From 084865d1a92f1b699a20d3535d7b5c6e3d4a6833 Mon Sep 17 00:00:00 2001 From: Ben Dunkin Date: Sun, 1 Dec 2024 04:56:53 -0800 Subject: [PATCH 001/922] fix: dependency resolver on windows only works when --enable_runfiles and --windows_enable_symlinks is used (#2457) `compile_pip_requirements` doesn't work as expected on Windows unless both `--enable_runfiles` and `--windows_enable_symlinks` are used. Both options default to off on Windows because the filesystem on Windows makes setting up the runfiles directories with actual files very slow. This means that anyone on Windows with a default set up has to search around the Github issues to try and figure out why things don't work as advertised. The `dependency_resolver.py` doesn't inherently require these options, it just had two bugs that prevented it from working. 1. calling pip_compile exits the whole program so it never gets to run the code that should copy the output to the source tree. Things just happen to work on linux because the runfiles are symlinks, and it does not need to copy anything. 2. it assumed the `runfiles` resolved file would be in the runfiles tree, but on Windows, when `--enable_runfiles` is not set, it actually gets resolved to a file in the source tree. Before: ```sh bazel run //third_party/python:requirements.update Starting local Bazel server and connecting to it... INFO: Invocation ID: 8aa3e832-78ce-4999-912b-c43e7ca3212b INFO: Analyzed target //third_party/python:requirements.update (129 packages loaded, 9563 targets configured). INFO: Found 1 target... Target //third_party/python:requirements.update up-to-date: bazel-bin/third_party/python/requirements.update.zip bazel-bin/third_party/python/requirements.update.exe INFO: Elapsed time: 60.964s, Critical Path: 0.77s INFO: 8 processes: 2 remote cache hit, 6 internal. INFO: Build completed successfully, 8 total actions INFO: Running command line: bazel-bin/third_party/python/requirements.update.exe '--src=_main/third_party/python/requirements.txt' _main/third_party/python/requirements_lock.txt //third_party/python:requirements.update '--resolver=backtracking' --allow-unsafe --generate-hashes '--requirements-windows=_main/third_party/python/requirements_windows.txt' --strip-extras Updating third_party/python/requirements_windows.txt Error: Could not open file 'third_party/python/requirements_windows.txt': No such file or directory ``` After: ```sh bazel run //third_party/python:requirements.update INFO: Invocation ID: 39f999a0-6c1d-4b2c-a1be-3d71e838916a INFO: Analyzed target //third_party/python:requirements.update (5 packages loaded, 45 targets configured). INFO: Found 1 target... Target //third_party/python:requirements.update up-to-date: bazel-bin/third_party/python/requirements.update.zip bazel-bin/third_party/python/requirements.update.exe INFO: Elapsed time: 5.410s, Critical Path: 4.79s INFO: 2 processes: 1 internal, 1 local. INFO: Build completed successfully, 2 total actions INFO: Running command line: bazel-bin/third_party/python/requirements.update.exe '--src=_main/third_party/python/requirements.txt' _main/third_party/python/requirements_lock.txt //third_party/python:requirements.update '--resolver=backtracking' --allow-unsafe --generate-hashes '--requirements-windows=_main/third_party/python/requirements_windows.txt' --strip-extras Updating third_party/python/requirements_windows.txt # # This file is autogenerated by pip-compile with Python 3.13 # by the following command: # # bazel run //third_party/python:requirements.update # mpmath==1.3.0 \ --hash=sha256:7a28eb2a9774d00c7bc92411c19a89209d5da7c4c9a9e227be8330a23a25b91f \ --hash=sha256:a0b2b9fe80bbcd81a6647ff13108738cfb482d481d826cc0e02f5b35e5c88d2c # via sympy sympy==1.13.3 \ --hash=sha256:54612cf55a62755ee71824ce692986f23c88ffa77207b30c1368eda4a7060f73 \ --hash=sha256:b27fd2c6530e0ab39e275fc9b683895367e51d5da91baa8d3d64db2565fec4d9 # via -r G:/projects/bedrock-engine/third_party/python/requirements.txt ``` And `//third_part/python:requirements_windows.txt` is updated. Fixes #1943 Fixes #1431 --------- Co-authored-by: Ignas Anikevicius <240938+aignas@users.noreply.github.com> --- CHANGELOG.md | 2 ++ .../pypi/dependency_resolver/dependency_resolver.py | 13 ++++++++++--- 2 files changed, 12 insertions(+), 3 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 37a9e710a1..1fa047b47d 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -95,6 +95,8 @@ Other changes: ([2169](https://github.com/bazelbuild/rules_python/issues/2169)). * (workspace) Corrected protobuf's name to com_google_protobuf, the name is hardcoded in Bazel, WORKSPACE mode. +* (pypi): {bzl:obj}`compile_pip_requirements` no longer fails on Windows when `--enable_runfiles` is not enabled. +* (pypi): {bzl:obj}`compile_pip_requirements` now correctly updates files in the source tree on Windows when `--windows_enable_symlinks` is not enabled. * (repositories): Add libs/python3.lib and pythonXY.dll to the `libpython` target defined by a repository template. This enables stable ABI builds of Python extensions on Windows (by defining Py_LIMITED_API). diff --git a/python/private/pypi/dependency_resolver/dependency_resolver.py b/python/private/pypi/dependency_resolver/dependency_resolver.py index 0ff9b2fb7c..293377dc6d 100644 --- a/python/private/pypi/dependency_resolver/dependency_resolver.py +++ b/python/private/pypi/dependency_resolver/dependency_resolver.py @@ -170,19 +170,26 @@ def main( if UPDATE: print("Updating " + requirements_file_relative) + + # Make sure the output file for pip_compile exists. It won't if we are on Windows and --enable_runfiles is not set. + if not os.path.exists(requirements_file_relative): + os.makedirs(os.path.dirname(requirements_file_relative), exist_ok=True) + shutil.copy(resolved_requirements_file, requirements_file_relative) + if "BUILD_WORKSPACE_DIRECTORY" in os.environ: workspace = os.environ["BUILD_WORKSPACE_DIRECTORY"] requirements_file_tree = os.path.join(workspace, requirements_file_relative) + absolute_output_file = Path(requirements_file_relative).absolute() # In most cases, requirements_file will be a symlink to the real file in the source tree. # If symlinks are not enabled (e.g. on Windows), then requirements_file will be a copy, # and we should copy the updated requirements back to the source tree. - if not os.path.samefile(resolved_requirements_file, requirements_file_tree): + if not absolute_output_file.samefile(requirements_file_tree): atexit.register( lambda: shutil.copy( - resolved_requirements_file, requirements_file_tree + absolute_output_file, requirements_file_tree ) ) - cli(argv) + cli(argv, standalone_mode = False) requirements_file_relative_path = Path(requirements_file_relative) content = requirements_file_relative_path.read_text() content = content.replace(absolute_path_prefix, "") From d0c1555b8b3db82d61e16cddbac11ff4117a86f0 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Mon, 2 Dec 2024 15:59:27 -0800 Subject: [PATCH 002/922] chore: remove defunct code from py_cc_link_params_info.bzl (#2465) When splitting the single large providers.bzl file into separate files, I forgot to remove the PyRuntimeInfo code from the py_cc_link_params_info.bzl file. --- python/private/py_cc_link_params_info.bzl | 284 ---------------------- 1 file changed, 284 deletions(-) diff --git a/python/private/py_cc_link_params_info.bzl b/python/private/py_cc_link_params_info.bzl index bfa2de5978..35919a04e2 100644 --- a/python/private/py_cc_link_params_info.bzl +++ b/python/private/py_cc_link_params_info.bzl @@ -16,290 +16,6 @@ load("@rules_cc//cc/common:cc_info.bzl", "CcInfo") load(":util.bzl", "define_bazel_6_provider") -DEFAULT_STUB_SHEBANG = "#!/usr/bin/env python3" - -DEFAULT_BOOTSTRAP_TEMPLATE = Label("//python/private:bootstrap_template") - -_PYTHON_VERSION_VALUES = ["PY2", "PY3"] - -def _optional_int(value): - return int(value) if value != None else None - -def interpreter_version_info_struct_from_dict(info_dict): - """Create a struct of interpreter version info from a dict from an attribute. - - Args: - info_dict: (dict | None) of version info fields. See interpreter_version_info - provider field docs. - - Returns: - struct of version info; see interpreter_version_info provider field docs. - """ - info_dict = dict(info_dict or {}) # Copy in case the original is frozen - if info_dict: - if not ("major" in info_dict and "minor" in info_dict): - fail("interpreter_version_info must have at least two keys, 'major' and 'minor'") - version_info_struct = struct( - major = _optional_int(info_dict.pop("major", None)), - minor = _optional_int(info_dict.pop("minor", None)), - micro = _optional_int(info_dict.pop("micro", None)), - releaselevel = str(info_dict.pop("releaselevel")) if "releaselevel" in info_dict else None, - serial = _optional_int(info_dict.pop("serial", None)), - ) - - if len(info_dict.keys()) > 0: - fail("unexpected keys {} in interpreter_version_info".format( - str(info_dict.keys()), - )) - - return version_info_struct - -def _PyRuntimeInfo_init( - *, - implementation_name = None, - interpreter_path = None, - interpreter = None, - files = None, - coverage_tool = None, - coverage_files = None, - pyc_tag = None, - python_version, - stub_shebang = None, - bootstrap_template = None, - interpreter_version_info = None, - stage2_bootstrap_template = None, - zip_main_template = None): - if (interpreter_path and interpreter) or (not interpreter_path and not interpreter): - fail("exactly one of interpreter or interpreter_path must be specified") - - if interpreter_path and files != None: - fail("cannot specify 'files' if 'interpreter_path' is given") - - if (coverage_tool and not coverage_files) or (not coverage_tool and coverage_files): - fail( - "coverage_tool and coverage_files must both be set or neither must be set, " + - "got coverage_tool={}, coverage_files={}".format( - coverage_tool, - coverage_files, - ), - ) - - if python_version not in _PYTHON_VERSION_VALUES: - fail("invalid python_version: '{}'; must be one of {}".format( - python_version, - _PYTHON_VERSION_VALUES, - )) - - if files != None and type(files) != type(depset()): - fail("invalid files: got value of type {}, want depset".format(type(files))) - - if interpreter: - if files == None: - files = depset() - else: - files = None - - if coverage_files == None: - coverage_files = depset() - - if not stub_shebang: - stub_shebang = DEFAULT_STUB_SHEBANG - - return { - "bootstrap_template": bootstrap_template, - "coverage_files": coverage_files, - "coverage_tool": coverage_tool, - "files": files, - "implementation_name": implementation_name, - "interpreter": interpreter, - "interpreter_path": interpreter_path, - "interpreter_version_info": interpreter_version_info_struct_from_dict(interpreter_version_info), - "pyc_tag": pyc_tag, - "python_version": python_version, - "stage2_bootstrap_template": stage2_bootstrap_template, - "stub_shebang": stub_shebang, - "zip_main_template": zip_main_template, - } - -PyRuntimeInfo, _unused_raw_py_runtime_info_ctor = define_bazel_6_provider( - doc = """Contains information about a Python runtime, as returned by the `py_runtime` -rule. - -A Python runtime describes either a *platform runtime* or an *in-build runtime*. -A platform runtime accesses a system-installed interpreter at a known path, -whereas an in-build runtime points to a `File` that acts as the interpreter. In -both cases, an "interpreter" is really any executable binary or wrapper script -that is capable of running a Python script passed on the command line, following -the same conventions as the standard CPython interpreter. -""", - init = _PyRuntimeInfo_init, - fields = { - "bootstrap_template": """ -:type: File - -A template of code responsible for the initial startup of a program. - -This code is responsible for: - -* Locating the target interpreter. Typically it is in runfiles, but not always. -* Setting necessary environment variables, command line flags, or other - configuration that can't be modified after the interpreter starts. -* Invoking the appropriate entry point. This is usually a second-stage bootstrap - that performs additional setup prior to running a program's actual entry point. - -The {obj}`--bootstrap_impl` flag affects how this stage 1 bootstrap -is expected to behave and the substutitions performed. - -* `--bootstrap_impl=system_python` substitutions: `%is_zipfile%`, `%python_binary%`, - `%target%`, `%workspace_name`, `%coverage_tool%`, `%import_all%`, `%imports%`, - `%main%`, `%shebang%` -* `--bootstrap_impl=script` substititions: `%is_zipfile%`, `%python_binary%`, - `%target%`, `%workspace_name`, `%shebang%, `%stage2_bootstrap%` - -Substitution definitions: - -* `%shebang%`: The shebang to use with the bootstrap; the bootstrap template - may choose to ignore this. -* `%stage2_bootstrap%`: A runfiles-relative path to the stage 2 bootstrap. -* `%python_binary%`: The path to the target Python interpreter. There are three - types of paths: - * An absolute path to a system interpreter (e.g. begins with `/`). - * A runfiles-relative path to an interpreter (e.g. `somerepo/bin/python3`) - * A program to search for on PATH, i.e. a word without spaces, e.g. `python3`. -* `%workspace_name%`: The name of the workspace the target belongs to. -* `%is_zipfile%`: The string `1` if this template is prepended to a zipfile to - create a self-executable zip file. The string `0` otherwise. - -For the other substitution definitions, see the {obj}`stage2_bootstrap_template` -docs. - -:::{versionchanged} 0.33.0 -The set of substitutions depends on {obj}`--bootstrap_impl` -::: -""", - "coverage_files": """ -:type: depset[File] | None - -The files required at runtime for using `coverage_tool`. Will be `None` if no -`coverage_tool` was provided. -""", - "coverage_tool": """ -:type: File | None - -If set, this field is a `File` representing tool used for collecting code -coverage information from python tests. Otherwise, this is `None`. -""", - "files": """ -:type: depset[File] | None - -If this is an in-build runtime, this field is a `depset` of `File`s that need to -be added to the runfiles of an executable target that uses this runtime (in -particular, files needed by `interpreter`). The value of `interpreter` need not -be included in this field. If this is a platform runtime then this field is -`None`. -""", - "implementation_name": """ -:type: str | None - -The Python implementation name (`sys.implementation.name`) -""", - "interpreter": """ -:type: File | None - -If this is an in-build runtime, this field is a `File` representing the -interpreter. Otherwise, this is `None`. Note that an in-build runtime can use -either a prebuilt, checked-in interpreter or an interpreter built from source. -""", - "interpreter_path": """ -:type: str | None - -If this is a platform runtime, this field is the absolute filesystem path to the -interpreter on the target platform. Otherwise, this is `None`. -""", - "interpreter_version_info": """ -:type: struct - -Version information about the interpreter this runtime provides. -It should match the format given by `sys.version_info`, however -for simplicity, the micro, releaselevel, and serial values are -optional. -A struct with the following fields: -* `major`: {type}`int`, the major version number -* `minor`: {type}`int`, the minor version number -* `micro`: {type}`int | None`, the micro version number -* `releaselevel`: {type}`str | None`, the release level -* `serial`: {type}`int | None`, the serial number of the release -""", - "pyc_tag": """ -:type: str | None - -The tag portion of a pyc filename, e.g. the `cpython-39` infix -of `foo.cpython-39.pyc`. See PEP 3147. If not specified, it will be computed -from {obj}`implementation_name` and {obj}`interpreter_version_info`. If no -pyc_tag is available, then only source-less pyc generation will function -correctly. -""", - "python_version": """ -:type: str - -Indicates whether this runtime uses Python major version 2 or 3. Valid values -are (only) `"PY2"` and `"PY3"`. -""", - "stage2_bootstrap_template": """ -:type: File - -A template of Python code that runs under the desired interpreter and is -responsible for orchestrating calling the program's actual main code. This -bootstrap is responsible for affecting the current runtime's state, such as -import paths or enabling coverage, so that, when it runs the program's actual -main code, it works properly under Bazel. - -The following substitutions are made during template expansion: -* `%main%`: A runfiles-relative path to the program's actual main file. This - can be a `.py` or `.pyc` file, depending on precompile settings. -* `%coverage_tool%`: Runfiles-relative path to the coverage library's entry point. - If coverage is not enabled or available, an empty string. -* `%import_all%`: The string `True` if all repositories in the runfiles should - be added to sys.path. The string `False` otherwise. -* `%imports%`: A colon-delimited string of runfiles-relative paths to add to - sys.path. -* `%target%`: The name of the target this is for. -* `%workspace_name%`: The name of the workspace the target belongs to. - -:::{versionadded} 0.33.0 -::: -""", - "stub_shebang": """ -:type: str - -"Shebang" expression prepended to the bootstrapping Python stub -script used when executing {obj}`py_binary` targets. Does not -apply to Windows. -""", - "zip_main_template": """ -:type: File - -A template of Python code that becomes a zip file's top-level `__main__.py` -file. The top-level `__main__.py` file is used when the zip file is explicitly -passed to a Python interpreter. See PEP 441 for more information about zipapp -support. Note that py_binary-generated zip files are self-executing and -skip calling `__main__.py`. - -The following substitutions are made during template expansion: -* `%stage2_bootstrap%`: A runfiles-relative string to the stage 2 bootstrap file. -* `%python_binary%`: The path to the target Python interpreter. There are three - types of paths: - * An absolute path to a system interpreter (e.g. begins with `/`). - * A runfiles-relative path to an interpreter (e.g. `somerepo/bin/python3`) - * A program to search for on PATH, i.e. a word without spaces, e.g. `python3`. -* `%workspace_name%`: The name of the workspace for the built target. - -:::{versionadded} 0.33.0 -::: -""", - }, -) - def _PyCcLinkParamsInfo_init(cc_info): return { "cc_info": CcInfo(linking_context = cc_info.linking_context), From a36233950d887a4932537dc7831544d2608d1d9b Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Mon, 2 Dec 2024 16:00:04 -0800 Subject: [PATCH 003/922] fix: add missing api distribution target (#2464) The distribution file groups are mostly used by integration tests, not releases, so these filegroups missing doesn't usually cause a problem. This was found when importing rules_python into Google, where filegroups of the rules sources are fed into various tests. --- python/private/BUILD.bazel | 1 + python/private/api/BUILD.bazel | 5 +++++ 2 files changed, 6 insertions(+) diff --git a/python/private/BUILD.bazel b/python/private/BUILD.bazel index 9772089e97..76e3a78778 100644 --- a/python/private/BUILD.bazel +++ b/python/private/BUILD.bazel @@ -30,6 +30,7 @@ licenses(["notice"]) filegroup( name = "distribution", srcs = glob(["**"]) + [ + "//python/private/api:distribution", "//python/private/proto:distribution", "//python/private/pypi:distribution", "//python/private/whl_filegroup:distribution", diff --git a/python/private/api/BUILD.bazel b/python/private/api/BUILD.bazel index 9e97dc2b59..0826b85d9b 100644 --- a/python/private/api/BUILD.bazel +++ b/python/private/api/BUILD.bazel @@ -19,6 +19,11 @@ package( default_visibility = ["//:__subpackages__"], ) +filegroup( + name = "distribution", + srcs = glob(["**"]), +) + py_common_api( name = "py_common_api", # NOTE: Not actually public. Implicit dependency of public rules. From bc8658ae6ed6750d07ed4b7d6ba58f0951fc68e5 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Mon, 2 Dec 2024 16:00:26 -0800 Subject: [PATCH 004/922] docs: mention calling register_toolchains in custom toolchain docs (#2463) Mention where register_toolchains() should be when defining custom toolchains. Also link to the Bazel docs about toolchains. This stems from Slack discussion where someone was trying to define custom toolchains and got hung up on the last step or registering them. --- docs/toolchains.md | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/docs/toolchains.md b/docs/toolchains.md index d6c59544a8..db4c6ba07a 100644 --- a/docs/toolchains.md +++ b/docs/toolchains.md @@ -444,8 +444,17 @@ toolchain( ], exec_comaptible_with = ["@platforms/os:linux"] ) + +# File: MODULE.bazel or WORKSPACE.bazel +# These toolchains will considered before others +register_toolchains("//toolchains:all") ``` +When registering custom toolchains, be aware of the the [toolchain registration +order](https://bazel.build/extending/toolchains#toolchain-resolution). In brief, +toolchain order is the BFS-order of the modules; see the bazel docs for a more +detailed description. + :::{note} The toolchain() calls should be in a separate BUILD file from everything else. This avoids Bazel having to perform unnecessary work when it discovers the list From 096a04fdcd2c3ff29f485d57129a1d838f022867 Mon Sep 17 00:00:00 2001 From: Nicholas Junge Date: Tue, 3 Dec 2024 01:01:26 +0100 Subject: [PATCH 005/922] test: Enable non-ABI3 libs linking test for Windows (#2461) Also explicitly include ABI3 in the name of the other Windows-only testcase, since that is the whole point of the test, and no ABI3-only libs exist on non-Windows platforms. ----------- Follow-up of #1820. --- tests/cc/current_py_cc_libs/BUILD.bazel | 11 +---------- 1 file changed, 1 insertion(+), 10 deletions(-) diff --git a/tests/cc/current_py_cc_libs/BUILD.bazel b/tests/cc/current_py_cc_libs/BUILD.bazel index 9f335990e6..9269553a3f 100644 --- a/tests/cc/current_py_cc_libs/BUILD.bazel +++ b/tests/cc/current_py_cc_libs/BUILD.bazel @@ -20,14 +20,6 @@ current_py_cc_libs_test_suite(name = "current_py_cc_libs_tests") cc_test( name = "python_libs_linking_test", srcs = ["python_libs_linking_test.cc"], - # Windows fails with linking errors, but its not clear why; someone - # with more C + Windows experience will have to figure it out. - # - rickeylev@ - target_compatible_with = select({ - "@platforms//os:linux": [], - "@platforms//os:osx": [], - "//conditions:default": ["@platforms//:incompatible"], - }), deps = [ "@rules_python//python/cc:current_py_cc_headers", "@rules_python//python/cc:current_py_cc_libs", @@ -41,10 +33,9 @@ cc_test( # for libs/python3.lib. # buildifier: disable=native-cc cc_test( - name = "python_libs_linking_windows_test", + name = "python_abi3_libs_linking_windows_test", srcs = ["python_libs_linking_test.cc"], defines = ["Py_LIMITED_API=0x030A0000"], - env = {"HELLO": "world"}, target_compatible_with = ["@platforms//os:windows"], deps = [ "@rules_python//python/cc:current_py_cc_headers", From 5eb139f36793494313aa84429c50588c802f82e3 Mon Sep 17 00:00:00 2001 From: Chowder <16789070+chowder@users.noreply.github.com> Date: Wed, 4 Dec 2024 04:51:35 +0000 Subject: [PATCH 006/922] fix: only delete first sys.path entry in the stage-2 bootstrap if PYTHONSAFEPATH is unset or unsupported (#2418) Unnconditionally deleting the first `sys.path` entry on the stage-2 bootstrap incorrecly removes a valid search path on Python 3.11 and above, since `PYTHONSAFEPATH` is already unconditionally set in stage-1. It should be deleted only if it is unset or unsupported. Fixes https://github.com/bazelbuild/rules_python/issues/2318 --------- Co-authored-by: Richard Levasseur Co-authored-by: Richard Levasseur --- CHANGELOG.md | 2 + python/private/stage2_bootstrap_template.py | 16 +++++--- tests/bootstrap_impls/BUILD.bazel | 18 +++------ tests/no_unsafe_paths/BUILD.bazel | 33 ++++++++++++++++ tests/no_unsafe_paths/test.py | 44 +++++++++++++++++++++ tests/support/support.bzl | 7 ++++ 6 files changed, 102 insertions(+), 18 deletions(-) create mode 100644 tests/no_unsafe_paths/BUILD.bazel create mode 100644 tests/no_unsafe_paths/test.py diff --git a/CHANGELOG.md b/CHANGELOG.md index 1fa047b47d..590a9c795b 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -100,6 +100,8 @@ Other changes: * (repositories): Add libs/python3.lib and pythonXY.dll to the `libpython` target defined by a repository template. This enables stable ABI builds of Python extensions on Windows (by defining Py_LIMITED_API). +* (rules) `py_test` and `py_binary` targets no longer incorrectly remove the + first `sys.path` entry when using {obj}`--bootstrap_impl=script` {#v0-0-0-added} ### Added diff --git a/python/private/stage2_bootstrap_template.py b/python/private/stage2_bootstrap_template.py index d2c7497795..1e19a71b64 100644 --- a/python/private/stage2_bootstrap_template.py +++ b/python/private/stage2_bootstrap_template.py @@ -4,13 +4,17 @@ import sys -# The Python interpreter unconditionally prepends the directory containing this +# By default the Python interpreter prepends the directory containing this # script (following symlinks) to the import path. This is the cause of #9239, -# and is a special case of #7091. We therefore explicitly delete that entry. -# TODO(#7091): Remove this hack when no longer necessary. -# TODO: Use sys.flags.safe_path to determine whether this removal should be -# performed -del sys.path[0] +# and is a special case of #7091. +# +# Python 3.11 introduced an PYTHONSAFEPATH (-P) option that disables this +# behaviour, which we set in the stage 1 bootstrap. +# So the prepended entry needs to be removed only if the above option is either +# unset or not supported by the interpreter. +# NOTE: This can be removed when Python 3.10 and below is no longer supported +if not getattr(sys.flags, "safe_path", False): + del sys.path[0] import contextlib import os diff --git a/tests/bootstrap_impls/BUILD.bazel b/tests/bootstrap_impls/BUILD.bazel index 2fb1f38ff0..8e50f34cfa 100644 --- a/tests/bootstrap_impls/BUILD.bazel +++ b/tests/bootstrap_impls/BUILD.bazel @@ -11,16 +11,10 @@ # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # See the License for the specific language governing permissions and # limitations under the License. - -load("//python/private:util.bzl", "IS_BAZEL_7_OR_HIGHER") # buildifier: disable=bzl-visibility load("//tests/support:sh_py_run_test.bzl", "py_reconfig_test", "sh_py_run_test") +load("//tests/support:support.bzl", "SUPPORTS_BOOTSTRAP_SCRIPT") load(":venv_relative_path_tests.bzl", "relative_path_test_suite") -_SUPPORTS_BOOTSTRAP_SCRIPT = select({ - "@platforms//os:windows": ["@platforms//:incompatible"], - "//conditions:default": [], -}) if IS_BAZEL_7_OR_HIGHER else ["@platforms//:incompatible"] - sh_py_run_test( name = "run_binary_zip_no_test", build_python_zip = "no", @@ -41,7 +35,7 @@ sh_py_run_test( build_python_zip = "yes", py_src = "bin.py", sh_src = "run_binary_zip_yes_test.sh", - target_compatible_with = _SUPPORTS_BOOTSTRAP_SCRIPT, + target_compatible_with = SUPPORTS_BOOTSTRAP_SCRIPT, ) sh_py_run_test( @@ -50,7 +44,7 @@ sh_py_run_test( build_python_zip = "no", py_src = "bin.py", sh_src = "run_binary_zip_no_test.sh", - target_compatible_with = _SUPPORTS_BOOTSTRAP_SCRIPT, + target_compatible_with = SUPPORTS_BOOTSTRAP_SCRIPT, ) py_reconfig_test( @@ -60,7 +54,7 @@ py_reconfig_test( env = {"BOOTSTRAP": "script"}, imports = ["./USER_IMPORT/site-packages"], main = "sys_path_order_test.py", - target_compatible_with = _SUPPORTS_BOOTSTRAP_SCRIPT, + target_compatible_with = SUPPORTS_BOOTSTRAP_SCRIPT, ) py_reconfig_test( @@ -77,7 +71,7 @@ sh_py_run_test( bootstrap_impl = "script", py_src = "bin.py", sh_src = "inherit_pythonsafepath_env_test.sh", - target_compatible_with = _SUPPORTS_BOOTSTRAP_SCRIPT, + target_compatible_with = SUPPORTS_BOOTSTRAP_SCRIPT, ) sh_py_run_test( @@ -86,7 +80,7 @@ sh_py_run_test( imports = ["./MARKER"], py_src = "call_sys_exe.py", sh_src = "sys_executable_inherits_sys_path_test.sh", - target_compatible_with = _SUPPORTS_BOOTSTRAP_SCRIPT, + target_compatible_with = SUPPORTS_BOOTSTRAP_SCRIPT, ) relative_path_test_suite(name = "relative_path_tests") diff --git a/tests/no_unsafe_paths/BUILD.bazel b/tests/no_unsafe_paths/BUILD.bazel new file mode 100644 index 0000000000..f12d1c9a70 --- /dev/null +++ b/tests/no_unsafe_paths/BUILD.bazel @@ -0,0 +1,33 @@ +# Copyright 2024 The Bazel Authors. All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +load("//tests/support:sh_py_run_test.bzl", "py_reconfig_test") +load("//tests/support:support.bzl", "SUPPORTS_BOOTSTRAP_SCRIPT") + +py_reconfig_test( + name = "no_unsafe_paths_3.10_test", + srcs = ["test.py"], + bootstrap_impl = "script", + main = "test.py", + python_version = "3.10", + target_compatible_with = SUPPORTS_BOOTSTRAP_SCRIPT, +) + +py_reconfig_test( + name = "no_unsafe_paths_3.11_test", + srcs = ["test.py"], + bootstrap_impl = "script", + main = "test.py", + python_version = "3.11", + target_compatible_with = SUPPORTS_BOOTSTRAP_SCRIPT, +) diff --git a/tests/no_unsafe_paths/test.py b/tests/no_unsafe_paths/test.py new file mode 100644 index 0000000000..1f6cd4e569 --- /dev/null +++ b/tests/no_unsafe_paths/test.py @@ -0,0 +1,44 @@ +# Copyright 2024 The Bazel Authors. All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +import os +import sys +import unittest + + +class NoUnsafePathsTest(unittest.TestCase): + def test_no_unsafe_paths_in_search_path(self): + # Based on sys.path documentation, the first item added is the zip + # archive + # (see: https://docs.python.org/3/library/sys_path_init.html) + # + # We can use this as a marker to verify that during bootstrapping, + # (1) no unexpected paths were prepended, and (2) no paths were + # accidentally dropped. + # + major, minor, *_ = sys.version_info + archive = f"python{major}{minor}.zip" + + # < Python 3.11 behaviour + if (major, minor) < (3, 11): + # Because of https://github.com/bazelbuild/rules_python/blob/0.39.0/python/private/stage2_bootstrap_template.py#L415-L436 + self.assertEqual(os.path.dirname(sys.argv[0]), sys.path[0]) + self.assertEqual(os.path.basename(sys.path[1]), archive) + # >= Python 3.11 behaviour + else: + self.assertEqual(os.path.basename(sys.path[0]), archive) + + +if __name__ == '__main__': + unittest.main() \ No newline at end of file diff --git a/tests/support/support.bzl b/tests/support/support.bzl index 7358a6b1ee..2b6703843b 100644 --- a/tests/support/support.bzl +++ b/tests/support/support.bzl @@ -19,6 +19,8 @@ # rules_testing or as config_setting values, which don't support Label in some # places. +load("//python/private:util.bzl", "IS_BAZEL_7_OR_HIGHER") # buildifier: disable=bzl-visibility + MAC = Label("//tests/support:mac") MAC_X86_64 = Label("//tests/support:mac_x86_64") LINUX = Label("//tests/support:linux") @@ -39,3 +41,8 @@ PRECOMPILE_SOURCE_RETENTION = str(Label("//python/config_settings:precompile_sou PYC_COLLECTION = str(Label("//python/config_settings:pyc_collection")) PYTHON_VERSION = str(Label("//python/config_settings:python_version")) VISIBLE_FOR_TESTING = str(Label("//python/private:visible_for_testing")) + +SUPPORTS_BOOTSTRAP_SCRIPT = select({ + "@platforms//os:windows": ["@platforms//:incompatible"], + "//conditions:default": [], +}) if IS_BAZEL_7_OR_HIGHER else ["@platforms//:incompatible"] From 60da4a39e24381486d3d3bd089d516e3e0cad732 Mon Sep 17 00:00:00 2001 From: hunshcn Date: Wed, 4 Dec 2024 15:43:51 +0800 Subject: [PATCH 007/922] fix(gazelle): empty list (#2099) This is a code error (from typo). --- gazelle/python/generate.go | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/gazelle/python/generate.go b/gazelle/python/generate.go index c563b47bf3..b1ac6689e4 100644 --- a/gazelle/python/generate.go +++ b/gazelle/python/generate.go @@ -309,7 +309,7 @@ func (py *Python) GenerateRules(args language.GenerateArgs) language.GenerateRes build() if pyLibrary.IsEmpty(py.Kinds()[pyLibrary.Kind()]) { - result.Empty = append(result.Gen, pyLibrary) + result.Empty = append(result.Empty, pyLibrary) } else { result.Gen = append(result.Gen, pyLibrary) result.Imports = append(result.Imports, pyLibrary.PrivateAttr(config.GazelleImportsKey)) From d24691f0a7891136cf338f12480c33ad33ca39e4 Mon Sep 17 00:00:00 2001 From: Ignas Anikevicius <240938+aignas@users.noreply.github.com> Date: Fri, 6 Dec 2024 09:29:02 +0900 Subject: [PATCH 008/922] chore: bump the changelog to 1.0 (#2470) Fixes #1361 Closes #2459 as won't do --------- Co-authored-by: Richard Levasseur --- CHANGELOG.md | 27 ++++++++++++++++++++++++--- 1 file changed, 24 insertions(+), 3 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 590a9c795b..7bdc5e4483 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -52,6 +52,27 @@ Unreleased changes template. {#v0-0-0-changed} ### Changed +* Nothing changed. + +{#v0-0-0-fixed} +### Fixed +* Nothing fixed. + +{#v0-0-0-added} +### Added +* Nothing added. + +{#v0-0-0-removed} +### Removed +* Nothing removed. + +{#v1-0-0} +## [1.0.0] - 2024-12-05 + +[1.0.0]: https://github.com/bazelbuild/rules_python/releases/tag/1.0.0 + +{#v1-0-0-changed} +### Changed **Breaking**: * (toolchains) stop exposing config settings in python toolchain alias repos. @@ -82,7 +103,7 @@ Other changes: * (deps) bazel_features 1.21.0; necessary for compatiblity with Bazel 8 rc3 * (deps) stardoc 0.7.2 to support Bazel 8. -{#v0-0-0-fixed} +{#v1-0-0-fixed} ### Fixed * (toolchains) stop depending on `uname` to get the value of the host platform. * (pypi): Correctly handle multiple versions of the same package in the requirements @@ -103,7 +124,7 @@ Other changes: * (rules) `py_test` and `py_binary` targets no longer incorrectly remove the first `sys.path` entry when using {obj}`--bootstrap_impl=script` -{#v0-0-0-added} +{#v1-0-0-added} ### Added * (gazelle): Parser failures will now be logged to the terminal. Additional details can be logged by setting `RULES_PYTHON_GAZELLE_VERBOSE=1`. @@ -121,7 +142,7 @@ Other changes: initialize the interpreter via venv startup hooks. * (runfiles) (Bazel 7.4+) Added support for spaces and newlines in runfiles paths -{#v0-0-0-removed} +{#v1-0-0-removed} ### Removed * (pypi): Remove `pypi_install_dependencies` macro that has been included in {bzl:obj}`py_repositories` for a long time. From b5ed3e4554b6e31fd2d5fe6b0b270d2f8adfa059 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Thu, 5 Dec 2024 16:57:27 -0800 Subject: [PATCH 009/922] tests: skip workspace-specific examples for bazel 9 (#2471) Bazel 9 won't support workspace builds, so skip them in the Bazel@head pipeline. The bzlmod equivalents of these examples is covered elsewhere. Along the way, enable workspace by default in the examples for a bit of futureproofing. Work towards https://github.com/bazelbuild/rules_python/issues/2469 --- .bazelci/presubmit.yml | 1 + examples/pip_repository_annotations/.bazelrc | 2 +- examples/py_proto_library/.bazelrc | 2 +- 3 files changed, 3 insertions(+), 2 deletions(-) diff --git a/.bazelci/presubmit.yml b/.bazelci/presubmit.yml index c45fc78990..8c0252c3c8 100644 --- a/.bazelci/presubmit.yml +++ b/.bazelci/presubmit.yml @@ -48,6 +48,7 @@ buildifier: - "--noenable_bzlmod" - "--test_tag_filters=-integration-test" .common_workspace_flags: &common_workspace_flags + skip_in_bazel_downstream_pipeline: "Bazel 9 doesn't support workspace" test_flags: - "--noenable_bzlmod" - "--enable_workspace" diff --git a/examples/pip_repository_annotations/.bazelrc b/examples/pip_repository_annotations/.bazelrc index 4f62c6e76f..d893227946 100644 --- a/examples/pip_repository_annotations/.bazelrc +++ b/examples/pip_repository_annotations/.bazelrc @@ -3,5 +3,5 @@ try-import %workspace%/user.bazelrc # This example is WORKSPACE specific. The equivalent functionality # is in examples/bzlmod as the `whl_mods` feature. -build --experimental_enable_bzlmod=false +build --experimental_enable_bzlmod=false --enable_workspace=true common:bazel7.x --incompatible_python_disallow_native_rules diff --git a/examples/py_proto_library/.bazelrc b/examples/py_proto_library/.bazelrc index 65d8a0a2f6..d73fc5387a 100644 --- a/examples/py_proto_library/.bazelrc +++ b/examples/py_proto_library/.bazelrc @@ -1,3 +1,3 @@ # The equivalent bzlmod behavior is covered by examples/bzlmod/py_proto_library -common --noenable_bzlmod +common --noenable_bzlmod --enable_workspace=true common:bazel7.x --incompatible_python_disallow_native_rules From ef48735fe2e786f340a1d920494be22d72aafec5 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Fri, 6 Dec 2024 15:57:51 -0800 Subject: [PATCH 010/922] chore: ignore examples/pip_repository_annotations bazel-bin symlink (#2472) If the examples/pip_repository_annotations had build performed in it, then outer invocations try to traverse the symlink, wasting memory and causing errors. To fix, add the path to `.bazelignore` so bazel doesn't try to visit it. --- .bazelignore | 1 + 1 file changed, 1 insertion(+) diff --git a/.bazelignore b/.bazelignore index d5fe879e83..60d680e9f0 100644 --- a/.bazelignore +++ b/.bazelignore @@ -22,6 +22,7 @@ examples/bzlmod_build_file_generation/bazel-bzlmod_build_file_generation examples/multi_python_versions/bazel-multi_python_versions examples/pip_parse/bazel-pip_parse examples/pip_parse_vendored/bazel-pip_parse_vendored +examples/pip_repository_annotations/bazel-pip_repository_annotations examples/py_proto_library/bazel-py_proto_library tests/integration/compile_pip_requirements/bazel-compile_pip_requirements tests/integration/ignore_root_user_error/bazel-ignore_root_user_error From 9b2b70adba5431162401a97b2bbab1dc938e7245 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Fri, 6 Dec 2024 15:58:32 -0800 Subject: [PATCH 011/922] chore: use per-rule loads in pip_compile.bzl (#2473) This is for overall code hygiene, but also because it seems to make some progress on Bazel 9 being able to load files in WORKSPACE mode (something about defs.bzl triggers loading more symbols which can't be found) Work towards https://github.com/bazelbuild/rules_python/issues/2469 --- python/private/pypi/BUILD.bazel | 3 ++- python/private/pypi/pip_compile.bzl | 3 ++- 2 files changed, 4 insertions(+), 2 deletions(-) diff --git a/python/private/pypi/BUILD.bazel b/python/private/pypi/BUILD.bazel index 2cc073a832..7e2d398fde 100644 --- a/python/private/pypi/BUILD.bazel +++ b/python/private/pypi/BUILD.bazel @@ -218,7 +218,8 @@ bzl_library( srcs = ["pip_compile.bzl"], deps = [ ":deps_bzl", - "//python:defs_bzl", + "//python:py_binary_bzl", + "//python:py_test_bzl", ], ) diff --git a/python/private/pypi/pip_compile.bzl b/python/private/pypi/pip_compile.bzl index dc5b186a6a..8e46947b99 100644 --- a/python/private/pypi/pip_compile.bzl +++ b/python/private/pypi/pip_compile.bzl @@ -19,7 +19,8 @@ NOTE @aignas 2024-06-23: We are using the implementation specific name here to make it possible to have multiple tools inside the `pypi` directory """ -load("//python:defs.bzl", _py_binary = "py_binary", _py_test = "py_test") +load("//python:py_binary.bzl", _py_binary = "py_binary") +load("//python:py_test.bzl", _py_test = "py_test") def pip_compile( name, From a7119c9e7c2c591db3ef02c1ba21f76aacae3c04 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Fri, 6 Dec 2024 15:59:36 -0800 Subject: [PATCH 012/922] tests: make multi_python_versions pass with Bazel 9 workspace (#2474) This is basically part of #2395, but for the workspace test. Same as that PR, the `$(rootpath)` expansion isn't valid for a target with multiple outputs. To fix, use `$(rootpaths)` and parse out the particular value of interest. Work towards https://github.com/bazelbuild/rules_python/issues/2469 --- examples/multi_python_versions/tests/BUILD.bazel | 12 ++++++------ examples/multi_python_versions/tests/version_test.sh | 6 +++++- 2 files changed, 11 insertions(+), 7 deletions(-) diff --git a/examples/multi_python_versions/tests/BUILD.bazel b/examples/multi_python_versions/tests/BUILD.bazel index d5c66e026f..177de22230 100644 --- a/examples/multi_python_versions/tests/BUILD.bazel +++ b/examples/multi_python_versions/tests/BUILD.bazel @@ -131,7 +131,7 @@ py_test( data = [":version_3_10"], env = { "SUBPROCESS_VERSION_CHECK": "3.10", - "SUBPROCESS_VERSION_PY_BINARY": "$(rootpath :version_3_10)", + "SUBPROCESS_VERSION_PY_BINARY": "$(rootpaths :version_3_10)", "VERSION_CHECK": "3.9", }, main = "cross_version_test.py", @@ -143,7 +143,7 @@ py_test_3_10( data = [":version_3_9"], env = { "SUBPROCESS_VERSION_CHECK": "3.9", - "SUBPROCESS_VERSION_PY_BINARY": "$(rootpath :version_3_9)", + "SUBPROCESS_VERSION_PY_BINARY": "$(rootpaths :version_3_9)", "VERSION_CHECK": "3.10", }, main = "cross_version_test.py", @@ -155,7 +155,7 @@ sh_test( data = [":version_default"], env = { "VERSION_CHECK": "3.9", # The default defined in the WORKSPACE. - "VERSION_PY_BINARY": "$(rootpath :version_default)", + "VERSION_PY_BINARY": "$(rootpaths :version_default)", }, ) @@ -165,7 +165,7 @@ sh_test( data = [":version_3_8"], env = { "VERSION_CHECK": "3.8", - "VERSION_PY_BINARY": "$(rootpath :version_3_8)", + "VERSION_PY_BINARY": "$(rootpaths :version_3_8)", }, ) @@ -175,7 +175,7 @@ sh_test( data = [":version_3_9"], env = { "VERSION_CHECK": "3.9", - "VERSION_PY_BINARY": "$(rootpath :version_3_9)", + "VERSION_PY_BINARY": "$(rootpaths :version_3_9)", }, ) @@ -185,7 +185,7 @@ sh_test( data = [":version_3_10"], env = { "VERSION_CHECK": "3.10", - "VERSION_PY_BINARY": "$(rootpath :version_3_10)", + "VERSION_PY_BINARY": "$(rootpaths :version_3_10)", }, ) diff --git a/examples/multi_python_versions/tests/version_test.sh b/examples/multi_python_versions/tests/version_test.sh index 3bedb95ef9..3f5fd960cb 100755 --- a/examples/multi_python_versions/tests/version_test.sh +++ b/examples/multi_python_versions/tests/version_test.sh @@ -16,7 +16,11 @@ set -o errexit -o nounset -o pipefail -version_py_binary=$("${VERSION_PY_BINARY}") +# VERSION_PY_BINARY is a space separate list of the executable and its main +# py file. We just want the executable. +bin=($VERSION_PY_BINARY) +bin="${bin[@]//*.py}" +version_py_binary=$($bin) if [[ "${version_py_binary}" != "${VERSION_CHECK}" ]]; then echo >&2 "expected version '${VERSION_CHECK}' is different than returned '${version_py_binary}'" From 0fb4ce12f0637077ea9a1064f6447d3aa81c6ffa Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Fri, 6 Dec 2024 16:47:10 -0800 Subject: [PATCH 013/922] fix: normalize argv0 so runfiles root can be found on windows with bazel 9 (#2481) When the shell test invokes the python binary, it uses a combination of forward slashes and backslashes. Under Bazel 9, that mixture of slashes is preserved. This later breaks a regex that looks for the OS-specific path separator. To fix, normalize forward slashes to the OS path separator. Oddly, it's not Bazel that is passing the mixture of slashes (it's the shell), but behavior seems to vary based on which version of Bazel is used. Along the way, copy the nicer `print_verbose` function from the stage2 bootstrap into the old bootstrap. It prints debug information in a nicer format. Work towards https://github.com/bazelbuild/rules_python/issues/2469 --- python/private/python_bootstrap_template.txt | 39 +++++++++++++++++--- 1 file changed, 34 insertions(+), 5 deletions(-) diff --git a/python/private/python_bootstrap_template.txt b/python/private/python_bootstrap_template.txt index 0f9c90b3b3..e3b39e30cd 100644 --- a/python/private/python_bootstrap_template.txt +++ b/python/private/python_bootstrap_template.txt @@ -89,9 +89,28 @@ def FindPythonBinary(module_space): """Finds the real Python binary if it's not a normal absolute path.""" return FindBinary(module_space, PYTHON_BINARY) -def PrintVerbose(*args): - if os.environ.get("RULES_PYTHON_BOOTSTRAP_VERBOSE"): - print("bootstrap:", *args, file=sys.stderr, flush=True) +def print_verbose(*args, mapping=None, values=None): + if os.environ.get("RULES_PYTHON_BOOTSTRAP_VERBOSE"): + if mapping is not None: + for key, value in sorted((mapping or {}).items()): + print( + "bootstrap:", + *args, + f"{key}={value!r}", + file=sys.stderr, + flush=True, + ) + elif values is not None: + for i, v in enumerate(values): + print( + "bootstrap:", + *args, + f"[{i}] {v!r}", + file=sys.stderr, + flush=True, + ) + else: + print("bootstrap:", *args, file=sys.stderr, flush=True) def PrintVerboseCoverage(*args): """Print output if VERBOSE_COVERAGE is non-empty in the environment.""" @@ -157,6 +176,12 @@ def FindModuleSpace(main_rel_path): return runfiles_dir stub_filename = sys.argv[0] + # On Windows, the path may contain both forward and backslashes. + # Normalize to the OS separator because the regex used later assumes + # the OS-specific separator. + if IsWindows: + stub_filename = stub_filename.replace("/", os.sep) + if not os.path.isabs(stub_filename): stub_filename = os.path.join(os.getcwd(), stub_filename) @@ -380,9 +405,9 @@ def _RunExecv(python_program, main_filename, args, env): # type: (str, str, list[str], dict[str, str]) -> ... """Executes the given Python file using the various environment settings.""" os.environ.update(env) - PrintVerbose("RunExecv: environ:", os.environ) + print_verbose("RunExecv: environ:", mapping=os.environ) argv = [python_program, main_filename] + args - PrintVerbose("RunExecv: argv:", python_program, argv) + print_verbose("RunExecv: argv:", python_program, argv) os.execv(python_program, argv) def _RunForCoverage(python_program, main_filename, args, env, @@ -453,6 +478,10 @@ relative_files = True return ret_code def Main(): + print_verbose("initial argv:", values=sys.argv) + print_verbose("initial cwd:", os.getcwd()) + print_verbose("initial environ:", mapping=os.environ) + print_verbose("initial sys.path:", values=sys.path) args = sys.argv[1:] new_env = {} From ca987735a04c2e20e9341b4ffd24082c99afd152 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Fri, 6 Dec 2024 18:12:00 -0800 Subject: [PATCH 014/922] chore: load specific bzl files instead of generic defs.bzl (#2483) Update code and examples to load the object-specific bzl files instead of the generic `defs.bzl`. This is mostly for code hygiene, but came out of trying to diagnose why Bazel 9 workspace builds kept erroing with defs.bzl somehow related. Removing the internal usages of defs.bzl doesn't seem to fully fix it, but does seem to eliminate some errors, make some progress, and narrow down what's going on. Work towards https://github.com/bazelbuild/rules_python/issues/2469 --- docs/getting-started.md | 2 +- docs/index.md | 2 +- examples/build_file_generation/BUILD.bazel | 4 +++- .../random_number_generator/BUILD.bazel | 3 ++- examples/bzlmod/BUILD.bazel | 4 +++- examples/bzlmod/entry_points/tests/BUILD.bazel | 2 +- examples/bzlmod/libs/my_lib/BUILD.bazel | 2 +- examples/bzlmod/other_module/other_module/pkg/BUILD.bazel | 2 +- examples/bzlmod/runfiles/BUILD.bazel | 2 +- examples/bzlmod/tests/BUILD.bazel | 3 ++- examples/bzlmod/whl_mods/BUILD.bazel | 2 +- examples/bzlmod_build_file_generation/BUILD.bazel | 4 +++- .../other_module/other_module/pkg/BUILD.bazel | 2 +- examples/bzlmod_build_file_generation/runfiles/BUILD.bazel | 2 +- examples/multi_python_versions/libs/my_lib/BUILD.bazel | 2 +- examples/multi_python_versions/tests/BUILD.bazel | 3 ++- examples/pip_parse/BUILD.bazel | 5 +++-- examples/pip_parse_vendored/BUILD.bazel | 2 +- examples/pip_repository_annotations/BUILD.bazel | 2 +- examples/py_proto_library/BUILD.bazel | 2 +- examples/wheel/BUILD.bazel | 3 ++- examples/wheel/lib/BUILD.bazel | 2 +- examples/wheel/private/BUILD.bazel | 2 +- python/private/proto/BUILD.bazel | 2 +- python/private/proto/py_proto_library.bzl | 2 +- python/private/pypi/deps.bzl | 2 +- python/private/pypi/generate_group_library_build_bazel.bzl | 2 +- python/private/pypi/whl_installer/BUILD.bazel | 3 ++- python/private/whl_filegroup/BUILD.bazel | 2 +- python/python.bzl | 6 +----- python/runfiles/BUILD.bazel | 2 +- tests/base_rules/base_tests.bzl | 2 +- tests/base_rules/py_binary/py_binary_tests.bzl | 2 +- tests/base_rules/py_library/py_library_tests.bzl | 3 ++- tests/base_rules/py_test/py_test_tests.bzl | 2 +- tests/load_from_macro/BUILD.bazel | 2 +- tests/pycross/BUILD.bazel | 2 +- .../generate_group_library_build_bazel_tests.bzl | 4 ++-- tests/pypi/whl_installer/BUILD.bazel | 2 +- tests/whl_filegroup/BUILD.bazel | 3 ++- third_party/rules_pycross/pycross/private/wheel_library.bzl | 2 +- tools/BUILD.bazel | 2 +- 42 files changed, 58 insertions(+), 48 deletions(-) diff --git a/docs/getting-started.md b/docs/getting-started.md index 9f52243fd1..b3b5409c7e 100644 --- a/docs/getting-started.md +++ b/docs/getting-started.md @@ -76,7 +76,7 @@ Once you've imported the rule set using either Bzlmod or WORKSPACE, you can then load the core rules in your `BUILD` files with the following: ```starlark -load("@rules_python//python:defs.bzl", "py_binary") +load("@rules_python//python:py_binary.bzl", "py_binary") py_binary( name = "main", diff --git a/docs/index.md b/docs/index.md index 378aac2faa..dd2e147c18 100644 --- a/docs/index.md +++ b/docs/index.md @@ -67,7 +67,7 @@ components have examples in the {gh-path}`examples` directory. The core rules are currently available in Bazel as built-in symbols, but this form is deprecated. Instead, you should depend on rules_python in your `WORKSPACE` or `MODULE.bazel` file and load the Python rules from -`@rules_python//python:defs.bzl` or load paths described in the API documentation. +`@rules_python//python:.bzl` or load paths described in the API documentation. A [buildifier](https://github.com/bazelbuild/buildtools/blob/master/buildifier/README.md) fix is available to automatically migrate `BUILD` and `.bzl` files to add the diff --git a/examples/build_file_generation/BUILD.bazel b/examples/build_file_generation/BUILD.bazel index 4d270dd850..a378775968 100644 --- a/examples/build_file_generation/BUILD.bazel +++ b/examples/build_file_generation/BUILD.bazel @@ -4,8 +4,10 @@ # ruleset. When the symbol is loaded you can use the rule. load("@bazel_gazelle//:def.bzl", "gazelle") load("@pip//:requirements.bzl", "all_whl_requirements") -load("@rules_python//python:defs.bzl", "py_binary", "py_library", "py_test") load("@rules_python//python:pip.bzl", "compile_pip_requirements") +load("@rules_python//python:py_binary.bzl", "py_binary") +load("@rules_python//python:py_library.bzl", "py_library") +load("@rules_python//python:py_test.bzl", "py_test") load("@rules_python_gazelle_plugin//manifest:defs.bzl", "gazelle_python_manifest") load("@rules_python_gazelle_plugin//modules_mapping:def.bzl", "modules_mapping") diff --git a/examples/build_file_generation/random_number_generator/BUILD.bazel b/examples/build_file_generation/random_number_generator/BUILD.bazel index 28370b418f..c77550084f 100644 --- a/examples/build_file_generation/random_number_generator/BUILD.bazel +++ b/examples/build_file_generation/random_number_generator/BUILD.bazel @@ -1,4 +1,5 @@ -load("@rules_python//python:defs.bzl", "py_library", "py_test") +load("@rules_python//python:py_library.bzl", "py_library") +load("@rules_python//python:py_test.bzl", "py_test") py_library( name = "random_number_generator", diff --git a/examples/bzlmod/BUILD.bazel b/examples/bzlmod/BUILD.bazel index 054b957b3b..df07385690 100644 --- a/examples/bzlmod/BUILD.bazel +++ b/examples/bzlmod/BUILD.bazel @@ -9,7 +9,9 @@ load("@bazel_skylib//rules:build_test.bzl", "build_test") load("@pip//:requirements.bzl", "all_data_requirements", "all_requirements", "all_whl_requirements", "requirement") load("@python_3_9//:defs.bzl", py_test_with_transition = "py_test") load("@python_versions//3.10:defs.bzl", compile_pip_requirements_3_10 = "compile_pip_requirements") -load("@rules_python//python:defs.bzl", "py_binary", "py_library", "py_test") +load("@rules_python//python:py_binary.bzl", "py_binary") +load("@rules_python//python:py_library.bzl", "py_library") +load("@rules_python//python:py_test.bzl", "py_test") # This stanza calls a rule that generates targets for managing pip dependencies # with pip-compile for a particular python version. diff --git a/examples/bzlmod/entry_points/tests/BUILD.bazel b/examples/bzlmod/entry_points/tests/BUILD.bazel index 5a65e9e1a3..3c6e02a3c4 100644 --- a/examples/bzlmod/entry_points/tests/BUILD.bazel +++ b/examples/bzlmod/entry_points/tests/BUILD.bazel @@ -1,5 +1,5 @@ load("@bazel_skylib//rules:run_binary.bzl", "run_binary") -load("@rules_python//python:defs.bzl", "py_test") +load("@rules_python//python:py_test.bzl", "py_test") # Below are targets for testing the `py_console_script_binary` feature and are # not part of the example how to use the feature. diff --git a/examples/bzlmod/libs/my_lib/BUILD.bazel b/examples/bzlmod/libs/my_lib/BUILD.bazel index 2679d0e4a0..77a059574d 100644 --- a/examples/bzlmod/libs/my_lib/BUILD.bazel +++ b/examples/bzlmod/libs/my_lib/BUILD.bazel @@ -1,5 +1,5 @@ load("@pip//:requirements.bzl", "requirement") -load("@rules_python//python:defs.bzl", "py_library") +load("@rules_python//python:py_library.bzl", "py_library") py_library( name = "my_lib", diff --git a/examples/bzlmod/other_module/other_module/pkg/BUILD.bazel b/examples/bzlmod/other_module/other_module/pkg/BUILD.bazel index 021c969802..4fe392841e 100644 --- a/examples/bzlmod/other_module/other_module/pkg/BUILD.bazel +++ b/examples/bzlmod/other_module/other_module/pkg/BUILD.bazel @@ -2,7 +2,7 @@ load( "@python_3_11//:defs.bzl", py_binary_311 = "py_binary", ) -load("@rules_python//python:defs.bzl", "py_library") +load("@rules_python//python:py_library.bzl", "py_library") py_library( name = "lib", diff --git a/examples/bzlmod/runfiles/BUILD.bazel b/examples/bzlmod/runfiles/BUILD.bazel index add56b3bd0..11a8ce0bb7 100644 --- a/examples/bzlmod/runfiles/BUILD.bazel +++ b/examples/bzlmod/runfiles/BUILD.bazel @@ -1,4 +1,4 @@ -load("@rules_python//python:defs.bzl", "py_test") +load("@rules_python//python:py_test.bzl", "py_test") py_test( name = "runfiles_test", diff --git a/examples/bzlmod/tests/BUILD.bazel b/examples/bzlmod/tests/BUILD.bazel index 96e4cdde25..dd50cf3294 100644 --- a/examples/bzlmod/tests/BUILD.bazel +++ b/examples/bzlmod/tests/BUILD.bazel @@ -2,7 +2,8 @@ load("@python_versions//3.10:defs.bzl", py_binary_3_10 = "py_binary", py_test_3_ load("@python_versions//3.11:defs.bzl", py_binary_3_11 = "py_binary", py_test_3_11 = "py_test") load("@python_versions//3.9:defs.bzl", py_binary_3_9 = "py_binary", py_test_3_9 = "py_test") load("@pythons_hub//:versions.bzl", "MINOR_MAPPING") -load("@rules_python//python:defs.bzl", "py_binary", "py_test") +load("@rules_python//python:py_binary.bzl", "py_binary") +load("@rules_python//python:py_test.bzl", "py_test") load("@rules_python//python/config_settings:transition.bzl", py_versioned_binary = "py_binary", py_versioned_test = "py_test") load("@rules_shell//shell:sh_test.bzl", "sh_test") diff --git a/examples/bzlmod/whl_mods/BUILD.bazel b/examples/bzlmod/whl_mods/BUILD.bazel index 241d9c1073..7c5ab5056e 100644 --- a/examples/bzlmod/whl_mods/BUILD.bazel +++ b/examples/bzlmod/whl_mods/BUILD.bazel @@ -1,4 +1,4 @@ -load("@rules_python//python:defs.bzl", "py_test") +load("@rules_python//python:py_test.bzl", "py_test") exports_files( glob(["data/**"]), diff --git a/examples/bzlmod_build_file_generation/BUILD.bazel b/examples/bzlmod_build_file_generation/BUILD.bazel index 33d01f4119..a0047668cb 100644 --- a/examples/bzlmod_build_file_generation/BUILD.bazel +++ b/examples/bzlmod_build_file_generation/BUILD.bazel @@ -7,8 +7,10 @@ # requirements. load("@bazel_gazelle//:def.bzl", "gazelle") load("@pip//:requirements.bzl", "all_whl_requirements") -load("@rules_python//python:defs.bzl", "py_binary", "py_library", "py_test") load("@rules_python//python:pip.bzl", "compile_pip_requirements") +load("@rules_python//python:py_binary.bzl", "py_binary") +load("@rules_python//python:py_library.bzl", "py_library") +load("@rules_python//python:py_test.bzl", "py_test") load("@rules_python_gazelle_plugin//manifest:defs.bzl", "gazelle_python_manifest") load("@rules_python_gazelle_plugin//modules_mapping:def.bzl", "modules_mapping") diff --git a/examples/bzlmod_build_file_generation/other_module/other_module/pkg/BUILD.bazel b/examples/bzlmod_build_file_generation/other_module/other_module/pkg/BUILD.bazel index 9a130e3554..90d41e752e 100644 --- a/examples/bzlmod_build_file_generation/other_module/other_module/pkg/BUILD.bazel +++ b/examples/bzlmod_build_file_generation/other_module/other_module/pkg/BUILD.bazel @@ -1,4 +1,4 @@ -load("@rules_python//python:defs.bzl", "py_library") +load("@rules_python//python:py_library.bzl", "py_library") py_library( name = "lib", diff --git a/examples/bzlmod_build_file_generation/runfiles/BUILD.bazel b/examples/bzlmod_build_file_generation/runfiles/BUILD.bazel index 3503ac3017..8806668a3f 100644 --- a/examples/bzlmod_build_file_generation/runfiles/BUILD.bazel +++ b/examples/bzlmod_build_file_generation/runfiles/BUILD.bazel @@ -1,4 +1,4 @@ -load("@rules_python//python:defs.bzl", "py_test") +load("@rules_python//python:py_test.bzl", "py_test") # gazelle:ignore py_test( diff --git a/examples/multi_python_versions/libs/my_lib/BUILD.bazel b/examples/multi_python_versions/libs/my_lib/BUILD.bazel index 8c29f6083c..7ff62249c4 100644 --- a/examples/multi_python_versions/libs/my_lib/BUILD.bazel +++ b/examples/multi_python_versions/libs/my_lib/BUILD.bazel @@ -1,5 +1,5 @@ load("@pypi//:requirements.bzl", "requirement") -load("@rules_python//python:defs.bzl", "py_library") +load("@rules_python//python:py_library.bzl", "py_library") py_library( name = "my_lib", diff --git a/examples/multi_python_versions/tests/BUILD.bazel b/examples/multi_python_versions/tests/BUILD.bazel index 177de22230..d04ac6bb0a 100644 --- a/examples/multi_python_versions/tests/BUILD.bazel +++ b/examples/multi_python_versions/tests/BUILD.bazel @@ -6,7 +6,8 @@ load("@python//3.11:defs.bzl", py_binary_3_11 = "py_binary", py_test_3_11 = "py_ load("@python//3.8:defs.bzl", py_binary_3_8 = "py_binary", py_test_3_8 = "py_test") load("@python//3.9:defs.bzl", py_binary_3_9 = "py_binary", py_test_3_9 = "py_test") load("@pythons_hub//:versions.bzl", "MINOR_MAPPING", "PYTHON_VERSIONS") -load("@rules_python//python:defs.bzl", "py_binary", "py_test") +load("@rules_python//python:py_binary.bzl", "py_binary") +load("@rules_python//python:py_test.bzl", "py_test") load("@rules_python//python:versions.bzl", DEFAULT_MINOR_MAPPING = "MINOR_MAPPING", DEFAULT_TOOL_VERSIONS = "TOOL_VERSIONS") load("@rules_python//python/private:text_util.bzl", "render") # buildifier: disable=bzl-visibility load("@rules_shell//shell:sh_test.bzl", "sh_test") diff --git a/examples/pip_parse/BUILD.bazel b/examples/pip_parse/BUILD.bazel index fd744a2836..8bdbd94b2c 100644 --- a/examples/pip_parse/BUILD.bazel +++ b/examples/pip_parse/BUILD.bazel @@ -1,11 +1,12 @@ -load("@rules_python//python:defs.bzl", "py_binary", "py_test") load("@rules_python//python:pip.bzl", "compile_pip_requirements") +load("@rules_python//python:py_binary.bzl", "py_binary") +load("@rules_python//python:py_test.bzl", "py_test") load("@rules_python//python/entry_points:py_console_script_binary.bzl", "py_console_script_binary") # Toolchain setup, this is optional. # Demonstrate that we can use the same python interpreter for the toolchain and executing pip in pip install (see WORKSPACE). # -#load("@rules_python//python:defs.bzl", "py_runtime_pair") +#load("@rules_python//python:py_runtime_pair.bzl", "py_runtime_pair") # #py_runtime( # name = "python3_runtime", diff --git a/examples/pip_parse_vendored/BUILD.bazel b/examples/pip_parse_vendored/BUILD.bazel index e2b1f5d49b..8d81e4ba8b 100644 --- a/examples/pip_parse_vendored/BUILD.bazel +++ b/examples/pip_parse_vendored/BUILD.bazel @@ -1,8 +1,8 @@ load("@bazel_skylib//rules:build_test.bzl", "build_test") load("@bazel_skylib//rules:diff_test.bzl", "diff_test") load("@bazel_skylib//rules:write_file.bzl", "write_file") -load("@rules_python//python:defs.bzl", "py_test") load("@rules_python//python:pip.bzl", "compile_pip_requirements") +load("@rules_python//python:py_test.bzl", "py_test") load("//:requirements.bzl", "all_data_requirements", "all_requirements", "all_whl_requirements", "requirement") # This rule adds a convenient way to update the requirements.txt diff --git a/examples/pip_repository_annotations/BUILD.bazel b/examples/pip_repository_annotations/BUILD.bazel index bdf9df1274..4e10c51658 100644 --- a/examples/pip_repository_annotations/BUILD.bazel +++ b/examples/pip_repository_annotations/BUILD.bazel @@ -1,5 +1,5 @@ -load("@rules_python//python:defs.bzl", "py_test") load("@rules_python//python:pip.bzl", "compile_pip_requirements") +load("@rules_python//python:py_test.bzl", "py_test") exports_files( glob(["data/**"]), diff --git a/examples/py_proto_library/BUILD.bazel b/examples/py_proto_library/BUILD.bazel index 0158aa2d37..d782fb296d 100644 --- a/examples/py_proto_library/BUILD.bazel +++ b/examples/py_proto_library/BUILD.bazel @@ -1,4 +1,4 @@ -load("@rules_python//python:defs.bzl", "py_test") +load("@rules_python//python:py_test.bzl", "py_test") py_test( name = "pricetag_test", diff --git a/examples/wheel/BUILD.bazel b/examples/wheel/BUILD.bazel index 1eaf03525a..58a4301523 100644 --- a/examples/wheel/BUILD.bazel +++ b/examples/wheel/BUILD.bazel @@ -15,9 +15,10 @@ load("@bazel_skylib//rules:build_test.bzl", "build_test") load("@bazel_skylib//rules:write_file.bzl", "write_file") load("//examples/wheel/private:wheel_utils.bzl", "directory_writer", "make_variable_tags") -load("//python:defs.bzl", "py_library", "py_test") load("//python:packaging.bzl", "py_package", "py_wheel") load("//python:pip.bzl", "compile_pip_requirements") +load("//python:py_library.bzl", "py_library") +load("//python:py_test.bzl", "py_test") load("//python:versions.bzl", "gen_python_config_settings") load("//python/entry_points:py_console_script_binary.bzl", "py_console_script_binary") load("//python/private:bzlmod_enabled.bzl", "BZLMOD_ENABLED") # buildifier: disable=bzl-visibility diff --git a/examples/wheel/lib/BUILD.bazel b/examples/wheel/lib/BUILD.bazel index 755818daa1..c182143c1d 100644 --- a/examples/wheel/lib/BUILD.bazel +++ b/examples/wheel/lib/BUILD.bazel @@ -12,7 +12,7 @@ # See the License for the specific language governing permissions and # limitations under the License. -load("//python:defs.bzl", "py_library") +load("//python:py_library.bzl", "py_library") package(default_visibility = ["//visibility:public"]) diff --git a/examples/wheel/private/BUILD.bazel b/examples/wheel/private/BUILD.bazel index 3462d354d4..326fc3538c 100644 --- a/examples/wheel/private/BUILD.bazel +++ b/examples/wheel/private/BUILD.bazel @@ -1,4 +1,4 @@ -load("@rules_python//python:defs.bzl", "py_binary") +load("@rules_python//python:py_binary.bzl", "py_binary") py_binary( name = "directory_writer", diff --git a/python/private/proto/BUILD.bazel b/python/private/proto/BUILD.bazel index 65c09444f7..222be40d09 100644 --- a/python/private/proto/BUILD.bazel +++ b/python/private/proto/BUILD.bazel @@ -30,7 +30,7 @@ bzl_library( srcs = ["py_proto_library.bzl"], visibility = ["//python:__pkg__"], deps = [ - "//python:defs_bzl", + "//python:py_info_bzl", "@rules_proto//proto:defs", ], ) diff --git a/python/private/proto/py_proto_library.bzl b/python/private/proto/py_proto_library.bzl index ecb0938bcd..ff2d3d2bb3 100644 --- a/python/private/proto/py_proto_library.bzl +++ b/python/private/proto/py_proto_library.bzl @@ -15,7 +15,7 @@ """The implementation of the `py_proto_library` rule and its aspect.""" load("@rules_proto//proto:defs.bzl", "ProtoInfo", "proto_common") -load("//python:defs.bzl", "PyInfo") +load("//python:py_info.bzl", "PyInfo") load("//python/api:api.bzl", _py_common = "py_common") PY_PROTO_TOOLCHAIN = "@rules_python//python/proto:toolchain_type" diff --git a/python/private/pypi/deps.bzl b/python/private/pypi/deps.bzl index 8949ed4abe..c6691d7059 100644 --- a/python/private/pypi/deps.bzl +++ b/python/private/pypi/deps.bzl @@ -100,7 +100,7 @@ _RULE_DEPS = [ _GENERIC_WHEEL = """\ package(default_visibility = ["//visibility:public"]) -load("@rules_python//python:defs.bzl", "py_library") +load("@rules_python//python:py_library.bzl", "py_library") load("@rules_python//python/private:glob_excludes.bzl", "glob_excludes") py_library( diff --git a/python/private/pypi/generate_group_library_build_bazel.bzl b/python/private/pypi/generate_group_library_build_bazel.bzl index 54da066b42..571cfd6b3f 100644 --- a/python/private/pypi/generate_group_library_build_bazel.bzl +++ b/python/private/pypi/generate_group_library_build_bazel.bzl @@ -25,7 +25,7 @@ load( ) _PRELUDE = """\ -load("@rules_python//python:defs.bzl", "py_library") +load("@rules_python//python:py_library.bzl", "py_library") """ _GROUP_TEMPLATE = """\ diff --git a/python/private/pypi/whl_installer/BUILD.bazel b/python/private/pypi/whl_installer/BUILD.bazel index 5bce1a5bcc..5fb617004d 100644 --- a/python/private/pypi/whl_installer/BUILD.bazel +++ b/python/private/pypi/whl_installer/BUILD.bazel @@ -1,4 +1,5 @@ -load("//python:defs.bzl", "py_binary", "py_library") +load("//python:py_binary.bzl", "py_binary") +load("//python:py_library.bzl", "py_library") py_library( name = "lib", diff --git a/python/private/whl_filegroup/BUILD.bazel b/python/private/whl_filegroup/BUILD.bazel index 398b9af0d8..b4246ca080 100644 --- a/python/private/whl_filegroup/BUILD.bazel +++ b/python/private/whl_filegroup/BUILD.bazel @@ -1,5 +1,5 @@ load("@bazel_skylib//:bzl_library.bzl", "bzl_library") -load("//python:defs.bzl", "py_binary") +load("//python:py_binary.bzl", "py_binary") filegroup( name = "distribution", diff --git a/python/python.bzl b/python/python.bzl index 3e739ca55d..cfbf25b5b5 100644 --- a/python/python.bzl +++ b/python/python.bzl @@ -14,11 +14,7 @@ """Re-exports for some of the core Bazel Python rules. -This file is deprecated; please use the exports in defs.bzl instead. This is to -follow the new naming convention of putting core rules for a language -underneath @rules_//:defs.bzl. The exports in this file will be -disallowed in a future Bazel release by -`--incompatible_load_python_rules_from_bzl`. +This file is deprecated; please use the exports in `.bzl` files instead. """ def py_library(*args, **kwargs): diff --git a/python/runfiles/BUILD.bazel b/python/runfiles/BUILD.bazel index c1fc027fa4..a541b296a8 100644 --- a/python/runfiles/BUILD.bazel +++ b/python/runfiles/BUILD.bazel @@ -12,8 +12,8 @@ # See the License for the specific language governing permissions and # limitations under the License. -load("//python:defs.bzl", "py_library") load("//python:packaging.bzl", "py_wheel") +load("//python:py_library.bzl", "py_library") load("//python/private:bzlmod_enabled.bzl", "BZLMOD_ENABLED") filegroup( diff --git a/tests/base_rules/base_tests.bzl b/tests/base_rules/base_tests.bzl index 3518e6f57a..8e0d10d729 100644 --- a/tests/base_rules/base_tests.bzl +++ b/tests/base_rules/base_tests.bzl @@ -16,7 +16,7 @@ load("@rules_testing//lib:analysis_test.bzl", "analysis_test") load("@rules_testing//lib:truth.bzl", "matching") load("@rules_testing//lib:util.bzl", "PREVENT_IMPLICIT_BUILDING_TAGS", rt_util = "util") -load("//python:defs.bzl", "PyInfo") +load("//python:py_info.bzl", "PyInfo") load("//python/private:reexports.bzl", "BuiltinPyInfo") # buildifier: disable=bzl-visibility load("//tests/base_rules:util.bzl", pt_util = "util") load("//tests/support:py_info_subject.bzl", "py_info_subject") diff --git a/tests/base_rules/py_binary/py_binary_tests.bzl b/tests/base_rules/py_binary/py_binary_tests.bzl index 571955d3c6..86a9548f79 100644 --- a/tests/base_rules/py_binary/py_binary_tests.bzl +++ b/tests/base_rules/py_binary/py_binary_tests.bzl @@ -13,7 +13,7 @@ # limitations under the License. """Tests for py_binary.""" -load("//python:defs.bzl", "py_binary") +load("//python:py_binary.bzl", "py_binary") load( "//tests/base_rules:py_executable_base_tests.bzl", "create_executable_tests", diff --git a/tests/base_rules/py_library/py_library_tests.bzl b/tests/base_rules/py_library/py_library_tests.bzl index 526735af71..9b585b17ef 100644 --- a/tests/base_rules/py_library/py_library_tests.bzl +++ b/tests/base_rules/py_library/py_library_tests.bzl @@ -3,7 +3,8 @@ load("@rules_testing//lib:analysis_test.bzl", "analysis_test") load("@rules_testing//lib:truth.bzl", "matching") load("@rules_testing//lib:util.bzl", rt_util = "util") -load("//python:defs.bzl", "PyRuntimeInfo", "py_library") +load("//python:py_library.bzl", "py_library") +load("//python:py_runtime_info.bzl", "PyRuntimeInfo") load("//tests/base_rules:base_tests.bzl", "create_base_tests") load("//tests/base_rules:util.bzl", pt_util = "util") diff --git a/tests/base_rules/py_test/py_test_tests.bzl b/tests/base_rules/py_test/py_test_tests.bzl index 6bd31ed3f9..d4d839b392 100644 --- a/tests/base_rules/py_test/py_test_tests.bzl +++ b/tests/base_rules/py_test/py_test_tests.bzl @@ -15,7 +15,7 @@ load("@rules_testing//lib:analysis_test.bzl", "analysis_test") load("@rules_testing//lib:util.bzl", rt_util = "util") -load("//python:defs.bzl", "py_test") +load("//python:py_test.bzl", "py_test") load( "//tests/base_rules:py_executable_base_tests.bzl", "create_executable_tests", diff --git a/tests/load_from_macro/BUILD.bazel b/tests/load_from_macro/BUILD.bazel index 00d7bf90ca..ecb5de51a7 100644 --- a/tests/load_from_macro/BUILD.bazel +++ b/tests/load_from_macro/BUILD.bazel @@ -12,7 +12,7 @@ # See the License for the specific language governing permissions and # limitations under the License. -load("//python:defs.bzl", "py_library") +load("//python:py_library.bzl", "py_library") load(":tags.bzl", "TAGS") licenses(["notice"]) diff --git a/tests/pycross/BUILD.bazel b/tests/pycross/BUILD.bazel index 52d1d18480..e90b60e17e 100644 --- a/tests/pycross/BUILD.bazel +++ b/tests/pycross/BUILD.bazel @@ -12,7 +12,7 @@ # See the License for the specific language governing permissions and # limitations under the License. -load("//python:defs.bzl", "py_test") +load("//python:py_test.bzl", "py_test") load("//third_party/rules_pycross/pycross/private:wheel_library.bzl", "py_wheel_library") # buildifier: disable=bzl-visibility py_wheel_library( diff --git a/tests/pypi/generate_group_library_build_bazel/generate_group_library_build_bazel_tests.bzl b/tests/pypi/generate_group_library_build_bazel/generate_group_library_build_bazel_tests.bzl index a46aa413a3..a91f861a36 100644 --- a/tests/pypi/generate_group_library_build_bazel/generate_group_library_build_bazel_tests.bzl +++ b/tests/pypi/generate_group_library_build_bazel/generate_group_library_build_bazel_tests.bzl @@ -21,7 +21,7 @@ _tests = [] def _test_simple(env): want = """\ -load("@rules_python//python:defs.bzl", "py_library") +load("@rules_python//python:py_library.bzl", "py_library") ## Group vbap @@ -62,7 +62,7 @@ _tests.append(_test_simple) def _test_in_hub(env): want = """\ -load("@rules_python//python:defs.bzl", "py_library") +load("@rules_python//python:py_library.bzl", "py_library") ## Group vbap diff --git a/tests/pypi/whl_installer/BUILD.bazel b/tests/pypi/whl_installer/BUILD.bazel index e25c4a06a4..040e4d765f 100644 --- a/tests/pypi/whl_installer/BUILD.bazel +++ b/tests/pypi/whl_installer/BUILD.bazel @@ -1,4 +1,4 @@ -load("//python:defs.bzl", "py_test") +load("//python:py_test.bzl", "py_test") alias( name = "lib", diff --git a/tests/whl_filegroup/BUILD.bazel b/tests/whl_filegroup/BUILD.bazel index 2176e9e03a..61c1aa49ac 100644 --- a/tests/whl_filegroup/BUILD.bazel +++ b/tests/whl_filegroup/BUILD.bazel @@ -1,9 +1,10 @@ load("@bazel_skylib//rules:write_file.bzl", "write_file") load("@rules_cc//cc:cc_library.bzl", "cc_library") load("@rules_cc//cc:cc_test.bzl", "cc_test") -load("//python:defs.bzl", "py_library", "py_test") load("//python:packaging.bzl", "py_package", "py_wheel") load("//python:pip.bzl", "whl_filegroup") +load("//python:py_library.bzl", "py_library") +load("//python:py_test.bzl", "py_test") load(":whl_filegroup_tests.bzl", "whl_filegroup_test_suite") whl_filegroup_test_suite(name = "whl_filegroup_tests") diff --git a/third_party/rules_pycross/pycross/private/wheel_library.bzl b/third_party/rules_pycross/pycross/private/wheel_library.bzl index 166e1d06eb..3d6ee32562 100644 --- a/third_party/rules_pycross/pycross/private/wheel_library.bzl +++ b/third_party/rules_pycross/pycross/private/wheel_library.bzl @@ -16,7 +16,7 @@ """Implementation of the py_wheel_library rule.""" load("@bazel_skylib//lib:paths.bzl", "paths") -load("//python:defs.bzl", "PyInfo") +load("//python:py_info.bzl", "PyInfo") load(":providers.bzl", "PyWheelInfo") def _py_wheel_library_impl(ctx): diff --git a/tools/BUILD.bazel b/tools/BUILD.bazel index 4f42bcb02d..0fcce8f729 100644 --- a/tools/BUILD.bazel +++ b/tools/BUILD.bazel @@ -11,7 +11,7 @@ # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # See the License for the specific language governing permissions and # limitations under the License. -load("//python:defs.bzl", "py_binary") +load("//python:py_binary.bzl", "py_binary") package(default_visibility = ["//visibility:public"]) From 094a25663918d04ed145de09f37cf74cf67d3445 Mon Sep 17 00:00:00 2001 From: Ignas Anikevicius <240938+aignas@users.noreply.github.com> Date: Sun, 8 Dec 2024 02:20:25 +0900 Subject: [PATCH 015/922] feat(pypi): support freethreaded in experimental_index_url (#2460) With this we: * Fix the previous behaviour where `abi3` wheels would be selected when freethreaded builds are selected. Whilst this may work in practise sometimes, I am not sure it has been supported by reading PEP703. * Start selecting `cp313t` wheels when we scan what is available on PyPI. * Ensure that the `whl_library` repository rule handles `cp313t` wheel extraction. * Generate `cp313t` config_settings so that we can use them in `pkg_aliases`. * Generate `cp313t` references in `pkg_aliases` macro. * Add the 3.13 deps to dev_pip for testing. Also tested by manually running: ``` $ bazel cquery --//python/config_settings:python_version=3.13 --//python/config_settings:py_freethreaded=yes 'kind("py_library rule", deps(@dev_pip//markupsafe))' INFO: Analyzed target @@_main~pip~dev_pip//markupsafe:markupsafe (3 packages loaded, 4091 targets configured). INFO: Found 1 target... @@_main~pip~dev_pip_313_markupsafe_cp313_cp313t_manylinux_2_17_x86_64_c0ef13ea//:pkg (008c5a5) $bazel build --//python/config_settings:python_version=3.13 --//python/config_settings:py_freethreaded=yes @dev_pip//markupsafe ``` Fixes #2386 --- MODULE.bazel | 7 ++ python/config_settings/BUILD.bazel | 6 ++ python/private/pypi/config_settings.bzl | 101 ++++++++++++------ python/private/pypi/pkg_aliases.bzl | 4 +- python/private/pypi/whl_library.bzl | 2 +- python/private/pypi/whl_target_platforms.bzl | 4 + .../config_settings/config_settings_tests.bzl | 50 +++++++++ tests/pypi/pkg_aliases/pkg_aliases_test.bzl | 10 ++ .../whl_target_platforms/select_whl_tests.bzl | 20 ++++ 9 files changed, 171 insertions(+), 33 deletions(-) diff --git a/MODULE.bazel b/MODULE.bazel index 2ae3173094..e4b113e785 100644 --- a/MODULE.bazel +++ b/MODULE.bazel @@ -121,6 +121,13 @@ dev_pip.parse( python_version = "3.11", requirements_lock = "//docs:requirements.txt", ) +dev_pip.parse( + download_only = True, + experimental_index_url = "https://pypi.org/simple", + hub_name = "dev_pip", + python_version = "3.13.0", + requirements_lock = "//docs:requirements.txt", +) dev_pip.parse( download_only = True, experimental_index_url = "https://pypi.org/simple", diff --git a/python/config_settings/BUILD.bazel b/python/config_settings/BUILD.bazel index aa26e6e669..5455f5aef7 100644 --- a/python/config_settings/BUILD.bazel +++ b/python/config_settings/BUILD.bazel @@ -106,6 +106,12 @@ config_setting( visibility = ["//visibility:public"], ) +config_setting( + name = "is_py_non_freethreaded", + flag_values = {":py_freethreaded": FreeThreadedFlag.NO}, + visibility = ["//visibility:public"], +) + # pip.parse related flags string_flag( diff --git a/python/private/pypi/config_settings.bzl b/python/private/pypi/config_settings.bzl index 6f927f2a4c..620e50e997 100644 --- a/python/private/pypi/config_settings.bzl +++ b/python/private/pypi/config_settings.bzl @@ -20,20 +20,21 @@ that matches the target platform. We can leverage this fact to ensure that the most specialized wheels are used by default with the users being able to configure string_flag values to select the less specialized ones. -The list of specialization of the dists goes like follows: +The list of specialization of the dists goes like follows (cpxyt stands for freethreaded +environments): * sdist * py*-none-any.whl * py*-abi3-any.whl -* py*-cpxy-any.whl +* py*-cpxy-any.whl or py*-cpxyt-any.whl * cp*-none-any.whl * cp*-abi3-any.whl -* cp*-cpxy-plat.whl +* cp*-cpxy-any.whl or cp*-cpxyt-any.whl * py*-none-plat.whl * py*-abi3-plat.whl -* py*-cpxy-plat.whl +* py*-cpxy-plat.whl or py*-cpxyt-plat.whl * cp*-none-plat.whl * cp*-abi3-plat.whl -* cp*-cpxy-plat.whl +* cp*-cpxy-plat.whl or cp*-cpxyt-plat.whl Note, that here the specialization of musl vs manylinux wheels is the same in order to ensure that the matching fails if the user requests for `musl` and we don't have it or vice versa. @@ -46,19 +47,24 @@ FLAGS = struct( **{ f: str(Label("//python/config_settings:" + f)) for f in [ - "python_version", + "is_pip_whl_auto", + "is_pip_whl_no", + "is_pip_whl_only", + "is_py_freethreaded", + "is_py_non_freethreaded", "pip_whl_glibc_version", "pip_whl_muslc_version", "pip_whl_osx_arch", "pip_whl_osx_version", "py_linux_libc", - "is_pip_whl_no", - "is_pip_whl_only", - "is_pip_whl_auto", + "python_version", ] } ) +_DEFAULT = "//conditions:default" +_INCOMPATIBLE = "@platforms//:incompatible" + # Here we create extra string flags that are just to work with the select # selecting the most specialized match. We don't allow the user to change # them. @@ -170,52 +176,70 @@ def _dist_config_settings(*, suffix, plat_flag_values, **kwargs): **kwargs ) - for name, f in [ - ("py_none", _flags.whl_py2_py3), - ("py3_none", _flags.whl_py3), - ("py3_abi3", _flags.whl_py3_abi3), - ("cp3x_none", _flags.whl_pycp3x), - ("cp3x_abi3", _flags.whl_pycp3x_abi3), - ("cp3x_cp", _flags.whl_pycp3x_abicp), + used_flags = {} + + # NOTE @aignas 2024-12-01: the abi3 is not compatible with freethreaded + # builds as per PEP703 (https://peps.python.org/pep-0703/#backwards-compatibility) + # + # The discussion here also reinforces this notion: + # https://discuss.python.org/t/pep-703-making-the-global-interpreter-lock-optional-3-12-updates/26503/99 + + for name, f, abi in [ + ("py_none", _flags.whl_py2_py3, None), + ("py3_none", _flags.whl_py3, None), + ("py3_abi3", _flags.whl_py3_abi3, (FLAGS.is_py_non_freethreaded,)), + ("cp3x_none", _flags.whl_pycp3x, None), + ("cp3x_abi3", _flags.whl_pycp3x_abi3, (FLAGS.is_py_non_freethreaded,)), + # The below are not specializations of one another, they are variants + ("cp3x_cp", _flags.whl_pycp3x_abicp, (FLAGS.is_py_non_freethreaded,)), + ("cp3x_cpt", _flags.whl_pycp3x_abicp, (FLAGS.is_py_freethreaded,)), ]: - if f in flag_values: + if (f, abi) in used_flags: # This should never happen as all of the different whls should have - # unique flag values. + # unique flag values fail("BUG: the flag {} is attempted to be added twice to the list".format(f)) else: flag_values[f] = "" + used_flags[(f, abi)] = True _dist_config_setting( name = "{}_any{}".format(name, suffix), flag_values = flag_values, is_pip_whl = FLAGS.is_pip_whl_only, + abi = abi, **kwargs ) generic_flag_values = flag_values + generic_used_flags = used_flags for (suffix, flag_values) in plat_flag_values: + used_flags = {(f, None): True for f in flag_values} | generic_used_flags flag_values = flag_values | generic_flag_values - for name, f in [ - ("py_none", _flags.whl_plat), - ("py3_none", _flags.whl_plat_py3), - ("py3_abi3", _flags.whl_plat_py3_abi3), - ("cp3x_none", _flags.whl_plat_pycp3x), - ("cp3x_abi3", _flags.whl_plat_pycp3x_abi3), - ("cp3x_cp", _flags.whl_plat_pycp3x_abicp), + for name, f, abi in [ + ("py_none", _flags.whl_plat, None), + ("py3_none", _flags.whl_plat_py3, None), + ("py3_abi3", _flags.whl_plat_py3_abi3, (FLAGS.is_py_non_freethreaded,)), + ("cp3x_none", _flags.whl_plat_pycp3x, None), + ("cp3x_abi3", _flags.whl_plat_pycp3x_abi3, (FLAGS.is_py_non_freethreaded,)), + # The below are not specializations of one another, they are variants + ("cp3x_cp", _flags.whl_plat_pycp3x_abicp, (FLAGS.is_py_non_freethreaded,)), + ("cp3x_cpt", _flags.whl_plat_pycp3x_abicp, (FLAGS.is_py_freethreaded,)), ]: - if f in flag_values: + if (f, abi) in used_flags: # This should never happen as all of the different whls should have # unique flag values. fail("BUG: the flag {} is attempted to be added twice to the list".format(f)) else: flag_values[f] = "" + used_flags[(f, abi)] = True _dist_config_setting( name = "{}_{}".format(name, suffix), flag_values = flag_values, is_pip_whl = FLAGS.is_pip_whl_only, + abi = abi, **kwargs ) @@ -285,7 +309,7 @@ def _plat_flag_values(os, cpu, osx_versions, glibc_versions, muslc_versions): return ret -def _dist_config_setting(*, name, is_python, python_version, is_pip_whl = None, native = native, **kwargs): +def _dist_config_setting(*, name, is_python, python_version, is_pip_whl = None, abi = None, native = native, **kwargs): """A macro to create a target that matches is_pip_whl_auto and one more value. Args: @@ -294,6 +318,10 @@ def _dist_config_setting(*, name, is_python, python_version, is_pip_whl = None, `is_pip_whl_auto` when evaluating the config setting. is_python: The python version config_setting to match. python_version: The python version name. + abi: {type}`tuple[Label]` A collection of ABI config settings that are + compatible with the given dist config setting. For example, if only + non-freethreaded python builds are allowed, add + FLAGS.is_py_non_freethreaded here. native (struct): The struct containing alias and config_setting rules to use for creating the objects. Can be overridden for unit tests reasons. @@ -306,9 +334,9 @@ def _dist_config_setting(*, name, is_python, python_version, is_pip_whl = None, native.alias( name = "is_cp{}_{}".format(python_version, name) if python_version else "is_{}".format(name), actual = select({ - # First match by the python version - is_python: _name, - "//conditions:default": is_python, + # First match by the python version and then by ABI + is_python: _name + ("_abi" if abi else ""), + _DEFAULT: _INCOMPATIBLE, }), visibility = visibility, ) @@ -325,12 +353,23 @@ def _dist_config_setting(*, name, is_python, python_version, is_pip_whl = None, config_setting_name = _name + "_setting" native.config_setting(name = config_setting_name, **kwargs) + if abi: + native.alias( + name = _name + "_abi", + actual = select( + {k: _name for k in abi} | { + _DEFAULT: _INCOMPATIBLE, + }, + ), + visibility = visibility, + ) + # Next match by the `pip_whl` flag value and then match by the flags that # are intrinsic to the distribution. native.alias( name = _name, actual = select({ - "//conditions:default": FLAGS.is_pip_whl_auto, + _DEFAULT: _INCOMPATIBLE, FLAGS.is_pip_whl_auto: config_setting_name, is_pip_whl: config_setting_name, }), diff --git a/python/private/pypi/pkg_aliases.bzl b/python/private/pypi/pkg_aliases.bzl index 5a3f84199b..a6872fdce9 100644 --- a/python/private/pypi/pkg_aliases.bzl +++ b/python/private/pypi/pkg_aliases.bzl @@ -308,7 +308,9 @@ def get_filename_config_settings( else: py = "py3" - if parsed.abi_tag.startswith("cp"): + if parsed.abi_tag.startswith("cp") and parsed.abi_tag.endswith("t"): + abi = "cpt" + elif parsed.abi_tag.startswith("cp"): abi = "cp" else: abi = parsed.abi_tag diff --git a/python/private/pypi/whl_library.bzl b/python/private/pypi/whl_library.bzl index 612ca2cfdf..79a58a81f2 100644 --- a/python/private/pypi/whl_library.bzl +++ b/python/private/pypi/whl_library.bzl @@ -287,7 +287,7 @@ def _whl_library_impl(rctx): p.target_platform for p in whl_target_platforms( platform_tag = parsed_whl.platform_tag, - abi_tag = parsed_whl.abi_tag, + abi_tag = parsed_whl.abi_tag.strip("tm"), ) ] diff --git a/python/private/pypi/whl_target_platforms.bzl b/python/private/pypi/whl_target_platforms.bzl index bdc44c697a..6823199bee 100644 --- a/python/private/pypi/whl_target_platforms.bzl +++ b/python/private/pypi/whl_target_platforms.bzl @@ -89,6 +89,10 @@ def select_whls(*, whls, want_platforms = [], logger = None): want_abis[abi] = None want_abis[abi + "m"] = None + # Also add freethreaded wheels if we find them since we started supporting them + _want_platforms["{}t_{}".format(abi, os_cpu)] = None + want_abis[abi + "t"] = None + want_platforms = sorted(_want_platforms) candidates = {} diff --git a/tests/pypi/config_settings/config_settings_tests.bzl b/tests/pypi/config_settings/config_settings_tests.bzl index a77fa5b66b..049556a4c6 100644 --- a/tests/pypi/config_settings/config_settings_tests.bzl +++ b/tests/pypi/config_settings/config_settings_tests.bzl @@ -39,6 +39,7 @@ _flag = struct( pip_whl_osx_arch = lambda x: (str(Label("//python/config_settings:pip_whl_osx_arch")), str(x)), py_linux_libc = lambda x: (str(Label("//python/config_settings:py_linux_libc")), str(x)), python_version = lambda x: (str(Label("//python/config_settings:python_version")), str(x)), + py_freethreaded = lambda x: (str(Label("//python/config_settings:py_freethreaded")), str(x)), ) def _analysis_test(*, name, dist, want, config_settings = [_flag.platform("linux_aarch64")]): @@ -286,6 +287,38 @@ def _test_py_none_any_versioned(name): _tests.append(_test_py_none_any_versioned) +def _test_cp_whl_is_not_prefered_over_py3_non_freethreaded(name): + _analysis_test( + name = name, + dist = { + "is_cp3.7_cp3x_abi3_any": "py3_abi3", + "is_cp3.7_cp3x_cpt_any": "cp", + "is_cp3.7_cp3x_none_any": "py3", + }, + want = "py3_abi3", + config_settings = [ + _flag.py_freethreaded("no"), + ], + ) + +_tests.append(_test_cp_whl_is_not_prefered_over_py3_non_freethreaded) + +def _test_cp_whl_is_not_prefered_over_py3_freethreaded(name): + _analysis_test( + name = name, + dist = { + "is_cp3.7_cp3x_abi3_any": "py3_abi3", + "is_cp3.7_cp3x_cp_any": "cp", + "is_cp3.7_cp3x_none_any": "py3", + }, + want = "py3", + config_settings = [ + _flag.py_freethreaded("yes"), + ], + ) + +_tests.append(_test_cp_whl_is_not_prefered_over_py3_freethreaded) + def _test_cp_cp_whl(name): _analysis_test( name = name, @@ -412,6 +445,7 @@ def _test_windows(name): name = name, dist = { "is_cp3.7_cp3x_cp_windows_x86_64": "whl", + "is_cp3.7_cp3x_cpt_windows_x86_64": "whl_freethreaded", }, want = "whl", config_settings = [ @@ -421,6 +455,22 @@ def _test_windows(name): _tests.append(_test_windows) +def _test_windows_freethreaded(name): + _analysis_test( + name = name, + dist = { + "is_cp3.7_cp3x_cp_windows_x86_64": "whl", + "is_cp3.7_cp3x_cpt_windows_x86_64": "whl_freethreaded", + }, + want = "whl_freethreaded", + config_settings = [ + _flag.platform("windows_x86_64"), + _flag.py_freethreaded("yes"), + ], + ) + +_tests.append(_test_windows_freethreaded) + def _test_osx(name): _analysis_test( name = name, diff --git a/tests/pypi/pkg_aliases/pkg_aliases_test.bzl b/tests/pypi/pkg_aliases/pkg_aliases_test.bzl index 0fa66d05eb..23a0f01db9 100644 --- a/tests/pypi/pkg_aliases/pkg_aliases_test.bzl +++ b/tests/pypi/pkg_aliases/pkg_aliases_test.bzl @@ -287,6 +287,14 @@ def _test_multiplatform_whl_aliases_filename(env): filename = "foo-0.0.1-py3-none-any.whl", version = "3.1", ): "foo-py3-0.0.1", + whl_config_setting( + filename = "foo-0.0.1-cp313-cp313-any.whl", + version = "3.1", + ): "foo-cp-0.0.1", + whl_config_setting( + filename = "foo-0.0.1-cp313-cp313t-any.whl", + version = "3.1", + ): "foo-cpt-0.0.1", whl_config_setting( filename = "foo-0.0.2-py3-none-any.whl", version = "3.1", @@ -303,6 +311,8 @@ def _test_multiplatform_whl_aliases_filename(env): osx_versions = [], ) want = { + "//_config:is_cp3.1_cp3x_cp_any": "foo-cp-0.0.1", + "//_config:is_cp3.1_cp3x_cpt_any": "foo-cpt-0.0.1", "//_config:is_cp3.1_py3_none_any": "foo-py3-0.0.1", "//_config:is_cp3.1_py3_none_any_linux_aarch64": "foo-0.0.2", "//_config:is_cp3.1_py3_none_any_linux_x86_64": "foo-0.0.2", diff --git a/tests/pypi/whl_target_platforms/select_whl_tests.bzl b/tests/pypi/whl_target_platforms/select_whl_tests.bzl index 2994bd513f..8ab24138d1 100644 --- a/tests/pypi/whl_target_platforms/select_whl_tests.bzl +++ b/tests/pypi/whl_target_platforms/select_whl_tests.bzl @@ -27,6 +27,10 @@ WHL_LIST = [ "pkg-0.0.1-cp311-cp311-manylinux_2_17_s390x.manylinux2014_s390x.whl", "pkg-0.0.1-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", "pkg-0.0.1-cp311-cp311-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", + "pkg-0.0.1-cp313-cp313t-musllinux_1_1_x86_64.whl", + "pkg-0.0.1-cp313-cp313-musllinux_1_1_x86_64.whl", + "pkg-0.0.1-cp313-abi3-musllinux_1_1_x86_64.whl", + "pkg-0.0.1-cp313-none-musllinux_1_1_x86_64.whl", "pkg-0.0.1-cp311-cp311-musllinux_1_1_aarch64.whl", "pkg-0.0.1-cp311-cp311-musllinux_1_1_i686.whl", "pkg-0.0.1-cp311-cp311-musllinux_1_1_ppc64le.whl", @@ -269,6 +273,22 @@ def _test_prefer_manylinux_wheels(env): _tests.append(_test_prefer_manylinux_wheels) +def _test_freethreaded_wheels(env): + # Check we prefer platform specific wheels + got = _select_whls(whls = WHL_LIST, want_platforms = ["cp313_linux_x86_64"]) + _match( + env, + got, + "pkg-0.0.1-cp313-cp313t-musllinux_1_1_x86_64.whl", + "pkg-0.0.1-cp313-cp313-musllinux_1_1_x86_64.whl", + "pkg-0.0.1-cp313-abi3-musllinux_1_1_x86_64.whl", + "pkg-0.0.1-cp313-none-musllinux_1_1_x86_64.whl", + "pkg-0.0.1-cp39-abi3-any.whl", + "pkg-0.0.1-py3-none-any.whl", + ) + +_tests.append(_test_freethreaded_wheels) + def select_whl_test_suite(name): """Create the test suite. From 42930ccf04277648781003a0f9add82fbcc6c812 Mon Sep 17 00:00:00 2001 From: Simon Stewart Date: Sat, 7 Dec 2024 22:47:41 +0000 Subject: [PATCH 016/922] fix: Make sure wheelmaker uses the default shell env (#2477) Clean macOS installs place `python` in `/usr/local/bin`, which is not searched by default when calling `actions.run()`. This caused builds to fail when they were executing python tools on an unmodified macOS host. --------- Co-authored-by: Richard Levasseur Co-authored-by: Richard Levasseur --- CHANGELOG.md | 3 ++- python/private/py_wheel.bzl | 3 +++ 2 files changed, 5 insertions(+), 1 deletion(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 7bdc5e4483..10be235f38 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -56,7 +56,8 @@ Unreleased changes template. {#v0-0-0-fixed} ### Fixed -* Nothing fixed. +* (py_wheel) Use the default shell environment when building wheels to allow + toolchains that search PATH to be used for the wheel builder tool. {#v0-0-0-added} ### Added diff --git a/python/private/py_wheel.bzl b/python/private/py_wheel.bzl index 6d047ad680..b5fbec9ce0 100644 --- a/python/private/py_wheel.bzl +++ b/python/private/py_wheel.bzl @@ -514,6 +514,9 @@ def _py_wheel_impl(ctx): outputs = [outfile, name_file], arguments = [args], executable = ctx.executable._wheelmaker, + # The default shell env is used to better support toolchains that look + # up python at runtime using PATH. + use_default_shell_env = True, progress_message = "Building wheel {}".format(ctx.label), ) return [ From 679c553a15c6afe23f41bdbcdf51e3b6f3afb858 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Sat, 7 Dec 2024 18:39:25 -0800 Subject: [PATCH 017/922] chore: remove find_requirements re-export from bazel_tools (#2484) This removes re-exporting the `find_requirements` aspect from `@bazel_tools`. The find_requirements aspect is only useful for analyzing the `srcs_version` relationship between targets, which was only needed as part of the Python 2 to 3 upgrade. With Python 2 no longer supported, it's defunct. Removing it entirely because I can't find any real references to it in the wild. It looks entirely unused. --- CHANGELOG.md | 2 +- python/BUILD.bazel | 1 - python/defs.bzl | 5 ----- 3 files changed, 1 insertion(+), 7 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 10be235f38..803b5081d3 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -65,7 +65,7 @@ Unreleased changes template. {#v0-0-0-removed} ### Removed -* Nothing removed. +* `find_requirements` in `//python:defs.bzl` has been removed. {#v1-0-0} ## [1.0.0] - 2024-12-05 diff --git a/python/BUILD.bazel b/python/BUILD.bazel index f3b5b136a7..3422ef14fa 100644 --- a/python/BUILD.bazel +++ b/python/BUILD.bazel @@ -71,7 +71,6 @@ bzl_library( ":py_runtime_info_bzl", ":py_runtime_pair_bzl", ":py_test_bzl", - "//python/private:bazel_tools_bzl", ], ) diff --git a/python/defs.bzl b/python/defs.bzl index bd89f5b1f2..bdf5dae2e4 100644 --- a/python/defs.bzl +++ b/python/defs.bzl @@ -13,7 +13,6 @@ # limitations under the License. """Core rules for building Python projects.""" -load("@bazel_tools//tools/python:srcs_version.bzl", _find_requirements = "find_requirements") load("//python:py_binary.bzl", _py_binary = "py_binary") load("//python:py_info.bzl", _PyInfo = "PyInfo") load("//python:py_library.bzl", _py_library = "py_library") @@ -34,12 +33,8 @@ current_py_toolchain = _current_py_toolchain py_import = _py_import -# Re-exports of Starlark-defined symbols in @bazel_tools//tools/python. - py_runtime_pair = _py_runtime_pair -find_requirements = _find_requirements - py_library = _py_library py_binary = _py_binary From 6b0e40a3305236cf1a2ac3ffeb5865ad6679bee5 Mon Sep 17 00:00:00 2001 From: "Elvis M. Wianda" <7077790+ewianda@users.noreply.github.com> Date: Sat, 7 Dec 2024 20:19:34 -0700 Subject: [PATCH 018/922] feat(gazelle): Include types/stubs packages (#2425) This PR adds logic that checks if a package has a corresponding `types` or `stubs` package and automatically adds that to the BUILD file. This is useful for typeckers e.g pyright , mypy --------- Co-authored-by: Ignas Anikevicius <240938+aignas@users.noreply.github.com> --- CHANGELOG.md | 5 +- .../bzlmod_build_file_generation/BUILD.bazel | 23 +++++++ .../gazelle_python.yaml | 5 ++ .../gazelle_python_with_types.yaml | 42 +++++++++++++ .../requirements.in | 3 + .../requirements_lock.txt | 40 ++++++++++++- .../requirements_windows.txt | 60 +++++++++++++++++-- gazelle/MODULE.bazel | 11 ++++ gazelle/README.md | 10 ++++ gazelle/WORKSPACE | 4 ++ gazelle/internal_dev_deps.bzl | 47 +++++++++++++++ gazelle/modules_mapping/BUILD.bazel | 27 ++++++++- gazelle/modules_mapping/def.bzl | 7 +++ gazelle/modules_mapping/generator.py | 18 +++++- gazelle/modules_mapping/test_generator.py | 44 ++++++++++++++ gazelle/python/resolve.go | 14 ++++- .../testdata/add_type_stub_packages/BUILD.in | 0 .../testdata/add_type_stub_packages/BUILD.out | 14 +++++ .../testdata/add_type_stub_packages/README.md | 4 ++ .../testdata/add_type_stub_packages/WORKSPACE | 1 + .../add_type_stub_packages/__main__.py | 16 +++++ .../gazelle_python.yaml | 22 +++++++ .../testdata/add_type_stub_packages/test.yaml | 15 +++++ gazelle/pythonconfig/pythonconfig.go | 6 +- 24 files changed, 424 insertions(+), 14 deletions(-) create mode 100644 examples/bzlmod_build_file_generation/gazelle_python_with_types.yaml create mode 100644 gazelle/internal_dev_deps.bzl create mode 100644 gazelle/modules_mapping/test_generator.py create mode 100644 gazelle/python/testdata/add_type_stub_packages/BUILD.in create mode 100644 gazelle/python/testdata/add_type_stub_packages/BUILD.out create mode 100644 gazelle/python/testdata/add_type_stub_packages/README.md create mode 100644 gazelle/python/testdata/add_type_stub_packages/WORKSPACE create mode 100644 gazelle/python/testdata/add_type_stub_packages/__main__.py create mode 100644 gazelle/python/testdata/add_type_stub_packages/gazelle_python.yaml create mode 100644 gazelle/python/testdata/add_type_stub_packages/test.yaml diff --git a/CHANGELOG.md b/CHANGELOG.md index 803b5081d3..10a2be4edf 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -61,7 +61,10 @@ Unreleased changes template. {#v0-0-0-added} ### Added -* Nothing added. +* (gazelle) Added `include_stub_packages` flag to `modules_mapping`. When set to `True`, this + automatically includes corresponding stub packages for third-party libraries + that are present and used (e.g., `boto3` → `boto3-stubs`), improving + type-checking support. {#v0-0-0-removed} ### Removed diff --git a/examples/bzlmod_build_file_generation/BUILD.bazel b/examples/bzlmod_build_file_generation/BUILD.bazel index a0047668cb..95bb5f88f4 100644 --- a/examples/bzlmod_build_file_generation/BUILD.bazel +++ b/examples/bzlmod_build_file_generation/BUILD.bazel @@ -32,11 +32,26 @@ modules_mapping( "^_|(\\._)+", # This is the default. "(\\.tests)+", # Add a custom one to get rid of the psutil tests. "^colorama", # Get rid of colorama on Windows. + "^tzdata", # Get rid of tzdata on Windows. "^lazy_object_proxy\\.cext$", # Get rid of this on Linux because it isn't included on Windows. ], wheels = all_whl_requirements, ) +modules_mapping( + name = "modules_map_with_types", + exclude_patterns = [ + "^_|(\\._)+", # This is the default. + "(\\.tests)+", # Add a custom one to get rid of the psutil tests. + "^colorama", # Get rid of colorama on Windows. + "^tzdata", # Get rid of tzdata on Windows. + "^lazy_object_proxy\\.cext$", # Get rid of this on Linux because it isn't included on Windows. + ], + include_stub_packages = True, + modules_mapping_name = "modules_mapping_with_types.json", + wheels = all_whl_requirements, +) + # Gazelle python extension needs a manifest file mapping from # an import to the installed package that provides it. # This macro produces two targets: @@ -54,6 +69,14 @@ gazelle_python_manifest( tags = ["exclusive"], ) +gazelle_python_manifest( + name = "gazelle_python_manifest_with_types", + manifest = "gazelle_python_with_types.yaml", + modules_mapping = ":modules_map_with_types", + pip_repository_name = "pip", + tags = ["exclusive"], +) + # Our gazelle target points to the python gazelle binary. # This is the simple case where we only need one language supported. # If you also had proto, go, or other gazelle-supported languages, diff --git a/examples/bzlmod_build_file_generation/gazelle_python.yaml b/examples/bzlmod_build_file_generation/gazelle_python.yaml index d0d322446e..c94f93a070 100644 --- a/examples/bzlmod_build_file_generation/gazelle_python.yaml +++ b/examples/bzlmod_build_file_generation/gazelle_python.yaml @@ -6,16 +6,20 @@ manifest: modules_mapping: S3: s3cmd + asgiref: asgiref astroid: astroid certifi: certifi chardet: chardet dateutil: python_dateutil dill: dill + django: Django + django_stubs_ext: django_stubs_ext idna: idna isort: isort lazy_object_proxy: lazy_object_proxy magic: python_magic mccabe: mccabe + mypy_django_plugin: django_stubs pathspec: pathspec pkg_resources: setuptools platformdirs: platformdirs @@ -23,6 +27,7 @@ manifest: requests: requests setuptools: setuptools six: six + sqlparse: sqlparse tabulate: tabulate tomli: tomli tomlkit: tomlkit diff --git a/examples/bzlmod_build_file_generation/gazelle_python_with_types.yaml b/examples/bzlmod_build_file_generation/gazelle_python_with_types.yaml new file mode 100644 index 0000000000..b6b0687ea4 --- /dev/null +++ b/examples/bzlmod_build_file_generation/gazelle_python_with_types.yaml @@ -0,0 +1,42 @@ +# GENERATED FILE - DO NOT EDIT! +# +# To update this file, run: +# bazel run //:gazelle_python_manifest_with_types.update + +manifest: + modules_mapping: + S3: s3cmd + asgiref: asgiref + astroid: astroid + certifi: certifi + chardet: chardet + dateutil: python_dateutil + dill: dill + django: Django + django_stubs: django_stubs + django_stubs_ext: django_stubs_ext + idna: idna + isort: isort + lazy_object_proxy: lazy_object_proxy + magic: python_magic + mccabe: mccabe + pathspec: pathspec + pkg_resources: setuptools + platformdirs: platformdirs + pylint: pylint + requests: requests + setuptools: setuptools + six: six + sqlparse: sqlparse + tabulate: tabulate + tomli: tomli + tomlkit: tomlkit + types_pyyaml: types_pyyaml + types_tabulate: types_tabulate + typing_extensions: typing_extensions + urllib3: urllib3 + wrapt: wrapt + yaml: PyYAML + yamllint: yamllint + pip_repository: + name: pip diff --git a/examples/bzlmod_build_file_generation/requirements.in b/examples/bzlmod_build_file_generation/requirements.in index a709195442..fb3b45176c 100644 --- a/examples/bzlmod_build_file_generation/requirements.in +++ b/examples/bzlmod_build_file_generation/requirements.in @@ -2,5 +2,8 @@ requests~=2.25.1 s3cmd~=2.1.0 yamllint>=1.28.0 tabulate~=0.9.0 +types-tabulate pylint~=2.15.5 python-dateutil>=2.8.2 +django +django-stubs diff --git a/examples/bzlmod_build_file_generation/requirements_lock.txt b/examples/bzlmod_build_file_generation/requirements_lock.txt index 9d9ad9453e..cdcebb72f7 100644 --- a/examples/bzlmod_build_file_generation/requirements_lock.txt +++ b/examples/bzlmod_build_file_generation/requirements_lock.txt @@ -4,6 +4,12 @@ # # bazel run //:requirements.update # +asgiref==3.8.1 \ + --hash=sha256:3e1e3ecc849832fe52ccf2cb6686b7a55f82bb1d6aee72a58826471390335e47 \ + --hash=sha256:c343bd80a0bec947a9860adb4c432ffa7db769836c64238fc34bdc3fec84d590 + # via + # django + # django-stubs astroid==2.12.13 \ --hash=sha256:10e0ad5f7b79c435179d0d0f0df69998c4eef4597534aae44910db060baeb907 \ --hash=sha256:1493fe8bd3dfd73dc35bd53c9d5b6e49ead98497c47b2307662556a5692d29d7 @@ -20,6 +26,21 @@ dill==0.3.6 \ --hash=sha256:a07ffd2351b8c678dfc4a856a3005f8067aea51d6ba6c700796a4d9e280f39f0 \ --hash=sha256:e5db55f3687856d8fbdab002ed78544e1c4559a130302693d839dfe8f93f2373 # via pylint +django==4.2.16 \ + --hash=sha256:1ddc333a16fc139fd253035a1606bb24261951bbc3a6ca256717fa06cc41a898 \ + --hash=sha256:6f1616c2786c408ce86ab7e10f792b8f15742f7b7b7460243929cb371e7f1dad + # via + # -r requirements.in + # django-stubs + # django-stubs-ext +django-stubs==5.0.0 \ + --hash=sha256:084484cbe16a6d388e80ec687e46f529d67a232f3befaf55c936b3b476be289d \ + --hash=sha256:b8a792bee526d6cab31e197cb414ee7fa218abd931a50948c66a80b3a2548621 + # via -r requirements.in +django-stubs-ext==5.1.1 \ + --hash=sha256:3907f99e178c93323e2ce908aef8352adb8c047605161f8d9e5e7b4efb5a6a9c \ + --hash=sha256:db7364e4f50ae7e5360993dbd58a3a57ea4b2e7e5bab0fbd525ccdb3e7975d1c + # via django-stubs idna==2.10 \ --hash=sha256:b307872f855b18632ce0c21c5e45be78c0ea7ae4c15c828c20788b26921eb3f6 \ --hash=sha256:b97d804b1e9b523befed77c48dacec60e6dcb0b5391d57af6a65a312a90648c0 @@ -129,6 +150,10 @@ six==1.16.0 \ --hash=sha256:1e61c37477a1626458e36f7b1d82aa5c9b094fa4802892072e49de9c60c4c926 \ --hash=sha256:8abb2f1d86890a2dfb989f9a77cfcfd3e47c2a354b01111771326f8aa26e0254 # via python-dateutil +sqlparse==0.5.2 \ + --hash=sha256:9e37b35e16d1cc652a2545f0997c1deb23ea28fa1f3eefe609eee3063c3b105f \ + --hash=sha256:e99bc85c78160918c3e1d9230834ab8d80fc06c59d03f8db2618f65f65dda55e + # via django tabulate==0.9.0 \ --hash=sha256:0095b12bf5966de529c0feb1fa08671671b3368eec77d7ef7ab114be2c068b3c \ --hash=sha256:024ca478df22e9340661486f85298cff5f6dcdba14f3813e8830015b9ed1948f @@ -136,16 +161,29 @@ tabulate==0.9.0 \ tomli==2.0.1 \ --hash=sha256:939de3e7a6161af0c887ef91b7d41a53e7c5a1ca976325f429cb46ea9bc30ecc \ --hash=sha256:de526c12914f0c550d15924c62d72abc48d6fe7364aa87328337a31007fe8a4f - # via pylint + # via + # django-stubs + # pylint tomlkit==0.11.6 \ --hash=sha256:07de26b0d8cfc18f871aec595fda24d95b08fef89d147caa861939f37230bf4b \ --hash=sha256:71b952e5721688937fb02cf9d354dbcf0785066149d2855e44531ebdd2b65d73 # via pylint +types-pyyaml==6.0.12.20240917 \ + --hash=sha256:392b267f1c0fe6022952462bf5d6523f31e37f6cea49b14cee7ad634b6301570 \ + --hash=sha256:d1405a86f9576682234ef83bcb4e6fff7c9305c8b1fbad5e0bcd4f7dbdc9c587 + # via django-stubs +types-tabulate==0.9.0.20240106 \ + --hash=sha256:0378b7b6fe0ccb4986299496d027a6d4c218298ecad67199bbd0e2d7e9d335a1 \ + --hash=sha256:c9b6db10dd7fcf55bd1712dd3537f86ddce72a08fd62bb1af4338c7096ce947e + # via -r requirements.in typing-extensions==4.4.0 \ --hash=sha256:1511434bb92bf8dd198c12b1cc812e800d4181cfcb867674e0f8279cc93087aa \ --hash=sha256:16fa4864408f655d35ec496218b85f79b3437c829e93320c7c9215ccfd92489e # via + # asgiref # astroid + # django-stubs + # django-stubs-ext # pylint urllib3==1.26.13 \ --hash=sha256:47cc05d99aaa09c9e72ed5809b60e7ba354e64b59c9c173ac3018642d8bb41fc \ diff --git a/examples/bzlmod_build_file_generation/requirements_windows.txt b/examples/bzlmod_build_file_generation/requirements_windows.txt index 5b31ff5541..e591c8dc80 100644 --- a/examples/bzlmod_build_file_generation/requirements_windows.txt +++ b/examples/bzlmod_build_file_generation/requirements_windows.txt @@ -4,6 +4,12 @@ # # bazel run //:requirements.update # +asgiref==3.8.1 \ + --hash=sha256:3e1e3ecc849832fe52ccf2cb6686b7a55f82bb1d6aee72a58826471390335e47 \ + --hash=sha256:c343bd80a0bec947a9860adb4c432ffa7db769836c64238fc34bdc3fec84d590 + # via + # django + # django-stubs astroid==2.12.13 \ --hash=sha256:10e0ad5f7b79c435179d0d0f0df69998c4eef4597534aae44910db060baeb907 \ --hash=sha256:1493fe8bd3dfd73dc35bd53c9d5b6e49ead98497c47b2307662556a5692d29d7 @@ -24,6 +30,21 @@ dill==0.3.6 \ --hash=sha256:a07ffd2351b8c678dfc4a856a3005f8067aea51d6ba6c700796a4d9e280f39f0 \ --hash=sha256:e5db55f3687856d8fbdab002ed78544e1c4559a130302693d839dfe8f93f2373 # via pylint +django==4.2.16 \ + --hash=sha256:1ddc333a16fc139fd253035a1606bb24261951bbc3a6ca256717fa06cc41a898 \ + --hash=sha256:6f1616c2786c408ce86ab7e10f792b8f15742f7b7b7460243929cb371e7f1dad + # via + # -r requirements.in + # django-stubs + # django-stubs-ext +django-stubs==5.1.1 \ + --hash=sha256:126d354bbdff4906c4e93e6361197f6fbfb6231c3df6def85a291dae6f9f577b \ + --hash=sha256:c4dc64260bd72e6d32b9e536e8dd0d9247922f0271f82d1d5132a18f24b388ac + # via -r requirements.in +django-stubs-ext==5.1.1 \ + --hash=sha256:3907f99e178c93323e2ce908aef8352adb8c047605161f8d9e5e7b4efb5a6a9c \ + --hash=sha256:db7364e4f50ae7e5360993dbd58a3a57ea4b2e7e5bab0fbd525ccdb3e7975d1c + # via django-stubs idna==2.10 \ --hash=sha256:b307872f855b18632ce0c21c5e45be78c0ea7ae4c15c828c20788b26921eb3f6 \ --hash=sha256:b97d804b1e9b523befed77c48dacec60e6dcb0b5391d57af6a65a312a90648c0 @@ -133,6 +154,10 @@ six==1.16.0 \ --hash=sha256:1e61c37477a1626458e36f7b1d82aa5c9b094fa4802892072e49de9c60c4c926 \ --hash=sha256:8abb2f1d86890a2dfb989f9a77cfcfd3e47c2a354b01111771326f8aa26e0254 # via python-dateutil +sqlparse==0.5.2 \ + --hash=sha256:9e37b35e16d1cc652a2545f0997c1deb23ea28fa1f3eefe609eee3063c3b105f \ + --hash=sha256:e99bc85c78160918c3e1d9230834ab8d80fc06c59d03f8db2618f65f65dda55e + # via django tabulate==0.9.0 \ --hash=sha256:0095b12bf5966de529c0feb1fa08671671b3368eec77d7ef7ab114be2c068b3c \ --hash=sha256:024ca478df22e9340661486f85298cff5f6dcdba14f3813e8830015b9ed1948f @@ -140,17 +165,34 @@ tabulate==0.9.0 \ tomli==2.0.1 \ --hash=sha256:939de3e7a6161af0c887ef91b7d41a53e7c5a1ca976325f429cb46ea9bc30ecc \ --hash=sha256:de526c12914f0c550d15924c62d72abc48d6fe7364aa87328337a31007fe8a4f - # via pylint + # via + # django-stubs + # pylint tomlkit==0.11.6 \ --hash=sha256:07de26b0d8cfc18f871aec595fda24d95b08fef89d147caa861939f37230bf4b \ --hash=sha256:71b952e5721688937fb02cf9d354dbcf0785066149d2855e44531ebdd2b65d73 # via pylint -typing-extensions==4.4.0 \ - --hash=sha256:1511434bb92bf8dd198c12b1cc812e800d4181cfcb867674e0f8279cc93087aa \ - --hash=sha256:16fa4864408f655d35ec496218b85f79b3437c829e93320c7c9215ccfd92489e +types-pyyaml==6.0.12.20240917 \ + --hash=sha256:392b267f1c0fe6022952462bf5d6523f31e37f6cea49b14cee7ad634b6301570 \ + --hash=sha256:d1405a86f9576682234ef83bcb4e6fff7c9305c8b1fbad5e0bcd4f7dbdc9c587 + # via django-stubs +types-tabulate==0.9.0.20240106 \ + --hash=sha256:0378b7b6fe0ccb4986299496d027a6d4c218298ecad67199bbd0e2d7e9d335a1 \ + --hash=sha256:c9b6db10dd7fcf55bd1712dd3537f86ddce72a08fd62bb1af4338c7096ce947e + # via -r requirements.in +typing-extensions==4.12.2 \ + --hash=sha256:04e5ca0351e0f3f85c6853954072df659d0d13fac324d0072316b67d7794700d \ + --hash=sha256:1a7ead55c7e559dd4dee8856e3a88b41225abfe1ce8df57b7c13915fe121ffb8 # via + # asgiref # astroid + # django-stubs + # django-stubs-ext # pylint +tzdata==2024.2 \ + --hash=sha256:7d85cc416e9382e69095b7bdf4afd9e3880418a2413feec7069d533d6b4e31cc \ + --hash=sha256:a48093786cdcde33cad18c2555e8532f34422074448fbc874186f0abd79565cd + # via django urllib3==1.26.13 \ --hash=sha256:47cc05d99aaa09c9e72ed5809b60e7ba354e64b59c9c173ac3018642d8bb41fc \ --hash=sha256:c083dd0dce68dbfbe1129d5271cb90f9447dea7d52097c6e0126120c521ddea8 @@ -162,23 +204,30 @@ wrapt==1.14.1 \ --hash=sha256:07f7a7d0f388028b2df1d916e94bbb40624c59b48ecc6cbc232546706fac74c2 \ --hash=sha256:11871514607b15cfeb87c547a49bca19fde402f32e2b1c24a632506c0a756656 \ --hash=sha256:1b376b3f4896e7930f1f772ac4b064ac12598d1c38d04907e696cc4d794b43d3 \ + --hash=sha256:2020f391008ef874c6d9e208b24f28e31bcb85ccff4f335f15a3251d222b92d9 \ --hash=sha256:21ac0156c4b089b330b7666db40feee30a5d52634cc4560e1905d6529a3897ff \ + --hash=sha256:240b1686f38ae665d1b15475966fe0472f78e71b1b4903c143a842659c8e4cb9 \ --hash=sha256:257fd78c513e0fb5cdbe058c27a0624c9884e735bbd131935fd49e9fe719d310 \ + --hash=sha256:26046cd03936ae745a502abf44dac702a5e6880b2b01c29aea8ddf3353b68224 \ --hash=sha256:2b39d38039a1fdad98c87279b48bc5dce2c0ca0d73483b12cb72aa9609278e8a \ --hash=sha256:2cf71233a0ed05ccdabe209c606fe0bac7379fdcf687f39b944420d2a09fdb57 \ --hash=sha256:2fe803deacd09a233e4762a1adcea5db5d31e6be577a43352936179d14d90069 \ + --hash=sha256:2feecf86e1f7a86517cab34ae6c2f081fd2d0dac860cb0c0ded96d799d20b335 \ --hash=sha256:3232822c7d98d23895ccc443bbdf57c7412c5a65996c30442ebe6ed3df335383 \ --hash=sha256:34aa51c45f28ba7f12accd624225e2b1e5a3a45206aa191f6f9aac931d9d56fe \ + --hash=sha256:358fe87cc899c6bb0ddc185bf3dbfa4ba646f05b1b0b9b5a27c2cb92c2cea204 \ --hash=sha256:36f582d0c6bc99d5f39cd3ac2a9062e57f3cf606ade29a0a0d6b323462f4dd87 \ --hash=sha256:380a85cf89e0e69b7cfbe2ea9f765f004ff419f34194018a6827ac0e3edfed4d \ --hash=sha256:40e7bc81c9e2b2734ea4bc1aceb8a8f0ceaac7c5299bc5d69e37c44d9081d43b \ --hash=sha256:43ca3bbbe97af00f49efb06e352eae40434ca9d915906f77def219b88e85d907 \ + --hash=sha256:49ef582b7a1152ae2766557f0550a9fcbf7bbd76f43fbdc94dd3bf07cc7168be \ --hash=sha256:4fcc4649dc762cddacd193e6b55bc02edca674067f5f98166d7713b193932b7f \ --hash=sha256:5a0f54ce2c092aaf439813735584b9537cad479575a09892b8352fea5e988dc0 \ --hash=sha256:5a9a0d155deafd9448baff28c08e150d9b24ff010e899311ddd63c45c2445e28 \ --hash=sha256:5b02d65b9ccf0ef6c34cba6cf5bf2aab1bb2f49c6090bafeecc9cd81ad4ea1c1 \ --hash=sha256:60db23fa423575eeb65ea430cee741acb7c26a1365d103f7b0f6ec412b893853 \ --hash=sha256:642c2e7a804fcf18c222e1060df25fc210b9c58db7c91416fb055897fc27e8cc \ + --hash=sha256:6447e9f3ba72f8e2b985a1da758767698efa72723d5b59accefd716e9e8272bf \ --hash=sha256:6a9a25751acb379b466ff6be78a315e2b439d4c94c1e99cb7266d40a537995d3 \ --hash=sha256:6b1a564e6cb69922c7fe3a678b9f9a3c54e72b469875aa8018f18b4d1dd1adf3 \ --hash=sha256:6d323e1554b3d22cfc03cd3243b5bb815a51f5249fdcbb86fda4bf62bab9e164 \ @@ -201,8 +250,10 @@ wrapt==1.14.1 \ --hash=sha256:9e0fd32e0148dd5dea6af5fee42beb949098564cc23211a88d799e434255a1f4 \ --hash=sha256:9f3e6f9e05148ff90002b884fbc2a86bd303ae847e472f44ecc06c2cd2fcdb2d \ --hash=sha256:a85d2b46be66a71bedde836d9e41859879cc54a2a04fad1191eb50c2066f6e9d \ + --hash=sha256:a9008dad07d71f68487c91e96579c8567c98ca4c3881b9b113bc7b33e9fd78b8 \ --hash=sha256:a9a52172be0b5aae932bef82a79ec0a0ce87288c7d132946d645eba03f0ad8a8 \ --hash=sha256:aa31fdcc33fef9eb2552cbcbfee7773d5a6792c137b359e82879c101e98584c5 \ + --hash=sha256:acae32e13a4153809db37405f5eba5bac5fbe2e2ba61ab227926a22901051c0a \ --hash=sha256:b014c23646a467558be7da3d6b9fa409b2c567d2110599b7cf9a0c5992b3b471 \ --hash=sha256:b21bb4c09ffabfa0e85e3a6b623e19b80e7acd709b9f91452b8297ace2a8ab00 \ --hash=sha256:b5901a312f4d14c59918c221323068fad0540e34324925c8475263841dbdfe68 \ @@ -217,6 +268,7 @@ wrapt==1.14.1 \ --hash=sha256:dee60e1de1898bde3b238f18340eec6148986da0455d8ba7848d50470a7a32fb \ --hash=sha256:e2f83e18fe2f4c9e7db597e988f72712c0c3676d337d8b101f6758107c42425b \ --hash=sha256:e3fb1677c720409d5f671e39bac6c9e0e422584e5f518bfd50aa4cbbea02433f \ + --hash=sha256:ecee4132c6cd2ce5308e21672015ddfed1ff975ad0ac8d27168ea82e71413f55 \ --hash=sha256:ee2b1b1769f6707a8a445162ea16dddf74285c3964f605877a20e38545c3c462 \ --hash=sha256:ee6acae74a2b91865910eef5e7de37dc6895ad96fa23603d1d27ea69df545015 \ --hash=sha256:ef3f72c9666bba2bab70d2a8b79f2c6d2c1a42a7f7e2b0ec83bb2f9e383950af diff --git a/gazelle/MODULE.bazel b/gazelle/MODULE.bazel index d216ad5dc1..0a553831c3 100644 --- a/gazelle/MODULE.bazel +++ b/gazelle/MODULE.bazel @@ -34,3 +34,14 @@ use_repo( python_stdlib_list, "python_stdlib_list", ) + +internal_dev_deps = use_extension( + "//:internal_dev_deps.bzl", + "internal_dev_deps_extension", + dev_dependency = True, +) +use_repo( + internal_dev_deps, + "django-types", + "pytest", +) diff --git a/gazelle/README.md b/gazelle/README.md index c0494d141b..55c9cc9bff 100644 --- a/gazelle/README.md +++ b/gazelle/README.md @@ -119,6 +119,16 @@ gazelle_python_manifest( # the integrity field is not added to the manifest which can help avoid # merge conflicts in large repos. requirements = "//:requirements_lock.txt", + # include_stub_packages: bool (default: False) + # If set to True, this flag automatically includes any corresponding type stub packages + # for the third-party libraries that are present and used. For example, if you have + # `boto3` as a dependency, and this flag is enabled, the corresponding `boto3-stubs` + # package will be automatically included in the BUILD file. + # + # Enabling this feature helps ensure that type hints and stubs are readily available + # for tools like type checkers and IDEs, improving the development experience and + # reducing manual overhead in managing separate stub packages. + include_stub_packages = True ) ``` diff --git a/gazelle/WORKSPACE b/gazelle/WORKSPACE index d9f0645071..14a124d5f2 100644 --- a/gazelle/WORKSPACE +++ b/gazelle/WORKSPACE @@ -38,6 +38,10 @@ load("@rules_python//python:repositories.bzl", "py_repositories") py_repositories() +load("//:internal_dev_deps.bzl", "internal_dev_deps") + +internal_dev_deps() + load("//:deps.bzl", _py_gazelle_deps = "gazelle_deps") # gazelle:repository_macro deps.bzl%go_deps diff --git a/gazelle/internal_dev_deps.bzl b/gazelle/internal_dev_deps.bzl new file mode 100644 index 0000000000..f05f5fbb88 --- /dev/null +++ b/gazelle/internal_dev_deps.bzl @@ -0,0 +1,47 @@ +# Copyright 2024 The Bazel Authors. All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +"""Module extension for internal dev_dependency=True setup.""" + +load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_file") + +def internal_dev_deps(): + """This extension creates internal rules_python_gazelle dev dependencies.""" + http_file( + name = "pytest", + downloaded_file_path = "pytest-8.3.3-py3-none-any.whl", + sha256 = "a6853c7375b2663155079443d2e45de913a911a11d669df02a50814944db57b2", + urls = [ + "https://files.pythonhosted.org/packages/6b/77/7440a06a8ead44c7757a64362dd22df5760f9b12dc5f11b6188cd2fc27a0/pytest-8.3.3-py3-none-any.whl", + ], + ) + http_file( + name = "django-types", + downloaded_file_path = "django_types-0.19.1-py3-none-any.whl", + sha256 = "b3f529de17f6374d41ca67232aa01330c531bbbaa3ac4097896f31ac33c96c30", + urls = [ + "https://files.pythonhosted.org/packages/25/cb/d088c67245a9d5759a08dbafb47e040ee436e06ee433a3cdc7f3233b3313/django_types-0.19.1-py3-none-any.whl", + ], + ) + +def _internal_dev_deps_impl(mctx): + _ = mctx # @unused + + # This wheel is purely here to validate the wheel extraction code. It's not + # intended for anything else. + internal_dev_deps() + +internal_dev_deps_extension = module_extension( + implementation = _internal_dev_deps_impl, + doc = "This extension creates internal rules_python_gazelle dev dependencies.", +) diff --git a/gazelle/modules_mapping/BUILD.bazel b/gazelle/modules_mapping/BUILD.bazel index d78b1fb51f..3a9a8a47f3 100644 --- a/gazelle/modules_mapping/BUILD.bazel +++ b/gazelle/modules_mapping/BUILD.bazel @@ -1,4 +1,5 @@ -load("@rules_python//python:defs.bzl", "py_binary") +load("@bazel_skylib//rules:copy_file.bzl", "copy_file") +load("@rules_python//python:defs.bzl", "py_binary", "py_test") # gazelle:exclude *.py @@ -8,6 +9,30 @@ py_binary( visibility = ["//visibility:public"], ) +copy_file( + name = "pytest_wheel", + src = "@pytest//file", + out = "pytest-8.3.3-py3-none-any.whl", +) + +copy_file( + name = "django_types_wheel", + src = "@django-types//file", + out = "django_types-0.19.1-py3-none-any.whl", +) + +py_test( + name = "test_generator", + srcs = ["test_generator.py"], + data = [ + "django_types_wheel", + "pytest_wheel", + ], + imports = ["."], + main = "test_generator.py", + deps = [":generator"], +) + filegroup( name = "distribution", srcs = glob(["**"]), diff --git a/gazelle/modules_mapping/def.bzl b/gazelle/modules_mapping/def.bzl index 4da6267493..eb17f5c3d4 100644 --- a/gazelle/modules_mapping/def.bzl +++ b/gazelle/modules_mapping/def.bzl @@ -31,6 +31,8 @@ def _modules_mapping_impl(ctx): transitive = [dep[DefaultInfo].files for dep in ctx.attr.wheels] + [dep[DefaultInfo].data_runfiles.files for dep in ctx.attr.wheels], ) args.add("--output_file", modules_mapping.path) + if ctx.attr.include_stub_packages: + args.add("--include_stub_packages") args.add_all("--exclude_patterns", ctx.attr.exclude_patterns) args.add_all("--wheels", [whl.path for whl in all_wheels.to_list()]) ctx.actions.run( @@ -50,6 +52,11 @@ modules_mapping = rule( doc = "A set of regex patterns to match against each calculated module path. By default, exclude the modules starting with underscores.", mandatory = False, ), + "include_stub_packages": attr.bool( + default = False, + doc = "Whether to include stub packages in the mapping.", + mandatory = False, + ), "modules_mapping_name": attr.string( default = "modules_mapping.json", doc = "The name for the output JSON file.", diff --git a/gazelle/modules_mapping/generator.py b/gazelle/modules_mapping/generator.py index bbd579d416..99f565e8d6 100644 --- a/gazelle/modules_mapping/generator.py +++ b/gazelle/modules_mapping/generator.py @@ -25,16 +25,25 @@ class Generator: stderr = None output_file = None excluded_patterns = None - mapping = {} - def __init__(self, stderr, output_file, excluded_patterns): + def __init__(self, stderr, output_file, excluded_patterns, include_stub_packages): self.stderr = stderr self.output_file = output_file self.excluded_patterns = [re.compile(pattern) for pattern in excluded_patterns] + self.include_stub_packages = include_stub_packages + self.mapping = {} # dig_wheel analyses the wheel .whl file determining the modules it provides # by looking at the directory structure. def dig_wheel(self, whl): + # Skip stubs and types wheels. + wheel_name = get_wheel_name(whl) + if self.include_stub_packages and ( + wheel_name.endswith(("_stubs", "_types")) + or wheel_name.startswith(("types_", "stubs_")) + ): + self.mapping[wheel_name.lower()] = wheel_name.lower() + return with zipfile.ZipFile(whl, "r") as zip_file: for path in zip_file.namelist(): if is_metadata(path): @@ -145,8 +154,11 @@ def data_has_purelib_or_platlib(path): description="Generates the modules mapping used by the Gazelle manifest.", ) parser.add_argument("--output_file", type=str) + parser.add_argument("--include_stub_packages", action="store_true") parser.add_argument("--exclude_patterns", nargs="+", default=[]) parser.add_argument("--wheels", nargs="+", default=[]) args = parser.parse_args() - generator = Generator(sys.stderr, args.output_file, args.exclude_patterns) + generator = Generator( + sys.stderr, args.output_file, args.exclude_patterns, args.include_stub_packages + ) exit(generator.run(args.wheels)) diff --git a/gazelle/modules_mapping/test_generator.py b/gazelle/modules_mapping/test_generator.py new file mode 100644 index 0000000000..d6d2f19039 --- /dev/null +++ b/gazelle/modules_mapping/test_generator.py @@ -0,0 +1,44 @@ +import pathlib +import unittest + +from generator import Generator + + +class GeneratorTest(unittest.TestCase): + def test_generator(self): + whl = pathlib.Path(__file__).parent / "pytest-8.3.3-py3-none-any.whl" + gen = Generator(None, None, {}, False) + gen.dig_wheel(whl) + self.assertLessEqual( + { + "_pytest": "pytest", + "_pytest.__init__": "pytest", + "_pytest._argcomplete": "pytest", + "_pytest.config.argparsing": "pytest", + }.items(), + gen.mapping.items(), + ) + + def test_stub_generator(self): + whl = pathlib.Path(__file__).parent / "django_types-0.19.1-py3-none-any.whl" + gen = Generator(None, None, {}, True) + gen.dig_wheel(whl) + self.assertLessEqual( + { + "django_types": "django_types", + }.items(), + gen.mapping.items(), + ) + + def test_stub_excluded(self): + whl = pathlib.Path(__file__).parent / "django_types-0.19.1-py3-none-any.whl" + gen = Generator(None, None, {}, False) + gen.dig_wheel(whl) + self.assertEqual( + {}.items(), + gen.mapping.items(), + ) + + +if __name__ == "__main__": + unittest.main() diff --git a/gazelle/python/resolve.go b/gazelle/python/resolve.go index a7b716a829..88a688fa85 100644 --- a/gazelle/python/resolve.go +++ b/gazelle/python/resolve.go @@ -189,8 +189,20 @@ func (py *Resolver) Resolve( continue MODULES_LOOP } } else { - if dep, ok := cfg.FindThirdPartyDependency(moduleName); ok { + if dep, distributionName, ok := cfg.FindThirdPartyDependency(moduleName); ok { deps.Add(dep) + // Add the type and stub dependencies if they exist. + modules := []string{ + fmt.Sprintf("%s_stubs", strings.ToLower(distributionName)), + fmt.Sprintf("%s_types", strings.ToLower(distributionName)), + fmt.Sprintf("types_%s", strings.ToLower(distributionName)), + fmt.Sprintf("stubs_%s", strings.ToLower(distributionName)), + } + for _, module := range modules { + if dep, _, ok := cfg.FindThirdPartyDependency(module); ok { + deps.Add(dep) + } + } if explainDependency == dep { log.Printf("Explaining dependency (%s): "+ "in the target %q, the file %q imports %q at line %d, "+ diff --git a/gazelle/python/testdata/add_type_stub_packages/BUILD.in b/gazelle/python/testdata/add_type_stub_packages/BUILD.in new file mode 100644 index 0000000000..e69de29bb2 diff --git a/gazelle/python/testdata/add_type_stub_packages/BUILD.out b/gazelle/python/testdata/add_type_stub_packages/BUILD.out new file mode 100644 index 0000000000..d30540f61a --- /dev/null +++ b/gazelle/python/testdata/add_type_stub_packages/BUILD.out @@ -0,0 +1,14 @@ +load("@rules_python//python:defs.bzl", "py_binary") + +py_binary( + name = "add_type_stub_packages_bin", + srcs = ["__main__.py"], + main = "__main__.py", + visibility = ["//:__subpackages__"], + deps = [ + "@gazelle_python_test//boto3", + "@gazelle_python_test//boto3_stubs", + "@gazelle_python_test//django", + "@gazelle_python_test//django_types", + ], +) diff --git a/gazelle/python/testdata/add_type_stub_packages/README.md b/gazelle/python/testdata/add_type_stub_packages/README.md new file mode 100644 index 0000000000..c42e76f8be --- /dev/null +++ b/gazelle/python/testdata/add_type_stub_packages/README.md @@ -0,0 +1,4 @@ +# Add stubs to `deps` of `py_library` target + +This test case asserts that +* if a package has the corresponding stub available, it is added to the `deps` of the `py_library` target. diff --git a/gazelle/python/testdata/add_type_stub_packages/WORKSPACE b/gazelle/python/testdata/add_type_stub_packages/WORKSPACE new file mode 100644 index 0000000000..faff6af87a --- /dev/null +++ b/gazelle/python/testdata/add_type_stub_packages/WORKSPACE @@ -0,0 +1 @@ +# This is a Bazel workspace for the Gazelle test data. diff --git a/gazelle/python/testdata/add_type_stub_packages/__main__.py b/gazelle/python/testdata/add_type_stub_packages/__main__.py new file mode 100644 index 0000000000..96384cfb13 --- /dev/null +++ b/gazelle/python/testdata/add_type_stub_packages/__main__.py @@ -0,0 +1,16 @@ +# Copyright 2023 The Bazel Authors. All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +import boto3 +import django diff --git a/gazelle/python/testdata/add_type_stub_packages/gazelle_python.yaml b/gazelle/python/testdata/add_type_stub_packages/gazelle_python.yaml new file mode 100644 index 0000000000..f498d07f2f --- /dev/null +++ b/gazelle/python/testdata/add_type_stub_packages/gazelle_python.yaml @@ -0,0 +1,22 @@ +# Copyright 2023 The Bazel Authors. All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +manifest: + modules_mapping: + boto3: boto3 + boto3_stubs: boto3_stubs + django_types: django_types + django: Django + + pip_deps_repository_name: gazelle_python_test diff --git a/gazelle/python/testdata/add_type_stub_packages/test.yaml b/gazelle/python/testdata/add_type_stub_packages/test.yaml new file mode 100644 index 0000000000..fcea77710f --- /dev/null +++ b/gazelle/python/testdata/add_type_stub_packages/test.yaml @@ -0,0 +1,15 @@ +# Copyright 2023 The Bazel Authors. All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +--- diff --git a/gazelle/pythonconfig/pythonconfig.go b/gazelle/pythonconfig/pythonconfig.go index a24a90efeb..55121381dd 100644 --- a/gazelle/pythonconfig/pythonconfig.go +++ b/gazelle/pythonconfig/pythonconfig.go @@ -278,7 +278,7 @@ func (c *Config) SetGazelleManifest(gazelleManifest *manifest.Manifest) { // FindThirdPartyDependency scans the gazelle manifests for the current config // and the parent configs up to the root finding if it can resolve the module // name. -func (c *Config) FindThirdPartyDependency(modName string) (string, bool) { +func (c *Config) FindThirdPartyDependency(modName string) (string, string, bool) { for currentCfg := c; currentCfg != nil; currentCfg = currentCfg.parent { if currentCfg.gazelleManifest != nil { gazelleManifest := currentCfg.gazelleManifest @@ -291,11 +291,11 @@ func (c *Config) FindThirdPartyDependency(modName string) (string, bool) { } lbl := currentCfg.FormatThirdPartyDependency(distributionRepositoryName, distributionName) - return lbl.String(), true + return lbl.String(), distributionName, true } } } - return "", false + return "", "", false } // AddIgnoreFile adds a file to the list of ignored files for a given package. From fac693f2a4952049f3b69881a31bd7f369ee4821 Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Sun, 8 Dec 2024 00:15:39 -0800 Subject: [PATCH 019/922] build(deps): bump django from 4.2.16 to 4.2.17 in /examples/bzlmod_build_file_generation (#2486) Bumps [django](https://github.com/django/django) from 4.2.16 to 4.2.17.
Commits
  • 1f0356f [4.2.x] Bumped version for 4.2.17 release.
  • 7376bcb [4.2.x] Fixed CVE-2024-53908 -- Prevented SQL injections in direct HasKeyLook...
  • 790eb05 [4.2.x] Fixed CVE-2024-53907 -- Mitigated potential DoS in strip_tags().
  • f663277 [4.2.x] Refs CVE-2024-11168 -- Updated vendored _urlsplit() to properly valid...
  • 0acff0f [4.2.x] Added stub release notes and release date for 4.2.17.
  • b381b19 [4.2.x] Fixed docs build on Sphinx 8.1+.
  • ea4a1fb [4.2.x] Refs #35844 -- Expanded compatibility for expected error messages in ...
  • 345a665 [4.2.x] Added GitHub Action workflow to test all Python versions listed in th...
  • 5211677 [4.2.x] Added CVE-2024-45230 and CVE-2024-45231 to security archive.
  • 8f6c362 [4.2.x] Post-release version bump.
  • See full diff in compare view

[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=django&package-manager=pip&previous-version=4.2.16&new-version=4.2.17)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) ---
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot show ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) You can disable automated security fix PRs for this repo from the [Security Alerts page](https://github.com/bazelbuild/rules_python/network/alerts).
Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> --- examples/bzlmod_build_file_generation/requirements_lock.txt | 6 +++--- .../bzlmod_build_file_generation/requirements_windows.txt | 6 +++--- 2 files changed, 6 insertions(+), 6 deletions(-) diff --git a/examples/bzlmod_build_file_generation/requirements_lock.txt b/examples/bzlmod_build_file_generation/requirements_lock.txt index cdcebb72f7..7bf1e2200f 100644 --- a/examples/bzlmod_build_file_generation/requirements_lock.txt +++ b/examples/bzlmod_build_file_generation/requirements_lock.txt @@ -26,9 +26,9 @@ dill==0.3.6 \ --hash=sha256:a07ffd2351b8c678dfc4a856a3005f8067aea51d6ba6c700796a4d9e280f39f0 \ --hash=sha256:e5db55f3687856d8fbdab002ed78544e1c4559a130302693d839dfe8f93f2373 # via pylint -django==4.2.16 \ - --hash=sha256:1ddc333a16fc139fd253035a1606bb24261951bbc3a6ca256717fa06cc41a898 \ - --hash=sha256:6f1616c2786c408ce86ab7e10f792b8f15742f7b7b7460243929cb371e7f1dad +django==4.2.17 \ + --hash=sha256:3a93350214ba25f178d4045c0786c61573e7dbfa3c509b3551374f1e11ba8de0 \ + --hash=sha256:6b56d834cc94c8b21a8f4e775064896be3b4a4ca387f2612d4406a5927cd2fdc # via # -r requirements.in # django-stubs diff --git a/examples/bzlmod_build_file_generation/requirements_windows.txt b/examples/bzlmod_build_file_generation/requirements_windows.txt index e591c8dc80..8a796a3718 100644 --- a/examples/bzlmod_build_file_generation/requirements_windows.txt +++ b/examples/bzlmod_build_file_generation/requirements_windows.txt @@ -30,9 +30,9 @@ dill==0.3.6 \ --hash=sha256:a07ffd2351b8c678dfc4a856a3005f8067aea51d6ba6c700796a4d9e280f39f0 \ --hash=sha256:e5db55f3687856d8fbdab002ed78544e1c4559a130302693d839dfe8f93f2373 # via pylint -django==4.2.16 \ - --hash=sha256:1ddc333a16fc139fd253035a1606bb24261951bbc3a6ca256717fa06cc41a898 \ - --hash=sha256:6f1616c2786c408ce86ab7e10f792b8f15742f7b7b7460243929cb371e7f1dad +django==4.2.17 \ + --hash=sha256:3a93350214ba25f178d4045c0786c61573e7dbfa3c509b3551374f1e11ba8de0 \ + --hash=sha256:6b56d834cc94c8b21a8f4e775064896be3b4a4ca387f2612d4406a5927cd2fdc # via # -r requirements.in # django-stubs From 7fd51912fa042993de34b126b1774b45bbc6c5a0 Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Mon, 9 Dec 2024 11:12:20 +0900 Subject: [PATCH 020/922] build(deps): bump rich from 13.9.3 to 13.9.4 in /tools/publish (#2371) Bumps [rich](https://github.com/Textualize/rich) from 13.9.3 to 13.9.4.
Release notes

Sourced from rich's releases.

The Faster is Faster release

[13.9.4] - 2024-11-01

Changed

Changelog

Sourced from rich's changelog.

[13.9.4] - 2024-11-01

Changed

Commits

[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=rich&package-manager=pip&previous-version=13.9.3&new-version=13.9.4)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) ---
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot show ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> --- tools/publish/requirements_darwin.txt | 6 +++--- tools/publish/requirements_linux.txt | 6 +++--- tools/publish/requirements_universal.txt | 6 +++--- tools/publish/requirements_windows.txt | 6 +++--- 4 files changed, 12 insertions(+), 12 deletions(-) diff --git a/tools/publish/requirements_darwin.txt b/tools/publish/requirements_darwin.txt index 2f8088e358..31c0a0402f 100644 --- a/tools/publish/requirements_darwin.txt +++ b/tools/publish/requirements_darwin.txt @@ -207,9 +207,9 @@ rfc3986==2.0.0 \ --hash=sha256:50b1502b60e289cb37883f3dfd34532b8873c7de9f49bb546641ce9cbd256ebd \ --hash=sha256:97aacf9dbd4bfd829baad6e6309fa6573aaf1be3f6fa735c8ab05e46cecb261c # via twine -rich==13.9.3 \ - --hash=sha256:9836f5096eb2172c9e77df411c1b009bace4193d6a481d534fea75ebba758283 \ - --hash=sha256:bc1e01b899537598cf02579d2b9f4a415104d3fc439313a7a2c165d76557a08e +rich==13.9.4 \ + --hash=sha256:439594978a49a09530cff7ebc4b5c7103ef57baf48d5ea3184f21d9a2befa098 \ + --hash=sha256:6049d5e6ec054bf2779ab3358186963bac2ea89175919d699e378b99738c2a90 # via twine twine==5.1.1 \ --hash=sha256:215dbe7b4b94c2c50a7315c0275d2258399280fbb7d04182c7e55e24b5f93997 \ diff --git a/tools/publish/requirements_linux.txt b/tools/publish/requirements_linux.txt index 785af7f9af..31ced6af74 100644 --- a/tools/publish/requirements_linux.txt +++ b/tools/publish/requirements_linux.txt @@ -315,9 +315,9 @@ rfc3986==2.0.0 \ --hash=sha256:50b1502b60e289cb37883f3dfd34532b8873c7de9f49bb546641ce9cbd256ebd \ --hash=sha256:97aacf9dbd4bfd829baad6e6309fa6573aaf1be3f6fa735c8ab05e46cecb261c # via twine -rich==13.9.3 \ - --hash=sha256:9836f5096eb2172c9e77df411c1b009bace4193d6a481d534fea75ebba758283 \ - --hash=sha256:bc1e01b899537598cf02579d2b9f4a415104d3fc439313a7a2c165d76557a08e +rich==13.9.4 \ + --hash=sha256:439594978a49a09530cff7ebc4b5c7103ef57baf48d5ea3184f21d9a2befa098 \ + --hash=sha256:6049d5e6ec054bf2779ab3358186963bac2ea89175919d699e378b99738c2a90 # via twine secretstorage==3.3.3 \ --hash=sha256:2403533ef369eca6d2ba81718576c5e0f564d5cca1b58f73a8b23e7d4eeebd77 \ diff --git a/tools/publish/requirements_universal.txt b/tools/publish/requirements_universal.txt index 06f93286f5..6e2502835e 100644 --- a/tools/publish/requirements_universal.txt +++ b/tools/publish/requirements_universal.txt @@ -319,9 +319,9 @@ rfc3986==2.0.0 \ --hash=sha256:50b1502b60e289cb37883f3dfd34532b8873c7de9f49bb546641ce9cbd256ebd \ --hash=sha256:97aacf9dbd4bfd829baad6e6309fa6573aaf1be3f6fa735c8ab05e46cecb261c # via twine -rich==13.9.3 \ - --hash=sha256:9836f5096eb2172c9e77df411c1b009bace4193d6a481d534fea75ebba758283 \ - --hash=sha256:bc1e01b899537598cf02579d2b9f4a415104d3fc439313a7a2c165d76557a08e +rich==13.9.4 \ + --hash=sha256:439594978a49a09530cff7ebc4b5c7103ef57baf48d5ea3184f21d9a2befa098 \ + --hash=sha256:6049d5e6ec054bf2779ab3358186963bac2ea89175919d699e378b99738c2a90 # via twine secretstorage==3.3.3 ; sys_platform == 'linux' \ --hash=sha256:2403533ef369eca6d2ba81718576c5e0f564d5cca1b58f73a8b23e7d4eeebd77 \ diff --git a/tools/publish/requirements_windows.txt b/tools/publish/requirements_windows.txt index 23d298643f..3733696678 100644 --- a/tools/publish/requirements_windows.txt +++ b/tools/publish/requirements_windows.txt @@ -211,9 +211,9 @@ rfc3986==2.0.0 \ --hash=sha256:50b1502b60e289cb37883f3dfd34532b8873c7de9f49bb546641ce9cbd256ebd \ --hash=sha256:97aacf9dbd4bfd829baad6e6309fa6573aaf1be3f6fa735c8ab05e46cecb261c # via twine -rich==13.9.3 \ - --hash=sha256:9836f5096eb2172c9e77df411c1b009bace4193d6a481d534fea75ebba758283 \ - --hash=sha256:bc1e01b899537598cf02579d2b9f4a415104d3fc439313a7a2c165d76557a08e +rich==13.9.4 \ + --hash=sha256:439594978a49a09530cff7ebc4b5c7103ef57baf48d5ea3184f21d9a2befa098 \ + --hash=sha256:6049d5e6ec054bf2779ab3358186963bac2ea89175919d699e378b99738c2a90 # via twine twine==5.1.1 \ --hash=sha256:215dbe7b4b94c2c50a7315c0275d2258399280fbb7d04182c7e55e24b5f93997 \ From b11f07783d88c28d72dbfcc15161cf1aed239aef Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Sun, 8 Dec 2024 18:50:37 -0800 Subject: [PATCH 021/922] fix: define rules_python_internal earlier so Bazel 9 doesn't try to use PyInfo et al builtins (#2485) For Bazel 9 workspace builds, if `@rules_python_internal` isn't defined early enough, an earlier version of `@rules_python` gets defined and the logic to not use the builtin PyInfo et al symbols doesn't occur. Since Bazel 9 doesn't have these builtins, an error occurs. This seems to only happen if the main module is rules_python. The example workspaces don't see to have an issue. I'm not sure why, but it seems similar to the behavior where autoloading is disabled for specific repos, rules_python among them. To fix, move the `@rules_python_internal` repo definition to be earlier in the WORKSPACE processing. With that repo defined, the conditional logic takes place, and things seem to be happy. --- internal_dev_deps.bzl | 2 ++ internal_dev_setup.bzl | 2 -- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/internal_dev_deps.bzl b/internal_dev_deps.bzl index 80196815a0..e76ee48e83 100644 --- a/internal_dev_deps.bzl +++ b/internal_dev_deps.bzl @@ -16,6 +16,7 @@ load("@bazel_tools//tools/build_defs/repo:http.bzl", _http_archive = "http_archive", _http_file = "http_file") load("@bazel_tools//tools/build_defs/repo:utils.bzl", "maybe") +load("//python/private:internal_config_repo.bzl", "internal_config_repo") # buildifier: disable=bzl-visibility def http_archive(name, **kwargs): maybe( @@ -39,6 +40,7 @@ def rules_python_internal_deps(): For dependencies needed by *users* of rules_python, see python/private/py_repositories.bzl. """ + internal_config_repo(name = "rules_python_internal") http_archive( name = "bazel_skylib", diff --git a/internal_dev_setup.bzl b/internal_dev_setup.bzl index 554ff926f2..26edcb9abb 100644 --- a/internal_dev_setup.bzl +++ b/internal_dev_setup.bzl @@ -25,14 +25,12 @@ load("@rules_proto//proto:repositories.bzl", "rules_proto_dependencies", "rules_ load("@rules_shell//shell:repositories.bzl", "rules_shell_dependencies", "rules_shell_toolchains") load("//:version.bzl", "SUPPORTED_BAZEL_VERSIONS") load("//python:versions.bzl", "MINOR_MAPPING", "TOOL_VERSIONS") -load("//python/private:internal_config_repo.bzl", "internal_config_repo") # buildifier: disable=bzl-visibility load("//python/private:pythons_hub.bzl", "hub_repo") # buildifier: disable=bzl-visibility load("//python/private/pypi:deps.bzl", "pypi_deps") # buildifier: disable=bzl-visibility def rules_python_internal_setup(): """Setup for development and testing of rules_python itself.""" - internal_config_repo(name = "rules_python_internal") hub_repo( name = "pythons_hub", minor_mapping = MINOR_MAPPING, From a7b7126ab6f4c7bc9755aa7224a4a90bd4598457 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Sun, 8 Dec 2024 20:46:56 -0800 Subject: [PATCH 022/922] chore: replace rules_proto with com_google_protobuf (#2487) rules_proto is deprecated and recent versions simply forward onto com_google_protobuf. Older versions (e.g. 6.x used by us today), however, use the rules_proto or native (Bazel builtin) implementation. When those older versions are used with Bazel 9, which has removed various proto things, errors occur. To fix, switch to using com_google_protobuf directly. More recent versions of rules_proto just forward onto com_google_protobuf anyways, so this just removes the extra dependency and having to deal with WORKSPACE setup. Work towards https://github.com/bazelbuild/rules_python/issues/2469 --- WORKSPACE | 4 ++++ .../example.com/another_proto/BUILD.bazel | 2 +- .../py_proto_library/example.com/proto/BUILD.bazel | 2 +- examples/py_proto_library/WORKSPACE | 13 ------------- .../example.com/another_proto/BUILD.bazel | 2 +- .../py_proto_library/example.com/proto/BUILD.bazel | 2 +- internal_dev_setup.bzl | 4 ---- python/private/proto/BUILD.bazel | 4 +++- python/private/proto/py_proto_library.bzl | 3 ++- 9 files changed, 13 insertions(+), 23 deletions(-) diff --git a/WORKSPACE b/WORKSPACE index f03cc26803..c0d9f33a9b 100644 --- a/WORKSPACE +++ b/WORKSPACE @@ -21,6 +21,10 @@ load("//:internal_dev_deps.bzl", "rules_python_internal_deps") rules_python_internal_deps() +load("@com_google_protobuf//:protobuf_deps.bzl", "protobuf_deps") + +protobuf_deps() + load("@rules_jvm_external//:repositories.bzl", "rules_jvm_external_deps") rules_jvm_external_deps() diff --git a/examples/bzlmod/py_proto_library/example.com/another_proto/BUILD.bazel b/examples/bzlmod/py_proto_library/example.com/another_proto/BUILD.bazel index 806fcb9dcc..785d90d01e 100644 --- a/examples/bzlmod/py_proto_library/example.com/another_proto/BUILD.bazel +++ b/examples/bzlmod/py_proto_library/example.com/another_proto/BUILD.bazel @@ -1,4 +1,4 @@ -load("@rules_proto//proto:defs.bzl", "proto_library") +load("@com_google_protobuf//bazel:proto_library.bzl", "proto_library") load("@rules_python//python:proto.bzl", "py_proto_library") py_proto_library( diff --git a/examples/bzlmod/py_proto_library/example.com/proto/BUILD.bazel b/examples/bzlmod/py_proto_library/example.com/proto/BUILD.bazel index fa20f2ce94..72af672219 100644 --- a/examples/bzlmod/py_proto_library/example.com/proto/BUILD.bazel +++ b/examples/bzlmod/py_proto_library/example.com/proto/BUILD.bazel @@ -1,4 +1,4 @@ -load("@rules_proto//proto:defs.bzl", "proto_library") +load("@com_google_protobuf//bazel:proto_library.bzl", "proto_library") load("@rules_python//python:proto.bzl", "py_proto_library") py_proto_library( diff --git a/examples/py_proto_library/WORKSPACE b/examples/py_proto_library/WORKSPACE index 81f189dbbf..9cda5b97f1 100644 --- a/examples/py_proto_library/WORKSPACE +++ b/examples/py_proto_library/WORKSPACE @@ -24,19 +24,6 @@ python_register_toolchains( # Then we need to setup dependencies in order to use py_proto_library load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive") -http_archive( - name = "rules_proto", - sha256 = "904a8097fae42a690c8e08d805210e40cccb069f5f9a0f6727cf4faa7bed2c9c", - strip_prefix = "rules_proto-6.0.0-rc1", - url = "https://github.com/bazelbuild/rules_proto/releases/download/6.0.0-rc1/rules_proto-6.0.0-rc1.tar.gz", -) - -load("@rules_proto//proto:repositories.bzl", "rules_proto_dependencies", "rules_proto_toolchains") - -rules_proto_dependencies() - -rules_proto_toolchains() - http_archive( name = "com_google_protobuf", sha256 = "4fc5ff1b2c339fb86cd3a25f0b5311478ab081e65ad258c6789359cd84d421f8", diff --git a/examples/py_proto_library/example.com/another_proto/BUILD.bazel b/examples/py_proto_library/example.com/another_proto/BUILD.bazel index dd58265bc9..3d841554e9 100644 --- a/examples/py_proto_library/example.com/another_proto/BUILD.bazel +++ b/examples/py_proto_library/example.com/another_proto/BUILD.bazel @@ -1,4 +1,4 @@ -load("@rules_proto//proto:defs.bzl", "proto_library") +load("@com_google_protobuf//bazel:proto_library.bzl", "proto_library") load("@rules_python//python:proto.bzl", "py_proto_library") py_proto_library( diff --git a/examples/py_proto_library/example.com/proto/BUILD.bazel b/examples/py_proto_library/example.com/proto/BUILD.bazel index dc91162aa6..f84454f531 100644 --- a/examples/py_proto_library/example.com/proto/BUILD.bazel +++ b/examples/py_proto_library/example.com/proto/BUILD.bazel @@ -1,4 +1,4 @@ -load("@rules_proto//proto:defs.bzl", "proto_library") +load("@com_google_protobuf//bazel:proto_library.bzl", "proto_library") load("@rules_python//python:proto.bzl", "py_proto_library") py_proto_library( diff --git a/internal_dev_setup.bzl b/internal_dev_setup.bzl index 26edcb9abb..d6e95e22ce 100644 --- a/internal_dev_setup.bzl +++ b/internal_dev_setup.bzl @@ -21,7 +21,6 @@ load("@com_google_protobuf//:protobuf_deps.bzl", "protobuf_deps") load("@rules_bazel_integration_test//bazel_integration_test:deps.bzl", "bazel_integration_test_rules_dependencies") load("@rules_bazel_integration_test//bazel_integration_test:repo_defs.bzl", "bazel_binaries") load("@rules_java//java:repositories.bzl", "rules_java_dependencies", "rules_java_toolchains") -load("@rules_proto//proto:repositories.bzl", "rules_proto_dependencies", "rules_proto_toolchains") load("@rules_shell//shell:repositories.bzl", "rules_shell_dependencies", "rules_shell_toolchains") load("//:version.bzl", "SUPPORTED_BAZEL_VERSIONS") load("//python:versions.bzl", "MINOR_MAPPING", "TOOL_VERSIONS") @@ -46,9 +45,6 @@ def rules_python_internal_setup(): bazel_skylib_workspace() - rules_proto_dependencies() - rules_proto_toolchains() - protobuf_deps() rules_java_dependencies() diff --git a/python/private/proto/BUILD.bazel b/python/private/proto/BUILD.bazel index 222be40d09..dd53845638 100644 --- a/python/private/proto/BUILD.bazel +++ b/python/private/proto/BUILD.bazel @@ -13,7 +13,7 @@ # limitations under the License. load("@bazel_skylib//:bzl_library.bzl", "bzl_library") -load("@rules_proto//proto:defs.bzl", "proto_lang_toolchain") +load("@com_google_protobuf//bazel/toolchains:proto_lang_toolchain.bzl", "proto_lang_toolchain") package(default_visibility = ["//visibility:private"]) @@ -31,6 +31,8 @@ bzl_library( visibility = ["//python:__pkg__"], deps = [ "//python:py_info_bzl", + "@com_google_protobuf//bazel/common:proto_common_bzl", + "@com_google_protobuf//bazel/common:proto_info_bzl", "@rules_proto//proto:defs", ], ) diff --git a/python/private/proto/py_proto_library.bzl b/python/private/proto/py_proto_library.bzl index ff2d3d2bb3..d810e58c24 100644 --- a/python/private/proto/py_proto_library.bzl +++ b/python/private/proto/py_proto_library.bzl @@ -14,7 +14,8 @@ """The implementation of the `py_proto_library` rule and its aspect.""" -load("@rules_proto//proto:defs.bzl", "ProtoInfo", "proto_common") +load("@com_google_protobuf//bazel/common:proto_common.bzl", "proto_common") +load("@com_google_protobuf//bazel/common:proto_info.bzl", "ProtoInfo") load("//python:py_info.bzl", "PyInfo") load("//python/api:api.bzl", _py_common = "py_common") From 622989e30568e960e958d5d9a3cf7e62cda5ffb4 Mon Sep 17 00:00:00 2001 From: Ignas Anikevicius <240938+aignas@users.noreply.github.com> Date: Thu, 12 Dec 2024 01:35:21 +0900 Subject: [PATCH 023/922] doc: freethreaded support changelog (#2497) Support is implemented, just documenting this is left. Fixes #2386 --- CHANGELOG.md | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/CHANGELOG.md b/CHANGELOG.md index 10a2be4edf..bcf9f52324 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -65,6 +65,11 @@ Unreleased changes template. automatically includes corresponding stub packages for third-party libraries that are present and used (e.g., `boto3` → `boto3-stubs`), improving type-checking support. +* (pypi) Freethreaded packages are now fully supported in the + {obj}`experimental_index_url` usage or the regular `pip.parse` usage. + To select the free-threaded interpreter in the repo phase, please use + the documented [env](/environment-variables.html) variables. + Fixes [#2386](https://github.com/bazelbuild/rules_python/issues/2386). {#v0-0-0-removed} ### Removed From 444ca887d9ee50e6f551f8e58396865fb9573485 Mon Sep 17 00:00:00 2001 From: Ignas Anikevicius <240938+aignas@users.noreply.github.com> Date: Thu, 12 Dec 2024 02:21:20 +0900 Subject: [PATCH 024/922] ci: fix CI after bazel 8 release (#2492) Summary: * Remove the old `7.x` config from all `.bazelrc`. * Drop bazel 6 from example testing/support. The `.bazelrc` for enabling `WORKSPACE` cannot work across bazel 6,7,8. * Add missing `BUILD.bazel` file for integration tests. * Remove an integration test runner for `bazel 6.x`. * RBE test for bazel 8 is still not working. * bump `rules_java` for internal WORKSPACE dependencies to a version that supports `8.x`. * start running bazel-in-bazel integration tests using bazel `8.x`. * until bazel-contrib/rules_bazel_integration_test#414 is merged, we need to use bazel 7 for the delete_packagese pre-commit hook, not sure about how to track this as the `cgrindel/bazel-lib` needs a new version. Fixes #2378 --------- Co-authored-by: Richard Levasseur --- .bazelci/presubmit.yml | 23 +++++++++----------- .bazelrc | 4 ++-- .bazelversion | 2 +- .pre-commit-config.yaml | 4 +++- CHANGELOG.md | 4 +++- MODULE.bazel | 8 +++---- WORKSPACE | 15 +++++++++++++ examples/build_file_generation/.bazelrc | 6 ++--- examples/pip_parse/.bazelrc | 2 +- examples/pip_parse_vendored/.bazelrc | 5 +++-- examples/pip_repository_annotations/.bazelrc | 5 +++-- examples/py_proto_library/.bazelrc | 5 +++-- internal_dev_deps.bzl | 5 ++--- internal_dev_setup.bzl | 4 ---- python/BUILD.bazel | 1 + python/runtime_env_toolchains/BUILD.bazel | 6 +++++ tests/integration/BUILD.bazel | 23 +------------------- version.bzl | 4 ++-- 18 files changed, 63 insertions(+), 63 deletions(-) diff --git a/.bazelci/presubmit.yml b/.bazelci/presubmit.yml index 8c0252c3c8..f1a912cf80 100644 --- a/.bazelci/presubmit.yml +++ b/.bazelci/presubmit.yml @@ -18,12 +18,12 @@ buildifier: # Use a specific version to avoid skew issues when new versions are released. version: 6.1.0 warnings: "all" -# NOTE: Minimum supported version is 6.x for workspace; 7.x for bzlmod +# NOTE: Minimum supported version is 7.x .minimum_supported_version: &minimum_supported_version # For testing minimum supported version. # NOTE: Keep in sync with //:version.bzl - bazel: 6.4.0 - skip_in_bazel_downstream_pipeline: "Bazel 6 required" + bazel: 7.x + skip_in_bazel_downstream_pipeline: "Bazel 7 required" .reusable_config: &reusable_config build_targets: - "--" @@ -34,7 +34,6 @@ buildifier: build_flags: - "--keep_going" - "--build_tag_filters=-integration-test" - - "--config=bazel7.x" test_targets: - "--" - "..." @@ -55,6 +54,7 @@ buildifier: build_flags: - "--noenable_bzlmod" - "--enable_workspace" + bazel: 7.x .common_bazelinbazel_config: &common_bazelinbazel_config build_flags: - "--build_tag_filters=integration-test" @@ -159,7 +159,6 @@ tasks: - "--enable_workspace" - "--keep_going" - "--build_tag_filters=-integration-test" - - "--config=bazel7.x" test_targets: - "--" - "..." @@ -187,7 +186,6 @@ tasks: <<: *reusable_config name: "RBE: Ubuntu, minimum Bazel" platform: rbe_ubuntu2004 - bazel: 7.x build_flags: # BazelCI sets --action_env=BAZEL_DO_NOT_DETECT_CPP_TOOLCHAIN=1, # which prevents cc toolchain autodetection from working correctly @@ -206,6 +204,9 @@ tasks: <<: *reusable_config name: "RBE: Ubuntu" platform: rbe_ubuntu2004 + # TODO @aignas 2024-12-11: get the RBE working in CI for bazel 8.0 + # See https://github.com/bazelbuild/rules_python/issues/2499 + bazel: 7.x test_flags: - "--test_tag_filters=-integration-test,-acceptance-test" - "--extra_toolchains=@buildkite_config//config:cc-toolchain" @@ -412,25 +413,21 @@ tasks: name: "examples/pip_parse_vendored: Ubuntu, workspace, minimum Bazel" working_directory: examples/pip_parse_vendored platform: ubuntu2004 - integration_test_pip_parse_vendored_ubuntu_min_bzlmod: - <<: *minimum_supported_version - <<: *reusable_build_test_all - name: "examples/pip_parse_vendored: Ubuntu, bzlmod, minimum Bazel" - working_directory: examples/pip_parse_vendored - platform: ubuntu2004 - bazel: 7.x integration_test_pip_parse_vendored_ubuntu: <<: *reusable_build_test_all + <<: *common_workspace_flags name: "examples/pip_parse_vendored: Ubuntu" working_directory: examples/pip_parse_vendored platform: ubuntu2004 integration_test_pip_parse_vendored_debian: <<: *reusable_build_test_all + <<: *common_workspace_flags name: "examples/pip_parse_vendored: Debian" working_directory: examples/pip_parse_vendored platform: debian11 integration_test_pip_parse_vendored_macos: <<: *reusable_build_test_all + <<: *common_workspace_flags name: "examples/pip_parse_vendored: MacOS" working_directory: examples/pip_parse_vendored platform: macos diff --git a/.bazelrc b/.bazelrc index c44124d961..ada5c5a0a7 100644 --- a/.bazelrc +++ b/.bazelrc @@ -23,7 +23,7 @@ common --incompatible_disallow_struct_provider_syntax # Windows makes use of runfiles for some rules build --enable_runfiles -# Make Bazel 6 use bzlmod by default +# Make Bazel 7 use bzlmod by default common --enable_bzlmod # Additional config to use for readthedocs builds. @@ -33,6 +33,6 @@ build:rtd --stamp # Some bzl files contain repos only available under bzlmod build:rtd --enable_bzlmod -common:bazel7.x --incompatible_python_disallow_native_rules +common --incompatible_python_disallow_native_rules build --lockfile_mode=update diff --git a/.bazelversion b/.bazelversion index 35907cd9ca..c6b7980b68 100644 --- a/.bazelversion +++ b/.bazelversion @@ -1 +1 @@ -7.x +8.x diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml index 707a8d78aa..2b451e89fa 100644 --- a/.pre-commit-config.yaml +++ b/.pre-commit-config.yaml @@ -46,6 +46,8 @@ repos: - id: update-deleted-packages name: Update deleted packages language: system - entry: bazel run @rules_bazel_integration_test//tools:update_deleted_packages + # 7.x is necessary until https://github.com/bazel-contrib/rules_bazel_integration_test/pull/414 + # is merged and released + entry: env USE_BAZEL_VERSION=7.x bazel run @rules_bazel_integration_test//tools:update_deleted_packages files: ^((examples|tests)/.*/(MODULE.bazel|WORKSPACE|WORKSPACE.bzlmod|BUILD.bazel)|.bazelrc)$ pass_filenames: false diff --git a/CHANGELOG.md b/CHANGELOG.md index bcf9f52324..1f84193e62 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -52,7 +52,9 @@ Unreleased changes template. {#v0-0-0-changed} ### Changed -* Nothing changed. +* Bazel 6 support is dropped and Bazel 7.4.1 is the minimum supported + version, per our Bazel support matrix. Earlier versions are not + tested by CI, so functionality cannot be guaranteed. {#v0-0-0-fixed} ### Fixed diff --git a/MODULE.bazel b/MODULE.bazel index e4b113e785..57780b2369 100644 --- a/MODULE.bazel +++ b/MODULE.bazel @@ -150,8 +150,8 @@ bazel_binaries.local( name = "self", path = "tests/integration/bazel_from_env", ) -bazel_binaries.download(version = "6.4.0") -bazel_binaries.download(version = "7.4.0") +bazel_binaries.download(version = "7.4.1") +bazel_binaries.download(version = "8.0.0") # For now, don't test with rolling, because that's Bazel 9, which is a ways # away. @@ -162,8 +162,8 @@ use_repo( # These don't appear necessary, but are reported as direct dependencies # that should be use_repo()'d, so we add them as requested "bazel_binaries_bazelisk", - "build_bazel_bazel_6_4_0", - "build_bazel_bazel_7_4_0", + "build_bazel_bazel_7_4_1", + "build_bazel_bazel_8_0_0", # "build_bazel_bazel_rolling", "build_bazel_bazel_self", ) diff --git a/WORKSPACE b/WORKSPACE index c0d9f33a9b..6e9e85ac1e 100644 --- a/WORKSPACE +++ b/WORKSPACE @@ -21,6 +21,21 @@ load("//:internal_dev_deps.bzl", "rules_python_internal_deps") rules_python_internal_deps() +load("@rules_java//java:rules_java_deps.bzl", "rules_java_dependencies") + +rules_java_dependencies() + +# note that the following line is what is minimally required from protobuf for the java rules +# consider using the protobuf_deps() public API from @com_google_protobuf//:protobuf_deps.bzl +load("@com_google_protobuf//bazel/private:proto_bazel_features.bzl", "proto_bazel_features") # buildifier: disable=bzl-visibility + +proto_bazel_features(name = "proto_bazel_features") + +# register toolchains +load("@rules_java//java:repositories.bzl", "rules_java_toolchains") + +rules_java_toolchains() + load("@com_google_protobuf//:protobuf_deps.bzl", "protobuf_deps") protobuf_deps() diff --git a/examples/build_file_generation/.bazelrc b/examples/build_file_generation/.bazelrc index fd0f731d2f..306954d7be 100644 --- a/examples/build_file_generation/.bazelrc +++ b/examples/build_file_generation/.bazelrc @@ -5,6 +5,6 @@ build --enable_runfiles # The bzlmod version of this example is in examples/bzlmod_build_file_generation # Once WORKSPACE support is dropped, this example can be entirely deleted. -build --experimental_enable_bzlmod=false - -common:bazel7.x --incompatible_python_disallow_native_rules +common --noenable_bzlmod +common --enable_workspace +common --incompatible_python_disallow_native_rules diff --git a/examples/pip_parse/.bazelrc b/examples/pip_parse/.bazelrc index a56904803c..f263a1744d 100644 --- a/examples/pip_parse/.bazelrc +++ b/examples/pip_parse/.bazelrc @@ -1,3 +1,3 @@ # https://docs.bazel.build/versions/main/best-practices.html#using-the-bazelrc-file try-import %workspace%/user.bazelrc -common:bazel7.x --incompatible_python_disallow_native_rules +common --incompatible_python_disallow_native_rules diff --git a/examples/pip_parse_vendored/.bazelrc b/examples/pip_parse_vendored/.bazelrc index be3555d1eb..a6ea2d9138 100644 --- a/examples/pip_parse_vendored/.bazelrc +++ b/examples/pip_parse_vendored/.bazelrc @@ -5,5 +5,6 @@ build --enable_runfiles # Vendoring requirements.bzl files isn't necessary under bzlmod # When workspace support is dropped, this example can be removed. -build --noexperimental_enable_bzlmod -common:bazel7.x --incompatible_python_disallow_native_rules +common --noenable_bzlmod +common --enable_workspace +common --incompatible_python_disallow_native_rules diff --git a/examples/pip_repository_annotations/.bazelrc b/examples/pip_repository_annotations/.bazelrc index d893227946..c16c5a24f2 100644 --- a/examples/pip_repository_annotations/.bazelrc +++ b/examples/pip_repository_annotations/.bazelrc @@ -3,5 +3,6 @@ try-import %workspace%/user.bazelrc # This example is WORKSPACE specific. The equivalent functionality # is in examples/bzlmod as the `whl_mods` feature. -build --experimental_enable_bzlmod=false --enable_workspace=true -common:bazel7.x --incompatible_python_disallow_native_rules +common --noenable_bzlmod +common --enable_workspace +common --incompatible_python_disallow_native_rules diff --git a/examples/py_proto_library/.bazelrc b/examples/py_proto_library/.bazelrc index d73fc5387a..2ed86f591e 100644 --- a/examples/py_proto_library/.bazelrc +++ b/examples/py_proto_library/.bazelrc @@ -1,3 +1,4 @@ # The equivalent bzlmod behavior is covered by examples/bzlmod/py_proto_library -common --noenable_bzlmod --enable_workspace=true -common:bazel7.x --incompatible_python_disallow_native_rules +common --noenable_bzlmod +common --enable_workspace +common --incompatible_python_disallow_native_rules diff --git a/internal_dev_deps.bzl b/internal_dev_deps.bzl index e76ee48e83..0304fb16b7 100644 --- a/internal_dev_deps.bzl +++ b/internal_dev_deps.bzl @@ -195,10 +195,9 @@ def rules_python_internal_deps(): http_archive( name = "rules_java", urls = [ - "https://mirror.bazel.build/github.com/bazelbuild/rules_java/releases/download/8.3.1/rules_java-8.3.1.tar.gz", - "https://github.com/bazelbuild/rules_java/releases/download/8.3.1/rules_java-8.3.1.tar.gz", + "https://github.com/bazelbuild/rules_java/releases/download/8.6.2/rules_java-8.6.2.tar.gz", ], - sha256 = "ee786b943e00da4fea7c233e70e5f5b8a01cc69b9341b3f49169f174fe0df1c5", + sha256 = "a64ab04616e76a448c2c2d8165d836f0d2fb0906200d0b7c7376f46dd62e59cc", ) RULES_JVM_EXTERNAL_TAG = "5.2" diff --git a/internal_dev_setup.bzl b/internal_dev_setup.bzl index d6e95e22ce..fc38e3f9c5 100644 --- a/internal_dev_setup.bzl +++ b/internal_dev_setup.bzl @@ -20,7 +20,6 @@ load("@cgrindel_bazel_starlib//:deps.bzl", "bazel_starlib_dependencies") load("@com_google_protobuf//:protobuf_deps.bzl", "protobuf_deps") load("@rules_bazel_integration_test//bazel_integration_test:deps.bzl", "bazel_integration_test_rules_dependencies") load("@rules_bazel_integration_test//bazel_integration_test:repo_defs.bzl", "bazel_binaries") -load("@rules_java//java:repositories.bzl", "rules_java_dependencies", "rules_java_toolchains") load("@rules_shell//shell:repositories.bzl", "rules_shell_dependencies", "rules_shell_toolchains") load("//:version.bzl", "SUPPORTED_BAZEL_VERSIONS") load("//python:versions.bzl", "MINOR_MAPPING", "TOOL_VERSIONS") @@ -47,9 +46,6 @@ def rules_python_internal_setup(): protobuf_deps() - rules_java_dependencies() - rules_java_toolchains() - bazel_integration_test_rules_dependencies() bazel_starlib_dependencies() bazel_binaries(versions = SUPPORTED_BAZEL_VERSIONS) diff --git a/python/BUILD.bazel b/python/BUILD.bazel index 3422ef14fa..b747e2fbc7 100644 --- a/python/BUILD.bazel +++ b/python/BUILD.bazel @@ -43,6 +43,7 @@ filegroup( "//python/pip_install:distribution", "//python/private:distribution", "//python/runfiles:distribution", + "//python/runtime_env_toolchains:distribution", "//python/uv:distribution", ], visibility = ["//:__pkg__"], diff --git a/python/runtime_env_toolchains/BUILD.bazel b/python/runtime_env_toolchains/BUILD.bazel index 21355ac939..5001d12556 100644 --- a/python/runtime_env_toolchains/BUILD.bazel +++ b/python/runtime_env_toolchains/BUILD.bazel @@ -17,3 +17,9 @@ load("//python/private:runtime_env_toolchain.bzl", "define_runtime_env_toolchain package(default_visibility = ["//:__subpackages__"]) define_runtime_env_toolchain(name = "runtime_env_toolchain") + +filegroup( + name = "distribution", + srcs = glob(["**"]), + visibility = ["//python:__pkg__"], +) diff --git a/tests/integration/BUILD.bazel b/tests/integration/BUILD.bazel index 289c85d38a..d178e0f01c 100644 --- a/tests/integration/BUILD.bazel +++ b/tests/integration/BUILD.bazel @@ -19,11 +19,8 @@ load(":integration_test.bzl", "rules_python_integration_test") licenses(["notice"]) -_LEGACY_WORKSPACE_FLAGS = [ +_WORKSPACE_FLAGS = [ "--noenable_bzlmod", -] - -_WORKSPACE_FLAGS = _LEGACY_WORKSPACE_FLAGS + [ "--enable_workspace", ] @@ -35,24 +32,6 @@ _GAZELLE_PLUGIN_FLAGS = [ "--override_module=rules_python_gazelle_plugin=../../../rules_python_gazelle_plugin", ] -default_test_runner( - name = "bazel_6_4_workspace_test_runner", - bazel_cmds = [ - "info {}".format(" ".join(_LEGACY_WORKSPACE_FLAGS)), - "test {} //...".format(" ".join(_LEGACY_WORKSPACE_FLAGS)), - ], - visibility = ["//visibility:public"], -) - -default_test_runner( - name = "bazel_6_4_workspace_test_runner_gazelle_plugin", - bazel_cmds = [ - "info {}".format(" ".join(_LEGACY_WORKSPACE_FLAGS + _WORKSPACE_GAZELLE_PLUGIN_FLAGS)), - "test {} //...".format(" ".join(_LEGACY_WORKSPACE_FLAGS + _WORKSPACE_GAZELLE_PLUGIN_FLAGS)), - ], - visibility = ["//visibility:public"], -) - default_test_runner( name = "workspace_test_runner", bazel_cmds = [ diff --git a/version.bzl b/version.bzl index 61fb81efd4..4d85b5c420 100644 --- a/version.bzl +++ b/version.bzl @@ -17,11 +17,11 @@ # against. # This version should be updated together with the version of Bazel # in .bazelversion. -BAZEL_VERSION = "7.x" +BAZEL_VERSION = "8.x" # NOTE: Keep in sync with .bazelci/presubmit.yml # This is the minimum supported bazel version, that we have some tests for. -MINIMUM_BAZEL_VERSION = "6.4.0" +MINIMUM_BAZEL_VERSION = "7.4.1" # Versions of Bazel which users should be able to use. # Ensures we don't break backwards-compatibility, From e8236572b8270e06222cc605432bc8284a902408 Mon Sep 17 00:00:00 2001 From: Alex Martani Date: Wed, 11 Dec 2024 11:43:21 -0800 Subject: [PATCH 025/922] fix: Strip trailing slash for repo url (#2495) Strip potential trailing slash when building url on the case of relative path without up-references. In particular, this fixes using `experimental_index_url` with a AWS CodeArtifact python repository, which currently fails package downloads due to an incorrect URL (with double `//`) being produced by this code. Co-authored-by: Richard Levasseur --- python/private/pypi/parse_simpleapi_html.bzl | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/python/private/pypi/parse_simpleapi_html.bzl b/python/private/pypi/parse_simpleapi_html.bzl index b4e7dd8330..e549e76181 100644 --- a/python/private/pypi/parse_simpleapi_html.bzl +++ b/python/private/pypi/parse_simpleapi_html.bzl @@ -138,4 +138,4 @@ def _absolute_url(index_url, candidate): return "{}/{}".format(index_url, last.strip("/")) # relative path without up-references - return "{}/{}".format(index_url, candidate) + return "{}/{}".format(index_url.rstrip("/"), candidate) From bda710c54731753b652041aa7c7d9967c012cef0 Mon Sep 17 00:00:00 2001 From: Ignas Anikevicius <240938+aignas@users.noreply.github.com> Date: Thu, 12 Dec 2024 09:17:52 +0900 Subject: [PATCH 026/922] fix(pypi): pass requirements without env markers to the whl_library (#2488) With this change the environment markers from the requirements.txt files no longer end up in the whl_library definitions. I am reusing a function that already is parsing each requirement line for `sha256` values and added logic to extract the `marker` at that point. This means that the change is also trivial to backport to the `WORKSPACE` and the logic in the extension becomes simpler and we don't rely only on integration tests. Expected changes to the users: * If they have vendored pip requirements in `WORKSPACE`, those will be reformatted and the env markers will be removed. * The `MODULE.bazel.lock` file will be likewise reformatted if users are not using `--experimental_index_url`. Also, the env markers will not be passed in the `requirement`. * `bazel query 'deps("@pypi//foo")'` should start working in more cases. Fixes #2450. --------- Co-authored-by: Richard Levasseur --- CHANGELOG.md | 5 ++ examples/pip_parse_vendored/requirements.bzl | 10 +-- python/private/pypi/extension.bzl | 4 +- python/private/pypi/index_sources.bzl | 39 ++++++---- python/private/pypi/parse_requirements.bzl | 25 ++++--- python/private/pypi/pip_repository.bzl | 2 +- tests/pypi/extension/extension_tests.bzl | 25 ++++--- .../index_sources/index_sources_tests.bzl | 62 +++++++++++----- .../parse_requirements_tests.bzl | 71 +++++++++++-------- 9 files changed, 160 insertions(+), 83 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 1f84193e62..8605a4a03d 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -60,6 +60,11 @@ Unreleased changes template. ### Fixed * (py_wheel) Use the default shell environment when building wheels to allow toolchains that search PATH to be used for the wheel builder tool. +* (pypi) The requirement argument parsed to `whl_library` will now not have env + marker information allowing `bazel query` to work in cases where the `whl` is + available for all of the platforms and the sdist can be built. This fix is + for both WORKSPACE and `bzlmod` setups. + Fixes [#2450](https://github.com/bazelbuild/rules_python/issues/2450). {#v0-0-0-added} ### Added diff --git a/examples/pip_parse_vendored/requirements.bzl b/examples/pip_parse_vendored/requirements.bzl index 50bfe9fe8e..ead5c49b26 100644 --- a/examples/pip_parse_vendored/requirements.bzl +++ b/examples/pip_parse_vendored/requirements.bzl @@ -33,11 +33,11 @@ all_data_requirements = [ ] _packages = [ - ("my_project_pip_deps_vendored_certifi", "certifi==2023.7.22 --hash=sha256:539cc1d13202e33ca466e88b2807e29f4c13049d6d87031a3c110744495cb082 --hash=sha256:92d6037539857d8206b8f6ae472e8b77db8058fec5937a1ef3f54304089edbb9"), - ("my_project_pip_deps_vendored_charset_normalizer", "charset-normalizer==2.1.1 --hash=sha256:5a3d016c7c547f69d6f81fb0db9449ce888b418b5b9952cc5e6e66843e9dd845 --hash=sha256:83e9a75d1911279afd89352c68b45348559d1fc0506b054b346651b5e7fee29f"), - ("my_project_pip_deps_vendored_idna", "idna==3.4 --hash=sha256:814f528e8dead7d329833b91c5faa87d60bf71824cd12a7530b5526063d02cb4 --hash=sha256:90b77e79eaa3eba6de819a0c442c0b4ceefc341a7a2ab77d7562bf49f425c5c2"), - ("my_project_pip_deps_vendored_requests", "requests==2.28.1 --hash=sha256:7c5599b102feddaa661c826c56ab4fee28bfd17f5abca1ebbe3e7f19d7c97983 --hash=sha256:8fefa2a1a1365bf5520aac41836fbee479da67864514bdb821f31ce07ce65349"), - ("my_project_pip_deps_vendored_urllib3", "urllib3==1.26.13 --hash=sha256:47cc05d99aaa09c9e72ed5809b60e7ba354e64b59c9c173ac3018642d8bb41fc --hash=sha256:c083dd0dce68dbfbe1129d5271cb90f9447dea7d52097c6e0126120c521ddea8"), + ("my_project_pip_deps_vendored_certifi", "certifi==2023.7.22 --hash=sha256:539cc1d13202e33ca466e88b2807e29f4c13049d6d87031a3c110744495cb082 --hash=sha256:92d6037539857d8206b8f6ae472e8b77db8058fec5937a1ef3f54304089edbb9"), + ("my_project_pip_deps_vendored_charset_normalizer", "charset-normalizer==2.1.1 --hash=sha256:5a3d016c7c547f69d6f81fb0db9449ce888b418b5b9952cc5e6e66843e9dd845 --hash=sha256:83e9a75d1911279afd89352c68b45348559d1fc0506b054b346651b5e7fee29f"), + ("my_project_pip_deps_vendored_idna", "idna==3.4 --hash=sha256:814f528e8dead7d329833b91c5faa87d60bf71824cd12a7530b5526063d02cb4 --hash=sha256:90b77e79eaa3eba6de819a0c442c0b4ceefc341a7a2ab77d7562bf49f425c5c2"), + ("my_project_pip_deps_vendored_requests", "requests==2.28.1 --hash=sha256:7c5599b102feddaa661c826c56ab4fee28bfd17f5abca1ebbe3e7f19d7c97983 --hash=sha256:8fefa2a1a1365bf5520aac41836fbee479da67864514bdb821f31ce07ce65349"), + ("my_project_pip_deps_vendored_urllib3", "urllib3==1.26.13 --hash=sha256:47cc05d99aaa09c9e72ed5809b60e7ba354e64b59c9c173ac3018642d8bb41fc --hash=sha256:c083dd0dce68dbfbe1129d5271cb90f9447dea7d52097c6e0126120c521ddea8"), ] _config = { "download_only": False, diff --git a/python/private/pypi/extension.bzl b/python/private/pypi/extension.bzl index edfd5809f4..9b150bdce0 100644 --- a/python/private/pypi/extension.bzl +++ b/python/private/pypi/extension.bzl @@ -321,10 +321,10 @@ def _create_whl_repos( for requirement in requirements: is_exposed = is_exposed or requirement.is_exposed if get_index_urls: - logger.warn(lambda: "falling back to pip for installing the right file for {}".format(requirement.requirement_line)) + logger.warn(lambda: "falling back to pip for installing the right file for {}".format(requirement.srcs.requirement_line)) args = dict(whl_library_args) # make a copy - args["requirement"] = requirement.requirement_line + args["requirement"] = requirement.srcs.requirement_line if requirement.extra_pip_args: args["extra_pip_args"] = requirement.extra_pip_args diff --git a/python/private/pypi/index_sources.bzl b/python/private/pypi/index_sources.bzl index 21660141db..8b3c300946 100644 --- a/python/private/pypi/index_sources.bzl +++ b/python/private/pypi/index_sources.bzl @@ -26,28 +26,43 @@ def index_sources(line): line(str): The requirements.txt entry. Returns: - A struct with shas attribute containing a list of shas to download from pypi_index. + A struct with shas attribute containing: + * `shas` - list[str]; shas to download from pypi_index. + * `version` - str; version of the package. + * `marker` - str; the marker expression, as per PEP508 spec. + * `requirement` - str; a requirement line without the marker. This can + be given to `pip` to install a package. """ + line = line.replace("\\", " ") head, _, maybe_hashes = line.partition(";") _, _, version = head.partition("==") version = version.partition(" ")[0].strip() - if "@" in head: - shas = [] - else: - maybe_hashes = maybe_hashes or line - shas = [ - sha.strip() - for sha in maybe_hashes.split("--hash=sha256:")[1:] - ] + marker, _, _ = maybe_hashes.partition("--hash=") + maybe_hashes = maybe_hashes or line + shas = [ + sha.strip() + for sha in maybe_hashes.split("--hash=sha256:")[1:] + ] + marker = marker.strip() if head == line: - head = line.partition("--hash=")[0].strip() + requirement = line.partition("--hash=")[0].strip() else: - head = head + ";" + maybe_hashes.partition("--hash=")[0].strip() + requirement = head.strip() + + requirement_line = "{} {}".format( + requirement, + " ".join(["--hash=sha256:{}".format(sha) for sha in shas]), + ).strip() + if "@" in head: + requirement = requirement_line + shas = [] return struct( - requirement = line if not shas else head, + requirement = requirement, + requirement_line = requirement_line, version = version, shas = sorted(shas), + marker = marker, ) diff --git a/python/private/pypi/parse_requirements.bzl b/python/private/pypi/parse_requirements.bzl index 133ed18db8..821913d6de 100644 --- a/python/private/pypi/parse_requirements.bzl +++ b/python/private/pypi/parse_requirements.bzl @@ -74,16 +74,22 @@ def parse_requirements( logger: repo_utils.logger or None, a simple struct to log diagnostic messages. Returns: - A tuple where the first element a dict of dicts where the first key is - the normalized distribution name (with underscores) and the second key - is the requirement_line, then value and the keys are structs with the - following attributes: - * distribution: The non-normalized distribution name. - * srcs: The Simple API downloadable source list. - * requirement_line: The original requirement line. - * target_platforms: The list of target platforms that this package is for. - * is_exposed: A boolean if the package should be exposed via the hub + {type}`dict[str, list[struct]]` where the key is the distribution name and the struct + contains the following attributes: + * `distribution`: {type}`str` The non-normalized distribution name. + * `srcs`: {type}`struct` The parsed requirement line for easier Simple + API downloading (see `index_sources` return value). + * `target_platforms`: {type}`list[str]` Target platforms that this package is for. + The format is `cp3{minor}_{os}_{arch}`. + * `is_exposed`: {type}`bool` `True` if the package should be exposed via the hub repository. + * `extra_pip_args`: {type}`list[str]` pip args to use in case we are + not using the bazel downloader to download the archives. This should + be passed to {obj}`whl_library`. + * `whls`: {type}`list[struct]` The list of whl entries that can be + downloaded using the bazel downloader. + * `sdist`: {type}`list[struct]` The sdist that can be downloaded using + the bazel downloader. The second element is extra_pip_args should be passed to `whl_library`. """ @@ -209,7 +215,6 @@ def parse_requirements( struct( distribution = r.distribution, srcs = r.srcs, - requirement_line = r.requirement_line, target_platforms = sorted(target_platforms), extra_pip_args = r.extra_pip_args, whls = whls, diff --git a/python/private/pypi/pip_repository.bzl b/python/private/pypi/pip_repository.bzl index 47fa31f1bc..4591591dc9 100644 --- a/python/private/pypi/pip_repository.bzl +++ b/python/private/pypi/pip_repository.bzl @@ -101,7 +101,7 @@ def _pip_repository_impl(rctx): if not r: continue options = options or r.extra_pip_args - selected_requirements[name] = r.requirement_line + selected_requirements[name] = r.srcs.requirement_line bzl_packages = sorted(selected_requirements.keys()) diff --git a/tests/pypi/extension/extension_tests.bzl b/tests/pypi/extension/extension_tests.bzl index b9427795ec..1caab23cea 100644 --- a/tests/pypi/extension/extension_tests.bzl +++ b/tests/pypi/extension/extension_tests.bzl @@ -28,7 +28,10 @@ def _mock_mctx(*modules, environ = {}, read = None): name = "unittest", arch = "exotic", ), - read = read or (lambda _: "simple==0.0.1 --hash=sha256:deadbeef --hash=sha256:deadbaaf"), + read = read or (lambda _: """\ +simple==0.0.1 \ + --hash=sha256:deadbeef \ + --hash=sha256:deadbaaf"""), modules = [ struct( name = modules[0].name, @@ -262,7 +265,8 @@ def _test_simple_with_markers(env): read = lambda x: { "universal.txt": """\ torch==2.4.1+cpu ; platform_machine == 'x86_64' -torch==2.4.1 ; platform_machine != 'x86_64' +torch==2.4.1 ; platform_machine != 'x86_64' \ + --hash=sha256:deadbeef """, }[x], ), @@ -313,13 +317,13 @@ torch==2.4.1 ; platform_machine != 'x86_64' "dep_template": "@pypi//{name}:{target}", "python_interpreter_target": "unit_test_interpreter_target", "repo": "pypi_315", - "requirement": "torch==2.4.1 ; platform_machine != 'x86_64'", + "requirement": "torch==2.4.1 --hash=sha256:deadbeef", }, "pypi_315_torch_linux_x86_64_osx_x86_64_windows_x86_64": { "dep_template": "@pypi//{name}:{target}", "python_interpreter_target": "unit_test_interpreter_target", "repo": "pypi_315", - "requirement": "torch==2.4.1+cpu ; platform_machine == 'x86_64'", + "requirement": "torch==2.4.1+cpu", }, }) pypi.whl_mods().contains_exactly({}) @@ -351,8 +355,10 @@ def _test_download_only_multiple(env): --implementation=cp --abi=cp315 -simple==0.0.1 --hash=sha256:deadbeef -extra==0.0.1 --hash=sha256:deadb00f +simple==0.0.1 \ + --hash=sha256:deadbeef +extra==0.0.1 \ + --hash=sha256:deadb00f """, "requirements.osx_aarch64.txt": """\ --platform=macosx_10_9_arm64 @@ -360,7 +366,8 @@ extra==0.0.1 --hash=sha256:deadb00f --implementation=cp --abi=cp315 -simple==0.0.3 --hash=sha256:deadbaaf +simple==0.0.3 \ + --hash=sha256:deadbaaf """, }[x], ), @@ -473,7 +480,9 @@ def _test_simple_get_index(env): ), read = lambda x: { "requirements.txt": """ -simple==0.0.1 --hash=sha256:deadbeef --hash=sha256:deadb00f +simple==0.0.1 \ + --hash=sha256:deadbeef \ + --hash=sha256:deadb00f some_pkg==0.0.1 """, }[x], diff --git a/tests/pypi/index_sources/index_sources_tests.bzl b/tests/pypi/index_sources/index_sources_tests.bzl index 0a767078ba..440957e2f0 100644 --- a/tests/pypi/index_sources/index_sources_tests.bzl +++ b/tests/pypi/index_sources/index_sources_tests.bzl @@ -20,34 +20,62 @@ load("//python/private/pypi:index_sources.bzl", "index_sources") # buildifier: _tests = [] def _test_no_simple_api_sources(env): - inputs = [ - "foo==0.0.1", - "foo==0.0.1 @ https://someurl.org", - "foo==0.0.1 @ https://someurl.org --hash=sha256:deadbeef", - "foo==0.0.1 @ https://someurl.org; python_version < 2.7 --hash=sha256:deadbeef", - ] - for input in inputs: + inputs = { + "foo==0.0.1": struct( + requirement = "foo==0.0.1", + marker = "", + ), + "foo==0.0.1 @ https://someurl.org": struct( + requirement = "foo==0.0.1 @ https://someurl.org", + marker = "", + ), + "foo==0.0.1 @ https://someurl.org --hash=sha256:deadbeef": struct( + requirement = "foo==0.0.1 @ https://someurl.org --hash=sha256:deadbeef", + marker = "", + ), + "foo==0.0.1 @ https://someurl.org; python_version < \"2.7\"\\ --hash=sha256:deadbeef": struct( + requirement = "foo==0.0.1 @ https://someurl.org --hash=sha256:deadbeef", + marker = "python_version < \"2.7\"", + ), + } + for input, want in inputs.items(): got = index_sources(input) env.expect.that_collection(got.shas).contains_exactly([]) env.expect.that_str(got.version).equals("0.0.1") + env.expect.that_str(got.requirement).equals(want.requirement) + env.expect.that_str(got.requirement_line).equals(got.requirement) + env.expect.that_str(got.marker).equals(want.marker) _tests.append(_test_no_simple_api_sources) def _test_simple_api_sources(env): tests = { - "foo==0.0.2 --hash=sha256:deafbeef --hash=sha256:deadbeef": [ - "deadbeef", - "deafbeef", - ], - "foo[extra]==0.0.2; (python_version < 2.7 or something_else == \"@\") --hash=sha256:deafbeef --hash=sha256:deadbeef": [ - "deadbeef", - "deafbeef", - ], + "foo==0.0.2 --hash=sha256:deafbeef --hash=sha256:deadbeef": struct( + shas = [ + "deadbeef", + "deafbeef", + ], + marker = "", + requirement = "foo==0.0.2", + requirement_line = "foo==0.0.2 --hash=sha256:deafbeef --hash=sha256:deadbeef", + ), + "foo[extra]==0.0.2; (python_version < 2.7 or extra == \"@\") --hash=sha256:deafbeef --hash=sha256:deadbeef": struct( + shas = [ + "deadbeef", + "deafbeef", + ], + marker = "(python_version < 2.7 or extra == \"@\")", + requirement = "foo[extra]==0.0.2", + requirement_line = "foo[extra]==0.0.2 --hash=sha256:deafbeef --hash=sha256:deadbeef", + ), } - for input, want_shas in tests.items(): + for input, want in tests.items(): got = index_sources(input) - env.expect.that_collection(got.shas).contains_exactly(want_shas) + env.expect.that_collection(got.shas).contains_exactly(want.shas) env.expect.that_str(got.version).equals("0.0.2") + env.expect.that_str(got.requirement).equals(want.requirement) + env.expect.that_str(got.requirement_line).equals(want.requirement_line) + env.expect.that_str(got.marker).equals(want.marker) _tests.append(_test_simple_api_sources) diff --git a/tests/pypi/parse_requirements/parse_requirements_tests.bzl b/tests/pypi/parse_requirements/parse_requirements_tests.bzl index dfa1fef5c3..77e22b825a 100644 --- a/tests/pypi/parse_requirements/parse_requirements_tests.bzl +++ b/tests/pypi/parse_requirements/parse_requirements_tests.bzl @@ -20,8 +20,10 @@ load("//python/private/pypi:parse_requirements.bzl", "parse_requirements", "sele def _mock_ctx(): testdata = { "requirements_different_package_version": """\ -foo==0.0.1+local --hash=sha256:deadbeef -foo==0.0.1 --hash=sha256:deadb00f +foo==0.0.1+local \ + --hash=sha256:deadbeef +foo==0.0.1 \ + --hash=sha256:deadb00f """, "requirements_direct": """\ foo[extra] @ https://some-url @@ -29,7 +31,8 @@ foo[extra] @ https://some-url "requirements_extra_args": """\ --index-url=example.org -foo[extra]==0.0.1 --hash=sha256:deadbeef +foo[extra]==0.0.1 \ + --hash=sha256:deadbeef """, "requirements_linux": """\ foo==0.0.3 --hash=sha256:deadbaaf @@ -95,9 +98,12 @@ def _test_simple(env): struct( distribution = "foo", extra_pip_args = [], - requirement_line = "foo[extra]==0.0.1 --hash=sha256:deadbeef", + sdist = None, + is_exposed = True, srcs = struct( + marker = "", requirement = "foo[extra]==0.0.1", + requirement_line = "foo[extra]==0.0.1 --hash=sha256:deadbeef", shas = ["deadbeef"], version = "0.0.1", ), @@ -106,8 +112,6 @@ def _test_simple(env): "windows_x86_64", ], whls = [], - sdist = None, - is_exposed = True, ), ], }) @@ -133,9 +137,12 @@ def _test_extra_pip_args(env): struct( distribution = "foo", extra_pip_args = ["--index-url=example.org", "--trusted-host=example.org"], - requirement_line = "foo[extra]==0.0.1 --hash=sha256:deadbeef", + sdist = None, + is_exposed = True, srcs = struct( + marker = "", requirement = "foo[extra]==0.0.1", + requirement_line = "foo[extra]==0.0.1 --hash=sha256:deadbeef", shas = ["deadbeef"], version = "0.0.1", ), @@ -143,8 +150,6 @@ def _test_extra_pip_args(env): "linux_x86_64", ], whls = [], - sdist = None, - is_exposed = True, ), ], }) @@ -169,16 +174,17 @@ def _test_dupe_requirements(env): struct( distribution = "foo", extra_pip_args = [], - requirement_line = "foo[extra,extra_2]==0.0.1 --hash=sha256:deadbeef", + sdist = None, + is_exposed = True, srcs = struct( + marker = "", requirement = "foo[extra,extra_2]==0.0.1", + requirement_line = "foo[extra,extra_2]==0.0.1 --hash=sha256:deadbeef", shas = ["deadbeef"], version = "0.0.1", ), target_platforms = ["linux_x86_64"], whls = [], - sdist = None, - is_exposed = True, ), ], }) @@ -199,9 +205,10 @@ def _test_multi_os(env): struct( distribution = "bar", extra_pip_args = [], - requirement_line = "bar==0.0.1 --hash=sha256:deadb00f", srcs = struct( + marker = "", requirement = "bar==0.0.1", + requirement_line = "bar==0.0.1 --hash=sha256:deadb00f", shas = ["deadb00f"], version = "0.0.1", ), @@ -215,9 +222,10 @@ def _test_multi_os(env): struct( distribution = "foo", extra_pip_args = [], - requirement_line = "foo==0.0.3 --hash=sha256:deadbaaf", srcs = struct( + marker = "", requirement = "foo==0.0.3", + requirement_line = "foo==0.0.3 --hash=sha256:deadbaaf", shas = ["deadbaaf"], version = "0.0.3", ), @@ -229,9 +237,10 @@ def _test_multi_os(env): struct( distribution = "foo", extra_pip_args = [], - requirement_line = "foo[extra]==0.0.2 --hash=sha256:deadbeef", srcs = struct( + marker = "", requirement = "foo[extra]==0.0.2", + requirement_line = "foo[extra]==0.0.2 --hash=sha256:deadbeef", shas = ["deadbeef"], version = "0.0.2", ), @@ -266,10 +275,11 @@ def _test_multi_os_legacy(env): distribution = "bar", extra_pip_args = ["--platform=manylinux_2_17_x86_64", "--python-version=39", "--implementation=cp", "--abi=cp39"], is_exposed = False, - requirement_line = "bar==0.0.1 --hash=sha256:deadb00f", sdist = None, srcs = struct( + marker = "", requirement = "bar==0.0.1", + requirement_line = "bar==0.0.1 --hash=sha256:deadb00f", shas = ["deadb00f"], version = "0.0.1", ), @@ -282,10 +292,11 @@ def _test_multi_os_legacy(env): distribution = "foo", extra_pip_args = ["--platform=manylinux_2_17_x86_64", "--python-version=39", "--implementation=cp", "--abi=cp39"], is_exposed = True, - requirement_line = "foo==0.0.1 --hash=sha256:deadbeef", sdist = None, srcs = struct( + marker = "", requirement = "foo==0.0.1", + requirement_line = "foo==0.0.1 --hash=sha256:deadbeef", shas = ["deadbeef"], version = "0.0.1", ), @@ -296,9 +307,10 @@ def _test_multi_os_legacy(env): distribution = "foo", extra_pip_args = ["--platform=macosx_10_9_arm64", "--python-version=39", "--implementation=cp", "--abi=cp39"], is_exposed = True, - requirement_line = "foo==0.0.3 --hash=sha256:deadbaaf", sdist = None, srcs = struct( + marker = "", + requirement_line = "foo==0.0.3 --hash=sha256:deadbaaf", requirement = "foo==0.0.3", shas = ["deadbaaf"], version = "0.0.3", @@ -348,10 +360,11 @@ def _test_env_marker_resolution(env): distribution = "bar", extra_pip_args = [], is_exposed = True, - requirement_line = "bar==0.0.1 --hash=sha256:deadbeef", sdist = None, srcs = struct( + marker = "", requirement = "bar==0.0.1", + requirement_line = "bar==0.0.1 --hash=sha256:deadbeef", shas = ["deadbeef"], version = "0.0.1", ), @@ -363,12 +376,12 @@ def _test_env_marker_resolution(env): struct( distribution = "foo", extra_pip_args = [], - # This is not exposed because we also have `linux_super_exotic` in the platform list is_exposed = False, - requirement_line = "foo[extra]==0.0.1 ;marker --hash=sha256:deadbeef", sdist = None, srcs = struct( - requirement = "foo[extra]==0.0.1 ;marker", + marker = "marker", + requirement = "foo[extra]==0.0.1", + requirement_line = "foo[extra]==0.0.1 --hash=sha256:deadbeef", shas = ["deadbeef"], version = "0.0.1", ), @@ -398,30 +411,32 @@ def _test_different_package_version(env): struct( distribution = "foo", extra_pip_args = [], - requirement_line = "foo==0.0.1 --hash=sha256:deadb00f", + is_exposed = True, + sdist = None, srcs = struct( + marker = "", requirement = "foo==0.0.1", + requirement_line = "foo==0.0.1 --hash=sha256:deadb00f", shas = ["deadb00f"], version = "0.0.1", ), target_platforms = ["linux_x86_64"], whls = [], - sdist = None, - is_exposed = True, ), struct( distribution = "foo", extra_pip_args = [], - requirement_line = "foo==0.0.1+local --hash=sha256:deadbeef", + is_exposed = True, + sdist = None, srcs = struct( + marker = "", requirement = "foo==0.0.1+local", + requirement_line = "foo==0.0.1+local --hash=sha256:deadbeef", shas = ["deadbeef"], version = "0.0.1+local", ), target_platforms = ["linux_x86_64"], whls = [], - sdist = None, - is_exposed = True, ), ], }) From 94c77e2fbe98e6a69599ddd58b4adf519c7d8316 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Fri, 13 Dec 2024 08:33:34 -0800 Subject: [PATCH 027/922] fix: make bazel 9 workspace recognize rules_python as the main module (#2501) Building with Bazel 9 using WORKSPACE results in an odd error that `PyInfo` isn't defined. Oddly, the error refers to `rules_python/python/private/reexports.bzl` for rules_python 0.28.0. This seems to only happen when the main module is rules_python. While Bazel 9 is supposed to drop workspace support, I've been advised it's better to keep testing WORKSPACE support until closer to when Bazel 9 fully removes it. My best guess about what's happening is Bazel's autoloading is triggering and somehow defining rules_python before it's recognized that the main module is rules_python. The autoloading appears to be triggered, eventually, by things in bazel_tools loading rules_python. While removing unnecessary `@bazel_tools` loads in rules_python helps, the particular case I can't find a clean solution to is when `@@rules_java//toolchains:toolchain_java11_definition` causes rules_python to be loaded. This appears to end up loading rules_python via `@bazel_tools//tools/jdk:BUILD`, which has has some py rules defined in it. To fix/work around this issue, `local_repository` can be used to define the `rules_python` repo before autoloading happens. This appears to take precedence over whatever logic autoloading has. Work towards https://github.com/bazelbuild/rules_python/issues/2469 --- WORKSPACE | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/WORKSPACE b/WORKSPACE index 6e9e85ac1e..7303b480f2 100644 --- a/WORKSPACE +++ b/WORKSPACE @@ -17,6 +17,14 @@ workspace(name = "rules_python") # Everything below this line is used only for developing rules_python. Users # should not copy it to their WORKSPACE. +# Necessary so that Bazel 9 recognizes this as rules_python and doesn't try +# to load the version Bazel itself uses by default. +# buildifier: disable=duplicated-name +local_repository( + name = "rules_python", + path = ".", +) + load("//:internal_dev_deps.bzl", "rules_python_internal_deps") rules_python_internal_deps() From 15cc0b3086ec002499121ab295587e6538e4f999 Mon Sep 17 00:00:00 2001 From: Douglas Thor Date: Fri, 13 Dec 2024 17:49:54 -0800 Subject: [PATCH 028/922] fix(gazelle): Support parsing files that use Python3.12 PEP 695 (Type Parameter Syntax) by using dougthor42's fork of go-tree-sitter (#2496) Replaces #2413. Fixes #2396. This updates the `go-tree-sitter` dependency to use my fork that includes `BUILD.bazel` files. Specifically, the `BUILD.bazel` files in the fork include references to top-level code like `array.h` which the original Gazelle-generated files for `go-tree-sitter` were not able to handle. I also include the test cases that @maffoo created in #2413 and verified that they (a) fail before the fix and (b) pass after the fix. The fork is: https://github.com/dougthor42/go-tree-sitter The branch that includes all changes is: https://github.com/dougthor42/go-tree-sitter/tree/for-rules-python-gazelle-plugin A couple notes: + I have a PR open to get `go-tree-sitter` into BCR [here](https://github.com/bazelbuild/bazel-central-registry/pull/3366). However: 1. I'm having trouble getting tests to pass and to get things running locally to validate it 2. Using BCR would not fix things for people who still use WORKSPACE (right?) + The fork is _mostly_ [autogenerated BUILD.bazel files from gazelle](https://github.com/smacker/go-tree-sitter/commit/cfa9bdf58beb30159807c1846e1c761bce1e6158) but also contains: + [manual updates so that build files reference the toplevel `array.h` and other files](https://github.com/smacker/go-tree-sitter/commit/63f89cd3d471e7e81b51dc7e7205b201fed70fc1) + [replace all `smacker` with `dougthor42` so that `go build` works](https://github.com/smacker/go-tree-sitter/commit/8a73cbdb0e9b5febc314f8b9b4241aa01cc86bd0) + various other more minor things. + I was unable to get `go mod edit -replace` to work, so I've just manually updated `go.mod` and whatnot everywhere. If someone with more go knowledge has a suggestion I'm happy to hear it. --------- Co-authored-by: Matthew Neeley Co-authored-by: Ignas Anikevicius <240938+aignas@users.noreply.github.com> --- CHANGELOG.md | 8 +++++-- gazelle/MODULE.bazel | 2 +- gazelle/deps.bzl | 8 +++---- gazelle/go.mod | 2 +- gazelle/go.sum | 10 ++------- gazelle/python/BUILD.bazel | 4 ++-- gazelle/python/file_parser.go | 8 +++---- gazelle/python/testdata/py312_syntax/BUILD.in | 1 + .../python/testdata/py312_syntax/BUILD.out | 16 ++++++++++++++ .../python/testdata/py312_syntax/README.md | 4 ++++ .../python/testdata/py312_syntax/WORKSPACE | 1 + .../python/testdata/py312_syntax/__init__.py | 0 .../testdata/py312_syntax/_other_module.py | 0 .../py312_syntax/pep_695_type_parameter.py | 22 +++++++++++++++++++ .../python/testdata/py312_syntax/test.yaml | 1 + 15 files changed, 65 insertions(+), 22 deletions(-) create mode 100644 gazelle/python/testdata/py312_syntax/BUILD.in create mode 100644 gazelle/python/testdata/py312_syntax/BUILD.out create mode 100644 gazelle/python/testdata/py312_syntax/README.md create mode 100644 gazelle/python/testdata/py312_syntax/WORKSPACE create mode 100644 gazelle/python/testdata/py312_syntax/__init__.py create mode 100644 gazelle/python/testdata/py312_syntax/_other_module.py create mode 100644 gazelle/python/testdata/py312_syntax/pep_695_type_parameter.py create mode 100644 gazelle/python/testdata/py312_syntax/test.yaml diff --git a/CHANGELOG.md b/CHANGELOG.md index 8605a4a03d..33719187f7 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -65,12 +65,16 @@ Unreleased changes template. available for all of the platforms and the sdist can be built. This fix is for both WORKSPACE and `bzlmod` setups. Fixes [#2450](https://github.com/bazelbuild/rules_python/issues/2450). +* (gazelle) Gazelle will now correctly parse Python3.12 files that use [PEP 695 Type + Parameter Syntax][pep-695]. (#2396) + +[pep-695]: https://peps.python.org/pep-0695/ {#v0-0-0-added} ### Added * (gazelle) Added `include_stub_packages` flag to `modules_mapping`. When set to `True`, this automatically includes corresponding stub packages for third-party libraries - that are present and used (e.g., `boto3` → `boto3-stubs`), improving + that are present and used (e.g., `boto3` → `boto3-stubs`), improving type-checking support. * (pypi) Freethreaded packages are now fully supported in the {obj}`experimental_index_url` usage or the regular `pip.parse` usage. @@ -137,7 +141,7 @@ Other changes: * (repositories): Add libs/python3.lib and pythonXY.dll to the `libpython` target defined by a repository template. This enables stable ABI builds of Python extensions on Windows (by defining Py_LIMITED_API). -* (rules) `py_test` and `py_binary` targets no longer incorrectly remove the +* (rules) `py_test` and `py_binary` targets no longer incorrectly remove the first `sys.path` entry when using {obj}`--bootstrap_impl=script` {#v1-0-0-added} diff --git a/gazelle/MODULE.bazel b/gazelle/MODULE.bazel index 0a553831c3..6bbc74bc61 100644 --- a/gazelle/MODULE.bazel +++ b/gazelle/MODULE.bazel @@ -21,9 +21,9 @@ use_repo( go_deps, "com_github_bazelbuild_buildtools", "com_github_bmatcuk_doublestar_v4", + "com_github_dougthor42_go_tree_sitter", "com_github_emirpasic_gods", "com_github_ghodss_yaml", - "com_github_smacker_go_tree_sitter", "com_github_stretchr_testify", "in_gopkg_yaml_v2", "org_golang_x_sync", diff --git a/gazelle/deps.bzl b/gazelle/deps.bzl index 948d61e5ae..1bdf179e98 100644 --- a/gazelle/deps.bzl +++ b/gazelle/deps.bzl @@ -186,10 +186,10 @@ def go_deps(): version = "v0.0.0-20190812154241-14fe0d1b01d4", ) go_repository( - name = "com_github_smacker_go_tree_sitter", - importpath = "github.com/smacker/go-tree-sitter", - sum = "h1:7QZKUmQfnxncZIJGyvX8M8YeMfn8kM10j3J/2KwVTN4=", - version = "v0.0.0-20240422154435-0628b34cbf9c", + name = "com_github_dougthor42_go_tree_sitter", + importpath = "github.com/dougthor42/go-tree-sitter", + sum = "h1:b9s96BulIARx0konX36sJ5oZhWvAvjQBBntxp1eUukQ=", + version = "v0.0.0-20241210060307-2737e1d0de6b", ) go_repository( name = "com_github_stretchr_objx", diff --git a/gazelle/go.mod b/gazelle/go.mod index 4b65e71d67..29a0b5cb0c 100644 --- a/gazelle/go.mod +++ b/gazelle/go.mod @@ -7,9 +7,9 @@ require ( github.com/bazelbuild/buildtools v0.0.0-20231103205921-433ea8554e82 github.com/bazelbuild/rules_go v0.41.0 github.com/bmatcuk/doublestar/v4 v4.6.1 + github.com/dougthor42/go-tree-sitter v0.0.0-20241210060307-2737e1d0de6b github.com/emirpasic/gods v1.18.1 github.com/ghodss/yaml v1.0.0 - github.com/smacker/go-tree-sitter v0.0.0-20240422154435-0628b34cbf9c github.com/stretchr/testify v1.9.0 golang.org/x/sync v0.2.0 gopkg.in/yaml.v2 v2.4.0 diff --git a/gazelle/go.sum b/gazelle/go.sum index 46e0127e8f..d48da9ece3 100644 --- a/gazelle/go.sum +++ b/gazelle/go.sum @@ -13,9 +13,10 @@ github.com/chzyer/logex v1.1.10/go.mod h1:+Ywpsq7O8HXn0nuIou7OrIPyXbp3wmkHB+jjWR github.com/chzyer/readline v0.0.0-20180603132655-2972be24d48e/go.mod h1:nSuG5e5PlCu98SY8svDHJxuZscDgtXS6KTTbou5AhLI= github.com/chzyer/test v0.0.0-20180213035817-a1ea475d72b1/go.mod h1:Q3SI9o4m/ZMnBNeIyt5eFwwo7qiLfzFZmjNmxjkiQlU= github.com/client9/misspell v0.3.4/go.mod h1:qj6jICC3Q7zFZvVWo7KLAzC3yx5G7kyvSDkc90ppPyw= -github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38= github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c= github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38= +github.com/dougthor42/go-tree-sitter v0.0.0-20241210060307-2737e1d0de6b h1:b9s96BulIARx0konX36sJ5oZhWvAvjQBBntxp1eUukQ= +github.com/dougthor42/go-tree-sitter v0.0.0-20241210060307-2737e1d0de6b/go.mod h1:87UkDyPt18bTH/FvinLc/kj587VNYOdRKZT1la4T8Hg= github.com/emirpasic/gods v1.18.1 h1:FXtiHYKDGKCW2KzwZKx0iC0PQmdlorYgdFG9jPXJ1Bc= github.com/emirpasic/gods v1.18.1/go.mod h1:8tpGGwCnJ5H4r6BWwaV6OrWmMoPhUl5jm/FMNAnJvWQ= github.com/envoyproxy/go-control-plane v0.9.1-0.20191026205805-5f8ba28d4473/go.mod h1:YTl/9mNaCwkRvm6d1a2C3ymFceY/DCBVvsKhRF0iEA4= @@ -44,12 +45,6 @@ github.com/google/go-cmp v0.5.9/go.mod h1:17dUlkBOakJ0+DkrSSNjCkIjxS6bF9zb3elmeN github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM= github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4= github.com/prometheus/client_model v0.0.0-20190812154241-14fe0d1b01d4/go.mod h1:xMI15A0UPsDsEKsMN9yxemIoYk6Tm2C1GtYGdfGttqA= -github.com/smacker/go-tree-sitter v0.0.0-20240422154435-0628b34cbf9c h1:7QZKUmQfnxncZIJGyvX8M8YeMfn8kM10j3J/2KwVTN4= -github.com/smacker/go-tree-sitter v0.0.0-20240422154435-0628b34cbf9c/go.mod h1:q99oHDsbP0xRwmn7Vmob8gbSMNyvJ83OauXPSuHQuKE= -github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME= -github.com/stretchr/objx v0.4.0/go.mod h1:YvHI0jy2hoMjB+UWwv71VJQ9isScKT/TqJzVSSt89Yw= -github.com/stretchr/testify v1.7.1/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg= -github.com/stretchr/testify v1.7.4/go.mod h1:yNjHg4UonilssWZ8iaSj1OCr/vHnekPRkoO+kdMU+MU= github.com/stretchr/testify v1.9.0 h1:HtqpIVDClZ4nwg75+f6Lvsy/wHu+3BoSGCbBAcpTsTg= github.com/stretchr/testify v1.9.0/go.mod h1:r2ic/lqez/lEtzL7wO/rwa5dbSLXVDPFyf8C91i36aY= go.starlark.net v0.0.0-20210223155950-e043a3d3c984/go.mod h1:t3mmBBPzAVvK0L0n1drDmrQsJ8FoIx4INCqVMTr/Zo0= @@ -105,7 +100,6 @@ gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405 h1:yhCVgyC4o1eVCa2tZl7eS0r+ gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0= gopkg.in/yaml.v2 v2.4.0 h1:D8xgwECY7CYvx+Y2n4sBz93Jn9JRvxdiyyo8CTfuKaY= gopkg.in/yaml.v2 v2.4.0/go.mod h1:RDklbk79AGWmwhnvt/jBztapEOGDOx6ZbXqjP6csGnQ= -gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM= gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA= gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM= honnef.co/go/tools v0.0.0-20190102054323-c2f93a96b099/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4= diff --git a/gazelle/python/BUILD.bazel b/gazelle/python/BUILD.bazel index 627a867c68..893c82e8e4 100644 --- a/gazelle/python/BUILD.bazel +++ b/gazelle/python/BUILD.bazel @@ -39,11 +39,11 @@ go_library( "@bazel_gazelle//rule:go_default_library", "@com_github_bazelbuild_buildtools//build:go_default_library", "@com_github_bmatcuk_doublestar_v4//:doublestar", + "@com_github_dougthor42_go_tree_sitter//:go-tree-sitter", + "@com_github_dougthor42_go_tree_sitter//python", "@com_github_emirpasic_gods//lists/singlylinkedlist", "@com_github_emirpasic_gods//sets/treeset", "@com_github_emirpasic_gods//utils", - "@com_github_smacker_go_tree_sitter//:go-tree-sitter", - "@com_github_smacker_go_tree_sitter//python", "@org_golang_x_sync//errgroup", ], ) diff --git a/gazelle/python/file_parser.go b/gazelle/python/file_parser.go index a1f47f400c..c147984fc3 100644 --- a/gazelle/python/file_parser.go +++ b/gazelle/python/file_parser.go @@ -22,8 +22,8 @@ import ( "path/filepath" "strings" - sitter "github.com/smacker/go-tree-sitter" - "github.com/smacker/go-tree-sitter/python" + sitter "github.com/dougthor42/go-tree-sitter" + "github.com/dougthor42/go-tree-sitter/python" ) const ( @@ -115,10 +115,10 @@ func (p *FileParser) parseMain(ctx context.Context, node *sitter.Node) bool { a, b = b, a } if a.Type() == sitterNodeTypeIdentifier && a.Content(p.code) == "__name__" && - // at github.com/smacker/go-tree-sitter@latest (after v0.0.0-20240422154435-0628b34cbf9c we used) + // at github.com/dougthor42/go-tree-sitter@latest (after v0.0.0-20240422154435-0628b34cbf9c we used) // "__main__" is the second child of b. But now, it isn't. // we cannot use the latest go-tree-sitter because of the top level reference in scanner.c. - // https://github.com/smacker/go-tree-sitter/blob/04d6b33fe138a98075210f5b770482ded024dc0f/python/scanner.c#L1 + // https://github.com/dougthor42/go-tree-sitter/blob/04d6b33fe138a98075210f5b770482ded024dc0f/python/scanner.c#L1 b.Type() == sitterNodeTypeString && string(p.code[b.StartByte()+1:b.EndByte()-1]) == "__main__" { return true } diff --git a/gazelle/python/testdata/py312_syntax/BUILD.in b/gazelle/python/testdata/py312_syntax/BUILD.in new file mode 100644 index 0000000000..af2c2cea4b --- /dev/null +++ b/gazelle/python/testdata/py312_syntax/BUILD.in @@ -0,0 +1 @@ +# gazelle:python_generation_mode file diff --git a/gazelle/python/testdata/py312_syntax/BUILD.out b/gazelle/python/testdata/py312_syntax/BUILD.out new file mode 100644 index 0000000000..7457f335a7 --- /dev/null +++ b/gazelle/python/testdata/py312_syntax/BUILD.out @@ -0,0 +1,16 @@ +load("@rules_python//python:defs.bzl", "py_binary", "py_library") + +# gazelle:python_generation_mode file + +py_library( + name = "_other_module", + srcs = ["_other_module.py"], + visibility = ["//:__subpackages__"], +) + +py_binary( + name = "pep_695_type_parameter", + srcs = ["pep_695_type_parameter.py"], + visibility = ["//:__subpackages__"], + deps = [":_other_module"], +) diff --git a/gazelle/python/testdata/py312_syntax/README.md b/gazelle/python/testdata/py312_syntax/README.md new file mode 100644 index 0000000000..854a0a3aa6 --- /dev/null +++ b/gazelle/python/testdata/py312_syntax/README.md @@ -0,0 +1,4 @@ +# py312 syntax + +This test case checks that we properly parse certain python 3.12 syntax, such +as pep 695 type parameters, with go-tree-sitter. diff --git a/gazelle/python/testdata/py312_syntax/WORKSPACE b/gazelle/python/testdata/py312_syntax/WORKSPACE new file mode 100644 index 0000000000..faff6af87a --- /dev/null +++ b/gazelle/python/testdata/py312_syntax/WORKSPACE @@ -0,0 +1 @@ +# This is a Bazel workspace for the Gazelle test data. diff --git a/gazelle/python/testdata/py312_syntax/__init__.py b/gazelle/python/testdata/py312_syntax/__init__.py new file mode 100644 index 0000000000..e69de29bb2 diff --git a/gazelle/python/testdata/py312_syntax/_other_module.py b/gazelle/python/testdata/py312_syntax/_other_module.py new file mode 100644 index 0000000000..e69de29bb2 diff --git a/gazelle/python/testdata/py312_syntax/pep_695_type_parameter.py b/gazelle/python/testdata/py312_syntax/pep_695_type_parameter.py new file mode 100644 index 0000000000..eff06de5a7 --- /dev/null +++ b/gazelle/python/testdata/py312_syntax/pep_695_type_parameter.py @@ -0,0 +1,22 @@ +def search_one_more_level[T]( + graph: dict[T, set[T]], seen: set[T], routes: list[list[T]], target: T +) -> list[T] | None: + """This function fails to parse with older versions of go-tree-sitter. + + Args: + graph: The graph to search as input. + seen: The nodes that have been visited as input/output. + routes: The current routes in the breadth-first search as input/output. + target: The target to search in this extra search level. + + Returns: + a route if it ends on the target, or None if no route reaches the + target. + """ + + +import _other_module + + +if __name__ == "__main__": + pass diff --git a/gazelle/python/testdata/py312_syntax/test.yaml b/gazelle/python/testdata/py312_syntax/test.yaml new file mode 100644 index 0000000000..ed97d539c0 --- /dev/null +++ b/gazelle/python/testdata/py312_syntax/test.yaml @@ -0,0 +1 @@ +--- From 727ab43107fb0b2d528140f609b873670a5c6c26 Mon Sep 17 00:00:00 2001 From: Garrett Holmstrom Date: Sat, 14 Dec 2024 17:46:23 -0800 Subject: [PATCH 029/922] fix(pypi): Fix use_hub_alias_dependencies with WORKSPACE (#2504) The code path pip_parse follows when using a WORKSPACE file with use_hub_alias_dependencies enabled forgets to pass requirement cycles along to alias creation, leading to the _groups package never being created and aliases skipping them. Requirement cycles are just ignored entirely. In this patch we attempt to fix that so grouping works more or less the same way as it does under bzlmod with that flag enabled. --- CHANGELOG.md | 3 +++ python/private/pypi/pip_repository.bzl | 1 + 2 files changed, 4 insertions(+) diff --git a/CHANGELOG.md b/CHANGELOG.md index 33719187f7..5583399e96 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -67,6 +67,9 @@ Unreleased changes template. Fixes [#2450](https://github.com/bazelbuild/rules_python/issues/2450). * (gazelle) Gazelle will now correctly parse Python3.12 files that use [PEP 695 Type Parameter Syntax][pep-695]. (#2396) +* (pypi) Using {bzl:obj}`pip_parse.experimental_requirement_cycles` and + {bzl:obj}`pip_parse.use_hub_alias_dependencies` together now works when + using WORKSPACE files. [pep-695]: https://peps.python.org/pep-0695/ diff --git a/python/private/pypi/pip_repository.bzl b/python/private/pypi/pip_repository.bzl index 4591591dc9..029566eea3 100644 --- a/python/private/pypi/pip_repository.bzl +++ b/python/private/pypi/pip_repository.bzl @@ -178,6 +178,7 @@ def _pip_repository_impl(rctx): for pkg in bzl_packages or [] }, extra_hub_aliases = rctx.attr.extra_hub_aliases, + requirement_cycles = requirement_cycles, ) for path, contents in aliases.items(): rctx.file(path, contents) From 95fe03a3590f06c6b9d8a9c12d956b44a0a5de0b Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Tue, 17 Dec 2024 17:16:35 +0900 Subject: [PATCH 030/922] build(deps): bump astroid from 3.3.5 to 3.3.6 in /docs (#2490) Bumps [astroid](https://github.com/pylint-dev/astroid) from 3.3.5 to 3.3.6.
Changelog

Sourced from astroid's changelog.

What's New in astroid 3.3.6?

Release date: 2024-12-08

  • Fix inability to import collections.abc in python 3.13.1.

    Closes pylint-dev/pylint#10112

  • Fix crash when typing._alias() call is missing arguments.

    Closes #2513

Commits
  • a132679 Bump astroid to 3.3.6, update changelog
  • 0834156 Add compatibility with python 3.13.1 (#2647) (#2649)
  • 80ce031 [Backport maintenance/3.3.x] Fix IndexError when typing._alias() has missing ...
  • See full diff in compare view

[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=astroid&package-manager=pip&previous-version=3.3.5&new-version=3.3.6)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) ---
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot show ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Richard Levasseur --- docs/requirements.txt | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/requirements.txt b/docs/requirements.txt index 2e306cd5b5..bc9b3b411b 100644 --- a/docs/requirements.txt +++ b/docs/requirements.txt @@ -10,9 +10,9 @@ alabaster==1.0.0 \ --hash=sha256:c00dca57bca26fa62a6d7d0a9fcce65f3e026e9bfe33e9c538fd3fbb2144fd9e \ --hash=sha256:fc6786402dc3fcb2de3cabd5fe455a2db534b371124f1f21de8731783dec828b # via sphinx -astroid==3.3.5 \ - --hash=sha256:5cfc40ae9f68311075d27ef68a4841bdc5cc7f6cf86671b49f00607d30188e2d \ - --hash=sha256:a9d1c946ada25098d790e079ba2a1b112157278f3fb7e718ae6a9252f5835dc8 +astroid==3.3.6 \ + --hash=sha256:6aaea045f938c735ead292204afdb977a36e989522b7833ef6fea94de743f442 \ + --hash=sha256:db676dc4f3ae6bfe31cda227dc60e03438378d7a896aec57422c95634e8d722f # via sphinx-autodoc2 babel==2.16.0 \ --hash=sha256:368b5b98b37c06b7daf6696391c3240c938b37767d4584413e8438c5c435fa8b \ From 930639335363b7673630a459d1925f341c24fd8c Mon Sep 17 00:00:00 2001 From: Nicholas Junge Date: Wed, 18 Dec 2024 18:22:22 +0100 Subject: [PATCH 031/922] docs: Fix toolchain implementation file name (#2512) The code example comment uses `toolchain_impls` (extra s), while the target references use `toolchain_impl` (no s). This made me go back and double-check when reading the custom toolchain example (especially the definitions in L425ff). Remove the extra "s" in the comment so it matches what the target references are. --- docs/toolchains.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/toolchains.md b/docs/toolchains.md index db4c6ba07a..4cc0948a8b 100644 --- a/docs/toolchains.md +++ b/docs/toolchains.md @@ -376,7 +376,7 @@ Here, we show an example for a semi-complicated toolchain suite, one that is: Defining toolchains for this might look something like this: ``` -# File: toolchain_impls/BUILD +# File: toolchain_impl/BUILD load("@rules_python//python:py_cc_toolchain.bzl", "py_cc_toolchain") load("@rules_python//python:py_exec_tools_toolchain.bzl", "py_exec_tools_toolchain") load("@rules_python//python:py_runtime.bzl", "py_runtime") From 66a8b5b595710bd107c31ad5d449593536effb76 Mon Sep 17 00:00:00 2001 From: Ignas Anikevicius <240938+aignas@users.noreply.github.com> Date: Thu, 19 Dec 2024 09:27:17 +0900 Subject: [PATCH 032/922] doc: correct toolchain usage in repository_rule context docs (#2510) This clarifies a few points about how to use the python interpreter in the repository context. It also fixes outdated documentation as noted by some users of 1.0. See discussion: https://github.com/bazelbuild/rules_python/discussions/2509 Fixes #2494 --------- Co-authored-by: Richard Levasseur --- docs/toolchains.md | 41 +++++++++++++++++++++++++-- examples/pip_parse_vendored/README.md | 3 +- 2 files changed, 39 insertions(+), 5 deletions(-) diff --git a/docs/toolchains.md b/docs/toolchains.md index 4cc0948a8b..2a59169752 100644 --- a/docs/toolchains.md +++ b/docs/toolchains.md @@ -184,6 +184,43 @@ existing attributes: * Adding additional Python versions via {bzl:obj}`python.single_version_override` or {bzl:obj}`python.single_version_platform_override`. +### Using defined toolchains from WORKSPACE + +It is possible to use toolchains defined in `MODULE.bazel` in `WORKSPACE`. For example +the following `MODULE.bazel` and `WORKSPACE` provides a working {bzl:obj}`pip_parse` setup: +```starlark +# File: WORKSPACE +load("@rules_python//python:repositories.bzl", "py_repositories") + +py_repositories() + +load("@rules_python//python:pip.bzl", "pip_parse") + +pip_parse( + name = "third_party", + requirements_lock = "//:requirements.txt", + python_interpreter_target = "@python_3_10_host//:python", +) + +load("@third_party//:requirements.bzl", "install_deps") + +install_deps() + +# File: MODULE.bazel +bazel_dep(name = "rules_python", version = "0.40.0") + +python = use_extension("@rules_python//python/extensions:python.bzl", "python") + +python.toolchain(is_default = True, python_version = "3.10") + +use_repo(python, "python_3_10", "python_3_10_host") +``` + +Note, the user has to import the `*_host` repository to use the python interpreter in the +{bzl:obj}`pip_parse` and {bzl:obj}`whl_library` repository rules and once that is done +users should be able to ensure the setting of the default toolchain even during the +transition period when some of the code is still defined in `WORKSPACE`. + ## Workspace configuration To import rules_python in your project, you first need to add it to your @@ -229,13 +266,11 @@ python_register_toolchains( python_version = "3.11", ) -load("@python_3_11//:defs.bzl", "interpreter") - load("@rules_python//python:pip.bzl", "pip_parse") pip_parse( ... - python_interpreter_target = interpreter, + python_interpreter_target = "@python_3_11_host//:python", ... ) ``` diff --git a/examples/pip_parse_vendored/README.md b/examples/pip_parse_vendored/README.md index f53260a175..fdf040c8e5 100644 --- a/examples/pip_parse_vendored/README.md +++ b/examples/pip_parse_vendored/README.md @@ -20,12 +20,11 @@ python_register_toolchains( name = "python39", python_version = "3.9", ) -load("@python39//:defs.bzl", "interpreter") # Load dependencies vendored by some other ruleset. load("@some_rules//:py_deps.bzl", "install_deps") install_deps( - python_interpreter_target = interpreter, + python_interpreter_target = "@python39_host//:python", ) ``` From e3c940681a38131a491263d721f14bd8fe528273 Mon Sep 17 00:00:00 2001 From: Ted Pudlik Date: Sat, 21 Dec 2024 19:40:09 -0800 Subject: [PATCH 033/922] fix: py_proto_library: external runfiles (#2516) Previously, the import path within the runfiles was only correct for the case --legacy_external_runfiles=True (which copied the runfiles into `$RUNFILES/
/external//` in addition to `$RUNFILES//`. This flag was flipped to False in Bazel 8.0.0. Fixes https://github.com/bazelbuild/rules_python/issues/2515. Tested locally against the minimal reproducer in that issue. --- .bazelignore | 1 + CHANGELOG.md | 1 + examples/bzlmod/.bazelignore | 1 + examples/bzlmod/MODULE.bazel | 7 ++++++ examples/bzlmod/py_proto_library/BUILD.bazel | 16 ++++++++++++++ .../py_proto_library/foo_external/BUILD.bazel | 22 +++++++++++++++++++ .../foo_external/MODULE.bazel | 8 +++++++ .../py_proto_library/foo_external/WORKSPACE | 0 .../foo_external/nested/foo/my_proto.proto | 6 +++++ .../foo_external/py_binary_with_proto.py | 5 +++++ python/private/proto/py_proto_library.bzl | 7 +++++- 11 files changed, 73 insertions(+), 1 deletion(-) create mode 100644 examples/bzlmod/py_proto_library/foo_external/BUILD.bazel create mode 100644 examples/bzlmod/py_proto_library/foo_external/MODULE.bazel create mode 100644 examples/bzlmod/py_proto_library/foo_external/WORKSPACE create mode 100644 examples/bzlmod/py_proto_library/foo_external/nested/foo/my_proto.proto create mode 100644 examples/bzlmod/py_proto_library/foo_external/py_binary_with_proto.py diff --git a/.bazelignore b/.bazelignore index 60d680e9f0..e10af2035d 100644 --- a/.bazelignore +++ b/.bazelignore @@ -18,6 +18,7 @@ examples/bzlmod/other_module/bazel-bin examples/bzlmod/other_module/bazel-other_module examples/bzlmod/other_module/bazel-out examples/bzlmod/other_module/bazel-testlogs +examples/bzlmod/py_proto_library/foo_external examples/bzlmod_build_file_generation/bazel-bzlmod_build_file_generation examples/multi_python_versions/bazel-multi_python_versions examples/pip_parse/bazel-pip_parse diff --git a/CHANGELOG.md b/CHANGELOG.md index 5583399e96..9976d2027c 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -70,6 +70,7 @@ Unreleased changes template. * (pypi) Using {bzl:obj}`pip_parse.experimental_requirement_cycles` and {bzl:obj}`pip_parse.use_hub_alias_dependencies` together now works when using WORKSPACE files. +* (py_proto_library) Fix import paths in Bazel 8. [pep-695]: https://peps.python.org/pep-0695/ diff --git a/examples/bzlmod/.bazelignore b/examples/bzlmod/.bazelignore index ab3eb1635c..3927f8e910 100644 --- a/examples/bzlmod/.bazelignore +++ b/examples/bzlmod/.bazelignore @@ -1 +1,2 @@ other_module +py_proto_library/foo_external diff --git a/examples/bzlmod/MODULE.bazel b/examples/bzlmod/MODULE.bazel index 0a31c3beb8..536e3b2b67 100644 --- a/examples/bzlmod/MODULE.bazel +++ b/examples/bzlmod/MODULE.bazel @@ -5,6 +5,7 @@ module( ) bazel_dep(name = "bazel_skylib", version = "1.7.1") +bazel_dep(name = "platforms", version = "0.0.4") bazel_dep(name = "rules_python", version = "0.0.0") local_path_override( module_name = "rules_python", @@ -272,5 +273,11 @@ local_path_override( path = "other_module", ) +bazel_dep(name = "foo_external", version = "") +local_path_override( + module_name = "foo_external", + path = "py_proto_library/foo_external", +) + # example test dependencies bazel_dep(name = "rules_shell", version = "0.3.0", dev_dependency = True) diff --git a/examples/bzlmod/py_proto_library/BUILD.bazel b/examples/bzlmod/py_proto_library/BUILD.bazel index d0bc683021..24436b48ea 100644 --- a/examples/bzlmod/py_proto_library/BUILD.bazel +++ b/examples/bzlmod/py_proto_library/BUILD.bazel @@ -1,3 +1,4 @@ +load("@bazel_skylib//rules:native_binary.bzl", "native_test") load("@rules_python//python:py_test.bzl", "py_test") py_test( @@ -16,3 +17,18 @@ py_test( "//py_proto_library/example.com/another_proto:message_proto_py_pb2", ], ) + +# Regression test for https://github.com/bazelbuild/rules_python/issues/2515 +# +# This test failed before https://github.com/bazelbuild/rules_python/pull/2516 +# when ran with --legacy_external_runfiles=False (default in Bazel 8.0.0). +native_test( + name = "external_import_test", + src = "@foo_external//:py_binary_with_proto", + # Incompatible with Windows: native_test wrapping a py_binary doesn't work + # on Windows. + target_compatible_with = select({ + "@platforms//os:windows": ["@platforms//:incompatible"], + "//conditions:default": [], + }), +) diff --git a/examples/bzlmod/py_proto_library/foo_external/BUILD.bazel b/examples/bzlmod/py_proto_library/foo_external/BUILD.bazel new file mode 100644 index 0000000000..3fa22e06e7 --- /dev/null +++ b/examples/bzlmod/py_proto_library/foo_external/BUILD.bazel @@ -0,0 +1,22 @@ +load("@rules_proto//proto:defs.bzl", "proto_library") +load("@rules_python//python:proto.bzl", "py_proto_library") +load("@rules_python//python:py_binary.bzl", "py_binary") + +package(default_visibility = ["//visibility:public"]) + +proto_library( + name = "proto_lib", + srcs = ["nested/foo/my_proto.proto"], + strip_import_prefix = "/nested/foo", +) + +py_proto_library( + name = "a_proto", + deps = [":proto_lib"], +) + +py_binary( + name = "py_binary_with_proto", + srcs = ["py_binary_with_proto.py"], + deps = [":a_proto"], +) diff --git a/examples/bzlmod/py_proto_library/foo_external/MODULE.bazel b/examples/bzlmod/py_proto_library/foo_external/MODULE.bazel new file mode 100644 index 0000000000..5063f9b2d1 --- /dev/null +++ b/examples/bzlmod/py_proto_library/foo_external/MODULE.bazel @@ -0,0 +1,8 @@ +module( + name = "foo_external", + version = "0.0.1", +) + +bazel_dep(name = "rules_python", version = "1.0.0") +bazel_dep(name = "protobuf", version = "28.2", repo_name = "com_google_protobuf") +bazel_dep(name = "rules_proto", version = "7.0.2") diff --git a/examples/bzlmod/py_proto_library/foo_external/WORKSPACE b/examples/bzlmod/py_proto_library/foo_external/WORKSPACE new file mode 100644 index 0000000000..e69de29bb2 diff --git a/examples/bzlmod/py_proto_library/foo_external/nested/foo/my_proto.proto b/examples/bzlmod/py_proto_library/foo_external/nested/foo/my_proto.proto new file mode 100644 index 0000000000..7b8440cbed --- /dev/null +++ b/examples/bzlmod/py_proto_library/foo_external/nested/foo/my_proto.proto @@ -0,0 +1,6 @@ +syntax = "proto3"; + +package my_proto; + +message MyMessage { +} diff --git a/examples/bzlmod/py_proto_library/foo_external/py_binary_with_proto.py b/examples/bzlmod/py_proto_library/foo_external/py_binary_with_proto.py new file mode 100644 index 0000000000..be34264b5a --- /dev/null +++ b/examples/bzlmod/py_proto_library/foo_external/py_binary_with_proto.py @@ -0,0 +1,5 @@ +import sys + +if __name__ == "__main__": + import my_proto_pb2 + sys.exit(0) diff --git a/python/private/proto/py_proto_library.bzl b/python/private/proto/py_proto_library.bzl index d810e58c24..1e9df848ab 100644 --- a/python/private/proto/py_proto_library.bzl +++ b/python/private/proto/py_proto_library.bzl @@ -98,7 +98,12 @@ def _py_proto_aspect_impl(target, ctx): proto_root = proto_root[len(ctx.bin_dir.path) + 1:] plugin_output = ctx.bin_dir.path + "/" + proto_root - proto_root = ctx.workspace_name + "/" + proto_root + + # Import path within the runfiles tree + if proto_root.startswith("external/"): + proto_root = proto_root[len("external") + 1:] + else: + proto_root = ctx.workspace_name + "/" + proto_root proto_common.compile( actions = ctx.actions, From be950f9c2448f85332322fd4f7918b940bf45bfd Mon Sep 17 00:00:00 2001 From: Ignas Anikevicius <240938+aignas@users.noreply.github.com> Date: Mon, 23 Dec 2024 21:44:54 +0900 Subject: [PATCH 034/922] refactor(pypi): A better error message when the wheel select hits no_match (#2519) With this change we get the current values of the python configuration values printed in addition to the message printed previously. This should help us advise users who don't have their builds configured correctly. We are adding an extra `build_setting` which we can set in order to get an error message instead of a `DEBUG` warning. This has been documented as part of our config settings and in the `no_match_error` in the `select` statement. Example output now ```console $ bazel cquery --@rules_python//python/config_settings:python_version=3.12 @dev_pip//sphinx DEBUG: /home/aignas/src/github/aignas/rules_python/python/private/config_settings.bzl:193:14: The current configuration rules_python config flags is: @@//python/config_settings:pip_whl: "auto" @@//python/config_settings:pip_whl_glibc_version: "" @@//python/config_settings:pip_whl_muslc_version: "" @@//python/config_settings:pip_whl_osx_arch: "arch" @@//python/config_settings:pip_whl_osx_version: "" @@//python/config_settings:py_freethreaded: "no" @@//python/config_settings:py_linux_libc: "glibc" @@//python/config_settings:python_version: "3.12" If the value is missing, then the default value is being used, see documentation: https://rules-python.readthedocs.io/en/latest/api/rules_python/python/config_settings ERROR: /home/aignas/.cache/bazel/_bazel_aignas/6f0de8c9128ee8d5dbf27ba6dcc48bdd/external/+pip+dev_pip/sphinx/BUILD.bazel:6:12: configurable attribute "actual" in @@+pip+dev_pip//sphinx:_no_matching_repository doesn't match this configuration: No matching wheel for current configuration's Python version. The current build configuration's Python version doesn't match any of the Python wheels available for this distribution. This distribution supports the following Python configuration settings: //_config:is_cp3.11_py3_none_any //_config:is_cp3.13_py3_none_any To determine the current configuration's Python version, run: `bazel config ` (shown further below) For the current configuration value see the debug message above that is printing the current flag values. If you can't see the message, then re-run the build to make it a failure instead by running the build with: --@@//python/config_settings:current_config=fail However, the command above will hide the `bazel config ` message. This instance of @@+pip+dev_pip//sphinx:_no_matching_repository has configuration identifier 29ffcf8. To inspect its configuration, run: bazel config 29ffcf8. For more help, see https://bazel.build/docs/configurable-attributes#faq-select-choose-condition. ERROR: Analysis of target '@@+pip+dev_pip//sphinx:sphinx' failed; build aborted: Analysis failed INFO: Elapsed time: 0.112s INFO: 0 processes. ERROR: Build did NOT complete successfully ``` Fixes #2466 --------- Co-authored-by: Richard Levasseur --- CHANGELOG.md | 3 + .../python/config_settings/index.md | 18 +++++ python/config_settings/BUILD.bazel | 9 +++ python/private/config_settings.bzl | 73 ++++++++++++++++++- python/private/pypi/pkg_aliases.bzl | 69 +++++++++--------- tests/pypi/pkg_aliases/pkg_aliases_test.bzl | 66 ++++++++++------- 6 files changed, 175 insertions(+), 63 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 9976d2027c..9a3436487e 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -70,6 +70,9 @@ Unreleased changes template. * (pypi) Using {bzl:obj}`pip_parse.experimental_requirement_cycles` and {bzl:obj}`pip_parse.use_hub_alias_dependencies` together now works when using WORKSPACE files. +* (pypi) The error messages when the wheel distributions do not match anything + are now printing more details and include the currently active flag + values. Fixes [#2466](https://github.com/bazelbuild/rules_python/issues/2466). * (py_proto_library) Fix import paths in Bazel 8. [pep-695]: https://peps.python.org/pep-0695/ diff --git a/docs/api/rules_python/python/config_settings/index.md b/docs/api/rules_python/python/config_settings/index.md index ef829bab76..793f6e08fd 100644 --- a/docs/api/rules_python/python/config_settings/index.md +++ b/docs/api/rules_python/python/config_settings/index.md @@ -240,3 +240,21 @@ instead. ::: :::: + +::::{bzl:flag} current_config +Fail the build if the current build configuration does not match the +{obj}`pip.parse` defined wheels. + +Values: +* `fail`: Will fail in the build action ensuring that we get the error + message no matter the action cache. +* ``: (empty string) The default value, that will just print a warning. + +:::{seealso} +{obj}`pip.parse` +::: + +:::{versionadded} 1.1.0 +::: + +:::: diff --git a/python/config_settings/BUILD.bazel b/python/config_settings/BUILD.bazel index 5455f5aef7..fcebcd76dc 100644 --- a/python/config_settings/BUILD.bazel +++ b/python/config_settings/BUILD.bazel @@ -29,6 +29,15 @@ filegroup( construct_config_settings( name = "construct_config_settings", default_version = DEFAULT_PYTHON_VERSION, + documented_flags = [ + ":pip_whl", + ":pip_whl_glibc_version", + ":pip_whl_muslc_version", + ":pip_whl_osx_arch", + ":pip_whl_osx_version", + ":py_freethreaded", + ":py_linux_libc", + ], minor_mapping = MINOR_MAPPING, versions = PYTHON_VERSIONS, ) diff --git a/python/private/config_settings.bzl b/python/private/config_settings.bzl index 10b4d686a7..e5f9d865d1 100644 --- a/python/private/config_settings.bzl +++ b/python/private/config_settings.bzl @@ -17,12 +17,21 @@ load("@bazel_skylib//lib:selects.bzl", "selects") load("@bazel_skylib//rules:common_settings.bzl", "BuildSettingInfo") +load("//python/private:text_util.bzl", "render") load(":semver.bzl", "semver") _PYTHON_VERSION_FLAG = Label("//python/config_settings:python_version") _PYTHON_VERSION_MAJOR_MINOR_FLAG = Label("//python/config_settings:python_version_major_minor") -def construct_config_settings(*, name, default_version, versions, minor_mapping): # buildifier: disable=function-docstring +_DEBUG_ENV_MESSAGE_TEMPLATE = """\ +The current configuration rules_python config flags is: + {flags} + +If the value is missing, then the default value is being used, see documentation: +{docs_url}/python/config_settings +""" + +def construct_config_settings(*, name, default_version, versions, minor_mapping, documented_flags): # buildifier: disable=function-docstring """Create a 'python_version' config flag and construct all config settings used in rules_python. This mainly includes the targets that are used in the toolchain and pip hub @@ -33,6 +42,8 @@ def construct_config_settings(*, name, default_version, versions, minor_mapping) default_version: {type}`str` the default value for the `python_version` flag. versions: {type}`list[str]` A list of versions to build constraint settings for. minor_mapping: {type}`dict[str, str]` A mapping from `X.Y` to `X.Y.Z` python versions. + documented_flags: {type}`list[str]` The labels of the documented settings + that affect build configuration. """ _ = name # @unused _python_version_flag( @@ -101,6 +112,25 @@ def construct_config_settings(*, name, default_version, versions, minor_mapping) visibility = ["//visibility:public"], ) + _current_config( + name = "current_config", + build_setting_default = "", + settings = documented_flags + [_PYTHON_VERSION_FLAG.name], + visibility = ["//visibility:private"], + ) + native.config_setting( + name = "is_not_matching_current_config", + # We use the rule above instead of @platforms//:incompatible so that the + # printing of the current env always happens when the _current_config rule + # is executed. + # + # NOTE: This should in practise only happen if there is a missing compatible + # `whl_library` in the hub repo created by `pip.parse`. + flag_values = {"current_config": "will-never-match"}, + # Only public so that PyPI hub repo can access it + visibility = ["//visibility:public"], + ) + def _python_version_flag_impl(ctx): value = ctx.build_setting_value return [ @@ -122,7 +152,7 @@ _python_version_flag = rule( ) def _python_version_major_minor_flag_impl(ctx): - input = ctx.attr._python_version_flag[config_common.FeatureFlagInfo].value + input = _flag_value(ctx.attr._python_version_flag) if input: version = semver(input) value = "{}.{}".format(version.major, version.minor) @@ -140,3 +170,42 @@ _python_version_major_minor_flag = rule( ), }, ) + +def _flag_value(s): + if config_common.FeatureFlagInfo in s: + return s[config_common.FeatureFlagInfo].value + else: + return s[BuildSettingInfo].value + +def _print_current_config_impl(ctx): + flags = "\n".join([ + "{}: \"{}\"".format(k, v) + for k, v in sorted({ + str(setting.label): _flag_value(setting) + for setting in ctx.attr.settings + }.items()) + ]) + + msg = ctx.attr._template.format( + docs_url = "https://rules-python.readthedocs.io/en/latest/api/rules_python", + flags = render.indent(flags).lstrip(), + ) + if ctx.build_setting_value and ctx.build_setting_value != "fail": + fail("Only 'fail' and empty build setting values are allowed for {}".format( + str(ctx.label), + )) + elif ctx.build_setting_value: + fail(msg) + else: + print(msg) # buildifier: disable=print + + return [config_common.FeatureFlagInfo(value = "")] + +_current_config = rule( + implementation = _print_current_config_impl, + build_setting = config.string(flag = True), + attrs = { + "settings": attr.label_list(mandatory = True), + "_template": attr.string(default = _DEBUG_ENV_MESSAGE_TEMPLATE), + }, +) diff --git a/python/private/pypi/pkg_aliases.bzl b/python/private/pypi/pkg_aliases.bzl index a6872fdce9..980921b474 100644 --- a/python/private/pypi/pkg_aliases.bzl +++ b/python/private/pypi/pkg_aliases.bzl @@ -36,8 +36,6 @@ load(":whl_target_platforms.bzl", "whl_target_platforms") # it. It is more of an internal consistency check. _VERSION_NONE = (0, 0) -_CONFIG_SETTINGS_PKG = str(Label("//python/config_settings:BUILD.bazel")).partition(":")[0] - _NO_MATCH_ERROR_TEMPLATE = """\ No matching wheel for current configuration's Python version. @@ -49,37 +47,18 @@ configuration settings: To determine the current configuration's Python version, run: `bazel config ` (shown further below) -and look for one of: - {settings_pkg}:python_version - {settings_pkg}:pip_whl - {settings_pkg}:pip_whl_glibc_version - {settings_pkg}:pip_whl_muslc_version - {settings_pkg}:pip_whl_osx_arch - {settings_pkg}:pip_whl_osx_version - {settings_pkg}:py_freethreaded - {settings_pkg}:py_linux_libc - -If the value is missing, then the default value is being used, see documentation: -{docs_url}/python/config_settings""" - -def _no_match_error(actual): - if type(actual) != type({}): - return None - - if "//conditions:default" in actual: - return None - - return _NO_MATCH_ERROR_TEMPLATE.format( - config_settings = render.indent( - "\n".join(sorted([ - value - for key in actual - for value in (key if type(key) == "tuple" else [key]) - ])), - ).lstrip(), - settings_pkg = _CONFIG_SETTINGS_PKG, - docs_url = "https://rules-python.readthedocs.io/en/latest/api/rules_python", - ) +For the current configuration value see the debug message above that is +printing the current flag values. If you can't see the message, then re-run the +build to make it a failure instead by running the build with: + --{current_flags}=fail + +However, the command above will hide the `bazel config ` message. +""" + +_LABEL_NONE = Label("//python:none") +_LABEL_CURRENT_CONFIG = Label("//python/config_settings:current_config") +_LABEL_CURRENT_CONFIG_NO_MATCH = Label("//python/config_settings:is_not_matching_current_config") +_INCOMPATIBLE = "_no_matching_repository" def pkg_aliases( *, @@ -120,7 +99,26 @@ def pkg_aliases( } actual = multiplatform_whl_aliases(aliases = actual, **kwargs) - no_match_error = _no_match_error(actual) + if type(actual) == type({}) and "//conditions:default" not in actual: + native.alias( + name = _INCOMPATIBLE, + actual = select( + {_LABEL_CURRENT_CONFIG_NO_MATCH: _LABEL_NONE}, + no_match_error = _NO_MATCH_ERROR_TEMPLATE.format( + config_settings = render.indent( + "\n".join(sorted([ + value + for key in actual + for value in (key if type(key) == "tuple" else [key]) + ])), + ).lstrip(), + current_flags = str(_LABEL_CURRENT_CONFIG), + ), + ), + visibility = ["//visibility:private"], + tags = ["manual"], + ) + actual["//conditions:default"] = _INCOMPATIBLE for name, target_name in target_names.items(): if type(actual) == type(""): @@ -134,10 +132,9 @@ def pkg_aliases( v: "@{repo}//:{target_name}".format( repo = repo, target_name = name, - ) + ) if repo != _INCOMPATIBLE else repo for v, repo in actual.items() }, - no_match_error = no_match_error, ) else: fail("The `actual` arg must be a dictionary or a string") diff --git a/tests/pypi/pkg_aliases/pkg_aliases_test.bzl b/tests/pypi/pkg_aliases/pkg_aliases_test.bzl index 23a0f01db9..f13b62f13d 100644 --- a/tests/pypi/pkg_aliases/pkg_aliases_test.bzl +++ b/tests/pypi/pkg_aliases/pkg_aliases_test.bzl @@ -56,12 +56,8 @@ def _test_config_setting_aliases(env): actual_no_match_error = [] def mock_select(value, no_match_error = None): - actual_no_match_error.append(no_match_error) - env.expect.that_str(no_match_error).contains("""\ -configuration settings: - //:my_config_setting - -""") + if no_match_error and no_match_error not in actual_no_match_error: + actual_no_match_error.append(no_match_error) return value pkg_aliases( @@ -71,7 +67,7 @@ configuration settings: }, extra_aliases = ["my_special"], native = struct( - alias = lambda name, actual: got.update({name: actual}), + alias = lambda *, name, actual, visibility = None, tags = None: got.update({name: actual}), ), select = mock_select, ) @@ -80,9 +76,22 @@ configuration settings: want = { "pkg": { "//:my_config_setting": "@bar_baz_repo//:pkg", + "//conditions:default": "_no_matching_repository", }, + # This will be printing the current config values and will make sure we + # have an error. + "_no_matching_repository": {Label("//python/config_settings:is_not_matching_current_config"): Label("//python:none")}, } env.expect.that_dict(got).contains_at_least(want) + env.expect.that_collection(actual_no_match_error).has_size(1) + env.expect.that_str(actual_no_match_error[0]).contains("""\ +configuration settings: + //:my_config_setting + +""") + env.expect.that_str(actual_no_match_error[0]).contains( + "//python/config_settings:current_config=fail", + ) _tests.append(_test_config_setting_aliases) @@ -92,13 +101,8 @@ def _test_config_setting_aliases_many(env): actual_no_match_error = [] def mock_select(value, no_match_error = None): - actual_no_match_error.append(no_match_error) - env.expect.that_str(no_match_error).contains("""\ -configuration settings: - //:another_config_setting - //:my_config_setting - //:third_config_setting -""") + if no_match_error and no_match_error not in actual_no_match_error: + actual_no_match_error.append(no_match_error) return value pkg_aliases( @@ -112,7 +116,8 @@ configuration settings: }, extra_aliases = ["my_special"], native = struct( - alias = lambda name, actual: got.update({name: actual}), + alias = lambda *, name, actual, visibility = None, tags = None: got.update({name: actual}), + config_setting = lambda **_: None, ), select = mock_select, ) @@ -125,9 +130,17 @@ configuration settings: "//:another_config_setting", ): "@bar_baz_repo//:my_special", "//:third_config_setting": "@foo_repo//:my_special", + "//conditions:default": "_no_matching_repository", }, } env.expect.that_dict(got).contains_at_least(want) + env.expect.that_collection(actual_no_match_error).has_size(1) + env.expect.that_str(actual_no_match_error[0]).contains("""\ +configuration settings: + //:another_config_setting + //:my_config_setting + //:third_config_setting +""") _tests.append(_test_config_setting_aliases_many) @@ -137,15 +150,8 @@ def _test_multiplatform_whl_aliases(env): actual_no_match_error = [] def mock_select(value, no_match_error = None): - actual_no_match_error.append(no_match_error) - env.expect.that_str(no_match_error).contains("""\ -configuration settings: - //:my_config_setting - //_config:is_cp3.9_linux_x86_64 - //_config:is_cp3.9_py3_none_any - //_config:is_cp3.9_py3_none_any_linux_x86_64 - -""") + if no_match_error and no_match_error not in actual_no_match_error: + actual_no_match_error.append(no_match_error) return value pkg_aliases( @@ -168,7 +174,7 @@ configuration settings: }, extra_aliases = [], native = struct( - alias = lambda name, actual: got.update({name: actual}), + alias = lambda *, name, actual, visibility = None, tags = None: got.update({name: actual}), ), select = mock_select, glibc_versions = [], @@ -183,9 +189,19 @@ configuration settings: "//_config:is_cp3.9_linux_x86_64": "@bzlmod_repo_for_a_particular_platform//:pkg", "//_config:is_cp3.9_py3_none_any": "@filename_repo//:pkg", "//_config:is_cp3.9_py3_none_any_linux_x86_64": "@filename_repo_for_platform//:pkg", + "//conditions:default": "_no_matching_repository", }, } env.expect.that_dict(got).contains_at_least(want) + env.expect.that_collection(actual_no_match_error).has_size(1) + env.expect.that_str(actual_no_match_error[0]).contains("""\ +configuration settings: + //:my_config_setting + //_config:is_cp3.9_linux_x86_64 + //_config:is_cp3.9_py3_none_any + //_config:is_cp3.9_py3_none_any_linux_x86_64 + +""") _tests.append(_test_multiplatform_whl_aliases) From 026b300d918ced0f4e9f99a22ab8407656ed20ac Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Mon, 23 Dec 2024 13:27:03 -0800 Subject: [PATCH 035/922] refactor: consolidate py_executable_bazel, common_bazel (#2523) This furthers the work of removing the artificial split of code that stemmed from when the implementation was part of Bazel itself. Summary of changes: * Move most of `py_executable_bazel.bzl` into `py_executable.bzl` * Move most of `common_bazel.bzl` into `common.bzl` * Create `precompile.bzl` for the precompile helpers. This is to avoid a circular dependency between common.bzl and attributes.bzl. Work towards https://github.com/bazelbuild/rules_python/issues/2522 --- python/private/BUILD.bazel | 54 +- python/private/common.bzl | 61 +- .../{common_bazel.bzl => precompile.bzl} | 73 -- python/private/py_binary_macro.bzl | 2 +- python/private/py_binary_rule.bzl | 6 +- python/private/py_executable.bzl | 752 ++++++++++++++++- python/private/py_executable_bazel.bzl | 772 ------------------ python/private/py_library_rule.bzl | 4 +- python/private/py_test_macro.bzl | 2 +- python/private/py_test_rule.bzl | 6 +- .../venv_relative_path_tests.bzl | 2 +- 11 files changed, 843 insertions(+), 891 deletions(-) rename python/private/{common_bazel.bzl => precompile.bzl} (78%) delete mode 100644 python/private/py_executable_bazel.bzl diff --git a/python/private/BUILD.bazel b/python/private/BUILD.bazel index 76e3a78778..706506a19c 100644 --- a/python/private/BUILD.bazel +++ b/python/private/BUILD.bazel @@ -104,30 +104,18 @@ bzl_library( deps = [":py_internal_bzl"], ) -bzl_library( - name = "common_bazel_bzl", - srcs = ["common_bazel.bzl"], - deps = [ - ":attributes_bzl", - ":common_bzl", - ":py_cc_link_params_info_bzl", - ":py_internal_bzl", - ":py_interpreter_program_bzl", - ":toolchain_types_bzl", - "@bazel_skylib//lib:paths", - ], -) - bzl_library( name = "common_bzl", srcs = ["common.bzl"], deps = [ ":cc_helper_bzl", + ":py_cc_link_params_info_bzl", ":py_info_bzl", ":py_internal_bzl", ":reexports_bzl", ":rules_cc_srcs_bzl", ":semantics_bzl", + "@bazel_skylib//lib:paths", ], ) @@ -199,6 +187,18 @@ bzl_library( srcs = ["normalize_name.bzl"], ) +bzl_library( + name = "precompile_bzl", + srcs = ["precompile.bzl"], + deps = [ + ":attributes_bzl", + ":py_internal_bzl", + ":py_interpreter_program_bzl", + ":toolchain_types_bzl", + "@bazel_skylib//lib:paths", + ], +) + bzl_library( name = "python_bzl", srcs = ["python.bzl"], @@ -265,8 +265,8 @@ bzl_library( name = "py_binary_macro_bzl", srcs = ["py_binary_macro.bzl"], deps = [ - ":common_bzl", ":py_binary_rule_bzl", + ":py_executable_bzl", ], ) @@ -275,7 +275,7 @@ bzl_library( srcs = ["py_binary_rule.bzl"], deps = [ ":attributes_bzl", - ":py_executable_bazel_bzl", + ":py_executable_bzl", ":semantics_bzl", "@bazel_skylib//lib:dicts", ], @@ -343,20 +343,6 @@ bzl_library( ], ) -bzl_library( - name = "py_executable_bazel_bzl", - srcs = ["py_executable_bazel.bzl"], - deps = [ - ":attributes_bzl", - ":common_bazel_bzl", - ":common_bzl", - ":py_executable_bzl", - ":py_internal_bzl", - ":py_runtime_info_bzl", - ":semantics_bzl", - ], -) - bzl_library( name = "py_executable_bzl", srcs = ["py_executable.bzl"], @@ -365,6 +351,7 @@ bzl_library( ":cc_helper_bzl", ":common_bzl", ":flags_bzl", + ":precompile_bzl", ":py_cc_link_params_info_bzl", ":py_executable_info_bzl", ":py_info_bzl", @@ -373,6 +360,7 @@ bzl_library( ":rules_cc_srcs_bzl", ":toolchain_types_bzl", "@bazel_skylib//lib:dicts", + "@bazel_skylib//lib:paths", "@bazel_skylib//lib:structs", "@bazel_skylib//rules:common_settings", ], @@ -431,8 +419,8 @@ bzl_library( name = "py_library_rule_bzl", srcs = ["py_library_rule.bzl"], deps = [ - ":common_bazel_bzl", ":common_bzl", + ":precompile_bzl", ":py_library_bzl", ], ) @@ -508,7 +496,7 @@ bzl_library( name = "py_test_macro_bzl", srcs = ["py_test_macro.bzl"], deps = [ - ":common_bazel_bzl", + ":py_executable_bzl", ":py_test_rule_bzl", ], ) @@ -519,7 +507,7 @@ bzl_library( deps = [ ":attributes_bzl", ":common_bzl", - ":py_executable_bazel_bzl", + ":py_executable_bzl", ":semantics_bzl", "@bazel_skylib//lib:dicts", ], diff --git a/python/private/common.bzl b/python/private/common.bzl index 97fabcebcb..9c285f97bc 100644 --- a/python/private/common.bzl +++ b/python/private/common.bzl @@ -11,9 +11,13 @@ # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # See the License for the specific language governing permissions and # limitations under the License. -"""Various things common to Bazel and Google rule implementations.""" +"""Various things common to rule implementations.""" +load("@bazel_skylib//lib:paths.bzl", "paths") +load("@rules_cc//cc/common:cc_common.bzl", "cc_common") +load("@rules_cc//cc/common:cc_info.bzl", "CcInfo") load(":cc_helper.bzl", "cc_helper") +load(":py_cc_link_params_info.bzl", "PyCcLinkParamsInfo") load(":py_info.bzl", "PyInfo", "PyInfoBuilder") load(":py_internal.bzl", "py_internal") load(":reexports.bzl", "BuiltinPyInfo") @@ -262,6 +266,30 @@ def filter_to_py_srcs(srcs): # as a valid extension. return [f for f in srcs if f.extension == "py"] +def collect_cc_info(ctx, extra_deps = []): + """Collect C++ information from dependencies for Bazel. + + Args: + ctx: Rule ctx; must have `deps` attribute. + extra_deps: list of Target to also collect C+ information from. + + Returns: + CcInfo provider of merged information. + """ + deps = ctx.attr.deps + if extra_deps: + deps = list(deps) + deps.extend(extra_deps) + cc_infos = [] + for dep in deps: + if CcInfo in dep: + cc_infos.append(dep[CcInfo]) + + if PyCcLinkParamsInfo in dep: + cc_infos.append(dep[PyCcLinkParamsInfo].cc_info) + + return cc_common.merge_cc_infos(cc_infos = cc_infos) + def collect_imports(ctx, semantics): """Collect the direct and transitive `imports` strings. @@ -280,6 +308,37 @@ def collect_imports(ctx, semantics): transitive.append(dep[BuiltinPyInfo].imports) return depset(direct = semantics.get_imports(ctx), transitive = transitive) +def get_imports(ctx): + """Gets the imports from a rule's `imports` attribute. + + See create_binary_semantics_struct for details about this function. + + Args: + ctx: Rule ctx. + + Returns: + List of strings. + """ + prefix = "{}/{}".format( + ctx.workspace_name, + py_internal.get_label_repo_runfiles_path(ctx.label), + ) + result = [] + for import_str in ctx.attr.imports: + import_str = ctx.expand_make_variables("imports", import_str, {}) + if import_str.startswith("/"): + continue + + # To prevent "escaping" out of the runfiles tree, we normalize + # the path and ensure it doesn't have up-level references. + import_path = paths.normalize("{}/{}".format(prefix, import_str)) + if import_path.startswith("../") or import_path == "..": + fail("Path '{}' references a path above the execution root".format( + import_str, + )) + result.append(import_path) + return result + def collect_runfiles(ctx, files = depset()): """Collects the necessary files from the rule's context. diff --git a/python/private/common_bazel.bzl b/python/private/precompile.bzl similarity index 78% rename from python/private/common_bazel.bzl rename to python/private/precompile.bzl index efbebd0252..23e8f81426 100644 --- a/python/private/common_bazel.bzl +++ b/python/private/precompile.bzl @@ -13,44 +13,12 @@ # limitations under the License. """Common functions that are specific to Bazel rule implementation""" -load("@bazel_skylib//lib:paths.bzl", "paths") load("@bazel_skylib//rules:common_settings.bzl", "BuildSettingInfo") -load("@rules_cc//cc/common:cc_common.bzl", "cc_common") -load("@rules_cc//cc/common:cc_info.bzl", "CcInfo") load(":attributes.bzl", "PrecompileAttr", "PrecompileInvalidationModeAttr", "PrecompileSourceRetentionAttr") -load(":common.bzl", "is_bool") load(":flags.bzl", "PrecompileFlag") -load(":py_cc_link_params_info.bzl", "PyCcLinkParamsInfo") -load(":py_internal.bzl", "py_internal") load(":py_interpreter_program.bzl", "PyInterpreterProgramInfo") load(":toolchain_types.bzl", "EXEC_TOOLS_TOOLCHAIN_TYPE", "TARGET_TOOLCHAIN_TYPE") -_py_builtins = py_internal - -def collect_cc_info(ctx, extra_deps = []): - """Collect C++ information from dependencies for Bazel. - - Args: - ctx: Rule ctx; must have `deps` attribute. - extra_deps: list of Target to also collect C+ information from. - - Returns: - CcInfo provider of merged information. - """ - deps = ctx.attr.deps - if extra_deps: - deps = list(deps) - deps.extend(extra_deps) - cc_infos = [] - for dep in deps: - if CcInfo in dep: - cc_infos.append(dep[CcInfo]) - - if PyCcLinkParamsInfo in dep: - cc_infos.append(dep[PyCcLinkParamsInfo].cc_info) - - return cc_common.merge_cc_infos(cc_infos = cc_infos) - def maybe_precompile(ctx, srcs): """Computes all the outputs (maybe precompiled) from the input srcs. @@ -237,44 +205,3 @@ def _precompile(ctx, src, *, use_pycache): toolchain = EXEC_TOOLS_TOOLCHAIN_TYPE, ) return pyc - -def get_imports(ctx): - """Gets the imports from a rule's `imports` attribute. - - See create_binary_semantics_struct for details about this function. - - Args: - ctx: Rule ctx. - - Returns: - List of strings. - """ - prefix = "{}/{}".format( - ctx.workspace_name, - _py_builtins.get_label_repo_runfiles_path(ctx.label), - ) - result = [] - for import_str in ctx.attr.imports: - import_str = ctx.expand_make_variables("imports", import_str, {}) - if import_str.startswith("/"): - continue - - # To prevent "escaping" out of the runfiles tree, we normalize - # the path and ensure it doesn't have up-level references. - import_path = paths.normalize("{}/{}".format(prefix, import_str)) - if import_path.startswith("../") or import_path == "..": - fail("Path '{}' references a path above the execution root".format( - import_str, - )) - result.append(import_path) - return result - -def convert_legacy_create_init_to_int(kwargs): - """Convert "legacy_create_init" key to int, in-place. - - Args: - kwargs: The kwargs to modify. The key "legacy_create_init", if present - and bool, will be converted to its integer value, in place. - """ - if is_bool(kwargs.get("legacy_create_init")): - kwargs["legacy_create_init"] = 1 if kwargs["legacy_create_init"] else 0 diff --git a/python/private/py_binary_macro.bzl b/python/private/py_binary_macro.bzl index 83b3c18677..d1269f2321 100644 --- a/python/private/py_binary_macro.bzl +++ b/python/private/py_binary_macro.bzl @@ -13,8 +13,8 @@ # limitations under the License. """Implementation of macro-half of py_binary rule.""" -load(":common_bazel.bzl", "convert_legacy_create_init_to_int") load(":py_binary_rule.bzl", py_binary_rule = "py_binary") +load(":py_executable.bzl", "convert_legacy_create_init_to_int") def py_binary(**kwargs): convert_legacy_create_init_to_int(kwargs) diff --git a/python/private/py_binary_rule.bzl b/python/private/py_binary_rule.bzl index 9ce0726c5e..f1c8eb1325 100644 --- a/python/private/py_binary_rule.bzl +++ b/python/private/py_binary_rule.bzl @@ -16,9 +16,9 @@ load("@bazel_skylib//lib:dicts.bzl", "dicts") load(":attributes.bzl", "AGNOSTIC_BINARY_ATTRS") load( - ":py_executable_bazel.bzl", + ":py_executable.bzl", "create_executable_rule", - "py_executable_bazel_impl", + "py_executable_impl", ) _PY_TEST_ATTRS = { @@ -39,7 +39,7 @@ _PY_TEST_ATTRS = { } def _py_binary_impl(ctx): - return py_executable_bazel_impl( + return py_executable_impl( ctx = ctx, is_test = False, inherited_environment = [], diff --git a/python/private/py_executable.bzl b/python/private/py_executable.bzl index 8c0487d6a1..40c74100f2 100644 --- a/python/private/py_executable.bzl +++ b/python/private/py_executable.bzl @@ -14,6 +14,7 @@ """Common functionality between test/binary executables.""" load("@bazel_skylib//lib:dicts.bzl", "dicts") +load("@bazel_skylib//lib:paths.bzl", "paths") load("@bazel_skylib//lib:structs.bzl", "structs") load("@bazel_skylib//rules:common_settings.bzl", "BuildSettingInfo") load("@rules_cc//cc/common:cc_common.bzl", "cc_common") @@ -21,6 +22,7 @@ load( ":attributes.bzl", "AGNOSTIC_EXECUTABLE_ATTRS", "COMMON_ATTRS", + "IMPORTS_ATTRS", "PY_SRCS_ATTRS", "PrecompileAttr", "PycCollectionAttr", @@ -33,21 +35,29 @@ load(":builders.bzl", "builders") load(":cc_helper.bzl", "cc_helper") load( ":common.bzl", + "collect_cc_info", "collect_imports", "collect_runfiles", + "create_binary_semantics_struct", + "create_cc_details_struct", + "create_executable_result_struct", "create_instrumented_files_info", "create_output_group_info", "create_py_info", "csv", "filter_to_py_srcs", + "get_imports", + "is_bool", "target_platform_has_any_constraint", "union_attrs", ) +load(":flags.bzl", "BootstrapImplFlag") +load(":precompile.bzl", "maybe_precompile") load(":py_cc_link_params_info.bzl", "PyCcLinkParamsInfo") load(":py_executable_info.bzl", "PyExecutableInfo") load(":py_info.bzl", "PyInfo") load(":py_internal.bzl", "py_internal") -load(":py_runtime_info.bzl", "PyRuntimeInfo") +load(":py_runtime_info.bzl", "DEFAULT_STUB_SHEBANG", "PyRuntimeInfo") load(":reexports.bzl", "BuiltinPyInfo", "BuiltinPyRuntimeInfo") load( ":semantics.bzl", @@ -59,10 +69,13 @@ load( load( ":toolchain_types.bzl", "EXEC_TOOLS_TOOLCHAIN_TYPE", + "TARGET_TOOLCHAIN_TYPE", TOOLCHAIN_TYPE = "TARGET_TOOLCHAIN_TYPE", ) _py_builtins = py_internal +_EXTERNAL_PATH_PREFIX = "external" +_ZIP_RUNFILES_DIRECTORY_NAME = "runfiles" # Bazel 5.4 doesn't have config_common.toolchain_type _CC_TOOLCHAINS = [config_common.toolchain_type( @@ -76,7 +89,21 @@ EXECUTABLE_ATTRS = union_attrs( COMMON_ATTRS, AGNOSTIC_EXECUTABLE_ATTRS, PY_SRCS_ATTRS, + IMPORTS_ATTRS, { + "legacy_create_init": attr.int( + default = -1, + values = [-1, 0, 1], + doc = """\ +Whether to implicitly create empty `__init__.py` files in the runfiles tree. +These are created in every directory containing Python source code or shared +libraries, and every parent directory of those directories, excluding the repo +root directory. The default, `-1` (auto), means true unless +`--incompatible_default_to_explicit_init_py` is used. If false, the user is +responsible for creating (possibly empty) `__init__.py` files and adding them to +the `srcs` of Python targets as required. + """, + ), # TODO(b/203567235): In the Java impl, any file is allowed. While marked # label, it is more treated as a string, and doesn't have to refer to # anything that exists because it gets treated as suffix-search string @@ -120,17 +147,732 @@ Valid values are: default = "//python/config_settings:bootstrap_impl", providers = [BuildSettingInfo], ), + "_bootstrap_template": attr.label( + allow_single_file = True, + default = "@bazel_tools//tools/python:python_bootstrap_template.txt", + ), + "_launcher": attr.label( + cfg = "target", + # NOTE: This is an executable, but is only used for Windows. It + # can't have executable=True because the backing target is an + # empty target for other platforms. + default = "//tools/launcher:launcher", + ), + "_py_interpreter": attr.label( + # The configuration_field args are validated when called; + # we use the precense of py_internal to indicate this Bazel + # build has that fragment and name. + default = configuration_field( + fragment = "bazel_py", + name = "python_top", + ) if py_internal else None, + ), + # TODO: This appears to be vestigial. It's only added because + # GraphlessQueryTest.testLabelsOperator relies on it to test for + # query behavior of implicit dependencies. + "_py_toolchain_type": attr.label( + default = TARGET_TOOLCHAIN_TYPE, + ), + "_python_version_flag": attr.label( + default = "//python/config_settings:python_version", + ), "_windows_constraints": attr.label_list( default = [ "@platforms//os:windows", ], ), + "_windows_launcher_maker": attr.label( + default = "@bazel_tools//tools/launcher:launcher_maker", + cfg = "exec", + executable = True, + ), + "_zipper": attr.label( + cfg = "exec", + executable = True, + default = "@bazel_tools//tools/zip:zipper", + ), }, create_srcs_version_attr(values = SRCS_VERSION_ALL_VALUES), create_srcs_attr(mandatory = True), allow_none = True, ) +def convert_legacy_create_init_to_int(kwargs): + """Convert "legacy_create_init" key to int, in-place. + + Args: + kwargs: The kwargs to modify. The key "legacy_create_init", if present + and bool, will be converted to its integer value, in place. + """ + if is_bool(kwargs.get("legacy_create_init")): + kwargs["legacy_create_init"] = 1 if kwargs["legacy_create_init"] else 0 + +def py_executable_impl(ctx, *, is_test, inherited_environment): + return py_executable_base_impl( + ctx = ctx, + semantics = create_binary_semantics(), + is_test = is_test, + inherited_environment = inherited_environment, + ) + +def create_binary_semantics(): + return create_binary_semantics_struct( + # keep-sorted start + create_executable = _create_executable, + get_cc_details_for_binary = _get_cc_details_for_binary, + get_central_uncachable_version_file = lambda ctx: None, + get_coverage_deps = _get_coverage_deps, + get_debugger_deps = _get_debugger_deps, + get_extra_common_runfiles_for_binary = lambda ctx: ctx.runfiles(), + get_extra_providers = _get_extra_providers, + get_extra_write_build_data_env = lambda ctx: {}, + get_imports = get_imports, + get_interpreter_path = _get_interpreter_path, + get_native_deps_dso_name = _get_native_deps_dso_name, + get_native_deps_user_link_flags = _get_native_deps_user_link_flags, + get_stamp_flag = _get_stamp_flag, + maybe_precompile = maybe_precompile, + should_build_native_deps_dso = lambda ctx: False, + should_create_init_files = _should_create_init_files, + should_include_build_data = lambda ctx: False, + # keep-sorted end + ) + +def _get_coverage_deps(ctx, runtime_details): + _ = ctx, runtime_details # @unused + return [] + +def _get_debugger_deps(ctx, runtime_details): + _ = ctx, runtime_details # @unused + return [] + +def _get_extra_providers(ctx, main_py, runtime_details): + _ = ctx, main_py, runtime_details # @unused + return [] + +def _get_stamp_flag(ctx): + # NOTE: Undocumented API; private to builtins + return ctx.configuration.stamp_binaries + +def _should_create_init_files(ctx): + if ctx.attr.legacy_create_init == -1: + return not ctx.fragments.py.default_to_explicit_init_py + else: + return bool(ctx.attr.legacy_create_init) + +def _create_executable( + ctx, + *, + executable, + main_py, + imports, + is_test, + runtime_details, + cc_details, + native_deps_details, + runfiles_details): + _ = is_test, cc_details, native_deps_details # @unused + + is_windows = target_platform_has_any_constraint(ctx, ctx.attr._windows_constraints) + + if is_windows: + if not executable.extension == "exe": + fail("Should not happen: somehow we are generating a non-.exe file on windows") + base_executable_name = executable.basename[0:-4] + else: + base_executable_name = executable.basename + + venv = None + + # The check for stage2_bootstrap_template is to support legacy + # BuiltinPyRuntimeInfo providers, which is likely to come from + # @bazel_tools//tools/python:autodetecting_toolchain, the toolchain used + # for workspace builds when no rules_python toolchain is configured. + if (BootstrapImplFlag.get_value(ctx) == BootstrapImplFlag.SCRIPT and + runtime_details.effective_runtime and + hasattr(runtime_details.effective_runtime, "stage2_bootstrap_template")): + venv = _create_venv( + ctx, + output_prefix = base_executable_name, + imports = imports, + runtime_details = runtime_details, + ) + + stage2_bootstrap = _create_stage2_bootstrap( + ctx, + output_prefix = base_executable_name, + output_sibling = executable, + main_py = main_py, + imports = imports, + runtime_details = runtime_details, + ) + extra_runfiles = ctx.runfiles([stage2_bootstrap] + venv.files_without_interpreter) + zip_main = _create_zip_main( + ctx, + stage2_bootstrap = stage2_bootstrap, + runtime_details = runtime_details, + venv = venv, + ) + else: + stage2_bootstrap = None + extra_runfiles = ctx.runfiles() + zip_main = ctx.actions.declare_file(base_executable_name + ".temp", sibling = executable) + _create_stage1_bootstrap( + ctx, + output = zip_main, + main_py = main_py, + imports = imports, + is_for_zip = True, + runtime_details = runtime_details, + ) + + zip_file = ctx.actions.declare_file(base_executable_name + ".zip", sibling = executable) + _create_zip_file( + ctx, + output = zip_file, + original_nonzip_executable = executable, + zip_main = zip_main, + runfiles = runfiles_details.default_runfiles.merge(extra_runfiles), + ) + + extra_files_to_build = [] + + # NOTE: --build_python_zip defaults to true on Windows + build_zip_enabled = ctx.fragments.py.build_python_zip + + # When --build_python_zip is enabled, then the zip file becomes + # one of the default outputs. + if build_zip_enabled: + extra_files_to_build.append(zip_file) + + # The logic here is a bit convoluted. Essentially, there are 3 types of + # executables produced: + # 1. (non-Windows) A bootstrap template based program. + # 2. (non-Windows) A self-executable zip file of a bootstrap template based program. + # 3. (Windows) A native Windows executable that finds and launches + # the actual underlying Bazel program (one of the above). Note that + # it implicitly assumes one of the above is located next to it, and + # that --build_python_zip defaults to true for Windows. + + should_create_executable_zip = False + bootstrap_output = None + if not is_windows: + if build_zip_enabled: + should_create_executable_zip = True + else: + bootstrap_output = executable + else: + _create_windows_exe_launcher( + ctx, + output = executable, + use_zip_file = build_zip_enabled, + python_binary_path = runtime_details.executable_interpreter_path, + ) + if not build_zip_enabled: + # On Windows, the main executable has an "exe" extension, so + # here we re-use the un-extensioned name for the bootstrap output. + bootstrap_output = ctx.actions.declare_file(base_executable_name) + + # The launcher looks for the non-zip executable next to + # itself, so add it to the default outputs. + extra_files_to_build.append(bootstrap_output) + + if should_create_executable_zip: + if bootstrap_output != None: + fail("Should not occur: bootstrap_output should not be used " + + "when creating an executable zip") + _create_executable_zip_file( + ctx, + output = executable, + zip_file = zip_file, + stage2_bootstrap = stage2_bootstrap, + runtime_details = runtime_details, + venv = venv, + ) + elif bootstrap_output: + _create_stage1_bootstrap( + ctx, + output = bootstrap_output, + stage2_bootstrap = stage2_bootstrap, + runtime_details = runtime_details, + is_for_zip = False, + imports = imports, + main_py = main_py, + venv = venv, + ) + else: + # Otherwise, this should be the Windows case of launcher + zip. + # Double check this just to make sure. + if not is_windows or not build_zip_enabled: + fail(("Should not occur: The non-executable-zip and " + + "non-bootstrap-template case should have windows and zip " + + "both true, but got " + + "is_windows={is_windows} " + + "build_zip_enabled={build_zip_enabled}").format( + is_windows = is_windows, + build_zip_enabled = build_zip_enabled, + )) + + # The interpreter is added this late in the process so that it isn't + # added to the zipped files. + if venv: + extra_runfiles = extra_runfiles.merge(ctx.runfiles([venv.interpreter])) + return create_executable_result_struct( + extra_files_to_build = depset(extra_files_to_build), + output_groups = {"python_zip_file": depset([zip_file])}, + extra_runfiles = extra_runfiles, + ) + +def _create_zip_main(ctx, *, stage2_bootstrap, runtime_details, venv): + python_binary = _runfiles_root_path(ctx, venv.interpreter.short_path) + python_binary_actual = venv.interpreter_actual_path + + # The location of this file doesn't really matter. It's added to + # the zip file as the top-level __main__.py file and not included + # elsewhere. + output = ctx.actions.declare_file(ctx.label.name + "_zip__main__.py") + ctx.actions.expand_template( + template = runtime_details.effective_runtime.zip_main_template, + output = output, + substitutions = { + "%python_binary%": python_binary, + "%python_binary_actual%": python_binary_actual, + "%stage2_bootstrap%": "{}/{}".format( + ctx.workspace_name, + stage2_bootstrap.short_path, + ), + "%workspace_name%": ctx.workspace_name, + }, + ) + return output + +def relative_path(from_, to): + """Compute a relative path from one path to another. + + Args: + from_: {type}`str` the starting directory. Note that it should be + a directory because relative-symlinks are relative to the + directory the symlink resides in. + to: {type}`str` the path that `from_` wants to point to + + Returns: + {type}`str` a relative path + """ + from_parts = from_.split("/") + to_parts = to.split("/") + + # Strip common leading parts from both paths + n = min(len(from_parts), len(to_parts)) + for _ in range(n): + if from_parts[0] == to_parts[0]: + from_parts.pop(0) + to_parts.pop(0) + else: + break + + # Impossible to compute a relative path without knowing what ".." is + if from_parts and from_parts[0] == "..": + fail("cannot compute relative path from '%s' to '%s'", from_, to) + + parts = ([".."] * len(from_parts)) + to_parts + return paths.join(*parts) + +# Create a venv the executable can use. +# For venv details and the venv startup process, see: +# * https://docs.python.org/3/library/venv.html +# * https://snarky.ca/how-virtual-environments-work/ +# * https://github.com/python/cpython/blob/main/Modules/getpath.py +# * https://github.com/python/cpython/blob/main/Lib/site.py +def _create_venv(ctx, output_prefix, imports, runtime_details): + venv = "_{}.venv".format(output_prefix.lstrip("_")) + + # The pyvenv.cfg file must be present to trigger the venv site hooks. + # Because it's paths are expected to be absolute paths, we can't reliably + # put much in it. See https://github.com/python/cpython/issues/83650 + pyvenv_cfg = ctx.actions.declare_file("{}/pyvenv.cfg".format(venv)) + ctx.actions.write(pyvenv_cfg, "") + + runtime = runtime_details.effective_runtime + if runtime.interpreter: + py_exe_basename = paths.basename(runtime.interpreter.short_path) + + # Even though ctx.actions.symlink() is used, using + # declare_symlink() is required to ensure that the resulting file + # in runfiles is always a symlink. An RBE implementation, for example, + # may choose to write what symlink() points to instead. + interpreter = ctx.actions.declare_symlink("{}/bin/{}".format(venv, py_exe_basename)) + + interpreter_actual_path = _runfiles_root_path(ctx, runtime.interpreter.short_path) + rel_path = relative_path( + # dirname is necessary because a relative symlink is relative to + # the directory the symlink resides within. + from_ = paths.dirname(_runfiles_root_path(ctx, interpreter.short_path)), + to = interpreter_actual_path, + ) + + ctx.actions.symlink(output = interpreter, target_path = rel_path) + else: + py_exe_basename = paths.basename(runtime.interpreter_path) + interpreter = ctx.actions.declare_symlink("{}/bin/{}".format(venv, py_exe_basename)) + ctx.actions.symlink(output = interpreter, target_path = runtime.interpreter_path) + interpreter_actual_path = runtime.interpreter_path + + if runtime.interpreter_version_info: + version = "{}.{}".format( + runtime.interpreter_version_info.major, + runtime.interpreter_version_info.minor, + ) + else: + version_flag = ctx.attr._python_version_flag[config_common.FeatureFlagInfo].value + version_flag_parts = version_flag.split(".")[0:2] + version = "{}.{}".format(*version_flag_parts) + + # See site.py logic: free-threaded builds append "t" to the venv lib dir name + if "t" in runtime.abi_flags: + version += "t" + + site_packages = "{}/lib/python{}/site-packages".format(venv, version) + pth = ctx.actions.declare_file("{}/bazel.pth".format(site_packages)) + ctx.actions.write(pth, "import _bazel_site_init\n") + + site_init = ctx.actions.declare_file("{}/_bazel_site_init.py".format(site_packages)) + computed_subs = ctx.actions.template_dict() + computed_subs.add_joined("%imports%", imports, join_with = ":", map_each = _map_each_identity) + ctx.actions.expand_template( + template = runtime.site_init_template, + output = site_init, + substitutions = { + "%import_all%": "True" if ctx.fragments.bazel_py.python_import_all_repositories else "False", + "%site_init_runfiles_path%": "{}/{}".format(ctx.workspace_name, site_init.short_path), + "%workspace_name%": ctx.workspace_name, + }, + computed_substitutions = computed_subs, + ) + + return struct( + interpreter = interpreter, + # Runfiles root relative path or absolute path + interpreter_actual_path = interpreter_actual_path, + files_without_interpreter = [pyvenv_cfg, pth, site_init], + ) + +def _map_each_identity(v): + return v + +def _create_stage2_bootstrap( + ctx, + *, + output_prefix, + output_sibling, + main_py, + imports, + runtime_details): + output = ctx.actions.declare_file( + # Prepend with underscore to prevent pytest from trying to + # process the bootstrap for files starting with `test_` + "_{}_stage2_bootstrap.py".format(output_prefix), + sibling = output_sibling, + ) + runtime = runtime_details.effective_runtime + if (ctx.configuration.coverage_enabled and + runtime and + runtime.coverage_tool): + coverage_tool_runfiles_path = "{}/{}".format( + ctx.workspace_name, + runtime.coverage_tool.short_path, + ) + else: + coverage_tool_runfiles_path = "" + + template = runtime.stage2_bootstrap_template + + ctx.actions.expand_template( + template = template, + output = output, + substitutions = { + "%coverage_tool%": coverage_tool_runfiles_path, + "%import_all%": "True" if ctx.fragments.bazel_py.python_import_all_repositories else "False", + "%imports%": ":".join(imports.to_list()), + "%main%": "{}/{}".format(ctx.workspace_name, main_py.short_path), + "%target%": str(ctx.label), + "%workspace_name%": ctx.workspace_name, + }, + is_executable = True, + ) + return output + +def _runfiles_root_path(ctx, short_path): + """Compute a runfiles-root relative path from `File.short_path` + + Args: + ctx: current target ctx + short_path: str, a main-repo relative path from `File.short_path` + + Returns: + {type}`str`, a runflies-root relative path + """ + + # The ../ comes from short_path is for files in other repos. + if short_path.startswith("../"): + return short_path[3:] + else: + return "{}/{}".format(ctx.workspace_name, short_path) + +def _create_stage1_bootstrap( + ctx, + *, + output, + main_py = None, + stage2_bootstrap = None, + imports = None, + is_for_zip, + runtime_details, + venv = None): + runtime = runtime_details.effective_runtime + + if venv: + python_binary_path = _runfiles_root_path(ctx, venv.interpreter.short_path) + else: + python_binary_path = runtime_details.executable_interpreter_path + + if is_for_zip and venv: + python_binary_actual = venv.interpreter_actual_path + else: + python_binary_actual = "" + + subs = { + "%is_zipfile%": "1" if is_for_zip else "0", + "%python_binary%": python_binary_path, + "%python_binary_actual%": python_binary_actual, + "%target%": str(ctx.label), + "%workspace_name%": ctx.workspace_name, + } + + if stage2_bootstrap: + subs["%stage2_bootstrap%"] = "{}/{}".format( + ctx.workspace_name, + stage2_bootstrap.short_path, + ) + template = runtime.bootstrap_template + subs["%shebang%"] = runtime.stub_shebang + else: + if (ctx.configuration.coverage_enabled and + runtime and + runtime.coverage_tool): + coverage_tool_runfiles_path = "{}/{}".format( + ctx.workspace_name, + runtime.coverage_tool.short_path, + ) + else: + coverage_tool_runfiles_path = "" + if runtime: + subs["%shebang%"] = runtime.stub_shebang + template = runtime.bootstrap_template + else: + subs["%shebang%"] = DEFAULT_STUB_SHEBANG + template = ctx.file._bootstrap_template + + subs["%coverage_tool%"] = coverage_tool_runfiles_path + subs["%import_all%"] = ("True" if ctx.fragments.bazel_py.python_import_all_repositories else "False") + subs["%imports%"] = ":".join(imports.to_list()) + subs["%main%"] = "{}/{}".format(ctx.workspace_name, main_py.short_path) + + ctx.actions.expand_template( + template = template, + output = output, + substitutions = subs, + ) + +def _create_windows_exe_launcher( + ctx, + *, + output, + python_binary_path, + use_zip_file): + launch_info = ctx.actions.args() + launch_info.use_param_file("%s", use_always = True) + launch_info.set_param_file_format("multiline") + launch_info.add("binary_type=Python") + launch_info.add(ctx.workspace_name, format = "workspace_name=%s") + launch_info.add( + "1" if py_internal.runfiles_enabled(ctx) else "0", + format = "symlink_runfiles_enabled=%s", + ) + launch_info.add(python_binary_path, format = "python_bin_path=%s") + launch_info.add("1" if use_zip_file else "0", format = "use_zip_file=%s") + + launcher = ctx.attr._launcher[DefaultInfo].files_to_run.executable + ctx.actions.run( + executable = ctx.executable._windows_launcher_maker, + arguments = [launcher.path, launch_info, output.path], + inputs = [launcher], + outputs = [output], + mnemonic = "PyBuildLauncher", + progress_message = "Creating launcher for %{label}", + # Needed to inherit PATH when using non-MSVC compilers like MinGW + use_default_shell_env = True, + ) + +def _create_zip_file(ctx, *, output, original_nonzip_executable, zip_main, runfiles): + """Create a Python zipapp (zip with __main__.py entry point).""" + workspace_name = ctx.workspace_name + legacy_external_runfiles = _py_builtins.get_legacy_external_runfiles(ctx) + + manifest = ctx.actions.args() + manifest.use_param_file("@%s", use_always = True) + manifest.set_param_file_format("multiline") + + manifest.add("__main__.py={}".format(zip_main.path)) + manifest.add("__init__.py=") + manifest.add( + "{}=".format( + _get_zip_runfiles_path("__init__.py", workspace_name, legacy_external_runfiles), + ), + ) + for path in runfiles.empty_filenames.to_list(): + manifest.add("{}=".format(_get_zip_runfiles_path(path, workspace_name, legacy_external_runfiles))) + + def map_zip_runfiles(file): + if file != original_nonzip_executable and file != output: + return "{}={}".format( + _get_zip_runfiles_path(file.short_path, workspace_name, legacy_external_runfiles), + file.path, + ) + else: + return None + + manifest.add_all(runfiles.files, map_each = map_zip_runfiles, allow_closure = True) + + inputs = [zip_main] + if _py_builtins.is_bzlmod_enabled(ctx): + zip_repo_mapping_manifest = ctx.actions.declare_file( + output.basename + ".repo_mapping", + sibling = output, + ) + _py_builtins.create_repo_mapping_manifest( + ctx = ctx, + runfiles = runfiles, + output = zip_repo_mapping_manifest, + ) + manifest.add("{}/_repo_mapping={}".format( + _ZIP_RUNFILES_DIRECTORY_NAME, + zip_repo_mapping_manifest.path, + )) + inputs.append(zip_repo_mapping_manifest) + + for artifact in runfiles.files.to_list(): + # Don't include the original executable because it isn't used by the + # zip file, so no need to build it for the action. + # Don't include the zipfile itself because it's an output. + if artifact != original_nonzip_executable and artifact != output: + inputs.append(artifact) + + zip_cli_args = ctx.actions.args() + zip_cli_args.add("cC") + zip_cli_args.add(output) + + ctx.actions.run( + executable = ctx.executable._zipper, + arguments = [zip_cli_args, manifest], + inputs = depset(inputs), + outputs = [output], + use_default_shell_env = True, + mnemonic = "PythonZipper", + progress_message = "Building Python zip: %{label}", + ) + +def _get_zip_runfiles_path(path, workspace_name, legacy_external_runfiles): + if legacy_external_runfiles and path.startswith(_EXTERNAL_PATH_PREFIX): + zip_runfiles_path = paths.relativize(path, _EXTERNAL_PATH_PREFIX) + else: + # NOTE: External runfiles (artifacts in other repos) will have a leading + # path component of "../" so that they refer outside the main workspace + # directory and into the runfiles root. By normalizing, we simplify e.g. + # "workspace/../foo/bar" to simply "foo/bar". + zip_runfiles_path = paths.normalize("{}/{}".format(workspace_name, path)) + return "{}/{}".format(_ZIP_RUNFILES_DIRECTORY_NAME, zip_runfiles_path) + +def _create_executable_zip_file( + ctx, + *, + output, + zip_file, + stage2_bootstrap, + runtime_details, + venv): + prelude = ctx.actions.declare_file( + "{}_zip_prelude.sh".format(output.basename), + sibling = output, + ) + if stage2_bootstrap: + _create_stage1_bootstrap( + ctx, + output = prelude, + stage2_bootstrap = stage2_bootstrap, + runtime_details = runtime_details, + is_for_zip = True, + venv = venv, + ) + else: + ctx.actions.write(prelude, "#!/usr/bin/env python3\n") + + ctx.actions.run_shell( + command = "cat {prelude} {zip} > {output}".format( + prelude = prelude.path, + zip = zip_file.path, + output = output.path, + ), + inputs = [prelude, zip_file], + outputs = [output], + use_default_shell_env = True, + mnemonic = "PyBuildExecutableZip", + progress_message = "Build Python zip executable: %{label}", + ) + +def _get_cc_details_for_binary(ctx, extra_deps): + cc_info = collect_cc_info(ctx, extra_deps = extra_deps) + return create_cc_details_struct( + cc_info_for_propagating = cc_info, + cc_info_for_self_link = cc_info, + cc_info_with_extra_link_time_libraries = None, + extra_runfiles = ctx.runfiles(), + # Though the rules require the CcToolchain, it isn't actually used. + cc_toolchain = None, + feature_config = None, + ) + +def _get_interpreter_path(ctx, *, runtime, flag_interpreter_path): + if runtime: + if runtime.interpreter_path: + interpreter_path = runtime.interpreter_path + else: + interpreter_path = "{}/{}".format( + ctx.workspace_name, + runtime.interpreter.short_path, + ) + + # NOTE: External runfiles (artifacts in other repos) will have a + # leading path component of "../" so that they refer outside the + # main workspace directory and into the runfiles root. By + # normalizing, we simplify e.g. "workspace/../foo/bar" to simply + # "foo/bar" + interpreter_path = paths.normalize(interpreter_path) + + elif flag_interpreter_path: + interpreter_path = flag_interpreter_path + else: + fail("Unable to determine interpreter path") + + return interpreter_path + +def _get_native_deps_dso_name(ctx): + _ = ctx # @unused + fail("Building native deps DSO not supported.") + +def _get_native_deps_user_link_flags(ctx): + _ = ctx # @unused + fail("Building native deps DSO not supported.") + def py_executable_base_impl(ctx, *, semantics, is_test, inherited_environment = []): """Base rule implementation for a Python executable. @@ -949,6 +1691,14 @@ def _create_run_environment_info(ctx, inherited_environment): inherited_environment = inherited_environment, ) +def create_executable_rule(*, attrs, **kwargs): + return create_base_executable_rule( + ##attrs = dicts.add(EXECUTABLE_ATTRS, attrs), + attrs = attrs, + fragments = ["py", "bazel_py"], + **kwargs + ) + def create_base_executable_rule(*, attrs, fragments = [], **kwargs): """Create a function for defining for Python binary/test targets. diff --git a/python/private/py_executable_bazel.bzl b/python/private/py_executable_bazel.bzl deleted file mode 100644 index 3778c192b4..0000000000 --- a/python/private/py_executable_bazel.bzl +++ /dev/null @@ -1,772 +0,0 @@ -# Copyright 2022 The Bazel Authors. All rights reserved. -# -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. -"""Implementation for Bazel Python executable.""" - -load("@bazel_skylib//lib:dicts.bzl", "dicts") -load("@bazel_skylib//lib:paths.bzl", "paths") -load(":attributes.bzl", "IMPORTS_ATTRS") -load( - ":common.bzl", - "create_binary_semantics_struct", - "create_cc_details_struct", - "create_executable_result_struct", - "target_platform_has_any_constraint", - "union_attrs", -) -load(":common_bazel.bzl", "collect_cc_info", "get_imports", "maybe_precompile") -load(":flags.bzl", "BootstrapImplFlag") -load( - ":py_executable.bzl", - "create_base_executable_rule", - "py_executable_base_impl", -) -load(":py_internal.bzl", "py_internal") -load(":py_runtime_info.bzl", "DEFAULT_STUB_SHEBANG") -load(":toolchain_types.bzl", "TARGET_TOOLCHAIN_TYPE") - -_py_builtins = py_internal -_EXTERNAL_PATH_PREFIX = "external" -_ZIP_RUNFILES_DIRECTORY_NAME = "runfiles" - -BAZEL_EXECUTABLE_ATTRS = union_attrs( - IMPORTS_ATTRS, - { - "legacy_create_init": attr.int( - default = -1, - values = [-1, 0, 1], - doc = """\ -Whether to implicitly create empty `__init__.py` files in the runfiles tree. -These are created in every directory containing Python source code or shared -libraries, and every parent directory of those directories, excluding the repo -root directory. The default, `-1` (auto), means true unless -`--incompatible_default_to_explicit_init_py` is used. If false, the user is -responsible for creating (possibly empty) `__init__.py` files and adding them to -the `srcs` of Python targets as required. - """, - ), - "_bootstrap_template": attr.label( - allow_single_file = True, - default = "@bazel_tools//tools/python:python_bootstrap_template.txt", - ), - "_launcher": attr.label( - cfg = "target", - # NOTE: This is an executable, but is only used for Windows. It - # can't have executable=True because the backing target is an - # empty target for other platforms. - default = "//tools/launcher:launcher", - ), - "_py_interpreter": attr.label( - # The configuration_field args are validated when called; - # we use the precense of py_internal to indicate this Bazel - # build has that fragment and name. - default = configuration_field( - fragment = "bazel_py", - name = "python_top", - ) if py_internal else None, - ), - # TODO: This appears to be vestigial. It's only added because - # GraphlessQueryTest.testLabelsOperator relies on it to test for - # query behavior of implicit dependencies. - "_py_toolchain_type": attr.label( - default = TARGET_TOOLCHAIN_TYPE, - ), - "_python_version_flag": attr.label( - default = "//python/config_settings:python_version", - ), - "_windows_launcher_maker": attr.label( - default = "@bazel_tools//tools/launcher:launcher_maker", - cfg = "exec", - executable = True, - ), - "_zipper": attr.label( - cfg = "exec", - executable = True, - default = "@bazel_tools//tools/zip:zipper", - ), - }, -) - -def create_executable_rule(*, attrs, **kwargs): - return create_base_executable_rule( - attrs = dicts.add(BAZEL_EXECUTABLE_ATTRS, attrs), - fragments = ["py", "bazel_py"], - **kwargs - ) - -def py_executable_bazel_impl(ctx, *, is_test, inherited_environment): - """Common code for executables for Bazel.""" - return py_executable_base_impl( - ctx = ctx, - semantics = create_binary_semantics_bazel(), - is_test = is_test, - inherited_environment = inherited_environment, - ) - -def create_binary_semantics_bazel(): - return create_binary_semantics_struct( - # keep-sorted start - create_executable = _create_executable, - get_cc_details_for_binary = _get_cc_details_for_binary, - get_central_uncachable_version_file = lambda ctx: None, - get_coverage_deps = _get_coverage_deps, - get_debugger_deps = _get_debugger_deps, - get_extra_common_runfiles_for_binary = lambda ctx: ctx.runfiles(), - get_extra_providers = _get_extra_providers, - get_extra_write_build_data_env = lambda ctx: {}, - get_imports = get_imports, - get_interpreter_path = _get_interpreter_path, - get_native_deps_dso_name = _get_native_deps_dso_name, - get_native_deps_user_link_flags = _get_native_deps_user_link_flags, - get_stamp_flag = _get_stamp_flag, - maybe_precompile = maybe_precompile, - should_build_native_deps_dso = lambda ctx: False, - should_create_init_files = _should_create_init_files, - should_include_build_data = lambda ctx: False, - # keep-sorted end - ) - -def _get_coverage_deps(ctx, runtime_details): - _ = ctx, runtime_details # @unused - return [] - -def _get_debugger_deps(ctx, runtime_details): - _ = ctx, runtime_details # @unused - return [] - -def _get_extra_providers(ctx, main_py, runtime_details): - _ = ctx, main_py, runtime_details # @unused - return [] - -def _get_stamp_flag(ctx): - # NOTE: Undocumented API; private to builtins - return ctx.configuration.stamp_binaries - -def _should_create_init_files(ctx): - if ctx.attr.legacy_create_init == -1: - return not ctx.fragments.py.default_to_explicit_init_py - else: - return bool(ctx.attr.legacy_create_init) - -def _create_executable( - ctx, - *, - executable, - main_py, - imports, - is_test, - runtime_details, - cc_details, - native_deps_details, - runfiles_details): - _ = is_test, cc_details, native_deps_details # @unused - - is_windows = target_platform_has_any_constraint(ctx, ctx.attr._windows_constraints) - - if is_windows: - if not executable.extension == "exe": - fail("Should not happen: somehow we are generating a non-.exe file on windows") - base_executable_name = executable.basename[0:-4] - else: - base_executable_name = executable.basename - - venv = None - - # The check for stage2_bootstrap_template is to support legacy - # BuiltinPyRuntimeInfo providers, which is likely to come from - # @bazel_tools//tools/python:autodetecting_toolchain, the toolchain used - # for workspace builds when no rules_python toolchain is configured. - if (BootstrapImplFlag.get_value(ctx) == BootstrapImplFlag.SCRIPT and - runtime_details.effective_runtime and - hasattr(runtime_details.effective_runtime, "stage2_bootstrap_template")): - venv = _create_venv( - ctx, - output_prefix = base_executable_name, - imports = imports, - runtime_details = runtime_details, - ) - - stage2_bootstrap = _create_stage2_bootstrap( - ctx, - output_prefix = base_executable_name, - output_sibling = executable, - main_py = main_py, - imports = imports, - runtime_details = runtime_details, - ) - extra_runfiles = ctx.runfiles([stage2_bootstrap] + venv.files_without_interpreter) - zip_main = _create_zip_main( - ctx, - stage2_bootstrap = stage2_bootstrap, - runtime_details = runtime_details, - venv = venv, - ) - else: - stage2_bootstrap = None - extra_runfiles = ctx.runfiles() - zip_main = ctx.actions.declare_file(base_executable_name + ".temp", sibling = executable) - _create_stage1_bootstrap( - ctx, - output = zip_main, - main_py = main_py, - imports = imports, - is_for_zip = True, - runtime_details = runtime_details, - ) - - zip_file = ctx.actions.declare_file(base_executable_name + ".zip", sibling = executable) - _create_zip_file( - ctx, - output = zip_file, - original_nonzip_executable = executable, - zip_main = zip_main, - runfiles = runfiles_details.default_runfiles.merge(extra_runfiles), - ) - - extra_files_to_build = [] - - # NOTE: --build_python_zip defaults to true on Windows - build_zip_enabled = ctx.fragments.py.build_python_zip - - # When --build_python_zip is enabled, then the zip file becomes - # one of the default outputs. - if build_zip_enabled: - extra_files_to_build.append(zip_file) - - # The logic here is a bit convoluted. Essentially, there are 3 types of - # executables produced: - # 1. (non-Windows) A bootstrap template based program. - # 2. (non-Windows) A self-executable zip file of a bootstrap template based program. - # 3. (Windows) A native Windows executable that finds and launches - # the actual underlying Bazel program (one of the above). Note that - # it implicitly assumes one of the above is located next to it, and - # that --build_python_zip defaults to true for Windows. - - should_create_executable_zip = False - bootstrap_output = None - if not is_windows: - if build_zip_enabled: - should_create_executable_zip = True - else: - bootstrap_output = executable - else: - _create_windows_exe_launcher( - ctx, - output = executable, - use_zip_file = build_zip_enabled, - python_binary_path = runtime_details.executable_interpreter_path, - ) - if not build_zip_enabled: - # On Windows, the main executable has an "exe" extension, so - # here we re-use the un-extensioned name for the bootstrap output. - bootstrap_output = ctx.actions.declare_file(base_executable_name) - - # The launcher looks for the non-zip executable next to - # itself, so add it to the default outputs. - extra_files_to_build.append(bootstrap_output) - - if should_create_executable_zip: - if bootstrap_output != None: - fail("Should not occur: bootstrap_output should not be used " + - "when creating an executable zip") - _create_executable_zip_file( - ctx, - output = executable, - zip_file = zip_file, - stage2_bootstrap = stage2_bootstrap, - runtime_details = runtime_details, - venv = venv, - ) - elif bootstrap_output: - _create_stage1_bootstrap( - ctx, - output = bootstrap_output, - stage2_bootstrap = stage2_bootstrap, - runtime_details = runtime_details, - is_for_zip = False, - imports = imports, - main_py = main_py, - venv = venv, - ) - else: - # Otherwise, this should be the Windows case of launcher + zip. - # Double check this just to make sure. - if not is_windows or not build_zip_enabled: - fail(("Should not occur: The non-executable-zip and " + - "non-bootstrap-template case should have windows and zip " + - "both true, but got " + - "is_windows={is_windows} " + - "build_zip_enabled={build_zip_enabled}").format( - is_windows = is_windows, - build_zip_enabled = build_zip_enabled, - )) - - # The interpreter is added this late in the process so that it isn't - # added to the zipped files. - if venv: - extra_runfiles = extra_runfiles.merge(ctx.runfiles([venv.interpreter])) - return create_executable_result_struct( - extra_files_to_build = depset(extra_files_to_build), - output_groups = {"python_zip_file": depset([zip_file])}, - extra_runfiles = extra_runfiles, - ) - -def _create_zip_main(ctx, *, stage2_bootstrap, runtime_details, venv): - python_binary = _runfiles_root_path(ctx, venv.interpreter.short_path) - python_binary_actual = venv.interpreter_actual_path - - # The location of this file doesn't really matter. It's added to - # the zip file as the top-level __main__.py file and not included - # elsewhere. - output = ctx.actions.declare_file(ctx.label.name + "_zip__main__.py") - ctx.actions.expand_template( - template = runtime_details.effective_runtime.zip_main_template, - output = output, - substitutions = { - "%python_binary%": python_binary, - "%python_binary_actual%": python_binary_actual, - "%stage2_bootstrap%": "{}/{}".format( - ctx.workspace_name, - stage2_bootstrap.short_path, - ), - "%workspace_name%": ctx.workspace_name, - }, - ) - return output - -def relative_path(from_, to): - """Compute a relative path from one path to another. - - Args: - from_: {type}`str` the starting directory. Note that it should be - a directory because relative-symlinks are relative to the - directory the symlink resides in. - to: {type}`str` the path that `from_` wants to point to - - Returns: - {type}`str` a relative path - """ - from_parts = from_.split("/") - to_parts = to.split("/") - - # Strip common leading parts from both paths - n = min(len(from_parts), len(to_parts)) - for _ in range(n): - if from_parts[0] == to_parts[0]: - from_parts.pop(0) - to_parts.pop(0) - else: - break - - # Impossible to compute a relative path without knowing what ".." is - if from_parts and from_parts[0] == "..": - fail("cannot compute relative path from '%s' to '%s'", from_, to) - - parts = ([".."] * len(from_parts)) + to_parts - return paths.join(*parts) - -# Create a venv the executable can use. -# For venv details and the venv startup process, see: -# * https://docs.python.org/3/library/venv.html -# * https://snarky.ca/how-virtual-environments-work/ -# * https://github.com/python/cpython/blob/main/Modules/getpath.py -# * https://github.com/python/cpython/blob/main/Lib/site.py -def _create_venv(ctx, output_prefix, imports, runtime_details): - venv = "_{}.venv".format(output_prefix.lstrip("_")) - - # The pyvenv.cfg file must be present to trigger the venv site hooks. - # Because it's paths are expected to be absolute paths, we can't reliably - # put much in it. See https://github.com/python/cpython/issues/83650 - pyvenv_cfg = ctx.actions.declare_file("{}/pyvenv.cfg".format(venv)) - ctx.actions.write(pyvenv_cfg, "") - - runtime = runtime_details.effective_runtime - if runtime.interpreter: - py_exe_basename = paths.basename(runtime.interpreter.short_path) - - # Even though ctx.actions.symlink() is used, using - # declare_symlink() is required to ensure that the resulting file - # in runfiles is always a symlink. An RBE implementation, for example, - # may choose to write what symlink() points to instead. - interpreter = ctx.actions.declare_symlink("{}/bin/{}".format(venv, py_exe_basename)) - - interpreter_actual_path = _runfiles_root_path(ctx, runtime.interpreter.short_path) - rel_path = relative_path( - # dirname is necessary because a relative symlink is relative to - # the directory the symlink resides within. - from_ = paths.dirname(_runfiles_root_path(ctx, interpreter.short_path)), - to = interpreter_actual_path, - ) - - ctx.actions.symlink(output = interpreter, target_path = rel_path) - else: - py_exe_basename = paths.basename(runtime.interpreter_path) - interpreter = ctx.actions.declare_symlink("{}/bin/{}".format(venv, py_exe_basename)) - ctx.actions.symlink(output = interpreter, target_path = runtime.interpreter_path) - interpreter_actual_path = runtime.interpreter_path - - if runtime.interpreter_version_info: - version = "{}.{}".format( - runtime.interpreter_version_info.major, - runtime.interpreter_version_info.minor, - ) - else: - version_flag = ctx.attr._python_version_flag[config_common.FeatureFlagInfo].value - version_flag_parts = version_flag.split(".")[0:2] - version = "{}.{}".format(*version_flag_parts) - - # See site.py logic: free-threaded builds append "t" to the venv lib dir name - if "t" in runtime.abi_flags: - version += "t" - - site_packages = "{}/lib/python{}/site-packages".format(venv, version) - pth = ctx.actions.declare_file("{}/bazel.pth".format(site_packages)) - ctx.actions.write(pth, "import _bazel_site_init\n") - - site_init = ctx.actions.declare_file("{}/_bazel_site_init.py".format(site_packages)) - computed_subs = ctx.actions.template_dict() - computed_subs.add_joined("%imports%", imports, join_with = ":", map_each = _map_each_identity) - ctx.actions.expand_template( - template = runtime.site_init_template, - output = site_init, - substitutions = { - "%import_all%": "True" if ctx.fragments.bazel_py.python_import_all_repositories else "False", - "%site_init_runfiles_path%": "{}/{}".format(ctx.workspace_name, site_init.short_path), - "%workspace_name%": ctx.workspace_name, - }, - computed_substitutions = computed_subs, - ) - - return struct( - interpreter = interpreter, - # Runfiles root relative path or absolute path - interpreter_actual_path = interpreter_actual_path, - files_without_interpreter = [pyvenv_cfg, pth, site_init], - ) - -def _map_each_identity(v): - return v - -def _create_stage2_bootstrap( - ctx, - *, - output_prefix, - output_sibling, - main_py, - imports, - runtime_details): - output = ctx.actions.declare_file( - # Prepend with underscore to prevent pytest from trying to - # process the bootstrap for files starting with `test_` - "_{}_stage2_bootstrap.py".format(output_prefix), - sibling = output_sibling, - ) - runtime = runtime_details.effective_runtime - if (ctx.configuration.coverage_enabled and - runtime and - runtime.coverage_tool): - coverage_tool_runfiles_path = "{}/{}".format( - ctx.workspace_name, - runtime.coverage_tool.short_path, - ) - else: - coverage_tool_runfiles_path = "" - - template = runtime.stage2_bootstrap_template - - ctx.actions.expand_template( - template = template, - output = output, - substitutions = { - "%coverage_tool%": coverage_tool_runfiles_path, - "%import_all%": "True" if ctx.fragments.bazel_py.python_import_all_repositories else "False", - "%imports%": ":".join(imports.to_list()), - "%main%": "{}/{}".format(ctx.workspace_name, main_py.short_path), - "%target%": str(ctx.label), - "%workspace_name%": ctx.workspace_name, - }, - is_executable = True, - ) - return output - -def _runfiles_root_path(ctx, short_path): - """Compute a runfiles-root relative path from `File.short_path` - - Args: - ctx: current target ctx - short_path: str, a main-repo relative path from `File.short_path` - - Returns: - {type}`str`, a runflies-root relative path - """ - - # The ../ comes from short_path is for files in other repos. - if short_path.startswith("../"): - return short_path[3:] - else: - return "{}/{}".format(ctx.workspace_name, short_path) - -def _create_stage1_bootstrap( - ctx, - *, - output, - main_py = None, - stage2_bootstrap = None, - imports = None, - is_for_zip, - runtime_details, - venv = None): - runtime = runtime_details.effective_runtime - - if venv: - python_binary_path = _runfiles_root_path(ctx, venv.interpreter.short_path) - else: - python_binary_path = runtime_details.executable_interpreter_path - - if is_for_zip and venv: - python_binary_actual = venv.interpreter_actual_path - else: - python_binary_actual = "" - - subs = { - "%is_zipfile%": "1" if is_for_zip else "0", - "%python_binary%": python_binary_path, - "%python_binary_actual%": python_binary_actual, - "%target%": str(ctx.label), - "%workspace_name%": ctx.workspace_name, - } - - if stage2_bootstrap: - subs["%stage2_bootstrap%"] = "{}/{}".format( - ctx.workspace_name, - stage2_bootstrap.short_path, - ) - template = runtime.bootstrap_template - subs["%shebang%"] = runtime.stub_shebang - else: - if (ctx.configuration.coverage_enabled and - runtime and - runtime.coverage_tool): - coverage_tool_runfiles_path = "{}/{}".format( - ctx.workspace_name, - runtime.coverage_tool.short_path, - ) - else: - coverage_tool_runfiles_path = "" - if runtime: - subs["%shebang%"] = runtime.stub_shebang - template = runtime.bootstrap_template - else: - subs["%shebang%"] = DEFAULT_STUB_SHEBANG - template = ctx.file._bootstrap_template - - subs["%coverage_tool%"] = coverage_tool_runfiles_path - subs["%import_all%"] = ("True" if ctx.fragments.bazel_py.python_import_all_repositories else "False") - subs["%imports%"] = ":".join(imports.to_list()) - subs["%main%"] = "{}/{}".format(ctx.workspace_name, main_py.short_path) - - ctx.actions.expand_template( - template = template, - output = output, - substitutions = subs, - ) - -def _create_windows_exe_launcher( - ctx, - *, - output, - python_binary_path, - use_zip_file): - launch_info = ctx.actions.args() - launch_info.use_param_file("%s", use_always = True) - launch_info.set_param_file_format("multiline") - launch_info.add("binary_type=Python") - launch_info.add(ctx.workspace_name, format = "workspace_name=%s") - launch_info.add( - "1" if py_internal.runfiles_enabled(ctx) else "0", - format = "symlink_runfiles_enabled=%s", - ) - launch_info.add(python_binary_path, format = "python_bin_path=%s") - launch_info.add("1" if use_zip_file else "0", format = "use_zip_file=%s") - - launcher = ctx.attr._launcher[DefaultInfo].files_to_run.executable - ctx.actions.run( - executable = ctx.executable._windows_launcher_maker, - arguments = [launcher.path, launch_info, output.path], - inputs = [launcher], - outputs = [output], - mnemonic = "PyBuildLauncher", - progress_message = "Creating launcher for %{label}", - # Needed to inherit PATH when using non-MSVC compilers like MinGW - use_default_shell_env = True, - ) - -def _create_zip_file(ctx, *, output, original_nonzip_executable, zip_main, runfiles): - """Create a Python zipapp (zip with __main__.py entry point).""" - workspace_name = ctx.workspace_name - legacy_external_runfiles = _py_builtins.get_legacy_external_runfiles(ctx) - - manifest = ctx.actions.args() - manifest.use_param_file("@%s", use_always = True) - manifest.set_param_file_format("multiline") - - manifest.add("__main__.py={}".format(zip_main.path)) - manifest.add("__init__.py=") - manifest.add( - "{}=".format( - _get_zip_runfiles_path("__init__.py", workspace_name, legacy_external_runfiles), - ), - ) - for path in runfiles.empty_filenames.to_list(): - manifest.add("{}=".format(_get_zip_runfiles_path(path, workspace_name, legacy_external_runfiles))) - - def map_zip_runfiles(file): - if file != original_nonzip_executable and file != output: - return "{}={}".format( - _get_zip_runfiles_path(file.short_path, workspace_name, legacy_external_runfiles), - file.path, - ) - else: - return None - - manifest.add_all(runfiles.files, map_each = map_zip_runfiles, allow_closure = True) - - inputs = [zip_main] - if _py_builtins.is_bzlmod_enabled(ctx): - zip_repo_mapping_manifest = ctx.actions.declare_file( - output.basename + ".repo_mapping", - sibling = output, - ) - _py_builtins.create_repo_mapping_manifest( - ctx = ctx, - runfiles = runfiles, - output = zip_repo_mapping_manifest, - ) - manifest.add("{}/_repo_mapping={}".format( - _ZIP_RUNFILES_DIRECTORY_NAME, - zip_repo_mapping_manifest.path, - )) - inputs.append(zip_repo_mapping_manifest) - - for artifact in runfiles.files.to_list(): - # Don't include the original executable because it isn't used by the - # zip file, so no need to build it for the action. - # Don't include the zipfile itself because it's an output. - if artifact != original_nonzip_executable and artifact != output: - inputs.append(artifact) - - zip_cli_args = ctx.actions.args() - zip_cli_args.add("cC") - zip_cli_args.add(output) - - ctx.actions.run( - executable = ctx.executable._zipper, - arguments = [zip_cli_args, manifest], - inputs = depset(inputs), - outputs = [output], - use_default_shell_env = True, - mnemonic = "PythonZipper", - progress_message = "Building Python zip: %{label}", - ) - -def _get_zip_runfiles_path(path, workspace_name, legacy_external_runfiles): - if legacy_external_runfiles and path.startswith(_EXTERNAL_PATH_PREFIX): - zip_runfiles_path = paths.relativize(path, _EXTERNAL_PATH_PREFIX) - else: - # NOTE: External runfiles (artifacts in other repos) will have a leading - # path component of "../" so that they refer outside the main workspace - # directory and into the runfiles root. By normalizing, we simplify e.g. - # "workspace/../foo/bar" to simply "foo/bar". - zip_runfiles_path = paths.normalize("{}/{}".format(workspace_name, path)) - return "{}/{}".format(_ZIP_RUNFILES_DIRECTORY_NAME, zip_runfiles_path) - -def _create_executable_zip_file( - ctx, - *, - output, - zip_file, - stage2_bootstrap, - runtime_details, - venv): - prelude = ctx.actions.declare_file( - "{}_zip_prelude.sh".format(output.basename), - sibling = output, - ) - if stage2_bootstrap: - _create_stage1_bootstrap( - ctx, - output = prelude, - stage2_bootstrap = stage2_bootstrap, - runtime_details = runtime_details, - is_for_zip = True, - venv = venv, - ) - else: - ctx.actions.write(prelude, "#!/usr/bin/env python3\n") - - ctx.actions.run_shell( - command = "cat {prelude} {zip} > {output}".format( - prelude = prelude.path, - zip = zip_file.path, - output = output.path, - ), - inputs = [prelude, zip_file], - outputs = [output], - use_default_shell_env = True, - mnemonic = "PyBuildExecutableZip", - progress_message = "Build Python zip executable: %{label}", - ) - -def _get_cc_details_for_binary(ctx, extra_deps): - cc_info = collect_cc_info(ctx, extra_deps = extra_deps) - return create_cc_details_struct( - cc_info_for_propagating = cc_info, - cc_info_for_self_link = cc_info, - cc_info_with_extra_link_time_libraries = None, - extra_runfiles = ctx.runfiles(), - # Though the rules require the CcToolchain, it isn't actually used. - cc_toolchain = None, - feature_config = None, - ) - -def _get_interpreter_path(ctx, *, runtime, flag_interpreter_path): - if runtime: - if runtime.interpreter_path: - interpreter_path = runtime.interpreter_path - else: - interpreter_path = "{}/{}".format( - ctx.workspace_name, - runtime.interpreter.short_path, - ) - - # NOTE: External runfiles (artifacts in other repos) will have a - # leading path component of "../" so that they refer outside the - # main workspace directory and into the runfiles root. By - # normalizing, we simplify e.g. "workspace/../foo/bar" to simply - # "foo/bar" - interpreter_path = paths.normalize(interpreter_path) - - elif flag_interpreter_path: - interpreter_path = flag_interpreter_path - else: - fail("Unable to determine interpreter path") - - return interpreter_path - -def _get_native_deps_dso_name(ctx): - _ = ctx # @unused - fail("Building native deps DSO not supported.") - -def _get_native_deps_user_link_flags(ctx): - _ = ctx # @unused - fail("Building native deps DSO not supported.") diff --git a/python/private/py_library_rule.bzl b/python/private/py_library_rule.bzl index ed64716122..8a8d6cf380 100644 --- a/python/private/py_library_rule.bzl +++ b/python/private/py_library_rule.bzl @@ -13,8 +13,8 @@ # limitations under the License. """Implementation of py_library rule.""" -load(":common.bzl", "create_library_semantics_struct") -load(":common_bazel.bzl", "collect_cc_info", "get_imports", "maybe_precompile") +load(":common.bzl", "collect_cc_info", "create_library_semantics_struct", "get_imports") +load(":precompile.bzl", "maybe_precompile") load(":py_library.bzl", "create_py_library_rule", "py_library_impl") def _py_library_impl_with_semantics(ctx): diff --git a/python/private/py_test_macro.bzl b/python/private/py_test_macro.bzl index 1f9330f8e5..348e877225 100644 --- a/python/private/py_test_macro.bzl +++ b/python/private/py_test_macro.bzl @@ -13,7 +13,7 @@ # limitations under the License. """Implementation of macro-half of py_test rule.""" -load(":common_bazel.bzl", "convert_legacy_create_init_to_int") +load(":py_executable.bzl", "convert_legacy_create_init_to_int") load(":py_test_rule.bzl", py_test_rule = "py_test") def py_test(**kwargs): diff --git a/python/private/py_test_rule.bzl b/python/private/py_test_rule.bzl index 64d5f21f81..63000c7255 100644 --- a/python/private/py_test_rule.bzl +++ b/python/private/py_test_rule.bzl @@ -17,9 +17,9 @@ load("@bazel_skylib//lib:dicts.bzl", "dicts") load(":attributes.bzl", "AGNOSTIC_TEST_ATTRS") load(":common.bzl", "maybe_add_test_execution_info") load( - ":py_executable_bazel.bzl", + ":py_executable.bzl", "create_executable_rule", - "py_executable_bazel_impl", + "py_executable_impl", ) _BAZEL_PY_TEST_ATTRS = { @@ -40,7 +40,7 @@ _BAZEL_PY_TEST_ATTRS = { } def _py_test_impl(ctx): - providers = py_executable_bazel_impl( + providers = py_executable_impl( ctx = ctx, is_test = True, inherited_environment = ctx.attr.env_inherit, diff --git a/tests/bootstrap_impls/venv_relative_path_tests.bzl b/tests/bootstrap_impls/venv_relative_path_tests.bzl index b21f220205..ad4870fe08 100644 --- a/tests/bootstrap_impls/venv_relative_path_tests.bzl +++ b/tests/bootstrap_impls/venv_relative_path_tests.bzl @@ -15,7 +15,7 @@ "Unit tests for relative_path computation" load("@rules_testing//lib:test_suite.bzl", "test_suite") -load("//python/private:py_executable_bazel.bzl", "relative_path") # buildifier: disable=bzl-visibility +load("//python/private:py_executable.bzl", "relative_path") # buildifier: disable=bzl-visibility _tests = [] From b5729b41ef18393c2609aa1633607695175a7419 Mon Sep 17 00:00:00 2001 From: Niko Wenselowski Date: Tue, 24 Dec 2024 01:34:04 +0100 Subject: [PATCH 036/922] feat(toolchain): Add support for Python 3.13.1. (#2482) Adds 3.13.1 Python toolchain. Also updates 3.13 to map to 3.13.1. Tests have been slightly altered to make spotting the override easier. --------- Co-authored-by: Richard Levasseur Co-authored-by: Ignas Anikevicius <240938+aignas@users.noreply.github.com> --- CHANGELOG.md | 9 +++ python/versions.bzl | 103 ++++++++++++++++++++++++++++++++-- tests/python/python_tests.bzl | 19 ++++--- 3 files changed, 118 insertions(+), 13 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 9a3436487e..eb4bcfa8da 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -52,6 +52,7 @@ Unreleased changes template. {#v0-0-0-changed} ### Changed +* (toolchains) 3.13 means 3.13.1 (previously 3.13.0) * Bazel 6 support is dropped and Bazel 7.4.1 is the minimum supported version, per our Bazel support matrix. Earlier versions are not tested by CI, so functionality cannot be guaranteed. @@ -88,6 +89,14 @@ Unreleased changes template. To select the free-threaded interpreter in the repo phase, please use the documented [env](/environment-variables.html) variables. Fixes [#2386](https://github.com/bazelbuild/rules_python/issues/2386). +* (toolchains) Use the latest astrahl-sh toolchain release [20241206] for Python versions: + * 3.9.21 + * 3.10.16 + * 3.11.11 + * 3.12.8 + * 3.13.1 + +[20241206]: https://github.com/astral-sh/python-build-standalone/releases/tag/20241206 {#v0-0-0-removed} ### Removed diff --git a/python/versions.bzl b/python/versions.bzl index 1fd0649f12..098362b7d3 100644 --- a/python/versions.bzl +++ b/python/versions.bzl @@ -252,6 +252,20 @@ TOOL_VERSIONS = { }, "strip_prefix": "python", }, + "3.9.21": { + "url": "20241206/cpython-{python_version}+20241206-{platform}-{build}.tar.gz", + "sha256": { + "aarch64-apple-darwin": "4bddc18228789d0316dcebc45b2242e0010fa6bc33c302b6b5a62a5ac39d2147", + "aarch64-unknown-linux-gnu": "7d3b4ab90f73fa9dab0c350ca64b1caa9b8e4655913acd098e594473c49921c8", + "ppc64le-unknown-linux-gnu": "966477345ca93f056cf18de9cff961aacda2318a8e641546e0fd7222f1362ee2", + "s390x-unknown-linux-gnu": "3ba05a408edce4e20ebd116643c8418e62f7c8066c8a35fe8d3b78371d90b46a", + "x86_64-apple-darwin": "619f5082288c771ad9b71e2daaf6df6bd39ca86e442638d150a71a6ccf62978d", + "x86_64-pc-windows-msvc": "82736b5a185c57b296188ce778ed865ff10edc5fe9ff1ec4cb33b39ac8e4819c", + "x86_64-unknown-linux-gnu": "208b2adc7c7e5d5df6d9385400dc7c4e3b4c3eed428e19a2326848978e98517e", + "x86_64-unknown-linux-musl": "67c058dbaae8fd8c4f68e13b10805a9227918afc94326f21a9a2ec2daca3ddbd", + }, + "strip_prefix": "python", + }, "3.10.2": { "url": "20220227/cpython-{python_version}+20220227-{platform}-{build}.tar.gz", "sha256": { @@ -372,6 +386,20 @@ TOOL_VERSIONS = { }, "strip_prefix": "python", }, + "3.10.16": { + "url": "20241206/cpython-{python_version}+20241206-{platform}-{build}.tar.gz", + "sha256": { + "aarch64-apple-darwin": "c2d25840756127f3583b04b0697bef79edacb15f1402cd980292c93488c3df22", + "aarch64-unknown-linux-gnu": "bbfc345615c5ed33916b4fd959fc16fa2e896a3c5eec1fb782c91b47c85c0542", + "ppc64le-unknown-linux-gnu": "cb474b392733d5ac2adaa1cfcc2b63b957611dc26697e76822706cc61ac21515", + "s390x-unknown-linux-gnu": "886a7effc8a3061d53cacc9cf54e82d6d57ac3665c258c6a2193528c16b557cd", + "x86_64-apple-darwin": "31a110b631eb79103675ed556255045deeea5ff533296d7f35b4d195a0df0315", + "x86_64-pc-windows-msvc": "fb7870717dc7e3aedcbab4a647782637da0046a4238db1d41eeaabb78566d814", + "x86_64-unknown-linux-gnu": "b15de0d63eed9871ed57285f81fd123cf6c4117251a9cac8f81f9cf0cccc0a53", + "x86_64-unknown-linux-musl": "bf956eeffcff002d2f38232faa750c279cbb76197b744761d1b253bf94d6f637", + }, + "strip_prefix": "python", + }, "3.11.1": { "url": "20230116/cpython-{python_version}+20230116-{platform}-{build}.tar.gz", "sha256": { @@ -487,6 +515,20 @@ TOOL_VERSIONS = { }, "strip_prefix": "python", }, + "3.11.11": { + "url": "20241206/cpython-{python_version}+20241206-{platform}-{build}.tar.gz", + "sha256": { + "aarch64-apple-darwin": "566c5e266f2c933d0c0b213a75496bc6a090e493097802f809dbe21c75cd5d13", + "aarch64-unknown-linux-gnu": "50ee364cfa24ee7d933eda955c9fe455bc0a8ebb9d998c9948f2909dac701dd9", + "ppc64le-unknown-linux-gnu": "e0cdc00e42a05191b9b75ba976fc0fca9205c66fdaef7571c20532346fd3db1e", + "s390x-unknown-linux-gnu": "3b106b8a3c5aa97ff76200cd0d9ba6eaed23d88ccb947e00ff6bb2d9f5422d2a", + "x86_64-apple-darwin": "8ecd267281fb5b2464ddcd2de79622cfa7aff42e929b17989da2721ba39d4a5e", + "x86_64-pc-windows-msvc": "d8986f026599074ddd206f3f62d6f2c323ca8fa7a854bf744989bfc0b12f5d0d", + "x86_64-unknown-linux-gnu": "57a171af687c926c5cabe3d1c7ce9950b98f00b932accd596eb60e14ca39c42d", + "x86_64-unknown-linux-musl": "8129a9a5c3f2654e1a9eed6093f5dc42399667b341050ff03219cb7df210c348", + }, + "strip_prefix": "python", + }, "3.12.0": { "url": "20231002/cpython-{python_version}+20231002-{platform}-{build}.tar.gz", "sha256": { @@ -566,6 +608,20 @@ TOOL_VERSIONS = { }, "strip_prefix": "python", }, + "3.12.8": { + "url": "20241206/cpython-{python_version}+20241206-{platform}-{build}.tar.gz", + "sha256": { + "aarch64-apple-darwin": "e3c4aa607717b23903ca2650d5c3ee24f89b97543e2db2b0f463bddc7a9e92f3", + "aarch64-unknown-linux-gnu": "ce674b55442b732973afb2932c281bb1ded4ad7e22bcf9b07071165770758c7e", + "ppc64le-unknown-linux-gnu": "b7214790b273de9ed0532420054b72ba1393d62d2fc844ec55ade193771bd90c", + "s390x-unknown-linux-gnu": "73102f5dbd7d1e7e9c2f2c80aedf2893d99a7fa407f6674ec8b2f57ba07daee5", + "x86_64-apple-darwin": "3ba35c706577d755e8e52a4c161a042464577c0e695e2a605362fa469e26de10", + "x86_64-pc-windows-msvc": "767b4be3ddf6b99e5ade519789c1615c191d8cf99d5aff4685cc18b48931f1e6", + "x86_64-unknown-linux-gnu": "b9d6ee5ddac1198e72d53112698773fc8bb597de095592eb849ca794306699ba", + "x86_64-unknown-linux-musl": "6f305888703691dd04cfff85284d23ea0b0146ed7c4415e472f1fb72b3f32cdf", + }, + "strip_prefix": "python", + }, "3.13.0": { "url": "20241016/cpython-{python_version}+20241016-{platform}-{build}.{ext}", "sha256": { @@ -603,16 +659,53 @@ TOOL_VERSIONS = { "x86_64-unknown-linux-gnu-freethreaded": "python/install", }, }, + "3.13.1": { + "url": "20241205/cpython-{python_version}+20241205-{platform}-{build}.{ext}", + "sha256": { + "aarch64-apple-darwin": "88b88b609129c12f4b3841845aca13230f61e97ba97bd0fb28ee64b0e442a34f", + "aarch64-unknown-linux-gnu": "fdfa86c2746d2ae700042c461846e6c37f70c249925b58de8cd02eb8d1423d4e", + "ppc64le-unknown-linux-gnu": "27b20b3237c55430ca1304e687d021f88373f906249f9cd272c5ff2803d5e5c3", + "s390x-unknown-linux-gnu": "7d0187e20cb5e36c689eec27e4d3de56d8b7f1c50dc5523550fc47377801521f", + "x86_64-apple-darwin": "47eef6efb8664e2d1d23a7cdaf56262d784f8ace48f3bfca1b183e95a49888d6", + "x86_64-pc-windows-msvc": "f51f0493a5f979ff0b8d8c598a8d74f2a4d86a190c2729c85e0af65c36a9cbbe", + "x86_64-unknown-linux-gnu": "242b2727df6c1e00de6a9f0f0dcb4562e168d27f428c785b0eb41a6aeb34d69a", + "x86_64-unknown-linux-musl": "76b30c6373b9c0aa2ba610e07da02f384aa210ac79643da38c66d3e6171c6ef5", + "aarch64-apple-darwin-freethreaded": "08f05618bdcf8064a7960b25d9ba92155447c9b08e0cf2f46a981e4c6a1bb5a5", + "aarch64-unknown-linux-gnu-freethreaded": "9f2fcb809f9ba6c7c014a8803073a88786701a98971135bce684355062e4bb35", + "ppc64le-unknown-linux-gnu-freethreaded": "15ceea78dff78ca8ccaac8d9c54b808af30daaa126f1f561e920a6896e098634", + "s390x-unknown-linux-gnu-freethreaded": "ed3c6118d1d12603309c930e93421ac7a30a69045ffd43006f63ecf71d72c317", + "x86_64-apple-darwin-freethreaded": "dc780fecd215d2cc9e573abf1e13a175fcfa8f6efd100ef888494a248a16cda8", + "x86_64-pc-windows-msvc-freethreaded": "7537b2ab361c0eabc0eabfca9ffd9862d7f5f6576eda13b97e98aceb5eea4fd3", + "x86_64-unknown-linux-gnu-freethreaded": "9ec1b81213f849d91f5ebe6a16196e85cd6ff7c05ca823ce0ab7ba5b0e9fee84", + }, + "strip_prefix": { + "aarch64-apple-darwin": "python", + "aarch64-unknown-linux-gnu": "python", + "ppc64le-unknown-linux-gnu": "python", + "s390x-unknown-linux-gnu": "python", + "x86_64-apple-darwin": "python", + "x86_64-pc-windows-msvc": "python", + "x86_64-unknown-linux-gnu": "python", + "x86_64-unknown-linux-musl": "python", + "aarch64-apple-darwin-freethreaded": "python/install", + "aarch64-unknown-linux-gnu-freethreaded": "python/install", + "ppc64le-unknown-linux-gnu-freethreaded": "python/install", + "s390x-unknown-linux-gnu-freethreaded": "python/install", + "x86_64-apple-darwin-freethreaded": "python/install", + "x86_64-pc-windows-msvc-freethreaded": "python/install", + "x86_64-unknown-linux-gnu-freethreaded": "python/install", + }, + }, } # buildifier: disable=unsorted-dict-items MINOR_MAPPING = { "3.8": "3.8.20", - "3.9": "3.9.20", - "3.10": "3.10.15", - "3.11": "3.11.10", - "3.12": "3.12.7", - "3.13": "3.13.0", + "3.9": "3.9.21", + "3.10": "3.10.16", + "3.11": "3.11.11", + "3.12": "3.12.8", + "3.13": "3.13.1", } def _generate_platforms(): diff --git a/tests/python/python_tests.bzl b/tests/python/python_tests.bzl index 40504302d1..e7828b92f5 100644 --- a/tests/python/python_tests.bzl +++ b/tests/python/python_tests.bzl @@ -413,7 +413,7 @@ def _test_add_new_version(env): strip_prefix = "python", platform = "aarch64-unknown-linux-gnu", coverage_tool = "specific_cov_tool", - python_version = "3.13.1", + python_version = "3.13.99", patch_strip = 2, patches = ["specific-patch.txt"], ), @@ -421,9 +421,9 @@ def _test_add_new_version(env): override = [ _override( base_url = "", - available_python_versions = ["3.12.4", "3.13.0", "3.13.1"], + available_python_versions = ["3.12.4", "3.13.0", "3.13.1", "3.13.99"], minor_mapping = { - "3.13": "3.13.0", + "3.13": "3.13.99", }, ), ], @@ -436,13 +436,14 @@ def _test_add_new_version(env): "3.12.4", "3.13.0", "3.13.1", + "3.13.99", ]) env.expect.that_dict(py.config.default["tool_versions"]["3.13.0"]).contains_exactly({ "sha256": {"aarch64-unknown-linux-gnu": "deadbeef"}, "strip_prefix": {"aarch64-unknown-linux-gnu": "prefix"}, "url": {"aarch64-unknown-linux-gnu": ["example.org"]}, }) - env.expect.that_dict(py.config.default["tool_versions"]["3.13.1"]).contains_exactly({ + env.expect.that_dict(py.config.default["tool_versions"]["3.13.99"]).contains_exactly({ "coverage_tool": {"aarch64-unknown-linux-gnu": "specific_cov_tool"}, "patch_strip": {"aarch64-unknown-linux-gnu": 2}, "patches": {"aarch64-unknown-linux-gnu": ["specific-patch.txt"]}, @@ -452,7 +453,7 @@ def _test_add_new_version(env): }) env.expect.that_dict(py.config.minor_mapping).contains_exactly({ "3.12": "3.12.4", # The `minor_mapping` will be overriden only for the missing keys - "3.13": "3.13.0", + "3.13": "3.13.99", }) env.expect.that_collection(py.toolchains).contains_exactly([ struct( @@ -484,13 +485,13 @@ def _test_register_all_versions(env): sha256 = "deadb00f", urls = ["something.org"], platform = "aarch64-unknown-linux-gnu", - python_version = "3.13.1", + python_version = "3.13.99", ), ], override = [ _override( base_url = "", - available_python_versions = ["3.12.4", "3.13.0", "3.13.1"], + available_python_versions = ["3.12.4", "3.13.0", "3.13.1", "3.13.99"], register_all_versions = True, ), ], @@ -503,11 +504,12 @@ def _test_register_all_versions(env): "3.12.4", "3.13.0", "3.13.1", + "3.13.99", ]) env.expect.that_dict(py.config.minor_mapping).contains_exactly({ # The mapping is calculated automatically "3.12": "3.12.4", - "3.13": "3.13.1", + "3.13": "3.13.99", }) env.expect.that_collection(py.toolchains).contains_exactly([ struct( @@ -521,6 +523,7 @@ def _test_register_all_versions(env): "python_3_13": "3.13", "python_3_13_0": "3.13.0", "python_3_13_1": "3.13.1", + "python_3_13_99": "3.13.99", }.items() ]) From 922929b6b7f8e9426e4a8d29aaebada9a4d14599 Mon Sep 17 00:00:00 2001 From: Ignas Anikevicius <240938+aignas@users.noreply.github.com> Date: Tue, 24 Dec 2024 16:12:25 +0900 Subject: [PATCH 037/922] refactor: stop warning if we don't find anything via SimpleAPI (#2532) The warning is somewhat non-actionable and the sources can be inspected via the MODULE.bazel.lock file if needed. This makes it easier to make this option a default at some point. At the same time cleanup the code since we are not using the `get_index_urls` to print the warning. Work towards #260 --- python/private/pypi/extension.bzl | 119 +++++++++------------ python/private/pypi/parse_requirements.bzl | 5 +- 2 files changed, 56 insertions(+), 68 deletions(-) diff --git a/python/private/pypi/extension.bzl b/python/private/pypi/extension.bzl index 9b150bdce0..e1904912fd 100644 --- a/python/private/pypi/extension.bzl +++ b/python/private/pypi/extension.bzl @@ -105,7 +105,6 @@ def _create_whl_repos( # containers to aggregate outputs from this function whl_map = {} - exposed_packages = {} extra_aliases = { whl_name: {alias: True for alias in aliases} for whl_name, aliases in pip_attr.extra_hub_aliases.items() @@ -219,8 +218,6 @@ def _create_whl_repos( ) for whl_name, requirements in requirements_by_platform.items(): - whl_name = normalize_name(whl_name) - group_name = whl_group_mapping.get(whl_name) group_deps = requirement_cycles.get(group_name, []) @@ -261,68 +258,55 @@ def _create_whl_repos( if v != default }) - is_exposed = False - if get_index_urls: - # TODO @aignas 2024-05-26: move to a separate function - found_something = False - for requirement in requirements: - is_exposed = is_exposed or requirement.is_exposed - dists = requirement.whls - if not pip_attr.download_only and requirement.sdist: - dists = dists + [requirement.sdist] - - for distribution in dists: - found_something = True - is_reproducible = False - - args = dict(whl_library_args) - if pip_attr.netrc: - args["netrc"] = pip_attr.netrc - if pip_attr.auth_patterns: - args["auth_patterns"] = pip_attr.auth_patterns - - if not distribution.filename.endswith(".whl"): - # pip is not used to download wheels and the python - # `whl_library` helpers are only extracting things, however - # for sdists, they will be built by `pip`, so we still - # need to pass the extra args there. - args["extra_pip_args"] = requirement.extra_pip_args - - # This is no-op because pip is not used to download the wheel. - args.pop("download_only", None) - - repo_name = whl_repo_name(pip_name, distribution.filename, distribution.sha256) - args["requirement"] = requirement.srcs.requirement - args["urls"] = [distribution.url] - args["sha256"] = distribution.sha256 - args["filename"] = distribution.filename - args["experimental_target_platforms"] = requirement.target_platforms - - # Pure python wheels or sdists may need to have a platform here - target_platforms = None - if distribution.filename.endswith("-any.whl") or not distribution.filename.endswith(".whl"): - if len(requirements) > 1: - target_platforms = requirement.target_platforms - - whl_libraries[repo_name] = args - - whl_map.setdefault(whl_name, {})[whl_config_setting( - version = major_minor, - filename = distribution.filename, - target_platforms = target_platforms, - )] = repo_name - - if found_something: - if is_exposed: - exposed_packages[whl_name] = None - continue - - is_exposed = False + # TODO @aignas 2024-05-26: move to a separate function for requirement in requirements: - is_exposed = is_exposed or requirement.is_exposed - if get_index_urls: - logger.warn(lambda: "falling back to pip for installing the right file for {}".format(requirement.srcs.requirement_line)) + dists = requirement.whls + if not pip_attr.download_only and requirement.sdist: + dists = dists + [requirement.sdist] + + for distribution in dists: + args = dict(whl_library_args) + if pip_attr.netrc: + args["netrc"] = pip_attr.netrc + if pip_attr.auth_patterns: + args["auth_patterns"] = pip_attr.auth_patterns + + if not distribution.filename.endswith(".whl"): + # pip is not used to download wheels and the python + # `whl_library` helpers are only extracting things, however + # for sdists, they will be built by `pip`, so we still + # need to pass the extra args there. + args["extra_pip_args"] = requirement.extra_pip_args + + # This is no-op because pip is not used to download the wheel. + args.pop("download_only", None) + + repo_name = whl_repo_name(pip_name, distribution.filename, distribution.sha256) + args["requirement"] = requirement.srcs.requirement + args["urls"] = [distribution.url] + args["sha256"] = distribution.sha256 + args["filename"] = distribution.filename + args["experimental_target_platforms"] = requirement.target_platforms + + # Pure python wheels or sdists may need to have a platform here + target_platforms = None + if distribution.filename.endswith("-any.whl") or not distribution.filename.endswith(".whl"): + if len(requirements) > 1: + target_platforms = requirement.target_platforms + + whl_libraries[repo_name] = args + + whl_map.setdefault(whl_name, {})[whl_config_setting( + version = major_minor, + filename = distribution.filename, + target_platforms = target_platforms, + )] = repo_name + + if dists: + is_reproducible = False + continue + # Fallback to a pip-installed wheel args = dict(whl_library_args) # make a copy args["requirement"] = requirement.srcs.requirement_line if requirement.extra_pip_args: @@ -343,13 +327,14 @@ def _create_whl_repos( target_platforms = target_platforms or None, )] = repo_name - if is_exposed: - exposed_packages[whl_name] = None - return struct( is_reproducible = is_reproducible, whl_map = whl_map, - exposed_packages = exposed_packages, + exposed_packages = { + whl_name: None + for whl_name, requirements in requirements_by_platform.items() + if len([r for r in requirements if r.is_exposed]) > 0 + }, extra_aliases = extra_aliases, whl_libraries = whl_libraries, ) diff --git a/python/private/pypi/parse_requirements.bzl b/python/private/pypi/parse_requirements.bzl index 821913d6de..d7ee285c0e 100644 --- a/python/private/pypi/parse_requirements.bzl +++ b/python/private/pypi/parse_requirements.bzl @@ -203,6 +203,9 @@ def parse_requirements( sorted(requirements), )) + # Return normalized names + ret_requirements = ret.setdefault(normalize_name(whl_name), []) + for r in sorted(reqs.values(), key = lambda r: r.requirement_line): whls, sdist = _add_dists( requirement = r, @@ -211,7 +214,7 @@ def parse_requirements( ) target_platforms = env_marker_target_platforms.get(r.requirement_line, r.target_platforms) - ret.setdefault(whl_name, []).append( + ret_requirements.append( struct( distribution = r.distribution, srcs = r.srcs, From a632044354054449f4e1534d24fdf4bd95a95a1f Mon Sep 17 00:00:00 2001 From: Ignas Anikevicius <240938+aignas@users.noreply.github.com> Date: Fri, 27 Dec 2024 16:37:55 +0900 Subject: [PATCH 038/922] feat(pypi): only query SimpleAPI for pkgs that have shas (#2527) Later in the code we would only use the results of SimpleAPI if the package has shas, so actually doing these calls is just wasting time, because we would be dropping the results anyway. Work towards #2100 --- CHANGELOG.md | 2 ++ python/private/pypi/parse_requirements.bzl | 1 + tests/pypi/extension/extension_tests.bzl | 2 +- 3 files changed, 4 insertions(+), 1 deletion(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index eb4bcfa8da..fd5d455147 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -56,6 +56,8 @@ Unreleased changes template. * Bazel 6 support is dropped and Bazel 7.4.1 is the minimum supported version, per our Bazel support matrix. Earlier versions are not tested by CI, so functionality cannot be guaranteed. +* ({bzl:obj}`pip.parse`) Only query SimpleAPI for packages that have + sha values in the `requirements.txt` file. {#v0-0-0-fixed} ### Fixed diff --git a/python/private/pypi/parse_requirements.bzl b/python/private/pypi/parse_requirements.bzl index d7ee285c0e..2bca8d8621 100644 --- a/python/private/pypi/parse_requirements.bzl +++ b/python/private/pypi/parse_requirements.bzl @@ -184,6 +184,7 @@ def parse_requirements( req.distribution: None for reqs in requirements_by_platform.values() for req in reqs.values() + if req.srcs.shas }), ) diff --git a/tests/pypi/extension/extension_tests.bzl b/tests/pypi/extension/extension_tests.bzl index 1caab23cea..5916a27e98 100644 --- a/tests/pypi/extension/extension_tests.bzl +++ b/tests/pypi/extension/extension_tests.bzl @@ -575,7 +575,7 @@ some_pkg==0.0.1 index_url = "pypi.org", index_url_overrides = {}, netrc = None, - sources = ["simple", "some_pkg"], + sources = ["simple"], ), "cache": {}, "parallel_download": False, From 21362155a181f6c3825676ae9351ef2cadde9cd4 Mon Sep 17 00:00:00 2001 From: Ignas Anikevicius <240938+aignas@users.noreply.github.com> Date: Fri, 27 Dec 2024 16:39:25 +0900 Subject: [PATCH 039/922] refactor(pypi): further cleanup of `pip.parse` code (#2534) Summary: - Move the `whl_library` creation into a separate function and remove the `TODO` note. - Move the creation of the `get_index_urls` functions into outer `parse_modules` function and simplify the reproducible extension setting logic. - Remove the `prefix` parameter from the `*repo_name` functions. - Add an extra error message, for ensuring that invariants are met. Work towards #260 --- python/private/pypi/extension.bzl | 213 ++++++++++-------- python/private/pypi/whl_repo_name.bzl | 9 +- .../whl_repo_name/whl_repo_name_tests.bzl | 11 +- 3 files changed, 124 insertions(+), 109 deletions(-) diff --git a/python/private/pypi/extension.bzl b/python/private/pypi/extension.bzl index e1904912fd..d16a7cce2f 100644 --- a/python/private/pypi/extension.bzl +++ b/python/private/pypi/extension.bzl @@ -67,20 +67,17 @@ def _create_whl_repos( *, pip_attr, whl_overrides, - simpleapi_cache, evaluate_markers = evaluate_markers, available_interpreters = INTERPRETER_LABELS, - simpleapi_download = simpleapi_download): + get_index_urls = None): """create all of the whl repositories Args: module_ctx: {type}`module_ctx`. pip_attr: {type}`struct` - the struct that comes from the tag class iteration. whl_overrides: {type}`dict[str, struct]` - per-wheel overrides. - simpleapi_cache: {type}`dict` - an opaque dictionary used for caching the results from calling - SimpleAPI evaluating all of the tag class invocations {bzl:obj}`pip.parse`. evaluate_markers: the function to use to evaluate markers. - simpleapi_download: Used for testing overrides + get_index_urls: A function used to get the index URLs available_interpreters: {type}`dict[str, Label]` The dictionary of available interpreters that have been registered using the `python` bzlmod extension. The keys are in the form `python_{snake_case_version}_host`. This is to be @@ -96,12 +93,9 @@ def _create_whl_repos( aparent repository names for the hub repo and the values are the arguments that will be passed to {bzl:obj}`whl_library` repository rule. - is_reproducible: {type}`bool` set to True if does not make calls to the - internet to evaluate the requirements files. """ logger = repo_utils.logger(module_ctx, "pypi:create_whl_repos") python_interpreter_target = pip_attr.python_interpreter_target - is_reproducible = True # containers to aggregate outputs from this function whl_map = {} @@ -158,26 +152,6 @@ def _create_whl_repos( whl_group_mapping = {} requirement_cycles = {} - # Create a new wheel library for each of the different whls - - get_index_urls = None - if pip_attr.experimental_index_url: - get_index_urls = lambda ctx, distributions: simpleapi_download( - ctx, - attr = struct( - index_url = pip_attr.experimental_index_url, - extra_index_urls = pip_attr.experimental_extra_index_urls or [], - index_url_overrides = pip_attr.experimental_index_url_overrides or {}, - sources = distributions, - envsubst = pip_attr.envsubst, - # Auth related info - netrc = pip_attr.netrc, - auth_patterns = pip_attr.auth_patterns, - ), - cache = simpleapi_cache, - parallel_download = pip_attr.parallel_download, - ) - requirements_by_platform = parse_requirements( module_ctx, requirements_by_platform = requirements_files_by_platform( @@ -258,77 +232,27 @@ def _create_whl_repos( if v != default }) - # TODO @aignas 2024-05-26: move to a separate function for requirement in requirements: - dists = requirement.whls - if not pip_attr.download_only and requirement.sdist: - dists = dists + [requirement.sdist] - - for distribution in dists: - args = dict(whl_library_args) - if pip_attr.netrc: - args["netrc"] = pip_attr.netrc - if pip_attr.auth_patterns: - args["auth_patterns"] = pip_attr.auth_patterns - - if not distribution.filename.endswith(".whl"): - # pip is not used to download wheels and the python - # `whl_library` helpers are only extracting things, however - # for sdists, they will be built by `pip`, so we still - # need to pass the extra args there. - args["extra_pip_args"] = requirement.extra_pip_args - - # This is no-op because pip is not used to download the wheel. - args.pop("download_only", None) - - repo_name = whl_repo_name(pip_name, distribution.filename, distribution.sha256) - args["requirement"] = requirement.srcs.requirement - args["urls"] = [distribution.url] - args["sha256"] = distribution.sha256 - args["filename"] = distribution.filename - args["experimental_target_platforms"] = requirement.target_platforms - - # Pure python wheels or sdists may need to have a platform here - target_platforms = None - if distribution.filename.endswith("-any.whl") or not distribution.filename.endswith(".whl"): - if len(requirements) > 1: - target_platforms = requirement.target_platforms + for repo_name, (args, config_setting) in _whl_repos( + requirement = requirement, + whl_library_args = whl_library_args, + download_only = pip_attr.download_only, + netrc = pip_attr.netrc, + auth_patterns = pip_attr.auth_patterns, + python_version = major_minor, + multiple_requirements_for_whl = len(requirements) > 1., + ).items(): + repo_name = "{}_{}".format(pip_name, repo_name) + if repo_name in whl_libraries: + fail("Attempting to creating a duplicate library {} for {}".format( + repo_name, + whl_name, + )) whl_libraries[repo_name] = args - - whl_map.setdefault(whl_name, {})[whl_config_setting( - version = major_minor, - filename = distribution.filename, - target_platforms = target_platforms, - )] = repo_name - - if dists: - is_reproducible = False - continue - - # Fallback to a pip-installed wheel - args = dict(whl_library_args) # make a copy - args["requirement"] = requirement.srcs.requirement_line - if requirement.extra_pip_args: - args["extra_pip_args"] = requirement.extra_pip_args - - if pip_attr.download_only: - args.setdefault("experimental_target_platforms", requirement.target_platforms) - - target_platforms = requirement.target_platforms if len(requirements) > 1 else [] - repo_name = pypi_repo_name( - pip_name, - whl_name, - *target_platforms - ) - whl_libraries[repo_name] = args - whl_map.setdefault(whl_name, {})[whl_config_setting( - version = major_minor, - target_platforms = target_platforms or None, - )] = repo_name + whl_map.setdefault(whl_name, {})[config_setting] = repo_name return struct( - is_reproducible = is_reproducible, whl_map = whl_map, exposed_packages = { whl_name: None @@ -339,11 +263,88 @@ def _create_whl_repos( whl_libraries = whl_libraries, ) -def parse_modules(module_ctx, _fail = fail, **kwargs): +def _whl_repos(*, requirement, whl_library_args, download_only, netrc, auth_patterns, multiple_requirements_for_whl = False, python_version): + ret = {} + + dists = requirement.whls + if not download_only and requirement.sdist: + dists = dists + [requirement.sdist] + + for distribution in dists: + args = dict(whl_library_args) + if netrc: + args["netrc"] = netrc + if auth_patterns: + args["auth_patterns"] = auth_patterns + + if not distribution.filename.endswith(".whl"): + # pip is not used to download wheels and the python + # `whl_library` helpers are only extracting things, however + # for sdists, they will be built by `pip`, so we still + # need to pass the extra args there. + args["extra_pip_args"] = requirement.extra_pip_args + + # This is no-op because pip is not used to download the wheel. + args.pop("download_only", None) + + args["requirement"] = requirement.srcs.requirement + args["urls"] = [distribution.url] + args["sha256"] = distribution.sha256 + args["filename"] = distribution.filename + args["experimental_target_platforms"] = requirement.target_platforms + + # Pure python wheels or sdists may need to have a platform here + target_platforms = None + if distribution.filename.endswith("-any.whl") or not distribution.filename.endswith(".whl"): + if multiple_requirements_for_whl: + target_platforms = requirement.target_platforms + + repo_name = whl_repo_name( + distribution.filename, + distribution.sha256, + ) + ret[repo_name] = ( + args, + whl_config_setting( + version = python_version, + filename = distribution.filename, + target_platforms = target_platforms, + ), + ) + + if ret: + return ret + + # Fallback to a pip-installed wheel + args = dict(whl_library_args) # make a copy + args["requirement"] = requirement.srcs.requirement_line + if requirement.extra_pip_args: + args["extra_pip_args"] = requirement.extra_pip_args + + if download_only: + args.setdefault("experimental_target_platforms", requirement.target_platforms) + + target_platforms = requirement.target_platforms if multiple_requirements_for_whl else [] + repo_name = pypi_repo_name( + normalize_name(requirement.distribution), + *target_platforms + ) + ret[repo_name] = ( + args, + whl_config_setting( + version = python_version, + target_platforms = target_platforms or None, + ), + ) + + return ret + +def parse_modules(module_ctx, _fail = fail, simpleapi_download = simpleapi_download, **kwargs): """Implementation of parsing the tag classes for the extension and return a struct for registering repositories. Args: module_ctx: {type}`module_ctx` module context. + simpleapi_download: Used for testing overrides _fail: {type}`function` the failure function, mainly for testing. **kwargs: Extra arguments passed to the layers below. @@ -460,10 +461,29 @@ You cannot use both the additive_build_content and additive_build_content_file a else: pip_hub_map[pip_attr.hub_name].python_versions.append(pip_attr.python_version) + get_index_urls = None + if pip_attr.experimental_index_url: + is_reproducible = False + get_index_urls = lambda ctx, distributions: simpleapi_download( + ctx, + attr = struct( + index_url = pip_attr.experimental_index_url, + extra_index_urls = pip_attr.experimental_extra_index_urls or [], + index_url_overrides = pip_attr.experimental_index_url_overrides or {}, + sources = distributions, + envsubst = pip_attr.envsubst, + # Auth related info + netrc = pip_attr.netrc, + auth_patterns = pip_attr.auth_patterns, + ), + cache = simpleapi_cache, + parallel_download = pip_attr.parallel_download, + ) + out = _create_whl_repos( module_ctx, pip_attr = pip_attr, - simpleapi_cache = simpleapi_cache, + get_index_urls = get_index_urls, whl_overrides = whl_overrides, **kwargs ) @@ -476,7 +496,6 @@ You cannot use both the additive_build_content and additive_build_content_file a extra_aliases[hub_name].setdefault(whl_name, {}).update(aliases) exposed_packages.setdefault(hub_name, {}).update(out.exposed_packages) whl_libraries.update(out.whl_libraries) - is_reproducible = is_reproducible and out.is_reproducible # TODO @aignas 2024-04-05: how do we support different requirement # cycles for different abis/oses? For now we will need the users to diff --git a/python/private/pypi/whl_repo_name.bzl b/python/private/pypi/whl_repo_name.bzl index 38ed600cd1..48bbd1a9b2 100644 --- a/python/private/pypi/whl_repo_name.bzl +++ b/python/private/pypi/whl_repo_name.bzl @@ -18,18 +18,17 @@ load("//python/private:normalize_name.bzl", "normalize_name") load(":parse_whl_name.bzl", "parse_whl_name") -def whl_repo_name(prefix, filename, sha256): +def whl_repo_name(filename, sha256): """Return a valid whl_library repo name given a distribution filename. Args: - prefix: {type}`str` the prefix of the whl_library. filename: {type}`str` the filename of the distribution. sha256: {type}`str` the sha256 of the distribution. Returns: a string that can be used in {obj}`whl_library`. """ - parts = [prefix] + parts = [] if not filename.endswith(".whl"): # Then the filename is basically foo-3.2.1. @@ -51,11 +50,10 @@ def whl_repo_name(prefix, filename, sha256): return "_".join(parts) -def pypi_repo_name(prefix, whl_name, *target_platforms): +def pypi_repo_name(whl_name, *target_platforms): """Return a valid whl_library given a requirement line. Args: - prefix: {type}`str` the prefix of the whl_library. whl_name: {type}`str` the whl_name to use. *target_platforms: {type}`list[str]` the target platforms to use in the name. @@ -63,7 +61,6 @@ def pypi_repo_name(prefix, whl_name, *target_platforms): {type}`str` that can be used in {obj}`whl_library`. """ parts = [ - prefix, normalize_name(whl_name), ] parts.extend([p.partition("_")[-1] for p in target_platforms]) diff --git a/tests/pypi/whl_repo_name/whl_repo_name_tests.bzl b/tests/pypi/whl_repo_name/whl_repo_name_tests.bzl index 8b7df83530..000941b55b 100644 --- a/tests/pypi/whl_repo_name/whl_repo_name_tests.bzl +++ b/tests/pypi/whl_repo_name/whl_repo_name_tests.bzl @@ -20,26 +20,25 @@ load("//python/private/pypi:whl_repo_name.bzl", "whl_repo_name") # buildifier: _tests = [] def _test_simple(env): - got = whl_repo_name("prefix", "foo-1.2.3-py3-none-any.whl", "deadbeef") - env.expect.that_str(got).equals("prefix_foo_py3_none_any_deadbeef") + got = whl_repo_name("foo-1.2.3-py3-none-any.whl", "deadbeef") + env.expect.that_str(got).equals("foo_py3_none_any_deadbeef") _tests.append(_test_simple) def _test_sdist(env): - got = whl_repo_name("prefix", "foo-1.2.3.tar.gz", "deadbeef000deadbeef") - env.expect.that_str(got).equals("prefix_foo_sdist_deadbeef") + got = whl_repo_name("foo-1.2.3.tar.gz", "deadbeef000deadbeef") + env.expect.that_str(got).equals("foo_sdist_deadbeef") _tests.append(_test_sdist) def _test_platform_whl(env): got = whl_repo_name( - "prefix", "foo-1.2.3-cp39.cp310-abi3-manylinux1_x86_64.manylinux_2_17_x86_64.whl", "deadbeef000deadbeef", ) # We only need the first segment of each - env.expect.that_str(got).equals("prefix_foo_cp39_abi3_manylinux_2_5_x86_64_deadbeef") + env.expect.that_str(got).equals("foo_cp39_abi3_manylinux_2_5_x86_64_deadbeef") _tests.append(_test_platform_whl) From c8346f9cba12301092a6946fd317220179ba3bea Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Mon, 30 Dec 2024 22:15:15 -0800 Subject: [PATCH 040/922] feat: add pyi attributes/fields, original source fields (#2538) This adds attributes and fields of use to static analysis. For type definition files (usually `.pyi` files), the `pyi_srcs` and `pyi_deps` fields are added to the rules. They end up in the PyInfo fields direct_pyi_files and transitive_pyi_files. So that static analysis tools can retain access to a target's Python source files, even if precompiling is enabled, `direct_original_sources` and `transitive_original_sources` fields are added to PyInfo. Work towards https://github.com/bazelbuild/rules_python/issues/2537, https://github.com/bazelbuild/rules_python/issues/296 --- CHANGELOG.md | 7 ++ python/private/attributes.bzl | 29 +++++ python/private/common.bzl | 11 +- python/private/py_executable.bzl | 5 + python/private/py_info.bzl | 135 ++++++++++++++++++++- python/private/py_library.bzl | 1 + tests/base_rules/base_tests.bzl | 45 +++++++ tests/base_rules/py_info/py_info_tests.bzl | 66 +++++++++- tests/support/py_info_subject.bzl | 38 ++++++ 9 files changed, 332 insertions(+), 5 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index fd5d455147..7c2f84f83a 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -97,6 +97,13 @@ Unreleased changes template. * 3.11.11 * 3.12.8 * 3.13.1 +* (rules) Attributes for type definition files (`.pyi` files) and type-checking + only dependencies added. See {obj}`py_library.pyi_srcs` and + `py_library.pyi_deps` (and the same named attributes for `py_binary` and + `py_test`). +* (providers) {obj}`PyInfo` has new fields to aid static analysis tools: + {obj}`direct_original_sources`, {obj}`direct_pyi_files`, + {obj}`transitive_original_sources`, {obj}`transitive_pyi_files`. [20241206]: https://github.com/astral-sh/python-build-standalone/releases/tag/20241206 diff --git a/python/private/attributes.bzl b/python/private/attributes.bzl index e62abf9f71..dfe0d4e716 100644 --- a/python/private/attributes.bzl +++ b/python/private/attributes.bzl @@ -375,6 +375,35 @@ in the resulting output or not. Valid values are: * `omit_source`: Don't include the original py source. """, ), + "pyi_deps": attr.label_list( + doc = """ +Dependencies providing type definitions the library needs. + +These are dependencies that satisfy imports guarded by `typing.TYPE_CHECKING`. +These are build-time only dependencies and not included as part of a runnable +program (packaging rules may include them, however). + +:::{versionadded} VERSION_NEXT_FEATURE +::: +""", + providers = [ + [PyInfo], + [CcInfo], + ] + _MaybeBuiltinPyInfo, + ), + "pyi_srcs": attr.label_list( + doc = """ +Type definition files for the library. + +These are typically `.pyi` files, but other file types for type-checker specific +formats are allowed. These files are build-time only dependencies and not included +as part of a runnable program (packaging rules may include them, however). + +:::{versionadded} VERSION_NEXT_FEATURE +::: +""", + allow_files = True, + ), # Required attribute, but details vary by rule. # Use create_srcs_attr to create one. "srcs": None, diff --git a/python/private/common.bzl b/python/private/common.bzl index 9c285f97bc..b6a54532d3 100644 --- a/python/private/common.bzl +++ b/python/private/common.bzl @@ -408,6 +408,7 @@ def collect_runfiles(ctx, files = depset()): def create_py_info( ctx, *, + original_sources, required_py_files, required_pyc_files, implicit_pyc_files, @@ -417,6 +418,7 @@ def create_py_info( Args: ctx: rule ctx. + original_sources: `depset[File]`; the original input sources from `srcs` required_py_files: `depset[File]`; the direct, `.py` sources for the target that **must** be included by downstream targets. This should only be Python source files. It should not include pyc files. @@ -435,10 +437,13 @@ def create_py_info( transitive sources collected from dependencies (the latter is only necessary for deprecated extra actions support). """ - py_info = PyInfoBuilder() + py_info.direct_original_sources.add(original_sources) py_info.direct_pyc_files.add(required_pyc_files) + py_info.direct_pyi_files.add(ctx.files.pyi_srcs) + py_info.transitive_original_sources.add(original_sources) py_info.transitive_pyc_files.add(required_pyc_files) + py_info.transitive_pyi_files.add(ctx.files.pyi_srcs) py_info.transitive_implicit_pyc_files.add(implicit_pyc_files) py_info.transitive_implicit_pyc_source_files.add(implicit_pyc_source_files) py_info.imports.add(imports) @@ -457,6 +462,10 @@ def create_py_info( if f.extension == "py": py_info.transitive_sources.add(f) py_info.merge_uses_shared_libraries(cc_helper.is_valid_shared_library_artifact(f)) + for target in ctx.attr.pyi_deps: + # PyInfo may not be present e.g. cc_library rules. + if PyInfo in target or (BuiltinPyInfo != None and BuiltinPyInfo in target): + py_info.merge(_get_py_info(target)) deps_transitive_sources = py_info.transitive_sources.build() py_info.transitive_sources.add(required_py_files) diff --git a/python/private/py_executable.bzl b/python/private/py_executable.bzl index 40c74100f2..20af98da9d 100644 --- a/python/private/py_executable.bzl +++ b/python/private/py_executable.bzl @@ -985,6 +985,7 @@ def py_executable_base_impl(ctx, *, semantics, is_test, inherited_environment = runfiles_details = runfiles_details, main_py = main_py, imports = imports, + original_sources = direct_sources, required_py_files = required_py_files, required_pyc_files = required_pyc_files, implicit_pyc_files = implicit_pyc_files, @@ -1548,6 +1549,7 @@ def _create_providers( ctx, executable, main_py, + original_sources, required_py_files, required_pyc_files, implicit_pyc_files, @@ -1566,6 +1568,8 @@ def _create_providers( ctx: The rule ctx. executable: File; the target's executable file. main_py: File; the main .py entry point. + original_sources: `depset[File]` the direct `.py` sources for the + target that were the original input sources. required_py_files: `depset[File]` the direct, `.py` sources for the target that **must** be included by downstream targets. This should only be Python source files. It should not include pyc files. @@ -1649,6 +1653,7 @@ def _create_providers( py_info, deps_transitive_sources, builtin_py_info = create_py_info( ctx, + original_sources = original_sources, required_py_files = required_py_files, required_pyc_files = required_pyc_files, implicit_pyc_files = implicit_pyc_files, diff --git a/python/private/py_info.bzl b/python/private/py_info.bzl index 4b2b8888c9..2a02685362 100644 --- a/python/private/py_info.bzl +++ b/python/private/py_info.bzl @@ -38,7 +38,11 @@ def _PyInfo_init( direct_pyc_files = depset(), transitive_pyc_files = depset(), transitive_implicit_pyc_files = depset(), - transitive_implicit_pyc_source_files = depset()): + transitive_implicit_pyc_source_files = depset(), + direct_original_sources = depset(), + transitive_original_sources = depset(), + direct_pyi_files = depset(), + transitive_pyi_files = depset()): _check_arg_type("transitive_sources", "depset", transitive_sources) # Verify it's postorder compatible, but retain is original ordering. @@ -53,14 +57,24 @@ def _PyInfo_init( _check_arg_type("transitive_implicit_pyc_files", "depset", transitive_pyc_files) _check_arg_type("transitive_implicit_pyc_source_files", "depset", transitive_pyc_files) + + _check_arg_type("direct_original_sources", "depset", direct_original_sources) + _check_arg_type("transitive_original_sources", "depset", transitive_original_sources) + + _check_arg_type("direct_pyi_files", "depset", direct_pyi_files) + _check_arg_type("transitive_pyi_files", "depset", transitive_pyi_files) return { + "direct_original_sources": direct_original_sources, "direct_pyc_files": direct_pyc_files, + "direct_pyi_files": direct_pyi_files, "has_py2_only_sources": has_py2_only_sources, "has_py3_only_sources": has_py2_only_sources, "imports": imports, "transitive_implicit_pyc_files": transitive_implicit_pyc_files, "transitive_implicit_pyc_source_files": transitive_implicit_pyc_source_files, + "transitive_original_sources": transitive_original_sources, "transitive_pyc_files": transitive_pyc_files, + "transitive_pyi_files": transitive_pyi_files, "transitive_sources": transitive_sources, "uses_shared_libraries": uses_shared_libraries, } @@ -69,6 +83,18 @@ PyInfo, _unused_raw_py_info_ctor = define_bazel_6_provider( doc = "Encapsulates information provided by the Python rules.", init = _PyInfo_init, fields = { + "direct_original_sources": """ +:type: depset[File] + +The `.py` source files (if any) that are considered directly provided by +the target. This field is intended so that static analysis tools can recover the +original Python source files, regardless of any build settings (e.g. +precompiling), so they can analyze source code. The values are typically the +`.py` files in the `srcs` attribute (or equivalent). + +::::{versionadded} 1.1.0 +:::: +""", "direct_pyc_files": """ :type: depset[File] @@ -78,6 +104,21 @@ by the target and **must be included**. These files usually come from, e.g., a library setting {attr}`precompile=enabled` to forcibly enable precompiling for itself. Downstream binaries are expected to always include these files, as the originating target expects them to exist. +""", + "direct_pyi_files": """ +:type: depset[File] + +Type definition files (usually `.pyi` files) for the Python modules provided by +this target. Usually they describe the source files listed in +`direct_original_sources`. This field is primarily for static analysis tools. + +:::{note} +This may contain implementation-specific file types specific to a particular +type checker. +::: + +::::{versionadded} 1.1.0 +:::: """, "has_py2_only_sources": """ :type: bool @@ -116,6 +157,21 @@ then {obj}`transitive_implicit_pyc_files` should be included instead. ::::{versionadded} 0.37.0 :::: +""", + "transitive_original_sources": """ +:type: depset[File] + +The transitive set of `.py` source files (if any) that are considered the +original sources for this target and its transitive dependencies. This field is +intended so that static analysis tools can recover the original Python source +files, regardless of any build settings (e.g. precompiling), so they can analyze +source code. The values are typically the `.py` files in the `srcs` attribute +(or equivalent). + +This is superset of `direct_original_sources`. + +::::{versionadded} 1.1.0 +:::: """, "transitive_pyc_files": """ :type: depset[File] @@ -125,6 +181,22 @@ The transitive set of precompiled files that must be included. These files usually come from, e.g., a library setting {attr}`precompile=enabled` to forcibly enable precompiling for itself. Downstream binaries are expected to always include these files, as the originating target expects them to exist. +""", + "transitive_pyi_files": """ +:type: depset[File] + +The transitive set of type definition files (usually `.pyi` files) for the +Python modules for this target and its transitive dependencies. this target. +Usually they describe the source files listed in `transitive_original_sources`. +This field is primarily for static analysis tools. + +:::{note} +This may contain implementation-specific file types specific to a particular +type checker. +::: + +::::{versionadded} 1.1.0 +:::: """, "transitive_sources": """\ :type: depset[File] @@ -165,7 +237,9 @@ def PyInfoBuilder(): _uses_shared_libraries = [False], build = lambda *a, **k: _PyInfoBuilder_build(self, *a, **k), build_builtin_py_info = lambda *a, **k: _PyInfoBuilder_build_builtin_py_info(self, *a, **k), + direct_original_sources = builders.DepsetBuilder(), direct_pyc_files = builders.DepsetBuilder(), + direct_pyi_files = builders.DepsetBuilder(), get_has_py2_only_sources = lambda *a, **k: _PyInfoBuilder_get_has_py2_only_sources(self, *a, **k), get_has_py3_only_sources = lambda *a, **k: _PyInfoBuilder_get_has_py3_only_sources(self, *a, **k), get_uses_shared_libraries = lambda *a, **k: _PyInfoBuilder_get_uses_shared_libraries(self, *a, **k), @@ -182,7 +256,9 @@ def PyInfoBuilder(): set_uses_shared_libraries = lambda *a, **k: _PyInfoBuilder_set_uses_shared_libraries(self, *a, **k), transitive_implicit_pyc_files = builders.DepsetBuilder(), transitive_implicit_pyc_source_files = builders.DepsetBuilder(), + transitive_original_sources = builders.DepsetBuilder(), transitive_pyc_files = builders.DepsetBuilder(), + transitive_pyi_files = builders.DepsetBuilder(), transitive_sources = builders.DepsetBuilder(), ) return self @@ -221,13 +297,39 @@ def _PyInfoBuilder_set_uses_shared_libraries(self, value): return self def _PyInfoBuilder_merge(self, *infos, direct = []): + """Merge other PyInfos into this PyInfo. + + Args: + self: implicitly added. + *infos: {type}`PyInfo` objects to merge in, but only merge in their + information into this object's transitive fields. + direct: {type}`list[PyInfo]` objects to merge in, but also merge their + direct fields into this object's direct fields. + + Returns: + {type}`PyInfoBuilder` the current object + """ return self.merge_all(list(infos), direct = direct) def _PyInfoBuilder_merge_all(self, transitive, *, direct = []): + """Merge other PyInfos into this PyInfo. + + Args: + self: implicitly added. + transitive: {type}`list[PyInfo]` objects to merge in, but only merge in + their information into this object's transitive fields. + direct: {type}`list[PyInfo]` objects to merge in, but also merge their + direct fields into this object's direct fields. + + Returns: + {type}`PyInfoBuilder` the current object + """ for info in direct: # BuiltinPyInfo doesn't have this field if hasattr(info, "direct_pyc_files"): + self.direct_original_sources.add(info.direct_original_sources) self.direct_pyc_files.add(info.direct_pyc_files) + self.direct_pyi_files.add(info.direct_pyi_files) for info in direct + transitive: self.imports.add(info.imports) @@ -240,11 +342,24 @@ def _PyInfoBuilder_merge_all(self, transitive, *, direct = []): if hasattr(info, "transitive_pyc_files"): self.transitive_implicit_pyc_files.add(info.transitive_implicit_pyc_files) self.transitive_implicit_pyc_source_files.add(info.transitive_implicit_pyc_source_files) + self.transitive_original_sources.add(info.transitive_original_sources) self.transitive_pyc_files.add(info.transitive_pyc_files) + self.transitive_pyi_files.add(info.transitive_pyi_files) return self def _PyInfoBuilder_merge_target(self, target): + """Merge a target's Python information in this object. + + Args: + self: implicitly added. + target: {type}`Target` targets that provide PyInfo, or other relevant + providers, will be merged into this object. If a target doesn't provide + any relevant providers, it is ignored. + + Returns: + {type}`PyInfoBuilder` the current object. + """ if PyInfo in target: self.merge(target[PyInfo]) elif BuiltinPyInfo != None and BuiltinPyInfo in target: @@ -252,6 +367,18 @@ def _PyInfoBuilder_merge_target(self, target): return self def _PyInfoBuilder_merge_targets(self, targets): + """Merge multiple targets into this object. + + Args: + self: implicitly added. + targets: {type}`list[Target]` + targets that provide PyInfo, or other relevant + providers, will be merged into this object. If a target doesn't provide + any relevant providers, it is ignored. + + Returns: + {type}`PyInfoBuilder` the current object. + """ for t in targets: self.merge_target(t) return self @@ -259,10 +386,14 @@ def _PyInfoBuilder_merge_targets(self, targets): def _PyInfoBuilder_build(self): if config.enable_pystar: kwargs = dict( + direct_original_sources = self.direct_original_sources.build(), direct_pyc_files = self.direct_pyc_files.build(), - transitive_pyc_files = self.transitive_pyc_files.build(), + direct_pyi_files = self.direct_pyi_files.build(), transitive_implicit_pyc_files = self.transitive_implicit_pyc_files.build(), transitive_implicit_pyc_source_files = self.transitive_implicit_pyc_source_files.build(), + transitive_original_sources = self.transitive_original_sources.build(), + transitive_pyc_files = self.transitive_pyc_files.build(), + transitive_pyi_files = self.transitive_pyi_files.build(), ) else: kwargs = {} diff --git a/python/private/py_library.bzl b/python/private/py_library.bzl index 6a65038e8a..350ea35aa6 100644 --- a/python/private/py_library.bzl +++ b/python/private/py_library.bzl @@ -102,6 +102,7 @@ def py_library_impl(ctx, *, semantics): cc_info = semantics.get_cc_info_for_library(ctx) py_info, deps_transitive_sources, builtins_py_info = create_py_info( ctx, + original_sources = direct_sources, required_py_files = required_py_files, required_pyc_files = required_pyc_files, implicit_pyc_files = implicit_pyc_files, diff --git a/tests/base_rules/base_tests.bzl b/tests/base_rules/base_tests.bzl index 8e0d10d729..a9fadd7564 100644 --- a/tests/base_rules/base_tests.bzl +++ b/tests/base_rules/base_tests.bzl @@ -17,6 +17,7 @@ load("@rules_testing//lib:analysis_test.bzl", "analysis_test") load("@rules_testing//lib:truth.bzl", "matching") load("@rules_testing//lib:util.bzl", "PREVENT_IMPLICIT_BUILDING_TAGS", rt_util = "util") load("//python:py_info.bzl", "PyInfo") +load("//python:py_library.bzl", "py_library") load("//python/private:reexports.bzl", "BuiltinPyInfo") # buildifier: disable=bzl-visibility load("//tests/base_rules:util.bzl", pt_util = "util") load("//tests/support:py_info_subject.bzl", "py_info_subject") @@ -58,6 +59,50 @@ _not_produces_py_info = rule( implementation = _not_produces_py_info_impl, ) +def _test_py_info_populated(name, config): + rt_util.helper_target( + config.base_test_rule, + name = name + "_subject", + srcs = [name + "_subject.py"], + pyi_srcs = ["subject.pyi"], + pyi_deps = [name + "_lib2"], + ) + rt_util.helper_target( + py_library, + name = name + "_lib2", + srcs = ["lib2.py"], + pyi_srcs = ["lib2.pyi"], + ) + + analysis_test( + name = name, + target = name + "_subject", + impl = _test_py_info_populated_impl, + ) + +def _test_py_info_populated_impl(env, target): + info = env.expect.that_target(target).provider( + PyInfo, + factory = py_info_subject, + ) + info.direct_original_sources().contains_exactly([ + "{package}/test_py_info_populated_subject.py", + ]) + info.transitive_original_sources().contains_exactly([ + "{package}/test_py_info_populated_subject.py", + "{package}/lib2.py", + ]) + + info.direct_pyi_files().contains_exactly([ + "{package}/subject.pyi", + ]) + info.transitive_pyi_files().contains_exactly([ + "{package}/lib2.pyi", + "{package}/subject.pyi", + ]) + +_tests.append(_test_py_info_populated) + def _py_info_propagation_setup(name, config, produce_py_info_rule, test_impl): rt_util.helper_target( config.base_test_rule, diff --git a/tests/base_rules/py_info/py_info_tests.bzl b/tests/base_rules/py_info/py_info_tests.bzl index 4067a59d24..e160e704de 100644 --- a/tests/base_rules/py_info/py_info_tests.bzl +++ b/tests/base_rules/py_info/py_info_tests.bzl @@ -24,9 +24,13 @@ load("//tests/support:py_info_subject.bzl", "py_info_subject") def _provide_py_info_impl(ctx): kwargs = { + "direct_original_sources": depset(ctx.files.direct_original_sources), "direct_pyc_files": depset(ctx.files.direct_pyc_files), + "direct_pyi_files": depset(ctx.files.direct_pyi_files), "imports": depset(ctx.attr.imports), + "transitive_original_sources": depset(ctx.files.transitive_original_sources), "transitive_pyc_files": depset(ctx.files.transitive_pyc_files), + "transitive_pyi_files": depset(ctx.files.transitive_pyi_files), "transitive_sources": depset(ctx.files.transitive_sources), } if ctx.attr.has_py2_only_sources != -1: @@ -56,11 +60,15 @@ def _provide_py_info_impl(ctx): provide_py_info = rule( implementation = _provide_py_info_impl, attrs = { + "direct_original_sources": attr.label_list(allow_files = True), "direct_pyc_files": attr.label_list(allow_files = True), + "direct_pyi_files": attr.label_list(allow_files = True), "has_py2_only_sources": attr.int(default = -1), "has_py3_only_sources": attr.int(default = -1), "imports": attr.string_list(), + "transitive_original_sources": attr.label_list(allow_files = True), "transitive_pyc_files": attr.label_list(allow_files = True), + "transitive_pyi_files": attr.label_list(allow_files = True), "transitive_sources": attr.label_list(allow_files = True), }, ) @@ -109,7 +117,15 @@ def _test_py_info_builder(name): rt_util.helper_target( native.filegroup, name = name + "_misc", - srcs = ["trans.py", "direct.pyc", "trans.pyc"], + srcs = [ + "trans.py", + "direct.pyc", + "trans.pyc", + "original.py", + "trans-original.py", + "direct.pyi", + "trans.pyi", + ], ) py_info_targets = {} @@ -123,6 +139,10 @@ def _test_py_info_builder(name): direct_pyc_files = ["py{}-direct.pyc".format(n)], imports = ["py{}import".format(n)], transitive_pyc_files = ["py{}-trans.pyc".format(n)], + direct_original_sources = ["py{}-original-direct.py".format(n)], + transitive_original_sources = ["py{}-original-trans.py".format(n)], + direct_pyi_files = ["py{}-direct.pyi".format(n)], + transitive_pyi_files = ["py{}-trans.pyi".format(n)], ) analysis_test( name = name, @@ -133,13 +153,25 @@ def _test_py_info_builder(name): ) def _test_py_info_builder_impl(env, targets): - trans, direct_pyc, trans_pyc = targets.misc[DefaultInfo].files.to_list() + ( + trans, + direct_pyc, + trans_pyc, + original_py, + trans_original_py, + direct_pyi, + trans_pyi, + ) = targets.misc[DefaultInfo].files.to_list() builder = PyInfoBuilder() builder.direct_pyc_files.add(direct_pyc) + builder.direct_original_sources.add(original_py) + builder.direct_pyi_files.add(direct_pyi) builder.merge_has_py2_only_sources(True) builder.merge_has_py3_only_sources(True) builder.imports.add("import-path") builder.transitive_pyc_files.add(trans_pyc) + builder.transitive_pyi_files.add(trans_pyi) + builder.transitive_original_sources.add(trans_original_py) builder.transitive_sources.add(trans) builder.merge_uses_shared_libraries(True) @@ -174,6 +206,8 @@ def _test_py_info_builder_impl(env, targets): "py5import", "py6import", ]) + + # Checks for non-Bazel builtin PyInfo if hasattr(actual, "direct_pyc_files"): subject.direct_pyc_files().contains_exactly([ "tests/base_rules/py_info/direct.pyc", @@ -189,6 +223,34 @@ def _test_py_info_builder_impl(env, targets): "tests/base_rules/py_info/py5-trans.pyc", "tests/base_rules/py_info/py6-trans.pyc", ]) + subject.direct_original_sources().contains_exactly([ + "tests/base_rules/py_info/original.py", + "tests/base_rules/py_info/py4-original-direct.py", + "tests/base_rules/py_info/py6-original-direct.py", + ]) + subject.transitive_original_sources().contains_exactly([ + "tests/base_rules/py_info/trans-original.py", + "tests/base_rules/py_info/py1-original-trans.py", + "tests/base_rules/py_info/py2-original-trans.py", + "tests/base_rules/py_info/py3-original-trans.py", + "tests/base_rules/py_info/py4-original-trans.py", + "tests/base_rules/py_info/py5-original-trans.py", + "tests/base_rules/py_info/py6-original-trans.py", + ]) + subject.direct_pyi_files().contains_exactly([ + "tests/base_rules/py_info/direct.pyi", + "tests/base_rules/py_info/py4-direct.pyi", + "tests/base_rules/py_info/py6-direct.pyi", + ]) + subject.transitive_pyi_files().contains_exactly([ + "tests/base_rules/py_info/trans.pyi", + "tests/base_rules/py_info/py1-trans.pyi", + "tests/base_rules/py_info/py2-trans.pyi", + "tests/base_rules/py_info/py3-trans.pyi", + "tests/base_rules/py_info/py4-trans.pyi", + "tests/base_rules/py_info/py5-trans.pyi", + "tests/base_rules/py_info/py6-trans.pyi", + ]) check(builder.build()) if BuiltinPyInfo != None: diff --git a/tests/support/py_info_subject.bzl b/tests/support/py_info_subject.bzl index bfed0b335d..9122eaa9fd 100644 --- a/tests/support/py_info_subject.bzl +++ b/tests/support/py_info_subject.bzl @@ -31,11 +31,15 @@ def py_info_subject(info, *, meta): # buildifier: disable=uninitialized public = struct( # go/keep-sorted start + direct_original_sources = lambda *a, **k: _py_info_subject_direct_original_sources(self, *a, **k), direct_pyc_files = lambda *a, **k: _py_info_subject_direct_pyc_files(self, *a, **k), + direct_pyi_files = lambda *a, **k: _py_info_subject_direct_pyi_files(self, *a, **k), has_py2_only_sources = lambda *a, **k: _py_info_subject_has_py2_only_sources(self, *a, **k), has_py3_only_sources = lambda *a, **k: _py_info_subject_has_py3_only_sources(self, *a, **k), imports = lambda *a, **k: _py_info_subject_imports(self, *a, **k), + transitive_original_sources = lambda *a, **k: _py_info_subject_transitive_original_sources(self, *a, **k), transitive_pyc_files = lambda *a, **k: _py_info_subject_transitive_pyc_files(self, *a, **k), + transitive_pyi_files = lambda *a, **k: _py_info_subject_transitive_pyi_files(self, *a, **k), transitive_sources = lambda *a, **k: _py_info_subject_transitive_sources(self, *a, **k), uses_shared_libraries = lambda *a, **k: _py_info_subject_uses_shared_libraries(self, *a, **k), # go/keep-sorted end @@ -46,6 +50,14 @@ def py_info_subject(info, *, meta): ) return public +def _py_info_subject_direct_original_sources(self): + """Returns a `DepsetFileSubject` for the `direct_original_sources` attribute. + """ + return subjects.depset_file( + self.actual.direct_original_sources, + meta = self.meta.derive("direct_original_sources()"), + ) + def _py_info_subject_direct_pyc_files(self): """Returns a `DepsetFileSubject` for the `direct_pyc_files` attribute. @@ -56,6 +68,14 @@ def _py_info_subject_direct_pyc_files(self): meta = self.meta.derive("direct_pyc_files()"), ) +def _py_info_subject_direct_pyi_files(self): + """Returns a `DepsetFileSubject` for the `direct_pyi_files` attribute. + """ + return subjects.depset_file( + self.actual.direct_pyi_files, + meta = self.meta.derive("direct_pyi_files()"), + ) + def _py_info_subject_has_py2_only_sources(self): """Returns a `BoolSubject` for the `has_py2_only_sources` attribute. @@ -86,6 +106,16 @@ def _py_info_subject_imports(self): meta = self.meta.derive("imports()"), ) +def _py_info_subject_transitive_original_sources(self): + """Returns a `DepsetFileSubject` for the `transitive_original_sources` attribute. + + Method: PyInfoSubject.transitive_original_sources + """ + return subjects.depset_file( + self.actual.transitive_original_sources, + meta = self.meta.derive("transitive_original_sources()"), + ) + def _py_info_subject_transitive_pyc_files(self): """Returns a `DepsetFileSubject` for the `transitive_pyc_files` attribute. @@ -96,6 +126,14 @@ def _py_info_subject_transitive_pyc_files(self): meta = self.meta.derive("transitive_pyc_files()"), ) +def _py_info_subject_transitive_pyi_files(self): + """Returns a `DepsetFileSubject` for the `transitive_pyi_files` attribute. + """ + return subjects.depset_file( + self.actual.transitive_pyi_files, + meta = self.meta.derive("transitive_pyi_files()"), + ) + def _py_info_subject_transitive_sources(self): """Returns a `DepsetFileSubject` for the `transitive_sources` attribute. From 9035db2d5e3dedf1392d83189ed4fd67244b0a14 Mon Sep 17 00:00:00 2001 From: Douglas Thor Date: Mon, 30 Dec 2024 22:16:17 -0800 Subject: [PATCH 041/922] fix(gazelle): Don't ignore `setup.py` files when running Gazelle (#2536) Don't ignore `setup.py` files when running Gazelle. Fixes #2108. I believe that `setup.py` was originally ignored because it, when found that the repo root, is part of `setuptools` config and may have caused problems with Gazelle. I've been running our Google Quantum code with this patch for a long while now and not seen any issues. I figured it was time to upstream it. --- CHANGELOG.md | 2 ++ .../python/testdata/dont_ignore_setup/BUILD.in | 1 + .../python/testdata/dont_ignore_setup/BUILD.out | 9 +++++++++ .../python/testdata/dont_ignore_setup/README.md | 8 ++++++++ .../python/testdata/dont_ignore_setup/WORKSPACE | 0 .../python/testdata/dont_ignore_setup/setup.py | 0 .../python/testdata/dont_ignore_setup/test.yaml | 15 +++++++++++++++ .../python_ignore_files_directive/BUILD.out | 5 ++++- gazelle/pythonconfig/pythonconfig.go | 1 - 9 files changed, 39 insertions(+), 2 deletions(-) create mode 100644 gazelle/python/testdata/dont_ignore_setup/BUILD.in create mode 100644 gazelle/python/testdata/dont_ignore_setup/BUILD.out create mode 100644 gazelle/python/testdata/dont_ignore_setup/README.md create mode 100644 gazelle/python/testdata/dont_ignore_setup/WORKSPACE create mode 100644 gazelle/python/testdata/dont_ignore_setup/setup.py create mode 100644 gazelle/python/testdata/dont_ignore_setup/test.yaml diff --git a/CHANGELOG.md b/CHANGELOG.md index 7c2f84f83a..3ae28a9678 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -77,6 +77,8 @@ Unreleased changes template. are now printing more details and include the currently active flag values. Fixes [#2466](https://github.com/bazelbuild/rules_python/issues/2466). * (py_proto_library) Fix import paths in Bazel 8. ++ (gazelle) Gazelle no longer ignores `setup.py` files by default. To restore + this behavior, apply the `# gazelle:python_ignore_files setup.py` directive. [pep-695]: https://peps.python.org/pep-0695/ diff --git a/gazelle/python/testdata/dont_ignore_setup/BUILD.in b/gazelle/python/testdata/dont_ignore_setup/BUILD.in new file mode 100644 index 0000000000..af2c2cea4b --- /dev/null +++ b/gazelle/python/testdata/dont_ignore_setup/BUILD.in @@ -0,0 +1 @@ +# gazelle:python_generation_mode file diff --git a/gazelle/python/testdata/dont_ignore_setup/BUILD.out b/gazelle/python/testdata/dont_ignore_setup/BUILD.out new file mode 100644 index 0000000000..acf9324d3d --- /dev/null +++ b/gazelle/python/testdata/dont_ignore_setup/BUILD.out @@ -0,0 +1,9 @@ +load("@rules_python//python:defs.bzl", "py_library") + +# gazelle:python_generation_mode file + +py_library( + name = "setup", + srcs = ["setup.py"], + visibility = ["//:__subpackages__"], +) diff --git a/gazelle/python/testdata/dont_ignore_setup/README.md b/gazelle/python/testdata/dont_ignore_setup/README.md new file mode 100644 index 0000000000..d170364cb2 --- /dev/null +++ b/gazelle/python/testdata/dont_ignore_setup/README.md @@ -0,0 +1,8 @@ +# Don't ignore setup.py files + +Make sure that files named `setup.py` are processed by Gazelle. + +It's believed that `setup.py` was originally ignored because it, when found +in the repository root directory, is part of the `setuptools` build system +and could cause some issues for Gazelle. However, files within source code can +also be called `setup.py` and thus should be processed by Gazelle. diff --git a/gazelle/python/testdata/dont_ignore_setup/WORKSPACE b/gazelle/python/testdata/dont_ignore_setup/WORKSPACE new file mode 100644 index 0000000000..e69de29bb2 diff --git a/gazelle/python/testdata/dont_ignore_setup/setup.py b/gazelle/python/testdata/dont_ignore_setup/setup.py new file mode 100644 index 0000000000..e69de29bb2 diff --git a/gazelle/python/testdata/dont_ignore_setup/test.yaml b/gazelle/python/testdata/dont_ignore_setup/test.yaml new file mode 100644 index 0000000000..c27e6c854b --- /dev/null +++ b/gazelle/python/testdata/dont_ignore_setup/test.yaml @@ -0,0 +1,15 @@ +# Copyright 2024 The Bazel Authors. All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +--- diff --git a/gazelle/python/testdata/python_ignore_files_directive/BUILD.out b/gazelle/python/testdata/python_ignore_files_directive/BUILD.out index 1fe6030053..234ff71b13 100644 --- a/gazelle/python/testdata/python_ignore_files_directive/BUILD.out +++ b/gazelle/python/testdata/python_ignore_files_directive/BUILD.out @@ -4,6 +4,9 @@ load("@rules_python//python:defs.bzl", "py_library") py_library( name = "python_ignore_files_directive", - srcs = ["__init__.py"], + srcs = [ + "__init__.py", + "setup.py", + ], visibility = ["//:__subpackages__"], ) diff --git a/gazelle/pythonconfig/pythonconfig.go b/gazelle/pythonconfig/pythonconfig.go index 55121381dd..fde0a98da2 100644 --- a/gazelle/pythonconfig/pythonconfig.go +++ b/gazelle/pythonconfig/pythonconfig.go @@ -126,7 +126,6 @@ const ( // defaultIgnoreFiles is the list of default values used in the // python_ignore_files option. var defaultIgnoreFiles = map[string]struct{}{ - "setup.py": {}, } // Configs is an extension of map[string]*Config. It provides finding methods From 475a99e283acbd602d584635a6672cb2b27ca37e Mon Sep 17 00:00:00 2001 From: Ignas Anikevicius <240938+aignas@users.noreply.github.com> Date: Tue, 31 Dec 2024 15:16:38 +0900 Subject: [PATCH 042/922] fix(pypi): change the parallelisation scheme for querying SimpleAPI (#2531) Instead of querying everything in parallel and yielding a lot of 404 warnings, let's query the main index first and then query the other indexes only for the packages that were not yet found. What is more, we can print the value of `experimental_index_url_overrides` for the users to use. Whilst at it, add a unit test to check the new logic. Fixes #2100, since this is the best `rules_python` can do for now. --------- Co-authored-by: Douglas Thor --- CHANGELOG.md | 4 + python/private/pypi/extension.bzl | 5 + python/private/pypi/simpleapi_download.bzl | 99 ++++++++------ tests/pypi/simpleapi_download/BUILD.bazel | 5 + .../simpleapi_download_tests.bzl | 128 ++++++++++++++++++ 5 files changed, 202 insertions(+), 39 deletions(-) create mode 100644 tests/pypi/simpleapi_download/BUILD.bazel create mode 100644 tests/pypi/simpleapi_download/simpleapi_download_tests.bzl diff --git a/CHANGELOG.md b/CHANGELOG.md index 3ae28a9678..ad3f0a6da9 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -56,6 +56,10 @@ Unreleased changes template. * Bazel 6 support is dropped and Bazel 7.4.1 is the minimum supported version, per our Bazel support matrix. Earlier versions are not tested by CI, so functionality cannot be guaranteed. +* ({bzl:obj}`pip.parse`) From now we will make fewer calls to indexes when + fetching the metadata from SimpleAPI. The calls will be done in parallel to + each index separately, so the extension evaluation time might slow down if + not using {bzl:obj}`pip.parse.experimental_index_url_overrides`. * ({bzl:obj}`pip.parse`) Only query SimpleAPI for packages that have sha values in the `requirements.txt` file. diff --git a/python/private/pypi/extension.bzl b/python/private/pypi/extension.bzl index d16a7cce2f..6409bccdd6 100644 --- a/python/private/pypi/extension.bzl +++ b/python/private/pypi/extension.bzl @@ -657,6 +657,11 @@ The indexes must support Simple API as described here: https://packaging.python.org/en/latest/specifications/simple-repository-api/ This is equivalent to `--extra-index-urls` `pip` option. + +:::{versionchanged} 1.1.0 +Starting with this version we will iterate over each index specified until +we find metadata for all references distributions. +::: """, default = [], ), diff --git a/python/private/pypi/simpleapi_download.bzl b/python/private/pypi/simpleapi_download.bzl index c730c20439..6401a066c2 100644 --- a/python/private/pypi/simpleapi_download.bzl +++ b/python/private/pypi/simpleapi_download.bzl @@ -20,9 +20,17 @@ load("@bazel_features//:features.bzl", "bazel_features") load("//python/private:auth.bzl", "get_auth") load("//python/private:envsubst.bzl", "envsubst") load("//python/private:normalize_name.bzl", "normalize_name") +load("//python/private:text_util.bzl", "render") load(":parse_simpleapi_html.bzl", "parse_simpleapi_html") -def simpleapi_download(ctx, *, attr, cache, parallel_download = True): +def simpleapi_download( + ctx, + *, + attr, + cache, + parallel_download = True, + read_simpleapi = None, + _fail = fail): """Download Simple API HTML. Args: @@ -49,6 +57,9 @@ def simpleapi_download(ctx, *, attr, cache, parallel_download = True): reflected when re-evaluating the extension unless we do `bazel clean --expunge`. parallel_download: A boolean to enable usage of bazel 7.1 non-blocking downloads. + read_simpleapi: a function for reading and parsing of the SimpleAPI contents. + Used in tests. + _fail: a function to print a failure. Used in tests. Returns: dict of pkg name to the parsed HTML contents - a list of structs. @@ -64,15 +75,22 @@ def simpleapi_download(ctx, *, attr, cache, parallel_download = True): # NOTE @aignas 2024-03-31: we are not merging results from multiple indexes # to replicate how `pip` would handle this case. - async_downloads = {} contents = {} index_urls = [attr.index_url] + attr.extra_index_urls - for pkg in attr.sources: - pkg_normalized = normalize_name(pkg) - - success = False - for index_url in index_urls: - result = _read_simpleapi( + read_simpleapi = read_simpleapi or _read_simpleapi + + found_on_index = {} + warn_overrides = False + for i, index_url in enumerate(index_urls): + if i != 0: + # Warn the user about a potential fix for the overrides + warn_overrides = True + + async_downloads = {} + sources = [pkg for pkg in attr.sources if pkg not in found_on_index] + for pkg in sources: + pkg_normalized = normalize_name(pkg) + result = read_simpleapi( ctx = ctx, url = "{}/{}/".format( index_url_overrides.get(pkg_normalized, index_url).rstrip("/"), @@ -84,42 +102,45 @@ def simpleapi_download(ctx, *, attr, cache, parallel_download = True): ) if hasattr(result, "wait"): # We will process it in a separate loop: - async_downloads.setdefault(pkg_normalized, []).append( - struct( - pkg_normalized = pkg_normalized, - wait = result.wait, - ), + async_downloads[pkg] = struct( + pkg_normalized = pkg_normalized, + wait = result.wait, ) - continue - - if result.success: + elif result.success: contents[pkg_normalized] = result.output - success = True - break - - if not async_downloads and not success: - fail("Failed to download metadata from urls: {}".format( - ", ".join(index_urls), - )) - - if not async_downloads: - return contents - - # If we use `block` == False, then we need to have a second loop that is - # collecting all of the results as they were being downloaded in parallel. - for pkg, downloads in async_downloads.items(): - success = False - for download in downloads: + found_on_index[pkg] = index_url + + if not async_downloads: + continue + + # If we use `block` == False, then we need to have a second loop that is + # collecting all of the results as they were being downloaded in parallel. + for pkg, download in async_downloads.items(): result = download.wait() - if result.success and download.pkg_normalized not in contents: + if result.success: contents[download.pkg_normalized] = result.output - success = True - - if not success: - fail("Failed to download metadata from urls: {}".format( - ", ".join(index_urls), - )) + found_on_index[pkg] = index_url + + failed_sources = [pkg for pkg in attr.sources if pkg not in found_on_index] + if failed_sources: + _fail("Failed to download metadata for {} for from urls: {}".format( + failed_sources, + index_urls, + )) + return None + + if warn_overrides: + index_url_overrides = { + pkg: found_on_index[pkg] + for pkg in attr.sources + if found_on_index[pkg] != attr.index_url + } + + # buildifier: disable=print + print("You can use the following `index_url_overrides` to avoid the 404 warnings:\n{}".format( + render.dict(index_url_overrides), + )) return contents diff --git a/tests/pypi/simpleapi_download/BUILD.bazel b/tests/pypi/simpleapi_download/BUILD.bazel new file mode 100644 index 0000000000..04747b6246 --- /dev/null +++ b/tests/pypi/simpleapi_download/BUILD.bazel @@ -0,0 +1,5 @@ +load("simpleapi_download_tests.bzl", "simpleapi_download_test_suite") + +simpleapi_download_test_suite( + name = "simpleapi_download_tests", +) diff --git a/tests/pypi/simpleapi_download/simpleapi_download_tests.bzl b/tests/pypi/simpleapi_download/simpleapi_download_tests.bzl new file mode 100644 index 0000000000..9b2967b0da --- /dev/null +++ b/tests/pypi/simpleapi_download/simpleapi_download_tests.bzl @@ -0,0 +1,128 @@ +# Copyright 2024 The Bazel Authors. All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +"" + +load("@rules_testing//lib:test_suite.bzl", "test_suite") +load("//python/private/pypi:simpleapi_download.bzl", "simpleapi_download") # buildifier: disable=bzl-visibility + +_tests = [] + +def _test_simple(env): + calls = [] + + def read_simpleapi(ctx, url, attr, cache, block): + _ = ctx # buildifier: disable=unused-variable + _ = attr + _ = cache + env.expect.that_bool(block).equals(False) + calls.append(url) + if "foo" in url and "main" in url: + return struct( + output = "", + success = False, + ) + else: + return struct( + output = "data from {}".format(url), + success = True, + ) + + contents = simpleapi_download( + ctx = struct( + os = struct(environ = {}), + ), + attr = struct( + index_url_overrides = {}, + index_url = "main", + extra_index_urls = ["extra"], + sources = ["foo", "bar", "baz"], + envsubst = [], + ), + cache = {}, + parallel_download = True, + read_simpleapi = read_simpleapi, + ) + + env.expect.that_collection(calls).contains_exactly([ + "extra/foo/", + "main/bar/", + "main/baz/", + "main/foo/", + ]) + env.expect.that_dict(contents).contains_exactly({ + "bar": "data from main/bar/", + "baz": "data from main/baz/", + "foo": "data from extra/foo/", + }) + +_tests.append(_test_simple) + +def _test_fail(env): + calls = [] + fails = [] + + def read_simpleapi(ctx, url, attr, cache, block): + _ = ctx # buildifier: disable=unused-variable + _ = attr + _ = cache + env.expect.that_bool(block).equals(False) + calls.append(url) + if "foo" in url: + return struct( + output = "", + success = False, + ) + else: + return struct( + output = "data from {}".format(url), + success = True, + ) + + simpleapi_download( + ctx = struct( + os = struct(environ = {}), + ), + attr = struct( + index_url_overrides = {}, + index_url = "main", + extra_index_urls = ["extra"], + sources = ["foo", "bar", "baz"], + envsubst = [], + ), + cache = {}, + parallel_download = True, + read_simpleapi = read_simpleapi, + _fail = fails.append, + ) + + env.expect.that_collection(fails).contains_exactly([ + """Failed to download metadata for ["foo"] for from urls: ["main", "extra"]""", + ]) + env.expect.that_collection(calls).contains_exactly([ + "extra/foo/", + "main/bar/", + "main/baz/", + "main/foo/", + ]) + +_tests.append(_test_fail) + +def simpleapi_download_test_suite(name): + """Create the test suite. + + Args: + name: the name of the test suite + """ + test_suite(name = name, basic_tests = _tests) From 611eda87e48a3943808216cf4e4de875040b09aa Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Tue, 31 Dec 2024 15:46:42 -0800 Subject: [PATCH 043/922] refactor: fold per-target python version into base rules (#2541) Today, specifying the Python version for a target requires using the version-aware rules in `transition.bzl` (or the generated equivalents bound to a specific Python version). With the rules rewritten in Bazel, that functionality can be moved into the base rules themselves. Moving the logic into the base rules simplifies the implementation and avoids having to re-implement subtle behaviors in the wrappers to correctly emulate the wrapped target. For backwards compatibility, the symbols in `transition.bzl` are left as aliases to the underlying rules. --- CHANGELOG.md | 8 + python/config_settings/transition.bzl | 291 ++---------------- python/private/py_executable.bzl | 55 +++- .../transition/multi_version_tests.bzl | 6 +- 4 files changed, 90 insertions(+), 270 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index ad3f0a6da9..e842cf3265 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -62,6 +62,14 @@ Unreleased changes template. not using {bzl:obj}`pip.parse.experimental_index_url_overrides`. * ({bzl:obj}`pip.parse`) Only query SimpleAPI for packages that have sha values in the `requirements.txt` file. +* (rules) The version-aware rules have been folded into the base rules and + the version-aware rules are now simply aliases for the base rules. The + `python_version` attribute is still used to specify the Python version. + +{#v0-0-0-deprecations} +#### Deprecations +* `//python/config_settings:transitions.bzl` and its `py_binary` and `py_test` + wrappers are deprecated. Use the regular rules instead. {#v0-0-0-fixed} ### Fixed diff --git a/python/config_settings/transition.bzl b/python/config_settings/transition.bzl index a7646dcda3..14e2a73dc4 100644 --- a/python/config_settings/transition.bzl +++ b/python/config_settings/transition.bzl @@ -14,266 +14,43 @@ """The transition module contains the rule definitions to wrap py_binary and py_test and transition them to the desired target platform. + +:::{versionchanged} VERSION_NEXT_PATCH +The `py_binary` and `py_test` symbols are aliases to the regular rules. Usages +of them should be changed to load the regular rules directly. +::: """ -load("@bazel_skylib//lib:dicts.bzl", "dicts") load("//python:py_binary.bzl", _py_binary = "py_binary") -load("//python:py_info.bzl", "PyInfo") -load("//python:py_runtime_info.bzl", "PyRuntimeInfo") load("//python:py_test.bzl", _py_test = "py_test") -load("//python/config_settings/private:py_args.bzl", "py_args") -load("//python/private:reexports.bzl", "BuiltinPyInfo", "BuiltinPyRuntimeInfo") - -def _transition_python_version_impl(_, attr): - return {"//python/config_settings:python_version": str(attr.python_version)} - -_transition_python_version = transition( - implementation = _transition_python_version_impl, - inputs = [], - outputs = ["//python/config_settings:python_version"], -) - -def _transition_py_impl(ctx): - target = ctx.attr.target - windows_constraint = ctx.attr._windows_constraint[platform_common.ConstraintValueInfo] - target_is_windows = ctx.target_platform_has_constraint(windows_constraint) - executable = ctx.actions.declare_file(ctx.attr.name + (".exe" if target_is_windows else "")) - ctx.actions.symlink( - is_executable = True, - output = executable, - target_file = target[DefaultInfo].files_to_run.executable, - ) - default_outputs = [] - if target_is_windows: - # NOTE: Bazel 6 + host=linux + target=windows results in the .exe extension missing - inner_bootstrap_path = _strip_suffix(target[DefaultInfo].files_to_run.executable.short_path, ".exe") - inner_bootstrap = None - inner_zip_file_path = inner_bootstrap_path + ".zip" - inner_zip_file = None - for file in target[DefaultInfo].files.to_list(): - if file.short_path == inner_bootstrap_path: - inner_bootstrap = file - elif file.short_path == inner_zip_file_path: - inner_zip_file = file - - # TODO: Use `fragments.py.build_python_zip` once Bazel 6 support is dropped. - # Which file the Windows .exe looks for depends on the --build_python_zip file. - # Bazel 7+ has APIs to know the effective value of that flag, but not Bazel 6. - # To work around this, we treat the existence of a .zip in the default outputs - # to mean --build_python_zip=true. - if inner_zip_file: - suffix = ".zip" - underlying_launched_file = inner_zip_file - else: - suffix = "" - underlying_launched_file = inner_bootstrap - - if underlying_launched_file: - launched_file_symlink = ctx.actions.declare_file(ctx.attr.name + suffix) - ctx.actions.symlink( - is_executable = True, - output = launched_file_symlink, - target_file = underlying_launched_file, - ) - default_outputs.append(launched_file_symlink) - - env = {} - for k, v in ctx.attr.env.items(): - env[k] = ctx.expand_location(v) - - providers = [ - DefaultInfo( - executable = executable, - files = depset(default_outputs, transitive = [target[DefaultInfo].files]), - runfiles = ctx.runfiles(default_outputs).merge(target[DefaultInfo].default_runfiles), - ), - # Ensure that the binary we're wrapping is included in code coverage. - coverage_common.instrumented_files_info( - ctx, - dependency_attributes = ["target"], - ), - target[OutputGroupInfo], - # TODO(f0rmiga): testing.TestEnvironment is deprecated in favour of RunEnvironmentInfo but - # RunEnvironmentInfo is not exposed in Bazel < 5.3. - # https://github.com/bazelbuild/rules_python/issues/901 - # https://github.com/bazelbuild/bazel/commit/dbdfa07e92f99497be9c14265611ad2920161483 - testing.TestEnvironment(env), - ] - if PyInfo in target: - providers.append(target[PyInfo]) - if BuiltinPyInfo != None and BuiltinPyInfo in target and PyInfo != BuiltinPyInfo: - providers.append(target[BuiltinPyInfo]) - - if PyRuntimeInfo in target: - providers.append(target[PyRuntimeInfo]) - if BuiltinPyRuntimeInfo != None and BuiltinPyRuntimeInfo in target and PyRuntimeInfo != BuiltinPyRuntimeInfo: - providers.append(target[BuiltinPyRuntimeInfo]) - return providers - -_COMMON_ATTRS = { - "deps": attr.label_list( - mandatory = False, - ), - "env": attr.string_dict( - mandatory = False, - ), - "python_version": attr.string( - mandatory = True, - ), - "srcs": attr.label_list( - allow_files = True, - mandatory = False, - ), - "target": attr.label( - executable = True, - cfg = "target", - mandatory = True, - providers = [PyInfo], - ), - # "tools" is a hack here. It should be "data" but "data" is not included by default in the - # location expansion in the same way it is in the native Python rules. The difference on how - # the Bazel deals with those special attributes differ on the LocationExpander, e.g.: - # https://github.com/bazelbuild/bazel/blob/ce611646/src/main/java/com/google/devtools/build/lib/analysis/LocationExpander.java#L415-L429 - # - # Since the default LocationExpander used by ctx.expand_location is not the same as the native - # rules (it doesn't set "allowDataAttributeEntriesInLabel"), we use "tools" temporarily while a - # proper fix in Bazel happens. - # - # A fix for this was proposed in https://github.com/bazelbuild/bazel/pull/16381. - "tools": attr.label_list( - allow_files = True, - mandatory = False, - ), - # Required to Opt-in to the transitions feature. - "_allowlist_function_transition": attr.label( - default = "@bazel_tools//tools/allowlists/function_transition_allowlist", - ), - "_windows_constraint": attr.label( - default = "@platforms//os:windows", - ), -} - -_PY_TEST_ATTRS = { - # Magic attribute to help C++ coverage work. There's no - # docs about this; see TestActionBuilder.java - "_collect_cc_coverage": attr.label( - default = "@bazel_tools//tools/test:collect_cc_coverage", - executable = True, - cfg = "exec", - ), - # Magic attribute to make coverage work. There's no - # docs about this; see TestActionBuilder.java - "_lcov_merger": attr.label( - default = configuration_field(fragment = "coverage", name = "output_generator"), - executable = True, - cfg = "exec", - ), -} -_transition_py_binary = rule( - _transition_py_impl, - attrs = _COMMON_ATTRS | _PY_TEST_ATTRS, - cfg = _transition_python_version, - executable = True, - fragments = ["py"], -) - -_transition_py_test = rule( - _transition_py_impl, - attrs = _COMMON_ATTRS | _PY_TEST_ATTRS, - cfg = _transition_python_version, - test = True, - fragments = ["py"], -) - -def _py_rule(rule_impl, transition_rule, name, python_version, **kwargs): - pyargs = py_args(name, kwargs) - args = pyargs["args"] - data = pyargs["data"] - env = pyargs["env"] - srcs = pyargs["srcs"] - deps = pyargs["deps"] - main = pyargs["main"] - - # Attributes common to all build rules. - # https://bazel.build/reference/be/common-definitions#common-attributes - compatible_with = kwargs.pop("compatible_with", None) - deprecation = kwargs.pop("deprecation", None) - exec_compatible_with = kwargs.pop("exec_compatible_with", None) - exec_properties = kwargs.pop("exec_properties", None) - features = kwargs.pop("features", None) - restricted_to = kwargs.pop("restricted_to", None) - tags = kwargs.pop("tags", None) - target_compatible_with = kwargs.pop("target_compatible_with", None) - testonly = kwargs.pop("testonly", None) - toolchains = kwargs.pop("toolchains", None) - visibility = kwargs.pop("visibility", None) - - common_attrs = { - "compatible_with": compatible_with, - "deprecation": deprecation, - "exec_compatible_with": exec_compatible_with, - "exec_properties": exec_properties, - "features": features, - "restricted_to": restricted_to, - "target_compatible_with": target_compatible_with, - "testonly": testonly, - "toolchains": toolchains, - } - - # Test-specific extra attributes. - if "env_inherit" in kwargs: - common_attrs["env_inherit"] = kwargs.pop("env_inherit") - if "size" in kwargs: - common_attrs["size"] = kwargs.pop("size") - if "timeout" in kwargs: - common_attrs["timeout"] = kwargs.pop("timeout") - if "flaky" in kwargs: - common_attrs["flaky"] = kwargs.pop("flaky") - if "shard_count" in kwargs: - common_attrs["shard_count"] = kwargs.pop("shard_count") - if "local" in kwargs: - common_attrs["local"] = kwargs.pop("local") - - # Binary-specific extra attributes. - if "output_licenses" in kwargs: - common_attrs["output_licenses"] = kwargs.pop("output_licenses") - - rule_impl( - name = "_" + name, - args = args, - data = data, - deps = deps, - env = env, - srcs = srcs, - main = main, - tags = ["manual"] + (tags if tags else []), - visibility = ["//visibility:private"], - **dicts.add(common_attrs, kwargs) - ) - - return transition_rule( - name = name, - args = args, - deps = deps, - env = env, - python_version = python_version, - srcs = srcs, - tags = tags, - target = ":_" + name, - tools = data, - visibility = visibility, - **common_attrs - ) - -def py_binary(name, python_version, **kwargs): - return _py_rule(_py_binary, _transition_py_binary, name, python_version, **kwargs) - -def py_test(name, python_version, **kwargs): - return _py_rule(_py_test, _transition_py_test, name, python_version, **kwargs) +_DEPRECATION_MESSAGE = """ +The {name} symbol in @rules_python//python/config_settings:transition.bzl +is deprecated. It is an alias to the regular rule; use it directly instead: + load("@rules_python//python:{name}.bzl", "{name}") +""" -def _strip_suffix(s, suffix): - if s.endswith(suffix): - return s[:-len(suffix)] - else: - return s +def py_binary(**kwargs): + """[DEPRECATED] Deprecated alias for py_binary. + + Args: + **kwargs: keyword args forwarded onto {obj}`py_binary`. + """ + + deprecation = _DEPRECATION_MESSAGE.format(name = "py_binary") + if kwargs.get("deprecation"): + deprecation = kwargs.get("deprecation") + "\n\n" + deprecation + kwargs["deprecation"] = deprecation + _py_binary(**kwargs) + +def py_test(**kwargs): + """[DEPRECATED] Deprecated alias for py_test. + + Args: + **kwargs: keyword args forwarded onto {obj}`py_binary`. + """ + deprecation = _DEPRECATION_MESSAGE.format(name = "py_test") + if kwargs.get("deprecation"): + deprecation = kwargs.get("deprecation") + "\n\n" + deprecation + kwargs["deprecation"] = deprecation + _py_test(**kwargs) diff --git a/python/private/py_executable.bzl b/python/private/py_executable.bzl index 20af98da9d..3b063aac95 100644 --- a/python/private/py_executable.bzl +++ b/python/private/py_executable.bzl @@ -76,6 +76,7 @@ load( _py_builtins = py_internal _EXTERNAL_PATH_PREFIX = "external" _ZIP_RUNFILES_DIRECTORY_NAME = "runfiles" +_PYTHON_VERSION_FLAG = str(Label("//python/config_settings:python_version")) # Bazel 5.4 doesn't have config_common.toolchain_type _CC_TOOLCHAINS = [config_common.toolchain_type( @@ -132,16 +133,34 @@ Valid values are: target level. """, ), - # TODO(b/203567235): In Google, this attribute is deprecated, and can - # only effectively be PY3. Externally, with Bazel, this attribute has - # a separate story. "python_version": attr.string( # TODO(b/203567235): In the Java impl, the default comes from # --python_version. Not clear what the Starlark equivalent is. - default = "PY3", - # NOTE: Some tests care about the order of these values. - values = ["PY2", "PY3"], - doc = "Defunct, unused, does nothing.", + doc = """ +The Python version this target should use. + +The value should be in `X.Y` or `X.Y.Z` (or compatible) format. If empty or +unspecified, the incoming configuration's {obj}`--python_version` flag is +inherited. For backwards compatibility, the values `PY2` and `PY3` are +accepted, but treated as an empty/unspecified value. + +:::{note} +In order for the requested version to be used, there must be a +toolchain configured to match the Python version. If there isn't, then it +may be silently ignored, or an error may occur, depending on the toolchain +configuration. +::: + +:::{versionchanged} VERSION_NEXT_PATCH + +This attribute was changed from only accepting `PY2` and `PY3` values to +accepting arbitrary Python versions. +::: +""", + ), + # Required to opt-in to the transition feature. + "_allowlist_function_transition": attr.label( + default = "@bazel_tools//tools/allowlists/function_transition_allowlist", ), "_bootstrap_impl_flag": attr.label( default = "//python/config_settings:bootstrap_impl", @@ -1009,7 +1028,7 @@ def _get_build_info(ctx, cc_toolchain): return build_info_files.redacted_build_info_files.to_list() def _validate_executable(ctx): - if ctx.attr.python_version != "PY3": + if ctx.attr.python_version == "PY2": fail("It is not allowed to use Python 2") def _declare_executable_file(ctx): @@ -1696,9 +1715,26 @@ def _create_run_environment_info(ctx, inherited_environment): inherited_environment = inherited_environment, ) +def _transition_executable_impl(input_settings, attr): + settings = { + _PYTHON_VERSION_FLAG: input_settings[_PYTHON_VERSION_FLAG], + } + if attr.python_version and attr.python_version not in ("PY2", "PY3"): + settings[_PYTHON_VERSION_FLAG] = attr.python_version + return settings + +_transition_executable = transition( + implementation = _transition_executable_impl, + inputs = [ + _PYTHON_VERSION_FLAG, + ], + outputs = [ + _PYTHON_VERSION_FLAG, + ], +) + def create_executable_rule(*, attrs, **kwargs): return create_base_executable_rule( - ##attrs = dicts.add(EXECUTABLE_ATTRS, attrs), attrs = attrs, fragments = ["py", "bazel_py"], **kwargs @@ -1720,6 +1756,7 @@ def create_base_executable_rule(*, attrs, fragments = [], **kwargs): fragments = fragments + ["py"] kwargs.setdefault("provides", []).append(PyExecutableInfo) kwargs["exec_groups"] = REQUIRED_EXEC_GROUPS | (kwargs.get("exec_groups") or {}) + kwargs.setdefault("cfg", _transition_executable) return rule( # TODO: add ability to remove attrs, i.e. for imports attr attrs = dicts.add(EXECUTABLE_ATTRS, attrs), diff --git a/tests/config_settings/transition/multi_version_tests.bzl b/tests/config_settings/transition/multi_version_tests.bzl index 5805bab32d..50b4402fce 100644 --- a/tests/config_settings/transition/multi_version_tests.bzl +++ b/tests/config_settings/transition/multi_version_tests.bzl @@ -109,8 +109,7 @@ def _test_py_binary_windows_build_python_zip_false_impl(env, target): # have the "_" prefix on them (those are coming from the underlying # wrapped binary). env.expect.that_target(target).default_outputs().contains_exactly([ - "{package}/_{test_name}_subject", - "{package}/_{test_name}_subject.exe", + "{package}/{test_name}_subject.exe", "{package}/{test_name}_subject", "{package}/{test_name}_subject.py", ]) @@ -136,8 +135,7 @@ def _test_py_binary_windows_build_python_zip_true_impl(env, target): # have the "_" prefix on them (those are coming from the underlying # wrapped binary). default_outputs.contains_exactly([ - "{package}/_{test_name}_subject.exe", - "{package}/_{test_name}_subject.zip", + "{package}/{test_name}_subject.exe", "{package}/{test_name}_subject.py", "{package}/{test_name}_subject.zip", ]) From 6a04d3832e82fec0a7b0675e9964b360bc358554 Mon Sep 17 00:00:00 2001 From: Ignas Anikevicius <240938+aignas@users.noreply.github.com> Date: Wed, 1 Jan 2025 14:25:12 +0900 Subject: [PATCH 044/922] fix(whl_library): track sources in whl_library (#2526) This is reusing a bit of code used in `evaluate_markers` and makes use of the RECORD files in the `whl` files that we use to extract whls in `whl_library`. This should be merged before #2514 to avoid any cache invalidation issues downstream. Fixes #2468 --- CHANGELOG.md | 3 +++ python/private/pypi/deps.bzl | 7 +++++++ python/private/pypi/evaluate_markers.bzl | 3 ++- python/private/pypi/whl_library.bzl | 18 +++++++++++++++--- 4 files changed, 27 insertions(+), 4 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index e842cf3265..da411c26f4 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -89,6 +89,9 @@ Unreleased changes template. are now printing more details and include the currently active flag values. Fixes [#2466](https://github.com/bazelbuild/rules_python/issues/2466). * (py_proto_library) Fix import paths in Bazel 8. +* (whl_library) Now the changes to the dependencies are correctly tracked when + PyPI packages used in {bzl:obj}`whl_library` during the `repository_rule` phase + change. Fixes [#2468](https://github.com/bazelbuild/rules_python/issues/2468). + (gazelle) Gazelle no longer ignores `setup.py` files by default. To restore this behavior, apply the `# gazelle:python_ignore_files setup.py` directive. diff --git a/python/private/pypi/deps.bzl b/python/private/pypi/deps.bzl index c6691d7059..31a5201659 100644 --- a/python/private/pypi/deps.bzl +++ b/python/private/pypi/deps.bzl @@ -124,6 +124,13 @@ py_library( # Collate all the repository names so they can be easily consumed all_repo_names = [name for (name, _, _) in _RULE_DEPS] +record_files = { + name: Label("@{}//:{}.dist-info/RECORD".format( + name, + url.rpartition("/")[-1].partition("-py3-none")[0], + )) + for (name, url, _) in _RULE_DEPS +} def pypi_deps(): """ diff --git a/python/private/pypi/evaluate_markers.bzl b/python/private/pypi/evaluate_markers.bzl index c805fd7a59..ec5f576945 100644 --- a/python/private/pypi/evaluate_markers.bzl +++ b/python/private/pypi/evaluate_markers.bzl @@ -14,13 +14,14 @@ """A simple function that evaluates markers using a python interpreter.""" +load(":deps.bzl", "record_files") load(":pypi_repo_utils.bzl", "pypi_repo_utils") # Used as a default value in a rule to ensure we fetch the dependencies. SRCS = [ # When the version, or any of the files in `packaging` package changes, # this file will change as well. - Label("@pypi__packaging//:packaging-24.0.dist-info/RECORD"), + record_files["pypi__packaging"], Label("//python/private/pypi/requirements_parser:resolve_target_platforms.py"), Label("//python/private/pypi/whl_installer:platform.py"), ] diff --git a/python/private/pypi/whl_library.bzl b/python/private/pypi/whl_library.bzl index 79a58a81f2..ef4077fa41 100644 --- a/python/private/pypi/whl_library.bzl +++ b/python/private/pypi/whl_library.bzl @@ -19,7 +19,7 @@ load("//python/private:envsubst.bzl", "envsubst") load("//python/private:is_standalone_interpreter.bzl", "is_standalone_interpreter") load("//python/private:repo_utils.bzl", "REPO_DEBUG_ENV_VAR", "repo_utils") load(":attrs.bzl", "ATTRS", "use_isolated") -load(":deps.bzl", "all_repo_names") +load(":deps.bzl", "all_repo_names", "record_files") load(":generate_whl_library_build_bazel.bzl", "generate_whl_library_build_bazel") load(":parse_whl_name.bzl", "parse_whl_name") load(":patch_whl.bzl", "patch_whl") @@ -242,7 +242,7 @@ def _whl_library_impl(rctx): else: op_tmpl = "whl_library.ResolveRequirement({name}, {requirement})" - repo_utils.execute_checked( + pypi_repo_utils.execute_checked( rctx, # truncate the requirement value when logging it / reporting # progress since it may contain several ' --hash=sha256:... @@ -250,6 +250,7 @@ def _whl_library_impl(rctx): op = op_tmpl.format(name = rctx.attr.name, requirement = rctx.attr.requirement.split(" ", 1)[0]), arguments = args, environment = environment, + srcs = rctx.attr._python_srcs, quiet = rctx.attr.quiet, timeout = rctx.attr.timeout, logger = logger, @@ -291,13 +292,14 @@ def _whl_library_impl(rctx): ) ] - repo_utils.execute_checked( + pypi_repo_utils.execute_checked( rctx, op = "whl_library.ExtractWheel({}, {})".format(rctx.attr.name, whl_path), arguments = args + [ "--whl-file", whl_path, ] + ["--platform={}".format(p) for p in target_platforms], + srcs = rctx.attr._python_srcs, environment = environment, quiet = rctx.attr.quiet, timeout = rctx.attr.timeout, @@ -450,6 +452,16 @@ attr makes `extra_pip_args` and `download_only` ignored.""", for repo in all_repo_names ], ), + "_python_srcs": attr.label_list( + # Used as a default value in a rule to ensure we fetch the dependencies. + default = [ + Label("//python/private/pypi/whl_installer:platform.py"), + Label("//python/private/pypi/whl_installer:wheel.py"), + Label("//python/private/pypi/whl_installer:wheel_installer.py"), + Label("//python/private/pypi/whl_installer:arguments.py"), + Label("//python/private/pypi/whl_installer:namespace_pkgs.py"), + ] + record_files.values(), + ), "_rule_name": attr.string(default = "whl_library"), }, **ATTRS) whl_library_attrs.update(AUTH_ATTRS) From fbf8bc10a466c498fc80c6b58c939a87a8d9e929 Mon Sep 17 00:00:00 2001 From: vfdev Date: Fri, 3 Jan 2025 14:10:27 +0100 Subject: [PATCH 045/922] Updated pip and packaging versions to work with free-threading packages (#2514) We had an issue to install jaxlib with bazel when running the following command (using rules_python v0.39): ```bash bazel test \ --repo_env=HERMETIC_PYTHON_VERSION=3.13-ft \ --repo_env=JAX_NUM_GENERATED_CASES=$JAX_NUM_GENERATED_CASES \ --repo_env=JAX_ENABLE_X64=$JAX_ENABLE_X64 \ --repo_env=JAX_SKIP_SLOW_TESTS=$JAX_SKIP_SLOW_TESTS \ --repo_env=PYTHON_GIL=$PYTHON_GIL \ --repo_env=TSAN_OPTIONS="halt_on_error=1" \ --//jax:build_jaxlib=false \ --nocache_test_results \ --test_output=all \ //tests:cpu_tests ``` According to @vam-google, this was due to old pip/packaging versions. We updated them and this helped to make work the whole building/testing pipeline: https://github.com/jax-ml/jax/pull/24898 So, we would like to upstream the patch: https://github.com/jax-ml/jax/pull/24898/files#diff-e3dc8d7d2bf5d057f95b86bcff7360b6c99fa1f458882fd112b58da4aceb53e4 --- CHANGELOG.md | 2 ++ .../dependency_resolver.py | 30 ++++++++++++++----- python/private/pypi/deps.bzl | 8 ++--- 3 files changed, 29 insertions(+), 11 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index da411c26f4..6ee44251ca 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -65,6 +65,8 @@ Unreleased changes template. * (rules) The version-aware rules have been folded into the base rules and the version-aware rules are now simply aliases for the base rules. The `python_version` attribute is still used to specify the Python version. +* (pypi) Updated versions of packages: `pip` to 24.3.1 and + `packaging` to 24.2. {#v0-0-0-deprecations} #### Deprecations diff --git a/python/private/pypi/dependency_resolver/dependency_resolver.py b/python/private/pypi/dependency_resolver/dependency_resolver.py index 293377dc6d..6f6c20241b 100644 --- a/python/private/pypi/dependency_resolver/dependency_resolver.py +++ b/python/private/pypi/dependency_resolver/dependency_resolver.py @@ -16,6 +16,7 @@ import atexit import os +import re import shutil import sys from pathlib import Path @@ -117,7 +118,6 @@ def main( absolute_path_prefix = resolved_requirements_file[ : -(len(requirements_file) - len(repository_prefix)) ] - # As srcs might contain references to generated files we want to # use the runfiles file first. Thus, we need to compute the relative path # from the execution root. @@ -162,12 +162,19 @@ def main( argv.append( f"--output-file={requirements_file_relative if UPDATE else requirements_out}" ) - argv.extend( + src_files = [ (src_relative if Path(src_relative).exists() else resolved_src) for src_relative, resolved_src in zip(srcs_relative, resolved_srcs) - ) + ] + argv.extend(src_files) argv.extend(extra_args) + # Replace in the output lock file + # the lines like: # via -r /absolute/path/to/ + # with: # via -r + # For Windows, we should explicitly call .as_posix() to convert \\ -> / + absolute_src_prefixes = [Path(src).absolute().parent.as_posix() + "/" for src in src_files] + if UPDATE: print("Updating " + requirements_file_relative) @@ -185,14 +192,14 @@ def main( # and we should copy the updated requirements back to the source tree. if not absolute_output_file.samefile(requirements_file_tree): atexit.register( - lambda: shutil.copy( - absolute_output_file, requirements_file_tree - ) + lambda: shutil.copy(absolute_output_file, requirements_file_tree) ) - cli(argv, standalone_mode = False) + cli(argv, standalone_mode=False) requirements_file_relative_path = Path(requirements_file_relative) content = requirements_file_relative_path.read_text() content = content.replace(absolute_path_prefix, "") + for absolute_src_prefix in absolute_src_prefixes: + content = content.replace(absolute_src_prefix, "") requirements_file_relative_path.write_text(content) else: # cli will exit(0) on success @@ -214,6 +221,15 @@ def main( golden = open(_locate(bazel_runfiles, requirements_file)).readlines() out = open(requirements_out).readlines() out = [line.replace(absolute_path_prefix, "") for line in out] + + def replace_via_minus_r(line): + if "# via -r " in line: + for absolute_src_prefix in absolute_src_prefixes: + line = line.replace(absolute_src_prefix, "") + return line + return line + + out = [replace_via_minus_r(line) for line in out] if golden != out: import difflib diff --git a/python/private/pypi/deps.bzl b/python/private/pypi/deps.bzl index 31a5201659..21dd7771fa 100644 --- a/python/private/pypi/deps.bzl +++ b/python/private/pypi/deps.bzl @@ -51,8 +51,8 @@ _RULE_DEPS = [ ), ( "pypi__packaging", - "https://files.pythonhosted.org/packages/49/df/1fceb2f8900f8639e278b056416d49134fb8d84c5942ffaa01ad34782422/packaging-24.0-py3-none-any.whl", - "2ddfb553fdf02fb784c234c7ba6ccc288296ceabec964ad2eae3777778130bc5", + "https://files.pythonhosted.org/packages/88/ef/eb23f262cca3c0c4eb7ab1933c3b1f03d021f2c48f54763065b6f0e321be/packaging-24.2-py3-none-any.whl", + "09abb1bccd265c01f4a3aa3f7a7db064b36514d2cba19a2f694fe6150451a759", ), ( "pypi__pep517", @@ -61,8 +61,8 @@ _RULE_DEPS = [ ), ( "pypi__pip", - "https://files.pythonhosted.org/packages/8a/6a/19e9fe04fca059ccf770861c7d5721ab4c2aebc539889e97c7977528a53b/pip-24.0-py3-none-any.whl", - "ba0d021a166865d2265246961bec0152ff124de910c5cc39f1156ce3fa7c69dc", + "https://files.pythonhosted.org/packages/ef/7d/500c9ad20238fcfcb4cb9243eede163594d7020ce87bd9610c9e02771876/pip-24.3.1-py3-none-any.whl", + "3790624780082365f47549d032f3770eeb2b1e8bd1f7b2e02dace1afa361b4ed", ), ( "pypi__pip_tools", From 1cb4f8fa979ea7582f6937025a9a43f143acb2ae Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Fri, 3 Jan 2025 10:42:37 -0800 Subject: [PATCH 046/922] chore: add check for version_next markers (#2542) In a couple recent PRs, I put "VERSION_NEXT_{feature,patch}" as place holders since I wasn't sure what the next appropriate version would be for unreleased code. e.g. specifying `1.0.1` becomes invalid if a subsequent PR would make it `1.1.0`. To help remind us to populate these values before a release, have the release workflow check for the marker strings. --- .github/workflows/create_archive_and_notes.sh | 8 ++++ CONTRIBUTING.md | 45 +++++++++++++++++-- 2 files changed, 50 insertions(+), 3 deletions(-) diff --git a/.github/workflows/create_archive_and_notes.sh b/.github/workflows/create_archive_and_notes.sh index 0bc14f936b..b425de96ac 100755 --- a/.github/workflows/create_archive_and_notes.sh +++ b/.github/workflows/create_archive_and_notes.sh @@ -15,6 +15,14 @@ set -o errexit -o nounset -o pipefail +# Exclude dot directories, specifically, this file so that we don't +# find the substring we're looking for in our own file. +if grep --exclude-dir=.* VERSION_NEXT_ -r; then + echo + echo "Found VERSION_NEXT markers indicating version needs to be specified" + exit 1 +fi + # Set by GH actions, see # https://docs.github.com/en/actions/learn-github-actions/environment-variables#default-environment-variables TAG=${GITHUB_REF_NAME} diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index d5f24a9365..8928246c93 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -137,19 +137,58 @@ If a breaking change is introduced, then `BREAKING CHANGE:` is required; see the [Breaking Changes](#breaking-changes) section for how to introduce breaking changes. +User visible changes, such as features, fixes, or notable refactors, should +be documneted in CHANGELOG.md and their respective API doc. See [Documenting +changes] for how to do so. + Common `type`s: * `build:` means it affects the building or development workflow. * `docs:` means only documentation is being added, updated, or fixed. -* `feat:` means a user-visible feature is being added. -* `fix:` means a user-visible behavior is being fixed. -* `refactor:` means some sort of code cleanup that doesn't change user-visible behavior. +* `feat:` means a user-visible feature is being added. See [Documenting version + changes] for how to documenAdd `{versionadded}` + to appropriate docs. +* `fix:` means a user-visible behavior is being fixed. If the fix is changing + behavior of a function, add `{versionchanged}` to appropriate docs, as necessary. +* `refactor:` means some sort of code cleanup that doesn't change user-visible + behavior. Add `{versionchanged}` to appropriate docs, as necessary. * `revert:` means a prior change is being reverted in some way. * `test:` means only tests are being added. For the full details of types, see [Conventional Commits](https://www.conventionalcommits.org/). +### Documenting changes + +Changes are documented in two places: CHANGELOG.md and API docs. + +CHANGELOG.md contains a brief, human friendly, description. This text is +intended for easy skimming so that, when people upgrade, they can quickly get a +sense of what's relevant to them. + +API documentation are the doc strings for functions, fields, attributes, etc. +When user-visible or notable behavior is added, changed, or removed, the +`{versionadded}`, `{versionchanged}` or `{versionremoved}` directives should be +used to note the change. When specifying the version, use the values +`VERSION_NEXT_FEATURE` or `VERSION_NEXT_PATCH` to indicate what sort of +version increase the change requires. + +These directives use Sphinx MyST syntax, e.g. + +``` +:::{versionadded} VERSION_NEXT_FEATURE +The `allow_new_thing` arg was added. +::: + +:::{versionchanged} VERSION_NEXT_PATCH +Large numbers no longer consume exponential memory. +::: + +:::{versionremoved} VERSION_NEXT_FEATURE +The `legacy_foo` arg was removed +::: +``` + ## Generated files Some checked-in files are generated and need to be updated when a new PR is From edd6bb68439ef9888add2194cac9cbb1b2aac5ef Mon Sep 17 00:00:00 2001 From: James Sharpe Date: Tue, 7 Jan 2025 22:12:14 +0000 Subject: [PATCH 047/922] feat: Add feature to expose whether the native rules are used (#2549) Internally rules_python decides whether the implementation should use the legacy builtin rules from bazel or not. The rules_python attributes have diverged from the builtin rules e.g. `precompile` and so a consumer of this library that wants to support bazel versions that are still using the legacy builtin rules needs a method to be able to query whether the rules are being used. This change adds a entry to features.bzl to expose whether the legacy builtin rules are being used. --- python/features.bzl | 3 +++ 1 file changed, 3 insertions(+) diff --git a/python/features.bzl b/python/features.bzl index 90a1121909..a7098f4710 100644 --- a/python/features.bzl +++ b/python/features.bzl @@ -13,6 +13,8 @@ # limitations under the License. """Allows detecting of rules_python features that aren't easily detected.""" +load("@rules_python_internal//:rules_python_config.bzl", "config") + # This is a magic string expanded by `git archive`, as set by `.gitattributes` # See https://git-scm.com/docs/git-archive/2.29.0#Documentation/git-archive.txt-export-subst _VERSION_PRIVATE = "$Format:%(describe:tags=true)$" @@ -20,4 +22,5 @@ _VERSION_PRIVATE = "$Format:%(describe:tags=true)$" features = struct( version = _VERSION_PRIVATE if "$Format" not in _VERSION_PRIVATE else "", precompile = True, + uses_builtin_rules = not config.enable_pystar, ) From 77cb18b59f3158b36352f2e75afc070180b9ee7f Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Tue, 7 Jan 2025 18:37:32 -0800 Subject: [PATCH 048/922] chore: update changelog for 1.1.0 (#2547) Update headers and version links in CHANGELOG.md for 1.1.0 release --- CHANGELOG.md | 29 +++++++++++++++++++++++++---- 1 file changed, 25 insertions(+), 4 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 6ee44251ca..24b419d1ec 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -52,6 +52,27 @@ Unreleased changes template. {#v0-0-0-changed} ### Changed +* Nothing changed. + +{#v0-0-0-fixed} +### Fixed +* Nothing fixed. + +{#v0-0-0-added} +### Added +* Nothing added. + +{#v0-0-0-removed} +### Removed +* Nothing removed. + +{#v1-1-0} +## [1.1.0] - 2025-01-07 + +[1.1.0]: https://github.com/bazelbuild/rules_python/releases/tag/1.1.0 + +{#v1-1-0-changed} +### Changed * (toolchains) 3.13 means 3.13.1 (previously 3.13.0) * Bazel 6 support is dropped and Bazel 7.4.1 is the minimum supported version, per our Bazel support matrix. Earlier versions are not @@ -68,12 +89,12 @@ Unreleased changes template. * (pypi) Updated versions of packages: `pip` to 24.3.1 and `packaging` to 24.2. -{#v0-0-0-deprecations} +{#v1-1-0-deprecations} #### Deprecations * `//python/config_settings:transitions.bzl` and its `py_binary` and `py_test` wrappers are deprecated. Use the regular rules instead. -{#v0-0-0-fixed} +{#v1-1-0-fixed} ### Fixed * (py_wheel) Use the default shell environment when building wheels to allow toolchains that search PATH to be used for the wheel builder tool. @@ -99,7 +120,7 @@ Unreleased changes template. [pep-695]: https://peps.python.org/pep-0695/ -{#v0-0-0-added} +{#v1-1-0-added} ### Added * (gazelle) Added `include_stub_packages` flag to `modules_mapping`. When set to `True`, this automatically includes corresponding stub packages for third-party libraries @@ -126,7 +147,7 @@ Unreleased changes template. [20241206]: https://github.com/astral-sh/python-build-standalone/releases/tag/20241206 -{#v0-0-0-removed} +{#v1-1-0-removed} ### Removed * `find_requirements` in `//python:defs.bzl` has been removed. From 50b4f87ea81400f81b633f2a9fc06ffdd92b4243 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Tue, 7 Jan 2025 18:37:40 -0800 Subject: [PATCH 049/922] chore: update version strings in docs. (#2546) * Change "VERSION_NEXT" markers to upcoming 1.1.0 release * Change incorrect 0.41.0 mention to 1.0.0 --- python/config_settings/transition.bzl | 2 +- python/private/attributes.bzl | 4 ++-- python/private/py_executable.bzl | 2 +- python/private/py_runtime_info.bzl | 4 ++-- 4 files changed, 6 insertions(+), 6 deletions(-) diff --git a/python/config_settings/transition.bzl b/python/config_settings/transition.bzl index 14e2a73dc4..c241f20746 100644 --- a/python/config_settings/transition.bzl +++ b/python/config_settings/transition.bzl @@ -15,7 +15,7 @@ """The transition module contains the rule definitions to wrap py_binary and py_test and transition them to the desired target platform. -:::{versionchanged} VERSION_NEXT_PATCH +:::{versionchanged} 1.1.0 The `py_binary` and `py_test` symbols are aliases to the regular rules. Usages of them should be changed to load the regular rules directly. ::: diff --git a/python/private/attributes.bzl b/python/private/attributes.bzl index dfe0d4e716..e167482eb1 100644 --- a/python/private/attributes.bzl +++ b/python/private/attributes.bzl @@ -383,7 +383,7 @@ These are dependencies that satisfy imports guarded by `typing.TYPE_CHECKING`. These are build-time only dependencies and not included as part of a runnable program (packaging rules may include them, however). -:::{versionadded} VERSION_NEXT_FEATURE +:::{versionadded} 1.1.0 ::: """, providers = [ @@ -399,7 +399,7 @@ These are typically `.pyi` files, but other file types for type-checker specific formats are allowed. These files are build-time only dependencies and not included as part of a runnable program (packaging rules may include them, however). -:::{versionadded} VERSION_NEXT_FEATURE +:::{versionadded} 1.1.0 ::: """, allow_files = True, diff --git a/python/private/py_executable.bzl b/python/private/py_executable.bzl index 3b063aac95..da7127e070 100644 --- a/python/private/py_executable.bzl +++ b/python/private/py_executable.bzl @@ -151,7 +151,7 @@ may be silently ignored, or an error may occur, depending on the toolchain configuration. ::: -:::{versionchanged} VERSION_NEXT_PATCH +:::{versionchanged} 1.1.0 This attribute was changed from only accepting `PY2` and `PY3` values to accepting arbitrary Python versions. diff --git a/python/private/py_runtime_info.bzl b/python/private/py_runtime_info.bzl index 34be0db69b..19857c9ede 100644 --- a/python/private/py_runtime_info.bzl +++ b/python/private/py_runtime_info.bzl @@ -147,7 +147,7 @@ the same conventions as the standard CPython interpreter. The runtime's ABI flags, i.e. `sys.abiflags`. -:::{versionadded} 0.41.0 +:::{versionadded} 1.0.0 ::: """, "bootstrap_template": """ @@ -281,7 +281,7 @@ are (only) `"PY2"` and `"PY3"`. The template to use for the binary-specific site-init hook run by the interpreter at startup. -:::{versionadded} 0.41.0 +:::{versionadded} 1.0.0 ::: """, "stage2_bootstrap_template": """ From 4e95a60a7376bb0efe3d05df3130cfcb43c3c509 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Tue, 7 Jan 2025 18:37:48 -0800 Subject: [PATCH 050/922] feat: make pypi-generated targets include pyi files (#2545) Make pypi-generated targets set `pyi_srcs` to include `*.pyi` files. Work towards https://github.com/bazelbuild/rules_python/issues/2537 --- CHANGELOG.md | 3 +++ python/private/pypi/whl_library_targets.bzl | 5 +++++ tests/pypi/whl_library_targets/whl_library_targets_tests.bzl | 4 ++++ 3 files changed, 12 insertions(+) diff --git a/CHANGELOG.md b/CHANGELOG.md index 24b419d1ec..d343ae255c 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -88,6 +88,8 @@ Unreleased changes template. `python_version` attribute is still used to specify the Python version. * (pypi) Updated versions of packages: `pip` to 24.3.1 and `packaging` to 24.2. +* (pypi) For pypi-generated targets, `*.pyi` files are included in the + `pyi_srcs` attribute instead of the `data` attribute. {#v1-1-0-deprecations} #### Deprecations @@ -141,6 +143,7 @@ Unreleased changes template. only dependencies added. See {obj}`py_library.pyi_srcs` and `py_library.pyi_deps` (and the same named attributes for `py_binary` and `py_test`). +* (pypi) pypi-generated targets set `pyi_srcs` to include `*.pyi` files. * (providers) {obj}`PyInfo` has new fields to aid static analysis tools: {obj}`direct_original_sources`, {obj}`direct_pyi_files`, {obj}`transitive_original_sources`, {obj}`transitive_pyi_files`. diff --git a/python/private/pypi/whl_library_targets.bzl b/python/private/pypi/whl_library_targets.bzl index a303bdcd9a..461a75cac3 100644 --- a/python/private/pypi/whl_library_targets.bzl +++ b/python/private/pypi/whl_library_targets.bzl @@ -226,6 +226,7 @@ def whl_library_targets( "**/*.py", "**/*.pyc", "**/*.pyc.*", # During pyc creation, temp files named *.pyc.NNNN are created + "**/*.pyi", # RECORD is known to contain sha256 checksums of files which might include the checksums # of generated files produced when wheels are installed. The file is ignored to avoid # Bazel caching issues. @@ -244,6 +245,10 @@ def whl_library_targets( # pure-Python code, e.g. pymssql, which is written in Cython. allow_empty = True, ), + pyi_srcs = native.glob( + ["site-packages/**/*.pyi"], + allow_empty = True, + ), data = data + native.glob( ["site-packages/**/*"], exclude = _data_exclude, diff --git a/tests/pypi/whl_library_targets/whl_library_targets_tests.bzl b/tests/pypi/whl_library_targets/whl_library_targets_tests.bzl index e69eb0f0e9..5d10cf0a5a 100644 --- a/tests/pypi/whl_library_targets/whl_library_targets_tests.bzl +++ b/tests/pypi/whl_library_targets/whl_library_targets_tests.bzl @@ -245,12 +245,14 @@ def _test_whl_and_library_deps(env): exclude = [], allow_empty = True, ), + "pyi_srcs": _glob(["site-packages/**/*.pyi"], allow_empty = True), "data": [] + _glob( ["site-packages/**/*"], exclude = [ "**/*.py", "**/*.pyc", "**/*.pyc.*", + "**/*.pyi", "**/*.dist-info/RECORD", ] + glob_excludes.version_dependent_exclusions(), ), @@ -316,12 +318,14 @@ def _test_group(env): { "name": "_pkg", "srcs": _glob(["site-packages/**/*.py"], exclude = [], allow_empty = True), + "pyi_srcs": _glob(["site-packages/**/*.pyi"], allow_empty = True), "data": [] + _glob( ["site-packages/**/*"], exclude = [ "**/*.py", "**/*.pyc", "**/*.pyc.*", + "**/*.pyi", "**/*.dist-info/RECORD", ] + glob_excludes.version_dependent_exclusions(), ), From 89d850aab819eb2dea9d6340beab1ca810dbe18d Mon Sep 17 00:00:00 2001 From: Brendan Linn Date: Wed, 8 Jan 2025 20:28:38 -0800 Subject: [PATCH 051/922] fix: _which_unchecked: don't watch PATH if binary exists. (#2552) Currently, the _which_unchecked helper unconditionally watches the `PATH` env var via repository_ctx.getenv. getenv is documented https://bazel.build/rules/lib/builtins/repository_ctx#getenv: > any change to the value of the variable named by name will cause this repository to be re-fetched. Thus, any change to `PATH` will cause any repository rule that transitively calls _which_unchecked to be re-fetched. This includes python_repository and whl_library. There are reasonable development workflows that modify `PATH`. In particular, when git runs a hook, it adds the value of `GIT_EXEC_PATH` to `PATH` before invoking the hook. If the hook invokes bazel (for example, a pre-commit hook running `bazel build ...`), it will cause the Python repository rules to be re-fetched. This commit lowers the repository_ctx.getenv("PATH") call to its only use site in _which_unchecked, which happens to be a failure case (when the binary is not found). This allows the success case to not watch `PATH`, and therefore not to re-fetch the repository rule when it changes. Fixes https://github.com/bazelbuild/rules_python/issues/2551. --- CHANGELOG.md | 3 +++ python/private/repo_utils.bzl | 4 ++-- 2 files changed, 5 insertions(+), 2 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index d343ae255c..6ccc568d26 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -119,6 +119,9 @@ Unreleased changes template. change. Fixes [#2468](https://github.com/bazelbuild/rules_python/issues/2468). + (gazelle) Gazelle no longer ignores `setup.py` files by default. To restore this behavior, apply the `# gazelle:python_ignore_files setup.py` directive. +* Don't re-fetch whl_library, python_repository, etc. repository rules + whenever `PATH` changes. Fixes + [#2551](https://github.com/bazelbuild/rules_python/issues/2551). [pep-695]: https://peps.python.org/pep-0695/ diff --git a/python/private/repo_utils.bzl b/python/private/repo_utils.bzl index 0e3f7b024b..e5c78be815 100644 --- a/python/private/repo_utils.bzl +++ b/python/private/repo_utils.bzl @@ -256,7 +256,7 @@ def _which_checked(mrctx, binary_name): def _which_unchecked(mrctx, binary_name): """Tests to see if a binary exists. - This is also watch the `PATH` environment variable. + Watches the `PATH` environment variable if the binary doesn't exist. Args: binary_name: name of the binary to find. @@ -268,12 +268,12 @@ def _which_unchecked(mrctx, binary_name): * `describe_failure`: `Callable | None`; takes no args. If the binary couldn't be found, provides a detailed error description. """ - path = _getenv(mrctx, "PATH", "") binary = mrctx.which(binary_name) if binary: _watch(mrctx, binary) describe_failure = None else: + path = _getenv(mrctx, "PATH", "") describe_failure = lambda: _which_describe_failure(binary_name, path) return struct( From 38135f7d7eab5916cd301e131e5c07698cb0e7b8 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Thu, 9 Jan 2025 13:13:48 -0800 Subject: [PATCH 052/922] chore: update release check to ignore VERSION_NEXT substring in CONTRIBUTING.md (#2553) Otherwise the release action fails, thinking there are version markers. --- .github/workflows/create_archive_and_notes.sh | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/.github/workflows/create_archive_and_notes.sh b/.github/workflows/create_archive_and_notes.sh index b425de96ac..29f9f8b9f7 100755 --- a/.github/workflows/create_archive_and_notes.sh +++ b/.github/workflows/create_archive_and_notes.sh @@ -17,7 +17,8 @@ set -o errexit -o nounset -o pipefail # Exclude dot directories, specifically, this file so that we don't # find the substring we're looking for in our own file. -if grep --exclude-dir=.* VERSION_NEXT_ -r; then +# Exclude CONTRIBUTING.md because it documents how to use these strings. +if grep --exclude=CONTRIBUTING.md --exclude-dir=.* VERSION_NEXT_ -r; then echo echo "Found VERSION_NEXT markers indicating version needs to be specified" exit 1 From eae098548f053c1bc3f2ab77620154d0444ce236 Mon Sep 17 00:00:00 2001 From: Spencer Putt Date: Fri, 10 Jan 2025 10:37:19 -0800 Subject: [PATCH 053/922] fix(gazelle): Fix the requirements arg to the gazelle python manifest generator. (#2533) Fix args passed into the gazelle manifest file generator if the requirements file is not a source file. --------- Co-authored-by: Douglas Thor --- CHANGELOG.md | 4 ++-- gazelle/manifest/defs.bzl | 2 +- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 6ccc568d26..7f4c60b4b3 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,7 +4,7 @@ # rules_python Changelog This is a human-friendly changelog in a keepachangelog.com style format. -Because this changelog is for end-user consumption of meaningful changes,only +Because this changelog is for end-user consumption of meaningful changes, only a summary of a release's changes is described. This means every commit is not necessarily mentioned, and internal refactors or code cleanups are omitted unless they're particularly notable. @@ -56,7 +56,7 @@ Unreleased changes template. {#v0-0-0-fixed} ### Fixed -* Nothing fixed. +* (gazelle) Providing multiple input requirements files to `gazelle_python_manifest` now works correctly. {#v0-0-0-added} ### Added diff --git a/gazelle/manifest/defs.bzl b/gazelle/manifest/defs.bzl index 3a65bffec4..6c0072a48b 100644 --- a/gazelle/manifest/defs.bzl +++ b/gazelle/manifest/defs.bzl @@ -79,7 +79,7 @@ def gazelle_python_manifest( update_args = [ "--manifest-generator-hash=$(execpath {})".format(manifest_generator_hash), - "--requirements=$(rootpath {})".format(requirements) if requirements else "--requirements=", + "--requirements=$(execpath {})".format(requirements) if requirements else "--requirements=", "--pip-repository-name={}".format(pip_repository_name), "--modules-mapping=$(execpath {})".format(modules_mapping), "--output=$(execpath {})".format(generated_manifest), From 4db0d919c8d12caa816c5b797bd28b0d457409d1 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Sat, 11 Jan 2025 06:30:04 -0800 Subject: [PATCH 054/922] docs: note direct_pyi_files/transitive_pyi_files are usually build-time only (#2555) The pyi_srcs and pyi_deps attributes have this disclaimer already. The provider should, too, so behavior is better specified. --- python/private/py_info.bzl | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/python/private/py_info.bzl b/python/private/py_info.bzl index 2a02685362..ef654c303e 100644 --- a/python/private/py_info.bzl +++ b/python/private/py_info.bzl @@ -112,6 +112,9 @@ Type definition files (usually `.pyi` files) for the Python modules provided by this target. Usually they describe the source files listed in `direct_original_sources`. This field is primarily for static analysis tools. +These files are _usually_ build-time only and not included as part of a runnable +program. + :::{note} This may contain implementation-specific file types specific to a particular type checker. @@ -190,6 +193,9 @@ Python modules for this target and its transitive dependencies. this target. Usually they describe the source files listed in `transitive_original_sources`. This field is primarily for static analysis tools. +These files are _usually_ build-time only and not included as part of a runnable +program. + :::{note} This may contain implementation-specific file types specific to a particular type checker. From 51f10470a906611979c5e62c92aa08648980bc94 Mon Sep 17 00:00:00 2001 From: Ignas Anikevicius <240938+aignas@users.noreply.github.com> Date: Sun, 12 Jan 2025 22:15:02 +0900 Subject: [PATCH 055/922] refactor(pypi): rename config settings and improve docs (#2556) Before the PR the `config_setting` names where following an internal logic and those names would be leaking into the error messages when no match is found. I thought that the names thus should be improved and maybe made more similar to the `whl` filename parts that they are derived from. As part of this change I have also added more docs and added them to sphinxdocs in the hopes that this documentation helps maintainers and users looking at error messages alike. Summary: * Make names more similar to the whl filenames. * Instead of having `osx__universal2` config settings for each `cpu` value, have a single `osx_universal2` config setting. * Stop creating redundant/unused config settings * Refactor the `_dist_config_setting` code to be simpler and create fewer targets by using a clever trick for the `whl` config setting flag value usage. The stats: ``` $ bazel query //tests/pypi/config_settings/... | rg ":(|_)is" | wc -l 2223 $ bazel query @dev_pip//_config/... | wc -l 1982 $ bazel query //tests/pypi/config_settings/... | rg ":(|_)is" | wc -l 1780 $ bazel query @dev_pip//_config/... | wc -l 1066 ``` Work towards #260 --- docs/BUILD.bazel | 2 + python/private/pypi/BUILD.bazel | 61 ++-- python/private/pypi/config_settings.bzl | 261 ++++++++---------- python/private/pypi/flags.bzl | 37 ++- python/private/pypi/pkg_aliases.bzl | 129 ++++++--- python/private/pypi/render_pkg_aliases.bzl | 2 +- .../config_settings/config_settings_tests.bzl | 214 +++++++++----- tests/pypi/pkg_aliases/pkg_aliases_test.bzl | 44 +-- .../render_pkg_aliases_test.bzl | 44 ++- 9 files changed, 474 insertions(+), 320 deletions(-) diff --git a/docs/BUILD.bazel b/docs/BUILD.bazel index a9a1db02a8..e365532d01 100644 --- a/docs/BUILD.bazel +++ b/docs/BUILD.bazel @@ -103,6 +103,8 @@ sphinx_stardocs( "//python/private:py_runtime_rule_bzl", "//python/private:py_test_rule_bzl", "//python/private/api:py_common_api_bzl", + "//python/private/pypi:config_settings_bzl", + "//python/private/pypi:pkg_aliases_bzl", ] + ([ # Bazel 6 + Stardoc isn't able to parse something about the python bzlmod extension "//python/extensions:python_bzl", diff --git a/python/private/pypi/BUILD.bazel b/python/private/pypi/BUILD.bazel index 7e2d398fde..6f80272af6 100644 --- a/python/private/pypi/BUILD.bazel +++ b/python/private/pypi/BUILD.bazel @@ -53,6 +53,32 @@ bzl_library( srcs = ["attrs.bzl"], ) +bzl_library( + name = "config_settings_bzl", + srcs = ["config_settings.bzl"], + deps = [ + ":flags_bzl", + "//python/private:flags_bzl", + ], +) + +bzl_library( + name = "deps_bzl", + srcs = ["deps.bzl"], + deps = [ + "//python/private:bazel_tools_bzl", + "//python/private:glob_excludes_bzl", + ], +) + +bzl_library( + name = "evaluate_markers_bzl", + srcs = ["evaluate_markers.bzl"], + deps = [ + ":pypi_repo_utils_bzl", + ], +) + bzl_library( name = "extension_bzl", srcs = ["extension.bzl"], @@ -76,29 +102,6 @@ bzl_library( ], ) -bzl_library( - name = "config_settings_bzl", - srcs = ["config_settings.bzl"], - deps = ["flags_bzl"], -) - -bzl_library( - name = "deps_bzl", - srcs = ["deps.bzl"], - deps = [ - "//python/private:bazel_tools_bzl", - "//python/private:glob_excludes_bzl", - ], -) - -bzl_library( - name = "evaluate_markers_bzl", - srcs = ["evaluate_markers.bzl"], - deps = [ - ":pypi_repo_utils_bzl", - ], -) - bzl_library( name = "flags_bzl", srcs = ["flags.bzl"], @@ -245,6 +248,18 @@ bzl_library( srcs = ["pip_repository_attrs.bzl"], ) +bzl_library( + name = "pkg_aliases_bzl", + srcs = ["pkg_aliases.bzl"], + deps = [ + ":labels_bzl", + ":parse_whl_name_bzl", + ":whl_target_platforms_bzl", + "//python/private:text_util_bzl", + "@bazel_skylib//lib:selects", + ], +) + bzl_library( name = "pypi_repo_utils_bzl", srcs = ["pypi_repo_utils.bzl"], diff --git a/python/private/pypi/config_settings.bzl b/python/private/pypi/config_settings.bzl index 620e50e997..1045ffef35 100644 --- a/python/private/pypi/config_settings.bzl +++ b/python/private/pypi/config_settings.bzl @@ -13,31 +13,59 @@ # limitations under the License. """ -This module is used to construct the config settings for selecting which distribution is used in the pip hub repository. +The {obj}`config_settings` macro is used to create the config setting targets +that can be used in the {obj}`pkg_aliases` macro for selecting the compatible +repositories. Bazel's selects work by selecting the most-specialized configuration setting -that matches the target platform. We can leverage this fact to ensure that the -most specialized wheels are used by default with the users being able to -configure string_flag values to select the less specialized ones. - -The list of specialization of the dists goes like follows (cpxyt stands for freethreaded -environments): -* sdist -* py*-none-any.whl -* py*-abi3-any.whl -* py*-cpxy-any.whl or py*-cpxyt-any.whl -* cp*-none-any.whl -* cp*-abi3-any.whl -* cp*-cpxy-any.whl or cp*-cpxyt-any.whl -* py*-none-plat.whl -* py*-abi3-plat.whl -* py*-cpxy-plat.whl or py*-cpxyt-plat.whl -* cp*-none-plat.whl -* cp*-abi3-plat.whl -* cp*-cpxy-plat.whl or cp*-cpxyt-plat.whl - -Note, that here the specialization of musl vs manylinux wheels is the same in -order to ensure that the matching fails if the user requests for `musl` and we don't have it or vice versa. +that matches the target platform, which is further described in [bazel documentation][docs]. +We can leverage this fact to ensure that the most specialized matches are used +by default with the users being able to configure string_flag values to select +the less specialized ones. + +[docs]: https://bazel.build/docs/configurable-attributes + +The config settings in the order from the least specialized to the most +specialized is as follows: +* `:is_cp3` +* `:is_cp3_sdist` +* `:is_cp3_py_none_any` +* `:is_cp3_py3_none_any` +* `:is_cp3_py3_abi3_any` +* `:is_cp3_none_any` +* `:is_cp3_any_any` +* `:is_cp3_cp3_any` and `:is_cp3_cp3t_any` +* `:is_cp3_py_none_` +* `:is_cp3_py3_none_` +* `:is_cp3_py3_abi3_` +* `:is_cp3_none_` +* `:is_cp3_abi3_` +* `:is_cp3_cp3_` and `:is_cp3_cp3t_` + +The specialization of free-threaded vs non-free-threaded wheels is the same as +they are just variants of each other. The same goes for the specialization of +`musllinux` vs `manylinux`. + +The goal of this macro is to provide config settings that provide unambigous +matches if any pair of them is used together for any target configuration +setting. We achieve this by using dummy internal `flag_values` keys to force the +items further down the list to appear to be more specialized than the ones above. + +What is more, the names of the config settings are as similar to the platform wheel +specification as possible. How the wheel names map to the config setting names defined +in here is described in {obj}`pkg_aliases` documentation. + +:::{note} +Right now the specialization of adjacent config settings where one is with +`constraint_values` and one is without is ambiguous. I.e. `py_none_any` and +`sdist_linux_x86_64` have the same specialization from bazel point of view +because one has one `flag_value` entry and `constraint_values` and the +other has 2 flag_value entries. And unfortunately there is no way to disambiguate +it, because we are essentially in two dimensions here (`flag_values` and +`constraint_values`). Hence, when using the `config_settings` from here, +either have all of them with empty `suffix` or all of them with a non-empty +suffix. +::: """ load("//python/private:flags.bzl", "LibcFlag") @@ -83,8 +111,7 @@ def config_settings( osx_versions = [], target_platforms = [], name = None, - visibility = None, - native = native): + **kwargs): """Generate all of the pip config settings. Args: @@ -99,35 +126,19 @@ def config_settings( config settings for. target_platforms (list[str]): The list of "{os}_{cpu}" for deriving constraint values for each condition. - visibility (list[str], optional): The visibility to be passed to the - exposed labels. All other labels will be private. - native (struct): The struct containing alias and config_setting rules - to use for creating the objects. Can be overridden for unit tests - reasons. + **kwargs: Other args passed to the underlying implementations, such as + {obj}`native`. """ glibc_versions = [""] + glibc_versions muslc_versions = [""] + muslc_versions osx_versions = [""] + osx_versions - target_platforms = [("", "")] + [ + target_platforms = [("", ""), ("osx", "universal2")] + [ t.split("_", 1) for t in target_platforms ] - for python_version in [""] + python_versions: - is_python = "is_python_{}".format(python_version or "version_unset") - - # The aliases defined in @rules_python//python/config_settings may not - # have config settings for the versions we need, so define our own - # config settings instead. - native.config_setting( - name = is_python, - flag_values = { - Label("//python/config_settings:python_version_major_minor"): python_version, - }, - visibility = visibility, - ) - + for python_version in python_versions: for os, cpu in target_platforms: constraint_values = [] suffix = "" @@ -135,8 +146,9 @@ def config_settings( constraint_values.append("@platforms//os:" + os) suffix += "_" + os if cpu: - constraint_values.append("@platforms//cpu:" + cpu) suffix += "_" + cpu + if cpu != "universal2": + constraint_values.append("@platforms//cpu:" + cpu) _dist_config_settings( suffix = suffix, @@ -149,20 +161,24 @@ def config_settings( ), constraint_values = constraint_values, python_version = python_version, - is_python = is_python, - visibility = visibility, - native = native, + **kwargs ) -def _dist_config_settings(*, suffix, plat_flag_values, **kwargs): - if kwargs.get("constraint_values"): - # Add python version + platform config settings - _dist_config_setting( - name = suffix.strip("_"), - **kwargs - ) +def _dist_config_settings(*, suffix, plat_flag_values, python_version, **kwargs): + flag_values = { + Label("//python/config_settings:python_version_major_minor"): python_version, + } - flag_values = {_flags.dist: ""} + cpv = "cp" + python_version.replace(".", "") + prefix = "is_{}".format(cpv) + + _dist_config_setting( + name = prefix + suffix, + flag_values = flag_values, + **kwargs + ) + + flag_values[_flags.dist] = "" # First create an sdist, we will be building upon the flag values, which # will ensure that each sdist config setting is the least specialized of @@ -170,9 +186,9 @@ def _dist_config_settings(*, suffix, plat_flag_values, **kwargs): # have `sdist` for any platform, hence we have a non-empty `flag_values` # here. _dist_config_setting( - name = "sdist{}".format(suffix), + name = "{}_sdist{}".format(prefix, suffix), flag_values = flag_values, - is_pip_whl = FLAGS.is_pip_whl_no, + compatible_with = (FLAGS.is_pip_whl_no, FLAGS.is_pip_whl_auto), **kwargs ) @@ -184,29 +200,28 @@ def _dist_config_settings(*, suffix, plat_flag_values, **kwargs): # The discussion here also reinforces this notion: # https://discuss.python.org/t/pep-703-making-the-global-interpreter-lock-optional-3-12-updates/26503/99 - for name, f, abi in [ - ("py_none", _flags.whl_py2_py3, None), + for name, f, compatible_with in [ + ("py_none", _flags.whl, None), ("py3_none", _flags.whl_py3, None), ("py3_abi3", _flags.whl_py3_abi3, (FLAGS.is_py_non_freethreaded,)), - ("cp3x_none", _flags.whl_pycp3x, None), - ("cp3x_abi3", _flags.whl_pycp3x_abi3, (FLAGS.is_py_non_freethreaded,)), + ("none", _flags.whl_pycp3x, None), + ("abi3", _flags.whl_pycp3x_abi3, (FLAGS.is_py_non_freethreaded,)), # The below are not specializations of one another, they are variants - ("cp3x_cp", _flags.whl_pycp3x_abicp, (FLAGS.is_py_non_freethreaded,)), - ("cp3x_cpt", _flags.whl_pycp3x_abicp, (FLAGS.is_py_freethreaded,)), + (cpv, _flags.whl_pycp3x_abicp, (FLAGS.is_py_non_freethreaded,)), + (cpv + "t", _flags.whl_pycp3x_abicp, (FLAGS.is_py_freethreaded,)), ]: - if (f, abi) in used_flags: + if (f, compatible_with) in used_flags: # This should never happen as all of the different whls should have # unique flag values fail("BUG: the flag {} is attempted to be added twice to the list".format(f)) else: - flag_values[f] = "" - used_flags[(f, abi)] = True + flag_values[f] = "yes" if f == _flags.whl else "" + used_flags[(f, compatible_with)] = True _dist_config_setting( - name = "{}_any{}".format(name, suffix), + name = "{}_{}_any{}".format(prefix, name, suffix), flag_values = flag_values, - is_pip_whl = FLAGS.is_pip_whl_only, - abi = abi, + compatible_with = compatible_with, **kwargs ) @@ -217,29 +232,28 @@ def _dist_config_settings(*, suffix, plat_flag_values, **kwargs): used_flags = {(f, None): True for f in flag_values} | generic_used_flags flag_values = flag_values | generic_flag_values - for name, f, abi in [ + for name, f, compatible_with in [ ("py_none", _flags.whl_plat, None), ("py3_none", _flags.whl_plat_py3, None), ("py3_abi3", _flags.whl_plat_py3_abi3, (FLAGS.is_py_non_freethreaded,)), - ("cp3x_none", _flags.whl_plat_pycp3x, None), - ("cp3x_abi3", _flags.whl_plat_pycp3x_abi3, (FLAGS.is_py_non_freethreaded,)), + ("none", _flags.whl_plat_pycp3x, None), + ("abi3", _flags.whl_plat_pycp3x_abi3, (FLAGS.is_py_non_freethreaded,)), # The below are not specializations of one another, they are variants - ("cp3x_cp", _flags.whl_plat_pycp3x_abicp, (FLAGS.is_py_non_freethreaded,)), - ("cp3x_cpt", _flags.whl_plat_pycp3x_abicp, (FLAGS.is_py_freethreaded,)), + (cpv, _flags.whl_plat_pycp3x_abicp, (FLAGS.is_py_non_freethreaded,)), + (cpv + "t", _flags.whl_plat_pycp3x_abicp, (FLAGS.is_py_freethreaded,)), ]: - if (f, abi) in used_flags: + if (f, compatible_with) in used_flags: # This should never happen as all of the different whls should have # unique flag values. fail("BUG: the flag {} is attempted to be added twice to the list".format(f)) else: flag_values[f] = "" - used_flags[(f, abi)] = True + used_flags[(f, compatible_with)] = True _dist_config_setting( - name = "{}_{}".format(name, suffix), + name = "{}_{}_{}".format(prefix, name, suffix), flag_values = flag_values, - is_pip_whl = FLAGS.is_pip_whl_only, - abi = abi, + compatible_with = compatible_with, **kwargs ) @@ -256,23 +270,19 @@ def _plat_flag_values(os, cpu, osx_versions, glibc_versions, muslc_versions): elif os == "windows": ret.append(("{}_{}".format(os, cpu), {})) elif os == "osx": - for cpu_, arch in { - cpu: UniversalWhlFlag.ARCH, - cpu + "_universal2": UniversalWhlFlag.UNIVERSAL, - }.items(): - for osx_version in osx_versions: - flags = { - FLAGS.pip_whl_osx_version: _to_version_string(osx_version), - } - if arch == UniversalWhlFlag.ARCH: - flags[FLAGS.pip_whl_osx_arch] = arch - - if not osx_version: - suffix = "{}_{}".format(os, cpu_) - else: - suffix = "{}_{}_{}".format(os, _to_version_string(osx_version, "_"), cpu_) + for osx_version in osx_versions: + flags = { + FLAGS.pip_whl_osx_version: _to_version_string(osx_version), + } + if cpu != "universal2": + flags[FLAGS.pip_whl_osx_arch] = UniversalWhlFlag.ARCH + + if not osx_version: + suffix = "{}_{}".format(os, cpu) + else: + suffix = "{}_{}_{}".format(os, _to_version_string(osx_version, "_"), cpu) - ret.append((suffix, flags)) + ret.append((suffix, flags)) elif os == "linux": for os_prefix, linux_libc in { @@ -309,16 +319,12 @@ def _plat_flag_values(os, cpu, osx_versions, glibc_versions, muslc_versions): return ret -def _dist_config_setting(*, name, is_python, python_version, is_pip_whl = None, abi = None, native = native, **kwargs): - """A macro to create a target that matches is_pip_whl_auto and one more value. +def _dist_config_setting(*, name, compatible_with = None, native = native, **kwargs): + """A macro to create a target for matching Python binary and source distributions. Args: name: The name of the public target. - is_pip_whl: The config setting to match in addition to - `is_pip_whl_auto` when evaluating the config setting. - is_python: The python version config_setting to match. - python_version: The python version name. - abi: {type}`tuple[Label]` A collection of ABI config settings that are + compatible_with: {type}`tuple[Label]` A collection of config settings that are compatible with the given dist config setting. For example, if only non-freethreaded python builds are allowed, add FLAGS.is_py_non_freethreaded here. @@ -328,50 +334,17 @@ def _dist_config_setting(*, name, is_python, python_version, is_pip_whl = None, **kwargs: The kwargs passed to the config_setting rule. Visibility of the main alias target is also taken from the kwargs. """ - _name = "_is_" + name - - visibility = kwargs.get("visibility") - native.alias( - name = "is_cp{}_{}".format(python_version, name) if python_version else "is_{}".format(name), - actual = select({ - # First match by the python version and then by ABI - is_python: _name + ("_abi" if abi else ""), - _DEFAULT: _INCOMPATIBLE, - }), - visibility = visibility, - ) - - if python_version: - # Reuse the config_setting targets that we use with the default - # `python_version` setting. - return - - if not is_pip_whl: - native.config_setting(name = _name, **kwargs) - return - - config_setting_name = _name + "_setting" - native.config_setting(name = config_setting_name, **kwargs) - - if abi: + if compatible_with: + dist_config_setting_name = "_" + name native.alias( - name = _name + "_abi", + name = name, actual = select( - {k: _name for k in abi} | { + {setting: dist_config_setting_name for setting in compatible_with} | { _DEFAULT: _INCOMPATIBLE, }, ), - visibility = visibility, + visibility = kwargs.get("visibility"), ) + name = dist_config_setting_name - # Next match by the `pip_whl` flag value and then match by the flags that - # are intrinsic to the distribution. - native.alias( - name = _name, - actual = select({ - _DEFAULT: _INCOMPATIBLE, - FLAGS.is_pip_whl_auto: config_setting_name, - is_pip_whl: config_setting_name, - }), - visibility = visibility, - ) + native.config_setting(name = name, **kwargs) diff --git a/python/private/pypi/flags.bzl b/python/private/pypi/flags.bzl index 11727b5853..a25579a2b8 100644 --- a/python/private/pypi/flags.bzl +++ b/python/private/pypi/flags.bzl @@ -18,7 +18,7 @@ NOTE: The transitive loads of this should be kept minimal. This avoids loading unnecessary files when all that are needed are flag definitions. """ -load("@bazel_skylib//rules:common_settings.bzl", "string_flag") +load("@bazel_skylib//rules:common_settings.bzl", "BuildSettingInfo", "string_flag") load("//python/private:enum.bzl", "enum") # Determines if we should use whls for third party @@ -44,7 +44,7 @@ UniversalWhlFlag = enum( UNIVERSAL = "universal", ) -INTERNAL_FLAGS = [ +_STRING_FLAGS = [ "dist", "whl_plat", "whl_plat_py3", @@ -52,7 +52,6 @@ INTERNAL_FLAGS = [ "whl_plat_pycp3x", "whl_plat_pycp3x_abi3", "whl_plat_pycp3x_abicp", - "whl_py2_py3", "whl_py3", "whl_py3_abi3", "whl_pycp3x", @@ -60,11 +59,41 @@ INTERNAL_FLAGS = [ "whl_pycp3x_abicp", ] +INTERNAL_FLAGS = [ + "whl", +] + _STRING_FLAGS + def define_pypi_internal_flags(name): - for flag in INTERNAL_FLAGS: + """define internal PyPI flags used in PyPI hub repository by pkg_aliases. + + Args: + name: not used + """ + for flag in _STRING_FLAGS: string_flag( name = "_internal_pip_" + flag, build_setting_default = "", values = [""], visibility = ["//visibility:public"], ) + + _allow_wheels_flag( + name = "_internal_pip_whl", + visibility = ["//visibility:public"], + ) + +def _allow_wheels_flag_impl(ctx): + input = ctx.attr._setting[BuildSettingInfo].value + value = "yes" if input in ["auto", "only"] else "no" + return [config_common.FeatureFlagInfo(value = value)] + +_allow_wheels_flag = rule( + implementation = _allow_wheels_flag_impl, + attrs = { + "_setting": attr.label(default = "//python/config_settings:pip_whl"), + }, + doc = """\ +This rule allows us to greatly reduce the number of config setting targets at no cost even +if we are duplicating some of the functionality of the `native.config_setting`. +""", +) diff --git a/python/private/pypi/pkg_aliases.bzl b/python/private/pypi/pkg_aliases.bzl index 980921b474..a9eee7be88 100644 --- a/python/private/pypi/pkg_aliases.bzl +++ b/python/private/pypi/pkg_aliases.bzl @@ -12,9 +12,66 @@ # See the License for the specific language governing permissions and # limitations under the License. -"""pkg_aliases is a macro to generate aliases for selecting the right wheel for the right target platform. - -This is used in bzlmod and non-bzlmod setups.""" +"""{obj}`pkg_aliases` is a macro to generate aliases for selecting the right wheel for the right target platform. + +If you see an error where the distribution selection error indicates the config setting names this +page may help to describe the naming convention and relationship between various flags and options +in `rules_python` and the error message contents. + +Definitions: +:minor_version: Python interpreter minor version that the distributions are compatible with. +:suffix: Can be either empty or `__`, which is usually used to distinguish multiple versions used for different target platforms. +:os: OS identifier that exists in `@platforms//os:`. +:cpu: CPU architecture identifier that exists in `@platforms//cpu:`. +:python_tag: The Python tag as defined by the [Python Packaging Authority][packaging_spec]. E.g. `py2.py3`, `py3`, `py311`, `cp311`. +:abi_tag: The ABI tag as defined by the [Python Packaging Authority][packaging_spec]. E.g. `none`, `abi3`, `cp311`, `cp311t`. +:platform_tag: The Platform tag as defined by the [Python Packaging Authority][packaging_spec]. E.g. `manylinux_2_17_x86_64`. +:platform_suffix: is a derivative of the `platform_tag` and is used to implement selection based on `libc` or `osx` version. + +All of the config settings used by this macro are generated by +{obj}`config_settings`, for more detailed documentation on what each config +setting maps to and their precedence, refer to documentation on that page. + +The first group of config settings that are as follows: + +* `//_config:is_cp3` is used to select legacy `pip` + based `whl` and `sdist` {obj}`whl_library` instances. Whereas other config + settings are created when {obj}`pip.parse.experimental_index_url` is used. +* `//_config:is_cp3_sdist` is for wheels built from + `sdist` in {obj}`whl_library`. +* `//_config:is_cp3_py__any` for wheels with + `py2.py3` `python_tag` value. +* `//_config:is_cp3_py3__any` for wheels with + `py3` `python_tag` value. +* `//_config:is_cp3__any` for any other wheels. +* `//_config:is_cp3_py__` for + platform-specific wheels with `py2.py3` `python_tag` value. +* `//_config:is_cp3_py3__` for + platform-specific wheels with `py3` `python_tag` value. +* `//_config:is_cp3__` for any other + platform-specific wheels. + +Note that wheels with `abi3` or `none` `abi_tag` values and `python_tag` values +other than `py2.py3` or `py3` are compatible with the python version that is +equal or higher than the one denoted in the `python_tag`. For example: `py37` +and `cp37` wheels are compatible with Python 3.7 and above and in the case of +the target python version being `3.11`, `rules_python` will use +`//_config:is_cp311__any` config settings. + +For platform-specific wheels, i.e. the ones that have their `platform_tag` as +something else than `any`, we treat them as below: +* `linux_` tags assume that the target `libc` flavour is `glibc`, so this + is in many ways equivalent to it being `manylinux`, but with an unspecified + `libc` version. +* For `osx` and `linux` OSes wheel filename will be mapped to multiple config settings: + * `osx_` and `osx___` where + `major_version` and `minor_version` are the compatible OSX versions. + * `linux_` and + `linux___` where the version + identifiers are the compatible libc versions. + +[packaging_spec]: https://packaging.python.org/en/latest/specifications/platform-compatibility-tags/ +""" load("@bazel_skylib//lib:selects.bzl", "selects") load("//python/private:text_util.bzl", "render") @@ -66,11 +123,11 @@ def pkg_aliases( actual, group_name = None, extra_aliases = None, - native = native, - select = selects.with_or, **kwargs): """Create aliases for an actual package. + Exposed only to be used from the hub repositories created by `rules_python`. + Args: name: {type}`str` The name of the package. actual: {type}`dict[Label | tuple, str] | str` The name of the repo the @@ -79,11 +136,12 @@ def pkg_aliases( to bazel skylib's `selects.with_or`, so they can be tuples as well. group_name: {type}`str` The group name that the pkg belongs to. extra_aliases: {type}`list[str]` The extra aliases to be created. - native: {type}`struct` used in unit tests. - select: {type}`select` used in unit tests. **kwargs: extra kwargs to pass to {bzl:obj}`get_filename_config_settings`. """ - native.alias( + alias = kwargs.pop("native", native).alias + select = kwargs.pop("select", selects.with_or) + + alias( name = name, actual = ":" + PY_LIBRARY_PUBLIC_LABEL, ) @@ -100,7 +158,7 @@ def pkg_aliases( actual = multiplatform_whl_aliases(aliases = actual, **kwargs) if type(actual) == type({}) and "//conditions:default" not in actual: - native.alias( + alias( name = _INCOMPATIBLE, actual = select( {_LABEL_CURRENT_CONFIG_NO_MATCH: _LABEL_NONE}, @@ -143,18 +201,18 @@ def pkg_aliases( if target_name.startswith("_"): kwargs["visibility"] = ["//_groups:__subpackages__"] - native.alias( + alias( name = target_name, actual = _actual, **kwargs ) if group_name: - native.alias( + alias( name = PY_LIBRARY_PUBLIC_LABEL, actual = "//_groups:{}_pkg".format(group_name), ) - native.alias( + alias( name = WHEEL_FILE_PUBLIC_LABEL, actual = "//_groups:{}_whl".format(group_name), ) @@ -176,6 +234,8 @@ def multiplatform_whl_aliases( osx_versions = []): """convert a list of aliases from filename to config_setting ones. + Exposed only for unit tests. + Args: aliases: {type}`str | dict[whl_config_setting | str, str]`: The aliases to process. Any aliases that have the filename set will be @@ -277,6 +337,8 @@ def get_filename_config_settings( non_whl_prefix = "sdist"): """Get the filename config settings. + Exposed only for unit tests. + Args: filename: the distribution filename (can be a whl or an sdist). target_platforms: list[str], target platforms in "{abi}_{os}_{cpu}" format. @@ -299,23 +361,20 @@ def get_filename_config_settings( if filename.endswith(".whl"): parsed = parse_whl_name(filename) if parsed.python_tag == "py2.py3": - py = "py" + py = "py_" + elif parsed.python_tag == "py3": + py = "py3_" elif parsed.python_tag.startswith("cp"): - py = "cp3x" + py = "" else: - py = "py3" + py = "py3_" - if parsed.abi_tag.startswith("cp") and parsed.abi_tag.endswith("t"): - abi = "cpt" - elif parsed.abi_tag.startswith("cp"): - abi = "cp" - else: - abi = parsed.abi_tag + abi = parsed.abi_tag if parsed.platform_tag == "any": - prefixes = ["_{}_{}_any".format(py, abi)] + prefixes = ["{}{}_any".format(py, abi)] else: - prefixes = ["_{}_{}".format(py, abi)] + prefixes = ["{}{}".format(py, abi)] suffixes = _whl_config_setting_suffixes( platform_tag = parsed.platform_tag, glibc_versions = glibc_versions, @@ -324,14 +383,20 @@ def get_filename_config_settings( setting_supported_versions = setting_supported_versions, ) else: - prefixes = [""] if not non_whl_prefix else ["_" + non_whl_prefix] + prefixes = [non_whl_prefix or ""] + + py = "cp{}".format(python_version).replace(".", "") + prefixes = [ + "{}_{}".format(py, prefix) if prefix else py + for prefix in prefixes + ] versioned = { - ":is_cp{}{}_{}".format(python_version, p, suffix): { - version: ":is_cp{}{}_{}".format(python_version, p, setting) + ":is_{}_{}".format(prefix, suffix): { + version: ":is_{}_{}".format(prefix, setting) for version, setting in versions.items() } - for p in prefixes + for prefix in prefixes for suffix, versions in setting_supported_versions.items() } @@ -339,12 +404,12 @@ def get_filename_config_settings( target_platforms = target_platforms or [] suffixes = suffixes or [_non_versioned_platform(p) for p in target_platforms] return [ - ":is_cp{}{}_{}".format(python_version, p, s) - for p in prefixes - for s in suffixes + ":is_{}_{}".format(prefix, suffix) + for prefix in prefixes + for suffix in suffixes ], versioned else: - return [":is_cp{}{}".format(python_version, p) for p in prefixes], setting_supported_versions + return [":is_{}".format(p) for p in prefixes], setting_supported_versions def _whl_config_setting_suffixes( platform_tag, @@ -368,7 +433,7 @@ def _whl_config_setting_suffixes( elif p.os == "osx": versions = osx_versions if "universal2" in platform_tag: - suffix += "_universal2" + suffix = "universal2" else: fail("Unsupported whl os: {}".format(p.os)) diff --git a/python/private/pypi/render_pkg_aliases.bzl b/python/private/pypi/render_pkg_aliases.bzl index 62c893d3e7..863d25095c 100644 --- a/python/private/pypi/render_pkg_aliases.bzl +++ b/python/private/pypi/render_pkg_aliases.bzl @@ -60,7 +60,7 @@ def _repr_config_setting(alias): ) else: return repr( - alias.config_setting or ("//_config:is_python_" + alias.version), + alias.config_setting or "//_config:is_cp{}".format(alias.version.replace(".", "")), ) def _repr_actual(aliases): diff --git a/tests/pypi/config_settings/config_settings_tests.bzl b/tests/pypi/config_settings/config_settings_tests.bzl index 049556a4c6..f111d0c55c 100644 --- a/tests/pypi/config_settings/config_settings_tests.bzl +++ b/tests/pypi/config_settings/config_settings_tests.bzl @@ -72,24 +72,61 @@ def _match(env, target, want): _tests = [] +# Legacy pip config setting tests + +def _test_legacy_default(name): + _analysis_test( + name = name, + dist = { + "is_cp37": "legacy", + }, + want = "legacy", + ) + +_tests.append(_test_legacy_default) + +def _test_legacy_with_constraint_values(name): + _analysis_test( + name = name, + dist = { + "is_cp37": "legacy", + "is_cp37_linux_aarch64": "legacy_platform_override", + }, + want = "legacy_platform_override", + ) + +_tests.append(_test_legacy_with_constraint_values) + # Tests when we only have an `sdist` present. def _test_sdist_default(name): _analysis_test( name = name, dist = { - "is_cp3.7_sdist": "sdist", + "is_cp37_sdist": "sdist", }, want = "sdist", ) _tests.append(_test_sdist_default) +def _test_legacy_less_specialized_than_sdist(name): + _analysis_test( + name = name, + dist = { + "is_cp37": "legacy", + "is_cp37_sdist": "sdist", + }, + want = "sdist", + ) + +_tests.append(_test_legacy_less_specialized_than_sdist) + def _test_sdist_no_whl(name): _analysis_test( name = name, dist = { - "is_cp3.7_sdist": "sdist", + "is_cp37_sdist": "sdist", }, config_settings = [ _flag.platform("linux_aarch64"), @@ -104,7 +141,7 @@ def _test_sdist_no_sdist(name): _analysis_test( name = name, dist = { - "is_cp3.7_sdist": "sdist", + "is_cp37_sdist": "sdist", }, config_settings = [ _flag.platform("linux_aarch64"), @@ -121,8 +158,8 @@ def _test_basic_whl_default(name): _analysis_test( name = name, dist = { - "is_cp3.7_py_none_any": "whl", - "is_cp3.7_sdist": "sdist", + "is_cp37_py_none_any": "whl", + "is_cp37_sdist": "sdist", }, want = "whl", ) @@ -133,8 +170,8 @@ def _test_basic_whl_nowhl(name): _analysis_test( name = name, dist = { - "is_cp3.7_py_none_any": "whl", - "is_cp3.7_sdist": "sdist", + "is_cp37_py_none_any": "whl", + "is_cp37_sdist": "sdist", }, config_settings = [ _flag.platform("linux_aarch64"), @@ -149,8 +186,8 @@ def _test_basic_whl_nosdist(name): _analysis_test( name = name, dist = { - "is_cp3.7_py_none_any": "whl", - "is_cp3.7_sdist": "sdist", + "is_cp37_py_none_any": "whl", + "is_cp37_sdist": "sdist", }, config_settings = [ _flag.platform("linux_aarch64"), @@ -165,8 +202,8 @@ def _test_whl_default(name): _analysis_test( name = name, dist = { - "is_cp3.7_py3_none_any": "whl", - "is_cp3.7_py_none_any": "basic_whl", + "is_cp37_py3_none_any": "whl", + "is_cp37_py_none_any": "basic_whl", }, want = "whl", ) @@ -177,8 +214,8 @@ def _test_whl_nowhl(name): _analysis_test( name = name, dist = { - "is_cp3.7_py3_none_any": "whl", - "is_cp3.7_py_none_any": "basic_whl", + "is_cp37_py3_none_any": "whl", + "is_cp37_py_none_any": "basic_whl", }, config_settings = [ _flag.platform("linux_aarch64"), @@ -193,7 +230,7 @@ def _test_whl_nosdist(name): _analysis_test( name = name, dist = { - "is_cp3.7_py3_none_any": "whl", + "is_cp37_py3_none_any": "whl", }, config_settings = [ _flag.platform("linux_aarch64"), @@ -208,8 +245,8 @@ def _test_abi_whl_is_prefered(name): _analysis_test( name = name, dist = { - "is_cp3.7_py3_abi3_any": "abi_whl", - "is_cp3.7_py3_none_any": "whl", + "is_cp37_py3_abi3_any": "abi_whl", + "is_cp37_py3_none_any": "whl", }, want = "abi_whl", ) @@ -220,9 +257,9 @@ def _test_whl_with_constraints_is_prefered(name): _analysis_test( name = name, dist = { - "is_cp3.7_py3_none_any": "default_whl", - "is_cp3.7_py3_none_any_linux_aarch64": "whl", - "is_cp3.7_py3_none_any_linux_x86_64": "amd64_whl", + "is_cp37_py3_none_any": "default_whl", + "is_cp37_py3_none_any_linux_aarch64": "whl", + "is_cp37_py3_none_any_linux_x86_64": "amd64_whl", }, want = "whl", ) @@ -233,9 +270,9 @@ def _test_cp_whl_is_prefered_over_py3(name): _analysis_test( name = name, dist = { - "is_cp3.7_cp3x_none_any": "cp", - "is_cp3.7_py3_abi3_any": "py3_abi3", - "is_cp3.7_py3_none_any": "py3", + "is_cp37_none_any": "cp", + "is_cp37_py3_abi3_any": "py3_abi3", + "is_cp37_py3_none_any": "py3", }, want = "cp", ) @@ -246,8 +283,8 @@ def _test_cp_abi_whl_is_prefered_over_py3(name): _analysis_test( name = name, dist = { - "is_cp3.7_cp3x_abi3_any": "cp", - "is_cp3.7_py3_abi3_any": "py3", + "is_cp37_abi3_any": "cp", + "is_cp37_py3_abi3_any": "py3", }, want = "cp", ) @@ -258,9 +295,9 @@ def _test_cp_version_is_selected_when_python_version_is_specified(name): _analysis_test( name = name, dist = { - "is_cp3.10_cp3x_none_any": "cp310", - "is_cp3.8_cp3x_none_any": "cp38", - "is_cp3.9_cp3x_none_any": "cp39", + "is_cp310_none_any": "cp310", + "is_cp38_none_any": "cp38", + "is_cp39_none_any": "cp39", }, want = "cp310", config_settings = [ @@ -275,8 +312,8 @@ def _test_py_none_any_versioned(name): _analysis_test( name = name, dist = { - "is_cp3.10_py_none_any": "whl", - "is_cp3.9_py_none_any": "too-low", + "is_cp310_py_none_any": "whl", + "is_cp39_py_none_any": "too-low", }, want = "whl", config_settings = [ @@ -291,9 +328,9 @@ def _test_cp_whl_is_not_prefered_over_py3_non_freethreaded(name): _analysis_test( name = name, dist = { - "is_cp3.7_cp3x_abi3_any": "py3_abi3", - "is_cp3.7_cp3x_cpt_any": "cp", - "is_cp3.7_cp3x_none_any": "py3", + "is_cp37_abi3_any": "py3_abi3", + "is_cp37_cp37t_any": "cp", + "is_cp37_none_any": "py3", }, want = "py3_abi3", config_settings = [ @@ -307,9 +344,9 @@ def _test_cp_whl_is_not_prefered_over_py3_freethreaded(name): _analysis_test( name = name, dist = { - "is_cp3.7_cp3x_abi3_any": "py3_abi3", - "is_cp3.7_cp3x_cp_any": "cp", - "is_cp3.7_cp3x_none_any": "py3", + "is_cp37_abi3_any": "py3_abi3", + "is_cp37_cp37_any": "cp", + "is_cp37_none_any": "py3", }, want = "py3", config_settings = [ @@ -323,7 +360,7 @@ def _test_cp_cp_whl(name): _analysis_test( name = name, dist = { - "is_cp3.10_cp3x_cp_linux_aarch64": "whl", + "is_cp310_cp310_linux_aarch64": "whl", }, want = "whl", config_settings = [ @@ -338,7 +375,7 @@ def _test_cp_version_sdist_is_selected(name): _analysis_test( name = name, dist = { - "is_cp3.10_sdist": "sdist", + "is_cp310_sdist": "sdist", }, want = "sdist", config_settings = [ @@ -349,15 +386,52 @@ def _test_cp_version_sdist_is_selected(name): _tests.append(_test_cp_version_sdist_is_selected) +# NOTE: Right now there is no way to get the following behaviour without +# breaking other tests. We need to choose either ta have the correct +# specialization behaviour between `is_cp37_cp37_any` and +# `is_cp37_cp37_any_linux_aarch64` or this commented out test case. +# +# I think having this behaviour not working is fine because the `suffix` +# will be either present on all of config settings of the same platform +# or none, because we use it as a way to select a separate version of the +# wheel for a single platform only. +# +# If we can think of a better way to handle it, then we can lift this +# limitation. +# +# def _test_any_whl_with_suffix_specialization(name): +# _analysis_test( +# name = name, +# dist = { +# "is_cp37_abi3_any_linux_aarch64": "abi3", +# "is_cp37_cp37_any": "cp37", +# }, +# want = "cp37", +# ) +# +# _tests.append(_test_any_whl_with_suffix_specialization) + +def _test_platform_vs_any_with_suffix_specialization(name): + _analysis_test( + name = name, + dist = { + "is_cp37_cp37_any_linux_aarch64": "any", + "is_cp37_py3_none_linux_aarch64": "platform_whl", + }, + want = "platform_whl", + ) + +_tests.append(_test_platform_vs_any_with_suffix_specialization) + def _test_platform_whl_is_prefered_over_any_whl_with_constraints(name): _analysis_test( name = name, dist = { - "is_cp3.7_py3_abi3_any": "better_default_whl", - "is_cp3.7_py3_abi3_any_linux_aarch64": "better_default_any_whl", - "is_cp3.7_py3_none_any": "default_whl", - "is_cp3.7_py3_none_any_linux_aarch64": "whl", - "is_cp3.7_py3_none_linux_aarch64": "platform_whl", + "is_cp37_py3_abi3_any": "better_default_whl", + "is_cp37_py3_abi3_any_linux_aarch64": "better_default_any_whl", + "is_cp37_py3_none_any": "default_whl", + "is_cp37_py3_none_any_linux_aarch64": "whl", + "is_cp37_py3_none_linux_aarch64": "platform_whl", }, want = "platform_whl", ) @@ -368,8 +442,8 @@ def _test_abi3_platform_whl_preference(name): _analysis_test( name = name, dist = { - "is_cp3.7_py3_abi3_linux_aarch64": "abi3_platform", - "is_cp3.7_py3_none_linux_aarch64": "platform", + "is_cp37_py3_abi3_linux_aarch64": "abi3_platform", + "is_cp37_py3_none_linux_aarch64": "platform", }, want = "abi3_platform", ) @@ -380,8 +454,8 @@ def _test_glibc(name): _analysis_test( name = name, dist = { - "is_cp3.7_cp3x_cp_manylinux_aarch64": "glibc", - "is_cp3.7_py3_abi3_linux_aarch64": "abi3_platform", + "is_cp37_cp37_manylinux_aarch64": "glibc", + "is_cp37_py3_abi3_linux_aarch64": "abi3_platform", }, want = "glibc", ) @@ -392,9 +466,9 @@ def _test_glibc_versioned(name): _analysis_test( name = name, dist = { - "is_cp3.7_cp3x_cp_manylinux_2_14_aarch64": "glibc", - "is_cp3.7_cp3x_cp_manylinux_2_17_aarch64": "glibc", - "is_cp3.7_py3_abi3_linux_aarch64": "abi3_platform", + "is_cp37_cp37_manylinux_2_14_aarch64": "glibc", + "is_cp37_cp37_manylinux_2_17_aarch64": "glibc", + "is_cp37_py3_abi3_linux_aarch64": "abi3_platform", }, want = "glibc", config_settings = [ @@ -412,8 +486,8 @@ def _test_glibc_compatible_exists(name): dist = { # Code using the conditions will need to construct selects, which # do the version matching correctly. - "is_cp3.7_cp3x_cp_manylinux_2_14_aarch64": "2_14_whl_via_2_14_branch", - "is_cp3.7_cp3x_cp_manylinux_2_17_aarch64": "2_14_whl_via_2_17_branch", + "is_cp37_cp37_manylinux_2_14_aarch64": "2_14_whl_via_2_14_branch", + "is_cp37_cp37_manylinux_2_17_aarch64": "2_14_whl_via_2_17_branch", }, want = "2_14_whl_via_2_17_branch", config_settings = [ @@ -429,7 +503,7 @@ def _test_musl(name): _analysis_test( name = name, dist = { - "is_cp3.7_cp3x_cp_musllinux_aarch64": "musl", + "is_cp37_cp37_musllinux_aarch64": "musl", }, want = "musl", config_settings = [ @@ -444,8 +518,8 @@ def _test_windows(name): _analysis_test( name = name, dist = { - "is_cp3.7_cp3x_cp_windows_x86_64": "whl", - "is_cp3.7_cp3x_cpt_windows_x86_64": "whl_freethreaded", + "is_cp37_cp37_windows_x86_64": "whl", + "is_cp37_cp37t_windows_x86_64": "whl_freethreaded", }, want = "whl", config_settings = [ @@ -459,8 +533,8 @@ def _test_windows_freethreaded(name): _analysis_test( name = name, dist = { - "is_cp3.7_cp3x_cp_windows_x86_64": "whl", - "is_cp3.7_cp3x_cpt_windows_x86_64": "whl_freethreaded", + "is_cp37_cp37_windows_x86_64": "whl", + "is_cp37_cp37t_windows_x86_64": "whl_freethreaded", }, want = "whl_freethreaded", config_settings = [ @@ -476,8 +550,8 @@ def _test_osx(name): name = name, dist = { # We prefer arch specific whls over universal - "is_cp3.7_cp3x_cp_osx_x86_64": "whl", - "is_cp3.7_cp3x_cp_osx_x86_64_universal2": "universal_whl", + "is_cp37_cp37_osx_universal2": "universal_whl", + "is_cp37_cp37_osx_x86_64": "whl", }, want = "whl", config_settings = [ @@ -492,7 +566,7 @@ def _test_osx_universal_default(name): name = name, dist = { # We default to universal if only that exists - "is_cp3.7_cp3x_cp_osx_x86_64_universal2": "whl", + "is_cp37_cp37_osx_universal2": "whl", }, want = "whl", config_settings = [ @@ -507,8 +581,8 @@ def _test_osx_universal_only(name): name = name, dist = { # If we prefer universal, then we use that - "is_cp3.7_cp3x_cp_osx_x86_64": "whl", - "is_cp3.7_cp3x_cp_osx_x86_64_universal2": "universal", + "is_cp37_cp37_osx_universal2": "universal", + "is_cp37_cp37_osx_x86_64": "whl", }, want = "universal", config_settings = [ @@ -525,7 +599,7 @@ def _test_osx_os_version(name): dist = { # Similarly to the libc version, the user of the config settings will have to # construct the select so that the version selection is correct. - "is_cp3.7_cp3x_cp_osx_10_9_x86_64": "whl", + "is_cp37_cp37_osx_10_9_x86_64": "whl", }, want = "whl", config_settings = [ @@ -540,15 +614,15 @@ def _test_all(name): _analysis_test( name = name, dist = { - "is_cp3.7_" + f: f + "is_cp37_" + f: f for f in [ - "{py}_{abi}_{plat}".format(py = valid_py, abi = valid_abi, plat = valid_plat) - # we have py2.py3, py3, cp3x - for valid_py in ["py", "py3", "cp3x"] + "{py}{abi}_{plat}".format(py = valid_py, abi = valid_abi, plat = valid_plat) + # we have py2.py3, py3, cp3 + for valid_py in ["py_", "py3_", ""] # cp abi usually comes with a version and we only need one # config setting variant for all of them because the python # version will discriminate between different versions. - for valid_abi in ["none", "abi3", "cp"] + for valid_abi in ["none", "abi3", "cp37"] for valid_plat in [ "any", "manylinux_2_17_x86_64", @@ -557,12 +631,12 @@ def _test_all(name): "windows_x86_64", ] if not ( - valid_abi == "abi3" and valid_py == "py" or - valid_abi == "cp" and valid_py != "cp3x" + valid_abi == "abi3" and valid_py == "py_" or + valid_abi == "cp37" and valid_py != "" ) ] }, - want = "cp3x_cp_manylinux_2_17_x86_64", + want = "cp37_manylinux_2_17_x86_64", config_settings = [ _flag.pip_whl_glibc_version("2.17"), _flag.platform("linux_x86_64"), diff --git a/tests/pypi/pkg_aliases/pkg_aliases_test.bzl b/tests/pypi/pkg_aliases/pkg_aliases_test.bzl index f13b62f13d..71ca811fee 100644 --- a/tests/pypi/pkg_aliases/pkg_aliases_test.bzl +++ b/tests/pypi/pkg_aliases/pkg_aliases_test.bzl @@ -186,9 +186,9 @@ def _test_multiplatform_whl_aliases(env): want = { "pkg": { "//:my_config_setting": "@bzlmod_repo//:pkg", - "//_config:is_cp3.9_linux_x86_64": "@bzlmod_repo_for_a_particular_platform//:pkg", - "//_config:is_cp3.9_py3_none_any": "@filename_repo//:pkg", - "//_config:is_cp3.9_py3_none_any_linux_x86_64": "@filename_repo_for_platform//:pkg", + "//_config:is_cp39_linux_x86_64": "@bzlmod_repo_for_a_particular_platform//:pkg", + "//_config:is_cp39_py3_none_any": "@filename_repo//:pkg", + "//_config:is_cp39_py3_none_any_linux_x86_64": "@filename_repo_for_platform//:pkg", "//conditions:default": "_no_matching_repository", }, } @@ -197,9 +197,9 @@ def _test_multiplatform_whl_aliases(env): env.expect.that_str(actual_no_match_error[0]).contains("""\ configuration settings: //:my_config_setting - //_config:is_cp3.9_linux_x86_64 - //_config:is_cp3.9_py3_none_any - //_config:is_cp3.9_py3_none_any_linux_x86_64 + //_config:is_cp39_linux_x86_64 + //_config:is_cp39_py3_none_any + //_config:is_cp39_py3_none_any_linux_x86_64 """) @@ -286,8 +286,8 @@ def _test_multiplatform_whl_aliases_nofilename_target_platforms(env): got = multiplatform_whl_aliases(aliases = aliases) want = { - "//_config:is_cp3.1_linux_aarch64": "foo", - "//_config:is_cp3.1_linux_x86_64": "foo", + "//_config:is_cp31_linux_aarch64": "foo", + "//_config:is_cp31_linux_x86_64": "foo", } env.expect.that_dict(got).contains_exactly(want) @@ -305,11 +305,11 @@ def _test_multiplatform_whl_aliases_filename(env): ): "foo-py3-0.0.1", whl_config_setting( filename = "foo-0.0.1-cp313-cp313-any.whl", - version = "3.1", + version = "3.13", ): "foo-cp-0.0.1", whl_config_setting( filename = "foo-0.0.1-cp313-cp313t-any.whl", - version = "3.1", + version = "3.13", ): "foo-cpt-0.0.1", whl_config_setting( filename = "foo-0.0.2-py3-none-any.whl", @@ -327,12 +327,12 @@ def _test_multiplatform_whl_aliases_filename(env): osx_versions = [], ) want = { - "//_config:is_cp3.1_cp3x_cp_any": "foo-cp-0.0.1", - "//_config:is_cp3.1_cp3x_cpt_any": "foo-cpt-0.0.1", - "//_config:is_cp3.1_py3_none_any": "foo-py3-0.0.1", - "//_config:is_cp3.1_py3_none_any_linux_aarch64": "foo-0.0.2", - "//_config:is_cp3.1_py3_none_any_linux_x86_64": "foo-0.0.2", - "//_config:is_cp3.2_py3_none_any": "foo-py3-0.0.3", + "//_config:is_cp313_cp313_any": "foo-cp-0.0.1", + "//_config:is_cp313_cp313t_any": "foo-cpt-0.0.1", + "//_config:is_cp31_py3_none_any": "foo-py3-0.0.1", + "//_config:is_cp31_py3_none_any_linux_aarch64": "foo-0.0.2", + "//_config:is_cp31_py3_none_any_linux_x86_64": "foo-0.0.2", + "//_config:is_cp32_py3_none_any": "foo-py3-0.0.3", } env.expect.that_dict(got).contains_exactly(want) @@ -378,12 +378,12 @@ def _test_multiplatform_whl_aliases_filename_versioned(env): # For this to fully work we need to have the pypi:config_settings.bzl to generate the # extra targets that use the FeatureFlagInfo and this to generate extra aliases for the # config settings. - "//_config:is_cp3.1_py3_none_manylinux_2_17_x86_64": "glibc-2.17", - "//_config:is_cp3.1_py3_none_manylinux_2_18_x86_64": "glibc-2.18", - "//_config:is_cp3.1_py3_none_manylinux_x86_64": "glibc-2.17", - "//_config:is_cp3.1_py3_none_musllinux_1_1_x86_64": "musl-1.1", - "//_config:is_cp3.1_py3_none_musllinux_1_2_x86_64": "musl-1.1", - "//_config:is_cp3.1_py3_none_musllinux_x86_64": "musl-1.1", + "//_config:is_cp31_py3_none_manylinux_2_17_x86_64": "glibc-2.17", + "//_config:is_cp31_py3_none_manylinux_2_18_x86_64": "glibc-2.18", + "//_config:is_cp31_py3_none_manylinux_x86_64": "glibc-2.17", + "//_config:is_cp31_py3_none_musllinux_1_1_x86_64": "musl-1.1", + "//_config:is_cp31_py3_none_musllinux_1_2_x86_64": "musl-1.1", + "//_config:is_cp31_py3_none_musllinux_x86_64": "musl-1.1", } env.expect.that_dict(got).contains_exactly(want) diff --git a/tests/pypi/render_pkg_aliases/render_pkg_aliases_test.bzl b/tests/pypi/render_pkg_aliases/render_pkg_aliases_test.bzl index ca1651aa1d..c60761bed7 100644 --- a/tests/pypi/render_pkg_aliases/render_pkg_aliases_test.bzl +++ b/tests/pypi/render_pkg_aliases/render_pkg_aliases_test.bzl @@ -341,7 +341,7 @@ def _test_sdist(env): env, filename = "foo-0.0.1" + ext, python_version = "3.2", - want = [":is_cp3.2_sdist"], + want = [":is_cp32_sdist"], ) ext = ".zip" @@ -354,8 +354,8 @@ def _test_sdist(env): "linux_x86_64", ], want = [ - ":is_cp3.2_sdist_linux_aarch64", - ":is_cp3.2_sdist_linux_x86_64", + ":is_cp32_sdist_linux_aarch64", + ":is_cp32_sdist_linux_x86_64", ], ) @@ -367,7 +367,7 @@ def _test_py2_py3_none_any(env): filename = "foo-0.0.1-py2.py3-none-any.whl", python_version = "3.2", want = [ - ":is_cp3.2_py_none_any", + ":is_cp32_py_none_any", ], ) @@ -378,7 +378,7 @@ def _test_py2_py3_none_any(env): target_platforms = [ "osx_x86_64", ], - want = [":is_cp3.2_py_none_any_osx_x86_64"], + want = [":is_cp32_py_none_any_osx_x86_64"], ) _tests.append(_test_py2_py3_none_any) @@ -388,7 +388,7 @@ def _test_py3_none_any(env): env, filename = "foo-0.0.1-py3-none-any.whl", python_version = "3.1", - want = [":is_cp3.1_py3_none_any"], + want = [":is_cp31_py3_none_any"], ) _test_config_settings( @@ -396,7 +396,7 @@ def _test_py3_none_any(env): filename = "foo-0.0.1-py3-none-any.whl", python_version = "3.1", target_platforms = ["linux_x86_64"], - want = [":is_cp3.1_py3_none_any_linux_x86_64"], + want = [":is_cp31_py3_none_any_linux_x86_64"], ) _tests.append(_test_py3_none_any) @@ -412,13 +412,9 @@ def _test_py3_none_macosx_10_9_universal2(env): ], want = [], want_versions = { - ":is_cp3.1_py3_none_osx_aarch64_universal2": { - (10, 9): ":is_cp3.1_py3_none_osx_10_9_aarch64_universal2", - (11, 0): ":is_cp3.1_py3_none_osx_11_0_aarch64_universal2", - }, - ":is_cp3.1_py3_none_osx_x86_64_universal2": { - (10, 9): ":is_cp3.1_py3_none_osx_10_9_x86_64_universal2", - (11, 0): ":is_cp3.1_py3_none_osx_11_0_x86_64_universal2", + ":is_cp31_py3_none_osx_universal2": { + (10, 9): ":is_cp31_py3_none_osx_10_9_universal2", + (11, 0): ":is_cp31_py3_none_osx_11_0_universal2", }, }, ) @@ -430,7 +426,7 @@ def _test_cp37_abi3_linux_x86_64(env): env, filename = "foo-0.0.1-cp37-abi3-linux_x86_64.whl", python_version = "3.7", - want = [":is_cp3.7_cp3x_abi3_linux_x86_64"], + want = [":is_cp37_abi3_linux_x86_64"], ) _tests.append(_test_cp37_abi3_linux_x86_64) @@ -440,7 +436,7 @@ def _test_cp37_abi3_windows_x86_64(env): env, filename = "foo-0.0.1-cp37-abi3-windows_x86_64.whl", python_version = "3.7", - want = [":is_cp3.7_cp3x_abi3_windows_x86_64"], + want = [":is_cp37_abi3_windows_x86_64"], ) _tests.append(_test_cp37_abi3_windows_x86_64) @@ -457,9 +453,9 @@ def _test_cp37_abi3_manylinux_2_17_x86_64(env): ], want = [], want_versions = { - ":is_cp3.7_cp3x_abi3_manylinux_x86_64": { - (2, 17): ":is_cp3.7_cp3x_abi3_manylinux_2_17_x86_64", - (2, 18): ":is_cp3.7_cp3x_abi3_manylinux_2_18_x86_64", + ":is_cp37_abi3_manylinux_x86_64": { + (2, 17): ":is_cp37_abi3_manylinux_2_17_x86_64", + (2, 18): ":is_cp37_abi3_manylinux_2_18_x86_64", }, }, ) @@ -482,12 +478,12 @@ def _test_cp37_abi3_manylinux_2_17_musllinux_1_1_aarch64(env): ], want = [], want_versions = { - ":is_cp3.7_cp3x_cp_manylinux_aarch64": { - (2, 17): ":is_cp3.7_cp3x_cp_manylinux_2_17_aarch64", - (2, 18): ":is_cp3.7_cp3x_cp_manylinux_2_18_aarch64", + ":is_cp37_cp37_manylinux_aarch64": { + (2, 17): ":is_cp37_cp37_manylinux_2_17_aarch64", + (2, 18): ":is_cp37_cp37_manylinux_2_18_aarch64", }, - ":is_cp3.7_cp3x_cp_musllinux_aarch64": { - (1, 1): ":is_cp3.7_cp3x_cp_musllinux_1_1_aarch64", + ":is_cp37_cp37_musllinux_aarch64": { + (1, 1): ":is_cp37_cp37_musllinux_1_1_aarch64", }, }, ) From 1aa0d9f63ed7b98c65a81c4e78ebef2a258ee673 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Sun, 12 Jan 2025 17:18:30 -0800 Subject: [PATCH 056/922] fix(pypi): include pyi files in data attribute (#2558) Restore the previous behavior of pyi files being included in data. This is because certain packages (librosa, at least) expect the pyi files to be available at runtime. --- CHANGELOG.md | 2 -- python/private/pypi/whl_library_targets.bzl | 4 +++- tests/pypi/whl_library_targets/whl_library_targets_tests.bzl | 2 -- 3 files changed, 3 insertions(+), 5 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 7f4c60b4b3..3c71f7e860 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -88,8 +88,6 @@ Unreleased changes template. `python_version` attribute is still used to specify the Python version. * (pypi) Updated versions of packages: `pip` to 24.3.1 and `packaging` to 24.2. -* (pypi) For pypi-generated targets, `*.pyi` files are included in the - `pyi_srcs` attribute instead of the `data` attribute. {#v1-1-0-deprecations} #### Deprecations diff --git a/python/private/pypi/whl_library_targets.bzl b/python/private/pypi/whl_library_targets.bzl index 461a75cac3..c390da2613 100644 --- a/python/private/pypi/whl_library_targets.bzl +++ b/python/private/pypi/whl_library_targets.bzl @@ -222,11 +222,13 @@ def whl_library_targets( ) if hasattr(rules, "py_library"): + # NOTE: pyi files should probably be excluded because they're carried + # by the pyi_srcs attribute. However, historical behavior included + # them in data and some tools currently rely on that. _data_exclude = [ "**/*.py", "**/*.pyc", "**/*.pyc.*", # During pyc creation, temp files named *.pyc.NNNN are created - "**/*.pyi", # RECORD is known to contain sha256 checksums of files which might include the checksums # of generated files produced when wheels are installed. The file is ignored to avoid # Bazel caching issues. diff --git a/tests/pypi/whl_library_targets/whl_library_targets_tests.bzl b/tests/pypi/whl_library_targets/whl_library_targets_tests.bzl index 5d10cf0a5a..ba04e1d887 100644 --- a/tests/pypi/whl_library_targets/whl_library_targets_tests.bzl +++ b/tests/pypi/whl_library_targets/whl_library_targets_tests.bzl @@ -252,7 +252,6 @@ def _test_whl_and_library_deps(env): "**/*.py", "**/*.pyc", "**/*.pyc.*", - "**/*.pyi", "**/*.dist-info/RECORD", ] + glob_excludes.version_dependent_exclusions(), ), @@ -325,7 +324,6 @@ def _test_group(env): "**/*.py", "**/*.pyc", "**/*.pyc.*", - "**/*.pyi", "**/*.dist-info/RECORD", ] + glob_excludes.version_dependent_exclusions(), ), From c0bd668136e57852ced9cc87127ee38c20426d37 Mon Sep 17 00:00:00 2001 From: Nicholas Junge Date: Tue, 14 Jan 2025 21:02:20 +0100 Subject: [PATCH 057/922] docs: Add horizontal spacers around filenames in custom toolchain guide (#2563) This improves readability by clearly marking the beginnings and ends of the three involved source files. Follow-up of #2512, as discussed. --- docs/toolchains.md | 11 +++++++++-- 1 file changed, 9 insertions(+), 2 deletions(-) diff --git a/docs/toolchains.md b/docs/toolchains.md index 2a59169752..32f4a541d9 100644 --- a/docs/toolchains.md +++ b/docs/toolchains.md @@ -411,7 +411,10 @@ Here, we show an example for a semi-complicated toolchain suite, one that is: Defining toolchains for this might look something like this: ``` +# ------------------------------------------------------- # File: toolchain_impl/BUILD +# Contains the tool definitions (runtime, headers, libs). +# ------------------------------------------------------- load("@rules_python//python:py_cc_toolchain.bzl", "py_cc_toolchain") load("@rules_python//python:py_exec_tools_toolchain.bzl", "py_exec_tools_toolchain") load("@rules_python//python:py_runtime.bzl", "py_runtime") @@ -453,9 +456,11 @@ cc_binary(name = "python3.12", ...) cc_library(name = "headers", ...) cc_library(name = "libs", ...) +# ------------------------------------------------------------------ # File: toolchains/BUILD # Putting toolchain() calls in a separate package from the toolchain -# implementations minimizes Bazel loading overhead +# implementations minimizes Bazel loading overhead. +# ------------------------------------------------------------------ toolchain( name = "runtime_toolchain", @@ -480,8 +485,10 @@ toolchain( exec_comaptible_with = ["@platforms/os:linux"] ) +# ----------------------------------------------- # File: MODULE.bazel or WORKSPACE.bazel -# These toolchains will considered before others +# These toolchains will considered before others. +# ----------------------------------------------- register_toolchains("//toolchains:all") ``` From eef839ba9a5edbda273c8d35d6bee4256fcd2249 Mon Sep 17 00:00:00 2001 From: Will Morrison Date: Wed, 15 Jan 2025 00:04:25 +0100 Subject: [PATCH 058/922] fix: Avoid creating URLs with empty path segments from index URLs in environment variables (#2557) This change updates `_read_simpleapi` such that it correctly handles the case where the index URL is specified in an environment variable and contains a trailing slash. The URL construction would have introduced an empty path segment, which is now removed. Fixes: #2554 --------- Co-authored-by: Ignas Anikevicius <240938+aignas@users.noreply.github.com> --- CHANGELOG.md | 2 + python/private/pypi/simpleapi_download.bzl | 35 ++++- .../simpleapi_download_tests.bzl | 123 +++++++++++++++++- 3 files changed, 153 insertions(+), 7 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 3c71f7e860..3ea933986f 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -57,6 +57,8 @@ Unreleased changes template. {#v0-0-0-fixed} ### Fixed * (gazelle) Providing multiple input requirements files to `gazelle_python_manifest` now works correctly. +* (pypi) Handle trailing slashes in pip index URLs in environment variables, + fixes [#2554](https://github.com/bazelbuild/rules_python/issues/2554). {#v0-0-0-added} ### Added diff --git a/python/private/pypi/simpleapi_download.bzl b/python/private/pypi/simpleapi_download.bzl index 6401a066c2..ef39fb8723 100644 --- a/python/private/pypi/simpleapi_download.bzl +++ b/python/private/pypi/simpleapi_download.bzl @@ -17,7 +17,7 @@ A file that houses private functions used in the `bzlmod` extension with the sam """ load("@bazel_features//:features.bzl", "bazel_features") -load("//python/private:auth.bzl", "get_auth") +load("//python/private:auth.bzl", _get_auth = "get_auth") load("//python/private:envsubst.bzl", "envsubst") load("//python/private:normalize_name.bzl", "normalize_name") load("//python/private:text_util.bzl", "render") @@ -30,6 +30,7 @@ def simpleapi_download( cache, parallel_download = True, read_simpleapi = None, + get_auth = None, _fail = fail): """Download Simple API HTML. @@ -59,6 +60,7 @@ def simpleapi_download( parallel_download: A boolean to enable usage of bazel 7.1 non-blocking downloads. read_simpleapi: a function for reading and parsing of the SimpleAPI contents. Used in tests. + get_auth: A function to get auth information passed to read_simpleapi. Used in tests. _fail: a function to print a failure. Used in tests. Returns: @@ -98,6 +100,7 @@ def simpleapi_download( ), attr = attr, cache = cache, + get_auth = get_auth, **download_kwargs ) if hasattr(result, "wait"): @@ -144,7 +147,7 @@ def simpleapi_download( return contents -def _read_simpleapi(ctx, url, attr, cache, **download_kwargs): +def _read_simpleapi(ctx, url, attr, cache, get_auth = None, **download_kwargs): """Read SimpleAPI. Args: @@ -157,6 +160,7 @@ def _read_simpleapi(ctx, url, attr, cache, **download_kwargs): * auth_patterns: The auth_patterns parameter for ctx.download, see http_file for docs. cache: A dict for storing the results. + get_auth: A function to get auth information. Used in tests. **download_kwargs: Any extra params to ctx.download. Note that output and auth will be passed for you. @@ -169,11 +173,11 @@ def _read_simpleapi(ctx, url, attr, cache, **download_kwargs): # them to ctx.download if we want to correctly handle the relative URLs. # TODO: Add a test that env subbed index urls do not leak into the lock file. - real_url = envsubst( + real_url = strip_empty_path_segments(envsubst( url, attr.envsubst, ctx.getenv if hasattr(ctx, "getenv") else ctx.os.environ.get, - ) + )) cache_key = real_url if cache_key in cache: @@ -194,6 +198,8 @@ def _read_simpleapi(ctx, url, attr, cache, **download_kwargs): output = ctx.path(output_str.strip("_").lower() + ".html") + get_auth = get_auth or _get_auth + # NOTE: this may have block = True or block = False in the download_kwargs download = ctx.download( url = [real_url], @@ -211,6 +217,27 @@ def _read_simpleapi(ctx, url, attr, cache, **download_kwargs): return _read_index_result(ctx, download, output, real_url, cache, cache_key) +def strip_empty_path_segments(url): + """Removes empty path segments from a URL. Does nothing for urls with no scheme. + + Public only for testing. + + Args: + url: The url to remove empty path segments from + + Returns: + The url with empty path segments removed and any trailing slash preserved. + If the url had no scheme it is returned unchanged. + """ + scheme, _, rest = url.partition("://") + if rest == "": + return url + stripped = "/".join([p for p in rest.split("/") if p]) + if url.endswith("/"): + return "{}://{}/".format(scheme, stripped) + else: + return "{}://{}".format(scheme, stripped) + def _read_index_result(ctx, result, output, url, cache, cache_key): if not result.success: return struct(success = False) diff --git a/tests/pypi/simpleapi_download/simpleapi_download_tests.bzl b/tests/pypi/simpleapi_download/simpleapi_download_tests.bzl index 9b2967b0da..964d3e25ea 100644 --- a/tests/pypi/simpleapi_download/simpleapi_download_tests.bzl +++ b/tests/pypi/simpleapi_download/simpleapi_download_tests.bzl @@ -15,17 +15,18 @@ "" load("@rules_testing//lib:test_suite.bzl", "test_suite") -load("//python/private/pypi:simpleapi_download.bzl", "simpleapi_download") # buildifier: disable=bzl-visibility +load("//python/private/pypi:simpleapi_download.bzl", "simpleapi_download", "strip_empty_path_segments") # buildifier: disable=bzl-visibility _tests = [] def _test_simple(env): calls = [] - def read_simpleapi(ctx, url, attr, cache, block): + def read_simpleapi(ctx, url, attr, cache, get_auth, block): _ = ctx # buildifier: disable=unused-variable _ = attr _ = cache + _ = get_auth env.expect.that_bool(block).equals(False) calls.append(url) if "foo" in url and "main" in url: @@ -73,10 +74,11 @@ def _test_fail(env): calls = [] fails = [] - def read_simpleapi(ctx, url, attr, cache, block): + def read_simpleapi(ctx, url, attr, cache, get_auth, block): _ = ctx # buildifier: disable=unused-variable _ = attr _ = cache + _ = get_auth env.expect.that_bool(block).equals(False) calls.append(url) if "foo" in url: @@ -119,6 +121,121 @@ def _test_fail(env): _tests.append(_test_fail) +def _test_download_url(env): + downloads = {} + + def download(url, output, **kwargs): + _ = kwargs # buildifier: disable=unused-variable + downloads[url[0]] = output + return struct(success = True) + + simpleapi_download( + ctx = struct( + os = struct(environ = {}), + download = download, + read = lambda i: "contents of " + i, + path = lambda i: "path/for/" + i, + ), + attr = struct( + index_url_overrides = {}, + index_url = "https://example.com/main/simple/", + extra_index_urls = [], + sources = ["foo", "bar", "baz"], + envsubst = [], + ), + cache = {}, + parallel_download = False, + get_auth = lambda ctx, urls, ctx_attr: struct(), + ) + + env.expect.that_dict(downloads).contains_exactly({ + "https://example.com/main/simple/bar/": "path/for/https___example_com_main_simple_bar.html", + "https://example.com/main/simple/baz/": "path/for/https___example_com_main_simple_baz.html", + "https://example.com/main/simple/foo/": "path/for/https___example_com_main_simple_foo.html", + }) + +_tests.append(_test_download_url) + +def _test_download_url_parallel(env): + downloads = {} + + def download(url, output, **kwargs): + _ = kwargs # buildifier: disable=unused-variable + downloads[url[0]] = output + return struct(wait = lambda: struct(success = True)) + + simpleapi_download( + ctx = struct( + os = struct(environ = {}), + download = download, + read = lambda i: "contents of " + i, + path = lambda i: "path/for/" + i, + ), + attr = struct( + index_url_overrides = {}, + index_url = "https://example.com/main/simple/", + extra_index_urls = [], + sources = ["foo", "bar", "baz"], + envsubst = [], + ), + cache = {}, + parallel_download = True, + get_auth = lambda ctx, urls, ctx_attr: struct(), + ) + + env.expect.that_dict(downloads).contains_exactly({ + "https://example.com/main/simple/bar/": "path/for/https___example_com_main_simple_bar.html", + "https://example.com/main/simple/baz/": "path/for/https___example_com_main_simple_baz.html", + "https://example.com/main/simple/foo/": "path/for/https___example_com_main_simple_foo.html", + }) + +_tests.append(_test_download_url_parallel) + +def _test_download_envsubst_url(env): + downloads = {} + + def download(url, output, **kwargs): + _ = kwargs # buildifier: disable=unused-variable + downloads[url[0]] = output + return struct(success = True) + + simpleapi_download( + ctx = struct( + os = struct(environ = {"INDEX_URL": "https://example.com/main/simple/"}), + download = download, + read = lambda i: "contents of " + i, + path = lambda i: "path/for/" + i, + ), + attr = struct( + index_url_overrides = {}, + index_url = "$INDEX_URL", + extra_index_urls = [], + sources = ["foo", "bar", "baz"], + envsubst = ["INDEX_URL"], + ), + cache = {}, + parallel_download = False, + get_auth = lambda ctx, urls, ctx_attr: struct(), + ) + + env.expect.that_dict(downloads).contains_exactly({ + "https://example.com/main/simple/bar/": "path/for/~index_url~_bar.html", + "https://example.com/main/simple/baz/": "path/for/~index_url~_baz.html", + "https://example.com/main/simple/foo/": "path/for/~index_url~_foo.html", + }) + +_tests.append(_test_download_envsubst_url) + +def _test_strip_empty_path_segments(env): + env.expect.that_str(strip_empty_path_segments("no/scheme//is/unchanged")).equals("no/scheme//is/unchanged") + env.expect.that_str(strip_empty_path_segments("scheme://with/no/empty/segments")).equals("scheme://with/no/empty/segments") + env.expect.that_str(strip_empty_path_segments("scheme://with//empty/segments")).equals("scheme://with/empty/segments") + env.expect.that_str(strip_empty_path_segments("scheme://with///multiple//empty/segments")).equals("scheme://with/multiple/empty/segments") + env.expect.that_str(strip_empty_path_segments("scheme://with//trailing/slash/")).equals("scheme://with/trailing/slash/") + env.expect.that_str(strip_empty_path_segments("scheme://with/trailing/slashes///")).equals("scheme://with/trailing/slashes/") + +_tests.append(_test_strip_empty_path_segments) + def simpleapi_download_test_suite(name): """Create the test suite. From f21911211de2d25bc744f6a1a1b5db6372379426 Mon Sep 17 00:00:00 2001 From: Alex Eagle Date: Fri, 17 Jan 2025 09:54:51 -0800 Subject: [PATCH 059/922] docs: update gazelle README.md (#2567) There's no longer a python subprocess since switching to tree-sitter in https://github.com/bazelbuild/rules_python/pull/1895 --- gazelle/README.md | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/gazelle/README.md b/gazelle/README.md index 55c9cc9bff..01cf45a938 100644 --- a/gazelle/README.md +++ b/gazelle/README.md @@ -654,8 +654,7 @@ code into a separate script without a `main` line. Gazelle will then create a ## Developer Notes -Gazelle extensions are written in Go. This gazelle plugin is a hybrid, as it uses Go to execute a -Python interpreter as a subprocess to parse Python source files. +Gazelle extensions are written in Go. See the gazelle documentation https://github.com/bazelbuild/bazel-gazelle/blob/master/extend.md for more information on extending Gazelle. From 50a9a2e59d98c56a26b1b9230609d16723037c46 Mon Sep 17 00:00:00 2001 From: mareld <70335127+mailto-jonas@users.noreply.github.com> Date: Tue, 21 Jan 2025 03:01:41 +0100 Subject: [PATCH 060/922] fix: Don't fail in override from a non-root module (#2566) This patch enable calls to pypi override from a non-root module without failing. The call will instead be silently ignored. Fixes #2550 --------- Co-authored-by: Ignas Anikevicius <240938+aignas@users.noreply.github.com> --- CHANGELOG.md | 3 ++- python/private/pypi/extension.bzl | 4 +++- 2 files changed, 5 insertions(+), 2 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 3ea933986f..2d8c1631ed 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -52,7 +52,8 @@ Unreleased changes template. {#v0-0-0-changed} ### Changed -* Nothing changed. +* (pypi) {obj}`pip.override` will now be ignored instead of raising an error, + fixes [#2550](https://github.com/bazelbuild/rules_python/issues/2550). {#v0-0-0-fixed} ### Fixed diff --git a/python/private/pypi/extension.bzl b/python/private/pypi/extension.bzl index 6409bccdd6..405c22f60e 100644 --- a/python/private/pypi/extension.bzl +++ b/python/private/pypi/extension.bzl @@ -387,7 +387,9 @@ You cannot use both the additive_build_content and additive_build_content_file a for module in module_ctx.modules: for attr in module.tags.override: if not module.is_root: - fail("overrides are only supported in root modules") + # Overrides are only supported in root modules. Silently + # ignore the override: + continue if not attr.file.endswith(".whl"): fail("Only whl overrides are supported at this time") From 188598ab8348ac1d84e417a66ebb8501506041e6 Mon Sep 17 00:00:00 2001 From: Ignas Anikevicius <240938+aignas@users.noreply.github.com> Date: Wed, 22 Jan 2025 11:31:12 +0900 Subject: [PATCH 061/922] chore: remove internal usage of deprecated py_binary ad py_test (#2569) This goes together with #2565 to remove the internal usage of the deprecated symbols. This also fixes the compile_pip_requirements symbol to print the correct deprecation message. Builds on top of 611eda8 The example message that would be printed is as follows: ``` The 'py_test' symbol in '@+python+python_3_11//:defs.bzl' is deprecated. It is an alias to the regular rule; use it directly instead: load("@rules_python//python:py_test.bzl", "py_test") py_test( name = "versioned_py_test", srcs = ["dummy.py"], main = "dummy.py", python_version = "3.11.11", ) ``` --------- Co-authored-by: Richard Levasseur --- CHANGELOG.md | 3 + MODULE.bazel | 7 +- WORKSPACE | 4 +- docs/environment-variables.md | 6 ++ examples/bzlmod/other_module/BUILD.bazel | 5 +- .../other_module/other_module/pkg/BUILD.bazel | 8 +- examples/bzlmod/tests/BUILD.bazel | 39 ++++---- .../requirements/BUILD.bazel | 17 ++-- .../multi_python_versions/tests/BUILD.bazel | 43 +++++---- python/config_settings/transition.bzl | 30 +++--- python/private/BUILD.bazel | 8 ++ python/private/deprecation.bzl | 59 ++++++++++++ python/private/internal_config_repo.bzl | 8 +- python/private/toolchains_repo.bzl | 90 +++++++++-------- python/uv/private/lock.bzl | 3 +- .../transition/multi_version_tests.bzl | 9 +- tests/deprecated/BUILD.bazel | 96 +++++++++++++++++++ tests/deprecated/dummy.py | 0 tests/deprecated/requirements.in | 0 tests/deprecated/requirements.txt | 6 ++ tests/deprecated/requirements_hub.txt | 6 ++ tools/publish/BUILD.bazel | 8 +- 22 files changed, 336 insertions(+), 119 deletions(-) create mode 100644 python/private/deprecation.bzl create mode 100644 tests/deprecated/BUILD.bazel create mode 100644 tests/deprecated/dummy.py create mode 100644 tests/deprecated/requirements.in create mode 100644 tests/deprecated/requirements.txt create mode 100644 tests/deprecated/requirements_hub.txt diff --git a/CHANGELOG.md b/CHANGELOG.md index 2d8c1631ed..00624db01f 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -54,6 +54,9 @@ Unreleased changes template. ### Changed * (pypi) {obj}`pip.override` will now be ignored instead of raising an error, fixes [#2550](https://github.com/bazelbuild/rules_python/issues/2550). +* (rules) deprecation warnings for deprecated symbols have been turned off by + default for now and can be enabled with `RULES_PYTHON_DEPRECATION_WARNINGS` + env var. {#v0-0-0-fixed} ### Fixed diff --git a/MODULE.bazel b/MODULE.bazel index 57780b2369..2ac5a27223 100644 --- a/MODULE.bazel +++ b/MODULE.bazel @@ -46,7 +46,12 @@ python.toolchain( is_default = True, python_version = "3.11", ) -use_repo(python, "python_3_11", "python_versions", "pythons_hub") +use_repo( + python, + "python_3_11", + "pythons_hub", + python = "python_versions", +) # This call registers the Python toolchains. register_toolchains("@pythons_hub//:all") diff --git a/WORKSPACE b/WORKSPACE index 7303b480f2..902af58ec8 100644 --- a/WORKSPACE +++ b/WORKSPACE @@ -68,12 +68,12 @@ load("//:internal_dev_setup.bzl", "rules_python_internal_setup") rules_python_internal_setup() -load("@pythons_hub//:versions.bzl", "MINOR_MAPPING", "PYTHON_VERSIONS") +load("@pythons_hub//:versions.bzl", "PYTHON_VERSIONS") load("//python:repositories.bzl", "python_register_multi_toolchains") python_register_multi_toolchains( name = "python", - default_version = MINOR_MAPPING.values()[-3], # Use 3.11.10 + default_version = "3.11", # Integration tests verify each version, so register all of them. python_versions = PYTHON_VERSIONS, ) diff --git a/docs/environment-variables.md b/docs/environment-variables.md index 12c1bcf0c2..fb9971597b 100644 --- a/docs/environment-variables.md +++ b/docs/environment-variables.md @@ -40,6 +40,12 @@ When `1`, bzlmod extensions will print debug information about what they're doing. This is mostly useful for development to debug errors. ::: +:::{envvar} RULES_PYTHON_DEPRECATION_WARNINGS + +When `1`, the rules_python will warn users about deprecated functionality that will +be removed in a subsequent major `rules_python` version. Defaults to `0` if unset. +::: + :::{envvar} RULES_PYTHON_ENABLE_PYSTAR When `1`, the rules_python Starlark implementation of the core rules is used diff --git a/examples/bzlmod/other_module/BUILD.bazel b/examples/bzlmod/other_module/BUILD.bazel index a93b92aaed..6294c5b0ae 100644 --- a/examples/bzlmod/other_module/BUILD.bazel +++ b/examples/bzlmod/other_module/BUILD.bazel @@ -1,9 +1,10 @@ -load("@python_versions//3.11:defs.bzl", compile_pip_requirements_311 = "compile_pip_requirements") +load("@rules_python//python:pip.bzl", "compile_pip_requirements") # NOTE: To update the requirements, you need to uncomment the rules_python # override in the MODULE.bazel. -compile_pip_requirements_311( +compile_pip_requirements( name = "requirements", src = "requirements.in", + python_version = "3.11", requirements_txt = "requirements_lock_3_11.txt", ) diff --git a/examples/bzlmod/other_module/other_module/pkg/BUILD.bazel b/examples/bzlmod/other_module/other_module/pkg/BUILD.bazel index 4fe392841e..53344c708a 100644 --- a/examples/bzlmod/other_module/other_module/pkg/BUILD.bazel +++ b/examples/bzlmod/other_module/other_module/pkg/BUILD.bazel @@ -1,7 +1,4 @@ -load( - "@python_3_11//:defs.bzl", - py_binary_311 = "py_binary", -) +load("@rules_python//python:py_binary.bzl", "py_binary") load("@rules_python//python:py_library.bzl", "py_library") py_library( @@ -15,11 +12,12 @@ py_library( # This is used for testing mulitple versions of Python. This is # used only when you need to support multiple versions of Python # in the same project. -py_binary_311( +py_binary( name = "bin", srcs = ["bin.py"], data = ["data/data.txt"], main = "bin.py", + python_version = "3.11", visibility = ["//visibility:public"], deps = [ ":lib", diff --git a/examples/bzlmod/tests/BUILD.bazel b/examples/bzlmod/tests/BUILD.bazel index dd50cf3294..4650fb8788 100644 --- a/examples/bzlmod/tests/BUILD.bazel +++ b/examples/bzlmod/tests/BUILD.bazel @@ -1,10 +1,6 @@ -load("@python_versions//3.10:defs.bzl", py_binary_3_10 = "py_binary", py_test_3_10 = "py_test") -load("@python_versions//3.11:defs.bzl", py_binary_3_11 = "py_binary", py_test_3_11 = "py_test") -load("@python_versions//3.9:defs.bzl", py_binary_3_9 = "py_binary", py_test_3_9 = "py_test") load("@pythons_hub//:versions.bzl", "MINOR_MAPPING") load("@rules_python//python:py_binary.bzl", "py_binary") load("@rules_python//python:py_test.bzl", "py_test") -load("@rules_python//python/config_settings:transition.bzl", py_versioned_binary = "py_binary", py_versioned_test = "py_test") load("@rules_shell//shell:sh_test.bzl", "sh_test") py_binary( @@ -13,25 +9,28 @@ py_binary( main = "version.py", ) -py_binary_3_9( +py_binary( name = "version_3_9", srcs = ["version.py"], main = "version.py", + python_version = "3.9", ) -py_binary_3_10( +py_binary( name = "version_3_10", srcs = ["version.py"], main = "version.py", + python_version = "3.10", ) -py_binary_3_11( +py_binary( name = "version_3_11", srcs = ["version.py"], main = "version.py", + python_version = "3.11", ) -py_versioned_binary( +py_binary( name = "version_3_10_versioned", srcs = ["version.py"], main = "version.py", @@ -49,21 +48,23 @@ py_test( deps = ["//libs/my_lib"], ) -py_test_3_9( +py_test( name = "my_lib_3_9_test", srcs = ["my_lib_test.py"], main = "my_lib_test.py", + python_version = "3.9", deps = ["//libs/my_lib"], ) -py_test_3_10( +py_test( name = "my_lib_3_10_test", srcs = ["my_lib_test.py"], main = "my_lib_test.py", + python_version = "3.10", deps = ["//libs/my_lib"], ) -py_versioned_test( +py_test( name = "my_lib_versioned_test", srcs = ["my_lib_test.py"], main = "my_lib_test.py", @@ -92,21 +93,23 @@ py_test( main = "version_test.py", ) -py_test_3_9( +py_test( name = "version_3_9_test", srcs = ["version_test.py"], env = {"VERSION_CHECK": "3.9"}, main = "version_test.py", + python_version = "3.9", ) -py_test_3_10( +py_test( name = "version_3_10_test", srcs = ["version_test.py"], env = {"VERSION_CHECK": "3.10"}, main = "version_test.py", + python_version = "3.10", ) -py_versioned_test( +py_test( name = "version_versioned_test", srcs = ["version_test.py"], env = {"VERSION_CHECK": "3.10"}, @@ -114,11 +117,12 @@ py_versioned_test( python_version = "3.10", ) -py_test_3_11( +py_test( name = "version_3_11_test", srcs = ["version_test.py"], env = {"VERSION_CHECK": "3.11"}, main = "version_test.py", + python_version = "3.11", ) py_test( @@ -133,7 +137,7 @@ py_test( main = "cross_version_test.py", ) -py_test_3_10( +py_test( name = "version_3_10_takes_3_9_subprocess_test", srcs = ["cross_version_test.py"], data = [":version_3_9"], @@ -143,9 +147,10 @@ py_test_3_10( "VERSION_CHECK": "3.10", }, main = "cross_version_test.py", + python_version = "3.10", ) -py_versioned_test( +py_test( name = "version_3_10_takes_3_9_subprocess_test_2", srcs = ["cross_version_test.py"], data = [":version_3_9"], diff --git a/examples/multi_python_versions/requirements/BUILD.bazel b/examples/multi_python_versions/requirements/BUILD.bazel index f67333a657..c9b695e8e4 100644 --- a/examples/multi_python_versions/requirements/BUILD.bazel +++ b/examples/multi_python_versions/requirements/BUILD.bazel @@ -1,28 +1,29 @@ -load("@python//3.10:defs.bzl", compile_pip_requirements_3_10 = "compile_pip_requirements") -load("@python//3.11:defs.bzl", compile_pip_requirements_3_11 = "compile_pip_requirements") -load("@python//3.8:defs.bzl", compile_pip_requirements_3_8 = "compile_pip_requirements") -load("@python//3.9:defs.bzl", compile_pip_requirements_3_9 = "compile_pip_requirements") +load("@rules_python//python:pip.bzl", "compile_pip_requirements") -compile_pip_requirements_3_8( +compile_pip_requirements( name = "requirements_3_8", src = "requirements.in", + python_version = "3.8", requirements_txt = "requirements_lock_3_8.txt", ) -compile_pip_requirements_3_9( +compile_pip_requirements( name = "requirements_3_9", src = "requirements.in", + python_version = "3.9", requirements_txt = "requirements_lock_3_9.txt", ) -compile_pip_requirements_3_10( +compile_pip_requirements( name = "requirements_3_10", src = "requirements.in", + python_version = "3.10", requirements_txt = "requirements_lock_3_10.txt", ) -compile_pip_requirements_3_11( +compile_pip_requirements( name = "requirements_3_11", src = "requirements.in", + python_version = "3.11", requirements_txt = "requirements_lock_3_11.txt", ) diff --git a/examples/multi_python_versions/tests/BUILD.bazel b/examples/multi_python_versions/tests/BUILD.bazel index d04ac6bb0a..e3dfb48cca 100644 --- a/examples/multi_python_versions/tests/BUILD.bazel +++ b/examples/multi_python_versions/tests/BUILD.bazel @@ -1,10 +1,6 @@ load("@bazel_skylib//rules:copy_file.bzl", "copy_file") load("@bazel_skylib//rules:diff_test.bzl", "diff_test") load("@bazel_skylib//rules:write_file.bzl", "write_file") -load("@python//3.10:defs.bzl", py_binary_3_10 = "py_binary", py_test_3_10 = "py_test") -load("@python//3.11:defs.bzl", py_binary_3_11 = "py_binary", py_test_3_11 = "py_test") -load("@python//3.8:defs.bzl", py_binary_3_8 = "py_binary", py_test_3_8 = "py_test") -load("@python//3.9:defs.bzl", py_binary_3_9 = "py_binary", py_test_3_9 = "py_test") load("@pythons_hub//:versions.bzl", "MINOR_MAPPING", "PYTHON_VERSIONS") load("@rules_python//python:py_binary.bzl", "py_binary") load("@rules_python//python:py_test.bzl", "py_test") @@ -26,28 +22,32 @@ py_binary( srcs = ["version_default.py"], ) -py_binary_3_8( +py_binary( name = "version_3_8", srcs = ["version.py"], main = "version.py", + python_version = "3.8", ) -py_binary_3_9( +py_binary( name = "version_3_9", srcs = ["version.py"], main = "version.py", + python_version = "3.9", ) -py_binary_3_10( +py_binary( name = "version_3_10", srcs = ["version.py"], main = "version.py", + python_version = "3.10", ) -py_binary_3_11( +py_binary( name = "version_3_11", srcs = ["version.py"], main = "version.py", + python_version = "3.11", ) py_test( @@ -57,31 +57,35 @@ py_test( deps = ["//libs/my_lib"], ) -py_test_3_8( +py_test( name = "my_lib_3_8_test", srcs = ["my_lib_test.py"], main = "my_lib_test.py", + python_version = "3.8", deps = ["//libs/my_lib"], ) -py_test_3_9( +py_test( name = "my_lib_3_9_test", srcs = ["my_lib_test.py"], main = "my_lib_test.py", + python_version = "3.9", deps = ["//libs/my_lib"], ) -py_test_3_10( +py_test( name = "my_lib_3_10_test", srcs = ["my_lib_test.py"], main = "my_lib_test.py", + python_version = "3.10", deps = ["//libs/my_lib"], ) -py_test_3_11( +py_test( name = "my_lib_3_11_test", srcs = ["my_lib_test.py"], main = "my_lib_test.py", + python_version = "3.11", deps = ["//libs/my_lib"], ) @@ -98,32 +102,36 @@ py_test( env = {"VERSION_CHECK": "3.9"}, # The default defined in the WORKSPACE. ) -py_test_3_8( +py_test( name = "version_3_8_test", srcs = ["version_test.py"], env = {"VERSION_CHECK": "3.8"}, main = "version_test.py", + python_version = "3.8", ) -py_test_3_9( +py_test( name = "version_3_9_test", srcs = ["version_test.py"], env = {"VERSION_CHECK": "3.9"}, main = "version_test.py", + python_version = "3.9", ) -py_test_3_10( +py_test( name = "version_3_10_test", srcs = ["version_test.py"], env = {"VERSION_CHECK": "3.10"}, main = "version_test.py", + python_version = "3.10", ) -py_test_3_11( +py_test( name = "version_3_11_test", srcs = ["version_test.py"], env = {"VERSION_CHECK": "3.11"}, main = "version_test.py", + python_version = "3.11", ) py_test( @@ -138,7 +146,7 @@ py_test( main = "cross_version_test.py", ) -py_test_3_10( +py_test( name = "version_3_10_takes_3_9_subprocess_test", srcs = ["cross_version_test.py"], data = [":version_3_9"], @@ -148,6 +156,7 @@ py_test_3_10( "VERSION_CHECK": "3.10", }, main = "cross_version_test.py", + python_version = "3.10", ) sh_test( diff --git a/python/config_settings/transition.bzl b/python/config_settings/transition.bzl index c241f20746..937f33bb88 100644 --- a/python/config_settings/transition.bzl +++ b/python/config_settings/transition.bzl @@ -23,12 +23,18 @@ of them should be changed to load the regular rules directly. load("//python:py_binary.bzl", _py_binary = "py_binary") load("//python:py_test.bzl", _py_test = "py_test") - -_DEPRECATION_MESSAGE = """ -The {name} symbol in @rules_python//python/config_settings:transition.bzl -is deprecated. It is an alias to the regular rule; use it directly instead: - load("@rules_python//python:{name}.bzl", "{name}") -""" +load("//python/private:deprecation.bzl", "with_deprecation") +load("//python/private:text_util.bzl", "render") + +def _with_deprecation(kwargs, *, name, python_version): + kwargs["python_version"] = python_version + return with_deprecation.symbol( + kwargs, + symbol_name = name, + old_load = "@rules_python//python/config_settings:transition.bzl", + new_load = "@rules_python//python:{}.bzl".format(name), + snippet = render.call(name, **{k: repr(v) for k, v in kwargs.items()}), + ) def py_binary(**kwargs): """[DEPRECATED] Deprecated alias for py_binary. @@ -37,11 +43,7 @@ def py_binary(**kwargs): **kwargs: keyword args forwarded onto {obj}`py_binary`. """ - deprecation = _DEPRECATION_MESSAGE.format(name = "py_binary") - if kwargs.get("deprecation"): - deprecation = kwargs.get("deprecation") + "\n\n" + deprecation - kwargs["deprecation"] = deprecation - _py_binary(**kwargs) + _py_binary(**_with_deprecation(kwargs, name = "py_binary", python_version = kwargs.get("python_version"))) def py_test(**kwargs): """[DEPRECATED] Deprecated alias for py_test. @@ -49,8 +51,4 @@ def py_test(**kwargs): Args: **kwargs: keyword args forwarded onto {obj}`py_binary`. """ - deprecation = _DEPRECATION_MESSAGE.format(name = "py_test") - if kwargs.get("deprecation"): - deprecation = kwargs.get("deprecation") + "\n\n" + deprecation - kwargs["deprecation"] = deprecation - _py_test(**kwargs) + _py_test(**_with_deprecation(kwargs, name = "py_test", python_version = kwargs.get("python_version"))) diff --git a/python/private/BUILD.bazel b/python/private/BUILD.bazel index 706506a19c..14f52c541b 100644 --- a/python/private/BUILD.bazel +++ b/python/private/BUILD.bazel @@ -138,6 +138,14 @@ bzl_library( ], ) +bzl_library( + name = "deprecation_bzl", + srcs = ["deprecation.bzl"], + deps = [ + "@rules_python_internal//:rules_python_config_bzl", + ], +) + bzl_library( name = "enum_bzl", srcs = ["enum.bzl"], diff --git a/python/private/deprecation.bzl b/python/private/deprecation.bzl new file mode 100644 index 0000000000..70461c2fa1 --- /dev/null +++ b/python/private/deprecation.bzl @@ -0,0 +1,59 @@ +# Copyright 2024 The Bazel Authors. All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +"""Helper functions to deprecation utilities. +""" + +load("@rules_python_internal//:rules_python_config.bzl", "config") + +_DEPRECATION_MESSAGE = """ +The '{name}' symbol in '{old_load}' +is deprecated. It is an alias to the regular rule; use it directly instead: + +load("{new_load}", "{name}") + +{snippet} +""" + +def _symbol(kwargs, *, symbol_name, new_load, old_load, snippet = ""): + """An internal function to propagate the deprecation warning. + + This is not an API that should be used outside `rules_python`. + + Args: + kwargs: Arguments to modify. + symbol_name: {type}`str` the symbol name that is deprecated. + new_load: {type}`str` the new load location under `//`. + old_load: {type}`str` the symbol import location that we are deprecating. + snippet: {type}`str` the usage snippet of the new symbol. + + Returns: + The kwargs to be used in the macro creation. + """ + + if config.enable_deprecation_warnings: + deprecation = _DEPRECATION_MESSAGE.format( + name = symbol_name, + old_load = old_load, + new_load = new_load, + snippet = snippet, + ) + if kwargs.get("deprecation"): + deprecation = kwargs.get("deprecation") + "\n\n" + deprecation + kwargs["deprecation"] = deprecation + return kwargs + +with_deprecation = struct( + symbol = _symbol, +) diff --git a/python/private/internal_config_repo.bzl b/python/private/internal_config_repo.bzl index 7b6869e9a5..a5c4787161 100644 --- a/python/private/internal_config_repo.bzl +++ b/python/private/internal_config_repo.bzl @@ -18,12 +18,17 @@ such as globals available to Bazel versions, or propagating user environment settings for rules to later use. """ +load(":repo_utils.bzl", "repo_utils") + _ENABLE_PYSTAR_ENVVAR_NAME = "RULES_PYTHON_ENABLE_PYSTAR" _ENABLE_PYSTAR_DEFAULT = "1" +_ENABLE_DEPRECATION_WARNINGS_ENVVAR_NAME = "RULES_PYTHON_DEPRECATION_WARNINGS" +_ENABLE_DEPRECATION_WARNINGS_DEFAULT = "0" _CONFIG_TEMPLATE = """\ config = struct( enable_pystar = {enable_pystar}, + enable_deprecation_warnings = {enable_deprecation_warnings}, BuiltinPyInfo = getattr(getattr(native, "legacy_globals", None), "PyInfo", {builtin_py_info_symbol}), BuiltinPyRuntimeInfo = getattr(getattr(native, "legacy_globals", None), "PyRuntimeInfo", {builtin_py_runtime_info_symbol}), BuiltinPyCcLinkParamsProvider = getattr(getattr(native, "legacy_globals", None), "PyCcLinkParamsProvider", {builtin_py_cc_link_params_provider}), @@ -79,6 +84,7 @@ def _internal_config_repo_impl(rctx): rctx.file("rules_python_config.bzl", _CONFIG_TEMPLATE.format( enable_pystar = enable_pystar, + enable_deprecation_warnings = _bool_from_environ(rctx, _ENABLE_DEPRECATION_WARNINGS_ENVVAR_NAME, _ENABLE_DEPRECATION_WARNINGS_DEFAULT), builtin_py_info_symbol = builtin_py_info_symbol, builtin_py_runtime_info_symbol = builtin_py_runtime_info_symbol, builtin_py_cc_link_params_provider = builtin_py_cc_link_params_provider, @@ -112,4 +118,4 @@ internal_config_repo = repository_rule( ) def _bool_from_environ(rctx, key, default): - return bool(int(rctx.os.environ.get(key, default))) + return bool(int(repo_utils.getenv(rctx, key, default))) diff --git a/python/private/toolchains_repo.bzl b/python/private/toolchains_repo.bzl index 7e9a0c7ff9..5082047135 100644 --- a/python/private/toolchains_repo.bzl +++ b/python/private/toolchains_repo.bzl @@ -151,47 +151,39 @@ toolchain_aliases( rctx.file("defs.bzl", content = """\ # Generated by python/private/toolchains_repo.bzl -load( - "{rules_python}//python/config_settings:transition.bzl", - _py_binary = "py_binary", - _py_test = "py_test", -) +load("{rules_python}//python:pip.bzl", _compile_pip_requirements = "compile_pip_requirements") +load("{rules_python}//python/private:deprecation.bzl", "with_deprecation") +load("{rules_python}//python/private:text_util.bzl", "render") +load("{rules_python}//python:py_binary.bzl", _py_binary = "py_binary") +load("{rules_python}//python:py_test.bzl", _py_test = "py_test") load( "{rules_python}//python/entry_points:py_console_script_binary.bzl", _py_console_script_binary = "py_console_script_binary", ) -load("{rules_python}//python:pip.bzl", _compile_pip_requirements = "compile_pip_requirements") -def py_binary(name, **kwargs): - return _py_binary( - name = name, - python_version = "{python_version}", - **kwargs +def _with_deprecation(kwargs, *, name): + kwargs["python_version"] = "{python_version}" + return with_deprecation.symbol( + kwargs, + symbol_name = name, + old_load = "@{name}//:defs.bzl", + new_load = "@rules_python//python:{{}}.bzl".format(name), + snippet = render.call(name, **{{k: repr(v) for k,v in kwargs.items()}}) ) -def py_console_script_binary(name, **kwargs): - return _py_console_script_binary( - name = name, - binary_rule = py_binary, - **kwargs - ) +def py_binary(**kwargs): + return _py_binary(**_with_deprecation(kwargs, name = "py_binary")) -def py_test(name, **kwargs): - return _py_test( - name = name, - python_version = "{python_version}", - **kwargs - ) +def py_console_script_binary(**kwargs): + return _py_console_script_binary(**_with_deprecation(kwargs, name = "py_console_script_binary")) -def compile_pip_requirements(name, **kwargs): - return _compile_pip_requirements( - name = name, - py_binary = py_binary, - py_test = py_test, - **kwargs - ) +def py_test(**kwargs): + return _py_test(**_with_deprecation(kwargs, name = "py_test")) +def compile_pip_requirements(**kwargs): + return _compile_pip_requirements(**_with_deprecation(kwargs, name = "compile_pip_requirements")) """.format( + name = rctx.attr.name, python_version = rctx.attr.python_version, rules_python = get_repository_name(rctx.attr._rules_python_workspace), )) @@ -316,20 +308,42 @@ def _multi_toolchain_aliases_impl(rctx): rctx.file(file, content = """\ # Generated by python/private/toolchains_repo.bzl +load("{rules_python}//python:pip.bzl", _compile_pip_requirements = "compile_pip_requirements") +load("{rules_python}//python/private:deprecation.bzl", "with_deprecation") +load("{rules_python}//python/private:text_util.bzl", "render") +load("{rules_python}//python:py_binary.bzl", _py_binary = "py_binary") +load("{rules_python}//python:py_test.bzl", _py_test = "py_test") load( - "@{repository_name}//:defs.bzl", - _compile_pip_requirements = "compile_pip_requirements", - _py_binary = "py_binary", + "{rules_python}//python/entry_points:py_console_script_binary.bzl", _py_console_script_binary = "py_console_script_binary", - _py_test = "py_test", ) -compile_pip_requirements = _compile_pip_requirements -py_binary = _py_binary -py_console_script_binary = _py_console_script_binary -py_test = _py_test +def _with_deprecation(kwargs, *, name): + kwargs["python_version"] = "{python_version}" + return with_deprecation.symbol( + kwargs, + symbol_name = name, + old_load = "@{name}//{python_version}:defs.bzl", + new_load = "@rules_python//python:{{}}.bzl".format(name), + snippet = render.call(name, **{{k: repr(v) for k,v in kwargs.items()}}) + ) + +def py_binary(**kwargs): + return _py_binary(**_with_deprecation(kwargs, name = "py_binary")) + +def py_console_script_binary(**kwargs): + return _py_console_script_binary(**_with_deprecation(kwargs, name = "py_console_script_binary")) + +def py_test(**kwargs): + return _py_test(**_with_deprecation(kwargs, name = "py_test")) + +def compile_pip_requirements(**kwargs): + return _compile_pip_requirements(**_with_deprecation(kwargs, name = "compile_pip_requirements")) """.format( repository_name = repository_name, + name = rctx.attr.name, + python_version = python_version, + rules_python = get_repository_name(rctx.attr._rules_python_workspace), )) rctx.file("{}/BUILD.bazel".format(python_version), "") diff --git a/python/uv/private/lock.bzl b/python/uv/private/lock.bzl index 217b6e4831..f4dfa36eff 100644 --- a/python/uv/private/lock.bzl +++ b/python/uv/private/lock.bzl @@ -17,7 +17,6 @@ load("@bazel_skylib//rules:write_file.bzl", "write_file") load("//python:py_binary.bzl", "py_binary") -load("//python/config_settings:transition.bzl", transition_py_binary = "py_binary") load("//python/private:bzlmod_enabled.bzl", "BZLMOD_ENABLED") # buildifier: disable=bzl-visibility visibility(["//..."]) @@ -94,7 +93,7 @@ def lock(*, name, srcs, out, upgrade = False, universal = True, python_version = ], ) if python_version: - py_binary_rule = lambda *args, **kwargs: transition_py_binary(python_version = python_version, *args, **kwargs) + py_binary_rule = lambda *args, **kwargs: py_binary(python_version = python_version, *args, **kwargs) else: py_binary_rule = py_binary diff --git a/tests/config_settings/transition/multi_version_tests.bzl b/tests/config_settings/transition/multi_version_tests.bzl index 50b4402fce..aca341a295 100644 --- a/tests/config_settings/transition/multi_version_tests.bzl +++ b/tests/config_settings/transition/multi_version_tests.bzl @@ -16,8 +16,9 @@ load("@rules_testing//lib:analysis_test.bzl", "analysis_test") load("@rules_testing//lib:test_suite.bzl", "test_suite") load("@rules_testing//lib:util.bzl", "TestingAspectInfo", rt_util = "util") +load("//python:py_binary.bzl", "py_binary") load("//python:py_info.bzl", "PyInfo") -load("//python/config_settings:transition.bzl", py_binary_transitioned = "py_binary", py_test_transitioned = "py_test") +load("//python:py_test.bzl", "py_test") load("//python/private:reexports.bzl", "BuiltinPyInfo") # buildifier: disable=bzl-visibility load("//python/private:util.bzl", "IS_BAZEL_7_OR_HIGHER") # buildifier: disable=bzl-visibility load("//tests/support:support.bzl", "CC_TOOLCHAIN") @@ -34,7 +35,7 @@ _tests = [] def _test_py_test_with_transition(name): rt_util.helper_target( - py_test_transitioned, + py_test, name = name + "_subject", srcs = [name + "_subject.py"], python_version = _PYTHON_VERSION, @@ -56,7 +57,7 @@ _tests.append(_test_py_test_with_transition) def _test_py_binary_with_transition(name): rt_util.helper_target( - py_binary_transitioned, + py_binary, name = name + "_subject", srcs = [name + "_subject.py"], python_version = _PYTHON_VERSION, @@ -78,7 +79,7 @@ _tests.append(_test_py_binary_with_transition) def _setup_py_binary_windows(name, *, impl, build_python_zip): rt_util.helper_target( - py_binary_transitioned, + py_binary, name = name + "_subject", srcs = [name + "_subject.py"], python_version = _PYTHON_VERSION, diff --git a/tests/deprecated/BUILD.bazel b/tests/deprecated/BUILD.bazel new file mode 100644 index 0000000000..4b920679f1 --- /dev/null +++ b/tests/deprecated/BUILD.bazel @@ -0,0 +1,96 @@ +load("@bazel_skylib//rules:build_test.bzl", "build_test") +load( + "@python//3.11:defs.bzl", + hub_compile_pip_requirements = "compile_pip_requirements", + hub_py_binary = "py_binary", + hub_py_console_script_binary = "py_console_script_binary", + hub_py_test = "py_test", +) +load( + "@python_3_11//:defs.bzl", + versioned_compile_pip_requirements = "compile_pip_requirements", + versioned_py_binary = "py_binary", + versioned_py_console_script_binary = "py_console_script_binary", + versioned_py_test = "py_test", +) +load("//python/config_settings:transition.bzl", transition_py_binary = "py_binary", transition_py_test = "py_test") + +# TODO @aignas 2025-01-22: remove the referenced symbols when releasing v2 + +transition_py_binary( + name = "transition_py_binary", + srcs = ["dummy.py"], + main = "dummy.py", + python_version = "3.11", +) + +transition_py_test( + name = "transition_py_test", + srcs = ["dummy.py"], + main = "dummy.py", + python_version = "3.11", +) + +versioned_py_binary( + name = "versioned_py_binary", + srcs = ["dummy.py"], + main = "dummy.py", +) + +versioned_py_test( + name = "versioned_py_test", + srcs = ["dummy.py"], + main = "dummy.py", +) + +versioned_py_console_script_binary( + name = "versioned_py_console_script_binary", + pkg = "@rules_python_publish_deps//twine", + script = "twine", +) + +versioned_compile_pip_requirements( + name = "versioned_compile_pip_requirements", + src = "requirements.in", + requirements_txt = "requirements.txt", +) + +hub_py_binary( + name = "hub_py_binary", + srcs = ["dummy.py"], + main = "dummy.py", +) + +hub_py_test( + name = "hub_py_test", + srcs = ["dummy.py"], + main = "dummy.py", +) + +hub_py_console_script_binary( + name = "hub_py_console_script_binary", + pkg = "@rules_python_publish_deps//twine", + script = "twine", +) + +hub_compile_pip_requirements( + name = "hub_compile_pip_requirements", + src = "requirements.in", + requirements_txt = "requirements_hub.txt", +) + +build_test( + name = "build_test", + targets = [ + "transition_py_binary", + "transition_py_test", + "versioned_py_binary", + "versioned_py_test", + "versioned_py_console_script_binary", + "versioned_compile_pip_requirements", + "hub_py_binary", + "hub_py_test", + "hub_py_console_script_binary", + "hub_compile_pip_requirements", + ], +) diff --git a/tests/deprecated/dummy.py b/tests/deprecated/dummy.py new file mode 100644 index 0000000000..e69de29bb2 diff --git a/tests/deprecated/requirements.in b/tests/deprecated/requirements.in new file mode 100644 index 0000000000..e69de29bb2 diff --git a/tests/deprecated/requirements.txt b/tests/deprecated/requirements.txt new file mode 100644 index 0000000000..4d53f7c4e3 --- /dev/null +++ b/tests/deprecated/requirements.txt @@ -0,0 +1,6 @@ +# +# This file is autogenerated by pip-compile with Python 3.11 +# by the following command: +# +# bazel run //tests/deprecated:versioned_compile_pip_requirements.update +# diff --git a/tests/deprecated/requirements_hub.txt b/tests/deprecated/requirements_hub.txt new file mode 100644 index 0000000000..444beb63a5 --- /dev/null +++ b/tests/deprecated/requirements_hub.txt @@ -0,0 +1,6 @@ +# +# This file is autogenerated by pip-compile with Python 3.11 +# by the following command: +# +# bazel run //tests/deprecated:hub_compile_pip_requirements.update +# diff --git a/tools/publish/BUILD.bazel b/tools/publish/BUILD.bazel index 1648ac85df..4cf99e4d97 100644 --- a/tools/publish/BUILD.bazel +++ b/tools/publish/BUILD.bazel @@ -1,14 +1,10 @@ -load("//python/config_settings:transition.bzl", "py_binary") load("//python/entry_points:py_console_script_binary.bzl", "py_console_script_binary") load("//tools/private:publish_deps.bzl", "publish_deps") py_console_script_binary( name = "twine", - # We use a py_binary rule with version transitions to ensure that we do not - # rely on the default version of the registered python toolchain. What is more - # we are using this instead of `@python_versions//3.11:defs.bzl` because loading - # that file relies on bzlmod being enabled. - binary_rule = py_binary, + # We transition to a specific python version in order to ensure that we + # don't rely on the default version configured by the root module. pkg = "@rules_python_publish_deps//twine", python_version = "3.11", script = "twine", From 626b03a9fadf076abe50c32b07242ce3bf29bdf3 Mon Sep 17 00:00:00 2001 From: Philipp Stephani Date: Wed, 22 Jan 2025 17:58:32 +0100 Subject: [PATCH 062/922] fix: Fix encoding of runfiles manifest and repository mapping files. (#2568) See https://github.com/bazelbuild/bazel/issues/374#issuecomment-2594713891: > all output files produced by Bazel should use UTF-8 and \n line endings on > all platforms, including Windows. Previously this would use the legacy ANSI codepage on Windows. --- CHANGELOG.md | 2 ++ examples/bzlmod/runfiles/runfiles_test.py | 12 ++++++------ .../runfiles/runfiles_test.py | 12 ++++++------ python/runfiles/runfiles.py | 4 ++-- tests/runfiles/runfiles_test.py | 2 +- 5 files changed, 17 insertions(+), 15 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 00624db01f..9fdce66550 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -63,6 +63,8 @@ Unreleased changes template. * (gazelle) Providing multiple input requirements files to `gazelle_python_manifest` now works correctly. * (pypi) Handle trailing slashes in pip index URLs in environment variables, fixes [#2554](https://github.com/bazelbuild/rules_python/issues/2554). +* (runfiles) Runfile manifest and repository mapping files are now interpreted + as UTF-8 on all platforms. {#v0-0-0-added} ### Added diff --git a/examples/bzlmod/runfiles/runfiles_test.py b/examples/bzlmod/runfiles/runfiles_test.py index e1ba14e569..7b7e87726a 100644 --- a/examples/bzlmod/runfiles/runfiles_test.py +++ b/examples/bzlmod/runfiles/runfiles_test.py @@ -27,36 +27,36 @@ def testCurrentRepository(self): def testRunfilesWithRepoMapping(self): data_path = runfiles.Create().Rlocation("example_bzlmod/runfiles/data/data.txt") - with open(data_path) as f: + with open(data_path, "rt", encoding="utf-8", newline="\n") as f: self.assertEqual(f.read().strip(), "Hello, example_bzlmod!") def testRunfileWithRlocationpath(self): data_rlocationpath = os.getenv("DATA_RLOCATIONPATH") data_path = runfiles.Create().Rlocation(data_rlocationpath) - with open(data_path) as f: + with open(data_path, "rt", encoding="utf-8", newline="\n") as f: self.assertEqual(f.read().strip(), "Hello, example_bzlmod!") def testRunfileInOtherModuleWithOurRepoMapping(self): data_path = runfiles.Create().Rlocation( "our_other_module/other_module/pkg/data/data.txt" ) - with open(data_path) as f: + with open(data_path, "rt", encoding="utf-8", newline="\n") as f: self.assertEqual(f.read().strip(), "Hello, other_module!") def testRunfileInOtherModuleWithItsRepoMapping(self): data_path = lib.GetRunfilePathWithRepoMapping() - with open(data_path) as f: + with open(data_path, "rt", encoding="utf-8", newline="\n") as f: self.assertEqual(f.read().strip(), "Hello, other_module!") def testRunfileInOtherModuleWithCurrentRepository(self): data_path = lib.GetRunfilePathWithCurrentRepository() - with open(data_path) as f: + with open(data_path, "rt", encoding="utf-8", newline="\n") as f: self.assertEqual(f.read().strip(), "Hello, other_module!") def testRunfileInOtherModuleWithRlocationpath(self): data_rlocationpath = os.getenv("OTHER_MODULE_DATA_RLOCATIONPATH") data_path = runfiles.Create().Rlocation(data_rlocationpath) - with open(data_path) as f: + with open(data_path, "rt", encoding="utf-8", newline="\n") as f: self.assertEqual(f.read().strip(), "Hello, other_module!") diff --git a/examples/bzlmod_build_file_generation/runfiles/runfiles_test.py b/examples/bzlmod_build_file_generation/runfiles/runfiles_test.py index 5bfa5302ef..6ce4c2db37 100644 --- a/examples/bzlmod_build_file_generation/runfiles/runfiles_test.py +++ b/examples/bzlmod_build_file_generation/runfiles/runfiles_test.py @@ -29,36 +29,36 @@ def testRunfilesWithRepoMapping(self): data_path = runfiles.Create().Rlocation( "example_bzlmod_build_file_generation/runfiles/data/data.txt" ) - with open(data_path) as f: + with open(data_path, "rt", encoding="utf-8", newline="\n") as f: self.assertEqual(f.read().strip(), "Hello, example_bzlmod!") def testRunfileWithRlocationpath(self): data_rlocationpath = os.getenv("DATA_RLOCATIONPATH") data_path = runfiles.Create().Rlocation(data_rlocationpath) - with open(data_path) as f: + with open(data_path, "rt", encoding="utf-8", newline="\n") as f: self.assertEqual(f.read().strip(), "Hello, example_bzlmod!") def testRunfileInOtherModuleWithOurRepoMapping(self): data_path = runfiles.Create().Rlocation( "our_other_module/other_module/pkg/data/data.txt" ) - with open(data_path) as f: + with open(data_path, "rt", encoding="utf-8", newline="\n") as f: self.assertEqual(f.read().strip(), "Hello, other_module!") def testRunfileInOtherModuleWithItsRepoMapping(self): data_path = lib.GetRunfilePathWithRepoMapping() - with open(data_path) as f: + with open(data_path, "rt", encoding="utf-8", newline="\n") as f: self.assertEqual(f.read().strip(), "Hello, other_module!") def testRunfileInOtherModuleWithCurrentRepository(self): data_path = lib.GetRunfilePathWithCurrentRepository() - with open(data_path) as f: + with open(data_path, "rt", encoding="utf-8", newline="\n") as f: self.assertEqual(f.read().strip(), "Hello, other_module!") def testRunfileInOtherModuleWithRlocationpath(self): data_rlocationpath = os.getenv("OTHER_MODULE_DATA_RLOCATIONPATH") data_path = runfiles.Create().Rlocation(data_rlocationpath) - with open(data_path) as f: + with open(data_path, "rt", encoding="utf-8", newline="\n") as f: self.assertEqual(f.read().strip(), "Hello, other_module!") diff --git a/python/runfiles/runfiles.py b/python/runfiles/runfiles.py index ea816c64fd..3943be5646 100644 --- a/python/runfiles/runfiles.py +++ b/python/runfiles/runfiles.py @@ -56,7 +56,7 @@ def RlocationChecked(self, path: str) -> Optional[str]: def _LoadRunfiles(path: str) -> Dict[str, str]: """Loads the runfiles manifest.""" result = {} - with open(path, "r") as f: + with open(path, "r", encoding="utf-8", newline="\n") as f: for line in f: line = line.rstrip("\n") if line.startswith(" "): @@ -367,7 +367,7 @@ def _ParseRepoMapping(repo_mapping_path: Optional[str]) -> Dict[Tuple[str, str], if not repo_mapping_path: return {} try: - with open(repo_mapping_path, "r") as f: + with open(repo_mapping_path, "r", encoding="utf-8", newline="\n") as f: content = f.read() except FileNotFoundError: return {} diff --git a/tests/runfiles/runfiles_test.py b/tests/runfiles/runfiles_test.py index cf6a70a020..a3837ac842 100644 --- a/tests/runfiles/runfiles_test.py +++ b/tests/runfiles/runfiles_test.py @@ -552,7 +552,7 @@ def __init__( def __enter__(self) -> Any: tmpdir = os.environ.get("TEST_TMPDIR") self._path = os.path.join(tempfile.mkdtemp(dir=tmpdir), self._name) - with open(self._path, "wt") as f: + with open(self._path, "wt", encoding="utf-8", newline="\n") as f: f.writelines(l + "\n" for l in self._contents) return self From ea716fed66f762d7e0f2307abe51751fa86075ab Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Wed, 22 Jan 2025 22:08:31 -0800 Subject: [PATCH 063/922] fix: make coverage work with bootstrap=script (#2574) The script based bootstrap wasn't expanding the coverage template variable, which prevented coverage from activating. This was introduced when it was switched to the venv layout. To fix, expand the `%coverage_tool%` template variable as done elsewhere. Tested manually, per repro instructions in #2572. While I did devise a way to mostly test this without an integration test, it was thwarted by some other bugs. Along the way, improve some of the bootstrap debug output and fix a comment. Fixes https://github.com/bazelbuild/rules_python/issues/2572 --- CHANGELOG.md | 2 ++ python/private/py_executable.bzl | 23 ++++++++++++--------- python/private/site_init_template.py | 5 +++-- python/private/stage2_bootstrap_template.py | 8 +++++-- 4 files changed, 24 insertions(+), 14 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 9fdce66550..24b83e3228 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -65,6 +65,8 @@ Unreleased changes template. fixes [#2554](https://github.com/bazelbuild/rules_python/issues/2554). * (runfiles) Runfile manifest and repository mapping files are now interpreted as UTF-8 on all platforms. +* (coverage) Coverage with `--bootstrap_impl=script` is fixed + ([#2572](https://github.com/bazelbuild/rules_python/issues/2572)). {#v0-0-0-added} ### Added diff --git a/python/private/py_executable.bzl b/python/private/py_executable.bzl index da7127e070..1e437f57e1 100644 --- a/python/private/py_executable.bzl +++ b/python/private/py_executable.bzl @@ -561,6 +561,7 @@ def _create_venv(ctx, output_prefix, imports, runtime_details): template = runtime.site_init_template, output = site_init, substitutions = { + "%coverage_tool%": _get_coverage_tool_runfiles_path(ctx, runtime), "%import_all%": "True" if ctx.fragments.bazel_py.python_import_all_repositories else "False", "%site_init_runfiles_path%": "{}/{}".format(ctx.workspace_name, site_init.short_path), "%workspace_name%": ctx.workspace_name, @@ -578,6 +579,17 @@ def _create_venv(ctx, output_prefix, imports, runtime_details): def _map_each_identity(v): return v +def _get_coverage_tool_runfiles_path(ctx, runtime): + if (ctx.configuration.coverage_enabled and + runtime and + runtime.coverage_tool): + return "{}/{}".format( + ctx.workspace_name, + runtime.coverage_tool.short_path, + ) + else: + return "" + def _create_stage2_bootstrap( ctx, *, @@ -593,15 +605,6 @@ def _create_stage2_bootstrap( sibling = output_sibling, ) runtime = runtime_details.effective_runtime - if (ctx.configuration.coverage_enabled and - runtime and - runtime.coverage_tool): - coverage_tool_runfiles_path = "{}/{}".format( - ctx.workspace_name, - runtime.coverage_tool.short_path, - ) - else: - coverage_tool_runfiles_path = "" template = runtime.stage2_bootstrap_template @@ -609,7 +612,7 @@ def _create_stage2_bootstrap( template = template, output = output, substitutions = { - "%coverage_tool%": coverage_tool_runfiles_path, + "%coverage_tool%": _get_coverage_tool_runfiles_path(ctx, runtime), "%import_all%": "True" if ctx.fragments.bazel_py.python_import_all_repositories else "False", "%imports%": ":".join(imports.to_list()), "%main%": "{}/{}".format(ctx.workspace_name, main_py.short_path), diff --git a/python/private/site_init_template.py b/python/private/site_init_template.py index 7a32210bff..dcbd799909 100644 --- a/python/private/site_init_template.py +++ b/python/private/site_init_template.py @@ -163,7 +163,7 @@ def _maybe_add_path(path): if cov_tool: _print_verbose_coverage(f"Using toolchain coverage_tool {cov_tool}") elif cov_tool := os.environ.get("PYTHON_COVERAGE"): - _print_verbose_coverage(f"PYTHON_COVERAGE: {cov_tool}") + _print_verbose_coverage(f"Using env var coverage: PYTHON_COVERAGE={cov_tool}") if cov_tool: if os.path.isabs(cov_tool): @@ -185,7 +185,7 @@ def _maybe_add_path(path): coverage_setup = True else: _print_verbose_coverage( - "Coverage was enabled, but python coverage tool was not configured." + "Coverage was enabled, but the coverage tool was not found or valid. " + "To enable coverage, consult the docs at " + "https://rules-python.readthedocs.io/en/latest/coverage.html" ) @@ -194,3 +194,4 @@ def _maybe_add_path(path): COVERAGE_SETUP = _setup_sys_path() +_print_verbose("DONE") diff --git a/python/private/stage2_bootstrap_template.py b/python/private/stage2_bootstrap_template.py index 1e19a71b64..b1f6b031aa 100644 --- a/python/private/stage2_bootstrap_template.py +++ b/python/private/stage2_bootstrap_template.py @@ -106,8 +106,8 @@ def print_verbose(*args, mapping=None, values=None): def print_verbose_coverage(*args): """Print output if VERBOSE_COVERAGE is non-empty in the environment.""" - if os.environ.get("VERBOSE_COVERAGE"): - print(*args, file=sys.stderr, flush=True) + if is_verbose_coverage(): + print("bootstrap: stage 2: coverage:", *args, file=sys.stderr, flush=True) def is_verbose_coverage(): @@ -271,6 +271,7 @@ def _run_py(main_filename, *, args, cwd=None): @contextlib.contextmanager def _maybe_collect_coverage(enable): + print_verbose_coverage("enabled:", enable) if not enable: yield return @@ -283,7 +284,9 @@ def _maybe_collect_coverage(enable): unique_id = uuid.uuid4() # We need for coveragepy to use relative paths. This can only be configured + # using an rc file. rcfile_name = os.path.join(coverage_dir, ".coveragerc_{}".format(unique_id)) + print_verbose_coverage("coveragerc file:", rcfile_name) with open(rcfile_name, "w") as rcfile: rcfile.write( """[run] @@ -318,6 +321,7 @@ def _maybe_collect_coverage(enable): finally: cov.stop() lcov_path = os.path.join(coverage_dir, "pylcov.dat") + print_verbose_coverage("generating lcov from:", lcov_path) cov.lcov_report( outfile=lcov_path, # Ignore errors because sometimes instrumented files aren't From 4de99abab053ab616570c41463b6f15a8200cf0b Mon Sep 17 00:00:00 2001 From: Ignas Anikevicius <240938+aignas@users.noreply.github.com> Date: Sun, 26 Jan 2025 04:35:33 +0900 Subject: [PATCH 064/922] doc: point users to our CHANGELOG at the top of the release note (#2582) This is so that we can start pointing users at a more helpful changelog that has announcements about deprecations, etc. --- .github/workflows/create_archive_and_notes.sh | 14 ++++++++++---- 1 file changed, 10 insertions(+), 4 deletions(-) diff --git a/.github/workflows/create_archive_and_notes.sh b/.github/workflows/create_archive_and_notes.sh index 29f9f8b9f7..dc7f8a6982 100755 --- a/.github/workflows/create_archive_and_notes.sh +++ b/.github/workflows/create_archive_and_notes.sh @@ -37,6 +37,8 @@ cat > release_notes.txt << EOF For more detailed setup instructions, see https://rules-python.readthedocs.io/en/latest/getting-started.html +For the user-facing changelog see [here](https://rules-python.readthedocs.io/en/latest/changelog.html#v${TAG//./-}) + ## Using Bzlmod Add to your \`MODULE.bazel\` file: @@ -44,15 +46,19 @@ Add to your \`MODULE.bazel\` file: \`\`\`starlark bazel_dep(name = "rules_python", version = "${TAG}") -pip = use_extension("@rules_python//python/extensions:pip.bzl", "pip") +python = use_extension("@rules_python//python/extensions:python.bzl", "python") +python.toolchain( + python_version = "3.13", +) +pip = use_extension("@rules_python//python/extensions:pip.bzl", "pip") pip.parse( - hub_name = "pip", - python_version = "3.11", + hub_name = "pypi", + python_version = "3.13", requirements_lock = "//:requirements_lock.txt", ) -use_repo(pip, "pip") +use_repo(pip, "pypi") \`\`\` ## Using WORKSPACE From 80aab4a2c8f2cfcf8b70a935b0302f5b7b2917e4 Mon Sep 17 00:00:00 2001 From: Philipp Schrader Date: Sat, 25 Jan 2025 22:12:26 -0800 Subject: [PATCH 065/922] fix: Enable location expansion for `sh_py_run_test` (#2583) I noticed that my `$(location //path/to:target)` wasn't getting expanded when writing a test. This patch fixes the issue by forwarding the already-expanded environment from the inner target to the outer target. --- tests/support/sh_py_run_test.bzl | 9 +++------ 1 file changed, 3 insertions(+), 6 deletions(-) diff --git a/tests/support/sh_py_run_test.bzl b/tests/support/sh_py_run_test.bzl index 7fb7016eec..9bf0a7402e 100644 --- a/tests/support/sh_py_run_test.bzl +++ b/tests/support/sh_py_run_test.bzl @@ -86,16 +86,14 @@ def _py_reconfig_impl(ctx): default_info.default_runfiles, ), ), - testing.TestEnvironment( - environment = ctx.attr.env, - ), + # Inherit the expanded environment from the inner target. + ctx.attr.target[RunEnvironmentInfo], ] def _make_reconfig_rule(**kwargs): attrs = { "bootstrap_impl": attr.string(), "build_python_zip": attr.string(default = "auto"), - "env": attr.string_dict(), "extra_toolchains": attr.string_list( doc = """ Value for the --extra_toolchains flag. @@ -133,7 +131,6 @@ def py_reconfig_test(*, name, **kwargs): reconfig_kwargs["bootstrap_impl"] = kwargs.pop("bootstrap_impl", None) reconfig_kwargs["extra_toolchains"] = kwargs.pop("extra_toolchains", None) reconfig_kwargs["python_version"] = kwargs.pop("python_version", None) - reconfig_kwargs["env"] = kwargs.get("env") reconfig_kwargs["target_compatible_with"] = kwargs.get("target_compatible_with") inner_name = "_{}_inner".format(name) @@ -172,7 +169,7 @@ def sh_py_run_test(*, name, sh_src, py_src, **kwargs): py_binary_kwargs = { key: kwargs.pop(key) - for key in ("imports", "deps") + for key in ("imports", "deps", "env") if key in kwargs } From 0475c9e63c399f6063371e246d382f1f43ae4fb1 Mon Sep 17 00:00:00 2001 From: Ignas Anikevicius <240938+aignas@users.noreply.github.com> Date: Mon, 27 Jan 2025 02:10:48 +0900 Subject: [PATCH 066/922] fix(sphinxdocs): do not crash when tag_class does not have doc (#2585) It seems that there was a typo in the code and instead of calling `self._write` we were calling `self.write`. It went unnoticed because of lack of coverage. This adds test code exercising the edge case and fixes the typo. Fixes #2579 --- CHANGELOG.md | 2 ++ sphinxdocs/private/proto_to_markdown.py | 2 +- .../tests/proto_to_markdown/proto_to_markdown_test.py | 11 +++++++++++ 3 files changed, 14 insertions(+), 1 deletion(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 24b83e3228..1848c1dc59 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -67,6 +67,8 @@ Unreleased changes template. as UTF-8 on all platforms. * (coverage) Coverage with `--bootstrap_impl=script` is fixed ([#2572](https://github.com/bazelbuild/rules_python/issues/2572)). +* (sphinxdocs) Do not crash when `tag_class` does not have a populated `doc` value. + Fixes ([#2579](https://github.com/bazelbuild/rules_python/issues/2579)). {#v0-0-0-added} ### Added diff --git a/sphinxdocs/private/proto_to_markdown.py b/sphinxdocs/private/proto_to_markdown.py index 18fbd12ede..9dac71d51c 100644 --- a/sphinxdocs/private/proto_to_markdown.py +++ b/sphinxdocs/private/proto_to_markdown.py @@ -197,7 +197,7 @@ def _render_module_extension(self, mod_ext: stardoc_output_pb2.ModuleExtensionIn # Ensure a newline between the directive and the doc fields, # otherwise they get parsed as directive options instead. if not doc_string and tag.attribute: - self.write("\n") + self._write("\n") self._render_attributes(tag.attribute) self._write(":::::\n") self._write("::::::\n") diff --git a/sphinxdocs/tests/proto_to_markdown/proto_to_markdown_test.py b/sphinxdocs/tests/proto_to_markdown/proto_to_markdown_test.py index 66e3224b20..9d15b830e3 100644 --- a/sphinxdocs/tests/proto_to_markdown/proto_to_markdown_test.py +++ b/sphinxdocs/tests/proto_to_markdown/proto_to_markdown_test.py @@ -82,6 +82,14 @@ default_value: "[BZLMOD_EXT_TAG_A_ATTRIBUTE_1_DEFAULT_VALUE]" } } + tag_class: { + tag_name: "bzlmod_ext_tag_no_doc" + attribute: { + name: "bzlmod_ext_tag_a_attribute_2", + type: STRING_LIST + default_value: "[BZLMOD_EXT_TAG_A_ATTRIBUTE_2_DEFAULT_VALUE]" + } + } } repository_rule_info: { rule_name: "repository_rule", @@ -151,6 +159,9 @@ def test_basic_rendering_everything(self): self.assertRegex(actual, "bzlmod_ext_tag_a_attribute_1") self.assertRegex(actual, "BZLMOD_EXT_TAG_A_ATTRIBUTE_1_DOC_STRING") self.assertRegex(actual, "BZLMOD_EXT_TAG_A_ATTRIBUTE_1_DEFAULT_VALUE") + self.assertRegex(actual, "{bzl:tag-class} bzlmod_ext_tag_no_doc") + self.assertRegex(actual, "bzlmod_ext_tag_a_attribute_2") + self.assertRegex(actual, "BZLMOD_EXT_TAG_A_ATTRIBUTE_2_DEFAULT_VALUE") self.assertRegex(actual, "{bzl:repo-rule} repository_rule") self.assertRegex(actual, "REPOSITORY_RULE_DOC_STRING") From 18f76f9cc4c3c6c4470f7a881e6ea4e8b80d7bab Mon Sep 17 00:00:00 2001 From: Ignas Anikevicius <240938+aignas@users.noreply.github.com> Date: Mon, 27 Jan 2025 02:21:03 +0900 Subject: [PATCH 067/922] refactor(uv): move around uv implementation files (#2580) This PR starts establishing a structure that will eventually become a part of our API. This is a prerequisite for #2578 which removes the versions.bzl file in favour of a more dynamic configuration of the extension. We also remove the `defs.bzl` to establish a one symbol per file convention. Things that I wish we could change is `//python/uv:extensions.bzl` and the fact that we have `extensions` in the load path. I think it cannot be removed, because that may break the BCR test. On the other hand, maybe we could remove it and do an alpha release to verify this assumption. Work towards #1975 --- MODULE.bazel | 2 +- docs/BUILD.bazel | 6 ++- examples/bzlmod/MODULE.bazel | 2 +- python/uv/BUILD.bazel | 28 +++++----- python/uv/lock.bzl | 22 ++++++++ python/uv/private/BUILD.bazel | 52 +++++++++++++++++-- python/uv/private/lock.bzl | 24 ++++----- python/uv/{extensions.bzl => private/uv.bzl} | 13 +++-- .../uv_repositories.bzl} | 16 +++--- .../uv_toolchain.bzl} | 2 +- .../{providers.bzl => uv_toolchain_info.bzl} | 0 ...chains_repo.bzl => uv_toolchains_repo.bzl} | 0 python/uv/uv.bzl | 22 ++++++++ python/uv/uv_toolchain.bzl | 22 ++++++++ python/uv/{defs.bzl => uv_toolchain_info.bzl} | 9 ++-- 15 files changed, 162 insertions(+), 58 deletions(-) create mode 100644 python/uv/lock.bzl rename python/uv/{extensions.bzl => private/uv.bzl} (85%) rename python/uv/{repositories.bzl => private/uv_repositories.bzl} (88%) rename python/uv/{toolchain.bzl => private/uv_toolchain.bzl} (96%) rename python/uv/private/{providers.bzl => uv_toolchain_info.bzl} (100%) rename python/uv/private/{toolchains_repo.bzl => uv_toolchains_repo.bzl} (100%) create mode 100644 python/uv/uv.bzl create mode 100644 python/uv/uv_toolchain.bzl rename python/uv/{defs.bzl => uv_toolchain_info.bzl} (78%) diff --git a/MODULE.bazel b/MODULE.bazel index 2ac5a27223..7034357f61 100644 --- a/MODULE.bazel +++ b/MODULE.bazel @@ -175,7 +175,7 @@ use_repo( # EXPERIMENTAL: This is experimental and may be removed without notice uv = use_extension( - "//python/uv:extensions.bzl", + "//python/uv:uv.bzl", "uv", dev_dependency = True, ) diff --git a/docs/BUILD.bazel b/docs/BUILD.bazel index e365532d01..ea386f114a 100644 --- a/docs/BUILD.bazel +++ b/docs/BUILD.bazel @@ -16,7 +16,7 @@ load("@bazel_skylib//:bzl_library.bzl", "bzl_library") load("@dev_pip//:requirements.bzl", "requirement") load("//python/private:bzlmod_enabled.bzl", "BZLMOD_ENABLED") # buildifier: disable=bzl-visibility load("//python/private:util.bzl", "IS_BAZEL_7_OR_HIGHER") # buildifier: disable=bzl-visibility -load("//python/uv/private:lock.bzl", "lock") # buildifier: disable=bzl-visibility +load("//python/uv:lock.bzl", "lock") # buildifier: disable=bzl-visibility load("//sphinxdocs:readthedocs.bzl", "readthedocs_install") load("//sphinxdocs:sphinx.bzl", "sphinx_build_binary", "sphinx_docs") load("//sphinxdocs:sphinx_docs_library.bzl", "sphinx_docs_library") @@ -105,6 +105,10 @@ sphinx_stardocs( "//python/private/api:py_common_api_bzl", "//python/private/pypi:config_settings_bzl", "//python/private/pypi:pkg_aliases_bzl", + "//python/uv:lock_bzl", + "//python/uv:uv_bzl", + "//python/uv:uv_toolchain_bzl", + "//python/uv:uv_toolchain_info_bzl", ] + ([ # Bazel 6 + Stardoc isn't able to parse something about the python bzlmod extension "//python/extensions:python_bzl", diff --git a/examples/bzlmod/MODULE.bazel b/examples/bzlmod/MODULE.bazel index 536e3b2b67..d8535a0115 100644 --- a/examples/bzlmod/MODULE.bazel +++ b/examples/bzlmod/MODULE.bazel @@ -105,7 +105,7 @@ python.single_version_platform_override( use_repo(python, "python_3_10", "python_3_9", "python_versions", "pythons_hub") # EXPERIMENTAL: This is experimental and may be removed without notice -uv = use_extension("@rules_python//python/uv:extensions.bzl", "uv") +uv = use_extension("@rules_python//python/uv:uv.bzl", "uv") uv.toolchain(uv_version = "0.4.25") use_repo(uv, "uv_toolchains") diff --git a/python/uv/BUILD.bazel b/python/uv/BUILD.bazel index 383bdfcc3c..7ce6ce0523 100644 --- a/python/uv/BUILD.bazel +++ b/python/uv/BUILD.bazel @@ -27,9 +27,6 @@ filegroup( visibility = ["//:__subpackages__"], ) -# For stardoc to reference the files -exports_files(["defs.bzl"]) - toolchain_type( name = "uv_toolchain_type", visibility = ["//visibility:public"], @@ -48,34 +45,33 @@ current_toolchain( ) bzl_library( - name = "defs", - srcs = ["defs.bzl"], + name = "lock_bzl", + srcs = ["lock.bzl"], # EXPERIMENTAL: Visibility is restricted to allow for changes. visibility = ["//:__subpackages__"], + deps = ["//python/uv/private:lock_bzl"], ) bzl_library( - name = "extensions", - srcs = ["extensions.bzl"], + name = "uv_bzl", + srcs = ["uv.bzl"], # EXPERIMENTAL: Visibility is restricted to allow for changes. visibility = ["//:__subpackages__"], - deps = [":repositories"], + deps = ["//python/uv/private:uv_bzl"], ) bzl_library( - name = "repositories", - srcs = ["repositories.bzl"], + name = "uv_toolchain_bzl", + srcs = ["uv_toolchain.bzl"], # EXPERIMENTAL: Visibility is restricted to allow for changes. visibility = ["//:__subpackages__"], - deps = [ - "//python/uv/private:toolchains_repo", - "//python/uv/private:versions", - ], + deps = ["//python/uv/private:uv_toolchain_bzl"], ) bzl_library( - name = "toolchain", - srcs = ["toolchain.bzl"], + name = "uv_toolchain_info_bzl", + srcs = ["uv_toolchain_info.bzl"], # EXPERIMENTAL: Visibility is restricted to allow for changes. visibility = ["//:__subpackages__"], + deps = ["//python/uv/private:uv_toolchain_info_bzl"], ) diff --git a/python/uv/lock.bzl b/python/uv/lock.bzl new file mode 100644 index 0000000000..edffe4728c --- /dev/null +++ b/python/uv/lock.bzl @@ -0,0 +1,22 @@ +# Copyright 2025 The Bazel Authors. All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +"""The `uv` locking rule. + +EXPERIMENTAL: This is experimental and may be removed without notice +""" + +load("//python/uv/private:lock.bzl", _lock = "lock") + +lock = _lock diff --git a/python/uv/private/BUILD.bazel b/python/uv/private/BUILD.bazel index 80fd23913f..006c856d02 100644 --- a/python/uv/private/BUILD.bazel +++ b/python/uv/private/BUILD.bazel @@ -21,20 +21,62 @@ filegroup( ) bzl_library( - name = "current_toolchain", + name = "current_toolchain_bzl", srcs = ["current_toolchain.bzl"], visibility = ["//python/uv:__subpackages__"], ) bzl_library( - name = "toolchain_types", + name = "lock_bzl", + srcs = ["lock.bzl"], + visibility = ["//python/uv:__subpackages__"], + deps = [ + "//python:py_binary_bzl", + "//python/private:bzlmod_enabled_bzl", + "@bazel_skylib//rules:write_file", + ], +) + +bzl_library( + name = "toolchain_types_bzl", srcs = ["toolchain_types.bzl"], visibility = ["//python/uv:__subpackages__"], ) bzl_library( - name = "toolchains_repo", - srcs = ["toolchains_repo.bzl"], + name = "uv_bzl", + srcs = ["uv.bzl"], + visibility = ["//python/uv:__subpackages__"], + deps = [":uv_repositories_bzl"], +) + +bzl_library( + name = "uv_repositories_bzl", + srcs = ["uv_repositories.bzl"], + visibility = ["//python/uv:__subpackages__"], + deps = [ + ":toolchain_types_bzl", + ":uv_toolchains_repo_bzl", + ":versions_bzl", + ], +) + +bzl_library( + name = "uv_toolchain_bzl", + srcs = ["uv_toolchain.bzl"], + visibility = ["//python/uv:__subpackages__"], + deps = [":uv_toolchain_info_bzl"], +) + +bzl_library( + name = "uv_toolchain_info_bzl", + srcs = ["uv_toolchain_info.bzl"], + visibility = ["//python/uv:__subpackages__"], +) + +bzl_library( + name = "uv_toolchains_repo_bzl", + srcs = ["uv_toolchains_repo.bzl"], visibility = ["//python/uv:__subpackages__"], deps = [ "//python/private:text_util_bzl", @@ -42,7 +84,7 @@ bzl_library( ) bzl_library( - name = "versions", + name = "versions_bzl", srcs = ["versions.bzl"], visibility = ["//python/uv:__subpackages__"], ) diff --git a/python/uv/private/lock.bzl b/python/uv/private/lock.bzl index f4dfa36eff..e0491b282c 100644 --- a/python/uv/private/lock.bzl +++ b/python/uv/private/lock.bzl @@ -26,9 +26,14 @@ _REQUIREMENTS_TARGET_COMPATIBLE_WITH = select({ "//conditions:default": [], }) if BZLMOD_ENABLED else ["@platforms//:incompatible"] -def lock(*, name, srcs, out, upgrade = False, universal = True, python_version = None, args = [], **kwargs): +def lock(*, name, srcs, out, upgrade = False, universal = True, args = [], **kwargs): """Pin the requirements based on the src files. + Differences with the current {obj}`compile_pip_requirements` rule: + - This is implemented in shell and uv. + - This does not error out if the output file does not exist yet. + - Supports transitions out of the box. + Args: name: The name of the target to run for updating the requirements. srcs: The srcs to use as inputs. @@ -36,15 +41,8 @@ def lock(*, name, srcs, out, upgrade = False, universal = True, python_version = upgrade: Tell `uv` to always upgrade the dependencies instead of keeping them as they are. universal: Tell `uv` to generate a universal lock file. - python_version: Tell `rules_python` to use a particular version. - Defaults to the default py toolchain. - args: Extra args to pass to the rule. - **kwargs: Extra kwargs passed to the binary rule. - - Differences with the current pip-compile rule: - - This is implemented in shell and uv. - - This does not error out if the output file does not exist yet. - - Supports transitions out of the box. + args: Extra args to pass to `uv`. + **kwargs: Extra kwargs passed to the {obj}`py_binary` rule. """ pkg = native.package_name() update_target = name + ".update" @@ -92,10 +90,6 @@ def lock(*, name, srcs, out, upgrade = False, universal = True, python_version = Label("//python:current_py_toolchain"), ], ) - if python_version: - py_binary_rule = lambda *args, **kwargs: py_binary(python_version = python_version, *args, **kwargs) - else: - py_binary_rule = py_binary # Write a script that can be used for updating the in-tree version of the # requirements file @@ -116,7 +110,7 @@ def lock(*, name, srcs, out, upgrade = False, universal = True, python_version = ], ) - py_binary_rule( + py_binary( name = update_target, srcs = [update_target + ".py"], main = update_target + ".py", diff --git a/python/uv/extensions.bzl b/python/uv/private/uv.bzl similarity index 85% rename from python/uv/extensions.bzl rename to python/uv/private/uv.bzl index 82560eb17c..886e7fe748 100644 --- a/python/uv/extensions.bzl +++ b/python/uv/private/uv.bzl @@ -18,15 +18,18 @@ EXPERIMENTAL: This is experimental and may be removed without notice A module extension for working with uv. """ -load("//python/uv:repositories.bzl", "uv_register_toolchains") +load(":uv_repositories.bzl", "uv_repositories") _DOC = """\ A module extension for working with uv. """ -uv_toolchain = tag_class(attrs = { - "uv_version": attr.string(doc = "Explicit version of uv.", mandatory = True), -}) +uv_toolchain = tag_class( + doc = "Configure uv toolchain for lock file generation.", + attrs = { + "uv_version": attr.string(doc = "Explicit version of uv.", mandatory = True), + }, +) def _uv_toolchain_extension(module_ctx): for mod in module_ctx.modules: @@ -38,7 +41,7 @@ def _uv_toolchain_extension(module_ctx): "NOTE: We may wish to enforce a policy where toolchain configuration is only allowed in the root module, or in rules_python. See https://github.com/bazelbuild/bazel/discussions/22024", ) - uv_register_toolchains( + uv_repositories( uv_version = toolchain.uv_version, register_toolchains = False, ) diff --git a/python/uv/repositories.bzl b/python/uv/private/uv_repositories.bzl similarity index 88% rename from python/uv/repositories.bzl rename to python/uv/private/uv_repositories.bzl index 0125b2033b..24fb9c2447 100644 --- a/python/uv/repositories.bzl +++ b/python/uv/private/uv_repositories.bzl @@ -18,13 +18,13 @@ EXPERIMENTAL: This is experimental and may be removed without notice Create repositories for uv toolchain dependencies """ -load("//python/uv/private:toolchain_types.bzl", "UV_TOOLCHAIN_TYPE") -load("//python/uv/private:toolchains_repo.bzl", "uv_toolchains_repo") -load("//python/uv/private:versions.bzl", "UV_PLATFORMS", "UV_TOOL_VERSIONS") +load(":toolchain_types.bzl", "UV_TOOLCHAIN_TYPE") +load(":uv_toolchains_repo.bzl", "uv_toolchains_repo") +load(":versions.bzl", "UV_PLATFORMS", "UV_TOOL_VERSIONS") UV_BUILD_TMPL = """\ # Generated by repositories.bzl -load("@rules_python//python/uv:toolchain.bzl", "uv_toolchain") +load("@rules_python//python/uv:uv_toolchain.bzl", "uv_toolchain") uv_toolchain( name = "uv_toolchain", @@ -77,13 +77,13 @@ uv_repository = repository_rule( }, ) -# buildifier: disable=unnamed-macro -def uv_register_toolchains(uv_version = None, register_toolchains = True): +def uv_repositories(name = "uv_toolchains", uv_version = None, register_toolchains = True): """Convenience macro which does typical toolchain setup Skip this macro if you need more control over the toolchain setup. Args: + name: {type}`str` The name of the toolchains repo. uv_version: The uv toolchain version to download. register_toolchains: If true, repositories will be generated to produce and register `uv_toolchain` targets. """ @@ -109,7 +109,7 @@ def uv_register_toolchains(uv_version = None, register_toolchains = True): toolchain_compatible_with_by_toolchain[toolchain_name] = UV_PLATFORMS[platform].compatible_with uv_toolchains_repo( - name = "uv_toolchains", + name = name, toolchain_type = str(UV_TOOLCHAIN_TYPE), toolchain_names = toolchain_names, toolchain_labels = toolchain_labels_by_toolchain, @@ -117,4 +117,4 @@ def uv_register_toolchains(uv_version = None, register_toolchains = True): ) if register_toolchains: - native.register_toolchains("@uv_toolchains//:all") + native.register_toolchains("@{}/:all".format(name)) diff --git a/python/uv/toolchain.bzl b/python/uv/private/uv_toolchain.bzl similarity index 96% rename from python/uv/toolchain.bzl rename to python/uv/private/uv_toolchain.bzl index 3cd5850acd..3b51f5f533 100644 --- a/python/uv/toolchain.bzl +++ b/python/uv/private/uv_toolchain.bzl @@ -18,7 +18,7 @@ EXPERIMENTAL: This is experimental and may be removed without notice This module implements the uv toolchain rule """ -load("//python/uv/private:providers.bzl", "UvToolchainInfo") +load(":uv_toolchain_info.bzl", "UvToolchainInfo") def _uv_toolchain_impl(ctx): uv = ctx.attr.uv diff --git a/python/uv/private/providers.bzl b/python/uv/private/uv_toolchain_info.bzl similarity index 100% rename from python/uv/private/providers.bzl rename to python/uv/private/uv_toolchain_info.bzl diff --git a/python/uv/private/toolchains_repo.bzl b/python/uv/private/uv_toolchains_repo.bzl similarity index 100% rename from python/uv/private/toolchains_repo.bzl rename to python/uv/private/uv_toolchains_repo.bzl diff --git a/python/uv/uv.bzl b/python/uv/uv.bzl new file mode 100644 index 0000000000..d72ab9dc3d --- /dev/null +++ b/python/uv/uv.bzl @@ -0,0 +1,22 @@ +# Copyright 2025 The Bazel Authors. All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +""" EXPERIMENTAL: This is experimental and may be removed without notice. + +The uv toolchain extension. +""" + +load("//python/uv/private:uv.bzl", _uv = "uv") + +uv = _uv diff --git a/python/uv/uv_toolchain.bzl b/python/uv/uv_toolchain.bzl new file mode 100644 index 0000000000..a4b466cb1b --- /dev/null +++ b/python/uv/uv_toolchain.bzl @@ -0,0 +1,22 @@ +# Copyright 2025 The Bazel Authors. All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +"""The `uv_toolchain` rule. + +EXPERIMENTAL: This is experimental and may be removed without notice +""" + +load("//python/uv/private:uv_toolchain.bzl", _uv_toolchain = "uv_toolchain") + +uv_toolchain = _uv_toolchain diff --git a/python/uv/defs.bzl b/python/uv/uv_toolchain_info.bzl similarity index 78% rename from python/uv/defs.bzl rename to python/uv/uv_toolchain_info.bzl index 20b426a355..1ae89636be 100644 --- a/python/uv/defs.bzl +++ b/python/uv/uv_toolchain_info.bzl @@ -1,4 +1,4 @@ -# Copyright 2024 The Bazel Authors. All rights reserved. +# Copyright 2025 The Bazel Authors. All rights reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); # you may not use this file except in compliance with the License. @@ -12,12 +12,11 @@ # See the License for the specific language governing permissions and # limitations under the License. -""" -EXPERIMENTAL: This is experimental and may be removed without notice +"""The `UvToolchainInfo` provider. -A toolchain for uv +EXPERIMENTAL: This is experimental and may be removed without notice """ -load("//python/uv/private:providers.bzl", _UvToolchainInfo = "UvToolchainInfo") +load("//python/uv/private:uv_toolchain_info.bzl", _UvToolchainInfo = "UvToolchainInfo") UvToolchainInfo = _UvToolchainInfo From 309ee59968cf6759c09d8669f56f675b2ce17108 Mon Sep 17 00:00:00 2001 From: Ignas Anikevicius <240938+aignas@users.noreply.github.com> Date: Tue, 28 Jan 2025 12:07:28 +0900 Subject: [PATCH 068/922] revert: Updated pip and packaging versions to work with free-threading packages (#2514) (#2584) This reverts commit fbf8bc10a466c498fc80c6b58c939a87a8d9e929 (#2514) Also, update the CHANGELOG about the reverting. Fixes #908, which is about the `pip-compile` not using the right files for performing the locking. It seems that the `pip` upgrade regressed this error. --- CHANGELOG.md | 5 ++++ .../dependency_resolver.py | 30 +++++-------------- python/private/pypi/deps.bzl | 8 ++--- 3 files changed, 16 insertions(+), 27 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 1848c1dc59..3f8da580a1 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -57,6 +57,8 @@ Unreleased changes template. * (rules) deprecation warnings for deprecated symbols have been turned off by default for now and can be enabled with `RULES_PYTHON_DEPRECATION_WARNINGS` env var. +* (pypi) Downgraded versions of packages: `pip` from `24.3.2` to `24.0.0` and + `packaging` from `24.2` to `24.0`. {#v0-0-0-fixed} ### Fixed @@ -67,6 +69,9 @@ Unreleased changes template. as UTF-8 on all platforms. * (coverage) Coverage with `--bootstrap_impl=script` is fixed ([#2572](https://github.com/bazelbuild/rules_python/issues/2572)). +* (pypi) Non deterministic behaviour in requirement file usage has been fixed + by reverting [#2514](https://github.com/bazelbuild/rules_python/pull/2514). + The related issue is [#908](https://github.com/bazelbuild/rules_python/issue/908). * (sphinxdocs) Do not crash when `tag_class` does not have a populated `doc` value. Fixes ([#2579](https://github.com/bazelbuild/rules_python/issues/2579)). diff --git a/python/private/pypi/dependency_resolver/dependency_resolver.py b/python/private/pypi/dependency_resolver/dependency_resolver.py index 6f6c20241b..293377dc6d 100644 --- a/python/private/pypi/dependency_resolver/dependency_resolver.py +++ b/python/private/pypi/dependency_resolver/dependency_resolver.py @@ -16,7 +16,6 @@ import atexit import os -import re import shutil import sys from pathlib import Path @@ -118,6 +117,7 @@ def main( absolute_path_prefix = resolved_requirements_file[ : -(len(requirements_file) - len(repository_prefix)) ] + # As srcs might contain references to generated files we want to # use the runfiles file first. Thus, we need to compute the relative path # from the execution root. @@ -162,19 +162,12 @@ def main( argv.append( f"--output-file={requirements_file_relative if UPDATE else requirements_out}" ) - src_files = [ + argv.extend( (src_relative if Path(src_relative).exists() else resolved_src) for src_relative, resolved_src in zip(srcs_relative, resolved_srcs) - ] - argv.extend(src_files) + ) argv.extend(extra_args) - # Replace in the output lock file - # the lines like: # via -r /absolute/path/to/ - # with: # via -r - # For Windows, we should explicitly call .as_posix() to convert \\ -> / - absolute_src_prefixes = [Path(src).absolute().parent.as_posix() + "/" for src in src_files] - if UPDATE: print("Updating " + requirements_file_relative) @@ -192,14 +185,14 @@ def main( # and we should copy the updated requirements back to the source tree. if not absolute_output_file.samefile(requirements_file_tree): atexit.register( - lambda: shutil.copy(absolute_output_file, requirements_file_tree) + lambda: shutil.copy( + absolute_output_file, requirements_file_tree + ) ) - cli(argv, standalone_mode=False) + cli(argv, standalone_mode = False) requirements_file_relative_path = Path(requirements_file_relative) content = requirements_file_relative_path.read_text() content = content.replace(absolute_path_prefix, "") - for absolute_src_prefix in absolute_src_prefixes: - content = content.replace(absolute_src_prefix, "") requirements_file_relative_path.write_text(content) else: # cli will exit(0) on success @@ -221,15 +214,6 @@ def main( golden = open(_locate(bazel_runfiles, requirements_file)).readlines() out = open(requirements_out).readlines() out = [line.replace(absolute_path_prefix, "") for line in out] - - def replace_via_minus_r(line): - if "# via -r " in line: - for absolute_src_prefix in absolute_src_prefixes: - line = line.replace(absolute_src_prefix, "") - return line - return line - - out = [replace_via_minus_r(line) for line in out] if golden != out: import difflib diff --git a/python/private/pypi/deps.bzl b/python/private/pypi/deps.bzl index 21dd7771fa..31a5201659 100644 --- a/python/private/pypi/deps.bzl +++ b/python/private/pypi/deps.bzl @@ -51,8 +51,8 @@ _RULE_DEPS = [ ), ( "pypi__packaging", - "https://files.pythonhosted.org/packages/88/ef/eb23f262cca3c0c4eb7ab1933c3b1f03d021f2c48f54763065b6f0e321be/packaging-24.2-py3-none-any.whl", - "09abb1bccd265c01f4a3aa3f7a7db064b36514d2cba19a2f694fe6150451a759", + "https://files.pythonhosted.org/packages/49/df/1fceb2f8900f8639e278b056416d49134fb8d84c5942ffaa01ad34782422/packaging-24.0-py3-none-any.whl", + "2ddfb553fdf02fb784c234c7ba6ccc288296ceabec964ad2eae3777778130bc5", ), ( "pypi__pep517", @@ -61,8 +61,8 @@ _RULE_DEPS = [ ), ( "pypi__pip", - "https://files.pythonhosted.org/packages/ef/7d/500c9ad20238fcfcb4cb9243eede163594d7020ce87bd9610c9e02771876/pip-24.3.1-py3-none-any.whl", - "3790624780082365f47549d032f3770eeb2b1e8bd1f7b2e02dace1afa361b4ed", + "https://files.pythonhosted.org/packages/8a/6a/19e9fe04fca059ccf770861c7d5721ab4c2aebc539889e97c7977528a53b/pip-24.0-py3-none-any.whl", + "ba0d021a166865d2265246961bec0152ff124de910c5cc39f1156ce3fa7c69dc", ), ( "pypi__pip_tools", From 466da1d9710289bfb01061b9be7bb124132996e0 Mon Sep 17 00:00:00 2001 From: J Schmidt Date: Tue, 28 Jan 2025 22:53:57 +0100 Subject: [PATCH 069/922] docs: using python_version attribute for specifying python version (#2589) Updates examples and docs to tell to use the base rules and the python_version attribute instead of the wrapper transition rules. --- CHANGELOG.md | 1 + docs/_includes/py_console_script_binary.md | 21 +++++-- docs/toolchains.md | 66 +++++++++++++++++++--- 3 files changed, 73 insertions(+), 15 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 3f8da580a1..cba9a8a8c5 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -62,6 +62,7 @@ Unreleased changes template. {#v0-0-0-fixed} ### Fixed +* (docs) Using `python_version` attribute for specifying python versions introduced in `v1.1.0` * (gazelle) Providing multiple input requirements files to `gazelle_python_manifest` now works correctly. * (pypi) Handle trailing slashes in pip index URLs in environment variables, fixes [#2554](https://github.com/bazelbuild/rules_python/issues/2554). diff --git a/docs/_includes/py_console_script_binary.md b/docs/_includes/py_console_script_binary.md index 7373c8a7b2..aa356e0e94 100644 --- a/docs/_includes/py_console_script_binary.md +++ b/docs/_includes/py_console_script_binary.md @@ -12,7 +12,8 @@ py_console_script_binary( ) ``` -Or for more advanced setups you can also specify extra dependencies and the +#### Specifying extra dependencies +You can also specify extra dependencies and the exact script name you want to call. It is useful for tools like `flake8`, `pylint`, `pytest`, which have plugin discovery methods and discover dependencies from the PyPI packages available in the `PYTHONPATH`. @@ -34,17 +35,26 @@ py_console_script_binary( ) ``` -A specific Python version can be forced by using the generated version-aware -wrappers, e.g. to force Python 3.9: +#### Using a specific Python version + +A specific Python version can be forced by passing the desired Python version, e.g. to force Python 3.9: ```starlark -load("@python_versions//3.9:defs.bzl", "py_console_script_binary") +load("@rules_python//python/entry_points:py_console_script_binary.bzl", "py_console_script_binary") py_console_script_binary( name = "yamllint", pkg = "@pip//yamllint", + python_version = "3.9" ) ``` +#### Using a specific Python Version directly from a Toolchain +:::{deprecated} 1.1.0 +The toolchain specific `py_binary` and `py_test` symbols are aliases to the regular rules. +i.e. Deprecated `load("@python_versions//3.11:defs.bzl", "py_binary")` and `load("@python_versions//3.11:defs.bzl", "py_test")` + +You should instead specify the desired python version with `python_version`; see above example. +::: Alternatively, the [`py_console_script_binary.binary_rule`] arg can be passed the version-bound `py_binary` symbol, or any other `py_binary`-compatible rule of your choosing: @@ -60,5 +70,4 @@ py_console_script_binary( ``` [specification]: https://packaging.python.org/en/latest/specifications/entry-points/ -[`py_console_script_binary.binary_rule`]: #py_console_script_binary_binary_rule - +[`py_console_script_binary.binary_rule`]: #py_console_script_binary_binary_rule \ No newline at end of file diff --git a/docs/toolchains.md b/docs/toolchains.md index 32f4a541d9..6eaa244b1f 100644 --- a/docs/toolchains.md +++ b/docs/toolchains.md @@ -116,9 +116,9 @@ python = use_extension("@rules_python//python/extensions:python.bzl", "python") python.toolchain(python_version = "3.12") # BUILD.bazel -load("@python_versions//3.12:defs.bzl", "py_binary") +load("@rules_python//python:py_binary.bzl", "py_binary") -py_binary(...) +py_binary(..., python_version="3.12") ``` ### Pinning to a Python version @@ -132,21 +132,59 @@ is most useful for two cases: typically in a mono-repo situation. To configure a submodule with the version-aware rules, request the particular -version you need, then use the `@python_versions` repo to use the rules that -force specific versions: +version you need when defining the toolchain: ```starlark +# MODULE.bazel python = use_extension("@rules_python//python/extensions:python.bzl", "python") python.toolchain( python_version = "3.11", ) -use_repo(python, "python_versions") +use_repo(python) +``` + +Then use the `@rules_python` repo in your BUILD file to explicity pin the Python version when calling the rule: + +```starlark +# BUILD.bazel +load("@rules_python//python:py_binary.bzl", "py_binary") + +py_binary(..., python_version = "3.11") +py_test(..., python_version = "3.11") ``` -Then use e.g. `load("@python_versions//3.11:defs.bzl", "py_binary")` to use -the rules that force that particular version. Multiple versions can be specified -and use within a single build. +Multiple versions can be specified and used within a single build. + +```starlark +# MODULE.bazel +python = use_extension("@rules_python//python/extensions:python.bzl", "python") + +python.toolchain( + python_version = "3.11", + is_default = True, +) + +python.toolchain( + python_version = "3.12", +) + +# BUILD.bazel +load("@rules_python//python:py_binary.bzl", "py_binary") +load("@rules_python//python:py_test.bzl", "py_test") + +# Defaults to 3.11 +py_binary(...) +py_test(...) + +# Explicitly use Python 3.11 +py_binary(..., python_version = "3.11") +py_test(..., python_version = "3.11") + +# Explicitly use Python 3.12 +py_binary(..., python_version = "3.12") +py_test(..., python_version = "3.12") +``` For more documentation, see the bzlmod examples under the {gh-path}`examples` folder. Look for the examples that contain a `MODULE.bazel` file. @@ -159,6 +197,16 @@ The `python.toolchain()` call makes its contents available under a repo named Remember to call `use_repo()` to make repos visible to your module: `use_repo(python, "python_3_11")` + +:::{deprecated} 1.1.0 +The toolchain specific `py_binary` and `py_test` symbols are aliases to the regular rules. +i.e. Deprecated `load("@python_versions//3.11:defs.bzl", "py_binary")` & `load("@python_versions//3.11:defs.bzl", "py_test")` + +Usages of them should be changed to load the regular rules directly; +i.e. Use `load("@rules_python//python:py_binary.bzl", "py_binary")` & `load("@rules_python//python:py_test.bzl", "py_test")` and then specify the `python_version` when using the rules corresponding to the python version you defined in your toolchain. {ref}`Library modules with version constraints` +::: + + #### Toolchain usage in other rules Python toolchains can be utilized in other bazel rules, such as `genrule()`, by @@ -508,4 +556,4 @@ of available toolchains. Currently the following flags are used to influence toolchain selection: * {obj}`--@rules_python//python/config_settings:py_linux_libc` for selecting the Linux libc variant. * {obj}`--@rules_python//python/config_settings:py_freethreaded` for selecting - the freethreaded experimental Python builds available from `3.13.0` onwards. + the freethreaded experimental Python builds available from `3.13.0` onwards. \ No newline at end of file From 33cb431c2f87b2bcf8211745ba36da218b2f03bd Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Sat, 1 Feb 2025 17:50:58 -0800 Subject: [PATCH 070/922] fix: make plain zipapp work with bootstrap=script (#2598) The `__main__.py` template (zip_main_template.py) was using the wrong path when creating the interpreter symlinks. It as computing it correctly, just the wrong variable was used in the symlink() call. To fix, pass the correct variable. Also adds a test to check that it's runnable. Fixes https://github.com/bazelbuild/rules_python/issues/2596 --- CHANGELOG.md | 2 + python/private/zip_main_template.py | 4 +- tests/bootstrap_impls/BUILD.bazel | 34 +++++++++++++- .../bootstrap_script_zipapp_test.sh | 47 +++++++++++++++++++ tests/support/sh_py_run_test.bzl | 36 ++++++++++---- 5 files changed, 111 insertions(+), 12 deletions(-) create mode 100755 tests/bootstrap_impls/bootstrap_script_zipapp_test.sh diff --git a/CHANGELOG.md b/CHANGELOG.md index cba9a8a8c5..82aeda8117 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -62,6 +62,8 @@ Unreleased changes template. {#v0-0-0-fixed} ### Fixed +* (rules) `python_zip_file` output with `--bootstrap_impl=script` works again + ([#2596](https://github.com/bazelbuild/rules_python/issues/2596)). * (docs) Using `python_version` attribute for specifying python versions introduced in `v1.1.0` * (gazelle) Providing multiple input requirements files to `gazelle_python_manifest` now works correctly. * (pypi) Handle trailing slashes in pip index URLs in environment variables, diff --git a/python/private/zip_main_template.py b/python/private/zip_main_template.py index b4c9d279a6..5ec5ba07fa 100644 --- a/python/private/zip_main_template.py +++ b/python/private/zip_main_template.py @@ -286,10 +286,10 @@ def main(): # The bin/ directory may not exist if it is empty. os.makedirs(os.path.dirname(python_program), exist_ok=True) try: - os.symlink(_PYTHON_BINARY_ACTUAL, python_program) + os.symlink(symlink_to, python_program) except OSError as e: raise Exception( - f"Unable to create venv python interpreter symlink: {python_program} -> {PYTHON_BINARY_ACTUAL}" + f"Unable to create venv python interpreter symlink: {python_program} -> {symlink_to}" ) from e # Some older Python versions on macOS (namely Python 3.7) may unintentionally diff --git a/tests/bootstrap_impls/BUILD.bazel b/tests/bootstrap_impls/BUILD.bazel index 8e50f34cfa..3df72a10ba 100644 --- a/tests/bootstrap_impls/BUILD.bazel +++ b/tests/bootstrap_impls/BUILD.bazel @@ -1,3 +1,5 @@ +load("@rules_shell//shell:sh_test.bzl", "sh_test") + # Copyright 2023 The Bazel Authors. All rights reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); @@ -11,10 +13,40 @@ # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # See the License for the specific language governing permissions and # limitations under the License. -load("//tests/support:sh_py_run_test.bzl", "py_reconfig_test", "sh_py_run_test") +load("//tests/support:sh_py_run_test.bzl", "py_reconfig_binary", "py_reconfig_test", "sh_py_run_test") load("//tests/support:support.bzl", "SUPPORTS_BOOTSTRAP_SCRIPT") load(":venv_relative_path_tests.bzl", "relative_path_test_suite") +py_reconfig_binary( + name = "bootstrap_script_zipapp_bin", + srcs = ["bin.py"], + bootstrap_impl = "script", + # Force it to not be self-executable + build_python_zip = "no", + main = "bin.py", + target_compatible_with = SUPPORTS_BOOTSTRAP_SCRIPT, +) + +filegroup( + name = "bootstrap_script_zipapp_zip", + testonly = 1, + srcs = [":bootstrap_script_zipapp_bin"], + output_group = "python_zip_file", +) + +sh_test( + name = "bootstrap_script_zipapp_test", + srcs = ["bootstrap_script_zipapp_test.sh"], + data = [":bootstrap_script_zipapp_zip"], + env = { + "ZIP_RLOCATION": "$(rlocationpaths :bootstrap_script_zipapp_zip)".format(), + }, + target_compatible_with = SUPPORTS_BOOTSTRAP_SCRIPT, + deps = [ + "@bazel_tools//tools/bash/runfiles", + ], +) + sh_py_run_test( name = "run_binary_zip_no_test", build_python_zip = "no", diff --git a/tests/bootstrap_impls/bootstrap_script_zipapp_test.sh b/tests/bootstrap_impls/bootstrap_script_zipapp_test.sh new file mode 100755 index 0000000000..558ca970d6 --- /dev/null +++ b/tests/bootstrap_impls/bootstrap_script_zipapp_test.sh @@ -0,0 +1,47 @@ +# Copyright 2024 The Bazel Authors. All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +# --- begin runfiles.bash initialization v3 --- +# Copy-pasted from the Bazel Bash runfiles library v3. +set -uo pipefail; set +e; f=bazel_tools/tools/bash/runfiles/runfiles.bash +source "${RUNFILES_DIR:-/dev/null}/$f" 2>/dev/null || \ + source "$(grep -sm1 "^$f " "${RUNFILES_MANIFEST_FILE:-/dev/null}" | cut -f2- -d' ')" 2>/dev/null || \ + source "$0.runfiles/$f" 2>/dev/null || \ + source "$(grep -sm1 "^$f " "$0.runfiles_manifest" | cut -f2- -d' ')" 2>/dev/null || \ + source "$(grep -sm1 "^$f " "$0.exe.runfiles_manifest" | cut -f2- -d' ')" 2>/dev/null || \ + { echo>&2 "ERROR: cannot find $f"; exit 1; }; f=; set -e +# --- end runfiles.bash initialization v3 --- +set +e + +bin=$(rlocation $ZIP_RLOCATION) +if [[ -z "$bin" ]]; then + echo "Unable to locate test binary: $ZIP_RLOCATION" + exit 1 +fi +set -x +actual=$(python3 $bin) + +# How we detect if a zip file was executed from depends on which bootstrap +# is used. +# bootstrap_impl=script outputs RULES_PYTHON_ZIP_DIR= +# bootstrap_impl=system_python outputs file:.*Bazel.runfiles +expected_pattern="Hello" +if ! (echo "$actual" | grep "$expected_pattern" ) >/dev/null; then + echo "Test case failed: $1" + echo "expected output to match: $expected_pattern" + echo "but got:\n$actual" + exit 1 +fi + +exit 0 diff --git a/tests/support/sh_py_run_test.bzl b/tests/support/sh_py_run_test.bzl index 9bf0a7402e..a76d2a335b 100644 --- a/tests/support/sh_py_run_test.bzl +++ b/tests/support/sh_py_run_test.bzl @@ -86,6 +86,7 @@ def _py_reconfig_impl(ctx): default_info.default_runfiles, ), ), + ctx.attr.target[OutputGroupInfo], # Inherit the expanded environment from the inner target. ctx.attr.target[RunEnvironmentInfo], ] @@ -120,31 +121,48 @@ _py_reconfig_binary = _make_reconfig_rule(executable = True) _py_reconfig_test = _make_reconfig_rule(test = True) -def py_reconfig_test(*, name, **kwargs): - """Create a py_test with customized build settings for testing. - - Args: - name: str, name of teset target. - **kwargs: kwargs to pass along to _py_reconfig_test and py_test. - """ +def _py_reconfig_executable(*, name, py_reconfig_rule, py_inner_rule, **kwargs): reconfig_kwargs = {} reconfig_kwargs["bootstrap_impl"] = kwargs.pop("bootstrap_impl", None) reconfig_kwargs["extra_toolchains"] = kwargs.pop("extra_toolchains", None) reconfig_kwargs["python_version"] = kwargs.pop("python_version", None) reconfig_kwargs["target_compatible_with"] = kwargs.get("target_compatible_with") + reconfig_kwargs["build_python_zip"] = kwargs.pop("build_python_zip", None) inner_name = "_{}_inner".format(name) - _py_reconfig_test( + py_reconfig_rule( name = name, target = inner_name, **reconfig_kwargs ) - py_test( + py_inner_rule( name = inner_name, tags = ["manual"], **kwargs ) +def py_reconfig_test(*, name, **kwargs): + """Create a py_test with customized build settings for testing. + + Args: + name: str, name of teset target. + **kwargs: kwargs to pass along to _py_reconfig_test and py_test. + """ + _py_reconfig_executable( + name = name, + py_reconfig_rule = _py_reconfig_test, + py_inner_rule = py_test, + **kwargs + ) + +def py_reconfig_binary(*, name, **kwargs): + _py_reconfig_executable( + name = name, + py_reconfig_rule = _py_reconfig_binary, + py_inner_rule = py_binary, + **kwargs + ) + def sh_py_run_test(*, name, sh_src, py_src, **kwargs): """Run a py_binary within a sh_test. From 2e6f8ad5fe4dd0cc81550dd533692638e8cffe52 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Mon, 3 Feb 2025 04:31:34 -0800 Subject: [PATCH 071/922] fix: add flag to use runtime venv creation when using bootstrap=script (#2590) The bootstrap=script implementation was changed to use declare_symlink() to create explicit symlinks so its venv works. Unfortunately, this broke packaging rules, which would treat the symlinks as regular files. To fix, introduce a flag that stops using declare_symlink() and instead creates the venv at runtime. Creating a venv at runtime is problematic for various reasons, but this should work well enough until packaging rules are able to handle these raw symlinks. The location of the venv can be somewhat controlled by setting the `RULES_PYTHON_VENVS_ROOT` environment variable. This is to better accommodate cases where using /tmp is problematic. Along the way, sort the environment variable docs by their name. Fixes https://github.com/bazelbuild/rules_python/issues/2489 --- CHANGELOG.md | 4 + MODULE.bazel | 1 + .../python/config_settings/index.md | 24 +++++ docs/environment-variables.md | 89 ++++++++++++------- python/config_settings/BUILD.bazel | 8 ++ python/private/flags.bzl | 15 ++++ python/private/py_executable.bzl | 33 +++++-- python/private/stage1_bootstrap_template.sh | 64 +++++++++++-- tests/bootstrap_impls/BUILD.bazel | 9 ++ tests/bootstrap_impls/bin.py | 1 + ...inary_venvs_use_declare_symlink_no_test.sh | 56 ++++++++++++ tests/packaging/BUILD.bazel | 44 +++++++++ tests/packaging/bin.py | 1 + tests/support/sh_py_run_test.bzl | 22 +++-- 14 files changed, 320 insertions(+), 51 deletions(-) create mode 100755 tests/bootstrap_impls/run_binary_venvs_use_declare_symlink_no_test.sh create mode 100644 tests/packaging/BUILD.bazel create mode 100644 tests/packaging/bin.py diff --git a/CHANGELOG.md b/CHANGELOG.md index 82aeda8117..61000a1b08 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -77,6 +77,10 @@ Unreleased changes template. The related issue is [#908](https://github.com/bazelbuild/rules_python/issue/908). * (sphinxdocs) Do not crash when `tag_class` does not have a populated `doc` value. Fixes ([#2579](https://github.com/bazelbuild/rules_python/issues/2579)). +* (binaries/tests) Fix packaging when using `--bootstrap_impl=script`: set + {obj}`--venvs_use_declare_symlink=no` to have it not create symlinks at + build time (they will be created at runtime instead). + (Fixes [#2489](https://github.com/bazelbuild/rules_python/issues/2489)) {#v0-0-0-added} ### Added diff --git a/MODULE.bazel b/MODULE.bazel index 7034357f61..89f1cd7961 100644 --- a/MODULE.bazel +++ b/MODULE.bazel @@ -84,6 +84,7 @@ bazel_dep(name = "rules_testing", version = "0.6.0", dev_dependency = True) bazel_dep(name = "rules_shell", version = "0.3.0", dev_dependency = True) bazel_dep(name = "rules_multirun", version = "0.9.0", dev_dependency = True) bazel_dep(name = "bazel_ci_rules", version = "1.0.0", dev_dependency = True) +bazel_dep(name = "rules_pkg", version = "1.0.1", dev_dependency = True) # Extra gazelle plugin deps so that WORKSPACE.bzlmod can continue including it for e2e tests. # We use `WORKSPACE.bzlmod` because it is impossible to have dev-only local overrides. diff --git a/docs/api/rules_python/python/config_settings/index.md b/docs/api/rules_python/python/config_settings/index.md index 793f6e08fd..b2163233ca 100644 --- a/docs/api/rules_python/python/config_settings/index.md +++ b/docs/api/rules_python/python/config_settings/index.md @@ -212,6 +212,7 @@ Values: ::: :::: + ::::{bzl:flag} bootstrap_impl Determine how programs implement their startup process. @@ -258,3 +259,26 @@ Values: ::: :::: + +::::{bzl:flag} venvs_use_declare_symlink + +Determines if relative symlinks are created using `declare_symlink()` at build +time. + +This is only intended to work around +[#2489](https://github.com/bazelbuild/rules_python/issues/2489), where some +packaging rules don't support `declare_symlink()` artifacts. + +Values: +* `yes`: Use `declare_symlink()` and create relative symlinks at build time. +* `no`: Do not use `declare_symlink()`. Instead, the venv will be created at + runtime. + +:::{seealso} +{envvar}`RULES_PYTHON_EXTRACT_ROOT` for customizing where the runtime venv +is created. +::: + +:::{versionadded} VERSION_NEXT_PATCH +::: +:::: diff --git a/docs/environment-variables.md b/docs/environment-variables.md index fb9971597b..dd4a700081 100644 --- a/docs/environment-variables.md +++ b/docs/environment-variables.md @@ -1,28 +1,56 @@ # Environment Variables -:::{envvar} RULES_PYTHON_REPO_DEBUG +:::{envvar} RULES_PYTHON_BOOTSTRAP_VERBOSE -When `1`, repository rules will print debug information about what they're +When `1`, debug information about bootstrapping of a program is printed to +stderr. +::: + +:::{envvar} RULES_PYTHON_BZLMOD_DEBUG + +When `1`, bzlmod extensions will print debug information about what they're doing. This is mostly useful for development to debug errors. ::: -:::{envvar} RULES_PYTHON_REPO_DEBUG_VERBOSITY +:::{envvar} RULES_PYTHON_DEPRECATION_WARNINGS -Determines the verbosity of logging output for repo rules. Valid values: +When `1`, the rules_python will warn users about deprecated functionality that will +be removed in a subsequent major `rules_python` version. Defaults to `0` if unset. +::: -* `DEBUG` -* `INFO` -* `TRACE` +:::{envvar} RULES_PYTHON_ENABLE_PYSTAR + +When `1`, the rules_python Starlark implementation of the core rules is used +instead of the Bazel-builtin rules. Note this requires Bazel 7+. ::: -:::{envvar} RULES_PYTHON_REPO_TOOLCHAIN_VERSION_OS_ARCH +::::{envvar} RULES_PYTHON_EXTRACT_ROOT -Determines the python interpreter platform to be used for a particular -interpreter `(version, os, arch)` triple to be used in repository rules. -Replace the `VERSION_OS_ARCH` part with actual values when using, e.g. -`3_13_0_linux_x86_64`. The version values must have `_` instead of `.` and the -os, arch values are the same as the ones mentioned in the -`//python:versions.bzl` file. +Directory to use as the root for creating files necessary for bootstrapping so +that a binary can run. + +Only applicable when {bzl:flag}`--venvs_use_declare_symlink=no` is used. + +When set, a binary will attempt to find a unique, reusable, location within this +directory for the files it needs to create to aid startup. The files may not be +deleted upon program exit; it is the responsibility of the caller to ensure +cleanup. + +Manually specifying the directory is useful to lower the overhead of +extracting/creating files on every program execution. By using a location +outside /tmp, longer lived programs don't have to worry about files in /tmp +being cleaned up by the OS. + +If not set, then a temporary directory will be created and deleted upon program +exit. + +:::{versionadded} VERSION_NEXT_PATCH +::: +:::: + +:::{envvar} RULES_PYTHON_GAZELLE_VERBOSE + +When `1`, debug information from gazelle is printed to stderr. ::: :::{envvar} RULES_PYTHON_PIP_ISOLATED @@ -34,37 +62,32 @@ Valid values: * Other non-empty values mean to use isolated mode. ::: -:::{envvar} RULES_PYTHON_BZLMOD_DEBUG +:::{envvar} RULES_PYTHON_REPO_DEBUG -When `1`, bzlmod extensions will print debug information about what they're +When `1`, repository rules will print debug information about what they're doing. This is mostly useful for development to debug errors. ::: -:::{envvar} RULES_PYTHON_DEPRECATION_WARNINGS - -When `1`, the rules_python will warn users about deprecated functionality that will -be removed in a subsequent major `rules_python` version. Defaults to `0` if unset. -::: +:::{envvar} RULES_PYTHON_REPO_DEBUG_VERBOSITY -:::{envvar} RULES_PYTHON_ENABLE_PYSTAR +Determines the verbosity of logging output for repo rules. Valid values: -When `1`, the rules_python Starlark implementation of the core rules is used -instead of the Bazel-builtin rules. Note this requires Bazel 7+. +* `DEBUG` +* `INFO` +* `TRACE` ::: -:::{envvar} RULES_PYTHON_BOOTSTRAP_VERBOSE +:::{envvar} RULES_PYTHON_REPO_TOOLCHAIN_VERSION_OS_ARCH -When `1`, debug information about bootstrapping of a program is printed to -stderr. +Determines the python interpreter platform to be used for a particular +interpreter `(version, os, arch)` triple to be used in repository rules. +Replace the `VERSION_OS_ARCH` part with actual values when using, e.g. +`3_13_0_linux_x86_64`. The version values must have `_` instead of `.` and the +os, arch values are the same as the ones mentioned in the +`//python:versions.bzl` file. ::: :::{envvar} VERBOSE_COVERAGE When `1`, debug information about coverage behavior is printed to stderr. ::: - - -:::{envvar} RULES_PYTHON_GAZELLE_VERBOSE - -When `1`, debug information from gazelle is printed to stderr. -::: diff --git a/python/config_settings/BUILD.bazel b/python/config_settings/BUILD.bazel index fcebcd76dc..796cf0c9c4 100644 --- a/python/config_settings/BUILD.bazel +++ b/python/config_settings/BUILD.bazel @@ -9,6 +9,7 @@ load( "LibcFlag", "PrecompileFlag", "PrecompileSourceRetentionFlag", + "VenvsUseDeclareSymlinkFlag", ) load( "//python/private/pypi:flags.bzl", @@ -121,6 +122,13 @@ config_setting( visibility = ["//visibility:public"], ) +string_flag( + name = "venvs_use_declare_symlink", + build_setting_default = VenvsUseDeclareSymlinkFlag.YES, + values = VenvsUseDeclareSymlinkFlag.flag_values(), + visibility = ["//visibility:public"], +) + # pip.parse related flags string_flag( diff --git a/python/private/flags.bzl b/python/private/flags.bzl index 9070f113ac..1019faa8d6 100644 --- a/python/private/flags.bzl +++ b/python/private/flags.bzl @@ -123,6 +123,21 @@ PrecompileSourceRetentionFlag = enum( get_effective_value = _precompile_source_retention_flag_get_effective_value, ) +def _venvs_use_declare_symlink_flag_get_value(ctx): + return ctx.attr._venvs_use_declare_symlink_flag[BuildSettingInfo].value + +# Decides if the venv created by bootstrap=script uses declare_file() to +# create relative symlinks. Workaround for #2489 (packaging rules not supporting +# declare_link() files). +# buildifier: disable=name-conventions +VenvsUseDeclareSymlinkFlag = FlagEnum( + # Use declare_file() and relative symlinks in the venv + YES = "yes", + # Do not use declare_file() and relative symlinks in the venv + NO = "no", + get_value = _venvs_use_declare_symlink_flag_get_value, +) + # Used for matching freethreaded toolchains and would have to be used in wheels # as well. # buildifier: disable=name-conventions diff --git a/python/private/py_executable.bzl b/python/private/py_executable.bzl index 1e437f57e1..18a7a707fc 100644 --- a/python/private/py_executable.bzl +++ b/python/private/py_executable.bzl @@ -51,7 +51,7 @@ load( "target_platform_has_any_constraint", "union_attrs", ) -load(":flags.bzl", "BootstrapImplFlag") +load(":flags.bzl", "BootstrapImplFlag", "VenvsUseDeclareSymlinkFlag") load(":precompile.bzl", "maybe_precompile") load(":py_cc_link_params_info.bzl", "PyCcLinkParamsInfo") load(":py_executable_info.bzl", "PyExecutableInfo") @@ -195,6 +195,10 @@ accepting arbitrary Python versions. "_python_version_flag": attr.label( default = "//python/config_settings:python_version", ), + "_venvs_use_declare_symlink_flag": attr.label( + default = "//python/config_settings:venvs_use_declare_symlink", + providers = [BuildSettingInfo], + ), "_windows_constraints": attr.label_list( default = [ "@platforms//os:windows", @@ -512,7 +516,25 @@ def _create_venv(ctx, output_prefix, imports, runtime_details): ctx.actions.write(pyvenv_cfg, "") runtime = runtime_details.effective_runtime - if runtime.interpreter: + venvs_use_declare_symlink_enabled = ( + VenvsUseDeclareSymlinkFlag.get_value(ctx) == VenvsUseDeclareSymlinkFlag.YES + ) + + if not venvs_use_declare_symlink_enabled: + if runtime.interpreter: + interpreter_actual_path = _runfiles_root_path(ctx, runtime.interpreter.short_path) + else: + interpreter_actual_path = runtime.interpreter_path + + py_exe_basename = paths.basename(interpreter_actual_path) + + # When the venv symlinks are disabled, the $venv/bin/python3 file isn't + # needed or used at runtime. However, the zip code uses the interpreter + # File object to figure out some paths. + interpreter = ctx.actions.declare_file("{}/bin/{}".format(venv, py_exe_basename)) + ctx.actions.write(interpreter, "actual:{}".format(interpreter_actual_path)) + + elif runtime.interpreter: py_exe_basename = paths.basename(runtime.interpreter.short_path) # Even though ctx.actions.symlink() is used, using @@ -571,6 +593,7 @@ def _create_venv(ctx, output_prefix, imports, runtime_details): return struct( interpreter = interpreter, + recreate_venv_at_runtime = not venvs_use_declare_symlink_enabled, # Runfiles root relative path or absolute path interpreter_actual_path = interpreter_actual_path, files_without_interpreter = [pyvenv_cfg, pth, site_init], @@ -657,15 +680,13 @@ def _create_stage1_bootstrap( else: python_binary_path = runtime_details.executable_interpreter_path - if is_for_zip and venv: - python_binary_actual = venv.interpreter_actual_path - else: - python_binary_actual = "" + python_binary_actual = venv.interpreter_actual_path if venv else "" subs = { "%is_zipfile%": "1" if is_for_zip else "0", "%python_binary%": python_binary_path, "%python_binary_actual%": python_binary_actual, + "%recreate_venv_at_runtime%": str(int(venv.recreate_venv_at_runtime)) if venv else "0", "%target%": str(ctx.label), "%workspace_name%": ctx.workspace_name, } diff --git a/python/private/stage1_bootstrap_template.sh b/python/private/stage1_bootstrap_template.sh index b05b4a54cd..19ff763094 100644 --- a/python/private/stage1_bootstrap_template.sh +++ b/python/private/stage1_bootstrap_template.sh @@ -9,15 +9,17 @@ fi # runfiles-relative path STAGE2_BOOTSTRAP="%stage2_bootstrap%" -# runfiles-relative path +# runfiles-relative path to python interpreter to use PYTHON_BINARY='%python_binary%' # The path that PYTHON_BINARY should symlink to. # runfiles-relative path, absolute path, or single word. -# Only applicable for zip files. +# Only applicable for zip files or when venv is recreated at runtime. PYTHON_BINARY_ACTUAL="%python_binary_actual%" # 0 or 1 IS_ZIPFILE="%is_zipfile%" +# 0 or 1 +RECREATE_VENV_AT_RUNTIME="%recreate_venv_at_runtime%" if [[ "$IS_ZIPFILE" == "1" ]]; then # NOTE: Macs have an old version of mktemp, so we must use only the @@ -104,6 +106,7 @@ python_exe=$(find_python_interpreter $RUNFILES_DIR $PYTHON_BINARY) # Zip files have to re-create the venv bin/python3 symlink because they # don't contain it already. if [[ "$IS_ZIPFILE" == "1" ]]; then + use_exec=0 # It should always be under runfiles, but double check this. We don't # want to accidentally create symlinks elsewhere. if [[ "$python_exe" != $RUNFILES_DIR/* ]]; then @@ -121,13 +124,60 @@ if [[ "$IS_ZIPFILE" == "1" ]]; then symlink_to=$(which $PYTHON_BINARY_ACTUAL) # Guard against trying to symlink to an empty value if [[ $? -ne 0 ]]; then - echo >&2 "ERROR: Python to use found on PATH: $PYTHON_BINARY_ACTUAL" + echo >&2 "ERROR: Python to use not found on PATH: $PYTHON_BINARY_ACTUAL" exit 1 fi fi # The bin/ directory may not exist if it is empty. mkdir -p "$(dirname $python_exe)" ln -s "$symlink_to" "$python_exe" +elif [[ "$RECREATE_VENV_AT_RUNTIME" == "1" ]]; then + if [[ -n "$RULES_PYTHON_EXTRACT_ROOT" ]]; then + use_exec=1 + # Use our runfiles path as a unique, reusable, location for the + # binary-specific venv being created. + venv="$RULES_PYTHON_EXTRACT_ROOT/$(dirname $(dirname $PYTHON_BINARY))" + mkdir -p $RULES_PYTHON_EXTRACT_ROOT + else + # Re-exec'ing can't be used because we have to clean up the temporary + # venv directory that is created. + use_exec=0 + venv=$(mktemp -d) + if [[ -n "$venv" && -z "${RULES_PYTHON_BOOTSTRAP_VERBOSE:-}" ]]; then + trap 'rm -fr "$venv"' EXIT + fi + fi + + if [[ "$PYTHON_BINARY_ACTUAL" == /* ]]; then + # An absolute path, i.e. platform runtime, e.g. /usr/bin/python3 + symlink_to=$PYTHON_BINARY_ACTUAL + elif [[ "$PYTHON_BINARY_ACTUAL" == */* ]]; then + # A runfiles-relative path + symlink_to="$RUNFILES_DIR/$PYTHON_BINARY_ACTUAL" + else + # A plain word, e.g. "python3". Symlink to where PATH leads + symlink_to=$(which $PYTHON_BINARY_ACTUAL) + # Guard against trying to symlink to an empty value + if [[ $? -ne 0 ]]; then + echo >&2 "ERROR: Python to use not found on PATH: $PYTHON_BINARY_ACTUAL" + exit 1 + fi + fi + mkdir -p "$venv/bin" + # Match the basename; some tools, e.g. pyvenv key off the executable name + python_exe="$venv/bin/$(basename $PYTHON_BINARY_ACTUAL)" + if [[ ! -e "$python_exe" ]]; then + ln -s "$symlink_to" "$python_exe" + fi + runfiles_venv="$RUNFILES_DIR/$(dirname $(dirname $PYTHON_BINARY))" + if [[ ! -e "$venv/pyvenv.cfg" ]]; then + ln -s "$runfiles_venv/pyvenv.cfg" "$venv/pyvenv.cfg" + fi + if [[ ! -e "$venv/lib" ]]; then + ln -s "$runfiles_venv/lib" "$venv/lib" + fi +else + use_exec=1 fi # At this point, we should have a valid reference to the interpreter. @@ -165,7 +215,6 @@ if [[ "$IS_ZIPFILE" == "1" ]]; then interpreter_args+=("-XRULES_PYTHON_ZIP_DIR=$zip_dir") fi - export RUNFILES_DIR command=( @@ -184,9 +233,10 @@ command=( # See https://github.com/bazelbuild/rules_python/issues/2043#issuecomment-2215469971 # for more information. # -# However, when running a zip file, we need to clean up the workspace after the -# process finishes so control must return here. -if [[ "$IS_ZIPFILE" == "1" ]]; then +# However, we can't use exec when there is cleanup to do afterwards. Control +# must return to this process so it can run the trap handlers. Such cases +# occur when zip mode or recreate_venv_at_runtime creates temporary files. +if [[ "$use_exec" == "0" ]]; then "${command[@]}" exit $? else diff --git a/tests/bootstrap_impls/BUILD.bazel b/tests/bootstrap_impls/BUILD.bazel index 3df72a10ba..8a64bf2b5b 100644 --- a/tests/bootstrap_impls/BUILD.bazel +++ b/tests/bootstrap_impls/BUILD.bazel @@ -61,6 +61,15 @@ sh_py_run_test( sh_src = "run_binary_zip_yes_test.sh", ) +sh_py_run_test( + name = "run_binary_venvs_use_declare_symlink_no_test", + bootstrap_impl = "script", + py_src = "bin.py", + sh_src = "run_binary_venvs_use_declare_symlink_no_test.sh", + target_compatible_with = SUPPORTS_BOOTSTRAP_SCRIPT, + venvs_use_declare_symlink = "no", +) + sh_py_run_test( name = "run_binary_bootstrap_script_zip_yes_test", bootstrap_impl = "script", diff --git a/tests/bootstrap_impls/bin.py b/tests/bootstrap_impls/bin.py index c46e43adc8..1176107384 100644 --- a/tests/bootstrap_impls/bin.py +++ b/tests/bootstrap_impls/bin.py @@ -22,3 +22,4 @@ print("PYTHONSAFEPATH:", os.environ.get("PYTHONSAFEPATH", "UNSET") or "EMPTY") print("sys.flags.safe_path:", sys.flags.safe_path) print("file:", __file__) +print("sys.executable:", sys.executable) diff --git a/tests/bootstrap_impls/run_binary_venvs_use_declare_symlink_no_test.sh b/tests/bootstrap_impls/run_binary_venvs_use_declare_symlink_no_test.sh new file mode 100755 index 0000000000..d4840116f9 --- /dev/null +++ b/tests/bootstrap_impls/run_binary_venvs_use_declare_symlink_no_test.sh @@ -0,0 +1,56 @@ +# Copyright 2024 The Bazel Authors. All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +# --- begin runfiles.bash initialization v3 --- +# Copy-pasted from the Bazel Bash runfiles library v3. +set -uo pipefail; set +e; f=bazel_tools/tools/bash/runfiles/runfiles.bash +source "${RUNFILES_DIR:-/dev/null}/$f" 2>/dev/null || \ + source "$(grep -sm1 "^$f " "${RUNFILES_MANIFEST_FILE:-/dev/null}" | cut -f2- -d' ')" 2>/dev/null || \ + source "$0.runfiles/$f" 2>/dev/null || \ + source "$(grep -sm1 "^$f " "$0.runfiles_manifest" | cut -f2- -d' ')" 2>/dev/null || \ + source "$(grep -sm1 "^$f " "$0.exe.runfiles_manifest" | cut -f2- -d' ')" 2>/dev/null || \ + { echo>&2 "ERROR: cannot find $f"; exit 1; }; f=; set -e +# --- end runfiles.bash initialization v3 --- +set +e + +bin=$(rlocation $BIN_RLOCATION) +if [[ -z "$bin" ]]; then + echo "Unable to locate test binary: $BIN_RLOCATION" + exit 1 +fi +actual=$($bin) + +function expect_match() { + local expected_pattern=$1 + local actual=$2 + if ! (echo "$actual" | grep "$expected_pattern" ) >/dev/null; then + echo "expected to match: $expected_pattern" + echo "===== actual START =====" + echo "$actual" + echo "===== actual END =====" + echo + touch EXPECTATION_FAILED + return 1 + fi +} + +expect_match "sys.executable:.*tmp.*python3" "$actual" + +# Now test that using a custom location for the bootstrap files works +venvs_root=$(mktemp -d) +actual=$(RULES_PYTHON_EXTRACT_ROOT=$venvs_root $bin) +expect_match "sys.executable:.*$venvs_root" "$actual" + +# Exit if any of the expects failed +[[ ! -e EXPECTATION_FAILED ]] diff --git a/tests/packaging/BUILD.bazel b/tests/packaging/BUILD.bazel new file mode 100644 index 0000000000..cc04c05ba9 --- /dev/null +++ b/tests/packaging/BUILD.bazel @@ -0,0 +1,44 @@ +# Copyright 2025 The Bazel Authors. All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +load("@bazel_skylib//rules:build_test.bzl", "build_test") +load("@rules_pkg//pkg:tar.bzl", "pkg_tar") +load("//tests/support:sh_py_run_test.bzl", "py_reconfig_test") +load("//tests/support:support.bzl", "SUPPORTS_BOOTSTRAP_SCRIPT") + +build_test( + name = "bzl_libraries_build_test", + targets = [ + # keep sorted + ":bin_tar", + ], +) + +py_reconfig_test( + name = "bin", + srcs = ["bin.py"], + bootstrap_impl = "script", + main = "bin.py", + target_compatible_with = SUPPORTS_BOOTSTRAP_SCRIPT, + # Needed until https://github.com/bazelbuild/rules_pkg/issues/929 is fixed + # See: https://github.com/bazelbuild/rules_python/issues/2489 + venvs_use_declare_symlink = "no", +) + +pkg_tar( + name = "bin_tar", + testonly = True, + srcs = [":bin"], + include_runfiles = True, +) diff --git a/tests/packaging/bin.py b/tests/packaging/bin.py new file mode 100644 index 0000000000..2f9a147db1 --- /dev/null +++ b/tests/packaging/bin.py @@ -0,0 +1 @@ +print("Hello") diff --git a/tests/support/sh_py_run_test.bzl b/tests/support/sh_py_run_test.bzl index a76d2a335b..4fa53ebd66 100644 --- a/tests/support/sh_py_run_test.bzl +++ b/tests/support/sh_py_run_test.bzl @@ -33,6 +33,8 @@ def _perform_transition_impl(input_settings, attr): settings["//command_line_option:extra_toolchains"] = attr.extra_toolchains if attr.python_version: settings["//python/config_settings:python_version"] = attr.python_version + if attr.venvs_use_declare_symlink: + settings["//python/config_settings:venvs_use_declare_symlink"] = attr.venvs_use_declare_symlink return settings _perform_transition = transition( @@ -41,12 +43,14 @@ _perform_transition = transition( "//python/config_settings:bootstrap_impl", "//command_line_option:extra_toolchains", "//python/config_settings:python_version", + "//python/config_settings:venvs_use_declare_symlink", ], outputs = [ "//command_line_option:build_python_zip", "//command_line_option:extra_toolchains", "//python/config_settings:bootstrap_impl", "//python/config_settings:python_version", + "//python/config_settings:venvs_use_declare_symlink", VISIBLE_FOR_TESTING, ], ) @@ -106,6 +110,7 @@ toolchain. ), "python_version": attr.string(), "target": attr.label(executable = True, cfg = "target"), + "venvs_use_declare_symlink": attr.string(), "_allowlist_function_transition": attr.label( default = "@bazel_tools//tools/allowlists/function_transition_allowlist", ), @@ -122,12 +127,19 @@ _py_reconfig_binary = _make_reconfig_rule(executable = True) _py_reconfig_test = _make_reconfig_rule(test = True) def _py_reconfig_executable(*, name, py_reconfig_rule, py_inner_rule, **kwargs): - reconfig_kwargs = {} - reconfig_kwargs["bootstrap_impl"] = kwargs.pop("bootstrap_impl", None) - reconfig_kwargs["extra_toolchains"] = kwargs.pop("extra_toolchains", None) - reconfig_kwargs["python_version"] = kwargs.pop("python_version", None) + reconfig_only_kwarg_names = [ + # keep sorted + "bootstrap_impl", + "build_python_zip", + "extra_toolchains", + "python_version", + "venvs_use_declare_symlink", + ] + reconfig_kwargs = { + key: kwargs.pop(key, None) + for key in reconfig_only_kwarg_names + } reconfig_kwargs["target_compatible_with"] = kwargs.get("target_compatible_with") - reconfig_kwargs["build_python_zip"] = kwargs.pop("build_python_zip", None) inner_name = "_{}_inner".format(name) py_reconfig_rule( From 428c1bbb2c81feacf5e61f44201484c7e3378434 Mon Sep 17 00:00:00 2001 From: Markus Hofbauer Date: Tue, 4 Feb 2025 18:08:13 +0100 Subject: [PATCH 072/922] docs: Update URL in gazelle example (#2602) The location of gazelle has changed to bazel-contrib, so update the example accordingly. --- examples/bzlmod_build_file_generation/BUILD.bazel | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/examples/bzlmod_build_file_generation/BUILD.bazel b/examples/bzlmod_build_file_generation/BUILD.bazel index 95bb5f88f4..5ab2790e04 100644 --- a/examples/bzlmod_build_file_generation/BUILD.bazel +++ b/examples/bzlmod_build_file_generation/BUILD.bazel @@ -81,7 +81,7 @@ gazelle_python_manifest( # This is the simple case where we only need one language supported. # If you also had proto, go, or other gazelle-supported languages, # you would also need a gazelle_binary rule. -# See https://github.com/bazelbuild/bazel-gazelle/blob/master/extend.rst#example +# See https://github.com/bazel-contrib/bazel-gazelle/blob/master/extend.md#example # This is the primary gazelle target to run, so that you can update BUILD.bazel files. # You can execute: # - bazel run //:gazelle update From 81c67981cbe488e01d25b1ae6306731167cfb2b7 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Tue, 4 Feb 2025 09:14:00 -0800 Subject: [PATCH 073/922] refactor: expose base rule construction via builders to allow customization for testing (#2600) The py_reconfig rules work by wrapping: The outer reconfig rule applies a transition, depends on an inner py base rule, then jumps through various hoops to ensure it looks and acts like the target it's wrapping. This is error prone, incomplete, and annoying code to maintain. Phil recently discovered it wasn't properly propagating the output group, so he had to add that. I wasted time trying to fix a bug I _thought_ was in it, but actually was working correctly. The logic within it is a bit hacky as it tries to emulate some of the platform-specific stuff for windows. Every time py_reconfig gains something to transition on, there's numerous places to define, propagate, and extract the pieces necessary to do it. To fix this, make the py_reconfig rules not wrap an inner base py rule. Instead, they use the same underlying rule args that the base rules do. This lets them act directly as the rule they're designed to test. Customization is done by capturing all the rule args in builder objects. The py_reconfig code constructs the same builder the base rules do, then modifies it as necessary (adding attributes, wrapping the base transition function). As a bonus, this sets some ground work to allow more easily defining derivative rules without having to copy/paste arbitrary parts of how the base rules are defined. Work towards https://github.com/bazelbuild/rules_python/issues/1647 --- python/private/builders.bzl | 228 ++++++++++++++++++++++ python/private/py_binary_macro.bzl | 5 +- python/private/py_binary_rule.bzl | 20 +- python/private/py_executable.bzl | 42 ++-- python/private/py_test_macro.bzl | 5 +- python/private/py_test_rule.bzl | 18 +- tests/support/sh_py_run_test.bzl | 193 +++++------------- tests/toolchains/python_toolchain_test.py | 9 +- 8 files changed, 330 insertions(+), 190 deletions(-) diff --git a/python/private/builders.bzl b/python/private/builders.bzl index 50aa3ed91a..bf5dbb8667 100644 --- a/python/private/builders.bzl +++ b/python/private/builders.bzl @@ -96,6 +96,145 @@ def _DepsetBuilder_build(self): kwargs["order"] = self._order[0] return depset(direct = self.direct, transitive = self.transitive, **kwargs) +def _Optional(*initial): + """A wrapper for a re-assignable value that may or may not be set. + + This allows structs to have attributes that aren't inherently mutable + and must be re-assigned to have their value updated. + + Args: + *initial: A single vararg to be the initial value, or no args + to leave it unset. + + Returns: + {type}`Optional` + """ + if len(initial) > 1: + fail("Only zero or one positional arg allowed") + + # buildifier: disable=uninitialized + self = struct( + _value = list(initial), + present = lambda *a, **k: _Optional_present(self, *a, **k), + set = lambda *a, **k: _Optional_set(self, *a, **k), + get = lambda *a, **k: _Optional_get(self, *a, **k), + ) + return self + +def _Optional_set(self, value): + """Sets the value of the optional. + + Args: + self: implicitly added + value: the value to set. + """ + if len(self._value) == 0: + self._value.append(value) + else: + self._value[0] = value + +def _Optional_get(self): + """Gets the value of the optional, or error. + + Args: + self: implicitly added + + Returns: + The stored value, or error if not set. + """ + if not len(self._value): + fail("Value not present") + return self._value[0] + +def _Optional_present(self): + """Tells if a value is present. + + Args: + self: implicitly added + + Returns: + {type}`bool` True if the value is set, False if not. + """ + return len(self._value) > 0 + +def _RuleBuilder(implementation = None, **kwargs): + """Builder for creating rules. + + Args: + implementation: {type}`callable` The rule implementation function. + **kwargs: The same as the `rule()` function, but using builders + for the non-mutable Bazel objects. + """ + + # buildifier: disable=uninitialized + self = struct( + attrs = dict(kwargs.pop("attrs", None) or {}), + cfg = kwargs.pop("cfg", None) or _TransitionBuilder(), + exec_groups = dict(kwargs.pop("exec_groups", None) or {}), + executable = _Optional(), + fragments = list(kwargs.pop("fragments", None) or []), + implementation = _Optional(implementation), + extra_kwargs = kwargs, + provides = list(kwargs.pop("provides", None) or []), + test = _Optional(), + toolchains = list(kwargs.pop("toolchains", None) or []), + build = lambda *a, **k: _RuleBuilder_build(self, *a, **k), + to_kwargs = lambda *a, **k: _RuleBuilder_to_kwargs(self, *a, **k), + ) + if "test" in kwargs: + self.test.set(kwargs.pop("test")) + if "executable" in kwargs: + self.executable.set(kwargs.pop("executable")) + return self + +def _RuleBuilder_build(self, debug = ""): + """Builds a `rule` object + + Args: + self: implicitly added + debug: {type}`str` If set, prints the args used to create the rule. + + Returns: + {type}`rule` + """ + kwargs = self.to_kwargs() + if debug: + lines = ["=" * 80, "rule kwargs: {}:".format(debug)] + for k, v in sorted(kwargs.items()): + lines.append(" {}={}".format(k, v)) + print("\n".join(lines)) # buildifier: disable=print + return rule(**kwargs) + +def _RuleBuilder_to_kwargs(self): + """Builds the arguments for calling `rule()`. + + Args: + self: implicitly added + + Returns: + {type}`dict` + """ + kwargs = {} + if self.executable.present(): + kwargs["executable"] = self.executable.get() + if self.test.present(): + kwargs["test"] = self.test.get() + + kwargs.update( + implementation = self.implementation.get(), + cfg = self.cfg.build() if self.cfg.implementation.present() else None, + attrs = { + k: (v.build() if hasattr(v, "build") else v) + for k, v in self.attrs.items() + }, + exec_groups = self.exec_groups, + fragments = self.fragments, + provides = self.provides, + toolchains = self.toolchains, + ) + kwargs.update(self.extra_kwargs) + return kwargs + def _RunfilesBuilder(): """Creates a `RunfilesBuilder`. @@ -177,6 +316,91 @@ def _RunfilesBuilder_build(self, ctx, **kwargs): **kwargs ).merge_all(self.runfiles) +def _SetBuilder(initial = None): + """Builder for list of unique values. + + Args: + initial: {type}`list | None` The initial values. + + Returns: + {type}`SetBuilder` + """ + initial = {} if not initial else {v: None for v in initial} + + # buildifier: disable=uninitialized + self = struct( + # TODO - Switch this to use set() builtin when available + # https://bazel.build/rules/lib/core/set + _values = initial, + update = lambda *a, **k: _SetBuilder_update(self, *a, **k), + build = lambda *a, **k: _SetBuilder_build(self, *a, **k), + ) + return self + +def _SetBuilder_build(self): + """Builds the values into a list + + Returns: + {type}`list` + """ + return self._values.keys() + +def _SetBuilder_update(self, *others): + """Adds values to the builder. + + Args: + self: implicitly added + *others: {type}`list` values to add to the set. + """ + for other in others: + for value in other: + if value not in self._values: + self._values[value] = None + +def _TransitionBuilder(implementation = None, inputs = None, outputs = None, **kwargs): + """Builder for transition objects. + + Args: + implementation: {type}`callable` the transition implementation function. + inputs: {type}`list[str]` the inputs for the transition. + outputs: {type}`list[str]` the outputs of the transition. + **kwargs: Extra keyword args to use when building. + + Returns: + {type}`TransitionBuilder` + """ + + # buildifier: disable=uninitialized + self = struct( + implementation = _Optional(implementation), + # Bazel requires transition.inputs to have unique values, so use set + # semantics so extenders of a transition can easily add/remove values. + # TODO - Use set builtin instead of custom builder, when available. + # https://bazel.build/rules/lib/core/set + inputs = _SetBuilder(inputs), + # Bazel requires transition.inputs to have unique values, so use set + # semantics so extenders of a transition can easily add/remove values. + # TODO - Use set builtin instead of custom builder, when available. + # https://bazel.build/rules/lib/core/set + outputs = _SetBuilder(outputs), + extra_kwargs = kwargs, + build = lambda *a, **k: _TransitionBuilder_build(self, *a, **k), + ) + return self + +def _TransitionBuilder_build(self): + """Creates a transition from the builder. + + Returns: + {type}`transition` + """ + return transition( + implementation = self.implementation.get(), + inputs = self.inputs.build(), + outputs = self.outputs.build(), + **self.extra_kwargs + ) + # Skylib's types module doesn't have is_file, so roll our own def _is_file(value): return type(value) == "File" @@ -187,4 +411,8 @@ def _is_runfiles(value): builders = struct( DepsetBuilder = _DepsetBuilder, RunfilesBuilder = _RunfilesBuilder, + RuleBuilder = _RuleBuilder, + TransitionBuilder = _TransitionBuilder, + SetBuilder = _SetBuilder, + Optional = _Optional, ) diff --git a/python/private/py_binary_macro.bzl b/python/private/py_binary_macro.bzl index d1269f2321..fa10f2e8a3 100644 --- a/python/private/py_binary_macro.bzl +++ b/python/private/py_binary_macro.bzl @@ -17,5 +17,8 @@ load(":py_binary_rule.bzl", py_binary_rule = "py_binary") load(":py_executable.bzl", "convert_legacy_create_init_to_int") def py_binary(**kwargs): + py_binary_macro(py_binary_rule, **kwargs) + +def py_binary_macro(py_rule, **kwargs): convert_legacy_create_init_to_int(kwargs) - py_binary_rule(**kwargs) + py_rule(**kwargs) diff --git a/python/private/py_binary_rule.bzl b/python/private/py_binary_rule.bzl index f1c8eb1325..5b40f52198 100644 --- a/python/private/py_binary_rule.bzl +++ b/python/private/py_binary_rule.bzl @@ -13,15 +13,14 @@ # limitations under the License. """Rule implementation of py_binary for Bazel.""" -load("@bazel_skylib//lib:dicts.bzl", "dicts") load(":attributes.bzl", "AGNOSTIC_BINARY_ATTRS") load( ":py_executable.bzl", - "create_executable_rule", + "create_executable_rule_builder", "py_executable_impl", ) -_PY_TEST_ATTRS = { +_COVERAGE_ATTRS = { # Magic attribute to help C++ coverage work. There's no # docs about this; see TestActionBuilder.java "_collect_cc_coverage": attr.label( @@ -45,8 +44,13 @@ def _py_binary_impl(ctx): inherited_environment = [], ) -py_binary = create_executable_rule( - implementation = _py_binary_impl, - attrs = dicts.add(AGNOSTIC_BINARY_ATTRS, _PY_TEST_ATTRS), - executable = True, -) +def create_binary_rule_builder(): + builder = create_executable_rule_builder( + implementation = _py_binary_impl, + executable = True, + ) + builder.attrs.update(AGNOSTIC_BINARY_ATTRS) + builder.attrs.update(_COVERAGE_ATTRS) + return builder + +py_binary = create_binary_rule_builder().build() diff --git a/python/private/py_executable.bzl b/python/private/py_executable.bzl index 18a7a707fc..2b2bf6636a 100644 --- a/python/private/py_executable.bzl +++ b/python/private/py_executable.bzl @@ -1747,16 +1747,6 @@ def _transition_executable_impl(input_settings, attr): settings[_PYTHON_VERSION_FLAG] = attr.python_version return settings -_transition_executable = transition( - implementation = _transition_executable_impl, - inputs = [ - _PYTHON_VERSION_FLAG, - ], - outputs = [ - _PYTHON_VERSION_FLAG, - ], -) - def create_executable_rule(*, attrs, **kwargs): return create_base_executable_rule( attrs = attrs, @@ -1764,33 +1754,33 @@ def create_executable_rule(*, attrs, **kwargs): **kwargs ) -def create_base_executable_rule(*, attrs, fragments = [], **kwargs): +def create_base_executable_rule(): """Create a function for defining for Python binary/test targets. - Args: - attrs: Rule attributes - fragments: List of str; extra config fragments that are required. - **kwargs: Additional args to pass onto `rule()` - Returns: A rule function """ - if "py" not in fragments: - # The list might be frozen, so use concatentation - fragments = fragments + ["py"] - kwargs.setdefault("provides", []).append(PyExecutableInfo) - kwargs["exec_groups"] = REQUIRED_EXEC_GROUPS | (kwargs.get("exec_groups") or {}) - kwargs.setdefault("cfg", _transition_executable) - return rule( - # TODO: add ability to remove attrs, i.e. for imports attr - attrs = dicts.add(EXECUTABLE_ATTRS, attrs), + return create_executable_rule_builder().build() + +def create_executable_rule_builder(implementation, **kwargs): + builder = builders.RuleBuilder( + implementation = implementation, + attrs = EXECUTABLE_ATTRS, + exec_groups = REQUIRED_EXEC_GROUPS, + fragments = ["py", "bazel_py"], + provides = [PyExecutableInfo], toolchains = [ TOOLCHAIN_TYPE, config_common.toolchain_type(EXEC_TOOLS_TOOLCHAIN_TYPE, mandatory = False), ] + _CC_TOOLCHAINS, - fragments = fragments, + cfg = builders.TransitionBuilder( + implementation = _transition_executable_impl, + inputs = [_PYTHON_VERSION_FLAG], + outputs = [_PYTHON_VERSION_FLAG], + ), **kwargs ) + return builder def cc_configure_features( ctx, diff --git a/python/private/py_test_macro.bzl b/python/private/py_test_macro.bzl index 348e877225..028dee6678 100644 --- a/python/private/py_test_macro.bzl +++ b/python/private/py_test_macro.bzl @@ -17,5 +17,8 @@ load(":py_executable.bzl", "convert_legacy_create_init_to_int") load(":py_test_rule.bzl", py_test_rule = "py_test") def py_test(**kwargs): + py_test_macro(py_test_rule, **kwargs) + +def py_test_macro(py_rule, **kwargs): convert_legacy_create_init_to_int(kwargs) - py_test_rule(**kwargs) + py_rule(**kwargs) diff --git a/python/private/py_test_rule.bzl b/python/private/py_test_rule.bzl index 63000c7255..6ad4fbddb8 100644 --- a/python/private/py_test_rule.bzl +++ b/python/private/py_test_rule.bzl @@ -13,12 +13,11 @@ # limitations under the License. """Implementation of py_test rule.""" -load("@bazel_skylib//lib:dicts.bzl", "dicts") load(":attributes.bzl", "AGNOSTIC_TEST_ATTRS") load(":common.bzl", "maybe_add_test_execution_info") load( ":py_executable.bzl", - "create_executable_rule", + "create_executable_rule_builder", "py_executable_impl", ) @@ -48,8 +47,13 @@ def _py_test_impl(ctx): maybe_add_test_execution_info(providers, ctx) return providers -py_test = create_executable_rule( - implementation = _py_test_impl, - attrs = dicts.add(AGNOSTIC_TEST_ATTRS, _BAZEL_PY_TEST_ATTRS), - test = True, -) +def create_test_rule_builder(): + builder = create_executable_rule_builder( + implementation = _py_test_impl, + test = True, + ) + builder.attrs.update(AGNOSTIC_TEST_ATTRS) + builder.attrs.update(_BAZEL_PY_TEST_ATTRS) + return builder + +py_test = create_test_rule_builder().build() diff --git a/tests/support/sh_py_run_test.bzl b/tests/support/sh_py_run_test.bzl index 4fa53ebd66..a1da285864 100644 --- a/tests/support/sh_py_run_test.bzl +++ b/tests/support/sh_py_run_test.bzl @@ -18,162 +18,77 @@ without the overhead of a bazel-in-bazel integration test. """ load("@rules_shell//shell:sh_test.bzl", "sh_test") -load("//python:py_binary.bzl", "py_binary") -load("//python:py_test.bzl", "py_test") +load("//python/private:py_binary_macro.bzl", "py_binary_macro") # buildifier: disable=bzl-visibility +load("//python/private:py_binary_rule.bzl", "create_binary_rule_builder") # buildifier: disable=bzl-visibility +load("//python/private:py_test_macro.bzl", "py_test_macro") # buildifier: disable=bzl-visibility +load("//python/private:py_test_rule.bzl", "create_test_rule_builder") # buildifier: disable=bzl-visibility load("//python/private:toolchain_types.bzl", "TARGET_TOOLCHAIN_TYPE") # buildifier: disable=bzl-visibility load("//tests/support:support.bzl", "VISIBLE_FOR_TESTING") -def _perform_transition_impl(input_settings, attr): - settings = dict(input_settings) +def _perform_transition_impl(input_settings, attr, base_impl): + settings = {k: input_settings[k] for k in _RECONFIG_INHERITED_OUTPUTS if k in input_settings} + settings.update(base_impl(input_settings, attr)) + settings[VISIBLE_FOR_TESTING] = True settings["//command_line_option:build_python_zip"] = attr.build_python_zip if attr.bootstrap_impl: settings["//python/config_settings:bootstrap_impl"] = attr.bootstrap_impl if attr.extra_toolchains: settings["//command_line_option:extra_toolchains"] = attr.extra_toolchains - if attr.python_version: - settings["//python/config_settings:python_version"] = attr.python_version if attr.venvs_use_declare_symlink: settings["//python/config_settings:venvs_use_declare_symlink"] = attr.venvs_use_declare_symlink return settings -_perform_transition = transition( - implementation = _perform_transition_impl, - inputs = [ - "//python/config_settings:bootstrap_impl", - "//command_line_option:extra_toolchains", - "//python/config_settings:python_version", - "//python/config_settings:venvs_use_declare_symlink", - ], - outputs = [ - "//command_line_option:build_python_zip", - "//command_line_option:extra_toolchains", - "//python/config_settings:bootstrap_impl", - "//python/config_settings:python_version", - "//python/config_settings:venvs_use_declare_symlink", - VISIBLE_FOR_TESTING, - ], -) - -def _py_reconfig_impl(ctx): - default_info = ctx.attr.target[DefaultInfo] - exe_ext = default_info.files_to_run.executable.extension - if exe_ext: - exe_ext = "." + exe_ext - exe_name = ctx.label.name + exe_ext - - executable = ctx.actions.declare_file(exe_name) - ctx.actions.symlink(output = executable, target_file = default_info.files_to_run.executable) - - default_outputs = [executable] - - # todo: could probably check target.owner vs src.owner to check if it should - # be symlinked or included as-is - # For simplicity of implementation, we're assuming the target being run is - # py_binary-like. In order for Windows to work, we need to make sure the - # file that the .exe launcher runs (the .zip or underlying non-exe - # executable) is a sibling of the .exe file with the same base name. - for src in default_info.files.to_list(): - if src.extension in ("", "zip"): - ext = ("." if src.extension else "") + src.extension - output = ctx.actions.declare_file(ctx.label.name + ext) - ctx.actions.symlink(output = output, target_file = src) - default_outputs.append(output) - - return [ - DefaultInfo( - executable = executable, - files = depset(default_outputs), - # On windows, the other default outputs must also be included - # in runfiles so the exe launcher can find the backing file. - runfiles = ctx.runfiles(default_outputs).merge( - default_info.default_runfiles, - ), - ), - ctx.attr.target[OutputGroupInfo], - # Inherit the expanded environment from the inner target. - ctx.attr.target[RunEnvironmentInfo], - ] - -def _make_reconfig_rule(**kwargs): - attrs = { - "bootstrap_impl": attr.string(), - "build_python_zip": attr.string(default = "auto"), - "extra_toolchains": attr.string_list( - doc = """ +_RECONFIG_INPUTS = [ + "//python/config_settings:bootstrap_impl", + "//command_line_option:extra_toolchains", + "//python/config_settings:venvs_use_declare_symlink", +] +_RECONFIG_OUTPUTS = _RECONFIG_INPUTS + [ + "//command_line_option:build_python_zip", + VISIBLE_FOR_TESTING, +] +_RECONFIG_INHERITED_OUTPUTS = [v for v in _RECONFIG_OUTPUTS if v in _RECONFIG_INPUTS] + +_RECONFIG_ATTRS = { + "bootstrap_impl": attr.string(), + "build_python_zip": attr.string(default = "auto"), + "extra_toolchains": attr.string_list( + doc = """ Value for the --extra_toolchains flag. NOTE: You'll likely have to also specify //tests/support/cc_toolchains:all (or some CC toolchain) to make the RBE presubmits happy, which disable auto-detection of a CC toolchain. """, - ), - "python_version": attr.string(), - "target": attr.label(executable = True, cfg = "target"), - "venvs_use_declare_symlink": attr.string(), - "_allowlist_function_transition": attr.label( - default = "@bazel_tools//tools/allowlists/function_transition_allowlist", - ), - } - return rule( - implementation = _py_reconfig_impl, - attrs = attrs, - cfg = _perform_transition, - **kwargs - ) + ), + "venvs_use_declare_symlink": attr.string(), +} -_py_reconfig_binary = _make_reconfig_rule(executable = True) - -_py_reconfig_test = _make_reconfig_rule(test = True) - -def _py_reconfig_executable(*, name, py_reconfig_rule, py_inner_rule, **kwargs): - reconfig_only_kwarg_names = [ - # keep sorted - "bootstrap_impl", - "build_python_zip", - "extra_toolchains", - "python_version", - "venvs_use_declare_symlink", - ] - reconfig_kwargs = { - key: kwargs.pop(key, None) - for key in reconfig_only_kwarg_names - } - reconfig_kwargs["target_compatible_with"] = kwargs.get("target_compatible_with") - - inner_name = "_{}_inner".format(name) - py_reconfig_rule( - name = name, - target = inner_name, - **reconfig_kwargs - ) - py_inner_rule( - name = inner_name, - tags = ["manual"], - **kwargs - ) +def _create_reconfig_rule(builder): + builder.attrs.update(_RECONFIG_ATTRS) + + base_cfg_impl = builder.cfg.implementation.get() + builder.cfg.implementation.set(lambda *args: _perform_transition_impl(base_impl = base_cfg_impl, *args)) + builder.cfg.inputs.update(_RECONFIG_INPUTS) + builder.cfg.outputs.update(_RECONFIG_OUTPUTS) + + return builder.build() + +_py_reconfig_binary = _create_reconfig_rule(create_binary_rule_builder()) -def py_reconfig_test(*, name, **kwargs): +_py_reconfig_test = _create_reconfig_rule(create_test_rule_builder()) + +def py_reconfig_test(**kwargs): """Create a py_test with customized build settings for testing. Args: - name: str, name of teset target. - **kwargs: kwargs to pass along to _py_reconfig_test and py_test. + **kwargs: kwargs to pass along to _py_reconfig_test. """ - _py_reconfig_executable( - name = name, - py_reconfig_rule = _py_reconfig_test, - py_inner_rule = py_test, - **kwargs - ) + py_test_macro(_py_reconfig_test, **kwargs) -def py_reconfig_binary(*, name, **kwargs): - _py_reconfig_executable( - name = name, - py_reconfig_rule = _py_reconfig_binary, - py_inner_rule = py_binary, - **kwargs - ) +def py_reconfig_binary(**kwargs): + py_binary_macro(_py_reconfig_binary, **kwargs) def sh_py_run_test(*, name, sh_src, py_src, **kwargs): """Run a py_binary within a sh_test. @@ -196,26 +111,12 @@ def sh_py_run_test(*, name, sh_src, py_src, **kwargs): "BIN_RLOCATION": "$(rlocationpaths {})".format(bin_name), }, ) - - py_binary_kwargs = { - key: kwargs.pop(key) - for key in ("imports", "deps", "env") - if key in kwargs - } - - _py_reconfig_binary( + py_reconfig_binary( name = bin_name, - tags = ["manual"], - target = "_{}_plain_bin".format(name), - **kwargs - ) - - py_binary( - name = "_{}_plain_bin".format(name), srcs = [py_src], main = py_src, tags = ["manual"], - **py_binary_kwargs + **kwargs ) def _current_build_settings_impl(ctx): diff --git a/tests/toolchains/python_toolchain_test.py b/tests/toolchains/python_toolchain_test.py index 371b252a4a..591d7dbe8a 100644 --- a/tests/toolchains/python_toolchain_test.py +++ b/tests/toolchains/python_toolchain_test.py @@ -1,6 +1,7 @@ import json import os import pathlib +import pprint import sys import unittest @@ -18,7 +19,13 @@ def test_expected_toolchain_matches(self): settings = json.loads(pathlib.Path(settings_path).read_text()) expected = "python_{}".format(expect_version.replace(".", "_")) - self.assertIn(expected, settings["toolchain_label"], str(settings)) + msg = ( + "Expected toolchain not found\n" + + f"Expected toolchain label to contain: {expected}\n" + + "Actual build settings:\n" + + pprint.pformat(settings) + ) + self.assertIn(expected, settings["toolchain_label"], msg) actual = "{v.major}.{v.minor}.{v.micro}".format(v=sys.version_info) self.assertEqual(actual, expect_version) From edfb4b34de1c2602f8ae5c8d402384c8e36a03cd Mon Sep 17 00:00:00 2001 From: Ivo List Date: Tue, 11 Feb 2025 23:28:37 +0100 Subject: [PATCH 074/922] feat: Remove and redirect py_proto_library to protobuf (#2604) Protobuf team is taking ownership of `py_proto_library` and the implementation was moved to protobuf repository. Remove py_proto_library from rules_python, to prevent divergent implementations. Make a redirect with a deprecation warning, so that this doesn't break any users. Previously this was attempted in: https://github.com/bazelbuild/rules_python/commit/d0e25cfb41446e481da6e85f04ad0ac5bcf7ea80 Work towards https://github.com/bazelbuild/rules_python/issues/2173, https://github.com/bazelbuild/rules_python/issues/2543 --- CHANGELOG.md | 3 + MODULE.bazel | 2 +- WORKSPACE | 6 - examples/bzlmod/MODULE.bazel | 3 - examples/bzlmod/py_proto_library/BUILD.bazel | 3 +- .../py_proto_library/foo_external/BUILD.bazel | 4 +- .../foo_external/MODULE.bazel | 1 - internal_dev_deps.bzl | 7 - python/BUILD.bazel | 2 +- python/private/BUILD.bazel | 1 - python/private/proto/BUILD.bazel | 48 ---- python/private/proto/py_proto_library.bzl | 244 ------------------ python/proto.bzl | 6 +- 13 files changed, 12 insertions(+), 318 deletions(-) delete mode 100644 python/private/proto/BUILD.bazel delete mode 100644 python/private/proto/py_proto_library.bzl diff --git a/CHANGELOG.md b/CHANGELOG.md index 61000a1b08..7255e9ffcd 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -52,6 +52,9 @@ Unreleased changes template. {#v0-0-0-changed} ### Changed +* (rules) `py_proto_library` is deprecated in favour of the + implementation in https://github.com/protocolbuffers/protobuf. It will be + removed in the future release. * (pypi) {obj}`pip.override` will now be ignored instead of raising an error, fixes [#2550](https://github.com/bazelbuild/rules_python/issues/2550). * (rules) deprecation warnings for deprecated symbols have been turned off by diff --git a/MODULE.bazel b/MODULE.bazel index 89f1cd7961..76710e4ac4 100644 --- a/MODULE.bazel +++ b/MODULE.bazel @@ -10,7 +10,7 @@ bazel_dep(name = "rules_cc", version = "0.0.16") bazel_dep(name = "platforms", version = "0.0.4") # Those are loaded only when using py_proto_library -bazel_dep(name = "rules_proto", version = "7.0.2") +# Use py_proto_library directly from protobuf repository bazel_dep(name = "protobuf", version = "29.0-rc2", repo_name = "com_google_protobuf") internal_deps = use_extension("//python/private:internal_deps.bzl", "internal_deps") diff --git a/WORKSPACE b/WORKSPACE index 902af58ec8..b97411e2d5 100644 --- a/WORKSPACE +++ b/WORKSPACE @@ -166,9 +166,3 @@ http_file( "https://files.pythonhosted.org/packages/50/67/3e966d99a07d60a21a21d7ec016e9e4c2642a86fea251ec68677daf71d4d/numpy-1.25.2-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", ], ) - -# rules_proto expects //external:python_headers to point at the python headers. -bind( - name = "python_headers", - actual = "//python/cc:current_py_cc_headers", -) diff --git a/examples/bzlmod/MODULE.bazel b/examples/bzlmod/MODULE.bazel index d8535a0115..eaed078d63 100644 --- a/examples/bzlmod/MODULE.bazel +++ b/examples/bzlmod/MODULE.bazel @@ -12,9 +12,6 @@ local_path_override( path = "../..", ) -# (py_proto_library specific) We are using rules_proto to define rules_proto targets to be consumed by py_proto_library. -bazel_dep(name = "rules_proto", version = "6.0.0-rc1") - # (py_proto_library specific) Add the protobuf library for well-known types (e.g. `Any`, `Timestamp`, etc) bazel_dep(name = "protobuf", version = "27.0", repo_name = "com_google_protobuf") diff --git a/examples/bzlmod/py_proto_library/BUILD.bazel b/examples/bzlmod/py_proto_library/BUILD.bazel index 24436b48ea..175589fbf9 100644 --- a/examples/bzlmod/py_proto_library/BUILD.bazel +++ b/examples/bzlmod/py_proto_library/BUILD.bazel @@ -20,11 +20,12 @@ py_test( # Regression test for https://github.com/bazelbuild/rules_python/issues/2515 # -# This test failed before https://github.com/bazelbuild/rules_python/pull/2516 +# This test fails before protobuf 30.0 release # when ran with --legacy_external_runfiles=False (default in Bazel 8.0.0). native_test( name = "external_import_test", src = "@foo_external//:py_binary_with_proto", + tags = ["manual"], # TODO: reenable when com_google_protobuf is upgraded # Incompatible with Windows: native_test wrapping a py_binary doesn't work # on Windows. target_compatible_with = select({ diff --git a/examples/bzlmod/py_proto_library/foo_external/BUILD.bazel b/examples/bzlmod/py_proto_library/foo_external/BUILD.bazel index 3fa22e06e7..183a3c28d2 100644 --- a/examples/bzlmod/py_proto_library/foo_external/BUILD.bazel +++ b/examples/bzlmod/py_proto_library/foo_external/BUILD.bazel @@ -1,5 +1,5 @@ -load("@rules_proto//proto:defs.bzl", "proto_library") -load("@rules_python//python:proto.bzl", "py_proto_library") +load("@com_google_protobuf//bazel:proto_library.bzl", "proto_library") +load("@com_google_protobuf//bazel:py_proto_library.bzl", "py_proto_library") load("@rules_python//python:py_binary.bzl", "py_binary") package(default_visibility = ["//visibility:public"]) diff --git a/examples/bzlmod/py_proto_library/foo_external/MODULE.bazel b/examples/bzlmod/py_proto_library/foo_external/MODULE.bazel index 5063f9b2d1..aca6f98eab 100644 --- a/examples/bzlmod/py_proto_library/foo_external/MODULE.bazel +++ b/examples/bzlmod/py_proto_library/foo_external/MODULE.bazel @@ -5,4 +5,3 @@ module( bazel_dep(name = "rules_python", version = "1.0.0") bazel_dep(name = "protobuf", version = "28.2", repo_name = "com_google_protobuf") -bazel_dep(name = "rules_proto", version = "7.0.2") diff --git a/internal_dev_deps.bzl b/internal_dev_deps.bzl index 0304fb16b7..cd33475f43 100644 --- a/internal_dev_deps.bzl +++ b/internal_dev_deps.bzl @@ -177,13 +177,6 @@ def rules_python_internal_deps(): ], ) - http_archive( - name = "rules_proto", - sha256 = "904a8097fae42a690c8e08d805210e40cccb069f5f9a0f6727cf4faa7bed2c9c", - strip_prefix = "rules_proto-6.0.0-rc1", - url = "https://github.com/bazelbuild/rules_proto/releases/download/6.0.0-rc1/rules_proto-6.0.0-rc1.tar.gz", - ) - http_archive( name = "com_google_protobuf", sha256 = "23082dca1ca73a1e9c6cbe40097b41e81f71f3b4d6201e36c134acc30a1b3660", diff --git a/python/BUILD.bazel b/python/BUILD.bazel index b747e2fbc7..5c6c6a4175 100644 --- a/python/BUILD.bazel +++ b/python/BUILD.bazel @@ -116,7 +116,7 @@ bzl_library( ], visibility = ["//visibility:public"], deps = [ - "//python/private/proto:py_proto_library_bzl", + "@com_google_protobuf//bazel:py_proto_library_bzl", ], ) diff --git a/python/private/BUILD.bazel b/python/private/BUILD.bazel index 14f52c541b..2928dab068 100644 --- a/python/private/BUILD.bazel +++ b/python/private/BUILD.bazel @@ -31,7 +31,6 @@ filegroup( name = "distribution", srcs = glob(["**"]) + [ "//python/private/api:distribution", - "//python/private/proto:distribution", "//python/private/pypi:distribution", "//python/private/whl_filegroup:distribution", "//tools/build_defs/python/private:distribution", diff --git a/python/private/proto/BUILD.bazel b/python/private/proto/BUILD.bazel deleted file mode 100644 index dd53845638..0000000000 --- a/python/private/proto/BUILD.bazel +++ /dev/null @@ -1,48 +0,0 @@ -# Copyright 2022 The Bazel Authors. All rights reserved. -# -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -load("@bazel_skylib//:bzl_library.bzl", "bzl_library") -load("@com_google_protobuf//bazel/toolchains:proto_lang_toolchain.bzl", "proto_lang_toolchain") - -package(default_visibility = ["//visibility:private"]) - -licenses(["notice"]) - -filegroup( - name = "distribution", - srcs = glob(["**"]), - visibility = ["//python/private:__pkg__"], -) - -bzl_library( - name = "py_proto_library_bzl", - srcs = ["py_proto_library.bzl"], - visibility = ["//python:__pkg__"], - deps = [ - "//python:py_info_bzl", - "@com_google_protobuf//bazel/common:proto_common_bzl", - "@com_google_protobuf//bazel/common:proto_info_bzl", - "@rules_proto//proto:defs", - ], -) - -proto_lang_toolchain( - name = "python_toolchain", - command_line = "--python_out=%s", - progress_message = "Generating Python proto_library %{label}", - runtime = "@com_google_protobuf//:protobuf_python", - # NOTE: This isn't *actually* public. It's an implicit dependency of py_proto_library, - # so must be public so user usages of the rule can reference it. - visibility = ["//visibility:public"], -) diff --git a/python/private/proto/py_proto_library.bzl b/python/private/proto/py_proto_library.bzl deleted file mode 100644 index 1e9df848ab..0000000000 --- a/python/private/proto/py_proto_library.bzl +++ /dev/null @@ -1,244 +0,0 @@ -# Copyright 2022 The Bazel Authors. All rights reserved. -# -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -"""The implementation of the `py_proto_library` rule and its aspect.""" - -load("@com_google_protobuf//bazel/common:proto_common.bzl", "proto_common") -load("@com_google_protobuf//bazel/common:proto_info.bzl", "ProtoInfo") -load("//python:py_info.bzl", "PyInfo") -load("//python/api:api.bzl", _py_common = "py_common") - -PY_PROTO_TOOLCHAIN = "@rules_python//python/proto:toolchain_type" - -_PyProtoInfo = provider( - doc = "Encapsulates information needed by the Python proto rules.", - fields = { - "imports": """ - (depset[str]) The field forwarding PyInfo.imports coming from - the proto language runtime dependency.""", - "py_info": "PyInfo from proto runtime (or other deps) to propagate.", - "runfiles_from_proto_deps": """ - (depset[File]) Files from the transitive closure implicit proto - dependencies""", - "transitive_sources": """(depset[File]) The Python sources.""", - }, -) - -def _filter_provider(provider, *attrs): - return [dep[provider] for attr in attrs for dep in attr if provider in dep] - -def _incompatible_toolchains_enabled(): - return getattr(proto_common, "INCOMPATIBLE_ENABLE_PROTO_TOOLCHAIN_RESOLUTION", False) - -def _py_proto_aspect_impl(target, ctx): - """Generates and compiles Python code for a proto_library. - - The function runs protobuf compiler on the `proto_library` target generating - a .py file for each .proto file. - - Args: - target: (Target) A target providing `ProtoInfo`. Usually this means a - `proto_library` target, but not always; you must expect to visit - non-`proto_library` targets, too. - ctx: (RuleContext) The rule context. - - Returns: - ([_PyProtoInfo]) Providers collecting transitive information about - generated files. - """ - _proto_library = ctx.rule.attr - - # Check Proto file names - for proto in target[ProtoInfo].direct_sources: - if proto.is_source and "-" in proto.dirname: - fail("Cannot generate Python code for a .proto whose path contains '-' ({}).".format( - proto.path, - )) - - if _incompatible_toolchains_enabled(): - toolchain = ctx.toolchains[PY_PROTO_TOOLCHAIN] - if not toolchain: - fail("No toolchains registered for '%s'." % PY_PROTO_TOOLCHAIN) - proto_lang_toolchain_info = toolchain.proto - else: - proto_lang_toolchain_info = getattr(ctx.attr, "_aspect_proto_toolchain")[proto_common.ProtoLangToolchainInfo] - - py_common = _py_common.get(ctx) - py_info = py_common.PyInfoBuilder().merge_target( - proto_lang_toolchain_info.runtime, - ).build() - - api_deps = [proto_lang_toolchain_info.runtime] - - generated_sources = [] - proto_info = target[ProtoInfo] - proto_root = proto_info.proto_source_root - if proto_info.direct_sources: - # Generate py files - generated_sources = proto_common.declare_generated_files( - actions = ctx.actions, - proto_info = proto_info, - extension = "_pb2.py", - name_mapper = lambda name: name.replace("-", "_").replace(".", "/"), - ) - - # Handles multiple repository and virtual import cases - if proto_root.startswith(ctx.bin_dir.path): - proto_root = proto_root[len(ctx.bin_dir.path) + 1:] - - plugin_output = ctx.bin_dir.path + "/" + proto_root - - # Import path within the runfiles tree - if proto_root.startswith("external/"): - proto_root = proto_root[len("external") + 1:] - else: - proto_root = ctx.workspace_name + "/" + proto_root - - proto_common.compile( - actions = ctx.actions, - proto_info = proto_info, - proto_lang_toolchain_info = proto_lang_toolchain_info, - generated_files = generated_sources, - plugin_output = plugin_output, - ) - - # Generated sources == Python sources - python_sources = generated_sources - - deps = _filter_provider(_PyProtoInfo, getattr(_proto_library, "deps", [])) - runfiles_from_proto_deps = depset( - transitive = [dep[DefaultInfo].default_runfiles.files for dep in api_deps] + - [dep.runfiles_from_proto_deps for dep in deps], - ) - transitive_sources = depset( - direct = python_sources, - transitive = [dep.transitive_sources for dep in deps], - ) - - return [ - _PyProtoInfo( - imports = depset( - # Adding to PYTHONPATH so the generated modules can be - # imported. This is necessary when there is - # strip_import_prefix, the Python modules are generated under - # _virtual_imports. But it's undesirable otherwise, because it - # will put the repo root at the top of the PYTHONPATH, ahead of - # directories added through `imports` attributes. - [proto_root] if "_virtual_imports" in proto_root else [], - transitive = [dep[PyInfo].imports for dep in api_deps] + [dep.imports for dep in deps], - ), - runfiles_from_proto_deps = runfiles_from_proto_deps, - transitive_sources = transitive_sources, - py_info = py_info, - ), - ] - -_py_proto_aspect = aspect( - implementation = _py_proto_aspect_impl, - attrs = _py_common.API_ATTRS | ( - {} if _incompatible_toolchains_enabled() else { - "_aspect_proto_toolchain": attr.label( - default = ":python_toolchain", - ), - } - ), - attr_aspects = ["deps"], - required_providers = [ProtoInfo], - provides = [_PyProtoInfo], - toolchains = [PY_PROTO_TOOLCHAIN] if _incompatible_toolchains_enabled() else [], -) - -def _py_proto_library_rule(ctx): - """Merges results of `py_proto_aspect` in `deps`. - - Args: - ctx: (RuleContext) The rule context. - Returns: - ([PyInfo, DefaultInfo, OutputGroupInfo]) - """ - if not ctx.attr.deps: - fail("'deps' attribute mustn't be empty.") - - pyproto_infos = _filter_provider(_PyProtoInfo, ctx.attr.deps) - default_outputs = depset( - transitive = [info.transitive_sources for info in pyproto_infos], - ) - - py_common = _py_common.get(ctx) - - py_info = py_common.PyInfoBuilder() - py_info.set_has_py2_only_sources(False) - py_info.set_has_py3_only_sources(False) - py_info.transitive_sources.add(default_outputs) - py_info.imports.add([info.imports for info in pyproto_infos]) - py_info.merge_all([ - pyproto_info.py_info - for pyproto_info in pyproto_infos - ]) - return [ - DefaultInfo( - files = default_outputs, - default_runfiles = ctx.runfiles(transitive_files = depset( - transitive = - [default_outputs] + - [info.runfiles_from_proto_deps for info in pyproto_infos], - )), - ), - OutputGroupInfo( - default = depset(), - ), - py_info.build(), - ] - -py_proto_library = rule( - implementation = _py_proto_library_rule, - doc = """ - Use `py_proto_library` to generate Python libraries from `.proto` files. - - The convention is to name the `py_proto_library` rule `foo_py_pb2`, - when it is wrapping `proto_library` rule `foo_proto`. - - `deps` must point to a `proto_library` rule. - - Example: - -```starlark -py_library( - name = "lib", - deps = [":foo_py_pb2"], -) - -py_proto_library( - name = "foo_py_pb2", - deps = [":foo_proto"], -) - -proto_library( - name = "foo_proto", - srcs = ["foo.proto"], -) -```""", - attrs = { - "deps": attr.label_list( - doc = """ - The list of `proto_library` rules to generate Python libraries for. - - Usually this is just the one target: the proto library of interest. - It can be any target providing `ProtoInfo`.""", - providers = [ProtoInfo], - aspects = [_py_proto_aspect], - ), - } | _py_common.API_ATTRS, - provides = [PyInfo], -) diff --git a/python/proto.bzl b/python/proto.bzl index 3f455aee58..2ea9bdb153 100644 --- a/python/proto.bzl +++ b/python/proto.bzl @@ -11,11 +11,11 @@ # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # See the License for the specific language governing permissions and # limitations under the License. - """ Python proto library. """ -load("//python/private/proto:py_proto_library.bzl", _py_proto_library = "py_proto_library") +load("@com_google_protobuf//bazel:py_proto_library.bzl", _py_proto_library = "py_proto_library") -py_proto_library = _py_proto_library +def py_proto_library(*, deprecation = "Use py_proto_library from protobuf repository", **kwargs): + _py_proto_library(deprecation = deprecation, **kwargs) From ae361c2de8290dd7f71716a55b29c3b07cef78fe Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Thu, 13 Feb 2025 21:14:20 -0800 Subject: [PATCH 075/922] chore: updates for 1.2.0 release (#2611) Update changelog and VERSION_NEXT markers --- CHANGELOG.md | 27 ++++++++++++++++--- .../python/config_settings/index.md | 2 +- docs/environment-variables.md | 2 +- 3 files changed, 26 insertions(+), 5 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 7255e9ffcd..e93cdc5327 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -52,6 +52,27 @@ Unreleased changes template. {#v0-0-0-changed} ### Changed +* Nothing changed. + +{#v0-0-0-fixed} +### Fixed +* Nothing fixed. + +{#v0-0-0-added} +### Added +* Nothing added. + +{#v0-0-0-removed} +### Removed +* Nothing removed. + +{#v1-2-0} +## Unreleased + +[1.2.0]: https://github.com/bazelbuild/rules_python/releases/tag/1.2.0 + +{#v1-2-0-changed} +### Changed * (rules) `py_proto_library` is deprecated in favour of the implementation in https://github.com/protocolbuffers/protobuf. It will be removed in the future release. @@ -63,7 +84,7 @@ Unreleased changes template. * (pypi) Downgraded versions of packages: `pip` from `24.3.2` to `24.0.0` and `packaging` from `24.2` to `24.0`. -{#v0-0-0-fixed} +{#v1-2-0-fixed} ### Fixed * (rules) `python_zip_file` output with `--bootstrap_impl=script` works again ([#2596](https://github.com/bazelbuild/rules_python/issues/2596)). @@ -85,11 +106,11 @@ Unreleased changes template. build time (they will be created at runtime instead). (Fixes [#2489](https://github.com/bazelbuild/rules_python/issues/2489)) -{#v0-0-0-added} +{#v1-2-0-added} ### Added * Nothing added. -{#v0-0-0-removed} +{#v1-2-0-removed} ### Removed * Nothing removed. diff --git a/docs/api/rules_python/python/config_settings/index.md b/docs/api/rules_python/python/config_settings/index.md index b2163233ca..cb44de97c7 100644 --- a/docs/api/rules_python/python/config_settings/index.md +++ b/docs/api/rules_python/python/config_settings/index.md @@ -279,6 +279,6 @@ Values: is created. ::: -:::{versionadded} VERSION_NEXT_PATCH +:::{versionadded} 1.2.0 ::: :::: diff --git a/docs/environment-variables.md b/docs/environment-variables.md index dd4a700081..d50070af55 100644 --- a/docs/environment-variables.md +++ b/docs/environment-variables.md @@ -44,7 +44,7 @@ being cleaned up by the OS. If not set, then a temporary directory will be created and deleted upon program exit. -:::{versionadded} VERSION_NEXT_PATCH +:::{versionadded} 1.2.0 ::: :::: From 9b5f5ddbfc25e93f872b18cbac231af630c6162d Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Thu, 13 Feb 2025 21:35:06 -0800 Subject: [PATCH 076/922] docs: update dev docs on how to pick next version (#2612) We're not using 0-version anymore, so update the docs to reflect that. --- DEVELOPING.md | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/DEVELOPING.md b/DEVELOPING.md index d816fba57f..7f9b6fc1b1 100644 --- a/DEVELOPING.md +++ b/DEVELOPING.md @@ -36,12 +36,13 @@ also test-drive the commit in an existing Bazel workspace to sanity check functi #### Determining Semantic Version -**rules_python** is currently using [Zero-based versioning](https://0ver.org/) and thus backwards-incompatible API -changes still come under the minor-version digit. So releases with API changes and new features bump the minor, and -those with only bug fixes and other minor changes bump the patch digit. +**rules_python** uses [semantic version](https://semver.org), so releases with +API changes and new features bump the minor, and those with only bug fixes and +other minor changes bump the patch digit. To find if there were any features added or incompatible changes made, review -the commit history. This can be done using github by going to the url: +[CHANGELOG.md](CHANGELOG.md) and the commit history. This can be done using +github by going to the url: `https://github.com/bazelbuild/rules_python/compare/...main`. ### Patch release with cherry picks From e509b7cac7410be051e85706b9eb7c66fe677176 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Fri, 14 Feb 2025 16:15:57 -0800 Subject: [PATCH 077/922] docs: tell how to create branches for releases (#2613) It's easier to do patch releases when the branch is already created. Some of the bugs fixes in recent releases we probably could have easily released as patch releases if we already had the branch ready. --- DEVELOPING.md | 23 +++++++++++++++-------- 1 file changed, 15 insertions(+), 8 deletions(-) diff --git a/DEVELOPING.md b/DEVELOPING.md index 7f9b6fc1b1..dfca9844f7 100644 --- a/DEVELOPING.md +++ b/DEVELOPING.md @@ -27,12 +27,20 @@ also test-drive the commit in an existing Bazel workspace to sanity check functi ### Releasing from HEAD #### Steps -1. [Determine the next semantic version number](#determining-semantic-version) -1. Create a tag and push, e.g. `git tag 0.5.0 upstream/main && git push upstream --tags` - NOTE: Pushing the tag will trigger release automation. -1. Watch the release automation run on https://github.com/bazelbuild/rules_python/actions -1. Add missing information to the release notes. The automatic release note - generation only includes commits associated with issues. +1. [Determine the next semantic version number](#determining-semantic-version). +1. Update CHANGELOG.md: replace the `v0-0-0` and `0.0.0` with `X.Y.0`. +1. Replace `VERSION_NEXT_*` strings with `X.Y.0`. +1. Send these changes for review and get them merged. +1. Create a branch for the new release, named `release/X.Y` + ``` + git branch --no-track release/X.Y upstream/main && git push upstream release/X.Y + ``` +1. Create a tag and push: + ``` + git tag X.Y.0 upstream/release/X.Y && git push upstream --tags + ``` + **NOTE:** Pushing the tag will trigger release automation. +1. Release automation will create a GitHub release and BCR pull request. #### Determining Semantic Version @@ -54,8 +62,7 @@ release tag and the patch changes cherry-picked into it. In this example, release `0.37.0` is being patched to create release `0.37.1`. The fix being included is commit `deadbeef`. -1. `git checkout -b release/0.37 0.37.0` -1. `git push upstream release/0.37` +1. `git checkout release/0.37` 1. `git cherry-pick -x deadbeef` 1. Fix merge conflicts, if any. 1. `git cherry-pick --continue` (if applicable) From 0a3704d1954d9fe6b21e7c937f3c1451b00862ae Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Sat, 15 Feb 2025 16:05:01 -0800 Subject: [PATCH 078/922] docs: split out release steps into separate doc (#2615) Move the steps for releasing into a separate doc. The release steps are specific to releases, which only maintainers do. This frees up space in the developing docs for more general tips, tricks, and guidance for others. Along the way... * Remove the text about the core rules being part of Bazel * Put the CLA text first -- if CLAs aren't signed _before_ code is given, it can result is large headaches. * Move some more internal dev steps out of contributing docs. --- CONTRIBUTING.md | 61 ++++++++++----------------------------- DEVELOPING.md | 76 ++++++------------------------------------------- RELEASING.md | 68 +++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 91 insertions(+), 114 deletions(-) create mode 100644 RELEASING.md diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 8928246c93..8805d458e8 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -3,6 +3,21 @@ We'd love to accept your patches and contributions to this project. There are just a few small guidelines you need to follow. +## Contributor License Agreement + +First, the most important step: signing the Contributor License Agreement. We +cannot look at any of your code unless one is signed. + +Contributions to this project must be accompanied by a Contributor License +Agreement. You (or your employer) retain the copyright to your contribution, +this simply gives us permission to use and redistribute your contributions as +part of the project. Head over to to see +your current agreements on file or to sign a new one. + +You generally only need to submit a CLA once, so if you've already submitted one +(even if it was for a different project), you probably don't need to do it +again. + ## Getting started Before we can work on the code, we need to get a copy of it and setup some @@ -65,15 +80,6 @@ and setup. Subsequent runs will be faster, but there are many tests, and some of them are slow. If you're working on a particular area of code, you can run just the tests in those directories instead, which can speed up your edit-run cycle. -## Updating tool dependencies - -It's suggested to routinely update the tool versions within our repo - some of the -tools are using requirement files compiled by `uv` and others use other means. In order -to have everything self-documented, we have a special target - -`//private:requirements.update`, which uses `rules_multirun` to run in sequence all -of the requirement updating scripts in one go. This can be done once per release as -we prepare for releases. - ## Formatting Starlark files should be formatted by @@ -99,18 +105,6 @@ $ buildifier --lint=fix --warnings=native-py -warnings=all WORKSPACE Replace the argument "WORKSPACE" with the file that you are linting. -## Contributor License Agreement - -Contributions to this project must be accompanied by a Contributor License -Agreement. You (or your employer) retain the copyright to your contribution, -this simply gives us permission to use and redistribute your contributions as -part of the project. Head over to to see -your current agreements on file or to sign a new one. - -You generally only need to submit a CLA once, so if you've already submitted one -(even if it was for a different project), you probably don't need to do it -again. - ## Code reviews All submissions, including submissions by project members, require review. We @@ -198,31 +192,6 @@ merged: `compile_pip_requirements` update target, which is usually in the same directory. e.g. `bazel run //docs:requirements.update` -## Core rules - -The bulk of this repo is owned and maintained by the Bazel Python community. -However, since the core Python rules (`py_binary` and friends) are still -bundled with Bazel itself, the Bazel team retains ownership of their stubs in -this repository. This will be the case at least until the Python rules are -fully migrated to Starlark code. - -Practically, this means that a Bazel team member should approve any PR -concerning the core Python logic. This includes everything under the `python/` -directory except for `pip.bzl` and `requirements.txt`. - -Issues should be triaged as follows: - -- Anything concerning the way Bazel implements the core Python rules should be - filed under [bazelbuild/bazel](https://github.com/bazelbuild/bazel), using - the label `team-Rules-python`. - -- If the issue specifically concerns the rules_python stubs, it should be filed - here in this repository and use the label `core-rules`. - -- Anything else, such as feature requests not related to existing core rules - functionality, should also be filed in this repository but without the - `core-rules` label. - (breaking-changes)= ## Breaking Changes diff --git a/DEVELOPING.md b/DEVELOPING.md index dfca9844f7..360c57a4b3 100644 --- a/DEVELOPING.md +++ b/DEVELOPING.md @@ -17,71 +17,11 @@ # bazel run //tools/private/update_deps:update_coverage_deps 7.6.1 ``` -## Releasing - -Start from a clean checkout at `main`. - -Before running through the release it's good to run the build and the tests locally, and make sure CI is passing. You can -also test-drive the commit in an existing Bazel workspace to sanity check functionality. - -### Releasing from HEAD - -#### Steps -1. [Determine the next semantic version number](#determining-semantic-version). -1. Update CHANGELOG.md: replace the `v0-0-0` and `0.0.0` with `X.Y.0`. -1. Replace `VERSION_NEXT_*` strings with `X.Y.0`. -1. Send these changes for review and get them merged. -1. Create a branch for the new release, named `release/X.Y` - ``` - git branch --no-track release/X.Y upstream/main && git push upstream release/X.Y - ``` -1. Create a tag and push: - ``` - git tag X.Y.0 upstream/release/X.Y && git push upstream --tags - ``` - **NOTE:** Pushing the tag will trigger release automation. -1. Release automation will create a GitHub release and BCR pull request. - -#### Determining Semantic Version - -**rules_python** uses [semantic version](https://semver.org), so releases with -API changes and new features bump the minor, and those with only bug fixes and -other minor changes bump the patch digit. - -To find if there were any features added or incompatible changes made, review -[CHANGELOG.md](CHANGELOG.md) and the commit history. This can be done using -github by going to the url: -`https://github.com/bazelbuild/rules_python/compare/...main`. - -### Patch release with cherry picks - -If a patch release from head would contain changes that aren't appropriate for -a patch release, then the patch release needs to be based on the original -release tag and the patch changes cherry-picked into it. - -In this example, release `0.37.0` is being patched to create release `0.37.1`. -The fix being included is commit `deadbeef`. - -1. `git checkout release/0.37` -1. `git cherry-pick -x deadbeef` -1. Fix merge conflicts, if any. -1. `git cherry-pick --continue` (if applicable) -1. `git push upstream` - -If multiple commits need to be applied, repeat the `git cherry-pick` step for -each. - -Once the release branch is in the desired state, use `git tag` to tag it, as -done with a release from head. Release automation will do the rest. - -#### After release creation in Github - -1. Announce the release in the #python channel in the Bazel slack (bazelbuild.slack.com). - -## Secrets - -### PyPI user rules-python - -Part of the release process uploads packages to PyPI as the user `rules-python`. -This account is managed by Google; contact rules-python-pyi@google.com if -something needs to be done with the PyPI account. +## Updating tool dependencies + +It's suggested to routinely update the tool versions within our repo - some of the +tools are using requirement files compiled by `uv` and others use other means. In order +to have everything self-documented, we have a special target - +`//private:requirements.update`, which uses `rules_multirun` to run in sequence all +of the requirement updating scripts in one go. This can be done once per release as +we prepare for releases. diff --git a/RELEASING.md b/RELEASING.md new file mode 100644 index 0000000000..42a29219f9 --- /dev/null +++ b/RELEASING.md @@ -0,0 +1,68 @@ +# Releasing + +Start from a clean checkout at `main`. + +Before running through the release it's good to run the build and the tests locally, and make sure CI is passing. You can +also test-drive the commit in an existing Bazel workspace to sanity check functionality. + +## Releasing from HEAD + +### Steps +1. [Determine the next semantic version number](#determining-semantic-version). +1. Update CHANGELOG.md: replace the `v0-0-0` and `0.0.0` with `X.Y.0`. +1. Replace `VERSION_NEXT_*` strings with `X.Y.0`. +1. Send these changes for review and get them merged. +1. Create a branch for the new release, named `release/X.Y` + ``` + git branch --no-track release/X.Y upstream/main && git push upstream release/X.Y + ``` +1. Create a tag and push: + ``` + git tag X.Y.0 upstream/release/X.Y && git push upstream --tags + ``` + **NOTE:** Pushing the tag will trigger release automation. +1. Release automation will create a GitHub release and BCR pull request. + +### Determining Semantic Version + +**rules_python** uses [semantic version](https://semver.org), so releases with +API changes and new features bump the minor, and those with only bug fixes and +other minor changes bump the patch digit. + +To find if there were any features added or incompatible changes made, review +[CHANGELOG.md](CHANGELOG.md) and the commit history. This can be done using +github by going to the url: +`https://github.com/bazelbuild/rules_python/compare/...main`. + +## Patch release with cherry picks + +If a patch release from head would contain changes that aren't appropriate for +a patch release, then the patch release needs to be based on the original +release tag and the patch changes cherry-picked into it. + +In this example, release `0.37.0` is being patched to create release `0.37.1`. +The fix being included is commit `deadbeef`. + +1. `git checkout release/0.37` +1. `git cherry-pick -x deadbeef` +1. Fix merge conflicts, if any. +1. `git cherry-pick --continue` (if applicable) +1. `git push upstream` + +If multiple commits need to be applied, repeat the `git cherry-pick` step for +each. + +Once the release branch is in the desired state, use `git tag` to tag it, as +done with a release from head. Release automation will do the rest. + +### After release creation in Github + +1. Announce the release in the #python channel in the Bazel slack (bazelbuild.slack.com). + +## Secrets + +### PyPI user rules-python + +Part of the release process uploads packages to PyPI as the user `rules-python`. +This account is managed by Google; contact rules-python-pyi@google.com if +something needs to be done with the PyPI account. From 34e82cd417438fd2233738bb004c2db060c18cfe Mon Sep 17 00:00:00 2001 From: Philipp Schrader Date: Sun, 16 Feb 2025 13:28:15 -0800 Subject: [PATCH 079/922] feat: provide access to arbitrary interpreters (#2507) There are some use cases that folks want to cover here. They are discussed in [this Slack thread][1]. The high-level summary is: 1. Users want to run the exact same interpreter that Bazel is running to minimize environmental issues. 2. It is useful to pass a target label to third-party tools like mypy so that they can use the correct interpreter. This patch adds to @rickeylev's work from #2359 by adding docs and a few integration tests. [1]: https://bazelbuild.slack.com/archives/CA306CEV6/p1730095371089259 --------- Co-authored-by: Richard Levasseur --- docs/api/rules_python/python/bin/index.md | 41 ++++++++++++ docs/toolchains.md | 45 ++++++++++++- python/BUILD.bazel | 1 + python/bin/BUILD.bazel | 24 +++++++ python/private/common.bzl | 17 +++++ python/private/interpreter.bzl | 82 +++++++++++++++++++++++ python/private/interpreter_tmpl.sh | 23 +++++++ python/private/py_executable.bzl | 28 ++------ python/private/site_init_template.py | 4 +- tests/interpreter/BUILD.bazel | 52 ++++++++++++++ tests/interpreter/interpreter_test.py | 80 ++++++++++++++++++++++ tests/interpreter/interpreter_tests.bzl | 54 +++++++++++++++ tests/support/sh_py_run_test.bzl | 4 ++ 13 files changed, 430 insertions(+), 25 deletions(-) create mode 100644 docs/api/rules_python/python/bin/index.md create mode 100644 python/bin/BUILD.bazel create mode 100644 python/private/interpreter.bzl create mode 100644 python/private/interpreter_tmpl.sh create mode 100644 tests/interpreter/BUILD.bazel create mode 100644 tests/interpreter/interpreter_test.py create mode 100644 tests/interpreter/interpreter_tests.bzl diff --git a/docs/api/rules_python/python/bin/index.md b/docs/api/rules_python/python/bin/index.md new file mode 100644 index 0000000000..ad6a4e7ed5 --- /dev/null +++ b/docs/api/rules_python/python/bin/index.md @@ -0,0 +1,41 @@ +:::{default-domain} bzl +::: +:::{bzl:currentfile} //python/bin:BUILD.bazel +::: + +# //python/bin + +:::{bzl:target} python + +A target to directly run a Python interpreter. + +By default, it uses the Python version that toolchain resolution matches +(typically the one marked `is_default=True` in `MODULE.bazel`). + +This runs a Python interpreter in a similar manner as when running `python3` +on the command line. It can be invoked using `bazel run`. Remember that in +order to pass flags onto the program `--` must be specified to separate +Bazel flags from the program flags. + +An example that will run Python 3.12 and have it print the version + +``` +bazel run @rules_python//python/bin:python \ + `--@rule_python//python/config_settings:python_verion=3.12 \ + -- \ + --version +``` + +::::{seealso} +The {flag}`--python_src` flag for using the intepreter a binary/test uses. +:::: + +::::{versionadded} VERSION_NEXT_FEATURE +:::: +::: + +:::{bzl:flag} python_src + +The target (one providing `PyRuntimeInfo`) whose python interpreter to use for +{obj}`:python`. +::: diff --git a/docs/toolchains.md b/docs/toolchains.md index 6eaa244b1f..3294c1732a 100644 --- a/docs/toolchains.md +++ b/docs/toolchains.md @@ -396,7 +396,7 @@ provide `Python.h`. This is typically implemented using {obj}`py_cc_toolchain()`, which provides {obj}`ToolchainInfo` with the field `py_cc_toolchain` set, which is a -{obj}`PyCcToolchainInfo` provider instance. +{obj}`PyCcToolchainInfo` provider instance. This toolchain type is intended to hold only _target configuration_ values relating to the C/C++ information for the Python runtime. As such, when defining @@ -556,4 +556,45 @@ of available toolchains. Currently the following flags are used to influence toolchain selection: * {obj}`--@rules_python//python/config_settings:py_linux_libc` for selecting the Linux libc variant. * {obj}`--@rules_python//python/config_settings:py_freethreaded` for selecting - the freethreaded experimental Python builds available from `3.13.0` onwards. \ No newline at end of file + the freethreaded experimental Python builds available from `3.13.0` onwards. + +## Running the underlying interpreter + +To run the interpreter that Bazel will use, you can use the +`@rules_python//python/bin:python` target. This is a binary target with +the executable pointing at the `python3` binary plus its relevent runfiles. + +```console +$ bazel run @rules_python//python/bin:python +Python 3.11.1 (main, Jan 16 2023, 22:41:20) [Clang 15.0.7 ] on linux +Type "help", "copyright", "credits" or "license" for more information. +>>> +$ bazel run @rules_python//python/bin:python --@rules_python//python/config_settings:python_version=3.12 +Python 3.12.0 (main, Oct 3 2023, 01:27:23) [Clang 17.0.1 ] on linux +Type "help", "copyright", "credits" or "license" for more information. +>>> +``` + +You can also access a specific binary's interpreter this way by using the +`@rules_python//python/bin:python_src` target. In the example below, it is +assumed that the `@rules_python//tools/publish:twine` binary is fixed at Python +3.11. + +```console +$ bazel run @rules_python//python/bin:python --@rules_python//python/bin:interpreter_src=@rules_python//tools/publish:twine +Python 3.11.1 (main, Jan 16 2023, 22:41:20) [Clang 15.0.7 ] on linux +Type "help", "copyright", "credits" or "license" for more information. +>>> +$ bazel run @rules_python//python/bin:python --@rules_python//python/bin:interpreter_src=@rules_python//tools/publish:twine --@rules_python//python/config_settings:python_version=3.12 +Python 3.11.1 (main, Jan 16 2023, 22:41:20) [Clang 15.0.7 ] on linux +Type "help", "copyright", "credits" or "license" for more information. +>>> +``` +Despite setting the Python version explicitly to 3.12 in the example above, the +interpreter comes from the `@rules_python//tools/publish:twine` binary. That is +a fixed version. + +:::{note} +The `python` target does not provide access to any modules from `py_*` +targets on its own. Please file a feature request if this is desired. +::: diff --git a/python/BUILD.bazel b/python/BUILD.bazel index 5c6c6a4175..c52e772666 100644 --- a/python/BUILD.bazel +++ b/python/BUILD.bazel @@ -35,6 +35,7 @@ filegroup( name = "distribution", srcs = glob(["**"]) + [ "//python/api:distribution", + "//python/bin:distribution", "//python/cc:distribution", "//python/config_settings:distribution", "//python/constraints:distribution", diff --git a/python/bin/BUILD.bazel b/python/bin/BUILD.bazel new file mode 100644 index 0000000000..57bee34378 --- /dev/null +++ b/python/bin/BUILD.bazel @@ -0,0 +1,24 @@ +load("//python/private:interpreter.bzl", _interpreter_binary = "interpreter_binary") + +filegroup( + name = "distribution", + srcs = glob(["**"]), + visibility = ["//:__subpackages__"], +) + +_interpreter_binary( + name = "python", + binary = ":python_src", + target_compatible_with = select({ + "@platforms//os:windows": ["@platforms//:incompatible"], + "//conditions:default": [], + }), + visibility = ["//visibility:public"], +) + +# The user can modify this flag to source different interpreters for the +# `python` target above. +label_flag( + name = "python_src", + build_setting_default = "//python:none", +) diff --git a/python/private/common.bzl b/python/private/common.bzl index b6a54532d3..137f0d23f3 100644 --- a/python/private/common.bzl +++ b/python/private/common.bzl @@ -543,3 +543,20 @@ def target_platform_has_any_constraint(ctx, constraints): if ctx.target_platform_has_constraint(constraint_value): return True return False + +def runfiles_root_path(ctx, short_path): + """Compute a runfiles-root relative path from `File.short_path` + + Args: + ctx: current target ctx + short_path: str, a main-repo relative path from `File.short_path` + + Returns: + {type}`str`, a runflies-root relative path + """ + + # The ../ comes from short_path is for files in other repos. + if short_path.startswith("../"): + return short_path[3:] + else: + return "{}/{}".format(ctx.workspace_name, short_path) diff --git a/python/private/interpreter.bzl b/python/private/interpreter.bzl new file mode 100644 index 0000000000..c66d3dc21e --- /dev/null +++ b/python/private/interpreter.bzl @@ -0,0 +1,82 @@ +# Copyright 2025 The Bazel Authors. All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +"""Implementation of the rules to access the underlying Python interpreter.""" + +load("@bazel_skylib//lib:paths.bzl", "paths") +load("//python:py_runtime_info.bzl", "PyRuntimeInfo") +load(":common.bzl", "runfiles_root_path") +load(":sentinel.bzl", "SentinelInfo") +load(":toolchain_types.bzl", "TARGET_TOOLCHAIN_TYPE") + +def _interpreter_binary_impl(ctx): + if SentinelInfo in ctx.attr.binary: + toolchain = ctx.toolchains[TARGET_TOOLCHAIN_TYPE] + runtime = toolchain.py3_runtime + else: + runtime = ctx.attr.binary[PyRuntimeInfo] + + # NOTE: We name the output filename after the underlying file name + # because of things like pyenv: they use $0 to determine what to + # re-exec. If it's not a recognized name, then they fail. + if runtime.interpreter: + # In order for this to work both locally and remotely, we create a + # shell script here that re-exec's into the real interpreter. Ideally, + # we'd just use a symlink, but that breaks under certain conditions. If + # we use a ctx.actions.symlink(target=...) then it fails under remote + # execution. If we use ctx.actions.symlink(target_path=...) then it + # behaves differently inside the runfiles tree and outside the runfiles + # tree. + # + # This currently does not work on Windows. Need to find a way to enable + # that. + executable = ctx.actions.declare_file(runtime.interpreter.basename) + ctx.actions.expand_template( + template = ctx.file._template, + output = executable, + substitutions = { + "%target_file%": runfiles_root_path(ctx, runtime.interpreter.short_path), + }, + is_executable = True, + ) + else: + executable = ctx.actions.declare_symlink(paths.basename(runtime.interpreter_path)) + ctx.actions.symlink(output = executable, target_path = runtime.interpreter_path) + + return [ + DefaultInfo( + executable = executable, + runfiles = ctx.runfiles([executable], transitive_files = runtime.files).merge_all([ + ctx.attr._bash_runfiles[DefaultInfo].default_runfiles, + ]), + ), + ] + +interpreter_binary = rule( + implementation = _interpreter_binary_impl, + toolchains = [TARGET_TOOLCHAIN_TYPE], + executable = True, + attrs = { + "binary": attr.label( + mandatory = True, + ), + "_bash_runfiles": attr.label( + default = "@bazel_tools//tools/bash/runfiles", + ), + "_template": attr.label( + default = "//python/private:interpreter_tmpl.sh", + allow_single_file = True, + ), + }, +) diff --git a/python/private/interpreter_tmpl.sh b/python/private/interpreter_tmpl.sh new file mode 100644 index 0000000000..cfe85ec1be --- /dev/null +++ b/python/private/interpreter_tmpl.sh @@ -0,0 +1,23 @@ +#!/bin/bash + +# --- begin runfiles.bash initialization v3 --- +# Copy-pasted from the Bazel Bash runfiles library v3. +set -uo pipefail; set +e; f=bazel_tools/tools/bash/runfiles/runfiles.bash +# shellcheck disable=SC1090 +source "${RUNFILES_DIR:-/dev/null}/$f" 2>/dev/null || \ + source "$(grep -sm1 "^$f " "${RUNFILES_MANIFEST_FILE:-/dev/null}" | cut -f2- -d' ')" 2>/dev/null || \ + source "$0.runfiles/$f" 2>/dev/null || \ + source "$(grep -sm1 "^$f " "$0.runfiles_manifest" | cut -f2- -d' ')" 2>/dev/null || \ + source "$(grep -sm1 "^$f " "$0.exe.runfiles_manifest" | cut -f2- -d' ')" 2>/dev/null || \ + { echo>&2 "ERROR: cannot find $f"; exit 1; }; f=; set -e +# --- end runfiles.bash initialization v3 --- + +set +e # allow us to check for errors more easily +readonly TARGET_FILE="%target_file%" +MAIN_BIN=$(rlocation "$TARGET_FILE") + +if [[ -z "$MAIN_BIN" || ! -e "$MAIN_BIN" ]]; then + echo "ERROR: interpreter executable not found: $MAIN_BIN (from $TARGET_FILE)" + exit 1 +fi +exec "${MAIN_BIN}" "$@" diff --git a/python/private/py_executable.bzl b/python/private/py_executable.bzl index 2b2bf6636a..a2ccdc65f3 100644 --- a/python/private/py_executable.bzl +++ b/python/private/py_executable.bzl @@ -48,6 +48,7 @@ load( "filter_to_py_srcs", "get_imports", "is_bool", + "runfiles_root_path", "target_platform_has_any_constraint", "union_attrs", ) @@ -447,7 +448,7 @@ def _create_executable( ) def _create_zip_main(ctx, *, stage2_bootstrap, runtime_details, venv): - python_binary = _runfiles_root_path(ctx, venv.interpreter.short_path) + python_binary = runfiles_root_path(ctx, venv.interpreter.short_path) python_binary_actual = venv.interpreter_actual_path # The location of this file doesn't really matter. It's added to @@ -522,7 +523,7 @@ def _create_venv(ctx, output_prefix, imports, runtime_details): if not venvs_use_declare_symlink_enabled: if runtime.interpreter: - interpreter_actual_path = _runfiles_root_path(ctx, runtime.interpreter.short_path) + interpreter_actual_path = runfiles_root_path(ctx, runtime.interpreter.short_path) else: interpreter_actual_path = runtime.interpreter_path @@ -543,11 +544,11 @@ def _create_venv(ctx, output_prefix, imports, runtime_details): # may choose to write what symlink() points to instead. interpreter = ctx.actions.declare_symlink("{}/bin/{}".format(venv, py_exe_basename)) - interpreter_actual_path = _runfiles_root_path(ctx, runtime.interpreter.short_path) + interpreter_actual_path = runfiles_root_path(ctx, runtime.interpreter.short_path) rel_path = relative_path( # dirname is necessary because a relative symlink is relative to # the directory the symlink resides within. - from_ = paths.dirname(_runfiles_root_path(ctx, interpreter.short_path)), + from_ = paths.dirname(runfiles_root_path(ctx, interpreter.short_path)), to = interpreter_actual_path, ) @@ -646,23 +647,6 @@ def _create_stage2_bootstrap( ) return output -def _runfiles_root_path(ctx, short_path): - """Compute a runfiles-root relative path from `File.short_path` - - Args: - ctx: current target ctx - short_path: str, a main-repo relative path from `File.short_path` - - Returns: - {type}`str`, a runflies-root relative path - """ - - # The ../ comes from short_path is for files in other repos. - if short_path.startswith("../"): - return short_path[3:] - else: - return "{}/{}".format(ctx.workspace_name, short_path) - def _create_stage1_bootstrap( ctx, *, @@ -676,7 +660,7 @@ def _create_stage1_bootstrap( runtime = runtime_details.effective_runtime if venv: - python_binary_path = _runfiles_root_path(ctx, venv.interpreter.short_path) + python_binary_path = runfiles_root_path(ctx, venv.interpreter.short_path) else: python_binary_path = runtime_details.executable_interpreter_path diff --git a/python/private/site_init_template.py b/python/private/site_init_template.py index dcbd799909..40fb4e4139 100644 --- a/python/private/site_init_template.py +++ b/python/private/site_init_template.py @@ -163,7 +163,9 @@ def _maybe_add_path(path): if cov_tool: _print_verbose_coverage(f"Using toolchain coverage_tool {cov_tool}") elif cov_tool := os.environ.get("PYTHON_COVERAGE"): - _print_verbose_coverage(f"Using env var coverage: PYTHON_COVERAGE={cov_tool}") + _print_verbose_coverage( + f"Using env var coverage: PYTHON_COVERAGE={cov_tool}" + ) if cov_tool: if os.path.isabs(cov_tool): diff --git a/tests/interpreter/BUILD.bazel b/tests/interpreter/BUILD.bazel new file mode 100644 index 0000000000..5d89ede28a --- /dev/null +++ b/tests/interpreter/BUILD.bazel @@ -0,0 +1,52 @@ +# Copyright 2024 The Bazel Authors. All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +load(":interpreter_tests.bzl", "PYTHON_VERSIONS_TO_TEST", "py_reconfig_interpreter_tests") + +# For this test the interpreter is sourced from the current configuration. That +# means both the interpreter and the test itself are expected to run under the +# same Python version. +py_reconfig_interpreter_tests( + name = "interpreter_version_test", + srcs = ["interpreter_test.py"], + data = [ + "//python/bin:python", + ], + env = { + "PYTHON_BIN": "$(rootpath //python/bin:python)", + }, + main = "interpreter_test.py", + python_versions = PYTHON_VERSIONS_TO_TEST, +) + +# For this test the interpreter is sourced from a binary pinned at a specific +# Python version. That means the interpreter and the test itself can run +# different Python versions. +py_reconfig_interpreter_tests( + name = "python_src_test", + srcs = ["interpreter_test.py"], + data = [ + "//python/bin:python", + ], + env = { + # Since we're grabbing the interpreter from a binary with a fixed + # version, we expect to always see that version. It doesn't matter what + # Python version the test itself is running with. + "EXPECTED_INTERPRETER_VERSION": "3.11", + "PYTHON_BIN": "$(rootpath //python/bin:python)", + }, + main = "interpreter_test.py", + python_src = "//tools/publish:twine", + python_versions = PYTHON_VERSIONS_TO_TEST, +) diff --git a/tests/interpreter/interpreter_test.py b/tests/interpreter/interpreter_test.py new file mode 100644 index 0000000000..0971fa2eba --- /dev/null +++ b/tests/interpreter/interpreter_test.py @@ -0,0 +1,80 @@ +# Copyright 2024 The Bazel Authors. All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +import os +import subprocess +import sys +import unittest + + +class InterpreterTest(unittest.TestCase): + def setUp(self): + super().setUp() + self.interpreter = os.environ["PYTHON_BIN"] + + v = sys.version_info + self.version = f"{v.major}.{v.minor}" + + def test_self_version(self): + """Performs a sanity check on the Python version used for this test.""" + expected_version = os.environ["EXPECTED_SELF_VERSION"] + self.assertEqual(expected_version, self.version) + + def test_interpreter_version(self): + """Validates that we can successfully execute arbitrary code from the CLI.""" + expected_version = os.environ.get("EXPECTED_INTERPRETER_VERSION", self.version) + + try: + result = subprocess.check_output( + [self.interpreter], + text=True, + stderr=subprocess.STDOUT, + input="\r".join( + [ + "import sys", + "v = sys.version_info", + "print(f'version: {v.major}.{v.minor}')", + ] + ), + ).strip() + except subprocess.CalledProcessError as error: + print("OUTPUT:", error.stdout) + raise + + self.assertEqual(result, f"version: {expected_version}") + + def test_json_tool(self): + """Validates that we can successfully invoke a module from the CLI.""" + # Pass unformatted JSON to the json.tool module. + try: + result = subprocess.check_output( + [ + self.interpreter, + "-m", + "json.tool", + ], + text=True, + stderr=subprocess.STDOUT, + input='{"json":"obj"}', + ).strip() + except subprocess.CalledProcessError as error: + print("OUTPUT:", error.stdout) + raise + + # Validate that we get formatted JSON back. + self.assertEqual(result, '{\n "json": "obj"\n}') + + +if __name__ == "__main__": + unittest.main() diff --git a/tests/interpreter/interpreter_tests.bzl b/tests/interpreter/interpreter_tests.bzl new file mode 100644 index 0000000000..ad94f43423 --- /dev/null +++ b/tests/interpreter/interpreter_tests.bzl @@ -0,0 +1,54 @@ +# Copyright 2025 The Bazel Authors. All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +"""This file contains helpers for testing the interpreter rule.""" + +load("//tests/support:sh_py_run_test.bzl", "py_reconfig_test") + +# The versions of Python that we want to run the interpreter tests against. +PYTHON_VERSIONS_TO_TEST = ( + "3.10", + "3.11", + "3.12", +) + +def py_reconfig_interpreter_tests(name, python_versions, env = {}, **kwargs): + """Runs the specified test against each of the specified Python versions. + + One test gets generated for each Python version. The following environment + variable gets set for the test: + + EXPECTED_SELF_VERSION: Contains the Python version that the test itself + is running under. + + Args: + name: Name of the test. + python_versions: A list of Python versions to test. + env: The environment to set on the test. + **kwargs: Passed to the underlying py_reconfig_test targets. + """ + for python_version in python_versions: + py_reconfig_test( + name = "{}_{}".format(name, python_version), + env = env | { + "EXPECTED_SELF_VERSION": python_version, + }, + python_version = python_version, + **kwargs + ) + + native.test_suite( + name = name, + tests = [":{}_{}".format(name, python_version) for python_version in python_versions], + ) diff --git a/tests/support/sh_py_run_test.bzl b/tests/support/sh_py_run_test.bzl index a1da285864..d116f0403f 100644 --- a/tests/support/sh_py_run_test.bzl +++ b/tests/support/sh_py_run_test.bzl @@ -35,12 +35,15 @@ def _perform_transition_impl(input_settings, attr, base_impl): settings["//python/config_settings:bootstrap_impl"] = attr.bootstrap_impl if attr.extra_toolchains: settings["//command_line_option:extra_toolchains"] = attr.extra_toolchains + if attr.python_src: + settings["//python/bin:python_src"] = attr.python_src if attr.venvs_use_declare_symlink: settings["//python/config_settings:venvs_use_declare_symlink"] = attr.venvs_use_declare_symlink return settings _RECONFIG_INPUTS = [ "//python/config_settings:bootstrap_impl", + "//python/bin:python_src", "//command_line_option:extra_toolchains", "//python/config_settings:venvs_use_declare_symlink", ] @@ -62,6 +65,7 @@ to make the RBE presubmits happy, which disable auto-detection of a CC toolchain. """, ), + "python_src": attr.label(), "venvs_use_declare_symlink": attr.string(), } From f2941df7562c4183c37d6ceeae23e7d390738d58 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Sun, 16 Feb 2025 16:59:04 -0800 Subject: [PATCH 080/922] docs: add changelog update for //python/bin (#2616) This was a forgotten part of the original PR (#2507) implementing it. --- CHANGELOG.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index e93cdc5327..203cc55b1a 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -60,7 +60,9 @@ Unreleased changes template. {#v0-0-0-added} ### Added -* Nothing added. +* {obj}`//python/bin:python`: convenience target for directly running an + interpreter. {obj}`--//python/bin:python_src` can be used to specify a + binary whose interpreter to use. {#v0-0-0-removed} ### Removed From f9779ee9c0a7b6dbfc1cdeb4a6d6a3f06d6206df Mon Sep 17 00:00:00 2001 From: Alex Eagle Date: Fri, 21 Feb 2025 22:04:22 -0800 Subject: [PATCH 081/922] refactor: cleanup now-unreferenced proto toolchain type (#2620) Follow-up to #2604, fixes a breaking change in v1.2.0-rc0 Note that this toolchain_type became unused in that PR. We leave behind an alias to make this a non-breaking change. Verified in a downstream repo that requires the toolchain_type to register pre-built `protoc`: https://github.com/aspect-build/toolchains_protoc/pull/50/files --------- Co-authored-by: Richard Levasseur --- python/proto/BUILD.bazel | 10 ++++++++-- 1 file changed, 8 insertions(+), 2 deletions(-) diff --git a/python/proto/BUILD.bazel b/python/proto/BUILD.bazel index 9f60574f26..4d5a92a93f 100644 --- a/python/proto/BUILD.bazel +++ b/python/proto/BUILD.bazel @@ -14,5 +14,11 @@ package(default_visibility = ["//visibility:public"]) -# Toolchain type provided by proto_lang_toolchain rule and used by py_proto_library -toolchain_type(name = "toolchain_type") +# Deprecated; use @com_google_protobuf//bazel/private:python_toolchain_type instead. +# Alias is here to provide backward-compatibility; see #2604 +# It will be removed in a future release. +alias( + name = "toolchain_type", + actual = "@com_google_protobuf//bazel/private:python_toolchain_type", + deprecation = "Use @com_google_protobuf//bazel/private:python_toolchain_type instead", +) From ef205f56d641069401893bc4929b2e55ec59c426 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Sat, 22 Feb 2025 15:49:54 -0800 Subject: [PATCH 082/922] docs: add some docs to help contributors get started (#2623) A common pattern I've seen with PRs is they lack tests. I suspect part of the reason is authors aren't sure how to write tests or where to start. So here's some basic docs to help. --- CONTRIBUTING.md | 25 ++++++++-------- DEVELOPING.md | 77 +++++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 89 insertions(+), 13 deletions(-) diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 8805d458e8..cd274861d7 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -65,20 +65,10 @@ git push origin my-feature Once the code is in your github repo, you can then turn it into a Pull Request to the actual rules_python project and begin the code review process. +## Developer guide -## Running tests - -Running tests is particularly easy thanks to Bazel, simply run: - -``` -bazel test //... -``` - -And it will run all the tests it can find. The first time you do this, it will -probably take long time because various dependencies will need to be downloaded -and setup. Subsequent runs will be faster, but there are many tests, and some of -them are slow. If you're working on a particular area of code, you can run just -the tests in those directories instead, which can speed up your edit-run cycle. +For more more details, guidance, and tips for working with the code base, +see [DEVELOPING.md](DEVELOPING.md) ## Formatting @@ -192,6 +182,15 @@ merged: `compile_pip_requirements` update target, which is usually in the same directory. e.g. `bazel run //docs:requirements.update` +## Binary artifacts + +Checking in binary artifacts is not allowed. This is because they are extremely +problematic to verify and ensure they're safe + +Examples include, but aren't limited to: prebuilt binaries, shared libraries, +zip files, or wheels. + + (breaking-changes)= ## Breaking Changes diff --git a/DEVELOPING.md b/DEVELOPING.md index 360c57a4b3..83026c1dbc 100644 --- a/DEVELOPING.md +++ b/DEVELOPING.md @@ -1,5 +1,82 @@ # For Developers +This document covers tips and guidance for working on the rules_python code +base. A primary audience for it is first time contributors. + +## Running tests + +Running tests is particularly easy thanks to Bazel, simply run: + +``` +bazel test //... +``` + +And it will run all the tests it can find. The first time you do this, it will +probably take long time because various dependencies will need to be downloaded +and setup. Subsequent runs will be faster, but there are many tests, and some of +them are slow. If you're working on a particular area of code, you can run just +the tests in those directories instead, which can speed up your edit-run cycle. + +## Writing Tests + +Most code should have tests of some sort. This helps us have confidence that +refactors didn't break anything and that releases won't have regressions. + +We don't require 100% test coverage, testing certain Bazel functionality is +difficult, and some edge cases are simply too hard to test or not worth the +extra complexity. We try to judiciously decide when not having tests is a good +idea. + +Tests go under `tests/`. They are loosely organized into directories for the +particular subsystem or functionality they are testing. If an existing directory +doesn't seem like a good match for the functionality being testing, then it's +fine to create a new directory. + +Re-usable test helpers and support code go in `tests/support`. Tests don't need +to be perfectly factored and not every common thing a test does needs to be +factored into a more generally reusable piece. Copying and pasting is fine. It's +more important for tests to balance understandability and maintainability. + +### sh_py_run_test + +The [`sh_py_run_test`](tests/support/sh_py_run_test.bzl) rule is a helper to +make it easy to run a Python program with custom build settings using a shell +script to perform setup and verification. This is best to use when verifying +behavior needs certain environment variables or directory structures to +correctly and reliably verify behavior. + +When adding a test, you may find the flag you need to set isn't supported by +the rule. To have it support setting a new flag, see the py_reconfig_test docs +below. + +### py_reconfig_test + +The `py_reconfig_test` and `py_reconfig_binary` rules are helpers for running +Python binaries and tests with custom build flags. This is best to use when +verifying behavior that requires specific flags to be set and when the program +itself can verify the desired state. + +When adding a test, you may find the flag you need to set isn't supported by +the rule. To have it support setting a new flag: + +* Add an attribute to the rule. It should have the same name as the flag + it's for. It should be a string, string_list, or label attribute -- this + allows distinguishing between if the value was specified or not. +* Modify the transition and add the flag to both the inputs and outputs + list, then modify the transition's logic to check the attribute and set + the flag value if the attribute is set. + +### Integration tests + +An integration test is one that runs a separate Bazel instance inside the test. +These tests are discouraged unless absolutely necessary because they are slow, +require much memory and CPU, and are generally harder to debug. Integration +tests are reserved for things that simple can't be tested otherwise, or for +simple high level verification tests. + +Integration tests live in `tests/integration`. When possible, add to an existing +integration test. + ## Updating internal dependencies 1. Modify the `./python/private/pypi/requirements.txt` file and run: From a04b2a4815721c09c1f8579ca1a3dfb20c9dadd5 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Sun, 23 Feb 2025 17:02:24 -0800 Subject: [PATCH 083/922] ci: use Python 3.9 for mypy workflow to fix ci (#2625) The mypy check on CI has been failing. The problem was the combination of: * We were using Python 3.8 * jpetrucciani/mypy-check@master updated to use mypy 1.15 * mypy 1.15 dropped support for Python 3.8 To fix, use Python 3.9. --- .github/workflows/mypy.yaml | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/.github/workflows/mypy.yaml b/.github/workflows/mypy.yaml index 429775172e..866c43abd1 100644 --- a/.github/workflows/mypy.yaml +++ b/.github/workflows/mypy.yaml @@ -22,11 +22,10 @@ jobs: - uses: jpetrucciani/mypy-check@master with: requirements: 1.6.0 - python_version: 3.8 + python_version: 3.9 path: 'python/runfiles' - uses: jpetrucciani/mypy-check@master with: requirements: 1.6.0 - python_version: 3.8 + python_version: 3.9 path: 'tests/runfiles' - From fa882817a7a69ae1e6bc3a63530ce158b64d2efd Mon Sep 17 00:00:00 2001 From: Ignas Anikevicius <240938+aignas@users.noreply.github.com> Date: Mon, 24 Feb 2025 17:26:30 +0900 Subject: [PATCH 084/922] fix(pypi): correctly translate ppc64le to bazel platforms (#2577) Bump the `platforms` version and correctly translate the ppc64le value. See https://github.com/bazelbuild/platforms/pull/105 --------- Co-authored-by: Richard Levasseur --- CHANGELOG.md | 4 ++-- MODULE.bazel | 2 +- python/private/pypi/whl_installer/platform.py | 8 +++++--- python/private/pypi/whl_target_platforms.bzl | 2 +- python/private/repo_utils.bzl | 4 +++- .../construct_config_settings_tests.bzl | 7 ++++--- tests/pypi/whl_installer/platform_test.py | 8 +++++--- .../whl_library_targets_tests.bzl | 12 ++++++------ .../whl_target_platforms_tests.bzl | 7 +++++-- 9 files changed, 32 insertions(+), 22 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 203cc55b1a..8a62ab7840 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -52,11 +52,11 @@ Unreleased changes template. {#v0-0-0-changed} ### Changed -* Nothing changed. +* (deps) platforms 0.0.4 -> 0.0.11 {#v0-0-0-fixed} ### Fixed -* Nothing fixed. +* (pypi) The `ppc64le` is now pointing to the right target in the `platforms` package. {#v0-0-0-added} ### Added diff --git a/MODULE.bazel b/MODULE.bazel index 76710e4ac4..3d7c3042a5 100644 --- a/MODULE.bazel +++ b/MODULE.bazel @@ -7,7 +7,7 @@ module( bazel_dep(name = "bazel_features", version = "1.21.0") bazel_dep(name = "bazel_skylib", version = "1.7.1") bazel_dep(name = "rules_cc", version = "0.0.16") -bazel_dep(name = "platforms", version = "0.0.4") +bazel_dep(name = "platforms", version = "0.0.11") # Those are loaded only when using py_proto_library # Use py_proto_library directly from protobuf repository diff --git a/python/private/pypi/whl_installer/platform.py b/python/private/pypi/whl_installer/platform.py index 83e42b0e46..11dd6e37ab 100644 --- a/python/private/pypi/whl_installer/platform.py +++ b/python/private/pypi/whl_installer/platform.py @@ -42,14 +42,14 @@ class Arch(Enum): x86_32 = 2 aarch64 = 3 ppc = 4 - s390x = 5 - arm = 6 + ppc64le = 5 + s390x = 6 + arm = 7 amd64 = x86_64 arm64 = aarch64 i386 = x86_32 i686 = x86_32 x86 = x86_32 - ppc64le = ppc @classmethod def interpreter(cls) -> "Arch": @@ -271,6 +271,8 @@ def platform_machine(self) -> str: return "arm64" elif self.os != OS.linux: return "" + elif self.arch == Arch.ppc: + return "ppc" elif self.arch == Arch.ppc64le: return "ppc64le" elif self.arch == Arch.s390x: diff --git a/python/private/pypi/whl_target_platforms.bzl b/python/private/pypi/whl_target_platforms.bzl index 6823199bee..9f47e625b3 100644 --- a/python/private/pypi/whl_target_platforms.bzl +++ b/python/private/pypi/whl_target_platforms.bzl @@ -31,7 +31,7 @@ _CPU_ALIASES = { "arm64": "aarch64", "ppc": "ppc", "ppc64": "ppc", - "ppc64le": "ppc", + "ppc64le": "ppc64le", "s390x": "s390x", "arm": "arm", "armv6l": "arm", diff --git a/python/private/repo_utils.bzl b/python/private/repo_utils.bzl index e5c78be815..d9ad2449f1 100644 --- a/python/private/repo_utils.bzl +++ b/python/private/repo_utils.bzl @@ -391,8 +391,10 @@ def _get_platforms_cpu_name(mrctx): return "x86_32" if arch in ["amd64", "x86_64", "x64"]: return "x86_64" - if arch in ["ppc", "ppc64", "ppc64le"]: + if arch in ["ppc", "ppc64"]: return "ppc" + if arch in ["ppc64le"]: + return "ppc64le" if arch in ["arm", "armv7l"]: return "arm" if arch in ["aarch64"]: diff --git a/tests/config_settings/construct_config_settings_tests.bzl b/tests/config_settings/construct_config_settings_tests.bzl index 087efbbc70..1d21a8680d 100644 --- a/tests/config_settings/construct_config_settings_tests.bzl +++ b/tests/config_settings/construct_config_settings_tests.bzl @@ -47,7 +47,7 @@ def _test_minor_version_matching(name): } minor_cpu_matches = { str(Label(":is_python_3.11_aarch64")): "matched-3.11-aarch64", - str(Label(":is_python_3.11_ppc")): "matched-3.11-ppc", + str(Label(":is_python_3.11_ppc64le")): "matched-3.11-ppc64le", str(Label(":is_python_3.11_s390x")): "matched-3.11-s390x", str(Label(":is_python_3.11_x86_64")): "matched-3.11-x86_64", } @@ -58,7 +58,7 @@ def _test_minor_version_matching(name): } minor_os_cpu_matches = { str(Label(":is_python_3.11_linux_aarch64")): "matched-3.11-linux-aarch64", - str(Label(":is_python_3.11_linux_ppc")): "matched-3.11-linux-ppc", + str(Label(":is_python_3.11_linux_ppc64le")): "matched-3.11-linux-ppc64le", str(Label(":is_python_3.11_linux_s390x")): "matched-3.11-linux-s390x", str(Label(":is_python_3.11_linux_x86_64")): "matched-3.11-linux-x86_64", str(Label(":is_python_3.11_osx_aarch64")): "matched-3.11-osx-aarch64", @@ -171,7 +171,7 @@ def construct_config_settings_test_suite(name): # buildifier: disable=function- }, ) - for cpu in ["s390x", "ppc", "x86_64", "aarch64"]: + for cpu in ["s390x", "ppc", "ppc64le", "x86_64", "aarch64"]: native.config_setting( name = "is_python_3.11_" + cpu, constraint_values = [ @@ -185,6 +185,7 @@ def construct_config_settings_test_suite(name): # buildifier: disable=function- for (os, cpu) in [ ("linux", "aarch64"), ("linux", "ppc"), + ("linux", "ppc64le"), ("linux", "s390x"), ("linux", "x86_64"), ("osx", "aarch64"), diff --git a/tests/pypi/whl_installer/platform_test.py b/tests/pypi/whl_installer/platform_test.py index 7ced1e9826..2aeb4caa69 100644 --- a/tests/pypi/whl_installer/platform_test.py +++ b/tests/pypi/whl_installer/platform_test.py @@ -34,17 +34,17 @@ def test_can_get_specific_from_string(self): def test_can_get_all_for_py_version(self): cp39 = Platform.all(minor_version=9) - self.assertEqual(18, len(cp39), f"Got {cp39}") + self.assertEqual(21, len(cp39), f"Got {cp39}") self.assertEqual(cp39, Platform.from_string("cp39_*")) def test_can_get_all_for_os(self): linuxes = Platform.all(OS.linux, minor_version=9) - self.assertEqual(6, len(linuxes)) + self.assertEqual(7, len(linuxes)) self.assertEqual(linuxes, Platform.from_string("cp39_linux_*")) def test_can_get_all_for_os_for_host_python(self): linuxes = Platform.all(OS.linux) - self.assertEqual(6, len(linuxes)) + self.assertEqual(7, len(linuxes)) self.assertEqual(linuxes, Platform.from_string("linux_*")) def test_specific_version_specializations(self): @@ -84,6 +84,7 @@ def test_linux_specializations(self): Platform(os=OS.linux, arch=Arch.x86_32), Platform(os=OS.linux, arch=Arch.aarch64), Platform(os=OS.linux, arch=Arch.ppc), + Platform(os=OS.linux, arch=Arch.ppc64le), Platform(os=OS.linux, arch=Arch.s390x), Platform(os=OS.linux, arch=Arch.arm), ] @@ -101,6 +102,7 @@ def test_osx_specializations(self): Platform(os=OS.osx, arch=Arch.x86_32), Platform(os=OS.osx, arch=Arch.aarch64), Platform(os=OS.osx, arch=Arch.ppc), + Platform(os=OS.osx, arch=Arch.ppc64le), Platform(os=OS.osx, arch=Arch.s390x), Platform(os=OS.osx, arch=Arch.arm), ] diff --git a/tests/pypi/whl_library_targets/whl_library_targets_tests.bzl b/tests/pypi/whl_library_targets/whl_library_targets_tests.bzl index ba04e1d887..a042ed0346 100644 --- a/tests/pypi/whl_library_targets/whl_library_targets_tests.bzl +++ b/tests/pypi/whl_library_targets/whl_library_targets_tests.bzl @@ -68,7 +68,7 @@ def _test_platforms(env): "@//python/config_settings:is_python_3.9": ["py39_dep"], "@platforms//cpu:aarch64": ["arm_dep"], "@platforms//os:windows": ["win_dep"], - "cp310_linux_ppc": ["py310_linux_ppc_dep"], + "cp310_linux_ppc64le": ["py310_linux_ppc64le_dep"], "cp39_anyos_aarch64": ["py39_arm_dep"], "cp39_linux_anyarch": ["py39_linux_dep"], "linux_x86_64": ["linux_intel_dep"], @@ -82,12 +82,12 @@ def _test_platforms(env): env.expect.that_collection(calls).contains_exactly([ { - "name": "is_python_3.10_linux_ppc", + "name": "is_python_3.10_linux_ppc64le", "flag_values": { "@rules_python//python/config_settings:python_version_major_minor": "3.10", }, "constraint_values": [ - "@platforms//cpu:ppc", + "@platforms//cpu:ppc64le", "@platforms//os:linux", ], "visibility": ["//visibility:private"], @@ -195,7 +195,7 @@ def _test_whl_and_library_deps(env): "@//python/config_settings:is_python_3.9": ["py39_dep"], "@platforms//cpu:aarch64": ["arm_dep"], "@platforms//os:windows": ["win_dep"], - "cp310_linux_ppc": ["py310_linux_ppc_dep"], + "cp310_linux_ppc64le": ["py310_linux_ppc64le_dep"], "cp39_anyos_aarch64": ["py39_arm_dep"], "cp39_linux_anyarch": ["py39_linux_dep"], "linux_x86_64": ["linux_intel_dep"], @@ -227,7 +227,7 @@ def _test_whl_and_library_deps(env): Label("//python/config_settings:is_python_3.9"): ["@pypi_py39_dep//:whl"], "@platforms//cpu:aarch64": ["@pypi_arm_dep//:whl"], "@platforms//os:windows": ["@pypi_win_dep//:whl"], - ":is_python_3.10_linux_ppc": ["@pypi_py310_linux_ppc_dep//:whl"], + ":is_python_3.10_linux_ppc64le": ["@pypi_py310_linux_ppc64le_dep//:whl"], ":is_python_3.9_anyos_aarch64": ["@pypi_py39_arm_dep//:whl"], ":is_python_3.9_linux_anyarch": ["@pypi_py39_linux_dep//:whl"], ":is_linux_x86_64": ["@pypi_linux_intel_dep//:whl"], @@ -264,7 +264,7 @@ def _test_whl_and_library_deps(env): Label("//python/config_settings:is_python_3.9"): ["@pypi_py39_dep//:pkg"], "@platforms//cpu:aarch64": ["@pypi_arm_dep//:pkg"], "@platforms//os:windows": ["@pypi_win_dep//:pkg"], - ":is_python_3.10_linux_ppc": ["@pypi_py310_linux_ppc_dep//:pkg"], + ":is_python_3.10_linux_ppc64le": ["@pypi_py310_linux_ppc64le_dep//:pkg"], ":is_python_3.9_anyos_aarch64": ["@pypi_py39_arm_dep//:pkg"], ":is_python_3.9_linux_anyarch": ["@pypi_py39_linux_dep//:pkg"], ":is_linux_x86_64": ["@pypi_linux_intel_dep//:pkg"], diff --git a/tests/pypi/whl_target_platforms/whl_target_platforms_tests.bzl b/tests/pypi/whl_target_platforms/whl_target_platforms_tests.bzl index a72bdc275f..a976a0cf95 100644 --- a/tests/pypi/whl_target_platforms/whl_target_platforms_tests.bzl +++ b/tests/pypi/whl_target_platforms/whl_target_platforms_tests.bzl @@ -32,7 +32,7 @@ def _test_simple(env): struct(os = "linux", cpu = "x86_32", abi = None, target_platform = "linux_x86_32", version = (2, 17)), ], "musllinux_1_1_ppc64le": [ - struct(os = "linux", cpu = "ppc", abi = None, target_platform = "linux_ppc", version = (1, 1)), + struct(os = "linux", cpu = "ppc64le", abi = None, target_platform = "linux_ppc64le", version = (1, 1)), ], "win_amd64": [ struct(os = "windows", cpu = "x86_64", abi = None, target_platform = "windows_x86_64", version = (0, 0)), @@ -60,9 +60,12 @@ def _test_with_abi(env): "manylinux1_i686.manylinux_2_17_i686": [ struct(os = "linux", cpu = "x86_32", abi = "cp38", target_platform = "cp38_linux_x86_32", version = (0, 0)), ], - "musllinux_1_1_ppc64le": [ + "musllinux_1_1_ppc64": [ struct(os = "linux", cpu = "ppc", abi = "cp311", target_platform = "cp311_linux_ppc", version = (1, 1)), ], + "musllinux_1_1_ppc64le": [ + struct(os = "linux", cpu = "ppc64le", abi = "cp311", target_platform = "cp311_linux_ppc64le", version = (1, 1)), + ], "win_amd64": [ struct(os = "windows", cpu = "x86_64", abi = "cp311", target_platform = "cp311_windows_x86_64", version = (0, 0)), ], From fcf7221c1e079307ff13d32239b7782d2f1dc48c Mon Sep 17 00:00:00 2001 From: Jimmy Tanner Date: Tue, 25 Feb 2025 10:34:27 -0800 Subject: [PATCH 085/922] fix: Gazelle bug with merging py_binary targets in per-file mode and partial update (#2619) This PR adds a new unit test. Currently, this is just a failing test without a fix, and I am still trying to understand the code well enough to find the root cause of the issue. Our team uses Python+Gazelle in a monorepo, and we have a handful of directories with multiple `.py` files containing `if __name__ == "__main__"`. Most of the time these are present for convenience or ad-hoc invocation. We're aware of the [recommendation to split these into separate files](https://github.com/bazelbuild/rules_python/tree/main/gazelle#binaries), but that can cause clutter, and it is non-obvious to most engineers what to do when encountering this issue, which presents either as a misleading error message or a no-op without creating the appropriate targets. **Update** This bug occurs when ALL of the following are true: * `python_generation_mode` is set to `file`. * Multiple python binary files (files with `if __name__ == "__main__"`) exist in the same directory. * The directory has no `__main__.py` file. * The `BUILD` file in the directory is partially complete, i.e. it contains `py_binary` targets for some of the python files, but not others. In this situation, previously absent `py_binary` targets are merged into existing `py_binary` targets instead of being created as new targets. --------- Co-authored-by: Jimmy Tanner --- CHANGELOG.md | 2 ++ gazelle/python/kinds.go | 3 ++- .../BUILD.in | 9 +++++++++ .../BUILD.out | 15 +++++++++++++++ .../README.md | 3 +++ .../WORKSPACE | 1 + .../a.py | 2 ++ .../b.py | 2 ++ .../test.yaml | 17 +++++++++++++++++ 9 files changed, 53 insertions(+), 1 deletion(-) create mode 100644 gazelle/python/testdata/binary_without_entrypoint_per_file_generation_partial_update/BUILD.in create mode 100644 gazelle/python/testdata/binary_without_entrypoint_per_file_generation_partial_update/BUILD.out create mode 100644 gazelle/python/testdata/binary_without_entrypoint_per_file_generation_partial_update/README.md create mode 100644 gazelle/python/testdata/binary_without_entrypoint_per_file_generation_partial_update/WORKSPACE create mode 100644 gazelle/python/testdata/binary_without_entrypoint_per_file_generation_partial_update/a.py create mode 100644 gazelle/python/testdata/binary_without_entrypoint_per_file_generation_partial_update/b.py create mode 100644 gazelle/python/testdata/binary_without_entrypoint_per_file_generation_partial_update/test.yaml diff --git a/CHANGELOG.md b/CHANGELOG.md index 8a62ab7840..1c075af80b 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -57,6 +57,8 @@ Unreleased changes template. {#v0-0-0-fixed} ### Fixed * (pypi) The `ppc64le` is now pointing to the right target in the `platforms` package. +* (gazelle) No longer incorrectly merge `py_binary` targets during partial updates in + `file` generation mode. Fixed in [#2619](https://github.com/bazelbuild/rules_python/pull/2619). {#v0-0-0-added} ### Added diff --git a/gazelle/python/kinds.go b/gazelle/python/kinds.go index a9483372e2..7a0639abd3 100644 --- a/gazelle/python/kinds.go +++ b/gazelle/python/kinds.go @@ -32,7 +32,8 @@ func (*Python) Kinds() map[string]rule.KindInfo { var pyKinds = map[string]rule.KindInfo{ pyBinaryKind: { - MatchAny: true, + MatchAny: false, + MatchAttrs: []string{"srcs"}, NonEmptyAttrs: map[string]bool{ "deps": true, "main": true, diff --git a/gazelle/python/testdata/binary_without_entrypoint_per_file_generation_partial_update/BUILD.in b/gazelle/python/testdata/binary_without_entrypoint_per_file_generation_partial_update/BUILD.in new file mode 100644 index 0000000000..63b547f0b3 --- /dev/null +++ b/gazelle/python/testdata/binary_without_entrypoint_per_file_generation_partial_update/BUILD.in @@ -0,0 +1,9 @@ +load("@rules_python//python:defs.bzl", "py_binary") + +# gazelle:python_generation_mode file + +py_binary( + name = "a", + srcs = ["a.py"], + visibility = ["//:__subpackages__"], +) diff --git a/gazelle/python/testdata/binary_without_entrypoint_per_file_generation_partial_update/BUILD.out b/gazelle/python/testdata/binary_without_entrypoint_per_file_generation_partial_update/BUILD.out new file mode 100644 index 0000000000..8f49cccd9f --- /dev/null +++ b/gazelle/python/testdata/binary_without_entrypoint_per_file_generation_partial_update/BUILD.out @@ -0,0 +1,15 @@ +load("@rules_python//python:defs.bzl", "py_binary") + +# gazelle:python_generation_mode file + +py_binary( + name = "a", + srcs = ["a.py"], + visibility = ["//:__subpackages__"], +) + +py_binary( + name = "b", + srcs = ["b.py"], + visibility = ["//:__subpackages__"], +) diff --git a/gazelle/python/testdata/binary_without_entrypoint_per_file_generation_partial_update/README.md b/gazelle/python/testdata/binary_without_entrypoint_per_file_generation_partial_update/README.md new file mode 100644 index 0000000000..5aa499f4ad --- /dev/null +++ b/gazelle/python/testdata/binary_without_entrypoint_per_file_generation_partial_update/README.md @@ -0,0 +1,3 @@ +# Partial update with multiple per-file binaries + +This test case asserts that when there are multiple binaries in a package, and no __main__.py, and the BUILD file already includes a py_binary for one of the files, a py_binary is generated for the other file. diff --git a/gazelle/python/testdata/binary_without_entrypoint_per_file_generation_partial_update/WORKSPACE b/gazelle/python/testdata/binary_without_entrypoint_per_file_generation_partial_update/WORKSPACE new file mode 100644 index 0000000000..faff6af87a --- /dev/null +++ b/gazelle/python/testdata/binary_without_entrypoint_per_file_generation_partial_update/WORKSPACE @@ -0,0 +1 @@ +# This is a Bazel workspace for the Gazelle test data. diff --git a/gazelle/python/testdata/binary_without_entrypoint_per_file_generation_partial_update/a.py b/gazelle/python/testdata/binary_without_entrypoint_per_file_generation_partial_update/a.py new file mode 100644 index 0000000000..9c97da4809 --- /dev/null +++ b/gazelle/python/testdata/binary_without_entrypoint_per_file_generation_partial_update/a.py @@ -0,0 +1,2 @@ +if __name__ == "__main__": + print("Hello, world!") diff --git a/gazelle/python/testdata/binary_without_entrypoint_per_file_generation_partial_update/b.py b/gazelle/python/testdata/binary_without_entrypoint_per_file_generation_partial_update/b.py new file mode 100644 index 0000000000..9c97da4809 --- /dev/null +++ b/gazelle/python/testdata/binary_without_entrypoint_per_file_generation_partial_update/b.py @@ -0,0 +1,2 @@ +if __name__ == "__main__": + print("Hello, world!") diff --git a/gazelle/python/testdata/binary_without_entrypoint_per_file_generation_partial_update/test.yaml b/gazelle/python/testdata/binary_without_entrypoint_per_file_generation_partial_update/test.yaml new file mode 100644 index 0000000000..346ecd7ae8 --- /dev/null +++ b/gazelle/python/testdata/binary_without_entrypoint_per_file_generation_partial_update/test.yaml @@ -0,0 +1,17 @@ +# Copyright 2025 The Bazel Authors. All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +--- +expect: + exit_code: 0 From effdce8d284d6ac1fe1bc1ded5ff02444547d90b Mon Sep 17 00:00:00 2001 From: Ed Schouten Date: Thu, 27 Feb 2025 06:29:23 +0100 Subject: [PATCH 086/922] refactor: stop using some deprecated Starlark APIs (#2626) I am currently working on an analysis tool that is capable of parsing BUILD/*.bzl files. It currently fails to process some of the Python rules, due to the rules depending on some features that are deprecated on the Bazel side. Instead of adding implementations of these deprecated features to my brand new analysis tool, I thought I'd simply patch up the Python rules instead. --- gazelle/manifest/defs.bzl | 2 +- python/private/pypi/multi_pip_parse.bzl | 4 +- python/private/pypi/whl_library_alias.bzl | 2 +- python/private/pythons_hub.bzl | 2 +- python/private/toolchains_repo.bzl | 40 +++++++++---------- .../pycross/private/wheel_library.bzl | 2 +- 6 files changed, 24 insertions(+), 28 deletions(-) diff --git a/gazelle/manifest/defs.bzl b/gazelle/manifest/defs.bzl index 6c0072a48b..45fdb32e7d 100644 --- a/gazelle/manifest/defs.bzl +++ b/gazelle/manifest/defs.bzl @@ -161,7 +161,7 @@ AllSourcesInfo = provider(fields = {"all_srcs": "All sources collected from the _rules_python_workspace = Label("@rules_python//:WORKSPACE") def _get_all_sources_impl(target, ctx): - is_rules_python = target.label.workspace_name == _rules_python_workspace.workspace_name + is_rules_python = target.label.repo_name == _rules_python_workspace.repo_name if not is_rules_python: # Avoid adding third-party dependency files to the checksum of the srcs. return AllSourcesInfo(all_srcs = depset()) diff --git a/python/private/pypi/multi_pip_parse.bzl b/python/private/pypi/multi_pip_parse.bzl index 6e824f674c..60496c2eca 100644 --- a/python/private/pypi/multi_pip_parse.bzl +++ b/python/private/pypi/multi_pip_parse.bzl @@ -18,7 +18,7 @@ load("//python/private:text_util.bzl", "render") load(":pip_repository.bzl", pip_parse = "pip_repository") def _multi_pip_parse_impl(rctx): - rules_python = rctx.attr._rules_python_workspace.workspace_name + rules_python = rctx.attr._rules_python_workspace.repo_name load_statements = [] install_deps_calls = [] process_requirements_calls = [] @@ -69,7 +69,7 @@ def _process_requirements(pkg_labels, python_version, repo_prefix): wheel_name = Label(pkg_label).package if not wheel_name: # We are dealing with the cases where we don't have aliases. - workspace_name = Label(pkg_label).workspace_name + workspace_name = Label(pkg_label).repo_name wheel_name = workspace_name[len(repo_prefix):] _wheel_names.append(wheel_name) diff --git a/python/private/pypi/whl_library_alias.bzl b/python/private/pypi/whl_library_alias.bzl index d34b34a51a..66c3504d90 100644 --- a/python/private/pypi/whl_library_alias.bzl +++ b/python/private/pypi/whl_library_alias.bzl @@ -18,7 +18,7 @@ load("//python/private:full_version.bzl", "full_version") load(":render_pkg_aliases.bzl", "NO_MATCH_ERROR_MESSAGE_TEMPLATE") def _whl_library_alias_impl(rctx): - rules_python = rctx.attr._rules_python_workspace.workspace_name + rules_python = rctx.attr._rules_python_workspace.repo_name if rctx.attr.default_version: default_repo_prefix = rctx.attr.version_map[rctx.attr.default_version] else: diff --git a/python/private/pythons_hub.bzl b/python/private/pythons_hub.bzl index ac928ffc96..b448d53097 100644 --- a/python/private/pythons_hub.bzl +++ b/python/private/pythons_hub.bzl @@ -79,7 +79,7 @@ def _hub_build_file_content( return _HUB_BUILD_FILE_TEMPLATE.format( toolchains = toolchains, - rules_python = workspace_location.workspace_name, + rules_python = workspace_location.repo_name, ) _interpreters_bzl_template = """ diff --git a/python/private/toolchains_repo.bzl b/python/private/toolchains_repo.bzl index 5082047135..4e4a5de501 100644 --- a/python/private/toolchains_repo.bzl +++ b/python/private/toolchains_repo.bzl @@ -31,10 +31,6 @@ load( load(":repo_utils.bzl", "REPO_DEBUG_ENV_VAR", "repo_utils") load(":text_util.bzl", "render") -def get_repository_name(repository_workspace): - dummy_label = "//:_" - return str(repository_workspace.relative(dummy_label))[:-len(dummy_label)] or "@" - def python_toolchain_build_file_content( prefix, python_version, @@ -90,10 +86,10 @@ def _toolchains_repo_impl(rctx): # python_register_toolchains macro so you don't normally need to interact with # these targets. -load("@{rules_python}//python/private:py_toolchain_suite.bzl", "py_toolchain_suite") +load("@@{rules_python}//python/private:py_toolchain_suite.bzl", "py_toolchain_suite") """.format( - rules_python = rctx.attr._rules_python_workspace.workspace_name, + rules_python = rctx.attr._rules_python_workspace.repo_name, ) toolchains = python_toolchain_build_file_content( @@ -151,13 +147,13 @@ toolchain_aliases( rctx.file("defs.bzl", content = """\ # Generated by python/private/toolchains_repo.bzl -load("{rules_python}//python:pip.bzl", _compile_pip_requirements = "compile_pip_requirements") -load("{rules_python}//python/private:deprecation.bzl", "with_deprecation") -load("{rules_python}//python/private:text_util.bzl", "render") -load("{rules_python}//python:py_binary.bzl", _py_binary = "py_binary") -load("{rules_python}//python:py_test.bzl", _py_test = "py_test") +load("@@{rules_python}//python:pip.bzl", _compile_pip_requirements = "compile_pip_requirements") +load("@@{rules_python}//python/private:deprecation.bzl", "with_deprecation") +load("@@{rules_python}//python/private:text_util.bzl", "render") +load("@@{rules_python}//python:py_binary.bzl", _py_binary = "py_binary") +load("@@{rules_python}//python:py_test.bzl", _py_test = "py_test") load( - "{rules_python}//python/entry_points:py_console_script_binary.bzl", + "@@{rules_python}//python/entry_points:py_console_script_binary.bzl", _py_console_script_binary = "py_console_script_binary", ) @@ -185,7 +181,7 @@ def compile_pip_requirements(**kwargs): """.format( name = rctx.attr.name, python_version = rctx.attr.python_version, - rules_python = get_repository_name(rctx.attr._rules_python_workspace), + rules_python = rctx.attr._rules_python_workspace.repo_name, )) toolchain_aliases = repository_rule( @@ -301,20 +297,20 @@ this repo causes an eager fetch of the toolchain for the host platform. ) def _multi_toolchain_aliases_impl(rctx): - rules_python = rctx.attr._rules_python_workspace.workspace_name + rules_python = rctx.attr._rules_python_workspace.repo_name for python_version, repository_name in rctx.attr.python_versions.items(): file = "{}/defs.bzl".format(python_version) rctx.file(file, content = """\ # Generated by python/private/toolchains_repo.bzl -load("{rules_python}//python:pip.bzl", _compile_pip_requirements = "compile_pip_requirements") -load("{rules_python}//python/private:deprecation.bzl", "with_deprecation") -load("{rules_python}//python/private:text_util.bzl", "render") -load("{rules_python}//python:py_binary.bzl", _py_binary = "py_binary") -load("{rules_python}//python:py_test.bzl", _py_test = "py_test") +load("@@{rules_python}//python:pip.bzl", _compile_pip_requirements = "compile_pip_requirements") +load("@@{rules_python}//python/private:deprecation.bzl", "with_deprecation") +load("@@{rules_python}//python/private:text_util.bzl", "render") +load("@@{rules_python}//python:py_binary.bzl", _py_binary = "py_binary") +load("@@{rules_python}//python:py_test.bzl", _py_test = "py_test") load( - "{rules_python}//python/entry_points:py_console_script_binary.bzl", + "@@{rules_python}//python/entry_points:py_console_script_binary.bzl", _py_console_script_binary = "py_console_script_binary", ) @@ -343,14 +339,14 @@ def compile_pip_requirements(**kwargs): repository_name = repository_name, name = rctx.attr.name, python_version = python_version, - rules_python = get_repository_name(rctx.attr._rules_python_workspace), + rules_python = rules_python, )) rctx.file("{}/BUILD.bazel".format(python_version), "") pip_bzl = """\ # Generated by python/private/toolchains_repo.bzl -load("@{rules_python}//python:pip.bzl", "pip_parse", _multi_pip_parse = "multi_pip_parse") +load("@@{rules_python}//python:pip.bzl", "pip_parse", _multi_pip_parse = "multi_pip_parse") def multi_pip_parse(name, requirements_lock, **kwargs): return _multi_pip_parse( diff --git a/third_party/rules_pycross/pycross/private/wheel_library.bzl b/third_party/rules_pycross/pycross/private/wheel_library.bzl index 3d6ee32562..00d85f71b1 100644 --- a/third_party/rules_pycross/pycross/private/wheel_library.bzl +++ b/third_party/rules_pycross/pycross/private/wheel_library.bzl @@ -83,7 +83,7 @@ def _py_wheel_library_impl(ctx): # TODO: Is there a more correct way to get this runfiles-relative import path? imp = paths.join( - ctx.label.workspace_name or ctx.workspace_name, # Default to the local workspace. + ctx.label.repo_name or ctx.workspace_name, # Default to the local workspace. ctx.label.package, ctx.label.name, "site-packages", # we put lib files in this subdirectory. From bb6249bf2f3786ed9e27fcfeb74b3762bf9eb1cb Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Fri, 28 Feb 2025 02:39:16 -0800 Subject: [PATCH 087/922] docs: fix changelog header for 1.2.0 entry (#2635) When adding the 1.2 section, everything was updated exception the section title. --- CHANGELOG.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 1c075af80b..e447012c98 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -71,7 +71,7 @@ Unreleased changes template. * Nothing removed. {#v1-2-0} -## Unreleased +## [1.2.0] - 2025-02-21 [1.2.0]: https://github.com/bazelbuild/rules_python/releases/tag/1.2.0 From c7aa9893c146e33a5e76dbbd83115e91a8836021 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?X=C3=B9d=C5=8Dng=20Y=C3=A1ng?= Date: Mon, 3 Mar 2025 02:35:43 -0500 Subject: [PATCH 088/922] fix: Downgrade "running as root" error to a warning by default (#2636) Currently, by default, rules_python immediately fails when Bazel is run as root. The reasoning behind this involves .pyc files being generated for hermetic toolchains when they're first used, causing cache misses; to work around this, rules_python opts to make the toolchain installation directory read-only, but running Bazel as root would circumvent this. So rules_python actively detects if the current user is root, and hard fails. This check can be disabled by the root module by setting `python.override(ignore_root_user_error=True)`. (See more context in the linked issues/PRs.) This causes a reverberating effect across the Bazel ecosystem, as rules_python is essentially a dependency of every single Bazel project through protobuf. Effectively, any Bazel project wishing to run as root need to add the override tag above, even if they don't have anything to do with Python at all. This PR changes the default value of the `ignore_root_user_error` to True instead. Besides, it now unconditionally tries to make the toolchain installation directory read-only, and only outputs a warning if it's detected that the current user is root. See previous discussions at #713, #749, #907, #1008, #1169, etc. Fixes https://github.com/bazelbuild/rules_python/issues/1169. --------- Co-authored-by: Richard Levasseur --- CHANGELOG.md | 4 ++ python/private/python.bzl | 39 ++++++++------------ python/private/python_repository.bzl | 55 ++++++++++++++-------------- tests/python/python_tests.bzl | 50 +++++-------------------- 4 files changed, 56 insertions(+), 92 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index e447012c98..849b458745 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -59,6 +59,10 @@ Unreleased changes template. * (pypi) The `ppc64le` is now pointing to the right target in the `platforms` package. * (gazelle) No longer incorrectly merge `py_binary` targets during partial updates in `file` generation mode. Fixed in [#2619](https://github.com/bazelbuild/rules_python/pull/2619). +* (bzlmod) Running as root is no longer an error. `ignore_root_user_error=True` + is now the default. Note that running as root may still cause spurious + Bazel cache invalidation + ([#1169](https://github.com/bazelbuild/rules_python/issues/1169)). {#v0-0-0-added} ### Added diff --git a/python/private/python.bzl b/python/private/python.bzl index ec6f73e41f..304a1d7745 100644 --- a/python/private/python.bzl +++ b/python/private/python.bzl @@ -72,9 +72,9 @@ def parse_modules(*, module_ctx, _fail = fail): logger = repo_utils.logger(module_ctx, "python") # if the root module does not register any toolchain then the - # ignore_root_user_error takes its default value: False + # ignore_root_user_error takes its default value: True if not module_ctx.modules[0].tags.toolchain: - ignore_root_user_error = False + ignore_root_user_error = True config = _get_toolchain_config(modules = module_ctx.modules, _fail = _fail) @@ -559,7 +559,7 @@ def _create_toolchain_attrs_struct(*, tag = None, python_version = None, toolcha is_default = is_default, python_version = python_version if python_version else tag.python_version, configure_coverage_tool = getattr(tag, "configure_coverage_tool", False), - ignore_root_user_error = getattr(tag, "ignore_root_user_error", False), + ignore_root_user_error = getattr(tag, "ignore_root_user_error", True), ) def _get_bazel_version_specific_kwargs(): @@ -636,16 +636,18 @@ Then the python interpreter will be available as `my_python_name`. doc = "Whether or not to configure the default coverage tool provided by `rules_python` for the compatible toolchains.", ), "ignore_root_user_error": attr.bool( - default = False, + default = True, doc = """\ -If `False`, the Python runtime installation will be made read only. This improves -the ability for Bazel to cache it, but prevents the interpreter from creating -`.pyc` files for the standard library dynamically at runtime as they are loaded. - -If `True`, the Python runtime installation is read-write. This allows the -interpreter to create `.pyc` files for the standard library, but, because they are -created as needed, it adversely affects Bazel's ability to cache the runtime and -can result in spurious build failures. +The Python runtime installation is made read only. This improves the ability for +Bazel to cache it by preventing the interpreter from creating `.pyc` files for +the standard library dynamically at runtime as they are loaded (this often leads +to spurious cache misses or build failures). + +However, if the user is running Bazel as root, this read-onlyness is not +respected. Bazel will print a warning message when it detects that the runtime +installation is writable despite being made read only (i.e. it's running with +root access). If this attribute is set to `False`, Bazel will make it a hard +error to run with root access instead. """, mandatory = False, ), @@ -690,17 +692,8 @@ dependencies are introduced. default = DEFAULT_RELEASE_BASE_URL, ), "ignore_root_user_error": attr.bool( - default = False, - doc = """\ -If `False`, the Python runtime installation will be made read only. This improves -the ability for Bazel to cache it, but prevents the interpreter from creating -`.pyc` files for the standard library dynamically at runtime as they are loaded. - -If `True`, the Python runtime installation is read-write. This allows the -interpreter to create `.pyc` files for the standard library, but, because they are -created as needed, it adversely affects Bazel's ability to cache the runtime and -can result in spurious build failures. -""", + default = True, + doc = """Deprecated; do not use. This attribute has no effect.""", mandatory = False, ), "minor_mapping": attr.string_dict( diff --git a/python/private/python_repository.bzl b/python/private/python_repository.bzl index c7407c8f2c..075d4b1195 100644 --- a/python/private/python_repository.bzl +++ b/python/private/python_repository.bzl @@ -127,37 +127,36 @@ def _python_repository_impl(rctx): # pycs being generated at runtime: # * The pycs are not deterministic (they contain timestamps) # * Multiple processes trying to write the same pycs can result in errors. - if not rctx.attr.ignore_root_user_error: - if "windows" not in platform: - lib_dir = "lib" if "windows" not in platform else "Lib" + if "windows" not in platform: + repo_utils.execute_checked( + rctx, + op = "python_repository.MakeReadOnly", + arguments = [repo_utils.which_checked(rctx, "chmod"), "-R", "ugo-w", "lib"], + logger = logger, + ) - repo_utils.execute_checked( - rctx, - op = "python_repository.MakeReadOnly", - arguments = [repo_utils.which_checked(rctx, "chmod"), "-R", "ugo-w", lib_dir], - logger = logger, - ) - exec_result = repo_utils.execute_unchecked( + fail_or_warn = logger.warn if rctx.attr.ignore_root_user_error else logger.fail + exec_result = repo_utils.execute_unchecked( + rctx, + op = "python_repository.TestReadOnly", + arguments = [repo_utils.which_checked(rctx, "touch"), "lib/.test"], + logger = logger, + ) + + # The issue with running as root is the installation is no longer + # read-only, so the problems due to pyc can resurface. + if exec_result.return_code == 0: + stdout = repo_utils.execute_checked_stdout( rctx, - op = "python_repository.TestReadOnly", - arguments = [repo_utils.which_checked(rctx, "touch"), "{}/.test".format(lib_dir)], + op = "python_repository.GetUserId", + arguments = [repo_utils.which_checked(rctx, "id"), "-u"], logger = logger, ) - - # The issue with running as root is the installation is no longer - # read-only, so the problems due to pyc can resurface. - if exec_result.return_code == 0: - stdout = repo_utils.execute_checked_stdout( - rctx, - op = "python_repository.GetUserId", - arguments = [repo_utils.which_checked(rctx, "id"), "-u"], - logger = logger, - ) - uid = int(stdout.strip()) - if uid == 0: - fail("The current user is root, please run as non-root when using the hermetic Python interpreter. See https://github.com/bazelbuild/rules_python/pull/713.") - else: - fail("The current user has CAP_DAC_OVERRIDE set, please drop this capability when using the hermetic Python interpreter. See https://github.com/bazelbuild/rules_python/pull/713.") + uid = int(stdout.strip()) + if uid == 0: + fail_or_warn("The current user is root, which can cause spurious cache misses or build failures with the hermetic Python interpreter. See https://github.com/bazelbuild/rules_python/pull/713.") + else: + fail_or_warn("The current user has CAP_DAC_OVERRIDE set, which can cause spurious cache misses or build failures with the hermetic Python interpreter. See https://github.com/bazelbuild/rules_python/pull/713.") python_bin = "python.exe" if ("windows" in platform) else "bin/python3" @@ -294,7 +293,7 @@ For more information see {attr}`py_runtime.coverage_tool`. mandatory = False, ), "ignore_root_user_error": attr.bool( - default = False, + default = True, doc = "Whether the check for root should be ignored or not. This causes cache misses with .pyc files.", mandatory = False, ), diff --git a/tests/python/python_tests.bzl b/tests/python/python_tests.bzl index e7828b92f5..6552251331 100644 --- a/tests/python/python_tests.bzl +++ b/tests/python/python_tests.bzl @@ -62,7 +62,7 @@ def _override( auth_patterns = {}, available_python_versions = [], base_url = "", - ignore_root_user_error = False, + ignore_root_user_error = True, minor_mapping = {}, netrc = "", register_all_versions = False): @@ -139,7 +139,7 @@ def _test_default(env): "ignore_root_user_error", "tool_versions", ]) - env.expect.that_bool(py.config.default["ignore_root_user_error"]).equals(False) + env.expect.that_bool(py.config.default["ignore_root_user_error"]).equals(True) env.expect.that_str(py.default_python_version).equals("3.11") want_toolchain = struct( @@ -212,13 +212,13 @@ def _test_default_non_rules_python_ignore_root_user_error(env): module_ctx = _mock_mctx( _mod( name = "my_module", - toolchain = [_toolchain("3.12", ignore_root_user_error = True)], + toolchain = [_toolchain("3.12", ignore_root_user_error = False)], ), _mod(name = "rules_python", toolchain = [_toolchain("3.11")]), ), ) - env.expect.that_bool(py.config.default["ignore_root_user_error"]).equals(True) + env.expect.that_bool(py.config.default["ignore_root_user_error"]).equals(False) env.expect.that_str(py.default_python_version).equals("3.12") my_module_toolchain = struct( @@ -238,49 +238,17 @@ def _test_default_non_rules_python_ignore_root_user_error(env): _tests.append(_test_default_non_rules_python_ignore_root_user_error) -def _test_default_non_rules_python_ignore_root_user_error_override(env): - py = parse_modules( - module_ctx = _mock_mctx( - _mod( - name = "my_module", - toolchain = [_toolchain("3.12")], - override = [_override(ignore_root_user_error = True)], - ), - _mod(name = "rules_python", toolchain = [_toolchain("3.11")]), - ), - ) - - env.expect.that_bool(py.config.default["ignore_root_user_error"]).equals(True) - env.expect.that_str(py.default_python_version).equals("3.12") - - my_module_toolchain = struct( - name = "python_3_12", - python_version = "3.12", - register_coverage_tool = False, - ) - rules_python_toolchain = struct( - name = "python_3_11", - python_version = "3.11", - register_coverage_tool = False, - ) - env.expect.that_collection(py.toolchains).contains_exactly([ - rules_python_toolchain, - my_module_toolchain, - ]).in_order() - -_tests.append(_test_default_non_rules_python_ignore_root_user_error_override) - def _test_default_non_rules_python_ignore_root_user_error_non_root_module(env): py = parse_modules( module_ctx = _mock_mctx( _mod(name = "my_module", toolchain = [_toolchain("3.13")]), - _mod(name = "some_module", toolchain = [_toolchain("3.12", ignore_root_user_error = True)]), + _mod(name = "some_module", toolchain = [_toolchain("3.12", ignore_root_user_error = False)]), _mod(name = "rules_python", toolchain = [_toolchain("3.11")]), ), ) env.expect.that_str(py.default_python_version).equals("3.13") - env.expect.that_bool(py.config.default["ignore_root_user_error"]).equals(False) + env.expect.that_bool(py.config.default["ignore_root_user_error"]).equals(True) my_module_toolchain = struct( name = "python_3_13", @@ -338,8 +306,8 @@ def _test_first_occurance_of_the_toolchain_wins(env): env.expect.that_dict(py.debug_info).contains_exactly({ "toolchains_registered": [ - {"ignore_root_user_error": False, "module": {"is_root": True, "name": "my_module"}, "name": "python_3_12"}, - {"ignore_root_user_error": False, "module": {"is_root": False, "name": "rules_python"}, "name": "python_3_11"}, + {"ignore_root_user_error": True, "module": {"is_root": True, "name": "my_module"}, "name": "python_3_12"}, + {"ignore_root_user_error": True, "module": {"is_root": False, "name": "rules_python"}, "name": "python_3_11"}, ], }) @@ -364,7 +332,7 @@ def _test_auth_overrides(env): env.expect.that_dict(py.config.default).contains_at_least({ "auth_patterns": {"foo": "bar"}, - "ignore_root_user_error": False, + "ignore_root_user_error": True, "netrc": "/my/netrc", }) env.expect.that_str(py.default_python_version).equals("3.12") From f4fde65a4e079f7d76e82c6bf05acc5dbd9091ea Mon Sep 17 00:00:00 2001 From: Matt Mackay Date: Mon, 3 Mar 2025 09:16:08 -0500 Subject: [PATCH 089/922] fix: spill module mapping args to a file (#2644) Calls to the modules mapping rule contains very long command line args due to the use of the full `wheels` parameter. This change adds support for spilling the args into a file as needed. In addition, it improves the performance of the `modules_mapping` rule: * Remove the calls `to_list` that are unnecessary on the depset. * Remove the iteration over the depset when passing to `args`, and other calls to `.path`, and instead let args do this lazily. --- CHANGELOG.md | 2 ++ gazelle/modules_mapping/def.bzl | 15 +++++++++++---- gazelle/modules_mapping/generator.py | 3 +++ 3 files changed, 16 insertions(+), 4 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 849b458745..8eaac3d9cc 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -63,6 +63,8 @@ Unreleased changes template. is now the default. Note that running as root may still cause spurious Bazel cache invalidation ([#1169](https://github.com/bazelbuild/rules_python/issues/1169)). +* (gazelle) Don't collapse depsets to a list or into args when generating the modules mapping file. + Support spilling modules mapping args into a params file. {#v0-0-0-added} ### Added diff --git a/gazelle/modules_mapping/def.bzl b/gazelle/modules_mapping/def.bzl index eb17f5c3d4..48a5477b93 100644 --- a/gazelle/modules_mapping/def.bzl +++ b/gazelle/modules_mapping/def.bzl @@ -25,18 +25,25 @@ module name doesn't match the wheel distribution name. def _modules_mapping_impl(ctx): modules_mapping = ctx.actions.declare_file(ctx.attr.modules_mapping_name) - args = ctx.actions.args() all_wheels = depset( [whl for whl in ctx.files.wheels], transitive = [dep[DefaultInfo].files for dep in ctx.attr.wheels] + [dep[DefaultInfo].data_runfiles.files for dep in ctx.attr.wheels], ) - args.add("--output_file", modules_mapping.path) + + args = ctx.actions.args() + + # Spill parameters to a file prefixed with '@'. Note, the '@' prefix is the same + # prefix as used in the `generator.py` in `fromfile_prefix_chars` attribute. + args.use_param_file(param_file_arg = "@%s") + args.set_param_file_format(format = "multiline") if ctx.attr.include_stub_packages: args.add("--include_stub_packages") + args.add("--output_file", modules_mapping) args.add_all("--exclude_patterns", ctx.attr.exclude_patterns) - args.add_all("--wheels", [whl.path for whl in all_wheels.to_list()]) + args.add_all("--wheels", all_wheels) + ctx.actions.run( - inputs = all_wheels.to_list(), + inputs = all_wheels, outputs = [modules_mapping], executable = ctx.executable._generator, arguments = [args], diff --git a/gazelle/modules_mapping/generator.py b/gazelle/modules_mapping/generator.py index 99f565e8d6..d5ddca2ef2 100644 --- a/gazelle/modules_mapping/generator.py +++ b/gazelle/modules_mapping/generator.py @@ -152,6 +152,9 @@ def data_has_purelib_or_platlib(path): parser = argparse.ArgumentParser( prog="generator", description="Generates the modules mapping used by the Gazelle manifest.", + # Automatically read parameters from a file. Note, the '@' is the same prefix + # as set in the 'args.use_param_file' in the bazel rule. + fromfile_prefix_chars="@", ) parser.add_argument("--output_file", type=str) parser.add_argument("--include_stub_packages", action="store_true") From 1226caa77cc469a2cb42f9bec4d6d3b7bf5a2e1f Mon Sep 17 00:00:00 2001 From: Keith Smiley Date: Mon, 3 Mar 2025 15:09:28 -0800 Subject: [PATCH 090/922] Add error for pip.parse attrs that require other attrs (#2646) This makes it more clear when you've misconfigured pip.parse --- python/private/pypi/extension.bzl | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/python/private/pypi/extension.bzl b/python/private/pypi/extension.bzl index 405c22f60e..1a7d1e12ea 100644 --- a/python/private/pypi/extension.bzl +++ b/python/private/pypi/extension.bzl @@ -481,6 +481,10 @@ You cannot use both the additive_build_content and additive_build_content_file a cache = simpleapi_cache, parallel_download = pip_attr.parallel_download, ) + elif pip_attr.experimental_extra_index_urls: + fail("'experimental_extra_index_urls' is a no-op unless 'experimental_index_url' is set") + elif pip_attr.experimental_index_url_overrides: + fail("'experimental_index_url_overrides' is a no-op unless 'experimental_index_url' is set") out = _create_whl_repos( module_ctx, From a816962e509311c23230730b4b28f9d52a229949 Mon Sep 17 00:00:00 2001 From: Mathias Laurin Date: Thu, 6 Mar 2025 06:34:36 +0100 Subject: [PATCH 091/922] feat: Package pyi files in wheel (#2609) 1.1.0 introduced separate attributes for the type definitions (`.pyi` files) and type checking. This patch adds those files to the wheel to ensure that they are distributed and available to users. https://github.com/bazelbuild/rules_python/pull/2538 introduced `pyi_srcs`. --------- Co-authored-by: Ignas Anikevicius <240938+aignas@users.noreply.github.com> --- CHANGELOG.md | 2 ++ examples/wheel/BUILD.bazel | 4 +++ examples/wheel/lib/BUILD.bazel | 6 ++++ .../wheel/lib/module_with_type_annotations.py | 16 +++++++++ .../lib/module_with_type_annotations.pyi | 15 ++++++++ examples/wheel/main.py | 2 ++ examples/wheel/test_publish.py | 2 +- examples/wheel/wheel_test.py | 34 ++++++++++++++----- python/private/py_package.bzl | 3 ++ python/private/py_wheel.bzl | 8 ++++- .../whl_filegroup/extract_wheel_files_test.py | 2 ++ 11 files changed, 83 insertions(+), 11 deletions(-) create mode 100644 examples/wheel/lib/module_with_type_annotations.py create mode 100644 examples/wheel/lib/module_with_type_annotations.pyi diff --git a/CHANGELOG.md b/CHANGELOG.md index 8eaac3d9cc..da775748f0 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -53,6 +53,8 @@ Unreleased changes template. {#v0-0-0-changed} ### Changed * (deps) platforms 0.0.4 -> 0.0.11 +* (py_wheel) Package `py_library.pyi_srcs` (`.pyi` files) in the wheel. +* (py_package) Package `py_library.pyi_srcs` (`.pyi` files) in `py_package`. {#v0-0-0-fixed} ### Fixed diff --git a/examples/wheel/BUILD.bazel b/examples/wheel/BUILD.bazel index 58a4301523..d9ba800125 100644 --- a/examples/wheel/BUILD.bazel +++ b/examples/wheel/BUILD.bazel @@ -33,6 +33,7 @@ py_library( deps = [ "//examples/wheel/lib:simple_module", "//examples/wheel/lib:module_with_data", + "//examples/wheel/lib:module_with_type_annotations", # Example dependency which is not packaged in the wheel # due to "packages" filter on py_package rule. "//tests/load_from_macro:foo", @@ -67,6 +68,7 @@ py_wheel( version = "0.0.1", deps = [ "//examples/wheel/lib:module_with_data", + "//examples/wheel/lib:module_with_type_annotations", "//examples/wheel/lib:simple_module", ], ) @@ -90,6 +92,7 @@ py_wheel( version = "$(VERSION)", deps = [ "//examples/wheel/lib:module_with_data", + "//examples/wheel/lib:module_with_type_annotations", "//examples/wheel/lib:simple_module", ], ) @@ -109,6 +112,7 @@ py_wheel( version = "0.1.{BUILD_TIMESTAMP}", deps = [ "//examples/wheel/lib:module_with_data", + "//examples/wheel/lib:module_with_type_annotations", "//examples/wheel/lib:simple_module", ], ) diff --git a/examples/wheel/lib/BUILD.bazel b/examples/wheel/lib/BUILD.bazel index c182143c1d..7fcd8572cf 100644 --- a/examples/wheel/lib/BUILD.bazel +++ b/examples/wheel/lib/BUILD.bazel @@ -23,6 +23,12 @@ py_library( srcs = ["simple_module.py"], ) +py_library( + name = "module_with_type_annotations", + srcs = ["module_with_type_annotations.py"], + pyi_srcs = ["module_with_type_annotations.pyi"], +) + py_library( name = "module_with_data", srcs = ["module_with_data.py"], diff --git a/examples/wheel/lib/module_with_type_annotations.py b/examples/wheel/lib/module_with_type_annotations.py new file mode 100644 index 0000000000..13e0895160 --- /dev/null +++ b/examples/wheel/lib/module_with_type_annotations.py @@ -0,0 +1,16 @@ +# Copyright 2025 The Bazel Authors. All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +def function(): + return "qux" diff --git a/examples/wheel/lib/module_with_type_annotations.pyi b/examples/wheel/lib/module_with_type_annotations.pyi new file mode 100644 index 0000000000..b250cd01cf --- /dev/null +++ b/examples/wheel/lib/module_with_type_annotations.pyi @@ -0,0 +1,15 @@ +# Copyright 2025 The Bazel Authors. All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +def function() -> str: ... diff --git a/examples/wheel/main.py b/examples/wheel/main.py index 7c4d323e87..37b4f69811 100644 --- a/examples/wheel/main.py +++ b/examples/wheel/main.py @@ -13,6 +13,7 @@ # limitations under the License. import examples.wheel.lib.module_with_data as module_with_data +import examples.wheel.lib.module_with_type_annotations as module_with_type_annotations import examples.wheel.lib.simple_module as simple_module @@ -23,6 +24,7 @@ def function(): def main(): print(function()) print(module_with_data.function()) + print(module_with_type_annotations.function()) print(simple_module.function()) diff --git a/examples/wheel/test_publish.py b/examples/wheel/test_publish.py index 496642acb7..47134d11f3 100644 --- a/examples/wheel/test_publish.py +++ b/examples/wheel/test_publish.py @@ -104,7 +104,7 @@ def test_upload_and_query_simple_api(self):

Links for example-minimal-library

- example_minimal_library-0.0.1-py3-none-any.whl
+ example_minimal_library-0.0.1-py3-none-any.whl
""" self.assertEqual( diff --git a/examples/wheel/wheel_test.py b/examples/wheel/wheel_test.py index 4494ee170d..a3d6034930 100644 --- a/examples/wheel/wheel_test.py +++ b/examples/wheel/wheel_test.py @@ -76,6 +76,8 @@ def test_py_library_wheel(self): zf.namelist(), [ "examples/wheel/lib/module_with_data.py", + "examples/wheel/lib/module_with_type_annotations.py", + "examples/wheel/lib/module_with_type_annotations.pyi", "examples/wheel/lib/simple_module.py", "example_minimal_library-0.0.1.dist-info/WHEEL", "example_minimal_library-0.0.1.dist-info/METADATA", @@ -83,7 +85,7 @@ def test_py_library_wheel(self): ], ) self.assertFileSha256Equal( - filename, "79a4e9c1838c0631d5d8fa49a26efd6e9a364f6b38d9597c0f6df112271a0e28" + filename, "0cbf4ec574676015af595f570caf4ae2812f994f6338e247b002b4e496b6fbd5" ) def test_py_package_wheel(self): @@ -98,6 +100,8 @@ def test_py_package_wheel(self): "examples/wheel/lib/data,with,commas.txt", "examples/wheel/lib/data.txt", "examples/wheel/lib/module_with_data.py", + "examples/wheel/lib/module_with_type_annotations.py", + "examples/wheel/lib/module_with_type_annotations.pyi", "examples/wheel/lib/simple_module.py", "examples/wheel/main.py", "example_minimal_package-0.0.1.dist-info/WHEEL", @@ -106,7 +110,7 @@ def test_py_package_wheel(self): ], ) self.assertFileSha256Equal( - filename, "82370bf61310e2d3c7b1218368457dc7e161bf5dc1a280d7d45102b5e56acf43" + filename, "22aff90dd3c8c30c3ce2b729bb793cab0bd2668a6810de232677a0354ce79cae" ) def test_customized_wheel(self): @@ -121,6 +125,8 @@ def test_customized_wheel(self): "examples/wheel/lib/data,with,commas.txt", "examples/wheel/lib/data.txt", "examples/wheel/lib/module_with_data.py", + "examples/wheel/lib/module_with_type_annotations.py", + "examples/wheel/lib/module_with_type_annotations.pyi", "examples/wheel/lib/simple_module.py", "examples/wheel/main.py", "example_customized-0.0.1.dist-info/WHEEL", @@ -145,8 +151,10 @@ def test_customized_wheel(self): "examples/wheel/lib/data,with,commas.txt",sha256=9vJKEdfLu8bZRArKLroPZJh1XKkK3qFMXiM79MBL2Sg,12 examples/wheel/lib/data.txt,sha256=9vJKEdfLu8bZRArKLroPZJh1XKkK3qFMXiM79MBL2Sg,12 examples/wheel/lib/module_with_data.py,sha256=8s0Khhcqz3yVsBKv2IB5u4l4TMKh7-c_V6p65WVHPms,637 +examples/wheel/lib/module_with_type_annotations.py,sha256=MM2cFQsCBaUnzGiEGT5r07jhKSaCVRh5Paw_YLyrS-w,636 +examples/wheel/lib/module_with_type_annotations.pyi,sha256=fja3ql_WRJ1qO8jyZjWWrTTMcg1J7EpOQivOHY_8vI4,630 examples/wheel/lib/simple_module.py,sha256=z2hwciab_XPNIBNH8B1Q5fYgnJvQTeYf0ZQJpY8yLLY,637 -examples/wheel/main.py,sha256=sgg5iWN_9inYBjm6_Zw27hYdmo-l24fA-2rfphT-IlY,909 +examples/wheel/main.py,sha256=mFiRfzQEDwCHr-WVNQhOH26M42bw1UMF6IoqvtuDTrw,1047 example_customized-0.0.1.dist-info/WHEEL,sha256=sobxWSyDDkdg_rinUth-jxhXHqoNqlmNMJY3aTZn2Us,91 example_customized-0.0.1.dist-info/METADATA,sha256=QYQcDJFQSIqan8eiXqL67bqsUfgEAwf2hoK_Lgi1S-0,559 example_customized-0.0.1.dist-info/entry_points.txt,sha256=pqzpbQ8MMorrJ3Jp0ntmpZcuvfByyqzMXXi2UujuXD0,137 @@ -197,7 +205,7 @@ def test_customized_wheel(self): second = second.main:s""", ) self.assertFileSha256Equal( - filename, "706e8dd45884d8cb26e92869f7d29ab7ed9f683b4e2d08f06c03dbdaa12191b8" + filename, "657a938a6fdd6f38bf73d1d91016ffff85d68cf29ca390692a3e9d923dd0e39e" ) def test_filename_escaping(self): @@ -211,6 +219,8 @@ def test_filename_escaping(self): "examples/wheel/lib/data,with,commas.txt", "examples/wheel/lib/data.txt", "examples/wheel/lib/module_with_data.py", + "examples/wheel/lib/module_with_type_annotations.py", + "examples/wheel/lib/module_with_type_annotations.pyi", "examples/wheel/lib/simple_module.py", "examples/wheel/main.py", # PEP calls for replacing only in the archive filename. @@ -248,6 +258,8 @@ def test_custom_package_root_wheel(self): "wheel/lib/data,with,commas.txt", "wheel/lib/data.txt", "wheel/lib/module_with_data.py", + "wheel/lib/module_with_type_annotations.py", + "wheel/lib/module_with_type_annotations.pyi", "wheel/lib/simple_module.py", "wheel/main.py", "examples_custom_package_root-0.0.1.dist-info/WHEEL", @@ -265,7 +277,7 @@ def test_custom_package_root_wheel(self): for line in record_contents.splitlines(): self.assertFalse(line.startswith("/")) self.assertFileSha256Equal( - filename, "568922541703f6edf4b090a8413991f9fa625df2844e644dd30bdbe9deb660be" + filename, "d415edbf8f326161674c1fa260e364dd44f2a0311e2f596284320ea52d2a8bdb" ) def test_custom_package_root_multi_prefix_wheel(self): @@ -281,6 +293,8 @@ def test_custom_package_root_multi_prefix_wheel(self): "data,with,commas.txt", "data.txt", "module_with_data.py", + "module_with_type_annotations.py", + "module_with_type_annotations.pyi", "simple_module.py", "main.py", "example_custom_package_root_multi_prefix-0.0.1.dist-info/WHEEL", @@ -297,7 +311,7 @@ def test_custom_package_root_multi_prefix_wheel(self): for line in record_contents.splitlines(): self.assertFalse(line.startswith("/")) self.assertFileSha256Equal( - filename, "a8b91ce9d6f570e97b40a357a292a6f595d3470f07c479cb08550257cc9c8306" + filename, "6b76a1178c90996feaf3f9417f350c4a67f90f4247647fd4fd552858dc372d4b" ) def test_custom_package_root_multi_prefix_reverse_order_wheel(self): @@ -313,6 +327,8 @@ def test_custom_package_root_multi_prefix_reverse_order_wheel(self): "lib/data,with,commas.txt", "lib/data.txt", "lib/module_with_data.py", + "lib/module_with_type_annotations.py", + "lib/module_with_type_annotations.pyi", "lib/simple_module.py", "main.py", "example_custom_package_root_multi_prefix_reverse_order-0.0.1.dist-info/WHEEL", @@ -329,7 +345,7 @@ def test_custom_package_root_multi_prefix_reverse_order_wheel(self): for line in record_contents.splitlines(): self.assertFalse(line.startswith("/")) self.assertFileSha256Equal( - filename, "8f44e940731757c186079a42cfe7ea3d43cd96b526e3fb2ca2a3ea3048a9d489" + filename, "f976f0bb1c7d753e8c41629d6b79fb09908c6ecd2fec006816879fc86b664f3f" ) def test_python_requires_wheel(self): @@ -354,7 +370,7 @@ def test_python_requires_wheel(self): """, ) self.assertFileSha256Equal( - filename, "ba32493f5e43e481346384aaab9e8fa09c23884276ad057c5f432096a0350101" + filename, "f3b74ce429c3324b87f8d1cc7dc33be1493f54bb88d546a7d53be7587b82c1a7" ) def test_python_abi3_binary_wheel(self): @@ -419,7 +435,7 @@ def test_rule_creates_directory_and_is_included_in_wheel(self): ], ) self.assertFileSha256Equal( - filename, "ac9216bd54dcae1a6270c35fccf8a73b0be87c1b026c28e963b7c76b2f9b722b" + filename, "d8e874b807e5574bd11a9312c58ce7fe7055afb80412d0d0e7ed21fc9223cd53" ) def test_rule_expands_workspace_status_keys_in_wheel_metadata(self): diff --git a/python/private/py_package.bzl b/python/private/py_package.bzl index fd8bc2724c..1d866a9d80 100644 --- a/python/private/py_package.bzl +++ b/python/private/py_package.bzl @@ -46,6 +46,9 @@ def _py_package_impl(ctx): if hasattr(py_info, "transitive_pyc_files"): inputs.add(py_info.transitive_pyc_files) + if hasattr(py_info, "transitive_pyi_files"): + inputs.add(py_info.transitive_pyi_files) + inputs = inputs.build() # TODO: '/' is wrong on windows, but the path separator is not available in starlark. diff --git a/python/private/py_wheel.bzl b/python/private/py_wheel.bzl index b5fbec9ce0..c196ca6ad0 100644 --- a/python/private/py_wheel.bzl +++ b/python/private/py_wheel.bzl @@ -14,6 +14,7 @@ "Implementation of py_wheel rule" +load(":py_info.bzl", "PyInfo") load(":py_package.bzl", "py_package_lib") load(":py_wheel_normalize_pep440.bzl", "normalize_pep440") load(":stamp.bzl", "is_stamping_enabled") @@ -319,8 +320,13 @@ def _py_wheel_impl(ctx): name_file = ctx.actions.declare_file(ctx.label.name + ".name") + direct_pyi_files = [] + for dep in ctx.attr.deps: + if PyInfo in dep: + direct_pyi_files.extend(dep[PyInfo].direct_pyi_files.to_list()) + inputs_to_package = depset( - direct = ctx.files.deps, + direct = ctx.files.deps + direct_pyi_files, ) # Inputs to this rule which are not to be packaged. diff --git a/tests/whl_filegroup/extract_wheel_files_test.py b/tests/whl_filegroup/extract_wheel_files_test.py index 434899d5cf..125d7f312c 100644 --- a/tests/whl_filegroup/extract_wheel_files_test.py +++ b/tests/whl_filegroup/extract_wheel_files_test.py @@ -14,6 +14,8 @@ def test_get_wheel_record(self) -> None: "examples/wheel/lib/data,with,commas.txt", "examples/wheel/lib/data.txt", "examples/wheel/lib/module_with_data.py", + "examples/wheel/lib/module_with_type_annotations.py", + "examples/wheel/lib/module_with_type_annotations.pyi", "examples/wheel/lib/simple_module.py", "examples/wheel/main.py", "example_minimal_package-0.0.1.dist-info/WHEEL", From 0fa6667de443ebbe75ffabddffe5734ea7c05bb1 Mon Sep 17 00:00:00 2001 From: Wyatt Hepler <255@users.noreply.github.com> Date: Thu, 6 Mar 2025 15:12:31 -0800 Subject: [PATCH 092/922] chore: Remove *_build_test targets from sphinx_docs (#2645) (#2650) Remove implicit `build_test`s from `sphinx_docs` targets. Instead, users can decide whether or not to add `build_tests` for docs. This also keeps `sphinx_docs` builds out of `bazel test //...`, which may not be desirable. Add `build_test`s to cover in-tree `sphinx_docs` targets. Rename the existing `build_test` for `//sphinxdocs/tests/sphinx_docs:docs` to match the new targets. Also, tag the `sphinx_docs` `*.run` and `*.serve` targets as `"manual"` so they are excluded from wildcards. These are only needed for interactive development. --------- Co-authored-by: Richard Levasseur --- docs/BUILD.bazel | 6 ++++++ sphinxdocs/private/sphinx.bzl | 14 +++++--------- sphinxdocs/tests/sphinx_docs/BUILD.bazel | 2 +- sphinxdocs/tests/sphinx_stardoc/BUILD.bazel | 6 ++++++ 4 files changed, 18 insertions(+), 10 deletions(-) diff --git a/docs/BUILD.bazel b/docs/BUILD.bazel index ea386f114a..0c07002a01 100644 --- a/docs/BUILD.bazel +++ b/docs/BUILD.bazel @@ -13,6 +13,7 @@ # limitations under the License. load("@bazel_skylib//:bzl_library.bzl", "bzl_library") +load("@bazel_skylib//rules:build_test.bzl", "build_test") load("@dev_pip//:requirements.bzl", "requirement") load("//python/private:bzlmod_enabled.bzl", "BZLMOD_ENABLED") # buildifier: disable=bzl-visibility load("//python/private:util.bzl", "IS_BAZEL_7_OR_HIGHER") # buildifier: disable=bzl-visibility @@ -77,6 +78,11 @@ sphinx_docs( ], ) +build_test( + name = "docs_build_test", + targets = [":docs"], +) + sphinx_stardocs( name = "bzl_api_docs", srcs = [ diff --git a/sphinxdocs/private/sphinx.bzl b/sphinxdocs/private/sphinx.bzl index 7ec35f9ab4..8d19d87052 100644 --- a/sphinxdocs/private/sphinx.bzl +++ b/sphinxdocs/private/sphinx.bzl @@ -15,7 +15,6 @@ """Implementation of sphinx rules.""" load("@bazel_skylib//lib:paths.bzl", "paths") -load("@bazel_skylib//rules:build_test.bzl", "build_test") load("@bazel_skylib//rules:common_settings.bzl", "BuildSettingInfo") load("//python:py_binary.bzl", "py_binary") load("//python/private:util.bzl", "add_tag", "copy_propagating_kwargs") # buildifier: disable=bzl-visibility @@ -177,6 +176,9 @@ def sphinx_docs( **common_kwargs ) + common_kwargs_with_manual_tag = dict(common_kwargs) + common_kwargs_with_manual_tag["tags"] = list(common_kwargs.get("tags") or []) + ["manual"] + py_binary( name = name + ".serve", srcs = [_SPHINX_SERVE_MAIN_SRC], @@ -185,18 +187,12 @@ def sphinx_docs( args = [ "$(execpath {})".format(html_name), ], - **common_kwargs + **common_kwargs_with_manual_tag ) sphinx_run( name = name + ".run", docs = name, - **common_kwargs - ) - - build_test( - name = name + "_build_test", - targets = [name], - **kwargs # kwargs used to pick up target_compatible_with + **common_kwargs_with_manual_tag ) def _sphinx_docs_impl(ctx): diff --git a/sphinxdocs/tests/sphinx_docs/BUILD.bazel b/sphinxdocs/tests/sphinx_docs/BUILD.bazel index 1a05db0ea3..f9c82967c1 100644 --- a/sphinxdocs/tests/sphinx_docs/BUILD.bazel +++ b/sphinxdocs/tests/sphinx_docs/BUILD.bazel @@ -40,6 +40,6 @@ sphinx_build_binary( ) build_test( - name = "build_tests", + name = "docs_build_test", targets = [":docs"], ) diff --git a/sphinxdocs/tests/sphinx_stardoc/BUILD.bazel b/sphinxdocs/tests/sphinx_stardoc/BUILD.bazel index 60a5e8d766..e3a68ea225 100644 --- a/sphinxdocs/tests/sphinx_stardoc/BUILD.bazel +++ b/sphinxdocs/tests/sphinx_stardoc/BUILD.bazel @@ -1,4 +1,5 @@ load("@bazel_skylib//:bzl_library.bzl", "bzl_library") +load("@bazel_skylib//rules:build_test.bzl", "build_test") load("//python:py_test.bzl", "py_test") load("//python/private:util.bzl", "IS_BAZEL_7_OR_HIGHER") # buildifier: disable=bzl-visibility load("//sphinxdocs:sphinx.bzl", "sphinx_build_binary", "sphinx_docs") @@ -40,6 +41,11 @@ sphinx_docs( ], ) +build_test( + name = "docs_build_test", + targets = [":docs"], +) + sphinx_stardocs( name = "simple_bzl_docs", srcs = [ From 49109619cfbd9794ffbe2026b32dba84d984c892 Mon Sep 17 00:00:00 2001 From: Ignas Anikevicius <240938+aignas@users.noreply.github.com> Date: Fri, 7 Mar 2025 10:54:28 +0900 Subject: [PATCH 093/922] fix(pypi): use python -B for repo-phase invocations (#2641) Before this change we would just invoke the Python interpreter. This means that in the `rules_python` directory there would be `__pycache__` folders created in the source tree and the same `__pycache__` folders would be created in the python interpreter repository rules if the directories were writable. This change ensures that we are executing `python` with `-B` in those contexts and reduces any likelihood of us doing the wrong thing. Work towards #1169. --------- Co-authored-by: Richard Levasseur --- CHANGELOG.md | 3 ++ python/private/pypi/evaluate_markers.bzl | 10 ++--- python/private/pypi/patch_whl.bzl | 10 +++-- python/private/pypi/pypi_repo_utils.bzl | 55 ++++++++++++++++++------ python/private/pypi/whl_library.bzl | 8 ++-- 5 files changed, 61 insertions(+), 25 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index da775748f0..8f97eef933 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -67,6 +67,9 @@ Unreleased changes template. ([#1169](https://github.com/bazelbuild/rules_python/issues/1169)). * (gazelle) Don't collapse depsets to a list or into args when generating the modules mapping file. Support spilling modules mapping args into a params file. +* (pypi) From now on `python` invocations in repository and module extension + evaluation contexts will invoke Python interpreter with `-B` to avoid + creating `.pyc` files. {#v0-0-0-added} ### Added diff --git a/python/private/pypi/evaluate_markers.bzl b/python/private/pypi/evaluate_markers.bzl index ec5f576945..028657f716 100644 --- a/python/private/pypi/evaluate_markers.bzl +++ b/python/private/pypi/evaluate_markers.bzl @@ -55,12 +55,12 @@ def evaluate_markers(mrctx, *, requirements, python_interpreter, python_interpre pypi_repo_utils.execute_checked( mrctx, op = "ResolveRequirementEnvMarkers({})".format(in_file), + python = pypi_repo_utils.resolve_python_interpreter( + mrctx, + python_interpreter = python_interpreter, + python_interpreter_target = python_interpreter_target, + ), arguments = [ - pypi_repo_utils.resolve_python_interpreter( - mrctx, - python_interpreter = python_interpreter, - python_interpreter_target = python_interpreter_target, - ), "-m", "python.private.pypi.requirements_parser.resolve_target_platforms", in_file, diff --git a/python/private/pypi/patch_whl.bzl b/python/private/pypi/patch_whl.bzl index a7da224321..c839f2e4d6 100644 --- a/python/private/pypi/patch_whl.bzl +++ b/python/private/pypi/patch_whl.bzl @@ -27,8 +27,8 @@ other patches ensures that the users have overview on exactly what has changed within the wheel. """ -load("//python/private:repo_utils.bzl", "repo_utils") load(":parse_whl_name.bzl", "parse_whl_name") +load(":pypi_repo_utils.bzl", "pypi_repo_utils") _rules_python_root = Label("//:BUILD.bazel") @@ -102,10 +102,14 @@ def patch_whl(rctx, *, python_interpreter, whl_path, patches, **kwargs): record_patch = rctx.path("RECORD.patch") whl_patched = patched_whl_name(whl_input.basename) - repo_utils.execute_checked( + pypi_repo_utils.execute_checked( rctx, + python = python_interpreter, + srcs = [ + Label("//python/private/pypi:repack_whl.py"), + Label("//tools:wheelmaker.py"), + ], arguments = [ - python_interpreter, "-m", "python.private.pypi.repack_whl", "--record-patch", diff --git a/python/private/pypi/pypi_repo_utils.bzl b/python/private/pypi/pypi_repo_utils.bzl index 196431636f..bb2acc850a 100644 --- a/python/private/pypi/pypi_repo_utils.bzl +++ b/python/private/pypi/pypi_repo_utils.bzl @@ -104,11 +104,30 @@ def _construct_pypath(mrctx, *, entries): ]) return pypath -def _execute_checked(mrctx, *, srcs, **kwargs): +def _execute_prep(mrctx, *, python, srcs, **kwargs): + for src in srcs: + # This will ensure that we will re-evaluate the bzlmod extension or + # refetch the repository_rule when the srcs change. This should work on + # Bazel versions without `mrctx.watch` as well. + repo_utils.watch(mrctx, mrctx.path(src)) + + environment = kwargs.pop("environment", {}) + pythonpath = environment.get("PYTHONPATH", "") + if pythonpath and not types.is_string(pythonpath): + environment["PYTHONPATH"] = _construct_pypath(mrctx, entries = pythonpath) + kwargs["environment"] = environment + + # -B is added to prevent the repo-phase invocation from creating timestamp + # based pyc files, which contributes to race conditions and non-determinism + kwargs["arguments"] = [python, "-B"] + kwargs.get("arguments", []) + return kwargs + +def _execute_checked(mrctx, *, python, srcs, **kwargs): """Helper function to run a python script and modify the PYTHONPATH to include external deps. Args: mrctx: Handle to the module_ctx or repository_ctx. + python: The python interpreter to use. srcs: The src files that the script depends on. This is important to ensure that the Bazel repository cache or the bzlmod lock file gets invalidated when any one file changes. It is advisable to use @@ -118,26 +137,34 @@ def _execute_checked(mrctx, *, srcs, **kwargs): the `environment` has a value `PYTHONPATH` and it is a list, then it will be passed to `construct_pythonpath` function. """ + return repo_utils.execute_checked( + mrctx, + **_execute_prep(mrctx, python = python, srcs = srcs, **kwargs) + ) - for src in srcs: - # This will ensure that we will re-evaluate the bzlmod extension or - # refetch the repository_rule when the srcs change. This should work on - # Bazel versions without `mrctx.watch` as well. - repo_utils.watch(mrctx, mrctx.path(src)) - - env = kwargs.pop("environment", {}) - pythonpath = env.get("PYTHONPATH", "") - if pythonpath and not types.is_string(pythonpath): - env["PYTHONPATH"] = _construct_pypath(mrctx, entries = pythonpath) +def _execute_checked_stdout(mrctx, *, python, srcs, **kwargs): + """Helper function to run a python script and modify the PYTHONPATH to include external deps. - return repo_utils.execute_checked( + Args: + mrctx: Handle to the module_ctx or repository_ctx. + python: The python interpreter to use. + srcs: The src files that the script depends on. This is important to + ensure that the Bazel repository cache or the bzlmod lock file gets + invalidated when any one file changes. It is advisable to use + `RECORD` files for external deps and the list of srcs from the + rules_python repo for any scripts. + **kwargs: Arguments forwarded to `repo_utils.execute_checked`. If + the `environment` has a value `PYTHONPATH` and it is a list, then + it will be passed to `construct_pythonpath` function. + """ + return repo_utils.execute_checked_stdout( mrctx, - environment = env, - **kwargs + **_execute_prep(mrctx, python = python, srcs = srcs, **kwargs) ) pypi_repo_utils = struct( construct_pythonpath = _construct_pypath, execute_checked = _execute_checked, + execute_checked_stdout = _execute_checked_stdout, resolve_python_interpreter = _resolve_python_interpreter, ) diff --git a/python/private/pypi/whl_library.bzl b/python/private/pypi/whl_library.bzl index ef4077fa41..bdcf7849ad 100644 --- a/python/private/pypi/whl_library.bzl +++ b/python/private/pypi/whl_library.bzl @@ -75,14 +75,15 @@ def _get_toolchain_unix_cflags(rctx, python_interpreter, logger = None): if not is_standalone_interpreter(rctx, python_interpreter, logger = logger): return [] - stdout = repo_utils.execute_checked_stdout( + stdout = pypi_repo_utils.execute_checked_stdout( rctx, op = "GetPythonVersionForUnixCflags", + python = python_interpreter, arguments = [ - python_interpreter, "-c", "import sys; print(f'{sys.version_info[0]}.{sys.version_info[1]}', end='')", ], + srcs = [], ) _python_version = stdout include_path = "{}/include/python{}".format( @@ -181,7 +182,6 @@ def _whl_library_impl(rctx): python_interpreter_target = rctx.attr.python_interpreter_target, ) args = [ - python_interpreter, "-m", "python.private.pypi.whl_installer.wheel_installer", "--requirement", @@ -247,6 +247,7 @@ def _whl_library_impl(rctx): # truncate the requirement value when logging it / reporting # progress since it may contain several ' --hash=sha256:... # --hash=sha256:...' substrings that fill up the console + python = python_interpreter, op = op_tmpl.format(name = rctx.attr.name, requirement = rctx.attr.requirement.split(" ", 1)[0]), arguments = args, environment = environment, @@ -295,6 +296,7 @@ def _whl_library_impl(rctx): pypi_repo_utils.execute_checked( rctx, op = "whl_library.ExtractWheel({}, {})".format(rctx.attr.name, whl_path), + python = python_interpreter, arguments = args + [ "--whl-file", whl_path, From b49956040752c2909685564ce752093bdb7bc537 Mon Sep 17 00:00:00 2001 From: Simon Stewart Date: Fri, 7 Mar 2025 04:54:45 +0000 Subject: [PATCH 094/922] build: Update doublestar to a version that works with the latest Gazelle (#2480) Co-authored-by: Douglas Thor Co-authored-by: Ignas Anikevicius <240938+aignas@users.noreply.github.com> --- CHANGELOG.md | 1 + gazelle/deps.bzl | 9 +++------ gazelle/go.mod | 2 +- gazelle/go.sum | 2 ++ 4 files changed, 7 insertions(+), 7 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 8f97eef933..c05204dbd2 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -70,6 +70,7 @@ Unreleased changes template. * (pypi) From now on `python` invocations in repository and module extension evaluation contexts will invoke Python interpreter with `-B` to avoid creating `.pyc` files. +* (deps) doublestar 4.7.1 (required for recent Gazelle versions) {#v0-0-0-added} ### Added diff --git a/gazelle/deps.bzl b/gazelle/deps.bzl index 1bdf179e98..fbb5285a4c 100644 --- a/gazelle/deps.bzl +++ b/gazelle/deps.bzl @@ -14,10 +14,7 @@ "This file managed by `bazel run //:gazelle_update_repos`" -load( - "@bazel_gazelle//:deps.bzl", - _go_repository = "go_repository", -) +load("@bazel_gazelle//:deps.bzl", _go_repository = "go_repository") load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive") def go_repository(name, **kwargs): @@ -70,8 +67,8 @@ def go_deps(): go_repository( name = "com_github_bmatcuk_doublestar_v4", importpath = "github.com/bmatcuk/doublestar/v4", - sum = "h1:FH9SifrbvJhnlQpztAx++wlkk70QBf0iBWDwNy7PA4I=", - version = "v4.6.1", + sum = "h1:fdDeAqgT47acgwd9bd9HxJRDmc9UAmPpc+2m0CXv75Q=", + version = "v4.7.1", ) go_repository( diff --git a/gazelle/go.mod b/gazelle/go.mod index 29a0b5cb0c..33ee6bb08a 100644 --- a/gazelle/go.mod +++ b/gazelle/go.mod @@ -6,7 +6,7 @@ require ( github.com/bazelbuild/bazel-gazelle v0.31.1 github.com/bazelbuild/buildtools v0.0.0-20231103205921-433ea8554e82 github.com/bazelbuild/rules_go v0.41.0 - github.com/bmatcuk/doublestar/v4 v4.6.1 + github.com/bmatcuk/doublestar/v4 v4.7.1 github.com/dougthor42/go-tree-sitter v0.0.0-20241210060307-2737e1d0de6b github.com/emirpasic/gods v1.18.1 github.com/ghodss/yaml v1.0.0 diff --git a/gazelle/go.sum b/gazelle/go.sum index d48da9ece3..5acd4a6db5 100644 --- a/gazelle/go.sum +++ b/gazelle/go.sum @@ -8,6 +8,8 @@ github.com/bazelbuild/rules_go v0.41.0 h1:JzlRxsFNhlX+g4drDRPhIaU5H5LnI978wdMJ0v github.com/bazelbuild/rules_go v0.41.0/go.mod h1:TMHmtfpvyfsxaqfL9WnahCsXMWDMICTw7XeK9yVb+YU= github.com/bmatcuk/doublestar/v4 v4.6.1 h1:FH9SifrbvJhnlQpztAx++wlkk70QBf0iBWDwNy7PA4I= github.com/bmatcuk/doublestar/v4 v4.6.1/go.mod h1:xBQ8jztBU6kakFMg+8WGxn0c6z1fTSPVIjEY1Wr7jzc= +github.com/bmatcuk/doublestar/v4 v4.7.1 h1:fdDeAqgT47acgwd9bd9HxJRDmc9UAmPpc+2m0CXv75Q= +github.com/bmatcuk/doublestar/v4 v4.7.1/go.mod h1:xBQ8jztBU6kakFMg+8WGxn0c6z1fTSPVIjEY1Wr7jzc= github.com/census-instrumentation/opencensus-proto v0.2.1/go.mod h1:f6KPmirojxKA12rnyqOA5BBL4O983OfeGPqjHWSTneU= github.com/chzyer/logex v1.1.10/go.mod h1:+Ywpsq7O8HXn0nuIou7OrIPyXbp3wmkHB+jjWRnGsAI= github.com/chzyer/readline v0.0.0-20180603132655-2972be24d48e/go.mod h1:nSuG5e5PlCu98SY8svDHJxuZscDgtXS6KTTbou5AhLI= From e7d2f09394dd14816310c4c661d2fefab33b2b1b Mon Sep 17 00:00:00 2001 From: Simon Stewart Date: Fri, 7 Mar 2025 07:02:29 +0000 Subject: [PATCH 095/922] fix: Add libdir to library search path (#2476) We discovered when dealing with libraries such as `psycopg2` that the wheel would attempt to link against `libpython.a`. This fix points the linker at the correct python version being used. --------- Co-authored-by: Ignas Anikevicius <240938+aignas@users.noreply.github.com> --- CHANGELOG.md | 2 ++ python/private/pypi/attrs.bzl | 9 +++++++++ python/private/pypi/extension.bzl | 1 + python/private/pypi/whl_library.bzl | 25 ++++++++++++++++++++---- tests/pypi/extension/extension_tests.bzl | 2 ++ 5 files changed, 35 insertions(+), 4 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index c05204dbd2..e59d225189 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -77,6 +77,8 @@ Unreleased changes template. * {obj}`//python/bin:python`: convenience target for directly running an interpreter. {obj}`--//python/bin:python_src` can be used to specify a binary whose interpreter to use. +* (pypi) An extra argument to add the interpreter lib dir to `LDFLAGS` when + building wheels from `sdist`. {#v0-0-0-removed} ### Removed diff --git a/python/private/pypi/attrs.bzl b/python/private/pypi/attrs.bzl index c9b7ea66a9..6717e9528c 100644 --- a/python/private/pypi/attrs.bzl +++ b/python/private/pypi/attrs.bzl @@ -15,6 +15,15 @@ "common attributes for whl_library and pip_repository" ATTRS = { + "add_libdir_to_library_search_path": attr.bool( + default = False, + doc = """ +If true, add the lib dir of the bundled interpreter to the library search path via `LDFLAGS`. + +:::{versionadded} VERSION_NEXT_FEATURE +::: +""", + ), "download_only": attr.bool( doc = """ Whether to use "pip download" instead of "pip wheel". Disables building wheels from source, but allows use of diff --git a/python/private/pypi/extension.bzl b/python/private/pypi/extension.bzl index 1a7d1e12ea..be00bf8ab3 100644 --- a/python/private/pypi/extension.bzl +++ b/python/private/pypi/extension.bzl @@ -203,6 +203,7 @@ def _create_whl_repos( ) maybe_args = dict( # The following values are safe to omit if they have false like values + add_libdir_to_library_search_path = pip_attr.add_libdir_to_library_search_path, annotation = whl_modifications.get(whl_name), download_only = pip_attr.download_only, enable_implicit_namespace_pkgs = pip_attr.enable_implicit_namespace_pkgs, diff --git a/python/private/pypi/whl_library.bzl b/python/private/pypi/whl_library.bzl index bdcf7849ad..dea61b23dc 100644 --- a/python/private/pypi/whl_library.bzl +++ b/python/private/pypi/whl_library.bzl @@ -140,11 +140,28 @@ def _parse_optional_attrs(rctx, args, extra_pip_args = None): if rctx.attr.enable_implicit_namespace_pkgs: args.append("--enable_implicit_namespace_pkgs") + env = {} if rctx.attr.environment != None: - args += [ - "--environment", - json.encode(struct(arg = rctx.attr.environment)), - ] + for key, value in rctx.attr.environment.items(): + env[key] = value + + # This is super hacky, but working out something nice is tricky. + # This is in particular needed for psycopg2 which attempts to link libpython.a, + # in order to point the linker at the correct python intepreter. + if rctx.attr.add_libdir_to_library_search_path: + if "LDFLAGS" in env: + fail("Can't set both environment LDFLAGS and add_libdir_to_library_search_path") + command = [pypi_repo_utils.resolve_python_interpreter(rctx), "-c", "import sys ; sys.stdout.write('{}/lib'.format(sys.exec_prefix))"] + result = rctx.execute(command) + if result.return_code != 0: + fail("Failed to get LDFLAGS path: command: {}, exit code: {}, stdout: {}, stderr: {}".format(command, result.return_code, result.stdout, result.stderr)) + libdir = result.stdout + env["LDFLAGS"] = "-L{}".format(libdir) + + args += [ + "--environment", + json.encode(struct(arg = env)), + ] return args diff --git a/tests/pypi/extension/extension_tests.bzl b/tests/pypi/extension/extension_tests.bzl index 5916a27e98..8c01a02271 100644 --- a/tests/pypi/extension/extension_tests.bzl +++ b/tests/pypi/extension/extension_tests.bzl @@ -77,6 +77,7 @@ def _parse( hub_name, python_version, _evaluate_markers_srcs = [], + add_libdir_to_library_search_path = False, auth_patterns = {}, download_only = False, enable_implicit_namespace_pkgs = False, @@ -105,6 +106,7 @@ def _parse( return struct( _evaluate_markers_srcs = _evaluate_markers_srcs, auth_patterns = auth_patterns, + add_libdir_to_library_search_path = add_libdir_to_library_search_path, download_only = download_only, enable_implicit_namespace_pkgs = enable_implicit_namespace_pkgs, environment = environment, From 52712b9279d2ab77e33ad43a65eba546bbd17ef3 Mon Sep 17 00:00:00 2001 From: Douglas Thor Date: Mon, 10 Mar 2025 19:40:14 -0700 Subject: [PATCH 096/922] fix(gazelle): Include YAML 'docstart' in gazelle manifest file (#2656) Update `gazelle_python.yaml` to include the YAML docstart string: ```diff -- a/gazelle_python.yaml +++ b/gazelle_python.yaml @@ -3,6 +3,7 @@ # To update this file, run: # bazel run //:gazelle_python_manifest.update +--- manifest: modules_mapping: 30fcd23745efe32ce681__mypyc: black ``` While _technically_ not required, it is good practice to include. And then users don't have to exclude `gazelle_python.yaml` from their linters :upside_down_face:. /cc @joshgc --- CHANGELOG.md | 5 ++++- examples/build_file_generation/gazelle_python.yaml | 1 + examples/bzlmod_build_file_generation/gazelle_python.yaml | 1 + .../gazelle_python_with_types.yaml | 1 + gazelle/manifest/generate/generate.go | 2 +- 5 files changed, 8 insertions(+), 2 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index e59d225189..d7ae4bf0a7 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -55,11 +55,14 @@ Unreleased changes template. * (deps) platforms 0.0.4 -> 0.0.11 * (py_wheel) Package `py_library.pyi_srcs` (`.pyi` files) in the wheel. * (py_package) Package `py_library.pyi_srcs` (`.pyi` files) in `py_package`. +* (gazelle) The generated manifest file (default: `gazelle_python.yaml`) will now include the + YAML document start `---` line. Implemented in + [#2656](https://github.com/bazelbuild/rules_python/pull/2656). {#v0-0-0-fixed} ### Fixed * (pypi) The `ppc64le` is now pointing to the right target in the `platforms` package. -* (gazelle) No longer incorrectly merge `py_binary` targets during partial updates in +* (gazelle) No longer incorrectly merge `py_binary` targets during partial updates in `file` generation mode. Fixed in [#2619](https://github.com/bazelbuild/rules_python/pull/2619). * (bzlmod) Running as root is no longer an error. `ignore_root_user_error=True` is now the default. Note that running as root may still cause spurious diff --git a/examples/build_file_generation/gazelle_python.yaml b/examples/build_file_generation/gazelle_python.yaml index cd5904dcba..6b34f3c688 100644 --- a/examples/build_file_generation/gazelle_python.yaml +++ b/examples/build_file_generation/gazelle_python.yaml @@ -3,6 +3,7 @@ # To update this file, run: # bazel run //:gazelle_python_manifest.update +--- manifest: modules_mapping: alabaster: alabaster diff --git a/examples/bzlmod_build_file_generation/gazelle_python.yaml b/examples/bzlmod_build_file_generation/gazelle_python.yaml index c94f93a070..019b051092 100644 --- a/examples/bzlmod_build_file_generation/gazelle_python.yaml +++ b/examples/bzlmod_build_file_generation/gazelle_python.yaml @@ -3,6 +3,7 @@ # To update this file, run: # bazel run //:gazelle_python_manifest.update +--- manifest: modules_mapping: S3: s3cmd diff --git a/examples/bzlmod_build_file_generation/gazelle_python_with_types.yaml b/examples/bzlmod_build_file_generation/gazelle_python_with_types.yaml index b6b0687ea4..7632235aa0 100644 --- a/examples/bzlmod_build_file_generation/gazelle_python_with_types.yaml +++ b/examples/bzlmod_build_file_generation/gazelle_python_with_types.yaml @@ -3,6 +3,7 @@ # To update this file, run: # bazel run //:gazelle_python_manifest_with_types.update +--- manifest: modules_mapping: S3: s3cmd diff --git a/gazelle/manifest/generate/generate.go b/gazelle/manifest/generate/generate.go index 27cf2a21d8..899b1514ee 100644 --- a/gazelle/manifest/generate/generate.go +++ b/gazelle/manifest/generate/generate.go @@ -151,7 +151,7 @@ func writeOutput( } defer outputFile.Close() - if _, err := fmt.Fprintf(outputFile, "%s\n", header); err != nil { + if _, err := fmt.Fprintf(outputFile, "%s\n---\n", header); err != nil { return fmt.Errorf("failed to write output: %w", err) } From 4cb8412dbb5d2df5b91d3e3102d210be6d8b8d6f Mon Sep 17 00:00:00 2001 From: Ignas Anikevicius <240938+aignas@users.noreply.github.com> Date: Tue, 11 Mar 2025 16:41:16 +0900 Subject: [PATCH 097/922] feat(uv): parse the dist-manifest.json to not hardcode sha256 in rules_python (#2578) Finalize the `uv` extension interface employing a builder pattern so that the users can specify the exact version that needs to be registered. This also moves the registration of the actual toolchain to `rules_python` itself and ensures that an incompatible noop toolchain is registered if nothing is configured. This ensures that the `register_toolchains("@uv//:all")` never fails. If the `url/sha256` values are not specified, this is falling back to using the `dist-manifest.json` on the GH releases page so that we can get the expected `sha256` value of each available file and download all of the usable archives. This means that `rules_python` no longer needs to be updated for `uv` version bumps. The remaining bits for closing the ticket: - [ ] Finalize the `lock` interface. - [ ] Add the locking target to the `pip.parse` hub repo if `pyproject.toml` is passed in. - [ ] Add a rule/target for `venv` creation. Work towards #1975. --- CHANGELOG.md | 5 + MODULE.bazel | 83 +++- examples/bzlmod/MODULE.bazel | 15 +- python/uv/private/BUILD.bazel | 21 +- python/uv/private/lock.bzl | 8 +- python/uv/private/toolchains_hub.bzl | 65 +++ python/uv/private/uv.bzl | 480 +++++++++++++++++- python/uv/private/uv_repositories.bzl | 120 ----- python/uv/private/uv_repository.bzl | 74 +++ python/uv/private/uv_toolchain.bzl | 2 + python/uv/private/uv_toolchain_info.bzl | 5 + python/uv/private/uv_toolchains_repo.bzl | 49 +- python/uv/private/versions.bzl | 94 ---- tests/uv/BUILD.bazel | 0 tests/uv/uv/BUILD.bazel | 17 + tests/uv/uv/uv_tests.bzl | 592 +++++++++++++++++++++++ tests/uv/uv_toolchains/BUILD.bazel | 25 + 17 files changed, 1371 insertions(+), 284 deletions(-) create mode 100644 python/uv/private/toolchains_hub.bzl delete mode 100644 python/uv/private/uv_repositories.bzl create mode 100644 python/uv/private/uv_repository.bzl delete mode 100644 python/uv/private/versions.bzl create mode 100644 tests/uv/BUILD.bazel create mode 100644 tests/uv/uv/BUILD.bazel create mode 100644 tests/uv/uv/uv_tests.bzl create mode 100644 tests/uv/uv_toolchains/BUILD.bazel diff --git a/CHANGELOG.md b/CHANGELOG.md index d7ae4bf0a7..413442eb99 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -80,6 +80,11 @@ Unreleased changes template. * {obj}`//python/bin:python`: convenience target for directly running an interpreter. {obj}`--//python/bin:python_src` can be used to specify a binary whose interpreter to use. +* (uv) Now the extension can be fully configured via `bzlmod` APIs without the + need to patch `rules_python`. The documentation has been added to `rules_python` + docs but usage of the extension may result in your setup breaking without any + notice. What is more, the URLs and SHA256 values will be retrieved from the + GitHub releases page metadata published by the `uv` project. * (pypi) An extra argument to add the interpreter lib dir to `LDFLAGS` when building wheels from `sdist`. diff --git a/MODULE.bazel b/MODULE.bazel index 3d7c3042a5..dc2193cec2 100644 --- a/MODULE.bazel +++ b/MODULE.bazel @@ -174,16 +174,83 @@ use_repo( "build_bazel_bazel_self", ) -# EXPERIMENTAL: This is experimental and may be removed without notice -uv = use_extension( +# TODO @aignas 2025-01-27: should this be moved to `//python/extensions:uv.bzl` or should +# it stay as it is? I think I may prefer to move it. +uv = use_extension("//python/uv:uv.bzl", "uv") + +# Here is how we can define platforms for the `uv` binaries - this will affect +# all of the downstream callers because we are using the extension without +# `dev_dependency = True`. +uv.default( + base_url = "https://github.com/astral-sh/uv/releases/download", + manifest_filename = "dist-manifest.json", + version = "0.6.3", +) +uv.default( + compatible_with = [ + "@platforms//os:macos", + "@platforms//cpu:aarch64", + ], + platform = "aarch64-apple-darwin", +) +uv.default( + compatible_with = [ + "@platforms//os:linux", + "@platforms//cpu:aarch64", + ], + platform = "aarch64-unknown-linux-gnu", +) +uv.default( + compatible_with = [ + "@platforms//os:linux", + "@platforms//cpu:ppc", + ], + platform = "powerpc64-unknown-linux-gnu", +) +uv.default( + compatible_with = [ + "@platforms//os:linux", + "@platforms//cpu:ppc64le", + ], + platform = "powerpc64le-unknown-linux-gnu", +) +uv.default( + compatible_with = [ + "@platforms//os:linux", + "@platforms//cpu:s390x", + ], + platform = "s390x-unknown-linux-gnu", +) +uv.default( + compatible_with = [ + "@platforms//os:macos", + "@platforms//cpu:x86_64", + ], + platform = "x86_64-apple-darwin", +) +uv.default( + compatible_with = [ + "@platforms//os:windows", + "@platforms//cpu:x86_64", + ], + platform = "x86_64-pc-windows-msvc", +) +uv.default( + compatible_with = [ + "@platforms//os:linux", + "@platforms//cpu:x86_64", + ], + platform = "x86_64-unknown-linux-gnu", +) +use_repo(uv, "uv") + +register_toolchains("@uv//:all") + +uv_dev = use_extension( "//python/uv:uv.bzl", "uv", dev_dependency = True, ) -uv.toolchain(uv_version = "0.4.25") -use_repo(uv, "uv_toolchains") - -register_toolchains( - "@uv_toolchains//:all", - dev_dependency = True, +uv_dev.configure( + version = "0.6.2", ) diff --git a/examples/bzlmod/MODULE.bazel b/examples/bzlmod/MODULE.bazel index eaed078d63..69e384e42b 100644 --- a/examples/bzlmod/MODULE.bazel +++ b/examples/bzlmod/MODULE.bazel @@ -101,12 +101,15 @@ python.single_version_platform_override( # rules based on the `python_version` arg values. use_repo(python, "python_3_10", "python_3_9", "python_versions", "pythons_hub") -# EXPERIMENTAL: This is experimental and may be removed without notice -uv = use_extension("@rules_python//python/uv:uv.bzl", "uv") -uv.toolchain(uv_version = "0.4.25") -use_repo(uv, "uv_toolchains") - -register_toolchains("@uv_toolchains//:all") +# EXPERIMENTAL: This is experimental and may be changed or removed without notice +uv = use_extension( + "@rules_python//python/uv:uv.bzl", + "uv", + # Use `dev_dependency` so that the toolchains are not defined pulled when your + # module is used elsewhere. + dev_dependency = True, +) +uv.configure(version = "0.6.2") # This extension allows a user to create modifications to how rules_python # creates different wheel repositories. Different attributes allow the user diff --git a/python/uv/private/BUILD.bazel b/python/uv/private/BUILD.bazel index 006c856d02..acf2a9c1f7 100644 --- a/python/uv/private/BUILD.bazel +++ b/python/uv/private/BUILD.bazel @@ -47,20 +47,19 @@ bzl_library( name = "uv_bzl", srcs = ["uv.bzl"], visibility = ["//python/uv:__subpackages__"], - deps = [":uv_repositories_bzl"], -) - -bzl_library( - name = "uv_repositories_bzl", - srcs = ["uv_repositories.bzl"], - visibility = ["//python/uv:__subpackages__"], deps = [ ":toolchain_types_bzl", + ":uv_repository_bzl", ":uv_toolchains_repo_bzl", - ":versions_bzl", ], ) +bzl_library( + name = "uv_repository_bzl", + srcs = ["uv_repository.bzl"], + visibility = ["//python/uv:__subpackages__"], +) + bzl_library( name = "uv_toolchain_bzl", srcs = ["uv_toolchain.bzl"], @@ -82,9 +81,3 @@ bzl_library( "//python/private:text_util_bzl", ], ) - -bzl_library( - name = "versions_bzl", - srcs = ["versions.bzl"], - visibility = ["//python/uv:__subpackages__"], -) diff --git a/python/uv/private/lock.bzl b/python/uv/private/lock.bzl index e0491b282c..9378f180db 100644 --- a/python/uv/private/lock.bzl +++ b/python/uv/private/lock.bzl @@ -30,9 +30,11 @@ def lock(*, name, srcs, out, upgrade = False, universal = True, args = [], **kwa """Pin the requirements based on the src files. Differences with the current {obj}`compile_pip_requirements` rule: - - This is implemented in shell and uv. + - This is implemented in shell and `uv`. - This does not error out if the output file does not exist yet. - Supports transitions out of the box. + - The execution of the lock file generation is happening inside of a build + action in a `genrule`. Args: name: The name of the target to run for updating the requirements. @@ -41,8 +43,8 @@ def lock(*, name, srcs, out, upgrade = False, universal = True, args = [], **kwa upgrade: Tell `uv` to always upgrade the dependencies instead of keeping them as they are. universal: Tell `uv` to generate a universal lock file. - args: Extra args to pass to `uv`. - **kwargs: Extra kwargs passed to the {obj}`py_binary` rule. + args: Extra args to pass to the rule. + **kwargs: Extra kwargs passed to the binary rule. """ pkg = native.package_name() update_target = name + ".update" diff --git a/python/uv/private/toolchains_hub.bzl b/python/uv/private/toolchains_hub.bzl new file mode 100644 index 0000000000..b39d84f0c2 --- /dev/null +++ b/python/uv/private/toolchains_hub.bzl @@ -0,0 +1,65 @@ +# Copyright 2025 The Bazel Authors. All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +"""A macro used from the uv_toolchain hub repo.""" + +load(":toolchain_types.bzl", "UV_TOOLCHAIN_TYPE") + +def toolchains_hub( + *, + name, + toolchains, + implementations, + target_compatible_with, + target_settings): + """Define the toolchains so that the lexicographical order registration is deterministic. + + TODO @aignas 2025-03-09: see if this can be reused in the python toolchains. + + Args: + name: The prefix to all of the targets, which goes after a numeric prefix. + toolchains: The toolchain names for the targets defined by this macro. + The earlier occurring items take precedence over the later items if + they match the target platform and target settings. + implementations: The name to label mapping. + target_compatible_with: The name to target_compatible_with list mapping. + target_settings: The name to target_settings list mapping. + """ + if len(toolchains) != len(implementations): + fail("Each name must have an implementation") + + # We are defining the toolchains so that the order of toolchain matching is + # the same as the order of the toolchains, because: + # * the toolchains are matched by target settings and target_compatible_with + # * the first toolchain satisfying the above wins + # + # this means we need to register the toolchains prefixed with a number of + # format 00xy, where x and y are some digits and the leading zeros to + # ensure lexicographical sorting. + # + # Add 1 so that there is always a leading zero + prefix_len = len(str(len(toolchains))) + 1 + prefix = "0" * (prefix_len - 1) + + for i, toolchain in enumerate(toolchains): + # prefix with a prefix and then truncate the string. + number_prefix = "{}{}".format(prefix, i)[-prefix_len:] + + native.toolchain( + name = "{}_{}_{}".format(number_prefix, name, toolchain), + target_compatible_with = target_compatible_with.get(toolchain, []), + target_settings = target_settings.get(toolchain, []), + toolchain = implementations[toolchain], + toolchain_type = UV_TOOLCHAIN_TYPE, + ) diff --git a/python/uv/private/uv.bzl b/python/uv/private/uv.bzl index 886e7fe748..55a05be032 100644 --- a/python/uv/private/uv.bzl +++ b/python/uv/private/uv.bzl @@ -18,36 +18,480 @@ EXPERIMENTAL: This is experimental and may be removed without notice A module extension for working with uv. """ -load(":uv_repositories.bzl", "uv_repositories") +load(":toolchain_types.bzl", "UV_TOOLCHAIN_TYPE") +load(":uv_repository.bzl", "uv_repository") +load(":uv_toolchains_repo.bzl", "uv_toolchains_repo") _DOC = """\ A module extension for working with uv. + +Basic usage: +```starlark +uv = use_extension( + "@rules_python//python/uv:uv.bzl", + "uv", + # Use `dev_dependency` so that the toolchains are not defined pulled when + # your module is used elsewhere. + dev_dependency = True, +) +uv.configure(version = "0.5.24") +``` + +Since this is only for locking the requirements files, it should be always +marked as a `dev_dependency`. """ -uv_toolchain = tag_class( - doc = "Configure uv toolchain for lock file generation.", - attrs = { - "uv_version": attr.string(doc = "Explicit version of uv.", mandatory = True), +_DEFAULT_ATTRS = { + "base_url": attr.string( + doc = """\ +Base URL to download metadata about the binaries and the binaries themselves. +""", + ), + "compatible_with": attr.label_list( + doc = """\ +The compatible with constraint values for toolchain resolution. +""", + ), + "manifest_filename": attr.string( + doc = """\ +The distribution manifest filename to use for the metadata fetching from GH. The +defaults for this are set in `rules_python` MODULE.bazel file that one can override +for a specific version. +""", + default = "dist-manifest.json", + ), + "platform": attr.string( + doc = """\ +The platform string used in the UV repository to denote the platform triple. +""", + ), + "target_settings": attr.label_list( + doc = """\ +The `target_settings` to add to platform definitions that then get used in `toolchain` +definitions. +""", + ), + "version": attr.string( + doc = """\ +The version of uv to configure the sources for. If this is not specified it will be the +last version used in the module or the default version set by `rules_python`. +""", + ), +} + +default = tag_class( + doc = """\ +Set the uv configuration defaults. +""", + attrs = _DEFAULT_ATTRS, +) + +configure = tag_class( + doc = """\ +Build the `uv` toolchain configuration by appending the provided configuration. +The information is appended to the version configuration that is specified by +{attr}`version` attribute, or if the version is unspecified, the version of the +last {obj}`uv.configure` call in the current module, or the version from the +defaults is used. + +Complex configuration example: +```starlark +# Configure the base_url for the default version. +uv.configure(base_url = "my_mirror") + +# Add an extra platform that can be used with your version. +uv.configure( + platform = "extra-platform", + target_settings = ["//my_config_setting_label"], + compatible_with = ["@platforms//os:exotic"], +) + +# Add an extra platform that can be used with your version. +uv.configure( + platform = "patched-binary", + target_settings = ["//my_super_config_setting"], + urls = ["https://example.zip"], + sha256 = "deadbeef", +) +``` +""", + attrs = _DEFAULT_ATTRS | { + "sha256": attr.string( + doc = "The sha256 of the downloaded artifact if the {attr}`urls` is specified.", + ), + "urls": attr.string_list( + doc = """\ +The urls to download the binary from. If this is used, {attr}`base_url` and +{attr}`manifest_name` are ignored for the given version. + +::::note +If the `urls` are specified, they need to be specified for all of the platforms +for a particular version. +:::: +""", + ), }, ) -def _uv_toolchain_extension(module_ctx): +def _configure(config, *, platform, compatible_with, target_settings, urls = [], sha256 = "", override = False, **values): + """Set the value in the config if the value is provided""" + for key, value in values.items(): + if not value: + continue + + if not override and config.get(key): + continue + + config[key] = value + + config.setdefault("platforms", {}) + if not platform: + if compatible_with or target_settings or urls: + fail("`platform` name must be specified when specifying `compatible_with`, `target_settings` or `urls`") + elif compatible_with or target_settings: + if not override and config.get("platforms", {}).get(platform): + return + + config["platforms"][platform] = struct( + name = platform.replace("-", "_").lower(), + compatible_with = compatible_with, + target_settings = target_settings, + ) + elif urls: + if not override and config.get("urls", {}).get(platform): + return + + config.setdefault("urls", {})[platform] = struct( + sha256 = sha256, + urls = urls, + ) + else: + config["platforms"].pop(platform) + +def process_modules( + module_ctx, + hub_name = "uv", + uv_repository = uv_repository, + toolchain_type = str(UV_TOOLCHAIN_TYPE), + hub_repo = uv_toolchains_repo): + """Parse the modules to get the config for 'uv' toolchains. + + Args: + module_ctx: the context. + hub_name: the name of the hub repository. + uv_repository: the rule to create a uv_repository override. + toolchain_type: the toolchain type to use here. + hub_repo: the hub repo factory function to use. + + Returns: + the result of the hub_repo. Mainly used for tests. + """ + + # default values to apply for version specific config + defaults = { + "base_url": "", + "manifest_filename": "", + "platforms": { + # The structure is as follows: + # "platform_name": struct( + # compatible_with = [], + # target_settings = [], + # ), + # + # NOTE: urls and sha256 cannot be set in defaults + }, + "version": "", + } for mod in module_ctx.modules: - for toolchain in mod.tags.toolchain: - if not mod.is_root: - fail( - "Only the root module may configure the uv toolchain.", - "This prevents conflicting registrations with any other modules.", - "NOTE: We may wish to enforce a policy where toolchain configuration is only allowed in the root module, or in rules_python. See https://github.com/bazelbuild/bazel/discussions/22024", - ) - - uv_repositories( - uv_version = toolchain.uv_version, - register_toolchains = False, + if not (mod.is_root or mod.name == "rules_python"): + continue + + for tag in mod.tags.default: + _configure( + defaults, + version = tag.version, + base_url = tag.base_url, + manifest_filename = tag.manifest_filename, + platform = tag.platform, + compatible_with = tag.compatible_with, + target_settings = tag.target_settings, + override = mod.is_root, + ) + + for key in [ + "version", + "manifest_filename", + "platforms", + ]: + if not defaults.get(key, None): + fail("defaults need to be set for '{}'".format(key)) + + # resolved per-version configuration. The shape is something like: + # versions = { + # "1.0.0": { + # "base_url": "", + # "manifest_filename": "", + # "platforms": { + # "platform_name": struct( + # compatible_with = [], + # target_settings = [], + # urls = [], # can be unset + # sha256 = "", # can be unset + # ), + # }, + # }, + # } + versions = {} + for mod in module_ctx.modules: + if not (mod.is_root or mod.name == "rules_python"): + continue + + # last_version is the last version used in the MODULE.bazel or the default + last_version = None + for tag in mod.tags.configure: + last_version = tag.version or last_version or defaults["version"] + specific_config = versions.setdefault( + last_version, + { + "base_url": defaults["base_url"], + "manifest_filename": defaults["manifest_filename"], + # shallow copy is enough as the values are structs and will + # be replaced on modification + "platforms": dict(defaults["platforms"]), + }, + ) + + _configure( + specific_config, + base_url = tag.base_url, + manifest_filename = tag.manifest_filename, + platform = tag.platform, + compatible_with = tag.compatible_with, + target_settings = tag.target_settings, + sha256 = tag.sha256, + urls = tag.urls, + override = mod.is_root, ) + if not versions: + return hub_repo( + name = hub_name, + toolchain_type = toolchain_type, + toolchain_names = ["none"], + toolchain_implementations = { + # NOTE @aignas 2025-02-24: the label to the toolchain can be anything + "none": str(Label("//python:none")), + }, + toolchain_compatible_with = { + "none": ["@platforms//:incompatible"], + }, + toolchain_target_settings = {}, + ) + + toolchain_names = [] + toolchain_implementations = {} + toolchain_compatible_with_by_toolchain = {} + toolchain_target_settings = {} + for version, config in versions.items(): + platforms = config["platforms"] + + # Use the manually specified urls + urls = { + platform: src + for platform, src in config.get("urls", {}).items() + if src.urls + } + + # Or fallback to fetching them from GH manifest file + # Example file: https://github.com/astral-sh/uv/releases/download/0.6.3/dist-manifest.json + if not urls: + urls = _get_tool_urls_from_dist_manifest( + module_ctx, + base_url = "{base_url}/{version}".format( + version = version, + base_url = config["base_url"], + ), + manifest_filename = config["manifest_filename"], + platforms = sorted(platforms), + ) + + for platform_name, platform in platforms.items(): + if platform_name not in urls: + continue + + toolchain_name = "{}_{}".format(version.replace(".", "_"), platform_name.lower().replace("-", "_")) + uv_repository_name = "{}_{}".format(hub_name, toolchain_name) + uv_repository( + name = uv_repository_name, + version = version, + platform = platform_name, + urls = urls[platform_name].urls, + sha256 = urls[platform_name].sha256, + ) + + toolchain_names.append(toolchain_name) + toolchain_implementations[toolchain_name] = "@{}//:uv_toolchain".format(uv_repository_name) + toolchain_compatible_with_by_toolchain[toolchain_name] = [ + str(label) + for label in platform.compatible_with + ] + if platform.target_settings: + toolchain_target_settings[toolchain_name] = [ + str(label) + for label in platform.target_settings + ] + + return hub_repo( + name = hub_name, + toolchain_type = toolchain_type, + toolchain_names = toolchain_names, + toolchain_implementations = toolchain_implementations, + toolchain_compatible_with = toolchain_compatible_with_by_toolchain, + toolchain_target_settings = toolchain_target_settings, + ) + +def _uv_toolchain_extension(module_ctx): + process_modules( + module_ctx, + hub_name = "uv", + ) + +def _overlap(first_collection, second_collection): + for x in first_collection: + if x in second_collection: + return True + + return False + +def _get_tool_urls_from_dist_manifest(module_ctx, *, base_url, manifest_filename, platforms): + """Download the results about remote tool sources. + + This relies on the tools using the cargo packaging to infer the actual + sha256 values for each binary. + + Example manifest url: https://github.com/astral-sh/uv/releases/download/0.6.5/dist-manifest.json + + The example format is as below + + dist_version "0.28.0" + announcement_tag "0.6.5" + announcement_tag_is_implicit false + announcement_is_prerelease false + announcement_title "0.6.5" + announcement_changelog "text" + announcement_github_body "MD text" + releases [ + { + app_name "uv" + app_version "0.6.5" + env + install_dir_env_var "UV_INSTALL_DIR" + unmanaged_dir_env_var "UV_UNMANAGED_INSTALL" + disable_update_env_var "UV_DISABLE_UPDATE" + no_modify_path_env_var "UV_NO_MODIFY_PATH" + github_base_url_env_var "UV_INSTALLER_GITHUB_BASE_URL" + ghe_base_url_env_var "UV_INSTALLER_GHE_BASE_URL" + display_name "uv" + display true + artifacts [ + "source.tar.gz" + "source.tar.gz.sha256" + "uv-installer.sh" + "uv-installer.ps1" + "sha256.sum" + "uv-aarch64-apple-darwin.tar.gz" + "uv-aarch64-apple-darwin.tar.gz.sha256" + "... + ] + artifacts + uv-aarch64-apple-darwin.tar.gz + name "uv-aarch64-apple-darwin.tar.gz" + kind "executable-zip" + target_triples [ + "aarch64-apple-darwin" + assets [ + { + id "uv-aarch64-apple-darwin-exe-uv" + name "uv" + path "uv" + kind "executable" + }, + { + id "uv-aarch64-apple-darwin-exe-uvx" + name "uvx" + path "uvx" + kind "executable" + } + ] + checksum "uv-aarch64-apple-darwin.tar.gz.sha256" + uv-aarch64-apple-darwin.tar.gz.sha256 + name "uv-aarch64-apple-darwin.tar.gz.sha256" + kind "checksum" + target_triples [ + "aarch64-apple-darwin" + ] + """ + dist_manifest = module_ctx.path(manifest_filename) + result = module_ctx.download( + base_url + "/" + manifest_filename, + output = dist_manifest, + ) + if not result.success: + fail(result) + dist_manifest = json.decode(module_ctx.read(dist_manifest)) + + artifacts = dist_manifest["artifacts"] + tool_sources = {} + downloads = {} + for fname, artifact in artifacts.items(): + if artifact.get("kind") != "executable-zip": + continue + + checksum = artifacts[artifact["checksum"]] + if not _overlap(checksum["target_triples"], platforms): + # we are not interested in this platform, so skip + continue + + checksum_fname = checksum["name"] + checksum_path = module_ctx.path(checksum_fname) + downloads[checksum_path] = struct( + download = module_ctx.download( + "{}/{}".format(base_url, checksum_fname), + output = checksum_path, + block = False, + ), + archive_fname = fname, + platforms = checksum["target_triples"], + ) + + for checksum_path, download in downloads.items(): + result = download.download.wait() + if not result.success: + fail(result) + + archive_fname = download.archive_fname + + sha256, _, checksummed_fname = module_ctx.read(checksum_path).partition(" ") + checksummed_fname = checksummed_fname.strip(" *\n") + if archive_fname != checksummed_fname: + fail("The checksum is for a different file, expected '{}' but got '{}'".format( + archive_fname, + checksummed_fname, + )) + + for platform in download.platforms: + tool_sources[platform] = struct( + urls = ["{}/{}".format(base_url, archive_fname)], + sha256 = sha256, + ) + + return tool_sources + uv = module_extension( doc = _DOC, implementation = _uv_toolchain_extension, - tag_classes = {"toolchain": uv_toolchain}, + tag_classes = { + "configure": configure, + "default": default, + }, ) diff --git a/python/uv/private/uv_repositories.bzl b/python/uv/private/uv_repositories.bzl deleted file mode 100644 index 24fb9c2447..0000000000 --- a/python/uv/private/uv_repositories.bzl +++ /dev/null @@ -1,120 +0,0 @@ -# Copyright 2024 The Bazel Authors. All rights reserved. -# -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -""" -EXPERIMENTAL: This is experimental and may be removed without notice - -Create repositories for uv toolchain dependencies -""" - -load(":toolchain_types.bzl", "UV_TOOLCHAIN_TYPE") -load(":uv_toolchains_repo.bzl", "uv_toolchains_repo") -load(":versions.bzl", "UV_PLATFORMS", "UV_TOOL_VERSIONS") - -UV_BUILD_TMPL = """\ -# Generated by repositories.bzl -load("@rules_python//python/uv:uv_toolchain.bzl", "uv_toolchain") - -uv_toolchain( - name = "uv_toolchain", - uv = "{binary}", - version = "{version}", -) -""" - -def _uv_repo_impl(repository_ctx): - platform = repository_ctx.attr.platform - uv_version = repository_ctx.attr.uv_version - - is_windows = "windows" in platform - - suffix = ".zip" if is_windows else ".tar.gz" - filename = "uv-{platform}{suffix}".format( - platform = platform, - suffix = suffix, - ) - url = "https://github.com/astral-sh/uv/releases/download/{version}/{filename}".format( - version = uv_version, - filename = filename, - ) - if filename.endswith(".tar.gz"): - strip_prefix = filename[:-len(".tar.gz")] - else: - strip_prefix = "" - - repository_ctx.download_and_extract( - url = url, - sha256 = UV_TOOL_VERSIONS[repository_ctx.attr.uv_version][repository_ctx.attr.platform].sha256, - stripPrefix = strip_prefix, - ) - - binary = "uv.exe" if is_windows else "uv" - repository_ctx.file( - "BUILD.bazel", - UV_BUILD_TMPL.format( - binary = binary, - version = uv_version, - ), - ) - -uv_repository = repository_rule( - _uv_repo_impl, - doc = "Fetch external tools needed for uv toolchain", - attrs = { - "platform": attr.string(mandatory = True, values = UV_PLATFORMS.keys()), - "uv_version": attr.string(mandatory = True, values = UV_TOOL_VERSIONS.keys()), - }, -) - -def uv_repositories(name = "uv_toolchains", uv_version = None, register_toolchains = True): - """Convenience macro which does typical toolchain setup - - Skip this macro if you need more control over the toolchain setup. - - Args: - name: {type}`str` The name of the toolchains repo. - uv_version: The uv toolchain version to download. - register_toolchains: If true, repositories will be generated to produce and register `uv_toolchain` targets. - """ - if not uv_version: - fail("uv_version is required") - - toolchain_names = [] - toolchain_labels_by_toolchain = {} - toolchain_compatible_with_by_toolchain = {} - - for platform in UV_PLATFORMS.keys(): - uv_repository_name = UV_PLATFORMS[platform].default_repo_name - - uv_repository( - name = uv_repository_name, - uv_version = uv_version, - platform = platform, - ) - - toolchain_name = uv_repository_name + "_toolchain" - toolchain_names.append(toolchain_name) - toolchain_labels_by_toolchain[toolchain_name] = "@{}//:uv_toolchain".format(uv_repository_name) - toolchain_compatible_with_by_toolchain[toolchain_name] = UV_PLATFORMS[platform].compatible_with - - uv_toolchains_repo( - name = name, - toolchain_type = str(UV_TOOLCHAIN_TYPE), - toolchain_names = toolchain_names, - toolchain_labels = toolchain_labels_by_toolchain, - toolchain_compatible_with = toolchain_compatible_with_by_toolchain, - ) - - if register_toolchains: - native.register_toolchains("@{}/:all".format(name)) diff --git a/python/uv/private/uv_repository.bzl b/python/uv/private/uv_repository.bzl new file mode 100644 index 0000000000..ba7d2a766c --- /dev/null +++ b/python/uv/private/uv_repository.bzl @@ -0,0 +1,74 @@ +# Copyright 2024 The Bazel Authors. All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +""" +EXPERIMENTAL: This is experimental and may be removed without notice + +Create repositories for uv toolchain dependencies +""" + +UV_BUILD_TMPL = """\ +# Generated by repositories.bzl +load("@rules_python//python/uv:uv_toolchain.bzl", "uv_toolchain") + +uv_toolchain( + name = "uv_toolchain", + uv = "{binary}", + version = "{version}", +) +""" + +def _uv_repo_impl(repository_ctx): + platform = repository_ctx.attr.platform + + is_windows = "windows" in platform + _, _, filename = repository_ctx.attr.urls[0].rpartition("/") + if filename.endswith(".tar.gz"): + strip_prefix = filename[:-len(".tar.gz")] + else: + strip_prefix = "" + + result = repository_ctx.download_and_extract( + url = repository_ctx.attr.urls, + sha256 = repository_ctx.attr.sha256, + stripPrefix = strip_prefix, + ) + + binary = "uv.exe" if is_windows else "uv" + repository_ctx.file( + "BUILD.bazel", + UV_BUILD_TMPL.format( + binary = binary, + version = repository_ctx.attr.version, + ), + ) + + return { + "name": repository_ctx.attr.name, + "platform": repository_ctx.attr.platform, + "sha256": result.sha256, + "urls": repository_ctx.attr.urls, + "version": repository_ctx.attr.version, + } + +uv_repository = repository_rule( + _uv_repo_impl, + doc = "Fetch external tools needed for uv toolchain", + attrs = { + "platform": attr.string(mandatory = True), + "sha256": attr.string(mandatory = False), + "urls": attr.string_list(mandatory = True), + "version": attr.string(mandatory = True), + }, +) diff --git a/python/uv/private/uv_toolchain.bzl b/python/uv/private/uv_toolchain.bzl index 3b51f5f533..b740fc304d 100644 --- a/python/uv/private/uv_toolchain.bzl +++ b/python/uv/private/uv_toolchain.bzl @@ -30,6 +30,8 @@ def _uv_toolchain_impl(ctx): uv_toolchain_info = UvToolchainInfo( uv = uv, version = ctx.attr.version, + # Exposed for testing/debugging + label = ctx.label, ) # Export all the providers inside our ToolchainInfo diff --git a/python/uv/private/uv_toolchain_info.bzl b/python/uv/private/uv_toolchain_info.bzl index ac1ef310ea..5d70766e7f 100644 --- a/python/uv/private/uv_toolchain_info.bzl +++ b/python/uv/private/uv_toolchain_info.bzl @@ -17,6 +17,11 @@ UvToolchainInfo = provider( doc = "Information about how to invoke the uv executable.", fields = { + "label": """ +:type: Label + +The uv toolchain implementation label returned by the toolchain. +""", "uv": """ :type: Target diff --git a/python/uv/private/uv_toolchains_repo.bzl b/python/uv/private/uv_toolchains_repo.bzl index 9a8858f1b0..7e11e0adb6 100644 --- a/python/uv/private/uv_toolchains_repo.bzl +++ b/python/uv/private/uv_toolchains_repo.bzl @@ -16,37 +16,44 @@ load("//python/private:text_util.bzl", "render") -_TOOLCHAIN_TEMPLATE = """ -toolchain( - name = "{name}", - target_compatible_with = {compatible_with}, - toolchain = "{toolchain_label}", - toolchain_type = "{toolchain_type}", -) -""" +_TEMPLATE = """\ +load("@rules_python//python/uv/private:toolchains_hub.bzl", "toolchains_hub") -def _toolchains_repo_impl(repository_ctx): - build_content = "" - for toolchain_name in repository_ctx.attr.toolchain_names: - toolchain_label = repository_ctx.attr.toolchain_labels[toolchain_name] - toolchain_compatible_with = repository_ctx.attr.toolchain_compatible_with[toolchain_name] +{} +""" - build_content += _TOOLCHAIN_TEMPLATE.format( - name = toolchain_name, - toolchain_type = repository_ctx.attr.toolchain_type, - toolchain_label = toolchain_label, - compatible_with = render.list(toolchain_compatible_with), - ) +def _non_empty(d): + return {k: v for k, v in d.items() if v} - repository_ctx.file("BUILD.bazel", build_content) +def _toolchains_repo_impl(repository_ctx): + contents = _TEMPLATE.format( + render.call( + "toolchains_hub", + name = repr("uv_toolchain"), + toolchains = render.list(repository_ctx.attr.toolchain_names), + implementations = render.dict( + repository_ctx.attr.toolchain_implementations, + ), + target_compatible_with = render.dict( + repository_ctx.attr.toolchain_compatible_with, + value_repr = render.list, + ), + target_settings = render.dict( + _non_empty(repository_ctx.attr.toolchain_target_settings), + value_repr = render.list, + ), + ), + ) + repository_ctx.file("BUILD.bazel", contents) uv_toolchains_repo = repository_rule( _toolchains_repo_impl, doc = "Generates a toolchain hub repository", attrs = { "toolchain_compatible_with": attr.string_list_dict(doc = "A list of platform constraints for this toolchain, keyed by toolchain name.", mandatory = True), - "toolchain_labels": attr.string_dict(doc = "The name of the toolchain implementation target, keyed by toolchain name.", mandatory = True), + "toolchain_implementations": attr.string_dict(doc = "The name of the toolchain implementation target, keyed by toolchain name.", mandatory = True), "toolchain_names": attr.string_list(doc = "List of toolchain names", mandatory = True), + "toolchain_target_settings": attr.string_list_dict(doc = "A list of target_settings constraints for this toolchain, keyed by toolchain name.", mandatory = True), "toolchain_type": attr.string(doc = "The toolchain type of the toolchains", mandatory = True), }, ) diff --git a/python/uv/private/versions.bzl b/python/uv/private/versions.bzl deleted file mode 100644 index 1d68302c74..0000000000 --- a/python/uv/private/versions.bzl +++ /dev/null @@ -1,94 +0,0 @@ -# Copyright 2024 The Bazel Authors. All rights reserved. -# -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -"""Version and integrity information for downloaded artifacts""" - -UV_PLATFORMS = { - "aarch64-apple-darwin": struct( - default_repo_name = "uv_darwin_aarch64", - compatible_with = [ - "@platforms//os:macos", - "@platforms//cpu:aarch64", - ], - ), - "aarch64-unknown-linux-gnu": struct( - default_repo_name = "uv_linux_aarch64", - compatible_with = [ - "@platforms//os:linux", - "@platforms//cpu:aarch64", - ], - ), - "powerpc64le-unknown-linux-gnu": struct( - default_repo_name = "uv_linux_ppc", - compatible_with = [ - "@platforms//os:linux", - "@platforms//cpu:ppc", - ], - ), - "s390x-unknown-linux-gnu": struct( - default_repo_name = "uv_linux_s390x", - compatible_with = [ - "@platforms//os:linux", - "@platforms//cpu:s390x", - ], - ), - "x86_64-apple-darwin": struct( - default_repo_name = "uv_darwin_x86_64", - compatible_with = [ - "@platforms//os:macos", - "@platforms//cpu:x86_64", - ], - ), - "x86_64-pc-windows-msvc": struct( - default_repo_name = "uv_windows_x86_64", - compatible_with = [ - "@platforms//os:windows", - "@platforms//cpu:x86_64", - ], - ), - "x86_64-unknown-linux-gnu": struct( - default_repo_name = "uv_linux_x86_64", - compatible_with = [ - "@platforms//os:linux", - "@platforms//cpu:x86_64", - ], - ), -} - -# From: https://github.com/astral-sh/uv/releases -UV_TOOL_VERSIONS = { - "0.4.25": { - "aarch64-apple-darwin": struct( - sha256 = "bb2ff4348114ef220ca52e44d5086640c4a1a18f797a5f1ab6f8559fc37b1230", - ), - "aarch64-unknown-linux-gnu": struct( - sha256 = "4485852eb8013530c4275cd222c0056ce123f92742321f012610f1b241463f39", - ), - "powerpc64le-unknown-linux-gnu": struct( - sha256 = "32421c61e8d497243171b28c7efd74f039251256ae9e57ce4a457fdd7d045e24", - ), - "s390x-unknown-linux-gnu": struct( - sha256 = "9afa342d87256f5178a592d3eeb44ece8a93e9359db37e31be1b092226338469", - ), - "x86_64-apple-darwin": struct( - sha256 = "f0ec1f79f4791294382bff242691c6502e95853acef080ae3f7c367a8e1beb6f", - ), - "x86_64-pc-windows-msvc": struct( - sha256 = "c5c7fa084ae4e8ac9e3b0b6c4c7b61e9355eb0c86801c4c7728c0cb142701f38", - ), - "x86_64-unknown-linux-gnu": struct( - sha256 = "6cb6eaf711cd7ce5fb1efaa539c5906374c762af547707a2041c9f6fd207769a", - ), - }, -} diff --git a/tests/uv/BUILD.bazel b/tests/uv/BUILD.bazel new file mode 100644 index 0000000000..e69de29bb2 diff --git a/tests/uv/uv/BUILD.bazel b/tests/uv/uv/BUILD.bazel new file mode 100644 index 0000000000..e1535ab5d8 --- /dev/null +++ b/tests/uv/uv/BUILD.bazel @@ -0,0 +1,17 @@ +# Copyright 2024 The Bazel Authors. All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +load(":uv_tests.bzl", "uv_test_suite") + +uv_test_suite(name = "uv_tests") diff --git a/tests/uv/uv/uv_tests.bzl b/tests/uv/uv/uv_tests.bzl new file mode 100644 index 0000000000..bf0deefa88 --- /dev/null +++ b/tests/uv/uv/uv_tests.bzl @@ -0,0 +1,592 @@ +# Copyright 2024 The Bazel Authors. All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +"" + +load("@rules_testing//lib:analysis_test.bzl", "analysis_test") +load("@rules_testing//lib:test_suite.bzl", "test_suite") +load("@rules_testing//lib:truth.bzl", "subjects") +load("//python/uv:uv_toolchain_info.bzl", "UvToolchainInfo") +load("//python/uv/private:uv.bzl", "process_modules") # buildifier: disable=bzl-visibility +load("//python/uv/private:uv_toolchain.bzl", "uv_toolchain") # buildifier: disable=bzl-visibility + +_tests = [] + +def _mock_mctx(*modules, download = None, read = None): + # Here we construct a fake minimal manifest file that we use to mock what would + # be otherwise read from GH files + manifest_files = { + "different.json": { + x: { + "checksum": x + ".sha256", + "kind": "executable-zip", + } + for x in ["linux", "osx"] + } | { + x + ".sha256": { + "name": x + ".sha256", + "target_triples": [x], + } + for x in ["linux", "osx"] + }, + "manifest.json": { + x: { + "checksum": x + ".sha256", + "kind": "executable-zip", + } + for x in ["linux", "os", "osx", "something_extra"] + } | { + x + ".sha256": { + "name": x + ".sha256", + "target_triples": [x], + } + for x in ["linux", "os", "osx", "something_extra"] + }, + } + + fake_fs = { + "linux.sha256": "deadbeef linux", + "os.sha256": "deadbeef os", + "osx.sha256": "deadb00f osx", + } | { + fname: json.encode({"artifacts": contents}) + for fname, contents in manifest_files.items() + } + + return struct( + path = str, + download = download or (lambda *_, **__: struct( + success = True, + wait = lambda: struct( + success = True, + ), + )), + read = read or (lambda x: fake_fs[x]), + modules = [ + struct( + name = modules[0].name, + tags = modules[0].tags, + is_root = modules[0].is_root, + ), + ] + [ + struct( + name = mod.name, + tags = mod.tags, + is_root = False, + ) + for mod in modules[1:] + ], + ) + +def _mod(*, name = None, default = [], configure = [], is_root = True): + return struct( + name = name, # module_name + tags = struct( + default = default, + configure = configure, + ), + is_root = is_root, + ) + +def _process_modules(env, **kwargs): + result = process_modules(hub_repo = struct, **kwargs) + + return env.expect.that_struct( + struct( + names = result.toolchain_names, + implementations = result.toolchain_implementations, + compatible_with = result.toolchain_compatible_with, + target_settings = result.toolchain_target_settings, + ), + attrs = dict( + names = subjects.collection, + implementations = subjects.dict, + compatible_with = subjects.dict, + target_settings = subjects.dict, + ), + ) + +def _default( + base_url = None, + compatible_with = None, + manifest_filename = None, + platform = None, + target_settings = None, + version = None, + **kwargs): + return struct( + base_url = base_url, + compatible_with = [] + (compatible_with or []), # ensure that the type is correct + manifest_filename = manifest_filename, + platform = platform, + target_settings = [] + (target_settings or []), # ensure that the type is correct + version = version, + **kwargs + ) + +def _configure(urls = None, sha256 = None, **kwargs): + # We have the same attributes + return _default(sha256 = sha256, urls = urls, **kwargs) + +def _test_only_defaults(env): + uv = _process_modules( + env, + module_ctx = _mock_mctx( + _mod( + default = [ + _default( + base_url = "https://example.org", + manifest_filename = "manifest.json", + version = "1.0.0", + platform = "some_name", + compatible_with = ["@platforms//:incompatible"], + ), + ], + ), + ), + ) + + # No defined platform means nothing gets registered + uv.names().contains_exactly([ + "none", + ]) + uv.implementations().contains_exactly({ + "none": str(Label("//python:none")), + }) + uv.compatible_with().contains_exactly({ + "none": ["@platforms//:incompatible"], + }) + uv.target_settings().contains_exactly({}) + +_tests.append(_test_only_defaults) + +def _test_manual_url_spec(env): + calls = [] + uv = _process_modules( + env, + module_ctx = _mock_mctx( + _mod( + default = [ + _default( + manifest_filename = "manifest.json", + version = "1.0.0", + ), + _default( + platform = "linux", + compatible_with = ["@platforms//os:linux"], + ), + # This will be ignored because urls are passed for some of + # the binaries. + _default( + platform = "osx", + compatible_with = ["@platforms//os:osx"], + ), + ], + configure = [ + _configure( + platform = "linux", + urls = ["https://example.org/download.zip"], + sha256 = "deadbeef", + ), + ], + ), + read = lambda *args, **kwargs: fail(args, kwargs), + ), + uv_repository = lambda **kwargs: calls.append(kwargs), + ) + + uv.names().contains_exactly([ + "1_0_0_linux", + ]) + uv.implementations().contains_exactly({ + "1_0_0_linux": "@uv_1_0_0_linux//:uv_toolchain", + }) + uv.compatible_with().contains_exactly({ + "1_0_0_linux": ["@platforms//os:linux"], + }) + uv.target_settings().contains_exactly({}) + env.expect.that_collection(calls).contains_exactly([ + { + "name": "uv_1_0_0_linux", + "platform": "linux", + "sha256": "deadbeef", + "urls": ["https://example.org/download.zip"], + "version": "1.0.0", + }, + ]) + +_tests.append(_test_manual_url_spec) + +def _test_defaults(env): + calls = [] + uv = _process_modules( + env, + module_ctx = _mock_mctx( + _mod( + default = [ + _default( + base_url = "https://example.org", + manifest_filename = "manifest.json", + version = "1.0.0", + platform = "linux", + compatible_with = ["@platforms//os:linux"], + target_settings = ["//:my_flag"], + ), + ], + configure = [ + _configure(), # use defaults + ], + ), + ), + uv_repository = lambda **kwargs: calls.append(kwargs), + ) + + uv.names().contains_exactly([ + "1_0_0_linux", + ]) + uv.implementations().contains_exactly({ + "1_0_0_linux": "@uv_1_0_0_linux//:uv_toolchain", + }) + uv.compatible_with().contains_exactly({ + "1_0_0_linux": ["@platforms//os:linux"], + }) + uv.target_settings().contains_exactly({ + "1_0_0_linux": ["//:my_flag"], + }) + env.expect.that_collection(calls).contains_exactly([ + { + "name": "uv_1_0_0_linux", + "platform": "linux", + "sha256": "deadbeef", + "urls": ["https://example.org/1.0.0/linux"], + "version": "1.0.0", + }, + ]) + +_tests.append(_test_defaults) + +def _test_default_building(env): + calls = [] + uv = _process_modules( + env, + module_ctx = _mock_mctx( + _mod( + default = [ + _default( + base_url = "https://example.org", + manifest_filename = "manifest.json", + version = "1.0.0", + ), + _default( + platform = "linux", + compatible_with = ["@platforms//os:linux"], + target_settings = ["//:my_flag"], + ), + _default( + platform = "osx", + compatible_with = ["@platforms//os:osx"], + ), + ], + configure = [ + _configure(), # use defaults + ], + ), + ), + uv_repository = lambda **kwargs: calls.append(kwargs), + ) + + uv.names().contains_exactly([ + "1_0_0_linux", + "1_0_0_osx", + ]) + uv.implementations().contains_exactly({ + "1_0_0_linux": "@uv_1_0_0_linux//:uv_toolchain", + "1_0_0_osx": "@uv_1_0_0_osx//:uv_toolchain", + }) + uv.compatible_with().contains_exactly({ + "1_0_0_linux": ["@platforms//os:linux"], + "1_0_0_osx": ["@platforms//os:osx"], + }) + uv.target_settings().contains_exactly({ + "1_0_0_linux": ["//:my_flag"], + }) + env.expect.that_collection(calls).contains_exactly([ + { + "name": "uv_1_0_0_linux", + "platform": "linux", + "sha256": "deadbeef", + "urls": ["https://example.org/1.0.0/linux"], + "version": "1.0.0", + }, + { + "name": "uv_1_0_0_osx", + "platform": "osx", + "sha256": "deadb00f", + "urls": ["https://example.org/1.0.0/osx"], + "version": "1.0.0", + }, + ]) + +_tests.append(_test_default_building) + +def _test_complex_configuring(env): + calls = [] + uv = _process_modules( + env, + module_ctx = _mock_mctx( + _mod( + default = [ + _default( + base_url = "https://example.org", + manifest_filename = "manifest.json", + version = "1.0.0", + platform = "osx", + compatible_with = ["@platforms//os:os"], + ), + ], + configure = [ + _configure(), # use defaults + _configure( + version = "1.0.1", + ), # use defaults + _configure( + version = "1.0.2", + base_url = "something_different", + manifest_filename = "different.json", + ), # use defaults + _configure( + platform = "osx", + compatible_with = ["@platforms//os:different"], + ), + _configure( + version = "1.0.3", + ), + _configure(platform = "osx"), # remove the default + _configure( + platform = "linux", + compatible_with = ["@platforms//os:linux"], + ), + ], + ), + ), + uv_repository = lambda **kwargs: calls.append(kwargs), + ) + + uv.names().contains_exactly([ + "1_0_0_osx", + "1_0_1_osx", + "1_0_2_osx", + "1_0_3_linux", + ]) + uv.implementations().contains_exactly({ + "1_0_0_osx": "@uv_1_0_0_osx//:uv_toolchain", + "1_0_1_osx": "@uv_1_0_1_osx//:uv_toolchain", + "1_0_2_osx": "@uv_1_0_2_osx//:uv_toolchain", + "1_0_3_linux": "@uv_1_0_3_linux//:uv_toolchain", + }) + uv.compatible_with().contains_exactly({ + "1_0_0_osx": ["@platforms//os:os"], + "1_0_1_osx": ["@platforms//os:os"], + "1_0_2_osx": ["@platforms//os:different"], + "1_0_3_linux": ["@platforms//os:linux"], + }) + uv.target_settings().contains_exactly({}) + env.expect.that_collection(calls).contains_exactly([ + { + "name": "uv_1_0_0_osx", + "platform": "osx", + "sha256": "deadb00f", + "urls": ["https://example.org/1.0.0/osx"], + "version": "1.0.0", + }, + { + "name": "uv_1_0_1_osx", + "platform": "osx", + "sha256": "deadb00f", + "urls": ["https://example.org/1.0.1/osx"], + "version": "1.0.1", + }, + { + "name": "uv_1_0_2_osx", + "platform": "osx", + "sha256": "deadb00f", + "urls": ["something_different/1.0.2/osx"], + "version": "1.0.2", + }, + { + "name": "uv_1_0_3_linux", + "platform": "linux", + "sha256": "deadbeef", + "urls": ["https://example.org/1.0.3/linux"], + "version": "1.0.3", + }, + ]) + +_tests.append(_test_complex_configuring) + +def _test_non_rules_python_non_root_is_ignored(env): + calls = [] + uv = _process_modules( + env, + module_ctx = _mock_mctx( + _mod( + default = [ + _default( + base_url = "https://example.org", + manifest_filename = "manifest.json", + version = "1.0.0", + platform = "osx", + compatible_with = ["@platforms//os:os"], + ), + ], + configure = [ + _configure(), # use defaults + ], + ), + _mod( + name = "something", + configure = [ + _configure(version = "6.6.6"), # use defaults whatever they are + ], + ), + ), + uv_repository = lambda **kwargs: calls.append(kwargs), + ) + + uv.names().contains_exactly([ + "1_0_0_osx", + ]) + uv.implementations().contains_exactly({ + "1_0_0_osx": "@uv_1_0_0_osx//:uv_toolchain", + }) + uv.compatible_with().contains_exactly({ + "1_0_0_osx": ["@platforms//os:os"], + }) + uv.target_settings().contains_exactly({}) + env.expect.that_collection(calls).contains_exactly([ + { + "name": "uv_1_0_0_osx", + "platform": "osx", + "sha256": "deadb00f", + "urls": ["https://example.org/1.0.0/osx"], + "version": "1.0.0", + }, + ]) + +_tests.append(_test_non_rules_python_non_root_is_ignored) + +def _test_rules_python_does_not_take_precedence(env): + calls = [] + uv = _process_modules( + env, + module_ctx = _mock_mctx( + _mod( + default = [ + _default( + base_url = "https://example.org", + manifest_filename = "manifest.json", + version = "1.0.0", + platform = "osx", + compatible_with = ["@platforms//os:os"], + ), + ], + configure = [ + _configure(), # use defaults + ], + ), + _mod( + name = "rules_python", + configure = [ + _configure( + version = "1.0.0", + base_url = "https://foobar.org", + platform = "osx", + compatible_with = ["@platforms//os:osx"], + ), + ], + ), + ), + uv_repository = lambda **kwargs: calls.append(kwargs), + ) + + uv.names().contains_exactly([ + "1_0_0_osx", + ]) + uv.implementations().contains_exactly({ + "1_0_0_osx": "@uv_1_0_0_osx//:uv_toolchain", + }) + uv.compatible_with().contains_exactly({ + "1_0_0_osx": ["@platforms//os:os"], + }) + uv.target_settings().contains_exactly({}) + env.expect.that_collection(calls).contains_exactly([ + { + "name": "uv_1_0_0_osx", + "platform": "osx", + "sha256": "deadb00f", + "urls": ["https://example.org/1.0.0/osx"], + "version": "1.0.0", + }, + ]) + +_tests.append(_test_rules_python_does_not_take_precedence) + +_analysis_tests = [] + +def _test_toolchain_precedence(name): + analysis_test( + name = name, + impl = _test_toolchain_precedence_impl, + target = "//python/uv:current_toolchain", + config_settings = { + "//command_line_option:extra_toolchains": [ + str(Label("//tests/uv/uv_toolchains:all")), + ], + "//command_line_option:platforms": str(Label("//tests/support:linux_aarch64")), + }, + ) + +def _test_toolchain_precedence_impl(env, target): + # Check that the forwarded UvToolchainInfo looks vaguely correct. + uv_info = env.expect.that_target(target).provider( + UvToolchainInfo, + factory = lambda v, meta: v, + ) + env.expect.that_str(str(uv_info.label)).contains("//tests/uv/uv:fake_foof") + +_analysis_tests.append(_test_toolchain_precedence) + +def uv_test_suite(name): + """Create the test suite. + + Args: + name: the name of the test suite + """ + test_suite( + name = name, + basic_tests = _tests, + tests = _analysis_tests, + ) + + uv_toolchain( + name = "fake_bar", + uv = ":BUILD.bazel", + version = "0.0.1", + ) + + uv_toolchain( + name = "fake_foof", + uv = ":BUILD.bazel", + version = "0.0.1", + ) diff --git a/tests/uv/uv_toolchains/BUILD.bazel b/tests/uv/uv_toolchains/BUILD.bazel new file mode 100644 index 0000000000..4e2a12dcae --- /dev/null +++ b/tests/uv/uv_toolchains/BUILD.bazel @@ -0,0 +1,25 @@ +load("//python/uv/private:toolchains_hub.bzl", "toolchains_hub") # buildifier: disable=bzl-visibility + +toolchains_hub( + name = "uv_unit_test", + implementations = { + "bar": "//tests/uv/uv:fake_bar", + "foo": "//tests/uv/uv:fake_foof", + }, + target_compatible_with = { + "bar": [ + "@platforms//os:linux", + "@platforms//cpu:aarch64", + ], + "foo": [ + "@platforms//os:linux", + "@platforms//cpu:aarch64", + ], + }, + target_settings = {}, + # We expect foo to take precedence over bar + toolchains = [ + "foo", + "bar", + ], +) From 8f517315d807ffd8a7ba330f1ed5e3065e18bc36 Mon Sep 17 00:00:00 2001 From: Kevin Lloyd Bernal Date: Tue, 11 Mar 2025 21:31:53 +1100 Subject: [PATCH 098/922] fix(coverage): missing files in the coverage report if they have no tests (#2607) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit This ensures that un-executed files _(i.e. files that aren't tested)_ are included in the coverage report. The current behavior is that coverage.py excludes them by default. This PR configures source files via the auto-generated `.coveragerc` file. See https://coverage.readthedocs.io/en/7.6.10/source.html#execution: > If the source option is specified, only code in those locations will be measured. Specifying the source option also enables coverage.py to report on un-executed files, since it can search the source tree for files that haven’t been measured at all. Closes #2599 Closes #2597 Fixes #2575 --------- Co-authored-by: Ignas Anikevicius <240938+aignas@users.noreply.github.com> --- CHANGELOG.md | 1 + examples/bzlmod/.python_version | 1 + python/private/python_bootstrap_template.txt | 11 ++++++++++- python/private/stage2_bootstrap_template.py | 11 ++++++++++- 4 files changed, 22 insertions(+), 2 deletions(-) create mode 100644 examples/bzlmod/.python_version diff --git a/CHANGELOG.md b/CHANGELOG.md index 413442eb99..403dbafade 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -70,6 +70,7 @@ Unreleased changes template. ([#1169](https://github.com/bazelbuild/rules_python/issues/1169)). * (gazelle) Don't collapse depsets to a list or into args when generating the modules mapping file. Support spilling modules mapping args into a params file. +* (coverage) Fix missing files in the coverage report if they have no tests. * (pypi) From now on `python` invocations in repository and module extension evaluation contexts will invoke Python interpreter with `-B` to avoid creating `.pyc` files. diff --git a/examples/bzlmod/.python_version b/examples/bzlmod/.python_version new file mode 100644 index 0000000000..bd28b9c5c2 --- /dev/null +++ b/examples/bzlmod/.python_version @@ -0,0 +1 @@ +3.9 diff --git a/python/private/python_bootstrap_template.txt b/python/private/python_bootstrap_template.txt index e3b39e30cd..9f671ddda5 100644 --- a/python/private/python_bootstrap_template.txt +++ b/python/private/python_bootstrap_template.txt @@ -425,12 +425,21 @@ def _RunForCoverage(python_program, main_filename, args, env, directory under the runfiles tree, and will recursively delete the runfiles directory if set. """ + instrumented_files = [abs_path for abs_path, _ in InstrumentedFilePaths()] + unique_dirs = {os.path.dirname(file) for file in instrumented_files} + source = "\n\t".join(unique_dirs) + + PrintVerboseCoverage("[coveragepy] Instrumented Files:\n" + "\n".join(instrumented_files)) + PrintVerboseCoverage("[coveragepy] Sources:\n" + "\n".join(unique_dirs)) + # We need for coveragepy to use relative paths. This can only be configured unique_id = uuid.uuid4() rcfile_name = os.path.join(os.environ['COVERAGE_DIR'], ".coveragerc_{}".format(unique_id)) with open(rcfile_name, "w") as rcfile: - rcfile.write('''[run] + rcfile.write(f'''[run] relative_files = True +source = +\t{source} ''') PrintVerboseCoverage('Coverage entrypoint:', coverage_entrypoint) # First run the target Python file via coveragepy to create a .coverage diff --git a/python/private/stage2_bootstrap_template.py b/python/private/stage2_bootstrap_template.py index b1f6b031aa..4687bc003f 100644 --- a/python/private/stage2_bootstrap_template.py +++ b/python/private/stage2_bootstrap_template.py @@ -276,6 +276,13 @@ def _maybe_collect_coverage(enable): yield return + instrumented_files = [abs_path for abs_path, _ in instrumented_file_paths()] + unique_dirs = {os.path.dirname(file) for file in instrumented_files} + source = "\n\t".join(unique_dirs) + + print_verbose_coverage("Instrumented Files:\n" + "\n".join(instrumented_files)) + print_verbose_coverage("Sources:\n" + "\n".join(unique_dirs)) + import uuid import coverage @@ -289,8 +296,10 @@ def _maybe_collect_coverage(enable): print_verbose_coverage("coveragerc file:", rcfile_name) with open(rcfile_name, "w") as rcfile: rcfile.write( - """[run] + f"""[run] relative_files = True +source = +\t{source} """ ) try: From 5a8f6c4acd4190421e58f5ecc6f099f1ce406cb8 Mon Sep 17 00:00:00 2001 From: Chris Chua Date: Wed, 12 Mar 2025 20:09:52 +0800 Subject: [PATCH 099/922] feat(pypi): support direct urls for wheels in bazel downloader (#2655) This PR adds support for installing wheels via direct urls in the requirements lock file: ``` foo==0.0.1 @ https://someurl.org/package.whl bar==0.0.1 @ https://someurl.org/package.tar.gz ``` This is to improve parity between bazel downloader and pip behavior. Before this change, direct urls used fallback to pip install. Partially addresses #2363 as it does not add support for git urls. --- CHANGELOG.md | 3 + python/private/pypi/index_sources.bzl | 7 +- python/private/pypi/parse_requirements.bzl | 17 +++ .../index_sources/index_sources_tests.bzl | 25 +++- .../parse_requirements_tests.bzl | 122 +++++++++++++++++- 5 files changed, 167 insertions(+), 7 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 403dbafade..9029794ffc 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -88,6 +88,9 @@ Unreleased changes template. GitHub releases page metadata published by the `uv` project. * (pypi) An extra argument to add the interpreter lib dir to `LDFLAGS` when building wheels from `sdist`. +* (pypi) Direct HTTP urls for wheels and sdists are now supported when using + {obj}`experimental_index_url` (bazel downloader). + Partially fixes [#2363](https://github.com/bazelbuild/rules_python/issues/2363). {#v0-0-0-removed} ### Removed diff --git a/python/private/pypi/index_sources.bzl b/python/private/pypi/index_sources.bzl index 8b3c300946..e3762d2a48 100644 --- a/python/private/pypi/index_sources.bzl +++ b/python/private/pypi/index_sources.bzl @@ -32,6 +32,7 @@ def index_sources(line): * `marker` - str; the marker expression, as per PEP508 spec. * `requirement` - str; a requirement line without the marker. This can be given to `pip` to install a package. + * `url` - str; URL if the requirement specifies a direct URL, empty string otherwise. """ line = line.replace("\\", " ") head, _, maybe_hashes = line.partition(";") @@ -55,9 +56,12 @@ def index_sources(line): requirement, " ".join(["--hash=sha256:{}".format(sha) for sha in shas]), ).strip() + + url = "" if "@" in head: requirement = requirement_line - shas = [] + _, _, url_and_rest = requirement.partition("@") + url = url_and_rest.strip().partition(" ")[0].strip() return struct( requirement = requirement, @@ -65,4 +69,5 @@ def index_sources(line): version = version, shas = sorted(shas), marker = marker, + url = url, ) diff --git a/python/private/pypi/parse_requirements.bzl b/python/private/pypi/parse_requirements.bzl index 2bca8d8621..dbff44ecb3 100644 --- a/python/private/pypi/parse_requirements.bzl +++ b/python/private/pypi/parse_requirements.bzl @@ -292,6 +292,23 @@ def _add_dists(*, requirement, index_urls, logger = None): index_urls: The result of simpleapi_download. logger: A logger for printing diagnostic info. """ + + # Handle direct URLs in requirements + if requirement.srcs.url: + url = requirement.srcs.url + _, _, filename = url.rpartition("/") + direct_url_dist = struct( + url = url, + filename = filename, + sha256 = requirement.srcs.shas[0] if requirement.srcs.shas else "", + yanked = False, + ) + + if filename.endswith(".whl"): + return [direct_url_dist], None + else: + return [], direct_url_dist + if not index_urls: return [], None diff --git a/tests/pypi/index_sources/index_sources_tests.bzl b/tests/pypi/index_sources/index_sources_tests.bzl index 440957e2f0..ffeed87a7b 100644 --- a/tests/pypi/index_sources/index_sources_tests.bzl +++ b/tests/pypi/index_sources/index_sources_tests.bzl @@ -24,27 +24,39 @@ def _test_no_simple_api_sources(env): "foo==0.0.1": struct( requirement = "foo==0.0.1", marker = "", + url = "", ), "foo==0.0.1 @ https://someurl.org": struct( requirement = "foo==0.0.1 @ https://someurl.org", marker = "", + url = "https://someurl.org", ), - "foo==0.0.1 @ https://someurl.org --hash=sha256:deadbeef": struct( - requirement = "foo==0.0.1 @ https://someurl.org --hash=sha256:deadbeef", + "foo==0.0.1 @ https://someurl.org/package.whl": struct( + requirement = "foo==0.0.1 @ https://someurl.org/package.whl", marker = "", + url = "https://someurl.org/package.whl", ), - "foo==0.0.1 @ https://someurl.org; python_version < \"2.7\"\\ --hash=sha256:deadbeef": struct( - requirement = "foo==0.0.1 @ https://someurl.org --hash=sha256:deadbeef", + "foo==0.0.1 @ https://someurl.org/package.whl --hash=sha256:deadbeef": struct( + requirement = "foo==0.0.1 @ https://someurl.org/package.whl --hash=sha256:deadbeef", + marker = "", + url = "https://someurl.org/package.whl", + shas = ["deadbeef"], + ), + "foo==0.0.1 @ https://someurl.org/package.whl; python_version < \"2.7\"\\ --hash=sha256:deadbeef": struct( + requirement = "foo==0.0.1 @ https://someurl.org/package.whl --hash=sha256:deadbeef", marker = "python_version < \"2.7\"", + url = "https://someurl.org/package.whl", + shas = ["deadbeef"], ), } for input, want in inputs.items(): got = index_sources(input) - env.expect.that_collection(got.shas).contains_exactly([]) + env.expect.that_collection(got.shas).contains_exactly(want.shas if hasattr(want, "shas") else []) env.expect.that_str(got.version).equals("0.0.1") env.expect.that_str(got.requirement).equals(want.requirement) env.expect.that_str(got.requirement_line).equals(got.requirement) env.expect.that_str(got.marker).equals(want.marker) + env.expect.that_str(got.url).equals(want.url) _tests.append(_test_no_simple_api_sources) @@ -58,6 +70,7 @@ def _test_simple_api_sources(env): marker = "", requirement = "foo==0.0.2", requirement_line = "foo==0.0.2 --hash=sha256:deafbeef --hash=sha256:deadbeef", + url = "", ), "foo[extra]==0.0.2; (python_version < 2.7 or extra == \"@\") --hash=sha256:deafbeef --hash=sha256:deadbeef": struct( shas = [ @@ -67,6 +80,7 @@ def _test_simple_api_sources(env): marker = "(python_version < 2.7 or extra == \"@\")", requirement = "foo[extra]==0.0.2", requirement_line = "foo[extra]==0.0.2 --hash=sha256:deafbeef --hash=sha256:deadbeef", + url = "", ), } for input, want in tests.items(): @@ -76,6 +90,7 @@ def _test_simple_api_sources(env): env.expect.that_str(got.requirement).equals(want.requirement) env.expect.that_str(got.requirement_line).equals(want.requirement_line) env.expect.that_str(got.marker).equals(want.marker) + env.expect.that_str(got.url).equals(want.url) _tests.append(_test_simple_api_sources) diff --git a/tests/pypi/parse_requirements/parse_requirements_tests.bzl b/tests/pypi/parse_requirements/parse_requirements_tests.bzl index 77e22b825a..8edc2689bf 100644 --- a/tests/pypi/parse_requirements/parse_requirements_tests.bzl +++ b/tests/pypi/parse_requirements/parse_requirements_tests.bzl @@ -26,7 +26,10 @@ foo==0.0.1 \ --hash=sha256:deadb00f """, "requirements_direct": """\ -foo[extra] @ https://some-url +foo[extra] @ https://some-url/package.whl +bar @ https://example.org/bar-1.0.whl --hash=sha256:deadbeef +baz @ https://test.com/baz-2.0.whl; python_version < "3.8" --hash=sha256:deadb00f +qux @ https://example.org/qux-1.0.tar.gz --hash=sha256:deadbe0f """, "requirements_extra_args": """\ --index-url=example.org @@ -106,6 +109,7 @@ def _test_simple(env): requirement_line = "foo[extra]==0.0.1 --hash=sha256:deadbeef", shas = ["deadbeef"], version = "0.0.1", + url = "", ), target_platforms = [ "linux_x86_64", @@ -124,6 +128,110 @@ def _test_simple(env): _tests.append(_test_simple) +def _test_direct_urls(env): + got = parse_requirements( + ctx = _mock_ctx(), + requirements_by_platform = { + "requirements_direct": ["linux_x86_64"], + }, + ) + env.expect.that_dict(got).contains_exactly({ + "bar": [ + struct( + distribution = "bar", + extra_pip_args = [], + sdist = None, + is_exposed = True, + srcs = struct( + marker = "", + requirement = "bar @ https://example.org/bar-1.0.whl --hash=sha256:deadbeef", + requirement_line = "bar @ https://example.org/bar-1.0.whl --hash=sha256:deadbeef", + shas = ["deadbeef"], + version = "", + url = "https://example.org/bar-1.0.whl", + ), + target_platforms = ["linux_x86_64"], + whls = [struct( + url = "https://example.org/bar-1.0.whl", + filename = "bar-1.0.whl", + sha256 = "deadbeef", + yanked = False, + )], + ), + ], + "baz": [ + struct( + distribution = "baz", + extra_pip_args = [], + sdist = None, + is_exposed = True, + srcs = struct( + marker = "python_version < \"3.8\"", + requirement = "baz @ https://test.com/baz-2.0.whl --hash=sha256:deadb00f", + requirement_line = "baz @ https://test.com/baz-2.0.whl --hash=sha256:deadb00f", + shas = ["deadb00f"], + version = "", + url = "https://test.com/baz-2.0.whl", + ), + target_platforms = ["linux_x86_64"], + whls = [struct( + url = "https://test.com/baz-2.0.whl", + filename = "baz-2.0.whl", + sha256 = "deadb00f", + yanked = False, + )], + ), + ], + "foo": [ + struct( + distribution = "foo", + extra_pip_args = [], + sdist = None, + is_exposed = True, + srcs = struct( + marker = "", + requirement = "foo[extra] @ https://some-url/package.whl", + requirement_line = "foo[extra] @ https://some-url/package.whl", + shas = [], + version = "", + url = "https://some-url/package.whl", + ), + target_platforms = ["linux_x86_64"], + whls = [struct( + url = "https://some-url/package.whl", + filename = "package.whl", + sha256 = "", + yanked = False, + )], + ), + ], + "qux": [ + struct( + distribution = "qux", + extra_pip_args = [], + sdist = struct( + url = "https://example.org/qux-1.0.tar.gz", + filename = "qux-1.0.tar.gz", + sha256 = "deadbe0f", + yanked = False, + ), + is_exposed = True, + srcs = struct( + marker = "", + requirement = "qux @ https://example.org/qux-1.0.tar.gz --hash=sha256:deadbe0f", + requirement_line = "qux @ https://example.org/qux-1.0.tar.gz --hash=sha256:deadbe0f", + shas = ["deadbe0f"], + version = "", + url = "https://example.org/qux-1.0.tar.gz", + ), + target_platforms = ["linux_x86_64"], + whls = [], + ), + ], + }) + +_tests.append(_test_direct_urls) + def _test_extra_pip_args(env): got = parse_requirements( ctx = _mock_ctx(), @@ -145,6 +253,7 @@ def _test_extra_pip_args(env): requirement_line = "foo[extra]==0.0.1 --hash=sha256:deadbeef", shas = ["deadbeef"], version = "0.0.1", + url = "", ), target_platforms = [ "linux_x86_64", @@ -182,6 +291,7 @@ def _test_dupe_requirements(env): requirement_line = "foo[extra,extra_2]==0.0.1 --hash=sha256:deadbeef", shas = ["deadbeef"], version = "0.0.1", + url = "", ), target_platforms = ["linux_x86_64"], whls = [], @@ -211,6 +321,7 @@ def _test_multi_os(env): requirement_line = "bar==0.0.1 --hash=sha256:deadb00f", shas = ["deadb00f"], version = "0.0.1", + url = "", ), target_platforms = ["windows_x86_64"], whls = [], @@ -228,6 +339,7 @@ def _test_multi_os(env): requirement_line = "foo==0.0.3 --hash=sha256:deadbaaf", shas = ["deadbaaf"], version = "0.0.3", + url = "", ), target_platforms = ["linux_x86_64"], whls = [], @@ -243,6 +355,7 @@ def _test_multi_os(env): requirement_line = "foo[extra]==0.0.2 --hash=sha256:deadbeef", shas = ["deadbeef"], version = "0.0.2", + url = "", ), target_platforms = ["windows_x86_64"], whls = [], @@ -282,6 +395,7 @@ def _test_multi_os_legacy(env): requirement_line = "bar==0.0.1 --hash=sha256:deadb00f", shas = ["deadb00f"], version = "0.0.1", + url = "", ), target_platforms = ["cp39_linux_x86_64"], whls = [], @@ -299,6 +413,7 @@ def _test_multi_os_legacy(env): requirement_line = "foo==0.0.1 --hash=sha256:deadbeef", shas = ["deadbeef"], version = "0.0.1", + url = "", ), target_platforms = ["cp39_linux_x86_64"], whls = [], @@ -314,6 +429,7 @@ def _test_multi_os_legacy(env): requirement = "foo==0.0.3", shas = ["deadbaaf"], version = "0.0.3", + url = "", ), target_platforms = ["cp39_osx_aarch64"], whls = [], @@ -367,6 +483,7 @@ def _test_env_marker_resolution(env): requirement_line = "bar==0.0.1 --hash=sha256:deadbeef", shas = ["deadbeef"], version = "0.0.1", + url = "", ), target_platforms = ["cp311_linux_super_exotic", "cp311_windows_x86_64"], whls = [], @@ -384,6 +501,7 @@ def _test_env_marker_resolution(env): requirement_line = "foo[extra]==0.0.1 --hash=sha256:deadbeef", shas = ["deadbeef"], version = "0.0.1", + url = "", ), target_platforms = ["cp311_windows_x86_64"], whls = [], @@ -419,6 +537,7 @@ def _test_different_package_version(env): requirement_line = "foo==0.0.1 --hash=sha256:deadb00f", shas = ["deadb00f"], version = "0.0.1", + url = "", ), target_platforms = ["linux_x86_64"], whls = [], @@ -434,6 +553,7 @@ def _test_different_package_version(env): requirement_line = "foo==0.0.1+local --hash=sha256:deadbeef", shas = ["deadbeef"], version = "0.0.1+local", + url = "", ), target_platforms = ["linux_x86_64"], whls = [], From 389431bba6f9a4b46b6cf15dd9cd24a1f52f6e16 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Thu, 13 Mar 2025 16:25:06 -0700 Subject: [PATCH 100/922] refactor: API for deriving customized versions of the base rules (#2610) This implements a "builder style" API to allow arbitrary modification of rule, attr, etc objects used when defining a rule. The net effect is users are able to use the base definition for our rules, but define their own with the modifications they need, without having to copy/paste portions our implementation, load private files, or patch source. The basic way it works is a mutable object ("builder") holds the args and state that would be used to create the immutable Bazel object. When `build()` is called, the immutable Bazel object (e.g. `attr.string()`) is created. Builders are implemented for most objects and their settings (rule, attrs, and supporting objects). This design is necessary because of three Bazel behaviors: * attr etc objects are immutable, which means we must keep our own state * attr etc objects aren't inspectable, which means we must store the arguments for creating the immutable objects. * Starlark objects are frozen after initial bzl file evaluation, which means creation of any mutable object must be done at the point of use. The resulting API resembles the builder APIs common in other languages: ``` r = create_py_binary_rule_builder() r.attrs.get("srcs").set_mandatory(True) r.attrs.get("deps").aspects().append(my_aspect) my_py_binary = r.build() ``` Most objects are thin wrappers for managing a kwargs dict. As such, and because they're wrapping a foreign API, they aren't strict in enforcing their internal state and the kwargs dict is publicly exposed as an escape hatch. As of this PR, no public API for e.g. `create_py_binary_rule_builder()` is exposed. That'll come in a separate PR (to add public access points under python/api). Work towards https://github.com/bazelbuild/rules_python/issues/1647 --- docs/BUILD.bazel | 3 + docs/_includes/field_kwargs_doc.md | 11 + python/private/BUILD.bazel | 32 + python/private/attr_builders.bzl | 1360 ++++++++++++++++++++ python/private/attributes.bzl | 206 ++- python/private/builders.bzl | 228 ---- python/private/builders_util.bzl | 116 ++ python/private/common.bzl | 46 - python/private/py_binary_rule.bzl | 18 - python/private/py_executable.bzl | 63 +- python/private/py_library.bzl | 33 +- python/private/py_library_rule.bzl | 6 +- python/private/py_runtime_rule.bzl | 139 +- python/private/py_test_rule.bzl | 18 - python/private/rule_builders.bzl | 692 ++++++++++ sphinxdocs/inventories/bazel_inventory.txt | 8 + tests/builders/BUILD.bazel | 36 + tests/builders/attr_builders_tests.bzl | 468 +++++++ tests/builders/rule_builders_tests.bzl | 256 ++++ tests/support/empty_toolchain/BUILD.bazel | 3 + tests/support/empty_toolchain/empty.bzl | 23 + tests/support/sh_py_run_test.bzl | 20 +- 22 files changed, 3238 insertions(+), 547 deletions(-) create mode 100644 docs/_includes/field_kwargs_doc.md create mode 100644 python/private/attr_builders.bzl create mode 100644 python/private/builders_util.bzl create mode 100644 python/private/rule_builders.bzl create mode 100644 tests/builders/attr_builders_tests.bzl create mode 100644 tests/builders/rule_builders_tests.bzl create mode 100644 tests/support/empty_toolchain/BUILD.bazel create mode 100644 tests/support/empty_toolchain/empty.bzl diff --git a/docs/BUILD.bazel b/docs/BUILD.bazel index 0c07002a01..e19c22113f 100644 --- a/docs/BUILD.bazel +++ b/docs/BUILD.bazel @@ -103,11 +103,14 @@ sphinx_stardocs( "//python/cc:py_cc_toolchain_bzl", "//python/cc:py_cc_toolchain_info_bzl", "//python/entry_points:py_console_script_binary_bzl", + "//python/private:attr_builders_bzl", + "//python/private:builders_util_bzl", "//python/private:py_binary_rule_bzl", "//python/private:py_cc_toolchain_rule_bzl", "//python/private:py_library_rule_bzl", "//python/private:py_runtime_rule_bzl", "//python/private:py_test_rule_bzl", + "//python/private:rule_builders_bzl", "//python/private/api:py_common_api_bzl", "//python/private/pypi:config_settings_bzl", "//python/private/pypi:pkg_aliases_bzl", diff --git a/docs/_includes/field_kwargs_doc.md b/docs/_includes/field_kwargs_doc.md new file mode 100644 index 0000000000..0241947b43 --- /dev/null +++ b/docs/_includes/field_kwargs_doc.md @@ -0,0 +1,11 @@ +:::{field} kwargs +:type: dict[str, Any] + +Additional kwargs to use when building. This is to allow manipulations that +aren't directly supported by the builder's API. The state of this dict +may or may not reflect prior API calls, and subsequent API calls may +modify this dict. The general contract is that modifications to this will +be respected when `build()` is called, assuming there were no API calls +in between. +::: + diff --git a/python/private/BUILD.bazel b/python/private/BUILD.bazel index 2928dab068..b7e52a35aa 100644 --- a/python/private/BUILD.bazel +++ b/python/private/BUILD.bazel @@ -51,10 +51,20 @@ filegroup( visibility = ["//python:__pkg__"], ) +bzl_library( + name = "attr_builders_bzl", + srcs = ["attr_builders.bzl"], + deps = [ + ":builders_util_bzl", + "@bazel_skylib//lib:types", + ], +) + bzl_library( name = "attributes_bzl", srcs = ["attributes.bzl"], deps = [ + ":attr_builders_bzl", ":common_bzl", ":enum_bzl", ":flags_bzl", @@ -92,6 +102,14 @@ bzl_library( ], ) +bzl_library( + name = "builders_util_bzl", + srcs = ["builders_util.bzl"], + deps = [ + "@bazel_skylib//lib:types", + ], +) + bzl_library( name = "bzlmod_enabled_bzl", srcs = ["bzlmod_enabled.bzl"], @@ -283,6 +301,7 @@ bzl_library( deps = [ ":attributes_bzl", ":py_executable_bzl", + ":rule_builders_bzl", ":semantics_bzl", "@bazel_skylib//lib:dicts", ], @@ -410,6 +429,7 @@ bzl_library( ":flags_bzl", ":py_cc_link_params_info_bzl", ":py_internal_bzl", + ":rule_builders_bzl", ":toolchain_types_bzl", "@bazel_skylib//lib:dicts", "@bazel_skylib//rules:common_settings", @@ -475,6 +495,7 @@ bzl_library( ":py_internal_bzl", ":py_runtime_info_bzl", ":reexports_bzl", + ":rule_builders_bzl", ":util_bzl", "@bazel_skylib//lib:dicts", "@bazel_skylib//lib:paths", @@ -515,6 +536,7 @@ bzl_library( ":attributes_bzl", ":common_bzl", ":py_executable_bzl", + ":rule_builders_bzl", ":semantics_bzl", "@bazel_skylib//lib:dicts", ], @@ -563,6 +585,16 @@ bzl_library( srcs = ["repo_utils.bzl"], ) +bzl_library( + name = "rule_builders_bzl", + srcs = ["rule_builders.bzl"], + deps = [ + ":builders_bzl", + ":builders_util_bzl", + "@bazel_skylib//lib:types", + ], +) + bzl_library( name = "semver_bzl", srcs = ["semver.bzl"], diff --git a/python/private/attr_builders.bzl b/python/private/attr_builders.bzl new file mode 100644 index 0000000000..acd1d40394 --- /dev/null +++ b/python/private/attr_builders.bzl @@ -0,0 +1,1360 @@ +# Copyright 2025 The Bazel Authors. All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +"""Builders for creating attributes et al.""" + +load("@bazel_skylib//lib:types.bzl", "types") +load( + ":builders_util.bzl", + "kwargs_getter", + "kwargs_getter_doc", + "kwargs_getter_mandatory", + "kwargs_set_default_doc", + "kwargs_set_default_ignore_none", + "kwargs_set_default_list", + "kwargs_set_default_mandatory", + "kwargs_setter", + "kwargs_setter_doc", + "kwargs_setter_mandatory", + "to_label_maybe", +) + +# Various string constants for kwarg key names used across two or more +# functions, or in contexts with optional lookups (e.g. dict.dict, key in dict). +# Constants are used to reduce the chance of typos. +# NOTE: These keys are often part of function signature via `**kwargs`; they +# are not simply internal names. +_ALLOW_FILES = "allow_files" +_ALLOW_EMPTY = "allow_empty" +_ALLOW_SINGLE_FILE = "allow_single_file" +_DEFAULT = "default" +_INPUTS = "inputs" +_OUTPUTS = "outputs" +_CFG = "cfg" +_VALUES = "values" + +def _kwargs_set_default_allow_empty(kwargs): + existing = kwargs.get(_ALLOW_EMPTY) + if existing == None: + kwargs[_ALLOW_EMPTY] = True + +def _kwargs_getter_allow_empty(kwargs): + return kwargs_getter(kwargs, _ALLOW_EMPTY) + +def _kwargs_setter_allow_empty(kwargs): + return kwargs_setter(kwargs, _ALLOW_EMPTY) + +def _kwargs_set_default_allow_files(kwargs): + existing = kwargs.get(_ALLOW_FILES) + if existing == None: + kwargs[_ALLOW_FILES] = False + +def _kwargs_getter_allow_files(kwargs): + return kwargs_getter(kwargs, _ALLOW_FILES) + +def _kwargs_setter_allow_files(kwargs): + return kwargs_setter(kwargs, _ALLOW_FILES) + +def _kwargs_set_default_aspects(kwargs): + kwargs_set_default_list(kwargs, "aspects") + +def _kwargs_getter_aspects(kwargs): + return kwargs_getter(kwargs, "aspects") + +def _kwargs_getter_providers(kwargs): + return kwargs_getter(kwargs, "providers") + +def _kwargs_set_default_providers(kwargs): + kwargs_set_default_list(kwargs, "providers") + +def _common_label_build(self, attr_factory): + kwargs = dict(self.kwargs) + kwargs[_CFG] = self.cfg.build() + return attr_factory(**kwargs) + +def _WhichCfg_typedef(): + """Values returned by `AttrCfg.which_cfg` + + :::{field} TARGET + + Indicates the target config is set. + ::: + + :::{field} EXEC + + Indicates the exec config is set. + ::: + :::{field} NONE + + Indicates the "none" config is set (see {obj}`config.none`). + ::: + :::{field} IMPL + + Indicates a custom transition is set. + ::: + """ + +# buildifier: disable=name-conventions +_WhichCfg = struct( + TYPEDEF = _WhichCfg_typedef, + TARGET = "target", + EXEC = "exec", + NONE = "none", + IMPL = "impl", +) + +def _AttrCfg_typedef(): + """Builder for `cfg` arg of label attributes. + + :::{function} inputs() -> list[Label] + ::: + + :::{function} outputs() -> list[Label] + ::: + + :::{function} which_cfg() -> attrb.WhichCfg + + Tells which of the cfg modes is set. Will be one of: target, exec, none, + or implementation + ::: + """ + +_ATTR_CFG_WHICH = "which" +_ATTR_CFG_VALUE = "value" + +def _AttrCfg_new( + inputs = None, + outputs = None, + **kwargs): + """Creates a builder for the `attr.cfg` attribute. + + Args: + inputs: {type}`list[Label] | None` inputs to use for a transition + outputs: {type}`list[Label] | None` outputs to use for a transition + **kwargs: {type}`dict` Three different keyword args are supported. + The presence of a keyword arg will mark the respective mode + returned by `which_cfg`. + - `cfg`: string of either "target" or "exec" + - `exec_group`: string of an exec group name to use. None means + to use regular exec config (i.e. `config.exec()`) + - `implementation`: callable for a custom transition function. + + Returns: + {type}`AttrCfg` + """ + state = { + _INPUTS: inputs, + _OUTPUTS: outputs, + # Value depends on _ATTR_CFG_WHICH key. See associated setters. + _ATTR_CFG_VALUE: True, + # str: one of the _WhichCfg values + _ATTR_CFG_WHICH: _WhichCfg.TARGET, + } + kwargs_set_default_list(state, _INPUTS) + kwargs_set_default_list(state, _OUTPUTS) + + # buildifier: disable=uninitialized + self = struct( + # keep sorted + _state = state, + build = lambda: _AttrCfg_build(self), + exec_group = lambda: _AttrCfg_exec_group(self), + implementation = lambda: _AttrCfg_implementation(self), + inputs = kwargs_getter(state, _INPUTS), + none = lambda: _AttrCfg_none(self), + outputs = kwargs_getter(state, _OUTPUTS), + set_exec = lambda *a, **k: _AttrCfg_set_exec(self, *a, **k), + set_implementation = lambda *a, **k: _AttrCfg_set_implementation(self, *a, **k), + set_none = lambda: _AttrCfg_set_none(self), + set_target = lambda: _AttrCfg_set_target(self), + target = lambda: _AttrCfg_target(self), + which_cfg = kwargs_getter(state, _ATTR_CFG_WHICH), + ) + + # Only one of the three kwargs should be present. We just process anything + # we see because it's simpler. + if _CFG in kwargs: + cfg = kwargs.pop(_CFG) + if cfg == "target" or cfg == None: + self.set_target() + elif cfg == "exec": + self.set_exec() + elif cfg == "none": + self.set_none() + else: + self.set_implementation(cfg) + if "exec_group" in kwargs: + self.set_exec(kwargs.pop("exec_group")) + + if "implementation" in kwargs: + self.set_implementation(kwargs.pop("implementation")) + + return self + +def _AttrCfg_from_attr_kwargs_pop(attr_kwargs): + """Creates a `AttrCfg` from the cfg arg passed to an attribute bulider. + + Args: + attr_kwargs: dict of attr kwargs, it's "cfg" key will be removed. + + Returns: + {type}`AttrCfg` + """ + cfg = attr_kwargs.pop(_CFG, None) + if not types.is_dict(cfg): + kwargs = {_CFG: cfg} + else: + kwargs = cfg + return _AttrCfg_new(**kwargs) + +def _AttrCfg_implementation(self): + """Tells the custom transition function, if any and applicable. + + Returns: + {type}`callable | None` the custom transition function to use, if + any, or `None` if a different config mode is being used. + """ + return self._state[_ATTR_CFG_VALUE] if self._state[_ATTR_CFG_WHICH] == _WhichCfg.IMPL else None + +def _AttrCfg_none(self): + """Tells if none cfg (`config.none()`) is set. + + Returns: + {type}`bool` True if none cfg is set, False if not. + """ + return self._state[_ATTR_CFG_VALUE] if self._state[_ATTR_CFG_WHICH] == _WhichCfg.NONE else False + +def _AttrCfg_target(self): + """Tells if target cfg is set. + + Returns: + {type}`bool` True if target cfg is set, False if not. + """ + return self._state[_ATTR_CFG_VALUE] if self._state[_ATTR_CFG_WHICH] == _WhichCfg.TARGET else False + +def _AttrCfg_exec_group(self): + """Tells the exec group to use if an exec transition is being used. + + Args: + self: implicitly added. + + Returns: + {type}`str | None` the name of the exec group to use if any, + or `None` if `which_cfg` isn't `exec` + """ + return self._state[_ATTR_CFG_VALUE] if self._state[_ATTR_CFG_WHICH] == _WhichCfg.EXEC else None + +def _AttrCfg_set_implementation(self, impl): + """Sets a custom transition function to use. + + Args: + self: implicitly added. + impl: {type}`callable` a transition implementation function. + """ + self._state[_ATTR_CFG_WHICH] = _WhichCfg.IMPL + self._state[_ATTR_CFG_VALUE] = impl + +def _AttrCfg_set_none(self): + """Sets to use the "none" transition.""" + self._state[_ATTR_CFG_WHICH] = _WhichCfg.NONE + self._state[_ATTR_CFG_VALUE] = True + +def _AttrCfg_set_exec(self, exec_group = None): + """Sets to use an exec transition. + + Args: + self: implicitly added. + exec_group: {type}`str | None` the exec group name to use, if any. + """ + self._state[_ATTR_CFG_WHICH] = _WhichCfg.EXEC + self._state[_ATTR_CFG_VALUE] = exec_group + +def _AttrCfg_set_target(self): + """Sets to use the target transition.""" + self._state[_ATTR_CFG_WHICH] = _WhichCfg.TARGET + self._state[_ATTR_CFG_VALUE] = True + +def _AttrCfg_build(self): + which = self._state[_ATTR_CFG_WHICH] + value = self._state[_ATTR_CFG_VALUE] + if which == None: + return None + elif which == _WhichCfg.TARGET: + # config.target is Bazel 8+ + if hasattr(config, "target"): + return config.target() + else: + return "target" + elif which == _WhichCfg.EXEC: + return config.exec(value) + elif which == _WhichCfg.NONE: + return config.none() + elif types.is_function(value): + return transition( + implementation = value, + # Transitions only accept unique lists of strings. + inputs = {str(v): None for v in self._state[_INPUTS]}.keys(), + outputs = {str(v): None for v in self._state[_OUTPUTS]}.keys(), + ) + else: + # Otherwise, just assume the value is valid and whoever set it knows + # what they're doing. + return value + +# buildifier: disable=name-conventions +AttrCfg = struct( + TYPEDEF = _AttrCfg_typedef, + new = _AttrCfg_new, + # keep sorted + exec_group = _AttrCfg_exec_group, + implementation = _AttrCfg_implementation, + none = _AttrCfg_none, + set_exec = _AttrCfg_set_exec, + set_implementation = _AttrCfg_set_implementation, + set_none = _AttrCfg_set_none, + set_target = _AttrCfg_set_target, + target = _AttrCfg_target, +) + +def _Bool_typedef(): + """Builder for attr.bool. + + :::{function} build() -> attr.bool + ::: + + :::{function} default() -> bool. + ::: + + :::{function} doc() -> str + ::: + + :::{include} /_includes/field_kwargs_doc.md + ::: + + :::{function} mandatory() -> bool + ::: + + :::{function} set_default(v: bool) + ::: + + :::{function} set_doc(v: str) + ::: + + :::{function} set_mandatory(v: bool) + ::: + + """ + +def _Bool_new(**kwargs): + """Creates a builder for `attr.bool`. + + Args: + **kwargs: Same kwargs as {obj}`attr.bool` + + Returns: + {type}`Bool` + """ + kwargs_set_default_ignore_none(kwargs, _DEFAULT, False) + kwargs_set_default_doc(kwargs) + kwargs_set_default_mandatory(kwargs) + + # buildifier: disable=uninitialized + self = struct( + # keep sorted + build = lambda: attr.bool(**self.kwargs), + default = kwargs_getter(kwargs, _DEFAULT), + doc = kwargs_getter_doc(kwargs), + kwargs = kwargs, + mandatory = kwargs_getter_mandatory(kwargs), + set_default = kwargs_setter(kwargs, _DEFAULT), + set_doc = kwargs_setter_doc(kwargs), + set_mandatory = kwargs_setter_mandatory(kwargs), + ) + return self + +# buildifier: disable=name-conventions +Bool = struct( + TYPEDEF = _Bool_typedef, + new = _Bool_new, +) + +def _Int_typedef(): + """Builder for attr.int. + + :::{function} build() -> attr.int + ::: + + :::{function} default() -> int + ::: + + :::{function} doc() -> str + ::: + + :::{include} /_includes/field_kwargs_doc.md + ::: + + :::{function} mandatory() -> bool + ::: + + :::{function} values() -> list[int] + + The returned value is a mutable reference to the underlying list. + ::: + + :::{function} set_default(v: int) + ::: + + :::{function} set_doc(v: str) + ::: + + :::{function} set_mandatory(v: bool) + ::: + """ + +def _Int_new(**kwargs): + """Creates a builder for `attr.int`. + + Args: + **kwargs: Same kwargs as {obj}`attr.int` + + Returns: + {type}`Int` + """ + kwargs_set_default_ignore_none(kwargs, _DEFAULT, 0) + kwargs_set_default_doc(kwargs) + kwargs_set_default_mandatory(kwargs) + kwargs_set_default_list(kwargs, _VALUES) + + # buildifier: disable=uninitialized + self = struct( + build = lambda: attr.int(**self.kwargs), + default = kwargs_getter(kwargs, _DEFAULT), + doc = kwargs_getter_doc(kwargs), + kwargs = kwargs, + mandatory = kwargs_getter_mandatory(kwargs), + values = kwargs_getter(kwargs, _VALUES), + set_default = kwargs_setter(kwargs, _DEFAULT), + set_doc = kwargs_setter_doc(kwargs), + set_mandatory = kwargs_setter_mandatory(kwargs), + ) + return self + +# buildifier: disable=name-conventions +Int = struct( + TYPEDEF = _Int_typedef, + new = _Int_new, +) + +def _IntList_typedef(): + """Builder for attr.int_list. + + :::{function} allow_empty() -> bool + ::: + + :::{function} build() -> attr.int_list + ::: + + :::{function} default() -> list[int] + ::: + + :::{function} doc() -> str + ::: + + :::{include} /_includes/field_kwargs_doc.md + ::: + + :::{function} mandatory() -> bool + ::: + + :::{function} set_allow_empty(v: bool) + ::: + + :::{function} set_doc(v: str) + ::: + + :::{function} set_mandatory(v: bool) + ::: + """ + +def _IntList_new(**kwargs): + """Creates a builder for `attr.int_list`. + + Args: + **kwargs: Same as {obj}`attr.int_list`. + + Returns: + {type}`IntList` + """ + kwargs_set_default_list(kwargs, _DEFAULT) + kwargs_set_default_doc(kwargs) + kwargs_set_default_mandatory(kwargs) + _kwargs_set_default_allow_empty(kwargs) + + # buildifier: disable=uninitialized + self = struct( + # keep sorted + allow_empty = _kwargs_getter_allow_empty(kwargs), + build = lambda: attr.int_list(**self.kwargs), + default = kwargs_getter(kwargs, _DEFAULT), + doc = kwargs_getter_doc(kwargs), + kwargs = kwargs, + mandatory = kwargs_getter_mandatory(kwargs), + set_allow_empty = _kwargs_setter_allow_empty(kwargs), + set_doc = kwargs_setter_doc(kwargs), + set_mandatory = kwargs_setter_mandatory(kwargs), + ) + return self + +# buildifier: disable=name-conventions +IntList = struct( + TYPEDEF = _IntList_typedef, + new = _IntList_new, +) + +def _Label_typedef(): + """Builder for `attr.label` objects. + + :::{function} allow_files() -> bool | list[str] | None + + Note that `allow_files` is mutually exclusive with `allow_single_file`. + Only one of the two can have a value set. + ::: + + :::{function} allow_single_file() -> bool | None + Note that `allow_single_file` is mutually exclusive with `allow_files`. + Only one of the two can have a value set. + ::: + + :::{function} aspects() -> list[aspect] + + The returned list is a mutable reference to the underlying list. + ::: + + :::{function} build() -> attr.label + ::: + + :::{field} cfg + :type: AttrCfg + ::: + + :::{function} default() -> str | label | configuration_field | None + ::: + + :::{function} doc() -> str + ::: + + :::{function} executable() -> bool + ::: + + :::{include} /_includes/field_kwargs_doc.md + ::: + + :::{function} mandatory() -> bool + ::: + + + :::{function} providers() -> list[list[provider]] + The returned list is a mutable reference to the underlying list. + ::: + + :::{function} set_default(v: str | Label) + ::: + + :::{function} set_doc(v: str) + ::: + + :::{function} set_executable(v: bool) + ::: + + :::{function} set_mandatory(v: bool) + ::: + """ + +def _Label_new(**kwargs): + """Creates a builder for `attr.label`. + + Args: + **kwargs: The same as {obj}`attr.label()`. + + Returns: + {type}`Label` + """ + kwargs_set_default_ignore_none(kwargs, "executable", False) + _kwargs_set_default_aspects(kwargs) + _kwargs_set_default_providers(kwargs) + kwargs_set_default_doc(kwargs) + kwargs_set_default_mandatory(kwargs) + + kwargs[_DEFAULT] = to_label_maybe(kwargs.get(_DEFAULT)) + + # buildifier: disable=uninitialized + self = struct( + # keep sorted + add_allow_files = lambda v: _Label_add_allow_files(self, v), + allow_files = _kwargs_getter_allow_files(kwargs), + allow_single_file = kwargs_getter(kwargs, _ALLOW_SINGLE_FILE), + aspects = _kwargs_getter_aspects(kwargs), + build = lambda: _common_label_build(self, attr.label), + cfg = _AttrCfg_from_attr_kwargs_pop(kwargs), + default = kwargs_getter(kwargs, _DEFAULT), + doc = kwargs_getter_doc(kwargs), + executable = kwargs_getter(kwargs, "executable"), + kwargs = kwargs, + mandatory = kwargs_getter_mandatory(kwargs), + providers = _kwargs_getter_providers(kwargs), + set_allow_files = lambda v: _Label_set_allow_files(self, v), + set_allow_single_file = lambda v: _Label_set_allow_single_file(self, v), + set_default = kwargs_setter(kwargs, _DEFAULT), + set_doc = kwargs_setter_doc(kwargs), + set_executable = kwargs_setter(kwargs, "executable"), + set_mandatory = kwargs_setter_mandatory(kwargs), + ) + return self + +def _Label_set_allow_files(self, v): + """Set the allow_files arg + + NOTE: Setting `allow_files` unsets `allow_single_file` + + Args: + self: implicitly added. + v: {type}`bool | list[str] | None` the value to set to. + If set to `None`, then `allow_files` is unset. + """ + if v == None: + self.kwargs.pop(_ALLOW_FILES, None) + else: + self.kwargs[_ALLOW_FILES] = v + self.kwargs.pop(_ALLOW_SINGLE_FILE, None) + +def _Label_add_allow_files(self, *values): + """Adds allowed file extensions + + NOTE: Add an allowed file extension unsets `allow_single_file` + + Args: + self: implicitly added. + *values: {type}`str` file extensions to allow (including dot) + """ + self.kwargs.pop(_ALLOW_SINGLE_FILE, None) + if not types.is_list(self.kwargs.get(_ALLOW_FILES)): + self.kwargs[_ALLOW_FILES] = [] + existing = self.kwargs[_ALLOW_FILES] + existing.extend([v for v in values if v not in existing]) + +def _Label_set_allow_single_file(self, v): + """Sets the allow_single_file arg. + + NOTE: Setting `allow_single_file` unsets `allow_file` + + Args: + self: implicitly added. + v: {type}`bool | None` the value to set to. + If set to `None`, then `allow_single_file` is unset. + """ + if v == None: + self.kwargs.pop(_ALLOW_SINGLE_FILE, None) + else: + self.kwargs[_ALLOW_SINGLE_FILE] = v + self.kwargs.pop(_ALLOW_FILES, None) + +# buildifier: disable=name-conventions +Label = struct( + TYPEDEF = _Label_typedef, + new = _Label_new, + set_allow_files = _Label_set_allow_files, + add_allow_files = _Label_add_allow_files, + set_allow_single_file = _Label_set_allow_single_file, +) + +def _LabelKeyedStringDict_typedef(): + """Builder for attr.label_keyed_string_dict. + + :::{function} aspects() -> list[aspect] + The returned list is a mutable reference to the underlying list. + ::: + + :::{function} allow_files() -> bool | list[str] + ::: + + :::{function} allow_empty() -> bool + ::: + + :::{field} cfg + :type: AttrCfg + ::: + + :::{function} default() -> dict[str | Label, str] | callable + ::: + + :::{function} doc() -> str + ::: + + :::{include} /_includes/field_kwargs_doc.md + ::: + + :::{function} mandatory() -> bool + ::: + + :::{function} providers() -> list[provider | list[provider]] + + Returns a mutable reference to the underlying list. + ::: + + :::{function} set_mandatory(v: bool) + ::: + :::{function} set_allow_empty(v: bool) + ::: + :::{function} set_default(v: dict[str | Label, str] | callable) + ::: + :::{function} set_doc(v: str) + ::: + :::{function} set_allow_files(v: bool | list[str]) + ::: + """ + +def _LabelKeyedStringDict_new(**kwargs): + """Creates a builder for `attr.label_keyed_string_dict`. + + Args: + **kwargs: Same as {obj}`attr.label_keyed_string_dict`. + + Returns: + {type}`LabelKeyedStringDict` + """ + kwargs_set_default_ignore_none(kwargs, _DEFAULT, {}) + _kwargs_set_default_aspects(kwargs) + _kwargs_set_default_providers(kwargs) + _kwargs_set_default_allow_empty(kwargs) + _kwargs_set_default_allow_files(kwargs) + kwargs_set_default_doc(kwargs) + kwargs_set_default_mandatory(kwargs) + + # buildifier: disable=uninitialized + self = struct( + # keep sorted + add_allow_files = lambda *v: _LabelKeyedStringDict_add_allow_files(self, *v), + allow_empty = _kwargs_getter_allow_empty(kwargs), + allow_files = _kwargs_getter_allow_files(kwargs), + aspects = _kwargs_getter_aspects(kwargs), + build = lambda: _common_label_build(self, attr.label_keyed_string_dict), + cfg = _AttrCfg_from_attr_kwargs_pop(kwargs), + default = kwargs_getter(kwargs, _DEFAULT), + doc = kwargs_getter_doc(kwargs), + kwargs = kwargs, + mandatory = kwargs_getter_mandatory(kwargs), + providers = _kwargs_getter_providers(kwargs), + set_allow_empty = _kwargs_setter_allow_empty(kwargs), + set_allow_files = _kwargs_setter_allow_files(kwargs), + set_default = kwargs_setter(kwargs, _DEFAULT), + set_doc = kwargs_setter_doc(kwargs), + set_mandatory = kwargs_setter_mandatory(kwargs), + ) + return self + +def _LabelKeyedStringDict_add_allow_files(self, *values): + """Adds allowed file extensions + + Args: + self: implicitly added. + *values: {type}`str` file extensions to allow (including dot) + """ + if not types.is_list(self.kwargs.get(_ALLOW_FILES)): + self.kwargs[_ALLOW_FILES] = [] + existing = self.kwargs[_ALLOW_FILES] + existing.extend([v for v in values if v not in existing]) + +# buildifier: disable=name-conventions +LabelKeyedStringDict = struct( + TYPEDEF = _LabelKeyedStringDict_typedef, + new = _LabelKeyedStringDict_new, + add_allow_files = _LabelKeyedStringDict_add_allow_files, +) + +def _LabelList_typedef(): + """Builder for `attr.label_list` + + :::{function} aspects() -> list[aspect] + ::: + + :::{function} allow_files() -> bool | list[str] + ::: + + :::{function} allow_empty() -> bool + ::: + + :::{function} build() -> attr.label_list + ::: + + :::{field} cfg + :type: AttrCfg + ::: + + :::{function} default() -> list[str|Label] | configuration_field | callable + ::: + + :::{function} doc() -> str + ::: + + :::{include} /_includes/field_kwargs_doc.md + ::: + + :::{function} mandatory() -> bool + ::: + + :::{function} providers() -> list[provider | list[provider]] + ::: + + :::{function} set_allow_empty(v: bool) + ::: + + :::{function} set_allow_files(v: bool | list[str]) + ::: + + :::{function} set_default(v: list[str|Label] | configuration_field | callable) + ::: + + :::{function} set_doc(v: str) + ::: + + :::{function} set_mandatory(v: bool) + ::: + """ + +def _LabelList_new(**kwargs): + """Creates a builder for `attr.label_list`. + + Args: + **kwargs: Same as {obj}`attr.label_list`. + + Returns: + {type}`LabelList` + """ + _kwargs_set_default_allow_empty(kwargs) + kwargs_set_default_mandatory(kwargs) + kwargs_set_default_doc(kwargs) + if kwargs.get(_ALLOW_FILES) == None: + kwargs[_ALLOW_FILES] = False + _kwargs_set_default_aspects(kwargs) + kwargs_set_default_list(kwargs, _DEFAULT) + _kwargs_set_default_providers(kwargs) + + # buildifier: disable=uninitialized + self = struct( + # keep sorted + allow_empty = _kwargs_getter_allow_empty(kwargs), + allow_files = _kwargs_getter_allow_files(kwargs), + aspects = _kwargs_getter_aspects(kwargs), + build = lambda: _common_label_build(self, attr.label_list), + cfg = _AttrCfg_from_attr_kwargs_pop(kwargs), + default = kwargs_getter(kwargs, _DEFAULT), + doc = kwargs_getter_doc(kwargs), + kwargs = kwargs, + mandatory = kwargs_getter_mandatory(kwargs), + providers = _kwargs_getter_providers(kwargs), + set_allow_empty = _kwargs_setter_allow_empty(kwargs), + set_allow_files = _kwargs_setter_allow_files(kwargs), + set_default = kwargs_setter(kwargs, _DEFAULT), + set_doc = kwargs_setter_doc(kwargs), + set_mandatory = kwargs_setter_mandatory(kwargs), + ) + return self + +# buildifier: disable=name-conventions +LabelList = struct( + TYPEDEF = _LabelList_typedef, + new = _LabelList_new, +) + +def _Output_typedef(): + """Builder for attr.output + + :::{function} build() -> attr.output + ::: + + :::{function} doc() -> str + ::: + + :::{include} /_includes/field_kwargs_doc.md + ::: + + :::{function} mandatory() -> bool + ::: + + :::{function} set_doc(v: str) + ::: + + :::{function} set_mandatory(v: bool) + ::: + """ + +def _Output_new(**kwargs): + """Creates a builder for `attr.output`. + + Args: + **kwargs: Same as {obj}`attr.output`. + + Returns: + {type}`Output` + """ + kwargs_set_default_doc(kwargs) + kwargs_set_default_mandatory(kwargs) + + # buildifier: disable=uninitialized + self = struct( + # keep sorted + build = lambda: attr.output(**self.kwargs), + doc = kwargs_getter_doc(kwargs), + kwargs = kwargs, + mandatory = kwargs_getter_mandatory(kwargs), + set_doc = kwargs_setter_doc(kwargs), + set_mandatory = kwargs_setter_mandatory(kwargs), + ) + return self + +# buildifier: disable=name-conventions +Output = struct( + TYPEDEF = _Output_typedef, + new = _Output_new, +) + +def _OutputList_typedef(): + """Builder for attr.output_list + + :::{function} allow_empty() -> bool + ::: + + :::{function} build() -> attr.output + ::: + + :::{function} doc() -> str + ::: + + :::{include} /_includes/field_kwargs_doc.md + ::: + + :::{function} mandatory() -> bool + ::: + + :::{function} set_allow_empty(v: bool) + ::: + :::{function} set_doc(v: str) + ::: + :::{function} set_mandatory(v: bool) + ::: + """ + +def _OutputList_new(**kwargs): + """Creates a builder for `attr.output_list`. + + Args: + **kwargs: Same as {obj}`attr.output_list`. + + Returns: + {type}`OutputList` + """ + kwargs_set_default_doc(kwargs) + kwargs_set_default_mandatory(kwargs) + _kwargs_set_default_allow_empty(kwargs) + + # buildifier: disable=uninitialized + self = struct( + allow_empty = _kwargs_getter_allow_empty(kwargs), + build = lambda: attr.output_list(**self.kwargs), + doc = kwargs_getter_doc(kwargs), + kwargs = kwargs, + mandatory = kwargs_getter_mandatory(kwargs), + set_allow_empty = _kwargs_setter_allow_empty(kwargs), + set_doc = kwargs_setter_doc(kwargs), + set_mandatory = kwargs_setter_mandatory(kwargs), + ) + return self + +# buildifier: disable=name-conventions +OutputList = struct( + TYPEDEF = _OutputList_typedef, + new = _OutputList_new, +) + +def _String_typedef(): + """Builder for `attr.string` + + :::{function} build() -> attr.string + ::: + + :::{function} default() -> str | configuration_field + ::: + + :::{function} doc() -> str + ::: + + :::{include} /_includes/field_kwargs_doc.md + ::: + + :::{function} mandatory() -> bool + ::: + + :::{function} values() -> list[str] + ::: + + :::{function} set_default(v: str | configuration_field) + ::: + + :::{function} set_doc(v: str) + ::: + + :::{function} set_mandatory(v: bool) + ::: + """ + +def _String_new(**kwargs): + """Creates a builder for `attr.string`. + + Args: + **kwargs: Same as {obj}`attr.string`. + + Returns: + {type}`String` + """ + kwargs_set_default_ignore_none(kwargs, _DEFAULT, "") + kwargs_set_default_list(kwargs, _VALUES) + kwargs_set_default_doc(kwargs) + kwargs_set_default_mandatory(kwargs) + + # buildifier: disable=uninitialized + self = struct( + default = kwargs_getter(kwargs, _DEFAULT), + doc = kwargs_getter_doc(kwargs), + mandatory = kwargs_getter_mandatory(kwargs), + build = lambda: attr.string(**self.kwargs), + kwargs = kwargs, + values = kwargs_getter(kwargs, _VALUES), + set_default = kwargs_setter(kwargs, _DEFAULT), + set_doc = kwargs_setter_doc(kwargs), + set_mandatory = kwargs_setter_mandatory(kwargs), + ) + return self + +# buildifier: disable=name-conventions +String = struct( + TYPEDEF = _String_typedef, + new = _String_new, +) + +def _StringDict_typedef(): + """Builder for `attr.string_dict` + + :::{function} default() -> dict[str, str] + ::: + + :::{function} doc() -> str + ::: + + :::{function} mandatory() -> bool + ::: + + :::{function} allow_empty() -> bool + ::: + + :::{function} build() -> attr.string_dict + ::: + + :::{include} /_includes/field_kwargs_doc.md + ::: + + :::{function} set_doc(v: str) + ::: + :::{function} set_mandatory(v: bool) + ::: + :::{function} set_allow_empty(v: bool) + ::: + """ + +def _StringDict_new(**kwargs): + """Creates a builder for `attr.string_dict`. + + Args: + **kwargs: The same args as for `attr.string_dict`. + + Returns: + {type}`StringDict` + """ + kwargs_set_default_ignore_none(kwargs, _DEFAULT, {}) + kwargs_set_default_doc(kwargs) + kwargs_set_default_mandatory(kwargs) + _kwargs_set_default_allow_empty(kwargs) + + # buildifier: disable=uninitialized + self = struct( + allow_empty = _kwargs_getter_allow_empty(kwargs), + build = lambda: attr.string_dict(**self.kwargs), + default = kwargs_getter(kwargs, _DEFAULT), + doc = kwargs_getter_doc(kwargs), + kwargs = kwargs, + mandatory = kwargs_getter_mandatory(kwargs), + set_allow_empty = _kwargs_setter_allow_empty(kwargs), + set_doc = kwargs_setter_doc(kwargs), + set_mandatory = kwargs_setter_mandatory(kwargs), + ) + return self + +# buildifier: disable=name-conventions +StringDict = struct( + TYPEDEF = _StringDict_typedef, + new = _StringDict_new, +) + +def _StringKeyedLabelDict_typedef(): + """Builder for attr.string_keyed_label_dict. + + :::{function} allow_empty() -> bool + ::: + + :::{function} allow_files() -> bool | list[str] + ::: + + :::{function} aspects() -> list[aspect] + ::: + + :::{function} build() -> attr.string_list + ::: + + :::{field} cfg + :type: AttrCfg + ::: + + :::{function} default() -> dict[str, Label] | callable + ::: + + :::{function} doc() -> str + ::: + + :::{function} mandatory() -> bool + ::: + + :::{function} providers() -> list[list[provider]] + ::: + + :::{include} /_includes/field_kwargs_doc.md + ::: + + :::{function} set_allow_empty(v: bool) + ::: + + :::{function} set_allow_files(v: bool | list[str]) + ::: + + :::{function} set_doc(v: str) + ::: + + :::{function} set_default(v: dict[str, Label] | callable) + ::: + + :::{function} set_mandatory(v: bool) + ::: + """ + +def _StringKeyedLabelDict_new(**kwargs): + """Creates a builder for `attr.string_keyed_label_dict`. + + Args: + **kwargs: Same as {obj}`attr.string_keyed_label_dict`. + + Returns: + {type}`StringKeyedLabelDict` + """ + kwargs_set_default_ignore_none(kwargs, _DEFAULT, {}) + kwargs_set_default_doc(kwargs) + kwargs_set_default_mandatory(kwargs) + _kwargs_set_default_allow_files(kwargs) + _kwargs_set_default_allow_empty(kwargs) + _kwargs_set_default_aspects(kwargs) + _kwargs_set_default_providers(kwargs) + + # buildifier: disable=uninitialized + self = struct( + allow_empty = _kwargs_getter_allow_empty(kwargs), + allow_files = _kwargs_getter_allow_files(kwargs), + build = lambda: _common_label_build(self, attr.string_keyed_label_dict), + cfg = _AttrCfg_from_attr_kwargs_pop(kwargs), + default = kwargs_getter(kwargs, _DEFAULT), + doc = kwargs_getter_doc(kwargs), + kwargs = kwargs, + mandatory = kwargs_getter_mandatory(kwargs), + set_allow_empty = _kwargs_setter_allow_empty(kwargs), + set_allow_files = _kwargs_setter_allow_files(kwargs), + set_default = kwargs_setter(kwargs, _DEFAULT), + set_doc = kwargs_setter_doc(kwargs), + set_mandatory = kwargs_setter_mandatory(kwargs), + providers = _kwargs_getter_providers(kwargs), + aspects = _kwargs_getter_aspects(kwargs), + ) + return self + +# buildifier: disable=name-conventions +StringKeyedLabelDict = struct( + TYPEDEF = _StringKeyedLabelDict_typedef, + new = _StringKeyedLabelDict_new, +) + +def _StringList_typedef(): + """Builder for `attr.string_list` + + :::{function} allow_empty() -> bool + ::: + + :::{function} build() -> attr.string_list + ::: + + :::{field} default + :type: Value[list[str] | configuration_field] + ::: + + :::{function} doc() -> str + ::: + + :::{function} mandatory() -> bool + ::: + + :::{include} /_includes/field_kwargs_doc.md + ::: + + :::{function} set_allow_empty(v: bool) + ::: + + :::{function} set_doc(v: str) + ::: + + :::{function} set_mandatory(v: bool) + ::: + """ + +def _StringList_new(**kwargs): + """Creates a builder for `attr.string_list`. + + Args: + **kwargs: Same as {obj}`attr.string_list`. + + Returns: + {type}`StringList` + """ + kwargs_set_default_ignore_none(kwargs, _DEFAULT, []) + kwargs_set_default_doc(kwargs) + kwargs_set_default_mandatory(kwargs) + _kwargs_set_default_allow_empty(kwargs) + + # buildifier: disable=uninitialized + self = struct( + allow_empty = _kwargs_getter_allow_empty(kwargs), + build = lambda: attr.string_list(**self.kwargs), + default = kwargs_getter(kwargs, _DEFAULT), + doc = kwargs_getter_doc(kwargs), + kwargs = kwargs, + mandatory = kwargs_getter_mandatory(kwargs), + set_allow_empty = _kwargs_setter_allow_empty(kwargs), + set_default = kwargs_setter(kwargs, _DEFAULT), + set_doc = kwargs_setter_doc(kwargs), + set_mandatory = kwargs_setter_mandatory(kwargs), + ) + return self + +# buildifier: disable=name-conventions +StringList = struct( + TYPEDEF = _StringList_typedef, + new = _StringList_new, +) + +def _StringListDict_typedef(): + """Builder for attr.string_list_dict. + + :::{function} allow_empty() -> bool + ::: + + :::{function} build() -> attr.string_list + ::: + + :::{function} default() -> dict[str, list[str]] + ::: + + :::{function} doc() -> str + ::: + + :::{function} mandatory() -> bool + ::: + + :::{include} /_includes/field_kwargs_doc.md + ::: + + :::{function} set_allow_empty(v: bool) + ::: + + :::{function} set_doc(v: str) + ::: + + :::{function} set_mandatory(v: bool) + ::: + """ + +def _StringListDict_new(**kwargs): + """Creates a builder for `attr.string_list_dict`. + + Args: + **kwargs: Same as {obj}`attr.string_list_dict`. + + Returns: + {type}`StringListDict` + """ + kwargs_set_default_ignore_none(kwargs, _DEFAULT, {}) + kwargs_set_default_doc(kwargs) + kwargs_set_default_mandatory(kwargs) + _kwargs_set_default_allow_empty(kwargs) + + # buildifier: disable=uninitialized + self = struct( + allow_empty = _kwargs_getter_allow_empty(kwargs), + build = lambda: attr.string_list_dict(**self.kwargs), + default = kwargs_getter(kwargs, _DEFAULT), + doc = kwargs_getter_doc(kwargs), + kwargs = kwargs, + mandatory = kwargs_getter_mandatory(kwargs), + set_allow_empty = _kwargs_setter_allow_empty(kwargs), + set_default = kwargs_setter(kwargs, _DEFAULT), + set_doc = kwargs_setter_doc(kwargs), + set_mandatory = kwargs_setter_mandatory(kwargs), + ) + return self + +# buildifier: disable=name-conventions +StringListDict = struct( + TYPEDEF = _StringListDict_typedef, + new = _StringListDict_new, +) + +attrb = struct( + # keep sorted + Bool = _Bool_new, + Int = _Int_new, + IntList = _IntList_new, + Label = _Label_new, + LabelKeyedStringDict = _LabelKeyedStringDict_new, + LabelList = _LabelList_new, + Output = _Output_new, + OutputList = _OutputList_new, + String = _String_new, + StringDict = _StringDict_new, + StringKeyedLabelDict = _StringKeyedLabelDict_new, + StringList = _StringList_new, + StringListDict = _StringListDict_new, + WhichCfg = _WhichCfg, +) diff --git a/python/private/attributes.bzl b/python/private/attributes.bzl index e167482eb1..b57e275406 100644 --- a/python/private/attributes.bzl +++ b/python/private/attributes.bzl @@ -13,14 +13,16 @@ # limitations under the License. """Attributes for Python rules.""" +load("@bazel_skylib//lib:dicts.bzl", "dicts") load("@bazel_skylib//rules:common_settings.bzl", "BuildSettingInfo") load("@rules_cc//cc/common:cc_info.bzl", "CcInfo") -load(":common.bzl", "union_attrs") +load(":attr_builders.bzl", "attrb") load(":enum.bzl", "enum") load(":flags.bzl", "PrecompileFlag", "PrecompileSourceRetentionFlag") load(":py_info.bzl", "PyInfo") load(":py_internal.bzl", "py_internal") load(":reexports.bzl", "BuiltinPyInfo") +load(":rule_builders.bzl", "ruleb") load( ":semantics.bzl", "DEPS_ATTR_ALLOW_RULES", @@ -41,12 +43,18 @@ _PackageSpecificationInfo = getattr(py_internal, "PackageSpecificationInfo", Non # NOTE: These are no-op/empty exec groups. If a rule *does* support an exec # group and needs custom settings, it should merge this dict with one that # overrides the supported key. -REQUIRED_EXEC_GROUPS = { +REQUIRED_EXEC_GROUP_BUILDERS = { # py_binary may invoke C++ linking, or py rules may be used in combination # with cc rules (e.g. within the same macro), so support that exec group. # This exec group is defined by rules_cc for the cc rules. - "cpp_link": exec_group(), - "py_precompile": exec_group(), + "cpp_link": lambda: ruleb.ExecGroup(), + "py_precompile": lambda: ruleb.ExecGroup(), +} + +# Backwards compatibility symbol for Google. +REQUIRED_EXEC_GROUPS = { + k: v().build() + for k, v in REQUIRED_EXEC_GROUP_BUILDERS.items() } _STAMP_VALUES = [-1, 0, 1] @@ -139,59 +147,6 @@ PycCollectionAttr = enum( is_pyc_collection_enabled = _pyc_collection_attr_is_pyc_collection_enabled, ) -def create_stamp_attr(**kwargs): - return { - "stamp": attr.int( - values = _STAMP_VALUES, - doc = """ -Whether to encode build information into the binary. Possible values: - -* `stamp = 1`: Always stamp the build information into the binary, even in - `--nostamp` builds. **This setting should be avoided**, since it potentially kills - remote caching for the binary and any downstream actions that depend on it. -* `stamp = 0`: Always replace build information by constant values. This gives - good build result caching. -* `stamp = -1`: Embedding of build information is controlled by the - `--[no]stamp` flag. - -Stamped binaries are not rebuilt unless their dependencies change. - -WARNING: Stamping can harm build performance by reducing cache hits and should -be avoided if possible. -""", - **kwargs - ), - } - -def create_srcs_attr(*, mandatory): - return { - "srcs": attr.label_list( - # Google builds change the set of allowed files. - allow_files = SRCS_ATTR_ALLOW_FILES, - mandatory = mandatory, - # Necessary for --compile_one_dependency to work. - flags = ["DIRECT_COMPILE_TIME_INPUT"], - doc = """ -The list of Python source files that are processed to create the target. This -includes all your checked-in code and may include generated source files. The -`.py` files belong in `srcs` and library targets belong in `deps`. Other binary -files that may be needed at run time belong in `data`. -""", - ), - } - -SRCS_VERSION_ALL_VALUES = ["PY2", "PY2ONLY", "PY2AND3", "PY3", "PY3ONLY"] -SRCS_VERSION_NON_CONVERSION_VALUES = ["PY2AND3", "PY2ONLY", "PY3ONLY"] - -def create_srcs_version_attr(values): - return { - "srcs_version": attr.string( - default = "PY2AND3", - values = values, - doc = "Defunct, unused, does nothing.", - ), - } - def copy_common_binary_kwargs(kwargs): return { key: kwargs[key] @@ -216,7 +171,7 @@ CC_TOOLCHAIN = { DATA_ATTRS = { # NOTE: The "flags" attribute is deprecated, but there isn't an alternative # way to specify that constraints should be ignored. - "data": attr.label_list( + "data": lambda: attrb.LabelList( allow_files = True, flags = ["SKIP_CONSTRAINTS_OVERRIDE"], doc = """ @@ -244,7 +199,7 @@ def _create_native_rules_allowlist_attrs(): providers = [] return { - "_native_rules_allowlist": attr.label( + "_native_rules_allowlist": lambda: attrb.Label( default = default, providers = providers, ), @@ -253,7 +208,7 @@ def _create_native_rules_allowlist_attrs(): NATIVE_RULES_ALLOWLIST_ATTRS = _create_native_rules_allowlist_attrs() # Attributes common to all rules. -COMMON_ATTRS = union_attrs( +COMMON_ATTRS = dicts.add( DATA_ATTRS, NATIVE_RULES_ALLOWLIST_ATTRS, # buildifier: disable=attr-licenses @@ -267,11 +222,10 @@ COMMON_ATTRS = union_attrs( # buildifier: disable=attr-license "licenses": attr.license() if hasattr(attr, "license") else attr.string_list(), }, - allow_none = True, ) IMPORTS_ATTRS = { - "imports": attr.string_list( + "imports": lambda: attrb.StringList( doc = """ List of import directories to be added to the PYTHONPATH. @@ -289,9 +243,9 @@ above the execution root are not allowed and will result in an error. _MaybeBuiltinPyInfo = [[BuiltinPyInfo]] if BuiltinPyInfo != None else [] # Attributes common to rules accepting Python sources and deps. -PY_SRCS_ATTRS = union_attrs( +PY_SRCS_ATTRS = dicts.add( { - "deps": attr.label_list( + "deps": lambda: attrb.LabelList( providers = [ [PyInfo], [CcInfo], @@ -310,7 +264,7 @@ Targets that only provide data files used at runtime belong in the `data` attribute. """, ), - "precompile": attr.string( + "precompile": lambda: attrb.String( doc = """ Whether py source files **for this target** should be precompiled. @@ -332,7 +286,7 @@ Values: default = PrecompileAttr.INHERIT, values = sorted(PrecompileAttr.__members__.values()), ), - "precompile_invalidation_mode": attr.string( + "precompile_invalidation_mode": lambda: attrb.String( doc = """ How precompiled files should be verified to be up-to-date with their associated source files. Possible values are: @@ -350,7 +304,7 @@ https://docs.python.org/3/library/py_compile.html#py_compile.PycInvalidationMode default = PrecompileInvalidationModeAttr.AUTO, values = sorted(PrecompileInvalidationModeAttr.__members__.values()), ), - "precompile_optimize_level": attr.int( + "precompile_optimize_level": lambda: attrb.Int( doc = """ The optimization level for precompiled files. @@ -363,7 +317,7 @@ runtime when the code actually runs. """, default = 0, ), - "precompile_source_retention": attr.string( + "precompile_source_retention": lambda: attrb.String( default = PrecompileSourceRetentionAttr.INHERIT, values = sorted(PrecompileSourceRetentionAttr.__members__.values()), doc = """ @@ -375,7 +329,7 @@ in the resulting output or not. Valid values are: * `omit_source`: Don't include the original py source. """, ), - "pyi_deps": attr.label_list( + "pyi_deps": lambda: attrb.LabelList( doc = """ Dependencies providing type definitions the library needs. @@ -391,7 +345,7 @@ program (packaging rules may include them, however). [CcInfo], ] + _MaybeBuiltinPyInfo, ), - "pyi_srcs": attr.label_list( + "pyi_srcs": lambda: attrb.LabelList( doc = """ Type definition files for the library. @@ -404,37 +358,61 @@ as part of a runnable program (packaging rules may include them, however). """, allow_files = True, ), - # Required attribute, but details vary by rule. - # Use create_srcs_attr to create one. - "srcs": None, - # NOTE: In Google, this attribute is deprecated, and can only - # effectively be PY3 or PY3ONLY. Externally, with Bazel, this attribute - # has a separate story. - # Required attribute, but the details vary by rule. - # Use create_srcs_version_attr to create one. - "srcs_version": None, - "_precompile_flag": attr.label( + "srcs": lambda: attrb.LabelList( + # Google builds change the set of allowed files. + allow_files = SRCS_ATTR_ALLOW_FILES, + # Necessary for --compile_one_dependency to work. + flags = ["DIRECT_COMPILE_TIME_INPUT"], + doc = """ +The list of Python source files that are processed to create the target. This +includes all your checked-in code and may include generated source files. The +`.py` files belong in `srcs` and library targets belong in `deps`. Other binary +files that may be needed at run time belong in `data`. +""", + ), + "srcs_version": lambda: attrb.String( + doc = "Defunct, unused, does nothing.", + ), + "_precompile_flag": lambda: attrb.Label( default = "//python/config_settings:precompile", providers = [BuildSettingInfo], ), - "_precompile_source_retention_flag": attr.label( + "_precompile_source_retention_flag": lambda: attrb.Label( default = "//python/config_settings:precompile_source_retention", providers = [BuildSettingInfo], ), # Force enabling auto exec groups, see # https://bazel.build/extending/auto-exec-groups#how-enable-particular-rule - "_use_auto_exec_groups": attr.bool(default = True), + "_use_auto_exec_groups": lambda: attrb.Bool( + default = True, + ), }, - allow_none = True, ) +COVERAGE_ATTRS = { + # Magic attribute to help C++ coverage work. There's no + # docs about this; see TestActionBuilder.java + "_collect_cc_coverage": lambda: attrb.Label( + default = "@bazel_tools//tools/test:collect_cc_coverage", + executable = True, + cfg = "exec", + ), + # Magic attribute to make coverage work. There's no + # docs about this; see TestActionBuilder.java + "_lcov_merger": lambda: attrb.Label( + default = configuration_field(fragment = "coverage", name = "output_generator"), + executable = True, + cfg = "exec", + ), +} + # Attributes specific to Python executable-equivalent rules. Such rules may not # accept Python sources (e.g. some packaged-version of a py_test/py_binary), but # still accept Python source-agnostic settings. -AGNOSTIC_EXECUTABLE_ATTRS = union_attrs( +AGNOSTIC_EXECUTABLE_ATTRS = dicts.add( DATA_ATTRS, { - "env": attr.string_dict( + "env": lambda: attrb.StringDict( doc = """\ Dictionary of strings; optional; values are subject to `$(location)` and "Make variable" substitution. @@ -443,22 +421,40 @@ Specifies additional environment variables to set when the target is executed by `test` or `run`. """, ), - # The value is required, but varies by rule and/or rule type. Use - # create_stamp_attr to create one. - "stamp": None, + "stamp": lambda: attrb.Int( + values = _STAMP_VALUES, + doc = """ +Whether to encode build information into the binary. Possible values: + +* `stamp = 1`: Always stamp the build information into the binary, even in + `--nostamp` builds. **This setting should be avoided**, since it potentially kills + remote caching for the binary and any downstream actions that depend on it. +* `stamp = 0`: Always replace build information by constant values. This gives + good build result caching. +* `stamp = -1`: Embedding of build information is controlled by the + `--[no]stamp` flag. + +Stamped binaries are not rebuilt unless their dependencies change. + +WARNING: Stamping can harm build performance by reducing cache hits and should +be avoided if possible. +""", + default = -1, + ), }, - allow_none = True, ) -# Attributes specific to Python test-equivalent executable rules. Such rules may -# not accept Python sources (e.g. some packaged-version of a py_test/py_binary), -# but still accept Python source-agnostic settings. -AGNOSTIC_TEST_ATTRS = union_attrs( - AGNOSTIC_EXECUTABLE_ATTRS, +def _init_agnostic_test_attrs(): + base_stamp = AGNOSTIC_EXECUTABLE_ATTRS["stamp"] + # Tests have stamping disabled by default. - create_stamp_attr(default = 0), - { - "env_inherit": attr.string_list( + def stamp_default_disabled(): + b = base_stamp() + b.set_default(0) + return b + + return dicts.add(AGNOSTIC_EXECUTABLE_ATTRS, { + "env_inherit": lambda: attrb.StringList( doc = """\ List of strings; optional @@ -466,8 +462,9 @@ Specifies additional environment variables to inherit from the external environment when the test is executed by bazel test. """, ), + "stamp": stamp_default_disabled, # TODO(b/176993122): Remove when Bazel automatically knows to run on darwin. - "_apple_constraints": attr.label_list( + "_apple_constraints": lambda: attrb.LabelList( default = [ "@platforms//os:ios", "@platforms//os:macos", @@ -476,16 +473,17 @@ environment when the test is executed by bazel test. "@platforms//os:watchos", ], ), - }, -) + }) + +# Attributes specific to Python test-equivalent executable rules. Such rules may +# not accept Python sources (e.g. some packaged-version of a py_test/py_binary), +# but still accept Python source-agnostic settings. +AGNOSTIC_TEST_ATTRS = _init_agnostic_test_attrs() # Attributes specific to Python binary-equivalent executable rules. Such rules may # not accept Python sources (e.g. some packaged-version of a py_test/py_binary), # but still accept Python source-agnostic settings. -AGNOSTIC_BINARY_ATTRS = union_attrs( - AGNOSTIC_EXECUTABLE_ATTRS, - create_stamp_attr(default = -1), -) +AGNOSTIC_BINARY_ATTRS = dicts.add(AGNOSTIC_EXECUTABLE_ATTRS) # Attribute names common to all Python rules COMMON_ATTR_NAMES = [ diff --git a/python/private/builders.bzl b/python/private/builders.bzl index bf5dbb8667..50aa3ed91a 100644 --- a/python/private/builders.bzl +++ b/python/private/builders.bzl @@ -96,145 +96,6 @@ def _DepsetBuilder_build(self): kwargs["order"] = self._order[0] return depset(direct = self.direct, transitive = self.transitive, **kwargs) -def _Optional(*initial): - """A wrapper for a re-assignable value that may or may not be set. - - This allows structs to have attributes that aren't inherently mutable - and must be re-assigned to have their value updated. - - Args: - *initial: A single vararg to be the initial value, or no args - to leave it unset. - - Returns: - {type}`Optional` - """ - if len(initial) > 1: - fail("Only zero or one positional arg allowed") - - # buildifier: disable=uninitialized - self = struct( - _value = list(initial), - present = lambda *a, **k: _Optional_present(self, *a, **k), - set = lambda *a, **k: _Optional_set(self, *a, **k), - get = lambda *a, **k: _Optional_get(self, *a, **k), - ) - return self - -def _Optional_set(self, value): - """Sets the value of the optional. - - Args: - self: implicitly added - value: the value to set. - """ - if len(self._value) == 0: - self._value.append(value) - else: - self._value[0] = value - -def _Optional_get(self): - """Gets the value of the optional, or error. - - Args: - self: implicitly added - - Returns: - The stored value, or error if not set. - """ - if not len(self._value): - fail("Value not present") - return self._value[0] - -def _Optional_present(self): - """Tells if a value is present. - - Args: - self: implicitly added - - Returns: - {type}`bool` True if the value is set, False if not. - """ - return len(self._value) > 0 - -def _RuleBuilder(implementation = None, **kwargs): - """Builder for creating rules. - - Args: - implementation: {type}`callable` The rule implementation function. - **kwargs: The same as the `rule()` function, but using builders - for the non-mutable Bazel objects. - """ - - # buildifier: disable=uninitialized - self = struct( - attrs = dict(kwargs.pop("attrs", None) or {}), - cfg = kwargs.pop("cfg", None) or _TransitionBuilder(), - exec_groups = dict(kwargs.pop("exec_groups", None) or {}), - executable = _Optional(), - fragments = list(kwargs.pop("fragments", None) or []), - implementation = _Optional(implementation), - extra_kwargs = kwargs, - provides = list(kwargs.pop("provides", None) or []), - test = _Optional(), - toolchains = list(kwargs.pop("toolchains", None) or []), - build = lambda *a, **k: _RuleBuilder_build(self, *a, **k), - to_kwargs = lambda *a, **k: _RuleBuilder_to_kwargs(self, *a, **k), - ) - if "test" in kwargs: - self.test.set(kwargs.pop("test")) - if "executable" in kwargs: - self.executable.set(kwargs.pop("executable")) - return self - -def _RuleBuilder_build(self, debug = ""): - """Builds a `rule` object - - Args: - self: implicitly added - debug: {type}`str` If set, prints the args used to create the rule. - - Returns: - {type}`rule` - """ - kwargs = self.to_kwargs() - if debug: - lines = ["=" * 80, "rule kwargs: {}:".format(debug)] - for k, v in sorted(kwargs.items()): - lines.append(" {}={}".format(k, v)) - print("\n".join(lines)) # buildifier: disable=print - return rule(**kwargs) - -def _RuleBuilder_to_kwargs(self): - """Builds the arguments for calling `rule()`. - - Args: - self: implicitly added - - Returns: - {type}`dict` - """ - kwargs = {} - if self.executable.present(): - kwargs["executable"] = self.executable.get() - if self.test.present(): - kwargs["test"] = self.test.get() - - kwargs.update( - implementation = self.implementation.get(), - cfg = self.cfg.build() if self.cfg.implementation.present() else None, - attrs = { - k: (v.build() if hasattr(v, "build") else v) - for k, v in self.attrs.items() - }, - exec_groups = self.exec_groups, - fragments = self.fragments, - provides = self.provides, - toolchains = self.toolchains, - ) - kwargs.update(self.extra_kwargs) - return kwargs - def _RunfilesBuilder(): """Creates a `RunfilesBuilder`. @@ -316,91 +177,6 @@ def _RunfilesBuilder_build(self, ctx, **kwargs): **kwargs ).merge_all(self.runfiles) -def _SetBuilder(initial = None): - """Builder for list of unique values. - - Args: - initial: {type}`list | None` The initial values. - - Returns: - {type}`SetBuilder` - """ - initial = {} if not initial else {v: None for v in initial} - - # buildifier: disable=uninitialized - self = struct( - # TODO - Switch this to use set() builtin when available - # https://bazel.build/rules/lib/core/set - _values = initial, - update = lambda *a, **k: _SetBuilder_update(self, *a, **k), - build = lambda *a, **k: _SetBuilder_build(self, *a, **k), - ) - return self - -def _SetBuilder_build(self): - """Builds the values into a list - - Returns: - {type}`list` - """ - return self._values.keys() - -def _SetBuilder_update(self, *others): - """Adds values to the builder. - - Args: - self: implicitly added - *others: {type}`list` values to add to the set. - """ - for other in others: - for value in other: - if value not in self._values: - self._values[value] = None - -def _TransitionBuilder(implementation = None, inputs = None, outputs = None, **kwargs): - """Builder for transition objects. - - Args: - implementation: {type}`callable` the transition implementation function. - inputs: {type}`list[str]` the inputs for the transition. - outputs: {type}`list[str]` the outputs of the transition. - **kwargs: Extra keyword args to use when building. - - Returns: - {type}`TransitionBuilder` - """ - - # buildifier: disable=uninitialized - self = struct( - implementation = _Optional(implementation), - # Bazel requires transition.inputs to have unique values, so use set - # semantics so extenders of a transition can easily add/remove values. - # TODO - Use set builtin instead of custom builder, when available. - # https://bazel.build/rules/lib/core/set - inputs = _SetBuilder(inputs), - # Bazel requires transition.inputs to have unique values, so use set - # semantics so extenders of a transition can easily add/remove values. - # TODO - Use set builtin instead of custom builder, when available. - # https://bazel.build/rules/lib/core/set - outputs = _SetBuilder(outputs), - extra_kwargs = kwargs, - build = lambda *a, **k: _TransitionBuilder_build(self, *a, **k), - ) - return self - -def _TransitionBuilder_build(self): - """Creates a transition from the builder. - - Returns: - {type}`transition` - """ - return transition( - implementation = self.implementation.get(), - inputs = self.inputs.build(), - outputs = self.outputs.build(), - **self.extra_kwargs - ) - # Skylib's types module doesn't have is_file, so roll our own def _is_file(value): return type(value) == "File" @@ -411,8 +187,4 @@ def _is_runfiles(value): builders = struct( DepsetBuilder = _DepsetBuilder, RunfilesBuilder = _RunfilesBuilder, - RuleBuilder = _RuleBuilder, - TransitionBuilder = _TransitionBuilder, - SetBuilder = _SetBuilder, - Optional = _Optional, ) diff --git a/python/private/builders_util.bzl b/python/private/builders_util.bzl new file mode 100644 index 0000000000..139084f79a --- /dev/null +++ b/python/private/builders_util.bzl @@ -0,0 +1,116 @@ +# Copyright 2025 The Bazel Authors. All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +"""Utilities for builders.""" + +load("@bazel_skylib//lib:types.bzl", "types") + +def to_label_maybe(value): + """Converts `value` to a `Label`, maybe. + + The "maybe" qualification is because invalid values for `Label()` + are returned as-is (e.g. None, or special values that might be + used with e.g. the `default` attribute arg). + + Args: + value: {type}`str | Label | None | object` the value to turn into a label, + or return as-is. + + Returns: + {type}`Label | input_value` + """ + if value == None: + return None + if is_label(value): + return value + if types.is_string(value): + return Label(value) + return value + +def is_label(obj): + """Tell if an object is a `Label`.""" + return type(obj) == "Label" + +def kwargs_set_default_ignore_none(kwargs, key, default): + """Normalize None/missing to `default`.""" + existing = kwargs.get(key) + if existing == None: + kwargs[key] = default + +def kwargs_set_default_list(kwargs, key): + """Normalizes None/missing to list.""" + existing = kwargs.get(key) + if existing == None: + kwargs[key] = [] + +def kwargs_set_default_dict(kwargs, key): + """Normalizes None/missing to list.""" + existing = kwargs.get(key) + if existing == None: + kwargs[key] = {} + +def kwargs_set_default_doc(kwargs): + """Sets the `doc` arg default.""" + existing = kwargs.get("doc") + if existing == None: + kwargs["doc"] = "" + +def kwargs_set_default_mandatory(kwargs): + """Sets `False` as the `mandatory` arg default.""" + existing = kwargs.get("mandatory") + if existing == None: + kwargs["mandatory"] = False + +def kwargs_getter(kwargs, key): + """Create a function to get `key` from `kwargs`.""" + return lambda: kwargs.get(key) + +def kwargs_setter(kwargs, key): + """Create a function to set `key` in `kwargs`.""" + + def setter(v): + kwargs[key] = v + + return setter + +def kwargs_getter_doc(kwargs): + """Creates a `kwargs_getter` for the `doc` key.""" + return kwargs_getter(kwargs, "doc") + +def kwargs_setter_doc(kwargs): + """Creates a `kwargs_setter` for the `doc` key.""" + return kwargs_setter(kwargs, "doc") + +def kwargs_getter_mandatory(kwargs): + """Creates a `kwargs_getter` for the `mandatory` key.""" + return kwargs_getter(kwargs, "mandatory") + +def kwargs_setter_mandatory(kwargs): + """Creates a `kwargs_setter` for the `mandatory` key.""" + return kwargs_setter(kwargs, "mandatory") + +def list_add_unique(add_to, others): + """Bulk add values to a list if not already present. + + Args: + add_to: {type}`list[T]` the list to add values to. It is modified + in-place. + others: {type}`collection[collection[T]]` collection of collections of + the values to add. + """ + existing = {v: None for v in add_to} + for values in others: + for value in values: + if value not in existing: + add_to.append(value) diff --git a/python/private/common.bzl b/python/private/common.bzl index 137f0d23f3..48e2653ebb 100644 --- a/python/private/common.bzl +++ b/python/private/common.bzl @@ -208,52 +208,6 @@ def create_executable_result_struct(*, extra_files_to_build, output_groups, extr extra_runfiles = extra_runfiles, ) -def union_attrs(*attr_dicts, allow_none = False): - """Helper for combining and building attriute dicts for rules. - - Similar to dict.update, except: - * Duplicate keys raise an error if they aren't equal. This is to prevent - unintentionally replacing an attribute with a potentially incompatible - definition. - * None values are special: They mean the attribute is required, but the - value should be provided by another attribute dict (depending on the - `allow_none` arg). - Args: - *attr_dicts: The dicts to combine. - allow_none: bool, if True, then None values are allowed. If False, - then one of `attrs_dicts` must set a non-None value for keys - with a None value. - - Returns: - dict of attributes. - """ - result = {} - missing = {} - for attr_dict in attr_dicts: - for attr_name, value in attr_dict.items(): - if value == None and not allow_none: - if attr_name not in result: - missing[attr_name] = None - else: - if attr_name in missing: - missing.pop(attr_name) - - if attr_name not in result or result[attr_name] == None: - result[attr_name] = value - elif value != None and result[attr_name] != value: - fail("Duplicate attribute name: '{}': existing={}, new={}".format( - attr_name, - result[attr_name], - value, - )) - - # Else, they're equal, so do nothing. This allows merging dicts - # that both define the same key from a common place. - - if missing and not allow_none: - fail("Required attributes missing: " + csv(missing.keys())) - return result - def csv(values): """Convert a list of strings to comma separated value string.""" return ", ".join(sorted(values)) diff --git a/python/private/py_binary_rule.bzl b/python/private/py_binary_rule.bzl index 5b40f52198..0e1912cf0c 100644 --- a/python/private/py_binary_rule.bzl +++ b/python/private/py_binary_rule.bzl @@ -20,23 +20,6 @@ load( "py_executable_impl", ) -_COVERAGE_ATTRS = { - # Magic attribute to help C++ coverage work. There's no - # docs about this; see TestActionBuilder.java - "_collect_cc_coverage": attr.label( - default = "@bazel_tools//tools/test:collect_cc_coverage", - executable = True, - cfg = "exec", - ), - # Magic attribute to make coverage work. There's no - # docs about this; see TestActionBuilder.java - "_lcov_merger": attr.label( - default = configuration_field(fragment = "coverage", name = "output_generator"), - executable = True, - cfg = "exec", - ), -} - def _py_binary_impl(ctx): return py_executable_impl( ctx = ctx, @@ -50,7 +33,6 @@ def create_binary_rule_builder(): executable = True, ) builder.attrs.update(AGNOSTIC_BINARY_ATTRS) - builder.attrs.update(_COVERAGE_ATTRS) return builder py_binary = create_binary_rule_builder().build() diff --git a/python/private/py_executable.bzl b/python/private/py_executable.bzl index a2ccdc65f3..f85f242bba 100644 --- a/python/private/py_executable.bzl +++ b/python/private/py_executable.bzl @@ -18,18 +18,17 @@ load("@bazel_skylib//lib:paths.bzl", "paths") load("@bazel_skylib//lib:structs.bzl", "structs") load("@bazel_skylib//rules:common_settings.bzl", "BuildSettingInfo") load("@rules_cc//cc/common:cc_common.bzl", "cc_common") +load(":attr_builders.bzl", "attrb") load( ":attributes.bzl", "AGNOSTIC_EXECUTABLE_ATTRS", "COMMON_ATTRS", + "COVERAGE_ATTRS", "IMPORTS_ATTRS", "PY_SRCS_ATTRS", "PrecompileAttr", "PycCollectionAttr", - "REQUIRED_EXEC_GROUPS", - "SRCS_VERSION_ALL_VALUES", - "create_srcs_attr", - "create_srcs_version_attr", + "REQUIRED_EXEC_GROUP_BUILDERS", ) load(":builders.bzl", "builders") load(":cc_helper.bzl", "cc_helper") @@ -50,7 +49,6 @@ load( "is_bool", "runfiles_root_path", "target_platform_has_any_constraint", - "union_attrs", ) load(":flags.bzl", "BootstrapImplFlag", "VenvsUseDeclareSymlinkFlag") load(":precompile.bzl", "maybe_precompile") @@ -60,6 +58,7 @@ load(":py_info.bzl", "PyInfo") load(":py_internal.bzl", "py_internal") load(":py_runtime_info.bzl", "DEFAULT_STUB_SHEBANG", "PyRuntimeInfo") load(":reexports.bzl", "BuiltinPyInfo", "BuiltinPyRuntimeInfo") +load(":rule_builders.bzl", "ruleb") load( ":semantics.bzl", "ALLOWED_MAIN_EXTENSIONS", @@ -79,21 +78,16 @@ _EXTERNAL_PATH_PREFIX = "external" _ZIP_RUNFILES_DIRECTORY_NAME = "runfiles" _PYTHON_VERSION_FLAG = str(Label("//python/config_settings:python_version")) -# Bazel 5.4 doesn't have config_common.toolchain_type -_CC_TOOLCHAINS = [config_common.toolchain_type( - "@bazel_tools//tools/cpp:toolchain_type", - mandatory = False, -)] if hasattr(config_common, "toolchain_type") else [] - # Non-Google-specific attributes for executables # These attributes are for rules that accept Python sources. -EXECUTABLE_ATTRS = union_attrs( +EXECUTABLE_ATTRS = dicts.add( COMMON_ATTRS, AGNOSTIC_EXECUTABLE_ATTRS, PY_SRCS_ATTRS, IMPORTS_ATTRS, + COVERAGE_ATTRS, { - "legacy_create_init": attr.int( + "legacy_create_init": lambda: attrb.Int( default = -1, values = [-1, 0, 1], doc = """\ @@ -110,7 +104,7 @@ the `srcs` of Python targets as required. # label, it is more treated as a string, and doesn't have to refer to # anything that exists because it gets treated as suffix-search string # over `srcs`. - "main": attr.label( + "main": lambda: attrb.Label( allow_single_file = True, doc = """\ Optional; the name of the source file that is the main entry point of the @@ -119,7 +113,7 @@ application. This file must also be listed in `srcs`. If left unspecified, filename in `srcs`, `main` must be specified. """, ), - "pyc_collection": attr.string( + "pyc_collection": lambda: attrb.String( default = PycCollectionAttr.INHERIT, values = sorted(PycCollectionAttr.__members__.values()), doc = """ @@ -134,7 +128,7 @@ Valid values are: target level. """, ), - "python_version": attr.string( + "python_version": lambda: attrb.String( # TODO(b/203567235): In the Java impl, the default comes from # --python_version. Not clear what the Starlark equivalent is. doc = """ @@ -160,25 +154,25 @@ accepting arbitrary Python versions. """, ), # Required to opt-in to the transition feature. - "_allowlist_function_transition": attr.label( + "_allowlist_function_transition": lambda: attrb.Label( default = "@bazel_tools//tools/allowlists/function_transition_allowlist", ), - "_bootstrap_impl_flag": attr.label( + "_bootstrap_impl_flag": lambda: attrb.Label( default = "//python/config_settings:bootstrap_impl", providers = [BuildSettingInfo], ), - "_bootstrap_template": attr.label( + "_bootstrap_template": lambda: attrb.Label( allow_single_file = True, default = "@bazel_tools//tools/python:python_bootstrap_template.txt", ), - "_launcher": attr.label( + "_launcher": lambda: attrb.Label( cfg = "target", # NOTE: This is an executable, but is only used for Windows. It # can't have executable=True because the backing target is an # empty target for other platforms. default = "//tools/launcher:launcher", ), - "_py_interpreter": attr.label( + "_py_interpreter": lambda: attrb.Label( # The configuration_field args are validated when called; # we use the precense of py_internal to indicate this Bazel # build has that fragment and name. @@ -193,32 +187,29 @@ accepting arbitrary Python versions. "_py_toolchain_type": attr.label( default = TARGET_TOOLCHAIN_TYPE, ), - "_python_version_flag": attr.label( + "_python_version_flag": lambda: attrb.Label( default = "//python/config_settings:python_version", ), - "_venvs_use_declare_symlink_flag": attr.label( + "_venvs_use_declare_symlink_flag": lambda: attrb.Label( default = "//python/config_settings:venvs_use_declare_symlink", providers = [BuildSettingInfo], ), - "_windows_constraints": attr.label_list( + "_windows_constraints": lambda: attrb.LabelList( default = [ "@platforms//os:windows", ], ), - "_windows_launcher_maker": attr.label( + "_windows_launcher_maker": lambda: attrb.Label( default = "@bazel_tools//tools/launcher:launcher_maker", cfg = "exec", executable = True, ), - "_zipper": attr.label( + "_zipper": lambda: attrb.Label( cfg = "exec", executable = True, default = "@bazel_tools//tools/zip:zipper", ), }, - create_srcs_version_attr(values = SRCS_VERSION_ALL_VALUES), - create_srcs_attr(mandatory = True), - allow_none = True, ) def convert_legacy_create_init_to_int(kwargs): @@ -1747,23 +1738,25 @@ def create_base_executable_rule(): return create_executable_rule_builder().build() def create_executable_rule_builder(implementation, **kwargs): - builder = builders.RuleBuilder( + builder = ruleb.Rule( implementation = implementation, attrs = EXECUTABLE_ATTRS, - exec_groups = REQUIRED_EXEC_GROUPS, + exec_groups = dict(REQUIRED_EXEC_GROUP_BUILDERS), # Mutable copy fragments = ["py", "bazel_py"], provides = [PyExecutableInfo], toolchains = [ - TOOLCHAIN_TYPE, - config_common.toolchain_type(EXEC_TOOLS_TOOLCHAIN_TYPE, mandatory = False), - ] + _CC_TOOLCHAINS, - cfg = builders.TransitionBuilder( + ruleb.ToolchainType(TOOLCHAIN_TYPE), + ruleb.ToolchainType(EXEC_TOOLS_TOOLCHAIN_TYPE, mandatory = False), + ruleb.ToolchainType("@bazel_tools//tools/cpp:toolchain_type", mandatory = False), + ], + cfg = dict( implementation = _transition_executable_impl, inputs = [_PYTHON_VERSION_FLAG], outputs = [_PYTHON_VERSION_FLAG], ), **kwargs ) + builder.attrs.get("srcs").set_mandatory(True) return builder def cc_configure_features( diff --git a/python/private/py_library.bzl b/python/private/py_library.bzl index 350ea35aa6..a774104dd2 100644 --- a/python/private/py_library.bzl +++ b/python/private/py_library.bzl @@ -15,16 +15,14 @@ load("@bazel_skylib//lib:dicts.bzl", "dicts") load("@bazel_skylib//rules:common_settings.bzl", "BuildSettingInfo") +load(":attr_builders.bzl", "attrb") load( ":attributes.bzl", "COMMON_ATTRS", "IMPORTS_ATTRS", "PY_SRCS_ATTRS", "PrecompileAttr", - "REQUIRED_EXEC_GROUPS", - "SRCS_VERSION_ALL_VALUES", - "create_srcs_attr", - "create_srcs_version_attr", + "REQUIRED_EXEC_GROUP_BUILDERS", ) load(":builders.bzl", "builders") load( @@ -35,11 +33,11 @@ load( "create_output_group_info", "create_py_info", "filter_to_py_srcs", - "union_attrs", ) load(":flags.bzl", "AddSrcsToRunfilesFlag", "PrecompileFlag") load(":py_cc_link_params_info.bzl", "PyCcLinkParamsInfo") load(":py_internal.bzl", "py_internal") +load(":rule_builders.bzl", "ruleb") load( ":toolchain_types.bzl", "EXEC_TOOLS_TOOLCHAIN_TYPE", @@ -48,14 +46,12 @@ load( _py_builtins = py_internal -LIBRARY_ATTRS = union_attrs( +LIBRARY_ATTRS = dicts.add( COMMON_ATTRS, PY_SRCS_ATTRS, IMPORTS_ATTRS, - create_srcs_version_attr(values = SRCS_VERSION_ALL_VALUES), - create_srcs_attr(mandatory = False), { - "_add_srcs_to_runfiles_flag": attr.label( + "_add_srcs_to_runfiles_flag": lambda: attrb.Label( default = "//python/config_settings:add_srcs_to_runfiles", ), }, @@ -145,14 +141,15 @@ Source files are no longer added to the runfiles directly. ::: """ -def create_py_library_rule(*, attrs = {}, **kwargs): +def create_py_library_rule_builder(*, attrs = {}, **kwargs): """Creates a py_library rule. Args: attrs: dict of rule attributes. - **kwargs: Additional kwargs to pass onto the rule() call. + **kwargs: Additional kwargs to pass onto {obj}`ruleb.Rule()`. + Returns: - A rule object + {type}`ruleb.Rule` builder object. """ # Within Google, the doc attribute is overridden @@ -161,13 +158,15 @@ def create_py_library_rule(*, attrs = {}, **kwargs): # TODO: b/253818097 - fragments=py is only necessary so that # RequiredConfigFragmentsTest passes fragments = kwargs.pop("fragments", None) or [] - kwargs["exec_groups"] = REQUIRED_EXEC_GROUPS | (kwargs.get("exec_groups") or {}) - return rule( + kwargs["exec_groups"] = REQUIRED_EXEC_GROUP_BUILDERS | (kwargs.get("exec_groups") or {}) + + builder = ruleb.Rule( attrs = dicts.add(LIBRARY_ATTRS, attrs), + fragments = fragments + ["py"], toolchains = [ - config_common.toolchain_type(TOOLCHAIN_TYPE, mandatory = False), - config_common.toolchain_type(EXEC_TOOLS_TOOLCHAIN_TYPE, mandatory = False), + ruleb.ToolchainType(TOOLCHAIN_TYPE, mandatory = False), + ruleb.ToolchainType(EXEC_TOOLS_TOOLCHAIN_TYPE, mandatory = False), ], - fragments = fragments + ["py"], **kwargs ) + return builder diff --git a/python/private/py_library_rule.bzl b/python/private/py_library_rule.bzl index 8a8d6cf380..44382a76d6 100644 --- a/python/private/py_library_rule.bzl +++ b/python/private/py_library_rule.bzl @@ -15,7 +15,7 @@ load(":common.bzl", "collect_cc_info", "create_library_semantics_struct", "get_imports") load(":precompile.bzl", "maybe_precompile") -load(":py_library.bzl", "create_py_library_rule", "py_library_impl") +load(":py_library.bzl", "create_py_library_rule_builder", "py_library_impl") def _py_library_impl_with_semantics(ctx): return py_library_impl( @@ -27,6 +27,6 @@ def _py_library_impl_with_semantics(ctx): ), ) -py_library = create_py_library_rule( +py_library = create_py_library_rule_builder( implementation = _py_library_impl_with_semantics, -) +).build() diff --git a/python/private/py_runtime_rule.bzl b/python/private/py_runtime_rule.bzl index 5ce8161cf0..9407cac50f 100644 --- a/python/private/py_runtime_rule.bzl +++ b/python/private/py_runtime_rule.bzl @@ -188,19 +188,21 @@ py_runtime( ``` """, fragments = ["py"], - attrs = dicts.add(NATIVE_RULES_ALLOWLIST_ATTRS, { - "abi_flags": attr.string( - default = "", - doc = """ + attrs = dicts.add( + {k: v().build() for k, v in NATIVE_RULES_ALLOWLIST_ATTRS.items()}, + { + "abi_flags": attr.string( + default = "", + doc = """ The runtime's ABI flags, i.e. `sys.abiflags`. If not set, then it will be set based on flags. """, - ), - "bootstrap_template": attr.label( - allow_single_file = True, - default = DEFAULT_BOOTSTRAP_TEMPLATE, - doc = """ + ), + "bootstrap_template": attr.label( + allow_single_file = True, + default = DEFAULT_BOOTSTRAP_TEMPLATE, + doc = """ The bootstrap script template file to use. Should have %python_binary%, %workspace_name%, %main%, and %imports%. @@ -218,10 +220,10 @@ itself. See @bazel_tools//tools/python:python_bootstrap_template.txt for more variables. """, - ), - "coverage_tool": attr.label( - allow_files = False, - doc = """ + ), + "coverage_tool": attr.label( + allow_files = False, + doc = """ This is a target to use for collecting code coverage information from {rule}`py_binary` and {rule}`py_test` targets. @@ -235,25 +237,25 @@ The entry point for the tool must be loadable by a Python interpreter (e.g. a of [`coverage.py`](https://coverage.readthedocs.io), at least including the `run` and `lcov` subcommands. """, - ), - "files": attr.label_list( - allow_files = True, - doc = """ + ), + "files": attr.label_list( + allow_files = True, + doc = """ For an in-build runtime, this is the set of files comprising this runtime. These files will be added to the runfiles of Python binaries that use this runtime. For a platform runtime this attribute must not be set. """, - ), - "implementation_name": attr.string( - doc = "The Python implementation name (`sys.implementation.name`)", - default = "cpython", - ), - "interpreter": attr.label( - # We set `allow_files = True` to allow specifying executable - # targets from rules that have more than one default output, - # e.g. sh_binary. - allow_files = True, - doc = """ + ), + "implementation_name": attr.string( + doc = "The Python implementation name (`sys.implementation.name`)", + default = "cpython", + ), + "interpreter": attr.label( + # We set `allow_files = True` to allow specifying executable + # targets from rules that have more than one default output, + # e.g. sh_binary. + allow_files = True, + doc = """ For an in-build runtime, this is the target to invoke as the interpreter. It can be either of: @@ -272,13 +274,13 @@ can be either of: For a platform runtime (i.e. `interpreter_path` being set) this attribute must not be set. """, - ), - "interpreter_path": attr.string(doc = """ + ), + "interpreter_path": attr.string(doc = """ For a platform runtime, this is the absolute path of a Python interpreter on the target platform. For an in-build runtime this attribute must not be set. """), - "interpreter_version_info": attr.string_dict( - doc = """ + "interpreter_version_info": attr.string_dict( + doc = """ Version information about the interpreter this runtime provides. If not specified, uses {obj}`--python_version` @@ -295,20 +297,20 @@ values are strings, most are converted to ints. The supported keys are: {obj}`--python_version` determines the default value. ::: """, - mandatory = False, - ), - "pyc_tag": attr.string( - doc = """ + mandatory = False, + ), + "pyc_tag": attr.string( + doc = """ Optional string; the tag portion of a pyc filename, e.g. the `cpython-39` infix of `foo.cpython-39.pyc`. See PEP 3147. If not specified, it will be computed from `implementation_name` and `interpreter_version_info`. If no pyc_tag is available, then only source-less pyc generation will function correctly. """, - ), - "python_version": attr.string( - default = "PY3", - values = ["PY2", "PY3"], - doc = """ + ), + "python_version": attr.string( + default = "PY3", + values = ["PY2", "PY3"], + doc = """ Whether this runtime is for Python major version 2 or 3. Valid values are `"PY2"` and `"PY3"`. @@ -316,32 +318,32 @@ The default value is controlled by the `--incompatible_py3_is_default` flag. However, in the future this attribute will be mandatory and have no default value. """, - ), - "site_init_template": attr.label( - allow_single_file = True, - default = "//python/private:site_init_template", - doc = """ + ), + "site_init_template": attr.label( + allow_single_file = True, + default = "//python/private:site_init_template", + doc = """ The template to use for the binary-specific site-init hook run by the interpreter at startup. :::{versionadded} 0.41.0 ::: """, - ), - "stage2_bootstrap_template": attr.label( - default = "//python/private:stage2_bootstrap_template", - allow_single_file = True, - doc = """ + ), + "stage2_bootstrap_template": attr.label( + default = "//python/private:stage2_bootstrap_template", + allow_single_file = True, + doc = """ The template to use when two stage bootstrapping is enabled :::{seealso} {obj}`PyRuntimeInfo.stage2_bootstrap_template` and {obj}`--bootstrap_impl` ::: """, - ), - "stub_shebang": attr.string( - default = DEFAULT_STUB_SHEBANG, - doc = """ + ), + "stub_shebang": attr.string( + default = DEFAULT_STUB_SHEBANG, + doc = """ "Shebang" expression prepended to the bootstrapping Python stub script used when executing {rule}`py_binary` targets. @@ -350,11 +352,11 @@ motivation. Does not apply to Windows. """, - ), - "zip_main_template": attr.label( - default = "//python/private:zip_main_template", - allow_single_file = True, - doc = """ + ), + "zip_main_template": attr.label( + default = "//python/private:zip_main_template", + allow_single_file = True, + doc = """ The template to use for a zip's top-level `__main__.py` file. This becomes the entry point executed when `python foo.zip` is run. @@ -363,14 +365,15 @@ This becomes the entry point executed when `python foo.zip` is run. The {obj}`PyRuntimeInfo.zip_main_template` field. ::: """, - ), - "_py_freethreaded_flag": attr.label( - default = "//python/config_settings:py_freethreaded", - ), - "_python_version_flag": attr.label( - default = "//python/config_settings:python_version", - ), - }), + ), + "_py_freethreaded_flag": attr.label( + default = "//python/config_settings:py_freethreaded", + ), + "_python_version_flag": attr.label( + default = "//python/config_settings:python_version", + ), + }, + ), ) def _is_singleton_depset(files): diff --git a/python/private/py_test_rule.bzl b/python/private/py_test_rule.bzl index 6ad4fbddb8..72e8bab805 100644 --- a/python/private/py_test_rule.bzl +++ b/python/private/py_test_rule.bzl @@ -21,23 +21,6 @@ load( "py_executable_impl", ) -_BAZEL_PY_TEST_ATTRS = { - # This *might* be a magic attribute to help C++ coverage work. There's no - # docs about this; see TestActionBuilder.java - "_collect_cc_coverage": attr.label( - default = "@bazel_tools//tools/test:collect_cc_coverage", - executable = True, - cfg = "exec", - ), - # This *might* be a magic attribute to help C++ coverage work. There's no - # docs about this; see TestActionBuilder.java - "_lcov_merger": attr.label( - default = configuration_field(fragment = "coverage", name = "output_generator"), - cfg = "exec", - executable = True, - ), -} - def _py_test_impl(ctx): providers = py_executable_impl( ctx = ctx, @@ -53,7 +36,6 @@ def create_test_rule_builder(): test = True, ) builder.attrs.update(AGNOSTIC_TEST_ATTRS) - builder.attrs.update(_BAZEL_PY_TEST_ATTRS) return builder py_test = create_test_rule_builder().build() diff --git a/python/private/rule_builders.bzl b/python/private/rule_builders.bzl new file mode 100644 index 0000000000..6d9fb3f964 --- /dev/null +++ b/python/private/rule_builders.bzl @@ -0,0 +1,692 @@ +# Copyright 2025 The Bazel Authors. All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +"""Builders for creating rules, aspects et al. + +When defining rules, Bazel only allows creating *immutable* objects that can't +be introspected. This makes it difficult to perform arbitrary customizations of +how a rule is defined, which makes extending a rule implementation prone to +copy/paste issues and version skew. + +These builders are, essentially, mutable and inspectable wrappers for those +Bazel objects. This allows defining a rule where the values are mutable and +callers can customize them to derive their own variant of the rule while still +inheriting everything else about the rule. + +To that end, the builders are not strict in how they handle values. They +generally assume that the values provided are valid and provide ways to +override their logic and force particular values to be used when they are +eventually converted to the args for calling e.g. `rule()`. + +:::{important} +When using builders, most lists, dicts, et al passed into them **must** be +locally created values, otherwise they won't be mutable. This is due to Bazel's +implicit immutability rules: after evaluating a `.bzl` file, its global +variables are frozen. +::: + +:::{tip} +To aid defining reusable pieces, many APIs accept no-arg callable functions +that create a builder. For example, common attributes can be stored +in a `dict[str, lambda]`, e.g. `ATTRS = {"srcs": lambda: LabelList(...)}`. +::: + +Example usage: + +``` + +load(":rule_builders.bzl", "ruleb") +load(":attr_builders.bzl", "attrb") + +# File: foo_binary.bzl +_COMMON_ATTRS = { + "srcs": lambda: attrb.LabelList(...), +} + +def create_foo_binary_builder(): + foo = ruleb.Rule( + executable = True, + ) + foo.implementation.set(_foo_binary_impl) + foo.attrs.update(COMMON_ATTRS) + return foo + +def create_foo_test_builder(): + foo = create_foo_binary_build() + + binary_impl = foo.implementation.get() + def foo_test_impl(ctx): + binary_impl(ctx) + ... + + foo.implementation.set(foo_test_impl) + foo.executable.set(False) + foo.test.test(True) + foo.attrs.update( + _coverage = attrb.Label(default="//:coverage") + ) + return foo + +foo_binary = create_foo_binary_builder().build() +foo_test = create_foo_test_builder().build() + +# File: custom_foo_binary.bzl +load(":foo_binary.bzl", "create_foo_binary_builder") + +def create_custom_foo_binary(): + r = create_foo_binary_builder() + r.attrs["srcs"].default.append("whatever.txt") + return r.build() + +custom_foo_binary = create_custom_foo_binary() +``` +""" + +load("@bazel_skylib//lib:types.bzl", "types") +load( + ":builders_util.bzl", + "kwargs_getter", + "kwargs_getter_doc", + "kwargs_set_default_dict", + "kwargs_set_default_doc", + "kwargs_set_default_ignore_none", + "kwargs_set_default_list", + "kwargs_setter", + "kwargs_setter_doc", + "list_add_unique", +) + +# Various string constants for kwarg key names used across two or more +# functions, or in contexts with optional lookups (e.g. dict.dict, key in dict). +# Constants are used to reduce the chance of typos. +# NOTE: These keys are often part of function signature via `**kwargs`; they +# are not simply internal names. +_ATTRS = "attrs" +_CFG = "cfg" +_EXEC_COMPATIBLE_WITH = "exec_compatible_with" +_EXEC_GROUPS = "exec_groups" +_IMPLEMENTATION = "implementation" +_INPUTS = "inputs" +_OUTPUTS = "outputs" +_TOOLCHAINS = "toolchains" + +def _is_builder(obj): + return hasattr(obj, "build") + +def _ExecGroup_typedef(): + """Builder for {external:bzl:obj}`exec_group` + + :::{function} toolchains() -> list[ToolchainType] + ::: + + :::{function} exec_compatible_with() -> list[str | Label] + ::: + + :::{include} /_includes/field_kwargs_doc.md + ::: + """ + +def _ExecGroup_new(**kwargs): + """Creates a builder for {external:bzl:obj}`exec_group`. + + Args: + **kwargs: Same as {external:bzl:obj}`exec_group` + + Returns: + {type}`ExecGroup` + """ + kwargs_set_default_list(kwargs, _TOOLCHAINS) + kwargs_set_default_list(kwargs, _EXEC_COMPATIBLE_WITH) + + for i, value in enumerate(kwargs[_TOOLCHAINS]): + kwargs[_TOOLCHAINS][i] = _ToolchainType_maybe_from(value) + + # buildifier: disable=uninitialized + self = struct( + toolchains = kwargs_getter(kwargs, _TOOLCHAINS), + exec_compatible_with = kwargs_getter(kwargs, _EXEC_COMPATIBLE_WITH), + kwargs = kwargs, + build = lambda: _ExecGroup_build(self), + ) + return self + +def _ExecGroup_maybe_from(obj): + if types.is_function(obj): + return obj() + else: + return obj + +def _ExecGroup_build(self): + kwargs = dict(self.kwargs) + if kwargs.get(_TOOLCHAINS): + kwargs[_TOOLCHAINS] = [ + v.build() if _is_builder(v) else v + for v in kwargs[_TOOLCHAINS] + ] + if kwargs.get(_EXEC_COMPATIBLE_WITH): + kwargs[_EXEC_COMPATIBLE_WITH] = [ + v.build() if _is_builder(v) else v + for v in kwargs[_EXEC_COMPATIBLE_WITH] + ] + return exec_group(**kwargs) + +# buildifier: disable=name-conventions +ExecGroup = struct( + TYPEDEF = _ExecGroup_typedef, + new = _ExecGroup_new, + build = _ExecGroup_build, +) + +def _ToolchainType_typedef(): + """Builder for {obj}`config_common.toolchain_type()` + + :::{include} /_includes/field_kwargs_doc.md + ::: + + :::{function} mandatory() -> bool + ::: + + :::{function} name() -> str | Label | None + ::: + + :::{function} set_name(v: str) + ::: + + :::{function} set_mandatory(v: bool) + ::: + """ + +def _ToolchainType_new(name = None, **kwargs): + """Creates a builder for `config_common.toolchain_type`. + + Args: + name: {type}`str | Label | None` the toolchain type target. + **kwargs: Same as {obj}`config_common.toolchain_type` + + Returns: + {type}`ToolchainType` + """ + kwargs["name"] = name + kwargs_set_default_ignore_none(kwargs, "mandatory", True) + + # buildifier: disable=uninitialized + self = struct( + # keep sorted + build = lambda: _ToolchainType_build(self), + kwargs = kwargs, + mandatory = kwargs_getter(kwargs, "mandatory"), + name = kwargs_getter(kwargs, "name"), + set_mandatory = kwargs_setter(kwargs, "mandatory"), + set_name = kwargs_setter(kwargs, "name"), + ) + return self + +def _ToolchainType_maybe_from(obj): + if types.is_string(obj) or type(obj) == "Label": + return ToolchainType.new(name = obj) + elif types.is_function(obj): + # A lambda to create a builder + return obj() + else: + # For lack of another option, return it as-is. + # Presumably it's already a builder or other valid object. + return obj + +def _ToolchainType_build(self): + """Builds a `config_common.toolchain_type` + + Args: + self: implicitly added + + Returns: + {type}`config_common.toolchain_type` + """ + kwargs = dict(self.kwargs) + name = kwargs.pop("name") # Name must be positional + return config_common.toolchain_type(name, **kwargs) + +# buildifier: disable=name-conventions +ToolchainType = struct( + TYPEDEF = _ToolchainType_typedef, + new = _ToolchainType_new, + build = _ToolchainType_build, +) + +def _RuleCfg_typedef(): + """Wrapper for `rule.cfg` arg. + + :::{function} implementation() -> str | callable | None | config.target | config.none + ::: + + ::::{function} inputs() -> list[Label] + + :::{seealso} + The {obj}`add_inputs()` and {obj}`update_inputs` methods for adding unique + values. + ::: + :::: + + :::{function} outputs() -> list[Label] + + :::{seealso} + The {obj}`add_outputs()` and {obj}`update_outputs` methods for adding unique + values. + ::: + ::: + + :::{function} set_implementation(v: str | callable | None | config.target | config.none) + + The string values "target" and "none" are supported. + ::: + """ + +def _RuleCfg_new(rule_cfg_arg): + """Creates a builder for the `rule.cfg` arg. + + Args: + rule_cfg_arg: {type}`str | dict | None` The `cfg` arg passed to Rule(). + + Returns: + {type}`RuleCfg` + """ + state = {} + if types.is_dict(rule_cfg_arg): + state.update(rule_cfg_arg) + else: + # Assume its a string, config.target, config.none, or other + # valid object. + state[_IMPLEMENTATION] = rule_cfg_arg + + kwargs_set_default_list(state, _INPUTS) + kwargs_set_default_list(state, _OUTPUTS) + + # buildifier: disable=uninitialized + self = struct( + add_inputs = lambda *a, **k: _RuleCfg_add_inputs(self, *a, **k), + add_outputs = lambda *a, **k: _RuleCfg_add_outputs(self, *a, **k), + _state = state, + build = lambda: _RuleCfg_build(self), + implementation = kwargs_getter(state, _IMPLEMENTATION), + inputs = kwargs_getter(state, _INPUTS), + outputs = kwargs_getter(state, _OUTPUTS), + set_implementation = kwargs_setter(state, _IMPLEMENTATION), + update_inputs = lambda *a, **k: _RuleCfg_update_inputs(self, *a, **k), + update_outputs = lambda *a, **k: _RuleCfg_update_outputs(self, *a, **k), + ) + return self + +def _RuleCfg_add_inputs(self, *inputs): + """Adds an input to the list of inputs, if not present already. + + :::{seealso} + The {obj}`update_inputs()` method for adding a collection of + values. + ::: + + Args: + self: implicitly arg. + *inputs: {type}`Label` the inputs to add. Note that a `Label`, + not `str`, should be passed to ensure different apparent labels + can be properly de-duplicated. + """ + self.update_inputs(inputs) + +def _RuleCfg_add_outputs(self, *outputs): + """Adds an output to the list of outputs, if not present already. + + :::{seealso} + The {obj}`update_outputs()` method for adding a collection of + values. + ::: + + Args: + self: implicitly arg. + *outputs: {type}`Label` the outputs to add. Note that a `Label`, + not `str`, should be passed to ensure different apparent labels + can be properly de-duplicated. + """ + self.update_outputs(outputs) + +def _RuleCfg_build(self): + """Builds the rule cfg into the value rule.cfg arg value. + + Returns: + {type}`transition` the transition object to apply to the rule. + """ + impl = self._state[_IMPLEMENTATION] + if impl == "target" or impl == None: + # config.target is Bazel 8+ + if hasattr(config, "target"): + return config.target() + else: + return None + elif impl == "none": + return config.none() + elif types.is_function(impl): + return transition( + implementation = impl, + # Transitions only accept unique lists of strings. + inputs = {str(v): None for v in self._state[_INPUTS]}.keys(), + outputs = {str(v): None for v in self._state[_OUTPUTS]}.keys(), + ) + else: + # Assume its valid. Probably an `config.XXX` object or manually + # set transition object. + return impl + +def _RuleCfg_update_inputs(self, *others): + """Add a collection of values to inputs. + + Args: + self: implicitly added + *others: {type}`collection[Label]` collection of labels to add to + inputs. Only values not already present are added. Note that a + `Label`, not `str`, should be passed to ensure different apparent + labels can be properly de-duplicated. + """ + list_add_unique(self._state[_INPUTS], others) + +def _RuleCfg_update_outputs(self, *others): + """Add a collection of values to outputs. + + Args: + self: implicitly added + *others: {type}`collection[Label]` collection of labels to add to + outputs. Only values not already present are added. Note that a + `Label`, not `str`, should be passed to ensure different apparent + labels can be properly de-duplicated. + """ + list_add_unique(self._state[_OUTPUTS], others) + +# buildifier: disable=name-conventions +RuleCfg = struct( + TYPEDEF = _RuleCfg_typedef, + new = _RuleCfg_new, + # keep sorted + add_inputs = _RuleCfg_add_inputs, + add_outputs = _RuleCfg_add_outputs, + build = _RuleCfg_build, + update_inputs = _RuleCfg_update_inputs, + update_outputs = _RuleCfg_update_outputs, +) + +def _Rule_typedef(): + """A builder to accumulate state for constructing a `rule` object. + + :::{field} attrs + :type: AttrsDict + ::: + + :::{field} cfg + :type: RuleCfg + ::: + + :::{function} doc() -> str + ::: + + :::{function} exec_groups() -> dict[str, ExecGroup] + ::: + + :::{function} executable() -> bool + ::: + + :::{include} /_includes/field_kwargs_doc.md + ::: + + :::{function} fragments() -> list[str] + ::: + + :::{function} implementation() -> callable | None + ::: + + :::{function} provides() -> list[provider | list[provider]] + ::: + + :::{function} set_doc(v: str) + ::: + + :::{function} set_executable(v: bool) + ::: + + :::{function} set_implementation(v: callable) + ::: + + :::{function} set_test(v: bool) + ::: + + :::{function} test() -> bool + ::: + + :::{function} toolchains() -> list[ToolchainType] + ::: + """ + +def _Rule_new(**kwargs): + """Builder for creating rules. + + Args: + **kwargs: The same as the `rule()` function, but using builders or + dicts to specify sub-objects instead of the immutable Bazel + objects. + """ + kwargs.setdefault(_IMPLEMENTATION, None) + kwargs_set_default_doc(kwargs) + kwargs_set_default_dict(kwargs, _EXEC_GROUPS) + kwargs_set_default_ignore_none(kwargs, "executable", False) + kwargs_set_default_list(kwargs, "fragments") + kwargs_set_default_list(kwargs, "provides") + kwargs_set_default_ignore_none(kwargs, "test", False) + kwargs_set_default_list(kwargs, _TOOLCHAINS) + + for name, value in kwargs[_EXEC_GROUPS].items(): + kwargs[_EXEC_GROUPS][name] = _ExecGroup_maybe_from(value) + + for i, value in enumerate(kwargs[_TOOLCHAINS]): + kwargs[_TOOLCHAINS][i] = _ToolchainType_maybe_from(value) + + # buildifier: disable=uninitialized + self = struct( + attrs = _AttrsDict_new(kwargs.pop(_ATTRS, None)), + build = lambda *a, **k: _Rule_build(self, *a, **k), + cfg = _RuleCfg_new(kwargs.pop(_CFG, None)), + doc = kwargs_getter_doc(kwargs), + exec_groups = kwargs_getter(kwargs, _EXEC_GROUPS), + executable = kwargs_getter(kwargs, "executable"), + fragments = kwargs_getter(kwargs, "fragments"), + implementation = kwargs_getter(kwargs, _IMPLEMENTATION), + kwargs = kwargs, + provides = kwargs_getter(kwargs, "provides"), + set_doc = kwargs_setter_doc(kwargs), + set_executable = kwargs_setter(kwargs, "executable"), + set_implementation = kwargs_setter(kwargs, _IMPLEMENTATION), + set_test = kwargs_setter(kwargs, "test"), + test = kwargs_getter(kwargs, "test"), + to_kwargs = lambda: _Rule_to_kwargs(self), + toolchains = kwargs_getter(kwargs, _TOOLCHAINS), + ) + return self + +def _Rule_build(self, debug = ""): + """Builds a `rule` object + + Args: + self: implicitly added + debug: {type}`str` If set, prints the args used to create the rule. + + Returns: + {type}`rule` + """ + kwargs = self.to_kwargs() + if debug: + lines = ["=" * 80, "rule kwargs: {}:".format(debug)] + for k, v in sorted(kwargs.items()): + if types.is_dict(v): + lines.append(" %s={" % k) + for k2, v2 in sorted(v.items()): + lines.append(" {}: {}".format(k2, v2)) + lines.append(" }") + elif types.is_list(v): + lines.append(" {}=[".format(k)) + for i, v2 in enumerate(v): + lines.append(" [{}] {}".format(i, v2)) + lines.append(" ]") + else: + lines.append(" {}={}".format(k, v)) + print("\n".join(lines)) # buildifier: disable=print + return rule(**kwargs) + +def _Rule_to_kwargs(self): + """Builds the arguments for calling `rule()`. + + This is added as an escape hatch to construct the final values `rule()` + kwarg values in case callers want to manually change them. + + Args: + self: implicitly added. + + Returns: + {type}`dict` + """ + kwargs = dict(self.kwargs) + if _EXEC_GROUPS in kwargs: + kwargs[_EXEC_GROUPS] = { + k: v.build() if _is_builder(v) else v + for k, v in kwargs[_EXEC_GROUPS].items() + } + if _TOOLCHAINS in kwargs: + kwargs[_TOOLCHAINS] = [ + v.build() if _is_builder(v) else v + for v in kwargs[_TOOLCHAINS] + ] + if _ATTRS not in kwargs: + kwargs[_ATTRS] = self.attrs.build() + if _CFG not in kwargs: + kwargs[_CFG] = self.cfg.build() + return kwargs + +# buildifier: disable=name-conventions +Rule = struct( + TYPEDEF = _Rule_typedef, + new = _Rule_new, + build = _Rule_build, + to_kwargs = _Rule_to_kwargs, +) + +def _AttrsDict_typedef(): + """Builder for the dictionary of rule attributes. + + :::{field} map + :type: dict[str, AttributeBuilder] + + The underlying dict of attributes. Directly accessible so that regular + dict operations (e.g. `x in y`) can be performed, if necessary. + ::: + + :::{function} get(key, default=None) + Get an entry from the dict. Convenience wrapper for `.map.get(...)` + ::: + + :::{function} items() -> list[tuple[str, object]] + Returns a list of key-value tuples. Convenience wrapper for `.map.items()` + ::: + + :::{function} pop(key, default) -> object + Removes a key from the attr dict + ::: + """ + +def _AttrsDict_new(initial): + """Creates a builder for the `rule.attrs` dict. + + Args: + initial: {type}`dict[str, callable | AttributeBuilder] | None` dict of + initial values to populate the attributes dict with. + + Returns: + {type}`AttrsDict` + """ + + # buildifier: disable=uninitialized + self = struct( + # keep sorted + build = lambda: _AttrsDict_build(self), + get = lambda *a, **k: self.map.get(*a, **k), + items = lambda: self.map.items(), + map = {}, + put = lambda key, value: _AttrsDict_put(self, key, value), + update = lambda *a, **k: _AttrsDict_update(self, *a, **k), + pop = lambda *a, **k: self.map.pop(*a, **k), + ) + if initial: + _AttrsDict_update(self, initial) + return self + +def _AttrsDict_put(self, name, value): + """Sets a value in the attrs dict. + + Args: + self: implicitly added + name: {type}`str` the attribute name to set in the dict + value: {type}`AttributeBuilder | callable` the value for the + attribute. If a callable, then it is treated as an + attribute builder factory (no-arg callable that returns an + attribute builder) and is called immediately. + """ + if types.is_function(value): + # Convert factory function to builder + value = value() + self.map[name] = value + +def _AttrsDict_update(self, other): + """Merge `other` into this object. + + Args: + self: implicitly added + other: {type}`dict[str, callable | AttributeBuilder]` the values to + merge into this object. If the value a function, it is called + with no args and expected to return an attribute builder. This + allows defining dicts of common attributes (where the values are + functions that create a builder) and merge them into the rule. + """ + for k, v in other.items(): + # Handle factory functions that create builders + if types.is_function(v): + self.map[k] = v() + else: + self.map[k] = v + +def _AttrsDict_build(self): + """Build an attribute dict for passing to `rule()`. + + Returns: + {type}`dict[str, attribute]` where the values are `attr.XXX` objects + """ + attrs = {} + for k, v in self.map.items(): + attrs[k] = v.build() if _is_builder(v) else v + return attrs + +# buildifier: disable=name-conventions +AttrsDict = struct( + TYPEDEF = _AttrsDict_typedef, + new = _AttrsDict_new, + update = _AttrsDict_update, + build = _AttrsDict_build, +) + +ruleb = struct( + Rule = _Rule_new, + ToolchainType = _ToolchainType_new, + ExecGroup = _ExecGroup_new, +) diff --git a/sphinxdocs/inventories/bazel_inventory.txt b/sphinxdocs/inventories/bazel_inventory.txt index 969c772386..dc11f02b5b 100644 --- a/sphinxdocs/inventories/bazel_inventory.txt +++ b/sphinxdocs/inventories/bazel_inventory.txt @@ -15,10 +15,17 @@ Target bzl:type 1 rules/lib/builtins/Target - ToolchainInfo bzl:type 1 rules/lib/providers/ToolchainInfo.html - attr.bool bzl:type 1 rules/lib/toplevel/attr#bool - attr.int bzl:type 1 rules/lib/toplevel/attr#int - +attr.int_list bzl:type 1 rules/lib/toplevel/attr#int_list - attr.label bzl:type 1 rules/lib/toplevel/attr#label - +attr.label_keyed_string_dict bzl:type 1 rules/lib/toplevel/attr#label_keyed_string_dict - attr.label_list bzl:type 1 rules/lib/toplevel/attr#label_list - +attr.output bzl:type 1 rules/lib/toplevel/attr#output - +attr.output_list bzl:type 1 rules/lib/toplevel/attr#output_list - attr.string bzl:type 1 rules/lib/toplevel/attr#string - +attr.string_dict bzl:type 1 rules/lib/toplevel/attr#string_dict - +attr.string_keyed_label_dict bzl:type 1 rules/lib/toplevel/attr#string_keyed_label_dict - attr.string_list bzl:type 1 rules/lib/toplevel/attr#string_list - +attr.string_list_dict bzl:type 1 rules/lib/toplevel/attr#string_list_dict - bool bzl:type 1 rules/lib/bool - callable bzl:type 1 rules/lib/core/function - config_common.FeatureFlagInfo bzl:type 1 rules/lib/toplevel/config_common#FeatureFlagInfo - @@ -60,6 +67,7 @@ ctx.workspace_name bzl:obj 1 rules/lib/builtins/ctx#workspace_name - depset bzl:type 1 rules/lib/depset - dict bzl:type 1 rules/lib/dict - exec_compatible_with bzl:attr 1 reference/be/common-definitions#common.exec_compatible_with - +exec_group bzl:function 1 rules/lib/globals/bzl#exec_group - int bzl:type 1 rules/lib/int - label bzl:type 1 concepts/labels - list bzl:type 1 rules/lib/list - diff --git a/tests/builders/BUILD.bazel b/tests/builders/BUILD.bazel index 3ad0c3e80c..f963cb0131 100644 --- a/tests/builders/BUILD.bazel +++ b/tests/builders/BUILD.bazel @@ -12,6 +12,42 @@ # See the License for the specific language governing permissions and # limitations under the License. +load(":attr_builders_tests.bzl", "attr_builders_test_suite") load(":builders_tests.bzl", "builders_test_suite") +load(":rule_builders_tests.bzl", "rule_builders_test_suite") builders_test_suite(name = "builders_test_suite") + +rule_builders_test_suite(name = "rule_builders_test_suite") + +attr_builders_test_suite(name = "attr_builders_test_suite") + +toolchain_type(name = "tct_1") + +toolchain_type(name = "tct_2") + +toolchain_type(name = "tct_3") + +toolchain_type(name = "tct_4") + +toolchain_type(name = "tct_5") + +filegroup(name = "empty") + +toolchain( + name = "tct_3_toolchain", + toolchain = "//tests/support/empty_toolchain:empty", + toolchain_type = "//tests/builders:tct_3", +) + +toolchain( + name = "tct_4_toolchain", + toolchain = "//tests/support/empty_toolchain:empty", + toolchain_type = ":tct_4", +) + +toolchain( + name = "tct_5_toolchain", + toolchain = "//tests/support/empty_toolchain:empty", + toolchain_type = ":tct_5", +) diff --git a/tests/builders/attr_builders_tests.bzl b/tests/builders/attr_builders_tests.bzl new file mode 100644 index 0000000000..58557cd633 --- /dev/null +++ b/tests/builders/attr_builders_tests.bzl @@ -0,0 +1,468 @@ +# Copyright 2025 The Bazel Authors. All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +"""Tests for attr_builders.""" + +load("@rules_testing//lib:analysis_test.bzl", "analysis_test") +load("@rules_testing//lib:test_suite.bzl", "test_suite") +load("@rules_testing//lib:truth.bzl", "truth") +load("//python/private:attr_builders.bzl", "attrb") # buildifier: disable=bzl-visibility + +def _expect_cfg_defaults(expect, cfg): + expect.where(expr = "cfg.outputs").that_collection(cfg.outputs()).contains_exactly([]) + expect.where(expr = "cfg.inputs").that_collection(cfg.inputs()).contains_exactly([]) + expect.where(expr = "cfg.implementation").that_bool(cfg.implementation()).equals(None) + expect.where(expr = "cfg.target").that_bool(cfg.target()).equals(True) + expect.where(expr = "cfg.exec_group").that_str(cfg.exec_group()).equals(None) + expect.where(expr = "cfg.which_cfg").that_str(cfg.which_cfg()).equals("target") + +_some_aspect = aspect(implementation = lambda target, ctx: None) + +_tests = [] + +def _report_failures(name, env): + failures = env.failures + + def _report_failures_impl(env, target): + _ = target # @unused + env._failures.extend(failures) + + analysis_test( + name = name, + target = "//python:none", + impl = _report_failures_impl, + ) + +# Calling attr.xxx() outside of the loading phase is an error, but rules_testing +# creates the expect/truth helpers during the analysis phase. To make the truth +# helpers available during the loading phase, fake out the ctx just enough to +# satify rules_testing. +def _loading_phase_expect(test_name): + env = struct( + ctx = struct( + workspace_name = "bogus", + label = Label(test_name), + attr = struct( + _impl_name = test_name, + ), + ), + failures = [], + ) + return env, truth.expect(env) + +def _expect_builds(expect, builder, attribute_type): + expect.that_str(str(builder.build())).contains(attribute_type) + +def _test_cfg_arg(name): + env, _ = _loading_phase_expect(name) + + def build_cfg(cfg): + attrb.Label(cfg = cfg).build() + + build_cfg(None) + build_cfg("target") + build_cfg("exec") + build_cfg(dict(exec_group = "eg")) + build_cfg(dict(implementation = (lambda settings, attr: None))) + build_cfg(config.exec()) + build_cfg(transition( + implementation = (lambda settings, attr: None), + inputs = [], + outputs = [], + )) + + # config.target is Bazel 8+ + if hasattr(config, "target"): + build_cfg(config.target()) + + # config.none is Bazel 8+ + if hasattr(config, "none"): + build_cfg("none") + build_cfg(config.none()) + + _report_failures(name, env) + +_tests.append(_test_cfg_arg) + +def _test_bool(name): + env, expect = _loading_phase_expect(name) + subject = attrb.Bool() + expect.that_str(subject.doc()).equals("") + expect.that_bool(subject.default()).equals(False) + expect.that_bool(subject.mandatory()).equals(False) + _expect_builds(expect, subject, "attr.bool") + + subject.set_default(True) + subject.set_mandatory(True) + subject.set_doc("doc") + + expect.that_str(subject.doc()).equals("doc") + expect.that_bool(subject.default()).equals(True) + expect.that_bool(subject.mandatory()).equals(True) + _expect_builds(expect, subject, "attr.bool") + + _report_failures(name, env) + +_tests.append(_test_bool) + +def _test_int(name): + env, expect = _loading_phase_expect(name) + + subject = attrb.Int() + expect.that_int(subject.default()).equals(0) + expect.that_str(subject.doc()).equals("") + expect.that_bool(subject.mandatory()).equals(False) + expect.that_collection(subject.values()).contains_exactly([]) + _expect_builds(expect, subject, "attr.int") + + subject.set_default(42) + subject.set_doc("doc") + subject.set_mandatory(True) + subject.values().append(42) + + expect.that_int(subject.default()).equals(42) + expect.that_str(subject.doc()).equals("doc") + expect.that_bool(subject.mandatory()).equals(True) + expect.that_collection(subject.values()).contains_exactly([42]) + _expect_builds(expect, subject, "attr.int") + + _report_failures(name, env) + +_tests.append(_test_int) + +def _test_int_list(name): + env, expect = _loading_phase_expect(name) + + subject = attrb.IntList() + expect.that_bool(subject.allow_empty()).equals(True) + expect.that_collection(subject.default()).contains_exactly([]) + expect.that_str(subject.doc()).equals("") + expect.that_bool(subject.mandatory()).equals(False) + _expect_builds(expect, subject, "attr.int_list") + + subject.default().append(99) + subject.set_doc("doc") + subject.set_mandatory(True) + + expect.that_collection(subject.default()).contains_exactly([99]) + expect.that_str(subject.doc()).equals("doc") + expect.that_bool(subject.mandatory()).equals(True) + _expect_builds(expect, subject, "attr.int_list") + + _report_failures(name, env) + +_tests.append(_test_int_list) + +def _test_label(name): + env, expect = _loading_phase_expect(name) + + subject = attrb.Label() + + expect.that_str(subject.default()).equals(None) + expect.that_str(subject.doc()).equals("") + expect.that_bool(subject.mandatory()).equals(False) + expect.that_bool(subject.executable()).equals(False) + expect.that_bool(subject.allow_files()).equals(None) + expect.that_bool(subject.allow_single_file()).equals(None) + expect.that_collection(subject.providers()).contains_exactly([]) + expect.that_collection(subject.aspects()).contains_exactly([]) + _expect_cfg_defaults(expect, subject.cfg) + _expect_builds(expect, subject, "attr.label") + + subject.set_default("//foo:bar") + subject.set_doc("doc") + subject.set_mandatory(True) + subject.set_executable(True) + subject.add_allow_files(".txt") + subject.cfg.set_target() + subject.providers().append("provider") + subject.aspects().append(_some_aspect) + subject.cfg.outputs().append(Label("//some:output")) + subject.cfg.inputs().append(Label("//some:input")) + impl = lambda: None + subject.cfg.set_implementation(impl) + + expect.that_str(subject.default()).equals("//foo:bar") + expect.that_str(subject.doc()).equals("doc") + expect.that_bool(subject.mandatory()).equals(True) + expect.that_bool(subject.executable()).equals(True) + expect.that_collection(subject.allow_files()).contains_exactly([".txt"]) + expect.that_bool(subject.allow_single_file()).equals(None) + expect.that_collection(subject.providers()).contains_exactly(["provider"]) + expect.that_collection(subject.aspects()).contains_exactly([_some_aspect]) + expect.that_collection(subject.cfg.outputs()).contains_exactly([Label("//some:output")]) + expect.that_collection(subject.cfg.inputs()).contains_exactly([Label("//some:input")]) + expect.that_bool(subject.cfg.implementation()).equals(impl) + _expect_builds(expect, subject, "attr.label") + + _report_failures(name, env) + +_tests.append(_test_label) + +def _test_label_keyed_string_dict(name): + env, expect = _loading_phase_expect(name) + + subject = attrb.LabelKeyedStringDict() + + expect.that_dict(subject.default()).contains_exactly({}) + expect.that_str(subject.doc()).equals("") + expect.that_bool(subject.mandatory()).equals(False) + expect.that_bool(subject.allow_files()).equals(False) + expect.that_collection(subject.providers()).contains_exactly([]) + expect.that_collection(subject.aspects()).contains_exactly([]) + _expect_cfg_defaults(expect, subject.cfg) + _expect_builds(expect, subject, "attr.label_keyed_string_dict") + + subject.default()["key"] = "//some:label" + subject.set_doc("doc") + subject.set_mandatory(True) + subject.set_allow_files(True) + subject.cfg.set_target() + subject.providers().append("provider") + subject.aspects().append(_some_aspect) + subject.cfg.outputs().append("//some:output") + subject.cfg.inputs().append("//some:input") + impl = lambda: None + subject.cfg.set_implementation(impl) + + expect.that_dict(subject.default()).contains_exactly({"key": "//some:label"}) + expect.that_str(subject.doc()).equals("doc") + expect.that_bool(subject.mandatory()).equals(True) + expect.that_bool(subject.allow_files()).equals(True) + expect.that_collection(subject.providers()).contains_exactly(["provider"]) + expect.that_collection(subject.aspects()).contains_exactly([_some_aspect]) + expect.that_collection(subject.cfg.outputs()).contains_exactly(["//some:output"]) + expect.that_collection(subject.cfg.inputs()).contains_exactly(["//some:input"]) + expect.that_bool(subject.cfg.implementation()).equals(impl) + + _expect_builds(expect, subject, "attr.label_keyed_string_dict") + + subject.add_allow_files(".txt") + expect.that_collection(subject.allow_files()).contains_exactly([".txt"]) + _expect_builds(expect, subject, "attr.label_keyed_string_dict") + + _report_failures(name, env) + +_tests.append(_test_label_keyed_string_dict) + +def _test_label_list(name): + env, expect = _loading_phase_expect(name) + + subject = attrb.LabelList() + + expect.that_collection(subject.default()).contains_exactly([]) + expect.that_str(subject.doc()).equals("") + expect.that_bool(subject.mandatory()).equals(False) + expect.that_bool(subject.allow_files()).equals(False) + expect.that_collection(subject.providers()).contains_exactly([]) + expect.that_collection(subject.aspects()).contains_exactly([]) + _expect_cfg_defaults(expect, subject.cfg) + _expect_builds(expect, subject, "attr.label_list") + + subject.default().append("//some:label") + subject.set_doc("doc") + subject.set_mandatory(True) + subject.set_allow_files([".txt"]) + subject.providers().append("provider") + subject.aspects().append(_some_aspect) + + expect.that_collection(subject.default()).contains_exactly(["//some:label"]) + expect.that_str(subject.doc()).equals("doc") + expect.that_bool(subject.mandatory()).equals(True) + expect.that_collection(subject.allow_files()).contains_exactly([".txt"]) + expect.that_collection(subject.providers()).contains_exactly(["provider"]) + expect.that_collection(subject.aspects()).contains_exactly([_some_aspect]) + + _expect_builds(expect, subject, "attr.label_list") + + _report_failures(name, env) + +_tests.append(_test_label_list) + +def _test_output(name): + env, expect = _loading_phase_expect(name) + + subject = attrb.Output() + expect.that_str(subject.doc()).equals("") + expect.that_bool(subject.mandatory()).equals(False) + _expect_builds(expect, subject, "attr.output") + + subject.set_doc("doc") + subject.set_mandatory(True) + expect.that_str(subject.doc()).equals("doc") + expect.that_bool(subject.mandatory()).equals(True) + _expect_builds(expect, subject, "attr.output") + + _report_failures(name, env) + +_tests.append(_test_output) + +def _test_output_list(name): + env, expect = _loading_phase_expect(name) + + subject = attrb.OutputList() + expect.that_bool(subject.allow_empty()).equals(True) + expect.that_str(subject.doc()).equals("") + expect.that_bool(subject.mandatory()).equals(False) + _expect_builds(expect, subject, "attr.output_list") + + subject.set_allow_empty(False) + subject.set_doc("doc") + subject.set_mandatory(True) + expect.that_bool(subject.allow_empty()).equals(False) + expect.that_str(subject.doc()).equals("doc") + expect.that_bool(subject.mandatory()).equals(True) + _expect_builds(expect, subject, "attr.output_list") + + _report_failures(name, env) + +_tests.append(_test_output_list) + +def _test_string(name): + env, expect = _loading_phase_expect(name) + + subject = attrb.String() + expect.that_str(subject.default()).equals("") + expect.that_str(subject.doc()).equals("") + expect.that_bool(subject.mandatory()).equals(False) + expect.that_collection(subject.values()).contains_exactly([]) + _expect_builds(expect, subject, "attr.string") + + subject.set_doc("doc") + subject.set_mandatory(True) + subject.values().append("green") + expect.that_str(subject.doc()).equals("doc") + expect.that_bool(subject.mandatory()).equals(True) + expect.that_collection(subject.values()).contains_exactly(["green"]) + _expect_builds(expect, subject, "attr.string") + + _report_failures(name, env) + +_tests.append(_test_string) + +def _test_string_dict(name): + env, expect = _loading_phase_expect(name) + + subject = attrb.StringDict() + + expect.that_dict(subject.default()).contains_exactly({}) + expect.that_str(subject.doc()).equals("") + expect.that_bool(subject.mandatory()).equals(False) + expect.that_bool(subject.allow_empty()).equals(True) + _expect_builds(expect, subject, "attr.string_dict") + + subject.default()["key"] = "value" + subject.set_doc("doc") + subject.set_mandatory(True) + subject.set_allow_empty(False) + + expect.that_dict(subject.default()).contains_exactly({"key": "value"}) + expect.that_str(subject.doc()).equals("doc") + expect.that_bool(subject.mandatory()).equals(True) + expect.that_bool(subject.allow_empty()).equals(False) + _expect_builds(expect, subject, "attr.string_dict") + + _report_failures(name, env) + +_tests.append(_test_string_dict) + +def _test_string_keyed_label_dict(name): + env, expect = _loading_phase_expect(name) + + subject = attrb.StringKeyedLabelDict() + + expect.that_dict(subject.default()).contains_exactly({}) + expect.that_str(subject.doc()).equals("") + expect.that_bool(subject.mandatory()).equals(False) + expect.that_bool(subject.allow_files()).equals(False) + expect.that_collection(subject.providers()).contains_exactly([]) + expect.that_collection(subject.aspects()).contains_exactly([]) + _expect_cfg_defaults(expect, subject.cfg) + _expect_builds(expect, subject, "attr.string_keyed_label_dict") + + subject.default()["key"] = "//some:label" + subject.set_doc("doc") + subject.set_mandatory(True) + subject.set_allow_files([".txt"]) + subject.providers().append("provider") + subject.aspects().append(_some_aspect) + + expect.that_dict(subject.default()).contains_exactly({"key": "//some:label"}) + expect.that_str(subject.doc()).equals("doc") + expect.that_bool(subject.mandatory()).equals(True) + expect.that_collection(subject.allow_files()).contains_exactly([".txt"]) + expect.that_collection(subject.providers()).contains_exactly(["provider"]) + expect.that_collection(subject.aspects()).contains_exactly([_some_aspect]) + + _expect_builds(expect, subject, "attr.string_keyed_label_dict") + + _report_failures(name, env) + +_tests.append(_test_string_keyed_label_dict) + +def _test_string_list(name): + env, expect = _loading_phase_expect(name) + + subject = attrb.StringList() + + expect.that_collection(subject.default()).contains_exactly([]) + expect.that_str(subject.doc()).equals("") + expect.that_bool(subject.mandatory()).equals(False) + expect.that_bool(subject.allow_empty()).equals(True) + _expect_builds(expect, subject, "attr.string_list") + + subject.set_doc("doc") + subject.set_mandatory(True) + subject.default().append("blue") + subject.set_allow_empty(False) + expect.that_str(subject.doc()).equals("doc") + expect.that_bool(subject.mandatory()).equals(True) + expect.that_bool(subject.allow_empty()).equals(False) + expect.that_collection(subject.default()).contains_exactly(["blue"]) + _expect_builds(expect, subject, "attr.string_list") + + _report_failures(name, env) + +_tests.append(_test_string_list) + +def _test_string_list_dict(name): + env, expect = _loading_phase_expect(name) + + subject = attrb.StringListDict() + + expect.that_dict(subject.default()).contains_exactly({}) + expect.that_str(subject.doc()).equals("") + expect.that_bool(subject.mandatory()).equals(False) + expect.that_bool(subject.allow_empty()).equals(True) + _expect_builds(expect, subject, "attr.string_list_dict") + + subject.set_doc("doc") + subject.set_mandatory(True) + subject.default()["key"] = ["red"] + subject.set_allow_empty(False) + expect.that_str(subject.doc()).equals("doc") + expect.that_bool(subject.mandatory()).equals(True) + expect.that_bool(subject.allow_empty()).equals(False) + expect.that_dict(subject.default()).contains_exactly({"key": ["red"]}) + _expect_builds(expect, subject, "attr.string_list_dict") + + _report_failures(name, env) + +_tests.append(_test_string_list_dict) + +def attr_builders_test_suite(name): + test_suite( + name = name, + tests = _tests, + ) diff --git a/tests/builders/rule_builders_tests.bzl b/tests/builders/rule_builders_tests.bzl new file mode 100644 index 0000000000..9a91ceb062 --- /dev/null +++ b/tests/builders/rule_builders_tests.bzl @@ -0,0 +1,256 @@ +# Copyright 2025 The Bazel Authors. All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +"""Tests for rule_builders.""" + +load("@rules_testing//lib:analysis_test.bzl", "analysis_test") +load("@rules_testing//lib:test_suite.bzl", "test_suite") +load("@rules_testing//lib:util.bzl", "TestingAspectInfo") +load("//python/private:attr_builders.bzl", "attrb") # buildifier: disable=bzl-visibility +load("//python/private:rule_builders.bzl", "ruleb") # buildifier: disable=bzl-visibility + +RuleInfo = provider(doc = "test provider", fields = []) + +_tests = [] # analysis-phase tests +_basic_tests = [] # loading-phase tests + +fruit = ruleb.Rule( + implementation = lambda ctx: [RuleInfo()], + attrs = { + "color": attrb.String(default = "yellow"), + "fertilizers": attrb.LabelList( + allow_files = True, + ), + "flavors": attrb.StringList(), + "nope": attr.label( + # config.none is Bazel 8+ + cfg = config.none() if hasattr(config, "none") else None, + ), + "organic": lambda: attrb.Bool(), + "origin": lambda: attrb.Label(), + "size": lambda: attrb.Int(default = 10), + }, +).build() + +def _test_fruit_rule(name): + fruit( + name = name + "_subject", + flavors = ["spicy", "sweet"], + organic = True, + size = 5, + origin = "//python:none", + fertilizers = [ + "nitrogen.txt", + "phosphorus.txt", + ], + ) + + analysis_test( + name = name, + target = name + "_subject", + impl = _test_fruit_rule_impl, + ) + +def _test_fruit_rule_impl(env, target): + attrs = target[TestingAspectInfo].attrs + env.expect.that_str(attrs.color).equals("yellow") + env.expect.that_collection(attrs.flavors).contains_exactly(["spicy", "sweet"]) + env.expect.that_bool(attrs.organic).equals(True) + env.expect.that_int(attrs.size).equals(5) + + # //python:none is an alias to //python/private:sentinel; we see the + # resolved value, not the intermediate alias + env.expect.that_target(attrs.origin).label().equals(Label("//python/private:sentinel")) + + env.expect.that_collection(attrs.fertilizers).transform( + desc = "target.label", + map_each = lambda t: t.label, + ).contains_exactly([ + Label(":nitrogen.txt"), + Label(":phosphorus.txt"), + ]) + +_tests.append(_test_fruit_rule) + +# NOTE: `Rule.build()` can't be called because it's not during the top-level +# bzl evaluation. +def _test_rule_api(env): + subject = ruleb.Rule() + expect = env.expect + + expect.that_dict(subject.attrs.map).contains_exactly({}) + expect.that_collection(subject.cfg.outputs()).contains_exactly([]) + expect.that_collection(subject.cfg.inputs()).contains_exactly([]) + expect.that_bool(subject.cfg.implementation()).equals(None) + expect.that_str(subject.doc()).equals("") + expect.that_dict(subject.exec_groups()).contains_exactly({}) + expect.that_bool(subject.executable()).equals(False) + expect.that_collection(subject.fragments()).contains_exactly([]) + expect.that_bool(subject.implementation()).equals(None) + expect.that_collection(subject.provides()).contains_exactly([]) + expect.that_bool(subject.test()).equals(False) + expect.that_collection(subject.toolchains()).contains_exactly([]) + + subject.attrs.update({ + "builder": attrb.String(), + "factory": lambda: attrb.String(), + }) + subject.attrs.put("put_factory", lambda: attrb.Int()) + subject.attrs.put("put_builder", attrb.Int()) + + expect.that_dict(subject.attrs.map).keys().contains_exactly([ + "factory", + "builder", + "put_factory", + "put_builder", + ]) + expect.that_collection(subject.attrs.map.values()).transform( + desc = "type() of attr value", + map_each = type, + ).contains_exactly(["struct", "struct", "struct", "struct"]) + + subject.set_doc("doc") + expect.that_str(subject.doc()).equals("doc") + + subject.exec_groups()["eg"] = ruleb.ExecGroup() + expect.that_dict(subject.exec_groups()).keys().contains_exactly(["eg"]) + + subject.set_executable(True) + expect.that_bool(subject.executable()).equals(True) + + subject.fragments().append("frag") + expect.that_collection(subject.fragments()).contains_exactly(["frag"]) + + impl = lambda: None + subject.set_implementation(impl) + expect.that_bool(subject.implementation()).equals(impl) + + subject.provides().append(RuleInfo) + expect.that_collection(subject.provides()).contains_exactly([RuleInfo]) + + subject.set_test(True) + expect.that_bool(subject.test()).equals(True) + + subject.toolchains().append(ruleb.ToolchainType()) + expect.that_collection(subject.toolchains()).has_size(1) + + expect.that_collection(subject.cfg.outputs()).contains_exactly([]) + expect.that_collection(subject.cfg.inputs()).contains_exactly([]) + expect.that_bool(subject.cfg.implementation()).equals(None) + + subject.cfg.set_implementation(impl) + expect.that_bool(subject.cfg.implementation()).equals(impl) + subject.cfg.add_inputs(Label("//some:input")) + expect.that_collection(subject.cfg.inputs()).contains_exactly([ + Label("//some:input"), + ]) + subject.cfg.add_outputs(Label("//some:output")) + expect.that_collection(subject.cfg.outputs()).contains_exactly([ + Label("//some:output"), + ]) + +_basic_tests.append(_test_rule_api) + +def _test_exec_group(env): + subject = ruleb.ExecGroup() + + env.expect.that_collection(subject.toolchains()).contains_exactly([]) + env.expect.that_collection(subject.exec_compatible_with()).contains_exactly([]) + env.expect.that_str(str(subject.build())).contains("ExecGroup") + + subject.toolchains().append(ruleb.ToolchainType("//python:none")) + subject.exec_compatible_with().append("//some:constraint") + env.expect.that_str(str(subject.build())).contains("ExecGroup") + +_basic_tests.append(_test_exec_group) + +def _test_toolchain_type(env): + subject = ruleb.ToolchainType() + + env.expect.that_str(subject.name()).equals(None) + env.expect.that_bool(subject.mandatory()).equals(True) + subject.set_name("//some:toolchain_type") + env.expect.that_str(str(subject.build())).contains("ToolchainType") + + subject.set_name("//some:toolchain_type") + subject.set_mandatory(False) + env.expect.that_str(subject.name()).equals("//some:toolchain_type") + env.expect.that_bool(subject.mandatory()).equals(False) + env.expect.that_str(str(subject.build())).contains("ToolchainType") + +_basic_tests.append(_test_toolchain_type) + +rule_with_toolchains = ruleb.Rule( + implementation = lambda ctx: [], + toolchains = [ + ruleb.ToolchainType("//tests/builders:tct_1", mandatory = False), + lambda: ruleb.ToolchainType("//tests/builders:tct_2", mandatory = False), + "//tests/builders:tct_3", + Label("//tests/builders:tct_4"), + ], + exec_groups = { + "eg1": ruleb.ExecGroup( + toolchains = [ + ruleb.ToolchainType("//tests/builders:tct_1", mandatory = False), + lambda: ruleb.ToolchainType("//tests/builders:tct_2", mandatory = False), + "//tests/builders:tct_3", + Label("//tests/builders:tct_4"), + ], + ), + "eg2": lambda: ruleb.ExecGroup(), + }, +).build() + +def _test_rule_with_toolchains(name): + rule_with_toolchains( + name = name + "_subject", + tags = ["manual"], # Can't be built without extra_toolchains set + ) + + analysis_test( + name = name, + impl = lambda env, target: None, + target = name + "_subject", + config_settings = { + "//command_line_option:extra_toolchains": [ + Label("//tests/builders:all"), + ], + }, + ) + +_tests.append(_test_rule_with_toolchains) + +rule_with_immutable_attrs = ruleb.Rule( + implementation = lambda ctx: [], + attrs = { + "foo": attr.string(), + }, +).build() + +def _test_rule_with_immutable_attrs(name): + rule_with_immutable_attrs(name = name + "_subject") + analysis_test( + name = name, + target = name + "_subject", + impl = lambda env, target: None, + ) + +_tests.append(_test_rule_with_immutable_attrs) + +def rule_builders_test_suite(name): + test_suite( + name = name, + basic_tests = _basic_tests, + tests = _tests, + ) diff --git a/tests/support/empty_toolchain/BUILD.bazel b/tests/support/empty_toolchain/BUILD.bazel new file mode 100644 index 0000000000..cab5f800ec --- /dev/null +++ b/tests/support/empty_toolchain/BUILD.bazel @@ -0,0 +1,3 @@ +load(":empty.bzl", "empty_toolchain") + +empty_toolchain(name = "empty") diff --git a/tests/support/empty_toolchain/empty.bzl b/tests/support/empty_toolchain/empty.bzl new file mode 100644 index 0000000000..e2839283c7 --- /dev/null +++ b/tests/support/empty_toolchain/empty.bzl @@ -0,0 +1,23 @@ +# Copyright 2025 The Bazel Authors. All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +"""Defines an empty toolchain that returns just ToolchainInfo.""" + +def _empty_toolchain_impl(ctx): + # Include the label so e.g. tests can identify what the target was. + return [platform_common.ToolchainInfo(label = ctx.label)] + +empty_toolchain = rule( + implementation = _empty_toolchain_impl, +) diff --git a/tests/support/sh_py_run_test.bzl b/tests/support/sh_py_run_test.bzl index d116f0403f..d1e3b8e9c8 100644 --- a/tests/support/sh_py_run_test.bzl +++ b/tests/support/sh_py_run_test.bzl @@ -18,6 +18,7 @@ without the overhead of a bazel-in-bazel integration test. """ load("@rules_shell//shell:sh_test.bzl", "sh_test") +load("//python/private:attr_builders.bzl", "attrb") # buildifier: disable=bzl-visibility load("//python/private:py_binary_macro.bzl", "py_binary_macro") # buildifier: disable=bzl-visibility load("//python/private:py_binary_rule.bzl", "create_binary_rule_builder") # buildifier: disable=bzl-visibility load("//python/private:py_test_macro.bzl", "py_test_macro") # buildifier: disable=bzl-visibility @@ -54,9 +55,9 @@ _RECONFIG_OUTPUTS = _RECONFIG_INPUTS + [ _RECONFIG_INHERITED_OUTPUTS = [v for v in _RECONFIG_OUTPUTS if v in _RECONFIG_INPUTS] _RECONFIG_ATTRS = { - "bootstrap_impl": attr.string(), - "build_python_zip": attr.string(default = "auto"), - "extra_toolchains": attr.string_list( + "bootstrap_impl": attrb.String(), + "build_python_zip": attrb.String(default = "auto"), + "extra_toolchains": attrb.StringList( doc = """ Value for the --extra_toolchains flag. @@ -65,18 +66,17 @@ to make the RBE presubmits happy, which disable auto-detection of a CC toolchain. """, ), - "python_src": attr.label(), - "venvs_use_declare_symlink": attr.string(), + "python_src": attrb.Label(), + "venvs_use_declare_symlink": attrb.String(), } def _create_reconfig_rule(builder): builder.attrs.update(_RECONFIG_ATTRS) - base_cfg_impl = builder.cfg.implementation.get() - builder.cfg.implementation.set(lambda *args: _perform_transition_impl(base_impl = base_cfg_impl, *args)) - builder.cfg.inputs.update(_RECONFIG_INPUTS) - builder.cfg.outputs.update(_RECONFIG_OUTPUTS) - + base_cfg_impl = builder.cfg.implementation() + builder.cfg.set_implementation(lambda *args: _perform_transition_impl(base_impl = base_cfg_impl, *args)) + builder.cfg.update_inputs(_RECONFIG_INPUTS) + builder.cfg.update_outputs(_RECONFIG_OUTPUTS) return builder.build() _py_reconfig_binary = _create_reconfig_rule(create_binary_rule_builder()) From c0b5075df0e8f61f83bf55dcbaa5c2912d248c70 Mon Sep 17 00:00:00 2001 From: Sam Schlegel Date: Fri, 14 Mar 2025 19:47:28 -0700 Subject: [PATCH 101/922] fix(gazelle): Explicitly call sys.exit in the modules_mapping generator (#2662) When running python with `-S` to disable the `site` module, `exit` isn't implicitly imported and you need to explicitly call `sys.exit` instead. Seems to be a remnant of the REPL --- gazelle/modules_mapping/generator.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/gazelle/modules_mapping/generator.py b/gazelle/modules_mapping/generator.py index d5ddca2ef2..ea11f3e236 100644 --- a/gazelle/modules_mapping/generator.py +++ b/gazelle/modules_mapping/generator.py @@ -164,4 +164,4 @@ def data_has_purelib_or_platlib(path): generator = Generator( sys.stderr, args.output_file, args.exclude_patterns, args.include_stub_packages ) - exit(generator.run(args.wheels)) + sys.exit(generator.run(args.wheels)) From 20ac9bc5b185cf7944727a60d65fd870de90ebef Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Sat, 15 Mar 2025 17:30:39 -0700 Subject: [PATCH 102/922] feat(rules): allow deriving custom rules from core rules (#2666) This exposes public functions for creating builders for py_binary, py_test, and py_library. It also adds some docs and examples for how to use them. I'm calling this a "volatile" API -- it's public, but the pieces that comprise it (e.g. all the rule args, attributes, the attribute args, etc) are likely to change in various ways, and not all modifications to them can be supported in a backward compatible way. Hence the "volatile" term: * hold it gently and its fine * shake it a bit and its probably fine * shake it moderately and something may or may not blow up * shake it a lot and something will certainly blow up. Work towards https://github.com/bazelbuild/rules_python/issues/1647 --------- Co-authored-by: Ignas Anikevicius <240938+aignas@users.noreply.github.com> --- CHANGELOG.md | 3 + docs/BUILD.bazel | 2 + docs/_includes/volatile_api.md | 5 + docs/extending.md | 143 +++++++++++++++++++++++++++++ docs/index.md | 1 + python/api/BUILD.bazel | 20 ++++ python/api/executables.bzl | 31 +++++++ python/api/libraries.bzl | 27 ++++++ python/private/BUILD.bazel | 3 +- python/private/attr_builders.bzl | 6 +- python/private/py_binary_rule.bzl | 17 +++- python/private/py_executable.bzl | 17 ++++ python/private/py_library.bzl | 54 +++++------ python/private/py_library_rule.bzl | 18 +--- python/private/py_test_rule.bzl | 17 +++- python/private/rule_builders.bzl | 3 + tests/support/sh_py_run_test.bzl | 8 +- 17 files changed, 321 insertions(+), 54 deletions(-) create mode 100644 docs/_includes/volatile_api.md create mode 100644 docs/extending.md create mode 100644 python/api/executables.bzl create mode 100644 python/api/libraries.bzl diff --git a/CHANGELOG.md b/CHANGELOG.md index 9029794ffc..c5bf986216 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -91,6 +91,9 @@ Unreleased changes template. * (pypi) Direct HTTP urls for wheels and sdists are now supported when using {obj}`experimental_index_url` (bazel downloader). Partially fixes [#2363](https://github.com/bazelbuild/rules_python/issues/2363). +* (rules) APIs for creating custom rules based on the core py_binary, py_test, + and py_library rules + ([#1647](https://github.com/bazelbuild/rules_python/issues/1647)) {#v0-0-0-removed} ### Removed diff --git a/docs/BUILD.bazel b/docs/BUILD.bazel index e19c22113f..09de21b86a 100644 --- a/docs/BUILD.bazel +++ b/docs/BUILD.bazel @@ -100,6 +100,8 @@ sphinx_stardocs( "//python:py_test_bzl", "//python:repositories_bzl", "//python/api:api_bzl", + "//python/api:executables_bzl", + "//python/api:libraries_bzl", "//python/cc:py_cc_toolchain_bzl", "//python/cc:py_cc_toolchain_info_bzl", "//python/entry_points:py_console_script_binary_bzl", diff --git a/docs/_includes/volatile_api.md b/docs/_includes/volatile_api.md new file mode 100644 index 0000000000..b79f5f7061 --- /dev/null +++ b/docs/_includes/volatile_api.md @@ -0,0 +1,5 @@ +:::{important} + +**Public, but volatile, API.** Some parts are stable, while others are +implementation details and may change more frequently. +::: diff --git a/docs/extending.md b/docs/extending.md new file mode 100644 index 0000000000..dbd63e5a4f --- /dev/null +++ b/docs/extending.md @@ -0,0 +1,143 @@ +# Extending the rules + +:::{important} +**This is public, but volatile, functionality.** + +Extending and customizing the rules is supported functionality, but with weaker +backwards compatibility guarantees, and is not fully subject to the normal +backwards compatibility procedures and policies. It's simply not feasible to +support every possible customization with strong backwards compatibility +guarantees. +::: + +Because of the rich ecosystem of tools and variety of use cases, APIs are +provided to make it easy to create custom rules using the existing rules as a +basis. This allows implementing behaviors that aren't possible using +wrapper macros around the core rules, and can make certain types of changes +much easier and transparent to implement. + +:::{note} +It is not required to extend a core rule. The minimum requirement for a custom +rule is to return the appropriate provider (e.g. {bzl:obj}`PyInfo` etc). +Extending the core rules is most useful when you want all or most of the +behavior of a core rule. +::: + +Follow or comment on https://github.com/bazelbuild/rules_python/issues/1647 +for the development of APIs to support custom derived rules. + +## Creating custom rules + +Custom rules can be created using the core rules as a basis by using their rule +builder APIs. + +* [`//python/apis:executables.bzl`](#python-apis-executables-bzl): builders for + executables. +* [`//python/apis:libraries.bzl`](#python-apis-libraries-bzl): builders for + libraries. + +These builders create {bzl:obj}`ruleb.Rule` objects, which are thin +wrappers around the keyword arguments eventually passed to the `rule()` +function. These builder APIs give access to the _entire_ rule definition and +allow arbitrary modifications. + +This is level of control is powerful, but also volatile. A rule definition +contains many details that _must_ change as the implementation changes. What +is more or less likely to change isn't known in advance, but some general +rules are: + +* Additive behavior to public attributes will be less prone to breaking. +* Internal attributes that directly support a public attribute are likely + reliable. +* Internal attributes that support an action are more likely to change. +* Rule toolchains are moderately stable (toolchains are mostly internal to + how a rule works, but custom toolchains are supported). + +## Example: validating a source file + +In this example, we derive from `py_library` a custom rule that verifies source +code contains the word "snakes". It does this by: + +* Adding an implicit dependency on a checker program +* Calling the base implementation function +* Running the checker on the srcs files +* Adding the result to the `_validation` output group (a special output + group for validation behaviors). + +To users, they can use `has_snakes_library` the same as `py_library`. The same +is true for other targets that might consume the rule. + +``` +load("@rules_python//python/api:libraries.bzl", "libraries") +load("@rules_python//python/api:attr_builders.bzl", "attrb") + +def _has_snakes_impl(ctx, base): + providers = base(ctx) + + out = ctx.actions.declare_file(ctx.label.name + "_snakes.check") + ctx.actions.run( + inputs = ctx.files.srcs, + outputs = [out], + executable = ctx.attr._checker[DefaultInfo].files_to_run, + args = [out.path] + [f.path for f in ctx.files.srcs], + ) + prior_ogi = None + for i, p in enumerate(providers): + if type(p) == "OutputGroupInfo": + prior_ogi = (i, p) + break + if prior_ogi: + groups = {k: getattr(prior_ogi[1], k) for k in dir(prior_ogi)} + if "_validation" in groups: + groups["_validation"] = depset([out], transitive=groups["_validation"]) + else: + groups["_validation"] = depset([out]) + providers[prior_ogi[0]] = OutputGroupInfo(**groups) + else: + providers.append(OutputGroupInfo(_validation=depset([out]))) + return providers + +def create_has_snakes_rule(): + r = libraries.py_library_builder() + base_impl = r.implementation() + r.set_implementation(lambda ctx: _has_snakes_impl(ctx, base_impl)) + r.attrs["_checker"] = attrb.Label( + default="//:checker", + executable = True, + ) + return r.build() +has_snakes_library = create_has_snakes_rule() +``` + +## Example: adding transitions + +In this example, we derive from `py_binary` to force building for a particular +platform. We do this by: + +* Adding an additional output to the rule's cfg +* Calling the base transition function +* Returning the new transition outputs + +```starlark + +load("@rules_python//python/api:executables.bzl", "executables") + +def _force_linux_impl(settings, attr, base_impl): + settings = base_impl(settings, attr) + settings["//command_line_option:platforms"] = ["//my/platforms:linux"] + return settings + +def create_rule(): + r = executables.py_binary_rule_builder() + base_impl = r.cfg.implementation() + r.cfg.set_implementation( + lambda settings, attr: _force_linux_impl(settings, attr, base_impl) + ) + r.cfg.add_output("//command_line_option:platforms") + return r.build() + +py_linux_binary = create_linux_binary_rule() +``` + +Users can then use `py_linux_binary` the same as a regular py_binary. It will +act as if `--platforms=//my/platforms:linux` was specified when building it. diff --git a/docs/index.md b/docs/index.md index dd2e147c18..04a7688850 100644 --- a/docs/index.md +++ b/docs/index.md @@ -101,6 +101,7 @@ pip coverage precompiling gazelle +Extending Contributing support Changelog diff --git a/python/api/BUILD.bazel b/python/api/BUILD.bazel index 1df6877ef8..f0e04948ac 100644 --- a/python/api/BUILD.bazel +++ b/python/api/BUILD.bazel @@ -25,6 +25,26 @@ bzl_library( deps = ["//python/private/api:api_bzl"], ) +bzl_library( + name = "executables_bzl", + srcs = ["executables.bzl"], + visibility = ["//visibility:public"], + deps = [ + "//python/private:py_binary_rule_bzl", + "//python/private:py_executable_bzl", + "//python/private:py_test_rule_bzl", + ], +) + +bzl_library( + name = "libraries_bzl", + srcs = ["libraries.bzl"], + visibility = ["//visibility:public"], + deps = [ + "//python/private:py_library_bzl", + ], +) + filegroup( name = "distribution", srcs = glob(["**"]), diff --git a/python/api/executables.bzl b/python/api/executables.bzl new file mode 100644 index 0000000000..4715c0f481 --- /dev/null +++ b/python/api/executables.bzl @@ -0,0 +1,31 @@ +# Copyright 2025 The Bazel Authors. All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +""" +{#python-apis-executables-bzl} +Loading-phase APIs specific to executables (binaries/tests). + +:::{versionadded} VERSION_NEXT_FEATURE +::: +""" + +load("//python/private:py_binary_rule.bzl", "create_py_binary_rule_builder") +load("//python/private:py_executable.bzl", "create_executable_rule_builder") +load("//python/private:py_test_rule.bzl", "create_py_test_rule_builder") + +executables = struct( + py_binary_rule_builder = create_py_binary_rule_builder, + py_test_rule_builder = create_py_test_rule_builder, + executable_rule_builder = create_executable_rule_builder, +) diff --git a/python/api/libraries.bzl b/python/api/libraries.bzl new file mode 100644 index 0000000000..c4ad598e3f --- /dev/null +++ b/python/api/libraries.bzl @@ -0,0 +1,27 @@ +# Copyright 2025 The Bazel Authors. All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +""" +{#python-apis-libraries-bzl} +Loading-phase APIs specific to libraries. + +:::{versionadded} VERSION_NEXT_FEATURE +::: +""" + +load("//python/private:py_library.bzl", "create_py_library_rule_builder") + +libraries = struct( + py_library_rule_builder = create_py_library_rule_builder, +) diff --git a/python/private/BUILD.bazel b/python/private/BUILD.bazel index b7e52a35aa..8b07fbd877 100644 --- a/python/private/BUILD.bazel +++ b/python/private/BUILD.bazel @@ -427,6 +427,7 @@ bzl_library( ":attributes_bzl", ":common_bzl", ":flags_bzl", + ":precompile_bzl", ":py_cc_link_params_info_bzl", ":py_internal_bzl", ":rule_builders_bzl", @@ -446,8 +447,6 @@ bzl_library( name = "py_library_rule_bzl", srcs = ["py_library_rule.bzl"], deps = [ - ":common_bzl", - ":precompile_bzl", ":py_library_bzl", ], ) diff --git a/python/private/attr_builders.bzl b/python/private/attr_builders.bzl index acd1d40394..efcbfa6e5b 100644 --- a/python/private/attr_builders.bzl +++ b/python/private/attr_builders.bzl @@ -12,7 +12,11 @@ # See the License for the specific language governing permissions and # limitations under the License. -"""Builders for creating attributes et al.""" +"""Builders for creating attributes et al. + +:::{versionadded} VERSION_NEXT_FEATURE +::: +""" load("@bazel_skylib//lib:types.bzl", "types") load( diff --git a/python/private/py_binary_rule.bzl b/python/private/py_binary_rule.bzl index 0e1912cf0c..38e3a697c7 100644 --- a/python/private/py_binary_rule.bzl +++ b/python/private/py_binary_rule.bzl @@ -27,7 +27,20 @@ def _py_binary_impl(ctx): inherited_environment = [], ) -def create_binary_rule_builder(): +# NOTE: Exported publicly +def create_py_binary_rule_builder(): + """Create a rule builder for a py_binary. + + :::{include} /_includes/volatile_api.md + ::: + + :::{versionadded} VERSION_NEXT_FEATURE + ::: + + Returns: + {type}`ruleb.Rule` with the necessary settings + for creating a `py_binary` rule. + """ builder = create_executable_rule_builder( implementation = _py_binary_impl, executable = True, @@ -35,4 +48,4 @@ def create_binary_rule_builder(): builder.attrs.update(AGNOSTIC_BINARY_ATTRS) return builder -py_binary = create_binary_rule_builder().build() +py_binary = create_py_binary_rule_builder().build() diff --git a/python/private/py_executable.bzl b/python/private/py_executable.bzl index f85f242bba..bcbff70bec 100644 --- a/python/private/py_executable.bzl +++ b/python/private/py_executable.bzl @@ -1737,7 +1737,24 @@ def create_base_executable_rule(): """ return create_executable_rule_builder().build() +# NOTE: Exported publicly def create_executable_rule_builder(implementation, **kwargs): + """Create a rule builder for an executable Python program. + + :::{include} /_includes/volatile_api.md + ::: + + An executable rule is one that sets either `executable=True` or `test=True`, + and the output is something that can be run directly (e.g. `bazel run`, + `exec(...)` etc) + + :::{versionadded} VERSION_NEXT_FEATURE + ::: + + Returns: + {type}`ruleb.Rule` with the necessary settings + for creating an executable Python rule. + """ builder = ruleb.Rule( implementation = implementation, attrs = EXECUTABLE_ATTRS, diff --git a/python/private/py_library.bzl b/python/private/py_library.bzl index a774104dd2..7b024a0f07 100644 --- a/python/private/py_library.bzl +++ b/python/private/py_library.bzl @@ -25,16 +25,9 @@ load( "REQUIRED_EXEC_GROUP_BUILDERS", ) load(":builders.bzl", "builders") -load( - ":common.bzl", - "collect_imports", - "collect_runfiles", - "create_instrumented_files_info", - "create_output_group_info", - "create_py_info", - "filter_to_py_srcs", -) +load(":common.bzl", "collect_cc_info", "collect_imports", "collect_runfiles", "create_instrumented_files_info", "create_library_semantics_struct", "create_output_group_info", "create_py_info", "filter_to_py_srcs", "get_imports") load(":flags.bzl", "AddSrcsToRunfilesFlag", "PrecompileFlag") +load(":precompile.bzl", "maybe_precompile") load(":py_cc_link_params_info.bzl", "PyCcLinkParamsInfo") load(":py_internal.bzl", "py_internal") load(":rule_builders.bzl", "ruleb") @@ -57,6 +50,16 @@ LIBRARY_ATTRS = dicts.add( }, ) +def _py_library_impl_with_semantics(ctx): + return py_library_impl( + ctx, + semantics = create_library_semantics_struct( + get_imports = get_imports, + maybe_precompile = maybe_precompile, + get_cc_info_for_library = collect_cc_info, + ), + ) + def py_library_impl(ctx, *, semantics): """Abstract implementation of py_library rule. @@ -141,32 +144,29 @@ Source files are no longer added to the runfiles directly. ::: """ -def create_py_library_rule_builder(*, attrs = {}, **kwargs): - """Creates a py_library rule. +# NOTE: Exported publicaly +def create_py_library_rule_builder(): + """Create a rule builder for a py_library. - Args: - attrs: dict of rule attributes. - **kwargs: Additional kwargs to pass onto {obj}`ruleb.Rule()`. + :::{include} /_includes/volatile_api.md + ::: + + :::{versionadded} VERSION_NEXT_FEATURE + ::: Returns: - {type}`ruleb.Rule` builder object. + {type}`ruleb.Rule` with the necessary settings + for creating a `py_library` rule. """ - - # Within Google, the doc attribute is overridden - kwargs.setdefault("doc", _DEFAULT_PY_LIBRARY_DOC) - - # TODO: b/253818097 - fragments=py is only necessary so that - # RequiredConfigFragmentsTest passes - fragments = kwargs.pop("fragments", None) or [] - kwargs["exec_groups"] = REQUIRED_EXEC_GROUP_BUILDERS | (kwargs.get("exec_groups") or {}) - builder = ruleb.Rule( - attrs = dicts.add(LIBRARY_ATTRS, attrs), - fragments = fragments + ["py"], + implementation = _py_library_impl_with_semantics, + doc = _DEFAULT_PY_LIBRARY_DOC, + exec_groups = dict(REQUIRED_EXEC_GROUP_BUILDERS), + attrs = LIBRARY_ATTRS, + fragments = ["py"], toolchains = [ ruleb.ToolchainType(TOOLCHAIN_TYPE, mandatory = False), ruleb.ToolchainType(EXEC_TOOLS_TOOLCHAIN_TYPE, mandatory = False), ], - **kwargs ) return builder diff --git a/python/private/py_library_rule.bzl b/python/private/py_library_rule.bzl index 44382a76d6..ac256bccc1 100644 --- a/python/private/py_library_rule.bzl +++ b/python/private/py_library_rule.bzl @@ -13,20 +13,6 @@ # limitations under the License. """Implementation of py_library rule.""" -load(":common.bzl", "collect_cc_info", "create_library_semantics_struct", "get_imports") -load(":precompile.bzl", "maybe_precompile") -load(":py_library.bzl", "create_py_library_rule_builder", "py_library_impl") +load(":py_library.bzl", "create_py_library_rule_builder") -def _py_library_impl_with_semantics(ctx): - return py_library_impl( - ctx, - semantics = create_library_semantics_struct( - get_imports = get_imports, - maybe_precompile = maybe_precompile, - get_cc_info_for_library = collect_cc_info, - ), - ) - -py_library = create_py_library_rule_builder( - implementation = _py_library_impl_with_semantics, -).build() +py_library = create_py_library_rule_builder().build() diff --git a/python/private/py_test_rule.bzl b/python/private/py_test_rule.bzl index 72e8bab805..f21fdc7557 100644 --- a/python/private/py_test_rule.bzl +++ b/python/private/py_test_rule.bzl @@ -30,7 +30,20 @@ def _py_test_impl(ctx): maybe_add_test_execution_info(providers, ctx) return providers -def create_test_rule_builder(): +# NOTE: Exported publicaly +def create_py_test_rule_builder(): + """Create a rule builder for a py_test. + + :::{include} /_includes/volatile_api.md + ::: + + :::{versionadded} VERSION_NEXT_FEATURE + ::: + + Returns: + {type}`ruleb.Rule` with the necessary settings + for creating a `py_test` rule. + """ builder = create_executable_rule_builder( implementation = _py_test_impl, test = True, @@ -38,4 +51,4 @@ def create_test_rule_builder(): builder.attrs.update(AGNOSTIC_TEST_ATTRS) return builder -py_test = create_test_rule_builder().build() +py_test = create_py_test_rule_builder().build() diff --git a/python/private/rule_builders.bzl b/python/private/rule_builders.bzl index 6d9fb3f964..4607285949 100644 --- a/python/private/rule_builders.bzl +++ b/python/private/rule_builders.bzl @@ -91,6 +91,9 @@ def create_custom_foo_binary(): custom_foo_binary = create_custom_foo_binary() ``` + +:::{versionadded} VERSION_NEXT_FEATURE +::: """ load("@bazel_skylib//lib:types.bzl", "types") diff --git a/tests/support/sh_py_run_test.bzl b/tests/support/sh_py_run_test.bzl index d1e3b8e9c8..7b3b617da1 100644 --- a/tests/support/sh_py_run_test.bzl +++ b/tests/support/sh_py_run_test.bzl @@ -20,9 +20,9 @@ without the overhead of a bazel-in-bazel integration test. load("@rules_shell//shell:sh_test.bzl", "sh_test") load("//python/private:attr_builders.bzl", "attrb") # buildifier: disable=bzl-visibility load("//python/private:py_binary_macro.bzl", "py_binary_macro") # buildifier: disable=bzl-visibility -load("//python/private:py_binary_rule.bzl", "create_binary_rule_builder") # buildifier: disable=bzl-visibility +load("//python/private:py_binary_rule.bzl", "create_py_binary_rule_builder") # buildifier: disable=bzl-visibility load("//python/private:py_test_macro.bzl", "py_test_macro") # buildifier: disable=bzl-visibility -load("//python/private:py_test_rule.bzl", "create_test_rule_builder") # buildifier: disable=bzl-visibility +load("//python/private:py_test_rule.bzl", "create_py_test_rule_builder") # buildifier: disable=bzl-visibility load("//python/private:toolchain_types.bzl", "TARGET_TOOLCHAIN_TYPE") # buildifier: disable=bzl-visibility load("//tests/support:support.bzl", "VISIBLE_FOR_TESTING") @@ -79,9 +79,9 @@ def _create_reconfig_rule(builder): builder.cfg.update_outputs(_RECONFIG_OUTPUTS) return builder.build() -_py_reconfig_binary = _create_reconfig_rule(create_binary_rule_builder()) +_py_reconfig_binary = _create_reconfig_rule(create_py_binary_rule_builder()) -_py_reconfig_test = _create_reconfig_rule(create_test_rule_builder()) +_py_reconfig_test = _create_reconfig_rule(create_py_test_rule_builder()) def py_reconfig_test(**kwargs): """Create a py_test with customized build settings for testing. From 4079953a8397b22ee30c3a1534d04211c566959c Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Sun, 16 Mar 2025 19:46:27 -0700 Subject: [PATCH 103/922] feat(binary/test): add interpreter_args attribute (#2669) Today, there's way to control what startup args are used for the interpreter. To fix, add an `interpreter_args` attribute. These are written into the bootstrap. This is only implemented for the bootstrap=script method Fixes https://github.com/bazelbuild/rules_python/issues/2668 --- CHANGELOG.md | 2 ++ python/private/py_executable.bzl | 19 ++++++++++++++ python/private/stage1_bootstrap_template.sh | 6 +++++ tests/bootstrap_impls/BUILD.bazel | 9 +++++++ .../bootstrap_impls/interpreter_args_test.py | 25 +++++++++++++++++++ 5 files changed, 61 insertions(+) create mode 100644 tests/bootstrap_impls/interpreter_args_test.py diff --git a/CHANGELOG.md b/CHANGELOG.md index c5bf986216..dc2419360c 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -94,6 +94,8 @@ Unreleased changes template. * (rules) APIs for creating custom rules based on the core py_binary, py_test, and py_library rules ([#1647](https://github.com/bazelbuild/rules_python/issues/1647)) +* (rules) Added {obj}`interpreter_args` attribute to `py_binary` and `py_test`, + which allows pass arguments to the interpreter before the regular args. {#v0-0-0-removed} ### Removed diff --git a/python/private/py_executable.bzl b/python/private/py_executable.bzl index bcbff70bec..d1905448a6 100644 --- a/python/private/py_executable.bzl +++ b/python/private/py_executable.bzl @@ -87,6 +87,21 @@ EXECUTABLE_ATTRS = dicts.add( IMPORTS_ATTRS, COVERAGE_ATTRS, { + "interpreter_args": lambda: attrb.StringList( + doc = """ +Arguments that are only applicable to the interpreter. + +The args an interpreter supports are specific to the interpreter. For +CPython, see https://docs.python.org/3/using/cmdline.html. + +:::{note} +Only supported for {obj}`--bootstrap_impl=script`. Ignored otherwise. +::: + +:::{versionadded} VERSION_NEXT_FEATURE +::: +""", + ), "legacy_create_init": lambda: attrb.Int( default = -1, values = [-1, 0, 1], @@ -658,6 +673,10 @@ def _create_stage1_bootstrap( python_binary_actual = venv.interpreter_actual_path if venv else "" subs = { + "%interpreter_args%": "\n".join([ + '"{}"'.format(v) + for v in ctx.attr.interpreter_args + ]), "%is_zipfile%": "1" if is_for_zip else "0", "%python_binary%": python_binary_path, "%python_binary_actual%": python_binary_actual, diff --git a/python/private/stage1_bootstrap_template.sh b/python/private/stage1_bootstrap_template.sh index 19ff763094..523210ad14 100644 --- a/python/private/stage1_bootstrap_template.sh +++ b/python/private/stage1_bootstrap_template.sh @@ -21,6 +21,11 @@ IS_ZIPFILE="%is_zipfile%" # 0 or 1 RECREATE_VENV_AT_RUNTIME="%recreate_venv_at_runtime%" +# array of strings +declare -a INTERPRETER_ARGS_FROM_TARGET=( +%interpreter_args% +) + if [[ "$IS_ZIPFILE" == "1" ]]; then # NOTE: Macs have an old version of mktemp, so we must use only the # minimal functionality of it. @@ -222,6 +227,7 @@ command=( "${interpreter_env[@]}" "$python_exe" "${interpreter_args[@]}" + "${INTERPRETER_ARGS_FROM_TARGET[@]}" "$stage2_bootstrap" "$@" ) diff --git a/tests/bootstrap_impls/BUILD.bazel b/tests/bootstrap_impls/BUILD.bazel index 8a64bf2b5b..7a5c4b46c6 100644 --- a/tests/bootstrap_impls/BUILD.bazel +++ b/tests/bootstrap_impls/BUILD.bazel @@ -124,4 +124,13 @@ sh_py_run_test( target_compatible_with = SUPPORTS_BOOTSTRAP_SCRIPT, ) +py_reconfig_test( + name = "interpreter_args_test", + srcs = ["interpreter_args_test.py"], + bootstrap_impl = "script", + interpreter_args = ["-XSPECIAL=1"], + main = "interpreter_args_test.py", + target_compatible_with = SUPPORTS_BOOTSTRAP_SCRIPT, +) + relative_path_test_suite(name = "relative_path_tests") diff --git a/tests/bootstrap_impls/interpreter_args_test.py b/tests/bootstrap_impls/interpreter_args_test.py new file mode 100644 index 0000000000..27744c647f --- /dev/null +++ b/tests/bootstrap_impls/interpreter_args_test.py @@ -0,0 +1,25 @@ +# Copyright 2025 The Bazel Authors. All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +import sys +import unittest + + +class InterpreterArgsTest(unittest.TestCase): + def test_interpreter_args(self): + self.assertEqual(sys._xoptions, {"SPECIAL": "1"}) + + +if __name__ == "__main__": + unittest.main() From ea80366b29219d3dad8c90191c21b77a4525875a Mon Sep 17 00:00:00 2001 From: "Andrew Lindesay [Canva]" <143454275+andponlin-canva@users.noreply.github.com> Date: Mon, 17 Mar 2025 16:30:18 +1300 Subject: [PATCH 104/922] feat: env-var for additional interpreter args in bootstrap stage 1 (#2654) There is no means to be able to provide additional interpreter arguments to the `bash`-based stage 1 bootstrap system at launch time. The Intelli-J / Bazel plugin typically launches a `py_*` rule build product with something like this (abridged) using a Python interpreter from the local environment; ``` python3 /path/to/pydev/pydevd.py --client 127.0.0.1 --port 12344 --file /path/to/built/python-file ``` When the `bash`-based bootstrap process is used, this mechanism not longer works. This PR will mean that a potential future Intelli-j / Bazel plugin version may be able to launch the build product differently and inject additional interpreter arguments so that the debug system can be stood up in this sort of a way; ``` RULES_PYTHON_ADDITIONAL_INTERPRETER_ARGS="/path/to/pydev/pydevd.py --client 127.0.0.1 --port 12344 --file" /path/to/bash-bootstrap-stage1-script ``` The work to support this in the Intelli-J / Bazel plugin has not been done; it would have to be undertaken some time after this change were available. --------- Co-authored-by: Ignas Anikevicius <240938+aignas@users.noreply.github.com> Co-authored-by: Richard Levasseur --- CHANGELOG.md | 3 +++ docs/environment-variables.md | 28 +++++++++++++++++++++ python/private/py_executable.bzl | 4 +++ python/private/stage1_bootstrap_template.sh | 7 ++++++ 4 files changed, 42 insertions(+) diff --git a/CHANGELOG.md b/CHANGELOG.md index dc2419360c..15fb211ce8 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -94,6 +94,9 @@ Unreleased changes template. * (rules) APIs for creating custom rules based on the core py_binary, py_test, and py_library rules ([#1647](https://github.com/bazelbuild/rules_python/issues/1647)) +* (rules) Added env-var to allow additional interpreter args for stage1 bootstrap. + See {obj}`RULES_PYTHON_ADDITIONAL_INTERPRETER_ARGS` environment variable. + Only applicable for {obj}`--bootstrap_impl=script`. * (rules) Added {obj}`interpreter_args` attribute to `py_binary` and `py_test`, which allows pass arguments to the interpreter before the regular args. diff --git a/docs/environment-variables.md b/docs/environment-variables.md index d50070af55..c7c0181d18 100644 --- a/docs/environment-variables.md +++ b/docs/environment-variables.md @@ -1,5 +1,33 @@ # Environment Variables +::::{envvar} RULES_PYTHON_ADDITIONAL_INTERPRETER_ARGS + +This variable allows for additional arguments to be provided to the Python interpreter +at bootstrap time when the `bash` bootstrap is used. If +`RULES_PYTHON_ADDITIONAL_INTERPRETER_ARGS` were provided as `-Xaaa`, then the command +would be; + +``` +python -Xaaa /path/to/file.py +``` + +This feature is likely to be useful for the integration of debuggers. For example, +it would be possible to configure the `RULES_PYTHON_ADDITIONAL_INTERPRETER_ARGS` to +be set to `/path/to/debugger.py --port 12344 --file` resulting +in the command executed being; + +``` +python /path/to/debugger.py --port 12345 --file /path/to/file.py +``` + +:::{seealso} +The {bzl:obj}`interpreter_args` attribute. +::: + +:::{versionadded} VERSION_NEXT_FEATURE + +:::: + :::{envvar} RULES_PYTHON_BOOTSTRAP_VERBOSE When `1`, debug information about bootstrapping of a program is printed to diff --git a/python/private/py_executable.bzl b/python/private/py_executable.bzl index d1905448a6..bbaed3104e 100644 --- a/python/private/py_executable.bzl +++ b/python/private/py_executable.bzl @@ -98,6 +98,10 @@ CPython, see https://docs.python.org/3/using/cmdline.html. Only supported for {obj}`--bootstrap_impl=script`. Ignored otherwise. ::: +:::{seealso} +The {obj}`RULES_PYTHON_ADDITIONAL_INTERPRETER_ARGS` environment variable +::: + :::{versionadded} VERSION_NEXT_FEATURE ::: """, diff --git a/python/private/stage1_bootstrap_template.sh b/python/private/stage1_bootstrap_template.sh index 523210ad14..bd142cf7c7 100644 --- a/python/private/stage1_bootstrap_template.sh +++ b/python/private/stage1_bootstrap_template.sh @@ -202,6 +202,7 @@ stage2_bootstrap="$RUNFILES_DIR/$STAGE2_BOOTSTRAP" declare -a interpreter_env declare -a interpreter_args +declare -a additional_interpreter_args # Don't prepend a potentially unsafe path to sys.path # See: https://docs.python.org/3.11/using/cmdline.html#envvar-PYTHONSAFEPATH @@ -220,6 +221,12 @@ if [[ "$IS_ZIPFILE" == "1" ]]; then interpreter_args+=("-XRULES_PYTHON_ZIP_DIR=$zip_dir") fi +if [[ -n "${RULES_PYTHON_ADDITIONAL_INTERPRETER_ARGS}" ]]; then + read -a additional_interpreter_args <<< "${RULES_PYTHON_ADDITIONAL_INTERPRETER_ARGS}" + interpreter_args+=("${additional_interpreter_args[@]}") + unset RULES_PYTHON_ADDITIONAL_INTERPRETER_ARGS +fi + export RUNFILES_DIR command=( From 2c1d9e062db844d56e44a84539562a14175bb7d9 Mon Sep 17 00:00:00 2001 From: Yun Peng Date: Mon, 17 Mar 2025 17:39:55 +0100 Subject: [PATCH 105/922] Update source repo in BCR metadata.json (#2672) Update after https://github.com/bazel-contrib/rules_python/issues/2638 --- .bcr/metadata.template.json | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/.bcr/metadata.template.json b/.bcr/metadata.template.json index b164e70443..579d6884cd 100644 --- a/.bcr/metadata.template.json +++ b/.bcr/metadata.template.json @@ -13,7 +13,8 @@ } ], "repository": [ - "github:bazelbuild/rules_python" + "github:bazelbuild/rules_python", + "github:bazel-contrib/rules_python" ], "versions": [], "yanked_versions": {} From 6e4abec6a3b2b1d72ca924ef02639d9c1a2b87ef Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Mon, 17 Mar 2025 14:08:05 -0700 Subject: [PATCH 106/922] build(deps): bump jinja2 from 3.1.4 to 3.1.6 in /examples/bzlmod (#2651) Bumps [jinja2](https://github.com/pallets/jinja) from 3.1.4 to 3.1.6.
Release notes

Sourced from jinja2's releases.

3.1.6

This is the Jinja 3.1.6 security release, which fixes security issues but does not otherwise change behavior and should not result in breaking changes compared to the latest feature release.

PyPI: https://pypi.org/project/Jinja2/3.1.6/ Changes: https://jinja.palletsprojects.com/en/stable/changes/#version-3-1-6

3.1.5

This is the Jinja 3.1.5 security fix release, which fixes security issues and bugs but does not otherwise change behavior and should not result in breaking changes compared to the latest feature release.

PyPI: https://pypi.org/project/Jinja2/3.1.5/ Changes: https://jinja.palletsprojects.com/changes/#version-3-1-5 Milestone: https://github.com/pallets/jinja/milestone/16?closed=1

  • The sandboxed environment handles indirect calls to str.format, such as by passing a stored reference to a filter that calls its argument. GHSA-q2x7-8rv6-6q7h
  • Escape template name before formatting it into error messages, to avoid issues with names that contain f-string syntax. #1792, GHSA-gmj6-6f8f-6699
  • Sandbox does not allow clear and pop on known mutable sequence types. #2032
  • Calling sync render for an async template uses asyncio.run. #1952
  • Avoid unclosed auto_aiter warnings. #1960
  • Return an aclose-able AsyncGenerator from Template.generate_async. #1960
  • Avoid leaving root_render_func() unclosed in Template.generate_async. #1960
  • Avoid leaving async generators unclosed in blocks, includes and extends. #1960
  • The runtime uses the correct concat function for the current environment when calling block references. #1701
  • Make |unique async-aware, allowing it to be used after another async-aware filter. #1781
  • |int filter handles OverflowError from scientific notation. #1921
  • Make compiling deterministic for tuple unpacking in a {% set ... %} call. #2021
  • Fix dunder protocol (copy/pickle/etc) interaction with Undefined objects. #2025
  • Fix copy/pickle support for the internal missing object. #2027
  • Environment.overlay(enable_async) is applied correctly. #2061
  • The error message from FileSystemLoader includes the paths that were searched. #1661
  • PackageLoader shows a clearer error message when the package does not contain the templates directory. #1705
  • Improve annotations for methods returning copies. #1880
  • urlize does not add mailto: to values like @a@b. #1870
  • Tests decorated with @pass_context can be used with the |select filter. #1624
  • Using set for multiple assignment (a, b = 1, 2) does not fail when the target is a namespace attribute. #1413
  • Using set in all branches of {% if %}{% elif %}{% else %} blocks does not cause the variable to be considered initially undefined. #1253
Changelog

Sourced from jinja2's changelog.

Version 3.1.6

Released 2025-03-05

  • The |attr filter does not bypass the environment's attribute lookup, allowing the sandbox to apply its checks. :ghsa:cpwx-vrp4-4pq7

Version 3.1.5

Released 2024-12-21

  • The sandboxed environment handles indirect calls to str.format, such as by passing a stored reference to a filter that calls its argument. :ghsa:q2x7-8rv6-6q7h
  • Escape template name before formatting it into error messages, to avoid issues with names that contain f-string syntax. :issue:1792, :ghsa:gmj6-6f8f-6699
  • Sandbox does not allow clear and pop on known mutable sequence types. :issue:2032
  • Calling sync render for an async template uses asyncio.run. :pr:1952
  • Avoid unclosed auto_aiter warnings. :pr:1960
  • Return an aclose-able AsyncGenerator from Template.generate_async. :pr:1960
  • Avoid leaving root_render_func() unclosed in Template.generate_async. :pr:1960
  • Avoid leaving async generators unclosed in blocks, includes and extends. :pr:1960
  • The runtime uses the correct concat function for the current environment when calling block references. :issue:1701
  • Make |unique async-aware, allowing it to be used after another async-aware filter. :issue:1781
  • |int filter handles OverflowError from scientific notation. :issue:1921
  • Make compiling deterministic for tuple unpacking in a {% set ... %} call. :issue:2021
  • Fix dunder protocol (copy/pickle/etc) interaction with Undefined objects. :issue:2025
  • Fix copy/pickle support for the internal missing object. :issue:2027
  • Environment.overlay(enable_async) is applied correctly. :pr:2061
  • The error message from FileSystemLoader includes the paths that were searched. :issue:1661
  • PackageLoader shows a clearer error message when the package does not contain the templates directory. :issue:1705
  • Improve annotations for methods returning copies. :pr:1880
  • urlize does not add mailto: to values like @a@b. :pr:1870

... (truncated)

Commits

[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=jinja2&package-manager=pip&previous-version=3.1.4&new-version=3.1.6)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) ---
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot show ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) You can disable automated security fix PRs for this repo from the [Security Alerts page](https://github.com/bazelbuild/rules_python/network/alerts).
Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> --- examples/bzlmod/requirements_lock_3_10.txt | 6 +++--- examples/bzlmod/requirements_lock_3_9.txt | 6 +++--- examples/bzlmod/requirements_windows_3_10.txt | 6 +++--- 3 files changed, 9 insertions(+), 9 deletions(-) diff --git a/examples/bzlmod/requirements_lock_3_10.txt b/examples/bzlmod/requirements_lock_3_10.txt index ace879f38e..c7e35a2b2c 100644 --- a/examples/bzlmod/requirements_lock_3_10.txt +++ b/examples/bzlmod/requirements_lock_3_10.txt @@ -50,9 +50,9 @@ isort==5.12.0 \ --hash=sha256:8bef7dde241278824a6d83f44a544709b065191b95b6e50894bdc722fcba0504 \ --hash=sha256:f84c2818376e66cf843d497486ea8fed8700b340f308f076c6fb1229dff318b6 # via pylint -jinja2==3.1.4 \ - --hash=sha256:4a3aee7acbbe7303aede8e9648d13b8bf88a429282aa6122a993f0ac800cb369 \ - --hash=sha256:bc5dd2abb727a5319567b7a813e6a2e7318c39f4f487cfe6c89c6f9c7d25197d +jinja2==3.1.6 \ + --hash=sha256:0137fb05990d35f1275a587e9aee6d56da821fc83491a0fb838183be43f66d6d \ + --hash=sha256:85ece4451f492d0c13c5dd7c13a64681a86afae63a5f347908daf103ce6d2f67 # via sphinx lazy-object-proxy==1.9.0 \ --hash=sha256:09763491ce220c0299688940f8dc2c5d05fd1f45af1e42e636b2e8b2303e4382 \ diff --git a/examples/bzlmod/requirements_lock_3_9.txt b/examples/bzlmod/requirements_lock_3_9.txt index bfabfd5fa5..d74d1d39b6 100644 --- a/examples/bzlmod/requirements_lock_3_9.txt +++ b/examples/bzlmod/requirements_lock_3_9.txt @@ -54,9 +54,9 @@ isort==5.11.4 \ --hash=sha256:6db30c5ded9815d813932c04c2f85a360bcdd35fed496f4d8f35495ef0a261b6 \ --hash=sha256:c033fd0edb91000a7f09527fe5c75321878f98322a77ddcc81adbd83724afb7b # via pylint -jinja2==3.1.4 \ - --hash=sha256:4a3aee7acbbe7303aede8e9648d13b8bf88a429282aa6122a993f0ac800cb369 \ - --hash=sha256:bc5dd2abb727a5319567b7a813e6a2e7318c39f4f487cfe6c89c6f9c7d25197d +jinja2==3.1.6 \ + --hash=sha256:0137fb05990d35f1275a587e9aee6d56da821fc83491a0fb838183be43f66d6d \ + --hash=sha256:85ece4451f492d0c13c5dd7c13a64681a86afae63a5f347908daf103ce6d2f67 # via sphinx lazy-object-proxy==1.10.0 \ --hash=sha256:009e6bb1f1935a62889ddc8541514b6a9e1fcf302667dcb049a0be5c8f613e56 \ diff --git a/examples/bzlmod/requirements_windows_3_10.txt b/examples/bzlmod/requirements_windows_3_10.txt index e4373c1682..0e43dbfe6b 100644 --- a/examples/bzlmod/requirements_windows_3_10.txt +++ b/examples/bzlmod/requirements_windows_3_10.txt @@ -53,9 +53,9 @@ isort==5.12.0 \ --hash=sha256:8bef7dde241278824a6d83f44a544709b065191b95b6e50894bdc722fcba0504 \ --hash=sha256:f84c2818376e66cf843d497486ea8fed8700b340f308f076c6fb1229dff318b6 # via pylint -jinja2==3.1.4 \ - --hash=sha256:4a3aee7acbbe7303aede8e9648d13b8bf88a429282aa6122a993f0ac800cb369 \ - --hash=sha256:bc5dd2abb727a5319567b7a813e6a2e7318c39f4f487cfe6c89c6f9c7d25197d +jinja2==3.1.6 \ + --hash=sha256:0137fb05990d35f1275a587e9aee6d56da821fc83491a0fb838183be43f66d6d \ + --hash=sha256:85ece4451f492d0c13c5dd7c13a64681a86afae63a5f347908daf103ce6d2f67 # via sphinx lazy-object-proxy==1.9.0 \ --hash=sha256:09763491ce220c0299688940f8dc2c5d05fd1f45af1e42e636b2e8b2303e4382 \ From 5ba2e705225d1a78de0e86fee000377a4a483834 Mon Sep 17 00:00:00 2001 From: Alex Eagle Date: Mon, 17 Mar 2025 15:03:59 -0700 Subject: [PATCH 107/922] chore(docs): fix forward-ref to 1.0 (#2673) It's been released so this was out-of-date. --- docs/index.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/index.md b/docs/index.md index 04a7688850..b10b445983 100644 --- a/docs/index.md +++ b/docs/index.md @@ -13,7 +13,7 @@ in this repository are simple aliases. On Bazel 7 and above `rules_python` uses a separate Starlark implementation, see {ref}`Migrating from the Bundled Rules` below. -Once rules_python 1.0 is released, they will follow +This repository follows [semantic versioning](https://semver.org) and the breaking change policy outlined in the [support](support) page. From 701ba456462eccce7d1dac4abaf24f9b6b8207e7 Mon Sep 17 00:00:00 2001 From: Alex Eagle Date: Mon, 17 Mar 2025 17:47:22 -0700 Subject: [PATCH 108/922] chore: account for new GH org of standalone interpreter (#2676) The repo was donated. It also trivially removes the need for a redirect on the URL that fetches artifacts. --- python/private/pypi/whl_library.bzl | 8 ++++---- python/private/python_repository.bzl | 4 ++-- python/versions.bzl | 2 +- 3 files changed, 7 insertions(+), 7 deletions(-) diff --git a/python/private/pypi/whl_library.bzl b/python/private/pypi/whl_library.bzl index dea61b23dc..9bbd842116 100644 --- a/python/private/pypi/whl_library.bzl +++ b/python/private/pypi/whl_library.bzl @@ -34,8 +34,8 @@ def _get_xcode_location_cflags(rctx): """Query the xcode sdk location to update cflags Figure out if this interpreter target comes from rules_python, and patch the xcode sdk location if so. - Pip won't be able to compile c extensions from sdists with the pre built python distributions from indygreg - otherwise. See https://github.com/indygreg/python-build-standalone/issues/103 + Pip won't be able to compile c extensions from sdists with the pre built python distributions from astral-sh + otherwise. See https://github.com/astral-sh/python-build-standalone/issues/103 """ # Only run on MacOS hosts @@ -63,8 +63,8 @@ def _get_xcode_location_cflags(rctx): def _get_toolchain_unix_cflags(rctx, python_interpreter, logger = None): """Gather cflags from a standalone toolchain for unix systems. - Pip won't be able to compile c extensions from sdists with the pre built python distributions from indygreg - otherwise. See https://github.com/indygreg/python-build-standalone/issues/103 + Pip won't be able to compile c extensions from sdists with the pre built python distributions from astral-sh + otherwise. See https://github.com/astral-sh/python-build-standalone/issues/103 """ # Only run on Unix systems diff --git a/python/private/python_repository.bzl b/python/private/python_repository.bzl index 075d4b1195..299dd36eae 100644 --- a/python/private/python_repository.bzl +++ b/python/private/python_repository.bzl @@ -161,7 +161,7 @@ def _python_repository_impl(rctx): python_bin = "python.exe" if ("windows" in platform) else "bin/python3" if "linux" in platform: - # Workaround around https://github.com/indygreg/python-build-standalone/issues/231 + # Workaround around https://github.com/astral-sh/python-build-standalone/issues/231 for url in urls: head_and_release, _, _ = url.rpartition("/") _, _, release = head_and_release.rpartition("/") @@ -177,7 +177,7 @@ def _python_repository_impl(rctx): # building on. # # Link to the first affected release: - # https://github.com/indygreg/python-build-standalone/releases/tag/20240224 + # https://github.com/astral-sh/python-build-standalone/releases/tag/20240224 rctx.delete("share/terminfo") break diff --git a/python/versions.bzl b/python/versions.bzl index 098362b7d3..b88aa47171 100644 --- a/python/versions.bzl +++ b/python/versions.bzl @@ -22,7 +22,7 @@ WINDOWS_NAME = "windows" FREETHREADED = "freethreaded" INSTALL_ONLY = "install_only" -DEFAULT_RELEASE_BASE_URL = "https://github.com/indygreg/python-build-standalone/releases/download" +DEFAULT_RELEASE_BASE_URL = "https://github.com/astral-sh/python-build-standalone/releases/download" # When updating the versions and releases, run the following command to get # the hashes: From 032f6aa738a673b13b605dabf55465c6fc1a56eb Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Tue, 18 Mar 2025 23:53:06 -0700 Subject: [PATCH 109/922] feat(rules): add main_module attribute to run a module name (python -m) (#2671) This implements the ability to run a module name instead of a file path, aka `python -m` style of invocation. This allows a binary/test to specify what the main module is without having to have a direct dependency on the entry point file. As a side effect, the `srcs` attribute is no longer required. Fixes https://github.com/bazelbuild/rules_python/issues/2539 --- CHANGELOG.md | 2 + python/private/py_executable.bzl | 36 ++++++- python/private/stage2_bootstrap_template.py | 114 ++++++++++++-------- tests/bootstrap_impls/BUILD.bazel | 9 ++ tests/bootstrap_impls/main_module.py | 17 +++ 5 files changed, 131 insertions(+), 47 deletions(-) create mode 100644 tests/bootstrap_impls/main_module.py diff --git a/CHANGELOG.md b/CHANGELOG.md index 15fb211ce8..7c6287da0b 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -99,6 +99,8 @@ Unreleased changes template. Only applicable for {obj}`--bootstrap_impl=script`. * (rules) Added {obj}`interpreter_args` attribute to `py_binary` and `py_test`, which allows pass arguments to the interpreter before the regular args. +* (rules) Added {obj}`main_module` attribute to `py_binary` and `py_test`, + which allows specifying a module name to run (i.e. `python -m `). {#v0-0-0-removed} ### Removed diff --git a/python/private/py_executable.bzl b/python/private/py_executable.bzl index bbaed3104e..d0ac3146ac 100644 --- a/python/private/py_executable.bzl +++ b/python/private/py_executable.bzl @@ -130,6 +130,24 @@ Optional; the name of the source file that is the main entry point of the application. This file must also be listed in `srcs`. If left unspecified, `name`, with `.py` appended, is used instead. If `name` does not match any filename in `srcs`, `main` must be specified. + +This is mutually exclusive with {obj}`main_module`. +""", + ), + "main_module": lambda: attrb.String( + doc = """ +Module name to execute as the main program. + +When set, `srcs` is not required, and it is assumed the module is +provided by a dependency. + +See https://docs.python.org/3/using/cmdline.html#cmdoption-m for more +information about running modules as the main program. + +This is mutually exclusive with {obj}`main`. + +:::{versionadded} VERSION_NEXT_FEATURE +::: """, ), "pyc_collection": lambda: attrb.String( @@ -642,6 +660,10 @@ def _create_stage2_bootstrap( template = runtime.stage2_bootstrap_template + if main_py: + main_py_path = "{}/{}".format(ctx.workspace_name, main_py.short_path) + else: + main_py_path = "" ctx.actions.expand_template( template = template, output = output, @@ -649,7 +671,8 @@ def _create_stage2_bootstrap( "%coverage_tool%": _get_coverage_tool_runfiles_path(ctx, runtime), "%import_all%": "True" if ctx.fragments.bazel_py.python_import_all_repositories else "False", "%imports%": ":".join(imports.to_list()), - "%main%": "{}/{}".format(ctx.workspace_name, main_py.short_path), + "%main%": main_py_path, + "%main_module%": ctx.attr.main_module, "%target%": str(ctx.label), "%workspace_name%": ctx.workspace_name, }, @@ -933,7 +956,10 @@ def py_executable_base_impl(ctx, *, semantics, is_test, inherited_environment = """ _validate_executable(ctx) - main_py = determine_main(ctx) + if not ctx.attr.main_module: + main_py = determine_main(ctx) + else: + main_py = None direct_sources = filter_to_py_srcs(ctx.files.srcs) precompile_result = semantics.maybe_precompile(ctx, direct_sources) @@ -1053,6 +1079,12 @@ def _validate_executable(ctx): if ctx.attr.python_version == "PY2": fail("It is not allowed to use Python 2") + if ctx.attr.main and ctx.attr.main_module: + fail(( + "Only one of main and main_module can be set, got: " + + "main={}, main_module={}" + ).format(ctx.attr.main, ctx.attr.main_module)) + def _declare_executable_file(ctx): if target_platform_has_any_constraint(ctx, ctx.attr._windows_constraints): executable = ctx.actions.declare_file(ctx.label.name + ".exe") diff --git a/python/private/stage2_bootstrap_template.py b/python/private/stage2_bootstrap_template.py index 4687bc003f..e8228edf3b 100644 --- a/python/private/stage2_bootstrap_template.py +++ b/python/private/stage2_bootstrap_template.py @@ -26,7 +26,11 @@ # We just put them in one place so its easy to tell which are used. # Runfiles-relative path to the main Python source file. -MAIN = "%main%" +# Empty if MAIN_MODULE is used +MAIN_PATH = "%main%" + +# Module name to execute. Empty if MAIN is used. +MAIN_MODULE = "%main_module%" # ===== Template substitutions end ===== @@ -249,7 +253,7 @@ def unresolve_symlinks(output_filename): os.unlink(unfixed_file) -def _run_py(main_filename, *, args, cwd=None): +def _run_py_path(main_filename, *, args, cwd=None): # type: (str, str, list[str], dict[str, str]) -> ... """Executes the given Python file using the various environment settings.""" @@ -269,6 +273,11 @@ def _run_py(main_filename, *, args, cwd=None): sys.argv = orig_argv +def _run_py_module(module_name): + # Match `python -m` behavior, so modify sys.argv and the run name + runpy.run_module(module_name, alter_sys=True, run_name="__main__") + + @contextlib.contextmanager def _maybe_collect_coverage(enable): print_verbose_coverage("enabled:", enable) @@ -356,64 +365,79 @@ def main(): print_verbose("initial environ:", mapping=os.environ) print_verbose("initial sys.path:", values=sys.path) - main_rel_path = MAIN - if is_windows(): - main_rel_path = main_rel_path.replace("/", os.sep) - - module_space = find_runfiles_root(main_rel_path) - print_verbose("runfiles root:", module_space) - - # Recreate the "add main's dir to sys.path[0]" behavior to match the - # system-python bootstrap / typical Python behavior. - # - # Without safe path enabled, when `python foo/bar.py` is run, python will - # resolve the foo/bar.py symlink to its real path, then add the directory - # of that path to sys.path. But, the resolved directory for the symlink - # depends on if the file is generated or not. - # - # When foo/bar.py is a source file, then it's a symlink pointing - # back to the client source directory. This means anything from that source - # directory becomes importable, i.e. most code is importable. - # - # When foo/bar.py is a generated file, then it's a symlink pointing to - # somewhere under bazel-out/.../bin, i.e. where generated files are. This - # means only other generated files are importable (not source files). - # - # To replicate this behavior, we add main's directory within the runfiles - # when safe path isn't enabled. - if not getattr(sys.flags, "safe_path", False): - prepend_path_entries = [ - os.path.join(module_space, os.path.dirname(main_rel_path)) - ] + main_rel_path = None + # todo: things happen to work because find_runfiles_root + # ends up using stage2_bootstrap, and ends up computing the proper + # runfiles root + if MAIN_PATH: + main_rel_path = MAIN_PATH + if is_windows(): + main_rel_path = main_rel_path.replace("/", os.sep) + + runfiles_root = find_runfiles_root(main_rel_path) else: - prepend_path_entries = [] + runfiles_root = find_runfiles_root("") + + print_verbose("runfiles root:", runfiles_root) - runfiles_envkey, runfiles_envvalue = runfiles_envvar(module_space) + runfiles_envkey, runfiles_envvalue = runfiles_envvar(runfiles_root) if runfiles_envkey: os.environ[runfiles_envkey] = runfiles_envvalue - main_filename = os.path.join(module_space, main_rel_path) - main_filename = get_windows_path_with_unc_prefix(main_filename) - assert os.path.exists(main_filename), ( - "Cannot exec() %r: file not found." % main_filename - ) - assert os.access(main_filename, os.R_OK), ( - "Cannot exec() %r: file not readable." % main_filename - ) + if MAIN_PATH: + # Recreate the "add main's dir to sys.path[0]" behavior to match the + # system-python bootstrap / typical Python behavior. + # + # Without safe path enabled, when `python foo/bar.py` is run, python will + # resolve the foo/bar.py symlink to its real path, then add the directory + # of that path to sys.path. But, the resolved directory for the symlink + # depends on if the file is generated or not. + # + # When foo/bar.py is a source file, then it's a symlink pointing + # back to the client source directory. This means anything from that source + # directory becomes importable, i.e. most code is importable. + # + # When foo/bar.py is a generated file, then it's a symlink pointing to + # somewhere under bazel-out/.../bin, i.e. where generated files are. This + # means only other generated files are importable (not source files). + # + # To replicate this behavior, we add main's directory within the runfiles + # when safe path isn't enabled. + if not getattr(sys.flags, "safe_path", False): + prepend_path_entries = [ + os.path.join(runfiles_root, os.path.dirname(main_rel_path)) + ] + else: + prepend_path_entries = [] + + main_filename = os.path.join(runfiles_root, main_rel_path) + main_filename = get_windows_path_with_unc_prefix(main_filename) + assert os.path.exists(main_filename), ( + "Cannot exec() %r: file not found." % main_filename + ) + assert os.access(main_filename, os.R_OK), ( + "Cannot exec() %r: file not readable." % main_filename + ) - sys.stdout.flush() + sys.stdout.flush() - sys.path[0:0] = prepend_path_entries + sys.path[0:0] = prepend_path_entries + else: + main_filename = None if os.environ.get("COVERAGE_DIR"): import _bazel_site_init + coverage_enabled = _bazel_site_init.COVERAGE_SETUP else: coverage_enabled = False with _maybe_collect_coverage(enable=coverage_enabled): - # The first arg is this bootstrap, so drop that for the re-invocation. - _run_py(main_filename, args=sys.argv[1:]) + if MAIN_PATH: + # The first arg is this bootstrap, so drop that for the re-invocation. + _run_py_path(main_filename, args=sys.argv[1:]) + else: + _run_py_module(MAIN_MODULE) sys.exit(0) diff --git a/tests/bootstrap_impls/BUILD.bazel b/tests/bootstrap_impls/BUILD.bazel index 7a5c4b46c6..e464a98e98 100644 --- a/tests/bootstrap_impls/BUILD.bazel +++ b/tests/bootstrap_impls/BUILD.bazel @@ -107,6 +107,15 @@ py_reconfig_test( main = "sys_path_order_test.py", ) +py_reconfig_test( + name = "main_module_test", + srcs = ["main_module.py"], + bootstrap_impl = "script", + imports = ["."], + main_module = "tests.bootstrap_impls.main_module", + target_compatible_with = SUPPORTS_BOOTSTRAP_SCRIPT, +) + sh_py_run_test( name = "inherit_pythonsafepath_env_test", bootstrap_impl = "script", diff --git a/tests/bootstrap_impls/main_module.py b/tests/bootstrap_impls/main_module.py new file mode 100644 index 0000000000..afb1ff6ba8 --- /dev/null +++ b/tests/bootstrap_impls/main_module.py @@ -0,0 +1,17 @@ +import sys +import unittest + + +class MainModuleTest(unittest.TestCase): + def test_run_as_module(self): + self.assertIsNotNone(__spec__, "__spec__ was none") + # If not run as a module, __spec__ is None + self.assertNotEqual(__name__, __spec__.name) + self.assertEqual(__spec__.name, "tests.bootstrap_impls.main_module") + + +if __name__ == "__main__": + unittest.main() +else: + # Guard against running it as a module in a non-main way. + sys.exit(f"__name__ should be __main__, got {__name__}") From 8396af0863aef47c7e411e31153350f8e215fec9 Mon Sep 17 00:00:00 2001 From: Ignas Anikevicius <240938+aignas@users.noreply.github.com> Date: Fri, 21 Mar 2025 01:42:29 +0900 Subject: [PATCH 110/922] fix: expose public attrb/ruleb bzl targets (#2682) PR #2666 forgot to add public load targets for the attr/rule builder apis and associated build targets for docs and bzl_library. --------- Co-authored-by: Richard Levasseur --- docs/BUILD.bazel | 2 ++ python/api/BUILD.bazel | 12 ++++++++++++ python/api/attr_builders.bzl | 5 +++++ python/api/rule_builders.bzl | 5 +++++ 4 files changed, 24 insertions(+) create mode 100644 python/api/attr_builders.bzl create mode 100644 python/api/rule_builders.bzl diff --git a/docs/BUILD.bazel b/docs/BUILD.bazel index 09de21b86a..ab996537c7 100644 --- a/docs/BUILD.bazel +++ b/docs/BUILD.bazel @@ -100,8 +100,10 @@ sphinx_stardocs( "//python:py_test_bzl", "//python:repositories_bzl", "//python/api:api_bzl", + "//python/api:attr_builders_bzl", "//python/api:executables_bzl", "//python/api:libraries_bzl", + "//python/api:rule_builders_bzl", "//python/cc:py_cc_toolchain_bzl", "//python/cc:py_cc_toolchain_info_bzl", "//python/entry_points:py_console_script_binary_bzl", diff --git a/python/api/BUILD.bazel b/python/api/BUILD.bazel index f0e04948ac..11fee103cb 100644 --- a/python/api/BUILD.bazel +++ b/python/api/BUILD.bazel @@ -25,6 +25,12 @@ bzl_library( deps = ["//python/private/api:api_bzl"], ) +bzl_library( + name = "attr_builders_bzl", + srcs = ["attr_builders.bzl"], + deps = ["//python/private:attr_builders_bzl"], +) + bzl_library( name = "executables_bzl", srcs = ["executables.bzl"], @@ -45,6 +51,12 @@ bzl_library( ], ) +bzl_library( + name = "rule_builders_bzl", + srcs = ["rule_builders.bzl"], + deps = ["//python/private:rule_builders_bzl"], +) + filegroup( name = "distribution", srcs = glob(["**"]), diff --git a/python/api/attr_builders.bzl b/python/api/attr_builders.bzl new file mode 100644 index 0000000000..573f9c6bc1 --- /dev/null +++ b/python/api/attr_builders.bzl @@ -0,0 +1,5 @@ +"""Public, attribute building APIs for Python rules.""" + +load("//python/private:attr_builders.bzl", _attrb = "attrb") + +attrb = _attrb diff --git a/python/api/rule_builders.bzl b/python/api/rule_builders.bzl new file mode 100644 index 0000000000..13ec4d39ea --- /dev/null +++ b/python/api/rule_builders.bzl @@ -0,0 +1,5 @@ +"""Public, rule building APIs for Python rules.""" + +load("//python/private:rule_builders.bzl", _ruleb = "ruleb") + +ruleb = _ruleb From d976228abe36bb08b58dccdfca398e0e660f37bd Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Thu, 20 Mar 2025 14:34:01 -0700 Subject: [PATCH 111/922] chore: update changelog and version markers for 1.3 release (#2683) Updates the changelog and VERSION_NEXT_XXX markers to specify 1.3.0 for the upcoming release. Work towards https://github.com/bazel-contrib/rules_python/pull/2683 --- CHANGELOG.md | 29 ++++++++++++++++++++--- docs/api/rules_python/python/bin/index.md | 2 +- docs/environment-variables.md | 2 +- python/api/executables.bzl | 2 +- python/api/libraries.bzl | 2 +- python/private/attr_builders.bzl | 2 +- python/private/py_binary_rule.bzl | 2 +- python/private/py_executable.bzl | 6 ++--- python/private/py_library.bzl | 2 +- python/private/py_test_rule.bzl | 2 +- python/private/pypi/attrs.bzl | 2 +- python/private/rule_builders.bzl | 2 +- 12 files changed, 39 insertions(+), 16 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 7c6287da0b..4e5f102b5a 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -45,6 +45,7 @@ Unreleased changes template. * Nothing removed. --> + {#v0-0-0} ## Unreleased @@ -52,6 +53,28 @@ Unreleased changes template. {#v0-0-0-changed} ### Changed +* Nothing changed. + +{#v0-0-0-fixed} +### Fixed +* Nothing fixed. + +{#v0-0-0-added} +### Added +* Nothing added. + +{#v0-0-0-removed} +### Removed +* Nothing removed. + + +{#v1-3-0} +## Unreleased + +[1.3.0]: https://github.com/bazelbuild/rules_python/releases/tag/1.3.0 + +{#v1-3-0-changed} +### Changed * (deps) platforms 0.0.4 -> 0.0.11 * (py_wheel) Package `py_library.pyi_srcs` (`.pyi` files) in the wheel. * (py_package) Package `py_library.pyi_srcs` (`.pyi` files) in `py_package`. @@ -59,7 +82,7 @@ Unreleased changes template. YAML document start `---` line. Implemented in [#2656](https://github.com/bazelbuild/rules_python/pull/2656). -{#v0-0-0-fixed} +{#v1-3-0-fixed} ### Fixed * (pypi) The `ppc64le` is now pointing to the right target in the `platforms` package. * (gazelle) No longer incorrectly merge `py_binary` targets during partial updates in @@ -76,7 +99,7 @@ Unreleased changes template. creating `.pyc` files. * (deps) doublestar 4.7.1 (required for recent Gazelle versions) -{#v0-0-0-added} +{#v1-3-0-added} ### Added * {obj}`//python/bin:python`: convenience target for directly running an interpreter. {obj}`--//python/bin:python_src` can be used to specify a @@ -102,7 +125,7 @@ Unreleased changes template. * (rules) Added {obj}`main_module` attribute to `py_binary` and `py_test`, which allows specifying a module name to run (i.e. `python -m `). -{#v0-0-0-removed} +{#v1-3-0-removed} ### Removed * Nothing removed. diff --git a/docs/api/rules_python/python/bin/index.md b/docs/api/rules_python/python/bin/index.md index ad6a4e7ed5..8bea6b54bd 100644 --- a/docs/api/rules_python/python/bin/index.md +++ b/docs/api/rules_python/python/bin/index.md @@ -30,7 +30,7 @@ bazel run @rules_python//python/bin:python \ The {flag}`--python_src` flag for using the intepreter a binary/test uses. :::: -::::{versionadded} VERSION_NEXT_FEATURE +::::{versionadded} 1.3.0 :::: ::: diff --git a/docs/environment-variables.md b/docs/environment-variables.md index c7c0181d18..d8735cb2d5 100644 --- a/docs/environment-variables.md +++ b/docs/environment-variables.md @@ -24,7 +24,7 @@ python /path/to/debugger.py --port 12345 --file /path/to/file.py The {bzl:obj}`interpreter_args` attribute. ::: -:::{versionadded} VERSION_NEXT_FEATURE +:::{versionadded} 1.3.0 :::: diff --git a/python/api/executables.bzl b/python/api/executables.bzl index 4715c0f481..99bb7cc603 100644 --- a/python/api/executables.bzl +++ b/python/api/executables.bzl @@ -16,7 +16,7 @@ {#python-apis-executables-bzl} Loading-phase APIs specific to executables (binaries/tests). -:::{versionadded} VERSION_NEXT_FEATURE +:::{versionadded} 1.3.0 ::: """ diff --git a/python/api/libraries.bzl b/python/api/libraries.bzl index c4ad598e3f..0b470a9ad4 100644 --- a/python/api/libraries.bzl +++ b/python/api/libraries.bzl @@ -16,7 +16,7 @@ {#python-apis-libraries-bzl} Loading-phase APIs specific to libraries. -:::{versionadded} VERSION_NEXT_FEATURE +:::{versionadded} 1.3.0 ::: """ diff --git a/python/private/attr_builders.bzl b/python/private/attr_builders.bzl index efcbfa6e5b..57fe476109 100644 --- a/python/private/attr_builders.bzl +++ b/python/private/attr_builders.bzl @@ -14,7 +14,7 @@ """Builders for creating attributes et al. -:::{versionadded} VERSION_NEXT_FEATURE +:::{versionadded} 1.3.0 ::: """ diff --git a/python/private/py_binary_rule.bzl b/python/private/py_binary_rule.bzl index 38e3a697c7..3df6bd87c4 100644 --- a/python/private/py_binary_rule.bzl +++ b/python/private/py_binary_rule.bzl @@ -34,7 +34,7 @@ def create_py_binary_rule_builder(): :::{include} /_includes/volatile_api.md ::: - :::{versionadded} VERSION_NEXT_FEATURE + :::{versionadded} 1.3.0 ::: Returns: diff --git a/python/private/py_executable.bzl b/python/private/py_executable.bzl index d0ac3146ac..d54a3d7f24 100644 --- a/python/private/py_executable.bzl +++ b/python/private/py_executable.bzl @@ -102,7 +102,7 @@ Only supported for {obj}`--bootstrap_impl=script`. Ignored otherwise. The {obj}`RULES_PYTHON_ADDITIONAL_INTERPRETER_ARGS` environment variable ::: -:::{versionadded} VERSION_NEXT_FEATURE +:::{versionadded} 1.3.0 ::: """, ), @@ -146,7 +146,7 @@ information about running modules as the main program. This is mutually exclusive with {obj}`main`. -:::{versionadded} VERSION_NEXT_FEATURE +:::{versionadded} 1.3.0 ::: """, ), @@ -1803,7 +1803,7 @@ def create_executable_rule_builder(implementation, **kwargs): and the output is something that can be run directly (e.g. `bazel run`, `exec(...)` etc) - :::{versionadded} VERSION_NEXT_FEATURE + :::{versionadded} 1.3.0 ::: Returns: diff --git a/python/private/py_library.bzl b/python/private/py_library.bzl index 7b024a0f07..f6c7b12578 100644 --- a/python/private/py_library.bzl +++ b/python/private/py_library.bzl @@ -151,7 +151,7 @@ def create_py_library_rule_builder(): :::{include} /_includes/volatile_api.md ::: - :::{versionadded} VERSION_NEXT_FEATURE + :::{versionadded} 1.3.0 ::: Returns: diff --git a/python/private/py_test_rule.bzl b/python/private/py_test_rule.bzl index f21fdc7557..bb35d6974e 100644 --- a/python/private/py_test_rule.bzl +++ b/python/private/py_test_rule.bzl @@ -37,7 +37,7 @@ def create_py_test_rule_builder(): :::{include} /_includes/volatile_api.md ::: - :::{versionadded} VERSION_NEXT_FEATURE + :::{versionadded} 1.3.0 ::: Returns: diff --git a/python/private/pypi/attrs.bzl b/python/private/pypi/attrs.bzl index 6717e9528c..9d88c1e32c 100644 --- a/python/private/pypi/attrs.bzl +++ b/python/private/pypi/attrs.bzl @@ -20,7 +20,7 @@ ATTRS = { doc = """ If true, add the lib dir of the bundled interpreter to the library search path via `LDFLAGS`. -:::{versionadded} VERSION_NEXT_FEATURE +:::{versionadded} 1.3.0 ::: """, ), diff --git a/python/private/rule_builders.bzl b/python/private/rule_builders.bzl index 4607285949..9b7c03136c 100644 --- a/python/private/rule_builders.bzl +++ b/python/private/rule_builders.bzl @@ -92,7 +92,7 @@ def create_custom_foo_binary(): custom_foo_binary = create_custom_foo_binary() ``` -:::{versionadded} VERSION_NEXT_FEATURE +:::{versionadded} 1.3.0 ::: """ From e6f79dc0cf8b8720336f4a5141369612c8478e08 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Thu, 20 Mar 2025 16:22:53 -0700 Subject: [PATCH 112/922] chore: ignore releasing.md for version string check (#2684) The RELEASING.md docs contain the VERSION_NEXT marker string in their docs, so also have to be ignored by the release script. --- .github/workflows/create_archive_and_notes.sh | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/.github/workflows/create_archive_and_notes.sh b/.github/workflows/create_archive_and_notes.sh index dc7f8a6982..26091a8989 100755 --- a/.github/workflows/create_archive_and_notes.sh +++ b/.github/workflows/create_archive_and_notes.sh @@ -17,8 +17,8 @@ set -o errexit -o nounset -o pipefail # Exclude dot directories, specifically, this file so that we don't # find the substring we're looking for in our own file. -# Exclude CONTRIBUTING.md because it documents how to use these strings. -if grep --exclude=CONTRIBUTING.md --exclude-dir=.* VERSION_NEXT_ -r; then +# Exclude CONTRIBUTING.md, RELEASING.md because they document how to use these strings. +if grep --exclude=CONTRIBUTING.md --exclude=RELEASING.md --exclude-dir=.* VERSION_NEXT_ -r; then echo echo "Found VERSION_NEXT markers indicating version needs to be specified" exit 1 From 14b559b569b6d21ddc723a2116a65adae3b97b5b Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Thu, 20 Mar 2025 18:59:32 -0700 Subject: [PATCH 113/922] chore: replace bazelbuild with bazel-contrib (#2688) This was done using `grep | xargs sed`. BCR presubmits require that the list of repositories match where downloads come from Along the way, also update the URL homepages to bazel-contrib and change the email to my personal instead of work email. --- .bcr/gazelle/metadata.template.json | 7 +- .bcr/metadata.template.json | 4 +- .github/workflows/create_archive_and_notes.sh | 4 +- .github/workflows/release.yml | 2 +- BZLMOD_SUPPORT.md | 4 +- CHANGELOG.md | 202 +++++++++--------- CONTRIBUTING.md | 2 +- RELEASING.md | 2 +- WORKSPACE | 4 +- .../python/config_settings/index.md | 2 +- docs/conf.py | 4 +- docs/extending.md | 2 +- docs/getting-started.md | 6 +- docs/pypi-dependencies.md | 4 +- docs/toolchains.md | 8 +- examples/build_file_generation/WORKSPACE | 4 +- examples/bzlmod/py_proto_library/BUILD.bazel | 2 +- .../bzlmod_build_file_generation/MODULE.bazel | 4 +- examples/pip_parse_vendored/README.md | 2 +- gazelle/BUILD.bazel | 2 +- gazelle/README.md | 8 +- gazelle/go.mod | 2 +- gazelle/manifest/BUILD.bazel | 2 +- gazelle/manifest/generate/BUILD.bazel | 2 +- gazelle/manifest/generate/generate.go | 2 +- gazelle/manifest/hasher/BUILD.bazel | 2 +- gazelle/manifest/manifest_test.go | 2 +- gazelle/manifest/test/test.go | 2 +- gazelle/python/BUILD.bazel | 2 +- gazelle/python/configure.go | 4 +- gazelle/python/generate.go | 2 +- gazelle/python/resolve.go | 2 +- .../README.md | 2 +- .../README.md | 2 +- .../README.md | 2 +- gazelle/pythonconfig/BUILD.bazel | 2 +- gazelle/pythonconfig/pythonconfig.go | 2 +- python/packaging.bzl | 2 +- python/private/py_cc_toolchain_rule.bzl | 2 +- python/private/py_console_script_gen.py | 4 +- python/private/py_runtime_rule.bzl | 2 +- python/private/pypi/patch_whl.bzl | 2 +- python/private/pypi/pip_repository.bzl | 4 +- .../pypi/whl_installer/namespace_pkgs.py | 2 +- python/private/pypi/whl_installer/wheel.py | 2 +- python/private/python_repository.bzl | 6 +- .../runtime_env_toolchain_interpreter.sh | 2 +- python/private/stage1_bootstrap_template.sh | 2 +- python/py_binary.bzl | 4 +- python/py_library.bzl | 2 +- python/py_runtime.bzl | 2 +- python/py_runtime_pair.bzl | 2 +- python/py_test.bzl | 4 +- python/runfiles/BUILD.bazel | 4 +- sphinxdocs/docs/readthedocs.md | 2 +- tests/integration/custom_commands_test.py | 2 +- tests/no_unsafe_paths/test.py | 2 +- tests/packaging/BUILD.bazel | 2 +- .../pycross/private/tools/wheel_installer.py | 2 +- 59 files changed, 186 insertions(+), 185 deletions(-) diff --git a/.bcr/gazelle/metadata.template.json b/.bcr/gazelle/metadata.template.json index 687f78e977..017f9d3774 100644 --- a/.bcr/gazelle/metadata.template.json +++ b/.bcr/gazelle/metadata.template.json @@ -1,9 +1,9 @@ { - "homepage": "https://github.com/bazelbuild/rules_python", + "homepage": "https://github.com/bazel-contrib/rules_python", "maintainers": [ { "name": "Richard Levasseur", - "email": "rlevasseur@google.com", + "email": "richardlev@gmail.com", "github": "rickeylev" }, { @@ -13,7 +13,8 @@ } ], "repository": [ - "github:bazelbuild/rules_python" + "github:bazelbuild/rules_python", + "github:bazel-contrib/rules_python" ], "versions": [], "yanked_versions": {} diff --git a/.bcr/metadata.template.json b/.bcr/metadata.template.json index 579d6884cd..9d85e22200 100644 --- a/.bcr/metadata.template.json +++ b/.bcr/metadata.template.json @@ -1,9 +1,9 @@ { - "homepage": "https://github.com/bazelbuild/rules_python", + "homepage": "https://github.com/bazel-contrib/rules_python", "maintainers": [ { "name": "Richard Levasseur", - "email": "rlevasseur@google.com", + "email": "richardlev@gmail.com", "github": "rickeylev" }, { diff --git a/.github/workflows/create_archive_and_notes.sh b/.github/workflows/create_archive_and_notes.sh index 26091a8989..a21585f866 100755 --- a/.github/workflows/create_archive_and_notes.sh +++ b/.github/workflows/create_archive_and_notes.sh @@ -72,7 +72,7 @@ http_archive( name = "rules_python", sha256 = "${SHA}", strip_prefix = "${PREFIX}", - url = "https://github.com/bazelbuild/rules_python/releases/download/${TAG}/rules_python-${TAG}.tar.gz", + url = "https://github.com/bazel-contrib/rules_python/releases/download/${TAG}/rules_python-${TAG}.tar.gz", ) load("@rules_python//python:repositories.bzl", "py_repositories") @@ -90,7 +90,7 @@ http_archive( name = "rules_python_gazelle_plugin", sha256 = "${SHA}", strip_prefix = "${PREFIX}/gazelle", - url = "https://github.com/bazelbuild/rules_python/releases/download/${TAG}/rules_python-${TAG}.tar.gz", + url = "https://github.com/bazel-contrib/rules_python/releases/download/${TAG}/rules_python-${TAG}.tar.gz", ) # To compile the rules_python gazelle extension from source, diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index 29b70ccc8f..436797e3ed 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -33,7 +33,7 @@ jobs: # This special value tells pypi that the user identity is supplied within the token TWINE_USERNAME: __token__ # Note, the PYPI_API_TOKEN is for the rules-python pypi user, added by @rickylev on - # https://github.com/bazelbuild/rules_python/settings/secrets/actions + # https://github.com/bazel-contrib/rules_python/settings/secrets/actions TWINE_PASSWORD: ${{ secrets.PYPI_API_TOKEN }} run: bazel run --stamp --embed_label=${{ github.ref_name }} //python/runfiles:wheel.publish - name: Release diff --git a/BZLMOD_SUPPORT.md b/BZLMOD_SUPPORT.md index 85e28acb1a..73fde463b7 100644 --- a/BZLMOD_SUPPORT.md +++ b/BZLMOD_SUPPORT.md @@ -11,7 +11,7 @@ In general `bzlmod` has more features than `WORKSPACE` and users are encouraged ## Configuration -The releases page will give you the latest version number, and a basic example. The release page is located [here](/bazelbuild/rules_python/releases). +The releases page will give you the latest version number, and a basic example. The release page is located [here](/bazel-contrib/rules_python/releases). ## What is bzlmod? @@ -53,7 +53,7 @@ better supported. the toolchains rules_python registers**. NOTE: Regardless of your toolchain, due to -[#691](https://github.com/bazelbuild/rules_python/issues/691), `rules_python` +[#691](https://github.com/bazel-contrib/rules_python/issues/691), `rules_python` still relies on a local Python being available to bootstrap the program before handing over execution to the toolchain Python. diff --git a/CHANGELOG.md b/CHANGELOG.md index 4e5f102b5a..dc40a25961 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -26,7 +26,7 @@ Unreleased changes template. {#v0-0-0} ## Unreleased -[0.0.0]: https://github.com/bazelbuild/rules_python/releases/tag/0.0.0 +[0.0.0]: https://github.com/bazel-contrib/rules_python/releases/tag/0.0.0 {#v0-0-0-changed} ### Changed @@ -49,7 +49,7 @@ Unreleased changes template. {#v0-0-0} ## Unreleased -[0.0.0]: https://github.com/bazelbuild/rules_python/releases/tag/0.0.0 +[0.0.0]: https://github.com/bazel-contrib/rules_python/releases/tag/0.0.0 {#v0-0-0-changed} ### Changed @@ -71,7 +71,7 @@ Unreleased changes template. {#v1-3-0} ## Unreleased -[1.3.0]: https://github.com/bazelbuild/rules_python/releases/tag/1.3.0 +[1.3.0]: https://github.com/bazel-contrib/rules_python/releases/tag/1.3.0 {#v1-3-0-changed} ### Changed @@ -80,17 +80,17 @@ Unreleased changes template. * (py_package) Package `py_library.pyi_srcs` (`.pyi` files) in `py_package`. * (gazelle) The generated manifest file (default: `gazelle_python.yaml`) will now include the YAML document start `---` line. Implemented in - [#2656](https://github.com/bazelbuild/rules_python/pull/2656). + [#2656](https://github.com/bazel-contrib/rules_python/pull/2656). {#v1-3-0-fixed} ### Fixed * (pypi) The `ppc64le` is now pointing to the right target in the `platforms` package. * (gazelle) No longer incorrectly merge `py_binary` targets during partial updates in - `file` generation mode. Fixed in [#2619](https://github.com/bazelbuild/rules_python/pull/2619). + `file` generation mode. Fixed in [#2619](https://github.com/bazel-contrib/rules_python/pull/2619). * (bzlmod) Running as root is no longer an error. `ignore_root_user_error=True` is now the default. Note that running as root may still cause spurious Bazel cache invalidation - ([#1169](https://github.com/bazelbuild/rules_python/issues/1169)). + ([#1169](https://github.com/bazel-contrib/rules_python/issues/1169)). * (gazelle) Don't collapse depsets to a list or into args when generating the modules mapping file. Support spilling modules mapping args into a params file. * (coverage) Fix missing files in the coverage report if they have no tests. @@ -113,10 +113,10 @@ Unreleased changes template. building wheels from `sdist`. * (pypi) Direct HTTP urls for wheels and sdists are now supported when using {obj}`experimental_index_url` (bazel downloader). - Partially fixes [#2363](https://github.com/bazelbuild/rules_python/issues/2363). + Partially fixes [#2363](https://github.com/bazel-contrib/rules_python/issues/2363). * (rules) APIs for creating custom rules based on the core py_binary, py_test, and py_library rules - ([#1647](https://github.com/bazelbuild/rules_python/issues/1647)) + ([#1647](https://github.com/bazel-contrib/rules_python/issues/1647)) * (rules) Added env-var to allow additional interpreter args for stage1 bootstrap. See {obj}`RULES_PYTHON_ADDITIONAL_INTERPRETER_ARGS` environment variable. Only applicable for {obj}`--bootstrap_impl=script`. @@ -132,7 +132,7 @@ Unreleased changes template. {#v1-2-0} ## [1.2.0] - 2025-02-21 -[1.2.0]: https://github.com/bazelbuild/rules_python/releases/tag/1.2.0 +[1.2.0]: https://github.com/bazel-contrib/rules_python/releases/tag/1.2.0 {#v1-2-0-changed} ### Changed @@ -140,7 +140,7 @@ Unreleased changes template. implementation in https://github.com/protocolbuffers/protobuf. It will be removed in the future release. * (pypi) {obj}`pip.override` will now be ignored instead of raising an error, - fixes [#2550](https://github.com/bazelbuild/rules_python/issues/2550). + fixes [#2550](https://github.com/bazel-contrib/rules_python/issues/2550). * (rules) deprecation warnings for deprecated symbols have been turned off by default for now and can be enabled with `RULES_PYTHON_DEPRECATION_WARNINGS` env var. @@ -150,24 +150,24 @@ Unreleased changes template. {#v1-2-0-fixed} ### Fixed * (rules) `python_zip_file` output with `--bootstrap_impl=script` works again - ([#2596](https://github.com/bazelbuild/rules_python/issues/2596)). + ([#2596](https://github.com/bazel-contrib/rules_python/issues/2596)). * (docs) Using `python_version` attribute for specifying python versions introduced in `v1.1.0` * (gazelle) Providing multiple input requirements files to `gazelle_python_manifest` now works correctly. * (pypi) Handle trailing slashes in pip index URLs in environment variables, - fixes [#2554](https://github.com/bazelbuild/rules_python/issues/2554). + fixes [#2554](https://github.com/bazel-contrib/rules_python/issues/2554). * (runfiles) Runfile manifest and repository mapping files are now interpreted as UTF-8 on all platforms. * (coverage) Coverage with `--bootstrap_impl=script` is fixed - ([#2572](https://github.com/bazelbuild/rules_python/issues/2572)). + ([#2572](https://github.com/bazel-contrib/rules_python/issues/2572)). * (pypi) Non deterministic behaviour in requirement file usage has been fixed - by reverting [#2514](https://github.com/bazelbuild/rules_python/pull/2514). - The related issue is [#908](https://github.com/bazelbuild/rules_python/issue/908). + by reverting [#2514](https://github.com/bazel-contrib/rules_python/pull/2514). + The related issue is [#908](https://github.com/bazel-contrib/rules_python/issue/908). * (sphinxdocs) Do not crash when `tag_class` does not have a populated `doc` value. - Fixes ([#2579](https://github.com/bazelbuild/rules_python/issues/2579)). + Fixes ([#2579](https://github.com/bazel-contrib/rules_python/issues/2579)). * (binaries/tests) Fix packaging when using `--bootstrap_impl=script`: set {obj}`--venvs_use_declare_symlink=no` to have it not create symlinks at build time (they will be created at runtime instead). - (Fixes [#2489](https://github.com/bazelbuild/rules_python/issues/2489)) + (Fixes [#2489](https://github.com/bazel-contrib/rules_python/issues/2489)) {#v1-2-0-added} ### Added @@ -180,7 +180,7 @@ Unreleased changes template. {#v1-1-0} ## [1.1.0] - 2025-01-07 -[1.1.0]: https://github.com/bazelbuild/rules_python/releases/tag/1.1.0 +[1.1.0]: https://github.com/bazel-contrib/rules_python/releases/tag/1.1.0 {#v1-1-0-changed} ### Changed @@ -213,7 +213,7 @@ Unreleased changes template. marker information allowing `bazel query` to work in cases where the `whl` is available for all of the platforms and the sdist can be built. This fix is for both WORKSPACE and `bzlmod` setups. - Fixes [#2450](https://github.com/bazelbuild/rules_python/issues/2450). + Fixes [#2450](https://github.com/bazel-contrib/rules_python/issues/2450). * (gazelle) Gazelle will now correctly parse Python3.12 files that use [PEP 695 Type Parameter Syntax][pep-695]. (#2396) * (pypi) Using {bzl:obj}`pip_parse.experimental_requirement_cycles` and @@ -221,16 +221,16 @@ Unreleased changes template. using WORKSPACE files. * (pypi) The error messages when the wheel distributions do not match anything are now printing more details and include the currently active flag - values. Fixes [#2466](https://github.com/bazelbuild/rules_python/issues/2466). + values. Fixes [#2466](https://github.com/bazel-contrib/rules_python/issues/2466). * (py_proto_library) Fix import paths in Bazel 8. * (whl_library) Now the changes to the dependencies are correctly tracked when PyPI packages used in {bzl:obj}`whl_library` during the `repository_rule` phase - change. Fixes [#2468](https://github.com/bazelbuild/rules_python/issues/2468). + change. Fixes [#2468](https://github.com/bazel-contrib/rules_python/issues/2468). + (gazelle) Gazelle no longer ignores `setup.py` files by default. To restore this behavior, apply the `# gazelle:python_ignore_files setup.py` directive. * Don't re-fetch whl_library, python_repository, etc. repository rules whenever `PATH` changes. Fixes - [#2551](https://github.com/bazelbuild/rules_python/issues/2551). + [#2551](https://github.com/bazel-contrib/rules_python/issues/2551). [pep-695]: https://peps.python.org/pep-0695/ @@ -244,7 +244,7 @@ Unreleased changes template. {obj}`experimental_index_url` usage or the regular `pip.parse` usage. To select the free-threaded interpreter in the repo phase, please use the documented [env](/environment-variables.html) variables. - Fixes [#2386](https://github.com/bazelbuild/rules_python/issues/2386). + Fixes [#2386](https://github.com/bazel-contrib/rules_python/issues/2386). * (toolchains) Use the latest astrahl-sh toolchain release [20241206] for Python versions: * 3.9.21 * 3.10.16 @@ -269,7 +269,7 @@ Unreleased changes template. {#v1-0-0} ## [1.0.0] - 2024-12-05 -[1.0.0]: https://github.com/bazelbuild/rules_python/releases/tag/1.0.0 +[1.0.0]: https://github.com/bazel-contrib/rules_python/releases/tag/1.0.0 {#v1-0-0-changed} ### Changed @@ -308,12 +308,12 @@ Other changes: * (toolchains) stop depending on `uname` to get the value of the host platform. * (pypi): Correctly handle multiple versions of the same package in the requirements files which is useful when including different PyTorch builds (e.g. vs ) for different target platforms. - Fixes ([2337](https://github.com/bazelbuild/rules_python/issues/2337)). + Fixes ([2337](https://github.com/bazel-contrib/rules_python/issues/2337)). * (uv): Correct the sha256sum for the `uv` binary for aarch64-apple-darwin. - Fixes ([2411](https://github.com/bazelbuild/rules_python/issues/2411)). + Fixes ([2411](https://github.com/bazel-contrib/rules_python/issues/2411)). * (binaries/tests) ({obj}`--bootstrap_impl=scipt`) Using `sys.executable` will use the same `sys.path` setup as the calling binary. - ([2169](https://github.com/bazelbuild/rules_python/issues/2169)). + ([2169](https://github.com/bazel-contrib/rules_python/issues/2169)). * (workspace) Corrected protobuf's name to com_google_protobuf, the name is hardcoded in Bazel, WORKSPACE mode. * (pypi): {bzl:obj}`compile_pip_requirements` no longer fails on Windows when `--enable_runfiles` is not enabled. @@ -352,7 +352,7 @@ Other changes: {#v0-40-0} ## [0.40.0] - 2024-11-17 -[0.40.0]: https://github.com/bazelbuild/rules_python/releases/tag/0.40.0 +[0.40.0]: https://github.com/bazel-contrib/rules_python/releases/tag/0.40.0 {#v0-40-changed} ### Changed @@ -361,7 +361,7 @@ Other changes: {#v0-40-fixed} ### Fixed * (rules) Don't drop custom import paths if Bazel-builtin PyInfo is removed. - ([2414](https://github.com/bazelbuild/rules_python/issues/2414)). + ([2414](https://github.com/bazel-contrib/rules_python/issues/2414)). {#v0-40-added} ### Added @@ -380,7 +380,7 @@ Other changes: {#v0-39-0} ## [0.39.0] - 2024-11-13 -[0.39.0]: https://github.com/bazelbuild/rules_python/releases/tag/0.39.0 +[0.39.0]: https://github.com/bazel-contrib/rules_python/releases/tag/0.39.0 {#v0-39-0-changed} ### Changed @@ -408,7 +408,7 @@ Other changes: ### Fixed * (precompiling) Skip precompiling (instead of erroring) if the legacy `@bazel_tools//tools/python:autodetecting_toolchain` is being used - ([#2364](https://github.com/bazelbuild/rules_python/issues/2364)). + ([#2364](https://github.com/bazel-contrib/rules_python/issues/2364)). {#v0-39-0-added} ### Added @@ -426,14 +426,14 @@ Other changes: {#v0-38-0} ## [0.38.0] - 2024-11-08 -[0.38.0]: https://github.com/bazelbuild/rules_python/releases/tag/0.38.0 +[0.38.0]: https://github.com/bazel-contrib/rules_python/releases/tag/0.38.0 {#v0-38-0-changed} ### Changed * (deps) (WORKSPACE only) rules_cc 0.0.13 and protobuf 27.0 is now the default version used; this for Bazel 8+ support (previously version was rules_cc 0.0.9 and no protobuf version specified) - ([2310](https://github.com/bazelbuild/rules_python/issues/2310)). + ([2310](https://github.com/bazel-contrib/rules_python/issues/2310)). * (publish) The dependencies have been updated to the latest available versions for the `twine` publishing rule. * (whl_library) Remove `--no-build-isolation` to allow non-hermetic sdist builds @@ -452,7 +452,7 @@ Other changes: {#v0-38-0-fixed} ### Fixed * (pypi) (Bazel 7.4+) Allow spaces in filenames included in `whl_library`s - ([617](https://github.com/bazelbuild/rules_python/issues/617)). + ([617](https://github.com/bazel-contrib/rules_python/issues/617)). * (pypi) When {attr}`pip.parse.experimental_index_url` is set, we need to still pass the `extra_pip_args` value when building an `sdist`. * (pypi) The patched wheel filenames from now on are using local version specifiers @@ -462,7 +462,7 @@ Other changes: or not. To opt into this behavior, set `pip.parse.parse_all_requirements_files`, which will become the default in future releases leading up to `1.0.0`. Fixes - [#2268](https://github.com/bazelbuild/rules_python/issues/2268). A known + [#2268](https://github.com/bazel-contrib/rules_python/issues/2268). A known issue is that it may break `bazel query` and in these use cases it is advisable to use `cquery` or switch to `download_only = True` @@ -476,7 +476,7 @@ Other changes: * The rules_python version is now reported in `//python/features.bzl#features.version` * (pip.parse) {attr}`pip.parse.extra_hub_aliases` can now be used to expose extra targets created by annotations in whl repositories. - Fixes [#2187](https://github.com/bazelbuild/rules_python/issues/2187). + Fixes [#2187](https://github.com/bazel-contrib/rules_python/issues/2187). * (bzlmod) `pip.parse` now supports `whl-only` setup using `download_only = True` where users can specify multiple requirements files and use the `pip` backend to do the downloading. This was only available for @@ -486,7 +486,7 @@ Other changes: {#v0-37-2} ## [0.37.2] - 2024-10-27 -[0.37.2]: https://github.com/bazelbuild/rules_python/releases/tag/0.37.2 +[0.37.2]: https://github.com/bazel-contrib/rules_python/releases/tag/0.37.2 {#v0-37-2-fixed} ### Fixed @@ -497,18 +497,18 @@ Other changes: {#v0-37-1} ## [0.37.1] - 2024-10-22 -[0.37.1]: https://github.com/bazelbuild/rules_python/releases/tag/0.37.1 +[0.37.1]: https://github.com/bazel-contrib/rules_python/releases/tag/0.37.1 {#v0-37-1-fixed} ### Fixed * (rules) Setting `--incompatible_python_disallow_native_rules` no longer causes rules_python rules to fail - ([#2326](https://github.com/bazelbuild/rules_python/issues/2326)). + ([#2326](https://github.com/bazel-contrib/rules_python/issues/2326)). {#v0-37-0} ## [0.37.0] - 2024-10-18 -[0.37.0]: https://github.com/bazelbuild/rules_python/releases/tag/0.37.0 +[0.37.0]: https://github.com/bazel-contrib/rules_python/releases/tag/0.37.0 {#v0-37-0-changed} ### Changed @@ -538,7 +538,7 @@ Other changes: way to {obj}`whl_library`. What is more we will pass the `extra_pip_args` to {obj}`whl_library` for `sdist` distributions when using {attr}`pip.parse.experimental_index_url`. See - [#2239](https://github.com/bazelbuild/rules_python/issues/2239). + [#2239](https://github.com/bazel-contrib/rules_python/issues/2239). * (whl_filegroup): Provide per default also the `RECORD` file * (py_wheel): `RECORD` file entry elements are now quoted if necessary when a wheel is created @@ -546,17 +546,17 @@ Other changes: case where a requirement has many `--hash=sha256:...` flags * (rules) `compile_pip_requirements` passes `env` to the `X.update` target (and not only to the `X_test` target, a bug introduced in - [#1067](https://github.com/bazelbuild/rules_python/pull/1067)). + [#1067](https://github.com/bazel-contrib/rules_python/pull/1067)). * (bzlmod) In hybrid bzlmod with WORKSPACE builds, `python_register_toolchains(register_toolchains=True)` is respected - ([#1675](https://github.com/bazelbuild/rules_python/issues/1675)). + ([#1675](https://github.com/bazel-contrib/rules_python/issues/1675)). * (precompiling) The {obj}`pyc_collection` attribute now correctly enables (or disables) using pyc files from targets transitively * (pip) Skip patching wheels not matching `pip.override`'s `file` - ([#2294](https://github.com/bazelbuild/rules_python/pull/2294)). + ([#2294](https://github.com/bazel-contrib/rules_python/pull/2294)). * (chore): Add a `rules_shell` dev dependency and moved a `sh_test` target outside of the `//:BUILD.bazel` file. - Fixes [#2299](https://github.com/bazelbuild/rules_python/issues/2299). + Fixes [#2299](https://github.com/bazel-contrib/rules_python/issues/2299). {#v0-37-0-added} ### Added @@ -593,7 +593,7 @@ Other changes: {#v0-36-0} ## [0.36.0] - 2024-09-24 -[0.36.0]: https://github.com/bazelbuild/rules_python/releases/tag/0.36.0 +[0.36.0]: https://github.com/bazel-contrib/rules_python/releases/tag/0.36.0 {#v0-36-0-changed} ### Changed @@ -632,7 +632,7 @@ Other changes: * (rules) Make `RUNFILES_MANIFEST_FILE`-based invocations work when used with {obj}`--bootstrap_impl=script`. This fixes invocations using non-sandboxed test execution with `--enable_runfiles=false --build_runfile_manifests=true`. - ([#2186](https://github.com/bazelbuild/rules_python/issues/2186)). + ([#2186](https://github.com/bazel-contrib/rules_python/issues/2186)). * (py_wheel) Fix incorrectly generated `Required-Dist` when specifying requirements with markers in extra_requires in py_wheel rule. * (rules) Prevent pytest from trying run the generated stage2 @@ -645,7 +645,7 @@ Other changes: * (bzlmod): Toolchain overrides can now be done using the new {bzl:obj}`python.override`, {bzl:obj}`python.single_version_override` and {bzl:obj}`python.single_version_platform_override` tag classes. - See [#2081](https://github.com/bazelbuild/rules_python/issues/2081). + See [#2081](https://github.com/bazel-contrib/rules_python/issues/2081). * (rules) Executables provide {obj}`PyExecutableInfo`, which contains executable-specific information useful for packaging an executable or or deriving a new one from the original. @@ -671,7 +671,7 @@ Other changes: {#v0-35-0} ## [0.35.0] - 2024-08-15 -[0.35.0]: https://github.com/bazelbuild/rules_python/releases/tag/0.35.0 +[0.35.0]: https://github.com/bazel-contrib/rules_python/releases/tag/0.35.0 {#v0-35-0-changed} ### Changed @@ -685,7 +685,7 @@ Other changes: * `3.12 -> 3.12.4` * (rules) `PYTHONSAFEPATH` is inherited from the calling environment to allow disabling it (Requires {obj}`--bootstrap_impl=script`) - ([#2060](https://github.com/bazelbuild/rules_python/issues/2060)). + ([#2060](https://github.com/bazel-contrib/rules_python/issues/2060)). {#v0-35-0-fixed} ### Fixed @@ -699,42 +699,42 @@ Other changes: execroot. * (rules) Signals are properly received when using {obj}`--bootstrap_impl=script` (for non-zip builds). - ([#2043](https://github.com/bazelbuild/rules_python/issues/2043)) + ([#2043](https://github.com/bazel-contrib/rules_python/issues/2043)) * (rules) Fixes Python builds when the `--build_python_zip` is set to `false` on - Windows. See [#1840](https://github.com/bazelbuild/rules_python/issues/1840). + Windows. See [#1840](https://github.com/bazel-contrib/rules_python/issues/1840). * (rules) Fixes Mac + `--build_python_zip` + {obj}`--bootstrap_impl=script` - ([#2030](https://github.com/bazelbuild/rules_python/issues/2030)). + ([#2030](https://github.com/bazel-contrib/rules_python/issues/2030)). * (rules) User dependencies come before runtime site-packages when using {obj}`--bootstrap_impl=script`. - ([#2064](https://github.com/bazelbuild/rules_python/issues/2064)). + ([#2064](https://github.com/bazel-contrib/rules_python/issues/2064)). * (rules) Version-aware rules now return both `@_builtins` and `@rules_python` providers instead of only one. - ([#2114](https://github.com/bazelbuild/rules_python/issues/2114)). + ([#2114](https://github.com/bazel-contrib/rules_python/issues/2114)). * (pip) Fixed pypi parse_simpleapi_html function for feeds with package metadata containing ">" sign * (toolchains) Added missing executable permission to `//python/runtime_env_toolchains` interpreter script so that it is runnable. - ([#2085](https://github.com/bazelbuild/rules_python/issues/2085)). + ([#2085](https://github.com/bazel-contrib/rules_python/issues/2085)). * (pip) Correctly use the `sdist` downloaded by the bazel downloader when using `experimental_index_url` feature. Fixes - [#2091](https://github.com/bazelbuild/rules_python/issues/2090). + [#2091](https://github.com/bazel-contrib/rules_python/issues/2090). * (gazelle) Make `gazelle_python_manifest.update` manual to avoid unnecessary network behavior. * (bzlmod): The conflicting toolchains during `python` extension will no longer cause warnings by default. In order to see the warnings for diagnostic purposes set the env var `RULES_PYTHON_REPO_DEBUG_VERBOSITY` to one of `INFO`, `DEBUG` or `TRACE`. - Fixes [#1818](https://github.com/bazelbuild/rules_python/issues/1818). + Fixes [#1818](https://github.com/bazel-contrib/rules_python/issues/1818). * (runfiles) Make runfiles lookups work for the situation of Bazel 7, Python 3.9 (or earlier, where safepath isn't present), and the Rlocation call in the same directory as the main file. - Fixes [#1631](https://github.com/bazelbuild/rules_python/issues/1631). + Fixes [#1631](https://github.com/bazel-contrib/rules_python/issues/1631). {#v0-35-0-added} ### Added * (rules) `compile_pip_requirements` supports multiple requirements input files as `srcs`. * (rules) `PYTHONSAFEPATH` is inherited from the calling environment to allow disabling it (Requires {obj}`--bootstrap_impl=script`) - ([#2060](https://github.com/bazelbuild/rules_python/issues/2060)). + ([#2060](https://github.com/bazel-contrib/rules_python/issues/2060)). * (gazelle) Added `python_generation_mode_per_package_require_test_entry_point` in order to better accommodate users who use a custom macro, [`pytest-bazel`][pytest_bazel], [rules_python_pytest] or `rules_py` @@ -756,7 +756,7 @@ Other changes: {#v0-34-0} ## [0.34.0] - 2024-07-04 -[0.34.0]: https://github.com/bazelbuild/rules_python/releases/tag/0.34.0 +[0.34.0]: https://github.com/bazel-contrib/rules_python/releases/tag/0.34.0 {#v0-34-0-changed} ### Changed @@ -797,7 +797,7 @@ Other changes: and drop the defaults from the lock file. * (whl_library) Correctly handle arch-specific dependencies when we encounter a platform specific wheel and use `experimental_target_platforms`. - Fixes [#1996](https://github.com/bazelbuild/rules_python/issues/1996). + Fixes [#1996](https://github.com/bazel-contrib/rules_python/issues/1996). * (rules) The first element of the default outputs is now the executable again. * (pip) Fixed crash when pypi packages lacked a sha (e.g. yanked packages) @@ -807,7 +807,7 @@ Other changes: replacement for the "autodetecting" toolchain. * (gazelle) Added new `python_label_convention` and `python_label_normalization` directives. These directive allows altering default Gazelle label format to third-party dependencies useful for re-using Gazelle plugin - with other rules, including `rules_pycross`. See [#1939](https://github.com/bazelbuild/rules_python/issues/1939). + with other rules, including `rules_pycross`. See [#1939](https://github.com/bazel-contrib/rules_python/issues/1939). {#v0-34-0-removed} ### Removed @@ -816,7 +816,7 @@ Other changes: {#v0-33-2} ## [0.33.2] - 2024-06-13 -[0.33.2]: https://github.com/bazelbuild/rules_python/releases/tag/0.33.2 +[0.33.2]: https://github.com/bazel-contrib/rules_python/releases/tag/0.33.2 {#v0-33-2-fixed} ### Fixed @@ -824,22 +824,22 @@ Other changes: To enable it, set {obj}`--//python/config_settings:exec_tools_toolchain=enabled`. This toolchain must be enabled for precompilation to work. This toolchain will be enabled by default in a future release. - Fixes [#1967](https://github.com/bazelbuild/rules_python/issues/1967). + Fixes [#1967](https://github.com/bazel-contrib/rules_python/issues/1967). {#v0-33-1} ## [0.33.1] - 2024-06-13 -[0.33.1]: https://github.com/bazelbuild/rules_python/releases/tag/0.33.1 +[0.33.1]: https://github.com/bazel-contrib/rules_python/releases/tag/0.33.1 {#v0-33-1-fixed} ### Fixed * (py_binary) Fix building of zip file when using `--build_python_zip` - argument. Fixes [#1954](https://github.com/bazelbuild/rules_python/issues/1954). + argument. Fixes [#1954](https://github.com/bazel-contrib/rules_python/issues/1954). {#v0-33-0} ## [0.33.0] - 2024-06-12 -[0.33.0]: https://github.com/bazelbuild/rules_python/releases/tag/0.33.0 +[0.33.0]: https://github.com/bazel-contrib/rules_python/releases/tag/0.33.0 {#v0-33-0-changed} ### Changed @@ -859,8 +859,8 @@ Other changes: * (pip.parse): Add references to all supported wheels when using `experimental_index_url` to allowing to correctly fetch the wheels for the right platform. See the updated docs on how to use the feature. This is work towards addressing - [#735](https://github.com/bazelbuild/rules_python/issues/735) and - [#260](https://github.com/bazelbuild/rules_python/issues/260). The spoke + [#735](https://github.com/bazel-contrib/rules_python/issues/735) and + [#260](https://github.com/bazel-contrib/rules_python/issues/260). The spoke repository names when using this flag will have a structure of `{pip_hub_prefix}_{wheel_name}_{py_tag}_{abi_tag}_{platform_tag}_{sha256}`, which is an implementation detail which should not be relied on and is there @@ -886,13 +886,13 @@ Other changes: * (bzlmod) remove `pip.parse(annotations)` attribute as it is unused and has been replaced by whl_modifications. * (pip) Correctly select wheels when the python tag includes minor versions. - See ([#1930](https://github.com/bazelbuild/rules_python/issues/1930)) + See ([#1930](https://github.com/bazel-contrib/rules_python/issues/1930)) * (pip.parse): The lock file is now reproducible on any host platform if the `experimental_index_url` is not used by any of the modules in the dependency chain. To make the lock file identical on each `os` and `arch`, please use the `experimental_index_url` feature which will fetch metadata from PyPI or a different private index and write the contents to the lock file. Fixes - [#1643](https://github.com/bazelbuild/rules_python/issues/1643). + [#1643](https://github.com/bazel-contrib/rules_python/issues/1643). * (pip.parse): Install `yanked` packages and print a warning instead of ignoring them. This better matches the behaviour of `uv pip install`. * (toolchains): Now matching of the default hermetic toolchain is more robust @@ -901,7 +901,7 @@ Other changes: to toolchain selection failures when the python toolchain is not registered, but is requested via `//python/config_settings:python_version` flag setting. * (doc) Fix the `WORKSPACE` requirement vendoring example. Fixes - [#1918](https://github.com/bazelbuild/rules_python/issues/1918). + [#1918](https://github.com/bazel-contrib/rules_python/issues/1918). {#v0-33-0-added} ### Added @@ -912,7 +912,7 @@ Other changes: [Precompiling docs][precompile-docs] and API reference docs for more information on precompiling. Note this requires Bazel 7+ and the Pystar rule implementation enabled. - ([#1761](https://github.com/bazelbuild/rules_python/issues/1761)) + ([#1761](https://github.com/bazel-contrib/rules_python/issues/1761)) * (rules) Attributes and flags to control precompile behavior: `precompile`, `precompile_optimize_level`, `precompile_source_retention`, `precompile_invalidation_mode`, and `pyc_collection` @@ -938,7 +938,7 @@ Other changes: is available. It can be enabled by setting {obj}`--@rules_python//python/config_settings:bootstrap_impl=script`. It will become the default in a subsequent release. - ([#691](https://github.com/bazelbuild/rules_python/issues/691)) + ([#691](https://github.com/bazel-contrib/rules_python/issues/691)) * (providers) `PyRuntimeInfo` has two new attributes: {obj}`PyRuntimeInfo.stage2_bootstrap_template` and {obj}`PyRuntimeInfo.zip_main_template`. @@ -960,7 +960,7 @@ Other changes: {#v0-32-2} ## [0.32.2] - 2024-05-14 -[0.32.2]: https://github.com/bazelbuild/rules_python/releases/tag/0.32.2 +[0.32.2]: https://github.com/bazel-contrib/rules_python/releases/tag/0.32.2 {#v0-32-2-fixed} ### Fixed @@ -968,12 +968,12 @@ Other changes: * Workaround existence of infinite symlink loops on case insensitive filesystems when targeting linux platforms with recent Python toolchains. Works around an upstream [issue][indygreg-231]. Fixes [#1800][rules_python_1800]. [indygreg-231]: https://github.com/indygreg/python-build-standalone/issues/231 -[rules_python_1800]: https://github.com/bazelbuild/rules_python/issues/1800 +[rules_python_1800]: https://github.com/bazel-contrib/rules_python/issues/1800 {#v0-32-0} ## [0.32.0] - 2024-05-12 -[0.32.0]: https://github.com/bazelbuild/rules_python/releases/tag/0.32.0 +[0.32.0]: https://github.com/bazel-contrib/rules_python/releases/tag/0.32.0 {#v0-32-0-changed} ### Changed @@ -998,22 +998,22 @@ Other changes: * (whl_library): Fix the experimental_target_platforms overriding for platform specific wheels when the wheels are for any python interpreter version. Fixes - [#1810](https://github.com/bazelbuild/rules_python/issues/1810). + [#1810](https://github.com/bazel-contrib/rules_python/issues/1810). * (whl_library): Stop generating duplicate dependencies when encountering duplicates in the METADATA. Fixes - [#1873](https://github.com/bazelbuild/rules_python/issues/1873). + [#1873](https://github.com/bazel-contrib/rules_python/issues/1873). * (gazelle) In `project` or `package` generation modes, do not generate `py_test` rules when there are no test files and do not set `main = "__test__.py"` when that file doesn't exist. * (whl_library) The group redirection is only added when the package is part of the group potentially fixing aspects that want to traverse a `py_library` graph. - Fixes [#1760](https://github.com/bazelbuild/rules_python/issues/1760). + Fixes [#1760](https://github.com/bazel-contrib/rules_python/issues/1760). * (bzlmod) Setting a particular micro version for the interpreter and the `pip.parse` extension is now possible, see the `examples/pip_parse/MODULE.bazel` for how to do it. - See [#1371](https://github.com/bazelbuild/rules_python/issues/1371). + See [#1371](https://github.com/bazel-contrib/rules_python/issues/1371). * (refactor) The pre-commit developer workflow should now pass `isort` and `black` - checks (see [#1674](https://github.com/bazelbuild/rules_python/issues/1674)). + checks (see [#1674](https://github.com/bazel-contrib/rules_python/issues/1674)). ### Added @@ -1031,13 +1031,13 @@ Other changes: [original issue][test_file_pattern_issue] and the [docs][test_file_pattern_docs] for details. * (wheel) Add support for `data_files` attributes in py_wheel rule - ([#1777](https://github.com/bazelbuild/rules_python/issues/1777)) + ([#1777](https://github.com/bazel-contrib/rules_python/issues/1777)) * (py_wheel) `bzlmod` installations now provide a `twine` setup for the default Python toolchain in `rules_python` for version 3.11. * (bzlmod) New `experimental_index_url`, `experimental_extra_index_urls` and `experimental_index_url_overrides` to `pip.parse` for using the bazel downloader. If you see any issues, report in - [#1357](https://github.com/bazelbuild/rules_python/issues/1357). The URLs for + [#1357](https://github.com/bazel-contrib/rules_python/issues/1357). The URLs for the whl and sdist files will be written to the lock file. Controlling whether the downloading of metadata is done in parallel can be done using `parallel_download` attribute. @@ -1053,7 +1053,7 @@ Other changes: `experimental_requirement_cycles`, now is a good time to migrate. [python_default_visibility]: gazelle/README.md#directive-python_default_visibility -[test_file_pattern_issue]: https://github.com/bazelbuild/rules_python/issues/1816 +[test_file_pattern_issue]: https://github.com/bazel-contrib/rules_python/issues/1816 [test_file_pattern_docs]: gazelle/README.md#directive-python_test_file_pattern [20240224]: https://github.com/indygreg/python-build-standalone/releases/tag/20240224. [20240415]: https://github.com/indygreg/python-build-standalone/releases/tag/20240415. @@ -1061,7 +1061,7 @@ Other changes: ## [0.31.0] - 2024-02-12 -[0.31.0]: https://github.com/bazelbuild/rules_python/releases/tag/0.31.0 +[0.31.0]: https://github.com/bazel-contrib/rules_python/releases/tag/0.31.0 ### Changed @@ -1073,7 +1073,7 @@ Other changes: ## [0.30.0] - 2024-02-12 -[0.30.0]: https://github.com/bazelbuild/rules_python/releases/tag/0.30.0 +[0.30.0]: https://github.com/bazel-contrib/rules_python/releases/tag/0.30.0 ### Changed @@ -1105,7 +1105,7 @@ Other changes: * (PyRuntimeInfo) Switch back to builtin PyRuntimeInfo for Bazel 6.4 and when pystar is disabled. This fixes an error about `target ... does not have ... PyRuntimeInfo`. - ([#1732](https://github.com/bazelbuild/rules_python/issues/1732)) + ([#1732](https://github.com/bazel-contrib/rules_python/issues/1732)) ### Added @@ -1147,7 +1147,7 @@ Other changes: ## [0.29.0] - 2024-01-22 -[0.29.0]: https://github.com/bazelbuild/rules_python/releases/tag/0.29.0 +[0.29.0]: https://github.com/bazel-contrib/rules_python/releases/tag/0.29.0 ### Changed @@ -1167,7 +1167,7 @@ Other changes: * (bzlmod pip.parse) Use a platform-independent reference to the interpreter pip uses. This reduces (but doesn't eliminate) the amount of platform-specific content in `MODULE.bazel.lock` files; Follow - [#1643](https://github.com/bazelbuild/rules_python/issues/1643) for removing + [#1643](https://github.com/bazel-contrib/rules_python/issues/1643) for removing platform-specific content in `MODULE.bazel.lock` files. * (wheel) The stamp variables inside the distribution name are no longer @@ -1199,7 +1199,7 @@ Other changes: ## [0.28.0] - 2024-01-07 -[0.28.0]: https://github.com/bazelbuild/rules_python/releases/tag/0.28.0 +[0.28.0]: https://github.com/bazel-contrib/rules_python/releases/tag/0.28.0 ### Changed @@ -1225,7 +1225,7 @@ Other changes: * (toolchains) `py_runtime` can now take an executable target. Note: runfiles from the target are not supported yet. - ([#1612](https://github.com/bazelbuild/rules_python/issues/1612)) + ([#1612](https://github.com/bazel-contrib/rules_python/issues/1612)) * (gazelle) When `python_generation_mode` is set to `file`, create one `py_binary` target for each file with `if __name__ == "__main__"` instead of just one @@ -1252,7 +1252,7 @@ Other changes: package (e.g. one for the package, one for an extra) now work. * (bzlmod python.toolchain) Submodules can now (re)register the Python version that rules_python has set as the default. - ([#1638](https://github.com/bazelbuild/rules_python/issues/1638)) + ([#1638](https://github.com/bazel-contrib/rules_python/issues/1638)) * (whl_library) Actually use the provided patches to patch the whl_library. On Windows the patching may result in files with CRLF line endings, as a result the RECORD file consistency requirement is lifted and now a warning is emitted @@ -1261,13 +1261,13 @@ Other changes: file if you decide to do so. * (coverage): coverage reports are now created when the version-aware rules are used. - ([#1600](https://github.com/bazelbuild/rules_python/issues/1600)) + ([#1600](https://github.com/bazel-contrib/rules_python/issues/1600)) * (toolchains) Workspace builds register the py cc toolchain (bzlmod already was). This makes e.g. `//python/cc:current_py_cc_headers` Just Work. - ([#1669](https://github.com/bazelbuild/rules_python/issues/1669)) + ([#1669](https://github.com/bazel-contrib/rules_python/issues/1669)) * (bzlmod python.toolchain) The value of `ignore_root_user_error` is now decided by the root module only. - ([#1658](https://github.com/bazelbuild/rules_python/issues/1658)) + ([#1658](https://github.com/bazel-contrib/rules_python/issues/1658)) ### Added @@ -1280,7 +1280,7 @@ Other changes: ## [0.27.0] - 2023-11-16 -[0.27.0]: https://github.com/bazelbuild/rules_python/releases/tag/0.27.0 +[0.27.0]: https://github.com/bazel-contrib/rules_python/releases/tag/0.27.0 ### Changed @@ -1446,7 +1446,7 @@ Breaking changes: * (gazelle) Improve runfiles lookup hermeticity. -[0.26.0]: https://github.com/bazelbuild/rules_python/releases/tag/0.26.0 +[0.26.0]: https://github.com/bazel-contrib/rules_python/releases/tag/0.26.0 ## [0.25.0] - 2023-08-22 @@ -1474,7 +1474,7 @@ Breaking changes: * (gazelle) Stop generating unnecessary imports. * (toolchains) s390x supported for Python 3.9.17, 3.10.12, and 3.11.4. -[0.25.0]: https://github.com/bazelbuild/rules_python/releases/tag/0.25.0 +[0.25.0]: https://github.com/bazel-contrib/rules_python/releases/tag/0.25.0 ## [0.24.0] - 2023-07-11 @@ -1510,4 +1510,4 @@ Breaking changes: * (pip) Create all_data_requirements alias * Expose Python C headers through the toolchain. -[0.24.0]: https://github.com/bazelbuild/rules_python/releases/tag/0.24.0 +[0.24.0]: https://github.com/bazel-contrib/rules_python/releases/tag/0.24.0 diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index cd274861d7..17558e1b23 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -30,7 +30,7 @@ the [GitHub `gh` tool](https://github.com/cli/cli) (More advanced users may prefer the GitHub UI and raw `git` commands). ```shell -gh repo fork bazelbuild/rules_python --clone --remote +gh repo fork bazel-contrib/rules_python --clone --remote ``` Next, make sure you have a new enough version of Python installed that supports the diff --git a/RELEASING.md b/RELEASING.md index 42a29219f9..6e441cbce6 100644 --- a/RELEASING.md +++ b/RELEASING.md @@ -32,7 +32,7 @@ other minor changes bump the patch digit. To find if there were any features added or incompatible changes made, review [CHANGELOG.md](CHANGELOG.md) and the commit history. This can be done using github by going to the url: -`https://github.com/bazelbuild/rules_python/compare/...main`. +`https://github.com/bazel-contrib/rules_python/compare/...main`. ## Patch release with cherry picks diff --git a/WORKSPACE b/WORKSPACE index b97411e2d5..3ad83ca04b 100644 --- a/WORKSPACE +++ b/WORKSPACE @@ -107,7 +107,7 @@ local_repository( # which we need to fetch in order to compile it. load("@rules_python_gazelle_plugin//:deps.bzl", _py_gazelle_deps = "gazelle_deps") -# See: https://github.com/bazelbuild/rules_python/blob/main/gazelle/README.md +# See: https://github.com/bazel-contrib/rules_python/blob/main/gazelle/README.md # This rule loads and compiles various go dependencies that running gazelle # for python requirements. _py_gazelle_deps() @@ -118,7 +118,7 @@ interpreter = "@python_3_11_9_host//:python" ##################### # Install twine for our own runfiles wheel publishing. # Eventually we might want to install twine automatically for users too, see: -# https://github.com/bazelbuild/rules_python/issues/1016. +# https://github.com/bazel-contrib/rules_python/issues/1016. load("@rules_python//python:pip.bzl", "pip_parse") pip_parse( diff --git a/docs/api/rules_python/python/config_settings/index.md b/docs/api/rules_python/python/config_settings/index.md index cb44de97c7..79c7d0c109 100644 --- a/docs/api/rules_python/python/config_settings/index.md +++ b/docs/api/rules_python/python/config_settings/index.md @@ -266,7 +266,7 @@ Determines if relative symlinks are created using `declare_symlink()` at build time. This is only intended to work around -[#2489](https://github.com/bazelbuild/rules_python/issues/2489), where some +[#2489](https://github.com/bazel-contrib/rules_python/issues/2489), where some packaging rules don't support `declare_symlink()` artifacts. Values: diff --git a/docs/conf.py b/docs/conf.py index 4c8e4a2a6b..f58baf5183 100644 --- a/docs/conf.py +++ b/docs/conf.py @@ -104,7 +104,7 @@ # Insert after the main extension extensions.insert(1, "readthedocs_ext.external_version_warning") readthedocs_vcs_url = ( - "http://github.com/bazelbuild/rules_python/pull/{}".format( + "http://github.com/bazel-contrib/rules_python/pull/{}".format( os.environ.get("READTHEDOCS_VERSION", "") ) ) @@ -133,7 +133,7 @@ # --- Extlinks configuration extlinks = { - "gh-path": (f"https://github.com/bazelbuild/rules_python/tree/main/%s", "%s"), + "gh-path": (f"https://github.com/bazel-contrib/rules_python/tree/main/%s", "%s"), } # --- MyST configuration diff --git a/docs/extending.md b/docs/extending.md index dbd63e5a4f..387310e6cf 100644 --- a/docs/extending.md +++ b/docs/extending.md @@ -23,7 +23,7 @@ Extending the core rules is most useful when you want all or most of the behavior of a core rule. ::: -Follow or comment on https://github.com/bazelbuild/rules_python/issues/1647 +Follow or comment on https://github.com/bazel-contrib/rules_python/issues/1647 for the development of APIs to support custom derived rules. ## Creating custom rules diff --git a/docs/getting-started.md b/docs/getting-started.md index b3b5409c7e..969716603c 100644 --- a/docs/getting-started.md +++ b/docs/getting-started.md @@ -18,7 +18,7 @@ your MODULE.bazel file: ```starlark # Update the version "0.0.0" to the release found here: -# https://github.com/bazelbuild/rules_python/releases. +# https://github.com/bazel-contrib/rules_python/releases. bazel_dep(name = "rules_python", version = "0.0.0") pip = use_extension("@rules_python//python/extensions:pip.bzl", "pip") @@ -39,13 +39,13 @@ using Bzlmod. Here is a simplified setup to download the prebuilt runtimes. load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive") # Update the snippet based on the latest release below -# https://github.com/bazelbuild/rules_python/releases +# https://github.com/bazel-contrib/rules_python/releases http_archive( name = "rules_python", sha256 = "ca77768989a7f311186a29747e3e95c936a41dffac779aff6b443db22290d913", strip_prefix = "rules_python-0.36.0", - url = "https://github.com/bazelbuild/rules_python/releases/download/0.36.0/rules_python-0.36.0.tar.gz", + url = "https://github.com/bazel-contrib/rules_python/releases/download/0.36.0/rules_python-0.36.0.tar.gz", ) load("@rules_python//python:repositories.bzl", "py_repositories") diff --git a/docs/pypi-dependencies.md b/docs/pypi-dependencies.md index 28e630c61d..039200dfd4 100644 --- a/docs/pypi-dependencies.md +++ b/docs/pypi-dependencies.md @@ -71,7 +71,7 @@ In some cases you may not want to generate the requirements.bzl file as a reposi while Bazel is fetching dependencies. For example, if you produce a reusable Bazel module such as a ruleset, you may want to include the requirements.bzl file rather than make your users install the WORKSPACE setup to generate it. -See https://github.com/bazelbuild/rules_python/issues/608 +See https://github.com/bazel-contrib/rules_python/issues/608 This is the same workflow as Gazelle, which creates `go_repository` rules with [`update-repos`](https://github.com/bazelbuild/bazel-gazelle#update-repos) @@ -180,7 +180,7 @@ buildozer command: buildozer 'substitute deps @old//([^/]+) @new//${1}' //...:* ``` -[requirements-drawbacks]: https://github.com/bazelbuild/rules_python/issues/414 +[requirements-drawbacks]: https://github.com/bazel-contrib/rules_python/issues/414 ### Entry points diff --git a/docs/toolchains.md b/docs/toolchains.md index 3294c1732a..0e4f5c2321 100644 --- a/docs/toolchains.md +++ b/docs/toolchains.md @@ -273,7 +273,7 @@ transition period when some of the code is still defined in `WORKSPACE`. To import rules_python in your project, you first need to add it to your `WORKSPACE` file, using the snippet provided in the -[release you choose](https://github.com/bazelbuild/rules_python/releases) +[release you choose](https://github.com/bazel-contrib/rules_python/releases) To depend on a particular unreleased version, you can do the following: @@ -282,7 +282,7 @@ load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive") # Update the SHA and VERSION to the lastest version available here: -# https://github.com/bazelbuild/rules_python/releases. +# https://github.com/bazel-contrib/rules_python/releases. SHA="84aec9e21cc56fbc7f1335035a71c850d1b9b5cc6ff497306f84cced9a769841" @@ -292,7 +292,7 @@ http_archive( name = "rules_python", sha256 = SHA, strip_prefix = "rules_python-{}".format(VERSION), - url = "https://github.com/bazelbuild/rules_python/releases/download/{}/rules_python-{}.tar.gz".format(VERSION,VERSION), + url = "https://github.com/bazel-contrib/rules_python/releases/download/{}/rules_python-{}.tar.gz".format(VERSION,VERSION), ) load("@rules_python//python:repositories.bzl", "py_repositories") @@ -324,7 +324,7 @@ pip_parse( ``` After registration, your Python targets will use the toolchain's interpreter during execution, but a system-installed interpreter -is still used to 'bootstrap' Python targets (see https://github.com/bazelbuild/rules_python/issues/691). +is still used to 'bootstrap' Python targets (see https://github.com/bazel-contrib/rules_python/issues/691). You may also find some quirks while using this toolchain. Please refer to [python-build-standalone documentation's _Quirks_ section](https://gregoryszorc.com/docs/python-build-standalone/main/quirks.html). ## Autodetecting toolchain diff --git a/examples/build_file_generation/WORKSPACE b/examples/build_file_generation/WORKSPACE index 3f1fad8a8d..6681ad6861 100644 --- a/examples/build_file_generation/WORKSPACE +++ b/examples/build_file_generation/WORKSPACE @@ -59,7 +59,7 @@ gazelle_dependencies() # DON'T COPY_PASTE THIS. # Our example uses `local_repository` to point to the HEAD version of rules_python. # Users should instead use the installation instructions from the release they use. -# See https://github.com/bazelbuild/rules_python/releases +# See https://github.com/bazel-contrib/rules_python/releases local_repository( name = "rules_python", path = "../..", @@ -128,7 +128,7 @@ install_deps() # which we need to fetch in order to compile it. load("@rules_python_gazelle_plugin//:deps.bzl", _py_gazelle_deps = "gazelle_deps") -# See: https://github.com/bazelbuild/rules_python/blob/main/gazelle/README.md +# See: https://github.com/bazel-contrib/rules_python/blob/main/gazelle/README.md # This rule loads and compiles various go dependencies that running gazelle # for python requirements. _py_gazelle_deps() diff --git a/examples/bzlmod/py_proto_library/BUILD.bazel b/examples/bzlmod/py_proto_library/BUILD.bazel index 175589fbf9..969cb8e9f7 100644 --- a/examples/bzlmod/py_proto_library/BUILD.bazel +++ b/examples/bzlmod/py_proto_library/BUILD.bazel @@ -18,7 +18,7 @@ py_test( ], ) -# Regression test for https://github.com/bazelbuild/rules_python/issues/2515 +# Regression test for https://github.com/bazel-contrib/rules_python/issues/2515 # # This test fails before protobuf 30.0 release # when ran with --legacy_external_runfiles=False (default in Bazel 8.0.0). diff --git a/examples/bzlmod_build_file_generation/MODULE.bazel b/examples/bzlmod_build_file_generation/MODULE.bazel index 30ad567879..9bec25fcbb 100644 --- a/examples/bzlmod_build_file_generation/MODULE.bazel +++ b/examples/bzlmod_build_file_generation/MODULE.bazel @@ -12,7 +12,7 @@ module( # The following stanza defines the dependency rules_python. # For typical setups you set the version. # See the releases page for available versions. -# https://github.com/bazelbuild/rules_python/releases +# https://github.com/bazel-contrib/rules_python/releases bazel_dep(name = "rules_python", version = "0.0.0") # The following loads rules_python from the file system. @@ -25,7 +25,7 @@ local_path_override( # The following stanza defines the dependency rules_python_gazelle_plugin. # For typical setups you set the version. # See the releases page for available versions. -# https://github.com/bazelbuild/rules_python/releases +# https://github.com/bazel-contrib/rules_python/releases bazel_dep(name = "rules_python_gazelle_plugin", version = "0.0.0") # The following starlark loads the gazelle plugin from the file system. diff --git a/examples/pip_parse_vendored/README.md b/examples/pip_parse_vendored/README.md index fdf040c8e5..baa51f5729 100644 --- a/examples/pip_parse_vendored/README.md +++ b/examples/pip_parse_vendored/README.md @@ -1,7 +1,7 @@ # pip_parse vendored This example is like pip_parse, however we avoid loading from the generated file. -See https://github.com/bazelbuild/rules_python/issues/608 +See https://github.com/bazel-contrib/rules_python/issues/608 and https://blog.aspect.dev/avoid-eager-fetches. The requirements now form a triple: diff --git a/gazelle/BUILD.bazel b/gazelle/BUILD.bazel index f74338d4b5..0938be3dfc 100644 --- a/gazelle/BUILD.bazel +++ b/gazelle/BUILD.bazel @@ -2,7 +2,7 @@ load("@bazel_gazelle//:def.bzl", "gazelle") # Gazelle configuration options. # See https://github.com/bazelbuild/bazel-gazelle#running-gazelle-with-bazel -# gazelle:prefix github.com/bazelbuild/rules_python/gazelle +# gazelle:prefix github.com/bazel-contrib/rules_python/gazelle # gazelle:exclude bazel-out gazelle( name = "gazelle", diff --git a/gazelle/README.md b/gazelle/README.md index 01cf45a938..89ebaef4cd 100644 --- a/gazelle/README.md +++ b/gazelle/README.md @@ -17,7 +17,7 @@ without using bzlmod as your dependency manager. ## Example -We have an example of using Gazelle with Python located [here](https://github.com/bazelbuild/rules_python/tree/main/examples/bzlmod). +We have an example of using Gazelle with Python located [here](https://github.com/bazel-contrib/rules_python/tree/main/examples/bzlmod). A fully-working example without using bzlmod is in [`examples/build_file_generation`](../examples/build_file_generation). The following documentation covers using bzlmod. @@ -29,7 +29,7 @@ Get the current version of Gazelle from there releases here: https://github.com See the installation `MODULE.bazel` snippet on the Releases page: -https://github.com/bazelbuild/rules_python/releases in order to configure rules_python. +https://github.com/bazel-contrib/rules_python/releases in order to configure rules_python. You will also need to add the `bazel_dep` for configuration for `rules_python_gazelle_plugin`. @@ -450,7 +450,7 @@ py_library( ) ``` -[issue-1826]: https://github.com/bazelbuild/rules_python/issues/1826 +[issue-1826]: https://github.com/bazel-contrib/rules_python/issues/1826 #### Directive: `python_generation_mode_per_package_require_test_entry_point`: When `# gazelle:python_generation_mode package`, whether a file called `__test__.py` or a target called `__test__`, a.k.a., entry point, is required to generate one test target per package. If this is set to true but no entry point is found, Gazelle will fall back to file mode and generate one test target per file. Setting this directive to false forces Gazelle to generate one test target per package even without entry point. However, this means the `main` attribute of the `py_test` will not be set and the target will not be runnable unless either: @@ -553,7 +553,7 @@ target, building will result in an error saying: ``` Adding non-Python targets to the generated target is a feature request being -tracked in [Issue #1865](https://github.com/bazelbuild/rules_python/issues/1865). +tracked in [Issue #1865](https://github.com/bazel-contrib/rules_python/issues/1865). The annotation can be added multiple times, and all values are combined and de-duplicated. diff --git a/gazelle/go.mod b/gazelle/go.mod index 33ee6bb08a..91d27fdd5a 100644 --- a/gazelle/go.mod +++ b/gazelle/go.mod @@ -1,4 +1,4 @@ -module github.com/bazelbuild/rules_python/gazelle +module github.com/bazel-contrib/rules_python/gazelle go 1.19 diff --git a/gazelle/manifest/BUILD.bazel b/gazelle/manifest/BUILD.bazel index 33b5a46947..ea81d85fbe 100644 --- a/gazelle/manifest/BUILD.bazel +++ b/gazelle/manifest/BUILD.bazel @@ -8,7 +8,7 @@ exports_files([ go_library( name = "manifest", srcs = ["manifest.go"], - importpath = "github.com/bazelbuild/rules_python/gazelle/manifest", + importpath = "github.com/bazel-contrib/rules_python/gazelle/manifest", visibility = ["//visibility:public"], deps = [ "@com_github_emirpasic_gods//sets/treeset", diff --git a/gazelle/manifest/generate/BUILD.bazel b/gazelle/manifest/generate/BUILD.bazel index 96248f4e08..77d2467cef 100644 --- a/gazelle/manifest/generate/BUILD.bazel +++ b/gazelle/manifest/generate/BUILD.bazel @@ -4,7 +4,7 @@ load("//manifest:defs.bzl", "sources_hash") go_library( name = "generate_lib", srcs = ["generate.go"], - importpath = "github.com/bazelbuild/rules_python/gazelle/manifest/generate", + importpath = "github.com/bazel-contrib/rules_python/gazelle/manifest/generate", visibility = ["//visibility:public"], deps = ["//manifest"], ) diff --git a/gazelle/manifest/generate/generate.go b/gazelle/manifest/generate/generate.go index 899b1514ee..52100713e3 100644 --- a/gazelle/manifest/generate/generate.go +++ b/gazelle/manifest/generate/generate.go @@ -28,7 +28,7 @@ import ( "os" "strings" - "github.com/bazelbuild/rules_python/gazelle/manifest" + "github.com/bazel-contrib/rules_python/gazelle/manifest" ) func main() { diff --git a/gazelle/manifest/hasher/BUILD.bazel b/gazelle/manifest/hasher/BUILD.bazel index 2e7b125cc0..c6e3c4c29b 100644 --- a/gazelle/manifest/hasher/BUILD.bazel +++ b/gazelle/manifest/hasher/BUILD.bazel @@ -3,7 +3,7 @@ load("@io_bazel_rules_go//go:def.bzl", "go_binary", "go_library") go_library( name = "hasher_lib", srcs = ["main.go"], - importpath = "github.com/bazelbuild/rules_python/gazelle/manifest/hasher", + importpath = "github.com/bazel-contrib/rules_python/gazelle/manifest/hasher", visibility = ["//visibility:private"], ) diff --git a/gazelle/manifest/manifest_test.go b/gazelle/manifest/manifest_test.go index e80c7fcccc..320361a8e1 100644 --- a/gazelle/manifest/manifest_test.go +++ b/gazelle/manifest/manifest_test.go @@ -22,7 +22,7 @@ import ( "strings" "testing" - "github.com/bazelbuild/rules_python/gazelle/manifest" + "github.com/bazel-contrib/rules_python/gazelle/manifest" ) var modulesMapping = manifest.ModulesMapping{ diff --git a/gazelle/manifest/test/test.go b/gazelle/manifest/test/test.go index a7647f3f7c..5804a7102e 100644 --- a/gazelle/manifest/test/test.go +++ b/gazelle/manifest/test/test.go @@ -27,7 +27,7 @@ import ( "testing" "github.com/bazelbuild/rules_go/go/runfiles" - "github.com/bazelbuild/rules_python/gazelle/manifest" + "github.com/bazel-contrib/rules_python/gazelle/manifest" ) func TestGazelleManifestIsUpdated(t *testing.T) { diff --git a/gazelle/python/BUILD.bazel b/gazelle/python/BUILD.bazel index 893c82e8e4..eb2d72e5eb 100644 --- a/gazelle/python/BUILD.bazel +++ b/gazelle/python/BUILD.bazel @@ -26,7 +26,7 @@ go_library( # See following for more info: # https://github.com/bazelbuild/bazel-gazelle/issues/1513 embedsrcs = ["stdlib_list.txt"], # keep # TODO: use user-defined version? - importpath = "github.com/bazelbuild/rules_python/gazelle/python", + importpath = "github.com/bazel-contrib/rules_python/gazelle/python", visibility = ["//visibility:public"], deps = [ "//manifest", diff --git a/gazelle/python/configure.go b/gazelle/python/configure.go index a369a64b8e..7b1f091b34 100644 --- a/gazelle/python/configure.go +++ b/gazelle/python/configure.go @@ -27,8 +27,8 @@ import ( "github.com/bazelbuild/bazel-gazelle/rule" "github.com/bmatcuk/doublestar/v4" - "github.com/bazelbuild/rules_python/gazelle/manifest" - "github.com/bazelbuild/rules_python/gazelle/pythonconfig" + "github.com/bazel-contrib/rules_python/gazelle/manifest" + "github.com/bazel-contrib/rules_python/gazelle/pythonconfig" ) // Configurer satisfies the config.Configurer interface. It's the diff --git a/gazelle/python/generate.go b/gazelle/python/generate.go index b1ac6689e4..27930c1025 100644 --- a/gazelle/python/generate.go +++ b/gazelle/python/generate.go @@ -32,7 +32,7 @@ import ( "github.com/emirpasic/gods/sets/treeset" godsutils "github.com/emirpasic/gods/utils" - "github.com/bazelbuild/rules_python/gazelle/pythonconfig" + "github.com/bazel-contrib/rules_python/gazelle/pythonconfig" ) const ( diff --git a/gazelle/python/resolve.go b/gazelle/python/resolve.go index 88a688fa85..7a2ec3d68a 100644 --- a/gazelle/python/resolve.go +++ b/gazelle/python/resolve.go @@ -30,7 +30,7 @@ import ( "github.com/emirpasic/gods/sets/treeset" godsutils "github.com/emirpasic/gods/utils" - "github.com/bazelbuild/rules_python/gazelle/pythonconfig" + "github.com/bazel-contrib/rules_python/gazelle/pythonconfig" ) const languageName = "py" diff --git a/gazelle/python/testdata/directive_python_default_visibility/README.md b/gazelle/python/testdata/directive_python_default_visibility/README.md index be42792375..60582d6407 100644 --- a/gazelle/python/testdata/directive_python_default_visibility/README.md +++ b/gazelle/python/testdata/directive_python_default_visibility/README.md @@ -18,4 +18,4 @@ correctly: they interact with sub-packages. -[gh-1682]: https://github.com/bazelbuild/rules_python/issues/1682 +[gh-1682]: https://github.com/bazel-contrib/rules_python/issues/1682 diff --git a/gazelle/python/testdata/directive_python_test_file_pattern_no_value/README.md b/gazelle/python/testdata/directive_python_test_file_pattern_no_value/README.md index 2c38eb78d2..d6fb0b6a72 100644 --- a/gazelle/python/testdata/directive_python_test_file_pattern_no_value/README.md +++ b/gazelle/python/testdata/directive_python_test_file_pattern_no_value/README.md @@ -5,4 +5,4 @@ fails with a nice message if the directive has no value. See discussion in [PR #1819 (comment)][comment]. -[comment]: https://github.com/bazelbuild/rules_python/pull/1819#discussion_r1536906287 +[comment]: https://github.com/bazel-contrib/rules_python/pull/1819#discussion_r1536906287 diff --git a/gazelle/python/testdata/with_third_party_requirements_from_imports/README.md b/gazelle/python/testdata/with_third_party_requirements_from_imports/README.md index c50a1ca100..8713d3d7e1 100644 --- a/gazelle/python/testdata/with_third_party_requirements_from_imports/README.md +++ b/gazelle/python/testdata/with_third_party_requirements_from_imports/README.md @@ -12,4 +12,4 @@ for example from google.cloud import aiplatform, storage ``` -See https://github.com/bazelbuild/rules_python/issues/709 and https://github.com/sramirezmartin/gazelle-toy-example. +See https://github.com/bazel-contrib/rules_python/issues/709 and https://github.com/sramirezmartin/gazelle-toy-example. diff --git a/gazelle/pythonconfig/BUILD.bazel b/gazelle/pythonconfig/BUILD.bazel index d80902e7ce..711bf2eb42 100644 --- a/gazelle/pythonconfig/BUILD.bazel +++ b/gazelle/pythonconfig/BUILD.bazel @@ -6,7 +6,7 @@ go_library( "pythonconfig.go", "types.go", ], - importpath = "github.com/bazelbuild/rules_python/gazelle/pythonconfig", + importpath = "github.com/bazel-contrib/rules_python/gazelle/pythonconfig", visibility = ["//visibility:public"], deps = [ "//manifest", diff --git a/gazelle/pythonconfig/pythonconfig.go b/gazelle/pythonconfig/pythonconfig.go index fde0a98da2..2183ec60a3 100644 --- a/gazelle/pythonconfig/pythonconfig.go +++ b/gazelle/pythonconfig/pythonconfig.go @@ -23,7 +23,7 @@ import ( "github.com/emirpasic/gods/lists/singlylinkedlist" "github.com/bazelbuild/bazel-gazelle/label" - "github.com/bazelbuild/rules_python/gazelle/manifest" + "github.com/bazel-contrib/rules_python/gazelle/manifest" ) // Directives diff --git a/python/packaging.bzl b/python/packaging.bzl index 17f72a7d67..629af2d6a4 100644 --- a/python/packaging.bzl +++ b/python/packaging.bzl @@ -139,7 +139,7 @@ def py_wheel( To publish the wheel to PyPI, the twine package is required and it is installed by default on `bzlmod` setups. On legacy `WORKSPACE`, `rules_python` doesn't provide `twine` itself - (see https://github.com/bazelbuild/rules_python/issues/1016), but + (see https://github.com/bazel-contrib/rules_python/issues/1016), but you can install it with `pip_parse`, just like we do any other dependencies. Once you've installed twine, you can pass its label to the `twine` diff --git a/python/private/py_cc_toolchain_rule.bzl b/python/private/py_cc_toolchain_rule.bzl index d5f3b685a4..f12933e245 100644 --- a/python/private/py_cc_toolchain_rule.bzl +++ b/python/private/py_cc_toolchain_rule.bzl @@ -15,7 +15,7 @@ """Implementation of py_cc_toolchain rule. NOTE: This is a beta-quality feature. APIs subject to change until -https://github.com/bazelbuild/rules_python/issues/824 is considered done. +https://github.com/bazel-contrib/rules_python/issues/824 is considered done. """ load("@bazel_skylib//rules:common_settings.bzl", "BuildSettingInfo") diff --git a/python/private/py_console_script_gen.py b/python/private/py_console_script_gen.py index 64ebea6ab7..ffc4e81b3a 100644 --- a/python/private/py_console_script_gen.py +++ b/python/private/py_console_script_gen.py @@ -17,7 +17,7 @@ For Python versions earlier than 3.11 and for earlier bazel versions than 7.0 we need to workaround the issue of sys.path[0] breaking out of the runfiles tree see the following for more context: -* https://github.com/bazelbuild/rules_python/issues/382 +* https://github.com/bazel-contrib/rules_python/issues/382 * https://github.com/bazelbuild/bazel/pull/15701 In affected bazel and Python versions we see in programs such as `flake8`, `pylint` or `pytest` errors because the @@ -130,7 +130,7 @@ def run( module, _, entry_point = entry_point.rpartition(":") attr, _, _ = entry_point.partition(".") # TODO: handle 'extras' in entry_point generation - # See https://github.com/bazelbuild/rules_python/issues/1383 + # See https://github.com/bazel-contrib/rules_python/issues/1383 # See https://packaging.python.org/en/latest/specifications/entry-points/ with open(out, "w") as f: diff --git a/python/private/py_runtime_rule.bzl b/python/private/py_runtime_rule.bzl index 9407cac50f..3dc00baa12 100644 --- a/python/private/py_runtime_rule.bzl +++ b/python/private/py_runtime_rule.bzl @@ -269,7 +269,7 @@ can be either of: NOTE: the runfiles of the target may not yet be properly respected/propagated to consumers of the toolchain/interpreter, see - bazelbuild/rules_python/issues/1612 + bazel-contrib/rules_python/issues/1612 For a platform runtime (i.e. `interpreter_path` being set) this attribute must not be set. diff --git a/python/private/pypi/patch_whl.bzl b/python/private/pypi/patch_whl.bzl index c839f2e4d6..7af9c4da2f 100644 --- a/python/private/pypi/patch_whl.bzl +++ b/python/private/pypi/patch_whl.bzl @@ -128,7 +128,7 @@ def patch_whl(rctx, *, python_interpreter, whl_path, patches, **kwargs): warning_msg = """WARNING: the resultant RECORD file of the patch wheel is different If you are patching on Windows, you may see this warning because of - a known issue (bazelbuild/rules_python#1639) with file endings. + a known issue (bazel-contrib/rules_python#1639) with file endings. If you would like to silence the warning, you can apply the patch that is stored in {record_patch}. The contents of the file are below: diff --git a/python/private/pypi/pip_repository.bzl b/python/private/pypi/pip_repository.bzl index 029566eea3..7976cfaae9 100644 --- a/python/private/pypi/pip_repository.bzl +++ b/python/private/pypi/pip_repository.bzl @@ -228,7 +228,7 @@ pip_repository = repository_rule( Optional annotations to apply to packages. Keys should be package names, with capitalization matching the input requirements file, and values should be generated using the `package_name` macro. For example usage, see [this WORKSPACE -file](https://github.com/bazelbuild/rules_python/blob/main/examples/pip_repository_annotations/WORKSPACE). +file](https://github.com/bazel-contrib/rules_python/blob/main/examples/pip_repository_annotations/WORKSPACE). """, ), _template = attr.label( @@ -336,7 +336,7 @@ In some cases you may not want to generate the requirements.bzl file as a reposi while Bazel is fetching dependencies. For example, if you produce a reusable Bazel module such as a ruleset, you may want to include the requirements.bzl file rather than make your users install the WORKSPACE setup to generate it. -See https://github.com/bazelbuild/rules_python/issues/608 +See https://github.com/bazel-contrib/rules_python/issues/608 This is the same workflow as Gazelle, which creates `go_repository` rules with [`update-repos`](https://github.com/bazelbuild/bazel-gazelle#update-repos) diff --git a/python/private/pypi/whl_installer/namespace_pkgs.py b/python/private/pypi/whl_installer/namespace_pkgs.py index 7d23c0e34b..b415844ace 100644 --- a/python/private/pypi/whl_installer/namespace_pkgs.py +++ b/python/private/pypi/whl_installer/namespace_pkgs.py @@ -92,7 +92,7 @@ def add_pkgutil_style_namespace_pkg_init(dir_path: Path) -> None: ns_pkg_init_f.write( textwrap.dedent( """\ - # __path__ manipulation added by bazelbuild/rules_python to support namespace pkgs. + # __path__ manipulation added by bazel-contrib/rules_python to support namespace pkgs. __path__ = __import__('pkgutil').extend_path(__path__, __name__) """ ) diff --git a/python/private/pypi/whl_installer/wheel.py b/python/private/pypi/whl_installer/wheel.py index 0f6bd27cdd..d95b33a194 100644 --- a/python/private/pypi/whl_installer/wheel.py +++ b/python/private/pypi/whl_installer/wheel.py @@ -378,6 +378,6 @@ def unzip(self, directory: str) -> None: source=wheel_source, destination=destination, additional_metadata={ - "INSTALLER": b"https://github.com/bazelbuild/rules_python", + "INSTALLER": b"https://github.com/bazel-contrib/rules_python", }, ) diff --git a/python/private/python_repository.bzl b/python/private/python_repository.bzl index 299dd36eae..0534f9cd69 100644 --- a/python/private/python_repository.bzl +++ b/python/private/python_repository.bzl @@ -154,9 +154,9 @@ def _python_repository_impl(rctx): ) uid = int(stdout.strip()) if uid == 0: - fail_or_warn("The current user is root, which can cause spurious cache misses or build failures with the hermetic Python interpreter. See https://github.com/bazelbuild/rules_python/pull/713.") + fail_or_warn("The current user is root, which can cause spurious cache misses or build failures with the hermetic Python interpreter. See https://github.com/bazel-contrib/rules_python/pull/713.") else: - fail_or_warn("The current user has CAP_DAC_OVERRIDE set, which can cause spurious cache misses or build failures with the hermetic Python interpreter. See https://github.com/bazelbuild/rules_python/pull/713.") + fail_or_warn("The current user has CAP_DAC_OVERRIDE set, which can cause spurious cache misses or build failures with the hermetic Python interpreter. See https://github.com/bazel-contrib/rules_python/pull/713.") python_bin = "python.exe" if ("windows" in platform) else "bin/python3" @@ -188,7 +188,7 @@ def _python_repository_impl(rctx): # These pycache files are created on first use of the associated python files. # Exclude them from the glob because otherwise between the first time and second time a python toolchain is used," # the definition of this filegroup will change, and depending rules will get invalidated." - # See https://github.com/bazelbuild/rules_python/issues/1008 for unconditionally adding these to toolchains so we can stop ignoring them." + # See https://github.com/bazel-contrib/rules_python/issues/1008 for unconditionally adding these to toolchains so we can stop ignoring them." "**/__pycache__/*.pyc", "**/__pycache__/*.pyo", ] diff --git a/python/private/runtime_env_toolchain_interpreter.sh b/python/private/runtime_env_toolchain_interpreter.sh index 2cb7cc7151..b09bc53e5c 100755 --- a/python/private/runtime_env_toolchain_interpreter.sh +++ b/python/private/runtime_env_toolchain_interpreter.sh @@ -50,7 +50,7 @@ $PATH Please ensure an interpreter is available on this platform (and marked \ executable), or else register an appropriate Python toolchain as per the \ documentation for py_runtime_pair \ -(https://github.com/bazelbuild/rules_python/blob/master/docs/python.md#py_runtime_pair)." +(https://github.com/bazel-contrib/rules_python/blob/master/docs/python.md#py_runtime_pair)." fi exec "$PYTHON_BIN" "$@" diff --git a/python/private/stage1_bootstrap_template.sh b/python/private/stage1_bootstrap_template.sh index bd142cf7c7..e548c848a5 100644 --- a/python/private/stage1_bootstrap_template.sh +++ b/python/private/stage1_bootstrap_template.sh @@ -243,7 +243,7 @@ command=( # using `kill`) to this process (the PID seen by the calling process) are # received by the Python process. Otherwise, this process receives the signal # and would have to manually propagate it. -# See https://github.com/bazelbuild/rules_python/issues/2043#issuecomment-2215469971 +# See https://github.com/bazel-contrib/rules_python/issues/2043#issuecomment-2215469971 # for more information. # # However, we can't use exec when there is cleanup to do afterwards. Control diff --git a/python/py_binary.bzl b/python/py_binary.bzl index c7d57dab49..48ea768948 100644 --- a/python/py_binary.bzl +++ b/python/py_binary.bzl @@ -38,9 +38,9 @@ def py_binary(**attrs): **attrs: Rule attributes forwarded onto the underlying {rule}`py_binary`. """ if attrs.get("python_version") == "PY2": - fail("Python 2 is no longer supported: https://github.com/bazelbuild/rules_python/issues/886") + fail("Python 2 is no longer supported: https://github.com/bazel-contrib/rules_python/issues/886") if attrs.get("srcs_version") in ("PY2", "PY2ONLY"): - fail("Python 2 is no longer supported: https://github.com/bazelbuild/rules_python/issues/886") + fail("Python 2 is no longer supported: https://github.com/bazel-contrib/rules_python/issues/886") _py_binary_impl(**add_migration_tag(attrs)) diff --git a/python/py_library.bzl b/python/py_library.bzl index 12354a7deb..8b8d46870b 100644 --- a/python/py_library.bzl +++ b/python/py_library.bzl @@ -37,7 +37,7 @@ def py_library(**attrs): **attrs: Rule attributes forwarded onto {rule}`py_library`. """ if attrs.get("srcs_version") in ("PY2", "PY2ONLY"): - fail("Python 2 is no longer supported: https://github.com/bazelbuild/rules_python/issues/886") + fail("Python 2 is no longer supported: https://github.com/bazel-contrib/rules_python/issues/886") _py_library_impl(**add_migration_tag(attrs)) diff --git a/python/py_runtime.bzl b/python/py_runtime.bzl index 2c44523505..dad2965cf5 100644 --- a/python/py_runtime.bzl +++ b/python/py_runtime.bzl @@ -37,6 +37,6 @@ def py_runtime(**attrs): **attrs: Rule attributes forwarded onto {rule}`py_runtime`. """ if attrs.get("python_version") == "PY2": - fail("Python 2 is no longer supported: see https://github.com/bazelbuild/rules_python/issues/886") + fail("Python 2 is no longer supported: see https://github.com/bazel-contrib/rules_python/issues/886") _py_runtime_impl(**add_migration_tag(attrs)) diff --git a/python/py_runtime_pair.bzl b/python/py_runtime_pair.bzl index b1e90414a2..26d378fce2 100644 --- a/python/py_runtime_pair.bzl +++ b/python/py_runtime_pair.bzl @@ -85,7 +85,7 @@ def py_runtime_pair(name, py2_runtime = None, py3_runtime = None, **attrs): **attrs: Extra attrs passed onto the native rule """ if attrs.get("py2_runtime"): - fail("PYthon 2 is no longer supported: see https://github.com/bazelbuild/rules_python/issues/886") + fail("PYthon 2 is no longer supported: see https://github.com/bazel-contrib/rules_python/issues/886") _py_runtime_pair( name = name, py2_runtime = py2_runtime, diff --git a/python/py_test.bzl b/python/py_test.bzl index 7f6626e0e5..b5657730b7 100644 --- a/python/py_test.bzl +++ b/python/py_test.bzl @@ -38,9 +38,9 @@ def py_test(**attrs): **attrs: Rule attributes forwarded onto {rule}`py_test`. """ if attrs.get("python_version") == "PY2": - fail("Python 2 is no longer supported: https://github.com/bazelbuild/rules_python/issues/886") + fail("Python 2 is no longer supported: https://github.com/bazel-contrib/rules_python/issues/886") if attrs.get("srcs_version") in ("PY2", "PY2ONLY"): - fail("Python 2 is no longer supported: https://github.com/bazelbuild/rules_python/issues/886") + fail("Python 2 is no longer supported: https://github.com/bazel-contrib/rules_python/issues/886") # buildifier: disable=native-python _py_test_impl(**add_migration_tag(attrs)) diff --git a/python/runfiles/BUILD.bazel b/python/runfiles/BUILD.bazel index a541b296a8..2040403b10 100644 --- a/python/runfiles/BUILD.bazel +++ b/python/runfiles/BUILD.bazel @@ -39,7 +39,7 @@ py_library( # This can be manually tested by running tests/runfiles/runfiles_wheel_integration_test.sh # We ought to have an automated integration test for it, too. -# see https://github.com/bazelbuild/rules_python/issues/1002 +# see https://github.com/bazel-contrib/rules_python/issues/1002 py_wheel( name = "wheel", # From https://pypi.org/classifiers/ @@ -50,7 +50,7 @@ py_wheel( description_file = "README.md", dist_folder = "dist", distribution = "bazel_runfiles", - homepage = "https://github.com/bazelbuild/rules_python", + homepage = "https://github.com/bazel-contrib/rules_python", python_requires = ">=3.7", strip_path_prefixes = ["python"], twine = None if BZLMOD_ENABLED else "@rules_python_publish_deps_twine//:pkg", diff --git a/sphinxdocs/docs/readthedocs.md b/sphinxdocs/docs/readthedocs.md index 66e4be82ea..c347d19850 100644 --- a/sphinxdocs/docs/readthedocs.md +++ b/sphinxdocs/docs/readthedocs.md @@ -119,7 +119,7 @@ if os.environ.get("READTHEDOCS") == "True": # Insert after the main extension extensions.insert(1, "readthedocs_ext.external_version_warning") readthedocs_vcs_url = ( - "http://github.com/bazelbuild/rules_python/pull/{}".format( + "http://github.com/bazel-contrib/rules_python/pull/{}".format( os.environ.get("READTHEDOCS_VERSION", "") ) ) diff --git a/tests/integration/custom_commands_test.py b/tests/integration/custom_commands_test.py index f78ee468bd..2e9cb741b0 100644 --- a/tests/integration/custom_commands_test.py +++ b/tests/integration/custom_commands_test.py @@ -19,7 +19,7 @@ class CustomCommandsTest(runner.TestCase): - # Regression test for https://github.com/bazelbuild/rules_python/issues/1840 + # Regression test for https://github.com/bazel-contrib/rules_python/issues/1840 def test_run_build_python_zip_false(self): result = self.run_bazel("run", "--build_python_zip=false", "//:bin") self.assert_result_matches(result, "bazel-out") diff --git a/tests/no_unsafe_paths/test.py b/tests/no_unsafe_paths/test.py index 1f6cd4e569..893add2f62 100644 --- a/tests/no_unsafe_paths/test.py +++ b/tests/no_unsafe_paths/test.py @@ -32,7 +32,7 @@ def test_no_unsafe_paths_in_search_path(self): # < Python 3.11 behaviour if (major, minor) < (3, 11): - # Because of https://github.com/bazelbuild/rules_python/blob/0.39.0/python/private/stage2_bootstrap_template.py#L415-L436 + # Because of https://github.com/bazel-contrib/rules_python/blob/0.39.0/python/private/stage2_bootstrap_template.py#L415-L436 self.assertEqual(os.path.dirname(sys.argv[0]), sys.path[0]) self.assertEqual(os.path.basename(sys.path[1]), archive) # >= Python 3.11 behaviour diff --git a/tests/packaging/BUILD.bazel b/tests/packaging/BUILD.bazel index cc04c05ba9..bb12269e3d 100644 --- a/tests/packaging/BUILD.bazel +++ b/tests/packaging/BUILD.bazel @@ -32,7 +32,7 @@ py_reconfig_test( main = "bin.py", target_compatible_with = SUPPORTS_BOOTSTRAP_SCRIPT, # Needed until https://github.com/bazelbuild/rules_pkg/issues/929 is fixed - # See: https://github.com/bazelbuild/rules_python/issues/2489 + # See: https://github.com/bazel-contrib/rules_python/issues/2489 venvs_use_declare_symlink = "no", ) diff --git a/third_party/rules_pycross/pycross/private/tools/wheel_installer.py b/third_party/rules_pycross/pycross/private/tools/wheel_installer.py index c03c4c2523..a122e67733 100644 --- a/third_party/rules_pycross/pycross/private/tools/wheel_installer.py +++ b/third_party/rules_pycross/pycross/private/tools/wheel_installer.py @@ -90,7 +90,7 @@ def main(args: Any) -> None: destination=destination, # Additional metadata that is generated by the installation tool. additional_metadata={ - "INSTALLER": b"https://github.com/bazelbuild/rules_python/tree/main/third_party/rules_pycross", + "INSTALLER": b"https://github.com/bazel-contrib/rules_python/tree/main/third_party/rules_pycross", }, ) finally: From dea960a759f22ee70603e92b2abafaa421a4b64b Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Thu, 20 Mar 2025 19:20:34 -0700 Subject: [PATCH 114/922] chore: update bcr metadata files to specify bazel-contrib (#2686) BCR presubmits require that the list of repositories match where downloads come from Along the way, also update the URL homepages to bazel-contrib and change the email to my personal instead of work email. From 175fe4cbe25f574abb2a516cd805cd664a4f7ddf Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Thu, 20 Mar 2025 19:54:05 -0700 Subject: [PATCH 115/922] docs: add steps for creating release candidates (#2687) We've done release candidates for the last couple releases and I think it's gone well, so document how to do them. --- RELEASING.md | 45 ++++++++++++++++++++++++++++++++++++++------- 1 file changed, 38 insertions(+), 7 deletions(-) diff --git a/RELEASING.md b/RELEASING.md index 6e441cbce6..82510b99c7 100644 --- a/RELEASING.md +++ b/RELEASING.md @@ -2,12 +2,16 @@ Start from a clean checkout at `main`. -Before running through the release it's good to run the build and the tests locally, and make sure CI is passing. You can -also test-drive the commit in an existing Bazel workspace to sanity check functionality. +Before running through the release it's good to run the build and the tests +locally, and make sure CI is passing. You can also test-drive the commit in an +existing Bazel workspace to sanity check functionality. ## Releasing from HEAD +These are the steps for a regularly scheduled release from HEAD. + ### Steps + 1. [Determine the next semantic version number](#determining-semantic-version). 1. Update CHANGELOG.md: replace the `v0-0-0` and `0.0.0` with `X.Y.0`. 1. Replace `VERSION_NEXT_*` strings with `X.Y.0`. @@ -16,12 +20,26 @@ also test-drive the commit in an existing Bazel workspace to sanity check functi ``` git branch --no-track release/X.Y upstream/main && git push upstream release/X.Y ``` -1. Create a tag and push: + +The next step is to create tags to trigger release workflow, **however** +we start by using release candidate tags (`X.Y.Z-rcN`) before tagging the +final release (`X.Y.Z`). + +1. Create release candidate tag and push. Increment `N` for each rc. + ``` + git tag X.Y.0-rcN upstream/release/X.Y && git push upstream --tags + ``` +2. Announce the RC release: see [Announcing Releases] +3. Wait a week for feedback. + * Follow [Patch release with cherry picks] to pull bug fixes into the + release branch. + * Repeat the RC tagging step, incrementing `N`. +4. Finally, tag the final release tag: ``` git tag X.Y.0 upstream/release/X.Y && git push upstream --tags ``` - **NOTE:** Pushing the tag will trigger release automation. -1. Release automation will create a GitHub release and BCR pull request. + +Release automation will create a GitHub release and BCR pull request. ### Determining Semantic Version @@ -55,9 +73,22 @@ each. Once the release branch is in the desired state, use `git tag` to tag it, as done with a release from head. Release automation will do the rest. -### After release creation in Github +### Announcing releases + +We announce releases in the #python channel in the Bazel slack +(bazelbuild.slack.com). Here's a template: + +``` +Greetings Pythonistas, + +rules_python X.Y.Z-rcN is now available +Changelog: https://rules-python.readthedocs.io/en/X.Y.Z-rcN/changelog.html#vX-Y-Z + +It will be promoted to stable next week, pending feedback. +``` -1. Announce the release in the #python channel in the Bazel slack (bazelbuild.slack.com). +It's traditional to include notable changes from the changelog, but not +required. ## Secrets From 1299307b939c9d7ec6df07d2082121885afd942e Mon Sep 17 00:00:00 2001 From: Ignas Anikevicius <240938+aignas@users.noreply.github.com> Date: Mon, 24 Mar 2025 08:46:30 +0900 Subject: [PATCH 116/922] fix(toolchain): no chmod on windows when downloading hermetic toolchain (#2693) Previously the code would not chmod for the Windows hermetic toolchains because there is usually no need - Windows does not have chmod and if you are downloading the Windows repo on a UNIX system, you won't run it, so it will stay as is. However, that left a single case where somebody may want to download the Linux toolchain on a Windows and the main cases are: * `bazel sync` * build a docker image on Windows using `rules_oci` or similar. Fixes #2660 --- CHANGELOG.md | 4 +++- python/private/python_repository.bzl | 4 +++- 2 files changed, 6 insertions(+), 2 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index dc40a25961..f8fd29fa5b 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -57,7 +57,9 @@ Unreleased changes template. {#v0-0-0-fixed} ### Fixed -* Nothing fixed. +* Do not try to run `chmod` when downloading non-windows hermetic toolchain + repositories on Windows. Fixes + [#2660](https://github.com/bazel-contrib/rules_python/issues/2660). {#v0-0-0-added} ### Added diff --git a/python/private/python_repository.bzl b/python/private/python_repository.bzl index 0534f9cd69..f3ec13d67d 100644 --- a/python/private/python_repository.bzl +++ b/python/private/python_repository.bzl @@ -127,7 +127,9 @@ def _python_repository_impl(rctx): # pycs being generated at runtime: # * The pycs are not deterministic (they contain timestamps) # * Multiple processes trying to write the same pycs can result in errors. - if "windows" not in platform: + # + # Note, when on Windows the `chmod` may not work + if "windows" not in platform and "windows" != repo_utils.get_platforms_os_name(rctx): repo_utils.execute_checked( rctx, op = "python_repository.MakeReadOnly", From d713ba704e9a6442c409134f7a701c0b6e1a9fe0 Mon Sep 17 00:00:00 2001 From: Logan Pulley Date: Sun, 23 Mar 2025 20:35:19 -0500 Subject: [PATCH 117/922] fix: correctly find runfiles root for symlinks (#2665) `$maybe_runfiles_root` doesn't seem to be a real variable. Based on the presence of the `while` loop, it seems that this code wants to try resolving the symlink one level at a time (`readlink`, not `realpath`) until it can find runfiles? --------- Co-authored-by: Ignas Anikevicius <240938+aignas@users.noreply.github.com> --- CHANGELOG.md | 1 + python/private/stage1_bootstrap_template.sh | 3 +- tests/bootstrap_impls/BUILD.bazel | 15 +++++ .../run_binary_find_runfiles_test.sh | 59 +++++++++++++++++++ 4 files changed, 76 insertions(+), 2 deletions(-) create mode 100755 tests/bootstrap_impls/run_binary_find_runfiles_test.sh diff --git a/CHANGELOG.md b/CHANGELOG.md index f8fd29fa5b..5e05096ceb 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -57,6 +57,7 @@ Unreleased changes template. {#v0-0-0-fixed} ### Fixed +* (runfiles) ({obj}`--bootstrap_impl=script`) Follow symlinks when searching for runfiles. * Do not try to run `chmod` when downloading non-windows hermetic toolchain repositories on Windows. Fixes [#2660](https://github.com/bazel-contrib/rules_python/issues/2660). diff --git a/python/private/stage1_bootstrap_template.sh b/python/private/stage1_bootstrap_template.sh index e548c848a5..c487624934 100644 --- a/python/private/stage1_bootstrap_template.sh +++ b/python/private/stage1_bootstrap_template.sh @@ -81,8 +81,7 @@ else if [[ ! -L "$stub_filename" ]]; then break fi - target=$(realpath $maybe_runfiles_root) - stub_filename="$target" + stub_filename=$(readlink $stub_filename) done echo >&2 "Unable to find runfiles directory for $1" exit 1 diff --git a/tests/bootstrap_impls/BUILD.bazel b/tests/bootstrap_impls/BUILD.bazel index e464a98e98..28a0d21fb7 100644 --- a/tests/bootstrap_impls/BUILD.bazel +++ b/tests/bootstrap_impls/BUILD.bazel @@ -70,6 +70,13 @@ sh_py_run_test( venvs_use_declare_symlink = "no", ) +sh_py_run_test( + name = "run_binary_find_runfiles_test", + py_src = "bin.py", + sh_src = "run_binary_find_runfiles_test.sh", + target_compatible_with = SUPPORTS_BOOTSTRAP_SCRIPT, +) + sh_py_run_test( name = "run_binary_bootstrap_script_zip_yes_test", bootstrap_impl = "script", @@ -88,6 +95,14 @@ sh_py_run_test( target_compatible_with = SUPPORTS_BOOTSTRAP_SCRIPT, ) +sh_py_run_test( + name = "run_binary_bootstrap_script_find_runfiles_test", + bootstrap_impl = "script", + py_src = "bin.py", + sh_src = "run_binary_find_runfiles_test.sh", + target_compatible_with = SUPPORTS_BOOTSTRAP_SCRIPT, +) + py_reconfig_test( name = "sys_path_order_bootstrap_script_test", srcs = ["sys_path_order_test.py"], diff --git a/tests/bootstrap_impls/run_binary_find_runfiles_test.sh b/tests/bootstrap_impls/run_binary_find_runfiles_test.sh new file mode 100755 index 0000000000..a6c1b565db --- /dev/null +++ b/tests/bootstrap_impls/run_binary_find_runfiles_test.sh @@ -0,0 +1,59 @@ +# Copyright 2023 The Bazel Authors. All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +# --- begin runfiles.bash initialization v3 --- +# Copy-pasted from the Bazel Bash runfiles library v3. +set -uo pipefail; set +e; f=bazel_tools/tools/bash/runfiles/runfiles.bash +source "${RUNFILES_DIR:-/dev/null}/$f" 2>/dev/null || \ + source "$(grep -sm1 "^$f " "${RUNFILES_MANIFEST_FILE:-/dev/null}" | cut -f2- -d' ')" 2>/dev/null || \ + source "$0.runfiles/$f" 2>/dev/null || \ + source "$(grep -sm1 "^$f " "$0.runfiles_manifest" | cut -f2- -d' ')" 2>/dev/null || \ + source "$(grep -sm1 "^$f " "$0.exe.runfiles_manifest" | cut -f2- -d' ')" 2>/dev/null || \ + { echo>&2 "ERROR: cannot find $f"; exit 1; }; f=; set -e +# --- end runfiles.bash initialization v3 --- +set +e + +bin=$(rlocation $BIN_RLOCATION) +if [[ -z "$bin" ]]; then + echo "Unable to locate test binary: $BIN_RLOCATION" + exit 1 +fi + +bin_link_layer_1=$TEST_TMPDIR/link1 +ln -s "$bin" "$bin_link_layer_1" +bin_link_layer_2=$TEST_TMPDIR/link2 +ln -s "$bin_link_layer_1" "$bin_link_layer_2" + +result=$(RUNFILES_DIR='' RUNFILES_MANIFEST_FILE='' $bin) +result_link_layer_1=$(RUNFILES_DIR='' RUNFILES_MANIFEST_FILE='' $bin_link_layer_1) +result_link_layer_2=$(RUNFILES_DIR='' RUNFILES_MANIFEST_FILE='' $bin_link_layer_2) + +if [[ "$result" != "$result_link_layer_1" ]]; then + echo "Output from test does not match output when invoked via a link;" + echo "Output from test:" + echo "$result" + echo "Output when invoked via a link:" + echo "$result_link_layer_1" + exit 1 +fi +if [[ "$result" != "$result_link_layer_2" ]]; then + echo "Output from test does not match output when invoked via a link to a link;" + echo "Output from test:" + echo "$result" + echo "Output when invoked via a link to a link:" + echo "$result_link_layer_2" + exit 1 +fi + +exit 0 From bfc03143d860109a2a8f2d13e5c129e4b9b4eb8a Mon Sep 17 00:00:00 2001 From: Levi Zim Date: Mon, 24 Mar 2025 16:08:57 +0800 Subject: [PATCH 118/922] feat: add riscv64 linux support (#2694) This patch introduces support for riscv64 linux platform, which is supported in python-build-standalone since https://github.com/astral-sh/python-build-standalone/releases/tag/20250115 Because it only gets supported recently, I updated python version maps to match latest release. The msvc `-shared` variant is no longer offered after 20250311 release. So I updated the corresponding msvc builds to the normal variant. --- CHANGELOG.md | 12 ++++- MODULE.bazel | 9 +++- gazelle/deps.bzl | 6 +-- python/versions.bzl | 120 +++++++++++++++++++++++++++++++++----------- 4 files changed, 111 insertions(+), 36 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 5e05096ceb..057ff78f14 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -53,7 +53,14 @@ Unreleased changes template. {#v0-0-0-changed} ### Changed -* Nothing changed. +* (toolchains) Use the latest astrahl-sh toolchain release [20250317] for Python versions: + * 3.9.21 + * 3.10.16 + * 3.11.11 + * 3.12.9 + * 3.13.2 + +[20250317]: https://github.com/astral-sh/python-build-standalone/releases/tag/20250317 {#v0-0-0-fixed} ### Fixed @@ -64,7 +71,8 @@ Unreleased changes template. {#v0-0-0-added} ### Added -* Nothing added. +* Add support for riscv64 linux platform. +* (toolchains) Add python 3.13.2 and 3.12.9 toolchains {#v0-0-0-removed} ### Removed diff --git a/MODULE.bazel b/MODULE.bazel index dc2193cec2..e4e45af7f0 100644 --- a/MODULE.bazel +++ b/MODULE.bazel @@ -131,7 +131,7 @@ dev_pip.parse( download_only = True, experimental_index_url = "https://pypi.org/simple", hub_name = "dev_pip", - python_version = "3.13.0", + python_version = "3.13", requirements_lock = "//docs:requirements.txt", ) dev_pip.parse( @@ -221,6 +221,13 @@ uv.default( ], platform = "s390x-unknown-linux-gnu", ) +uv.default( + compatible_with = [ + "@platforms//os:linux", + "@platforms//cpu:riscv64", + ], + platform = "riscv64-unknown-linux-gnu", +) uv.default( compatible_with = [ "@platforms//os:macos", diff --git a/gazelle/deps.bzl b/gazelle/deps.bzl index fbb5285a4c..7253ef8194 100644 --- a/gazelle/deps.bzl +++ b/gazelle/deps.bzl @@ -26,9 +26,9 @@ def python_stdlib_list_deps(): http_archive( name = "python_stdlib_list", build_file_content = """exports_files(glob(["stdlib_list/lists/*.txt"]))""", - sha256 = "3f6fc8fba0a99ce8fa76c1b794a24f38962f6275ea9d5cfb43a874abe472571e", - strip_prefix = "stdlib-list-0.10.0", - url = "https://github.com/pypi/stdlib-list/releases/download/v0.10.0/v0.10.0.tar.gz", + sha256 = "aa21a4f219530e85ecc364f0bbff2df4e6097a8954c63652af060f4e64afa65d", + strip_prefix = "stdlib-list-0.11.0", + url = "https://github.com/pypi/stdlib-list/releases/download/v0.11.0/v0.11.0.tar.gz", ) def gazelle_deps(): diff --git a/python/versions.bzl b/python/versions.bzl index b88aa47171..57a960c6a9 100644 --- a/python/versions.bzl +++ b/python/versions.bzl @@ -253,16 +253,17 @@ TOOL_VERSIONS = { "strip_prefix": "python", }, "3.9.21": { - "url": "20241206/cpython-{python_version}+20241206-{platform}-{build}.tar.gz", + "url": "20250317/cpython-{python_version}+20250317-{platform}-{build}.tar.gz", "sha256": { - "aarch64-apple-darwin": "4bddc18228789d0316dcebc45b2242e0010fa6bc33c302b6b5a62a5ac39d2147", - "aarch64-unknown-linux-gnu": "7d3b4ab90f73fa9dab0c350ca64b1caa9b8e4655913acd098e594473c49921c8", - "ppc64le-unknown-linux-gnu": "966477345ca93f056cf18de9cff961aacda2318a8e641546e0fd7222f1362ee2", - "s390x-unknown-linux-gnu": "3ba05a408edce4e20ebd116643c8418e62f7c8066c8a35fe8d3b78371d90b46a", - "x86_64-apple-darwin": "619f5082288c771ad9b71e2daaf6df6bd39ca86e442638d150a71a6ccf62978d", - "x86_64-pc-windows-msvc": "82736b5a185c57b296188ce778ed865ff10edc5fe9ff1ec4cb33b39ac8e4819c", - "x86_64-unknown-linux-gnu": "208b2adc7c7e5d5df6d9385400dc7c4e3b4c3eed428e19a2326848978e98517e", - "x86_64-unknown-linux-musl": "67c058dbaae8fd8c4f68e13b10805a9227918afc94326f21a9a2ec2daca3ddbd", + "aarch64-apple-darwin": "2a7d83db10c082ce59e9c4b8bd6c5790310198fb759a7c94aceebac1d93676d3", + "aarch64-unknown-linux-gnu": "758ebbc4d60b3ca26cf21720232043ad626373fbeb6632122e5db622a1f55465", + "ppc64le-unknown-linux-gnu": "3c7c0cc16468659049ac2f843ffba29144dd987869c943b83c2730569b7f57bd", + "riscv64-unknown-linux-gnu": "ef1463ad5349419309060854a5f942b0bd7bd0b9245b53980129836187e68ad9", + "s390x-unknown-linux-gnu": "e66e52dcbe3e20153e7d5844451bf58a69f41b858348e0f59c547444bfe191ee", + "x86_64-apple-darwin": "786ebd91e4dd0920acf60aa3428a627a937342d2455f7eb5e9a491517c32db3d", + "x86_64-pc-windows-msvc": "5392cee2ef7cd20b34128384d0b31864fb3c02bdb7a8ae6995cfec621bb657bc", + "x86_64-unknown-linux-gnu": "6f426b5494e90701ffa2753e229252e8b3ac61151a09c8cd6c0a649512df8ab2", + "x86_64-unknown-linux-musl": "6113c6c5f88d295bb26279b8a49d74126ee12db137854e0d8c3077051a4eddc4", }, "strip_prefix": "python", }, @@ -387,16 +388,17 @@ TOOL_VERSIONS = { "strip_prefix": "python", }, "3.10.16": { - "url": "20241206/cpython-{python_version}+20241206-{platform}-{build}.tar.gz", + "url": "20250317/cpython-{python_version}+20250317-{platform}-{build}.tar.gz", "sha256": { - "aarch64-apple-darwin": "c2d25840756127f3583b04b0697bef79edacb15f1402cd980292c93488c3df22", - "aarch64-unknown-linux-gnu": "bbfc345615c5ed33916b4fd959fc16fa2e896a3c5eec1fb782c91b47c85c0542", - "ppc64le-unknown-linux-gnu": "cb474b392733d5ac2adaa1cfcc2b63b957611dc26697e76822706cc61ac21515", - "s390x-unknown-linux-gnu": "886a7effc8a3061d53cacc9cf54e82d6d57ac3665c258c6a2193528c16b557cd", - "x86_64-apple-darwin": "31a110b631eb79103675ed556255045deeea5ff533296d7f35b4d195a0df0315", - "x86_64-pc-windows-msvc": "fb7870717dc7e3aedcbab4a647782637da0046a4238db1d41eeaabb78566d814", - "x86_64-unknown-linux-gnu": "b15de0d63eed9871ed57285f81fd123cf6c4117251a9cac8f81f9cf0cccc0a53", - "x86_64-unknown-linux-musl": "bf956eeffcff002d2f38232faa750c279cbb76197b744761d1b253bf94d6f637", + "aarch64-apple-darwin": "e99f8457d9c79592c036489c5cfa78df76e4762d170665e499833e045d82608f", + "aarch64-unknown-linux-gnu": "76d0f04d2444e77200fdc70d1c57480e29cca78cb7420d713bc1c523709c198d", + "ppc64le-unknown-linux-gnu": "39c9b3486de984fe1d72d90278229c70d6b08bcf69cd55796881b2d75077b603", + "riscv64-unknown-linux-gnu": "ebe949ada9293581c17d9bcdaa8f645f67d95f73eac65def760a71ef9dd6600d", + "s390x-unknown-linux-gnu": "9b2fc0b7f1c75b48e799b6fa14f7e24f5c61f2db82e3c65d13ed25e08f7f0857", + "x86_64-apple-darwin": "e03e62dbe95afa2f56b7344ff3bd061b180a0b690ff77f9a1d7e6601935e05ca", + "x86_64-pc-windows-msvc": "c7e0eb0ff5b36758b7a8cacd42eb223c056b9c4d36eded9bf5b9fe0c0b9aeb08", + "x86_64-unknown-linux-gnu": "b350c7e63956ca8edb856b91316328e0fd003a840cbd63d08253af43b2c63643", + "x86_64-unknown-linux-musl": "6ed64923ee4fbea4c5780f1a5a66651d239191ac10bd23420db4f5e4e0bf79c4", }, "strip_prefix": "python", }, @@ -516,16 +518,17 @@ TOOL_VERSIONS = { "strip_prefix": "python", }, "3.11.11": { - "url": "20241206/cpython-{python_version}+20241206-{platform}-{build}.tar.gz", + "url": "20250317/cpython-{python_version}+20250317-{platform}-{build}.tar.gz", "sha256": { - "aarch64-apple-darwin": "566c5e266f2c933d0c0b213a75496bc6a090e493097802f809dbe21c75cd5d13", - "aarch64-unknown-linux-gnu": "50ee364cfa24ee7d933eda955c9fe455bc0a8ebb9d998c9948f2909dac701dd9", - "ppc64le-unknown-linux-gnu": "e0cdc00e42a05191b9b75ba976fc0fca9205c66fdaef7571c20532346fd3db1e", - "s390x-unknown-linux-gnu": "3b106b8a3c5aa97ff76200cd0d9ba6eaed23d88ccb947e00ff6bb2d9f5422d2a", - "x86_64-apple-darwin": "8ecd267281fb5b2464ddcd2de79622cfa7aff42e929b17989da2721ba39d4a5e", - "x86_64-pc-windows-msvc": "d8986f026599074ddd206f3f62d6f2c323ca8fa7a854bf744989bfc0b12f5d0d", - "x86_64-unknown-linux-gnu": "57a171af687c926c5cabe3d1c7ce9950b98f00b932accd596eb60e14ca39c42d", - "x86_64-unknown-linux-musl": "8129a9a5c3f2654e1a9eed6093f5dc42399667b341050ff03219cb7df210c348", + "aarch64-apple-darwin": "19b147c7e4b742656da4cb6ba35bc3ea2f15aa5f4d1bbbc38d09e2e85551e927", + "aarch64-unknown-linux-gnu": "7d52b5206afe617de2899af477f5a1d275ecbce80fb8300301b254ebf1da5a90", + "ppc64le-unknown-linux-gnu": "17c049f70ce719adc89dd0ae26f4e6a28f6aaedc63c2efef6bbb9c112ea4d692", + "riscv64-unknown-linux-gnu": "83ed50713409576756f5708e8f0549a15c17071bea22b71f15e11a7084f09481", + "s390x-unknown-linux-gnu": "298507f1f8d962b1bb98cb506c99e7e0d291a63eb9117e1521141e6b3825fd56", + "x86_64-apple-darwin": "a870cd965e7dded5100d13b1d34cab1c32a92811e000d10fbfe9bbdb36cdaa0e", + "x86_64-pc-windows-msvc": "1cf5760eea0a9df3308ca2c4111b5cc18fd638b2a912dbe07606193e3f9aa123", + "x86_64-unknown-linux-gnu": "51e47bc0d1b9f4bf68dd395f7a39f60c58a87cde854cab47264a859eb666bb69", + "x86_64-unknown-linux-musl": "ee4d84f992c6a1df42096e26b970fe5938fd6c1eadd245894bc94c5737ff9977", }, "strip_prefix": "python", }, @@ -622,6 +625,21 @@ TOOL_VERSIONS = { }, "strip_prefix": "python", }, + "3.12.9": { + "url": "20250317/cpython-{python_version}+20250317-{platform}-{build}.tar.gz", + "sha256": { + "aarch64-apple-darwin": "7c7fd9809da0382a601a79287b5d62d61ce0b15f5a5ee836233727a516e85381", + "aarch64-unknown-linux-gnu": "00c6bf9acef21ac741fea24dc449d0149834d30e9113429e50a95cce4b00bb80", + "ppc64le-unknown-linux-gnu": "25d77599dfd5849f17391d92da0da99079e4e94f19a881f763f5cc62530ef7e1", + "riscv64-unknown-linux-gnu": "e97ab0fdf443b302c56a52b4fd08f513bf3be66aa47263f0f9df3c6e60e05f2e", + "s390x-unknown-linux-gnu": "7492d079ffa8425c8f6c58e43b237c37e3fb7b31e2e14635927bb4d3397ba21e", + "x86_64-apple-darwin": "1ee1b1bb9fbce5c145c4bec9a3c98d7a4fa22543e09a7c1d932bc8599283c2dc", + "x86_64-pc-windows-msvc": "d15361fd202dd74ae9c3eece1abdab7655f1eba90bf6255cad1d7c53d463ed4d", + "x86_64-unknown-linux-gnu": "ef382fb88cbb41a3b0801690bd716b8a1aec07a6c6471010bcc6bd14cd575226", + "x86_64-unknown-linux-musl": "94e3837da1adf9964aab2d6047b33f70167de3096d1f9a2d1fa9340b1bbf537d", + }, + "strip_prefix": "python", + }, "3.13.0": { "url": "20241016/cpython-{python_version}+20241016-{platform}-{build}.{ext}", "sha256": { @@ -696,6 +714,47 @@ TOOL_VERSIONS = { "x86_64-unknown-linux-gnu-freethreaded": "python/install", }, }, + "3.13.2": { + "url": "20250317/cpython-{python_version}+20250317-{platform}-{build}.{ext}", + "sha256": { + "aarch64-apple-darwin": "faa44274a331eb39786362818b21b3a4e74514e8805000b20b0e55c590cecb94", + "aarch64-unknown-linux-gnu": "9c67260446fee6ea706dad577a0b32936c63f449c25d66e4383d5846b2ab2e36", + "ppc64le-unknown-linux-gnu": "345b53d2f86c9dbd7f1320657cb227ff9a42ef63ff21f129abbbc8c82a375147", + "riscv64-unknown-linux-gnu": "172d22b2330737f3a028ea538ffe497c39a066a8d3200b22dd4d177a3332ad85", + "s390x-unknown-linux-gnu": "ec3b16ea8a97e3138acec72bc5ff35949950c62c8994a8ec8e213fd93f0e806b", + "x86_64-apple-darwin": "ee4526e84b5ce5b11141c50060b385320f2773616249a741f90c96d460ce8e8f", + "x86_64-pc-windows-msvc": "84d7b52f3558c8e35c670a4fa14080c75e3ec584adfae49fec8b51008b75b21e", + "x86_64-unknown-linux-gnu": "db011f0cd29cab2291584958f4e2eb001b0e6051848d89b38a2dc23c5c54e512", + "x86_64-unknown-linux-musl": "00bb2d629f7eacbb5c6b44dc04af26d1f1da64cee3425b0d8eb5135a93830296", + "aarch64-apple-darwin-freethreaded": "c98c9c977e6fa05c3813bd49f3553904d89d60fed27e2e36468da7afa1d6d5e2", + "aarch64-unknown-linux-gnu-freethreaded": "b8635e59e3143fd17f19a3dfe8ccc246ee6587c87da359bd1bcab35eefbb5f19", + "ppc64le-unknown-linux-gnu-freethreaded": "6ae8fa44cb2edf4ab49cff1820b53c40c10349c0f39e11b8cd76ce7f3e7e1def", + "riscv64-unknown-linux-gnu-freethreaded": "2af1b8850c52801fb6189e7a17a51e0c93d9e46ddefcca72247b76329c97d02a", + "s390x-unknown-linux-gnu-freethreaded": "c074144cc80c2af32c420b79a9df26e8db405212619990c1fbdd308bd75afe3f", + "x86_64-apple-darwin-freethreaded": "0d73e4348d8d4b5159058609d2303705190405b485dd09ad05d870d7e0f36e0f", + "x86_64-pc-windows-msvc-freethreaded": "c51b4845fda5421e044067c111192f645234081d704313f74ee77fa013a186ea", + "x86_64-unknown-linux-gnu-freethreaded": "1aea5062614c036904b55c1cc2fb4b500b7f6f7a4cacc263f4888889d355eef8", + }, + "strip_prefix": { + "aarch64-apple-darwin": "python", + "aarch64-unknown-linux-gnu": "python", + "ppc64le-unknown-linux-gnu": "python", + "s390x-unknown-linux-gnu": "python", + "riscv64-unknown-linux-gnu": "python", + "x86_64-apple-darwin": "python", + "x86_64-pc-windows-msvc": "python", + "x86_64-unknown-linux-gnu": "python", + "x86_64-unknown-linux-musl": "python", + "aarch64-apple-darwin-freethreaded": "python/install", + "aarch64-unknown-linux-gnu-freethreaded": "python/install", + "ppc64le-unknown-linux-gnu-freethreaded": "python/install", + "riscv64-unknown-linux-gnu-freethreaded": "python/install", + "s390x-unknown-linux-gnu-freethreaded": "python/install", + "x86_64-apple-darwin-freethreaded": "python/install", + "x86_64-pc-windows-msvc-freethreaded": "python/install", + "x86_64-unknown-linux-gnu-freethreaded": "python/install", + }, + }, } # buildifier: disable=unsorted-dict-items @@ -704,8 +763,8 @@ MINOR_MAPPING = { "3.9": "3.9.21", "3.10": "3.10.16", "3.11": "3.11.11", - "3.12": "3.12.8", - "3.13": "3.13.1", + "3.12": "3.12.9", + "3.13": "3.13.2", } def _generate_platforms(): @@ -895,6 +954,7 @@ def get_release_info(platform, python_version, base_url = DEFAULT_RELEASE_BASE_U "aarch64-apple-darwin": "pgo+lto", "aarch64-unknown-linux-gnu": "lto", "ppc64le-unknown-linux-gnu": "lto", + "riscv64-unknown-linux-gnu": "lto", "s390x-unknown-linux-gnu": "lto", "x86_64-apple-darwin": "pgo+lto", "x86_64-pc-windows-msvc": "pgo", @@ -904,7 +964,7 @@ def get_release_info(platform, python_version, base_url = DEFAULT_RELEASE_BASE_U else: build = INSTALL_ONLY - if WINDOWS_NAME in platform: + if WINDOWS_NAME in platform and int(u.split("/")[0]) < 20250317: build = "shared-" + build release_filename = u.format( From 6acff2ae607f6caf927980ac28d3948458b881f8 Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Mon, 24 Mar 2025 19:07:31 -0700 Subject: [PATCH 119/922] build(deps): bump urllib3 from 2.2.3 to 2.3.0 in /tools/publish (#2699) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Bumps [urllib3](https://github.com/urllib3/urllib3) from 2.2.3 to 2.3.0.
Release notes

Sourced from urllib3's releases.

2.3.0

🚀 urllib3 is fundraising for HTTP/2 support

urllib3 is raising ~$40,000 USD to release HTTP/2 support and ensure long-term sustainable maintenance of the project after a sharp decline in financial support for 2023. If your company or organization uses Python and would benefit from HTTP/2 support in Requests, pip, cloud SDKs, and thousands of other projects please consider contributing financially to ensure HTTP/2 support is developed sustainably and maintained for the long-haul.

Thank you for your support.

Features

  • Added HTTPResponse.shutdown() to stop any ongoing or future reads for a specific response. It calls shutdown(SHUT_RD) on the underlying socket. This feature was sponsored by LaunchDarkly. (urllib3/urllib3#2868)
  • Added support for JavaScript Promise Integration on Emscripten. This enables more efficient WebAssembly requests and streaming, and makes it possible to use in Node.js if you launch it as node --experimental-wasm-stack-switching. (urllib3/urllib3#3400)
  • Added the proxy_is_tunneling property to HTTPConnection and HTTPSConnection. (urllib3/urllib3#3285)
  • Added pickling support to NewConnectionError and NameResolutionError. (urllib3/urllib3#3480)

Bugfixes

  • Fixed an issue in debug logs where the HTTP version was rendering as "HTTP/11" instead of "HTTP/1.1". (urllib3/urllib3#3489)

Deprecations and Removals

Full Changelog: https://github.com/urllib3/urllib3/compare/2.2.3...2.3.0

Changelog

Sourced from urllib3's changelog.

2.3.0 (2024-12-22)

Features

  • Added HTTPResponse.shutdown() to stop any ongoing or future reads for a specific response. It calls shutdown(SHUT_RD) on the underlying socket. This feature was sponsored by LaunchDarkly <https://opencollective.com/urllib3/contributions/815307>. ([#2868](https://github.com/urllib3/urllib3/issues/2868) <https://github.com/urllib3/urllib3/issues/2868>)
  • Added support for JavaScript Promise Integration on Emscripten. This enables more efficient WebAssembly requests and streaming, and makes it possible to use in Node.js if you launch it as node --experimental-wasm-stack-switching. ([#3400](https://github.com/urllib3/urllib3/issues/3400) <https://github.com/urllib3/urllib3/issues/3400>__)
  • Added the proxy_is_tunneling property to HTTPConnection and HTTPSConnection. ([#3285](https://github.com/urllib3/urllib3/issues/3285) <https://github.com/urllib3/urllib3/issues/3285>__)
  • Added pickling support to NewConnectionError and NameResolutionError. ([#3480](https://github.com/urllib3/urllib3/issues/3480) <https://github.com/urllib3/urllib3/issues/3480>__)

Bugfixes

  • Fixed an issue in debug logs where the HTTP version was rendering as "HTTP/11" instead of "HTTP/1.1". ([#3489](https://github.com/urllib3/urllib3/issues/3489) <https://github.com/urllib3/urllib3/issues/3489>__)

Deprecations and Removals

  • Removed support for Python 3.8. ([#3492](https://github.com/urllib3/urllib3/issues/3492) <https://github.com/urllib3/urllib3/issues/3492>__)
Commits

[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=urllib3&package-manager=pip&previous-version=2.2.3&new-version=2.3.0)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) ---
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot show ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> --- tools/publish/requirements_darwin.txt | 6 +++--- tools/publish/requirements_linux.txt | 6 +++--- tools/publish/requirements_universal.txt | 6 +++--- tools/publish/requirements_windows.txt | 6 +++--- 4 files changed, 12 insertions(+), 12 deletions(-) diff --git a/tools/publish/requirements_darwin.txt b/tools/publish/requirements_darwin.txt index 31c0a0402f..2517b22ff7 100644 --- a/tools/publish/requirements_darwin.txt +++ b/tools/publish/requirements_darwin.txt @@ -215,9 +215,9 @@ twine==5.1.1 \ --hash=sha256:215dbe7b4b94c2c50a7315c0275d2258399280fbb7d04182c7e55e24b5f93997 \ --hash=sha256:9aa0825139c02b3434d913545c7b847a21c835e11597f5255842d457da2322db # via -r tools/publish/requirements.in -urllib3==2.2.3 \ - --hash=sha256:ca899ca043dcb1bafa3e262d73aa25c465bfb49e0bd9dd5d59f1d0acba2f8fac \ - --hash=sha256:e7d814a81dad81e6caf2ec9fdedb284ecc9c73076b62654547cc64ccdcae26e9 +urllib3==2.3.0 \ + --hash=sha256:1cee9ad369867bfdbbb48b7dd50374c0967a0bb7710050facf0dd6911440e3df \ + --hash=sha256:f8c5449b3cf0861679ce7e0503c7b44b5ec981bec0d1d3795a07f1ba96f0204d # via # requests # twine diff --git a/tools/publish/requirements_linux.txt b/tools/publish/requirements_linux.txt index 31ced6af74..8aeed63726 100644 --- a/tools/publish/requirements_linux.txt +++ b/tools/publish/requirements_linux.txt @@ -327,9 +327,9 @@ twine==5.1.1 \ --hash=sha256:215dbe7b4b94c2c50a7315c0275d2258399280fbb7d04182c7e55e24b5f93997 \ --hash=sha256:9aa0825139c02b3434d913545c7b847a21c835e11597f5255842d457da2322db # via -r tools/publish/requirements.in -urllib3==2.2.3 \ - --hash=sha256:ca899ca043dcb1bafa3e262d73aa25c465bfb49e0bd9dd5d59f1d0acba2f8fac \ - --hash=sha256:e7d814a81dad81e6caf2ec9fdedb284ecc9c73076b62654547cc64ccdcae26e9 +urllib3==2.3.0 \ + --hash=sha256:1cee9ad369867bfdbbb48b7dd50374c0967a0bb7710050facf0dd6911440e3df \ + --hash=sha256:f8c5449b3cf0861679ce7e0503c7b44b5ec981bec0d1d3795a07f1ba96f0204d # via # requests # twine diff --git a/tools/publish/requirements_universal.txt b/tools/publish/requirements_universal.txt index 6e2502835e..1528b85244 100644 --- a/tools/publish/requirements_universal.txt +++ b/tools/publish/requirements_universal.txt @@ -331,9 +331,9 @@ twine==5.1.1 \ --hash=sha256:215dbe7b4b94c2c50a7315c0275d2258399280fbb7d04182c7e55e24b5f93997 \ --hash=sha256:9aa0825139c02b3434d913545c7b847a21c835e11597f5255842d457da2322db # via -r tools/publish/requirements.in -urllib3==2.2.3 \ - --hash=sha256:ca899ca043dcb1bafa3e262d73aa25c465bfb49e0bd9dd5d59f1d0acba2f8fac \ - --hash=sha256:e7d814a81dad81e6caf2ec9fdedb284ecc9c73076b62654547cc64ccdcae26e9 +urllib3==2.3.0 \ + --hash=sha256:1cee9ad369867bfdbbb48b7dd50374c0967a0bb7710050facf0dd6911440e3df \ + --hash=sha256:f8c5449b3cf0861679ce7e0503c7b44b5ec981bec0d1d3795a07f1ba96f0204d # via # requests # twine diff --git a/tools/publish/requirements_windows.txt b/tools/publish/requirements_windows.txt index 3733696678..ba6a30d737 100644 --- a/tools/publish/requirements_windows.txt +++ b/tools/publish/requirements_windows.txt @@ -219,9 +219,9 @@ twine==5.1.1 \ --hash=sha256:215dbe7b4b94c2c50a7315c0275d2258399280fbb7d04182c7e55e24b5f93997 \ --hash=sha256:9aa0825139c02b3434d913545c7b847a21c835e11597f5255842d457da2322db # via -r tools/publish/requirements.in -urllib3==2.2.3 \ - --hash=sha256:ca899ca043dcb1bafa3e262d73aa25c465bfb49e0bd9dd5d59f1d0acba2f8fac \ - --hash=sha256:e7d814a81dad81e6caf2ec9fdedb284ecc9c73076b62654547cc64ccdcae26e9 +urllib3==2.3.0 \ + --hash=sha256:1cee9ad369867bfdbbb48b7dd50374c0967a0bb7710050facf0dd6911440e3df \ + --hash=sha256:f8c5449b3cf0861679ce7e0503c7b44b5ec981bec0d1d3795a07f1ba96f0204d # via # requests # twine From 86708181feefd0e8654cd6aafc56738704b10273 Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Mon, 24 Mar 2025 19:07:47 -0700 Subject: [PATCH 120/922] build(deps): bump urllib3 from 2.2.3 to 2.3.0 in /docs (#2698) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Bumps [urllib3](https://github.com/urllib3/urllib3) from 2.2.3 to 2.3.0.
Release notes

Sourced from urllib3's releases.

2.3.0

🚀 urllib3 is fundraising for HTTP/2 support

urllib3 is raising ~$40,000 USD to release HTTP/2 support and ensure long-term sustainable maintenance of the project after a sharp decline in financial support for 2023. If your company or organization uses Python and would benefit from HTTP/2 support in Requests, pip, cloud SDKs, and thousands of other projects please consider contributing financially to ensure HTTP/2 support is developed sustainably and maintained for the long-haul.

Thank you for your support.

Features

  • Added HTTPResponse.shutdown() to stop any ongoing or future reads for a specific response. It calls shutdown(SHUT_RD) on the underlying socket. This feature was sponsored by LaunchDarkly. (urllib3/urllib3#2868)
  • Added support for JavaScript Promise Integration on Emscripten. This enables more efficient WebAssembly requests and streaming, and makes it possible to use in Node.js if you launch it as node --experimental-wasm-stack-switching. (urllib3/urllib3#3400)
  • Added the proxy_is_tunneling property to HTTPConnection and HTTPSConnection. (urllib3/urllib3#3285)
  • Added pickling support to NewConnectionError and NameResolutionError. (urllib3/urllib3#3480)

Bugfixes

  • Fixed an issue in debug logs where the HTTP version was rendering as "HTTP/11" instead of "HTTP/1.1". (urllib3/urllib3#3489)

Deprecations and Removals

Full Changelog: https://github.com/urllib3/urllib3/compare/2.2.3...2.3.0

Changelog

Sourced from urllib3's changelog.

2.3.0 (2024-12-22)

Features

  • Added HTTPResponse.shutdown() to stop any ongoing or future reads for a specific response. It calls shutdown(SHUT_RD) on the underlying socket. This feature was sponsored by LaunchDarkly <https://opencollective.com/urllib3/contributions/815307>. ([#2868](https://github.com/urllib3/urllib3/issues/2868) <https://github.com/urllib3/urllib3/issues/2868>)
  • Added support for JavaScript Promise Integration on Emscripten. This enables more efficient WebAssembly requests and streaming, and makes it possible to use in Node.js if you launch it as node --experimental-wasm-stack-switching. ([#3400](https://github.com/urllib3/urllib3/issues/3400) <https://github.com/urllib3/urllib3/issues/3400>__)
  • Added the proxy_is_tunneling property to HTTPConnection and HTTPSConnection. ([#3285](https://github.com/urllib3/urllib3/issues/3285) <https://github.com/urllib3/urllib3/issues/3285>__)
  • Added pickling support to NewConnectionError and NameResolutionError. ([#3480](https://github.com/urllib3/urllib3/issues/3480) <https://github.com/urllib3/urllib3/issues/3480>__)

Bugfixes

  • Fixed an issue in debug logs where the HTTP version was rendering as "HTTP/11" instead of "HTTP/1.1". ([#3489](https://github.com/urllib3/urllib3/issues/3489) <https://github.com/urllib3/urllib3/issues/3489>__)

Deprecations and Removals

  • Removed support for Python 3.8. ([#3492](https://github.com/urllib3/urllib3/issues/3492) <https://github.com/urllib3/urllib3/issues/3492>__)
Commits

[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=urllib3&package-manager=pip&previous-version=2.2.3&new-version=2.3.0)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) ---
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot show ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> --- docs/requirements.txt | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/requirements.txt b/docs/requirements.txt index bc9b3b411b..581feb0893 100644 --- a/docs/requirements.txt +++ b/docs/requirements.txt @@ -370,7 +370,7 @@ typing-extensions==4.12.2 \ # via # rules-python-docs (docs/pyproject.toml) # sphinx-autodoc2 -urllib3==2.2.3 \ - --hash=sha256:ca899ca043dcb1bafa3e262d73aa25c465bfb49e0bd9dd5d59f1d0acba2f8fac \ - --hash=sha256:e7d814a81dad81e6caf2ec9fdedb284ecc9c73076b62654547cc64ccdcae26e9 +urllib3==2.3.0 \ + --hash=sha256:1cee9ad369867bfdbbb48b7dd50374c0967a0bb7710050facf0dd6911440e3df \ + --hash=sha256:f8c5449b3cf0861679ce7e0503c7b44b5ec981bec0d1d3795a07f1ba96f0204d # via requests From ab70bca371521bd6e223296ae02a686703a06778 Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Mon, 24 Mar 2025 19:08:42 -0700 Subject: [PATCH 121/922] build(deps): bump babel from 2.16.0 to 2.17.0 in /docs (#2696) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Bumps [babel](https://github.com/python-babel/babel) from 2.16.0 to 2.17.0.
Release notes

Sourced from babel's releases.

v2.17.0

Happy 2025! This release is being made from FOSDEM 2025, in Brussels, Belgium. 🇧🇪

Thank you to all contributors, new and old, and here's to another great year of internationalization and localization!


The changelog below is auto-generated by GitHub.

Please see CHANGELOG.rst for additional details.


What's Changed

New Contributors

... (truncated)

Changelog

Sourced from babel's changelog.

Version 2.17.0

Happy 2025! This release is being made from FOSDEM 2025, in Brussels, Belgium.

Thank you to all contributors, new and old, and here's to another great year of internationalization and localization!

Features


* CLDR: Babel now uses CLDR 46, by @tomasr8 in :gh:`1145`
* Dates: Allow specifying an explicit format in parse_date/parse_time by
@tomasr8 in :gh:`1131`
* Dates: More alternate characters are now supported by
`format_skeleton`. By @tomasr8 in :gh:`1122`
* Dates: Support short and narrow formats for format_timedelta when
using `add_direction`, by @akx in :gh:`1163`
* Messages: .po files now enclose white spaces in filenames like GNU
gettext does. By @Dunedan in :gh:`1105`, and @tomasr8 in :gh:`1120`
* Messages: Initial support for `Message.python_brace_format`, by
@tomasr8 in :gh:`1169`
* Numbers: LC_MONETARY is now preferred when formatting currencies, by
@akx in :gh:`1173`

Bugfixes

  • Dates: Make seconds optional in parse_time time formats by @​tomasr8 in :gh:1141
  • Dates: Replace str.index with str.find by @​tomasr8 in :gh:1130
  • Dates: Strip extra leading slashes in /etc/localtime by @​akx in :gh:1165
  • Dates: Week numbering and formatting of dates with week numbers was repaired by @​jun66j5 in :gh:1179
  • General: Improve handling for locale=None by @​akx in :gh:1164
  • General: Remove redundant assignment in Catalog.__setitem__ by @​tomasr8 in :gh:1167
  • Messages: Fix extracted lineno with nested calls, by @​dylankiss in :gh:1126
  • Messages: Fix of list index out of range when translations is empty, by @​gabe-sherman in :gh:1135
  • Messages: Fix the way obsolete messages are stored by @​tomasr8 in :gh:1132
  • Messages: Simplify read_mo logic regarding catalog.charset by @​tomasr8 in :gh:1148
  • Messages: Use the first matching method & options, rather than first matching method & last options, by @​jpmckinney in :gh:1121

Deprecation and compatibility


* Dates: Fix deprecation warnings for `datetime.utcnow()` by @tomasr8 in
:gh:`1119`
* Docs: Adjust docs/conf.py to add compatibility with sphinx 8 by
@hrnciar in :gh:`1155`
* General: Import `Literal` from the typing module by @tomasr8 in
:gh:`1175`
* General: Replace `OrderedDict` with just `dict` by @tomasr8 in
:gh:`1149`
* Messages: Mark `wraptext` deprecated; use `TextWrapper` directly in
`write_po` by @akx in :gh:`1140`

Infrastructure


* Add tzdata as dev dependency and sync with tox.ini by @wandrew004 in
:gh:`1159`
* Duplicate test code was deleted by @mattdiaz007 in :gh:`1138`
* Increase test coverage of the `python_format` checker by @tomasr8 in
:gh:`1176`
* Small cleanups by @akx in :gh:`1160`, :gh:`1166`, :gh:`1170` and
:gh:`1172`
&lt;/tr&gt;&lt;/table&gt;
</code></pre>
</blockquote>
<p>... (truncated)</p>
</details>
<details>
<summary>Commits</summary>

<ul>
<li><a
href="https://github.com/python-babel/babel/commit/b50a1d2186c20f3359f7e10853d2b2225a46ed40"><code>b50a1d2</code></a>
Prepare for 2.17.0 (<a
href="https://redirect.github.com/python-babel/babel/issues/1182">#1182</a>)</li>
<li><a
href="https://github.com/python-babel/babel/commit/5f117b2689573aa98acc8a47108c49b99f4d1394"><code>5f117b2</code></a>
Increase test coverage of the <code>python_format</code>
checker (<a
href="https://redirect.github.com/python-babel/babel/issues/1176">#1176</a>)</li>
<li><a
href="https://github.com/python-babel/babel/commit/363ad7531fb5dcdc3e9844573592b0b44afb914b"><code>363ad75</code></a>
Fix dates formatting <code>Y</code>,
<code>w</code> and <code>W</code> symbols for
week-numbering (<a
href="https://redirect.github.com/python-babel/babel/issues/1179">#1179</a>)</li>
<li><a
href="https://github.com/python-babel/babel/commit/e9c3ef8d0de3080ca59f7f8dbabf9b52983adc7d"><code>e9c3ef8</code></a>
Merge pull request <a
href="https://redirect.github.com/python-babel/babel/issues/1173">#1173</a>
from python-babel/lc-monetary-2</li>
<li><a
href="https://github.com/python-babel/babel/commit/56ef7c7f578a904917464c187e399abb762bd5e3"><code>56ef7c7</code></a>
Prefer LC_MONETARY when formatting currency</li>
<li><a
href="https://github.com/python-babel/babel/commit/aee6d698b541dc50439280d7e093092cc0d4b832"><code>aee6d69</code></a>
<code>default_locale</code>: support multiple
keys</li>
<li><a
href="https://github.com/python-babel/babel/commit/2d8a808864d1aae5d3d02d4f95917c79740c5d35"><code>2d8a808</code></a>
Import <code>Literal</code> &amp;
<code>TypedDict</code> from the typing module (<a
href="https://redirect.github.com/python-babel/babel/issues/1175">#1175</a>)</li>
<li><a
href="https://github.com/python-babel/babel/commit/98b9562c05e5276038c27ec12c12f3e92dc027b6"><code>98b9562</code></a>
Add basic support for
<code>Message.python_brace_format</code> (<a
href="https://redirect.github.com/python-babel/babel/issues/1169">#1169</a>)</li>
<li><a
href="https://github.com/python-babel/babel/commit/0c1091c9de9543e30bc4b845eb10b5bf84516d7b"><code>0c1091c</code></a>
Small test cleanup (<a
href="https://redirect.github.com/python-babel/babel/issues/1172">#1172</a>)</li>
<li><a
href="https://github.com/python-babel/babel/commit/db4879136a7fbcef475f26b75dbdd65d0ce488f9"><code>db48791</code></a>
Merge pull request <a
href="https://redirect.github.com/python-babel/babel/issues/1170">#1170</a>
from python-babel/small-cleanup</li>
<li>Additional commits viewable in <a
href="https://github.com/python-babel/babel/compare/v2.16.0...v2.17.0">compare
view</a></li>
</ul>
</details>

<br />
[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=babel&package-manager=pip&previous-version=2.16.0&new-version=2.17.0)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) ---
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot show ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> --- docs/requirements.txt | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/requirements.txt b/docs/requirements.txt index 581feb0893..a49e8f9fe2 100644 --- a/docs/requirements.txt +++ b/docs/requirements.txt @@ -14,9 +14,9 @@ astroid==3.3.6 \ --hash=sha256:6aaea045f938c735ead292204afdb977a36e989522b7833ef6fea94de743f442 \ --hash=sha256:db676dc4f3ae6bfe31cda227dc60e03438378d7a896aec57422c95634e8d722f # via sphinx-autodoc2 -babel==2.16.0 \ - --hash=sha256:368b5b98b37c06b7daf6696391c3240c938b37767d4584413e8438c5c435fa8b \ - --hash=sha256:d1f3554ca26605fe173f3de0c65f750f5a42f924499bf134de6423582298e316 +babel==2.17.0 \ + --hash=sha256:0c54cffb19f690cdcc52a3b50bcbf71e07a808d1c80d549f2459b9d2cf0afb9d \ + --hash=sha256:4d0b53093fdfb4b21c92b5213dba5a1b23885afa8383709427046b21c366e5f2 # via sphinx certifi==2024.8.30 \ --hash=sha256:922820b53db7a7257ffbda3f597266d435245903d80737e34f8a45ff3e3230d8 \ From 8485290b38275dfa75a902ff264f74282bbcd2e8 Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Tue, 25 Mar 2025 11:17:21 +0900 Subject: [PATCH 122/922] build(deps): bump keyring from 25.4.1 to 25.5.0 in /tools/publish (#2355) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Bumps [keyring](https://github.com/jaraco/keyring) from 25.4.1 to 25.5.0.
Changelog

Sourced from keyring's changelog.

v25.5.0

Features

  • When parsing keyring_path from the config, the home directory is now expanded from ~. (#696)

Bugfixes

  • In get_credential, now returns None when the indicated username is not found. (#698)
Commits

[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=keyring&package-manager=pip&previous-version=25.4.1&new-version=25.5.0)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) You can trigger a rebase of this PR by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) ---
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot show ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
> **Note** > Automatic rebases have been disabled on this pull request as it has been open for over 30 days. Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> --- tools/publish/requirements_darwin.txt | 6 +++--- tools/publish/requirements_linux.txt | 6 +++--- tools/publish/requirements_universal.txt | 6 +++--- tools/publish/requirements_windows.txt | 6 +++--- 4 files changed, 12 insertions(+), 12 deletions(-) diff --git a/tools/publish/requirements_darwin.txt b/tools/publish/requirements_darwin.txt index 2517b22ff7..9c9398ade5 100644 --- a/tools/publish/requirements_darwin.txt +++ b/tools/publish/requirements_darwin.txt @@ -143,9 +143,9 @@ jaraco-functools==4.1.0 \ --hash=sha256:70f7e0e2ae076498e212562325e805204fc092d7b4c17e0e86c959e249701a9d \ --hash=sha256:ad159f13428bc4acbf5541ad6dec511f91573b90fba04df61dafa2a1231cf649 # via keyring -keyring==25.4.1 \ - --hash=sha256:5426f817cf7f6f007ba5ec722b1bcad95a75b27d780343772ad76b17cb47b0bf \ - --hash=sha256:b07ebc55f3e8ed86ac81dd31ef14e81ace9dd9c3d4b5d77a6e9a2016d0d71a1b +keyring==25.5.0 \ + --hash=sha256:4c753b3ec91717fe713c4edd522d625889d8973a349b0e582622f49766de58e6 \ + --hash=sha256:e67f8ac32b04be4714b42fe84ce7dad9c40985b9ca827c592cc303e7c26d9741 # via twine markdown-it-py==3.0.0 \ --hash=sha256:355216845c60bd96232cd8d8c40e8f9765cc86f46880e43a8fd22dc1a1a8cab1 \ diff --git a/tools/publish/requirements_linux.txt b/tools/publish/requirements_linux.txt index 8aeed63726..147fb2d206 100644 --- a/tools/publish/requirements_linux.txt +++ b/tools/publish/requirements_linux.txt @@ -247,9 +247,9 @@ jeepney==0.8.0 \ # via # keyring # secretstorage -keyring==25.4.1 \ - --hash=sha256:5426f817cf7f6f007ba5ec722b1bcad95a75b27d780343772ad76b17cb47b0bf \ - --hash=sha256:b07ebc55f3e8ed86ac81dd31ef14e81ace9dd9c3d4b5d77a6e9a2016d0d71a1b +keyring==25.5.0 \ + --hash=sha256:4c753b3ec91717fe713c4edd522d625889d8973a349b0e582622f49766de58e6 \ + --hash=sha256:e67f8ac32b04be4714b42fe84ce7dad9c40985b9ca827c592cc303e7c26d9741 # via twine markdown-it-py==3.0.0 \ --hash=sha256:355216845c60bd96232cd8d8c40e8f9765cc86f46880e43a8fd22dc1a1a8cab1 \ diff --git a/tools/publish/requirements_universal.txt b/tools/publish/requirements_universal.txt index 1528b85244..2ad13f5688 100644 --- a/tools/publish/requirements_universal.txt +++ b/tools/publish/requirements_universal.txt @@ -247,9 +247,9 @@ jeepney==0.8.0 ; sys_platform == 'linux' \ # via # keyring # secretstorage -keyring==25.4.1 \ - --hash=sha256:5426f817cf7f6f007ba5ec722b1bcad95a75b27d780343772ad76b17cb47b0bf \ - --hash=sha256:b07ebc55f3e8ed86ac81dd31ef14e81ace9dd9c3d4b5d77a6e9a2016d0d71a1b +keyring==25.5.0 \ + --hash=sha256:4c753b3ec91717fe713c4edd522d625889d8973a349b0e582622f49766de58e6 \ + --hash=sha256:e67f8ac32b04be4714b42fe84ce7dad9c40985b9ca827c592cc303e7c26d9741 # via twine markdown-it-py==3.0.0 \ --hash=sha256:355216845c60bd96232cd8d8c40e8f9765cc86f46880e43a8fd22dc1a1a8cab1 \ diff --git a/tools/publish/requirements_windows.txt b/tools/publish/requirements_windows.txt index ba6a30d737..bb87804df5 100644 --- a/tools/publish/requirements_windows.txt +++ b/tools/publish/requirements_windows.txt @@ -143,9 +143,9 @@ jaraco-functools==4.1.0 \ --hash=sha256:70f7e0e2ae076498e212562325e805204fc092d7b4c17e0e86c959e249701a9d \ --hash=sha256:ad159f13428bc4acbf5541ad6dec511f91573b90fba04df61dafa2a1231cf649 # via keyring -keyring==25.4.1 \ - --hash=sha256:5426f817cf7f6f007ba5ec722b1bcad95a75b27d780343772ad76b17cb47b0bf \ - --hash=sha256:b07ebc55f3e8ed86ac81dd31ef14e81ace9dd9c3d4b5d77a6e9a2016d0d71a1b +keyring==25.5.0 \ + --hash=sha256:4c753b3ec91717fe713c4edd522d625889d8973a349b0e582622f49766de58e6 \ + --hash=sha256:e67f8ac32b04be4714b42fe84ce7dad9c40985b9ca827c592cc303e7c26d9741 # via twine markdown-it-py==3.0.0 \ --hash=sha256:355216845c60bd96232cd8d8c40e8f9765cc86f46880e43a8fd22dc1a1a8cab1 \ From e2d4ac8ed6d64cc4646db71e7c94ebdf8ca0b93a Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Tue, 25 Mar 2025 11:27:09 +0900 Subject: [PATCH 123/922] build(deps): bump django from 4.2.17 to 4.2.20 in /examples/bzlmod_build_file_generation (#2689) Bumps [django](https://github.com/django/django) from 4.2.17 to 4.2.20.
Commits
  • 35c58a7 [4.2.x] Bumped version for 4.2.20 release.
  • e88f737 [4.2.x] Fixed CVE-2025-26699 -- Mitigated potential DoS in wordwrap template ...
  • 348e46a [4.2.x] Added stub release notes and release date for 4.2.20.
  • 73e2107 [4.2.x] Post-release version bump.
  • db89d2f [4.2.x] Bumped version for 4.2.19 release.
  • 83231cc [4.2.x] Added release date for 4.2.19.
  • 7bd1ddf [4.2.x] Refs #34060 -- Adjusted CVE-2024-53908 regression test for psycopg2.
  • 57b0229 [4.2.x] Refs #36098 -- Fixed validate_ipv4_address() crash for non-string val...
  • 043dfad [4.2.x] Fixed #36098 -- Fixed validate_ipv6_address()/validate_ipv46_address(...
  • 8769b44 [4.2.x] Added CVE-2024-56374 to security archive.
  • Additional commits viewable in compare view

[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=django&package-manager=pip&previous-version=4.2.17&new-version=4.2.20)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) ---
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot show ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) You can disable automated security fix PRs for this repo from the [Security Alerts page](https://github.com/bazel-contrib/rules_python/network/alerts).
Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> --- examples/bzlmod_build_file_generation/requirements_lock.txt | 6 +++--- .../bzlmod_build_file_generation/requirements_windows.txt | 6 +++--- 2 files changed, 6 insertions(+), 6 deletions(-) diff --git a/examples/bzlmod_build_file_generation/requirements_lock.txt b/examples/bzlmod_build_file_generation/requirements_lock.txt index 7bf1e2200f..5c1b7a86e8 100644 --- a/examples/bzlmod_build_file_generation/requirements_lock.txt +++ b/examples/bzlmod_build_file_generation/requirements_lock.txt @@ -26,9 +26,9 @@ dill==0.3.6 \ --hash=sha256:a07ffd2351b8c678dfc4a856a3005f8067aea51d6ba6c700796a4d9e280f39f0 \ --hash=sha256:e5db55f3687856d8fbdab002ed78544e1c4559a130302693d839dfe8f93f2373 # via pylint -django==4.2.17 \ - --hash=sha256:3a93350214ba25f178d4045c0786c61573e7dbfa3c509b3551374f1e11ba8de0 \ - --hash=sha256:6b56d834cc94c8b21a8f4e775064896be3b4a4ca387f2612d4406a5927cd2fdc +django==4.2.20 \ + --hash=sha256:213381b6e4405f5c8703fffc29cd719efdf189dec60c67c04f76272b3dc845b9 \ + --hash=sha256:92bac5b4432a64532abb73b2ac27203f485e40225d2640a7fbef2b62b876e789 # via # -r requirements.in # django-stubs diff --git a/examples/bzlmod_build_file_generation/requirements_windows.txt b/examples/bzlmod_build_file_generation/requirements_windows.txt index 8a796a3718..309dfbcf40 100644 --- a/examples/bzlmod_build_file_generation/requirements_windows.txt +++ b/examples/bzlmod_build_file_generation/requirements_windows.txt @@ -30,9 +30,9 @@ dill==0.3.6 \ --hash=sha256:a07ffd2351b8c678dfc4a856a3005f8067aea51d6ba6c700796a4d9e280f39f0 \ --hash=sha256:e5db55f3687856d8fbdab002ed78544e1c4559a130302693d839dfe8f93f2373 # via pylint -django==4.2.17 \ - --hash=sha256:3a93350214ba25f178d4045c0786c61573e7dbfa3c509b3551374f1e11ba8de0 \ - --hash=sha256:6b56d834cc94c8b21a8f4e775064896be3b4a4ca387f2612d4406a5927cd2fdc +django==4.2.20 \ + --hash=sha256:213381b6e4405f5c8703fffc29cd719efdf189dec60c67c04f76272b3dc845b9 \ + --hash=sha256:92bac5b4432a64532abb73b2ac27203f485e40225d2640a7fbef2b62b876e789 # via # -r requirements.in # django-stubs From 06f6f316c27cf5dd57930536c2264fb99ddd18a9 Mon Sep 17 00:00:00 2001 From: Simon Stewart Date: Tue, 25 Mar 2025 02:30:21 +0000 Subject: [PATCH 124/922] fix: Correctly resolve macOS SDK paths (#2478) XCode has facilities for accurately telling us where SDKs are installed. This is important to use, particularly when there may be multiple SDKs or versions of XCode installed. --------- Co-authored-by: Ignas Anikevicius <240938+aignas@users.noreply.github.com> --- CHANGELOG.md | 1 + python/private/pypi/whl_library.bzl | 52 ++++++++++++++++++++++++----- 2 files changed, 44 insertions(+), 9 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 057ff78f14..96bf33dbd5 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -59,6 +59,7 @@ Unreleased changes template. * 3.11.11 * 3.12.9 * 3.13.2 +* (pypi) Use `xcrun xcodebuild --showsdks` to find XCode root. [20250317]: https://github.com/astral-sh/python-build-standalone/releases/tag/20250317 diff --git a/python/private/pypi/whl_library.bzl b/python/private/pypi/whl_library.bzl index 9bbd842116..38ac9dcd92 100644 --- a/python/private/pypi/whl_library.bzl +++ b/python/private/pypi/whl_library.bzl @@ -30,7 +30,7 @@ _CPPFLAGS = "CPPFLAGS" _COMMAND_LINE_TOOLS_PATH_SLUG = "commandlinetools" _WHEEL_ENTRY_POINT_PREFIX = "rules_python_wheel_entry_point" -def _get_xcode_location_cflags(rctx): +def _get_xcode_location_cflags(rctx, logger = None): """Query the xcode sdk location to update cflags Figure out if this interpreter target comes from rules_python, and patch the xcode sdk location if so. @@ -46,6 +46,7 @@ def _get_xcode_location_cflags(rctx): rctx, op = "GetXcodeLocation", arguments = [repo_utils.which_checked(rctx, "xcode-select"), "--print-path"], + logger = logger, ) if xcode_sdk_location.return_code != 0: return [] @@ -55,9 +56,37 @@ def _get_xcode_location_cflags(rctx): # This is a full xcode installation somewhere like /Applications/Xcode13.0.app/Contents/Developer # so we need to change the path to to the macos specific tools which are in a different relative # path than xcode installed command line tools. - xcode_root = "{}/Platforms/MacOSX.platform/Developer".format(xcode_root) + xcode_sdks_json = repo_utils.execute_checked( + rctx, + op = "LocateXCodeSDKs", + arguments = [ + repo_utils.which_checked(rctx, "xcrun"), + "xcodebuild", + "-showsdks", + "-json", + ], + environment = { + "DEVELOPER_DIR": xcode_root, + }, + logger = logger, + ).stdout + xcode_sdks = json.decode(xcode_sdks_json) + potential_sdks = [ + sdk + for sdk in xcode_sdks + if "productName" in sdk and + sdk["productName"] == "macOS" and + "darwinos" not in sdk["canonicalName"] + ] + + # Now we'll get two entries here (one for internal and another one for public) + # It shouldn't matter which one we pick. + xcode_sdk_path = potential_sdks[0]["sdkPath"] + else: + xcode_sdk_path = "{}/SDKs/MacOSX.sdk".format(xcode_root) + return [ - "-isysroot {}/SDKs/MacOSX.sdk".format(xcode_root), + "-isysroot {}".format(xcode_sdk_path), ] def _get_toolchain_unix_cflags(rctx, python_interpreter, logger = None): @@ -84,6 +113,7 @@ def _get_toolchain_unix_cflags(rctx, python_interpreter, logger = None): "import sys; print(f'{sys.version_info[0]}.{sys.version_info[1]}', end='')", ], srcs = [], + logger = logger, ) _python_version = stdout include_path = "{}/include/python{}".format( @@ -176,19 +206,23 @@ def _create_repository_execution_environment(rctx, python_interpreter, logger = Dictionary of environment variable suitable to pass to rctx.execute. """ - # Gather any available CPPFLAGS values - cppflags = [] - cppflags.extend(_get_xcode_location_cflags(rctx)) - cppflags.extend(_get_toolchain_unix_cflags(rctx, python_interpreter, logger = logger)) - env = { "PYTHONPATH": pypi_repo_utils.construct_pythonpath( rctx, entries = rctx.attr._python_path_entries, ), - _CPPFLAGS: " ".join(cppflags), } + # Gather any available CPPFLAGS values + # + # We may want to build in an environment without a cc toolchain. + # In those cases, we're limited to --download-only, but we should respect that here. + is_wheel = rctx.attr.filename and rctx.attr.filename.endswith(".whl") + if not (rctx.attr.download_only or is_wheel): + cppflags = [] + cppflags.extend(_get_xcode_location_cflags(rctx, logger = logger)) + cppflags.extend(_get_toolchain_unix_cflags(rctx, python_interpreter, logger = logger)) + env[_CPPFLAGS] = " ".join(cppflags) return env def _whl_library_impl(rctx): From bfa59b93dead3e6c5c9f91063078b9e09c91ea5a Mon Sep 17 00:00:00 2001 From: Ignas Anikevicius <240938+aignas@users.noreply.github.com> Date: Thu, 27 Mar 2025 08:53:50 +0900 Subject: [PATCH 125/922] chore: remove old versions of Python 3.8 (#2700) Python 3.8 has reached EOL and this PR removes old toolchains and most of the tests. Users can still use it if they register the toolchains themselves, but `rules_python` will no longer keep testing the toolchains. Removing the toolchains all-together will be done at a later stage which may require us to be more clever how we handle asks to include `3.8`. Maybe we can just fail if the user asks for a python version that does not exist, but I am concerned that `rules_python` depending on `protobuf` may pull in code that requests `3.8`. I'll look at this at some later time. --- .bazelci/presubmit.yml | 2 - CHANGELOG.md | 5 +- examples/multi_python_versions/MODULE.bazel | 9 -- examples/multi_python_versions/WORKSPACE | 3 - .../requirements/BUILD.bazel | 7 -- .../requirements/requirements_lock_3_8.txt | 78 ----------------- .../multi_python_versions/tests/BUILD.bazel | 33 ------- python/versions.bzl | 85 ------------------- 8 files changed, 4 insertions(+), 218 deletions(-) delete mode 100644 examples/multi_python_versions/requirements/requirements_lock_3_8.txt diff --git a/.bazelci/presubmit.yml b/.bazelci/presubmit.yml index f1a912cf80..3b70734eff 100644 --- a/.bazelci/presubmit.yml +++ b/.bazelci/presubmit.yml @@ -78,12 +78,10 @@ buildifier: coverage_targets: - //tests:my_lib_3_10_test - //tests:my_lib_3_11_test - - //tests:my_lib_3_8_test - //tests:my_lib_3_9_test - //tests:my_lib_default_test - //tests:version_3_10_test - //tests:version_3_11_test - - //tests:version_3_8_test - //tests:version_3_9_test - //tests:version_default_test tasks: diff --git a/CHANGELOG.md b/CHANGELOG.md index 96bf33dbd5..c64241ccbf 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -60,13 +60,16 @@ Unreleased changes template. * 3.12.9 * 3.13.2 * (pypi) Use `xcrun xcodebuild --showsdks` to find XCode root. +* (toolchains) Remove all but `3.8.20` versions of the Python `3.8` interpreter who has + reached EOL. If users still need other versions of the `3.8` interpreter, please supply + the URLs manually {bzl:ob}`python.toolchain` or {bzl:obj}`python_register_toolchains` calls. [20250317]: https://github.com/astral-sh/python-build-standalone/releases/tag/20250317 {#v0-0-0-fixed} ### Fixed * (runfiles) ({obj}`--bootstrap_impl=script`) Follow symlinks when searching for runfiles. -* Do not try to run `chmod` when downloading non-windows hermetic toolchain +* (toolchains) Do not try to run `chmod` when downloading non-windows hermetic toolchain repositories on Windows. Fixes [#2660](https://github.com/bazel-contrib/rules_python/issues/2660). diff --git a/examples/multi_python_versions/MODULE.bazel b/examples/multi_python_versions/MODULE.bazel index 578315741f..74cb4b01df 100644 --- a/examples/multi_python_versions/MODULE.bazel +++ b/examples/multi_python_versions/MODULE.bazel @@ -10,10 +10,6 @@ local_path_override( ) python = use_extension("@rules_python//python/extensions:python.bzl", "python") -python.toolchain( - configure_coverage_tool = True, - python_version = "3.8", -) python.toolchain( configure_coverage_tool = True, # Only set when you have mulitple toolchain versions. @@ -36,11 +32,6 @@ use_repo( pip = use_extension("@rules_python//python/extensions:pip.bzl", "pip") use_repo(pip, "pypi") -pip.parse( - hub_name = "pypi", - python_version = "3.8", - requirements_lock = "//requirements:requirements_lock_3_8.txt", -) pip.parse( hub_name = "pypi", python_version = "3.9", diff --git a/examples/multi_python_versions/WORKSPACE b/examples/multi_python_versions/WORKSPACE index 48d2065282..6b69e0a891 100644 --- a/examples/multi_python_versions/WORKSPACE +++ b/examples/multi_python_versions/WORKSPACE @@ -15,7 +15,6 @@ python_register_multi_toolchains( name = "python", default_version = default_python_version, python_versions = [ - "3.8", "3.9", "3.10", "3.11", @@ -31,13 +30,11 @@ multi_pip_parse( python_interpreter_target = { "3.10": "@python_3_10_host//:python", "3.11": "@python_3_11_host//:python", - "3.8": "@python_3_8_host//:python", "3.9": "@python_3_9_host//:python", }, requirements_lock = { "3.10": "//requirements:requirements_lock_3_10.txt", "3.11": "//requirements:requirements_lock_3_11.txt", - "3.8": "//requirements:requirements_lock_3_8.txt", "3.9": "//requirements:requirements_lock_3_9.txt", }, ) diff --git a/examples/multi_python_versions/requirements/BUILD.bazel b/examples/multi_python_versions/requirements/BUILD.bazel index c9b695e8e4..516a378df8 100644 --- a/examples/multi_python_versions/requirements/BUILD.bazel +++ b/examples/multi_python_versions/requirements/BUILD.bazel @@ -1,12 +1,5 @@ load("@rules_python//python:pip.bzl", "compile_pip_requirements") -compile_pip_requirements( - name = "requirements_3_8", - src = "requirements.in", - python_version = "3.8", - requirements_txt = "requirements_lock_3_8.txt", -) - compile_pip_requirements( name = "requirements_3_9", src = "requirements.in", diff --git a/examples/multi_python_versions/requirements/requirements_lock_3_8.txt b/examples/multi_python_versions/requirements/requirements_lock_3_8.txt deleted file mode 100644 index 10b5df4830..0000000000 --- a/examples/multi_python_versions/requirements/requirements_lock_3_8.txt +++ /dev/null @@ -1,78 +0,0 @@ -# -# This file is autogenerated by pip-compile with Python 3.8 -# by the following command: -# -# bazel run //requirements:requirements_3_8.update -# -websockets==11.0.3 \ - --hash=sha256:01f5567d9cf6f502d655151645d4e8b72b453413d3819d2b6f1185abc23e82dd \ - --hash=sha256:03aae4edc0b1c68498f41a6772d80ac7c1e33c06c6ffa2ac1c27a07653e79d6f \ - --hash=sha256:0ac56b661e60edd453585f4bd68eb6a29ae25b5184fd5ba51e97652580458998 \ - --hash=sha256:0ee68fe502f9031f19d495dae2c268830df2760c0524cbac5d759921ba8c8e82 \ - --hash=sha256:1553cb82942b2a74dd9b15a018dce645d4e68674de2ca31ff13ebc2d9f283788 \ - --hash=sha256:1a073fc9ab1c8aff37c99f11f1641e16da517770e31a37265d2755282a5d28aa \ - --hash=sha256:1d2256283fa4b7f4c7d7d3e84dc2ece74d341bce57d5b9bf385df109c2a1a82f \ - --hash=sha256:1d5023a4b6a5b183dc838808087033ec5df77580485fc533e7dab2567851b0a4 \ - --hash=sha256:1fdf26fa8a6a592f8f9235285b8affa72748dc12e964a5518c6c5e8f916716f7 \ - --hash=sha256:2529338a6ff0eb0b50c7be33dc3d0e456381157a31eefc561771ee431134a97f \ - --hash=sha256:279e5de4671e79a9ac877427f4ac4ce93751b8823f276b681d04b2156713b9dd \ - --hash=sha256:2d903ad4419f5b472de90cd2d40384573b25da71e33519a67797de17ef849b69 \ - --hash=sha256:332d126167ddddec94597c2365537baf9ff62dfcc9db4266f263d455f2f031cb \ - --hash=sha256:34fd59a4ac42dff6d4681d8843217137f6bc85ed29722f2f7222bd619d15e95b \ - --hash=sha256:3580dd9c1ad0701169e4d6fc41e878ffe05e6bdcaf3c412f9d559389d0c9e016 \ - --hash=sha256:3ccc8a0c387629aec40f2fc9fdcb4b9d5431954f934da3eaf16cdc94f67dbfac \ - --hash=sha256:41f696ba95cd92dc047e46b41b26dd24518384749ed0d99bea0a941ca87404c4 \ - --hash=sha256:42cc5452a54a8e46a032521d7365da775823e21bfba2895fb7b77633cce031bb \ - --hash=sha256:4841ed00f1026dfbced6fca7d963c4e7043aa832648671b5138008dc5a8f6d99 \ - --hash=sha256:4b253869ea05a5a073ebfdcb5cb3b0266a57c3764cf6fe114e4cd90f4bfa5f5e \ - --hash=sha256:54c6e5b3d3a8936a4ab6870d46bdd6ec500ad62bde9e44462c32d18f1e9a8e54 \ - --hash=sha256:619d9f06372b3a42bc29d0cd0354c9bb9fb39c2cbc1a9c5025b4538738dbffaf \ - --hash=sha256:6505c1b31274723ccaf5f515c1824a4ad2f0d191cec942666b3d0f3aa4cb4007 \ - --hash=sha256:660e2d9068d2bedc0912af508f30bbeb505bbbf9774d98def45f68278cea20d3 \ - --hash=sha256:6681ba9e7f8f3b19440921e99efbb40fc89f26cd71bf539e45d8c8a25c976dc6 \ - --hash=sha256:68b977f21ce443d6d378dbd5ca38621755f2063d6fdb3335bda981d552cfff86 \ - --hash=sha256:69269f3a0b472e91125b503d3c0b3566bda26da0a3261c49f0027eb6075086d1 \ - --hash=sha256:6f1a3f10f836fab6ca6efa97bb952300b20ae56b409414ca85bff2ad241d2a61 \ - --hash=sha256:7622a89d696fc87af8e8d280d9b421db5133ef5b29d3f7a1ce9f1a7bf7fcfa11 \ - --hash=sha256:777354ee16f02f643a4c7f2b3eff8027a33c9861edc691a2003531f5da4f6bc8 \ - --hash=sha256:84d27a4832cc1a0ee07cdcf2b0629a8a72db73f4cf6de6f0904f6661227f256f \ - --hash=sha256:8531fdcad636d82c517b26a448dcfe62f720e1922b33c81ce695d0edb91eb931 \ - --hash=sha256:86d2a77fd490ae3ff6fae1c6ceaecad063d3cc2320b44377efdde79880e11526 \ - --hash=sha256:88fc51d9a26b10fc331be344f1781224a375b78488fc343620184e95a4b27016 \ - --hash=sha256:8a34e13a62a59c871064dfd8ffb150867e54291e46d4a7cf11d02c94a5275bae \ - --hash=sha256:8c82f11964f010053e13daafdc7154ce7385ecc538989a354ccc7067fd7028fd \ - --hash=sha256:92b2065d642bf8c0a82d59e59053dd2fdde64d4ed44efe4870fa816c1232647b \ - --hash=sha256:97b52894d948d2f6ea480171a27122d77af14ced35f62e5c892ca2fae9344311 \ - --hash=sha256:9d9acd80072abcc98bd2c86c3c9cd4ac2347b5a5a0cae7ed5c0ee5675f86d9af \ - --hash=sha256:9f59a3c656fef341a99e3d63189852be7084c0e54b75734cde571182c087b152 \ - --hash=sha256:aa5003845cdd21ac0dc6c9bf661c5beddd01116f6eb9eb3c8e272353d45b3288 \ - --hash=sha256:b16fff62b45eccb9c7abb18e60e7e446998093cdcb50fed33134b9b6878836de \ - --hash=sha256:b30c6590146e53149f04e85a6e4fcae068df4289e31e4aee1fdf56a0dead8f97 \ - --hash=sha256:b58cbf0697721120866820b89f93659abc31c1e876bf20d0b3d03cef14faf84d \ - --hash=sha256:b67c6f5e5a401fc56394f191f00f9b3811fe843ee93f4a70df3c389d1adf857d \ - --hash=sha256:bceab846bac555aff6427d060f2fcfff71042dba6f5fca7dc4f75cac815e57ca \ - --hash=sha256:bee9fcb41db2a23bed96c6b6ead6489702c12334ea20a297aa095ce6d31370d0 \ - --hash=sha256:c114e8da9b475739dde229fd3bc6b05a6537a88a578358bc8eb29b4030fac9c9 \ - --hash=sha256:c1f0524f203e3bd35149f12157438f406eff2e4fb30f71221c8a5eceb3617b6b \ - --hash=sha256:c792ea4eabc0159535608fc5658a74d1a81020eb35195dd63214dcf07556f67e \ - --hash=sha256:c7f3cb904cce8e1be667c7e6fef4516b98d1a6a0635a58a57528d577ac18a128 \ - --hash=sha256:d67ac60a307f760c6e65dad586f556dde58e683fab03323221a4e530ead6f74d \ - --hash=sha256:dcacf2c7a6c3a84e720d1bb2b543c675bf6c40e460300b628bab1b1efc7c034c \ - --hash=sha256:de36fe9c02995c7e6ae6efe2e205816f5f00c22fd1fbf343d4d18c3d5ceac2f5 \ - --hash=sha256:def07915168ac8f7853812cc593c71185a16216e9e4fa886358a17ed0fd9fcf6 \ - --hash=sha256:df41b9bc27c2c25b486bae7cf42fccdc52ff181c8c387bfd026624a491c2671b \ - --hash=sha256:e052b8467dd07d4943936009f46ae5ce7b908ddcac3fda581656b1b19c083d9b \ - --hash=sha256:e063b1865974611313a3849d43f2c3f5368093691349cf3c7c8f8f75ad7cb280 \ - --hash=sha256:e1459677e5d12be8bbc7584c35b992eea142911a6236a3278b9b5ce3326f282c \ - --hash=sha256:e1a99a7a71631f0efe727c10edfba09ea6bee4166a6f9c19aafb6c0b5917d09c \ - --hash=sha256:e590228200fcfc7e9109509e4d9125eace2042fd52b595dd22bbc34bb282307f \ - --hash=sha256:e6316827e3e79b7b8e7d8e3b08f4e331af91a48e794d5d8b099928b6f0b85f20 \ - --hash=sha256:e7837cb169eca3b3ae94cc5787c4fed99eef74c0ab9506756eea335e0d6f3ed8 \ - --hash=sha256:e848f46a58b9fcf3d06061d17be388caf70ea5b8cc3466251963c8345e13f7eb \ - --hash=sha256:ed058398f55163a79bb9f06a90ef9ccc063b204bb346c4de78efc5d15abfe602 \ - --hash=sha256:f2e58f2c36cc52d41f2659e4c0cbf7353e28c8c9e63e30d8c6d3494dc9fdedcf \ - --hash=sha256:f467ba0050b7de85016b43f5a22b46383ef004c4f672148a8abf32bc999a87f0 \ - --hash=sha256:f61bdb1df43dc9c131791fbc2355535f9024b9a04398d3bd0684fc16ab07df74 \ - --hash=sha256:fb06eea71a00a7af0ae6aefbb932fb8a7df3cb390cc217d51a9ad7343de1b8d0 \ - --hash=sha256:ffd7dcaf744f25f82190856bc26ed81721508fc5cbf2a330751e135ff1283564 - # via -r requirements/requirements.in diff --git a/examples/multi_python_versions/tests/BUILD.bazel b/examples/multi_python_versions/tests/BUILD.bazel index e3dfb48cca..11fb98ca61 100644 --- a/examples/multi_python_versions/tests/BUILD.bazel +++ b/examples/multi_python_versions/tests/BUILD.bazel @@ -22,13 +22,6 @@ py_binary( srcs = ["version_default.py"], ) -py_binary( - name = "version_3_8", - srcs = ["version.py"], - main = "version.py", - python_version = "3.8", -) - py_binary( name = "version_3_9", srcs = ["version.py"], @@ -57,14 +50,6 @@ py_test( deps = ["//libs/my_lib"], ) -py_test( - name = "my_lib_3_8_test", - srcs = ["my_lib_test.py"], - main = "my_lib_test.py", - python_version = "3.8", - deps = ["//libs/my_lib"], -) - py_test( name = "my_lib_3_9_test", srcs = ["my_lib_test.py"], @@ -102,14 +87,6 @@ py_test( env = {"VERSION_CHECK": "3.9"}, # The default defined in the WORKSPACE. ) -py_test( - name = "version_3_8_test", - srcs = ["version_test.py"], - env = {"VERSION_CHECK": "3.8"}, - main = "version_test.py", - python_version = "3.8", -) - py_test( name = "version_3_9_test", srcs = ["version_test.py"], @@ -169,16 +146,6 @@ sh_test( }, ) -sh_test( - name = "version_test_binary_3_8", - srcs = ["version_test.sh"], - data = [":version_3_8"], - env = { - "VERSION_CHECK": "3.8", - "VERSION_PY_BINARY": "$(rootpaths :version_3_8)", - }, -) - sh_test( name = "version_test_binary_3_9", srcs = ["version_test.sh"], diff --git a/python/versions.bzl b/python/versions.bzl index 57a960c6a9..6343ee49c8 100644 --- a/python/versions.bzl +++ b/python/versions.bzl @@ -47,91 +47,6 @@ DEFAULT_RELEASE_BASE_URL = "https://github.com/astral-sh/python-build-standalone # # buildifier: disable=unsorted-dict-items TOOL_VERSIONS = { - "3.8.10": { - "url": "20210506/cpython-{python_version}-{platform}-pgo+lto-20210506T0943.tar.zst", - "sha256": { - "x86_64-apple-darwin": "8d06bec08db8cdd0f64f4f05ee892cf2fcbc58cfb1dd69da2caab78fac420238", - "x86_64-unknown-linux-gnu": "aec8c4c53373b90be7e2131093caa26063be6d9d826f599c935c0e1042af3355", - }, - "strip_prefix": "python/install", - }, - "3.8.12": { - "url": "20220227/cpython-{python_version}+20220227-{platform}-{build}.tar.gz", - "sha256": { - "aarch64-apple-darwin": "f9a3cbb81e0463d6615125964762d133387d561b226a30199f5b039b20f1d944", - # no aarch64-unknown-linux-gnu build available for 3.8.12 - "x86_64-apple-darwin": "f323fbc558035c13a85ce2267d0fad9e89282268ecb810e364fff1d0a079d525", - "x86_64-pc-windows-msvc": "4658e08a00d60b1e01559b74d58ff4dd04da6df935d55f6268a15d6d0a679d74", - "x86_64-unknown-linux-gnu": "5be9c6d61e238b90dfd94755051c0d3a2d8023ebffdb4b0fa4e8fedd09a6cab6", - }, - "strip_prefix": "python", - }, - "3.8.13": { - "url": "20220802/cpython-{python_version}+20220802-{platform}-{build}.tar.gz", - "sha256": { - "aarch64-apple-darwin": "ae4131253d890b013171cb5f7b03cadc585ae263719506f7b7e063a7cf6fde76", - # no aarch64-unknown-linux-gnu build available for 3.8.13 - "x86_64-apple-darwin": "cd6e7c0a27daf7df00f6882eaba01490dd963f698e99aeee9706877333e0df69", - "x86_64-pc-windows-msvc": "f20643f1b3e263a56287319aea5c3888530c09ad9de3a5629b1a5d207807e6b9", - "x86_64-unknown-linux-gnu": "fb566629ccb5f76ef56d275a3f8017d683f1c20c5beb5d5f38b155ed11e16187", - }, - "strip_prefix": "python", - }, - "3.8.15": { - "url": "20221106/cpython-{python_version}+20221106-{platform}-{build}.tar.gz", - "sha256": { - "aarch64-apple-darwin": "1e0a92d1a4f5e6d4a99f86b1cbf9773d703fe7fd032590f3e9c285c7a5eeb00a", - "aarch64-unknown-linux-gnu": "886ab33ced13c84bf59ce8ff79eba6448365bfcafea1bf415bd1d75e21b690aa", - "x86_64-apple-darwin": "70b57f28c2b5e1e3dd89f0d30edd5bc414e8b20195766cf328e1b26bed7890e1", - "x86_64-pc-windows-msvc": "2fdc3fa1c95f982179bbbaedae2b328197658638799b6dcb63f9f494b0de59e2", - "x86_64-unknown-linux-gnu": "e47edfb2ceaf43fc699e20c179ec428b6f3e497cf8e2dcd8e9c936d4b96b1e56", - }, - "strip_prefix": "python", - }, - "3.8.16": { - "url": "20230116/cpython-{python_version}+20230116-{platform}-{build}.tar.gz", - "sha256": { - "aarch64-apple-darwin": "d1f408569d8807c1053939d7822b082a17545e363697e1ce3cfb1ee75834c7be", - "aarch64-unknown-linux-gnu": "15d00bc8400ed6d94c665a797dc8ed7a491ae25c5022e738dcd665cd29beec42", - "x86_64-apple-darwin": "484ba901f64fc7888bec5994eb49343dc3f9d00ed43df17ee9c40935aad4aa18", - "x86_64-pc-windows-msvc": "b446bec833eaba1bac9063bb9b4aeadfdf67fa81783b4487a90c56d408fb7994", - "x86_64-unknown-linux-gnu": "c890de112f1ae31283a31fefd2061d5c97bdd4d1bdd795552c7abddef2697ea1", - }, - "strip_prefix": "python", - }, - "3.8.17": { - "url": "20230826/cpython-{python_version}+20230826-{platform}-{build}.tar.gz", - "sha256": { - "aarch64-apple-darwin": "c6f7a130d0044a78e39648f4dae56dcff5a41eba91888a99f6e560507162e6a1", - "aarch64-unknown-linux-gnu": "9f6d585091fe26906ff1dbb80437a3fe37a1e3db34d6ecc0098f3d6a78356682", - "x86_64-apple-darwin": "155b06821607bae1a58ecc60a7d036b358c766f19e493b8876190765c883a5c2", - "x86_64-pc-windows-msvc": "6428e1b4e0b4482d390828de7d4c82815257443416cb786abe10cb2466ca68cd", - "x86_64-unknown-linux-gnu": "8d3e1826c0bb7821ec63288038644808a2d45553245af106c685ef5892fabcd8", - }, - "strip_prefix": "python", - }, - "3.8.18": { - "url": "20240224/cpython-{python_version}+20240224-{platform}-{build}.tar.gz", - "sha256": { - "aarch64-apple-darwin": "4d493a1792bf211f37f98404cc1468f09bd781adc2602dea0df82ad264c11abc", - "aarch64-unknown-linux-gnu": "6588c9eed93833d9483d01fe40ac8935f691a1af8e583d404ec7666631b52487", - "x86_64-apple-darwin": "7d2cd8d289d5e3cdd0a8c06c028c7c621d3d00ce44b7e2f08c1724ae0471c626", - "x86_64-pc-windows-msvc": "dba923ee5df8f99db04f599e826be92880746c02247c8d8e4d955d4bc711af11", - "x86_64-unknown-linux-gnu": "5ae36825492372554c02708bdd26b8dcd57e3dbf34b3d6d599ad91d93540b2b7", - }, - "strip_prefix": "python", - }, - "3.8.19": { - "url": "20240726/cpython-{python_version}+20240726-{platform}-{build}.tar.gz", - "sha256": { - "aarch64-apple-darwin": "fe4af1b6bc59478d027ede43f6249cf7b9143558e171bdf8711247337623af57", - "aarch64-unknown-linux-gnu": "8dc598aca7ad43ea20119324af98862d198d8990151c734a69f0fc9d16384b46", - "x86_64-apple-darwin": "4bc990b35384c83b5b0b3071e91455ec203517e569f29f691b159f1a6b2a19b2", - "x86_64-pc-windows-msvc": "4e8e9ddda82062d6e111108ab72f439acac4ba41b77d694548ef5dbf6b2b3319", - "x86_64-unknown-linux-gnu": "e81ea4dd16e6057c8121bdbcb7b64e2956068ca019f244c814bc3ad907cb2765", - }, - "strip_prefix": "python", - }, "3.8.20": { "url": "20241002/cpython-{python_version}+20241002-{platform}-{build}.tar.gz", "sha256": { From 09145b9f628d482246eaa70421bf0cbae9acb096 Mon Sep 17 00:00:00 2001 From: Ignas Anikevicius <240938+aignas@users.noreply.github.com> Date: Thu, 27 Mar 2025 23:32:39 +0900 Subject: [PATCH 126/922] feat: uv lock rule instead of genrule (#2657) This change re-implements the `uv pip compile` as a set of rules instead of using a `genrule`. This makes the setup more RBE friendly and it also fixes some of existing issues in the exec tools toolchain. The `lock` macro in the `//python/uv:lock.bzl` now creates three public targets: ``, `.update` and `.run`. The first will provide you with the locked `requirements.txt` file that is used in the `.update` executable target when updating the in-source copy of the file. The `.run` provides an executable target that hardcodes all of the `uv` args from the `` rule in a shell script and allows user to debug the execution and add extra arguments at the command line. The `test` target is no longer included, but users can define it themselves with the help of `native_test`. Things that I could not test and would benefit from the community help: * Windows support - the repository has a rudimentary script, but I am almost sure that it is likely not working, so PRs there are welcome. * The integration tests are not running on RBE because of the current RBE cluster setup. If you see issues in your RBE setup, PRs are welcome. * `keyring` integration to pull packages from private index servers is untested as of now, but I see no reason why it should not work. Work towards #1325 Work towards #1975 Related #2663 --- CHANGELOG.md | 12 + docs/BUILD.bazel | 8 +- examples/BUILD.bazel | 5 + examples/bzlmod/requirements_lock_3_9.txt | 8 +- private/BUILD.bazel | 2 + python/private/BUILD.bazel | 1 + python/private/py_exec_tools_info.bzl | 19 +- python/private/py_exec_tools_toolchain.bzl | 24 +- python/private/sentinel.bzl | 6 +- python/uv/lock.bzl | 28 +- python/uv/private/BUILD.bazel | 25 +- python/uv/private/lock.bat | 7 + python/uv/private/lock.bzl | 531 +++++++++++++++--- python/uv/private/lock.sh | 9 + python/uv/private/lock_copier.py | 69 +++ python/uv/private/uv_toolchain.bzl | 2 +- tests/uv/lock/BUILD.bazel | 5 + tests/uv/lock/lock_run_test.py | 165 ++++++ tests/uv/lock/lock_tests.bzl | 105 ++++ tests/uv/lock/testdata/build_constraints.txt | 1 + tests/uv/lock/testdata/build_constraints2.txt | 1 + tests/uv/lock/testdata/constraints.txt | 1 + tests/uv/lock/testdata/constraints2.txt | 1 + tests/uv/lock/testdata/requirements.in | 1 + tests/uv/lock/testdata/requirements.txt | 128 +++++ tools/private/publish_deps.bzl | 22 +- tools/publish/BUILD.bazel | 5 +- 27 files changed, 1085 insertions(+), 106 deletions(-) create mode 100755 python/uv/private/lock.bat create mode 100755 python/uv/private/lock.sh create mode 100644 python/uv/private/lock_copier.py create mode 100644 tests/uv/lock/BUILD.bazel create mode 100644 tests/uv/lock/lock_run_test.py create mode 100644 tests/uv/lock/lock_tests.bzl create mode 100644 tests/uv/lock/testdata/build_constraints.txt create mode 100644 tests/uv/lock/testdata/build_constraints2.txt create mode 100644 tests/uv/lock/testdata/constraints.txt create mode 100644 tests/uv/lock/testdata/constraints2.txt create mode 100644 tests/uv/lock/testdata/requirements.in create mode 100644 tests/uv/lock/testdata/requirements.txt diff --git a/CHANGELOG.md b/CHANGELOG.md index c64241ccbf..80466fc3f9 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -53,6 +53,10 @@ Unreleased changes template. {#v0-0-0-changed} ### Changed +* (toolchain) The `exec` configuration toolchain now has the forwarded + `exec_interpreter` now also forwards the `ToolchainInfo` provider. This is + for increased compatibility with the `RBE` setups where access to the `exec` + configuration interpreter is needed. * (toolchains) Use the latest astrahl-sh toolchain release [20250317] for Python versions: * 3.9.21 * 3.10.16 @@ -75,6 +79,14 @@ Unreleased changes template. {#v0-0-0-added} ### Added +* (uv) A {obj}`lock` rule that is the replacement for the + {obj}`compile_pip_requirements`. This may still have rough corners + so please report issues with it in the + [#1975](https://github.com/bazel-contrib/rules_python/issues/1975). + Main highlights - the locking can be done within a build action or outside + it, there is no more automatic `test` target (but it can be added on the user + side by using `native_test`). For customizing the `uv` version that is used, + please check the {obj}`uv.configure` tag class. * Add support for riscv64 linux platform. * (toolchains) Add python 3.13.2 and 3.12.9 toolchains diff --git a/docs/BUILD.bazel b/docs/BUILD.bazel index ab996537c7..bebecd18b2 100644 --- a/docs/BUILD.bazel +++ b/docs/BUILD.bazel @@ -176,8 +176,12 @@ lock( name = "requirements", srcs = ["pyproject.toml"], out = "requirements.txt", - upgrade = True, - visibility = ["//private:__pkg__"], + args = [ + "--emit-index-url", + "--universal", + "--upgrade", + ], + visibility = ["//:__subpackages__"], ) # Temporary compatibility aliases for some other projects depending on the old diff --git a/examples/BUILD.bazel b/examples/BUILD.bazel index 92ca8e7199..d2fddc44c5 100644 --- a/examples/BUILD.bazel +++ b/examples/BUILD.bazel @@ -21,5 +21,10 @@ lock( name = "bzlmod_requirements_3_9", srcs = ["bzlmod/requirements.in"], out = "bzlmod/requirements_lock_3_9.txt", + args = [ + "--emit-index-url", + "--universal", + "--python-version=3.9", + ], python_version = "3.9.19", ) diff --git a/examples/bzlmod/requirements_lock_3_9.txt b/examples/bzlmod/requirements_lock_3_9.txt index d74d1d39b6..c48f406451 100644 --- a/examples/bzlmod/requirements_lock_3_9.txt +++ b/examples/bzlmod/requirements_lock_3_9.txt @@ -46,7 +46,7 @@ imagesize==1.4.1 \ --hash=sha256:0d8d18d08f840c19d0ee7ca1fd82490fdc3729b7ac93f49870406ddde8ef8d8b \ --hash=sha256:69150444affb9cb0d5cc5a92b3676f0b2fb7cd9ae39e947a5e11a36b4497cd4a # via sphinx -importlib-metadata==8.4.0 ; python_version < '3.10' \ +importlib-metadata==8.4.0 ; python_full_version < '3.10' \ --hash=sha256:66f342cc6ac9818fc6ff340576acd24d65ba0b3efabb2b4ac08b598965a4a2f1 \ --hash=sha256:9a547d3bc3608b025f93d403fdd1aae741c24fbb8314df4b155675742ce303c5 # via sphinx @@ -316,7 +316,7 @@ tabulate==0.9.0 \ --hash=sha256:0095b12bf5966de529c0feb1fa08671671b3368eec77d7ef7ab114be2c068b3c \ --hash=sha256:024ca478df22e9340661486f85298cff5f6dcdba14f3813e8830015b9ed1948f # via -r examples/bzlmod/requirements.in -tomli==2.0.1 ; python_version < '3.11' \ +tomli==2.0.1 ; python_full_version < '3.11' \ --hash=sha256:939de3e7a6161af0c887ef91b7d41a53e7c5a1ca976325f429cb46ea9bc30ecc \ --hash=sha256:de526c12914f0c550d15924c62d72abc48d6fe7364aa87328337a31007fe8a4f # via pylint @@ -324,7 +324,7 @@ tomlkit==0.11.6 \ --hash=sha256:07de26b0d8cfc18f871aec595fda24d95b08fef89d147caa861939f37230bf4b \ --hash=sha256:71b952e5721688937fb02cf9d354dbcf0785066149d2855e44531ebdd2b65d73 # via pylint -typing-extensions==4.12.2 ; python_version < '3.10' \ +typing-extensions==4.12.2 ; python_full_version < '3.10' \ --hash=sha256:04e5ca0351e0f3f85c6853954072df659d0d13fac324d0072316b67d7794700d \ --hash=sha256:1a7ead55c7e559dd4dee8856e3a88b41225abfe1ce8df57b7c13915fe121ffb8 # via @@ -480,7 +480,7 @@ yamllint==1.28.0 \ --hash=sha256:89bb5b5ac33b1ade059743cf227de73daa34d5e5a474b06a5e17fc16583b0cf2 \ --hash=sha256:9e3d8ddd16d0583214c5fdffe806c9344086721f107435f68bad990e5a88826b # via -r examples/bzlmod/requirements.in -zipp==3.20.0 ; python_version < '3.10' \ +zipp==3.20.0 ; python_full_version < '3.10' \ --hash=sha256:0145e43d89664cfe1a2e533adc75adafed82fe2da404b4bbb6b026c0157bdb31 \ --hash=sha256:58da6168be89f0be59beb194da1250516fdaa062ccebd30127ac65d30045e10d # via importlib-metadata diff --git a/private/BUILD.bazel b/private/BUILD.bazel index 68fefe910f..ef5652b826 100644 --- a/private/BUILD.bazel +++ b/private/BUILD.bazel @@ -15,6 +15,7 @@ multirun( ] + [ "//docs:requirements.update", ], + tags = ["manual"], ) # NOTE: The requirements for the pip dependencies may sometimes break the build @@ -24,4 +25,5 @@ multirun( alias( name = "whl_library_requirements.update", actual = "//tools/private/update_deps:update_pip_deps", + tags = ["manual"], ) diff --git a/python/private/BUILD.bazel b/python/private/BUILD.bazel index 8b07fbd877..0f6668fa93 100644 --- a/python/private/BUILD.bazel +++ b/python/private/BUILD.bazel @@ -361,6 +361,7 @@ bzl_library( name = "py_exec_tools_toolchain_bzl", srcs = ["py_exec_tools_toolchain.bzl"], deps = [ + ":common_bzl", ":py_exec_tools_info_bzl", ":sentinel_bzl", ":toolchain_types_bzl", diff --git a/python/private/py_exec_tools_info.bzl b/python/private/py_exec_tools_info.bzl index b74f480fab..ad9a7b0c5e 100644 --- a/python/private/py_exec_tools_info.bzl +++ b/python/private/py_exec_tools_info.bzl @@ -24,15 +24,26 @@ When running it in an action, use `DefaultInfo.files_to_run` to ensure all its files are appropriately available. An exec interpreter may not be available, e.g. if all the exec tools are prebuilt binaries. -NOTE: this interpreter is really only for use when a build tool cannot use +:::{note} +this interpreter is really only for use when a build tool cannot use the Python toolchain itself. When possible, prefeer to define a `py_binary` instead and use it via a `cfg=exec` attribute; this makes it much easier to setup the runtime environment for the binary. See also: `py_interpreter_program` rule. +::: -NOTE: What interpreter is used depends on the toolchain constraints. Ensure -the proper target constraints are being applied when obtaining this from -the toolchain. +:::{note} +What interpreter is used depends on the toolchain constraints. Ensure the +proper target constraints are being applied when obtaining this from the +toolchain. +::: + +:::{warning} +This does not work correctly in case of RBE, please use exec_runtime instead. + +Once https://github.com/bazelbuild/bazel/issues/23620 is resolved this warning +may be removed. +::: """, "precompiler": """ :type: Target | None diff --git a/python/private/py_exec_tools_toolchain.bzl b/python/private/py_exec_tools_toolchain.bzl index edf9159759..ff30431ff4 100644 --- a/python/private/py_exec_tools_toolchain.bzl +++ b/python/private/py_exec_tools_toolchain.bzl @@ -29,13 +29,15 @@ def _py_exec_tools_toolchain_impl(ctx): if SentinelInfo in ctx.attr.exec_interpreter: exec_interpreter = None - return [platform_common.ToolchainInfo( - exec_tools = PyExecToolsInfo( - exec_interpreter = exec_interpreter, - precompiler = ctx.attr.precompiler, + return [ + platform_common.ToolchainInfo( + exec_tools = PyExecToolsInfo( + exec_interpreter = exec_interpreter, + precompiler = ctx.attr.precompiler, + ), + **extra_kwargs ), - **extra_kwargs - )] + ] py_exec_tools_toolchain = rule( implementation = _py_exec_tools_toolchain_impl, @@ -51,6 +53,11 @@ This provides `ToolchainInfo` with the following attributes: attrs = { "exec_interpreter": attr.label( default = "//python/private:current_interpreter_executable", + providers = [ + DefaultInfo, + # Add the toolchain provider so that we can forward provider fields. + platform_common.ToolchainInfo, + ], cfg = "exec", doc = """ An interpreter that is directly usable in the exec configuration @@ -69,6 +76,11 @@ handle all the necessary transitions and runtime setup to invoke a program. ::: See {obj}`PyExecToolsInfo.exec_interpreter` for further docs. + +:::{versionchanged} VERSION_NEXT_FEATURE +From now on the provided target also needs to provide `platform_common.ToolchainInfo` +so that the toolchain `py_runtime` field can be correctly forwarded. +::: """, ), "precompiler": attr.label( diff --git a/python/private/sentinel.bzl b/python/private/sentinel.bzl index 6d753e1983..8b69682b49 100644 --- a/python/private/sentinel.bzl +++ b/python/private/sentinel.bzl @@ -25,6 +25,10 @@ SentinelInfo = provider( def _sentinel_impl(ctx): _ = ctx # @unused - return [SentinelInfo()] + return [ + SentinelInfo(), + # Also output ToolchainInfo to allow it to be used for noop toolchains + platform_common.ToolchainInfo(), + ] sentinel = rule(implementation = _sentinel_impl) diff --git a/python/uv/lock.bzl b/python/uv/lock.bzl index edffe4728c..82b00bc2d2 100644 --- a/python/uv/lock.bzl +++ b/python/uv/lock.bzl @@ -14,7 +14,33 @@ """The `uv` locking rule. -EXPERIMENTAL: This is experimental and may be removed without notice +Differences with the legacy {obj}`compile_pip_requirements` rule: +- This is implemented as a rule that performs locking in a build action. +- Additionally one can use the runnable target. +- Uses `uv`. +- This does not error out if the output file does not exist yet. +- Supports transitions out of the box. + +Note, this does not provide a `test` target, if you would like to add a test +target that always does the locking automatically to ensure that the +`requirements.txt` file is up-to-date, add something similar to: + +```starlark +load("@bazel_skylib//rules:native_binary.bzl", "native_test") +load("@rules_python//python/uv:lock.bzl", "lock") + +lock( + name = "requirements", + srcs = ["pyproject.toml"], +) + +native_test( + name = "requirements_test", + src = "requirements.update", +) +``` + +EXPERIMENTAL: This is experimental and may be changed without notice. """ load("//python/uv/private:lock.bzl", _lock = "lock") diff --git a/python/uv/private/BUILD.bazel b/python/uv/private/BUILD.bazel index acf2a9c1f7..d17ca39490 100644 --- a/python/uv/private/BUILD.bazel +++ b/python/uv/private/BUILD.bazel @@ -13,6 +13,15 @@ # limitations under the License. load("@bazel_skylib//:bzl_library.bzl", "bzl_library") +load("//python/private:bzlmod_enabled.bzl", "BZLMOD_ENABLED") # buildifier: disable=bzl-visibility + +exports_files( + srcs = [ + "lock_copier.py", + ], + # only because this is used from a macro to template + visibility = ["//visibility:public"], +) filegroup( name = "distribution", @@ -31,9 +40,13 @@ bzl_library( srcs = ["lock.bzl"], visibility = ["//python/uv:__subpackages__"], deps = [ + ":toolchain_types_bzl", "//python:py_binary_bzl", "//python/private:bzlmod_enabled_bzl", - "@bazel_skylib//rules:write_file", + "//python/private:full_version_bzl", + "//python/private:toolchain_types_bzl", + "@bazel_skylib//lib:shell", + "@pythons_hub//:versions_bzl", ], ) @@ -81,3 +94,13 @@ bzl_library( "//python/private:text_util_bzl", ], ) + +filegroup( + name = "lock_template", + srcs = select({ + "@platforms//os:windows": ["lock.bat"], + "//conditions:default": ["lock.sh"], + }), + target_compatible_with = [] if BZLMOD_ENABLED else ["@platforms//:incompatible"], + visibility = ["//visibility:public"], +) diff --git a/python/uv/private/lock.bat b/python/uv/private/lock.bat new file mode 100755 index 0000000000..3954c10347 --- /dev/null +++ b/python/uv/private/lock.bat @@ -0,0 +1,7 @@ +if defined BUILD_WORKSPACE_DIRECTORY ( + set "out=%BUILD_WORKSPACE_DIRECTORY%\{{src_out}}" +) else ( + exit /b 1 +) + +"{{args}}" --output-file "%out%" %* diff --git a/python/uv/private/lock.bzl b/python/uv/private/lock.bzl index 9378f180db..69d277d653 100644 --- a/python/uv/private/lock.bzl +++ b/python/uv/private/lock.bzl @@ -12,114 +12,483 @@ # See the License for the specific language governing permissions and # limitations under the License. -"""A simple macro to lock the requirements. +"""An implementation for a simple macro to lock the requirements. """ -load("@bazel_skylib//rules:write_file.bzl", "write_file") +load("@bazel_skylib//lib:shell.bzl", "shell") +load("@pythons_hub//:versions.bzl", "DEFAULT_PYTHON_VERSION", "MINOR_MAPPING") load("//python:py_binary.bzl", "py_binary") load("//python/private:bzlmod_enabled.bzl", "BZLMOD_ENABLED") # buildifier: disable=bzl-visibility +load("//python/private:full_version.bzl", "full_version") +load("//python/private:toolchain_types.bzl", "EXEC_TOOLS_TOOLCHAIN_TYPE") # buildifier: disable=bzl-visibility +load(":toolchain_types.bzl", "UV_TOOLCHAIN_TYPE") visibility(["//..."]) -_REQUIREMENTS_TARGET_COMPATIBLE_WITH = select({ - "@platforms//os:windows": ["@platforms//:incompatible"], - "//conditions:default": [], -}) if BZLMOD_ENABLED else ["@platforms//:incompatible"] +_PYTHON_VERSION_FLAG = "//python/config_settings:python_version" -def lock(*, name, srcs, out, upgrade = False, universal = True, args = [], **kwargs): - """Pin the requirements based on the src files. +_RunLockInfo = provider( + doc = "", + fields = { + "args": "The args passed to the `uv` by default when running the runnable target.", + "env": "The env passed to the execution.", + "srcs": "Source files required to run the runnable target.", + }, +) + +def _args(ctx): + """A small helper to ensure that the right args are pushed to the _RunLockInfo provider""" + run_info = [] + args = ctx.actions.args() + + def _add_args(arg, maybe_value = None): + run_info.append(arg) + if maybe_value: + args.add(arg, maybe_value) + run_info.append(maybe_value) + else: + args.add(arg) + + def _add_all(name, all_args = None, **kwargs): + if not all_args and type(name) == "list": + all_args = name + name = None + + before_each = kwargs.get("before_each") + if name: + args.add_all(name, all_args, **kwargs) + run_info.append(name) + else: + args.add_all(all_args, **kwargs) + + for arg in all_args: + if before_each: + run_info.append(before_each) + run_info.append(arg) + + return struct( + run_info = run_info, + run_shell = args, + add = _add_args, + add_all = _add_all, + ) + +def _lock_impl(ctx): + srcs = ctx.files.srcs + python_version = full_version( + version = ctx.attr.python_version or DEFAULT_PYTHON_VERSION, + minor_mapping = MINOR_MAPPING, + ) + output = ctx.actions.declare_file("{}.{}.out".format( + ctx.label.name, + python_version.replace(".", "_"), + )) + + toolchain_info = ctx.toolchains[UV_TOOLCHAIN_TYPE] + uv = toolchain_info.uv_toolchain_info.uv[DefaultInfo].files_to_run.executable + + args = _args(ctx) + args.add_all([ + uv, + "pip", + "compile", + "--no-python-downloads", + "--no-cache", + ]) + pkg = ctx.label.package + update_target = ctx.attr.update_target + args.add("--custom-compile-command", "bazel run //{}:{}".format(pkg, update_target)) + if ctx.attr.generate_hashes: + args.add("--generate-hashes") + if not ctx.attr.strip_extras: + args.add("--no-strip-extras") + args.add_all(ctx.files.build_constraints, before_each = "--build-constraints") + args.add_all(ctx.files.constraints, before_each = "--constraints") + args.add_all(ctx.attr.args) + + exec_tools = ctx.toolchains[EXEC_TOOLS_TOOLCHAIN_TYPE].exec_tools + runtime = exec_tools.exec_interpreter[platform_common.ToolchainInfo].py3_runtime + python = runtime.interpreter or runtime.interpreter_path + python_files = runtime.files + args.add("--python", python) + args.add_all(srcs) + + args.run_shell.add("--output-file", output) + + # These arguments does not change behaviour, but it reduces the output from + # the command, which is especially verbose in stderr. + args.run_shell.add("--no-progress") + args.run_shell.add("--quiet") - Differences with the current {obj}`compile_pip_requirements` rule: - - This is implemented in shell and `uv`. - - This does not error out if the output file does not exist yet. - - Supports transitions out of the box. - - The execution of the lock file generation is happening inside of a build - action in a `genrule`. + if ctx.files.existing_output: + command = '{python} -c {python_cmd} && "$@"'.format( + python = getattr(python, "path", python), + python_cmd = shell.quote( + "from shutil import copy; copy(\"{src}\", \"{dst}\")".format( + src = ctx.files.existing_output[0].path, + dst = output.path, + ), + ), + ) + else: + command = '"$@"' + + srcs = srcs + ctx.files.build_constraints + ctx.files.constraints + + ctx.actions.run_shell( + command = command, + inputs = srcs + ctx.files.existing_output, + mnemonic = "PyRequirementsLockUv", + outputs = [output], + arguments = [args.run_shell], + tools = [ + uv, + python_files, + ], + progress_message = "Creating a requirements.txt with uv: %{label}", + env = ctx.attr.env, + ) + + return [ + DefaultInfo(files = depset([output])), + _RunLockInfo( + args = args.run_info, + env = ctx.attr.env, + srcs = depset( + srcs + [uv], + transitive = [python_files], + ), + ), + ] + +def _transition_impl(input_settings, attr): + settings = { + _PYTHON_VERSION_FLAG: input_settings[_PYTHON_VERSION_FLAG], + } + if attr.python_version: + # FIXME @aignas 2025-03-20: using `full_version` is a workaround for a bug in + # how we order toolchains in bazel. If I set the `python_version` flag + # to `3.12`, I would expect the latest version to be selected, i.e. the + # one that is in MINOR_MAPPING, but it seems that 3.12.0 is selected, + # because of how the targets are ordered. + settings[_PYTHON_VERSION_FLAG] = full_version( + version = attr.python_version, + minor_mapping = MINOR_MAPPING, + ) + return settings + +_python_version_transition = transition( + implementation = _transition_impl, + inputs = [_PYTHON_VERSION_FLAG], + outputs = [_PYTHON_VERSION_FLAG], +) + +_lock = rule( + implementation = _lock_impl, + doc = """\ +The lock rule that does the locking in a build action (that makes it possible +to use RBE) and also prepares information for a `bazel run` executable rule. +""", + attrs = { + "args": attr.string_list( + doc = "Public, see the docs in the macro.", + ), + "build_constraints": attr.label_list( + allow_files = True, + doc = "Public, see the docs in the macro.", + ), + "constraints": attr.label_list( + allow_files = True, + doc = "Public, see the docs in the macro.", + ), + "env": attr.string_dict( + doc = "Public, see the docs in the macro.", + ), + "existing_output": attr.label( + mandatory = False, + allow_single_file = True, + doc = """\ +An already existing output file that is used as a basis for further +modifications and the locking is not done from scratch. +""", + ), + "generate_hashes": attr.bool( + doc = "Public, see the docs in the macro.", + default = True, + ), + "output": attr.string( + doc = "Public, see the docs in the macro.", + mandatory = True, + ), + "python_version": attr.string( + doc = "Public, see the docs in the macro.", + ), + "srcs": attr.label_list( + mandatory = True, + allow_files = True, + doc = "Public, see the docs in the macro.", + ), + "strip_extras": attr.bool( + doc = "Public, see the docs in the macro.", + default = False, + ), + "update_target": attr.string( + mandatory = True, + doc = """\ +The string to input for the 'uv pip compile'. +""", + ), + "_allowlist_function_transition": attr.label( + default = "@bazel_tools//tools/allowlists/function_transition_allowlist", + ), + }, + toolchains = [ + EXEC_TOOLS_TOOLCHAIN_TYPE, + UV_TOOLCHAIN_TYPE, + ], + cfg = _python_version_transition, +) + +def _lock_run_impl(ctx): + if ctx.attr.is_windows: + path_sep = "\\" + ext = ".exe" + else: + path_sep = "/" + ext = "" + + def _maybe_path(arg): + if hasattr(arg, "short_path"): + arg = arg.short_path + + return shell.quote(arg.replace("/", path_sep)) + + info = ctx.attr.lock[_RunLockInfo] + executable = ctx.actions.declare_file(ctx.label.name + ext) + ctx.actions.expand_template( + template = ctx.files._template[0], + substitutions = { + '"{{args}}"': " ".join([_maybe_path(arg) for arg in info.args]), + "{{src_out}}": "{}/{}".format(ctx.label.package, ctx.attr.output).replace( + "/", + path_sep, + ), + }, + output = executable, + is_executable = True, + ) + + return [ + DefaultInfo( + executable = executable, + runfiles = ctx.runfiles(transitive_files = info.srcs), + ), + RunEnvironmentInfo( + environment = info.env, + ), + ] + +_lock_run = rule( + implementation = _lock_run_impl, + doc = """\ +""", + attrs = { + "is_windows": attr.bool(mandatory = True), + "lock": attr.label( + doc = "The lock target that is doing locking in a build action.", + providers = [_RunLockInfo], + cfg = "exec", + ), + "output": attr.string( + doc = """\ +The output that we would be updated, relative to the package the macro is used in. +""", + ), + "_template": attr.label( + default = "//python/uv/private:lock_template", + doc = """\ +The template to be used for 'uv pip compile'. This is either .ps1 or bash +script depending on what the target platform is executed on. +""", + ), + }, + executable = True, +) + +def _maybe_file(path): + """A small function to return a list of existing outputs. + + If the file referenced by the input argument exists, then it will return + it, otherwise it will return an empty list. This is useful to for programs + like pip-compile which behave differently if the output file exists and + update the output file in place. + + The API of the function ensures that path is not a glob itself. Args: - name: The name of the target to run for updating the requirements. - srcs: The srcs to use as inputs. - out: The output file. - upgrade: Tell `uv` to always upgrade the dependencies instead of - keeping them as they are. - universal: Tell `uv` to generate a universal lock file. - args: Extra args to pass to the rule. - **kwargs: Extra kwargs passed to the binary rule. + path: {type}`str` the file name. """ - pkg = native.package_name() - update_target = name + ".update" - - _args = [ - "--custom-compile-command='bazel run //{}:{}'".format(pkg, update_target), - "--generate-hashes", - "--emit-index-url", - "--no-strip-extras", - "--python=$(PYTHON3)", - ] + args + [ - "$(location {})".format(src) - for src in srcs - ] - if upgrade: - _args.append("--upgrade") - if universal: - _args.append("--universal") - _args.append("--output-file=$@") - cmd = "$(UV_BIN) pip compile " + " ".join(_args) + for p in native.glob([path], allow_empty = True): + if path == p: + return p + + return None + +def _expand_template_impl(ctx): + pkg = ctx.label.package + update_src = ctx.actions.declare_file(ctx.attr.update_target + ".py") + ctx.actions.expand_template( + template = ctx.files._template[0], + substitutions = { + "{{dst}}": "{}/{}".format(pkg, ctx.attr.output), + "{{src}}": "{}".format(ctx.files.src[0].short_path), + "{{update_target}}": "//{}:{}".format(pkg, ctx.attr.update_target), + }, + output = update_src, + ) + return DefaultInfo(files = depset([update_src])) + +_expand_template = rule( + implementation = _expand_template_impl, + attrs = { + "output": attr.string(mandatory = True), + "src": attr.label(mandatory = True), + "update_target": attr.string(mandatory = True), + "_template": attr.label( + default = "//python/uv/private:lock_copier.py", + allow_single_file = True, + ), + }, + doc = "Expand the template for the update script allowing us to use `select` statements in the {attr}`output` attribute.", +) + +def lock( + *, + name, + srcs, + out, + args = [], + build_constraints = [], + constraints = [], + env = None, + generate_hashes = True, + python_version = None, + strip_extras = False, + **kwargs): + """Pin the requirements based on the src files. + + This macro creates the following targets: + - `name`: the target that creates the requirements.txt file in a build + action. This target will have `no-cache` and `requires-network` added + to its tags. + - `name.run`: a runnable target that can be used to pass extra parameters + to the same command that would be run in the `name` action. This will + update the source copy of the requirements file. You can customize the + args via the command line, but it requires being able to run `uv` (and + possibly `python`) directly on your host. + - `name.update`: a target that can be run to update the source-tree version + of the requirements lock file. The output can be fed to the + {obj}`pip.parse` bzlmod extension tag class. Note, you can use + `native_test` to wrap this target to make a test. You can't customize the + args via command line, but you can use RBE to generate requirements + (offload execution and run for different platforms). Note, that for RBE + to be usable, one needs to ensure that the nodes running the action have + internet connectivity or the indexes are provided in a different way for + a fully offline operation. - # Make a copy to ensure that we are not modifying the initial list - srcs = list(srcs) + :::{note} + All of the targets have `manual` tags as locking results cannot be cached. + ::: + + Args: + name: {type}`str` The prefix of all targets created by this macro. + srcs: {type}`list[Label]` The sources that will be used. Add all of the + files that would be passed as srcs to the `uv pip compile` command. + out: {type}`str` The output file relative to the package. + args: {type}`list[str]` The list of args to pass to uv. Note, these are + written into the runnable `name.run` target. + env: {type}`dict[str, str]` the environment variables to set. Note, this + is passed as is and the environment variables are not expanded. + build_constraints: {type}`list[Label]` The list of build constraints to use. + constraints: {type}`list[Label]` The list of constraints files to use. + generate_hashes: {type}`bool` Generate hashes for all of the + requirements. This is a must if you want to use + {attr}`pip.parse.experimental_index_url`. Defaults to `True`. + strip_extras: {type}`bool` whether to strip extras from the output. + Currently `rules_python` requires `--no-strip-extras` to properly + function, but sometimes one may want to not have the extras if you + are compiling the requirements file for using it as a constraints + file. Defaults to `False`. + python_version: {type}`str | None` the python_version to transition to + when locking the requirements. Defaults to the default python version + configured by the {obj}`python` module extension. + **kwargs: common kwargs passed to rules. + """ + update_target = "{}.update".format(name) + locker_target = "{}.run".format(name) # Check if the output file already exists, if yes, first copy it to the # output file location in order to make `uv` not change the requirements if # we are just running the command. - if native.glob([out]): - cmd = "cp -v $(location {}) $@; {}".format(out, cmd) - srcs.append(out) + maybe_out = _maybe_file(out) + + tags = ["manual"] + kwargs.pop("tags", []) + if not BZLMOD_ENABLED: + kwargs["target_compatible_with"] = ["@platforms//:incompatible"] - native.genrule( + # FIXME @aignas 2025-03-17: should we have one more target that transitions + # the python_version to ensure that if somebody calls `bazel build + # :requirements` that it is locked with the right `python_version`? + _lock( name = name, + args = args, + build_constraints = build_constraints, + constraints = constraints, + env = env, + existing_output = maybe_out, + generate_hashes = generate_hashes, + python_version = python_version, srcs = srcs, - outs = [out + ".new"], - cmd_bash = cmd, + strip_extras = strip_extras, + update_target = update_target, + output = out, tags = [ - "local", - "manual", "no-cache", - ], - target_compatible_with = _REQUIREMENTS_TARGET_COMPATIBLE_WITH, - toolchains = [ - Label("//python/uv:current_toolchain"), - Label("//python:current_py_toolchain"), - ], + "requires-network", + ] + tags, + **kwargs ) - # Write a script that can be used for updating the in-tree version of the - # requirements file - write_file( - name = name + ".update_gen", - out = update_target + ".py", - content = [ - "from os import environ", - "from pathlib import Path", - "from sys import stderr", - "", - 'src = Path(environ["REQUIREMENTS_FILE"])', - 'assert src.exists(), f"the {src} file does not exist"', - 'dst = Path(environ["BUILD_WORKSPACE_DIRECTORY"]) / "{}" / "{}"'.format(pkg, out), - 'print(f"Writing requirements contents\\n from {src.absolute()}\\n to {dst.absolute()}", file=stderr)', - "dst.write_text(src.read_text())", - 'print("Success!", file=stderr)', - ], + # A target for updating the in-tree version directly by skipping the in-action + # uv pip compile. + _lock_run( + name = locker_target, + lock = name, + output = out, + is_windows = select({ + "@platforms//os:windows": True, + "//conditions:default": False, + }), + tags = tags, + **kwargs + ) + + # FIXME @aignas 2025-03-20: is it possible to extend `py_binary` so that the + # srcs are generated before `py_binary` is run? I found that + # `ctx.files.srcs` usage in the base implementation is making it difficult. + template_target = "_{}_gen".format(name) + _expand_template( + name = template_target, + src = name, + output = out, + update_target = update_target, + tags = tags, ) py_binary( name = update_target, - srcs = [update_target + ".py"], - main = update_target + ".py", - data = [name], - env = { - "REQUIREMENTS_FILE": "$(rootpath {})".format(name), - }, - tags = ["manual"], + srcs = [template_target], + data = [name] + ([maybe_out] if maybe_out else []), + tags = tags, **kwargs ) diff --git a/python/uv/private/lock.sh b/python/uv/private/lock.sh new file mode 100755 index 0000000000..b6ba0c6c48 --- /dev/null +++ b/python/uv/private/lock.sh @@ -0,0 +1,9 @@ +#!/bin/bash +set -euo pipefail + +if [[ -n "${BUILD_WORKSPACE_DIRECTORY:-}" ]]; then + readonly out="${BUILD_WORKSPACE_DIRECTORY}/{{src_out}}" +else + exit 1 +fi +exec "{{args}}" --output-file "$out" "$@" diff --git a/python/uv/private/lock_copier.py b/python/uv/private/lock_copier.py new file mode 100644 index 0000000000..bcc64c1661 --- /dev/null +++ b/python/uv/private/lock_copier.py @@ -0,0 +1,69 @@ +import sys +from difflib import unified_diff +from os import environ +from pathlib import Path + +_LINE = "=" * 80 + + +def main(): + src = "{{src}}" + dst = "{{dst}}" + + src = Path(src) + if not src.exists(): + raise AssertionError(f"The {src} file does not exist") + + if "TEST_SRCDIR" in environ: + # Running as a bazel test + dst = Path(dst) + a = dst.read_text() if dst.exists() else "\n" + b = src.read_text() + + diff = unified_diff( + a.splitlines(), + b.splitlines(), + str(dst), + str(src), + lineterm="", + ) + diff = "\n".join(list(diff)) + if not diff: + print( + f"""\ +{_LINE} +The in source file copy is up-to-date. +{_LINE} +""" + ) + return 0 + + print(diff) + print( + f"""\ +{_LINE} +The in source file copy is out of date, please run: + + bazel run {{update_target}} +{_LINE} +""" + ) + return 1 + + if "BUILD_WORKSPACE_DIRECTORY" not in environ: + raise RuntimeError( + "This must be either run as `bazel test` via a `native_test` or similar or via `bazel run`" + ) + + print(f"cp /{src} /{dst}") + build_workspace = Path(environ["BUILD_WORKSPACE_DIRECTORY"]) + + dst_real_path = build_workspace / dst + dst_real_path.parent.mkdir(parents=True, exist_ok=True) + dst_real_path.write_text(src.read_text()) + print(f"OK: updated {dst_real_path}") + return 0 + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/python/uv/private/uv_toolchain.bzl b/python/uv/private/uv_toolchain.bzl index b740fc304d..8c7f1b4b8c 100644 --- a/python/uv/private/uv_toolchain.bzl +++ b/python/uv/private/uv_toolchain.bzl @@ -53,7 +53,7 @@ uv_toolchain = rule( mandatory = True, allow_single_file = True, executable = True, - cfg = "target", + cfg = "exec", ), "version": attr.string(mandatory = True, doc = "Version of the uv binary."), }, diff --git a/tests/uv/lock/BUILD.bazel b/tests/uv/lock/BUILD.bazel new file mode 100644 index 0000000000..6b6902da44 --- /dev/null +++ b/tests/uv/lock/BUILD.bazel @@ -0,0 +1,5 @@ +load(":lock_tests.bzl", "lock_test_suite") + +lock_test_suite( + name = "lock_tests", +) diff --git a/tests/uv/lock/lock_run_test.py b/tests/uv/lock/lock_run_test.py new file mode 100644 index 0000000000..ef57f23d31 --- /dev/null +++ b/tests/uv/lock/lock_run_test.py @@ -0,0 +1,165 @@ +import subprocess +import sys +import tempfile +import unittest +from pathlib import Path + +from python import runfiles + +rfiles = runfiles.Create() + + +def _relative_rpath(path: str) -> Path: + p = (Path("_main") / "tests" / "uv" / "lock" / path).as_posix() + rpath = rfiles.Rlocation(p) + if not rpath: + raise ValueError(f"Could not find file: {p}") + + return Path(rpath) + + +class LockTests(unittest.TestCase): + def test_requirements_updating_for_the_first_time(self): + # Given + copier_path = _relative_rpath("requirements_new_file.update") + + # When + with tempfile.TemporaryDirectory() as dir: + workspace_dir = Path(dir) + want_path = workspace_dir / "tests" / "uv" / "lock" / "does_not_exist.txt" + + self.assertFalse( + want_path.exists(), "The path should not exist after the test" + ) + output = subprocess.run( + copier_path, + capture_output=True, + env={ + "BUILD_WORKSPACE_DIRECTORY": f"{workspace_dir}", + }, + ) + + # Then + self.assertEqual(0, output.returncode, output.stderr) + self.assertIn( + "cp /tests/uv/lock/requirements_new_file", + output.stdout.decode("utf-8"), + ) + self.assertTrue(want_path.exists(), "The path should exist after the test") + self.assertNotEqual(want_path.read_text(), "") + + def test_requirements_updating(self): + # Given + copier_path = _relative_rpath("requirements.update") + existing_file = _relative_rpath("testdata/requirements.txt") + want_text = existing_file.read_text() + + # When + with tempfile.TemporaryDirectory() as dir: + workspace_dir = Path(dir) + want_path = ( + workspace_dir + / "tests" + / "uv" + / "lock" + / "testdata" + / "requirements.txt" + ) + want_path.parent.mkdir(parents=True) + want_path.write_text( + want_text + "\n\n" + ) # Write something else to see that it is restored + + output = subprocess.run( + copier_path, + capture_output=True, + env={ + "BUILD_WORKSPACE_DIRECTORY": f"{workspace_dir}", + }, + ) + + # Then + self.assertEqual(0, output.returncode) + self.assertIn( + "cp /tests/uv/lock/requirements", + output.stdout.decode("utf-8"), + ) + self.assertEqual(want_path.read_text(), want_text) + + def test_requirements_run_on_the_first_time(self): + # Given + copier_path = _relative_rpath("requirements_new_file.run") + + # When + with tempfile.TemporaryDirectory() as dir: + workspace_dir = Path(dir) + want_path = workspace_dir / "tests" / "uv" / "lock" / "does_not_exist.txt" + # NOTE @aignas 2025-03-18: right now we require users to have the folder + # there already + want_path.parent.mkdir(parents=True) + + self.assertFalse( + want_path.exists(), "The path should not exist after the test" + ) + output = subprocess.run( + copier_path, + capture_output=True, + env={ + "BUILD_WORKSPACE_DIRECTORY": f"{workspace_dir}", + }, + ) + + # Then + self.assertEqual(0, output.returncode, output.stderr) + self.assertTrue(want_path.exists(), "The path should exist after the test") + got_contents = want_path.read_text() + self.assertNotEqual(got_contents, "") + self.assertIn( + got_contents, + output.stdout.decode("utf-8"), + ) + + def test_requirements_run(self): + # Given + copier_path = _relative_rpath("requirements.run") + existing_file = _relative_rpath("testdata/requirements.txt") + want_text = existing_file.read_text() + + # When + with tempfile.TemporaryDirectory() as dir: + workspace_dir = Path(dir) + want_path = ( + workspace_dir + / "tests" + / "uv" + / "lock" + / "testdata" + / "requirements.txt" + ) + + want_path.parent.mkdir(parents=True) + want_path.write_text( + want_text + "\n\n" + ) # Write something else to see that it is restored + + output = subprocess.run( + copier_path, + capture_output=True, + env={ + "BUILD_WORKSPACE_DIRECTORY": f"{workspace_dir}", + }, + ) + + # Then + self.assertEqual(0, output.returncode, output.stderr) + self.assertTrue(want_path.exists(), "The path should exist after the test") + got_contents = want_path.read_text() + self.assertNotEqual(got_contents, "") + self.assertIn( + got_contents, + output.stdout.decode("utf-8"), + ) + + +if __name__ == "__main__": + unittest.main() diff --git a/tests/uv/lock/lock_tests.bzl b/tests/uv/lock/lock_tests.bzl new file mode 100644 index 0000000000..35c7c19328 --- /dev/null +++ b/tests/uv/lock/lock_tests.bzl @@ -0,0 +1,105 @@ +# Copyright 2025 The Bazel Authors. All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +"" + +load("@bazel_skylib//rules:native_binary.bzl", "native_test") +load("//python/uv:lock.bzl", "lock") +load("//tests/support:sh_py_run_test.bzl", "py_reconfig_test") + +def lock_test_suite(name): + """The test suite with various lock-related integration tests + + Args: + name: {type}`str` the name of the test suite + """ + lock( + name = "requirements", + srcs = ["testdata/requirements.in"], + constraints = [ + "testdata/constraints.txt", + "testdata/constraints2.txt", + ], + build_constraints = [ + "testdata/build_constraints.txt", + "testdata/build_constraints2.txt", + ], + # It seems that the CI remote executors for the RBE do not have network + # connectivity due to current CI setup. + tags = ["no-remote-exec"], + out = "testdata/requirements.txt", + ) + + lock( + name = "requirements_new_file", + srcs = ["testdata/requirements.in"], + out = "does_not_exist.txt", + # It seems that the CI remote executors for the RBE do not have network + # connectivity due to current CI setup. + tags = ["no-remote-exec"], + ) + + py_reconfig_test( + name = "requirements_run_tests", + env = { + "BUILD_WORKSPACE_DIRECTORY": "foo", + }, + srcs = ["lock_run_test.py"], + deps = [ + "//python/runfiles", + ], + data = [ + "requirements_new_file.update", + "requirements_new_file.run", + "requirements.update", + "requirements.run", + "testdata/requirements.txt", + ], + main = "lock_run_test.py", + tags = [ + "requires-network", + # FIXME @aignas 2025-03-19: it seems that the RBE tests are failing + # to execute the `requirements.run` targets that require network. + # + # We could potentially dump the required `.html` files and somehow + # provide it to the `uv`, but may rely on internal uv handling of + # `--index-url`. + "no-remote-exec", + ], + # FIXME @aignas 2025-03-19: It seems that currently: + # 1. The Windows runners are not compatible with the `uv` Windows binaries. + # 2. The Python launcher is having trouble launching scripts from within the Python test. + target_compatible_with = select({ + "@platforms//os:windows": ["@platforms//:incompatible"], + "//conditions:default": [], + }), + ) + + # document and check that this actually works + native_test( + name = "requirements_test", + src = ":requirements.update", + target_compatible_with = select({ + "@platforms//os:windows": ["@platforms//:incompatible"], + "//conditions:default": [], + }), + ) + + native.test_suite( + name = name, + tests = [ + ":requirements_test", + ":requirements_run_tests", + ], + ) diff --git a/tests/uv/lock/testdata/build_constraints.txt b/tests/uv/lock/testdata/build_constraints.txt new file mode 100644 index 0000000000..34c3ebe3de --- /dev/null +++ b/tests/uv/lock/testdata/build_constraints.txt @@ -0,0 +1 @@ +certifi==2025.1.31 diff --git a/tests/uv/lock/testdata/build_constraints2.txt b/tests/uv/lock/testdata/build_constraints2.txt new file mode 100644 index 0000000000..34c3ebe3de --- /dev/null +++ b/tests/uv/lock/testdata/build_constraints2.txt @@ -0,0 +1 @@ +certifi==2025.1.31 diff --git a/tests/uv/lock/testdata/constraints.txt b/tests/uv/lock/testdata/constraints.txt new file mode 100644 index 0000000000..18ade2c5b9 --- /dev/null +++ b/tests/uv/lock/testdata/constraints.txt @@ -0,0 +1 @@ +charset-normalizer==3.4.0 diff --git a/tests/uv/lock/testdata/constraints2.txt b/tests/uv/lock/testdata/constraints2.txt new file mode 100644 index 0000000000..18ade2c5b9 --- /dev/null +++ b/tests/uv/lock/testdata/constraints2.txt @@ -0,0 +1 @@ +charset-normalizer==3.4.0 diff --git a/tests/uv/lock/testdata/requirements.in b/tests/uv/lock/testdata/requirements.in new file mode 100644 index 0000000000..f2293605cf --- /dev/null +++ b/tests/uv/lock/testdata/requirements.in @@ -0,0 +1 @@ +requests diff --git a/tests/uv/lock/testdata/requirements.txt b/tests/uv/lock/testdata/requirements.txt new file mode 100644 index 0000000000..d02844636d --- /dev/null +++ b/tests/uv/lock/testdata/requirements.txt @@ -0,0 +1,128 @@ +# This file was autogenerated by uv via the following command: +# bazel run //tests/uv/lock:requirements.update +certifi==2025.1.31 \ + --hash=sha256:3d5da6925056f6f18f119200434a4780a94263f10d1c21d032a6f6b2baa20651 \ + --hash=sha256:ca78db4565a652026a4db2bcdf68f2fb589ea80d0be70e03929ed730746b84fe + # via requests +charset-normalizer==3.4.0 \ + --hash=sha256:0099d79bdfcf5c1f0c2c72f91516702ebf8b0b8ddd8905f97a8aecf49712c621 \ + --hash=sha256:0713f3adb9d03d49d365b70b84775d0a0d18e4ab08d12bc46baa6132ba78aaf6 \ + --hash=sha256:07afec21bbbbf8a5cc3651aa96b980afe2526e7f048fdfb7f1014d84acc8b6d8 \ + --hash=sha256:0b309d1747110feb25d7ed6b01afdec269c647d382c857ef4663bbe6ad95a912 \ + --hash=sha256:0d99dd8ff461990f12d6e42c7347fd9ab2532fb70e9621ba520f9e8637161d7c \ + --hash=sha256:0de7b687289d3c1b3e8660d0741874abe7888100efe14bd0f9fd7141bcbda92b \ + --hash=sha256:1110e22af8ca26b90bd6364fe4c763329b0ebf1ee213ba32b68c73de5752323d \ + --hash=sha256:130272c698667a982a5d0e626851ceff662565379baf0ff2cc58067b81d4f11d \ + --hash=sha256:136815f06a3ae311fae551c3df1f998a1ebd01ddd424aa5603a4336997629e95 \ + --hash=sha256:14215b71a762336254351b00ec720a8e85cada43b987da5a042e4ce3e82bd68e \ + --hash=sha256:1db4e7fefefd0f548d73e2e2e041f9df5c59e178b4c72fbac4cc6f535cfb1565 \ + --hash=sha256:1ffd9493de4c922f2a38c2bf62b831dcec90ac673ed1ca182fe11b4d8e9f2a64 \ + --hash=sha256:2006769bd1640bdf4d5641c69a3d63b71b81445473cac5ded39740a226fa88ab \ + --hash=sha256:20587d20f557fe189b7947d8e7ec5afa110ccf72a3128d61a2a387c3313f46be \ + --hash=sha256:223217c3d4f82c3ac5e29032b3f1c2eb0fb591b72161f86d93f5719079dae93e \ + --hash=sha256:27623ba66c183eca01bf9ff833875b459cad267aeeb044477fedac35e19ba907 \ + --hash=sha256:285e96d9d53422efc0d7a17c60e59f37fbf3dfa942073f666db4ac71e8d726d0 \ + --hash=sha256:2de62e8801ddfff069cd5c504ce3bc9672b23266597d4e4f50eda28846c322f2 \ + --hash=sha256:2f6c34da58ea9c1a9515621f4d9ac379871a8f21168ba1b5e09d74250de5ad62 \ + --hash=sha256:309a7de0a0ff3040acaebb35ec45d18db4b28232f21998851cfa709eeff49d62 \ + --hash=sha256:35c404d74c2926d0287fbd63ed5d27eb911eb9e4a3bb2c6d294f3cfd4a9e0c23 \ + --hash=sha256:3710a9751938947e6327ea9f3ea6332a09bf0ba0c09cae9cb1f250bd1f1549bc \ + --hash=sha256:3d59d125ffbd6d552765510e3f31ed75ebac2c7470c7274195b9161a32350284 \ + --hash=sha256:40d3ff7fc90b98c637bda91c89d51264a3dcf210cade3a2c6f838c7268d7a4ca \ + --hash=sha256:425c5f215d0eecee9a56cdb703203dda90423247421bf0d67125add85d0c4455 \ + --hash=sha256:43193c5cda5d612f247172016c4bb71251c784d7a4d9314677186a838ad34858 \ + --hash=sha256:44aeb140295a2f0659e113b31cfe92c9061622cadbc9e2a2f7b8ef6b1e29ef4b \ + --hash=sha256:47334db71978b23ebcf3c0f9f5ee98b8d65992b65c9c4f2d34c2eaf5bcaf0594 \ + --hash=sha256:4796efc4faf6b53a18e3d46343535caed491776a22af773f366534056c4e1fbc \ + --hash=sha256:4a51b48f42d9358460b78725283f04bddaf44a9358197b889657deba38f329db \ + --hash=sha256:4b67fdab07fdd3c10bb21edab3cbfe8cf5696f453afce75d815d9d7223fbe88b \ + --hash=sha256:4ec9dd88a5b71abfc74e9df5ebe7921c35cbb3b641181a531ca65cdb5e8e4dea \ + --hash=sha256:4f9fc98dad6c2eaa32fc3af1417d95b5e3d08aff968df0cd320066def971f9a6 \ + --hash=sha256:54b6a92d009cbe2fb11054ba694bc9e284dad30a26757b1e372a1fdddaf21920 \ + --hash=sha256:55f56e2ebd4e3bc50442fbc0888c9d8c94e4e06a933804e2af3e89e2f9c1c749 \ + --hash=sha256:5726cf76c982532c1863fb64d8c6dd0e4c90b6ece9feb06c9f202417a31f7dd7 \ + --hash=sha256:5d447056e2ca60382d460a604b6302d8db69476fd2015c81e7c35417cfabe4cd \ + --hash=sha256:5ed2e36c3e9b4f21dd9422f6893dec0abf2cca553af509b10cd630f878d3eb99 \ + --hash=sha256:5ff2ed8194587faf56555927b3aa10e6fb69d931e33953943bc4f837dfee2242 \ + --hash=sha256:62f60aebecfc7f4b82e3f639a7d1433a20ec32824db2199a11ad4f5e146ef5ee \ + --hash=sha256:63bc5c4ae26e4bc6be6469943b8253c0fd4e4186c43ad46e713ea61a0ba49129 \ + --hash=sha256:6b40e8d38afe634559e398cc32b1472f376a4099c75fe6299ae607e404c033b2 \ + --hash=sha256:6b493a043635eb376e50eedf7818f2f322eabbaa974e948bd8bdd29eb7ef2a51 \ + --hash=sha256:6dba5d19c4dfab08e58d5b36304b3f92f3bd5d42c1a3fa37b5ba5cdf6dfcbcee \ + --hash=sha256:6fd30dc99682dc2c603c2b315bded2799019cea829f8bf57dc6b61efde6611c8 \ + --hash=sha256:707b82d19e65c9bd28b81dde95249b07bf9f5b90ebe1ef17d9b57473f8a64b7b \ + --hash=sha256:7706f5850360ac01d80c89bcef1640683cc12ed87f42579dab6c5d3ed6888613 \ + --hash=sha256:7782afc9b6b42200f7362858f9e73b1f8316afb276d316336c0ec3bd73312742 \ + --hash=sha256:79983512b108e4a164b9c8d34de3992f76d48cadc9554c9e60b43f308988aabe \ + --hash=sha256:7f683ddc7eedd742e2889d2bfb96d69573fde1d92fcb811979cdb7165bb9c7d3 \ + --hash=sha256:82357d85de703176b5587dbe6ade8ff67f9f69a41c0733cf2425378b49954de5 \ + --hash=sha256:84450ba661fb96e9fd67629b93d2941c871ca86fc38d835d19d4225ff946a631 \ + --hash=sha256:86f4e8cca779080f66ff4f191a685ced73d2f72d50216f7112185dc02b90b9b7 \ + --hash=sha256:8cda06946eac330cbe6598f77bb54e690b4ca93f593dee1568ad22b04f347c15 \ + --hash=sha256:8ce7fd6767a1cc5a92a639b391891bf1c268b03ec7e021c7d6d902285259685c \ + --hash=sha256:8ff4e7cdfdb1ab5698e675ca622e72d58a6fa2a8aa58195de0c0061288e6e3ea \ + --hash=sha256:9289fd5dddcf57bab41d044f1756550f9e7cf0c8e373b8cdf0ce8773dc4bd417 \ + --hash=sha256:92a7e36b000bf022ef3dbb9c46bfe2d52c047d5e3f3343f43204263c5addc250 \ + --hash=sha256:92db3c28b5b2a273346bebb24857fda45601aef6ae1c011c0a997106581e8a88 \ + --hash=sha256:95c3c157765b031331dd4db3c775e58deaee050a3042fcad72cbc4189d7c8dca \ + --hash=sha256:980b4f289d1d90ca5efcf07958d3eb38ed9c0b7676bf2831a54d4f66f9c27dfa \ + --hash=sha256:9ae4ef0b3f6b41bad6366fb0ea4fc1d7ed051528e113a60fa2a65a9abb5b1d99 \ + --hash=sha256:9c98230f5042f4945f957d006edccc2af1e03ed5e37ce7c373f00a5a4daa6149 \ + --hash=sha256:9fa2566ca27d67c86569e8c85297aaf413ffab85a8960500f12ea34ff98e4c41 \ + --hash=sha256:a14969b8691f7998e74663b77b4c36c0337cb1df552da83d5c9004a93afdb574 \ + --hash=sha256:a8aacce6e2e1edcb6ac625fb0f8c3a9570ccc7bfba1f63419b3769ccf6a00ed0 \ + --hash=sha256:a8e538f46104c815be19c975572d74afb53f29650ea2025bbfaef359d2de2f7f \ + --hash=sha256:aa41e526a5d4a9dfcfbab0716c7e8a1b215abd3f3df5a45cf18a12721d31cb5d \ + --hash=sha256:aa693779a8b50cd97570e5a0f343538a8dbd3e496fa5dcb87e29406ad0299654 \ + --hash=sha256:ab22fbd9765e6954bc0bcff24c25ff71dcbfdb185fcdaca49e81bac68fe724d3 \ + --hash=sha256:ab2e5bef076f5a235c3774b4f4028a680432cded7cad37bba0fd90d64b187d19 \ + --hash=sha256:ab973df98fc99ab39080bfb0eb3a925181454d7c3ac8a1e695fddfae696d9e90 \ + --hash=sha256:af73657b7a68211996527dbfeffbb0864e043d270580c5aef06dc4b659a4b578 \ + --hash=sha256:b197e7094f232959f8f20541ead1d9862ac5ebea1d58e9849c1bf979255dfac9 \ + --hash=sha256:b295729485b06c1a0683af02a9e42d2caa9db04a373dc38a6a58cdd1e8abddf1 \ + --hash=sha256:b8831399554b92b72af5932cdbbd4ddc55c55f631bb13ff8fe4e6536a06c5c51 \ + --hash=sha256:b8dcd239c743aa2f9c22ce674a145e0a25cb1566c495928440a181ca1ccf6719 \ + --hash=sha256:bcb4f8ea87d03bc51ad04add8ceaf9b0f085ac045ab4d74e73bbc2dc033f0236 \ + --hash=sha256:bd7af3717683bea4c87acd8c0d3d5b44d56120b26fd3f8a692bdd2d5260c620a \ + --hash=sha256:bf4475b82be41b07cc5e5ff94810e6a01f276e37c2d55571e3fe175e467a1a1c \ + --hash=sha256:c3e446d253bd88f6377260d07c895816ebf33ffffd56c1c792b13bff9c3e1ade \ + --hash=sha256:c57516e58fd17d03ebe67e181a4e4e2ccab1168f8c2976c6a334d4f819fe5944 \ + --hash=sha256:c94057af19bc953643a33581844649a7fdab902624d2eb739738a30e2b3e60fc \ + --hash=sha256:cab5d0b79d987c67f3b9e9c53f54a61360422a5a0bc075f43cab5621d530c3b6 \ + --hash=sha256:ce031db0408e487fd2775d745ce30a7cd2923667cf3b69d48d219f1d8f5ddeb6 \ + --hash=sha256:cee4373f4d3ad28f1ab6290684d8e2ebdb9e7a1b74fdc39e4c211995f77bec27 \ + --hash=sha256:d5b054862739d276e09928de37c79ddeec42a6e1bfc55863be96a36ba22926f6 \ + --hash=sha256:dbe03226baf438ac4fda9e2d0715022fd579cb641c4cf639fa40d53b2fe6f3e2 \ + --hash=sha256:dc15e99b2d8a656f8e666854404f1ba54765871104e50c8e9813af8a7db07f12 \ + --hash=sha256:dcaf7c1524c0542ee2fc82cc8ec337f7a9f7edee2532421ab200d2b920fc97cf \ + --hash=sha256:dd4eda173a9fcccb5f2e2bd2a9f423d180194b1bf17cf59e3269899235b2a114 \ + --hash=sha256:dd9a8bd8900e65504a305bf8ae6fa9fbc66de94178c420791d0293702fce2df7 \ + --hash=sha256:de7376c29d95d6719048c194a9cf1a1b0393fbe8488a22008610b0361d834ecf \ + --hash=sha256:e7fdd52961feb4c96507aa649550ec2a0d527c086d284749b2f582f2d40a2e0d \ + --hash=sha256:e91f541a85298cf35433bf66f3fab2a4a2cff05c127eeca4af174f6d497f0d4b \ + --hash=sha256:e9e3c4c9e1ed40ea53acf11e2a386383c3304212c965773704e4603d589343ed \ + --hash=sha256:ee803480535c44e7f5ad00788526da7d85525cfefaf8acf8ab9a310000be4b03 \ + --hash=sha256:f09cb5a7bbe1ecae6e87901a2eb23e0256bb524a79ccc53eb0b7629fbe7677c4 \ + --hash=sha256:f19c1585933c82098c2a520f8ec1227f20e339e33aca8fa6f956f6691b784e67 \ + --hash=sha256:f1a2f519ae173b5b6a2c9d5fa3116ce16e48b3462c8b96dfdded11055e3d6365 \ + --hash=sha256:f28f891ccd15c514a0981f3b9db9aa23d62fe1a99997512b0491d2ed323d229a \ + --hash=sha256:f3e73a4255342d4eb26ef6df01e3962e73aa29baa3124a8e824c5d3364a65748 \ + --hash=sha256:f606a1881d2663630ea5b8ce2efe2111740df4b687bd78b34a8131baa007f79b \ + --hash=sha256:fe9f97feb71aa9896b81973a7bbada8c49501dc73e58a10fcef6663af95e5079 \ + --hash=sha256:ffc519621dce0c767e96b9c53f09c5d215578e10b02c285809f76509a3931482 + # via + # -c tests/uv/lock/testdata/constraints.txt + # -c tests/uv/lock/testdata/constraints2.txt + # requests +idna==3.10 \ + --hash=sha256:12f65c9b470abda6dc35cf8e63cc574b1c52b11df2c86030af0ac09b01b13ea9 \ + --hash=sha256:946d195a0d259cbba61165e88e65941f16e9b36ea6ddb97f00452bae8b1287d3 + # via requests +requests==2.32.3 \ + --hash=sha256:55365417734eb18255590a9ff9eb97e9e1da868d4ccd6402399eaf68af20a760 \ + --hash=sha256:70761cfe03c773ceb22aa2f671b4757976145175cdfca038c02654d061d6dcc6 + # via -r tests/uv/lock/testdata/requirements.in +urllib3==2.3.0 \ + --hash=sha256:1cee9ad369867bfdbbb48b7dd50374c0967a0bb7710050facf0dd6911440e3df \ + --hash=sha256:f8c5449b3cf0861679ce7e0503c7b44b5ec981bec0d1d3795a07f1ba96f0204d + # via requests diff --git a/tools/private/publish_deps.bzl b/tools/private/publish_deps.bzl index 538cc1d583..a9b0dbc562 100644 --- a/tools/private/publish_deps.bzl +++ b/tools/private/publish_deps.bzl @@ -17,13 +17,27 @@ load("//python/uv/private:lock.bzl", "lock") # buildifier: disable=bzl-visibility -def publish_deps(*, name, outs, **kwargs): - """Generate all of the requirements files for all platforms.""" +def publish_deps(*, name, args, outs, **kwargs): + """Generate all of the requirements files for all platforms. + + Args: + name: {type}`str`: the currently unused. + args: {type}`list[str]`: the common args to apply. + outs: {type}`dict[Label, str]`: the output files mapping to the platform + for each requirement file to be generated. + **kwargs: Extra args passed to the {rule}`lock` rule. + """ + all_args = args for out, platform in outs.items(): + args = [] + all_args + if platform: + args.append("--python-platform=" + platform) + else: + args.append("--universal") + lock( name = out.replace(".txt", ""), out = out, - universal = platform == "", - args = [] if not platform else ["--python-platform=" + platform], + args = args, **kwargs ) diff --git a/tools/publish/BUILD.bazel b/tools/publish/BUILD.bazel index 4cf99e4d97..2f02809ccd 100644 --- a/tools/publish/BUILD.bazel +++ b/tools/publish/BUILD.bazel @@ -33,6 +33,9 @@ publish_deps( "requirements_universal.txt": "", # universal "requirements_windows.txt": "windows", }, - upgrade = True, + args = [ + "--emit-index-url", + "--upgrade", # always upgrade + ], visibility = ["//private:__pkg__"], ) From 5d6827eb016e4a1024a7b1fcdeab71ea9f978081 Mon Sep 17 00:00:00 2001 From: Christian von Schultz Date: Sat, 29 Mar 2025 14:53:57 +0100 Subject: [PATCH 127/922] feat(python.toolchain): support file-based default Python version (#2588) This change adds a new `default_version_file` attribute to `python.toolchain`. If set, the toolchain compares the file's contents to its `python_version`, and if they match, treats that toolchain as default (ignoring `is_default`). This allows Bazel to synchronize the default Python version with external tools (e.g., pyenv) that use a `.python-version` file or environment variables. Fixes #2587. --------- Co-authored-by: Ignas Anikevicius <240938+aignas@users.noreply.github.com> --- CHANGELOG.md | 3 + examples/multi_python_versions/MODULE.bazel | 5 + python/private/python.bzl | 156 +++++++++++++++++++- tests/python/python_tests.bzl | 95 +++++++++++- 4 files changed, 254 insertions(+), 5 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 80466fc3f9..3a2ff25b12 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -128,6 +128,9 @@ Unreleased changes template. {#v1-3-0-added} ### Added +* (python) {attr}`python.defaults` has been added to allow users to + set the default python version in the root module by reading the + default version number from a file or an environment variable. * {obj}`//python/bin:python`: convenience target for directly running an interpreter. {obj}`--//python/bin:python_src` can be used to specify a binary whose interpreter to use. diff --git a/examples/multi_python_versions/MODULE.bazel b/examples/multi_python_versions/MODULE.bazel index 74cb4b01df..85140360bb 100644 --- a/examples/multi_python_versions/MODULE.bazel +++ b/examples/multi_python_versions/MODULE.bazel @@ -10,6 +10,11 @@ local_path_override( ) python = use_extension("@rules_python//python/extensions:python.bzl", "python") +python.defaults( + # The environment variable takes precedence if set. + python_version = "3.9", + python_version_env = "BAZEL_PYTHON_VERSION", +) python.toolchain( configure_coverage_tool = True, # Only set when you have mulitple toolchain versions. diff --git a/python/private/python.bzl b/python/private/python.bzl index 304a1d7745..44eb09f766 100644 --- a/python/private/python.bzl +++ b/python/private/python.bzl @@ -78,6 +78,47 @@ def parse_modules(*, module_ctx, _fail = fail): config = _get_toolchain_config(modules = module_ctx.modules, _fail = _fail) + default_python_version = None + for mod in module_ctx.modules: + defaults_attr_structs = _create_defaults_attr_structs(mod = mod) + default_python_version_env = None + default_python_version_file = None + + # Only the root module and rules_python are allowed to specify the default + # toolchain for a couple reasons: + # * It prevents submodules from specifying different defaults and only + # one of them winning. + # * rules_python needs to set a soft default in case the root module doesn't, + # e.g. if the root module doesn't use Python itself. + # * The root module is allowed to override the rules_python default. + if mod.is_root or (mod.name == "rules_python" and not default_python_version): + for defaults_attr in defaults_attr_structs: + default_python_version = _one_or_the_same( + default_python_version, + defaults_attr.python_version, + onerror = _fail_multiple_defaults_python_version, + ) + default_python_version_env = _one_or_the_same( + default_python_version_env, + defaults_attr.python_version_env, + onerror = _fail_multiple_defaults_python_version_env, + ) + default_python_version_file = _one_or_the_same( + default_python_version_file, + defaults_attr.python_version_file, + onerror = _fail_multiple_defaults_python_version_file, + ) + if default_python_version_file: + default_python_version = _one_or_the_same( + default_python_version, + module_ctx.read(default_python_version_file, watch = "yes").strip(), + ) + if default_python_version_env: + default_python_version = module_ctx.getenv( + default_python_version_env, + default_python_version, + ) + seen_versions = {} for mod in module_ctx.modules: module_toolchain_versions = [] @@ -104,7 +145,13 @@ def parse_modules(*, module_ctx, _fail = fail): # * rules_python needs to set a soft default in case the root module doesn't, # e.g. if the root module doesn't use Python itself. # * The root module is allowed to override the rules_python default. - is_default = toolchain_attr.is_default + if default_python_version: + is_default = default_python_version == toolchain_version + if toolchain_attr.is_default and not is_default: + fail("The 'is_default' attribute doesn't work if you set " + + "the default Python version with the `defaults` tag.") + else: + is_default = toolchain_attr.is_default # Also only the root module should be able to decide ignore_root_user_error. # Modules being depended upon don't know the final environment, so they aren't @@ -115,7 +162,7 @@ def parse_modules(*, module_ctx, _fail = fail): fail("Toolchains in the root module must have consistent 'ignore_root_user_error' attributes") ignore_root_user_error = toolchain_attr.ignore_root_user_error - elif mod.name == "rules_python" and not default_toolchain: + elif mod.name == "rules_python" and not default_toolchain and not default_python_version: # We don't do the len() check because we want the default that rules_python # sets to be clearly visible. is_default = toolchain_attr.is_default @@ -282,6 +329,19 @@ def _python_impl(module_ctx): else: return None +def _one_or_the_same(first, second, *, onerror = None): + if not first: + return second + if not second or second == first: + return first + if onerror: + return onerror(first, second) + else: + fail("Unique value needed, got both '{}' and '{}', which are different".format( + first, + second, + )) + def _fail_duplicate_module_toolchain_version(version, module): fail(("Duplicate module toolchain version: module '{module}' attempted " + "to use version '{version}' multiple times in itself").format( @@ -305,6 +365,30 @@ def _warn_duplicate_global_toolchain_version(version, first, second_toolchain_na version = version, )) +def _fail_multiple_defaults_python_version(first, second): + fail(("Multiple python_version entries in defaults: " + + "First default was python_version '{first}'. " + + "Second was python_version '{second}'").format( + first = first, + second = second, + )) + +def _fail_multiple_defaults_python_version_file(first, second): + fail(("Multiple python_version_file entries in defaults: " + + "First default was python_version_file '{first}'. " + + "Second was python_version_file '{second}'").format( + first = first, + second = second, + )) + +def _fail_multiple_defaults_python_version_env(first, second): + fail(("Multiple python_version_env entries in defaults: " + + "First default was python_version_env '{first}'. " + + "Second was python_version_env '{second}'").format( + first = first, + second = second, + )) + def _fail_multiple_default_toolchains(first, second): fail(("Multiple default toolchains: only one toolchain " + "can have is_default=True. First default " + @@ -526,6 +610,21 @@ def _get_toolchain_config(*, modules, _fail = fail): register_all_versions = register_all_versions, ) +def _create_defaults_attr_structs(*, mod): + arg_structs = [] + + for tag in mod.tags.defaults: + arg_structs.append(_create_defaults_attr_struct(tag = tag)) + + return arg_structs + +def _create_defaults_attr_struct(*, tag): + return struct( + python_version = getattr(tag, "python_version", None), + python_version_env = getattr(tag, "python_version_env", None), + python_version_file = getattr(tag, "python_version_file", None), + ) + def _create_toolchain_attr_structs(*, mod, config, seen_versions): arg_structs = [] @@ -570,6 +669,49 @@ def _get_bazel_version_specific_kwargs(): return kwargs +_defaults = tag_class( + doc = """Tag class to specify the default Python version.""", + attrs = { + "python_version": attr.string( + mandatory = False, + doc = """\ +String saying what the default Python version should be. If the string +matches the {attr}`python_version` attribute of a toolchain, this +toolchain is the default version. If this attribute is set, the +{attr}`is_default` attribute of the toolchain is ignored. + +:::{versionadded} VERSION_NEXT_FEATURE +::: +""", + ), + "python_version_env": attr.string( + mandatory = False, + doc = """\ +Environment variable saying what the default Python version should be. +If the string matches the {attr}`python_version` attribute of a +toolchain, this toolchain is the default version. If this attribute is +set, the {attr}`is_default` attribute of the toolchain is ignored. + +:::{versionadded} VERSION_NEXT_FEATURE +::: +""", + ), + "python_version_file": attr.label( + mandatory = False, + allow_single_file = True, + doc = """\ +File saying what the default Python version should be. If the contents +of the file match the {attr}`python_version` attribute of a toolchain, +this toolchain is the default version. If this attribute is set, the +{attr}`is_default` attribute of the toolchain is ignored. + +:::{versionadded} VERSION_NEXT_FEATURE +::: +""", + ), + }, +) + _toolchain = tag_class( doc = """Tag class used to register Python toolchains. Use this tag class to register one or more Python toolchains. This class @@ -653,7 +795,14 @@ error to run with root access instead. ), "is_default": attr.bool( mandatory = False, - doc = "Whether the toolchain is the default version", + doc = """\ +Whether the toolchain is the default version. + +:::{versionchanged} VERSION_NEXT_FEATURE +This setting is ignored if the default version is set using the `defaults` +tag class. +::: +""", ), "python_version": attr.string( mandatory = True, @@ -852,6 +1001,7 @@ python = module_extension( """, implementation = _python_impl, tag_classes = { + "defaults": _defaults, "override": _override, "single_version_override": _single_version_override, "single_version_platform_override": _single_version_platform_override, diff --git a/tests/python/python_tests.bzl b/tests/python/python_tests.bzl index 6552251331..1679794e15 100644 --- a/tests/python/python_tests.bzl +++ b/tests/python/python_tests.bzl @@ -20,8 +20,11 @@ load("//python/private:python.bzl", "parse_modules") # buildifier: disable=bzl- _tests = [] -def _mock_mctx(*modules, environ = {}): +def _mock_mctx(*modules, environ = {}, mocked_files = {}): return struct( + path = lambda x: struct(exists = x in mocked_files, _file = x), + read = lambda x, watch = None: mocked_files[x._file if "_file" in dir(x) else x], + getenv = environ.get, os = struct(environ = environ), modules = [ struct( @@ -39,10 +42,11 @@ def _mock_mctx(*modules, environ = {}): ], ) -def _mod(*, name, toolchain = [], override = [], single_version_override = [], single_version_platform_override = [], is_root = True): +def _mod(*, name, defaults = [], toolchain = [], override = [], single_version_override = [], single_version_platform_override = [], is_root = True): return struct( name = name, tags = struct( + defaults = defaults, toolchain = toolchain, override = override, single_version_override = single_version_override, @@ -51,6 +55,13 @@ def _mod(*, name, toolchain = [], override = [], single_version_override = [], s is_root = is_root, ) +def _defaults(python_version = None, python_version_env = None, python_version_file = None): + return struct( + python_version = python_version, + python_version_env = python_version_env, + python_version_file = python_version_file, + ) + def _toolchain(python_version, *, is_default = False, **kwargs): return struct( is_default = is_default, @@ -273,6 +284,86 @@ def _test_default_non_rules_python_ignore_root_user_error_non_root_module(env): _tests.append(_test_default_non_rules_python_ignore_root_user_error_non_root_module) +def _test_default_from_defaults(env): + py = parse_modules( + module_ctx = _mock_mctx( + _mod( + name = "my_root_module", + defaults = [_defaults(python_version = "3.11")], + toolchain = [_toolchain("3.10"), _toolchain("3.11"), _toolchain("3.12")], + is_root = True, + ), + ), + ) + + env.expect.that_str(py.default_python_version).equals("3.11") + + want_toolchains = [ + struct( + name = "python_3_" + minor_version, + python_version = "3." + minor_version, + register_coverage_tool = False, + ) + for minor_version in ["10", "11", "12"] + ] + env.expect.that_collection(py.toolchains).contains_exactly(want_toolchains) + +_tests.append(_test_default_from_defaults) + +def _test_default_from_defaults_env(env): + py = parse_modules( + module_ctx = _mock_mctx( + _mod( + name = "my_root_module", + defaults = [_defaults(python_version = "3.11", python_version_env = "PYENV_VERSION")], + toolchain = [_toolchain("3.10"), _toolchain("3.11"), _toolchain("3.12")], + is_root = True, + ), + environ = {"PYENV_VERSION": "3.12"}, + ), + ) + + env.expect.that_str(py.default_python_version).equals("3.12") + + want_toolchains = [ + struct( + name = "python_3_" + minor_version, + python_version = "3." + minor_version, + register_coverage_tool = False, + ) + for minor_version in ["10", "11", "12"] + ] + env.expect.that_collection(py.toolchains).contains_exactly(want_toolchains) + +_tests.append(_test_default_from_defaults_env) + +def _test_default_from_defaults_file(env): + py = parse_modules( + module_ctx = _mock_mctx( + _mod( + name = "my_root_module", + defaults = [_defaults(python_version_file = "@@//:.python-version")], + toolchain = [_toolchain("3.10"), _toolchain("3.11"), _toolchain("3.12")], + is_root = True, + ), + mocked_files = {"@@//:.python-version": "3.12\n"}, + ), + ) + + env.expect.that_str(py.default_python_version).equals("3.12") + + want_toolchains = [ + struct( + name = "python_3_" + minor_version, + python_version = "3." + minor_version, + register_coverage_tool = False, + ) + for minor_version in ["10", "11", "12"] + ] + env.expect.that_collection(py.toolchains).contains_exactly(want_toolchains) + +_tests.append(_test_default_from_defaults_file) + def _test_first_occurance_of_the_toolchain_wins(env): py = parse_modules( module_ctx = _mock_mctx( From 67e233f491c16f9083181c40957223724e7a61b8 Mon Sep 17 00:00:00 2001 From: Ignas Anikevicius <240938+aignas@users.noreply.github.com> Date: Sun, 30 Mar 2025 07:58:44 +0900 Subject: [PATCH 128/922] fix(pypi): output only necessary target_platforms (#2710) This change reduces the number of lines we are going to write to the MODULE.bazel.lock file by not writing `experimental_target_platforms` to the lock file that eventually get discarded in the `whl_library` if the wheel is platform specific [1]. This means that the tests will become more easy to understand, but technically this is a no-op change, only resulting in a smaller lock file: ``` $ wc -l MODULE.bazel.lock 6536 MODULE.bazel.lock $ bazel mod deps --lockfile_mode=refresh ... $ wc -l MODULE.bazel.lock 6154 MODULE.bazel.lock ``` Work related to #2622 [1]: https://github.com/bazel-contrib/rules_python/blob/09145b9f628d482246eaa70421bf0cbae9acb096/python/private/pypi/whl_library.bzl#L337 --- CHANGELOG.md | 2 + python/private/pypi/BUILD.bazel | 1 + python/private/pypi/extension.bzl | 16 ++- tests/pypi/extension/extension_tests.bzl | 174 +++++++++++++++++++++++ 4 files changed, 190 insertions(+), 3 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 3a2ff25b12..cc742e6160 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -64,6 +64,8 @@ Unreleased changes template. * 3.12.9 * 3.13.2 * (pypi) Use `xcrun xcodebuild --showsdks` to find XCode root. +* (pypi) The `bzlmod` extension will now generate smaller lock files for when + using `experimental_index_url`. * (toolchains) Remove all but `3.8.20` versions of the Python `3.8` interpreter who has reached EOL. If users still need other versions of the `3.8` interpreter, please supply the URLs manually {bzl:ob}`python.toolchain` or {bzl:obj}`python_register_toolchains` calls. diff --git a/python/private/pypi/BUILD.bazel b/python/private/pypi/BUILD.bazel index 6f80272af6..79eb4dba46 100644 --- a/python/private/pypi/BUILD.bazel +++ b/python/private/pypi/BUILD.bazel @@ -93,6 +93,7 @@ bzl_library( ":whl_config_setting_bzl", ":whl_library_bzl", ":whl_repo_name_bzl", + ":whl_target_platforms_bzl", "//python/private:full_version_bzl", "//python/private:normalize_name_bzl", "//python/private:semver_bzl", diff --git a/python/private/pypi/extension.bzl b/python/private/pypi/extension.bzl index be00bf8ab3..be3067d04a 100644 --- a/python/private/pypi/extension.bzl +++ b/python/private/pypi/extension.bzl @@ -32,6 +32,7 @@ load(":simpleapi_download.bzl", "simpleapi_download") load(":whl_config_setting.bzl", "whl_config_setting") load(":whl_library.bzl", "whl_library") load(":whl_repo_name.bzl", "pypi_repo_name", "whl_repo_name") +load(":whl_target_platforms.bzl", "whl_target_platforms") def _major_minor_version(version): version = semver(version) @@ -296,9 +297,18 @@ def _whl_repos(*, requirement, whl_library_args, download_only, netrc, auth_patt # Pure python wheels or sdists may need to have a platform here target_platforms = None - if distribution.filename.endswith("-any.whl") or not distribution.filename.endswith(".whl"): - if multiple_requirements_for_whl: - target_platforms = requirement.target_platforms + if distribution.filename.endswith(".whl") and not distribution.filename.endswith("-any.whl"): + parsed_whl = parse_whl_name(distribution.filename) + whl_platforms = whl_target_platforms( + platform_tag = parsed_whl.platform_tag, + ) + args["experimental_target_platforms"] = [ + p + for p in requirement.target_platforms + if [None for wp in whl_platforms if p.endswith(wp.target_platform)] + ] + elif multiple_requirements_for_whl: + target_platforms = requirement.target_platforms repo_name = whl_repo_name( distribution.filename, diff --git a/tests/pypi/extension/extension_tests.bzl b/tests/pypi/extension/extension_tests.bzl index 8c01a02271..1b18d2a339 100644 --- a/tests/pypi/extension/extension_tests.bzl +++ b/tests/pypi/extension/extension_tests.bzl @@ -17,6 +17,7 @@ load("@rules_testing//lib:test_suite.bzl", "test_suite") load("@rules_testing//lib:truth.bzl", "subjects") load("//python/private/pypi:extension.bzl", "parse_modules") # buildifier: disable=bzl-visibility +load("//python/private/pypi:parse_simpleapi_html.bzl", "parse_simpleapi_html") # buildifier: disable=bzl-visibility load("//python/private/pypi:whl_config_setting.bzl", "whl_config_setting") # buildifier: disable=bzl-visibility _tests = [] @@ -332,6 +333,179 @@ torch==2.4.1 ; platform_machine != 'x86_64' \ _tests.append(_test_simple_with_markers) +def _test_torch_experimental_index_url(env): + def mocksimpleapi_download(*_, **__): + return { + "torch": parse_simpleapi_html( + url = "https://torch.index", + content = """\ + torch-2.4.1+cpu-cp310-cp310-linux_x86_64.whl
+ torch-2.4.1+cpu-cp310-cp310-win_amd64.whl
+ torch-2.4.1+cpu-cp311-cp311-linux_x86_64.whl
+ torch-2.4.1+cpu-cp311-cp311-win_amd64.whl
+ torch-2.4.1+cpu-cp312-cp312-linux_x86_64.whl
+ torch-2.4.1+cpu-cp312-cp312-win_amd64.whl
+ torch-2.4.1+cpu-cp38-cp38-linux_x86_64.whl
+ torch-2.4.1+cpu-cp38-cp38-win_amd64.whl
+ torch-2.4.1+cpu-cp39-cp39-linux_x86_64.whl
+ torch-2.4.1+cpu-cp39-cp39-win_amd64.whl
+ torch-2.4.1-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
+ torch-2.4.1-cp310-none-macosx_11_0_arm64.whl
+ torch-2.4.1-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
+ torch-2.4.1-cp311-none-macosx_11_0_arm64.whl
+ torch-2.4.1-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
+ torch-2.4.1-cp312-none-macosx_11_0_arm64.whl
+ torch-2.4.1-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
+ torch-2.4.1-cp38-none-macosx_11_0_arm64.whl
+ torch-2.4.1-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
+ torch-2.4.1-cp39-none-macosx_11_0_arm64.whl
+""", + ), + } + + pypi = _parse_modules( + env, + module_ctx = _mock_mctx( + _mod( + name = "rules_python", + parse = [ + _parse( + hub_name = "pypi", + python_version = "3.12", + experimental_index_url = "https://torch.index", + requirements_lock = "universal.txt", + ), + ], + ), + read = lambda x: { + "universal.txt": """\ +torch==2.4.1 ; platform_machine != 'x86_64' \ + --hash=sha256:1495132f30f722af1a091950088baea383fe39903db06b20e6936fd99402803e \ + --hash=sha256:30be2844d0c939161a11073bfbaf645f1c7cb43f62f46cc6e4df1c119fb2a798 \ + --hash=sha256:36109432b10bd7163c9b30ce896f3c2cca1b86b9765f956a1594f0ff43091e2a \ + --hash=sha256:56ad2a760b7a7882725a1eebf5657abbb3b5144eb26bcb47b52059357463c548 \ + --hash=sha256:5fc1d4d7ed265ef853579caf272686d1ed87cebdcd04f2a498f800ffc53dab71 \ + --hash=sha256:72b484d5b6cec1a735bf3fa5a1c4883d01748698c5e9cfdbeb4ffab7c7987e0d \ + --hash=sha256:a38de2803ee6050309aac032676536c3d3b6a9804248537e38e098d0e14817ec \ + --hash=sha256:d36a8ef100f5bff3e9c3cea934b9e0d7ea277cb8210c7152d34a9a6c5830eadd \ + --hash=sha256:ddddbd8b066e743934a4200b3d54267a46db02106876d21cf31f7da7a96f98ea \ + --hash=sha256:fa27b048d32198cda6e9cff0bf768e8683d98743903b7e5d2b1f5098ded1d343 + # via -r requirements.in +torch==2.4.1+cpu ; platform_machine == 'x86_64' \ + --hash=sha256:0c0a7cc4f7c74ff024d5a5e21230a01289b65346b27a626f6c815d94b4b8c955 \ + --hash=sha256:1dd062d296fb78aa7cfab8690bf03704995a821b5ef69cfc807af5c0831b4202 \ + --hash=sha256:2b03e20f37557d211d14e3fb3f71709325336402db132a1e0dd8b47392185baf \ + --hash=sha256:330e780f478707478f797fdc82c2a96e9b8c5f60b6f1f57bb6ad1dd5b1e7e97e \ + --hash=sha256:3a570e5c553415cdbddfe679207327b3a3806b21c6adea14fba77684d1619e97 \ + --hash=sha256:3c99506980a2fb4b634008ccb758f42dd82f93ae2830c1e41f64536e310bf562 \ + --hash=sha256:76a6fe7b10491b650c630bc9ae328df40f79a948296b41d3b087b29a8a63cbad \ + --hash=sha256:833490a28ac156762ed6adaa7c695879564fa2fd0dc51bcf3fdb2c7b47dc55e6 \ + --hash=sha256:8800deef0026011d502c0c256cc4b67d002347f63c3a38cd8e45f1f445c61364 \ + --hash=sha256:c4f2c3c026e876d4dad7629170ec14fff48c076d6c2ae0e354ab3fdc09024f00 + # via -r requirements.in +""", + }[x], + ), + available_interpreters = { + "python_3_12_host": "unit_test_interpreter_target", + }, + evaluate_markers = lambda _, requirements, **__: { + # todo once 2692 is merged, this is going to be easier to test. + key: [ + platform + for platform in platforms + if ("x86_64" in platform and "platform_machine ==" in key) or ("x86_64" not in platform and "platform_machine !=" in key) + ] + for key, platforms in requirements.items() + }, + simpleapi_download = mocksimpleapi_download, + ) + + pypi.is_reproducible().equals(False) + pypi.exposed_packages().contains_exactly({"pypi": ["torch"]}) + pypi.hub_group_map().contains_exactly({"pypi": {}}) + pypi.hub_whl_map().contains_exactly({"pypi": { + "torch": { + "pypi_312_torch_cp312_cp312_linux_x86_64_8800deef": [ + struct( + config_setting = None, + filename = "torch-2.4.1+cpu-cp312-cp312-linux_x86_64.whl", + target_platforms = None, + version = "3.12", + ), + ], + "pypi_312_torch_cp312_cp312_manylinux_2_17_aarch64_36109432": [ + struct( + config_setting = None, + filename = "torch-2.4.1-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", + target_platforms = None, + version = "3.12", + ), + ], + "pypi_312_torch_cp312_cp312_win_amd64_3a570e5c": [ + struct( + config_setting = None, + filename = "torch-2.4.1+cpu-cp312-cp312-win_amd64.whl", + target_platforms = None, + version = "3.12", + ), + ], + "pypi_312_torch_cp312_none_macosx_11_0_arm64_72b484d5": [ + struct( + config_setting = None, + filename = "torch-2.4.1-cp312-none-macosx_11_0_arm64.whl", + target_platforms = None, + version = "3.12", + ), + ], + }, + }}) + pypi.whl_libraries().contains_exactly({ + "pypi_312_torch_cp312_cp312_linux_x86_64_8800deef": { + "dep_template": "@pypi//{name}:{target}", + "experimental_target_platforms": ["cp312_linux_x86_64"], + "filename": "torch-2.4.1+cpu-cp312-cp312-linux_x86_64.whl", + "python_interpreter_target": "unit_test_interpreter_target", + "repo": "pypi_312", + "requirement": "torch==2.4.1+cpu", + "sha256": "8800deef0026011d502c0c256cc4b67d002347f63c3a38cd8e45f1f445c61364", + "urls": ["https://torch.index/whl/cpu/torch-2.4.1%2Bcpu-cp312-cp312-linux_x86_64.whl"], + }, + "pypi_312_torch_cp312_cp312_manylinux_2_17_aarch64_36109432": { + "dep_template": "@pypi//{name}:{target}", + "experimental_target_platforms": ["cp312_linux_aarch64"], + "filename": "torch-2.4.1-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", + "python_interpreter_target": "unit_test_interpreter_target", + "repo": "pypi_312", + "requirement": "torch==2.4.1", + "sha256": "36109432b10bd7163c9b30ce896f3c2cca1b86b9765f956a1594f0ff43091e2a", + "urls": ["https://torch.index/whl/cpu/torch-2.4.1-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl"], + }, + "pypi_312_torch_cp312_cp312_win_amd64_3a570e5c": { + "dep_template": "@pypi//{name}:{target}", + "experimental_target_platforms": ["cp312_windows_x86_64"], + "filename": "torch-2.4.1+cpu-cp312-cp312-win_amd64.whl", + "python_interpreter_target": "unit_test_interpreter_target", + "repo": "pypi_312", + "requirement": "torch==2.4.1+cpu", + "sha256": "3a570e5c553415cdbddfe679207327b3a3806b21c6adea14fba77684d1619e97", + "urls": ["https://torch.index/whl/cpu/torch-2.4.1%2Bcpu-cp312-cp312-win_amd64.whl"], + }, + "pypi_312_torch_cp312_none_macosx_11_0_arm64_72b484d5": { + "dep_template": "@pypi//{name}:{target}", + "experimental_target_platforms": ["cp312_osx_aarch64"], + "filename": "torch-2.4.1-cp312-none-macosx_11_0_arm64.whl", + "python_interpreter_target": "unit_test_interpreter_target", + "repo": "pypi_312", + "requirement": "torch==2.4.1", + "sha256": "72b484d5b6cec1a735bf3fa5a1c4883d01748698c5e9cfdbeb4ffab7c7987e0d", + "urls": ["https://torch.index/whl/cpu/torch-2.4.1-cp312-none-macosx_11_0_arm64.whl"], + }, + }) + pypi.whl_mods().contains_exactly({}) + +_tests.append(_test_torch_experimental_index_url) + def _test_download_only_multiple(env): pypi = _parse_modules( env, From bfad5078acee76f9743e6f1210784288571b0c0c Mon Sep 17 00:00:00 2001 From: Ignas Anikevicius <240938+aignas@users.noreply.github.com> Date: Sun, 30 Mar 2025 23:21:05 +0900 Subject: [PATCH 129/922] refactor(pypi): implement PEP508 compliant marker evaluation (#2692) This implements the PEP508 compliant marker evaluation in starlark and removes the need for the Python interpreter when evaluating requirements files passed to `pip.parse`. This makes the evaluation faster and allows us to fix a few known issues (#2690). In the future the intent is to move the `METADATA` parsing to pure starlark so that the `RequiresDist` could be parsed in starlark at the macro evaluation or analysis phases. This should make it possible to more easily solve the design problem that more and more things need to be passed to `whl_library` as args to have a robust dependency parsing: * #2319 needs the full Python version to have correct cross-platform compatible `METADATA` parsing and passing it to `Python` and back makes it difficult/annoying to implement. * Parsing the `METADATA` file requires the precise list of target platform or the list of available packages in the `requirements.txt`. This means that without it we cannot trim the dependency tree in the `whl_library`. Doing this at macro loading phase allows us to depend on `.bzl` files in the `hub_repository` and more effectively pass information. I can remotely see that this could become useful in `py_wheel` or an building wheels from sdists as the environment markers may be present in various source metadata as well. What is more `uv.lock` file has the env markers as part of the lock file information, so this might be useful there. Work towards #2423 Work towards #260 Split from #2629 --- python/private/pypi/BUILD.bazel | 35 +- python/private/pypi/evaluate_markers.bzl | 67 +-- python/private/pypi/extension.bzl | 35 +- python/private/pypi/parse_requirements.bzl | 12 +- python/private/pypi/pep508.bzl | 23 + python/private/pypi/pep508_env.bzl | 117 ++++ python/private/pypi/pep508_evaluate.bzl | 500 ++++++++++++++++++ python/private/pypi/pep508_req.bzl | 42 ++ python/private/pypi/pip_repository.bzl | 17 +- .../pypi/requirements_parser/BUILD.bazel | 0 .../resolve_target_platforms.py | 63 --- python/private/semver.bzl | 55 +- tests/pypi/extension/extension_tests.bzl | 19 - .../parse_requirements_tests.bzl | 2 +- tests/pypi/pep508/BUILD.bazel | 5 + tests/pypi/pep508/evaluate_tests.bzl | 271 ++++++++++ tests/semver/semver_test.bzl | 18 + 17 files changed, 1083 insertions(+), 198 deletions(-) create mode 100644 python/private/pypi/pep508.bzl create mode 100644 python/private/pypi/pep508_env.bzl create mode 100644 python/private/pypi/pep508_evaluate.bzl create mode 100644 python/private/pypi/pep508_req.bzl delete mode 100644 python/private/pypi/requirements_parser/BUILD.bazel delete mode 100755 python/private/pypi/requirements_parser/resolve_target_platforms.py create mode 100644 tests/pypi/pep508/BUILD.bazel create mode 100644 tests/pypi/pep508/evaluate_tests.bzl diff --git a/python/private/pypi/BUILD.bazel b/python/private/pypi/BUILD.bazel index 79eb4dba46..21e05f2895 100644 --- a/python/private/pypi/BUILD.bazel +++ b/python/private/pypi/BUILD.bazel @@ -75,7 +75,9 @@ bzl_library( name = "evaluate_markers_bzl", srcs = ["evaluate_markers.bzl"], deps = [ - ":pypi_repo_utils_bzl", + ":pep508_env_bzl", + ":pep508_evaluate_bzl", + ":pep508_req_bzl", ], ) @@ -209,6 +211,37 @@ bzl_library( ], ) +bzl_library( + name = "pep508_bzl", + srcs = ["pep508.bzl"], + deps = [ + ":pep508_env_bzl", + ":pep508_evaluate_bzl", + ], +) + +bzl_library( + name = "pep508_env_bzl", + srcs = ["pep508_env.bzl"], +) + +bzl_library( + name = "pep508_evaluate_bzl", + srcs = ["pep508_evaluate.bzl"], + deps = [ + "//python/private:enum_bzl", + "//python/private:semver_bzl", + ], +) + +bzl_library( + name = "pep508_req_bzl", + srcs = ["pep508_req.bzl"], + deps = [ + "//python/private:normalize_name_bzl", + ], +) + bzl_library( name = "pip_bzl", srcs = ["pip.bzl"], diff --git a/python/private/pypi/evaluate_markers.bzl b/python/private/pypi/evaluate_markers.bzl index 028657f716..1d4c30753f 100644 --- a/python/private/pypi/evaluate_markers.bzl +++ b/python/private/pypi/evaluate_markers.bzl @@ -14,65 +14,24 @@ """A simple function that evaluates markers using a python interpreter.""" -load(":deps.bzl", "record_files") -load(":pypi_repo_utils.bzl", "pypi_repo_utils") +load(":pep508_env.bzl", "env", _platform_from_str = "platform_from_str") +load(":pep508_evaluate.bzl", "evaluate") +load(":pep508_req.bzl", _req = "requirement") -# Used as a default value in a rule to ensure we fetch the dependencies. -SRCS = [ - # When the version, or any of the files in `packaging` package changes, - # this file will change as well. - record_files["pypi__packaging"], - Label("//python/private/pypi/requirements_parser:resolve_target_platforms.py"), - Label("//python/private/pypi/whl_installer:platform.py"), -] - -def evaluate_markers(mrctx, *, requirements, python_interpreter, python_interpreter_target, srcs, logger = None): +def evaluate_markers(requirements): """Return the list of supported platforms per requirements line. Args: - mrctx: repository_ctx or module_ctx. - requirements: list[str] of the requirement file lines to evaluate. - python_interpreter: str, path to the python_interpreter to use to - evaluate the env markers in the given requirements files. It will - be only called if the requirements files have env markers. This - should be something that is in your PATH or an absolute path. - python_interpreter_target: Label, same as python_interpreter, but in a - label format. - srcs: list[Label], the value of SRCS passed from the `rctx` or `mctx` to this function. - logger: repo_utils.logger or None, a simple struct to log diagnostic - messages. Defaults to None. + requirements: dict[str, list[str]] of the requirement file lines to evaluate. Returns: dict of string lists with target platforms """ - if not requirements: - return {} - - in_file = mrctx.path("requirements_with_markers.in.json") - out_file = mrctx.path("requirements_with_markers.out.json") - mrctx.file(in_file, json.encode(requirements)) - - pypi_repo_utils.execute_checked( - mrctx, - op = "ResolveRequirementEnvMarkers({})".format(in_file), - python = pypi_repo_utils.resolve_python_interpreter( - mrctx, - python_interpreter = python_interpreter, - python_interpreter_target = python_interpreter_target, - ), - arguments = [ - "-m", - "python.private.pypi.requirements_parser.resolve_target_platforms", - in_file, - out_file, - ], - srcs = srcs, - environment = { - "PYTHONPATH": [ - Label("@pypi__packaging//:BUILD.bazel"), - Label("//:BUILD.bazel"), - ], - }, - logger = logger, - ) - return json.decode(mrctx.read(out_file)) + ret = {} + for req_string, platforms in requirements.items(): + req = _req(req_string) + for platform in platforms: + if evaluate(req.marker, env = env(_platform_from_str(platform, None))): + ret.setdefault(req_string, []).append(platform) + + return ret diff --git a/python/private/pypi/extension.bzl b/python/private/pypi/extension.bzl index be3067d04a..490bd05f11 100644 --- a/python/private/pypi/extension.bzl +++ b/python/private/pypi/extension.bzl @@ -22,7 +22,7 @@ load("//python/private:repo_utils.bzl", "repo_utils") load("//python/private:semver.bzl", "semver") load("//python/private:version_label.bzl", "version_label") load(":attrs.bzl", "use_isolated") -load(":evaluate_markers.bzl", "evaluate_markers", EVALUATE_MARKERS_SRCS = "SRCS") +load(":evaluate_markers.bzl", "evaluate_markers") load(":hub_repository.bzl", "hub_repository", "whl_config_settings_to_json") load(":parse_requirements.bzl", "parse_requirements") load(":parse_whl_name.bzl", "parse_whl_name") @@ -167,28 +167,10 @@ def _create_whl_repos( ), extra_pip_args = pip_attr.extra_pip_args, get_index_urls = get_index_urls, - # NOTE @aignas 2024-08-02: , we will execute any interpreter that we find either - # in the PATH or if specified as a label. We will configure the env - # markers when evaluating the requirement lines based on the output - # from the `requirements_files_by_platform` which should have something - # similar to: - # { - # "//:requirements.txt": ["cp311_linux_x86_64", ...] - # } - # - # We know the target python versions that we need to evaluate the - # markers for and thus we don't need to use multiple python interpreter - # instances to perform this manipulation. This function should be executed - # only once by the underlying code to minimize the overhead needed to - # spin up a Python interpreter. - evaluate_markers = lambda module_ctx, requirements: evaluate_markers( - module_ctx, - requirements = requirements, - python_interpreter = pip_attr.python_interpreter, - python_interpreter_target = python_interpreter_target, - srcs = pip_attr._evaluate_markers_srcs, - logger = logger, - ), + # NOTE @aignas 2025-02-24: we will use the "cp3xx_os_arch" platform labels + # for converting to the PEP508 environment and will evaluate them in starlark + # without involving the interpreter at all. + evaluate_markers = evaluate_markers, logger = logger, ) @@ -774,13 +756,6 @@ a corresponding `python.toolchain()` configured. doc = """\ A dict of labels to wheel names that is typically generated by the whl_modifications. The labels are JSON config files describing the modifications. -""", - ), - "_evaluate_markers_srcs": attr.label_list( - default = EVALUATE_MARKERS_SRCS, - doc = """\ -The list of labels to use as SRCS for the marker evaluation code. This ensures that the -code will be re-evaluated when any of files in the default changes. """, ), }, **ATTRS) diff --git a/python/private/pypi/parse_requirements.bzl b/python/private/pypi/parse_requirements.bzl index dbff44ecb3..7aadc15eac 100644 --- a/python/private/pypi/parse_requirements.bzl +++ b/python/private/pypi/parse_requirements.bzl @@ -67,10 +67,10 @@ def parse_requirements( of the distribution URLs from a PyPI index. Accepts ctx and distribution names to query. evaluate_markers: A function to use to evaluate the requirements. - Accepts the ctx and a dict where keys are requirement lines to - evaluate against the platforms stored as values in the input dict. - Returns the same dict, but with values being platforms that are - compatible with the requirements line. + Accepts a dict where keys are requirement lines to evaluate against + the platforms stored as values in the input dict. Returns the same + dict, but with values being platforms that are compatible with the + requirements line. logger: repo_utils.logger or None, a simple struct to log diagnostic messages. Returns: @@ -93,7 +93,7 @@ def parse_requirements( The second element is extra_pip_args should be passed to `whl_library`. """ - evaluate_markers = evaluate_markers or (lambda *_: {}) + evaluate_markers = evaluate_markers or (lambda _: {}) options = {} requirements = {} for file, plats in requirements_by_platform.items(): @@ -168,7 +168,7 @@ def parse_requirements( # to do, we could use Python to parse the requirement lines and infer the # URL of the files to download things from. This should be important for # VCS package references. - env_marker_target_platforms = evaluate_markers(ctx, reqs_with_env_markers) + env_marker_target_platforms = evaluate_markers(reqs_with_env_markers) if logger: logger.debug(lambda: "Evaluated env markers from:\n{}\n\nTo:\n{}".format( reqs_with_env_markers, diff --git a/python/private/pypi/pep508.bzl b/python/private/pypi/pep508.bzl new file mode 100644 index 0000000000..e74352def2 --- /dev/null +++ b/python/private/pypi/pep508.bzl @@ -0,0 +1,23 @@ +# Copyright 2025 The Bazel Authors. All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +"""This module is for implementing PEP508 in starlark as FeatureFlagInfo +""" + +load(":pep508_env.bzl", _env = "env") +load(":pep508_evaluate.bzl", _evaluate = "evaluate", _to_string = "to_string") + +to_string = _to_string +evaluate = _evaluate +env = _env diff --git a/python/private/pypi/pep508_env.bzl b/python/private/pypi/pep508_env.bzl new file mode 100644 index 0000000000..17d41871d1 --- /dev/null +++ b/python/private/pypi/pep508_env.bzl @@ -0,0 +1,117 @@ +# Copyright 2025 The Bazel Authors. All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +"""This module is for implementing PEP508 environment definition. +""" + +# See https://stackoverflow.com/questions/45125516/possible-values-for-uname-m +_platform_machine_aliases = { + # These pairs mean the same hardware, but different values may be used + # on different host platforms. + "amd64": "x86_64", + "arm64": "aarch64", + "i386": "x86_32", + "i686": "x86_32", +} +_platform_system_values = { + "linux": "Linux", + "osx": "Darwin", + "windows": "Windows", +} +_sys_platform_values = { + "linux": "posix", + "osx": "darwin", + "windows": "win32", +} +_os_name_values = { + "linux": "posix", + "osx": "posix", + "windows": "nt", +} + +def env(target_platform, *, extra = None): + """Return an env target platform + + Args: + target_platform: {type}`str` the target platform identifier, e.g. + `cp33_linux_aarch64` + extra: {type}`str` the extra value to be added into the env. + + Returns: + A dict that can be used as `env` in the marker evaluation. + """ + + # TODO @aignas 2025-02-13: consider moving this into config settings. + + env = {"extra": extra} if extra != None else {} + env = env | { + "implementation_name": "cpython", + "platform_python_implementation": "CPython", + "platform_release": "", + "platform_version": "", + } + if type(target_platform) == type(""): + target_platform = platform_from_str(target_platform, python_version = "") + + if target_platform.abi: + minor_version, _, micro_version = target_platform.abi[3:].partition(".") + micro_version = micro_version or "0" + env = env | { + "implementation_version": "3.{}.{}".format(minor_version, micro_version), + "python_full_version": "3.{}.{}".format(minor_version, micro_version), + "python_version": "3.{}".format(minor_version), + } + if target_platform.os and target_platform.arch: + os = target_platform.os + env = env | { + "os_name": _os_name_values.get(os, ""), + "platform_machine": target_platform.arch, + "platform_system": _platform_system_values.get(os, ""), + "sys_platform": _sys_platform_values.get(os, ""), + } + + # This is split by topic + return env | { + "_aliases": { + "platform_machine": _platform_machine_aliases, + }, + } + +def _platform(*, abi = None, os = None, arch = None): + return struct( + abi = abi, + os = os, + arch = arch, + ) + +def platform_from_str(p, python_version): + """Return a platform from a string. + + Args: + p: {type}`str` the actual string. + python_version: {type}`str` the python version to add to platform if needed. + + Returns: + A struct that is returned by the `_platform` function. + """ + if p.startswith("cp"): + abi, _, p = p.partition("_") + elif python_version: + major, _, tail = python_version.partition(".") + abi = "cp{}{}".format(major, tail) + else: + abi = None + + os, _, arch = p.partition("_") + return _platform(abi = abi, os = os or None, arch = arch or None) diff --git a/python/private/pypi/pep508_evaluate.bzl b/python/private/pypi/pep508_evaluate.bzl new file mode 100644 index 0000000000..f45eb75cdb --- /dev/null +++ b/python/private/pypi/pep508_evaluate.bzl @@ -0,0 +1,500 @@ +# Copyright 2025 The Bazel Authors. All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +"""This module is for implementing PEP508 in starlark as FeatureFlagInfo +""" + +load("//python/private:enum.bzl", "enum") +load("//python/private:semver.bzl", "semver") + +# The expression parsing and resolution for the PEP508 is below +# + +# Taken from +# https://peps.python.org/pep-0508/#grammar +# +# version_cmp = wsp* '<' | '<=' | '!=' | '==' | '>=' | '>' | '~=' | '===' +_VERSION_CMP = sorted( + [ + i.strip(" '") + for i in "'<' | '<=' | '!=' | '==' | '>=' | '>' | '~=' | '==='".split(" | ") + ], + key = lambda x: (-len(x), x), +) + +_STATE = enum( + STRING = "string", + VAR = "var", + OP = "op", + NONE = "none", +) +_BRACKETS = "()" +_OPCHARS = "<>!=~" +_QUOTES = "'\"" +_WSP = " \t" +_NON_VERSION_VAR_NAMES = [ + "implementation_name", + "os_name", + "platform_machine", + "platform_python_implementation", + "platform_release", + "platform_system", + "sys_platform", + "extra", +] +_AND = "and" +_OR = "or" +_NOT = "not" +_ENV_ALIASES = "_aliases" + +def tokenize(marker): + """Tokenize the input string. + + The output will have double-quoted values (i.e. the quoting will be normalized) and all of the whitespace will be trimmed. + + Args: + marker: {type}`str` The input to tokenize. + + Returns: + The {type}`str` that is the list of recognized tokens that should be parsed. + """ + if not marker: + return [] + + tokens = [] + token = "" + state = _STATE.NONE + char = "" + + # Due to the `continue` in the loop, we will be processing chars at a slower pace + for _ in range(2 * len(marker)): + if token and (state == _STATE.NONE or not marker): + if tokens and token == "in" and tokens[-1] == _NOT: + tokens[-1] += " " + token + else: + tokens.append(token) + token = "" + + if not marker: + return tokens + + char = marker[0] + if char in _BRACKETS: + state = _STATE.NONE + token = char + elif state == _STATE.STRING and char in _QUOTES: + state = _STATE.NONE + token = '"{}"'.format(token) + elif ( + (state == _STATE.VAR and not char.isalnum() and char != "_") or + (state == _STATE.OP and char not in _OPCHARS) + ): + state = _STATE.NONE + continue # Skip consuming the char below + elif state == _STATE.NONE: + # Transition from _STATE.NONE to something or stay in NONE + if char in _QUOTES: + state = _STATE.STRING + elif char.isalnum(): + state = _STATE.VAR + token += char + elif char in _OPCHARS: + state = _STATE.OP + token += char + elif char in _WSP: + state = _STATE.NONE + else: + fail("BUG: Cannot parse '{}' in {} ({})".format(char, state, marker)) + else: + token += char + + # Consume the char + marker = marker[1:] + + return fail("BUG: failed to process the marker in allocated cycles: {}".format(marker)) + +def evaluate(marker, *, env, strict = True, **kwargs): + """Evaluate the marker against a given env. + + Args: + marker: {type}`str` The string marker to evaluate. + env: {type}`dict` The environment to evaluate the marker against. + strict: {type}`bool` A setting to not fail on missing values in the env. + **kwargs: Extra kwargs to be passed to the expression evaluator. + + Returns: + The {type}`bool` If the marker is compatible with the given env. + """ + tokens = tokenize(marker) + + ast = _new_expr(**kwargs) + for _ in range(len(tokens) * 2): + if not tokens: + break + + tokens = ast.parse(env = env, tokens = tokens, strict = strict) + + if not tokens: + return ast.value() + + fail("Could not evaluate: {}".format(marker)) + +_STRING_REPLACEMENTS = { + "!=": "neq", + "(": "_", + ")": "_", + "<": "lt", + "<=": "lteq", + "==": "eq", + "===": "eeq", + ">": "gt", + ">=": "gteq", + "not in": "not_in", + "~==": "cmp", +} + +def to_string(marker): + return "_".join([ + _STRING_REPLACEMENTS.get(t, t) + for t in tokenize(marker) + ]).replace("\"", "") + +def _and_fn(x, y): + """Our custom `and` evaluation function. + + Allow partial evaluation if one of the values is a string, return the + string value because that means that `marker_expr` was set to + `strict = False` and we are only evaluating what we can. + """ + if not (x and y): + return False + + x_is_str = type(x) == type("") + y_is_str = type(y) == type("") + if x_is_str and y_is_str: + return "{} and {}".format(x, y) + elif x_is_str: + return x + else: + return y + +def _or_fn(x, y): + """Our custom `or` evaluation function. + + Allow partial evaluation if one of the values is a string, return the + string value because that means that `marker_expr` was set to + `strict = False` and we are only evaluating what we can. + """ + x_is_str = type(x) == type("") + y_is_str = type(y) == type("") + + if x_is_str and y_is_str: + return "{} or {}".format(x, y) if x and y else "" + elif x_is_str: + return "" if y else x + elif y_is_str: + return "" if x else y + else: + return x or y + +def _not_fn(x): + """Our custom `not` evaluation function. + + Allow partial evaluation if the value is a string. + """ + if type(x) == type(""): + return "not {}".format(x) + else: + return not x + +def _new_expr( + and_fn = _and_fn, + or_fn = _or_fn, + not_fn = _not_fn): + # buildifier: disable=uninitialized + self = struct( + tree = [], + parse = lambda **kwargs: _parse(self, **kwargs), + value = lambda: _value(self), + # This is a way for us to have a handle to the currently constructed + # expression tree branch. + current = lambda: self._current[0] if self._current else None, + _current = [], + _and = and_fn, + _or = or_fn, + _not = not_fn, + ) + return self + +def _parse(self, *, env, tokens, strict = False): + """The parse function takes the consumed tokens and returns the remaining.""" + token, remaining = tokens[0], tokens[1:] + + if token == "(": + expr = _open_parenthesis(self) + elif token == ")": + expr = _close_parenthesis(self) + elif token == _AND: + expr = _and_expr(self) + elif token == _OR: + expr = _or_expr(self) + elif token == _NOT: + expr = _not_expr(self) + else: + expr = marker_expr(env = env, strict = strict, *tokens[:3]) + remaining = tokens[3:] + + _append(self, expr) + return remaining + +def _value(self): + """Evaluate the expression tree""" + if not self.tree: + # Basic case where no marker should evaluate to True + return True + + for _ in range(len(self.tree)): + if len(self.tree) == 1: + return self.tree[0] + + # Resolve all of the `or` expressions as it is safe to do now since all + # `and` and `not` expressions have been taken care of by now. + if getattr(self.tree[-2], "op", None) == _OR: + current = self.tree.pop() + self.tree[-1] = self.tree[-1].value(current) + else: + break + + fail("BUG: invalid state: {}".format(self.tree)) + +def marker_expr(left, op, right, *, env, strict = True): + """Evaluate a marker expression + + Args: + left: {type}`str` the env identifier or a value quoted in `"`. + op: {type}`str` the operation to carry out. + right: {type}`str` the env identifier or a value quoted in `"`. + strict: {type}`bool` if false, only evaluates the values that are present + in the environment, otherwise returns the original expression. + env: {type}`dict[str, str]` the `env` to substitute `env` identifiers in + the ` ` expression. Note, if `env` has a key + "_aliases", then we will do normalization so that we can ensure + that e.g. `aarch64` evaluation in the `platform_machine` works the + same way irrespective if the marker uses `arm64` or `aarch64` value + in the expression. + + Returns: + {type}`bool` if the expression evaluation result or {type}`str` if the expression + could not be evaluated. + """ + var_name = None + if right not in env and left not in env and not strict: + return "{} {} {}".format(left, op, right) + if left[0] == '"': + var_name = right + right = env[right] + left = left.strip("\"") + + if _ENV_ALIASES in env: + # On Windows, Linux, OSX different values may mean the same hardware, + # e.g. Python on Windows returns arm64, but on Linux returns aarch64. + # e.g. Python on Windows returns amd64, but on Linux returns x86_64. + # + # The following normalizes the values + left = env.get(_ENV_ALIASES, {}).get(var_name, {}).get(left, left) + else: + var_name = left + left = env[left] + right = right.strip("\"") + + if _ENV_ALIASES in env: + # See the note above on normalization + right = env.get(_ENV_ALIASES, {}).get(var_name, {}).get(right, right) + + if var_name in _NON_VERSION_VAR_NAMES: + return _env_expr(left, op, right) + elif var_name.endswith("_version"): + return _version_expr(left, op, right) + else: + # Do not fail here, just evaluate the expression to False. + return False + +def _env_expr(left, op, right): + """Evaluate a string comparison expression""" + if op == "==": + return left == right + elif op == "!=": + return left != right + elif op == "in": + return left in right + elif op == "not in": + return left not in right + else: + return fail("TODO: op unsupported: '{}'".format(op)) + +def _version_expr(left, op, right): + """Evaluate a version comparison expression""" + left = semver(left) + right = semver(right) + _left = left.key() + _right = right.key() + if op == "<": + return _left < _right + elif op == ">": + return _left > _right + elif op == "<=": + return _left <= _right + elif op == ">=": + return _left >= _right + elif op == "!=": + return _left != _right + elif op == "==": + # Matching of major, minor, patch only + return _left[:3] == _right[:3] + elif op == "~=": + right_plus = right.upper() + _right_plus = right_plus.key() + return _left >= _right and _left < _right_plus + elif op == "===": + # Strict matching + return _left == _right + elif op in _VERSION_CMP: + fail("TODO: op unsupported: '{}'".format(op)) + else: + return False # Let's just ignore the invalid ops + +# Code to allowing to combine expressions with logical operators + +def _append(self, value): + if value == None: + return + + current = self.current() or self + op = getattr(value, "op", None) + + if op == _NOT: + current.tree.append(value) + elif op in [_AND, _OR]: + value.append(current.tree[-1]) + current.tree[-1] = value + elif not current.tree: + current.tree.append(value) + elif hasattr(current.tree[-1], "append"): + current.tree[-1].append(value) + else: + current.tree._append(value) + +def _open_parenthesis(self): + """Add an extra node into the tree to perform evaluate inside parenthesis.""" + self._current.append(_new_expr( + and_fn = self._and, + or_fn = self._or, + not_fn = self._not, + )) + +def _close_parenthesis(self): + """Backtrack and evaluate the expression within parenthesis.""" + value = self._current.pop().value() + if type(value) == type(""): + return "({})".format(value) + else: + return value + +def _not_expr(self): + """Add an extra node into the tree to perform an 'not' operation.""" + + def _append(value): + """Append a value to the not expression node. + + This codifies `not` precedence over `and` and performs backtracking to + evaluate any `not` statements and forward the value to the first `and` + statement if needed. + """ + + current = self.current() or self + current.tree[-1] = self._not(value) + + for _ in range(len(current.tree)): + if not len(current.tree) > 1: + break + + op = getattr(current.tree[-2], "op", None) + if op == None: + pass + elif op == _NOT: + value = current.tree.pop() + current.tree[-1] = self._not(value) + continue + elif op == _AND: + value = current.tree.pop() + current.tree[-1].append(value) + elif op != _OR: + fail("BUG: '{} not' compound is unsupported".format(current.tree[-1])) + + break + + return struct( + op = _NOT, + append = _append, + ) + +def _and_expr(self): + """Add an extra node into the tree to perform an 'and' operation""" + maybe_value = [None] + + def _append(value): + """Append a value to the and expression node. + + Here we backtrack, but we only evaluate the current `and` statement - + all of the `not` statements will be by now evaluated and `or` + statements need to be evaluated later. + """ + if maybe_value[0] == None: + maybe_value[0] = value + return + + current = self.current() or self + current.tree[-1] = self._and(maybe_value[0], value) + + return struct( + op = _AND, + append = _append, + # private fields that help debugging + _maybe_value = maybe_value, + ) + +def _or_expr(self): + """Add an extra node into the tree to perform an 'or' operation""" + maybe_value = [None] + + def _append(value): + """Append a value to the or expression node. + + Here we just append the extra values to the tree and the `or` + statements will be evaluated in the _value() function. + """ + if maybe_value[0] == None: + maybe_value[0] = value + return + + current = self.current() or self + current.tree.append(value) + + return struct( + op = _OR, + value = lambda x: self._or(maybe_value[0], x), + append = _append, + # private fields that help debugging + _maybe_value = maybe_value, + ) diff --git a/python/private/pypi/pep508_req.bzl b/python/private/pypi/pep508_req.bzl new file mode 100644 index 0000000000..618ffaf17a --- /dev/null +++ b/python/private/pypi/pep508_req.bzl @@ -0,0 +1,42 @@ +# Copyright 2025 The Bazel Authors. All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +"""This module is for parsing PEP508 requires-dist and requirements lines. +""" + +load("//python/private:normalize_name.bzl", "normalize_name") + +_STRIP = ["(", " ", ">", "=", "<", "~", "!"] + +def requirement(spec): + """Parse a PEP508 requirement line + + Args: + spec: {type}`str` requirement line that will be parsed. + + Returns: + A struct with the information. + """ + requires, _, maybe_hashes = spec.partition(";") + marker, _, _ = maybe_hashes.partition("--hash") + requires, _, extras_unparsed = requires.partition("[") + for char in _STRIP: + requires, _, _ = requires.partition(char) + extras = extras_unparsed.strip("]").split(",") + + return struct( + name = normalize_name(requires.strip(" ")), + marker = marker.strip(" "), + extras = extras, + ) diff --git a/python/private/pypi/pip_repository.bzl b/python/private/pypi/pip_repository.bzl index 7976cfaae9..01a541cf2f 100644 --- a/python/private/pypi/pip_repository.bzl +++ b/python/private/pypi/pip_repository.bzl @@ -18,7 +18,7 @@ load("@bazel_skylib//lib:sets.bzl", "sets") load("//python/private:normalize_name.bzl", "normalize_name") load("//python/private:repo_utils.bzl", "REPO_DEBUG_ENV_VAR") load("//python/private:text_util.bzl", "render") -load(":evaluate_markers.bzl", "evaluate_markers", EVALUATE_MARKERS_SRCS = "SRCS") +load(":evaluate_markers.bzl", "evaluate_markers") load(":parse_requirements.bzl", "host_platform", "parse_requirements", "select_requirement") load(":pip_repository_attrs.bzl", "ATTRS") load(":render_pkg_aliases.bzl", "render_pkg_aliases") @@ -82,13 +82,7 @@ def _pip_repository_impl(rctx): extra_pip_args = rctx.attr.extra_pip_args, ), extra_pip_args = rctx.attr.extra_pip_args, - evaluate_markers = lambda rctx, requirements: evaluate_markers( - rctx, - requirements = requirements, - python_interpreter = rctx.attr.python_interpreter, - python_interpreter_target = rctx.attr.python_interpreter_target, - srcs = rctx.attr._evaluate_markers_srcs, - ), + evaluate_markers = evaluate_markers, ) selected_requirements = {} options = None @@ -234,13 +228,6 @@ file](https://github.com/bazel-contrib/rules_python/blob/main/examples/pip_repos _template = attr.label( default = ":requirements.bzl.tmpl.workspace", ), - _evaluate_markers_srcs = attr.label_list( - default = EVALUATE_MARKERS_SRCS, - doc = """\ -The list of labels to use as SRCS for the marker evaluation code. This ensures that the -code will be re-evaluated when any of files in the default changes. -""", - ), **ATTRS ), doc = """Accepts a locked/compiled requirements file and installs the dependencies listed within. diff --git a/python/private/pypi/requirements_parser/BUILD.bazel b/python/private/pypi/requirements_parser/BUILD.bazel deleted file mode 100644 index e69de29bb2..0000000000 diff --git a/python/private/pypi/requirements_parser/resolve_target_platforms.py b/python/private/pypi/requirements_parser/resolve_target_platforms.py deleted file mode 100755 index c899a943cc..0000000000 --- a/python/private/pypi/requirements_parser/resolve_target_platforms.py +++ /dev/null @@ -1,63 +0,0 @@ -"""A CLI to evaluate env markers for requirements files. - -A simple script to evaluate the `requirements.txt` files. Currently it is only -handling environment markers in the requirements files, but in the future it -may handle more things. We require a `python` interpreter that can run on the -host platform and then we depend on the [packaging] PyPI wheel. - -In order to be able to resolve requirements files for any platform, we are -re-using the same code that is used in the `whl_library` installer. See -[here](../whl_installer/wheel.py). - -Requirements for the code are: -- Depends only on `packaging` and core Python. -- Produces the same result irrespective of the Python interpreter platform or version. - -[packaging]: https://packaging.pypa.io/en/stable/ -""" - -import argparse -import json -import pathlib - -from packaging.requirements import Requirement - -from python.private.pypi.whl_installer.platform import Platform - -INPUT_HELP = """\ -Input path to read the requirements as a json file, the keys in the dictionary -are the requirements lines and the values are strings of target platforms. -""" -OUTPUT_HELP = """\ -Output to write the requirements as a json filepath, the keys in the dictionary -are the requirements lines and the values are strings of target platforms, which -got changed based on the evaluated markers. -""" - - -def main(): - parser = argparse.ArgumentParser(description=__doc__) - parser.add_argument("input_path", type=pathlib.Path, help=INPUT_HELP.strip()) - parser.add_argument("output_path", type=pathlib.Path, help=OUTPUT_HELP.strip()) - args = parser.parse_args() - - with args.input_path.open() as f: - reqs = json.load(f) - - response = {} - for requirement_line, target_platforms in reqs.items(): - entry, prefix, hashes = requirement_line.partition("--hash") - hashes = prefix + hashes - - req = Requirement(entry) - for p in target_platforms: - (platform,) = Platform.from_string(p) - if not req.marker or req.marker.evaluate(platform.env_markers("")): - response.setdefault(requirement_line, []).append(p) - - with args.output_path.open("w") as f: - json.dump(response, f) - - -if __name__ == "__main__": - main() diff --git a/python/private/semver.bzl b/python/private/semver.bzl index 73d6b130ae..cc9ae6ecb6 100644 --- a/python/private/semver.bzl +++ b/python/private/semver.bzl @@ -43,6 +43,49 @@ def _to_dict(self): "pre_release": self.pre_release, } +def _upper(self): + major = self.major + minor = self.minor + patch = self.patch + build = "" + pre_release = "" + version = self.str() + + if patch != None: + minor = minor + 1 + patch = 0 + elif minor != None: + major = major + 1 + minor = 0 + elif minor == None: + major = major + 1 + + return _new( + major = major, + minor = minor, + patch = patch, + build = build, + pre_release = pre_release, + version = "~" + version, + ) + +def _new(*, major, minor, patch, pre_release, build, version = None): + # buildifier: disable=uninitialized + self = struct( + major = int(major), + minor = None if minor == None else int(minor), + # NOTE: this is called `micro` in the Python interpreter versioning scheme + patch = None if patch == None else int(patch), + pre_release = pre_release, + build = build, + # buildifier: disable=uninitialized + key = lambda: _key(self), + str = lambda: version, + to_dict = lambda: _to_dict(self), + upper = lambda: _upper(self), + ) + return self + def semver(version): """Parse the semver version and return the values as a struct. @@ -59,17 +102,11 @@ def semver(version): patch, _, build = tail.partition("+") patch, _, pre_release = patch.partition("-") - # buildifier: disable=uninitialized - self = struct( + return _new( major = int(major), minor = int(minor) if minor.isdigit() else None, - # NOTE: this is called `micro` in the Python interpreter versioning scheme patch = int(patch) if patch.isdigit() else None, - pre_release = pre_release, build = build, - # buildifier: disable=uninitialized - key = lambda: _key(self), - str = lambda: version, - to_dict = lambda: _to_dict(self), + pre_release = pre_release, + version = version, ) - return self diff --git a/tests/pypi/extension/extension_tests.bzl b/tests/pypi/extension/extension_tests.bzl index 1b18d2a339..858c026df8 100644 --- a/tests/pypi/extension/extension_tests.bzl +++ b/tests/pypi/extension/extension_tests.bzl @@ -77,7 +77,6 @@ def _parse( *, hub_name, python_version, - _evaluate_markers_srcs = [], add_libdir_to_library_search_path = False, auth_patterns = {}, download_only = False, @@ -105,7 +104,6 @@ def _parse( whl_modifications = {}, **kwargs): return struct( - _evaluate_markers_srcs = _evaluate_markers_srcs, auth_patterns = auth_patterns, add_libdir_to_library_search_path = add_libdir_to_library_search_path, download_only = download_only, @@ -276,14 +274,6 @@ torch==2.4.1 ; platform_machine != 'x86_64' \ available_interpreters = { "python_3_15_host": "unit_test_interpreter_target", }, - evaluate_markers = lambda _, requirements, **__: { - key: [ - platform - for platform in platforms - if ("x86_64" in platform and "platform_machine ==" in key) or ("x86_64" not in platform and "platform_machine !=" in key) - ] - for key, platforms in requirements.items() - }, ) pypi.is_reproducible().equals(True) @@ -409,15 +399,6 @@ torch==2.4.1+cpu ; platform_machine == 'x86_64' \ available_interpreters = { "python_3_12_host": "unit_test_interpreter_target", }, - evaluate_markers = lambda _, requirements, **__: { - # todo once 2692 is merged, this is going to be easier to test. - key: [ - platform - for platform in platforms - if ("x86_64" in platform and "platform_machine ==" in key) or ("x86_64" not in platform and "platform_machine !=" in key) - ] - for key, platforms in requirements.items() - }, simpleapi_download = mocksimpleapi_download, ) diff --git a/tests/pypi/parse_requirements/parse_requirements_tests.bzl b/tests/pypi/parse_requirements/parse_requirements_tests.bzl index 8edc2689bf..7bbd696afa 100644 --- a/tests/pypi/parse_requirements/parse_requirements_tests.bzl +++ b/tests/pypi/parse_requirements/parse_requirements_tests.bzl @@ -454,7 +454,7 @@ def _test_select_requirement_none_platform(env): _tests.append(_test_select_requirement_none_platform) def _test_env_marker_resolution(env): - def _mock_eval_markers(_, input): + def _mock_eval_markers(input): ret = { "foo[extra]==0.0.1 ;marker --hash=sha256:deadbeef": ["cp311_windows_x86_64"], } diff --git a/tests/pypi/pep508/BUILD.bazel b/tests/pypi/pep508/BUILD.bazel new file mode 100644 index 0000000000..b795db0591 --- /dev/null +++ b/tests/pypi/pep508/BUILD.bazel @@ -0,0 +1,5 @@ +load(":evaluate_tests.bzl", "evaluate_test_suite") + +evaluate_test_suite( + name = "evaluate_tests", +) diff --git a/tests/pypi/pep508/evaluate_tests.bzl b/tests/pypi/pep508/evaluate_tests.bzl new file mode 100644 index 0000000000..80b70f4dad --- /dev/null +++ b/tests/pypi/pep508/evaluate_tests.bzl @@ -0,0 +1,271 @@ +# Copyright 2024 The Bazel Authors. All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +"""Tests for construction of Python version matching config settings.""" + +load("@rules_testing//lib:test_suite.bzl", "test_suite") +load("//python/private/pypi:pep508_env.bzl", pep508_env = "env") # buildifier: disable=bzl-visibility +load("//python/private/pypi:pep508_evaluate.bzl", "evaluate", "tokenize") # buildifier: disable=bzl-visibility + +_tests = [] + +def _tokenize_tests(env): + for input, want in { + "": [], + "'osx' == os_name": ['"osx"', "==", "os_name"], + "'x' not in os_name": ['"x"', "not in", "os_name"], + "()": ["(", ")"], + "(os_name == 'osx' and not os_name == 'posix') or os_name == \"win\"": [ + "(", + "os_name", + "==", + '"osx"', + "and", + "not", + "os_name", + "==", + '"posix"', + ")", + "or", + "os_name", + "==", + '"win"', + ], + "os_name\t==\t'osx'": ["os_name", "==", '"osx"'], + "os_name == 'osx'": ["os_name", "==", '"osx"'], + "python_version <= \"1.0\"": ["python_version", "<=", '"1.0"'], + "python_version>='1.0.0'": ["python_version", ">=", '"1.0.0"'], + "python_version~='1.0.0'": ["python_version", "~=", '"1.0.0"'], + }.items(): + got = tokenize(input) + env.expect.that_collection(got).contains_exactly(want).in_order() + +_tests.append(_tokenize_tests) + +def _evaluate_non_version_env_tests(env): + for var_name in [ + "implementation_name", + "os_name", + "platform_machine", + "platform_python_implementation", + "platform_release", + "platform_system", + "sys_platform", + "extra", + ]: + # Given + marker_env = {var_name: "osx"} + + # When + for input, want in { + "{} == 'osx'".format(var_name): True, + "{} != 'osx'".format(var_name): False, + "'osx' == {}".format(var_name): True, + "'osx' != {}".format(var_name): False, + "'x' in {}".format(var_name): True, + "'w' not in {}".format(var_name): True, + }.items(): # buildifier: @unsorted-dict-items + got = evaluate( + input, + env = marker_env, + ) + env.expect.that_bool(got).equals(want) + + # Check that the non-strict eval gives us back the input when no + # env is supplied. + got = evaluate( + input, + env = {}, + strict = False, + ) + env.expect.that_bool(got).equals(input.replace("'", '"')) + +_tests.append(_evaluate_non_version_env_tests) + +def _evaluate_version_env_tests(env): + for var_name in [ + "python_version", + "implementation_version", + "platform_version", + "python_full_version", + ]: + # Given + marker_env = {var_name: "3.7.9"} + + # When + for input, want in { + "{} < '3.8'".format(var_name): True, + "{} > '3.7'".format(var_name): True, + "{} >= '3.7.9'".format(var_name): True, + "{} >= '3.7.10'".format(var_name): False, + "{} >= '3.7.8'".format(var_name): True, + "{} <= '3.7.9'".format(var_name): True, + "{} <= '3.7.10'".format(var_name): True, + "{} <= '3.7.8'".format(var_name): False, + "{} == '3.7.9'".format(var_name): True, + "{} != '3.7.9'".format(var_name): False, + "{} ~= '3.7.1'".format(var_name): True, + "{} ~= '3.7.10'".format(var_name): False, + "{} ~= '3.8.0'".format(var_name): False, + "{} === '3.7.9+rc2'".format(var_name): False, + "{} === '3.7.9'".format(var_name): True, + "{} == '3.7.9+rc2'".format(var_name): True, + }.items(): # buildifier: @unsorted-dict-items + got = evaluate( + input, + env = marker_env, + ) + env.expect.that_collection((input, got)).contains_exactly((input, want)) + + # Check that the non-strict eval gives us back the input when no + # env is supplied. + got = evaluate( + input, + env = {}, + strict = False, + ) + env.expect.that_bool(got).equals(input.replace("'", '"')) + +_tests.append(_evaluate_version_env_tests) + +def _logical_expression_tests(env): + for input, want in { + # Basic + "": True, + "(())": True, + "()": True, + + # expr + "os_name == 'fo'": False, + "(os_name == 'fo')": False, + "not (os_name == 'fo')": True, + + # and + "os_name == 'fo' and os_name == 'foo'": False, + + # and not + "os_name == 'fo' and not os_name == 'foo'": False, + + # or + "os_name == 'oo' or os_name == 'foo'": True, + + # or not + "os_name == 'foo' or not os_name == 'foo'": True, + + # multiple or + "os_name == 'oo' or os_name == 'fo' or os_name == 'foo'": True, + "os_name == 'oo' or os_name == 'foo' or os_name == 'fo'": True, + + # multiple and + "os_name == 'foo' and os_name == 'foo' and os_name == 'fo'": False, + + # x or not y and z != (x or not y), but is instead evaluated as x or (not y and z) + "os_name == 'foo' or not os_name == 'fo' and os_name == 'fo'": True, + + # x or y and z != (x or y) and z, but is instead evaluated as x or (y and z) + "os_name == 'foo' or os_name == 'fo' and os_name == 'fo'": True, + "not (os_name == 'foo' or os_name == 'fo' and os_name == 'fo')": False, + + # x or y and z and w != (x or y and z) and w, but is instead evaluated as x or (y and z and w) + "os_name == 'foo' or os_name == 'fo' and os_name == 'fo' and os_name == 'fo'": True, + + # not not True + "not not os_name == 'foo'": True, + "not not not os_name == 'foo'": False, + }.items(): # buildifier: @unsorted-dict-items + got = evaluate( + input, + env = { + "os_name": "foo", + }, + ) + env.expect.that_collection((input, got)).contains_exactly((input, want)) + + if not input.strip("()"): + # These cases will just return True, because they will be evaluated + # and the brackets will be processed. + continue + + # Check that the non-strict eval gives us back the input when no env + # is supplied. + got = evaluate( + input, + env = {}, + strict = False, + ) + env.expect.that_bool(got).equals(input.replace("'", '"')) + +_tests.append(_logical_expression_tests) + +def _evaluate_partial_only_extra(env): + # Given + extra = "foo" + + # When + for input, want in { + "os_name == 'osx' and extra == 'bar'": False, + "os_name == 'osx' and extra == 'foo'": "os_name == \"osx\"", + "platform_system == 'aarch64' and os_name == 'osx' and extra == 'foo'": "platform_system == \"aarch64\" and os_name == \"osx\"", + "platform_system == 'aarch64' and extra == 'foo' and os_name == 'osx'": "platform_system == \"aarch64\" and os_name == \"osx\"", + "os_name == 'osx' or extra == 'bar'": "os_name == \"osx\"", + "os_name == 'osx' or extra == 'foo'": "", + "extra == 'bar' or os_name == 'osx'": "os_name == \"osx\"", + "extra == 'foo' or os_name == 'osx'": "", + "os_name == 'win' or extra == 'bar' or os_name == 'osx'": "os_name == \"win\" or os_name == \"osx\"", + "os_name == 'win' or extra == 'foo' or os_name == 'osx'": "", + }.items(): # buildifier: @unsorted-dict-items + got = evaluate( + input, + env = { + "extra": extra, + }, + strict = False, + ) + env.expect.that_bool(got).equals(want) + +_tests.append(_evaluate_partial_only_extra) + +def _evaluate_with_aliases(env): + # When + for target_platform, tests in { + # buildifier: @unsorted-dict-items + "osx_aarch64": { + "platform_system == 'Darwin' and platform_machine == 'arm64'": True, + "platform_system == 'Darwin' and platform_machine == 'aarch64'": True, + "platform_system == 'Darwin' and platform_machine == 'amd64'": False, + }, + "osx_x86_64": { + "platform_system == 'Darwin' and platform_machine == 'amd64'": True, + "platform_system == 'Darwin' and platform_machine == 'x86_64'": True, + }, + "osx_x86_32": { + "platform_system == 'Darwin' and platform_machine == 'i386'": True, + "platform_system == 'Darwin' and platform_machine == 'i686'": True, + "platform_system == 'Darwin' and platform_machine == 'x86_32'": True, + "platform_system == 'Darwin' and platform_machine == 'x86_64'": False, + }, + }.items(): # buildifier: @unsorted-dict-items + for input, want in tests.items(): + got = evaluate( + input, + env = pep508_env(target_platform), + ) + env.expect.that_bool(got).equals(want) + +_tests.append(_evaluate_with_aliases) + +def evaluate_test_suite(name): # buildifier: disable=function-docstring + test_suite( + name = name, + basic_tests = _tests, + ) diff --git a/tests/semver/semver_test.bzl b/tests/semver/semver_test.bzl index 9d13402c92..aef3deca82 100644 --- a/tests/semver/semver_test.bzl +++ b/tests/semver/semver_test.bzl @@ -104,6 +104,24 @@ def _test_semver_sort(env): _tests.append(_test_semver_sort) +def _test_upper(env): + for input, want in { + # Depending on how many version numbers are specified we will increase + # the upper bound differently. See https://packaging.python.org/en/latest/specifications/version-specifiers/#compatible-release for docs + "0.0.1": "0.1.0", + "0.1": "1.0", + "0.1.0": "0.2.0", + "1": "2", + "1.0.0-pre": "1.1.0", # pre-release info is dropped + "1.2.0": "1.3.0", + "2.0.0+build0": "2.1.0", # build info is dropped + }.items(): + actual = semver(input).upper().key() + want = semver(want).key() + env.expect.that_collection(actual).contains_exactly(want).in_order() + +_tests.append(_test_upper) + def semver_test_suite(name): """Create the test suite. From 43e3d75f666654fcaf6d116f48cc16696da6ba4b Mon Sep 17 00:00:00 2001 From: Logan Pulley Date: Mon, 31 Mar 2025 17:06:26 -0500 Subject: [PATCH 130/922] fix(docs): CHANGELOG "astral" typo (#2715) It appears to have been copied from the 1.1.0 "Added" section, but I'm not sure whether "fixing" old changelogs is acceptable. --- CHANGELOG.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index cc742e6160..5974a656a6 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -57,7 +57,7 @@ Unreleased changes template. `exec_interpreter` now also forwards the `ToolchainInfo` provider. This is for increased compatibility with the `RBE` setups where access to the `exec` configuration interpreter is needed. -* (toolchains) Use the latest astrahl-sh toolchain release [20250317] for Python versions: +* (toolchains) Use the latest astral-sh toolchain release [20250317] for Python versions: * 3.9.21 * 3.10.16 * 3.11.11 From 5cfd948d5c0567a9bc555a1ee1dbd5434a98c9c5 Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Tue, 1 Apr 2025 08:40:07 +0900 Subject: [PATCH 131/922] build(deps): bump certifi from 2024.8.30 to 2025.1.31 in /docs (#2718) Bumps [certifi](https://github.com/certifi/python-certifi) from 2024.8.30 to 2025.1.31.
Commits
  • 088f931 2025.01.31 (#336)
  • 1c17795 Bump pypa/gh-action-pypi-publish from 1.12.3 to 1.12.4 (#335)
  • a2e88f0 Bump actions/upload-artifact from 4.5.0 to 4.6.0 (#334)
  • 82284ed Bump peter-evans/create-pull-request from 7.0.5 to 7.0.6 (#333)
  • 10d3d1d Bump actions/upload-artifact from 4.4.3 to 4.5.0 (#332)
  • 4ba3900 2024.12.14 (#329)
  • 9164660 Bump pypa/gh-action-pypi-publish from 1.12.2 to 1.12.3 (#331)
  • 3dc3651 Bump pypa/gh-action-pypi-publish from 1.11.0 to 1.12.2 (#328)
  • c5bf18d Bump pypa/gh-action-pypi-publish from 1.10.3 to 1.11.0 (#327)
  • b908391 Bump actions/setup-python from 5.2.0 to 5.3.0 (#326)
  • Additional commits viewable in compare view

[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=certifi&package-manager=pip&previous-version=2024.8.30&new-version=2025.1.31)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) ---
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot show ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> --- docs/requirements.txt | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/requirements.txt b/docs/requirements.txt index a49e8f9fe2..eb39af0da5 100644 --- a/docs/requirements.txt +++ b/docs/requirements.txt @@ -18,9 +18,9 @@ babel==2.17.0 \ --hash=sha256:0c54cffb19f690cdcc52a3b50bcbf71e07a808d1c80d549f2459b9d2cf0afb9d \ --hash=sha256:4d0b53093fdfb4b21c92b5213dba5a1b23885afa8383709427046b21c366e5f2 # via sphinx -certifi==2024.8.30 \ - --hash=sha256:922820b53db7a7257ffbda3f597266d435245903d80737e34f8a45ff3e3230d8 \ - --hash=sha256:bec941d2aa8195e248a60b31ff9f0558284cf01a52591ceda73ea9afffd69fd9 +certifi==2025.1.31 \ + --hash=sha256:3d5da6925056f6f18f119200434a4780a94263f10d1c21d032a6f6b2baa20651 \ + --hash=sha256:ca78db4565a652026a4db2bcdf68f2fb589ea80d0be70e03929ed730746b84fe # via requests charset-normalizer==3.4.0 \ --hash=sha256:0099d79bdfcf5c1f0c2c72f91516702ebf8b0b8ddd8905f97a8aecf49712c621 \ From 20aa5269718a98d2514ee4651b3b899f277e7cf8 Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Tue, 1 Apr 2025 08:40:36 +0900 Subject: [PATCH 132/922] build(deps): bump sphinx-reredirects from 0.1.5 to 0.1.6 in /docs (#2716) Bumps [sphinx-reredirects](https://github.com/documatt/sphinx-reredirects) from 0.1.5 to 0.1.6.
Commits
  • 9c21d3b chore: release 0.1.6
  • 638f011 Merge branch 'davidekete-preserve-url-fragments'
  • e50560f Merge branch 'main' into preserve-url-fragments
  • a0822b5 feat: update default HTML template to preserve url fragments
  • 29503e3 style: reformatted with prettier
  • 19207de chore: setup maintenance tools
  • 4671309 feat: update FAQ to match new default template
  • 36c6a8b feat: update default HTML template to preserve url fragments
  • 7b3cf64 docs: Update LICENSE to MIT
  • 1fb15c8 docs: create README.md
  • Additional commits viewable in compare view

[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=sphinx-reredirects&package-manager=pip&previous-version=0.1.5&new-version=0.1.6)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) ---
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot show ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> --- docs/requirements.txt | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/requirements.txt b/docs/requirements.txt index eb39af0da5..8f8b18d3f2 100644 --- a/docs/requirements.txt +++ b/docs/requirements.txt @@ -328,9 +328,9 @@ sphinx-autodoc2==0.5.0 \ --hash=sha256:7d76044aa81d6af74447080182b6868c7eb066874edc835e8ddf810735b6565a \ --hash=sha256:e867013b1512f9d6d7e6f6799f8b537d6884462acd118ef361f3f619a60b5c9e # via rules-python-docs (docs/pyproject.toml) -sphinx-reredirects==0.1.5 \ - --hash=sha256:444ae1438fba4418242ca76d6a6de3eaee82aaf0d8f2b0cac71a15d32ce6eba2 \ - --hash=sha256:cfa753b441020a22708ce8eb17d4fd553a28fc87a609330092917ada2a6da0d8 +sphinx-reredirects==0.1.6 \ + --hash=sha256:c491cba545f67be9697508727818d8626626366245ae64456fe29f37e9bbea64 \ + --hash=sha256:efd50c766fbc5bf40cd5148e10c00f2c00d143027de5c5e48beece93cc40eeea # via rules-python-docs (docs/pyproject.toml) sphinx-rtd-theme==3.0.1 \ --hash=sha256:921c0ece75e90633ee876bd7b148cfaad136b481907ad154ac3669b6fc957916 \ From 7d102062675f9306a1120d9f90049b50f3137eb8 Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Mon, 31 Mar 2025 23:48:02 +0000 Subject: [PATCH 133/922] build(deps): bump certifi from 2024.8.30 to 2025.1.31 in /tools/publish (#2719) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit [//]: # (dependabot-start) ⚠️ **Dependabot is rebasing this PR** ⚠️ Rebasing might not happen immediately, so don't worry if this takes some time. Note: if you make any changes to this PR yourself, they will take precedence over the rebase. --- [//]: # (dependabot-end) Bumps [certifi](https://github.com/certifi/python-certifi) from 2024.8.30 to 2025.1.31.
Commits
  • 088f931 2025.01.31 (#336)
  • 1c17795 Bump pypa/gh-action-pypi-publish from 1.12.3 to 1.12.4 (#335)
  • a2e88f0 Bump actions/upload-artifact from 4.5.0 to 4.6.0 (#334)
  • 82284ed Bump peter-evans/create-pull-request from 7.0.5 to 7.0.6 (#333)
  • 10d3d1d Bump actions/upload-artifact from 4.4.3 to 4.5.0 (#332)
  • 4ba3900 2024.12.14 (#329)
  • 9164660 Bump pypa/gh-action-pypi-publish from 1.12.2 to 1.12.3 (#331)
  • 3dc3651 Bump pypa/gh-action-pypi-publish from 1.11.0 to 1.12.2 (#328)
  • c5bf18d Bump pypa/gh-action-pypi-publish from 1.10.3 to 1.11.0 (#327)
  • b908391 Bump actions/setup-python from 5.2.0 to 5.3.0 (#326)
  • Additional commits viewable in compare view

[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=certifi&package-manager=pip&previous-version=2024.8.30&new-version=2025.1.31)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) ---
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot show ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> --- tools/publish/requirements_darwin.txt | 6 +++--- tools/publish/requirements_linux.txt | 6 +++--- tools/publish/requirements_universal.txt | 6 +++--- tools/publish/requirements_windows.txt | 6 +++--- 4 files changed, 12 insertions(+), 12 deletions(-) diff --git a/tools/publish/requirements_darwin.txt b/tools/publish/requirements_darwin.txt index 9c9398ade5..e8ee1e9b89 100644 --- a/tools/publish/requirements_darwin.txt +++ b/tools/publish/requirements_darwin.txt @@ -6,9 +6,9 @@ backports-tarfile==1.2.0 \ --hash=sha256:77e284d754527b01fb1e6fa8a1afe577858ebe4e9dad8919e34c862cb399bc34 \ --hash=sha256:d75e02c268746e1b8144c278978b6e98e85de6ad16f8e4b0844a154557eca991 # via jaraco-context -certifi==2024.8.30 \ - --hash=sha256:922820b53db7a7257ffbda3f597266d435245903d80737e34f8a45ff3e3230d8 \ - --hash=sha256:bec941d2aa8195e248a60b31ff9f0558284cf01a52591ceda73ea9afffd69fd9 +certifi==2025.1.31 \ + --hash=sha256:3d5da6925056f6f18f119200434a4780a94263f10d1c21d032a6f6b2baa20651 \ + --hash=sha256:ca78db4565a652026a4db2bcdf68f2fb589ea80d0be70e03929ed730746b84fe # via requests charset-normalizer==3.4.0 \ --hash=sha256:0099d79bdfcf5c1f0c2c72f91516702ebf8b0b8ddd8905f97a8aecf49712c621 \ diff --git a/tools/publish/requirements_linux.txt b/tools/publish/requirements_linux.txt index 147fb2d206..892b8b26b3 100644 --- a/tools/publish/requirements_linux.txt +++ b/tools/publish/requirements_linux.txt @@ -6,9 +6,9 @@ backports-tarfile==1.2.0 \ --hash=sha256:77e284d754527b01fb1e6fa8a1afe577858ebe4e9dad8919e34c862cb399bc34 \ --hash=sha256:d75e02c268746e1b8144c278978b6e98e85de6ad16f8e4b0844a154557eca991 # via jaraco-context -certifi==2024.8.30 \ - --hash=sha256:922820b53db7a7257ffbda3f597266d435245903d80737e34f8a45ff3e3230d8 \ - --hash=sha256:bec941d2aa8195e248a60b31ff9f0558284cf01a52591ceda73ea9afffd69fd9 +certifi==2025.1.31 \ + --hash=sha256:3d5da6925056f6f18f119200434a4780a94263f10d1c21d032a6f6b2baa20651 \ + --hash=sha256:ca78db4565a652026a4db2bcdf68f2fb589ea80d0be70e03929ed730746b84fe # via requests cffi==1.17.1 \ --hash=sha256:045d61c734659cc045141be4bae381a41d89b741f795af1dd018bfb532fd0df8 \ diff --git a/tools/publish/requirements_universal.txt b/tools/publish/requirements_universal.txt index 2ad13f5688..337073ac25 100644 --- a/tools/publish/requirements_universal.txt +++ b/tools/publish/requirements_universal.txt @@ -6,9 +6,9 @@ backports-tarfile==1.2.0 ; python_full_version < '3.12' \ --hash=sha256:77e284d754527b01fb1e6fa8a1afe577858ebe4e9dad8919e34c862cb399bc34 \ --hash=sha256:d75e02c268746e1b8144c278978b6e98e85de6ad16f8e4b0844a154557eca991 # via jaraco-context -certifi==2024.8.30 \ - --hash=sha256:922820b53db7a7257ffbda3f597266d435245903d80737e34f8a45ff3e3230d8 \ - --hash=sha256:bec941d2aa8195e248a60b31ff9f0558284cf01a52591ceda73ea9afffd69fd9 +certifi==2025.1.31 \ + --hash=sha256:3d5da6925056f6f18f119200434a4780a94263f10d1c21d032a6f6b2baa20651 \ + --hash=sha256:ca78db4565a652026a4db2bcdf68f2fb589ea80d0be70e03929ed730746b84fe # via requests cffi==1.17.1 ; platform_python_implementation != 'PyPy' and sys_platform == 'linux' \ --hash=sha256:045d61c734659cc045141be4bae381a41d89b741f795af1dd018bfb532fd0df8 \ diff --git a/tools/publish/requirements_windows.txt b/tools/publish/requirements_windows.txt index bb87804df5..1c6b9808fb 100644 --- a/tools/publish/requirements_windows.txt +++ b/tools/publish/requirements_windows.txt @@ -6,9 +6,9 @@ backports-tarfile==1.2.0 \ --hash=sha256:77e284d754527b01fb1e6fa8a1afe577858ebe4e9dad8919e34c862cb399bc34 \ --hash=sha256:d75e02c268746e1b8144c278978b6e98e85de6ad16f8e4b0844a154557eca991 # via jaraco-context -certifi==2024.8.30 \ - --hash=sha256:922820b53db7a7257ffbda3f597266d435245903d80737e34f8a45ff3e3230d8 \ - --hash=sha256:bec941d2aa8195e248a60b31ff9f0558284cf01a52591ceda73ea9afffd69fd9 +certifi==2025.1.31 \ + --hash=sha256:3d5da6925056f6f18f119200434a4780a94263f10d1c21d032a6f6b2baa20651 \ + --hash=sha256:ca78db4565a652026a4db2bcdf68f2fb589ea80d0be70e03929ed730746b84fe # via requests charset-normalizer==3.4.0 \ --hash=sha256:0099d79bdfcf5c1f0c2c72f91516702ebf8b0b8ddd8905f97a8aecf49712c621 \ From 1f8659c816c7d81b29ff9d534565cfb78dfcb72e Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Tue, 1 Apr 2025 00:04:51 +0000 Subject: [PATCH 134/922] build(deps): bump pygments from 2.18.0 to 2.19.1 in /docs (#2720) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Bumps [pygments](https://github.com/pygments/pygments) from 2.18.0 to 2.19.1.
Release notes

Sourced from pygments's releases.

2.19.1

  • Updated lexers:

    • Ini: Fix quoted string regression introduced in 2.19.0
    • Lua: Fix a regression introduced in 2.19.0

2.19.0

  • New lexers:

  • Updated lexers:

    • BQN: Various improvements (#2789)
    • C#: Fix number highlighting (#986, #2727), add file keyword (#2726, #2805, #2806), add various other keywords (#2745, #2770)
    • CSS: Add revert (#2766, #2775)
    • Debian control: Add Change-By field (#2757)
    • Elip: Improve punctuation handling (#2651)
    • Igor: Add int (#2801)
    • Ini: Fix quoted strings with embedded comment characters (#2767, #2720)
    • Java: Support functions returning types containing a question mark (#2737)
    • JavaScript: Support private identiiers (#2729, #2671)
    • LLVM: Add splat, improve floating-point number parsing (#2755)
    • Lua: Improve variable detection, add built-in functions (#2829)
    • Macaulay2: Update to 1.24.11 (#2800)
    • PostgreSQL: Add more EXPLAIN keywords (#2785), handle / (#2774)
    • S-Lexer: Fix keywords (#2082, #2750)
    • TransactSQL: Fix single-line comments (#2717)
    • Turtle: Fix triple quoted strings (#2744, #2758)
    • Typst: Various improvements (#2724)
    • Various: Add ^ as an operator to Matlab, Octave and Scilab (#2798)
    • Vyper: Add staticcall and extcall (#2719)
  • Mark file extensions for HTML/XML+Evoque as aliases (#2743)
  • Add a color for Operator.Word to the rrt style (#2709)
  • Fix broken link in the documentation (#2803, #2804)
  • Drop executable bit where not needed (#2781)
  • Reduce Mojo priority relative to Python in ``analyze_text´` (#2771, #2772)
  • Fix documentation builds (#2712)
  • Match example file names to the lexer's name (#2713, #2715)

... (truncated)

Changelog

Sourced from pygments's changelog.

Version 2.19.1

(released January 6th, 2025)

  • Updated lexers:

    • Ini: Fix quoted string regression introduced in 2.19.0
    • Lua: Fix a regression introduced in 2.19.0

Version 2.19.0

(released January 5th, 2025)

  • New lexers:

  • Updated lexers:

    • BQN: Various improvements (#2789)
    • C#: Fix number highlighting (#986, #2727), add file keyword (#2726, #2805, #2806), add various other keywords (#2745, #2770)
    • CSS: Add revert (#2766, #2775)
    • Debian control: Add Change-By field (#2757)
    • Elip: Improve punctuation handling (#2651)
    • Igor: Add int (#2801)
    • Ini: Fix quoted strings with embedded comment characters (#2767, #2720)
    • Java: Support functions returning types containing a question mark (#2737)
    • JavaScript: Support private identiiers (#2729, #2671)
    • LLVM: Add splat, improve floating-point number parsing (#2755)
    • Lua: Improve variable detection, add built-in functions (#2829)
    • Macaulay2: Update to 1.24.11 (#2800)
    • PostgreSQL: Add more EXPLAIN keywords (#2785), handle / (#2774)
    • S-Lexer: Fix keywords (#2082, #2750)
    • TransactSQL: Fix single-line comments (#2717)
    • Turtle: Fix triple quoted strings (#2744, #2758)
    • Typst: Various improvements (#2724)
    • Various: Add ^ as an operator to Matlab, Octave and Scilab (#2798)
    • Vyper: Add staticcall and extcall (#2719)
  • Mark file extensions for HTML/XML+Evoque as aliases (#2743)

... (truncated)

Commits

[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=pygments&package-manager=pip&previous-version=2.18.0&new-version=2.19.1)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) ---
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot show ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Ignas Anikevicius <240938+aignas@users.noreply.github.com> --- docs/requirements.txt | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/requirements.txt b/docs/requirements.txt index 8f8b18d3f2..7e62e94fab 100644 --- a/docs/requirements.txt +++ b/docs/requirements.txt @@ -242,9 +242,9 @@ packaging==24.1 \ # via # readthedocs-sphinx-ext # sphinx -pygments==2.18.0 \ - --hash=sha256:786ff802f32e91311bff3889f6e9a86e81505fe99f2735bb6d60ae0c5004f199 \ - --hash=sha256:b8e6aca0523f3ab76fee51799c488e38782ac06eafcf95e7ba832985c8e7b13a +pygments==2.19.1 \ + --hash=sha256:61c16d2a8576dc0649d9f39e089b5f02bcd27fba10d8fb4dcc28173f7a45151f \ + --hash=sha256:9ea1544ad55cecf4b8242fab6dd35a93bbce657034b0611ee383099054ab6d8c # via sphinx pyyaml==6.0.2 \ --hash=sha256:01179a4a8559ab5de078078f37e5c1a30d76bb88519906844fd7bdea1b7729ff \ From 481db1354d27712567bfaed0a31dcc2a7241beb1 Mon Sep 17 00:00:00 2001 From: armandomontanez Date: Mon, 31 Mar 2025 20:54:56 -0700 Subject: [PATCH 135/922] fix: Fix Python 3.4.x compatibilty with bootstrap (#2709) (#2714) Fixes some f-strings, trailing commas, and out-of-order argument unpacking in the bootstrap template to restore compatibility with Python 3.4.x. --- python/private/python_bootstrap_template.txt | 10 ++++------ 1 file changed, 4 insertions(+), 6 deletions(-) diff --git a/python/private/python_bootstrap_template.txt b/python/private/python_bootstrap_template.txt index 9f671ddda5..babff075b5 100644 --- a/python/private/python_bootstrap_template.txt +++ b/python/private/python_bootstrap_template.txt @@ -95,19 +95,17 @@ def print_verbose(*args, mapping=None, values=None): for key, value in sorted((mapping or {}).items()): print( "bootstrap:", - *args, - f"{key}={value!r}", + *(list(args) + ["{}={}".format(key, repr(value))]), file=sys.stderr, - flush=True, + flush=True ) elif values is not None: for i, v in enumerate(values): print( "bootstrap:", - *args, - f"[{i}] {v!r}", + *(list(args) + ["[{}] {}".format(i, repr(v))]), file=sys.stderr, - flush=True, + flush=True ) else: print("bootstrap:", *args, file=sys.stderr, flush=True) From 7d431d84c43b0251485b4e3ba3be76aa9b140775 Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Tue, 1 Apr 2025 12:56:26 +0900 Subject: [PATCH 136/922] build(deps): bump packaging from 24.1 to 24.2 in /docs (#2721) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Bumps [packaging](https://github.com/pypa/packaging) from 24.1 to 24.2.
Release notes

Sourced from packaging's releases.

24.2

What's Changed

New Contributors

Full Changelog: https://github.com/pypa/packaging/compare/24.1...24.2

Changelog

Sourced from packaging's changelog.

24.2 - 2024-11-08


* PEP 639: Implement License-Expression and License-File (:issue:`828`)
* Use ``!r`` formatter for error messages with filenames (:issue:`844`)
* Add support for PEP 730 iOS tags (:issue:`832`)
* Fix prerelease detection for ``>`` and ``<`` (:issue:`794`)
* Fix uninformative error message (:issue:`830`)
* Refactor ``canonicalize_version`` (:issue:`793`)
* Patch python_full_version unconditionally (:issue:`825`)
* Fix doc for ``canonicalize_version`` to mention
``strip_trailing_zero`` and a typo in a docstring (:issue:`801`)
* Fix typo in Version ``__str__`` (:issue:`817`)
* Support creating a ``SpecifierSet`` from an iterable of ``Specifier``
objects (:issue:`775`)
Commits
  • d8e3b31 Bump for release
  • 2de393d Update changelog for release
  • 9c66f5c Remove extraneous quotes in f-strings by using !r (#848)
  • 4dc334c Upgrade to latest mypy (#853)
  • d1a9f93 Bump the github-actions group with 4 updates (#852)
  • 029f415 PEP 639: Implement License-Expression and License-File (#828)
  • 6c338a8 Use !r formatter for error messages with filenames. (#844)
  • 28e7da7 Add a comment as to why Metadata.name isn't normalized (#842)
  • ce0d79c Mention updating changelog in release process (#841)
  • ac5bdf3 Update the changelog to reflect 24.1 changes (#840)
  • Additional commits viewable in compare view

[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=packaging&package-manager=pip&previous-version=24.1&new-version=24.2)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) ---
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot show ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> --- docs/requirements.txt | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/requirements.txt b/docs/requirements.txt index 7e62e94fab..e838daca8f 100644 --- a/docs/requirements.txt +++ b/docs/requirements.txt @@ -236,9 +236,9 @@ myst-parser==4.0.0 \ --hash=sha256:851c9dfb44e36e56d15d05e72f02b80da21a9e0d07cba96baf5e2d476bb91531 \ --hash=sha256:b9317997552424448c6096c2558872fdb6f81d3ecb3a40ce84a7518798f3f28d # via rules-python-docs (docs/pyproject.toml) -packaging==24.1 \ - --hash=sha256:026ed72c8ed3fcce5bf8950572258698927fd1dbda10a5e981cdf0ac37f4f002 \ - --hash=sha256:5b8f2217dbdbd2f7f384c41c628544e6d52f2d0f53c6d0c3ea61aa5d1d7ff124 +packaging==24.2 \ + --hash=sha256:09abb1bccd265c01f4a3aa3f7a7db064b36514d2cba19a2f694fe6150451a759 \ + --hash=sha256:c228a6dc5e932d346bc5739379109d49e8853dd8223571c7c5b55260edc0b97f # via # readthedocs-sphinx-ext # sphinx From 24b9c51fa669d15dec0dd05ebe1ef60e4b9112be Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Tue, 1 Apr 2025 02:07:44 -0700 Subject: [PATCH 137/922] chore: remove semantics.bzl (#2725) semantics.bzl is an artifact of how the rules avoided patching when they were part of Bazel. With the code moved out of Bazel, such helper files aren't necessary anymore. Work towards https://github.com/bazel-contrib/rules_python/issues/2522 --- python/private/BUILD.bazel | 9 -------- python/private/attributes.bzl | 11 +--------- python/private/py_executable.bzl | 36 ++++++++++---------------------- python/private/semantics.bzl | 31 --------------------------- 4 files changed, 12 insertions(+), 75 deletions(-) delete mode 100644 python/private/semantics.bzl diff --git a/python/private/BUILD.bazel b/python/private/BUILD.bazel index 0f6668fa93..ef4580e1ce 100644 --- a/python/private/BUILD.bazel +++ b/python/private/BUILD.bazel @@ -72,7 +72,6 @@ bzl_library( ":py_internal_bzl", ":reexports_bzl", ":rules_cc_srcs_bzl", - ":semantics_bzl", "@bazel_skylib//rules:common_settings", ], ) @@ -131,7 +130,6 @@ bzl_library( ":py_internal_bzl", ":reexports_bzl", ":rules_cc_srcs_bzl", - ":semantics_bzl", "@bazel_skylib//lib:paths", ], ) @@ -302,7 +300,6 @@ bzl_library( ":attributes_bzl", ":py_executable_bzl", ":rule_builders_bzl", - ":semantics_bzl", "@bazel_skylib//lib:dicts", ], ) @@ -537,7 +534,6 @@ bzl_library( ":common_bzl", ":py_executable_bzl", ":rule_builders_bzl", - ":semantics_bzl", "@bazel_skylib//lib:dicts", ], ) @@ -677,11 +673,6 @@ bzl_library( ], ) -bzl_library( - name = "semantics_bzl", - srcs = ["semantics.bzl"], -) - # Needed to define bzl_library targets for docgen. (We don't define the # bzl_library target here because it'd give our users a transitive dependency # on Skylib.) diff --git a/python/private/attributes.bzl b/python/private/attributes.bzl index b57e275406..b042b3db6a 100644 --- a/python/private/attributes.bzl +++ b/python/private/attributes.bzl @@ -23,11 +23,6 @@ load(":py_info.bzl", "PyInfo") load(":py_internal.bzl", "py_internal") load(":reexports.bzl", "BuiltinPyInfo") load(":rule_builders.bzl", "ruleb") -load( - ":semantics.bzl", - "DEPS_ATTR_ALLOW_RULES", - "SRCS_ATTR_ALLOW_FILES", -) _PackageSpecificationInfo = getattr(py_internal, "PackageSpecificationInfo", None) @@ -250,9 +245,6 @@ PY_SRCS_ATTRS = dicts.add( [PyInfo], [CcInfo], ] + _MaybeBuiltinPyInfo, - # TODO(b/228692666): Google-specific; remove these allowances once - # the depot is cleaned up. - allow_rules = DEPS_ATTR_ALLOW_RULES, doc = """ List of additional libraries to be linked in to the target. See comments about @@ -359,8 +351,7 @@ as part of a runnable program (packaging rules may include them, however). allow_files = True, ), "srcs": lambda: attrb.LabelList( - # Google builds change the set of allowed files. - allow_files = SRCS_ATTR_ALLOW_FILES, + allow_files = [".py", ".py3"], # Necessary for --compile_one_dependency to work. flags = ["DIRECT_COMPILE_TIME_INPUT"], doc = """ diff --git a/python/private/py_executable.bzl b/python/private/py_executable.bzl index d54a3d7f24..fed46ab223 100644 --- a/python/private/py_executable.bzl +++ b/python/private/py_executable.bzl @@ -59,13 +59,6 @@ load(":py_internal.bzl", "py_internal") load(":py_runtime_info.bzl", "DEFAULT_STUB_SHEBANG", "PyRuntimeInfo") load(":reexports.bzl", "BuiltinPyInfo", "BuiltinPyRuntimeInfo") load(":rule_builders.bzl", "ruleb") -load( - ":semantics.bzl", - "ALLOWED_MAIN_EXTENSIONS", - "BUILD_DATA_SYMLINK_PATH", - "IS_BAZEL", - "PY_RUNTIME_ATTR_NAME", -) load( ":toolchain_types.bzl", "EXEC_TOOLS_TOOLCHAIN_TYPE", @@ -1116,19 +1109,12 @@ def _get_runtime_details(ctx, semantics): # # TOOD(bazelbuild/bazel#7901): Remove this once --python_path flag is removed. - if IS_BAZEL: - flag_interpreter_path = ctx.fragments.bazel_py.python_path - toolchain_runtime, effective_runtime = _maybe_get_runtime_from_ctx(ctx) - if not effective_runtime: - # Clear these just in case - toolchain_runtime = None - effective_runtime = None - - else: # Google code path - flag_interpreter_path = None - toolchain_runtime, effective_runtime = _maybe_get_runtime_from_ctx(ctx) - if not effective_runtime: - fail("Unable to find Python runtime") + flag_interpreter_path = ctx.fragments.bazel_py.python_path + toolchain_runtime, effective_runtime = _maybe_get_runtime_from_ctx(ctx) + if not effective_runtime: + # Clear these just in case + toolchain_runtime = None + effective_runtime = None if effective_runtime: direct = [] # List of files @@ -1207,7 +1193,7 @@ def _maybe_get_runtime_from_ctx(ctx): effective_runtime = toolchain_runtime else: toolchain_runtime = None - attr_target = getattr(ctx.attr, PY_RUNTIME_ATTR_NAME) + attr_target = ctx.attr._py_interpreter # In Bazel, --python_top is null by default. if attr_target and PyRuntimeInfo in attr_target: @@ -1335,9 +1321,9 @@ def _create_runfiles_with_build_data( central_uncachable_version_file, extra_write_build_data_env, ) - build_data_runfiles = ctx.runfiles(symlinks = { - BUILD_DATA_SYMLINK_PATH: build_data_file, - }) + build_data_runfiles = ctx.runfiles(files = [ + build_data_file, + ]) return build_data_file, build_data_runfiles def _write_build_data(ctx, central_uncachable_version_file, extra_write_build_data_env): @@ -1552,7 +1538,7 @@ def determine_main(ctx): """ if ctx.attr.main: proposed_main = ctx.attr.main.label.name - if not proposed_main.endswith(tuple(ALLOWED_MAIN_EXTENSIONS)): + if not proposed_main.endswith(".py"): fail("main must end in '.py'") else: if ctx.label.name.endswith(".py"): diff --git a/python/private/semantics.bzl b/python/private/semantics.bzl deleted file mode 100644 index 3811b17414..0000000000 --- a/python/private/semantics.bzl +++ /dev/null @@ -1,31 +0,0 @@ -# Copyright 2022 The Bazel Authors. All rights reserved. -# -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. -"""Contains constants that vary between Bazel and Google-internal""" - -IMPORTS_ATTR_SUPPORTED = True - -SRCS_ATTR_ALLOW_FILES = [".py", ".py3"] - -DEPS_ATTR_ALLOW_RULES = None - -PY_RUNTIME_ATTR_NAME = "_py_interpreter" - -BUILD_DATA_SYMLINK_PATH = None - -IS_BAZEL = True - -NATIVE_RULES_MIGRATION_HELP_URL = "https://github.com/bazelbuild/bazel/issues/17773" -NATIVE_RULES_MIGRATION_FIX_CMD = "add_python_loads" - -ALLOWED_MAIN_EXTENSIONS = [".py"] From ca91cea20a19a73ad81eccd4a497b72acc842633 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Tue, 1 Apr 2025 02:09:59 -0700 Subject: [PATCH 138/922] chore: remove defunct comment about py2 compatibility (#2724) The comment in the bootstrap about requiring compatibility with older Python versions is defunct and outdated. Python 2 support was dropped years ago. While compatibility with older Python versions is best effort for the system_python bootstrap, Python 2 doesn't need to be supported --- python/private/python_bootstrap_template.txt | 6 ------ 1 file changed, 6 deletions(-) diff --git a/python/private/python_bootstrap_template.txt b/python/private/python_bootstrap_template.txt index babff075b5..eb5595f4a1 100644 --- a/python/private/python_bootstrap_template.txt +++ b/python/private/python_bootstrap_template.txt @@ -1,11 +1,5 @@ %shebang% -# This script must retain compatibility with a wide variety of Python versions -# since it is run for every py_binary target. Currently we guarantee support -# going back to Python 2.7, and try to support even Python 2.6 on a best-effort -# basis. We might abandon 2.6 support once users have the ability to control the -# above shebang string via the Python toolchain (#8685). - from __future__ import absolute_import from __future__ import division from __future__ import print_function From 965dd51065e0a9bebd157518b19a2b1bb5f24321 Mon Sep 17 00:00:00 2001 From: Yuji Wang <146617342+Yanpei-Wang@users.noreply.github.com> Date: Wed, 2 Apr 2025 22:57:41 +0800 Subject: [PATCH 139/922] feat(pypi/parse_requirements): get dists by version when no hash provied (#2695) This pull request modifies the SimpleAPI HTML parsing to add a new field where we can get the `sha256` values by package version. This allows us to very easily fallback to all packages of a particular version when using `experimental_index_url` if the hashes are not specified. The code deciding which packages to query the SimpleAPI for has been also modified to only omit queries for packages that are included via direct URL references. If we fail to get the data from the SimpleAPI, we will fallback to `pip` and try to install it via the legacy behaviour. Fixes #2023 Work towards #260 Work towards #1357 Work towards #2363 --------- Co-authored-by: Ignas Anikevicius <240938+aignas@users.noreply.github.com> --- CHANGELOG.md | 8 + docs/pypi-dependencies.md | 12 +- python/private/pypi/extension.bzl | 27 ++- python/private/pypi/parse_requirements.bzl | 15 +- python/private/pypi/parse_simpleapi_html.bzl | 35 +++- python/private/pypi/simpleapi_download.bzl | 15 +- python/private/pypi/whl_library.bzl | 6 + python/private/pypi/whl_repo_name.bzl | 17 +- tests/pypi/extension/extension_tests.bzl | 159 +++++++++++++----- .../parse_requirements_tests.bzl | 60 +++++++ .../parse_simpleapi_html_tests.bzl | 30 +++- .../simpleapi_download_tests.bzl | 5 +- .../whl_repo_name/whl_repo_name_tests.bzl | 12 ++ 13 files changed, 331 insertions(+), 70 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 5974a656a6..bbcf2561c8 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -81,6 +81,14 @@ Unreleased changes template. {#v0-0-0-added} ### Added +* (pypi) From now on `sha256` values in the `requirements.txt` is no longer + mandatory when enabling {attr}`pip.parse.experimental_index_url` feature. + This means that `rules_python` will attempt to fetch metadata for all + packages through SimpleAPI unless they are pulled through direct URL + references. Fixes [#2023](https://github.com/bazel-contrib/rules_python/issues/2023). + In case you see issues with `rules_python` being too eager to fetch the SimpleAPI + metadata, you can use the newly added {attr}`pip.parse.experimental_skip_sources` + to skip metadata fetching for those packages. * (uv) A {obj}`lock` rule that is the replacement for the {obj}`compile_pip_requirements`. This may still have rough corners so please report issues with it in the diff --git a/docs/pypi-dependencies.md b/docs/pypi-dependencies.md index 039200dfd4..6cc0da6cb4 100644 --- a/docs/pypi-dependencies.md +++ b/docs/pypi-dependencies.md @@ -386,11 +386,13 @@ This does not mean that `rules_python` is fetching the wheels eagerly, but it rather means that it is calling the PyPI server to get the Simple API response to get the list of all available source and wheel distributions. Once it has got all of the available distributions, it will select the right ones depending -on the `sha256` values in your `requirements_lock.txt` file. The compatible -distribution URLs will be then written to the `MODULE.bazel.lock` file. Currently -users wishing to use the lock file with `rules_python` with this feature have -to set an environment variable `RULES_PYTHON_OS_ARCH_LOCK_FILE=0` which will -become default in the next release. +on the `sha256` values in your `requirements_lock.txt` file. If `sha256` hashes +are not present in the requirements file, we will fallback to matching by version +specified in the lock file. The compatible distribution URLs will be then +written to the `MODULE.bazel.lock` file. Currently users wishing to use the +lock file with `rules_python` with this feature have to set an environment +variable `RULES_PYTHON_OS_ARCH_LOCK_FILE=0` which will become default in the +next release. Fetching the distribution information from the PyPI allows `rules_python` to know which `whl` should be used on which target platform and it will determine diff --git a/python/private/pypi/extension.bzl b/python/private/pypi/extension.bzl index 490bd05f11..f782e69a45 100644 --- a/python/private/pypi/extension.bzl +++ b/python/private/pypi/extension.bzl @@ -459,13 +459,21 @@ You cannot use both the additive_build_content and additive_build_content_file a get_index_urls = None if pip_attr.experimental_index_url: is_reproducible = False + skip_sources = [ + normalize_name(s) + for s in pip_attr.simpleapi_skip + ] get_index_urls = lambda ctx, distributions: simpleapi_download( ctx, attr = struct( index_url = pip_attr.experimental_index_url, extra_index_urls = pip_attr.experimental_extra_index_urls or [], index_url_overrides = pip_attr.experimental_index_url_overrides or {}, - sources = distributions, + sources = [ + d + for d in distributions + if normalize_name(d) not in skip_sources + ], envsubst = pip_attr.envsubst, # Auth related info netrc = pip_attr.netrc, @@ -682,6 +690,11 @@ This is equivalent to `--index-url` `pip` option. If {attr}`download_only` is set, then `sdist` archives will be discarded and `pip.parse` will operate in wheel-only mode. ::: + +:::{versionchanged} VERSION_NEXT_FEATURE +Index metadata will be used to deduct `sha256` values for packages even if the +`sha256` values are not present in the requirements.txt lock file. +::: """, ), "experimental_index_url_overrides": attr.string_dict( @@ -749,6 +762,18 @@ The Python version the dependencies are targetting, in Major.Minor format If an interpreter isn't explicitly provided (using `python_interpreter` or `python_interpreter_target`), then the version specified here must have a corresponding `python.toolchain()` configured. +""", + ), + "simpleapi_skip": attr.string_list( + doc = """\ +The list of packages to skip fetching metadata for from SimpleAPI index. You should +normally not need this attribute, but in case you do, please report this as a bug +to `rules_python` and use this attribute until the bug is fixed. + +EXPERIMENTAL: this may be removed without notice. + +:::{versionadded} VERSION_NEXT_FEATURE +::: """, ), "whl_modifications": attr.label_keyed_string_dict( diff --git a/python/private/pypi/parse_requirements.bzl b/python/private/pypi/parse_requirements.bzl index 7aadc15eac..3280ce8df1 100644 --- a/python/private/pypi/parse_requirements.bzl +++ b/python/private/pypi/parse_requirements.bzl @@ -184,7 +184,7 @@ def parse_requirements( req.distribution: None for reqs in requirements_by_platform.values() for req in reqs.values() - if req.srcs.shas + if not req.srcs.url }), ) @@ -315,10 +315,15 @@ def _add_dists(*, requirement, index_urls, logger = None): whls = [] sdist = None - # TODO @aignas 2024-05-22: it is in theory possible to add all - # requirements by version instead of by sha256. This may be useful - # for some projects. - for sha256 in requirement.srcs.shas: + # First try to find distributions by SHA256 if provided + shas_to_use = requirement.srcs.shas + if not shas_to_use: + version = requirement.srcs.version + shas_to_use = index_urls.sha256s_by_version.get(version, []) + if logger: + logger.warn(lambda: "requirement file has been generated without hashes, will use all hashes for the given version {} that could find on the index:\n {}".format(version, shas_to_use)) + + for sha256 in shas_to_use: # For now if the artifact is marked as yanked we just ignore it. # # See https://packaging.python.org/en/latest/specifications/simple-repository-api/#adding-yank-support-to-the-simple-api diff --git a/python/private/pypi/parse_simpleapi_html.bzl b/python/private/pypi/parse_simpleapi_html.bzl index e549e76181..8c6f739fe3 100644 --- a/python/private/pypi/parse_simpleapi_html.bzl +++ b/python/private/pypi/parse_simpleapi_html.bzl @@ -26,6 +26,7 @@ def parse_simpleapi_html(*, url, content): Returns: A list of structs with: * filename: The filename of the artifact. + * version: The version of the artifact. * url: The URL to download the artifact. * sha256: The sha256 of the artifact. * metadata_sha256: The whl METADATA sha256 if we can download it. If this is @@ -51,8 +52,11 @@ def parse_simpleapi_html(*, url, content): # Each line follows the following pattern # filename
+ sha256_by_version = {} for line in lines[1:]: dist_url, _, tail = line.partition("#sha256=") + dist_url = _absolute_url(url, dist_url) + sha256, _, tail = tail.partition("\"") # See https://packaging.python.org/en/latest/specifications/simple-repository-api/#adding-yank-support-to-the-simple-api @@ -60,6 +64,8 @@ def parse_simpleapi_html(*, url, content): head, _, _ = tail.rpartition("") maybe_metadata, _, filename = head.rpartition(">") + version = _version(filename) + sha256_by_version.setdefault(version, []).append(sha256) metadata_sha256 = "" metadata_url = "" @@ -75,7 +81,8 @@ def parse_simpleapi_html(*, url, content): if filename.endswith(".whl"): whls[sha256] = struct( filename = filename, - url = _absolute_url(url, dist_url), + version = version, + url = dist_url, sha256 = sha256, metadata_sha256 = metadata_sha256, metadata_url = _absolute_url(url, metadata_url) if metadata_url else "", @@ -84,7 +91,8 @@ def parse_simpleapi_html(*, url, content): else: sdists[sha256] = struct( filename = filename, - url = _absolute_url(url, dist_url), + version = version, + url = dist_url, sha256 = sha256, metadata_sha256 = "", metadata_url = "", @@ -94,8 +102,31 @@ def parse_simpleapi_html(*, url, content): return struct( sdists = sdists, whls = whls, + sha256_by_version = sha256_by_version, ) +_SDIST_EXTS = [ + ".tar", # handles any compression + ".zip", +] + +def _version(filename): + # See https://packaging.python.org/en/latest/specifications/binary-distribution-format/#binary-distribution-format + + _, _, tail = filename.partition("-") + version, _, _ = tail.partition("-") + if version != tail: + # The format is {name}-{version}-{whl_specifiers}.whl + return version + + # NOTE @aignas 2025-03-29: most of the files are wheels, so this is not the common path + + # {name}-{version}.{ext} + for ext in _SDIST_EXTS: + version, _, _ = version.partition(ext) # build or name + + return version + def _get_root_directory(url): scheme_end = url.find("://") if scheme_end == -1: diff --git a/python/private/pypi/simpleapi_download.bzl b/python/private/pypi/simpleapi_download.bzl index ef39fb8723..e8d7d0941a 100644 --- a/python/private/pypi/simpleapi_download.bzl +++ b/python/private/pypi/simpleapi_download.bzl @@ -127,10 +127,17 @@ def simpleapi_download( failed_sources = [pkg for pkg in attr.sources if pkg not in found_on_index] if failed_sources: - _fail("Failed to download metadata for {} for from urls: {}".format( - failed_sources, - index_urls, - )) + _fail( + "\n".join([ + "Failed to download metadata for {} for from urls: {}.".format( + failed_sources, + index_urls, + ), + "If you would like to skip downloading metadata for these packages please add 'simpleapi_skip={}' to your 'pip.parse' call.".format( + render.list(failed_sources), + ), + ]), + ) return None if warn_overrides: diff --git a/python/private/pypi/whl_library.bzl b/python/private/pypi/whl_library.bzl index 38ac9dcd92..2904f85f1b 100644 --- a/python/private/pypi/whl_library.bzl +++ b/python/private/pypi/whl_library.bzl @@ -270,6 +270,12 @@ def _whl_library_impl(rctx): sha256 = rctx.attr.sha256, auth = get_auth(rctx, urls), ) + if not rctx.attr.sha256: + # this is only seen when there is a direct URL reference without sha256 + logger.warn("Please update the requirement line to include the hash:\n{} \\\n --hash=sha256:{}".format( + rctx.attr.requirement, + result.sha256, + )) if not result.success: fail("could not download the '{}' from {}:\n{}".format(filename, urls, result)) diff --git a/python/private/pypi/whl_repo_name.bzl b/python/private/pypi/whl_repo_name.bzl index 48bbd1a9b2..02a7c8142c 100644 --- a/python/private/pypi/whl_repo_name.bzl +++ b/python/private/pypi/whl_repo_name.bzl @@ -32,11 +32,19 @@ def whl_repo_name(filename, sha256): if not filename.endswith(".whl"): # Then the filename is basically foo-3.2.1. - parts.append(normalize_name(filename.rpartition("-")[0])) - parts.append("sdist") + name, _, tail = filename.rpartition("-") + parts.append(normalize_name(name)) + if sha256: + parts.append("sdist") + version = "" + else: + for ext in [".tar", ".zip"]: + tail, _, _ = tail.partition(ext) + version = tail.replace(".", "_").replace("!", "_") else: parsed = parse_whl_name(filename) name = normalize_name(parsed.distribution) + version = parsed.version.replace(".", "_").replace("!", "_") python_tag, _, _ = parsed.python_tag.partition(".") abi_tag, _, _ = parsed.abi_tag.partition(".") platform_tag, _, _ = parsed.platform_tag.partition(".") @@ -46,7 +54,10 @@ def whl_repo_name(filename, sha256): parts.append(abi_tag) parts.append(platform_tag) - parts.append(sha256[:8]) + if sha256: + parts.append(sha256[:8]) + elif version: + parts.insert(1, version) return "_".join(parts) diff --git a/tests/pypi/extension/extension_tests.bzl b/tests/pypi/extension/extension_tests.bzl index 858c026df8..3a91c7b108 100644 --- a/tests/pypi/extension/extension_tests.bzl +++ b/tests/pypi/extension/extension_tests.bzl @@ -100,6 +100,7 @@ def _parse( requirements_linux = None, requirements_lock = None, requirements_windows = None, + simpleapi_skip = [], timeout = 600, whl_modifications = {}, **kwargs): @@ -135,6 +136,7 @@ def _parse( experimental_extra_index_urls = [], parallel_download = False, experimental_index_url_overrides = {}, + simpleapi_skip = simpleapi_skip, **kwargs ) @@ -616,6 +618,21 @@ def _test_simple_get_index(env): ), }, ), + "some_other_pkg": struct( + whls = { + "deadb33f": struct( + yanked = False, + filename = "some-other-pkg-0.0.1-py3-none-any.whl", + sha256 = "deadb33f", + url = "example2.org/index/some_other_pkg/", + ), + }, + sdists = {}, + sha256s_by_version = { + "0.0.1": ["deadb33f"], + "0.0.3": ["deadbeef"], + }, + ), } pypi = _parse_modules( @@ -640,7 +657,11 @@ def _test_simple_get_index(env): simple==0.0.1 \ --hash=sha256:deadbeef \ --hash=sha256:deadb00f -some_pkg==0.0.1 +some_pkg==0.0.1 @ example-direct.org/some_pkg-0.0.1-py3-none-any.whl \ + --hash=sha256:deadbaaf +direct_without_sha==0.0.1 @ example-direct.org/direct_without_sha-0.0.1-py3-none-any.whl +some_other_pkg==0.0.1 +pip_fallback==0.0.1 """, }[x], ), @@ -651,42 +672,91 @@ some_pkg==0.0.1 ) pypi.is_reproducible().equals(False) - pypi.exposed_packages().contains_exactly({"pypi": ["simple", "some_pkg"]}) + pypi.exposed_packages().contains_exactly({"pypi": ["direct_without_sha", "pip_fallback", "simple", "some_other_pkg", "some_pkg"]}) pypi.hub_group_map().contains_exactly({"pypi": {}}) pypi.hub_whl_map().contains_exactly({ "pypi": { + "direct_without_sha": { + "pypi_315_direct_without_sha_0_0_1_py3_none_any": [ + struct( + config_setting = None, + filename = "direct_without_sha-0.0.1-py3-none-any.whl", + target_platforms = None, + version = "3.15", + ), + ], + }, + "pip_fallback": { + "pypi_315_pip_fallback": [ + struct( + config_setting = None, + filename = None, + target_platforms = None, + version = "3.15", + ), + ], + }, "simple": { "pypi_315_simple_py3_none_any_deadb00f": [ - whl_config_setting( + struct( + config_setting = None, filename = "simple-0.0.1-py3-none-any.whl", + target_platforms = None, version = "3.15", ), ], "pypi_315_simple_sdist_deadbeef": [ - whl_config_setting( + struct( + config_setting = None, filename = "simple-0.0.1.tar.gz", + target_platforms = None, + version = "3.15", + ), + ], + }, + "some_other_pkg": { + "pypi_315_some_py3_none_any_deadb33f": [ + struct( + config_setting = None, + filename = "some-other-pkg-0.0.1-py3-none-any.whl", + target_platforms = None, version = "3.15", ), ], }, "some_pkg": { - "pypi_315_some_pkg": [whl_config_setting(version = "3.15")], + "pypi_315_some_pkg_py3_none_any_deadbaaf": [ + struct( + config_setting = None, + filename = "some_pkg-0.0.1-py3-none-any.whl", + target_platforms = None, + version = "3.15", + ), + ], }, }, }) pypi.whl_libraries().contains_exactly({ + "pypi_315_direct_without_sha_0_0_1_py3_none_any": { + "dep_template": "@pypi//{name}:{target}", + "experimental_target_platforms": ["cp315_linux_aarch64", "cp315_linux_arm", "cp315_linux_ppc", "cp315_linux_s390x", "cp315_linux_x86_64", "cp315_osx_aarch64", "cp315_osx_x86_64", "cp315_windows_x86_64"], + "filename": "direct_without_sha-0.0.1-py3-none-any.whl", + "python_interpreter_target": "unit_test_interpreter_target", + "repo": "pypi_315", + "requirement": "direct_without_sha==0.0.1 @ example-direct.org/direct_without_sha-0.0.1-py3-none-any.whl", + "sha256": "", + "urls": ["example-direct.org/direct_without_sha-0.0.1-py3-none-any.whl"], + }, + "pypi_315_pip_fallback": { + "dep_template": "@pypi//{name}:{target}", + "extra_pip_args": ["--extra-args-for-sdist-building"], + "python_interpreter_target": "unit_test_interpreter_target", + "repo": "pypi_315", + "requirement": "pip_fallback==0.0.1", + }, "pypi_315_simple_py3_none_any_deadb00f": { "dep_template": "@pypi//{name}:{target}", - "experimental_target_platforms": [ - "cp315_linux_aarch64", - "cp315_linux_arm", - "cp315_linux_ppc", - "cp315_linux_s390x", - "cp315_linux_x86_64", - "cp315_osx_aarch64", - "cp315_osx_x86_64", - "cp315_windows_x86_64", - ], + "experimental_target_platforms": ["cp315_linux_aarch64", "cp315_linux_arm", "cp315_linux_ppc", "cp315_linux_s390x", "cp315_linux_x86_64", "cp315_osx_aarch64", "cp315_osx_x86_64", "cp315_windows_x86_64"], "filename": "simple-0.0.1-py3-none-any.whl", "python_interpreter_target": "unit_test_interpreter_target", "repo": "pypi_315", @@ -696,16 +766,7 @@ some_pkg==0.0.1 }, "pypi_315_simple_sdist_deadbeef": { "dep_template": "@pypi//{name}:{target}", - "experimental_target_platforms": [ - "cp315_linux_aarch64", - "cp315_linux_arm", - "cp315_linux_ppc", - "cp315_linux_s390x", - "cp315_linux_x86_64", - "cp315_osx_aarch64", - "cp315_osx_x86_64", - "cp315_windows_x86_64", - ], + "experimental_target_platforms": ["cp315_linux_aarch64", "cp315_linux_arm", "cp315_linux_ppc", "cp315_linux_s390x", "cp315_linux_x86_64", "cp315_osx_aarch64", "cp315_osx_x86_64", "cp315_windows_x86_64"], "extra_pip_args": ["--extra-args-for-sdist-building"], "filename": "simple-0.0.1.tar.gz", "python_interpreter_target": "unit_test_interpreter_target", @@ -714,29 +775,43 @@ some_pkg==0.0.1 "sha256": "deadbeef", "urls": ["example.org"], }, - # We are falling back to regular `pip` - "pypi_315_some_pkg": { + "pypi_315_some_pkg_py3_none_any_deadbaaf": { "dep_template": "@pypi//{name}:{target}", - "extra_pip_args": ["--extra-args-for-sdist-building"], + "experimental_target_platforms": ["cp315_linux_aarch64", "cp315_linux_arm", "cp315_linux_ppc", "cp315_linux_s390x", "cp315_linux_x86_64", "cp315_osx_aarch64", "cp315_osx_x86_64", "cp315_windows_x86_64"], + "filename": "some_pkg-0.0.1-py3-none-any.whl", + "python_interpreter_target": "unit_test_interpreter_target", + "repo": "pypi_315", + "requirement": "some_pkg==0.0.1 @ example-direct.org/some_pkg-0.0.1-py3-none-any.whl --hash=sha256:deadbaaf", + "sha256": "deadbaaf", + "urls": ["example-direct.org/some_pkg-0.0.1-py3-none-any.whl"], + }, + "pypi_315_some_py3_none_any_deadb33f": { + "dep_template": "@pypi//{name}:{target}", + "experimental_target_platforms": ["cp315_linux_aarch64", "cp315_linux_arm", "cp315_linux_ppc", "cp315_linux_s390x", "cp315_linux_x86_64", "cp315_osx_aarch64", "cp315_osx_x86_64", "cp315_windows_x86_64"], + "filename": "some-other-pkg-0.0.1-py3-none-any.whl", "python_interpreter_target": "unit_test_interpreter_target", "repo": "pypi_315", - "requirement": "some_pkg==0.0.1", + "requirement": "some_other_pkg==0.0.1", + "sha256": "deadb33f", + "urls": ["example2.org/index/some_other_pkg/"], }, }) pypi.whl_mods().contains_exactly({}) - env.expect.that_dict(got_simpleapi_download_kwargs).contains_exactly({ - "attr": struct( - auth_patterns = {}, - envsubst = {}, - extra_index_urls = [], - index_url = "pypi.org", - index_url_overrides = {}, - netrc = None, - sources = ["simple"], - ), - "cache": {}, - "parallel_download": False, - }) + env.expect.that_dict(got_simpleapi_download_kwargs).contains_exactly( + { + "attr": struct( + auth_patterns = {}, + envsubst = {}, + extra_index_urls = [], + index_url = "pypi.org", + index_url_overrides = {}, + netrc = None, + sources = ["simple", "pip_fallback", "some_other_pkg"], + ), + "cache": {}, + "parallel_download": False, + }, + ) _tests.append(_test_simple_get_index) diff --git a/tests/pypi/parse_requirements/parse_requirements_tests.bzl b/tests/pypi/parse_requirements/parse_requirements_tests.bzl index 7bbd696afa..c50482127b 100644 --- a/tests/pypi/parse_requirements/parse_requirements_tests.bzl +++ b/tests/pypi/parse_requirements/parse_requirements_tests.bzl @@ -61,6 +61,10 @@ foo[extra]==0.0.1 --hash=sha256:deadbeef "requirements_marker": """\ foo[extra]==0.0.1 ;marker --hash=sha256:deadbeef bar==0.0.1 --hash=sha256:deadbeef +""", + "requirements_optional_hash": """ +foo==0.0.4 @ https://example.org/foo-0.0.4.whl +foo==0.0.5 @ https://example.org/foo-0.0.5.whl --hash=sha256:deadbeef """, "requirements_osx": """\ foo==0.0.3 --hash=sha256:deadbaaf @@ -563,6 +567,62 @@ def _test_different_package_version(env): _tests.append(_test_different_package_version) +def _test_optional_hash(env): + got = parse_requirements( + ctx = _mock_ctx(), + requirements_by_platform = { + "requirements_optional_hash": ["linux_x86_64"], + }, + ) + env.expect.that_dict(got).contains_exactly({ + "foo": [ + struct( + distribution = "foo", + extra_pip_args = [], + sdist = None, + is_exposed = True, + srcs = struct( + marker = "", + requirement = "foo==0.0.4 @ https://example.org/foo-0.0.4.whl", + requirement_line = "foo==0.0.4 @ https://example.org/foo-0.0.4.whl", + shas = [], + version = "0.0.4", + url = "https://example.org/foo-0.0.4.whl", + ), + target_platforms = ["linux_x86_64"], + whls = [struct( + url = "https://example.org/foo-0.0.4.whl", + filename = "foo-0.0.4.whl", + sha256 = "", + yanked = False, + )], + ), + struct( + distribution = "foo", + extra_pip_args = [], + sdist = None, + is_exposed = True, + srcs = struct( + marker = "", + requirement = "foo==0.0.5 @ https://example.org/foo-0.0.5.whl --hash=sha256:deadbeef", + requirement_line = "foo==0.0.5 @ https://example.org/foo-0.0.5.whl --hash=sha256:deadbeef", + shas = ["deadbeef"], + version = "0.0.5", + url = "https://example.org/foo-0.0.5.whl", + ), + target_platforms = ["linux_x86_64"], + whls = [struct( + url = "https://example.org/foo-0.0.5.whl", + filename = "foo-0.0.5.whl", + sha256 = "deadbeef", + yanked = False, + )], + ), + ], + }) + +_tests.append(_test_optional_hash) + def parse_requirements_test_suite(name): """Create the test suite. diff --git a/tests/pypi/parse_simpleapi_html/parse_simpleapi_html_tests.bzl b/tests/pypi/parse_simpleapi_html/parse_simpleapi_html_tests.bzl index d3c42a8864..abaa7a6a49 100644 --- a/tests/pypi/parse_simpleapi_html/parse_simpleapi_html_tests.bzl +++ b/tests/pypi/parse_simpleapi_html/parse_simpleapi_html_tests.bzl @@ -52,13 +52,14 @@ def _test_sdist(env): 'data-requires-python=">=3.7"', ], filename = "foo-0.0.1.tar.gz", - url = "ignored", + url = "foo", ), struct( filename = "foo-0.0.1.tar.gz", sha256 = "deadbeefasource", url = "https://example.org/full-url/foo-0.0.1.tar.gz", yanked = False, + version = "0.0.1", ), ), ( @@ -68,12 +69,13 @@ def _test_sdist(env): 'data-requires-python=">=3.7"', ], filename = "foo-0.0.1.tar.gz", - url = "ignored", + url = "foo", ), struct( filename = "foo-0.0.1.tar.gz", sha256 = "deadbeefasource", url = "https://example.org/full-url/foo-0.0.1.tar.gz", + version = "0.0.1", yanked = False, ), ), @@ -94,12 +96,14 @@ def _test_sdist(env): sha256 = subjects.str, url = subjects.str, yanked = subjects.bool, + version = subjects.str, ), ) actual.filename().equals(want.filename) actual.sha256().equals(want.sha256) actual.url().equals(want.url) actual.yanked().equals(want.yanked) + actual.version().equals(want.version) _tests.append(_test_sdist) @@ -115,7 +119,7 @@ def _test_whls(env): 'data-core-metadata="sha256=deadb00f"', ], filename = "foo-0.0.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", - url = "ignored", + url = "foo", ), struct( filename = "foo-0.0.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", @@ -123,6 +127,7 @@ def _test_whls(env): metadata_url = "https://example.org/full-url/foo-0.0.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata", sha256 = "deadbeef", url = "https://example.org/full-url/foo-0.0.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", + version = "0.0.2", yanked = False, ), ), @@ -135,7 +140,7 @@ def _test_whls(env): 'data-core-metadata="sha256=deadb00f"', ], filename = "foo-0.0.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", - url = "ignored", + url = "foo", ), struct( filename = "foo-0.0.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", @@ -143,6 +148,7 @@ def _test_whls(env): metadata_url = "https://example.org/full-url/foo-0.0.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata", sha256 = "deadbeef", url = "https://example.org/full-url/foo-0.0.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", + version = "0.0.2", yanked = False, ), ), @@ -154,13 +160,14 @@ def _test_whls(env): 'data-core-metadata="sha256=deadb00f"', ], filename = "foo-0.0.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", - url = "ignored", + url = "foo", ), struct( filename = "foo-0.0.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", metadata_sha256 = "deadb00f", metadata_url = "https://example.org/full-url/foo-0.0.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata", sha256 = "deadbeef", + version = "0.0.2", url = "https://example.org/full-url/foo-0.0.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", yanked = False, ), @@ -173,13 +180,14 @@ def _test_whls(env): 'data-dist-info-metadata="sha256=deadb00f"', ], filename = "foo-0.0.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", - url = "ignored", + url = "foo", ), struct( filename = "foo-0.0.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", metadata_sha256 = "deadb00f", metadata_url = "https://example.org/full-url/foo-0.0.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata", sha256 = "deadbeef", + version = "0.0.2", url = "https://example.org/full-url/foo-0.0.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", yanked = False, ), @@ -191,7 +199,7 @@ def _test_whls(env): 'data-requires-python=">=3.7"', ], filename = "foo-0.0.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", - url = "ignored", + url = "foo", ), struct( filename = "foo-0.0.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", @@ -199,6 +207,7 @@ def _test_whls(env): metadata_url = "", sha256 = "deadbeef", url = "https://example.org/full-url/foo-0.0.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", + version = "0.0.2", yanked = False, ), ), @@ -217,6 +226,7 @@ def _test_whls(env): metadata_sha256 = "deadb00f", metadata_url = "https://example.org/python-wheels/foo-0.0.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata", sha256 = "deadbeef", + version = "0.0.2", url = "https://example.org/python-wheels/foo-0.0.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", yanked = False, ), @@ -235,6 +245,7 @@ def _test_whls(env): metadata_url = "", sha256 = "deadbeef", url = "https://download.pytorch.org/whl/torch-2.0.0-cp38-cp38-manylinux2014_aarch64.whl", + version = "2.0.0", yanked = False, ), ), @@ -252,6 +263,7 @@ def _test_whls(env): metadata_url = "", sha256 = "notdeadbeef", url = "http://download.pytorch.org/whl/torch-2.0.0-cp38-cp38-manylinux2014_aarch64.whl", + version = "2.0.0", yanked = False, ), ), @@ -267,6 +279,7 @@ def _test_whls(env): filename = "mypy_extensions-1.0.0-py3-none-any.whl", metadata_sha256 = "", metadata_url = "", + version = "1.0.0", sha256 = "deadbeef", url = "https://example.org/simple/mypy_extensions/1.0.0/mypy_extensions-1.0.0-py3-none-any.whl", yanked = False, @@ -285,6 +298,7 @@ def _test_whls(env): metadata_sha256 = "", metadata_url = "", sha256 = "deadbeef", + version = "1.0.0", url = "https://example.org/simple/mypy_extensions/unknown://example.com/mypy_extensions-1.0.0-py3-none-any.whl", yanked = False, ), @@ -308,6 +322,7 @@ def _test_whls(env): sha256 = subjects.str, url = subjects.str, yanked = subjects.bool, + version = subjects.str, ), ) actual.filename().equals(want.filename) @@ -316,6 +331,7 @@ def _test_whls(env): actual.sha256().equals(want.sha256) actual.url().equals(want.url) actual.yanked().equals(want.yanked) + actual.version().equals(want.version) _tests.append(_test_whls) diff --git a/tests/pypi/simpleapi_download/simpleapi_download_tests.bzl b/tests/pypi/simpleapi_download/simpleapi_download_tests.bzl index 964d3e25ea..ce214d6e34 100644 --- a/tests/pypi/simpleapi_download/simpleapi_download_tests.bzl +++ b/tests/pypi/simpleapi_download/simpleapi_download_tests.bzl @@ -110,7 +110,10 @@ def _test_fail(env): ) env.expect.that_collection(fails).contains_exactly([ - """Failed to download metadata for ["foo"] for from urls: ["main", "extra"]""", + """\ +Failed to download metadata for ["foo"] for from urls: ["main", "extra"]. +If you would like to skip downloading metadata for these packages please add 'simpleapi_skip=["foo"]' to your 'pip.parse' call.\ +""", ]) env.expect.that_collection(calls).contains_exactly([ "extra/foo/", diff --git a/tests/pypi/whl_repo_name/whl_repo_name_tests.bzl b/tests/pypi/whl_repo_name/whl_repo_name_tests.bzl index 000941b55b..f0d1d059e1 100644 --- a/tests/pypi/whl_repo_name/whl_repo_name_tests.bzl +++ b/tests/pypi/whl_repo_name/whl_repo_name_tests.bzl @@ -25,12 +25,24 @@ def _test_simple(env): _tests.append(_test_simple) +def _test_simple_no_sha(env): + got = whl_repo_name("foo-1.2.3-py3-none-any.whl", "") + env.expect.that_str(got).equals("foo_1_2_3_py3_none_any") + +_tests.append(_test_simple_no_sha) + def _test_sdist(env): got = whl_repo_name("foo-1.2.3.tar.gz", "deadbeef000deadbeef") env.expect.that_str(got).equals("foo_sdist_deadbeef") _tests.append(_test_sdist) +def _test_sdist_no_sha(env): + got = whl_repo_name("foo-1.2.3.tar.gz", "") + env.expect.that_str(got).equals("foo_1_2_3") + +_tests.append(_test_sdist_no_sha) + def _test_platform_whl(env): got = whl_repo_name( "foo-1.2.3-cp39.cp310-abi3-manylinux1_x86_64.manylinux_2_17_x86_64.whl", From 3d98aeea9c70b2a7336d9ea8f7397b5c6d07d405 Mon Sep 17 00:00:00 2001 From: Ignas Anikevicius <240938+aignas@users.noreply.github.com> Date: Sat, 5 Apr 2025 22:17:28 +0900 Subject: [PATCH 140/922] fix(toolchains): correctly order the toolchains (#2735) Since toolchain matching is done by matching the first target that matches target settings, the `minor_mapping` config setting is special, because e.g. all `3.11.X` toolchains match the `python_version = "3.11"` setting. This just reshuffles the list so that we have toolchains that are in the `minor_mapping` before the rest. At the same time remove the workaround from the `lock.bzl` where the bug was initially discovered. Fixes #2685 --- CHANGELOG.md | 3 + python/private/python.bzl | 17 +- python/uv/private/BUILD.bazel | 2 - python/uv/private/lock.bzl | 31 +-- .../transition/multi_version_tests.bzl | 3 +- tests/python/python_tests.bzl | 52 +++++ tests/toolchains/transitions/BUILD.bazel | 5 + .../transitions/transitions_tests.bzl | 182 ++++++++++++++++++ 8 files changed, 269 insertions(+), 26 deletions(-) create mode 100644 tests/toolchains/transitions/BUILD.bazel create mode 100644 tests/toolchains/transitions/transitions_tests.bzl diff --git a/CHANGELOG.md b/CHANGELOG.md index bbcf2561c8..b11270cb25 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -78,6 +78,9 @@ Unreleased changes template. * (toolchains) Do not try to run `chmod` when downloading non-windows hermetic toolchain repositories on Windows. Fixes [#2660](https://github.com/bazel-contrib/rules_python/issues/2660). +* (toolchains) The toolchain matching is has been fixed when writing + transitions transitioning on the `python_version` flag. + Fixes [#2685](https://github.com/bazel-contrib/rules_python/issues/2685). {#v0-0-0-added} ### Added diff --git a/python/private/python.bzl b/python/private/python.bzl index 44eb09f766..296fb0ab7d 100644 --- a/python/private/python.bzl +++ b/python/private/python.bzl @@ -243,10 +243,25 @@ def parse_modules(*, module_ctx, _fail = fail): if len(toolchains) > _MAX_NUM_TOOLCHAINS: fail("more than {} python versions are not supported".format(_MAX_NUM_TOOLCHAINS)) + # sort the toolchains so that the toolchain versions that are in the + # `minor_mapping` are coming first. This ensures that `python_version = + # "3.X"` transitions work as expected. + minor_version_toolchains = [] + other_toolchains = [] + minor_mapping = list(config.minor_mapping.values()) + for t in toolchains: + # FIXME @aignas 2025-04-04: How can we unit test that this ordering is + # consistent with what would actually work? + if config.minor_mapping.get(t.python_version, t.python_version) in minor_mapping: + minor_version_toolchains.append(t) + else: + other_toolchains.append(t) + toolchains = minor_version_toolchains + other_toolchains + return struct( config = config, debug_info = debug_info, - default_python_version = toolchains[-1].python_version, + default_python_version = default_toolchain.python_version, toolchains = [ struct( python_version = t.python_version, diff --git a/python/uv/private/BUILD.bazel b/python/uv/private/BUILD.bazel index d17ca39490..587ad9a0f9 100644 --- a/python/uv/private/BUILD.bazel +++ b/python/uv/private/BUILD.bazel @@ -43,10 +43,8 @@ bzl_library( ":toolchain_types_bzl", "//python:py_binary_bzl", "//python/private:bzlmod_enabled_bzl", - "//python/private:full_version_bzl", "//python/private:toolchain_types_bzl", "@bazel_skylib//lib:shell", - "@pythons_hub//:versions_bzl", ], ) diff --git a/python/uv/private/lock.bzl b/python/uv/private/lock.bzl index 69d277d653..45a3819ee6 100644 --- a/python/uv/private/lock.bzl +++ b/python/uv/private/lock.bzl @@ -16,10 +16,8 @@ """ load("@bazel_skylib//lib:shell.bzl", "shell") -load("@pythons_hub//:versions.bzl", "DEFAULT_PYTHON_VERSION", "MINOR_MAPPING") load("//python:py_binary.bzl", "py_binary") load("//python/private:bzlmod_enabled.bzl", "BZLMOD_ENABLED") # buildifier: disable=bzl-visibility -load("//python/private:full_version.bzl", "full_version") load("//python/private:toolchain_types.bzl", "EXEC_TOOLS_TOOLCHAIN_TYPE") # buildifier: disable=bzl-visibility load(":toolchain_types.bzl", "UV_TOOLCHAIN_TYPE") @@ -75,15 +73,15 @@ def _args(ctx): def _lock_impl(ctx): srcs = ctx.files.srcs - python_version = full_version( - version = ctx.attr.python_version or DEFAULT_PYTHON_VERSION, - minor_mapping = MINOR_MAPPING, - ) - output = ctx.actions.declare_file("{}.{}.out".format( - ctx.label.name, - python_version.replace(".", "_"), - )) + fname = "{}.out".format(ctx.label.name) + python_version = ctx.attr.python_version + if python_version: + fname = "{}.{}.out".format( + ctx.label.name, + python_version.replace(".", "_"), + ) + output = ctx.actions.declare_file(fname) toolchain_info = ctx.toolchains[UV_TOOLCHAIN_TYPE] uv = toolchain_info.uv_toolchain_info.uv[DefaultInfo].files_to_run.executable @@ -166,15 +164,7 @@ def _transition_impl(input_settings, attr): _PYTHON_VERSION_FLAG: input_settings[_PYTHON_VERSION_FLAG], } if attr.python_version: - # FIXME @aignas 2025-03-20: using `full_version` is a workaround for a bug in - # how we order toolchains in bazel. If I set the `python_version` flag - # to `3.12`, I would expect the latest version to be selected, i.e. the - # one that is in MINOR_MAPPING, but it seems that 3.12.0 is selected, - # because of how the targets are ordered. - settings[_PYTHON_VERSION_FLAG] = full_version( - version = attr.python_version, - minor_mapping = MINOR_MAPPING, - ) + settings[_PYTHON_VERSION_FLAG] = attr.python_version return settings _python_version_transition = transition( @@ -436,9 +426,6 @@ def lock( if not BZLMOD_ENABLED: kwargs["target_compatible_with"] = ["@platforms//:incompatible"] - # FIXME @aignas 2025-03-17: should we have one more target that transitions - # the python_version to ensure that if somebody calls `bazel build - # :requirements` that it is locked with the right `python_version`? _lock( name = name, args = args, diff --git a/tests/config_settings/transition/multi_version_tests.bzl b/tests/config_settings/transition/multi_version_tests.bzl index aca341a295..93f6efd728 100644 --- a/tests/config_settings/transition/multi_version_tests.bzl +++ b/tests/config_settings/transition/multi_version_tests.bzl @@ -13,6 +13,7 @@ # limitations under the License. """Tests for py_test.""" +load("@pythons_hub//:versions.bzl", "DEFAULT_PYTHON_VERSION") load("@rules_testing//lib:analysis_test.bzl", "analysis_test") load("@rules_testing//lib:test_suite.bzl", "test_suite") load("@rules_testing//lib:util.bzl", "TestingAspectInfo", rt_util = "util") @@ -29,7 +30,7 @@ load("//tests/support:support.bzl", "CC_TOOLCHAIN") # If the toolchain is not resolved then you will have a weird message telling # you that your transition target does not have a PyRuntime provider, which is # caused by there not being a toolchain detected for the target. -_PYTHON_VERSION = "3.11" +_PYTHON_VERSION = DEFAULT_PYTHON_VERSION _tests = [] diff --git a/tests/python/python_tests.bzl b/tests/python/python_tests.bzl index 1679794e15..97c47b57db 100644 --- a/tests/python/python_tests.bzl +++ b/tests/python/python_tests.bzl @@ -284,6 +284,58 @@ def _test_default_non_rules_python_ignore_root_user_error_non_root_module(env): _tests.append(_test_default_non_rules_python_ignore_root_user_error_non_root_module) +def _test_toolchain_ordering(env): + py = parse_modules( + module_ctx = _mock_mctx( + _mod( + name = "my_module", + toolchain = [ + _toolchain("3.10"), + _toolchain("3.10.15"), + _toolchain("3.10.16"), + _toolchain("3.10.11"), + _toolchain("3.11.1"), + _toolchain("3.11.10"), + _toolchain("3.11.11", is_default = True), + ], + ), + _mod(name = "rules_python", toolchain = [_toolchain("3.11")]), + ), + ) + got_versions = [ + t.python_version + for t in py.toolchains + ] + + env.expect.that_str(py.default_python_version).equals("3.11.11") + env.expect.that_dict(py.config.minor_mapping).contains_exactly({ + "3.10": "3.10.16", + "3.11": "3.11.11", + "3.12": "3.12.9", + "3.13": "3.13.2", + "3.8": "3.8.20", + "3.9": "3.9.21", + }) + env.expect.that_collection(got_versions).contains_exactly([ + # First the full-version toolchains that are in minor_mapping + # so that they get matched first if only the `python_version` is in MINOR_MAPPING + # + # The default version is always set in the `python_version` flag, so know, that + # the default match will be somewhere in the first bunch. + "3.10", + "3.10.16", + "3.11", + "3.11.11", + # Next, the rest, where we will match things based on the `python_version` being + # the same + "3.10.15", + "3.10.11", + "3.11.1", + "3.11.10", + ]).in_order() + +_tests.append(_test_toolchain_ordering) + def _test_default_from_defaults(env): py = parse_modules( module_ctx = _mock_mctx( diff --git a/tests/toolchains/transitions/BUILD.bazel b/tests/toolchains/transitions/BUILD.bazel new file mode 100644 index 0000000000..a7bef8c0e5 --- /dev/null +++ b/tests/toolchains/transitions/BUILD.bazel @@ -0,0 +1,5 @@ +load(":transitions_tests.bzl", "transitions_test_suite") + +transitions_test_suite( + name = "transitions_tests", +) diff --git a/tests/toolchains/transitions/transitions_tests.bzl b/tests/toolchains/transitions/transitions_tests.bzl new file mode 100644 index 0000000000..bddd1745f0 --- /dev/null +++ b/tests/toolchains/transitions/transitions_tests.bzl @@ -0,0 +1,182 @@ +# Copyright 2022 The Bazel Authors. All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +"" + +load("@pythons_hub//:versions.bzl", "DEFAULT_PYTHON_VERSION", "MINOR_MAPPING") +load("@rules_testing//lib:analysis_test.bzl", "analysis_test") +load("@rules_testing//lib:test_suite.bzl", "test_suite") +load("@rules_testing//lib:util.bzl", rt_util = "util") +load("//python:versions.bzl", "TOOL_VERSIONS") +load("//python/private:bzlmod_enabled.bzl", "BZLMOD_ENABLED") # buildifier: disable=bzl-visibility +load("//python/private:full_version.bzl", "full_version") # buildifier: disable=bzl-visibility +load("//python/private:toolchain_types.bzl", "EXEC_TOOLS_TOOLCHAIN_TYPE") # buildifier: disable=bzl-visibility +load("//tests/support:support.bzl", "PYTHON_VERSION") + +_analysis_tests = [] + +def _transition_impl(input_settings, attr): + """Transition based on python_version flag. + + This is a simple transition impl that a user of rules_python may implement + for their own rule. + """ + settings = { + PYTHON_VERSION: input_settings[PYTHON_VERSION], + } + if attr.python_version: + settings[PYTHON_VERSION] = attr.python_version + return settings + +_python_version_transition = transition( + implementation = _transition_impl, + inputs = [PYTHON_VERSION], + outputs = [PYTHON_VERSION], +) + +TestInfo = provider( + doc = "A simple test provider to forward the values for the assertion.", + fields = {"got": "", "want": ""}, +) + +def _impl(ctx): + if ctx.attr.skip: + return [TestInfo(got = "", want = "")] + + exec_tools = ctx.toolchains[EXEC_TOOLS_TOOLCHAIN_TYPE].exec_tools + got_version = exec_tools.exec_interpreter[platform_common.ToolchainInfo].py3_runtime.interpreter_version_info + + return [ + TestInfo( + got = "{}.{}.{}".format( + got_version.major, + got_version.minor, + got_version.micro, + ), + want = ctx.attr.want_version, + ), + ] + +_simple_transition = rule( + implementation = _impl, + attrs = { + "python_version": attr.string( + doc = "The input python version which we transition on.", + ), + "skip": attr.bool( + doc = "Whether to skip the test", + ), + "want_version": attr.string( + doc = "The python version that we actually expect to receive.", + ), + "_allowlist_function_transition": attr.label( + default = "@bazel_tools//tools/allowlists/function_transition_allowlist", + ), + }, + toolchains = [ + config_common.toolchain_type( + EXEC_TOOLS_TOOLCHAIN_TYPE, + mandatory = False, + ), + ], + cfg = _python_version_transition, +) + +def _test_transitions(*, name, tests, skip = False): + """A reusable rule so that we can split the tests.""" + targets = {} + for test_name, (input_version, want_version) in tests.items(): + target_name = "{}_{}".format(name, test_name) + targets["python_" + test_name] = target_name + rt_util.helper_target( + _simple_transition, + name = target_name, + python_version = input_version, + want_version = want_version, + skip = skip, + ) + + analysis_test( + name = name, + impl = _test_transition_impl, + targets = targets, + ) + +def _test_transition_impl(env, targets): + # Check that the forwarded version from the PyRuntimeInfo is correct + for target in dir(targets): + if not target.startswith("python"): + # Skip other attributes that might be not the ones we set (e.g. to_json, to_proto). + continue + + test_info = env.expect.that_target(getattr(targets, target)).provider( + TestInfo, + factory = lambda v, meta: v, + ) + env.expect.that_str(test_info.got).equals(test_info.want) + +def _test_full_version(name): + """Check that python_version transitions work. + + Expectation is to get the same full version that we input. + """ + _test_transitions( + name = name, + tests = { + v.replace(".", "_"): (v, v) + for v in TOOL_VERSIONS + }, + ) + +_analysis_tests.append(_test_full_version) + +def _test_minor_versions(name): + """Ensure that MINOR_MAPPING versions are correctly selected.""" + _test_transitions( + name = name, + skip = not BZLMOD_ENABLED, + tests = { + minor.replace(".", "_"): (minor, full) + for minor, full in MINOR_MAPPING.items() + }, + ) + +_analysis_tests.append(_test_minor_versions) + +def _test_default(name): + """Check the default version. + + Lastly, if we don't provide any version to the transition, we should + get the default version + """ + default_version = full_version( + version = DEFAULT_PYTHON_VERSION, + minor_mapping = MINOR_MAPPING, + ) if DEFAULT_PYTHON_VERSION else "" + + _test_transitions( + name = name, + skip = not BZLMOD_ENABLED, + tests = { + "default": (None, default_version), + }, + ) + +_analysis_tests.append(_test_default) + +def transitions_test_suite(name): + test_suite( + name = name, + tests = _analysis_tests, + ) From f685fe9a192dcdc8b65376821d9f25b990aa54fa Mon Sep 17 00:00:00 2001 From: Matt Mackay Date: Sat, 5 Apr 2025 09:43:16 -0400 Subject: [PATCH 141/922] fix: allow warn logging to be disabled via RULES_PYTHON_REPO_DEBUG_VERBOSITY (#2737) Allows the logging level to be set to `FAIL`, removing `WARN` logging. --------- Co-authored-by: Ignas Anikevicius <240938+aignas@users.noreply.github.com> --- CHANGELOG.md | 1 + docs/environment-variables.md | 1 + python/private/repo_utils.bzl | 1 + 3 files changed, 3 insertions(+) diff --git a/CHANGELOG.md b/CHANGELOG.md index b11270cb25..33acd38706 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -78,6 +78,7 @@ Unreleased changes template. * (toolchains) Do not try to run `chmod` when downloading non-windows hermetic toolchain repositories on Windows. Fixes [#2660](https://github.com/bazel-contrib/rules_python/issues/2660). +* (logging) Allow repo rule logging level to be set to `FAIL` via the `RULES_PYTHON_REPO_DEBUG_VERBOSITY` environment variable. * (toolchains) The toolchain matching is has been fixed when writing transitions transitioning on the `python_version` flag. Fixes [#2685](https://github.com/bazel-contrib/rules_python/issues/2685). diff --git a/docs/environment-variables.md b/docs/environment-variables.md index d8735cb2d5..9500fa8295 100644 --- a/docs/environment-variables.md +++ b/docs/environment-variables.md @@ -101,6 +101,7 @@ doing. This is mostly useful for development to debug errors. Determines the verbosity of logging output for repo rules. Valid values: * `DEBUG` +* `FAIL` * `INFO` * `TRACE` ::: diff --git a/python/private/repo_utils.bzl b/python/private/repo_utils.bzl index d9ad2449f1..73883a9244 100644 --- a/python/private/repo_utils.bzl +++ b/python/private/repo_utils.bzl @@ -56,6 +56,7 @@ def _logger(mrctx, name = None): verbosity = { "DEBUG": 2, + "FAIL": -1, "INFO": 1, "TRACE": 3, }.get(verbosity_level, 0) From f65b2ac7b20354cf18400cb6512548405a88639c Mon Sep 17 00:00:00 2001 From: Matt Mackay Date: Sat, 5 Apr 2025 11:51:41 -0400 Subject: [PATCH 142/922] fix: run check on interpreter in isolated mode (#2738) Runs the check on the interpreter in the toolchain repo in isolated mode via `-I`. This ensures it's not influenced by userland environment variables, such as `PYTHONPATH` which will cause issues if it allows this invocation to use into another interpreter versions site-packages. --- CHANGELOG.md | 1 + python/private/toolchains_repo.bzl | 9 ++++++++- 2 files changed, 9 insertions(+), 1 deletion(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 33acd38706..ac41e81f6b 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -82,6 +82,7 @@ Unreleased changes template. * (toolchains) The toolchain matching is has been fixed when writing transitions transitioning on the `python_version` flag. Fixes [#2685](https://github.com/bazel-contrib/rules_python/issues/2685). +* (toolchains) Run the check on the Python interpreter in isolated mode, to ensure it's not affected by userland environment variables, such as `PYTHONPATH`. {#v0-0-0-added} ### Added diff --git a/python/private/toolchains_repo.bzl b/python/private/toolchains_repo.bzl index 4e4a5de501..23c4643c0a 100644 --- a/python/private/toolchains_repo.bzl +++ b/python/private/toolchains_repo.bzl @@ -275,7 +275,14 @@ assert want_python == got_python, \ repo_utils.execute_checked( rctx, op = "CheckHostInterpreter", - arguments = [rctx.path(python_binary), python_tester], + arguments = [ + rctx.path(python_binary), + # Run the interpreter in isolated mode, this options implies -E, -P and -s. + # This ensures that environment variables are ignored that are set in userspace, such as PYTHONPATH, + # which may interfere with this invocation. + "-I", + python_tester, + ], ) if not rctx.delete(python_tester): fail("Failed to delete the python tester") From 537fc4b9e461639144083a1542e10f7589c5251f Mon Sep 17 00:00:00 2001 From: Ignas Anikevicius <240938+aignas@users.noreply.github.com> Date: Sun, 6 Apr 2025 00:51:57 +0900 Subject: [PATCH 143/922] fix(pypi): correctly fallback to pip for git direct URLs (#2732) Whilst integrating #2695 I introduced a regression and here I add a test for that and fix it. The code that was getting the filename from the URL was too eager and would break if there was a git ref as noted in the test. Before this commit and #2695 the code was not handling all of the cases that are tested now either, so I think now we are in a good place. I am not sure how we should handle the `git_repository` URLs. Maybe having `http_archive` and `git_repository` usage would be nice, but I am not sure how we can introduce it at the moment. Work towards #2363 --- python/private/pypi/parse_requirements.bzl | 6 +++ tests/pypi/extension/extension_tests.bzl | 50 +++++++++++++++++++++- 2 files changed, 55 insertions(+), 1 deletion(-) diff --git a/python/private/pypi/parse_requirements.bzl b/python/private/pypi/parse_requirements.bzl index 3280ce8df1..d2014a7eb9 100644 --- a/python/private/pypi/parse_requirements.bzl +++ b/python/private/pypi/parse_requirements.bzl @@ -297,6 +297,12 @@ def _add_dists(*, requirement, index_urls, logger = None): if requirement.srcs.url: url = requirement.srcs.url _, _, filename = url.rpartition("/") + if "." not in filename: + # detected filename has no extension, it might be an sdist ref + # TODO @aignas 2025-04-03: should be handled if the following is fixed: + # https://github.com/bazel-contrib/rules_python/issues/2363 + return [], None + direct_url_dist = struct( url = url, filename = filename, diff --git a/tests/pypi/extension/extension_tests.bzl b/tests/pypi/extension/extension_tests.bzl index 3a91c7b108..ab7a1358ad 100644 --- a/tests/pypi/extension/extension_tests.bzl +++ b/tests/pypi/extension/extension_tests.bzl @@ -662,6 +662,8 @@ some_pkg==0.0.1 @ example-direct.org/some_pkg-0.0.1-py3-none-any.whl \ direct_without_sha==0.0.1 @ example-direct.org/direct_without_sha-0.0.1-py3-none-any.whl some_other_pkg==0.0.1 pip_fallback==0.0.1 +direct_sdist_without_sha @ some-archive/any-name.tar.gz +git_dep @ git+https://git.server/repo/project@deadbeefdeadbeef """, }[x], ), @@ -672,10 +674,28 @@ pip_fallback==0.0.1 ) pypi.is_reproducible().equals(False) - pypi.exposed_packages().contains_exactly({"pypi": ["direct_without_sha", "pip_fallback", "simple", "some_other_pkg", "some_pkg"]}) + pypi.exposed_packages().contains_exactly({"pypi": [ + "direct_sdist_without_sha", + "direct_without_sha", + "git_dep", + "pip_fallback", + "simple", + "some_other_pkg", + "some_pkg", + ]}) pypi.hub_group_map().contains_exactly({"pypi": {}}) pypi.hub_whl_map().contains_exactly({ "pypi": { + "direct_sdist_without_sha": { + "pypi_315_any_name": [ + struct( + config_setting = None, + filename = "any-name.tar.gz", + target_platforms = None, + version = "3.15", + ), + ], + }, "direct_without_sha": { "pypi_315_direct_without_sha_0_0_1_py3_none_any": [ struct( @@ -686,6 +706,16 @@ pip_fallback==0.0.1 ), ], }, + "git_dep": { + "pypi_315_git_dep": [ + struct( + config_setting = None, + filename = None, + target_platforms = None, + version = "3.15", + ), + ], + }, "pip_fallback": { "pypi_315_pip_fallback": [ struct( @@ -737,6 +767,17 @@ pip_fallback==0.0.1 }, }) pypi.whl_libraries().contains_exactly({ + "pypi_315_any_name": { + "dep_template": "@pypi//{name}:{target}", + "experimental_target_platforms": ["cp315_linux_aarch64", "cp315_linux_arm", "cp315_linux_ppc", "cp315_linux_s390x", "cp315_linux_x86_64", "cp315_osx_aarch64", "cp315_osx_x86_64", "cp315_windows_x86_64"], + "extra_pip_args": ["--extra-args-for-sdist-building"], + "filename": "any-name.tar.gz", + "python_interpreter_target": "unit_test_interpreter_target", + "repo": "pypi_315", + "requirement": "direct_sdist_without_sha @ some-archive/any-name.tar.gz", + "sha256": "", + "urls": ["some-archive/any-name.tar.gz"], + }, "pypi_315_direct_without_sha_0_0_1_py3_none_any": { "dep_template": "@pypi//{name}:{target}", "experimental_target_platforms": ["cp315_linux_aarch64", "cp315_linux_arm", "cp315_linux_ppc", "cp315_linux_s390x", "cp315_linux_x86_64", "cp315_osx_aarch64", "cp315_osx_x86_64", "cp315_windows_x86_64"], @@ -747,6 +788,13 @@ pip_fallback==0.0.1 "sha256": "", "urls": ["example-direct.org/direct_without_sha-0.0.1-py3-none-any.whl"], }, + "pypi_315_git_dep": { + "dep_template": "@pypi//{name}:{target}", + "extra_pip_args": ["--extra-args-for-sdist-building"], + "python_interpreter_target": "unit_test_interpreter_target", + "repo": "pypi_315", + "requirement": "git_dep @ git+https://git.server/repo/project@deadbeefdeadbeef", + }, "pypi_315_pip_fallback": { "dep_template": "@pypi//{name}:{target}", "extra_pip_args": ["--extra-args-for-sdist-building"], From 69a99200fa38096675bd37ba2856eb3077cd3b86 Mon Sep 17 00:00:00 2001 From: Jason Bedard Date: Sat, 5 Apr 2025 09:02:59 -0700 Subject: [PATCH 144/922] fix: support gazelle generation_mode:update_only (#2708) This just fixes a crash when `generation_mode: update_only` causes `GenerateRules` to not be invoked for 100% of directories. Fix #2707 --- gazelle/pythonconfig/pythonconfig.go | 25 +++++++++++------- gazelle/pythonconfig/pythonconfig_test.go | 32 +++++++++++++++++++++++ 2 files changed, 48 insertions(+), 9 deletions(-) diff --git a/gazelle/pythonconfig/pythonconfig.go b/gazelle/pythonconfig/pythonconfig.go index 2183ec60a3..23c0cfd572 100644 --- a/gazelle/pythonconfig/pythonconfig.go +++ b/gazelle/pythonconfig/pythonconfig.go @@ -22,8 +22,8 @@ import ( "github.com/emirpasic/gods/lists/singlylinkedlist" - "github.com/bazelbuild/bazel-gazelle/label" "github.com/bazel-contrib/rules_python/gazelle/manifest" + "github.com/bazelbuild/bazel-gazelle/label" ) // Directives @@ -125,21 +125,28 @@ const ( // defaultIgnoreFiles is the list of default values used in the // python_ignore_files option. -var defaultIgnoreFiles = map[string]struct{}{ -} +var defaultIgnoreFiles = map[string]struct{}{} // Configs is an extension of map[string]*Config. It provides finding methods // on top of the mapping. type Configs map[string]*Config // ParentForPackage returns the parent Config for the given Bazel package. -func (c *Configs) ParentForPackage(pkg string) *Config { - dir := path.Dir(pkg) - if dir == "." { - dir = "" +func (c Configs) ParentForPackage(pkg string) *Config { + for { + dir := path.Dir(pkg) + if dir == "." { + dir = "" + } + parent := (map[string]*Config)(c)[dir] + if parent != nil { + return parent + } + if dir == "" { + return nil + } + pkg = dir } - parent := (map[string]*Config)(*c)[dir] - return parent } // Config represents a config extension for a specific Bazel package. diff --git a/gazelle/pythonconfig/pythonconfig_test.go b/gazelle/pythonconfig/pythonconfig_test.go index 7cdb9af1d1..fe21ce236e 100644 --- a/gazelle/pythonconfig/pythonconfig_test.go +++ b/gazelle/pythonconfig/pythonconfig_test.go @@ -248,3 +248,35 @@ func TestFormatThirdPartyDependency(t *testing.T) { }) } } + +func TestConfigsMap(t *testing.T) { + t.Run("only root", func(t *testing.T) { + configs := Configs{"": New("root/dir", "")} + + if configs.ParentForPackage("") == nil { + t.Fatal("expected non-nil for root config") + } + + if configs.ParentForPackage("a/b/c") != configs[""] { + t.Fatal("expected root for subpackage") + } + }) + + t.Run("sparse child configs", func(t *testing.T) { + configs := Configs{"": New("root/dir", "")} + configs["a"] = configs[""].NewChild() + configs["a/b/c"] = configs["a"].NewChild() + + if configs.ParentForPackage("a/b/c/d") != configs["a/b/c"] { + t.Fatal("child should match direct parent") + } + + if configs.ParentForPackage("a/b/c/d/e") != configs["a/b/c"] { + t.Fatal("grandchild should match first parant") + } + + if configs.ParentForPackage("other/root/path") != configs[""] { + t.Fatal("non-configured subpackage should match root") + } + }) +} From 2bc357787e8d6e76fd2f58e401cf3062bcf4f415 Mon Sep 17 00:00:00 2001 From: Ignas Anikevicius <240938+aignas@users.noreply.github.com> Date: Sun, 6 Apr 2025 01:27:12 +0900 Subject: [PATCH 145/922] fix(pypi): mark the extension reproducible (#2730) This will remove the merge conflicts and improve the usability when the `MODULE.bazel.lock` is used together with `rules_python`. This means that the lock file will not be used to read the `URL` and `sha256` values for the Python sources when the `experimental_index_url` is used, but the idea is that that information will be kept in repo cache. Fixes #2434 Created #2731 to leverage the bazel feature to write immutable facts to the lock file once it becomes available. --- CHANGELOG.md | 3 +++ python/private/pypi/extension.bzl | 6 +----- tests/pypi/extension/extension_tests.bzl | 7 ------- 3 files changed, 4 insertions(+), 12 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index ac41e81f6b..69e9330f64 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -69,6 +69,9 @@ Unreleased changes template. * (toolchains) Remove all but `3.8.20` versions of the Python `3.8` interpreter who has reached EOL. If users still need other versions of the `3.8` interpreter, please supply the URLs manually {bzl:ob}`python.toolchain` or {bzl:obj}`python_register_toolchains` calls. +* (pypi) The PyPI extension will no longer write the lock file entries as the + extension has been marked reproducible. + Fixes [#2434](https://github.com/bazel-contrib/rules_python/issues/2434). [20250317]: https://github.com/astral-sh/python-build-standalone/releases/tag/20250317 diff --git a/python/private/pypi/extension.bzl b/python/private/pypi/extension.bzl index f782e69a45..8fce47656b 100644 --- a/python/private/pypi/extension.bzl +++ b/python/private/pypi/extension.bzl @@ -419,8 +419,6 @@ You cannot use both the additive_build_content and additive_build_content_file a extra_aliases = {} whl_libraries = {} - is_reproducible = True - for mod in module_ctx.modules: for pip_attr in mod.tags.parse: hub_name = pip_attr.hub_name @@ -458,7 +456,6 @@ You cannot use both the additive_build_content and additive_build_content_file a get_index_urls = None if pip_attr.experimental_index_url: - is_reproducible = False skip_sources = [ normalize_name(s) for s in pip_attr.simpleapi_skip @@ -543,7 +540,6 @@ You cannot use both the additive_build_content and additive_build_content_file a k: dict(sorted(args.items())) for k, args in sorted(whl_libraries.items()) }, - is_reproducible = is_reproducible, ) def _pip_impl(module_ctx): @@ -640,7 +636,7 @@ def _pip_impl(module_ctx): # In order to be able to dogfood the `experimental_index_url` feature before it gets # stabilized, we have created the `_pip_non_reproducible` function, that will result # in extra entries in the lock file. - return module_ctx.extension_metadata(reproducible = mods.is_reproducible) + return module_ctx.extension_metadata(reproducible = True) else: return None diff --git a/tests/pypi/extension/extension_tests.bzl b/tests/pypi/extension/extension_tests.bzl index ab7a1358ad..1652e76156 100644 --- a/tests/pypi/extension/extension_tests.bzl +++ b/tests/pypi/extension/extension_tests.bzl @@ -64,7 +64,6 @@ def _parse_modules(env, **kwargs): return env.expect.that_struct( parse_modules(**kwargs), attrs = dict( - is_reproducible = subjects.bool, exposed_packages = subjects.dict, hub_group_map = subjects.dict, hub_whl_map = subjects.dict, @@ -160,7 +159,6 @@ def _test_simple(env): }, ) - pypi.is_reproducible().equals(True) pypi.exposed_packages().contains_exactly({"pypi": ["simple"]}) pypi.hub_group_map().contains_exactly({"pypi": {}}) pypi.hub_whl_map().contains_exactly({"pypi": { @@ -209,7 +207,6 @@ def _test_simple_multiple_requirements(env): }, ) - pypi.is_reproducible().equals(True) pypi.exposed_packages().contains_exactly({"pypi": ["simple"]}) pypi.hub_group_map().contains_exactly({"pypi": {}}) pypi.hub_whl_map().contains_exactly({"pypi": { @@ -278,7 +275,6 @@ torch==2.4.1 ; platform_machine != 'x86_64' \ }, ) - pypi.is_reproducible().equals(True) pypi.exposed_packages().contains_exactly({"pypi": ["torch"]}) pypi.hub_group_map().contains_exactly({"pypi": {}}) pypi.hub_whl_map().contains_exactly({"pypi": { @@ -404,7 +400,6 @@ torch==2.4.1+cpu ; platform_machine == 'x86_64' \ simpleapi_download = mocksimpleapi_download, ) - pypi.is_reproducible().equals(False) pypi.exposed_packages().contains_exactly({"pypi": ["torch"]}) pypi.hub_group_map().contains_exactly({"pypi": {}}) pypi.hub_whl_map().contains_exactly({"pypi": { @@ -535,7 +530,6 @@ simple==0.0.3 \ }, ) - pypi.is_reproducible().equals(True) pypi.exposed_packages().contains_exactly({"pypi": ["simple"]}) pypi.hub_group_map().contains_exactly({"pypi": {}}) pypi.hub_whl_map().contains_exactly({"pypi": { @@ -673,7 +667,6 @@ git_dep @ git+https://git.server/repo/project@deadbeefdeadbeef simpleapi_download = mocksimpleapi_download, ) - pypi.is_reproducible().equals(False) pypi.exposed_packages().contains_exactly({"pypi": [ "direct_sdist_without_sha", "direct_without_sha", From 01968255660aa99041c0c8989a0d68c01aa2978e Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Sat, 5 Apr 2025 09:37:21 -0700 Subject: [PATCH 146/922] feat: allow populating binary's venv site-packages with symlinks (#2617) This implements functionality to allow libraries to populate the site-packages directory of downstream binaries. The basic implementation is: * Libraries provide tuples of `(runfile path, site packages path)` in the `PyInfo.site_packages_symlinks` field. * Binaries create symlinks (using declare_symlink) in their site-packages directory pointing to the runfiles paths libraries provide. The design was chosen because of the following properties: * The site-packages directory is relocatable * Populating site packages is cheap ( `O(number 3p dependencies)` ) * Dependencies are only created once in the runfiles, no matter how many how many binaries there that use them. This minimizes disk usage, file counts, inodes, etc. The `site_packages_symlinks` field is a depset with topological ordering. Using topological ordering allows dependencies closer to the binary to have precedence, which gives some basic control over what entries are used. Additionally, the runfiles path to link to can be None/empty, in which case, the directory in site-packages won't be created. This allows binaries to prevent creation of directories that might e.g. conflict. For now, this functionality is disabled by default. The flag `--venvs_site_packages=yes` can be set to allow using it, which is automatically enable it for pypi generated targets. When enabled, it does basic detection of implicit namespace directories, which allows multiple distributions to "install" into the the same site-packages directory. Though this functionality is primarily useful for dependencies from pypi (e.g. via pip.parse), it is not yet activated for those targets, for two main reasons: 1. The wheel extraction code creates pkgutil-style `__init__.py` shims during the repo-phase. The build phase can't distinguish these artifical rules_python generated shims from actual `__init__.py` files, which breaks the implicit namespace detection logic. 2. A flag guard is needed before changing the behavior. Even though how 3p libraries are added to sys.path is an implementation detail, the behavior has been there for many years, so an escape hatch should be added. Work towards https://github.com/bazelbuild/rules_python/issues/2156 --- .bazelrc | 4 +- CHANGELOG.md | 4 + MODULE.bazel | 6 + docs/BUILD.bazel | 1 + docs/_includes/experimental_api.md | 5 + .../python/config_settings/index.md | 17 ++ internal_dev_deps.bzl | 6 + python/BUILD.bazel | 3 + python/config_settings/BUILD.bazel | 8 + python/features.bzl | 43 ++++- python/private/attributes.bzl | 11 ++ python/private/builders.bzl | 13 +- python/private/common.bzl | 17 +- python/private/enum.bzl | 20 +++ python/private/flags.bzl | 38 ++--- python/private/py_executable.bzl | 76 ++++++++- python/private/py_info.bzl | 35 +++- python/private/py_library.bzl | 161 +++++++++++++++++- python/private/pypi/whl_library_targets.bzl | 1 + tests/modules/other/BUILD.bazel | 0 tests/modules/other/MODULE.bazel | 3 + tests/modules/other/nspkg_delta/BUILD.bazel | 10 ++ .../nspkg/subnspkg/delta/__init__.py | 1 + tests/modules/other/nspkg_gamma/BUILD.bazel | 10 ++ .../nspkg/subnspkg/gamma/__init__.py | 1 + .../whl_library_targets_tests.bzl | 2 + tests/support/sh_py_run_test.bzl | 4 + tests/venv_site_packages_libs/BUILD.bazel | 17 ++ tests/venv_site_packages_libs/bin.py | 32 ++++ .../nspkg_alpha/BUILD.bazel | 10 ++ .../nspkg/subnspkg/alpha/__init__.py | 1 + .../nspkg_beta/BUILD.bazel | 10 ++ .../nspkg/subnspkg/beta/__init__.py | 1 + .../venv_site_packages_pypi_test.py | 36 ++++ 34 files changed, 574 insertions(+), 33 deletions(-) create mode 100644 docs/_includes/experimental_api.md create mode 100644 tests/modules/other/BUILD.bazel create mode 100644 tests/modules/other/MODULE.bazel create mode 100644 tests/modules/other/nspkg_delta/BUILD.bazel create mode 100644 tests/modules/other/nspkg_delta/site-packages/nspkg/subnspkg/delta/__init__.py create mode 100644 tests/modules/other/nspkg_gamma/BUILD.bazel create mode 100644 tests/modules/other/nspkg_gamma/site-packages/nspkg/subnspkg/gamma/__init__.py create mode 100644 tests/venv_site_packages_libs/BUILD.bazel create mode 100644 tests/venv_site_packages_libs/bin.py create mode 100644 tests/venv_site_packages_libs/nspkg_alpha/BUILD.bazel create mode 100644 tests/venv_site_packages_libs/nspkg_alpha/site-packages/nspkg/subnspkg/alpha/__init__.py create mode 100644 tests/venv_site_packages_libs/nspkg_beta/BUILD.bazel create mode 100644 tests/venv_site_packages_libs/nspkg_beta/site-packages/nspkg/subnspkg/beta/__init__.py create mode 100644 tests/venv_site_packages_libs/venv_site_packages_pypi_test.py diff --git a/.bazelrc b/.bazelrc index ada5c5a0a7..4e6f2fa187 100644 --- a/.bazelrc +++ b/.bazelrc @@ -4,8 +4,8 @@ # (Note, we cannot use `common --deleted_packages` because the bazel version command doesn't support it) # To update these lines, execute # `bazel run @rules_bazel_integration_test//tools:update_deleted_packages` -build --deleted_packages=examples/build_file_generation,examples/build_file_generation/random_number_generator,examples/bzlmod,examples/bzlmod_build_file_generation,examples/bzlmod_build_file_generation/other_module/other_module/pkg,examples/bzlmod_build_file_generation/runfiles,examples/bzlmod/entry_points,examples/bzlmod/entry_points/tests,examples/bzlmod/libs/my_lib,examples/bzlmod/other_module,examples/bzlmod/other_module/other_module/pkg,examples/bzlmod/patches,examples/bzlmod/py_proto_library,examples/bzlmod/py_proto_library/example.com/another_proto,examples/bzlmod/py_proto_library/example.com/proto,examples/bzlmod/runfiles,examples/bzlmod/tests,examples/bzlmod/tests/other_module,examples/bzlmod/whl_mods,examples/multi_python_versions/libs/my_lib,examples/multi_python_versions/requirements,examples/multi_python_versions/tests,examples/pip_parse,examples/pip_parse_vendored,examples/pip_repository_annotations,examples/py_proto_library,examples/py_proto_library/example.com/another_proto,examples/py_proto_library/example.com/proto,gazelle,gazelle/manifest,gazelle/manifest/generate,gazelle/manifest/hasher,gazelle/manifest/test,gazelle/modules_mapping,gazelle/python,gazelle/pythonconfig,gazelle/python/private,tests/integration/compile_pip_requirements,tests/integration/compile_pip_requirements_test_from_external_repo,tests/integration/custom_commands,tests/integration/ignore_root_user_error,tests/integration/ignore_root_user_error/submodule,tests/integration/local_toolchains,tests/integration/pip_parse,tests/integration/pip_parse/empty,tests/integration/py_cc_toolchain_registered -query --deleted_packages=examples/build_file_generation,examples/build_file_generation/random_number_generator,examples/bzlmod,examples/bzlmod_build_file_generation,examples/bzlmod_build_file_generation/other_module/other_module/pkg,examples/bzlmod_build_file_generation/runfiles,examples/bzlmod/entry_points,examples/bzlmod/entry_points/tests,examples/bzlmod/libs/my_lib,examples/bzlmod/other_module,examples/bzlmod/other_module/other_module/pkg,examples/bzlmod/patches,examples/bzlmod/py_proto_library,examples/bzlmod/py_proto_library/example.com/another_proto,examples/bzlmod/py_proto_library/example.com/proto,examples/bzlmod/runfiles,examples/bzlmod/tests,examples/bzlmod/tests/other_module,examples/bzlmod/whl_mods,examples/multi_python_versions/libs/my_lib,examples/multi_python_versions/requirements,examples/multi_python_versions/tests,examples/pip_parse,examples/pip_parse_vendored,examples/pip_repository_annotations,examples/py_proto_library,examples/py_proto_library/example.com/another_proto,examples/py_proto_library/example.com/proto,gazelle,gazelle/manifest,gazelle/manifest/generate,gazelle/manifest/hasher,gazelle/manifest/test,gazelle/modules_mapping,gazelle/python,gazelle/pythonconfig,gazelle/python/private,tests/integration/compile_pip_requirements,tests/integration/compile_pip_requirements_test_from_external_repo,tests/integration/custom_commands,tests/integration/ignore_root_user_error,tests/integration/ignore_root_user_error/submodule,tests/integration/local_toolchains,tests/integration/pip_parse,tests/integration/pip_parse/empty,tests/integration/py_cc_toolchain_registered +build --deleted_packages=examples/build_file_generation,examples/build_file_generation/random_number_generator,examples/bzlmod,examples/bzlmod_build_file_generation,examples/bzlmod_build_file_generation/other_module/other_module/pkg,examples/bzlmod_build_file_generation/runfiles,examples/bzlmod/entry_points,examples/bzlmod/entry_points/tests,examples/bzlmod/libs/my_lib,examples/bzlmod/other_module,examples/bzlmod/other_module/other_module/pkg,examples/bzlmod/patches,examples/bzlmod/py_proto_library,examples/bzlmod/py_proto_library/example.com/another_proto,examples/bzlmod/py_proto_library/example.com/proto,examples/bzlmod/runfiles,examples/bzlmod/tests,examples/bzlmod/tests/other_module,examples/bzlmod/whl_mods,examples/multi_python_versions/libs/my_lib,examples/multi_python_versions/requirements,examples/multi_python_versions/tests,examples/pip_parse,examples/pip_parse_vendored,examples/pip_repository_annotations,examples/py_proto_library,examples/py_proto_library/example.com/another_proto,examples/py_proto_library/example.com/proto,gazelle,gazelle/manifest,gazelle/manifest/generate,gazelle/manifest/hasher,gazelle/manifest/test,gazelle/modules_mapping,gazelle/python,gazelle/pythonconfig,gazelle/python/private,tests/integration/compile_pip_requirements,tests/integration/compile_pip_requirements_test_from_external_repo,tests/integration/custom_commands,tests/integration/ignore_root_user_error,tests/integration/ignore_root_user_error/submodule,tests/integration/local_toolchains,tests/integration/pip_parse,tests/integration/pip_parse/empty,tests/integration/py_cc_toolchain_registered,tests/modules/other,tests/modules/other/nspkg_delta,tests/modules/other/nspkg_gamma +query --deleted_packages=examples/build_file_generation,examples/build_file_generation/random_number_generator,examples/bzlmod,examples/bzlmod_build_file_generation,examples/bzlmod_build_file_generation/other_module/other_module/pkg,examples/bzlmod_build_file_generation/runfiles,examples/bzlmod/entry_points,examples/bzlmod/entry_points/tests,examples/bzlmod/libs/my_lib,examples/bzlmod/other_module,examples/bzlmod/other_module/other_module/pkg,examples/bzlmod/patches,examples/bzlmod/py_proto_library,examples/bzlmod/py_proto_library/example.com/another_proto,examples/bzlmod/py_proto_library/example.com/proto,examples/bzlmod/runfiles,examples/bzlmod/tests,examples/bzlmod/tests/other_module,examples/bzlmod/whl_mods,examples/multi_python_versions/libs/my_lib,examples/multi_python_versions/requirements,examples/multi_python_versions/tests,examples/pip_parse,examples/pip_parse_vendored,examples/pip_repository_annotations,examples/py_proto_library,examples/py_proto_library/example.com/another_proto,examples/py_proto_library/example.com/proto,gazelle,gazelle/manifest,gazelle/manifest/generate,gazelle/manifest/hasher,gazelle/manifest/test,gazelle/modules_mapping,gazelle/python,gazelle/pythonconfig,gazelle/python/private,tests/integration/compile_pip_requirements,tests/integration/compile_pip_requirements_test_from_external_repo,tests/integration/custom_commands,tests/integration/ignore_root_user_error,tests/integration/ignore_root_user_error/submodule,tests/integration/local_toolchains,tests/integration/pip_parse,tests/integration/pip_parse/empty,tests/integration/py_cc_toolchain_registered,tests/modules/other,tests/modules/other/nspkg_delta,tests/modules/other/nspkg_gamma test --test_output=errors diff --git a/CHANGELOG.md b/CHANGELOG.md index 69e9330f64..818773e589 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -107,6 +107,10 @@ Unreleased changes template. please check the {obj}`uv.configure` tag class. * Add support for riscv64 linux platform. * (toolchains) Add python 3.13.2 and 3.12.9 toolchains +* (providers) (experimental) {obj}`PyInfo.site_packages_symlinks` field added to + allow specifying links to create within the venv site packages (only + applicable with {obj}`--bootstrap_impl=script`) + ([#2156](https://github.com/bazelbuild/rules_python/issues/2156)). {#v0-0-0-removed} ### Removed diff --git a/MODULE.bazel b/MODULE.bazel index e4e45af7f0..c649896344 100644 --- a/MODULE.bazel +++ b/MODULE.bazel @@ -85,6 +85,7 @@ bazel_dep(name = "rules_shell", version = "0.3.0", dev_dependency = True) bazel_dep(name = "rules_multirun", version = "0.9.0", dev_dependency = True) bazel_dep(name = "bazel_ci_rules", version = "1.0.0", dev_dependency = True) bazel_dep(name = "rules_pkg", version = "1.0.1", dev_dependency = True) +bazel_dep(name = "other", version = "0", dev_dependency = True) # Extra gazelle plugin deps so that WORKSPACE.bzlmod can continue including it for e2e tests. # We use `WORKSPACE.bzlmod` because it is impossible to have dev-only local overrides. @@ -106,6 +107,11 @@ local_path_override( path = "gazelle", ) +local_path_override( + module_name = "other", + path = "tests/modules/other", +) + dev_python = use_extension( "//python/extensions:python.bzl", "python", diff --git a/docs/BUILD.bazel b/docs/BUILD.bazel index bebecd18b2..29eac6e714 100644 --- a/docs/BUILD.bazel +++ b/docs/BUILD.bazel @@ -87,6 +87,7 @@ sphinx_stardocs( name = "bzl_api_docs", srcs = [ "//python:defs_bzl", + "//python:features_bzl", "//python:packaging_bzl", "//python:pip_bzl", "//python:py_binary_bzl", diff --git a/docs/_includes/experimental_api.md b/docs/_includes/experimental_api.md new file mode 100644 index 0000000000..45473a7cbf --- /dev/null +++ b/docs/_includes/experimental_api.md @@ -0,0 +1,5 @@ +:::{warning} + +**Experimental API.** This API is still under development and may change or be +removed without notice. +::: diff --git a/docs/api/rules_python/python/config_settings/index.md b/docs/api/rules_python/python/config_settings/index.md index 79c7d0c109..340335d9b1 100644 --- a/docs/api/rules_python/python/config_settings/index.md +++ b/docs/api/rules_python/python/config_settings/index.md @@ -213,6 +213,23 @@ Values: :::: +:::: + +:::{flag} venvs_site_packages + +Determines if libraries use a site-packages layout for their files. + +Note this flag only affects PyPI dependencies of `--bootstrap_impl=script` binaries + +:::{include} /_includes/experimental_api.md +::: + + +Values: +* `no` (default): Make libraries importable by adding to `sys.path` +* `yes`: Make libraries importable by creating paths in a binary's site-packages directory. +:::: + ::::{bzl:flag} bootstrap_impl Determine how programs implement their startup process. diff --git a/internal_dev_deps.bzl b/internal_dev_deps.bzl index cd33475f43..87690be1ad 100644 --- a/internal_dev_deps.bzl +++ b/internal_dev_deps.bzl @@ -15,6 +15,7 @@ """Dependencies that are needed for development and testing of rules_python itself.""" load("@bazel_tools//tools/build_defs/repo:http.bzl", _http_archive = "http_archive", _http_file = "http_file") +load("@bazel_tools//tools/build_defs/repo:local.bzl", "local_repository") load("@bazel_tools//tools/build_defs/repo:utils.bzl", "maybe") load("//python/private:internal_config_repo.bzl", "internal_config_repo") # buildifier: disable=bzl-visibility @@ -42,6 +43,11 @@ def rules_python_internal_deps(): """ internal_config_repo(name = "rules_python_internal") + local_repository( + name = "other", + path = "tests/modules/other", + ) + http_archive( name = "bazel_skylib", sha256 = "bc283cdfcd526a52c3201279cda4bc298652efa898b10b4db0837dc51652756f", diff --git a/python/BUILD.bazel b/python/BUILD.bazel index c52e772666..a699c81cc4 100644 --- a/python/BUILD.bazel +++ b/python/BUILD.bazel @@ -79,6 +79,9 @@ bzl_library( bzl_library( name = "features_bzl", srcs = ["features.bzl"], + deps = [ + "@rules_python_internal//:rules_python_config_bzl", + ], ) bzl_library( diff --git a/python/config_settings/BUILD.bazel b/python/config_settings/BUILD.bazel index 796cf0c9c4..45354e24d9 100644 --- a/python/config_settings/BUILD.bazel +++ b/python/config_settings/BUILD.bazel @@ -9,6 +9,7 @@ load( "LibcFlag", "PrecompileFlag", "PrecompileSourceRetentionFlag", + "VenvsSitePackages", "VenvsUseDeclareSymlinkFlag", ) load( @@ -195,6 +196,13 @@ string_flag( visibility = ["//visibility:public"], ) +string_flag( + name = "venvs_site_packages", + build_setting_default = VenvsSitePackages.NO, + # NOTE: Only public because it is used in pip hub repos. + visibility = ["//visibility:public"], +) + define_pypi_internal_flags( name = "define_pypi_internal_flags", ) diff --git a/python/features.bzl b/python/features.bzl index a7098f4710..8edfb698fc 100644 --- a/python/features.bzl +++ b/python/features.bzl @@ -19,8 +19,49 @@ load("@rules_python_internal//:rules_python_config.bzl", "config") # See https://git-scm.com/docs/git-archive/2.29.0#Documentation/git-archive.txt-export-subst _VERSION_PRIVATE = "$Format:%(describe:tags=true)$" +def _features_typedef(): + """Information about features rules_python has implemented. + + ::::{field} precompile + :type: bool + + True if the precompile attributes are available. + + :::{versionadded} 0.33.0 + ::: + :::: + + ::::{field} py_info_site_packages_symlinks + + True if the `PyInfo.site_packages_symlinks` field is available. + + :::{versionadded} VERSION_NEXT_FEATURE + ::: + :::: + + ::::{field} uses_builtin_rules + :type: bool + + True if the rules are using the Bazel-builtin implementation. + + :::{versionadded} 1.1.0 + ::: + :::: + + ::::{field} version + :type: str + + The rules_python version. This is a semver format, e.g. `X.Y.Z` with + optional trailing `-rcN`. For unreleased versions, it is an empty string. + :::{versionadded} 0.38.0 + :::: + """ + features = struct( - version = _VERSION_PRIVATE if "$Format" not in _VERSION_PRIVATE else "", + TYPEDEF = _features_typedef, + # keep sorted precompile = True, + py_info_site_packages_symlinks = True, uses_builtin_rules = not config.enable_pystar, + version = _VERSION_PRIVATE if "$Format" not in _VERSION_PRIVATE else "", ) diff --git a/python/private/attributes.bzl b/python/private/attributes.bzl index b042b3db6a..8543caba7b 100644 --- a/python/private/attributes.bzl +++ b/python/private/attributes.bzl @@ -254,6 +254,17 @@ These are typically `py_library` rules. Targets that only provide data files used at runtime belong in the `data` attribute. + +:::{note} +The order of this list can matter because it affects the order that information +from dependencies is merged in, which can be relevant depending on the ordering +mode of depsets that are merged. + +* {obj}`PyInfo.site_packages_symlinks` uses topological ordering. + +See {obj}`PyInfo` for more information about the ordering of its depsets and +how its fields are merged. +::: """, ), "precompile": lambda: attrb.String( diff --git a/python/private/builders.bzl b/python/private/builders.bzl index 50aa3ed91a..54d46c2af2 100644 --- a/python/private/builders.bzl +++ b/python/private/builders.bzl @@ -15,12 +15,19 @@ load("@bazel_skylib//lib:types.bzl", "types") -def _DepsetBuilder(): - """Create a builder for a depset.""" +def _DepsetBuilder(order = None): + """Create a builder for a depset. + + Args: + order: {type}`str | None` The order to initialize the depset to, if any. + + Returns: + {type}`DepsetBuilder` + """ # buildifier: disable=uninitialized self = struct( - _order = [None], + _order = [order], add = lambda *a, **k: _DepsetBuilder_add(self, *a, **k), build = lambda *a, **k: _DepsetBuilder_build(self, *a, **k), direct = [], diff --git a/python/private/common.bzl b/python/private/common.bzl index 48e2653ebb..072a1bb296 100644 --- a/python/private/common.bzl +++ b/python/private/common.bzl @@ -30,6 +30,16 @@ PackageSpecificationInfo = getattr(py_internal, "PackageSpecificationInfo", None # Extensions without the dot _PYTHON_SOURCE_EXTENSIONS = ["py"] +# Extensions that mean a file is relevant to Python +PYTHON_FILE_EXTENSIONS = [ + "dll", # Python C modules, Windows specific + "dylib", # Python C modules, Mac specific + "py", + "pyc", + "pyi", + "so", # Python C modules, usually Linux +] + def create_binary_semantics_struct( *, create_executable, @@ -367,7 +377,8 @@ def create_py_info( required_pyc_files, implicit_pyc_files, implicit_pyc_source_files, - imports): + imports, + site_packages_symlinks = []): """Create PyInfo provider. Args: @@ -385,6 +396,9 @@ def create_py_info( implicit_pyc_files: {type}`depset[File]` Implicitly generated pyc files that a binary can choose to include. imports: depset of strings; the import path values to propagate. + site_packages_symlinks: {type}`list[tuple[str, str]]` tuples of + `(runfiles_path, site_packages_path)` for symlinks to create + in the consuming binary's venv site packages. Returns: A tuple of the PyInfo instance and a depset of the @@ -392,6 +406,7 @@ def create_py_info( necessary for deprecated extra actions support). """ py_info = PyInfoBuilder() + py_info.site_packages_symlinks.add(site_packages_symlinks) py_info.direct_original_sources.add(original_sources) py_info.direct_pyc_files.add(required_pyc_files) py_info.direct_pyi_files.add(ctx.files.pyi_srcs) diff --git a/python/private/enum.bzl b/python/private/enum.bzl index d71442e3b5..4d0fb10699 100644 --- a/python/private/enum.bzl +++ b/python/private/enum.bzl @@ -43,3 +43,23 @@ def enum(methods = {}, **kwargs): self = struct(__members__ = members, **kwargs) return self + +def _FlagEnum_flag_values(self): + return sorted(self.__members__.values()) + +def FlagEnum(**kwargs): + """Define an enum specialized for flags. + + Args: + **kwargs: members of the enum. + + Returns: + {type}`FlagEnum` struct. This is an enum with the following extras: + * `flag_values`: A function that returns a sorted list of the + flag values (enum `__members__`). Useful for passing to the + `values` attribute for string flags. + """ + return enum( + methods = dict(flag_values = _FlagEnum_flag_values), + **kwargs + ) diff --git a/python/private/flags.bzl b/python/private/flags.bzl index 1019faa8d6..c53e4610ff 100644 --- a/python/private/flags.bzl +++ b/python/private/flags.bzl @@ -19,27 +19,7 @@ unnecessary files when all that are needed are flag definitions. """ load("@bazel_skylib//rules:common_settings.bzl", "BuildSettingInfo") -load(":enum.bzl", "enum") - -def _FlagEnum_flag_values(self): - return sorted(self.__members__.values()) - -def FlagEnum(**kwargs): - """Define an enum specialized for flags. - - Args: - **kwargs: members of the enum. - - Returns: - {type}`FlagEnum` struct. This is an enum with the following extras: - * `flag_values`: A function that returns a sorted list of the - flag values (enum `__members__`). Useful for passing to the - `values` attribute for string flags. - """ - return enum( - methods = dict(flag_values = _FlagEnum_flag_values), - **kwargs - ) +load(":enum.bzl", "FlagEnum", "enum") def _AddSrcsToRunfilesFlag_is_enabled(ctx): value = ctx.attr._add_srcs_to_runfiles_flag[BuildSettingInfo].value @@ -138,6 +118,22 @@ VenvsUseDeclareSymlinkFlag = FlagEnum( get_value = _venvs_use_declare_symlink_flag_get_value, ) +def _venvs_site_packages_is_enabled(ctx): + if not ctx.attr.experimental_venvs_site_packages: + return False + flag_value = ctx.attr.experimental_venvs_site_packages[BuildSettingInfo].value + return flag_value == VenvsSitePackages.YES + +# Decides if libraries try to use a site-packages layout using site_packages_symlinks +# buildifier: disable=name-conventions +VenvsSitePackages = FlagEnum( + # Use site_packages_symlinks + YES = "yes", + # Don't use site_packages_symlinks + NO = "no", + is_enabled = _venvs_site_packages_is_enabled, +) + # Used for matching freethreaded toolchains and would have to be used in wheels # as well. # buildifier: disable=name-conventions diff --git a/python/private/py_executable.bzl b/python/private/py_executable.bzl index fed46ab223..f33c2b6ca1 100644 --- a/python/private/py_executable.bzl +++ b/python/private/py_executable.bzl @@ -612,15 +612,89 @@ def _create_venv(ctx, output_prefix, imports, runtime_details): }, computed_substitutions = computed_subs, ) + site_packages_symlinks = _create_site_packages_symlinks(ctx, site_packages) return struct( interpreter = interpreter, recreate_venv_at_runtime = not venvs_use_declare_symlink_enabled, # Runfiles root relative path or absolute path interpreter_actual_path = interpreter_actual_path, - files_without_interpreter = [pyvenv_cfg, pth, site_init], + files_without_interpreter = [pyvenv_cfg, pth, site_init] + site_packages_symlinks, ) +def _create_site_packages_symlinks(ctx, site_packages): + """Creates symlinks within site-packages. + + Args: + ctx: current rule ctx + site_packages: runfiles-root-relative path to the site-packages directory + + Returns: + {type}`list[File]` list of the File symlink objects created. + """ + + # maps site-package symlink to the runfiles path it should point to + entries = depset( + # NOTE: Topological ordering is used so that dependencies closer to the + # binary have precedence in creating their symlinks. This allows the + # binary a modicum of control over the result. + order = "topological", + transitive = [ + dep[PyInfo].site_packages_symlinks + for dep in ctx.attr.deps + if PyInfo in dep + ], + ).to_list() + link_map = _build_link_map(entries) + + sp_files = [] + for sp_dir_path, link_to in link_map.items(): + sp_link = ctx.actions.declare_symlink(paths.join(site_packages, sp_dir_path)) + sp_link_rf_path = runfiles_root_path(ctx, sp_link.short_path) + rel_path = relative_path( + # dirname is necessary because a relative symlink is relative to + # the directory the symlink resides within. + from_ = paths.dirname(sp_link_rf_path), + to = link_to, + ) + ctx.actions.symlink(output = sp_link, target_path = rel_path) + sp_files.append(sp_link) + return sp_files + +def _build_link_map(entries): + link_map = {} + for link_to_runfiles_path, site_packages_path in entries: + if site_packages_path in link_map: + # We ignore duplicates by design. The dependency closer to the + # binary gets precedence due to the topological ordering. + continue + else: + link_map[site_packages_path] = link_to_runfiles_path + + # An empty link_to value means to not create the site package symlink. + # Because of the topological ordering, this allows binaries to remove + # entries by having an earlier dependency produce empty link_to values. + for sp_dir_path, link_to in link_map.items(): + if not link_to: + link_map.pop(sp_dir_path) + + # Remove entries that would be a child path of a created symlink. + # Earlier entries have precedence to match how exact matches are handled. + keep_link_map = {} + for _ in range(len(link_map)): + if not link_map: + break + dirname, value = link_map.popitem() + keep_link_map[dirname] = value + + prefix = dirname + "/" # Add slash to prevent /X matching /XY + for maybe_suffix in link_map.keys(): + maybe_suffix += "/" # Add slash to prevent /X matching /XY + if maybe_suffix.startswith(prefix) or prefix.startswith(maybe_suffix): + link_map.pop(maybe_suffix) + + return keep_link_map + def _map_each_identity(v): return v diff --git a/python/private/py_info.bzl b/python/private/py_info.bzl index ef654c303e..4ecd02a438 100644 --- a/python/private/py_info.bzl +++ b/python/private/py_info.bzl @@ -42,7 +42,8 @@ def _PyInfo_init( direct_original_sources = depset(), transitive_original_sources = depset(), direct_pyi_files = depset(), - transitive_pyi_files = depset()): + transitive_pyi_files = depset(), + site_packages_symlinks = depset()): _check_arg_type("transitive_sources", "depset", transitive_sources) # Verify it's postorder compatible, but retain is original ordering. @@ -70,6 +71,7 @@ def _PyInfo_init( "has_py2_only_sources": has_py2_only_sources, "has_py3_only_sources": has_py2_only_sources, "imports": imports, + "site_packages_symlinks": site_packages_symlinks, "transitive_implicit_pyc_files": transitive_implicit_pyc_files, "transitive_implicit_pyc_source_files": transitive_implicit_pyc_source_files, "transitive_original_sources": transitive_original_sources, @@ -140,6 +142,34 @@ A depset of import path strings to be added to the `PYTHONPATH` of executable Python targets. These are accumulated from the transitive `deps`. The order of the depset is not guaranteed and may be changed in the future. It is recommended to use `default` order (the default). +""", + "site_packages_symlinks": """ +:type: depset[tuple[str | None, str]] + +A depset with `topological` ordering. + +Tuples of `(runfiles_path, site_packages_path)`. Where +* `runfiles_path` is a runfiles-root relative path. It is the path that + has the code to make importable. If `None` or empty string, then it means + to not create a site packages directory with the `site_packages_path` + name. +* `site_packages_path` is a path relative to the site-packages directory of + the venv for whatever creates the venv (typically py_binary). It makes + the code in `runfiles_path` available for import. Note that this + is created as a "raw" symlink (via `declare_symlink`). + +:::{include} /_includes/experimental_api.md +::: + +:::{tip} +The topological ordering means dependencies earlier and closer to the consumer +have precedence. This allows e.g. a binary to add dependencies that override +values from further way dependencies, such as forcing symlinks to point to +specific paths or preventing symlinks from being created. +::: + +:::{versionadded} VERSION_NEXT_FEATURE +::: """, "transitive_implicit_pyc_files": """ :type: depset[File] @@ -266,6 +296,7 @@ def PyInfoBuilder(): transitive_pyc_files = builders.DepsetBuilder(), transitive_pyi_files = builders.DepsetBuilder(), transitive_sources = builders.DepsetBuilder(), + site_packages_symlinks = builders.DepsetBuilder(order = "topological"), ) return self @@ -351,6 +382,7 @@ def _PyInfoBuilder_merge_all(self, transitive, *, direct = []): self.transitive_original_sources.add(info.transitive_original_sources) self.transitive_pyc_files.add(info.transitive_pyc_files) self.transitive_pyi_files.add(info.transitive_pyi_files) + self.site_packages_symlinks.add(info.site_packages_symlinks) return self @@ -400,6 +432,7 @@ def _PyInfoBuilder_build(self): transitive_original_sources = self.transitive_original_sources.build(), transitive_pyc_files = self.transitive_pyc_files.build(), transitive_pyi_files = self.transitive_pyi_files.build(), + site_packages_symlinks = self.site_packages_symlinks.build(), ) else: kwargs = {} diff --git a/python/private/py_library.bzl b/python/private/py_library.bzl index f6c7b12578..edd0db579f 100644 --- a/python/private/py_library.bzl +++ b/python/private/py_library.bzl @@ -14,6 +14,7 @@ """Common code for implementing py_library rules.""" load("@bazel_skylib//lib:dicts.bzl", "dicts") +load("@bazel_skylib//lib:paths.bzl", "paths") load("@bazel_skylib//rules:common_settings.bzl", "BuildSettingInfo") load(":attr_builders.bzl", "attrb") load( @@ -25,8 +26,21 @@ load( "REQUIRED_EXEC_GROUP_BUILDERS", ) load(":builders.bzl", "builders") -load(":common.bzl", "collect_cc_info", "collect_imports", "collect_runfiles", "create_instrumented_files_info", "create_library_semantics_struct", "create_output_group_info", "create_py_info", "filter_to_py_srcs", "get_imports") -load(":flags.bzl", "AddSrcsToRunfilesFlag", "PrecompileFlag") +load( + ":common.bzl", + "PYTHON_FILE_EXTENSIONS", + "collect_cc_info", + "collect_imports", + "collect_runfiles", + "create_instrumented_files_info", + "create_library_semantics_struct", + "create_output_group_info", + "create_py_info", + "filter_to_py_srcs", + "get_imports", + "runfiles_root_path", +) +load(":flags.bzl", "AddSrcsToRunfilesFlag", "PrecompileFlag", "VenvsSitePackages") load(":precompile.bzl", "maybe_precompile") load(":py_cc_link_params_info.bzl", "PyCcLinkParamsInfo") load(":py_internal.bzl", "py_internal") @@ -44,6 +58,46 @@ LIBRARY_ATTRS = dicts.add( PY_SRCS_ATTRS, IMPORTS_ATTRS, { + "experimental_venvs_site_packages": lambda: attrb.Label( + doc = """ +**INTERNAL ATTRIBUTE. SHOULD ONLY BE SET BY rules_python-INTERNAL CODE.** + +:::{include} /_includes/experimental_api.md +::: + +A flag that decides whether the library should treat its sources as a +site-packages layout. + +When the flag is `yes`, then the `srcs` files are treated as a site-packages +layout that is relative to the `imports` attribute. The `imports` attribute +can have only a single element. It is a repo-relative runfiles path. + +For example, in the `my/pkg/BUILD.bazel` file, given +`srcs=["site-packages/foo/bar.py"]`, specifying +`imports=["my/pkg/site-packages"]` means `foo/bar.py` is the file path +under the binary's venv site-packages directory that should be made available (i.e. +`import foo.bar` will work). + +`__init__.py` files are treated specially to provide basic support for [implicit +namespace packages]( +https://packaging.python.org/en/latest/guides/packaging-namespace-packages/#native-namespace-packages). +However, the *content* of the files cannot be taken into account, merely their +presence or absense. Stated another way: [pkgutil-style namespace packages]( +https://packaging.python.org/en/latest/guides/packaging-namespace-packages/#pkgutil-style-namespace-packages) +won't be understood as namespace packages; they'll be seen as regular packages. This will +likely lead to conflicts with other targets that contribute to the namespace. + +:::{tip} +This attributes populates {obj}`PyInfo.site_packages_symlinks`, which is +a topologically ordered depset. This means dependencies closer and earlier +to a consumer have precedence. See {obj}`PyInfo.site_packages_symlinks` for +more information. +::: + +:::{versionadded} VERSION_NEXT_FEATURE +::: +""", + ), "_add_srcs_to_runfiles_flag": lambda: attrb.Label( default = "//python/config_settings:add_srcs_to_runfiles", ), @@ -98,6 +152,11 @@ def py_library_impl(ctx, *, semantics): runfiles.add(collect_runfiles(ctx)) runfiles = runfiles.build(ctx) + imports = [] + site_packages_symlinks = [] + + imports, site_packages_symlinks = _get_imports_and_site_packages_symlinks(ctx, semantics) + cc_info = semantics.get_cc_info_for_library(ctx) py_info, deps_transitive_sources, builtins_py_info = create_py_info( ctx, @@ -106,7 +165,8 @@ def py_library_impl(ctx, *, semantics): required_pyc_files = required_pyc_files, implicit_pyc_files = implicit_pyc_files, implicit_pyc_source_files = implicit_pyc_source_files, - imports = collect_imports(ctx, semantics), + imports = imports, + site_packages_symlinks = site_packages_symlinks, ) # TODO(b/253059598): Remove support for extra actions; https://github.com/bazelbuild/bazel/issues/16455 @@ -144,6 +204,101 @@ Source files are no longer added to the runfiles directly. ::: """ +def _get_imports_and_site_packages_symlinks(ctx, semantics): + imports = depset() + site_packages_symlinks = depset() + if VenvsSitePackages.is_enabled(ctx): + site_packages_symlinks = _get_site_packages_symlinks(ctx) + else: + imports = collect_imports(ctx, semantics) + return imports, site_packages_symlinks + +def _get_site_packages_symlinks(ctx): + imports = ctx.attr.imports + if len(imports) == 0: + fail("When venvs_site_packages is enabled, exactly one `imports` " + + "value must be specified, got 0") + elif len(imports) > 1: + fail("When venvs_site_packages is enabled, exactly one `imports` " + + "value must be specified, got {}".format(imports)) + else: + site_packages_root = imports[0] + + if site_packages_root.endswith("/"): + fail("The site packages root value from `imports` cannot end in " + + "slash, got {}".format(site_packages_root)) + if site_packages_root.startswith("/"): + fail("The site packages root value from `imports` cannot start with " + + "slash, got {}".format(site_packages_root)) + + # Append slash to prevent incorrectly prefix-string matches + site_packages_root += "/" + + # We have to build a list of (runfiles path, site-packages path) pairs of + # the files to create in the consuming binary's venv site-packages directory. + # To minimize the number of files to create, we just return the paths + # to the directories containing the code of interest. + # + # However, namespace packages complicate matters: multiple + # distributions install in the same directory in site-packages. This + # works out because they don't overlap in their files. Typically, they + # install to different directories within the namespace package + # directory. Namespace package directories are simply directories + # within site-packages that *don't* have an `__init__.py` file, which + # can be arbitrarily deep. Thus, we simply have to look for the + # directories that _do_ have an `__init__.py` file and treat those as + # the path to symlink to. + + repo_runfiles_dirname = None + dirs_with_init = {} # dirname -> runfile path + for src in ctx.files.srcs: + if src.extension not in PYTHON_FILE_EXTENSIONS: + continue + path = _repo_relative_short_path(src.short_path) + if not path.startswith(site_packages_root): + continue + path = path.removeprefix(site_packages_root) + dir_name, _, filename = path.rpartition("/") + if not dir_name: + # This would be e.g. `site-packages/__init__.py`, which isn't valid + # because it's not within a directory for an importable Python package. + # However, the pypi integration over-eagerly adds a pkgutil-style + # __init__.py file during the repo phase. Just ignore them for now. + continue + + if filename.startswith("__init__."): + dirs_with_init[dir_name] = None + repo_runfiles_dirname = runfiles_root_path(ctx, src.short_path).partition("/")[0] + + # Sort so that we encounter `foo` before `foo/bar`. This ensures we + # see the top-most explicit package first. + dirnames = sorted(dirs_with_init.keys()) + first_level_explicit_packages = [] + for d in dirnames: + is_sub_package = False + for existing in first_level_explicit_packages: + # Suffix with / to prevent foo matching foobar + if d.startswith(existing + "/"): + is_sub_package = True + break + if not is_sub_package: + first_level_explicit_packages.append(d) + + site_packages_symlinks = [] + for dirname in first_level_explicit_packages: + site_packages_symlinks.append(( + paths.join(repo_runfiles_dirname, site_packages_root, dirname), + dirname, + )) + return site_packages_symlinks + +def _repo_relative_short_path(short_path): + # Convert `../+pypi+foo/some/file.py` to `some/file.py` + if short_path.startswith("../"): + return short_path[3:].partition("/")[2] + else: + return short_path + # NOTE: Exported publicaly def create_py_library_rule_builder(): """Create a rule builder for a py_library. diff --git a/python/private/pypi/whl_library_targets.bzl b/python/private/pypi/whl_library_targets.bzl index c390da2613..95031e6181 100644 --- a/python/private/pypi/whl_library_targets.bzl +++ b/python/private/pypi/whl_library_targets.bzl @@ -266,6 +266,7 @@ def whl_library_targets( ), tags = tags, visibility = impl_vis, + experimental_venvs_site_packages = Label("@rules_python//python/config_settings:venvs_site_packages"), ) def _config_settings(dependencies_by_platform, native = native, **kwargs): diff --git a/tests/modules/other/BUILD.bazel b/tests/modules/other/BUILD.bazel new file mode 100644 index 0000000000..e69de29bb2 diff --git a/tests/modules/other/MODULE.bazel b/tests/modules/other/MODULE.bazel new file mode 100644 index 0000000000..7cd3118b81 --- /dev/null +++ b/tests/modules/other/MODULE.bazel @@ -0,0 +1,3 @@ +module(name = "other") + +bazel_dep(name = "rules_python", version = "0") diff --git a/tests/modules/other/nspkg_delta/BUILD.bazel b/tests/modules/other/nspkg_delta/BUILD.bazel new file mode 100644 index 0000000000..457033aacf --- /dev/null +++ b/tests/modules/other/nspkg_delta/BUILD.bazel @@ -0,0 +1,10 @@ +load("@rules_python//python:py_library.bzl", "py_library") + +package(default_visibility = ["//visibility:public"]) + +py_library( + name = "nspkg_delta", + srcs = glob(["site-packages/**/*.py"]), + experimental_venvs_site_packages = "@rules_python//python/config_settings:venvs_site_packages", + imports = [package_name() + "/site-packages"], +) diff --git a/tests/modules/other/nspkg_delta/site-packages/nspkg/subnspkg/delta/__init__.py b/tests/modules/other/nspkg_delta/site-packages/nspkg/subnspkg/delta/__init__.py new file mode 100644 index 0000000000..bb7b160deb --- /dev/null +++ b/tests/modules/other/nspkg_delta/site-packages/nspkg/subnspkg/delta/__init__.py @@ -0,0 +1 @@ +# Intentionally empty diff --git a/tests/modules/other/nspkg_gamma/BUILD.bazel b/tests/modules/other/nspkg_gamma/BUILD.bazel new file mode 100644 index 0000000000..89038e80d2 --- /dev/null +++ b/tests/modules/other/nspkg_gamma/BUILD.bazel @@ -0,0 +1,10 @@ +load("@rules_python//python:py_library.bzl", "py_library") + +package(default_visibility = ["//visibility:public"]) + +py_library( + name = "nspkg_gamma", + srcs = glob(["site-packages/**/*.py"]), + experimental_venvs_site_packages = "@rules_python//python/config_settings:venvs_site_packages", + imports = [package_name() + "/site-packages"], +) diff --git a/tests/modules/other/nspkg_gamma/site-packages/nspkg/subnspkg/gamma/__init__.py b/tests/modules/other/nspkg_gamma/site-packages/nspkg/subnspkg/gamma/__init__.py new file mode 100644 index 0000000000..bb7b160deb --- /dev/null +++ b/tests/modules/other/nspkg_gamma/site-packages/nspkg/subnspkg/gamma/__init__.py @@ -0,0 +1 @@ +# Intentionally empty diff --git a/tests/pypi/whl_library_targets/whl_library_targets_tests.bzl b/tests/pypi/whl_library_targets/whl_library_targets_tests.bzl index a042ed0346..f738e03b5d 100644 --- a/tests/pypi/whl_library_targets/whl_library_targets_tests.bzl +++ b/tests/pypi/whl_library_targets/whl_library_targets_tests.bzl @@ -273,6 +273,7 @@ def _test_whl_and_library_deps(env): ), "tags": ["tag1", "tag2"], "visibility": ["//visibility:public"], + "experimental_venvs_site_packages": Label("//python/config_settings:venvs_site_packages"), }, ]) # buildifier: @unsorted-dict-items @@ -335,6 +336,7 @@ def _test_group(env): }), "tags": [], "visibility": ["@pypi__groups//:__pkg__"], + "experimental_venvs_site_packages": Label("//python/config_settings:venvs_site_packages"), }, ]) # buildifier: @unsorted-dict-items diff --git a/tests/support/sh_py_run_test.bzl b/tests/support/sh_py_run_test.bzl index 7b3b617da1..9c8134ff40 100644 --- a/tests/support/sh_py_run_test.bzl +++ b/tests/support/sh_py_run_test.bzl @@ -40,6 +40,8 @@ def _perform_transition_impl(input_settings, attr, base_impl): settings["//python/bin:python_src"] = attr.python_src if attr.venvs_use_declare_symlink: settings["//python/config_settings:venvs_use_declare_symlink"] = attr.venvs_use_declare_symlink + if attr.venvs_site_packages: + settings["//python/config_settings:venvs_site_packages"] = attr.venvs_site_packages return settings _RECONFIG_INPUTS = [ @@ -47,6 +49,7 @@ _RECONFIG_INPUTS = [ "//python/bin:python_src", "//command_line_option:extra_toolchains", "//python/config_settings:venvs_use_declare_symlink", + "//python/config_settings:venvs_site_packages", ] _RECONFIG_OUTPUTS = _RECONFIG_INPUTS + [ "//command_line_option:build_python_zip", @@ -67,6 +70,7 @@ toolchain. """, ), "python_src": attrb.Label(), + "venvs_site_packages": attrb.String(), "venvs_use_declare_symlink": attrb.String(), } diff --git a/tests/venv_site_packages_libs/BUILD.bazel b/tests/venv_site_packages_libs/BUILD.bazel new file mode 100644 index 0000000000..5d02708800 --- /dev/null +++ b/tests/venv_site_packages_libs/BUILD.bazel @@ -0,0 +1,17 @@ +load("//tests/support:sh_py_run_test.bzl", "py_reconfig_test") +load("//tests/support:support.bzl", "SUPPORTS_BOOTSTRAP_SCRIPT") + +py_reconfig_test( + name = "venvs_site_packages_libs_test", + srcs = ["bin.py"], + bootstrap_impl = "script", + main = "bin.py", + target_compatible_with = SUPPORTS_BOOTSTRAP_SCRIPT, + venvs_site_packages = "yes", + deps = [ + "//tests/venv_site_packages_libs/nspkg_alpha", + "//tests/venv_site_packages_libs/nspkg_beta", + "@other//nspkg_delta", + "@other//nspkg_gamma", + ], +) diff --git a/tests/venv_site_packages_libs/bin.py b/tests/venv_site_packages_libs/bin.py new file mode 100644 index 0000000000..b944be69e3 --- /dev/null +++ b/tests/venv_site_packages_libs/bin.py @@ -0,0 +1,32 @@ +import importlib +import os +import sys +import unittest + + +class VenvSitePackagesLibraryTest(unittest.TestCase): + def setUp(self): + super().setUp() + if sys.prefix == sys.base_prefix: + raise AssertionError("Not running under a venv") + self.venv = sys.prefix + + def assert_imported_from_venv(self, module_name): + module = importlib.import_module(module_name) + self.assertEqual(module.__name__, module_name) + self.assertTrue( + module.__file__.startswith(self.venv), + f"\n{module_name} was imported, but not from the venv.\n" + + f"venv : {self.venv}\n" + + f"actual: {module.__file__}", + ) + + def test_imported_from_venv(self): + self.assert_imported_from_venv("nspkg.subnspkg.alpha") + self.assert_imported_from_venv("nspkg.subnspkg.beta") + self.assert_imported_from_venv("nspkg.subnspkg.gamma") + self.assert_imported_from_venv("nspkg.subnspkg.delta") + + +if __name__ == "__main__": + unittest.main() diff --git a/tests/venv_site_packages_libs/nspkg_alpha/BUILD.bazel b/tests/venv_site_packages_libs/nspkg_alpha/BUILD.bazel new file mode 100644 index 0000000000..c40c3b4080 --- /dev/null +++ b/tests/venv_site_packages_libs/nspkg_alpha/BUILD.bazel @@ -0,0 +1,10 @@ +load("@rules_python//python:py_library.bzl", "py_library") + +package(default_visibility = ["//visibility:public"]) + +py_library( + name = "nspkg_alpha", + srcs = glob(["site-packages/**/*.py"]), + experimental_venvs_site_packages = "//python/config_settings:venvs_site_packages", + imports = [package_name() + "/site-packages"], +) diff --git a/tests/venv_site_packages_libs/nspkg_alpha/site-packages/nspkg/subnspkg/alpha/__init__.py b/tests/venv_site_packages_libs/nspkg_alpha/site-packages/nspkg/subnspkg/alpha/__init__.py new file mode 100644 index 0000000000..b5ee093672 --- /dev/null +++ b/tests/venv_site_packages_libs/nspkg_alpha/site-packages/nspkg/subnspkg/alpha/__init__.py @@ -0,0 +1 @@ +whoami = "alpha" diff --git a/tests/venv_site_packages_libs/nspkg_beta/BUILD.bazel b/tests/venv_site_packages_libs/nspkg_beta/BUILD.bazel new file mode 100644 index 0000000000..5d402183bd --- /dev/null +++ b/tests/venv_site_packages_libs/nspkg_beta/BUILD.bazel @@ -0,0 +1,10 @@ +load("@rules_python//python:py_library.bzl", "py_library") + +package(default_visibility = ["//visibility:public"]) + +py_library( + name = "nspkg_beta", + srcs = glob(["site-packages/**/*.py"]), + experimental_venvs_site_packages = "//python/config_settings:venvs_site_packages", + imports = [package_name() + "/site-packages"], +) diff --git a/tests/venv_site_packages_libs/nspkg_beta/site-packages/nspkg/subnspkg/beta/__init__.py b/tests/venv_site_packages_libs/nspkg_beta/site-packages/nspkg/subnspkg/beta/__init__.py new file mode 100644 index 0000000000..a2a65910c7 --- /dev/null +++ b/tests/venv_site_packages_libs/nspkg_beta/site-packages/nspkg/subnspkg/beta/__init__.py @@ -0,0 +1 @@ +whoami = "beta" diff --git a/tests/venv_site_packages_libs/venv_site_packages_pypi_test.py b/tests/venv_site_packages_libs/venv_site_packages_pypi_test.py new file mode 100644 index 0000000000..519b258044 --- /dev/null +++ b/tests/venv_site_packages_libs/venv_site_packages_pypi_test.py @@ -0,0 +1,36 @@ +import os +import sys +import unittest + + +class VenvSitePackagesLibraryTest(unittest.TestCase): + def test_imported_from_venv(self): + self.assertNotEqual(sys.prefix, sys.base_prefix, "Not running under a venv") + venv = sys.prefix + + from nspkg.subnspkg import alpha + + self.assertEqual(alpha.whoami, "alpha") + self.assertEqual(alpha.__name__, "nspkg.subnspkg.alpha") + + self.assertTrue( + alpha.__file__.startswith(sys.prefix), + f"\nalpha was imported, not from within the venv.\n" + + f"venv : {venv}\n" + + f"actual: {alpha.__file__}", + ) + + from nspkg.subnspkg import beta + + self.assertEqual(beta.whoami, "beta") + self.assertEqual(beta.__name__, "nspkg.subnspkg.beta") + self.assertTrue( + beta.__file__.startswith(sys.prefix), + f"\nbeta was imported, not from within the venv.\n" + + f"venv : {venv}\n" + + f"actual: {beta.__file__}", + ) + + +if __name__ == "__main__": + unittest.main() From e5fa023b27cf3583eb9e45efcbcb887e660ce65f Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Sat, 5 Apr 2025 10:05:08 -0700 Subject: [PATCH 147/922] docs: fix a few xrefs (#2740) Fixes a few xrefs in the docs that had typos or missing external bazel links. --- CHANGELOG.md | 2 +- docs/api/rules_python/python/config_settings/index.md | 2 +- docs/toolchains.md | 4 ++-- python/private/py_executable.bzl | 2 +- sphinxdocs/inventories/bazel_inventory.txt | 8 ++++++++ 5 files changed, 13 insertions(+), 5 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 818773e589..5172e742c9 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -68,7 +68,7 @@ Unreleased changes template. using `experimental_index_url`. * (toolchains) Remove all but `3.8.20` versions of the Python `3.8` interpreter who has reached EOL. If users still need other versions of the `3.8` interpreter, please supply - the URLs manually {bzl:ob}`python.toolchain` or {bzl:obj}`python_register_toolchains` calls. + the URLs manually {bzl:obj}`python.toolchain` or {bzl:obj}`python_register_toolchains` calls. * (pypi) The PyPI extension will no longer write the lock file entries as the extension has been marked reproducible. Fixes [#2434](https://github.com/bazel-contrib/rules_python/issues/2434). diff --git a/docs/api/rules_python/python/config_settings/index.md b/docs/api/rules_python/python/config_settings/index.md index 340335d9b1..ed6444298e 100644 --- a/docs/api/rules_python/python/config_settings/index.md +++ b/docs/api/rules_python/python/config_settings/index.md @@ -46,7 +46,7 @@ of builtin, known versions. If you need to match a version that isn't present, then you have two options: 1. Manually define a `config_setting` and have it match {obj}`--python_version` - or {ob}`python_version_major_minor`. This works best when you don't control the + or {obj}`python_version_major_minor`. This works best when you don't control the root module, or don't want to rely on the MODULE.bazel configuration. Such a config settings would look like: ``` diff --git a/docs/toolchains.md b/docs/toolchains.md index 0e4f5c2321..73a8a48121 100644 --- a/docs/toolchains.md +++ b/docs/toolchains.md @@ -265,7 +265,7 @@ use_repo(python, "python_3_10", "python_3_10_host") ``` Note, the user has to import the `*_host` repository to use the python interpreter in the -{bzl:obj}`pip_parse` and {bzl:obj}`whl_library` repository rules and once that is done +{bzl:obj}`pip_parse` and `whl_library` repository rules and once that is done users should be able to ensure the setting of the default toolchain even during the transition period when some of the code is still defined in `WORKSPACE`. @@ -364,7 +364,7 @@ toolchains a "toolchain suite". One of the underlying design goals of the toolchains is to support complex and bespoke environments. Such environments may use an arbitrary combination of -{obj}`RBE`, cross-platform building, multiple Python versions, +{bzl:obj}`RBE`, cross-platform building, multiple Python versions, building Python from source, embeding Python (as opposed to building separate interpreters), using prebuilt binaries, or using binaries built from source. To that end, many of the attributes they accept, and fields they provide, are diff --git a/python/private/py_executable.bzl b/python/private/py_executable.bzl index f33c2b6ca1..e6f4700b20 100644 --- a/python/private/py_executable.bzl +++ b/python/private/py_executable.bzl @@ -92,7 +92,7 @@ Only supported for {obj}`--bootstrap_impl=script`. Ignored otherwise. ::: :::{seealso} -The {obj}`RULES_PYTHON_ADDITIONAL_INTERPRETER_ARGS` environment variable +The {any}`RULES_PYTHON_ADDITIONAL_INTERPRETER_ARGS` environment variable ::: :::{versionadded} 1.3.0 diff --git a/sphinxdocs/inventories/bazel_inventory.txt b/sphinxdocs/inventories/bazel_inventory.txt index dc11f02b5b..458126a849 100644 --- a/sphinxdocs/inventories/bazel_inventory.txt +++ b/sphinxdocs/inventories/bazel_inventory.txt @@ -28,6 +28,14 @@ attr.string_list bzl:type 1 rules/lib/toplevel/attr#string_list - attr.string_list_dict bzl:type 1 rules/lib/toplevel/attr#string_list_dict - bool bzl:type 1 rules/lib/bool - callable bzl:type 1 rules/lib/core/function - +config bzl:obj 1 rules/lib/toplevel/config - +config.bool bzl:function 1 rules/lib/toplevel/config#bool - +config.exec bzl:function 1 rules/lib/toplevel/config#exec - +config.int bzl:function 1 rules/lib/toplevel/config#int - +config.none bzl:function 1 rules/lib/toplevel/config#none - +config.string bzl:function 1 rules/lib/toplevel/config#string - +config.string_list bzl:function 1 rules/lib/toplevel/config#string_list - +config.target bzl:function 1 rules/lib/toplevel/config#target - config_common.FeatureFlagInfo bzl:type 1 rules/lib/toplevel/config_common#FeatureFlagInfo - config_common.toolchain_type bzl:function 1 rules/lib/toplevel/config_common#toolchain_type - ctx.actions bzl:obj 1 rules/lib/builtins/ctx#actions - From 6854dc3880b1ff81659ad4a36fb2e6551f41d0e2 Mon Sep 17 00:00:00 2001 From: Matt Mackay Date: Sat, 5 Apr 2025 14:42:03 -0400 Subject: [PATCH 148/922] fix: treat ignore_root_user_error either ignored or warning (#2739) Previously [#2636](https://github.com/bazel-contrib/rules_python/pull/2636) changed the semantics of `ignore_root_user_error` from "ignore" to "warning". This is now flipped back to ignoring the issue, and will only emit a warning when the attribute is set `False`. This does also change the semantics of what #2636 did by flipping the attribute, as now there is no warning, and the user would have to explicitly set it to `False` (they don't want to ignore the error) to see the warning. Co-authored-by: Richard Levasseur --- CHANGELOG.md | 4 +++ python/private/python.bzl | 4 +-- python/private/python_repository.bzl | 40 +++++++++++++++------------- 3 files changed, 27 insertions(+), 21 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 5172e742c9..dbb0c03e59 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -69,6 +69,10 @@ Unreleased changes template. * (toolchains) Remove all but `3.8.20` versions of the Python `3.8` interpreter who has reached EOL. If users still need other versions of the `3.8` interpreter, please supply the URLs manually {bzl:obj}`python.toolchain` or {bzl:obj}`python_register_toolchains` calls. +* (toolchains) Previously [#2636](https://github.com/bazel-contrib/rules_python/pull/2636) + changed the semantics of `ignore_root_user_error` from "ignore" to "warning". This is now + flipped back to ignoring the issue, and will only emit a warning when the attribute is set + `False`. * (pypi) The PyPI extension will no longer write the lock file entries as the extension has been marked reproducible. Fixes [#2434](https://github.com/bazel-contrib/rules_python/issues/2434). diff --git a/python/private/python.bzl b/python/private/python.bzl index 296fb0ab7d..efc429420e 100644 --- a/python/private/python.bzl +++ b/python/private/python.bzl @@ -803,8 +803,8 @@ to spurious cache misses or build failures). However, if the user is running Bazel as root, this read-onlyness is not respected. Bazel will print a warning message when it detects that the runtime installation is writable despite being made read only (i.e. it's running with -root access). If this attribute is set to `False`, Bazel will make it a hard -error to run with root access instead. +root access) while this attribute is set `False`, however this messaging can be ignored by setting +this to `False`. """, mandatory = False, ), diff --git a/python/private/python_repository.bzl b/python/private/python_repository.bzl index f3ec13d67d..cfc06452a9 100644 --- a/python/private/python_repository.bzl +++ b/python/private/python_repository.bzl @@ -137,28 +137,30 @@ def _python_repository_impl(rctx): logger = logger, ) - fail_or_warn = logger.warn if rctx.attr.ignore_root_user_error else logger.fail - exec_result = repo_utils.execute_unchecked( - rctx, - op = "python_repository.TestReadOnly", - arguments = [repo_utils.which_checked(rctx, "touch"), "lib/.test"], - logger = logger, - ) - - # The issue with running as root is the installation is no longer - # read-only, so the problems due to pyc can resurface. - if exec_result.return_code == 0: - stdout = repo_utils.execute_checked_stdout( + # If the user is not ignoring the warnings, then proceed to run a check, + # otherwise these steps can be skipped, as they both result in some warning. + if not rctx.attr.ignore_root_user_error: + exec_result = repo_utils.execute_unchecked( rctx, - op = "python_repository.GetUserId", - arguments = [repo_utils.which_checked(rctx, "id"), "-u"], + op = "python_repository.TestReadOnly", + arguments = [repo_utils.which_checked(rctx, "touch"), "lib/.test"], logger = logger, ) - uid = int(stdout.strip()) - if uid == 0: - fail_or_warn("The current user is root, which can cause spurious cache misses or build failures with the hermetic Python interpreter. See https://github.com/bazel-contrib/rules_python/pull/713.") - else: - fail_or_warn("The current user has CAP_DAC_OVERRIDE set, which can cause spurious cache misses or build failures with the hermetic Python interpreter. See https://github.com/bazel-contrib/rules_python/pull/713.") + + # The issue with running as root is the installation is no longer + # read-only, so the problems due to pyc can resurface. + if exec_result.return_code == 0: + stdout = repo_utils.execute_checked_stdout( + rctx, + op = "python_repository.GetUserId", + arguments = [repo_utils.which_checked(rctx, "id"), "-u"], + logger = logger, + ) + uid = int(stdout.strip()) + if uid == 0: + logger.warn("The current user is root, which can cause spurious cache misses or build failures with the hermetic Python interpreter. See https://github.com/bazel-contrib/rules_python/pull/713.") + else: + logger.warn("The current user has CAP_DAC_OVERRIDE set, which can cause spurious cache misses or build failures with the hermetic Python interpreter. See https://github.com/bazel-contrib/rules_python/pull/713.") python_bin = "python.exe" if ("windows" in platform) else "bin/python3" From 7f5a1b5a0e6fbe29c5c33d8e164b4cda6ded99b7 Mon Sep 17 00:00:00 2001 From: Matt Mackay Date: Sat, 5 Apr 2025 18:48:14 -0400 Subject: [PATCH 149/922] fix: Ensure temporary .pyc & .pyo files are excluded from the interpreters repository files (#2743) We've seen cases the temporary versions for the `.pyc` and `.pyo` files are unstable on certain interpreter toolchains. The temp files take for form of `.pyc.NNN`, so the amended glob patten will still match both the `.pyc` and `.pyc.NNN` versions of the file names. --------- Co-authored-by: Richard Levasseur --- CHANGELOG.md | 1 + python/private/python_repository.bzl | 5 +++-- 2 files changed, 4 insertions(+), 2 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index dbb0c03e59..abe718c389 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -90,6 +90,7 @@ Unreleased changes template. transitions transitioning on the `python_version` flag. Fixes [#2685](https://github.com/bazel-contrib/rules_python/issues/2685). * (toolchains) Run the check on the Python interpreter in isolated mode, to ensure it's not affected by userland environment variables, such as `PYTHONPATH`. +* (toolchains) Ensure temporary `.pyc` and `.pyo` files are also excluded from the interpreters repository files. {#v0-0-0-added} ### Added diff --git a/python/private/python_repository.bzl b/python/private/python_repository.bzl index cfc06452a9..fd86b415cc 100644 --- a/python/private/python_repository.bzl +++ b/python/private/python_repository.bzl @@ -193,8 +193,9 @@ def _python_repository_impl(rctx): # Exclude them from the glob because otherwise between the first time and second time a python toolchain is used," # the definition of this filegroup will change, and depending rules will get invalidated." # See https://github.com/bazel-contrib/rules_python/issues/1008 for unconditionally adding these to toolchains so we can stop ignoring them." - "**/__pycache__/*.pyc", - "**/__pycache__/*.pyo", + # pyc* is ignored because pyc creation creates temporary .pyc.NNNN files + "**/__pycache__/*.pyc*", + "**/__pycache__/*.pyo*", ] if "windows" in platform: From da0e52f59047ab47bcb561787d42a8f93537dc41 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Sat, 5 Apr 2025 16:47:44 -0700 Subject: [PATCH 150/922] chore: remove unnecessary DEFAULT_BOOTSTRAP_TEMPLATE global (#2744) I think the DEFAULT_BOOTSTRAP_TEMPLATE global was used by something in the original Bazel impl, but now it's just used in one place. Remove the shared global and just inline the single usage. --- python/private/py_runtime_info.bzl | 2 -- python/private/py_runtime_rule.bzl | 4 ++-- 2 files changed, 2 insertions(+), 4 deletions(-) diff --git a/python/private/py_runtime_info.bzl b/python/private/py_runtime_info.bzl index 19857c9ede..4297391068 100644 --- a/python/private/py_runtime_info.bzl +++ b/python/private/py_runtime_info.bzl @@ -17,8 +17,6 @@ load(":util.bzl", "define_bazel_6_provider") DEFAULT_STUB_SHEBANG = "#!/usr/bin/env python3" -DEFAULT_BOOTSTRAP_TEMPLATE = Label("//python/private:bootstrap_template") - _PYTHON_VERSION_VALUES = ["PY2", "PY3"] def _optional_int(value): diff --git a/python/private/py_runtime_rule.bzl b/python/private/py_runtime_rule.bzl index 3dc00baa12..a85f5b25f2 100644 --- a/python/private/py_runtime_rule.bzl +++ b/python/private/py_runtime_rule.bzl @@ -19,7 +19,7 @@ load("@bazel_skylib//rules:common_settings.bzl", "BuildSettingInfo") load(":attributes.bzl", "NATIVE_RULES_ALLOWLIST_ATTRS") load(":flags.bzl", "FreeThreadedFlag") load(":py_internal.bzl", "py_internal") -load(":py_runtime_info.bzl", "DEFAULT_BOOTSTRAP_TEMPLATE", "DEFAULT_STUB_SHEBANG", "PyRuntimeInfo") +load(":py_runtime_info.bzl", "DEFAULT_STUB_SHEBANG", "PyRuntimeInfo") load(":reexports.bzl", "BuiltinPyRuntimeInfo") load(":util.bzl", "IS_BAZEL_7_OR_HIGHER") @@ -201,7 +201,7 @@ If not set, then it will be set based on flags. ), "bootstrap_template": attr.label( allow_single_file = True, - default = DEFAULT_BOOTSTRAP_TEMPLATE, + default = Label("//python/private:bootstrap_template"), doc = """ The bootstrap script template file to use. Should have %python_binary%, %workspace_name%, %main%, and %imports%. From 996ae2658bffe7163a5abc384eff57ff28d4f409 Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Mon, 7 Apr 2025 23:16:24 +0000 Subject: [PATCH 151/922] build(deps): bump jinja2 from 3.1.4 to 3.1.6 in /docs (#2750) Bumps [jinja2](https://github.com/pallets/jinja) from 3.1.4 to 3.1.6.
Release notes

Sourced from jinja2's releases.

3.1.6

This is the Jinja 3.1.6 security release, which fixes security issues but does not otherwise change behavior and should not result in breaking changes compared to the latest feature release.

PyPI: https://pypi.org/project/Jinja2/3.1.6/ Changes: https://jinja.palletsprojects.com/en/stable/changes/#version-3-1-6

3.1.5

This is the Jinja 3.1.5 security fix release, which fixes security issues and bugs but does not otherwise change behavior and should not result in breaking changes compared to the latest feature release.

PyPI: https://pypi.org/project/Jinja2/3.1.5/ Changes: https://jinja.palletsprojects.com/changes/#version-3-1-5 Milestone: https://github.com/pallets/jinja/milestone/16?closed=1

  • The sandboxed environment handles indirect calls to str.format, such as by passing a stored reference to a filter that calls its argument. GHSA-q2x7-8rv6-6q7h
  • Escape template name before formatting it into error messages, to avoid issues with names that contain f-string syntax. #1792, GHSA-gmj6-6f8f-6699
  • Sandbox does not allow clear and pop on known mutable sequence types. #2032
  • Calling sync render for an async template uses asyncio.run. #1952
  • Avoid unclosed auto_aiter warnings. #1960
  • Return an aclose-able AsyncGenerator from Template.generate_async. #1960
  • Avoid leaving root_render_func() unclosed in Template.generate_async. #1960
  • Avoid leaving async generators unclosed in blocks, includes and extends. #1960
  • The runtime uses the correct concat function for the current environment when calling block references. #1701
  • Make |unique async-aware, allowing it to be used after another async-aware filter. #1781
  • |int filter handles OverflowError from scientific notation. #1921
  • Make compiling deterministic for tuple unpacking in a {% set ... %} call. #2021
  • Fix dunder protocol (copy/pickle/etc) interaction with Undefined objects. #2025
  • Fix copy/pickle support for the internal missing object. #2027
  • Environment.overlay(enable_async) is applied correctly. #2061
  • The error message from FileSystemLoader includes the paths that were searched. #1661
  • PackageLoader shows a clearer error message when the package does not contain the templates directory. #1705
  • Improve annotations for methods returning copies. #1880
  • urlize does not add mailto: to values like @a@b. #1870
  • Tests decorated with @pass_context can be used with the |select filter. #1624
  • Using set for multiple assignment (a, b = 1, 2) does not fail when the target is a namespace attribute. #1413
  • Using set in all branches of {% if %}{% elif %}{% else %} blocks does not cause the variable to be considered initially undefined. #1253
Changelog

Sourced from jinja2's changelog.

Version 3.1.6

Released 2025-03-05

  • The |attr filter does not bypass the environment's attribute lookup, allowing the sandbox to apply its checks. :ghsa:cpwx-vrp4-4pq7

Version 3.1.5

Released 2024-12-21

  • The sandboxed environment handles indirect calls to str.format, such as by passing a stored reference to a filter that calls its argument. :ghsa:q2x7-8rv6-6q7h
  • Escape template name before formatting it into error messages, to avoid issues with names that contain f-string syntax. :issue:1792, :ghsa:gmj6-6f8f-6699
  • Sandbox does not allow clear and pop on known mutable sequence types. :issue:2032
  • Calling sync render for an async template uses asyncio.run. :pr:1952
  • Avoid unclosed auto_aiter warnings. :pr:1960
  • Return an aclose-able AsyncGenerator from Template.generate_async. :pr:1960
  • Avoid leaving root_render_func() unclosed in Template.generate_async. :pr:1960
  • Avoid leaving async generators unclosed in blocks, includes and extends. :pr:1960
  • The runtime uses the correct concat function for the current environment when calling block references. :issue:1701
  • Make |unique async-aware, allowing it to be used after another async-aware filter. :issue:1781
  • |int filter handles OverflowError from scientific notation. :issue:1921
  • Make compiling deterministic for tuple unpacking in a {% set ... %} call. :issue:2021
  • Fix dunder protocol (copy/pickle/etc) interaction with Undefined objects. :issue:2025
  • Fix copy/pickle support for the internal missing object. :issue:2027
  • Environment.overlay(enable_async) is applied correctly. :pr:2061
  • The error message from FileSystemLoader includes the paths that were searched. :issue:1661
  • PackageLoader shows a clearer error message when the package does not contain the templates directory. :issue:1705
  • Improve annotations for methods returning copies. :pr:1880
  • urlize does not add mailto: to values like @a@b. :pr:1870

... (truncated)

Commits

[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=jinja2&package-manager=pip&previous-version=3.1.4&new-version=3.1.6)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) ---
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot show ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> --- docs/requirements.txt | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/requirements.txt b/docs/requirements.txt index e838daca8f..0b4909535a 100644 --- a/docs/requirements.txt +++ b/docs/requirements.txt @@ -148,9 +148,9 @@ imagesize==1.4.1 \ --hash=sha256:0d8d18d08f840c19d0ee7ca1fd82490fdc3729b7ac93f49870406ddde8ef8d8b \ --hash=sha256:69150444affb9cb0d5cc5a92b3676f0b2fb7cd9ae39e947a5e11a36b4497cd4a # via sphinx -jinja2==3.1.4 \ - --hash=sha256:4a3aee7acbbe7303aede8e9648d13b8bf88a429282aa6122a993f0ac800cb369 \ - --hash=sha256:bc5dd2abb727a5319567b7a813e6a2e7318c39f4f487cfe6c89c6f9c7d25197d +jinja2==3.1.6 \ + --hash=sha256:0137fb05990d35f1275a587e9aee6d56da821fc83491a0fb838183be43f66d6d \ + --hash=sha256:85ece4451f492d0c13c5dd7c13a64681a86afae63a5f347908daf103ce6d2f67 # via # myst-parser # readthedocs-sphinx-ext From 8bda670add1c490477a3ac9914405c802a087847 Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Mon, 7 Apr 2025 23:18:33 +0000 Subject: [PATCH 152/922] build(deps): bump absl-py from 2.1.0 to 2.2.2 in /docs (#2751) Bumps [absl-py](https://github.com/abseil/abseil-py) from 2.1.0 to 2.2.2.
Release notes

Sourced from absl-py's releases.

v2.2.2

Added

  • (testing) Added a new method absltest.TestCase.assertMappingEqual that tests equality of Mapping objects not requiring them to be dicts. Similar to assertSequenceEqual but for mappings.
  • (testing) Added a new method absltest.assertDictContainsSubset that checks that a dictionary contains a subset of keys and values. Similar to a removed method unittest.assertDictContainsSubset (existed until Python 3.11).
  • Added type annotations that are compliant with MyPy.

Changed

  • Removed support for Python 3.7.

Fixed

  • (testing) Fixed an issue where the test reporter crashes with exceptions with no string representation, starting with Python 3.11.

(The change log also includes changes in 2.2.0 and 2.2.1.)

Changelog

Sourced from absl-py's changelog.

Python Absl Changelog

All notable changes to Python Absl are recorded here.

The format is based on Keep a Changelog.

Unreleased

Nothing notable unreleased.

  • (testing) Added a new method absltest.TestCase.assertMappingEqual that tests equality of Mapping objects not requiring them to be dicts. Similar to assertSequenceEqual but for mappings.

  • (testing) Added a new method absltest.assertDictContainsSubset that checks that a dictionary contains a subset of keys and values. Similar to a removed method unittest.assertDictContainsSubset (existed until Python 3.11).

Fixed

  • (testing) Fixed an issue where the test reporter crashes with exceptions with no string representation, starting with Python 3.11.
Commits
  • 4de3812 Fixing a typo in hex regex in logging_functional_test.py
  • e889843 Exclude files and bump version to 2.2.2
  • d45bb4b Bump absl-py version to 2.2.1 to prepare for a release
  • 014aa0a Fixing the behavior of assertDictAlmostEqual
  • 57ea862 Bump absl-py version to 2.2 to prepare for a release
  • 214f0ff Changing assertMappingEqual to support arbitrary equality function. Also addi...
  • c98852f Avoid double negation in the error message for required flags.
  • f1cd92d Updating string substitution with modern f-string style in assertMappingEqual...
  • f63fe8d pytype fails to build the target in Python 3.12. suppress a misleading type w...
  • 6609299 Minor improvements of assertDictContainsSubset method.
  • Additional commits viewable in compare view

[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=absl-py&package-manager=pip&previous-version=2.1.0&new-version=2.2.2)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) ---
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot show ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> --- docs/requirements.txt | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/requirements.txt b/docs/requirements.txt index 0b4909535a..66d41a963f 100644 --- a/docs/requirements.txt +++ b/docs/requirements.txt @@ -2,9 +2,9 @@ # bazel run //docs:requirements.update --index-url https://pypi.org/simple -absl-py==2.1.0 \ - --hash=sha256:526a04eadab8b4ee719ce68f204172ead1027549089702d99b9059f129ff1308 \ - --hash=sha256:7820790efbb316739cde8b4e19357243fc3608a152024288513dd968d7d959ff +absl-py==2.2.2 \ + --hash=sha256:bf25b2c2eed013ca456918c453d687eab4e8309fba81ee2f4c1a6aa2494175eb \ + --hash=sha256:e5797bc6abe45f64fd95dc06394ca3f2bedf3b5d895e9da691c9ee3397d70092 # via rules-python-docs (docs/pyproject.toml) alabaster==1.0.0 \ --hash=sha256:c00dca57bca26fa62a6d7d0a9fcce65f3e026e9bfe33e9c538fd3fbb2144fd9e \ From 23157f96117cc82adb540030e9da737b8811608d Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Tue, 8 Apr 2025 10:46:41 +0900 Subject: [PATCH 153/922] build(deps): bump charset-normalizer from 3.4.0 to 3.4.1 in /tools/publish (#2753) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Bumps [charset-normalizer](https://github.com/jawah/charset_normalizer) from 3.4.0 to 3.4.1.
Release notes

Sourced from charset-normalizer's releases.

Version 3.4.1

🚀 We're still raising awareness around HTTP/2, and HTTP/3!

Did you know that Internet Explorer 11 shipped with an optional HTTP/2 support back in 2013? also libcurl did ship it in 2014[...] Using Requests today is the rough equivalent of using EOL Windows 8! We promptly invite Python developers to look at the first drop-in replacement for Requests, namely Niquests. Ship with native WebSocket, SSE, Happy Eyeballs, DNS over HTTPS, and so on[...] All of this while remaining compatible with all Requests prior plug-ins / add-ons.

It leverages charset-normalizer in a better way! Check it out, you will gain up to being 3X faster and get a real/respectable support with it.

3.4.1 (2024-12-24)

Changed

  • Project metadata are now stored using pyproject.toml instead of setup.cfg using setuptools as the build backend.
  • Enforce annotation delayed loading for a simpler and consistent types in the project.
  • Optional mypyc compilation upgraded to version 1.14 for Python >= 3.8

Added

  • pre-commit configuration.
  • noxfile.

Removed

  • build-requirements.txt as per using pyproject.toml native build configuration.
  • bin/integration.py and bin/serve.py in favor of downstream integration test (see noxfile).
  • setup.cfg in favor of pyproject.toml metadata configuration.
  • Unused utils.range_scan function.

Fixed

  • Converting content to Unicode bytes may insert utf_8 instead of preferred utf-8. (#572)
  • Deprecation warning "'count' is passed as positional argument" when converting to Unicode bytes on Python 3.13+
Changelog

Sourced from charset-normalizer's changelog.

3.4.1 (2024-12-24)

Changed

  • Project metadata are now stored using pyproject.toml instead of setup.cfg using setuptools as the build backend.
  • Enforce annotation delayed loading for a simpler and consistent types in the project.
  • Optional mypyc compilation upgraded to version 1.14 for Python >= 3.8

Added

  • pre-commit configuration.
  • noxfile.

Removed

  • build-requirements.txt as per using pyproject.toml native build configuration.
  • bin/integration.py and bin/serve.py in favor of downstream integration test (see noxfile).
  • setup.cfg in favor of pyproject.toml metadata configuration.
  • Unused utils.range_scan function.

Fixed

  • Converting content to Unicode bytes may insert utf_8 instead of preferred utf-8. (#572)
  • Deprecation warning "'count' is passed as positional argument" when converting to Unicode bytes on Python 3.13+
Commits
  • ffdf7f5 :wrench: fix long description content-type inferred as rst instead of md
  • c7197b7 :pencil: fix changelog entries (#582)
  • c390e1f Merge pull request #581 from jawah/refresh-part-2
  • f9d6b8c :lock: add CODEOWNERS
  • 7ce1ef1 :wrench: use ubuntu-22.04 for cibuildwheel in continuous deployment workflow
  • deed205 :wrench: update LICENSE copyright
  • f11f571 :wrench: include noxfile in sdist
  • 1ec7c06 :wrench: update changelog
  • 14b4649 :bug: output(...) replace declarative mark using non iana compliant encoding ...
  • 1b06bc0 Merge branch 'refresh-part-2' of github.com:jawah/charset_normalizer into ref...
  • Additional commits viewable in compare view

[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=charset-normalizer&package-manager=pip&previous-version=3.4.0&new-version=3.4.1)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) ---
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot show ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> --- tools/publish/requirements_darwin.txt | 199 +++++++++++------------ tools/publish/requirements_linux.txt | 199 +++++++++++------------ tools/publish/requirements_universal.txt | 199 +++++++++++------------ tools/publish/requirements_windows.txt | 199 +++++++++++------------ 4 files changed, 372 insertions(+), 424 deletions(-) diff --git a/tools/publish/requirements_darwin.txt b/tools/publish/requirements_darwin.txt index e8ee1e9b89..5f8a33c3f5 100644 --- a/tools/publish/requirements_darwin.txt +++ b/tools/publish/requirements_darwin.txt @@ -10,112 +10,99 @@ certifi==2025.1.31 \ --hash=sha256:3d5da6925056f6f18f119200434a4780a94263f10d1c21d032a6f6b2baa20651 \ --hash=sha256:ca78db4565a652026a4db2bcdf68f2fb589ea80d0be70e03929ed730746b84fe # via requests -charset-normalizer==3.4.0 \ - --hash=sha256:0099d79bdfcf5c1f0c2c72f91516702ebf8b0b8ddd8905f97a8aecf49712c621 \ - --hash=sha256:0713f3adb9d03d49d365b70b84775d0a0d18e4ab08d12bc46baa6132ba78aaf6 \ - --hash=sha256:07afec21bbbbf8a5cc3651aa96b980afe2526e7f048fdfb7f1014d84acc8b6d8 \ - --hash=sha256:0b309d1747110feb25d7ed6b01afdec269c647d382c857ef4663bbe6ad95a912 \ - --hash=sha256:0d99dd8ff461990f12d6e42c7347fd9ab2532fb70e9621ba520f9e8637161d7c \ - --hash=sha256:0de7b687289d3c1b3e8660d0741874abe7888100efe14bd0f9fd7141bcbda92b \ - --hash=sha256:1110e22af8ca26b90bd6364fe4c763329b0ebf1ee213ba32b68c73de5752323d \ - --hash=sha256:130272c698667a982a5d0e626851ceff662565379baf0ff2cc58067b81d4f11d \ - --hash=sha256:136815f06a3ae311fae551c3df1f998a1ebd01ddd424aa5603a4336997629e95 \ - --hash=sha256:14215b71a762336254351b00ec720a8e85cada43b987da5a042e4ce3e82bd68e \ - --hash=sha256:1db4e7fefefd0f548d73e2e2e041f9df5c59e178b4c72fbac4cc6f535cfb1565 \ - --hash=sha256:1ffd9493de4c922f2a38c2bf62b831dcec90ac673ed1ca182fe11b4d8e9f2a64 \ - --hash=sha256:2006769bd1640bdf4d5641c69a3d63b71b81445473cac5ded39740a226fa88ab \ - --hash=sha256:20587d20f557fe189b7947d8e7ec5afa110ccf72a3128d61a2a387c3313f46be \ - --hash=sha256:223217c3d4f82c3ac5e29032b3f1c2eb0fb591b72161f86d93f5719079dae93e \ - --hash=sha256:27623ba66c183eca01bf9ff833875b459cad267aeeb044477fedac35e19ba907 \ - --hash=sha256:285e96d9d53422efc0d7a17c60e59f37fbf3dfa942073f666db4ac71e8d726d0 \ - --hash=sha256:2de62e8801ddfff069cd5c504ce3bc9672b23266597d4e4f50eda28846c322f2 \ - --hash=sha256:2f6c34da58ea9c1a9515621f4d9ac379871a8f21168ba1b5e09d74250de5ad62 \ - --hash=sha256:309a7de0a0ff3040acaebb35ec45d18db4b28232f21998851cfa709eeff49d62 \ - --hash=sha256:35c404d74c2926d0287fbd63ed5d27eb911eb9e4a3bb2c6d294f3cfd4a9e0c23 \ - --hash=sha256:3710a9751938947e6327ea9f3ea6332a09bf0ba0c09cae9cb1f250bd1f1549bc \ - --hash=sha256:3d59d125ffbd6d552765510e3f31ed75ebac2c7470c7274195b9161a32350284 \ - --hash=sha256:40d3ff7fc90b98c637bda91c89d51264a3dcf210cade3a2c6f838c7268d7a4ca \ - --hash=sha256:425c5f215d0eecee9a56cdb703203dda90423247421bf0d67125add85d0c4455 \ - --hash=sha256:43193c5cda5d612f247172016c4bb71251c784d7a4d9314677186a838ad34858 \ - --hash=sha256:44aeb140295a2f0659e113b31cfe92c9061622cadbc9e2a2f7b8ef6b1e29ef4b \ - --hash=sha256:47334db71978b23ebcf3c0f9f5ee98b8d65992b65c9c4f2d34c2eaf5bcaf0594 \ - --hash=sha256:4796efc4faf6b53a18e3d46343535caed491776a22af773f366534056c4e1fbc \ - --hash=sha256:4a51b48f42d9358460b78725283f04bddaf44a9358197b889657deba38f329db \ - --hash=sha256:4b67fdab07fdd3c10bb21edab3cbfe8cf5696f453afce75d815d9d7223fbe88b \ - --hash=sha256:4ec9dd88a5b71abfc74e9df5ebe7921c35cbb3b641181a531ca65cdb5e8e4dea \ - --hash=sha256:4f9fc98dad6c2eaa32fc3af1417d95b5e3d08aff968df0cd320066def971f9a6 \ - --hash=sha256:54b6a92d009cbe2fb11054ba694bc9e284dad30a26757b1e372a1fdddaf21920 \ - --hash=sha256:55f56e2ebd4e3bc50442fbc0888c9d8c94e4e06a933804e2af3e89e2f9c1c749 \ - --hash=sha256:5726cf76c982532c1863fb64d8c6dd0e4c90b6ece9feb06c9f202417a31f7dd7 \ - --hash=sha256:5d447056e2ca60382d460a604b6302d8db69476fd2015c81e7c35417cfabe4cd \ - --hash=sha256:5ed2e36c3e9b4f21dd9422f6893dec0abf2cca553af509b10cd630f878d3eb99 \ - --hash=sha256:5ff2ed8194587faf56555927b3aa10e6fb69d931e33953943bc4f837dfee2242 \ - --hash=sha256:62f60aebecfc7f4b82e3f639a7d1433a20ec32824db2199a11ad4f5e146ef5ee \ - --hash=sha256:63bc5c4ae26e4bc6be6469943b8253c0fd4e4186c43ad46e713ea61a0ba49129 \ - --hash=sha256:6b40e8d38afe634559e398cc32b1472f376a4099c75fe6299ae607e404c033b2 \ - --hash=sha256:6b493a043635eb376e50eedf7818f2f322eabbaa974e948bd8bdd29eb7ef2a51 \ - --hash=sha256:6dba5d19c4dfab08e58d5b36304b3f92f3bd5d42c1a3fa37b5ba5cdf6dfcbcee \ - --hash=sha256:6fd30dc99682dc2c603c2b315bded2799019cea829f8bf57dc6b61efde6611c8 \ - --hash=sha256:707b82d19e65c9bd28b81dde95249b07bf9f5b90ebe1ef17d9b57473f8a64b7b \ - --hash=sha256:7706f5850360ac01d80c89bcef1640683cc12ed87f42579dab6c5d3ed6888613 \ - --hash=sha256:7782afc9b6b42200f7362858f9e73b1f8316afb276d316336c0ec3bd73312742 \ - --hash=sha256:79983512b108e4a164b9c8d34de3992f76d48cadc9554c9e60b43f308988aabe \ - --hash=sha256:7f683ddc7eedd742e2889d2bfb96d69573fde1d92fcb811979cdb7165bb9c7d3 \ - --hash=sha256:82357d85de703176b5587dbe6ade8ff67f9f69a41c0733cf2425378b49954de5 \ - --hash=sha256:84450ba661fb96e9fd67629b93d2941c871ca86fc38d835d19d4225ff946a631 \ - --hash=sha256:86f4e8cca779080f66ff4f191a685ced73d2f72d50216f7112185dc02b90b9b7 \ - --hash=sha256:8cda06946eac330cbe6598f77bb54e690b4ca93f593dee1568ad22b04f347c15 \ - --hash=sha256:8ce7fd6767a1cc5a92a639b391891bf1c268b03ec7e021c7d6d902285259685c \ - --hash=sha256:8ff4e7cdfdb1ab5698e675ca622e72d58a6fa2a8aa58195de0c0061288e6e3ea \ - --hash=sha256:9289fd5dddcf57bab41d044f1756550f9e7cf0c8e373b8cdf0ce8773dc4bd417 \ - --hash=sha256:92a7e36b000bf022ef3dbb9c46bfe2d52c047d5e3f3343f43204263c5addc250 \ - --hash=sha256:92db3c28b5b2a273346bebb24857fda45601aef6ae1c011c0a997106581e8a88 \ - --hash=sha256:95c3c157765b031331dd4db3c775e58deaee050a3042fcad72cbc4189d7c8dca \ - --hash=sha256:980b4f289d1d90ca5efcf07958d3eb38ed9c0b7676bf2831a54d4f66f9c27dfa \ - --hash=sha256:9ae4ef0b3f6b41bad6366fb0ea4fc1d7ed051528e113a60fa2a65a9abb5b1d99 \ - --hash=sha256:9c98230f5042f4945f957d006edccc2af1e03ed5e37ce7c373f00a5a4daa6149 \ - --hash=sha256:9fa2566ca27d67c86569e8c85297aaf413ffab85a8960500f12ea34ff98e4c41 \ - --hash=sha256:a14969b8691f7998e74663b77b4c36c0337cb1df552da83d5c9004a93afdb574 \ - --hash=sha256:a8aacce6e2e1edcb6ac625fb0f8c3a9570ccc7bfba1f63419b3769ccf6a00ed0 \ - --hash=sha256:a8e538f46104c815be19c975572d74afb53f29650ea2025bbfaef359d2de2f7f \ - --hash=sha256:aa41e526a5d4a9dfcfbab0716c7e8a1b215abd3f3df5a45cf18a12721d31cb5d \ - --hash=sha256:aa693779a8b50cd97570e5a0f343538a8dbd3e496fa5dcb87e29406ad0299654 \ - --hash=sha256:ab22fbd9765e6954bc0bcff24c25ff71dcbfdb185fcdaca49e81bac68fe724d3 \ - --hash=sha256:ab2e5bef076f5a235c3774b4f4028a680432cded7cad37bba0fd90d64b187d19 \ - --hash=sha256:ab973df98fc99ab39080bfb0eb3a925181454d7c3ac8a1e695fddfae696d9e90 \ - --hash=sha256:af73657b7a68211996527dbfeffbb0864e043d270580c5aef06dc4b659a4b578 \ - --hash=sha256:b197e7094f232959f8f20541ead1d9862ac5ebea1d58e9849c1bf979255dfac9 \ - --hash=sha256:b295729485b06c1a0683af02a9e42d2caa9db04a373dc38a6a58cdd1e8abddf1 \ - --hash=sha256:b8831399554b92b72af5932cdbbd4ddc55c55f631bb13ff8fe4e6536a06c5c51 \ - --hash=sha256:b8dcd239c743aa2f9c22ce674a145e0a25cb1566c495928440a181ca1ccf6719 \ - --hash=sha256:bcb4f8ea87d03bc51ad04add8ceaf9b0f085ac045ab4d74e73bbc2dc033f0236 \ - --hash=sha256:bd7af3717683bea4c87acd8c0d3d5b44d56120b26fd3f8a692bdd2d5260c620a \ - --hash=sha256:bf4475b82be41b07cc5e5ff94810e6a01f276e37c2d55571e3fe175e467a1a1c \ - --hash=sha256:c3e446d253bd88f6377260d07c895816ebf33ffffd56c1c792b13bff9c3e1ade \ - --hash=sha256:c57516e58fd17d03ebe67e181a4e4e2ccab1168f8c2976c6a334d4f819fe5944 \ - --hash=sha256:c94057af19bc953643a33581844649a7fdab902624d2eb739738a30e2b3e60fc \ - --hash=sha256:cab5d0b79d987c67f3b9e9c53f54a61360422a5a0bc075f43cab5621d530c3b6 \ - --hash=sha256:ce031db0408e487fd2775d745ce30a7cd2923667cf3b69d48d219f1d8f5ddeb6 \ - --hash=sha256:cee4373f4d3ad28f1ab6290684d8e2ebdb9e7a1b74fdc39e4c211995f77bec27 \ - --hash=sha256:d5b054862739d276e09928de37c79ddeec42a6e1bfc55863be96a36ba22926f6 \ - --hash=sha256:dbe03226baf438ac4fda9e2d0715022fd579cb641c4cf639fa40d53b2fe6f3e2 \ - --hash=sha256:dc15e99b2d8a656f8e666854404f1ba54765871104e50c8e9813af8a7db07f12 \ - --hash=sha256:dcaf7c1524c0542ee2fc82cc8ec337f7a9f7edee2532421ab200d2b920fc97cf \ - --hash=sha256:dd4eda173a9fcccb5f2e2bd2a9f423d180194b1bf17cf59e3269899235b2a114 \ - --hash=sha256:dd9a8bd8900e65504a305bf8ae6fa9fbc66de94178c420791d0293702fce2df7 \ - --hash=sha256:de7376c29d95d6719048c194a9cf1a1b0393fbe8488a22008610b0361d834ecf \ - --hash=sha256:e7fdd52961feb4c96507aa649550ec2a0d527c086d284749b2f582f2d40a2e0d \ - --hash=sha256:e91f541a85298cf35433bf66f3fab2a4a2cff05c127eeca4af174f6d497f0d4b \ - --hash=sha256:e9e3c4c9e1ed40ea53acf11e2a386383c3304212c965773704e4603d589343ed \ - --hash=sha256:ee803480535c44e7f5ad00788526da7d85525cfefaf8acf8ab9a310000be4b03 \ - --hash=sha256:f09cb5a7bbe1ecae6e87901a2eb23e0256bb524a79ccc53eb0b7629fbe7677c4 \ - --hash=sha256:f19c1585933c82098c2a520f8ec1227f20e339e33aca8fa6f956f6691b784e67 \ - --hash=sha256:f1a2f519ae173b5b6a2c9d5fa3116ce16e48b3462c8b96dfdded11055e3d6365 \ - --hash=sha256:f28f891ccd15c514a0981f3b9db9aa23d62fe1a99997512b0491d2ed323d229a \ - --hash=sha256:f3e73a4255342d4eb26ef6df01e3962e73aa29baa3124a8e824c5d3364a65748 \ - --hash=sha256:f606a1881d2663630ea5b8ce2efe2111740df4b687bd78b34a8131baa007f79b \ - --hash=sha256:fe9f97feb71aa9896b81973a7bbada8c49501dc73e58a10fcef6663af95e5079 \ - --hash=sha256:ffc519621dce0c767e96b9c53f09c5d215578e10b02c285809f76509a3931482 +charset-normalizer==3.4.1 \ + --hash=sha256:0167ddc8ab6508fe81860a57dd472b2ef4060e8d378f0cc555707126830f2537 \ + --hash=sha256:01732659ba9b5b873fc117534143e4feefecf3b2078b0a6a2e925271bb6f4cfa \ + --hash=sha256:01ad647cdd609225c5350561d084b42ddf732f4eeefe6e678765636791e78b9a \ + --hash=sha256:04432ad9479fa40ec0f387795ddad4437a2b50417c69fa275e212933519ff294 \ + --hash=sha256:0907f11d019260cdc3f94fbdb23ff9125f6b5d1039b76003b5b0ac9d6a6c9d5b \ + --hash=sha256:0924e81d3d5e70f8126529951dac65c1010cdf117bb75eb02dd12339b57749dd \ + --hash=sha256:09b26ae6b1abf0d27570633b2b078a2a20419c99d66fb2823173d73f188ce601 \ + --hash=sha256:09b5e6733cbd160dcc09589227187e242a30a49ca5cefa5a7edd3f9d19ed53fd \ + --hash=sha256:0af291f4fe114be0280cdd29d533696a77b5b49cfde5467176ecab32353395c4 \ + --hash=sha256:0f55e69f030f7163dffe9fd0752b32f070566451afe180f99dbeeb81f511ad8d \ + --hash=sha256:1a2bc9f351a75ef49d664206d51f8e5ede9da246602dc2d2726837620ea034b2 \ + --hash=sha256:22e14b5d70560b8dd51ec22863f370d1e595ac3d024cb8ad7d308b4cd95f8313 \ + --hash=sha256:234ac59ea147c59ee4da87a0c0f098e9c8d169f4dc2a159ef720f1a61bbe27cd \ + --hash=sha256:2369eea1ee4a7610a860d88f268eb39b95cb588acd7235e02fd5a5601773d4fa \ + --hash=sha256:237bdbe6159cff53b4f24f397d43c6336c6b0b42affbe857970cefbb620911c8 \ + --hash=sha256:28bf57629c75e810b6ae989f03c0828d64d6b26a5e205535585f96093e405ed1 \ + --hash=sha256:2967f74ad52c3b98de4c3b32e1a44e32975e008a9cd2a8cc8966d6a5218c5cb2 \ + --hash=sha256:2a75d49014d118e4198bcee5ee0a6f25856b29b12dbf7cd012791f8a6cc5c496 \ + --hash=sha256:2bdfe3ac2e1bbe5b59a1a63721eb3b95fc9b6817ae4a46debbb4e11f6232428d \ + --hash=sha256:2d074908e1aecee37a7635990b2c6d504cd4766c7bc9fc86d63f9c09af3fa11b \ + --hash=sha256:2fb9bd477fdea8684f78791a6de97a953c51831ee2981f8e4f583ff3b9d9687e \ + --hash=sha256:311f30128d7d333eebd7896965bfcfbd0065f1716ec92bd5638d7748eb6f936a \ + --hash=sha256:329ce159e82018d646c7ac45b01a430369d526569ec08516081727a20e9e4af4 \ + --hash=sha256:345b0426edd4e18138d6528aed636de7a9ed169b4aaf9d61a8c19e39d26838ca \ + --hash=sha256:363e2f92b0f0174b2f8238240a1a30142e3db7b957a5dd5689b0e75fb717cc78 \ + --hash=sha256:3a3bd0dcd373514dcec91c411ddb9632c0d7d92aed7093b8c3bbb6d69ca74408 \ + --hash=sha256:3bed14e9c89dcb10e8f3a29f9ccac4955aebe93c71ae803af79265c9ca5644c5 \ + --hash=sha256:44251f18cd68a75b56585dd00dae26183e102cd5e0f9f1466e6df5da2ed64ea3 \ + --hash=sha256:44ecbf16649486d4aebafeaa7ec4c9fed8b88101f4dd612dcaf65d5e815f837f \ + --hash=sha256:4532bff1b8421fd0a320463030c7520f56a79c9024a4e88f01c537316019005a \ + --hash=sha256:49402233c892a461407c512a19435d1ce275543138294f7ef013f0b63d5d3765 \ + --hash=sha256:4c0907b1928a36d5a998d72d64d8eaa7244989f7aaaf947500d3a800c83a3fd6 \ + --hash=sha256:4d86f7aff21ee58f26dcf5ae81a9addbd914115cdebcbb2217e4f0ed8982e146 \ + --hash=sha256:5777ee0881f9499ed0f71cc82cf873d9a0ca8af166dfa0af8ec4e675b7df48e6 \ + --hash=sha256:5df196eb874dae23dcfb968c83d4f8fdccb333330fe1fc278ac5ceeb101003a9 \ + --hash=sha256:619a609aa74ae43d90ed2e89bdd784765de0a25ca761b93e196d938b8fd1dbbd \ + --hash=sha256:6e27f48bcd0957c6d4cb9d6fa6b61d192d0b13d5ef563e5f2ae35feafc0d179c \ + --hash=sha256:6ff8a4a60c227ad87030d76e99cd1698345d4491638dfa6673027c48b3cd395f \ + --hash=sha256:73d94b58ec7fecbc7366247d3b0b10a21681004153238750bb67bd9012414545 \ + --hash=sha256:7461baadb4dc00fd9e0acbe254e3d7d2112e7f92ced2adc96e54ef6501c5f176 \ + --hash=sha256:75832c08354f595c760a804588b9357d34ec00ba1c940c15e31e96d902093770 \ + --hash=sha256:7709f51f5f7c853f0fb938bcd3bc59cdfdc5203635ffd18bf354f6967ea0f824 \ + --hash=sha256:78baa6d91634dfb69ec52a463534bc0df05dbd546209b79a3880a34487f4b84f \ + --hash=sha256:7974a0b5ecd505609e3b19742b60cee7aa2aa2fb3151bc917e6e2646d7667dcf \ + --hash=sha256:7a4f97a081603d2050bfaffdefa5b02a9ec823f8348a572e39032caa8404a487 \ + --hash=sha256:7b1bef6280950ee6c177b326508f86cad7ad4dff12454483b51d8b7d673a2c5d \ + --hash=sha256:7d053096f67cd1241601111b698f5cad775f97ab25d81567d3f59219b5f1adbd \ + --hash=sha256:804a4d582ba6e5b747c625bf1255e6b1507465494a40a2130978bda7b932c90b \ + --hash=sha256:807f52c1f798eef6cf26beb819eeb8819b1622ddfeef9d0977a8502d4db6d534 \ + --hash=sha256:80ed5e856eb7f30115aaf94e4a08114ccc8813e6ed1b5efa74f9f82e8509858f \ + --hash=sha256:8417cb1f36cc0bc7eaba8ccb0e04d55f0ee52df06df3ad55259b9a323555fc8b \ + --hash=sha256:8436c508b408b82d87dc5f62496973a1805cd46727c34440b0d29d8a2f50a6c9 \ + --hash=sha256:89149166622f4db9b4b6a449256291dc87a99ee53151c74cbd82a53c8c2f6ccd \ + --hash=sha256:8bfa33f4f2672964266e940dd22a195989ba31669bd84629f05fab3ef4e2d125 \ + --hash=sha256:8c60ca7339acd497a55b0ea5d506b2a2612afb2826560416f6894e8b5770d4a9 \ + --hash=sha256:91b36a978b5ae0ee86c394f5a54d6ef44db1de0815eb43de826d41d21e4af3de \ + --hash=sha256:955f8851919303c92343d2f66165294848d57e9bba6cf6e3625485a70a038d11 \ + --hash=sha256:97f68b8d6831127e4787ad15e6757232e14e12060bec17091b85eb1486b91d8d \ + --hash=sha256:9b23ca7ef998bc739bf6ffc077c2116917eabcc901f88da1b9856b210ef63f35 \ + --hash=sha256:9f0b8b1c6d84c8034a44893aba5e767bf9c7a211e313a9605d9c617d7083829f \ + --hash=sha256:aabfa34badd18f1da5ec1bc2715cadc8dca465868a4e73a0173466b688f29dda \ + --hash=sha256:ab36c8eb7e454e34e60eb55ca5d241a5d18b2c6244f6827a30e451c42410b5f7 \ + --hash=sha256:b010a7a4fd316c3c484d482922d13044979e78d1861f0e0650423144c616a46a \ + --hash=sha256:b1ac5992a838106edb89654e0aebfc24f5848ae2547d22c2c3f66454daa11971 \ + --hash=sha256:b7b2d86dd06bfc2ade3312a83a5c364c7ec2e3498f8734282c6c3d4b07b346b8 \ + --hash=sha256:b97e690a2118911e39b4042088092771b4ae3fc3aa86518f84b8cf6888dbdb41 \ + --hash=sha256:bc2722592d8998c870fa4e290c2eec2c1569b87fe58618e67d38b4665dfa680d \ + --hash=sha256:c0429126cf75e16c4f0ad00ee0eae4242dc652290f940152ca8c75c3a4b6ee8f \ + --hash=sha256:c30197aa96e8eed02200a83fba2657b4c3acd0f0aa4bdc9f6c1af8e8962e0757 \ + --hash=sha256:c4c3e6da02df6fa1410a7680bd3f63d4f710232d3139089536310d027950696a \ + --hash=sha256:c75cb2a3e389853835e84a2d8fb2b81a10645b503eca9bcb98df6b5a43eb8886 \ + --hash=sha256:c96836c97b1238e9c9e3fe90844c947d5afbf4f4c92762679acfe19927d81d77 \ + --hash=sha256:d7f50a1f8c450f3925cb367d011448c39239bb3eb4117c36a6d354794de4ce76 \ + --hash=sha256:d973f03c0cb71c5ed99037b870f2be986c3c05e63622c017ea9816881d2dd247 \ + --hash=sha256:d98b1668f06378c6dbefec3b92299716b931cd4e6061f3c875a71ced1780ab85 \ + --hash=sha256:d9c3cdf5390dcd29aa8056d13e8e99526cda0305acc038b96b30352aff5ff2bb \ + --hash=sha256:dad3e487649f498dd991eeb901125411559b22e8d7ab25d3aeb1af367df5efd7 \ + --hash=sha256:dccbe65bd2f7f7ec22c4ff99ed56faa1e9f785482b9bbd7c717e26fd723a1d1e \ + --hash=sha256:dd78cfcda14a1ef52584dbb008f7ac81c1328c0f58184bf9a84c49c605002da6 \ + --hash=sha256:e218488cd232553829be0664c2292d3af2eeeb94b32bea483cf79ac6a694e037 \ + --hash=sha256:e358e64305fe12299a08e08978f51fc21fac060dcfcddd95453eabe5b93ed0e1 \ + --hash=sha256:ea0d8d539afa5eb2728aa1932a988a9a7af94f18582ffae4bc10b3fbdad0626e \ + --hash=sha256:eab677309cdb30d047996b36d34caeda1dc91149e4fdca0b1a039b3f79d9a807 \ + --hash=sha256:eb8178fe3dba6450a3e024e95ac49ed3400e506fd4e9e5c32d30adda88cbd407 \ + --hash=sha256:ecddf25bee22fe4fe3737a399d0d177d72bc22be6913acfab364b40bce1ba83c \ + --hash=sha256:eea6ee1db730b3483adf394ea72f808b6e18cf3cb6454b4d86e04fa8c4327a12 \ + --hash=sha256:f08ff5e948271dc7e18a35641d2f11a4cd8dfd5634f55228b691e62b37125eb3 \ + --hash=sha256:f30bf9fd9be89ecb2360c7d94a711f00c09b976258846efe40db3d05828e8089 \ + --hash=sha256:fa88b843d6e211393a37219e6a1c1df99d35e8fd90446f1118f4216e307e48cd \ + --hash=sha256:fc54db6c8593ef7d4b2a331b58653356cf04f67c960f584edb7c3d8c97e8f39e \ + --hash=sha256:fd4ec41f914fa74ad1b8304bbc634b3de73d2a0889bd32076342a573e0779e00 \ + --hash=sha256:ffc9202a29ab3920fa812879e95a9e78b2465fd10be7fcbd042899695d75e616 # via requests docutils==0.21.2 \ --hash=sha256:3a6b18732edf182daa3cd12775bbb338cf5691468f91eeeb109deff6ebfa986f \ diff --git a/tools/publish/requirements_linux.txt b/tools/publish/requirements_linux.txt index 892b8b26b3..90b07d4c97 100644 --- a/tools/publish/requirements_linux.txt +++ b/tools/publish/requirements_linux.txt @@ -79,112 +79,99 @@ cffi==1.17.1 \ --hash=sha256:f7f5baafcc48261359e14bcd6d9bff6d4b28d9103847c9e136694cb0501aef87 \ --hash=sha256:fc48c783f9c87e60831201f2cce7f3b2e4846bf4d8728eabe54d60700b318a0b # via cryptography -charset-normalizer==3.4.0 \ - --hash=sha256:0099d79bdfcf5c1f0c2c72f91516702ebf8b0b8ddd8905f97a8aecf49712c621 \ - --hash=sha256:0713f3adb9d03d49d365b70b84775d0a0d18e4ab08d12bc46baa6132ba78aaf6 \ - --hash=sha256:07afec21bbbbf8a5cc3651aa96b980afe2526e7f048fdfb7f1014d84acc8b6d8 \ - --hash=sha256:0b309d1747110feb25d7ed6b01afdec269c647d382c857ef4663bbe6ad95a912 \ - --hash=sha256:0d99dd8ff461990f12d6e42c7347fd9ab2532fb70e9621ba520f9e8637161d7c \ - --hash=sha256:0de7b687289d3c1b3e8660d0741874abe7888100efe14bd0f9fd7141bcbda92b \ - --hash=sha256:1110e22af8ca26b90bd6364fe4c763329b0ebf1ee213ba32b68c73de5752323d \ - --hash=sha256:130272c698667a982a5d0e626851ceff662565379baf0ff2cc58067b81d4f11d \ - --hash=sha256:136815f06a3ae311fae551c3df1f998a1ebd01ddd424aa5603a4336997629e95 \ - --hash=sha256:14215b71a762336254351b00ec720a8e85cada43b987da5a042e4ce3e82bd68e \ - --hash=sha256:1db4e7fefefd0f548d73e2e2e041f9df5c59e178b4c72fbac4cc6f535cfb1565 \ - --hash=sha256:1ffd9493de4c922f2a38c2bf62b831dcec90ac673ed1ca182fe11b4d8e9f2a64 \ - --hash=sha256:2006769bd1640bdf4d5641c69a3d63b71b81445473cac5ded39740a226fa88ab \ - --hash=sha256:20587d20f557fe189b7947d8e7ec5afa110ccf72a3128d61a2a387c3313f46be \ - --hash=sha256:223217c3d4f82c3ac5e29032b3f1c2eb0fb591b72161f86d93f5719079dae93e \ - --hash=sha256:27623ba66c183eca01bf9ff833875b459cad267aeeb044477fedac35e19ba907 \ - --hash=sha256:285e96d9d53422efc0d7a17c60e59f37fbf3dfa942073f666db4ac71e8d726d0 \ - --hash=sha256:2de62e8801ddfff069cd5c504ce3bc9672b23266597d4e4f50eda28846c322f2 \ - --hash=sha256:2f6c34da58ea9c1a9515621f4d9ac379871a8f21168ba1b5e09d74250de5ad62 \ - --hash=sha256:309a7de0a0ff3040acaebb35ec45d18db4b28232f21998851cfa709eeff49d62 \ - --hash=sha256:35c404d74c2926d0287fbd63ed5d27eb911eb9e4a3bb2c6d294f3cfd4a9e0c23 \ - --hash=sha256:3710a9751938947e6327ea9f3ea6332a09bf0ba0c09cae9cb1f250bd1f1549bc \ - --hash=sha256:3d59d125ffbd6d552765510e3f31ed75ebac2c7470c7274195b9161a32350284 \ - --hash=sha256:40d3ff7fc90b98c637bda91c89d51264a3dcf210cade3a2c6f838c7268d7a4ca \ - --hash=sha256:425c5f215d0eecee9a56cdb703203dda90423247421bf0d67125add85d0c4455 \ - --hash=sha256:43193c5cda5d612f247172016c4bb71251c784d7a4d9314677186a838ad34858 \ - --hash=sha256:44aeb140295a2f0659e113b31cfe92c9061622cadbc9e2a2f7b8ef6b1e29ef4b \ - --hash=sha256:47334db71978b23ebcf3c0f9f5ee98b8d65992b65c9c4f2d34c2eaf5bcaf0594 \ - --hash=sha256:4796efc4faf6b53a18e3d46343535caed491776a22af773f366534056c4e1fbc \ - --hash=sha256:4a51b48f42d9358460b78725283f04bddaf44a9358197b889657deba38f329db \ - --hash=sha256:4b67fdab07fdd3c10bb21edab3cbfe8cf5696f453afce75d815d9d7223fbe88b \ - --hash=sha256:4ec9dd88a5b71abfc74e9df5ebe7921c35cbb3b641181a531ca65cdb5e8e4dea \ - --hash=sha256:4f9fc98dad6c2eaa32fc3af1417d95b5e3d08aff968df0cd320066def971f9a6 \ - --hash=sha256:54b6a92d009cbe2fb11054ba694bc9e284dad30a26757b1e372a1fdddaf21920 \ - --hash=sha256:55f56e2ebd4e3bc50442fbc0888c9d8c94e4e06a933804e2af3e89e2f9c1c749 \ - --hash=sha256:5726cf76c982532c1863fb64d8c6dd0e4c90b6ece9feb06c9f202417a31f7dd7 \ - --hash=sha256:5d447056e2ca60382d460a604b6302d8db69476fd2015c81e7c35417cfabe4cd \ - --hash=sha256:5ed2e36c3e9b4f21dd9422f6893dec0abf2cca553af509b10cd630f878d3eb99 \ - --hash=sha256:5ff2ed8194587faf56555927b3aa10e6fb69d931e33953943bc4f837dfee2242 \ - --hash=sha256:62f60aebecfc7f4b82e3f639a7d1433a20ec32824db2199a11ad4f5e146ef5ee \ - --hash=sha256:63bc5c4ae26e4bc6be6469943b8253c0fd4e4186c43ad46e713ea61a0ba49129 \ - --hash=sha256:6b40e8d38afe634559e398cc32b1472f376a4099c75fe6299ae607e404c033b2 \ - --hash=sha256:6b493a043635eb376e50eedf7818f2f322eabbaa974e948bd8bdd29eb7ef2a51 \ - --hash=sha256:6dba5d19c4dfab08e58d5b36304b3f92f3bd5d42c1a3fa37b5ba5cdf6dfcbcee \ - --hash=sha256:6fd30dc99682dc2c603c2b315bded2799019cea829f8bf57dc6b61efde6611c8 \ - --hash=sha256:707b82d19e65c9bd28b81dde95249b07bf9f5b90ebe1ef17d9b57473f8a64b7b \ - --hash=sha256:7706f5850360ac01d80c89bcef1640683cc12ed87f42579dab6c5d3ed6888613 \ - --hash=sha256:7782afc9b6b42200f7362858f9e73b1f8316afb276d316336c0ec3bd73312742 \ - --hash=sha256:79983512b108e4a164b9c8d34de3992f76d48cadc9554c9e60b43f308988aabe \ - --hash=sha256:7f683ddc7eedd742e2889d2bfb96d69573fde1d92fcb811979cdb7165bb9c7d3 \ - --hash=sha256:82357d85de703176b5587dbe6ade8ff67f9f69a41c0733cf2425378b49954de5 \ - --hash=sha256:84450ba661fb96e9fd67629b93d2941c871ca86fc38d835d19d4225ff946a631 \ - --hash=sha256:86f4e8cca779080f66ff4f191a685ced73d2f72d50216f7112185dc02b90b9b7 \ - --hash=sha256:8cda06946eac330cbe6598f77bb54e690b4ca93f593dee1568ad22b04f347c15 \ - --hash=sha256:8ce7fd6767a1cc5a92a639b391891bf1c268b03ec7e021c7d6d902285259685c \ - --hash=sha256:8ff4e7cdfdb1ab5698e675ca622e72d58a6fa2a8aa58195de0c0061288e6e3ea \ - --hash=sha256:9289fd5dddcf57bab41d044f1756550f9e7cf0c8e373b8cdf0ce8773dc4bd417 \ - --hash=sha256:92a7e36b000bf022ef3dbb9c46bfe2d52c047d5e3f3343f43204263c5addc250 \ - --hash=sha256:92db3c28b5b2a273346bebb24857fda45601aef6ae1c011c0a997106581e8a88 \ - --hash=sha256:95c3c157765b031331dd4db3c775e58deaee050a3042fcad72cbc4189d7c8dca \ - --hash=sha256:980b4f289d1d90ca5efcf07958d3eb38ed9c0b7676bf2831a54d4f66f9c27dfa \ - --hash=sha256:9ae4ef0b3f6b41bad6366fb0ea4fc1d7ed051528e113a60fa2a65a9abb5b1d99 \ - --hash=sha256:9c98230f5042f4945f957d006edccc2af1e03ed5e37ce7c373f00a5a4daa6149 \ - --hash=sha256:9fa2566ca27d67c86569e8c85297aaf413ffab85a8960500f12ea34ff98e4c41 \ - --hash=sha256:a14969b8691f7998e74663b77b4c36c0337cb1df552da83d5c9004a93afdb574 \ - --hash=sha256:a8aacce6e2e1edcb6ac625fb0f8c3a9570ccc7bfba1f63419b3769ccf6a00ed0 \ - --hash=sha256:a8e538f46104c815be19c975572d74afb53f29650ea2025bbfaef359d2de2f7f \ - --hash=sha256:aa41e526a5d4a9dfcfbab0716c7e8a1b215abd3f3df5a45cf18a12721d31cb5d \ - --hash=sha256:aa693779a8b50cd97570e5a0f343538a8dbd3e496fa5dcb87e29406ad0299654 \ - --hash=sha256:ab22fbd9765e6954bc0bcff24c25ff71dcbfdb185fcdaca49e81bac68fe724d3 \ - --hash=sha256:ab2e5bef076f5a235c3774b4f4028a680432cded7cad37bba0fd90d64b187d19 \ - --hash=sha256:ab973df98fc99ab39080bfb0eb3a925181454d7c3ac8a1e695fddfae696d9e90 \ - --hash=sha256:af73657b7a68211996527dbfeffbb0864e043d270580c5aef06dc4b659a4b578 \ - --hash=sha256:b197e7094f232959f8f20541ead1d9862ac5ebea1d58e9849c1bf979255dfac9 \ - --hash=sha256:b295729485b06c1a0683af02a9e42d2caa9db04a373dc38a6a58cdd1e8abddf1 \ - --hash=sha256:b8831399554b92b72af5932cdbbd4ddc55c55f631bb13ff8fe4e6536a06c5c51 \ - --hash=sha256:b8dcd239c743aa2f9c22ce674a145e0a25cb1566c495928440a181ca1ccf6719 \ - --hash=sha256:bcb4f8ea87d03bc51ad04add8ceaf9b0f085ac045ab4d74e73bbc2dc033f0236 \ - --hash=sha256:bd7af3717683bea4c87acd8c0d3d5b44d56120b26fd3f8a692bdd2d5260c620a \ - --hash=sha256:bf4475b82be41b07cc5e5ff94810e6a01f276e37c2d55571e3fe175e467a1a1c \ - --hash=sha256:c3e446d253bd88f6377260d07c895816ebf33ffffd56c1c792b13bff9c3e1ade \ - --hash=sha256:c57516e58fd17d03ebe67e181a4e4e2ccab1168f8c2976c6a334d4f819fe5944 \ - --hash=sha256:c94057af19bc953643a33581844649a7fdab902624d2eb739738a30e2b3e60fc \ - --hash=sha256:cab5d0b79d987c67f3b9e9c53f54a61360422a5a0bc075f43cab5621d530c3b6 \ - --hash=sha256:ce031db0408e487fd2775d745ce30a7cd2923667cf3b69d48d219f1d8f5ddeb6 \ - --hash=sha256:cee4373f4d3ad28f1ab6290684d8e2ebdb9e7a1b74fdc39e4c211995f77bec27 \ - --hash=sha256:d5b054862739d276e09928de37c79ddeec42a6e1bfc55863be96a36ba22926f6 \ - --hash=sha256:dbe03226baf438ac4fda9e2d0715022fd579cb641c4cf639fa40d53b2fe6f3e2 \ - --hash=sha256:dc15e99b2d8a656f8e666854404f1ba54765871104e50c8e9813af8a7db07f12 \ - --hash=sha256:dcaf7c1524c0542ee2fc82cc8ec337f7a9f7edee2532421ab200d2b920fc97cf \ - --hash=sha256:dd4eda173a9fcccb5f2e2bd2a9f423d180194b1bf17cf59e3269899235b2a114 \ - --hash=sha256:dd9a8bd8900e65504a305bf8ae6fa9fbc66de94178c420791d0293702fce2df7 \ - --hash=sha256:de7376c29d95d6719048c194a9cf1a1b0393fbe8488a22008610b0361d834ecf \ - --hash=sha256:e7fdd52961feb4c96507aa649550ec2a0d527c086d284749b2f582f2d40a2e0d \ - --hash=sha256:e91f541a85298cf35433bf66f3fab2a4a2cff05c127eeca4af174f6d497f0d4b \ - --hash=sha256:e9e3c4c9e1ed40ea53acf11e2a386383c3304212c965773704e4603d589343ed \ - --hash=sha256:ee803480535c44e7f5ad00788526da7d85525cfefaf8acf8ab9a310000be4b03 \ - --hash=sha256:f09cb5a7bbe1ecae6e87901a2eb23e0256bb524a79ccc53eb0b7629fbe7677c4 \ - --hash=sha256:f19c1585933c82098c2a520f8ec1227f20e339e33aca8fa6f956f6691b784e67 \ - --hash=sha256:f1a2f519ae173b5b6a2c9d5fa3116ce16e48b3462c8b96dfdded11055e3d6365 \ - --hash=sha256:f28f891ccd15c514a0981f3b9db9aa23d62fe1a99997512b0491d2ed323d229a \ - --hash=sha256:f3e73a4255342d4eb26ef6df01e3962e73aa29baa3124a8e824c5d3364a65748 \ - --hash=sha256:f606a1881d2663630ea5b8ce2efe2111740df4b687bd78b34a8131baa007f79b \ - --hash=sha256:fe9f97feb71aa9896b81973a7bbada8c49501dc73e58a10fcef6663af95e5079 \ - --hash=sha256:ffc519621dce0c767e96b9c53f09c5d215578e10b02c285809f76509a3931482 +charset-normalizer==3.4.1 \ + --hash=sha256:0167ddc8ab6508fe81860a57dd472b2ef4060e8d378f0cc555707126830f2537 \ + --hash=sha256:01732659ba9b5b873fc117534143e4feefecf3b2078b0a6a2e925271bb6f4cfa \ + --hash=sha256:01ad647cdd609225c5350561d084b42ddf732f4eeefe6e678765636791e78b9a \ + --hash=sha256:04432ad9479fa40ec0f387795ddad4437a2b50417c69fa275e212933519ff294 \ + --hash=sha256:0907f11d019260cdc3f94fbdb23ff9125f6b5d1039b76003b5b0ac9d6a6c9d5b \ + --hash=sha256:0924e81d3d5e70f8126529951dac65c1010cdf117bb75eb02dd12339b57749dd \ + --hash=sha256:09b26ae6b1abf0d27570633b2b078a2a20419c99d66fb2823173d73f188ce601 \ + --hash=sha256:09b5e6733cbd160dcc09589227187e242a30a49ca5cefa5a7edd3f9d19ed53fd \ + --hash=sha256:0af291f4fe114be0280cdd29d533696a77b5b49cfde5467176ecab32353395c4 \ + --hash=sha256:0f55e69f030f7163dffe9fd0752b32f070566451afe180f99dbeeb81f511ad8d \ + --hash=sha256:1a2bc9f351a75ef49d664206d51f8e5ede9da246602dc2d2726837620ea034b2 \ + --hash=sha256:22e14b5d70560b8dd51ec22863f370d1e595ac3d024cb8ad7d308b4cd95f8313 \ + --hash=sha256:234ac59ea147c59ee4da87a0c0f098e9c8d169f4dc2a159ef720f1a61bbe27cd \ + --hash=sha256:2369eea1ee4a7610a860d88f268eb39b95cb588acd7235e02fd5a5601773d4fa \ + --hash=sha256:237bdbe6159cff53b4f24f397d43c6336c6b0b42affbe857970cefbb620911c8 \ + --hash=sha256:28bf57629c75e810b6ae989f03c0828d64d6b26a5e205535585f96093e405ed1 \ + --hash=sha256:2967f74ad52c3b98de4c3b32e1a44e32975e008a9cd2a8cc8966d6a5218c5cb2 \ + --hash=sha256:2a75d49014d118e4198bcee5ee0a6f25856b29b12dbf7cd012791f8a6cc5c496 \ + --hash=sha256:2bdfe3ac2e1bbe5b59a1a63721eb3b95fc9b6817ae4a46debbb4e11f6232428d \ + --hash=sha256:2d074908e1aecee37a7635990b2c6d504cd4766c7bc9fc86d63f9c09af3fa11b \ + --hash=sha256:2fb9bd477fdea8684f78791a6de97a953c51831ee2981f8e4f583ff3b9d9687e \ + --hash=sha256:311f30128d7d333eebd7896965bfcfbd0065f1716ec92bd5638d7748eb6f936a \ + --hash=sha256:329ce159e82018d646c7ac45b01a430369d526569ec08516081727a20e9e4af4 \ + --hash=sha256:345b0426edd4e18138d6528aed636de7a9ed169b4aaf9d61a8c19e39d26838ca \ + --hash=sha256:363e2f92b0f0174b2f8238240a1a30142e3db7b957a5dd5689b0e75fb717cc78 \ + --hash=sha256:3a3bd0dcd373514dcec91c411ddb9632c0d7d92aed7093b8c3bbb6d69ca74408 \ + --hash=sha256:3bed14e9c89dcb10e8f3a29f9ccac4955aebe93c71ae803af79265c9ca5644c5 \ + --hash=sha256:44251f18cd68a75b56585dd00dae26183e102cd5e0f9f1466e6df5da2ed64ea3 \ + --hash=sha256:44ecbf16649486d4aebafeaa7ec4c9fed8b88101f4dd612dcaf65d5e815f837f \ + --hash=sha256:4532bff1b8421fd0a320463030c7520f56a79c9024a4e88f01c537316019005a \ + --hash=sha256:49402233c892a461407c512a19435d1ce275543138294f7ef013f0b63d5d3765 \ + --hash=sha256:4c0907b1928a36d5a998d72d64d8eaa7244989f7aaaf947500d3a800c83a3fd6 \ + --hash=sha256:4d86f7aff21ee58f26dcf5ae81a9addbd914115cdebcbb2217e4f0ed8982e146 \ + --hash=sha256:5777ee0881f9499ed0f71cc82cf873d9a0ca8af166dfa0af8ec4e675b7df48e6 \ + --hash=sha256:5df196eb874dae23dcfb968c83d4f8fdccb333330fe1fc278ac5ceeb101003a9 \ + --hash=sha256:619a609aa74ae43d90ed2e89bdd784765de0a25ca761b93e196d938b8fd1dbbd \ + --hash=sha256:6e27f48bcd0957c6d4cb9d6fa6b61d192d0b13d5ef563e5f2ae35feafc0d179c \ + --hash=sha256:6ff8a4a60c227ad87030d76e99cd1698345d4491638dfa6673027c48b3cd395f \ + --hash=sha256:73d94b58ec7fecbc7366247d3b0b10a21681004153238750bb67bd9012414545 \ + --hash=sha256:7461baadb4dc00fd9e0acbe254e3d7d2112e7f92ced2adc96e54ef6501c5f176 \ + --hash=sha256:75832c08354f595c760a804588b9357d34ec00ba1c940c15e31e96d902093770 \ + --hash=sha256:7709f51f5f7c853f0fb938bcd3bc59cdfdc5203635ffd18bf354f6967ea0f824 \ + --hash=sha256:78baa6d91634dfb69ec52a463534bc0df05dbd546209b79a3880a34487f4b84f \ + --hash=sha256:7974a0b5ecd505609e3b19742b60cee7aa2aa2fb3151bc917e6e2646d7667dcf \ + --hash=sha256:7a4f97a081603d2050bfaffdefa5b02a9ec823f8348a572e39032caa8404a487 \ + --hash=sha256:7b1bef6280950ee6c177b326508f86cad7ad4dff12454483b51d8b7d673a2c5d \ + --hash=sha256:7d053096f67cd1241601111b698f5cad775f97ab25d81567d3f59219b5f1adbd \ + --hash=sha256:804a4d582ba6e5b747c625bf1255e6b1507465494a40a2130978bda7b932c90b \ + --hash=sha256:807f52c1f798eef6cf26beb819eeb8819b1622ddfeef9d0977a8502d4db6d534 \ + --hash=sha256:80ed5e856eb7f30115aaf94e4a08114ccc8813e6ed1b5efa74f9f82e8509858f \ + --hash=sha256:8417cb1f36cc0bc7eaba8ccb0e04d55f0ee52df06df3ad55259b9a323555fc8b \ + --hash=sha256:8436c508b408b82d87dc5f62496973a1805cd46727c34440b0d29d8a2f50a6c9 \ + --hash=sha256:89149166622f4db9b4b6a449256291dc87a99ee53151c74cbd82a53c8c2f6ccd \ + --hash=sha256:8bfa33f4f2672964266e940dd22a195989ba31669bd84629f05fab3ef4e2d125 \ + --hash=sha256:8c60ca7339acd497a55b0ea5d506b2a2612afb2826560416f6894e8b5770d4a9 \ + --hash=sha256:91b36a978b5ae0ee86c394f5a54d6ef44db1de0815eb43de826d41d21e4af3de \ + --hash=sha256:955f8851919303c92343d2f66165294848d57e9bba6cf6e3625485a70a038d11 \ + --hash=sha256:97f68b8d6831127e4787ad15e6757232e14e12060bec17091b85eb1486b91d8d \ + --hash=sha256:9b23ca7ef998bc739bf6ffc077c2116917eabcc901f88da1b9856b210ef63f35 \ + --hash=sha256:9f0b8b1c6d84c8034a44893aba5e767bf9c7a211e313a9605d9c617d7083829f \ + --hash=sha256:aabfa34badd18f1da5ec1bc2715cadc8dca465868a4e73a0173466b688f29dda \ + --hash=sha256:ab36c8eb7e454e34e60eb55ca5d241a5d18b2c6244f6827a30e451c42410b5f7 \ + --hash=sha256:b010a7a4fd316c3c484d482922d13044979e78d1861f0e0650423144c616a46a \ + --hash=sha256:b1ac5992a838106edb89654e0aebfc24f5848ae2547d22c2c3f66454daa11971 \ + --hash=sha256:b7b2d86dd06bfc2ade3312a83a5c364c7ec2e3498f8734282c6c3d4b07b346b8 \ + --hash=sha256:b97e690a2118911e39b4042088092771b4ae3fc3aa86518f84b8cf6888dbdb41 \ + --hash=sha256:bc2722592d8998c870fa4e290c2eec2c1569b87fe58618e67d38b4665dfa680d \ + --hash=sha256:c0429126cf75e16c4f0ad00ee0eae4242dc652290f940152ca8c75c3a4b6ee8f \ + --hash=sha256:c30197aa96e8eed02200a83fba2657b4c3acd0f0aa4bdc9f6c1af8e8962e0757 \ + --hash=sha256:c4c3e6da02df6fa1410a7680bd3f63d4f710232d3139089536310d027950696a \ + --hash=sha256:c75cb2a3e389853835e84a2d8fb2b81a10645b503eca9bcb98df6b5a43eb8886 \ + --hash=sha256:c96836c97b1238e9c9e3fe90844c947d5afbf4f4c92762679acfe19927d81d77 \ + --hash=sha256:d7f50a1f8c450f3925cb367d011448c39239bb3eb4117c36a6d354794de4ce76 \ + --hash=sha256:d973f03c0cb71c5ed99037b870f2be986c3c05e63622c017ea9816881d2dd247 \ + --hash=sha256:d98b1668f06378c6dbefec3b92299716b931cd4e6061f3c875a71ced1780ab85 \ + --hash=sha256:d9c3cdf5390dcd29aa8056d13e8e99526cda0305acc038b96b30352aff5ff2bb \ + --hash=sha256:dad3e487649f498dd991eeb901125411559b22e8d7ab25d3aeb1af367df5efd7 \ + --hash=sha256:dccbe65bd2f7f7ec22c4ff99ed56faa1e9f785482b9bbd7c717e26fd723a1d1e \ + --hash=sha256:dd78cfcda14a1ef52584dbb008f7ac81c1328c0f58184bf9a84c49c605002da6 \ + --hash=sha256:e218488cd232553829be0664c2292d3af2eeeb94b32bea483cf79ac6a694e037 \ + --hash=sha256:e358e64305fe12299a08e08978f51fc21fac060dcfcddd95453eabe5b93ed0e1 \ + --hash=sha256:ea0d8d539afa5eb2728aa1932a988a9a7af94f18582ffae4bc10b3fbdad0626e \ + --hash=sha256:eab677309cdb30d047996b36d34caeda1dc91149e4fdca0b1a039b3f79d9a807 \ + --hash=sha256:eb8178fe3dba6450a3e024e95ac49ed3400e506fd4e9e5c32d30adda88cbd407 \ + --hash=sha256:ecddf25bee22fe4fe3737a399d0d177d72bc22be6913acfab364b40bce1ba83c \ + --hash=sha256:eea6ee1db730b3483adf394ea72f808b6e18cf3cb6454b4d86e04fa8c4327a12 \ + --hash=sha256:f08ff5e948271dc7e18a35641d2f11a4cd8dfd5634f55228b691e62b37125eb3 \ + --hash=sha256:f30bf9fd9be89ecb2360c7d94a711f00c09b976258846efe40db3d05828e8089 \ + --hash=sha256:fa88b843d6e211393a37219e6a1c1df99d35e8fd90446f1118f4216e307e48cd \ + --hash=sha256:fc54db6c8593ef7d4b2a331b58653356cf04f67c960f584edb7c3d8c97e8f39e \ + --hash=sha256:fd4ec41f914fa74ad1b8304bbc634b3de73d2a0889bd32076342a573e0779e00 \ + --hash=sha256:ffc9202a29ab3920fa812879e95a9e78b2465fd10be7fcbd042899695d75e616 # via requests cryptography==43.0.3 \ --hash=sha256:0c580952eef9bf68c4747774cde7ec1d85a6e61de97281f2dba83c7d2c806362 \ diff --git a/tools/publish/requirements_universal.txt b/tools/publish/requirements_universal.txt index 337073ac25..9b145fce49 100644 --- a/tools/publish/requirements_universal.txt +++ b/tools/publish/requirements_universal.txt @@ -79,112 +79,99 @@ cffi==1.17.1 ; platform_python_implementation != 'PyPy' and sys_platform == 'lin --hash=sha256:f7f5baafcc48261359e14bcd6d9bff6d4b28d9103847c9e136694cb0501aef87 \ --hash=sha256:fc48c783f9c87e60831201f2cce7f3b2e4846bf4d8728eabe54d60700b318a0b # via cryptography -charset-normalizer==3.4.0 \ - --hash=sha256:0099d79bdfcf5c1f0c2c72f91516702ebf8b0b8ddd8905f97a8aecf49712c621 \ - --hash=sha256:0713f3adb9d03d49d365b70b84775d0a0d18e4ab08d12bc46baa6132ba78aaf6 \ - --hash=sha256:07afec21bbbbf8a5cc3651aa96b980afe2526e7f048fdfb7f1014d84acc8b6d8 \ - --hash=sha256:0b309d1747110feb25d7ed6b01afdec269c647d382c857ef4663bbe6ad95a912 \ - --hash=sha256:0d99dd8ff461990f12d6e42c7347fd9ab2532fb70e9621ba520f9e8637161d7c \ - --hash=sha256:0de7b687289d3c1b3e8660d0741874abe7888100efe14bd0f9fd7141bcbda92b \ - --hash=sha256:1110e22af8ca26b90bd6364fe4c763329b0ebf1ee213ba32b68c73de5752323d \ - --hash=sha256:130272c698667a982a5d0e626851ceff662565379baf0ff2cc58067b81d4f11d \ - --hash=sha256:136815f06a3ae311fae551c3df1f998a1ebd01ddd424aa5603a4336997629e95 \ - --hash=sha256:14215b71a762336254351b00ec720a8e85cada43b987da5a042e4ce3e82bd68e \ - --hash=sha256:1db4e7fefefd0f548d73e2e2e041f9df5c59e178b4c72fbac4cc6f535cfb1565 \ - --hash=sha256:1ffd9493de4c922f2a38c2bf62b831dcec90ac673ed1ca182fe11b4d8e9f2a64 \ - --hash=sha256:2006769bd1640bdf4d5641c69a3d63b71b81445473cac5ded39740a226fa88ab \ - --hash=sha256:20587d20f557fe189b7947d8e7ec5afa110ccf72a3128d61a2a387c3313f46be \ - --hash=sha256:223217c3d4f82c3ac5e29032b3f1c2eb0fb591b72161f86d93f5719079dae93e \ - --hash=sha256:27623ba66c183eca01bf9ff833875b459cad267aeeb044477fedac35e19ba907 \ - --hash=sha256:285e96d9d53422efc0d7a17c60e59f37fbf3dfa942073f666db4ac71e8d726d0 \ - --hash=sha256:2de62e8801ddfff069cd5c504ce3bc9672b23266597d4e4f50eda28846c322f2 \ - --hash=sha256:2f6c34da58ea9c1a9515621f4d9ac379871a8f21168ba1b5e09d74250de5ad62 \ - --hash=sha256:309a7de0a0ff3040acaebb35ec45d18db4b28232f21998851cfa709eeff49d62 \ - --hash=sha256:35c404d74c2926d0287fbd63ed5d27eb911eb9e4a3bb2c6d294f3cfd4a9e0c23 \ - --hash=sha256:3710a9751938947e6327ea9f3ea6332a09bf0ba0c09cae9cb1f250bd1f1549bc \ - --hash=sha256:3d59d125ffbd6d552765510e3f31ed75ebac2c7470c7274195b9161a32350284 \ - --hash=sha256:40d3ff7fc90b98c637bda91c89d51264a3dcf210cade3a2c6f838c7268d7a4ca \ - --hash=sha256:425c5f215d0eecee9a56cdb703203dda90423247421bf0d67125add85d0c4455 \ - --hash=sha256:43193c5cda5d612f247172016c4bb71251c784d7a4d9314677186a838ad34858 \ - --hash=sha256:44aeb140295a2f0659e113b31cfe92c9061622cadbc9e2a2f7b8ef6b1e29ef4b \ - --hash=sha256:47334db71978b23ebcf3c0f9f5ee98b8d65992b65c9c4f2d34c2eaf5bcaf0594 \ - --hash=sha256:4796efc4faf6b53a18e3d46343535caed491776a22af773f366534056c4e1fbc \ - --hash=sha256:4a51b48f42d9358460b78725283f04bddaf44a9358197b889657deba38f329db \ - --hash=sha256:4b67fdab07fdd3c10bb21edab3cbfe8cf5696f453afce75d815d9d7223fbe88b \ - --hash=sha256:4ec9dd88a5b71abfc74e9df5ebe7921c35cbb3b641181a531ca65cdb5e8e4dea \ - --hash=sha256:4f9fc98dad6c2eaa32fc3af1417d95b5e3d08aff968df0cd320066def971f9a6 \ - --hash=sha256:54b6a92d009cbe2fb11054ba694bc9e284dad30a26757b1e372a1fdddaf21920 \ - --hash=sha256:55f56e2ebd4e3bc50442fbc0888c9d8c94e4e06a933804e2af3e89e2f9c1c749 \ - --hash=sha256:5726cf76c982532c1863fb64d8c6dd0e4c90b6ece9feb06c9f202417a31f7dd7 \ - --hash=sha256:5d447056e2ca60382d460a604b6302d8db69476fd2015c81e7c35417cfabe4cd \ - --hash=sha256:5ed2e36c3e9b4f21dd9422f6893dec0abf2cca553af509b10cd630f878d3eb99 \ - --hash=sha256:5ff2ed8194587faf56555927b3aa10e6fb69d931e33953943bc4f837dfee2242 \ - --hash=sha256:62f60aebecfc7f4b82e3f639a7d1433a20ec32824db2199a11ad4f5e146ef5ee \ - --hash=sha256:63bc5c4ae26e4bc6be6469943b8253c0fd4e4186c43ad46e713ea61a0ba49129 \ - --hash=sha256:6b40e8d38afe634559e398cc32b1472f376a4099c75fe6299ae607e404c033b2 \ - --hash=sha256:6b493a043635eb376e50eedf7818f2f322eabbaa974e948bd8bdd29eb7ef2a51 \ - --hash=sha256:6dba5d19c4dfab08e58d5b36304b3f92f3bd5d42c1a3fa37b5ba5cdf6dfcbcee \ - --hash=sha256:6fd30dc99682dc2c603c2b315bded2799019cea829f8bf57dc6b61efde6611c8 \ - --hash=sha256:707b82d19e65c9bd28b81dde95249b07bf9f5b90ebe1ef17d9b57473f8a64b7b \ - --hash=sha256:7706f5850360ac01d80c89bcef1640683cc12ed87f42579dab6c5d3ed6888613 \ - --hash=sha256:7782afc9b6b42200f7362858f9e73b1f8316afb276d316336c0ec3bd73312742 \ - --hash=sha256:79983512b108e4a164b9c8d34de3992f76d48cadc9554c9e60b43f308988aabe \ - --hash=sha256:7f683ddc7eedd742e2889d2bfb96d69573fde1d92fcb811979cdb7165bb9c7d3 \ - --hash=sha256:82357d85de703176b5587dbe6ade8ff67f9f69a41c0733cf2425378b49954de5 \ - --hash=sha256:84450ba661fb96e9fd67629b93d2941c871ca86fc38d835d19d4225ff946a631 \ - --hash=sha256:86f4e8cca779080f66ff4f191a685ced73d2f72d50216f7112185dc02b90b9b7 \ - --hash=sha256:8cda06946eac330cbe6598f77bb54e690b4ca93f593dee1568ad22b04f347c15 \ - --hash=sha256:8ce7fd6767a1cc5a92a639b391891bf1c268b03ec7e021c7d6d902285259685c \ - --hash=sha256:8ff4e7cdfdb1ab5698e675ca622e72d58a6fa2a8aa58195de0c0061288e6e3ea \ - --hash=sha256:9289fd5dddcf57bab41d044f1756550f9e7cf0c8e373b8cdf0ce8773dc4bd417 \ - --hash=sha256:92a7e36b000bf022ef3dbb9c46bfe2d52c047d5e3f3343f43204263c5addc250 \ - --hash=sha256:92db3c28b5b2a273346bebb24857fda45601aef6ae1c011c0a997106581e8a88 \ - --hash=sha256:95c3c157765b031331dd4db3c775e58deaee050a3042fcad72cbc4189d7c8dca \ - --hash=sha256:980b4f289d1d90ca5efcf07958d3eb38ed9c0b7676bf2831a54d4f66f9c27dfa \ - --hash=sha256:9ae4ef0b3f6b41bad6366fb0ea4fc1d7ed051528e113a60fa2a65a9abb5b1d99 \ - --hash=sha256:9c98230f5042f4945f957d006edccc2af1e03ed5e37ce7c373f00a5a4daa6149 \ - --hash=sha256:9fa2566ca27d67c86569e8c85297aaf413ffab85a8960500f12ea34ff98e4c41 \ - --hash=sha256:a14969b8691f7998e74663b77b4c36c0337cb1df552da83d5c9004a93afdb574 \ - --hash=sha256:a8aacce6e2e1edcb6ac625fb0f8c3a9570ccc7bfba1f63419b3769ccf6a00ed0 \ - --hash=sha256:a8e538f46104c815be19c975572d74afb53f29650ea2025bbfaef359d2de2f7f \ - --hash=sha256:aa41e526a5d4a9dfcfbab0716c7e8a1b215abd3f3df5a45cf18a12721d31cb5d \ - --hash=sha256:aa693779a8b50cd97570e5a0f343538a8dbd3e496fa5dcb87e29406ad0299654 \ - --hash=sha256:ab22fbd9765e6954bc0bcff24c25ff71dcbfdb185fcdaca49e81bac68fe724d3 \ - --hash=sha256:ab2e5bef076f5a235c3774b4f4028a680432cded7cad37bba0fd90d64b187d19 \ - --hash=sha256:ab973df98fc99ab39080bfb0eb3a925181454d7c3ac8a1e695fddfae696d9e90 \ - --hash=sha256:af73657b7a68211996527dbfeffbb0864e043d270580c5aef06dc4b659a4b578 \ - --hash=sha256:b197e7094f232959f8f20541ead1d9862ac5ebea1d58e9849c1bf979255dfac9 \ - --hash=sha256:b295729485b06c1a0683af02a9e42d2caa9db04a373dc38a6a58cdd1e8abddf1 \ - --hash=sha256:b8831399554b92b72af5932cdbbd4ddc55c55f631bb13ff8fe4e6536a06c5c51 \ - --hash=sha256:b8dcd239c743aa2f9c22ce674a145e0a25cb1566c495928440a181ca1ccf6719 \ - --hash=sha256:bcb4f8ea87d03bc51ad04add8ceaf9b0f085ac045ab4d74e73bbc2dc033f0236 \ - --hash=sha256:bd7af3717683bea4c87acd8c0d3d5b44d56120b26fd3f8a692bdd2d5260c620a \ - --hash=sha256:bf4475b82be41b07cc5e5ff94810e6a01f276e37c2d55571e3fe175e467a1a1c \ - --hash=sha256:c3e446d253bd88f6377260d07c895816ebf33ffffd56c1c792b13bff9c3e1ade \ - --hash=sha256:c57516e58fd17d03ebe67e181a4e4e2ccab1168f8c2976c6a334d4f819fe5944 \ - --hash=sha256:c94057af19bc953643a33581844649a7fdab902624d2eb739738a30e2b3e60fc \ - --hash=sha256:cab5d0b79d987c67f3b9e9c53f54a61360422a5a0bc075f43cab5621d530c3b6 \ - --hash=sha256:ce031db0408e487fd2775d745ce30a7cd2923667cf3b69d48d219f1d8f5ddeb6 \ - --hash=sha256:cee4373f4d3ad28f1ab6290684d8e2ebdb9e7a1b74fdc39e4c211995f77bec27 \ - --hash=sha256:d5b054862739d276e09928de37c79ddeec42a6e1bfc55863be96a36ba22926f6 \ - --hash=sha256:dbe03226baf438ac4fda9e2d0715022fd579cb641c4cf639fa40d53b2fe6f3e2 \ - --hash=sha256:dc15e99b2d8a656f8e666854404f1ba54765871104e50c8e9813af8a7db07f12 \ - --hash=sha256:dcaf7c1524c0542ee2fc82cc8ec337f7a9f7edee2532421ab200d2b920fc97cf \ - --hash=sha256:dd4eda173a9fcccb5f2e2bd2a9f423d180194b1bf17cf59e3269899235b2a114 \ - --hash=sha256:dd9a8bd8900e65504a305bf8ae6fa9fbc66de94178c420791d0293702fce2df7 \ - --hash=sha256:de7376c29d95d6719048c194a9cf1a1b0393fbe8488a22008610b0361d834ecf \ - --hash=sha256:e7fdd52961feb4c96507aa649550ec2a0d527c086d284749b2f582f2d40a2e0d \ - --hash=sha256:e91f541a85298cf35433bf66f3fab2a4a2cff05c127eeca4af174f6d497f0d4b \ - --hash=sha256:e9e3c4c9e1ed40ea53acf11e2a386383c3304212c965773704e4603d589343ed \ - --hash=sha256:ee803480535c44e7f5ad00788526da7d85525cfefaf8acf8ab9a310000be4b03 \ - --hash=sha256:f09cb5a7bbe1ecae6e87901a2eb23e0256bb524a79ccc53eb0b7629fbe7677c4 \ - --hash=sha256:f19c1585933c82098c2a520f8ec1227f20e339e33aca8fa6f956f6691b784e67 \ - --hash=sha256:f1a2f519ae173b5b6a2c9d5fa3116ce16e48b3462c8b96dfdded11055e3d6365 \ - --hash=sha256:f28f891ccd15c514a0981f3b9db9aa23d62fe1a99997512b0491d2ed323d229a \ - --hash=sha256:f3e73a4255342d4eb26ef6df01e3962e73aa29baa3124a8e824c5d3364a65748 \ - --hash=sha256:f606a1881d2663630ea5b8ce2efe2111740df4b687bd78b34a8131baa007f79b \ - --hash=sha256:fe9f97feb71aa9896b81973a7bbada8c49501dc73e58a10fcef6663af95e5079 \ - --hash=sha256:ffc519621dce0c767e96b9c53f09c5d215578e10b02c285809f76509a3931482 +charset-normalizer==3.4.1 \ + --hash=sha256:0167ddc8ab6508fe81860a57dd472b2ef4060e8d378f0cc555707126830f2537 \ + --hash=sha256:01732659ba9b5b873fc117534143e4feefecf3b2078b0a6a2e925271bb6f4cfa \ + --hash=sha256:01ad647cdd609225c5350561d084b42ddf732f4eeefe6e678765636791e78b9a \ + --hash=sha256:04432ad9479fa40ec0f387795ddad4437a2b50417c69fa275e212933519ff294 \ + --hash=sha256:0907f11d019260cdc3f94fbdb23ff9125f6b5d1039b76003b5b0ac9d6a6c9d5b \ + --hash=sha256:0924e81d3d5e70f8126529951dac65c1010cdf117bb75eb02dd12339b57749dd \ + --hash=sha256:09b26ae6b1abf0d27570633b2b078a2a20419c99d66fb2823173d73f188ce601 \ + --hash=sha256:09b5e6733cbd160dcc09589227187e242a30a49ca5cefa5a7edd3f9d19ed53fd \ + --hash=sha256:0af291f4fe114be0280cdd29d533696a77b5b49cfde5467176ecab32353395c4 \ + --hash=sha256:0f55e69f030f7163dffe9fd0752b32f070566451afe180f99dbeeb81f511ad8d \ + --hash=sha256:1a2bc9f351a75ef49d664206d51f8e5ede9da246602dc2d2726837620ea034b2 \ + --hash=sha256:22e14b5d70560b8dd51ec22863f370d1e595ac3d024cb8ad7d308b4cd95f8313 \ + --hash=sha256:234ac59ea147c59ee4da87a0c0f098e9c8d169f4dc2a159ef720f1a61bbe27cd \ + --hash=sha256:2369eea1ee4a7610a860d88f268eb39b95cb588acd7235e02fd5a5601773d4fa \ + --hash=sha256:237bdbe6159cff53b4f24f397d43c6336c6b0b42affbe857970cefbb620911c8 \ + --hash=sha256:28bf57629c75e810b6ae989f03c0828d64d6b26a5e205535585f96093e405ed1 \ + --hash=sha256:2967f74ad52c3b98de4c3b32e1a44e32975e008a9cd2a8cc8966d6a5218c5cb2 \ + --hash=sha256:2a75d49014d118e4198bcee5ee0a6f25856b29b12dbf7cd012791f8a6cc5c496 \ + --hash=sha256:2bdfe3ac2e1bbe5b59a1a63721eb3b95fc9b6817ae4a46debbb4e11f6232428d \ + --hash=sha256:2d074908e1aecee37a7635990b2c6d504cd4766c7bc9fc86d63f9c09af3fa11b \ + --hash=sha256:2fb9bd477fdea8684f78791a6de97a953c51831ee2981f8e4f583ff3b9d9687e \ + --hash=sha256:311f30128d7d333eebd7896965bfcfbd0065f1716ec92bd5638d7748eb6f936a \ + --hash=sha256:329ce159e82018d646c7ac45b01a430369d526569ec08516081727a20e9e4af4 \ + --hash=sha256:345b0426edd4e18138d6528aed636de7a9ed169b4aaf9d61a8c19e39d26838ca \ + --hash=sha256:363e2f92b0f0174b2f8238240a1a30142e3db7b957a5dd5689b0e75fb717cc78 \ + --hash=sha256:3a3bd0dcd373514dcec91c411ddb9632c0d7d92aed7093b8c3bbb6d69ca74408 \ + --hash=sha256:3bed14e9c89dcb10e8f3a29f9ccac4955aebe93c71ae803af79265c9ca5644c5 \ + --hash=sha256:44251f18cd68a75b56585dd00dae26183e102cd5e0f9f1466e6df5da2ed64ea3 \ + --hash=sha256:44ecbf16649486d4aebafeaa7ec4c9fed8b88101f4dd612dcaf65d5e815f837f \ + --hash=sha256:4532bff1b8421fd0a320463030c7520f56a79c9024a4e88f01c537316019005a \ + --hash=sha256:49402233c892a461407c512a19435d1ce275543138294f7ef013f0b63d5d3765 \ + --hash=sha256:4c0907b1928a36d5a998d72d64d8eaa7244989f7aaaf947500d3a800c83a3fd6 \ + --hash=sha256:4d86f7aff21ee58f26dcf5ae81a9addbd914115cdebcbb2217e4f0ed8982e146 \ + --hash=sha256:5777ee0881f9499ed0f71cc82cf873d9a0ca8af166dfa0af8ec4e675b7df48e6 \ + --hash=sha256:5df196eb874dae23dcfb968c83d4f8fdccb333330fe1fc278ac5ceeb101003a9 \ + --hash=sha256:619a609aa74ae43d90ed2e89bdd784765de0a25ca761b93e196d938b8fd1dbbd \ + --hash=sha256:6e27f48bcd0957c6d4cb9d6fa6b61d192d0b13d5ef563e5f2ae35feafc0d179c \ + --hash=sha256:6ff8a4a60c227ad87030d76e99cd1698345d4491638dfa6673027c48b3cd395f \ + --hash=sha256:73d94b58ec7fecbc7366247d3b0b10a21681004153238750bb67bd9012414545 \ + --hash=sha256:7461baadb4dc00fd9e0acbe254e3d7d2112e7f92ced2adc96e54ef6501c5f176 \ + --hash=sha256:75832c08354f595c760a804588b9357d34ec00ba1c940c15e31e96d902093770 \ + --hash=sha256:7709f51f5f7c853f0fb938bcd3bc59cdfdc5203635ffd18bf354f6967ea0f824 \ + --hash=sha256:78baa6d91634dfb69ec52a463534bc0df05dbd546209b79a3880a34487f4b84f \ + --hash=sha256:7974a0b5ecd505609e3b19742b60cee7aa2aa2fb3151bc917e6e2646d7667dcf \ + --hash=sha256:7a4f97a081603d2050bfaffdefa5b02a9ec823f8348a572e39032caa8404a487 \ + --hash=sha256:7b1bef6280950ee6c177b326508f86cad7ad4dff12454483b51d8b7d673a2c5d \ + --hash=sha256:7d053096f67cd1241601111b698f5cad775f97ab25d81567d3f59219b5f1adbd \ + --hash=sha256:804a4d582ba6e5b747c625bf1255e6b1507465494a40a2130978bda7b932c90b \ + --hash=sha256:807f52c1f798eef6cf26beb819eeb8819b1622ddfeef9d0977a8502d4db6d534 \ + --hash=sha256:80ed5e856eb7f30115aaf94e4a08114ccc8813e6ed1b5efa74f9f82e8509858f \ + --hash=sha256:8417cb1f36cc0bc7eaba8ccb0e04d55f0ee52df06df3ad55259b9a323555fc8b \ + --hash=sha256:8436c508b408b82d87dc5f62496973a1805cd46727c34440b0d29d8a2f50a6c9 \ + --hash=sha256:89149166622f4db9b4b6a449256291dc87a99ee53151c74cbd82a53c8c2f6ccd \ + --hash=sha256:8bfa33f4f2672964266e940dd22a195989ba31669bd84629f05fab3ef4e2d125 \ + --hash=sha256:8c60ca7339acd497a55b0ea5d506b2a2612afb2826560416f6894e8b5770d4a9 \ + --hash=sha256:91b36a978b5ae0ee86c394f5a54d6ef44db1de0815eb43de826d41d21e4af3de \ + --hash=sha256:955f8851919303c92343d2f66165294848d57e9bba6cf6e3625485a70a038d11 \ + --hash=sha256:97f68b8d6831127e4787ad15e6757232e14e12060bec17091b85eb1486b91d8d \ + --hash=sha256:9b23ca7ef998bc739bf6ffc077c2116917eabcc901f88da1b9856b210ef63f35 \ + --hash=sha256:9f0b8b1c6d84c8034a44893aba5e767bf9c7a211e313a9605d9c617d7083829f \ + --hash=sha256:aabfa34badd18f1da5ec1bc2715cadc8dca465868a4e73a0173466b688f29dda \ + --hash=sha256:ab36c8eb7e454e34e60eb55ca5d241a5d18b2c6244f6827a30e451c42410b5f7 \ + --hash=sha256:b010a7a4fd316c3c484d482922d13044979e78d1861f0e0650423144c616a46a \ + --hash=sha256:b1ac5992a838106edb89654e0aebfc24f5848ae2547d22c2c3f66454daa11971 \ + --hash=sha256:b7b2d86dd06bfc2ade3312a83a5c364c7ec2e3498f8734282c6c3d4b07b346b8 \ + --hash=sha256:b97e690a2118911e39b4042088092771b4ae3fc3aa86518f84b8cf6888dbdb41 \ + --hash=sha256:bc2722592d8998c870fa4e290c2eec2c1569b87fe58618e67d38b4665dfa680d \ + --hash=sha256:c0429126cf75e16c4f0ad00ee0eae4242dc652290f940152ca8c75c3a4b6ee8f \ + --hash=sha256:c30197aa96e8eed02200a83fba2657b4c3acd0f0aa4bdc9f6c1af8e8962e0757 \ + --hash=sha256:c4c3e6da02df6fa1410a7680bd3f63d4f710232d3139089536310d027950696a \ + --hash=sha256:c75cb2a3e389853835e84a2d8fb2b81a10645b503eca9bcb98df6b5a43eb8886 \ + --hash=sha256:c96836c97b1238e9c9e3fe90844c947d5afbf4f4c92762679acfe19927d81d77 \ + --hash=sha256:d7f50a1f8c450f3925cb367d011448c39239bb3eb4117c36a6d354794de4ce76 \ + --hash=sha256:d973f03c0cb71c5ed99037b870f2be986c3c05e63622c017ea9816881d2dd247 \ + --hash=sha256:d98b1668f06378c6dbefec3b92299716b931cd4e6061f3c875a71ced1780ab85 \ + --hash=sha256:d9c3cdf5390dcd29aa8056d13e8e99526cda0305acc038b96b30352aff5ff2bb \ + --hash=sha256:dad3e487649f498dd991eeb901125411559b22e8d7ab25d3aeb1af367df5efd7 \ + --hash=sha256:dccbe65bd2f7f7ec22c4ff99ed56faa1e9f785482b9bbd7c717e26fd723a1d1e \ + --hash=sha256:dd78cfcda14a1ef52584dbb008f7ac81c1328c0f58184bf9a84c49c605002da6 \ + --hash=sha256:e218488cd232553829be0664c2292d3af2eeeb94b32bea483cf79ac6a694e037 \ + --hash=sha256:e358e64305fe12299a08e08978f51fc21fac060dcfcddd95453eabe5b93ed0e1 \ + --hash=sha256:ea0d8d539afa5eb2728aa1932a988a9a7af94f18582ffae4bc10b3fbdad0626e \ + --hash=sha256:eab677309cdb30d047996b36d34caeda1dc91149e4fdca0b1a039b3f79d9a807 \ + --hash=sha256:eb8178fe3dba6450a3e024e95ac49ed3400e506fd4e9e5c32d30adda88cbd407 \ + --hash=sha256:ecddf25bee22fe4fe3737a399d0d177d72bc22be6913acfab364b40bce1ba83c \ + --hash=sha256:eea6ee1db730b3483adf394ea72f808b6e18cf3cb6454b4d86e04fa8c4327a12 \ + --hash=sha256:f08ff5e948271dc7e18a35641d2f11a4cd8dfd5634f55228b691e62b37125eb3 \ + --hash=sha256:f30bf9fd9be89ecb2360c7d94a711f00c09b976258846efe40db3d05828e8089 \ + --hash=sha256:fa88b843d6e211393a37219e6a1c1df99d35e8fd90446f1118f4216e307e48cd \ + --hash=sha256:fc54db6c8593ef7d4b2a331b58653356cf04f67c960f584edb7c3d8c97e8f39e \ + --hash=sha256:fd4ec41f914fa74ad1b8304bbc634b3de73d2a0889bd32076342a573e0779e00 \ + --hash=sha256:ffc9202a29ab3920fa812879e95a9e78b2465fd10be7fcbd042899695d75e616 # via requests cryptography==43.0.3 ; sys_platform == 'linux' \ --hash=sha256:0c580952eef9bf68c4747774cde7ec1d85a6e61de97281f2dba83c7d2c806362 \ diff --git a/tools/publish/requirements_windows.txt b/tools/publish/requirements_windows.txt index 1c6b9808fb..1980812d15 100644 --- a/tools/publish/requirements_windows.txt +++ b/tools/publish/requirements_windows.txt @@ -10,112 +10,99 @@ certifi==2025.1.31 \ --hash=sha256:3d5da6925056f6f18f119200434a4780a94263f10d1c21d032a6f6b2baa20651 \ --hash=sha256:ca78db4565a652026a4db2bcdf68f2fb589ea80d0be70e03929ed730746b84fe # via requests -charset-normalizer==3.4.0 \ - --hash=sha256:0099d79bdfcf5c1f0c2c72f91516702ebf8b0b8ddd8905f97a8aecf49712c621 \ - --hash=sha256:0713f3adb9d03d49d365b70b84775d0a0d18e4ab08d12bc46baa6132ba78aaf6 \ - --hash=sha256:07afec21bbbbf8a5cc3651aa96b980afe2526e7f048fdfb7f1014d84acc8b6d8 \ - --hash=sha256:0b309d1747110feb25d7ed6b01afdec269c647d382c857ef4663bbe6ad95a912 \ - --hash=sha256:0d99dd8ff461990f12d6e42c7347fd9ab2532fb70e9621ba520f9e8637161d7c \ - --hash=sha256:0de7b687289d3c1b3e8660d0741874abe7888100efe14bd0f9fd7141bcbda92b \ - --hash=sha256:1110e22af8ca26b90bd6364fe4c763329b0ebf1ee213ba32b68c73de5752323d \ - --hash=sha256:130272c698667a982a5d0e626851ceff662565379baf0ff2cc58067b81d4f11d \ - --hash=sha256:136815f06a3ae311fae551c3df1f998a1ebd01ddd424aa5603a4336997629e95 \ - --hash=sha256:14215b71a762336254351b00ec720a8e85cada43b987da5a042e4ce3e82bd68e \ - --hash=sha256:1db4e7fefefd0f548d73e2e2e041f9df5c59e178b4c72fbac4cc6f535cfb1565 \ - --hash=sha256:1ffd9493de4c922f2a38c2bf62b831dcec90ac673ed1ca182fe11b4d8e9f2a64 \ - --hash=sha256:2006769bd1640bdf4d5641c69a3d63b71b81445473cac5ded39740a226fa88ab \ - --hash=sha256:20587d20f557fe189b7947d8e7ec5afa110ccf72a3128d61a2a387c3313f46be \ - --hash=sha256:223217c3d4f82c3ac5e29032b3f1c2eb0fb591b72161f86d93f5719079dae93e \ - --hash=sha256:27623ba66c183eca01bf9ff833875b459cad267aeeb044477fedac35e19ba907 \ - --hash=sha256:285e96d9d53422efc0d7a17c60e59f37fbf3dfa942073f666db4ac71e8d726d0 \ - --hash=sha256:2de62e8801ddfff069cd5c504ce3bc9672b23266597d4e4f50eda28846c322f2 \ - --hash=sha256:2f6c34da58ea9c1a9515621f4d9ac379871a8f21168ba1b5e09d74250de5ad62 \ - --hash=sha256:309a7de0a0ff3040acaebb35ec45d18db4b28232f21998851cfa709eeff49d62 \ - --hash=sha256:35c404d74c2926d0287fbd63ed5d27eb911eb9e4a3bb2c6d294f3cfd4a9e0c23 \ - --hash=sha256:3710a9751938947e6327ea9f3ea6332a09bf0ba0c09cae9cb1f250bd1f1549bc \ - --hash=sha256:3d59d125ffbd6d552765510e3f31ed75ebac2c7470c7274195b9161a32350284 \ - --hash=sha256:40d3ff7fc90b98c637bda91c89d51264a3dcf210cade3a2c6f838c7268d7a4ca \ - --hash=sha256:425c5f215d0eecee9a56cdb703203dda90423247421bf0d67125add85d0c4455 \ - --hash=sha256:43193c5cda5d612f247172016c4bb71251c784d7a4d9314677186a838ad34858 \ - --hash=sha256:44aeb140295a2f0659e113b31cfe92c9061622cadbc9e2a2f7b8ef6b1e29ef4b \ - --hash=sha256:47334db71978b23ebcf3c0f9f5ee98b8d65992b65c9c4f2d34c2eaf5bcaf0594 \ - --hash=sha256:4796efc4faf6b53a18e3d46343535caed491776a22af773f366534056c4e1fbc \ - --hash=sha256:4a51b48f42d9358460b78725283f04bddaf44a9358197b889657deba38f329db \ - --hash=sha256:4b67fdab07fdd3c10bb21edab3cbfe8cf5696f453afce75d815d9d7223fbe88b \ - --hash=sha256:4ec9dd88a5b71abfc74e9df5ebe7921c35cbb3b641181a531ca65cdb5e8e4dea \ - --hash=sha256:4f9fc98dad6c2eaa32fc3af1417d95b5e3d08aff968df0cd320066def971f9a6 \ - --hash=sha256:54b6a92d009cbe2fb11054ba694bc9e284dad30a26757b1e372a1fdddaf21920 \ - --hash=sha256:55f56e2ebd4e3bc50442fbc0888c9d8c94e4e06a933804e2af3e89e2f9c1c749 \ - --hash=sha256:5726cf76c982532c1863fb64d8c6dd0e4c90b6ece9feb06c9f202417a31f7dd7 \ - --hash=sha256:5d447056e2ca60382d460a604b6302d8db69476fd2015c81e7c35417cfabe4cd \ - --hash=sha256:5ed2e36c3e9b4f21dd9422f6893dec0abf2cca553af509b10cd630f878d3eb99 \ - --hash=sha256:5ff2ed8194587faf56555927b3aa10e6fb69d931e33953943bc4f837dfee2242 \ - --hash=sha256:62f60aebecfc7f4b82e3f639a7d1433a20ec32824db2199a11ad4f5e146ef5ee \ - --hash=sha256:63bc5c4ae26e4bc6be6469943b8253c0fd4e4186c43ad46e713ea61a0ba49129 \ - --hash=sha256:6b40e8d38afe634559e398cc32b1472f376a4099c75fe6299ae607e404c033b2 \ - --hash=sha256:6b493a043635eb376e50eedf7818f2f322eabbaa974e948bd8bdd29eb7ef2a51 \ - --hash=sha256:6dba5d19c4dfab08e58d5b36304b3f92f3bd5d42c1a3fa37b5ba5cdf6dfcbcee \ - --hash=sha256:6fd30dc99682dc2c603c2b315bded2799019cea829f8bf57dc6b61efde6611c8 \ - --hash=sha256:707b82d19e65c9bd28b81dde95249b07bf9f5b90ebe1ef17d9b57473f8a64b7b \ - --hash=sha256:7706f5850360ac01d80c89bcef1640683cc12ed87f42579dab6c5d3ed6888613 \ - --hash=sha256:7782afc9b6b42200f7362858f9e73b1f8316afb276d316336c0ec3bd73312742 \ - --hash=sha256:79983512b108e4a164b9c8d34de3992f76d48cadc9554c9e60b43f308988aabe \ - --hash=sha256:7f683ddc7eedd742e2889d2bfb96d69573fde1d92fcb811979cdb7165bb9c7d3 \ - --hash=sha256:82357d85de703176b5587dbe6ade8ff67f9f69a41c0733cf2425378b49954de5 \ - --hash=sha256:84450ba661fb96e9fd67629b93d2941c871ca86fc38d835d19d4225ff946a631 \ - --hash=sha256:86f4e8cca779080f66ff4f191a685ced73d2f72d50216f7112185dc02b90b9b7 \ - --hash=sha256:8cda06946eac330cbe6598f77bb54e690b4ca93f593dee1568ad22b04f347c15 \ - --hash=sha256:8ce7fd6767a1cc5a92a639b391891bf1c268b03ec7e021c7d6d902285259685c \ - --hash=sha256:8ff4e7cdfdb1ab5698e675ca622e72d58a6fa2a8aa58195de0c0061288e6e3ea \ - --hash=sha256:9289fd5dddcf57bab41d044f1756550f9e7cf0c8e373b8cdf0ce8773dc4bd417 \ - --hash=sha256:92a7e36b000bf022ef3dbb9c46bfe2d52c047d5e3f3343f43204263c5addc250 \ - --hash=sha256:92db3c28b5b2a273346bebb24857fda45601aef6ae1c011c0a997106581e8a88 \ - --hash=sha256:95c3c157765b031331dd4db3c775e58deaee050a3042fcad72cbc4189d7c8dca \ - --hash=sha256:980b4f289d1d90ca5efcf07958d3eb38ed9c0b7676bf2831a54d4f66f9c27dfa \ - --hash=sha256:9ae4ef0b3f6b41bad6366fb0ea4fc1d7ed051528e113a60fa2a65a9abb5b1d99 \ - --hash=sha256:9c98230f5042f4945f957d006edccc2af1e03ed5e37ce7c373f00a5a4daa6149 \ - --hash=sha256:9fa2566ca27d67c86569e8c85297aaf413ffab85a8960500f12ea34ff98e4c41 \ - --hash=sha256:a14969b8691f7998e74663b77b4c36c0337cb1df552da83d5c9004a93afdb574 \ - --hash=sha256:a8aacce6e2e1edcb6ac625fb0f8c3a9570ccc7bfba1f63419b3769ccf6a00ed0 \ - --hash=sha256:a8e538f46104c815be19c975572d74afb53f29650ea2025bbfaef359d2de2f7f \ - --hash=sha256:aa41e526a5d4a9dfcfbab0716c7e8a1b215abd3f3df5a45cf18a12721d31cb5d \ - --hash=sha256:aa693779a8b50cd97570e5a0f343538a8dbd3e496fa5dcb87e29406ad0299654 \ - --hash=sha256:ab22fbd9765e6954bc0bcff24c25ff71dcbfdb185fcdaca49e81bac68fe724d3 \ - --hash=sha256:ab2e5bef076f5a235c3774b4f4028a680432cded7cad37bba0fd90d64b187d19 \ - --hash=sha256:ab973df98fc99ab39080bfb0eb3a925181454d7c3ac8a1e695fddfae696d9e90 \ - --hash=sha256:af73657b7a68211996527dbfeffbb0864e043d270580c5aef06dc4b659a4b578 \ - --hash=sha256:b197e7094f232959f8f20541ead1d9862ac5ebea1d58e9849c1bf979255dfac9 \ - --hash=sha256:b295729485b06c1a0683af02a9e42d2caa9db04a373dc38a6a58cdd1e8abddf1 \ - --hash=sha256:b8831399554b92b72af5932cdbbd4ddc55c55f631bb13ff8fe4e6536a06c5c51 \ - --hash=sha256:b8dcd239c743aa2f9c22ce674a145e0a25cb1566c495928440a181ca1ccf6719 \ - --hash=sha256:bcb4f8ea87d03bc51ad04add8ceaf9b0f085ac045ab4d74e73bbc2dc033f0236 \ - --hash=sha256:bd7af3717683bea4c87acd8c0d3d5b44d56120b26fd3f8a692bdd2d5260c620a \ - --hash=sha256:bf4475b82be41b07cc5e5ff94810e6a01f276e37c2d55571e3fe175e467a1a1c \ - --hash=sha256:c3e446d253bd88f6377260d07c895816ebf33ffffd56c1c792b13bff9c3e1ade \ - --hash=sha256:c57516e58fd17d03ebe67e181a4e4e2ccab1168f8c2976c6a334d4f819fe5944 \ - --hash=sha256:c94057af19bc953643a33581844649a7fdab902624d2eb739738a30e2b3e60fc \ - --hash=sha256:cab5d0b79d987c67f3b9e9c53f54a61360422a5a0bc075f43cab5621d530c3b6 \ - --hash=sha256:ce031db0408e487fd2775d745ce30a7cd2923667cf3b69d48d219f1d8f5ddeb6 \ - --hash=sha256:cee4373f4d3ad28f1ab6290684d8e2ebdb9e7a1b74fdc39e4c211995f77bec27 \ - --hash=sha256:d5b054862739d276e09928de37c79ddeec42a6e1bfc55863be96a36ba22926f6 \ - --hash=sha256:dbe03226baf438ac4fda9e2d0715022fd579cb641c4cf639fa40d53b2fe6f3e2 \ - --hash=sha256:dc15e99b2d8a656f8e666854404f1ba54765871104e50c8e9813af8a7db07f12 \ - --hash=sha256:dcaf7c1524c0542ee2fc82cc8ec337f7a9f7edee2532421ab200d2b920fc97cf \ - --hash=sha256:dd4eda173a9fcccb5f2e2bd2a9f423d180194b1bf17cf59e3269899235b2a114 \ - --hash=sha256:dd9a8bd8900e65504a305bf8ae6fa9fbc66de94178c420791d0293702fce2df7 \ - --hash=sha256:de7376c29d95d6719048c194a9cf1a1b0393fbe8488a22008610b0361d834ecf \ - --hash=sha256:e7fdd52961feb4c96507aa649550ec2a0d527c086d284749b2f582f2d40a2e0d \ - --hash=sha256:e91f541a85298cf35433bf66f3fab2a4a2cff05c127eeca4af174f6d497f0d4b \ - --hash=sha256:e9e3c4c9e1ed40ea53acf11e2a386383c3304212c965773704e4603d589343ed \ - --hash=sha256:ee803480535c44e7f5ad00788526da7d85525cfefaf8acf8ab9a310000be4b03 \ - --hash=sha256:f09cb5a7bbe1ecae6e87901a2eb23e0256bb524a79ccc53eb0b7629fbe7677c4 \ - --hash=sha256:f19c1585933c82098c2a520f8ec1227f20e339e33aca8fa6f956f6691b784e67 \ - --hash=sha256:f1a2f519ae173b5b6a2c9d5fa3116ce16e48b3462c8b96dfdded11055e3d6365 \ - --hash=sha256:f28f891ccd15c514a0981f3b9db9aa23d62fe1a99997512b0491d2ed323d229a \ - --hash=sha256:f3e73a4255342d4eb26ef6df01e3962e73aa29baa3124a8e824c5d3364a65748 \ - --hash=sha256:f606a1881d2663630ea5b8ce2efe2111740df4b687bd78b34a8131baa007f79b \ - --hash=sha256:fe9f97feb71aa9896b81973a7bbada8c49501dc73e58a10fcef6663af95e5079 \ - --hash=sha256:ffc519621dce0c767e96b9c53f09c5d215578e10b02c285809f76509a3931482 +charset-normalizer==3.4.1 \ + --hash=sha256:0167ddc8ab6508fe81860a57dd472b2ef4060e8d378f0cc555707126830f2537 \ + --hash=sha256:01732659ba9b5b873fc117534143e4feefecf3b2078b0a6a2e925271bb6f4cfa \ + --hash=sha256:01ad647cdd609225c5350561d084b42ddf732f4eeefe6e678765636791e78b9a \ + --hash=sha256:04432ad9479fa40ec0f387795ddad4437a2b50417c69fa275e212933519ff294 \ + --hash=sha256:0907f11d019260cdc3f94fbdb23ff9125f6b5d1039b76003b5b0ac9d6a6c9d5b \ + --hash=sha256:0924e81d3d5e70f8126529951dac65c1010cdf117bb75eb02dd12339b57749dd \ + --hash=sha256:09b26ae6b1abf0d27570633b2b078a2a20419c99d66fb2823173d73f188ce601 \ + --hash=sha256:09b5e6733cbd160dcc09589227187e242a30a49ca5cefa5a7edd3f9d19ed53fd \ + --hash=sha256:0af291f4fe114be0280cdd29d533696a77b5b49cfde5467176ecab32353395c4 \ + --hash=sha256:0f55e69f030f7163dffe9fd0752b32f070566451afe180f99dbeeb81f511ad8d \ + --hash=sha256:1a2bc9f351a75ef49d664206d51f8e5ede9da246602dc2d2726837620ea034b2 \ + --hash=sha256:22e14b5d70560b8dd51ec22863f370d1e595ac3d024cb8ad7d308b4cd95f8313 \ + --hash=sha256:234ac59ea147c59ee4da87a0c0f098e9c8d169f4dc2a159ef720f1a61bbe27cd \ + --hash=sha256:2369eea1ee4a7610a860d88f268eb39b95cb588acd7235e02fd5a5601773d4fa \ + --hash=sha256:237bdbe6159cff53b4f24f397d43c6336c6b0b42affbe857970cefbb620911c8 \ + --hash=sha256:28bf57629c75e810b6ae989f03c0828d64d6b26a5e205535585f96093e405ed1 \ + --hash=sha256:2967f74ad52c3b98de4c3b32e1a44e32975e008a9cd2a8cc8966d6a5218c5cb2 \ + --hash=sha256:2a75d49014d118e4198bcee5ee0a6f25856b29b12dbf7cd012791f8a6cc5c496 \ + --hash=sha256:2bdfe3ac2e1bbe5b59a1a63721eb3b95fc9b6817ae4a46debbb4e11f6232428d \ + --hash=sha256:2d074908e1aecee37a7635990b2c6d504cd4766c7bc9fc86d63f9c09af3fa11b \ + --hash=sha256:2fb9bd477fdea8684f78791a6de97a953c51831ee2981f8e4f583ff3b9d9687e \ + --hash=sha256:311f30128d7d333eebd7896965bfcfbd0065f1716ec92bd5638d7748eb6f936a \ + --hash=sha256:329ce159e82018d646c7ac45b01a430369d526569ec08516081727a20e9e4af4 \ + --hash=sha256:345b0426edd4e18138d6528aed636de7a9ed169b4aaf9d61a8c19e39d26838ca \ + --hash=sha256:363e2f92b0f0174b2f8238240a1a30142e3db7b957a5dd5689b0e75fb717cc78 \ + --hash=sha256:3a3bd0dcd373514dcec91c411ddb9632c0d7d92aed7093b8c3bbb6d69ca74408 \ + --hash=sha256:3bed14e9c89dcb10e8f3a29f9ccac4955aebe93c71ae803af79265c9ca5644c5 \ + --hash=sha256:44251f18cd68a75b56585dd00dae26183e102cd5e0f9f1466e6df5da2ed64ea3 \ + --hash=sha256:44ecbf16649486d4aebafeaa7ec4c9fed8b88101f4dd612dcaf65d5e815f837f \ + --hash=sha256:4532bff1b8421fd0a320463030c7520f56a79c9024a4e88f01c537316019005a \ + --hash=sha256:49402233c892a461407c512a19435d1ce275543138294f7ef013f0b63d5d3765 \ + --hash=sha256:4c0907b1928a36d5a998d72d64d8eaa7244989f7aaaf947500d3a800c83a3fd6 \ + --hash=sha256:4d86f7aff21ee58f26dcf5ae81a9addbd914115cdebcbb2217e4f0ed8982e146 \ + --hash=sha256:5777ee0881f9499ed0f71cc82cf873d9a0ca8af166dfa0af8ec4e675b7df48e6 \ + --hash=sha256:5df196eb874dae23dcfb968c83d4f8fdccb333330fe1fc278ac5ceeb101003a9 \ + --hash=sha256:619a609aa74ae43d90ed2e89bdd784765de0a25ca761b93e196d938b8fd1dbbd \ + --hash=sha256:6e27f48bcd0957c6d4cb9d6fa6b61d192d0b13d5ef563e5f2ae35feafc0d179c \ + --hash=sha256:6ff8a4a60c227ad87030d76e99cd1698345d4491638dfa6673027c48b3cd395f \ + --hash=sha256:73d94b58ec7fecbc7366247d3b0b10a21681004153238750bb67bd9012414545 \ + --hash=sha256:7461baadb4dc00fd9e0acbe254e3d7d2112e7f92ced2adc96e54ef6501c5f176 \ + --hash=sha256:75832c08354f595c760a804588b9357d34ec00ba1c940c15e31e96d902093770 \ + --hash=sha256:7709f51f5f7c853f0fb938bcd3bc59cdfdc5203635ffd18bf354f6967ea0f824 \ + --hash=sha256:78baa6d91634dfb69ec52a463534bc0df05dbd546209b79a3880a34487f4b84f \ + --hash=sha256:7974a0b5ecd505609e3b19742b60cee7aa2aa2fb3151bc917e6e2646d7667dcf \ + --hash=sha256:7a4f97a081603d2050bfaffdefa5b02a9ec823f8348a572e39032caa8404a487 \ + --hash=sha256:7b1bef6280950ee6c177b326508f86cad7ad4dff12454483b51d8b7d673a2c5d \ + --hash=sha256:7d053096f67cd1241601111b698f5cad775f97ab25d81567d3f59219b5f1adbd \ + --hash=sha256:804a4d582ba6e5b747c625bf1255e6b1507465494a40a2130978bda7b932c90b \ + --hash=sha256:807f52c1f798eef6cf26beb819eeb8819b1622ddfeef9d0977a8502d4db6d534 \ + --hash=sha256:80ed5e856eb7f30115aaf94e4a08114ccc8813e6ed1b5efa74f9f82e8509858f \ + --hash=sha256:8417cb1f36cc0bc7eaba8ccb0e04d55f0ee52df06df3ad55259b9a323555fc8b \ + --hash=sha256:8436c508b408b82d87dc5f62496973a1805cd46727c34440b0d29d8a2f50a6c9 \ + --hash=sha256:89149166622f4db9b4b6a449256291dc87a99ee53151c74cbd82a53c8c2f6ccd \ + --hash=sha256:8bfa33f4f2672964266e940dd22a195989ba31669bd84629f05fab3ef4e2d125 \ + --hash=sha256:8c60ca7339acd497a55b0ea5d506b2a2612afb2826560416f6894e8b5770d4a9 \ + --hash=sha256:91b36a978b5ae0ee86c394f5a54d6ef44db1de0815eb43de826d41d21e4af3de \ + --hash=sha256:955f8851919303c92343d2f66165294848d57e9bba6cf6e3625485a70a038d11 \ + --hash=sha256:97f68b8d6831127e4787ad15e6757232e14e12060bec17091b85eb1486b91d8d \ + --hash=sha256:9b23ca7ef998bc739bf6ffc077c2116917eabcc901f88da1b9856b210ef63f35 \ + --hash=sha256:9f0b8b1c6d84c8034a44893aba5e767bf9c7a211e313a9605d9c617d7083829f \ + --hash=sha256:aabfa34badd18f1da5ec1bc2715cadc8dca465868a4e73a0173466b688f29dda \ + --hash=sha256:ab36c8eb7e454e34e60eb55ca5d241a5d18b2c6244f6827a30e451c42410b5f7 \ + --hash=sha256:b010a7a4fd316c3c484d482922d13044979e78d1861f0e0650423144c616a46a \ + --hash=sha256:b1ac5992a838106edb89654e0aebfc24f5848ae2547d22c2c3f66454daa11971 \ + --hash=sha256:b7b2d86dd06bfc2ade3312a83a5c364c7ec2e3498f8734282c6c3d4b07b346b8 \ + --hash=sha256:b97e690a2118911e39b4042088092771b4ae3fc3aa86518f84b8cf6888dbdb41 \ + --hash=sha256:bc2722592d8998c870fa4e290c2eec2c1569b87fe58618e67d38b4665dfa680d \ + --hash=sha256:c0429126cf75e16c4f0ad00ee0eae4242dc652290f940152ca8c75c3a4b6ee8f \ + --hash=sha256:c30197aa96e8eed02200a83fba2657b4c3acd0f0aa4bdc9f6c1af8e8962e0757 \ + --hash=sha256:c4c3e6da02df6fa1410a7680bd3f63d4f710232d3139089536310d027950696a \ + --hash=sha256:c75cb2a3e389853835e84a2d8fb2b81a10645b503eca9bcb98df6b5a43eb8886 \ + --hash=sha256:c96836c97b1238e9c9e3fe90844c947d5afbf4f4c92762679acfe19927d81d77 \ + --hash=sha256:d7f50a1f8c450f3925cb367d011448c39239bb3eb4117c36a6d354794de4ce76 \ + --hash=sha256:d973f03c0cb71c5ed99037b870f2be986c3c05e63622c017ea9816881d2dd247 \ + --hash=sha256:d98b1668f06378c6dbefec3b92299716b931cd4e6061f3c875a71ced1780ab85 \ + --hash=sha256:d9c3cdf5390dcd29aa8056d13e8e99526cda0305acc038b96b30352aff5ff2bb \ + --hash=sha256:dad3e487649f498dd991eeb901125411559b22e8d7ab25d3aeb1af367df5efd7 \ + --hash=sha256:dccbe65bd2f7f7ec22c4ff99ed56faa1e9f785482b9bbd7c717e26fd723a1d1e \ + --hash=sha256:dd78cfcda14a1ef52584dbb008f7ac81c1328c0f58184bf9a84c49c605002da6 \ + --hash=sha256:e218488cd232553829be0664c2292d3af2eeeb94b32bea483cf79ac6a694e037 \ + --hash=sha256:e358e64305fe12299a08e08978f51fc21fac060dcfcddd95453eabe5b93ed0e1 \ + --hash=sha256:ea0d8d539afa5eb2728aa1932a988a9a7af94f18582ffae4bc10b3fbdad0626e \ + --hash=sha256:eab677309cdb30d047996b36d34caeda1dc91149e4fdca0b1a039b3f79d9a807 \ + --hash=sha256:eb8178fe3dba6450a3e024e95ac49ed3400e506fd4e9e5c32d30adda88cbd407 \ + --hash=sha256:ecddf25bee22fe4fe3737a399d0d177d72bc22be6913acfab364b40bce1ba83c \ + --hash=sha256:eea6ee1db730b3483adf394ea72f808b6e18cf3cb6454b4d86e04fa8c4327a12 \ + --hash=sha256:f08ff5e948271dc7e18a35641d2f11a4cd8dfd5634f55228b691e62b37125eb3 \ + --hash=sha256:f30bf9fd9be89ecb2360c7d94a711f00c09b976258846efe40db3d05828e8089 \ + --hash=sha256:fa88b843d6e211393a37219e6a1c1df99d35e8fd90446f1118f4216e307e48cd \ + --hash=sha256:fc54db6c8593ef7d4b2a331b58653356cf04f67c960f584edb7c3d8c97e8f39e \ + --hash=sha256:fd4ec41f914fa74ad1b8304bbc634b3de73d2a0889bd32076342a573e0779e00 \ + --hash=sha256:ffc9202a29ab3920fa812879e95a9e78b2465fd10be7fcbd042899695d75e616 # via requests docutils==0.21.2 \ --hash=sha256:3a6b18732edf182daa3cd12775bbb338cf5691468f91eeeb109deff6ebfa986f \ From 97637d2451647561205b10494b410f3b6edc3f83 Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Tue, 8 Apr 2025 10:46:55 +0900 Subject: [PATCH 154/922] build(deps): bump charset-normalizer from 3.4.0 to 3.4.1 in /docs (#2752) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Bumps [charset-normalizer](https://github.com/jawah/charset_normalizer) from 3.4.0 to 3.4.1.
Release notes

Sourced from charset-normalizer's releases.

Version 3.4.1

🚀 We're still raising awareness around HTTP/2, and HTTP/3!

Did you know that Internet Explorer 11 shipped with an optional HTTP/2 support back in 2013? also libcurl did ship it in 2014[...] Using Requests today is the rough equivalent of using EOL Windows 8! We promptly invite Python developers to look at the first drop-in replacement for Requests, namely Niquests. Ship with native WebSocket, SSE, Happy Eyeballs, DNS over HTTPS, and so on[...] All of this while remaining compatible with all Requests prior plug-ins / add-ons.

It leverages charset-normalizer in a better way! Check it out, you will gain up to being 3X faster and get a real/respectable support with it.

3.4.1 (2024-12-24)

Changed

  • Project metadata are now stored using pyproject.toml instead of setup.cfg using setuptools as the build backend.
  • Enforce annotation delayed loading for a simpler and consistent types in the project.
  • Optional mypyc compilation upgraded to version 1.14 for Python >= 3.8

Added

  • pre-commit configuration.
  • noxfile.

Removed

  • build-requirements.txt as per using pyproject.toml native build configuration.
  • bin/integration.py and bin/serve.py in favor of downstream integration test (see noxfile).
  • setup.cfg in favor of pyproject.toml metadata configuration.
  • Unused utils.range_scan function.

Fixed

  • Converting content to Unicode bytes may insert utf_8 instead of preferred utf-8. (#572)
  • Deprecation warning "'count' is passed as positional argument" when converting to Unicode bytes on Python 3.13+
Changelog

Sourced from charset-normalizer's changelog.

3.4.1 (2024-12-24)

Changed

  • Project metadata are now stored using pyproject.toml instead of setup.cfg using setuptools as the build backend.
  • Enforce annotation delayed loading for a simpler and consistent types in the project.
  • Optional mypyc compilation upgraded to version 1.14 for Python >= 3.8

Added

  • pre-commit configuration.
  • noxfile.

Removed

  • build-requirements.txt as per using pyproject.toml native build configuration.
  • bin/integration.py and bin/serve.py in favor of downstream integration test (see noxfile).
  • setup.cfg in favor of pyproject.toml metadata configuration.
  • Unused utils.range_scan function.

Fixed

  • Converting content to Unicode bytes may insert utf_8 instead of preferred utf-8. (#572)
  • Deprecation warning "'count' is passed as positional argument" when converting to Unicode bytes on Python 3.13+
Commits
  • ffdf7f5 :wrench: fix long description content-type inferred as rst instead of md
  • c7197b7 :pencil: fix changelog entries (#582)
  • c390e1f Merge pull request #581 from jawah/refresh-part-2
  • f9d6b8c :lock: add CODEOWNERS
  • 7ce1ef1 :wrench: use ubuntu-22.04 for cibuildwheel in continuous deployment workflow
  • deed205 :wrench: update LICENSE copyright
  • f11f571 :wrench: include noxfile in sdist
  • 1ec7c06 :wrench: update changelog
  • 14b4649 :bug: output(...) replace declarative mark using non iana compliant encoding ...
  • 1b06bc0 Merge branch 'refresh-part-2' of github.com:jawah/charset_normalizer into ref...
  • Additional commits viewable in compare view

[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=charset-normalizer&package-manager=pip&previous-version=3.4.0&new-version=3.4.1)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) ---
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot show ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> --- docs/requirements.txt | 199 ++++++++++++++++++++---------------------- 1 file changed, 93 insertions(+), 106 deletions(-) diff --git a/docs/requirements.txt b/docs/requirements.txt index 66d41a963f..8d1cbabffc 100644 --- a/docs/requirements.txt +++ b/docs/requirements.txt @@ -22,112 +22,99 @@ certifi==2025.1.31 \ --hash=sha256:3d5da6925056f6f18f119200434a4780a94263f10d1c21d032a6f6b2baa20651 \ --hash=sha256:ca78db4565a652026a4db2bcdf68f2fb589ea80d0be70e03929ed730746b84fe # via requests -charset-normalizer==3.4.0 \ - --hash=sha256:0099d79bdfcf5c1f0c2c72f91516702ebf8b0b8ddd8905f97a8aecf49712c621 \ - --hash=sha256:0713f3adb9d03d49d365b70b84775d0a0d18e4ab08d12bc46baa6132ba78aaf6 \ - --hash=sha256:07afec21bbbbf8a5cc3651aa96b980afe2526e7f048fdfb7f1014d84acc8b6d8 \ - --hash=sha256:0b309d1747110feb25d7ed6b01afdec269c647d382c857ef4663bbe6ad95a912 \ - --hash=sha256:0d99dd8ff461990f12d6e42c7347fd9ab2532fb70e9621ba520f9e8637161d7c \ - --hash=sha256:0de7b687289d3c1b3e8660d0741874abe7888100efe14bd0f9fd7141bcbda92b \ - --hash=sha256:1110e22af8ca26b90bd6364fe4c763329b0ebf1ee213ba32b68c73de5752323d \ - --hash=sha256:130272c698667a982a5d0e626851ceff662565379baf0ff2cc58067b81d4f11d \ - --hash=sha256:136815f06a3ae311fae551c3df1f998a1ebd01ddd424aa5603a4336997629e95 \ - --hash=sha256:14215b71a762336254351b00ec720a8e85cada43b987da5a042e4ce3e82bd68e \ - --hash=sha256:1db4e7fefefd0f548d73e2e2e041f9df5c59e178b4c72fbac4cc6f535cfb1565 \ - --hash=sha256:1ffd9493de4c922f2a38c2bf62b831dcec90ac673ed1ca182fe11b4d8e9f2a64 \ - --hash=sha256:2006769bd1640bdf4d5641c69a3d63b71b81445473cac5ded39740a226fa88ab \ - --hash=sha256:20587d20f557fe189b7947d8e7ec5afa110ccf72a3128d61a2a387c3313f46be \ - --hash=sha256:223217c3d4f82c3ac5e29032b3f1c2eb0fb591b72161f86d93f5719079dae93e \ - --hash=sha256:27623ba66c183eca01bf9ff833875b459cad267aeeb044477fedac35e19ba907 \ - --hash=sha256:285e96d9d53422efc0d7a17c60e59f37fbf3dfa942073f666db4ac71e8d726d0 \ - --hash=sha256:2de62e8801ddfff069cd5c504ce3bc9672b23266597d4e4f50eda28846c322f2 \ - --hash=sha256:2f6c34da58ea9c1a9515621f4d9ac379871a8f21168ba1b5e09d74250de5ad62 \ - --hash=sha256:309a7de0a0ff3040acaebb35ec45d18db4b28232f21998851cfa709eeff49d62 \ - --hash=sha256:35c404d74c2926d0287fbd63ed5d27eb911eb9e4a3bb2c6d294f3cfd4a9e0c23 \ - --hash=sha256:3710a9751938947e6327ea9f3ea6332a09bf0ba0c09cae9cb1f250bd1f1549bc \ - --hash=sha256:3d59d125ffbd6d552765510e3f31ed75ebac2c7470c7274195b9161a32350284 \ - --hash=sha256:40d3ff7fc90b98c637bda91c89d51264a3dcf210cade3a2c6f838c7268d7a4ca \ - --hash=sha256:425c5f215d0eecee9a56cdb703203dda90423247421bf0d67125add85d0c4455 \ - --hash=sha256:43193c5cda5d612f247172016c4bb71251c784d7a4d9314677186a838ad34858 \ - --hash=sha256:44aeb140295a2f0659e113b31cfe92c9061622cadbc9e2a2f7b8ef6b1e29ef4b \ - --hash=sha256:47334db71978b23ebcf3c0f9f5ee98b8d65992b65c9c4f2d34c2eaf5bcaf0594 \ - --hash=sha256:4796efc4faf6b53a18e3d46343535caed491776a22af773f366534056c4e1fbc \ - --hash=sha256:4a51b48f42d9358460b78725283f04bddaf44a9358197b889657deba38f329db \ - --hash=sha256:4b67fdab07fdd3c10bb21edab3cbfe8cf5696f453afce75d815d9d7223fbe88b \ - --hash=sha256:4ec9dd88a5b71abfc74e9df5ebe7921c35cbb3b641181a531ca65cdb5e8e4dea \ - --hash=sha256:4f9fc98dad6c2eaa32fc3af1417d95b5e3d08aff968df0cd320066def971f9a6 \ - --hash=sha256:54b6a92d009cbe2fb11054ba694bc9e284dad30a26757b1e372a1fdddaf21920 \ - --hash=sha256:55f56e2ebd4e3bc50442fbc0888c9d8c94e4e06a933804e2af3e89e2f9c1c749 \ - --hash=sha256:5726cf76c982532c1863fb64d8c6dd0e4c90b6ece9feb06c9f202417a31f7dd7 \ - --hash=sha256:5d447056e2ca60382d460a604b6302d8db69476fd2015c81e7c35417cfabe4cd \ - --hash=sha256:5ed2e36c3e9b4f21dd9422f6893dec0abf2cca553af509b10cd630f878d3eb99 \ - --hash=sha256:5ff2ed8194587faf56555927b3aa10e6fb69d931e33953943bc4f837dfee2242 \ - --hash=sha256:62f60aebecfc7f4b82e3f639a7d1433a20ec32824db2199a11ad4f5e146ef5ee \ - --hash=sha256:63bc5c4ae26e4bc6be6469943b8253c0fd4e4186c43ad46e713ea61a0ba49129 \ - --hash=sha256:6b40e8d38afe634559e398cc32b1472f376a4099c75fe6299ae607e404c033b2 \ - --hash=sha256:6b493a043635eb376e50eedf7818f2f322eabbaa974e948bd8bdd29eb7ef2a51 \ - --hash=sha256:6dba5d19c4dfab08e58d5b36304b3f92f3bd5d42c1a3fa37b5ba5cdf6dfcbcee \ - --hash=sha256:6fd30dc99682dc2c603c2b315bded2799019cea829f8bf57dc6b61efde6611c8 \ - --hash=sha256:707b82d19e65c9bd28b81dde95249b07bf9f5b90ebe1ef17d9b57473f8a64b7b \ - --hash=sha256:7706f5850360ac01d80c89bcef1640683cc12ed87f42579dab6c5d3ed6888613 \ - --hash=sha256:7782afc9b6b42200f7362858f9e73b1f8316afb276d316336c0ec3bd73312742 \ - --hash=sha256:79983512b108e4a164b9c8d34de3992f76d48cadc9554c9e60b43f308988aabe \ - --hash=sha256:7f683ddc7eedd742e2889d2bfb96d69573fde1d92fcb811979cdb7165bb9c7d3 \ - --hash=sha256:82357d85de703176b5587dbe6ade8ff67f9f69a41c0733cf2425378b49954de5 \ - --hash=sha256:84450ba661fb96e9fd67629b93d2941c871ca86fc38d835d19d4225ff946a631 \ - --hash=sha256:86f4e8cca779080f66ff4f191a685ced73d2f72d50216f7112185dc02b90b9b7 \ - --hash=sha256:8cda06946eac330cbe6598f77bb54e690b4ca93f593dee1568ad22b04f347c15 \ - --hash=sha256:8ce7fd6767a1cc5a92a639b391891bf1c268b03ec7e021c7d6d902285259685c \ - --hash=sha256:8ff4e7cdfdb1ab5698e675ca622e72d58a6fa2a8aa58195de0c0061288e6e3ea \ - --hash=sha256:9289fd5dddcf57bab41d044f1756550f9e7cf0c8e373b8cdf0ce8773dc4bd417 \ - --hash=sha256:92a7e36b000bf022ef3dbb9c46bfe2d52c047d5e3f3343f43204263c5addc250 \ - --hash=sha256:92db3c28b5b2a273346bebb24857fda45601aef6ae1c011c0a997106581e8a88 \ - --hash=sha256:95c3c157765b031331dd4db3c775e58deaee050a3042fcad72cbc4189d7c8dca \ - --hash=sha256:980b4f289d1d90ca5efcf07958d3eb38ed9c0b7676bf2831a54d4f66f9c27dfa \ - --hash=sha256:9ae4ef0b3f6b41bad6366fb0ea4fc1d7ed051528e113a60fa2a65a9abb5b1d99 \ - --hash=sha256:9c98230f5042f4945f957d006edccc2af1e03ed5e37ce7c373f00a5a4daa6149 \ - --hash=sha256:9fa2566ca27d67c86569e8c85297aaf413ffab85a8960500f12ea34ff98e4c41 \ - --hash=sha256:a14969b8691f7998e74663b77b4c36c0337cb1df552da83d5c9004a93afdb574 \ - --hash=sha256:a8aacce6e2e1edcb6ac625fb0f8c3a9570ccc7bfba1f63419b3769ccf6a00ed0 \ - --hash=sha256:a8e538f46104c815be19c975572d74afb53f29650ea2025bbfaef359d2de2f7f \ - --hash=sha256:aa41e526a5d4a9dfcfbab0716c7e8a1b215abd3f3df5a45cf18a12721d31cb5d \ - --hash=sha256:aa693779a8b50cd97570e5a0f343538a8dbd3e496fa5dcb87e29406ad0299654 \ - --hash=sha256:ab22fbd9765e6954bc0bcff24c25ff71dcbfdb185fcdaca49e81bac68fe724d3 \ - --hash=sha256:ab2e5bef076f5a235c3774b4f4028a680432cded7cad37bba0fd90d64b187d19 \ - --hash=sha256:ab973df98fc99ab39080bfb0eb3a925181454d7c3ac8a1e695fddfae696d9e90 \ - --hash=sha256:af73657b7a68211996527dbfeffbb0864e043d270580c5aef06dc4b659a4b578 \ - --hash=sha256:b197e7094f232959f8f20541ead1d9862ac5ebea1d58e9849c1bf979255dfac9 \ - --hash=sha256:b295729485b06c1a0683af02a9e42d2caa9db04a373dc38a6a58cdd1e8abddf1 \ - --hash=sha256:b8831399554b92b72af5932cdbbd4ddc55c55f631bb13ff8fe4e6536a06c5c51 \ - --hash=sha256:b8dcd239c743aa2f9c22ce674a145e0a25cb1566c495928440a181ca1ccf6719 \ - --hash=sha256:bcb4f8ea87d03bc51ad04add8ceaf9b0f085ac045ab4d74e73bbc2dc033f0236 \ - --hash=sha256:bd7af3717683bea4c87acd8c0d3d5b44d56120b26fd3f8a692bdd2d5260c620a \ - --hash=sha256:bf4475b82be41b07cc5e5ff94810e6a01f276e37c2d55571e3fe175e467a1a1c \ - --hash=sha256:c3e446d253bd88f6377260d07c895816ebf33ffffd56c1c792b13bff9c3e1ade \ - --hash=sha256:c57516e58fd17d03ebe67e181a4e4e2ccab1168f8c2976c6a334d4f819fe5944 \ - --hash=sha256:c94057af19bc953643a33581844649a7fdab902624d2eb739738a30e2b3e60fc \ - --hash=sha256:cab5d0b79d987c67f3b9e9c53f54a61360422a5a0bc075f43cab5621d530c3b6 \ - --hash=sha256:ce031db0408e487fd2775d745ce30a7cd2923667cf3b69d48d219f1d8f5ddeb6 \ - --hash=sha256:cee4373f4d3ad28f1ab6290684d8e2ebdb9e7a1b74fdc39e4c211995f77bec27 \ - --hash=sha256:d5b054862739d276e09928de37c79ddeec42a6e1bfc55863be96a36ba22926f6 \ - --hash=sha256:dbe03226baf438ac4fda9e2d0715022fd579cb641c4cf639fa40d53b2fe6f3e2 \ - --hash=sha256:dc15e99b2d8a656f8e666854404f1ba54765871104e50c8e9813af8a7db07f12 \ - --hash=sha256:dcaf7c1524c0542ee2fc82cc8ec337f7a9f7edee2532421ab200d2b920fc97cf \ - --hash=sha256:dd4eda173a9fcccb5f2e2bd2a9f423d180194b1bf17cf59e3269899235b2a114 \ - --hash=sha256:dd9a8bd8900e65504a305bf8ae6fa9fbc66de94178c420791d0293702fce2df7 \ - --hash=sha256:de7376c29d95d6719048c194a9cf1a1b0393fbe8488a22008610b0361d834ecf \ - --hash=sha256:e7fdd52961feb4c96507aa649550ec2a0d527c086d284749b2f582f2d40a2e0d \ - --hash=sha256:e91f541a85298cf35433bf66f3fab2a4a2cff05c127eeca4af174f6d497f0d4b \ - --hash=sha256:e9e3c4c9e1ed40ea53acf11e2a386383c3304212c965773704e4603d589343ed \ - --hash=sha256:ee803480535c44e7f5ad00788526da7d85525cfefaf8acf8ab9a310000be4b03 \ - --hash=sha256:f09cb5a7bbe1ecae6e87901a2eb23e0256bb524a79ccc53eb0b7629fbe7677c4 \ - --hash=sha256:f19c1585933c82098c2a520f8ec1227f20e339e33aca8fa6f956f6691b784e67 \ - --hash=sha256:f1a2f519ae173b5b6a2c9d5fa3116ce16e48b3462c8b96dfdded11055e3d6365 \ - --hash=sha256:f28f891ccd15c514a0981f3b9db9aa23d62fe1a99997512b0491d2ed323d229a \ - --hash=sha256:f3e73a4255342d4eb26ef6df01e3962e73aa29baa3124a8e824c5d3364a65748 \ - --hash=sha256:f606a1881d2663630ea5b8ce2efe2111740df4b687bd78b34a8131baa007f79b \ - --hash=sha256:fe9f97feb71aa9896b81973a7bbada8c49501dc73e58a10fcef6663af95e5079 \ - --hash=sha256:ffc519621dce0c767e96b9c53f09c5d215578e10b02c285809f76509a3931482 +charset-normalizer==3.4.1 \ + --hash=sha256:0167ddc8ab6508fe81860a57dd472b2ef4060e8d378f0cc555707126830f2537 \ + --hash=sha256:01732659ba9b5b873fc117534143e4feefecf3b2078b0a6a2e925271bb6f4cfa \ + --hash=sha256:01ad647cdd609225c5350561d084b42ddf732f4eeefe6e678765636791e78b9a \ + --hash=sha256:04432ad9479fa40ec0f387795ddad4437a2b50417c69fa275e212933519ff294 \ + --hash=sha256:0907f11d019260cdc3f94fbdb23ff9125f6b5d1039b76003b5b0ac9d6a6c9d5b \ + --hash=sha256:0924e81d3d5e70f8126529951dac65c1010cdf117bb75eb02dd12339b57749dd \ + --hash=sha256:09b26ae6b1abf0d27570633b2b078a2a20419c99d66fb2823173d73f188ce601 \ + --hash=sha256:09b5e6733cbd160dcc09589227187e242a30a49ca5cefa5a7edd3f9d19ed53fd \ + --hash=sha256:0af291f4fe114be0280cdd29d533696a77b5b49cfde5467176ecab32353395c4 \ + --hash=sha256:0f55e69f030f7163dffe9fd0752b32f070566451afe180f99dbeeb81f511ad8d \ + --hash=sha256:1a2bc9f351a75ef49d664206d51f8e5ede9da246602dc2d2726837620ea034b2 \ + --hash=sha256:22e14b5d70560b8dd51ec22863f370d1e595ac3d024cb8ad7d308b4cd95f8313 \ + --hash=sha256:234ac59ea147c59ee4da87a0c0f098e9c8d169f4dc2a159ef720f1a61bbe27cd \ + --hash=sha256:2369eea1ee4a7610a860d88f268eb39b95cb588acd7235e02fd5a5601773d4fa \ + --hash=sha256:237bdbe6159cff53b4f24f397d43c6336c6b0b42affbe857970cefbb620911c8 \ + --hash=sha256:28bf57629c75e810b6ae989f03c0828d64d6b26a5e205535585f96093e405ed1 \ + --hash=sha256:2967f74ad52c3b98de4c3b32e1a44e32975e008a9cd2a8cc8966d6a5218c5cb2 \ + --hash=sha256:2a75d49014d118e4198bcee5ee0a6f25856b29b12dbf7cd012791f8a6cc5c496 \ + --hash=sha256:2bdfe3ac2e1bbe5b59a1a63721eb3b95fc9b6817ae4a46debbb4e11f6232428d \ + --hash=sha256:2d074908e1aecee37a7635990b2c6d504cd4766c7bc9fc86d63f9c09af3fa11b \ + --hash=sha256:2fb9bd477fdea8684f78791a6de97a953c51831ee2981f8e4f583ff3b9d9687e \ + --hash=sha256:311f30128d7d333eebd7896965bfcfbd0065f1716ec92bd5638d7748eb6f936a \ + --hash=sha256:329ce159e82018d646c7ac45b01a430369d526569ec08516081727a20e9e4af4 \ + --hash=sha256:345b0426edd4e18138d6528aed636de7a9ed169b4aaf9d61a8c19e39d26838ca \ + --hash=sha256:363e2f92b0f0174b2f8238240a1a30142e3db7b957a5dd5689b0e75fb717cc78 \ + --hash=sha256:3a3bd0dcd373514dcec91c411ddb9632c0d7d92aed7093b8c3bbb6d69ca74408 \ + --hash=sha256:3bed14e9c89dcb10e8f3a29f9ccac4955aebe93c71ae803af79265c9ca5644c5 \ + --hash=sha256:44251f18cd68a75b56585dd00dae26183e102cd5e0f9f1466e6df5da2ed64ea3 \ + --hash=sha256:44ecbf16649486d4aebafeaa7ec4c9fed8b88101f4dd612dcaf65d5e815f837f \ + --hash=sha256:4532bff1b8421fd0a320463030c7520f56a79c9024a4e88f01c537316019005a \ + --hash=sha256:49402233c892a461407c512a19435d1ce275543138294f7ef013f0b63d5d3765 \ + --hash=sha256:4c0907b1928a36d5a998d72d64d8eaa7244989f7aaaf947500d3a800c83a3fd6 \ + --hash=sha256:4d86f7aff21ee58f26dcf5ae81a9addbd914115cdebcbb2217e4f0ed8982e146 \ + --hash=sha256:5777ee0881f9499ed0f71cc82cf873d9a0ca8af166dfa0af8ec4e675b7df48e6 \ + --hash=sha256:5df196eb874dae23dcfb968c83d4f8fdccb333330fe1fc278ac5ceeb101003a9 \ + --hash=sha256:619a609aa74ae43d90ed2e89bdd784765de0a25ca761b93e196d938b8fd1dbbd \ + --hash=sha256:6e27f48bcd0957c6d4cb9d6fa6b61d192d0b13d5ef563e5f2ae35feafc0d179c \ + --hash=sha256:6ff8a4a60c227ad87030d76e99cd1698345d4491638dfa6673027c48b3cd395f \ + --hash=sha256:73d94b58ec7fecbc7366247d3b0b10a21681004153238750bb67bd9012414545 \ + --hash=sha256:7461baadb4dc00fd9e0acbe254e3d7d2112e7f92ced2adc96e54ef6501c5f176 \ + --hash=sha256:75832c08354f595c760a804588b9357d34ec00ba1c940c15e31e96d902093770 \ + --hash=sha256:7709f51f5f7c853f0fb938bcd3bc59cdfdc5203635ffd18bf354f6967ea0f824 \ + --hash=sha256:78baa6d91634dfb69ec52a463534bc0df05dbd546209b79a3880a34487f4b84f \ + --hash=sha256:7974a0b5ecd505609e3b19742b60cee7aa2aa2fb3151bc917e6e2646d7667dcf \ + --hash=sha256:7a4f97a081603d2050bfaffdefa5b02a9ec823f8348a572e39032caa8404a487 \ + --hash=sha256:7b1bef6280950ee6c177b326508f86cad7ad4dff12454483b51d8b7d673a2c5d \ + --hash=sha256:7d053096f67cd1241601111b698f5cad775f97ab25d81567d3f59219b5f1adbd \ + --hash=sha256:804a4d582ba6e5b747c625bf1255e6b1507465494a40a2130978bda7b932c90b \ + --hash=sha256:807f52c1f798eef6cf26beb819eeb8819b1622ddfeef9d0977a8502d4db6d534 \ + --hash=sha256:80ed5e856eb7f30115aaf94e4a08114ccc8813e6ed1b5efa74f9f82e8509858f \ + --hash=sha256:8417cb1f36cc0bc7eaba8ccb0e04d55f0ee52df06df3ad55259b9a323555fc8b \ + --hash=sha256:8436c508b408b82d87dc5f62496973a1805cd46727c34440b0d29d8a2f50a6c9 \ + --hash=sha256:89149166622f4db9b4b6a449256291dc87a99ee53151c74cbd82a53c8c2f6ccd \ + --hash=sha256:8bfa33f4f2672964266e940dd22a195989ba31669bd84629f05fab3ef4e2d125 \ + --hash=sha256:8c60ca7339acd497a55b0ea5d506b2a2612afb2826560416f6894e8b5770d4a9 \ + --hash=sha256:91b36a978b5ae0ee86c394f5a54d6ef44db1de0815eb43de826d41d21e4af3de \ + --hash=sha256:955f8851919303c92343d2f66165294848d57e9bba6cf6e3625485a70a038d11 \ + --hash=sha256:97f68b8d6831127e4787ad15e6757232e14e12060bec17091b85eb1486b91d8d \ + --hash=sha256:9b23ca7ef998bc739bf6ffc077c2116917eabcc901f88da1b9856b210ef63f35 \ + --hash=sha256:9f0b8b1c6d84c8034a44893aba5e767bf9c7a211e313a9605d9c617d7083829f \ + --hash=sha256:aabfa34badd18f1da5ec1bc2715cadc8dca465868a4e73a0173466b688f29dda \ + --hash=sha256:ab36c8eb7e454e34e60eb55ca5d241a5d18b2c6244f6827a30e451c42410b5f7 \ + --hash=sha256:b010a7a4fd316c3c484d482922d13044979e78d1861f0e0650423144c616a46a \ + --hash=sha256:b1ac5992a838106edb89654e0aebfc24f5848ae2547d22c2c3f66454daa11971 \ + --hash=sha256:b7b2d86dd06bfc2ade3312a83a5c364c7ec2e3498f8734282c6c3d4b07b346b8 \ + --hash=sha256:b97e690a2118911e39b4042088092771b4ae3fc3aa86518f84b8cf6888dbdb41 \ + --hash=sha256:bc2722592d8998c870fa4e290c2eec2c1569b87fe58618e67d38b4665dfa680d \ + --hash=sha256:c0429126cf75e16c4f0ad00ee0eae4242dc652290f940152ca8c75c3a4b6ee8f \ + --hash=sha256:c30197aa96e8eed02200a83fba2657b4c3acd0f0aa4bdc9f6c1af8e8962e0757 \ + --hash=sha256:c4c3e6da02df6fa1410a7680bd3f63d4f710232d3139089536310d027950696a \ + --hash=sha256:c75cb2a3e389853835e84a2d8fb2b81a10645b503eca9bcb98df6b5a43eb8886 \ + --hash=sha256:c96836c97b1238e9c9e3fe90844c947d5afbf4f4c92762679acfe19927d81d77 \ + --hash=sha256:d7f50a1f8c450f3925cb367d011448c39239bb3eb4117c36a6d354794de4ce76 \ + --hash=sha256:d973f03c0cb71c5ed99037b870f2be986c3c05e63622c017ea9816881d2dd247 \ + --hash=sha256:d98b1668f06378c6dbefec3b92299716b931cd4e6061f3c875a71ced1780ab85 \ + --hash=sha256:d9c3cdf5390dcd29aa8056d13e8e99526cda0305acc038b96b30352aff5ff2bb \ + --hash=sha256:dad3e487649f498dd991eeb901125411559b22e8d7ab25d3aeb1af367df5efd7 \ + --hash=sha256:dccbe65bd2f7f7ec22c4ff99ed56faa1e9f785482b9bbd7c717e26fd723a1d1e \ + --hash=sha256:dd78cfcda14a1ef52584dbb008f7ac81c1328c0f58184bf9a84c49c605002da6 \ + --hash=sha256:e218488cd232553829be0664c2292d3af2eeeb94b32bea483cf79ac6a694e037 \ + --hash=sha256:e358e64305fe12299a08e08978f51fc21fac060dcfcddd95453eabe5b93ed0e1 \ + --hash=sha256:ea0d8d539afa5eb2728aa1932a988a9a7af94f18582ffae4bc10b3fbdad0626e \ + --hash=sha256:eab677309cdb30d047996b36d34caeda1dc91149e4fdca0b1a039b3f79d9a807 \ + --hash=sha256:eb8178fe3dba6450a3e024e95ac49ed3400e506fd4e9e5c32d30adda88cbd407 \ + --hash=sha256:ecddf25bee22fe4fe3737a399d0d177d72bc22be6913acfab364b40bce1ba83c \ + --hash=sha256:eea6ee1db730b3483adf394ea72f808b6e18cf3cb6454b4d86e04fa8c4327a12 \ + --hash=sha256:f08ff5e948271dc7e18a35641d2f11a4cd8dfd5634f55228b691e62b37125eb3 \ + --hash=sha256:f30bf9fd9be89ecb2360c7d94a711f00c09b976258846efe40db3d05828e8089 \ + --hash=sha256:fa88b843d6e211393a37219e6a1c1df99d35e8fd90446f1118f4216e307e48cd \ + --hash=sha256:fc54db6c8593ef7d4b2a331b58653356cf04f67c960f584edb7c3d8c97e8f39e \ + --hash=sha256:fd4ec41f914fa74ad1b8304bbc634b3de73d2a0889bd32076342a573e0779e00 \ + --hash=sha256:ffc9202a29ab3920fa812879e95a9e78b2465fd10be7fcbd042899695d75e616 # via requests colorama==0.4.6 ; sys_platform == 'win32' \ --hash=sha256:08695f5cb7ed6e0531a20572697297273c47b8cae5a63ffc6d6ed5c201be6e44 \ From 2a710f07c2eafd5c6d32d4721ee4403a34769361 Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Tue, 8 Apr 2025 10:47:08 +0900 Subject: [PATCH 155/922] build(deps): bump jinja2 from 3.1.4 to 3.1.6 in /examples/pip_parse (#2754) Bumps [jinja2](https://github.com/pallets/jinja) from 3.1.4 to 3.1.6.
Release notes

Sourced from jinja2's releases.

3.1.6

This is the Jinja 3.1.6 security release, which fixes security issues but does not otherwise change behavior and should not result in breaking changes compared to the latest feature release.

PyPI: https://pypi.org/project/Jinja2/3.1.6/ Changes: https://jinja.palletsprojects.com/en/stable/changes/#version-3-1-6

3.1.5

This is the Jinja 3.1.5 security fix release, which fixes security issues and bugs but does not otherwise change behavior and should not result in breaking changes compared to the latest feature release.

PyPI: https://pypi.org/project/Jinja2/3.1.5/ Changes: https://jinja.palletsprojects.com/changes/#version-3-1-5 Milestone: https://github.com/pallets/jinja/milestone/16?closed=1

  • The sandboxed environment handles indirect calls to str.format, such as by passing a stored reference to a filter that calls its argument. GHSA-q2x7-8rv6-6q7h
  • Escape template name before formatting it into error messages, to avoid issues with names that contain f-string syntax. #1792, GHSA-gmj6-6f8f-6699
  • Sandbox does not allow clear and pop on known mutable sequence types. #2032
  • Calling sync render for an async template uses asyncio.run. #1952
  • Avoid unclosed auto_aiter warnings. #1960
  • Return an aclose-able AsyncGenerator from Template.generate_async. #1960
  • Avoid leaving root_render_func() unclosed in Template.generate_async. #1960
  • Avoid leaving async generators unclosed in blocks, includes and extends. #1960
  • The runtime uses the correct concat function for the current environment when calling block references. #1701
  • Make |unique async-aware, allowing it to be used after another async-aware filter. #1781
  • |int filter handles OverflowError from scientific notation. #1921
  • Make compiling deterministic for tuple unpacking in a {% set ... %} call. #2021
  • Fix dunder protocol (copy/pickle/etc) interaction with Undefined objects. #2025
  • Fix copy/pickle support for the internal missing object. #2027
  • Environment.overlay(enable_async) is applied correctly. #2061
  • The error message from FileSystemLoader includes the paths that were searched. #1661
  • PackageLoader shows a clearer error message when the package does not contain the templates directory. #1705
  • Improve annotations for methods returning copies. #1880
  • urlize does not add mailto: to values like @a@b. #1870
  • Tests decorated with @pass_context can be used with the |select filter. #1624
  • Using set for multiple assignment (a, b = 1, 2) does not fail when the target is a namespace attribute. #1413
  • Using set in all branches of {% if %}{% elif %}{% else %} blocks does not cause the variable to be considered initially undefined. #1253
Changelog

Sourced from jinja2's changelog.

Version 3.1.6

Released 2025-03-05

  • The |attr filter does not bypass the environment's attribute lookup, allowing the sandbox to apply its checks. :ghsa:cpwx-vrp4-4pq7

Version 3.1.5

Released 2024-12-21

  • The sandboxed environment handles indirect calls to str.format, such as by passing a stored reference to a filter that calls its argument. :ghsa:q2x7-8rv6-6q7h
  • Escape template name before formatting it into error messages, to avoid issues with names that contain f-string syntax. :issue:1792, :ghsa:gmj6-6f8f-6699
  • Sandbox does not allow clear and pop on known mutable sequence types. :issue:2032
  • Calling sync render for an async template uses asyncio.run. :pr:1952
  • Avoid unclosed auto_aiter warnings. :pr:1960
  • Return an aclose-able AsyncGenerator from Template.generate_async. :pr:1960
  • Avoid leaving root_render_func() unclosed in Template.generate_async. :pr:1960
  • Avoid leaving async generators unclosed in blocks, includes and extends. :pr:1960
  • The runtime uses the correct concat function for the current environment when calling block references. :issue:1701
  • Make |unique async-aware, allowing it to be used after another async-aware filter. :issue:1781
  • |int filter handles OverflowError from scientific notation. :issue:1921
  • Make compiling deterministic for tuple unpacking in a {% set ... %} call. :issue:2021
  • Fix dunder protocol (copy/pickle/etc) interaction with Undefined objects. :issue:2025
  • Fix copy/pickle support for the internal missing object. :issue:2027
  • Environment.overlay(enable_async) is applied correctly. :pr:2061
  • The error message from FileSystemLoader includes the paths that were searched. :issue:1661
  • PackageLoader shows a clearer error message when the package does not contain the templates directory. :issue:1705
  • Improve annotations for methods returning copies. :pr:1880
  • urlize does not add mailto: to values like @a@b. :pr:1870

... (truncated)

Commits

[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=jinja2&package-manager=pip&previous-version=3.1.4&new-version=3.1.6)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) ---
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot show ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) You can disable automated security fix PRs for this repo from the [Security Alerts page](https://github.com/bazel-contrib/rules_python/network/alerts).
Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> --- examples/pip_parse/requirements_lock.txt | 6 +++--- examples/pip_parse/requirements_windows.txt | 6 +++--- 2 files changed, 6 insertions(+), 6 deletions(-) diff --git a/examples/pip_parse/requirements_lock.txt b/examples/pip_parse/requirements_lock.txt index 5e7a198c38..aeac61eff9 100644 --- a/examples/pip_parse/requirements_lock.txt +++ b/examples/pip_parse/requirements_lock.txt @@ -36,9 +36,9 @@ importlib-metadata==6.8.0 \ --hash=sha256:3ebb78df84a805d7698245025b975d9d67053cd94c79245ba4b3eb694abe68bb \ --hash=sha256:dbace7892d8c0c4ac1ad096662232f831d4e64f4c4545bd53016a3e9d4654743 # via sphinx -jinja2==3.1.4 \ - --hash=sha256:4a3aee7acbbe7303aede8e9648d13b8bf88a429282aa6122a993f0ac800cb369 \ - --hash=sha256:bc5dd2abb727a5319567b7a813e6a2e7318c39f4f487cfe6c89c6f9c7d25197d +jinja2==3.1.6 \ + --hash=sha256:0137fb05990d35f1275a587e9aee6d56da821fc83491a0fb838183be43f66d6d \ + --hash=sha256:85ece4451f492d0c13c5dd7c13a64681a86afae63a5f347908daf103ce6d2f67 # via sphinx markupsafe==2.1.3 \ --hash=sha256:05fb21170423db021895e1ea1e1f3ab3adb85d1c2333cbc2310f2a26bc77272e \ diff --git a/examples/pip_parse/requirements_windows.txt b/examples/pip_parse/requirements_windows.txt index 4b1969255a..61a6682047 100644 --- a/examples/pip_parse/requirements_windows.txt +++ b/examples/pip_parse/requirements_windows.txt @@ -40,9 +40,9 @@ importlib-metadata==6.8.0 \ --hash=sha256:3ebb78df84a805d7698245025b975d9d67053cd94c79245ba4b3eb694abe68bb \ --hash=sha256:dbace7892d8c0c4ac1ad096662232f831d4e64f4c4545bd53016a3e9d4654743 # via sphinx -jinja2==3.1.4 \ - --hash=sha256:4a3aee7acbbe7303aede8e9648d13b8bf88a429282aa6122a993f0ac800cb369 \ - --hash=sha256:bc5dd2abb727a5319567b7a813e6a2e7318c39f4f487cfe6c89c6f9c7d25197d +jinja2==3.1.6 \ + --hash=sha256:0137fb05990d35f1275a587e9aee6d56da821fc83491a0fb838183be43f66d6d \ + --hash=sha256:85ece4451f492d0c13c5dd7c13a64681a86afae63a5f347908daf103ce6d2f67 # via sphinx markupsafe==2.1.3 \ --hash=sha256:05fb21170423db021895e1ea1e1f3ab3adb85d1c2333cbc2310f2a26bc77272e \ From 6821709d7c79e9a1156287d06522de674e5c376d Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Tue, 8 Apr 2025 02:21:43 +0000 Subject: [PATCH 156/922] build(deps): bump cryptography from 43.0.3 to 44.0.1 in /tools/publish (#2756) Bumps [cryptography](https://github.com/pyca/cryptography) from 43.0.3 to 44.0.1.
Changelog

Sourced from cryptography's changelog.

44.0.1 - 2025-02-11


* Updated Windows, macOS, and Linux wheels to be compiled with OpenSSL
3.4.1.
* We now build ``armv7l`` ``manylinux`` wheels and publish them to PyPI.
* We now build ``manylinux_2_34`` wheels and publish them to PyPI.

.. _v44-0-0:

44.0.0 - 2024-11-27

  • BACKWARDS INCOMPATIBLE: Dropped support for LibreSSL < 3.9.
  • Deprecated Python 3.7 support. Python 3.7 is no longer supported by the Python core team. Support for Python 3.7 will be removed in a future cryptography release.
  • Updated Windows, macOS, and Linux wheels to be compiled with OpenSSL 3.4.0.
  • macOS wheels are now built against the macOS 10.13 SDK. Users on older versions of macOS should upgrade, or they will need to build cryptography themselves.
  • Enforce the :rfc:5280 requirement that extended key usage extensions must not be empty.
  • Added support for timestamp extraction to the :class:~cryptography.fernet.MultiFernet class.
  • Relax the Authority Key Identifier requirements on root CA certificates during X.509 verification to allow fields permitted by :rfc:5280 but forbidden by the CA/Browser BRs.
  • Added support for :class:~cryptography.hazmat.primitives.kdf.argon2.Argon2id when using OpenSSL 3.2.0+.
  • Added support for the :class:~cryptography.x509.Admissions certificate extension.
  • Added basic support for PKCS7 decryption (including S/MIME 3.2) via :func:~cryptography.hazmat.primitives.serialization.pkcs7.pkcs7_decrypt_der, :func:~cryptography.hazmat.primitives.serialization.pkcs7.pkcs7_decrypt_pem, and :func:~cryptography.hazmat.primitives.serialization.pkcs7.pkcs7_decrypt_smime.

.. _v43-0-3:

Commits

[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=cryptography&package-manager=pip&previous-version=43.0.3&new-version=44.0.1)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) ---
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot show ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) You can disable automated security fix PRs for this repo from the [Security Alerts page](https://github.com/bazel-contrib/rules_python/network/alerts).
Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> --- tools/publish/requirements_linux.txt | 60 +++++++++++++----------- tools/publish/requirements_universal.txt | 60 +++++++++++++----------- 2 files changed, 64 insertions(+), 56 deletions(-) diff --git a/tools/publish/requirements_linux.txt b/tools/publish/requirements_linux.txt index 90b07d4c97..40d987b16d 100644 --- a/tools/publish/requirements_linux.txt +++ b/tools/publish/requirements_linux.txt @@ -173,34 +173,38 @@ charset-normalizer==3.4.1 \ --hash=sha256:fd4ec41f914fa74ad1b8304bbc634b3de73d2a0889bd32076342a573e0779e00 \ --hash=sha256:ffc9202a29ab3920fa812879e95a9e78b2465fd10be7fcbd042899695d75e616 # via requests -cryptography==43.0.3 \ - --hash=sha256:0c580952eef9bf68c4747774cde7ec1d85a6e61de97281f2dba83c7d2c806362 \ - --hash=sha256:0f996e7268af62598f2fc1204afa98a3b5712313a55c4c9d434aef49cadc91d4 \ - --hash=sha256:1ec0bcf7e17c0c5669d881b1cd38c4972fade441b27bda1051665faaa89bdcaa \ - --hash=sha256:281c945d0e28c92ca5e5930664c1cefd85efe80e5c0d2bc58dd63383fda29f83 \ - --hash=sha256:2ce6fae5bdad59577b44e4dfed356944fbf1d925269114c28be377692643b4ff \ - --hash=sha256:315b9001266a492a6ff443b61238f956b214dbec9910a081ba5b6646a055a805 \ - --hash=sha256:443c4a81bb10daed9a8f334365fe52542771f25aedaf889fd323a853ce7377d6 \ - --hash=sha256:4a02ded6cd4f0a5562a8887df8b3bd14e822a90f97ac5e544c162899bc467664 \ - --hash=sha256:53a583b6637ab4c4e3591a15bc9db855b8d9dee9a669b550f311480acab6eb08 \ - --hash=sha256:63efa177ff54aec6e1c0aefaa1a241232dcd37413835a9b674b6e3f0ae2bfd3e \ - --hash=sha256:74f57f24754fe349223792466a709f8e0c093205ff0dca557af51072ff47ab18 \ - --hash=sha256:7e1ce50266f4f70bf41a2c6dc4358afadae90e2a1e5342d3c08883df1675374f \ - --hash=sha256:81ef806b1fef6b06dcebad789f988d3b37ccaee225695cf3e07648eee0fc6b73 \ - --hash=sha256:846da004a5804145a5f441b8530b4bf35afbf7da70f82409f151695b127213d5 \ - --hash=sha256:8ac43ae87929a5982f5948ceda07001ee5e83227fd69cf55b109144938d96984 \ - --hash=sha256:9762ea51a8fc2a88b70cf2995e5675b38d93bf36bd67d91721c309df184f49bd \ - --hash=sha256:a2a431ee15799d6db9fe80c82b055bae5a752bef645bba795e8e52687c69efe3 \ - --hash=sha256:bf7a1932ac4176486eab36a19ed4c0492da5d97123f1406cf15e41b05e787d2e \ - --hash=sha256:c2e6fc39c4ab499049df3bdf567f768a723a5e8464816e8f009f121a5a9f4405 \ - --hash=sha256:cbeb489927bd7af4aa98d4b261af9a5bc025bd87f0e3547e11584be9e9427be2 \ - --hash=sha256:d03b5621a135bffecad2c73e9f4deb1a0f977b9a8ffe6f8e002bf6c9d07b918c \ - --hash=sha256:d56e96520b1020449bbace2b78b603442e7e378a9b3bd68de65c782db1507995 \ - --hash=sha256:df6b6c6d742395dd77a23ea3728ab62f98379eff8fb61be2744d4679ab678f73 \ - --hash=sha256:e1be4655c7ef6e1bbe6b5d0403526601323420bcf414598955968c9ef3eb7d16 \ - --hash=sha256:f18c716be16bc1fea8e95def49edf46b82fccaa88587a45f8dc0ff6ab5d8e0a7 \ - --hash=sha256:f46304d6f0c6ab8e52770addfa2fc41e6629495548862279641972b6215451cd \ - --hash=sha256:f7b178f11ed3664fd0e995a47ed2b5ff0a12d893e41dd0494f406d1cf555cab7 +cryptography==44.0.1 \ + --hash=sha256:00918d859aa4e57db8299607086f793fa7813ae2ff5a4637e318a25ef82730f7 \ + --hash=sha256:1e8d181e90a777b63f3f0caa836844a1182f1f265687fac2115fcf245f5fbec3 \ + --hash=sha256:1f9a92144fa0c877117e9748c74501bea842f93d21ee00b0cf922846d9d0b183 \ + --hash=sha256:21377472ca4ada2906bc313168c9dc7b1d7ca417b63c1c3011d0c74b7de9ae69 \ + --hash=sha256:24979e9f2040c953a94bf3c6782e67795a4c260734e5264dceea65c8f4bae64a \ + --hash=sha256:2a46a89ad3e6176223b632056f321bc7de36b9f9b93b2cc1cccf935a3849dc62 \ + --hash=sha256:322eb03ecc62784536bc173f1483e76747aafeb69c8728df48537eb431cd1911 \ + --hash=sha256:436df4f203482f41aad60ed1813811ac4ab102765ecae7a2bbb1dbb66dcff5a7 \ + --hash=sha256:4f422e8c6a28cf8b7f883eb790695d6d45b0c385a2583073f3cec434cc705e1a \ + --hash=sha256:53f23339864b617a3dfc2b0ac8d5c432625c80014c25caac9082314e9de56f41 \ + --hash=sha256:5fed5cd6102bb4eb843e3315d2bf25fede494509bddadb81e03a859c1bc17b83 \ + --hash=sha256:610a83540765a8d8ce0f351ce42e26e53e1f774a6efb71eb1b41eb01d01c3d12 \ + --hash=sha256:6c8acf6f3d1f47acb2248ec3ea261171a671f3d9428e34ad0357148d492c7864 \ + --hash=sha256:6f76fdd6fd048576a04c5210d53aa04ca34d2ed63336d4abd306d0cbe298fddf \ + --hash=sha256:72198e2b5925155497a5a3e8c216c7fb3e64c16ccee11f0e7da272fa93b35c4c \ + --hash=sha256:887143b9ff6bad2b7570da75a7fe8bbf5f65276365ac259a5d2d5147a73775f2 \ + --hash=sha256:888fcc3fce0c888785a4876ca55f9f43787f4c5c1cc1e2e0da71ad481ff82c5b \ + --hash=sha256:8e6a85a93d0642bd774460a86513c5d9d80b5c002ca9693e63f6e540f1815ed0 \ + --hash=sha256:94f99f2b943b354a5b6307d7e8d19f5c423a794462bde2bf310c770ba052b1c4 \ + --hash=sha256:9b336599e2cb77b1008cb2ac264b290803ec5e8e89d618a5e978ff5eb6f715d9 \ + --hash=sha256:a2d8a7045e1ab9b9f803f0d9531ead85f90c5f2859e653b61497228b18452008 \ + --hash=sha256:b8272f257cf1cbd3f2e120f14c68bff2b6bdfcc157fafdee84a1b795efd72862 \ + --hash=sha256:bf688f615c29bfe9dfc44312ca470989279f0e94bb9f631f85e3459af8efc009 \ + --hash=sha256:d9c5b9f698a83c8bd71e0f4d3f9f839ef244798e5ffe96febfa9714717db7af7 \ + --hash=sha256:dd7c7e2d71d908dc0f8d2027e1604102140d84b155e658c20e8ad1304317691f \ + --hash=sha256:df978682c1504fc93b3209de21aeabf2375cb1571d4e61907b3e7a2540e83026 \ + --hash=sha256:e403f7f766ded778ecdb790da786b418a9f2394f36e8cc8b796cc056ab05f44f \ + --hash=sha256:eb3889330f2a4a148abead555399ec9a32b13b7c8ba969b72d8e500eb7ef84cd \ + --hash=sha256:f4daefc971c2d1f82f03097dc6f216744a6cd2ac0f04c68fb935ea2ba2a0d420 \ + --hash=sha256:f51f5705ab27898afda1aaa430f34ad90dc117421057782022edf0600bec5f14 \ + --hash=sha256:fd0ee90072861e276b0ff08bd627abec29e32a53b2be44e41dbcdf87cbee2b00 # via secretstorage docutils==0.21.2 \ --hash=sha256:3a6b18732edf182daa3cd12775bbb338cf5691468f91eeeb109deff6ebfa986f \ diff --git a/tools/publish/requirements_universal.txt b/tools/publish/requirements_universal.txt index 9b145fce49..c8bc0bb258 100644 --- a/tools/publish/requirements_universal.txt +++ b/tools/publish/requirements_universal.txt @@ -173,34 +173,38 @@ charset-normalizer==3.4.1 \ --hash=sha256:fd4ec41f914fa74ad1b8304bbc634b3de73d2a0889bd32076342a573e0779e00 \ --hash=sha256:ffc9202a29ab3920fa812879e95a9e78b2465fd10be7fcbd042899695d75e616 # via requests -cryptography==43.0.3 ; sys_platform == 'linux' \ - --hash=sha256:0c580952eef9bf68c4747774cde7ec1d85a6e61de97281f2dba83c7d2c806362 \ - --hash=sha256:0f996e7268af62598f2fc1204afa98a3b5712313a55c4c9d434aef49cadc91d4 \ - --hash=sha256:1ec0bcf7e17c0c5669d881b1cd38c4972fade441b27bda1051665faaa89bdcaa \ - --hash=sha256:281c945d0e28c92ca5e5930664c1cefd85efe80e5c0d2bc58dd63383fda29f83 \ - --hash=sha256:2ce6fae5bdad59577b44e4dfed356944fbf1d925269114c28be377692643b4ff \ - --hash=sha256:315b9001266a492a6ff443b61238f956b214dbec9910a081ba5b6646a055a805 \ - --hash=sha256:443c4a81bb10daed9a8f334365fe52542771f25aedaf889fd323a853ce7377d6 \ - --hash=sha256:4a02ded6cd4f0a5562a8887df8b3bd14e822a90f97ac5e544c162899bc467664 \ - --hash=sha256:53a583b6637ab4c4e3591a15bc9db855b8d9dee9a669b550f311480acab6eb08 \ - --hash=sha256:63efa177ff54aec6e1c0aefaa1a241232dcd37413835a9b674b6e3f0ae2bfd3e \ - --hash=sha256:74f57f24754fe349223792466a709f8e0c093205ff0dca557af51072ff47ab18 \ - --hash=sha256:7e1ce50266f4f70bf41a2c6dc4358afadae90e2a1e5342d3c08883df1675374f \ - --hash=sha256:81ef806b1fef6b06dcebad789f988d3b37ccaee225695cf3e07648eee0fc6b73 \ - --hash=sha256:846da004a5804145a5f441b8530b4bf35afbf7da70f82409f151695b127213d5 \ - --hash=sha256:8ac43ae87929a5982f5948ceda07001ee5e83227fd69cf55b109144938d96984 \ - --hash=sha256:9762ea51a8fc2a88b70cf2995e5675b38d93bf36bd67d91721c309df184f49bd \ - --hash=sha256:a2a431ee15799d6db9fe80c82b055bae5a752bef645bba795e8e52687c69efe3 \ - --hash=sha256:bf7a1932ac4176486eab36a19ed4c0492da5d97123f1406cf15e41b05e787d2e \ - --hash=sha256:c2e6fc39c4ab499049df3bdf567f768a723a5e8464816e8f009f121a5a9f4405 \ - --hash=sha256:cbeb489927bd7af4aa98d4b261af9a5bc025bd87f0e3547e11584be9e9427be2 \ - --hash=sha256:d03b5621a135bffecad2c73e9f4deb1a0f977b9a8ffe6f8e002bf6c9d07b918c \ - --hash=sha256:d56e96520b1020449bbace2b78b603442e7e378a9b3bd68de65c782db1507995 \ - --hash=sha256:df6b6c6d742395dd77a23ea3728ab62f98379eff8fb61be2744d4679ab678f73 \ - --hash=sha256:e1be4655c7ef6e1bbe6b5d0403526601323420bcf414598955968c9ef3eb7d16 \ - --hash=sha256:f18c716be16bc1fea8e95def49edf46b82fccaa88587a45f8dc0ff6ab5d8e0a7 \ - --hash=sha256:f46304d6f0c6ab8e52770addfa2fc41e6629495548862279641972b6215451cd \ - --hash=sha256:f7b178f11ed3664fd0e995a47ed2b5ff0a12d893e41dd0494f406d1cf555cab7 +cryptography==44.0.1 ; sys_platform == 'linux' \ + --hash=sha256:00918d859aa4e57db8299607086f793fa7813ae2ff5a4637e318a25ef82730f7 \ + --hash=sha256:1e8d181e90a777b63f3f0caa836844a1182f1f265687fac2115fcf245f5fbec3 \ + --hash=sha256:1f9a92144fa0c877117e9748c74501bea842f93d21ee00b0cf922846d9d0b183 \ + --hash=sha256:21377472ca4ada2906bc313168c9dc7b1d7ca417b63c1c3011d0c74b7de9ae69 \ + --hash=sha256:24979e9f2040c953a94bf3c6782e67795a4c260734e5264dceea65c8f4bae64a \ + --hash=sha256:2a46a89ad3e6176223b632056f321bc7de36b9f9b93b2cc1cccf935a3849dc62 \ + --hash=sha256:322eb03ecc62784536bc173f1483e76747aafeb69c8728df48537eb431cd1911 \ + --hash=sha256:436df4f203482f41aad60ed1813811ac4ab102765ecae7a2bbb1dbb66dcff5a7 \ + --hash=sha256:4f422e8c6a28cf8b7f883eb790695d6d45b0c385a2583073f3cec434cc705e1a \ + --hash=sha256:53f23339864b617a3dfc2b0ac8d5c432625c80014c25caac9082314e9de56f41 \ + --hash=sha256:5fed5cd6102bb4eb843e3315d2bf25fede494509bddadb81e03a859c1bc17b83 \ + --hash=sha256:610a83540765a8d8ce0f351ce42e26e53e1f774a6efb71eb1b41eb01d01c3d12 \ + --hash=sha256:6c8acf6f3d1f47acb2248ec3ea261171a671f3d9428e34ad0357148d492c7864 \ + --hash=sha256:6f76fdd6fd048576a04c5210d53aa04ca34d2ed63336d4abd306d0cbe298fddf \ + --hash=sha256:72198e2b5925155497a5a3e8c216c7fb3e64c16ccee11f0e7da272fa93b35c4c \ + --hash=sha256:887143b9ff6bad2b7570da75a7fe8bbf5f65276365ac259a5d2d5147a73775f2 \ + --hash=sha256:888fcc3fce0c888785a4876ca55f9f43787f4c5c1cc1e2e0da71ad481ff82c5b \ + --hash=sha256:8e6a85a93d0642bd774460a86513c5d9d80b5c002ca9693e63f6e540f1815ed0 \ + --hash=sha256:94f99f2b943b354a5b6307d7e8d19f5c423a794462bde2bf310c770ba052b1c4 \ + --hash=sha256:9b336599e2cb77b1008cb2ac264b290803ec5e8e89d618a5e978ff5eb6f715d9 \ + --hash=sha256:a2d8a7045e1ab9b9f803f0d9531ead85f90c5f2859e653b61497228b18452008 \ + --hash=sha256:b8272f257cf1cbd3f2e120f14c68bff2b6bdfcc157fafdee84a1b795efd72862 \ + --hash=sha256:bf688f615c29bfe9dfc44312ca470989279f0e94bb9f631f85e3459af8efc009 \ + --hash=sha256:d9c5b9f698a83c8bd71e0f4d3f9f839ef244798e5ffe96febfa9714717db7af7 \ + --hash=sha256:dd7c7e2d71d908dc0f8d2027e1604102140d84b155e658c20e8ad1304317691f \ + --hash=sha256:df978682c1504fc93b3209de21aeabf2375cb1571d4e61907b3e7a2540e83026 \ + --hash=sha256:e403f7f766ded778ecdb790da786b418a9f2394f36e8cc8b796cc056ab05f44f \ + --hash=sha256:eb3889330f2a4a148abead555399ec9a32b13b7c8ba969b72d8e500eb7ef84cd \ + --hash=sha256:f4daefc971c2d1f82f03097dc6f216744a6cd2ac0f04c68fb935ea2ba2a0d420 \ + --hash=sha256:f51f5705ab27898afda1aaa430f34ad90dc117421057782022edf0600bec5f14 \ + --hash=sha256:fd0ee90072861e276b0ff08bd627abec29e32a53b2be44e41dbcdf87cbee2b00 # via secretstorage docutils==0.21.2 \ --hash=sha256:3a6b18732edf182daa3cd12775bbb338cf5691468f91eeeb109deff6ebfa986f \ From 34e433b75373aa9ad5645f370a0e0a4025e328da Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Tue, 8 Apr 2025 22:43:06 -0700 Subject: [PATCH 157/922] feat(toolchains): create toolchains from locally installed python (#2742) This adds docs and public APIs for using a locally installed python for a toolchain. Work towards https://github.com/bazel-contrib/rules_python/issues/2070 --------- Co-authored-by: Ignas Anikevicius <240938+aignas@users.noreply.github.com> --- CHANGELOG.md | 4 + docs/BUILD.bazel | 1 + docs/toolchains.md | 97 ++++++++++++++++++- python/BUILD.bazel | 1 + python/local_toolchains/BUILD.bazel | 18 ++++ python/local_toolchains/repos.bzl | 18 ++++ python/private/BUILD.bazel | 18 ++++ .../integration/local_toolchains/MODULE.bazel | 4 +- 8 files changed, 155 insertions(+), 6 deletions(-) create mode 100644 python/local_toolchains/BUILD.bazel create mode 100644 python/local_toolchains/repos.bzl diff --git a/CHANGELOG.md b/CHANGELOG.md index abe718c389..7aeb135788 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -116,6 +116,10 @@ Unreleased changes template. allow specifying links to create within the venv site packages (only applicable with {obj}`--bootstrap_impl=script`) ([#2156](https://github.com/bazelbuild/rules_python/issues/2156)). +* (toolchains) Local Python installs can be used to create a toolchain + equivalent to the standard toolchains. See [Local toolchains] docs for how to + configure them. + {#v0-0-0-removed} ### Removed diff --git a/docs/BUILD.bazel b/docs/BUILD.bazel index 29eac6e714..25da682012 100644 --- a/docs/BUILD.bazel +++ b/docs/BUILD.bazel @@ -108,6 +108,7 @@ sphinx_stardocs( "//python/cc:py_cc_toolchain_bzl", "//python/cc:py_cc_toolchain_info_bzl", "//python/entry_points:py_console_script_binary_bzl", + "//python/local_toolchains:repos_bzl", "//python/private:attr_builders_bzl", "//python/private:builders_util_bzl", "//python/private:py_binary_rule_bzl", diff --git a/docs/toolchains.md b/docs/toolchains.md index 73a8a48121..5cd9eb268e 100644 --- a/docs/toolchains.md +++ b/docs/toolchains.md @@ -199,10 +199,10 @@ Remember to call `use_repo()` to make repos visible to your module: :::{deprecated} 1.1.0 -The toolchain specific `py_binary` and `py_test` symbols are aliases to the regular rules. +The toolchain specific `py_binary` and `py_test` symbols are aliases to the regular rules. i.e. Deprecated `load("@python_versions//3.11:defs.bzl", "py_binary")` & `load("@python_versions//3.11:defs.bzl", "py_test")` -Usages of them should be changed to load the regular rules directly; +Usages of them should be changed to load the regular rules directly; i.e. Use `load("@rules_python//python:py_binary.bzl", "py_binary")` & `load("@rules_python//python:py_test.bzl", "py_test")` and then specify the `python_version` when using the rules corresponding to the python version you defined in your toolchain. {ref}`Library modules with version constraints` ::: @@ -327,7 +327,97 @@ After registration, your Python targets will use the toolchain's interpreter dur is still used to 'bootstrap' Python targets (see https://github.com/bazel-contrib/rules_python/issues/691). You may also find some quirks while using this toolchain. Please refer to [python-build-standalone documentation's _Quirks_ section](https://gregoryszorc.com/docs/python-build-standalone/main/quirks.html). -## Autodetecting toolchain +## Local toolchain + +It's possible to use a locally installed Python runtime instead of the regular +prebuilt, remotely downloaded ones. A local toolchain contains the Python +runtime metadata (Python version, headers, ABI flags, etc) that the regular +remotely downloaded runtimes contain, which makes it possible to build e.g. C +extensions (unlike the autodetecting and runtime environment toolchains). + +For simple cases, some rules are provided that will introspect +a Python installation and create an appropriate Bazel definition from +it. To do this, three pieces need to be wired together: + +1. Specify a path or command to a Python interpreter (multiple can be defined). +2. Create toolchains for the runtimes in (1) +3. Register the toolchains created by (2) + +The below is an example that will use `python3` from PATH to find the +interpreter, then introspect its installation to generate a full toolchain. + +```starlark +# File: MODULE.bazel + +local_runtime_repo = use_repo_rule( + "@rules_python//python/local_toolchains:repos.bzl", + "local_runtime_repo", + dev_dependency = True, +) + +local_runtime_toolchains_repo = use_repo_rule( + "@rules_python//python/local_toolchains:repos.bzl" + "local_runtime_toolchains_repo" + dev_dependency = True, +) + +# Step 1: Define the Python runtime +local_runtime_repo( + name = "local_python3", + interpreter_path = "python3", + on_failure = "fail", +) + +# Step 2: Create toolchains for the runtimes +local_runtime_toolchains_repo( + name = "local_toolchains", + runtimes = ["local_python3"], +) + +# Step 3: Register the toolchains +register_toolchains("@local_toolchains//:all", dev_dependency = True) +``` + +Note that `register_toolchains` will insert the local toolchain earlier in the +toolchain ordering, so it will take precedence over other registered toolchains. + +:::{important} +Be sure to set `dev_dependency = True`. Using a local toolchain only makes sense +for the root module. + +If an intermediate module does it, then the `register_toolchains()` call will +take precedence over the default rules_python toolchains and cause problems for +downstream modules. +::: + +Multiple runtimes and/or toolchains can be defined, which allows for multiple +Python versions and/or platforms to be configured in a single `MODULE.bazel`. + +## Runtime environment toolchain + +The runtime environment toolchain is a minimal toolchain that doesn't provide +information about Python at build time. In particular, this means it is not able +to build C extensions -- doing so requires knowing, at build time, what Python +headers to use. + +In effect, all it does is generate a small wrapper script that simply calls e.g. +`/usr/bin/env python3` to run a program. This makes it easy to change what +Python is used to run a program, but also makes it easy to use a Python version +that isn't compatible with build-time assumptions. + +``` +register_toolchains("@rules_python//python/runtime_env_toolchains:all") +``` + +Note that this toolchain has no constraints, i.e. it will match any platform, +Python version, etc. + +:::{seealso} +[Local toolchain], which creates a more full featured toolchain from a +locally installed Python. +::: + +### Autodetecting toolchain The autodetecting toolchain is a deprecated toolchain that is built into Bazel. It's name is a bit misleading: it doesn't autodetect anything. All it does is @@ -345,7 +435,6 @@ To aid migration off the Bazel-builtin toolchain, rules_python provides {bzl:obj}`@rules_python//python/runtime_env_toolchains:all`. This is an equivalent toolchain, but is implemented using rules_python's objects. - ## Custom toolchains While rules_python provides toolchains by default, it is not required to use diff --git a/python/BUILD.bazel b/python/BUILD.bazel index a699c81cc4..3389a0dacc 100644 --- a/python/BUILD.bazel +++ b/python/BUILD.bazel @@ -41,6 +41,7 @@ filegroup( "//python/constraints:distribution", "//python/entry_points:distribution", "//python/extensions:distribution", + "//python/local_toolchains:distribution", "//python/pip_install:distribution", "//python/private:distribution", "//python/runfiles:distribution", diff --git a/python/local_toolchains/BUILD.bazel b/python/local_toolchains/BUILD.bazel new file mode 100644 index 0000000000..211f3e21a7 --- /dev/null +++ b/python/local_toolchains/BUILD.bazel @@ -0,0 +1,18 @@ +load("@bazel_skylib//:bzl_library.bzl", "bzl_library") + +package(default_visibility = ["//:__subpackages__"]) + +bzl_library( + name = "repos_bzl", + srcs = ["repos.bzl"], + visibility = ["//visibility:public"], + deps = [ + "//python/private:local_runtime_repo_bzl", + "//python/private:local_runtime_toolchains_repo_bzl", + ], +) + +filegroup( + name = "distribution", + srcs = glob(["**"]), +) diff --git a/python/local_toolchains/repos.bzl b/python/local_toolchains/repos.bzl new file mode 100644 index 0000000000..d1b45cfd7f --- /dev/null +++ b/python/local_toolchains/repos.bzl @@ -0,0 +1,18 @@ +"""Rules/macros for repository phase for local toolchains. + +:::{versionadded} VERSION_NEXT_FEATURE +::: +""" + +load( + "@rules_python//python/private:local_runtime_repo.bzl", + _local_runtime_repo = "local_runtime_repo", +) +load( + "@rules_python//python/private:local_runtime_toolchains_repo.bzl", + _local_runtime_toolchains_repo = "local_runtime_toolchains_repo", +) + +local_runtime_repo = _local_runtime_repo + +local_runtime_toolchains_repo = _local_runtime_toolchains_repo diff --git a/python/private/BUILD.bazel b/python/private/BUILD.bazel index ef4580e1ce..b63f446be3 100644 --- a/python/private/BUILD.bazel +++ b/python/private/BUILD.bazel @@ -205,6 +205,24 @@ bzl_library( ], ) +bzl_library( + name = "local_runtime_repo_bzl", + srcs = ["local_runtime_repo.bzl"], + deps = [ + ":enum_bzl", + ":repo_utils.bzl", + ], +) + +bzl_library( + name = "local_runtime_toolchains_repo_bzl", + srcs = ["local_runtime_toolchains_repo.bzl"], + deps = [ + ":repo_utils.bzl", + ":text_util_bzl", + ], +) + bzl_library( name = "normalize_name_bzl", srcs = ["normalize_name.bzl"], diff --git a/tests/integration/local_toolchains/MODULE.bazel b/tests/integration/local_toolchains/MODULE.bazel index d4ef12e952..98f1ed9ac4 100644 --- a/tests/integration/local_toolchains/MODULE.bazel +++ b/tests/integration/local_toolchains/MODULE.bazel @@ -19,9 +19,9 @@ local_path_override( path = "../../..", ) -local_runtime_repo = use_repo_rule("@rules_python//python/private:local_runtime_repo.bzl", "local_runtime_repo") +local_runtime_repo = use_repo_rule("@rules_python//python/local_toolchains:repos.bzl", "local_runtime_repo") -local_runtime_toolchains_repo = use_repo_rule("@rules_python//python/private:local_runtime_toolchains_repo.bzl", "local_runtime_toolchains_repo") +local_runtime_toolchains_repo = use_repo_rule("@rules_python//python/local_toolchains:repos.bzl", "local_runtime_toolchains_repo") local_runtime_repo( name = "local_python3", From 9fb13ec1af33ecc9da8beb7dcea7bb25b4dbc241 Mon Sep 17 00:00:00 2001 From: Matt Mackay Date: Wed, 9 Apr 2025 08:37:57 -0400 Subject: [PATCH 158/922] fix: run python version call in isolated mode (#2761) Similar to https://github.com/bazel-contrib/rules_python/pull/2738, runs the call to get the Python interpreter version in isolated mode via `-I`, ensuring userland Python variables do not affect this call. --- CHANGELOG.md | 1 + python/private/pypi/whl_library.bzl | 4 ++++ 2 files changed, 5 insertions(+) diff --git a/CHANGELOG.md b/CHANGELOG.md index 7aeb135788..f38732f7d8 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -91,6 +91,7 @@ Unreleased changes template. Fixes [#2685](https://github.com/bazel-contrib/rules_python/issues/2685). * (toolchains) Run the check on the Python interpreter in isolated mode, to ensure it's not affected by userland environment variables, such as `PYTHONPATH`. * (toolchains) Ensure temporary `.pyc` and `.pyo` files are also excluded from the interpreters repository files. +* (pypi) Run interpreter version call in isolated mode, to ensure it's not affected by userland environment variables, such as `PYTHONPATH`. {#v0-0-0-added} ### Added diff --git a/python/private/pypi/whl_library.bzl b/python/private/pypi/whl_library.bzl index 2904f85f1b..493f11353e 100644 --- a/python/private/pypi/whl_library.bzl +++ b/python/private/pypi/whl_library.bzl @@ -109,6 +109,10 @@ def _get_toolchain_unix_cflags(rctx, python_interpreter, logger = None): op = "GetPythonVersionForUnixCflags", python = python_interpreter, arguments = [ + # Run the interpreter in isolated mode, this options implies -E, -P and -s. + # Ensures environment variables are ignored that are set in userspace, such as PYTHONPATH, + # which may interfere with this invocation. + "-I", "-c", "import sys; print(f'{sys.version_info[0]}.{sys.version_info[1]}', end='')", ], From 55d68369e37da847ee8ac2be0358ef4969f1b194 Mon Sep 17 00:00:00 2001 From: Ignas Anikevicius <240938+aignas@users.noreply.github.com> Date: Fri, 11 Apr 2025 03:43:17 +0900 Subject: [PATCH 159/922] fix(pypi): fixes to the marker evaluation and utils (#2767) These are just bugfixes to already merged code: * Fix nested bracket parsing in PEP508 marker parser. * Fix the sys_platform constants, which I noticed in #2629 but they got also pointed out in #2766. * Port some of python tests for requirement parsing and improve the implementation. Those tests will be removed in #2629. * Move the platform related code to a separate file. * Rename `pep508_req.bzl` to `pep508_requirement.bzl` to follow the convention. All of the bug fixes have added tests. Work towards #2423. --- python/private/pypi/BUILD.bazel | 15 ++++- python/private/pypi/evaluate_markers.bzl | 9 +-- python/private/pypi/pep508_env.bzl | 63 ++++++++++--------- python/private/pypi/pep508_platform.bzl | 57 +++++++++++++++++ ...{pep508_req.bzl => pep508_requirement.bzl} | 9 ++- tests/pypi/pep508/BUILD.bazel | 5 ++ tests/pypi/pep508/requirement_tests.bzl | 47 ++++++++++++++ 7 files changed, 165 insertions(+), 40 deletions(-) create mode 100644 python/private/pypi/pep508_platform.bzl rename python/private/pypi/{pep508_req.bzl => pep508_requirement.bzl} (82%) create mode 100644 tests/pypi/pep508/requirement_tests.bzl diff --git a/python/private/pypi/BUILD.bazel b/python/private/pypi/BUILD.bazel index 21e05f2895..e0a2f20c14 100644 --- a/python/private/pypi/BUILD.bazel +++ b/python/private/pypi/BUILD.bazel @@ -77,7 +77,8 @@ bzl_library( deps = [ ":pep508_env_bzl", ":pep508_evaluate_bzl", - ":pep508_req_bzl", + ":pep508_platform_bzl", + ":pep508_requirement_bzl", ], ) @@ -223,6 +224,9 @@ bzl_library( bzl_library( name = "pep508_env_bzl", srcs = ["pep508_env.bzl"], + deps = [ + ":pep508_platform_bzl", + ], ) bzl_library( @@ -235,8 +239,13 @@ bzl_library( ) bzl_library( - name = "pep508_req_bzl", - srcs = ["pep508_req.bzl"], + name = "pep508_platform_bzl", + srcs = ["pep508_platform.bzl"], +) + +bzl_library( + name = "pep508_requirement_bzl", + srcs = ["pep508_requirement.bzl"], deps = [ "//python/private:normalize_name_bzl", ], diff --git a/python/private/pypi/evaluate_markers.bzl b/python/private/pypi/evaluate_markers.bzl index 1d4c30753f..a0223abdc8 100644 --- a/python/private/pypi/evaluate_markers.bzl +++ b/python/private/pypi/evaluate_markers.bzl @@ -14,9 +14,10 @@ """A simple function that evaluates markers using a python interpreter.""" -load(":pep508_env.bzl", "env", _platform_from_str = "platform_from_str") +load(":pep508_env.bzl", "env") load(":pep508_evaluate.bzl", "evaluate") -load(":pep508_req.bzl", _req = "requirement") +load(":pep508_platform.bzl", "platform_from_str") +load(":pep508_requirement.bzl", "requirement") def evaluate_markers(requirements): """Return the list of supported platforms per requirements line. @@ -29,9 +30,9 @@ def evaluate_markers(requirements): """ ret = {} for req_string, platforms in requirements.items(): - req = _req(req_string) + req = requirement(req_string) for platform in platforms: - if evaluate(req.marker, env = env(_platform_from_str(platform, None))): + if evaluate(req.marker, env = env(platform_from_str(platform, None))): ret.setdefault(req_string, []).append(platform) return ret diff --git a/python/private/pypi/pep508_env.bzl b/python/private/pypi/pep508_env.bzl index 17d41871d1..265a8e9b99 100644 --- a/python/private/pypi/pep508_env.bzl +++ b/python/private/pypi/pep508_env.bzl @@ -15,7 +15,9 @@ """This module is for implementing PEP508 environment definition. """ -# See https://stackoverflow.com/questions/45125516/possible-values-for-uname-m +load(":pep508_platform.bzl", "platform_from_str") + +# See https://stackoverflow.com/a/45125525 _platform_machine_aliases = { # These pairs mean the same hardware, but different values may be used # on different host platforms. @@ -24,13 +26,41 @@ _platform_machine_aliases = { "i386": "x86_32", "i686": "x86_32", } + +# Platform system returns results from the `uname` call. _platform_system_values = { "linux": "Linux", "osx": "Darwin", "windows": "Windows", } + +# The copy of SO [answer](https://stackoverflow.com/a/13874620) containing +# all of the platforms: +# ┍━━━━━━━━━━━━━━━━━━━━━┯━━━━━━━━━━━━━━━━━━━━━┑ +# │ System │ Value │ +# ┝━━━━━━━━━━━━━━━━━━━━━┿━━━━━━━━━━━━━━━━━━━━━┥ +# │ Linux │ linux or linux2 (*) │ +# │ Windows │ win32 │ +# │ Windows/Cygwin │ cygwin │ +# │ Windows/MSYS2 │ msys │ +# │ Mac OS X │ darwin │ +# │ OS/2 │ os2 │ +# │ OS/2 EMX │ os2emx │ +# │ RiscOS │ riscos │ +# │ AtheOS │ atheos │ +# │ FreeBSD 7 │ freebsd7 │ +# │ FreeBSD 8 │ freebsd8 │ +# │ FreeBSD N │ freebsdN │ +# │ OpenBSD 6 │ openbsd6 │ +# │ AIX │ aix (**) │ +# ┕━━━━━━━━━━━━━━━━━━━━━┷━━━━━━━━━━━━━━━━━━━━━┙ +# +# (*) Prior to Python 3.3, the value for any Linux version is always linux2; after, it is linux. +# (**) Prior Python 3.8 could also be aix5 or aix7; use sys.platform.startswith() +# +# We are using only the subset that we actually support. _sys_platform_values = { - "linux": "posix", + "linux": "linux", "osx": "darwin", "windows": "win32", } @@ -61,6 +91,7 @@ def env(target_platform, *, extra = None): "platform_release": "", "platform_version": "", } + if type(target_platform) == type(""): target_platform = platform_from_str(target_platform, python_version = "") @@ -87,31 +118,3 @@ def env(target_platform, *, extra = None): "platform_machine": _platform_machine_aliases, }, } - -def _platform(*, abi = None, os = None, arch = None): - return struct( - abi = abi, - os = os, - arch = arch, - ) - -def platform_from_str(p, python_version): - """Return a platform from a string. - - Args: - p: {type}`str` the actual string. - python_version: {type}`str` the python version to add to platform if needed. - - Returns: - A struct that is returned by the `_platform` function. - """ - if p.startswith("cp"): - abi, _, p = p.partition("_") - elif python_version: - major, _, tail = python_version.partition(".") - abi = "cp{}{}".format(major, tail) - else: - abi = None - - os, _, arch = p.partition("_") - return _platform(abi = abi, os = os or None, arch = arch or None) diff --git a/python/private/pypi/pep508_platform.bzl b/python/private/pypi/pep508_platform.bzl new file mode 100644 index 0000000000..381a8d7a08 --- /dev/null +++ b/python/private/pypi/pep508_platform.bzl @@ -0,0 +1,57 @@ +# Copyright 2025 The Bazel Authors. All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +"""The platform abstraction +""" + +def platform(*, abi = None, os = None, arch = None): + """platform returns a struct for the platform. + + Args: + abi: {type}`str | None` the target ABI, e.g. `"cp39"`. + os: {type}`str | None` the target os, e.g. `"linux"`. + arch: {type}`str | None` the target CPU, e.g. `"aarch64"`. + + Returns: + A struct. + """ + + # Note, this is used a lot as a key in dictionaries, so it cannot contain + # methods. + return struct( + abi = abi, + os = os, + arch = arch, + ) + +def platform_from_str(p, python_version): + """Return a platform from a string. + + Args: + p: {type}`str` the actual string. + python_version: {type}`str` the python version to add to platform if needed. + + Returns: + A struct that is returned by the `_platform` function. + """ + if p.startswith("cp"): + abi, _, p = p.partition("_") + elif python_version: + major, _, tail = python_version.partition(".") + abi = "cp{}{}".format(major, tail) + else: + abi = None + + os, _, arch = p.partition("_") + return platform(abi = abi, os = os or None, arch = arch or None) diff --git a/python/private/pypi/pep508_req.bzl b/python/private/pypi/pep508_requirement.bzl similarity index 82% rename from python/private/pypi/pep508_req.bzl rename to python/private/pypi/pep508_requirement.bzl index 618ffaf17a..11f2b3e8fa 100644 --- a/python/private/pypi/pep508_req.bzl +++ b/python/private/pypi/pep508_requirement.bzl @@ -17,7 +17,7 @@ load("//python/private:normalize_name.bzl", "normalize_name") -_STRIP = ["(", " ", ">", "=", "<", "~", "!"] +_STRIP = ["(", " ", ">", "=", "<", "~", "!", "@"] def requirement(spec): """Parse a PEP508 requirement line @@ -28,15 +28,18 @@ def requirement(spec): Returns: A struct with the information. """ + spec = spec.strip() requires, _, maybe_hashes = spec.partition(";") marker, _, _ = maybe_hashes.partition("--hash") requires, _, extras_unparsed = requires.partition("[") + extras_unparsed, _, _ = extras_unparsed.partition("]") for char in _STRIP: requires, _, _ = requires.partition(char) - extras = extras_unparsed.strip("]").split(",") + extras = extras_unparsed.replace(" ", "").split(",") + name = requires.strip(" ") return struct( - name = normalize_name(requires.strip(" ")), + name = normalize_name(name).replace("_", "-"), marker = marker.strip(" "), extras = extras, ) diff --git a/tests/pypi/pep508/BUILD.bazel b/tests/pypi/pep508/BUILD.bazel index b795db0591..575f28ada6 100644 --- a/tests/pypi/pep508/BUILD.bazel +++ b/tests/pypi/pep508/BUILD.bazel @@ -1,5 +1,10 @@ load(":evaluate_tests.bzl", "evaluate_test_suite") +load(":requirement_tests.bzl", "requirement_test_suite") evaluate_test_suite( name = "evaluate_tests", ) + +requirement_test_suite( + name = "requirement_tests", +) diff --git a/tests/pypi/pep508/requirement_tests.bzl b/tests/pypi/pep508/requirement_tests.bzl new file mode 100644 index 0000000000..7c81ea50fc --- /dev/null +++ b/tests/pypi/pep508/requirement_tests.bzl @@ -0,0 +1,47 @@ +# Copyright 2025 The Bazel Authors. All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +"""Tests for parsing the requirement specifier.""" + +load("@rules_testing//lib:test_suite.bzl", "test_suite") +load("//python/private/pypi:pep508_requirement.bzl", "requirement") # buildifier: disable=bzl-visibility + +_tests = [] + +def _test_requirement_line_parsing(env): + want = { + " name1[ foo ] ": ("name1", ["foo"]), + "Name[foo]": ("name", ["foo"]), + "name [fred,bar] @ http://foo.com ; python_version=='2.7'": ("name", ["fred", "bar"]), + "name; (os_name=='a' or os_name=='b') and os_name=='c'": ("name", [""]), + "name@http://foo.com": ("name", [""]), + "name[ Foo123 ]": ("name", ["Foo123"]), + "name[extra]@http://foo.com": ("name", ["extra"]), + "name[foo]": ("name", ["foo"]), + "name[quux, strange];python_version<'2.7' and platform_version=='2'": ("name", ["quux", "strange"]), + "name_foo[bar]": ("name-foo", ["bar"]), + } + + got = { + i: (parsed.name, parsed.extras) + for i, parsed in {case: requirement(case) for case in want}.items() + } + env.expect.that_dict(got).contains_exactly(want) + +_tests.append(_test_requirement_line_parsing) + +def requirement_test_suite(name): # buildifier: disable=function-docstring + test_suite( + name = name, + basic_tests = _tests, + ) From 6e2d493f3e8e12c7cf208a4e9a398c5eabb65f24 Mon Sep 17 00:00:00 2001 From: asa <96153+asa@users.noreply.github.com> Date: Thu, 10 Apr 2025 17:44:56 -0700 Subject: [PATCH 160/922] fix: Prevent absolute path creation in uv lock template (#2769) This change fixes a bug in the `lock` rule where, when the package is at the root level, the path to `requirements.txt` is constructed incorrectly with a leading double slash (`//requirements.txt`), causing it to be interpreted as an absolute path. This change detects if the package is empty before constructing the output path. Work towards #1975 --------- Co-authored-by: Ignas Anikevicius <240938+aignas@users.noreply.github.com> --- python/uv/private/lock.bzl | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/python/uv/private/lock.bzl b/python/uv/private/lock.bzl index 45a3819ee6..2731d6b009 100644 --- a/python/uv/private/lock.bzl +++ b/python/uv/private/lock.bzl @@ -327,10 +327,15 @@ def _maybe_file(path): def _expand_template_impl(ctx): pkg = ctx.label.package update_src = ctx.actions.declare_file(ctx.attr.update_target + ".py") + + # Fix the path construction to avoid absolute paths + # If package is empty (root), don't add a leading slash + dst = "{}/{}".format(pkg, ctx.attr.output) if pkg else ctx.attr.output + ctx.actions.expand_template( template = ctx.files._template[0], substitutions = { - "{{dst}}": "{}/{}".format(pkg, ctx.attr.output), + "{{dst}}": dst, "{{src}}": "{}".format(ctx.files.src[0].short_path), "{{update_target}}": "//{}:{}".format(pkg, ctx.attr.update_target), }, From 84351d4ec14e474bc196c0b8cd70e04fcc9a25ca Mon Sep 17 00:00:00 2001 From: "Elvis M. Wianda" <7077790+ewianda@users.noreply.github.com> Date: Fri, 11 Apr 2025 17:18:46 -0600 Subject: [PATCH 161/922] fix: Resolve incorrect platform specific dependency (#2766) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit This change addresses a bug where `pip.parse` selects the wrong requirement entry when multiple extras are listed with platform-specific markers. #### 🔍 Problem: In a `requirements.txt` generated by tools like `uv` or `poetry`, it's valid to have multiple entries for the same package, each with different extras and `sys_platform` markers, for example: ```ini optimum[onnxruntime]==1.17.1 ; sys_platform == 'darwin' optimum[onnxruntime-gpu]==1.17.1 ; sys_platform == 'linux' ``` The current implementation in [`[parse_requirements.bzl](https://github.com/bazel-contrib/rules_python/blob/032f6aa738a673b13b605dabf55465c6fc1a56eb/python/private/pypi/parse_requirements.bzl#L114-L126)`](https://github.com/bazel-contrib/rules_python/blob/032f6aa738a673b13b605dabf55465c6fc1a56eb/python/private/pypi/parse_requirements.bzl#L114-L126) uses a sort-by-length heuristic to select the “best” requirement when there are multiple entries with the same base name. This works well in legacy `requirements.txt` files where: ``` my_dep my_dep[foo] my_dep[foo,bar] ``` ...would indicate an intent to select the **most specific subset of extras** (i.e. the longest name). However, this heuristic **breaks** in the presence of **platform markers**, where extras are **not subsets**, but distinct variants. In the example above, Bazel mistakenly selects `optimum[onnxruntime-gpu]` on macOS because it's a longer match, even though it is guarded by a Linux-only marker. #### ✅ Fix: This PR modifies the behavior to: 1. **Add the requirement marker** as part of the sorting key. 2. **Then apply the longest-match logic** to drop duplicate requirements with different extras but the same markers. This ensures that only applicable requirements are considered during resolution, preserving correctness in multi-platform environments. #### 🧪 Before: On macOS, the following entry is incorrectly selected: ``` optimum[onnxruntime-gpu]==1.17.1 ; sys_platform == 'linux' ``` #### ✅ After: Correct entry is selected: ``` optimum[onnxruntime]==1.17.1 ; sys_platform == 'darwin' ``` close https://github.com/bazel-contrib/rules_python/issues/2690 --------- Co-authored-by: Ignas Anikevicius <240938+aignas@users.noreply.github.com> --- CHANGELOG.md | 2 + python/private/pypi/parse_requirements.bzl | 44 +++++------- python/private/pypi/pep508_requirement.bzl | 11 +++ tests/pypi/extension/extension_tests.bzl | 78 ++++++++++++++++++++++ tests/pypi/pep508/requirement_tests.bzl | 23 ++++--- 5 files changed, 119 insertions(+), 39 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index f38732f7d8..7d9b648bea 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -81,6 +81,8 @@ Unreleased changes template. {#v0-0-0-fixed} ### Fixed +* (pypi) Platform specific extras are now correctly handled when using + universal lock files with environment markers. Fixes [#2690](https://github.com/bazel-contrib/rules_python/pull/2690). * (runfiles) ({obj}`--bootstrap_impl=script`) Follow symlinks when searching for runfiles. * (toolchains) Do not try to run `chmod` when downloading non-windows hermetic toolchain repositories on Windows. Fixes diff --git a/python/private/pypi/parse_requirements.bzl b/python/private/pypi/parse_requirements.bzl index d2014a7eb9..1cbf094f5c 100644 --- a/python/private/pypi/parse_requirements.bzl +++ b/python/private/pypi/parse_requirements.bzl @@ -30,22 +30,9 @@ load("//python/private:normalize_name.bzl", "normalize_name") load("//python/private:repo_utils.bzl", "repo_utils") load(":index_sources.bzl", "index_sources") load(":parse_requirements_txt.bzl", "parse_requirements_txt") +load(":pep508_requirement.bzl", "requirement") load(":whl_target_platforms.bzl", "select_whls") -def _extract_version(entry): - """Extract the version part from the requirement string. - - - Args: - entry: {type}`str` The requirement string. - """ - version_start = entry.find("==") - if version_start != -1: - # Extract everything after '==' until the next space or end of the string - version, _, _ = entry[version_start + 2:].partition(" ") - return version - return None - def parse_requirements( ctx, *, @@ -111,19 +98,20 @@ def parse_requirements( # The requirement lines might have duplicate names because lines for extras # are returned as just the base package name. e.g., `foo[bar]` results # in an entry like `("foo", "foo[bar] == 1.0 ...")`. - requirements_dict = { - (normalize_name(entry[0]), _extract_version(entry[1])): entry - for entry in sorted( - parse_result.requirements, - # Get the longest match and fallback to original WORKSPACE sorting, - # which should get us the entry with most extras. - # - # FIXME @aignas 2024-05-13: The correct behaviour might be to get an - # entry with all aggregated extras, but it is unclear if we - # should do this now. - key = lambda x: (len(x[1].partition("==")[0]), x), - ) - }.values() + # Lines with different markers are not condidered duplicates. + requirements_dict = {} + for entry in sorted( + parse_result.requirements, + # Get the longest match and fallback to original WORKSPACE sorting, + # which should get us the entry with most extras. + # + # FIXME @aignas 2024-05-13: The correct behaviour might be to get an + # entry with all aggregated extras, but it is unclear if we + # should do this now. + key = lambda x: (len(x[1].partition("==")[0]), x), + ): + req = requirement(entry[1]) + requirements_dict[(req.name, req.version, req.marker)] = entry tokenized_options = [] for opt in parse_result.options: @@ -132,7 +120,7 @@ def parse_requirements( pip_args = tokenized_options + extra_pip_args for plat in plats: - requirements[plat] = requirements_dict + requirements[plat] = requirements_dict.values() options[plat] = pip_args requirements_by_platform = {} diff --git a/python/private/pypi/pep508_requirement.bzl b/python/private/pypi/pep508_requirement.bzl index 11f2b3e8fa..ee7b5dfc35 100644 --- a/python/private/pypi/pep508_requirement.bzl +++ b/python/private/pypi/pep508_requirement.bzl @@ -30,6 +30,16 @@ def requirement(spec): """ spec = spec.strip() requires, _, maybe_hashes = spec.partition(";") + + version_start = requires.find("==") + version = None + if version_start != -1: + # Extract everything after '==' until the next space or end of the string + version, _, _ = requires[version_start + 2:].partition(" ") + + # Remove any trailing characters from the version string + version = version.strip(" ") + marker, _, _ = maybe_hashes.partition("--hash") requires, _, extras_unparsed = requires.partition("[") extras_unparsed, _, _ = extras_unparsed.partition("]") @@ -42,4 +52,5 @@ def requirement(spec): name = normalize_name(name).replace("_", "-"), marker = marker.strip(" "), extras = extras, + version = version, ) diff --git a/tests/pypi/extension/extension_tests.bzl b/tests/pypi/extension/extension_tests.bzl index 1652e76156..66c9e0549e 100644 --- a/tests/pypi/extension/extension_tests.bzl +++ b/tests/pypi/extension/extension_tests.bzl @@ -856,6 +856,84 @@ git_dep @ git+https://git.server/repo/project@deadbeefdeadbeef _tests.append(_test_simple_get_index) +def _test_optimum_sys_platform_extra(env): + pypi = _parse_modules( + env, + module_ctx = _mock_mctx( + _mod( + name = "rules_python", + parse = [ + _parse( + hub_name = "pypi", + python_version = "3.15", + requirements_lock = "universal.txt", + ), + ], + ), + read = lambda x: { + "universal.txt": """\ +optimum[onnxruntime]==1.17.1 ; sys_platform == 'darwin' +optimum[onnxruntime-gpu]==1.17.1 ; sys_platform == 'linux' +""", + }[x], + ), + available_interpreters = { + "python_3_15_host": "unit_test_interpreter_target", + }, + ) + + pypi.exposed_packages().contains_exactly({"pypi": []}) + pypi.hub_group_map().contains_exactly({"pypi": {}}) + pypi.hub_whl_map().contains_exactly({ + "pypi": { + "optimum": { + "pypi_315_optimum_linux_aarch64_linux_arm_linux_ppc_linux_s390x_linux_x86_64": [ + whl_config_setting( + version = "3.15", + target_platforms = [ + "cp315_linux_aarch64", + "cp315_linux_arm", + "cp315_linux_ppc", + "cp315_linux_s390x", + "cp315_linux_x86_64", + ], + config_setting = None, + filename = None, + ), + ], + "pypi_315_optimum_osx_aarch64_osx_x86_64": [ + whl_config_setting( + version = "3.15", + target_platforms = [ + "cp315_osx_aarch64", + "cp315_osx_x86_64", + ], + config_setting = None, + filename = None, + ), + ], + }, + }, + }) + + pypi.whl_libraries().contains_exactly({ + "pypi_315_optimum_linux_aarch64_linux_arm_linux_ppc_linux_s390x_linux_x86_64": { + "dep_template": "@pypi//{name}:{target}", + "python_interpreter_target": "unit_test_interpreter_target", + "repo": "pypi_315", + "requirement": "optimum[onnxruntime-gpu]==1.17.1", + }, + "pypi_315_optimum_osx_aarch64_osx_x86_64": { + "dep_template": "@pypi//{name}:{target}", + "python_interpreter_target": "unit_test_interpreter_target", + "repo": "pypi_315", + "requirement": "optimum[onnxruntime]==1.17.1", + }, + }) + pypi.whl_mods().contains_exactly({}) + +_tests.append(_test_optimum_sys_platform_extra) + def extension_test_suite(name): """Create the test suite. diff --git a/tests/pypi/pep508/requirement_tests.bzl b/tests/pypi/pep508/requirement_tests.bzl index 7c81ea50fc..9afb43a437 100644 --- a/tests/pypi/pep508/requirement_tests.bzl +++ b/tests/pypi/pep508/requirement_tests.bzl @@ -20,20 +20,21 @@ _tests = [] def _test_requirement_line_parsing(env): want = { - " name1[ foo ] ": ("name1", ["foo"]), - "Name[foo]": ("name", ["foo"]), - "name [fred,bar] @ http://foo.com ; python_version=='2.7'": ("name", ["fred", "bar"]), - "name; (os_name=='a' or os_name=='b') and os_name=='c'": ("name", [""]), - "name@http://foo.com": ("name", [""]), - "name[ Foo123 ]": ("name", ["Foo123"]), - "name[extra]@http://foo.com": ("name", ["extra"]), - "name[foo]": ("name", ["foo"]), - "name[quux, strange];python_version<'2.7' and platform_version=='2'": ("name", ["quux", "strange"]), - "name_foo[bar]": ("name-foo", ["bar"]), + " name1[ foo ] ": ("name1", ["foo"], None, ""), + "Name[foo]": ("name", ["foo"], None, ""), + "name [fred,bar] @ http://foo.com ; python_version=='2.7'": ("name", ["fred", "bar"], None, "python_version=='2.7'"), + "name; (os_name=='a' or os_name=='b') and os_name=='c'": ("name", [""], None, "(os_name=='a' or os_name=='b') and os_name=='c'"), + "name@http://foo.com": ("name", [""], None, ""), + "name[ Foo123 ]": ("name", ["Foo123"], None, ""), + "name[extra]@http://foo.com": ("name", ["extra"], None, ""), + "name[foo]": ("name", ["foo"], None, ""), + "name[quux, strange];python_version<'2.7' and platform_version=='2'": ("name", ["quux", "strange"], None, "python_version<'2.7' and platform_version=='2'"), + "name_foo[bar]": ("name-foo", ["bar"], None, ""), + "name_foo[bar]==0.25": ("name-foo", ["bar"], "0.25", ""), } got = { - i: (parsed.name, parsed.extras) + i: (parsed.name, parsed.extras, parsed.version, parsed.marker) for i, parsed in {case: requirement(case) for case in want}.items() } env.expect.that_dict(got).contains_exactly(want) From aa0d16c1463e4e26f6ed633ae83d9785a2ea9dfa Mon Sep 17 00:00:00 2001 From: Ignas Anikevicius <240938+aignas@users.noreply.github.com> Date: Mon, 14 Apr 2025 07:10:51 +0900 Subject: [PATCH 162/922] fix(rules): make the srcs trully optional (#2768) With this PR we mark the srcs attribute as optional as we can leverage the `main_module` to just run things from the deps. This also removes a long-standing `TODO` note. Fixes #2765 --------- Co-authored-by: Richard Levasseur --- CHANGELOG.md | 2 + python/private/py_executable.bzl | 3 +- tests/base_rules/py_executable_base_tests.bzl | 72 ++++++++++++------- tests/support/support.bzl | 1 + 4 files changed, 53 insertions(+), 25 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 7d9b648bea..33d99dfaa1 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -76,6 +76,8 @@ Unreleased changes template. * (pypi) The PyPI extension will no longer write the lock file entries as the extension has been marked reproducible. Fixes [#2434](https://github.com/bazel-contrib/rules_python/issues/2434). +* (rules) {attr}`py_binary.srcs` and {attr}`py_test.srcs` is no longer mandatory when + `main_module` is specified (for `--bootstrap_impl=script`) [20250317]: https://github.com/astral-sh/python-build-standalone/releases/tag/20250317 diff --git a/python/private/py_executable.bzl b/python/private/py_executable.bzl index e6f4700b20..dd3ad869fa 100644 --- a/python/private/py_executable.bzl +++ b/python/private/py_executable.bzl @@ -786,6 +786,8 @@ def _create_stage1_bootstrap( ) template = runtime.bootstrap_template subs["%shebang%"] = runtime.stub_shebang + elif not ctx.files.srcs: + fail("mandatory 'srcs' files have not been provided") else: if (ctx.configuration.coverage_enabled and runtime and @@ -1888,7 +1890,6 @@ def create_executable_rule_builder(implementation, **kwargs): ), **kwargs ) - builder.attrs.get("srcs").set_mandatory(True) return builder def cc_configure_features( diff --git a/tests/base_rules/py_executable_base_tests.bzl b/tests/base_rules/py_executable_base_tests.bzl index 3cc6dfb702..37707831fc 100644 --- a/tests/base_rules/py_executable_base_tests.bzl +++ b/tests/base_rules/py_executable_base_tests.bzl @@ -24,7 +24,7 @@ load("//python/private:util.bzl", "IS_BAZEL_7_OR_HIGHER") # buildifier: disable load("//tests/base_rules:base_tests.bzl", "create_base_tests") load("//tests/base_rules:util.bzl", "WINDOWS_ATTR", pt_util = "util") load("//tests/support:py_executable_info_subject.bzl", "PyExecutableInfoSubject") -load("//tests/support:support.bzl", "CC_TOOLCHAIN", "CROSSTOOL_TOP", "LINUX_X86_64", "WINDOWS_X86_64") +load("//tests/support:support.bzl", "BOOTSTRAP_IMPL", "CC_TOOLCHAIN", "CROSSTOOL_TOP", "LINUX_X86_64", "WINDOWS_X86_64") _tests = [] @@ -342,6 +342,53 @@ def _test_name_cannot_end_in_py_impl(env, target): matching.str_matches("name must not end in*.py"), ) +def _test_main_module_bootstrap_system_python(name, config): + rt_util.helper_target( + config.rule, + name = name + "_subject", + main_module = "dummy", + ) + analysis_test( + name = name, + impl = _test_main_module_bootstrap_system_python_impl, + target = name + "_subject", + config_settings = { + BOOTSTRAP_IMPL: "system_python", + "//command_line_option:platforms": [LINUX_X86_64], + }, + expect_failure = True, + ) + +def _test_main_module_bootstrap_system_python_impl(env, target): + env.expect.that_target(target).failures().contains_predicate( + matching.str_matches("mandatory*srcs"), + ) + +_tests.append(_test_main_module_bootstrap_system_python) + +def _test_main_module_bootstrap_script(name, config): + rt_util.helper_target( + config.rule, + name = name + "_subject", + main_module = "dummy", + ) + analysis_test( + name = name, + impl = _test_main_module_bootstrap_script_impl, + target = name + "_subject", + config_settings = { + BOOTSTRAP_IMPL: "script", + "//command_line_option:platforms": [LINUX_X86_64], + }, + ) + +def _test_main_module_bootstrap_script_impl(env, target): + env.expect.that_target(target).default_outputs().contains( + "{package}/{test_name}_subject", + ) + +_tests.append(_test_main_module_bootstrap_script) + def _test_py_runtime_info_provided(name, config): rt_util.helper_target( config.rule, @@ -365,29 +412,6 @@ def _test_py_runtime_info_provided_impl(env, target): _tests.append(_test_py_runtime_info_provided) -# Can't test this -- mandatory validation happens before analysis test -# can intercept it -# TODO(#1069): Once re-implemented in Starlark, modify rule logic to make this -# testable. -# def _test_srcs_is_mandatory(name, config): -# rt_util.helper_target( -# config.rule, -# name = name + "_subject", -# ) -# analysis_test( -# name = name, -# impl = _test_srcs_is_mandatory, -# target = name + "_subject", -# expect_failure = True, -# ) -# -# _tests.append(_test_srcs_is_mandatory) -# -# def _test_srcs_is_mandatory_impl(env, target): -# env.expect.that_target(target).failures().contains_predicate( -# matching.str_matches("mandatory*srcs"), -# ) - # ===== # You were gonna add a test at the end, weren't you? # Nope. Please keep them sorted; put it in its alphabetical location. diff --git a/tests/support/support.bzl b/tests/support/support.bzl index 2b6703843b..6330155d8c 100644 --- a/tests/support/support.bzl +++ b/tests/support/support.bzl @@ -35,6 +35,7 @@ CROSSTOOL_TOP = Label("//tests/support/cc_toolchains:cc_toolchain_suite") # str() around Label() is necessary because rules_testing's config_settings # doesn't accept yet Label objects. ADD_SRCS_TO_RUNFILES = str(Label("//python/config_settings:add_srcs_to_runfiles")) +BOOTSTRAP_IMPL = str(Label("//python/config_settings:bootstrap_impl")) EXEC_TOOLS_TOOLCHAIN = str(Label("//python/config_settings:exec_tools_toolchain")) PRECOMPILE = str(Label("//python/config_settings:precompile")) PRECOMPILE_SOURCE_RETENTION = str(Label("//python/config_settings:precompile_source_retention")) From 2cb920c1e52a85239d6bcc38919fbf143b514dac Mon Sep 17 00:00:00 2001 From: Ignas Anikevicius <240938+aignas@users.noreply.github.com> Date: Mon, 14 Apr 2025 08:32:10 +0900 Subject: [PATCH 163/922] refactor(pypi): translate wheel METADATA parsing to starlark (#2629) This PR starts using the newly introduced (#2692) PEP508 compliant requirement marker parser in starlark and moves the dependency generation from the Python language (`whl_installer`) to the Starlark in the `whl_library` repository rule. This PR is (almost) a pure refactor where no bugs are fixed, but this is foundational work that also adds notes on how things will be moved to macros (i.e. analysis phase) so that we can fix a few long standing bugs and prepare for stabilizing the `experimental_index_url` (#260). Refactor: * I have migrated all of the unit tests from Python to starlark for deps generation from METADATA `Requires-Dist` fields. * Read the `METADATA` file itself in Starlark. Work towards #260, #2319, #2241 Fixes #2423 --- python/private/pypi/BUILD.bazel | 19 + python/private/pypi/pep508_deps.bzl | 351 ++++++++++++++++ python/private/pypi/pep508_evaluate.bzl | 13 +- python/private/pypi/whl_installer/BUILD.bazel | 1 - .../private/pypi/whl_installer/arguments.py | 8 - python/private/pypi/whl_installer/platform.py | 304 -------------- python/private/pypi/whl_installer/wheel.py | 281 ------------- .../pypi/whl_installer/wheel_installer.py | 37 +- python/private/pypi/whl_library.bzl | 57 ++- python/private/pypi/whl_library_targets.bzl | 2 - python/private/pypi/whl_metadata.bzl | 108 +++++ tests/pypi/pep508/BUILD.bazel | 5 + tests/pypi/pep508/deps_tests.bzl | 385 ++++++++++++++++++ tests/pypi/pep508/evaluate_tests.bzl | 2 + tests/pypi/whl_installer/BUILD.bazel | 24 -- tests/pypi/whl_installer/arguments_test.py | 14 +- tests/pypi/whl_installer/platform_test.py | 154 ------- .../whl_installer/wheel_installer_test.py | 42 +- tests/pypi/whl_installer/wheel_test.py | 371 ----------------- tests/pypi/whl_metadata/BUILD.bazel | 5 + .../pypi/whl_metadata/whl_metadata_tests.bzl | 147 +++++++ 21 files changed, 1099 insertions(+), 1231 deletions(-) create mode 100644 python/private/pypi/pep508_deps.bzl delete mode 100644 python/private/pypi/whl_installer/platform.py create mode 100644 python/private/pypi/whl_metadata.bzl create mode 100644 tests/pypi/pep508/deps_tests.bzl delete mode 100644 tests/pypi/whl_installer/platform_test.py delete mode 100644 tests/pypi/whl_installer/wheel_test.py create mode 100644 tests/pypi/whl_metadata/BUILD.bazel create mode 100644 tests/pypi/whl_metadata/whl_metadata_tests.bzl diff --git a/python/private/pypi/BUILD.bazel b/python/private/pypi/BUILD.bazel index e0a2f20c14..7297238cb4 100644 --- a/python/private/pypi/BUILD.bazel +++ b/python/private/pypi/BUILD.bazel @@ -221,6 +221,18 @@ bzl_library( ], ) +bzl_library( + name = "pep508_deps_bzl", + srcs = ["pep508_deps.bzl"], + deps = [ + ":pep508_env_bzl", + ":pep508_evaluate_bzl", + ":pep508_platform_bzl", + ":pep508_requirement_bzl", + "//python/private:normalize_name_bzl", + ], +) + bzl_library( name = "pep508_env_bzl", srcs = ["pep508_env.bzl"], @@ -368,7 +380,9 @@ bzl_library( ":generate_whl_library_build_bazel_bzl", ":parse_whl_name_bzl", ":patch_whl_bzl", + ":pep508_deps_bzl", ":pypi_repo_utils_bzl", + ":whl_metadata_bzl", ":whl_target_platforms_bzl", "//python/private:auth_bzl", "//python/private:envsubst_bzl", @@ -377,6 +391,11 @@ bzl_library( ], ) +bzl_library( + name = "whl_metadata_bzl", + srcs = ["whl_metadata.bzl"], +) + bzl_library( name = "whl_repo_name_bzl", srcs = ["whl_repo_name.bzl"], diff --git a/python/private/pypi/pep508_deps.bzl b/python/private/pypi/pep508_deps.bzl new file mode 100644 index 0000000000..af0a75362b --- /dev/null +++ b/python/private/pypi/pep508_deps.bzl @@ -0,0 +1,351 @@ +# Copyright 2025 The Bazel Authors. All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +"""This module is for implementing PEP508 compliant METADATA deps parsing. +""" + +load("//python/private:normalize_name.bzl", "normalize_name") +load(":pep508_env.bzl", "env") +load(":pep508_evaluate.bzl", "evaluate") +load(":pep508_platform.bzl", "platform", "platform_from_str") +load(":pep508_requirement.bzl", "requirement") + +_ALL_OS_VALUES = [ + "windows", + "osx", + "linux", +] +_ALL_ARCH_VALUES = [ + "aarch64", + "ppc64", + "ppc64le", + "s390x", + "x86_32", + "x86_64", +] + +def deps(name, *, requires_dist, platforms = [], extras = [], host_python_version = None): + """Parse the RequiresDist from wheel METADATA + + Args: + name: {type}`str` the name of the wheel. + requires_dist: {type}`list[str]` the list of RequiresDist lines from the + METADATA file. + extras: {type}`list[str]` the requested extras to generate targets for. + platforms: {type}`list[str]` the list of target platform strings. + host_python_version: {type}`str` the host python version. + + Returns: + A struct with attributes: + * deps: {type}`list[str]` dependencies to include unconditionally. + * deps_select: {type}`dict[str, list[str]]` dependencies to include on particular + subset of target platforms. + """ + reqs = sorted( + [requirement(r) for r in requires_dist], + key = lambda x: "{}:{}:".format(x.name, sorted(x.extras), x.marker), + ) + deps = {} + deps_select = {} + name = normalize_name(name) + want_extras = _resolve_extras(name, reqs, extras) + + # drop self edges + reqs = [r for r in reqs if r.name != name] + + platforms = [ + platform_from_str(p, python_version = host_python_version) + for p in platforms + ] or [ + platform_from_str("", python_version = host_python_version), + ] + + abis = sorted({p.abi: True for p in platforms if p.abi}) + if host_python_version and len(abis) > 1: + _, _, minor_version = host_python_version.partition(".") + minor_version, _, _ = minor_version.partition(".") + default_abi = "cp3" + minor_version + elif len(abis) > 1: + fail( + "all python versions need to be specified explicitly, got: {}".format(platforms), + ) + else: + default_abi = None + + for req in reqs: + _add_req( + deps, + deps_select, + req, + extras = want_extras, + platforms = platforms, + default_abi = default_abi, + ) + + return struct( + deps = sorted(deps), + deps_select = { + _platform_str(p): sorted(deps) + for p, deps in deps_select.items() + }, + ) + +def _platform_str(self): + if self.abi == None: + if not self.os and not self.arch: + return "//conditions:default" + elif not self.arch: + return "@platforms//os:{}".format(self.os) + else: + return "{}_{}".format(self.os, self.arch) + + minor_version = self.abi[3:] + if self.arch == None and self.os == None: + return str(Label("//python/config_settings:is_python_3.{}".format(minor_version))) + + return "cp3{}_{}_{}".format( + minor_version, + self.os or "anyos", + self.arch or "anyarch", + ) + +def _platform_specializations(self, cpu_values = _ALL_ARCH_VALUES, os_values = _ALL_OS_VALUES): + """Return the platform itself and all its unambiguous specializations. + + For more info about specializations see + https://bazel.build/docs/configurable-attributes + """ + specializations = [] + specializations.append(self) + if self.arch == None: + specializations.extend([ + platform(os = self.os, arch = arch, abi = self.abi) + for arch in cpu_values + ]) + if self.os == None: + specializations.extend([ + platform(os = os, arch = self.arch, abi = self.abi) + for os in os_values + ]) + if self.os == None and self.arch == None: + specializations.extend([ + platform(os = os, arch = arch, abi = self.abi) + for os in os_values + for arch in cpu_values + ]) + return specializations + +def _add(deps, deps_select, dep, platform): + dep = normalize_name(dep) + + if platform == None: + deps[dep] = True + + # If the dep is in the platform-specific list, remove it from the select. + pop_keys = [] + for p, _deps in deps_select.items(): + if dep not in _deps: + continue + + _deps.pop(dep) + if not _deps: + pop_keys.append(p) + + for p in pop_keys: + deps_select.pop(p) + return + + if dep in deps: + # If the dep is already in the main dependency list, no need to add it in the + # platform-specific dependency list. + return + + # Add the platform-specific branch + deps_select.setdefault(platform, {}) + + # Add the dep to specializations of the given platform if they + # exist in the select statement. + for p in _platform_specializations(platform): + if p not in deps_select: + continue + + deps_select[p][dep] = True + + if len(deps_select[platform]) == 1: + # We are adding a new item to the select and we need to ensure that + # existing dependencies from less specialized platforms are propagated + # to the newly added dependency set. + for p, _deps in deps_select.items(): + # Check if the existing platform overlaps with the given platform + if p == platform or platform not in _platform_specializations(p): + continue + + deps_select[platform].update(_deps) + +def _maybe_add_common_dep(deps, deps_select, platforms, dep): + abis = sorted({p.abi: True for p in platforms if p.abi}) + if len(abis) < 2: + return + + platforms = [platform()] + [ + platform(abi = abi) + for abi in abis + ] + + # If the dep is targeting all target python versions, lets add it to + # the common dependency list to simplify the select statements. + for p in platforms: + if p not in deps_select: + return + + if dep not in deps_select[p]: + return + + # All of the python version-specific branches have the dep, so lets add + # it to the common deps. + deps[dep] = True + for p in platforms: + deps_select[p].pop(dep) + if not deps_select[p]: + deps_select.pop(p) + +def _resolve_extras(self_name, reqs, extras): + """Resolve extras which are due to depending on self[some_other_extra]. + + Some packages may have cyclic dependencies resulting from extras being used, one example is + `etils`, where we have one set of extras as aliases for other extras + and we have an extra called 'all' that includes all other extras. + + Example: github.com/google/etils/blob/a0b71032095db14acf6b33516bca6d885fe09e35/pyproject.toml#L32. + + When the `requirements.txt` is generated by `pip-tools`, then it is likely that + this step is not needed, but for other `requirements.txt` files this may be useful. + + NOTE @aignas 2023-12-08: the extra resolution is not platform dependent, + but in order for it to become platform dependent we would have to have + separate targets for each extra in extras. + """ + + # Resolve any extra extras due to self-edges, empty string means no + # extras The empty string in the set is just a way to make the handling + # of no extras and a single extra easier and having a set of {"", "foo"} + # is equivalent to having {"foo"}. + extras = extras or [""] + + self_reqs = [] + for req in reqs: + if req.name != self_name: + continue + + if req.marker == None: + # I am pretty sure we cannot reach this code as it does not + # make sense to specify packages in this way, but since it is + # easy to handle, lets do it. + # + # TODO @aignas 2023-12-08: add a test + extras = extras + req.extras + else: + # process these in a separate loop + self_reqs.append(req) + + # A double loop is not strictly optimal, but always correct without recursion + for req in self_reqs: + if [True for extra in extras if evaluate(req.marker, env = {"extra": extra})]: + extras = extras + req.extras + else: + continue + + # Iterate through all packages to ensure that we include all of the extras from previously + # visited packages. + for req_ in self_reqs: + if [True for extra in extras if evaluate(req.marker, env = {"extra": extra})]: + extras = extras + req_.extras + + # Poor mans set + return sorted({x: None for x in extras}) + +def _add_req(deps, deps_select, req, *, extras, platforms, default_abi = None): + if not req.marker: + _add(deps, deps_select, req.name, None) + return + + # NOTE @aignas 2023-12-08: in order to have reasonable select statements + # we do have to have some parsing of the markers, so it begs the question + # if packaging should be reimplemented in Starlark to have the best solution + # for now we will implement it in Python and see what the best parsing result + # can be before making this decision. + match_os = len([ + tag + for tag in [ + "os_name", + "sys_platform", + "platform_system", + ] + if tag in req.marker + ]) > 0 + match_arch = "platform_machine" in req.marker + match_version = "version" in req.marker + + if not (match_os or match_arch or match_version): + if [ + True + for extra in extras + for p in platforms + if evaluate( + req.marker, + env = env( + target_platform = p, + extra = extra, + ), + ) + ]: + _add(deps, deps_select, req.name, None) + return + + for plat in platforms: + if not [ + True + for extra in extras + if evaluate( + req.marker, + env = env( + target_platform = plat, + extra = extra, + ), + ) + ]: + continue + + if match_arch and default_abi: + _add(deps, deps_select, req.name, plat) + if plat.abi == default_abi: + _add(deps, deps_select, req.name, platform(os = plat.os, arch = plat.arch)) + elif match_arch: + _add(deps, deps_select, req.name, platform(os = plat.os, arch = plat.arch)) + elif match_os and default_abi: + _add(deps, deps_select, req.name, platform(os = plat.os, abi = plat.abi)) + if plat.abi == default_abi: + _add(deps, deps_select, req.name, platform(os = plat.os)) + elif match_os: + _add(deps, deps_select, req.name, platform(os = plat.os)) + elif match_version and default_abi: + _add(deps, deps_select, req.name, platform(abi = plat.abi)) + if plat.abi == default_abi: + _add(deps, deps_select, req.name, platform()) + elif match_version: + _add(deps, deps_select, req.name, None) + else: + fail("BUG: {} support is not implemented".format(req.marker)) + + _maybe_add_common_dep(deps, deps_select, platforms, req.name) diff --git a/python/private/pypi/pep508_evaluate.bzl b/python/private/pypi/pep508_evaluate.bzl index f45eb75cdb..f8ef553034 100644 --- a/python/private/pypi/pep508_evaluate.bzl +++ b/python/private/pypi/pep508_evaluate.bzl @@ -138,7 +138,7 @@ def evaluate(marker, *, env, strict = True, **kwargs): """ tokens = tokenize(marker) - ast = _new_expr(**kwargs) + ast = _new_expr(marker = marker, **kwargs) for _ in range(len(tokens) * 2): if not tokens: break @@ -219,17 +219,20 @@ def _not_fn(x): return not x def _new_expr( + *, + marker, and_fn = _and_fn, or_fn = _or_fn, not_fn = _not_fn): # buildifier: disable=uninitialized self = struct( + marker = marker, tree = [], parse = lambda **kwargs: _parse(self, **kwargs), value = lambda: _value(self), # This is a way for us to have a handle to the currently constructed # expression tree branch. - current = lambda: self._current[0] if self._current else None, + current = lambda: self._current[-1] if self._current else None, _current = [], _and = and_fn, _or = or_fn, @@ -313,6 +316,7 @@ def marker_expr(left, op, right, *, env, strict = True): # # The following normalizes the values left = env.get(_ENV_ALIASES, {}).get(var_name, {}).get(left, left) + else: var_name = left left = env[left] @@ -392,12 +396,15 @@ def _append(self, value): current.tree.append(value) elif hasattr(current.tree[-1], "append"): current.tree[-1].append(value) - else: + elif hasattr(current.tree, "_append"): current.tree._append(value) + else: + fail("Cannot evaluate '{}' in '{}', current: {}".format(value, self.marker, current)) def _open_parenthesis(self): """Add an extra node into the tree to perform evaluate inside parenthesis.""" self._current.append(_new_expr( + marker = self.marker, and_fn = self._and, or_fn = self._or, not_fn = self._not, diff --git a/python/private/pypi/whl_installer/BUILD.bazel b/python/private/pypi/whl_installer/BUILD.bazel index 5fb617004d..49f1a119c1 100644 --- a/python/private/pypi/whl_installer/BUILD.bazel +++ b/python/private/pypi/whl_installer/BUILD.bazel @@ -6,7 +6,6 @@ py_library( srcs = [ "arguments.py", "namespace_pkgs.py", - "platform.py", "wheel.py", "wheel_installer.py", ], diff --git a/python/private/pypi/whl_installer/arguments.py b/python/private/pypi/whl_installer/arguments.py index 29bea8026e..bb841ea9ab 100644 --- a/python/private/pypi/whl_installer/arguments.py +++ b/python/private/pypi/whl_installer/arguments.py @@ -17,8 +17,6 @@ import pathlib from typing import Any, Dict, Set -from python.private.pypi.whl_installer.platform import Platform - def parser(**kwargs: Any) -> argparse.ArgumentParser: """Create a parser for the wheel_installer tool.""" @@ -41,12 +39,6 @@ def parser(**kwargs: Any) -> argparse.ArgumentParser: action="store", help="Extra arguments to pass down to pip.", ) - parser.add_argument( - "--platform", - action="extend", - type=Platform.from_string, - help="Platforms to target dependencies. Can be used multiple times.", - ) parser.add_argument( "--pip_data_exclude", action="store", diff --git a/python/private/pypi/whl_installer/platform.py b/python/private/pypi/whl_installer/platform.py deleted file mode 100644 index 11dd6e37ab..0000000000 --- a/python/private/pypi/whl_installer/platform.py +++ /dev/null @@ -1,304 +0,0 @@ -# Copyright 2024 The Bazel Authors. All rights reserved. -# -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -"""Utility class to inspect an extracted wheel directory""" - -import platform -import sys -from dataclasses import dataclass -from enum import Enum -from typing import Any, Dict, Iterator, List, Optional, Union - - -class OS(Enum): - linux = 1 - osx = 2 - windows = 3 - darwin = osx - win32 = windows - - @classmethod - def interpreter(cls) -> "OS": - "Return the interpreter operating system." - return cls[sys.platform.lower()] - - def __str__(self) -> str: - return self.name.lower() - - -class Arch(Enum): - x86_64 = 1 - x86_32 = 2 - aarch64 = 3 - ppc = 4 - ppc64le = 5 - s390x = 6 - arm = 7 - amd64 = x86_64 - arm64 = aarch64 - i386 = x86_32 - i686 = x86_32 - x86 = x86_32 - - @classmethod - def interpreter(cls) -> "Arch": - "Return the currently running interpreter architecture." - # FIXME @aignas 2023-12-13: Hermetic toolchain on Windows 3.11.6 - # is returning an empty string here, so lets default to x86_64 - return cls[platform.machine().lower() or "x86_64"] - - def __str__(self) -> str: - return self.name.lower() - - -def _as_int(value: Optional[Union[OS, Arch]]) -> int: - """Convert one of the enums above to an int for easier sorting algorithms. - - Args: - value: The value of an enum or None. - - Returns: - -1 if we get None, otherwise, the numeric value of the given enum. - """ - if value is None: - return -1 - - return int(value.value) - - -def host_interpreter_minor_version() -> int: - return sys.version_info.minor - - -@dataclass(frozen=True) -class Platform: - os: Optional[OS] = None - arch: Optional[Arch] = None - minor_version: Optional[int] = None - - @classmethod - def all( - cls, - want_os: Optional[OS] = None, - minor_version: Optional[int] = None, - ) -> List["Platform"]: - return sorted( - [ - cls(os=os, arch=arch, minor_version=minor_version) - for os in OS - for arch in Arch - if not want_os or want_os == os - ] - ) - - @classmethod - def host(cls) -> List["Platform"]: - """Use the Python interpreter to detect the platform. - - We extract `os` from sys.platform and `arch` from platform.machine - - Returns: - A list of parsed values which makes the signature the same as - `Platform.all` and `Platform.from_string`. - """ - return [ - Platform( - os=OS.interpreter(), - arch=Arch.interpreter(), - minor_version=host_interpreter_minor_version(), - ) - ] - - def all_specializations(self) -> Iterator["Platform"]: - """Return the platform itself and all its unambiguous specializations. - - For more info about specializations see - https://bazel.build/docs/configurable-attributes - """ - yield self - if self.arch is None: - for arch in Arch: - yield Platform(os=self.os, arch=arch, minor_version=self.minor_version) - if self.os is None: - for os in OS: - yield Platform(os=os, arch=self.arch, minor_version=self.minor_version) - if self.arch is None and self.os is None: - for os in OS: - for arch in Arch: - yield Platform(os=os, arch=arch, minor_version=self.minor_version) - - def __lt__(self, other: Any) -> bool: - """Add a comparison method, so that `sorted` returns the most specialized platforms first.""" - if not isinstance(other, Platform) or other is None: - raise ValueError(f"cannot compare {other} with Platform") - - self_arch, self_os = _as_int(self.arch), _as_int(self.os) - other_arch, other_os = _as_int(other.arch), _as_int(other.os) - - if self_os == other_os: - return self_arch < other_arch - else: - return self_os < other_os - - def __str__(self) -> str: - if self.minor_version is None: - if self.os is None and self.arch is None: - return "//conditions:default" - - if self.arch is None: - return f"@platforms//os:{self.os}" - else: - return f"{self.os}_{self.arch}" - - if self.arch is None and self.os is None: - return f"@//python/config_settings:is_python_3.{self.minor_version}" - - if self.arch is None: - return f"cp3{self.minor_version}_{self.os}_anyarch" - - if self.os is None: - return f"cp3{self.minor_version}_anyos_{self.arch}" - - return f"cp3{self.minor_version}_{self.os}_{self.arch}" - - @classmethod - def from_string(cls, platform: Union[str, List[str]]) -> List["Platform"]: - """Parse a string and return a list of platforms""" - platform = [platform] if isinstance(platform, str) else list(platform) - ret = set() - for p in platform: - if p == "host": - ret.update(cls.host()) - continue - - abi, _, tail = p.partition("_") - if not abi.startswith("cp"): - # The first item is not an abi - tail = p - abi = "" - os, _, arch = tail.partition("_") - arch = arch or "*" - - minor_version = int(abi[len("cp3") :]) if abi else None - - if arch != "*": - ret.add( - cls( - os=OS[os] if os != "*" else None, - arch=Arch[arch], - minor_version=minor_version, - ) - ) - - else: - ret.update( - cls.all( - want_os=OS[os] if os != "*" else None, - minor_version=minor_version, - ) - ) - - return sorted(ret) - - # NOTE @aignas 2023-12-05: below is the minimum number of accessors that are defined in - # https://peps.python.org/pep-0496/ to make rules_python generate dependencies. - # - # WARNING: It may not work in cases where the python implementation is different between - # different platforms. - - # derived from OS - @property - def os_name(self) -> str: - if self.os == OS.linux or self.os == OS.osx: - return "posix" - elif self.os == OS.windows: - return "nt" - else: - return "" - - @property - def sys_platform(self) -> str: - if self.os == OS.linux: - return "linux" - elif self.os == OS.osx: - return "darwin" - elif self.os == OS.windows: - return "win32" - else: - return "" - - @property - def platform_system(self) -> str: - if self.os == OS.linux: - return "Linux" - elif self.os == OS.osx: - return "Darwin" - elif self.os == OS.windows: - return "Windows" - else: - return "" - - # derived from OS and Arch - @property - def platform_machine(self) -> str: - """Guess the target 'platform_machine' marker. - - NOTE @aignas 2023-12-05: this may not work on really new systems, like - Windows if they define the platform markers in a different way. - """ - if self.arch == Arch.x86_64: - return "x86_64" - elif self.arch == Arch.x86_32 and self.os != OS.osx: - return "i386" - elif self.arch == Arch.x86_32: - return "" - elif self.arch == Arch.aarch64 and self.os == OS.linux: - return "aarch64" - elif self.arch == Arch.aarch64: - # Assuming that OSX and Windows use this one since the precedent is set here: - # https://github.com/cgohlke/win_arm64-wheels - return "arm64" - elif self.os != OS.linux: - return "" - elif self.arch == Arch.ppc: - return "ppc" - elif self.arch == Arch.ppc64le: - return "ppc64le" - elif self.arch == Arch.s390x: - return "s390x" - else: - return "" - - def env_markers(self, extra: str) -> Dict[str, str]: - # If it is None, use the host version - minor_version = self.minor_version or host_interpreter_minor_version() - - return { - "extra": extra, - "os_name": self.os_name, - "sys_platform": self.sys_platform, - "platform_machine": self.platform_machine, - "platform_system": self.platform_system, - "platform_release": "", # unset - "platform_version": "", # unset - "python_version": f"3.{minor_version}", - # FIXME @aignas 2024-01-14: is putting zero last a good idea? Maybe we should - # use `20` or something else to avoid having weird issues where the full version is used for - # matching and the author decides to only support 3.y.5 upwards. - "implementation_version": f"3.{minor_version}.0", - "python_full_version": f"3.{minor_version}.0", - # we assume that the following are the same as the interpreter used to setup the deps: - # "implementation_name": "cpython" - # "platform_python_implementation: "CPython", - } diff --git a/python/private/pypi/whl_installer/wheel.py b/python/private/pypi/whl_installer/wheel.py index d95b33a194..da81b5ea9f 100644 --- a/python/private/pypi/whl_installer/wheel.py +++ b/python/private/pypi/whl_installer/wheel.py @@ -25,275 +25,6 @@ from packaging.requirements import Requirement from pip._vendor.packaging.utils import canonicalize_name -from python.private.pypi.whl_installer.platform import ( - Platform, - host_interpreter_minor_version, -) - - -@dataclass(frozen=True) -class FrozenDeps: - deps: List[str] - deps_select: Dict[str, List[str]] - - -class Deps: - """Deps is a dependency builder that has a build() method to return FrozenDeps.""" - - def __init__( - self, - name: str, - requires_dist: List[str], - *, - extras: Optional[Set[str]] = None, - platforms: Optional[Set[Platform]] = None, - ): - """Create a new instance and parse the requires_dist - - Args: - name (str): The name of the whl distribution - requires_dist (list[Str]): The Requires-Dist from the METADATA of the whl - distribution. - extras (set[str], optional): The list of requested extras, defaults to None. - platforms (set[Platform], optional): The list of target platforms, defaults to - None. If the list of platforms has multiple `minor_version` values, it - will change the code to generate the select statements using - `@rules_python//python/config_settings:is_python_3.y` conditions. - """ - self.name: str = Deps._normalize(name) - self._platforms: Set[Platform] = platforms or set() - self._target_versions = {p.minor_version for p in platforms or {}} - self._default_minor_version = None - if platforms and len(self._target_versions) > 2: - # TODO @aignas 2024-06-23: enable this to be set via a CLI arg - # for being more explicit. - self._default_minor_version = host_interpreter_minor_version() - - if None in self._target_versions and len(self._target_versions) > 2: - raise ValueError( - f"all python versions need to be specified explicitly, got: {platforms}" - ) - - # Sort so that the dictionary order in the FrozenDeps is deterministic - # without the final sort because Python retains insertion order. That way - # the sorting by platform is limited within the Platform class itself and - # the unit-tests for the Deps can be simpler. - reqs = sorted( - (Requirement(wheel_req) for wheel_req in requires_dist), - key=lambda x: f"{x.name}:{sorted(x.extras)}", - ) - - want_extras = self._resolve_extras(reqs, extras) - - # Then add all of the requirements in order - self._deps: Set[str] = set() - self._select: Dict[Platform, Set[str]] = defaultdict(set) - for req in reqs: - self._add_req(req, want_extras) - - def _add(self, dep: str, platform: Optional[Platform]): - dep = Deps._normalize(dep) - - # Self-edges are processed in _resolve_extras - if dep == self.name: - return - - if not platform: - self._deps.add(dep) - - # If the dep is in the platform-specific list, remove it from the select. - pop_keys = [] - for p, deps in self._select.items(): - if dep not in deps: - continue - - deps.remove(dep) - if not deps: - pop_keys.append(p) - - for p in pop_keys: - self._select.pop(p) - return - - if dep in self._deps: - # If the dep is already in the main dependency list, no need to add it in the - # platform-specific dependency list. - return - - # Add the platform-specific dep - self._select[platform].add(dep) - - # Add the dep to specializations of the given platform if they - # exist in the select statement. - for p in platform.all_specializations(): - if p not in self._select: - continue - - self._select[p].add(dep) - - if len(self._select[platform]) == 1: - # We are adding a new item to the select and we need to ensure that - # existing dependencies from less specialized platforms are propagated - # to the newly added dependency set. - for p, deps in self._select.items(): - # Check if the existing platform overlaps with the given platform - if p == platform or platform not in p.all_specializations(): - continue - - self._select[platform].update(self._select[p]) - - def _maybe_add_common_dep(self, dep): - if len(self._target_versions) < 2: - return - - platforms = [Platform()] + [ - Platform(minor_version=v) for v in self._target_versions - ] - - # If the dep is targeting all target python versions, lets add it to - # the common dependency list to simplify the select statements. - for p in platforms: - if p not in self._select: - return - - if dep not in self._select[p]: - return - - # All of the python version-specific branches have the dep, so lets add - # it to the common deps. - self._deps.add(dep) - for p in platforms: - self._select[p].remove(dep) - if not self._select[p]: - self._select.pop(p) - - @staticmethod - def _normalize(name: str) -> str: - return re.sub(r"[-_.]+", "_", name).lower() - - def _resolve_extras( - self, reqs: List[Requirement], extras: Optional[Set[str]] - ) -> Set[str]: - """Resolve extras which are due to depending on self[some_other_extra]. - - Some packages may have cyclic dependencies resulting from extras being used, one example is - `etils`, where we have one set of extras as aliases for other extras - and we have an extra called 'all' that includes all other extras. - - Example: github.com/google/etils/blob/a0b71032095db14acf6b33516bca6d885fe09e35/pyproject.toml#L32. - - When the `requirements.txt` is generated by `pip-tools`, then it is likely that - this step is not needed, but for other `requirements.txt` files this may be useful. - - NOTE @aignas 2023-12-08: the extra resolution is not platform dependent, - but in order for it to become platform dependent we would have to have - separate targets for each extra in extras. - """ - - # Resolve any extra extras due to self-edges, empty string means no - # extras The empty string in the set is just a way to make the handling - # of no extras and a single extra easier and having a set of {"", "foo"} - # is equivalent to having {"foo"}. - extras = extras or {""} - - self_reqs = [] - for req in reqs: - if Deps._normalize(req.name) != self.name: - continue - - if req.marker is None: - # I am pretty sure we cannot reach this code as it does not - # make sense to specify packages in this way, but since it is - # easy to handle, lets do it. - # - # TODO @aignas 2023-12-08: add a test - extras = extras | req.extras - else: - # process these in a separate loop - self_reqs.append(req) - - # A double loop is not strictly optimal, but always correct without recursion - for req in self_reqs: - if any(req.marker.evaluate({"extra": extra}) for extra in extras): - extras = extras | req.extras - else: - continue - - # Iterate through all packages to ensure that we include all of the extras from previously - # visited packages. - for req_ in self_reqs: - if any(req_.marker.evaluate({"extra": extra}) for extra in extras): - extras = extras | req_.extras - - return extras - - def _add_req(self, req: Requirement, extras: Set[str]) -> None: - if req.marker is None: - self._add(req.name, None) - return - - marker_str = str(req.marker) - - if not self._platforms: - if any(req.marker.evaluate({"extra": extra}) for extra in extras): - self._add(req.name, None) - return - - # NOTE @aignas 2023-12-08: in order to have reasonable select statements - # we do have to have some parsing of the markers, so it begs the question - # if packaging should be reimplemented in Starlark to have the best solution - # for now we will implement it in Python and see what the best parsing result - # can be before making this decision. - match_os = any( - tag in marker_str - for tag in [ - "os_name", - "sys_platform", - "platform_system", - ] - ) - match_arch = "platform_machine" in marker_str - match_version = "version" in marker_str - - if not (match_os or match_arch or match_version): - if any(req.marker.evaluate({"extra": extra}) for extra in extras): - self._add(req.name, None) - return - - for plat in self._platforms: - if not any( - req.marker.evaluate(plat.env_markers(extra)) for extra in extras - ): - continue - - if match_arch and self._default_minor_version: - self._add(req.name, plat) - if plat.minor_version == self._default_minor_version: - self._add(req.name, Platform(plat.os, plat.arch)) - elif match_arch: - self._add(req.name, Platform(plat.os, plat.arch)) - elif match_os and self._default_minor_version: - self._add(req.name, Platform(plat.os, minor_version=plat.minor_version)) - if plat.minor_version == self._default_minor_version: - self._add(req.name, Platform(plat.os)) - elif match_os: - self._add(req.name, Platform(plat.os)) - elif match_version and self._default_minor_version: - self._add(req.name, Platform(minor_version=plat.minor_version)) - if plat.minor_version == self._default_minor_version: - self._add(req.name, Platform()) - elif match_version: - self._add(req.name, None) - - # Merge to common if possible after processing all platforms - self._maybe_add_common_dep(req.name) - - def build(self) -> FrozenDeps: - return FrozenDeps( - deps=sorted(self._deps), - deps_select={str(p): sorted(deps) for p, deps in self._select.items()}, - ) - class Wheel: """Representation of the compressed .whl file""" @@ -344,18 +75,6 @@ def entry_points(self) -> Dict[str, Tuple[str, str]]: return entry_points_mapping - def dependencies( - self, - extras_requested: Set[str] = None, - platforms: Optional[Set[Platform]] = None, - ) -> FrozenDeps: - return Deps( - self.name, - extras=extras_requested, - platforms=platforms, - requires_dist=self.metadata.get_all("Requires-Dist", []), - ).build() - def unzip(self, directory: str) -> None: installation_schemes = { "purelib": "/site-packages", diff --git a/python/private/pypi/whl_installer/wheel_installer.py b/python/private/pypi/whl_installer/wheel_installer.py index ef8181c30d..c7695d92e8 100644 --- a/python/private/pypi/whl_installer/wheel_installer.py +++ b/python/private/pypi/whl_installer/wheel_installer.py @@ -23,7 +23,7 @@ import sys from pathlib import Path from tempfile import NamedTemporaryFile -from typing import Dict, List, Optional, Set, Tuple +from typing import Dict, Optional, Set, Tuple from pip._vendor.packaging.utils import canonicalize_name @@ -103,9 +103,7 @@ def _setup_namespace_pkg_compatibility(wheel_dir: str) -> None: def _extract_wheel( wheel_file: str, - extras: Dict[str, Set[str]], enable_implicit_namespace_pkgs: bool, - platforms: List[wheel.Platform], installation_dir: Path = Path("."), ) -> None: """Extracts wheel into given directory and creates py_library and filegroup targets. @@ -113,7 +111,6 @@ def _extract_wheel( Args: wheel_file: the filepath of the .whl installation_dir: the destination directory for installation of the wheel. - extras: a list of extras to add as dependencies for the installed wheel enable_implicit_namespace_pkgs: if true, disables conversion of implicit namespace packages and will unzip as-is """ @@ -123,25 +120,19 @@ def _extract_wheel( if not enable_implicit_namespace_pkgs: _setup_namespace_pkg_compatibility(installation_dir) - extras_requested = extras[whl.name] if whl.name in extras else set() - - dependencies = whl.dependencies(extras_requested, platforms) + metadata = { + "python_version": sys.version.partition(" ")[0], + "entry_points": [ + { + "name": name, + "module": module, + "attribute": attribute, + } + for name, (module, attribute) in sorted(whl.entry_points().items()) + ], + } with open(os.path.join(installation_dir, "metadata.json"), "w") as f: - metadata = { - "name": whl.name, - "version": whl.version, - "deps": dependencies.deps, - "deps_by_platform": dependencies.deps_select, - "entry_points": [ - { - "name": name, - "module": module, - "attribute": attribute, - } - for name, (module, attribute) in sorted(whl.entry_points().items()) - ], - } json.dump(metadata, f) @@ -155,13 +146,9 @@ def main() -> None: if args.whl_file: whl = Path(args.whl_file) - name, extras_for_pkg = _parse_requirement_for_extra(args.requirement) - extras = {name: extras_for_pkg} if extras_for_pkg and name else dict() _extract_wheel( wheel_file=whl, - extras=extras, enable_implicit_namespace_pkgs=args.enable_implicit_namespace_pkgs, - platforms=arguments.get_platforms(args), ) return diff --git a/python/private/pypi/whl_library.bzl b/python/private/pypi/whl_library.bzl index 493f11353e..54f9ff3909 100644 --- a/python/private/pypi/whl_library.bzl +++ b/python/private/pypi/whl_library.bzl @@ -21,9 +21,13 @@ load("//python/private:repo_utils.bzl", "REPO_DEBUG_ENV_VAR", "repo_utils") load(":attrs.bzl", "ATTRS", "use_isolated") load(":deps.bzl", "all_repo_names", "record_files") load(":generate_whl_library_build_bazel.bzl", "generate_whl_library_build_bazel") +load(":parse_requirements.bzl", "host_platform") load(":parse_whl_name.bzl", "parse_whl_name") load(":patch_whl.bzl", "patch_whl") +load(":pep508_deps.bzl", "deps") +load(":pep508_requirement.bzl", "requirement") load(":pypi_repo_utils.bzl", "pypi_repo_utils") +load(":whl_metadata.bzl", "whl_metadata") load(":whl_target_platforms.bzl", "whl_target_platforms") _CPPFLAGS = "CPPFLAGS" @@ -361,7 +365,7 @@ def _whl_library_impl(rctx): arguments = args + [ "--whl-file", whl_path, - ] + ["--platform={}".format(p) for p in target_platforms], + ], srcs = rctx.attr._python_srcs, environment = environment, quiet = rctx.attr.quiet, @@ -396,17 +400,60 @@ def _whl_library_impl(rctx): ) entry_points[entry_point_without_py] = entry_point_script_name + # TODO @aignas 2025-04-04: move this to whl_library_targets.bzl to have + # this in the analysis phase. + # + # This means that whl_library_targets will have to accept the following args: + # * name - the name of the package in the METADATA. + # * requires_dist - the list of METADATA Requires-Dist. + # * platforms - the list of target platforms. The target_platforms + # should come from the hub repo via a 'load' statement so that they don't + # need to be passed as an argument to `whl_library`. + # * extras - the list of required extras. This comes from the + # `rctx.attr.requirement` for now. In the future the required extras could + # stay in the hub repo, where we calculate the extra aliases that we need + # to create automatically and this way expose the targets for the specific + # extras. The first step will be to generate a target per extra for the + # `py_library` and `filegroup`. Maybe we need to have a special provider + # or an output group so that we can return the `whl` file from the + # `py_library` target? filegroup can use output groups to expose files. + # * host_python_version/versons - the list of python versions to support + # should come from the hub, similar to how the target platforms are specified. + # + # Extra things that we should move at the same time: + # * group_name, group_deps - this info can stay in the hub repository so that + # it is piped at the analysis time and changing the requirement groups does + # cause to re-fetch the deps. + python_version = metadata["python_version"] + metadata = whl_metadata( + install_dir = rctx.path("site-packages"), + read_fn = rctx.read, + logger = logger, + ) + + # TODO @aignas 2025-04-09: this will later be removed when loaded through the hub + major_minor, _, _ = python_version.rpartition(".") + package_deps = deps( + name = metadata.name, + requires_dist = metadata.requires_dist, + platforms = target_platforms or [ + "cp{}_{}".format(major_minor.replace(".", ""), host_platform(rctx)), + ], + extras = requirement(rctx.attr.requirement).extras, + host_python_version = python_version, + ) + build_file_contents = generate_whl_library_build_bazel( name = whl_path.basename, dep_template = rctx.attr.dep_template or "@{}{{name}}//:{{target}}".format(rctx.attr.repo_prefix), - dependencies = metadata["deps"], - dependencies_by_platform = metadata["deps_by_platform"], + dependencies = package_deps.deps, + dependencies_by_platform = package_deps.deps_select, group_name = rctx.attr.group_name, group_deps = rctx.attr.group_deps, data_exclude = rctx.attr.pip_data_exclude, tags = [ - "pypi_name=" + metadata["name"], - "pypi_version=" + metadata["version"], + "pypi_name=" + metadata.name, + "pypi_version=" + metadata.version, ], entry_points = entry_points, annotation = None if not rctx.attr.annotation else struct(**json.decode(rctx.read(rctx.attr.annotation))), diff --git a/python/private/pypi/whl_library_targets.bzl b/python/private/pypi/whl_library_targets.bzl index 95031e6181..d32746b604 100644 --- a/python/private/pypi/whl_library_targets.bzl +++ b/python/private/pypi/whl_library_targets.bzl @@ -90,8 +90,6 @@ def whl_library_targets( native: {type}`native` The native struct for overriding in tests. rules: {type}`struct` A struct with references to rules for creating targets. """ - _ = name # buildifier: @unused - dependencies = sorted([normalize_name(d) for d in dependencies]) dependencies_by_platform = { platform: sorted([normalize_name(d) for d in deps]) diff --git a/python/private/pypi/whl_metadata.bzl b/python/private/pypi/whl_metadata.bzl new file mode 100644 index 0000000000..8a86ffbff1 --- /dev/null +++ b/python/private/pypi/whl_metadata.bzl @@ -0,0 +1,108 @@ +"""A simple function to find the METADATA file and parse it""" + +_NAME = "Name: " +_PROVIDES_EXTRA = "Provides-Extra: " +_REQUIRES_DIST = "Requires-Dist: " +_VERSION = "Version: " + +def whl_metadata(*, install_dir, read_fn, logger): + """Find and parse the METADATA file in the extracted whl contents dir. + + Args: + install_dir: {type}`path` location where the wheel has been extracted. + read_fn: the function used to read files. + logger: the function used to log failures. + + Returns: + A struct with parsed values: + * `name`: {type}`str` the name of the wheel. + * `version`: {type}`str` the version of the wheel. + * `requires_dist`: {type}`list[str]` the list of requirements. + * `provides_extra`: {type}`list[str]` the list of extras that this package + provides. + """ + metadata_file = find_whl_metadata(install_dir = install_dir, logger = logger) + contents = read_fn(metadata_file) + result = parse_whl_metadata(contents) + + if not (result.name and result.version): + logger.fail("Failed to parsed the wheel METADATA file:\n{}".format(contents)) + return None + + return result + +def parse_whl_metadata(contents): + """Parse .whl METADATA file + + Args: + contents: {type}`str` the contents of the file. + + Returns: + A struct with parsed values: + * `name`: {type}`str` the name of the wheel. + * `version`: {type}`str` the version of the wheel. + * `requires_dist`: {type}`list[str]` the list of requirements. + * `provides_extra`: {type}`list[str]` the list of extras that this package + provides. + """ + parsed = { + "name": "", + "provides_extra": [], + "requires_dist": [], + "version": "", + } + for line in contents.strip().split("\n"): + if not line.strip(): + # Stop parsing on first empty line, which marks the end of the + # headers containing the metadata. + break + + if line.startswith(_NAME): + _, _, value = line.partition(_NAME) + parsed["name"] = value.strip() + elif line.startswith(_VERSION): + _, _, value = line.partition(_VERSION) + parsed["version"] = value.strip() + elif line.startswith(_REQUIRES_DIST): + _, _, value = line.partition(_REQUIRES_DIST) + parsed["requires_dist"].append(value.strip(" ")) + elif line.startswith(_PROVIDES_EXTRA): + _, _, value = line.partition(_PROVIDES_EXTRA) + parsed["provides_extra"].append(value.strip(" ")) + + return struct( + name = parsed["name"], + provides_extra = parsed["provides_extra"], + requires_dist = parsed["requires_dist"], + version = parsed["version"], + ) + +def find_whl_metadata(*, install_dir, logger): + """Find the whl METADATA file in the install_dir. + + Args: + install_dir: {type}`path` location where the wheel has been extracted. + logger: the function used to log failures. + + Returns: + {type}`path` The path to the METADATA file. + """ + dist_info = None + for maybe_dist_info in install_dir.readdir(): + # first find the ".dist-info" folder + if not (maybe_dist_info.is_dir and maybe_dist_info.basename.endswith(".dist-info")): + continue + + dist_info = maybe_dist_info + metadata_file = dist_info.get_child("METADATA") + + if metadata_file.exists: + return metadata_file + + break + + if dist_info: + logger.fail("The METADATA file for the wheel could not be found in '{}/{}'".format(install_dir.basename, dist_info.basename)) + else: + logger.fail("The '*.dist-info' directory could not be found in '{}'".format(install_dir.basename)) + return None diff --git a/tests/pypi/pep508/BUILD.bazel b/tests/pypi/pep508/BUILD.bazel index 575f28ada6..7eab2e096a 100644 --- a/tests/pypi/pep508/BUILD.bazel +++ b/tests/pypi/pep508/BUILD.bazel @@ -1,6 +1,11 @@ +load(":deps_tests.bzl", "deps_test_suite") load(":evaluate_tests.bzl", "evaluate_test_suite") load(":requirement_tests.bzl", "requirement_test_suite") +deps_test_suite( + name = "deps_tests", +) + evaluate_test_suite( name = "evaluate_tests", ) diff --git a/tests/pypi/pep508/deps_tests.bzl b/tests/pypi/pep508/deps_tests.bzl new file mode 100644 index 0000000000..44031ab6a5 --- /dev/null +++ b/tests/pypi/pep508/deps_tests.bzl @@ -0,0 +1,385 @@ +# Copyright 2025 The Bazel Authors. All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +"""Tests for construction of Python version matching config settings.""" + +load("@rules_testing//lib:test_suite.bzl", "test_suite") +load("//python/private/pypi:pep508_deps.bzl", "deps") # buildifier: disable=bzl-visibility + +_tests = [] + +def test_simple_deps(env): + got = deps( + "foo", + requires_dist = ["bar-Bar"], + ) + env.expect.that_collection(got.deps).contains_exactly(["bar_bar"]) + env.expect.that_dict(got.deps_select).contains_exactly({}) + +_tests.append(test_simple_deps) + +def test_can_add_os_specific_deps(env): + got = deps( + "foo", + requires_dist = [ + "bar", + "an_osx_dep; sys_platform=='darwin'", + "posix_dep; os_name=='posix'", + "win_dep; os_name=='nt'", + ], + platforms = [ + "linux_x86_64", + "osx_x86_64", + "osx_aarch64", + "windows_x86_64", + ], + host_python_version = "3.3.1", + ) + + env.expect.that_collection(got.deps).contains_exactly(["bar"]) + env.expect.that_dict(got.deps_select).contains_exactly({ + "@platforms//os:linux": ["posix_dep"], + "@platforms//os:osx": ["an_osx_dep", "posix_dep"], + "@platforms//os:windows": ["win_dep"], + }) + +_tests.append(test_can_add_os_specific_deps) + +def test_can_add_os_specific_deps_with_python_version(env): + got = deps( + "foo", + requires_dist = [ + "bar", + "an_osx_dep; sys_platform=='darwin'", + "posix_dep; os_name=='posix'", + "win_dep; os_name=='nt'", + ], + platforms = [ + "cp33_linux_x86_64", + "cp33_osx_x86_64", + "cp33_osx_aarch64", + "cp33_windows_x86_64", + ], + ) + + env.expect.that_collection(got.deps).contains_exactly(["bar"]) + env.expect.that_dict(got.deps_select).contains_exactly({ + "@platforms//os:linux": ["posix_dep"], + "@platforms//os:osx": ["an_osx_dep", "posix_dep"], + "@platforms//os:windows": ["win_dep"], + }) + +_tests.append(test_can_add_os_specific_deps_with_python_version) + +def test_deps_are_added_to_more_specialized_platforms(env): + got = deps( + "foo", + requires_dist = [ + "m1_dep; sys_platform=='darwin' and platform_machine=='arm64'", + "mac_dep; sys_platform=='darwin'", + ], + platforms = [ + "osx_x86_64", + "osx_aarch64", + ], + host_python_version = "3.8.4", + ) + + env.expect.that_collection(got.deps).contains_exactly([]) + env.expect.that_dict(got.deps_select).contains_exactly({ + "@platforms//os:osx": ["mac_dep"], + "osx_aarch64": ["m1_dep", "mac_dep"], + }) + +_tests.append(test_deps_are_added_to_more_specialized_platforms) + +def test_deps_from_more_specialized_platforms_are_propagated(env): + got = deps( + "foo", + requires_dist = [ + "a_mac_dep; sys_platform=='darwin'", + "m1_dep; sys_platform=='darwin' and platform_machine=='arm64'", + ], + platforms = [ + "osx_x86_64", + "osx_aarch64", + ], + host_python_version = "3.8.4", + ) + + env.expect.that_collection(got.deps).contains_exactly([]) + env.expect.that_dict(got.deps_select).contains_exactly( + { + "@platforms//os:osx": ["a_mac_dep"], + "osx_aarch64": ["a_mac_dep", "m1_dep"], + }, + ) + +_tests.append(test_deps_from_more_specialized_platforms_are_propagated) + +def test_non_platform_markers_are_added_to_common_deps(env): + got = deps( + "foo", + requires_dist = [ + "bar", + "baz; implementation_name=='cpython'", + "m1_dep; sys_platform=='darwin' and platform_machine=='arm64'", + ], + platforms = [ + "linux_x86_64", + "osx_x86_64", + "osx_aarch64", + "windows_x86_64", + ], + host_python_version = "3.8.4", + ) + + env.expect.that_collection(got.deps).contains_exactly(["bar", "baz"]) + env.expect.that_dict(got.deps_select).contains_exactly({ + "osx_aarch64": ["m1_dep"], + }) + +_tests.append(test_non_platform_markers_are_added_to_common_deps) + +def test_self_is_ignored(env): + got = deps( + "foo", + requires_dist = [ + "bar", + "req_dep; extra == 'requests'", + "foo[requests]; extra == 'ssl'", + "ssl_lib; extra == 'ssl'", + ], + extras = ["ssl"], + ) + + env.expect.that_collection(got.deps).contains_exactly(["bar", "req_dep", "ssl_lib"]) + env.expect.that_dict(got.deps_select).contains_exactly({}) + +_tests.append(test_self_is_ignored) + +def test_self_dependencies_can_come_in_any_order(env): + got = deps( + "foo", + requires_dist = [ + "bar", + "baz; extra == 'feat'", + "foo[feat2]; extra == 'all'", + "foo[feat]; extra == 'feat2'", + "zdep; extra == 'all'", + ], + extras = ["all"], + ) + + env.expect.that_collection(got.deps).contains_exactly(["bar", "baz", "zdep"]) + env.expect.that_dict(got.deps_select).contains_exactly({}) + +_tests.append(test_self_dependencies_can_come_in_any_order) + +def _test_can_get_deps_based_on_specific_python_version(env): + requires_dist = [ + "bar", + "baz; python_version < '3.8'", + "posix_dep; os_name=='posix' and python_version >= '3.8'", + ] + + py38 = deps( + "foo", + requires_dist = requires_dist, + platforms = ["cp38_linux_x86_64"], + ) + py37 = deps( + "foo", + requires_dist = requires_dist, + platforms = ["cp37_linux_x86_64"], + ) + + env.expect.that_collection(py37.deps).contains_exactly(["bar", "baz"]) + env.expect.that_dict(py37.deps_select).contains_exactly({}) + env.expect.that_collection(py38.deps).contains_exactly(["bar"]) + env.expect.that_dict(py38.deps_select).contains_exactly({"@platforms//os:linux": ["posix_dep"]}) + +_tests.append(_test_can_get_deps_based_on_specific_python_version) + +def _test_no_version_select_when_single_version(env): + requires_dist = [ + "bar", + "baz; python_version >= '3.8'", + "posix_dep; os_name=='posix'", + "posix_dep_with_version; os_name=='posix' and python_version >= '3.8'", + "arch_dep; platform_machine=='x86_64' and python_version >= '3.8'", + ] + host_python_version = "3.7.5" + + got = deps( + "foo", + requires_dist = requires_dist, + platforms = [ + "cp38_linux_x86_64", + "cp38_windows_x86_64", + ], + host_python_version = host_python_version, + ) + + env.expect.that_collection(got.deps).contains_exactly(["bar", "baz"]) + env.expect.that_dict(got.deps_select).contains_exactly({ + "@platforms//os:linux": ["posix_dep", "posix_dep_with_version"], + "linux_x86_64": ["arch_dep", "posix_dep", "posix_dep_with_version"], + "windows_x86_64": ["arch_dep"], + }) + +_tests.append(_test_no_version_select_when_single_version) + +def _test_can_get_version_select(env): + requires_dist = [ + "bar", + "baz; python_version < '3.8'", + "baz_new; python_version >= '3.8'", + "posix_dep; os_name=='posix'", + "posix_dep_with_version; os_name=='posix' and python_version >= '3.8'", + "arch_dep; platform_machine=='x86_64' and python_version < '3.8'", + ] + host_python_version = "3.7.4" + + got = deps( + "foo", + requires_dist = requires_dist, + platforms = [ + "cp3{}_{}_x86_64".format(minor, os) + for minor in [7, 8, 9] + for os in ["linux", "windows"] + ], + host_python_version = host_python_version, + ) + + env.expect.that_collection(got.deps).contains_exactly(["bar"]) + env.expect.that_dict(got.deps_select).contains_exactly({ + str(Label("//python/config_settings:is_python_3.7")): ["baz"], + str(Label("//python/config_settings:is_python_3.8")): ["baz_new"], + str(Label("//python/config_settings:is_python_3.9")): ["baz_new"], + "@platforms//os:linux": ["baz", "posix_dep"], + "cp37_linux_anyarch": ["baz", "posix_dep"], + "cp37_linux_x86_64": ["arch_dep", "baz", "posix_dep"], + "cp37_windows_x86_64": ["arch_dep", "baz"], + "cp38_linux_anyarch": [ + "baz_new", + "posix_dep", + "posix_dep_with_version", + ], + "cp39_linux_anyarch": [ + "baz_new", + "posix_dep", + "posix_dep_with_version", + ], + "linux_x86_64": ["arch_dep", "baz", "posix_dep"], + "windows_x86_64": ["arch_dep", "baz"], + "//conditions:default": ["baz"], + }) + +_tests.append(_test_can_get_version_select) + +def _test_deps_spanning_all_target_py_versions_are_added_to_common(env): + requires_dist = [ + "bar", + "baz (<2,>=1.11) ; python_version < '3.8'", + "baz (<2,>=1.14) ; python_version >= '3.8'", + ] + host_python_version = "3.8.4" + + got = deps( + "foo", + requires_dist = requires_dist, + platforms = [ + "cp3{}_linux_x86_64".format(minor) + for minor in [7, 8, 9] + ], + host_python_version = host_python_version, + ) + + env.expect.that_collection(got.deps).contains_exactly(["bar", "baz"]) + env.expect.that_dict(got.deps_select).contains_exactly({}) + +_tests.append(_test_deps_spanning_all_target_py_versions_are_added_to_common) + +def _test_deps_are_not_duplicated(env): + host_python_version = "3.7.4" + + # See an example in + # https://files.pythonhosted.org/packages/76/9e/db1c2d56c04b97981c06663384f45f28950a73d9acf840c4006d60d0a1ff/opencv_python-4.9.0.80-cp37-abi3-win32.whl.metadata + requires_dist = [ + "bar >=0.1.0 ; python_version < '3.7'", + "bar >=0.2.0 ; python_version >= '3.7'", + "bar >=0.4.0 ; python_version >= '3.6' and platform_system == 'Linux' and platform_machine == 'aarch64'", + "bar >=0.4.0 ; python_version >= '3.9'", + "bar >=0.5.0 ; python_version <= '3.9' and platform_system == 'Darwin' and platform_machine == 'arm64'", + "bar >=0.5.0 ; python_version >= '3.10' and platform_system == 'Darwin'", + "bar >=0.5.0 ; python_version >= '3.10'", + "bar >=0.6.0 ; python_version >= '3.11'", + ] + + got = deps( + "foo", + requires_dist = requires_dist, + platforms = [ + "cp3{}_{}_{}".format(minor, os, arch) + for minor in [7, 10] + for os in ["linux", "osx", "windows"] + for arch in ["x86_64", "aarch64"] + ], + host_python_version = host_python_version, + ) + + env.expect.that_collection(got.deps).contains_exactly(["bar"]) + env.expect.that_dict(got.deps_select).contains_exactly({}) + +_tests.append(_test_deps_are_not_duplicated) + +def _test_deps_are_not_duplicated_when_encountering_platform_dep_first(env): + host_python_version = "3.7.1" + + # Note, that we are sorting the incoming `requires_dist` and we need to ensure that we are not getting any + # issues even if the platform-specific line comes first. + requires_dist = [ + "bar >=0.4.0 ; python_version >= '3.6' and platform_system == 'Linux' and platform_machine == 'aarch64'", + "bar >=0.5.0 ; python_version >= '3.9'", + ] + + got = deps( + "foo", + requires_dist = requires_dist, + platforms = [ + "cp37_linux_aarch64", + "cp37_linux_x86_64", + "cp310_linux_aarch64", + "cp310_linux_x86_64", + ], + host_python_version = host_python_version, + ) + + # TODO @aignas 2025-02-24: this test case in the python version is passing but + # I am not sure why. The starlark version behaviour looks more correct. + env.expect.that_collection(got.deps).contains_exactly([]) + env.expect.that_dict(got.deps_select).contains_exactly({ + str(Label("//python/config_settings:is_python_3.10")): ["bar"], + "cp310_linux_aarch64": ["bar"], + "cp37_linux_aarch64": ["bar"], + "linux_aarch64": ["bar"], + }) + +_tests.append(_test_deps_are_not_duplicated_when_encountering_platform_dep_first) + +def deps_test_suite(name): # buildifier: disable=function-docstring + test_suite( + name = name, + basic_tests = _tests, + ) diff --git a/tests/pypi/pep508/evaluate_tests.bzl b/tests/pypi/pep508/evaluate_tests.bzl index 80b70f4dad..14e5e40b43 100644 --- a/tests/pypi/pep508/evaluate_tests.bzl +++ b/tests/pypi/pep508/evaluate_tests.bzl @@ -148,6 +148,8 @@ def _logical_expression_tests(env): # expr "os_name == 'fo'": False, "(os_name == 'fo')": False, + "((os_name == 'fo'))": False, + "((os_name == 'foo'))": True, "not (os_name == 'fo')": True, # and diff --git a/tests/pypi/whl_installer/BUILD.bazel b/tests/pypi/whl_installer/BUILD.bazel index 040e4d765f..fea6a46d01 100644 --- a/tests/pypi/whl_installer/BUILD.bazel +++ b/tests/pypi/whl_installer/BUILD.bazel @@ -27,18 +27,6 @@ py_test( ], ) -py_test( - name = "platform_test", - size = "small", - srcs = [ - "platform_test.py", - ], - data = ["//examples/wheel:minimal_with_py_package"], - deps = [ - ":lib", - ], -) - py_test( name = "wheel_installer_test", size = "small", @@ -50,15 +38,3 @@ py_test( ":lib", ], ) - -py_test( - name = "wheel_test", - size = "small", - srcs = [ - "wheel_test.py", - ], - data = ["//examples/wheel:minimal_with_py_package"], - deps = [ - ":lib", - ], -) diff --git a/tests/pypi/whl_installer/arguments_test.py b/tests/pypi/whl_installer/arguments_test.py index 5538054a59..9f73ae96a9 100644 --- a/tests/pypi/whl_installer/arguments_test.py +++ b/tests/pypi/whl_installer/arguments_test.py @@ -15,7 +15,7 @@ import json import unittest -from python.private.pypi.whl_installer import arguments, wheel +from python.private.pypi.whl_installer import arguments class ArgumentsTestCase(unittest.TestCase): @@ -49,18 +49,6 @@ def test_deserialize_structured_args(self) -> None: self.assertEqual(args["environment"], {"PIP_DO_SOMETHING": "True"}) self.assertEqual(args["extra_pip_args"], []) - def test_platform_aggregation(self) -> None: - parser = arguments.parser() - args = parser.parse_args( - args=[ - "--platform=linux_*", - "--platform=osx_*", - "--platform=windows_*", - "--requirement=foo", - ] - ) - self.assertEqual(set(wheel.Platform.all()), arguments.get_platforms(args)) - if __name__ == "__main__": unittest.main() diff --git a/tests/pypi/whl_installer/platform_test.py b/tests/pypi/whl_installer/platform_test.py deleted file mode 100644 index 2aeb4caa69..0000000000 --- a/tests/pypi/whl_installer/platform_test.py +++ /dev/null @@ -1,154 +0,0 @@ -import unittest -from random import shuffle - -from python.private.pypi.whl_installer.platform import ( - OS, - Arch, - Platform, - host_interpreter_minor_version, -) - - -class MinorVersionTest(unittest.TestCase): - def test_host(self): - host = host_interpreter_minor_version() - self.assertIsNotNone(host) - - -class PlatformTest(unittest.TestCase): - def test_can_get_host(self): - host = Platform.host() - self.assertIsNotNone(host) - self.assertEqual(1, len(Platform.from_string("host"))) - self.assertEqual(host, Platform.from_string("host")) - - def test_can_get_linux_x86_64_without_py_version(self): - got = Platform.from_string("linux_x86_64") - want = Platform(os=OS.linux, arch=Arch.x86_64) - self.assertEqual(want, got[0]) - - def test_can_get_specific_from_string(self): - got = Platform.from_string("cp33_linux_x86_64") - want = Platform(os=OS.linux, arch=Arch.x86_64, minor_version=3) - self.assertEqual(want, got[0]) - - def test_can_get_all_for_py_version(self): - cp39 = Platform.all(minor_version=9) - self.assertEqual(21, len(cp39), f"Got {cp39}") - self.assertEqual(cp39, Platform.from_string("cp39_*")) - - def test_can_get_all_for_os(self): - linuxes = Platform.all(OS.linux, minor_version=9) - self.assertEqual(7, len(linuxes)) - self.assertEqual(linuxes, Platform.from_string("cp39_linux_*")) - - def test_can_get_all_for_os_for_host_python(self): - linuxes = Platform.all(OS.linux) - self.assertEqual(7, len(linuxes)) - self.assertEqual(linuxes, Platform.from_string("linux_*")) - - def test_specific_version_specializations(self): - any_py33 = Platform(minor_version=3) - - # When - all_specializations = list(any_py33.all_specializations()) - - want = ( - [any_py33] - + [ - Platform(arch=arch, minor_version=any_py33.minor_version) - for arch in Arch - ] - + [Platform(os=os, minor_version=any_py33.minor_version) for os in OS] - + Platform.all(minor_version=any_py33.minor_version) - ) - self.assertEqual(want, all_specializations) - - def test_aarch64_specializations(self): - any_aarch64 = Platform(arch=Arch.aarch64) - all_specializations = list(any_aarch64.all_specializations()) - want = [ - Platform(os=None, arch=Arch.aarch64), - Platform(os=OS.linux, arch=Arch.aarch64), - Platform(os=OS.osx, arch=Arch.aarch64), - Platform(os=OS.windows, arch=Arch.aarch64), - ] - self.assertEqual(want, all_specializations) - - def test_linux_specializations(self): - any_linux = Platform(os=OS.linux) - all_specializations = list(any_linux.all_specializations()) - want = [ - Platform(os=OS.linux, arch=None), - Platform(os=OS.linux, arch=Arch.x86_64), - Platform(os=OS.linux, arch=Arch.x86_32), - Platform(os=OS.linux, arch=Arch.aarch64), - Platform(os=OS.linux, arch=Arch.ppc), - Platform(os=OS.linux, arch=Arch.ppc64le), - Platform(os=OS.linux, arch=Arch.s390x), - Platform(os=OS.linux, arch=Arch.arm), - ] - self.assertEqual(want, all_specializations) - - def test_osx_specializations(self): - any_osx = Platform(os=OS.osx) - all_specializations = list(any_osx.all_specializations()) - # NOTE @aignas 2024-01-14: even though in practice we would only have - # Python on osx aarch64 and osx x86_64, we return all arch posibilities - # to make the code simpler. - want = [ - Platform(os=OS.osx, arch=None), - Platform(os=OS.osx, arch=Arch.x86_64), - Platform(os=OS.osx, arch=Arch.x86_32), - Platform(os=OS.osx, arch=Arch.aarch64), - Platform(os=OS.osx, arch=Arch.ppc), - Platform(os=OS.osx, arch=Arch.ppc64le), - Platform(os=OS.osx, arch=Arch.s390x), - Platform(os=OS.osx, arch=Arch.arm), - ] - self.assertEqual(want, all_specializations) - - def test_platform_sort(self): - platforms = [ - Platform(os=OS.linux, arch=None), - Platform(os=OS.linux, arch=Arch.x86_64), - Platform(os=OS.osx, arch=None), - Platform(os=OS.osx, arch=Arch.x86_64), - Platform(os=OS.osx, arch=Arch.aarch64), - ] - shuffle(platforms) - platforms.sort() - want = [ - Platform(os=OS.linux, arch=None), - Platform(os=OS.linux, arch=Arch.x86_64), - Platform(os=OS.osx, arch=None), - Platform(os=OS.osx, arch=Arch.x86_64), - Platform(os=OS.osx, arch=Arch.aarch64), - ] - - self.assertEqual(want, platforms) - - def test_wheel_os_alias(self): - self.assertEqual("osx", str(OS.osx)) - self.assertEqual(str(OS.darwin), str(OS.osx)) - - def test_wheel_arch_alias(self): - self.assertEqual("x86_64", str(Arch.x86_64)) - self.assertEqual(str(Arch.amd64), str(Arch.x86_64)) - - def test_wheel_platform_alias(self): - give = Platform( - os=OS.darwin, - arch=Arch.amd64, - ) - alias = Platform( - os=OS.osx, - arch=Arch.x86_64, - ) - - self.assertEqual("osx_x86_64", str(give)) - self.assertEqual(str(alias), str(give)) - - -if __name__ == "__main__": - unittest.main() diff --git a/tests/pypi/whl_installer/wheel_installer_test.py b/tests/pypi/whl_installer/wheel_installer_test.py index 7139779c3e..3c118af3c4 100644 --- a/tests/pypi/whl_installer/wheel_installer_test.py +++ b/tests/pypi/whl_installer/wheel_installer_test.py @@ -22,39 +22,6 @@ from python.private.pypi.whl_installer import wheel_installer -class TestRequirementExtrasParsing(unittest.TestCase): - def test_parses_requirement_for_extra(self) -> None: - cases = [ - ("name[foo]", ("name", frozenset(["foo"]))), - ("name[ Foo123 ]", ("name", frozenset(["Foo123"]))), - (" name1[ foo ] ", ("name1", frozenset(["foo"]))), - ("Name[foo]", ("name", frozenset(["foo"]))), - ("name_foo[bar]", ("name-foo", frozenset(["bar"]))), - ( - "name [fred,bar] @ http://foo.com ; python_version=='2.7'", - ("name", frozenset(["fred", "bar"])), - ), - ( - "name[quux, strange];python_version<'2.7' and platform_version=='2'", - ("name", frozenset(["quux", "strange"])), - ), - ( - "name; (os_name=='a' or os_name=='b') and os_name=='c'", - (None, None), - ), - ( - "name@http://foo.com", - (None, None), - ), - ] - - for case, expected in cases: - with self.subTest(): - self.assertTupleEqual( - wheel_installer._parse_requirement_for_extra(case), expected - ) - - class TestWhlFilegroup(unittest.TestCase): def setUp(self) -> None: self.wheel_name = "example_minimal_package-0.0.1-py3-none-any.whl" @@ -68,10 +35,8 @@ def tearDown(self): def test_wheel_exists(self) -> None: wheel_installer._extract_wheel( Path(self.wheel_path), - installation_dir=Path(self.wheel_dir), - extras={}, enable_implicit_namespace_pkgs=False, - platforms=[], + installation_dir=Path(self.wheel_dir), ) want_files = [ @@ -92,11 +57,8 @@ def test_wheel_exists(self) -> None: metadata_file_content = json.load(metadata_file) want = dict( - version="0.0.1", - name="example-minimal-package", - deps=[], - deps_by_platform={}, entry_points=[], + python_version="3.11.11", ) self.assertEqual(want, metadata_file_content) diff --git a/tests/pypi/whl_installer/wheel_test.py b/tests/pypi/whl_installer/wheel_test.py deleted file mode 100644 index 404218e12b..0000000000 --- a/tests/pypi/whl_installer/wheel_test.py +++ /dev/null @@ -1,371 +0,0 @@ -import unittest -from unittest import mock - -from python.private.pypi.whl_installer import wheel -from python.private.pypi.whl_installer.platform import OS, Arch, Platform - -_HOST_INTERPRETER_FN = ( - "python.private.pypi.whl_installer.wheel.host_interpreter_minor_version" -) - - -class DepsTest(unittest.TestCase): - def test_simple(self): - deps = wheel.Deps("foo", requires_dist=["bar"]) - - got = deps.build() - - self.assertIsInstance(got, wheel.FrozenDeps) - self.assertEqual(["bar"], got.deps) - self.assertEqual({}, got.deps_select) - - def test_can_add_os_specific_deps(self): - deps = wheel.Deps( - "foo", - requires_dist=[ - "bar", - "an_osx_dep; sys_platform=='darwin'", - "posix_dep; os_name=='posix'", - "win_dep; os_name=='nt'", - ], - platforms={ - Platform(os=OS.linux, arch=Arch.x86_64), - Platform(os=OS.osx, arch=Arch.x86_64), - Platform(os=OS.osx, arch=Arch.aarch64), - Platform(os=OS.windows, arch=Arch.x86_64), - }, - ) - - got = deps.build() - - self.assertEqual(["bar"], got.deps) - self.assertEqual( - { - "@platforms//os:linux": ["posix_dep"], - "@platforms//os:osx": ["an_osx_dep", "posix_dep"], - "@platforms//os:windows": ["win_dep"], - }, - got.deps_select, - ) - - def test_can_add_os_specific_deps_with_specific_python_version(self): - deps = wheel.Deps( - "foo", - requires_dist=[ - "bar", - "an_osx_dep; sys_platform=='darwin'", - "posix_dep; os_name=='posix'", - "win_dep; os_name=='nt'", - ], - platforms={ - Platform(os=OS.linux, arch=Arch.x86_64, minor_version=8), - Platform(os=OS.osx, arch=Arch.x86_64, minor_version=8), - Platform(os=OS.osx, arch=Arch.aarch64, minor_version=8), - Platform(os=OS.windows, arch=Arch.x86_64, minor_version=8), - }, - ) - - got = deps.build() - - self.assertEqual(["bar"], got.deps) - self.assertEqual( - { - "@platforms//os:linux": ["posix_dep"], - "@platforms//os:osx": ["an_osx_dep", "posix_dep"], - "@platforms//os:windows": ["win_dep"], - }, - got.deps_select, - ) - - def test_deps_are_added_to_more_specialized_platforms(self): - got = wheel.Deps( - "foo", - requires_dist=[ - "m1_dep; sys_platform=='darwin' and platform_machine=='arm64'", - "mac_dep; sys_platform=='darwin'", - ], - platforms={ - Platform(os=OS.osx, arch=Arch.x86_64), - Platform(os=OS.osx, arch=Arch.aarch64), - }, - ).build() - - self.assertEqual( - wheel.FrozenDeps( - deps=[], - deps_select={ - "osx_aarch64": ["m1_dep", "mac_dep"], - "@platforms//os:osx": ["mac_dep"], - }, - ), - got, - ) - - def test_deps_from_more_specialized_platforms_are_propagated(self): - got = wheel.Deps( - "foo", - requires_dist=[ - "a_mac_dep; sys_platform=='darwin'", - "m1_dep; sys_platform=='darwin' and platform_machine=='arm64'", - ], - platforms={ - Platform(os=OS.osx, arch=Arch.x86_64), - Platform(os=OS.osx, arch=Arch.aarch64), - }, - ).build() - - self.assertEqual([], got.deps) - self.assertEqual( - { - "osx_aarch64": ["a_mac_dep", "m1_dep"], - "@platforms//os:osx": ["a_mac_dep"], - }, - got.deps_select, - ) - - def test_non_platform_markers_are_added_to_common_deps(self): - got = wheel.Deps( - "foo", - requires_dist=[ - "bar", - "baz; implementation_name=='cpython'", - "m1_dep; sys_platform=='darwin' and platform_machine=='arm64'", - ], - platforms={ - Platform(os=OS.linux, arch=Arch.x86_64), - Platform(os=OS.osx, arch=Arch.x86_64), - Platform(os=OS.osx, arch=Arch.aarch64), - Platform(os=OS.windows, arch=Arch.x86_64), - }, - ).build() - - self.assertEqual(["bar", "baz"], got.deps) - self.assertEqual( - { - "osx_aarch64": ["m1_dep"], - }, - got.deps_select, - ) - - def test_self_is_ignored(self): - deps = wheel.Deps( - "foo", - requires_dist=[ - "bar", - "req_dep; extra == 'requests'", - "foo[requests]; extra == 'ssl'", - "ssl_lib; extra == 'ssl'", - ], - extras={"ssl"}, - ) - - got = deps.build() - - self.assertEqual(["bar", "req_dep", "ssl_lib"], got.deps) - self.assertEqual({}, got.deps_select) - - def test_self_dependencies_can_come_in_any_order(self): - deps = wheel.Deps( - "foo", - requires_dist=[ - "bar", - "baz; extra == 'feat'", - "foo[feat2]; extra == 'all'", - "foo[feat]; extra == 'feat2'", - "zdep; extra == 'all'", - ], - extras={"all"}, - ) - - got = deps.build() - - self.assertEqual(["bar", "baz", "zdep"], got.deps) - self.assertEqual({}, got.deps_select) - - def test_can_get_deps_based_on_specific_python_version(self): - requires_dist = [ - "bar", - "baz; python_version < '3.8'", - "posix_dep; os_name=='posix' and python_version >= '3.8'", - ] - - py38_deps = wheel.Deps( - "foo", - requires_dist=requires_dist, - platforms=[ - Platform(os=OS.linux, arch=Arch.x86_64, minor_version=8), - ], - ).build() - py37_deps = wheel.Deps( - "foo", - requires_dist=requires_dist, - platforms=[ - Platform(os=OS.linux, arch=Arch.x86_64, minor_version=7), - ], - ).build() - - self.assertEqual(["bar", "baz"], py37_deps.deps) - self.assertEqual({}, py37_deps.deps_select) - self.assertEqual(["bar"], py38_deps.deps) - self.assertEqual({"@platforms//os:linux": ["posix_dep"]}, py38_deps.deps_select) - - @mock.patch(_HOST_INTERPRETER_FN) - def test_no_version_select_when_single_version(self, mock_host_interpreter_version): - requires_dist = [ - "bar", - "baz; python_version >= '3.8'", - "posix_dep; os_name=='posix'", - "posix_dep_with_version; os_name=='posix' and python_version >= '3.8'", - "arch_dep; platform_machine=='x86_64' and python_version >= '3.8'", - ] - mock_host_interpreter_version.return_value = 7 - - self.maxDiff = None - - deps = wheel.Deps( - "foo", - requires_dist=requires_dist, - platforms=[ - Platform(os=os, arch=Arch.x86_64, minor_version=minor) - for minor in [8] - for os in [OS.linux, OS.windows] - ], - ) - got = deps.build() - - self.assertEqual(["bar", "baz"], got.deps) - self.assertEqual( - { - "@platforms//os:linux": ["posix_dep", "posix_dep_with_version"], - "linux_x86_64": ["arch_dep", "posix_dep", "posix_dep_with_version"], - "windows_x86_64": ["arch_dep"], - }, - got.deps_select, - ) - - @mock.patch(_HOST_INTERPRETER_FN) - def test_can_get_version_select(self, mock_host_interpreter_version): - requires_dist = [ - "bar", - "baz; python_version < '3.8'", - "baz_new; python_version >= '3.8'", - "posix_dep; os_name=='posix'", - "posix_dep_with_version; os_name=='posix' and python_version >= '3.8'", - "arch_dep; platform_machine=='x86_64' and python_version < '3.8'", - ] - mock_host_interpreter_version.return_value = 7 - - self.maxDiff = None - - deps = wheel.Deps( - "foo", - requires_dist=requires_dist, - platforms=[ - Platform(os=os, arch=Arch.x86_64, minor_version=minor) - for minor in [7, 8, 9] - for os in [OS.linux, OS.windows] - ], - ) - got = deps.build() - - self.assertEqual(["bar"], got.deps) - self.assertEqual( - { - "//conditions:default": ["baz"], - "@//python/config_settings:is_python_3.7": ["baz"], - "@//python/config_settings:is_python_3.8": ["baz_new"], - "@//python/config_settings:is_python_3.9": ["baz_new"], - "@platforms//os:linux": ["baz", "posix_dep"], - "cp37_linux_x86_64": ["arch_dep", "baz", "posix_dep"], - "cp37_windows_x86_64": ["arch_dep", "baz"], - "cp37_linux_anyarch": ["baz", "posix_dep"], - "cp38_linux_anyarch": [ - "baz_new", - "posix_dep", - "posix_dep_with_version", - ], - "cp39_linux_anyarch": [ - "baz_new", - "posix_dep", - "posix_dep_with_version", - ], - "linux_x86_64": ["arch_dep", "baz", "posix_dep"], - "windows_x86_64": ["arch_dep", "baz"], - }, - got.deps_select, - ) - - @mock.patch(_HOST_INTERPRETER_FN) - def test_deps_spanning_all_target_py_versions_are_added_to_common( - self, mock_host_version - ): - requires_dist = [ - "bar", - "baz (<2,>=1.11) ; python_version < '3.8'", - "baz (<2,>=1.14) ; python_version >= '3.8'", - ] - mock_host_version.return_value = 8 - - deps = wheel.Deps( - "foo", - requires_dist=requires_dist, - platforms=Platform.from_string(["cp37_*", "cp38_*", "cp39_*"]), - ) - got = deps.build() - - self.assertEqual(["bar", "baz"], got.deps) - self.assertEqual({}, got.deps_select) - - @mock.patch(_HOST_INTERPRETER_FN) - def test_deps_are_not_duplicated(self, mock_host_version): - mock_host_version.return_value = 7 - - # See an example in - # https://files.pythonhosted.org/packages/76/9e/db1c2d56c04b97981c06663384f45f28950a73d9acf840c4006d60d0a1ff/opencv_python-4.9.0.80-cp37-abi3-win32.whl.metadata - requires_dist = [ - "bar >=0.1.0 ; python_version < '3.7'", - "bar >=0.2.0 ; python_version >= '3.7'", - "bar >=0.4.0 ; python_version >= '3.6' and platform_system == 'Linux' and platform_machine == 'aarch64'", - "bar >=0.4.0 ; python_version >= '3.9'", - "bar >=0.5.0 ; python_version <= '3.9' and platform_system == 'Darwin' and platform_machine == 'arm64'", - "bar >=0.5.0 ; python_version >= '3.10' and platform_system == 'Darwin'", - "bar >=0.5.0 ; python_version >= '3.10'", - "bar >=0.6.0 ; python_version >= '3.11'", - ] - - deps = wheel.Deps( - "foo", - requires_dist=requires_dist, - platforms=Platform.from_string(["cp37_*", "cp310_*"]), - ) - got = deps.build() - - self.assertEqual(["bar"], got.deps) - self.assertEqual({}, got.deps_select) - - @mock.patch(_HOST_INTERPRETER_FN) - def test_deps_are_not_duplicated_when_encountering_platform_dep_first( - self, mock_host_version - ): - mock_host_version.return_value = 7 - - # Note, that we are sorting the incoming `requires_dist` and we need to ensure that we are not getting any - # issues even if the platform-specific line comes first. - requires_dist = [ - "bar >=0.4.0 ; python_version >= '3.6' and platform_system == 'Linux' and platform_machine == 'aarch64'", - "bar >=0.5.0 ; python_version >= '3.9'", - ] - - deps = wheel.Deps( - "foo", - requires_dist=requires_dist, - platforms=Platform.from_string(["cp37_*", "cp310_*"]), - ) - got = deps.build() - - self.assertEqual(["bar"], got.deps) - self.assertEqual({}, got.deps_select) - - -if __name__ == "__main__": - unittest.main() diff --git a/tests/pypi/whl_metadata/BUILD.bazel b/tests/pypi/whl_metadata/BUILD.bazel new file mode 100644 index 0000000000..3f1d665dd2 --- /dev/null +++ b/tests/pypi/whl_metadata/BUILD.bazel @@ -0,0 +1,5 @@ +load(":whl_metadata_tests.bzl", "whl_metadata_test_suite") + +whl_metadata_test_suite( + name = "whl_metadata_tests", +) diff --git a/tests/pypi/whl_metadata/whl_metadata_tests.bzl b/tests/pypi/whl_metadata/whl_metadata_tests.bzl new file mode 100644 index 0000000000..4acbc9213d --- /dev/null +++ b/tests/pypi/whl_metadata/whl_metadata_tests.bzl @@ -0,0 +1,147 @@ +"" + +load("@rules_testing//lib:test_suite.bzl", "test_suite") +load("@rules_testing//lib:truth.bzl", "subjects") +load( + "//python/private/pypi:whl_metadata.bzl", + "find_whl_metadata", + "parse_whl_metadata", +) # buildifier: disable=bzl-visibility + +_tests = [] + +def _test_empty(env): + fake_path = struct( + basename = "site-packages", + readdir = lambda watch = None: [], + ) + fail_messages = [] + find_whl_metadata(install_dir = fake_path, logger = struct( + fail = fail_messages.append, + )) + env.expect.that_collection(fail_messages).contains_exactly([ + "The '*.dist-info' directory could not be found in 'site-packages'", + ]) + +_tests.append(_test_empty) + +def _test_contains_dist_info_but_no_metadata(env): + fake_path = struct( + basename = "site-packages", + readdir = lambda watch = None: [ + struct( + basename = "something.dist-info", + is_dir = True, + get_child = lambda basename: struct( + basename = basename, + exists = False, + ), + ), + ], + ) + fail_messages = [] + find_whl_metadata(install_dir = fake_path, logger = struct( + fail = fail_messages.append, + )) + env.expect.that_collection(fail_messages).contains_exactly([ + "The METADATA file for the wheel could not be found in 'site-packages/something.dist-info'", + ]) + +_tests.append(_test_contains_dist_info_but_no_metadata) + +def _test_contains_metadata(env): + fake_path = struct( + basename = "site-packages", + readdir = lambda watch = None: [ + struct( + basename = "something.dist-info", + is_dir = True, + get_child = lambda basename: struct( + basename = basename, + exists = True, + ), + ), + ], + ) + fail_messages = [] + got = find_whl_metadata(install_dir = fake_path, logger = struct( + fail = fail_messages.append, + )) + env.expect.that_collection(fail_messages).contains_exactly([]) + env.expect.that_str(got.basename).equals("METADATA") + +_tests.append(_test_contains_metadata) + +def _parse_whl_metadata(env, **kwargs): + result = parse_whl_metadata(**kwargs) + + return env.expect.that_struct( + struct( + name = result.name, + version = result.version, + requires_dist = result.requires_dist, + provides_extra = result.provides_extra, + ), + attrs = dict( + name = subjects.str, + version = subjects.str, + requires_dist = subjects.collection, + provides_extra = subjects.collection, + ), + ) + +def _test_parse_metadata_invalid(env): + got = _parse_whl_metadata( + env, + contents = "", + ) + got.name().equals("") + got.version().equals("") + got.requires_dist().contains_exactly([]) + got.provides_extra().contains_exactly([]) + +_tests.append(_test_parse_metadata_invalid) + +def _test_parse_metadata_basic(env): + got = _parse_whl_metadata( + env, + contents = """\ +Name: foo +Version: 0.0.1 +""", + ) + got.name().equals("foo") + got.version().equals("0.0.1") + got.requires_dist().contains_exactly([]) + got.provides_extra().contains_exactly([]) + +_tests.append(_test_parse_metadata_basic) + +def _test_parse_metadata_all(env): + got = _parse_whl_metadata( + env, + contents = """\ +Name: foo +Version: 0.0.1 +Requires-Dist: bar; extra == "all" +Provides-Extra: all + +Requires-Dist: this will be ignored +""", + ) + got.name().equals("foo") + got.version().equals("0.0.1") + got.requires_dist().contains_exactly([ + "bar; extra == \"all\"", + ]) + got.provides_extra().contains_exactly([ + "all", + ]) + +_tests.append(_test_parse_metadata_all) + +def whl_metadata_test_suite(name): # buildifier: disable=function-docstring + test_suite( + name = name, + basic_tests = _tests, + ) From 79abef898ece1a6ae2af8cb855418ac342dd27d8 Mon Sep 17 00:00:00 2001 From: Ivo List Date: Tue, 15 Apr 2025 04:21:33 +0200 Subject: [PATCH 164/922] fix: replace string with modern providers in tests (#2773) Strings used to refer to legacy struct providers, which were removed from Bazel. Legacy struct providers have been deprecated by Bazel. Replacing them with modern providers, will make it possible to simplify and remove legacy handling from Blaze. The change is a no-op. More information: https://github.com/bazelbuild/bazel/issues/25836 --- tests/builders/attr_builders_tests.bzl | 17 +++++++++-------- 1 file changed, 9 insertions(+), 8 deletions(-) diff --git a/tests/builders/attr_builders_tests.bzl b/tests/builders/attr_builders_tests.bzl index 58557cd633..e92ba2ae0a 100644 --- a/tests/builders/attr_builders_tests.bzl +++ b/tests/builders/attr_builders_tests.bzl @@ -28,6 +28,7 @@ def _expect_cfg_defaults(expect, cfg): expect.where(expr = "cfg.which_cfg").that_str(cfg.which_cfg()).equals("target") _some_aspect = aspect(implementation = lambda target, ctx: None) +_SomeInfo = provider("MyInfo", fields = []) _tests = [] @@ -186,7 +187,7 @@ def _test_label(name): subject.set_executable(True) subject.add_allow_files(".txt") subject.cfg.set_target() - subject.providers().append("provider") + subject.providers().append(_SomeInfo) subject.aspects().append(_some_aspect) subject.cfg.outputs().append(Label("//some:output")) subject.cfg.inputs().append(Label("//some:input")) @@ -199,7 +200,7 @@ def _test_label(name): expect.that_bool(subject.executable()).equals(True) expect.that_collection(subject.allow_files()).contains_exactly([".txt"]) expect.that_bool(subject.allow_single_file()).equals(None) - expect.that_collection(subject.providers()).contains_exactly(["provider"]) + expect.that_collection(subject.providers()).contains_exactly([_SomeInfo]) expect.that_collection(subject.aspects()).contains_exactly([_some_aspect]) expect.that_collection(subject.cfg.outputs()).contains_exactly([Label("//some:output")]) expect.that_collection(subject.cfg.inputs()).contains_exactly([Label("//some:input")]) @@ -229,7 +230,7 @@ def _test_label_keyed_string_dict(name): subject.set_mandatory(True) subject.set_allow_files(True) subject.cfg.set_target() - subject.providers().append("provider") + subject.providers().append(_SomeInfo) subject.aspects().append(_some_aspect) subject.cfg.outputs().append("//some:output") subject.cfg.inputs().append("//some:input") @@ -240,7 +241,7 @@ def _test_label_keyed_string_dict(name): expect.that_str(subject.doc()).equals("doc") expect.that_bool(subject.mandatory()).equals(True) expect.that_bool(subject.allow_files()).equals(True) - expect.that_collection(subject.providers()).contains_exactly(["provider"]) + expect.that_collection(subject.providers()).contains_exactly([_SomeInfo]) expect.that_collection(subject.aspects()).contains_exactly([_some_aspect]) expect.that_collection(subject.cfg.outputs()).contains_exactly(["//some:output"]) expect.that_collection(subject.cfg.inputs()).contains_exactly(["//some:input"]) @@ -274,14 +275,14 @@ def _test_label_list(name): subject.set_doc("doc") subject.set_mandatory(True) subject.set_allow_files([".txt"]) - subject.providers().append("provider") + subject.providers().append(_SomeInfo) subject.aspects().append(_some_aspect) expect.that_collection(subject.default()).contains_exactly(["//some:label"]) expect.that_str(subject.doc()).equals("doc") expect.that_bool(subject.mandatory()).equals(True) expect.that_collection(subject.allow_files()).contains_exactly([".txt"]) - expect.that_collection(subject.providers()).contains_exactly(["provider"]) + expect.that_collection(subject.providers()).contains_exactly([_SomeInfo]) expect.that_collection(subject.aspects()).contains_exactly([_some_aspect]) _expect_builds(expect, subject, "attr.label_list") @@ -395,14 +396,14 @@ def _test_string_keyed_label_dict(name): subject.set_doc("doc") subject.set_mandatory(True) subject.set_allow_files([".txt"]) - subject.providers().append("provider") + subject.providers().append(_SomeInfo) subject.aspects().append(_some_aspect) expect.that_dict(subject.default()).contains_exactly({"key": "//some:label"}) expect.that_str(subject.doc()).equals("doc") expect.that_bool(subject.mandatory()).equals(True) expect.that_collection(subject.allow_files()).contains_exactly([".txt"]) - expect.that_collection(subject.providers()).contains_exactly(["provider"]) + expect.that_collection(subject.providers()).contains_exactly([_SomeInfo]) expect.that_collection(subject.aspects()).contains_exactly([_some_aspect]) _expect_builds(expect, subject, "attr.string_keyed_label_dict") From a0400e9a832d554de032fe44d8b8375ceaa32db8 Mon Sep 17 00:00:00 2001 From: Frank Portman Date: Tue, 15 Apr 2025 04:37:01 -0400 Subject: [PATCH 165/922] feat(toolchain): Add new make vars for Python interpreter path compliant with `--no_legacy_external_runfiles` (#2772) Using these new make vars in `py_binary` or `py_test` will correctly find the interpreter when setting `--no_legacy_external_runfiles`. Fixes #2728 --- CHANGELOG.md | 2 ++ docs/toolchains.md | 6 +++++- python/current_py_toolchain.bzl | 7 +++++++ 3 files changed, 14 insertions(+), 1 deletion(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 33d99dfaa1..6f86851bdf 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -124,6 +124,8 @@ Unreleased changes template. * (toolchains) Local Python installs can be used to create a toolchain equivalent to the standard toolchains. See [Local toolchains] docs for how to configure them. +* (toolchains) Expose `$(PYTHON2_ROOTPATH)` and `$(PYTHON3_ROOTPATH)` which are runfiles + locations equivalents of `$(PYTHON2)` and `$(PYTHON3) respectively. {#v0-0-0-removed} diff --git a/docs/toolchains.md b/docs/toolchains.md index 5cd9eb268e..320e16335b 100644 --- a/docs/toolchains.md +++ b/docs/toolchains.md @@ -215,7 +215,11 @@ attribute. You can obtain the path to the Python interpreter using the `$(PYTHON2)` and `$(PYTHON3)` ["Make" Variables](https://bazel.build/reference/be/make-variables). See the {gh-path}`test_current_py_toolchain ` target -for an example. +for an example. We also make available `$(PYTHON2_ROOTPATH)` and `$(PYTHON3_ROOTPATH)` +which are Make Variable equivalents of `$(PYTHON2)` and `$(PYTHON3)` but for runfiles +locations. These will be helpful if you need to set env vars of binary/test rules +while using [`--nolegacy_external_runfiles`](https://bazel.build/reference/command-line-reference#flag--legacy_external_runfiles). +The original make variables still work in exec contexts such as genrules. ### Overriding toolchain defaults and adding more versions diff --git a/python/current_py_toolchain.bzl b/python/current_py_toolchain.bzl index f3ff2ace07..f5c5638a88 100644 --- a/python/current_py_toolchain.bzl +++ b/python/current_py_toolchain.bzl @@ -27,11 +27,13 @@ def _current_py_toolchain_impl(ctx): direct.append(toolchain.py3_runtime.interpreter) transitive.append(toolchain.py3_runtime.files) vars["PYTHON3"] = toolchain.py3_runtime.interpreter.path + vars["PYTHON3_ROOTPATH"] = toolchain.py3_runtime.interpreter.short_path if toolchain.py2_runtime and toolchain.py2_runtime.interpreter: direct.append(toolchain.py2_runtime.interpreter) transitive.append(toolchain.py2_runtime.files) vars["PYTHON2"] = toolchain.py2_runtime.interpreter.path + vars["PYTHON2_ROOTPATH"] = toolchain.py2_runtime.interpreter.short_path files = depset(direct, transitive = transitive) return [ @@ -49,6 +51,11 @@ current_py_toolchain = rule( other rules, such as genrule. It allows exposing a python toolchain after toolchain resolution has happened, to a rule which expects a concrete implementation of a toolchain, rather than a toolchain_type which could be resolved to that toolchain. + + :::{versionchanged} VERSION_NEXT_FEATURE + From now on, we also expose `$(PYTHON2_ROOTPATH)` and `$(PYTHON3_ROOTPATH)` which are runfiles + locations equivalents of `$(PYTHON2)` and `$(PYTHON3) respectively. + ::: """, implementation = _current_py_toolchain_impl, attrs = { From ccf3141bbe85f1bd7396febe08ff367101826205 Mon Sep 17 00:00:00 2001 From: Frank Portman Date: Tue, 15 Apr 2025 04:38:54 -0400 Subject: [PATCH 166/922] fix(packaging): Format `METADATA` correctly if given empty `requires_file` (#2771) An empty `requires_file` used to be okay, but at some point regressed to leaving an empty line (due to the `metadata.replace(...)`) in the `METADATA` file - rendering the wheel uninstallable. This PR initially attempted to solve that by introducing a new list that processed `METADATA` lines go into, rather than relying on repeated string replacement. But it seems like the repeated string replace actually did more than simply process one line at a time, so I reverted to a single substitution at the end. --- CHANGELOG.md | 1 + examples/wheel/BUILD.bazel | 16 ++++++++++++++++ examples/wheel/wheel_test.py | 24 +++++++++++++++++++++++- python/packaging.bzl | 5 +++++ tools/wheelmaker.py | 7 ++++++- 5 files changed, 51 insertions(+), 2 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 6f86851bdf..e7f9fe30e2 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -96,6 +96,7 @@ Unreleased changes template. * (toolchains) Run the check on the Python interpreter in isolated mode, to ensure it's not affected by userland environment variables, such as `PYTHONPATH`. * (toolchains) Ensure temporary `.pyc` and `.pyo` files are also excluded from the interpreters repository files. * (pypi) Run interpreter version call in isolated mode, to ensure it's not affected by userland environment variables, such as `PYTHONPATH`. +* (packaging) An empty `requires_file` is treated as if it were omitted, resulting in a valid `METADATA` file. {#v0-0-0-added} ### Added diff --git a/examples/wheel/BUILD.bazel b/examples/wheel/BUILD.bazel index d9ba800125..b434e67405 100644 --- a/examples/wheel/BUILD.bazel +++ b/examples/wheel/BUILD.bazel @@ -294,6 +294,12 @@ starlark # Example comment """.splitlines(), ) +write_file( + name = "empty_requires_file", + out = "empty_requires.txt", + content = [""], +) + write_file( name = "extra_requires_file", out = "extra_requires.txt", @@ -324,6 +330,15 @@ py_wheel( deps = [":example_pkg"], ) +py_wheel( + name = "empty_requires_files", + distribution = "empty_requires_files", + python_tag = "py3", + requires_file = ":empty_requires.txt", + version = "0.0.1", + deps = [":example_pkg"], +) + # Package just a specific py_libraries, without their dependencies py_wheel( name = "minimal_data_files", @@ -367,6 +382,7 @@ py_test( ":custom_package_root_multi_prefix", ":custom_package_root_multi_prefix_reverse_order", ":customized", + ":empty_requires_files", ":extra_requires", ":filename_escaping", ":minimal_data_files", diff --git a/examples/wheel/wheel_test.py b/examples/wheel/wheel_test.py index a3d6034930..9ec150301d 100644 --- a/examples/wheel/wheel_test.py +++ b/examples/wheel/wheel_test.py @@ -483,7 +483,6 @@ def test_requires_file_and_extra_requires_files(self): if line.startswith(b"Requires-Dist:"): requires.append(line.decode("utf-8").strip()) - print(requires) self.assertEqual( [ "Requires-Dist: tomli>=2.0.0", @@ -495,6 +494,29 @@ def test_requires_file_and_extra_requires_files(self): requires, ) + def test_empty_requires_file(self): + filename = self._get_path("empty_requires_files-0.0.1-py3-none-any.whl") + + with zipfile.ZipFile(filename) as zf: + self.assertAllEntriesHasReproducibleMetadata(zf) + metadata_file = None + for f in zf.namelist(): + if os.path.basename(f) == "METADATA": + metadata_file = f + self.assertIsNotNone(metadata_file) + + metadata = zf.read(metadata_file).decode("utf-8") + metadata_lines = metadata.splitlines() + + requires = [] + for i, line in enumerate(metadata_lines): + if line.startswith("Name:"): + self.assertTrue(metadata_lines[i + 1].startswith("Version:")) + if line.startswith("Requires-Dist:"): + requires.append(line.strip()) + + self.assertEqual([], requires) + def test_minimal_data_files(self): filename = self._get_path("minimal_data_files-0.0.1-py3-none-any.whl") diff --git a/python/packaging.bzl b/python/packaging.bzl index 629af2d6a4..b190635cfe 100644 --- a/python/packaging.bzl +++ b/python/packaging.bzl @@ -101,6 +101,11 @@ def py_wheel( Currently only pure-python wheels are supported. + :::{versionchanged} VERSION_NEXT_FEATURE + From now on, an empty `requires_file` is treated as if it were omitted, resulting in a valid + `METADATA` file. + ::: + Examples: ```python diff --git a/tools/wheelmaker.py b/tools/wheelmaker.py index 23b18eca5f..908b3fe956 100644 --- a/tools/wheelmaker.py +++ b/tools/wheelmaker.py @@ -599,7 +599,12 @@ def get_new_requirement_line(reqs_text, extra): reqs.append(get_new_requirement_line(reqs_text, extra)) - metadata = metadata.replace(meta_line, "\n".join(reqs)) + if reqs: + metadata = metadata.replace(meta_line, "\n".join(reqs)) + # File is empty + # So replace the meta_line entirely, including removing newline chars + else: + metadata = re.sub(re.escape(meta_line) + r"(?:\r?\n)?", "", metadata, count=1) maker.add_metadata( metadata=metadata, From ff1388356b0d47b6249dc606ae4ba521df54a06f Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Tue, 15 Apr 2025 17:40:02 +0900 Subject: [PATCH 167/922] build(deps): bump typing-extensions from 4.12.2 to 4.13.2 in /docs (#2776) Bumps [typing-extensions](https://github.com/python/typing_extensions) from 4.12.2 to 4.13.2.
Release notes

Sourced from typing-extensions's releases.

4.13.2

  • Fix TypeError when taking the union of typing_extensions.TypeAliasType and a typing.TypeAliasType on Python 3.12 and 3.13. Patch by Joren Hammudoglu.
  • Backport from CPython PR #132160 to avoid having user arguments shadowed in generated __new__ by @typing_extensions.deprecated. Patch by Victorien Plot.

4.13.1

This is a bugfix release fixing two edge cases that appear on old bugfix releases of CPython.

Bugfixes:

  • Fix regression in 4.13.0 on Python 3.10.2 causing a TypeError when using Concatenate. Patch by Daraan.
  • Fix TypeError when using evaluate_forward_ref on Python 3.10.1-2 and 3.9.8-10. Patch by Daraan.

4.13.0

New features:

  • Add typing_extensions.TypeForm from PEP 747. Patch by Jelle Zijlstra.
  • Add typing_extensions.get_annotations, a backport of inspect.get_annotations that adds features specified by PEP 649. Patches by Jelle Zijlstra and Alex Waygood.
  • Backport evaluate_forward_ref from CPython PR #119891 to evaluate ForwardRefs. Patch by Daraan, backporting a CPython PR by Jelle Zijlstra.

Bugfixes and changed features:

  • Update PEP 728 implementation to a newer version of the PEP. Patch by Jelle Zijlstra.
  • Copy the coroutine status of functions and methods wrapped with @typing_extensions.deprecated. Patch by Sebastian Rittau.
  • Fix bug where TypeAliasType instances could be subscripted even where they were not generic. Patch by Daraan.
  • Fix bug where a subscripted TypeAliasType instance did not have all attributes of the original TypeAliasType instance on older Python versions. Patch by Daraan and Alex Waygood.
  • Fix bug where subscripted TypeAliasType instances (and some other subscripted objects) had wrong parameters if they were directly subscripted with an Unpack object. Patch by Daraan.
  • Backport to Python 3.10 the ability to substitute ... in generic Callable aliases that have a Concatenate special form as their argument. Patch by Daraan.
  • Extended the Concatenate backport for Python 3.8-3.10 to now accept Ellipsis as an argument. Patch by Daraan.
  • Fix backport of get_type_hints to reflect Python 3.11+ behavior which does not add

... (truncated)

Changelog

Sourced from typing-extensions's changelog.

Release 4.13.2 (April 10, 2025)

  • Fix TypeError when taking the union of typing_extensions.TypeAliasType and a typing.TypeAliasType on Python 3.12 and 3.13. Patch by Joren Hammudoglu.
  • Backport from CPython PR #132160 to avoid having user arguments shadowed in generated __new__ by @typing_extensions.deprecated. Patch by Victorien Plot.

Release 4.13.1 (April 3, 2025)

Bugfixes:

  • Fix regression in 4.13.0 on Python 3.10.2 causing a TypeError when using Concatenate. Patch by Daraan.
  • Fix TypeError when using evaluate_forward_ref on Python 3.10.1-2 and 3.9.8-10. Patch by Daraan.

Release 4.13.0 (March 25, 2025)

No user-facing changes since 4.13.0rc1.

Release 4.13.0rc1 (March 18, 2025)

New features:

  • Add typing_extensions.TypeForm from PEP 747. Patch by Jelle Zijlstra.
  • Add typing_extensions.get_annotations, a backport of inspect.get_annotations that adds features specified by PEP 649. Patches by Jelle Zijlstra and Alex Waygood.
  • Backport evaluate_forward_ref from CPython PR #119891 to evaluate ForwardRefs. Patch by Daraan, backporting a CPython PR by Jelle Zijlstra.

Bugfixes and changed features:

  • Update PEP 728 implementation to a newer version of the PEP. Patch by Jelle Zijlstra.
  • Copy the coroutine status of functions and methods wrapped with @typing_extensions.deprecated. Patch by Sebastian Rittau.
  • Fix bug where TypeAliasType instances could be subscripted even where they were not generic. Patch by Daraan.
  • Fix bug where a subscripted TypeAliasType instance did not have all attributes of the original TypeAliasType instance on older Python versions. Patch by Daraan and Alex Waygood.
  • Fix bug where subscripted TypeAliasType instances (and some other subscripted objects) had wrong parameters if they were directly subscripted with an Unpack object. Patch by Daraan.
  • Backport to Python 3.10 the ability to substitute ... in generic Callable

... (truncated)

Commits
  • 4525e9d Prepare release 4.13.2 (#583)
  • 88a0c20 Do not shadow user arguments in generated __new__ by @deprecated (#581)
  • 281d7b0 Add 3rd party tests for litestar (#578)
  • 8092c39 fix TypeAliasType union with typing.TypeAliasType (#575)
  • 45a8847 Prepare release 4.13.1 (#573)
  • f264e58 Move CI to "ubuntu-latest" (round 2) (#570)
  • 5ce0e69 Fix TypeError with evaluate_forward_ref on some 3.10 and 3.9 versions (#558)
  • 304f5cb Add SQLAlchemy to third-party daily tests (#561)
  • ebe2b94 Fix duplicated keywords for typing._ConcatenateGenericAlias in 3.10.2 (#557)
  • 9f93d6f Add intersphinx links for 3.13 typing features (#550)
  • Additional commits viewable in compare view

[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=typing-extensions&package-manager=pip&previous-version=4.12.2&new-version=4.13.2)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) ---
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot show ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> --- docs/requirements.txt | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/requirements.txt b/docs/requirements.txt index 8d1cbabffc..e2fb59565a 100644 --- a/docs/requirements.txt +++ b/docs/requirements.txt @@ -351,9 +351,9 @@ sphinxcontrib-serializinghtml==2.0.0 \ --hash=sha256:6e2cb0eef194e10c27ec0023bfeb25badbbb5868244cf5bc5bdc04e4464bf331 \ --hash=sha256:e9d912827f872c029017a53f0ef2180b327c3f7fd23c87229f7a8e8b70031d4d # via sphinx -typing-extensions==4.12.2 \ - --hash=sha256:04e5ca0351e0f3f85c6853954072df659d0d13fac324d0072316b67d7794700d \ - --hash=sha256:1a7ead55c7e559dd4dee8856e3a88b41225abfe1ce8df57b7c13915fe121ffb8 +typing-extensions==4.13.2 \ + --hash=sha256:a439e7c04b49fec3e5d3e2beaa21755cadbbdc391694e28ccdd36ca4a1408f8c \ + --hash=sha256:e6c81219bd689f51865d9e372991c540bda33a0379d5573cddb9a3a23f7caaef # via # rules-python-docs (docs/pyproject.toml) # sphinx-autodoc2 From 2cf7ba4bb76f630ff7f2c83cab0b5294db65107b Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Tue, 15 Apr 2025 17:40:24 +0900 Subject: [PATCH 168/922] build(deps): bump urllib3 from 2.3.0 to 2.4.0 in /tools/publish (#2775) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Bumps [urllib3](https://github.com/urllib3/urllib3) from 2.3.0 to 2.4.0.
Release notes

Sourced from urllib3's releases.

2.4.0

🚀 urllib3 is fundraising for HTTP/2 support

urllib3 is raising ~$40,000 USD to release HTTP/2 support and ensure long-term sustainable maintenance of the project after a sharp decline in financial support. If your company or organization uses Python and would benefit from HTTP/2 support in Requests, pip, cloud SDKs, and thousands of other projects please consider contributing financially to ensure HTTP/2 support is developed sustainably and maintained for the long-haul.

Thank you for your support.

Features

  • Applied PEP 639 by specifying the license fields in pyproject.toml. (#3522)
  • Updated exceptions to save and restore more properties during the pickle/serialization process. (#3567)
  • Added verify_flags option to create_urllib3_context with a default of VERIFY_X509_PARTIAL_CHAIN and VERIFY_X509_STRICT for Python 3.13+. (#3571)

Bugfixes

  • Fixed a bug with partial reads of streaming data in Emscripten. (#3555)

Misc

  • Switched to uv for installing development dependecies. (#3550)
  • Removed the multiple.intoto.jsonl asset from GitHub releases. Attestation of release files since v2.3.0 can be found on PyPI. (#3566)
Changelog

Sourced from urllib3's changelog.

2.4.0 (2025-04-10)

Features

  • Applied PEP 639 by specifying the license fields in pyproject.toml. ([#3522](https://github.com/urllib3/urllib3/issues/3522) <https://github.com/urllib3/urllib3/issues/3522>__)
  • Updated exceptions to save and restore more properties during the pickle/serialization process. ([#3567](https://github.com/urllib3/urllib3/issues/3567) <https://github.com/urllib3/urllib3/issues/3567>__)
  • Added verify_flags option to create_urllib3_context with a default of VERIFY_X509_PARTIAL_CHAIN and VERIFY_X509_STRICT for Python 3.13+. ([#3571](https://github.com/urllib3/urllib3/issues/3571) <https://github.com/urllib3/urllib3/issues/3571>__)

Bugfixes

  • Fixed a bug with partial reads of streaming data in Emscripten. ([#3555](https://github.com/urllib3/urllib3/issues/3555) <https://github.com/urllib3/urllib3/issues/3555>__)

Misc

  • Switched to uv for installing development dependecies. ([#3550](https://github.com/urllib3/urllib3/issues/3550) <https://github.com/urllib3/urllib3/issues/3550>__)
  • Removed the multiple.intoto.jsonl asset from GitHub releases. Attestation of release files since v2.3.0 can be found on PyPI. ([#3566](https://github.com/urllib3/urllib3/issues/3566) <https://github.com/urllib3/urllib3/issues/3566>__)
Commits

[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=urllib3&package-manager=pip&previous-version=2.3.0&new-version=2.4.0)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) ---
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot show ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> --- tools/publish/requirements_darwin.txt | 6 +++--- tools/publish/requirements_linux.txt | 6 +++--- tools/publish/requirements_universal.txt | 6 +++--- tools/publish/requirements_windows.txt | 6 +++--- 4 files changed, 12 insertions(+), 12 deletions(-) diff --git a/tools/publish/requirements_darwin.txt b/tools/publish/requirements_darwin.txt index 5f8a33c3f5..eaec72c01c 100644 --- a/tools/publish/requirements_darwin.txt +++ b/tools/publish/requirements_darwin.txt @@ -202,9 +202,9 @@ twine==5.1.1 \ --hash=sha256:215dbe7b4b94c2c50a7315c0275d2258399280fbb7d04182c7e55e24b5f93997 \ --hash=sha256:9aa0825139c02b3434d913545c7b847a21c835e11597f5255842d457da2322db # via -r tools/publish/requirements.in -urllib3==2.3.0 \ - --hash=sha256:1cee9ad369867bfdbbb48b7dd50374c0967a0bb7710050facf0dd6911440e3df \ - --hash=sha256:f8c5449b3cf0861679ce7e0503c7b44b5ec981bec0d1d3795a07f1ba96f0204d +urllib3==2.4.0 \ + --hash=sha256:414bc6535b787febd7567804cc015fee39daab8ad86268f1310a9250697de466 \ + --hash=sha256:4e16665048960a0900c702d4a66415956a584919c03361cac9f1df5c5dd7e813 # via # requests # twine diff --git a/tools/publish/requirements_linux.txt b/tools/publish/requirements_linux.txt index 40d987b16d..5fdc742a88 100644 --- a/tools/publish/requirements_linux.txt +++ b/tools/publish/requirements_linux.txt @@ -318,9 +318,9 @@ twine==5.1.1 \ --hash=sha256:215dbe7b4b94c2c50a7315c0275d2258399280fbb7d04182c7e55e24b5f93997 \ --hash=sha256:9aa0825139c02b3434d913545c7b847a21c835e11597f5255842d457da2322db # via -r tools/publish/requirements.in -urllib3==2.3.0 \ - --hash=sha256:1cee9ad369867bfdbbb48b7dd50374c0967a0bb7710050facf0dd6911440e3df \ - --hash=sha256:f8c5449b3cf0861679ce7e0503c7b44b5ec981bec0d1d3795a07f1ba96f0204d +urllib3==2.4.0 \ + --hash=sha256:414bc6535b787febd7567804cc015fee39daab8ad86268f1310a9250697de466 \ + --hash=sha256:4e16665048960a0900c702d4a66415956a584919c03361cac9f1df5c5dd7e813 # via # requests # twine diff --git a/tools/publish/requirements_universal.txt b/tools/publish/requirements_universal.txt index c8bc0bb258..97cbef0221 100644 --- a/tools/publish/requirements_universal.txt +++ b/tools/publish/requirements_universal.txt @@ -322,9 +322,9 @@ twine==5.1.1 \ --hash=sha256:215dbe7b4b94c2c50a7315c0275d2258399280fbb7d04182c7e55e24b5f93997 \ --hash=sha256:9aa0825139c02b3434d913545c7b847a21c835e11597f5255842d457da2322db # via -r tools/publish/requirements.in -urllib3==2.3.0 \ - --hash=sha256:1cee9ad369867bfdbbb48b7dd50374c0967a0bb7710050facf0dd6911440e3df \ - --hash=sha256:f8c5449b3cf0861679ce7e0503c7b44b5ec981bec0d1d3795a07f1ba96f0204d +urllib3==2.4.0 \ + --hash=sha256:414bc6535b787febd7567804cc015fee39daab8ad86268f1310a9250697de466 \ + --hash=sha256:4e16665048960a0900c702d4a66415956a584919c03361cac9f1df5c5dd7e813 # via # requests # twine diff --git a/tools/publish/requirements_windows.txt b/tools/publish/requirements_windows.txt index 1980812d15..458414009e 100644 --- a/tools/publish/requirements_windows.txt +++ b/tools/publish/requirements_windows.txt @@ -206,9 +206,9 @@ twine==5.1.1 \ --hash=sha256:215dbe7b4b94c2c50a7315c0275d2258399280fbb7d04182c7e55e24b5f93997 \ --hash=sha256:9aa0825139c02b3434d913545c7b847a21c835e11597f5255842d457da2322db # via -r tools/publish/requirements.in -urllib3==2.3.0 \ - --hash=sha256:1cee9ad369867bfdbbb48b7dd50374c0967a0bb7710050facf0dd6911440e3df \ - --hash=sha256:f8c5449b3cf0861679ce7e0503c7b44b5ec981bec0d1d3795a07f1ba96f0204d +urllib3==2.4.0 \ + --hash=sha256:414bc6535b787febd7567804cc015fee39daab8ad86268f1310a9250697de466 \ + --hash=sha256:4e16665048960a0900c702d4a66415956a584919c03361cac9f1df5c5dd7e813 # via # requests # twine From 101962aecbe048525248361d7a8e6341655fa30f Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Tue, 15 Apr 2025 17:40:45 +0900 Subject: [PATCH 169/922] build(deps): bump urllib3 from 2.3.0 to 2.4.0 in /docs (#2774) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Bumps [urllib3](https://github.com/urllib3/urllib3) from 2.3.0 to 2.4.0.
Release notes

Sourced from urllib3's releases.

2.4.0

🚀 urllib3 is fundraising for HTTP/2 support

urllib3 is raising ~$40,000 USD to release HTTP/2 support and ensure long-term sustainable maintenance of the project after a sharp decline in financial support. If your company or organization uses Python and would benefit from HTTP/2 support in Requests, pip, cloud SDKs, and thousands of other projects please consider contributing financially to ensure HTTP/2 support is developed sustainably and maintained for the long-haul.

Thank you for your support.

Features

  • Applied PEP 639 by specifying the license fields in pyproject.toml. (#3522)
  • Updated exceptions to save and restore more properties during the pickle/serialization process. (#3567)
  • Added verify_flags option to create_urllib3_context with a default of VERIFY_X509_PARTIAL_CHAIN and VERIFY_X509_STRICT for Python 3.13+. (#3571)

Bugfixes

  • Fixed a bug with partial reads of streaming data in Emscripten. (#3555)

Misc

  • Switched to uv for installing development dependecies. (#3550)
  • Removed the multiple.intoto.jsonl asset from GitHub releases. Attestation of release files since v2.3.0 can be found on PyPI. (#3566)
Changelog

Sourced from urllib3's changelog.

2.4.0 (2025-04-10)

Features

  • Applied PEP 639 by specifying the license fields in pyproject.toml. ([#3522](https://github.com/urllib3/urllib3/issues/3522) <https://github.com/urllib3/urllib3/issues/3522>__)
  • Updated exceptions to save and restore more properties during the pickle/serialization process. ([#3567](https://github.com/urllib3/urllib3/issues/3567) <https://github.com/urllib3/urllib3/issues/3567>__)
  • Added verify_flags option to create_urllib3_context with a default of VERIFY_X509_PARTIAL_CHAIN and VERIFY_X509_STRICT for Python 3.13+. ([#3571](https://github.com/urllib3/urllib3/issues/3571) <https://github.com/urllib3/urllib3/issues/3571>__)

Bugfixes

  • Fixed a bug with partial reads of streaming data in Emscripten. ([#3555](https://github.com/urllib3/urllib3/issues/3555) <https://github.com/urllib3/urllib3/issues/3555>__)

Misc

  • Switched to uv for installing development dependecies. ([#3550](https://github.com/urllib3/urllib3/issues/3550) <https://github.com/urllib3/urllib3/issues/3550>__)
  • Removed the multiple.intoto.jsonl asset from GitHub releases. Attestation of release files since v2.3.0 can be found on PyPI. ([#3566](https://github.com/urllib3/urllib3/issues/3566) <https://github.com/urllib3/urllib3/issues/3566>__)
Commits

[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=urllib3&package-manager=pip&previous-version=2.3.0&new-version=2.4.0)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) ---
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot show ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> --- docs/requirements.txt | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/requirements.txt b/docs/requirements.txt index e2fb59565a..5e308b00f4 100644 --- a/docs/requirements.txt +++ b/docs/requirements.txt @@ -357,7 +357,7 @@ typing-extensions==4.13.2 \ # via # rules-python-docs (docs/pyproject.toml) # sphinx-autodoc2 -urllib3==2.3.0 \ - --hash=sha256:1cee9ad369867bfdbbb48b7dd50374c0967a0bb7710050facf0dd6911440e3df \ - --hash=sha256:f8c5449b3cf0861679ce7e0503c7b44b5ec981bec0d1d3795a07f1ba96f0204d +urllib3==2.4.0 \ + --hash=sha256:414bc6535b787febd7567804cc015fee39daab8ad86268f1310a9250697de466 \ + --hash=sha256:4e16665048960a0900c702d4a66415956a584919c03361cac9f1df5c5dd7e813 # via requests From 8fc25de7dcec1d1106edd8e076c9fcb58497b40b Mon Sep 17 00:00:00 2001 From: Ignas Anikevicius <240938+aignas@users.noreply.github.com> Date: Wed, 16 Apr 2025 14:45:34 +0900 Subject: [PATCH 170/922] refactor(bzlmod): stop using 'repo' attr in whl_library (#2779) A simple non-functional cleanup that just removes legacy code paths from bzlmod PyPI integration. --- python/private/pypi/extension.bzl | 1 - python/private/pypi/whl_library.bzl | 6 ++++-- tests/pypi/extension/extension_tests.bzl | 22 ---------------------- 3 files changed, 4 insertions(+), 25 deletions(-) diff --git a/python/private/pypi/extension.bzl b/python/private/pypi/extension.bzl index 8fce47656b..d2ae132741 100644 --- a/python/private/pypi/extension.bzl +++ b/python/private/pypi/extension.bzl @@ -181,7 +181,6 @@ def _create_whl_repos( # Construct args separately so that the lock file can be smaller and does not include unused # attrs. whl_library_args = dict( - repo = pip_name, dep_template = "@{}//{{name}}:{{target}}".format(hub_name), ) maybe_args = dict( diff --git a/python/private/pypi/whl_library.bzl b/python/private/pypi/whl_library.bzl index 54f9ff3909..0a580011ab 100644 --- a/python/private/pypi/whl_library.bzl +++ b/python/private/pypi/whl_library.bzl @@ -517,8 +517,10 @@ and the target that we need respectively. doc = "Name of the group, if any.", ), "repo": attr.string( - mandatory = True, - doc = "Pointer to parent repo name. Used to make these rules rerun if the parent repo changes.", + doc = """\ +Pointer to parent repo name. Used to make these rules rerun if the parent repo changes. +Only used in WORKSPACE when the {attr}`dep_template` is not set. +""", ), "repo_prefix": attr.string( doc = """ diff --git a/tests/pypi/extension/extension_tests.bzl b/tests/pypi/extension/extension_tests.bzl index 66c9e0549e..4d86d6a6e0 100644 --- a/tests/pypi/extension/extension_tests.bzl +++ b/tests/pypi/extension/extension_tests.bzl @@ -174,7 +174,6 @@ def _test_simple(env): "pypi_315_simple": { "dep_template": "@pypi//{name}:{target}", "python_interpreter_target": "unit_test_interpreter_target", - "repo": "pypi_315", "requirement": "simple==0.0.1 --hash=sha256:deadbeef --hash=sha256:deadbaaf", }, }) @@ -234,13 +233,11 @@ def _test_simple_multiple_requirements(env): "pypi_315_simple_osx_aarch64_osx_x86_64": { "dep_template": "@pypi//{name}:{target}", "python_interpreter_target": "unit_test_interpreter_target", - "repo": "pypi_315", "requirement": "simple==0.0.2 --hash=sha256:deadb00f", }, "pypi_315_simple_windows_x86_64": { "dep_template": "@pypi//{name}:{target}", "python_interpreter_target": "unit_test_interpreter_target", - "repo": "pypi_315", "requirement": "simple==0.0.1 --hash=sha256:deadbeef", }, }) @@ -307,13 +304,11 @@ torch==2.4.1 ; platform_machine != 'x86_64' \ "pypi_315_torch_linux_aarch64_linux_arm_linux_ppc_linux_s390x_osx_aarch64": { "dep_template": "@pypi//{name}:{target}", "python_interpreter_target": "unit_test_interpreter_target", - "repo": "pypi_315", "requirement": "torch==2.4.1 --hash=sha256:deadbeef", }, "pypi_315_torch_linux_x86_64_osx_x86_64_windows_x86_64": { "dep_template": "@pypi//{name}:{target}", "python_interpreter_target": "unit_test_interpreter_target", - "repo": "pypi_315", "requirement": "torch==2.4.1+cpu", }, }) @@ -444,7 +439,6 @@ torch==2.4.1+cpu ; platform_machine == 'x86_64' \ "experimental_target_platforms": ["cp312_linux_x86_64"], "filename": "torch-2.4.1+cpu-cp312-cp312-linux_x86_64.whl", "python_interpreter_target": "unit_test_interpreter_target", - "repo": "pypi_312", "requirement": "torch==2.4.1+cpu", "sha256": "8800deef0026011d502c0c256cc4b67d002347f63c3a38cd8e45f1f445c61364", "urls": ["https://torch.index/whl/cpu/torch-2.4.1%2Bcpu-cp312-cp312-linux_x86_64.whl"], @@ -454,7 +448,6 @@ torch==2.4.1+cpu ; platform_machine == 'x86_64' \ "experimental_target_platforms": ["cp312_linux_aarch64"], "filename": "torch-2.4.1-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", "python_interpreter_target": "unit_test_interpreter_target", - "repo": "pypi_312", "requirement": "torch==2.4.1", "sha256": "36109432b10bd7163c9b30ce896f3c2cca1b86b9765f956a1594f0ff43091e2a", "urls": ["https://torch.index/whl/cpu/torch-2.4.1-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl"], @@ -464,7 +457,6 @@ torch==2.4.1+cpu ; platform_machine == 'x86_64' \ "experimental_target_platforms": ["cp312_windows_x86_64"], "filename": "torch-2.4.1+cpu-cp312-cp312-win_amd64.whl", "python_interpreter_target": "unit_test_interpreter_target", - "repo": "pypi_312", "requirement": "torch==2.4.1+cpu", "sha256": "3a570e5c553415cdbddfe679207327b3a3806b21c6adea14fba77684d1619e97", "urls": ["https://torch.index/whl/cpu/torch-2.4.1%2Bcpu-cp312-cp312-win_amd64.whl"], @@ -474,7 +466,6 @@ torch==2.4.1+cpu ; platform_machine == 'x86_64' \ "experimental_target_platforms": ["cp312_osx_aarch64"], "filename": "torch-2.4.1-cp312-none-macosx_11_0_arm64.whl", "python_interpreter_target": "unit_test_interpreter_target", - "repo": "pypi_312", "requirement": "torch==2.4.1", "sha256": "72b484d5b6cec1a735bf3fa5a1c4883d01748698c5e9cfdbeb4ffab7c7987e0d", "urls": ["https://torch.index/whl/cpu/torch-2.4.1-cp312-none-macosx_11_0_arm64.whl"], @@ -560,7 +551,6 @@ simple==0.0.3 \ "experimental_target_platforms": ["cp315_linux_x86_64"], "extra_pip_args": ["--platform=manylinux_2_17_x86_64", "--python-version=315", "--implementation=cp", "--abi=cp315"], "python_interpreter_target": "unit_test_interpreter_target", - "repo": "pypi_315", "requirement": "extra==0.0.1 --hash=sha256:deadb00f", }, "pypi_315_simple_linux_x86_64": { @@ -569,7 +559,6 @@ simple==0.0.3 \ "experimental_target_platforms": ["cp315_linux_x86_64"], "extra_pip_args": ["--platform=manylinux_2_17_x86_64", "--python-version=315", "--implementation=cp", "--abi=cp315"], "python_interpreter_target": "unit_test_interpreter_target", - "repo": "pypi_315", "requirement": "simple==0.0.1 --hash=sha256:deadbeef", }, "pypi_315_simple_osx_aarch64": { @@ -578,7 +567,6 @@ simple==0.0.3 \ "experimental_target_platforms": ["cp315_osx_aarch64"], "extra_pip_args": ["--platform=macosx_10_9_arm64", "--python-version=315", "--implementation=cp", "--abi=cp315"], "python_interpreter_target": "unit_test_interpreter_target", - "repo": "pypi_315", "requirement": "simple==0.0.3 --hash=sha256:deadbaaf", }, }) @@ -766,7 +754,6 @@ git_dep @ git+https://git.server/repo/project@deadbeefdeadbeef "extra_pip_args": ["--extra-args-for-sdist-building"], "filename": "any-name.tar.gz", "python_interpreter_target": "unit_test_interpreter_target", - "repo": "pypi_315", "requirement": "direct_sdist_without_sha @ some-archive/any-name.tar.gz", "sha256": "", "urls": ["some-archive/any-name.tar.gz"], @@ -776,7 +763,6 @@ git_dep @ git+https://git.server/repo/project@deadbeefdeadbeef "experimental_target_platforms": ["cp315_linux_aarch64", "cp315_linux_arm", "cp315_linux_ppc", "cp315_linux_s390x", "cp315_linux_x86_64", "cp315_osx_aarch64", "cp315_osx_x86_64", "cp315_windows_x86_64"], "filename": "direct_without_sha-0.0.1-py3-none-any.whl", "python_interpreter_target": "unit_test_interpreter_target", - "repo": "pypi_315", "requirement": "direct_without_sha==0.0.1 @ example-direct.org/direct_without_sha-0.0.1-py3-none-any.whl", "sha256": "", "urls": ["example-direct.org/direct_without_sha-0.0.1-py3-none-any.whl"], @@ -785,14 +771,12 @@ git_dep @ git+https://git.server/repo/project@deadbeefdeadbeef "dep_template": "@pypi//{name}:{target}", "extra_pip_args": ["--extra-args-for-sdist-building"], "python_interpreter_target": "unit_test_interpreter_target", - "repo": "pypi_315", "requirement": "git_dep @ git+https://git.server/repo/project@deadbeefdeadbeef", }, "pypi_315_pip_fallback": { "dep_template": "@pypi//{name}:{target}", "extra_pip_args": ["--extra-args-for-sdist-building"], "python_interpreter_target": "unit_test_interpreter_target", - "repo": "pypi_315", "requirement": "pip_fallback==0.0.1", }, "pypi_315_simple_py3_none_any_deadb00f": { @@ -800,7 +784,6 @@ git_dep @ git+https://git.server/repo/project@deadbeefdeadbeef "experimental_target_platforms": ["cp315_linux_aarch64", "cp315_linux_arm", "cp315_linux_ppc", "cp315_linux_s390x", "cp315_linux_x86_64", "cp315_osx_aarch64", "cp315_osx_x86_64", "cp315_windows_x86_64"], "filename": "simple-0.0.1-py3-none-any.whl", "python_interpreter_target": "unit_test_interpreter_target", - "repo": "pypi_315", "requirement": "simple==0.0.1", "sha256": "deadb00f", "urls": ["example2.org"], @@ -811,7 +794,6 @@ git_dep @ git+https://git.server/repo/project@deadbeefdeadbeef "extra_pip_args": ["--extra-args-for-sdist-building"], "filename": "simple-0.0.1.tar.gz", "python_interpreter_target": "unit_test_interpreter_target", - "repo": "pypi_315", "requirement": "simple==0.0.1", "sha256": "deadbeef", "urls": ["example.org"], @@ -821,7 +803,6 @@ git_dep @ git+https://git.server/repo/project@deadbeefdeadbeef "experimental_target_platforms": ["cp315_linux_aarch64", "cp315_linux_arm", "cp315_linux_ppc", "cp315_linux_s390x", "cp315_linux_x86_64", "cp315_osx_aarch64", "cp315_osx_x86_64", "cp315_windows_x86_64"], "filename": "some_pkg-0.0.1-py3-none-any.whl", "python_interpreter_target": "unit_test_interpreter_target", - "repo": "pypi_315", "requirement": "some_pkg==0.0.1 @ example-direct.org/some_pkg-0.0.1-py3-none-any.whl --hash=sha256:deadbaaf", "sha256": "deadbaaf", "urls": ["example-direct.org/some_pkg-0.0.1-py3-none-any.whl"], @@ -831,7 +812,6 @@ git_dep @ git+https://git.server/repo/project@deadbeefdeadbeef "experimental_target_platforms": ["cp315_linux_aarch64", "cp315_linux_arm", "cp315_linux_ppc", "cp315_linux_s390x", "cp315_linux_x86_64", "cp315_osx_aarch64", "cp315_osx_x86_64", "cp315_windows_x86_64"], "filename": "some-other-pkg-0.0.1-py3-none-any.whl", "python_interpreter_target": "unit_test_interpreter_target", - "repo": "pypi_315", "requirement": "some_other_pkg==0.0.1", "sha256": "deadb33f", "urls": ["example2.org/index/some_other_pkg/"], @@ -920,13 +900,11 @@ optimum[onnxruntime-gpu]==1.17.1 ; sys_platform == 'linux' "pypi_315_optimum_linux_aarch64_linux_arm_linux_ppc_linux_s390x_linux_x86_64": { "dep_template": "@pypi//{name}:{target}", "python_interpreter_target": "unit_test_interpreter_target", - "repo": "pypi_315", "requirement": "optimum[onnxruntime-gpu]==1.17.1", }, "pypi_315_optimum_osx_aarch64_osx_x86_64": { "dep_template": "@pypi//{name}:{target}", "python_interpreter_target": "unit_test_interpreter_target", - "repo": "pypi_315", "requirement": "optimum[onnxruntime]==1.17.1", }, }) From c813d845b959e37d4949e368c86bc1277d153b38 Mon Sep 17 00:00:00 2001 From: Matt Mackay Date: Wed, 16 Apr 2025 23:45:50 -0400 Subject: [PATCH 171/922] perf: lazily load gazelle manifest files (#2746) In large repositories where Python may not be the only language, the gazelle manifest loading is done unnecessarily, and is done during the configuration walk. This means that even for non-python gazelle invocations (eg `bazel run gazelle -- web/`), Python manifest files are being parsed and loaded into memory. This issue compounds if the repository uses multiple dependency closures, ie multiple `gazelle_python.yaml` files. In our repo, we currently have ~250 Python manifests, so loading them when Gazelle is only running over other languages is time consuming. Co-authored-by: Douglas Thor --- CHANGELOG.md | 3 +++ gazelle/python/configure.go | 24 +---------------- gazelle/pythonconfig/pythonconfig.go | 40 +++++++++++++++++++++++++--- 3 files changed, 40 insertions(+), 27 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index e7f9fe30e2..299a43e1ff 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -76,6 +76,9 @@ Unreleased changes template. * (pypi) The PyPI extension will no longer write the lock file entries as the extension has been marked reproducible. Fixes [#2434](https://github.com/bazel-contrib/rules_python/issues/2434). +* (gazelle) Lazily load and parse manifest files when running Gazelle. This ensures no + manifest files are loaded when Gazelle is run over a set of non-python directories + [PR #2746](https://github.com/bazel-contrib/rules_python/pull/2746). * (rules) {attr}`py_binary.srcs` and {attr}`py_test.srcs` is no longer mandatory when `main_module` is specified (for `--bootstrap_impl=script`) diff --git a/gazelle/python/configure.go b/gazelle/python/configure.go index 7b1f091b34..a00b0ba0ba 100644 --- a/gazelle/python/configure.go +++ b/gazelle/python/configure.go @@ -18,7 +18,6 @@ import ( "flag" "fmt" "log" - "os" "path/filepath" "strconv" "strings" @@ -27,7 +26,6 @@ import ( "github.com/bazelbuild/bazel-gazelle/rule" "github.com/bmatcuk/doublestar/v4" - "github.com/bazel-contrib/rules_python/gazelle/manifest" "github.com/bazel-contrib/rules_python/gazelle/pythonconfig" ) @@ -228,25 +226,5 @@ func (py *Configurer) Configure(c *config.Config, rel string, f *rule.File) { } gazelleManifestPath := filepath.Join(c.RepoRoot, rel, gazelleManifestFilename) - gazelleManifest, err := py.loadGazelleManifest(gazelleManifestPath) - if err != nil { - log.Fatal(err) - } - if gazelleManifest != nil { - config.SetGazelleManifest(gazelleManifest) - } -} - -func (py *Configurer) loadGazelleManifest(gazelleManifestPath string) (*manifest.Manifest, error) { - if _, err := os.Stat(gazelleManifestPath); err != nil { - if os.IsNotExist(err) { - return nil, nil - } - return nil, fmt.Errorf("failed to load Gazelle manifest at %q: %w", gazelleManifestPath, err) - } - manifestFile := new(manifest.File) - if err := manifestFile.Decode(gazelleManifestPath); err != nil { - return nil, fmt.Errorf("failed to load Gazelle manifest at %q: %w", gazelleManifestPath, err) - } - return manifestFile.Manifest, nil + config.SetGazelleManifestPath(gazelleManifestPath) } diff --git a/gazelle/pythonconfig/pythonconfig.go b/gazelle/pythonconfig/pythonconfig.go index 23c0cfd572..866339d449 100644 --- a/gazelle/pythonconfig/pythonconfig.go +++ b/gazelle/pythonconfig/pythonconfig.go @@ -16,6 +16,8 @@ package pythonconfig import ( "fmt" + "log" + "os" "path" "regexp" "strings" @@ -153,10 +155,11 @@ func (c Configs) ParentForPackage(pkg string) *Config { type Config struct { parent *Config - extensionEnabled bool - repoRoot string - pythonProjectRoot string - gazelleManifest *manifest.Manifest + extensionEnabled bool + repoRoot string + pythonProjectRoot string + gazelleManifestPath string + gazelleManifest *manifest.Manifest excludedPatterns *singlylinkedlist.List ignoreFiles map[string]struct{} @@ -281,11 +284,26 @@ func (c *Config) SetGazelleManifest(gazelleManifest *manifest.Manifest) { c.gazelleManifest = gazelleManifest } +// SetGazelleManifestPath sets the path to the gazelle_python.yaml file +// for the current configuration. +func (c *Config) SetGazelleManifestPath(gazelleManifestPath string) { + c.gazelleManifestPath = gazelleManifestPath +} + // FindThirdPartyDependency scans the gazelle manifests for the current config // and the parent configs up to the root finding if it can resolve the module // name. func (c *Config) FindThirdPartyDependency(modName string) (string, string, bool) { for currentCfg := c; currentCfg != nil; currentCfg = currentCfg.parent { + // Attempt to load the manifest if needed. + if currentCfg.gazelleManifestPath != "" && currentCfg.gazelleManifest == nil { + currentCfgManifest, err := loadGazelleManifest(currentCfg.gazelleManifestPath) + if err != nil { + log.Fatal(err) + } + currentCfg.SetGazelleManifest(currentCfgManifest) + } + if currentCfg.gazelleManifest != nil { gazelleManifest := currentCfg.gazelleManifest if distributionName, ok := gazelleManifest.ModulesMapping[modName]; ok { @@ -526,3 +544,17 @@ func (c *Config) FormatThirdPartyDependency(repositoryName string, distributionN return label.New(repositoryName, normConventionalDistributionName, normConventionalDistributionName) } + +func loadGazelleManifest(gazelleManifestPath string) (*manifest.Manifest, error) { + if _, err := os.Stat(gazelleManifestPath); err != nil { + if os.IsNotExist(err) { + return nil, nil + } + return nil, fmt.Errorf("failed to load Gazelle manifest at %q: %w", gazelleManifestPath, err) + } + manifestFile := new(manifest.File) + if err := manifestFile.Decode(gazelleManifestPath); err != nil { + return nil, fmt.Errorf("failed to load Gazelle manifest at %q: %w", gazelleManifestPath, err) + } + return manifestFile.Manifest, nil +} From d0950c5648789071667b852a6d736cf865e2ff07 Mon Sep 17 00:00:00 2001 From: Ignas Anikevicius <240938+aignas@users.noreply.github.com> Date: Fri, 18 Apr 2025 07:00:05 +0900 Subject: [PATCH 172/922] fix(ci): use ubuntu-latest for mypy action (#2784) --- .github/workflows/mypy.yaml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.github/workflows/mypy.yaml b/.github/workflows/mypy.yaml index 866c43abd1..e774b9b03b 100644 --- a/.github/workflows/mypy.yaml +++ b/.github/workflows/mypy.yaml @@ -15,7 +15,7 @@ defaults: jobs: ci: - runs-on: ubuntu-20.04 + runs-on: ubuntu-latest steps: # Checkout the code - uses: actions/checkout@v4 From 183d2973060c653fc393209241b46e4ec807dd7b Mon Sep 17 00:00:00 2001 From: Ignas Anikevicius <240938+aignas@users.noreply.github.com> Date: Fri, 18 Apr 2025 08:43:27 +0900 Subject: [PATCH 173/922] doc: better document supported platform tiers (#2783) Fixes #2722. Related #2734, #2276, #1579 --- docs/support.md | 28 ++++++++++++++++++++++++++-- 1 file changed, 26 insertions(+), 2 deletions(-) diff --git a/docs/support.md b/docs/support.md index ea099650bd..5e6de57fcb 100644 --- a/docs/support.md +++ b/docs/support.md @@ -31,11 +31,35 @@ minor/patch versions. See [Bazel's release support matrix](https://bazel.build/release#support-matrix) for what versions are the rolling, active, and prior releases. +## Supported Python versions + +As a general rule we test all released non-EOL Python versions. Different +interpreter versions may work but are not guaranteed. We are interested in +staying compatible with upcoming unreleased versions, so if you see that things +stop working, please create tickets or, more preferably, pull requests. + ## Supported Platforms We only support the platforms that our continuous integration jobs run, which -is Linux, Mac, and Windows. Code to support other platforms is allowed, but -can only be on a best-effort basis. +is Linux, Mac, and Windows. + +In order to better describe different support levels, the below acts as a rough +guideline for different platform tiers: +* Tier 0 - The platforms that our CI runs: `linux_x86_64`, `osx_x86_64`, `RBE linux_x86_64`. +* Tier 1 - The platforms that are similar enough to what the CI runs: `linux_aarch64`, `osx_arm64`. + What is more, `windows_x86_64` is in this list as we run tests in CI but + developing for Windows is more challenging and features may come later to + this platform. +* Tier 2 - The rest of the platforms that may have varying level of support, e.g. + `linux_s390x`, `linux_ppc64le`, `windows_arm64`. + +:::{note} +Code to support Tier 2 platforms is allowed, but regressions will be fixed on a +best-effort basis, so feel free to contribute by creating PRs. + +If you would like to provide/sponsor CI setup for a platform that is not Tier 0, +please create a ticket or contact the maintainers on Slack. +::: ## Compatibility Policy From abdf560f56490beb43c1e4d72338f8553bc4d73f Mon Sep 17 00:00:00 2001 From: David Sanderson <32687193+dws@users.noreply.github.com> Date: Fri, 18 Apr 2025 16:04:22 -0400 Subject: [PATCH 174/922] fix(rules): copy_propagating_kwargs() now also copies target_compatible_with (#2788) This routine already copies `compatible_with`, which is little used, but does not copy `target_compatible_with`, which is broadly used. This seems like an oversight. I noticed this discrepancy when working on a system that assumes that any `tags` or `target_compatible_with` parameters supplied to a macro will propagate to all rules created by that macro. In rules_python, this already works for `tags`, but not for `target_compatible_with`. It would be great to get this accepted upstream, so that I can stop patching rules_python. --------- Co-authored-by: Richard Levasseur --- CHANGELOG.md | 2 ++ python/private/util.bzl | 2 +- 2 files changed, 3 insertions(+), 1 deletion(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 299a43e1ff..47ccd2459a 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -100,6 +100,8 @@ Unreleased changes template. * (toolchains) Ensure temporary `.pyc` and `.pyo` files are also excluded from the interpreters repository files. * (pypi) Run interpreter version call in isolated mode, to ensure it's not affected by userland environment variables, such as `PYTHONPATH`. * (packaging) An empty `requires_file` is treated as if it were omitted, resulting in a valid `METADATA` file. +* (rules) py_wheel and sphinxdocs rules now propagate `target_compatible_with` to all targets they create. + [PR #2788](https://github.com/bazel-contrib/rules_python/pull/2788). {#v0-0-0-added} ### Added diff --git a/python/private/util.bzl b/python/private/util.bzl index 33261befaf..4d2da57760 100644 --- a/python/private/util.bzl +++ b/python/private/util.bzl @@ -42,7 +42,7 @@ def copy_propagating_kwargs(from_kwargs, into_kwargs = None): into_kwargs = {} # Include tags because people generally expect tags to propagate. - for attr in ("testonly", "tags", "compatible_with", "restricted_to"): + for attr in ("testonly", "tags", "compatible_with", "restricted_to", "target_compatible_with"): if attr in from_kwargs and attr not in into_kwargs: into_kwargs[attr] = from_kwargs[attr] return into_kwargs From 844e7ada6738fc0e1f040df3c967e778af2af1c7 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Sat, 19 Apr 2025 20:18:40 -0700 Subject: [PATCH 175/922] release: 1.4.0 release prep (#2789) Updates changelog and version markers. Also updates the release docs with some shell-one liners to copy and paste to make it a bit more mechanical. --- CHANGELOG.md | 22 ++++++++++++---------- RELEASING.md | 21 +++++++++++++++++++++ python/current_py_toolchain.bzl | 2 +- python/features.bzl | 2 +- python/local_toolchains/repos.bzl | 2 +- python/packaging.bzl | 2 +- python/private/py_exec_tools_toolchain.bzl | 2 +- python/private/py_info.bzl | 2 +- python/private/py_library.bzl | 2 +- python/private/pypi/extension.bzl | 4 ++-- python/private/python.bzl | 8 ++++---- 11 files changed, 46 insertions(+), 23 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 47ccd2459a..1378853626 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -21,7 +21,7 @@ A brief description of the categories of changes: `(docs)`. -{#v0-0-0} -## Unreleased +{#1-4-0} +## [1.4.0] - 2025-04-19 -[0.0.0]: https://github.com/bazel-contrib/rules_python/releases/tag/0.0.0 +[1.4.0]: https://github.com/bazel-contrib/rules_python/releases/tag/1.4.0 -{#v0-0-0-changed} +{#1-4-0-changed} ### Changed * (toolchain) The `exec` configuration toolchain now has the forwarded `exec_interpreter` now also forwards the `ToolchainInfo` provider. This is @@ -72,7 +74,7 @@ Unreleased changes template. * (toolchains) Previously [#2636](https://github.com/bazel-contrib/rules_python/pull/2636) changed the semantics of `ignore_root_user_error` from "ignore" to "warning". This is now flipped back to ignoring the issue, and will only emit a warning when the attribute is set - `False`. + `False`. * (pypi) The PyPI extension will no longer write the lock file entries as the extension has been marked reproducible. Fixes [#2434](https://github.com/bazel-contrib/rules_python/issues/2434). @@ -84,7 +86,7 @@ Unreleased changes template. [20250317]: https://github.com/astral-sh/python-build-standalone/releases/tag/20250317 -{#v0-0-0-fixed} +{#1-4-0-fixed} ### Fixed * (pypi) Platform specific extras are now correctly handled when using universal lock files with environment markers. Fixes [#2690](https://github.com/bazel-contrib/rules_python/pull/2690). @@ -103,7 +105,7 @@ Unreleased changes template. * (rules) py_wheel and sphinxdocs rules now propagate `target_compatible_with` to all targets they create. [PR #2788](https://github.com/bazel-contrib/rules_python/pull/2788). -{#v0-0-0-added} +{#1-4-0-added} ### Added * (pypi) From now on `sha256` values in the `requirements.txt` is no longer mandatory when enabling {attr}`pip.parse.experimental_index_url` feature. @@ -134,13 +136,13 @@ Unreleased changes template. locations equivalents of `$(PYTHON2)` and `$(PYTHON3) respectively. -{#v0-0-0-removed} +{#1-4-0-removed} ### Removed * Nothing removed. {#v1-3-0} -## Unreleased +## [1.3.0] - 2025-03-27 [1.3.0]: https://github.com/bazel-contrib/rules_python/releases/tag/1.3.0 diff --git a/RELEASING.md b/RELEASING.md index 82510b99c7..c9d46c39f0 100644 --- a/RELEASING.md +++ b/RELEASING.md @@ -14,7 +14,14 @@ These are the steps for a regularly scheduled release from HEAD. 1. [Determine the next semantic version number](#determining-semantic-version). 1. Update CHANGELOG.md: replace the `v0-0-0` and `0.0.0` with `X.Y.0`. + ``` + awk -v version=X.Y.0 'BEGIN { hv=version; gsub(/\./, "-", hv) } /END_UNRELEASED_TEMPLATE/ { found_marker = 1 } found_marker { gsub(/v0-0-0/, hv, $0); gsub(/Unreleased/, "[" version "] - " strftime("%Y-%m-%d"), $0); gsub(/0.0.0/, version, $0); } { print } ' CHANGELOG.md > /tmp/changelog && cp /tmp/changelog CHANGELOG.md + ``` 1. Replace `VERSION_NEXT_*` strings with `X.Y.0`. + ``` + grep -l --exclude=CONTRIBUTING.md --exclude=RELEASING.md --exclude-dir=.* VERSION_NEXT_ -r \ + | xargs sed -i -e 's/VERSION_NEXT_FEATURE/X.Y.0/' -e 's/VERSION_NEXT_PATCH/X.Y.0/' + ``` 1. Send these changes for review and get them merged. 1. Create a branch for the new release, named `release/X.Y` ``` @@ -90,6 +97,20 @@ It will be promoted to stable next week, pending feedback. It's traditional to include notable changes from the changelog, but not required. +### Re-releasing a version + +Re-releasing a version (i.e. changing the commit a tag points to) is +*sometimes* possible, but it depends on how far into the release process it got. + +The two points of no return are: + * If the PyPI package has been published: PyPI disallows using the same + filename/version twice. Once published, it cannot be replaced. + * If the BCR package has been published: Once it's been committed to the BCR + registry, it cannot be replaced. + +If release steps fail _prior_ to those steps, then its OK to change the tag. You +may need to manually delete the GitHub release. + ## Secrets ### PyPI user rules-python diff --git a/python/current_py_toolchain.bzl b/python/current_py_toolchain.bzl index f5c5638a88..0ca5c90ccc 100644 --- a/python/current_py_toolchain.bzl +++ b/python/current_py_toolchain.bzl @@ -52,7 +52,7 @@ current_py_toolchain = rule( happened, to a rule which expects a concrete implementation of a toolchain, rather than a toolchain_type which could be resolved to that toolchain. - :::{versionchanged} VERSION_NEXT_FEATURE + :::{versionchanged} 1.4.0 From now on, we also expose `$(PYTHON2_ROOTPATH)` and `$(PYTHON3_ROOTPATH)` which are runfiles locations equivalents of `$(PYTHON2)` and `$(PYTHON3) respectively. ::: diff --git a/python/features.bzl b/python/features.bzl index 8edfb698fc..917bd3800c 100644 --- a/python/features.bzl +++ b/python/features.bzl @@ -35,7 +35,7 @@ def _features_typedef(): True if the `PyInfo.site_packages_symlinks` field is available. - :::{versionadded} VERSION_NEXT_FEATURE + :::{versionadded} 1.4.0 ::: :::: diff --git a/python/local_toolchains/repos.bzl b/python/local_toolchains/repos.bzl index d1b45cfd7f..320e503e1a 100644 --- a/python/local_toolchains/repos.bzl +++ b/python/local_toolchains/repos.bzl @@ -1,6 +1,6 @@ """Rules/macros for repository phase for local toolchains. -:::{versionadded} VERSION_NEXT_FEATURE +:::{versionadded} 1.4.0 ::: """ diff --git a/python/packaging.bzl b/python/packaging.bzl index b190635cfe..223aba142d 100644 --- a/python/packaging.bzl +++ b/python/packaging.bzl @@ -101,7 +101,7 @@ def py_wheel( Currently only pure-python wheels are supported. - :::{versionchanged} VERSION_NEXT_FEATURE + :::{versionchanged} 1.4.0 From now on, an empty `requires_file` is treated as if it were omitted, resulting in a valid `METADATA` file. ::: diff --git a/python/private/py_exec_tools_toolchain.bzl b/python/private/py_exec_tools_toolchain.bzl index ff30431ff4..332570b26b 100644 --- a/python/private/py_exec_tools_toolchain.bzl +++ b/python/private/py_exec_tools_toolchain.bzl @@ -77,7 +77,7 @@ handle all the necessary transitions and runtime setup to invoke a program. See {obj}`PyExecToolsInfo.exec_interpreter` for further docs. -:::{versionchanged} VERSION_NEXT_FEATURE +:::{versionchanged} 1.4.0 From now on the provided target also needs to provide `platform_common.ToolchainInfo` so that the toolchain `py_runtime` field can be correctly forwarded. ::: diff --git a/python/private/py_info.bzl b/python/private/py_info.bzl index 4ecd02a438..dc3cb24c51 100644 --- a/python/private/py_info.bzl +++ b/python/private/py_info.bzl @@ -168,7 +168,7 @@ values from further way dependencies, such as forcing symlinks to point to specific paths or preventing symlinks from being created. ::: -:::{versionadded} VERSION_NEXT_FEATURE +:::{versionadded} 1.4.0 ::: """, "transitive_implicit_pyc_files": """ diff --git a/python/private/py_library.bzl b/python/private/py_library.bzl index edd0db579f..6b5882de5a 100644 --- a/python/private/py_library.bzl +++ b/python/private/py_library.bzl @@ -94,7 +94,7 @@ to a consumer have precedence. See {obj}`PyInfo.site_packages_symlinks` for more information. ::: -:::{versionadded} VERSION_NEXT_FEATURE +:::{versionadded} 1.4.0 ::: """, ), diff --git a/python/private/pypi/extension.bzl b/python/private/pypi/extension.bzl index d2ae132741..68776e32d0 100644 --- a/python/private/pypi/extension.bzl +++ b/python/private/pypi/extension.bzl @@ -686,7 +686,7 @@ If {attr}`download_only` is set, then `sdist` archives will be discarded and `pi operate in wheel-only mode. ::: -:::{versionchanged} VERSION_NEXT_FEATURE +:::{versionchanged} 1.4.0 Index metadata will be used to deduct `sha256` values for packages even if the `sha256` values are not present in the requirements.txt lock file. ::: @@ -767,7 +767,7 @@ to `rules_python` and use this attribute until the bug is fixed. EXPERIMENTAL: this may be removed without notice. -:::{versionadded} VERSION_NEXT_FEATURE +:::{versionadded} 1.4.0 ::: """, ), diff --git a/python/private/python.bzl b/python/private/python.bzl index efc429420e..f49fb26d52 100644 --- a/python/private/python.bzl +++ b/python/private/python.bzl @@ -695,7 +695,7 @@ matches the {attr}`python_version` attribute of a toolchain, this toolchain is the default version. If this attribute is set, the {attr}`is_default` attribute of the toolchain is ignored. -:::{versionadded} VERSION_NEXT_FEATURE +:::{versionadded} 1.4.0 ::: """, ), @@ -707,7 +707,7 @@ If the string matches the {attr}`python_version` attribute of a toolchain, this toolchain is the default version. If this attribute is set, the {attr}`is_default` attribute of the toolchain is ignored. -:::{versionadded} VERSION_NEXT_FEATURE +:::{versionadded} 1.4.0 ::: """, ), @@ -720,7 +720,7 @@ of the file match the {attr}`python_version` attribute of a toolchain, this toolchain is the default version. If this attribute is set, the {attr}`is_default` attribute of the toolchain is ignored. -:::{versionadded} VERSION_NEXT_FEATURE +:::{versionadded} 1.4.0 ::: """, ), @@ -813,7 +813,7 @@ this to `False`. doc = """\ Whether the toolchain is the default version. -:::{versionchanged} VERSION_NEXT_FEATURE +:::{versionchanged} 1.4.0 This setting is ignored if the default version is set using the `defaults` tag class. ::: From cc46fb26d629b9e440371861f031cb2a85fd9c55 Mon Sep 17 00:00:00 2001 From: Guillaume Maudoux Date: Sun, 20 Apr 2025 08:05:13 +0200 Subject: [PATCH 176/922] fix: declare PyInfo as provided by test/binary/library (#2777) Currently, the rules don't advertise the PyInfo provider through the provides argument to the rule function. This means that aspects that want to consume PyInfo can't use `required_providers` to restrict themselves to the Python rules, and instead have to apply to all rules. To fix, add PyInfo to the provides arg of the rules. Fixes https://github.com/bazel-contrib/rules_python/issues/2506 --------- Co-authored-by: Richard Levasseur Co-authored-by: Richard Levasseur --- CHANGELOG.md | 22 ++++++++++++++++++++++ python/private/py_executable.bzl | 4 +++- python/private/py_library.bzl | 5 +++++ 3 files changed, 30 insertions(+), 1 deletion(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 1378853626..cad074e6a6 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -47,6 +47,28 @@ BEGIN_UNRELEASED_TEMPLATE END_UNRELEASED_TEMPLATE --> +{#v0-0-0} +## Unreleased + +[0.0.0]: https://github.com/bazel-contrib/rules_python/releases/tag/0.0.0 + +{#v0-0-0-changed} +### Changed +* Nothing changed. + +{#v0-0-0-fixed} +### Fixed +* (rules) PyInfo provider is now advertised by py_test, py_binary, and py_library; + this allows aspects using required_providers to function correctly. + ([#2506](https://github.com/bazel-contrib/rules_python/issues/2506)). + +{#v0-0-0-added} +### Added +* Nothing added. + +{#v0-0-0-removed} +### Removed +* Nothing removed. {#1-4-0} ## [1.4.0] - 2025-04-19 diff --git a/python/private/py_executable.bzl b/python/private/py_executable.bzl index dd3ad869fa..b4cda21b1d 100644 --- a/python/private/py_executable.bzl +++ b/python/private/py_executable.bzl @@ -1854,6 +1854,8 @@ def create_base_executable_rule(): """ return create_executable_rule_builder().build() +_MaybeBuiltinPyInfo = [BuiltinPyInfo] if BuiltinPyInfo != None else [] + # NOTE: Exported publicly def create_executable_rule_builder(implementation, **kwargs): """Create a rule builder for an executable Python program. @@ -1877,7 +1879,7 @@ def create_executable_rule_builder(implementation, **kwargs): attrs = EXECUTABLE_ATTRS, exec_groups = dict(REQUIRED_EXEC_GROUP_BUILDERS), # Mutable copy fragments = ["py", "bazel_py"], - provides = [PyExecutableInfo], + provides = [PyExecutableInfo, PyInfo] + _MaybeBuiltinPyInfo, toolchains = [ ruleb.ToolchainType(TOOLCHAIN_TYPE), ruleb.ToolchainType(EXEC_TOOLS_TOOLCHAIN_TYPE, mandatory = False), diff --git a/python/private/py_library.bzl b/python/private/py_library.bzl index 6b5882de5a..bf0c25439e 100644 --- a/python/private/py_library.bzl +++ b/python/private/py_library.bzl @@ -43,7 +43,9 @@ load( load(":flags.bzl", "AddSrcsToRunfilesFlag", "PrecompileFlag", "VenvsSitePackages") load(":precompile.bzl", "maybe_precompile") load(":py_cc_link_params_info.bzl", "PyCcLinkParamsInfo") +load(":py_info.bzl", "PyInfo") load(":py_internal.bzl", "py_internal") +load(":reexports.bzl", "BuiltinPyInfo") load(":rule_builders.bzl", "ruleb") load( ":toolchain_types.bzl", @@ -299,6 +301,8 @@ def _repo_relative_short_path(short_path): else: return short_path +_MaybeBuiltinPyInfo = [BuiltinPyInfo] if BuiltinPyInfo != None else [] + # NOTE: Exported publicaly def create_py_library_rule_builder(): """Create a rule builder for a py_library. @@ -319,6 +323,7 @@ def create_py_library_rule_builder(): exec_groups = dict(REQUIRED_EXEC_GROUP_BUILDERS), attrs = LIBRARY_ATTRS, fragments = ["py"], + provides = [PyCcLinkParamsInfo, PyInfo] + _MaybeBuiltinPyInfo, toolchains = [ ruleb.ToolchainType(TOOLCHAIN_TYPE, mandatory = False), ruleb.ToolchainType(EXEC_TOOLS_TOOLCHAIN_TYPE, mandatory = False), From a19e1e41a609dd10ae6cdc49d76eb1f119145d2e Mon Sep 17 00:00:00 2001 From: Ignas Anikevicius <240938+aignas@users.noreply.github.com> Date: Sun, 20 Apr 2025 19:17:59 +0900 Subject: [PATCH 177/922] fix: load target_platforms through the hub (#2781) This PR moves the parsing of `Requires-Dist` to the loading phase within the `whl_library_targets_from_requires` macro. The original `whl_library_targets` macro has been left unchanged so that I don't have to reinvent the unit tests - it is well covered under tests. Before this PR we had to wire the `target_platforms` via the `experimental_target_platforms` attr in the `whl_library`, which means that whenever this would change (e.g. the minor Python version changes), the wheel would be re-extracted even though the final result may be the same. This refactor uncovered that the dependency graph creation was incorrect if we had multiple target Python versions due to various heuristics that this had. In hindsight I had them to make the generated `BUILD.bazel` files more readable when the unit test coverage was not great. Now this is unnecessary and since everything is happening in Starlark I thought that having a simpler algorithm that does the right thing always is the best way. This also cleans up the code by removing left over TODO notes or code that no longer make sense. Work towards #260, #2319 --- CHANGELOG.md | 7 + config.bzl.tmpl.bzlmod | 0 python/private/pypi/BUILD.bazel | 14 +- python/private/pypi/attrs.bzl | 3 + python/private/pypi/config.bzl.tmpl.bzlmod | 9 + python/private/pypi/extension.bzl | 41 ++-- .../pypi/generate_whl_library_build_bazel.bzl | 27 +- python/private/pypi/hub_repository.bzl | 18 +- python/private/pypi/pep508.bzl | 23 -- python/private/pypi/pep508_deps.bzl | 231 ++++-------------- python/private/pypi/pep508_requirement.bzl | 4 +- python/private/pypi/whl_library.bzl | 97 +++----- python/private/pypi/whl_library_targets.bzl | 83 +++++++ tests/pypi/extension/extension_tests.bzl | 10 - ...generate_whl_library_build_bazel_tests.bzl | 92 +++++-- tests/pypi/pep508/deps_tests.bzl | 191 ++++++--------- .../whl_library_targets_tests.bzl | 67 ++++- 17 files changed, 451 insertions(+), 466 deletions(-) create mode 100644 config.bzl.tmpl.bzlmod create mode 100644 python/private/pypi/config.bzl.tmpl.bzlmod delete mode 100644 python/private/pypi/pep508.bzl diff --git a/CHANGELOG.md b/CHANGELOG.md index cad074e6a6..154b66114b 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -105,6 +105,13 @@ END_UNRELEASED_TEMPLATE [PR #2746](https://github.com/bazel-contrib/rules_python/pull/2746). * (rules) {attr}`py_binary.srcs` and {attr}`py_test.srcs` is no longer mandatory when `main_module` is specified (for `--bootstrap_impl=script`) +* (pypi) From now on the `Requires-Dist` from the wheel metadata is analysed in + the loading phase instead of repository rule phase giving better caching + performance when the target platforms are changed (e.g. target python + versions). This is preparatory work for stabilizing the cross-platform wheel + support. From now on the usage of `experimental_target_platforms` should be + avoided and the `requirements_by_platform` values should be instead used to + specify the target platforms for the given dependencies. [20250317]: https://github.com/astral-sh/python-build-standalone/releases/tag/20250317 diff --git a/config.bzl.tmpl.bzlmod b/config.bzl.tmpl.bzlmod new file mode 100644 index 0000000000..e69de29bb2 diff --git a/python/private/pypi/BUILD.bazel b/python/private/pypi/BUILD.bazel index 7297238cb4..a758b3f153 100644 --- a/python/private/pypi/BUILD.bazel +++ b/python/private/pypi/BUILD.bazel @@ -212,15 +212,6 @@ bzl_library( ], ) -bzl_library( - name = "pep508_bzl", - srcs = ["pep508.bzl"], - deps = [ - ":pep508_env_bzl", - ":pep508_evaluate_bzl", - ], -) - bzl_library( name = "pep508_deps_bzl", srcs = ["pep508_deps.bzl"], @@ -378,13 +369,12 @@ bzl_library( ":attrs_bzl", ":deps_bzl", ":generate_whl_library_build_bazel_bzl", - ":parse_whl_name_bzl", ":patch_whl_bzl", - ":pep508_deps_bzl", + ":pep508_requirement_bzl", ":pypi_repo_utils_bzl", ":whl_metadata_bzl", - ":whl_target_platforms_bzl", "//python/private:auth_bzl", + "//python/private:bzlmod_enabled_bzl", "//python/private:envsubst_bzl", "//python/private:is_standalone_interpreter_bzl", "//python/private:repo_utils_bzl", diff --git a/python/private/pypi/attrs.bzl b/python/private/pypi/attrs.bzl index 9d88c1e32c..fe35d8bf7d 100644 --- a/python/private/pypi/attrs.bzl +++ b/python/private/pypi/attrs.bzl @@ -123,6 +123,9 @@ Warning: "experimental_target_platforms": attr.string_list( default = [], doc = """\ +*NOTE*: This will be removed in the next major version, so please consider migrating +to `bzlmod` and rely on {attr}`pip.parse.requirements_by_platform` for this feature. + A list of platforms that we will generate the conditional dependency graph for cross platform wheels by parsing the wheel metadata. This will generate the correct dependencies for packages like `sphinx` or `pylint`, which include diff --git a/python/private/pypi/config.bzl.tmpl.bzlmod b/python/private/pypi/config.bzl.tmpl.bzlmod new file mode 100644 index 0000000000..deb53631d1 --- /dev/null +++ b/python/private/pypi/config.bzl.tmpl.bzlmod @@ -0,0 +1,9 @@ +"""Extra configuration values that are exposed from the hub repository for spoke repositories to access. + +NOTE: This is internal `rules_python` API and if you would like to depend on it, please raise an issue +with your usecase. This may change in between rules_python versions without any notice. + +@generated by rules_python pip.parse bzlmod extension. +""" + +target_platforms = %%TARGET_PLATFORMS%% diff --git a/python/private/pypi/extension.bzl b/python/private/pypi/extension.bzl index 68776e32d0..d1895ca211 100644 --- a/python/private/pypi/extension.bzl +++ b/python/private/pypi/extension.bzl @@ -32,7 +32,6 @@ load(":simpleapi_download.bzl", "simpleapi_download") load(":whl_config_setting.bzl", "whl_config_setting") load(":whl_library.bzl", "whl_library") load(":whl_repo_name.bzl", "pypi_repo_name", "whl_repo_name") -load(":whl_target_platforms.bzl", "whl_target_platforms") def _major_minor_version(version): version = semver(version) @@ -68,7 +67,6 @@ def _create_whl_repos( *, pip_attr, whl_overrides, - evaluate_markers = evaluate_markers, available_interpreters = INTERPRETER_LABELS, get_index_urls = None): """create all of the whl repositories @@ -77,7 +75,6 @@ def _create_whl_repos( module_ctx: {type}`module_ctx`. pip_attr: {type}`struct` - the struct that comes from the tag class iteration. whl_overrides: {type}`dict[str, struct]` - per-wheel overrides. - evaluate_markers: the function to use to evaluate markers. get_index_urls: A function used to get the index URLs available_interpreters: {type}`dict[str, Label]` The dictionary of available interpreters that have been registered using the `python` bzlmod extension. @@ -162,14 +159,12 @@ def _create_whl_repos( requirements_osx = pip_attr.requirements_darwin, requirements_windows = pip_attr.requirements_windows, extra_pip_args = pip_attr.extra_pip_args, + # TODO @aignas 2025-04-15: pass the full version into here python_version = major_minor, logger = logger, ), extra_pip_args = pip_attr.extra_pip_args, get_index_urls = get_index_urls, - # NOTE @aignas 2025-02-24: we will use the "cp3xx_os_arch" platform labels - # for converting to the PEP508 environment and will evaluate them in starlark - # without involving the interpreter at all. evaluate_markers = evaluate_markers, logger = logger, ) @@ -191,7 +186,6 @@ def _create_whl_repos( enable_implicit_namespace_pkgs = pip_attr.enable_implicit_namespace_pkgs, environment = pip_attr.environment, envsubst = pip_attr.envsubst, - experimental_target_platforms = pip_attr.experimental_target_platforms, group_deps = group_deps, group_name = group_name, pip_data_exclude = pip_attr.pip_data_exclude, @@ -244,6 +238,12 @@ def _create_whl_repos( }, extra_aliases = extra_aliases, whl_libraries = whl_libraries, + target_platforms = { + plat: None + for reqs in requirements_by_platform.values() + for req in reqs + for plat in req.target_platforms + }, ) def _whl_repos(*, requirement, whl_library_args, download_only, netrc, auth_patterns, multiple_requirements_for_whl = False, python_version): @@ -274,20 +274,11 @@ def _whl_repos(*, requirement, whl_library_args, download_only, netrc, auth_patt args["urls"] = [distribution.url] args["sha256"] = distribution.sha256 args["filename"] = distribution.filename - args["experimental_target_platforms"] = requirement.target_platforms # Pure python wheels or sdists may need to have a platform here target_platforms = None if distribution.filename.endswith(".whl") and not distribution.filename.endswith("-any.whl"): - parsed_whl = parse_whl_name(distribution.filename) - whl_platforms = whl_target_platforms( - platform_tag = parsed_whl.platform_tag, - ) - args["experimental_target_platforms"] = [ - p - for p in requirement.target_platforms - if [None for wp in whl_platforms if p.endswith(wp.target_platform)] - ] + pass elif multiple_requirements_for_whl: target_platforms = requirement.target_platforms @@ -416,6 +407,7 @@ You cannot use both the additive_build_content and additive_build_content_file a hub_group_map = {} exposed_packages = {} extra_aliases = {} + target_platforms = {} whl_libraries = {} for mod in module_ctx.modules: @@ -498,6 +490,7 @@ You cannot use both the additive_build_content and additive_build_content_file a for whl_name, aliases in out.extra_aliases.items(): extra_aliases[hub_name].setdefault(whl_name, {}).update(aliases) exposed_packages.setdefault(hub_name, {}).update(out.exposed_packages) + target_platforms.setdefault(hub_name, {}).update(out.target_platforms) whl_libraries.update(out.whl_libraries) # TODO @aignas 2024-04-05: how do we support different requirement @@ -535,6 +528,10 @@ You cannot use both the additive_build_content and additive_build_content_file a } for hub_name, extra_whl_aliases in extra_aliases.items() }, + target_platforms = { + hub_name: sorted(p) + for hub_name, p in target_platforms.items() + }, whl_libraries = { k: dict(sorted(args.items())) for k, args in sorted(whl_libraries.items()) @@ -626,15 +623,13 @@ def _pip_impl(module_ctx): }, packages = mods.exposed_packages.get(hub_name, []), groups = mods.hub_group_map.get(hub_name), + target_platforms = mods.target_platforms.get(hub_name, []), ) if bazel_features.external_deps.extension_metadata_has_reproducible: - # If we are not using the `experimental_index_url feature, the extension is fully - # deterministic and we don't need to create a lock entry for it. - # - # In order to be able to dogfood the `experimental_index_url` feature before it gets - # stabilized, we have created the `_pip_non_reproducible` function, that will result - # in extra entries in the lock file. + # NOTE @aignas 2025-04-15: this is set to be reproducible, because the + # results after calling the PyPI index should be reproducible on each + # machine. return module_ctx.extension_metadata(reproducible = True) else: return None diff --git a/python/private/pypi/generate_whl_library_build_bazel.bzl b/python/private/pypi/generate_whl_library_build_bazel.bzl index 8050cd22ad..7988aca1c4 100644 --- a/python/private/pypi/generate_whl_library_build_bazel.bzl +++ b/python/private/pypi/generate_whl_library_build_bazel.bzl @@ -21,23 +21,23 @@ _RENDER = { "copy_files": render.dict, "data": render.list, "data_exclude": render.list, - "dependencies": render.list, - "dependencies_by_platform": lambda x: render.dict(x, value_repr = render.list), "entry_points": render.dict, + "extras": render.list, "group_deps": render.list, + "requires_dist": render.list, "srcs_exclude": render.list, - "tags": render.list, + "target_platforms": lambda x: render.list(x) if x else "target_platforms", } # NOTE @aignas 2024-10-25: We have to keep this so that files in # this repository can be publicly visible without the need for # export_files _TEMPLATE = """\ -load("@rules_python//python/private/pypi:whl_library_targets.bzl", "whl_library_targets") +{loads} package(default_visibility = ["//visibility:public"]) -whl_library_targets( +whl_library_targets_from_requires( {kwargs} ) """ @@ -45,11 +45,13 @@ whl_library_targets( def generate_whl_library_build_bazel( *, annotation = None, + default_python_version = None, **kwargs): """Generate a BUILD file for an unzipped Wheel Args: annotation: The annotation for the build file. + default_python_version: The python version to use to parse the METADATA. **kwargs: Extra args serialized to be passed to the {obj}`whl_library_targets`. @@ -57,6 +59,18 @@ def generate_whl_library_build_bazel( A complete BUILD file as a string """ + loads = [ + """load("@rules_python//python/private/pypi:whl_library_targets.bzl", "whl_library_targets_from_requires")""", + ] + if not kwargs.setdefault("target_platforms", None): + dep_template = kwargs["dep_template"] + loads.append( + "load(\"{}\", \"{}\")".format( + dep_template.format(name = "", target = "config.bzl"), + "target_platforms", + ), + ) + additional_content = [] if annotation: kwargs["data"] = annotation.data @@ -66,10 +80,13 @@ def generate_whl_library_build_bazel( kwargs["srcs_exclude"] = annotation.srcs_exclude_glob if annotation.additive_build_content: additional_content.append(annotation.additive_build_content) + if default_python_version: + kwargs["default_python_version"] = default_python_version contents = "\n".join( [ _TEMPLATE.format( + loads = "\n".join(loads), kwargs = render.indent("\n".join([ "{} = {},".format(k, _RENDER.get(k, repr)(v)) for k, v in sorted(kwargs.items()) diff --git a/python/private/pypi/hub_repository.bzl b/python/private/pypi/hub_repository.bzl index 48245b4106..d2cbf88c24 100644 --- a/python/private/pypi/hub_repository.bzl +++ b/python/private/pypi/hub_repository.bzl @@ -45,7 +45,14 @@ def _impl(rctx): macro_tmpl = "@@{name}//{{}}:{{}}".format(name = rctx.attr.name) rctx.file("BUILD.bazel", _BUILD_FILE_CONTENTS) - rctx.template("requirements.bzl", rctx.attr._template, substitutions = { + rctx.template( + "config.bzl", + rctx.attr._config_template, + substitutions = { + "%%TARGET_PLATFORMS%%": render.list(rctx.attr.target_platforms), + }, + ) + rctx.template("requirements.bzl", rctx.attr._requirements_bzl_template, substitutions = { "%%ALL_DATA_REQUIREMENTS%%": render.list([ macro_tmpl.format(p, "data") for p in bzl_packages @@ -80,6 +87,10 @@ The list of packages that will be exposed via all_*requirements macros. Defaults mandatory = True, doc = "The apparent name of the repo. This is needed because in bzlmod, the name attribute becomes the canonical name.", ), + "target_platforms": attr.string_list( + mandatory = True, + doc = "All of the target platforms for the hub repo", + ), "whl_map": attr.string_dict( mandatory = True, doc = """\ @@ -87,7 +98,10 @@ The wheel map where values are json.encoded strings of the whl_map constructed in the pip.parse tag class. """, ), - "_template": attr.label( + "_config_template": attr.label( + default = ":config.bzl.tmpl.bzlmod", + ), + "_requirements_bzl_template": attr.label( default = ":requirements.bzl.tmpl.bzlmod", ), }, diff --git a/python/private/pypi/pep508.bzl b/python/private/pypi/pep508.bzl deleted file mode 100644 index e74352def2..0000000000 --- a/python/private/pypi/pep508.bzl +++ /dev/null @@ -1,23 +0,0 @@ -# Copyright 2025 The Bazel Authors. All rights reserved. -# -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -"""This module is for implementing PEP508 in starlark as FeatureFlagInfo -""" - -load(":pep508_env.bzl", _env = "env") -load(":pep508_evaluate.bzl", _evaluate = "evaluate", _to_string = "to_string") - -to_string = _to_string -evaluate = _evaluate -env = _env diff --git a/python/private/pypi/pep508_deps.bzl b/python/private/pypi/pep508_deps.bzl index af0a75362b..115bbd78d8 100644 --- a/python/private/pypi/pep508_deps.bzl +++ b/python/private/pypi/pep508_deps.bzl @@ -15,36 +15,24 @@ """This module is for implementing PEP508 compliant METADATA deps parsing. """ +load("@pythons_hub//:versions.bzl", "DEFAULT_PYTHON_VERSION") load("//python/private:normalize_name.bzl", "normalize_name") load(":pep508_env.bzl", "env") load(":pep508_evaluate.bzl", "evaluate") load(":pep508_platform.bzl", "platform", "platform_from_str") load(":pep508_requirement.bzl", "requirement") -_ALL_OS_VALUES = [ - "windows", - "osx", - "linux", -] -_ALL_ARCH_VALUES = [ - "aarch64", - "ppc64", - "ppc64le", - "s390x", - "x86_32", - "x86_64", -] - -def deps(name, *, requires_dist, platforms = [], extras = [], host_python_version = None): +def deps(name, *, requires_dist, platforms = [], extras = [], excludes = [], default_python_version = None): """Parse the RequiresDist from wheel METADATA Args: name: {type}`str` the name of the wheel. requires_dist: {type}`list[str]` the list of RequiresDist lines from the METADATA file. + excludes: {type}`list[str]` what packages should we exclude. extras: {type}`list[str]` the requested extras to generate targets for. platforms: {type}`list[str]` the list of target platform strings. - host_python_version: {type}`str` the host python version. + default_python_version: {type}`str` the host python version. Returns: A struct with attributes: @@ -62,18 +50,17 @@ def deps(name, *, requires_dist, platforms = [], extras = [], host_python_versio want_extras = _resolve_extras(name, reqs, extras) # drop self edges - reqs = [r for r in reqs if r.name != name] + excludes = [name] + [normalize_name(x) for x in excludes] + default_python_version = default_python_version or DEFAULT_PYTHON_VERSION platforms = [ - platform_from_str(p, python_version = host_python_version) + platform_from_str(p, python_version = default_python_version) for p in platforms - ] or [ - platform_from_str("", python_version = host_python_version), ] abis = sorted({p.abi: True for p in platforms if p.abi}) - if host_python_version and len(abis) > 1: - _, _, minor_version = host_python_version.partition(".") + if default_python_version and len(abis) > 1: + _, _, minor_version = default_python_version.partition(".") minor_version, _, _ = minor_version.partition(".") default_abi = "cp3" + minor_version elif len(abis) > 1: @@ -83,11 +70,20 @@ def deps(name, *, requires_dist, platforms = [], extras = [], host_python_versio else: default_abi = None + reqs_by_name = {} + for req in reqs: - _add_req( + if req.name_ in excludes: + continue + + reqs_by_name.setdefault(req.name, []).append(req) + + for name, reqs in reqs_by_name.items(): + _add_reqs( deps, deps_select, - req, + normalize_name(name), + reqs, extras = want_extras, platforms = platforms, default_abi = default_abi, @@ -103,49 +99,14 @@ def deps(name, *, requires_dist, platforms = [], extras = [], host_python_versio def _platform_str(self): if self.abi == None: - if not self.os and not self.arch: - return "//conditions:default" - elif not self.arch: - return "@platforms//os:{}".format(self.os) - else: - return "{}_{}".format(self.os, self.arch) + return "{}_{}".format(self.os, self.arch) - minor_version = self.abi[3:] - if self.arch == None and self.os == None: - return str(Label("//python/config_settings:is_python_3.{}".format(minor_version))) - - return "cp3{}_{}_{}".format( - minor_version, + return "{}_{}_{}".format( + self.abi, self.os or "anyos", self.arch or "anyarch", ) -def _platform_specializations(self, cpu_values = _ALL_ARCH_VALUES, os_values = _ALL_OS_VALUES): - """Return the platform itself and all its unambiguous specializations. - - For more info about specializations see - https://bazel.build/docs/configurable-attributes - """ - specializations = [] - specializations.append(self) - if self.arch == None: - specializations.extend([ - platform(os = self.os, arch = arch, abi = self.abi) - for arch in cpu_values - ]) - if self.os == None: - specializations.extend([ - platform(os = os, arch = self.arch, abi = self.abi) - for os in os_values - ]) - if self.os == None and self.arch == None: - specializations.extend([ - platform(os = os, arch = arch, abi = self.abi) - for os in os_values - for arch in cpu_values - ]) - return specializations - def _add(deps, deps_select, dep, platform): dep = normalize_name(dep) @@ -172,53 +133,7 @@ def _add(deps, deps_select, dep, platform): return # Add the platform-specific branch - deps_select.setdefault(platform, {}) - - # Add the dep to specializations of the given platform if they - # exist in the select statement. - for p in _platform_specializations(platform): - if p not in deps_select: - continue - - deps_select[p][dep] = True - - if len(deps_select[platform]) == 1: - # We are adding a new item to the select and we need to ensure that - # existing dependencies from less specialized platforms are propagated - # to the newly added dependency set. - for p, _deps in deps_select.items(): - # Check if the existing platform overlaps with the given platform - if p == platform or platform not in _platform_specializations(p): - continue - - deps_select[platform].update(_deps) - -def _maybe_add_common_dep(deps, deps_select, platforms, dep): - abis = sorted({p.abi: True for p in platforms if p.abi}) - if len(abis) < 2: - return - - platforms = [platform()] + [ - platform(abi = abi) - for abi in abis - ] - - # If the dep is targeting all target python versions, lets add it to - # the common dependency list to simplify the select statements. - for p in platforms: - if p not in deps_select: - return - - if dep not in deps_select[p]: - return - - # All of the python version-specific branches have the dep, so lets add - # it to the common deps. - deps[dep] = True - for p in platforms: - deps_select[p].pop(dep) - if not deps_select[p]: - deps_select.pop(p) + deps_select.setdefault(platform, {})[dep] = True def _resolve_extras(self_name, reqs, extras): """Resolve extras which are due to depending on self[some_other_extra]. @@ -275,77 +190,37 @@ def _resolve_extras(self_name, reqs, extras): # Poor mans set return sorted({x: None for x in extras}) -def _add_req(deps, deps_select, req, *, extras, platforms, default_abi = None): - if not req.marker: - _add(deps, deps_select, req.name, None) - return - - # NOTE @aignas 2023-12-08: in order to have reasonable select statements - # we do have to have some parsing of the markers, so it begs the question - # if packaging should be reimplemented in Starlark to have the best solution - # for now we will implement it in Python and see what the best parsing result - # can be before making this decision. - match_os = len([ - tag - for tag in [ - "os_name", - "sys_platform", - "platform_system", - ] - if tag in req.marker - ]) > 0 - match_arch = "platform_machine" in req.marker - match_version = "version" in req.marker - - if not (match_os or match_arch or match_version): - if [ - True - for extra in extras - for p in platforms - if evaluate( - req.marker, - env = env( - target_platform = p, - extra = extra, - ), - ) - ]: - _add(deps, deps_select, req.name, None) - return +def _add_reqs(deps, deps_select, dep, reqs, *, extras, platforms, default_abi = None): + for req in reqs: + if not req.marker: + _add(deps, deps_select, dep, None) + return + platforms_to_add = {} for plat in platforms: - if not [ - True - for extra in extras - if evaluate( - req.marker, - env = env( - target_platform = plat, - extra = extra, - ), - ) - ]: + if plat in platforms_to_add: + # marker evaluation is more expensive than this check continue - if match_arch and default_abi: - _add(deps, deps_select, req.name, plat) - if plat.abi == default_abi: - _add(deps, deps_select, req.name, platform(os = plat.os, arch = plat.arch)) - elif match_arch: - _add(deps, deps_select, req.name, platform(os = plat.os, arch = plat.arch)) - elif match_os and default_abi: - _add(deps, deps_select, req.name, platform(os = plat.os, abi = plat.abi)) - if plat.abi == default_abi: - _add(deps, deps_select, req.name, platform(os = plat.os)) - elif match_os: - _add(deps, deps_select, req.name, platform(os = plat.os)) - elif match_version and default_abi: - _add(deps, deps_select, req.name, platform(abi = plat.abi)) - if plat.abi == default_abi: - _add(deps, deps_select, req.name, platform()) - elif match_version: - _add(deps, deps_select, req.name, None) - else: - fail("BUG: {} support is not implemented".format(req.marker)) + added = False + for extra in extras: + if added: + break + + for req in reqs: + if evaluate(req.marker, env = env(target_platform = plat, extra = extra)): + platforms_to_add[plat] = True + added = True + break + + if len(platforms_to_add) == len(platforms): + # the dep is in all target platforms, let's just add it to the regular + # list + _add(deps, deps_select, dep, None) + return - _maybe_add_common_dep(deps, deps_select, platforms, req.name) + for plat in platforms_to_add: + if default_abi: + _add(deps, deps_select, dep, plat) + if plat.abi == default_abi or not default_abi: + _add(deps, deps_select, dep, platform(os = plat.os, arch = plat.arch)) diff --git a/python/private/pypi/pep508_requirement.bzl b/python/private/pypi/pep508_requirement.bzl index ee7b5dfc35..b5be17f890 100644 --- a/python/private/pypi/pep508_requirement.bzl +++ b/python/private/pypi/pep508_requirement.bzl @@ -47,9 +47,11 @@ def requirement(spec): requires, _, _ = requires.partition(char) extras = extras_unparsed.replace(" ", "").split(",") name = requires.strip(" ") + name = normalize_name(name) return struct( - name = normalize_name(name).replace("_", "-"), + name = name.replace("_", "-"), + name_ = name, marker = marker.strip(" "), extras = extras, version = version, diff --git a/python/private/pypi/whl_library.bzl b/python/private/pypi/whl_library.bzl index 0a580011ab..630dc8519f 100644 --- a/python/private/pypi/whl_library.bzl +++ b/python/private/pypi/whl_library.bzl @@ -15,6 +15,7 @@ "" load("//python/private:auth.bzl", "AUTH_ATTRS", "get_auth") +load("//python/private:bzlmod_enabled.bzl", "BZLMOD_ENABLED") load("//python/private:envsubst.bzl", "envsubst") load("//python/private:is_standalone_interpreter.bzl", "is_standalone_interpreter") load("//python/private:repo_utils.bzl", "REPO_DEBUG_ENV_VAR", "repo_utils") @@ -22,13 +23,10 @@ load(":attrs.bzl", "ATTRS", "use_isolated") load(":deps.bzl", "all_repo_names", "record_files") load(":generate_whl_library_build_bazel.bzl", "generate_whl_library_build_bazel") load(":parse_requirements.bzl", "host_platform") -load(":parse_whl_name.bzl", "parse_whl_name") load(":patch_whl.bzl", "patch_whl") -load(":pep508_deps.bzl", "deps") load(":pep508_requirement.bzl", "requirement") load(":pypi_repo_utils.bzl", "pypi_repo_utils") load(":whl_metadata.bzl", "whl_metadata") -load(":whl_target_platforms.bzl", "whl_target_platforms") _CPPFLAGS = "CPPFLAGS" _COMMAND_LINE_TOOLS_PATH_SLUG = "commandlinetools" @@ -344,20 +342,6 @@ def _whl_library_impl(rctx): timeout = rctx.attr.timeout, ) - target_platforms = rctx.attr.experimental_target_platforms - if target_platforms: - parsed_whl = parse_whl_name(whl_path.basename) - if parsed_whl.platform_tag != "any": - # NOTE @aignas 2023-12-04: if the wheel is a platform specific - # wheel, we only include deps for that target platform - target_platforms = [ - p.target_platform - for p in whl_target_platforms( - platform_tag = parsed_whl.platform_tag, - abi_tag = parsed_whl.abi_tag.strip("tm"), - ) - ] - pypi_repo_utils.execute_checked( rctx, op = "whl_library.ExtractWheel({}, {})".format(rctx.attr.name, whl_path), @@ -400,63 +384,45 @@ def _whl_library_impl(rctx): ) entry_points[entry_point_without_py] = entry_point_script_name - # TODO @aignas 2025-04-04: move this to whl_library_targets.bzl to have - # this in the analysis phase. - # - # This means that whl_library_targets will have to accept the following args: - # * name - the name of the package in the METADATA. - # * requires_dist - the list of METADATA Requires-Dist. - # * platforms - the list of target platforms. The target_platforms - # should come from the hub repo via a 'load' statement so that they don't - # need to be passed as an argument to `whl_library`. - # * extras - the list of required extras. This comes from the - # `rctx.attr.requirement` for now. In the future the required extras could - # stay in the hub repo, where we calculate the extra aliases that we need - # to create automatically and this way expose the targets for the specific - # extras. The first step will be to generate a target per extra for the - # `py_library` and `filegroup`. Maybe we need to have a special provider - # or an output group so that we can return the `whl` file from the - # `py_library` target? filegroup can use output groups to expose files. - # * host_python_version/versons - the list of python versions to support - # should come from the hub, similar to how the target platforms are specified. - # - # Extra things that we should move at the same time: - # * group_name, group_deps - this info can stay in the hub repository so that - # it is piped at the analysis time and changing the requirement groups does - # cause to re-fetch the deps. - python_version = metadata["python_version"] + if BZLMOD_ENABLED: + # The following attributes are unset on bzlmod and we pass data through + # the hub via load statements. + default_python_version = None + target_platforms = [] + else: + # NOTE @aignas 2025-04-16: if BZLMOD_ENABLED, we should use + # DEFAULT_PYTHON_VERSION since platforms always come with the actual + # python version otherwise we should use the version of the interpreter + # here. In WORKSPACE `multi_pip_parse` is using an interpreter for each + # `pip_parse` invocation, so we will have the host target platform + # only. Even if somebody would change the code to support + # `experimental_target_platforms`, they would be for a single python + # version. Hence, using the `default_python_version` that we get from the + # interpreter is correct. Hence, we unset the argument if we are on bzlmod. + default_python_version = metadata["python_version"] + target_platforms = rctx.attr.experimental_target_platforms or [host_platform(rctx)] + metadata = whl_metadata( install_dir = rctx.path("site-packages"), read_fn = rctx.read, logger = logger, ) - # TODO @aignas 2025-04-09: this will later be removed when loaded through the hub - major_minor, _, _ = python_version.rpartition(".") - package_deps = deps( - name = metadata.name, - requires_dist = metadata.requires_dist, - platforms = target_platforms or [ - "cp{}_{}".format(major_minor.replace(".", ""), host_platform(rctx)), - ], - extras = requirement(rctx.attr.requirement).extras, - host_python_version = python_version, - ) - build_file_contents = generate_whl_library_build_bazel( name = whl_path.basename, + metadata_name = metadata.name, + metadata_version = metadata.version, + requires_dist = metadata.requires_dist, dep_template = rctx.attr.dep_template or "@{}{{name}}//:{{target}}".format(rctx.attr.repo_prefix), - dependencies = package_deps.deps, - dependencies_by_platform = package_deps.deps_select, - group_name = rctx.attr.group_name, - group_deps = rctx.attr.group_deps, - data_exclude = rctx.attr.pip_data_exclude, - tags = [ - "pypi_name=" + metadata.name, - "pypi_version=" + metadata.version, - ], entry_points = entry_points, + target_platforms = target_platforms, + default_python_version = default_python_version, + # TODO @aignas 2025-04-14: load through the hub: annotation = None if not rctx.attr.annotation else struct(**json.decode(rctx.read(rctx.attr.annotation))), + data_exclude = rctx.attr.pip_data_exclude, + extras = requirement(rctx.attr.requirement).extras, + group_deps = rctx.attr.group_deps, + group_name = rctx.attr.group_name, ) rctx.file("BUILD.bazel", build_file_contents) @@ -517,10 +483,7 @@ and the target that we need respectively. doc = "Name of the group, if any.", ), "repo": attr.string( - doc = """\ -Pointer to parent repo name. Used to make these rules rerun if the parent repo changes. -Only used in WORKSPACE when the {attr}`dep_template` is not set. -""", + doc = "Pointer to parent repo name. Used to make these rules rerun if the parent repo changes.", ), "repo_prefix": attr.string( doc = """ diff --git a/python/private/pypi/whl_library_targets.bzl b/python/private/pypi/whl_library_targets.bzl index d32746b604..cf3df133c4 100644 --- a/python/private/pypi/whl_library_targets.bzl +++ b/python/private/pypi/whl_library_targets.bzl @@ -29,6 +29,89 @@ load( "WHEEL_FILE_IMPL_LABEL", "WHEEL_FILE_PUBLIC_LABEL", ) +load(":parse_whl_name.bzl", "parse_whl_name") +load(":pep508_deps.bzl", "deps") +load(":whl_target_platforms.bzl", "whl_target_platforms") + +def whl_library_targets_from_requires( + *, + name, + metadata_name = "", + metadata_version = "", + requires_dist = [], + extras = [], + target_platforms = [], + default_python_version = None, + group_deps = [], + **kwargs): + """The macro to create whl targets from the METADATA. + + Args: + name: {type}`str` The wheel filename + metadata_name: {type}`str` The package name as written in wheel `METADATA`. + metadata_version: {type}`str` The package version as written in wheel `METADATA`. + group_deps: {type}`list[str]` names of fellow members of the group (if + any). These will be excluded from generated deps lists so as to avoid + direct cycles. These dependencies will be provided at runtime by the + group rules which wrap this library and its fellows together. + requires_dist: {type}`list[str]` The list of `Requires-Dist` values from + the whl `METADATA`. + extras: {type}`list[str]` The list of requested extras. This essentially includes extra transitive dependencies in the final targets depending on the wheel `METADATA`. + target_platforms: {type}`list[str]` The list of target platforms to create + dependency closures for. + default_python_version: {type}`str` The python version to assume when parsing + the `METADATA`. This is only used when the `target_platforms` do not + include the version information. + **kwargs: Extra args passed to the {obj}`whl_library_targets` + """ + package_deps = _parse_requires_dist( + name = name, + default_python_version = default_python_version, + requires_dist = requires_dist, + excludes = group_deps, + extras = extras, + target_platforms = target_platforms, + ) + whl_library_targets( + name = name, + dependencies = package_deps.deps, + dependencies_by_platform = package_deps.deps_select, + tags = [ + "pypi_name={}".format(metadata_name), + "pypi_version={}".format(metadata_version), + ], + **kwargs + ) + +def _parse_requires_dist( + *, + name, + default_python_version, + requires_dist, + excludes, + extras, + target_platforms): + parsed_whl = parse_whl_name(name) + + # NOTE @aignas 2023-12-04: if the wheel is a platform specific wheel, we + # only include deps for that target platform + if parsed_whl.platform_tag != "any": + target_platforms = [ + p.target_platform + for p in whl_target_platforms( + platform_tag = parsed_whl.platform_tag, + abi_tag = parsed_whl.abi_tag.strip("tm"), + ) + ] + + return deps( + name = normalize_name(parsed_whl.distribution), + requires_dist = requires_dist, + platforms = target_platforms, + excludes = excludes, + extras = extras, + default_python_version = default_python_version, + ) def whl_library_targets( *, diff --git a/tests/pypi/extension/extension_tests.bzl b/tests/pypi/extension/extension_tests.bzl index 4d86d6a6e0..ce5474e35b 100644 --- a/tests/pypi/extension/extension_tests.bzl +++ b/tests/pypi/extension/extension_tests.bzl @@ -436,7 +436,6 @@ torch==2.4.1+cpu ; platform_machine == 'x86_64' \ pypi.whl_libraries().contains_exactly({ "pypi_312_torch_cp312_cp312_linux_x86_64_8800deef": { "dep_template": "@pypi//{name}:{target}", - "experimental_target_platforms": ["cp312_linux_x86_64"], "filename": "torch-2.4.1+cpu-cp312-cp312-linux_x86_64.whl", "python_interpreter_target": "unit_test_interpreter_target", "requirement": "torch==2.4.1+cpu", @@ -445,7 +444,6 @@ torch==2.4.1+cpu ; platform_machine == 'x86_64' \ }, "pypi_312_torch_cp312_cp312_manylinux_2_17_aarch64_36109432": { "dep_template": "@pypi//{name}:{target}", - "experimental_target_platforms": ["cp312_linux_aarch64"], "filename": "torch-2.4.1-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", "python_interpreter_target": "unit_test_interpreter_target", "requirement": "torch==2.4.1", @@ -454,7 +452,6 @@ torch==2.4.1+cpu ; platform_machine == 'x86_64' \ }, "pypi_312_torch_cp312_cp312_win_amd64_3a570e5c": { "dep_template": "@pypi//{name}:{target}", - "experimental_target_platforms": ["cp312_windows_x86_64"], "filename": "torch-2.4.1+cpu-cp312-cp312-win_amd64.whl", "python_interpreter_target": "unit_test_interpreter_target", "requirement": "torch==2.4.1+cpu", @@ -463,7 +460,6 @@ torch==2.4.1+cpu ; platform_machine == 'x86_64' \ }, "pypi_312_torch_cp312_none_macosx_11_0_arm64_72b484d5": { "dep_template": "@pypi//{name}:{target}", - "experimental_target_platforms": ["cp312_osx_aarch64"], "filename": "torch-2.4.1-cp312-none-macosx_11_0_arm64.whl", "python_interpreter_target": "unit_test_interpreter_target", "requirement": "torch==2.4.1", @@ -750,7 +746,6 @@ git_dep @ git+https://git.server/repo/project@deadbeefdeadbeef pypi.whl_libraries().contains_exactly({ "pypi_315_any_name": { "dep_template": "@pypi//{name}:{target}", - "experimental_target_platforms": ["cp315_linux_aarch64", "cp315_linux_arm", "cp315_linux_ppc", "cp315_linux_s390x", "cp315_linux_x86_64", "cp315_osx_aarch64", "cp315_osx_x86_64", "cp315_windows_x86_64"], "extra_pip_args": ["--extra-args-for-sdist-building"], "filename": "any-name.tar.gz", "python_interpreter_target": "unit_test_interpreter_target", @@ -760,7 +755,6 @@ git_dep @ git+https://git.server/repo/project@deadbeefdeadbeef }, "pypi_315_direct_without_sha_0_0_1_py3_none_any": { "dep_template": "@pypi//{name}:{target}", - "experimental_target_platforms": ["cp315_linux_aarch64", "cp315_linux_arm", "cp315_linux_ppc", "cp315_linux_s390x", "cp315_linux_x86_64", "cp315_osx_aarch64", "cp315_osx_x86_64", "cp315_windows_x86_64"], "filename": "direct_without_sha-0.0.1-py3-none-any.whl", "python_interpreter_target": "unit_test_interpreter_target", "requirement": "direct_without_sha==0.0.1 @ example-direct.org/direct_without_sha-0.0.1-py3-none-any.whl", @@ -781,7 +775,6 @@ git_dep @ git+https://git.server/repo/project@deadbeefdeadbeef }, "pypi_315_simple_py3_none_any_deadb00f": { "dep_template": "@pypi//{name}:{target}", - "experimental_target_platforms": ["cp315_linux_aarch64", "cp315_linux_arm", "cp315_linux_ppc", "cp315_linux_s390x", "cp315_linux_x86_64", "cp315_osx_aarch64", "cp315_osx_x86_64", "cp315_windows_x86_64"], "filename": "simple-0.0.1-py3-none-any.whl", "python_interpreter_target": "unit_test_interpreter_target", "requirement": "simple==0.0.1", @@ -790,7 +783,6 @@ git_dep @ git+https://git.server/repo/project@deadbeefdeadbeef }, "pypi_315_simple_sdist_deadbeef": { "dep_template": "@pypi//{name}:{target}", - "experimental_target_platforms": ["cp315_linux_aarch64", "cp315_linux_arm", "cp315_linux_ppc", "cp315_linux_s390x", "cp315_linux_x86_64", "cp315_osx_aarch64", "cp315_osx_x86_64", "cp315_windows_x86_64"], "extra_pip_args": ["--extra-args-for-sdist-building"], "filename": "simple-0.0.1.tar.gz", "python_interpreter_target": "unit_test_interpreter_target", @@ -800,7 +792,6 @@ git_dep @ git+https://git.server/repo/project@deadbeefdeadbeef }, "pypi_315_some_pkg_py3_none_any_deadbaaf": { "dep_template": "@pypi//{name}:{target}", - "experimental_target_platforms": ["cp315_linux_aarch64", "cp315_linux_arm", "cp315_linux_ppc", "cp315_linux_s390x", "cp315_linux_x86_64", "cp315_osx_aarch64", "cp315_osx_x86_64", "cp315_windows_x86_64"], "filename": "some_pkg-0.0.1-py3-none-any.whl", "python_interpreter_target": "unit_test_interpreter_target", "requirement": "some_pkg==0.0.1 @ example-direct.org/some_pkg-0.0.1-py3-none-any.whl --hash=sha256:deadbaaf", @@ -809,7 +800,6 @@ git_dep @ git+https://git.server/repo/project@deadbeefdeadbeef }, "pypi_315_some_py3_none_any_deadb33f": { "dep_template": "@pypi//{name}:{target}", - "experimental_target_platforms": ["cp315_linux_aarch64", "cp315_linux_arm", "cp315_linux_ppc", "cp315_linux_s390x", "cp315_linux_x86_64", "cp315_osx_aarch64", "cp315_osx_x86_64", "cp315_windows_x86_64"], "filename": "some-other-pkg-0.0.1-py3-none-any.whl", "python_interpreter_target": "unit_test_interpreter_target", "requirement": "some_other_pkg==0.0.1", diff --git a/tests/pypi/generate_whl_library_build_bazel/generate_whl_library_build_bazel_tests.bzl b/tests/pypi/generate_whl_library_build_bazel/generate_whl_library_build_bazel_tests.bzl index b0d8f6d17e..7bd19b65c1 100644 --- a/tests/pypi/generate_whl_library_build_bazel/generate_whl_library_build_bazel_tests.bzl +++ b/tests/pypi/generate_whl_library_build_bazel/generate_whl_library_build_bazel_tests.bzl @@ -21,11 +21,11 @@ _tests = [] def _test_all(env): want = """\ -load("@rules_python//python/private/pypi:whl_library_targets.bzl", "whl_library_targets") +load("@rules_python//python/private/pypi:whl_library_targets.bzl", "whl_library_targets_from_requires") package(default_visibility = ["//visibility:public"]) -whl_library_targets( +whl_library_targets_from_requires( copy_executables = { "exec_src": "exec_dest", }, @@ -38,19 +38,71 @@ whl_library_targets( "data_exclude_all", ], dep_template = "@pypi//{name}:{target}", - dependencies = [ + entry_points = { + "foo": "bar.py", + }, + group_deps = [ + "foo", + "fox", + "qux", + ], + group_name = "qux", + name = "foo.whl", + requires_dist = [ "foo", "bar-baz", "qux", ], - dependencies_by_platform = { - "linux_x86_64": [ - "box", - "box-amd64", - ], - "windows_x86_64": ["fox"], - "@platforms//os:linux": ["box"], + srcs_exclude = ["srcs_exclude_all"], + target_platforms = ["foo"], +) + +# SOMETHING SPECIAL AT THE END +""" + actual = generate_whl_library_build_bazel( + dep_template = "@pypi//{name}:{target}", + name = "foo.whl", + requires_dist = ["foo", "bar-baz", "qux"], + entry_points = { + "foo": "bar.py", + }, + data_exclude = ["exclude_via_attr"], + annotation = struct( + copy_files = {"file_src": "file_dest"}, + copy_executables = {"exec_src": "exec_dest"}, + data = ["extra_target"], + data_exclude_glob = ["data_exclude_all"], + srcs_exclude_glob = ["srcs_exclude_all"], + additive_build_content = """# SOMETHING SPECIAL AT THE END""", + ), + group_name = "qux", + target_platforms = ["foo"], + group_deps = ["foo", "fox", "qux"], + ) + env.expect.that_str(actual.replace("@@", "@")).equals(want) + +_tests.append(_test_all) + +def _test_all_with_loads(env): + want = """\ +load("@rules_python//python/private/pypi:whl_library_targets.bzl", "whl_library_targets_from_requires") +load("@pypi//:config.bzl", "target_platforms") + +package(default_visibility = ["//visibility:public"]) + +whl_library_targets_from_requires( + copy_executables = { + "exec_src": "exec_dest", }, + copy_files = { + "file_src": "file_dest", + }, + data = ["extra_target"], + data_exclude = [ + "exclude_via_attr", + "data_exclude_all", + ], + dep_template = "@pypi//{name}:{target}", entry_points = { "foo": "bar.py", }, @@ -61,11 +113,13 @@ whl_library_targets( ], group_name = "qux", name = "foo.whl", - srcs_exclude = ["srcs_exclude_all"], - tags = [ - "tag2", - "tag1", + requires_dist = [ + "foo", + "bar-baz", + "qux", ], + srcs_exclude = ["srcs_exclude_all"], + target_platforms = target_platforms, ) # SOMETHING SPECIAL AT THE END @@ -73,13 +127,7 @@ whl_library_targets( actual = generate_whl_library_build_bazel( dep_template = "@pypi//{name}:{target}", name = "foo.whl", - dependencies = ["foo", "bar-baz", "qux"], - dependencies_by_platform = { - "linux_x86_64": ["box", "box-amd64"], - "windows_x86_64": ["fox"], - "@platforms//os:linux": ["box"], # buildifier: disable=unsorted-dict-items to check that we sort inside the test - }, - tags = ["tag2", "tag1"], + requires_dist = ["foo", "bar-baz", "qux"], entry_points = { "foo": "bar.py", }, @@ -97,7 +145,7 @@ whl_library_targets( ) env.expect.that_str(actual.replace("@@", "@")).equals(want) -_tests.append(_test_all) +_tests.append(_test_all_with_loads) def generate_whl_library_build_bazel_test_suite(name): """Create the test suite. diff --git a/tests/pypi/pep508/deps_tests.bzl b/tests/pypi/pep508/deps_tests.bzl index 44031ab6a5..d362925080 100644 --- a/tests/pypi/pep508/deps_tests.bzl +++ b/tests/pypi/pep508/deps_tests.bzl @@ -29,58 +29,48 @@ def test_simple_deps(env): _tests.append(test_simple_deps) def test_can_add_os_specific_deps(env): - got = deps( - "foo", - requires_dist = [ - "bar", - "an_osx_dep; sys_platform=='darwin'", - "posix_dep; os_name=='posix'", - "win_dep; os_name=='nt'", - ], - platforms = [ - "linux_x86_64", - "osx_x86_64", - "osx_aarch64", - "windows_x86_64", - ], - host_python_version = "3.3.1", - ) - - env.expect.that_collection(got.deps).contains_exactly(["bar"]) - env.expect.that_dict(got.deps_select).contains_exactly({ - "@platforms//os:linux": ["posix_dep"], - "@platforms//os:osx": ["an_osx_dep", "posix_dep"], - "@platforms//os:windows": ["win_dep"], - }) + for target in [ + struct( + platforms = [ + "linux_x86_64", + "osx_x86_64", + "osx_aarch64", + "windows_x86_64", + ], + python_version = "3.3.1", + ), + struct( + platforms = [ + "cp33_linux_x86_64", + "cp33_osx_x86_64", + "cp33_osx_aarch64", + "cp33_windows_x86_64", + ], + python_version = "", + ), + ]: + got = deps( + "foo", + requires_dist = [ + "bar", + "an_osx_dep; sys_platform=='darwin'", + "posix_dep; os_name=='posix'", + "win_dep; os_name=='nt'", + ], + platforms = target.platforms, + default_python_version = target.python_version, + ) + + env.expect.that_collection(got.deps).contains_exactly(["bar"]) + env.expect.that_dict(got.deps_select).contains_exactly({ + "linux_x86_64": ["posix_dep"], + "osx_aarch64": ["an_osx_dep", "posix_dep"], + "osx_x86_64": ["an_osx_dep", "posix_dep"], + "windows_x86_64": ["win_dep"], + }) _tests.append(test_can_add_os_specific_deps) -def test_can_add_os_specific_deps_with_python_version(env): - got = deps( - "foo", - requires_dist = [ - "bar", - "an_osx_dep; sys_platform=='darwin'", - "posix_dep; os_name=='posix'", - "win_dep; os_name=='nt'", - ], - platforms = [ - "cp33_linux_x86_64", - "cp33_osx_x86_64", - "cp33_osx_aarch64", - "cp33_windows_x86_64", - ], - ) - - env.expect.that_collection(got.deps).contains_exactly(["bar"]) - env.expect.that_dict(got.deps_select).contains_exactly({ - "@platforms//os:linux": ["posix_dep"], - "@platforms//os:osx": ["an_osx_dep", "posix_dep"], - "@platforms//os:windows": ["win_dep"], - }) - -_tests.append(test_can_add_os_specific_deps_with_python_version) - def test_deps_are_added_to_more_specialized_platforms(env): got = deps( "foo", @@ -92,41 +82,16 @@ def test_deps_are_added_to_more_specialized_platforms(env): "osx_x86_64", "osx_aarch64", ], - host_python_version = "3.8.4", + default_python_version = "3.8.4", ) - env.expect.that_collection(got.deps).contains_exactly([]) + env.expect.that_collection(got.deps).contains_exactly(["mac_dep"]) env.expect.that_dict(got.deps_select).contains_exactly({ - "@platforms//os:osx": ["mac_dep"], - "osx_aarch64": ["m1_dep", "mac_dep"], + "osx_aarch64": ["m1_dep"], }) _tests.append(test_deps_are_added_to_more_specialized_platforms) -def test_deps_from_more_specialized_platforms_are_propagated(env): - got = deps( - "foo", - requires_dist = [ - "a_mac_dep; sys_platform=='darwin'", - "m1_dep; sys_platform=='darwin' and platform_machine=='arm64'", - ], - platforms = [ - "osx_x86_64", - "osx_aarch64", - ], - host_python_version = "3.8.4", - ) - - env.expect.that_collection(got.deps).contains_exactly([]) - env.expect.that_dict(got.deps_select).contains_exactly( - { - "@platforms//os:osx": ["a_mac_dep"], - "osx_aarch64": ["a_mac_dep", "m1_dep"], - }, - ) - -_tests.append(test_deps_from_more_specialized_platforms_are_propagated) - def test_non_platform_markers_are_added_to_common_deps(env): got = deps( "foo", @@ -141,7 +106,7 @@ def test_non_platform_markers_are_added_to_common_deps(env): "osx_aarch64", "windows_x86_64", ], - host_python_version = "3.8.4", + default_python_version = "3.8.4", ) env.expect.that_collection(got.deps).contains_exactly(["bar", "baz"]) @@ -204,38 +169,34 @@ def _test_can_get_deps_based_on_specific_python_version(env): platforms = ["cp37_linux_x86_64"], ) + # since there is a single target platform, the deps_select will be empty env.expect.that_collection(py37.deps).contains_exactly(["bar", "baz"]) env.expect.that_dict(py37.deps_select).contains_exactly({}) - env.expect.that_collection(py38.deps).contains_exactly(["bar"]) - env.expect.that_dict(py38.deps_select).contains_exactly({"@platforms//os:linux": ["posix_dep"]}) + env.expect.that_collection(py38.deps).contains_exactly(["bar", "posix_dep"]) + env.expect.that_dict(py38.deps_select).contains_exactly({}) _tests.append(_test_can_get_deps_based_on_specific_python_version) def _test_no_version_select_when_single_version(env): - requires_dist = [ - "bar", - "baz; python_version >= '3.8'", - "posix_dep; os_name=='posix'", - "posix_dep_with_version; os_name=='posix' and python_version >= '3.8'", - "arch_dep; platform_machine=='x86_64' and python_version >= '3.8'", - ] - host_python_version = "3.7.5" - got = deps( "foo", - requires_dist = requires_dist, + requires_dist = [ + "bar", + "baz; python_version >= '3.8'", + "posix_dep; os_name=='posix'", + "posix_dep_with_version; os_name=='posix' and python_version >= '3.8'", + "arch_dep; platform_machine=='x86_64' and python_version >= '3.8'", + ], platforms = [ "cp38_linux_x86_64", "cp38_windows_x86_64", ], - host_python_version = host_python_version, + default_python_version = "", ) - env.expect.that_collection(got.deps).contains_exactly(["bar", "baz"]) + env.expect.that_collection(got.deps).contains_exactly(["bar", "baz", "arch_dep"]) env.expect.that_dict(got.deps_select).contains_exactly({ - "@platforms//os:linux": ["posix_dep", "posix_dep_with_version"], - "linux_x86_64": ["arch_dep", "posix_dep", "posix_dep_with_version"], - "windows_x86_64": ["arch_dep"], + "linux_x86_64": ["posix_dep", "posix_dep_with_version"], }) _tests.append(_test_no_version_select_when_single_version) @@ -249,7 +210,7 @@ def _test_can_get_version_select(env): "posix_dep_with_version; os_name=='posix' and python_version >= '3.8'", "arch_dep; platform_machine=='x86_64' and python_version < '3.8'", ] - host_python_version = "3.7.4" + default_python_version = "3.7.4" got = deps( "foo", @@ -259,31 +220,19 @@ def _test_can_get_version_select(env): for minor in [7, 8, 9] for os in ["linux", "windows"] ], - host_python_version = host_python_version, + default_python_version = default_python_version, ) env.expect.that_collection(got.deps).contains_exactly(["bar"]) env.expect.that_dict(got.deps_select).contains_exactly({ - str(Label("//python/config_settings:is_python_3.7")): ["baz"], - str(Label("//python/config_settings:is_python_3.8")): ["baz_new"], - str(Label("//python/config_settings:is_python_3.9")): ["baz_new"], - "@platforms//os:linux": ["baz", "posix_dep"], - "cp37_linux_anyarch": ["baz", "posix_dep"], "cp37_linux_x86_64": ["arch_dep", "baz", "posix_dep"], "cp37_windows_x86_64": ["arch_dep", "baz"], - "cp38_linux_anyarch": [ - "baz_new", - "posix_dep", - "posix_dep_with_version", - ], - "cp39_linux_anyarch": [ - "baz_new", - "posix_dep", - "posix_dep_with_version", - ], + "cp38_linux_x86_64": ["baz_new", "posix_dep", "posix_dep_with_version"], + "cp38_windows_x86_64": ["baz_new"], + "cp39_linux_x86_64": ["baz_new", "posix_dep", "posix_dep_with_version"], + "cp39_windows_x86_64": ["baz_new"], "linux_x86_64": ["arch_dep", "baz", "posix_dep"], "windows_x86_64": ["arch_dep", "baz"], - "//conditions:default": ["baz"], }) _tests.append(_test_can_get_version_select) @@ -294,7 +243,7 @@ def _test_deps_spanning_all_target_py_versions_are_added_to_common(env): "baz (<2,>=1.11) ; python_version < '3.8'", "baz (<2,>=1.14) ; python_version >= '3.8'", ] - host_python_version = "3.8.4" + default_python_version = "3.8.4" got = deps( "foo", @@ -303,7 +252,7 @@ def _test_deps_spanning_all_target_py_versions_are_added_to_common(env): "cp3{}_linux_x86_64".format(minor) for minor in [7, 8, 9] ], - host_python_version = host_python_version, + default_python_version = default_python_version, ) env.expect.that_collection(got.deps).contains_exactly(["bar", "baz"]) @@ -312,7 +261,7 @@ def _test_deps_spanning_all_target_py_versions_are_added_to_common(env): _tests.append(_test_deps_spanning_all_target_py_versions_are_added_to_common) def _test_deps_are_not_duplicated(env): - host_python_version = "3.7.4" + default_python_version = "3.7.4" # See an example in # https://files.pythonhosted.org/packages/76/9e/db1c2d56c04b97981c06663384f45f28950a73d9acf840c4006d60d0a1ff/opencv_python-4.9.0.80-cp37-abi3-win32.whl.metadata @@ -336,7 +285,7 @@ def _test_deps_are_not_duplicated(env): for os in ["linux", "osx", "windows"] for arch in ["x86_64", "aarch64"] ], - host_python_version = host_python_version, + default_python_version = default_python_version, ) env.expect.that_collection(got.deps).contains_exactly(["bar"]) @@ -345,7 +294,7 @@ def _test_deps_are_not_duplicated(env): _tests.append(_test_deps_are_not_duplicated) def _test_deps_are_not_duplicated_when_encountering_platform_dep_first(env): - host_python_version = "3.7.1" + default_python_version = "3.7.1" # Note, that we are sorting the incoming `requires_dist` and we need to ensure that we are not getting any # issues even if the platform-specific line comes first. @@ -363,15 +312,13 @@ def _test_deps_are_not_duplicated_when_encountering_platform_dep_first(env): "cp310_linux_aarch64", "cp310_linux_x86_64", ], - host_python_version = host_python_version, + default_python_version = default_python_version, ) - # TODO @aignas 2025-02-24: this test case in the python version is passing but - # I am not sure why. The starlark version behaviour looks more correct. env.expect.that_collection(got.deps).contains_exactly([]) env.expect.that_dict(got.deps_select).contains_exactly({ - str(Label("//python/config_settings:is_python_3.10")): ["bar"], "cp310_linux_aarch64": ["bar"], + "cp310_linux_x86_64": ["bar"], "cp37_linux_aarch64": ["bar"], "linux_aarch64": ["bar"], }) diff --git a/tests/pypi/whl_library_targets/whl_library_targets_tests.bzl b/tests/pypi/whl_library_targets/whl_library_targets_tests.bzl index f738e03b5d..61e5441050 100644 --- a/tests/pypi/whl_library_targets/whl_library_targets_tests.bzl +++ b/tests/pypi/whl_library_targets/whl_library_targets_tests.bzl @@ -16,7 +16,7 @@ load("@rules_testing//lib:test_suite.bzl", "test_suite") load("//python/private:glob_excludes.bzl", "glob_excludes") # buildifier: disable=bzl-visibility -load("//python/private/pypi:whl_library_targets.bzl", "whl_library_targets") # buildifier: disable=bzl-visibility +load("//python/private/pypi:whl_library_targets.bzl", "whl_library_targets", "whl_library_targets_from_requires") # buildifier: disable=bzl-visibility _tests = [] @@ -183,6 +183,71 @@ def _test_entrypoints(env): _tests.append(_test_entrypoints) +def _test_whl_and_library_deps_from_requires(env): + filegroup_calls = [] + py_library_calls = [] + + whl_library_targets_from_requires( + name = "foo-0-py3-none-any.whl", + metadata_name = "Foo", + metadata_version = "0", + dep_template = "@pypi_{name}//:{target}", + requires_dist = [ + "foo", # this self-edge will be ignored + "bar-baz", + ], + target_platforms = ["cp38_linux_x86_64"], + default_python_version = "3.8.1", + data_exclude = [], + # Overrides for testing + filegroups = {}, + native = struct( + filegroup = lambda **kwargs: filegroup_calls.append(kwargs), + config_setting = lambda **_: None, + glob = _glob, + select = _select, + ), + rules = struct( + py_library = lambda **kwargs: py_library_calls.append(kwargs), + ), + ) + + env.expect.that_collection(filegroup_calls).contains_exactly([ + { + "name": "whl", + "srcs": ["foo-0-py3-none-any.whl"], + "data": ["@pypi_bar_baz//:whl"], + "visibility": ["//visibility:public"], + }, + ]) # buildifier: @unsorted-dict-items + env.expect.that_collection(py_library_calls).contains_exactly([ + { + "name": "pkg", + "srcs": _glob( + ["site-packages/**/*.py"], + exclude = [], + allow_empty = True, + ), + "pyi_srcs": _glob(["site-packages/**/*.pyi"], allow_empty = True), + "data": [] + _glob( + ["site-packages/**/*"], + exclude = [ + "**/*.py", + "**/*.pyc", + "**/*.pyc.*", + "**/*.dist-info/RECORD", + ] + glob_excludes.version_dependent_exclusions(), + ), + "imports": ["site-packages"], + "deps": ["@pypi_bar_baz//:pkg"], + "tags": ["pypi_name=Foo", "pypi_version=0"], + "visibility": ["//visibility:public"], + "experimental_venvs_site_packages": Label("//python/config_settings:venvs_site_packages"), + }, + ]) # buildifier: @unsorted-dict-items + +_tests.append(_test_whl_and_library_deps_from_requires) + def _test_whl_and_library_deps(env): filegroup_calls = [] py_library_calls = [] From c981569cc89c76eb57a78f0bbc47f1566211c924 Mon Sep 17 00:00:00 2001 From: Ignas Anikevicius <240938+aignas@users.noreply.github.com> Date: Mon, 21 Apr 2025 15:13:10 +0900 Subject: [PATCH 178/922] chore: remove a stray file (#2795) Remove a stray file --- config.bzl.tmpl.bzlmod | 0 1 file changed, 0 insertions(+), 0 deletions(-) delete mode 100644 config.bzl.tmpl.bzlmod diff --git a/config.bzl.tmpl.bzlmod b/config.bzl.tmpl.bzlmod deleted file mode 100644 index e69de29bb2..0000000000 From e11873323ffc2694489131fd2f861c0619907bc1 Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Mon, 21 Apr 2025 22:19:07 +0000 Subject: [PATCH 179/922] build(deps): bump sphinx-rtd-theme from 3.0.1 to 3.0.2 in /docs (#2802) Bumps [sphinx-rtd-theme](https://github.com/readthedocs/sphinx_rtd_theme) from 3.0.1 to 3.0.2.
Changelog

Sourced from sphinx-rtd-theme's changelog.

3.0.2

  • Show current translation when the flyout is attached
  • Fix JavaScript issue that didn't allow users to disable selectors

.. _release-3.0.1:

Commits

[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=sphinx-rtd-theme&package-manager=pip&previous-version=3.0.1&new-version=3.0.2)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) ---
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot show ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> --- docs/requirements.txt | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/requirements.txt b/docs/requirements.txt index 5e308b00f4..747ae59e1a 100644 --- a/docs/requirements.txt +++ b/docs/requirements.txt @@ -319,9 +319,9 @@ sphinx-reredirects==0.1.6 \ --hash=sha256:c491cba545f67be9697508727818d8626626366245ae64456fe29f37e9bbea64 \ --hash=sha256:efd50c766fbc5bf40cd5148e10c00f2c00d143027de5c5e48beece93cc40eeea # via rules-python-docs (docs/pyproject.toml) -sphinx-rtd-theme==3.0.1 \ - --hash=sha256:921c0ece75e90633ee876bd7b148cfaad136b481907ad154ac3669b6fc957916 \ - --hash=sha256:a4c5745d1b06dfcb80b7704fe532eb765b44065a8fad9851e4258c8804140703 +sphinx-rtd-theme==3.0.2 \ + --hash=sha256:422ccc750c3a3a311de4ae327e82affdaf59eb695ba4936538552f3b00f4ee13 \ + --hash=sha256:b7457bc25dda723b20b086a670b9953c859eab60a2a03ee8eb2bb23e176e5f85 # via rules-python-docs (docs/pyproject.toml) sphinxcontrib-applehelp==2.0.0 \ --hash=sha256:2f29ef331735ce958efa4734873f084941970894c6090408b079c61b2e1c06d1 \ From a57c4de9dbb722765685cd2deae71fc73efcde75 Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Mon, 21 Apr 2025 22:19:54 +0000 Subject: [PATCH 180/922] build(deps): bump astroid from 3.3.6 to 3.3.9 in /docs (#2803) Bumps [astroid](https://github.com/pylint-dev/astroid) from 3.3.6 to 3.3.9.
Release notes

Sourced from astroid's releases.

v3.3.9

What's New in astroid 3.3.9?

Release date: 2025-03-09

v3.3.8

What's New in astroid 3.3.8?

Release date: 2024-12-23

  • Fix inability to import collections.abc in python 3.13.1. The reported fixes in astroid 3.3.6 and 3.3.7 did not actually fix this issue.

    Closes pylint-dev/pylint#10112

v3.3.7

What's New in astroid 3.3.7?

Release date: 2024-12-21

  • Fix inability to import collections.abc in python 3.13.1. The reported fix in astroid 3.3.6 did not actually fix this issue.

    Closes pylint-dev/pylint#10112

Changelog

Sourced from astroid's changelog.

What's New in astroid 3.3.9?

Release date: 2025-03-09

What's New in astroid 3.3.8?

Release date: 2024-12-23

  • Fix inability to import collections.abc in python 3.13.1. The reported fixes in astroid 3.3.6 and 3.3.7 did not actually fix this issue.

    Closes pylint-dev/pylint#10112

What's New in astroid 3.3.7?

Release date: 2024-12-20

This release was yanked.

  • Fix inability to import collections.abc in python 3.13.1. The reported fix in astroid 3.3.6 did not actually fix this issue.

    Closes pylint-dev/pylint#10112

Commits
  • a6ccad5 Bump astroid to 3.3.9, update changelog
  • ec2df97 Add setuptools in order to run 3.12/3.13 tests
  • 74c34fb Bump actions/cache from 4.2.0 to 4.2.2 (#2692)
  • 5512bf2 Update release workflow to use Trusted Publishing (#2696)
  • aad8e68 [Backport maintenance/3.3.x] Fix missing dict (#2685) (#2690)
  • 234be58 Fix RuntimeError caused by analyzing live objects with __getattribute__ or ...
  • 6aeafd5 Bump pylint in pre-commit configuration to 3.2.7
  • d52799b Bump astroid to 3.3.8, update changelog
  • 68714df [Backport maintenance/3.3.x] Another attempt at fixing the collections.abc ...
  • 7cfbad1 Skip flaky recursion test on PyPy (#2661) (#2663)
  • Additional commits viewable in compare view

[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=astroid&package-manager=pip&previous-version=3.3.6&new-version=3.3.9)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) ---
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot show ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> --- docs/requirements.txt | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/requirements.txt b/docs/requirements.txt index 747ae59e1a..ee242e07d0 100644 --- a/docs/requirements.txt +++ b/docs/requirements.txt @@ -10,9 +10,9 @@ alabaster==1.0.0 \ --hash=sha256:c00dca57bca26fa62a6d7d0a9fcce65f3e026e9bfe33e9c538fd3fbb2144fd9e \ --hash=sha256:fc6786402dc3fcb2de3cabd5fe455a2db534b371124f1f21de8731783dec828b # via sphinx -astroid==3.3.6 \ - --hash=sha256:6aaea045f938c735ead292204afdb977a36e989522b7833ef6fea94de743f442 \ - --hash=sha256:db676dc4f3ae6bfe31cda227dc60e03438378d7a896aec57422c95634e8d722f +astroid==3.3.9 \ + --hash=sha256:622cc8e3048684aa42c820d9d218978021c3c3d174fb03a9f0d615921744f550 \ + --hash=sha256:d05bfd0acba96a7bd43e222828b7d9bc1e138aaeb0649707908d3702a9831248 # via sphinx-autodoc2 babel==2.17.0 \ --hash=sha256:0c54cffb19f690cdcc52a3b50bcbf71e07a808d1c80d549f2459b9d2cf0afb9d \ From aaf8ce8adb43536f24ecfe38038351afafcbfa65 Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Mon, 21 Apr 2025 22:22:05 +0000 Subject: [PATCH 181/922] build(deps): bump packaging from 24.2 to 25.0 in /docs (#2804) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Bumps [packaging](https://github.com/pypa/packaging) from 24.2 to 25.0.
Release notes

Sourced from packaging's releases.

25.0

What's Changed

New Contributors

Full Changelog: https://github.com/pypa/packaging/compare/24.2...25.0

Changelog

Sourced from packaging's changelog.

25.0 - 2025-04-19


* PEP 751: Add support for ``extras`` and ``dependency_groups`` markers.
(:issue:`885`)
* PEP 738: Add support for Android platform tags. (:issue:`880`)
Commits
  • f585376 Bump for release
  • 600ecea Add changelog entries
  • 3910129 support 'extras' and 'dependency_groups' markers (#888)
  • 8e49b43 Add support for PEP 738 Android tags (#880)
  • e624d8e Bump the github-actions group with 3 updates (#886)
  • 71f38d8 Bump the github-actions group with 2 updates (#878)
  • 9b4922d Bump the github-actions group with 3 updates (#870)
  • 8510bd9 Upgrade to ruff 0.9.1 (#865)
  • 9375ec2 Re-add tests for Unicode file name parsing (#863)
  • 2256ed4 Bump the github-actions group across 1 directory with 2 updates (#864)
  • Additional commits viewable in compare view

[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=packaging&package-manager=pip&previous-version=24.2&new-version=25.0)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) ---
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot show ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> --- docs/requirements.txt | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/requirements.txt b/docs/requirements.txt index ee242e07d0..e4ec16fa5e 100644 --- a/docs/requirements.txt +++ b/docs/requirements.txt @@ -223,9 +223,9 @@ myst-parser==4.0.0 \ --hash=sha256:851c9dfb44e36e56d15d05e72f02b80da21a9e0d07cba96baf5e2d476bb91531 \ --hash=sha256:b9317997552424448c6096c2558872fdb6f81d3ecb3a40ce84a7518798f3f28d # via rules-python-docs (docs/pyproject.toml) -packaging==24.2 \ - --hash=sha256:09abb1bccd265c01f4a3aa3f7a7db064b36514d2cba19a2f694fe6150451a759 \ - --hash=sha256:c228a6dc5e932d346bc5739379109d49e8853dd8223571c7c5b55260edc0b97f +packaging==25.0 \ + --hash=sha256:29572ef2b1f17581046b3a2227d5c611fb25ec70ca1ba8554b24b0e69331a484 \ + --hash=sha256:d443872c98d677bf60f6a1f2f8c1cb748e8fe762d2bf9d3148b5599295b0fc4f # via # readthedocs-sphinx-ext # sphinx From f4780f7b71dc224ea3b51b4ec8048b829e1f3375 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Mon, 21 Apr 2025 15:35:13 -0700 Subject: [PATCH 182/922] fix: fixes to prepare for making bootstrap=script the default for Linux (#2760) Various cleanup and prep work to switch bootstrap=script to be the default. * Change `bootstrap_impl` to always be disabled for windows. This allows setting it to true in a bazelrc without worrying about the target platform. This is done by using FeatureFlagInfo to force the value to disabled for windows. This allows any downstream usages of the flag to Just Work and not have to add selects() for windows themselves. * Switch pip_repository_annotations test to `import python.runfiles`. The script bootstrap doesn't add the runfiles root to sys.path, so `import rules_python` stops working. * Switch gazelle workspace to using the runtime-env toolchain. It was previously implicitly using the deprecated one built into bazel, which doesn't provide various necessary provider fields. * Make the local toolchain use `sys._base_executable` instead of `sys.executable` when finding the interpreter. Otherwise, it might find a venv interpreter or not properly handle wrapper scripts like pyenv. * Adds a toolchain attribute/field to indicate if the toolchain supports a build-time created venv. This is due to the runtime_env toolchain. See PR comments for details, but in short: if we don't know the python interpreter path and version at build time, the venv may not properly activate or find site-packages. If it isn't supported, then the stage1 bootstrap creates a temporary venv, similar to how the zip case is handled. Unfortunately, this requires invoking Python itself as part of program startup, but I don't see a way around that -- note this is only triggered by the runtime-env toolchain. * Make the runtime-env toolchain better support virtualenvs. Because it's a wrapper that re-invokes Python, Python can't automatically detect its in a venv. Two tricks are used (`exec -a` and PYTHONEXECUTABLE) to help address this (but they aren't guaranteed to work, hence the "recreate at runtime" logic). * Fix a subtle issue where `sys._base_executable` isn't set correctly due to `home` missing in the pyvenv.cfg file. This mostly only affected the creation of venvs from within the bazel-created venv. * Change the bazel site init to always add the build-time created site-packages (if it exists) as a site directory. This matches the system_python bootstrap behavior a bit better, which just shoved everything onto sys.path using PYTHONPATH. * Skip running runtime_env_toolchains tests on RBE. RBE's system python is 3.6, but the script bootstrap uses 3.9 features. (Running it on RBE is questionable anyways). Along the way... * Ignore gazelle convenience symlinks * Switch pip_repository_annotations test to use non-legacy_external_runfiles based paths. The legacy behavior is disabled in Bazel 8+ by default. * Also document why the script bootstrap doesn't add the runfiles root to sys.path. Work towards https://github.com/bazel-contrib/rules_python/issues/2521 --------- Co-authored-by: Ignas Anikevicius <240938+aignas@users.noreply.github.com> --- .bazelignore | 1 + CHANGELOG.md | 10 +- examples/pip_repository_annotations/.bazelrc | 1 + .../pip_repository_annotations_test.py | 25 ++--- gazelle/WORKSPACE | 2 + python/config_settings/BUILD.bazel | 16 +++- python/private/BUILD.bazel | 1 + python/private/config_settings.bzl | 30 ++++++ python/private/flags.bzl | 32 ++++++- python/private/get_local_runtime_info.py | 1 + python/private/local_runtime_repo.bzl | 14 +++ python/private/py_executable.bzl | 35 ++++++- python/private/py_runtime_info.bzl | 26 ++++- python/private/py_runtime_rule.bzl | 12 +++ python/private/runtime_env_toolchain.bzl | 12 +++ .../runtime_env_toolchain_interpreter.sh | 26 ++++- python/private/site_init_template.py | 30 ++++++ python/private/stage1_bootstrap_template.sh | 94 ++++++++++++++----- python/private/stage2_bootstrap_template.py | 22 +++++ .../integration/local_toolchains/BUILD.bazel | 2 + tests/integration/local_toolchains/test.py | 53 +++++++++-- tests/runtime_env_toolchain/BUILD.bazel | 4 + 22 files changed, 393 insertions(+), 56 deletions(-) diff --git a/.bazelignore b/.bazelignore index e10af2035d..fb999097f5 100644 --- a/.bazelignore +++ b/.bazelignore @@ -25,6 +25,7 @@ examples/pip_parse/bazel-pip_parse examples/pip_parse_vendored/bazel-pip_parse_vendored examples/pip_repository_annotations/bazel-pip_repository_annotations examples/py_proto_library/bazel-py_proto_library +gazelle/bazel-gazelle tests/integration/compile_pip_requirements/bazel-compile_pip_requirements tests/integration/ignore_root_user_error/bazel-ignore_root_user_error tests/integration/local_toolchains/bazel-local_toolchains diff --git a/CHANGELOG.md b/CHANGELOG.md index 154b66114b..f696cefde2 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -54,13 +54,21 @@ END_UNRELEASED_TEMPLATE {#v0-0-0-changed} ### Changed -* Nothing changed. +* (rules) On Windows, {obj}`--bootstrap_impl=system_python` is forced. This + allows setting `--bootstrap_impl=script` in bazelrc for mixed-platform + environments. {#v0-0-0-fixed} ### Fixed + * (rules) PyInfo provider is now advertised by py_test, py_binary, and py_library; this allows aspects using required_providers to function correctly. ([#2506](https://github.com/bazel-contrib/rules_python/issues/2506)). +* Fixes when using {obj}`--bootstrap_impl=script`: + * `compile_pip_requirements` now works with it + * The `sys._base_executable` value will reflect the underlying interpreter, + not venv interpreter. + * The {obj}`//python/runtime_env_toolchains:all` toolchain now works with it. {#v0-0-0-added} ### Added diff --git a/examples/pip_repository_annotations/.bazelrc b/examples/pip_repository_annotations/.bazelrc index c16c5a24f2..9397bd31b8 100644 --- a/examples/pip_repository_annotations/.bazelrc +++ b/examples/pip_repository_annotations/.bazelrc @@ -5,4 +5,5 @@ try-import %workspace%/user.bazelrc # is in examples/bzlmod as the `whl_mods` feature. common --noenable_bzlmod common --enable_workspace +common --legacy_external_runfiles=false common --incompatible_python_disallow_native_rules diff --git a/examples/pip_repository_annotations/pip_repository_annotations_test.py b/examples/pip_repository_annotations/pip_repository_annotations_test.py index e41dd4f0f6..219be1ba03 100644 --- a/examples/pip_repository_annotations/pip_repository_annotations_test.py +++ b/examples/pip_repository_annotations/pip_repository_annotations_test.py @@ -21,7 +21,7 @@ import unittest from pathlib import Path -from rules_python.python.runfiles import runfiles +from python.runfiles import runfiles class PipRepositoryAnnotationsTest(unittest.TestCase): @@ -34,11 +34,7 @@ def wheel_pkg_dir(self) -> str: def test_build_content_and_data(self): r = runfiles.Create() - rpath = r.Rlocation( - "pip_repository_annotations_example/external/{}/generated_file.txt".format( - self.wheel_pkg_dir() - ) - ) + rpath = r.Rlocation("{}/generated_file.txt".format(self.wheel_pkg_dir())) generated_file = Path(rpath) self.assertTrue(generated_file.exists()) @@ -47,11 +43,7 @@ def test_build_content_and_data(self): def test_copy_files(self): r = runfiles.Create() - rpath = r.Rlocation( - "pip_repository_annotations_example/external/{}/copied_content/file.txt".format( - self.wheel_pkg_dir() - ) - ) + rpath = r.Rlocation("{}/copied_content/file.txt".format(self.wheel_pkg_dir())) copied_file = Path(rpath) self.assertTrue(copied_file.exists()) @@ -61,7 +53,7 @@ def test_copy_files(self): def test_copy_executables(self): r = runfiles.Create() rpath = r.Rlocation( - "pip_repository_annotations_example/external/{}/copied_content/executable{}".format( + "{}/copied_content/executable{}".format( self.wheel_pkg_dir(), ".exe" if platform.system() == "windows" else ".py", ) @@ -82,7 +74,7 @@ def test_data_exclude_glob(self): current_wheel_version = "0.38.4" r = runfiles.Create() - dist_info_dir = "pip_repository_annotations_example/external/{}/site-packages/wheel-{}.dist-info".format( + dist_info_dir = "{}/site-packages/wheel-{}.dist-info".format( self.wheel_pkg_dir(), current_wheel_version, ) @@ -113,11 +105,8 @@ def test_extra(self): # This test verifies that annotations work correctly for pip packages with extras # specified, in this case requests[security]. r = runfiles.Create() - rpath = r.Rlocation( - "pip_repository_annotations_example/external/{}/generated_file.txt".format( - self.requests_pkg_dir() - ) - ) + path = "{}/generated_file.txt".format(self.requests_pkg_dir()) + rpath = r.Rlocation(path) generated_file = Path(rpath) self.assertTrue(generated_file.exists()) diff --git a/gazelle/WORKSPACE b/gazelle/WORKSPACE index 14a124d5f2..ad428b10cd 100644 --- a/gazelle/WORKSPACE +++ b/gazelle/WORKSPACE @@ -42,6 +42,8 @@ load("//:internal_dev_deps.bzl", "internal_dev_deps") internal_dev_deps() +register_toolchains("@rules_python//python/runtime_env_toolchains:all") + load("//:deps.bzl", _py_gazelle_deps = "gazelle_deps") # gazelle:repository_macro deps.bzl%go_deps diff --git a/python/config_settings/BUILD.bazel b/python/config_settings/BUILD.bazel index 45354e24d9..872d7d1bda 100644 --- a/python/config_settings/BUILD.bazel +++ b/python/config_settings/BUILD.bazel @@ -11,6 +11,7 @@ load( "PrecompileSourceRetentionFlag", "VenvsSitePackages", "VenvsUseDeclareSymlinkFlag", + rp_string_flag = "string_flag", ) load( "//python/private/pypi:flags.bzl", @@ -87,14 +88,27 @@ string_flag( visibility = ["//visibility:public"], ) -string_flag( +rp_string_flag( name = "bootstrap_impl", build_setting_default = BootstrapImplFlag.SYSTEM_PYTHON, + override = select({ + # Windows doesn't yet support bootstrap=script, so force disable it + ":_is_windows": BootstrapImplFlag.SYSTEM_PYTHON, + "//conditions:default": "", + }), values = sorted(BootstrapImplFlag.__members__.values()), # NOTE: Only public because it's an implicit dependency visibility = ["//visibility:public"], ) +# For some reason, @platforms//os:windows can't be directly used +# in the select() for the flag. But it can be used when put behind +# a config_setting(). +config_setting( + name = "_is_windows", + constraint_values = ["@platforms//os:windows"], +) + # This is used for pip and hermetic toolchain resolution. string_flag( name = "py_linux_libc", diff --git a/python/private/BUILD.bazel b/python/private/BUILD.bazel index b63f446be3..9cc8ffc62c 100644 --- a/python/private/BUILD.bazel +++ b/python/private/BUILD.bazel @@ -86,6 +86,7 @@ bzl_library( name = "runtime_env_toolchain_bzl", srcs = ["runtime_env_toolchain.bzl"], deps = [ + ":config_settings_bzl", ":py_exec_tools_toolchain_bzl", ":toolchain_types_bzl", "//python:py_runtime_bzl", diff --git a/python/private/config_settings.bzl b/python/private/config_settings.bzl index e5f9d865d1..2cf7968061 100644 --- a/python/private/config_settings.bzl +++ b/python/private/config_settings.bzl @@ -209,3 +209,33 @@ _current_config = rule( "_template": attr.string(default = _DEBUG_ENV_MESSAGE_TEMPLATE), }, ) + +def is_python_version_at_least(name, **kwargs): + flag_name = "_{}_flag".format(name) + native.config_setting( + name = name, + flag_values = { + flag_name: "yes", + }, + ) + _python_version_at_least( + name = flag_name, + visibility = ["//visibility:private"], + **kwargs + ) + +def _python_version_at_least_impl(ctx): + at_least = tuple(ctx.attr.at_least.split(".")) + current = tuple( + ctx.attr._major_minor[config_common.FeatureFlagInfo].value.split("."), + ) + value = "yes" if current >= at_least else "no" + return [config_common.FeatureFlagInfo(value = value)] + +_python_version_at_least = rule( + implementation = _python_version_at_least_impl, + attrs = { + "at_least": attr.string(mandatory = True), + "_major_minor": attr.label(default = _PYTHON_VERSION_MAJOR_MINOR_FLAG), + }, +) diff --git a/python/private/flags.bzl b/python/private/flags.bzl index c53e4610ff..40ce63b3b0 100644 --- a/python/private/flags.bzl +++ b/python/private/flags.bzl @@ -35,8 +35,38 @@ AddSrcsToRunfilesFlag = FlagEnum( is_enabled = _AddSrcsToRunfilesFlag_is_enabled, ) +def _string_flag_impl(ctx): + if ctx.attr.override: + value = ctx.attr.override + else: + value = ctx.build_setting_value + + if value not in ctx.attr.values: + fail(( + "Invalid value for {name}: got {value}, must " + + "be one of {allowed}" + ).format( + name = ctx.label, + value = value, + allowed = ctx.attr.values, + )) + + return [ + BuildSettingInfo(value = value), + config_common.FeatureFlagInfo(value = value), + ] + +string_flag = rule( + implementation = _string_flag_impl, + build_setting = config.string(flag = True), + attrs = { + "override": attr.string(), + "values": attr.string_list(), + }, +) + def _bootstrap_impl_flag_get_value(ctx): - return ctx.attr._bootstrap_impl_flag[BuildSettingInfo].value + return ctx.attr._bootstrap_impl_flag[config_common.FeatureFlagInfo].value # buildifier: disable=name-conventions BootstrapImplFlag = enum( diff --git a/python/private/get_local_runtime_info.py b/python/private/get_local_runtime_info.py index 0207f56bef..19db3a2935 100644 --- a/python/private/get_local_runtime_info.py +++ b/python/private/get_local_runtime_info.py @@ -22,6 +22,7 @@ "micro": sys.version_info.micro, "include": sysconfig.get_path("include"), "implementation_name": sys.implementation.name, + "base_executable": sys._base_executable, } config_vars = [ diff --git a/python/private/local_runtime_repo.bzl b/python/private/local_runtime_repo.bzl index fb1a8e29ac..ec0643e497 100644 --- a/python/private/local_runtime_repo.bzl +++ b/python/private/local_runtime_repo.bzl @@ -84,6 +84,20 @@ def _local_runtime_repo_impl(rctx): info = json.decode(exec_result.stdout) logger.info(lambda: _format_get_info_result(info)) + # We use base_executable because we want the path within a Python + # installation directory ("PYTHONHOME"). The problems with sys.executable + # are: + # * If we're in an activated venv, then we don't want the venv's + # `bin/python3` path to be used -- it isn't an actual Python installation. + # * If sys.executable is a wrapper (e.g. pyenv), then (1) it may not be + # located within an actual Python installation directory, and (2) it + # can interfer with Python recognizing when it's within a venv. + # + # In some cases, it may be a symlink (usually e.g. `python3->python3.12`), + # but we don't realpath() it to respect what it has decided is the + # appropriate path. + interpreter_path = info["base_executable"] + # NOTE: Keep in sync with recursive glob in define_local_runtime_toolchain_impl repo_utils.watch_tree(rctx, rctx.path(info["include"])) diff --git a/python/private/py_executable.bzl b/python/private/py_executable.bzl index b4cda21b1d..a8c669afd9 100644 --- a/python/private/py_executable.bzl +++ b/python/private/py_executable.bzl @@ -350,6 +350,7 @@ def _create_executable( main_py = main_py, imports = imports, runtime_details = runtime_details, + venv = venv, ) extra_runfiles = ctx.runfiles([stage2_bootstrap] + venv.files_without_interpreter) zip_main = _create_zip_main( @@ -538,11 +539,14 @@ def _create_venv(ctx, output_prefix, imports, runtime_details): ctx.actions.write(pyvenv_cfg, "") runtime = runtime_details.effective_runtime + venvs_use_declare_symlink_enabled = ( VenvsUseDeclareSymlinkFlag.get_value(ctx) == VenvsUseDeclareSymlinkFlag.YES ) + recreate_venv_at_runtime = False - if not venvs_use_declare_symlink_enabled: + if not venvs_use_declare_symlink_enabled or not runtime.supports_build_time_venv: + recreate_venv_at_runtime = True if runtime.interpreter: interpreter_actual_path = runfiles_root_path(ctx, runtime.interpreter.short_path) else: @@ -557,6 +561,8 @@ def _create_venv(ctx, output_prefix, imports, runtime_details): ctx.actions.write(interpreter, "actual:{}".format(interpreter_actual_path)) elif runtime.interpreter: + # Some wrappers around the interpreter (e.g. pyenv) use the program + # name to decide what to do, so preserve the name. py_exe_basename = paths.basename(runtime.interpreter.short_path) # Even though ctx.actions.symlink() is used, using @@ -594,7 +600,8 @@ def _create_venv(ctx, output_prefix, imports, runtime_details): if "t" in runtime.abi_flags: version += "t" - site_packages = "{}/lib/python{}/site-packages".format(venv, version) + venv_site_packages = "lib/python{}/site-packages".format(version) + site_packages = "{}/{}".format(venv, venv_site_packages) pth = ctx.actions.declare_file("{}/bazel.pth".format(site_packages)) ctx.actions.write(pth, "import _bazel_site_init\n") @@ -616,10 +623,12 @@ def _create_venv(ctx, output_prefix, imports, runtime_details): return struct( interpreter = interpreter, - recreate_venv_at_runtime = not venvs_use_declare_symlink_enabled, + recreate_venv_at_runtime = recreate_venv_at_runtime, # Runfiles root relative path or absolute path interpreter_actual_path = interpreter_actual_path, files_without_interpreter = [pyvenv_cfg, pth, site_init] + site_packages_symlinks, + # string; venv-relative path to the site-packages directory. + venv_site_packages = venv_site_packages, ) def _create_site_packages_symlinks(ctx, site_packages): @@ -716,7 +725,8 @@ def _create_stage2_bootstrap( output_sibling, main_py, imports, - runtime_details): + runtime_details, + venv = None): output = ctx.actions.declare_file( # Prepend with underscore to prevent pytest from trying to # process the bootstrap for files starting with `test_` @@ -731,6 +741,14 @@ def _create_stage2_bootstrap( main_py_path = "{}/{}".format(ctx.workspace_name, main_py.short_path) else: main_py_path = "" + + # The stage2 bootstrap uses the venv site-packages location to fix up issues + # that occur when the toolchain doesn't support the build-time venv. + if venv and not runtime.supports_build_time_venv: + venv_rel_site_packages = venv.venv_site_packages + else: + venv_rel_site_packages = "" + ctx.actions.expand_template( template = template, output = output, @@ -741,6 +759,7 @@ def _create_stage2_bootstrap( "%main%": main_py_path, "%main_module%": ctx.attr.main_module, "%target%": str(ctx.label), + "%venv_rel_site_packages%": venv_rel_site_packages, "%workspace_name%": ctx.workspace_name, }, is_executable = True, @@ -766,6 +785,12 @@ def _create_stage1_bootstrap( python_binary_actual = venv.interpreter_actual_path if venv else "" + # Runtime may be None on Windows due to the --python_path flag. + if runtime and runtime.supports_build_time_venv: + resolve_python_binary_at_runtime = "0" + else: + resolve_python_binary_at_runtime = "1" + subs = { "%interpreter_args%": "\n".join([ '"{}"'.format(v) @@ -775,7 +800,9 @@ def _create_stage1_bootstrap( "%python_binary%": python_binary_path, "%python_binary_actual%": python_binary_actual, "%recreate_venv_at_runtime%": str(int(venv.recreate_venv_at_runtime)) if venv else "0", + "%resolve_python_binary_at_runtime%": resolve_python_binary_at_runtime, "%target%": str(ctx.label), + "%venv_rel_site_packages%": venv.venv_site_packages if venv else "", "%workspace_name%": ctx.workspace_name, } diff --git a/python/private/py_runtime_info.bzl b/python/private/py_runtime_info.bzl index 4297391068..d2ae17e360 100644 --- a/python/private/py_runtime_info.bzl +++ b/python/private/py_runtime_info.bzl @@ -67,7 +67,8 @@ def _PyRuntimeInfo_init( stage2_bootstrap_template = None, zip_main_template = None, abi_flags = "", - site_init_template = None): + site_init_template = None, + supports_build_time_venv = True): if (interpreter_path and interpreter) or (not interpreter_path and not interpreter): fail("exactly one of interpreter or interpreter_path must be specified") @@ -119,6 +120,7 @@ def _PyRuntimeInfo_init( "site_init_template": site_init_template, "stage2_bootstrap_template": stage2_bootstrap_template, "stub_shebang": stub_shebang, + "supports_build_time_venv": supports_build_time_venv, "zip_main_template": zip_main_template, } @@ -312,6 +314,28 @@ The following substitutions are made during template expansion: "Shebang" expression prepended to the bootstrapping Python stub script used when executing {obj}`py_binary` targets. Does not apply to Windows. +""", + "supports_build_time_venv": """ +:type: bool + +True if this toolchain supports the build-time created virtual environment. +False if not or unknown. If build-time venv creation isn't supported, then binaries may +fallback to non-venv solutions or creating a venv at runtime. + +In order to use the build-time created virtual environment, a toolchain needs +to meet two criteria: +1. Specifying the underlying executable (e.g. `/usr/bin/python3`, as reported by + `sys._base_executable`) for the venv executable (`$venv/bin/python3`, as reported + by `sys.executable`). This typically requires relative symlinking the venv + path to the underlying path at build time, or using the `PYTHONEXECUTABLE` + environment variable (Python 3.11+) at runtime. +2. Having the build-time created site-packages directory + (`/lib/python{version}/site-packages`) recognized by the runtime + interpreter. This typically requires the Python version to be known at + build-time and match at runtime. + +:::{versionadded} VERSION_NEXT_FEATURE +::: """, "zip_main_template": """ :type: File diff --git a/python/private/py_runtime_rule.bzl b/python/private/py_runtime_rule.bzl index a85f5b25f2..6dadcfeac3 100644 --- a/python/private/py_runtime_rule.bzl +++ b/python/private/py_runtime_rule.bzl @@ -130,6 +130,7 @@ def _py_runtime_impl(ctx): zip_main_template = ctx.file.zip_main_template, abi_flags = abi_flags, site_init_template = ctx.file.site_init_template, + supports_build_time_venv = ctx.attr.supports_build_time_venv, )) if not IS_BAZEL_7_OR_HIGHER: @@ -353,6 +354,17 @@ motivation. Does not apply to Windows. """, ), + "supports_build_time_venv": attr.bool( + doc = """ +Whether this runtime supports virtualenvs created at build time. + +See {obj}`PyRuntimeInfo.supports_build_time_venv` for docs. + +:::{versionadded} VERSION_NEXT_FEATURE +::: +""", + default = True, + ), "zip_main_template": attr.label( default = "//python/private:zip_main_template", allow_single_file = True, diff --git a/python/private/runtime_env_toolchain.bzl b/python/private/runtime_env_toolchain.bzl index 2116012c03..1956ad5e95 100644 --- a/python/private/runtime_env_toolchain.bzl +++ b/python/private/runtime_env_toolchain.bzl @@ -17,6 +17,7 @@ load("@rules_cc//cc:cc_library.bzl", "cc_library") load("//python:py_runtime.bzl", "py_runtime") load("//python:py_runtime_pair.bzl", "py_runtime_pair") load("//python/cc:py_cc_toolchain.bzl", "py_cc_toolchain") +load("//python/private:config_settings.bzl", "is_python_version_at_least") load(":py_exec_tools_toolchain.bzl", "py_exec_tools_toolchain") load(":toolchain_types.bzl", "EXEC_TOOLS_TOOLCHAIN_TYPE", "PY_CC_TOOLCHAIN_TYPE", "TARGET_TOOLCHAIN_TYPE") @@ -38,6 +39,11 @@ def define_runtime_env_toolchain(name): """ base_name = name.replace("_toolchain", "") + supports_build_time_venv = select({ + ":_is_at_least_py3.11": True, + "//conditions:default": False, + }) + py_runtime( name = "_runtime_env_py3_runtime", interpreter = "//python/private:runtime_env_toolchain_interpreter.sh", @@ -45,6 +51,7 @@ def define_runtime_env_toolchain(name): stub_shebang = "#!/usr/bin/env python3", visibility = ["//visibility:private"], tags = ["manual"], + supports_build_time_venv = supports_build_time_venv, ) # This is a dummy runtime whose interpreter_path triggers the native rule @@ -56,6 +63,7 @@ def define_runtime_env_toolchain(name): python_version = "PY3", visibility = ["//visibility:private"], tags = ["manual"], + supports_build_time_venv = supports_build_time_venv, ) py_runtime_pair( @@ -110,3 +118,7 @@ def define_runtime_env_toolchain(name): toolchain_type = PY_CC_TOOLCHAIN_TYPE, visibility = ["//visibility:public"], ) + is_python_version_at_least( + name = "_is_at_least_py3.11", + at_least = "3.11", + ) diff --git a/python/private/runtime_env_toolchain_interpreter.sh b/python/private/runtime_env_toolchain_interpreter.sh index b09bc53e5c..6159d4f38c 100755 --- a/python/private/runtime_env_toolchain_interpreter.sh +++ b/python/private/runtime_env_toolchain_interpreter.sh @@ -53,5 +53,29 @@ documentation for py_runtime_pair \ (https://github.com/bazel-contrib/rules_python/blob/master/docs/python.md#py_runtime_pair)." fi -exec "$PYTHON_BIN" "$@" +# Because this is a wrapper script that invokes Python, it prevents Python from +# detecting virtualenvs like normal (i.e. using the venv symlink to find the +# real interpreter). To work around this, we have to manually detect the venv, +# then trick the interpreter into understanding we're in a virtual env. +self_dir=$(dirname "$0") +if [ -e "$self_dir/pyvenv.cfg" ] || [ -e "$self_dir/../pyvenv.cfg" ]; then + case "$0" in + /*) + venv_bin="$0" + ;; + *) + venv_bin="$PWD/$0" + ;; + esac + # PYTHONEXECUTABLE is also used because `exec -a` doesn't fully trick the + # pyenv wrappers. + # NOTE: The PYTHONEXECUTABLE envvar only works for non-Mac starting in Python 3.11 + export PYTHONEXECUTABLE="$venv_bin" + # Python looks at argv[0] to determine sys.executable, so use exec -a + # to make it think it's the venv's binary, not the actual one invoked. + # NOTE: exec -a isn't strictly posix-compatible, but very widespread + exec -a "$venv_bin" "$PYTHON_BIN" "$@" +else + exec "$PYTHON_BIN" "$@" +fi diff --git a/python/private/site_init_template.py b/python/private/site_init_template.py index 40fb4e4139..a87a0d2a8f 100644 --- a/python/private/site_init_template.py +++ b/python/private/site_init_template.py @@ -125,6 +125,14 @@ def _search_path(name): def _setup_sys_path(): + """Perform Bazel/binary specific sys.path setup. + + NOTE: We do not add _RUNFILES_ROOT to sys.path for two reasons: + 1. Under workspace, it makes every external repository importable. If a Bazel + repository matches a Python import name, they conflict. + 2. Under bzlmod, the repo names in the runfiles directory aren't importable + Python names, so there's no point in adding the runfiles root to sys.path. + """ seen = set(sys.path) python_path_entries = [] @@ -195,5 +203,27 @@ def _maybe_add_path(path): return coverage_setup +def _fixup_sys_base_executable(): + """Fixup sys._base_executable to account for Bazel-specific pyvenv.cfg + + The pyvenv.cfg created for py_binary leaves the `home` key unset. A + side-effect of this is `sys._base_executable` points to the venv executable, + not the actual executable. This mostly doesn't matter, but does affect + using the venv module to create venvs (they point to the venv executable, not + the actual executable). + """ + # Must have been set correctly? + if sys.executable != sys._base_executable: + return + # Not in a venv, so don't touch anything. + if sys.prefix == sys.base_prefix: + return + exe = os.path.realpath(sys.executable) + _print_verbose("setting sys._base_executable:", exe) + sys._base_executable = exe + + +_fixup_sys_base_executable() + COVERAGE_SETUP = _setup_sys_path() _print_verbose("DONE") diff --git a/python/private/stage1_bootstrap_template.sh b/python/private/stage1_bootstrap_template.sh index c487624934..d992b55cae 100644 --- a/python/private/stage1_bootstrap_template.sh +++ b/python/private/stage1_bootstrap_template.sh @@ -9,7 +9,8 @@ fi # runfiles-relative path STAGE2_BOOTSTRAP="%stage2_bootstrap%" -# runfiles-relative path to python interpreter to use +# runfiles-relative path to python interpreter to use. +# This is the `bin/python3` path in the binary's venv. PYTHON_BINARY='%python_binary%' # The path that PYTHON_BINARY should symlink to. # runfiles-relative path, absolute path, or single word. @@ -18,8 +19,17 @@ PYTHON_BINARY_ACTUAL="%python_binary_actual%" # 0 or 1 IS_ZIPFILE="%is_zipfile%" -# 0 or 1 +# 0 or 1. +# If 1, then a venv will be created at runtime that replicates what would have +# been the build-time structure. RECREATE_VENV_AT_RUNTIME="%recreate_venv_at_runtime%" +# 0 or 1 +# If 1, then the path to python will be resolved by running +# PYTHON_BINARY_ACTUAL to determine the actual underlying interpreter. +RESOLVE_PYTHON_BINARY_AT_RUNTIME="%resolve_python_binary_at_runtime%" +# venv-relative path to the site-packages +# e.g. lib/python3.12t/site-packages +VENV_REL_SITE_PACKAGES="%venv_rel_site_packages%" # array of strings declare -a INTERPRETER_ARGS_FROM_TARGET=( @@ -152,34 +162,72 @@ elif [[ "$RECREATE_VENV_AT_RUNTIME" == "1" ]]; then fi fi - if [[ "$PYTHON_BINARY_ACTUAL" == /* ]]; then - # An absolute path, i.e. platform runtime, e.g. /usr/bin/python3 - symlink_to=$PYTHON_BINARY_ACTUAL - elif [[ "$PYTHON_BINARY_ACTUAL" == */* ]]; then - # A runfiles-relative path - symlink_to="$RUNFILES_DIR/$PYTHON_BINARY_ACTUAL" - else - # A plain word, e.g. "python3". Symlink to where PATH leads - symlink_to=$(which $PYTHON_BINARY_ACTUAL) - # Guard against trying to symlink to an empty value - if [[ $? -ne 0 ]]; then - echo >&2 "ERROR: Python to use not found on PATH: $PYTHON_BINARY_ACTUAL" - exit 1 - fi - fi - mkdir -p "$venv/bin" # Match the basename; some tools, e.g. pyvenv key off the executable name python_exe="$venv/bin/$(basename $PYTHON_BINARY_ACTUAL)" + if [[ ! -e "$python_exe" ]]; then - ln -s "$symlink_to" "$python_exe" + if [[ "$PYTHON_BINARY_ACTUAL" == /* ]]; then + # An absolute path, i.e. platform runtime, e.g. /usr/bin/python3 + python_exe_actual=$PYTHON_BINARY_ACTUAL + elif [[ "$PYTHON_BINARY_ACTUAL" == */* ]]; then + # A runfiles-relative path + python_exe_actual="$RUNFILES_DIR/$PYTHON_BINARY_ACTUAL" + else + # A plain word, e.g. "python3". Symlink to where PATH leads + python_exe_actual=$(which $PYTHON_BINARY_ACTUAL) + # Guard against trying to symlink to an empty value + if [[ $? -ne 0 ]]; then + echo >&2 "ERROR: Python to use not found on PATH: $PYTHON_BINARY_ACTUAL" + exit 1 + fi + fi + + runfiles_venv="$RUNFILES_DIR/$(dirname $(dirname $PYTHON_BINARY))" + # When RESOLVE_PYTHON_BINARY_AT_RUNTIME is true, it means the toolchain + # has thrown two complications at us: + # 1. The build-time assumption of the Python version may not match the + # runtime Python version. The site-packages directory path includes the + # Python version, so when the versions don't match, the runtime won't + # find it. + # 2. The interpreter might be a wrapper script, which interferes with Python's + # ability to detect when it's within a venv. Starting in Python 3.11, + # the PYTHONEXECUTABLE environment variable can fix this, but due to (1), + # we don't know if that is supported without running Python. + # To fix (1), we symlink the desired site-packages path to the build-time + # directory. Hopefully the version mismatch is OK :D. + # To fix (2), we determine the actual underlying interpreter and symlink + # to that. + if [[ "$RESOLVE_PYTHON_BINARY_AT_RUNTIME" == "1" ]]; then + { + read -r resolved_py_exe + read -r resolved_site_packages + } < <("$python_exe_actual" -I < Date: Mon, 21 Apr 2025 17:00:40 -0700 Subject: [PATCH 183/922] fix: escape more invalid repo string characters (#2801) Also escape plus and percent when generating the repo name from the wheel version. Sometimes they have such characters in them. Fixes https://github.com/bazel-contrib/rules_python/issues/2799 Co-authored-by: Richard Levasseur --- python/private/pypi/whl_repo_name.bzl | 2 +- tests/pypi/whl_repo_name/whl_repo_name_tests.bzl | 12 ++++++++++++ 2 files changed, 13 insertions(+), 1 deletion(-) diff --git a/python/private/pypi/whl_repo_name.bzl b/python/private/pypi/whl_repo_name.bzl index 02a7c8142c..2b3b5418aa 100644 --- a/python/private/pypi/whl_repo_name.bzl +++ b/python/private/pypi/whl_repo_name.bzl @@ -44,7 +44,7 @@ def whl_repo_name(filename, sha256): else: parsed = parse_whl_name(filename) name = normalize_name(parsed.distribution) - version = parsed.version.replace(".", "_").replace("!", "_") + version = parsed.version.replace(".", "_").replace("!", "_").replace("+", "_").replace("%", "_") python_tag, _, _ = parsed.python_tag.partition(".") abi_tag, _, _ = parsed.abi_tag.partition(".") platform_tag, _, _ = parsed.platform_tag.partition(".") diff --git a/tests/pypi/whl_repo_name/whl_repo_name_tests.bzl b/tests/pypi/whl_repo_name/whl_repo_name_tests.bzl index f0d1d059e1..35e6bcdf9f 100644 --- a/tests/pypi/whl_repo_name/whl_repo_name_tests.bzl +++ b/tests/pypi/whl_repo_name/whl_repo_name_tests.bzl @@ -54,6 +54,18 @@ def _test_platform_whl(env): _tests.append(_test_platform_whl) +def _test_name_with_plus(env): + got = whl_repo_name("gptqmodel-2.0.0+cu126torch2.6-cp312-cp312-linux_x86_64.whl", "") + env.expect.that_str(got).equals("gptqmodel_2_0_0_cu126torch2_6_cp312_cp312_linux_x86_64") + +_tests.append(_test_name_with_plus) + +def _test_name_with_percent(env): + got = whl_repo_name("gptqmodel-2.0.0%2Bcu126torch2.6-cp312-cp312-linux_x86_64.whl", "") + env.expect.that_str(got).equals("gptqmodel_2_0_0_2Bcu126torch2_6_cp312_cp312_linux_x86_64") + +_tests.append(_test_name_with_percent) + def whl_repo_name_test_suite(name): """Create the test suite. From 1d69ad68d7959570acde61d8705f1f437c0691b0 Mon Sep 17 00:00:00 2001 From: Keith Smiley Date: Tue, 22 Apr 2025 05:49:15 -0700 Subject: [PATCH 184/922] fix: parsing metadata with inline licenses (#2806) The wheel `METADATA` parsing implemented in 1.4 missed the fact that whitespace is significant and sometimes License is included inline in the `METADATA` file itself. This change ensures that we stop parsing the `METADATA` file only on first completely empty line. Fixes https://github.com/bazel-contrib/rules_python/issues/2796 --------- Co-authored-by: Ignas Anikevicius <240938+aignas@users.noreply.github.com> --- python/private/pypi/whl_metadata.bzl | 2 +- .../pypi/whl_metadata/whl_metadata_tests.bzl | 31 +++++++++++++++++++ 2 files changed, 32 insertions(+), 1 deletion(-) diff --git a/python/private/pypi/whl_metadata.bzl b/python/private/pypi/whl_metadata.bzl index 8a86ffbff1..cf2d51afda 100644 --- a/python/private/pypi/whl_metadata.bzl +++ b/python/private/pypi/whl_metadata.bzl @@ -52,7 +52,7 @@ def parse_whl_metadata(contents): "version": "", } for line in contents.strip().split("\n"): - if not line.strip(): + if not line: # Stop parsing on first empty line, which marks the end of the # headers containing the metadata. break diff --git a/tests/pypi/whl_metadata/whl_metadata_tests.bzl b/tests/pypi/whl_metadata/whl_metadata_tests.bzl index 4acbc9213d..329423a26c 100644 --- a/tests/pypi/whl_metadata/whl_metadata_tests.bzl +++ b/tests/pypi/whl_metadata/whl_metadata_tests.bzl @@ -140,6 +140,37 @@ Requires-Dist: this will be ignored _tests.append(_test_parse_metadata_all) +def _test_parse_metadata_multiline_license(env): + got = _parse_whl_metadata( + env, + # NOTE: The trailing whitespace here is meaningful as an empty line + # denotes the end of the header. + contents = """\ +Name: foo +Version: 0.0.1 +License: some License + + some line + + another line + +Requires-Dist: bar; extra == "all" +Provides-Extra: all + +Requires-Dist: this will be ignored +""", + ) + got.name().equals("foo") + got.version().equals("0.0.1") + got.requires_dist().contains_exactly([ + "bar; extra == \"all\"", + ]) + got.provides_extra().contains_exactly([ + "all", + ]) + +_tests.append(_test_parse_metadata_multiline_license) + def whl_metadata_test_suite(name): # buildifier: disable=function-docstring test_suite( name = name, From 830261e4b1c427c7f646f689fedf45117dd54aad Mon Sep 17 00:00:00 2001 From: Ignas Anikevicius <240938+aignas@users.noreply.github.com> Date: Wed, 23 Apr 2025 01:45:10 +0900 Subject: [PATCH 185/922] test(pypi): add a test case for simpleapi html parsing with % (#2811) In addition to #2801 I wanted to ensure that we are getting the correct filename when downloading wheels. It seems that the `%` in the wheel filename might get through wheels that get referenced via direct URL in the requirements.txt files. --------- Co-authored-by: Richard Levasseur Co-authored-by: Richard Levasseur --- .../parse_simpleapi_html_tests.bzl | 19 +++++++++++++++++++ 1 file changed, 19 insertions(+) diff --git a/tests/pypi/parse_simpleapi_html/parse_simpleapi_html_tests.bzl b/tests/pypi/parse_simpleapi_html/parse_simpleapi_html_tests.bzl index abaa7a6a49..191079d214 100644 --- a/tests/pypi/parse_simpleapi_html/parse_simpleapi_html_tests.bzl +++ b/tests/pypi/parse_simpleapi_html/parse_simpleapi_html_tests.bzl @@ -303,6 +303,25 @@ def _test_whls(env): yanked = False, ), ), + ( + struct( + attrs = [ + 'href="/whl/cpu/torch-2.6.0%2Bcpu-cp39-cp39-manylinux_2_28_aarch64.whl#sha256=deadbeef"', + ], + filename = "torch-2.6.0+cpu-cp39-cp39-manylinux_2_28_aarch64.whl", + url = "https://example.org/", + ), + struct( + filename = "torch-2.6.0+cpu-cp39-cp39-manylinux_2_28_aarch64.whl", + metadata_sha256 = "", + metadata_url = "", + sha256 = "deadbeef", + version = "2.6.0+cpu", + # A URL with % could occur if directly written in requirements. + url = "https://example.org/whl/cpu/torch-2.6.0%2Bcpu-cp39-cp39-manylinux_2_28_aarch64.whl", + yanked = False, + ), + ), ] for (input, want) in tests: From fe88b2381b5d272437593dc3604fc834114e4a15 Mon Sep 17 00:00:00 2001 From: Brandon Chinn Date: Tue, 22 Apr 2025 23:39:02 -0700 Subject: [PATCH 186/922] build: Run pre-commit everywhere (#2808) Fix pre-commit issues. Would be nice to run `pre-commit run -a` in CI, but won't fix that now --------- Co-authored-by: Douglas Thor --- .bazelrc | 4 ++-- .pre-commit-config.yaml | 2 +- .../foo_external/py_binary_with_proto.py | 1 + .../wheel/lib/module_with_type_annotations.py | 1 + examples/wheel/test_publish.py | 2 +- examples/wheel/wheel_test.py | 17 +++++++++-------- .../dependency_resolution_order/__init__.py | 3 +-- .../py312_syntax/pep_695_type_parameter.py | 1 - .../dependency_resolver/dependency_resolver.py | 6 ++---- tests/integration/runner.py | 5 ++++- tests/no_unsafe_paths/test.py | 4 ++-- tools/wheelmaker.py | 12 ++++++++---- 12 files changed, 32 insertions(+), 26 deletions(-) diff --git a/.bazelrc b/.bazelrc index 4e6f2fa187..d2e0721526 100644 --- a/.bazelrc +++ b/.bazelrc @@ -4,8 +4,8 @@ # (Note, we cannot use `common --deleted_packages` because the bazel version command doesn't support it) # To update these lines, execute # `bazel run @rules_bazel_integration_test//tools:update_deleted_packages` -build --deleted_packages=examples/build_file_generation,examples/build_file_generation/random_number_generator,examples/bzlmod,examples/bzlmod_build_file_generation,examples/bzlmod_build_file_generation/other_module/other_module/pkg,examples/bzlmod_build_file_generation/runfiles,examples/bzlmod/entry_points,examples/bzlmod/entry_points/tests,examples/bzlmod/libs/my_lib,examples/bzlmod/other_module,examples/bzlmod/other_module/other_module/pkg,examples/bzlmod/patches,examples/bzlmod/py_proto_library,examples/bzlmod/py_proto_library/example.com/another_proto,examples/bzlmod/py_proto_library/example.com/proto,examples/bzlmod/runfiles,examples/bzlmod/tests,examples/bzlmod/tests/other_module,examples/bzlmod/whl_mods,examples/multi_python_versions/libs/my_lib,examples/multi_python_versions/requirements,examples/multi_python_versions/tests,examples/pip_parse,examples/pip_parse_vendored,examples/pip_repository_annotations,examples/py_proto_library,examples/py_proto_library/example.com/another_proto,examples/py_proto_library/example.com/proto,gazelle,gazelle/manifest,gazelle/manifest/generate,gazelle/manifest/hasher,gazelle/manifest/test,gazelle/modules_mapping,gazelle/python,gazelle/pythonconfig,gazelle/python/private,tests/integration/compile_pip_requirements,tests/integration/compile_pip_requirements_test_from_external_repo,tests/integration/custom_commands,tests/integration/ignore_root_user_error,tests/integration/ignore_root_user_error/submodule,tests/integration/local_toolchains,tests/integration/pip_parse,tests/integration/pip_parse/empty,tests/integration/py_cc_toolchain_registered,tests/modules/other,tests/modules/other/nspkg_delta,tests/modules/other/nspkg_gamma -query --deleted_packages=examples/build_file_generation,examples/build_file_generation/random_number_generator,examples/bzlmod,examples/bzlmod_build_file_generation,examples/bzlmod_build_file_generation/other_module/other_module/pkg,examples/bzlmod_build_file_generation/runfiles,examples/bzlmod/entry_points,examples/bzlmod/entry_points/tests,examples/bzlmod/libs/my_lib,examples/bzlmod/other_module,examples/bzlmod/other_module/other_module/pkg,examples/bzlmod/patches,examples/bzlmod/py_proto_library,examples/bzlmod/py_proto_library/example.com/another_proto,examples/bzlmod/py_proto_library/example.com/proto,examples/bzlmod/runfiles,examples/bzlmod/tests,examples/bzlmod/tests/other_module,examples/bzlmod/whl_mods,examples/multi_python_versions/libs/my_lib,examples/multi_python_versions/requirements,examples/multi_python_versions/tests,examples/pip_parse,examples/pip_parse_vendored,examples/pip_repository_annotations,examples/py_proto_library,examples/py_proto_library/example.com/another_proto,examples/py_proto_library/example.com/proto,gazelle,gazelle/manifest,gazelle/manifest/generate,gazelle/manifest/hasher,gazelle/manifest/test,gazelle/modules_mapping,gazelle/python,gazelle/pythonconfig,gazelle/python/private,tests/integration/compile_pip_requirements,tests/integration/compile_pip_requirements_test_from_external_repo,tests/integration/custom_commands,tests/integration/ignore_root_user_error,tests/integration/ignore_root_user_error/submodule,tests/integration/local_toolchains,tests/integration/pip_parse,tests/integration/pip_parse/empty,tests/integration/py_cc_toolchain_registered,tests/modules/other,tests/modules/other/nspkg_delta,tests/modules/other/nspkg_gamma +build --deleted_packages=examples/build_file_generation,examples/build_file_generation/random_number_generator,examples/bzlmod,examples/bzlmod/entry_points,examples/bzlmod/entry_points/tests,examples/bzlmod/libs/my_lib,examples/bzlmod/other_module,examples/bzlmod/other_module/other_module/pkg,examples/bzlmod/patches,examples/bzlmod/py_proto_library,examples/bzlmod/py_proto_library/example.com/another_proto,examples/bzlmod/py_proto_library/example.com/proto,examples/bzlmod/runfiles,examples/bzlmod/tests,examples/bzlmod/tests/other_module,examples/bzlmod/whl_mods,examples/bzlmod_build_file_generation,examples/bzlmod_build_file_generation/other_module/other_module/pkg,examples/bzlmod_build_file_generation/runfiles,examples/multi_python_versions/libs/my_lib,examples/multi_python_versions/requirements,examples/multi_python_versions/tests,examples/pip_parse,examples/pip_parse_vendored,examples/pip_repository_annotations,examples/py_proto_library,examples/py_proto_library/example.com/another_proto,examples/py_proto_library/example.com/proto,gazelle,gazelle/manifest,gazelle/manifest/generate,gazelle/manifest/hasher,gazelle/manifest/test,gazelle/modules_mapping,gazelle/python,gazelle/python/private,gazelle/pythonconfig,tests/integration/compile_pip_requirements,tests/integration/compile_pip_requirements_test_from_external_repo,tests/integration/custom_commands,tests/integration/ignore_root_user_error,tests/integration/ignore_root_user_error/submodule,tests/integration/local_toolchains,tests/integration/pip_parse,tests/integration/pip_parse/empty,tests/integration/py_cc_toolchain_registered,tests/modules/other,tests/modules/other/nspkg_delta,tests/modules/other/nspkg_gamma +query --deleted_packages=examples/build_file_generation,examples/build_file_generation/random_number_generator,examples/bzlmod,examples/bzlmod/entry_points,examples/bzlmod/entry_points/tests,examples/bzlmod/libs/my_lib,examples/bzlmod/other_module,examples/bzlmod/other_module/other_module/pkg,examples/bzlmod/patches,examples/bzlmod/py_proto_library,examples/bzlmod/py_proto_library/example.com/another_proto,examples/bzlmod/py_proto_library/example.com/proto,examples/bzlmod/runfiles,examples/bzlmod/tests,examples/bzlmod/tests/other_module,examples/bzlmod/whl_mods,examples/bzlmod_build_file_generation,examples/bzlmod_build_file_generation/other_module/other_module/pkg,examples/bzlmod_build_file_generation/runfiles,examples/multi_python_versions/libs/my_lib,examples/multi_python_versions/requirements,examples/multi_python_versions/tests,examples/pip_parse,examples/pip_parse_vendored,examples/pip_repository_annotations,examples/py_proto_library,examples/py_proto_library/example.com/another_proto,examples/py_proto_library/example.com/proto,gazelle,gazelle/manifest,gazelle/manifest/generate,gazelle/manifest/hasher,gazelle/manifest/test,gazelle/modules_mapping,gazelle/python,gazelle/python/private,gazelle/pythonconfig,tests/integration/compile_pip_requirements,tests/integration/compile_pip_requirements_test_from_external_repo,tests/integration/custom_commands,tests/integration/ignore_root_user_error,tests/integration/ignore_root_user_error/submodule,tests/integration/local_toolchains,tests/integration/pip_parse,tests/integration/pip_parse/empty,tests/integration/py_cc_toolchain_registered,tests/modules/other,tests/modules/other/nspkg_delta,tests/modules/other/nspkg_gamma test --test_output=errors diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml index 2b451e89fa..67a02fc6c0 100644 --- a/.pre-commit-config.yaml +++ b/.pre-commit-config.yaml @@ -38,7 +38,7 @@ repos: - --profile - black - repo: https://github.com/psf/black - rev: 23.1.0 + rev: 25.1.0 hooks: - id: black - repo: local diff --git a/examples/bzlmod/py_proto_library/foo_external/py_binary_with_proto.py b/examples/bzlmod/py_proto_library/foo_external/py_binary_with_proto.py index be34264b5a..67e798bb8f 100644 --- a/examples/bzlmod/py_proto_library/foo_external/py_binary_with_proto.py +++ b/examples/bzlmod/py_proto_library/foo_external/py_binary_with_proto.py @@ -2,4 +2,5 @@ if __name__ == "__main__": import my_proto_pb2 + sys.exit(0) diff --git a/examples/wheel/lib/module_with_type_annotations.py b/examples/wheel/lib/module_with_type_annotations.py index 13e0895160..eda57bae6a 100644 --- a/examples/wheel/lib/module_with_type_annotations.py +++ b/examples/wheel/lib/module_with_type_annotations.py @@ -12,5 +12,6 @@ # See the License for the specific language governing permissions and # limitations under the License. + def function(): return "qux" diff --git a/examples/wheel/test_publish.py b/examples/wheel/test_publish.py index 47134d11f3..e6ec80721b 100644 --- a/examples/wheel/test_publish.py +++ b/examples/wheel/test_publish.py @@ -104,7 +104,7 @@ def test_upload_and_query_simple_api(self):

Links for example-minimal-library

- example_minimal_library-0.0.1-py3-none-any.whl
+ example_minimal_library-0.0.1-py3-none-any.whl
""" self.assertEqual( diff --git a/examples/wheel/wheel_test.py b/examples/wheel/wheel_test.py index 9ec150301d..35803da742 100644 --- a/examples/wheel/wheel_test.py +++ b/examples/wheel/wheel_test.py @@ -85,7 +85,7 @@ def test_py_library_wheel(self): ], ) self.assertFileSha256Equal( - filename, "0cbf4ec574676015af595f570caf4ae2812f994f6338e247b002b4e496b6fbd5" + filename, "a73acae23590c7a8d4365c888c1f12f0399b7af27169ea99fc7a00f402833926" ) def test_py_package_wheel(self): @@ -110,7 +110,7 @@ def test_py_package_wheel(self): ], ) self.assertFileSha256Equal( - filename, "22aff90dd3c8c30c3ce2b729bb793cab0bd2668a6810de232677a0354ce79cae" + filename, "a76001500453dbd1d778821dcaba165d56db502c854cef9381dd3f8f89caee11" ) def test_customized_wheel(self): @@ -144,6 +144,7 @@ def test_customized_wheel(self): "example_customized-0.0.1.dist-info/entry_points.txt" ) + print(record_contents) self.assertEqual( record_contents, # The entries are guaranteed to be sorted. @@ -151,7 +152,7 @@ def test_customized_wheel(self): "examples/wheel/lib/data,with,commas.txt",sha256=9vJKEdfLu8bZRArKLroPZJh1XKkK3qFMXiM79MBL2Sg,12 examples/wheel/lib/data.txt,sha256=9vJKEdfLu8bZRArKLroPZJh1XKkK3qFMXiM79MBL2Sg,12 examples/wheel/lib/module_with_data.py,sha256=8s0Khhcqz3yVsBKv2IB5u4l4TMKh7-c_V6p65WVHPms,637 -examples/wheel/lib/module_with_type_annotations.py,sha256=MM2cFQsCBaUnzGiEGT5r07jhKSaCVRh5Paw_YLyrS-w,636 +examples/wheel/lib/module_with_type_annotations.py,sha256=2p_0YFT0TBUufbGCAR_u2vtxF1nM0lf3dX4VGeUtYq0,637 examples/wheel/lib/module_with_type_annotations.pyi,sha256=fja3ql_WRJ1qO8jyZjWWrTTMcg1J7EpOQivOHY_8vI4,630 examples/wheel/lib/simple_module.py,sha256=z2hwciab_XPNIBNH8B1Q5fYgnJvQTeYf0ZQJpY8yLLY,637 examples/wheel/main.py,sha256=mFiRfzQEDwCHr-WVNQhOH26M42bw1UMF6IoqvtuDTrw,1047 @@ -205,7 +206,7 @@ def test_customized_wheel(self): second = second.main:s""", ) self.assertFileSha256Equal( - filename, "657a938a6fdd6f38bf73d1d91016ffff85d68cf29ca390692a3e9d923dd0e39e" + filename, "941c0d79f4ca67cfa0028248bd0606db7fc69953ff9c7c73ac26a3e6d3c23587" ) def test_filename_escaping(self): @@ -277,7 +278,7 @@ def test_custom_package_root_wheel(self): for line in record_contents.splitlines(): self.assertFalse(line.startswith("/")) self.assertFileSha256Equal( - filename, "d415edbf8f326161674c1fa260e364dd44f2a0311e2f596284320ea52d2a8bdb" + filename, "7bd959b7efe9e325b30a6559177a1a4f22ac7a68fade310845916276110e9287" ) def test_custom_package_root_multi_prefix_wheel(self): @@ -311,7 +312,7 @@ def test_custom_package_root_multi_prefix_wheel(self): for line in record_contents.splitlines(): self.assertFalse(line.startswith("/")) self.assertFileSha256Equal( - filename, "6b76a1178c90996feaf3f9417f350c4a67f90f4247647fd4fd552858dc372d4b" + filename, "caf51e22bdcd3c6c766c8903319ce717daeb6caac577d14e16326a8597981854" ) def test_custom_package_root_multi_prefix_reverse_order_wheel(self): @@ -345,7 +346,7 @@ def test_custom_package_root_multi_prefix_reverse_order_wheel(self): for line in record_contents.splitlines(): self.assertFalse(line.startswith("/")) self.assertFileSha256Equal( - filename, "f976f0bb1c7d753e8c41629d6b79fb09908c6ecd2fec006816879fc86b664f3f" + filename, "9e8c0baa408b829dec691a5e8d3bc040be0bbfcc95c0eee19e1e5ffadea4a059" ) def test_python_requires_wheel(self): @@ -370,7 +371,7 @@ def test_python_requires_wheel(self): """, ) self.assertFileSha256Equal( - filename, "f3b74ce429c3324b87f8d1cc7dc33be1493f54bb88d546a7d53be7587b82c1a7" + filename, "b47f3eaf4f9fa4685a58c7415ba1feddd39635ae26c18473504f7d7e62e8ce07" ) def test_python_abi3_binary_wheel(self): diff --git a/gazelle/python/testdata/dependency_resolution_order/__init__.py b/gazelle/python/testdata/dependency_resolution_order/__init__.py index e2d0a8a979..4b40aa9f54 100644 --- a/gazelle/python/testdata/dependency_resolution_order/__init__.py +++ b/gazelle/python/testdata/dependency_resolution_order/__init__.py @@ -22,9 +22,8 @@ # we can still override "third_party.foo.bar" import third_party.foo.bar -from third_party import baz - import third_party +from third_party import baz _ = sys _ = bar diff --git a/gazelle/python/testdata/py312_syntax/pep_695_type_parameter.py b/gazelle/python/testdata/py312_syntax/pep_695_type_parameter.py index eff06de5a7..eb6263b334 100644 --- a/gazelle/python/testdata/py312_syntax/pep_695_type_parameter.py +++ b/gazelle/python/testdata/py312_syntax/pep_695_type_parameter.py @@ -17,6 +17,5 @@ def search_one_more_level[T]( import _other_module - if __name__ == "__main__": pass diff --git a/python/private/pypi/dependency_resolver/dependency_resolver.py b/python/private/pypi/dependency_resolver/dependency_resolver.py index 293377dc6d..89c9123a61 100644 --- a/python/private/pypi/dependency_resolver/dependency_resolver.py +++ b/python/private/pypi/dependency_resolver/dependency_resolver.py @@ -185,11 +185,9 @@ def main( # and we should copy the updated requirements back to the source tree. if not absolute_output_file.samefile(requirements_file_tree): atexit.register( - lambda: shutil.copy( - absolute_output_file, requirements_file_tree - ) + lambda: shutil.copy(absolute_output_file, requirements_file_tree) ) - cli(argv, standalone_mode = False) + cli(argv, standalone_mode=False) requirements_file_relative_path = Path(requirements_file_relative) content = requirements_file_relative_path.read_text() content = content.replace(absolute_path_prefix, "") diff --git a/tests/integration/runner.py b/tests/integration/runner.py index 9414a865c0..2534ab2d90 100644 --- a/tests/integration/runner.py +++ b/tests/integration/runner.py @@ -23,12 +23,15 @@ _logger = logging.getLogger(__name__) + class ExecuteError(Exception): def __init__(self, result): self.result = result + def __str__(self): return self.result.describe() + class ExecuteResult: def __init__( self, @@ -83,7 +86,7 @@ def setUp(self): "TMP": str(self.tmp_dir), # For some reason, this is necessary for Bazel 6.4 to work. # If not present, it can't find some bash helpers in @bazel_tools - "RUNFILES_DIR": os.environ["TEST_SRCDIR"] + "RUNFILES_DIR": os.environ["TEST_SRCDIR"], } def run_bazel(self, *args: str, check: bool = True) -> ExecuteResult: diff --git a/tests/no_unsafe_paths/test.py b/tests/no_unsafe_paths/test.py index 893add2f62..4727a02995 100644 --- a/tests/no_unsafe_paths/test.py +++ b/tests/no_unsafe_paths/test.py @@ -40,5 +40,5 @@ def test_no_unsafe_paths_in_search_path(self): self.assertEqual(os.path.basename(sys.path[0]), archive) -if __name__ == '__main__': - unittest.main() \ No newline at end of file +if __name__ == "__main__": + unittest.main() diff --git a/tools/wheelmaker.py b/tools/wheelmaker.py index 908b3fe956..28ec039741 100644 --- a/tools/wheelmaker.py +++ b/tools/wheelmaker.py @@ -217,9 +217,11 @@ def add_recordfile(self): filename = filename.lstrip("/") writer.writerow( ( - c - if isinstance(c, str) - else c.decode("utf-8", "surrogateescape") + ( + c + if isinstance(c, str) + else c.decode("utf-8", "surrogateescape") + ) for c in (filename, digest, size) ) ) @@ -604,7 +606,9 @@ def get_new_requirement_line(reqs_text, extra): # File is empty # So replace the meta_line entirely, including removing newline chars else: - metadata = re.sub(re.escape(meta_line) + r"(?:\r?\n)?", "", metadata, count=1) + metadata = re.sub( + re.escape(meta_line) + r"(?:\r?\n)?", "", metadata, count=1 + ) maker.add_metadata( metadata=metadata, From e32b08f2b01b972aed2e94def5c22512604ded93 Mon Sep 17 00:00:00 2001 From: Brandon Chinn Date: Wed, 23 Apr 2025 09:31:08 -0700 Subject: [PATCH 187/922] refactor/docs: improve compile_pip_requirements error message and docs (#2792) Resolution failure is the most common error from pip-compile, so we should make sure the error message is as clean as it can be. Previously, the output was cluttered with the exception traceback, which makes the actual error hard to see (several nested traceback). The new output shortens it with a nicer message: ``` Checking _main/requirements_lock.txt ERROR: Cannot install requests<2.24 and requests~=2.25.1 because these package versions have conflicting dependencies. ResolutionImpossible: for help visit https://pip.pypa.io/en/latest/topics/dependency-resolution/#dealing-with-dependency-conflicts ``` Fixes #2763 --------- Co-authored-by: Richard Levasseur --- docs/pypi-dependencies.md | 39 +++++- .../dependency_resolver.py | 111 +++++++++++------- python/private/pypi/pip_compile.bzl | 2 +- 3 files changed, 105 insertions(+), 47 deletions(-) diff --git a/docs/pypi-dependencies.md b/docs/pypi-dependencies.md index 6cc0da6cb4..4ec40bc889 100644 --- a/docs/pypi-dependencies.md +++ b/docs/pypi-dependencies.md @@ -5,8 +5,40 @@ Using PyPI packages (aka "pip install") involves two main steps. -1. [Installing third party packages](#installing-third-party-packages) -2. [Using third party packages as dependencies](#using-third-party-packages) +1. [Generating requirements file](#generating-requirements-file) +2. [Installing third party packages](#installing-third-party-packages) +3. [Using third party packages as dependencies](#using-third-party-packages) + +{#generating-requirements-file} +## Generating requirements file + +Generally, when working on a Python project, you'll have some dependencies that themselves have other dependencies. You might also specify dependency bounds instead of specific versions. So you'll need to generate a full list of all transitive dependencies and pinned versions for every dependency. + +Typically, you'd have your dependencies specified in `pyproject.toml` or `requirements.in` and generate the full pinned list of dependencies in `requirements_lock.txt`, which you can manage with the `compile_pip_requirements` Bazel rule: + +```starlark +load("@rules_python//python:pip.bzl", "compile_pip_requirements") + +compile_pip_requirements( + name = "requirements", + src = "requirements.in", + requirements_txt = "requirements_lock.txt", +) +``` + +This rule generates two targets: +- `bazel run [name].update` will regenerate the `requirements_txt` file +- `bazel test [name]_test` will test that the `requirements_txt` file is up to date + +For more documentation, see the API docs under {obj}`@rules_python//python:pip.bzl`. + +Once you generate this fully specified list of requirements, you can install the requirements with the instructions in [Installing third party packages](#installing-third-party-packages). + +:::{warning} +If you're specifying dependencies in `pyproject.toml`, make sure to include the `[build-system]` configuration, with pinned dependencies. `compile_pip_requirements` will use the build system specified to read your project's metadata, and you might see non-hermetic behavior if you don't pin the build system. + +Not specifying `[build-system]` at all will result in using a default `[build-system]` configuration, which uses unpinned versions ([ref](https://peps.python.org/pep-0518/#build-system-table)). +::: {#installing-third-party-packages} ## Installing third party packages @@ -27,8 +59,7 @@ pip.parse( ) use_repo(pip, "my_deps") ``` -For more documentation, including how the rules can update/create a requirements -file, see the bzlmod examples under the {gh-path}`examples` folder or the documentation +For more documentation, see the bzlmod examples under the {gh-path}`examples` folder or the documentation for the {obj}`@rules_python//python/extensions:pip.bzl` extension. ```{note} diff --git a/python/private/pypi/dependency_resolver/dependency_resolver.py b/python/private/pypi/dependency_resolver/dependency_resolver.py index 89c9123a61..ada0763558 100644 --- a/python/private/pypi/dependency_resolver/dependency_resolver.py +++ b/python/private/pypi/dependency_resolver/dependency_resolver.py @@ -15,14 +15,17 @@ "Set defaults for the pip-compile command to run it under Bazel" import atexit +import functools import os import shutil import sys from pathlib import Path -from typing import Optional, Tuple +from typing import List, Optional, Tuple import click import piptools.writer as piptools_writer +from pip._internal.exceptions import DistributionNotFound +from pip._vendor.resolvelib.resolvers import ResolutionImpossible from piptools.scripts.compile import cli from python.runfiles import runfiles @@ -82,7 +85,7 @@ def _locate(bazel_runfiles, file): @click.command(context_settings={"ignore_unknown_options": True}) @click.option("--src", "srcs", multiple=True, required=True) @click.argument("requirements_txt") -@click.argument("update_target_label") +@click.argument("target_label_prefix") @click.option("--requirements-linux") @click.option("--requirements-darwin") @click.option("--requirements-windows") @@ -90,7 +93,7 @@ def _locate(bazel_runfiles, file): def main( srcs: Tuple[str, ...], requirements_txt: str, - update_target_label: str, + target_label_prefix: str, requirements_linux: Optional[str], requirements_darwin: Optional[str], requirements_windows: Optional[str], @@ -152,9 +155,10 @@ def main( # or shutil.copyfile, as they will fail with OSError: [Errno 18] Invalid cross-device link. shutil.copy(resolved_requirements_file, requirements_out) - update_command = os.getenv("CUSTOM_COMPILE_COMMAND") or "bazel run %s" % ( - update_target_label, + update_command = ( + os.getenv("CUSTOM_COMPILE_COMMAND") or f"bazel run {target_label_prefix}.update" ) + test_command = f"bazel test {target_label_prefix}_test" os.environ["CUSTOM_COMPILE_COMMAND"] = update_command os.environ["PIP_CONFIG_FILE"] = os.getenv("PIP_CONFIG_FILE") or os.devnull @@ -168,6 +172,12 @@ def main( ) argv.extend(extra_args) + _run_pip_compile = functools.partial( + run_pip_compile, + argv, + srcs_relative=srcs_relative, + ) + if UPDATE: print("Updating " + requirements_file_relative) @@ -187,49 +197,66 @@ def main( atexit.register( lambda: shutil.copy(absolute_output_file, requirements_file_tree) ) - cli(argv, standalone_mode=False) + _run_pip_compile(verbose_command=f"{update_command} -- --verbose") requirements_file_relative_path = Path(requirements_file_relative) content = requirements_file_relative_path.read_text() content = content.replace(absolute_path_prefix, "") requirements_file_relative_path.write_text(content) else: - # cli will exit(0) on success - try: - print("Checking " + requirements_file) - cli(argv) - print("cli() should exit", file=sys.stderr) + print("Checking " + requirements_file) + sys.stdout.flush() + _run_pip_compile(verbose_command=f"{test_command} --test_arg=--verbose") + golden = open(_locate(bazel_runfiles, requirements_file)).readlines() + out = open(requirements_out).readlines() + out = [line.replace(absolute_path_prefix, "") for line in out] + if golden != out: + import difflib + + print("".join(difflib.unified_diff(golden, out)), file=sys.stderr) + print( + f"Lock file out of date. Run '{update_command}' to update.", + file=sys.stderr, + ) + sys.exit(1) + + +def run_pip_compile( + args: List[str], + *, + srcs_relative: List[str], + verbose_command: str, +) -> None: + try: + cli(args, standalone_mode=False) + except DistributionNotFound as e: + if isinstance(e.__cause__, ResolutionImpossible): + # pip logs an informative error to stderr already + # just render the error and exit + print(e) + sys.exit(1) + else: + raise + except SystemExit as e: + if e.code == 0: + return # shouldn't happen, but just in case + elif e.code == 2: + print( + "pip-compile exited with code 2. This means that pip-compile found " + "incompatible requirements or could not find a version that matches " + f"the install requirement in one of {srcs_relative}.\n" + "Try re-running with verbose:\n" + f" {verbose_command}", + file=sys.stderr, + ) + sys.exit(1) + else: + print( + f"pip-compile unexpectedly exited with code {e.code}.\n" + "Try re-running with verbose:\n" + f" {verbose_command}", + file=sys.stderr, + ) sys.exit(1) - except SystemExit as e: - if e.code == 2: - print( - "pip-compile exited with code 2. This means that pip-compile found " - "incompatible requirements or could not find a version that matches " - f"the install requirement in one of {srcs_relative}.", - file=sys.stderr, - ) - sys.exit(1) - elif e.code == 0: - golden = open(_locate(bazel_runfiles, requirements_file)).readlines() - out = open(requirements_out).readlines() - out = [line.replace(absolute_path_prefix, "") for line in out] - if golden != out: - import difflib - - print("".join(difflib.unified_diff(golden, out)), file=sys.stderr) - print( - "Lock file out of date. Run '" - + update_command - + "' to update.", - file=sys.stderr, - ) - sys.exit(1) - sys.exit(0) - else: - print( - f"pip-compile unexpectedly exited with code {e.code}.", - file=sys.stderr, - ) - sys.exit(1) if __name__ == "__main__": diff --git a/python/private/pypi/pip_compile.bzl b/python/private/pypi/pip_compile.bzl index 8e46947b99..7edbf7dc2c 100644 --- a/python/private/pypi/pip_compile.bzl +++ b/python/private/pypi/pip_compile.bzl @@ -110,7 +110,7 @@ def pip_compile( args = ["--src=%s" % loc.format(src) for src in srcs] + [ loc.format(requirements_txt), - "//%s:%s.update" % (native.package_name(), name), + "//%s:%s" % (native.package_name(), name), "--resolver=backtracking", "--allow-unsafe", ] From b7e58d1795d9f7858d3e1ba669cd84422fedc6f1 Mon Sep 17 00:00:00 2001 From: Douglas Thor Date: Wed, 23 Apr 2025 13:59:11 -0700 Subject: [PATCH 188/922] feat: Have `pip_compile` generate a `*.test` target; deprecate `*_test` (#2812) Fixes #2794. The `pip_compile` macro generates `*_test` and `*.update` targets. This pattern does not match with other macros that generate similar targets, namely `gazelle_python_manifest` and uv `lock` (though that's `.run` instead of `.test` but either way, it uses a dot `.` instead of underscore `_`). Adjust the macro so that a `.test` target is made. The `_test` target is aliased with a deprecation warning, to be removed in the next major version. --- CHANGELOG.md | 3 +++ python/private/pypi/pip_compile.bzl | 10 ++++++++-- 2 files changed, 11 insertions(+), 2 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index f696cefde2..b1767664ef 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -57,6 +57,9 @@ END_UNRELEASED_TEMPLATE * (rules) On Windows, {obj}`--bootstrap_impl=system_python` is forced. This allows setting `--bootstrap_impl=script` in bazelrc for mixed-platform environments. +* (rules) {obj}`pip_compile` now generates a `.test` target. The `_test` target is deprecated + and will be removed in the next major release. + ([#2794](https://github.com/bazel-contrib/rules_python/issues/2794) {#v0-0-0-fixed} ### Fixed diff --git a/python/private/pypi/pip_compile.bzl b/python/private/pypi/pip_compile.bzl index 7edbf7dc2c..e5b62c4ab0 100644 --- a/python/private/pypi/pip_compile.bzl +++ b/python/private/pypi/pip_compile.bzl @@ -47,7 +47,7 @@ def pip_compile( It also generates two targets for running pip-compile: - - validate with `bazel test [name]_test` + - validate with `bazel test [name].test` - update with `bazel run [name].update` If you are using a version control system, the requirements.txt generated by this rule should @@ -166,7 +166,7 @@ def pip_compile( timeout = kwargs.pop("timeout", "short") py_test( - name = name + "_test", + name = name + ".test", timeout = timeout, # setuptools (the default python build tool) attempts to find user # configuration in the user's home direcotory. This seems to work fine on @@ -180,3 +180,9 @@ def pip_compile( # kwargs could contain test-specific attributes like size **dict(attrs, **kwargs) ) + + native.alias( + name = "{}_test".format(name), + actual = ":{}.test".format(name), + deprecation = "Use '{}.test' instead. The '*_test' target will be removed in the next major release.".format(name), + ) From bb7b164fc1214b319a085222f5ce2a8ef41841c9 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Wed, 23 Apr 2025 16:36:30 -0700 Subject: [PATCH 189/922] fix: try multiple times to get win32 version to handle flakes (#2814) The Google tensorflow/jax devinfra team reported that Windows 2022 with Python 3.12.8 has a tendency to be flaky when calling the platform.win32 APIs. I'm very certain I saw similar behavior in the past myself. To fix, just call the APIs a couple times; it seems to fix itself. cc @vam-google --- CHANGELOG.md | 2 ++ python/private/python_bootstrap_template.txt | 10 +++++++++- python/private/stage2_bootstrap_template.py | 10 +++++++++- 3 files changed, 20 insertions(+), 2 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index b1767664ef..8d11187cdf 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -72,6 +72,8 @@ END_UNRELEASED_TEMPLATE * The `sys._base_executable` value will reflect the underlying interpreter, not venv interpreter. * The {obj}`//python/runtime_env_toolchains:all` toolchain now works with it. +* (rules) Better handle flakey platform.win32_ver() calls by calling them + multiple times. {#v0-0-0-added} ### Added diff --git a/python/private/python_bootstrap_template.txt b/python/private/python_bootstrap_template.txt index eb5595f4a1..210987abf9 100644 --- a/python/private/python_bootstrap_template.txt +++ b/python/private/python_bootstrap_template.txt @@ -46,7 +46,15 @@ def GetWindowsPathWithUNCPrefix(path): # removed from common Win32 file and directory functions. # Related doc: https://docs.microsoft.com/en-us/windows/win32/fileio/maximum-file-path-limitation?tabs=cmd#enable-long-paths-in-windows-10-version-1607-and-later import platform - if platform.win32_ver()[1] >= '10.0.14393': + win32_version = None + # Windows 2022 with Python 3.12.8 gives flakey errors, so try a couple times. + for _ in range(3): + try: + win32_version = platform.win32_ver()[1] + break + except (ValueError, KeyError): + pass + if win32_version and win32_version >= '10.0.14393': return path # import sysconfig only now to maintain python 2.6 compatibility diff --git a/python/private/stage2_bootstrap_template.py b/python/private/stage2_bootstrap_template.py index fcc323e8ca..689602d3aa 100644 --- a/python/private/stage2_bootstrap_template.py +++ b/python/private/stage2_bootstrap_template.py @@ -58,7 +58,15 @@ def get_windows_path_with_unc_prefix(path): # Related doc: https://docs.microsoft.com/en-us/windows/win32/fileio/maximum-file-path-limitation?tabs=cmd#enable-long-paths-in-windows-10-version-1607-and-later import platform - if platform.win32_ver()[1] >= "10.0.14393": + win32_version = None + # Windows 2022 with Python 3.12.8 gives flakey errors, so try a couple times. + for _ in range(3): + try: + win32_version = platform.win32_ver()[1] + break + except (ValueError, KeyError): + pass + if win32_version and win32_version >= '10.0.14393': return path # import sysconfig only now to maintain python 2.6 compatibility From 7164477cc97ea98a72ca3dc769ac63bc2c061de6 Mon Sep 17 00:00:00 2001 From: Douglas Thor Date: Wed, 23 Apr 2025 23:24:56 -0700 Subject: [PATCH 190/922] refactor: Add log_std(out|err) bools to repo_utils that execute a subprocess (#2817) While making a local patch to work around #2640, I found that I had a need for running a subprocess (`gcloud auth print-access-token`) via `repo_utils.execute_checked_stdout`. However, doing so would log that access token when debug logging was enabled via `RULES_PYTHON_REPO_DEBUG=1`. This is a security concern for us, so I hacked in an option to allow a particular `execute_(un)checked(_stdout)` call to disable logging stdout, stderr, or both. I figure this might be useful to others so I thought I'd upstream it. `execute_(un)checked(_stdout)` now support `log_stdout` and `log_stderr` bools that default to `True` (which is the same behavior as before this PR. When the subprocess writes to stdout and `log_stdout = False`, the logged message will show: ``` ===== stdout start ===== ===== stdout end ===== ``` If the subprocess does not write to stdout, the debug log shows the same as before: ``` ``` The above also applies for stderr, with text adjusted accordingly. --- CHANGELOG.md | 4 +++- python/private/repo_utils.bzl | 31 ++++++++++++++++++++++++------- 2 files changed, 27 insertions(+), 8 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 8d11187cdf..88defb8e84 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -77,7 +77,9 @@ END_UNRELEASED_TEMPLATE {#v0-0-0-added} ### Added -* Nothing added. +* Repo utilities `execute_unchecked`, `execute_checked`, and `execute_checked_stdout` now + support `log_stdout` and `log_stderr` keyword arg booleans. When these are `True` + (the default), the subprocess's stdout/stderr will be logged. {#v0-0-0-removed} ### Removed diff --git a/python/private/repo_utils.bzl b/python/private/repo_utils.bzl index 73883a9244..eee56ec86c 100644 --- a/python/private/repo_utils.bzl +++ b/python/private/repo_utils.bzl @@ -98,6 +98,8 @@ def _execute_internal( arguments, environment = {}, logger = None, + log_stdout = True, + log_stderr = True, **kwargs): """Execute a subprocess with debugging instrumentation. @@ -116,6 +118,10 @@ def _execute_internal( logger: optional `Logger` to use for logging execution details. Must be specified when using module_ctx. If not specified, a default will be created. + log_stdout: If True (the default), write stdout to the logged message. Setting + to False can be useful for large stdout messages or for secrets. + log_stderr: If True (the default), write stderr to the logged message. Setting + to False can be useful for large stderr messages or for secrets. **kwargs: additional kwargs to pass onto rctx.execute Returns: @@ -160,7 +166,7 @@ def _execute_internal( cwd = _cwd_to_str(mrctx, kwargs), timeout = _timeout_to_str(kwargs), env_str = _env_to_str(environment), - output = _outputs_to_str(result), + output = _outputs_to_str(result, log_stdout = log_stdout, log_stderr = log_stderr), )) elif _is_repo_debug_enabled(mrctx): logger.debug(( @@ -171,7 +177,7 @@ def _execute_internal( op = op, status = "success" if result.return_code == 0 else "failure", return_code = result.return_code, - output = _outputs_to_str(result), + output = _outputs_to_str(result, log_stdout = log_stdout, log_stderr = log_stderr), )) result_kwargs = {k: getattr(result, k) for k in dir(result)} @@ -183,6 +189,8 @@ def _execute_internal( mrctx = mrctx, kwargs = kwargs, environment = environment, + log_stdout = log_stdout, + log_stderr = log_stderr, ), **result_kwargs ) @@ -220,7 +228,16 @@ def _execute_checked_stdout(*args, **kwargs): """Calls execute_checked, but only returns the stdout value.""" return _execute_checked(*args, **kwargs).stdout -def _execute_describe_failure(*, op, arguments, result, mrctx, kwargs, environment): +def _execute_describe_failure( + *, + op, + arguments, + result, + mrctx, + kwargs, + environment, + log_stdout = True, + log_stderr = True): return ( "repo.execute: {op}: failure:\n" + " command: {cmd}\n" + @@ -236,7 +253,7 @@ def _execute_describe_failure(*, op, arguments, result, mrctx, kwargs, environme cwd = _cwd_to_str(mrctx, kwargs), timeout = _timeout_to_str(kwargs), env_str = _env_to_str(environment), - output = _outputs_to_str(result), + output = _outputs_to_str(result, log_stdout = log_stdout, log_stderr = log_stderr), ) def _which_checked(mrctx, binary_name): @@ -331,11 +348,11 @@ def _env_to_str(environment): def _timeout_to_str(kwargs): return kwargs.get("timeout", "") -def _outputs_to_str(result): +def _outputs_to_str(result, log_stdout = True, log_stderr = True): lines = [] items = [ - ("stdout", result.stdout), - ("stderr", result.stderr), + ("stdout", result.stdout if log_stdout else ""), + ("stderr", result.stderr if log_stderr else ""), ] for name, content in items: if content: From 1e21dbdbba45a3fa7a3bcb2495d72f89eae1fb98 Mon Sep 17 00:00:00 2001 From: Ignas Anikevicius <240938+aignas@users.noreply.github.com> Date: Thu, 24 Apr 2025 22:05:45 +0900 Subject: [PATCH 191/922] fix: use the python micro version to parse whl metadata in bzlmod (#2793) Add `` version to the target platform. Instead of `cpxy_os_cpu` the target platform string format becomes `cpxy.z_os_cpu`. This is a temporary measure until we get a better API for defining target platforms. Summary: - [x] test `select_whls` function needs to be tested to ensure that the whl selection is not impacted when we have the full version in the target platform. - [ ] `download_only` legacy whl code path in `bzlmod` needs further testing. - [x] test `whl_config_setting` handling and config setting creation. The config settings in the hub repo should not use the full version, because from the outside, the whl is compatible with all `micro` versions of a given `3.` of the Python interpreter. This means that the already documented config setting do not need to be changed. - [x] `pep508_deps` tests for handling the `full_python_version` correctly. - [x] `pep508_deps` tests for ensuring the `default_abi` is being handled correctly. Fixes #2319 --- .bazelrc | 4 +- CHANGELOG.md | 3 ++ examples/bzlmod/entry_points/BUILD.bazel | 8 +-- python/private/pypi/BUILD.bazel | 3 ++ python/private/pypi/config_settings.bzl | 2 + python/private/pypi/extension.bzl | 14 ++++-- python/private/pypi/pep508_deps.bzl | 27 ++++++++-- python/private/pypi/pkg_aliases.bzl | 3 ++ python/private/pypi/render_pkg_aliases.bzl | 14 +++++- .../pypi/requirements_files_by_platform.bzl | 7 ++- python/private/pypi/whl_config_setting.bzl | 12 ++++- python/private/pypi/whl_library_targets.bzl | 16 +++--- python/private/pypi/whl_target_platforms.bzl | 5 +- tests/pypi/extension/extension_tests.bzl | 12 +++-- tests/pypi/pep508/deps_tests.bzl | 49 +++++++++++++------ .../render_pkg_aliases_test.bzl | 9 ++-- .../whl_library_targets_tests.bzl | 30 +++++------- .../whl_target_platforms/select_whl_tests.bzl | 16 ++++++ 18 files changed, 160 insertions(+), 74 deletions(-) diff --git a/.bazelrc b/.bazelrc index d2e0721526..4e6f2fa187 100644 --- a/.bazelrc +++ b/.bazelrc @@ -4,8 +4,8 @@ # (Note, we cannot use `common --deleted_packages` because the bazel version command doesn't support it) # To update these lines, execute # `bazel run @rules_bazel_integration_test//tools:update_deleted_packages` -build --deleted_packages=examples/build_file_generation,examples/build_file_generation/random_number_generator,examples/bzlmod,examples/bzlmod/entry_points,examples/bzlmod/entry_points/tests,examples/bzlmod/libs/my_lib,examples/bzlmod/other_module,examples/bzlmod/other_module/other_module/pkg,examples/bzlmod/patches,examples/bzlmod/py_proto_library,examples/bzlmod/py_proto_library/example.com/another_proto,examples/bzlmod/py_proto_library/example.com/proto,examples/bzlmod/runfiles,examples/bzlmod/tests,examples/bzlmod/tests/other_module,examples/bzlmod/whl_mods,examples/bzlmod_build_file_generation,examples/bzlmod_build_file_generation/other_module/other_module/pkg,examples/bzlmod_build_file_generation/runfiles,examples/multi_python_versions/libs/my_lib,examples/multi_python_versions/requirements,examples/multi_python_versions/tests,examples/pip_parse,examples/pip_parse_vendored,examples/pip_repository_annotations,examples/py_proto_library,examples/py_proto_library/example.com/another_proto,examples/py_proto_library/example.com/proto,gazelle,gazelle/manifest,gazelle/manifest/generate,gazelle/manifest/hasher,gazelle/manifest/test,gazelle/modules_mapping,gazelle/python,gazelle/python/private,gazelle/pythonconfig,tests/integration/compile_pip_requirements,tests/integration/compile_pip_requirements_test_from_external_repo,tests/integration/custom_commands,tests/integration/ignore_root_user_error,tests/integration/ignore_root_user_error/submodule,tests/integration/local_toolchains,tests/integration/pip_parse,tests/integration/pip_parse/empty,tests/integration/py_cc_toolchain_registered,tests/modules/other,tests/modules/other/nspkg_delta,tests/modules/other/nspkg_gamma -query --deleted_packages=examples/build_file_generation,examples/build_file_generation/random_number_generator,examples/bzlmod,examples/bzlmod/entry_points,examples/bzlmod/entry_points/tests,examples/bzlmod/libs/my_lib,examples/bzlmod/other_module,examples/bzlmod/other_module/other_module/pkg,examples/bzlmod/patches,examples/bzlmod/py_proto_library,examples/bzlmod/py_proto_library/example.com/another_proto,examples/bzlmod/py_proto_library/example.com/proto,examples/bzlmod/runfiles,examples/bzlmod/tests,examples/bzlmod/tests/other_module,examples/bzlmod/whl_mods,examples/bzlmod_build_file_generation,examples/bzlmod_build_file_generation/other_module/other_module/pkg,examples/bzlmod_build_file_generation/runfiles,examples/multi_python_versions/libs/my_lib,examples/multi_python_versions/requirements,examples/multi_python_versions/tests,examples/pip_parse,examples/pip_parse_vendored,examples/pip_repository_annotations,examples/py_proto_library,examples/py_proto_library/example.com/another_proto,examples/py_proto_library/example.com/proto,gazelle,gazelle/manifest,gazelle/manifest/generate,gazelle/manifest/hasher,gazelle/manifest/test,gazelle/modules_mapping,gazelle/python,gazelle/python/private,gazelle/pythonconfig,tests/integration/compile_pip_requirements,tests/integration/compile_pip_requirements_test_from_external_repo,tests/integration/custom_commands,tests/integration/ignore_root_user_error,tests/integration/ignore_root_user_error/submodule,tests/integration/local_toolchains,tests/integration/pip_parse,tests/integration/pip_parse/empty,tests/integration/py_cc_toolchain_registered,tests/modules/other,tests/modules/other/nspkg_delta,tests/modules/other/nspkg_gamma +build --deleted_packages=examples/build_file_generation,examples/build_file_generation/random_number_generator,examples/bzlmod,examples/bzlmod_build_file_generation,examples/bzlmod_build_file_generation/other_module/other_module/pkg,examples/bzlmod_build_file_generation/runfiles,examples/bzlmod/entry_points,examples/bzlmod/entry_points/tests,examples/bzlmod/libs/my_lib,examples/bzlmod/other_module,examples/bzlmod/other_module/other_module/pkg,examples/bzlmod/patches,examples/bzlmod/py_proto_library,examples/bzlmod/py_proto_library/example.com/another_proto,examples/bzlmod/py_proto_library/example.com/proto,examples/bzlmod/runfiles,examples/bzlmod/tests,examples/bzlmod/tests/other_module,examples/bzlmod/whl_mods,examples/multi_python_versions/libs/my_lib,examples/multi_python_versions/requirements,examples/multi_python_versions/tests,examples/pip_parse,examples/pip_parse_vendored,examples/pip_repository_annotations,examples/py_proto_library,examples/py_proto_library/example.com/another_proto,examples/py_proto_library/example.com/proto,gazelle,gazelle/manifest,gazelle/manifest/generate,gazelle/manifest/hasher,gazelle/manifest/test,gazelle/modules_mapping,gazelle/python,gazelle/pythonconfig,gazelle/python/private,tests/integration/compile_pip_requirements,tests/integration/compile_pip_requirements_test_from_external_repo,tests/integration/custom_commands,tests/integration/ignore_root_user_error,tests/integration/ignore_root_user_error/submodule,tests/integration/local_toolchains,tests/integration/pip_parse,tests/integration/pip_parse/empty,tests/integration/py_cc_toolchain_registered,tests/modules/other,tests/modules/other/nspkg_delta,tests/modules/other/nspkg_gamma +query --deleted_packages=examples/build_file_generation,examples/build_file_generation/random_number_generator,examples/bzlmod,examples/bzlmod_build_file_generation,examples/bzlmod_build_file_generation/other_module/other_module/pkg,examples/bzlmod_build_file_generation/runfiles,examples/bzlmod/entry_points,examples/bzlmod/entry_points/tests,examples/bzlmod/libs/my_lib,examples/bzlmod/other_module,examples/bzlmod/other_module/other_module/pkg,examples/bzlmod/patches,examples/bzlmod/py_proto_library,examples/bzlmod/py_proto_library/example.com/another_proto,examples/bzlmod/py_proto_library/example.com/proto,examples/bzlmod/runfiles,examples/bzlmod/tests,examples/bzlmod/tests/other_module,examples/bzlmod/whl_mods,examples/multi_python_versions/libs/my_lib,examples/multi_python_versions/requirements,examples/multi_python_versions/tests,examples/pip_parse,examples/pip_parse_vendored,examples/pip_repository_annotations,examples/py_proto_library,examples/py_proto_library/example.com/another_proto,examples/py_proto_library/example.com/proto,gazelle,gazelle/manifest,gazelle/manifest/generate,gazelle/manifest/hasher,gazelle/manifest/test,gazelle/modules_mapping,gazelle/python,gazelle/pythonconfig,gazelle/python/private,tests/integration/compile_pip_requirements,tests/integration/compile_pip_requirements_test_from_external_repo,tests/integration/custom_commands,tests/integration/ignore_root_user_error,tests/integration/ignore_root_user_error/submodule,tests/integration/local_toolchains,tests/integration/pip_parse,tests/integration/pip_parse/empty,tests/integration/py_cc_toolchain_registered,tests/modules/other,tests/modules/other/nspkg_delta,tests/modules/other/nspkg_gamma test --test_output=errors diff --git a/CHANGELOG.md b/CHANGELOG.md index 88defb8e84..984af8bad2 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -148,6 +148,9 @@ END_UNRELEASED_TEMPLATE * (packaging) An empty `requires_file` is treated as if it were omitted, resulting in a valid `METADATA` file. * (rules) py_wheel and sphinxdocs rules now propagate `target_compatible_with` to all targets they create. [PR #2788](https://github.com/bazel-contrib/rules_python/pull/2788). +* (pypi) Correctly handle `METADATA` entries when `python_full_version` is used in + the environment marker. + Fixes [#2319](https://github.com/bazel-contrib/rules_python/issues/2319). {#1-4-0-added} ### Added diff --git a/examples/bzlmod/entry_points/BUILD.bazel b/examples/bzlmod/entry_points/BUILD.bazel index a0939cb65b..4ca5b53568 100644 --- a/examples/bzlmod/entry_points/BUILD.bazel +++ b/examples/bzlmod/entry_points/BUILD.bazel @@ -1,4 +1,3 @@ -load("@python_versions//3.9:defs.bzl", py_console_script_binary_3_9 = "py_console_script_binary") load("@rules_python//python/entry_points:py_console_script_binary.bzl", "py_console_script_binary") # This is how you can define a `pylint` entrypoint which uses the default python version. @@ -24,10 +23,11 @@ py_console_script_binary( ], ) -# A specific Python version can be forced by using the generated version-aware -# wrappers, e.g. to force Python 3.9: -py_console_script_binary_3_9( +# A specific Python version can be forced by passing `python_version` +# attribute, e.g. to force Python 3.9: +py_console_script_binary( name = "yamllint", pkg = "@pip//yamllint:pkg", + python_version = "3.9", visibility = ["//entry_points:__subpackages__"], ) diff --git a/python/private/pypi/BUILD.bazel b/python/private/pypi/BUILD.bazel index a758b3f153..bfb0be2d59 100644 --- a/python/private/pypi/BUILD.bazel +++ b/python/private/pypi/BUILD.bazel @@ -103,6 +103,7 @@ bzl_library( "//python/private:version_label_bzl", "@bazel_features//:features", "@pythons_hub//:interpreters_bzl", + "@pythons_hub//:versions_bzl", ], ) @@ -220,7 +221,9 @@ bzl_library( ":pep508_evaluate_bzl", ":pep508_platform_bzl", ":pep508_requirement_bzl", + "//python/private:full_version_bzl", "//python/private:normalize_name_bzl", + "@pythons_hub//:versions_bzl", ], ) diff --git a/python/private/pypi/config_settings.bzl b/python/private/pypi/config_settings.bzl index 1045ffef35..d1b85d16c1 100644 --- a/python/private/pypi/config_settings.bzl +++ b/python/private/pypi/config_settings.bzl @@ -42,6 +42,8 @@ specialized is as follows: * `:is_cp3_abi3_` * `:is_cp3_cp3_` and `:is_cp3_cp3t_` +Optionally instead of `` there sometimes may be `.` used in order to fully specify the versions + The specialization of free-threaded vs non-free-threaded wheels is the same as they are just variants of each other. The same goes for the specialization of `musllinux` vs `manylinux`. diff --git a/python/private/pypi/extension.bzl b/python/private/pypi/extension.bzl index d1895ca211..e9eba684f8 100644 --- a/python/private/pypi/extension.bzl +++ b/python/private/pypi/extension.bzl @@ -16,7 +16,9 @@ load("@bazel_features//:features.bzl", "bazel_features") load("@pythons_hub//:interpreters.bzl", "INTERPRETER_LABELS") +load("@pythons_hub//:versions.bzl", "MINOR_MAPPING") load("//python/private:auth.bzl", "AUTH_ATTRS") +load("//python/private:full_version.bzl", "full_version") load("//python/private:normalize_name.bzl", "normalize_name") load("//python/private:repo_utils.bzl", "repo_utils") load("//python/private:semver.bzl", "semver") @@ -68,6 +70,7 @@ def _create_whl_repos( pip_attr, whl_overrides, available_interpreters = INTERPRETER_LABELS, + minor_mapping = MINOR_MAPPING, get_index_urls = None): """create all of the whl repositories @@ -80,6 +83,8 @@ def _create_whl_repos( interpreters that have been registered using the `python` bzlmod extension. The keys are in the form `python_{snake_case_version}_host`. This is to be used during the `repository_rule` and must be always compatible with the host. + minor_mapping: {type}`dict[str, str]` The dictionary needed to resolve the full + python version used to parse package METADATA files. Returns a {type}`struct` with the following attributes: whl_map: {type}`dict[str, list[struct]]` the output is keyed by the @@ -159,8 +164,10 @@ def _create_whl_repos( requirements_osx = pip_attr.requirements_darwin, requirements_windows = pip_attr.requirements_windows, extra_pip_args = pip_attr.extra_pip_args, - # TODO @aignas 2025-04-15: pass the full version into here - python_version = major_minor, + python_version = full_version( + version = pip_attr.python_version, + minor_mapping = minor_mapping, + ), logger = logger, ), extra_pip_args = pip_attr.extra_pip_args, @@ -304,9 +311,6 @@ def _whl_repos(*, requirement, whl_library_args, download_only, netrc, auth_patt if requirement.extra_pip_args: args["extra_pip_args"] = requirement.extra_pip_args - if download_only: - args.setdefault("experimental_target_platforms", requirement.target_platforms) - target_platforms = requirement.target_platforms if multiple_requirements_for_whl else [] repo_name = pypi_repo_name( normalize_name(requirement.distribution), diff --git a/python/private/pypi/pep508_deps.bzl b/python/private/pypi/pep508_deps.bzl index 115bbd78d8..bcc4845cf1 100644 --- a/python/private/pypi/pep508_deps.bzl +++ b/python/private/pypi/pep508_deps.bzl @@ -15,14 +15,23 @@ """This module is for implementing PEP508 compliant METADATA deps parsing. """ -load("@pythons_hub//:versions.bzl", "DEFAULT_PYTHON_VERSION") +load("@pythons_hub//:versions.bzl", "DEFAULT_PYTHON_VERSION", "MINOR_MAPPING") +load("//python/private:full_version.bzl", "full_version") load("//python/private:normalize_name.bzl", "normalize_name") load(":pep508_env.bzl", "env") load(":pep508_evaluate.bzl", "evaluate") load(":pep508_platform.bzl", "platform", "platform_from_str") load(":pep508_requirement.bzl", "requirement") -def deps(name, *, requires_dist, platforms = [], extras = [], excludes = [], default_python_version = None): +def deps( + name, + *, + requires_dist, + platforms = [], + extras = [], + excludes = [], + default_python_version = None, + minor_mapping = MINOR_MAPPING): """Parse the RequiresDist from wheel METADATA Args: @@ -33,6 +42,9 @@ def deps(name, *, requires_dist, platforms = [], extras = [], excludes = [], def extras: {type}`list[str]` the requested extras to generate targets for. platforms: {type}`list[str]` the list of target platform strings. default_python_version: {type}`str` the host python version. + minor_mapping: {type}`type[str, str]` the minor mapping to use when + resolving to the full python version as DEFAULT_PYTHON_VERSION can by + of format `3.x`. Returns: A struct with attributes: @@ -53,6 +65,12 @@ def deps(name, *, requires_dist, platforms = [], extras = [], excludes = [], def excludes = [name] + [normalize_name(x) for x in excludes] default_python_version = default_python_version or DEFAULT_PYTHON_VERSION + if default_python_version: + # if it is not bzlmod, then DEFAULT_PYTHON_VERSION may be unset + default_python_version = full_version( + version = default_python_version, + minor_mapping = minor_mapping, + ) platforms = [ platform_from_str(p, python_version = default_python_version) for p in platforms @@ -60,9 +78,8 @@ def deps(name, *, requires_dist, platforms = [], extras = [], excludes = [], def abis = sorted({p.abi: True for p in platforms if p.abi}) if default_python_version and len(abis) > 1: - _, _, minor_version = default_python_version.partition(".") - minor_version, _, _ = minor_version.partition(".") - default_abi = "cp3" + minor_version + _, _, tail = default_python_version.partition(".") + default_abi = "cp3" + tail elif len(abis) > 1: fail( "all python versions need to be specified explicitly, got: {}".format(platforms), diff --git a/python/private/pypi/pkg_aliases.bzl b/python/private/pypi/pkg_aliases.bzl index a9eee7be88..28d70ff715 100644 --- a/python/private/pypi/pkg_aliases.bzl +++ b/python/private/pypi/pkg_aliases.bzl @@ -371,6 +371,9 @@ def get_filename_config_settings( abi = parsed.abi_tag + # TODO @aignas 2025-04-20: test + abi, _, _ = abi.partition(".") + if parsed.platform_tag == "any": prefixes = ["{}{}_any".format(py, abi)] else: diff --git a/python/private/pypi/render_pkg_aliases.bzl b/python/private/pypi/render_pkg_aliases.bzl index 863d25095c..28f32edc78 100644 --- a/python/private/pypi/render_pkg_aliases.bzl +++ b/python/private/pypi/render_pkg_aliases.bzl @@ -143,6 +143,18 @@ def render_pkg_aliases(*, aliases, requirement_cycles = None, extra_hub_aliases files["_groups/BUILD.bazel"] = generate_group_library_build_bazel("", requirement_cycles) return files +def _major_minor(python_version): + major, _, tail = python_version.partition(".") + minor, _, _ = tail.partition(".") + return "{}.{}".format(major, minor) + +def _major_minor_versions(python_versions): + if not python_versions: + return [] + + # Use a dict as a simple set + return sorted({_major_minor(v): None for v in python_versions}) + def render_multiplatform_pkg_aliases(*, aliases, **kwargs): """Render the multi-platform pkg aliases. @@ -174,7 +186,7 @@ def render_multiplatform_pkg_aliases(*, aliases, **kwargs): glibc_versions = flag_versions.get("glibc_versions", []), muslc_versions = flag_versions.get("muslc_versions", []), osx_versions = flag_versions.get("osx_versions", []), - python_versions = flag_versions.get("python_versions", []), + python_versions = _major_minor_versions(flag_versions.get("python_versions", [])), target_platforms = flag_versions.get("target_platforms", []), visibility = ["//:__subpackages__"], ) diff --git a/python/private/pypi/requirements_files_by_platform.bzl b/python/private/pypi/requirements_files_by_platform.bzl index e3aafc083f..9165c05bed 100644 --- a/python/private/pypi/requirements_files_by_platform.bzl +++ b/python/private/pypi/requirements_files_by_platform.bzl @@ -91,13 +91,12 @@ def _platforms_from_args(extra_pip_args): return list(platforms.keys()) def _platform(platform_string, python_version = None): - if not python_version or platform_string.startswith("cp3"): + if not python_version or platform_string.startswith("cp"): return platform_string - _, _, tail = python_version.partition(".") - minor, _, _ = tail.partition(".") + major, _, tail = python_version.partition(".") - return "cp3{}_{}".format(minor, platform_string) + return "cp{}{}_{}".format(major, tail, platform_string) def requirements_files_by_platform( *, diff --git a/python/private/pypi/whl_config_setting.bzl b/python/private/pypi/whl_config_setting.bzl index d966206372..6e10eb4d27 100644 --- a/python/private/pypi/whl_config_setting.bzl +++ b/python/private/pypi/whl_config_setting.bzl @@ -35,10 +35,20 @@ def whl_config_setting(*, version = None, config_setting = None, filename = None a struct with the validated and parsed values. """ if target_platforms: - for p in target_platforms: + target_platforms_input = target_platforms + target_platforms = [] + for p in target_platforms_input: if not p.startswith("cp"): fail("target_platform should start with 'cp' denoting the python version, got: " + p) + abi, _, tail = p.partition("_") + + # drop the micro version here, currently there is no usecase to use + # multiple python interpreters with the same minor version but + # different micro version. + abi, _, _ = abi.partition(".") + target_platforms.append("{}_{}".format(abi, tail)) + return struct( config_setting = config_setting, filename = filename, diff --git a/python/private/pypi/whl_library_targets.bzl b/python/private/pypi/whl_library_targets.bzl index cf3df133c4..21e4a54a3a 100644 --- a/python/private/pypi/whl_library_targets.bzl +++ b/python/private/pypi/whl_library_targets.bzl @@ -369,26 +369,22 @@ def _config_settings(dependencies_by_platform, native = native, **kwargs): if p.startswith("@") or p.endswith("default"): continue + # TODO @aignas 2025-04-20: add tests here abi, _, tail = p.partition("_") if not abi.startswith("cp"): tail = p abi = "" - os, _, arch = tail.partition("_") - os = "" if os == "anyos" else os - arch = "" if arch == "anyarch" else arch _kwargs = dict(kwargs) - if arch: - _kwargs.setdefault("constraint_values", []).append("@platforms//cpu:{}".format(arch)) - if os: - _kwargs.setdefault("constraint_values", []).append("@platforms//os:{}".format(os)) + _kwargs["constraint_values"] = [ + "@platforms//cpu:{}".format(arch), + "@platforms//os:{}".format(os), + ] if abi: _kwargs["flag_values"] = { - "@rules_python//python/config_settings:python_version_major_minor": "3.{minor_version}".format( - minor_version = abi[len("cp3"):], - ), + Label("//python/config_settings:python_version"): "3.{}".format(abi[len("cp3"):]), } native.config_setting( diff --git a/python/private/pypi/whl_target_platforms.bzl b/python/private/pypi/whl_target_platforms.bzl index 9f47e625b3..6ea3f120c3 100644 --- a/python/private/pypi/whl_target_platforms.bzl +++ b/python/private/pypi/whl_target_platforms.bzl @@ -75,8 +75,11 @@ def select_whls(*, whls, want_platforms = [], logger = None): fail("expected all platforms to start with ABI, but got: {}".format(p)) abi, _, os_cpu = p.partition("_") + abi, _, _ = abi.partition(".") _want_platforms[os_cpu] = None - _want_platforms[p] = None + + # TODO @aignas 2025-04-20: add a test + _want_platforms["{}_{}".format(abi, os_cpu)] = None version_limit_candidate = int(abi[3:]) if not version_limit: diff --git a/tests/pypi/extension/extension_tests.bzl b/tests/pypi/extension/extension_tests.bzl index ce5474e35b..5de3bb58d3 100644 --- a/tests/pypi/extension/extension_tests.bzl +++ b/tests/pypi/extension/extension_tests.bzl @@ -157,6 +157,7 @@ def _test_simple(env): available_interpreters = { "python_3_15_host": "unit_test_interpreter_target", }, + minor_mapping = {"3.15": "3.15.19"}, ) pypi.exposed_packages().contains_exactly({"pypi": ["simple"]}) @@ -204,6 +205,7 @@ def _test_simple_multiple_requirements(env): available_interpreters = { "python_3_15_host": "unit_test_interpreter_target", }, + minor_mapping = {"3.15": "3.15.19"}, ) pypi.exposed_packages().contains_exactly({"pypi": ["simple"]}) @@ -270,6 +272,7 @@ torch==2.4.1 ; platform_machine != 'x86_64' \ available_interpreters = { "python_3_15_host": "unit_test_interpreter_target", }, + minor_mapping = {"3.15": "3.15.19"}, ) pypi.exposed_packages().contains_exactly({"pypi": ["torch"]}) @@ -392,6 +395,7 @@ torch==2.4.1+cpu ; platform_machine == 'x86_64' \ available_interpreters = { "python_3_12_host": "unit_test_interpreter_target", }, + minor_mapping = {"3.12": "3.12.19"}, simpleapi_download = mocksimpleapi_download, ) @@ -515,6 +519,7 @@ simple==0.0.3 \ available_interpreters = { "python_3_15_host": "unit_test_interpreter_target", }, + minor_mapping = {"3.15": "3.15.19"}, ) pypi.exposed_packages().contains_exactly({"pypi": ["simple"]}) @@ -544,7 +549,8 @@ simple==0.0.3 \ "pypi_315_extra": { "dep_template": "@pypi//{name}:{target}", "download_only": True, - "experimental_target_platforms": ["cp315_linux_x86_64"], + # TODO @aignas 2025-04-20: ensure that this is in the hub repo + # "experimental_target_platforms": ["cp315_linux_x86_64"], "extra_pip_args": ["--platform=manylinux_2_17_x86_64", "--python-version=315", "--implementation=cp", "--abi=cp315"], "python_interpreter_target": "unit_test_interpreter_target", "requirement": "extra==0.0.1 --hash=sha256:deadb00f", @@ -552,7 +558,6 @@ simple==0.0.3 \ "pypi_315_simple_linux_x86_64": { "dep_template": "@pypi//{name}:{target}", "download_only": True, - "experimental_target_platforms": ["cp315_linux_x86_64"], "extra_pip_args": ["--platform=manylinux_2_17_x86_64", "--python-version=315", "--implementation=cp", "--abi=cp315"], "python_interpreter_target": "unit_test_interpreter_target", "requirement": "simple==0.0.1 --hash=sha256:deadbeef", @@ -560,7 +565,6 @@ simple==0.0.3 \ "pypi_315_simple_osx_aarch64": { "dep_template": "@pypi//{name}:{target}", "download_only": True, - "experimental_target_platforms": ["cp315_osx_aarch64"], "extra_pip_args": ["--platform=macosx_10_9_arm64", "--python-version=315", "--implementation=cp", "--abi=cp315"], "python_interpreter_target": "unit_test_interpreter_target", "requirement": "simple==0.0.3 --hash=sha256:deadbaaf", @@ -648,6 +652,7 @@ git_dep @ git+https://git.server/repo/project@deadbeefdeadbeef available_interpreters = { "python_3_15_host": "unit_test_interpreter_target", }, + minor_mapping = {"3.15": "3.15.19"}, simpleapi_download = mocksimpleapi_download, ) @@ -850,6 +855,7 @@ optimum[onnxruntime-gpu]==1.17.1 ; sys_platform == 'linux' available_interpreters = { "python_3_15_host": "unit_test_interpreter_target", }, + minor_mapping = {"3.15": "3.15.19"}, ) pypi.exposed_packages().contains_exactly({"pypi": []}) diff --git a/tests/pypi/pep508/deps_tests.bzl b/tests/pypi/pep508/deps_tests.bzl index d362925080..118cd50092 100644 --- a/tests/pypi/pep508/deps_tests.bzl +++ b/tests/pypi/pep508/deps_tests.bzl @@ -48,6 +48,15 @@ def test_can_add_os_specific_deps(env): ], python_version = "", ), + struct( + platforms = [ + "cp33.1_linux_x86_64", + "cp33.1_osx_x86_64", + "cp33.1_osx_aarch64", + "cp33.1_windows_x86_64", + ], + python_version = "", + ), ]: got = deps( "foo", @@ -154,7 +163,7 @@ _tests.append(test_self_dependencies_can_come_in_any_order) def _test_can_get_deps_based_on_specific_python_version(env): requires_dist = [ "bar", - "baz; python_version < '3.8'", + "baz; python_full_version < '3.7.3'", "posix_dep; os_name=='posix' and python_version >= '3.8'", ] @@ -163,6 +172,11 @@ def _test_can_get_deps_based_on_specific_python_version(env): requires_dist = requires_dist, platforms = ["cp38_linux_x86_64"], ) + py373 = deps( + "foo", + requires_dist = requires_dist, + platforms = ["cp37.3_linux_x86_64"], + ) py37 = deps( "foo", requires_dist = requires_dist, @@ -174,6 +188,8 @@ def _test_can_get_deps_based_on_specific_python_version(env): env.expect.that_dict(py37.deps_select).contains_exactly({}) env.expect.that_collection(py38.deps).contains_exactly(["bar", "posix_dep"]) env.expect.that_dict(py38.deps_select).contains_exactly({}) + env.expect.that_collection(py373.deps).contains_exactly(["bar"]) + env.expect.that_dict(py373.deps_select).contains_exactly({}) _tests.append(_test_can_get_deps_based_on_specific_python_version) @@ -210,27 +226,29 @@ def _test_can_get_version_select(env): "posix_dep_with_version; os_name=='posix' and python_version >= '3.8'", "arch_dep; platform_machine=='x86_64' and python_version < '3.8'", ] - default_python_version = "3.7.4" got = deps( "foo", requires_dist = requires_dist, platforms = [ "cp3{}_{}_x86_64".format(minor, os) - for minor in [7, 8, 9] + for minor in ["7.4", "8.8", "9.8"] for os in ["linux", "windows"] ], - default_python_version = default_python_version, + default_python_version = "3.7", + minor_mapping = { + "3.7": "3.7.4", + }, ) env.expect.that_collection(got.deps).contains_exactly(["bar"]) env.expect.that_dict(got.deps_select).contains_exactly({ - "cp37_linux_x86_64": ["arch_dep", "baz", "posix_dep"], - "cp37_windows_x86_64": ["arch_dep", "baz"], - "cp38_linux_x86_64": ["baz_new", "posix_dep", "posix_dep_with_version"], - "cp38_windows_x86_64": ["baz_new"], - "cp39_linux_x86_64": ["baz_new", "posix_dep", "posix_dep_with_version"], - "cp39_windows_x86_64": ["baz_new"], + "cp37.4_linux_x86_64": ["arch_dep", "baz", "posix_dep"], + "cp37.4_windows_x86_64": ["arch_dep", "baz"], + "cp38.8_linux_x86_64": ["baz_new", "posix_dep", "posix_dep_with_version"], + "cp38.8_windows_x86_64": ["baz_new"], + "cp39.8_linux_x86_64": ["baz_new", "posix_dep", "posix_dep_with_version"], + "cp39.8_windows_x86_64": ["baz_new"], "linux_x86_64": ["arch_dep", "baz", "posix_dep"], "windows_x86_64": ["arch_dep", "baz"], }) @@ -294,8 +312,6 @@ def _test_deps_are_not_duplicated(env): _tests.append(_test_deps_are_not_duplicated) def _test_deps_are_not_duplicated_when_encountering_platform_dep_first(env): - default_python_version = "3.7.1" - # Note, that we are sorting the incoming `requires_dist` and we need to ensure that we are not getting any # issues even if the platform-specific line comes first. requires_dist = [ @@ -307,19 +323,20 @@ def _test_deps_are_not_duplicated_when_encountering_platform_dep_first(env): "foo", requires_dist = requires_dist, platforms = [ - "cp37_linux_aarch64", - "cp37_linux_x86_64", + "cp37.1_linux_aarch64", + "cp37.1_linux_x86_64", "cp310_linux_aarch64", "cp310_linux_x86_64", ], - default_python_version = default_python_version, + default_python_version = "3.7.1", + minor_mapping = {}, ) env.expect.that_collection(got.deps).contains_exactly([]) env.expect.that_dict(got.deps_select).contains_exactly({ "cp310_linux_aarch64": ["bar"], "cp310_linux_x86_64": ["bar"], - "cp37_linux_aarch64": ["bar"], + "cp37.1_linux_aarch64": ["bar"], "linux_aarch64": ["bar"], }) diff --git a/tests/pypi/render_pkg_aliases/render_pkg_aliases_test.bzl b/tests/pypi/render_pkg_aliases/render_pkg_aliases_test.bzl index c60761bed7..416d50bd80 100644 --- a/tests/pypi/render_pkg_aliases/render_pkg_aliases_test.bzl +++ b/tests/pypi/render_pkg_aliases/render_pkg_aliases_test.bzl @@ -68,7 +68,8 @@ def _test_bzlmod_aliases(env): aliases = { "bar-baz": { whl_config_setting( - version = "3.2", + # Add one with micro version to mimic construction in the extension + version = "3.2.2", config_setting = "//:my_config_setting", ): "pypi_32_bar_baz", whl_config_setting( @@ -83,10 +84,10 @@ def _test_bzlmod_aliases(env): filename = "foo-0.0.0-py3-none-any.whl", ): "filename_repo", whl_config_setting( - version = "3.2", + version = "3.2.2", filename = "foo-0.0.0-py3-none-any.whl", target_platforms = [ - "cp32_linux_x86_64", + "cp32.2_linux_x86_64", ], ): "filename_repo_linux_x86_64", }, @@ -117,7 +118,7 @@ pkg_aliases( whl_config_setting( filename = "foo-0.0.0-py3-none-any.whl", target_platforms = ("cp32_linux_x86_64",), - version = "3.2", + version = "3.2.2", ): "filename_repo_linux_x86_64", }, extra_aliases = ["foo"], diff --git a/tests/pypi/whl_library_targets/whl_library_targets_tests.bzl b/tests/pypi/whl_library_targets/whl_library_targets_tests.bzl index 61e5441050..432cdbfa1b 100644 --- a/tests/pypi/whl_library_targets/whl_library_targets_tests.bzl +++ b/tests/pypi/whl_library_targets/whl_library_targets_tests.bzl @@ -68,9 +68,8 @@ def _test_platforms(env): "@//python/config_settings:is_python_3.9": ["py39_dep"], "@platforms//cpu:aarch64": ["arm_dep"], "@platforms//os:windows": ["win_dep"], + "cp310.11_linux_ppc64le": ["full_version_dep"], "cp310_linux_ppc64le": ["py310_linux_ppc64le_dep"], - "cp39_anyos_aarch64": ["py39_arm_dep"], - "cp39_linux_anyarch": ["py39_linux_dep"], "linux_x86_64": ["linux_intel_dep"], }, filegroups = {}, @@ -82,39 +81,34 @@ def _test_platforms(env): env.expect.that_collection(calls).contains_exactly([ { - "name": "is_python_3.10_linux_ppc64le", - "flag_values": { - "@rules_python//python/config_settings:python_version_major_minor": "3.10", - }, + "name": "is_python_3.10.11_linux_ppc64le", + "visibility": ["//visibility:private"], "constraint_values": [ "@platforms//cpu:ppc64le", "@platforms//os:linux", ], - "visibility": ["//visibility:private"], - }, - { - "name": "is_python_3.9_anyos_aarch64", "flag_values": { - "@rules_python//python/config_settings:python_version_major_minor": "3.9", + Label("//python/config_settings:python_version"): "3.10.11", }, - "constraint_values": ["@platforms//cpu:aarch64"], - "visibility": ["//visibility:private"], }, { - "name": "is_python_3.9_linux_anyarch", + "name": "is_python_3.10_linux_ppc64le", + "visibility": ["//visibility:private"], + "constraint_values": [ + "@platforms//cpu:ppc64le", + "@platforms//os:linux", + ], "flag_values": { - "@rules_python//python/config_settings:python_version_major_minor": "3.9", + Label("//python/config_settings:python_version"): "3.10", }, - "constraint_values": ["@platforms//os:linux"], - "visibility": ["//visibility:private"], }, { "name": "is_linux_x86_64", + "visibility": ["//visibility:private"], "constraint_values": [ "@platforms//cpu:x86_64", "@platforms//os:linux", ], - "visibility": ["//visibility:private"], }, ]) # buildifier: @unsorted-dict-items diff --git a/tests/pypi/whl_target_platforms/select_whl_tests.bzl b/tests/pypi/whl_target_platforms/select_whl_tests.bzl index 8ab24138d1..1674ac5ef2 100644 --- a/tests/pypi/whl_target_platforms/select_whl_tests.bzl +++ b/tests/pypi/whl_target_platforms/select_whl_tests.bzl @@ -289,6 +289,22 @@ def _test_freethreaded_wheels(env): _tests.append(_test_freethreaded_wheels) +def _test_micro_version_freethreaded(env): + # Check we prefer platform specific wheels + got = _select_whls(whls = WHL_LIST, want_platforms = ["cp313.3_linux_x86_64"]) + _match( + env, + got, + "pkg-0.0.1-cp313-cp313t-musllinux_1_1_x86_64.whl", + "pkg-0.0.1-cp313-cp313-musllinux_1_1_x86_64.whl", + "pkg-0.0.1-cp313-abi3-musllinux_1_1_x86_64.whl", + "pkg-0.0.1-cp313-none-musllinux_1_1_x86_64.whl", + "pkg-0.0.1-cp39-abi3-any.whl", + "pkg-0.0.1-py3-none-any.whl", + ) + +_tests.append(_test_micro_version_freethreaded) + def select_whl_test_suite(name): """Create the test suite. From ee3440986f422c6a02d52d594816e571d0c633d8 Mon Sep 17 00:00:00 2001 From: Ignas Anikevicius <240938+aignas@users.noreply.github.com> Date: Fri, 25 Apr 2025 03:37:31 +0900 Subject: [PATCH 192/922] fix(pypi): call python --version before marker eval (#2819) `bzlmod` has the full python version information statically and we don't need to call Python to get its version, but for `WORKSPACE` that is not the case and we have to call it before evaluating the markers in universal requirements files. This also fixes transitions in the `compile_pip_requirements` macro where the `.update` target would not transition correctly based on the `python_version` parameter. Fixes #2818 --- CHANGELOG.md | 4 +++ .../requirements/requirements.in | 2 +- .../requirements/requirements_lock_3_10.txt | 2 +- .../requirements/requirements_lock_3_11.txt | 2 +- .../requirements/requirements_lock_3_9.txt | 2 +- python/private/pypi/BUILD.bazel | 1 + python/private/pypi/evaluate_markers.bzl | 7 +++--- python/private/pypi/pip_compile.bzl | 1 + python/private/pypi/pip_repository.bzl | 25 +++++++++++++++++-- 9 files changed, 37 insertions(+), 9 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 984af8bad2..8fc00ca25f 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -151,6 +151,10 @@ END_UNRELEASED_TEMPLATE * (pypi) Correctly handle `METADATA` entries when `python_full_version` is used in the environment marker. Fixes [#2319](https://github.com/bazel-contrib/rules_python/issues/2319). +* (pypi) Correctly handle `python_version` parameter and transition the requirement + locking to the right interpreter version when using + {obj}`compile_pip_requirements` rule. + See [#2819](https://github.com/bazel-contrib/rules_python/pull/2819). {#1-4-0-added} ### Added diff --git a/examples/multi_python_versions/requirements/requirements.in b/examples/multi_python_versions/requirements/requirements.in index 14774b465e..4d1474b9a2 100644 --- a/examples/multi_python_versions/requirements/requirements.in +++ b/examples/multi_python_versions/requirements/requirements.in @@ -1 +1 @@ -websockets +websockets ; python_full_version > "3.9.1" diff --git a/examples/multi_python_versions/requirements/requirements_lock_3_10.txt b/examples/multi_python_versions/requirements/requirements_lock_3_10.txt index 4910d13844..3a8453223f 100644 --- a/examples/multi_python_versions/requirements/requirements_lock_3_10.txt +++ b/examples/multi_python_versions/requirements/requirements_lock_3_10.txt @@ -4,7 +4,7 @@ # # bazel run //requirements:requirements_3_10.update # -websockets==11.0.3 \ +websockets==11.0.3 ; python_full_version > "3.9.1" \ --hash=sha256:01f5567d9cf6f502d655151645d4e8b72b453413d3819d2b6f1185abc23e82dd \ --hash=sha256:03aae4edc0b1c68498f41a6772d80ac7c1e33c06c6ffa2ac1c27a07653e79d6f \ --hash=sha256:0ac56b661e60edd453585f4bd68eb6a29ae25b5184fd5ba51e97652580458998 \ diff --git a/examples/multi_python_versions/requirements/requirements_lock_3_11.txt b/examples/multi_python_versions/requirements/requirements_lock_3_11.txt index 35666b54b1..f1fa8f56f5 100644 --- a/examples/multi_python_versions/requirements/requirements_lock_3_11.txt +++ b/examples/multi_python_versions/requirements/requirements_lock_3_11.txt @@ -4,7 +4,7 @@ # # bazel run //requirements:requirements_3_11.update # -websockets==11.0.3 \ +websockets==11.0.3 ; python_full_version > "3.9.1" \ --hash=sha256:01f5567d9cf6f502d655151645d4e8b72b453413d3819d2b6f1185abc23e82dd \ --hash=sha256:03aae4edc0b1c68498f41a6772d80ac7c1e33c06c6ffa2ac1c27a07653e79d6f \ --hash=sha256:0ac56b661e60edd453585f4bd68eb6a29ae25b5184fd5ba51e97652580458998 \ diff --git a/examples/multi_python_versions/requirements/requirements_lock_3_9.txt b/examples/multi_python_versions/requirements/requirements_lock_3_9.txt index 0001f88d48..3c696a865e 100644 --- a/examples/multi_python_versions/requirements/requirements_lock_3_9.txt +++ b/examples/multi_python_versions/requirements/requirements_lock_3_9.txt @@ -4,7 +4,7 @@ # # bazel run //requirements:requirements_3_9.update # -websockets==11.0.3 \ +websockets==11.0.3 ; python_full_version > "3.9.1" \ --hash=sha256:01f5567d9cf6f502d655151645d4e8b72b453413d3819d2b6f1185abc23e82dd \ --hash=sha256:03aae4edc0b1c68498f41a6772d80ac7c1e33c06c6ffa2ac1c27a07653e79d6f \ --hash=sha256:0ac56b661e60edd453585f4bd68eb6a29ae25b5184fd5ba51e97652580458998 \ diff --git a/python/private/pypi/BUILD.bazel b/python/private/pypi/BUILD.bazel index bfb0be2d59..9216134857 100644 --- a/python/private/pypi/BUILD.bazel +++ b/python/private/pypi/BUILD.bazel @@ -283,6 +283,7 @@ bzl_library( ":evaluate_markers_bzl", ":parse_requirements_bzl", ":pip_repository_attrs_bzl", + ":pypi_repo_utils_bzl", ":render_pkg_aliases_bzl", ":whl_config_setting_bzl", "//python/private:normalize_name_bzl", diff --git a/python/private/pypi/evaluate_markers.bzl b/python/private/pypi/evaluate_markers.bzl index a0223abdc8..f966aa32be 100644 --- a/python/private/pypi/evaluate_markers.bzl +++ b/python/private/pypi/evaluate_markers.bzl @@ -19,11 +19,12 @@ load(":pep508_evaluate.bzl", "evaluate") load(":pep508_platform.bzl", "platform_from_str") load(":pep508_requirement.bzl", "requirement") -def evaluate_markers(requirements): +def evaluate_markers(requirements, python_version = None): """Return the list of supported platforms per requirements line. Args: - requirements: dict[str, list[str]] of the requirement file lines to evaluate. + requirements: {type}`dict[str, list[str]]` of the requirement file lines to evaluate. + python_version: {type}`str | None` the version that can be used when evaluating the markers. Returns: dict of string lists with target platforms @@ -32,7 +33,7 @@ def evaluate_markers(requirements): for req_string, platforms in requirements.items(): req = requirement(req_string) for platform in platforms: - if evaluate(req.marker, env = env(platform_from_str(platform, None))): + if evaluate(req.marker, env = env(platform_from_str(platform, python_version))): ret.setdefault(req_string, []).append(platform) return ret diff --git a/python/private/pypi/pip_compile.bzl b/python/private/pypi/pip_compile.bzl index e5b62c4ab0..9782d3ce21 100644 --- a/python/private/pypi/pip_compile.bzl +++ b/python/private/pypi/pip_compile.bzl @@ -160,6 +160,7 @@ def pip_compile( py_binary( name = name + ".update", env = env, + python_version = kwargs.get("python_version", None), **attrs ) diff --git a/python/private/pypi/pip_repository.bzl b/python/private/pypi/pip_repository.bzl index 01a541cf2f..b7ed1659d1 100644 --- a/python/private/pypi/pip_repository.bzl +++ b/python/private/pypi/pip_repository.bzl @@ -16,11 +16,12 @@ load("@bazel_skylib//lib:sets.bzl", "sets") load("//python/private:normalize_name.bzl", "normalize_name") -load("//python/private:repo_utils.bzl", "REPO_DEBUG_ENV_VAR") +load("//python/private:repo_utils.bzl", "REPO_DEBUG_ENV_VAR", "repo_utils") load("//python/private:text_util.bzl", "render") load(":evaluate_markers.bzl", "evaluate_markers") load(":parse_requirements.bzl", "host_platform", "parse_requirements", "select_requirement") load(":pip_repository_attrs.bzl", "ATTRS") +load(":pypi_repo_utils.bzl", "pypi_repo_utils") load(":render_pkg_aliases.bzl", "render_pkg_aliases") load(":requirements_files_by_platform.bzl", "requirements_files_by_platform") @@ -70,7 +71,27 @@ package(default_visibility = ["//visibility:public"]) exports_files(["requirements.bzl"]) """ +def _evaluate_markers(rctx, requirements, logger = None): + python_interpreter = _get_python_interpreter_attr(rctx) + stdout = pypi_repo_utils.execute_checked_stdout( + rctx, + op = "GetPythonVersionForMarkerEval", + python = python_interpreter, + arguments = [ + # Run the interpreter in isolated mode, this options implies -E, -P and -s. + # Ensures environment variables are ignored that are set in userspace, such as PYTHONPATH, + # which may interfere with this invocation. + "-I", + "-c", + "import sys; print(f'{sys.version_info[0]}.{sys.version_info[1]}.{sys.version_info[2]}', end='')", + ], + srcs = [], + logger = logger, + ) + return evaluate_markers(requirements, python_version = stdout) + def _pip_repository_impl(rctx): + logger = repo_utils.logger(rctx) requirements_by_platform = parse_requirements( rctx, requirements_by_platform = requirements_files_by_platform( @@ -82,7 +103,7 @@ def _pip_repository_impl(rctx): extra_pip_args = rctx.attr.extra_pip_args, ), extra_pip_args = rctx.attr.extra_pip_args, - evaluate_markers = evaluate_markers, + evaluate_markers = lambda requirements: _evaluate_markers(rctx, requirements, logger), ) selected_requirements = {} options = None From 070aa43745810950d572367f7fd6acbf517a76c7 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Thu, 24 Apr 2025 16:41:45 -0700 Subject: [PATCH 193/922] docs: add xrefs for local toolchains rules (#2823) This is to make it easier to find the API docs for the rules the docs talk about. --- docs/toolchains.md | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/docs/toolchains.md b/docs/toolchains.md index 320e16335b..2f8db66595 100644 --- a/docs/toolchains.md +++ b/docs/toolchains.md @@ -339,9 +339,10 @@ runtime metadata (Python version, headers, ABI flags, etc) that the regular remotely downloaded runtimes contain, which makes it possible to build e.g. C extensions (unlike the autodetecting and runtime environment toolchains). -For simple cases, some rules are provided that will introspect -a Python installation and create an appropriate Bazel definition from -it. To do this, three pieces need to be wired together: +For simple cases, the {obj}`local_runtime_repo` and +{obj}`local_runtime_toolchains_repo` rules are provided that will introspect a +Python installation and create an appropriate Bazel definition from it. To do +this, three pieces need to be wired together: 1. Specify a path or command to a Python interpreter (multiple can be defined). 2. Create toolchains for the runtimes in (1) From 7234ddae6debeea091d88233c9d974756e64d6e4 Mon Sep 17 00:00:00 2001 From: Fabian Meumertzheim Date: Fri, 25 Apr 2025 17:05:44 +0200 Subject: [PATCH 194/922] docs: Improve bazel-runfiles docs (#2824) --- python/runfiles/README.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/python/runfiles/README.md b/python/runfiles/README.md index 2a57c76846..b5315a48f5 100644 --- a/python/runfiles/README.md +++ b/python/runfiles/README.md @@ -59,6 +59,8 @@ with open(r.Rlocation("my_workspace/path/to/my/data.txt"), "r") as f: # ... ``` +Here `my_workspace` is the name you specified via `module(name = "...")` in your `MODULE.bazel` file (with `--enable_bzlmod`, default as of Bazel 7) or `workspace(name = "...")` in `WORKSPACE` (with `--noenable_bzlmod`). + The code above creates a manifest- or directory-based implementation based on the environment variables in `os.environ`. See `Runfiles.Create()` for more info. If you want to explicitly create a manifest- or directory-based @@ -70,9 +72,7 @@ r1 = Runfiles.CreateManifestBased("path/to/foo.runfiles_manifest") r2 = Runfiles.CreateDirectoryBased("path/to/foo.runfiles/") ``` -If you want to start subprocesses, and the subprocess can't automatically -find the correct runfiles directory, you can explicitly set the right -environment variables for them: +If you want to start subprocesses that access runfiles, you have to set the right environment variables for them: ```python import subprocess From 61c91fe9bd322f91af77db2f57e5b6b40792628f Mon Sep 17 00:00:00 2001 From: Ignas Anikevicius <240938+aignas@users.noreply.github.com> Date: Sun, 27 Apr 2025 12:43:38 +0900 Subject: [PATCH 195/922] revert(pypi): bring back Python PEP508 code with tests (#2831) This just adds the code back at the original state before the following PRs have been made to remove them: #2629, #2781. This has not been hooked up yet in `evaluate_markers` and `whl_library` yet and I'll need extra PRs to do that. No CHANGELOG entries for now, will be done once the integration is back. Work towards #2830 --- .../pypi/requirements_parser/BUILD.bazel | 0 .../resolve_target_platforms.py | 63 +++ python/private/pypi/whl_installer/BUILD.bazel | 1 + .../private/pypi/whl_installer/arguments.py | 8 + python/private/pypi/whl_installer/platform.py | 304 ++++++++++++++ python/private/pypi/whl_installer/wheel.py | 281 +++++++++++++ .../pypi/whl_installer/wheel_installer.py | 38 +- tests/pypi/whl_installer/BUILD.bazel | 24 ++ tests/pypi/whl_installer/arguments_test.py | 14 +- tests/pypi/whl_installer/platform_test.py | 154 ++++++++ .../whl_installer/wheel_installer_test.py | 41 +- tests/pypi/whl_installer/wheel_test.py | 371 ++++++++++++++++++ 12 files changed, 1285 insertions(+), 14 deletions(-) create mode 100644 python/private/pypi/requirements_parser/BUILD.bazel create mode 100755 python/private/pypi/requirements_parser/resolve_target_platforms.py create mode 100644 python/private/pypi/whl_installer/platform.py create mode 100644 tests/pypi/whl_installer/platform_test.py create mode 100644 tests/pypi/whl_installer/wheel_test.py diff --git a/python/private/pypi/requirements_parser/BUILD.bazel b/python/private/pypi/requirements_parser/BUILD.bazel new file mode 100644 index 0000000000..e69de29bb2 diff --git a/python/private/pypi/requirements_parser/resolve_target_platforms.py b/python/private/pypi/requirements_parser/resolve_target_platforms.py new file mode 100755 index 0000000000..c899a943cc --- /dev/null +++ b/python/private/pypi/requirements_parser/resolve_target_platforms.py @@ -0,0 +1,63 @@ +"""A CLI to evaluate env markers for requirements files. + +A simple script to evaluate the `requirements.txt` files. Currently it is only +handling environment markers in the requirements files, but in the future it +may handle more things. We require a `python` interpreter that can run on the +host platform and then we depend on the [packaging] PyPI wheel. + +In order to be able to resolve requirements files for any platform, we are +re-using the same code that is used in the `whl_library` installer. See +[here](../whl_installer/wheel.py). + +Requirements for the code are: +- Depends only on `packaging` and core Python. +- Produces the same result irrespective of the Python interpreter platform or version. + +[packaging]: https://packaging.pypa.io/en/stable/ +""" + +import argparse +import json +import pathlib + +from packaging.requirements import Requirement + +from python.private.pypi.whl_installer.platform import Platform + +INPUT_HELP = """\ +Input path to read the requirements as a json file, the keys in the dictionary +are the requirements lines and the values are strings of target platforms. +""" +OUTPUT_HELP = """\ +Output to write the requirements as a json filepath, the keys in the dictionary +are the requirements lines and the values are strings of target platforms, which +got changed based on the evaluated markers. +""" + + +def main(): + parser = argparse.ArgumentParser(description=__doc__) + parser.add_argument("input_path", type=pathlib.Path, help=INPUT_HELP.strip()) + parser.add_argument("output_path", type=pathlib.Path, help=OUTPUT_HELP.strip()) + args = parser.parse_args() + + with args.input_path.open() as f: + reqs = json.load(f) + + response = {} + for requirement_line, target_platforms in reqs.items(): + entry, prefix, hashes = requirement_line.partition("--hash") + hashes = prefix + hashes + + req = Requirement(entry) + for p in target_platforms: + (platform,) = Platform.from_string(p) + if not req.marker or req.marker.evaluate(platform.env_markers("")): + response.setdefault(requirement_line, []).append(p) + + with args.output_path.open("w") as f: + json.dump(response, f) + + +if __name__ == "__main__": + main() diff --git a/python/private/pypi/whl_installer/BUILD.bazel b/python/private/pypi/whl_installer/BUILD.bazel index 49f1a119c1..5fb617004d 100644 --- a/python/private/pypi/whl_installer/BUILD.bazel +++ b/python/private/pypi/whl_installer/BUILD.bazel @@ -6,6 +6,7 @@ py_library( srcs = [ "arguments.py", "namespace_pkgs.py", + "platform.py", "wheel.py", "wheel_installer.py", ], diff --git a/python/private/pypi/whl_installer/arguments.py b/python/private/pypi/whl_installer/arguments.py index bb841ea9ab..29bea8026e 100644 --- a/python/private/pypi/whl_installer/arguments.py +++ b/python/private/pypi/whl_installer/arguments.py @@ -17,6 +17,8 @@ import pathlib from typing import Any, Dict, Set +from python.private.pypi.whl_installer.platform import Platform + def parser(**kwargs: Any) -> argparse.ArgumentParser: """Create a parser for the wheel_installer tool.""" @@ -39,6 +41,12 @@ def parser(**kwargs: Any) -> argparse.ArgumentParser: action="store", help="Extra arguments to pass down to pip.", ) + parser.add_argument( + "--platform", + action="extend", + type=Platform.from_string, + help="Platforms to target dependencies. Can be used multiple times.", + ) parser.add_argument( "--pip_data_exclude", action="store", diff --git a/python/private/pypi/whl_installer/platform.py b/python/private/pypi/whl_installer/platform.py new file mode 100644 index 0000000000..11dd6e37ab --- /dev/null +++ b/python/private/pypi/whl_installer/platform.py @@ -0,0 +1,304 @@ +# Copyright 2024 The Bazel Authors. All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +"""Utility class to inspect an extracted wheel directory""" + +import platform +import sys +from dataclasses import dataclass +from enum import Enum +from typing import Any, Dict, Iterator, List, Optional, Union + + +class OS(Enum): + linux = 1 + osx = 2 + windows = 3 + darwin = osx + win32 = windows + + @classmethod + def interpreter(cls) -> "OS": + "Return the interpreter operating system." + return cls[sys.platform.lower()] + + def __str__(self) -> str: + return self.name.lower() + + +class Arch(Enum): + x86_64 = 1 + x86_32 = 2 + aarch64 = 3 + ppc = 4 + ppc64le = 5 + s390x = 6 + arm = 7 + amd64 = x86_64 + arm64 = aarch64 + i386 = x86_32 + i686 = x86_32 + x86 = x86_32 + + @classmethod + def interpreter(cls) -> "Arch": + "Return the currently running interpreter architecture." + # FIXME @aignas 2023-12-13: Hermetic toolchain on Windows 3.11.6 + # is returning an empty string here, so lets default to x86_64 + return cls[platform.machine().lower() or "x86_64"] + + def __str__(self) -> str: + return self.name.lower() + + +def _as_int(value: Optional[Union[OS, Arch]]) -> int: + """Convert one of the enums above to an int for easier sorting algorithms. + + Args: + value: The value of an enum or None. + + Returns: + -1 if we get None, otherwise, the numeric value of the given enum. + """ + if value is None: + return -1 + + return int(value.value) + + +def host_interpreter_minor_version() -> int: + return sys.version_info.minor + + +@dataclass(frozen=True) +class Platform: + os: Optional[OS] = None + arch: Optional[Arch] = None + minor_version: Optional[int] = None + + @classmethod + def all( + cls, + want_os: Optional[OS] = None, + minor_version: Optional[int] = None, + ) -> List["Platform"]: + return sorted( + [ + cls(os=os, arch=arch, minor_version=minor_version) + for os in OS + for arch in Arch + if not want_os or want_os == os + ] + ) + + @classmethod + def host(cls) -> List["Platform"]: + """Use the Python interpreter to detect the platform. + + We extract `os` from sys.platform and `arch` from platform.machine + + Returns: + A list of parsed values which makes the signature the same as + `Platform.all` and `Platform.from_string`. + """ + return [ + Platform( + os=OS.interpreter(), + arch=Arch.interpreter(), + minor_version=host_interpreter_minor_version(), + ) + ] + + def all_specializations(self) -> Iterator["Platform"]: + """Return the platform itself and all its unambiguous specializations. + + For more info about specializations see + https://bazel.build/docs/configurable-attributes + """ + yield self + if self.arch is None: + for arch in Arch: + yield Platform(os=self.os, arch=arch, minor_version=self.minor_version) + if self.os is None: + for os in OS: + yield Platform(os=os, arch=self.arch, minor_version=self.minor_version) + if self.arch is None and self.os is None: + for os in OS: + for arch in Arch: + yield Platform(os=os, arch=arch, minor_version=self.minor_version) + + def __lt__(self, other: Any) -> bool: + """Add a comparison method, so that `sorted` returns the most specialized platforms first.""" + if not isinstance(other, Platform) or other is None: + raise ValueError(f"cannot compare {other} with Platform") + + self_arch, self_os = _as_int(self.arch), _as_int(self.os) + other_arch, other_os = _as_int(other.arch), _as_int(other.os) + + if self_os == other_os: + return self_arch < other_arch + else: + return self_os < other_os + + def __str__(self) -> str: + if self.minor_version is None: + if self.os is None and self.arch is None: + return "//conditions:default" + + if self.arch is None: + return f"@platforms//os:{self.os}" + else: + return f"{self.os}_{self.arch}" + + if self.arch is None and self.os is None: + return f"@//python/config_settings:is_python_3.{self.minor_version}" + + if self.arch is None: + return f"cp3{self.minor_version}_{self.os}_anyarch" + + if self.os is None: + return f"cp3{self.minor_version}_anyos_{self.arch}" + + return f"cp3{self.minor_version}_{self.os}_{self.arch}" + + @classmethod + def from_string(cls, platform: Union[str, List[str]]) -> List["Platform"]: + """Parse a string and return a list of platforms""" + platform = [platform] if isinstance(platform, str) else list(platform) + ret = set() + for p in platform: + if p == "host": + ret.update(cls.host()) + continue + + abi, _, tail = p.partition("_") + if not abi.startswith("cp"): + # The first item is not an abi + tail = p + abi = "" + os, _, arch = tail.partition("_") + arch = arch or "*" + + minor_version = int(abi[len("cp3") :]) if abi else None + + if arch != "*": + ret.add( + cls( + os=OS[os] if os != "*" else None, + arch=Arch[arch], + minor_version=minor_version, + ) + ) + + else: + ret.update( + cls.all( + want_os=OS[os] if os != "*" else None, + minor_version=minor_version, + ) + ) + + return sorted(ret) + + # NOTE @aignas 2023-12-05: below is the minimum number of accessors that are defined in + # https://peps.python.org/pep-0496/ to make rules_python generate dependencies. + # + # WARNING: It may not work in cases where the python implementation is different between + # different platforms. + + # derived from OS + @property + def os_name(self) -> str: + if self.os == OS.linux or self.os == OS.osx: + return "posix" + elif self.os == OS.windows: + return "nt" + else: + return "" + + @property + def sys_platform(self) -> str: + if self.os == OS.linux: + return "linux" + elif self.os == OS.osx: + return "darwin" + elif self.os == OS.windows: + return "win32" + else: + return "" + + @property + def platform_system(self) -> str: + if self.os == OS.linux: + return "Linux" + elif self.os == OS.osx: + return "Darwin" + elif self.os == OS.windows: + return "Windows" + else: + return "" + + # derived from OS and Arch + @property + def platform_machine(self) -> str: + """Guess the target 'platform_machine' marker. + + NOTE @aignas 2023-12-05: this may not work on really new systems, like + Windows if they define the platform markers in a different way. + """ + if self.arch == Arch.x86_64: + return "x86_64" + elif self.arch == Arch.x86_32 and self.os != OS.osx: + return "i386" + elif self.arch == Arch.x86_32: + return "" + elif self.arch == Arch.aarch64 and self.os == OS.linux: + return "aarch64" + elif self.arch == Arch.aarch64: + # Assuming that OSX and Windows use this one since the precedent is set here: + # https://github.com/cgohlke/win_arm64-wheels + return "arm64" + elif self.os != OS.linux: + return "" + elif self.arch == Arch.ppc: + return "ppc" + elif self.arch == Arch.ppc64le: + return "ppc64le" + elif self.arch == Arch.s390x: + return "s390x" + else: + return "" + + def env_markers(self, extra: str) -> Dict[str, str]: + # If it is None, use the host version + minor_version = self.minor_version or host_interpreter_minor_version() + + return { + "extra": extra, + "os_name": self.os_name, + "sys_platform": self.sys_platform, + "platform_machine": self.platform_machine, + "platform_system": self.platform_system, + "platform_release": "", # unset + "platform_version": "", # unset + "python_version": f"3.{minor_version}", + # FIXME @aignas 2024-01-14: is putting zero last a good idea? Maybe we should + # use `20` or something else to avoid having weird issues where the full version is used for + # matching and the author decides to only support 3.y.5 upwards. + "implementation_version": f"3.{minor_version}.0", + "python_full_version": f"3.{minor_version}.0", + # we assume that the following are the same as the interpreter used to setup the deps: + # "implementation_name": "cpython" + # "platform_python_implementation: "CPython", + } diff --git a/python/private/pypi/whl_installer/wheel.py b/python/private/pypi/whl_installer/wheel.py index da81b5ea9f..d95b33a194 100644 --- a/python/private/pypi/whl_installer/wheel.py +++ b/python/private/pypi/whl_installer/wheel.py @@ -25,6 +25,275 @@ from packaging.requirements import Requirement from pip._vendor.packaging.utils import canonicalize_name +from python.private.pypi.whl_installer.platform import ( + Platform, + host_interpreter_minor_version, +) + + +@dataclass(frozen=True) +class FrozenDeps: + deps: List[str] + deps_select: Dict[str, List[str]] + + +class Deps: + """Deps is a dependency builder that has a build() method to return FrozenDeps.""" + + def __init__( + self, + name: str, + requires_dist: List[str], + *, + extras: Optional[Set[str]] = None, + platforms: Optional[Set[Platform]] = None, + ): + """Create a new instance and parse the requires_dist + + Args: + name (str): The name of the whl distribution + requires_dist (list[Str]): The Requires-Dist from the METADATA of the whl + distribution. + extras (set[str], optional): The list of requested extras, defaults to None. + platforms (set[Platform], optional): The list of target platforms, defaults to + None. If the list of platforms has multiple `minor_version` values, it + will change the code to generate the select statements using + `@rules_python//python/config_settings:is_python_3.y` conditions. + """ + self.name: str = Deps._normalize(name) + self._platforms: Set[Platform] = platforms or set() + self._target_versions = {p.minor_version for p in platforms or {}} + self._default_minor_version = None + if platforms and len(self._target_versions) > 2: + # TODO @aignas 2024-06-23: enable this to be set via a CLI arg + # for being more explicit. + self._default_minor_version = host_interpreter_minor_version() + + if None in self._target_versions and len(self._target_versions) > 2: + raise ValueError( + f"all python versions need to be specified explicitly, got: {platforms}" + ) + + # Sort so that the dictionary order in the FrozenDeps is deterministic + # without the final sort because Python retains insertion order. That way + # the sorting by platform is limited within the Platform class itself and + # the unit-tests for the Deps can be simpler. + reqs = sorted( + (Requirement(wheel_req) for wheel_req in requires_dist), + key=lambda x: f"{x.name}:{sorted(x.extras)}", + ) + + want_extras = self._resolve_extras(reqs, extras) + + # Then add all of the requirements in order + self._deps: Set[str] = set() + self._select: Dict[Platform, Set[str]] = defaultdict(set) + for req in reqs: + self._add_req(req, want_extras) + + def _add(self, dep: str, platform: Optional[Platform]): + dep = Deps._normalize(dep) + + # Self-edges are processed in _resolve_extras + if dep == self.name: + return + + if not platform: + self._deps.add(dep) + + # If the dep is in the platform-specific list, remove it from the select. + pop_keys = [] + for p, deps in self._select.items(): + if dep not in deps: + continue + + deps.remove(dep) + if not deps: + pop_keys.append(p) + + for p in pop_keys: + self._select.pop(p) + return + + if dep in self._deps: + # If the dep is already in the main dependency list, no need to add it in the + # platform-specific dependency list. + return + + # Add the platform-specific dep + self._select[platform].add(dep) + + # Add the dep to specializations of the given platform if they + # exist in the select statement. + for p in platform.all_specializations(): + if p not in self._select: + continue + + self._select[p].add(dep) + + if len(self._select[platform]) == 1: + # We are adding a new item to the select and we need to ensure that + # existing dependencies from less specialized platforms are propagated + # to the newly added dependency set. + for p, deps in self._select.items(): + # Check if the existing platform overlaps with the given platform + if p == platform or platform not in p.all_specializations(): + continue + + self._select[platform].update(self._select[p]) + + def _maybe_add_common_dep(self, dep): + if len(self._target_versions) < 2: + return + + platforms = [Platform()] + [ + Platform(minor_version=v) for v in self._target_versions + ] + + # If the dep is targeting all target python versions, lets add it to + # the common dependency list to simplify the select statements. + for p in platforms: + if p not in self._select: + return + + if dep not in self._select[p]: + return + + # All of the python version-specific branches have the dep, so lets add + # it to the common deps. + self._deps.add(dep) + for p in platforms: + self._select[p].remove(dep) + if not self._select[p]: + self._select.pop(p) + + @staticmethod + def _normalize(name: str) -> str: + return re.sub(r"[-_.]+", "_", name).lower() + + def _resolve_extras( + self, reqs: List[Requirement], extras: Optional[Set[str]] + ) -> Set[str]: + """Resolve extras which are due to depending on self[some_other_extra]. + + Some packages may have cyclic dependencies resulting from extras being used, one example is + `etils`, where we have one set of extras as aliases for other extras + and we have an extra called 'all' that includes all other extras. + + Example: github.com/google/etils/blob/a0b71032095db14acf6b33516bca6d885fe09e35/pyproject.toml#L32. + + When the `requirements.txt` is generated by `pip-tools`, then it is likely that + this step is not needed, but for other `requirements.txt` files this may be useful. + + NOTE @aignas 2023-12-08: the extra resolution is not platform dependent, + but in order for it to become platform dependent we would have to have + separate targets for each extra in extras. + """ + + # Resolve any extra extras due to self-edges, empty string means no + # extras The empty string in the set is just a way to make the handling + # of no extras and a single extra easier and having a set of {"", "foo"} + # is equivalent to having {"foo"}. + extras = extras or {""} + + self_reqs = [] + for req in reqs: + if Deps._normalize(req.name) != self.name: + continue + + if req.marker is None: + # I am pretty sure we cannot reach this code as it does not + # make sense to specify packages in this way, but since it is + # easy to handle, lets do it. + # + # TODO @aignas 2023-12-08: add a test + extras = extras | req.extras + else: + # process these in a separate loop + self_reqs.append(req) + + # A double loop is not strictly optimal, but always correct without recursion + for req in self_reqs: + if any(req.marker.evaluate({"extra": extra}) for extra in extras): + extras = extras | req.extras + else: + continue + + # Iterate through all packages to ensure that we include all of the extras from previously + # visited packages. + for req_ in self_reqs: + if any(req_.marker.evaluate({"extra": extra}) for extra in extras): + extras = extras | req_.extras + + return extras + + def _add_req(self, req: Requirement, extras: Set[str]) -> None: + if req.marker is None: + self._add(req.name, None) + return + + marker_str = str(req.marker) + + if not self._platforms: + if any(req.marker.evaluate({"extra": extra}) for extra in extras): + self._add(req.name, None) + return + + # NOTE @aignas 2023-12-08: in order to have reasonable select statements + # we do have to have some parsing of the markers, so it begs the question + # if packaging should be reimplemented in Starlark to have the best solution + # for now we will implement it in Python and see what the best parsing result + # can be before making this decision. + match_os = any( + tag in marker_str + for tag in [ + "os_name", + "sys_platform", + "platform_system", + ] + ) + match_arch = "platform_machine" in marker_str + match_version = "version" in marker_str + + if not (match_os or match_arch or match_version): + if any(req.marker.evaluate({"extra": extra}) for extra in extras): + self._add(req.name, None) + return + + for plat in self._platforms: + if not any( + req.marker.evaluate(plat.env_markers(extra)) for extra in extras + ): + continue + + if match_arch and self._default_minor_version: + self._add(req.name, plat) + if plat.minor_version == self._default_minor_version: + self._add(req.name, Platform(plat.os, plat.arch)) + elif match_arch: + self._add(req.name, Platform(plat.os, plat.arch)) + elif match_os and self._default_minor_version: + self._add(req.name, Platform(plat.os, minor_version=plat.minor_version)) + if plat.minor_version == self._default_minor_version: + self._add(req.name, Platform(plat.os)) + elif match_os: + self._add(req.name, Platform(plat.os)) + elif match_version and self._default_minor_version: + self._add(req.name, Platform(minor_version=plat.minor_version)) + if plat.minor_version == self._default_minor_version: + self._add(req.name, Platform()) + elif match_version: + self._add(req.name, None) + + # Merge to common if possible after processing all platforms + self._maybe_add_common_dep(req.name) + + def build(self) -> FrozenDeps: + return FrozenDeps( + deps=sorted(self._deps), + deps_select={str(p): sorted(deps) for p, deps in self._select.items()}, + ) + class Wheel: """Representation of the compressed .whl file""" @@ -75,6 +344,18 @@ def entry_points(self) -> Dict[str, Tuple[str, str]]: return entry_points_mapping + def dependencies( + self, + extras_requested: Set[str] = None, + platforms: Optional[Set[Platform]] = None, + ) -> FrozenDeps: + return Deps( + self.name, + extras=extras_requested, + platforms=platforms, + requires_dist=self.metadata.get_all("Requires-Dist", []), + ).build() + def unzip(self, directory: str) -> None: installation_schemes = { "purelib": "/site-packages", diff --git a/python/private/pypi/whl_installer/wheel_installer.py b/python/private/pypi/whl_installer/wheel_installer.py index c7695d92e8..a48df699ba 100644 --- a/python/private/pypi/whl_installer/wheel_installer.py +++ b/python/private/pypi/whl_installer/wheel_installer.py @@ -23,7 +23,7 @@ import sys from pathlib import Path from tempfile import NamedTemporaryFile -from typing import Dict, Optional, Set, Tuple +from typing import Dict, List, Optional, Set, Tuple from pip._vendor.packaging.utils import canonicalize_name @@ -103,7 +103,9 @@ def _setup_namespace_pkg_compatibility(wheel_dir: str) -> None: def _extract_wheel( wheel_file: str, + extras: Dict[str, Set[str]], enable_implicit_namespace_pkgs: bool, + platforms: List[wheel.Platform], installation_dir: Path = Path("."), ) -> None: """Extracts wheel into given directory and creates py_library and filegroup targets. @@ -111,6 +113,7 @@ def _extract_wheel( Args: wheel_file: the filepath of the .whl installation_dir: the destination directory for installation of the wheel. + extras: a list of extras to add as dependencies for the installed wheel enable_implicit_namespace_pkgs: if true, disables conversion of implicit namespace packages and will unzip as-is """ @@ -120,19 +123,26 @@ def _extract_wheel( if not enable_implicit_namespace_pkgs: _setup_namespace_pkg_compatibility(installation_dir) - metadata = { - "python_version": sys.version.partition(" ")[0], - "entry_points": [ - { - "name": name, - "module": module, - "attribute": attribute, - } - for name, (module, attribute) in sorted(whl.entry_points().items()) - ], - } + extras_requested = extras[whl.name] if whl.name in extras else set() + + dependencies = whl.dependencies(extras_requested, platforms) with open(os.path.join(installation_dir, "metadata.json"), "w") as f: + metadata = { + "name": whl.name, + "version": whl.version, + "deps": dependencies.deps, + "python_version": f"{sys.version_info[0]}.{sys.version_info[1]}.{sys.version_info[2]}", + "deps_by_platform": dependencies.deps_select, + "entry_points": [ + { + "name": name, + "module": module, + "attribute": attribute, + } + for name, (module, attribute) in sorted(whl.entry_points().items()) + ], + } json.dump(metadata, f) @@ -146,9 +156,13 @@ def main() -> None: if args.whl_file: whl = Path(args.whl_file) + name, extras_for_pkg = _parse_requirement_for_extra(args.requirement) + extras = {name: extras_for_pkg} if extras_for_pkg and name else dict() _extract_wheel( wheel_file=whl, + extras=extras, enable_implicit_namespace_pkgs=args.enable_implicit_namespace_pkgs, + platforms=arguments.get_platforms(args), ) return diff --git a/tests/pypi/whl_installer/BUILD.bazel b/tests/pypi/whl_installer/BUILD.bazel index fea6a46d01..040e4d765f 100644 --- a/tests/pypi/whl_installer/BUILD.bazel +++ b/tests/pypi/whl_installer/BUILD.bazel @@ -27,6 +27,18 @@ py_test( ], ) +py_test( + name = "platform_test", + size = "small", + srcs = [ + "platform_test.py", + ], + data = ["//examples/wheel:minimal_with_py_package"], + deps = [ + ":lib", + ], +) + py_test( name = "wheel_installer_test", size = "small", @@ -38,3 +50,15 @@ py_test( ":lib", ], ) + +py_test( + name = "wheel_test", + size = "small", + srcs = [ + "wheel_test.py", + ], + data = ["//examples/wheel:minimal_with_py_package"], + deps = [ + ":lib", + ], +) diff --git a/tests/pypi/whl_installer/arguments_test.py b/tests/pypi/whl_installer/arguments_test.py index 9f73ae96a9..5538054a59 100644 --- a/tests/pypi/whl_installer/arguments_test.py +++ b/tests/pypi/whl_installer/arguments_test.py @@ -15,7 +15,7 @@ import json import unittest -from python.private.pypi.whl_installer import arguments +from python.private.pypi.whl_installer import arguments, wheel class ArgumentsTestCase(unittest.TestCase): @@ -49,6 +49,18 @@ def test_deserialize_structured_args(self) -> None: self.assertEqual(args["environment"], {"PIP_DO_SOMETHING": "True"}) self.assertEqual(args["extra_pip_args"], []) + def test_platform_aggregation(self) -> None: + parser = arguments.parser() + args = parser.parse_args( + args=[ + "--platform=linux_*", + "--platform=osx_*", + "--platform=windows_*", + "--requirement=foo", + ] + ) + self.assertEqual(set(wheel.Platform.all()), arguments.get_platforms(args)) + if __name__ == "__main__": unittest.main() diff --git a/tests/pypi/whl_installer/platform_test.py b/tests/pypi/whl_installer/platform_test.py new file mode 100644 index 0000000000..2aeb4caa69 --- /dev/null +++ b/tests/pypi/whl_installer/platform_test.py @@ -0,0 +1,154 @@ +import unittest +from random import shuffle + +from python.private.pypi.whl_installer.platform import ( + OS, + Arch, + Platform, + host_interpreter_minor_version, +) + + +class MinorVersionTest(unittest.TestCase): + def test_host(self): + host = host_interpreter_minor_version() + self.assertIsNotNone(host) + + +class PlatformTest(unittest.TestCase): + def test_can_get_host(self): + host = Platform.host() + self.assertIsNotNone(host) + self.assertEqual(1, len(Platform.from_string("host"))) + self.assertEqual(host, Platform.from_string("host")) + + def test_can_get_linux_x86_64_without_py_version(self): + got = Platform.from_string("linux_x86_64") + want = Platform(os=OS.linux, arch=Arch.x86_64) + self.assertEqual(want, got[0]) + + def test_can_get_specific_from_string(self): + got = Platform.from_string("cp33_linux_x86_64") + want = Platform(os=OS.linux, arch=Arch.x86_64, minor_version=3) + self.assertEqual(want, got[0]) + + def test_can_get_all_for_py_version(self): + cp39 = Platform.all(minor_version=9) + self.assertEqual(21, len(cp39), f"Got {cp39}") + self.assertEqual(cp39, Platform.from_string("cp39_*")) + + def test_can_get_all_for_os(self): + linuxes = Platform.all(OS.linux, minor_version=9) + self.assertEqual(7, len(linuxes)) + self.assertEqual(linuxes, Platform.from_string("cp39_linux_*")) + + def test_can_get_all_for_os_for_host_python(self): + linuxes = Platform.all(OS.linux) + self.assertEqual(7, len(linuxes)) + self.assertEqual(linuxes, Platform.from_string("linux_*")) + + def test_specific_version_specializations(self): + any_py33 = Platform(minor_version=3) + + # When + all_specializations = list(any_py33.all_specializations()) + + want = ( + [any_py33] + + [ + Platform(arch=arch, minor_version=any_py33.minor_version) + for arch in Arch + ] + + [Platform(os=os, minor_version=any_py33.minor_version) for os in OS] + + Platform.all(minor_version=any_py33.minor_version) + ) + self.assertEqual(want, all_specializations) + + def test_aarch64_specializations(self): + any_aarch64 = Platform(arch=Arch.aarch64) + all_specializations = list(any_aarch64.all_specializations()) + want = [ + Platform(os=None, arch=Arch.aarch64), + Platform(os=OS.linux, arch=Arch.aarch64), + Platform(os=OS.osx, arch=Arch.aarch64), + Platform(os=OS.windows, arch=Arch.aarch64), + ] + self.assertEqual(want, all_specializations) + + def test_linux_specializations(self): + any_linux = Platform(os=OS.linux) + all_specializations = list(any_linux.all_specializations()) + want = [ + Platform(os=OS.linux, arch=None), + Platform(os=OS.linux, arch=Arch.x86_64), + Platform(os=OS.linux, arch=Arch.x86_32), + Platform(os=OS.linux, arch=Arch.aarch64), + Platform(os=OS.linux, arch=Arch.ppc), + Platform(os=OS.linux, arch=Arch.ppc64le), + Platform(os=OS.linux, arch=Arch.s390x), + Platform(os=OS.linux, arch=Arch.arm), + ] + self.assertEqual(want, all_specializations) + + def test_osx_specializations(self): + any_osx = Platform(os=OS.osx) + all_specializations = list(any_osx.all_specializations()) + # NOTE @aignas 2024-01-14: even though in practice we would only have + # Python on osx aarch64 and osx x86_64, we return all arch posibilities + # to make the code simpler. + want = [ + Platform(os=OS.osx, arch=None), + Platform(os=OS.osx, arch=Arch.x86_64), + Platform(os=OS.osx, arch=Arch.x86_32), + Platform(os=OS.osx, arch=Arch.aarch64), + Platform(os=OS.osx, arch=Arch.ppc), + Platform(os=OS.osx, arch=Arch.ppc64le), + Platform(os=OS.osx, arch=Arch.s390x), + Platform(os=OS.osx, arch=Arch.arm), + ] + self.assertEqual(want, all_specializations) + + def test_platform_sort(self): + platforms = [ + Platform(os=OS.linux, arch=None), + Platform(os=OS.linux, arch=Arch.x86_64), + Platform(os=OS.osx, arch=None), + Platform(os=OS.osx, arch=Arch.x86_64), + Platform(os=OS.osx, arch=Arch.aarch64), + ] + shuffle(platforms) + platforms.sort() + want = [ + Platform(os=OS.linux, arch=None), + Platform(os=OS.linux, arch=Arch.x86_64), + Platform(os=OS.osx, arch=None), + Platform(os=OS.osx, arch=Arch.x86_64), + Platform(os=OS.osx, arch=Arch.aarch64), + ] + + self.assertEqual(want, platforms) + + def test_wheel_os_alias(self): + self.assertEqual("osx", str(OS.osx)) + self.assertEqual(str(OS.darwin), str(OS.osx)) + + def test_wheel_arch_alias(self): + self.assertEqual("x86_64", str(Arch.x86_64)) + self.assertEqual(str(Arch.amd64), str(Arch.x86_64)) + + def test_wheel_platform_alias(self): + give = Platform( + os=OS.darwin, + arch=Arch.amd64, + ) + alias = Platform( + os=OS.osx, + arch=Arch.x86_64, + ) + + self.assertEqual("osx_x86_64", str(give)) + self.assertEqual(str(alias), str(give)) + + +if __name__ == "__main__": + unittest.main() diff --git a/tests/pypi/whl_installer/wheel_installer_test.py b/tests/pypi/whl_installer/wheel_installer_test.py index 3c118af3c4..b736877e81 100644 --- a/tests/pypi/whl_installer/wheel_installer_test.py +++ b/tests/pypi/whl_installer/wheel_installer_test.py @@ -22,6 +22,39 @@ from python.private.pypi.whl_installer import wheel_installer +class TestRequirementExtrasParsing(unittest.TestCase): + def test_parses_requirement_for_extra(self) -> None: + cases = [ + ("name[foo]", ("name", frozenset(["foo"]))), + ("name[ Foo123 ]", ("name", frozenset(["Foo123"]))), + (" name1[ foo ] ", ("name1", frozenset(["foo"]))), + ("Name[foo]", ("name", frozenset(["foo"]))), + ("name_foo[bar]", ("name-foo", frozenset(["bar"]))), + ( + "name [fred,bar] @ http://foo.com ; python_version=='2.7'", + ("name", frozenset(["fred", "bar"])), + ), + ( + "name[quux, strange];python_version<'2.7' and platform_version=='2'", + ("name", frozenset(["quux", "strange"])), + ), + ( + "name; (os_name=='a' or os_name=='b') and os_name=='c'", + (None, None), + ), + ( + "name@http://foo.com", + (None, None), + ), + ] + + for case, expected in cases: + with self.subTest(): + self.assertTupleEqual( + wheel_installer._parse_requirement_for_extra(case), expected + ) + + class TestWhlFilegroup(unittest.TestCase): def setUp(self) -> None: self.wheel_name = "example_minimal_package-0.0.1-py3-none-any.whl" @@ -35,8 +68,10 @@ def tearDown(self): def test_wheel_exists(self) -> None: wheel_installer._extract_wheel( Path(self.wheel_path), - enable_implicit_namespace_pkgs=False, installation_dir=Path(self.wheel_dir), + extras={}, + enable_implicit_namespace_pkgs=False, + platforms=[], ) want_files = [ @@ -57,8 +92,12 @@ def test_wheel_exists(self) -> None: metadata_file_content = json.load(metadata_file) want = dict( + deps=[], + deps_by_platform={}, entry_points=[], + name="example-minimal-package", python_version="3.11.11", + version="0.0.1", ) self.assertEqual(want, metadata_file_content) diff --git a/tests/pypi/whl_installer/wheel_test.py b/tests/pypi/whl_installer/wheel_test.py new file mode 100644 index 0000000000..404218e12b --- /dev/null +++ b/tests/pypi/whl_installer/wheel_test.py @@ -0,0 +1,371 @@ +import unittest +from unittest import mock + +from python.private.pypi.whl_installer import wheel +from python.private.pypi.whl_installer.platform import OS, Arch, Platform + +_HOST_INTERPRETER_FN = ( + "python.private.pypi.whl_installer.wheel.host_interpreter_minor_version" +) + + +class DepsTest(unittest.TestCase): + def test_simple(self): + deps = wheel.Deps("foo", requires_dist=["bar"]) + + got = deps.build() + + self.assertIsInstance(got, wheel.FrozenDeps) + self.assertEqual(["bar"], got.deps) + self.assertEqual({}, got.deps_select) + + def test_can_add_os_specific_deps(self): + deps = wheel.Deps( + "foo", + requires_dist=[ + "bar", + "an_osx_dep; sys_platform=='darwin'", + "posix_dep; os_name=='posix'", + "win_dep; os_name=='nt'", + ], + platforms={ + Platform(os=OS.linux, arch=Arch.x86_64), + Platform(os=OS.osx, arch=Arch.x86_64), + Platform(os=OS.osx, arch=Arch.aarch64), + Platform(os=OS.windows, arch=Arch.x86_64), + }, + ) + + got = deps.build() + + self.assertEqual(["bar"], got.deps) + self.assertEqual( + { + "@platforms//os:linux": ["posix_dep"], + "@platforms//os:osx": ["an_osx_dep", "posix_dep"], + "@platforms//os:windows": ["win_dep"], + }, + got.deps_select, + ) + + def test_can_add_os_specific_deps_with_specific_python_version(self): + deps = wheel.Deps( + "foo", + requires_dist=[ + "bar", + "an_osx_dep; sys_platform=='darwin'", + "posix_dep; os_name=='posix'", + "win_dep; os_name=='nt'", + ], + platforms={ + Platform(os=OS.linux, arch=Arch.x86_64, minor_version=8), + Platform(os=OS.osx, arch=Arch.x86_64, minor_version=8), + Platform(os=OS.osx, arch=Arch.aarch64, minor_version=8), + Platform(os=OS.windows, arch=Arch.x86_64, minor_version=8), + }, + ) + + got = deps.build() + + self.assertEqual(["bar"], got.deps) + self.assertEqual( + { + "@platforms//os:linux": ["posix_dep"], + "@platforms//os:osx": ["an_osx_dep", "posix_dep"], + "@platforms//os:windows": ["win_dep"], + }, + got.deps_select, + ) + + def test_deps_are_added_to_more_specialized_platforms(self): + got = wheel.Deps( + "foo", + requires_dist=[ + "m1_dep; sys_platform=='darwin' and platform_machine=='arm64'", + "mac_dep; sys_platform=='darwin'", + ], + platforms={ + Platform(os=OS.osx, arch=Arch.x86_64), + Platform(os=OS.osx, arch=Arch.aarch64), + }, + ).build() + + self.assertEqual( + wheel.FrozenDeps( + deps=[], + deps_select={ + "osx_aarch64": ["m1_dep", "mac_dep"], + "@platforms//os:osx": ["mac_dep"], + }, + ), + got, + ) + + def test_deps_from_more_specialized_platforms_are_propagated(self): + got = wheel.Deps( + "foo", + requires_dist=[ + "a_mac_dep; sys_platform=='darwin'", + "m1_dep; sys_platform=='darwin' and platform_machine=='arm64'", + ], + platforms={ + Platform(os=OS.osx, arch=Arch.x86_64), + Platform(os=OS.osx, arch=Arch.aarch64), + }, + ).build() + + self.assertEqual([], got.deps) + self.assertEqual( + { + "osx_aarch64": ["a_mac_dep", "m1_dep"], + "@platforms//os:osx": ["a_mac_dep"], + }, + got.deps_select, + ) + + def test_non_platform_markers_are_added_to_common_deps(self): + got = wheel.Deps( + "foo", + requires_dist=[ + "bar", + "baz; implementation_name=='cpython'", + "m1_dep; sys_platform=='darwin' and platform_machine=='arm64'", + ], + platforms={ + Platform(os=OS.linux, arch=Arch.x86_64), + Platform(os=OS.osx, arch=Arch.x86_64), + Platform(os=OS.osx, arch=Arch.aarch64), + Platform(os=OS.windows, arch=Arch.x86_64), + }, + ).build() + + self.assertEqual(["bar", "baz"], got.deps) + self.assertEqual( + { + "osx_aarch64": ["m1_dep"], + }, + got.deps_select, + ) + + def test_self_is_ignored(self): + deps = wheel.Deps( + "foo", + requires_dist=[ + "bar", + "req_dep; extra == 'requests'", + "foo[requests]; extra == 'ssl'", + "ssl_lib; extra == 'ssl'", + ], + extras={"ssl"}, + ) + + got = deps.build() + + self.assertEqual(["bar", "req_dep", "ssl_lib"], got.deps) + self.assertEqual({}, got.deps_select) + + def test_self_dependencies_can_come_in_any_order(self): + deps = wheel.Deps( + "foo", + requires_dist=[ + "bar", + "baz; extra == 'feat'", + "foo[feat2]; extra == 'all'", + "foo[feat]; extra == 'feat2'", + "zdep; extra == 'all'", + ], + extras={"all"}, + ) + + got = deps.build() + + self.assertEqual(["bar", "baz", "zdep"], got.deps) + self.assertEqual({}, got.deps_select) + + def test_can_get_deps_based_on_specific_python_version(self): + requires_dist = [ + "bar", + "baz; python_version < '3.8'", + "posix_dep; os_name=='posix' and python_version >= '3.8'", + ] + + py38_deps = wheel.Deps( + "foo", + requires_dist=requires_dist, + platforms=[ + Platform(os=OS.linux, arch=Arch.x86_64, minor_version=8), + ], + ).build() + py37_deps = wheel.Deps( + "foo", + requires_dist=requires_dist, + platforms=[ + Platform(os=OS.linux, arch=Arch.x86_64, minor_version=7), + ], + ).build() + + self.assertEqual(["bar", "baz"], py37_deps.deps) + self.assertEqual({}, py37_deps.deps_select) + self.assertEqual(["bar"], py38_deps.deps) + self.assertEqual({"@platforms//os:linux": ["posix_dep"]}, py38_deps.deps_select) + + @mock.patch(_HOST_INTERPRETER_FN) + def test_no_version_select_when_single_version(self, mock_host_interpreter_version): + requires_dist = [ + "bar", + "baz; python_version >= '3.8'", + "posix_dep; os_name=='posix'", + "posix_dep_with_version; os_name=='posix' and python_version >= '3.8'", + "arch_dep; platform_machine=='x86_64' and python_version >= '3.8'", + ] + mock_host_interpreter_version.return_value = 7 + + self.maxDiff = None + + deps = wheel.Deps( + "foo", + requires_dist=requires_dist, + platforms=[ + Platform(os=os, arch=Arch.x86_64, minor_version=minor) + for minor in [8] + for os in [OS.linux, OS.windows] + ], + ) + got = deps.build() + + self.assertEqual(["bar", "baz"], got.deps) + self.assertEqual( + { + "@platforms//os:linux": ["posix_dep", "posix_dep_with_version"], + "linux_x86_64": ["arch_dep", "posix_dep", "posix_dep_with_version"], + "windows_x86_64": ["arch_dep"], + }, + got.deps_select, + ) + + @mock.patch(_HOST_INTERPRETER_FN) + def test_can_get_version_select(self, mock_host_interpreter_version): + requires_dist = [ + "bar", + "baz; python_version < '3.8'", + "baz_new; python_version >= '3.8'", + "posix_dep; os_name=='posix'", + "posix_dep_with_version; os_name=='posix' and python_version >= '3.8'", + "arch_dep; platform_machine=='x86_64' and python_version < '3.8'", + ] + mock_host_interpreter_version.return_value = 7 + + self.maxDiff = None + + deps = wheel.Deps( + "foo", + requires_dist=requires_dist, + platforms=[ + Platform(os=os, arch=Arch.x86_64, minor_version=minor) + for minor in [7, 8, 9] + for os in [OS.linux, OS.windows] + ], + ) + got = deps.build() + + self.assertEqual(["bar"], got.deps) + self.assertEqual( + { + "//conditions:default": ["baz"], + "@//python/config_settings:is_python_3.7": ["baz"], + "@//python/config_settings:is_python_3.8": ["baz_new"], + "@//python/config_settings:is_python_3.9": ["baz_new"], + "@platforms//os:linux": ["baz", "posix_dep"], + "cp37_linux_x86_64": ["arch_dep", "baz", "posix_dep"], + "cp37_windows_x86_64": ["arch_dep", "baz"], + "cp37_linux_anyarch": ["baz", "posix_dep"], + "cp38_linux_anyarch": [ + "baz_new", + "posix_dep", + "posix_dep_with_version", + ], + "cp39_linux_anyarch": [ + "baz_new", + "posix_dep", + "posix_dep_with_version", + ], + "linux_x86_64": ["arch_dep", "baz", "posix_dep"], + "windows_x86_64": ["arch_dep", "baz"], + }, + got.deps_select, + ) + + @mock.patch(_HOST_INTERPRETER_FN) + def test_deps_spanning_all_target_py_versions_are_added_to_common( + self, mock_host_version + ): + requires_dist = [ + "bar", + "baz (<2,>=1.11) ; python_version < '3.8'", + "baz (<2,>=1.14) ; python_version >= '3.8'", + ] + mock_host_version.return_value = 8 + + deps = wheel.Deps( + "foo", + requires_dist=requires_dist, + platforms=Platform.from_string(["cp37_*", "cp38_*", "cp39_*"]), + ) + got = deps.build() + + self.assertEqual(["bar", "baz"], got.deps) + self.assertEqual({}, got.deps_select) + + @mock.patch(_HOST_INTERPRETER_FN) + def test_deps_are_not_duplicated(self, mock_host_version): + mock_host_version.return_value = 7 + + # See an example in + # https://files.pythonhosted.org/packages/76/9e/db1c2d56c04b97981c06663384f45f28950a73d9acf840c4006d60d0a1ff/opencv_python-4.9.0.80-cp37-abi3-win32.whl.metadata + requires_dist = [ + "bar >=0.1.0 ; python_version < '3.7'", + "bar >=0.2.0 ; python_version >= '3.7'", + "bar >=0.4.0 ; python_version >= '3.6' and platform_system == 'Linux' and platform_machine == 'aarch64'", + "bar >=0.4.0 ; python_version >= '3.9'", + "bar >=0.5.0 ; python_version <= '3.9' and platform_system == 'Darwin' and platform_machine == 'arm64'", + "bar >=0.5.0 ; python_version >= '3.10' and platform_system == 'Darwin'", + "bar >=0.5.0 ; python_version >= '3.10'", + "bar >=0.6.0 ; python_version >= '3.11'", + ] + + deps = wheel.Deps( + "foo", + requires_dist=requires_dist, + platforms=Platform.from_string(["cp37_*", "cp310_*"]), + ) + got = deps.build() + + self.assertEqual(["bar"], got.deps) + self.assertEqual({}, got.deps_select) + + @mock.patch(_HOST_INTERPRETER_FN) + def test_deps_are_not_duplicated_when_encountering_platform_dep_first( + self, mock_host_version + ): + mock_host_version.return_value = 7 + + # Note, that we are sorting the incoming `requires_dist` and we need to ensure that we are not getting any + # issues even if the platform-specific line comes first. + requires_dist = [ + "bar >=0.4.0 ; python_version >= '3.6' and platform_system == 'Linux' and platform_machine == 'aarch64'", + "bar >=0.5.0 ; python_version >= '3.9'", + ] + + deps = wheel.Deps( + "foo", + requires_dist=requires_dist, + platforms=Platform.from_string(["cp37_*", "cp310_*"]), + ) + got = deps.build() + + self.assertEqual(["bar"], got.deps) + self.assertEqual({}, got.deps_select) + + +if __name__ == "__main__": + unittest.main() From 9e613d58cecda3f370698f37f7ca26bf38486db3 Mon Sep 17 00:00:00 2001 From: Ignas Anikevicius <240938+aignas@users.noreply.github.com> Date: Mon, 28 Apr 2025 18:44:32 +0900 Subject: [PATCH 196/922] fix(pypi) backport python_full_version fix to Python (#2833) Handling of `python_full_version` correctly has been fixed in the Starlark implementation in #2793 and in this PR I am backporting the changes to handle the full python version target platform strings so that we can have the same behaviour for now. At the same time I have simplified and got rid of the specialization handling in the Python algorithm just like I did in the starlark, which simplifies the tests and makes the algorithm more correct. Summary: * Handle `cp3x.y_os_arch` strings in the `platform.py` * Produce correct strings when the `micro_version` is unset. Note, that we use version `0` in evaluating but we use the default version in the config setting. This is to keep compatibility with the current behaviour when the target platform is not fully specified (which would be the case for WORKSPACE users). * Adjust the tests and the code to be more similar to the starlark impl. Work towards #2830 --- python/private/pypi/whl_installer/platform.py | 90 ++++---- python/private/pypi/whl_installer/wheel.py | 140 +++-------- tests/pypi/whl_installer/platform_test.py | 73 +----- tests/pypi/whl_installer/wheel_test.py | 218 ++++++++---------- 4 files changed, 185 insertions(+), 336 deletions(-) diff --git a/python/private/pypi/whl_installer/platform.py b/python/private/pypi/whl_installer/platform.py index 11dd6e37ab..ff267fe4aa 100644 --- a/python/private/pypi/whl_installer/platform.py +++ b/python/private/pypi/whl_installer/platform.py @@ -18,7 +18,7 @@ import sys from dataclasses import dataclass from enum import Enum -from typing import Any, Dict, Iterator, List, Optional, Union +from typing import Any, Dict, Iterator, List, Optional, Tuple, Union class OS(Enum): @@ -77,8 +77,8 @@ def _as_int(value: Optional[Union[OS, Arch]]) -> int: return int(value.value) -def host_interpreter_minor_version() -> int: - return sys.version_info.minor +def host_interpreter_version() -> Tuple[int, int]: + return (sys.version_info.minor, sys.version_info.micro) @dataclass(frozen=True) @@ -86,16 +86,23 @@ class Platform: os: Optional[OS] = None arch: Optional[Arch] = None minor_version: Optional[int] = None + micro_version: Optional[int] = None @classmethod def all( cls, want_os: Optional[OS] = None, minor_version: Optional[int] = None, + micro_version: Optional[int] = None, ) -> List["Platform"]: return sorted( [ - cls(os=os, arch=arch, minor_version=minor_version) + cls( + os=os, + arch=arch, + minor_version=minor_version, + micro_version=micro_version, + ) for os in OS for arch in Arch if not want_os or want_os == os @@ -112,32 +119,16 @@ def host(cls) -> List["Platform"]: A list of parsed values which makes the signature the same as `Platform.all` and `Platform.from_string`. """ + minor, micro = host_interpreter_version() return [ Platform( os=OS.interpreter(), arch=Arch.interpreter(), - minor_version=host_interpreter_minor_version(), + minor_version=minor, + micro_version=micro, ) ] - def all_specializations(self) -> Iterator["Platform"]: - """Return the platform itself and all its unambiguous specializations. - - For more info about specializations see - https://bazel.build/docs/configurable-attributes - """ - yield self - if self.arch is None: - for arch in Arch: - yield Platform(os=self.os, arch=arch, minor_version=self.minor_version) - if self.os is None: - for os in OS: - yield Platform(os=os, arch=self.arch, minor_version=self.minor_version) - if self.arch is None and self.os is None: - for os in OS: - for arch in Arch: - yield Platform(os=os, arch=arch, minor_version=self.minor_version) - def __lt__(self, other: Any) -> bool: """Add a comparison method, so that `sorted` returns the most specialized platforms first.""" if not isinstance(other, Platform) or other is None: @@ -153,24 +144,15 @@ def __lt__(self, other: Any) -> bool: def __str__(self) -> str: if self.minor_version is None: - if self.os is None and self.arch is None: - return "//conditions:default" - - if self.arch is None: - return f"@platforms//os:{self.os}" - else: - return f"{self.os}_{self.arch}" - - if self.arch is None and self.os is None: - return f"@//python/config_settings:is_python_3.{self.minor_version}" + return f"{self.os}_{self.arch}" - if self.arch is None: - return f"cp3{self.minor_version}_{self.os}_anyarch" + minor_version = self.minor_version + micro_version = self.micro_version - if self.os is None: - return f"cp3{self.minor_version}_anyos_{self.arch}" - - return f"cp3{self.minor_version}_{self.os}_{self.arch}" + if micro_version is None: + return f"cp3{minor_version}_{self.os}_{self.arch}" + else: + return f"cp3{minor_version}.{micro_version}_{self.os}_{self.arch}" @classmethod def from_string(cls, platform: Union[str, List[str]]) -> List["Platform"]: @@ -190,7 +172,17 @@ def from_string(cls, platform: Union[str, List[str]]) -> List["Platform"]: os, _, arch = tail.partition("_") arch = arch or "*" - minor_version = int(abi[len("cp3") :]) if abi else None + if abi: + tail = abi[len("cp3") :] + minor_version, _, micro_version = tail.partition(".") + minor_version = int(minor_version) + if micro_version == "": + micro_version = None + else: + micro_version = int(micro_version) + else: + minor_version = None + micro_version = None if arch != "*": ret.add( @@ -198,6 +190,7 @@ def from_string(cls, platform: Union[str, List[str]]) -> List["Platform"]: os=OS[os] if os != "*" else None, arch=Arch[arch], minor_version=minor_version, + micro_version=micro_version, ) ) @@ -206,6 +199,7 @@ def from_string(cls, platform: Union[str, List[str]]) -> List["Platform"]: cls.all( want_os=OS[os] if os != "*" else None, minor_version=minor_version, + micro_version=micro_version, ) ) @@ -282,7 +276,12 @@ def platform_machine(self) -> str: def env_markers(self, extra: str) -> Dict[str, str]: # If it is None, use the host version - minor_version = self.minor_version or host_interpreter_minor_version() + if self.minor_version is None: + minor, micro = host_interpreter_version() + else: + minor, micro = self.minor_version, self.micro_version + + micro = micro or 0 return { "extra": extra, @@ -292,12 +291,9 @@ def env_markers(self, extra: str) -> Dict[str, str]: "platform_system": self.platform_system, "platform_release": "", # unset "platform_version": "", # unset - "python_version": f"3.{minor_version}", - # FIXME @aignas 2024-01-14: is putting zero last a good idea? Maybe we should - # use `20` or something else to avoid having weird issues where the full version is used for - # matching and the author decides to only support 3.y.5 upwards. - "implementation_version": f"3.{minor_version}.0", - "python_full_version": f"3.{minor_version}.0", + "python_version": f"3.{minor}", + "implementation_version": f"3.{minor}.{micro}", + "python_full_version": f"3.{minor}.{micro}", # we assume that the following are the same as the interpreter used to setup the deps: # "implementation_name": "cpython" # "platform_python_implementation: "CPython", diff --git a/python/private/pypi/whl_installer/wheel.py b/python/private/pypi/whl_installer/wheel.py index d95b33a194..fce706acfb 100644 --- a/python/private/pypi/whl_installer/wheel.py +++ b/python/private/pypi/whl_installer/wheel.py @@ -27,7 +27,7 @@ from python.private.pypi.whl_installer.platform import ( Platform, - host_interpreter_minor_version, + host_interpreter_version, ) @@ -62,12 +62,13 @@ def __init__( """ self.name: str = Deps._normalize(name) self._platforms: Set[Platform] = platforms or set() - self._target_versions = {p.minor_version for p in platforms or {}} - self._default_minor_version = None - if platforms and len(self._target_versions) > 2: + self._target_versions = {(p.minor_version, p.micro_version) for p in platforms or {}} + if platforms and len(self._target_versions) > 1: # TODO @aignas 2024-06-23: enable this to be set via a CLI arg # for being more explicit. - self._default_minor_version = host_interpreter_minor_version() + self._default_minor_version, _ = host_interpreter_version() + else: + self._default_minor_version = None if None in self._target_versions and len(self._target_versions) > 2: raise ValueError( @@ -88,8 +89,13 @@ def __init__( # Then add all of the requirements in order self._deps: Set[str] = set() self._select: Dict[Platform, Set[str]] = defaultdict(set) + + reqs_by_name = {} for req in reqs: - self._add_req(req, want_extras) + reqs_by_name.setdefault(req.name, []).append(req) + + for reqs in reqs_by_name.values(): + self._add_req(reqs, want_extras) def _add(self, dep: str, platform: Optional[Platform]): dep = Deps._normalize(dep) @@ -123,50 +129,6 @@ def _add(self, dep: str, platform: Optional[Platform]): # Add the platform-specific dep self._select[platform].add(dep) - # Add the dep to specializations of the given platform if they - # exist in the select statement. - for p in platform.all_specializations(): - if p not in self._select: - continue - - self._select[p].add(dep) - - if len(self._select[platform]) == 1: - # We are adding a new item to the select and we need to ensure that - # existing dependencies from less specialized platforms are propagated - # to the newly added dependency set. - for p, deps in self._select.items(): - # Check if the existing platform overlaps with the given platform - if p == platform or platform not in p.all_specializations(): - continue - - self._select[platform].update(self._select[p]) - - def _maybe_add_common_dep(self, dep): - if len(self._target_versions) < 2: - return - - platforms = [Platform()] + [ - Platform(minor_version=v) for v in self._target_versions - ] - - # If the dep is targeting all target python versions, lets add it to - # the common dependency list to simplify the select statements. - for p in platforms: - if p not in self._select: - return - - if dep not in self._select[p]: - return - - # All of the python version-specific branches have the dep, so lets add - # it to the common deps. - self._deps.add(dep) - for p in platforms: - self._select[p].remove(dep) - if not self._select[p]: - self._select.pop(p) - @staticmethod def _normalize(name: str) -> str: return re.sub(r"[-_.]+", "_", name).lower() @@ -227,66 +189,40 @@ def _resolve_extras( return extras - def _add_req(self, req: Requirement, extras: Set[str]) -> None: - if req.marker is None: - self._add(req.name, None) - return + def _add_req(self, reqs: List[Requirement], extras: Set[str]) -> None: + platforms_to_add = set() + for req in reqs: + if req.marker is None: + self._add(req.name, None) + return - marker_str = str(req.marker) + for plat in self._platforms: + if plat in platforms_to_add: + # marker evaluation is more expensive than this check + continue - if not self._platforms: - if any(req.marker.evaluate({"extra": extra}) for extra in extras): - self._add(req.name, None) - return + added = False + for extra in extras: + if added: + break - # NOTE @aignas 2023-12-08: in order to have reasonable select statements - # we do have to have some parsing of the markers, so it begs the question - # if packaging should be reimplemented in Starlark to have the best solution - # for now we will implement it in Python and see what the best parsing result - # can be before making this decision. - match_os = any( - tag in marker_str - for tag in [ - "os_name", - "sys_platform", - "platform_system", - ] - ) - match_arch = "platform_machine" in marker_str - match_version = "version" in marker_str + if req.marker.evaluate(plat.env_markers(extra)): + platforms_to_add.add(plat) + added = True + break - if not (match_os or match_arch or match_version): - if any(req.marker.evaluate({"extra": extra}) for extra in extras): - self._add(req.name, None) + if len(platforms_to_add) == len(self._platforms): + # the dep is in all target platforms, let's just add it to the regular + # list + self._add(req.name, None) return - for plat in self._platforms: - if not any( - req.marker.evaluate(plat.env_markers(extra)) for extra in extras - ): - continue - - if match_arch and self._default_minor_version: + for plat in platforms_to_add: + if self._default_minor_version is not None: self._add(req.name, plat) - if plat.minor_version == self._default_minor_version: - self._add(req.name, Platform(plat.os, plat.arch)) - elif match_arch: - self._add(req.name, Platform(plat.os, plat.arch)) - elif match_os and self._default_minor_version: - self._add(req.name, Platform(plat.os, minor_version=plat.minor_version)) - if plat.minor_version == self._default_minor_version: - self._add(req.name, Platform(plat.os)) - elif match_os: - self._add(req.name, Platform(plat.os)) - elif match_version and self._default_minor_version: - self._add(req.name, Platform(minor_version=plat.minor_version)) - if plat.minor_version == self._default_minor_version: - self._add(req.name, Platform()) - elif match_version: - self._add(req.name, None) - # Merge to common if possible after processing all platforms - self._maybe_add_common_dep(req.name) + if self._default_minor_version is None or plat.minor_version == self._default_minor_version: + self._add(req.name, Platform(os = plat.os, arch = plat.arch)) def build(self) -> FrozenDeps: return FrozenDeps( diff --git a/tests/pypi/whl_installer/platform_test.py b/tests/pypi/whl_installer/platform_test.py index 2aeb4caa69..ad65650779 100644 --- a/tests/pypi/whl_installer/platform_test.py +++ b/tests/pypi/whl_installer/platform_test.py @@ -5,13 +5,13 @@ OS, Arch, Platform, - host_interpreter_minor_version, + host_interpreter_version, ) class MinorVersionTest(unittest.TestCase): def test_host(self): - host = host_interpreter_minor_version() + host = host_interpreter_version() self.assertIsNotNone(host) @@ -32,10 +32,14 @@ def test_can_get_specific_from_string(self): want = Platform(os=OS.linux, arch=Arch.x86_64, minor_version=3) self.assertEqual(want, got[0]) + got = Platform.from_string("cp33.0_linux_x86_64") + want = Platform(os=OS.linux, arch=Arch.x86_64, minor_version=3, micro_version=0) + self.assertEqual(want, got[0]) + def test_can_get_all_for_py_version(self): - cp39 = Platform.all(minor_version=9) + cp39 = Platform.all(minor_version=9, micro_version=0) self.assertEqual(21, len(cp39), f"Got {cp39}") - self.assertEqual(cp39, Platform.from_string("cp39_*")) + self.assertEqual(cp39, Platform.from_string("cp39.0_*")) def test_can_get_all_for_os(self): linuxes = Platform.all(OS.linux, minor_version=9) @@ -47,67 +51,6 @@ def test_can_get_all_for_os_for_host_python(self): self.assertEqual(7, len(linuxes)) self.assertEqual(linuxes, Platform.from_string("linux_*")) - def test_specific_version_specializations(self): - any_py33 = Platform(minor_version=3) - - # When - all_specializations = list(any_py33.all_specializations()) - - want = ( - [any_py33] - + [ - Platform(arch=arch, minor_version=any_py33.minor_version) - for arch in Arch - ] - + [Platform(os=os, minor_version=any_py33.minor_version) for os in OS] - + Platform.all(minor_version=any_py33.minor_version) - ) - self.assertEqual(want, all_specializations) - - def test_aarch64_specializations(self): - any_aarch64 = Platform(arch=Arch.aarch64) - all_specializations = list(any_aarch64.all_specializations()) - want = [ - Platform(os=None, arch=Arch.aarch64), - Platform(os=OS.linux, arch=Arch.aarch64), - Platform(os=OS.osx, arch=Arch.aarch64), - Platform(os=OS.windows, arch=Arch.aarch64), - ] - self.assertEqual(want, all_specializations) - - def test_linux_specializations(self): - any_linux = Platform(os=OS.linux) - all_specializations = list(any_linux.all_specializations()) - want = [ - Platform(os=OS.linux, arch=None), - Platform(os=OS.linux, arch=Arch.x86_64), - Platform(os=OS.linux, arch=Arch.x86_32), - Platform(os=OS.linux, arch=Arch.aarch64), - Platform(os=OS.linux, arch=Arch.ppc), - Platform(os=OS.linux, arch=Arch.ppc64le), - Platform(os=OS.linux, arch=Arch.s390x), - Platform(os=OS.linux, arch=Arch.arm), - ] - self.assertEqual(want, all_specializations) - - def test_osx_specializations(self): - any_osx = Platform(os=OS.osx) - all_specializations = list(any_osx.all_specializations()) - # NOTE @aignas 2024-01-14: even though in practice we would only have - # Python on osx aarch64 and osx x86_64, we return all arch posibilities - # to make the code simpler. - want = [ - Platform(os=OS.osx, arch=None), - Platform(os=OS.osx, arch=Arch.x86_64), - Platform(os=OS.osx, arch=Arch.x86_32), - Platform(os=OS.osx, arch=Arch.aarch64), - Platform(os=OS.osx, arch=Arch.ppc), - Platform(os=OS.osx, arch=Arch.ppc64le), - Platform(os=OS.osx, arch=Arch.s390x), - Platform(os=OS.osx, arch=Arch.arm), - ] - self.assertEqual(want, all_specializations) - def test_platform_sort(self): platforms = [ Platform(os=OS.linux, arch=None), diff --git a/tests/pypi/whl_installer/wheel_test.py b/tests/pypi/whl_installer/wheel_test.py index 404218e12b..6921fe6d3f 100644 --- a/tests/pypi/whl_installer/wheel_test.py +++ b/tests/pypi/whl_installer/wheel_test.py @@ -5,7 +5,7 @@ from python.private.pypi.whl_installer.platform import OS, Arch, Platform _HOST_INTERPRETER_FN = ( - "python.private.pypi.whl_installer.wheel.host_interpreter_minor_version" + "python.private.pypi.whl_installer.wheel.host_interpreter_version" ) @@ -20,108 +20,56 @@ def test_simple(self): self.assertEqual({}, got.deps_select) def test_can_add_os_specific_deps(self): - deps = wheel.Deps( - "foo", - requires_dist=[ - "bar", - "an_osx_dep; sys_platform=='darwin'", - "posix_dep; os_name=='posix'", - "win_dep; os_name=='nt'", - ], - platforms={ + for platforms in [ + { Platform(os=OS.linux, arch=Arch.x86_64), Platform(os=OS.osx, arch=Arch.x86_64), Platform(os=OS.osx, arch=Arch.aarch64), Platform(os=OS.windows, arch=Arch.x86_64), }, - ) - - got = deps.build() - - self.assertEqual(["bar"], got.deps) - self.assertEqual( { - "@platforms//os:linux": ["posix_dep"], - "@platforms//os:osx": ["an_osx_dep", "posix_dep"], - "@platforms//os:windows": ["win_dep"], - }, - got.deps_select, - ) - - def test_can_add_os_specific_deps_with_specific_python_version(self): - deps = wheel.Deps( - "foo", - requires_dist=[ - "bar", - "an_osx_dep; sys_platform=='darwin'", - "posix_dep; os_name=='posix'", - "win_dep; os_name=='nt'", - ], - platforms={ Platform(os=OS.linux, arch=Arch.x86_64, minor_version=8), Platform(os=OS.osx, arch=Arch.x86_64, minor_version=8), Platform(os=OS.osx, arch=Arch.aarch64, minor_version=8), Platform(os=OS.windows, arch=Arch.x86_64, minor_version=8), }, - ) - - got = deps.build() - - self.assertEqual(["bar"], got.deps) - self.assertEqual( { - "@platforms//os:linux": ["posix_dep"], - "@platforms//os:osx": ["an_osx_dep", "posix_dep"], - "@platforms//os:windows": ["win_dep"], - }, - got.deps_select, - ) - - def test_deps_are_added_to_more_specialized_platforms(self): - got = wheel.Deps( - "foo", - requires_dist=[ - "m1_dep; sys_platform=='darwin' and platform_machine=='arm64'", - "mac_dep; sys_platform=='darwin'", - ], - platforms={ - Platform(os=OS.osx, arch=Arch.x86_64), - Platform(os=OS.osx, arch=Arch.aarch64), + Platform( + os=OS.linux, arch=Arch.x86_64, minor_version=8, micro_version=1 + ), + Platform(os=OS.osx, arch=Arch.x86_64, minor_version=8, micro_version=1), + Platform( + os=OS.osx, arch=Arch.aarch64, minor_version=8, micro_version=1 + ), + Platform( + os=OS.windows, arch=Arch.x86_64, minor_version=8, micro_version=1 + ), }, - ).build() - - self.assertEqual( - wheel.FrozenDeps( - deps=[], - deps_select={ - "osx_aarch64": ["m1_dep", "mac_dep"], - "@platforms//os:osx": ["mac_dep"], - }, - ), - got, - ) - - def test_deps_from_more_specialized_platforms_are_propagated(self): - got = wheel.Deps( - "foo", - requires_dist=[ - "a_mac_dep; sys_platform=='darwin'", - "m1_dep; sys_platform=='darwin' and platform_machine=='arm64'", - ], - platforms={ - Platform(os=OS.osx, arch=Arch.x86_64), - Platform(os=OS.osx, arch=Arch.aarch64), - }, - ).build() - - self.assertEqual([], got.deps) - self.assertEqual( - { - "osx_aarch64": ["a_mac_dep", "m1_dep"], - "@platforms//os:osx": ["a_mac_dep"], - }, - got.deps_select, - ) + ]: + with self.subTest(): + deps = wheel.Deps( + "foo", + requires_dist=[ + "bar", + "an_osx_dep; sys_platform=='darwin'", + "posix_dep; os_name=='posix'", + "win_dep; os_name=='nt'", + ], + platforms=platforms, + ) + + got = deps.build() + + self.assertEqual(["bar"], got.deps) + self.assertEqual( + { + "linux_x86_64": ["posix_dep"], + "osx_aarch64": ["an_osx_dep", "posix_dep"], + "osx_x86_64": ["an_osx_dep", "posix_dep"], + "windows_x86_64": ["win_dep"], + }, + got.deps_select, + ) def test_non_platform_markers_are_added_to_common_deps(self): got = wheel.Deps( @@ -185,7 +133,7 @@ def test_self_dependencies_can_come_in_any_order(self): def test_can_get_deps_based_on_specific_python_version(self): requires_dist = [ "bar", - "baz; python_version < '3.8'", + "baz; python_full_version < '3.7.3'", "posix_dep; os_name=='posix' and python_version >= '3.8'", ] @@ -196,6 +144,15 @@ def test_can_get_deps_based_on_specific_python_version(self): Platform(os=OS.linux, arch=Arch.x86_64, minor_version=8), ], ).build() + py373_deps = wheel.Deps( + "foo", + requires_dist=requires_dist, + platforms=[ + Platform( + os=OS.linux, arch=Arch.x86_64, minor_version=7, micro_version=3 + ), + ], + ).build() py37_deps = wheel.Deps( "foo", requires_dist=requires_dist, @@ -206,11 +163,12 @@ def test_can_get_deps_based_on_specific_python_version(self): self.assertEqual(["bar", "baz"], py37_deps.deps) self.assertEqual({}, py37_deps.deps_select) - self.assertEqual(["bar"], py38_deps.deps) - self.assertEqual({"@platforms//os:linux": ["posix_dep"]}, py38_deps.deps_select) + self.assertEqual(["bar"], py373_deps.deps) + self.assertEqual({}, py37_deps.deps_select) + self.assertEqual(["bar", "posix_dep"], py38_deps.deps) + self.assertEqual({}, py38_deps.deps_select) - @mock.patch(_HOST_INTERPRETER_FN) - def test_no_version_select_when_single_version(self, mock_host_interpreter_version): + def test_no_version_select_when_single_version(self): requires_dist = [ "bar", "baz; python_version >= '3.8'", @@ -218,7 +176,6 @@ def test_no_version_select_when_single_version(self, mock_host_interpreter_versi "posix_dep_with_version; os_name=='posix' and python_version >= '3.8'", "arch_dep; platform_machine=='x86_64' and python_version >= '3.8'", ] - mock_host_interpreter_version.return_value = 7 self.maxDiff = None @@ -226,19 +183,19 @@ def test_no_version_select_when_single_version(self, mock_host_interpreter_versi "foo", requires_dist=requires_dist, platforms=[ - Platform(os=os, arch=Arch.x86_64, minor_version=minor) - for minor in [8] + Platform( + os=os, arch=Arch.x86_64, minor_version=minor, micro_version=micro + ) + for minor, micro in [(8, 4)] for os in [OS.linux, OS.windows] ], ) got = deps.build() - self.assertEqual(["bar", "baz"], got.deps) + self.assertEqual(["arch_dep", "bar", "baz"], got.deps) self.assertEqual( { - "@platforms//os:linux": ["posix_dep", "posix_dep_with_version"], - "linux_x86_64": ["arch_dep", "posix_dep", "posix_dep_with_version"], - "windows_x86_64": ["arch_dep"], + "linux_x86_64": ["posix_dep", "posix_dep_with_version"], }, got.deps_select, ) @@ -253,7 +210,7 @@ def test_can_get_version_select(self, mock_host_interpreter_version): "posix_dep_with_version; os_name=='posix' and python_version >= '3.8'", "arch_dep; platform_machine=='x86_64' and python_version < '3.8'", ] - mock_host_interpreter_version.return_value = 7 + mock_host_interpreter_version.return_value = (7, 4) self.maxDiff = None @@ -261,8 +218,10 @@ def test_can_get_version_select(self, mock_host_interpreter_version): "foo", requires_dist=requires_dist, platforms=[ - Platform(os=os, arch=Arch.x86_64, minor_version=minor) - for minor in [7, 8, 9] + Platform( + os=os, arch=Arch.x86_64, minor_version=minor, micro_version=micro + ) + for minor, micro in [(7, 4), (8, 8), (9, 8)] for os in [OS.linux, OS.windows] ], ) @@ -271,24 +230,20 @@ def test_can_get_version_select(self, mock_host_interpreter_version): self.assertEqual(["bar"], got.deps) self.assertEqual( { - "//conditions:default": ["baz"], - "@//python/config_settings:is_python_3.7": ["baz"], - "@//python/config_settings:is_python_3.8": ["baz_new"], - "@//python/config_settings:is_python_3.9": ["baz_new"], - "@platforms//os:linux": ["baz", "posix_dep"], - "cp37_linux_x86_64": ["arch_dep", "baz", "posix_dep"], - "cp37_windows_x86_64": ["arch_dep", "baz"], - "cp37_linux_anyarch": ["baz", "posix_dep"], - "cp38_linux_anyarch": [ + "cp37.4_linux_x86_64": ["arch_dep", "baz", "posix_dep"], + "cp37.4_windows_x86_64": ["arch_dep", "baz"], + "cp38.8_linux_x86_64": [ "baz_new", "posix_dep", "posix_dep_with_version", ], - "cp39_linux_anyarch": [ + "cp38.8_windows_x86_64": ["baz_new"], + "cp39.8_linux_x86_64": [ "baz_new", "posix_dep", "posix_dep_with_version", ], + "cp39.8_windows_x86_64": ["baz_new"], "linux_x86_64": ["arch_dep", "baz", "posix_dep"], "windows_x86_64": ["arch_dep", "baz"], }, @@ -304,7 +259,9 @@ def test_deps_spanning_all_target_py_versions_are_added_to_common( "baz (<2,>=1.11) ; python_version < '3.8'", "baz (<2,>=1.14) ; python_version >= '3.8'", ] - mock_host_version.return_value = 8 + mock_host_version.return_value = (8, 4) + + self.maxDiff = None deps = wheel.Deps( "foo", @@ -313,12 +270,12 @@ def test_deps_spanning_all_target_py_versions_are_added_to_common( ) got = deps.build() - self.assertEqual(["bar", "baz"], got.deps) self.assertEqual({}, got.deps_select) + self.assertEqual(["bar", "baz"], got.deps) @mock.patch(_HOST_INTERPRETER_FN) def test_deps_are_not_duplicated(self, mock_host_version): - mock_host_version.return_value = 7 + mock_host_version.return_value = (7, 4) # See an example in # https://files.pythonhosted.org/packages/76/9e/db1c2d56c04b97981c06663384f45f28950a73d9acf840c4006d60d0a1ff/opencv_python-4.9.0.80-cp37-abi3-win32.whl.metadata @@ -347,7 +304,7 @@ def test_deps_are_not_duplicated(self, mock_host_version): def test_deps_are_not_duplicated_when_encountering_platform_dep_first( self, mock_host_version ): - mock_host_version.return_value = 7 + mock_host_version.return_value = (7, 1) # Note, that we are sorting the incoming `requires_dist` and we need to ensure that we are not getting any # issues even if the platform-specific line comes first. @@ -356,15 +313,32 @@ def test_deps_are_not_duplicated_when_encountering_platform_dep_first( "bar >=0.5.0 ; python_version >= '3.9'", ] + self.maxDiff = None + deps = wheel.Deps( "foo", requires_dist=requires_dist, - platforms=Platform.from_string(["cp37_*", "cp310_*"]), + platforms=Platform.from_string( + [ + "cp37.1_linux_x86_64", + "cp37.1_linux_aarch64", + "cp310_linux_x86_64", + "cp310_linux_aarch64", + ] + ), ) got = deps.build() - self.assertEqual(["bar"], got.deps) - self.assertEqual({}, got.deps_select) + self.assertEqual([], got.deps) + self.assertEqual( + { + "cp310_linux_aarch64": ["bar"], + "cp310_linux_x86_64": ["bar"], + "cp37.1_linux_aarch64": ["bar"], + "linux_aarch64": ["bar"], + }, + got.deps_select, + ) if __name__ == "__main__": From 5b9d545220e5956e0686de91a14e6ded89df651a Mon Sep 17 00:00:00 2001 From: Ignas Anikevicius <240938+aignas@users.noreply.github.com> Date: Tue, 29 Apr 2025 05:37:37 +0900 Subject: [PATCH 197/922] revert(pypi): use Python for marker eval and METADATA parsing (#2834) Summary: - Revert to using Python for marker evaluation during parsing of requirements (partial revert of #2692). - Use Python to parse whl METADATA. - Bugfix the new simpler algorithm and add a new unit test. Fixes #2830 --- CHANGELOG.md | 9 -- python/private/pypi/evaluate_markers.bzl | 62 ++++++++++ python/private/pypi/extension.bzl | 42 ++++++- .../pypi/generate_whl_library_build_bazel.bzl | 35 ++++-- python/private/pypi/parse_requirements.bzl | 4 +- python/private/pypi/pip_repository.bzl | 40 +++---- python/private/pypi/whl_installer/wheel.py | 33 ++++-- python/private/pypi/whl_library.bzl | 59 ++++------ tests/pypi/extension/extension_tests.bzl | 110 ++++++++++++++++++ ...generate_whl_library_build_bazel_tests.bzl | 2 - .../parse_requirements_tests.bzl | 2 +- tests/pypi/whl_installer/wheel_test.py | 2 +- 12 files changed, 304 insertions(+), 96 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 8fc00ca25f..a8cac4c5cd 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -103,8 +103,6 @@ END_UNRELEASED_TEMPLATE * 3.12.9 * 3.13.2 * (pypi) Use `xcrun xcodebuild --showsdks` to find XCode root. -* (pypi) The `bzlmod` extension will now generate smaller lock files for when - using `experimental_index_url`. * (toolchains) Remove all but `3.8.20` versions of the Python `3.8` interpreter who has reached EOL. If users still need other versions of the `3.8` interpreter, please supply the URLs manually {bzl:obj}`python.toolchain` or {bzl:obj}`python_register_toolchains` calls. @@ -120,13 +118,6 @@ END_UNRELEASED_TEMPLATE [PR #2746](https://github.com/bazel-contrib/rules_python/pull/2746). * (rules) {attr}`py_binary.srcs` and {attr}`py_test.srcs` is no longer mandatory when `main_module` is specified (for `--bootstrap_impl=script`) -* (pypi) From now on the `Requires-Dist` from the wheel metadata is analysed in - the loading phase instead of repository rule phase giving better caching - performance when the target platforms are changed (e.g. target python - versions). This is preparatory work for stabilizing the cross-platform wheel - support. From now on the usage of `experimental_target_platforms` should be - avoided and the `requirements_by_platform` values should be instead used to - specify the target platforms for the given dependencies. [20250317]: https://github.com/astral-sh/python-build-standalone/releases/tag/20250317 diff --git a/python/private/pypi/evaluate_markers.bzl b/python/private/pypi/evaluate_markers.bzl index f966aa32be..191933596e 100644 --- a/python/private/pypi/evaluate_markers.bzl +++ b/python/private/pypi/evaluate_markers.bzl @@ -14,10 +14,21 @@ """A simple function that evaluates markers using a python interpreter.""" +load(":deps.bzl", "record_files") load(":pep508_env.bzl", "env") load(":pep508_evaluate.bzl", "evaluate") load(":pep508_platform.bzl", "platform_from_str") load(":pep508_requirement.bzl", "requirement") +load(":pypi_repo_utils.bzl", "pypi_repo_utils") + +# Used as a default value in a rule to ensure we fetch the dependencies. +SRCS = [ + # When the version, or any of the files in `packaging` package changes, + # this file will change as well. + record_files["pypi__packaging"], + Label("//python/private/pypi/requirements_parser:resolve_target_platforms.py"), + Label("//python/private/pypi/whl_installer:platform.py"), +] def evaluate_markers(requirements, python_version = None): """Return the list of supported platforms per requirements line. @@ -37,3 +48,54 @@ def evaluate_markers(requirements, python_version = None): ret.setdefault(req_string, []).append(platform) return ret + +def evaluate_markers_py(mrctx, *, requirements, python_interpreter, python_interpreter_target, srcs, logger = None): + """Return the list of supported platforms per requirements line. + + Args: + mrctx: repository_ctx or module_ctx. + requirements: list[str] of the requirement file lines to evaluate. + python_interpreter: str, path to the python_interpreter to use to + evaluate the env markers in the given requirements files. It will + be only called if the requirements files have env markers. This + should be something that is in your PATH or an absolute path. + python_interpreter_target: Label, same as python_interpreter, but in a + label format. + srcs: list[Label], the value of SRCS passed from the `rctx` or `mctx` to this function. + logger: repo_utils.logger or None, a simple struct to log diagnostic + messages. Defaults to None. + + Returns: + dict of string lists with target platforms + """ + if not requirements: + return {} + + in_file = mrctx.path("requirements_with_markers.in.json") + out_file = mrctx.path("requirements_with_markers.out.json") + mrctx.file(in_file, json.encode(requirements)) + + pypi_repo_utils.execute_checked( + mrctx, + op = "ResolveRequirementEnvMarkers({})".format(in_file), + python = pypi_repo_utils.resolve_python_interpreter( + mrctx, + python_interpreter = python_interpreter, + python_interpreter_target = python_interpreter_target, + ), + arguments = [ + "-m", + "python.private.pypi.requirements_parser.resolve_target_platforms", + in_file, + out_file, + ], + srcs = srcs, + environment = { + "PYTHONPATH": [ + Label("@pypi__packaging//:BUILD.bazel"), + Label("//:BUILD.bazel"), + ], + }, + logger = logger, + ) + return json.decode(mrctx.read(out_file)) diff --git a/python/private/pypi/extension.bzl b/python/private/pypi/extension.bzl index e9eba684f8..647407f16f 100644 --- a/python/private/pypi/extension.bzl +++ b/python/private/pypi/extension.bzl @@ -24,7 +24,7 @@ load("//python/private:repo_utils.bzl", "repo_utils") load("//python/private:semver.bzl", "semver") load("//python/private:version_label.bzl", "version_label") load(":attrs.bzl", "use_isolated") -load(":evaluate_markers.bzl", "evaluate_markers") +load(":evaluate_markers.bzl", "evaluate_markers_py", EVALUATE_MARKERS_SRCS = "SRCS") load(":hub_repository.bzl", "hub_repository", "whl_config_settings_to_json") load(":parse_requirements.bzl", "parse_requirements") load(":parse_whl_name.bzl", "parse_whl_name") @@ -71,6 +71,7 @@ def _create_whl_repos( whl_overrides, available_interpreters = INTERPRETER_LABELS, minor_mapping = MINOR_MAPPING, + evaluate_markers = evaluate_markers_py, get_index_urls = None): """create all of the whl repositories @@ -85,6 +86,7 @@ def _create_whl_repos( used during the `repository_rule` and must be always compatible with the host. minor_mapping: {type}`dict[str, str]` The dictionary needed to resolve the full python version used to parse package METADATA files. + evaluate_markers: the function used to evaluate the markers. Returns a {type}`struct` with the following attributes: whl_map: {type}`dict[str, list[struct]]` the output is keyed by the @@ -172,7 +174,28 @@ def _create_whl_repos( ), extra_pip_args = pip_attr.extra_pip_args, get_index_urls = get_index_urls, - evaluate_markers = evaluate_markers, + # NOTE @aignas 2024-08-02: , we will execute any interpreter that we find either + # in the PATH or if specified as a label. We will configure the env + # markers when evaluating the requirement lines based on the output + # from the `requirements_files_by_platform` which should have something + # similar to: + # { + # "//:requirements.txt": ["cp311_linux_x86_64", ...] + # } + # + # We know the target python versions that we need to evaluate the + # markers for and thus we don't need to use multiple python interpreter + # instances to perform this manipulation. This function should be executed + # only once by the underlying code to minimize the overhead needed to + # spin up a Python interpreter. + evaluate_markers = lambda module_ctx, requirements: evaluate_markers( + module_ctx, + requirements = requirements, + python_interpreter = pip_attr.python_interpreter, + python_interpreter_target = python_interpreter_target, + srcs = pip_attr._evaluate_markers_srcs, + logger = logger, + ), logger = logger, ) @@ -193,6 +216,7 @@ def _create_whl_repos( enable_implicit_namespace_pkgs = pip_attr.enable_implicit_namespace_pkgs, environment = pip_attr.environment, envsubst = pip_attr.envsubst, + experimental_target_platforms = pip_attr.experimental_target_platforms, group_deps = group_deps, group_name = group_name, pip_data_exclude = pip_attr.pip_data_exclude, @@ -281,6 +305,13 @@ def _whl_repos(*, requirement, whl_library_args, download_only, netrc, auth_patt args["urls"] = [distribution.url] args["sha256"] = distribution.sha256 args["filename"] = distribution.filename + args["experimental_target_platforms"] = [ + # Get rid of the version fot the target platforms because we are + # passing the interpreter any way. Ideally we should search of ways + # how to pass the target platforms through the hub repo. + p.partition("_")[2] + for p in requirement.target_platforms + ] # Pure python wheels or sdists may need to have a platform here target_platforms = None @@ -775,6 +806,13 @@ EXPERIMENTAL: this may be removed without notice. doc = """\ A dict of labels to wheel names that is typically generated by the whl_modifications. The labels are JSON config files describing the modifications. +""", + ), + "_evaluate_markers_srcs": attr.label_list( + default = EVALUATE_MARKERS_SRCS, + doc = """\ +The list of labels to use as SRCS for the marker evaluation code. This ensures that the +code will be re-evaluated when any of files in the default changes. """, ), }, **ATTRS) diff --git a/python/private/pypi/generate_whl_library_build_bazel.bzl b/python/private/pypi/generate_whl_library_build_bazel.bzl index 7988aca1c4..31c9d4da60 100644 --- a/python/private/pypi/generate_whl_library_build_bazel.bzl +++ b/python/private/pypi/generate_whl_library_build_bazel.bzl @@ -21,11 +21,14 @@ _RENDER = { "copy_files": render.dict, "data": render.list, "data_exclude": render.list, + "dependencies": render.list, + "dependencies_by_platform": lambda x: render.dict(x, value_repr = render.list), "entry_points": render.dict, "extras": render.list, "group_deps": render.list, "requires_dist": render.list, "srcs_exclude": render.list, + "tags": render.list, "target_platforms": lambda x: render.list(x) if x else "target_platforms", } @@ -37,7 +40,7 @@ _TEMPLATE = """\ package(default_visibility = ["//visibility:public"]) -whl_library_targets_from_requires( +{fn}( {kwargs} ) """ @@ -59,17 +62,28 @@ def generate_whl_library_build_bazel( A complete BUILD file as a string """ + fn = "whl_library_targets" + if kwargs.get("tags"): + # legacy path + unsupported_args = [ + "requires", + "metadata_name", + "metadata_version", + ] + else: + fn = "{}_from_requires".format(fn) + unsupported_args = [ + "dependencies", + "dependencies_by_platform", + ] + + for arg in unsupported_args: + if kwargs.get(arg): + fail("BUG, unsupported arg: '{}'".format(arg)) + loads = [ - """load("@rules_python//python/private/pypi:whl_library_targets.bzl", "whl_library_targets_from_requires")""", + """load("@rules_python//python/private/pypi:whl_library_targets.bzl", "{}")""".format(fn), ] - if not kwargs.setdefault("target_platforms", None): - dep_template = kwargs["dep_template"] - loads.append( - "load(\"{}\", \"{}\")".format( - dep_template.format(name = "", target = "config.bzl"), - "target_platforms", - ), - ) additional_content = [] if annotation: @@ -87,6 +101,7 @@ def generate_whl_library_build_bazel( [ _TEMPLATE.format( loads = "\n".join(loads), + fn = fn, kwargs = render.indent("\n".join([ "{} = {},".format(k, _RENDER.get(k, repr)(v)) for k, v in sorted(kwargs.items()) diff --git a/python/private/pypi/parse_requirements.bzl b/python/private/pypi/parse_requirements.bzl index 1cbf094f5c..5633328cf9 100644 --- a/python/private/pypi/parse_requirements.bzl +++ b/python/private/pypi/parse_requirements.bzl @@ -80,7 +80,7 @@ def parse_requirements( The second element is extra_pip_args should be passed to `whl_library`. """ - evaluate_markers = evaluate_markers or (lambda _: {}) + evaluate_markers = evaluate_markers or (lambda _ctx, _requirements: {}) options = {} requirements = {} for file, plats in requirements_by_platform.items(): @@ -156,7 +156,7 @@ def parse_requirements( # to do, we could use Python to parse the requirement lines and infer the # URL of the files to download things from. This should be important for # VCS package references. - env_marker_target_platforms = evaluate_markers(reqs_with_env_markers) + env_marker_target_platforms = evaluate_markers(ctx, reqs_with_env_markers) if logger: logger.debug(lambda: "Evaluated env markers from:\n{}\n\nTo:\n{}".format( reqs_with_env_markers, diff --git a/python/private/pypi/pip_repository.bzl b/python/private/pypi/pip_repository.bzl index b7ed1659d1..8ca94f7f9b 100644 --- a/python/private/pypi/pip_repository.bzl +++ b/python/private/pypi/pip_repository.bzl @@ -16,12 +16,11 @@ load("@bazel_skylib//lib:sets.bzl", "sets") load("//python/private:normalize_name.bzl", "normalize_name") -load("//python/private:repo_utils.bzl", "REPO_DEBUG_ENV_VAR", "repo_utils") +load("//python/private:repo_utils.bzl", "REPO_DEBUG_ENV_VAR") load("//python/private:text_util.bzl", "render") -load(":evaluate_markers.bzl", "evaluate_markers") +load(":evaluate_markers.bzl", "evaluate_markers_py", EVALUATE_MARKERS_SRCS = "SRCS") load(":parse_requirements.bzl", "host_platform", "parse_requirements", "select_requirement") load(":pip_repository_attrs.bzl", "ATTRS") -load(":pypi_repo_utils.bzl", "pypi_repo_utils") load(":render_pkg_aliases.bzl", "render_pkg_aliases") load(":requirements_files_by_platform.bzl", "requirements_files_by_platform") @@ -71,27 +70,7 @@ package(default_visibility = ["//visibility:public"]) exports_files(["requirements.bzl"]) """ -def _evaluate_markers(rctx, requirements, logger = None): - python_interpreter = _get_python_interpreter_attr(rctx) - stdout = pypi_repo_utils.execute_checked_stdout( - rctx, - op = "GetPythonVersionForMarkerEval", - python = python_interpreter, - arguments = [ - # Run the interpreter in isolated mode, this options implies -E, -P and -s. - # Ensures environment variables are ignored that are set in userspace, such as PYTHONPATH, - # which may interfere with this invocation. - "-I", - "-c", - "import sys; print(f'{sys.version_info[0]}.{sys.version_info[1]}.{sys.version_info[2]}', end='')", - ], - srcs = [], - logger = logger, - ) - return evaluate_markers(requirements, python_version = stdout) - def _pip_repository_impl(rctx): - logger = repo_utils.logger(rctx) requirements_by_platform = parse_requirements( rctx, requirements_by_platform = requirements_files_by_platform( @@ -103,7 +82,13 @@ def _pip_repository_impl(rctx): extra_pip_args = rctx.attr.extra_pip_args, ), extra_pip_args = rctx.attr.extra_pip_args, - evaluate_markers = lambda requirements: _evaluate_markers(rctx, requirements, logger), + evaluate_markers = lambda rctx, requirements: evaluate_markers_py( + rctx, + requirements = requirements, + python_interpreter = rctx.attr.python_interpreter, + python_interpreter_target = rctx.attr.python_interpreter_target, + srcs = rctx.attr._evaluate_markers_srcs, + ), ) selected_requirements = {} options = None @@ -249,6 +234,13 @@ file](https://github.com/bazel-contrib/rules_python/blob/main/examples/pip_repos _template = attr.label( default = ":requirements.bzl.tmpl.workspace", ), + _evaluate_markers_srcs = attr.label_list( + default = EVALUATE_MARKERS_SRCS, + doc = """\ +The list of labels to use as SRCS for the marker evaluation code. This ensures that the +code will be re-evaluated when any of files in the default changes. +""", + ), **ATTRS ), doc = """Accepts a locked/compiled requirements file and installs the dependencies listed within. diff --git a/python/private/pypi/whl_installer/wheel.py b/python/private/pypi/whl_installer/wheel.py index fce706acfb..25003e6280 100644 --- a/python/private/pypi/whl_installer/wheel.py +++ b/python/private/pypi/whl_installer/wheel.py @@ -62,7 +62,9 @@ def __init__( """ self.name: str = Deps._normalize(name) self._platforms: Set[Platform] = platforms or set() - self._target_versions = {(p.minor_version, p.micro_version) for p in platforms or {}} + self._target_versions = { + (p.minor_version, p.micro_version) for p in platforms or {} + } if platforms and len(self._target_versions) > 1: # TODO @aignas 2024-06-23: enable this to be set via a CLI arg # for being more explicit. @@ -94,8 +96,8 @@ def __init__( for req in reqs: reqs_by_name.setdefault(req.name, []).append(req) - for reqs in reqs_by_name.values(): - self._add_req(reqs, want_extras) + for req_name, reqs in reqs_by_name.items(): + self._add_req(req_name, reqs, want_extras) def _add(self, dep: str, platform: Optional[Platform]): dep = Deps._normalize(dep) @@ -134,7 +136,7 @@ def _normalize(name: str) -> str: return re.sub(r"[-_.]+", "_", name).lower() def _resolve_extras( - self, reqs: List[Requirement], extras: Optional[Set[str]] + self, reqs: List[Requirement], want_extras: Optional[Set[str]] ) -> Set[str]: """Resolve extras which are due to depending on self[some_other_extra]. @@ -156,7 +158,7 @@ def _resolve_extras( # extras The empty string in the set is just a way to make the handling # of no extras and a single extra easier and having a set of {"", "foo"} # is equivalent to having {"foo"}. - extras = extras or {""} + extras: Set[str] = want_extras or {""} self_reqs = [] for req in reqs: @@ -189,13 +191,18 @@ def _resolve_extras( return extras - def _add_req(self, reqs: List[Requirement], extras: Set[str]) -> None: + def _add_req(self, req_name, reqs: List[Requirement], extras: Set[str]) -> None: platforms_to_add = set() for req in reqs: if req.marker is None: self._add(req.name, None) return + if not self._platforms: + if any(req.marker.evaluate({"extra": extra}) for extra in extras): + self._add(req.name, None) + return + for plat in self._platforms: if plat in platforms_to_add: # marker evaluation is more expensive than this check @@ -211,18 +218,24 @@ def _add_req(self, reqs: List[Requirement], extras: Set[str]) -> None: added = True break + if not self._platforms: + return + if len(platforms_to_add) == len(self._platforms): # the dep is in all target platforms, let's just add it to the regular # list - self._add(req.name, None) + self._add(req_name, None) return for plat in platforms_to_add: if self._default_minor_version is not None: - self._add(req.name, plat) + self._add(req_name, plat) - if self._default_minor_version is None or plat.minor_version == self._default_minor_version: - self._add(req.name, Platform(os = plat.os, arch = plat.arch)) + if ( + self._default_minor_version is None + or plat.minor_version == self._default_minor_version + ): + self._add(req_name, Platform(os=plat.os, arch=plat.arch)) def build(self) -> FrozenDeps: return FrozenDeps( diff --git a/python/private/pypi/whl_library.bzl b/python/private/pypi/whl_library.bzl index 630dc8519f..0c09f7960a 100644 --- a/python/private/pypi/whl_library.bzl +++ b/python/private/pypi/whl_library.bzl @@ -15,18 +15,16 @@ "" load("//python/private:auth.bzl", "AUTH_ATTRS", "get_auth") -load("//python/private:bzlmod_enabled.bzl", "BZLMOD_ENABLED") load("//python/private:envsubst.bzl", "envsubst") load("//python/private:is_standalone_interpreter.bzl", "is_standalone_interpreter") load("//python/private:repo_utils.bzl", "REPO_DEBUG_ENV_VAR", "repo_utils") load(":attrs.bzl", "ATTRS", "use_isolated") load(":deps.bzl", "all_repo_names", "record_files") load(":generate_whl_library_build_bazel.bzl", "generate_whl_library_build_bazel") -load(":parse_requirements.bzl", "host_platform") +load(":parse_whl_name.bzl", "parse_whl_name") load(":patch_whl.bzl", "patch_whl") -load(":pep508_requirement.bzl", "requirement") load(":pypi_repo_utils.bzl", "pypi_repo_utils") -load(":whl_metadata.bzl", "whl_metadata") +load(":whl_target_platforms.bzl", "whl_target_platforms") _CPPFLAGS = "CPPFLAGS" _COMMAND_LINE_TOOLS_PATH_SLUG = "commandlinetools" @@ -342,6 +340,21 @@ def _whl_library_impl(rctx): timeout = rctx.attr.timeout, ) + target_platforms = rctx.attr.experimental_target_platforms or [] + if target_platforms: + parsed_whl = parse_whl_name(whl_path.basename) + + # NOTE @aignas 2023-12-04: if the wheel is a platform specific wheel, we + # only include deps for that target platform + if parsed_whl.platform_tag != "any": + target_platforms = [ + p.target_platform + for p in whl_target_platforms( + platform_tag = parsed_whl.platform_tag, + abi_tag = parsed_whl.abi_tag.strip("tm"), + ) + ] + pypi_repo_utils.execute_checked( rctx, op = "whl_library.ExtractWheel({}, {})".format(rctx.attr.name, whl_path), @@ -349,7 +362,7 @@ def _whl_library_impl(rctx): arguments = args + [ "--whl-file", whl_path, - ], + ] + ["--platform={}".format(p) for p in target_platforms], srcs = rctx.attr._python_srcs, environment = environment, quiet = rctx.attr.quiet, @@ -384,45 +397,21 @@ def _whl_library_impl(rctx): ) entry_points[entry_point_without_py] = entry_point_script_name - if BZLMOD_ENABLED: - # The following attributes are unset on bzlmod and we pass data through - # the hub via load statements. - default_python_version = None - target_platforms = [] - else: - # NOTE @aignas 2025-04-16: if BZLMOD_ENABLED, we should use - # DEFAULT_PYTHON_VERSION since platforms always come with the actual - # python version otherwise we should use the version of the interpreter - # here. In WORKSPACE `multi_pip_parse` is using an interpreter for each - # `pip_parse` invocation, so we will have the host target platform - # only. Even if somebody would change the code to support - # `experimental_target_platforms`, they would be for a single python - # version. Hence, using the `default_python_version` that we get from the - # interpreter is correct. Hence, we unset the argument if we are on bzlmod. - default_python_version = metadata["python_version"] - target_platforms = rctx.attr.experimental_target_platforms or [host_platform(rctx)] - - metadata = whl_metadata( - install_dir = rctx.path("site-packages"), - read_fn = rctx.read, - logger = logger, - ) - build_file_contents = generate_whl_library_build_bazel( name = whl_path.basename, - metadata_name = metadata.name, - metadata_version = metadata.version, - requires_dist = metadata.requires_dist, dep_template = rctx.attr.dep_template or "@{}{{name}}//:{{target}}".format(rctx.attr.repo_prefix), entry_points = entry_points, - target_platforms = target_platforms, - default_python_version = default_python_version, # TODO @aignas 2025-04-14: load through the hub: + dependencies = metadata["deps"], + dependencies_by_platform = metadata["deps_by_platform"], annotation = None if not rctx.attr.annotation else struct(**json.decode(rctx.read(rctx.attr.annotation))), data_exclude = rctx.attr.pip_data_exclude, - extras = requirement(rctx.attr.requirement).extras, group_deps = rctx.attr.group_deps, group_name = rctx.attr.group_name, + tags = [ + "pypi_name={}".format(metadata["name"]), + "pypi_version={}".format(metadata["version"]), + ], ) rctx.file("BUILD.bazel", build_file_contents) diff --git a/tests/pypi/extension/extension_tests.bzl b/tests/pypi/extension/extension_tests.bzl index 5de3bb58d3..1cd6869c84 100644 --- a/tests/pypi/extension/extension_tests.bzl +++ b/tests/pypi/extension/extension_tests.bzl @@ -136,6 +136,7 @@ def _parse( parallel_download = False, experimental_index_url_overrides = {}, simpleapi_skip = simpleapi_skip, + _evaluate_markers_srcs = [], **kwargs ) @@ -273,6 +274,14 @@ torch==2.4.1 ; platform_machine != 'x86_64' \ "python_3_15_host": "unit_test_interpreter_target", }, minor_mapping = {"3.15": "3.15.19"}, + evaluate_markers = lambda _, requirements, **__: { + key: [ + platform + for platform in platforms + if ("x86_64" in platform and "platform_machine ==" in key) or ("x86_64" not in platform and "platform_machine !=" in key) + ] + for key, platforms in requirements.items() + }, ) pypi.exposed_packages().contains_exactly({"pypi": ["torch"]}) @@ -397,6 +406,15 @@ torch==2.4.1+cpu ; platform_machine == 'x86_64' \ }, minor_mapping = {"3.12": "3.12.19"}, simpleapi_download = mocksimpleapi_download, + evaluate_markers = lambda _, requirements, **__: { + # todo once 2692 is merged, this is going to be easier to test. + key: [ + platform + for platform in platforms + if ("x86_64" in platform and "platform_machine ==" in key) or ("x86_64" not in platform and "platform_machine !=" in key) + ] + for key, platforms in requirements.items() + }, ) pypi.exposed_packages().contains_exactly({"pypi": ["torch"]}) @@ -440,6 +458,11 @@ torch==2.4.1+cpu ; platform_machine == 'x86_64' \ pypi.whl_libraries().contains_exactly({ "pypi_312_torch_cp312_cp312_linux_x86_64_8800deef": { "dep_template": "@pypi//{name}:{target}", + "experimental_target_platforms": [ + "linux_x86_64", + "osx_x86_64", + "windows_x86_64", + ], "filename": "torch-2.4.1+cpu-cp312-cp312-linux_x86_64.whl", "python_interpreter_target": "unit_test_interpreter_target", "requirement": "torch==2.4.1+cpu", @@ -448,6 +471,13 @@ torch==2.4.1+cpu ; platform_machine == 'x86_64' \ }, "pypi_312_torch_cp312_cp312_manylinux_2_17_aarch64_36109432": { "dep_template": "@pypi//{name}:{target}", + "experimental_target_platforms": [ + "linux_aarch64", + "linux_arm", + "linux_ppc", + "linux_s390x", + "osx_aarch64", + ], "filename": "torch-2.4.1-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", "python_interpreter_target": "unit_test_interpreter_target", "requirement": "torch==2.4.1", @@ -456,6 +486,11 @@ torch==2.4.1+cpu ; platform_machine == 'x86_64' \ }, "pypi_312_torch_cp312_cp312_win_amd64_3a570e5c": { "dep_template": "@pypi//{name}:{target}", + "experimental_target_platforms": [ + "linux_x86_64", + "osx_x86_64", + "windows_x86_64", + ], "filename": "torch-2.4.1+cpu-cp312-cp312-win_amd64.whl", "python_interpreter_target": "unit_test_interpreter_target", "requirement": "torch==2.4.1+cpu", @@ -464,6 +499,13 @@ torch==2.4.1+cpu ; platform_machine == 'x86_64' \ }, "pypi_312_torch_cp312_none_macosx_11_0_arm64_72b484d5": { "dep_template": "@pypi//{name}:{target}", + "experimental_target_platforms": [ + "linux_aarch64", + "linux_arm", + "linux_ppc", + "linux_s390x", + "osx_aarch64", + ], "filename": "torch-2.4.1-cp312-none-macosx_11_0_arm64.whl", "python_interpreter_target": "unit_test_interpreter_target", "requirement": "torch==2.4.1", @@ -751,6 +793,16 @@ git_dep @ git+https://git.server/repo/project@deadbeefdeadbeef pypi.whl_libraries().contains_exactly({ "pypi_315_any_name": { "dep_template": "@pypi//{name}:{target}", + "experimental_target_platforms": [ + "linux_aarch64", + "linux_arm", + "linux_ppc", + "linux_s390x", + "linux_x86_64", + "osx_aarch64", + "osx_x86_64", + "windows_x86_64", + ], "extra_pip_args": ["--extra-args-for-sdist-building"], "filename": "any-name.tar.gz", "python_interpreter_target": "unit_test_interpreter_target", @@ -760,6 +812,16 @@ git_dep @ git+https://git.server/repo/project@deadbeefdeadbeef }, "pypi_315_direct_without_sha_0_0_1_py3_none_any": { "dep_template": "@pypi//{name}:{target}", + "experimental_target_platforms": [ + "linux_aarch64", + "linux_arm", + "linux_ppc", + "linux_s390x", + "linux_x86_64", + "osx_aarch64", + "osx_x86_64", + "windows_x86_64", + ], "filename": "direct_without_sha-0.0.1-py3-none-any.whl", "python_interpreter_target": "unit_test_interpreter_target", "requirement": "direct_without_sha==0.0.1 @ example-direct.org/direct_without_sha-0.0.1-py3-none-any.whl", @@ -780,6 +842,16 @@ git_dep @ git+https://git.server/repo/project@deadbeefdeadbeef }, "pypi_315_simple_py3_none_any_deadb00f": { "dep_template": "@pypi//{name}:{target}", + "experimental_target_platforms": [ + "linux_aarch64", + "linux_arm", + "linux_ppc", + "linux_s390x", + "linux_x86_64", + "osx_aarch64", + "osx_x86_64", + "windows_x86_64", + ], "filename": "simple-0.0.1-py3-none-any.whl", "python_interpreter_target": "unit_test_interpreter_target", "requirement": "simple==0.0.1", @@ -788,6 +860,16 @@ git_dep @ git+https://git.server/repo/project@deadbeefdeadbeef }, "pypi_315_simple_sdist_deadbeef": { "dep_template": "@pypi//{name}:{target}", + "experimental_target_platforms": [ + "linux_aarch64", + "linux_arm", + "linux_ppc", + "linux_s390x", + "linux_x86_64", + "osx_aarch64", + "osx_x86_64", + "windows_x86_64", + ], "extra_pip_args": ["--extra-args-for-sdist-building"], "filename": "simple-0.0.1.tar.gz", "python_interpreter_target": "unit_test_interpreter_target", @@ -797,6 +879,16 @@ git_dep @ git+https://git.server/repo/project@deadbeefdeadbeef }, "pypi_315_some_pkg_py3_none_any_deadbaaf": { "dep_template": "@pypi//{name}:{target}", + "experimental_target_platforms": [ + "linux_aarch64", + "linux_arm", + "linux_ppc", + "linux_s390x", + "linux_x86_64", + "osx_aarch64", + "osx_x86_64", + "windows_x86_64", + ], "filename": "some_pkg-0.0.1-py3-none-any.whl", "python_interpreter_target": "unit_test_interpreter_target", "requirement": "some_pkg==0.0.1 @ example-direct.org/some_pkg-0.0.1-py3-none-any.whl --hash=sha256:deadbaaf", @@ -805,6 +897,16 @@ git_dep @ git+https://git.server/repo/project@deadbeefdeadbeef }, "pypi_315_some_py3_none_any_deadb33f": { "dep_template": "@pypi//{name}:{target}", + "experimental_target_platforms": [ + "linux_aarch64", + "linux_arm", + "linux_ppc", + "linux_s390x", + "linux_x86_64", + "osx_aarch64", + "osx_x86_64", + "windows_x86_64", + ], "filename": "some-other-pkg-0.0.1-py3-none-any.whl", "python_interpreter_target": "unit_test_interpreter_target", "requirement": "some_other_pkg==0.0.1", @@ -856,6 +958,14 @@ optimum[onnxruntime-gpu]==1.17.1 ; sys_platform == 'linux' "python_3_15_host": "unit_test_interpreter_target", }, minor_mapping = {"3.15": "3.15.19"}, + evaluate_markers = lambda _, requirements, **__: { + key: [ + platform + for platform in platforms + if ("darwin" in key and "osx" in platform) or ("linux" in key and "linux" in platform) + ] + for key, platforms in requirements.items() + }, ) pypi.exposed_packages().contains_exactly({"pypi": []}) diff --git a/tests/pypi/generate_whl_library_build_bazel/generate_whl_library_build_bazel_tests.bzl b/tests/pypi/generate_whl_library_build_bazel/generate_whl_library_build_bazel_tests.bzl index 7bd19b65c1..83be7395d4 100644 --- a/tests/pypi/generate_whl_library_build_bazel/generate_whl_library_build_bazel_tests.bzl +++ b/tests/pypi/generate_whl_library_build_bazel/generate_whl_library_build_bazel_tests.bzl @@ -86,7 +86,6 @@ _tests.append(_test_all) def _test_all_with_loads(env): want = """\ load("@rules_python//python/private/pypi:whl_library_targets.bzl", "whl_library_targets_from_requires") -load("@pypi//:config.bzl", "target_platforms") package(default_visibility = ["//visibility:public"]) @@ -119,7 +118,6 @@ whl_library_targets_from_requires( "qux", ], srcs_exclude = ["srcs_exclude_all"], - target_platforms = target_platforms, ) # SOMETHING SPECIAL AT THE END diff --git a/tests/pypi/parse_requirements/parse_requirements_tests.bzl b/tests/pypi/parse_requirements/parse_requirements_tests.bzl index c50482127b..723bb605ce 100644 --- a/tests/pypi/parse_requirements/parse_requirements_tests.bzl +++ b/tests/pypi/parse_requirements/parse_requirements_tests.bzl @@ -458,7 +458,7 @@ def _test_select_requirement_none_platform(env): _tests.append(_test_select_requirement_none_platform) def _test_env_marker_resolution(env): - def _mock_eval_markers(input): + def _mock_eval_markers(_, input): ret = { "foo[extra]==0.0.1 ;marker --hash=sha256:deadbeef": ["cp311_windows_x86_64"], } diff --git a/tests/pypi/whl_installer/wheel_test.py b/tests/pypi/whl_installer/wheel_test.py index 6921fe6d3f..3599fd1868 100644 --- a/tests/pypi/whl_installer/wheel_test.py +++ b/tests/pypi/whl_installer/wheel_test.py @@ -11,7 +11,7 @@ class DepsTest(unittest.TestCase): def test_simple(self): - deps = wheel.Deps("foo", requires_dist=["bar"]) + deps = wheel.Deps("foo", requires_dist=["bar", 'baz; extra=="foo"']) got = deps.build() From 1c35e4c84674ce25c9d9963125d335258f257ce7 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Mon, 28 Apr 2025 20:07:31 -0700 Subject: [PATCH 198/922] feat: implement less/greater operators for string for env marker evaluation (#2827) Right now, if two strings are compared, it results in an error. Per spec, strings are suppose to "use the python behavior". Starlark is going to use Java semantics underneath, but it should behave close enough for the (almost exclusively) ASCII input that will be used. Work towards https://github.com/bazel-contrib/rules_python/issues/2826 --- python/private/pypi/pep508_evaluate.bzl | 8 ++++++++ tests/pypi/pep508/evaluate_tests.bzl | 22 ++++++++++++++++------ 2 files changed, 24 insertions(+), 6 deletions(-) diff --git a/python/private/pypi/pep508_evaluate.bzl b/python/private/pypi/pep508_evaluate.bzl index f8ef553034..70840c76c6 100644 --- a/python/private/pypi/pep508_evaluate.bzl +++ b/python/private/pypi/pep508_evaluate.bzl @@ -344,6 +344,14 @@ def _env_expr(left, op, right): return left in right elif op == "not in": return left not in right + elif op == "<": + return left < right + elif op == "<=": + return left <= right + elif op == ">": + return left > right + elif op == ">=": + return left >= right else: return fail("TODO: op unsupported: '{}'".format(op)) diff --git a/tests/pypi/pep508/evaluate_tests.bzl b/tests/pypi/pep508/evaluate_tests.bzl index 14e5e40b43..303c167900 100644 --- a/tests/pypi/pep508/evaluate_tests.bzl +++ b/tests/pypi/pep508/evaluate_tests.bzl @@ -68,18 +68,28 @@ def _evaluate_non_version_env_tests(env): # When for input, want in { - "{} == 'osx'".format(var_name): True, - "{} != 'osx'".format(var_name): False, - "'osx' == {}".format(var_name): True, "'osx' != {}".format(var_name): False, - "'x' in {}".format(var_name): True, + "'osx' < {}".format(var_name): False, + "'osx' <= {}".format(var_name): True, + "'osx' == {}".format(var_name): True, + "'osx' >= {}".format(var_name): True, "'w' not in {}".format(var_name): True, - }.items(): # buildifier: @unsorted-dict-items + "'x' in {}".format(var_name): True, + "{} != 'osx'".format(var_name): False, + "{} < 'osx'".format(var_name): False, + "{} <= 'osx'".format(var_name): True, + "{} == 'osx'".format(var_name): True, + "{} > 'osx'".format(var_name): False, + "{} >= 'osx'".format(var_name): True, + }.items(): got = evaluate( input, env = marker_env, ) - env.expect.that_bool(got).equals(want) + env.expect.where( + expr = input, + env = marker_env, + ).that_bool(got).equals(want) # Check that the non-strict eval gives us back the input when no # env is supplied. From 704ecdd835c8a79ac415c81567eae5785df4b7e3 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Mon, 28 Apr 2025 20:07:57 -0700 Subject: [PATCH 199/922] docs: doc version when RULES_PYTHON_ENABLE_PYSTAR was introduced (#2838) While figuring out an upgrade from an old rules_python version, I had to look up when the environment variable first became available. Also note what version it defaulted to 1. --- docs/environment-variables.md | 12 ++++++++++-- 1 file changed, 10 insertions(+), 2 deletions(-) diff --git a/docs/environment-variables.md b/docs/environment-variables.md index 9500fa8295..49fdf766f6 100644 --- a/docs/environment-variables.md +++ b/docs/environment-variables.md @@ -46,11 +46,19 @@ When `1`, the rules_python will warn users about deprecated functionality that w be removed in a subsequent major `rules_python` version. Defaults to `0` if unset. ::: -:::{envvar} RULES_PYTHON_ENABLE_PYSTAR +::::{envvar} RULES_PYTHON_ENABLE_PYSTAR When `1`, the rules_python Starlark implementation of the core rules is used -instead of the Bazel-builtin rules. Note this requires Bazel 7+. +instead of the Bazel-builtin rules. Note this requires Bazel 7+. Defaults +to `1`. + +:::{versionadded} 0.26.0 +Defaults to `0` if unspecified. +::: +:::{versionchanged} 0.40.0 +The default became `1` if unspecified ::: +:::: ::::{envvar} RULES_PYTHON_EXTRACT_ROOT From a79bbfaece3e41f361b7d5baf89aec269184eb4d Mon Sep 17 00:00:00 2001 From: Ignas Anikevicius <240938+aignas@users.noreply.github.com> Date: Tue, 29 Apr 2025 14:52:46 +0900 Subject: [PATCH 200/922] fix(pypi): handle more URL patterns for requirement sources (#2843) Summary: - Better handle git references for sdists. - Better handle direct whl references. - Add an extra test that turned out to be not needed in the end, but I left it to increase the code coverage. Work towards #2363 Fixes #2828 --- python/private/pypi/parse_requirements.bzl | 5 ++ .../index_sources/index_sources_tests.bzl | 14 ++++- .../parse_requirements_tests.bzl | 59 +++++++++++++++++++ 3 files changed, 77 insertions(+), 1 deletion(-) diff --git a/python/private/pypi/parse_requirements.bzl b/python/private/pypi/parse_requirements.bzl index 5633328cf9..1583c89199 100644 --- a/python/private/pypi/parse_requirements.bzl +++ b/python/private/pypi/parse_requirements.bzl @@ -285,12 +285,17 @@ def _add_dists(*, requirement, index_urls, logger = None): if requirement.srcs.url: url = requirement.srcs.url _, _, filename = url.rpartition("/") + filename, _, _ = filename.partition("#sha256=") if "." not in filename: # detected filename has no extension, it might be an sdist ref # TODO @aignas 2025-04-03: should be handled if the following is fixed: # https://github.com/bazel-contrib/rules_python/issues/2363 return [], None + if "@" in filename: + # this is most likely foo.git@git_sha, skip special handling of these + return [], None + direct_url_dist = struct( url = url, filename = filename, diff --git a/tests/pypi/index_sources/index_sources_tests.bzl b/tests/pypi/index_sources/index_sources_tests.bzl index ffeed87a7b..9d12bc6399 100644 --- a/tests/pypi/index_sources/index_sources_tests.bzl +++ b/tests/pypi/index_sources/index_sources_tests.bzl @@ -21,38 +21,50 @@ _tests = [] def _test_no_simple_api_sources(env): inputs = { + "foo @ git+https://github.com/org/foo.git@deadbeef": struct( + requirement = "foo @ git+https://github.com/org/foo.git@deadbeef", + marker = "", + url = "git+https://github.com/org/foo.git@deadbeef", + shas = [], + version = "", + ), "foo==0.0.1": struct( requirement = "foo==0.0.1", marker = "", url = "", + version = "0.0.1", ), "foo==0.0.1 @ https://someurl.org": struct( requirement = "foo==0.0.1 @ https://someurl.org", marker = "", url = "https://someurl.org", + version = "0.0.1", ), "foo==0.0.1 @ https://someurl.org/package.whl": struct( requirement = "foo==0.0.1 @ https://someurl.org/package.whl", marker = "", url = "https://someurl.org/package.whl", + version = "0.0.1", ), "foo==0.0.1 @ https://someurl.org/package.whl --hash=sha256:deadbeef": struct( requirement = "foo==0.0.1 @ https://someurl.org/package.whl --hash=sha256:deadbeef", marker = "", url = "https://someurl.org/package.whl", shas = ["deadbeef"], + version = "0.0.1", ), "foo==0.0.1 @ https://someurl.org/package.whl; python_version < \"2.7\"\\ --hash=sha256:deadbeef": struct( requirement = "foo==0.0.1 @ https://someurl.org/package.whl --hash=sha256:deadbeef", marker = "python_version < \"2.7\"", url = "https://someurl.org/package.whl", shas = ["deadbeef"], + version = "0.0.1", ), } for input, want in inputs.items(): got = index_sources(input) env.expect.that_collection(got.shas).contains_exactly(want.shas if hasattr(want, "shas") else []) - env.expect.that_str(got.version).equals("0.0.1") + env.expect.that_str(got.version).equals(want.version) env.expect.that_str(got.requirement).equals(want.requirement) env.expect.that_str(got.requirement_line).equals(got.requirement) env.expect.that_str(got.marker).equals(want.marker) diff --git a/tests/pypi/parse_requirements/parse_requirements_tests.bzl b/tests/pypi/parse_requirements/parse_requirements_tests.bzl index 723bb605ce..c5b24870ea 100644 --- a/tests/pypi/parse_requirements/parse_requirements_tests.bzl +++ b/tests/pypi/parse_requirements/parse_requirements_tests.bzl @@ -30,12 +30,16 @@ foo[extra] @ https://some-url/package.whl bar @ https://example.org/bar-1.0.whl --hash=sha256:deadbeef baz @ https://test.com/baz-2.0.whl; python_version < "3.8" --hash=sha256:deadb00f qux @ https://example.org/qux-1.0.tar.gz --hash=sha256:deadbe0f +torch @ https://download.pytorch.org/whl/cpu/torch-2.6.0%2Bcpu-cp311-cp311-linux_x86_64.whl#sha256=5b6ae523bfb67088a17ca7734d131548a2e60346c622621e4248ed09dd0790cc """, "requirements_extra_args": """\ --index-url=example.org foo[extra]==0.0.1 \ --hash=sha256:deadbeef +""", + "requirements_git": """ +foo @ git+https://github.com/org/foo.git@deadbeef """, "requirements_linux": """\ foo==0.0.3 --hash=sha256:deadbaaf @@ -232,6 +236,31 @@ def _test_direct_urls(env): whls = [], ), ], + "torch": [ + struct( + distribution = "torch", + extra_pip_args = [], + is_exposed = True, + sdist = None, + srcs = struct( + marker = "", + requirement = "torch @ https://download.pytorch.org/whl/cpu/torch-2.6.0%2Bcpu-cp311-cp311-linux_x86_64.whl#sha256=5b6ae523bfb67088a17ca7734d131548a2e60346c622621e4248ed09dd0790cc", + requirement_line = "torch @ https://download.pytorch.org/whl/cpu/torch-2.6.0%2Bcpu-cp311-cp311-linux_x86_64.whl#sha256=5b6ae523bfb67088a17ca7734d131548a2e60346c622621e4248ed09dd0790cc", + shas = [], + url = "https://download.pytorch.org/whl/cpu/torch-2.6.0%2Bcpu-cp311-cp311-linux_x86_64.whl#sha256=5b6ae523bfb67088a17ca7734d131548a2e60346c622621e4248ed09dd0790cc", + version = "", + ), + target_platforms = ["linux_x86_64"], + whls = [ + struct( + filename = "torch-2.6.0%2Bcpu-cp311-cp311-linux_x86_64.whl", + sha256 = "", + url = "https://download.pytorch.org/whl/cpu/torch-2.6.0%2Bcpu-cp311-cp311-linux_x86_64.whl#sha256=5b6ae523bfb67088a17ca7734d131548a2e60346c622621e4248ed09dd0790cc", + yanked = False, + ), + ], + ), + ], }) _tests.append(_test_direct_urls) @@ -623,6 +652,36 @@ def _test_optional_hash(env): _tests.append(_test_optional_hash) +def _test_git_sources(env): + got = parse_requirements( + ctx = _mock_ctx(), + requirements_by_platform = { + "requirements_git": ["linux_x86_64"], + }, + ) + env.expect.that_dict(got).contains_exactly({ + "foo": [ + struct( + distribution = "foo", + extra_pip_args = [], + is_exposed = True, + sdist = None, + srcs = struct( + marker = "", + requirement = "foo @ git+https://github.com/org/foo.git@deadbeef", + requirement_line = "foo @ git+https://github.com/org/foo.git@deadbeef", + shas = [], + url = "git+https://github.com/org/foo.git@deadbeef", + version = "", + ), + target_platforms = ["linux_x86_64"], + whls = [], + ), + ], + }) + +_tests.append(_test_git_sources) + def parse_requirements_test_suite(name): """Create the test suite. From 189e30df4001d34aba590e0267d3e5f72e6d8b19 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Tue, 29 Apr 2025 10:08:37 -0700 Subject: [PATCH 201/922] docs: document some of our project styles/conventions (#2816) Spurred by the discussion to converge on using `.` to separate generated targets, I wrote down some of the conventions we've adopted. --------- Co-authored-by: Ignas Anikevicius <240938+aignas@users.noreply.github.com> --- .editorconfig | 17 +++++++++++++++++ CONTRIBUTING.md | 49 +++++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 66 insertions(+) create mode 100644 .editorconfig diff --git a/.editorconfig b/.editorconfig new file mode 100644 index 0000000000..26bb52ffac --- /dev/null +++ b/.editorconfig @@ -0,0 +1,17 @@ +# Unix-style newlines with a newline ending every file +[*] +end_of_line = lf +insert_final_newline = true + +# Set default charset +[*] +charset = utf-8 + +# Line width +[*] +max_line_length = 100 + +# 4 space indentation +[*.{py,bzl}] +indent_style = space +indent_size = 4 diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 17558e1b23..b087119dc6 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -173,6 +173,55 @@ The `legacy_foo` arg was removed ::: ``` +## Style and idioms + +For the most part, we just accept whatever the code formatters do, so there +isn't much style to enforce. + +Some miscellanous style, idioms, and conventions we have are: + +### Markdown/Sphinx Style + +* Use colons for prose sections of text, e.g. `:::{note}`, not backticks. +* Use backticks for code blocks. +* Max line length: 100. + +### BUILD/bzl Style + +* When a macro generates public targets, use a dot (`.`) to separate the + user-provided name from the generted name. e.g. `foo(name="x")` generates + `x.test`. The `.` is our convention to communicate that it's a generated + target, and thus one should look for `name="x"` when searching for the + definition. +* The different build phases shouldn't load code that defines objects that + aren't valid for their phase. e.g. + * The bzlmod phase shouldn't load code defining regular rules or providers. + * The repository phase shouldn't load code defining module extensions, regular + rules, or providers. + * The loading phase shouldn't load code defining module extensions or + repository rules. + * Loading utility libraries or generic code is OK, but should strive to load + code that is usable for its phase. e.g. loading-phase code shouldn't + load utility code that is predominately only usable to the bzlmod phase. +* Providers should be in their own files. This allows implementing a custom rule + that implements the provider without loading a specific implementation. +* One rule per file is preferred, but not required. The goal is that defining an + e.g. library shouldn't incur loading all the code for binaries, tests, + packaging, etc; things that may be niche or uncommonly used. +* Separate files should be used to expose public APIs. This ensures our public + API is well defined and prevents accidentally exposing a package-private + symbol as a public symbol. + + :::{note} + The public API file's docstring becomes part of the user-facing docs. That + file's docstring must be used for module-level API documentation. + ::: +* Repository rules should have name ending in `_repo`. This helps distinguish + them from regular rules. +* Each bzlmod extension, the "X" of `use_repo("//foo:foo.bzl", "X")` should be + in its own file. The path given in the `use_repo()` expression is the identity + Bazel uses and cannot be changed. + ## Generated files Some checked-in files are generated and need to be updated when a new PR is From 76b221e668d7038b8a069bf44b81682876dbea38 Mon Sep 17 00:00:00 2001 From: Vein Kong Date: Thu, 1 May 2025 23:36:56 -0700 Subject: [PATCH 202/922] fix: requires_file preserves extras that package depends on (#2807) When requirements are passed in through `requires_file` the extras are not preserved. eg if the contents of requires file is `example[extras]==1.1.1`, bazel will currently write to the METADATA file `Requires-Dist: example==1.1.1`. This PR attempts to fix that by adding that back if there are any extras. The expected output should be `Requires-Dist: example[extras]==1.1.1` --- .bazelrc | 4 +-- CHANGELOG.md | 2 ++ examples/wheel/BUILD.bazel | 29 +++++++++++++++++++++ examples/wheel/wheel_test.py | 50 ++++++++++++++++++++++++++++++++++++ tools/wheelmaker.py | 7 ++--- 5 files changed, 87 insertions(+), 5 deletions(-) diff --git a/.bazelrc b/.bazelrc index 4e6f2fa187..d2e0721526 100644 --- a/.bazelrc +++ b/.bazelrc @@ -4,8 +4,8 @@ # (Note, we cannot use `common --deleted_packages` because the bazel version command doesn't support it) # To update these lines, execute # `bazel run @rules_bazel_integration_test//tools:update_deleted_packages` -build --deleted_packages=examples/build_file_generation,examples/build_file_generation/random_number_generator,examples/bzlmod,examples/bzlmod_build_file_generation,examples/bzlmod_build_file_generation/other_module/other_module/pkg,examples/bzlmod_build_file_generation/runfiles,examples/bzlmod/entry_points,examples/bzlmod/entry_points/tests,examples/bzlmod/libs/my_lib,examples/bzlmod/other_module,examples/bzlmod/other_module/other_module/pkg,examples/bzlmod/patches,examples/bzlmod/py_proto_library,examples/bzlmod/py_proto_library/example.com/another_proto,examples/bzlmod/py_proto_library/example.com/proto,examples/bzlmod/runfiles,examples/bzlmod/tests,examples/bzlmod/tests/other_module,examples/bzlmod/whl_mods,examples/multi_python_versions/libs/my_lib,examples/multi_python_versions/requirements,examples/multi_python_versions/tests,examples/pip_parse,examples/pip_parse_vendored,examples/pip_repository_annotations,examples/py_proto_library,examples/py_proto_library/example.com/another_proto,examples/py_proto_library/example.com/proto,gazelle,gazelle/manifest,gazelle/manifest/generate,gazelle/manifest/hasher,gazelle/manifest/test,gazelle/modules_mapping,gazelle/python,gazelle/pythonconfig,gazelle/python/private,tests/integration/compile_pip_requirements,tests/integration/compile_pip_requirements_test_from_external_repo,tests/integration/custom_commands,tests/integration/ignore_root_user_error,tests/integration/ignore_root_user_error/submodule,tests/integration/local_toolchains,tests/integration/pip_parse,tests/integration/pip_parse/empty,tests/integration/py_cc_toolchain_registered,tests/modules/other,tests/modules/other/nspkg_delta,tests/modules/other/nspkg_gamma -query --deleted_packages=examples/build_file_generation,examples/build_file_generation/random_number_generator,examples/bzlmod,examples/bzlmod_build_file_generation,examples/bzlmod_build_file_generation/other_module/other_module/pkg,examples/bzlmod_build_file_generation/runfiles,examples/bzlmod/entry_points,examples/bzlmod/entry_points/tests,examples/bzlmod/libs/my_lib,examples/bzlmod/other_module,examples/bzlmod/other_module/other_module/pkg,examples/bzlmod/patches,examples/bzlmod/py_proto_library,examples/bzlmod/py_proto_library/example.com/another_proto,examples/bzlmod/py_proto_library/example.com/proto,examples/bzlmod/runfiles,examples/bzlmod/tests,examples/bzlmod/tests/other_module,examples/bzlmod/whl_mods,examples/multi_python_versions/libs/my_lib,examples/multi_python_versions/requirements,examples/multi_python_versions/tests,examples/pip_parse,examples/pip_parse_vendored,examples/pip_repository_annotations,examples/py_proto_library,examples/py_proto_library/example.com/another_proto,examples/py_proto_library/example.com/proto,gazelle,gazelle/manifest,gazelle/manifest/generate,gazelle/manifest/hasher,gazelle/manifest/test,gazelle/modules_mapping,gazelle/python,gazelle/pythonconfig,gazelle/python/private,tests/integration/compile_pip_requirements,tests/integration/compile_pip_requirements_test_from_external_repo,tests/integration/custom_commands,tests/integration/ignore_root_user_error,tests/integration/ignore_root_user_error/submodule,tests/integration/local_toolchains,tests/integration/pip_parse,tests/integration/pip_parse/empty,tests/integration/py_cc_toolchain_registered,tests/modules/other,tests/modules/other/nspkg_delta,tests/modules/other/nspkg_gamma +build --deleted_packages=examples/build_file_generation,examples/build_file_generation/random_number_generator,examples/bzlmod,examples/bzlmod/entry_points,examples/bzlmod/entry_points/tests,examples/bzlmod/libs/my_lib,examples/bzlmod/other_module,examples/bzlmod/other_module/other_module/pkg,examples/bzlmod/patches,examples/bzlmod/py_proto_library,examples/bzlmod/py_proto_library/example.com/another_proto,examples/bzlmod/py_proto_library/example.com/proto,examples/bzlmod/runfiles,examples/bzlmod/tests,examples/bzlmod/tests/other_module,examples/bzlmod/whl_mods,examples/bzlmod_build_file_generation,examples/bzlmod_build_file_generation/other_module/other_module/pkg,examples/bzlmod_build_file_generation/runfiles,examples/multi_python_versions/libs/my_lib,examples/multi_python_versions/requirements,examples/multi_python_versions/tests,examples/pip_parse,examples/pip_parse_vendored,examples/pip_repository_annotations,examples/py_proto_library,examples/py_proto_library/example.com/another_proto,examples/py_proto_library/example.com/proto,gazelle,gazelle/manifest,gazelle/manifest/generate,gazelle/manifest/hasher,gazelle/manifest/test,gazelle/modules_mapping,gazelle/python,gazelle/python/private,gazelle/pythonconfig,tests/integration/compile_pip_requirements,tests/integration/compile_pip_requirements_test_from_external_repo,tests/integration/custom_commands,tests/integration/ignore_root_user_error,tests/integration/ignore_root_user_error/submodule,tests/integration/local_toolchains,tests/integration/pip_parse,tests/integration/pip_parse/empty,tests/integration/py_cc_toolchain_registered,tests/modules/other,tests/modules/other/nspkg_delta,tests/modules/other/nspkg_gamma +query --deleted_packages=examples/build_file_generation,examples/build_file_generation/random_number_generator,examples/bzlmod,examples/bzlmod/entry_points,examples/bzlmod/entry_points/tests,examples/bzlmod/libs/my_lib,examples/bzlmod/other_module,examples/bzlmod/other_module/other_module/pkg,examples/bzlmod/patches,examples/bzlmod/py_proto_library,examples/bzlmod/py_proto_library/example.com/another_proto,examples/bzlmod/py_proto_library/example.com/proto,examples/bzlmod/runfiles,examples/bzlmod/tests,examples/bzlmod/tests/other_module,examples/bzlmod/whl_mods,examples/bzlmod_build_file_generation,examples/bzlmod_build_file_generation/other_module/other_module/pkg,examples/bzlmod_build_file_generation/runfiles,examples/multi_python_versions/libs/my_lib,examples/multi_python_versions/requirements,examples/multi_python_versions/tests,examples/pip_parse,examples/pip_parse_vendored,examples/pip_repository_annotations,examples/py_proto_library,examples/py_proto_library/example.com/another_proto,examples/py_proto_library/example.com/proto,gazelle,gazelle/manifest,gazelle/manifest/generate,gazelle/manifest/hasher,gazelle/manifest/test,gazelle/modules_mapping,gazelle/python,gazelle/python/private,gazelle/pythonconfig,tests/integration/compile_pip_requirements,tests/integration/compile_pip_requirements_test_from_external_repo,tests/integration/custom_commands,tests/integration/ignore_root_user_error,tests/integration/ignore_root_user_error/submodule,tests/integration/local_toolchains,tests/integration/pip_parse,tests/integration/pip_parse/empty,tests/integration/py_cc_toolchain_registered,tests/modules/other,tests/modules/other/nspkg_delta,tests/modules/other/nspkg_gamma test --test_output=errors diff --git a/CHANGELOG.md b/CHANGELOG.md index a8cac4c5cd..19fe636bc3 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -74,6 +74,8 @@ END_UNRELEASED_TEMPLATE * The {obj}`//python/runtime_env_toolchains:all` toolchain now works with it. * (rules) Better handle flakey platform.win32_ver() calls by calling them multiple times. +* (tools/wheelmaker.py) Extras are now preserved in Requires-Dist metadata when using requires_file + to specify the requirements. {#v0-0-0-added} ### Added diff --git a/examples/wheel/BUILD.bazel b/examples/wheel/BUILD.bazel index b434e67405..e52e0fc3a3 100644 --- a/examples/wheel/BUILD.bazel +++ b/examples/wheel/BUILD.bazel @@ -313,6 +313,17 @@ wheel; python_version == "3.11" or python_version == "3.12" # Example comment """.splitlines(), ) +write_file( + name = "requires_dist_depends_on_extras_file", + out = "requires_dist_depends_on_extras.txt", + content = """\ +# Requirements file +--index-url https://pypi.com + +extra_requires[example]==0.0.1 +""".splitlines(), +) + # py_wheel can use text files to specify their requirements. This # can be convenient for users of `compile_pip_requirements` who have # granular `requirements.in` files per package. This target shows @@ -374,6 +385,22 @@ py_wheel( deps = [":example_pkg"], ) +py_wheel( + name = "requires_dist_depends_on_extras", + distribution = "requires_dist_depends_on_extras", + requires = [ + "extra_requires[example]==0.0.1", + ], + version = "0.0.1", +) + +py_wheel( + name = "requires_dist_depends_on_extras_using_file", + distribution = "requires_dist_depends_on_extras_using_file", + requires_file = ":requires_dist_depends_on_extras.txt", + version = "0.0.1", +) + py_test( name = "wheel_test", srcs = ["wheel_test.py"], @@ -391,6 +418,8 @@ py_test( ":minimal_with_py_package", ":python_abi3_binary_wheel", ":python_requires_in_a_package", + ":requires_dist_depends_on_extras", + ":requires_dist_depends_on_extras_using_file", ":requires_files", ":use_rule_with_dir_in_outs", ], diff --git a/examples/wheel/wheel_test.py b/examples/wheel/wheel_test.py index 35803da742..43e56cfc17 100644 --- a/examples/wheel/wheel_test.py +++ b/examples/wheel/wheel_test.py @@ -565,6 +565,56 @@ def test_extra_requires(self): requires, ) + def test_requires_dist_depends_on_extras(self): + filename = self._get_path("requires_dist_depends_on_extras-0.0.1-py3-none-any.whl") + + with zipfile.ZipFile(filename) as zf: + self.assertAllEntriesHasReproducibleMetadata(zf) + metadata_file = None + for f in zf.namelist(): + if os.path.basename(f) == "METADATA": + metadata_file = f + self.assertIsNotNone(metadata_file) + + requires = [] + with zf.open(metadata_file) as fp: + for line in fp: + if line.startswith(b"Requires-Dist:"): + requires.append(line.decode("utf-8").strip()) + + print(requires) + self.assertEqual( + [ + "Requires-Dist: extra_requires[example]==0.0.1", + ], + requires, + ) + + def test_requires_dist_depends_on_extras_file(self): + filename = self._get_path("requires_dist_depends_on_extras_using_file-0.0.1-py3-none-any.whl") + + with zipfile.ZipFile(filename) as zf: + self.assertAllEntriesHasReproducibleMetadata(zf) + metadata_file = None + for f in zf.namelist(): + if os.path.basename(f) == "METADATA": + metadata_file = f + self.assertIsNotNone(metadata_file) + + requires = [] + with zf.open(metadata_file) as fp: + for line in fp: + if line.startswith(b"Requires-Dist:"): + requires.append(line.decode("utf-8").strip()) + + print(requires) + self.assertEqual( + [ + "Requires-Dist: extra_requires[example]==0.0.1", + ], + requires, + ) + if __name__ == "__main__": unittest.main() diff --git a/tools/wheelmaker.py b/tools/wheelmaker.py index 28ec039741..de584650d1 100644 --- a/tools/wheelmaker.py +++ b/tools/wheelmaker.py @@ -562,13 +562,14 @@ def main() -> None: def get_new_requirement_line(reqs_text, extra): req = Requirement(reqs_text.strip()) + req_extra_deps = f"[{','.join(req.extras)}]" if req.extras else "" if req.marker: if extra: - return f"Requires-Dist: {req.name}{req.specifier}; ({req.marker}) and {extra}" + return f"Requires-Dist: {req.name}{req_extra_deps}{req.specifier}; ({req.marker}) and {extra}" else: - return f"Requires-Dist: {req.name}{req.specifier}; {req.marker}" + return f"Requires-Dist: {req.name}{req_extra_deps}{req.specifier}; {req.marker}" else: - return f"Requires-Dist: {req.name}{req.specifier}; {extra}".strip(" ;") + return f"Requires-Dist: {req.name}{req_extra_deps}{req.specifier}; {extra}".strip(" ;") for meta_line in metadata.splitlines(): if not meta_line.startswith("Requires-Dist: "): From 8e76bd451a29d2728008a7094e850141a172cfe9 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Fri, 2 May 2025 14:46:20 -0700 Subject: [PATCH 203/922] refactor: add rule to do analysis time evaluation of environment markers (#2832) wip/prototype to help bootstrap the impl of an analysis-time flag that evaluates the pep508 dep specs Creating a PR to make collab easier (maintainers can directly edit) TODO: * Remove the todo markers after discussion Work towards https://github.com/bazel-contrib/rules_python/issues/2826 --------- Co-authored-by: Ignas Anikevicius <240938+aignas@users.noreply.github.com> --- python/private/pypi/env_marker_setting.bzl | 186 ++++++++++++++++++ python/private/pypi/pep508_env.bzl | 94 ++++++++- tests/pypi/env_marker_setting/BUILD.bazel | 5 + .../env_marker_setting_tests.bzl | 69 +++++++ 4 files changed, 351 insertions(+), 3 deletions(-) create mode 100644 python/private/pypi/env_marker_setting.bzl create mode 100644 tests/pypi/env_marker_setting/BUILD.bazel create mode 100644 tests/pypi/env_marker_setting/env_marker_setting_tests.bzl diff --git a/python/private/pypi/env_marker_setting.bzl b/python/private/pypi/env_marker_setting.bzl new file mode 100644 index 0000000000..bbc59ab110 --- /dev/null +++ b/python/private/pypi/env_marker_setting.bzl @@ -0,0 +1,186 @@ +"""Implement a flag for matching the dependency specifiers at analysis time.""" + +load("@bazel_skylib//rules:common_settings.bzl", "BuildSettingInfo") +load("//python/private:toolchain_types.bzl", "TARGET_TOOLCHAIN_TYPE") +load( + ":pep508_env.bzl", + "env_aliases", + "os_name_select_map", + "platform_machine_select_map", + "platform_system_select_map", + "sys_platform_select_map", +) +load(":pep508_evaluate.bzl", "evaluate") + +# Use capitals to hint its not an actual boolean type. +_ENV_MARKER_TRUE = "TRUE" +_ENV_MARKER_FALSE = "FALSE" + +def env_marker_setting(*, name, expression, **kwargs): + """Creates an env_marker setting. + + Generated targets: + + * `is_{name}_true`: config_setting that matches when the expression is true. + * `{name}`: env marker target that evalutes the expression. + + Args: + name: {type}`str` target name + expression: {type}`str` the environment marker string to evaluate + **kwargs: {type}`dict` additional common kwargs. + """ + native.config_setting( + name = "is_{}_true".format(name), + flag_values = { + ":{}".format(name): _ENV_MARKER_TRUE, + }, + **kwargs + ) + _env_marker_setting( + name = name, + expression = expression, + os_name = select(os_name_select_map), + sys_platform = select(sys_platform_select_map), + platform_machine = select(platform_machine_select_map), + platform_system = select(platform_system_select_map), + platform_release = select({ + "@platforms//os:osx": "USE_OSX_VERSION_FLAG", + "//conditions:default": "", + }), + **kwargs + ) + +def _env_marker_setting_impl(ctx): + env = {} + + runtime = ctx.toolchains[TARGET_TOOLCHAIN_TYPE].py3_runtime + if runtime.interpreter_version_info: + version_info = runtime.interpreter_version_info + env["python_version"] = "{major}.{minor}".format( + major = version_info.major, + minor = version_info.minor, + ) + full_version = _format_full_version(version_info) + env["python_full_version"] = full_version + env["implementation_version"] = full_version + else: + env["python_version"] = _get_flag(ctx.attr._python_version_major_minor_flag) + full_version = _get_flag(ctx.attr._python_full_version_flag) + env["python_full_version"] = full_version + env["implementation_version"] = full_version + + # We assume cpython if the toolchain doesn't specify because it's most + # likely to be true. + env["implementation_name"] = runtime.implementation_name or "cpython" + env["os_name"] = ctx.attr.os_name + env["sys_platform"] = ctx.attr.sys_platform + env["platform_machine"] = ctx.attr.platform_machine + + # The `platform_python_implementation` marker value is supposed to come + # from `platform.python_implementation()`, however, PEP 421 introduced + # `sys.implementation.name` and the `implementation_name` env marker to + # replace it. Per the platform.python_implementation docs, there's now + # essentially just two possible "registered" values: CPython or PyPy. + # Rather than add a field to the toolchain, we just special case the value + # from `sys.implementation.name` to handle the two documented values. + platform_python_impl = runtime.implementation_name + if platform_python_impl == "cpython": + platform_python_impl = "CPython" + elif platform_python_impl == "pypy": + platform_python_impl = "PyPy" + env["platform_python_implementation"] = platform_python_impl + + # NOTE: Platform release for Android will be Android version: + # https://peps.python.org/pep-0738/#platform + # Similar for iOS: + # https://peps.python.org/pep-0730/#platform + platform_release = ctx.attr.platform_release + if platform_release == "USE_OSX_VERSION_FLAG": + platform_release = _get_flag(ctx.attr._pip_whl_osx_version_flag) + env["platform_release"] = platform_release + env["platform_system"] = ctx.attr.platform_system + + # For lack of a better option, just use an empty string for now. + env["platform_version"] = "" + + env.update(env_aliases()) + + if evaluate(ctx.attr.expression, env = env): + value = _ENV_MARKER_TRUE + else: + value = _ENV_MARKER_FALSE + return [config_common.FeatureFlagInfo(value = value)] + +_env_marker_setting = rule( + doc = """ +Evaluates an environment marker expression using target configuration info. + +See +https://packaging.python.org/en/latest/specifications/dependency-specifiers +for the specification of behavior. +""", + implementation = _env_marker_setting_impl, + attrs = { + "expression": attr.string( + mandatory = True, + doc = "Environment marker expression to evaluate.", + ), + "os_name": attr.string(), + "platform_machine": attr.string(), + "platform_release": attr.string(), + "platform_system": attr.string(), + "sys_platform": attr.string(), + "_pip_whl_osx_version_flag": attr.label( + default = "//python/config_settings:pip_whl_osx_version", + providers = [[BuildSettingInfo], [config_common.FeatureFlagInfo]], + ), + "_python_full_version_flag": attr.label( + default = "//python/config_settings:python_version", + providers = [config_common.FeatureFlagInfo], + ), + "_python_version_major_minor_flag": attr.label( + default = "//python/config_settings:python_version_major_minor", + providers = [config_common.FeatureFlagInfo], + ), + }, + provides = [config_common.FeatureFlagInfo], + toolchains = [ + TARGET_TOOLCHAIN_TYPE, + ], +) + +def _format_full_version(info): + """Format the full python interpreter version. + + Adapted from spec code at: + https://packaging.python.org/en/latest/specifications/dependency-specifiers/#environment-markers + + Args: + info: The provider from the Python runtime. + + Returns: + a {type}`str` with the version + """ + kind = info.releaselevel + if kind == "final": + kind = "" + serial = "" + else: + kind = kind[0] if kind else "" + serial = str(info.serial) if info.serial else "" + + return "{major}.{minor}.{micro}{kind}{serial}".format( + v = info, + major = info.major, + minor = info.minor, + micro = info.micro, + kind = kind, + serial = serial, + ) + +def _get_flag(t): + if config_common.FeatureFlagInfo in t: + return t[config_common.FeatureFlagInfo].value + if BuildSettingInfo in t: + return t[BuildSettingInfo].value + fail("Should not occur: {} does not have necessary providers") diff --git a/python/private/pypi/pep508_env.bzl b/python/private/pypi/pep508_env.bzl index 265a8e9b99..3708c46f1d 100644 --- a/python/private/pypi/pep508_env.bzl +++ b/python/private/pypi/pep508_env.bzl @@ -18,7 +18,7 @@ load(":pep508_platform.bzl", "platform_from_str") # See https://stackoverflow.com/a/45125525 -_platform_machine_aliases = { +platform_machine_aliases = { # These pairs mean the same hardware, but different values may be used # on different host platforms. "amd64": "x86_64", @@ -27,6 +27,41 @@ _platform_machine_aliases = { "i686": "x86_32", } +# NOTE: There are many cpus, and unfortunately, the value isn't directly +# accessible to Starlark. Using CcToolchain.cpu might work, though. +platform_machine_select_map = { + "@platforms//cpu:aarch32": "aarch32", + "@platforms//cpu:aarch64": "aarch64", + "@platforms//cpu:arm": "arm", + "@platforms//cpu:arm64": "arm64", + "@platforms//cpu:arm64_32": "arm64_32", + "@platforms//cpu:arm64e": "arm64e", + "@platforms//cpu:armv6-m": "armv6-m", + "@platforms//cpu:armv7": "armv7", + "@platforms//cpu:armv7-m": "armv7-m", + "@platforms//cpu:armv7e-m": "armv7e-m", + "@platforms//cpu:armv7e-mf": "armv7e-mf", + "@platforms//cpu:armv7k": "armv7k", + "@platforms//cpu:armv8-m": "armv8-m", + "@platforms//cpu:cortex-r52": "cortex-r52", + "@platforms//cpu:cortex-r82": "cortex-r82", + "@platforms//cpu:i386": "i386", + "@platforms//cpu:mips64": "mips64", + "@platforms//cpu:ppc": "ppc", + "@platforms//cpu:ppc32": "ppc32", + "@platforms//cpu:ppc64le": "ppc64le", + "@platforms//cpu:riscv32": "riscv32", + "@platforms//cpu:riscv64": "riscv64", + "@platforms//cpu:s390x": "s390x", + "@platforms//cpu:wasm32": "wasm32", + "@platforms//cpu:wasm64": "wasm64", + "@platforms//cpu:x86_32": "x86_32", + "@platforms//cpu:x86_64": "x86_64", + # The value is empty string if it cannot be determined: + # https://docs.python.org/3/library/platform.html#platform.machine + "//conditions:default": "", +} + # Platform system returns results from the `uname` call. _platform_system_values = { "linux": "Linux", @@ -34,6 +69,23 @@ _platform_system_values = { "windows": "Windows", } +platform_system_select_map = { + # See https://peps.python.org/pep-0738/#platform + "@platforms//os:android": "Android", + "@platforms//os:freebsd": "FreeBSD", + # See https://peps.python.org/pep-0730/#platform + # NOTE: Per Pep 730, "iPadOS" is also an acceptable value + "@platforms//os:ios": "iOS", + "@platforms//os:linux": "Linux", + "@platforms//os:netbsd": "NetBSD", + "@platforms//os:openbsd": "OpenBSD", + "@platforms//os:osx": "Darwin", + "@platforms//os:windows": "Windows", + # The value is empty string if it cannot be determined: + # https://docs.python.org/3/library/platform.html#platform.machine + "//conditions:default": "", +} + # The copy of SO [answer](https://stackoverflow.com/a/13874620) containing # all of the platforms: # ┍━━━━━━━━━━━━━━━━━━━━━┯━━━━━━━━━━━━━━━━━━━━━┑ @@ -64,12 +116,45 @@ _sys_platform_values = { "osx": "darwin", "windows": "win32", } + +# Taken from +# https://docs.python.org/3/library/sys.html#sys.platform +sys_platform_select_map = { + # These values are decided by the sys.platform docs. + "@platforms//os:android": "android", + "@platforms//os:emscripten": "emscripten", + # NOTE: The below values are approximations. The sys.platform() docs + # don't have documented values for these OSes. Per docs, the + # sys.platform() value reflects the OS at the time Python was *built* + # instead of the runtime (target) OS value. + "@platforms//os:freebsd": "freebsd", + "@platforms//os:ios": "ios", + "@platforms//os:linux": "linux", + "@platforms//os:openbsd": "openbsd", + "@platforms//os:osx": "darwin", + "@platforms//os:wasi": "wasi", + "@platforms//os:windows": "win32", + # For lack of a better option, use empty string. No standard doc/spec + # about sys_platform value. + "//conditions:default": "", +} + _os_name_values = { "linux": "posix", "osx": "posix", "windows": "nt", } +os_name_select_map = { + # The "java" value is documented, but with Jython defunct, + # shouldn't occur in practice. + # The os.name value is technically a property of the runtime, not the + # targetted runtime OS, but the distinction shouldn't matter if + # things are properly configured. + "@platforms//os:windows": "nt", + "//conditions:default": "posix", +} + def env(target_platform, *, extra = None): """Return an env target platform @@ -113,8 +198,11 @@ def env(target_platform, *, extra = None): } # This is split by topic - return env | { + return env | env_aliases() + +def env_aliases(): + return { "_aliases": { - "platform_machine": _platform_machine_aliases, + "platform_machine": platform_machine_aliases, }, } diff --git a/tests/pypi/env_marker_setting/BUILD.bazel b/tests/pypi/env_marker_setting/BUILD.bazel new file mode 100644 index 0000000000..9605e650ce --- /dev/null +++ b/tests/pypi/env_marker_setting/BUILD.bazel @@ -0,0 +1,5 @@ +load(":env_marker_setting_tests.bzl", "env_marker_setting_test_suite") + +env_marker_setting_test_suite( + name = "env_marker_setting_tests", +) diff --git a/tests/pypi/env_marker_setting/env_marker_setting_tests.bzl b/tests/pypi/env_marker_setting/env_marker_setting_tests.bzl new file mode 100644 index 0000000000..549c15c20b --- /dev/null +++ b/tests/pypi/env_marker_setting/env_marker_setting_tests.bzl @@ -0,0 +1,69 @@ +"""env_marker_setting tests.""" + +load("@rules_testing//lib:analysis_test.bzl", "analysis_test") +load("@rules_testing//lib:test_suite.bzl", "test_suite") +load("@rules_testing//lib:util.bzl", "TestingAspectInfo") +load("//python/private/pypi:env_marker_setting.bzl", "env_marker_setting") # buildifier: disable=bzl-visibility +load("//tests/support:support.bzl", "PYTHON_VERSION") + +_tests = [] + +def _test_expr(name): + def impl(env, target): + env.expect.where( + expression = target[TestingAspectInfo].attrs.expression, + ).that_str( + target[config_common.FeatureFlagInfo].value, + ).equals( + env.ctx.attr.expected, + ) + + cases = { + "python_full_version_lt_negative": { + "config_settings": { + PYTHON_VERSION: "3.12.0", + }, + "expected": "FALSE", + "expression": "python_full_version < '3.8'", + }, + "python_version_gte": { + "config_settings": { + PYTHON_VERSION: "3.12.0", + }, + "expected": "TRUE", + "expression": "python_version >= '3.12.0'", + }, + } + + tests = [] + for case_name, case in cases.items(): + test_name = name + "_" + case_name + tests.append(test_name) + env_marker_setting( + name = test_name + "_subject", + expression = case["expression"], + ) + analysis_test( + name = test_name, + impl = impl, + target = test_name + "_subject", + config_settings = case["config_settings"], + attr_values = { + "expected": case["expected"], + }, + attrs = { + "expected": attr.string(), + }, + ) + native.test_suite( + name = name, + tests = tests, + ) + +_tests.append(_test_expr) + +def env_marker_setting_test_suite(name): + test_suite( + name = name, + tests = _tests, + ) From ccbe5dcdb84a2c194deaf34165e43201e17a3826 Mon Sep 17 00:00:00 2001 From: Tobias Fuchs <9053039+devtbi@users.noreply.github.com> Date: Sat, 3 May 2025 05:22:48 +0200 Subject: [PATCH 204/922] py_wheel: always generate zip64-capable wheels (#2711) Currently, there is no possibility to pass the force zip64 option to the wheel creation. This hinders creation of packages that contain >2Gb files (e.g. large projects with debug symbols). To fix, always generate zip64 capable wheels. zip64 support is wide spread. Fixes https://github.com/bazel-contrib/rules_python/issues/2852 --------- Co-authored-by: Richard Levasseur Co-authored-by: Richard Levasseur --- CHANGELOG.md | 2 ++ examples/wheel/test_publish.py | 2 +- examples/wheel/wheel_test.py | 16 ++++++++-------- tools/wheelmaker.py | 2 +- 4 files changed, 12 insertions(+), 10 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 19fe636bc3..17e3cd3c86 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -54,12 +54,14 @@ END_UNRELEASED_TEMPLATE {#v0-0-0-changed} ### Changed + * (rules) On Windows, {obj}`--bootstrap_impl=system_python` is forced. This allows setting `--bootstrap_impl=script` in bazelrc for mixed-platform environments. * (rules) {obj}`pip_compile` now generates a `.test` target. The `_test` target is deprecated and will be removed in the next major release. ([#2794](https://github.com/bazel-contrib/rules_python/issues/2794) +* (py_wheel) py_wheel always creates zip64-capable wheel zips {#v0-0-0-fixed} ### Fixed diff --git a/examples/wheel/test_publish.py b/examples/wheel/test_publish.py index e6ec80721b..7665629c19 100644 --- a/examples/wheel/test_publish.py +++ b/examples/wheel/test_publish.py @@ -104,7 +104,7 @@ def test_upload_and_query_simple_api(self):

Links for example-minimal-library

- example_minimal_library-0.0.1-py3-none-any.whl
+ example_minimal_library-0.0.1-py3-none-any.whl
""" self.assertEqual( diff --git a/examples/wheel/wheel_test.py b/examples/wheel/wheel_test.py index 43e56cfc17..7f19ecd9f9 100644 --- a/examples/wheel/wheel_test.py +++ b/examples/wheel/wheel_test.py @@ -85,7 +85,7 @@ def test_py_library_wheel(self): ], ) self.assertFileSha256Equal( - filename, "a73acae23590c7a8d4365c888c1f12f0399b7af27169ea99fc7a00f402833926" + filename, "ef5afd9f6c3ff569ef7e5b2799d3a2ec9675d029414f341e0abd7254d6b9a25d" ) def test_py_package_wheel(self): @@ -110,7 +110,7 @@ def test_py_package_wheel(self): ], ) self.assertFileSha256Equal( - filename, "a76001500453dbd1d778821dcaba165d56db502c854cef9381dd3f8f89caee11" + filename, "39bec133cf79431e8d057eae550cd91aa9dfbddfedb53d98ebd36e3ade2753d0" ) def test_customized_wheel(self): @@ -206,7 +206,7 @@ def test_customized_wheel(self): second = second.main:s""", ) self.assertFileSha256Equal( - filename, "941c0d79f4ca67cfa0028248bd0606db7fc69953ff9c7c73ac26a3e6d3c23587" + filename, "685f68fc6665f53c9b769fd1ba12cce9937ab7f40ef4e60c82ef2de8653935de" ) def test_filename_escaping(self): @@ -278,7 +278,7 @@ def test_custom_package_root_wheel(self): for line in record_contents.splitlines(): self.assertFalse(line.startswith("/")) self.assertFileSha256Equal( - filename, "7bd959b7efe9e325b30a6559177a1a4f22ac7a68fade310845916276110e9287" + filename, "2fbfc3baaf6fccca0f97d02316b8344507fe6c8136991a66ee5f162235adb19f" ) def test_custom_package_root_multi_prefix_wheel(self): @@ -312,7 +312,7 @@ def test_custom_package_root_multi_prefix_wheel(self): for line in record_contents.splitlines(): self.assertFalse(line.startswith("/")) self.assertFileSha256Equal( - filename, "caf51e22bdcd3c6c766c8903319ce717daeb6caac577d14e16326a8597981854" + filename, "3e67971ca1e8a9ba36a143df7532e641f5661c56235e41d818309316c955ba58" ) def test_custom_package_root_multi_prefix_reverse_order_wheel(self): @@ -346,7 +346,7 @@ def test_custom_package_root_multi_prefix_reverse_order_wheel(self): for line in record_contents.splitlines(): self.assertFalse(line.startswith("/")) self.assertFileSha256Equal( - filename, "9e8c0baa408b829dec691a5e8d3bc040be0bbfcc95c0eee19e1e5ffadea4a059" + filename, "372ef9e11fb79f1952172993718a326b5adda192d94884b54377c34b44394982" ) def test_python_requires_wheel(self): @@ -371,7 +371,7 @@ def test_python_requires_wheel(self): """, ) self.assertFileSha256Equal( - filename, "b47f3eaf4f9fa4685a58c7415ba1feddd39635ae26c18473504f7d7e62e8ce07" + filename, "10a325ba8f77428b5cfcff6345d508f5eb77c140889eb62490d7382f60d4ebfe" ) def test_python_abi3_binary_wheel(self): @@ -436,7 +436,7 @@ def test_rule_creates_directory_and_is_included_in_wheel(self): ], ) self.assertFileSha256Equal( - filename, "d8e874b807e5574bd11a9312c58ce7fe7055afb80412d0d0e7ed21fc9223cd53" + filename, "85e44c43cc19ccae9fe2e1d629230203aa11791bed1f7f68a069fb58d1c93cd2" ) def test_rule_expands_workspace_status_keys_in_wheel_metadata(self): diff --git a/tools/wheelmaker.py b/tools/wheelmaker.py index de584650d1..8b775e1541 100644 --- a/tools/wheelmaker.py +++ b/tools/wheelmaker.py @@ -154,7 +154,7 @@ def arcname_from(name): hash = hashlib.sha256() size = 0 with open(real_filename, "rb") as fsrc: - with self.open(zinfo, "w") as fdst: + with self.open(zinfo, "w", force_zip64=True) as fdst: while True: block = fsrc.read(2**20) if not block: From 4ccf5b23be8e2396b1fc358f1d83d1b7923c5ea7 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Sat, 3 May 2025 01:37:15 -0700 Subject: [PATCH 205/922] feat: allow specifying arbitrary constraints for local toolchains (#2829) This adds the ability for local toolchains to have arbitrary constraints set on them. This allows accomplishing two goals: 1. Makes it easier to enable/disable them on the command line, instead of having them entirely override an existing config and having to comment/uncomment the MODULE.bazel file sections. 2. Allows configuring them so that the repository is never initialized, which avoids the repository from being initialized during toolchain resolution, even if it will never match because of (1). --- .bazelci/presubmit.yml | 2 + CHANGELOG.md | 2 + docs/toolchains.md | 73 +++++++++++- .../private/local_runtime_toolchains_repo.bzl | 109 ++++++++++++++++++ python/private/py_toolchain_suite.bzl | 71 ++++++++++-- python/private/text_util.bzl | 5 + tests/integration/local_toolchains/.bazelrc | 2 + .../integration/local_toolchains/BUILD.bazel | 15 +++ .../integration/local_toolchains/MODULE.bazel | 13 +++ 9 files changed, 278 insertions(+), 14 deletions(-) diff --git a/.bazelci/presubmit.yml b/.bazelci/presubmit.yml index 3b70734eff..7e9d4dea53 100644 --- a/.bazelci/presubmit.yml +++ b/.bazelci/presubmit.yml @@ -51,9 +51,11 @@ buildifier: test_flags: - "--noenable_bzlmod" - "--enable_workspace" + - "--test_tag_filters=-integration-test" build_flags: - "--noenable_bzlmod" - "--enable_workspace" + - "--build_tag_filters=-integration-test" bazel: 7.x .common_bazelinbazel_config: &common_bazelinbazel_config build_flags: diff --git a/CHANGELOG.md b/CHANGELOG.md index 17e3cd3c86..d9cb14459d 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -84,6 +84,8 @@ END_UNRELEASED_TEMPLATE * Repo utilities `execute_unchecked`, `execute_checked`, and `execute_checked_stdout` now support `log_stdout` and `log_stderr` keyword arg booleans. When these are `True` (the default), the subprocess's stdout/stderr will be logged. +* (toolchains) Local toolchains can be activated with custom flags. See + [Conditionally using local toolchains] docs for how to configure. {#v0-0-0-removed} ### Removed diff --git a/docs/toolchains.md b/docs/toolchains.md index 2f8db66595..c8305e8f0d 100644 --- a/docs/toolchains.md +++ b/docs/toolchains.md @@ -377,15 +377,14 @@ local_runtime_repo( local_runtime_toolchains_repo( name = "local_toolchains", runtimes = ["local_python3"], + # TIP: The `target_settings` arg can be used to activate them based on + # command line flags; see docs below. ) # Step 3: Register the toolchains register_toolchains("@local_toolchains//:all", dev_dependency = True) ``` -Note that `register_toolchains` will insert the local toolchain earlier in the -toolchain ordering, so it will take precedence over other registered toolchains. - :::{important} Be sure to set `dev_dependency = True`. Using a local toolchain only makes sense for the root module. @@ -397,6 +396,72 @@ downstream modules. Multiple runtimes and/or toolchains can be defined, which allows for multiple Python versions and/or platforms to be configured in a single `MODULE.bazel`. +Note that `register_toolchains` will insert the local toolchain earlier in the +toolchain ordering, so it will take precedence over other registered toolchains. +To better control when the toolchain is used, see [Conditionally using local +toolchains] + +### Conditionally using local toolchains + +By default, a local toolchain has few constraints and is early in the toolchain +ordering, which means it will usually be used no matter what. This can be +problematic for CI (where it shouldn't be used), expensive for CI (CI must +initialize/download the repository to determine its Python version), and +annoying for iterative development (enabling/disabling it requires modifying +MODULE.bazel). + +These behaviors can be mitigated, but it requires additional configuration +to avoid triggering the local toolchain repository to initialize (i.e. run +local commands and perform downloads). + +The two settings to change are +{obj}`local_runtime_toolchains_repo.target_compatible_with` and +{obj}`local_runtime_toolchains_repo.target_settings`, which control how Bazel +decides if a toolchain should match. By default, they point to targets *within* +the local runtime repository (trigger repo initialization). We have to override +them to *not* reference the local runtime repository at all. + +In the example below, we reconfigure the local toolchains so they are only +activated if the custom flag `--//:py=local` is set and the target platform +matches the Bazel host platform. The net effect is CI won't use the local +toolchain (nor initialize its repository), and developers can easily +enable/disable the local toolchain with a command line flag. + +``` +# File: MODULE.bazel +bazel_dep(name = "bazel_skylib", version = "1.7.1") + +local_runtime_toolchains_repo( + name = "local_toolchains", + runtimes = ["local_python3"], + target_compatible_with = { + "local_python3": ["HOST_CONSTRAINTS"], + }, + target_settings = { + "local_python3": ["@//:is_py_local"] + } +) + +# File: BUILD.bazel +load("@bazel_skylib//rules:common_settings.bzl", "string_flag") + +config_setting( + name = "is_py_local", + flag_values = {":py": "local"}, +) + +string_flag( + name = "py", + build_setting_default = "", +) +``` + +:::{tip} +Easily switching between *multiple* local toolchains can be accomplished by +adding additional `:is_py_X` targets and setting `--//:py` to match. +to easily switch between different local toolchains. +::: + ## Runtime environment toolchain @@ -425,7 +490,7 @@ locally installed Python. ### Autodetecting toolchain The autodetecting toolchain is a deprecated toolchain that is built into Bazel. -It's name is a bit misleading: it doesn't autodetect anything. All it does is +**It's name is a bit misleading: it doesn't autodetect anything**. All it does is use `python3` from the environment a binary runs within. This provides extremely limited functionality to the rules (at build time, nothing is knowable about the Python runtime). diff --git a/python/private/local_runtime_toolchains_repo.bzl b/python/private/local_runtime_toolchains_repo.bzl index adb3bb560d..004ca664ad 100644 --- a/python/private/local_runtime_toolchains_repo.bzl +++ b/python/private/local_runtime_toolchains_repo.bzl @@ -26,6 +26,9 @@ define_local_toolchain_suites( name = "toolchains", version_aware_repo_names = {version_aware_names}, version_unaware_repo_names = {version_unaware_names}, + repo_exec_compatible_with = {repo_exec_compatible_with}, + repo_target_compatible_with = {repo_target_compatible_with}, + repo_target_settings = {repo_target_settings}, ) """ @@ -39,6 +42,9 @@ def _local_runtime_toolchains_repo(rctx): rctx.file("BUILD.bazel", _TOOLCHAIN_TEMPLATE.format( version_aware_names = render.list(rctx.attr.runtimes), + repo_target_settings = render.string_list_dict(rctx.attr.target_settings), + repo_target_compatible_with = render.string_list_dict(rctx.attr.target_compatible_with), + repo_exec_compatible_with = render.string_list_dict(rctx.attr.exec_compatible_with), version_unaware_names = render.list(rctx.attr.default_runtimes or rctx.attr.runtimes), )) @@ -62,8 +68,36 @@ These will be defined as *version-unaware* toolchains. This means they will match any Python version. As such, they are registered after the version-aware toolchains defined by the `runtimes` attribute. +If not set, then the `runtimes` values will be used. + Note that order matters: it determines the toolchain priority within the package. +""", + ), + "exec_compatible_with": attr.string_list_dict( + doc = """ +Constraints that must be satisfied by an exec platform for a toolchain to be used. + +This is a `dict[str, list[str]]`, where the keys are repo names from the +`runtimes` or `default_runtimes` args, and the values are constraint +target labels (e.g. OS, CPU, etc). + +:::{note} +Specify `@//foo:bar`, not simply `//foo:bar` or `:bar`. The additional `@` is +needed because the strings are evaluated in a different context than where +they originate. +::: + +The list of settings become the {obj}`toolchain.exec_compatible_with` value for +each respective repo. + +This allows a local toolchain to only be used if certain exec platform +conditions are met, typically values from `@platforms`. + +See the [Local toolchains] docs for examples and further information. + +:::{versionadded} VERSION_NEXT_FEATURE +::: """, ), "runtimes": attr.string_list( @@ -76,6 +110,81 @@ are registered before `default_runtimes`. Note that order matters: it determines the toolchain priority within the package. +""", + ), + "target_compatible_with": attr.string_list_dict( + doc = """ +Constraints that must be satisfied for a toolchain to be used. + + +This is a `dict[str, list[str]]`, where the keys are repo names from the +`runtimes` or `default_runtimes` args, and the values are constraint +target labels (e.g. OS, CPU, etc), or the special string `"HOST_CONSTRAINTS"` +(which will be replaced with the current Bazel hosts's constraints). + +If a repo's entry is missing or empty, it defaults to the supported OS the +underlying runtime repository detects as compatible. + +:::{note} +Specify `@//foo:bar`, not simply `//foo:bar` or `:bar`. The additional `@` is +needed because the strings are evaluated in a different context than where +they originate. +::: + +The list of settings **becomes the** the {obj}`toolchain.target_compatible_with` +value for each respective repo; i.e. they _replace_ the auto-detected values +the local runtime itself computes. + +This allows a local toolchain to only be used if certain target platform +conditions are met, typically values from `@platforms`. + +See the [Local toolchains] docs for examples and further information. + +:::{seealso} +The `target_settings` attribute, which handles `config_setting` values, +instead of constraints. +::: + +:::{versionadded} VERSION_NEXT_FEATURE +::: +""", + ), + "target_settings": attr.string_list_dict( + doc = """ +Config settings that must be satisfied for a toolchain to be used. + +This is a `dict[str, list[str]]`, where the keys are repo names from the +`runtimes` or `default_runtimes` args, and the values are {obj}`config_setting()` +target labels. + +If a repo's entry is missing or empty, it will default to +`@//:is_match_python_version` (for repos in `runtimes`) or an empty list +(for repos in `default_runtimes`). + +:::{note} +Specify `@//foo:bar`, not simply `//foo:bar` or `:bar`. The additional `@` is +needed because the strings are evaluated in a different context than where +they originate. +::: + +The list of settings will be applied atop of any of the local runtime's +settings that are used for {obj}`toolchain.target_settings`. i.e. they are +evaluated first and guard the checking of the local runtime's auto-detected +conditions. + +This allows a local toolchain to only be used if certain flags or +config setting conditions are met. Such conditions can include user-defined +flags, platform constraints, etc. + +See the [Local toolchains] docs for examples and further information. + +:::{seealso} +The `target_compatible_with` attribute, which handles *constraint* values, +instead of `config_settings`. +::: + +:::{versionadded} VERSION_NEXT_FEATURE +::: """, ), "_rule_name": attr.string(default = "local_toolchains_repo"), diff --git a/python/private/py_toolchain_suite.bzl b/python/private/py_toolchain_suite.bzl index a69be376b4..e71882dafd 100644 --- a/python/private/py_toolchain_suite.bzl +++ b/python/private/py_toolchain_suite.bzl @@ -15,6 +15,7 @@ """Create the toolchain defs in a BUILD.bazel file.""" load("@bazel_skylib//lib:selects.bzl", "selects") +load("@platforms//host:constraints.bzl", "HOST_CONSTRAINTS") load(":text_util.bzl", "render") load( ":toolchain_types.bzl", @@ -95,9 +96,15 @@ def py_toolchain_suite( runtime_repo_name = user_repository_name, target_settings = target_settings, target_compatible_with = target_compatible_with, + exec_compatible_with = [], ) -def _internal_toolchain_suite(prefix, runtime_repo_name, target_compatible_with, target_settings): +def _internal_toolchain_suite( + prefix, + runtime_repo_name, + target_compatible_with, + target_settings, + exec_compatible_with): native.toolchain( name = "{prefix}_toolchain".format(prefix = prefix), toolchain = "@{runtime_repo_name}//:python_runtimes".format( @@ -106,6 +113,7 @@ def _internal_toolchain_suite(prefix, runtime_repo_name, target_compatible_with, toolchain_type = TARGET_TOOLCHAIN_TYPE, target_settings = target_settings, target_compatible_with = target_compatible_with, + exec_compatible_with = exec_compatible_with, ) native.toolchain( @@ -116,6 +124,7 @@ def _internal_toolchain_suite(prefix, runtime_repo_name, target_compatible_with, toolchain_type = PY_CC_TOOLCHAIN_TYPE, target_settings = target_settings, target_compatible_with = target_compatible_with, + exec_compatible_with = exec_compatible_with, ) native.toolchain( @@ -142,7 +151,13 @@ def _internal_toolchain_suite(prefix, runtime_repo_name, target_compatible_with, # call in python/repositories.bzl. Bzlmod doesn't need anything; it will # register `:all`. -def define_local_toolchain_suites(name, version_aware_repo_names, version_unaware_repo_names): +def define_local_toolchain_suites( + name, + version_aware_repo_names, + version_unaware_repo_names, + repo_exec_compatible_with, + repo_target_compatible_with, + repo_target_settings): """Define toolchains for `local_runtime_repo` backed toolchains. This generates `toolchain` targets that can be registered using `:all`. The @@ -156,24 +171,60 @@ def define_local_toolchain_suites(name, version_aware_repo_names, version_unawar version-aware toolchains defined. version_unaware_repo_names: `list[str]` of the repo names that will have version-unaware toolchains defined. + repo_target_settings: {type}`dict[str, list[str]]` mapping of repo names + to string labels that are added to the `target_settings` for the + respective repo's toolchain. + repo_target_compatible_with: {type}`dict[str, list[str]]` mapping of repo names + to string labels that are added to the `target_compatible_with` for + the respective repo's toolchain. + repo_exec_compatible_with: {type}`dict[str, list[str]]` mapping of repo names + to string labels that are added to the `exec_compatible_with` for + the respective repo's toolchain. """ + i = 0 for i, repo in enumerate(version_aware_repo_names, start = i): - prefix = render.left_pad_zero(i, 4) + target_settings = ["@{}//:is_matching_python_version".format(repo)] + + if repo_target_settings.get(repo): + selects.config_setting_group( + name = "_{}_user_guard".format(repo), + match_all = repo_target_settings.get(repo, []) + target_settings, + ) + target_settings = ["_{}_user_guard".format(repo)] _internal_toolchain_suite( - prefix = prefix, + prefix = render.left_pad_zero(i, 4), runtime_repo_name = repo, - target_compatible_with = ["@{}//:os".format(repo)], - target_settings = ["@{}//:is_matching_python_version".format(repo)], + target_compatible_with = _get_local_toolchain_target_compatible_with( + repo, + repo_target_compatible_with, + ), + target_settings = target_settings, + exec_compatible_with = repo_exec_compatible_with.get(repo, []), ) # The version unaware entries must go last because they will match any Python # version. for i, repo in enumerate(version_unaware_repo_names, start = i + 1): - prefix = render.left_pad_zero(i, 4) _internal_toolchain_suite( - prefix = prefix, + prefix = render.left_pad_zero(i, 4) + "_default", runtime_repo_name = repo, - target_settings = [], - target_compatible_with = ["@{}//:os".format(repo)], + target_compatible_with = _get_local_toolchain_target_compatible_with( + repo, + repo_target_compatible_with, + ), + # We don't call _get_local_toolchain_target_settings because that + # will add the version matching condition by default. + target_settings = repo_target_settings.get(repo, []), + exec_compatible_with = repo_exec_compatible_with.get(repo, []), ) + +def _get_local_toolchain_target_compatible_with(repo, repo_target_compatible_with): + if repo in repo_target_compatible_with: + target_compatible_with = repo_target_compatible_with[repo] + if "HOST_CONSTRAINTS" in target_compatible_with: + target_compatible_with.remove("HOST_CONSTRAINTS") + target_compatible_with.extend(HOST_CONSTRAINTS) + else: + target_compatible_with = ["@{}//:os".format(repo)] + return target_compatible_with diff --git a/python/private/text_util.bzl b/python/private/text_util.bzl index a64b5d6243..28979d8981 100644 --- a/python/private/text_util.bzl +++ b/python/private/text_util.bzl @@ -108,6 +108,10 @@ def _render_list(items, *, hanging_indent = ""): def _render_str(value): return repr(value) +def _render_string_list_dict(value): + """Render an attr.string_list_dict value (`dict[str, list[str]`)""" + return _render_dict(value, value_repr = _render_list) + def _render_tuple(items, *, value_repr = repr): if not items: return "tuple()" @@ -166,4 +170,5 @@ render = struct( str = _render_str, toolchain_prefix = _toolchain_prefix, tuple = _render_tuple, + string_list_dict = _render_string_list_dict, ) diff --git a/tests/integration/local_toolchains/.bazelrc b/tests/integration/local_toolchains/.bazelrc index 39df41d9f4..aed08b0790 100644 --- a/tests/integration/local_toolchains/.bazelrc +++ b/tests/integration/local_toolchains/.bazelrc @@ -4,3 +4,5 @@ test --test_output=errors # Windows requires these for multi-python support: build --enable_runfiles common:bazel7.x --incompatible_python_disallow_native_rules +build --//:py=local +common --announce_rc diff --git a/tests/integration/local_toolchains/BUILD.bazel b/tests/integration/local_toolchains/BUILD.bazel index 02b126b0ea..6b731181a6 100644 --- a/tests/integration/local_toolchains/BUILD.bazel +++ b/tests/integration/local_toolchains/BUILD.bazel @@ -12,6 +12,7 @@ # See the License for the specific language governing permissions and # limitations under the License. +load("@bazel_skylib//rules:common_settings.bzl", "string_flag") load("@rules_python//python:py_test.bzl", "py_test") py_test( @@ -20,3 +21,17 @@ py_test( # Make this test better respect pyenv env_inherit = ["PYENV_VERSION"], ) + +config_setting( + name = "is_py_local", + flag_values = { + ":py": "local", + }, +) + +# Set `--//:py=local` to use the local toolchain +# (This is set in this example's .bazelrc) +string_flag( + name = "py", + build_setting_default = "", +) diff --git a/tests/integration/local_toolchains/MODULE.bazel b/tests/integration/local_toolchains/MODULE.bazel index 98f1ed9ac4..6c06909cd7 100644 --- a/tests/integration/local_toolchains/MODULE.bazel +++ b/tests/integration/local_toolchains/MODULE.bazel @@ -14,6 +14,9 @@ module(name = "module_under_test") bazel_dep(name = "rules_python", version = "0.0.0") +bazel_dep(name = "bazel_skylib", version = "1.7.1") +bazel_dep(name = "platforms", version = "0.0.11") + local_path_override( module_name = "rules_python", path = "../../..", @@ -32,6 +35,16 @@ local_runtime_repo( local_runtime_toolchains_repo( name = "local_toolchains", runtimes = ["local_python3"], + target_compatible_with = { + "local_python3": [ + "HOST_CONSTRAINTS", + ], + }, + target_settings = { + "local_python3": [ + "@//:is_py_local", + ], + }, ) python = use_extension("@rules_python//python/extensions:python.bzl", "python") From a4b946bbe1b3e83ca4602a0d059fea823b0ded65 Mon Sep 17 00:00:00 2001 From: Ignas Anikevicius <240938+aignas@users.noreply.github.com> Date: Mon, 5 May 2025 14:22:38 +0900 Subject: [PATCH 206/922] feat: add an env variable to toggle pipstar (#2855) This is a flag to start leveraging of the new code paths. The Starlark implementation has been added in 1.4 and has been reverted in the latest release candidates. The `env` variable will be a good way to roll it out more gradually and get more testing. For now we are switching only the `whl_library` internals as the `requirements.txt` files from `uv` may use `*` in `python_full_version` and `platform_version` that are not yet fully supported (#2826). Main goals for this is to start using Starlark implementation so that we don't have any hidden variables. What is more, having this in Starlark is the most maintainable long-term solution for supporting cross-platform builds. Work towards #260 --------- Co-authored-by: Richard Levasseur --- CHANGELOG.md | 3 + docs/environment-variables.md | 9 + python/private/internal_config_repo.bzl | 4 + .../private/pypi/whl_installer/arguments.py | 5 + .../pypi/whl_installer/wheel_installer.py | 44 ++-- python/private/pypi/whl_library.bzl | 207 ++++++++++++------ .../whl_installer/wheel_installer_test.py | 1 + 7 files changed, 187 insertions(+), 86 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index d9cb14459d..7d73613a07 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -86,6 +86,9 @@ END_UNRELEASED_TEMPLATE (the default), the subprocess's stdout/stderr will be logged. * (toolchains) Local toolchains can be activated with custom flags. See [Conditionally using local toolchains] docs for how to configure. +* (pypi) `RULES_PYTHON_ENABLE_PIPSTAR` environment variable: when `1`, the Starlark + implementation of wheel METADATA parsing is used (which has improved multi-platform + build support). {#v0-0-0-removed} ### Removed diff --git a/docs/environment-variables.md b/docs/environment-variables.md index 49fdf766f6..26c171095d 100644 --- a/docs/environment-variables.md +++ b/docs/environment-variables.md @@ -60,6 +60,15 @@ The default became `1` if unspecified ::: :::: +::::{envvar} RULES_PYTHON_ENABLE_PIPSTAR + +When `1`, the rules_python Starlark implementation of the pypi/pip integration is used +instead of the legacy Python scripts. + +:::{versionadded} VERSION_NEXT_FEATURE +::: +:::: + ::::{envvar} RULES_PYTHON_EXTRACT_ROOT Directory to use as the root for creating files necessary for bootstrapping so diff --git a/python/private/internal_config_repo.bzl b/python/private/internal_config_repo.bzl index a5c4787161..cfe2fdfd77 100644 --- a/python/private/internal_config_repo.bzl +++ b/python/private/internal_config_repo.bzl @@ -20,6 +20,8 @@ settings for rules to later use. load(":repo_utils.bzl", "repo_utils") +_ENABLE_PIPSTAR_ENVVAR_NAME = "RULES_PYTHON_ENABLE_PIPSTAR" +_ENABLE_PIPSTAR_DEFAULT = "0" _ENABLE_PYSTAR_ENVVAR_NAME = "RULES_PYTHON_ENABLE_PYSTAR" _ENABLE_PYSTAR_DEFAULT = "1" _ENABLE_DEPRECATION_WARNINGS_ENVVAR_NAME = "RULES_PYTHON_DEPRECATION_WARNINGS" @@ -28,6 +30,7 @@ _ENABLE_DEPRECATION_WARNINGS_DEFAULT = "0" _CONFIG_TEMPLATE = """\ config = struct( enable_pystar = {enable_pystar}, + enable_pipstar = {enable_pipstar}, enable_deprecation_warnings = {enable_deprecation_warnings}, BuiltinPyInfo = getattr(getattr(native, "legacy_globals", None), "PyInfo", {builtin_py_info_symbol}), BuiltinPyRuntimeInfo = getattr(getattr(native, "legacy_globals", None), "PyRuntimeInfo", {builtin_py_runtime_info_symbol}), @@ -84,6 +87,7 @@ def _internal_config_repo_impl(rctx): rctx.file("rules_python_config.bzl", _CONFIG_TEMPLATE.format( enable_pystar = enable_pystar, + enable_pipstar = _bool_from_environ(rctx, _ENABLE_PIPSTAR_ENVVAR_NAME, _ENABLE_PIPSTAR_DEFAULT), enable_deprecation_warnings = _bool_from_environ(rctx, _ENABLE_DEPRECATION_WARNINGS_ENVVAR_NAME, _ENABLE_DEPRECATION_WARNINGS_DEFAULT), builtin_py_info_symbol = builtin_py_info_symbol, builtin_py_runtime_info_symbol = builtin_py_runtime_info_symbol, diff --git a/python/private/pypi/whl_installer/arguments.py b/python/private/pypi/whl_installer/arguments.py index 29bea8026e..ea609bef9d 100644 --- a/python/private/pypi/whl_installer/arguments.py +++ b/python/private/pypi/whl_installer/arguments.py @@ -47,6 +47,11 @@ def parser(**kwargs: Any) -> argparse.ArgumentParser: type=Platform.from_string, help="Platforms to target dependencies. Can be used multiple times.", ) + parser.add_argument( + "--enable-pipstar", + action="store_true", + help="Disable certain code paths if we expect to process the whl in Starlark.", + ) parser.add_argument( "--pip_data_exclude", action="store", diff --git a/python/private/pypi/whl_installer/wheel_installer.py b/python/private/pypi/whl_installer/wheel_installer.py index a48df699ba..2db03e039d 100644 --- a/python/private/pypi/whl_installer/wheel_installer.py +++ b/python/private/pypi/whl_installer/wheel_installer.py @@ -104,6 +104,7 @@ def _setup_namespace_pkg_compatibility(wheel_dir: str) -> None: def _extract_wheel( wheel_file: str, extras: Dict[str, Set[str]], + enable_pipstar: bool, enable_implicit_namespace_pkgs: bool, platforms: List[wheel.Platform], installation_dir: Path = Path("."), @@ -114,6 +115,7 @@ def _extract_wheel( wheel_file: the filepath of the .whl installation_dir: the destination directory for installation of the wheel. extras: a list of extras to add as dependencies for the installed wheel + enable_pipstar: if true, turns off certain operations. enable_implicit_namespace_pkgs: if true, disables conversion of implicit namespace packages and will unzip as-is """ @@ -123,26 +125,31 @@ def _extract_wheel( if not enable_implicit_namespace_pkgs: _setup_namespace_pkg_compatibility(installation_dir) - extras_requested = extras[whl.name] if whl.name in extras else set() - - dependencies = whl.dependencies(extras_requested, platforms) + metadata = { + "python_version": f"{sys.version_info[0]}.{sys.version_info[1]}.{sys.version_info[2]}", + "entry_points": [ + { + "name": name, + "module": module, + "attribute": attribute, + } + for name, (module, attribute) in sorted(whl.entry_points().items()) + ], + } + if not enable_pipstar: + extras_requested = extras[whl.name] if whl.name in extras else set() + dependencies = whl.dependencies(extras_requested, platforms) + + metadata.update( + { + "name": whl.name, + "version": whl.version, + "deps": dependencies.deps, + "deps_by_platform": dependencies.deps_select, + } + ) with open(os.path.join(installation_dir, "metadata.json"), "w") as f: - metadata = { - "name": whl.name, - "version": whl.version, - "deps": dependencies.deps, - "python_version": f"{sys.version_info[0]}.{sys.version_info[1]}.{sys.version_info[2]}", - "deps_by_platform": dependencies.deps_select, - "entry_points": [ - { - "name": name, - "module": module, - "attribute": attribute, - } - for name, (module, attribute) in sorted(whl.entry_points().items()) - ], - } json.dump(metadata, f) @@ -161,6 +168,7 @@ def main() -> None: _extract_wheel( wheel_file=whl, extras=extras, + enable_pipstar=args.enable_pipstar, enable_implicit_namespace_pkgs=args.enable_implicit_namespace_pkgs, platforms=arguments.get_platforms(args), ) diff --git a/python/private/pypi/whl_library.bzl b/python/private/pypi/whl_library.bzl index 0c09f7960a..160bb5b799 100644 --- a/python/private/pypi/whl_library.bzl +++ b/python/private/pypi/whl_library.bzl @@ -14,6 +14,7 @@ "" +load("@rules_python_internal//:rules_python_config.bzl", rp_config = "config") load("//python/private:auth.bzl", "AUTH_ATTRS", "get_auth") load("//python/private:envsubst.bzl", "envsubst") load("//python/private:is_standalone_interpreter.bzl", "is_standalone_interpreter") @@ -21,9 +22,11 @@ load("//python/private:repo_utils.bzl", "REPO_DEBUG_ENV_VAR", "repo_utils") load(":attrs.bzl", "ATTRS", "use_isolated") load(":deps.bzl", "all_repo_names", "record_files") load(":generate_whl_library_build_bazel.bzl", "generate_whl_library_build_bazel") +load(":parse_requirements.bzl", "host_platform") load(":parse_whl_name.bzl", "parse_whl_name") load(":patch_whl.bzl", "patch_whl") load(":pypi_repo_utils.bzl", "pypi_repo_utils") +load(":whl_metadata.bzl", "whl_metadata") load(":whl_target_platforms.bzl", "whl_target_platforms") _CPPFLAGS = "CPPFLAGS" @@ -340,79 +343,147 @@ def _whl_library_impl(rctx): timeout = rctx.attr.timeout, ) - target_platforms = rctx.attr.experimental_target_platforms or [] - if target_platforms: - parsed_whl = parse_whl_name(whl_path.basename) - - # NOTE @aignas 2023-12-04: if the wheel is a platform specific wheel, we - # only include deps for that target platform - if parsed_whl.platform_tag != "any": - target_platforms = [ - p.target_platform - for p in whl_target_platforms( - platform_tag = parsed_whl.platform_tag, - abi_tag = parsed_whl.abi_tag.strip("tm"), - ) - ] - - pypi_repo_utils.execute_checked( - rctx, - op = "whl_library.ExtractWheel({}, {})".format(rctx.attr.name, whl_path), - python = python_interpreter, - arguments = args + [ - "--whl-file", - whl_path, - ] + ["--platform={}".format(p) for p in target_platforms], - srcs = rctx.attr._python_srcs, - environment = environment, - quiet = rctx.attr.quiet, - timeout = rctx.attr.timeout, - logger = logger, - ) + if rp_config.enable_pipstar: + pypi_repo_utils.execute_checked( + rctx, + op = "whl_library.ExtractWheel({}, {})".format(rctx.attr.name, whl_path), + python = python_interpreter, + arguments = args + [ + "--whl-file", + whl_path, + "--enable-pipstar", + ], + srcs = rctx.attr._python_srcs, + environment = environment, + quiet = rctx.attr.quiet, + timeout = rctx.attr.timeout, + logger = logger, + ) - metadata = json.decode(rctx.read("metadata.json")) - rctx.delete("metadata.json") + metadata = json.decode(rctx.read("metadata.json")) + rctx.delete("metadata.json") + python_version = metadata["python_version"] - # NOTE @aignas 2024-06-22: this has to live on until we stop supporting - # passing `twine` as a `:pkg` library via the `WORKSPACE` builds. - # - # See ../../packaging.bzl line 190 - entry_points = {} - for item in metadata["entry_points"]: - name = item["name"] - module = item["module"] - attribute = item["attribute"] - - # There is an extreme edge-case with entry_points that end with `.py` - # See: https://github.com/bazelbuild/bazel/blob/09c621e4cf5b968f4c6cdf905ab142d5961f9ddc/src/test/java/com/google/devtools/build/lib/rules/python/PyBinaryConfiguredTargetTest.java#L174 - entry_point_without_py = name[:-3] + "_py" if name.endswith(".py") else name - entry_point_target_name = ( - _WHEEL_ENTRY_POINT_PREFIX + "_" + entry_point_without_py + # NOTE @aignas 2024-06-22: this has to live on until we stop supporting + # passing `twine` as a `:pkg` library via the `WORKSPACE` builds. + # + # See ../../packaging.bzl line 190 + entry_points = {} + for item in metadata["entry_points"]: + name = item["name"] + module = item["module"] + attribute = item["attribute"] + + # There is an extreme edge-case with entry_points that end with `.py` + # See: https://github.com/bazelbuild/bazel/blob/09c621e4cf5b968f4c6cdf905ab142d5961f9ddc/src/test/java/com/google/devtools/build/lib/rules/python/PyBinaryConfiguredTargetTest.java#L174 + entry_point_without_py = name[:-3] + "_py" if name.endswith(".py") else name + entry_point_target_name = ( + _WHEEL_ENTRY_POINT_PREFIX + "_" + entry_point_without_py + ) + entry_point_script_name = entry_point_target_name + ".py" + + rctx.file( + entry_point_script_name, + _generate_entry_point_contents(module, attribute), + ) + entry_points[entry_point_without_py] = entry_point_script_name + + metadata = whl_metadata( + install_dir = whl_path.dirname.get_child("site-packages"), + read_fn = rctx.read, + logger = logger, ) - entry_point_script_name = entry_point_target_name + ".py" - rctx.file( - entry_point_script_name, - _generate_entry_point_contents(module, attribute), + build_file_contents = generate_whl_library_build_bazel( + name = whl_path.basename, + dep_template = rctx.attr.dep_template or "@{}{{name}}//:{{target}}".format(rctx.attr.repo_prefix), + entry_points = entry_points, + metadata_name = metadata.name, + metadata_version = metadata.version, + default_python_version = python_version, + requires_dist = metadata.requires_dist, + target_platforms = rctx.attr.experimental_target_platforms or [host_platform(rctx)], + # TODO @aignas 2025-04-14: load through the hub: + annotation = None if not rctx.attr.annotation else struct(**json.decode(rctx.read(rctx.attr.annotation))), + data_exclude = rctx.attr.pip_data_exclude, + group_deps = rctx.attr.group_deps, + group_name = rctx.attr.group_name, ) - entry_points[entry_point_without_py] = entry_point_script_name - - build_file_contents = generate_whl_library_build_bazel( - name = whl_path.basename, - dep_template = rctx.attr.dep_template or "@{}{{name}}//:{{target}}".format(rctx.attr.repo_prefix), - entry_points = entry_points, - # TODO @aignas 2025-04-14: load through the hub: - dependencies = metadata["deps"], - dependencies_by_platform = metadata["deps_by_platform"], - annotation = None if not rctx.attr.annotation else struct(**json.decode(rctx.read(rctx.attr.annotation))), - data_exclude = rctx.attr.pip_data_exclude, - group_deps = rctx.attr.group_deps, - group_name = rctx.attr.group_name, - tags = [ - "pypi_name={}".format(metadata["name"]), - "pypi_version={}".format(metadata["version"]), - ], - ) + else: + target_platforms = rctx.attr.experimental_target_platforms or [] + if target_platforms: + parsed_whl = parse_whl_name(whl_path.basename) + + # NOTE @aignas 2023-12-04: if the wheel is a platform specific wheel, we + # only include deps for that target platform + if parsed_whl.platform_tag != "any": + target_platforms = [ + p.target_platform + for p in whl_target_platforms( + platform_tag = parsed_whl.platform_tag, + abi_tag = parsed_whl.abi_tag.strip("tm"), + ) + ] + + pypi_repo_utils.execute_checked( + rctx, + op = "whl_library.ExtractWheel({}, {})".format(rctx.attr.name, whl_path), + python = python_interpreter, + arguments = args + [ + "--whl-file", + whl_path, + ] + ["--platform={}".format(p) for p in target_platforms], + srcs = rctx.attr._python_srcs, + environment = environment, + quiet = rctx.attr.quiet, + timeout = rctx.attr.timeout, + logger = logger, + ) + + metadata = json.decode(rctx.read("metadata.json")) + rctx.delete("metadata.json") + + # NOTE @aignas 2024-06-22: this has to live on until we stop supporting + # passing `twine` as a `:pkg` library via the `WORKSPACE` builds. + # + # See ../../packaging.bzl line 190 + entry_points = {} + for item in metadata["entry_points"]: + name = item["name"] + module = item["module"] + attribute = item["attribute"] + + # There is an extreme edge-case with entry_points that end with `.py` + # See: https://github.com/bazelbuild/bazel/blob/09c621e4cf5b968f4c6cdf905ab142d5961f9ddc/src/test/java/com/google/devtools/build/lib/rules/python/PyBinaryConfiguredTargetTest.java#L174 + entry_point_without_py = name[:-3] + "_py" if name.endswith(".py") else name + entry_point_target_name = ( + _WHEEL_ENTRY_POINT_PREFIX + "_" + entry_point_without_py + ) + entry_point_script_name = entry_point_target_name + ".py" + + rctx.file( + entry_point_script_name, + _generate_entry_point_contents(module, attribute), + ) + entry_points[entry_point_without_py] = entry_point_script_name + + build_file_contents = generate_whl_library_build_bazel( + name = whl_path.basename, + dep_template = rctx.attr.dep_template or "@{}{{name}}//:{{target}}".format(rctx.attr.repo_prefix), + entry_points = entry_points, + # TODO @aignas 2025-04-14: load through the hub: + dependencies = metadata["deps"], + dependencies_by_platform = metadata["deps_by_platform"], + annotation = None if not rctx.attr.annotation else struct(**json.decode(rctx.read(rctx.attr.annotation))), + data_exclude = rctx.attr.pip_data_exclude, + group_deps = rctx.attr.group_deps, + group_name = rctx.attr.group_name, + tags = [ + "pypi_name={}".format(metadata["name"]), + "pypi_version={}".format(metadata["version"]), + ], + ) + rctx.file("BUILD.bazel", build_file_contents) return diff --git a/tests/pypi/whl_installer/wheel_installer_test.py b/tests/pypi/whl_installer/wheel_installer_test.py index b736877e81..e838047925 100644 --- a/tests/pypi/whl_installer/wheel_installer_test.py +++ b/tests/pypi/whl_installer/wheel_installer_test.py @@ -72,6 +72,7 @@ def test_wheel_exists(self) -> None: extras={}, enable_implicit_namespace_pkgs=False, platforms=[], + enable_pipstar = False, ) want_files = [ From 78647318f94b3a94e11b77f03e0314bd77e1e0fe Mon Sep 17 00:00:00 2001 From: Fabian Meumertzheim Date: Mon, 5 May 2025 18:27:27 +0200 Subject: [PATCH 207/922] fix: add target platform to extra exec platforms in analysis tests (#2861) This is required as of https://github.com/bazelbuild/bazel/commit/2780393d35ad0607cf5e344ae082b00a5569a964 as tests now require an execution platform that matches their target constraints by default. Fixes #2850 --- tests/base_rules/py_executable_base_tests.bzl | 2 ++ tests/base_rules/py_test/py_test_tests.bzl | 2 ++ 2 files changed, 4 insertions(+) diff --git a/tests/base_rules/py_executable_base_tests.bzl b/tests/base_rules/py_executable_base_tests.bzl index 37707831fc..55a8958b82 100644 --- a/tests/base_rules/py_executable_base_tests.bzl +++ b/tests/base_rules/py_executable_base_tests.bzl @@ -51,6 +51,7 @@ def _test_basic_windows(name, config): "//command_line_option:build_python_zip": "true", "//command_line_option:cpu": "windows_x86_64", "//command_line_option:crosstool_top": CROSSTOOL_TOP, + "//command_line_option:extra_execution_platforms": [WINDOWS_X86_64], "//command_line_option:extra_toolchains": [CC_TOOLCHAIN], "//command_line_option:platforms": [WINDOWS_X86_64], }, @@ -96,6 +97,7 @@ def _test_basic_zip(name, config): "//command_line_option:build_python_zip": "true", "//command_line_option:cpu": "linux_x86_64", "//command_line_option:crosstool_top": CROSSTOOL_TOP, + "//command_line_option:extra_execution_platforms": [LINUX_X86_64], "//command_line_option:extra_toolchains": [CC_TOOLCHAIN], "//command_line_option:platforms": [LINUX_X86_64], }, diff --git a/tests/base_rules/py_test/py_test_tests.bzl b/tests/base_rules/py_test/py_test_tests.bzl index d4d839b392..c51aa53a95 100644 --- a/tests/base_rules/py_test/py_test_tests.bzl +++ b/tests/base_rules/py_test/py_test_tests.bzl @@ -59,6 +59,7 @@ def _test_mac_requires_darwin_for_execution(name, config): config_settings = { "//command_line_option:cpu": "darwin_x86_64", "//command_line_option:crosstool_top": CROSSTOOL_TOP, + "//command_line_option:extra_execution_platforms": [MAC_X86_64], "//command_line_option:extra_toolchains": CC_TOOLCHAIN, "//command_line_option:platforms": [MAC_X86_64], }, @@ -92,6 +93,7 @@ def _test_non_mac_doesnt_require_darwin_for_execution(name, config): config_settings = { "//command_line_option:cpu": "k8", "//command_line_option:crosstool_top": CROSSTOOL_TOP, + "//command_line_option:extra_execution_platforms": [LINUX_X86_64], "//command_line_option:extra_toolchains": CC_TOOLCHAIN, "//command_line_option:platforms": [LINUX_X86_64], }, From 1492ae4b53c6ace19cfc67f542b574b2ccd7e40b Mon Sep 17 00:00:00 2001 From: Fabian Meumertzheim Date: Mon, 5 May 2025 18:29:59 +0200 Subject: [PATCH 208/922] fix: configure coverage helpers for test exec group (#2857) They are run on the test action's execution platform, which is resolved for the `test` exec group, not the default one. --- python/private/attributes.bzl | 4 ++-- python/private/py_executable.bzl | 3 +-- 2 files changed, 3 insertions(+), 4 deletions(-) diff --git a/python/private/attributes.bzl b/python/private/attributes.bzl index 8543caba7b..98aba4eb23 100644 --- a/python/private/attributes.bzl +++ b/python/private/attributes.bzl @@ -397,14 +397,14 @@ COVERAGE_ATTRS = { "_collect_cc_coverage": lambda: attrb.Label( default = "@bazel_tools//tools/test:collect_cc_coverage", executable = True, - cfg = "exec", + cfg = config.exec(exec_group = "test"), ), # Magic attribute to make coverage work. There's no # docs about this; see TestActionBuilder.java "_lcov_merger": lambda: attrb.Label( default = configuration_field(fragment = "coverage", name = "output_generator"), executable = True, - cfg = "exec", + cfg = config.exec(exec_group = "test"), ), } diff --git a/python/private/py_executable.bzl b/python/private/py_executable.bzl index a8c669afd9..24be8dd2ad 100644 --- a/python/private/py_executable.bzl +++ b/python/private/py_executable.bzl @@ -78,7 +78,6 @@ EXECUTABLE_ATTRS = dicts.add( AGNOSTIC_EXECUTABLE_ATTRS, PY_SRCS_ATTRS, IMPORTS_ATTRS, - COVERAGE_ATTRS, { "interpreter_args": lambda: attrb.StringList( doc = """ @@ -1903,7 +1902,7 @@ def create_executable_rule_builder(implementation, **kwargs): """ builder = ruleb.Rule( implementation = implementation, - attrs = EXECUTABLE_ATTRS, + attrs = EXECUTABLE_ATTRS | (COVERAGE_ATTRS if kwargs.get("test") else {}), exec_groups = dict(REQUIRED_EXEC_GROUP_BUILDERS), # Mutable copy fragments = ["py", "bazel_py"], provides = [PyExecutableInfo, PyInfo] + _MaybeBuiltinPyInfo, From 63555e1fdf708b6a44f166aa5a3dfa344325e0d0 Mon Sep 17 00:00:00 2001 From: Fabian Meumertzheim Date: Tue, 6 May 2025 10:34:20 +0200 Subject: [PATCH 209/922] fix: fix test analysis error on macOS arm64 (#2860) Fixes: ``` ERROR: /Users/fmeum/git/rules_python/tests/pypi/env_marker_setting/BUILD.bazel:3:30: Illegal ambiguous match on configurable attribute "platform_machine" in //tests/pypi/env_marker_setting:test_expr_python_full_version_lt_negative_subject: @@platforms//cpu:aarch64 @@platforms//cpu:arm64 Multiple matches are not allowed unless one is unambiguously more specialized or they resolve to the same value. See https://bazel.build/reference/be/functions#select. ``` Work towards #2850. Work towards #2826. --- python/private/pypi/pep508_env.bzl | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/python/private/pypi/pep508_env.bzl b/python/private/pypi/pep508_env.bzl index 3708c46f1d..d618535674 100644 --- a/python/private/pypi/pep508_env.bzl +++ b/python/private/pypi/pep508_env.bzl @@ -29,11 +29,13 @@ platform_machine_aliases = { # NOTE: There are many cpus, and unfortunately, the value isn't directly # accessible to Starlark. Using CcToolchain.cpu might work, though. +# Some targets are aliases and are omitted below as their value is implied +# by the target they resolve to. platform_machine_select_map = { "@platforms//cpu:aarch32": "aarch32", "@platforms//cpu:aarch64": "aarch64", - "@platforms//cpu:arm": "arm", - "@platforms//cpu:arm64": "arm64", + # @platforms//cpu:arm is an alias for @platforms//cpu:aarch32 + # @platforms//cpu:arm64 is an alias for @platforms//cpu:aarch64 "@platforms//cpu:arm64_32": "arm64_32", "@platforms//cpu:arm64e": "arm64e", "@platforms//cpu:armv6-m": "armv6-m", From 0b3d845ed1803ed27083f850c7c542bd2d3fc52c Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Tue, 6 May 2025 01:38:02 -0700 Subject: [PATCH 210/922] refactor: make env marker config available through target and flag (#2853) This factors creation of (most of) the env marker dict into a separate target and provides a label flag to allow customizing the target that provides it. This makes it easier for users to override how env marker values are computed. The `env_marker_setting` rule will still, if necessary, compute values from the toolchain, but existing keys (computed from the env marker config target) have precedence. The `EnvMarkerInfo` provider is the interface for implementing a custom env marker config target; it will be publically exposed in a subsequent PR. Along the way, unify how the env dict and defaults are set. Work towards https://github.com/bazel-contrib/rules_python/issues/2826 --- .../python/config_settings/index.md | 12 ++ docs/pypi-dependencies.md | 32 ++++- python/config_settings/BUILD.bazel | 7 ++ python/private/pypi/BUILD.bazel | 19 +++ python/private/pypi/env_marker_info.bzl | 26 ++++ python/private/pypi/env_marker_setting.bzl | 104 +++++----------- python/private/pypi/flags.bzl | 68 ++++++++++ python/private/pypi/pep508_env.bzl | 117 +++++++++++------- .../env_marker_setting_tests.bzl | 37 +++++- tests/support/support.bzl | 1 + 10 files changed, 300 insertions(+), 123 deletions(-) create mode 100644 python/private/pypi/env_marker_info.bzl diff --git a/docs/api/rules_python/python/config_settings/index.md b/docs/api/rules_python/python/config_settings/index.md index ed6444298e..f4618ff967 100644 --- a/docs/api/rules_python/python/config_settings/index.md +++ b/docs/api/rules_python/python/config_settings/index.md @@ -159,6 +159,18 @@ Values: ::: :::: +::::{bzl:flag} pip_env_marker_config +The target that provides the values for pip env marker evaluation. + +Default: `//python/config_settings:_pip_env_marker_default_config` + +This flag points to a target providing {obj}`EnvMarkerInfo`, which determines +the values used when environment markers are resolved at build time. + +:::{versionadded} VERSION_NEXT_FEATURE +::: +:::: + ::::{bzl:flag} pip_whl Set what distributions are used in the `pip` integration. diff --git a/docs/pypi-dependencies.md b/docs/pypi-dependencies.md index 4ec40bc889..b3ae7fe594 100644 --- a/docs/pypi-dependencies.md +++ b/docs/pypi-dependencies.md @@ -338,7 +338,6 @@ leg of the dependency manually. For instance by making perhaps `apache-airflow-providers-common-sql`. -(bazel-downloader)= ### Multi-platform support Multi-platform support of cross-building the wheels can be done in two ways - either @@ -391,6 +390,31 @@ compatible indexes. This is only supported on `bzlmd`. ``` + + (bazel-downloader)= ### Bazel downloader and multi-platform wheel hub repository. @@ -487,3 +511,9 @@ Bazel will call this file like `cred_helper.sh get` and use the returned JSON to into whatever HTTP(S) request it performs against `example.com`. [rfc7617]: https://datatracker.ietf.org/doc/html/rfc7617 + + diff --git a/python/config_settings/BUILD.bazel b/python/config_settings/BUILD.bazel index 872d7d1bda..24bbe665c7 100644 --- a/python/config_settings/BUILD.bazel +++ b/python/config_settings/BUILD.bazel @@ -220,3 +220,10 @@ string_flag( define_pypi_internal_flags( name = "define_pypi_internal_flags", ) + +label_flag( + name = "pip_env_marker_config", + build_setting_default = ":_pip_env_marker_default_config", + # NOTE: Only public because it is used in pip hub repos. + visibility = ["//visibility:public"], +) diff --git a/python/private/pypi/BUILD.bazel b/python/private/pypi/BUILD.bazel index 9216134857..d5d897ef8c 100644 --- a/python/private/pypi/BUILD.bazel +++ b/python/private/pypi/BUILD.bazel @@ -71,6 +71,23 @@ bzl_library( ], ) +bzl_library( + name = "env_marker_info_bzl", + srcs = ["env_marker_info.bzl"], +) + +bzl_library( + name = "env_marker_setting_bzl", + srcs = ["env_marker_setting.bzl"], + deps = [ + ":env_marker_info_bzl", + ":pep508_env_bzl", + ":pep508_evaluate_bzl", + "//python/private:toolchain_types_bzl", + "@bazel_skylib//rules:common_settings", + ], +) + bzl_library( name = "evaluate_markers_bzl", srcs = ["evaluate_markers.bzl"], @@ -111,6 +128,8 @@ bzl_library( name = "flags_bzl", srcs = ["flags.bzl"], deps = [ + ":env_marker_info.bzl", + ":pep508_env_bzl", "//python/private:enum_bzl", "@bazel_skylib//rules:common_settings", ], diff --git a/python/private/pypi/env_marker_info.bzl b/python/private/pypi/env_marker_info.bzl new file mode 100644 index 0000000000..b483436d98 --- /dev/null +++ b/python/private/pypi/env_marker_info.bzl @@ -0,0 +1,26 @@ +"""Provider for implementing environment marker values.""" + +EnvMarkerInfo = provider( + doc = """ +The values to use during environment marker evaluation. + +:::{seealso} +The {obj}`--//python/config_settings:pip_env_marker_config` flag. +::: + +:::{versionadded} VERSION_NEXT_FEATURE +""", + fields = { + "env": """ +:type: dict[str, str] + +The values to use for environment markers when evaluating an expression. + +The keys and values should be compatible with the [PyPA dependency specifiers +specification](https://packaging.python.org/en/latest/specifications/dependency-specifiers/) + +Missing values will be set to the specification's defaults or computed using +available toolchain information. +""", + }, +) diff --git a/python/private/pypi/env_marker_setting.bzl b/python/private/pypi/env_marker_setting.bzl index bbc59ab110..2bfdf42ef0 100644 --- a/python/private/pypi/env_marker_setting.bzl +++ b/python/private/pypi/env_marker_setting.bzl @@ -2,14 +2,8 @@ load("@bazel_skylib//rules:common_settings.bzl", "BuildSettingInfo") load("//python/private:toolchain_types.bzl", "TARGET_TOOLCHAIN_TYPE") -load( - ":pep508_env.bzl", - "env_aliases", - "os_name_select_map", - "platform_machine_select_map", - "platform_system_select_map", - "sys_platform_select_map", -) +load(":env_marker_info.bzl", "EnvMarkerInfo") +load(":pep508_env.bzl", "create_env", "set_missing_env_defaults") load(":pep508_evaluate.bzl", "evaluate") # Use capitals to hint its not an actual boolean type. @@ -39,72 +33,37 @@ def env_marker_setting(*, name, expression, **kwargs): _env_marker_setting( name = name, expression = expression, - os_name = select(os_name_select_map), - sys_platform = select(sys_platform_select_map), - platform_machine = select(platform_machine_select_map), - platform_system = select(platform_system_select_map), - platform_release = select({ - "@platforms//os:osx": "USE_OSX_VERSION_FLAG", - "//conditions:default": "", - }), **kwargs ) def _env_marker_setting_impl(ctx): - env = {} + env = create_env() + env.update( + ctx.attr._env_marker_config_flag[EnvMarkerInfo].env, + ) runtime = ctx.toolchains[TARGET_TOOLCHAIN_TYPE].py3_runtime - if runtime.interpreter_version_info: - version_info = runtime.interpreter_version_info - env["python_version"] = "{major}.{minor}".format( - major = version_info.major, - minor = version_info.minor, - ) - full_version = _format_full_version(version_info) - env["python_full_version"] = full_version - env["implementation_version"] = full_version - else: - env["python_version"] = _get_flag(ctx.attr._python_version_major_minor_flag) - full_version = _get_flag(ctx.attr._python_full_version_flag) - env["python_full_version"] = full_version - env["implementation_version"] = full_version - - # We assume cpython if the toolchain doesn't specify because it's most - # likely to be true. - env["implementation_name"] = runtime.implementation_name or "cpython" - env["os_name"] = ctx.attr.os_name - env["sys_platform"] = ctx.attr.sys_platform - env["platform_machine"] = ctx.attr.platform_machine - - # The `platform_python_implementation` marker value is supposed to come - # from `platform.python_implementation()`, however, PEP 421 introduced - # `sys.implementation.name` and the `implementation_name` env marker to - # replace it. Per the platform.python_implementation docs, there's now - # essentially just two possible "registered" values: CPython or PyPy. - # Rather than add a field to the toolchain, we just special case the value - # from `sys.implementation.name` to handle the two documented values. - platform_python_impl = runtime.implementation_name - if platform_python_impl == "cpython": - platform_python_impl = "CPython" - elif platform_python_impl == "pypy": - platform_python_impl = "PyPy" - env["platform_python_implementation"] = platform_python_impl - - # NOTE: Platform release for Android will be Android version: - # https://peps.python.org/pep-0738/#platform - # Similar for iOS: - # https://peps.python.org/pep-0730/#platform - platform_release = ctx.attr.platform_release - if platform_release == "USE_OSX_VERSION_FLAG": - platform_release = _get_flag(ctx.attr._pip_whl_osx_version_flag) - env["platform_release"] = platform_release - env["platform_system"] = ctx.attr.platform_system - - # For lack of a better option, just use an empty string for now. - env["platform_version"] = "" - - env.update(env_aliases()) + if "python_version" not in env: + if runtime.interpreter_version_info: + version_info = runtime.interpreter_version_info + env["python_version"] = "{major}.{minor}".format( + major = version_info.major, + minor = version_info.minor, + ) + full_version = _format_full_version(version_info) + env["python_full_version"] = full_version + env["implementation_version"] = full_version + else: + env["python_version"] = _get_flag(ctx.attr._python_version_major_minor_flag) + full_version = _get_flag(ctx.attr._python_full_version_flag) + env["python_full_version"] = full_version + env["implementation_version"] = full_version + + if "implementation_name" not in env and runtime.implementation_name: + env["implementation_name"] = runtime.implementation_name + + set_missing_env_defaults(env) if evaluate(ctx.attr.expression, env = env): value = _ENV_MARKER_TRUE else: @@ -125,14 +84,9 @@ for the specification of behavior. mandatory = True, doc = "Environment marker expression to evaluate.", ), - "os_name": attr.string(), - "platform_machine": attr.string(), - "platform_release": attr.string(), - "platform_system": attr.string(), - "sys_platform": attr.string(), - "_pip_whl_osx_version_flag": attr.label( - default = "//python/config_settings:pip_whl_osx_version", - providers = [[BuildSettingInfo], [config_common.FeatureFlagInfo]], + "_env_marker_config_flag": attr.label( + default = "//python/config_settings:pip_env_marker_config", + providers = [EnvMarkerInfo], ), "_python_full_version_flag": attr.label( default = "//python/config_settings:python_version", diff --git a/python/private/pypi/flags.bzl b/python/private/pypi/flags.bzl index a25579a2b8..037383910e 100644 --- a/python/private/pypi/flags.bzl +++ b/python/private/pypi/flags.bzl @@ -20,6 +20,15 @@ unnecessary files when all that are needed are flag definitions. load("@bazel_skylib//rules:common_settings.bzl", "BuildSettingInfo", "string_flag") load("//python/private:enum.bzl", "enum") +load(":env_marker_info.bzl", "EnvMarkerInfo") +load( + ":pep508_env.bzl", + "create_env", + "os_name_select_map", + "platform_machine_select_map", + "platform_system_select_map", + "sys_platform_select_map", +) # Determines if we should use whls for third party # @@ -82,6 +91,10 @@ def define_pypi_internal_flags(name): visibility = ["//visibility:public"], ) + _default_env_marker_config( + name = "_pip_env_marker_default_config", + ) + def _allow_wheels_flag_impl(ctx): input = ctx.attr._setting[BuildSettingInfo].value value = "yes" if input in ["auto", "only"] else "no" @@ -97,3 +110,58 @@ This rule allows us to greatly reduce the number of config setting targets at no if we are duplicating some of the functionality of the `native.config_setting`. """, ) + +def _default_env_marker_config(**kwargs): + _env_marker_config( + os_name = select(os_name_select_map), + sys_platform = select(sys_platform_select_map), + platform_machine = select(platform_machine_select_map), + platform_system = select(platform_system_select_map), + platform_release = select({ + "@platforms//os:osx": "USE_OSX_VERSION_FLAG", + "//conditions:default": "", + }), + **kwargs + ) + +def _env_marker_config_impl(ctx): + env = create_env() + env["os_name"] = ctx.attr.os_name + env["sys_platform"] = ctx.attr.sys_platform + env["platform_machine"] = ctx.attr.platform_machine + + # NOTE: Platform release for Android will be Android version: + # https://peps.python.org/pep-0738/#platform + # Similar for iOS: + # https://peps.python.org/pep-0730/#platform + platform_release = ctx.attr.platform_release + if platform_release == "USE_OSX_VERSION_FLAG": + platform_release = _get_flag(ctx.attr._pip_whl_osx_version_flag) + env["platform_release"] = platform_release + env["platform_system"] = ctx.attr.platform_system + + # NOTE: We intentionally do not call set_missing_env_defaults() here because + # `env_marker_setting()` computes missing values using the toolchain. + return [EnvMarkerInfo(env = env)] + +_env_marker_config = rule( + implementation = _env_marker_config_impl, + attrs = { + "os_name": attr.string(), + "platform_machine": attr.string(), + "platform_release": attr.string(), + "platform_system": attr.string(), + "sys_platform": attr.string(), + "_pip_whl_osx_version_flag": attr.label( + default = "//python/config_settings:pip_whl_osx_version", + providers = [[BuildSettingInfo], [config_common.FeatureFlagInfo]], + ), + }, +) + +def _get_flag(t): + if config_common.FeatureFlagInfo in t: + return t[config_common.FeatureFlagInfo].value + if BuildSettingInfo in t: + return t[BuildSettingInfo].value + fail("Should not occur: {} does not have necessary providers") diff --git a/python/private/pypi/pep508_env.bzl b/python/private/pypi/pep508_env.bzl index d618535674..a6efb3c50c 100644 --- a/python/private/pypi/pep508_env.bzl +++ b/python/private/pypi/pep508_env.bzl @@ -66,23 +66,23 @@ platform_machine_select_map = { # Platform system returns results from the `uname` call. _platform_system_values = { + # See https://peps.python.org/pep-0738/#platform + "android": "Android", + "freebsd": "FreeBSD", + # See https://peps.python.org/pep-0730/#platform + # NOTE: Per Pep 730, "iPadOS" is also an acceptable value + "ios": "iOS", "linux": "Linux", + "netbsd": "NetBSD", + "openbsd": "OpenBSD", "osx": "Darwin", "windows": "Windows", } platform_system_select_map = { - # See https://peps.python.org/pep-0738/#platform - "@platforms//os:android": "Android", - "@platforms//os:freebsd": "FreeBSD", - # See https://peps.python.org/pep-0730/#platform - # NOTE: Per Pep 730, "iPadOS" is also an acceptable value - "@platforms//os:ios": "iOS", - "@platforms//os:linux": "Linux", - "@platforms//os:netbsd": "NetBSD", - "@platforms//os:openbsd": "OpenBSD", - "@platforms//os:osx": "Darwin", - "@platforms//os:windows": "Windows", + "@platforms//os:{}".format(bazel_os): py_system + for bazel_os, py_system in _platform_system_values.items() +} | { # The value is empty string if it cannot be determined: # https://docs.python.org/3/library/platform.html#platform.machine "//conditions:default": "", @@ -114,33 +114,36 @@ platform_system_select_map = { # # We are using only the subset that we actually support. _sys_platform_values = { + # These values are decided by the sys.platform docs. + "android": "android", + "emscripten": "emscripten", + # NOTE: The below values are approximations. The sys.platform() docs + # don't have documented values for these OSes. Per docs, the + # sys.platform() value reflects the OS at the time Python was *built* + # instead of the runtime (target) OS value. + "freebsd": "freebsd", + "ios": "ios", "linux": "linux", + "openbsd": "openbsd", "osx": "darwin", + "wasi": "wasi", "windows": "win32", } -# Taken from -# https://docs.python.org/3/library/sys.html#sys.platform sys_platform_select_map = { - # These values are decided by the sys.platform docs. - "@platforms//os:android": "android", - "@platforms//os:emscripten": "emscripten", - # NOTE: The below values are approximations. The sys.platform() docs - # don't have documented values for these OSes. Per docs, the - # sys.platform() value reflects the OS at the time Python was *built* - # instead of the runtime (target) OS value. - "@platforms//os:freebsd": "freebsd", - "@platforms//os:ios": "ios", - "@platforms//os:linux": "linux", - "@platforms//os:openbsd": "openbsd", - "@platforms//os:osx": "darwin", - "@platforms//os:wasi": "wasi", - "@platforms//os:windows": "win32", + "@platforms//os:{}".format(bazel_os): py_platform + for bazel_os, py_platform in _sys_platform_values.items() +} | { # For lack of a better option, use empty string. No standard doc/spec # about sys_platform value. "//conditions:default": "", } +# The "java" value is documented, but with Jython defunct, +# shouldn't occur in practice. +# The os.name value is technically a property of the runtime, not the +# targetted runtime OS, but the distinction shouldn't matter if +# things are properly configured. _os_name_values = { "linux": "posix", "osx": "posix", @@ -148,18 +151,18 @@ _os_name_values = { } os_name_select_map = { - # The "java" value is documented, but with Jython defunct, - # shouldn't occur in practice. - # The os.name value is technically a property of the runtime, not the - # targetted runtime OS, but the distinction shouldn't matter if - # things are properly configured. - "@platforms//os:windows": "nt", + "@platforms//os:{}".format(bazel_os): py_os + for bazel_os, py_os in _os_name_values.items() +} | { "//conditions:default": "posix", } def env(target_platform, *, extra = None): """Return an env target platform + NOTE: This is for use during the loading phase. For the analysis phase, + `env_marker_setting()` constructs the env dict. + Args: target_platform: {type}`str` the target platform identifier, e.g. `cp33_linux_aarch64` @@ -168,16 +171,9 @@ def env(target_platform, *, extra = None): Returns: A dict that can be used as `env` in the marker evaluation. """ - - # TODO @aignas 2025-02-13: consider moving this into config settings. - - env = {"extra": extra} if extra != None else {} - env = env | { - "implementation_name": "cpython", - "platform_python_implementation": "CPython", - "platform_release": "", - "platform_version": "", - } + env = create_env() + if extra != None: + env["extra"] = extra if type(target_platform) == type(""): target_platform = platform_from_str(target_platform, python_version = "") @@ -198,13 +194,42 @@ def env(target_platform, *, extra = None): "platform_system": _platform_system_values.get(os, ""), "sys_platform": _sys_platform_values.get(os, ""), } + set_missing_env_defaults(env) - # This is split by topic - return env | env_aliases() + return env -def env_aliases(): +def create_env(): return { + # This is split by topic "_aliases": { "platform_machine": platform_machine_aliases, }, } + +def set_missing_env_defaults(env): + """Sets defaults based on existing values. + + Args: + env: dict; NOTE: modified in-place + """ + if "implementation_name" not in env: + # Use cpython as the default because it's likely the correct value. + env["implementation_name"] = "cpython" + if "platform_python_implementation" not in env: + # The `platform_python_implementation` marker value is supposed to come + # from `platform.python_implementation()`, however, PEP 421 introduced + # `sys.implementation.name` and the `implementation_name` env marker to + # replace it. Per the platform.python_implementation docs, there's now + # essentially just two possible "registered" values: CPython or PyPy. + # Rather than add a field to the toolchain, we just special case the value + # from `sys.implementation.name` to handle the two documented values. + platform_python_impl = env["implementation_name"] + if platform_python_impl == "cpython": + platform_python_impl = "CPython" + elif platform_python_impl == "pypy": + platform_python_impl = "PyPy" + env["platform_python_implementation"] = platform_python_impl + if "platform_release" not in env: + env["platform_release"] = "" + if "platform_version" not in env: + env["platform_version"] = "0" diff --git a/tests/pypi/env_marker_setting/env_marker_setting_tests.bzl b/tests/pypi/env_marker_setting/env_marker_setting_tests.bzl index 549c15c20b..e16f2c8ef6 100644 --- a/tests/pypi/env_marker_setting/env_marker_setting_tests.bzl +++ b/tests/pypi/env_marker_setting/env_marker_setting_tests.bzl @@ -3,11 +3,46 @@ load("@rules_testing//lib:analysis_test.bzl", "analysis_test") load("@rules_testing//lib:test_suite.bzl", "test_suite") load("@rules_testing//lib:util.bzl", "TestingAspectInfo") +load("//python/private/pypi:env_marker_info.bzl", "EnvMarkerInfo") # buildifier: disable=bzl-visibility load("//python/private/pypi:env_marker_setting.bzl", "env_marker_setting") # buildifier: disable=bzl-visibility -load("//tests/support:support.bzl", "PYTHON_VERSION") +load("//tests/support:support.bzl", "PIP_ENV_MARKER_CONFIG", "PYTHON_VERSION") + +def _custom_env_markers_impl(ctx): + _ = ctx # @unused + return [EnvMarkerInfo(env = { + "os_name": "testos", + })] + +_custom_env_markers = rule( + implementation = _custom_env_markers_impl, +) _tests = [] +def _test_custom_env_markers(name): + def _impl(env, target): + env.expect.where( + expression = target[TestingAspectInfo].attrs.expression, + ).that_str( + target[config_common.FeatureFlagInfo].value, + ).equals("TRUE") + + env_marker_setting( + name = name + "_subject", + expression = "os_name == 'testos'", + ) + _custom_env_markers(name = name + "_env") + analysis_test( + name = name, + impl = _impl, + target = name + "_subject", + config_settings = { + PIP_ENV_MARKER_CONFIG: str(Label(name + "_env")), + }, + ) + +_tests.append(_test_custom_env_markers) + def _test_expr(name): def impl(env, target): env.expect.where( diff --git a/tests/support/support.bzl b/tests/support/support.bzl index 6330155d8c..7bab263c66 100644 --- a/tests/support/support.bzl +++ b/tests/support/support.bzl @@ -37,6 +37,7 @@ CROSSTOOL_TOP = Label("//tests/support/cc_toolchains:cc_toolchain_suite") ADD_SRCS_TO_RUNFILES = str(Label("//python/config_settings:add_srcs_to_runfiles")) BOOTSTRAP_IMPL = str(Label("//python/config_settings:bootstrap_impl")) EXEC_TOOLS_TOOLCHAIN = str(Label("//python/config_settings:exec_tools_toolchain")) +PIP_ENV_MARKER_CONFIG = str(Label("//python/config_settings:pip_env_marker_config")) PRECOMPILE = str(Label("//python/config_settings:precompile")) PRECOMPILE_SOURCE_RETENTION = str(Label("//python/config_settings:precompile_source_retention")) PYC_COLLECTION = str(Label("//python/config_settings:pyc_collection")) From 9f3512fe0cc6d7229170e45724e22e64be0b8300 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Tue, 6 May 2025 11:33:12 -0700 Subject: [PATCH 211/922] feat: default to bootstrap script for non-windows (#2858) This makes non-Windows use the script bootstrap by default. It's been a couple releases without any reported issues, so it seems ready to become the default. Work towards https://github.com/bazel-contrib/rules_python/issues/2156 --- CHANGELOG.md | 8 ++++ MODULE.bazel | 7 +++- .../python/config_settings/index.md | 9 ++++ internal_dev_setup.bzl | 3 ++ python/config_settings/BUILD.bazel | 2 +- python/private/config_settings.bzl | 17 ++++++-- python/private/internal_dev_deps.bzl | 2 + python/private/runtime_env_repo.bzl | 41 +++++++++++++++++++ .../runtime_env_toolchain_interpreter.sh | 3 ++ tests/runtime_env_toolchain/BUILD.bazel | 4 ++ 10 files changed, 90 insertions(+), 6 deletions(-) create mode 100644 python/private/runtime_env_repo.bzl diff --git a/CHANGELOG.md b/CHANGELOG.md index 7d73613a07..8fdb7edd6a 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -55,6 +55,14 @@ END_UNRELEASED_TEMPLATE {#v0-0-0-changed} ### Changed +* If using the (deprecated) autodetecting/runtime_env toolchain, then the Python + version specified at build-time *must* match the Python version used at + runtime (the {obj}`--@rules_python//python/config_settings:python_version` + flag and the {attr}`python_version` attribute control the build-time version + for a target). If they don't match, dependencies won't be importable. (Such a + misconfiguration was unlikely to work to begin with; this is called out as an + FYI). +* (rules) {obj}`--bootstrap_impl=script` is the default for non-Windows. * (rules) On Windows, {obj}`--bootstrap_impl=system_python` is forced. This allows setting `--bootstrap_impl=script` in bazelrc for mixed-platform environments. diff --git a/MODULE.bazel b/MODULE.bazel index c649896344..d0f7cc4afa 100644 --- a/MODULE.bazel +++ b/MODULE.bazel @@ -98,7 +98,12 @@ internal_dev_deps = use_extension( "internal_dev_deps", dev_dependency = True, ) -use_repo(internal_dev_deps, "buildkite_config", "wheel_for_testing") +use_repo( + internal_dev_deps, + "buildkite_config", + "rules_python_runtime_env_tc_info", + "wheel_for_testing", +) # Add gazelle plugin so that we can run the gazelle example as an e2e integration # test and include the distribution files. diff --git a/docs/api/rules_python/python/config_settings/index.md b/docs/api/rules_python/python/config_settings/index.md index f4618ff967..ae84d40b13 100644 --- a/docs/api/rules_python/python/config_settings/index.md +++ b/docs/api/rules_python/python/config_settings/index.md @@ -245,6 +245,10 @@ Values: ::::{bzl:flag} bootstrap_impl Determine how programs implement their startup process. +The default for this depends on the platform: +* Windows: `system_python` (**always** used) +* Other: `script` + Values: * `system_python`: Use a bootstrap that requires a system Python available in order to start programs. This requires @@ -269,6 +273,11 @@ instead. :::{versionadded} 0.33.0 ::: +:::{versionchanged} VERSION_NEXT_FEATURE +* The default for non-Windows changed from `system_python` to `script`. +* On Windows, the value is forced to `system_python`. +::: + :::: ::::{bzl:flag} current_config diff --git a/internal_dev_setup.bzl b/internal_dev_setup.bzl index fc38e3f9c5..f33908049f 100644 --- a/internal_dev_setup.bzl +++ b/internal_dev_setup.bzl @@ -24,6 +24,7 @@ load("@rules_shell//shell:repositories.bzl", "rules_shell_dependencies", "rules_ load("//:version.bzl", "SUPPORTED_BAZEL_VERSIONS") load("//python:versions.bzl", "MINOR_MAPPING", "TOOL_VERSIONS") load("//python/private:pythons_hub.bzl", "hub_repo") # buildifier: disable=bzl-visibility +load("//python/private:runtime_env_repo.bzl", "runtime_env_repo") # buildifier: disable=bzl-visibility load("//python/private/pypi:deps.bzl", "pypi_deps") # buildifier: disable=bzl-visibility def rules_python_internal_setup(): @@ -40,6 +41,8 @@ def rules_python_internal_setup(): python_versions = sorted(TOOL_VERSIONS.keys()), ) + runtime_env_repo(name = "rules_python_runtime_env_tc_info") + pypi_deps() bazel_skylib_workspace() diff --git a/python/config_settings/BUILD.bazel b/python/config_settings/BUILD.bazel index 24bbe665c7..1772a3403e 100644 --- a/python/config_settings/BUILD.bazel +++ b/python/config_settings/BUILD.bazel @@ -90,7 +90,7 @@ string_flag( rp_string_flag( name = "bootstrap_impl", - build_setting_default = BootstrapImplFlag.SYSTEM_PYTHON, + build_setting_default = BootstrapImplFlag.SCRIPT, override = select({ # Windows doesn't yet support bootstrap=script, so force disable it ":_is_windows": BootstrapImplFlag.SYSTEM_PYTHON, diff --git a/python/private/config_settings.bzl b/python/private/config_settings.bzl index 2cf7968061..1685195b78 100644 --- a/python/private/config_settings.bzl +++ b/python/private/config_settings.bzl @@ -225,10 +225,19 @@ def is_python_version_at_least(name, **kwargs): ) def _python_version_at_least_impl(ctx): - at_least = tuple(ctx.attr.at_least.split(".")) - current = tuple( - ctx.attr._major_minor[config_common.FeatureFlagInfo].value.split("."), - ) + flag_value = ctx.attr._major_minor[config_common.FeatureFlagInfo].value + + # CI is, somehow, getting an empty string for the current flag value. + # How isn't clear. + if not flag_value: + return [config_common.FeatureFlagInfo(value = "no")] + + current = tuple([ + int(x) + for x in flag_value.split(".") + ]) + at_least = tuple([int(x) for x in ctx.attr.at_least.split(".")]) + value = "yes" if current >= at_least else "no" return [config_common.FeatureFlagInfo(value = value)] diff --git a/python/private/internal_dev_deps.bzl b/python/private/internal_dev_deps.bzl index 2a3b84e7df..4f2cca0b42 100644 --- a/python/private/internal_dev_deps.bzl +++ b/python/private/internal_dev_deps.bzl @@ -15,6 +15,7 @@ load("@bazel_ci_rules//:rbe_repo.bzl", "rbe_preconfig") load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_file") +load(":runtime_env_repo.bzl", "runtime_env_repo") def _internal_dev_deps_impl(mctx): _ = mctx # @unused @@ -37,6 +38,7 @@ def _internal_dev_deps_impl(mctx): name = "buildkite_config", toolchain = "ubuntu1804-bazel-java11", ) + runtime_env_repo(name = "rules_python_runtime_env_tc_info") internal_dev_deps = module_extension( implementation = _internal_dev_deps_impl, diff --git a/python/private/runtime_env_repo.bzl b/python/private/runtime_env_repo.bzl new file mode 100644 index 0000000000..cade1968bb --- /dev/null +++ b/python/private/runtime_env_repo.bzl @@ -0,0 +1,41 @@ +"""Internal setup to help the runtime_env toolchain.""" + +load("//python/private:repo_utils.bzl", "repo_utils") + +def _runtime_env_repo_impl(rctx): + pyenv = repo_utils.which_unchecked(rctx, "pyenv").binary + if pyenv != None: + pyenv_version_file = repo_utils.execute_checked( + rctx, + op = "GetPyenvVersionFile", + arguments = [pyenv, "version-file"], + ).stdout.strip() + + # When pyenv is used, the version file is what decided the + # version used. Watch it so we compute the correct value if the + # user changes it. + rctx.watch(pyenv_version_file) + + version = repo_utils.execute_checked( + rctx, + op = "GetPythonVersion", + arguments = [ + "python3", + "-I", + "-c", + """import sys; print(f"{sys.version_info.major}.{sys.version_info.minor}")""", + ], + environment = { + # Prevent the user's current shell from influencing the result. + # This envvar won't be present when a test is run. + # NOTE: This should be None, but Bazel 7 doesn't support None + # values. Thankfully, pyenv treats empty string the same as missing. + "PYENV_VERSION": "", + }, + ).stdout.strip() + rctx.file("info.bzl", "PYTHON_VERSION = '{}'\n".format(version)) + rctx.file("BUILD.bazel", "") + +runtime_env_repo = repository_rule( + implementation = _runtime_env_repo_impl, +) diff --git a/python/private/runtime_env_toolchain_interpreter.sh b/python/private/runtime_env_toolchain_interpreter.sh index 6159d4f38c..7b3ec598b2 100755 --- a/python/private/runtime_env_toolchain_interpreter.sh +++ b/python/private/runtime_env_toolchain_interpreter.sh @@ -68,6 +68,9 @@ if [ -e "$self_dir/pyvenv.cfg" ] || [ -e "$self_dir/../pyvenv.cfg" ]; then ;; esac + if [ ! -e "$PYTHON_BIN" ]; then + die "ERROR: Python interpreter does not exist: $PYTHON_BIN" + fi # PYTHONEXECUTABLE is also used because `exec -a` doesn't fully trick the # pyenv wrappers. # NOTE: The PYTHONEXECUTABLE envvar only works for non-Mac starting in Python 3.11 diff --git a/tests/runtime_env_toolchain/BUILD.bazel b/tests/runtime_env_toolchain/BUILD.bazel index 59ca93ba49..ad2bd4eeb5 100644 --- a/tests/runtime_env_toolchain/BUILD.bazel +++ b/tests/runtime_env_toolchain/BUILD.bazel @@ -12,6 +12,7 @@ # See the License for the specific language governing permissions and # limitations under the License. +load("@rules_python_runtime_env_tc_info//:info.bzl", "PYTHON_VERSION") load("//tests/support:sh_py_run_test.bzl", "py_reconfig_test") load("//tests/support:support.bzl", "CC_TOOLCHAIN") load(":runtime_env_toolchain_tests.bzl", "runtime_env_toolchain_test_suite") @@ -30,6 +31,9 @@ py_reconfig_test( CC_TOOLCHAIN, ], main = "toolchain_runs_test.py", + # With bootstrap=script, the build version must match the runtime version + # because the venv has the version in the lib/site-packages dir name. + python_version = PYTHON_VERSION, # Our RBE has Python 3.6, which is too old for the language features # we use now. Using the runtime-env toolchain on RBE is pretty # questionable anyways. From 9dfa3abba293488a9a1899832a340f7b44525cad Mon Sep 17 00:00:00 2001 From: Ignas Anikevicius <240938+aignas@users.noreply.github.com> Date: Thu, 8 May 2025 16:12:17 +0900 Subject: [PATCH 212/922] fix(pypi): fix a typo in parse_simpleapi_html (#2866) It seems that the integration tests that I thought were covering this had the same time. Added an assertion to the unit tests as well Fixes #2863. --- CHANGELOG.md | 11 +++++++++++ python/private/pypi/parse_simpleapi_html.bzl | 6 +++--- .../parse_simpleapi_html_tests.bzl | 1 + 3 files changed, 15 insertions(+), 3 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 8fdb7edd6a..5f67c8a5ec 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -102,6 +102,17 @@ END_UNRELEASED_TEMPLATE ### Removed * Nothing removed. +{#1-4-1} +## [1.4.1] - 2025-05-08 + +[1.4.1]: https://github.com/bazel-contrib/rules_python/releases/tag/1.4.1 + +{#1-4-1-fixed} +### Fixed +* (pypi) Fix a typo not allowing users to benefit from using the downloader when the hashes in the + requirements file are not present. Fixes + [#2863](https://github.com/bazel-contrib/rules_python/issues/2863). + {#1-4-0} ## [1.4.0] - 2025-04-19 diff --git a/python/private/pypi/parse_simpleapi_html.bzl b/python/private/pypi/parse_simpleapi_html.bzl index 8c6f739fe3..a41f0750c4 100644 --- a/python/private/pypi/parse_simpleapi_html.bzl +++ b/python/private/pypi/parse_simpleapi_html.bzl @@ -52,7 +52,7 @@ def parse_simpleapi_html(*, url, content): # Each line follows the following pattern # filename
- sha256_by_version = {} + sha256s_by_version = {} for line in lines[1:]: dist_url, _, tail = line.partition("#sha256=") dist_url = _absolute_url(url, dist_url) @@ -65,7 +65,7 @@ def parse_simpleapi_html(*, url, content): head, _, _ = tail.rpartition("") maybe_metadata, _, filename = head.rpartition(">") version = _version(filename) - sha256_by_version.setdefault(version, []).append(sha256) + sha256s_by_version.setdefault(version, []).append(sha256) metadata_sha256 = "" metadata_url = "" @@ -102,7 +102,7 @@ def parse_simpleapi_html(*, url, content): return struct( sdists = sdists, whls = whls, - sha256_by_version = sha256_by_version, + sha256s_by_version = sha256s_by_version, ) _SDIST_EXTS = [ diff --git a/tests/pypi/parse_simpleapi_html/parse_simpleapi_html_tests.bzl b/tests/pypi/parse_simpleapi_html/parse_simpleapi_html_tests.bzl index 191079d214..b96d02f990 100644 --- a/tests/pypi/parse_simpleapi_html/parse_simpleapi_html_tests.bzl +++ b/tests/pypi/parse_simpleapi_html/parse_simpleapi_html_tests.bzl @@ -86,6 +86,7 @@ def _test_sdist(env): got = parse_simpleapi_html(url = input.url, content = html) env.expect.that_collection(got.sdists).has_size(1) env.expect.that_collection(got.whls).has_size(0) + env.expect.that_collection(got.sha256s_by_version).has_size(1) if not got: fail("expected at least one element, but did not get anything from:\n{}".format(html)) From a2ff7daba62da590d7395701a145acd900f29908 Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Fri, 9 May 2025 10:20:38 +0900 Subject: [PATCH 213/922] build(deps): bump more-itertools from 10.5.0 to 10.7.0 in /tools/publish (#2841) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Bumps [more-itertools](https://github.com/more-itertools/more-itertools) from 10.5.0 to 10.7.0.
Release notes

Sourced from more-itertools's releases.

Version 10.7.0

See the change log here for details.

Version 10.6.0

  • New functions:

    • is_prime and nth_prime were added (thanks to JamesParrott and rhettinger)
    • loops was added (thanks to rhettinger)
  • Changes to existing functions:

    • factor was optimized to handle larger inputs and use less memory (thanks to rhettinger)
    • spy was optimized to enable nested calls (thanks to rhettinger)
    • polynomial_from_roots was made non-recursive and able to handle larger numbers of roots (thanks to pochmann3 and rhettinger)
    • is_sorted now only relies on less than comparisons (thanks to rhettinger)
    • The docstring for outer_product was improved (thanks to rhettinger)
    • The type annotations for sample were improved (thanks to rhettinger)
  • Other changes:

    • Python 3.13 is officially supported. Python 3.8 is no longer officially supported. (thanks to hugovk, JamesParrott, and stankudrow)
    • mypy checks were fixed (thanks to JamesParrott)
Commits
  • 28ab736 Merge pull request #977 from more-itertools/version-10.7.0
  • 4c1a0c7 Bump version: 10.6.0 → 10.7.0
  • f2d5c9f Late-breaking changes for 10.7.0
  • 5d5a9e6 Merge remote-tracking branch 'origin/master' into version-10.7.0
  • 8988de6 Merge pull request #975 from rhettinger/groupby_transform_overloads
  • c925c2e Fix inner Iterable types as well
  • cc38c74 Fix #974: Inconsistent @​overload signatures
  • 3742de9 Merge pull request #972 from ricbit/master
  • c904030 Fix some typos
  • 6d0fe02 Merge pull request #971 from rhettinger/small_doc_edits
  • Additional commits viewable in compare view

[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=more-itertools&package-manager=pip&previous-version=10.5.0&new-version=10.7.0)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) ---
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot show ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> --- tools/publish/requirements_darwin.txt | 6 +++--- tools/publish/requirements_linux.txt | 6 +++--- tools/publish/requirements_universal.txt | 6 +++--- tools/publish/requirements_windows.txt | 6 +++--- 4 files changed, 12 insertions(+), 12 deletions(-) diff --git a/tools/publish/requirements_darwin.txt b/tools/publish/requirements_darwin.txt index eaec72c01c..483f88444e 100644 --- a/tools/publish/requirements_darwin.txt +++ b/tools/publish/requirements_darwin.txt @@ -142,9 +142,9 @@ mdurl==0.1.2 \ --hash=sha256:84008a41e51615a49fc9966191ff91509e3c40b939176e643fd50a5c2196b8f8 \ --hash=sha256:bb413d29f5eea38f31dd4754dd7377d4465116fb207585f97bf925588687c1ba # via markdown-it-py -more-itertools==10.5.0 \ - --hash=sha256:037b0d3203ce90cca8ab1defbbdac29d5f993fc20131f3664dc8d6acfa872aef \ - --hash=sha256:5482bfef7849c25dc3c6dd53a6173ae4795da2a41a80faea6700d9f5846c5da6 +more-itertools==10.7.0 \ + --hash=sha256:9fddd5403be01a94b204faadcff459ec3568cf110265d3c54323e1e866ad29d3 \ + --hash=sha256:d43980384673cb07d2f7d2d918c616b30c659c089ee23953f601d6609c67510e # via # jaraco-classes # jaraco-functools diff --git a/tools/publish/requirements_linux.txt b/tools/publish/requirements_linux.txt index 5fdc742a88..62dbf1eb77 100644 --- a/tools/publish/requirements_linux.txt +++ b/tools/publish/requirements_linux.txt @@ -250,9 +250,9 @@ mdurl==0.1.2 \ --hash=sha256:84008a41e51615a49fc9966191ff91509e3c40b939176e643fd50a5c2196b8f8 \ --hash=sha256:bb413d29f5eea38f31dd4754dd7377d4465116fb207585f97bf925588687c1ba # via markdown-it-py -more-itertools==10.5.0 \ - --hash=sha256:037b0d3203ce90cca8ab1defbbdac29d5f993fc20131f3664dc8d6acfa872aef \ - --hash=sha256:5482bfef7849c25dc3c6dd53a6173ae4795da2a41a80faea6700d9f5846c5da6 +more-itertools==10.7.0 \ + --hash=sha256:9fddd5403be01a94b204faadcff459ec3568cf110265d3c54323e1e866ad29d3 \ + --hash=sha256:d43980384673cb07d2f7d2d918c616b30c659c089ee23953f601d6609c67510e # via # jaraco-classes # jaraco-functools diff --git a/tools/publish/requirements_universal.txt b/tools/publish/requirements_universal.txt index 97cbef0221..e4e876b176 100644 --- a/tools/publish/requirements_universal.txt +++ b/tools/publish/requirements_universal.txt @@ -250,9 +250,9 @@ mdurl==0.1.2 \ --hash=sha256:84008a41e51615a49fc9966191ff91509e3c40b939176e643fd50a5c2196b8f8 \ --hash=sha256:bb413d29f5eea38f31dd4754dd7377d4465116fb207585f97bf925588687c1ba # via markdown-it-py -more-itertools==10.5.0 \ - --hash=sha256:037b0d3203ce90cca8ab1defbbdac29d5f993fc20131f3664dc8d6acfa872aef \ - --hash=sha256:5482bfef7849c25dc3c6dd53a6173ae4795da2a41a80faea6700d9f5846c5da6 +more-itertools==10.7.0 \ + --hash=sha256:9fddd5403be01a94b204faadcff459ec3568cf110265d3c54323e1e866ad29d3 \ + --hash=sha256:d43980384673cb07d2f7d2d918c616b30c659c089ee23953f601d6609c67510e # via # jaraco-classes # jaraco-functools diff --git a/tools/publish/requirements_windows.txt b/tools/publish/requirements_windows.txt index 458414009e..043de9ecb1 100644 --- a/tools/publish/requirements_windows.txt +++ b/tools/publish/requirements_windows.txt @@ -142,9 +142,9 @@ mdurl==0.1.2 \ --hash=sha256:84008a41e51615a49fc9966191ff91509e3c40b939176e643fd50a5c2196b8f8 \ --hash=sha256:bb413d29f5eea38f31dd4754dd7377d4465116fb207585f97bf925588687c1ba # via markdown-it-py -more-itertools==10.5.0 \ - --hash=sha256:037b0d3203ce90cca8ab1defbbdac29d5f993fc20131f3664dc8d6acfa872aef \ - --hash=sha256:5482bfef7849c25dc3c6dd53a6173ae4795da2a41a80faea6700d9f5846c5da6 +more-itertools==10.7.0 \ + --hash=sha256:9fddd5403be01a94b204faadcff459ec3568cf110265d3c54323e1e866ad29d3 \ + --hash=sha256:d43980384673cb07d2f7d2d918c616b30c659c089ee23953f601d6609c67510e # via # jaraco-classes # jaraco-functools From efc7589af6ba7fddf249b082ebfa29d7e260e0e6 Mon Sep 17 00:00:00 2001 From: Ignas Anikevicius <240938+aignas@users.noreply.github.com> Date: Sun, 11 May 2025 11:27:25 +0900 Subject: [PATCH 214/922] fix(pypi): finish PEP508/PEP440 impl for version matching (#2856) This reuses the previous work by @vonschultz who implemented a PEP440 version normalizer. We extend it and use it in the PEP508 marker evaluation. Summary: - Extend the normalization parser to output individual parts of the versions to the parsing context. - Re-implement all of the version comparison calls to use the parsed version. - Add extra validation for `.*` usage in the environment markers - Fallback to non-version matching in the environment markers if one of the sides is not a version. - Rename the original normalizer file to `version.bzl` because as far as Python is concerned this is the only version that there can be. We could in theory probably reuse this in other code where we are parsing the Python interpreter version many times, but this is left for the future. Fixes #2826 Work towards #2821 --------- Co-authored-by: Richard Levasseur Co-authored-by: Richard Levasseur --- .bazelrc | 4 +- CHANGELOG.md | 6 +- python/BUILD.bazel | 2 +- python/private/BUILD.bazel | 7 +- python/private/py_wheel.bzl | 8 +- python/private/pypi/BUILD.bazel | 1 + python/private/pypi/pep508_evaluate.bzl | 58 +-- python/private/semver.bzl | 27 -- ...wheel_normalize_pep440.bzl => version.bzl} | 379 +++++++++++++++++- tests/py_wheel/py_wheel_tests.bzl | 101 ----- tests/pypi/pep508/evaluate_tests.bzl | 127 ++++-- tests/semver/semver_test.bzl | 18 - tests/version/BUILD.bazel | 3 + tests/version/version_test.bzl | 157 ++++++++ 14 files changed, 641 insertions(+), 257 deletions(-) rename python/private/{py_wheel_normalize_pep440.bzl => version.bzl} (52%) create mode 100644 tests/version/BUILD.bazel create mode 100644 tests/version/version_test.bzl diff --git a/.bazelrc b/.bazelrc index d2e0721526..4e6f2fa187 100644 --- a/.bazelrc +++ b/.bazelrc @@ -4,8 +4,8 @@ # (Note, we cannot use `common --deleted_packages` because the bazel version command doesn't support it) # To update these lines, execute # `bazel run @rules_bazel_integration_test//tools:update_deleted_packages` -build --deleted_packages=examples/build_file_generation,examples/build_file_generation/random_number_generator,examples/bzlmod,examples/bzlmod/entry_points,examples/bzlmod/entry_points/tests,examples/bzlmod/libs/my_lib,examples/bzlmod/other_module,examples/bzlmod/other_module/other_module/pkg,examples/bzlmod/patches,examples/bzlmod/py_proto_library,examples/bzlmod/py_proto_library/example.com/another_proto,examples/bzlmod/py_proto_library/example.com/proto,examples/bzlmod/runfiles,examples/bzlmod/tests,examples/bzlmod/tests/other_module,examples/bzlmod/whl_mods,examples/bzlmod_build_file_generation,examples/bzlmod_build_file_generation/other_module/other_module/pkg,examples/bzlmod_build_file_generation/runfiles,examples/multi_python_versions/libs/my_lib,examples/multi_python_versions/requirements,examples/multi_python_versions/tests,examples/pip_parse,examples/pip_parse_vendored,examples/pip_repository_annotations,examples/py_proto_library,examples/py_proto_library/example.com/another_proto,examples/py_proto_library/example.com/proto,gazelle,gazelle/manifest,gazelle/manifest/generate,gazelle/manifest/hasher,gazelle/manifest/test,gazelle/modules_mapping,gazelle/python,gazelle/python/private,gazelle/pythonconfig,tests/integration/compile_pip_requirements,tests/integration/compile_pip_requirements_test_from_external_repo,tests/integration/custom_commands,tests/integration/ignore_root_user_error,tests/integration/ignore_root_user_error/submodule,tests/integration/local_toolchains,tests/integration/pip_parse,tests/integration/pip_parse/empty,tests/integration/py_cc_toolchain_registered,tests/modules/other,tests/modules/other/nspkg_delta,tests/modules/other/nspkg_gamma -query --deleted_packages=examples/build_file_generation,examples/build_file_generation/random_number_generator,examples/bzlmod,examples/bzlmod/entry_points,examples/bzlmod/entry_points/tests,examples/bzlmod/libs/my_lib,examples/bzlmod/other_module,examples/bzlmod/other_module/other_module/pkg,examples/bzlmod/patches,examples/bzlmod/py_proto_library,examples/bzlmod/py_proto_library/example.com/another_proto,examples/bzlmod/py_proto_library/example.com/proto,examples/bzlmod/runfiles,examples/bzlmod/tests,examples/bzlmod/tests/other_module,examples/bzlmod/whl_mods,examples/bzlmod_build_file_generation,examples/bzlmod_build_file_generation/other_module/other_module/pkg,examples/bzlmod_build_file_generation/runfiles,examples/multi_python_versions/libs/my_lib,examples/multi_python_versions/requirements,examples/multi_python_versions/tests,examples/pip_parse,examples/pip_parse_vendored,examples/pip_repository_annotations,examples/py_proto_library,examples/py_proto_library/example.com/another_proto,examples/py_proto_library/example.com/proto,gazelle,gazelle/manifest,gazelle/manifest/generate,gazelle/manifest/hasher,gazelle/manifest/test,gazelle/modules_mapping,gazelle/python,gazelle/python/private,gazelle/pythonconfig,tests/integration/compile_pip_requirements,tests/integration/compile_pip_requirements_test_from_external_repo,tests/integration/custom_commands,tests/integration/ignore_root_user_error,tests/integration/ignore_root_user_error/submodule,tests/integration/local_toolchains,tests/integration/pip_parse,tests/integration/pip_parse/empty,tests/integration/py_cc_toolchain_registered,tests/modules/other,tests/modules/other/nspkg_delta,tests/modules/other/nspkg_gamma +build --deleted_packages=examples/build_file_generation,examples/build_file_generation/random_number_generator,examples/bzlmod,examples/bzlmod_build_file_generation,examples/bzlmod_build_file_generation/other_module/other_module/pkg,examples/bzlmod_build_file_generation/runfiles,examples/bzlmod/entry_points,examples/bzlmod/entry_points/tests,examples/bzlmod/libs/my_lib,examples/bzlmod/other_module,examples/bzlmod/other_module/other_module/pkg,examples/bzlmod/patches,examples/bzlmod/py_proto_library,examples/bzlmod/py_proto_library/example.com/another_proto,examples/bzlmod/py_proto_library/example.com/proto,examples/bzlmod/runfiles,examples/bzlmod/tests,examples/bzlmod/tests/other_module,examples/bzlmod/whl_mods,examples/multi_python_versions/libs/my_lib,examples/multi_python_versions/requirements,examples/multi_python_versions/tests,examples/pip_parse,examples/pip_parse_vendored,examples/pip_repository_annotations,examples/py_proto_library,examples/py_proto_library/example.com/another_proto,examples/py_proto_library/example.com/proto,gazelle,gazelle/manifest,gazelle/manifest/generate,gazelle/manifest/hasher,gazelle/manifest/test,gazelle/modules_mapping,gazelle/python,gazelle/pythonconfig,gazelle/python/private,tests/integration/compile_pip_requirements,tests/integration/compile_pip_requirements_test_from_external_repo,tests/integration/custom_commands,tests/integration/ignore_root_user_error,tests/integration/ignore_root_user_error/submodule,tests/integration/local_toolchains,tests/integration/pip_parse,tests/integration/pip_parse/empty,tests/integration/py_cc_toolchain_registered,tests/modules/other,tests/modules/other/nspkg_delta,tests/modules/other/nspkg_gamma +query --deleted_packages=examples/build_file_generation,examples/build_file_generation/random_number_generator,examples/bzlmod,examples/bzlmod_build_file_generation,examples/bzlmod_build_file_generation/other_module/other_module/pkg,examples/bzlmod_build_file_generation/runfiles,examples/bzlmod/entry_points,examples/bzlmod/entry_points/tests,examples/bzlmod/libs/my_lib,examples/bzlmod/other_module,examples/bzlmod/other_module/other_module/pkg,examples/bzlmod/patches,examples/bzlmod/py_proto_library,examples/bzlmod/py_proto_library/example.com/another_proto,examples/bzlmod/py_proto_library/example.com/proto,examples/bzlmod/runfiles,examples/bzlmod/tests,examples/bzlmod/tests/other_module,examples/bzlmod/whl_mods,examples/multi_python_versions/libs/my_lib,examples/multi_python_versions/requirements,examples/multi_python_versions/tests,examples/pip_parse,examples/pip_parse_vendored,examples/pip_repository_annotations,examples/py_proto_library,examples/py_proto_library/example.com/another_proto,examples/py_proto_library/example.com/proto,gazelle,gazelle/manifest,gazelle/manifest/generate,gazelle/manifest/hasher,gazelle/manifest/test,gazelle/modules_mapping,gazelle/python,gazelle/pythonconfig,gazelle/python/private,tests/integration/compile_pip_requirements,tests/integration/compile_pip_requirements_test_from_external_repo,tests/integration/custom_commands,tests/integration/ignore_root_user_error,tests/integration/ignore_root_user_error/submodule,tests/integration/local_toolchains,tests/integration/pip_parse,tests/integration/pip_parse/empty,tests/integration/py_cc_toolchain_registered,tests/modules/other,tests/modules/other/nspkg_delta,tests/modules/other/nspkg_gamma test --test_output=errors diff --git a/CHANGELOG.md b/CHANGELOG.md index 5f67c8a5ec..aa7fc9d415 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -94,9 +94,9 @@ END_UNRELEASED_TEMPLATE (the default), the subprocess's stdout/stderr will be logged. * (toolchains) Local toolchains can be activated with custom flags. See [Conditionally using local toolchains] docs for how to configure. -* (pypi) `RULES_PYTHON_ENABLE_PIPSTAR` environment variable: when `1`, the Starlark - implementation of wheel METADATA parsing is used (which has improved multi-platform - build support). +* (pypi) Starlark-based evaluation of environment markers (requirements.txt conditionals) + available (not enabled by default) for improved multi-platform build support. + Set the `RULES_PYTHON_ENABLE_PIPSTAR=1` environment variable to enable it. {#v0-0-0-removed} ### Removed diff --git a/python/BUILD.bazel b/python/BUILD.bazel index 3389a0dacc..867c43478a 100644 --- a/python/BUILD.bazel +++ b/python/BUILD.bazel @@ -93,9 +93,9 @@ bzl_library( "//python/private:bzlmod_enabled_bzl", "//python/private:py_package.bzl", "//python/private:py_wheel_bzl", - "//python/private:py_wheel_normalize_pep440.bzl", "//python/private:stamp_bzl", "//python/private:util_bzl", + "//python/private:version.bzl", "@bazel_skylib//rules:native_binary", ], ) diff --git a/python/private/BUILD.bazel b/python/private/BUILD.bazel index 9cc8ffc62c..e72a8fcaa7 100644 --- a/python/private/BUILD.bazel +++ b/python/private/BUILD.bazel @@ -658,6 +658,11 @@ bzl_library( ], ) +bzl_library( + name = "version_bzl", + srcs = ["version.bzl"], +) + bzl_library( name = "version_label_bzl", srcs = ["version_label.bzl"], @@ -701,7 +706,7 @@ exports_files( "repack_whl.py", "py_package.bzl", "py_wheel.bzl", - "py_wheel_normalize_pep440.bzl", + "version.bzl", "reexports.bzl", "stamp.bzl", "util.bzl", diff --git a/python/private/py_wheel.bzl b/python/private/py_wheel.bzl index c196ca6ad0..ffc24f6846 100644 --- a/python/private/py_wheel.bzl +++ b/python/private/py_wheel.bzl @@ -16,8 +16,8 @@ load(":py_info.bzl", "PyInfo") load(":py_package.bzl", "py_package_lib") -load(":py_wheel_normalize_pep440.bzl", "normalize_pep440") load(":stamp.bzl", "is_stamping_enabled") +load(":version.bzl", "version") PyWheelInfo = provider( doc = "Information about a wheel produced by `py_wheel`", @@ -306,11 +306,11 @@ def _input_file_to_arg(input_file): def _py_wheel_impl(ctx): abi = _replace_make_variables(ctx.attr.abi, ctx) python_tag = _replace_make_variables(ctx.attr.python_tag, ctx) - version = _replace_make_variables(ctx.attr.version, ctx) + version_str = _replace_make_variables(ctx.attr.version, ctx) filename_segments = [ _escape_filename_distribution_name(ctx.attr.distribution), - normalize_pep440(version), + version.normalize(version_str), _escape_filename_segment(python_tag), _escape_filename_segment(abi), _escape_filename_segment(ctx.attr.platform), @@ -343,7 +343,7 @@ def _py_wheel_impl(ctx): args = ctx.actions.args() args.add("--name", ctx.attr.distribution) - args.add("--version", version) + args.add("--version", version_str) args.add("--python_tag", python_tag) args.add("--abi", abi) args.add("--platform", ctx.attr.platform) diff --git a/python/private/pypi/BUILD.bazel b/python/private/pypi/BUILD.bazel index d5d897ef8c..f541cbe98b 100644 --- a/python/private/pypi/BUILD.bazel +++ b/python/private/pypi/BUILD.bazel @@ -251,6 +251,7 @@ bzl_library( srcs = ["pep508_env.bzl"], deps = [ ":pep508_platform_bzl", + "//python/private:version_bzl", ], ) diff --git a/python/private/pypi/pep508_evaluate.bzl b/python/private/pypi/pep508_evaluate.bzl index 70840c76c6..61a5b19999 100644 --- a/python/private/pypi/pep508_evaluate.bzl +++ b/python/private/pypi/pep508_evaluate.bzl @@ -16,23 +16,11 @@ """ load("//python/private:enum.bzl", "enum") -load("//python/private:semver.bzl", "semver") +load("//python/private:version.bzl", "version") # The expression parsing and resolution for the PEP508 is below # -# Taken from -# https://peps.python.org/pep-0508/#grammar -# -# version_cmp = wsp* '<' | '<=' | '!=' | '==' | '>=' | '>' | '~=' | '===' -_VERSION_CMP = sorted( - [ - i.strip(" '") - for i in "'<' | '<=' | '!=' | '==' | '>=' | '>' | '~=' | '==='".split(" | ") - ], - key = lambda x: (-len(x), x), -) - _STATE = enum( STRING = "string", VAR = "var", @@ -353,36 +341,34 @@ def _env_expr(left, op, right): elif op == ">=": return left >= right else: - return fail("TODO: op unsupported: '{}'".format(op)) + return fail("unsupported op: '{}' {} '{}'".format(left, op, right)) def _version_expr(left, op, right): """Evaluate a version comparison expression""" - left = semver(left) - right = semver(right) - _left = left.key() - _right = right.key() - if op == "<": - return _left < _right + _left = version.parse(left) + _right = version.parse(right) + if _left == None or _right == None: + # Per spec, if either can't be normalized to a version, then + # fallback to simple string comparison. Usually this is `platform_version` + # or `platform_release`, which vary depending on platform. + return _env_expr(left, op, right) + + if op == "===": + return version.is_eeq(_left, _right) + elif op == "!=": + return version.is_ne(_left, _right) + elif op == "==": + return version.is_eq(_left, _right) + elif op == "<": + return version.is_lt(_left, _right) elif op == ">": - return _left > _right + return version.is_gt(_left, _right) elif op == "<=": - return _left <= _right + return version.is_le(_left, _right) elif op == ">=": - return _left >= _right - elif op == "!=": - return _left != _right - elif op == "==": - # Matching of major, minor, patch only - return _left[:3] == _right[:3] + return version.is_ge(_left, _right) elif op == "~=": - right_plus = right.upper() - _right_plus = right_plus.key() - return _left >= _right and _left < _right_plus - elif op == "===": - # Strict matching - return _left == _right - elif op in _VERSION_CMP: - fail("TODO: op unsupported: '{}'".format(op)) + return version.is_compatible(_left, _right) else: return False # Let's just ignore the invalid ops diff --git a/python/private/semver.bzl b/python/private/semver.bzl index cc9ae6ecb6..0cbd172348 100644 --- a/python/private/semver.bzl +++ b/python/private/semver.bzl @@ -43,32 +43,6 @@ def _to_dict(self): "pre_release": self.pre_release, } -def _upper(self): - major = self.major - minor = self.minor - patch = self.patch - build = "" - pre_release = "" - version = self.str() - - if patch != None: - minor = minor + 1 - patch = 0 - elif minor != None: - major = major + 1 - minor = 0 - elif minor == None: - major = major + 1 - - return _new( - major = major, - minor = minor, - patch = patch, - build = build, - pre_release = pre_release, - version = "~" + version, - ) - def _new(*, major, minor, patch, pre_release, build, version = None): # buildifier: disable=uninitialized self = struct( @@ -82,7 +56,6 @@ def _new(*, major, minor, patch, pre_release, build, version = None): key = lambda: _key(self), str = lambda: version, to_dict = lambda: _to_dict(self), - upper = lambda: _upper(self), ) return self diff --git a/python/private/py_wheel_normalize_pep440.bzl b/python/private/version.bzl similarity index 52% rename from python/private/py_wheel_normalize_pep440.bzl rename to python/private/version.bzl index 9566348987..4425cc7661 100644 --- a/python/private/py_wheel_normalize_pep440.bzl +++ b/python/private/version.bzl @@ -59,18 +59,23 @@ def _open_context(self): self.contexts.append(_ctx(_context(self)["start"])) return self.contexts[-1] -def _accept(self): +def _accept(self, key = None): """Close the current ctx successfully and merge the results.""" finished = self.contexts.pop() self.contexts[-1]["norm"] += finished["norm"] + if key: + self.contexts[-1][key] = finished["norm"] + self.contexts[-1]["start"] = finished["start"] return True def _context(self): return self.contexts[-1] -def _discard(self): +def _discard(self, key = None): self.contexts.pop() + if key: + self.contexts[-1][key] = "" return False def _new(input): @@ -313,9 +318,9 @@ def accept_epoch(parser): if accept_digits(parser) and accept(parser, _is("!"), "!"): if ctx["norm"] == "0!": ctx["norm"] = "" - return parser.accept() + return parser.accept("epoch") else: - return parser.discard() + return parser.discard("epoch") def accept_release(parser): """Accept the release segment, numbers separated by dots. @@ -329,10 +334,10 @@ def accept_release(parser): parser.open_context() if not accept_digits(parser): - return parser.discard() + return parser.discard("release") accept_dot_number_sequence(parser) - return parser.accept() + return parser.accept("release") def accept_pre_l(parser): """PEP 440: Pre-release spelling. @@ -374,7 +379,7 @@ def accept_prerelease(parser): accept(parser, _in(["-", "_", "."]), "") if not accept_pre_l(parser): - return parser.discard() + return parser.discard("pre") accept(parser, _in(["-", "_", "."]), "") @@ -382,7 +387,7 @@ def accept_prerelease(parser): # PEP 440: Implicit pre-release number ctx["norm"] += "0" - return parser.accept() + return parser.accept("pre") def accept_implicit_postrelease(parser): """PEP 440: Implicit post releases. @@ -444,9 +449,9 @@ def accept_postrelease(parser): parser.open_context() if accept_implicit_postrelease(parser) or accept_explicit_postrelease(parser): - return parser.accept() + return parser.accept("post") - return parser.discard() + return parser.discard("post") def accept_devrelease(parser): """PEP 440: Developmental releases. @@ -470,9 +475,9 @@ def accept_devrelease(parser): # PEP 440: Implicit development release number ctx["norm"] += "0" - return parser.accept() + return parser.accept("dev") - return parser.discard() + return parser.discard("dev") def accept_local(parser): """PEP 440: Local version identifiers. @@ -487,9 +492,9 @@ def accept_local(parser): if accept(parser, _is("+"), "+") and accept_alnum(parser): accept_separator_alnum_sequence(parser) - return parser.accept() + return parser.accept("local") - return parser.discard() + return parser.discard("local") def normalize_pep440(version): """Escape the version component of a filename. @@ -503,7 +508,31 @@ def normalize_pep440(version): Returns: string containing the normalized version. """ - parser = _new(version.strip()) # PEP 440: Leading and Trailing Whitespace + return _parse(version, strict = True)["norm"] + +def _parse(version_str, strict = True): + """Escape the version component of a filename. + + See https://packaging.python.org/en/latest/specifications/binary-distribution-format/#escaping-and-unicode + and https://peps.python.org/pep-0440/ + + Args: + version_str: version string to be normalized according to PEP 440. + strict: fail if the version is invalid, defaults to True. + + Returns: + string containing the normalized version. + """ + + # https://packaging.python.org/en/latest/specifications/version-specifiers/#leading-and-trailing-whitespace + version = version_str.strip() + is_prefix = False + + if not strict: + is_prefix = version.endswith(".*") + version = version.strip(" .*") # PEP 440: Leading and Trailing Whitespace and ".*" + + parser = _new(version) accept(parser, _is("v"), "") # PEP 440: Preceding v character accept_epoch(parser) accept_release(parser) @@ -511,9 +540,317 @@ def normalize_pep440(version): accept_postrelease(parser) accept_devrelease(parser) accept_local(parser) - if parser.input[parser.context()["start"]:]: - fail( - "Failed to parse PEP 440 version identifier '%s'." % parser.input, - "Parse error at '%s'" % parser.input[parser.context()["start"]:], - ) - return parser.context()["norm"] + + parser_ctx = parser.context() + if parser.input[parser_ctx["start"]:]: + if strict: + fail( + "Failed to parse PEP 440 version identifier '%s'." % parser.input, + "Parse error at '%s'" % parser.input[parser_ctx["start"]:], + ) + + return None + + parser_ctx["is_prefix"] = is_prefix + return parser_ctx + +def parse(version_str, strict = False): + """Parse a PEP4408 compliant version. + + This is similar to `normalize_pep440`, but it parses individual components to + comparable types. + + Args: + version_str: version string to be normalized according to PEP 440. + strict: fail if the version is invalid. + + Returns: + a struct with individual components of a version: + * `epoch` {type}`int`, defaults to `0` + * `release` {type}`tuple[int]` an n-tuple of ints + * `pre` {type}`tuple[str, int] | None` a tuple of a string and an int, + e.g. ("a", 1) + * `post` {type}`tuple[str, int] | None` a tuple of a string and an int, + e.g. ("~", 1) + * `dev` {type}`tuple[str, int] | None` a tuple of a string and an int, + e.g. ("", 1) + * `local` {type}`tuple[str, int] | None` a tuple of components in the local + version, e.g. ("abc", 123). + * `is_prefix` {type}`bool` whether the version_str ends with `.*`. + * `string` {type}`str` normalized value of the input. + """ + + parts = _parse(version_str, strict = strict) + if not parts: + return None + + if parts["is_prefix"] and (parts["local"] or parts["post"] or parts["dev"] or parts["pre"]): + if strict: + fail("local version part has been obtained, but only public segments can have prefix matches") + + # https://peps.python.org/pep-0440/#public-version-identifiers + return None + + return struct( + epoch = _parse_epoch(parts["epoch"]), + release = _parse_release(parts["release"]), + pre = _parse_pre(parts["pre"]), + post = _parse_post(parts["post"]), + dev = _parse_dev(parts["dev"]), + local = _parse_local(parts["local"]), + string = parts["norm"], + is_prefix = parts["is_prefix"], + ) + +def _parse_epoch(value): + if not value: + return 0 + + if not value.endswith("!"): + fail("epoch string segment needs to end with '!', got: {}".format(value)) + + return int(value[:-1]) + +def _parse_release(value): + return tuple([int(d) for d in value.split(".")]) + +def _parse_local(value): + if not value: + return None + + if not value.startswith("+"): + fail("local release identifier must start with '+', got: {}".format(value)) + + # If the part is numerical, handle it as a number + return tuple([int(part) if part.isdigit() else part for part in value[1:].split(".")]) + +def _parse_dev(value): + if not value: + return None + + if not value.startswith(".dev"): + fail("dev release identifier must start with '.dev', got: {}".format(value)) + dev = int(value[len(".dev"):]) + + # Empty string goes first when comparing + return ("", dev) + +def _parse_pre(value): + if not value: + return None + + if value.startswith("rc"): + prefix = "rc" + else: + prefix = value[0] + + return (prefix, int(value[len(prefix):])) + +def _parse_post(value): + if not value: + return None + + if not value.startswith(".post"): + fail("post release identifier must start with '.post', got: {}".format(value)) + post = int(value[len(".post"):]) + + # We choose `~` since almost all of the ASCII characters will be before + # it. Use `ord` and `chr` functions to find a good value. + return ("~", post) + +def _pad_zeros(release, n): + padding = n - len(release) + if padding <= 0: + return release + + release = list(release) + [0] * padding + return tuple(release) + +def _prefix_err(left, op, right): + if left.is_prefix or right.is_prefix: + fail("PEP440: only '==' and '!=' operators can use prefix matching: {} {} {}".format( + left.string, + op, + right.string, + )) + +def _version_eeq(left, right): + """=== operator""" + if left.is_prefix or right.is_prefix: + fail(_prefix_err(left, "===", right)) + + # https://peps.python.org/pep-0440/#arbitrary-equality + # > simple string equality operations + return left.string == right.string + +def _version_eq(left, right): + """== operator""" + if left.is_prefix and right.is_prefix: + fail("Invalid comparison: both versions cannot be prefix matching") + if left.is_prefix: + return right.string.startswith("{}.".format(left.string)) + if right.is_prefix: + return left.string.startswith("{}.".format(right.string)) + + if left.epoch != right.epoch: + return False + + release_len = max(len(left.release), len(right.release)) + left_release = _pad_zeros(left.release, release_len) + right_release = _pad_zeros(right.release, release_len) + + if left_release != right_release: + return False + + return ( + left.pre == right.pre and + left.post == right.post and + left.dev == right.dev + # local is ignored for == checks + ) + +def _version_compatible(left, right): + """~= operator""" + if left.is_prefix or right.is_prefix: + fail(_prefix_err(left, "~=", right)) + + # https://peps.python.org/pep-0440/#compatible-release + # Note, the ~= operator can be also expressed as: + # >= V.N, == V.* + + right_star = ".".join([str(d) for d in right.release[:-1]]) + if right.epoch: + right_star = "{}!{}.".format(right.epoch, right_star) + else: + right_star = "{}.".format(right_star) + + return _version_ge(left, right) and left.string.startswith(right_star) + +def _version_ne(left, right): + """!= operator""" + return not _version_eq(left, right) + +def _version_lt(left, right): + """< operator""" + if left.is_prefix or right.is_prefix: + fail(_prefix_err(left, "<", right)) + + if left.epoch > right.epoch: + return False + elif left.epoch < right.epoch: + return True + + release_len = max(len(left.release), len(right.release)) + left_release = _pad_zeros(left.release, release_len) + right_release = _pad_zeros(right.release, release_len) + + if left_release > right_release: + return False + elif left_release < right_release: + return True + + # From PEP440, this is not a simple ordering check and we need to check the version + # semantically: + # * The exclusive ordered comparison operator""" + if left.is_prefix or right.is_prefix: + fail(_prefix_err(left, ">", right)) + + if left.epoch > right.epoch: + return True + elif left.epoch < right.epoch: + return False + + release_len = max(len(left.release), len(right.release)) + left_release = _pad_zeros(left.release, release_len) + right_release = _pad_zeros(right.release, release_len) + + if left_release > right_release: + return True + elif left_release < right_release: + return False + + # From PEP440, this is not a simple ordering check and we need to check the version + # semantically: + # * The exclusive ordered comparison >V MUST NOT allow a post-release of the given version + # unless V itself is a post release. + # + # * The exclusive ordered comparison >V MUST NOT match a local version of the specified + # version. + + if left.post and right.post: + return left.post > right.post + else: + # ignore the left.post if right is not a post if right is a post, then this evaluates to + # False anyway. + return False + +def _version_le(left, right): + """<= operator""" + if left.is_prefix or right.is_prefix: + fail(_prefix_err(left, "<=", right)) + + # PEP440: simple order check + # https://peps.python.org/pep-0440/#inclusive-ordered-comparison + _left = _version_key(left, local = False) + _right = _version_key(right, local = False) + return _left < _right or _version_eq(left, right) + +def _version_ge(left, right): + """>= operator""" + if left.is_prefix or right.is_prefix: + fail(_prefix_err(left, ">=", right)) + + # PEP440: simple order check + # https://peps.python.org/pep-0440/#inclusive-ordered-comparison + _left = _version_key(left, local = False) + _right = _version_key(right, local = False) + return _left > _right or _version_eq(left, right) + +def _version_key(self, *, local = True): + """This function returns a tuple that can be used in 'sorted' calls. + + This implements the PEP440 version sorting. + """ + release_key = ("z",) + local = self.local if local else [] + local = local or [] + + return ( + self.epoch, + self.release, + # PEP440 Within a pre-release, post-release or development release segment with + # a shared prefix, ordering MUST be by the value of the numeric component. + # PEP440 release ordering: .devN, aN, bN, rcN, , .postN + # We choose to first match the pre-release, then post release, then dev and + # then stable + self.pre or self.post or self.dev or release_key, + # PEP440 local versions go before post versions + tuple([(type(item) == "int", item) for item in local]), + # PEP440 - pre-release ordering: .devN, , .postN + self.post or self.dev or release_key, + # PEP440 - post release ordering: .devN, + self.dev or release_key, + ) + +version = struct( + normalize = normalize_pep440, + parse = parse, + # methods, keep sorted + key = _version_key, + is_compatible = _version_compatible, + is_eq = _version_eq, + is_eeq = _version_eeq, + is_ge = _version_ge, + is_gt = _version_gt, + is_le = _version_le, + is_lt = _version_lt, + is_ne = _version_ne, +) diff --git a/tests/py_wheel/py_wheel_tests.bzl b/tests/py_wheel/py_wheel_tests.bzl index 091e01c37d..43c068e597 100644 --- a/tests/py_wheel/py_wheel_tests.bzl +++ b/tests/py_wheel/py_wheel_tests.bzl @@ -17,7 +17,6 @@ load("@rules_testing//lib:analysis_test.bzl", "analysis_test", "test_suite") load("@rules_testing//lib:truth.bzl", "matching") load("@rules_testing//lib:util.bzl", rt_util = "util") load("//python:packaging.bzl", "py_wheel") -load("//python/private:py_wheel_normalize_pep440.bzl", "normalize_pep440") # buildifier: disable=bzl-visibility _basic_tests = [] _tests = [] @@ -168,106 +167,6 @@ def _test_content_type_from_description_impl(env, target): _tests.append(_test_content_type_from_description) -def _test_pep440_normalization(env): - prefixes = ["v", " v", " \t\r\nv"] - epochs = { - "": ["", "0!", "00!"], - "1!": ["1!", "001!"], - "200!": ["200!", "00200!"], - } - releases = { - "0.1": ["0.1", "0.01"], - "2023.7.19": ["2023.7.19", "2023.07.19"], - } - pres = { - "": [""], - "a0": ["a", ".a", "-ALPHA0", "_alpha0", ".a0"], - "a4": ["alpha4", ".a04"], - "b0": ["b", ".b", "-BETA0", "_beta0", ".b0"], - "b5": ["beta05", ".b5"], - "rc0": ["C", "_c0", "RC", "_rc0", "-preview_0"], - } - explicit_posts = { - "": [""], - ".post0": [], - ".post1": [".post1", "-r1", "_rev1"], - } - implicit_posts = [[".post1", "-1"], [".post2", "-2"]] - devs = { - "": [""], - ".dev0": ["dev", "-DEV", "_Dev-0"], - ".dev9": ["DEV9", ".dev09", ".dev9"], - ".dev{BUILD_TIMESTAMP}": [ - "-DEV{BUILD_TIMESTAMP}", - "_dev_{BUILD_TIMESTAMP}", - ], - } - locals = { - "": [""], - "+ubuntu.7": ["+Ubuntu_7", "+ubuntu-007"], - "+ubuntu.r007": ["+Ubuntu_R007"], - } - epochs = [ - [normalized_epoch, input_epoch] - for normalized_epoch, input_epochs in epochs.items() - for input_epoch in input_epochs - ] - releases = [ - [normalized_release, input_release] - for normalized_release, input_releases in releases.items() - for input_release in input_releases - ] - pres = [ - [normalized_pre, input_pre] - for normalized_pre, input_pres in pres.items() - for input_pre in input_pres - ] - explicit_posts = [ - [normalized_post, input_post] - for normalized_post, input_posts in explicit_posts.items() - for input_post in input_posts - ] - pres_and_posts = [ - [normalized_pre + normalized_post, input_pre + input_post] - for normalized_pre, input_pre in pres - for normalized_post, input_post in explicit_posts - ] + [ - [normalized_pre + normalized_post, input_pre + input_post] - for normalized_pre, input_pre in pres - for normalized_post, input_post in implicit_posts - if input_pre == "" or input_pre[-1].isdigit() - ] - devs = [ - [normalized_dev, input_dev] - for normalized_dev, input_devs in devs.items() - for input_dev in input_devs - ] - locals = [ - [normalized_local, input_local] - for normalized_local, input_locals in locals.items() - for input_local in input_locals - ] - postfixes = ["", " ", " \t\r\n"] - i = 0 - for nepoch, iepoch in epochs: - for nrelease, irelease in releases: - for nprepost, iprepost in pres_and_posts: - for ndev, idev in devs: - for nlocal, ilocal in locals: - prefix = prefixes[i % len(prefixes)] - postfix = postfixes[(i // len(prefixes)) % len(postfixes)] - env.expect.that_str( - normalize_pep440( - prefix + iepoch + irelease + iprepost + - idev + ilocal + postfix, - ), - ).equals( - nepoch + nrelease + nprepost + ndev + nlocal, - ) - i += 1 - -_basic_tests.append(_test_pep440_normalization) - def py_wheel_test_suite(name): test_suite( name = name, diff --git a/tests/pypi/pep508/evaluate_tests.bzl b/tests/pypi/pep508/evaluate_tests.bzl index 303c167900..7b6c064b94 100644 --- a/tests/pypi/pep508/evaluate_tests.bzl +++ b/tests/pypi/pep508/evaluate_tests.bzl @@ -19,6 +19,12 @@ load("//python/private/pypi:pep508_evaluate.bzl", "evaluate", "tokenize") # bui _tests = [] +def _check_evaluate(env, expr, expected, values, strict = True): + env.expect.where( + expression = expr, + values = values, + ).that_bool(evaluate(expr, env = values, strict = strict)).equals(expected) + def _tokenize_tests(env): for input, want in { "": [], @@ -82,23 +88,11 @@ def _evaluate_non_version_env_tests(env): "{} > 'osx'".format(var_name): False, "{} >= 'osx'".format(var_name): True, }.items(): - got = evaluate( - input, - env = marker_env, - ) - env.expect.where( - expr = input, - env = marker_env, - ).that_bool(got).equals(want) + _check_evaluate(env, input, want, marker_env) # Check that the non-strict eval gives us back the input when no # env is supplied. - got = evaluate( - input, - env = {}, - strict = False, - ) - env.expect.that_bool(got).equals(input.replace("'", '"')) + _check_evaluate(env, input, input.replace("'", '"'), {}, strict = False) _tests.append(_evaluate_non_version_env_tests) @@ -123,6 +117,7 @@ def _evaluate_version_env_tests(env): "{} <= '3.7.10'".format(var_name): True, "{} <= '3.7.8'".format(var_name): False, "{} == '3.7.9'".format(var_name): True, + "{} == '3.7.*'".format(var_name): True, "{} != '3.7.9'".format(var_name): False, "{} ~= '3.7.1'".format(var_name): True, "{} ~= '3.7.10'".format(var_name): False, @@ -131,23 +126,32 @@ def _evaluate_version_env_tests(env): "{} === '3.7.9'".format(var_name): True, "{} == '3.7.9+rc2'".format(var_name): True, }.items(): # buildifier: @unsorted-dict-items - got = evaluate( - input, - env = marker_env, - ) - env.expect.that_collection((input, got)).contains_exactly((input, want)) + _check_evaluate(env, input, want, marker_env) # Check that the non-strict eval gives us back the input when no # env is supplied. - got = evaluate( - input, - env = {}, - strict = False, - ) - env.expect.that_bool(got).equals(input.replace("'", '"')) + _check_evaluate(env, input, input.replace("'", '"'), {}, strict = False) _tests.append(_evaluate_version_env_tests) +def _evaluate_platform_version_is_special(env): + # Given + marker_env = {"platform_version": "FooBar Linux v1.2.3"} + + # When the platform version is not + input = "platform_version == '0'" + _check_evaluate(env, input, False, marker_env) + + # And when I compare it as string + input = "'FooBar' in platform_version" + _check_evaluate(env, input, True, marker_env) + + # Check that the non-strict eval gives us back the input when no + # env is supplied. + _check_evaluate(env, input, input.replace("'", '"'), {}, strict = False) + +_tests.append(_evaluate_platform_version_is_special) + def _logical_expression_tests(env): for input, want in { # Basic @@ -195,13 +199,7 @@ def _logical_expression_tests(env): "not not os_name == 'foo'": True, "not not not os_name == 'foo'": False, }.items(): # buildifier: @unsorted-dict-items - got = evaluate( - input, - env = { - "os_name": "foo", - }, - ) - env.expect.that_collection((input, got)).contains_exactly((input, want)) + _check_evaluate(env, input, want, {"os_name": "foo"}) if not input.strip("()"): # These cases will just return True, because they will be evaluated @@ -210,12 +208,7 @@ def _logical_expression_tests(env): # Check that the non-strict eval gives us back the input when no env # is supplied. - got = evaluate( - input, - env = {}, - strict = False, - ) - env.expect.that_bool(got).equals(input.replace("'", '"')) + _check_evaluate(env, input, input.replace("'", '"'), {}, strict = False) _tests.append(_logical_expression_tests) @@ -244,6 +237,7 @@ def _evaluate_partial_only_extra(env): strict = False, ) env.expect.that_bool(got).equals(want) + _check_evaluate(env, input, want, {"extra": extra}, strict = False) _tests.append(_evaluate_partial_only_extra) @@ -268,14 +262,61 @@ def _evaluate_with_aliases(env): }, }.items(): # buildifier: @unsorted-dict-items for input, want in tests.items(): - got = evaluate( - input, - env = pep508_env(target_platform), - ) - env.expect.that_bool(got).equals(want) + _check_evaluate(env, input, want, pep508_env(target_platform)) _tests.append(_evaluate_with_aliases) +def _expr_case(expr, want, env): + return struct(expr = expr.strip(), want = want, env = env) + +_MISC_EXPRESSIONS = [ + _expr_case('python_version == "3.*"', True, {"python_version": "3.10.1"}), + _expr_case('python_version != "3.10.*"', False, {"python_version": "3.10.1"}), + _expr_case('python_version != "3.11.*"', True, {"python_version": "3.10.1"}), + _expr_case('python_version != "3.10"', False, {"python_version": "3.10.0"}), + _expr_case('python_version == "3.10"', True, {"python_version": "3.10.0"}), + # Cases for the '>' operator + # Taken from spec: https://peps.python.org/pep-0440/#exclusive-ordered-comparison + _expr_case('python_version > "1.7"', True, {"python_version": "1.7.1"}), + _expr_case('python_version > "1.7"', False, {"python_version": "1.7.0.post0"}), + _expr_case('python_version > "1.7"', True, {"python_version": "1.7.1"}), + _expr_case('python_version > "1.7.post2"', True, {"python_version": "1.7.1"}), + _expr_case('python_version > "1.7.post2"', True, {"python_version": "1.7.post3"}), + _expr_case('python_version > "1.7.post2"', False, {"python_version": "1.7.0"}), + _expr_case('python_version > "1.7.1+local"', False, {"python_version": "1.7.1"}), + _expr_case('python_version > "1.7.1+local"', True, {"python_version": "1.7.2"}), + # Extra cases for the '<' operator + _expr_case('python_version < "1.7.1"', False, {"python_version": "1.7.2"}), + _expr_case('python_version < "1.7.3"', True, {"python_version": "1.7.2"}), + _expr_case('python_version < "1.7.1"', True, {"python_version": "1.7"}), + _expr_case('python_version < "1.7.1"', False, {"python_version": "1.7.1-rc2"}), + _expr_case('python_version < "1.7.1-rc3"', True, {"python_version": "1.7.1-rc2"}), + _expr_case('python_version < "1.7.1-rc1"', False, {"python_version": "1.7.1-rc2"}), + # Extra tests + _expr_case('python_version <= "1.7.1"', True, {"python_version": "1.7.1"}), + _expr_case('python_version <= "1.7.2"', True, {"python_version": "1.7.1"}), + _expr_case('python_version >= "1.7.1"', True, {"python_version": "1.7.1"}), + _expr_case('python_version >= "1.7.0"', True, {"python_version": "1.7.1"}), + # Compatible version tests: + # https://packaging.python.org/en/latest/specifications/version-specifiers/#compatible-release + _expr_case('python_version ~= "2.2"', True, {"python_version": "2.3"}), + _expr_case('python_version ~= "2.2"', False, {"python_version": "2.1"}), + _expr_case('python_version ~= "2.2.post3"', False, {"python_version": "2.2"}), + _expr_case('python_version ~= "2.2.post3"', True, {"python_version": "2.3"}), + _expr_case('python_version ~= "2.2.post3"', False, {"python_version": "3.0"}), + _expr_case('python_version ~= "1!2.2"', False, {"python_version": "2.7"}), + _expr_case('python_version ~= "0!2.2"', True, {"python_version": "2.7"}), + _expr_case('python_version ~= "1!2.2"', True, {"python_version": "1!2.7"}), + _expr_case('python_version ~= "1.2.3"', True, {"python_version": "1.2.4"}), + _expr_case('python_version ~= "1.2.3"', False, {"python_version": "1.3.2"}), +] + +def _misc_expressions(env): + for case in _MISC_EXPRESSIONS: + _check_evaluate(env, case.expr, case.want, case.env) + +_tests.append(_misc_expressions) + def evaluate_test_suite(name): # buildifier: disable=function-docstring test_suite( name = name, diff --git a/tests/semver/semver_test.bzl b/tests/semver/semver_test.bzl index aef3deca82..9d13402c92 100644 --- a/tests/semver/semver_test.bzl +++ b/tests/semver/semver_test.bzl @@ -104,24 +104,6 @@ def _test_semver_sort(env): _tests.append(_test_semver_sort) -def _test_upper(env): - for input, want in { - # Depending on how many version numbers are specified we will increase - # the upper bound differently. See https://packaging.python.org/en/latest/specifications/version-specifiers/#compatible-release for docs - "0.0.1": "0.1.0", - "0.1": "1.0", - "0.1.0": "0.2.0", - "1": "2", - "1.0.0-pre": "1.1.0", # pre-release info is dropped - "1.2.0": "1.3.0", - "2.0.0+build0": "2.1.0", # build info is dropped - }.items(): - actual = semver(input).upper().key() - want = semver(want).key() - env.expect.that_collection(actual).contains_exactly(want).in_order() - -_tests.append(_test_upper) - def semver_test_suite(name): """Create the test suite. diff --git a/tests/version/BUILD.bazel b/tests/version/BUILD.bazel new file mode 100644 index 0000000000..d6fdecd4cf --- /dev/null +++ b/tests/version/BUILD.bazel @@ -0,0 +1,3 @@ +load(":version_test.bzl", "version_test_suite") + +version_test_suite(name = "version_tests") diff --git a/tests/version/version_test.bzl b/tests/version/version_test.bzl new file mode 100644 index 0000000000..589f9ac05d --- /dev/null +++ b/tests/version/version_test.bzl @@ -0,0 +1,157 @@ +"" + +load("@rules_testing//lib:analysis_test.bzl", "test_suite") +load("//python/private:version.bzl", "version") # buildifier: disable=bzl-visibility + +_tests = [] + +def _test_normalization(env): + prefixes = ["v", " v", " \t\r\nv"] + epochs = { + "": ["", "0!", "00!"], + "1!": ["1!", "001!"], + "200!": ["200!", "00200!"], + } + releases = { + "0.1": ["0.1", "0.01"], + "2023.7.19": ["2023.7.19", "2023.07.19"], + } + pres = { + "": [""], + "a0": ["a", ".a", "-ALPHA0", "_alpha0", ".a0"], + "a4": ["alpha4", ".a04"], + "b0": ["b", ".b", "-BETA0", "_beta0", ".b0"], + "b5": ["beta05", ".b5"], + "rc0": ["C", "_c0", "RC", "_rc0", "-preview_0"], + } + explicit_posts = { + "": [""], + ".post0": [], + ".post1": [".post1", "-r1", "_rev1"], + } + implicit_posts = [[".post1", "-1"], [".post2", "-2"]] + devs = { + "": [""], + ".dev0": ["dev", "-DEV", "_Dev-0"], + ".dev9": ["DEV9", ".dev09", ".dev9"], + ".dev{BUILD_TIMESTAMP}": [ + "-DEV{BUILD_TIMESTAMP}", + "_dev_{BUILD_TIMESTAMP}", + ], + } + locals = { + "": [""], + "+ubuntu.7": ["+Ubuntu_7", "+ubuntu-007"], + "+ubuntu.r007": ["+Ubuntu_R007"], + } + epochs = [ + [normalized_epoch, input_epoch] + for normalized_epoch, input_epochs in epochs.items() + for input_epoch in input_epochs + ] + releases = [ + [normalized_release, input_release] + for normalized_release, input_releases in releases.items() + for input_release in input_releases + ] + pres = [ + [normalized_pre, input_pre] + for normalized_pre, input_pres in pres.items() + for input_pre in input_pres + ] + explicit_posts = [ + [normalized_post, input_post] + for normalized_post, input_posts in explicit_posts.items() + for input_post in input_posts + ] + pres_and_posts = [ + [normalized_pre + normalized_post, input_pre + input_post] + for normalized_pre, input_pre in pres + for normalized_post, input_post in explicit_posts + ] + [ + [normalized_pre + normalized_post, input_pre + input_post] + for normalized_pre, input_pre in pres + for normalized_post, input_post in implicit_posts + if input_pre == "" or input_pre[-1].isdigit() + ] + devs = [ + [normalized_dev, input_dev] + for normalized_dev, input_devs in devs.items() + for input_dev in input_devs + ] + locals = [ + [normalized_local, input_local] + for normalized_local, input_locals in locals.items() + for input_local in input_locals + ] + postfixes = ["", " ", " \t\r\n"] + i = 0 + for nepoch, iepoch in epochs: + for nrelease, irelease in releases: + for nprepost, iprepost in pres_and_posts: + for ndev, idev in devs: + for nlocal, ilocal in locals: + prefix = prefixes[i % len(prefixes)] + postfix = postfixes[(i // len(prefixes)) % len(postfixes)] + env.expect.that_str( + version.normalize( + prefix + iepoch + irelease + iprepost + + idev + ilocal + postfix, + ), + ).equals( + nepoch + nrelease + nprepost + ndev + nlocal, + ) + i += 1 + +_tests.append(_test_normalization) + +def _test_ordering(env): + want = [ + # Taken from https://peps.python.org/pep-0440/#summary-of-permitted-suffixes-and-relative-ordering + "1.dev0", + "1.0.dev456", + "1.0a1", + "1.0a2.dev456", + "1.0a12.dev456", + "1.0a12", + "1.0b1.dev456", + "1.0b1.dev457", + "1.0b2", + "1.0b2.post345.dev456", + "1.0b2.post345.dev457", + "1.0b2.post345", + "1.0rc1.dev456", + "1.0rc1", + "1.0", + "1.0+abc.5", + "1.0+abc.7", + "1.0+5", + "1.0.post456.dev34", + "1.0.post456", + "1.0.15", + "1.1.dev1", + "1!0.1", + ] + + for lower, higher in zip(want[:-1], want[1:]): + lower = version.parse(lower, strict = True) + higher = version.parse(higher, strict = True) + + lower_key = version.key(lower) + higher_key = version.key(higher) + + if not lower_key < higher_key: + env.fail("Expected '{}'.key() to be smaller than '{}'.key(), but got otherwise: {} > {}".format( + lower.string, + higher.string, + lower_key, + higher_key, + )) + +_tests.append(_test_ordering) + +def version_test_suite(name): + test_suite( + name = name, + basic_tests = _tests, + ) From e54060b68c5d4fa7a34c6132efbab6761735c25e Mon Sep 17 00:00:00 2001 From: Fabian Meumertzheim Date: Mon, 12 May 2025 17:46:53 +0200 Subject: [PATCH 215/922] tests: make some analysis tests work for when test's exec platform is required (#2869) An upcoming change in Bazel makes the test toolchain required, which means a compatible exec platform amongst toolchains must be found (https://github.com/bazelbuild/bazel/commit/2780393d35ad0607cf5e344ae082b00a5569a964). Some analysis tests of `py_test` force the target platform to a specific platform, but before this change didn't register a compatible exec platform. This can be fixed by registering the target platform as an exec platform. Since Python targets currently depend on a C++ toolchain through Bazel's `launcher` and `launcher_maker` and the default toolchain can't cross-compile to Linux, the host platform still needs to be kept at highest priority to ensure that cross-compilation isn't needed on macOS. Work towards #2850 --- tests/base_rules/py_executable_base_tests.bzl | 2 ++ 1 file changed, 2 insertions(+) diff --git a/tests/base_rules/py_executable_base_tests.bzl b/tests/base_rules/py_executable_base_tests.bzl index 55a8958b82..49cbb1586c 100644 --- a/tests/base_rules/py_executable_base_tests.bzl +++ b/tests/base_rules/py_executable_base_tests.bzl @@ -356,6 +356,7 @@ def _test_main_module_bootstrap_system_python(name, config): target = name + "_subject", config_settings = { BOOTSTRAP_IMPL: "system_python", + "//command_line_option:extra_execution_platforms": ["@bazel_tools//tools:host_platform", LINUX_X86_64], "//command_line_option:platforms": [LINUX_X86_64], }, expect_failure = True, @@ -380,6 +381,7 @@ def _test_main_module_bootstrap_script(name, config): target = name + "_subject", config_settings = { BOOTSTRAP_IMPL: "script", + "//command_line_option:extra_execution_platforms": ["@bazel_tools//tools:host_platform", LINUX_X86_64], "//command_line_option:platforms": [LINUX_X86_64], }, ) From c383c3b2799b5255783545590482f59d78a42163 Mon Sep 17 00:00:00 2001 From: Ignas Anikevicius <240938+aignas@users.noreply.github.com> Date: Tue, 13 May 2025 08:22:38 +0900 Subject: [PATCH 216/922] fix(pypi): make the URL/filename extraction from requirement more robust (#2871) Summary: - Make the requirement line the same as the one that is used in whls. It only contains extras and the version if it is present. - Add debug log statements if we fail to get the version from a direct URL reference. - Move some tests from `parse_requirements_tests` to `index_sources_tests` to improve test maintenance. - Replace the URL encoded `+` to a regular `+` in the filename. - Correctly handle the case when the `=sha256:` is used in the URL. Once this is merged I plan to tackle #2648 by changing the `parse_requirements` code to de-duplicate entries returned by the `parse_requirements` function. I cannot think of anything else that we can do for this as of now, so will mark the associated issue as resolved. Fixes #2363 Work towards #2648 --- .editorconfig | 4 + CHANGELOG.md | 4 + python/private/pypi/extension.bzl | 4 +- python/private/pypi/index_sources.bzl | 42 ++- python/private/pypi/parse_requirements.bzl | 35 +-- python/private/pypi/pip_repository.bzl | 9 +- python/private/pypi/whl_library.bzl | 12 +- tests/pypi/extension/extension_tests.bzl | 6 +- .../index_sources/index_sources_tests.bzl | 39 ++- .../parse_requirements_tests.bzl | 278 ++---------------- 10 files changed, 132 insertions(+), 301 deletions(-) diff --git a/.editorconfig b/.editorconfig index 26bb52ffac..2737b0f184 100644 --- a/.editorconfig +++ b/.editorconfig @@ -15,3 +15,7 @@ max_line_length = 100 [*.{py,bzl}] indent_style = space indent_size = 4 + +# different overrides for git commit messages +[.git/COMMIT_EDITMSG] +max_line_length = 72 diff --git a/CHANGELOG.md b/CHANGELOG.md index aa7fc9d415..b94072d655 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -86,6 +86,10 @@ END_UNRELEASED_TEMPLATE multiple times. * (tools/wheelmaker.py) Extras are now preserved in Requires-Dist metadata when using requires_file to specify the requirements. +* (pypi) Use bazel downloader for direct URL references and correctly detect the filenames from + various URL formats - URL encoded version strings get correctly resolved, sha256 value can be + also retrieved from the URL as opposed to only the `--hash` parameter. Fixes + [#2363](https://github.com/bazel-contrib/rules_python/issues/2363). {#v0-0-0-added} ### Added diff --git a/python/private/pypi/extension.bzl b/python/private/pypi/extension.bzl index 647407f16f..9368e6539f 100644 --- a/python/private/pypi/extension.bzl +++ b/python/private/pypi/extension.bzl @@ -301,7 +301,7 @@ def _whl_repos(*, requirement, whl_library_args, download_only, netrc, auth_patt # This is no-op because pip is not used to download the wheel. args.pop("download_only", None) - args["requirement"] = requirement.srcs.requirement + args["requirement"] = requirement.line args["urls"] = [distribution.url] args["sha256"] = distribution.sha256 args["filename"] = distribution.filename @@ -338,7 +338,7 @@ def _whl_repos(*, requirement, whl_library_args, download_only, netrc, auth_patt # Fallback to a pip-installed wheel args = dict(whl_library_args) # make a copy - args["requirement"] = requirement.srcs.requirement_line + args["requirement"] = requirement.line if requirement.extra_pip_args: args["extra_pip_args"] = requirement.extra_pip_args diff --git a/python/private/pypi/index_sources.bzl b/python/private/pypi/index_sources.bzl index e3762d2a48..803670c3e4 100644 --- a/python/private/pypi/index_sources.bzl +++ b/python/private/pypi/index_sources.bzl @@ -16,6 +16,23 @@ A file that houses private functions used in the `bzlmod` extension with the same name. """ +# Just list them here and me super conservative +_KNOWN_EXTS = [ + # Note, the following source in pip has more extensions + # https://github.com/pypa/pip/blob/3c5a189141a965f21a473e46c3107e689eb9f79f/src/pip/_vendor/distlib/locators.py#L90 + # + # ".tar.bz2", + # ".tar", + # ".tgz", + # ".tbz", + # + # But we support only the following, used in 'packaging' + # https://github.com/pypa/pip/blob/3c5a189141a965f21a473e46c3107e689eb9f79f/src/pip/_vendor/packaging/utils.py#L137 + ".whl", + ".tar.gz", + ".zip", +] + def index_sources(line): """Get PyPI sources from a requirements.txt line. @@ -58,11 +75,31 @@ def index_sources(line): ).strip() url = "" + filename = "" if "@" in head: - requirement = requirement_line - _, _, url_and_rest = requirement.partition("@") + maybe_requirement, _, url_and_rest = requirement.partition("@") url = url_and_rest.strip().partition(" ")[0].strip() + url, _, sha256 = url.partition("#sha256=") + if sha256: + shas.append(sha256) + _, _, filename = url.rpartition("/") + + # Replace URL encoded characters and luckily there is only one case + filename = filename.replace("%2B", "+") + is_known_ext = False + for ext in _KNOWN_EXTS: + if filename.endswith(ext): + is_known_ext = True + break + + if is_known_ext: + requirement = maybe_requirement.strip() + else: + # could not detect filename from the URL + filename = "" + requirement = requirement_line + return struct( requirement = requirement, requirement_line = requirement_line, @@ -70,4 +107,5 @@ def index_sources(line): shas = sorted(shas), marker = marker, url = url, + filename = filename, ) diff --git a/python/private/pypi/parse_requirements.bzl b/python/private/pypi/parse_requirements.bzl index 1583c89199..bdfac46ed6 100644 --- a/python/private/pypi/parse_requirements.bzl +++ b/python/private/pypi/parse_requirements.bzl @@ -40,6 +40,7 @@ def parse_requirements( extra_pip_args = [], get_index_urls = None, evaluate_markers = None, + extract_url_srcs = True, logger = None): """Get the requirements with platforms that the requirements apply to. @@ -58,6 +59,8 @@ def parse_requirements( the platforms stored as values in the input dict. Returns the same dict, but with values being platforms that are compatible with the requirements line. + extract_url_srcs: A boolean to enable extracting URLs from requirement + lines to enable using bazel downloader. logger: repo_utils.logger or None, a simple struct to log diagnostic messages. Returns: @@ -206,7 +209,7 @@ def parse_requirements( ret_requirements.append( struct( distribution = r.distribution, - srcs = r.srcs, + line = r.srcs.requirement if extract_url_srcs and (whls or sdist) else r.srcs.requirement_line, target_platforms = sorted(target_platforms), extra_pip_args = r.extra_pip_args, whls = whls, @@ -281,32 +284,26 @@ def _add_dists(*, requirement, index_urls, logger = None): logger: A logger for printing diagnostic info. """ - # Handle direct URLs in requirements if requirement.srcs.url: - url = requirement.srcs.url - _, _, filename = url.rpartition("/") - filename, _, _ = filename.partition("#sha256=") - if "." not in filename: - # detected filename has no extension, it might be an sdist ref - # TODO @aignas 2025-04-03: should be handled if the following is fixed: - # https://github.com/bazel-contrib/rules_python/issues/2363 + if not requirement.srcs.filename: + if logger: + logger.debug(lambda: "Could not detect the filename from the URL, falling back to pip: {}".format( + requirement.srcs.url, + )) return [], None - if "@" in filename: - # this is most likely foo.git@git_sha, skip special handling of these - return [], None - - direct_url_dist = struct( - url = url, - filename = filename, + # Handle direct URLs in requirements + dist = struct( + url = requirement.srcs.url, + filename = requirement.srcs.filename, sha256 = requirement.srcs.shas[0] if requirement.srcs.shas else "", yanked = False, ) - if filename.endswith(".whl"): - return [direct_url_dist], None + if dist.filename.endswith(".whl"): + return [dist], None else: - return [], direct_url_dist + return [], dist if not index_urls: return [], None diff --git a/python/private/pypi/pip_repository.bzl b/python/private/pypi/pip_repository.bzl index 8ca94f7f9b..c8d23f471f 100644 --- a/python/private/pypi/pip_repository.bzl +++ b/python/private/pypi/pip_repository.bzl @@ -89,19 +89,20 @@ def _pip_repository_impl(rctx): python_interpreter_target = rctx.attr.python_interpreter_target, srcs = rctx.attr._evaluate_markers_srcs, ), + extract_url_srcs = False, ) selected_requirements = {} options = None repository_platform = host_platform(rctx) for name, requirements in requirements_by_platform.items(): - r = select_requirement( + requirement = select_requirement( requirements, platform = None if rctx.attr.download_only else repository_platform, ) - if not r: + if not requirement: continue - options = options or r.extra_pip_args - selected_requirements[name] = r.srcs.requirement_line + options = options or requirement.extra_pip_args + selected_requirements[name] = requirement.line bzl_packages = sorted(selected_requirements.keys()) diff --git a/python/private/pypi/whl_library.bzl b/python/private/pypi/whl_library.bzl index 160bb5b799..4427c0e3ef 100644 --- a/python/private/pypi/whl_library.bzl +++ b/python/private/pypi/whl_library.bzl @@ -258,19 +258,9 @@ def _whl_library_impl(rctx): # Simulate the behaviour where the whl is present in the current directory. rctx.symlink(whl_path, whl_path.basename) whl_path = rctx.path(whl_path.basename) - elif rctx.attr.urls: + elif rctx.attr.urls and rctx.attr.filename: filename = rctx.attr.filename urls = rctx.attr.urls - if not filename: - _, _, filename = urls[0].rpartition("/") - - if not (filename.endswith(".whl") or filename.endswith("tar.gz") or filename.endswith(".zip")): - if rctx.attr.filename: - msg = "got '{}'".format(filename) - else: - msg = "detected '{}' from url:\n{}".format(filename, urls[0]) - fail("Only '.whl', '.tar.gz' or '.zip' files are supported, {}".format(msg)) - result = rctx.download( url = urls, output = filename, diff --git a/tests/pypi/extension/extension_tests.bzl b/tests/pypi/extension/extension_tests.bzl index 1cd6869c84..b4a7746271 100644 --- a/tests/pypi/extension/extension_tests.bzl +++ b/tests/pypi/extension/extension_tests.bzl @@ -806,7 +806,7 @@ git_dep @ git+https://git.server/repo/project@deadbeefdeadbeef "extra_pip_args": ["--extra-args-for-sdist-building"], "filename": "any-name.tar.gz", "python_interpreter_target": "unit_test_interpreter_target", - "requirement": "direct_sdist_without_sha @ some-archive/any-name.tar.gz", + "requirement": "direct_sdist_without_sha", "sha256": "", "urls": ["some-archive/any-name.tar.gz"], }, @@ -824,7 +824,7 @@ git_dep @ git+https://git.server/repo/project@deadbeefdeadbeef ], "filename": "direct_without_sha-0.0.1-py3-none-any.whl", "python_interpreter_target": "unit_test_interpreter_target", - "requirement": "direct_without_sha==0.0.1 @ example-direct.org/direct_without_sha-0.0.1-py3-none-any.whl", + "requirement": "direct_without_sha==0.0.1", "sha256": "", "urls": ["example-direct.org/direct_without_sha-0.0.1-py3-none-any.whl"], }, @@ -891,7 +891,7 @@ git_dep @ git+https://git.server/repo/project@deadbeefdeadbeef ], "filename": "some_pkg-0.0.1-py3-none-any.whl", "python_interpreter_target": "unit_test_interpreter_target", - "requirement": "some_pkg==0.0.1 @ example-direct.org/some_pkg-0.0.1-py3-none-any.whl --hash=sha256:deadbaaf", + "requirement": "some_pkg==0.0.1", "sha256": "deadbaaf", "urls": ["example-direct.org/some_pkg-0.0.1-py3-none-any.whl"], }, diff --git a/tests/pypi/index_sources/index_sources_tests.bzl b/tests/pypi/index_sources/index_sources_tests.bzl index 9d12bc6399..d4062b47fe 100644 --- a/tests/pypi/index_sources/index_sources_tests.bzl +++ b/tests/pypi/index_sources/index_sources_tests.bzl @@ -23,42 +23,72 @@ def _test_no_simple_api_sources(env): inputs = { "foo @ git+https://github.com/org/foo.git@deadbeef": struct( requirement = "foo @ git+https://github.com/org/foo.git@deadbeef", + requirement_line = "foo @ git+https://github.com/org/foo.git@deadbeef", marker = "", url = "git+https://github.com/org/foo.git@deadbeef", shas = [], version = "", + filename = "", ), "foo==0.0.1": struct( requirement = "foo==0.0.1", + requirement_line = "foo==0.0.1", marker = "", url = "", version = "0.0.1", + filename = "", ), "foo==0.0.1 @ https://someurl.org": struct( requirement = "foo==0.0.1 @ https://someurl.org", + requirement_line = "foo==0.0.1 @ https://someurl.org", marker = "", url = "https://someurl.org", version = "0.0.1", + filename = "", ), "foo==0.0.1 @ https://someurl.org/package.whl": struct( - requirement = "foo==0.0.1 @ https://someurl.org/package.whl", + requirement = "foo==0.0.1", + requirement_line = "foo==0.0.1 @ https://someurl.org/package.whl", marker = "", url = "https://someurl.org/package.whl", version = "0.0.1", + filename = "package.whl", ), "foo==0.0.1 @ https://someurl.org/package.whl --hash=sha256:deadbeef": struct( - requirement = "foo==0.0.1 @ https://someurl.org/package.whl --hash=sha256:deadbeef", + requirement = "foo==0.0.1", + requirement_line = "foo==0.0.1 @ https://someurl.org/package.whl --hash=sha256:deadbeef", marker = "", url = "https://someurl.org/package.whl", shas = ["deadbeef"], version = "0.0.1", + filename = "package.whl", ), "foo==0.0.1 @ https://someurl.org/package.whl; python_version < \"2.7\"\\ --hash=sha256:deadbeef": struct( - requirement = "foo==0.0.1 @ https://someurl.org/package.whl --hash=sha256:deadbeef", + requirement = "foo==0.0.1", + requirement_line = "foo==0.0.1 @ https://someurl.org/package.whl --hash=sha256:deadbeef", marker = "python_version < \"2.7\"", url = "https://someurl.org/package.whl", shas = ["deadbeef"], version = "0.0.1", + filename = "package.whl", + ), + "foo[extra] @ https://example.org/foo-1.0.tar.gz --hash=sha256:deadbe0f": struct( + requirement = "foo[extra]", + requirement_line = "foo[extra] @ https://example.org/foo-1.0.tar.gz --hash=sha256:deadbe0f", + marker = "", + url = "https://example.org/foo-1.0.tar.gz", + shas = ["deadbe0f"], + version = "", + filename = "foo-1.0.tar.gz", + ), + "torch @ https://download.pytorch.org/whl/cpu/torch-2.6.0%2Bcpu-cp311-cp311-linux_x86_64.whl#sha256=deadbeef": struct( + requirement = "torch", + requirement_line = "torch @ https://download.pytorch.org/whl/cpu/torch-2.6.0%2Bcpu-cp311-cp311-linux_x86_64.whl#sha256=deadbeef", + marker = "", + url = "https://download.pytorch.org/whl/cpu/torch-2.6.0%2Bcpu-cp311-cp311-linux_x86_64.whl", + shas = ["deadbeef"], + version = "", + filename = "torch-2.6.0+cpu-cp311-cp311-linux_x86_64.whl", ), } for input, want in inputs.items(): @@ -66,9 +96,10 @@ def _test_no_simple_api_sources(env): env.expect.that_collection(got.shas).contains_exactly(want.shas if hasattr(want, "shas") else []) env.expect.that_str(got.version).equals(want.version) env.expect.that_str(got.requirement).equals(want.requirement) - env.expect.that_str(got.requirement_line).equals(got.requirement) + env.expect.that_str(got.requirement_line).equals(got.requirement_line) env.expect.that_str(got.marker).equals(want.marker) env.expect.that_str(got.url).equals(want.url) + env.expect.that_str(got.filename).equals(want.filename) _tests.append(_test_no_simple_api_sources) diff --git a/tests/pypi/parse_requirements/parse_requirements_tests.bzl b/tests/pypi/parse_requirements/parse_requirements_tests.bzl index c5b24870ea..497e08361f 100644 --- a/tests/pypi/parse_requirements/parse_requirements_tests.bzl +++ b/tests/pypi/parse_requirements/parse_requirements_tests.bzl @@ -27,10 +27,6 @@ foo==0.0.1 \ """, "requirements_direct": """\ foo[extra] @ https://some-url/package.whl -bar @ https://example.org/bar-1.0.whl --hash=sha256:deadbeef -baz @ https://test.com/baz-2.0.whl; python_version < "3.8" --hash=sha256:deadb00f -qux @ https://example.org/qux-1.0.tar.gz --hash=sha256:deadbe0f -torch @ https://download.pytorch.org/whl/cpu/torch-2.6.0%2Bcpu-cp311-cp311-linux_x86_64.whl#sha256=5b6ae523bfb67088a17ca7734d131548a2e60346c622621e4248ed09dd0790cc """, "requirements_extra_args": """\ --index-url=example.org @@ -111,14 +107,7 @@ def _test_simple(env): extra_pip_args = [], sdist = None, is_exposed = True, - srcs = struct( - marker = "", - requirement = "foo[extra]==0.0.1", - requirement_line = "foo[extra]==0.0.1 --hash=sha256:deadbeef", - shas = ["deadbeef"], - version = "0.0.1", - url = "", - ), + line = "foo[extra]==0.0.1 --hash=sha256:deadbeef", target_platforms = [ "linux_x86_64", "windows_x86_64", @@ -127,16 +116,11 @@ def _test_simple(env): ), ], }) - env.expect.that_str( - select_requirement( - got["foo"], - platform = "linux_x86_64", - ).srcs.version, - ).equals("0.0.1") _tests.append(_test_simple) -def _test_direct_urls(env): +def _test_direct_urls_integration(env): + """Check that we are using the filename from index_sources.""" got = parse_requirements( ctx = _mock_ctx(), requirements_by_platform = { @@ -144,66 +128,13 @@ def _test_direct_urls(env): }, ) env.expect.that_dict(got).contains_exactly({ - "bar": [ - struct( - distribution = "bar", - extra_pip_args = [], - sdist = None, - is_exposed = True, - srcs = struct( - marker = "", - requirement = "bar @ https://example.org/bar-1.0.whl --hash=sha256:deadbeef", - requirement_line = "bar @ https://example.org/bar-1.0.whl --hash=sha256:deadbeef", - shas = ["deadbeef"], - version = "", - url = "https://example.org/bar-1.0.whl", - ), - target_platforms = ["linux_x86_64"], - whls = [struct( - url = "https://example.org/bar-1.0.whl", - filename = "bar-1.0.whl", - sha256 = "deadbeef", - yanked = False, - )], - ), - ], - "baz": [ - struct( - distribution = "baz", - extra_pip_args = [], - sdist = None, - is_exposed = True, - srcs = struct( - marker = "python_version < \"3.8\"", - requirement = "baz @ https://test.com/baz-2.0.whl --hash=sha256:deadb00f", - requirement_line = "baz @ https://test.com/baz-2.0.whl --hash=sha256:deadb00f", - shas = ["deadb00f"], - version = "", - url = "https://test.com/baz-2.0.whl", - ), - target_platforms = ["linux_x86_64"], - whls = [struct( - url = "https://test.com/baz-2.0.whl", - filename = "baz-2.0.whl", - sha256 = "deadb00f", - yanked = False, - )], - ), - ], "foo": [ struct( distribution = "foo", extra_pip_args = [], sdist = None, is_exposed = True, - srcs = struct( - marker = "", - requirement = "foo[extra] @ https://some-url/package.whl", - requirement_line = "foo[extra] @ https://some-url/package.whl", - shas = [], - version = "", - url = "https://some-url/package.whl", - ), + line = "foo[extra]", target_platforms = ["linux_x86_64"], whls = [struct( url = "https://some-url/package.whl", @@ -213,57 +144,9 @@ def _test_direct_urls(env): )], ), ], - "qux": [ - struct( - distribution = "qux", - extra_pip_args = [], - sdist = struct( - url = "https://example.org/qux-1.0.tar.gz", - filename = "qux-1.0.tar.gz", - sha256 = "deadbe0f", - yanked = False, - ), - is_exposed = True, - srcs = struct( - marker = "", - requirement = "qux @ https://example.org/qux-1.0.tar.gz --hash=sha256:deadbe0f", - requirement_line = "qux @ https://example.org/qux-1.0.tar.gz --hash=sha256:deadbe0f", - shas = ["deadbe0f"], - version = "", - url = "https://example.org/qux-1.0.tar.gz", - ), - target_platforms = ["linux_x86_64"], - whls = [], - ), - ], - "torch": [ - struct( - distribution = "torch", - extra_pip_args = [], - is_exposed = True, - sdist = None, - srcs = struct( - marker = "", - requirement = "torch @ https://download.pytorch.org/whl/cpu/torch-2.6.0%2Bcpu-cp311-cp311-linux_x86_64.whl#sha256=5b6ae523bfb67088a17ca7734d131548a2e60346c622621e4248ed09dd0790cc", - requirement_line = "torch @ https://download.pytorch.org/whl/cpu/torch-2.6.0%2Bcpu-cp311-cp311-linux_x86_64.whl#sha256=5b6ae523bfb67088a17ca7734d131548a2e60346c622621e4248ed09dd0790cc", - shas = [], - url = "https://download.pytorch.org/whl/cpu/torch-2.6.0%2Bcpu-cp311-cp311-linux_x86_64.whl#sha256=5b6ae523bfb67088a17ca7734d131548a2e60346c622621e4248ed09dd0790cc", - version = "", - ), - target_platforms = ["linux_x86_64"], - whls = [ - struct( - filename = "torch-2.6.0%2Bcpu-cp311-cp311-linux_x86_64.whl", - sha256 = "", - url = "https://download.pytorch.org/whl/cpu/torch-2.6.0%2Bcpu-cp311-cp311-linux_x86_64.whl#sha256=5b6ae523bfb67088a17ca7734d131548a2e60346c622621e4248ed09dd0790cc", - yanked = False, - ), - ], - ), - ], }) -_tests.append(_test_direct_urls) +_tests.append(_test_direct_urls_integration) def _test_extra_pip_args(env): got = parse_requirements( @@ -280,14 +163,7 @@ def _test_extra_pip_args(env): extra_pip_args = ["--index-url=example.org", "--trusted-host=example.org"], sdist = None, is_exposed = True, - srcs = struct( - marker = "", - requirement = "foo[extra]==0.0.1", - requirement_line = "foo[extra]==0.0.1 --hash=sha256:deadbeef", - shas = ["deadbeef"], - version = "0.0.1", - url = "", - ), + line = "foo[extra]==0.0.1 --hash=sha256:deadbeef", target_platforms = [ "linux_x86_64", ], @@ -295,12 +171,6 @@ def _test_extra_pip_args(env): ), ], }) - env.expect.that_str( - select_requirement( - got["foo"], - platform = "linux_x86_64", - ).srcs.version, - ).equals("0.0.1") _tests.append(_test_extra_pip_args) @@ -318,14 +188,7 @@ def _test_dupe_requirements(env): extra_pip_args = [], sdist = None, is_exposed = True, - srcs = struct( - marker = "", - requirement = "foo[extra,extra_2]==0.0.1", - requirement_line = "foo[extra,extra_2]==0.0.1 --hash=sha256:deadbeef", - shas = ["deadbeef"], - version = "0.0.1", - url = "", - ), + line = "foo[extra,extra_2]==0.0.1 --hash=sha256:deadbeef", target_platforms = ["linux_x86_64"], whls = [], ), @@ -348,14 +211,7 @@ def _test_multi_os(env): struct( distribution = "bar", extra_pip_args = [], - srcs = struct( - marker = "", - requirement = "bar==0.0.1", - requirement_line = "bar==0.0.1 --hash=sha256:deadb00f", - shas = ["deadb00f"], - version = "0.0.1", - url = "", - ), + line = "bar==0.0.1 --hash=sha256:deadb00f", target_platforms = ["windows_x86_64"], whls = [], sdist = None, @@ -366,14 +222,7 @@ def _test_multi_os(env): struct( distribution = "foo", extra_pip_args = [], - srcs = struct( - marker = "", - requirement = "foo==0.0.3", - requirement_line = "foo==0.0.3 --hash=sha256:deadbaaf", - shas = ["deadbaaf"], - version = "0.0.3", - url = "", - ), + line = "foo==0.0.3 --hash=sha256:deadbaaf", target_platforms = ["linux_x86_64"], whls = [], sdist = None, @@ -382,14 +231,7 @@ def _test_multi_os(env): struct( distribution = "foo", extra_pip_args = [], - srcs = struct( - marker = "", - requirement = "foo[extra]==0.0.2", - requirement_line = "foo[extra]==0.0.2 --hash=sha256:deadbeef", - shas = ["deadbeef"], - version = "0.0.2", - url = "", - ), + line = "foo[extra]==0.0.2 --hash=sha256:deadbeef", target_platforms = ["windows_x86_64"], whls = [], sdist = None, @@ -401,8 +243,8 @@ def _test_multi_os(env): select_requirement( got["foo"], platform = "windows_x86_64", - ).srcs.version, - ).equals("0.0.2") + ).line, + ).equals("foo[extra]==0.0.2 --hash=sha256:deadbeef") _tests.append(_test_multi_os) @@ -422,14 +264,7 @@ def _test_multi_os_legacy(env): extra_pip_args = ["--platform=manylinux_2_17_x86_64", "--python-version=39", "--implementation=cp", "--abi=cp39"], is_exposed = False, sdist = None, - srcs = struct( - marker = "", - requirement = "bar==0.0.1", - requirement_line = "bar==0.0.1 --hash=sha256:deadb00f", - shas = ["deadb00f"], - version = "0.0.1", - url = "", - ), + line = "bar==0.0.1 --hash=sha256:deadb00f", target_platforms = ["cp39_linux_x86_64"], whls = [], ), @@ -440,14 +275,7 @@ def _test_multi_os_legacy(env): extra_pip_args = ["--platform=manylinux_2_17_x86_64", "--python-version=39", "--implementation=cp", "--abi=cp39"], is_exposed = True, sdist = None, - srcs = struct( - marker = "", - requirement = "foo==0.0.1", - requirement_line = "foo==0.0.1 --hash=sha256:deadbeef", - shas = ["deadbeef"], - version = "0.0.1", - url = "", - ), + line = "foo==0.0.1 --hash=sha256:deadbeef", target_platforms = ["cp39_linux_x86_64"], whls = [], ), @@ -456,14 +284,7 @@ def _test_multi_os_legacy(env): extra_pip_args = ["--platform=macosx_10_9_arm64", "--python-version=39", "--implementation=cp", "--abi=cp39"], is_exposed = True, sdist = None, - srcs = struct( - marker = "", - requirement_line = "foo==0.0.3 --hash=sha256:deadbaaf", - requirement = "foo==0.0.3", - shas = ["deadbaaf"], - version = "0.0.3", - url = "", - ), + line = "foo==0.0.3 --hash=sha256:deadbaaf", target_platforms = ["cp39_osx_aarch64"], whls = [], ), @@ -510,14 +331,7 @@ def _test_env_marker_resolution(env): extra_pip_args = [], is_exposed = True, sdist = None, - srcs = struct( - marker = "", - requirement = "bar==0.0.1", - requirement_line = "bar==0.0.1 --hash=sha256:deadbeef", - shas = ["deadbeef"], - version = "0.0.1", - url = "", - ), + line = "bar==0.0.1 --hash=sha256:deadbeef", target_platforms = ["cp311_linux_super_exotic", "cp311_windows_x86_64"], whls = [], ), @@ -528,25 +342,12 @@ def _test_env_marker_resolution(env): extra_pip_args = [], is_exposed = False, sdist = None, - srcs = struct( - marker = "marker", - requirement = "foo[extra]==0.0.1", - requirement_line = "foo[extra]==0.0.1 --hash=sha256:deadbeef", - shas = ["deadbeef"], - version = "0.0.1", - url = "", - ), + line = "foo[extra]==0.0.1 --hash=sha256:deadbeef", target_platforms = ["cp311_windows_x86_64"], whls = [], ), ], }) - env.expect.that_str( - select_requirement( - got["foo"], - platform = "windows_x86_64", - ).srcs.version, - ).equals("0.0.1") _tests.append(_test_env_marker_resolution) @@ -564,14 +365,7 @@ def _test_different_package_version(env): extra_pip_args = [], is_exposed = True, sdist = None, - srcs = struct( - marker = "", - requirement = "foo==0.0.1", - requirement_line = "foo==0.0.1 --hash=sha256:deadb00f", - shas = ["deadb00f"], - version = "0.0.1", - url = "", - ), + line = "foo==0.0.1 --hash=sha256:deadb00f", target_platforms = ["linux_x86_64"], whls = [], ), @@ -580,14 +374,7 @@ def _test_different_package_version(env): extra_pip_args = [], is_exposed = True, sdist = None, - srcs = struct( - marker = "", - requirement = "foo==0.0.1+local", - requirement_line = "foo==0.0.1+local --hash=sha256:deadbeef", - shas = ["deadbeef"], - version = "0.0.1+local", - url = "", - ), + line = "foo==0.0.1+local --hash=sha256:deadbeef", target_platforms = ["linux_x86_64"], whls = [], ), @@ -610,14 +397,7 @@ def _test_optional_hash(env): extra_pip_args = [], sdist = None, is_exposed = True, - srcs = struct( - marker = "", - requirement = "foo==0.0.4 @ https://example.org/foo-0.0.4.whl", - requirement_line = "foo==0.0.4 @ https://example.org/foo-0.0.4.whl", - shas = [], - version = "0.0.4", - url = "https://example.org/foo-0.0.4.whl", - ), + line = "foo==0.0.4", target_platforms = ["linux_x86_64"], whls = [struct( url = "https://example.org/foo-0.0.4.whl", @@ -631,14 +411,7 @@ def _test_optional_hash(env): extra_pip_args = [], sdist = None, is_exposed = True, - srcs = struct( - marker = "", - requirement = "foo==0.0.5 @ https://example.org/foo-0.0.5.whl --hash=sha256:deadbeef", - requirement_line = "foo==0.0.5 @ https://example.org/foo-0.0.5.whl --hash=sha256:deadbeef", - shas = ["deadbeef"], - version = "0.0.5", - url = "https://example.org/foo-0.0.5.whl", - ), + line = "foo==0.0.5", target_platforms = ["linux_x86_64"], whls = [struct( url = "https://example.org/foo-0.0.5.whl", @@ -666,14 +439,7 @@ def _test_git_sources(env): extra_pip_args = [], is_exposed = True, sdist = None, - srcs = struct( - marker = "", - requirement = "foo @ git+https://github.com/org/foo.git@deadbeef", - requirement_line = "foo @ git+https://github.com/org/foo.git@deadbeef", - shas = [], - url = "git+https://github.com/org/foo.git@deadbeef", - version = "", - ), + line = "foo @ git+https://github.com/org/foo.git@deadbeef", target_platforms = ["linux_x86_64"], whls = [], ), From a13fcd77cd27bd131e1b4ce227372312614613f2 Mon Sep 17 00:00:00 2001 From: Ignas Anikevicius <240938+aignas@users.noreply.github.com> Date: Wed, 14 May 2025 07:16:11 +0900 Subject: [PATCH 217/922] feat(pypi): actually start using env_marker_setting (#2873) Summary: - `pep508_deps` is now much simpler, because the hard work is done in analysis phase - `whl_library` BUILD.bazel tests now also have a test for the legacy flow. One thing that I noticed is that now we have an implicit dependency on the python toolchain when getting all of the `whl` target tree. This is a filegroup target that includes dependent wheels. However, we fallback to the flag values if we don't have the toolchain, so we should be good in general. Overall I like how this is turning out because we don't need to pipe the `target_platforms` anymore when we enable `PIPSTAR` feature. This means that we can start creating fewer whl_library instances - e.g. a `py3-none-any` wheel can be fetched once instead of once per python interpreter version. I'll leave this optimization for a later time. Work towards #260 --------- Co-authored-by: Richard Levasseur --- python/private/pypi/BUILD.bazel | 4 - python/private/pypi/config.bzl.tmpl.bzlmod | 2 +- python/private/pypi/extension.bzl | 35 ++- .../pypi/generate_whl_library_build_bazel.bzl | 27 +- python/private/pypi/hub_repository.bzl | 2 +- python/private/pypi/pep508_deps.bzl | 131 +++------- python/private/pypi/pep508_evaluate.bzl | 4 +- .../pypi/whl_installer/wheel_installer.py | 1 - python/private/pypi/whl_library.bzl | 4 - python/private/pypi/whl_library_targets.bzl | 77 +++--- tests/pypi/extension/extension_tests.bzl | 7 +- ...generate_whl_library_build_bazel_tests.bzl | 70 ++++- tests/pypi/pep508/deps_tests.bzl | 241 +++--------------- .../whl_installer/wheel_installer_test.py | 3 +- .../whl_library_targets_tests.bzl | 28 +- 15 files changed, 249 insertions(+), 387 deletions(-) diff --git a/python/private/pypi/BUILD.bazel b/python/private/pypi/BUILD.bazel index f541cbe98b..06ca3a8e34 100644 --- a/python/private/pypi/BUILD.bazel +++ b/python/private/pypi/BUILD.bazel @@ -236,13 +236,9 @@ bzl_library( name = "pep508_deps_bzl", srcs = ["pep508_deps.bzl"], deps = [ - ":pep508_env_bzl", ":pep508_evaluate_bzl", - ":pep508_platform_bzl", ":pep508_requirement_bzl", - "//python/private:full_version_bzl", "//python/private:normalize_name_bzl", - "@pythons_hub//:versions_bzl", ], ) diff --git a/python/private/pypi/config.bzl.tmpl.bzlmod b/python/private/pypi/config.bzl.tmpl.bzlmod index deb53631d1..c3ada70d27 100644 --- a/python/private/pypi/config.bzl.tmpl.bzlmod +++ b/python/private/pypi/config.bzl.tmpl.bzlmod @@ -6,4 +6,4 @@ with your usecase. This may change in between rules_python versions without any @generated by rules_python pip.parse bzlmod extension. """ -target_platforms = %%TARGET_PLATFORMS%% +whl_map = %%WHL_MAP%% diff --git a/python/private/pypi/extension.bzl b/python/private/pypi/extension.bzl index 9368e6539f..84caa0aee7 100644 --- a/python/private/pypi/extension.bzl +++ b/python/private/pypi/extension.bzl @@ -17,6 +17,7 @@ load("@bazel_features//:features.bzl", "bazel_features") load("@pythons_hub//:interpreters.bzl", "INTERPRETER_LABELS") load("@pythons_hub//:versions.bzl", "MINOR_MAPPING") +load("@rules_python_internal//:rules_python_config.bzl", rp_config = "config") load("//python/private:auth.bzl", "AUTH_ATTRS") load("//python/private:full_version.bzl", "full_version") load("//python/private:normalize_name.bzl", "normalize_name") @@ -72,7 +73,8 @@ def _create_whl_repos( available_interpreters = INTERPRETER_LABELS, minor_mapping = MINOR_MAPPING, evaluate_markers = evaluate_markers_py, - get_index_urls = None): + get_index_urls = None, + enable_pipstar = False): """create all of the whl repositories Args: @@ -87,6 +89,7 @@ def _create_whl_repos( minor_mapping: {type}`dict[str, str]` The dictionary needed to resolve the full python version used to parse package METADATA files. evaluate_markers: the function used to evaluate the markers. + enable_pipstar: enable the pipstar feature. Returns a {type}`struct` with the following attributes: whl_map: {type}`dict[str, list[struct]]` the output is keyed by the @@ -216,7 +219,6 @@ def _create_whl_repos( enable_implicit_namespace_pkgs = pip_attr.enable_implicit_namespace_pkgs, environment = pip_attr.environment, envsubst = pip_attr.envsubst, - experimental_target_platforms = pip_attr.experimental_target_platforms, group_deps = group_deps, group_name = group_name, pip_data_exclude = pip_attr.pip_data_exclude, @@ -227,6 +229,9 @@ def _create_whl_repos( for p, args in whl_overrides.get(whl_name, {}).items() }, ) + if not enable_pipstar: + maybe_args["experimental_target_platforms"] = pip_attr.experimental_target_platforms + whl_library_args.update({k: v for k, v in maybe_args.items() if v}) maybe_args_with_default = dict( # The following values have defaults next to them @@ -249,6 +254,7 @@ def _create_whl_repos( auth_patterns = pip_attr.auth_patterns, python_version = major_minor, multiple_requirements_for_whl = len(requirements) > 1., + enable_pipstar = enable_pipstar, ).items(): repo_name = "{}_{}".format(pip_name, repo_name) if repo_name in whl_libraries: @@ -277,7 +283,7 @@ def _create_whl_repos( }, ) -def _whl_repos(*, requirement, whl_library_args, download_only, netrc, auth_patterns, multiple_requirements_for_whl = False, python_version): +def _whl_repos(*, requirement, whl_library_args, download_only, netrc, auth_patterns, multiple_requirements_for_whl = False, python_version, enable_pipstar = False): ret = {} dists = requirement.whls @@ -305,13 +311,14 @@ def _whl_repos(*, requirement, whl_library_args, download_only, netrc, auth_patt args["urls"] = [distribution.url] args["sha256"] = distribution.sha256 args["filename"] = distribution.filename - args["experimental_target_platforms"] = [ - # Get rid of the version fot the target platforms because we are - # passing the interpreter any way. Ideally we should search of ways - # how to pass the target platforms through the hub repo. - p.partition("_")[2] - for p in requirement.target_platforms - ] + if not enable_pipstar: + args["experimental_target_platforms"] = [ + # Get rid of the version fot the target platforms because we are + # passing the interpreter any way. Ideally we should search of ways + # how to pass the target platforms through the hub repo. + p.partition("_")[2] + for p in requirement.target_platforms + ] # Pure python wheels or sdists may need to have a platform here target_platforms = None @@ -357,7 +364,11 @@ def _whl_repos(*, requirement, whl_library_args, download_only, netrc, auth_patt return ret -def parse_modules(module_ctx, _fail = fail, simpleapi_download = simpleapi_download, **kwargs): +def parse_modules( + module_ctx, + _fail = fail, + simpleapi_download = simpleapi_download, + **kwargs): """Implementation of parsing the tag classes for the extension and return a struct for registering repositories. Args: @@ -639,7 +650,7 @@ def _pip_impl(module_ctx): module_ctx: module contents """ - mods = parse_modules(module_ctx) + mods = parse_modules(module_ctx, enable_pipstar = rp_config.enable_pipstar) # Build all of the wheel modifications if the tag class is called. _whl_mods_impl(mods.whl_mods) diff --git a/python/private/pypi/generate_whl_library_build_bazel.bzl b/python/private/pypi/generate_whl_library_build_bazel.bzl index 31c9d4da60..3764e720c0 100644 --- a/python/private/pypi/generate_whl_library_build_bazel.bzl +++ b/python/private/pypi/generate_whl_library_build_bazel.bzl @@ -26,10 +26,11 @@ _RENDER = { "entry_points": render.dict, "extras": render.list, "group_deps": render.list, + "include": str, "requires_dist": render.list, "srcs_exclude": render.list, "tags": render.list, - "target_platforms": lambda x: render.list(x) if x else "target_platforms", + "target_platforms": render.list, } # NOTE @aignas 2024-10-25: We have to keep this so that files in @@ -62,28 +63,44 @@ def generate_whl_library_build_bazel( A complete BUILD file as a string """ - fn = "whl_library_targets" + loads = [] if kwargs.get("tags"): + fn = "whl_library_targets" + # legacy path unsupported_args = [ "requires", "metadata_name", "metadata_version", + "include", ] else: - fn = "{}_from_requires".format(fn) + fn = "whl_library_targets_from_requires" unsupported_args = [ "dependencies", "dependencies_by_platform", + "target_platforms", + "default_python_version", ] + dep_template = kwargs.get("dep_template") + loads.append( + """load("{}", "{}")""".format( + dep_template.format( + name = "", + target = "config.bzl", + ), + "whl_map", + ), + ) + kwargs["include"] = "whl_map" for arg in unsupported_args: if kwargs.get(arg): fail("BUG, unsupported arg: '{}'".format(arg)) - loads = [ + loads.extend([ """load("@rules_python//python/private/pypi:whl_library_targets.bzl", "{}")""".format(fn), - ] + ]) additional_content = [] if annotation: diff --git a/python/private/pypi/hub_repository.bzl b/python/private/pypi/hub_repository.bzl index d2cbf88c24..0a1e772d05 100644 --- a/python/private/pypi/hub_repository.bzl +++ b/python/private/pypi/hub_repository.bzl @@ -49,7 +49,7 @@ def _impl(rctx): "config.bzl", rctx.attr._config_template, substitutions = { - "%%TARGET_PLATFORMS%%": render.list(rctx.attr.target_platforms), + "%%WHL_MAP%%": render.dict(rctx.attr.whl_map, value_repr = lambda x: "None"), }, ) rctx.template("requirements.bzl", rctx.attr._requirements_bzl_template, substitutions = { diff --git a/python/private/pypi/pep508_deps.bzl b/python/private/pypi/pep508_deps.bzl index bcc4845cf1..e73f747bed 100644 --- a/python/private/pypi/pep508_deps.bzl +++ b/python/private/pypi/pep508_deps.bzl @@ -15,23 +15,17 @@ """This module is for implementing PEP508 compliant METADATA deps parsing. """ -load("@pythons_hub//:versions.bzl", "DEFAULT_PYTHON_VERSION", "MINOR_MAPPING") -load("//python/private:full_version.bzl", "full_version") load("//python/private:normalize_name.bzl", "normalize_name") -load(":pep508_env.bzl", "env") load(":pep508_evaluate.bzl", "evaluate") -load(":pep508_platform.bzl", "platform", "platform_from_str") load(":pep508_requirement.bzl", "requirement") def deps( name, *, requires_dist, - platforms = [], extras = [], excludes = [], - default_python_version = None, - minor_mapping = MINOR_MAPPING): + include = []): """Parse the RequiresDist from wheel METADATA Args: @@ -39,12 +33,9 @@ def deps( requires_dist: {type}`list[str]` the list of RequiresDist lines from the METADATA file. excludes: {type}`list[str]` what packages should we exclude. + include: {type}`list[str]` what packages should we exclude. If it is not + specified, then we will include all deps from `requires_dist`. extras: {type}`list[str]` the requested extras to generate targets for. - platforms: {type}`list[str]` the list of target platform strings. - default_python_version: {type}`str` the host python version. - minor_mapping: {type}`type[str, str]` the minor mapping to use when - resolving to the full python version as DEFAULT_PYTHON_VERSION can by - of format `3.x`. Returns: A struct with attributes: @@ -60,39 +51,20 @@ def deps( deps_select = {} name = normalize_name(name) want_extras = _resolve_extras(name, reqs, extras) + include = [normalize_name(n) for n in include] # drop self edges excludes = [name] + [normalize_name(x) for x in excludes] - default_python_version = default_python_version or DEFAULT_PYTHON_VERSION - if default_python_version: - # if it is not bzlmod, then DEFAULT_PYTHON_VERSION may be unset - default_python_version = full_version( - version = default_python_version, - minor_mapping = minor_mapping, - ) - platforms = [ - platform_from_str(p, python_version = default_python_version) - for p in platforms - ] - - abis = sorted({p.abi: True for p in platforms if p.abi}) - if default_python_version and len(abis) > 1: - _, _, tail = default_python_version.partition(".") - default_abi = "cp3" + tail - elif len(abis) > 1: - fail( - "all python versions need to be specified explicitly, got: {}".format(platforms), - ) - else: - default_abi = None - reqs_by_name = {} for req in reqs: if req.name_ in excludes: continue + if include and req.name_ not in include: + continue + reqs_by_name.setdefault(req.name, []).append(req) for name, reqs in reqs_by_name.items(): @@ -102,55 +74,25 @@ def deps( normalize_name(name), reqs, extras = want_extras, - platforms = platforms, - default_abi = default_abi, ) return struct( deps = sorted(deps), deps_select = { - _platform_str(p): sorted(deps) - for p, deps in deps_select.items() + d: markers + for d, markers in sorted(deps_select.items()) }, ) -def _platform_str(self): - if self.abi == None: - return "{}_{}".format(self.os, self.arch) - - return "{}_{}_{}".format( - self.abi, - self.os or "anyos", - self.arch or "anyarch", - ) - -def _add(deps, deps_select, dep, platform): +def _add(deps, deps_select, dep, markers = None): dep = normalize_name(dep) - if platform == None: + if not markers: deps[dep] = True - - # If the dep is in the platform-specific list, remove it from the select. - pop_keys = [] - for p, _deps in deps_select.items(): - if dep not in _deps: - continue - - _deps.pop(dep) - if not _deps: - pop_keys.append(p) - - for p in pop_keys: - deps_select.pop(p) - return - - if dep in deps: - # If the dep is already in the main dependency list, no need to add it in the - # platform-specific dependency list. - return - - # Add the platform-specific branch - deps_select.setdefault(platform, {})[dep] = True + elif len(markers) == 1: + deps_select[dep] = markers[0] + else: + deps_select[dep] = "({})".format(") or (".join(sorted(markers))) def _resolve_extras(self_name, reqs, extras): """Resolve extras which are due to depending on self[some_other_extra]. @@ -207,37 +149,24 @@ def _resolve_extras(self_name, reqs, extras): # Poor mans set return sorted({x: None for x in extras}) -def _add_reqs(deps, deps_select, dep, reqs, *, extras, platforms, default_abi = None): +def _add_reqs(deps, deps_select, dep, reqs, *, extras): for req in reqs: if not req.marker: - _add(deps, deps_select, dep, None) + _add(deps, deps_select, dep) return - platforms_to_add = {} - for plat in platforms: - if plat in platforms_to_add: - # marker evaluation is more expensive than this check - continue - - added = False - for extra in extras: - if added: + markers = {} + for req in reqs: + for x in extras: + m = evaluate(req.marker, env = {"extra": x}, strict = False) + if m == False: + continue + elif m == True: + _add(deps, deps_select, dep) break + else: + markers[m] = None + continue - for req in reqs: - if evaluate(req.marker, env = env(target_platform = plat, extra = extra)): - platforms_to_add[plat] = True - added = True - break - - if len(platforms_to_add) == len(platforms): - # the dep is in all target platforms, let's just add it to the regular - # list - _add(deps, deps_select, dep, None) - return - - for plat in platforms_to_add: - if default_abi: - _add(deps, deps_select, dep, plat) - if plat.abi == default_abi or not default_abi: - _add(deps, deps_select, dep, platform(os = plat.os, arch = plat.arch)) + if markers: + _add(deps, deps_select, dep, sorted(markers)) diff --git a/python/private/pypi/pep508_evaluate.bzl b/python/private/pypi/pep508_evaluate.bzl index 61a5b19999..d4492a75bb 100644 --- a/python/private/pypi/pep508_evaluate.bzl +++ b/python/private/pypi/pep508_evaluate.bzl @@ -122,7 +122,9 @@ def evaluate(marker, *, env, strict = True, **kwargs): **kwargs: Extra kwargs to be passed to the expression evaluator. Returns: - The {type}`bool` If the marker is compatible with the given env. + The {type}`bool | str` If the marker is compatible with the given env. If strict is + `False`, then the output type is `str` which will represent the remaining + expression that has not been evaluated. """ tokens = tokenize(marker) diff --git a/python/private/pypi/whl_installer/wheel_installer.py b/python/private/pypi/whl_installer/wheel_installer.py index 2db03e039d..600d45f940 100644 --- a/python/private/pypi/whl_installer/wheel_installer.py +++ b/python/private/pypi/whl_installer/wheel_installer.py @@ -126,7 +126,6 @@ def _extract_wheel( _setup_namespace_pkg_compatibility(installation_dir) metadata = { - "python_version": f"{sys.version_info[0]}.{sys.version_info[1]}.{sys.version_info[2]}", "entry_points": [ { "name": name, diff --git a/python/private/pypi/whl_library.bzl b/python/private/pypi/whl_library.bzl index 4427c0e3ef..b370de448a 100644 --- a/python/private/pypi/whl_library.bzl +++ b/python/private/pypi/whl_library.bzl @@ -22,7 +22,6 @@ load("//python/private:repo_utils.bzl", "REPO_DEBUG_ENV_VAR", "repo_utils") load(":attrs.bzl", "ATTRS", "use_isolated") load(":deps.bzl", "all_repo_names", "record_files") load(":generate_whl_library_build_bazel.bzl", "generate_whl_library_build_bazel") -load(":parse_requirements.bzl", "host_platform") load(":parse_whl_name.bzl", "parse_whl_name") load(":patch_whl.bzl", "patch_whl") load(":pypi_repo_utils.bzl", "pypi_repo_utils") @@ -352,7 +351,6 @@ def _whl_library_impl(rctx): metadata = json.decode(rctx.read("metadata.json")) rctx.delete("metadata.json") - python_version = metadata["python_version"] # NOTE @aignas 2024-06-22: this has to live on until we stop supporting # passing `twine` as a `:pkg` library via the `WORKSPACE` builds. @@ -390,9 +388,7 @@ def _whl_library_impl(rctx): entry_points = entry_points, metadata_name = metadata.name, metadata_version = metadata.version, - default_python_version = python_version, requires_dist = metadata.requires_dist, - target_platforms = rctx.attr.experimental_target_platforms or [host_platform(rctx)], # TODO @aignas 2025-04-14: load through the hub: annotation = None if not rctx.attr.annotation else struct(**json.decode(rctx.read(rctx.attr.annotation))), data_exclude = rctx.attr.pip_data_exclude, diff --git a/python/private/pypi/whl_library_targets.bzl b/python/private/pypi/whl_library_targets.bzl index 21e4a54a3a..e0c03a1505 100644 --- a/python/private/pypi/whl_library_targets.bzl +++ b/python/private/pypi/whl_library_targets.bzl @@ -19,6 +19,7 @@ load("//python:py_binary.bzl", "py_binary") load("//python:py_library.bzl", "py_library") load("//python/private:glob_excludes.bzl", "glob_excludes") load("//python/private:normalize_name.bzl", "normalize_name") +load(":env_marker_setting.bzl", "env_marker_setting") load( ":labels.bzl", "DATA_LABEL", @@ -29,9 +30,7 @@ load( "WHEEL_FILE_IMPL_LABEL", "WHEEL_FILE_PUBLIC_LABEL", ) -load(":parse_whl_name.bzl", "parse_whl_name") load(":pep508_deps.bzl", "deps") -load(":whl_target_platforms.bzl", "whl_target_platforms") def whl_library_targets_from_requires( *, @@ -40,8 +39,7 @@ def whl_library_targets_from_requires( metadata_version = "", requires_dist = [], extras = [], - target_platforms = [], - default_python_version = None, + include = [], group_deps = [], **kwargs): """The macro to create whl targets from the METADATA. @@ -57,25 +55,21 @@ def whl_library_targets_from_requires( requires_dist: {type}`list[str]` The list of `Requires-Dist` values from the whl `METADATA`. extras: {type}`list[str]` The list of requested extras. This essentially includes extra transitive dependencies in the final targets depending on the wheel `METADATA`. - target_platforms: {type}`list[str]` The list of target platforms to create - dependency closures for. - default_python_version: {type}`str` The python version to assume when parsing - the `METADATA`. This is only used when the `target_platforms` do not - include the version information. + include: {type}`list[str]` The list of packages to include. **kwargs: Extra args passed to the {obj}`whl_library_targets` """ package_deps = _parse_requires_dist( - name = name, - default_python_version = default_python_version, + name = metadata_name, requires_dist = requires_dist, excludes = group_deps, extras = extras, - target_platforms = target_platforms, + include = include, ) + whl_library_targets( name = name, dependencies = package_deps.deps, - dependencies_by_platform = package_deps.deps_select, + dependencies_with_markers = package_deps.deps_select, tags = [ "pypi_name={}".format(metadata_name), "pypi_version={}".format(metadata_version), @@ -86,31 +80,16 @@ def whl_library_targets_from_requires( def _parse_requires_dist( *, name, - default_python_version, requires_dist, excludes, - extras, - target_platforms): - parsed_whl = parse_whl_name(name) - - # NOTE @aignas 2023-12-04: if the wheel is a platform specific wheel, we - # only include deps for that target platform - if parsed_whl.platform_tag != "any": - target_platforms = [ - p.target_platform - for p in whl_target_platforms( - platform_tag = parsed_whl.platform_tag, - abi_tag = parsed_whl.abi_tag.strip("tm"), - ) - ] - + include, + extras): return deps( - name = normalize_name(parsed_whl.distribution), + name = normalize_name(name), requires_dist = requires_dist, - platforms = target_platforms, excludes = excludes, + include = include, extras = extras, - default_python_version = default_python_version, ) def whl_library_targets( @@ -126,6 +105,7 @@ def whl_library_targets( }, dependencies = [], dependencies_by_platform = {}, + dependencies_with_markers = {}, group_deps = [], group_name = "", data = [], @@ -137,6 +117,7 @@ def whl_library_targets( copy_file = copy_file, py_binary = py_binary, py_library = py_library, + env_marker_setting = env_marker_setting, )): """Create all of the whl_library targets. @@ -149,6 +130,8 @@ def whl_library_targets( dependencies: {type}`list[str]` A list of dependencies. dependencies_by_platform: {type}`dict[str, list[str]]` A list of dependencies by platform key. + dependencies_with_markers: {type}`dict[str, str]` A marker to evaluate + in order for the dep to be included. filegroups: {type}`dict[str, list[str]]` A dictionary of the target names and the glob matches. group_name: {type}`str` name of the dependency group (if any) which @@ -207,10 +190,16 @@ def whl_library_targets( data.append(dest) _config_settings( - dependencies_by_platform.keys(), + dependencies_by_platform = dependencies_by_platform.keys(), + dependencies_with_markers = dependencies_with_markers, native = native, + rules = rules, visibility = ["//visibility:private"], ) + deps_conditional = { + d: "is_include_{}_true".format(d) + for d in dependencies_with_markers + } # TODO @aignas 2024-10-25: remove the entry_point generation once # `py_console_script_binary` is the only way to use entry points. @@ -290,6 +279,7 @@ def whl_library_targets( data = _deps( deps = dependencies, deps_by_platform = dependencies_by_platform, + deps_conditional = deps_conditional, tmpl = dep_template.format(name = "{}", target = WHEEL_FILE_PUBLIC_LABEL), # NOTE @aignas 2024-10-28: Actually, `select` is not part of # `native`, but in order to support bazel 6.4 in unit tests, I @@ -342,6 +332,7 @@ def whl_library_targets( deps = _deps( deps = dependencies, deps_by_platform = dependencies_by_platform, + deps_conditional = deps_conditional, tmpl = dep_template.format(name = "{}", target = PY_LIBRARY_PUBLIC_LABEL), select = getattr(native, "select", select), ), @@ -350,7 +341,7 @@ def whl_library_targets( experimental_venvs_site_packages = Label("@rules_python//python/config_settings:venvs_site_packages"), ) -def _config_settings(dependencies_by_platform, native = native, **kwargs): +def _config_settings(dependencies_by_platform, dependencies_with_markers, rules, native = native, **kwargs): """Generate config settings for the targets. Args: @@ -362,9 +353,19 @@ def _config_settings(dependencies_by_platform, native = native, **kwargs): * `@//python/config_settings:is_python_3.{minor_version}` * `{os}_{cpu}` * `cp3{minor_version}_{os}_{cpu}` + dependencies_with_markers: {type}`dict[str, str]` The markers to evaluate by + each dep. + rules: used for testing native: {type}`native` The native struct for overriding in tests. **kwargs: Extra kwargs to pass to the rule. """ + for dep, expression in dependencies_with_markers.items(): + rules.env_marker_setting( + name = "include_{}".format(dep), + expression = expression, + **kwargs + ) + for p in dependencies_by_platform: if p.startswith("@") or p.endswith("default"): continue @@ -404,9 +405,15 @@ def _plat_label(plat): else: return ":is_" + plat.replace("cp3", "python_3.") -def _deps(deps, deps_by_platform, tmpl, select = select): +def _deps(deps, deps_by_platform, deps_conditional, tmpl, select = select): deps = [tmpl.format(d) for d in sorted(deps)] + for dep, setting in deps_conditional.items(): + deps = deps + select({ + ":{}".format(setting): [tmpl.format(dep)], + "//conditions:default": [], + }) + if not deps_by_platform: return deps diff --git a/tests/pypi/extension/extension_tests.bzl b/tests/pypi/extension/extension_tests.bzl index b4a7746271..8e325724f4 100644 --- a/tests/pypi/extension/extension_tests.bzl +++ b/tests/pypi/extension/extension_tests.bzl @@ -62,7 +62,12 @@ def _mod(*, name, parse = [], override = [], whl_mods = [], is_root = True): def _parse_modules(env, **kwargs): return env.expect.that_struct( - parse_modules(**kwargs), + parse_modules( + # TODO @aignas 2025-05-11: start integration testing the branch which + # includes this. + enable_pipstar = 0, + **kwargs + ), attrs = dict( exposed_packages = subjects.dict, hub_group_map = subjects.dict, diff --git a/tests/pypi/generate_whl_library_build_bazel/generate_whl_library_build_bazel_tests.bzl b/tests/pypi/generate_whl_library_build_bazel/generate_whl_library_build_bazel_tests.bzl index 83be7395d4..225b296ebf 100644 --- a/tests/pypi/generate_whl_library_build_bazel/generate_whl_library_build_bazel_tests.bzl +++ b/tests/pypi/generate_whl_library_build_bazel/generate_whl_library_build_bazel_tests.bzl @@ -19,8 +19,73 @@ load("//python/private/pypi:generate_whl_library_build_bazel.bzl", "generate_whl _tests = [] +def _test_all_legacy(env): + want = """\ +load("@rules_python//python/private/pypi:whl_library_targets.bzl", "whl_library_targets") + +package(default_visibility = ["//visibility:public"]) + +whl_library_targets( + copy_executables = { + "exec_src": "exec_dest", + }, + copy_files = { + "file_src": "file_dest", + }, + data = ["extra_target"], + data_exclude = [ + "exclude_via_attr", + "data_exclude_all", + ], + dep_template = "@pypi//{name}:{target}", + dependencies = ["foo"], + dependencies_by_platform = { + "baz": ["bar"], + }, + entry_points = { + "foo": "bar.py", + }, + group_deps = [ + "foo", + "fox", + "qux", + ], + group_name = "qux", + name = "foo.whl", + srcs_exclude = ["srcs_exclude_all"], + tags = ["tag1"], +) + +# SOMETHING SPECIAL AT THE END +""" + actual = generate_whl_library_build_bazel( + dep_template = "@pypi//{name}:{target}", + name = "foo.whl", + dependencies = ["foo"], + dependencies_by_platform = {"baz": ["bar"]}, + entry_points = { + "foo": "bar.py", + }, + data_exclude = ["exclude_via_attr"], + annotation = struct( + copy_files = {"file_src": "file_dest"}, + copy_executables = {"exec_src": "exec_dest"}, + data = ["extra_target"], + data_exclude_glob = ["data_exclude_all"], + srcs_exclude_glob = ["srcs_exclude_all"], + additive_build_content = """# SOMETHING SPECIAL AT THE END""", + ), + group_name = "qux", + group_deps = ["foo", "fox", "qux"], + tags = ["tag1"], + ) + env.expect.that_str(actual.replace("@@", "@")).equals(want) + +_tests.append(_test_all_legacy) + def _test_all(env): want = """\ +load("@pypi//:config.bzl", "whl_map") load("@rules_python//python/private/pypi:whl_library_targets.bzl", "whl_library_targets_from_requires") package(default_visibility = ["//visibility:public"]) @@ -47,6 +112,7 @@ whl_library_targets_from_requires( "qux", ], group_name = "qux", + include = whl_map, name = "foo.whl", requires_dist = [ "foo", @@ -54,7 +120,6 @@ whl_library_targets_from_requires( "qux", ], srcs_exclude = ["srcs_exclude_all"], - target_platforms = ["foo"], ) # SOMETHING SPECIAL AT THE END @@ -76,7 +141,6 @@ whl_library_targets_from_requires( additive_build_content = """# SOMETHING SPECIAL AT THE END""", ), group_name = "qux", - target_platforms = ["foo"], group_deps = ["foo", "fox", "qux"], ) env.expect.that_str(actual.replace("@@", "@")).equals(want) @@ -85,6 +149,7 @@ _tests.append(_test_all) def _test_all_with_loads(env): want = """\ +load("@pypi//:config.bzl", "whl_map") load("@rules_python//python/private/pypi:whl_library_targets.bzl", "whl_library_targets_from_requires") package(default_visibility = ["//visibility:public"]) @@ -111,6 +176,7 @@ whl_library_targets_from_requires( "qux", ], group_name = "qux", + include = whl_map, name = "foo.whl", requires_dist = [ "foo", diff --git a/tests/pypi/pep508/deps_tests.bzl b/tests/pypi/pep508/deps_tests.bzl index 118cd50092..aaa3b2f7dd 100644 --- a/tests/pypi/pep508/deps_tests.bzl +++ b/tests/pypi/pep508/deps_tests.bzl @@ -29,101 +29,41 @@ def test_simple_deps(env): _tests.append(test_simple_deps) def test_can_add_os_specific_deps(env): - for target in [ - struct( - platforms = [ - "linux_x86_64", - "osx_x86_64", - "osx_aarch64", - "windows_x86_64", - ], - python_version = "3.3.1", - ), - struct( - platforms = [ - "cp33_linux_x86_64", - "cp33_osx_x86_64", - "cp33_osx_aarch64", - "cp33_windows_x86_64", - ], - python_version = "", - ), - struct( - platforms = [ - "cp33.1_linux_x86_64", - "cp33.1_osx_x86_64", - "cp33.1_osx_aarch64", - "cp33.1_windows_x86_64", - ], - python_version = "", - ), - ]: - got = deps( - "foo", - requires_dist = [ - "bar", - "an_osx_dep; sys_platform=='darwin'", - "posix_dep; os_name=='posix'", - "win_dep; os_name=='nt'", - ], - platforms = target.platforms, - default_python_version = target.python_version, - ) - - env.expect.that_collection(got.deps).contains_exactly(["bar"]) - env.expect.that_dict(got.deps_select).contains_exactly({ - "linux_x86_64": ["posix_dep"], - "osx_aarch64": ["an_osx_dep", "posix_dep"], - "osx_x86_64": ["an_osx_dep", "posix_dep"], - "windows_x86_64": ["win_dep"], - }) - -_tests.append(test_can_add_os_specific_deps) - -def test_deps_are_added_to_more_specialized_platforms(env): got = deps( "foo", requires_dist = [ - "m1_dep; sys_platform=='darwin' and platform_machine=='arm64'", - "mac_dep; sys_platform=='darwin'", - ], - platforms = [ - "osx_x86_64", - "osx_aarch64", + "bar", + "an_osx_dep; sys_platform=='darwin'", + "posix_dep; os_name=='posix'", + "win_dep; os_name=='nt'", ], - default_python_version = "3.8.4", ) - env.expect.that_collection(got.deps).contains_exactly(["mac_dep"]) + env.expect.that_collection(got.deps).contains_exactly(["bar"]) env.expect.that_dict(got.deps_select).contains_exactly({ - "osx_aarch64": ["m1_dep"], + "an_osx_dep": "sys_platform == \"darwin\"", + "posix_dep": "os_name == \"posix\"", + "win_dep": "os_name == \"nt\"", }) -_tests.append(test_deps_are_added_to_more_specialized_platforms) +_tests.append(test_can_add_os_specific_deps) -def test_non_platform_markers_are_added_to_common_deps(env): +def test_deps_are_added_to_more_specialized_platforms(env): got = deps( "foo", requires_dist = [ - "bar", - "baz; implementation_name=='cpython'", "m1_dep; sys_platform=='darwin' and platform_machine=='arm64'", + "mac_dep; sys_platform=='darwin'", ], - platforms = [ - "linux_x86_64", - "osx_x86_64", - "osx_aarch64", - "windows_x86_64", - ], - default_python_version = "3.8.4", ) - env.expect.that_collection(got.deps).contains_exactly(["bar", "baz"]) + env.expect.that_collection(got.deps).contains_exactly([]) env.expect.that_dict(got.deps_select).contains_exactly({ - "osx_aarch64": ["m1_dep"], + "m1_dep": "sys_platform == \"darwin\" and platform_machine == \"arm64\"", + "mac_dep": "sys_platform == \"darwin\"", }) -_tests.append(test_non_platform_markers_are_added_to_common_deps) +_tests.append(test_deps_are_added_to_more_specialized_platforms) def test_self_is_ignored(env): got = deps( @@ -167,180 +107,59 @@ def _test_can_get_deps_based_on_specific_python_version(env): "posix_dep; os_name=='posix' and python_version >= '3.8'", ] - py38 = deps( - "foo", - requires_dist = requires_dist, - platforms = ["cp38_linux_x86_64"], - ) - py373 = deps( - "foo", - requires_dist = requires_dist, - platforms = ["cp37.3_linux_x86_64"], - ) - py37 = deps( + got = deps( "foo", requires_dist = requires_dist, - platforms = ["cp37_linux_x86_64"], ) # since there is a single target platform, the deps_select will be empty - env.expect.that_collection(py37.deps).contains_exactly(["bar", "baz"]) - env.expect.that_dict(py37.deps_select).contains_exactly({}) - env.expect.that_collection(py38.deps).contains_exactly(["bar", "posix_dep"]) - env.expect.that_dict(py38.deps_select).contains_exactly({}) - env.expect.that_collection(py373.deps).contains_exactly(["bar"]) - env.expect.that_dict(py373.deps_select).contains_exactly({}) - -_tests.append(_test_can_get_deps_based_on_specific_python_version) - -def _test_no_version_select_when_single_version(env): - got = deps( - "foo", - requires_dist = [ - "bar", - "baz; python_version >= '3.8'", - "posix_dep; os_name=='posix'", - "posix_dep_with_version; os_name=='posix' and python_version >= '3.8'", - "arch_dep; platform_machine=='x86_64' and python_version >= '3.8'", - ], - platforms = [ - "cp38_linux_x86_64", - "cp38_windows_x86_64", - ], - default_python_version = "", - ) - - env.expect.that_collection(got.deps).contains_exactly(["bar", "baz", "arch_dep"]) + env.expect.that_collection(got.deps).contains_exactly(["bar"]) env.expect.that_dict(got.deps_select).contains_exactly({ - "linux_x86_64": ["posix_dep", "posix_dep_with_version"], + "baz": "python_full_version < \"3.7.3\"", + "posix_dep": "os_name == \"posix\" and python_version >= \"3.8\"", }) -_tests.append(_test_no_version_select_when_single_version) +_tests.append(_test_can_get_deps_based_on_specific_python_version) -def _test_can_get_version_select(env): +def _test_include_only_particular_deps(env): requires_dist = [ "bar", - "baz; python_version < '3.8'", - "baz_new; python_version >= '3.8'", - "posix_dep; os_name=='posix'", - "posix_dep_with_version; os_name=='posix' and python_version >= '3.8'", - "arch_dep; platform_machine=='x86_64' and python_version < '3.8'", + "baz; python_full_version < '3.7.3'", + "posix_dep; os_name=='posix' and python_version >= '3.8'", ] got = deps( "foo", requires_dist = requires_dist, - platforms = [ - "cp3{}_{}_x86_64".format(minor, os) - for minor in ["7.4", "8.8", "9.8"] - for os in ["linux", "windows"] - ], - default_python_version = "3.7", - minor_mapping = { - "3.7": "3.7.4", - }, + include = ["bar", "posix_dep"], ) + # since there is a single target platform, the deps_select will be empty env.expect.that_collection(got.deps).contains_exactly(["bar"]) env.expect.that_dict(got.deps_select).contains_exactly({ - "cp37.4_linux_x86_64": ["arch_dep", "baz", "posix_dep"], - "cp37.4_windows_x86_64": ["arch_dep", "baz"], - "cp38.8_linux_x86_64": ["baz_new", "posix_dep", "posix_dep_with_version"], - "cp38.8_windows_x86_64": ["baz_new"], - "cp39.8_linux_x86_64": ["baz_new", "posix_dep", "posix_dep_with_version"], - "cp39.8_windows_x86_64": ["baz_new"], - "linux_x86_64": ["arch_dep", "baz", "posix_dep"], - "windows_x86_64": ["arch_dep", "baz"], + "posix_dep": "os_name == \"posix\" and python_version >= \"3.8\"", }) -_tests.append(_test_can_get_version_select) +_tests.append(_test_include_only_particular_deps) -def _test_deps_spanning_all_target_py_versions_are_added_to_common(env): +def test_all_markers_are_added(env): requires_dist = [ "bar", "baz (<2,>=1.11) ; python_version < '3.8'", "baz (<2,>=1.14) ; python_version >= '3.8'", ] - default_python_version = "3.8.4" got = deps( "foo", requires_dist = requires_dist, - platforms = [ - "cp3{}_linux_x86_64".format(minor) - for minor in [7, 8, 9] - ], - default_python_version = default_python_version, - ) - - env.expect.that_collection(got.deps).contains_exactly(["bar", "baz"]) - env.expect.that_dict(got.deps_select).contains_exactly({}) - -_tests.append(_test_deps_spanning_all_target_py_versions_are_added_to_common) - -def _test_deps_are_not_duplicated(env): - default_python_version = "3.7.4" - - # See an example in - # https://files.pythonhosted.org/packages/76/9e/db1c2d56c04b97981c06663384f45f28950a73d9acf840c4006d60d0a1ff/opencv_python-4.9.0.80-cp37-abi3-win32.whl.metadata - requires_dist = [ - "bar >=0.1.0 ; python_version < '3.7'", - "bar >=0.2.0 ; python_version >= '3.7'", - "bar >=0.4.0 ; python_version >= '3.6' and platform_system == 'Linux' and platform_machine == 'aarch64'", - "bar >=0.4.0 ; python_version >= '3.9'", - "bar >=0.5.0 ; python_version <= '3.9' and platform_system == 'Darwin' and platform_machine == 'arm64'", - "bar >=0.5.0 ; python_version >= '3.10' and platform_system == 'Darwin'", - "bar >=0.5.0 ; python_version >= '3.10'", - "bar >=0.6.0 ; python_version >= '3.11'", - ] - - got = deps( - "foo", - requires_dist = requires_dist, - platforms = [ - "cp3{}_{}_{}".format(minor, os, arch) - for minor in [7, 10] - for os in ["linux", "osx", "windows"] - for arch in ["x86_64", "aarch64"] - ], - default_python_version = default_python_version, ) env.expect.that_collection(got.deps).contains_exactly(["bar"]) - env.expect.that_dict(got.deps_select).contains_exactly({}) - -_tests.append(_test_deps_are_not_duplicated) - -def _test_deps_are_not_duplicated_when_encountering_platform_dep_first(env): - # Note, that we are sorting the incoming `requires_dist` and we need to ensure that we are not getting any - # issues even if the platform-specific line comes first. - requires_dist = [ - "bar >=0.4.0 ; python_version >= '3.6' and platform_system == 'Linux' and platform_machine == 'aarch64'", - "bar >=0.5.0 ; python_version >= '3.9'", - ] - - got = deps( - "foo", - requires_dist = requires_dist, - platforms = [ - "cp37.1_linux_aarch64", - "cp37.1_linux_x86_64", - "cp310_linux_aarch64", - "cp310_linux_x86_64", - ], - default_python_version = "3.7.1", - minor_mapping = {}, - ) - - env.expect.that_collection(got.deps).contains_exactly([]) env.expect.that_dict(got.deps_select).contains_exactly({ - "cp310_linux_aarch64": ["bar"], - "cp310_linux_x86_64": ["bar"], - "cp37.1_linux_aarch64": ["bar"], - "linux_aarch64": ["bar"], + "baz": "(python_version < \"3.8\") or (python_version >= \"3.8\")", }) -_tests.append(_test_deps_are_not_duplicated_when_encountering_platform_dep_first) +_tests.append(test_all_markers_are_added) def deps_test_suite(name): # buildifier: disable=function-docstring test_suite( diff --git a/tests/pypi/whl_installer/wheel_installer_test.py b/tests/pypi/whl_installer/wheel_installer_test.py index e838047925..ef5a2483ab 100644 --- a/tests/pypi/whl_installer/wheel_installer_test.py +++ b/tests/pypi/whl_installer/wheel_installer_test.py @@ -72,7 +72,7 @@ def test_wheel_exists(self) -> None: extras={}, enable_implicit_namespace_pkgs=False, platforms=[], - enable_pipstar = False, + enable_pipstar=False, ) want_files = [ @@ -97,7 +97,6 @@ def test_wheel_exists(self) -> None: deps_by_platform={}, entry_points=[], name="example-minimal-package", - python_version="3.11.11", version="0.0.1", ) self.assertEqual(want, metadata_file_content) diff --git a/tests/pypi/whl_library_targets/whl_library_targets_tests.bzl b/tests/pypi/whl_library_targets/whl_library_targets_tests.bzl index 432cdbfa1b..f0e5f57ac0 100644 --- a/tests/pypi/whl_library_targets/whl_library_targets_tests.bzl +++ b/tests/pypi/whl_library_targets/whl_library_targets_tests.bzl @@ -180,18 +180,20 @@ _tests.append(_test_entrypoints) def _test_whl_and_library_deps_from_requires(env): filegroup_calls = [] py_library_calls = [] + env_marker_setting_calls = [] whl_library_targets_from_requires( name = "foo-0-py3-none-any.whl", metadata_name = "Foo", metadata_version = "0", - dep_template = "@pypi_{name}//:{target}", + dep_template = "@pypi//{name}:{target}", requires_dist = [ "foo", # this self-edge will be ignored - "bar-baz", + "bar", + "bar-baz; python_version < \"8.2\"", + "booo", # this is effectively excluded due to the list below ], - target_platforms = ["cp38_linux_x86_64"], - default_python_version = "3.8.1", + include = ["foo", "bar", "bar_baz"], data_exclude = [], # Overrides for testing filegroups = {}, @@ -203,6 +205,7 @@ def _test_whl_and_library_deps_from_requires(env): ), rules = struct( py_library = lambda **kwargs: py_library_calls.append(kwargs), + env_marker_setting = lambda **kwargs: env_marker_setting_calls.append(kwargs), ), ) @@ -210,7 +213,10 @@ def _test_whl_and_library_deps_from_requires(env): { "name": "whl", "srcs": ["foo-0-py3-none-any.whl"], - "data": ["@pypi_bar_baz//:whl"], + "data": ["@pypi//bar:whl"] + _select({ + ":is_include_bar_baz_true": ["@pypi//bar_baz:whl"], + "//conditions:default": [], + }), "visibility": ["//visibility:public"], }, ]) # buildifier: @unsorted-dict-items @@ -233,12 +239,22 @@ def _test_whl_and_library_deps_from_requires(env): ] + glob_excludes.version_dependent_exclusions(), ), "imports": ["site-packages"], - "deps": ["@pypi_bar_baz//:pkg"], + "deps": ["@pypi//bar:pkg"] + _select({ + ":is_include_bar_baz_true": ["@pypi//bar_baz:pkg"], + "//conditions:default": [], + }), "tags": ["pypi_name=Foo", "pypi_version=0"], "visibility": ["//visibility:public"], "experimental_venvs_site_packages": Label("//python/config_settings:venvs_site_packages"), }, ]) # buildifier: @unsorted-dict-items + env.expect.that_collection(env_marker_setting_calls).contains_exactly([ + { + "name": "include_bar_baz", + "expression": "python_version < \"8.2\"", + "visibility": ["//visibility:private"], + }, + ]) # buildifier: @unsorted-dict-items _tests.append(_test_whl_and_library_deps_from_requires) From 367d09ec01ce5640ee9587398b6a8ce56a7eb0ba Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Tue, 13 May 2025 15:31:40 -0700 Subject: [PATCH 218/922] refactor: make python extension generate platform toolchains (#2875) This makes the python bzlmod extension handle generating the platform-specific toolchain entries ("python_3_10_{platform}"). This is to eventually allow adding additional toolchains that aren't part of the PLATFORMS mapping in versions.bzl and have their own custom constraints. The main things this refactor does are: 1. The bzlmod phase passes the full list of implementation toolchains to create (previously, it relied on `hub_repo` to generate the implementation names). 2. The name of a toolchain (the toolchain.name arg) and the repo that implements it (the py_toolchain_suite.user_repository_repo arg) are separate. This allows future work to mixin toolchains that point to arbitrary repos. 3. The platform meta data uses a list of target settings instead of dict of flag values. This allows more arbitrary target settings. For now, flag values on the platform metadata is still looked for because it's known that users patch python/versions.bzl. Along the way: * Factor out a platform_info helper in versions.bzl * Factor out a NOT_ACTUALLY_PUBLIC constants to better denote things that are public visibility, but actually internal. * Add some docs to some internals so we don't have to chase down their definitions. Work towards https://github.com/bazel-contrib/rules_python/issues/2081 --- internal_dev_setup.bzl | 12 ++- python/private/config_settings.bzl | 29 +++++- python/private/py_repositories.bzl | 12 ++- python/private/py_toolchain_suite.bzl | 11 ++- python/private/python.bzl | 102 ++++++++++++++------ python/private/pythons_hub.bzl | 116 +++++++++++----------- python/private/toolchains_repo.bzl | 67 ++++++++----- python/versions.bzl | 133 +++++++++++++++----------- 8 files changed, 309 insertions(+), 173 deletions(-) diff --git a/internal_dev_setup.bzl b/internal_dev_setup.bzl index f33908049f..62a11ab1d4 100644 --- a/internal_dev_setup.bzl +++ b/internal_dev_setup.bzl @@ -34,11 +34,15 @@ def rules_python_internal_setup(): name = "pythons_hub", minor_mapping = MINOR_MAPPING, default_python_version = "", - toolchain_prefixes = [], - toolchain_python_versions = [], - toolchain_set_python_version_constraints = [], - toolchain_user_repository_names = [], python_versions = sorted(TOOL_VERSIONS.keys()), + toolchain_names = [], + toolchain_repo_names = {}, + toolchain_target_compatible_with_map = {}, + toolchain_target_settings_map = {}, + toolchain_platform_keys = {}, + toolchain_python_versions = {}, + toolchain_set_python_version_constraints = {}, + base_toolchain_repo_names = [], ) runtime_env_repo(name = "rules_python_runtime_env_tc_info") diff --git a/python/private/config_settings.bzl b/python/private/config_settings.bzl index 1685195b78..5eb858e2e4 100644 --- a/python/private/config_settings.bzl +++ b/python/private/config_settings.bzl @@ -31,6 +31,10 @@ If the value is missing, then the default value is being used, see documentation {docs_url}/python/config_settings """ +# Indicates something needs public visibility so that other generated code can +# access it, but it's not intended for general public usage. +_NOT_ACTUALLY_PUBLIC = ["//visibility:public"] + def construct_config_settings(*, name, default_version, versions, minor_mapping, documented_flags): # buildifier: disable=function-docstring """Create a 'python_version' config flag and construct all config settings used in rules_python. @@ -128,7 +132,30 @@ def construct_config_settings(*, name, default_version, versions, minor_mapping, # `whl_library` in the hub repo created by `pip.parse`. flag_values = {"current_config": "will-never-match"}, # Only public so that PyPI hub repo can access it - visibility = ["//visibility:public"], + visibility = _NOT_ACTUALLY_PUBLIC, + ) + + libc = Label("//python/config_settings:py_linux_libc") + native.config_setting( + name = "_is_py_linux_libc_glibc", + flag_values = {libc: "glibc"}, + visibility = _NOT_ACTUALLY_PUBLIC, + ) + native.config_setting( + name = "_is_py_linux_libc_musl", + flag_values = {libc: "glibc"}, + visibility = _NOT_ACTUALLY_PUBLIC, + ) + freethreaded = Label("//python/config_settings:py_freethreaded") + native.config_setting( + name = "_is_py_freethreaded_yes", + flag_values = {freethreaded: "yes"}, + visibility = _NOT_ACTUALLY_PUBLIC, + ) + native.config_setting( + name = "_is_py_freethreaded_no", + flag_values = {freethreaded: "no"}, + visibility = _NOT_ACTUALLY_PUBLIC, ) def _python_version_flag_impl(ctx): diff --git a/python/private/py_repositories.bzl b/python/private/py_repositories.bzl index 46ca903df4..b5bd93b7c1 100644 --- a/python/private/py_repositories.bzl +++ b/python/private/py_repositories.bzl @@ -39,11 +39,15 @@ def py_repositories(): name = "pythons_hub", minor_mapping = MINOR_MAPPING, default_python_version = "", - toolchain_prefixes = [], - toolchain_python_versions = [], - toolchain_set_python_version_constraints = [], - toolchain_user_repository_names = [], python_versions = sorted(TOOL_VERSIONS.keys()), + toolchain_names = [], + toolchain_repo_names = {}, + toolchain_target_compatible_with_map = {}, + toolchain_target_settings_map = {}, + toolchain_platform_keys = {}, + toolchain_python_versions = {}, + toolchain_set_python_version_constraints = {}, + base_toolchain_repo_names = [], ) http_archive( name = "bazel_skylib", diff --git a/python/private/py_toolchain_suite.bzl b/python/private/py_toolchain_suite.bzl index e71882dafd..fa73d5daa3 100644 --- a/python/private/py_toolchain_suite.bzl +++ b/python/private/py_toolchain_suite.bzl @@ -34,15 +34,20 @@ def py_toolchain_suite( python_version, set_python_version_constraint, flag_values, + target_settings = [], target_compatible_with = []): """For internal use only. Args: prefix: Prefix for toolchain target names. - user_repository_name: The name of the user repository. + user_repository_name: The name of the repository with the toolchain + implementation (it's assumed to have particular target names within + it). Does not include the leading "@". python_version: The full (X.Y.Z) version of the interpreter. set_python_version_constraint: True or False as a string. - flag_values: Extra flag values to match for this toolchain. + flag_values: Extra flag values to match for this toolchain. These + are prepended to target_settings. + target_settings: Extra target_settings to match for this toolchain. target_compatible_with: list constraints the toolchains are compatible with. """ @@ -82,7 +87,7 @@ def py_toolchain_suite( match_any = match_any, visibility = ["//visibility:private"], ) - target_settings = [name] + target_settings = [name] + target_settings else: fail(("Invalid set_python_version_constraint value: got {} {}, wanted " + "either the string 'True' or the string 'False'; " + diff --git a/python/private/python.bzl b/python/private/python.bzl index f49fb26d52..53cd5e9cd2 100644 --- a/python/private/python.bzl +++ b/python/private/python.bzl @@ -22,15 +22,9 @@ load(":python_register_toolchains.bzl", "python_register_toolchains") load(":pythons_hub.bzl", "hub_repo") load(":repo_utils.bzl", "repo_utils") load(":semver.bzl", "semver") -load(":text_util.bzl", "render") load(":toolchains_repo.bzl", "multi_toolchain_aliases") load(":util.bzl", "IS_BAZEL_6_4_OR_HIGHER") -# This limit can be increased essentially arbitrarily, but doing so will cause a rebuild of all -# targets using any of these toolchains due to the changed repository name. -_MAX_NUM_TOOLCHAINS = 9999 -_TOOLCHAIN_INDEX_PAD_LENGTH = len(str(_MAX_NUM_TOOLCHAINS)) - def parse_modules(*, module_ctx, _fail = fail): """Parse the modules and return a struct for registrations. @@ -240,9 +234,6 @@ def parse_modules(*, module_ctx, _fail = fail): # toolchain. We need the default last. toolchains.append(default_toolchain) - if len(toolchains) > _MAX_NUM_TOOLCHAINS: - fail("more than {} python versions are not supported".format(_MAX_NUM_TOOLCHAINS)) - # sort the toolchains so that the toolchain versions that are in the # `minor_mapping` are coming first. This ensures that `python_version = # "3.X"` transitions work as expected. @@ -275,6 +266,9 @@ def parse_modules(*, module_ctx, _fail = fail): def _python_impl(module_ctx): py = parse_modules(module_ctx = module_ctx) + # dict[str version, list[str] platforms]; where version is full + # python version string ("3.4.5"), and platforms are keys from + # the PLATFORMS global. loaded_platforms = {} for toolchain_info in py.toolchains: # Ensure that we pass the full version here. @@ -297,30 +291,82 @@ def _python_impl(module_ctx): **kwargs ) - # Create the pythons_hub repo for the interpreter meta data and the - # the various toolchains. + # List of the base names ("python_3_10") for the toolchain repos + base_toolchain_repo_names = [] + + # list[str] The infix to use for the resulting toolchain() `name` arg. + toolchain_names = [] + + # dict[str i, str repo]; where repo is the full repo name + # ("python_3_10_unknown-linux-x86_64") for the toolchain + # i corresponds to index `i` in toolchain_names + toolchain_repo_names = {} + + # dict[str i, list[str] constraints]; where constraints is a list + # of labels for target_compatible_with + # i corresponds to index `i` in toolchain_names + toolchain_tcw_map = {} + + # dict[str i, list[str] settings]; where settings is a list + # of labels for target_settings + # i corresponds to index `i` in toolchain_names + toolchain_ts_map = {} + + # dict[str i, str set_constraint]; where set_constraint is the string + # "True" or "False". + # i corresponds to index `i` in toolchain_names + toolchain_set_python_version_constraints = {} + + # dict[str i, str python_version]; where python_version is the full + # python version ("3.4.5"). + toolchain_python_versions = {} + + # dict[str i, str platform_key]; where platform_key is the key within + # the PLATFORMS global for this toolchain + toolchain_platform_keys = {} + + # Split the toolchain info into separate objects so they can be passed onto + # the repository rule. + for i, t in enumerate(py.toolchains): + is_last = (i + 1) == len(py.toolchains) + base_name = t.name + base_toolchain_repo_names.append(base_name) + fv = full_version(version = t.python_version, minor_mapping = py.config.minor_mapping) + for platform in loaded_platforms[fv]: + if platform not in PLATFORMS: + continue + key = str(len(toolchain_names)) + + full_name = "{}_{}".format(base_name, platform) + toolchain_names.append(full_name) + toolchain_repo_names[key] = full_name + toolchain_tcw_map[key] = PLATFORMS[platform].compatible_with + + # The target_settings attribute may not be present for users + # patching python/versions.bzl. + toolchain_ts_map[key] = getattr(PLATFORMS[platform], "target_settings", []) + toolchain_platform_keys[key] = platform + toolchain_python_versions[key] = fv + + # The last toolchain is the default; it can't have version constraints + # Despite the implication of the arg name, the values are strs, not bools + toolchain_set_python_version_constraints[key] = ( + "True" if not is_last else "False" + ) + hub_repo( name = "pythons_hub", - # Last toolchain is default + toolchain_names = toolchain_names, + toolchain_repo_names = toolchain_repo_names, + toolchain_target_compatible_with_map = toolchain_tcw_map, + toolchain_target_settings_map = toolchain_ts_map, + toolchain_platform_keys = toolchain_platform_keys, + toolchain_python_versions = toolchain_python_versions, + toolchain_set_python_version_constraints = toolchain_set_python_version_constraints, + base_toolchain_repo_names = [t.name for t in py.toolchains], default_python_version = py.default_python_version, minor_mapping = py.config.minor_mapping, python_versions = list(py.config.default["tool_versions"].keys()), - toolchain_prefixes = [ - render.toolchain_prefix(index, toolchain.name, _TOOLCHAIN_INDEX_PAD_LENGTH) - for index, toolchain in enumerate(py.toolchains) - ], - toolchain_python_versions = [ - full_version(version = t.python_version, minor_mapping = py.config.minor_mapping) - for t in py.toolchains - ], - # The last toolchain is the default; it can't have version constraints - # Despite the implication of the arg name, the values are strs, not bools - toolchain_set_python_version_constraints = [ - "True" if i != len(py.toolchains) - 1 else "False" - for i in range(len(py.toolchains)) - ], - toolchain_user_repository_names = [t.name for t in py.toolchains], - loaded_platforms = loaded_platforms, ) # This is require in order to support multiple version py_test diff --git a/python/private/pythons_hub.bzl b/python/private/pythons_hub.bzl index b448d53097..53351cacb9 100644 --- a/python/private/pythons_hub.bzl +++ b/python/private/pythons_hub.bzl @@ -16,7 +16,7 @@ load("//python:versions.bzl", "PLATFORMS") load(":text_util.bzl", "render") -load(":toolchains_repo.bzl", "python_toolchain_build_file_content") +load(":toolchains_repo.bzl", "toolchain_suite_content") def _have_same_length(*lists): if not lists: @@ -24,8 +24,10 @@ def _have_same_length(*lists): return len({len(length): None for length in lists}) == 1 _HUB_BUILD_FILE_TEMPLATE = """\ -load("@bazel_skylib//:bzl_library.bzl", "bzl_library") +# Generated by @rules_python//python/private:pythons_hub.bzl + load("@@{rules_python}//python/private:py_toolchain_suite.bzl", "py_toolchain_suite") +load("@bazel_skylib//:bzl_library.bzl", "bzl_library") bzl_library( name = "interpreters_bzl", @@ -42,44 +44,43 @@ bzl_library( {toolchains} """ -def _hub_build_file_content( - prefixes, - python_versions, - set_python_version_constraints, - user_repository_names, - workspace_location, - loaded_platforms): - """This macro iterates over each of the lists and returns the toolchain content. - - python_toolchain_build_file_content is called to generate each of the toolchain - definitions. - """ - - if not _have_same_length(python_versions, set_python_version_constraints, user_repository_names): +def _hub_build_file_content(rctx): + # Verify a precondition. If these don't match, then something went wrong. + if not _have_same_length( + rctx.attr.toolchain_names, + rctx.attr.toolchain_platform_keys, + rctx.attr.toolchain_repo_names, + rctx.attr.toolchain_target_compatible_with_map, + rctx.attr.toolchain_target_settings_map, + rctx.attr.toolchain_set_python_version_constraints, + rctx.attr.toolchain_python_versions, + ): fail("all lists must have the same length") - # Iterate over the length of python_versions and call - # build the toolchain content by calling python_toolchain_build_file_content - toolchains = "\n".join( - [ - python_toolchain_build_file_content( - prefix = prefixes[i], - python_version = python_versions[i], - set_python_version_constraint = set_python_version_constraints[i], - user_repository_name = user_repository_names[i], - loaded_platforms = { - k: v - for k, v in PLATFORMS.items() - if k in loaded_platforms[python_versions[i]] - }, - ) - for i in range(len(python_versions)) - ], - ) + #pad_length = len(str(len(rctx.attr.toolchain_names))) + 1 + pad_length = 4 + toolchains = [] + for i, base_name in enumerate(rctx.attr.toolchain_names): + key = str(i) + platform = rctx.attr.toolchain_platform_keys[key] + if platform in PLATFORMS: + flag_values = PLATFORMS[platform].flag_values + else: + flag_values = {} + + toolchains.append(toolchain_suite_content( + prefix = "_{}_{}".format(render.left_pad_zero(i, pad_length), base_name), + user_repository_name = rctx.attr.toolchain_repo_names[key], + target_compatible_with = rctx.attr.toolchain_target_compatible_with_map[key], + flag_values = flag_values, + target_settings = rctx.attr.toolchain_target_settings_map[key], + set_python_version_constraint = rctx.attr.toolchain_set_python_version_constraints[key], + python_version = rctx.attr.toolchain_python_versions[key], + )) return _HUB_BUILD_FILE_TEMPLATE.format( - toolchains = toolchains, - rules_python = workspace_location.repo_name, + toolchains = "\n".join(toolchains), + rules_python = rctx.attr._rules_python_workspace.repo_name, ) _interpreters_bzl_template = """ @@ -103,14 +104,7 @@ def _hub_repo_impl(rctx): # write them to the BUILD file. rctx.file( "BUILD.bazel", - _hub_build_file_content( - rctx.attr.toolchain_prefixes, - rctx.attr.toolchain_python_versions, - rctx.attr.toolchain_set_python_version_constraints, - rctx.attr.toolchain_user_repository_names, - rctx.attr._rules_python_workspace, - rctx.attr.loaded_platforms, - ), + _hub_build_file_content(rctx), executable = False, ) @@ -118,7 +112,7 @@ def _hub_repo_impl(rctx): # a symlink to a interpreter. interpreter_labels = "".join([ _line_for_hub_template.format(name = name) - for name in rctx.attr.toolchain_user_repository_names + for name in rctx.attr.base_toolchain_repo_names ]) rctx.file( @@ -150,13 +144,15 @@ This rule also writes out the various toolchains for the different Python versio """, implementation = _hub_repo_impl, attrs = { + "base_toolchain_repo_names": attr.string_list( + doc = "The base repo name for toolchains ('python_3_10', no " + + "platform suffix)", + mandatory = True, + ), "default_python_version": attr.string( doc = "Default Python version for the build in `X.Y` or `X.Y.Z` format.", mandatory = True, ), - "loaded_platforms": attr.string_list_dict( - doc = "The list of loaded platforms keyed by the toolchain full python version", - ), "minor_mapping": attr.string_dict( doc = "The minor mapping of the `X.Y` to `X.Y.Z` format that is used in config settings.", mandatory = True, @@ -165,20 +161,32 @@ This rule also writes out the various toolchains for the different Python versio doc = "The list of python versions to include in the `interpreters.bzl` if the toolchains are not specified. Used in `WORKSPACE` builds.", mandatory = False, ), - "toolchain_prefixes": attr.string_list( - doc = "List prefixed for the toolchains", + "toolchain_names": attr.string_list( + doc = "Names of toolchains", + mandatory = True, + ), + "toolchain_platform_keys": attr.string_dict( + doc = "The platform key in PLATFORMS for toolchains.", mandatory = True, ), - "toolchain_python_versions": attr.string_list( + "toolchain_python_versions": attr.string_dict( doc = "List of Python versions for the toolchains. In `X.Y.Z` format.", mandatory = True, ), - "toolchain_set_python_version_constraints": attr.string_list( + "toolchain_repo_names": attr.string_dict( + doc = "The repo names containing toolchain implementations.", + mandatory = True, + ), + "toolchain_set_python_version_constraints": attr.string_dict( doc = "List of version contraints for the toolchains", mandatory = True, ), - "toolchain_user_repository_names": attr.string_list( - doc = "List of the user repo names for the toolchains", + "toolchain_target_compatible_with_map": attr.string_list_dict( + doc = "The target_compatible_with settings for toolchains.", + mandatory = True, + ), + "toolchain_target_settings_map": attr.string_list_dict( + doc = "The target_settings for toolchains", mandatory = True, ), "_rules_python_workspace": attr.label(default = Label("//:does_not_matter_what_this_name_is")), diff --git a/python/private/toolchains_repo.bzl b/python/private/toolchains_repo.bzl index 23c4643c0a..d0814b66d5 100644 --- a/python/private/toolchains_repo.bzl +++ b/python/private/toolchains_repo.bzl @@ -31,6 +31,18 @@ load( load(":repo_utils.bzl", "REPO_DEBUG_ENV_VAR", "repo_utils") load(":text_util.bzl", "render") +_SUITE_TEMPLATE = """ +py_toolchain_suite( + flag_values = {flag_values}, + target_settings = {target_settings}, + prefix = {prefix}, + python_version = {python_version}, + set_python_version_constraint = {set_python_version_constraint}, + target_compatible_with = {target_compatible_with}, + user_repository_name = {user_repository_name}, +) +""".lstrip() + def python_toolchain_build_file_content( prefix, python_version, @@ -53,29 +65,40 @@ def python_toolchain_build_file_content( build_content: Text containing toolchain definitions """ - return "\n\n".join([ - """\ -py_toolchain_suite( - user_repository_name = "{user_repository_name}_{platform}", - prefix = "{prefix}{platform}", - target_compatible_with = {compatible_with}, - flag_values = {flag_values}, - python_version = "{python_version}", - set_python_version_constraint = "{set_python_version_constraint}", -)""".format( - compatible_with = render.indent(render.list(meta.compatible_with)).lstrip(), - flag_values = render.indent(render.dict( - meta.flag_values, - key_repr = lambda x: repr(str(x)), # this is to correctly display labels - )).lstrip(), - platform = platform, - set_python_version_constraint = set_python_version_constraint, - user_repository_name = user_repository_name, - prefix = prefix, + entries = [] + for platform, meta in loaded_platforms.items(): + entries.append(toolchain_suite_content( + target_compatible_with = meta.compatible_with, + flag_values = meta.flag_values, + prefix = "{}{}".format(prefix, platform), + user_repository_name = "{}_{}".format(user_repository_name, platform), python_version = python_version, - ) - for platform, meta in loaded_platforms.items() - ]) + set_python_version_constraint = set_python_version_constraint, + target_settings = [], + )) + return "\n\n".join(entries) + +def toolchain_suite_content( + *, + flag_values, + prefix, + python_version, + set_python_version_constraint, + target_compatible_with, + target_settings, + user_repository_name): + return _SUITE_TEMPLATE.format( + prefix = render.str(prefix), + user_repository_name = render.str(user_repository_name), + target_compatible_with = render.indent(render.list(target_compatible_with)).lstrip(), + flag_values = render.indent(render.dict( + flag_values, + key_repr = lambda x: repr(str(x)), # this is to correctly display labels + )).lstrip(), + target_settings = render.list(target_settings, hanging_indent = " "), + set_python_version_constraint = render.str(set_python_version_constraint), + python_version = render.str(python_version), + ) def _toolchains_repo_impl(rctx): build_content = """\ diff --git a/python/versions.bzl b/python/versions.bzl index 6343ee49c8..4a2a4cb758 100644 --- a/python/versions.bzl +++ b/python/versions.bzl @@ -682,151 +682,170 @@ MINOR_MAPPING = { "3.13": "3.13.2", } +def _platform_info( + *, + compatible_with = [], + flag_values = {}, + target_settings = [], + os_name, + arch): + """Creates a struct of platform metadata. + + Args: + compatible_with: list[str], where the values are string labels. These + are the target_compatible_with values to use with the toolchain + flag_values: dict[str|Label, Any] of config_setting.flag_values + compatible values. DEPRECATED -- use target_settings instead + target_settings: list[str], where the values are string labels. These + are the target_settings values to use with the toolchain. + os_name: str, the os name; must match the name used in `@platfroms//os` + arch: str, the cpu name; must match the name used in `@platforms//cpu` + + Returns: + A struct with attributes and values matching the args. + """ + return struct( + compatible_with = compatible_with, + flag_values = flag_values, + target_settings = target_settings, + os_name = os_name, + arch = arch, + ) + def _generate_platforms(): - libc = Label("//python/config_settings:py_linux_libc") + is_libc_glibc = str(Label("//python/config_settings:_is_py_linux_libc_glibc")) + is_libc_musl = str(Label("//python/config_settings:_is_py_linux_libc_musl")) platforms = { - "aarch64-apple-darwin": struct( + "aarch64-apple-darwin": _platform_info( compatible_with = [ "@platforms//os:macos", "@platforms//cpu:aarch64", ], - flag_values = {}, os_name = MACOS_NAME, - # Matches the value in @platforms//cpu package arch = "aarch64", ), - "aarch64-unknown-linux-gnu": struct( + "aarch64-unknown-linux-gnu": _platform_info( compatible_with = [ "@platforms//os:linux", "@platforms//cpu:aarch64", ], - flag_values = { - libc: "glibc", - }, + target_settings = [ + is_libc_glibc, + ], os_name = LINUX_NAME, - # Matches the value in @platforms//cpu package arch = "aarch64", ), - "armv7-unknown-linux-gnu": struct( + "armv7-unknown-linux-gnu": _platform_info( compatible_with = [ "@platforms//os:linux", "@platforms//cpu:armv7", ], - flag_values = { - libc: "glibc", - }, + target_settings = [ + is_libc_glibc, + ], os_name = LINUX_NAME, - # Matches the value in @platforms//cpu package arch = "arm", ), - "i386-unknown-linux-gnu": struct( + "i386-unknown-linux-gnu": _platform_info( compatible_with = [ "@platforms//os:linux", "@platforms//cpu:i386", ], - flag_values = { - libc: "glibc", - }, + target_settings = [ + is_libc_glibc, + ], os_name = LINUX_NAME, - # Matches the value in @platforms//cpu package arch = "x86_32", ), - "ppc64le-unknown-linux-gnu": struct( + "ppc64le-unknown-linux-gnu": _platform_info( compatible_with = [ "@platforms//os:linux", "@platforms//cpu:ppc", ], - flag_values = { - libc: "glibc", - }, + target_settings = [ + is_libc_glibc, + ], os_name = LINUX_NAME, - # Matches the value in @platforms//cpu package arch = "ppc", ), - "riscv64-unknown-linux-gnu": struct( + "riscv64-unknown-linux-gnu": _platform_info( compatible_with = [ "@platforms//os:linux", "@platforms//cpu:riscv64", ], - flag_values = { - Label("//python/config_settings:py_linux_libc"): "glibc", - }, + target_settings = [ + is_libc_glibc, + ], os_name = LINUX_NAME, - # Matches the value in @platforms//cpu package arch = "riscv64", ), - "s390x-unknown-linux-gnu": struct( + "s390x-unknown-linux-gnu": _platform_info( compatible_with = [ "@platforms//os:linux", "@platforms//cpu:s390x", ], - flag_values = { - Label("//python/config_settings:py_linux_libc"): "glibc", - }, + target_settings = [ + is_libc_glibc, + ], os_name = LINUX_NAME, - # Matches the value in @platforms//cpu package arch = "s390x", ), - "x86_64-apple-darwin": struct( + "x86_64-apple-darwin": _platform_info( compatible_with = [ "@platforms//os:macos", "@platforms//cpu:x86_64", ], - flag_values = {}, os_name = MACOS_NAME, - # Matches the value in @platforms//cpu package arch = "x86_64", ), - "x86_64-pc-windows-msvc": struct( + "x86_64-pc-windows-msvc": _platform_info( compatible_with = [ "@platforms//os:windows", "@platforms//cpu:x86_64", ], - flag_values = {}, os_name = WINDOWS_NAME, - # Matches the value in @platforms//cpu package arch = "x86_64", ), - "x86_64-unknown-linux-gnu": struct( + "x86_64-unknown-linux-gnu": _platform_info( compatible_with = [ "@platforms//os:linux", "@platforms//cpu:x86_64", ], - flag_values = { - libc: "glibc", - }, + target_settings = [ + is_libc_glibc, + ], os_name = LINUX_NAME, - # Matches the value in @platforms//cpu package arch = "x86_64", ), - "x86_64-unknown-linux-musl": struct( + "x86_64-unknown-linux-musl": _platform_info( compatible_with = [ "@platforms//os:linux", "@platforms//cpu:x86_64", ], - flag_values = { - libc: "musl", - }, + target_settings = [ + is_libc_musl, + ], os_name = LINUX_NAME, arch = "x86_64", ), } - freethreaded = Label("//python/config_settings:py_freethreaded") + is_freethreaded_yes = str(Label("//python/config_settings:_is_py_freethreaded_yes")) + is_freethreaded_no = str(Label("//python/config_settings:_is_py_freethreaded_no")) return { - p + suffix: struct( + p + suffix: _platform_info( compatible_with = v.compatible_with, - flag_values = { - freethreaded: freethreaded_value, - } | v.flag_values, + target_settings = [ + freethreadedness, + ] + v.target_settings, os_name = v.os_name, arch = v.arch, ) for p, v in platforms.items() - for suffix, freethreaded_value in { - "": "no", - "-" + FREETHREADED: "yes", + for suffix, freethreadedness in { + "": is_freethreaded_no, + "-" + FREETHREADED: is_freethreaded_yes, }.items() } From 61b5a8d738a5478f6ffd354e3dc2459e089c287c Mon Sep 17 00:00:00 2001 From: Garrett Holmstrom Date: Tue, 13 May 2025 21:08:27 -0700 Subject: [PATCH 219/922] Fix whl_library file path inference (#2876) When given .whl file URLs and no file name, `whl_library` writes the wheel it downloads to a file with the same file name as the first URL's. But then at extraction time, it always consults ctx.attr.filename for that file name, leading to failure when that attribute is None. This patch should fix that. Related #2363 --- CHANGELOG.md | 1 + python/private/pypi/whl_library.bzl | 2 +- 2 files changed, 2 insertions(+), 1 deletion(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index b94072d655..94487219bc 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -90,6 +90,7 @@ END_UNRELEASED_TEMPLATE various URL formats - URL encoded version strings get correctly resolved, sha256 value can be also retrieved from the URL as opposed to only the `--hash` parameter. Fixes [#2363](https://github.com/bazel-contrib/rules_python/issues/2363). +* (pypi) `whl_library` now infers file names from its `urls` attribute correctly. {#v0-0-0-added} ### Added diff --git a/python/private/pypi/whl_library.bzl b/python/private/pypi/whl_library.bzl index b370de448a..17ee3d3cfe 100644 --- a/python/private/pypi/whl_library.bzl +++ b/python/private/pypi/whl_library.bzl @@ -277,7 +277,7 @@ def _whl_library_impl(rctx): fail("could not download the '{}' from {}:\n{}".format(filename, urls, result)) if filename.endswith(".whl"): - whl_path = rctx.path(rctx.attr.filename) + whl_path = rctx.path(filename) else: # It is an sdist and we need to tell PyPI to use a file in this directory # and, allow getting build dependencies from PYTHONPATH, which we From ee8d7d618cff43811779bb710d330ccd9bd577f2 Mon Sep 17 00:00:00 2001 From: Ignas Anikevicius <240938+aignas@users.noreply.github.com> Date: Fri, 16 May 2025 00:40:29 +0900 Subject: [PATCH 220/922] refactor: consolidate version parsing (#2874) This PR removes all of the custom version parsing functions where we try to make sense about the version (e.g. extracting major/minor versions). Whilst doing this I actually think that I made it easier to support #2837. --- python/private/BUILD.bazel | 9 +- python/private/config_settings.bzl | 6 +- .../private/hermetic_runtime_repo_setup.bzl | 15 ++- python/private/pypi/BUILD.bazel | 4 +- python/private/pypi/extension.bzl | 8 +- python/private/python.bzl | 37 +++--- python/private/semver.bzl | 85 ------------- python/private/version.bzl | 28 +++-- tests/python/python_tests.bzl | 14 +-- tests/semver/BUILD.bazel | 17 --- tests/semver/semver_test.bzl | 113 ------------------ 11 files changed, 61 insertions(+), 275 deletions(-) delete mode 100644 python/private/semver.bzl delete mode 100644 tests/semver/BUILD.bazel delete mode 100644 tests/semver/semver_test.bzl diff --git a/python/private/BUILD.bazel b/python/private/BUILD.bazel index e72a8fcaa7..0b50ccf0b7 100644 --- a/python/private/BUILD.bazel +++ b/python/private/BUILD.bazel @@ -139,7 +139,7 @@ bzl_library( name = "config_settings_bzl", srcs = ["config_settings.bzl"], deps = [ - ":semver_bzl", + ":version_bzl", "@bazel_skylib//lib:selects", "@bazel_skylib//rules:common_settings", ], @@ -249,9 +249,9 @@ bzl_library( ":python_register_toolchains_bzl", ":pythons_hub_bzl", ":repo_utils_bzl", - ":semver_bzl", ":toolchains_repo_bzl", ":util_bzl", + ":version_bzl", "@bazel_features//:features", ], ) @@ -610,11 +610,6 @@ bzl_library( ], ) -bzl_library( - name = "semver_bzl", - srcs = ["semver.bzl"], -) - bzl_library( name = "sentinel_bzl", srcs = ["sentinel.bzl"], diff --git a/python/private/config_settings.bzl b/python/private/config_settings.bzl index 5eb858e2e4..aff5d016fb 100644 --- a/python/private/config_settings.bzl +++ b/python/private/config_settings.bzl @@ -18,7 +18,7 @@ load("@bazel_skylib//lib:selects.bzl", "selects") load("@bazel_skylib//rules:common_settings.bzl", "BuildSettingInfo") load("//python/private:text_util.bzl", "render") -load(":semver.bzl", "semver") +load(":version.bzl", "version") _PYTHON_VERSION_FLAG = Label("//python/config_settings:python_version") _PYTHON_VERSION_MAJOR_MINOR_FLAG = Label("//python/config_settings:python_version_major_minor") @@ -181,8 +181,8 @@ _python_version_flag = rule( def _python_version_major_minor_flag_impl(ctx): input = _flag_value(ctx.attr._python_version_flag) if input: - version = semver(input) - value = "{}.{}".format(version.major, version.minor) + ver = version.parse(input) + value = "{}.{}".format(ver.release[0], ver.release[1]) else: value = "" diff --git a/python/private/hermetic_runtime_repo_setup.bzl b/python/private/hermetic_runtime_repo_setup.bzl index 64d721ecad..f944b0b914 100644 --- a/python/private/hermetic_runtime_repo_setup.bzl +++ b/python/private/hermetic_runtime_repo_setup.bzl @@ -20,7 +20,7 @@ load("//python:py_runtime_pair.bzl", "py_runtime_pair") load("//python/cc:py_cc_toolchain.bzl", "py_cc_toolchain") load(":glob_excludes.bzl", "glob_excludes") load(":py_exec_tools_toolchain.bzl", "py_exec_tools_toolchain") -load(":semver.bzl", "semver") +load(":version.bzl", "version") _IS_FREETHREADED = Label("//python/config_settings:is_py_freethreaded") @@ -53,8 +53,11 @@ def define_hermetic_runtime_toolchain_impl( use. """ _ = name # @unused - version_info = semver(python_version) - version_dict = version_info.to_dict() + version_info = version.parse(python_version) + version_dict = { + "major": version_info.release[0], + "minor": version_info.release[1], + } native.filegroup( name = "files", srcs = native.glob( @@ -198,9 +201,9 @@ def define_hermetic_runtime_toolchain_impl( files = [":files"], interpreter = python_bin, interpreter_version_info = { - "major": str(version_info.major), - "micro": str(version_info.patch), - "minor": str(version_info.minor), + "major": str(version_info.release[0]), + "micro": str(version_info.release[2]), + "minor": str(version_info.release[1]), }, coverage_tool = select({ # Convert empty string to None diff --git a/python/private/pypi/BUILD.bazel b/python/private/pypi/BUILD.bazel index 06ca3a8e34..84e0535289 100644 --- a/python/private/pypi/BUILD.bazel +++ b/python/private/pypi/BUILD.bazel @@ -116,7 +116,7 @@ bzl_library( ":whl_target_platforms_bzl", "//python/private:full_version_bzl", "//python/private:normalize_name_bzl", - "//python/private:semver_bzl", + "//python/private:version_bzl", "//python/private:version_label_bzl", "@bazel_features//:features", "@pythons_hub//:interpreters_bzl", @@ -256,7 +256,7 @@ bzl_library( srcs = ["pep508_evaluate.bzl"], deps = [ "//python/private:enum_bzl", - "//python/private:semver_bzl", + "//python/private:version_bzl", ], ) diff --git a/python/private/pypi/extension.bzl b/python/private/pypi/extension.bzl index 84caa0aee7..3896f2940a 100644 --- a/python/private/pypi/extension.bzl +++ b/python/private/pypi/extension.bzl @@ -22,7 +22,7 @@ load("//python/private:auth.bzl", "AUTH_ATTRS") load("//python/private:full_version.bzl", "full_version") load("//python/private:normalize_name.bzl", "normalize_name") load("//python/private:repo_utils.bzl", "repo_utils") -load("//python/private:semver.bzl", "semver") +load("//python/private:version.bzl", "version") load("//python/private:version_label.bzl", "version_label") load(":attrs.bzl", "use_isolated") load(":evaluate_markers.bzl", "evaluate_markers_py", EVALUATE_MARKERS_SRCS = "SRCS") @@ -36,9 +36,9 @@ load(":whl_config_setting.bzl", "whl_config_setting") load(":whl_library.bzl", "whl_library") load(":whl_repo_name.bzl", "pypi_repo_name", "whl_repo_name") -def _major_minor_version(version): - version = semver(version) - return "{}.{}".format(version.major, version.minor) +def _major_minor_version(version_str): + ver = version.parse(version_str) + return "{}.{}".format(ver.release[0], ver.release[1]) def _whl_mods_impl(whl_mods_dict): """Implementation of the pip.whl_mods tag class. diff --git a/python/private/python.bzl b/python/private/python.bzl index 53cd5e9cd2..c187904322 100644 --- a/python/private/python.bzl +++ b/python/private/python.bzl @@ -21,9 +21,9 @@ load(":full_version.bzl", "full_version") load(":python_register_toolchains.bzl", "python_register_toolchains") load(":pythons_hub.bzl", "hub_repo") load(":repo_utils.bzl", "repo_utils") -load(":semver.bzl", "semver") load(":toolchains_repo.bzl", "multi_toolchain_aliases") load(":util.bzl", "IS_BAZEL_6_4_OR_HIGHER") +load(":version.bzl", "version") def parse_modules(*, module_ctx, _fail = fail): """Parse the modules and return a struct for registrations. @@ -458,16 +458,20 @@ def _fail_multiple_default_toolchains(first, second): second = second, )) -def _validate_version(*, version, _fail = fail): - parsed = semver(version) - if parsed.patch == None or parsed.build or parsed.pre_release: - _fail("The 'python_version' attribute needs to specify an 'X.Y.Z' semver-compatible version, got: '{}'".format(version)) +def _validate_version(version_str, *, _fail = fail): + v = version.parse(version_str, strict = True, _fail = _fail) + if v == None: + # Only reachable in tests + return False + + if len(v.release) < 3: + _fail("The 'python_version' attribute needs to specify the full version in at least 'X.Y.Z' format, got: '{}'".format(v.string)) return False return True def _process_single_version_overrides(*, tag, _fail = fail, default): - if not _validate_version(version = tag.python_version, _fail = _fail): + if not _validate_version(tag.python_version, _fail = _fail): return available_versions = default["tool_versions"] @@ -517,7 +521,7 @@ def _process_single_version_overrides(*, tag, _fail = fail, default): kwargs.setdefault(tag.python_version, {})["distutils"] = tag.distutils def _process_single_version_platform_overrides(*, tag, _fail = fail, default): - if not _validate_version(version = tag.python_version, _fail = _fail): + if not _validate_version(tag.python_version, _fail = _fail): return available_versions = default["tool_versions"] @@ -558,12 +562,12 @@ def _process_global_overrides(*, tag, default, _fail = fail): if tag.minor_mapping: for minor_version, full_version in tag.minor_mapping.items(): - parsed = semver(minor_version) - if parsed.patch != None or parsed.build or parsed.pre_release: - fail("Expected the key to be of `X.Y` format but got `{}`".format(minor_version)) - parsed = semver(full_version) - if parsed.patch == None: - fail("Expected the value to at least be of `X.Y.Z` format but got `{}`".format(minor_version)) + parsed = version.parse(minor_version, strict = True, _fail = _fail) + if len(parsed.release) > 2 or parsed.pre or parsed.post or parsed.dev or parsed.local: + fail("Expected the key to be of `X.Y` format but got `{}`".format(parsed.string)) + + # Ensure that the version is valid + version.parse(full_version, strict = True, _fail = _fail) default["minor_mapping"] = tag.minor_mapping @@ -651,8 +655,11 @@ def _get_toolchain_config(*, modules, _fail = fail): versions = {} for version_string in available_versions: - v = semver(version_string) - versions.setdefault("{}.{}".format(v.major, v.minor), []).append((int(v.patch), version_string)) + v = version.parse(version_string, strict = True) + versions.setdefault( + "{}.{}".format(v.release[0], v.release[1]), + [], + ).append((version.key(v), v.string)) minor_mapping = { major_minor: max(subset)[1] diff --git a/python/private/semver.bzl b/python/private/semver.bzl deleted file mode 100644 index 0cbd172348..0000000000 --- a/python/private/semver.bzl +++ /dev/null @@ -1,85 +0,0 @@ -# Copyright 2024 The Bazel Authors. All rights reserved. -# -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -"A semver version parser" - -def _key(version): - return ( - version.major, - version.minor or 0, - version.patch or 0, - # non pre-release versions are higher - version.pre_release == "", - # then we compare each element of the pre_release tag separately - tuple([ - ( - i if not i.isdigit() else "", - # digit values take precedence - int(i) if i.isdigit() else 0, - ) - for i in version.pre_release.split(".") - ]) if version.pre_release else None, - # And build info is just alphabetic - version.build, - ) - -def _to_dict(self): - return { - "build": self.build, - "major": self.major, - "minor": self.minor, - "patch": self.patch, - "pre_release": self.pre_release, - } - -def _new(*, major, minor, patch, pre_release, build, version = None): - # buildifier: disable=uninitialized - self = struct( - major = int(major), - minor = None if minor == None else int(minor), - # NOTE: this is called `micro` in the Python interpreter versioning scheme - patch = None if patch == None else int(patch), - pre_release = pre_release, - build = build, - # buildifier: disable=uninitialized - key = lambda: _key(self), - str = lambda: version, - to_dict = lambda: _to_dict(self), - ) - return self - -def semver(version): - """Parse the semver version and return the values as a struct. - - Args: - version: {type}`str` the version string. - - Returns: - A {type}`struct` with `major`, `minor`, `patch` and `build` attributes. - """ - - # Implement the https://semver.org/ spec - major, _, tail = version.partition(".") - minor, _, tail = tail.partition(".") - patch, _, build = tail.partition("+") - patch, _, pre_release = patch.partition("-") - - return _new( - major = int(major), - minor = int(minor) if minor.isdigit() else None, - patch = int(patch) if patch.isdigit() else None, - build = build, - pre_release = pre_release, - version = version, - ) diff --git a/python/private/version.bzl b/python/private/version.bzl index 4425cc7661..f98165d391 100644 --- a/python/private/version.bzl +++ b/python/private/version.bzl @@ -510,7 +510,7 @@ def normalize_pep440(version): """ return _parse(version, strict = True)["norm"] -def _parse(version_str, strict = True): +def _parse(version_str, strict = True, _fail = fail): """Escape the version component of a filename. See https://packaging.python.org/en/latest/specifications/binary-distribution-format/#escaping-and-unicode @@ -519,6 +519,7 @@ def _parse(version_str, strict = True): Args: version_str: version string to be normalized according to PEP 440. strict: fail if the version is invalid, defaults to True. + _fail: Used for tests Returns: string containing the normalized version. @@ -544,7 +545,7 @@ def _parse(version_str, strict = True): parser_ctx = parser.context() if parser.input[parser_ctx["start"]:]: if strict: - fail( + _fail( "Failed to parse PEP 440 version identifier '%s'." % parser.input, "Parse error at '%s'" % parser.input[parser_ctx["start"]:], ) @@ -554,7 +555,7 @@ def _parse(version_str, strict = True): parser_ctx["is_prefix"] = is_prefix return parser_ctx -def parse(version_str, strict = False): +def parse(version_str, strict = False, _fail = fail): """Parse a PEP4408 compliant version. This is similar to `normalize_pep440`, but it parses individual components to @@ -563,6 +564,7 @@ def parse(version_str, strict = False): Args: version_str: version string to be normalized according to PEP 440. strict: fail if the version is invalid. + _fail: used for tests Returns: a struct with individual components of a version: @@ -580,29 +582,29 @@ def parse(version_str, strict = False): * `string` {type}`str` normalized value of the input. """ - parts = _parse(version_str, strict = strict) + parts = _parse(version_str, strict = strict, _fail = _fail) if not parts: return None if parts["is_prefix"] and (parts["local"] or parts["post"] or parts["dev"] or parts["pre"]): if strict: - fail("local version part has been obtained, but only public segments can have prefix matches") + _fail("local version part has been obtained, but only public segments can have prefix matches") # https://peps.python.org/pep-0440/#public-version-identifiers return None return struct( - epoch = _parse_epoch(parts["epoch"]), + epoch = _parse_epoch(parts["epoch"], _fail), release = _parse_release(parts["release"]), pre = _parse_pre(parts["pre"]), - post = _parse_post(parts["post"]), - dev = _parse_dev(parts["dev"]), - local = _parse_local(parts["local"]), + post = _parse_post(parts["post"], _fail), + dev = _parse_dev(parts["dev"], _fail), + local = _parse_local(parts["local"], _fail), string = parts["norm"], is_prefix = parts["is_prefix"], ) -def _parse_epoch(value): +def _parse_epoch(value, fail): if not value: return 0 @@ -614,7 +616,7 @@ def _parse_epoch(value): def _parse_release(value): return tuple([int(d) for d in value.split(".")]) -def _parse_local(value): +def _parse_local(value, fail): if not value: return None @@ -624,7 +626,7 @@ def _parse_local(value): # If the part is numerical, handle it as a number return tuple([int(part) if part.isdigit() else part for part in value[1:].split(".")]) -def _parse_dev(value): +def _parse_dev(value, fail): if not value: return None @@ -646,7 +648,7 @@ def _parse_pre(value): return (prefix, int(value[len(prefix):])) -def _parse_post(value): +def _parse_post(value, fail): if not value: return None diff --git a/tests/python/python_tests.bzl b/tests/python/python_tests.bzl index 97c47b57db..443174c966 100644 --- a/tests/python/python_tests.bzl +++ b/tests/python/python_tests.bzl @@ -746,12 +746,6 @@ def _test_single_version_override_errors(env): ], want_error = "Only a single 'python.single_version_override' can be present for '3.12.4'", ), - struct( - overrides = [ - _single_version_override(python_version = "3.12.4+3", distutils_content = "foo"), - ], - want_error = "The 'python_version' attribute needs to specify an 'X.Y.Z' semver-compatible version, got: '3.12.4+3'", - ), ]: errors = [] parse_modules( @@ -781,13 +775,13 @@ def _test_single_version_platform_override_errors(env): overrides = [ _single_version_platform_override(python_version = "3.12", platform = "foo"), ], - want_error = "The 'python_version' attribute needs to specify an 'X.Y.Z' semver-compatible version, got: '3.12'", + want_error = "The 'python_version' attribute needs to specify the full version in at least 'X.Y.Z' format, got: '3.12'", ), struct( overrides = [ - _single_version_platform_override(python_version = "3.12.1+my_build", platform = "foo"), + _single_version_platform_override(python_version = "foo", platform = "foo"), ], - want_error = "The 'python_version' attribute needs to specify an 'X.Y.Z' semver-compatible version, got: '3.12.1+my_build'", + want_error = "Failed to parse PEP 440 version identifier 'foo'. Parse error at 'foo'", ), ]: errors = [] @@ -799,7 +793,7 @@ def _test_single_version_platform_override_errors(env): single_version_platform_override = test.overrides, ), ), - _fail = errors.append, + _fail = lambda *a: errors.append(" ".join(a)), ) env.expect.that_collection(errors).contains_exactly([test.want_error]) diff --git a/tests/semver/BUILD.bazel b/tests/semver/BUILD.bazel deleted file mode 100644 index e12b1e5300..0000000000 --- a/tests/semver/BUILD.bazel +++ /dev/null @@ -1,17 +0,0 @@ -# Copyright 2024 The Bazel Authors. All rights reserved. -# -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -load(":semver_test.bzl", "semver_test_suite") - -semver_test_suite(name = "semver_tests") diff --git a/tests/semver/semver_test.bzl b/tests/semver/semver_test.bzl deleted file mode 100644 index 9d13402c92..0000000000 --- a/tests/semver/semver_test.bzl +++ /dev/null @@ -1,113 +0,0 @@ -# Copyright 2023 The Bazel Authors. All rights reserved. -# -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -"" - -load("@rules_testing//lib:test_suite.bzl", "test_suite") -load("//python/private:semver.bzl", "semver") # buildifier: disable=bzl-visibility - -_tests = [] - -def _test_semver_from_major(env): - actual = semver("3") - env.expect.that_int(actual.major).equals(3) - env.expect.that_int(actual.minor).equals(None) - env.expect.that_int(actual.patch).equals(None) - env.expect.that_str(actual.build).equals("") - -_tests.append(_test_semver_from_major) - -def _test_semver_from_major_minor_version(env): - actual = semver("4.9") - env.expect.that_int(actual.major).equals(4) - env.expect.that_int(actual.minor).equals(9) - env.expect.that_int(actual.patch).equals(None) - env.expect.that_str(actual.build).equals("") - -_tests.append(_test_semver_from_major_minor_version) - -def _test_semver_with_build_info(env): - actual = semver("1.2.3+mybuild") - env.expect.that_int(actual.major).equals(1) - env.expect.that_int(actual.minor).equals(2) - env.expect.that_int(actual.patch).equals(3) - env.expect.that_str(actual.build).equals("mybuild") - -_tests.append(_test_semver_with_build_info) - -def _test_semver_with_build_info_multiple_pluses(env): - actual = semver("1.2.3-rc0+build+info") - env.expect.that_int(actual.major).equals(1) - env.expect.that_int(actual.minor).equals(2) - env.expect.that_int(actual.patch).equals(3) - env.expect.that_str(actual.pre_release).equals("rc0") - env.expect.that_str(actual.build).equals("build+info") - -_tests.append(_test_semver_with_build_info_multiple_pluses) - -def _test_semver_alpha_beta(env): - actual = semver("1.2.3-alpha.beta") - env.expect.that_int(actual.major).equals(1) - env.expect.that_int(actual.minor).equals(2) - env.expect.that_int(actual.patch).equals(3) - env.expect.that_str(actual.pre_release).equals("alpha.beta") - -_tests.append(_test_semver_alpha_beta) - -def _test_semver_sort(env): - want = [ - semver(item) - for item in [ - # The items are sorted from lowest to highest version - "0.0.1", - "0.1.0-rc", - "0.1.0", - "0.9.11", - "0.9.12", - "1.0.0-alpha", - "1.0.0-alpha.1", - "1.0.0-alpha.beta", - "1.0.0-beta", - "1.0.0-beta.2", - "1.0.0-beta.11", - "1.0.0-rc.1", - "1.0.0-rc.2", - "1.0.0", - # Also handle missing minor and patch version strings - "2.0", - "3", - # Alphabetic comparison for different builds - "3.0.0+build0", - "3.0.0+build1", - ] - ] - actual = sorted(want, key = lambda x: x.key()) - env.expect.that_collection(actual).contains_exactly(want).in_order() - for i, greater in enumerate(want[1:]): - smaller = actual[i] - if greater.key() <= smaller.key(): - env.fail("Expected '{}' to be smaller than '{}', but got otherwise".format( - smaller.str(), - greater.str(), - )) - -_tests.append(_test_semver_sort) - -def semver_test_suite(name): - """Create the test suite. - - Args: - name: the name of the test suite - """ - test_suite(name = name, basic_tests = _tests) From ea4714d33adb82c68739bdfd74038faa4d60b061 Mon Sep 17 00:00:00 2001 From: Philipp Schrader Date: Thu, 15 May 2025 19:11:34 -0700 Subject: [PATCH 221/922] feat: Add support for REPLs (#2723) This patch adds a new target that lets users invoke a REPL for a given `PyInfo` target. For example, the following command will spawn a REPL for any target that provides `PyInfo`: ```console $ bazel run --//python/config_settings:bootstrap_impl=script //python/bin:repl --//python/bin:repl_dep=//tools:wheelmaker Python 3.11.1 (main, Jan 16 2023, 22:41:20) [Clang 15.0.7 ] on linux Type "help", "copyright", "credits" or "license" for more information. (InteractiveConsole) >>> import tools.wheelmaker >>> ``` If the user wants an IPython shell instead, they can create a file like this: ```python import IPython IPython.start_ipython() ``` Then they can set this up in their `.bazelrc` file: ``` # Allow the REPL stub to import ipython. In this case, @my_deps is the name # of the pip.parse() repository. build --@rules_python//python/bin:repl_stub_dep=@my_deps//ipython # Point the REPL at the stub created above. build --@rules_python//python/bin:repl_stub=//path/to:ipython_stub.py ``` --------- Co-authored-by: Ignas Anikevicius <240938+aignas@users.noreply.github.com> --- CHANGELOG.md | 2 + docs/index.md | 1 + docs/repl.md | 66 +++++++++++++++++++++++++ docs/toolchains.md | 10 ++++ python/bin/BUILD.bazel | 33 +++++++++++++ python/bin/repl_stub.py | 29 +++++++++++ python/private/BUILD.bazel | 4 ++ python/private/repl.bzl | 84 ++++++++++++++++++++++++++++++++ python/private/repl_template.py | 37 ++++++++++++++ tests/repl/BUILD.bazel | 44 +++++++++++++++++ tests/repl/helper/test_module.py | 5 ++ tests/repl/repl_test.py | 74 ++++++++++++++++++++++++++++ tests/support/sh_py_run_test.bzl | 4 ++ 13 files changed, 393 insertions(+) create mode 100644 docs/repl.md create mode 100644 python/bin/repl_stub.py create mode 100644 python/private/repl.bzl create mode 100644 python/private/repl_template.py create mode 100644 tests/repl/BUILD.bazel create mode 100644 tests/repl/helper/test_module.py create mode 100644 tests/repl/repl_test.py diff --git a/CHANGELOG.md b/CHANGELOG.md index 94487219bc..a6ba65eb2f 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -102,6 +102,8 @@ END_UNRELEASED_TEMPLATE * (pypi) Starlark-based evaluation of environment markers (requirements.txt conditionals) available (not enabled by default) for improved multi-platform build support. Set the `RULES_PYTHON_ENABLE_PIPSTAR=1` environment variable to enable it. +* (utils) Add a way to run a REPL for any `rules_python` target that returns + a `PyInfo` provider. {#v0-0-0-removed} ### Removed diff --git a/docs/index.md b/docs/index.md index b10b445983..285b1cd66e 100644 --- a/docs/index.md +++ b/docs/index.md @@ -101,6 +101,7 @@ pip coverage precompiling gazelle +REPL Extending Contributing support diff --git a/docs/repl.md b/docs/repl.md new file mode 100644 index 0000000000..edcf37e811 --- /dev/null +++ b/docs/repl.md @@ -0,0 +1,66 @@ +# Getting a REPL or Interactive Shell + +rules_python provides a REPL to help with debugging and developing. The goal of +the REPL is to present an environment identical to what a {bzl:obj}`py_binary` creates +for your code. + +## Usage + +Start the REPL with the following command: +```console +$ bazel run @rules_python//python/bin:repl +Python 3.11.11 (main, Mar 17 2025, 21:02:09) [Clang 20.1.0 ] on linux +Type "help", "copyright", "credits" or "license" for more information. +>>> +``` + +Settings like `//python/config_settings:python_version` will influence the exact +behaviour. +```console +$ bazel run @rules_python//python/bin:repl --@rules_python//python/config_settings:python_version=3.13 +Python 3.13.2 (main, Mar 17 2025, 21:02:54) [Clang 20.1.0 ] on linux +Type "help", "copyright", "credits" or "license" for more information. +>>> +``` + +See [//python/config_settings](api/rules_python/python/config_settings/index) +and [Environment Variables](environment-variables) for more settings. + +## Importing Python targets + +The `//python/bin:repl_dep` command line flag gives the REPL access to a target +that provides the {bzl:obj}`PyInfo` provider. + +```console +$ bazel run @rules_python//python/bin:repl --@rules_python//python/bin:repl_dep=@rules_python//tools:wheelmaker +Python 3.11.11 (main, Mar 17 2025, 21:02:09) [Clang 20.1.0 ] on linux +Type "help", "copyright", "credits" or "license" for more information. +>>> import tools.wheelmaker +>>> +``` + +## Customizing the shell + +By default, the `//python/bin:repl` target will invoke the shell from the `code` +module. It's possible to switch to another shell by writing a custom "stub" and +pointing the target at the necessary dependencies. + +### IPython Example + +For an IPython shell, create a file as follows. + +```python +import IPython +IPython.start_ipython() +``` + +Assuming the file is called `ipython_stub.py` and the `pip.parse` hub's name is +`my_deps`, set this up in the .bazelrc file: +``` +# Allow the REPL stub to import ipython. In this case, @my_deps is the hub name +# of the pip.parse() call. +build --@rules_python//python/bin:repl_stub_dep=@my_deps//ipython + +# Point the REPL at the stub created above. +build --@rules_python//python/bin:repl_stub=//path/to:ipython_stub.py +``` diff --git a/docs/toolchains.md b/docs/toolchains.md index c8305e8f0d..121b398f7a 100644 --- a/docs/toolchains.md +++ b/docs/toolchains.md @@ -757,3 +757,13 @@ a fixed version. The `python` target does not provide access to any modules from `py_*` targets on its own. Please file a feature request if this is desired. ::: + +### Differences from `//python/bin:repl` + +The `//python/bin:python` target provides access to the underlying interpreter +without any hermeticity guarantees. + +The [`//python/bin:repl` target](repl) provides an environment indentical to +what `py_binary` provides. That means it handles things like the +[`PYTHONSAFEPATH`](https://docs.python.org/3/using/cmdline.html#envvar-PYTHONSAFEPATH) +environment variable automatically. The `//python/bin:python` target will not. diff --git a/python/bin/BUILD.bazel b/python/bin/BUILD.bazel index 57bee34378..30af7d1b9f 100644 --- a/python/bin/BUILD.bazel +++ b/python/bin/BUILD.bazel @@ -1,4 +1,5 @@ load("//python/private:interpreter.bzl", _interpreter_binary = "interpreter_binary") +load("//python/private:repl.bzl", "py_repl_binary") filegroup( name = "distribution", @@ -22,3 +23,35 @@ label_flag( name = "python_src", build_setting_default = "//python:none", ) + +py_repl_binary( + name = "repl", + stub = ":repl_stub", + visibility = ["//visibility:public"], + deps = [ + ":repl_dep", + ":repl_stub_dep", + ], +) + +# The user can replace this with their own stub. E.g. they can use this to +# import ipython instead of the default shell. +label_flag( + name = "repl_stub", + build_setting_default = "repl_stub.py", +) + +# The user can modify this flag to make an interpreter shell library available +# for the stub. E.g. if they switch the stub for an ipython-based one, then they +# can point this at their version of ipython. +label_flag( + name = "repl_stub_dep", + build_setting_default = "//python/private:empty", +) + +# The user can modify this flag to make arbitrary PyInfo targets available for +# import on the REPL. +label_flag( + name = "repl_dep", + build_setting_default = "//python/private:empty", +) diff --git a/python/bin/repl_stub.py b/python/bin/repl_stub.py new file mode 100644 index 0000000000..86452aa869 --- /dev/null +++ b/python/bin/repl_stub.py @@ -0,0 +1,29 @@ +"""Simulates the REPL that Python spawns when invoking the binary with no arguments. + +The code module is responsible for the default shell. + +The import and `ocde.interact()` call here his is equivalent to doing: + + $ python3 -m code + Python 3.11.2 (main, Mar 13 2023, 12:18:29) [GCC 12.2.0] on linux + Type "help", "copyright", "credits" or "license" for more information. + (InteractiveConsole) + >>> + +The logic for PYTHONSTARTUP is handled in python/private/repl_template.py. +""" + +import code +import sys + +if sys.stdin.isatty(): + # Use the default options. + exitmsg = None +else: + # On a non-interactive console, we want to suppress the >>> and the exit message. + exitmsg = "" + sys.ps1 = "" + sys.ps2 = "" + +# We set the banner to an empty string because the repl_template.py file already prints the banner. +code.interact(banner="", exitmsg=exitmsg) diff --git a/python/private/BUILD.bazel b/python/private/BUILD.bazel index 0b50ccf0b7..ce22421300 100644 --- a/python/private/BUILD.bazel +++ b/python/private/BUILD.bazel @@ -817,6 +817,10 @@ current_interpreter_executable( visibility = ["//visibility:public"], ) +py_library( + name = "empty", +) + sentinel( name = "sentinel", ) diff --git a/python/private/repl.bzl b/python/private/repl.bzl new file mode 100644 index 0000000000..838166a187 --- /dev/null +++ b/python/private/repl.bzl @@ -0,0 +1,84 @@ +"""Implementation of the rules to expose a REPL.""" + +load("//python:py_binary.bzl", _py_binary = "py_binary") + +def _generate_repl_main_impl(ctx): + stub_repo = ctx.attr.stub.label.repo_name or ctx.workspace_name + stub_path = "/".join([stub_repo, ctx.file.stub.short_path]) + + out = ctx.actions.declare_file(ctx.label.name + ".py") + + # Point the generated main file at the stub. + ctx.actions.expand_template( + template = ctx.file._template, + output = out, + substitutions = { + "%stub_path%": stub_path, + }, + ) + + return [DefaultInfo(files = depset([out]))] + +_generate_repl_main = rule( + implementation = _generate_repl_main_impl, + attrs = { + "stub": attr.label( + mandatory = True, + allow_single_file = True, + doc = ("The stub responsible for actually invoking the final shell. " + + "See the \"Customizing the REPL\" docs for details."), + ), + "_template": attr.label( + default = "//python/private:repl_template.py", + allow_single_file = True, + doc = "The template to use for generating `out`.", + ), + }, + doc = """\ +Generates a "main" script for a py_binary target that starts a Python REPL. + +The template is designed to take care of the majority of the logic. The user +customizes the exact shell that will be started via the stub. The stub is a +simple shell script that imports the desired shell and then executes it. + +The target's name is used for the output filename (with a .py extension). +""", +) + +def py_repl_binary(name, stub, deps = [], data = [], **kwargs): + """A py_binary target that executes a REPL when run. + + The stub is the script that ultimately decides which shell the REPL will run. + It can be as simple as this: + + import code + code.interact() + + Or it can load something like IPython instead. + + Args: + name: Name of the generated py_binary target. + stub: The script that invokes the shell. + deps: The dependencies of the py_binary. + data: The runtime dependencies of the py_binary. + **kwargs: Forwarded to the py_binary. + """ + _generate_repl_main( + name = "%s_py" % name, + stub = stub, + ) + + _py_binary( + name = name, + srcs = [ + ":%s_py" % name, + ], + main = "%s_py.py" % name, + data = data + [ + stub, + ], + deps = deps + [ + "//python/runfiles", + ], + **kwargs + ) diff --git a/python/private/repl_template.py b/python/private/repl_template.py new file mode 100644 index 0000000000..0e058b23ae --- /dev/null +++ b/python/private/repl_template.py @@ -0,0 +1,37 @@ +import os +import runpy +import sys +from pathlib import Path + +from python.runfiles import runfiles + +STUB_PATH = "%stub_path%" + + +def start_repl(): + if sys.stdin.isatty(): + # Print the banner similar to how python does it on startup when running interactively. + cprt = 'Type "help", "copyright", "credits" or "license" for more information.' + sys.stderr.write("Python %s on %s\n%s\n" % (sys.version, sys.platform, cprt)) + + # Simulate Python's behavior when a valid startup script is defined by the + # PYTHONSTARTUP variable. If this file path fails to load, print the error + # and revert to the default behavior. + # + # See upstream for more information: + # https://docs.python.org/3/using/cmdline.html#envvar-PYTHONSTARTUP + if startup_file := os.getenv("PYTHONSTARTUP"): + try: + source_code = Path(startup_file).read_text() + except Exception as error: + print(f"{type(error).__name__}: {error}") + else: + compiled_code = compile(source_code, filename=startup_file, mode="exec") + eval(compiled_code, {}) + + bazel_runfiles = runfiles.Create() + runpy.run_path(bazel_runfiles.Rlocation(STUB_PATH), run_name="__main__") + + +if __name__ == "__main__": + start_repl() diff --git a/tests/repl/BUILD.bazel b/tests/repl/BUILD.bazel new file mode 100644 index 0000000000..62c7377d53 --- /dev/null +++ b/tests/repl/BUILD.bazel @@ -0,0 +1,44 @@ +load("//python:py_library.bzl", "py_library") +load("//tests/support:sh_py_run_test.bzl", "py_reconfig_test") + +# A library that adds a special import path only when this is specified as a +# dependency. This makes it easy for a dependency to have this import path +# available without the top-level target being able to import the module. +py_library( + name = "helper/test_module", + srcs = [ + "helper/test_module.py", + ], + imports = [ + "helper", + ], +) + +py_reconfig_test( + name = "repl_without_dep_test", + srcs = ["repl_test.py"], + data = [ + "//python/bin:repl", + ], + env = { + # The helper/test_module should _not_ be importable for this test. + "EXPECT_TEST_MODULE_IMPORTABLE": "0", + }, + main = "repl_test.py", + python_version = "3.12", +) + +py_reconfig_test( + name = "repl_with_dep_test", + srcs = ["repl_test.py"], + data = [ + "//python/bin:repl", + ], + env = { + # The helper/test_module _should_ be importable for this test. + "EXPECT_TEST_MODULE_IMPORTABLE": "1", + }, + main = "repl_test.py", + python_version = "3.12", + repl_dep = ":helper/test_module", +) diff --git a/tests/repl/helper/test_module.py b/tests/repl/helper/test_module.py new file mode 100644 index 0000000000..0c4a309b01 --- /dev/null +++ b/tests/repl/helper/test_module.py @@ -0,0 +1,5 @@ +"""This is a file purely intended for validating //python/bin:repl.""" + + +def print_hello(): + print("Hello World") diff --git a/tests/repl/repl_test.py b/tests/repl/repl_test.py new file mode 100644 index 0000000000..51ca951110 --- /dev/null +++ b/tests/repl/repl_test.py @@ -0,0 +1,74 @@ +import os +import subprocess +import sys +import unittest +from typing import Iterable + +from python import runfiles + +rfiles = runfiles.Create() + +# Signals the tests below whether we should be expecting the import of +# helpers/test_module.py on the REPL to work or not. +EXPECT_TEST_MODULE_IMPORTABLE = os.environ["EXPECT_TEST_MODULE_IMPORTABLE"] == "1" + + +class ReplTest(unittest.TestCase): + def setUp(self): + self.repl = rfiles.Rlocation("rules_python/python/bin/repl") + assert self.repl + + def run_code_in_repl(self, lines: Iterable[str]) -> str: + """Runs the lines of code in the REPL and returns the text output.""" + return subprocess.check_output( + [self.repl], + text=True, + stderr=subprocess.STDOUT, + input="\n".join(lines), + ).strip() + + def test_repl_version(self): + """Validates that we can successfully execute arbitrary code on the REPL.""" + + result = self.run_code_in_repl( + [ + "import sys", + "v = sys.version_info", + "print(f'version: {v.major}.{v.minor}')", + ] + ) + self.assertIn("version: 3.12", result) + + def test_cannot_import_test_module_directly(self): + """Validates that we cannot import helper/test_module.py since it's not a direct dep.""" + with self.assertRaises(ModuleNotFoundError): + import test_module + + @unittest.skipIf( + not EXPECT_TEST_MODULE_IMPORTABLE, "test only works without repl_dep set" + ) + def test_import_test_module_success(self): + """Validates that we can import helper/test_module.py when repl_dep is set.""" + result = self.run_code_in_repl( + [ + "import test_module", + "test_module.print_hello()", + ] + ) + self.assertIn("Hello World", result) + + @unittest.skipIf( + EXPECT_TEST_MODULE_IMPORTABLE, "test only works without repl_dep set" + ) + def test_import_test_module_failure(self): + """Validates that we cannot import helper/test_module.py when repl_dep isn't set.""" + result = self.run_code_in_repl( + [ + "import test_module", + ] + ) + self.assertIn("ModuleNotFoundError: No module named 'test_module'", result) + + +if __name__ == "__main__": + unittest.main() diff --git a/tests/support/sh_py_run_test.bzl b/tests/support/sh_py_run_test.bzl index 9c8134ff40..f6ebc506cc 100644 --- a/tests/support/sh_py_run_test.bzl +++ b/tests/support/sh_py_run_test.bzl @@ -38,6 +38,8 @@ def _perform_transition_impl(input_settings, attr, base_impl): settings["//command_line_option:extra_toolchains"] = attr.extra_toolchains if attr.python_src: settings["//python/bin:python_src"] = attr.python_src + if attr.repl_dep: + settings["//python/bin:repl_dep"] = attr.repl_dep if attr.venvs_use_declare_symlink: settings["//python/config_settings:venvs_use_declare_symlink"] = attr.venvs_use_declare_symlink if attr.venvs_site_packages: @@ -47,6 +49,7 @@ def _perform_transition_impl(input_settings, attr, base_impl): _RECONFIG_INPUTS = [ "//python/config_settings:bootstrap_impl", "//python/bin:python_src", + "//python/bin:repl_dep", "//command_line_option:extra_toolchains", "//python/config_settings:venvs_use_declare_symlink", "//python/config_settings:venvs_site_packages", @@ -70,6 +73,7 @@ toolchain. """, ), "python_src": attrb.Label(), + "repl_dep": attrb.Label(), "venvs_site_packages": attrb.String(), "venvs_use_declare_symlink": attrb.String(), } From ce50f6a05c85cb93fd9de6f2863d6131fb7609c4 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Sat, 17 May 2025 04:54:07 -0700 Subject: [PATCH 222/922] cleanup: remove unused sanitize_platform_name function (#2887) The sanitize_platform_name function is unused, so remove it. --- python/private/toolchains_repo.bzl | 3 --- 1 file changed, 3 deletions(-) diff --git a/python/private/toolchains_repo.bzl b/python/private/toolchains_repo.bzl index d0814b66d5..7557c9f7d0 100644 --- a/python/private/toolchains_repo.bzl +++ b/python/private/toolchains_repo.bzl @@ -404,9 +404,6 @@ multi_toolchain_aliases = repository_rule( }, ) -def sanitize_platform_name(platform): - return platform.replace("-", "_") - def _get_host_platform(*, rctx, logger, python_version, os_name, cpu_name, platforms): """Gets the host platform. From 9ad9ce5ce0d5b2cd7fbde1d3c5b2a241056d0bbf Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Sat, 17 May 2025 04:55:39 -0700 Subject: [PATCH 223/922] refactor: move inline code strings to top-level constants (#2886) This moves all the inline triple-quote strings of generated code to be in top-level constants. Because they're so large and dedented, they look like regular code. This makes it hard to visually parse. These large inline strings of code also confuse my editor's syntax highlighting (it does local, partial, parsing to highlight tokens, which gets thrown off by the seemingly valid looking code). --- python/private/toolchains_repo.bzl | 277 +++++++++++++++-------------- 1 file changed, 147 insertions(+), 130 deletions(-) diff --git a/python/private/toolchains_repo.bzl b/python/private/toolchains_repo.bzl index 7557c9f7d0..d00f5ae34d 100644 --- a/python/private/toolchains_repo.bzl +++ b/python/private/toolchains_repo.bzl @@ -43,6 +43,144 @@ py_toolchain_suite( ) """.lstrip() +_WORKSPACE_TOOLCHAINS_BUILD_TEMPLATE = """ +# Generated by python/private/toolchains_repo.bzl +# +# These can be registered in the workspace file or passed to --extra_toolchains +# flag. By default all these toolchains are registered by the +# python_register_toolchains macro so you don't normally need to interact with +# these targets. + +load("@@{rules_python}//python/private:py_toolchain_suite.bzl", "py_toolchain_suite") + +""".lstrip() + +_TOOLCHAIN_ALIASES_BUILD_TEMPLATE = """ +# Generated by python/private/toolchains_repo.bzl +load("@rules_python//python/private:toolchain_aliases.bzl", "toolchain_aliases") + +package(default_visibility = ["//visibility:public"]) + +exports_files(["defs.bzl"]) + +PLATFORMS = [ +{loaded_platforms} +] +toolchain_aliases( + name = "{py_repository}", + platforms = PLATFORMS, +) +""".lstrip() + +_TOOLCHAIN_ALIASES_DEFS_TEMPLATE = """ +# Generated by python/private/toolchains_repo.bzl + +load("@@{rules_python}//python:pip.bzl", _compile_pip_requirements = "compile_pip_requirements") +load("@@{rules_python}//python/private:deprecation.bzl", "with_deprecation") +load("@@{rules_python}//python/private:text_util.bzl", "render") +load("@@{rules_python}//python:py_binary.bzl", _py_binary = "py_binary") +load("@@{rules_python}//python:py_test.bzl", _py_test = "py_test") +load( + "@@{rules_python}//python/entry_points:py_console_script_binary.bzl", + _py_console_script_binary = "py_console_script_binary", +) + +def _with_deprecation(kwargs, *, name): + kwargs["python_version"] = "{python_version}" + return with_deprecation.symbol( + kwargs, + symbol_name = name, + old_load = "@{name}//:defs.bzl", + new_load = "@rules_python//python:{{}}.bzl".format(name), + snippet = render.call(name, **{{k: repr(v) for k,v in kwargs.items()}}) + ) + +def py_binary(**kwargs): + return _py_binary(**_with_deprecation(kwargs, name = "py_binary")) + +def py_console_script_binary(**kwargs): + return _py_console_script_binary(**_with_deprecation(kwargs, name = "py_console_script_binary")) + +def py_test(**kwargs): + return _py_test(**_with_deprecation(kwargs, name = "py_test")) + +def compile_pip_requirements(**kwargs): + return _compile_pip_requirements(**_with_deprecation(kwargs, name = "compile_pip_requirements")) +""".lstrip() + +_HOST_TOOLCHAIN_BUILD_CONTENT = """ +# Generated by python/private/toolchains_repo.bzl + +exports_files(["python"], visibility = ["//visibility:public"]) +""".lstrip() + +_HOST_PYTHON_TESTER_TEMPLATE = """ +from pathlib import Path +import sys + +python = Path(sys.executable) +want_python = str(Path("{python}").resolve()) +got_python = str(Path(sys.executable).resolve()) + +assert want_python == got_python, \ + "Expected to use a different interpreter:\\nwant: '{{}}'\\n got: '{{}}'".format( + want_python, + got_python, + ) +""".lstrip() + +_MULTI_TOOLCHAIN_ALIASES_DEFS_TEMPLATE = """ +# Generated by python/private/toolchains_repo.bzl + +load("@@{rules_python}//python:pip.bzl", _compile_pip_requirements = "compile_pip_requirements") +load("@@{rules_python}//python/private:deprecation.bzl", "with_deprecation") +load("@@{rules_python}//python/private:text_util.bzl", "render") +load("@@{rules_python}//python:py_binary.bzl", _py_binary = "py_binary") +load("@@{rules_python}//python:py_test.bzl", _py_test = "py_test") +load( + "@@{rules_python}//python/entry_points:py_console_script_binary.bzl", + _py_console_script_binary = "py_console_script_binary", +) + +def _with_deprecation(kwargs, *, name): + kwargs["python_version"] = "{python_version}" + return with_deprecation.symbol( + kwargs, + symbol_name = name, + old_load = "@{name}//{python_version}:defs.bzl", + new_load = "@rules_python//python:{{}}.bzl".format(name), + snippet = render.call(name, **{{k: repr(v) for k,v in kwargs.items()}}) + ) + +def py_binary(**kwargs): + return _py_binary(**_with_deprecation(kwargs, name = "py_binary")) + +def py_console_script_binary(**kwargs): + return _py_console_script_binary(**_with_deprecation(kwargs, name = "py_console_script_binary")) + +def py_test(**kwargs): + return _py_test(**_with_deprecation(kwargs, name = "py_test")) + +def compile_pip_requirements(**kwargs): + return _compile_pip_requirements(**_with_deprecation(kwargs, name = "compile_pip_requirements")) +""".lstrip() + +_MULTI_TOOLCHAIN_ALIASES_PIP_TEMPLATE = """ +# Generated by python/private/toolchains_repo.bzl + +load("@@{rules_python}//python:pip.bzl", "pip_parse", _multi_pip_parse = "multi_pip_parse") + +def multi_pip_parse(name, requirements_lock, **kwargs): + return _multi_pip_parse( + name = name, + python_versions = {python_versions}, + requirements_lock = requirements_lock, + minor_mapping = {minor_mapping}, + **kwargs + ) + +""".lstrip() + def python_toolchain_build_file_content( prefix, python_version, @@ -101,17 +239,7 @@ def toolchain_suite_content( ) def _toolchains_repo_impl(rctx): - build_content = """\ -# Generated by python/private/toolchains_repo.bzl -# -# These can be registered in the workspace file or passed to --extra_toolchains -# flag. By default all these toolchains are registered by the -# python_register_toolchains macro so you don't normally need to interact with -# these targets. - -load("@@{rules_python}//python/private:py_toolchain_suite.bzl", "py_toolchain_suite") - -""".format( + build_content = _WORKSPACE_TOOLCHAINS_BUILD_TEMPLATE.format( rules_python = rctx.attr._rules_python_workspace.repo_name, ) @@ -144,22 +272,7 @@ toolchains_repo = repository_rule( def _toolchain_aliases_impl(rctx): # Base BUILD file for this repository. - build_contents = """\ -# Generated by python/private/toolchains_repo.bzl -load("@rules_python//python/private:toolchain_aliases.bzl", "toolchain_aliases") - -package(default_visibility = ["//visibility:public"]) - -exports_files(["defs.bzl"]) - -PLATFORMS = [ -{loaded_platforms} -] -toolchain_aliases( - name = "{py_repository}", - platforms = PLATFORMS, -) -""".format( + build_contents = _TOOLCHAIN_ALIASES_BUILD_TEMPLATE.format( py_repository = rctx.attr.user_repository_name, loaded_platforms = "\n".join([" \"{}\",".format(p) for p in rctx.attr.platforms]), ) @@ -167,41 +280,7 @@ toolchain_aliases( # Expose a Starlark file so rules can know what host platform we used and where to find an interpreter # when using repository_ctx.path, which doesn't understand aliases. - rctx.file("defs.bzl", content = """\ -# Generated by python/private/toolchains_repo.bzl - -load("@@{rules_python}//python:pip.bzl", _compile_pip_requirements = "compile_pip_requirements") -load("@@{rules_python}//python/private:deprecation.bzl", "with_deprecation") -load("@@{rules_python}//python/private:text_util.bzl", "render") -load("@@{rules_python}//python:py_binary.bzl", _py_binary = "py_binary") -load("@@{rules_python}//python:py_test.bzl", _py_test = "py_test") -load( - "@@{rules_python}//python/entry_points:py_console_script_binary.bzl", - _py_console_script_binary = "py_console_script_binary", -) - -def _with_deprecation(kwargs, *, name): - kwargs["python_version"] = "{python_version}" - return with_deprecation.symbol( - kwargs, - symbol_name = name, - old_load = "@{name}//:defs.bzl", - new_load = "@rules_python//python:{{}}.bzl".format(name), - snippet = render.call(name, **{{k: repr(v) for k,v in kwargs.items()}}) - ) - -def py_binary(**kwargs): - return _py_binary(**_with_deprecation(kwargs, name = "py_binary")) - -def py_console_script_binary(**kwargs): - return _py_console_script_binary(**_with_deprecation(kwargs, name = "py_console_script_binary")) - -def py_test(**kwargs): - return _py_test(**_with_deprecation(kwargs, name = "py_test")) - -def compile_pip_requirements(**kwargs): - return _compile_pip_requirements(**_with_deprecation(kwargs, name = "compile_pip_requirements")) -""".format( + rctx.file("defs.bzl", content = _TOOLCHAIN_ALIASES_DEFS_TEMPLATE.format( name = rctx.attr.name, python_version = rctx.attr.python_version, rules_python = rctx.attr._rules_python_workspace.repo_name, @@ -229,11 +308,7 @@ actions.""", ) def _host_toolchain_impl(rctx): - rctx.file("BUILD.bazel", """\ -# Generated by python/private/toolchains_repo.bzl - -exports_files(["python"], visibility = ["//visibility:public"]) -""") + rctx.file("BUILD.bazel", _HOST_TOOLCHAIN_BUILD_CONTENT) os_name = repo_utils.get_platforms_os_name(rctx) host_platform = _get_host_platform( @@ -279,20 +354,10 @@ exports_files(["python"], visibility = ["//visibility:public"]) # Ensure that we can run the interpreter and check that we are not # using the host interpreter. - python_tester_contents = """\ -from pathlib import Path -import sys - -python = Path(sys.executable) -want_python = str(Path("{python}").resolve()) -got_python = str(Path(sys.executable).resolve()) - -assert want_python == got_python, \ - "Expected to use a different interpreter:\\nwant: '{{}}'\\n got: '{{}}'".format( - want_python, - got_python, + python_tester_contents = _HOST_PYTHON_TESTER_TEMPLATE.format( + repo = repo.strip("@"), + python = python_binary, ) -""".format(repo = repo.strip("@"), python = python_binary) python_tester = rctx.path("python_tester.py") rctx.file(python_tester, python_tester_contents) repo_utils.execute_checked( @@ -331,41 +396,7 @@ def _multi_toolchain_aliases_impl(rctx): for python_version, repository_name in rctx.attr.python_versions.items(): file = "{}/defs.bzl".format(python_version) - rctx.file(file, content = """\ -# Generated by python/private/toolchains_repo.bzl - -load("@@{rules_python}//python:pip.bzl", _compile_pip_requirements = "compile_pip_requirements") -load("@@{rules_python}//python/private:deprecation.bzl", "with_deprecation") -load("@@{rules_python}//python/private:text_util.bzl", "render") -load("@@{rules_python}//python:py_binary.bzl", _py_binary = "py_binary") -load("@@{rules_python}//python:py_test.bzl", _py_test = "py_test") -load( - "@@{rules_python}//python/entry_points:py_console_script_binary.bzl", - _py_console_script_binary = "py_console_script_binary", -) - -def _with_deprecation(kwargs, *, name): - kwargs["python_version"] = "{python_version}" - return with_deprecation.symbol( - kwargs, - symbol_name = name, - old_load = "@{name}//{python_version}:defs.bzl", - new_load = "@rules_python//python:{{}}.bzl".format(name), - snippet = render.call(name, **{{k: repr(v) for k,v in kwargs.items()}}) - ) - -def py_binary(**kwargs): - return _py_binary(**_with_deprecation(kwargs, name = "py_binary")) - -def py_console_script_binary(**kwargs): - return _py_console_script_binary(**_with_deprecation(kwargs, name = "py_console_script_binary")) - -def py_test(**kwargs): - return _py_test(**_with_deprecation(kwargs, name = "py_test")) - -def compile_pip_requirements(**kwargs): - return _compile_pip_requirements(**_with_deprecation(kwargs, name = "compile_pip_requirements")) -""".format( + rctx.file(file, content = _MULTI_TOOLCHAIN_ALIASES_DEFS_TEMPLATE.format( repository_name = repository_name, name = rctx.attr.name, python_version = python_version, @@ -373,21 +404,7 @@ def compile_pip_requirements(**kwargs): )) rctx.file("{}/BUILD.bazel".format(python_version), "") - pip_bzl = """\ -# Generated by python/private/toolchains_repo.bzl - -load("@@{rules_python}//python:pip.bzl", "pip_parse", _multi_pip_parse = "multi_pip_parse") - -def multi_pip_parse(name, requirements_lock, **kwargs): - return _multi_pip_parse( - name = name, - python_versions = {python_versions}, - requirements_lock = requirements_lock, - minor_mapping = {minor_mapping}, - **kwargs - ) - -""".format( + pip_bzl = _MULTI_TOOLCHAIN_ALIASES_PIP_TEMPLATE.format( python_versions = rctx.attr.python_versions.keys(), minor_mapping = render.indent(render.dict(rctx.attr.minor_mapping), indent = " " * 8).lstrip(), rules_python = rules_python, From 50d59e5e8ca5bc7fb50c4cec8b706938c003b778 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Sat, 17 May 2025 04:57:24 -0700 Subject: [PATCH 224/922] dev: add .python-version file so pyenv isn't user/system specific (#2883) The `.python-version` file is read by pyenv to decide what Python version to use. This makes it easier to get started with the project with installing things like pre-comment since you don't have to figure out what version of Python you need. --- .python-version | 1 + 1 file changed, 1 insertion(+) create mode 100644 .python-version diff --git a/.python-version b/.python-version new file mode 100644 index 0000000000..2c20ac9bea --- /dev/null +++ b/.python-version @@ -0,0 +1 @@ +3.13.3 From d6af2b795043ce623dfefe80d34a35268b212e1c Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Sat, 17 May 2025 06:39:36 -0700 Subject: [PATCH 225/922] refactor: have bzlmod pass platforms to python_register_toolchains (#2884) This is to facilitate eventually allowing overrides to add additional platforms. Instead of the PLATFORMS global being used, the python bzlmod extension passing the mapping directly to python_register_toolchains, then receives back the subset of platforms that had repos defined. That subset is then later used when (re)constructing the list of repo names for the toolchains. --- python/private/python.bzl | 54 +++++++++++++------ python/private/python_register_toolchains.bzl | 21 +++++--- tests/python/python_tests.bzl | 1 + 3 files changed, 53 insertions(+), 23 deletions(-) diff --git a/python/private/python.bzl b/python/private/python.bzl index c187904322..c87beefdcc 100644 --- a/python/private/python.bzl +++ b/python/private/python.bzl @@ -34,12 +34,13 @@ def parse_modules(*, module_ctx, _fail = fail): Returns: A struct with the following attributes: - * `toolchains`: The list of toolchains to register. The last - element is special and is treated as the default toolchain. - * `defaults`: The default `kwargs` passed to - {bzl:obj}`python_register_toolchains`. - * `debug_info`: {type}`None | dict` extra information to be passed - to the debug repo. + * `toolchains`: The list of toolchains to register. The last + element is special and is treated as the default toolchain. + * `config`: Various toolchain config, see `_get_toolchain_config`. + * `debug_info`: {type}`None | dict` extra information to be passed + to the debug repo. + * `platforms`: {type}`dict[str, platform_info]` of the base set of + platforms toolchains should be created for, if possible. """ if module_ctx.os.environ.get("RULES_PYTHON_BZLMOD_DEBUG", "0") == "1": debug_info = { @@ -285,11 +286,12 @@ def _python_impl(module_ctx): kwargs.update(py.config.kwargs.get(toolchain_info.python_version, {})) kwargs.update(py.config.kwargs.get(full_python_version, {})) kwargs.update(py.config.default) - loaded_platforms[full_python_version] = python_register_toolchains( + toolchain_registered_platforms = python_register_toolchains( name = toolchain_info.name, _internal_bzlmod_toolchain_call = True, **kwargs ) + loaded_platforms[full_python_version] = toolchain_registered_platforms # List of the base names ("python_3_10") for the toolchain repos base_toolchain_repo_names = [] @@ -332,20 +334,19 @@ def _python_impl(module_ctx): base_name = t.name base_toolchain_repo_names.append(base_name) fv = full_version(version = t.python_version, minor_mapping = py.config.minor_mapping) - for platform in loaded_platforms[fv]: - if platform not in PLATFORMS: - continue + platforms = loaded_platforms[fv] + for platform_name, platform_info in platforms.items(): key = str(len(toolchain_names)) - full_name = "{}_{}".format(base_name, platform) + full_name = "{}_{}".format(base_name, platform_name) toolchain_names.append(full_name) toolchain_repo_names[key] = full_name - toolchain_tcw_map[key] = PLATFORMS[platform].compatible_with + toolchain_tcw_map[key] = platform_info.compatible_with # The target_settings attribute may not be present for users # patching python/versions.bzl. - toolchain_ts_map[key] = getattr(PLATFORMS[platform], "target_settings", []) - toolchain_platform_keys[key] = platform + toolchain_ts_map[key] = getattr(platform_info, "target_settings", []) + toolchain_platform_keys[key] = platform_name toolchain_python_versions[key] = fv # The last toolchain is the default; it can't have version constraints @@ -483,9 +484,9 @@ def _process_single_version_overrides(*, tag, _fail = fail, default): return for platform in (tag.sha256 or []): - if platform not in PLATFORMS: + if platform not in default["platforms"]: _fail("The platform must be one of {allowed} but got '{got}'".format( - allowed = sorted(PLATFORMS), + allowed = sorted(default["platforms"]), got = platform, )) return @@ -602,6 +603,26 @@ def _override_defaults(*overrides, modules, _fail = fail, default): override.fn(tag = tag, _fail = _fail, default = default) def _get_toolchain_config(*, modules, _fail = fail): + """Computes the configs for toolchains. + + Args: + modules: The modules from module_ctx + _fail: Function to call for failing; only used for testing. + + Returns: + A struct with the following: + * `kwargs`: {type}`dict[str, dict[str, object]` custom kwargs to pass to + `python_register_toolchains`, keyed by python version. + The first key is either a Major.Minor or Major.Minor.Patch + string. + * `minor_mapping`: {type}`dict[str, str]` the mapping of Major.Minor + to Major.Minor.Patch. + * `default`: {type}`dict[str, object]` of kwargs passed along to + `python_register_toolchains`. These keys take final precedence. + * `register_all_versions`: {type}`bool` whether all known versions + should be registered. + """ + # Items that can be overridden available_versions = { version: { @@ -621,6 +642,7 @@ def _get_toolchain_config(*, modules, _fail = fail): } default = { "base_url": DEFAULT_RELEASE_BASE_URL, + "platforms": dict(PLATFORMS), # Copy so it's mutable. "tool_versions": available_versions, } diff --git a/python/private/python_register_toolchains.bzl b/python/private/python_register_toolchains.bzl index cd3e9cbed7..6a4c0c310f 100644 --- a/python/private/python_register_toolchains.bzl +++ b/python/private/python_register_toolchains.bzl @@ -41,6 +41,7 @@ def python_register_toolchains( register_coverage_tool = False, set_python_version_constraint = False, tool_versions = None, + platforms = PLATFORMS, minor_mapping = None, **kwargs): """Convenience macro for users which does typical setup. @@ -70,12 +71,18 @@ def python_register_toolchains( tool_versions: {type}`dict` contains a mapping of version with SHASUM and platform info. If not supplied, the defaults in python/versions.bzl will be used. + platforms: {type}`dict[str, platform_info]` platforms to create toolchain + repositories for. Note that only a subset is created, depending + on what's available in `tool_versions`. minor_mapping: {type}`dict[str, str]` contains a mapping from `X.Y` to `X.Y.Z` version. **kwargs: passed to each {obj}`python_repository` call. Returns: - On bzlmod this returns the loaded platform labels. Otherwise None. + On workspace, returns None. + + On bzlmod, returns a `dict[str, platform_info]`, which is the + subset of `platforms` that it created repositories for. """ bzlmod_toolchain_call = kwargs.pop("_internal_bzlmod_toolchain_call", False) if bzlmod_toolchain_call: @@ -104,13 +111,13 @@ def python_register_toolchains( )) register_coverage_tool = False - loaded_platforms = [] - for platform in PLATFORMS.keys(): + loaded_platforms = {} + for platform in platforms.keys(): sha256 = tool_versions[python_version]["sha256"].get(platform, None) if not sha256: continue - loaded_platforms.append(platform) + loaded_platforms[platform] = platforms[platform] (release_filename, urls, strip_prefix, patches, patch_strip) = get_release_info(platform, python_version, base_url, tool_versions) # allow passing in a tool version @@ -162,7 +169,7 @@ def python_register_toolchains( host_toolchain( name = name + "_host", - platforms = loaded_platforms, + platforms = loaded_platforms.keys(), python_version = python_version, ) @@ -170,7 +177,7 @@ def python_register_toolchains( name = name, python_version = python_version, user_repository_name = name, - platforms = loaded_platforms, + platforms = loaded_platforms.keys(), ) # in bzlmod we write out our own toolchain repos @@ -182,6 +189,6 @@ def python_register_toolchains( python_version = python_version, set_python_version_constraint = set_python_version_constraint, user_repository_name = name, - platforms = loaded_platforms, + platforms = loaded_platforms.keys(), ) return None diff --git a/tests/python/python_tests.bzl b/tests/python/python_tests.bzl index 443174c966..19be1c478e 100644 --- a/tests/python/python_tests.bzl +++ b/tests/python/python_tests.bzl @@ -149,6 +149,7 @@ def _test_default(env): "base_url", "ignore_root_user_error", "tool_versions", + "platforms", ]) env.expect.that_bool(py.config.default["ignore_root_user_error"]).equals(True) env.expect.that_str(py.default_python_version).equals("3.11") From 60c1c8ec045ca62d4e9ee9e1e8f651833cf8d4da Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Sat, 17 May 2025 13:49:23 -0700 Subject: [PATCH 226/922] sphinxdocs: close repo rule directives (#2892) When converting the protos for repo rules to markdown, their blocks weren't being properly closed. To fix, just add the closing colons. Also added a test of the text generation. --- sphinxdocs/private/proto_to_markdown.py | 4 ++- .../proto_to_markdown_test.py | 35 +++++++++++++++++++ 2 files changed, 38 insertions(+), 1 deletion(-) diff --git a/sphinxdocs/private/proto_to_markdown.py b/sphinxdocs/private/proto_to_markdown.py index 9dac71d51c..58fb79393d 100644 --- a/sphinxdocs/private/proto_to_markdown.py +++ b/sphinxdocs/private/proto_to_markdown.py @@ -216,7 +216,9 @@ def _render_repository_rule(self, repo_rule: stardoc_output_pb2.RepositoryRuleIn self._render_attributes(repo_rule.attribute) if repo_rule.environ: self._write(":envvars: ", ", ".join(sorted(repo_rule.environ))) - self._write("\n") + self._write("\n\n") + + self._write("::::::\n") def _render_rule(self, rule: stardoc_output_pb2.RuleInfo): rule_name = rule.rule_name diff --git a/sphinxdocs/tests/proto_to_markdown/proto_to_markdown_test.py b/sphinxdocs/tests/proto_to_markdown/proto_to_markdown_test.py index 9d15b830e3..da6edb21d4 100644 --- a/sphinxdocs/tests/proto_to_markdown/proto_to_markdown_test.py +++ b/sphinxdocs/tests/proto_to_markdown/proto_to_markdown_test.py @@ -272,6 +272,41 @@ def test_render_module_extension(self): ::::: +:::::: +""" + self.assertIn(expected, actual) + + def test_render_repo_rule(self): + proto_text = """ +file: "@repo//pkg:foo.bzl" +repository_rule_info: { + rule_name: "repository_rule", + doc_string: "REPOSITORY_RULE_DOC_STRING" + attribute: { + name: "repository_rule_attribute_a", + doc_string: "REPOSITORY_RULE_ATTRIBUTE_A_DOC_STRING" + type: BOOLEAN + default_value: "True" + } + environ: "ENV_VAR_A" +} +""" + actual = self._render(proto_text) + expected = """ +::::::{bzl:repo-rule} repository_rule(repository_rule_attribute_a=True) + +REPOSITORY_RULE_DOC_STRING + +:attr repository_rule_attribute_a: + {bzl:default-value}`True` + {type}`bool` + REPOSITORY_RULE_ATTRIBUTE_A_DOC_STRING + :::{bzl:attr-info} Info + ::: + + +:envvars: ENV_VAR_A + :::::: """ self.assertIn(expected, actual) From d91e9b256f9ea797899ef45a221968f2382cf7f4 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Sat, 17 May 2025 13:50:00 -0700 Subject: [PATCH 227/922] sphinxdocs: make xrefs to bzl:obj in inventories work (#2894) Apparently, the `object_type` dict controls what object types are recognized from inventory files. The bazel inventory of terms includes several bzl:obj entries for things that don't have a more appropriate type. This fixes xrefs for terms like RBE, config, and some others. --- sphinxdocs/src/sphinx_bzl/bzl.py | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/sphinxdocs/src/sphinx_bzl/bzl.py b/sphinxdocs/src/sphinx_bzl/bzl.py index 90fb109614..169f998749 100644 --- a/sphinxdocs/src/sphinx_bzl/bzl.py +++ b/sphinxdocs/src/sphinx_bzl/bzl.py @@ -1463,6 +1463,8 @@ class _BzlDomain(domains.Domain): # :obj:. # NOTE: We also use these object types for categorizing things in the # generated index page. + # NOTE: The object type keys control what object types are recognized + # in inventory files. object_types = { "arg": domains.ObjType("arg", "arg", "obj"), # macro/function arg "aspect": domains.ObjType("aspect", "aspect", "obj"), @@ -1486,6 +1488,8 @@ class _BzlDomain(domains.Domain): # types are objects that have a constructor and methods/attrs "type": domains.ObjType("type", "type", "obj"), "typedef": domains.ObjType("typedef", "typedef", "type", "obj"), + # generic objs usually come from inventories + "obj": domains.ObjType("object", "obj") } # This controls: From 9cfdfd823d0196a0e33b7004208199f35a19bdd8 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Sat, 17 May 2025 13:56:52 -0700 Subject: [PATCH 228/922] sphinxdocs: make xrefs to tag class attributes using attr role work (#2895) The role assigned to attributes within the tag class directive were being given the role `arg`, when they should be `attr`. This caused xrefs using the attr role to be unable to find them. To fix, set them to have the correct role, like the repo rule and regular rule directives do. Also add a test. --- sphinxdocs/src/sphinx_bzl/bzl.py | 4 ++-- sphinxdocs/tests/sphinx_stardoc/sphinx_output_test.py | 1 + sphinxdocs/tests/sphinx_stardoc/xrefs.md | 4 ++++ 3 files changed, 7 insertions(+), 2 deletions(-) diff --git a/sphinxdocs/src/sphinx_bzl/bzl.py b/sphinxdocs/src/sphinx_bzl/bzl.py index 169f998749..dc922056a6 100644 --- a/sphinxdocs/src/sphinx_bzl/bzl.py +++ b/sphinxdocs/src/sphinx_bzl/bzl.py @@ -1156,10 +1156,10 @@ class _BzlTagClass(_BzlCallable): doc_field_types = [ _BzlGroupedField( - "arg", + "attr", label=_("Attributes"), names=["attr"], - rolename="arg", + rolename="attr", can_collapse=False, ), ] diff --git a/sphinxdocs/tests/sphinx_stardoc/sphinx_output_test.py b/sphinxdocs/tests/sphinx_stardoc/sphinx_output_test.py index 6d65c920e1..565c5ef68e 100644 --- a/sphinxdocs/tests/sphinx_stardoc/sphinx_output_test.py +++ b/sphinxdocs/tests/sphinx_stardoc/sphinx_output_test.py @@ -63,6 +63,7 @@ def _doc_element(self, doc): ("full_repo_provider", "@testrepo//lang:provider.bzl%LangInfo", "provider.html#LangInfo"), ("full_repo_aspect", "@testrepo//lang:aspect.bzl%myaspect", "aspect.html#myaspect"), ("full_repo_target", "@testrepo//lang:relativetarget", "target.html#relativetarget"), + ("tag_class_attr_using_attr_role", "myext.mytag.ta1", "module_extension.html#myext.mytag.ta1"), # fmt: on ) def test_xrefs(self, text, href): diff --git a/sphinxdocs/tests/sphinx_stardoc/xrefs.md b/sphinxdocs/tests/sphinx_stardoc/xrefs.md index 83f6869a48..8ff3e75d43 100644 --- a/sphinxdocs/tests/sphinx_stardoc/xrefs.md +++ b/sphinxdocs/tests/sphinx_stardoc/xrefs.md @@ -41,3 +41,7 @@ Various tests of cross referencing support ## Any xref * {any}`LangInfo` + +## Tag class refs + +* tag class attribute using attr role: {attr}`myext.mytag.ta1` From dcf0511675a417203e6222d2ead84ce863152977 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Sat, 17 May 2025 16:57:31 -0700 Subject: [PATCH 229/922] sphinxdocs: allow unqualified arg/attr name for xref (#2896) It's common to simply refer to an arg or attribute by its sole name, especially for ones that are unique. This fixes about ~20 broken xrefs in our docs. Also add test for this behavior. --- sphinxdocs/src/sphinx_bzl/bzl.py | 8 ++++++-- sphinxdocs/tests/sphinx_stardoc/sphinx_output_test.py | 2 ++ sphinxdocs/tests/sphinx_stardoc/xrefs.md | 1 + 3 files changed, 9 insertions(+), 2 deletions(-) diff --git a/sphinxdocs/src/sphinx_bzl/bzl.py b/sphinxdocs/src/sphinx_bzl/bzl.py index dc922056a6..44d6cd9994 100644 --- a/sphinxdocs/src/sphinx_bzl/bzl.py +++ b/sphinxdocs/src/sphinx_bzl/bzl.py @@ -390,8 +390,12 @@ def _make_xrefs_for_arg_attr( descr=index_description, ), ), - # This allows referencing an arg as e.g `funcname.argname` - alt_names=[anchor_id], + alt_names=[ + # This allows referencing an arg as e.g `funcname.argname` + anchor_id, + # This allows referencing an arg as simply `argname` + arg_name + ], ) # Two changes to how arg xrefs are created: diff --git a/sphinxdocs/tests/sphinx_stardoc/sphinx_output_test.py b/sphinxdocs/tests/sphinx_stardoc/sphinx_output_test.py index 565c5ef68e..042d2bb533 100644 --- a/sphinxdocs/tests/sphinx_stardoc/sphinx_output_test.py +++ b/sphinxdocs/tests/sphinx_stardoc/sphinx_output_test.py @@ -64,6 +64,8 @@ def _doc_element(self, doc): ("full_repo_aspect", "@testrepo//lang:aspect.bzl%myaspect", "aspect.html#myaspect"), ("full_repo_target", "@testrepo//lang:relativetarget", "target.html#relativetarget"), ("tag_class_attr_using_attr_role", "myext.mytag.ta1", "module_extension.html#myext.mytag.ta1"), + ("tag_class_attr_using_attr_role_just_attr_name", "ta1", "module_extension.html#myext.mytag.ta1"), + # fmt: on ) def test_xrefs(self, text, href): diff --git a/sphinxdocs/tests/sphinx_stardoc/xrefs.md b/sphinxdocs/tests/sphinx_stardoc/xrefs.md index 8ff3e75d43..85055e1f5c 100644 --- a/sphinxdocs/tests/sphinx_stardoc/xrefs.md +++ b/sphinxdocs/tests/sphinx_stardoc/xrefs.md @@ -45,3 +45,4 @@ Various tests of cross referencing support ## Tag class refs * tag class attribute using attr role: {attr}`myext.mytag.ta1` +* tag class attribute, just attr name, attr role: {attr}`ta1` From 5c20268eee7f45e0d8f783429a33d5274c2df4f3 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Sat, 17 May 2025 17:46:03 -0700 Subject: [PATCH 230/922] docs: fix xref to toolchain docs from getting starting (#2899) The "toolchains" xref is ambiguous. Create a unique header for the toolchain configuration section and link to that name instead. --- docs/getting-started.md | 2 +- docs/toolchains.md | 1 + 2 files changed, 2 insertions(+), 1 deletion(-) diff --git a/docs/getting-started.md b/docs/getting-started.md index 969716603c..60d5d5e0be 100644 --- a/docs/getting-started.md +++ b/docs/getting-started.md @@ -7,7 +7,7 @@ and the older way of using `WORKSPACE`. It assumes you have a `requirements.txt` file with your PyPI dependencies. For more details information about configuring `rules_python`, see: -* [Configuring the runtime](toolchains) +* [Configuring the runtime](configuring-toolchains) * [Configuring third party dependencies (pip/pypi)](pypi-dependencies) * [API docs](api/index) diff --git a/docs/toolchains.md b/docs/toolchains.md index 121b398f7a..a2a2b5b63e 100644 --- a/docs/toolchains.md +++ b/docs/toolchains.md @@ -1,6 +1,7 @@ :::{default-domain} bzl ::: +(configuring-toolchains)= # Configuring Python toolchains and runtimes This documents how to configure the Python toolchain and runtimes for different From 53fd252358d1b2184b0429b816cd9420de0e3651 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Sat, 17 May 2025 18:14:39 -0700 Subject: [PATCH 231/922] sphinxdocs: allow files to be xref (#2897) This allows files to be specified as xrefs. Also adds a test for the functionality. --- sphinxdocs/src/sphinx_bzl/bzl.py | 36 +++++++++++++++++-- .../sphinx_stardoc/sphinx_output_test.py | 3 +- sphinxdocs/tests/sphinx_stardoc/xrefs.md | 5 +++ 3 files changed, 41 insertions(+), 3 deletions(-) diff --git a/sphinxdocs/src/sphinx_bzl/bzl.py b/sphinxdocs/src/sphinx_bzl/bzl.py index 44d6cd9994..7804e7f5e2 100644 --- a/sphinxdocs/src/sphinx_bzl/bzl.py +++ b/sphinxdocs/src/sphinx_bzl/bzl.py @@ -502,7 +502,39 @@ def run(self) -> list[docutils_nodes.Node]: self.env.ref_context["bzl:file"] = file_label self.env.ref_context["bzl:object_id_stack"] = [] self.env.ref_context["bzl:doc_id_stack"] = [] - return [] + + _, _, basename = file_label.partition(":") + index_description = f"File {label}" + absolute_label = repo + label + self.env.get_domain("bzl").add_object( + _ObjectEntry( + full_id=absolute_label, + display_name=absolute_label, + object_type="obj", + search_priority=1, + index_entry=domains.IndexEntry( + name=basename, + subtype=_INDEX_SUBTYPE_NORMAL, + docname=self.env.docname, + anchor="", + extra="", + qualifier="", + descr=index_description, + ), + ), + alt_names=[ + # Allow xref //foo:bar.bzl + file_label, + # Allow xref bar.bzl + basename, + ], + ) + index_node = addnodes.index( + entries=[ + _index_node_tuple("single", f"File; {label}", ""), + ] + ) + return [index_node] class _BzlAttrInfo(sphinx_docutils.SphinxDirective): @@ -1493,7 +1525,7 @@ class _BzlDomain(domains.Domain): "type": domains.ObjType("type", "type", "obj"), "typedef": domains.ObjType("typedef", "typedef", "type", "obj"), # generic objs usually come from inventories - "obj": domains.ObjType("object", "obj") + "obj": domains.ObjType("object", "obj"), } # This controls: diff --git a/sphinxdocs/tests/sphinx_stardoc/sphinx_output_test.py b/sphinxdocs/tests/sphinx_stardoc/sphinx_output_test.py index 042d2bb533..aa21369b40 100644 --- a/sphinxdocs/tests/sphinx_stardoc/sphinx_output_test.py +++ b/sphinxdocs/tests/sphinx_stardoc/sphinx_output_test.py @@ -65,7 +65,8 @@ def _doc_element(self, doc): ("full_repo_target", "@testrepo//lang:relativetarget", "target.html#relativetarget"), ("tag_class_attr_using_attr_role", "myext.mytag.ta1", "module_extension.html#myext.mytag.ta1"), ("tag_class_attr_using_attr_role_just_attr_name", "ta1", "module_extension.html#myext.mytag.ta1"), - + ("file_without_repo", "//lang:rule.bzl", "rule.html"), + ("file_with_repo", "@testrepo//lang:rule.bzl", "rule.html"), # fmt: on ) def test_xrefs(self, text, href): diff --git a/sphinxdocs/tests/sphinx_stardoc/xrefs.md b/sphinxdocs/tests/sphinx_stardoc/xrefs.md index 85055e1f5c..a32bb10339 100644 --- a/sphinxdocs/tests/sphinx_stardoc/xrefs.md +++ b/sphinxdocs/tests/sphinx_stardoc/xrefs.md @@ -46,3 +46,8 @@ Various tests of cross referencing support * tag class attribute using attr role: {attr}`myext.mytag.ta1` * tag class attribute, just attr name, attr role: {attr}`ta1` + +## File refs + +* without repo {obj}`//lang:rule.bzl` +* with repo {obj}`@testrepo//lang:rule.bzl` From 8f9ef76d358f2c08ba9558164d333b058915e44d Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Sat, 17 May 2025 18:14:59 -0700 Subject: [PATCH 232/922] docs: move devguide to sphinx for more powerful markup (#2898) Having the devguide processed by Sphinx will let us use more powerful markup, which will help make it possible to create richer documentation. --- DEVELOPING.md => docs/devguide.md | 2 +- docs/index.md | 1 + 2 files changed, 2 insertions(+), 1 deletion(-) rename DEVELOPING.md => docs/devguide.md (99%) diff --git a/DEVELOPING.md b/docs/devguide.md similarity index 99% rename from DEVELOPING.md rename to docs/devguide.md index 83026c1dbc..4d88b2817d 100644 --- a/DEVELOPING.md +++ b/docs/devguide.md @@ -1,4 +1,4 @@ -# For Developers +# Dev Guide This document covers tips and guidance for working on the rules_python code base. A primary audience for it is first time contributors. diff --git a/docs/index.md b/docs/index.md index 285b1cd66e..4983a6a029 100644 --- a/docs/index.md +++ b/docs/index.md @@ -104,6 +104,7 @@ gazelle REPL Extending Contributing +devguide support Changelog api/index From 2e96b3f08bb3fd3b626e036c404a99f9fed7218b Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Sat, 17 May 2025 19:33:52 -0700 Subject: [PATCH 233/922] sphinxdocs: make bazel package xrefs work (#2903) This allows using xrefs like `//python/runtime_env_toolchains` and `runtime_env_toolchains`. --- sphinxdocs/src/sphinx_bzl/bzl.py | 22 ++++++++++++++++--- .../sphinx_stardoc/sphinx_output_test.py | 2 ++ sphinxdocs/tests/sphinx_stardoc/xrefs.md | 5 +++++ 3 files changed, 26 insertions(+), 3 deletions(-) diff --git a/sphinxdocs/src/sphinx_bzl/bzl.py b/sphinxdocs/src/sphinx_bzl/bzl.py index 7804e7f5e2..4468ec660c 100644 --- a/sphinxdocs/src/sphinx_bzl/bzl.py +++ b/sphinxdocs/src/sphinx_bzl/bzl.py @@ -394,7 +394,7 @@ def _make_xrefs_for_arg_attr( # This allows referencing an arg as e.g `funcname.argname` anchor_id, # This allows referencing an arg as simply `argname` - arg_name + arg_name, ], ) @@ -503,7 +503,22 @@ def run(self) -> list[docutils_nodes.Node]: self.env.ref_context["bzl:object_id_stack"] = [] self.env.ref_context["bzl:doc_id_stack"] = [] - _, _, basename = file_label.partition(":") + package_label, _, basename = file_label.partition(":") + + # Transform //foo/bar:BUILD.bazel into "bar" + # This allows referencing "bar" as itself + extra_alt_names = [] + if basename in ("BUILD.bazel", "BUILD"): + # Allow xref //foo + extra_alt_names.append(package_label) + basename = os.path.basename(package_label) + # Handle //:BUILD.bazel + if not basename: + # There isn't a convention for referring to the root package + # besides `//:`, which is already the file_label. So just + # use some obvious value + basename = "__ROOT_BAZEL_PACKAGE__" + index_description = f"File {label}" absolute_label = repo + label self.env.get_domain("bzl").add_object( @@ -527,7 +542,8 @@ def run(self) -> list[docutils_nodes.Node]: file_label, # Allow xref bar.bzl basename, - ], + ] + + extra_alt_names, ) index_node = addnodes.index( entries=[ diff --git a/sphinxdocs/tests/sphinx_stardoc/sphinx_output_test.py b/sphinxdocs/tests/sphinx_stardoc/sphinx_output_test.py index aa21369b40..c78089ac14 100644 --- a/sphinxdocs/tests/sphinx_stardoc/sphinx_output_test.py +++ b/sphinxdocs/tests/sphinx_stardoc/sphinx_output_test.py @@ -67,6 +67,8 @@ def _doc_element(self, doc): ("tag_class_attr_using_attr_role_just_attr_name", "ta1", "module_extension.html#myext.mytag.ta1"), ("file_without_repo", "//lang:rule.bzl", "rule.html"), ("file_with_repo", "@testrepo//lang:rule.bzl", "rule.html"), + ("package_absolute", "//lang", "target.html"), + ("package_basename", "lang", "target.html"), # fmt: on ) def test_xrefs(self, text, href): diff --git a/sphinxdocs/tests/sphinx_stardoc/xrefs.md b/sphinxdocs/tests/sphinx_stardoc/xrefs.md index a32bb10339..bbd415ce19 100644 --- a/sphinxdocs/tests/sphinx_stardoc/xrefs.md +++ b/sphinxdocs/tests/sphinx_stardoc/xrefs.md @@ -51,3 +51,8 @@ Various tests of cross referencing support * without repo {obj}`//lang:rule.bzl` * with repo {obj}`@testrepo//lang:rule.bzl` + +## Package refs + +* absolute label {obj}`//lang` +* package basename {obj}`lang` From 459e1df4a92acfa82e7bfa08b2574dca1437913a Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Sat, 17 May 2025 19:45:25 -0700 Subject: [PATCH 234/922] docs: fix most broken xrefs in changelog (#2902) The changelog has a variety of broken xrefs. This fixes most of them. --- CHANGELOG.md | 30 +++++++++++++++--------------- 1 file changed, 15 insertions(+), 15 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index a6ba65eb2f..a76241018d 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -66,8 +66,8 @@ END_UNRELEASED_TEMPLATE * (rules) On Windows, {obj}`--bootstrap_impl=system_python` is forced. This allows setting `--bootstrap_impl=script` in bazelrc for mixed-platform environments. -* (rules) {obj}`pip_compile` now generates a `.test` target. The `_test` target is deprecated - and will be removed in the next major release. +* (rules) {obj}`compile_pip_requirements` now generates a `.test` target. The + `_test` target is deprecated and will be removed in the next major release. ([#2794](https://github.com/bazel-contrib/rules_python/issues/2794) * (py_wheel) py_wheel always creates zip64-capable wheel zips @@ -190,7 +190,7 @@ END_UNRELEASED_TEMPLATE packages through SimpleAPI unless they are pulled through direct URL references. Fixes [#2023](https://github.com/bazel-contrib/rules_python/issues/2023). In case you see issues with `rules_python` being too eager to fetch the SimpleAPI - metadata, you can use the newly added {attr}`pip.parse.experimental_skip_sources` + metadata, you can use the newly added {attr}`pip.parse.simpleapi_skip` to skip metadata fetching for those packages. * (uv) A {obj}`lock` rule that is the replacement for the {obj}`compile_pip_requirements`. This may still have rough corners @@ -251,7 +251,7 @@ END_UNRELEASED_TEMPLATE {#v1-3-0-added} ### Added -* (python) {attr}`python.defaults` has been added to allow users to +* (python) {obj}`python.defaults` has been added to allow users to set the default python version in the root module by reading the default version number from a file or an environment variable. * {obj}`//python/bin:python`: convenience target for directly running an @@ -271,7 +271,7 @@ END_UNRELEASED_TEMPLATE and py_library rules ([#1647](https://github.com/bazel-contrib/rules_python/issues/1647)) * (rules) Added env-var to allow additional interpreter args for stage1 bootstrap. - See {obj}`RULES_PYTHON_ADDITIONAL_INTERPRETER_ARGS` environment variable. + See {any}`RULES_PYTHON_ADDITIONAL_INTERPRETER_ARGS` environment variable. Only applicable for {obj}`--bootstrap_impl=script`. * (rules) Added {obj}`interpreter_args` attribute to `py_binary` and `py_test`, which allows pass arguments to the interpreter before the regular args. @@ -377,7 +377,7 @@ END_UNRELEASED_TEMPLATE values. Fixes [#2466](https://github.com/bazel-contrib/rules_python/issues/2466). * (py_proto_library) Fix import paths in Bazel 8. * (whl_library) Now the changes to the dependencies are correctly tracked when - PyPI packages used in {bzl:obj}`whl_library` during the `repository_rule` phase + PyPI packages used in `whl_library` during the repository rule phase change. Fixes [#2468](https://github.com/bazel-contrib/rules_python/issues/2468). + (gazelle) Gazelle no longer ignores `setup.py` files by default. To restore this behavior, apply the `# gazelle:python_ignore_files setup.py` directive. @@ -396,7 +396,7 @@ END_UNRELEASED_TEMPLATE * (pypi) Freethreaded packages are now fully supported in the {obj}`experimental_index_url` usage or the regular `pip.parse` usage. To select the free-threaded interpreter in the repo phase, please use - the documented [env](/environment-variables.html) variables. + the documented [env](environment-variables) variables. Fixes [#2386](https://github.com/bazel-contrib/rules_python/issues/2386). * (toolchains) Use the latest astrahl-sh toolchain release [20241206] for Python versions: * 3.9.21 @@ -490,7 +490,7 @@ Other changes: for the latest toolchain versions for each minor Python version. You can control the toolchain selection by using the {bzl:obj}`//python/config_settings:py_linux_libc` build flag. -* (providers) Added {obj}`py_runtime_info.site_init_template` and +* (providers) Added {obj}`PyRuntimeInfo.site_init_template` and {obj}`PyRuntimeInfo.site_init_template` for specifying the template to use to initialize the interpreter via venv startup hooks. * (runfiles) (Bazel 7.4+) Added support for spaces and newlines in runfiles paths @@ -688,8 +688,8 @@ Other changes: * (bzlmod) The default value for the {obj}`--python_version` flag will now be always set to the default python toolchain version value. * (bzlmod) correctly wire the {attr}`pip.parse.extra_pip_args` all the - way to {obj}`whl_library`. What is more we will pass the `extra_pip_args` to - {obj}`whl_library` for `sdist` distributions when using + way to `whl_library`. What is more we will pass the `extra_pip_args` to + `whl_library` for `sdist` distributions when using {attr}`pip.parse.experimental_index_url`. See [#2239](https://github.com/bazel-contrib/rules_python/issues/2239). * (whl_filegroup): Provide per default also the `RECORD` file @@ -737,8 +737,8 @@ Other changes: {#v0-37-0-removed} ### Removed -* (precompiling) {obj}`--precompile_add_to_runfiles` has been removed. -* (precompiling) {obj}`--pyc_collection` has been removed. The `pyc_collection` +* (precompiling) `--precompile_add_to_runfiles` has been removed. +* (precompiling) `--pyc_collection` has been removed. The `pyc_collection` attribute now bases its default on {obj}`--precompile`. * (precompiling) The {obj}`precompile=if_generated_source` value has been removed. * (precompiling) The {obj}`precompile_source_retention=omit_if_generated_source` value has been removed. @@ -790,7 +790,7 @@ Other changes: in extra_requires in py_wheel rule. * (rules) Prevent pytest from trying run the generated stage2 bootstrap .py file when using {obj}`--bootstrap_impl=script` -* (toolchain) The {bzl:obj}`gen_python_config_settings` has been fixed to include +* (toolchain) The `gen_python_config_settings` has been fixed to include the flag_values from the platform definitions. {#v0-36-0-added} @@ -1205,9 +1205,9 @@ Other changes: depend on legacy labels instead of the hub repo aliases and you use the `experimental_requirement_cycles`, now is a good time to migrate. -[python_default_visibility]: gazelle/README.md#directive-python_default_visibility +[python_default_visibility]: https://github.com/bazel-contrib/rules_python/tree/main/gazelle/README.md#directive-python_default_visibility [test_file_pattern_issue]: https://github.com/bazel-contrib/rules_python/issues/1816 -[test_file_pattern_docs]: gazelle/README.md#directive-python_test_file_pattern +[test_file_pattern_docs]: https://github.com/bazel-contrib/rules_python/tree/main/gazelle/README.md#directive-python_test_file_pattern [20240224]: https://github.com/indygreg/python-build-standalone/releases/tag/20240224. [20240415]: https://github.com/indygreg/python-build-standalone/releases/tag/20240415. From 8e6f73b026af31a4064da55b72fb17eb4d0809c5 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Sat, 17 May 2025 19:46:17 -0700 Subject: [PATCH 235/922] tests: move py_reconfig rules to their own file (#2900) The py_reconfig code is pretty large, so move it to its own file. It's also be easier to find in its own file rather that part of something named after "shell testing". --- tests/bootstrap_impls/BUILD.bazel | 3 +- tests/bootstrap_impls/a/b/c/BUILD.bazel | 2 +- tests/interpreter/interpreter_tests.bzl | 2 +- tests/no_unsafe_paths/BUILD.bazel | 2 +- tests/packaging/BUILD.bazel | 2 +- tests/repl/BUILD.bazel | 2 +- tests/runtime_env_toolchain/BUILD.bazel | 2 +- tests/support/py_reconfig.bzl | 101 ++++++++++++++++++++++ tests/support/sh_py_run_test.bzl | 88 +------------------ tests/toolchains/defs.bzl | 2 +- tests/uv/lock/lock_tests.bzl | 2 +- tests/venv_site_packages_libs/BUILD.bazel | 2 +- 12 files changed, 116 insertions(+), 94 deletions(-) create mode 100644 tests/support/py_reconfig.bzl diff --git a/tests/bootstrap_impls/BUILD.bazel b/tests/bootstrap_impls/BUILD.bazel index 28a0d21fb7..b669da5669 100644 --- a/tests/bootstrap_impls/BUILD.bazel +++ b/tests/bootstrap_impls/BUILD.bazel @@ -13,7 +13,8 @@ load("@rules_shell//shell:sh_test.bzl", "sh_test") # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # See the License for the specific language governing permissions and # limitations under the License. -load("//tests/support:sh_py_run_test.bzl", "py_reconfig_binary", "py_reconfig_test", "sh_py_run_test") +load("//tests/support:py_reconfig.bzl", "py_reconfig_binary", "py_reconfig_test") +load("//tests/support:sh_py_run_test.bzl", "sh_py_run_test") load("//tests/support:support.bzl", "SUPPORTS_BOOTSTRAP_SCRIPT") load(":venv_relative_path_tests.bzl", "relative_path_test_suite") diff --git a/tests/bootstrap_impls/a/b/c/BUILD.bazel b/tests/bootstrap_impls/a/b/c/BUILD.bazel index 8ffcbcd479..1659ef25bc 100644 --- a/tests/bootstrap_impls/a/b/c/BUILD.bazel +++ b/tests/bootstrap_impls/a/b/c/BUILD.bazel @@ -1,5 +1,5 @@ load("//python/private:util.bzl", "IS_BAZEL_7_OR_HIGHER") # buildifier: disable=bzl-visibility -load("//tests/support:sh_py_run_test.bzl", "py_reconfig_test") +load("//tests/support:py_reconfig.bzl", "py_reconfig_test") _SUPPORTS_BOOTSTRAP_SCRIPT = select({ "@platforms//os:windows": ["@platforms//:incompatible"], diff --git a/tests/interpreter/interpreter_tests.bzl b/tests/interpreter/interpreter_tests.bzl index ad94f43423..3c5882afa0 100644 --- a/tests/interpreter/interpreter_tests.bzl +++ b/tests/interpreter/interpreter_tests.bzl @@ -14,7 +14,7 @@ """This file contains helpers for testing the interpreter rule.""" -load("//tests/support:sh_py_run_test.bzl", "py_reconfig_test") +load("//tests/support:py_reconfig.bzl", "py_reconfig_test") # The versions of Python that we want to run the interpreter tests against. PYTHON_VERSIONS_TO_TEST = ( diff --git a/tests/no_unsafe_paths/BUILD.bazel b/tests/no_unsafe_paths/BUILD.bazel index f12d1c9a70..c9a681daa9 100644 --- a/tests/no_unsafe_paths/BUILD.bazel +++ b/tests/no_unsafe_paths/BUILD.bazel @@ -11,7 +11,7 @@ # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # See the License for the specific language governing permissions and # limitations under the License. -load("//tests/support:sh_py_run_test.bzl", "py_reconfig_test") +load("//tests/support:py_reconfig.bzl", "py_reconfig_test") load("//tests/support:support.bzl", "SUPPORTS_BOOTSTRAP_SCRIPT") py_reconfig_test( diff --git a/tests/packaging/BUILD.bazel b/tests/packaging/BUILD.bazel index bb12269e3d..d88a593006 100644 --- a/tests/packaging/BUILD.bazel +++ b/tests/packaging/BUILD.bazel @@ -14,7 +14,7 @@ load("@bazel_skylib//rules:build_test.bzl", "build_test") load("@rules_pkg//pkg:tar.bzl", "pkg_tar") -load("//tests/support:sh_py_run_test.bzl", "py_reconfig_test") +load("//tests/support:py_reconfig.bzl", "py_reconfig_test") load("//tests/support:support.bzl", "SUPPORTS_BOOTSTRAP_SCRIPT") build_test( diff --git a/tests/repl/BUILD.bazel b/tests/repl/BUILD.bazel index 62c7377d53..b3986cc023 100644 --- a/tests/repl/BUILD.bazel +++ b/tests/repl/BUILD.bazel @@ -1,5 +1,5 @@ load("//python:py_library.bzl", "py_library") -load("//tests/support:sh_py_run_test.bzl", "py_reconfig_test") +load("//tests/support:py_reconfig.bzl", "py_reconfig_test") # A library that adds a special import path only when this is specified as a # dependency. This makes it easy for a dependency to have this import path diff --git a/tests/runtime_env_toolchain/BUILD.bazel b/tests/runtime_env_toolchain/BUILD.bazel index ad2bd4eeb5..2f82d204ff 100644 --- a/tests/runtime_env_toolchain/BUILD.bazel +++ b/tests/runtime_env_toolchain/BUILD.bazel @@ -13,7 +13,7 @@ # limitations under the License. load("@rules_python_runtime_env_tc_info//:info.bzl", "PYTHON_VERSION") -load("//tests/support:sh_py_run_test.bzl", "py_reconfig_test") +load("//tests/support:py_reconfig.bzl", "py_reconfig_test") load("//tests/support:support.bzl", "CC_TOOLCHAIN") load(":runtime_env_toolchain_tests.bzl", "runtime_env_toolchain_test_suite") diff --git a/tests/support/py_reconfig.bzl b/tests/support/py_reconfig.bzl new file mode 100644 index 0000000000..b33f679e77 --- /dev/null +++ b/tests/support/py_reconfig.bzl @@ -0,0 +1,101 @@ +# Copyright 2024 The Bazel Authors. All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +"""Run a py_binary/py_test with altered config settings. + +This facilitates verify running binaries with different configuration settings +without the overhead of a bazel-in-bazel integration test. +""" + +load("//python/private:attr_builders.bzl", "attrb") # buildifier: disable=bzl-visibility +load("//python/private:py_binary_macro.bzl", "py_binary_macro") # buildifier: disable=bzl-visibility +load("//python/private:py_binary_rule.bzl", "create_py_binary_rule_builder") # buildifier: disable=bzl-visibility +load("//python/private:py_test_macro.bzl", "py_test_macro") # buildifier: disable=bzl-visibility +load("//python/private:py_test_rule.bzl", "create_py_test_rule_builder") # buildifier: disable=bzl-visibility +load("//tests/support:support.bzl", "VISIBLE_FOR_TESTING") + +def _perform_transition_impl(input_settings, attr, base_impl): + settings = {k: input_settings[k] for k in _RECONFIG_INHERITED_OUTPUTS if k in input_settings} + settings.update(base_impl(input_settings, attr)) + + settings[VISIBLE_FOR_TESTING] = True + settings["//command_line_option:build_python_zip"] = attr.build_python_zip + if attr.bootstrap_impl: + settings["//python/config_settings:bootstrap_impl"] = attr.bootstrap_impl + if attr.extra_toolchains: + settings["//command_line_option:extra_toolchains"] = attr.extra_toolchains + if attr.python_src: + settings["//python/bin:python_src"] = attr.python_src + if attr.repl_dep: + settings["//python/bin:repl_dep"] = attr.repl_dep + if attr.venvs_use_declare_symlink: + settings["//python/config_settings:venvs_use_declare_symlink"] = attr.venvs_use_declare_symlink + if attr.venvs_site_packages: + settings["//python/config_settings:venvs_site_packages"] = attr.venvs_site_packages + return settings + +_RECONFIG_INPUTS = [ + "//python/config_settings:bootstrap_impl", + "//python/bin:python_src", + "//python/bin:repl_dep", + "//command_line_option:extra_toolchains", + "//python/config_settings:venvs_use_declare_symlink", + "//python/config_settings:venvs_site_packages", +] +_RECONFIG_OUTPUTS = _RECONFIG_INPUTS + [ + "//command_line_option:build_python_zip", + VISIBLE_FOR_TESTING, +] +_RECONFIG_INHERITED_OUTPUTS = [v for v in _RECONFIG_OUTPUTS if v in _RECONFIG_INPUTS] + +_RECONFIG_ATTRS = { + "bootstrap_impl": attrb.String(), + "build_python_zip": attrb.String(default = "auto"), + "extra_toolchains": attrb.StringList( + doc = """ +Value for the --extra_toolchains flag. + +NOTE: You'll likely have to also specify //tests/support/cc_toolchains:all (or some CC toolchain) +to make the RBE presubmits happy, which disable auto-detection of a CC +toolchain. +""", + ), + "python_src": attrb.Label(), + "repl_dep": attrb.Label(), + "venvs_site_packages": attrb.String(), + "venvs_use_declare_symlink": attrb.String(), +} + +def _create_reconfig_rule(builder): + builder.attrs.update(_RECONFIG_ATTRS) + + base_cfg_impl = builder.cfg.implementation() + builder.cfg.set_implementation(lambda *args: _perform_transition_impl(base_impl = base_cfg_impl, *args)) + builder.cfg.update_inputs(_RECONFIG_INPUTS) + builder.cfg.update_outputs(_RECONFIG_OUTPUTS) + return builder.build() + +_py_reconfig_binary = _create_reconfig_rule(create_py_binary_rule_builder()) + +_py_reconfig_test = _create_reconfig_rule(create_py_test_rule_builder()) + +def py_reconfig_test(**kwargs): + """Create a py_test with customized build settings for testing. + + Args: + **kwargs: kwargs to pass along to _py_reconfig_test. + """ + py_test_macro(_py_reconfig_test, **kwargs) + +def py_reconfig_binary(**kwargs): + py_binary_macro(_py_reconfig_binary, **kwargs) diff --git a/tests/support/sh_py_run_test.bzl b/tests/support/sh_py_run_test.bzl index f6ebc506cc..1a61de9bd3 100644 --- a/tests/support/sh_py_run_test.bzl +++ b/tests/support/sh_py_run_test.bzl @@ -13,94 +13,14 @@ # limitations under the License. """Run a py_binary with altered config settings in an sh_test. -This facilitates verify running binaries with different configuration settings -without the overhead of a bazel-in-bazel integration test. +This facilitates verify running binaries with different outer environmental +settings and verifying their output without the overhead of a bazel-in-bazel +integration test. """ load("@rules_shell//shell:sh_test.bzl", "sh_test") -load("//python/private:attr_builders.bzl", "attrb") # buildifier: disable=bzl-visibility -load("//python/private:py_binary_macro.bzl", "py_binary_macro") # buildifier: disable=bzl-visibility -load("//python/private:py_binary_rule.bzl", "create_py_binary_rule_builder") # buildifier: disable=bzl-visibility -load("//python/private:py_test_macro.bzl", "py_test_macro") # buildifier: disable=bzl-visibility -load("//python/private:py_test_rule.bzl", "create_py_test_rule_builder") # buildifier: disable=bzl-visibility load("//python/private:toolchain_types.bzl", "TARGET_TOOLCHAIN_TYPE") # buildifier: disable=bzl-visibility -load("//tests/support:support.bzl", "VISIBLE_FOR_TESTING") - -def _perform_transition_impl(input_settings, attr, base_impl): - settings = {k: input_settings[k] for k in _RECONFIG_INHERITED_OUTPUTS if k in input_settings} - settings.update(base_impl(input_settings, attr)) - - settings[VISIBLE_FOR_TESTING] = True - settings["//command_line_option:build_python_zip"] = attr.build_python_zip - if attr.bootstrap_impl: - settings["//python/config_settings:bootstrap_impl"] = attr.bootstrap_impl - if attr.extra_toolchains: - settings["//command_line_option:extra_toolchains"] = attr.extra_toolchains - if attr.python_src: - settings["//python/bin:python_src"] = attr.python_src - if attr.repl_dep: - settings["//python/bin:repl_dep"] = attr.repl_dep - if attr.venvs_use_declare_symlink: - settings["//python/config_settings:venvs_use_declare_symlink"] = attr.venvs_use_declare_symlink - if attr.venvs_site_packages: - settings["//python/config_settings:venvs_site_packages"] = attr.venvs_site_packages - return settings - -_RECONFIG_INPUTS = [ - "//python/config_settings:bootstrap_impl", - "//python/bin:python_src", - "//python/bin:repl_dep", - "//command_line_option:extra_toolchains", - "//python/config_settings:venvs_use_declare_symlink", - "//python/config_settings:venvs_site_packages", -] -_RECONFIG_OUTPUTS = _RECONFIG_INPUTS + [ - "//command_line_option:build_python_zip", - VISIBLE_FOR_TESTING, -] -_RECONFIG_INHERITED_OUTPUTS = [v for v in _RECONFIG_OUTPUTS if v in _RECONFIG_INPUTS] - -_RECONFIG_ATTRS = { - "bootstrap_impl": attrb.String(), - "build_python_zip": attrb.String(default = "auto"), - "extra_toolchains": attrb.StringList( - doc = """ -Value for the --extra_toolchains flag. - -NOTE: You'll likely have to also specify //tests/support/cc_toolchains:all (or some CC toolchain) -to make the RBE presubmits happy, which disable auto-detection of a CC -toolchain. -""", - ), - "python_src": attrb.Label(), - "repl_dep": attrb.Label(), - "venvs_site_packages": attrb.String(), - "venvs_use_declare_symlink": attrb.String(), -} - -def _create_reconfig_rule(builder): - builder.attrs.update(_RECONFIG_ATTRS) - - base_cfg_impl = builder.cfg.implementation() - builder.cfg.set_implementation(lambda *args: _perform_transition_impl(base_impl = base_cfg_impl, *args)) - builder.cfg.update_inputs(_RECONFIG_INPUTS) - builder.cfg.update_outputs(_RECONFIG_OUTPUTS) - return builder.build() - -_py_reconfig_binary = _create_reconfig_rule(create_py_binary_rule_builder()) - -_py_reconfig_test = _create_reconfig_rule(create_py_test_rule_builder()) - -def py_reconfig_test(**kwargs): - """Create a py_test with customized build settings for testing. - - Args: - **kwargs: kwargs to pass along to _py_reconfig_test. - """ - py_test_macro(_py_reconfig_test, **kwargs) - -def py_reconfig_binary(**kwargs): - py_binary_macro(_py_reconfig_binary, **kwargs) +load(":py_reconfig.bzl", "py_reconfig_binary") def sh_py_run_test(*, name, sh_src, py_src, **kwargs): """Run a py_binary within a sh_test. diff --git a/tests/toolchains/defs.bzl b/tests/toolchains/defs.bzl index fbb70820c9..a883b0af33 100644 --- a/tests/toolchains/defs.bzl +++ b/tests/toolchains/defs.bzl @@ -15,7 +15,7 @@ "" load("//python:versions.bzl", "PLATFORMS", "TOOL_VERSIONS") -load("//tests/support:sh_py_run_test.bzl", "py_reconfig_test") +load("//tests/support:py_reconfig.bzl", "py_reconfig_test") def define_toolchain_tests(name): """Define the toolchain tests. diff --git a/tests/uv/lock/lock_tests.bzl b/tests/uv/lock/lock_tests.bzl index 35c7c19328..1eb5b1d903 100644 --- a/tests/uv/lock/lock_tests.bzl +++ b/tests/uv/lock/lock_tests.bzl @@ -16,7 +16,7 @@ load("@bazel_skylib//rules:native_binary.bzl", "native_test") load("//python/uv:lock.bzl", "lock") -load("//tests/support:sh_py_run_test.bzl", "py_reconfig_test") +load("//tests/support:py_reconfig.bzl", "py_reconfig_test") def lock_test_suite(name): """The test suite with various lock-related integration tests diff --git a/tests/venv_site_packages_libs/BUILD.bazel b/tests/venv_site_packages_libs/BUILD.bazel index 5d02708800..1f48331ff2 100644 --- a/tests/venv_site_packages_libs/BUILD.bazel +++ b/tests/venv_site_packages_libs/BUILD.bazel @@ -1,4 +1,4 @@ -load("//tests/support:sh_py_run_test.bzl", "py_reconfig_test") +load("//tests/support:py_reconfig.bzl", "py_reconfig_test") load("//tests/support:support.bzl", "SUPPORTS_BOOTSTRAP_SCRIPT") py_reconfig_test( From 945e46478e8b6af4cbe0ca9faa2d5852fe3f42f0 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Sat, 17 May 2025 19:48:38 -0700 Subject: [PATCH 236/922] docs: fix link to py_reconfig and sh_py_run_test files (#2901) Our test files aren't part of the sphinx docs, so use the gh-path external link hook to link to the files directly on github. --- docs/devguide.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/docs/devguide.md b/docs/devguide.md index 4d88b2817d..f233611cad 100644 --- a/docs/devguide.md +++ b/docs/devguide.md @@ -39,7 +39,7 @@ more important for tests to balance understandability and maintainability. ### sh_py_run_test -The [`sh_py_run_test`](tests/support/sh_py_run_test.bzl) rule is a helper to +The {gh-path}`sh_py_run_test Date: Sat, 17 May 2025 20:08:04 -0700 Subject: [PATCH 237/922] sphinxdocs: make Any and object types no-ops to avoid missing xrefs (#2905) The "Any" and "object" types are useful in expression starlark types, but aren't actually real things. Treat them like None and make them no-ops so they aren't treated like missing xrefs. --- sphinxdocs/src/sphinx_bzl/bzl.py | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/sphinxdocs/src/sphinx_bzl/bzl.py b/sphinxdocs/src/sphinx_bzl/bzl.py index 4468ec660c..8303b4d2a5 100644 --- a/sphinxdocs/src/sphinx_bzl/bzl.py +++ b/sphinxdocs/src/sphinx_bzl/bzl.py @@ -1766,6 +1766,11 @@ def _on_missing_reference(app, env: environment.BuildEnvironment, node, contnode # There's no Bazel docs for None, so prevent missing xrefs warning if node["reftarget"] == "None": return contnode + + # Any and object are just conventions from Python, but useful for + # indicating what something is in Starlark, so treat them specially. + if node["reftarget"] in ("Any", "object"): + return contnode return None From 1ea9102ed87fc878e152d64df79e34b604c43572 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Sat, 17 May 2025 23:29:19 -0700 Subject: [PATCH 238/922] docs: correct some xrefs, add various missing Bazel external xrefs (#2907) Adds a variety of Bazel builtins to the external Bazel inventory. Along the way, fFixes a couple of bad xrefs in rule_builders. --- python/private/rule_builders.bzl | 4 ++-- sphinxdocs/inventories/bazel_inventory.txt | 10 +++++++++- 2 files changed, 11 insertions(+), 3 deletions(-) diff --git a/python/private/rule_builders.bzl b/python/private/rule_builders.bzl index 9b7c03136c..892f2ea343 100644 --- a/python/private/rule_builders.bzl +++ b/python/private/rule_builders.bzl @@ -253,7 +253,7 @@ def _ToolchainType_build(self): self: implicitly added Returns: - {type}`config_common.toolchain_type` + {type}`toolchain_type` """ kwargs = dict(self.kwargs) name = kwargs.pop("name") # Name must be positional @@ -673,7 +673,7 @@ def _AttrsDict_build(self): """Build an attribute dict for passing to `rule()`. Returns: - {type}`dict[str, attribute]` where the values are `attr.XXX` objects + {type}`dict[str, Attribute]` where the values are `attr.XXX` objects """ attrs = {} for k, v in self.map.items(): diff --git a/sphinxdocs/inventories/bazel_inventory.txt b/sphinxdocs/inventories/bazel_inventory.txt index 458126a849..bbd200ddb5 100644 --- a/sphinxdocs/inventories/bazel_inventory.txt +++ b/sphinxdocs/inventories/bazel_inventory.txt @@ -3,8 +3,10 @@ # Version: 7.3.0 # The remainder of this file is compressed using zlib Action bzl:type 1 rules/lib/Action - +Attribute bzl:type 1 rules/lib/builtins/Attribute - CcInfo bzl:provider 1 rules/lib/providers/CcInfo - CcInfo.linking_context bzl:provider-field 1 rules/lib/providers/CcInfo#linking_context - +DefaultInfo bzl:type 1 rules/lib/providers/DefaultInfo - ExecutionInfo bzl:type 1 rules/lib/providers/ExecutionInfo - File bzl:type 1 rules/lib/File - Label bzl:type 1 rules/lib/Label - @@ -38,6 +40,7 @@ config.string_list bzl:function 1 rules/lib/toplevel/config#string_list - config.target bzl:function 1 rules/lib/toplevel/config#target - config_common.FeatureFlagInfo bzl:type 1 rules/lib/toplevel/config_common#FeatureFlagInfo - config_common.toolchain_type bzl:function 1 rules/lib/toplevel/config_common#toolchain_type - +ctx bzl:type 1 rules/lib/builtins/repository_ctx - ctx.actions bzl:obj 1 rules/lib/builtins/ctx#actions - ctx.aspect_ids bzl:obj 1 rules/lib/builtins/ctx#aspect_ids - ctx.attr bzl:obj 1 rules/lib/builtins/ctx#attr - @@ -96,6 +99,7 @@ module_ctx.report_progress bzl:function 1 rules/lib/builtins/module_ctx#report_p module_ctx.root_module_has_non_dev_dependency bzl:function 1 rules/lib/builtins/module_ctx#root_module_has_non_dev_dependency - module_ctx.watch bzl:function 1 rules/lib/builtins/module_ctx#watch - module_ctx.which bzl:function 1 rules/lib/builtins/module_ctx#which - +native bzl:obj 1 rules/lib/toplevel/native - native.existing_rule bzl:function 1 rules/lib/toplevel/native#existing_rule - native.existing_rules bzl:function 1 rules/lib/toplevel/native#existing_rules - native.exports_files bzl:function 1 rules/lib/toplevel/native#exports_files - @@ -140,6 +144,8 @@ repository_os bzl:type 1 rules/lib/builtins/repository_os - repository_os.arch bzl:obj 1 rules/lib/builtins/repository_os#arch repository_os.environ bzl:obj 1 rules/lib/builtins/repository_os#environ repository_os.name bzl:obj 1 rules/lib/builtins/repository_os#name +rule bzl:type 1 rules/lib/builtins/rule - +rule bzl:function rules/lib/globals/bzl.html#rule - runfiles bzl:type 1 rules/lib/builtins/runfiles - runfiles.empty_filenames bzl:type 1 rules/lib/builtins/runfiles#empty_filenames - runfiles.files bzl:type 1 rules/lib/builtins/runfiles#files - @@ -156,6 +162,8 @@ testing.TestEnvironment bzl:function 1 rules/lib/toplevel/testing#TestEnvironmen testing.analysis_test bzl:rule 1 rules/lib/toplevel/testing#analysis_test - toolchain bzl:rule 1 reference/be/platforms-and-toolchains#toolchain - toolchain.exec_compatible_with bzl:rule 1 reference/be/platforms-and-toolchains#toolchain.exec_compatible_with - -toolchain.target_settings bzl:attr 1 reference/be/platforms-and-toolchains#toolchain.target_settings - toolchain.target_compatible_with bzl:attr 1 reference/be/platforms-and-toolchains#toolchain.target_compatible_with - +toolchain.target_settings bzl:attr 1 reference/be/platforms-and-toolchains#toolchain.target_settings - toolchain_type bzl:type 1 rules/lib/builtins/toolchain_type.html - +transition bzl:type 1 rules/lib/builtins/transition - +tuple bzl:type 1 rules/lib/core/tuple - From 6ffeff643ad75433c3c65aedf45705b8b6c564e0 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Sun, 18 May 2025 05:01:58 -0700 Subject: [PATCH 239/922] docs: ignore warnings about missing external py xrefs (#2904) Crossreferencing to py code outside our project isn't setup, so these are just a lot of warning spam. Disable them for now. Co-authored-by: Ignas Anikevicius <240938+aignas@users.noreply.github.com> --- docs/conf.py | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/docs/conf.py b/docs/conf.py index f58baf5183..96bbdb50ab 100644 --- a/docs/conf.py +++ b/docs/conf.py @@ -125,6 +125,12 @@ primary_domain = None # The default is 'py', which we don't make much use of nitpicky = True +nitpick_ignore_regex = [ + # External xrefs aren't setup: ignore missing xref warnings + # External xrefs to sphinx isn't setup: ignore missing xref warnings + ("py:.*", "(sphinx|docutils|ast|enum|collections|typing_extensions).*"), +] + # --- Intersphinx configuration intersphinx_mapping = { From 9de326ec8fc6994cf663bcdbdd61677073f25a46 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Sun, 18 May 2025 15:29:05 -0700 Subject: [PATCH 240/922] refactor: make bzlmod directly aware of created toolchain repo names (#2885) This makes python_register_toolchains return the repo names it created, which allows the bzlmod code to be directly aware of the repos that were created instead of having to rely on assuming the names via the platform keys. This is to facilitate python_register_toolchains creating a more arbitrary subset of platform-specific repos. Work towards https://github.com/bazel-contrib/rules_python/issues/2081 --- python/private/python.bzl | 79 +++++++++++-------- python/private/python_register_toolchains.bzl | 28 ++++--- 2 files changed, 64 insertions(+), 43 deletions(-) diff --git a/python/private/python.bzl b/python/private/python.bzl index c87beefdcc..1a1fcaab8a 100644 --- a/python/private/python.bzl +++ b/python/private/python.bzl @@ -267,11 +267,18 @@ def parse_modules(*, module_ctx, _fail = fail): def _python_impl(module_ctx): py = parse_modules(module_ctx = module_ctx) - # dict[str version, list[str] platforms]; where version is full - # python version string ("3.4.5"), and platforms are keys from - # the PLATFORMS global. - loaded_platforms = {} - for toolchain_info in py.toolchains: + # list of structs; see inline struct call within the loop below. + toolchain_impls = [] + + # list[str] of the base names of toolchain repos + base_toolchain_repo_names = [] + + # Create the underlying python_repository repos that contain the + # python runtimes and their toolchain implementation definitions. + for i, toolchain_info in enumerate(py.toolchains): + is_last = (i + 1) == len(py.toolchains) + base_toolchain_repo_names.append(toolchain_info.name) + # Ensure that we pass the full version here. full_python_version = full_version( version = toolchain_info.python_version, @@ -286,12 +293,28 @@ def _python_impl(module_ctx): kwargs.update(py.config.kwargs.get(toolchain_info.python_version, {})) kwargs.update(py.config.kwargs.get(full_python_version, {})) kwargs.update(py.config.default) - toolchain_registered_platforms = python_register_toolchains( + register_result = python_register_toolchains( name = toolchain_info.name, _internal_bzlmod_toolchain_call = True, **kwargs ) - loaded_platforms[full_python_version] = toolchain_registered_platforms + for repo_name, (platform_name, platform_info) in register_result.impl_repos.items(): + toolchain_impls.append(struct( + # str: The base name to use for the toolchain() target + name = repo_name, + # str: The repo name the toolchain() target points to. + impl_repo_name = repo_name, + # str: platform key in the passed-in platforms dict + platform_name = platform_name, + # struct: platform_info() struct + platform = platform_info, + # str: Major.Minor.Micro python version + full_python_version = full_python_version, + # bool: whether to implicitly add the python version constraint + # to the toolchain's target_settings. + # The last toolchain is the default; it can't have version constraints + set_python_version_constraint = is_last, + )) # List of the base names ("python_3_10") for the toolchain repos base_toolchain_repo_names = [] @@ -329,31 +352,23 @@ def _python_impl(module_ctx): # Split the toolchain info into separate objects so they can be passed onto # the repository rule. - for i, t in enumerate(py.toolchains): - is_last = (i + 1) == len(py.toolchains) - base_name = t.name - base_toolchain_repo_names.append(base_name) - fv = full_version(version = t.python_version, minor_mapping = py.config.minor_mapping) - platforms = loaded_platforms[fv] - for platform_name, platform_info in platforms.items(): - key = str(len(toolchain_names)) - - full_name = "{}_{}".format(base_name, platform_name) - toolchain_names.append(full_name) - toolchain_repo_names[key] = full_name - toolchain_tcw_map[key] = platform_info.compatible_with - - # The target_settings attribute may not be present for users - # patching python/versions.bzl. - toolchain_ts_map[key] = getattr(platform_info, "target_settings", []) - toolchain_platform_keys[key] = platform_name - toolchain_python_versions[key] = fv - - # The last toolchain is the default; it can't have version constraints - # Despite the implication of the arg name, the values are strs, not bools - toolchain_set_python_version_constraints[key] = ( - "True" if not is_last else "False" - ) + for entry in toolchain_impls: + key = str(len(toolchain_names)) + + toolchain_names.append(entry.name) + toolchain_repo_names[key] = entry.impl_repo_name + toolchain_tcw_map[key] = entry.platform.compatible_with + + # The target_settings attribute may not be present for users + # patching python/versions.bzl. + toolchain_ts_map[key] = getattr(entry.platform, "target_settings", []) + toolchain_platform_keys[key] = entry.platform_name + toolchain_python_versions[key] = entry.full_python_version + + # Repo rules can't accept dict[str, bool], so encode them as a string value. + toolchain_set_python_version_constraints[key] = ( + "True" if entry.set_python_version_constraint else "False" + ) hub_repo( name = "pythons_hub", diff --git a/python/private/python_register_toolchains.bzl b/python/private/python_register_toolchains.bzl index 6a4c0c310f..e16a96e763 100644 --- a/python/private/python_register_toolchains.bzl +++ b/python/private/python_register_toolchains.bzl @@ -111,13 +111,17 @@ def python_register_toolchains( )) register_coverage_tool = False - loaded_platforms = {} - for platform in platforms.keys(): + # list[str] of the platform names that were used + loaded_platforms = [] + + # dict[str repo name, tuple[str, platform_info]] + impl_repos = {} + for platform, platform_info in platforms.items(): sha256 = tool_versions[python_version]["sha256"].get(platform, None) if not sha256: continue - loaded_platforms[platform] = platforms[platform] + loaded_platforms.append(platform) (release_filename, urls, strip_prefix, patches, patch_strip) = get_release_info(platform, python_version, base_url, tool_versions) # allow passing in a tool version @@ -137,11 +141,10 @@ def python_register_toolchains( )], ) + impl_repo_name = "{}_{}".format(name, platform) + impl_repos[impl_repo_name] = (platform, platform_info) python_repository( - name = "{name}_{platform}".format( - name = name, - platform = platform, - ), + name = impl_repo_name, sha256 = sha256, patches = patches, patch_strip = patch_strip, @@ -169,7 +172,7 @@ def python_register_toolchains( host_toolchain( name = name + "_host", - platforms = loaded_platforms.keys(), + platforms = loaded_platforms, python_version = python_version, ) @@ -177,18 +180,21 @@ def python_register_toolchains( name = name, python_version = python_version, user_repository_name = name, - platforms = loaded_platforms.keys(), + platforms = loaded_platforms, ) # in bzlmod we write out our own toolchain repos if bzlmod_toolchain_call: - return loaded_platforms + return struct( + # dict[str name, tuple[str platform_name, platform_info]] + impl_repos = impl_repos, + ) toolchains_repo( name = toolchain_repo_name, python_version = python_version, set_python_version_constraint = set_python_version_constraint, user_repository_name = name, - platforms = loaded_platforms.keys(), + platforms = loaded_platforms, ) return None From acc8f8202832252aae398cc6aba0b11de62c5179 Mon Sep 17 00:00:00 2001 From: Philipp Schrader Date: Mon, 19 May 2025 01:38:03 -0700 Subject: [PATCH 241/922] fix: Allow PYTHONSTARTUP to define variables (#2911) With the current `//python/bin:repl` implementation, any variables defined in `PYTHONSTARTUP` are not actually available in the REPL itself. I accidentally omitted this in the first patch. This patch fixes the issue and adds appropriate tests. --- python/bin/repl_stub.py | 5 +++- python/private/repl_template.py | 12 ++++++-- tests/repl/repl_test.py | 50 ++++++++++++++++++++++++++++++++- 3 files changed, 63 insertions(+), 4 deletions(-) diff --git a/python/bin/repl_stub.py b/python/bin/repl_stub.py index 86452aa869..1e21b26dc3 100644 --- a/python/bin/repl_stub.py +++ b/python/bin/repl_stub.py @@ -13,6 +13,9 @@ The logic for PYTHONSTARTUP is handled in python/private/repl_template.py. """ +# Capture the globals from PYTHONSTARTUP so we can pass them on to the console. +console_locals = globals().copy() + import code import sys @@ -26,4 +29,4 @@ sys.ps2 = "" # We set the banner to an empty string because the repl_template.py file already prints the banner. -code.interact(banner="", exitmsg=exitmsg) +code.interact(local=console_locals, banner="", exitmsg=exitmsg) diff --git a/python/private/repl_template.py b/python/private/repl_template.py index 0e058b23ae..37f4529fbe 100644 --- a/python/private/repl_template.py +++ b/python/private/repl_template.py @@ -14,6 +14,10 @@ def start_repl(): cprt = 'Type "help", "copyright", "credits" or "license" for more information.' sys.stderr.write("Python %s on %s\n%s\n" % (sys.version, sys.platform, cprt)) + # If there's a PYTHONSTARTUP script, we need to capture the new variables + # that it defines. + new_globals = {} + # Simulate Python's behavior when a valid startup script is defined by the # PYTHONSTARTUP variable. If this file path fails to load, print the error # and revert to the default behavior. @@ -27,10 +31,14 @@ def start_repl(): print(f"{type(error).__name__}: {error}") else: compiled_code = compile(source_code, filename=startup_file, mode="exec") - eval(compiled_code, {}) + eval(compiled_code, new_globals) bazel_runfiles = runfiles.Create() - runpy.run_path(bazel_runfiles.Rlocation(STUB_PATH), run_name="__main__") + runpy.run_path( + bazel_runfiles.Rlocation(STUB_PATH), + init_globals=new_globals, + run_name="__main__", + ) if __name__ == "__main__": diff --git a/tests/repl/repl_test.py b/tests/repl/repl_test.py index 51ca951110..37c9a37a0d 100644 --- a/tests/repl/repl_test.py +++ b/tests/repl/repl_test.py @@ -1,7 +1,9 @@ import os import subprocess import sys +import tempfile import unittest +from pathlib import Path from typing import Iterable from python import runfiles @@ -13,18 +15,26 @@ EXPECT_TEST_MODULE_IMPORTABLE = os.environ["EXPECT_TEST_MODULE_IMPORTABLE"] == "1" +# An arbitrary piece of code that sets some kind of variable. The variable needs to persist into the +# actual shell. +PYTHONSTARTUP_SETS_VAR = """\ +foo = 1234 +""" + + class ReplTest(unittest.TestCase): def setUp(self): self.repl = rfiles.Rlocation("rules_python/python/bin/repl") assert self.repl - def run_code_in_repl(self, lines: Iterable[str]) -> str: + def run_code_in_repl(self, lines: Iterable[str], *, env=None) -> str: """Runs the lines of code in the REPL and returns the text output.""" return subprocess.check_output( [self.repl], text=True, stderr=subprocess.STDOUT, input="\n".join(lines), + env=env, ).strip() def test_repl_version(self): @@ -69,6 +79,44 @@ def test_import_test_module_failure(self): ) self.assertIn("ModuleNotFoundError: No module named 'test_module'", result) + def test_pythonstartup_gets_executed(self): + """Validates that we can use the variables from PYTHONSTARTUP in the console itself.""" + with tempfile.TemporaryDirectory() as tempdir: + pythonstartup = Path(tempdir) / "pythonstartup.py" + pythonstartup.write_text(PYTHONSTARTUP_SETS_VAR) + + env = os.environ.copy() + env["PYTHONSTARTUP"] = str(pythonstartup) + + result = self.run_code_in_repl( + [ + "print(f'The value of foo is {foo}')", + ], + env=env, + ) + + self.assertIn("The value of foo is 1234", result) + + def test_pythonstartup_doesnt_leak(self): + """Validates that we don't accidentally leak code into the console. + + This test validates that a few of the variables we use in the template and stub are not + accessible in the REPL itself. + """ + with tempfile.TemporaryDirectory() as tempdir: + pythonstartup = Path(tempdir) / "pythonstartup.py" + pythonstartup.write_text(PYTHONSTARTUP_SETS_VAR) + + env = os.environ.copy() + env["PYTHONSTARTUP"] = str(pythonstartup) + + for var_name in ("exitmsg", "sys", "code", "bazel_runfiles", "STUB_PATH"): + with self.subTest(var_name=var_name): + result = self.run_code_in_repl([f"print({var_name})"], env=env) + self.assertIn( + f"NameError: name '{var_name}' is not defined", result + ) + if __name__ == "__main__": unittest.main() From e2e9a43853c7dfb97c408e39903a362dad2d0565 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Mon, 19 May 2025 12:24:09 -0700 Subject: [PATCH 242/922] docs: fix some more bad xrefs (#2910) Misc updates to fix doc warnings * Replace `collection` with `list`. Collections aren't a formal type. Just use list as a stand-in. * Create faux AttributeBuilder typedef so that AttributeBuilder references don't give a xref warning. * Change to object-lookup for `python` name (its a module extension, not rule) --- python/private/python_register_toolchains.bzl | 9 +++++---- python/private/rule_builders.bzl | 18 +++++++++++++++--- python/uv/private/uv.bzl | 2 +- 3 files changed, 21 insertions(+), 8 deletions(-) diff --git a/python/private/python_register_toolchains.bzl b/python/private/python_register_toolchains.bzl index e16a96e763..29da663844 100644 --- a/python/private/python_register_toolchains.bzl +++ b/python/private/python_register_toolchains.bzl @@ -48,7 +48,7 @@ def python_register_toolchains( With `bzlmod` enabled, this function is not needed since `rules_python` is handling everything. In order to override the default behaviour from the - root module one can see the docs for the {rule}`python` extension. + root module one can see the docs for the {obj}`python` extension. - Create a repository for each built-in platform like "python_3_8_linux_amd64" - this repository is lazily fetched when Python is needed for that platform. @@ -71,9 +71,10 @@ def python_register_toolchains( tool_versions: {type}`dict` contains a mapping of version with SHASUM and platform info. If not supplied, the defaults in python/versions.bzl will be used. - platforms: {type}`dict[str, platform_info]` platforms to create toolchain - repositories for. Note that only a subset is created, depending - on what's available in `tool_versions`. + platforms: {type}`dict[str, struct]` platforms to create toolchain + repositories for. Keys are platform names, and values are platform_info + structs. Note that only a subset is created, depending on what's + available in `tool_versions`. minor_mapping: {type}`dict[str, str]` contains a mapping from `X.Y` to `X.Y.Z` version. **kwargs: passed to each {obj}`python_repository` call. diff --git a/python/private/rule_builders.bzl b/python/private/rule_builders.bzl index 892f2ea343..360503b21b 100644 --- a/python/private/rule_builders.bzl +++ b/python/private/rule_builders.bzl @@ -192,7 +192,7 @@ ExecGroup = struct( ) def _ToolchainType_typedef(): - """Builder for {obj}`config_common.toolchain_type()` + """Builder for {obj}`config_common.toolchain_type` :::{include} /_includes/field_kwargs_doc.md ::: @@ -393,7 +393,7 @@ def _RuleCfg_update_inputs(self, *others): Args: self: implicitly added - *others: {type}`collection[Label]` collection of labels to add to + *others: {type}`list[Label]` collection of labels to add to inputs. Only values not already present are added. Note that a `Label`, not `str`, should be passed to ensure different apparent labels can be properly de-duplicated. @@ -405,7 +405,7 @@ def _RuleCfg_update_outputs(self, *others): Args: self: implicitly added - *others: {type}`collection[Label]` collection of labels to add to + *others: {type}`list[Label]` collection of labels to add to outputs. Only values not already present are added. Note that a `Label`, not `str`, should be passed to ensure different apparent labels can be properly de-duplicated. @@ -680,6 +680,18 @@ def _AttrsDict_build(self): attrs[k] = v.build() if _is_builder(v) else v return attrs +def _AttributeBuilder_typedef(): + """An abstract base typedef for builder for a Bazel {obj}`Attribute` + + Instances of this are a builder for a particular `Attribute` type, + e.g. `attr.label`, `attr.string`, etc. + """ + +# buildifier: disable=name-conventions +AttributeBuilder = struct( + TYPEDEF = _AttributeBuilder_typedef, +) + # buildifier: disable=name-conventions AttrsDict = struct( TYPEDEF = _AttrsDict_typedef, diff --git a/python/uv/private/uv.bzl b/python/uv/private/uv.bzl index 55a05be032..09fb78322f 100644 --- a/python/uv/private/uv.bzl +++ b/python/uv/private/uv.bzl @@ -122,7 +122,7 @@ uv.configure( "urls": attr.string_list( doc = """\ The urls to download the binary from. If this is used, {attr}`base_url` and -{attr}`manifest_name` are ignored for the given version. +{attr}`manifest_filename` are ignored for the given version. ::::note If the `urls` are specified, they need to be specified for all of the platforms From 9abd323cdf59248c212032fc7173b9e595f877c9 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Mon, 19 May 2025 13:13:18 -0700 Subject: [PATCH 243/922] refactor: make bzlmod create host repos for toolchains (#2888) This moves the creation of the host_toolchain repos into the bzlmod phase. This is to facilitate future work to allow for when a a particular version doesn't provide a host-compatible variant Work towards https://github.com/bazel-contrib/rules_python/issues/2081 --- python/private/python.bzl | 17 ++++++++++++++++- python/private/python_register_toolchains.bzl | 14 +++++++------- python/private/toolchains_repo.bzl | 2 ++ 3 files changed, 25 insertions(+), 8 deletions(-) diff --git a/python/private/python.bzl b/python/private/python.bzl index 1a1fcaab8a..ea794ecf86 100644 --- a/python/private/python.bzl +++ b/python/private/python.bzl @@ -21,7 +21,7 @@ load(":full_version.bzl", "full_version") load(":python_register_toolchains.bzl", "python_register_toolchains") load(":pythons_hub.bzl", "hub_repo") load(":repo_utils.bzl", "repo_utils") -load(":toolchains_repo.bzl", "multi_toolchain_aliases") +load(":toolchains_repo.bzl", "host_toolchain", "multi_toolchain_aliases") load(":util.bzl", "IS_BAZEL_6_4_OR_HIGHER") load(":version.bzl", "version") @@ -298,6 +298,7 @@ def _python_impl(module_ctx): _internal_bzlmod_toolchain_call = True, **kwargs ) + host_compatible = [] for repo_name, (platform_name, platform_info) in register_result.impl_repos.items(): toolchain_impls.append(struct( # str: The base name to use for the toolchain() target @@ -315,6 +316,15 @@ def _python_impl(module_ctx): # The last toolchain is the default; it can't have version constraints set_python_version_constraint = is_last, )) + if _is_compatible_with_host(module_ctx, platform_info): + host_compatible.append(platform_name) + + host_toolchain( + name = toolchain_info.name + "_host", + # NOTE: Order matters. The first found to be compatible is (usually) used. + platforms = host_compatible, + python_version = full_python_version, + ) # List of the base names ("python_3_10") for the toolchain repos base_toolchain_repo_names = [] @@ -406,6 +416,11 @@ def _python_impl(module_ctx): else: return None +def _is_compatible_with_host(mctx, platform_info): + os_name = repo_utils.get_platforms_os_name(mctx) + cpu_name = repo_utils.get_platforms_cpu_name(mctx) + return platform_info.os_name == os_name and platform_info.arch == cpu_name + def _one_or_the_same(first, second, *, onerror = None): if not first: return second diff --git a/python/private/python_register_toolchains.bzl b/python/private/python_register_toolchains.bzl index 29da663844..e821bae5e7 100644 --- a/python/private/python_register_toolchains.bzl +++ b/python/private/python_register_toolchains.bzl @@ -171,12 +171,6 @@ def python_register_toolchains( platform = platform, )) - host_toolchain( - name = name + "_host", - platforms = loaded_platforms, - python_version = python_version, - ) - toolchain_aliases( name = name, python_version = python_version, @@ -184,13 +178,19 @@ def python_register_toolchains( platforms = loaded_platforms, ) - # in bzlmod we write out our own toolchain repos + # in bzlmod we write out our own toolchain repos and host repos if bzlmod_toolchain_call: return struct( # dict[str name, tuple[str platform_name, platform_info]] impl_repos = impl_repos, ) + host_toolchain( + name = name + "_host", + platforms = loaded_platforms, + python_version = python_version, + ) + toolchains_repo( name = toolchain_repo_name, python_version = python_version, diff --git a/python/private/toolchains_repo.bzl b/python/private/toolchains_repo.bzl index d00f5ae34d..a4188a739a 100644 --- a/python/private/toolchains_repo.bzl +++ b/python/private/toolchains_repo.bzl @@ -375,6 +375,8 @@ def _host_toolchain_impl(rctx): if not rctx.delete(python_tester): fail("Failed to delete the python tester") +# NOTE: The term "toolchain" is a misnomer for this rule. This doesn't define +# a repo with toolchains or toolchain implementations. host_toolchain = repository_rule( _host_toolchain_impl, doc = """\ From 67c5cf0546d9d22b4ce3a36d6f3badebaafd2e10 Mon Sep 17 00:00:00 2001 From: Ignas Anikevicius <240938+aignas@users.noreply.github.com> Date: Tue, 20 May 2025 05:31:45 +0900 Subject: [PATCH 244/922] refactor: remove unused target_platforms hub_repository attr (#2912) The target_platforms attribute is unused. The attribute gets used, but the values it computes are never used. --- python/private/pypi/extension.bzl | 13 ------------- python/private/pypi/hub_repository.bzl | 4 ---- 2 files changed, 17 deletions(-) diff --git a/python/private/pypi/extension.bzl b/python/private/pypi/extension.bzl index 3896f2940a..d3a15dfc44 100644 --- a/python/private/pypi/extension.bzl +++ b/python/private/pypi/extension.bzl @@ -275,12 +275,6 @@ def _create_whl_repos( }, extra_aliases = extra_aliases, whl_libraries = whl_libraries, - target_platforms = { - plat: None - for reqs in requirements_by_platform.values() - for req in reqs - for plat in req.target_platforms - }, ) def _whl_repos(*, requirement, whl_library_args, download_only, netrc, auth_patterns, multiple_requirements_for_whl = False, python_version, enable_pipstar = False): @@ -453,7 +447,6 @@ You cannot use both the additive_build_content and additive_build_content_file a hub_group_map = {} exposed_packages = {} extra_aliases = {} - target_platforms = {} whl_libraries = {} for mod in module_ctx.modules: @@ -536,7 +529,6 @@ You cannot use both the additive_build_content and additive_build_content_file a for whl_name, aliases in out.extra_aliases.items(): extra_aliases[hub_name].setdefault(whl_name, {}).update(aliases) exposed_packages.setdefault(hub_name, {}).update(out.exposed_packages) - target_platforms.setdefault(hub_name, {}).update(out.target_platforms) whl_libraries.update(out.whl_libraries) # TODO @aignas 2024-04-05: how do we support different requirement @@ -574,10 +566,6 @@ You cannot use both the additive_build_content and additive_build_content_file a } for hub_name, extra_whl_aliases in extra_aliases.items() }, - target_platforms = { - hub_name: sorted(p) - for hub_name, p in target_platforms.items() - }, whl_libraries = { k: dict(sorted(args.items())) for k, args in sorted(whl_libraries.items()) @@ -669,7 +657,6 @@ def _pip_impl(module_ctx): }, packages = mods.exposed_packages.get(hub_name, []), groups = mods.hub_group_map.get(hub_name), - target_platforms = mods.target_platforms.get(hub_name, []), ) if bazel_features.external_deps.extension_metadata_has_reproducible: diff --git a/python/private/pypi/hub_repository.bzl b/python/private/pypi/hub_repository.bzl index 0a1e772d05..0dbc6c29c2 100644 --- a/python/private/pypi/hub_repository.bzl +++ b/python/private/pypi/hub_repository.bzl @@ -87,10 +87,6 @@ The list of packages that will be exposed via all_*requirements macros. Defaults mandatory = True, doc = "The apparent name of the repo. This is needed because in bzlmod, the name attribute becomes the canonical name.", ), - "target_platforms": attr.string_list( - mandatory = True, - doc = "All of the target platforms for the hub repo", - ), "whl_map": attr.string_dict( mandatory = True, doc = """\ From c7efa25aabc0f74f008e683d7ce266f0f3f4ff38 Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Mon, 19 May 2025 13:32:10 -0700 Subject: [PATCH 245/922] build(deps): bump setuptools from 65.6.3 to 78.1.1 in /examples/bzlmod (#2914) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Bumps [setuptools](https://github.com/pypa/setuptools) from 65.6.3 to 78.1.1.
Changelog

Sourced from setuptools's changelog.

v78.1.1

Bugfixes

  • More fully sanitized the filename in PackageIndex._download. (#4946)

v78.1.0

Features

  • Restore access to _get_vc_env with a warning. (#4874)

v78.0.2

Bugfixes

  • Postponed removals of deprecated dash-separated and uppercase fields in setup.cfg. All packages with deprecated configurations are advised to move before 2026. (#4911)

v78.0.1

Misc

v78.0.0

Bugfixes

  • Reverted distutils changes that broke the monkey patching of command classes. (#4902)

Deprecations and Removals

  • Setuptools no longer accepts options containing uppercase or dash characters in setup.cfg.

... (truncated)

Commits

[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=setuptools&package-manager=pip&previous-version=65.6.3&new-version=78.1.1)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) ---
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot show ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) You can disable automated security fix PRs for this repo from the [Security Alerts page](https://github.com/bazel-contrib/rules_python/network/alerts).
Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> --- examples/bzlmod/requirements_lock_3_9.txt | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/examples/bzlmod/requirements_lock_3_9.txt b/examples/bzlmod/requirements_lock_3_9.txt index c48f406451..8a6d41441a 100644 --- a/examples/bzlmod/requirements_lock_3_9.txt +++ b/examples/bzlmod/requirements_lock_3_9.txt @@ -262,9 +262,9 @@ s3cmd==2.1.0 \ --hash=sha256:49cd23d516b17974b22b611a95ce4d93fe326feaa07320bd1d234fed68cbccfa \ --hash=sha256:966b0a494a916fc3b4324de38f089c86c70ee90e8e1cae6d59102103a4c0cc03 # via -r examples/bzlmod/requirements.in -setuptools==65.6.3 \ - --hash=sha256:57f6f22bde4e042978bcd50176fdb381d7c21a9efa4041202288d3737a0c6a54 \ - --hash=sha256:a7620757bf984b58deaf32fc8a4577a9bbc0850cf92c20e1ce41c38c19e5fb75 +setuptools==78.1.1 \ + --hash=sha256:c3a9c4211ff4c309edb8b8c4f1cbfa7ae324c4ba9f91ff254e3d305b9fd54561 \ + --hash=sha256:fcc17fd9cd898242f6b4adfaca46137a9edef687f43e6f78469692a5e70d851d # via # babel # yamllint From a746b8fba6472afc935b6e018d5364cc33bad90b Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Mon, 19 May 2025 14:28:27 -0700 Subject: [PATCH 246/922] refactor: make bzlmod pass platform mapping to host repo creation (#2889) This makes bzlmod pass the platform metadata to the host_toolchain rule instead of the host toolchain rule using the fixed PLATFORMS global. This allows the bzlmod extension to modify the platforms that are available, where the fixed PLATFORM global can't be changed. Work towards https://github.com/bazel-contrib/rules_python/issues/2081 --------- Co-authored-by: Ignas Anikevicius <240938+aignas@users.noreply.github.com> --- python/private/python.bzl | 13 ++++++++++--- python/private/toolchains_repo.bzl | 23 ++++++++++++++++++++++- 2 files changed, 32 insertions(+), 4 deletions(-) diff --git a/python/private/python.bzl b/python/private/python.bzl index ea794ecf86..0f3bbee4ac 100644 --- a/python/private/python.bzl +++ b/python/private/python.bzl @@ -298,7 +298,9 @@ def _python_impl(module_ctx): _internal_bzlmod_toolchain_call = True, **kwargs ) - host_compatible = [] + host_platforms = [] + host_os_names = {} + host_archs = {} for repo_name, (platform_name, platform_info) in register_result.impl_repos.items(): toolchain_impls.append(struct( # str: The base name to use for the toolchain() target @@ -317,12 +319,17 @@ def _python_impl(module_ctx): set_python_version_constraint = is_last, )) if _is_compatible_with_host(module_ctx, platform_info): - host_compatible.append(platform_name) + host_key = str(len(host_platforms)) + host_platforms.append(platform_name) + host_os_names[host_key] = platform_info.os_name + host_archs[host_key] = platform_info.arch host_toolchain( name = toolchain_info.name + "_host", # NOTE: Order matters. The first found to be compatible is (usually) used. - platforms = host_compatible, + platforms = host_platforms, + os_names = host_os_names, + arch_names = host_archs, python_version = full_python_version, ) diff --git a/python/private/toolchains_repo.bzl b/python/private/toolchains_repo.bzl index a4188a739a..29ac694fd5 100644 --- a/python/private/toolchains_repo.bzl +++ b/python/private/toolchains_repo.bzl @@ -386,6 +386,16 @@ toolchain_aliases repo because referencing the `python` interpreter target from this repo causes an eager fetch of the toolchain for the host platform. """, attrs = { + "arch_names": attr.string_dict( + doc = """ +If set, overrides the platform metadata. Keyed by index in `platforms` +""", + ), + "os_names": attr.string_dict( + doc = """ +If set, overrides the platform metadata. Keyed by index in `platforms` +""", + ), "platforms": attr.string_list(mandatory = True), "python_version": attr.string(mandatory = True), "_rule_name": attr.string(default = "host_toolchain"), @@ -436,9 +446,20 @@ def _get_host_platform(*, rctx, logger, python_version, os_name, cpu_name, platf Returns: The host platform. """ + if rctx.attr.os_names: + platform_map = {} + for i, platform_name in enumerate(platforms): + key = str(i) + platform_map[platform_name] = struct( + os_name = rctx.attr.os_names[key], + arch = rctx.attr.arch_names[key], + ) + else: + platform_map = PLATFORMS + candidates = [] for platform in platforms: - meta = PLATFORMS[platform] + meta = platform_map[platform] if meta.os_name == os_name and meta.arch == cpu_name: candidates.append(platform) From f36d1205294c9a67a9ac969051ac75e47e27c689 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Mon, 19 May 2025 20:59:47 -0700 Subject: [PATCH 247/922] docs: fix xrefs in (#2917) More misc xrefs fixes in: * attr_builders * py_console_script_binary * pypi envvar ref --- python/private/attr_builders.bzl | 5 ++++- python/private/py_console_script_binary.bzl | 14 +++++++------- python/private/pypi/attrs.bzl | 2 +- 3 files changed, 12 insertions(+), 9 deletions(-) diff --git a/python/private/attr_builders.bzl b/python/private/attr_builders.bzl index 57fe476109..be9fa22138 100644 --- a/python/private/attr_builders.bzl +++ b/python/private/attr_builders.bzl @@ -1222,7 +1222,7 @@ def _StringList_typedef(): ::: :::{field} default - :type: Value[list[str] | configuration_field] + :type: list[str] | configuration_field ::: :::{function} doc() -> str @@ -1237,6 +1237,9 @@ def _StringList_typedef(): :::{function} set_allow_empty(v: bool) ::: + :::{function} set_default(v: list[str] | configuration_field) + ::: + :::{function} set_doc(v: str) ::: diff --git a/python/private/py_console_script_binary.bzl b/python/private/py_console_script_binary.bzl index 7347ebe16a..154fa3bf2f 100644 --- a/python/private/py_console_script_binary.bzl +++ b/python/private/py_console_script_binary.bzl @@ -56,18 +56,18 @@ def py_console_script_binary( """Generate a py_binary for a console_script entry_point. Args: - name: [`target-name`] The name of the resulting target. - pkg: {any}`simple label` the package for which to generate the script. - entry_points_txt: optional [`label`], the entry_points.txt file to parse + name: {type}`Name` The name of the resulting target. + pkg: {type}`Label` the package for which to generate the script. + entry_points_txt: {type}`label | None`, the entry_points.txt file to parse for available console_script values. It may be a single file, or a group of files, but must contain a file named `entry_points.txt`. If not specified, defaults to the `dist_info` target in the same package as the `pkg` Label. - script: [`str`], The console script name that the py_binary is going to be + script: {type}`str`, The console script name that the py_binary is going to be generated for. Defaults to the normalized name attribute. - binary_rule: {any}`rule callable`, The rule/macro to use to instantiate - the target. It's expected to behave like {any}`py_binary`. - Defaults to {any}`py_binary`. + binary_rule: {type}`callable`, The rule/macro to use to instantiate + the target. It's expected to behave like {obj}`py_binary`. + Defaults to {obj}`py_binary`. **kwargs: Extra parameters forwarded to `binary_rule`. """ main = "rules_python_entry_point_{}.py".format(name) diff --git a/python/private/pypi/attrs.bzl b/python/private/pypi/attrs.bzl index fe35d8bf7d..7ea19d106a 100644 --- a/python/private/pypi/attrs.bzl +++ b/python/private/pypi/attrs.bzl @@ -210,7 +210,7 @@ If True, suppress printing stdout and stderr output to the terminal. If you would like to get more diagnostic output, set {envvar}`RULES_PYTHON_REPO_DEBUG=1 ` or -{envvar}`RULES_PYTHON_REPO_DEBUG_VERBOSITY= ` +{envvar}`RULES_PYTHON_REPO_DEBUG_VERBOSITY=INFO|DEBUG|TRACE ` """, ), # 600 is documented as default here: https://docs.bazel.build/versions/master/skylark/lib/repository_ctx.html#execute From c678623fce4b5213b3c7661c166c0dac1ee22661 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Mon, 19 May 2025 21:07:54 -0700 Subject: [PATCH 248/922] refactor: explicitly define host platform ordering (#2890) It turns out the `unsorted-dict-items` disable in versions.bzl is load-bearing: the precedence of what host-compatible runtime is selected depends on the order of keys. Hence, the keys are carefully defined such that freethreaded and musl come after the regular runtimes. Make this subtle and implicit behavior explicit by having an ordering function that sorts keys in the order we want. Work towards https://github.com/bazel-contrib/rules_python/issues/2081 --------- Co-authored-by: Ignas Anikevicius <240938+aignas@users.noreply.github.com> --- python/private/python.bzl | 25 ++++++++++-------- python/private/toolchains_repo.bzl | 42 +++++++++++++++++++++++++++++- python/versions.bzl | 12 +++++---- 3 files changed, 62 insertions(+), 17 deletions(-) diff --git a/python/private/python.bzl b/python/private/python.bzl index 0f3bbee4ac..0cc19382d0 100644 --- a/python/private/python.bzl +++ b/python/private/python.bzl @@ -21,7 +21,7 @@ load(":full_version.bzl", "full_version") load(":python_register_toolchains.bzl", "python_register_toolchains") load(":pythons_hub.bzl", "hub_repo") load(":repo_utils.bzl", "repo_utils") -load(":toolchains_repo.bzl", "host_toolchain", "multi_toolchain_aliases") +load(":toolchains_repo.bzl", "host_toolchain", "multi_toolchain_aliases", "sorted_host_platforms") load(":util.bzl", "IS_BAZEL_6_4_OR_HIGHER") load(":version.bzl", "version") @@ -298,9 +298,8 @@ def _python_impl(module_ctx): _internal_bzlmod_toolchain_call = True, **kwargs ) - host_platforms = [] - host_os_names = {} - host_archs = {} + + host_platforms = {} for repo_name, (platform_name, platform_info) in register_result.impl_repos.items(): toolchain_impls.append(struct( # str: The base name to use for the toolchain() target @@ -319,17 +318,21 @@ def _python_impl(module_ctx): set_python_version_constraint = is_last, )) if _is_compatible_with_host(module_ctx, platform_info): - host_key = str(len(host_platforms)) - host_platforms.append(platform_name) - host_os_names[host_key] = platform_info.os_name - host_archs[host_key] = platform_info.arch + host_platforms[platform_name] = platform_info + host_platforms = sorted_host_platforms(host_platforms) host_toolchain( name = toolchain_info.name + "_host", # NOTE: Order matters. The first found to be compatible is (usually) used. - platforms = host_platforms, - os_names = host_os_names, - arch_names = host_archs, + platforms = host_platforms.keys(), + os_names = { + str(i): platform_info.os_name + for i, platform_info in enumerate(host_platforms.values()) + }, + arch_names = { + str(i): platform_info.arch + for i, platform_info in enumerate(host_platforms.values()) + }, python_version = full_python_version, ) diff --git a/python/private/toolchains_repo.bzl b/python/private/toolchains_repo.bzl index 29ac694fd5..0fd05c6625 100644 --- a/python/private/toolchains_repo.bzl +++ b/python/private/toolchains_repo.bzl @@ -25,6 +25,8 @@ platform-specific repositories. load( "//python:versions.bzl", + "FREETHREADED", + "MUSL", "PLATFORMS", "WINDOWS_NAME", ) @@ -433,6 +435,44 @@ multi_toolchain_aliases = repository_rule( }, ) +def sorted_host_platforms(platform_map): + """Sort the keys in the platform map to give correct precedence. + + The order of keys in the platform mapping matters for the host toolchain + selection. When multiple runtimes are compatible with the host, we take the + first that is compatible (usually; there's also the + `RULES_PYTHON_REPO_TOOLCHAIN_*` environment variables). The historical + behavior carefully constructed the ordering of platform keys such that + the ordering was: + * Regular platforms + * The "-freethreaded" suffix + * The "-musl" suffix + + Here, we formalize that so it isn't subtly encoded in the ordering of keys + in a dict that autoformatters like to clobber and whose only documentation + is an innocous looking formatter disable directive. + + Args: + platform_map: a mapping of platforms and their metadata. + + Returns: + dict; the same values, but with the keys inserted in the desired + order so that iteration happens in the desired order. + """ + + def platform_keyer(name): + # Ascending sort: lower is higher precedence + return ( + 1 if MUSL in name else 0, + 1 if FREETHREADED in name else 0, + ) + + sorted_platform_keys = sorted(platform_map.keys(), key = platform_keyer) + return { + key: platform_map[key] + for key in sorted_platform_keys + } + def _get_host_platform(*, rctx, logger, python_version, os_name, cpu_name, platforms): """Gets the host platform. @@ -455,7 +495,7 @@ def _get_host_platform(*, rctx, logger, python_version, os_name, cpu_name, platf arch = rctx.attr.arch_names[key], ) else: - platform_map = PLATFORMS + platform_map = sorted_host_platforms(PLATFORMS) candidates = [] for platform in platforms: diff --git a/python/versions.bzl b/python/versions.bzl index 4a2a4cb758..166cc98851 100644 --- a/python/versions.bzl +++ b/python/versions.bzl @@ -19,7 +19,9 @@ MACOS_NAME = "osx" LINUX_NAME = "linux" WINDOWS_NAME = "windows" -FREETHREADED = "freethreaded" + +FREETHREADED = "-freethreaded" +MUSL = "-musl" INSTALL_ONLY = "install_only" DEFAULT_RELEASE_BASE_URL = "https://github.com/astral-sh/python-build-standalone/releases/download" @@ -845,7 +847,7 @@ def _generate_platforms(): for p, v in platforms.items() for suffix, freethreadedness in { "": is_freethreaded_no, - "-" + FREETHREADED: is_freethreaded_yes, + FREETHREADED: is_freethreaded_yes, }.items() } @@ -879,11 +881,11 @@ def get_release_info(platform, python_version, base_url = DEFAULT_RELEASE_BASE_U release_filename = None rendered_urls = [] for u in url: - p, _, _ = platform.partition("-" + FREETHREADED) + p, _, _ = platform.partition(FREETHREADED) - if FREETHREADED in platform: + if FREETHREADED.lstrip("-") in platform: build = "{}+{}-full".format( - FREETHREADED, + FREETHREADED.lstrip("-"), { "aarch64-apple-darwin": "pgo+lto", "aarch64-unknown-linux-gnu": "lto", From cd550d9e77989c021c6603f960100818fea6683f Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Tue, 20 May 2025 17:03:09 -0700 Subject: [PATCH 249/922] docs: generate docs for py_common, PyInfoBuilder APIs (#2920) I wrote up the docs awhile, but didn't fully wire them through to the doc gen. Fixes some various issues with the generated docs along the way. --- docs/BUILD.bazel | 1 + python/api/api.bzl | 21 ++- python/private/api/api.bzl | 12 ++ python/private/api/py_common_api.bzl | 29 ++- python/private/common.bzl | 2 +- python/private/py_info.bzl | 204 +++++++++++++++++++-- python/private/py_package.bzl | 2 +- tests/base_rules/py_info/py_info_tests.bzl | 2 +- 8 files changed, 255 insertions(+), 18 deletions(-) diff --git a/docs/BUILD.bazel b/docs/BUILD.bazel index 25da682012..b3e5f52022 100644 --- a/docs/BUILD.bazel +++ b/docs/BUILD.bazel @@ -113,6 +113,7 @@ sphinx_stardocs( "//python/private:builders_util_bzl", "//python/private:py_binary_rule_bzl", "//python/private:py_cc_toolchain_rule_bzl", + "//python/private:py_info_bzl", "//python/private:py_library_rule_bzl", "//python/private:py_runtime_rule_bzl", "//python/private:py_test_rule_bzl", diff --git a/python/api/api.bzl b/python/api/api.bzl index c8fb921c12..d41ec739cd 100644 --- a/python/api/api.bzl +++ b/python/api/api.bzl @@ -1,4 +1,23 @@ -"""Public, analysis phase APIs for Python rules.""" +"""Public, analysis phase APIs for Python rules. + +To use the analyis-time API, add the attributes to your rule, then +use `py_common.get()` to get the api object: + +``` +load("@rules_python//python/api:api.bzl", "py_common") + +def _impl(ctx): + py_api = py_common.get(ctx) + +myrule = rule( + implementation = _impl, + attrs = {...} | py_common.API_ATTRS +) +``` + +:::{versionadded} 0.37.0 +::: +""" load("//python/private/api:api.bzl", _py_common = "py_common") diff --git a/python/private/api/api.bzl b/python/private/api/api.bzl index 06fb7294b9..44f9ab4e77 100644 --- a/python/private/api/api.bzl +++ b/python/private/api/api.bzl @@ -27,6 +27,17 @@ will depend on the target that is providing the API struct. }, ) +def _py_common_typedef(): + """Typedef for py_common. + + :::{field} API_ATTRS + :type: dict[str, Attribute] + + The attributes that rules must have for `py_common.get()` to work. + ::: + + """ + def _py_common_get(ctx): """Get the py_common API instance. @@ -45,6 +56,7 @@ def _py_common_get(ctx): return ctx.attr._py_common_api[ApiImplInfo].impl py_common = struct( + TYPEDEF = _py_common_typedef, get = _py_common_get, API_ATTRS = { "_py_common_api": attr.label( diff --git a/python/private/api/py_common_api.bzl b/python/private/api/py_common_api.bzl index 401b35973e..6fed245257 100644 --- a/python/private/api/py_common_api.bzl +++ b/python/private/api/py_common_api.bzl @@ -22,17 +22,40 @@ def _py_common_api_impl(ctx): py_common_api = rule( implementation = _py_common_api_impl, - doc = "Rule implementing py_common API.", + doc = "Internal Rule implementing py_common API.", ) +def _py_common_api_typedef(): + """The py_common API implementation. + + An instance of this object is obtained using {obj}`py_common.get()` + """ + def _merge_py_infos(transitive, *, direct = []): - builder = PyInfoBuilder() + """Merge PyInfo objects into a single PyInfo. + + This is a convenience wrapper around {obj}`PyInfoBuilder.merge_all`. For + more control over merging PyInfo objects, use {obj}`PyInfoBuilder`. + + Args: + transitive: {type}`list[PyInfo]` The PyInfo objects with info + considered indirectly provided by something (e.g. via + its deps attribute). + direct: {type}`list[PyInfo]` The PyInfo objects that are + considered directly provided by something (e.g. via + the srcs attribute). + + Returns: + {type}`PyInfo` A PyInfo containing the merged values. + """ + builder = PyInfoBuilder.new() builder.merge_all(transitive, direct = direct) return builder.build() # Exposed for doc generation, not directly used. # buildifier: disable=name-conventions PyCommonApi = struct( + TYPEDEF = _py_common_api_typedef, merge_py_infos = _merge_py_infos, - PyInfoBuilder = PyInfoBuilder, + PyInfoBuilder = PyInfoBuilder.new, ) diff --git a/python/private/common.bzl b/python/private/common.bzl index 072a1bb296..a58a9c00a4 100644 --- a/python/private/common.bzl +++ b/python/private/common.bzl @@ -405,7 +405,7 @@ def create_py_info( transitive sources collected from dependencies (the latter is only necessary for deprecated extra actions support). """ - py_info = PyInfoBuilder() + py_info = PyInfoBuilder.new() py_info.site_packages_symlinks.add(site_packages_symlinks) py_info.direct_original_sources.add(original_sources) py_info.direct_pyc_files.add(required_pyc_files) diff --git a/python/private/py_info.bzl b/python/private/py_info.bzl index dc3cb24c51..d175eefb69 100644 --- a/python/private/py_info.bzl +++ b/python/private/py_info.bzl @@ -82,7 +82,11 @@ def _PyInfo_init( } PyInfo, _unused_raw_py_info_ctor = define_bazel_6_provider( - doc = "Encapsulates information provided by the Python rules.", + doc = """Encapsulates information provided by the Python rules. + +Instead of creating this object directly, use {obj}`PyInfoBuilder` and +the {obj}`PyCommonApi` utilities. +""", init = _PyInfo_init, fields = { "direct_original_sources": """ @@ -265,7 +269,65 @@ This field is currently unused in Bazel and may go away in the future. # The "effective" PyInfo is what the canonical //python:py_info.bzl%PyInfo symbol refers to _EffectivePyInfo = PyInfo if (config.enable_pystar or BuiltinPyInfo == None) else BuiltinPyInfo -def PyInfoBuilder(): +def _PyInfoBuilder_typedef(): + """Builder for PyInfo. + + To create an instance, use {obj}`py_common.get()` and call `PyInfoBuilder()` + + :::{field} direct_original_sources + :type: DepsetBuilder[File] + ::: + + :::{field} direct_pyc_files + :type: DepsetBuilder[File] + ::: + + :::{field} direct_pyi_files + :type: DepsetBuilder[File] + ::: + + :::{field} imports + :type: DepsetBuilder[str] + ::: + + :::{field} transitive_implicit_pyc_files + :type: DepsetBuilder[File] + ::: + + :::{field} transitive_implicit_pyc_source_files + :type: DepsetBuilder[File] + ::: + + :::{field} transitive_original_sources + :type: DepsetBuilder[File] + ::: + + :::{field} transitive_pyc_files + :type: DepsetBuilder[File] + ::: + + :::{field} transitive_pyi_files + :type: DepsetBuilder[File] + ::: + + :::{field} transitive_sources + :type: DepsetBuilder[File] + ::: + + :::{field} site_packages_symlinks + :type: DepsetBuilder[tuple[str | None, str]] + + NOTE: This depset has `topological` order + ::: + """ + +def _PyInfoBuilder_new(): + """Creates an instance. + + Returns: + {type}`PyInfoBuilder` + """ + # buildifier: disable=uninitialized self = struct( _has_py2_only_sources = [False], @@ -301,35 +363,116 @@ def PyInfoBuilder(): return self def _PyInfoBuilder_get_has_py3_only_sources(self): + """Get the `has_py3_only_sources` value. + + Args: + self: implicitly added. + + Returns: + {type}`bool` + """ return self._has_py3_only_sources[0] def _PyInfoBuilder_get_has_py2_only_sources(self): + """Get the `has_py2_only_sources` value. + + Args: + self: implicitly added. + + Returns: + {type}`bool` + """ return self._has_py2_only_sources[0] def _PyInfoBuilder_set_has_py2_only_sources(self, value): + """Sets `has_py2_only_sources` to `value`. + + Args: + self: implicitly added. + value: {type}`bool` The value to set. + + Returns: + {type}`PyInfoBuilder` self + """ self._has_py2_only_sources[0] = value return self def _PyInfoBuilder_set_has_py3_only_sources(self, value): + """Sets `has_py3_only_sources` to `value`. + + Args: + self: implicitly added. + value: {type}`bool` The value to set. + + Returns: + {type}`PyInfoBuilder` self + """ self._has_py3_only_sources[0] = value return self def _PyInfoBuilder_merge_has_py2_only_sources(self, value): + """Sets `has_py2_only_sources` based on current and incoming `value`. + + Args: + self: implicitly added. + value: {type}`bool` Another `has_py2_only_sources` value. It will + be merged into this builder's state. + + Returns: + {type}`PyInfoBuilder` self + """ self._has_py2_only_sources[0] = self._has_py2_only_sources[0] or value return self def _PyInfoBuilder_merge_has_py3_only_sources(self, value): + """Sets `has_py3_only_sources` based on current and incoming `value`. + + Args: + self: implicitly added. + value: {type}`bool` Another `has_py3_only_sources` value. It will + be merged into this builder's state. + + Returns: + {type}`PyInfoBuilder` self + """ self._has_py3_only_sources[0] = self._has_py3_only_sources[0] or value return self def _PyInfoBuilder_merge_uses_shared_libraries(self, value): + """Sets `uses_shared_libraries` based on current and incoming `value`. + + Args: + self: implicitly added. + value: {type}`bool` Another `uses_shared_libraries` value. It will + be merged into this builder's state. + + Returns: + {type}`PyInfoBuilder` self + """ self._uses_shared_libraries[0] = self._uses_shared_libraries[0] or value return self def _PyInfoBuilder_get_uses_shared_libraries(self): + """Get the `uses_shared_libraries` value. + + Args: + self: implicitly added. + + Returns: + {type}`bool` + """ return self._uses_shared_libraries[0] def _PyInfoBuilder_set_uses_shared_libraries(self, value): + """Sets `uses_shared_libraries` to `value`. + + Args: + self: implicitly added. + value: {type}`bool` The value to set. + + Returns: + {type}`PyInfoBuilder` self + """ self._uses_shared_libraries[0] = value return self @@ -344,7 +487,7 @@ def _PyInfoBuilder_merge(self, *infos, direct = []): direct fields into this object's direct fields. Returns: - {type}`PyInfoBuilder` the current object + {type}`PyInfoBuilder` self """ return self.merge_all(list(infos), direct = direct) @@ -359,7 +502,7 @@ def _PyInfoBuilder_merge_all(self, transitive, *, direct = []): direct fields into this object's direct fields. Returns: - {type}`PyInfoBuilder` the current object + {type}`PyInfoBuilder` self """ for info in direct: # BuiltinPyInfo doesn't have this field @@ -392,11 +535,11 @@ def _PyInfoBuilder_merge_target(self, target): Args: self: implicitly added. target: {type}`Target` targets that provide PyInfo, or other relevant - providers, will be merged into this object. If a target doesn't provide - any relevant providers, it is ignored. + providers, will be merged into this object. If a target doesn't provide + any relevant providers, it is ignored. Returns: - {type}`PyInfoBuilder` the current object. + {type}`PyInfoBuilder` self. """ if PyInfo in target: self.merge(target[PyInfo]) @@ -410,18 +553,26 @@ def _PyInfoBuilder_merge_targets(self, targets): Args: self: implicitly added. targets: {type}`list[Target]` - targets that provide PyInfo, or other relevant - providers, will be merged into this object. If a target doesn't provide - any relevant providers, it is ignored. + targets that provide PyInfo, or other relevant + providers, will be merged into this object. If a target doesn't provide + any relevant providers, it is ignored. Returns: - {type}`PyInfoBuilder` the current object. + {type}`PyInfoBuilder` self. """ for t in targets: self.merge_target(t) return self def _PyInfoBuilder_build(self): + """Builds into a {obj}`PyInfo` object. + + Args: + self: implicitly added. + + Returns: + {type}`PyInfo` + """ if config.enable_pystar: kwargs = dict( direct_original_sources = self.direct_original_sources.build(), @@ -447,6 +598,15 @@ def _PyInfoBuilder_build(self): ) def _PyInfoBuilder_build_builtin_py_info(self): + """Builds into a Bazel-builtin PyInfo object, if available. + + Args: + self: implicitly added. + + Returns: + {type}`BuiltinPyInfo | None` None is returned if Bazel's + builtin PyInfo object is disabled. + """ if BuiltinPyInfo == None: return None @@ -457,3 +617,25 @@ def _PyInfoBuilder_build_builtin_py_info(self): transitive_sources = self.transitive_sources.build(), uses_shared_libraries = self._uses_shared_libraries[0], ) + +# Provided for documentation purposes +# buildifier: disable=name-conventions +PyInfoBuilder = struct( + TYPEDEF = _PyInfoBuilder_typedef, + new = _PyInfoBuilder_new, + build = _PyInfoBuilder_build, + build_builtin_py_info = _PyInfoBuilder_build_builtin_py_info, + get_has_py2_only_sources = _PyInfoBuilder_get_has_py2_only_sources, + get_has_py3_only_sources = _PyInfoBuilder_get_has_py3_only_sources, + get_uses_shared_libraries = _PyInfoBuilder_get_uses_shared_libraries, + merge = _PyInfoBuilder_merge, + merge_all = _PyInfoBuilder_merge_all, + merge_has_py2_only_sources = _PyInfoBuilder_merge_has_py2_only_sources, + merge_has_py3_only_sources = _PyInfoBuilder_merge_has_py3_only_sources, + merge_target = _PyInfoBuilder_merge_target, + merge_targets = _PyInfoBuilder_merge_targets, + merge_uses_shared_libraries = _PyInfoBuilder_merge_uses_shared_libraries, + set_has_py2_only_sources = _PyInfoBuilder_set_has_py2_only_sources, + set_has_py3_only_sources = _PyInfoBuilder_set_has_py3_only_sources, + set_uses_shared_libraries = _PyInfoBuilder_set_uses_shared_libraries, +) diff --git a/python/private/py_package.bzl b/python/private/py_package.bzl index 1d866a9d80..adf2b6deef 100644 --- a/python/private/py_package.bzl +++ b/python/private/py_package.bzl @@ -34,7 +34,7 @@ def _path_inside_wheel(input_file): def _py_package_impl(ctx): inputs = builders.DepsetBuilder() - py_info = PyInfoBuilder() + py_info = PyInfoBuilder.new() for dep in ctx.attr.deps: inputs.add(dep[DefaultInfo].data_runfiles.files) inputs.add(dep[DefaultInfo].default_runfiles.files) diff --git a/tests/base_rules/py_info/py_info_tests.bzl b/tests/base_rules/py_info/py_info_tests.bzl index e160e704de..aa252a2937 100644 --- a/tests/base_rules/py_info/py_info_tests.bzl +++ b/tests/base_rules/py_info/py_info_tests.bzl @@ -162,7 +162,7 @@ def _test_py_info_builder_impl(env, targets): direct_pyi, trans_pyi, ) = targets.misc[DefaultInfo].files.to_list() - builder = PyInfoBuilder() + builder = PyInfoBuilder.new() builder.direct_pyc_files.add(direct_pyc) builder.direct_original_sources.add(original_py) builder.direct_pyi_files.add(direct_pyi) From 85fcd7aef8beed5e5fdbc1d65596345badae3e70 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Thu, 22 May 2025 17:56:52 -0700 Subject: [PATCH 250/922] refactor: rename host_toolchain rule to host_compatible_python_repo (#2926) The host_toolchain name is misleading, so rename it to some more accurate. Work towards https://github.com/bazel-contrib/rules_python/issues/2913 --- python/private/python.bzl | 4 ++-- python/private/python_register_toolchains.bzl | 4 ++-- python/private/toolchains_repo.bzl | 4 ++-- 3 files changed, 6 insertions(+), 6 deletions(-) diff --git a/python/private/python.bzl b/python/private/python.bzl index 0cc19382d0..24ce38ad3d 100644 --- a/python/private/python.bzl +++ b/python/private/python.bzl @@ -21,7 +21,7 @@ load(":full_version.bzl", "full_version") load(":python_register_toolchains.bzl", "python_register_toolchains") load(":pythons_hub.bzl", "hub_repo") load(":repo_utils.bzl", "repo_utils") -load(":toolchains_repo.bzl", "host_toolchain", "multi_toolchain_aliases", "sorted_host_platforms") +load(":toolchains_repo.bzl", "host_compatible_python_repo", "multi_toolchain_aliases", "sorted_host_platforms") load(":util.bzl", "IS_BAZEL_6_4_OR_HIGHER") load(":version.bzl", "version") @@ -321,7 +321,7 @@ def _python_impl(module_ctx): host_platforms[platform_name] = platform_info host_platforms = sorted_host_platforms(host_platforms) - host_toolchain( + host_compatible_python_repo( name = toolchain_info.name + "_host", # NOTE: Order matters. The first found to be compatible is (usually) used. platforms = host_platforms.keys(), diff --git a/python/private/python_register_toolchains.bzl b/python/private/python_register_toolchains.bzl index e821bae5e7..2e0748deb0 100644 --- a/python/private/python_register_toolchains.bzl +++ b/python/private/python_register_toolchains.bzl @@ -28,7 +28,7 @@ load(":full_version.bzl", "full_version") load(":python_repository.bzl", "python_repository") load( ":toolchains_repo.bzl", - "host_toolchain", + "host_compatible_python_repo", "toolchain_aliases", "toolchains_repo", ) @@ -185,7 +185,7 @@ def python_register_toolchains( impl_repos = impl_repos, ) - host_toolchain( + host_compatible_python_repo( name = name + "_host", platforms = loaded_platforms, python_version = python_version, diff --git a/python/private/toolchains_repo.bzl b/python/private/toolchains_repo.bzl index 0fd05c6625..cf4373932b 100644 --- a/python/private/toolchains_repo.bzl +++ b/python/private/toolchains_repo.bzl @@ -379,7 +379,7 @@ def _host_toolchain_impl(rctx): # NOTE: The term "toolchain" is a misnomer for this rule. This doesn't define # a repo with toolchains or toolchain implementations. -host_toolchain = repository_rule( +host_compatible_python_repo = repository_rule( _host_toolchain_impl, doc = """\ Creates a repository with a shorter name meant to be used in the repository_ctx, @@ -400,7 +400,7 @@ If set, overrides the platform metadata. Keyed by index in `platforms` ), "platforms": attr.string_list(mandatory = True), "python_version": attr.string(mandatory = True), - "_rule_name": attr.string(default = "host_toolchain"), + "_rule_name": attr.string(default = "host_compatible_python_repo"), "_rules_python_workspace": attr.label(default = Label("//:WORKSPACE")), }, ) From 46f08dea00288300aaefbb1186074f9d0f0779b5 Mon Sep 17 00:00:00 2001 From: Christian von Schultz Date: Fri, 23 May 2025 02:58:25 +0200 Subject: [PATCH 251/922] docs/refactor: Use python.defaults, not is_default (#2924) When there are multiple Python toolchains, there are currently two ways of setting the default version: the `is_default` attribute of the `python.toolchain()` tag class and the `python.defaults()` tag class. The latter is more powerful, since it also supports files and environment variables. This patch updates the examples and the docs to use `python.defaults()`. Relates to pull request #2588 and issue #2587. --- docs/api/rules_python/python/bin/index.md | 3 ++- docs/toolchains.md | 15 +++++++++++---- examples/bzlmod/MODULE.bazel | 7 +++++-- examples/bzlmod/other_module/MODULE.bazel | 6 ++++-- .../bzlmod_build_file_generation/MODULE.bazel | 6 +++++- examples/multi_python_versions/MODULE.bazel | 2 -- python/extensions/python.bzl | 6 ++---- python/private/python.bzl | 10 ++++------ 8 files changed, 33 insertions(+), 22 deletions(-) diff --git a/docs/api/rules_python/python/bin/index.md b/docs/api/rules_python/python/bin/index.md index 8bea6b54bd..873b644341 100644 --- a/docs/api/rules_python/python/bin/index.md +++ b/docs/api/rules_python/python/bin/index.md @@ -10,7 +10,8 @@ A target to directly run a Python interpreter. By default, it uses the Python version that toolchain resolution matches -(typically the one marked `is_default=True` in `MODULE.bazel`). +(typically the one set with `python.defaults(python_version = ...)` in +`MODULE.bazel`). This runs a Python interpreter in a similar manner as when running `python3` on the command line. It can be invoked using `bazel run`. Remember that in diff --git a/docs/toolchains.md b/docs/toolchains.md index a2a2b5b63e..ada887c945 100644 --- a/docs/toolchains.md +++ b/docs/toolchains.md @@ -44,7 +44,8 @@ you should read the dev-only library module section. bazel_dep(name="rules_python", version=...) python = use_extension("@rules_python//python/extensions:python.bzl", "python") -python.toolchain(python_version = "3.12", is_default = True) +python.defaults(python_version = "3.12") +python.toolchain(python_version = "3.12") ``` ### Library modules @@ -72,7 +73,8 @@ python = use_extension( dev_dependency = True ) -python.toolchain(python_version = "3.12", is_default=True) +python.defaults(python_version = "3.12") +python.toolchain(python_version = "3.12") ``` #### Library modules without version constraints @@ -161,9 +163,13 @@ Multiple versions can be specified and used within a single build. # MODULE.bazel python = use_extension("@rules_python//python/extensions:python.bzl", "python") +python.defaults( + # The environment variable takes precedence if set. + python_version = "3.11", + python_version_env = "BAZEL_PYTHON_VERSION", +) python.toolchain( python_version = "3.11", - is_default = True, ) python.toolchain( @@ -264,7 +270,8 @@ bazel_dep(name = "rules_python", version = "0.40.0") python = use_extension("@rules_python//python/extensions:python.bzl", "python") -python.toolchain(is_default = True, python_version = "3.10") +python.defaults(python_version = "3.10") +python.toolchain(python_version = "3.10") use_repo(python, "python_3_10", "python_3_10_host") ``` diff --git a/examples/bzlmod/MODULE.bazel b/examples/bzlmod/MODULE.bazel index 69e384e42b..841c096dcf 100644 --- a/examples/bzlmod/MODULE.bazel +++ b/examples/bzlmod/MODULE.bazel @@ -28,10 +28,13 @@ bazel_dep(name = "rules_rust", version = "0.54.1") # We next initialize the python toolchain using the extension. # You can set different Python versions in this block. python = use_extension("@rules_python//python/extensions:python.bzl", "python") +python.defaults( + # Use python.defaults if you have defined multiple toolchain versions. + python_version = "3.9", + python_version_env = "BAZEL_PYTHON_VERSION", +) python.toolchain( configure_coverage_tool = True, - # Only set when you have multiple toolchain versions. - is_default = True, python_version = "3.9", ) diff --git a/examples/bzlmod/other_module/MODULE.bazel b/examples/bzlmod/other_module/MODULE.bazel index 959501abc2..f9d6706120 100644 --- a/examples/bzlmod/other_module/MODULE.bazel +++ b/examples/bzlmod/other_module/MODULE.bazel @@ -25,14 +25,16 @@ PYTHON_NAME_39 = "python_3_9" PYTHON_NAME_311 = "python_3_11" python = use_extension("@rules_python//python/extensions:python.bzl", "python") +python.defaults( + # In a submodule this is ignored + python_version = "3.11", +) python.toolchain( configure_coverage_tool = True, python_version = "3.9", ) python.toolchain( configure_coverage_tool = True, - # In a submodule this is ignored - is_default = True, python_version = "3.11", ) diff --git a/examples/bzlmod_build_file_generation/MODULE.bazel b/examples/bzlmod_build_file_generation/MODULE.bazel index 9bec25fcbb..b9b428d365 100644 --- a/examples/bzlmod_build_file_generation/MODULE.bazel +++ b/examples/bzlmod_build_file_generation/MODULE.bazel @@ -46,9 +46,13 @@ python = use_extension("@rules_python//python/extensions:python.bzl", "python") # We next initialize the python toolchain using the extension. # You can set different Python versions in this block. +python.defaults( + # The environment variable takes precedence if set. + python_version = "3.9", + python_version_env = "BAZEL_PYTHON_VERSION", +) python.toolchain( configure_coverage_tool = True, - is_default = True, python_version = "3.9", ) diff --git a/examples/multi_python_versions/MODULE.bazel b/examples/multi_python_versions/MODULE.bazel index 85140360bb..4e4a0473c2 100644 --- a/examples/multi_python_versions/MODULE.bazel +++ b/examples/multi_python_versions/MODULE.bazel @@ -17,8 +17,6 @@ python.defaults( ) python.toolchain( configure_coverage_tool = True, - # Only set when you have mulitple toolchain versions. - is_default = True, python_version = "3.9", ) python.toolchain( diff --git a/python/extensions/python.bzl b/python/extensions/python.bzl index abd5080dd8..b8b755ebca 100644 --- a/python/extensions/python.bzl +++ b/python/extensions/python.bzl @@ -20,10 +20,8 @@ The simplest way to configure the toolchain with `rules_python` is as follows. ```starlark python = use_extension("@rules_python//python/extensions:python.bzl", "python") -python.toolchain( - is_default = True, - python_version = "3.11", -) +python.defaults(python_version = "3.11") +python.toolchain(python_version = "3.11") use_repo(python, "python_3_11") ``` diff --git a/python/private/python.bzl b/python/private/python.bzl index 24ce38ad3d..a7e257601f 100644 --- a/python/private/python.bzl +++ b/python/private/python.bzl @@ -223,7 +223,7 @@ def parse_modules(*, module_ctx, _fail = fail): # A default toolchain is required so that the non-version-specific rules # are able to match a toolchain. if default_toolchain == None: - fail("No default Python toolchain configured. Is rules_python missing `is_default=True`?") + fail("No default Python toolchain configured. Is rules_python missing `python.defaults()`?") elif default_toolchain.python_version not in global_toolchain_versions: fail('Default version "{python_version}" selected by module ' + '"{module_name}", but no toolchain with that version registered'.format( @@ -891,10 +891,8 @@ In order to use a different name than the above, you can use the following `MODU syntax: ```starlark python = use_extension("@rules_python//python/extensions:python.bzl", "python") -python.toolchain( - is_default = True, - python_version = "3.11", -) +python.defaults(python_version = "3.11") +python.toolchain(python_version = "3.11") use_repo(python, my_python_name = "python_3_11") ``` @@ -930,7 +928,7 @@ Whether the toolchain is the default version. :::{versionchanged} 1.4.0 This setting is ignored if the default version is set using the `defaults` -tag class. +tag class (encouraged). ::: """, ), From 2036571e90f3af5318cae40bd504b59939923ec2 Mon Sep 17 00:00:00 2001 From: Marcel Date: Fri, 23 May 2025 22:09:15 +0200 Subject: [PATCH 252/922] fix: Normalize main script path in Python bootstrap (#2925) Use `os.path.normpath()` to resolve `_main/../repo/` to `repo/` and convert forward slashes to backward slashes on Windows. This fixes an issue where `_main` doesn't exist within runfiles and in turn the later assertion that the path to main exists fails (~L542). This happens, for example, when packaging a `py_binary` from a foreign repo into a tar/container. --------- Co-authored-by: Richard Levasseur Co-authored-by: Richard Levasseur --- CHANGELOG.md | 1 + internal_dev_deps.bzl | 6 ++--- python/private/python_bootstrap_template.txt | 8 +++++-- tests/bootstrap_impls/BUILD.bazel | 24 +++++++++++++++++-- tests/bootstrap_impls/external_binary_test.sh | 9 +++++++ tests/modules/other/BUILD.bazel | 14 +++++++++++ tests/modules/other/external_main.py | 1 + 7 files changed, 56 insertions(+), 7 deletions(-) create mode 100755 tests/bootstrap_impls/external_binary_test.sh create mode 100644 tests/modules/other/external_main.py diff --git a/CHANGELOG.md b/CHANGELOG.md index a76241018d..9655b90487 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -91,6 +91,7 @@ END_UNRELEASED_TEMPLATE also retrieved from the URL as opposed to only the `--hash` parameter. Fixes [#2363](https://github.com/bazel-contrib/rules_python/issues/2363). * (pypi) `whl_library` now infers file names from its `urls` attribute correctly. +* (py_test, py_binary) Allow external files to be used for main {#v0-0-0-added} ### Added diff --git a/internal_dev_deps.bzl b/internal_dev_deps.bzl index 87690be1ad..f2b33e279e 100644 --- a/internal_dev_deps.bzl +++ b/internal_dev_deps.bzl @@ -68,10 +68,10 @@ def rules_python_internal_deps(): http_archive( name = "rules_pkg", urls = [ - "https://mirror.bazel.build/github.com/bazelbuild/rules_pkg/releases/download/0.7.0/rules_pkg-0.7.0.tar.gz", - "https://github.com/bazelbuild/rules_pkg/releases/download/0.7.0/rules_pkg-0.7.0.tar.gz", + "https://mirror.bazel.build/github.com/bazelbuild/rules_pkg/releases/download/1.0.1/rules_pkg-1.0.1.tar.gz", + "https://github.com/bazelbuild/rules_pkg/releases/download/1.0.1/rules_pkg-1.0.1.tar.gz", ], - sha256 = "8a298e832762eda1830597d64fe7db58178aa84cd5926d76d5b744d6558941c2", + sha256 = "d20c951960ed77cb7b341c2a59488534e494d5ad1d30c4818c736d57772a9fef", ) http_archive( diff --git a/python/private/python_bootstrap_template.txt b/python/private/python_bootstrap_template.txt index 210987abf9..a979fd4422 100644 --- a/python/private/python_bootstrap_template.txt +++ b/python/private/python_bootstrap_template.txt @@ -499,8 +499,12 @@ def Main(): # The magic string percent-main-percent is replaced with the runfiles-relative # filename of the main file of the Python binary in BazelPythonSemantics.java. main_rel_path = '%main%' - if IsWindows(): - main_rel_path = main_rel_path.replace('/', os.sep) + # NOTE: We call normpath for two reasons: + # 1. Transform Bazel `foo/bar` to Windows `foo\bar` + # 2. Transform `_main/../foo/main.py` to simply `foo/main.py`, which + # matters if `_main` doesn't exist (which can occur if a binary + # is packaged and needs no artifacts from the main repo) + main_rel_path = os.path.normpath(main_rel_path) if IsRunningFromZip(): module_space = CreateModuleSpace() diff --git a/tests/bootstrap_impls/BUILD.bazel b/tests/bootstrap_impls/BUILD.bazel index b669da5669..c3d44df240 100644 --- a/tests/bootstrap_impls/BUILD.bazel +++ b/tests/bootstrap_impls/BUILD.bazel @@ -1,5 +1,3 @@ -load("@rules_shell//shell:sh_test.bzl", "sh_test") - # Copyright 2023 The Bazel Authors. All rights reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); @@ -13,6 +11,8 @@ load("@rules_shell//shell:sh_test.bzl", "sh_test") # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # See the License for the specific language governing permissions and # limitations under the License. +load("@rules_pkg//pkg:tar.bzl", "pkg_tar") +load("@rules_shell//shell:sh_test.bzl", "sh_test") load("//tests/support:py_reconfig.bzl", "py_reconfig_binary", "py_reconfig_test") load("//tests/support:sh_py_run_test.bzl", "sh_py_run_test") load("//tests/support:support.bzl", "SUPPORTS_BOOTSTRAP_SCRIPT") @@ -158,4 +158,24 @@ py_reconfig_test( target_compatible_with = SUPPORTS_BOOTSTRAP_SCRIPT, ) +pkg_tar( + name = "external_binary", + testonly = True, + srcs = ["@other//:external_main"], + include_runfiles = True, + tags = ["manual"], # Don't build as part of wildcards +) + +sh_test( + name = "external_binary_test", + srcs = ["external_binary_test.sh"], + data = [":external_binary"], + # For now, skip this test on Windows because it fails for reasons + # other than the code path being tested. + target_compatible_with = select({ + "@platforms//os:windows": ["@platforms//:incompatible"], + "//conditions:default": [], + }), +) + relative_path_test_suite(name = "relative_path_tests") diff --git a/tests/bootstrap_impls/external_binary_test.sh b/tests/bootstrap_impls/external_binary_test.sh new file mode 100755 index 0000000000..e3516af18e --- /dev/null +++ b/tests/bootstrap_impls/external_binary_test.sh @@ -0,0 +1,9 @@ +#!/bin/bash +set -euxo pipefail + +tmpdir="${TEST_TMPDIR}/external_binary" +mkdir -p "${tmpdir}" +tar xf "tests/bootstrap_impls/external_binary.tar" -C "${tmpdir}" +test -x "${tmpdir}/external_main" +output="$("${tmpdir}/external_main")" +test "$output" = "token" diff --git a/tests/modules/other/BUILD.bazel b/tests/modules/other/BUILD.bazel index e69de29bb2..46f1b96faa 100644 --- a/tests/modules/other/BUILD.bazel +++ b/tests/modules/other/BUILD.bazel @@ -0,0 +1,14 @@ +load("@rules_python//tests/support:py_reconfig.bzl", "py_reconfig_binary") + +package( + default_visibility = ["//visibility:public"], +) + +py_reconfig_binary( + name = "external_main", + srcs = [":external_main.py"], + # We're testing a system_python specific code path, + # so force using that bootstrap + bootstrap_impl = "system_python", + main = "external_main.py", +) diff --git a/tests/modules/other/external_main.py b/tests/modules/other/external_main.py new file mode 100644 index 0000000000..f742ebab60 --- /dev/null +++ b/tests/modules/other/external_main.py @@ -0,0 +1 @@ +print("token") From 3aea414aa2e5f1e0e915f58a434c01428a90382c Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Fri, 23 May 2025 19:52:32 -0700 Subject: [PATCH 253/922] refactor: also rename host toolchain impl function name (#2930) The implementation function name got missed when the repo rule name itself was changed. --- python/private/toolchains_repo.bzl | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/python/private/toolchains_repo.bzl b/python/private/toolchains_repo.bzl index cf4373932b..2476889583 100644 --- a/python/private/toolchains_repo.bzl +++ b/python/private/toolchains_repo.bzl @@ -309,7 +309,7 @@ actions.""", environ = [REPO_DEBUG_ENV_VAR], ) -def _host_toolchain_impl(rctx): +def _host_compatible_python_repo(rctx): rctx.file("BUILD.bazel", _HOST_TOOLCHAIN_BUILD_CONTENT) os_name = repo_utils.get_platforms_os_name(rctx) @@ -380,7 +380,7 @@ def _host_toolchain_impl(rctx): # NOTE: The term "toolchain" is a misnomer for this rule. This doesn't define # a repo with toolchains or toolchain implementations. host_compatible_python_repo = repository_rule( - _host_toolchain_impl, + _host_compatible_python_repo, doc = """\ Creates a repository with a shorter name meant to be used in the repository_ctx, which needs to have `symlinks` for the interpreter. This is separate from the From 28fda8664a1e89f2f055ed7183ad28dbdbeaafc9 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Sat, 24 May 2025 13:35:03 -0700 Subject: [PATCH 254/922] tests: refactor py_reconfig rules so less boilerplate is needed to add attrs (#2933) Just some minor refactoring of the py_reconfig rule so that it's easier to add attributes that affect transition state. After this, just two spots have to be modified to add an attribute (map of attrs, map of attr to transition label). --- tests/support/sh_py_run_test.bzl | 82 ++++++++++++++++++++++++++++++-- 1 file changed, 78 insertions(+), 4 deletions(-) diff --git a/tests/support/sh_py_run_test.bzl b/tests/support/sh_py_run_test.bzl index 1a61de9bd3..04a2883fde 100644 --- a/tests/support/sh_py_run_test.bzl +++ b/tests/support/sh_py_run_test.bzl @@ -13,14 +13,88 @@ # limitations under the License. """Run a py_binary with altered config settings in an sh_test. -This facilitates verify running binaries with different outer environmental -settings and verifying their output without the overhead of a bazel-in-bazel -integration test. +This facilitates verify running binaries with different configuration settings +without the overhead of a bazel-in-bazel integration test. """ load("@rules_shell//shell:sh_test.bzl", "sh_test") +load("//python/private:attr_builders.bzl", "attrb") # buildifier: disable=bzl-visibility +load("//python/private:py_binary_macro.bzl", "py_binary_macro") # buildifier: disable=bzl-visibility +load("//python/private:py_binary_rule.bzl", "create_py_binary_rule_builder") # buildifier: disable=bzl-visibility +load("//python/private:py_test_macro.bzl", "py_test_macro") # buildifier: disable=bzl-visibility +load("//python/private:py_test_rule.bzl", "create_py_test_rule_builder") # buildifier: disable=bzl-visibility load("//python/private:toolchain_types.bzl", "TARGET_TOOLCHAIN_TYPE") # buildifier: disable=bzl-visibility -load(":py_reconfig.bzl", "py_reconfig_binary") +load("//tests/support:support.bzl", "VISIBLE_FOR_TESTING") + +def _perform_transition_impl(input_settings, attr, base_impl): + settings = {k: input_settings[k] for k in _RECONFIG_INHERITED_OUTPUTS if k in input_settings} + settings.update(base_impl(input_settings, attr)) + + settings[VISIBLE_FOR_TESTING] = True + settings["//command_line_option:build_python_zip"] = attr.build_python_zip + + for attr_name, setting_label in _RECONFIG_ATTR_SETTING_MAP.items(): + if getattr(attr, attr_name): + settings[setting_label] = getattr(attr, attr_name) + return settings + +# Attributes that, if non-falsey (`if attr.`), will copy their +# value into the output settings +_RECONFIG_ATTR_SETTING_MAP = { + "bootstrap_impl": "//python/config_settings:bootstrap_impl", + "extra_toolchains": "//command_line_option:extra_toolchains", + "python_src": "//python/bin:python_src", + "venvs_site_packages": "//python/config_settings:venvs_site_packages", + "venvs_use_declare_symlink": "//python/config_settings:venvs_use_declare_symlink", +} + +_RECONFIG_INPUTS = _RECONFIG_ATTR_SETTING_MAP.values() +_RECONFIG_OUTPUTS = _RECONFIG_INPUTS + [ + "//command_line_option:build_python_zip", + VISIBLE_FOR_TESTING, +] +_RECONFIG_INHERITED_OUTPUTS = [v for v in _RECONFIG_OUTPUTS if v in _RECONFIG_INPUTS] + +_RECONFIG_ATTRS = { + "bootstrap_impl": attrb.String(), + "build_python_zip": attrb.String(default = "auto"), + "extra_toolchains": attrb.StringList( + doc = """ +Value for the --extra_toolchains flag. + +NOTE: You'll likely have to also specify //tests/support/cc_toolchains:all (or some CC toolchain) +to make the RBE presubmits happy, which disable auto-detection of a CC +toolchain. +""", + ), + "python_src": attrb.Label(), + "venvs_site_packages": attrb.String(), + "venvs_use_declare_symlink": attrb.String(), +} + +def _create_reconfig_rule(builder): + builder.attrs.update(_RECONFIG_ATTRS) + + base_cfg_impl = builder.cfg.implementation() + builder.cfg.set_implementation(lambda *args: _perform_transition_impl(base_impl = base_cfg_impl, *args)) + builder.cfg.update_inputs(_RECONFIG_INPUTS) + builder.cfg.update_outputs(_RECONFIG_OUTPUTS) + return builder.build() + +_py_reconfig_binary = _create_reconfig_rule(create_py_binary_rule_builder()) + +_py_reconfig_test = _create_reconfig_rule(create_py_test_rule_builder()) + +def py_reconfig_test(**kwargs): + """Create a py_test with customized build settings for testing. + + Args: + **kwargs: kwargs to pass along to _py_reconfig_test. + """ + py_test_macro(_py_reconfig_test, **kwargs) + +def py_reconfig_binary(**kwargs): + py_binary_macro(_py_reconfig_binary, **kwargs) def sh_py_run_test(*, name, sh_src, py_src, **kwargs): """Run a py_binary within a sh_test. From e73dccf7b1827b1ea1216646ac82c97ae8d1e64b Mon Sep 17 00:00:00 2001 From: Chris Chua Date: Sun, 25 May 2025 13:21:44 +0800 Subject: [PATCH 255/922] feat: add shebang attribute on py_console_script_binary (#2867) # Background Use case: user is setting up the environment for a docker image, and needs a bash executable from the py_console_script (e.g. to run `ray` from command line without full bazel bootstrapping). User is responsible of setting up the right paths (and hermeticity concerns). There's no change in default behavior per this diff. Previously, prior to Bazel mod, this was possible and simple through the use of `rules_python_wheel_entry_points` ([per here](https://github.com/bazel-contrib/rules_python/blob/9dfa3abba293488a9a1899832a340f7b44525cad/python/private/pypi/whl_library.bzl#L507)) but these are not reachable now via Bazel mod. # Approach Add a shebang attribute that allows users of the console binary to use it like a binary executable. This is similar to the functionality that came with wheel entry points here: https://github.com/bazel-contrib/rules_python/blob/9dfa3abba293488a9a1899832a340f7b44525cad/python/private/pypi/whl_library.bzl#L507 With this change, one can specify a shebang like: ```starlark py_console_script_binary( name = "yamllint", pkg = "@pip//yamllint", shebang = "#!/usr/bin/env python3", ) ``` Summary: - Update tests - Add test for this functionality - Leave default to without shebang so this is a non-breaking change - Documentation (want to hear more about the general approach first, and also want to hear whether this warrants specific docs, or can just leave it to API docs) --------- Co-authored-by: Ignas Anikevicius <240938+aignas@users.noreply.github.com> --- docs/_includes/py_console_script_binary.md | 23 ++++++++++++- python/private/py_console_script_binary.bzl | 4 +++ python/private/py_console_script_gen.bzl | 5 +++ python/private/py_console_script_gen.py | 11 +++++- .../py_console_script_gen_test.py | 34 +++++++++++++++++++ 5 files changed, 75 insertions(+), 2 deletions(-) diff --git a/docs/_includes/py_console_script_binary.md b/docs/_includes/py_console_script_binary.md index aa356e0e94..d327091630 100644 --- a/docs/_includes/py_console_script_binary.md +++ b/docs/_includes/py_console_script_binary.md @@ -48,6 +48,26 @@ py_console_script_binary( ) ``` +#### Adding a Shebang Line + +You can specify a shebang line for the generated binary, useful for Unix-like +systems where the shebang line determines which interpreter is used to execute +the script, per [PEP441]: + +```starlark +load("@rules_python//python/entry_points:py_console_script_binary.bzl", "py_console_script_binary") + +py_console_script_binary( + name = "black", + pkg = "@pip//black", + shebang = "#!/usr/bin/env python3", +) +``` + +Note that to execute via the shebang line, you need to ensure the specified +Python interpreter is available in the environment. + + #### Using a specific Python Version directly from a Toolchain :::{deprecated} 1.1.0 The toolchain specific `py_binary` and `py_test` symbols are aliases to the regular rules. @@ -70,4 +90,5 @@ py_console_script_binary( ``` [specification]: https://packaging.python.org/en/latest/specifications/entry-points/ -[`py_console_script_binary.binary_rule`]: #py_console_script_binary_binary_rule \ No newline at end of file +[`py_console_script_binary.binary_rule`]: #py_console_script_binary_binary_rule +[PEP441]: https://peps.python.org/pep-0441/#minimal-tooling-the-zipapp-module diff --git a/python/private/py_console_script_binary.bzl b/python/private/py_console_script_binary.bzl index 154fa3bf2f..d98457dbe1 100644 --- a/python/private/py_console_script_binary.bzl +++ b/python/private/py_console_script_binary.bzl @@ -52,6 +52,7 @@ def py_console_script_binary( entry_points_txt = None, script = None, binary_rule = py_binary, + shebang = "", **kwargs): """Generate a py_binary for a console_script entry_point. @@ -68,6 +69,8 @@ def py_console_script_binary( binary_rule: {type}`callable`, The rule/macro to use to instantiate the target. It's expected to behave like {obj}`py_binary`. Defaults to {obj}`py_binary`. + shebang: {type}`str`, The shebang to use for the entry point python file. + Defaults to empty string. **kwargs: Extra parameters forwarded to `binary_rule`. """ main = "rules_python_entry_point_{}.py".format(name) @@ -81,6 +84,7 @@ def py_console_script_binary( out = main, console_script = script, console_script_guess = name, + shebang = shebang, visibility = ["//visibility:private"], ) diff --git a/python/private/py_console_script_gen.bzl b/python/private/py_console_script_gen.bzl index 7dd4dd2dad..de016036b2 100644 --- a/python/private/py_console_script_gen.bzl +++ b/python/private/py_console_script_gen.bzl @@ -42,6 +42,7 @@ def _py_console_script_gen_impl(ctx): args = ctx.actions.args() args.add("--console-script", ctx.attr.console_script) args.add("--console-script-guess", ctx.attr.console_script_guess) + args.add("--shebang", ctx.attr.shebang) args.add(entry_points_txt) args.add(ctx.outputs.out) @@ -81,6 +82,10 @@ py_console_script_gen = rule( doc = "Output file location.", mandatory = True, ), + "shebang": attr.string( + doc = "The shebang to use for the entry point python file.", + default = "", + ), "_tool": attr.label( default = ":py_console_script_gen_py", executable = True, diff --git a/python/private/py_console_script_gen.py b/python/private/py_console_script_gen.py index ffc4e81b3a..4b4f2f6986 100644 --- a/python/private/py_console_script_gen.py +++ b/python/private/py_console_script_gen.py @@ -44,7 +44,7 @@ _ENTRY_POINTS_TXT = "entry_points.txt" _TEMPLATE = """\ -import sys +{shebang}import sys # See @rules_python//python/private:py_console_script_gen.py for explanation if getattr(sys.flags, "safe_path", False): @@ -87,6 +87,7 @@ def run( out: pathlib.Path, console_script: str, console_script_guess: str, + shebang: str, ): """Run the generator @@ -94,6 +95,8 @@ def run( entry_points: The entry_points.txt file to be parsed. out: The output file. console_script: The console_script entry in the entry_points.txt file. + console_script_guess: The string used for guessing the console_script if it is not provided. + shebang: The shebang to use for the entry point python file. Defaults to empty string (no shebang). """ config = EntryPointsParser() config.read(entry_points) @@ -136,6 +139,7 @@ def run( with open(out, "w") as f: f.write( _TEMPLATE.format( + shebang=f"{shebang}\n" if shebang else "", module=module, attr=attr, entry_point=entry_point, @@ -154,6 +158,10 @@ def main(): required=True, help="The string used for guessing the console_script if it is not provided.", ) + parser.add_argument( + "--shebang", + help="The shebang to use for the entry point python file.", + ) parser.add_argument( "entry_points", metavar="ENTRY_POINTS_TXT", @@ -173,6 +181,7 @@ def main(): out=args.out, console_script=args.console_script, console_script_guess=args.console_script_guess, + shebang=args.shebang, ) diff --git a/tests/entry_points/py_console_script_gen_test.py b/tests/entry_points/py_console_script_gen_test.py index a5fceb67f9..1bbf5fbf25 100644 --- a/tests/entry_points/py_console_script_gen_test.py +++ b/tests/entry_points/py_console_script_gen_test.py @@ -47,6 +47,7 @@ def test_no_console_scripts_error(self): out=outfile, console_script=None, console_script_guess="", + shebang="", ) self.assertEqual( @@ -76,6 +77,7 @@ def test_no_entry_point_selected_error(self): out=outfile, console_script=None, console_script_guess="bar-baz", + shebang="", ) self.assertEqual( @@ -106,6 +108,7 @@ def test_incorrect_entry_point(self): out=outfile, console_script="baz", console_script_guess="", + shebang="", ) self.assertEqual( @@ -134,6 +137,7 @@ def test_a_single_entry_point(self): out=out, console_script=None, console_script_guess="foo", + shebang="", ) got = out.read_text() @@ -185,6 +189,7 @@ def test_a_second_entry_point_class_method(self): out=out, console_script="bar", console_script_guess="", + shebang="", ) got = out.read_text() @@ -192,6 +197,35 @@ def test_a_second_entry_point_class_method(self): self.assertRegex(got, "from foo\.baz import Bar") self.assertRegex(got, "sys\.exit\(Bar\.baz\(\)\)") + def test_shebang_included(self): + with tempfile.TemporaryDirectory() as tmpdir: + tmpdir = pathlib.Path(tmpdir) + given_contents = ( + textwrap.dedent( + """ + [console_scripts] + foo = foo.bar:baz + """ + ).strip() + + "\n" + ) + entry_points = tmpdir / "entry_points.txt" + entry_points.write_text(given_contents) + out = tmpdir / "foo.py" + + shebang = "#!/usr/bin/env python3" + run( + entry_points=entry_points, + out=out, + console_script=None, + console_script_guess="foo", + shebang=shebang, + ) + + got = out.read_text() + + self.assertTrue(got.startswith(shebang + "\n")) + if __name__ == "__main__": unittest.main() From b40d96aba36d675c60b03424aa22f31c09e0ea4f Mon Sep 17 00:00:00 2001 From: Kayce Basques Date: Mon, 26 May 2025 06:36:58 -0700 Subject: [PATCH 256/922] fix: update the stub type alias names (#2929) Co-authored-by: Kayce Basques --- tools/precompiler/precompiler.py | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/tools/precompiler/precompiler.py b/tools/precompiler/precompiler.py index 310f2eb097..e7c693c195 100644 --- a/tools/precompiler/precompiler.py +++ b/tools/precompiler/precompiler.py @@ -68,12 +68,12 @@ def _compile(options: "argparse.Namespace") -> None: # A stub type alias for readability. # See the Bazel WorkRequest object definition: # https://github.com/bazelbuild/bazel/blob/master/src/main/protobuf/worker_protocol.proto -JsonWorkerRequest = object +JsonWorkRequest = object # A stub type alias for readability. # See the Bazel WorkResponse object definition: # https://github.com/bazelbuild/bazel/blob/master/src/main/protobuf/worker_protocol.proto -JsonWorkerResponse = object +JsonWorkResponse = object class _SerialPersistentWorker: From dce5120249f62bd04d2bafa48fd053732854e1ad Mon Sep 17 00:00:00 2001 From: Ignas Anikevicius <240938+aignas@users.noreply.github.com> Date: Wed, 28 May 2025 00:47:49 +0900 Subject: [PATCH 257/922] refactor: reimplement writing namespace pkgs in Starlark (#2882) With this PR I would like to facilitate the implementation of the venv layouts because we can in theory take the `srcs` and the `data` within the `py_library` and then use the `expand_template` to write the extra Python files if the namespace_pkgs flag is enabled. The old Python code has been removed and the extra generated files are written out with `bazel_skylib` `copy_file`. The implicit `namespace_pkg` init files are included to `py_library` if the `site-packages` config flag is set to false and I think this may help with continuing the implementation, but it currently is still not working as expected (see comment). Work towards #2156 --- python/config_settings/BUILD.bazel | 9 + python/private/pypi/BUILD.bazel | 5 + python/private/pypi/namespace_pkg_tmpl.py | 2 + python/private/pypi/namespace_pkgs.bzl | 83 ++++++++ .../private/pypi/whl_installer/arguments.py | 5 - .../pypi/whl_installer/wheel_installer.py | 32 +-- python/private/pypi/whl_library.bzl | 8 +- python/private/pypi/whl_library_targets.bzl | 50 +++-- tests/pypi/namespace_pkgs/BUILD.bazel | 5 + .../namespace_pkgs/namespace_pkgs_tests.bzl | 167 +++++++++++++++ tests/pypi/whl_installer/BUILD.bazel | 11 - tests/pypi/whl_installer/arguments_test.py | 1 - .../pypi/whl_installer/namespace_pkgs_test.py | 192 ------------------ .../whl_installer/wheel_installer_test.py | 1 - 14 files changed, 311 insertions(+), 260 deletions(-) create mode 100644 python/private/pypi/namespace_pkg_tmpl.py create mode 100644 python/private/pypi/namespace_pkgs.bzl create mode 100644 tests/pypi/namespace_pkgs/BUILD.bazel create mode 100644 tests/pypi/namespace_pkgs/namespace_pkgs_tests.bzl delete mode 100644 tests/pypi/whl_installer/namespace_pkgs_test.py diff --git a/python/config_settings/BUILD.bazel b/python/config_settings/BUILD.bazel index 1772a3403e..ee15828fa5 100644 --- a/python/config_settings/BUILD.bazel +++ b/python/config_settings/BUILD.bazel @@ -217,6 +217,15 @@ string_flag( visibility = ["//visibility:public"], ) +config_setting( + name = "is_venvs_site_packages", + flag_values = { + ":venvs_site_packages": VenvsSitePackages.YES, + }, + # NOTE: Only public because it is used in whl_library repos. + visibility = ["//visibility:public"], +) + define_pypi_internal_flags( name = "define_pypi_internal_flags", ) diff --git a/python/private/pypi/BUILD.bazel b/python/private/pypi/BUILD.bazel index 84e0535289..e9036c3013 100644 --- a/python/private/pypi/BUILD.bazel +++ b/python/private/pypi/BUILD.bazel @@ -18,6 +18,11 @@ package(default_visibility = ["//:__subpackages__"]) licenses(["notice"]) +exports_files( + srcs = ["namespace_pkg_tmpl.py"], + visibility = ["//visibility:public"], +) + filegroup( name = "distribution", srcs = glob( diff --git a/python/private/pypi/namespace_pkg_tmpl.py b/python/private/pypi/namespace_pkg_tmpl.py new file mode 100644 index 0000000000..a21b846e76 --- /dev/null +++ b/python/private/pypi/namespace_pkg_tmpl.py @@ -0,0 +1,2 @@ +# __path__ manipulation added by bazel-contrib/rules_python to support namespace pkgs. +__path__ = __import__("pkgutil").extend_path(__path__, __name__) diff --git a/python/private/pypi/namespace_pkgs.bzl b/python/private/pypi/namespace_pkgs.bzl new file mode 100644 index 0000000000..bf4689a5ea --- /dev/null +++ b/python/private/pypi/namespace_pkgs.bzl @@ -0,0 +1,83 @@ +"""Utilities to get where we should write namespace pkg paths.""" + +load("@bazel_skylib//rules:copy_file.bzl", "copy_file") + +_ext = struct( + py = ".py", + pyd = ".pyd", + so = ".so", + pyc = ".pyc", +) + +_TEMPLATE = Label("//python/private/pypi:namespace_pkg_tmpl.py") + +def _add_all(dirname, dirs): + dir_path = "." + for dir_name in dirname.split("/"): + dir_path = "{}/{}".format(dir_path, dir_name) + dirs[dir_path[2:]] = None + +def get_files(*, srcs, ignored_dirnames = [], root = None): + """Get the list of filenames to write the namespace pkg files. + + Args: + srcs: {type}`src` a list of files to be passed to {bzl:obj}`py_library` + as `srcs` and `data`. This is usually a result of a {obj}`glob`. + ignored_dirnames: {type}`str` a list of patterns to ignore. + root: {type}`str` the prefix to use as the root. + + Returns: + {type}`src` a list of paths to write the namespace pkg `__init__.py` file. + """ + dirs = {} + ignored = {i: None for i in ignored_dirnames} + + if root: + _add_all(root, ignored) + + for file in srcs: + dirname, _, filename = file.rpartition("/") + + if filename == "__init__.py": + ignored[dirname] = None + dirname, _, _ = dirname.rpartition("/") + elif filename.endswith(_ext.py): + pass + elif filename.endswith(_ext.pyc): + pass + elif filename.endswith(_ext.pyd): + pass + elif filename.endswith(_ext.so): + pass + else: + continue + + if dirname in dirs or not dirname: + continue + + _add_all(dirname, dirs) + + return sorted([d for d in dirs if d not in ignored]) + +def create_inits(**kwargs): + """Create init files and return the list to be included `py_library` srcs. + + Args: + **kwargs: passed to {obj}`get_files`. + + Returns: + {type}`list[str]` to be included as part of `py_library`. + """ + srcs = [] + for out in get_files(**kwargs): + src = "{}/__init__.py".format(out) + srcs.append(srcs) + + copy_file( + name = "_cp_{}_namespace".format(out), + src = _TEMPLATE, + out = src, + **kwargs + ) + + return srcs diff --git a/python/private/pypi/whl_installer/arguments.py b/python/private/pypi/whl_installer/arguments.py index ea609bef9d..57dae45ae9 100644 --- a/python/private/pypi/whl_installer/arguments.py +++ b/python/private/pypi/whl_installer/arguments.py @@ -57,11 +57,6 @@ def parser(**kwargs: Any) -> argparse.ArgumentParser: action="store", help="Additional data exclusion parameters to add to the pip packages BUILD file.", ) - parser.add_argument( - "--enable_implicit_namespace_pkgs", - action="store_true", - help="Disables conversion of implicit namespace packages into pkg-util style packages.", - ) parser.add_argument( "--environment", action="store", diff --git a/python/private/pypi/whl_installer/wheel_installer.py b/python/private/pypi/whl_installer/wheel_installer.py index 600d45f940..a6a9dd0429 100644 --- a/python/private/pypi/whl_installer/wheel_installer.py +++ b/python/private/pypi/whl_installer/wheel_installer.py @@ -27,7 +27,7 @@ from pip._vendor.packaging.utils import canonicalize_name -from python.private.pypi.whl_installer import arguments, namespace_pkgs, wheel +from python.private.pypi.whl_installer import arguments, wheel def _configure_reproducible_wheels() -> None: @@ -77,35 +77,10 @@ def _parse_requirement_for_extra( return None, None -def _setup_namespace_pkg_compatibility(wheel_dir: str) -> None: - """Converts native namespace packages to pkgutil-style packages - - Namespace packages can be created in one of three ways. They are detailed here: - https://packaging.python.org/guides/packaging-namespace-packages/#creating-a-namespace-package - - 'pkgutil-style namespace packages' (2) and 'pkg_resources-style namespace packages' (3) works in Bazel, but - 'native namespace packages' (1) do not. - - We ensure compatibility with Bazel of method 1 by converting them into method 2. - - Args: - wheel_dir: the directory of the wheel to convert - """ - - namespace_pkg_dirs = namespace_pkgs.implicit_namespace_packages( - wheel_dir, - ignored_dirnames=["%s/bin" % wheel_dir], - ) - - for ns_pkg_dir in namespace_pkg_dirs: - namespace_pkgs.add_pkgutil_style_namespace_pkg_init(ns_pkg_dir) - - def _extract_wheel( wheel_file: str, extras: Dict[str, Set[str]], enable_pipstar: bool, - enable_implicit_namespace_pkgs: bool, platforms: List[wheel.Platform], installation_dir: Path = Path("."), ) -> None: @@ -116,15 +91,11 @@ def _extract_wheel( installation_dir: the destination directory for installation of the wheel. extras: a list of extras to add as dependencies for the installed wheel enable_pipstar: if true, turns off certain operations. - enable_implicit_namespace_pkgs: if true, disables conversion of implicit namespace packages and will unzip as-is """ whl = wheel.Wheel(wheel_file) whl.unzip(installation_dir) - if not enable_implicit_namespace_pkgs: - _setup_namespace_pkg_compatibility(installation_dir) - metadata = { "entry_points": [ { @@ -168,7 +139,6 @@ def main() -> None: wheel_file=whl, extras=extras, enable_pipstar=args.enable_pipstar, - enable_implicit_namespace_pkgs=args.enable_implicit_namespace_pkgs, platforms=arguments.get_platforms(args), ) return diff --git a/python/private/pypi/whl_library.bzl b/python/private/pypi/whl_library.bzl index 17ee3d3cfe..c271449b3d 100644 --- a/python/private/pypi/whl_library.bzl +++ b/python/private/pypi/whl_library.bzl @@ -173,9 +173,6 @@ def _parse_optional_attrs(rctx, args, extra_pip_args = None): json.encode(struct(arg = rctx.attr.pip_data_exclude)), ] - if rctx.attr.enable_implicit_namespace_pkgs: - args.append("--enable_implicit_namespace_pkgs") - env = {} if rctx.attr.environment != None: for key, value in rctx.attr.environment.items(): @@ -389,6 +386,8 @@ def _whl_library_impl(rctx): metadata_name = metadata.name, metadata_version = metadata.version, requires_dist = metadata.requires_dist, + # TODO @aignas 2025-05-17: maybe have a build flag for this instead + enable_implicit_namespace_pkgs = rctx.attr.enable_implicit_namespace_pkgs, # TODO @aignas 2025-04-14: load through the hub: annotation = None if not rctx.attr.annotation else struct(**json.decode(rctx.read(rctx.attr.annotation))), data_exclude = rctx.attr.pip_data_exclude, @@ -457,6 +456,8 @@ def _whl_library_impl(rctx): name = whl_path.basename, dep_template = rctx.attr.dep_template or "@{}{{name}}//:{{target}}".format(rctx.attr.repo_prefix), entry_points = entry_points, + # TODO @aignas 2025-05-17: maybe have a build flag for this instead + enable_implicit_namespace_pkgs = rctx.attr.enable_implicit_namespace_pkgs, # TODO @aignas 2025-04-14: load through the hub: dependencies = metadata["deps"], dependencies_by_platform = metadata["deps_by_platform"], @@ -580,7 +581,6 @@ attr makes `extra_pip_args` and `download_only` ignored.""", Label("//python/private/pypi/whl_installer:wheel.py"), Label("//python/private/pypi/whl_installer:wheel_installer.py"), Label("//python/private/pypi/whl_installer:arguments.py"), - Label("//python/private/pypi/whl_installer:namespace_pkgs.py"), ] + record_files.values(), ), "_rule_name": attr.string(default = "whl_library"), diff --git a/python/private/pypi/whl_library_targets.bzl b/python/private/pypi/whl_library_targets.bzl index e0c03a1505..3529566c49 100644 --- a/python/private/pypi/whl_library_targets.bzl +++ b/python/private/pypi/whl_library_targets.bzl @@ -30,6 +30,7 @@ load( "WHEEL_FILE_IMPL_LABEL", "WHEEL_FILE_PUBLIC_LABEL", ) +load(":namespace_pkgs.bzl", "create_inits") load(":pep508_deps.bzl", "deps") def whl_library_targets_from_requires( @@ -113,6 +114,7 @@ def whl_library_targets( copy_executables = {}, entry_points = {}, native = native, + enable_implicit_namespace_pkgs = False, rules = struct( copy_file = copy_file, py_binary = py_binary, @@ -153,6 +155,8 @@ def whl_library_targets( data: {type}`list[str]` A list of labels to include as part of the `data` attribute in `py_library`. entry_points: {type}`dict[str, str]` The mapping between the script name and the python file to use. DEPRECATED. + enable_implicit_namespace_pkgs: {type}`boolean` generate __init__.py + files for namespace pkgs. native: {type}`native` The native struct for overriding in tests. rules: {type}`struct` A struct with references to rules for creating targets. """ @@ -293,6 +297,14 @@ def whl_library_targets( ) if hasattr(rules, "py_library"): + srcs = native.glob( + ["site-packages/**/*.py"], + exclude = srcs_exclude, + # Empty sources are allowed to support wheels that don't have any + # pure-Python code, e.g. pymssql, which is written in Cython. + allow_empty = True, + ) + # NOTE: pyi files should probably be excluded because they're carried # by the pyi_srcs attribute. However, historical behavior included # them in data and some tools currently rely on that. @@ -309,23 +321,31 @@ def whl_library_targets( if item not in _data_exclude: _data_exclude.append(item) + data = data + native.glob( + ["site-packages/**/*"], + exclude = _data_exclude, + ) + + pyi_srcs = native.glob( + ["site-packages/**/*.pyi"], + allow_empty = True, + ) + + if enable_implicit_namespace_pkgs: + srcs = srcs + getattr(native, "select", select)({ + Label("//python/config_settings:is_venvs_site_packages"): [], + "//conditions:default": create_inits( + srcs = srcs + data + pyi_srcs, + ignore_dirnames = [], # If you need to ignore certain folders, you can patch rules_python here to do so. + root = "site-packages", + ), + }) + rules.py_library( name = py_library_label, - srcs = native.glob( - ["site-packages/**/*.py"], - exclude = srcs_exclude, - # Empty sources are allowed to support wheels that don't have any - # pure-Python code, e.g. pymssql, which is written in Cython. - allow_empty = True, - ), - pyi_srcs = native.glob( - ["site-packages/**/*.pyi"], - allow_empty = True, - ), - data = data + native.glob( - ["site-packages/**/*"], - exclude = _data_exclude, - ), + srcs = srcs, + pyi_srcs = pyi_srcs, + data = data, # This makes this directory a top-level in the python import # search path for anything that depends on this. imports = ["site-packages"], diff --git a/tests/pypi/namespace_pkgs/BUILD.bazel b/tests/pypi/namespace_pkgs/BUILD.bazel new file mode 100644 index 0000000000..57f7962524 --- /dev/null +++ b/tests/pypi/namespace_pkgs/BUILD.bazel @@ -0,0 +1,5 @@ +load(":namespace_pkgs_tests.bzl", "namespace_pkgs_test_suite") + +namespace_pkgs_test_suite( + name = "namespace_pkgs_tests", +) diff --git a/tests/pypi/namespace_pkgs/namespace_pkgs_tests.bzl b/tests/pypi/namespace_pkgs/namespace_pkgs_tests.bzl new file mode 100644 index 0000000000..7ac938ff17 --- /dev/null +++ b/tests/pypi/namespace_pkgs/namespace_pkgs_tests.bzl @@ -0,0 +1,167 @@ +"" + +load("@rules_testing//lib:analysis_test.bzl", "test_suite") +load("//python/private/pypi:namespace_pkgs.bzl", "get_files") # buildifier: disable=bzl-visibility + +_tests = [] + +def test_in_current_dir(env): + srcs = [ + "foo/bar/biz.py", + "foo/bee/boo.py", + "foo/buu/__init__.py", + "foo/buu/bii.py", + ] + got = get_files(srcs = srcs) + expected = [ + "foo", + "foo/bar", + "foo/bee", + ] + env.expect.that_collection(got).contains_exactly(expected) + +_tests.append(test_in_current_dir) + +def test_find_correct_namespace_packages(env): + srcs = [ + "nested/root/foo/bar/biz.py", + "nested/root/foo/bee/boo.py", + "nested/root/foo/buu/__init__.py", + "nested/root/foo/buu/bii.py", + ] + + got = get_files(srcs = srcs, root = "nested/root") + expected = [ + "nested/root/foo", + "nested/root/foo/bar", + "nested/root/foo/bee", + ] + env.expect.that_collection(got).contains_exactly(expected) + +_tests.append(test_find_correct_namespace_packages) + +def test_ignores_empty_directories(_): + # because globs do not add directories, this test is not needed + pass + +_tests.append(test_ignores_empty_directories) + +def test_empty_case(env): + srcs = [ + "foo/__init__.py", + "foo/bar/__init__.py", + "foo/bar/biz.py", + ] + + got = get_files(srcs = srcs) + expected = [] + env.expect.that_collection(got).contains_exactly(expected) + +_tests.append(test_empty_case) + +def test_ignores_non_module_files_in_directories(env): + srcs = [ + "foo/__init__.pyi", + "foo/py.typed", + ] + + got = get_files(srcs = srcs) + expected = [] + env.expect.that_collection(got).contains_exactly(expected) + +_tests.append(test_ignores_non_module_files_in_directories) + +def test_parent_child_relationship_of_namespace_pkgs(env): + srcs = [ + "foo/bar/biff/my_module.py", + "foo/bar/biff/another_module.py", + ] + + got = get_files(srcs = srcs) + expected = [ + "foo", + "foo/bar", + "foo/bar/biff", + ] + env.expect.that_collection(got).contains_exactly(expected) + +_tests.append(test_parent_child_relationship_of_namespace_pkgs) + +def test_parent_child_relationship_of_namespace_and_standard_pkgs(env): + srcs = [ + "foo/bar/biff/__init__.py", + "foo/bar/biff/another_module.py", + ] + + got = get_files(srcs = srcs) + expected = [ + "foo", + "foo/bar", + ] + env.expect.that_collection(got).contains_exactly(expected) + +_tests.append(test_parent_child_relationship_of_namespace_and_standard_pkgs) + +def test_parent_child_relationship_of_namespace_and_nested_standard_pkgs(env): + srcs = [ + "foo/bar/__init__.py", + "foo/bar/biff/another_module.py", + "foo/bar/biff/__init__.py", + "foo/bar/boof/big_module.py", + "foo/bar/boof/__init__.py", + "fim/in_a_ns_pkg.py", + ] + + got = get_files(srcs = srcs) + expected = [ + "foo", + "fim", + ] + env.expect.that_collection(got).contains_exactly(expected) + +_tests.append(test_parent_child_relationship_of_namespace_and_nested_standard_pkgs) + +def test_recognized_all_nonstandard_module_types(env): + srcs = [ + "ayy/my_module.pyc", + "bee/ccc/dee/eee.so", + "eff/jee/aych.pyd", + ] + + expected = [ + "ayy", + "bee", + "bee/ccc", + "bee/ccc/dee", + "eff", + "eff/jee", + ] + got = get_files(srcs = srcs) + env.expect.that_collection(got).contains_exactly(expected) + +_tests.append(test_recognized_all_nonstandard_module_types) + +def test_skips_ignored_directories(env): + srcs = [ + "root/foo/boo/my_module.py", + "root/foo/bar/another_module.py", + ] + + expected = [ + "root/foo", + "root/foo/bar", + ] + got = get_files( + srcs = srcs, + ignored_dirnames = ["root/foo/boo"], + root = "root", + ) + env.expect.that_collection(got).contains_exactly(expected) + +_tests.append(test_skips_ignored_directories) + +def namespace_pkgs_test_suite(name): + test_suite( + name = name, + basic_tests = _tests, + ) diff --git a/tests/pypi/whl_installer/BUILD.bazel b/tests/pypi/whl_installer/BUILD.bazel index 040e4d765f..060d2bce62 100644 --- a/tests/pypi/whl_installer/BUILD.bazel +++ b/tests/pypi/whl_installer/BUILD.bazel @@ -16,17 +16,6 @@ py_test( ], ) -py_test( - name = "namespace_pkgs_test", - size = "small", - srcs = [ - "namespace_pkgs_test.py", - ], - deps = [ - ":lib", - ], -) - py_test( name = "platform_test", size = "small", diff --git a/tests/pypi/whl_installer/arguments_test.py b/tests/pypi/whl_installer/arguments_test.py index 5538054a59..2352d8e48b 100644 --- a/tests/pypi/whl_installer/arguments_test.py +++ b/tests/pypi/whl_installer/arguments_test.py @@ -36,7 +36,6 @@ def test_arguments(self) -> None: self.assertIn("requirement", args_dict) self.assertIn("extra_pip_args", args_dict) self.assertEqual(args_dict["pip_data_exclude"], []) - self.assertEqual(args_dict["enable_implicit_namespace_pkgs"], False) self.assertEqual(args_dict["extra_pip_args"], extra_pip_args) def test_deserialize_structured_args(self) -> None: diff --git a/tests/pypi/whl_installer/namespace_pkgs_test.py b/tests/pypi/whl_installer/namespace_pkgs_test.py deleted file mode 100644 index fbbd50926a..0000000000 --- a/tests/pypi/whl_installer/namespace_pkgs_test.py +++ /dev/null @@ -1,192 +0,0 @@ -# Copyright 2023 The Bazel Authors. All rights reserved. -# -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -import os -import pathlib -import shutil -import tempfile -import unittest -from typing import Optional, Set - -from python.private.pypi.whl_installer import namespace_pkgs - - -class TempDir: - def __init__(self) -> None: - self.dir = tempfile.mkdtemp() - - def root(self) -> str: - return self.dir - - def add_dir(self, rel_path: str) -> None: - d = pathlib.Path(self.dir, rel_path) - d.mkdir(parents=True) - - def add_file(self, rel_path: str, contents: Optional[str] = None) -> None: - f = pathlib.Path(self.dir, rel_path) - f.parent.mkdir(parents=True, exist_ok=True) - if contents: - with open(str(f), "w") as writeable_f: - writeable_f.write(contents) - else: - f.touch() - - def remove(self) -> None: - shutil.rmtree(self.dir) - - -class TestImplicitNamespacePackages(unittest.TestCase): - def assertPathsEqual(self, actual: Set[pathlib.Path], expected: Set[str]) -> None: - self.assertEqual(actual, {pathlib.Path(p) for p in expected}) - - def test_in_current_directory(self) -> None: - directory = TempDir() - directory.add_file("foo/bar/biz.py") - directory.add_file("foo/bee/boo.py") - directory.add_file("foo/buu/__init__.py") - directory.add_file("foo/buu/bii.py") - cwd = os.getcwd() - os.chdir(directory.root()) - expected = { - "foo", - "foo/bar", - "foo/bee", - } - try: - actual = namespace_pkgs.implicit_namespace_packages(".") - self.assertPathsEqual(actual, expected) - finally: - os.chdir(cwd) - directory.remove() - - def test_finds_correct_namespace_packages(self) -> None: - directory = TempDir() - directory.add_file("foo/bar/biz.py") - directory.add_file("foo/bee/boo.py") - directory.add_file("foo/buu/__init__.py") - directory.add_file("foo/buu/bii.py") - - expected = { - directory.root() + "/foo", - directory.root() + "/foo/bar", - directory.root() + "/foo/bee", - } - actual = namespace_pkgs.implicit_namespace_packages(directory.root()) - self.assertPathsEqual(actual, expected) - - def test_ignores_empty_directories(self) -> None: - directory = TempDir() - directory.add_file("foo/bar/biz.py") - directory.add_dir("foo/cat") - - expected = { - directory.root() + "/foo", - directory.root() + "/foo/bar", - } - actual = namespace_pkgs.implicit_namespace_packages(directory.root()) - self.assertPathsEqual(actual, expected) - - def test_empty_case(self) -> None: - directory = TempDir() - directory.add_file("foo/__init__.py") - directory.add_file("foo/bar/__init__.py") - directory.add_file("foo/bar/biz.py") - - actual = namespace_pkgs.implicit_namespace_packages(directory.root()) - self.assertEqual(actual, set()) - - def test_ignores_non_module_files_in_directories(self) -> None: - directory = TempDir() - directory.add_file("foo/__init__.pyi") - directory.add_file("foo/py.typed") - - actual = namespace_pkgs.implicit_namespace_packages(directory.root()) - self.assertEqual(actual, set()) - - def test_parent_child_relationship_of_namespace_pkgs(self): - directory = TempDir() - directory.add_file("foo/bar/biff/my_module.py") - directory.add_file("foo/bar/biff/another_module.py") - - expected = { - directory.root() + "/foo", - directory.root() + "/foo/bar", - directory.root() + "/foo/bar/biff", - } - actual = namespace_pkgs.implicit_namespace_packages(directory.root()) - self.assertPathsEqual(actual, expected) - - def test_parent_child_relationship_of_namespace_and_standard_pkgs(self): - directory = TempDir() - directory.add_file("foo/bar/biff/__init__.py") - directory.add_file("foo/bar/biff/another_module.py") - - expected = { - directory.root() + "/foo", - directory.root() + "/foo/bar", - } - actual = namespace_pkgs.implicit_namespace_packages(directory.root()) - self.assertPathsEqual(actual, expected) - - def test_parent_child_relationship_of_namespace_and_nested_standard_pkgs(self): - directory = TempDir() - directory.add_file("foo/bar/__init__.py") - directory.add_file("foo/bar/biff/another_module.py") - directory.add_file("foo/bar/biff/__init__.py") - directory.add_file("foo/bar/boof/big_module.py") - directory.add_file("foo/bar/boof/__init__.py") - directory.add_file("fim/in_a_ns_pkg.py") - - expected = { - directory.root() + "/foo", - directory.root() + "/fim", - } - actual = namespace_pkgs.implicit_namespace_packages(directory.root()) - self.assertPathsEqual(actual, expected) - - def test_recognized_all_nonstandard_module_types(self): - directory = TempDir() - directory.add_file("ayy/my_module.pyc") - directory.add_file("bee/ccc/dee/eee.so") - directory.add_file("eff/jee/aych.pyd") - - expected = { - directory.root() + "/ayy", - directory.root() + "/bee", - directory.root() + "/bee/ccc", - directory.root() + "/bee/ccc/dee", - directory.root() + "/eff", - directory.root() + "/eff/jee", - } - actual = namespace_pkgs.implicit_namespace_packages(directory.root()) - self.assertPathsEqual(actual, expected) - - def test_skips_ignored_directories(self): - directory = TempDir() - directory.add_file("foo/boo/my_module.py") - directory.add_file("foo/bar/another_module.py") - - expected = { - directory.root() + "/foo", - directory.root() + "/foo/bar", - } - actual = namespace_pkgs.implicit_namespace_packages( - directory.root(), - ignored_dirnames=[directory.root() + "/foo/boo"], - ) - self.assertPathsEqual(actual, expected) - - -if __name__ == "__main__": - unittest.main() diff --git a/tests/pypi/whl_installer/wheel_installer_test.py b/tests/pypi/whl_installer/wheel_installer_test.py index ef5a2483ab..7040b0cfd8 100644 --- a/tests/pypi/whl_installer/wheel_installer_test.py +++ b/tests/pypi/whl_installer/wheel_installer_test.py @@ -70,7 +70,6 @@ def test_wheel_exists(self) -> None: Path(self.wheel_path), installation_dir=Path(self.wheel_dir), extras={}, - enable_implicit_namespace_pkgs=False, platforms=[], enable_pipstar=False, ) From c0415c67e6f9c0951176354e0256a55e85e475aa Mon Sep 17 00:00:00 2001 From: Ignas Anikevicius <240938+aignas@users.noreply.github.com> Date: Wed, 28 May 2025 00:54:48 +0900 Subject: [PATCH 258/922] cleanup(pycross): remove the partially migrated code (#2906) The migration effort has stalled and we closed the initiative. #1360 --- MODULE.bazel | 1 - WORKSPACE | 13 +- python/private/internal_dev_deps.bzl | 12 -- ...d-new-file-for-testing-patch-support.patch | 17 -- tests/pycross/BUILD.bazel | 64 ------ .../pycross/patched_py_wheel_library_test.py | 40 ---- tests/pycross/py_wheel_library_test.py | 46 ---- third_party/rules_pycross/LICENSE | 201 ------------------ .../rules_pycross/pycross/private/BUILD.bazel | 14 -- .../pycross/private/providers.bzl | 32 --- .../pycross/private/tools/BUILD.bazel | 26 --- .../pycross/private/tools/wheel_installer.py | 196 ----------------- .../pycross/private/wheel_library.bzl | 174 --------------- 13 files changed, 1 insertion(+), 835 deletions(-) delete mode 100644 tests/pycross/0001-Add-new-file-for-testing-patch-support.patch delete mode 100644 tests/pycross/BUILD.bazel delete mode 100644 tests/pycross/patched_py_wheel_library_test.py delete mode 100644 tests/pycross/py_wheel_library_test.py delete mode 100644 third_party/rules_pycross/LICENSE delete mode 100644 third_party/rules_pycross/pycross/private/BUILD.bazel delete mode 100644 third_party/rules_pycross/pycross/private/providers.bzl delete mode 100644 third_party/rules_pycross/pycross/private/tools/BUILD.bazel delete mode 100644 third_party/rules_pycross/pycross/private/tools/wheel_installer.py delete mode 100644 third_party/rules_pycross/pycross/private/wheel_library.bzl diff --git a/MODULE.bazel b/MODULE.bazel index d0f7cc4afa..fa24ed04ba 100644 --- a/MODULE.bazel +++ b/MODULE.bazel @@ -102,7 +102,6 @@ use_repo( internal_dev_deps, "buildkite_config", "rules_python_runtime_env_tc_info", - "wheel_for_testing", ) # Add gazelle plugin so that we can run the gazelle example as an e2e integration diff --git a/WORKSPACE b/WORKSPACE index 3ad83ca04b..dddc5105ed 100644 --- a/WORKSPACE +++ b/WORKSPACE @@ -78,7 +78,7 @@ python_register_multi_toolchains( python_versions = PYTHON_VERSIONS, ) -load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive", "http_file") +load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive") # Used for Bazel CI http_archive( @@ -155,14 +155,3 @@ pip_parse( load("@dev_pip//:requirements.bzl", docs_install_deps = "install_deps") docs_install_deps() - -# This wheel is purely here to validate the wheel extraction code. It's not -# intended for anything else. -http_file( - name = "wheel_for_testing", - downloaded_file_path = "numpy-1.25.2-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", - sha256 = "0d60fbae8e0019865fc4784745814cff1c421df5afee233db6d88ab4f14655a2", - urls = [ - "https://files.pythonhosted.org/packages/50/67/3e966d99a07d60a21a21d7ec016e9e4c2642a86fea251ec68677daf71d4d/numpy-1.25.2-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", - ], -) diff --git a/python/private/internal_dev_deps.bzl b/python/private/internal_dev_deps.bzl index 4f2cca0b42..600c934ace 100644 --- a/python/private/internal_dev_deps.bzl +++ b/python/private/internal_dev_deps.bzl @@ -14,23 +14,11 @@ """Module extension for internal dev_dependency=True setup.""" load("@bazel_ci_rules//:rbe_repo.bzl", "rbe_preconfig") -load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_file") load(":runtime_env_repo.bzl", "runtime_env_repo") def _internal_dev_deps_impl(mctx): _ = mctx # @unused - # This wheel is purely here to validate the wheel extraction code. It's not - # intended for anything else. - http_file( - name = "wheel_for_testing", - downloaded_file_path = "numpy-1.25.2-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", - sha256 = "0d60fbae8e0019865fc4784745814cff1c421df5afee233db6d88ab4f14655a2", - urls = [ - "https://files.pythonhosted.org/packages/50/67/3e966d99a07d60a21a21d7ec016e9e4c2642a86fea251ec68677daf71d4d/numpy-1.25.2-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", - ], - ) - # Creates a default toolchain config for RBE. # Use this as is if you are using the rbe_ubuntu16_04 container, # otherwise refer to RBE docs. diff --git a/tests/pycross/0001-Add-new-file-for-testing-patch-support.patch b/tests/pycross/0001-Add-new-file-for-testing-patch-support.patch deleted file mode 100644 index fcbc3096ef..0000000000 --- a/tests/pycross/0001-Add-new-file-for-testing-patch-support.patch +++ /dev/null @@ -1,17 +0,0 @@ -From b2ebe6fe67ff48edaf2ae937d24b1f0b67c16f81 Mon Sep 17 00:00:00 2001 -From: Philipp Schrader -Date: Thu, 28 Sep 2023 09:02:44 -0700 -Subject: [PATCH] Add new file for testing patch support - ---- - site-packages/numpy/file_added_via_patch.txt | 1 + - 1 file changed, 1 insertion(+) - create mode 100644 site-packages/numpy/file_added_via_patch.txt - -diff --git a/site-packages/numpy/file_added_via_patch.txt b/site-packages/numpy/file_added_via_patch.txt -new file mode 100644 -index 0000000..9d947a4 ---- /dev/null -+++ b/site-packages/numpy/file_added_via_patch.txt -@@ -0,0 +1 @@ -+Hello from a patch! diff --git a/tests/pycross/BUILD.bazel b/tests/pycross/BUILD.bazel deleted file mode 100644 index e90b60e17e..0000000000 --- a/tests/pycross/BUILD.bazel +++ /dev/null @@ -1,64 +0,0 @@ -# Copyright 2023 The Bazel Authors. All rights reserved. -# -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -load("//python:py_test.bzl", "py_test") -load("//third_party/rules_pycross/pycross/private:wheel_library.bzl", "py_wheel_library") # buildifier: disable=bzl-visibility - -py_wheel_library( - name = "extracted_wheel_for_testing", - wheel = "@wheel_for_testing//file", -) - -py_test( - name = "py_wheel_library_test", - srcs = [ - "py_wheel_library_test.py", - ], - data = [ - ":extracted_wheel_for_testing", - ], - deps = [ - "//python/runfiles", - ], -) - -py_wheel_library( - name = "patched_extracted_wheel_for_testing", - patch_args = [ - "-p1", - ], - patch_tool = "patch", - patches = [ - "0001-Add-new-file-for-testing-patch-support.patch", - ], - target_compatible_with = select({ - # We don't have `patch` available on the Windows CI machines. - "@platforms//os:windows": ["@platforms//:incompatible"], - "//conditions:default": [], - }), - wheel = "@wheel_for_testing//file", -) - -py_test( - name = "patched_py_wheel_library_test", - srcs = [ - "patched_py_wheel_library_test.py", - ], - data = [ - ":patched_extracted_wheel_for_testing", - ], - deps = [ - "//python/runfiles", - ], -) diff --git a/tests/pycross/patched_py_wheel_library_test.py b/tests/pycross/patched_py_wheel_library_test.py deleted file mode 100644 index e1b404a0ef..0000000000 --- a/tests/pycross/patched_py_wheel_library_test.py +++ /dev/null @@ -1,40 +0,0 @@ -# Copyright 2023 The Bazel Authors. All rights reserved. -# -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -import unittest -from pathlib import Path - -from python.runfiles import runfiles - -RUNFILES = runfiles.Create() - - -class TestPyWheelLibrary(unittest.TestCase): - def setUp(self): - self.extraction_dir = Path( - RUNFILES.Rlocation( - "rules_python/tests/pycross/patched_extracted_wheel_for_testing" - ) - ) - self.assertTrue(self.extraction_dir.exists(), self.extraction_dir) - self.assertTrue(self.extraction_dir.is_dir(), self.extraction_dir) - - def test_patched_file_contents(self): - """Validate that the patch got applied correctly.""" - file = self.extraction_dir / "site-packages/numpy/file_added_via_patch.txt" - self.assertEqual(file.read_text(), "Hello from a patch!\n") - - -if __name__ == "__main__": - unittest.main() diff --git a/tests/pycross/py_wheel_library_test.py b/tests/pycross/py_wheel_library_test.py deleted file mode 100644 index 25d896a1ae..0000000000 --- a/tests/pycross/py_wheel_library_test.py +++ /dev/null @@ -1,46 +0,0 @@ -# Copyright 2023 The Bazel Authors. All rights reserved. -# -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -import unittest -from pathlib import Path - -from python.runfiles import runfiles - -RUNFILES = runfiles.Create() - - -class TestPyWheelLibrary(unittest.TestCase): - def setUp(self): - self.extraction_dir = Path( - RUNFILES.Rlocation("rules_python/tests/pycross/extracted_wheel_for_testing") - ) - self.assertTrue(self.extraction_dir.exists(), self.extraction_dir) - self.assertTrue(self.extraction_dir.is_dir(), self.extraction_dir) - - def test_file_presence(self): - """Validate that the basic file layout looks good.""" - for path in ( - "bin/f2py", - "site-packages/numpy.libs/libgfortran-daac5196.so.5.0.0", - "site-packages/numpy/dtypes.py", - "site-packages/numpy/core/_umath_tests.cpython-311-aarch64-linux-gnu.so", - ): - print(self.extraction_dir / path) - self.assertTrue( - (self.extraction_dir / path).exists(), f"{path} does not exist" - ) - - -if __name__ == "__main__": - unittest.main() diff --git a/third_party/rules_pycross/LICENSE b/third_party/rules_pycross/LICENSE deleted file mode 100644 index 261eeb9e9f..0000000000 --- a/third_party/rules_pycross/LICENSE +++ /dev/null @@ -1,201 +0,0 @@ - Apache License - Version 2.0, January 2004 - http://www.apache.org/licenses/ - - TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION - - 1. Definitions. - - "License" shall mean the terms and conditions for use, reproduction, - and distribution as defined by Sections 1 through 9 of this document. - - "Licensor" shall mean the copyright owner or entity authorized by - the copyright owner that is granting the License. - - "Legal Entity" shall mean the union of the acting entity and all - other entities that control, are controlled by, or are under common - control with that entity. For the purposes of this definition, - "control" means (i) the power, direct or indirect, to cause the - direction or management of such entity, whether by contract or - otherwise, or (ii) ownership of fifty percent (50%) or more of the - outstanding shares, or (iii) beneficial ownership of such entity. - - "You" (or "Your") shall mean an individual or Legal Entity - exercising permissions granted by this License. - - "Source" form shall mean the preferred form for making modifications, - including but not limited to software source code, documentation - source, and configuration files. - - "Object" form shall mean any form resulting from mechanical - transformation or translation of a Source form, including but - not limited to compiled object code, generated documentation, - and conversions to other media types. - - "Work" shall mean the work of authorship, whether in Source or - Object form, made available under the License, as indicated by a - copyright notice that is included in or attached to the work - (an example is provided in the Appendix below). - - "Derivative Works" shall mean any work, whether in Source or Object - form, that is based on (or derived from) the Work and for which the - editorial revisions, annotations, elaborations, or other modifications - represent, as a whole, an original work of authorship. For the purposes - of this License, Derivative Works shall not include works that remain - separable from, or merely link (or bind by name) to the interfaces of, - the Work and Derivative Works thereof. - - "Contribution" shall mean any work of authorship, including - the original version of the Work and any modifications or additions - to that Work or Derivative Works thereof, that is intentionally - submitted to Licensor for inclusion in the Work by the copyright owner - or by an individual or Legal Entity authorized to submit on behalf of - the copyright owner. For the purposes of this definition, "submitted" - means any form of electronic, verbal, or written communication sent - to the Licensor or its representatives, including but not limited to - communication on electronic mailing lists, source code control systems, - and issue tracking systems that are managed by, or on behalf of, the - Licensor for the purpose of discussing and improving the Work, but - excluding communication that is conspicuously marked or otherwise - designated in writing by the copyright owner as "Not a Contribution." - - "Contributor" shall mean Licensor and any individual or Legal Entity - on behalf of whom a Contribution has been received by Licensor and - subsequently incorporated within the Work. - - 2. Grant of Copyright License. Subject to the terms and conditions of - this License, each Contributor hereby grants to You a perpetual, - worldwide, non-exclusive, no-charge, royalty-free, irrevocable - copyright license to reproduce, prepare Derivative Works of, - publicly display, publicly perform, sublicense, and distribute the - Work and such Derivative Works in Source or Object form. - - 3. Grant of Patent License. Subject to the terms and conditions of - this License, each Contributor hereby grants to You a perpetual, - worldwide, non-exclusive, no-charge, royalty-free, irrevocable - (except as stated in this section) patent license to make, have made, - use, offer to sell, sell, import, and otherwise transfer the Work, - where such license applies only to those patent claims licensable - by such Contributor that are necessarily infringed by their - Contribution(s) alone or by combination of their Contribution(s) - with the Work to which such Contribution(s) was submitted. If You - institute patent litigation against any entity (including a - cross-claim or counterclaim in a lawsuit) alleging that the Work - or a Contribution incorporated within the Work constitutes direct - or contributory patent infringement, then any patent licenses - granted to You under this License for that Work shall terminate - as of the date such litigation is filed. - - 4. Redistribution. You may reproduce and distribute copies of the - Work or Derivative Works thereof in any medium, with or without - modifications, and in Source or Object form, provided that You - meet the following conditions: - - (a) You must give any other recipients of the Work or - Derivative Works a copy of this License; and - - (b) You must cause any modified files to carry prominent notices - stating that You changed the files; and - - (c) You must retain, in the Source form of any Derivative Works - that You distribute, all copyright, patent, trademark, and - attribution notices from the Source form of the Work, - excluding those notices that do not pertain to any part of - the Derivative Works; and - - (d) If the Work includes a "NOTICE" text file as part of its - distribution, then any Derivative Works that You distribute must - include a readable copy of the attribution notices contained - within such NOTICE file, excluding those notices that do not - pertain to any part of the Derivative Works, in at least one - of the following places: within a NOTICE text file distributed - as part of the Derivative Works; within the Source form or - documentation, if provided along with the Derivative Works; or, - within a display generated by the Derivative Works, if and - wherever such third-party notices normally appear. The contents - of the NOTICE file are for informational purposes only and - do not modify the License. You may add Your own attribution - notices within Derivative Works that You distribute, alongside - or as an addendum to the NOTICE text from the Work, provided - that such additional attribution notices cannot be construed - as modifying the License. - - You may add Your own copyright statement to Your modifications and - may provide additional or different license terms and conditions - for use, reproduction, or distribution of Your modifications, or - for any such Derivative Works as a whole, provided Your use, - reproduction, and distribution of the Work otherwise complies with - the conditions stated in this License. - - 5. Submission of Contributions. Unless You explicitly state otherwise, - any Contribution intentionally submitted for inclusion in the Work - by You to the Licensor shall be under the terms and conditions of - this License, without any additional terms or conditions. - Notwithstanding the above, nothing herein shall supersede or modify - the terms of any separate license agreement you may have executed - with Licensor regarding such Contributions. - - 6. Trademarks. This License does not grant permission to use the trade - names, trademarks, service marks, or product names of the Licensor, - except as required for reasonable and customary use in describing the - origin of the Work and reproducing the content of the NOTICE file. - - 7. Disclaimer of Warranty. Unless required by applicable law or - agreed to in writing, Licensor provides the Work (and each - Contributor provides its Contributions) on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or - implied, including, without limitation, any warranties or conditions - of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A - PARTICULAR PURPOSE. You are solely responsible for determining the - appropriateness of using or redistributing the Work and assume any - risks associated with Your exercise of permissions under this License. - - 8. Limitation of Liability. In no event and under no legal theory, - whether in tort (including negligence), contract, or otherwise, - unless required by applicable law (such as deliberate and grossly - negligent acts) or agreed to in writing, shall any Contributor be - liable to You for damages, including any direct, indirect, special, - incidental, or consequential damages of any character arising as a - result of this License or out of the use or inability to use the - Work (including but not limited to damages for loss of goodwill, - work stoppage, computer failure or malfunction, or any and all - other commercial damages or losses), even if such Contributor - has been advised of the possibility of such damages. - - 9. Accepting Warranty or Additional Liability. While redistributing - the Work or Derivative Works thereof, You may choose to offer, - and charge a fee for, acceptance of support, warranty, indemnity, - or other liability obligations and/or rights consistent with this - License. However, in accepting such obligations, You may act only - on Your own behalf and on Your sole responsibility, not on behalf - of any other Contributor, and only if You agree to indemnify, - defend, and hold each Contributor harmless for any liability - incurred by, or claims asserted against, such Contributor by reason - of your accepting any such warranty or additional liability. - - END OF TERMS AND CONDITIONS - - APPENDIX: How to apply the Apache License to your work. - - To apply the Apache License to your work, attach the following - boilerplate notice, with the fields enclosed by brackets "[]" - replaced with your own identifying information. (Don't include - the brackets!) The text should be enclosed in the appropriate - comment syntax for the file format. We also recommend that a - file or class name and description of purpose be included on the - same "printed page" as the copyright notice for easier - identification within third-party archives. - - Copyright [yyyy] [name of copyright owner] - - Licensed under the Apache License, Version 2.0 (the "License"); - you may not use this file except in compliance with the License. - You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. diff --git a/third_party/rules_pycross/pycross/private/BUILD.bazel b/third_party/rules_pycross/pycross/private/BUILD.bazel deleted file mode 100644 index f59b087027..0000000000 --- a/third_party/rules_pycross/pycross/private/BUILD.bazel +++ /dev/null @@ -1,14 +0,0 @@ -# Copyright 2023 Jeremy Volkman. All rights reserved. -# Copyright 2023 The Bazel Authors. All rights reserved. -# -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. diff --git a/third_party/rules_pycross/pycross/private/providers.bzl b/third_party/rules_pycross/pycross/private/providers.bzl deleted file mode 100644 index 47fc9f7271..0000000000 --- a/third_party/rules_pycross/pycross/private/providers.bzl +++ /dev/null @@ -1,32 +0,0 @@ -# Copyright 2023 Jeremy Volkman. All rights reserved. -# Copyright 2023 The Bazel Authors. All rights reserved. -# -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -"""Python providers.""" - -PyWheelInfo = provider( - doc = "Information about a Python wheel.", - fields = { - "name_file": "File: A file containing the canonical name of the wheel.", - "wheel_file": "File: The wheel file itself.", - }, -) - -PyTargetEnvironmentInfo = provider( - doc = "A target environment description.", - fields = { - "file": "The JSON file containing target environment information.", - "python_compatible_with": "A list of constraints used to select this platform.", - }, -) diff --git a/third_party/rules_pycross/pycross/private/tools/BUILD.bazel b/third_party/rules_pycross/pycross/private/tools/BUILD.bazel deleted file mode 100644 index 41485c18a3..0000000000 --- a/third_party/rules_pycross/pycross/private/tools/BUILD.bazel +++ /dev/null @@ -1,26 +0,0 @@ -# Copyright 2023 Jeremy Volkman. All rights reserved. -# Copyright 2023 The Bazel Authors. All rights reserved. -# -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -load("//python:defs.bzl", "py_binary") - -py_binary( - name = "wheel_installer", - srcs = ["wheel_installer.py"], - visibility = ["//visibility:public"], - deps = [ - "//python/private/pypi/whl_installer:lib", - "@pypi__installer//:lib", - ], -) diff --git a/third_party/rules_pycross/pycross/private/tools/wheel_installer.py b/third_party/rules_pycross/pycross/private/tools/wheel_installer.py deleted file mode 100644 index a122e67733..0000000000 --- a/third_party/rules_pycross/pycross/private/tools/wheel_installer.py +++ /dev/null @@ -1,196 +0,0 @@ -# Copyright 2023 Jeremy Volkman. All rights reserved. -# Copyright 2023 The Bazel Authors. All rights reserved. -# -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -""" -A tool that invokes pypa/build to build the given sdist tarball. -""" - -import argparse -import os -import shutil -import subprocess -import sys -import tempfile -from pathlib import Path -from typing import Any - -from installer import install -from installer.destinations import SchemeDictionaryDestination -from installer.sources import WheelFile - -from python.private.pypi.whl_installer import namespace_pkgs - - -def setup_namespace_pkg_compatibility(wheel_dir: Path) -> None: - """Converts native namespace packages to pkgutil-style packages - - Namespace packages can be created in one of three ways. They are detailed here: - https://packaging.python.org/guides/packaging-namespace-packages/#creating-a-namespace-package - - 'pkgutil-style namespace packages' (2) and 'pkg_resources-style namespace packages' (3) works in Bazel, but - 'native namespace packages' (1) do not. - - We ensure compatibility with Bazel of method 1 by converting them into method 2. - - Args: - wheel_dir: the directory of the wheel to convert - """ - - namespace_pkg_dirs = namespace_pkgs.implicit_namespace_packages( - str(wheel_dir), - ignored_dirnames=["%s/bin" % wheel_dir], - ) - - for ns_pkg_dir in namespace_pkg_dirs: - namespace_pkgs.add_pkgutil_style_namespace_pkg_init(ns_pkg_dir) - - -def main(args: Any) -> None: - dest_dir = args.directory - lib_dir = dest_dir / "site-packages" - destination = SchemeDictionaryDestination( - scheme_dict={ - "platlib": str(lib_dir), - "purelib": str(lib_dir), - "headers": str(dest_dir / "include"), - "scripts": str(dest_dir / "bin"), - "data": str(dest_dir / "data"), - }, - interpreter="/usr/bin/env python3", # Generic; it's not feasible to run these scripts directly. - script_kind="posix", - bytecode_optimization_levels=[0, 1], - ) - - link_dir = Path(tempfile.mkdtemp()) - if args.wheel_name_file: - with open(args.wheel_name_file, "r") as f: - wheel_name = f.read().strip() - else: - wheel_name = os.path.basename(args.wheel) - - link_path = link_dir / wheel_name - os.symlink(os.path.join(os.getcwd(), args.wheel), link_path) - - try: - with WheelFile.open(link_path) as source: - install( - source=source, - destination=destination, - # Additional metadata that is generated by the installation tool. - additional_metadata={ - "INSTALLER": b"https://github.com/bazel-contrib/rules_python/tree/main/third_party/rules_pycross", - }, - ) - finally: - shutil.rmtree(link_dir, ignore_errors=True) - - setup_namespace_pkg_compatibility(lib_dir) - - if args.patch: - if not args.patch_tool and not args.patch_tool_target: - raise ValueError("Specify one of 'patch_tool' or 'patch_tool_target'.") - - patch_args = [ - args.patch_tool or Path.cwd() / args.patch_tool_target - ] + args.patch_arg - for patch in args.patch: - with patch.open("r") as stdin: - try: - subprocess.run( - patch_args, - stdin=stdin, - check=True, - stdout=subprocess.PIPE, - stderr=subprocess.STDOUT, - cwd=args.directory, - ) - except subprocess.CalledProcessError as error: - print(f"Patch {patch} failed to apply:") - print(error.stdout.decode("utf-8")) - raise - - -def parse_flags(argv) -> Any: - parser = argparse.ArgumentParser(description="Extract a Python wheel.") - - parser.add_argument( - "--wheel", - type=Path, - required=True, - help="The wheel file path.", - ) - - parser.add_argument( - "--wheel-name-file", - type=Path, - required=False, - help="A file containing the canonical name of the wheel.", - ) - - parser.add_argument( - "--enable-implicit-namespace-pkgs", - action="store_true", - help="If true, disables conversion of implicit namespace packages and will unzip as-is.", - ) - - parser.add_argument( - "--directory", - type=Path, - help="The output path.", - ) - - parser.add_argument( - "--patch", - type=Path, - default=[], - action="append", - help="A patch file to apply.", - ) - - parser.add_argument( - "--patch-arg", - type=str, - default=[], - action="append", - help="An argument for the patch tool when applying the patches.", - ) - - parser.add_argument( - "--patch-tool", - type=str, - help=( - "The tool from PATH to invoke when applying patches. " - "If set, --patch-tool-target is ignored." - ), - ) - - parser.add_argument( - "--patch-tool-target", - type=Path, - help=( - "The path to the tool to invoke when applying patches. " - "Ignored when --patch-tool is set." - ), - ) - - return parser.parse_args(argv[1:]) - - -if __name__ == "__main__": - # When under `bazel run`, change to the actual working dir. - if "BUILD_WORKING_DIRECTORY" in os.environ: - os.chdir(os.environ["BUILD_WORKING_DIRECTORY"]) - - main(parse_flags(sys.argv)) diff --git a/third_party/rules_pycross/pycross/private/wheel_library.bzl b/third_party/rules_pycross/pycross/private/wheel_library.bzl deleted file mode 100644 index 00d85f71b1..0000000000 --- a/third_party/rules_pycross/pycross/private/wheel_library.bzl +++ /dev/null @@ -1,174 +0,0 @@ -# Copyright 2023 Jeremy Volkman. All rights reserved. -# Copyright 2023 The Bazel Authors. All rights reserved. -# -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -"""Implementation of the py_wheel_library rule.""" - -load("@bazel_skylib//lib:paths.bzl", "paths") -load("//python:py_info.bzl", "PyInfo") -load(":providers.bzl", "PyWheelInfo") - -def _py_wheel_library_impl(ctx): - out = ctx.actions.declare_directory(ctx.attr.name) - - wheel_target = ctx.attr.wheel - if PyWheelInfo in wheel_target: - wheel_file = wheel_target[PyWheelInfo].wheel_file - name_file = wheel_target[PyWheelInfo].name_file - else: - wheel_file = ctx.file.wheel - name_file = None - - args = ctx.actions.args().use_param_file("--flagfile=%s") - args.add("--wheel", wheel_file) - args.add("--directory", out.path) - args.add_all(ctx.files.patches, format_each = "--patch=%s") - args.add_all(ctx.attr.patch_args, format_each = "--patch-arg=%s") - args.add("--patch-tool", ctx.attr.patch_tool) - - tools = [] - inputs = [wheel_file] + ctx.files.patches - if name_file: - inputs.append(name_file) - args.add("--wheel-name-file", name_file) - - if ctx.attr.patch_tool_target: - args.add("--patch-tool-target", ctx.attr.patch_tool_target.files_to_run.executable) - tools.append(ctx.executable.patch_tool_target) - - if ctx.attr.enable_implicit_namespace_pkgs: - args.add("--enable-implicit-namespace-pkgs") - - # We apply patches in the same action as the extraction to minimize the - # number of times we cache the wheel contents. If we were to split this - # into 2 actions, then the wheel contents would be cached twice. - ctx.actions.run( - inputs = inputs, - outputs = [out], - executable = ctx.executable._tool, - tools = tools, - arguments = [args], - # Set environment variables to make generated .pyc files reproducible. - env = { - "PYTHONHASHSEED": "0", - "SOURCE_DATE_EPOCH": "315532800", - }, - mnemonic = "WheelInstall", - progress_message = "Installing %s" % ctx.file.wheel.basename, - ) - - has_py2_only_sources = ctx.attr.python_version == "PY2" - has_py3_only_sources = ctx.attr.python_version == "PY3" - if not has_py2_only_sources: - for d in ctx.attr.deps: - if d[PyInfo].has_py2_only_sources: - has_py2_only_sources = True - break - if not has_py3_only_sources: - for d in ctx.attr.deps: - if d[PyInfo].has_py3_only_sources: - has_py3_only_sources = True - break - - # TODO: Is there a more correct way to get this runfiles-relative import path? - imp = paths.join( - ctx.label.repo_name or ctx.workspace_name, # Default to the local workspace. - ctx.label.package, - ctx.label.name, - "site-packages", # we put lib files in this subdirectory. - ) - - imports = depset( - direct = [imp], - transitive = [d[PyInfo].imports for d in ctx.attr.deps], - ) - transitive_sources = depset( - direct = [out], - transitive = [dep[PyInfo].transitive_sources for dep in ctx.attr.deps if PyInfo in dep], - ) - runfiles = ctx.runfiles(files = [out]) - for d in ctx.attr.deps: - runfiles = runfiles.merge(d[DefaultInfo].default_runfiles) - - return [ - DefaultInfo( - files = depset(direct = [out]), - runfiles = runfiles, - ), - PyInfo( - has_py2_only_sources = has_py2_only_sources, - has_py3_only_sources = has_py3_only_sources, - imports = imports, - transitive_sources = transitive_sources, - uses_shared_libraries = True, # Docs say this is unused - ), - ] - -py_wheel_library = rule( - implementation = _py_wheel_library_impl, - attrs = { - "deps": attr.label_list( - doc = "A list of this wheel's Python library dependencies.", - providers = [DefaultInfo, PyInfo], - ), - "enable_implicit_namespace_pkgs": attr.bool( - default = True, - doc = """ -If true, disables conversion of native namespace packages into pkg-util style namespace packages. When set all py_binary -and py_test targets must specify either `legacy_create_init=False` or the global Bazel option -`--incompatible_default_to_explicit_init_py` to prevent `__init__.py` being automatically generated in every directory. -This option is required to support some packages which cannot handle the conversion to pkg-util style. - """, - ), - "patch_args": attr.string_list( - default = ["-p0"], - doc = - "The arguments given to the patch tool. Defaults to -p0, " + - "however -p1 will usually be needed for patches generated by " + - "git. If multiple -p arguments are specified, the last one will take effect.", - ), - "patch_tool": attr.string( - doc = "The patch(1) utility from the host to use. " + - "If set, overrides `patch_tool_target`. Please note that setting " + - "this means that builds are not completely hermetic.", - ), - "patch_tool_target": attr.label( - executable = True, - cfg = "exec", - doc = "The label of the patch(1) utility to use. " + - "Only used if `patch_tool` is not set.", - ), - "patches": attr.label_list( - allow_files = True, - default = [], - doc = - "A list of files that are to be applied as patches after " + - "extracting the archive. This will use the patch command line tool.", - ), - "python_version": attr.string( - doc = "The python version required for this wheel ('PY2' or 'PY3')", - values = ["PY2", "PY3", ""], - ), - "wheel": attr.label( - doc = "The wheel file.", - allow_single_file = [".whl"], - mandatory = True, - ), - "_tool": attr.label( - default = Label("//third_party/rules_pycross/pycross/private/tools:wheel_installer"), - cfg = "exec", - executable = True, - ), - }, -) From 369ca91fe346a7dac760a883d36352510eac8f1d Mon Sep 17 00:00:00 2001 From: Ignas Anikevicius <240938+aignas@users.noreply.github.com> Date: Wed, 28 May 2025 09:50:03 +0900 Subject: [PATCH 259/922] refactor(pypi): return a list from parse_requirements (#2931) The modeling of the data structures returned by the `parse_requirements` function was not optimal and this was because historically there was more logic in the `extension.bzl` and more things were decided there. With the recent refactors it is possible to have a harder to misuse data structure from the `parse_requirements`. For each `package` we will return a struct which will have a `srcs` field that will contain easy to consume values. With this in place we can do the fix that is outlined in the referenced issue. Work towards #2648 --- python/private/pypi/extension.bzl | 172 +++---- python/private/pypi/parse_requirements.bzl | 92 +++- python/private/pypi/pip_repository.bzl | 6 +- .../parse_requirements_tests.bzl | 485 ++++++++++-------- 4 files changed, 424 insertions(+), 331 deletions(-) diff --git a/python/private/pypi/extension.bzl b/python/private/pypi/extension.bzl index d3a15dfc44..b79be6e038 100644 --- a/python/private/pypi/extension.bzl +++ b/python/private/pypi/extension.bzl @@ -202,8 +202,12 @@ def _create_whl_repos( logger = logger, ) - for whl_name, requirements in requirements_by_platform.items(): - group_name = whl_group_mapping.get(whl_name) + exposed_packages = {} + for whl in requirements_by_platform: + if whl.is_exposed: + exposed_packages[whl.name] = None + + group_name = whl_group_mapping.get(whl.name) group_deps = requirement_cycles.get(group_name, []) # Construct args separately so that the lock file can be smaller and does not include unused @@ -214,7 +218,7 @@ def _create_whl_repos( maybe_args = dict( # The following values are safe to omit if they have false like values add_libdir_to_library_search_path = pip_attr.add_libdir_to_library_search_path, - annotation = whl_modifications.get(whl_name), + annotation = whl_modifications.get(whl.name), download_only = pip_attr.download_only, enable_implicit_namespace_pkgs = pip_attr.enable_implicit_namespace_pkgs, environment = pip_attr.environment, @@ -226,7 +230,7 @@ def _create_whl_repos( python_interpreter_target = python_interpreter_target, whl_patches = { p: json.encode(args) - for p, args in whl_overrides.get(whl_name, {}).items() + for p, args in whl_overrides.get(whl.name, {}).items() }, ) if not enable_pipstar: @@ -245,119 +249,99 @@ def _create_whl_repos( if v != default }) - for requirement in requirements: - for repo_name, (args, config_setting) in _whl_repos( - requirement = requirement, + for src in whl.srcs: + repo = _whl_repo( + src = src, whl_library_args = whl_library_args, download_only = pip_attr.download_only, netrc = pip_attr.netrc, auth_patterns = pip_attr.auth_patterns, python_version = major_minor, - multiple_requirements_for_whl = len(requirements) > 1., + is_multiple_versions = whl.is_multiple_versions, enable_pipstar = enable_pipstar, - ).items(): - repo_name = "{}_{}".format(pip_name, repo_name) - if repo_name in whl_libraries: - fail("Attempting to creating a duplicate library {} for {}".format( - repo_name, - whl_name, - )) + ) - whl_libraries[repo_name] = args - whl_map.setdefault(whl_name, {})[config_setting] = repo_name + repo_name = "{}_{}".format(pip_name, repo.repo_name) + if repo_name in whl_libraries: + fail("Attempting to creating a duplicate library {} for {}".format( + repo_name, + whl.name, + )) + + whl_libraries[repo_name] = repo.args + whl_map.setdefault(whl.name, {})[repo.config_setting] = repo_name return struct( whl_map = whl_map, - exposed_packages = { - whl_name: None - for whl_name, requirements in requirements_by_platform.items() - if len([r for r in requirements if r.is_exposed]) > 0 - }, + exposed_packages = exposed_packages, extra_aliases = extra_aliases, whl_libraries = whl_libraries, ) -def _whl_repos(*, requirement, whl_library_args, download_only, netrc, auth_patterns, multiple_requirements_for_whl = False, python_version, enable_pipstar = False): - ret = {} - - dists = requirement.whls - if not download_only and requirement.sdist: - dists = dists + [requirement.sdist] - - for distribution in dists: - args = dict(whl_library_args) - if netrc: - args["netrc"] = netrc - if auth_patterns: - args["auth_patterns"] = auth_patterns - - if not distribution.filename.endswith(".whl"): - # pip is not used to download wheels and the python - # `whl_library` helpers are only extracting things, however - # for sdists, they will be built by `pip`, so we still - # need to pass the extra args there. - args["extra_pip_args"] = requirement.extra_pip_args - - # This is no-op because pip is not used to download the wheel. - args.pop("download_only", None) - - args["requirement"] = requirement.line - args["urls"] = [distribution.url] - args["sha256"] = distribution.sha256 - args["filename"] = distribution.filename - if not enable_pipstar: - args["experimental_target_platforms"] = [ - # Get rid of the version fot the target platforms because we are - # passing the interpreter any way. Ideally we should search of ways - # how to pass the target platforms through the hub repo. - p.partition("_")[2] - for p in requirement.target_platforms - ] - - # Pure python wheels or sdists may need to have a platform here - target_platforms = None - if distribution.filename.endswith(".whl") and not distribution.filename.endswith("-any.whl"): - pass - elif multiple_requirements_for_whl: - target_platforms = requirement.target_platforms - - repo_name = whl_repo_name( - distribution.filename, - distribution.sha256, - ) - ret[repo_name] = ( - args, - whl_config_setting( +def _whl_repo(*, src, whl_library_args, is_multiple_versions, download_only, netrc, auth_patterns, python_version, enable_pipstar = False): + args = dict(whl_library_args) + args["requirement"] = src.requirement_line + is_whl = src.filename.endswith(".whl") + + if src.extra_pip_args and not is_whl: + # pip is not used to download wheels and the python + # `whl_library` helpers are only extracting things, however + # for sdists, they will be built by `pip`, so we still + # need to pass the extra args there, so only pop this for whls + args["extra_pip_args"] = src.extra_pip_args + + if not src.url or (not is_whl and download_only): + # Fallback to a pip-installed wheel + target_platforms = src.target_platforms if is_multiple_versions else [] + return struct( + repo_name = pypi_repo_name( + normalize_name(src.distribution), + *target_platforms + ), + args = args, + config_setting = whl_config_setting( version = python_version, - filename = distribution.filename, - target_platforms = target_platforms, + target_platforms = target_platforms or None, ), ) - if ret: - return ret - - # Fallback to a pip-installed wheel - args = dict(whl_library_args) # make a copy - args["requirement"] = requirement.line - if requirement.extra_pip_args: - args["extra_pip_args"] = requirement.extra_pip_args + # This is no-op because pip is not used to download the wheel. + args.pop("download_only", None) + + if netrc: + args["netrc"] = netrc + if auth_patterns: + args["auth_patterns"] = auth_patterns + + args["urls"] = [src.url] + args["sha256"] = src.sha256 + args["filename"] = src.filename + if not enable_pipstar: + args["experimental_target_platforms"] = [ + # Get rid of the version fot the target platforms because we are + # passing the interpreter any way. Ideally we should search of ways + # how to pass the target platforms through the hub repo. + p.partition("_")[2] + for p in src.target_platforms + ] + + # Pure python wheels or sdists may need to have a platform here + target_platforms = None + if is_whl and not src.filename.endswith("-any.whl"): + pass + elif is_multiple_versions: + target_platforms = src.target_platforms - target_platforms = requirement.target_platforms if multiple_requirements_for_whl else [] - repo_name = pypi_repo_name( - normalize_name(requirement.distribution), - *target_platforms - ) - ret[repo_name] = ( - args, - whl_config_setting( + return struct( + repo_name = whl_repo_name(src.filename, src.sha256), + args = args, + config_setting = whl_config_setting( version = python_version, - target_platforms = target_platforms or None, + filename = src.filename, + target_platforms = target_platforms, ), ) - return ret - def parse_modules( module_ctx, _fail = fail, diff --git a/python/private/pypi/parse_requirements.bzl b/python/private/pypi/parse_requirements.bzl index bdfac46ed6..bd2981efc0 100644 --- a/python/private/pypi/parse_requirements.bzl +++ b/python/private/pypi/parse_requirements.bzl @@ -179,49 +179,91 @@ def parse_requirements( }), ) - ret = {} - for whl_name, reqs in sorted(requirements_by_platform.items()): + ret = [] + for name, reqs in sorted(requirements_by_platform.items()): requirement_target_platforms = {} for r in reqs.values(): target_platforms = env_marker_target_platforms.get(r.requirement_line, r.target_platforms) for p in target_platforms: requirement_target_platforms[p] = None - is_exposed = len(requirement_target_platforms) == len(requirements) - if not is_exposed and logger: + item = struct( + # Return normalized names + name = normalize_name(name), + is_exposed = len(requirement_target_platforms) == len(requirements), + is_multiple_versions = len(reqs.values()) > 1, + srcs = _package_srcs( + name = name, + reqs = reqs, + index_urls = index_urls, + env_marker_target_platforms = env_marker_target_platforms, + extract_url_srcs = extract_url_srcs, + logger = logger, + ), + ) + ret.append(item) + if not item.is_exposed and logger: logger.debug(lambda: "Package '{}' will not be exposed because it is only present on a subset of platforms: {} out of {}".format( - whl_name, + name, sorted(requirement_target_platforms), sorted(requirements), )) - # Return normalized names - ret_requirements = ret.setdefault(normalize_name(whl_name), []) + if logger: + logger.debug(lambda: "Will configure whl repos: {}".format([w.name for w in ret])) - for r in sorted(reqs.values(), key = lambda r: r.requirement_line): - whls, sdist = _add_dists( - requirement = r, - index_urls = index_urls.get(whl_name), - logger = logger, - ) + return ret - target_platforms = env_marker_target_platforms.get(r.requirement_line, r.target_platforms) - ret_requirements.append( +def _package_srcs( + *, + name, + reqs, + index_urls, + logger, + env_marker_target_platforms, + extract_url_srcs): + """A function to return sources for a particular package.""" + srcs = [] + for r in sorted(reqs.values(), key = lambda r: r.requirement_line): + whls, sdist = _add_dists( + requirement = r, + index_urls = index_urls.get(name), + logger = logger, + ) + + target_platforms = env_marker_target_platforms.get(r.requirement_line, r.target_platforms) + target_platforms = sorted(target_platforms) + + all_dists = [] + whls + if sdist: + all_dists.append(sdist) + + if extract_url_srcs and all_dists: + req_line = r.srcs.requirement + else: + all_dists = [struct( + url = "", + filename = "", + sha256 = "", + yanked = False, + )] + req_line = r.srcs.requirement_line + + for dist in all_dists: + srcs.append( struct( - distribution = r.distribution, - line = r.srcs.requirement if extract_url_srcs and (whls or sdist) else r.srcs.requirement_line, - target_platforms = sorted(target_platforms), + distribution = name, extra_pip_args = r.extra_pip_args, - whls = whls, - sdist = sdist, - is_exposed = is_exposed, + requirement_line = req_line, + target_platforms = target_platforms, + filename = dist.filename, + sha256 = dist.sha256, + url = dist.url, + yanked = dist.yanked, ), ) - if logger: - logger.debug(lambda: "Will configure whl repos: {}".format(ret.keys())) - - return ret + return srcs def select_requirement(requirements, *, platform): """A simple function to get a requirement for a particular platform. diff --git a/python/private/pypi/pip_repository.bzl b/python/private/pypi/pip_repository.bzl index c8d23f471f..724fb6ddba 100644 --- a/python/private/pypi/pip_repository.bzl +++ b/python/private/pypi/pip_repository.bzl @@ -94,15 +94,15 @@ def _pip_repository_impl(rctx): selected_requirements = {} options = None repository_platform = host_platform(rctx) - for name, requirements in requirements_by_platform.items(): + for whl in requirements_by_platform: requirement = select_requirement( - requirements, + whl.srcs, platform = None if rctx.attr.download_only else repository_platform, ) if not requirement: continue options = options or requirement.extra_pip_args - selected_requirements[name] = requirement.line + selected_requirements[whl.name] = requirement.requirement_line bzl_packages = sorted(selected_requirements.keys()) diff --git a/tests/pypi/parse_requirements/parse_requirements_tests.bzl b/tests/pypi/parse_requirements/parse_requirements_tests.bzl index 497e08361f..926a7e0c50 100644 --- a/tests/pypi/parse_requirements/parse_requirements_tests.bzl +++ b/tests/pypi/parse_requirements/parse_requirements_tests.bzl @@ -100,22 +100,28 @@ def _test_simple(env): "requirements_lock": ["linux_x86_64", "windows_x86_64"], }, ) - env.expect.that_dict(got).contains_exactly({ - "foo": [ - struct( - distribution = "foo", - extra_pip_args = [], - sdist = None, - is_exposed = True, - line = "foo[extra]==0.0.1 --hash=sha256:deadbeef", - target_platforms = [ - "linux_x86_64", - "windows_x86_64", - ], - whls = [], - ), - ], - }) + env.expect.that_collection(got).contains_exactly([ + struct( + name = "foo", + is_exposed = True, + is_multiple_versions = False, + srcs = [ + struct( + distribution = "foo", + extra_pip_args = [], + requirement_line = "foo[extra]==0.0.1 --hash=sha256:deadbeef", + target_platforms = [ + "linux_x86_64", + "windows_x86_64", + ], + url = "", + filename = "", + sha256 = "", + yanked = False, + ), + ], + ), + ]) _tests.append(_test_simple) @@ -127,24 +133,25 @@ def _test_direct_urls_integration(env): "requirements_direct": ["linux_x86_64"], }, ) - env.expect.that_dict(got).contains_exactly({ - "foo": [ - struct( - distribution = "foo", - extra_pip_args = [], - sdist = None, - is_exposed = True, - line = "foo[extra]", - target_platforms = ["linux_x86_64"], - whls = [struct( + env.expect.that_collection(got).contains_exactly([ + struct( + name = "foo", + is_exposed = True, + is_multiple_versions = False, + srcs = [ + struct( + distribution = "foo", + extra_pip_args = [], + requirement_line = "foo[extra]", + target_platforms = ["linux_x86_64"], url = "https://some-url/package.whl", filename = "package.whl", sha256 = "", yanked = False, - )], - ), - ], - }) + ), + ], + ), + ]) _tests.append(_test_direct_urls_integration) @@ -156,21 +163,27 @@ def _test_extra_pip_args(env): }, extra_pip_args = ["--trusted-host=example.org"], ) - env.expect.that_dict(got).contains_exactly({ - "foo": [ - struct( - distribution = "foo", - extra_pip_args = ["--index-url=example.org", "--trusted-host=example.org"], - sdist = None, - is_exposed = True, - line = "foo[extra]==0.0.1 --hash=sha256:deadbeef", - target_platforms = [ - "linux_x86_64", - ], - whls = [], - ), - ], - }) + env.expect.that_collection(got).contains_exactly([ + struct( + name = "foo", + is_exposed = True, + is_multiple_versions = False, + srcs = [ + struct( + distribution = "foo", + extra_pip_args = ["--index-url=example.org", "--trusted-host=example.org"], + requirement_line = "foo[extra]==0.0.1 --hash=sha256:deadbeef", + target_platforms = [ + "linux_x86_64", + ], + url = "", + filename = "", + sha256 = "", + yanked = False, + ), + ], + ), + ]) _tests.append(_test_extra_pip_args) @@ -181,19 +194,25 @@ def _test_dupe_requirements(env): "requirements_lock_dupe": ["linux_x86_64"], }, ) - env.expect.that_dict(got).contains_exactly({ - "foo": [ - struct( - distribution = "foo", - extra_pip_args = [], - sdist = None, - is_exposed = True, - line = "foo[extra,extra_2]==0.0.1 --hash=sha256:deadbeef", - target_platforms = ["linux_x86_64"], - whls = [], - ), - ], - }) + env.expect.that_collection(got).contains_exactly([ + struct( + name = "foo", + is_exposed = True, + is_multiple_versions = False, + srcs = [ + struct( + distribution = "foo", + extra_pip_args = [], + requirement_line = "foo[extra,extra_2]==0.0.1 --hash=sha256:deadbeef", + target_platforms = ["linux_x86_64"], + url = "", + filename = "", + sha256 = "", + yanked = False, + ), + ], + ), + ]) _tests.append(_test_dupe_requirements) @@ -206,44 +225,57 @@ def _test_multi_os(env): }, ) - env.expect.that_dict(got).contains_exactly({ - "bar": [ - struct( - distribution = "bar", - extra_pip_args = [], - line = "bar==0.0.1 --hash=sha256:deadb00f", - target_platforms = ["windows_x86_64"], - whls = [], - sdist = None, - is_exposed = False, - ), - ], - "foo": [ - struct( - distribution = "foo", - extra_pip_args = [], - line = "foo==0.0.3 --hash=sha256:deadbaaf", - target_platforms = ["linux_x86_64"], - whls = [], - sdist = None, - is_exposed = True, - ), - struct( - distribution = "foo", - extra_pip_args = [], - line = "foo[extra]==0.0.2 --hash=sha256:deadbeef", - target_platforms = ["windows_x86_64"], - whls = [], - sdist = None, - is_exposed = True, - ), - ], - }) + env.expect.that_collection(got).contains_exactly([ + struct( + name = "bar", + is_exposed = False, + is_multiple_versions = False, + srcs = [ + struct( + distribution = "bar", + extra_pip_args = [], + requirement_line = "bar==0.0.1 --hash=sha256:deadb00f", + target_platforms = ["windows_x86_64"], + url = "", + filename = "", + sha256 = "", + yanked = False, + ), + ], + ), + struct( + name = "foo", + is_exposed = True, + is_multiple_versions = True, + srcs = [ + struct( + distribution = "foo", + extra_pip_args = [], + requirement_line = "foo==0.0.3 --hash=sha256:deadbaaf", + target_platforms = ["linux_x86_64"], + url = "", + filename = "", + sha256 = "", + yanked = False, + ), + struct( + distribution = "foo", + extra_pip_args = [], + requirement_line = "foo[extra]==0.0.2 --hash=sha256:deadbeef", + target_platforms = ["windows_x86_64"], + url = "", + filename = "", + sha256 = "", + yanked = False, + ), + ], + ), + ]) env.expect.that_str( select_requirement( - got["foo"], + got[1].srcs, platform = "windows_x86_64", - ).line, + ).requirement_line, ).equals("foo[extra]==0.0.2 --hash=sha256:deadbeef") _tests.append(_test_multi_os) @@ -257,39 +289,52 @@ def _test_multi_os_legacy(env): }, ) - env.expect.that_dict(got).contains_exactly({ - "bar": [ - struct( - distribution = "bar", - extra_pip_args = ["--platform=manylinux_2_17_x86_64", "--python-version=39", "--implementation=cp", "--abi=cp39"], - is_exposed = False, - sdist = None, - line = "bar==0.0.1 --hash=sha256:deadb00f", - target_platforms = ["cp39_linux_x86_64"], - whls = [], - ), - ], - "foo": [ - struct( - distribution = "foo", - extra_pip_args = ["--platform=manylinux_2_17_x86_64", "--python-version=39", "--implementation=cp", "--abi=cp39"], - is_exposed = True, - sdist = None, - line = "foo==0.0.1 --hash=sha256:deadbeef", - target_platforms = ["cp39_linux_x86_64"], - whls = [], - ), - struct( - distribution = "foo", - extra_pip_args = ["--platform=macosx_10_9_arm64", "--python-version=39", "--implementation=cp", "--abi=cp39"], - is_exposed = True, - sdist = None, - line = "foo==0.0.3 --hash=sha256:deadbaaf", - target_platforms = ["cp39_osx_aarch64"], - whls = [], - ), - ], - }) + env.expect.that_collection(got).contains_exactly([ + struct( + name = "bar", + is_exposed = False, + is_multiple_versions = False, + srcs = [ + struct( + distribution = "bar", + extra_pip_args = ["--platform=manylinux_2_17_x86_64", "--python-version=39", "--implementation=cp", "--abi=cp39"], + requirement_line = "bar==0.0.1 --hash=sha256:deadb00f", + target_platforms = ["cp39_linux_x86_64"], + url = "", + filename = "", + sha256 = "", + yanked = False, + ), + ], + ), + struct( + name = "foo", + is_exposed = True, + is_multiple_versions = True, + srcs = [ + struct( + distribution = "foo", + extra_pip_args = ["--platform=manylinux_2_17_x86_64", "--python-version=39", "--implementation=cp", "--abi=cp39"], + requirement_line = "foo==0.0.1 --hash=sha256:deadbeef", + target_platforms = ["cp39_linux_x86_64"], + url = "", + filename = "", + sha256 = "", + yanked = False, + ), + struct( + distribution = "foo", + extra_pip_args = ["--platform=macosx_10_9_arm64", "--python-version=39", "--implementation=cp", "--abi=cp39"], + requirement_line = "foo==0.0.3 --hash=sha256:deadbaaf", + target_platforms = ["cp39_osx_aarch64"], + url = "", + filename = "", + sha256 = "", + yanked = False, + ), + ], + ), + ]) _tests.append(_test_multi_os_legacy) @@ -324,30 +369,42 @@ def _test_env_marker_resolution(env): }, evaluate_markers = _mock_eval_markers, ) - env.expect.that_dict(got).contains_exactly({ - "bar": [ - struct( - distribution = "bar", - extra_pip_args = [], - is_exposed = True, - sdist = None, - line = "bar==0.0.1 --hash=sha256:deadbeef", - target_platforms = ["cp311_linux_super_exotic", "cp311_windows_x86_64"], - whls = [], - ), - ], - "foo": [ - struct( - distribution = "foo", - extra_pip_args = [], - is_exposed = False, - sdist = None, - line = "foo[extra]==0.0.1 --hash=sha256:deadbeef", - target_platforms = ["cp311_windows_x86_64"], - whls = [], - ), - ], - }) + env.expect.that_collection(got).contains_exactly([ + struct( + name = "bar", + is_exposed = True, + is_multiple_versions = False, + srcs = [ + struct( + distribution = "bar", + extra_pip_args = [], + requirement_line = "bar==0.0.1 --hash=sha256:deadbeef", + target_platforms = ["cp311_linux_super_exotic", "cp311_windows_x86_64"], + url = "", + filename = "", + sha256 = "", + yanked = False, + ), + ], + ), + struct( + name = "foo", + is_exposed = False, + is_multiple_versions = False, + srcs = [ + struct( + distribution = "foo", + extra_pip_args = [], + requirement_line = "foo[extra]==0.0.1 --hash=sha256:deadbeef", + target_platforms = ["cp311_windows_x86_64"], + url = "", + filename = "", + sha256 = "", + yanked = False, + ), + ], + ), + ]) _tests.append(_test_env_marker_resolution) @@ -358,28 +415,35 @@ def _test_different_package_version(env): "requirements_different_package_version": ["linux_x86_64"], }, ) - env.expect.that_dict(got).contains_exactly({ - "foo": [ - struct( - distribution = "foo", - extra_pip_args = [], - is_exposed = True, - sdist = None, - line = "foo==0.0.1 --hash=sha256:deadb00f", - target_platforms = ["linux_x86_64"], - whls = [], - ), - struct( - distribution = "foo", - extra_pip_args = [], - is_exposed = True, - sdist = None, - line = "foo==0.0.1+local --hash=sha256:deadbeef", - target_platforms = ["linux_x86_64"], - whls = [], - ), - ], - }) + env.expect.that_collection(got).contains_exactly([ + struct( + name = "foo", + is_exposed = True, + is_multiple_versions = True, + srcs = [ + struct( + distribution = "foo", + extra_pip_args = [], + requirement_line = "foo==0.0.1 --hash=sha256:deadb00f", + target_platforms = ["linux_x86_64"], + url = "", + filename = "", + sha256 = "", + yanked = False, + ), + struct( + distribution = "foo", + extra_pip_args = [], + requirement_line = "foo==0.0.1+local --hash=sha256:deadbeef", + target_platforms = ["linux_x86_64"], + url = "", + filename = "", + sha256 = "", + yanked = False, + ), + ], + ), + ]) _tests.append(_test_different_package_version) @@ -390,38 +454,35 @@ def _test_optional_hash(env): "requirements_optional_hash": ["linux_x86_64"], }, ) - env.expect.that_dict(got).contains_exactly({ - "foo": [ - struct( - distribution = "foo", - extra_pip_args = [], - sdist = None, - is_exposed = True, - line = "foo==0.0.4", - target_platforms = ["linux_x86_64"], - whls = [struct( + env.expect.that_collection(got).contains_exactly([ + struct( + name = "foo", + is_exposed = True, + is_multiple_versions = True, + srcs = [ + struct( + distribution = "foo", + extra_pip_args = [], + requirement_line = "foo==0.0.4", + target_platforms = ["linux_x86_64"], url = "https://example.org/foo-0.0.4.whl", filename = "foo-0.0.4.whl", sha256 = "", yanked = False, - )], - ), - struct( - distribution = "foo", - extra_pip_args = [], - sdist = None, - is_exposed = True, - line = "foo==0.0.5", - target_platforms = ["linux_x86_64"], - whls = [struct( + ), + struct( + distribution = "foo", + extra_pip_args = [], + requirement_line = "foo==0.0.5", + target_platforms = ["linux_x86_64"], url = "https://example.org/foo-0.0.5.whl", filename = "foo-0.0.5.whl", sha256 = "deadbeef", yanked = False, - )], - ), - ], - }) + ), + ], + ), + ]) _tests.append(_test_optional_hash) @@ -432,19 +493,25 @@ def _test_git_sources(env): "requirements_git": ["linux_x86_64"], }, ) - env.expect.that_dict(got).contains_exactly({ - "foo": [ - struct( - distribution = "foo", - extra_pip_args = [], - is_exposed = True, - sdist = None, - line = "foo @ git+https://github.com/org/foo.git@deadbeef", - target_platforms = ["linux_x86_64"], - whls = [], - ), - ], - }) + env.expect.that_collection(got).contains_exactly([ + struct( + name = "foo", + is_exposed = True, + is_multiple_versions = False, + srcs = [ + struct( + distribution = "foo", + extra_pip_args = [], + requirement_line = "foo @ git+https://github.com/org/foo.git@deadbeef", + target_platforms = ["linux_x86_64"], + url = "", + filename = "", + sha256 = "", + yanked = False, + ), + ], + ), + ]) _tests.append(_test_git_sources) From 3464c14c36e5d20a56e61952c5e06ef608aa0ed9 Mon Sep 17 00:00:00 2001 From: Ignas Anikevicius <240938+aignas@users.noreply.github.com> Date: Wed, 28 May 2025 22:53:50 +0900 Subject: [PATCH 260/922] fix: symlink root-level python files to the venv (#2908) As found in #2882 testing, packages like `typing-extensions` which have `.py` files at the root of the `site-packages` folder don't work and it seems that the comment about `rules_python` being too eager is only half-correct. Since `namespace_pkgs` are no longer there, we can just include all of the files and if there are collisions, they will be highlighted as build errors. Now the following works: ``` bazel build //docs --@rules_python//python/config_settings:venvs_site_packages=yes ``` Work towards #2156 --- .bazelrc | 4 +-- python/private/py_library.bzl | 19 +++++----- tests/modules/other/nspkg_single/BUILD.bazel | 10 ++++++ .../nspkg_single/site-packages/__init__.py | 1 + .../nspkg_single/site-packages/single_file.py | 5 +++ tests/venv_site_packages_libs/BUILD.bazel | 1 + tests/venv_site_packages_libs/bin.py | 1 + .../nspkg_alpha/BUILD.bazel | 2 +- .../venv_site_packages_pypi_test.py | 36 ------------------- 9 files changed, 32 insertions(+), 47 deletions(-) create mode 100644 tests/modules/other/nspkg_single/BUILD.bazel create mode 100644 tests/modules/other/nspkg_single/site-packages/__init__.py create mode 100644 tests/modules/other/nspkg_single/site-packages/single_file.py delete mode 100644 tests/venv_site_packages_libs/venv_site_packages_pypi_test.py diff --git a/.bazelrc b/.bazelrc index 4e6f2fa187..7e744fb67a 100644 --- a/.bazelrc +++ b/.bazelrc @@ -4,8 +4,8 @@ # (Note, we cannot use `common --deleted_packages` because the bazel version command doesn't support it) # To update these lines, execute # `bazel run @rules_bazel_integration_test//tools:update_deleted_packages` -build --deleted_packages=examples/build_file_generation,examples/build_file_generation/random_number_generator,examples/bzlmod,examples/bzlmod_build_file_generation,examples/bzlmod_build_file_generation/other_module/other_module/pkg,examples/bzlmod_build_file_generation/runfiles,examples/bzlmod/entry_points,examples/bzlmod/entry_points/tests,examples/bzlmod/libs/my_lib,examples/bzlmod/other_module,examples/bzlmod/other_module/other_module/pkg,examples/bzlmod/patches,examples/bzlmod/py_proto_library,examples/bzlmod/py_proto_library/example.com/another_proto,examples/bzlmod/py_proto_library/example.com/proto,examples/bzlmod/runfiles,examples/bzlmod/tests,examples/bzlmod/tests/other_module,examples/bzlmod/whl_mods,examples/multi_python_versions/libs/my_lib,examples/multi_python_versions/requirements,examples/multi_python_versions/tests,examples/pip_parse,examples/pip_parse_vendored,examples/pip_repository_annotations,examples/py_proto_library,examples/py_proto_library/example.com/another_proto,examples/py_proto_library/example.com/proto,gazelle,gazelle/manifest,gazelle/manifest/generate,gazelle/manifest/hasher,gazelle/manifest/test,gazelle/modules_mapping,gazelle/python,gazelle/pythonconfig,gazelle/python/private,tests/integration/compile_pip_requirements,tests/integration/compile_pip_requirements_test_from_external_repo,tests/integration/custom_commands,tests/integration/ignore_root_user_error,tests/integration/ignore_root_user_error/submodule,tests/integration/local_toolchains,tests/integration/pip_parse,tests/integration/pip_parse/empty,tests/integration/py_cc_toolchain_registered,tests/modules/other,tests/modules/other/nspkg_delta,tests/modules/other/nspkg_gamma -query --deleted_packages=examples/build_file_generation,examples/build_file_generation/random_number_generator,examples/bzlmod,examples/bzlmod_build_file_generation,examples/bzlmod_build_file_generation/other_module/other_module/pkg,examples/bzlmod_build_file_generation/runfiles,examples/bzlmod/entry_points,examples/bzlmod/entry_points/tests,examples/bzlmod/libs/my_lib,examples/bzlmod/other_module,examples/bzlmod/other_module/other_module/pkg,examples/bzlmod/patches,examples/bzlmod/py_proto_library,examples/bzlmod/py_proto_library/example.com/another_proto,examples/bzlmod/py_proto_library/example.com/proto,examples/bzlmod/runfiles,examples/bzlmod/tests,examples/bzlmod/tests/other_module,examples/bzlmod/whl_mods,examples/multi_python_versions/libs/my_lib,examples/multi_python_versions/requirements,examples/multi_python_versions/tests,examples/pip_parse,examples/pip_parse_vendored,examples/pip_repository_annotations,examples/py_proto_library,examples/py_proto_library/example.com/another_proto,examples/py_proto_library/example.com/proto,gazelle,gazelle/manifest,gazelle/manifest/generate,gazelle/manifest/hasher,gazelle/manifest/test,gazelle/modules_mapping,gazelle/python,gazelle/pythonconfig,gazelle/python/private,tests/integration/compile_pip_requirements,tests/integration/compile_pip_requirements_test_from_external_repo,tests/integration/custom_commands,tests/integration/ignore_root_user_error,tests/integration/ignore_root_user_error/submodule,tests/integration/local_toolchains,tests/integration/pip_parse,tests/integration/pip_parse/empty,tests/integration/py_cc_toolchain_registered,tests/modules/other,tests/modules/other/nspkg_delta,tests/modules/other/nspkg_gamma +build --deleted_packages=examples/build_file_generation,examples/build_file_generation/random_number_generator,examples/bzlmod,examples/bzlmod_build_file_generation,examples/bzlmod_build_file_generation/other_module/other_module/pkg,examples/bzlmod_build_file_generation/runfiles,examples/bzlmod/entry_points,examples/bzlmod/entry_points/tests,examples/bzlmod/libs/my_lib,examples/bzlmod/other_module,examples/bzlmod/other_module/other_module/pkg,examples/bzlmod/patches,examples/bzlmod/py_proto_library,examples/bzlmod/py_proto_library/example.com/another_proto,examples/bzlmod/py_proto_library/example.com/proto,examples/bzlmod/runfiles,examples/bzlmod/tests,examples/bzlmod/tests/other_module,examples/bzlmod/whl_mods,examples/multi_python_versions/libs/my_lib,examples/multi_python_versions/requirements,examples/multi_python_versions/tests,examples/pip_parse,examples/pip_parse_vendored,examples/pip_repository_annotations,examples/py_proto_library,examples/py_proto_library/example.com/another_proto,examples/py_proto_library/example.com/proto,gazelle,gazelle/manifest,gazelle/manifest/generate,gazelle/manifest/hasher,gazelle/manifest/test,gazelle/modules_mapping,gazelle/python,gazelle/pythonconfig,gazelle/python/private,tests/integration/compile_pip_requirements,tests/integration/compile_pip_requirements_test_from_external_repo,tests/integration/custom_commands,tests/integration/ignore_root_user_error,tests/integration/ignore_root_user_error/submodule,tests/integration/local_toolchains,tests/integration/pip_parse,tests/integration/pip_parse/empty,tests/integration/py_cc_toolchain_registered,tests/modules/other,tests/modules/other/nspkg_delta,tests/modules/other/nspkg_gamma,tests/modules/other/nspkg_single +query --deleted_packages=examples/build_file_generation,examples/build_file_generation/random_number_generator,examples/bzlmod,examples/bzlmod_build_file_generation,examples/bzlmod_build_file_generation/other_module/other_module/pkg,examples/bzlmod_build_file_generation/runfiles,examples/bzlmod/entry_points,examples/bzlmod/entry_points/tests,examples/bzlmod/libs/my_lib,examples/bzlmod/other_module,examples/bzlmod/other_module/other_module/pkg,examples/bzlmod/patches,examples/bzlmod/py_proto_library,examples/bzlmod/py_proto_library/example.com/another_proto,examples/bzlmod/py_proto_library/example.com/proto,examples/bzlmod/runfiles,examples/bzlmod/tests,examples/bzlmod/tests/other_module,examples/bzlmod/whl_mods,examples/multi_python_versions/libs/my_lib,examples/multi_python_versions/requirements,examples/multi_python_versions/tests,examples/pip_parse,examples/pip_parse_vendored,examples/pip_repository_annotations,examples/py_proto_library,examples/py_proto_library/example.com/another_proto,examples/py_proto_library/example.com/proto,gazelle,gazelle/manifest,gazelle/manifest/generate,gazelle/manifest/hasher,gazelle/manifest/test,gazelle/modules_mapping,gazelle/python,gazelle/pythonconfig,gazelle/python/private,tests/integration/compile_pip_requirements,tests/integration/compile_pip_requirements_test_from_external_repo,tests/integration/custom_commands,tests/integration/ignore_root_user_error,tests/integration/ignore_root_user_error/submodule,tests/integration/local_toolchains,tests/integration/pip_parse,tests/integration/pip_parse/empty,tests/integration/py_cc_toolchain_registered,tests/modules/other,tests/modules/other/nspkg_delta,tests/modules/other/nspkg_gamma,tests/modules/other/nspkg_single test --test_output=errors diff --git a/python/private/py_library.bzl b/python/private/py_library.bzl index bf0c25439e..fd9dad9f20 100644 --- a/python/private/py_library.bzl +++ b/python/private/py_library.bzl @@ -253,6 +253,7 @@ def _get_site_packages_symlinks(ctx): repo_runfiles_dirname = None dirs_with_init = {} # dirname -> runfile path + site_packages_symlinks = [] for src in ctx.files.srcs: if src.extension not in PYTHON_FILE_EXTENSIONS: continue @@ -261,16 +262,19 @@ def _get_site_packages_symlinks(ctx): continue path = path.removeprefix(site_packages_root) dir_name, _, filename = path.rpartition("/") - if not dir_name: - # This would be e.g. `site-packages/__init__.py`, which isn't valid - # because it's not within a directory for an importable Python package. - # However, the pypi integration over-eagerly adds a pkgutil-style - # __init__.py file during the repo phase. Just ignore them for now. - continue - if filename.startswith("__init__."): + if dir_name and filename.startswith("__init__."): dirs_with_init[dir_name] = None repo_runfiles_dirname = runfiles_root_path(ctx, src.short_path).partition("/")[0] + elif not dir_name: + repo_runfiles_dirname = runfiles_root_path(ctx, src.short_path).partition("/")[0] + + # This would be files that do not have directories and we just need to add + # direct symlinks to them as is: + site_packages_symlinks.append(( + paths.join(repo_runfiles_dirname, site_packages_root, filename), + filename, + )) # Sort so that we encounter `foo` before `foo/bar`. This ensures we # see the top-most explicit package first. @@ -286,7 +290,6 @@ def _get_site_packages_symlinks(ctx): if not is_sub_package: first_level_explicit_packages.append(d) - site_packages_symlinks = [] for dirname in first_level_explicit_packages: site_packages_symlinks.append(( paths.join(repo_runfiles_dirname, site_packages_root, dirname), diff --git a/tests/modules/other/nspkg_single/BUILD.bazel b/tests/modules/other/nspkg_single/BUILD.bazel new file mode 100644 index 0000000000..08cb4f373e --- /dev/null +++ b/tests/modules/other/nspkg_single/BUILD.bazel @@ -0,0 +1,10 @@ +load("@rules_python//python:py_library.bzl", "py_library") + +package(default_visibility = ["//visibility:public"]) + +py_library( + name = "nspkg_single", + srcs = glob(["site-packages/**/*.py"]), + experimental_venvs_site_packages = "@rules_python//python/config_settings:venvs_site_packages", + imports = [package_name() + "/site-packages"], +) diff --git a/tests/modules/other/nspkg_single/site-packages/__init__.py b/tests/modules/other/nspkg_single/site-packages/__init__.py new file mode 100644 index 0000000000..bb26c87599 --- /dev/null +++ b/tests/modules/other/nspkg_single/site-packages/__init__.py @@ -0,0 +1 @@ +# empty, will not be added to the site-packages dir diff --git a/tests/modules/other/nspkg_single/site-packages/single_file.py b/tests/modules/other/nspkg_single/site-packages/single_file.py new file mode 100644 index 0000000000..f6d7dfd640 --- /dev/null +++ b/tests/modules/other/nspkg_single/site-packages/single_file.py @@ -0,0 +1,5 @@ +__all__ = [ + "SOMETHING", +] + +SOMETHING = "nothing" diff --git a/tests/venv_site_packages_libs/BUILD.bazel b/tests/venv_site_packages_libs/BUILD.bazel index 1f48331ff2..d5a4fe6750 100644 --- a/tests/venv_site_packages_libs/BUILD.bazel +++ b/tests/venv_site_packages_libs/BUILD.bazel @@ -13,5 +13,6 @@ py_reconfig_test( "//tests/venv_site_packages_libs/nspkg_beta", "@other//nspkg_delta", "@other//nspkg_gamma", + "@other//nspkg_single", ], ) diff --git a/tests/venv_site_packages_libs/bin.py b/tests/venv_site_packages_libs/bin.py index b944be69e3..58572a2a1e 100644 --- a/tests/venv_site_packages_libs/bin.py +++ b/tests/venv_site_packages_libs/bin.py @@ -26,6 +26,7 @@ def test_imported_from_venv(self): self.assert_imported_from_venv("nspkg.subnspkg.beta") self.assert_imported_from_venv("nspkg.subnspkg.gamma") self.assert_imported_from_venv("nspkg.subnspkg.delta") + self.assert_imported_from_venv("single_file") if __name__ == "__main__": diff --git a/tests/venv_site_packages_libs/nspkg_alpha/BUILD.bazel b/tests/venv_site_packages_libs/nspkg_alpha/BUILD.bazel index c40c3b4080..aec415f7a0 100644 --- a/tests/venv_site_packages_libs/nspkg_alpha/BUILD.bazel +++ b/tests/venv_site_packages_libs/nspkg_alpha/BUILD.bazel @@ -1,4 +1,4 @@ -load("@rules_python//python:py_library.bzl", "py_library") +load("//python:py_library.bzl", "py_library") package(default_visibility = ["//visibility:public"]) diff --git a/tests/venv_site_packages_libs/venv_site_packages_pypi_test.py b/tests/venv_site_packages_libs/venv_site_packages_pypi_test.py deleted file mode 100644 index 519b258044..0000000000 --- a/tests/venv_site_packages_libs/venv_site_packages_pypi_test.py +++ /dev/null @@ -1,36 +0,0 @@ -import os -import sys -import unittest - - -class VenvSitePackagesLibraryTest(unittest.TestCase): - def test_imported_from_venv(self): - self.assertNotEqual(sys.prefix, sys.base_prefix, "Not running under a venv") - venv = sys.prefix - - from nspkg.subnspkg import alpha - - self.assertEqual(alpha.whoami, "alpha") - self.assertEqual(alpha.__name__, "nspkg.subnspkg.alpha") - - self.assertTrue( - alpha.__file__.startswith(sys.prefix), - f"\nalpha was imported, not from within the venv.\n" - + f"venv : {venv}\n" - + f"actual: {alpha.__file__}", - ) - - from nspkg.subnspkg import beta - - self.assertEqual(beta.whoami, "beta") - self.assertEqual(beta.__name__, "nspkg.subnspkg.beta") - self.assertTrue( - beta.__file__.startswith(sys.prefix), - f"\nbeta was imported, not from within the venv.\n" - + f"venv : {venv}\n" - + f"actual: {beta.__file__}", - ) - - -if __name__ == "__main__": - unittest.main() From 0d203a95d9ba6ec3365119fc709dc9eb3885f6d7 Mon Sep 17 00:00:00 2001 From: Ignas Anikevicius <240938+aignas@users.noreply.github.com> Date: Thu, 29 May 2025 11:27:28 +0900 Subject: [PATCH 261/922] docs: split PyPI docs up and add more (#2935) Summary: - Split the PyPI docs per topic. - Move everything to its own folder. - Separate the `bzlmod` and `WORKSPACE` documentation. Some of the features are only available in `bzlmod` and since `bzlmod` is the future having that as the default makes things a little easier. - Fix a few warnings. Fixes #2810. --- CONTRIBUTING.md | 2 +- MODULE.bazel | 1 + docs/BUILD.bazel | 3 + docs/conf.py | 4 + docs/getting-started.md | 10 +- docs/index.md | 3 +- docs/pip.md | 4 - docs/pypi-dependencies.md | 519 ------------------ docs/pypi/circular-dependencies.md | 82 +++ docs/pypi/download-workspace.md | 107 ++++ docs/pypi/download.md | 302 ++++++++++ docs/pypi/index.md | 27 + docs/pypi/lock.md | 46 ++ docs/pypi/patch.md | 10 + docs/pypi/use.md | 133 +++++ docs/requirements.txt | 1 - python/private/pypi/BUILD.bazel | 1 + python/private/pypi/pkg_aliases.bzl | 5 +- python/private/pypi/simpleapi_download.bzl | 1 + python/private/pypi/whl_config_setting.bzl | 8 +- sphinxdocs/inventories/bazel_inventory.txt | 4 + .../simpleapi_download_tests.bzl | 5 + 22 files changed, 740 insertions(+), 538 deletions(-) delete mode 100644 docs/pip.md delete mode 100644 docs/pypi-dependencies.md create mode 100644 docs/pypi/circular-dependencies.md create mode 100644 docs/pypi/download-workspace.md create mode 100644 docs/pypi/download.md create mode 100644 docs/pypi/index.md create mode 100644 docs/pypi/lock.md create mode 100644 docs/pypi/patch.md create mode 100644 docs/pypi/use.md diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index b087119dc6..324801cfc3 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -68,7 +68,7 @@ to the actual rules_python project and begin the code review process. ## Developer guide For more more details, guidance, and tips for working with the code base, -see [DEVELOPING.md](DEVELOPING.md) +see [docs/devguide.md](./devguide) ## Formatting diff --git a/MODULE.bazel b/MODULE.bazel index fa24ed04ba..d3a95350e5 100644 --- a/MODULE.bazel +++ b/MODULE.bazel @@ -134,6 +134,7 @@ dev_pip.parse( download_only = True, experimental_index_url = "https://pypi.org/simple", hub_name = "dev_pip", + parallel_download = False, python_version = "3.11", requirements_lock = "//docs:requirements.txt", ) diff --git a/docs/BUILD.bazel b/docs/BUILD.bazel index b3e5f52022..852c4d4fa6 100644 --- a/docs/BUILD.bazel +++ b/docs/BUILD.bazel @@ -120,7 +120,10 @@ sphinx_stardocs( "//python/private:rule_builders_bzl", "//python/private/api:py_common_api_bzl", "//python/private/pypi:config_settings_bzl", + "//python/private/pypi:env_marker_info_bzl", "//python/private/pypi:pkg_aliases_bzl", + "//python/private/pypi:whl_config_setting_bzl", + "//python/private/pypi:whl_library_bzl", "//python/uv:lock_bzl", "//python/uv:uv_bzl", "//python/uv:uv_toolchain_bzl", diff --git a/docs/conf.py b/docs/conf.py index 96bbdb50ab..1d9f526b93 100644 --- a/docs/conf.py +++ b/docs/conf.py @@ -91,6 +91,8 @@ "api/sphinxdocs/private/sphinx_docs_library": "/api/sphinxdocs/sphinxdocs/private/sphinx_docs_library.html", "api/sphinxdocs/sphinx_docs_library": "/api/sphinxdocs/sphinxdocs/sphinx_docs_library.html", "api/sphinxdocs/inventories/index": "/api/sphinxdocs/sphinxdocs/inventories/index.html", + "pip.html": "pypi/index.html", + "pypi-dependencies.html": "pypi/index.html", } # Adapted from the template code: @@ -139,7 +141,9 @@ # --- Extlinks configuration extlinks = { + "gh-issue": (f"https://github.com/bazel-contrib/rules_python/issues/%s", "#%s issue"), "gh-path": (f"https://github.com/bazel-contrib/rules_python/tree/main/%s", "%s"), + "gh-pr": (f"https://github.com/bazel-contrib/rules_python/pulls/%s", "#%s PR"), } # --- MyST configuration diff --git a/docs/getting-started.md b/docs/getting-started.md index 60d5d5e0be..7e7b88aa8a 100644 --- a/docs/getting-started.md +++ b/docs/getting-started.md @@ -8,13 +8,13 @@ It assumes you have a `requirements.txt` file with your PyPI dependencies. For more details information about configuring `rules_python`, see: * [Configuring the runtime](configuring-toolchains) -* [Configuring third party dependencies (pip/pypi)](pypi-dependencies) +* [Configuring third party dependencies (pip/pypi)](./pypi/index) * [API docs](api/index) -## Using bzlmod +## Including dependencies -The first step to using rules_python with bzlmod is to add the dependency to -your MODULE.bazel file: +The first step to using `rules_python` is to add the dependency to +your `MODULE.bazel` file: ```starlark # Update the version "0.0.0" to the release found here: @@ -30,7 +30,7 @@ pip.parse( use_repo(pip, "pypi") ``` -## Using a WORKSPACE file +### Using a WORKSPACE file Using WORKSPACE is deprecated, but still supported, and a bit more involved than using Bzlmod. Here is a simplified setup to download the prebuilt runtimes. diff --git a/docs/index.md b/docs/index.md index 4983a6a029..82023f3ad8 100644 --- a/docs/index.md +++ b/docs/index.md @@ -95,9 +95,8 @@ See {gh-path}`Bzlmod support ` for any behaviour differences :hidden: self getting-started -pypi-dependencies +pypi/index Toolchains -pip coverage precompiling gazelle diff --git a/docs/pip.md b/docs/pip.md deleted file mode 100644 index 43d8fc4978..0000000000 --- a/docs/pip.md +++ /dev/null @@ -1,4 +0,0 @@ -(pip-integration)= -# Pip Integration - -See [PyPI dependencies](./pypi-dependencies). diff --git a/docs/pypi-dependencies.md b/docs/pypi-dependencies.md deleted file mode 100644 index b3ae7fe594..0000000000 --- a/docs/pypi-dependencies.md +++ /dev/null @@ -1,519 +0,0 @@ -:::{default-domain} bzl -::: - -# Using dependencies from PyPI - -Using PyPI packages (aka "pip install") involves two main steps. - -1. [Generating requirements file](#generating-requirements-file) -2. [Installing third party packages](#installing-third-party-packages) -3. [Using third party packages as dependencies](#using-third-party-packages) - -{#generating-requirements-file} -## Generating requirements file - -Generally, when working on a Python project, you'll have some dependencies that themselves have other dependencies. You might also specify dependency bounds instead of specific versions. So you'll need to generate a full list of all transitive dependencies and pinned versions for every dependency. - -Typically, you'd have your dependencies specified in `pyproject.toml` or `requirements.in` and generate the full pinned list of dependencies in `requirements_lock.txt`, which you can manage with the `compile_pip_requirements` Bazel rule: - -```starlark -load("@rules_python//python:pip.bzl", "compile_pip_requirements") - -compile_pip_requirements( - name = "requirements", - src = "requirements.in", - requirements_txt = "requirements_lock.txt", -) -``` - -This rule generates two targets: -- `bazel run [name].update` will regenerate the `requirements_txt` file -- `bazel test [name]_test` will test that the `requirements_txt` file is up to date - -For more documentation, see the API docs under {obj}`@rules_python//python:pip.bzl`. - -Once you generate this fully specified list of requirements, you can install the requirements with the instructions in [Installing third party packages](#installing-third-party-packages). - -:::{warning} -If you're specifying dependencies in `pyproject.toml`, make sure to include the `[build-system]` configuration, with pinned dependencies. `compile_pip_requirements` will use the build system specified to read your project's metadata, and you might see non-hermetic behavior if you don't pin the build system. - -Not specifying `[build-system]` at all will result in using a default `[build-system]` configuration, which uses unpinned versions ([ref](https://peps.python.org/pep-0518/#build-system-table)). -::: - -{#installing-third-party-packages} -## Installing third party packages - -### Using bzlmod - -To add pip dependencies to your `MODULE.bazel` file, use the `pip.parse` -extension, and call it to create the central external repo and individual wheel -external repos. Include in the `MODULE.bazel` the toolchain extension as shown -in the first bzlmod example above. - -```starlark -pip = use_extension("@rules_python//python/extensions:pip.bzl", "pip") -pip.parse( - hub_name = "my_deps", - python_version = "3.11", - requirements_lock = "//:requirements_lock_3_11.txt", -) -use_repo(pip, "my_deps") -``` -For more documentation, see the bzlmod examples under the {gh-path}`examples` folder or the documentation -for the {obj}`@rules_python//python/extensions:pip.bzl` extension. - -```{note} -We are using a host-platform compatible toolchain by default to setup pip dependencies. -During the setup phase, we create some symlinks, which may be inefficient on Windows -by default. In that case use the following `.bazelrc` options to improve performance if -you have admin privileges: - - startup --windows_enable_symlinks - -This will enable symlinks on Windows and help with bootstrap performance of setting up the -hermetic host python interpreter on this platform. Linux and OSX users should see no -difference. -``` - -### Using a WORKSPACE file - -To add pip dependencies to your `WORKSPACE`, load the `pip_parse` function and -call it to create the central external repo and individual wheel external repos. - -```starlark -load("@rules_python//python:pip.bzl", "pip_parse") - -# Create a central repo that knows about the dependencies needed from -# requirements_lock.txt. -pip_parse( - name = "my_deps", - requirements_lock = "//path/to:requirements_lock.txt", -) -# Load the starlark macro, which will define your dependencies. -load("@my_deps//:requirements.bzl", "install_deps") -# Call it to define repos for your requirements. -install_deps() -``` - -(vendoring-requirements)= -#### Vendoring the requirements.bzl file - -In some cases you may not want to generate the requirements.bzl file as a repository rule -while Bazel is fetching dependencies. For example, if you produce a reusable Bazel module -such as a ruleset, you may want to include the requirements.bzl file rather than make your users -install the WORKSPACE setup to generate it. -See https://github.com/bazel-contrib/rules_python/issues/608 - -This is the same workflow as Gazelle, which creates `go_repository` rules with -[`update-repos`](https://github.com/bazelbuild/bazel-gazelle#update-repos) - -To do this, use the "write to source file" pattern documented in -https://blog.aspect.dev/bazel-can-write-to-the-source-folder -to put a copy of the generated requirements.bzl into your project. -Then load the requirements.bzl file directly rather than from the generated repository. -See the example in rules_python/examples/pip_parse_vendored. - -(per-os-arch-requirements)= -### Requirements for a specific OS/Architecture - -In some cases you may need to use different requirements files for different OS, Arch combinations. This is enabled via the `requirements_by_platform` attribute in `pip.parse` extension and the `pip_parse` repository rule. The keys of the dictionary are labels to the file and the values are a list of comma separated target (os, arch) tuples. - -For example: -```starlark - # ... - requirements_by_platform = { - "requirements_linux_x86_64.txt": "linux_x86_64", - "requirements_osx.txt": "osx_*", - "requirements_linux_exotic.txt": "linux_exotic", - "requirements_some_platforms.txt": "linux_aarch64,windows_*", - }, - # For the list of standard platforms that the rules_python has toolchains for, default to - # the following requirements file. - requirements_lock = "requirements_lock.txt", -``` - -In case of duplicate platforms, `rules_python` will raise an error as there has -to be unambiguous mapping of the requirement files to the (os, arch) tuples. - -An alternative way is to use per-OS requirement attributes. -```starlark - # ... - requirements_windows = "requirements_windows.txt", - requirements_darwin = "requirements_darwin.txt", - # For the remaining platforms (which is basically only linux OS), use this file. - requirements_lock = "requirements_lock.txt", -) -``` - -### pip rules - -Note that since `pip_parse` and `pip.parse` are executed at evaluation time, -Bazel has no information about the Python toolchain and cannot enforce that the -interpreter used to invoke `pip` matches the interpreter used to run -`py_binary` targets. By default, `pip_parse` uses the system command -`"python3"`. To override this, pass in the `python_interpreter` attribute or -`python_interpreter_target` attribute to `pip_parse`. The `pip.parse` `bzlmod` extension -by default uses the hermetic python toolchain for the host platform. - -You can have multiple `pip_parse`s in the same workspace, or use the pip -extension multiple times when using bzlmod. This configuration will create -multiple external repos that have no relation to one another and may result in -downloading the same wheels numerous times. - -As with any repository rule, if you would like to ensure that `pip_parse` is -re-executed to pick up a non-hermetic change to your environment (e.g., updating -your system `python` interpreter), you can force it to re-execute by running -`bazel sync --only [pip_parse name]`. - -{#using-third-party-packages} -## Using third party packages as dependencies - -Each extracted wheel repo contains a `py_library` target representing -the wheel's contents. There are two ways to access this library. The -first uses the `requirement()` function defined in the central -repo's `//:requirements.bzl` file. This function maps a pip package -name to a label: - -```starlark -load("@my_deps//:requirements.bzl", "requirement") - -py_library( - name = "mylib", - srcs = ["mylib.py"], - deps = [ - ":myotherlib", - requirement("some_pip_dep"), - requirement("another_pip_dep"), - ] -) -``` - -The reason `requirement()` exists is to insulate from -changes to the underlying repository and label strings. However, those -labels have become directly used, so aren't able to easily change regardless. - -On the other hand, using `requirement()` has several drawbacks; see -[this issue][requirements-drawbacks] for an enumeration. If you don't -want to use `requirement()`, you can use the library -labels directly instead. For `pip_parse`, the labels are of the following form: - -```starlark -@{name}//{package} -``` - -Here `name` is the `name` attribute that was passed to `pip_parse` and -`package` is the pip package name with characters that are illegal in -Bazel label names (e.g. `-`, `.`) replaced with `_`. If you need to -update `name` from "old" to "new", then you can run the following -buildozer command: - -```shell -buildozer 'substitute deps @old//([^/]+) @new//${1}' //...:* -``` - -[requirements-drawbacks]: https://github.com/bazel-contrib/rules_python/issues/414 - -### Entry points - -If you would like to access [entry points][whl_ep], see the `py_console_script_binary` rule documentation, -which can help you create a `py_binary` target for a particular console script exposed by a package. - -[whl_ep]: https://packaging.python.org/specifications/entry-points/ - -### 'Extras' dependencies - -Any 'extras' specified in the requirements lock file will be automatically added -as transitive dependencies of the package. In the example above, you'd just put -`requirement("useful_dep")` or `@pypi//useful_dep`. - -### Consuming Wheel Dists Directly - -If you need to depend on the wheel dists themselves, for instance, to pass them -to some other packaging tool, you can get a handle to them with the -`whl_requirement` macro. For example: - -```starlark -load("@pypi//:requirements.bzl", "whl_requirement") - -filegroup( - name = "whl_files", - data = [ - # This is equivalent to "@pypi//boto3:whl" - whl_requirement("boto3"), - ] -) -``` - -### Creating a filegroup of files within a whl - -The rule {obj}`whl_filegroup` exists as an easy way to extract the necessary files -from a whl file without the need to modify the `BUILD.bazel` contents of the -whl repositories generated via `pip_repository`. Use it similarly to the `filegroup` -above. See the API docs for more information. - -(advance-topics)= -## Advanced topics - -(circular-deps)= -### Circular dependencies - -Sometimes PyPi packages contain dependency cycles -- for instance a particular -version `sphinx` (this is no longer the case in the latest version as of -2024-06-02) depends on `sphinxcontrib-serializinghtml`. When using them as -`requirement()`s, ala - -``` -py_binary( - name = "doctool", - ... - deps = [ - requirement("sphinx"), - ], -) -``` - -Bazel will protest because it doesn't support cycles in the build graph -- - -``` -ERROR: .../external/pypi_sphinxcontrib_serializinghtml/BUILD.bazel:44:6: in alias rule @pypi_sphinxcontrib_serializinghtml//:pkg: cycle in dependency graph: - //:doctool (...) - @pypi//sphinxcontrib_serializinghtml:pkg (...) -.-> @pypi_sphinxcontrib_serializinghtml//:pkg (...) -| @pypi_sphinxcontrib_serializinghtml//:_pkg (...) -| @pypi_sphinx//:pkg (...) -| @pypi_sphinx//:_pkg (...) -`-- @pypi_sphinxcontrib_serializinghtml//:pkg (...) -``` - -The `experimental_requirement_cycles` argument allows you to work around these -issues by specifying groups of packages which form cycles. `pip_parse` will -transparently fix the cycles for you and provide the cyclic dependencies -simultaneously. - -```starlark -pip_parse( - ... - experimental_requirement_cycles = { - "sphinx": [ - "sphinx", - "sphinxcontrib-serializinghtml", - ] - }, -) -``` - -`pip_parse` supports fixing multiple cycles simultaneously, however cycles must -be distinct. `apache-airflow` for instance has dependency cycles with a number -of its optional dependencies, which means those optional dependencies must all -be a part of the `airflow` cycle. For instance -- - -```starlark -pip_parse( - ... - experimental_requirement_cycles = { - "airflow": [ - "apache-airflow", - "apache-airflow-providers-common-sql", - "apache-airflow-providers-postgres", - "apache-airflow-providers-sqlite", - ] - } -) -``` - -Alternatively, one could resolve the cycle by removing one leg of it. - -For example while `apache-airflow-providers-sqlite` is "baked into" the Airflow -package, `apache-airflow-providers-postgres` is not and is an optional feature. -Rather than listing `apache-airflow[postgres]` in your `requirements.txt` which -would expose a cycle via the extra, one could either _manually_ depend on -`apache-airflow` and `apache-airflow-providers-postgres` separately as -requirements. Bazel rules which need only `apache-airflow` can take it as a -dependency, and rules which explicitly want to mix in -`apache-airflow-providers-postgres` now can. - -Alternatively, one could use `rules_python`'s patching features to remove one -leg of the dependency manually. For instance by making -`apache-airflow-providers-postgres` not explicitly depend on `apache-airflow` or -perhaps `apache-airflow-providers-common-sql`. - - -### Multi-platform support - -Multi-platform support of cross-building the wheels can be done in two ways - either -using {bzl:attr}`experimental_index_url` for the {bzl:obj}`pip.parse` bzlmod tag class -or by using the {bzl:attr}`pip.parse.download_only` setting. In this section we -are going to outline quickly how one can use the latter option. - -Let's say you have 2 requirements files: -``` -# requirements.linux_x86_64.txt ---platform=manylinux_2_17_x86_64 ---python-version=39 ---implementation=cp ---abi=cp39 - -foo==0.0.1 --hash=sha256:deadbeef -bar==0.0.1 --hash=sha256:deadb00f -``` - -``` -# requirements.osx_aarch64.txt contents ---platform=macosx_10_9_arm64 ---python-version=39 ---implementation=cp ---abi=cp39 - -foo==0.0.3 --hash=sha256:deadbaaf -``` - -With these 2 files your {bzl:obj}`pip.parse` could look like: -``` -pip.parse( - hub_name = "pip", - python_version = "3.9", - # Tell `pip` to ignore sdists - download_only = True, - requirements_by_platform = { - "requirements.linux_x86_64.txt": "linux_x86_64", - "requirements.osx_aarch64.txt": "osx_aarch64", - }, -) -``` - -With this, the `pip.parse` will create a hub repository that is going to -support only two platforms - `cp39_osx_aarch64` and `cp39_linux_x86_64` and it -will only use `wheels` and ignore any sdists that it may find on the PyPI -compatible indexes. - -```{note} -This is only supported on `bzlmd`. -``` - - - -(bazel-downloader)= -### Bazel downloader and multi-platform wheel hub repository. - -The `bzlmod` `pip.parse` call supports pulling information from `PyPI` (or a -compatible mirror) and it will ensure that the [bazel -downloader][bazel_downloader] is used for downloading the wheels. This allows -the users to use the [credential helper](#credential-helper) to authenticate -with the mirror and it also ensures that the distribution downloads are cached. -It also avoids using `pip` altogether and results in much faster dependency -fetching. - -This can be enabled by `experimental_index_url` and related flags as shown in -the {gh-path}`examples/bzlmod/MODULE.bazel` example. - -When using this feature during the `pip` extension evaluation you will see the accessed indexes similar to below: -```console -Loading: 0 packages loaded - currently loading: docs/ - Fetching module extension pip in @@//python/extensions:pip.bzl; starting - Fetching https://pypi.org/simple/twine/ -``` - -This does not mean that `rules_python` is fetching the wheels eagerly, but it -rather means that it is calling the PyPI server to get the Simple API response -to get the list of all available source and wheel distributions. Once it has -got all of the available distributions, it will select the right ones depending -on the `sha256` values in your `requirements_lock.txt` file. If `sha256` hashes -are not present in the requirements file, we will fallback to matching by version -specified in the lock file. The compatible distribution URLs will be then -written to the `MODULE.bazel.lock` file. Currently users wishing to use the -lock file with `rules_python` with this feature have to set an environment -variable `RULES_PYTHON_OS_ARCH_LOCK_FILE=0` which will become default in the -next release. - -Fetching the distribution information from the PyPI allows `rules_python` to -know which `whl` should be used on which target platform and it will determine -that by parsing the `whl` filename based on [PEP600], [PEP656] standards. This -allows the user to configure the behaviour by using the following publicly -available flags: -* {obj}`--@rules_python//python/config_settings:py_linux_libc` for selecting the Linux libc variant. -* {obj}`--@rules_python//python/config_settings:pip_whl` for selecting `whl` distribution preference. -* {obj}`--@rules_python//python/config_settings:pip_whl_osx_arch` for selecting MacOS wheel preference. -* {obj}`--@rules_python//python/config_settings:pip_whl_glibc_version` for selecting the GLIBC version compatibility. -* {obj}`--@rules_python//python/config_settings:pip_whl_muslc_version` for selecting the musl version compatibility. -* {obj}`--@rules_python//python/config_settings:pip_whl_osx_version` for selecting MacOS version compatibility. - -[bazel_downloader]: https://bazel.build/rules/lib/builtins/repository_ctx#download -[pep600]: https://peps.python.org/pep-0600/ -[pep656]: https://peps.python.org/pep-0656/ - -(credential-helper)= -### Credential Helper - -The "use Bazel downloader for python wheels" experimental feature includes support for the Bazel -[Credential Helper][cred-helper-design]. - -Your python artifact registry may provide a credential helper for you. Refer to your index's docs -to see if one is provided. - -See the [Credential Helper Spec][cred-helper-spec] for details. - -[cred-helper-design]: https://github.com/bazelbuild/proposals/blob/main/designs/2022-06-07-bazel-credential-helpers.md -[cred-helper-spec]: https://github.com/EngFlow/credential-helper-spec/blob/main/spec.md - - -#### Basic Example: - -The simplest form of a credential helper is a bash script that accepts an arg and spits out JSON to -stdout. For a service like Google Artifact Registry that uses ['Basic' HTTP Auth][rfc7617] and does -not provide a credential helper that conforms to the [spec][cred-helper-spec], the script might -look like: - -```bash -#!/bin/bash -# cred_helper.sh -ARG=$1 # but we don't do anything with it as it's always "get" - -# formatting is optional -echo '{' -echo ' "headers": {' -echo ' "Authorization": ["Basic dGVzdDoxMjPCow=="]' -echo ' }' -echo '}' -``` - -Configure Bazel to use this credential helper for your python index `example.com`: - -``` -# .bazelrc -build --credential_helper=example.com=/full/path/to/cred_helper.sh -``` - -Bazel will call this file like `cred_helper.sh get` and use the returned JSON to inject headers -into whatever HTTP(S) request it performs against `example.com`. - -[rfc7617]: https://datatracker.ietf.org/doc/html/rfc7617 - - diff --git a/docs/pypi/circular-dependencies.md b/docs/pypi/circular-dependencies.md new file mode 100644 index 0000000000..d22f5b36a7 --- /dev/null +++ b/docs/pypi/circular-dependencies.md @@ -0,0 +1,82 @@ +:::{default-domain} bzl +::: + +# Circular dependencies + +Sometimes PyPi packages contain dependency cycles -- for instance a particular +version `sphinx` (this is no longer the case in the latest version as of +2024-06-02) depends on `sphinxcontrib-serializinghtml`. When using them as +`requirement()`s, ala + +```starlark +py_binary( + name = "doctool", + ... + deps = [ + requirement("sphinx"), + ], +) +``` + +Bazel will protest because it doesn't support cycles in the build graph -- + +``` +ERROR: .../external/pypi_sphinxcontrib_serializinghtml/BUILD.bazel:44:6: in alias rule @pypi_sphinxcontrib_serializinghtml//:pkg: cycle in dependency graph: + //:doctool (...) + @pypi//sphinxcontrib_serializinghtml:pkg (...) +.-> @pypi_sphinxcontrib_serializinghtml//:pkg (...) +| @pypi_sphinxcontrib_serializinghtml//:_pkg (...) +| @pypi_sphinx//:pkg (...) +| @pypi_sphinx//:_pkg (...) +`-- @pypi_sphinxcontrib_serializinghtml//:pkg (...) +``` + +The `experimental_requirement_cycles` attribute allows you to work around these +issues by specifying groups of packages which form cycles. `pip_parse` will +transparently fix the cycles for you and provide the cyclic dependencies +simultaneously. + +```starlark + ... + experimental_requirement_cycles = { + "sphinx": [ + "sphinx", + "sphinxcontrib-serializinghtml", + ] + }, +) +``` + +`pip_parse` supports fixing multiple cycles simultaneously, however cycles must +be distinct. `apache-airflow` for instance has dependency cycles with a number +of its optional dependencies, which means those optional dependencies must all +be a part of the `airflow` cycle. For instance -- + +```starlark + ... + experimental_requirement_cycles = { + "airflow": [ + "apache-airflow", + "apache-airflow-providers-common-sql", + "apache-airflow-providers-postgres", + "apache-airflow-providers-sqlite", + ] + } +) +``` + +Alternatively, one could resolve the cycle by removing one leg of it. + +For example while `apache-airflow-providers-sqlite` is "baked into" the Airflow +package, `apache-airflow-providers-postgres` is not and is an optional feature. +Rather than listing `apache-airflow[postgres]` in your `requirements.txt` which +would expose a cycle via the extra, one could either _manually_ depend on +`apache-airflow` and `apache-airflow-providers-postgres` separately as +requirements. Bazel rules which need only `apache-airflow` can take it as a +dependency, and rules which explicitly want to mix in +`apache-airflow-providers-postgres` now can. + +Alternatively, one could use `rules_python`'s patching features to remove one +leg of the dependency manually. For instance by making +`apache-airflow-providers-postgres` not explicitly depend on `apache-airflow` or +perhaps `apache-airflow-providers-common-sql`. diff --git a/docs/pypi/download-workspace.md b/docs/pypi/download-workspace.md new file mode 100644 index 0000000000..48710095a4 --- /dev/null +++ b/docs/pypi/download-workspace.md @@ -0,0 +1,107 @@ +:::{default-domain} bzl +::: + +# Download (WORKSPACE) + +This documentation page covers how to download the PyPI dependencies in the legacy `WORKSPACE` setup. + +To add pip dependencies to your `WORKSPACE`, load the `pip_parse` function and +call it to create the central external repo and individual wheel external repos. + +```starlark +load("@rules_python//python:pip.bzl", "pip_parse") + +# Create a central repo that knows about the dependencies needed from +# requirements_lock.txt. +pip_parse( + name = "my_deps", + requirements_lock = "//path/to:requirements_lock.txt", +) + +# Load the starlark macro, which will define your dependencies. +load("@my_deps//:requirements.bzl", "install_deps") + +# Call it to define repos for your requirements. +install_deps() +``` + +## Interpreter selection + +Note that pip parse runs before the Bazel before decides which Python toolchain to use, it cannot +enforce that the interpreter used to invoke `pip` matches the interpreter used to run `py_binary` +targets. By default, `pip_parse` uses the system command `"python3"`. To override this, pass in the +{attr}`pip_parse.python_interpreter` attribute or {attr}`pip_parse.python_interpreter_target`. + +You can have multiple `pip_parse`s in the same workspace. This configuration will create multiple +external repos that have no relation to one another and may result in downloading the same wheels +numerous times. + +As with any repository rule, if you would like to ensure that `pip_parse` is +re-executed to pick up a non-hermetic change to your environment (e.g., updating +your system `python` interpreter), you can force it to re-execute by running +`bazel sync --only [pip_parse name]`. + +(per-os-arch-requirements)= +## Requirements for a specific OS/Architecture + +In some cases you may need to use different requirements files for different OS, Arch combinations. +This is enabled via the {attr}`pip_parse.requirements_by_platform` attribute. The keys of the +dictionary are labels to the file and the values are a list of comma separated target (os, arch) +tuples. + +For example: +```starlark + # ... + requirements_by_platform = { + "requirements_linux_x86_64.txt": "linux_x86_64", + "requirements_osx.txt": "osx_*", + "requirements_linux_exotic.txt": "linux_exotic", + "requirements_some_platforms.txt": "linux_aarch64,windows_*", + }, + # For the list of standard platforms that the rules_python has toolchains for, default to + # the following requirements file. + requirements_lock = "requirements_lock.txt", +``` + +In case of duplicate platforms, `rules_python` will raise an error as there has +to be unambiguous mapping of the requirement files to the (os, arch) tuples. + +An alternative way is to use per-OS requirement attributes. +```starlark + # ... + requirements_windows = "requirements_windows.txt", + requirements_darwin = "requirements_darwin.txt", + # For the remaining platforms (which is basically only linux OS), use this file. + requirements_lock = "requirements_lock.txt", +) +``` + +:::{note} +If you are using a universal lock file but want to restrict the list of platforms that +the lock file will be evaluated against, consider using the aforementioned +`requirements_by_platform` attribute and listing the platforms explicitly. +::: + +(vendoring-requirements)= +## Vendoring the requirements.bzl file + +:::{note} +For `bzlmod`, refer to standard `bazel vendor` usage if you want to really vendor it, otherwise +just use the `pip` extension as you would normally. + +However, be aware that there are caveats when doing so. +::: + +In some cases you may not want to generate the requirements.bzl file as a repository rule +while Bazel is fetching dependencies. For example, if you produce a reusable Bazel module +such as a ruleset, you may want to include the `requirements.bzl` file rather than make your users +install the `WORKSPACE` setup to generate it, see {gh-issue}`608`. + +This is the same workflow as Gazelle, which creates `go_repository` rules with +[`update-repos`](https://github.com/bazelbuild/bazel-gazelle#update-repos) + +To do this, use the "write to source file" pattern documented in + +to put a copy of the generated `requirements.bzl` into your project. +Then load the requirements.bzl file directly rather than from the generated repository. +See the example in {gh-path}`examples/pip_parse_vendored`. diff --git a/docs/pypi/download.md b/docs/pypi/download.md new file mode 100644 index 0000000000..18d6699ab3 --- /dev/null +++ b/docs/pypi/download.md @@ -0,0 +1,302 @@ +:::{default-domain} bzl +::: + +# Download (bzlmod) + +:::{seealso} +For WORKSPACE instructions see [here](./download-workspace). +::: + +To add PyPI dependencies to your `MODULE.bazel` file, use the `pip.parse` +extension, and call it to create the central external repo and individual wheel +external repos. Include in the `MODULE.bazel` the toolchain extension as shown +in the first bzlmod example above. + +```starlark +pip = use_extension("@rules_python//python/extensions:pip.bzl", "pip") + +pip.parse( + hub_name = "my_deps", + python_version = "3.13", + requirements_lock = "//:requirements_lock_3_11.txt", +) + +use_repo(pip, "my_deps") +``` + +For more documentation, see the bzlmod examples under the {gh-path}`examples` folder or the documentation +for the {obj}`@rules_python//python/extensions:pip.bzl` extension. + +:::note} +We are using a host-platform compatible toolchain by default to setup pip dependencies. +During the setup phase, we create some symlinks, which may be inefficient on Windows +by default. In that case use the following `.bazelrc` options to improve performance if +you have admin privileges: + + startup --windows_enable_symlinks + +This will enable symlinks on Windows and help with bootstrap performance of setting up the +hermetic host python interpreter on this platform. Linux and OSX users should see no +difference. +::: + +## Interpreter selection + +The {obj}`pip.parse` `bzlmod` extension by default uses the hermetic python toolchain for the host +platform, but you can customize the interpreter using {attr}`pip.parse.python_interpreter` and +{attr}`pip.parse.python_interpreter_target`. + +You can use the pip extension multiple times. This configuration will create +multiple external repos that have no relation to one another and may result in +downloading the same wheels numerous times. + +As with any repository rule or extension, if you would like to ensure that `pip_parse` is +re-executed to pick up a non-hermetic change to your environment (e.g., updating your system +`python` interpreter), you can force it to re-execute by running `bazel sync --only [pip_parse +name]`. + +(per-os-arch-requirements)= +## Requirements for a specific OS/Architecture + +In some cases you may need to use different requirements files for different OS, Arch combinations. +This is enabled via the `requirements_by_platform` attribute in `pip.parse` extension and the +{obj}`pip.parse` tag class. The keys of the dictionary are labels to the file and the values are a +list of comma separated target (os, arch) tuples. + +For example: +```starlark + # ... + requirements_by_platform = { + "requirements_linux_x86_64.txt": "linux_x86_64", + "requirements_osx.txt": "osx_*", + "requirements_linux_exotic.txt": "linux_exotic", + "requirements_some_platforms.txt": "linux_aarch64,windows_*", + }, + # For the list of standard platforms that the rules_python has toolchains for, default to + # the following requirements file. + requirements_lock = "requirements_lock.txt", +``` + +In case of duplicate platforms, `rules_python` will raise an error as there has +to be unambiguous mapping of the requirement files to the (os, arch) tuples. + +An alternative way is to use per-OS requirement attributes. +```starlark + # ... + requirements_windows = "requirements_windows.txt", + requirements_darwin = "requirements_darwin.txt", + # For the remaining platforms (which is basically only linux OS), use this file. + requirements_lock = "requirements_lock.txt", +) +``` + +:::{note} +If you are using a universal lock file but want to restrict the list of platforms that +the lock file will be evaluated against, consider using the aforementioned +`requirements_by_platform` attribute and listing the platforms explicitly. +::: + +## Multi-platform support + +Historically the {obj}`pip_parse` and {obj}`pip.parse` have been only downloading/building +Python dependencies for the host platform that the `bazel` commands are executed on. Over +the years people started needing support for building containers and usually that involves +fetching dependencies for a particular target platform that may be other than the host +platform. + +Multi-platform support of cross-building the wheels can be done in two ways: +1. using {attr}`experimental_index_url` for the {bzl:obj}`pip.parse` bzlmod tag class +2. using {attr}`pip.parse.download_only` setting. + +:::{warning} +This will not for sdists with C extensions, but pure Python sdists may still work using the first +approach. +::: + +### Using `download_only` attribute + +Let's say you have 2 requirements files: +``` +# requirements.linux_x86_64.txt +--platform=manylinux_2_17_x86_64 +--python-version=39 +--implementation=cp +--abi=cp39 + +foo==0.0.1 --hash=sha256:deadbeef +bar==0.0.1 --hash=sha256:deadb00f +``` + +``` +# requirements.osx_aarch64.txt contents +--platform=macosx_10_9_arm64 +--python-version=39 +--implementation=cp +--abi=cp39 + +foo==0.0.3 --hash=sha256:deadbaaf +``` + +With these 2 files your {bzl:obj}`pip.parse` could look like: +```starlark +pip.parse( + hub_name = "pip", + python_version = "3.9", + # Tell `pip` to ignore sdists + download_only = True, + requirements_by_platform = { + "requirements.linux_x86_64.txt": "linux_x86_64", + "requirements.osx_aarch64.txt": "osx_aarch64", + }, +) +``` + +With this, the `pip.parse` will create a hub repository that is going to +support only two platforms - `cp39_osx_aarch64` and `cp39_linux_x86_64` and it +will only use `wheels` and ignore any sdists that it may find on the PyPI +compatible indexes. + +:::{warning} +Because bazel is not aware what exactly is downloaded, the same wheel may be downloaded +multiple times. +::: + +:::{note} +This will only work for wheel-only setups, i.e. all of your dependencies need to have wheels +available on the PyPI index that you use. +::: + +### Customizing `Requires-Dist` resolution + +:::{note} +Currently this is disabled by default, but you can turn it on using +{envvar}`RULES_PYTHON_ENABLE_PIPSTAR` environment variable. +::: + +In order to understand what dependencies to pull for a particular package +`rules_python` parses the `whl` file [`METADATA`][metadata]. +Packages can express dependencies via `Requires-Dist` and they can add conditions using +"environment markers", which represent the Python version, OS, etc. + +While the PyPI integration provides reasonable defaults to support most +platforms and environment markers, the values it uses can be customized in case +more esoteric configurations are needed. + +To customize the values used, you need to do two things: +1. Define a target that returns {obj}`EnvMarkerInfo` +2. Set the {obj}`//python/config_settings:pip_env_marker_config` flag to + the target defined in (1). + +The keys and values should be compatible with the [PyPA dependency specifiers +specification](https://packaging.python.org/en/latest/specifications/dependency-specifiers/). +This is not strictly enforced, however, so you can return a subset of keys or +additional keys, which become available during dependency evaluation. + +[metadata]: https://packaging.python.org/en/latest/specifications/core-metadata/ + +(bazel-downloader)= +### Bazel downloader and multi-platform wheel hub repository. + +:::{warning} +This is currently still experimental and whilst it has been proven to work in quite a few +environments, the APIs are still being finalized and there may be changes to the APIs for this +feature without much notice. + +The issues that you can subscribe to for updates are: +* {gh-issue}`260` +* {gh-issue}`1357` +::: + +The {obj}`pip` extension supports pulling information from `PyPI` (or a compatible mirror) and it +will ensure that the [bazel downloader][bazel_downloader] is used for downloading the wheels. + +This provides the following benefits: +* Integration with the [credential_helper](#credential-helper) to authenticate with private + mirrors. +* Cache the downloaded wheels speeding up the consecutive re-initialization of the repositories. +* Reuse the same instance of the wheel for multiple target platforms. +* Allow using transitions and targeting free-threaded and musl platforms more easily. +* Avoids `pip` for wheel fetching and results in much faster dependency fetching. + +To enable the feature specify {attr}`pip.parse.experimental_index_url` as shown in +the {gh-path}`examples/bzlmod/MODULE.bazel` example. + +Similar to [uv](https://docs.astral.sh/uv/configuration/indexes/), one can override the +index that is used for a single package. By default we first search in the index specified by +{attr}`pip.parse.experimental_index_url`, then we iterate through the +{attr}`pip.parse.experimental_extra_index_urls` unless there are overrides specified via +{attr}`pip.parse.experimental_index_url_overrides`. + +When using this feature during the `pip` extension evaluation you will see the accessed indexes similar to below: +```console +Loading: 0 packages loaded + Fetching module extension @@//python/extensions:pip.bzl%pip; Fetch package lists from PyPI index + Fetching https://pypi.org/simple/jinja2/ + +``` + +This does not mean that `rules_python` is fetching the wheels eagerly, but it +rather means that it is calling the PyPI server to get the Simple API response +to get the list of all available source and wheel distributions. Once it has +got all of the available distributions, it will select the right ones depending +on the `sha256` values in your `requirements_lock.txt` file. If `sha256` hashes +are not present in the requirements file, we will fallback to matching by version +specified in the lock file. + +Fetching the distribution information from the PyPI allows `rules_python` to +know which `whl` should be used on which target platform and it will determine +that by parsing the `whl` filename based on [PEP600], [PEP656] standards. This +allows the user to configure the behaviour by using the following publicly +available flags: +* {obj}`--@rules_python//python/config_settings:py_linux_libc` for selecting the Linux libc variant. +* {obj}`--@rules_python//python/config_settings:pip_whl` for selecting `whl` distribution preference. +* {obj}`--@rules_python//python/config_settings:pip_whl_osx_arch` for selecting MacOS wheel preference. +* {obj}`--@rules_python//python/config_settings:pip_whl_glibc_version` for selecting the GLIBC version compatibility. +* {obj}`--@rules_python//python/config_settings:pip_whl_muslc_version` for selecting the musl version compatibility. +* {obj}`--@rules_python//python/config_settings:pip_whl_osx_version` for selecting MacOS version compatibility. + +[bazel_downloader]: https://bazel.build/rules/lib/builtins/repository_ctx#download +[pep600]: https://peps.python.org/pep-0600/ +[pep656]: https://peps.python.org/pep-0656/ + +(credential-helper)= +## Credential Helper + +The [Bazel downloader](#bazel-downloader) usage allows for the Bazel +[Credential Helper][cred-helper-design]. +Your python artifact registry may provide a credential helper for you. +Refer to your index's docs to see if one is provided. + +The simplest form of a credential helper is a bash script that accepts an arg and spits out JSON to +stdout. For a service like Google Artifact Registry that uses ['Basic' HTTP Auth][rfc7617] and does +not provide a credential helper that conforms to the [spec][cred-helper-spec], the script might +look like: + +```bash +#!/bin/bash +# cred_helper.sh +ARG=$1 # but we don't do anything with it as it's always "get" + +# formatting is optional +echo '{' +echo ' "headers": {' +echo ' "Authorization": ["Basic dGVzdDoxMjPCow=="]' +echo ' }' +echo '}' +``` + +Configure Bazel to use this credential helper for your python index `example.com`: + +``` +# .bazelrc +build --credential_helper=example.com=/full/path/to/cred_helper.sh +``` + +Bazel will call this file like `cred_helper.sh get` and use the returned JSON to inject headers +into whatever HTTP(S) request it performs against `example.com`. + +See the [Credential Helper Spec][cred-helper-spec] for more details. + +[rfc7617]: https://datatracker.ietf.org/doc/html/rfc7617 +[cred-helper-design]: https://github.com/bazelbuild/proposals/blob/main/designs/2022-06-07-bazel-credential-helpers.md +[cred-helper-spec]: https://github.com/EngFlow/credential-helper-spec/blob/main/spec.md diff --git a/docs/pypi/index.md b/docs/pypi/index.md new file mode 100644 index 0000000000..c300124398 --- /dev/null +++ b/docs/pypi/index.md @@ -0,0 +1,27 @@ +:::{default-domain} bzl +::: + +# Using PyPI + +Using PyPI packages (aka "pip install") involves the following main steps. + +1. [Generating requirements file](./lock) +2. Installing third party packages in [bzlmod](./download) or [WORKSPACE](./download-workspace). +3. [Using third party packages as dependencies](./use) + +With the advanced topics covered separately: +* Dealing with [circular dependencies](./circular-dependencies). + +```{toctree} +lock +download +download-workspace +use +``` + +## Advanced topics + +```{toctree} +circular-dependencies +patch +``` diff --git a/docs/pypi/lock.md b/docs/pypi/lock.md new file mode 100644 index 0000000000..c9376036fb --- /dev/null +++ b/docs/pypi/lock.md @@ -0,0 +1,46 @@ +:::{default-domain} bzl +::: + +# Lock + +:::{note} +Currently `rules_python` only supports `requirements.txt` format. +::: + +## requirements.txt + +### pip compile + +Generally, when working on a Python project, you'll have some dependencies that themselves have other dependencies. You might also specify dependency bounds instead of specific versions. So you'll need to generate a full list of all transitive dependencies and pinned versions for every dependency. + +Typically, you'd have your project dependencies specified in `pyproject.toml` or `requirements.in` and generate the full pinned list of dependencies in `requirements_lock.txt`, which you can manage with the {obj}`compile_pip_requirements`: + +```starlark +load("@rules_python//python:pip.bzl", "compile_pip_requirements") + +compile_pip_requirements( + name = "requirements", + src = "requirements.in", + requirements_txt = "requirements_lock.txt", +) +``` + +This rule generates two targets: +- `bazel run [name].update` will regenerate the `requirements_txt` file +- `bazel test [name]_test` will test that the `requirements_txt` file is up to date + +Once you generate this fully specified list of requirements, you can install the requirements ([bzlmod](./download)/[WORKSPACE](./download-workspace)). + +:::{warning} +If you're specifying dependencies in `pyproject.toml`, make sure to include the `[build-system]` configuration, with pinned dependencies. `compile_pip_requirements` will use the build system specified to read your project's metadata, and you might see non-hermetic behavior if you don't pin the build system. + +Not specifying `[build-system]` at all will result in using a default `[build-system]` configuration, which uses unpinned versions ([ref](https://peps.python.org/pep-0518/#build-system-table)). +::: + +### uv pip compile (bzlmod only) + +We also have experimental setup for the `uv pip compile` way of generating lock files. +This is well tested with the public PyPI index, but you may hit some rough edges with private +mirrors. + +For more documentation see {obj}`lock` documentation. diff --git a/docs/pypi/patch.md b/docs/pypi/patch.md new file mode 100644 index 0000000000..f341bd1091 --- /dev/null +++ b/docs/pypi/patch.md @@ -0,0 +1,10 @@ +:::{default-domain} bzl +::: + +# Patching wheels + +Sometimes the wheels have to be patched to: +* Workaround the lack of a standard `site-packages` layout ({gh-issue}`2156`) +* Include certain PRs of your choice on top of wheels and avoid building from sdist, + +You can patch the wheels by using the {attr}`pip.override.patches` attribute. diff --git a/docs/pypi/use.md b/docs/pypi/use.md new file mode 100644 index 0000000000..7a16b7d9e9 --- /dev/null +++ b/docs/pypi/use.md @@ -0,0 +1,133 @@ +:::{default-domain} bzl +::: + +# Use in BUILD.bazel files + +Once you have setup the dependencies, you are ready to start using them in your `BUILD.bazel` +files. If you haven't done so yet, set it up by following the following docs: +1. [WORKSPACE](./download-workspace) +1. [bzlmod](./download) + +To refer to targets in a hub repo `pypi`, you can do one of two things: +```starlark +py_library( + name = "my_lib", + deps = [ + "@pypi//numpy", + ], +) +``` + +Or use the `requirement` helper that needs to be loaded from the `hub` repo itself: +```starlark +load("@pypi//:requirements.bzl", "requirement") + +py_library( + deps = [ + requirement("numpy") + ], +) +``` + +Note, that the usage of the `requirement` helper is not advised and can be problematic. See the +[notes below](#requirement-helper). + +Note, that the hub repo contains the following targets for each package: +* `@pypi//numpy` which is a shorthand for `@pypi//numpy:numpy`. This is an {obj}`alias` to + `@pypi//numpy:pkg`. +* `@pypi//numpy:pkg` - the {obj}`py_library` target automatically generated by the repository + rules. +* `@pypi//numpy:data` - the {obj}`filegroup` that is for all of the extra files that are included + as data in the `pkg` target. +* `@pypi//numpy:dist_info` - the {obj}`filegroup` that is for all of the files in the `.distinfo` directory. +* `@pypi//numpy:whl` - the {obj}`filegroup` that is the `.whl` file itself which includes all of + the transitive dependencies via the {attr}`filegroup.data` attribute. + +## Entry points + +If you would like to access [entry points][whl_ep], see the `py_console_script_binary` rule documentation, +which can help you create a `py_binary` target for a particular console script exposed by a package. + +[whl_ep]: https://packaging.python.org/specifications/entry-points/ + +## 'Extras' dependencies + +Any 'extras' specified in the requirements lock file will be automatically added +as transitive dependencies of the package. In the example above, you'd just put +`requirement("useful_dep")` or `@pypi//useful_dep`. + +## Consuming Wheel Dists Directly + +If you need to depend on the wheel dists themselves, for instance, to pass them +to some other packaging tool, you can get a handle to them with the +`whl_requirement` macro. For example: + +```starlark +load("@pypi//:requirements.bzl", "whl_requirement") + +filegroup( + name = "whl_files", + data = [ + # This is equivalent to "@pypi//boto3:whl" + whl_requirement("boto3"), + ] +) +``` + +## Creating a filegroup of files within a whl + +The rule {obj}`whl_filegroup` exists as an easy way to extract the necessary files +from a whl file without the need to modify the `BUILD.bazel` contents of the +whl repositories generated via `pip_repository`. Use it similarly to the `filegroup` +above. See the API docs for more information. + +(requirement-helper)= +## A note about using the requirement helper + +Each extracted wheel repo contains a `py_library` target representing +the wheel's contents. There are two ways to access this library. The +first uses the `requirement()` function defined in the central +repo's `//:requirements.bzl` file. This function maps a pip package +name to a label: + +```starlark +load("@my_deps//:requirements.bzl", "requirement") + +py_library( + name = "mylib", + srcs = ["mylib.py"], + deps = [ + ":myotherlib", + requirement("some_pip_dep"), + requirement("another_pip_dep"), + ] +) +``` + +The reason `requirement()` exists is to insulate from +changes to the underlying repository and label strings. However, those +labels have become directly used, so aren't able to easily change regardless. + +On the other hand, using `requirement()` helper has several drawbacks: + +- It doesn't work with `buildifier` +- It doesn't work with `buildozer` +- It adds extra layer on top of normal mechanisms to refer to targets. +- It does not scale well as each type of target needs a new macro to be loaded and imported. + +If you don't want to use `requirement()`, you can use the library labels directly instead. For +`pip_parse`, the labels are of the following form: + +```starlark +@{name}//{package} +``` + +Here `name` is the `name` attribute that was passed to `pip_parse` and +`package` is the pip package name with characters that are illegal in +Bazel label names (e.g. `-`, `.`) replaced with `_`. If you need to +update `name` from "old" to "new", then you can run the following +`buildozer` command: + +```shell +buildozer 'substitute deps @old//([^/]+) @new//${1}' //...:* +``` diff --git a/docs/requirements.txt b/docs/requirements.txt index e4ec16fa5e..87c13aa8ba 100644 --- a/docs/requirements.txt +++ b/docs/requirements.txt @@ -1,6 +1,5 @@ # This file was autogenerated by uv via the following command: # bazel run //docs:requirements.update ---index-url https://pypi.org/simple absl-py==2.2.2 \ --hash=sha256:bf25b2c2eed013ca456918c453d687eab4e8309fba81ee2f4c1a6aa2494175eb \ diff --git a/python/private/pypi/BUILD.bazel b/python/private/pypi/BUILD.bazel index e9036c3013..d89dc6c228 100644 --- a/python/private/pypi/BUILD.bazel +++ b/python/private/pypi/BUILD.bazel @@ -398,6 +398,7 @@ bzl_library( ":pep508_requirement_bzl", ":pypi_repo_utils_bzl", ":whl_metadata_bzl", + ":whl_target_platforms_bzl", "//python/private:auth_bzl", "//python/private:bzlmod_enabled_bzl", "//python/private:envsubst_bzl", diff --git a/python/private/pypi/pkg_aliases.bzl b/python/private/pypi/pkg_aliases.bzl index 28d70ff715..d71c37cb4b 100644 --- a/python/private/pypi/pkg_aliases.bzl +++ b/python/private/pypi/pkg_aliases.bzl @@ -237,9 +237,10 @@ def multiplatform_whl_aliases( Exposed only for unit tests. Args: - aliases: {type}`str | dict[whl_config_setting | str, str]`: The aliases + aliases: {type}`str | dict[struct | str, str]`: The aliases to process. Any aliases that have the filename set will be - converted to a dict of config settings to repo names. + converted to a dict of config settings to repo names. The + struct is created by {func}`whl_config_setting`. glibc_versions: {type}`list[tuple[int, int]]` list of versions that can be used in this hub repo. muslc_versions: {type}`list[tuple[int, int]]` list of versions that can be diff --git a/python/private/pypi/simpleapi_download.bzl b/python/private/pypi/simpleapi_download.bzl index e8d7d0941a..164d4e8dbd 100644 --- a/python/private/pypi/simpleapi_download.bzl +++ b/python/private/pypi/simpleapi_download.bzl @@ -83,6 +83,7 @@ def simpleapi_download( found_on_index = {} warn_overrides = False + ctx.report_progress("Fetch package lists from PyPI index") for i, index_url in enumerate(index_urls): if i != 0: # Warn the user about a potential fix for the overrides diff --git a/python/private/pypi/whl_config_setting.bzl b/python/private/pypi/whl_config_setting.bzl index 6e10eb4d27..3b81e4694f 100644 --- a/python/private/pypi/whl_config_setting.bzl +++ b/python/private/pypi/whl_config_setting.bzl @@ -21,14 +21,14 @@ def whl_config_setting(*, version = None, config_setting = None, filename = None aliases in a hub repository. Args: - version: optional(str), the version of the python toolchain that this + version: {type}`str | None`the version of the python toolchain that this whl alias is for. If not set, then non-version aware aliases will be constructed. This is mainly used for better error messages when there is no match found during a select. - config_setting: optional(Label or str), the config setting that we should use. Defaults + config_setting: {type}`str | Label | None` the config setting that we should use. Defaults to "//_config:is_python_{version}". - filename: optional(str), the distribution filename to derive the config_setting. - target_platforms: optional(list[str]), the list of target_platforms for this + filename: {type}`str | None` the distribution filename to derive the config_setting. + target_platforms: {type}`list[str] | None` the list of target_platforms for this distribution. Returns: diff --git a/sphinxdocs/inventories/bazel_inventory.txt b/sphinxdocs/inventories/bazel_inventory.txt index bbd200ddb5..e14ea76067 100644 --- a/sphinxdocs/inventories/bazel_inventory.txt +++ b/sphinxdocs/inventories/bazel_inventory.txt @@ -15,6 +15,7 @@ RBE bzl:obj 1 remote/rbe - RunEnvironmentInfo bzl:type 1 rules/lib/providers/RunEnvironmentInfo - Target bzl:type 1 rules/lib/builtins/Target - ToolchainInfo bzl:type 1 rules/lib/providers/ToolchainInfo.html - +alias bzl:rule 1 reference/be/general#alias - attr.bool bzl:type 1 rules/lib/toplevel/attr#bool - attr.int bzl:type 1 rules/lib/toplevel/attr#int - attr.int_list bzl:type 1 rules/lib/toplevel/attr#int_list - @@ -40,6 +41,7 @@ config.string_list bzl:function 1 rules/lib/toplevel/config#string_list - config.target bzl:function 1 rules/lib/toplevel/config#target - config_common.FeatureFlagInfo bzl:type 1 rules/lib/toplevel/config_common#FeatureFlagInfo - config_common.toolchain_type bzl:function 1 rules/lib/toplevel/config_common#toolchain_type - +config_setting bzl:rule 1 reference/be/general#config_setting - ctx bzl:type 1 rules/lib/builtins/repository_ctx - ctx.actions bzl:obj 1 rules/lib/builtins/ctx#actions - ctx.aspect_ids bzl:obj 1 rules/lib/builtins/ctx#aspect_ids - @@ -79,6 +81,8 @@ depset bzl:type 1 rules/lib/depset - dict bzl:type 1 rules/lib/dict - exec_compatible_with bzl:attr 1 reference/be/common-definitions#common.exec_compatible_with - exec_group bzl:function 1 rules/lib/globals/bzl#exec_group - +filegroup bzl:rule 1 reference/be/general#filegroup - +filegroup.data bzl:attr 1 reference/be/general#filegroup.data - int bzl:type 1 rules/lib/int - label bzl:type 1 concepts/labels - list bzl:type 1 rules/lib/list - diff --git a/tests/pypi/simpleapi_download/simpleapi_download_tests.bzl b/tests/pypi/simpleapi_download/simpleapi_download_tests.bzl index ce214d6e34..a96815c12c 100644 --- a/tests/pypi/simpleapi_download/simpleapi_download_tests.bzl +++ b/tests/pypi/simpleapi_download/simpleapi_download_tests.bzl @@ -43,6 +43,7 @@ def _test_simple(env): contents = simpleapi_download( ctx = struct( os = struct(environ = {}), + report_progress = lambda _: None, ), attr = struct( index_url_overrides = {}, @@ -95,6 +96,7 @@ def _test_fail(env): simpleapi_download( ctx = struct( os = struct(environ = {}), + report_progress = lambda _: None, ), attr = struct( index_url_overrides = {}, @@ -136,6 +138,7 @@ def _test_download_url(env): ctx = struct( os = struct(environ = {}), download = download, + report_progress = lambda _: None, read = lambda i: "contents of " + i, path = lambda i: "path/for/" + i, ), @@ -171,6 +174,7 @@ def _test_download_url_parallel(env): ctx = struct( os = struct(environ = {}), download = download, + report_progress = lambda _: None, read = lambda i: "contents of " + i, path = lambda i: "path/for/" + i, ), @@ -206,6 +210,7 @@ def _test_download_envsubst_url(env): ctx = struct( os = struct(environ = {"INDEX_URL": "https://example.com/main/simple/"}), download = download, + report_progress = lambda _: None, read = lambda i: "contents of " + i, path = lambda i: "path/for/" + i, ), From fd29d273e41180c56d691a67004ade742f7c7b2f Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Wed, 28 May 2025 23:37:23 -0700 Subject: [PATCH 262/922] refactor: change site_packages_symlinks to venv_symlinks (#2939) This generalizes the ability to populate the venv directory by adding and additional field, `kind`, which tells which directory of the venv to populate. A symbolic constant is used to indicate which directory so that users don't have to re-derive the platform and version specific paths that make up the venv directory names. This follows the design described by https://github.com/bazel-contrib/rules_python/issues/2156#issuecomment-2855580026 This also changes it to a depset of structs to make it more forward compatible. A provider is used because they're slightly more memory efficient than regular structs. Work towards https://github.com/bazel-contrib/rules_python/issues/2156 --- CHANGELOG.md | 4 +- python/features.bzl | 8 +- python/private/attributes.bzl | 2 +- python/private/common.bzl | 6 +- python/private/flags.bzl | 6 +- python/private/py_executable.bzl | 102 ++++++++++++++----------- python/private/py_info.bzl | 127 ++++++++++++++++++++++--------- python/private/py_library.bzl | 40 +++++----- 8 files changed, 187 insertions(+), 108 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 9655b90487..4a6bdf0a96 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -70,6 +70,8 @@ END_UNRELEASED_TEMPLATE `_test` target is deprecated and will be removed in the next major release. ([#2794](https://github.com/bazel-contrib/rules_python/issues/2794) * (py_wheel) py_wheel always creates zip64-capable wheel zips +* (providers) (experimental) {obj}`PyInfo.venv_symlinks` replaces + `PyInfo.site_packages_symlinks` {#v0-0-0-fixed} ### Fixed @@ -203,7 +205,7 @@ END_UNRELEASED_TEMPLATE please check the {obj}`uv.configure` tag class. * Add support for riscv64 linux platform. * (toolchains) Add python 3.13.2 and 3.12.9 toolchains -* (providers) (experimental) {obj}`PyInfo.site_packages_symlinks` field added to +* (providers) (experimental) `PyInfo.site_packages_symlinks` field added to allow specifying links to create within the venv site packages (only applicable with {obj}`--bootstrap_impl=script`) ([#2156](https://github.com/bazelbuild/rules_python/issues/2156)). diff --git a/python/features.bzl b/python/features.bzl index 917bd3800c..b678a45241 100644 --- a/python/features.bzl +++ b/python/features.bzl @@ -31,11 +31,11 @@ def _features_typedef(): ::: :::: - ::::{field} py_info_site_packages_symlinks + ::::{field} py_info_venv_symlinks - True if the `PyInfo.site_packages_symlinks` field is available. + True if the `PyInfo.venv_symlinks` field is available. - :::{versionadded} 1.4.0 + :::{versionadded} VERSION_NEXT_FEATURE ::: :::: @@ -61,7 +61,7 @@ features = struct( TYPEDEF = _features_typedef, # keep sorted precompile = True, - py_info_site_packages_symlinks = True, + py_info_venv_symlinks = True, uses_builtin_rules = not config.enable_pystar, version = _VERSION_PRIVATE if "$Format" not in _VERSION_PRIVATE else "", ) diff --git a/python/private/attributes.bzl b/python/private/attributes.bzl index 98aba4eb23..ad8cba2e6c 100644 --- a/python/private/attributes.bzl +++ b/python/private/attributes.bzl @@ -260,7 +260,7 @@ The order of this list can matter because it affects the order that information from dependencies is merged in, which can be relevant depending on the ordering mode of depsets that are merged. -* {obj}`PyInfo.site_packages_symlinks` uses topological ordering. +* {obj}`PyInfo.venv_symlinks` uses topological ordering. See {obj}`PyInfo` for more information about the ordering of its depsets and how its fields are merged. diff --git a/python/private/common.bzl b/python/private/common.bzl index a58a9c00a4..e49dbad20c 100644 --- a/python/private/common.bzl +++ b/python/private/common.bzl @@ -378,7 +378,7 @@ def create_py_info( implicit_pyc_files, implicit_pyc_source_files, imports, - site_packages_symlinks = []): + venv_symlinks = []): """Create PyInfo provider. Args: @@ -396,7 +396,7 @@ def create_py_info( implicit_pyc_files: {type}`depset[File]` Implicitly generated pyc files that a binary can choose to include. imports: depset of strings; the import path values to propagate. - site_packages_symlinks: {type}`list[tuple[str, str]]` tuples of + venv_symlinks: {type}`list[tuple[str, str]]` tuples of `(runfiles_path, site_packages_path)` for symlinks to create in the consuming binary's venv site packages. @@ -406,7 +406,7 @@ def create_py_info( necessary for deprecated extra actions support). """ py_info = PyInfoBuilder.new() - py_info.site_packages_symlinks.add(site_packages_symlinks) + py_info.venv_symlinks.add(venv_symlinks) py_info.direct_original_sources.add(original_sources) py_info.direct_pyc_files.add(required_pyc_files) py_info.direct_pyi_files.add(ctx.files.pyi_srcs) diff --git a/python/private/flags.bzl b/python/private/flags.bzl index 40ce63b3b0..710402ba68 100644 --- a/python/private/flags.bzl +++ b/python/private/flags.bzl @@ -154,12 +154,12 @@ def _venvs_site_packages_is_enabled(ctx): flag_value = ctx.attr.experimental_venvs_site_packages[BuildSettingInfo].value return flag_value == VenvsSitePackages.YES -# Decides if libraries try to use a site-packages layout using site_packages_symlinks +# Decides if libraries try to use a site-packages layout using venv_symlinks # buildifier: disable=name-conventions VenvsSitePackages = FlagEnum( - # Use site_packages_symlinks + # Use venv_symlinks YES = "yes", - # Don't use site_packages_symlinks + # Don't use venv_symlinks NO = "no", is_enabled = _venvs_site_packages_is_enabled, ) diff --git a/python/private/py_executable.bzl b/python/private/py_executable.bzl index 24be8dd2ad..7c3e0cb757 100644 --- a/python/private/py_executable.bzl +++ b/python/private/py_executable.bzl @@ -54,7 +54,7 @@ load(":flags.bzl", "BootstrapImplFlag", "VenvsUseDeclareSymlinkFlag") load(":precompile.bzl", "maybe_precompile") load(":py_cc_link_params_info.bzl", "PyCcLinkParamsInfo") load(":py_executable_info.bzl", "PyExecutableInfo") -load(":py_info.bzl", "PyInfo") +load(":py_info.bzl", "PyInfo", "VenvSymlinkKind") load(":py_internal.bzl", "py_internal") load(":py_runtime_info.bzl", "DEFAULT_STUB_SHEBANG", "PyRuntimeInfo") load(":reexports.bzl", "BuiltinPyInfo", "BuiltinPyRuntimeInfo") @@ -543,6 +543,7 @@ def _create_venv(ctx, output_prefix, imports, runtime_details): VenvsUseDeclareSymlinkFlag.get_value(ctx) == VenvsUseDeclareSymlinkFlag.YES ) recreate_venv_at_runtime = False + bin_dir = "{}/bin".format(venv) if not venvs_use_declare_symlink_enabled or not runtime.supports_build_time_venv: recreate_venv_at_runtime = True @@ -556,7 +557,7 @@ def _create_venv(ctx, output_prefix, imports, runtime_details): # When the venv symlinks are disabled, the $venv/bin/python3 file isn't # needed or used at runtime. However, the zip code uses the interpreter # File object to figure out some paths. - interpreter = ctx.actions.declare_file("{}/bin/{}".format(venv, py_exe_basename)) + interpreter = ctx.actions.declare_file("{}/{}".format(bin_dir, py_exe_basename)) ctx.actions.write(interpreter, "actual:{}".format(interpreter_actual_path)) elif runtime.interpreter: @@ -568,7 +569,7 @@ def _create_venv(ctx, output_prefix, imports, runtime_details): # declare_symlink() is required to ensure that the resulting file # in runfiles is always a symlink. An RBE implementation, for example, # may choose to write what symlink() points to instead. - interpreter = ctx.actions.declare_symlink("{}/bin/{}".format(venv, py_exe_basename)) + interpreter = ctx.actions.declare_symlink("{}/{}".format(bin_dir, py_exe_basename)) interpreter_actual_path = runfiles_root_path(ctx, runtime.interpreter.short_path) rel_path = relative_path( @@ -581,7 +582,7 @@ def _create_venv(ctx, output_prefix, imports, runtime_details): ctx.actions.symlink(output = interpreter, target_path = rel_path) else: py_exe_basename = paths.basename(runtime.interpreter_path) - interpreter = ctx.actions.declare_symlink("{}/bin/{}".format(venv, py_exe_basename)) + interpreter = ctx.actions.declare_symlink("{}/{}".format(bin_dir, py_exe_basename)) ctx.actions.symlink(output = interpreter, target_path = runtime.interpreter_path) interpreter_actual_path = runtime.interpreter_path @@ -618,89 +619,104 @@ def _create_venv(ctx, output_prefix, imports, runtime_details): }, computed_substitutions = computed_subs, ) - site_packages_symlinks = _create_site_packages_symlinks(ctx, site_packages) + + venv_dir_map = { + VenvSymlinkKind.BIN: bin_dir, + VenvSymlinkKind.LIB: site_packages, + } + venv_symlinks = _create_venv_symlinks(ctx, venv_dir_map) return struct( interpreter = interpreter, recreate_venv_at_runtime = recreate_venv_at_runtime, # Runfiles root relative path or absolute path interpreter_actual_path = interpreter_actual_path, - files_without_interpreter = [pyvenv_cfg, pth, site_init] + site_packages_symlinks, + files_without_interpreter = [pyvenv_cfg, pth, site_init] + venv_symlinks, # string; venv-relative path to the site-packages directory. venv_site_packages = venv_site_packages, ) -def _create_site_packages_symlinks(ctx, site_packages): - """Creates symlinks within site-packages. +def _create_venv_symlinks(ctx, venv_dir_map): + """Creates symlinks within the venv. Args: ctx: current rule ctx - site_packages: runfiles-root-relative path to the site-packages directory + venv_dir_map: mapping of VenvSymlinkKind constants to the + venv path. Returns: {type}`list[File]` list of the File symlink objects created. """ - # maps site-package symlink to the runfiles path it should point to + # maps venv-relative path to the runfiles path it should point to entries = depset( # NOTE: Topological ordering is used so that dependencies closer to the # binary have precedence in creating their symlinks. This allows the # binary a modicum of control over the result. order = "topological", transitive = [ - dep[PyInfo].site_packages_symlinks + dep[PyInfo].venv_symlinks for dep in ctx.attr.deps if PyInfo in dep ], ).to_list() + link_map = _build_link_map(entries) + venv_files = [] + for kind, kind_map in link_map.items(): + base = venv_dir_map[kind] + for venv_path, link_to in kind_map.items(): + venv_link = ctx.actions.declare_symlink(paths.join(base, venv_path)) + venv_link_rf_path = runfiles_root_path(ctx, venv_link.short_path) + rel_path = relative_path( + # dirname is necessary because a relative symlink is relative to + # the directory the symlink resides within. + from_ = paths.dirname(venv_link_rf_path), + to = link_to, + ) + ctx.actions.symlink(output = venv_link, target_path = rel_path) + venv_files.append(venv_link) - sp_files = [] - for sp_dir_path, link_to in link_map.items(): - sp_link = ctx.actions.declare_symlink(paths.join(site_packages, sp_dir_path)) - sp_link_rf_path = runfiles_root_path(ctx, sp_link.short_path) - rel_path = relative_path( - # dirname is necessary because a relative symlink is relative to - # the directory the symlink resides within. - from_ = paths.dirname(sp_link_rf_path), - to = link_to, - ) - ctx.actions.symlink(output = sp_link, target_path = rel_path) - sp_files.append(sp_link) - return sp_files + return venv_files def _build_link_map(entries): + # dict[str kind, dict[str rel_path, str link_to_path]] link_map = {} - for link_to_runfiles_path, site_packages_path in entries: - if site_packages_path in link_map: + for entry in entries: + kind = entry.kind + kind_map = link_map.setdefault(kind, {}) + if entry.venv_path in kind_map: # We ignore duplicates by design. The dependency closer to the # binary gets precedence due to the topological ordering. continue else: - link_map[site_packages_path] = link_to_runfiles_path + kind_map[entry.venv_path] = entry.link_to_path # An empty link_to value means to not create the site package symlink. # Because of the topological ordering, this allows binaries to remove # entries by having an earlier dependency produce empty link_to values. - for sp_dir_path, link_to in link_map.items(): - if not link_to: - link_map.pop(sp_dir_path) + for kind, kind_map in link_map.items(): + for dir_path, link_to in kind_map.items(): + if not link_to: + kind_map.pop(dir_path) - # Remove entries that would be a child path of a created symlink. - # Earlier entries have precedence to match how exact matches are handled. + # dict[str kind, dict[str rel_path, str link_to_path]] keep_link_map = {} - for _ in range(len(link_map)): - if not link_map: - break - dirname, value = link_map.popitem() - keep_link_map[dirname] = value - - prefix = dirname + "/" # Add slash to prevent /X matching /XY - for maybe_suffix in link_map.keys(): - maybe_suffix += "/" # Add slash to prevent /X matching /XY - if maybe_suffix.startswith(prefix) or prefix.startswith(maybe_suffix): - link_map.pop(maybe_suffix) + # Remove entries that would be a child path of a created symlink. + # Earlier entries have precedence to match how exact matches are handled. + for kind, kind_map in link_map.items(): + keep_kind_map = keep_link_map.setdefault(kind, {}) + for _ in range(len(kind_map)): + if not kind_map: + break + dirname, value = kind_map.popitem() + keep_kind_map[dirname] = value + prefix = dirname + "/" # Add slash to prevent /X matching /XY + for maybe_suffix in kind_map.keys(): + maybe_suffix += "/" # Add slash to prevent /X matching /XY + if maybe_suffix.startswith(prefix) or prefix.startswith(maybe_suffix): + kind_map.pop(maybe_suffix) return keep_link_map def _map_each_identity(v): diff --git a/python/private/py_info.bzl b/python/private/py_info.bzl index d175eefb69..2a2f4554e3 100644 --- a/python/private/py_info.bzl +++ b/python/private/py_info.bzl @@ -18,6 +18,64 @@ load(":builders.bzl", "builders") load(":reexports.bzl", "BuiltinPyInfo") load(":util.bzl", "define_bazel_6_provider") +def _VenvSymlinkKind_typedef(): + """An enum of types of venv directories. + + :::{field} BIN + :type: object + + Indicates to create paths under the directory that has binaries + within the venv. + ::: + + :::{field} LIB + :type: object + + Indicates to create paths under the venv's site-packages directory. + ::: + + :::{field} INCLUDE + :type: object + + Indicates to create paths under the venv's include directory. + ::: + """ + +# buildifier: disable=name-conventions +VenvSymlinkKind = struct( + TYPEDEF = _VenvSymlinkKind_typedef, + BIN = "BIN", + LIB = "LIB", + INCLUDE = "INCLUDE", +) + +# A provider is used for memory efficiency. +# buildifier: disable=name-conventions +VenvSymlinkEntry = provider( + doc = """ +An entry in `PyInfo.venv_symlinks` +""", + fields = { + "kind": """ +:type: str + +One of the {obj}`VenvSymlinkKind` values. It represents which directory within +the venv to create the path under. +""", + "link_to_path": """ +:type: str | None + +A runfiles-root relative path that `venv_path` will symlink to. If `None`, +it means to not create a symlink. +""", + "venv_path": """ +:type: str + +A path relative to the `kind` directory within the venv. +""", + }, +) + def _check_arg_type(name, required_type, value): """Check that a value is of an expected type.""" value_type = type(value) @@ -43,7 +101,7 @@ def _PyInfo_init( transitive_original_sources = depset(), direct_pyi_files = depset(), transitive_pyi_files = depset(), - site_packages_symlinks = depset()): + venv_symlinks = depset()): _check_arg_type("transitive_sources", "depset", transitive_sources) # Verify it's postorder compatible, but retain is original ordering. @@ -71,7 +129,6 @@ def _PyInfo_init( "has_py2_only_sources": has_py2_only_sources, "has_py3_only_sources": has_py2_only_sources, "imports": imports, - "site_packages_symlinks": site_packages_symlinks, "transitive_implicit_pyc_files": transitive_implicit_pyc_files, "transitive_implicit_pyc_source_files": transitive_implicit_pyc_source_files, "transitive_original_sources": transitive_original_sources, @@ -79,6 +136,7 @@ def _PyInfo_init( "transitive_pyi_files": transitive_pyi_files, "transitive_sources": transitive_sources, "uses_shared_libraries": uses_shared_libraries, + "venv_symlinks": venv_symlinks, } PyInfo, _unused_raw_py_info_ctor = define_bazel_6_provider( @@ -146,34 +204,6 @@ A depset of import path strings to be added to the `PYTHONPATH` of executable Python targets. These are accumulated from the transitive `deps`. The order of the depset is not guaranteed and may be changed in the future. It is recommended to use `default` order (the default). -""", - "site_packages_symlinks": """ -:type: depset[tuple[str | None, str]] - -A depset with `topological` ordering. - -Tuples of `(runfiles_path, site_packages_path)`. Where -* `runfiles_path` is a runfiles-root relative path. It is the path that - has the code to make importable. If `None` or empty string, then it means - to not create a site packages directory with the `site_packages_path` - name. -* `site_packages_path` is a path relative to the site-packages directory of - the venv for whatever creates the venv (typically py_binary). It makes - the code in `runfiles_path` available for import. Note that this - is created as a "raw" symlink (via `declare_symlink`). - -:::{include} /_includes/experimental_api.md -::: - -:::{tip} -The topological ordering means dependencies earlier and closer to the consumer -have precedence. This allows e.g. a binary to add dependencies that override -values from further way dependencies, such as forcing symlinks to point to -specific paths or preventing symlinks from being created. -::: - -:::{versionadded} 1.4.0 -::: """, "transitive_implicit_pyc_files": """ :type: depset[File] @@ -262,6 +292,35 @@ Whether any of this target's transitive `deps` has a shared library file (such as a `.so` file). This field is currently unused in Bazel and may go away in the future. +""", + "venv_symlinks": """ +:type: depset[VenvSymlinkEntry] + +A depset with `topological` ordering. + + +Tuples of `(runfiles_path, site_packages_path)`. Where +* `runfiles_path` is a runfiles-root relative path. It is the path that + has the code to make importable. If `None` or empty string, then it means + to not create a site packages directory with the `site_packages_path` + name. +* `site_packages_path` is a path relative to the site-packages directory of + the venv for whatever creates the venv (typically py_binary). It makes + the code in `runfiles_path` available for import. Note that this + is created as a "raw" symlink (via `declare_symlink`). + +:::{include} /_includes/experimental_api.md +::: + +:::{tip} +The topological ordering means dependencies earlier and closer to the consumer +have precedence. This allows e.g. a binary to add dependencies that override +values from further way dependencies, such as forcing symlinks to point to +specific paths or preventing symlinks from being created. +::: + +:::{versionadded} VERSION_NEXT_FEATURE +::: """, }, ) @@ -314,7 +373,7 @@ def _PyInfoBuilder_typedef(): :type: DepsetBuilder[File] ::: - :::{field} site_packages_symlinks + :::{field} venv_symlinks :type: DepsetBuilder[tuple[str | None, str]] NOTE: This depset has `topological` order @@ -358,7 +417,7 @@ def _PyInfoBuilder_new(): transitive_pyc_files = builders.DepsetBuilder(), transitive_pyi_files = builders.DepsetBuilder(), transitive_sources = builders.DepsetBuilder(), - site_packages_symlinks = builders.DepsetBuilder(order = "topological"), + venv_symlinks = builders.DepsetBuilder(order = "topological"), ) return self @@ -525,7 +584,7 @@ def _PyInfoBuilder_merge_all(self, transitive, *, direct = []): self.transitive_original_sources.add(info.transitive_original_sources) self.transitive_pyc_files.add(info.transitive_pyc_files) self.transitive_pyi_files.add(info.transitive_pyi_files) - self.site_packages_symlinks.add(info.site_packages_symlinks) + self.venv_symlinks.add(info.venv_symlinks) return self @@ -583,7 +642,7 @@ def _PyInfoBuilder_build(self): transitive_original_sources = self.transitive_original_sources.build(), transitive_pyc_files = self.transitive_pyc_files.build(), transitive_pyi_files = self.transitive_pyi_files.build(), - site_packages_symlinks = self.site_packages_symlinks.build(), + venv_symlinks = self.venv_symlinks.build(), ) else: kwargs = {} diff --git a/python/private/py_library.bzl b/python/private/py_library.bzl index fd9dad9f20..fabc880a8d 100644 --- a/python/private/py_library.bzl +++ b/python/private/py_library.bzl @@ -43,7 +43,7 @@ load( load(":flags.bzl", "AddSrcsToRunfilesFlag", "PrecompileFlag", "VenvsSitePackages") load(":precompile.bzl", "maybe_precompile") load(":py_cc_link_params_info.bzl", "PyCcLinkParamsInfo") -load(":py_info.bzl", "PyInfo") +load(":py_info.bzl", "PyInfo", "VenvSymlinkEntry", "VenvSymlinkKind") load(":py_internal.bzl", "py_internal") load(":reexports.bzl", "BuiltinPyInfo") load(":rule_builders.bzl", "ruleb") @@ -90,9 +90,9 @@ won't be understood as namespace packages; they'll be seen as regular packages. likely lead to conflicts with other targets that contribute to the namespace. :::{tip} -This attributes populates {obj}`PyInfo.site_packages_symlinks`, which is +This attributes populates {obj}`PyInfo.venv_symlinks`, which is a topologically ordered depset. This means dependencies closer and earlier -to a consumer have precedence. See {obj}`PyInfo.site_packages_symlinks` for +to a consumer have precedence. See {obj}`PyInfo.venv_symlinks` for more information. ::: @@ -155,9 +155,9 @@ def py_library_impl(ctx, *, semantics): runfiles = runfiles.build(ctx) imports = [] - site_packages_symlinks = [] + venv_symlinks = [] - imports, site_packages_symlinks = _get_imports_and_site_packages_symlinks(ctx, semantics) + imports, venv_symlinks = _get_imports_and_venv_symlinks(ctx, semantics) cc_info = semantics.get_cc_info_for_library(ctx) py_info, deps_transitive_sources, builtins_py_info = create_py_info( @@ -168,7 +168,7 @@ def py_library_impl(ctx, *, semantics): implicit_pyc_files = implicit_pyc_files, implicit_pyc_source_files = implicit_pyc_source_files, imports = imports, - site_packages_symlinks = site_packages_symlinks, + venv_symlinks = venv_symlinks, ) # TODO(b/253059598): Remove support for extra actions; https://github.com/bazelbuild/bazel/issues/16455 @@ -206,16 +206,16 @@ Source files are no longer added to the runfiles directly. ::: """ -def _get_imports_and_site_packages_symlinks(ctx, semantics): +def _get_imports_and_venv_symlinks(ctx, semantics): imports = depset() - site_packages_symlinks = depset() + venv_symlinks = depset() if VenvsSitePackages.is_enabled(ctx): - site_packages_symlinks = _get_site_packages_symlinks(ctx) + venv_symlinks = _get_venv_symlinks(ctx) else: imports = collect_imports(ctx, semantics) - return imports, site_packages_symlinks + return imports, venv_symlinks -def _get_site_packages_symlinks(ctx): +def _get_venv_symlinks(ctx): imports = ctx.attr.imports if len(imports) == 0: fail("When venvs_site_packages is enabled, exactly one `imports` " + @@ -253,7 +253,7 @@ def _get_site_packages_symlinks(ctx): repo_runfiles_dirname = None dirs_with_init = {} # dirname -> runfile path - site_packages_symlinks = [] + venv_symlinks = [] for src in ctx.files.srcs: if src.extension not in PYTHON_FILE_EXTENSIONS: continue @@ -271,9 +271,10 @@ def _get_site_packages_symlinks(ctx): # This would be files that do not have directories and we just need to add # direct symlinks to them as is: - site_packages_symlinks.append(( - paths.join(repo_runfiles_dirname, site_packages_root, filename), - filename, + venv_symlinks.append(VenvSymlinkEntry( + kind = VenvSymlinkKind.LIB, + link_to_path = paths.join(repo_runfiles_dirname, site_packages_root, filename), + venv_path = filename, )) # Sort so that we encounter `foo` before `foo/bar`. This ensures we @@ -291,11 +292,12 @@ def _get_site_packages_symlinks(ctx): first_level_explicit_packages.append(d) for dirname in first_level_explicit_packages: - site_packages_symlinks.append(( - paths.join(repo_runfiles_dirname, site_packages_root, dirname), - dirname, + venv_symlinks.append(VenvSymlinkEntry( + kind = VenvSymlinkKind.LIB, + link_to_path = paths.join(repo_runfiles_dirname, site_packages_root, dirname), + venv_path = dirname, )) - return site_packages_symlinks + return venv_symlinks def _repo_relative_short_path(short_path): # Convert `../+pypi+foo/some/file.py` to `some/file.py` From bbf3ab8956007f48fc012fb9316debffde8b0495 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Thu, 29 May 2025 06:21:27 -0700 Subject: [PATCH 263/922] docs: fix sphinxdocs mis-redirect (#2940) The redirect was going to a non-existent URL when viewed on the deployed docs. This was happening because the absolute paths `/api/whatever` don't exist in the deployed site -- it's actually `/en/latest/api/whatever`. This went unnoticed because it works locally (where there is no /en/latest prefix). To fix, use a relative url (relative urls are relative to the path that is redirected from) --- docs/conf.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/conf.py b/docs/conf.py index 1d9f526b93..8537d9996c 100644 --- a/docs/conf.py +++ b/docs/conf.py @@ -87,7 +87,7 @@ "api/sphinxdocs/sphinx": "/api/sphinxdocs/sphinxdocs/sphinx.html", "api/sphinxdocs/sphinx_stardoc": "/api/sphinxdocs/sphinxdocs/sphinx_stardoc.html", "api/sphinxdocs/readthedocs": "/api/sphinxdocs/sphinxdocs/readthedocs.html", - "api/sphinxdocs/index": "/api/sphinxdocs/sphinxdocs/index.html", + "api/sphinxdocs/index": "sphinxdocs/index.html", "api/sphinxdocs/private/sphinx_docs_library": "/api/sphinxdocs/sphinxdocs/private/sphinx_docs_library.html", "api/sphinxdocs/sphinx_docs_library": "/api/sphinxdocs/sphinxdocs/sphinx_docs_library.html", "api/sphinxdocs/inventories/index": "/api/sphinxdocs/sphinxdocs/inventories/index.html", From d60cee2623bf6cedb4dbd9899eb99ac84432fb37 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Thu, 29 May 2025 08:43:56 -0700 Subject: [PATCH 264/922] feat: allow custom platform when overriding (#2880) This basically allows using any python-build-standalone archive and using it if custom flags are set. This is done through the `single_version_platform_override()` API, because such archives are inherently version and platform specific. Key changes: * The `platform` arg can be any value (mostly; it ends up in repo names) * Added `target_compatible_with` and `target_settings` args, which become the settings used on the generated toolchain() definition. The platform settings are version specific, i.e. the key `(python_version, platform)` is what maps to the TCW/TS values. If an existing platform is used, it'll override the defaults that normally come from the PLATFORMS global for the particular version. If a new platform is used, it creates a new platform entry with those settings. Along the way: * Added various docs about internal variables so they're easier to grok at a glance. Work towards https://github.com/bazel-contrib/rules_python/issues/2081 --- CHANGELOG.md | 4 + MODULE.bazel | 16 + docs/toolchains.md | 67 ++++ internal_dev_setup.bzl | 2 +- python/BUILD.bazel | 1 + python/private/BUILD.bazel | 6 + python/private/platform_info.bzl | 34 ++ python/private/py_repositories.bzl | 2 +- python/private/python.bzl | 311 +++++++++++++++--- python/private/python_repository.bzl | 3 +- python/private/pythons_hub.bzl | 30 +- python/private/repo_utils.bzl | 20 +- python/private/toolchains_repo.bzl | 131 ++++++-- python/versions.bzl | 56 +--- tests/bootstrap_impls/bin.py | 1 + tests/python/python_tests.bzl | 23 ++ tests/support/BUILD.bazel | 13 + tests/support/sh_py_run_test.bzl | 2 + tests/toolchains/BUILD.bazel | 13 + .../custom_platform_toolchain_test.py | 15 + 20 files changed, 609 insertions(+), 141 deletions(-) create mode 100644 python/private/platform_info.bzl create mode 100644 tests/toolchains/custom_platform_toolchain_test.py diff --git a/CHANGELOG.md b/CHANGELOG.md index 4a6bdf0a96..a113c7411f 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -107,6 +107,10 @@ END_UNRELEASED_TEMPLATE Set the `RULES_PYTHON_ENABLE_PIPSTAR=1` environment variable to enable it. * (utils) Add a way to run a REPL for any `rules_python` target that returns a `PyInfo` provider. +* (toolchains) Arbitrary python-build-standalone runtimes can be registered + and activated with custom flags. See the [Registering custom runtimes] + docs and {obj}`single_version_platform_override()` API docs for more + information. {#v0-0-0-removed} ### Removed diff --git a/MODULE.bazel b/MODULE.bazel index d3a95350e5..144e130c1b 100644 --- a/MODULE.bazel +++ b/MODULE.bazel @@ -125,6 +125,22 @@ dev_python.override( register_all_versions = True, ) +# For testing an arbitrary runtime triggered by a custom flag. +# See //tests/toolchains:custom_platform_toolchain_test +dev_python.single_version_platform_override( + platform = "linux-x86-install-only-stripped", + python_version = "3.13.1", + sha256 = "56817aa976e4886bec1677699c136cb01c1cdfe0495104c0d8ef546541864bbb", + target_compatible_with = [ + "@platforms//os:linux", + "@platforms//cpu:x86_64", + ], + target_settings = [ + "@@//tests/support:is_custom_runtime_linux-x86-install-only-stripped", + ], + urls = ["https://github.com/astral-sh/python-build-standalone/releases/download/20250115/cpython-3.13.1+20250115-x86_64-unknown-linux-gnu-install_only_stripped.tar.gz"], +) + dev_pip = use_extension( "//python/extensions:pip.bzl", "pip", diff --git a/docs/toolchains.md b/docs/toolchains.md index ada887c945..57d43d27f1 100644 --- a/docs/toolchains.md +++ b/docs/toolchains.md @@ -243,6 +243,73 @@ existing attributes: * Adding additional Python versions via {bzl:obj}`python.single_version_override` or {bzl:obj}`python.single_version_platform_override`. +### Registering custom runtimes + +Because the python-build-standalone project has _thousands_ of prebuilt runtimes +available, rules_python only includes popular runtimes in its built in +configurations. If you want to use a runtime that isn't already known to +rules_python then {obj}`single_version_platform_override()` can be used to do +so. In short, it allows specifying an arbitrary URL and using custom flags +to control when a runtime is used. + +In the example below, we register a particular python-build-standalone runtime +that is activated for Linux x86 builds when the custom flag +`--//:runtime=my-custom-runtime` is set. + +``` +# File: MODULE.bazel +bazel_dep(name = "bazel_skylib", version = "1.7.1.") +bazel_dep(name = "rules_python", version = "1.5.0") +python = use_extension("@rules_python//python/extensions:python.bzl", "python") +python.single_version_platform_override( + platform = "my-platform", + python_version = "3.13.3", + sha256 = "01d08b9bc8a96698b9d64c2fc26da4ecc4fa9e708ce0a34fb88f11ab7e552cbd", + os_name = "linux", + arch = "x86_64", + target_settings = [ + "@@//:runtime=my-custom-runtime", + ], + urls = ["https://github.com/astral-sh/python-build-standalone/releases/download/20250409/cpython-3.13.3+20250409-x86_64-unknown-linux-gnu-install_only_stripped.tar.gz"], +) +# File: //:BUILD.bazel +load("@bazel_skylib//rules:common_settings.bzl", "string_flag") +string_flag( + name = "custom_runtime", + build_setting_default = "", +) +config_setting( + name = "is_custom_runtime_linux-x86-install-only-stripped", + flag_values = { + ":custom_runtime": "linux-x86-install-only-stripped", + }, +) +``` + +Notes: +- While any URL and archive can be used, it's assumed their content looks how + a python-build-standalone archive looks. +- A "version aware" toolchain is registered, which means the Python version flag + must also match (e.g. `--@rules_python//python/config_settings:python_version=3.13.3` + must be set -- see `minor_mapping` and `is_default` for controls and docs + about version matching and selection). +- The `target_compatible_with` attribute can be used to entirely specify the + arg of the same name the toolchain uses. +- The labels in `target_settings` must be absolute; `@@` refers to the main repo. +- The `target_settings` are `config_setting` targets, which means you can + customize how matching occurs. + +:::{seealso} +See {obj}`//python/config_settings` for flags rules_python already defines +that can be used with `target_settings`. Some particular ones of note are: +{flag}`--py_linux_libc` and {flag}`--py_freethreaded`, among others. +::: + +:::{versionadded} VERSION_NEXT_FEATURE +Added support for custom platform names, `target_compatible_with`, and +`target_settings` with `single_version_platform_override`. +::: + ### Using defined toolchains from WORKSPACE It is possible to use toolchains defined in `MODULE.bazel` in `WORKSPACE`. For example diff --git a/internal_dev_setup.bzl b/internal_dev_setup.bzl index 62a11ab1d4..c37c59a5da 100644 --- a/internal_dev_setup.bzl +++ b/internal_dev_setup.bzl @@ -42,7 +42,7 @@ def rules_python_internal_setup(): toolchain_platform_keys = {}, toolchain_python_versions = {}, toolchain_set_python_version_constraints = {}, - base_toolchain_repo_names = [], + host_compatible_repo_names = [], ) runtime_env_repo(name = "rules_python_runtime_env_tc_info") diff --git a/python/BUILD.bazel b/python/BUILD.bazel index 867c43478a..58cff5b99d 100644 --- a/python/BUILD.bazel +++ b/python/BUILD.bazel @@ -247,6 +247,7 @@ bzl_library( name = "versions_bzl", srcs = ["versions.bzl"], visibility = ["//:__subpackages__"], + deps = ["//python/private:platform_info_bzl"], ) # NOTE: Remember to add bzl_library targets to //tests:bzl_libraries diff --git a/python/private/BUILD.bazel b/python/private/BUILD.bazel index ce22421300..b319919305 100644 --- a/python/private/BUILD.bazel +++ b/python/private/BUILD.bazel @@ -241,11 +241,17 @@ bzl_library( ], ) +bzl_library( + name = "platform_info_bzl", + srcs = ["platform_info.bzl"], +) + bzl_library( name = "python_bzl", srcs = ["python.bzl"], deps = [ ":full_version_bzl", + ":platform_info_bzl", ":python_register_toolchains_bzl", ":pythons_hub_bzl", ":repo_utils_bzl", diff --git a/python/private/platform_info.bzl b/python/private/platform_info.bzl new file mode 100644 index 0000000000..3f7dc00165 --- /dev/null +++ b/python/private/platform_info.bzl @@ -0,0 +1,34 @@ +"""Helper to define a struct used to define platform metadata.""" + +def platform_info( + *, + compatible_with = [], + flag_values = {}, + target_settings = [], + os_name, + arch): + """Creates a struct of platform metadata. + + This is just a helper to ensure structs are created the same and + the meaning/values are documented. + + Args: + compatible_with: list[str], where the values are string labels. These + are the target_compatible_with values to use with the toolchain + flag_values: dict[str|Label, Any] of config_setting.flag_values + compatible values. DEPRECATED -- use target_settings instead + target_settings: list[str], where the values are string labels. These + are the target_settings values to use with the toolchain. + os_name: str, the os name; must match the name used in `@platfroms//os` + arch: str, the cpu name; must match the name used in `@platforms//cpu` + + Returns: + A struct with attributes and values matching the args. + """ + return struct( + compatible_with = compatible_with, + flag_values = flag_values, + target_settings = target_settings, + os_name = os_name, + arch = arch, + ) diff --git a/python/private/py_repositories.bzl b/python/private/py_repositories.bzl index b5bd93b7c1..10bc06630b 100644 --- a/python/private/py_repositories.bzl +++ b/python/private/py_repositories.bzl @@ -47,7 +47,7 @@ def py_repositories(): toolchain_platform_keys = {}, toolchain_python_versions = {}, toolchain_set_python_version_constraints = {}, - base_toolchain_repo_names = [], + host_compatible_repo_names = [], ) http_archive( name = "bazel_skylib", diff --git a/python/private/python.bzl b/python/private/python.bzl index a7e257601f..8e23668879 100644 --- a/python/private/python.bzl +++ b/python/private/python.bzl @@ -18,29 +18,44 @@ load("@bazel_features//:features.bzl", "bazel_features") load("//python:versions.bzl", "DEFAULT_RELEASE_BASE_URL", "PLATFORMS", "TOOL_VERSIONS") load(":auth.bzl", "AUTH_ATTRS") load(":full_version.bzl", "full_version") +load(":platform_info.bzl", "platform_info") load(":python_register_toolchains.bzl", "python_register_toolchains") load(":pythons_hub.bzl", "hub_repo") load(":repo_utils.bzl", "repo_utils") -load(":toolchains_repo.bzl", "host_compatible_python_repo", "multi_toolchain_aliases", "sorted_host_platforms") +load( + ":toolchains_repo.bzl", + "host_compatible_python_repo", + "multi_toolchain_aliases", + "sorted_host_platform_names", + "sorted_host_platforms", +) load(":util.bzl", "IS_BAZEL_6_4_OR_HIGHER") load(":version.bzl", "version") -def parse_modules(*, module_ctx, _fail = fail): +def parse_modules(*, module_ctx, logger, _fail = fail): """Parse the modules and return a struct for registrations. Args: module_ctx: {type}`module_ctx` module context. + logger: {type}`repo_utils.logger` A logger to use. _fail: {type}`function` the failure function, mainly for testing. Returns: A struct with the following attributes: - * `toolchains`: The list of toolchains to register. The last - element is special and is treated as the default toolchain. + * `toolchains`: {type}`list[ToolchainConfig]` The list of toolchains to + register. The last element is special and is treated as the default + toolchain. * `config`: Various toolchain config, see `_get_toolchain_config`. * `debug_info`: {type}`None | dict` extra information to be passed to the debug repo. * `platforms`: {type}`dict[str, platform_info]` of the base set of platforms toolchains should be created for, if possible. + + ToolchainConfig struct: + * python_version: str, full python version string + * name: str, the base toolchain name, e.g., "python_3_10", no + platform suffix. + * register_coverage_tool: bool """ if module_ctx.os.environ.get("RULES_PYTHON_BZLMOD_DEBUG", "0") == "1": debug_info = { @@ -64,8 +79,6 @@ def parse_modules(*, module_ctx, _fail = fail): ignore_root_user_error = None - logger = repo_utils.logger(module_ctx, "python") - # if the root module does not register any toolchain then the # ignore_root_user_error takes its default value: True if not module_ctx.modules[0].tags.toolchain: @@ -265,19 +278,37 @@ def parse_modules(*, module_ctx, _fail = fail): ) def _python_impl(module_ctx): - py = parse_modules(module_ctx = module_ctx) + logger = repo_utils.logger(module_ctx, "python") + py = parse_modules(module_ctx = module_ctx, logger = logger) + + # Host compatible runtime repos + # dict[str version, struct] where struct has: + # * full_python_version: str + # * platform: platform_info struct + # * platform_name: str platform name + # * impl_repo_name: str repo name of the runtime's python_repository() repo + all_host_compatible_impls = {} + + # Host compatible repos that still need to be created because, when + # creating the actual runtime repo, there wasn't a host-compatible + # variant defined for it. + # dict[str reponame, struct] where struct has: + # * compatible_version: str, e.g. 3.10 or 3.10.1. The version the host + # repo should be compatible with + # * full_python_version: str, e.g. 3.10.1, the full python version of + # the toolchain that still needs a host repo created. + needed_host_repos = {} # list of structs; see inline struct call within the loop below. toolchain_impls = [] - # list[str] of the base names of toolchain repos - base_toolchain_repo_names = [] + # list[str] of the repo names for host compatible repos + all_host_compatible_repo_names = [] # Create the underlying python_repository repos that contain the # python runtimes and their toolchain implementation definitions. for i, toolchain_info in enumerate(py.toolchains): is_last = (i + 1) == len(py.toolchains) - base_toolchain_repo_names.append(toolchain_info.name) # Ensure that we pass the full version here. full_python_version = full_version( @@ -298,6 +329,8 @@ def _python_impl(module_ctx): _internal_bzlmod_toolchain_call = True, **kwargs ) + if not register_result.impl_repos: + continue host_platforms = {} for repo_name, (platform_name, platform_info) in register_result.impl_repos.items(): @@ -318,27 +351,81 @@ def _python_impl(module_ctx): set_python_version_constraint = is_last, )) if _is_compatible_with_host(module_ctx, platform_info): - host_platforms[platform_name] = platform_info + host_compat_entry = struct( + full_python_version = full_python_version, + platform = platform_info, + platform_name = platform_name, + impl_repo_name = repo_name, + ) + host_platforms[platform_name] = host_compat_entry + all_host_compatible_impls.setdefault(full_python_version, []).append( + host_compat_entry, + ) + parsed_version = version.parse(full_python_version) + all_host_compatible_impls.setdefault( + "{}.{}".format(*parsed_version.release[0:2]), + [], + ).append(host_compat_entry) + + host_repo_name = toolchain_info.name + "_host" + if host_platforms: + all_host_compatible_repo_names.append(host_repo_name) + host_platforms = sorted_host_platforms(host_platforms) + entries = host_platforms.values() + host_compatible_python_repo( + name = host_repo_name, + base_name = host_repo_name, + # NOTE: Order matters. The first found to be compatible is + # (usually) used. + platforms = host_platforms.keys(), + os_names = {str(i): e.platform.os_name for i, e in enumerate(entries)}, + arch_names = {str(i): e.platform.arch for i, e in enumerate(entries)}, + python_versions = {str(i): e.full_python_version for i, e in enumerate(entries)}, + impl_repo_names = {str(i): e.impl_repo_name for i, e in enumerate(entries)}, + ) + else: + needed_host_repos[host_repo_name] = struct( + compatible_version = toolchain_info.python_version, + full_python_version = full_python_version, + ) + + if needed_host_repos: + for key, entries in all_host_compatible_impls.items(): + all_host_compatible_impls[key] = sorted( + entries, + reverse = True, + key = lambda e: version.key(version.parse(e.full_python_version)), + ) - host_platforms = sorted_host_platforms(host_platforms) + for host_repo_name, info in needed_host_repos.items(): + choices = [] + if info.compatible_version not in all_host_compatible_impls: + logger.warn("No host compatible runtime found compatible with version {}".format(info.compatible_version)) + continue + + choices = all_host_compatible_impls[info.compatible_version] + platform_keys = [ + # We have to prepend the offset because the same platform + # name might occur across different versions + "{}_{}".format(i, entry.platform_name) + for i, entry in enumerate(choices) + ] + platform_keys = sorted_host_platform_names(platform_keys) + + all_host_compatible_repo_names.append(host_repo_name) host_compatible_python_repo( - name = toolchain_info.name + "_host", - # NOTE: Order matters. The first found to be compatible is (usually) used. - platforms = host_platforms.keys(), - os_names = { - str(i): platform_info.os_name - for i, platform_info in enumerate(host_platforms.values()) - }, - arch_names = { - str(i): platform_info.arch - for i, platform_info in enumerate(host_platforms.values()) + name = host_repo_name, + base_name = host_repo_name, + platforms = platform_keys, + impl_repo_names = { + str(i): entry.impl_repo_name + for i, entry in enumerate(choices) }, - python_version = full_python_version, + os_names = {str(i): entry.platform.os_name for i, entry in enumerate(choices)}, + arch_names = {str(i): entry.platform.arch for i, entry in enumerate(choices)}, + python_versions = {str(i): entry.full_python_version for i, entry in enumerate(choices)}, ) - # List of the base names ("python_3_10") for the toolchain repos - base_toolchain_repo_names = [] - # list[str] The infix to use for the resulting toolchain() `name` arg. toolchain_names = [] @@ -399,7 +486,7 @@ def _python_impl(module_ctx): toolchain_platform_keys = toolchain_platform_keys, toolchain_python_versions = toolchain_python_versions, toolchain_set_python_version_constraints = toolchain_set_python_version_constraints, - base_toolchain_repo_names = [t.name for t in py.toolchains], + host_compatible_repo_names = sorted(all_host_compatible_repo_names), default_python_version = py.default_python_version, minor_mapping = py.config.minor_mapping, python_versions = list(py.config.default["tool_versions"].keys()), @@ -583,9 +670,56 @@ def _process_single_version_platform_overrides(*, tag, _fail = fail, default): available_versions[tag.python_version].setdefault("sha256", {})[tag.platform] = tag.sha256 if tag.strip_prefix: available_versions[tag.python_version].setdefault("strip_prefix", {})[tag.platform] = tag.strip_prefix + if tag.urls: available_versions[tag.python_version].setdefault("url", {})[tag.platform] = tag.urls + # If platform is customized, or doesn't exist, (re)define one. + if ((tag.target_compatible_with or tag.target_settings or tag.os_name or tag.arch) or + tag.platform not in default["platforms"]): + os_name = tag.os_name + arch = tag.arch + + if not tag.target_compatible_with: + target_compatible_with = [] + if os_name: + target_compatible_with.append("@platforms//os:{}".format( + repo_utils.get_platforms_os_name(os_name), + )) + if arch: + target_compatible_with.append("@platforms//cpu:{}".format( + repo_utils.get_platforms_cpu_name(arch), + )) + else: + target_compatible_with = tag.target_compatible_with + + # For lack of a better option, give a bogus value. It only affects + # if the runtime is considered host-compatible. + if not os_name: + os_name = "UNKNOWN_CUSTOM_OS" + if not arch: + arch = "UNKNOWN_CUSTOM_ARCH" + + # Move the override earlier in the ordering -- the platform key ordering + # becomes the toolchain ordering within the version. This allows the + # override to have a superset of constraints from a regular runtimes + # (e.g. same platform, but with a custom flag required). + override_first = { + tag.platform: platform_info( + compatible_with = target_compatible_with, + target_settings = tag.target_settings, + os_name = os_name, + arch = arch, + ), + } + for key, value in default["platforms"].items(): + # Don't replace our override with the old value + if key in override_first: + continue + override_first[key] = value + + default["platforms"] = override_first + def _process_global_overrides(*, tag, default, _fail = fail): if tag.available_python_versions: available_versions = default["tool_versions"] @@ -664,22 +798,29 @@ def _get_toolchain_config(*, modules, _fail = fail): """ # Items that can be overridden - available_versions = { - version: { - # Use a dicts straight away so that we could do URL overrides for a - # single version. - "sha256": dict(item["sha256"]), - "strip_prefix": { - platform: item["strip_prefix"] - for platform in item["sha256"] - } if type(item["strip_prefix"]) == type("") else item["strip_prefix"], - "url": { - platform: [item["url"]] - for platform in item["sha256"] - } if type(item["url"]) == type("") else item["url"], - } - for version, item in TOOL_VERSIONS.items() - } + available_versions = {} + for py_version, item in TOOL_VERSIONS.items(): + available_versions[py_version] = {} + available_versions[py_version]["sha256"] = dict(item["sha256"]) + platforms = item["sha256"].keys() + + strip_prefix = item["strip_prefix"] + if type(strip_prefix) == type(""): + available_versions[py_version]["strip_prefix"] = { + platform: strip_prefix + for platform in platforms + } + else: + available_versions[py_version]["strip_prefix"] = dict(strip_prefix) + url = item["url"] + if type(url) == type(""): + available_versions[py_version]["url"] = { + platform: url + for platform in platforms + } + else: + available_versions[py_version]["url"] = dict(url) + default = { "base_url": DEFAULT_RELEASE_BASE_URL, "platforms": dict(PLATFORMS), # Copy so it's mutable. @@ -1084,10 +1225,48 @@ configuration, please use {obj}`single_version_override`. ::: """, attrs = { + "arch": attr.string( + doc = """ +The arch (cpu) the runtime is compatible with. + +If not set, then the runtime cannot be used as a `python_X_Y_host` runtime. + +If set, the `os_name`, `target_compatible_with` and `target_settings` attributes +should also be set. + +The values should be one of the values in `@platforms//cpu` + +:::{seealso} +Docs for [Registering custom runtimes] +::: + +:::{{versionadded}} VERSION_NEXT_FEATURE +::: +""", + ), "coverage_tool": attr.label( doc = """\ The coverage tool to be used for a particular Python interpreter. This can override `rules_python` defaults. +""", + ), + "os_name": attr.string( + doc = """ +The host OS the runtime is compatible with. + +If not set, then the runtime cannot be used as a `python_X_Y_host` runtime. + +If set, the `os_name`, `target_compatible_with` and `target_settings` attributes +should also be set. + +The values should be one of the values in `@platforms//os` + +:::{seealso} +Docs for [Registering custom runtimes] +::: + +:::{{versionadded}} VERSION_NEXT_FEATURE +::: """, ), "patch_strip": attr.int( @@ -1101,8 +1280,20 @@ The coverage tool to be used for a particular Python interpreter. This can overr ), "platform": attr.string( mandatory = True, - values = PLATFORMS.keys(), - doc = "The platform to override the values for, must be one of:\n{}.".format("\n".join(sorted(["* `{}`".format(p) for p in PLATFORMS]))), + doc = """ +The platform to override the values for, typically one of:\n +{platforms} + +Other values are allowed, in which case, `target_compatible_with`, +`target_settings`, `os_name`, and `arch` should be specified so the toolchain is +only used when appropriate. + +:::{{versionchanged}} VERSION_NEXT_FEATURE +Arbitrary platform strings allowed. +::: +""".format( + platforms = "\n".join(sorted(["* `{}`".format(p) for p in PLATFORMS])), + ), ), "python_version": attr.string( mandatory = True, @@ -1117,6 +1308,36 @@ The coverage tool to be used for a particular Python interpreter. This can overr doc = "The 'strip_prefix' for the archive, defaults to 'python'.", default = "python", ), + "target_compatible_with": attr.string_list( + doc = """ +The `target_compatible_with` values to use for the toolchain definition. + +If not set, then `os_name` and `arch` will be used to populate it. + +If set, `target_settings`, `os_name`, and `arch` should also be set. + +:::{seealso} +Docs for [Registering custom runtimes] +::: + +:::{{versionadded}} VERSION_NEXT_FEATURE +::: +""", + ), + "target_settings": attr.string_list( + doc = """ +The `target_setings` values to use for the toolchain definition. + +If set, `target_compatible_with`, `os_name`, and `arch` should also be set. + +:::{seealso} +Docs for [Registering custom runtimes] +::: + +:::{{versionadded}} VERSION_NEXT_FEATURE +::: +""", + ), "urls": attr.string_list( mandatory = False, doc = "The URL template to fetch releases for this Python version. If the URL template results in a relative fragment, default base URL is going to be used. Occurrences of `{python_version}`, `{platform}` and `{build}` will be interpolated based on the contents in the override and the known {attr}`platform` values.", diff --git a/python/private/python_repository.bzl b/python/private/python_repository.bzl index fd86b415cc..cb0731e6eb 100644 --- a/python/private/python_repository.bzl +++ b/python/private/python_repository.bzl @@ -15,7 +15,7 @@ """This file contains repository rules and macros to support toolchain registration. """ -load("//python:versions.bzl", "FREETHREADED", "INSTALL_ONLY", "PLATFORMS") +load("//python:versions.bzl", "FREETHREADED", "INSTALL_ONLY") load(":auth.bzl", "get_auth") load(":repo_utils.bzl", "REPO_DEBUG_ENV_VAR", "repo_utils") load(":text_util.bzl", "render") @@ -327,7 +327,6 @@ function defaults (e.g. `single_version_override` for `MODULE.bazel` files. "platform": attr.string( doc = "The platform name for the Python interpreter tarball.", mandatory = True, - values = PLATFORMS.keys(), ), "python_version": attr.string( doc = "The Python version.", diff --git a/python/private/pythons_hub.bzl b/python/private/pythons_hub.bzl index 53351cacb9..cc25b4ba1d 100644 --- a/python/private/pythons_hub.bzl +++ b/python/private/pythons_hub.bzl @@ -84,13 +84,7 @@ def _hub_build_file_content(rctx): ) _interpreters_bzl_template = """ -INTERPRETER_LABELS = {{ -{interpreter_labels} -}} -""" - -_line_for_hub_template = """\ - "{name}_host": Label("@{name}_host//:python"), +INTERPRETER_LABELS = {labels} """ _versions_bzl_template = """ @@ -110,15 +104,16 @@ def _hub_repo_impl(rctx): # Create a dict that is later used to create # a symlink to a interpreter. - interpreter_labels = "".join([ - _line_for_hub_template.format(name = name) - for name in rctx.attr.base_toolchain_repo_names - ]) - rctx.file( "interpreters.bzl", _interpreters_bzl_template.format( - interpreter_labels = interpreter_labels, + labels = render.dict( + { + name: 'Label("@{}//:python")'.format(name) + for name in rctx.attr.host_compatible_repo_names + }, + value_repr = str, + ), ), executable = False, ) @@ -144,15 +139,14 @@ This rule also writes out the various toolchains for the different Python versio """, implementation = _hub_repo_impl, attrs = { - "base_toolchain_repo_names": attr.string_list( - doc = "The base repo name for toolchains ('python_3_10', no " + - "platform suffix)", - mandatory = True, - ), "default_python_version": attr.string( doc = "Default Python version for the build in `X.Y` or `X.Y.Z` format.", mandatory = True, ), + "host_compatible_repo_names": attr.string_list( + doc = "Names of `host_compatible_python_repo` repos.", + mandatory = True, + ), "minor_mapping": attr.string_dict( doc = "The minor mapping of the `X.Y` to `X.Y.Z` format that is used in config settings.", mandatory = True, diff --git a/python/private/repo_utils.bzl b/python/private/repo_utils.bzl index eee56ec86c..32a5b70e15 100644 --- a/python/private/repo_utils.bzl +++ b/python/private/repo_utils.bzl @@ -31,13 +31,15 @@ def _is_repo_debug_enabled(mrctx): """ return _getenv(mrctx, REPO_DEBUG_ENV_VAR) == "1" -def _logger(mrctx, name = None): +def _logger(mrctx = None, name = None, verbosity_level = None): """Creates a logger instance for printing messages. Args: mrctx: repository_ctx or module_ctx object. If the attribute `_rule_name` is present, it will be included in log messages. name: name for the logger. Optional for repository_ctx usage. + verbosity_level: {type}`int | None` verbosity level. If not set, + taken from `mrctx` Returns: A struct with attributes logging: trace, debug, info, warn, fail. @@ -46,13 +48,14 @@ def _logger(mrctx, name = None): the logger injected into the function work as expected by terminating on the given line. """ - if _is_repo_debug_enabled(mrctx): - verbosity_level = "DEBUG" - else: - verbosity_level = "WARN" + if verbosity_level == None: + if _is_repo_debug_enabled(mrctx): + verbosity_level = "DEBUG" + else: + verbosity_level = "WARN" - env_var_verbosity = _getenv(mrctx, REPO_VERBOSITY_ENV_VAR) - verbosity_level = env_var_verbosity or verbosity_level + env_var_verbosity = _getenv(mrctx, REPO_VERBOSITY_ENV_VAR) + verbosity_level = env_var_verbosity or verbosity_level verbosity = { "DEBUG": 2, @@ -376,7 +379,7 @@ def _get_platforms_os_name(mrctx): """Return the name in @platforms//os for the host os. Args: - mrctx: module_ctx or repository_ctx. + mrctx: {type}`module_ctx | repository_ctx` Returns: `str`. The target name. @@ -405,6 +408,7 @@ def _get_platforms_cpu_name(mrctx): `str`. The target name. """ arch = mrctx.os.arch.lower() + if arch in ["i386", "i486", "i586", "i686", "i786", "x86"]: return "x86_32" if arch in ["amd64", "x86_64", "x64"]: diff --git a/python/private/toolchains_repo.bzl b/python/private/toolchains_repo.bzl index 2476889583..93bbb52108 100644 --- a/python/private/toolchains_repo.bzl +++ b/python/private/toolchains_repo.bzl @@ -309,11 +309,11 @@ actions.""", environ = [REPO_DEBUG_ENV_VAR], ) -def _host_compatible_python_repo(rctx): +def _host_compatible_python_repo_impl(rctx): rctx.file("BUILD.bazel", _HOST_TOOLCHAIN_BUILD_CONTENT) os_name = repo_utils.get_platforms_os_name(rctx) - host_platform = _get_host_platform( + impl_repo_name = _get_host_impl_repo_name( rctx = rctx, logger = repo_utils.logger(rctx), python_version = rctx.attr.python_version, @@ -321,10 +321,11 @@ def _host_compatible_python_repo(rctx): cpu_name = repo_utils.get_platforms_cpu_name(rctx), platforms = rctx.attr.platforms, ) - repo = "@@{py_repository}_{host_platform}".format( - py_repository = rctx.attr.name[:-len("_host")], - host_platform = host_platform, - ) + + # Bzlmod quirk: A repository rule can't, in its **implemention function**, + # resolve an apparent repo name referring to a repo created by the same + # bzlmod extension. To work around this, we use a canonical label. + repo = "@@{}".format(impl_repo_name) rctx.report_progress("Symlinking interpreter files to the target platform") host_python_repo = rctx.path(Label("{repo}//:BUILD.bazel".format(repo = repo))) @@ -380,26 +381,76 @@ def _host_compatible_python_repo(rctx): # NOTE: The term "toolchain" is a misnomer for this rule. This doesn't define # a repo with toolchains or toolchain implementations. host_compatible_python_repo = repository_rule( - _host_compatible_python_repo, + implementation = _host_compatible_python_repo_impl, doc = """\ Creates a repository with a shorter name meant to be used in the repository_ctx, which needs to have `symlinks` for the interpreter. This is separate from the toolchain_aliases repo because referencing the `python` interpreter target from this repo causes an eager fetch of the toolchain for the host platform. - """, + +This repo has two ways in which is it called: + +1. Workspace. The `platforms` attribute is set, which are keys into the + PLATFORMS global. It assumes `name` + is a + valid repo name which it can use as the backing repo. + +2. Bzlmod. All platform and backing repo information is passed in via the + arch_names, impl_repo_names, os_names, python_versions attributes. +""", attrs = { "arch_names": attr.string_dict( doc = """ -If set, overrides the platform metadata. Keyed by index in `platforms` +Arch (cpu) names. Only set in bzlmod. Keyed by index in `platforms` +""", + ), + "base_name": attr.string( + doc = """ +The name arg, but without bzlmod canonicalization applied. Only set in bzlmod. +""", + ), + "impl_repo_names": attr.string_dict( + doc = """ +The names of backing runtime repos. Only set in bzlmod. The names must be repos +in the same extension as creates the host repo. Keyed by index in `platforms`. """, ), "os_names": attr.string_dict( doc = """ -If set, overrides the platform metadata. Keyed by index in `platforms` +If set, overrides the platform metadata. Only set in bzlmod. Keyed by +index in `platforms` +""", + ), + "platforms": attr.string_list( + mandatory = True, + doc = """ +Platform names (workspace) or platform name-like keys (bzlmod) + +NOTE: The order of this list matters. The first platform that is compatible +with the host will be selected; this can be customized by using the +`RULES_PYTHON_REPO_TOOLCHAIN_*` env vars. + +The values passed vary depending on workspace vs bzlmod. + +Workspace: the values are keys into the `PLATFORMS` dict and are the suffix +to append to `name` to point to the backing repo name. + +Bzlmod: The values are arbitrary keys to create the platform map from the +other attributes (os_name, arch_names, et al). +""", + ), + "python_version": attr.string( + doc = """ +Full python version, Major.Minor.Micro. + +Only set in workspace calls. +""", + ), + "python_versions": attr.string_dict( + doc = """ +If set, the Python version for the corresponding selected platform. Values in +Major.Minor.Micro format. Keyed by index in `platforms`. """, ), - "platforms": attr.string_list(mandatory = True), - "python_version": attr.string(mandatory = True), "_rule_name": attr.string(default = "host_compatible_python_repo"), "_rules_python_workspace": attr.label(default = Label("//:WORKSPACE")), }, @@ -435,8 +486,8 @@ multi_toolchain_aliases = repository_rule( }, ) -def sorted_host_platforms(platform_map): - """Sort the keys in the platform map to give correct precedence. +def sorted_host_platform_names(platform_names): + """Sort platform names to give correct precedence. The order of keys in the platform mapping matters for the host toolchain selection. When multiple runtimes are compatible with the host, we take the @@ -453,11 +504,10 @@ def sorted_host_platforms(platform_map): is an innocous looking formatter disable directive. Args: - platform_map: a mapping of platforms and their metadata. + platform_names: a list of platform names Returns: - dict; the same values, but with the keys inserted in the desired - order so that iteration happens in the desired order. + list[str] the same values, but in the desired order. """ def platform_keyer(name): @@ -467,13 +517,26 @@ def sorted_host_platforms(platform_map): 1 if FREETHREADED in name else 0, ) - sorted_platform_keys = sorted(platform_map.keys(), key = platform_keyer) + return sorted(platform_names, key = platform_keyer) + +def sorted_host_platforms(platform_map): + """Sort the keys in the platform map to give correct precedence. + + See sorted_host_platform_names for explanation. + + Args: + platform_map: a mapping of platforms and their metadata. + + Returns: + dict; the same values, but with the keys inserted in the desired + order so that iteration happens in the desired order. + """ return { key: platform_map[key] - for key in sorted_platform_keys + for key in sorted_host_platform_names(platform_map.keys()) } -def _get_host_platform(*, rctx, logger, python_version, os_name, cpu_name, platforms): +def _get_host_impl_repo_name(*, rctx, logger, python_version, os_name, cpu_name, platforms): """Gets the host platform. Args: @@ -488,24 +551,40 @@ def _get_host_platform(*, rctx, logger, python_version, os_name, cpu_name, platf """ if rctx.attr.os_names: platform_map = {} + base_name = rctx.attr.base_name + if not base_name: + fail("The `base_name` attribute must be set under bzlmod") for i, platform_name in enumerate(platforms): key = str(i) + impl_repo_name = rctx.attr.impl_repo_names[key] + impl_repo_name = rctx.name.replace(base_name, impl_repo_name) platform_map[platform_name] = struct( os_name = rctx.attr.os_names[key], arch = rctx.attr.arch_names[key], + python_version = rctx.attr.python_versions[key], + impl_repo_name = impl_repo_name, ) else: - platform_map = sorted_host_platforms(PLATFORMS) + base_name = rctx.name.removesuffix("_host") + platform_map = {} + for platform_name, info in sorted_host_platforms(PLATFORMS).items(): + platform_map[platform_name] = struct( + os_name = info.os_name, + arch = info.arch, + python_version = python_version, + impl_repo_name = "{}_{}".format(base_name, platform_name), + ) candidates = [] for platform in platforms: meta = platform_map[platform] if meta.os_name == os_name and meta.arch == cpu_name: - candidates.append(platform) + candidates.append((platform, meta)) if len(candidates) == 1: - return candidates[0] + platform_name, meta = candidates[0] + return meta.impl_repo_name if candidates: env_var = "RULES_PYTHON_REPO_TOOLCHAIN_{}_{}_{}".format( @@ -525,7 +604,11 @@ def _get_host_platform(*, rctx, logger, python_version, os_name, cpu_name, platf candidates = [preference] if candidates: - return candidates[0] + platform_name, meta = candidates[0] + suffix = meta.impl_repo_name + if not suffix: + suffix = platform_name + return suffix return logger.fail("Could not find a compatible 'host' python for '{os_name}', '{cpu_name}' from the loaded platforms: {platforms}".format( os_name = os_name, diff --git a/python/versions.bzl b/python/versions.bzl index 166cc98851..e712a2e126 100644 --- a/python/versions.bzl +++ b/python/versions.bzl @@ -15,6 +15,8 @@ """The Python versions we use for the toolchains. """ +load("//python/private:platform_info.bzl", "platform_info") + # Values present in the @platforms//os package MACOS_NAME = "osx" LINUX_NAME = "linux" @@ -684,42 +686,12 @@ MINOR_MAPPING = { "3.13": "3.13.2", } -def _platform_info( - *, - compatible_with = [], - flag_values = {}, - target_settings = [], - os_name, - arch): - """Creates a struct of platform metadata. - - Args: - compatible_with: list[str], where the values are string labels. These - are the target_compatible_with values to use with the toolchain - flag_values: dict[str|Label, Any] of config_setting.flag_values - compatible values. DEPRECATED -- use target_settings instead - target_settings: list[str], where the values are string labels. These - are the target_settings values to use with the toolchain. - os_name: str, the os name; must match the name used in `@platfroms//os` - arch: str, the cpu name; must match the name used in `@platforms//cpu` - - Returns: - A struct with attributes and values matching the args. - """ - return struct( - compatible_with = compatible_with, - flag_values = flag_values, - target_settings = target_settings, - os_name = os_name, - arch = arch, - ) - def _generate_platforms(): is_libc_glibc = str(Label("//python/config_settings:_is_py_linux_libc_glibc")) is_libc_musl = str(Label("//python/config_settings:_is_py_linux_libc_musl")) platforms = { - "aarch64-apple-darwin": _platform_info( + "aarch64-apple-darwin": platform_info( compatible_with = [ "@platforms//os:macos", "@platforms//cpu:aarch64", @@ -727,7 +699,7 @@ def _generate_platforms(): os_name = MACOS_NAME, arch = "aarch64", ), - "aarch64-unknown-linux-gnu": _platform_info( + "aarch64-unknown-linux-gnu": platform_info( compatible_with = [ "@platforms//os:linux", "@platforms//cpu:aarch64", @@ -738,7 +710,7 @@ def _generate_platforms(): os_name = LINUX_NAME, arch = "aarch64", ), - "armv7-unknown-linux-gnu": _platform_info( + "armv7-unknown-linux-gnu": platform_info( compatible_with = [ "@platforms//os:linux", "@platforms//cpu:armv7", @@ -749,7 +721,7 @@ def _generate_platforms(): os_name = LINUX_NAME, arch = "arm", ), - "i386-unknown-linux-gnu": _platform_info( + "i386-unknown-linux-gnu": platform_info( compatible_with = [ "@platforms//os:linux", "@platforms//cpu:i386", @@ -760,7 +732,7 @@ def _generate_platforms(): os_name = LINUX_NAME, arch = "x86_32", ), - "ppc64le-unknown-linux-gnu": _platform_info( + "ppc64le-unknown-linux-gnu": platform_info( compatible_with = [ "@platforms//os:linux", "@platforms//cpu:ppc", @@ -771,7 +743,7 @@ def _generate_platforms(): os_name = LINUX_NAME, arch = "ppc", ), - "riscv64-unknown-linux-gnu": _platform_info( + "riscv64-unknown-linux-gnu": platform_info( compatible_with = [ "@platforms//os:linux", "@platforms//cpu:riscv64", @@ -782,7 +754,7 @@ def _generate_platforms(): os_name = LINUX_NAME, arch = "riscv64", ), - "s390x-unknown-linux-gnu": _platform_info( + "s390x-unknown-linux-gnu": platform_info( compatible_with = [ "@platforms//os:linux", "@platforms//cpu:s390x", @@ -793,7 +765,7 @@ def _generate_platforms(): os_name = LINUX_NAME, arch = "s390x", ), - "x86_64-apple-darwin": _platform_info( + "x86_64-apple-darwin": platform_info( compatible_with = [ "@platforms//os:macos", "@platforms//cpu:x86_64", @@ -801,7 +773,7 @@ def _generate_platforms(): os_name = MACOS_NAME, arch = "x86_64", ), - "x86_64-pc-windows-msvc": _platform_info( + "x86_64-pc-windows-msvc": platform_info( compatible_with = [ "@platforms//os:windows", "@platforms//cpu:x86_64", @@ -809,7 +781,7 @@ def _generate_platforms(): os_name = WINDOWS_NAME, arch = "x86_64", ), - "x86_64-unknown-linux-gnu": _platform_info( + "x86_64-unknown-linux-gnu": platform_info( compatible_with = [ "@platforms//os:linux", "@platforms//cpu:x86_64", @@ -820,7 +792,7 @@ def _generate_platforms(): os_name = LINUX_NAME, arch = "x86_64", ), - "x86_64-unknown-linux-musl": _platform_info( + "x86_64-unknown-linux-musl": platform_info( compatible_with = [ "@platforms//os:linux", "@platforms//cpu:x86_64", @@ -836,7 +808,7 @@ def _generate_platforms(): is_freethreaded_yes = str(Label("//python/config_settings:_is_py_freethreaded_yes")) is_freethreaded_no = str(Label("//python/config_settings:_is_py_freethreaded_no")) return { - p + suffix: _platform_info( + p + suffix: platform_info( compatible_with = v.compatible_with, target_settings = [ freethreadedness, diff --git a/tests/bootstrap_impls/bin.py b/tests/bootstrap_impls/bin.py index 1176107384..3d467dcf29 100644 --- a/tests/bootstrap_impls/bin.py +++ b/tests/bootstrap_impls/bin.py @@ -23,3 +23,4 @@ print("sys.flags.safe_path:", sys.flags.safe_path) print("file:", __file__) print("sys.executable:", sys.executable) +print("sys._base_executable:", sys._base_executable) diff --git a/tests/python/python_tests.bzl b/tests/python/python_tests.bzl index 19be1c478e..116afa76ad 100644 --- a/tests/python/python_tests.bzl +++ b/tests/python/python_tests.bzl @@ -17,6 +17,7 @@ load("@pythons_hub//:versions.bzl", "MINOR_MAPPING") load("@rules_testing//lib:test_suite.bzl", "test_suite") load("//python/private:python.bzl", "parse_modules") # buildifier: disable=bzl-visibility +load("//python/private:repo_utils.bzl", "repo_utils") # buildifier: disable=bzl-visibility _tests = [] @@ -131,6 +132,10 @@ def _single_version_platform_override( python_version = python_version, patch_strip = patch_strip, patches = patches, + target_compatible_with = [], + target_settings = [], + os_name = "", + arch = "", ) def _test_default(env): @@ -138,6 +143,7 @@ def _test_default(env): module_ctx = _mock_mctx( _mod(name = "rules_python", toolchain = [_toolchain("3.11")]), ), + logger = repo_utils.logger(verbosity_level = 0, name = "python"), ) # The value there should be consistent in bzlmod with the automatically @@ -168,6 +174,7 @@ def _test_default_some_module(env): module_ctx = _mock_mctx( _mod(name = "rules_python", toolchain = [_toolchain("3.11")], is_root = False), ), + logger = repo_utils.logger(verbosity_level = 0, name = "python"), ) env.expect.that_str(py.default_python_version).equals("3.11") @@ -186,6 +193,7 @@ def _test_default_with_patch_version(env): module_ctx = _mock_mctx( _mod(name = "rules_python", toolchain = [_toolchain("3.11.2")]), ), + logger = repo_utils.logger(verbosity_level = 0, name = "python"), ) env.expect.that_str(py.default_python_version).equals("3.11.2") @@ -207,6 +215,7 @@ def _test_default_non_rules_python(env): # does not make any calls to the extension. _mod(name = "rules_python", toolchain = [_toolchain("3.11")], is_root = False), ), + logger = repo_utils.logger(verbosity_level = 0, name = "python"), ) env.expect.that_str(py.default_python_version).equals("3.11") @@ -228,6 +237,7 @@ def _test_default_non_rules_python_ignore_root_user_error(env): ), _mod(name = "rules_python", toolchain = [_toolchain("3.11")]), ), + logger = repo_utils.logger(verbosity_level = 0, name = "python"), ) env.expect.that_bool(py.config.default["ignore_root_user_error"]).equals(False) @@ -257,6 +267,7 @@ def _test_default_non_rules_python_ignore_root_user_error_non_root_module(env): _mod(name = "some_module", toolchain = [_toolchain("3.12", ignore_root_user_error = False)]), _mod(name = "rules_python", toolchain = [_toolchain("3.11")]), ), + logger = repo_utils.logger(verbosity_level = 0, name = "python"), ) env.expect.that_str(py.default_python_version).equals("3.13") @@ -302,6 +313,7 @@ def _test_toolchain_ordering(env): ), _mod(name = "rules_python", toolchain = [_toolchain("3.11")]), ), + logger = repo_utils.logger(verbosity_level = 0, name = "python"), ) got_versions = [ t.python_version @@ -347,6 +359,7 @@ def _test_default_from_defaults(env): is_root = True, ), ), + logger = repo_utils.logger(verbosity_level = 0, name = "python"), ) env.expect.that_str(py.default_python_version).equals("3.11") @@ -374,6 +387,7 @@ def _test_default_from_defaults_env(env): ), environ = {"PYENV_VERSION": "3.12"}, ), + logger = repo_utils.logger(verbosity_level = 0, name = "python"), ) env.expect.that_str(py.default_python_version).equals("3.12") @@ -401,6 +415,7 @@ def _test_default_from_defaults_file(env): ), mocked_files = {"@@//:.python-version": "3.12\n"}, ), + logger = repo_utils.logger(verbosity_level = 0, name = "python"), ) env.expect.that_str(py.default_python_version).equals("3.12") @@ -427,6 +442,7 @@ def _test_first_occurance_of_the_toolchain_wins(env): "RULES_PYTHON_BZLMOD_DEBUG": "1", }, ), + logger = repo_utils.logger(verbosity_level = 0, name = "python"), ) env.expect.that_str(py.default_python_version).equals("3.12") @@ -472,6 +488,7 @@ def _test_auth_overrides(env): ), _mod(name = "rules_python", toolchain = [_toolchain("3.11")]), ), + logger = repo_utils.logger(verbosity_level = 0, name = "python"), ) env.expect.that_dict(py.config.default).contains_at_least({ @@ -541,6 +558,7 @@ def _test_add_new_version(env): ], ), ), + logger = repo_utils.logger(verbosity_level = 0, name = "python"), ) env.expect.that_str(py.default_python_version).equals("3.13") @@ -609,6 +627,7 @@ def _test_register_all_versions(env): ], ), ), + logger = repo_utils.logger(verbosity_level = 0, name = "python"), ) env.expect.that_str(py.default_python_version).equals("3.13") @@ -685,6 +704,7 @@ def _test_add_patches(env): ], ), ), + logger = repo_utils.logger(verbosity_level = 0, name = "python"), ) env.expect.that_str(py.default_python_version).equals("3.13") @@ -731,6 +751,7 @@ def _test_fail_two_overrides(env): ), ), _fail = errors.append, + logger = repo_utils.logger(verbosity_level = 0, name = "python"), ) env.expect.that_collection(errors).contains_exactly([ "Only a single 'python.override' can be present", @@ -758,6 +779,7 @@ def _test_single_version_override_errors(env): ), ), _fail = errors.append, + logger = repo_utils.logger(verbosity_level = 0, name = "python"), ) env.expect.that_collection(errors).contains_exactly([test.want_error]) @@ -795,6 +817,7 @@ def _test_single_version_platform_override_errors(env): ), ), _fail = lambda *a: errors.append(" ".join(a)), + logger = repo_utils.logger(verbosity_level = 0, name = "python"), ) env.expect.that_collection(errors).contains_exactly([test.want_error]) diff --git a/tests/support/BUILD.bazel b/tests/support/BUILD.bazel index 9fb5cd0760..303dbafbdf 100644 --- a/tests/support/BUILD.bazel +++ b/tests/support/BUILD.bazel @@ -18,6 +18,7 @@ # to force them to resolve in the proper context. # ==================== +load("@bazel_skylib//rules:common_settings.bzl", "string_flag") load(":sh_py_run_test.bzl", "current_build_settings") package( @@ -90,3 +91,15 @@ platform( current_build_settings( name = "current_build_settings", ) + +string_flag( + name = "custom_runtime", + build_setting_default = "", +) + +config_setting( + name = "is_custom_runtime_linux-x86-install-only-stripped", + flag_values = { + ":custom_runtime": "linux-x86-install-only-stripped", + }, +) diff --git a/tests/support/sh_py_run_test.bzl b/tests/support/sh_py_run_test.bzl index 04a2883fde..69141fe8a4 100644 --- a/tests/support/sh_py_run_test.bzl +++ b/tests/support/sh_py_run_test.bzl @@ -42,6 +42,7 @@ def _perform_transition_impl(input_settings, attr, base_impl): # value into the output settings _RECONFIG_ATTR_SETTING_MAP = { "bootstrap_impl": "//python/config_settings:bootstrap_impl", + "custom_runtime": "//tests/support:custom_runtime", "extra_toolchains": "//command_line_option:extra_toolchains", "python_src": "//python/bin:python_src", "venvs_site_packages": "//python/config_settings:venvs_site_packages", @@ -58,6 +59,7 @@ _RECONFIG_INHERITED_OUTPUTS = [v for v in _RECONFIG_OUTPUTS if v in _RECONFIG_IN _RECONFIG_ATTRS = { "bootstrap_impl": attrb.String(), "build_python_zip": attrb.String(default = "auto"), + "custom_runtime": attrb.String(), "extra_toolchains": attrb.StringList( doc = """ Value for the --extra_toolchains flag. diff --git a/tests/toolchains/BUILD.bazel b/tests/toolchains/BUILD.bazel index c55dc92a7d..f346651d46 100644 --- a/tests/toolchains/BUILD.bazel +++ b/tests/toolchains/BUILD.bazel @@ -12,8 +12,21 @@ # See the License for the specific language governing permissions and # limitations under the License. +load("//python/private:bzlmod_enabled.bzl", "BZLMOD_ENABLED") # buildifier: disable=bzl-visibility +load("//tests/support:sh_py_run_test.bzl", "py_reconfig_test") load(":defs.bzl", "define_toolchain_tests") define_toolchain_tests( name = "toolchain_tests", ) + +py_reconfig_test( + name = "custom_platform_toolchain_test", + srcs = ["custom_platform_toolchain_test.py"], + custom_runtime = "linux-x86-install-only-stripped", + python_version = "3.13.1", + target_compatible_with = [ + "@platforms//os:linux", + "@platforms//cpu:x86_64", + ] if BZLMOD_ENABLED else ["@platforms//:incompatible"], +) diff --git a/tests/toolchains/custom_platform_toolchain_test.py b/tests/toolchains/custom_platform_toolchain_test.py new file mode 100644 index 0000000000..d6c083a6a2 --- /dev/null +++ b/tests/toolchains/custom_platform_toolchain_test.py @@ -0,0 +1,15 @@ +import sys +import unittest + + +class VerifyCustomPlatformToolchainTest(unittest.TestCase): + + def test_custom_platform_interpreter_used(self): + # We expect the repo name, and thus path, to have the + # platform name in it. + self.assertIn("linux-x86-install-only-stripped", sys._base_executable) + print(sys._base_executable) + + +if __name__ == "__main__": + unittest.main() From ce80db6a8640cc7552e4b5eada891cd19c4550f2 Mon Sep 17 00:00:00 2001 From: Vihang Mehta Date: Thu, 29 May 2025 18:16:03 -0700 Subject: [PATCH 265/922] feat: Support constraints in pip_compile (#2916) This adds in support to pass in a constraints file to pip-compile. This is extremly useful when you want to uprade an indirect/intermediate dependency to pull in security fixes but don't want to add said dependency to the requirements.in file. --------- Signed-off-by: Vihang Mehta Co-authored-by: Ignas Anikevicius <240938+aignas@users.noreply.github.com> --- CHANGELOG.md | 3 +++ examples/pip_parse/BUILD.bazel | 4 ++++ examples/pip_parse/constraints_certifi.txt | 1 + examples/pip_parse/constraints_urllib3.txt | 1 + examples/pip_parse/requirements_lock.txt | 20 ++++++++++++-------- examples/pip_parse/requirements_windows.txt | 20 ++++++++++++-------- python/private/pypi/pip_compile.bzl | 6 +++++- 7 files changed, 38 insertions(+), 17 deletions(-) create mode 100644 examples/pip_parse/constraints_certifi.txt create mode 100644 examples/pip_parse/constraints_urllib3.txt diff --git a/CHANGELOG.md b/CHANGELOG.md index a113c7411f..355f1fe9ef 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -111,6 +111,9 @@ END_UNRELEASED_TEMPLATE and activated with custom flags. See the [Registering custom runtimes] docs and {obj}`single_version_platform_override()` API docs for more information. +* (rules) Added support for a using constraints files with `compile_pip_requirements`. + Useful when an intermediate dependency needs to be upgraded to pull in + security patches. {#v0-0-0-removed} ### Removed diff --git a/examples/pip_parse/BUILD.bazel b/examples/pip_parse/BUILD.bazel index 8bdbd94b2c..6ed8d26286 100644 --- a/examples/pip_parse/BUILD.bazel +++ b/examples/pip_parse/BUILD.bazel @@ -57,6 +57,10 @@ py_console_script_binary( compile_pip_requirements( name = "requirements", src = "requirements.in", + constraints = [ + "constraints_certifi.txt", + "constraints_urllib3.txt", + ], requirements_txt = "requirements_lock.txt", requirements_windows = "requirements_windows.txt", ) diff --git a/examples/pip_parse/constraints_certifi.txt b/examples/pip_parse/constraints_certifi.txt new file mode 100644 index 0000000000..7dc4eac259 --- /dev/null +++ b/examples/pip_parse/constraints_certifi.txt @@ -0,0 +1 @@ +certifi>=2025.1.31 \ No newline at end of file diff --git a/examples/pip_parse/constraints_urllib3.txt b/examples/pip_parse/constraints_urllib3.txt new file mode 100644 index 0000000000..3818262552 --- /dev/null +++ b/examples/pip_parse/constraints_urllib3.txt @@ -0,0 +1 @@ +urllib3>1.26.18 diff --git a/examples/pip_parse/requirements_lock.txt b/examples/pip_parse/requirements_lock.txt index aeac61eff9..dc34b45a45 100644 --- a/examples/pip_parse/requirements_lock.txt +++ b/examples/pip_parse/requirements_lock.txt @@ -12,10 +12,12 @@ babel==2.13.1 \ --hash=sha256:33e0952d7dd6374af8dbf6768cc4ddf3ccfefc244f9986d4074704f2fbd18900 \ --hash=sha256:7077a4984b02b6727ac10f1f7294484f737443d7e2e66c5e4380e41a3ae0b4ed # via sphinx -certifi==2024.7.4 \ - --hash=sha256:5a1e7645bc0ec61a09e26c36f6106dd4cf40c6db3a1fb6352b0244e7fb057c7b \ - --hash=sha256:c198e21b1289c2ab85ee4e67bb4b4ef3ead0892059901a8d5b622f24a1101e90 - # via requests +certifi==2025.4.26 \ + --hash=sha256:0a816057ea3cdefcef70270d2c515e4506bbc954f417fa5ade2021213bb8f0c6 \ + --hash=sha256:30350364dfe371162649852c63336a15c70c6510c2ad5015b21c2345311805f3 + # via + # -c ./constraints_certifi.txt + # requests chardet==4.0.0 \ --hash=sha256:0d6f53a15db4120f2b08c94f11e7d93d2c911ee118b6b30a04ec3ee8310179fa \ --hash=sha256:f864054d66fd9118f2e67044ac8981a54775ec5b67aed0441892edb553d21da5 @@ -218,10 +220,12 @@ sphinxcontrib-serializinghtml==1.1.9 \ # via # -r requirements.in # sphinx -urllib3==1.26.18 \ - --hash=sha256:34b97092d7e0a3a8cf7cd10e386f401b3737364026c45e622aa02903dffe0f07 \ - --hash=sha256:f8ecc1bba5667413457c529ab955bf8c67b45db799d159066261719e328580a0 - # via requests +urllib3==1.26.20 \ + --hash=sha256:0ed14ccfbf1c30a9072c7ca157e4319b70d65f623e91e7b32fadb2853431016e \ + --hash=sha256:40c2dc0c681e47eb8f90e7e27bf6ff7df2e677421fd46756da1161c39ca70d32 + # via + # -c ./constraints_urllib3.txt + # requests yamllint==1.28.0 \ --hash=sha256:89bb5b5ac33b1ade059743cf227de73daa34d5e5a474b06a5e17fc16583b0cf2 \ --hash=sha256:9e3d8ddd16d0583214c5fdffe806c9344086721f107435f68bad990e5a88826b diff --git a/examples/pip_parse/requirements_windows.txt b/examples/pip_parse/requirements_windows.txt index 61a6682047..78c1a45690 100644 --- a/examples/pip_parse/requirements_windows.txt +++ b/examples/pip_parse/requirements_windows.txt @@ -12,10 +12,12 @@ babel==2.13.1 \ --hash=sha256:33e0952d7dd6374af8dbf6768cc4ddf3ccfefc244f9986d4074704f2fbd18900 \ --hash=sha256:7077a4984b02b6727ac10f1f7294484f737443d7e2e66c5e4380e41a3ae0b4ed # via sphinx -certifi==2024.7.4 \ - --hash=sha256:5a1e7645bc0ec61a09e26c36f6106dd4cf40c6db3a1fb6352b0244e7fb057c7b \ - --hash=sha256:c198e21b1289c2ab85ee4e67bb4b4ef3ead0892059901a8d5b622f24a1101e90 - # via requests +certifi==2025.4.26 \ + --hash=sha256:0a816057ea3cdefcef70270d2c515e4506bbc954f417fa5ade2021213bb8f0c6 \ + --hash=sha256:30350364dfe371162649852c63336a15c70c6510c2ad5015b21c2345311805f3 + # via + # -c ./constraints_certifi.txt + # requests chardet==4.0.0 \ --hash=sha256:0d6f53a15db4120f2b08c94f11e7d93d2c911ee118b6b30a04ec3ee8310179fa \ --hash=sha256:f864054d66fd9118f2e67044ac8981a54775ec5b67aed0441892edb553d21da5 @@ -222,10 +224,12 @@ sphinxcontrib-serializinghtml==1.1.9 \ # via # -r requirements.in # sphinx -urllib3==1.26.18 \ - --hash=sha256:34b97092d7e0a3a8cf7cd10e386f401b3737364026c45e622aa02903dffe0f07 \ - --hash=sha256:f8ecc1bba5667413457c529ab955bf8c67b45db799d159066261719e328580a0 - # via requests +urllib3==1.26.20 \ + --hash=sha256:0ed14ccfbf1c30a9072c7ca157e4319b70d65f623e91e7b32fadb2853431016e \ + --hash=sha256:40c2dc0c681e47eb8f90e7e27bf6ff7df2e677421fd46756da1161c39ca70d32 + # via + # -c ./constraints_urllib3.txt + # requests yamllint==1.28.0 \ --hash=sha256:89bb5b5ac33b1ade059743cf227de73daa34d5e5a474b06a5e17fc16583b0cf2 \ --hash=sha256:9e3d8ddd16d0583214c5fdffe806c9344086721f107435f68bad990e5a88826b diff --git a/python/private/pypi/pip_compile.bzl b/python/private/pypi/pip_compile.bzl index 9782d3ce21..c9899503d6 100644 --- a/python/private/pypi/pip_compile.bzl +++ b/python/private/pypi/pip_compile.bzl @@ -38,6 +38,7 @@ def pip_compile( requirements_windows = None, visibility = ["//visibility:private"], tags = None, + constraints = [], **kwargs): """Generates targets for managing pip dependencies with pip-compile. @@ -77,6 +78,7 @@ def pip_compile( requirements_windows: File of windows specific resolve output to check validate if requirement.in has changes. tags: tagging attribute common to all build rules, passed to both the _test and .update rules. visibility: passed to both the _test and .update rules. + constraints: a list of files containing constraints to pass to pip-compile with `--constraint`. **kwargs: other bazel attributes passed to the "_test" rule. """ if len([x for x in [srcs, src, requirements_in] if x != None]) > 1: @@ -100,7 +102,7 @@ def pip_compile( visibility = visibility, ) - data = [name, requirements_txt] + srcs + [f for f in (requirements_linux, requirements_darwin, requirements_windows) if f != None] + data = [name, requirements_txt] + srcs + [f for f in (requirements_linux, requirements_darwin, requirements_windows) if f != None] + constraints # Use the Label constructor so this is expanded in the context of the file # where it appears, which is to say, in @rules_python @@ -122,6 +124,8 @@ def pip_compile( args.append("--requirements-darwin={}".format(loc.format(requirements_darwin))) if requirements_windows: args.append("--requirements-windows={}".format(loc.format(requirements_windows))) + for constraint in constraints: + args.append("--constraint=$(location {})".format(constraint)) args.extend(extra_args) deps = [ From af9e959538f34878ca0ccccd97d51dc7b3ffdadd Mon Sep 17 00:00:00 2001 From: rbeasley-avgo Date: Fri, 30 May 2025 05:25:03 -0400 Subject: [PATCH 266/922] fix(pypi): allow pip_compile to work with read-only sources (#2712) The validating `py_test` generated by `compile_pip_requirements` chokes when the source `requirements.txt` is stored read-only, such as when managed by the Perforce Helix Core SCM. Though `dependency_resolver` makes a temporary copy of this file, it does so w/ `shutil.copy` which preserves the original read-only file mode. To address this, this commit replaces `shutil.copy` with a `shutil.copyfileobj` such that the temporary file is created w/ permissions according to the user's umask. Resolves (#2608). --------- Co-authored-by: Ignas Anikevicius <240938+aignas@users.noreply.github.com> --- CHANGELOG.md | 2 ++ .../pypi/dependency_resolver/dependency_resolver.py | 9 ++++++++- 2 files changed, 10 insertions(+), 1 deletion(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 355f1fe9ef..0a2dc413ae 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -93,6 +93,8 @@ END_UNRELEASED_TEMPLATE also retrieved from the URL as opposed to only the `--hash` parameter. Fixes [#2363](https://github.com/bazel-contrib/rules_python/issues/2363). * (pypi) `whl_library` now infers file names from its `urls` attribute correctly. +* (pypi) When running under `bazel test`, be sure that temporary `requirements` file + remains writable. * (py_test, py_binary) Allow external files to be used for main {#v0-0-0-added} diff --git a/python/private/pypi/dependency_resolver/dependency_resolver.py b/python/private/pypi/dependency_resolver/dependency_resolver.py index ada0763558..a42821c458 100644 --- a/python/private/pypi/dependency_resolver/dependency_resolver.py +++ b/python/private/pypi/dependency_resolver/dependency_resolver.py @@ -151,9 +151,16 @@ def main( requirements_out = os.path.join( os.environ["TEST_TMPDIR"], os.path.basename(requirements_file) + ".out" ) + # Why this uses shutil.copyfileobj: + # # Those two files won't necessarily be on the same filesystem, so we can't use os.replace # or shutil.copyfile, as they will fail with OSError: [Errno 18] Invalid cross-device link. - shutil.copy(resolved_requirements_file, requirements_out) + # + # Further, shutil.copy preserves the source file's mode, and so if + # our source file is read-only (the default under Perforce Helix), + # this scratch file will also be read-only, defeating its purpose. + with open(resolved_requirements_file, "rb") as fsrc, open(requirements_out, "wb") as fdst: + shutil.copyfileobj(fsrc, fdst) update_command = ( os.getenv("CUSTOM_COMPILE_COMMAND") or f"bazel run {target_label_prefix}.update" From 02198f622ee1b496111bef6b880ea35e0d24b600 Mon Sep 17 00:00:00 2001 From: Ignas Anikevicius <240938+aignas@users.noreply.github.com> Date: Sat, 31 May 2025 15:52:54 +0900 Subject: [PATCH 267/922] feat(uv): handle credential helpers and .netrc (#2872) This allows one to download the uv binaries from private mirrors. The plumbing of the auth attrs allows us to correctly use the `~/.netrc` or the credential helper for downloading from mirrors that require authentication. Testing notes: * When I tested this, it seems that the dist manifest json may not work with private mirrors, but I think it is fine for users in such cases to define the `uv` srcs using the `urls` attribute. Work towards #1975. --- CHANGELOG.md | 2 ++ python/uv/private/BUILD.bazel | 2 ++ python/uv/private/uv.bzl | 35 +++++++++++++++++++++++------ python/uv/private/uv_repository.bzl | 5 ++++- tests/uv/uv/uv_tests.bzl | 23 ++++++++++++++++++- 5 files changed, 58 insertions(+), 9 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 0a2dc413ae..f82df5aad0 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -109,6 +109,8 @@ END_UNRELEASED_TEMPLATE Set the `RULES_PYTHON_ENABLE_PIPSTAR=1` environment variable to enable it. * (utils) Add a way to run a REPL for any `rules_python` target that returns a `PyInfo` provider. +* (uv) Handle `.netrc` and `auth_patterns` auth when downloading `uv`. Work towards + [#1975](https://github.com/bazel-contrib/rules_python/issues/1975). * (toolchains) Arbitrary python-build-standalone runtimes can be registered and activated with custom flags. See the [Registering custom runtimes] docs and {obj}`single_version_platform_override()` API docs for more diff --git a/python/uv/private/BUILD.bazel b/python/uv/private/BUILD.bazel index 587ad9a0f9..a07d8591ad 100644 --- a/python/uv/private/BUILD.bazel +++ b/python/uv/private/BUILD.bazel @@ -62,6 +62,7 @@ bzl_library( ":toolchain_types_bzl", ":uv_repository_bzl", ":uv_toolchains_repo_bzl", + "//python/private:auth_bzl", ], ) @@ -69,6 +70,7 @@ bzl_library( name = "uv_repository_bzl", srcs = ["uv_repository.bzl"], visibility = ["//python/uv:__subpackages__"], + deps = ["//python/private:auth_bzl"], ) bzl_library( diff --git a/python/uv/private/uv.bzl b/python/uv/private/uv.bzl index 09fb78322f..2cc2df1b21 100644 --- a/python/uv/private/uv.bzl +++ b/python/uv/private/uv.bzl @@ -18,6 +18,7 @@ EXPERIMENTAL: This is experimental and may be removed without notice A module extension for working with uv. """ +load("//python/private:auth.bzl", "AUTH_ATTRS", "get_auth") load(":toolchain_types.bzl", "UV_TOOLCHAIN_TYPE") load(":uv_repository.bzl", "uv_repository") load(":uv_toolchains_repo.bzl", "uv_toolchains_repo") @@ -77,7 +78,7 @@ The version of uv to configure the sources for. If this is not specified it will last version used in the module or the default version set by `rules_python`. """, ), -} +} | AUTH_ATTRS default = tag_class( doc = """\ @@ -133,7 +134,7 @@ for a particular version. }, ) -def _configure(config, *, platform, compatible_with, target_settings, urls = [], sha256 = "", override = False, **values): +def _configure(config, *, platform, compatible_with, target_settings, auth_patterns, urls = [], sha256 = "", override = False, **values): """Set the value in the config if the value is provided""" for key, value in values.items(): if not value: @@ -144,6 +145,7 @@ def _configure(config, *, platform, compatible_with, target_settings, urls = [], config[key] = value + config.setdefault("auth_patterns", {}).update(auth_patterns) config.setdefault("platforms", {}) if not platform: if compatible_with or target_settings or urls: @@ -173,7 +175,8 @@ def process_modules( hub_name = "uv", uv_repository = uv_repository, toolchain_type = str(UV_TOOLCHAIN_TYPE), - hub_repo = uv_toolchains_repo): + hub_repo = uv_toolchains_repo, + get_auth = get_auth): """Parse the modules to get the config for 'uv' toolchains. Args: @@ -182,6 +185,7 @@ def process_modules( uv_repository: the rule to create a uv_repository override. toolchain_type: the toolchain type to use here. hub_repo: the hub repo factory function to use. + get_auth: the auth function to use. Returns: the result of the hub_repo. Mainly used for tests. @@ -216,6 +220,8 @@ def process_modules( compatible_with = tag.compatible_with, target_settings = tag.target_settings, override = mod.is_root, + netrc = tag.netrc, + auth_patterns = tag.auth_patterns, ) for key in [ @@ -271,6 +277,8 @@ def process_modules( sha256 = tag.sha256, urls = tag.urls, override = mod.is_root, + netrc = tag.netrc, + auth_patterns = tag.auth_patterns, ) if not versions: @@ -301,6 +309,11 @@ def process_modules( for platform, src in config.get("urls", {}).items() if src.urls } + auth = { + "auth_patterns": config.get("auth_patterns"), + "netrc": config.get("netrc"), + } + auth = {k: v for k, v in auth.items() if v} # Or fallback to fetching them from GH manifest file # Example file: https://github.com/astral-sh/uv/releases/download/0.6.3/dist-manifest.json @@ -313,6 +326,8 @@ def process_modules( ), manifest_filename = config["manifest_filename"], platforms = sorted(platforms), + get_auth = get_auth, + **auth ) for platform_name, platform in platforms.items(): @@ -327,6 +342,7 @@ def process_modules( platform = platform_name, urls = urls[platform_name].urls, sha256 = urls[platform_name].sha256, + **auth ) toolchain_names.append(toolchain_name) @@ -363,7 +379,7 @@ def _overlap(first_collection, second_collection): return False -def _get_tool_urls_from_dist_manifest(module_ctx, *, base_url, manifest_filename, platforms): +def _get_tool_urls_from_dist_manifest(module_ctx, *, base_url, manifest_filename, platforms, get_auth = get_auth, **auth_attrs): """Download the results about remote tool sources. This relies on the tools using the cargo packaging to infer the actual @@ -431,10 +447,13 @@ def _get_tool_urls_from_dist_manifest(module_ctx, *, base_url, manifest_filename "aarch64-apple-darwin" ] """ + auth_attr = struct(**auth_attrs) dist_manifest = module_ctx.path(manifest_filename) + urls = [base_url + "/" + manifest_filename] result = module_ctx.download( - base_url + "/" + manifest_filename, + url = urls, output = dist_manifest, + auth = get_auth(module_ctx, urls, ctx_attr = auth_attr), ) if not result.success: fail(result) @@ -454,11 +473,13 @@ def _get_tool_urls_from_dist_manifest(module_ctx, *, base_url, manifest_filename checksum_fname = checksum["name"] checksum_path = module_ctx.path(checksum_fname) + urls = ["{}/{}".format(base_url, checksum_fname)] downloads[checksum_path] = struct( download = module_ctx.download( - "{}/{}".format(base_url, checksum_fname), + url = urls, output = checksum_path, block = False, + auth = get_auth(module_ctx, urls, ctx_attr = auth_attr), ), archive_fname = fname, platforms = checksum["target_triples"], @@ -473,7 +494,7 @@ def _get_tool_urls_from_dist_manifest(module_ctx, *, base_url, manifest_filename sha256, _, checksummed_fname = module_ctx.read(checksum_path).partition(" ") checksummed_fname = checksummed_fname.strip(" *\n") - if archive_fname != checksummed_fname: + if checksummed_fname and archive_fname != checksummed_fname: fail("The checksum is for a different file, expected '{}' but got '{}'".format( archive_fname, checksummed_fname, diff --git a/python/uv/private/uv_repository.bzl b/python/uv/private/uv_repository.bzl index ba7d2a766c..fed4f576d3 100644 --- a/python/uv/private/uv_repository.bzl +++ b/python/uv/private/uv_repository.bzl @@ -18,6 +18,8 @@ EXPERIMENTAL: This is experimental and may be removed without notice Create repositories for uv toolchain dependencies """ +load("//python/private:auth.bzl", "AUTH_ATTRS", "get_auth") + UV_BUILD_TMPL = """\ # Generated by repositories.bzl load("@rules_python//python/uv:uv_toolchain.bzl", "uv_toolchain") @@ -43,6 +45,7 @@ def _uv_repo_impl(repository_ctx): url = repository_ctx.attr.urls, sha256 = repository_ctx.attr.sha256, stripPrefix = strip_prefix, + auth = get_auth(repository_ctx, repository_ctx.attr.urls), ) binary = "uv.exe" if is_windows else "uv" @@ -70,5 +73,5 @@ uv_repository = repository_rule( "sha256": attr.string(mandatory = False), "urls": attr.string_list(mandatory = True), "version": attr.string(mandatory = True), - }, + } | AUTH_ATTRS, ) diff --git a/tests/uv/uv/uv_tests.bzl b/tests/uv/uv/uv_tests.bzl index bf0deefa88..b464dab55c 100644 --- a/tests/uv/uv/uv_tests.bzl +++ b/tests/uv/uv/uv_tests.bzl @@ -100,7 +100,7 @@ def _mod(*, name = None, default = [], configure = [], is_root = True): ) def _process_modules(env, **kwargs): - result = process_modules(hub_repo = struct, **kwargs) + result = process_modules(hub_repo = struct, get_auth = lambda *_, **__: None, **kwargs) return env.expect.that_struct( struct( @@ -124,6 +124,8 @@ def _default( platform = None, target_settings = None, version = None, + netrc = None, + auth_patterns = None, **kwargs): return struct( base_url = base_url, @@ -132,6 +134,8 @@ def _default( platform = platform, target_settings = [] + (target_settings or []), # ensure that the type is correct version = version, + netrc = netrc, + auth_patterns = {} | (auth_patterns or {}), # ensure that the type is correct **kwargs ) @@ -377,6 +381,11 @@ def _test_complex_configuring(env): platform = "linux", compatible_with = ["@platforms//os:linux"], ), + _configure( + version = "1.0.4", + netrc = "~/.my_netrc", + auth_patterns = {"foo": "bar"}, + ), # use auth ], ), ), @@ -388,18 +397,21 @@ def _test_complex_configuring(env): "1_0_1_osx", "1_0_2_osx", "1_0_3_linux", + "1_0_4_osx", ]) uv.implementations().contains_exactly({ "1_0_0_osx": "@uv_1_0_0_osx//:uv_toolchain", "1_0_1_osx": "@uv_1_0_1_osx//:uv_toolchain", "1_0_2_osx": "@uv_1_0_2_osx//:uv_toolchain", "1_0_3_linux": "@uv_1_0_3_linux//:uv_toolchain", + "1_0_4_osx": "@uv_1_0_4_osx//:uv_toolchain", }) uv.compatible_with().contains_exactly({ "1_0_0_osx": ["@platforms//os:os"], "1_0_1_osx": ["@platforms//os:os"], "1_0_2_osx": ["@platforms//os:different"], "1_0_3_linux": ["@platforms//os:linux"], + "1_0_4_osx": ["@platforms//os:os"], }) uv.target_settings().contains_exactly({}) env.expect.that_collection(calls).contains_exactly([ @@ -431,6 +443,15 @@ def _test_complex_configuring(env): "urls": ["https://example.org/1.0.3/linux"], "version": "1.0.3", }, + { + "auth_patterns": {"foo": "bar"}, + "name": "uv_1_0_4_osx", + "netrc": "~/.my_netrc", + "platform": "osx", + "sha256": "deadb00f", + "urls": ["https://example.org/1.0.4/osx"], + "version": "1.0.4", + }, ]) _tests.append(_test_complex_configuring) From 948fcec44edbe12f4edf94db098c761570a72763 Mon Sep 17 00:00:00 2001 From: Ignas Anikevicius <240938+aignas@users.noreply.github.com> Date: Tue, 3 Jun 2025 00:44:57 +0900 Subject: [PATCH 268/922] fix(pypi): correctly aggregate the requirements files (#2932) This implements the actual fix where we are aggregating the whls and sdists correctly from multiple different requirements lines. Fixes #2648. Closes #2658. --- CHANGELOG.md | 2 + python/private/pypi/parse_requirements.bzl | 18 +++- .../parse_requirements_tests.bzl | 84 ++++++++++++++++++- 3 files changed, 97 insertions(+), 7 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index f82df5aad0..c9668c507f 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -96,6 +96,8 @@ END_UNRELEASED_TEMPLATE * (pypi) When running under `bazel test`, be sure that temporary `requirements` file remains writable. * (py_test, py_binary) Allow external files to be used for main +* (pypi) Correctly aggregate the sources when the hashes specified in the lockfile differ + by platform even though the same version is used. Fixes [#2648](https://github.com/bazel-contrib/rules_python/issues/2648). {#v0-0-0-added} ### Added diff --git a/python/private/pypi/parse_requirements.bzl b/python/private/pypi/parse_requirements.bzl index bd2981efc0..e4a8b90acb 100644 --- a/python/private/pypi/parse_requirements.bzl +++ b/python/private/pypi/parse_requirements.bzl @@ -223,7 +223,7 @@ def _package_srcs( env_marker_target_platforms, extract_url_srcs): """A function to return sources for a particular package.""" - srcs = [] + srcs = {} for r in sorted(reqs.values(), key = lambda r: r.requirement_line): whls, sdist = _add_dists( requirement = r, @@ -249,21 +249,31 @@ def _package_srcs( )] req_line = r.srcs.requirement_line + extra_pip_args = tuple(r.extra_pip_args) for dist in all_dists: - srcs.append( + key = ( + dist.filename, + req_line, + extra_pip_args, + ) + entry = srcs.setdefault( + key, struct( distribution = name, extra_pip_args = r.extra_pip_args, requirement_line = req_line, - target_platforms = target_platforms, + target_platforms = [], filename = dist.filename, sha256 = dist.sha256, url = dist.url, yanked = dist.yanked, ), ) + for p in target_platforms: + if p not in entry.target_platforms: + entry.target_platforms.append(p) - return srcs + return srcs.values() def select_requirement(requirements, *, platform): """A simple function to get a requirement for a particular platform. diff --git a/tests/pypi/parse_requirements/parse_requirements_tests.bzl b/tests/pypi/parse_requirements/parse_requirements_tests.bzl index 926a7e0c50..82fdd0a051 100644 --- a/tests/pypi/parse_requirements/parse_requirements_tests.bzl +++ b/tests/pypi/parse_requirements/parse_requirements_tests.bzl @@ -38,7 +38,7 @@ foo[extra]==0.0.1 \ foo @ git+https://github.com/org/foo.git@deadbeef """, "requirements_linux": """\ -foo==0.0.3 --hash=sha256:deadbaaf +foo==0.0.3 --hash=sha256:deadbaaf --hash=sha256:5d15t """, # download_only = True "requirements_linux_download_only": """\ @@ -67,7 +67,7 @@ foo==0.0.4 @ https://example.org/foo-0.0.4.whl foo==0.0.5 @ https://example.org/foo-0.0.5.whl --hash=sha256:deadbeef """, "requirements_osx": """\ -foo==0.0.3 --hash=sha256:deadbaaf +foo==0.0.3 --hash=sha256:deadbaaf --hash=sha256:deadb11f --hash=sha256:5d15t """, "requirements_osx_download_only": """\ --platform=macosx_10_9_arm64 @@ -251,7 +251,7 @@ def _test_multi_os(env): struct( distribution = "foo", extra_pip_args = [], - requirement_line = "foo==0.0.3 --hash=sha256:deadbaaf", + requirement_line = "foo==0.0.3 --hash=sha256:deadbaaf --hash=sha256:5d15t", target_platforms = ["linux_x86_64"], url = "", filename = "", @@ -515,6 +515,84 @@ def _test_git_sources(env): _tests.append(_test_git_sources) +def _test_overlapping_shas_with_index_results(env): + got = parse_requirements( + ctx = _mock_ctx(), + requirements_by_platform = { + "requirements_linux": ["cp39_linux_x86_64"], + "requirements_osx": ["cp39_osx_x86_64"], + }, + get_index_urls = lambda _, __: { + "foo": struct( + sdists = { + "5d15t": struct( + url = "sdist", + sha256 = "5d15t", + filename = "foo-0.0.1.tar.gz", + yanked = False, + ), + }, + whls = { + "deadb11f": struct( + url = "super2", + sha256 = "deadb11f", + filename = "foo-0.0.1-py3-none-macosx_14_0_x86_64.whl", + yanked = False, + ), + "deadbaaf": struct( + url = "super2", + sha256 = "deadbaaf", + filename = "foo-0.0.1-py3-none-any.whl", + yanked = False, + ), + }, + ), + }, + ) + + env.expect.that_collection(got).contains_exactly([ + struct( + name = "foo", + is_exposed = True, + # TODO @aignas 2025-05-25: how do we rename this? + is_multiple_versions = True, + srcs = [ + struct( + distribution = "foo", + extra_pip_args = [], + filename = "foo-0.0.1-py3-none-any.whl", + requirement_line = "foo==0.0.3", + sha256 = "deadbaaf", + target_platforms = ["cp39_linux_x86_64", "cp39_osx_x86_64"], + url = "super2", + yanked = False, + ), + struct( + distribution = "foo", + extra_pip_args = [], + filename = "foo-0.0.1.tar.gz", + requirement_line = "foo==0.0.3", + sha256 = "5d15t", + target_platforms = ["cp39_linux_x86_64", "cp39_osx_x86_64"], + url = "sdist", + yanked = False, + ), + struct( + distribution = "foo", + extra_pip_args = [], + filename = "foo-0.0.1-py3-none-macosx_14_0_x86_64.whl", + requirement_line = "foo==0.0.3", + sha256 = "deadb11f", + target_platforms = ["cp39_osx_x86_64"], + url = "super2", + yanked = False, + ), + ], + ), + ]) + +_tests.append(_test_overlapping_shas_with_index_results) + def parse_requirements_test_suite(name): """Create the test suite. From 9429ae6446935059e79047654d3fe53d60aadc31 Mon Sep 17 00:00:00 2001 From: Mike Toldov Date: Tue, 3 Jun 2025 09:13:19 +0200 Subject: [PATCH 269/922] fix(pypi): inherit proxy env variables in compile_pip_requirements test (#2941) Bazel does not pass environment variables implicitly (even running test outside of sandbox). This forces compile_pip_requirements test to fail with timeout when attempting to run it behind the proxy. Also changes test_command in dependency_resolver string helper to use dot instead of underscore following deprecation notice --- CHANGELOG.md | 1 + .../pypi/dependency_resolver/dependency_resolver.py | 2 +- python/private/pypi/pip_compile.bzl | 8 +++++++- 3 files changed, 9 insertions(+), 2 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index c9668c507f..e48e3d4f3d 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -98,6 +98,7 @@ END_UNRELEASED_TEMPLATE * (py_test, py_binary) Allow external files to be used for main * (pypi) Correctly aggregate the sources when the hashes specified in the lockfile differ by platform even though the same version is used. Fixes [#2648](https://github.com/bazel-contrib/rules_python/issues/2648). +* (pypi) `compile_pip_requirements` test rule works behind the proxy {#v0-0-0-added} ### Added diff --git a/python/private/pypi/dependency_resolver/dependency_resolver.py b/python/private/pypi/dependency_resolver/dependency_resolver.py index a42821c458..f3a339f929 100644 --- a/python/private/pypi/dependency_resolver/dependency_resolver.py +++ b/python/private/pypi/dependency_resolver/dependency_resolver.py @@ -165,7 +165,7 @@ def main( update_command = ( os.getenv("CUSTOM_COMPILE_COMMAND") or f"bazel run {target_label_prefix}.update" ) - test_command = f"bazel test {target_label_prefix}_test" + test_command = f"bazel test {target_label_prefix}.test" os.environ["CUSTOM_COMPILE_COMMAND"] = update_command os.environ["PIP_CONFIG_FILE"] = os.getenv("PIP_CONFIG_FILE") or os.devnull diff --git a/python/private/pypi/pip_compile.bzl b/python/private/pypi/pip_compile.bzl index c9899503d6..78b681b4ad 100644 --- a/python/private/pypi/pip_compile.bzl +++ b/python/private/pypi/pip_compile.bzl @@ -45,7 +45,6 @@ def pip_compile( By default this rules generates a filegroup named "[name]" which can be included in the data of some other compile_pip_requirements rule that references these requirements (e.g. with `-r ../other/requirements.txt`). - It also generates two targets for running pip-compile: - validate with `bazel test [name].test` @@ -160,6 +159,12 @@ def pip_compile( } env = kwargs.pop("env", {}) + env_inherit = kwargs.pop("env_inherit", []) + proxy_variables = ["https_proxy", "http_proxy", "no_proxy", "HTTPS_PROXY", "HTTP_PROXY", "NO_PROXY"] + + for var in proxy_variables: + if var not in env_inherit: + env_inherit.append(var) py_binary( name = name + ".update", @@ -182,6 +187,7 @@ def pip_compile( "@@platforms//os:windows": {"USERPROFILE": "Z:\\FakeSetuptoolsHomeDirectoryHack"}, "//conditions:default": {}, }) | env, + env_inherit = env_inherit, # kwargs could contain test-specific attributes like size **dict(attrs, **kwargs) ) From 049866442fee7bb54fcb1a09e920953a0666e4b3 Mon Sep 17 00:00:00 2001 From: Kayce Basques Date: Thu, 5 Jun 2025 14:29:07 -0700 Subject: [PATCH 270/922] feat: add persistent worker for sphinxdocs (#2938) This implements a simple, serialized persistent worker for Sphinxdocs with several optimizations. It is enabled by default. * The worker computes what inputs have changed, allowing Sphinx to only rebuild what is necessary. * Doctrees are written to a separate directory so they are retained between builds. * The worker tells Sphinx to write output to an internal directory, then copies it to the expected Bazel output directory afterwards. This allows Sphinx to only write output files that need to be updated. This works by having the worker compute what files have changed and having a Sphinx extension use the `get-env-outdated` event to tell Sphinx which files have changed. The extension is based on https://pwrev.dev/294057, but re-implemented to be in-memory as part of the worker instead of a separate extension projects must configure. For rules_python's doc building, this reduces incremental building from about 8 seconds to about 0.8 seconds. From what I can tell, about half the time is spent generating doctrees, and the other half generating the output files. Worker mode is enabled by default and can be disabled on the target or by adjusting the Bazel flags controlling execution strategy. Docs added to explain how. Because `--doctree-dir` is now always specified and outside the output dir, non-worker invocations can benefit, too, if run without sandboxing. Docs added to explain how to do this. Along the way: * Remove `--write-all` and `--fresh-env` from run args. This lets direct invocations benefit from the normal caching Sphinx does. * Change the args formatting to `--foo=bar` so they are a single element; just a bit nicer to see when debugging. Work towards https://github.com/bazel-contrib/rules_python/issues/2878, https://github.com/bazel-contrib/rules_python/issues/2879 --------- Co-authored-by: Kayce Basques Co-authored-by: Richard Levasseur --- sphinxdocs/docs/index.md | 23 +++ sphinxdocs/private/sphinx.bzl | 55 +++++-- sphinxdocs/private/sphinx_build.py | 231 ++++++++++++++++++++++++++- sphinxdocs/tests/sphinx_docs/doc1.md | 3 + sphinxdocs/tests/sphinx_docs/doc2.md | 3 + 5 files changed, 302 insertions(+), 13 deletions(-) create mode 100644 sphinxdocs/tests/sphinx_docs/doc1.md create mode 100644 sphinxdocs/tests/sphinx_docs/doc2.md diff --git a/sphinxdocs/docs/index.md b/sphinxdocs/docs/index.md index bd6448ced9..2ea1146e1b 100644 --- a/sphinxdocs/docs/index.md +++ b/sphinxdocs/docs/index.md @@ -11,6 +11,29 @@ documentation. It comes with: While it is primarily oriented towards docgen for Starlark code, the core of it is agnostic as to what is being documented. +### Optimization + +Normally, Sphinx keeps various cache files to improve incremental building. +Unfortunately, programs performing their own caching don't interact well +with Bazel's model of precisely declaring and strictly enforcing what are +inputs, what are outputs, and what files are available when running a program. +The net effect is programs don't have a prior invocation's cache files +available. + +There are two mechanisms available to make some cache available to Sphinx under +Bazel: + +* Disable sandboxing, which allows some files from prior invocations to be + visible to subsequent invocations. This can be done multiple ways: + * Set `tags = ["no-sandbox"]` on the `sphinx_docs` target + * `--modify_execution_info=SphinxBuildDocs=+no-sandbox` (Bazel flag) + * `--strategy=SphinxBuildDocs=local` (Bazel flag) +* Use persistent workers (enabled by default) by setting + `allow_persistent_workers=True` on the `sphinx_docs` target. Note that other + Bazel flags can disable using workers even if an action supports it. Setting + `--strategy=SphinxBuildDocs=dynamic,worker,local,sandbox` should tell Bazel + to use workers if possible, otherwise fallback to non-worker invocations. + ```{toctree} :hidden: diff --git a/sphinxdocs/private/sphinx.bzl b/sphinxdocs/private/sphinx.bzl index 8d19d87052..ee6b994e2e 100644 --- a/sphinxdocs/private/sphinx.bzl +++ b/sphinxdocs/private/sphinx.bzl @@ -103,6 +103,7 @@ def sphinx_docs( strip_prefix = "", extra_opts = [], tools = [], + allow_persistent_workers = True, **kwargs): """Generate docs using Sphinx. @@ -142,6 +143,9 @@ def sphinx_docs( tools: {type}`list[label]` Additional tools that are used by Sphinx and its plugins. This just makes the tools available during Sphinx execution. To locate them, use {obj}`extra_opts` and `$(location)`. + allow_persistent_workers: {type}`bool` (experimental) If true, allow + using persistent workers for running Sphinx, if Bazel decides to do so. + This can improve incremental building of docs. **kwargs: {type}`dict` Common attributes to pass onto rules. """ add_tag(kwargs, "@rules_python//sphinxdocs:sphinx_docs") @@ -165,6 +169,7 @@ def sphinx_docs( source_tree = internal_name + "/_sources", extra_opts = extra_opts, tools = tools, + allow_persistent_workers = allow_persistent_workers, **kwargs ) @@ -209,6 +214,7 @@ def _sphinx_docs_impl(ctx): source_path = source_dir_path, output_prefix = paths.join(ctx.label.name, "_build"), inputs = inputs, + allow_persistent_workers = ctx.attr.allow_persistent_workers, ) outputs[format] = output_dir per_format_args[format] = args_env @@ -229,6 +235,10 @@ def _sphinx_docs_impl(ctx): _sphinx_docs = rule( implementation = _sphinx_docs_impl, attrs = { + "allow_persistent_workers": attr.bool( + doc = "(experimental) Whether to invoke Sphinx as a persistent worker.", + default = False, + ), "extra_opts": attr.string_list( doc = "Additional options to pass onto Sphinx. These are added after " + "other options, but before the source/output args.", @@ -254,16 +264,27 @@ _sphinx_docs = rule( }, ) -def _run_sphinx(ctx, format, source_path, inputs, output_prefix): +def _run_sphinx(ctx, format, source_path, inputs, output_prefix, allow_persistent_workers): output_dir = ctx.actions.declare_directory(paths.join(output_prefix, format)) run_args = [] # Copy of the args to forward along to debug runner args = ctx.actions.args() # Args passed to the action + # An args file is required for persistent workers, but we don't know if + # the action will use worker mode or not (settings we can't see may + # force non-worker mode). For consistency, always use a params file. + args.use_param_file("@%s", use_always = True) + args.set_param_file_format("multiline") + + # NOTE: sphinx_build.py relies on the first two args being the srcdir and + # outputdir, in that order. + args.add(source_path) + args.add(output_dir.path) + args.add("--show-traceback") # Full tracebacks on error run_args.append("--show-traceback") - args.add("--builder", format) - run_args.extend(("--builder", format)) + args.add(format, format = "--builder=%s") + run_args.append("--builder={}".format(format)) if ctx.attr._quiet_flag[BuildSettingInfo].value: # Not added to run_args because run_args is for debugging @@ -271,11 +292,17 @@ def _run_sphinx(ctx, format, source_path, inputs, output_prefix): # Build in parallel, if possible # Don't add to run_args: parallel building breaks interactive debugging - args.add("--jobs", "auto") - args.add("--fresh-env") # Don't try to use cache files. Bazel can't make use of them. - run_args.append("--fresh-env") - args.add("--write-all") # Write all files; don't try to detect "changed" files - run_args.append("--write-all") + args.add("--jobs=auto") + + # Put the doctree dir outside of the output directory. + # This allows it to be reused between invocations when possible; Bazel + # clears the output directory every action invocation. + # * For workers, they can fully re-use it. + # * For non-workers, it can be reused when sandboxing is disabled via + # the `no-sandbox` tag or execution requirement. + # + # We also use a non-dot prefixed name so it shows up more visibly. + args.add(paths.join(output_dir.path + "_doctrees"), format = "--doctree-dir=%s") for opt in ctx.attr.extra_opts: expanded = ctx.expand_location(opt) @@ -287,9 +314,6 @@ def _run_sphinx(ctx, format, source_path, inputs, output_prefix): for define in extra_defines: run_args.extend(("--define", define)) - args.add(source_path) - args.add(output_dir.path) - env = dict([ v.split("=", 1) for v in ctx.attr._extra_env_flag[_FlagInfo].value @@ -299,6 +323,14 @@ def _run_sphinx(ctx, format, source_path, inputs, output_prefix): for tool in ctx.attr.tools: tools.append(tool[DefaultInfo].files_to_run) + # NOTE: Command line flags or RBE capabilities may override the execution + # requirements and disable workers. Thus, we can't assume that these + # exec requirements will actually be respected. + execution_requirements = {} + if allow_persistent_workers: + execution_requirements["supports-workers"] = "1" + execution_requirements["requires-worker-protocol"] = "json" + ctx.actions.run( executable = ctx.executable.sphinx, arguments = [args], @@ -308,6 +340,7 @@ def _run_sphinx(ctx, format, source_path, inputs, output_prefix): mnemonic = "SphinxBuildDocs", progress_message = "Sphinx building {} for %{{label}}".format(format), env = env, + execution_requirements = execution_requirements, ) return output_dir, struct(args = run_args, env = env) diff --git a/sphinxdocs/private/sphinx_build.py b/sphinxdocs/private/sphinx_build.py index 3b7b32eaf6..e9711042f6 100644 --- a/sphinxdocs/private/sphinx_build.py +++ b/sphinxdocs/private/sphinx_build.py @@ -1,8 +1,235 @@ +import contextlib +import io +import json +import logging import os -import pathlib +import shutil import sys +import traceback +import typing +import sphinx.application from sphinx.cmd.build import main +WorkRequest = object +WorkResponse = object + +logger = logging.getLogger("sphinxdocs_build") + +_WORKER_SPHINX_EXT_MODULE_NAME = "bazel_worker_sphinx_ext" + +# Config value name for getting the path to the request info file +_REQUEST_INFO_CONFIG_NAME = "bazel_worker_request_info_path" + + +class Worker: + + def __init__( + self, instream: "typing.TextIO", outstream: "typing.TextIO", exec_root: str + ): + # NOTE: Sphinx performs its own logging re-configuration, so any + # logging config we do isn't respected by Sphinx. Controlling where + # stdout and stderr goes are the main mechanisms. Recall that + # Bazel send worker stderr to the worker log file. + # outputBase=$(bazel info output_base) + # find $outputBase/bazel-workers/ -type f -printf '%T@ %p\n' | sort -n | tail -1 | awk '{print $2}' + logging.basicConfig(level=logging.WARN) + logger.info("Initializing worker") + + # The directory that paths are relative to. + self._exec_root = exec_root + # Where requests are read from. + self._instream = instream + # Where responses are written to. + self._outstream = outstream + + # dict[str srcdir, dict[str path, str digest]] + self._digests = {} + + # Internal output directories the worker gives to Sphinx that need + # to be cleaned up upon exit. + # set[str path] + self._worker_outdirs = set() + self._extension = BazelWorkerExtension() + + sys.modules[_WORKER_SPHINX_EXT_MODULE_NAME] = self._extension + sphinx.application.builtin_extensions += (_WORKER_SPHINX_EXT_MODULE_NAME,) + + def __enter__(self): + return self + + def __exit__(self): + for worker_outdir in self._worker_outdirs: + shutil.rmtree(worker_outdir, ignore_errors=True) + + def run(self) -> None: + logger.info("Worker started") + try: + while True: + request = None + try: + request = self._get_next_request() + if request is None: + logger.info("Empty request: exiting") + break + response = self._process_request(request) + if response: + self._send_response(response) + except Exception: + logger.exception("Unhandled error: request=%s", request) + output = ( + f"Unhandled error:\nRequest id: {request.get('id')}\n" + + traceback.format_exc() + ) + request_id = 0 if not request else request.get("requestId", 0) + self._send_response( + { + "exitCode": 3, + "output": output, + "requestId": request_id, + } + ) + finally: + logger.info("Worker shutting down") + + def _get_next_request(self) -> "object | None": + line = self._instream.readline() + if not line: + return None + return json.loads(line) + + def _send_response(self, response: "WorkResponse") -> None: + self._outstream.write(json.dumps(response) + "\n") + self._outstream.flush() + + def _prepare_sphinx(self, request): + sphinx_args = request["arguments"] + srcdir = sphinx_args[0] + + incoming_digests = {} + current_digests = self._digests.setdefault(srcdir, {}) + changed_paths = [] + request_info = {"exec_root": self._exec_root, "inputs": request["inputs"]} + for entry in request["inputs"]: + path = entry["path"] + digest = entry["digest"] + # Make the path srcdir-relative so Sphinx understands it. + path = path.removeprefix(srcdir + "/") + incoming_digests[path] = digest + + if path not in current_digests: + logger.info("path %s new", path) + changed_paths.append(path) + elif current_digests[path] != digest: + logger.info("path %s changed", path) + changed_paths.append(path) + + self._digests[srcdir] = incoming_digests + self._extension.changed_paths = changed_paths + request_info["changed_sources"] = changed_paths + + bazel_outdir = sphinx_args[1] + worker_outdir = bazel_outdir + ".worker-out.d" + self._worker_outdirs.add(worker_outdir) + sphinx_args[1] = worker_outdir + + request_info_path = os.path.join(srcdir, "_bazel_worker_request_info.json") + with open(request_info_path, "w") as fp: + json.dump(request_info, fp) + sphinx_args.append(f"--define={_REQUEST_INFO_CONFIG_NAME}={request_info_path}") + + return worker_outdir, bazel_outdir, sphinx_args + + @contextlib.contextmanager + def _redirect_streams(self): + out = io.StringIO() + orig_stdout = sys.stdout + try: + sys.stdout = out + yield out + finally: + sys.stdout = orig_stdout + + def _process_request(self, request: "WorkRequest") -> "WorkResponse | None": + logger.info("Request: %s", json.dumps(request, sort_keys=True, indent=2)) + if request.get("cancel"): + return None + + worker_outdir, bazel_outdir, sphinx_args = self._prepare_sphinx(request) + + # Prevent anything from going to stdout because it breaks the worker + # protocol. We have limited control over where Sphinx sends output. + with self._redirect_streams() as stdout: + logger.info("main args: %s", sphinx_args) + exit_code = main(sphinx_args) + + if exit_code: + raise Exception( + "Sphinx main() returned failure: " + + f" exit code: {exit_code}\n" + + "========== STDOUT START ==========\n" + + stdout.getvalue().rstrip("\n") + + "\n" + + "========== STDOUT END ==========\n" + ) + + # Copying is unfortunately necessary because Bazel doesn't know to + # implicily bring along what the symlinks point to. + shutil.copytree(worker_outdir, bazel_outdir, dirs_exist_ok=True) + + response = { + "requestId": request.get("requestId", 0), + "output": stdout.getvalue(), + "exitCode": 0, + } + return response + + +class BazelWorkerExtension: + """A Sphinx extension implemented as a class acting like a module.""" + + def __init__(self): + # Make it look like a Module object + self.__name__ = _WORKER_SPHINX_EXT_MODULE_NAME + # set[str] of src-dir relative path names + self.changed_paths = set() + + def setup(self, app): + app.add_config_value(_REQUEST_INFO_CONFIG_NAME, "", "") + app.connect("env-get-outdated", self._handle_env_get_outdated) + return {"parallel_read_safe": True, "parallel_write_safe": True} + + def _handle_env_get_outdated(self, app, env, added, changed, removed): + changed = { + # NOTE: path2doc returns None if it's not a doc path + env.path2doc(p) + for p in self.changed_paths + } + + logger.info("changed docs: %s", changed) + return changed + + +def _worker_main(stdin, stdout, exec_root): + with Worker(stdin, stdout, exec_root) as worker: + return worker.run() + + +def _non_worker_main(): + args = [] + for arg in sys.argv: + if arg.startswith("@"): + with open(arg.removeprefix("@")) as fp: + lines = [line.strip() for line in fp if line.strip()] + args.extend(lines) + else: + args.append(arg) + sys.argv[:] = args + return main() + + if __name__ == "__main__": - sys.exit(main()) + if "--persistent_worker" in sys.argv: + sys.exit(_worker_main(sys.stdin, sys.stdout, os.getcwd())) + else: + sys.exit(_non_worker_main()) diff --git a/sphinxdocs/tests/sphinx_docs/doc1.md b/sphinxdocs/tests/sphinx_docs/doc1.md new file mode 100644 index 0000000000..f6f70ba28c --- /dev/null +++ b/sphinxdocs/tests/sphinx_docs/doc1.md @@ -0,0 +1,3 @@ +# doc1 + +hello doc 1 diff --git a/sphinxdocs/tests/sphinx_docs/doc2.md b/sphinxdocs/tests/sphinx_docs/doc2.md new file mode 100644 index 0000000000..06eb76a596 --- /dev/null +++ b/sphinxdocs/tests/sphinx_docs/doc2.md @@ -0,0 +1,3 @@ +# doc 2 + +hello doc 3 From d98547e8ec6bdbf4f250dd01c2921c2d91dc6db6 Mon Sep 17 00:00:00 2001 From: Aaron Levy Date: Tue, 10 Jun 2025 01:20:22 -0700 Subject: [PATCH 271/922] fix: Updating setuptools to patch CVE-2025-47273 (#2955) Update setuptools to patch CVE-2025-47273 --- CHANGELOG.md | 1 + python/private/pypi/deps.bzl | 4 ++-- 2 files changed, 3 insertions(+), 2 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index e48e3d4f3d..eeafc70bae 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -72,6 +72,7 @@ END_UNRELEASED_TEMPLATE * (py_wheel) py_wheel always creates zip64-capable wheel zips * (providers) (experimental) {obj}`PyInfo.venv_symlinks` replaces `PyInfo.site_packages_symlinks` +* (deps) Updating setuptools to patch CVE-2025-47273. {#v0-0-0-fixed} ### Fixed diff --git a/python/private/pypi/deps.bzl b/python/private/pypi/deps.bzl index 31a5201659..73b30c69ee 100644 --- a/python/private/pypi/deps.bzl +++ b/python/private/pypi/deps.bzl @@ -76,8 +76,8 @@ _RULE_DEPS = [ ), ( "pypi__setuptools", - "https://files.pythonhosted.org/packages/de/88/70c5767a0e43eb4451c2200f07d042a4bcd7639276003a9c54a68cfcc1f8/setuptools-70.0.0-py3-none-any.whl", - "54faa7f2e8d2d11bcd2c07bed282eef1046b5c080d1c32add737d7b5817b1ad4", + "https://files.pythonhosted.org/packages/90/99/158ad0609729111163fc1f674a5a42f2605371a4cf036d0441070e2f7455/setuptools-78.1.1-py3-none-any.whl", + "c3a9c4211ff4c309edb8b8c4f1cbfa7ae324c4ba9f91ff254e3d305b9fd54561", ), ( "pypi__tomli", From 013acd944643dfc939639cdc6e8b49ef685ed314 Mon Sep 17 00:00:00 2001 From: Ignas Anikevicius <240938+aignas@users.noreply.github.com> Date: Tue, 10 Jun 2025 19:58:08 +0900 Subject: [PATCH 272/922] feat: data and pyi files in the venv (#2936) This adds the remaining of the files into the venv and should get us reasonably close to handling 99% of the cases. The expected differences from this and a `venv` built by `uv` would be: * The `RECORD` files are excluded from the `venv`s for better cache hit rate in `bazel`. Topological ordering is removed because topo ordering doesn't provide the "closer target first" guarantees desired. For now, just use default ordering and document conflicts as undefined behavior. Internally, it continues to use first-wins (i.e. first in depset.to_list() order) semantics. Work towards #2156 --- .bazelrc | 4 +- MODULE.bazel | 6 + internal_dev_deps.bzl | 5 + python/private/BUILD.bazel | 2 + python/private/attributes.bzl | 2 +- python/private/common.bzl | 7 +- python/private/py_executable.bzl | 65 +++++---- python/private/py_info.bzl | 38 ++--- python/private/py_library.bzl | 137 ++++++++++++------ tests/modules/another_module/BUILD.bazel | 5 + tests/modules/another_module/MODULE.bazel | 1 + .../another_module/another_module_data.txt | 1 + tests/modules/other/MODULE.bazel | 2 + tests/modules/other/simple_v1/BUILD.bazel | 14 ++ .../simple-1.0.0.dist-info/METADATA | 1 + .../site-packages/simple/__init__.py | 1 + .../site-packages/simple_v1_extras/data.txt | 0 tests/modules/other/simple_v2/BUILD.bazel | 15 ++ .../simple-2.0.0.dist-info/METADATA | 1 + .../simple-2.0.0.dist-info/licenses/LICENSE | 1 + .../site-packages/simple.libs/data.so | 2 + .../site-packages/simple/__init__.py | 1 + .../site-packages/simple/__init__.pyi | 1 + .../other/with_external_data/BUILD.bazel | 23 +++ .../site-packages/with_external_data.py | 1 + tests/venv_site_packages_libs/BUILD.bazel | 16 ++ tests/venv_site_packages_libs/bin.py | 49 ++++++- 27 files changed, 296 insertions(+), 105 deletions(-) create mode 100644 tests/modules/another_module/BUILD.bazel create mode 100644 tests/modules/another_module/MODULE.bazel create mode 100644 tests/modules/another_module/another_module_data.txt create mode 100644 tests/modules/other/simple_v1/BUILD.bazel create mode 100644 tests/modules/other/simple_v1/site-packages/simple-1.0.0.dist-info/METADATA create mode 100644 tests/modules/other/simple_v1/site-packages/simple/__init__.py create mode 100644 tests/modules/other/simple_v1/site-packages/simple_v1_extras/data.txt create mode 100644 tests/modules/other/simple_v2/BUILD.bazel create mode 100644 tests/modules/other/simple_v2/site-packages/simple-2.0.0.dist-info/METADATA create mode 100644 tests/modules/other/simple_v2/site-packages/simple-2.0.0.dist-info/licenses/LICENSE create mode 100644 tests/modules/other/simple_v2/site-packages/simple.libs/data.so create mode 100644 tests/modules/other/simple_v2/site-packages/simple/__init__.py create mode 100644 tests/modules/other/simple_v2/site-packages/simple/__init__.pyi create mode 100644 tests/modules/other/with_external_data/BUILD.bazel create mode 100644 tests/modules/other/with_external_data/site-packages/with_external_data.py diff --git a/.bazelrc b/.bazelrc index 7e744fb67a..f7f31aed98 100644 --- a/.bazelrc +++ b/.bazelrc @@ -4,8 +4,8 @@ # (Note, we cannot use `common --deleted_packages` because the bazel version command doesn't support it) # To update these lines, execute # `bazel run @rules_bazel_integration_test//tools:update_deleted_packages` -build --deleted_packages=examples/build_file_generation,examples/build_file_generation/random_number_generator,examples/bzlmod,examples/bzlmod_build_file_generation,examples/bzlmod_build_file_generation/other_module/other_module/pkg,examples/bzlmod_build_file_generation/runfiles,examples/bzlmod/entry_points,examples/bzlmod/entry_points/tests,examples/bzlmod/libs/my_lib,examples/bzlmod/other_module,examples/bzlmod/other_module/other_module/pkg,examples/bzlmod/patches,examples/bzlmod/py_proto_library,examples/bzlmod/py_proto_library/example.com/another_proto,examples/bzlmod/py_proto_library/example.com/proto,examples/bzlmod/runfiles,examples/bzlmod/tests,examples/bzlmod/tests/other_module,examples/bzlmod/whl_mods,examples/multi_python_versions/libs/my_lib,examples/multi_python_versions/requirements,examples/multi_python_versions/tests,examples/pip_parse,examples/pip_parse_vendored,examples/pip_repository_annotations,examples/py_proto_library,examples/py_proto_library/example.com/another_proto,examples/py_proto_library/example.com/proto,gazelle,gazelle/manifest,gazelle/manifest/generate,gazelle/manifest/hasher,gazelle/manifest/test,gazelle/modules_mapping,gazelle/python,gazelle/pythonconfig,gazelle/python/private,tests/integration/compile_pip_requirements,tests/integration/compile_pip_requirements_test_from_external_repo,tests/integration/custom_commands,tests/integration/ignore_root_user_error,tests/integration/ignore_root_user_error/submodule,tests/integration/local_toolchains,tests/integration/pip_parse,tests/integration/pip_parse/empty,tests/integration/py_cc_toolchain_registered,tests/modules/other,tests/modules/other/nspkg_delta,tests/modules/other/nspkg_gamma,tests/modules/other/nspkg_single -query --deleted_packages=examples/build_file_generation,examples/build_file_generation/random_number_generator,examples/bzlmod,examples/bzlmod_build_file_generation,examples/bzlmod_build_file_generation/other_module/other_module/pkg,examples/bzlmod_build_file_generation/runfiles,examples/bzlmod/entry_points,examples/bzlmod/entry_points/tests,examples/bzlmod/libs/my_lib,examples/bzlmod/other_module,examples/bzlmod/other_module/other_module/pkg,examples/bzlmod/patches,examples/bzlmod/py_proto_library,examples/bzlmod/py_proto_library/example.com/another_proto,examples/bzlmod/py_proto_library/example.com/proto,examples/bzlmod/runfiles,examples/bzlmod/tests,examples/bzlmod/tests/other_module,examples/bzlmod/whl_mods,examples/multi_python_versions/libs/my_lib,examples/multi_python_versions/requirements,examples/multi_python_versions/tests,examples/pip_parse,examples/pip_parse_vendored,examples/pip_repository_annotations,examples/py_proto_library,examples/py_proto_library/example.com/another_proto,examples/py_proto_library/example.com/proto,gazelle,gazelle/manifest,gazelle/manifest/generate,gazelle/manifest/hasher,gazelle/manifest/test,gazelle/modules_mapping,gazelle/python,gazelle/pythonconfig,gazelle/python/private,tests/integration/compile_pip_requirements,tests/integration/compile_pip_requirements_test_from_external_repo,tests/integration/custom_commands,tests/integration/ignore_root_user_error,tests/integration/ignore_root_user_error/submodule,tests/integration/local_toolchains,tests/integration/pip_parse,tests/integration/pip_parse/empty,tests/integration/py_cc_toolchain_registered,tests/modules/other,tests/modules/other/nspkg_delta,tests/modules/other/nspkg_gamma,tests/modules/other/nspkg_single +build --deleted_packages=examples/build_file_generation,examples/build_file_generation/random_number_generator,examples/bzlmod,examples/bzlmod_build_file_generation,examples/bzlmod_build_file_generation/other_module/other_module/pkg,examples/bzlmod_build_file_generation/runfiles,examples/bzlmod/entry_points,examples/bzlmod/entry_points/tests,examples/bzlmod/libs/my_lib,examples/bzlmod/other_module,examples/bzlmod/other_module/other_module/pkg,examples/bzlmod/patches,examples/bzlmod/py_proto_library,examples/bzlmod/py_proto_library/example.com/another_proto,examples/bzlmod/py_proto_library/example.com/proto,examples/bzlmod/runfiles,examples/bzlmod/tests,examples/bzlmod/tests/other_module,examples/bzlmod/whl_mods,examples/multi_python_versions/libs/my_lib,examples/multi_python_versions/requirements,examples/multi_python_versions/tests,examples/pip_parse,examples/pip_parse_vendored,examples/pip_repository_annotations,examples/py_proto_library,examples/py_proto_library/example.com/another_proto,examples/py_proto_library/example.com/proto,gazelle,gazelle/manifest,gazelle/manifest/generate,gazelle/manifest/hasher,gazelle/manifest/test,gazelle/modules_mapping,gazelle/python,gazelle/pythonconfig,gazelle/python/private,tests/integration/compile_pip_requirements,tests/integration/compile_pip_requirements_test_from_external_repo,tests/integration/custom_commands,tests/integration/ignore_root_user_error,tests/integration/ignore_root_user_error/submodule,tests/integration/local_toolchains,tests/integration/pip_parse,tests/integration/pip_parse/empty,tests/integration/py_cc_toolchain_registered,tests/modules/another_module,tests/modules/other,tests/modules/other/nspkg_delta,tests/modules/other/nspkg_gamma,tests/modules/other/nspkg_single,tests/modules/other/simple_v1,tests/modules/other/simple_v2,tests/modules/other/with_external_data +query --deleted_packages=examples/build_file_generation,examples/build_file_generation/random_number_generator,examples/bzlmod,examples/bzlmod_build_file_generation,examples/bzlmod_build_file_generation/other_module/other_module/pkg,examples/bzlmod_build_file_generation/runfiles,examples/bzlmod/entry_points,examples/bzlmod/entry_points/tests,examples/bzlmod/libs/my_lib,examples/bzlmod/other_module,examples/bzlmod/other_module/other_module/pkg,examples/bzlmod/patches,examples/bzlmod/py_proto_library,examples/bzlmod/py_proto_library/example.com/another_proto,examples/bzlmod/py_proto_library/example.com/proto,examples/bzlmod/runfiles,examples/bzlmod/tests,examples/bzlmod/tests/other_module,examples/bzlmod/whl_mods,examples/multi_python_versions/libs/my_lib,examples/multi_python_versions/requirements,examples/multi_python_versions/tests,examples/pip_parse,examples/pip_parse_vendored,examples/pip_repository_annotations,examples/py_proto_library,examples/py_proto_library/example.com/another_proto,examples/py_proto_library/example.com/proto,gazelle,gazelle/manifest,gazelle/manifest/generate,gazelle/manifest/hasher,gazelle/manifest/test,gazelle/modules_mapping,gazelle/python,gazelle/pythonconfig,gazelle/python/private,tests/integration/compile_pip_requirements,tests/integration/compile_pip_requirements_test_from_external_repo,tests/integration/custom_commands,tests/integration/ignore_root_user_error,tests/integration/ignore_root_user_error/submodule,tests/integration/local_toolchains,tests/integration/pip_parse,tests/integration/pip_parse/empty,tests/integration/py_cc_toolchain_registered,tests/modules/another_module,tests/modules/other,tests/modules/other/nspkg_delta,tests/modules/other/nspkg_gamma,tests/modules/other/nspkg_single,tests/modules/other/simple_v1,tests/modules/other/simple_v2,tests/modules/other/with_external_data test --test_output=errors diff --git a/MODULE.bazel b/MODULE.bazel index 144e130c1b..77fa12d113 100644 --- a/MODULE.bazel +++ b/MODULE.bazel @@ -86,6 +86,7 @@ bazel_dep(name = "rules_multirun", version = "0.9.0", dev_dependency = True) bazel_dep(name = "bazel_ci_rules", version = "1.0.0", dev_dependency = True) bazel_dep(name = "rules_pkg", version = "1.0.1", dev_dependency = True) bazel_dep(name = "other", version = "0", dev_dependency = True) +bazel_dep(name = "another_module", version = "0", dev_dependency = True) # Extra gazelle plugin deps so that WORKSPACE.bzlmod can continue including it for e2e tests. # We use `WORKSPACE.bzlmod` because it is impossible to have dev-only local overrides. @@ -116,6 +117,11 @@ local_path_override( path = "tests/modules/other", ) +local_path_override( + module_name = "another_module", + path = "tests/modules/another_module", +) + dev_python = use_extension( "//python/extensions:python.bzl", "python", diff --git a/internal_dev_deps.bzl b/internal_dev_deps.bzl index f2b33e279e..e6ade4035c 100644 --- a/internal_dev_deps.bzl +++ b/internal_dev_deps.bzl @@ -48,6 +48,11 @@ def rules_python_internal_deps(): path = "tests/modules/other", ) + local_repository( + name = "another_module", + path = "tests/modules/another_module", + ) + http_archive( name = "bazel_skylib", sha256 = "bc283cdfcd526a52c3201279cda4bc298652efa898b10b4db0837dc51652756f", diff --git a/python/private/BUILD.bazel b/python/private/BUILD.bazel index b319919305..8bcc6eaebe 100644 --- a/python/private/BUILD.bazel +++ b/python/private/BUILD.bazel @@ -450,11 +450,13 @@ bzl_library( ":attributes_bzl", ":common_bzl", ":flags_bzl", + ":normalize_name_bzl", ":precompile_bzl", ":py_cc_link_params_info_bzl", ":py_internal_bzl", ":rule_builders_bzl", ":toolchain_types_bzl", + ":version_bzl", "@bazel_skylib//lib:dicts", "@bazel_skylib//rules:common_settings", ], diff --git a/python/private/attributes.bzl b/python/private/attributes.bzl index ad8cba2e6c..c3b1cade91 100644 --- a/python/private/attributes.bzl +++ b/python/private/attributes.bzl @@ -260,7 +260,7 @@ The order of this list can matter because it affects the order that information from dependencies is merged in, which can be relevant depending on the ordering mode of depsets that are merged. -* {obj}`PyInfo.venv_symlinks` uses topological ordering. +* {obj}`PyInfo.venv_symlinks` uses default ordering. See {obj}`PyInfo` for more information about the ordering of its depsets and how its fields are merged. diff --git a/python/private/common.bzl b/python/private/common.bzl index e49dbad20c..163fb54d77 100644 --- a/python/private/common.bzl +++ b/python/private/common.bzl @@ -331,7 +331,7 @@ def collect_runfiles(ctx, files = depset()): # If the target is a File, then add that file to the runfiles. # Otherwise, add the target's **data runfiles** to the runfiles. # - # Note that, contray to best practice, the default outputs of the + # Note that, contrary to best practice, the default outputs of the # targets in `data` are *not* added, nor are the default runfiles. # # This ends up being important for several reasons, some of which are @@ -396,9 +396,8 @@ def create_py_info( implicit_pyc_files: {type}`depset[File]` Implicitly generated pyc files that a binary can choose to include. imports: depset of strings; the import path values to propagate. - venv_symlinks: {type}`list[tuple[str, str]]` tuples of - `(runfiles_path, site_packages_path)` for symlinks to create - in the consuming binary's venv site packages. + venv_symlinks: {type}`list[VenvSymlinkEntry]` instances for + symlinks to create in the consuming binary's venv. Returns: A tuple of the PyInfo instance and a depset of the diff --git a/python/private/py_executable.bzl b/python/private/py_executable.bzl index 7c3e0cb757..7e50247e61 100644 --- a/python/private/py_executable.bzl +++ b/python/private/py_executable.bzl @@ -650,10 +650,6 @@ def _create_venv_symlinks(ctx, venv_dir_map): # maps venv-relative path to the runfiles path it should point to entries = depset( - # NOTE: Topological ordering is used so that dependencies closer to the - # binary have precedence in creating their symlinks. This allows the - # binary a modicum of control over the result. - order = "topological", transitive = [ dep[PyInfo].venv_symlinks for dep in ctx.attr.deps @@ -680,43 +676,52 @@ def _create_venv_symlinks(ctx, venv_dir_map): return venv_files def _build_link_map(entries): - # dict[str kind, dict[str rel_path, str link_to_path]] - link_map = {} + # dict[str package, dict[str kind, dict[str rel_path, str link_to_path]]] + pkg_link_map = {} + + # dict[str package, str version] + version_by_pkg = {} + for entry in entries: - kind = entry.kind - kind_map = link_map.setdefault(kind, {}) - if entry.venv_path in kind_map: - # We ignore duplicates by design. The dependency closer to the - # binary gets precedence due to the topological ordering. + link_map = pkg_link_map.setdefault(entry.package, {}) + kind_map = link_map.setdefault(entry.kind, {}) + + if version_by_pkg.setdefault(entry.package, entry.version) != entry.version: + # We ignore duplicates by design. + continue + elif entry.venv_path in kind_map: + # We ignore duplicates by design. continue else: kind_map[entry.venv_path] = entry.link_to_path - # An empty link_to value means to not create the site package symlink. - # Because of the topological ordering, this allows binaries to remove - # entries by having an earlier dependency produce empty link_to values. - for kind, kind_map in link_map.items(): - for dir_path, link_to in kind_map.items(): - if not link_to: - kind_map.pop(dir_path) + # An empty link_to value means to not create the site package symlink. Because of the + # ordering, this allows binaries to remove entries by having an earlier dependency produce + # empty link_to values. + for link_map in pkg_link_map.values(): + for kind, kind_map in link_map.items(): + for dir_path, link_to in kind_map.items(): + if not link_to: + kind_map.pop(dir_path) # dict[str kind, dict[str rel_path, str link_to_path]] keep_link_map = {} # Remove entries that would be a child path of a created symlink. # Earlier entries have precedence to match how exact matches are handled. - for kind, kind_map in link_map.items(): - keep_kind_map = keep_link_map.setdefault(kind, {}) - for _ in range(len(kind_map)): - if not kind_map: - break - dirname, value = kind_map.popitem() - keep_kind_map[dirname] = value - prefix = dirname + "/" # Add slash to prevent /X matching /XY - for maybe_suffix in kind_map.keys(): - maybe_suffix += "/" # Add slash to prevent /X matching /XY - if maybe_suffix.startswith(prefix) or prefix.startswith(maybe_suffix): - kind_map.pop(maybe_suffix) + for link_map in pkg_link_map.values(): + for kind, kind_map in link_map.items(): + keep_kind_map = keep_link_map.setdefault(kind, {}) + for _ in range(len(kind_map)): + if not kind_map: + break + dirname, value = kind_map.popitem() + keep_kind_map[dirname] = value + prefix = dirname + "/" # Add slash to prevent /X matching /XY + for maybe_suffix in kind_map.keys(): + maybe_suffix += "/" # Add slash to prevent /X matching /XY + if maybe_suffix.startswith(prefix) or prefix.startswith(maybe_suffix): + kind_map.pop(maybe_suffix) return keep_link_map def _map_each_identity(v): diff --git a/python/private/py_info.bzl b/python/private/py_info.bzl index 2a2f4554e3..17c5e4e79e 100644 --- a/python/private/py_info.bzl +++ b/python/private/py_info.bzl @@ -67,11 +67,24 @@ the venv to create the path under. A runfiles-root relative path that `venv_path` will symlink to. If `None`, it means to not create a symlink. +""", + "package": """ +:type: str | None + +Represents the PyPI package name that the code originates from. It is normalized according to the +PEP440 with all `-` replaced with `_`, i.e. the same as the package name in the hub repository that +it would come from. """, "venv_path": """ :type: str A path relative to the `kind` directory within the venv. +""", + "version": """ +:type: str | None + +Represents the PyPI package version that the code originates from. It is normalized according to the +PEP440 standard. """, }, ) @@ -296,29 +309,9 @@ This field is currently unused in Bazel and may go away in the future. "venv_symlinks": """ :type: depset[VenvSymlinkEntry] -A depset with `topological` ordering. - - -Tuples of `(runfiles_path, site_packages_path)`. Where -* `runfiles_path` is a runfiles-root relative path. It is the path that - has the code to make importable. If `None` or empty string, then it means - to not create a site packages directory with the `site_packages_path` - name. -* `site_packages_path` is a path relative to the site-packages directory of - the venv for whatever creates the venv (typically py_binary). It makes - the code in `runfiles_path` available for import. Note that this - is created as a "raw" symlink (via `declare_symlink`). - :::{include} /_includes/experimental_api.md ::: -:::{tip} -The topological ordering means dependencies earlier and closer to the consumer -have precedence. This allows e.g. a binary to add dependencies that override -values from further way dependencies, such as forcing symlinks to point to -specific paths or preventing symlinks from being created. -::: - :::{versionadded} VERSION_NEXT_FEATURE ::: """, @@ -375,9 +368,6 @@ def _PyInfoBuilder_typedef(): :::{field} venv_symlinks :type: DepsetBuilder[tuple[str | None, str]] - - NOTE: This depset has `topological` order - ::: """ def _PyInfoBuilder_new(): @@ -417,7 +407,7 @@ def _PyInfoBuilder_new(): transitive_pyc_files = builders.DepsetBuilder(), transitive_pyi_files = builders.DepsetBuilder(), transitive_sources = builders.DepsetBuilder(), - venv_symlinks = builders.DepsetBuilder(order = "topological"), + venv_symlinks = builders.DepsetBuilder(), ) return self diff --git a/python/private/py_library.bzl b/python/private/py_library.bzl index fabc880a8d..e727694b32 100644 --- a/python/private/py_library.bzl +++ b/python/private/py_library.bzl @@ -41,6 +41,7 @@ load( "runfiles_root_path", ) load(":flags.bzl", "AddSrcsToRunfilesFlag", "PrecompileFlag", "VenvsSitePackages") +load(":normalize_name.bzl", "normalize_name") load(":precompile.bzl", "maybe_precompile") load(":py_cc_link_params_info.bzl", "PyCcLinkParamsInfo") load(":py_info.bzl", "PyInfo", "VenvSymlinkEntry", "VenvSymlinkKind") @@ -52,6 +53,7 @@ load( "EXEC_TOOLS_TOOLCHAIN_TYPE", TOOLCHAIN_TYPE = "TARGET_TOOLCHAIN_TYPE", ) +load(":version.bzl", "version") _py_builtins = py_internal @@ -84,20 +86,22 @@ under the binary's venv site-packages directory that should be made available (i namespace packages]( https://packaging.python.org/en/latest/guides/packaging-namespace-packages/#native-namespace-packages). However, the *content* of the files cannot be taken into account, merely their -presence or absense. Stated another way: [pkgutil-style namespace packages]( +presence or absence. Stated another way: [pkgutil-style namespace packages]( https://packaging.python.org/en/latest/guides/packaging-namespace-packages/#pkgutil-style-namespace-packages) won't be understood as namespace packages; they'll be seen as regular packages. This will likely lead to conflicts with other targets that contribute to the namespace. -:::{tip} -This attributes populates {obj}`PyInfo.venv_symlinks`, which is -a topologically ordered depset. This means dependencies closer and earlier -to a consumer have precedence. See {obj}`PyInfo.venv_symlinks` for -more information. +:::{seealso} +This attributes populates {obj}`PyInfo.venv_symlinks`. ::: :::{versionadded} 1.4.0 ::: +:::{versionchanged} VERSION_NEXT_FEATURE +The topological order has been removed and if 2 different versions of the same PyPI +package are observed, the behaviour has no guarantees except that it is deterministic +and that only one package version will be included. +::: """, ), "_add_srcs_to_runfiles_flag": lambda: attrb.Label( @@ -157,7 +161,8 @@ def py_library_impl(ctx, *, semantics): imports = [] venv_symlinks = [] - imports, venv_symlinks = _get_imports_and_venv_symlinks(ctx, semantics) + package, version_str = _get_package_and_version(ctx) + imports, venv_symlinks = _get_imports_and_venv_symlinks(ctx, semantics, package, version_str) cc_info = semantics.get_cc_info_for_library(ctx) py_info, deps_transitive_sources, builtins_py_info = create_py_info( @@ -206,16 +211,46 @@ Source files are no longer added to the runfiles directly. ::: """ -def _get_imports_and_venv_symlinks(ctx, semantics): +def _get_package_and_version(ctx): + """Return package name and version + + If the package comes from PyPI then it will have a `.dist-info` as part of `data`, which + allows us to get the name of the package and its version. + """ + dist_info_metadata = None + for d in ctx.files.data: + # work on case insensitive FSes + if d.basename.lower() != "metadata": + continue + + if d.dirname.endswith(".dist-info"): + dist_info_metadata = d + + if not dist_info_metadata: + return None, None + + # in order to be able to have replacements in the venv, we have to add a + # third value into the venv_symlinks, which would be the normalized + # package name. This allows us to ensure that we can replace the `dist-info` + # directories by checking if the package key is there. + dist_info_dir = paths.basename(dist_info_metadata.dirname) + package, _, _suffix = dist_info_dir.rpartition(".dist-info") + package, _, version_str = package.rpartition("-") + return ( + normalize_name(package), # will have no dashes + version.normalize(version_str), # will have no dashes either + ) + +def _get_imports_and_venv_symlinks(ctx, semantics, package, version_str): imports = depset() - venv_symlinks = depset() + venv_symlinks = [] if VenvsSitePackages.is_enabled(ctx): - venv_symlinks = _get_venv_symlinks(ctx) + venv_symlinks = _get_venv_symlinks(ctx, package, version_str) else: imports = collect_imports(ctx, semantics) return imports, venv_symlinks -def _get_venv_symlinks(ctx): +def _get_venv_symlinks(ctx, package, version_str): imports = ctx.attr.imports if len(imports) == 0: fail("When venvs_site_packages is enabled, exactly one `imports` " + @@ -236,50 +271,61 @@ def _get_venv_symlinks(ctx): # Append slash to prevent incorrectly prefix-string matches site_packages_root += "/" - # We have to build a list of (runfiles path, site-packages path) pairs of - # the files to create in the consuming binary's venv site-packages directory. - # To minimize the number of files to create, we just return the paths - # to the directories containing the code of interest. + # We have to build a list of (runfiles path, site-packages path) pairs of the files to + # create in the consuming binary's venv site-packages directory. To minimize the number of + # files to create, we just return the paths to the directories containing the code of + # interest. + # + # However, namespace packages complicate matters: multiple distributions install in the + # same directory in site-packages. This works out because they don't overlap in their + # files. Typically, they install to different directories within the namespace package + # directory. We also need to ensure that we can handle a case where the main package (e.g. + # airflow) has directories only containing data files and then namespace packages coming + # along and being next to it. # - # However, namespace packages complicate matters: multiple - # distributions install in the same directory in site-packages. This - # works out because they don't overlap in their files. Typically, they - # install to different directories within the namespace package - # directory. Namespace package directories are simply directories - # within site-packages that *don't* have an `__init__.py` file, which - # can be arbitrarily deep. Thus, we simply have to look for the - # directories that _do_ have an `__init__.py` file and treat those as - # the path to symlink to. - - repo_runfiles_dirname = None - dirs_with_init = {} # dirname -> runfile path + # Lastly we have to assume python modules just being `.py` files (e.g. typing-extensions) + # is just a single Python file. + + dir_symlinks = {} # dirname -> runfile path venv_symlinks = [] - for src in ctx.files.srcs: - if src.extension not in PYTHON_FILE_EXTENSIONS: - continue + for src in ctx.files.srcs + ctx.files.data + ctx.files.pyi_srcs: path = _repo_relative_short_path(src.short_path) if not path.startswith(site_packages_root): continue path = path.removeprefix(site_packages_root) dir_name, _, filename = path.rpartition("/") - if dir_name and filename.startswith("__init__."): - dirs_with_init[dir_name] = None - repo_runfiles_dirname = runfiles_root_path(ctx, src.short_path).partition("/")[0] - elif not dir_name: - repo_runfiles_dirname = runfiles_root_path(ctx, src.short_path).partition("/")[0] + if dir_name in dir_symlinks: + # we already have this dir, this allows us to short-circuit since most of the + # ctx.files.data might share the same directories as ctx.files.srcs + continue + runfiles_dir_name, _, _ = runfiles_root_path(ctx, src.short_path).partition("/") + if dir_name: + # This can be either: + # * a directory with libs (e.g. numpy.libs, created by auditwheel) + # * a directory with `__init__.py` file that potentially also needs to be + # symlinked. + # * `.dist-info` directory + # + # This could be also regular files, that just need to be symlinked, so we will + # add the directory here. + dir_symlinks[dir_name] = runfiles_dir_name + elif src.extension in PYTHON_FILE_EXTENSIONS: # This would be files that do not have directories and we just need to add - # direct symlinks to them as is: - venv_symlinks.append(VenvSymlinkEntry( + # direct symlinks to them as is, we only allow Python files in here + entry = VenvSymlinkEntry( kind = VenvSymlinkKind.LIB, - link_to_path = paths.join(repo_runfiles_dirname, site_packages_root, filename), + link_to_path = paths.join(runfiles_dir_name, site_packages_root, filename), + package = package, + version = version_str, venv_path = filename, - )) + ) + venv_symlinks.append(entry) # Sort so that we encounter `foo` before `foo/bar`. This ensures we # see the top-most explicit package first. - dirnames = sorted(dirs_with_init.keys()) + dirnames = sorted(dir_symlinks.keys()) first_level_explicit_packages = [] for d in dirnames: is_sub_package = False @@ -292,11 +338,16 @@ def _get_venv_symlinks(ctx): first_level_explicit_packages.append(d) for dirname in first_level_explicit_packages: - venv_symlinks.append(VenvSymlinkEntry( + prefix = dir_symlinks[dirname] + entry = VenvSymlinkEntry( kind = VenvSymlinkKind.LIB, - link_to_path = paths.join(repo_runfiles_dirname, site_packages_root, dirname), + link_to_path = paths.join(prefix, site_packages_root, dirname), + package = package, + version = version_str, venv_path = dirname, - )) + ) + venv_symlinks.append(entry) + return venv_symlinks def _repo_relative_short_path(short_path): diff --git a/tests/modules/another_module/BUILD.bazel b/tests/modules/another_module/BUILD.bazel new file mode 100644 index 0000000000..3b56b6ee83 --- /dev/null +++ b/tests/modules/another_module/BUILD.bazel @@ -0,0 +1,5 @@ +filegroup( + name = "data", + srcs = ["another_module_data.txt"], + visibility = ["//visibility:public"], +) diff --git a/tests/modules/another_module/MODULE.bazel b/tests/modules/another_module/MODULE.bazel new file mode 100644 index 0000000000..8ed5a5543b --- /dev/null +++ b/tests/modules/another_module/MODULE.bazel @@ -0,0 +1 @@ +module(name = "another_module") diff --git a/tests/modules/another_module/another_module_data.txt b/tests/modules/another_module/another_module_data.txt new file mode 100644 index 0000000000..f742ebab60 --- /dev/null +++ b/tests/modules/another_module/another_module_data.txt @@ -0,0 +1 @@ +print("token") diff --git a/tests/modules/other/MODULE.bazel b/tests/modules/other/MODULE.bazel index 7cd3118b81..11a633d56b 100644 --- a/tests/modules/other/MODULE.bazel +++ b/tests/modules/other/MODULE.bazel @@ -1,3 +1,5 @@ module(name = "other") bazel_dep(name = "rules_python", version = "0") +bazel_dep(name = "bazel_skylib", version = "1.7.1") +bazel_dep(name = "another_module", version = "0") diff --git a/tests/modules/other/simple_v1/BUILD.bazel b/tests/modules/other/simple_v1/BUILD.bazel new file mode 100644 index 0000000000..da5db8164a --- /dev/null +++ b/tests/modules/other/simple_v1/BUILD.bazel @@ -0,0 +1,14 @@ +load("@rules_python//python:py_library.bzl", "py_library") + +package(default_visibility = ["//visibility:public"]) + +py_library( + name = "simple_v1", + srcs = glob(["site-packages/**/*.py"]), + data = glob( + ["**/*"], + exclude = ["site-packages/**/*.py"], + ), + experimental_venvs_site_packages = "@rules_python//python/config_settings:venvs_site_packages", + imports = [package_name() + "/site-packages"], +) diff --git a/tests/modules/other/simple_v1/site-packages/simple-1.0.0.dist-info/METADATA b/tests/modules/other/simple_v1/site-packages/simple-1.0.0.dist-info/METADATA new file mode 100644 index 0000000000..ee76ec48a4 --- /dev/null +++ b/tests/modules/other/simple_v1/site-packages/simple-1.0.0.dist-info/METADATA @@ -0,0 +1 @@ +inside is v1 diff --git a/tests/modules/other/simple_v1/site-packages/simple/__init__.py b/tests/modules/other/simple_v1/site-packages/simple/__init__.py new file mode 100644 index 0000000000..5becc17c04 --- /dev/null +++ b/tests/modules/other/simple_v1/site-packages/simple/__init__.py @@ -0,0 +1 @@ +__version__ = "1.0.0" diff --git a/tests/modules/other/simple_v1/site-packages/simple_v1_extras/data.txt b/tests/modules/other/simple_v1/site-packages/simple_v1_extras/data.txt new file mode 100644 index 0000000000..e69de29bb2 diff --git a/tests/modules/other/simple_v2/BUILD.bazel b/tests/modules/other/simple_v2/BUILD.bazel new file mode 100644 index 0000000000..45f83a5a88 --- /dev/null +++ b/tests/modules/other/simple_v2/BUILD.bazel @@ -0,0 +1,15 @@ +load("@rules_python//python:py_library.bzl", "py_library") + +package(default_visibility = ["//visibility:public"]) + +py_library( + name = "simple_v2", + srcs = glob(["site-packages/**/*.py"]), + data = glob( + ["**/*"], + exclude = ["site-packages/**/*.py"], + ), + experimental_venvs_site_packages = "@rules_python//python/config_settings:venvs_site_packages", + imports = [package_name() + "/site-packages"], + pyi_srcs = glob(["**/*.pyi"]), +) diff --git a/tests/modules/other/simple_v2/site-packages/simple-2.0.0.dist-info/METADATA b/tests/modules/other/simple_v2/site-packages/simple-2.0.0.dist-info/METADATA new file mode 100644 index 0000000000..ee76ec48a4 --- /dev/null +++ b/tests/modules/other/simple_v2/site-packages/simple-2.0.0.dist-info/METADATA @@ -0,0 +1 @@ +inside is v1 diff --git a/tests/modules/other/simple_v2/site-packages/simple-2.0.0.dist-info/licenses/LICENSE b/tests/modules/other/simple_v2/site-packages/simple-2.0.0.dist-info/licenses/LICENSE new file mode 100644 index 0000000000..0cb5e79499 --- /dev/null +++ b/tests/modules/other/simple_v2/site-packages/simple-2.0.0.dist-info/licenses/LICENSE @@ -0,0 +1 @@ +Some License diff --git a/tests/modules/other/simple_v2/site-packages/simple.libs/data.so b/tests/modules/other/simple_v2/site-packages/simple.libs/data.so new file mode 100644 index 0000000000..f023e3b9ae --- /dev/null +++ b/tests/modules/other/simple_v2/site-packages/simple.libs/data.so @@ -0,0 +1,2 @@ +# This is usually created by auditwheel when processing linux wheels and including +# dependencies. diff --git a/tests/modules/other/simple_v2/site-packages/simple/__init__.py b/tests/modules/other/simple_v2/site-packages/simple/__init__.py new file mode 100644 index 0000000000..8c0d5d5bb2 --- /dev/null +++ b/tests/modules/other/simple_v2/site-packages/simple/__init__.py @@ -0,0 +1 @@ +__version__ = "2.0.0" diff --git a/tests/modules/other/simple_v2/site-packages/simple/__init__.pyi b/tests/modules/other/simple_v2/site-packages/simple/__init__.pyi new file mode 100644 index 0000000000..bb7b160deb --- /dev/null +++ b/tests/modules/other/simple_v2/site-packages/simple/__init__.pyi @@ -0,0 +1 @@ +# Intentionally empty diff --git a/tests/modules/other/with_external_data/BUILD.bazel b/tests/modules/other/with_external_data/BUILD.bazel new file mode 100644 index 0000000000..fc047aadab --- /dev/null +++ b/tests/modules/other/with_external_data/BUILD.bazel @@ -0,0 +1,23 @@ +load("@bazel_skylib//rules:copy_file.bzl", "copy_file") +load("@rules_python//python:py_library.bzl", "py_library") + +package(default_visibility = ["//visibility:public"]) + +# The users may include data through other repos via annotations and copy_file +# just add this edge case. +# +# NOTE: if the data is not copied to `site-packages/` then it will not +# appear. +copy_file( + name = "external_data", + src = "@another_module//:data", + out = "site-packages/external_data/another_module_data.txt", +) + +py_library( + name = "with_external_data", + srcs = ["site-packages/with_external_data.py"], + data = [":external_data"], + experimental_venvs_site_packages = "@rules_python//python/config_settings:venvs_site_packages", + imports = [package_name() + "/site-packages"], +) diff --git a/tests/modules/other/with_external_data/site-packages/with_external_data.py b/tests/modules/other/with_external_data/site-packages/with_external_data.py new file mode 100644 index 0000000000..ccd9dcef9e --- /dev/null +++ b/tests/modules/other/with_external_data/site-packages/with_external_data.py @@ -0,0 +1 @@ +# Intentionally blank diff --git a/tests/venv_site_packages_libs/BUILD.bazel b/tests/venv_site_packages_libs/BUILD.bazel index d5a4fe6750..e64299e1ad 100644 --- a/tests/venv_site_packages_libs/BUILD.bazel +++ b/tests/venv_site_packages_libs/BUILD.bazel @@ -1,6 +1,20 @@ +load("//python:py_library.bzl", "py_library") load("//tests/support:py_reconfig.bzl", "py_reconfig_test") load("//tests/support:support.bzl", "SUPPORTS_BOOTSTRAP_SCRIPT") +py_library( + name = "user_lib", + deps = ["@other//simple_v1"], +) + +py_library( + name = "closer_lib", + deps = [ + ":user_lib", + "@other//simple_v2", + ], +) + py_reconfig_test( name = "venvs_site_packages_libs_test", srcs = ["bin.py"], @@ -9,10 +23,12 @@ py_reconfig_test( target_compatible_with = SUPPORTS_BOOTSTRAP_SCRIPT, venvs_site_packages = "yes", deps = [ + ":closer_lib", "//tests/venv_site_packages_libs/nspkg_alpha", "//tests/venv_site_packages_libs/nspkg_beta", "@other//nspkg_delta", "@other//nspkg_gamma", "@other//nspkg_single", + "@other//with_external_data", ], ) diff --git a/tests/venv_site_packages_libs/bin.py b/tests/venv_site_packages_libs/bin.py index 58572a2a1e..7e5838d2c2 100644 --- a/tests/venv_site_packages_libs/bin.py +++ b/tests/venv_site_packages_libs/bin.py @@ -1,7 +1,7 @@ import importlib -import os import sys import unittest +from pathlib import Path class VenvSitePackagesLibraryTest(unittest.TestCase): @@ -27,6 +27,53 @@ def test_imported_from_venv(self): self.assert_imported_from_venv("nspkg.subnspkg.gamma") self.assert_imported_from_venv("nspkg.subnspkg.delta") self.assert_imported_from_venv("single_file") + self.assert_imported_from_venv("simple") + + def test_data_is_included(self): + self.assert_imported_from_venv("simple") + module = importlib.import_module("simple") + module_path = Path(module.__file__) + + site_packages = module_path.parent.parent + + # Ensure that packages from simple v1 are not present + files = [p.name for p in site_packages.glob("*")] + self.assertIn("simple_v1_extras", files) + + def test_override_pkg(self): + self.assert_imported_from_venv("simple") + module = importlib.import_module("simple") + self.assertEqual( + "1.0.0", + module.__version__, + ) + + def test_dirs_from_replaced_package_are_not_present(self): + self.assert_imported_from_venv("simple") + module = importlib.import_module("simple") + module_path = Path(module.__file__) + + site_packages = module_path.parent.parent + dist_info_dirs = [p.name for p in site_packages.glob("*.dist-info")] + self.assertEqual( + ["simple-1.0.0.dist-info"], + dist_info_dirs, + ) + + # Ensure that packages from simple v1 are not present + files = [p.name for p in site_packages.glob("*")] + self.assertNotIn("simple.libs", files) + + def test_data_from_another_pkg_is_included_via_copy_file(self): + self.assert_imported_from_venv("simple") + module = importlib.import_module("simple") + module_path = Path(module.__file__) + + site_packages = module_path.parent.parent + # Ensure that packages from simple v1 are not present + d = site_packages / "external_data" + files = [p.name for p in d.glob("*")] + self.assertIn("another_module_data.txt", files) if __name__ == "__main__": From cb1c382144f59f7190781fb19e090aee23536e65 Mon Sep 17 00:00:00 2001 From: Ted Kaplan Date: Tue, 10 Jun 2025 19:07:41 -0700 Subject: [PATCH 273/922] fix(pypi): Only show index_url_overrides warnings when they are needed (#2967) Fixes #2966 --- python/private/pypi/simpleapi_download.bzl | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/python/private/pypi/simpleapi_download.bzl b/python/private/pypi/simpleapi_download.bzl index 164d4e8dbd..a3ba9691cd 100644 --- a/python/private/pypi/simpleapi_download.bzl +++ b/python/private/pypi/simpleapi_download.bzl @@ -148,10 +148,11 @@ def simpleapi_download( if found_on_index[pkg] != attr.index_url } - # buildifier: disable=print - print("You can use the following `index_url_overrides` to avoid the 404 warnings:\n{}".format( - render.dict(index_url_overrides), - )) + if index_url_overrides: + # buildifier: disable=print + print("You can use the following `index_url_overrides` to avoid the 404 warnings:\n{}".format( + render.dict(index_url_overrides), + )) return contents From 95fb54a5e7146fd9c743f2984814f444798c9233 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Tue, 10 Jun 2025 22:11:52 -0700 Subject: [PATCH 274/922] revert: change default bootstrap back to system_python (#2968) Switch the default bootstrap back to system_python, per maintainer discussion. The main reason is downstream consumers are unlikely to be fully ready for the usage of raw symlinks (declare_symlink artifacts). APIs to detect them aren't available until Bazel 8, which makes it difficult for packaging rules, such as rules_pkg, bazel-lib, or tar rules. This reverts the core part of commit 9f3512fe0cc6d7229170e45724e22e64be0b8300 --- CHANGELOG.md | 8 -------- docs/api/rules_python/python/config_settings/index.md | 11 +---------- python/config_settings/BUILD.bazel | 2 +- 3 files changed, 2 insertions(+), 19 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index eeafc70bae..e8fa1751c2 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -55,14 +55,6 @@ END_UNRELEASED_TEMPLATE {#v0-0-0-changed} ### Changed -* If using the (deprecated) autodetecting/runtime_env toolchain, then the Python - version specified at build-time *must* match the Python version used at - runtime (the {obj}`--@rules_python//python/config_settings:python_version` - flag and the {attr}`python_version` attribute control the build-time version - for a target). If they don't match, dependencies won't be importable. (Such a - misconfiguration was unlikely to work to begin with; this is called out as an - FYI). -* (rules) {obj}`--bootstrap_impl=script` is the default for non-Windows. * (rules) On Windows, {obj}`--bootstrap_impl=system_python` is forced. This allows setting `--bootstrap_impl=script` in bazelrc for mixed-platform environments. diff --git a/docs/api/rules_python/python/config_settings/index.md b/docs/api/rules_python/python/config_settings/index.md index ae84d40b13..7fe25888dd 100644 --- a/docs/api/rules_python/python/config_settings/index.md +++ b/docs/api/rules_python/python/config_settings/index.md @@ -245,12 +245,8 @@ Values: ::::{bzl:flag} bootstrap_impl Determine how programs implement their startup process. -The default for this depends on the platform: -* Windows: `system_python` (**always** used) -* Other: `script` - Values: -* `system_python`: Use a bootstrap that requires a system Python available +* `system_python`: (default) Use a bootstrap that requires a system Python available in order to start programs. This requires {obj}`PyRuntimeInfo.bootstrap_template` to be a Python program. * `script`: Use a bootstrap that uses an arbitrary executable script (usually a @@ -273,11 +269,6 @@ instead. :::{versionadded} 0.33.0 ::: -:::{versionchanged} VERSION_NEXT_FEATURE -* The default for non-Windows changed from `system_python` to `script`. -* On Windows, the value is forced to `system_python`. -::: - :::: ::::{bzl:flag} current_config diff --git a/python/config_settings/BUILD.bazel b/python/config_settings/BUILD.bazel index ee15828fa5..b11580c4cb 100644 --- a/python/config_settings/BUILD.bazel +++ b/python/config_settings/BUILD.bazel @@ -90,7 +90,7 @@ string_flag( rp_string_flag( name = "bootstrap_impl", - build_setting_default = BootstrapImplFlag.SCRIPT, + build_setting_default = BootstrapImplFlag.SYSTEM_PYTHON, override = select({ # Windows doesn't yet support bootstrap=script, so force disable it ":_is_windows": BootstrapImplFlag.SYSTEM_PYTHON, From fb2298a7f2e6789186d63ef645ceed96261d94a9 Mon Sep 17 00:00:00 2001 From: Benjamin Peterson Date: Wed, 11 Jun 2025 13:55:19 -0700 Subject: [PATCH 275/922] fix: grammar in an error message (#2971) --- python/private/pypi/extension.bzl | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/python/private/pypi/extension.bzl b/python/private/pypi/extension.bzl index b79be6e038..867abe0898 100644 --- a/python/private/pypi/extension.bzl +++ b/python/private/pypi/extension.bzl @@ -263,7 +263,7 @@ def _create_whl_repos( repo_name = "{}_{}".format(pip_name, repo.repo_name) if repo_name in whl_libraries: - fail("Attempting to creating a duplicate library {} for {}".format( + fail("attempting to create a duplicate library {} for {}".format( repo_name, whl.name, )) From e03b63c725cbef77a5c9af254331086de4649e15 Mon Sep 17 00:00:00 2001 From: Keith Smiley Date: Wed, 11 Jun 2025 15:09:44 -0700 Subject: [PATCH 276/922] refactor: Add missing uses of DefaultInfo (#2972) Required for compatibility with https://github.com/bazelbuild/bazel/issues/20183 --- python/private/common.bzl | 4 ++-- python/private/py_wheel.bzl | 4 ++-- python/uv/private/uv_toolchain.bzl | 2 +- sphinxdocs/private/sphinx.bzl | 2 +- 4 files changed, 6 insertions(+), 6 deletions(-) diff --git a/python/private/common.bzl b/python/private/common.bzl index 163fb54d77..96f8ebeab4 100644 --- a/python/private/common.bzl +++ b/python/private/common.bzl @@ -425,7 +425,7 @@ def create_py_info( else: # TODO(b/228692666): Remove this once non-PyInfo targets are no # longer supported in `deps`. - files = target.files.to_list() + files = target[DefaultInfo].files.to_list() for f in files: if f.extension == "py": py_info.transitive_sources.add(f) @@ -449,7 +449,7 @@ def create_py_info( info = _get_py_info(target) py_info.merge_uses_shared_libraries(info.uses_shared_libraries) else: - files = target.files.to_list() + files = target[DefaultInfo].files.to_list() for f in files: py_info.merge_uses_shared_libraries(cc_helper.is_valid_shared_library_artifact(f)) if py_info.get_uses_shared_libraries(): diff --git a/python/private/py_wheel.bzl b/python/private/py_wheel.bzl index ffc24f6846..cfd4efdcda 100644 --- a/python/private/py_wheel.bzl +++ b/python/private/py_wheel.bzl @@ -480,7 +480,7 @@ def _py_wheel_impl(ctx): args.add("--no_compress") for target, filename in ctx.attr.extra_distinfo_files.items(): - target_files = target.files.to_list() + target_files = target[DefaultInfo].files.to_list() if len(target_files) != 1: fail( "Multi-file target listed in extra_distinfo_files %s", @@ -493,7 +493,7 @@ def _py_wheel_impl(ctx): ) for target, filename in ctx.attr.data_files.items(): - target_files = target.files.to_list() + target_files = target[DefaultInfo].files.to_list() if len(target_files) != 1: fail( "Multi-file target listed in data_files %s", diff --git a/python/uv/private/uv_toolchain.bzl b/python/uv/private/uv_toolchain.bzl index 8c7f1b4b8c..bd82e7452f 100644 --- a/python/uv/private/uv_toolchain.bzl +++ b/python/uv/private/uv_toolchain.bzl @@ -24,7 +24,7 @@ def _uv_toolchain_impl(ctx): uv = ctx.attr.uv default_info = DefaultInfo( - files = uv.files, + files = uv[DefaultInfo].files, runfiles = uv[DefaultInfo].default_runfiles, ) uv_toolchain_info = UvToolchainInfo( diff --git a/sphinxdocs/private/sphinx.bzl b/sphinxdocs/private/sphinx.bzl index ee6b994e2e..c1efda3508 100644 --- a/sphinxdocs/private/sphinx.bzl +++ b/sphinxdocs/private/sphinx.bzl @@ -386,7 +386,7 @@ def _sphinx_source_tree_impl(ctx): _relocate(orig_file) for src_target, dest in ctx.attr.renamed_srcs.items(): - src_files = src_target.files.to_list() + src_files = src_target[DefaultInfo].files.to_list() if len(src_files) != 1: fail("A single file must be specified to be renamed. Target {} " + "generate {} files: {}".format( From 108a66cefe3206ba1a15eac4b9dcc586b649aa0b Mon Sep 17 00:00:00 2001 From: honglooker Date: Wed, 11 Jun 2025 18:11:14 -0400 Subject: [PATCH 277/922] docs: fix typo in toolchains.md example code (#2970) Added missing commas in `local toolchains` instructions --- docs/toolchains.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/toolchains.md b/docs/toolchains.md index 57d43d27f1..be85a2471e 100644 --- a/docs/toolchains.md +++ b/docs/toolchains.md @@ -436,8 +436,8 @@ local_runtime_repo = use_repo_rule( ) local_runtime_toolchains_repo = use_repo_rule( - "@rules_python//python/local_toolchains:repos.bzl" - "local_runtime_toolchains_repo" + "@rules_python//python/local_toolchains:repos.bzl", + "local_runtime_toolchains_repo", dev_dependency = True, ) From ef14ae2143a3707da1b1c865a7b451b154df5353 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Wed, 11 Jun 2025 16:43:26 -0700 Subject: [PATCH 278/922] chore: prepare for 1.5 release (#2973) Update version markers with upcoming version. --- CHANGELOG.md | 14 +++++++------- .../rules_python/python/config_settings/index.md | 2 +- docs/environment-variables.md | 2 +- docs/toolchains.md | 2 +- python/features.bzl | 2 +- python/private/local_runtime_toolchains_repo.bzl | 6 +++--- python/private/py_info.bzl | 2 +- python/private/py_library.bzl | 2 +- python/private/py_runtime_info.bzl | 2 +- python/private/py_runtime_rule.bzl | 2 +- python/private/pypi/env_marker_info.bzl | 2 +- python/private/python.bzl | 10 +++++----- 12 files changed, 24 insertions(+), 24 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index e8fa1751c2..57001ca44f 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -47,12 +47,12 @@ BEGIN_UNRELEASED_TEMPLATE END_UNRELEASED_TEMPLATE --> -{#v0-0-0} -## Unreleased +{#1-5-0} +## [1.5.0] - 2025-06-11 -[0.0.0]: https://github.com/bazel-contrib/rules_python/releases/tag/0.0.0 +[1.5.0]: https://github.com/bazel-contrib/rules_python/releases/tag/1.5.0 -{#v0-0-0-changed} +{#1-5-0-changed} ### Changed * (rules) On Windows, {obj}`--bootstrap_impl=system_python` is forced. This @@ -66,7 +66,7 @@ END_UNRELEASED_TEMPLATE `PyInfo.site_packages_symlinks` * (deps) Updating setuptools to patch CVE-2025-47273. -{#v0-0-0-fixed} +{#1-5-0-fixed} ### Fixed * (rules) PyInfo provider is now advertised by py_test, py_binary, and py_library; @@ -93,7 +93,7 @@ END_UNRELEASED_TEMPLATE by platform even though the same version is used. Fixes [#2648](https://github.com/bazel-contrib/rules_python/issues/2648). * (pypi) `compile_pip_requirements` test rule works behind the proxy -{#v0-0-0-added} +{#1-5-0-added} ### Added * Repo utilities `execute_unchecked`, `execute_checked`, and `execute_checked_stdout` now support `log_stdout` and `log_stderr` keyword arg booleans. When these are `True` @@ -115,7 +115,7 @@ END_UNRELEASED_TEMPLATE Useful when an intermediate dependency needs to be upgraded to pull in security patches. -{#v0-0-0-removed} +{#1-5-0-removed} ### Removed * Nothing removed. diff --git a/docs/api/rules_python/python/config_settings/index.md b/docs/api/rules_python/python/config_settings/index.md index 7fe25888dd..989ebf1128 100644 --- a/docs/api/rules_python/python/config_settings/index.md +++ b/docs/api/rules_python/python/config_settings/index.md @@ -167,7 +167,7 @@ Default: `//python/config_settings:_pip_env_marker_default_config` This flag points to a target providing {obj}`EnvMarkerInfo`, which determines the values used when environment markers are resolved at build time. -:::{versionadded} VERSION_NEXT_FEATURE +:::{versionadded} 1.5.0 ::: :::: diff --git a/docs/environment-variables.md b/docs/environment-variables.md index 26c171095d..8a51bcbfd2 100644 --- a/docs/environment-variables.md +++ b/docs/environment-variables.md @@ -65,7 +65,7 @@ The default became `1` if unspecified When `1`, the rules_python Starlark implementation of the pypi/pip integration is used instead of the legacy Python scripts. -:::{versionadded} VERSION_NEXT_FEATURE +:::{versionadded} 1.5.0 ::: :::: diff --git a/docs/toolchains.md b/docs/toolchains.md index be85a2471e..668a458156 100644 --- a/docs/toolchains.md +++ b/docs/toolchains.md @@ -305,7 +305,7 @@ that can be used with `target_settings`. Some particular ones of note are: {flag}`--py_linux_libc` and {flag}`--py_freethreaded`, among others. ::: -:::{versionadded} VERSION_NEXT_FEATURE +:::{versionadded} 1.5.0 Added support for custom platform names, `target_compatible_with`, and `target_settings` with `single_version_platform_override`. ::: diff --git a/python/features.bzl b/python/features.bzl index b678a45241..e3d1ffdf61 100644 --- a/python/features.bzl +++ b/python/features.bzl @@ -35,7 +35,7 @@ def _features_typedef(): True if the `PyInfo.venv_symlinks` field is available. - :::{versionadded} VERSION_NEXT_FEATURE + :::{versionadded} 1.5.0 ::: :::: diff --git a/python/private/local_runtime_toolchains_repo.bzl b/python/private/local_runtime_toolchains_repo.bzl index 004ca664ad..8ef5ee9728 100644 --- a/python/private/local_runtime_toolchains_repo.bzl +++ b/python/private/local_runtime_toolchains_repo.bzl @@ -96,7 +96,7 @@ conditions are met, typically values from `@platforms`. See the [Local toolchains] docs for examples and further information. -:::{versionadded} VERSION_NEXT_FEATURE +:::{versionadded} 1.5.0 ::: """, ), @@ -145,7 +145,7 @@ The `target_settings` attribute, which handles `config_setting` values, instead of constraints. ::: -:::{versionadded} VERSION_NEXT_FEATURE +:::{versionadded} 1.5.0 ::: """, ), @@ -183,7 +183,7 @@ The `target_compatible_with` attribute, which handles *constraint* values, instead of `config_settings`. ::: -:::{versionadded} VERSION_NEXT_FEATURE +:::{versionadded} 1.5.0 ::: """, ), diff --git a/python/private/py_info.bzl b/python/private/py_info.bzl index 17c5e4e79e..31df5cfbde 100644 --- a/python/private/py_info.bzl +++ b/python/private/py_info.bzl @@ -312,7 +312,7 @@ This field is currently unused in Bazel and may go away in the future. :::{include} /_includes/experimental_api.md ::: -:::{versionadded} VERSION_NEXT_FEATURE +:::{versionadded} 1.5.0 ::: """, }, diff --git a/python/private/py_library.bzl b/python/private/py_library.bzl index e727694b32..24adb5f3ca 100644 --- a/python/private/py_library.bzl +++ b/python/private/py_library.bzl @@ -97,7 +97,7 @@ This attributes populates {obj}`PyInfo.venv_symlinks`. :::{versionadded} 1.4.0 ::: -:::{versionchanged} VERSION_NEXT_FEATURE +:::{versionchanged} 1.5.0 The topological order has been removed and if 2 different versions of the same PyPI package are observed, the behaviour has no guarantees except that it is deterministic and that only one package version will be included. diff --git a/python/private/py_runtime_info.bzl b/python/private/py_runtime_info.bzl index d2ae17e360..efe14b2c06 100644 --- a/python/private/py_runtime_info.bzl +++ b/python/private/py_runtime_info.bzl @@ -334,7 +334,7 @@ to meet two criteria: interpreter. This typically requires the Python version to be known at build-time and match at runtime. -:::{versionadded} VERSION_NEXT_FEATURE +:::{versionadded} 1.5.0 ::: """, "zip_main_template": """ diff --git a/python/private/py_runtime_rule.bzl b/python/private/py_runtime_rule.bzl index 6dadcfeac3..861014e117 100644 --- a/python/private/py_runtime_rule.bzl +++ b/python/private/py_runtime_rule.bzl @@ -360,7 +360,7 @@ Whether this runtime supports virtualenvs created at build time. See {obj}`PyRuntimeInfo.supports_build_time_venv` for docs. -:::{versionadded} VERSION_NEXT_FEATURE +:::{versionadded} 1.5.0 ::: """, default = True, diff --git a/python/private/pypi/env_marker_info.bzl b/python/private/pypi/env_marker_info.bzl index b483436d98..c3c5ec69ed 100644 --- a/python/private/pypi/env_marker_info.bzl +++ b/python/private/pypi/env_marker_info.bzl @@ -8,7 +8,7 @@ The values to use during environment marker evaluation. The {obj}`--//python/config_settings:pip_env_marker_config` flag. ::: -:::{versionadded} VERSION_NEXT_FEATURE +:::{versionadded} 1.5.0 """, fields = { "env": """ diff --git a/python/private/python.bzl b/python/private/python.bzl index 8e23668879..6eb8a3742e 100644 --- a/python/private/python.bzl +++ b/python/private/python.bzl @@ -1240,7 +1240,7 @@ The values should be one of the values in `@platforms//cpu` Docs for [Registering custom runtimes] ::: -:::{{versionadded}} VERSION_NEXT_FEATURE +:::{{versionadded}} 1.5.0 ::: """, ), @@ -1265,7 +1265,7 @@ The values should be one of the values in `@platforms//os` Docs for [Registering custom runtimes] ::: -:::{{versionadded}} VERSION_NEXT_FEATURE +:::{{versionadded}} 1.5.0 ::: """, ), @@ -1288,7 +1288,7 @@ Other values are allowed, in which case, `target_compatible_with`, `target_settings`, `os_name`, and `arch` should be specified so the toolchain is only used when appropriate. -:::{{versionchanged}} VERSION_NEXT_FEATURE +:::{{versionchanged}} 1.5.0 Arbitrary platform strings allowed. ::: """.format( @@ -1320,7 +1320,7 @@ If set, `target_settings`, `os_name`, and `arch` should also be set. Docs for [Registering custom runtimes] ::: -:::{{versionadded}} VERSION_NEXT_FEATURE +:::{{versionadded}} 1.5.0 ::: """, ), @@ -1334,7 +1334,7 @@ If set, `target_compatible_with`, `os_name`, and `arch` should also be set. Docs for [Registering custom runtimes] ::: -:::{{versionadded}} VERSION_NEXT_FEATURE +:::{{versionadded}} 1.5.0 ::: """, ), From 9b8f6501e8b814b4120ff23d787f2cb7ba8422c6 Mon Sep 17 00:00:00 2001 From: Ignas Anikevicius <240938+aignas@users.noreply.github.com> Date: Thu, 12 Jun 2025 11:51:50 +0900 Subject: [PATCH 279/922] fix: support pre-release versions and add new toolchain versions (#2969) Add latest toolchain builds and attempt adding a beta build. This shows/tests that we can handle pre-release versions just fine and we are able to test the toolchain matching. Whilst at it it implements the static advertising of the remaining interpreter information. Fixes #2837 --------- Co-authored-by: Richard Levasseur --- CHANGELOG.md | 9 + .../private/hermetic_runtime_repo_setup.bzl | 10 ++ python/versions.bzl | 160 ++++++++++++++++-- tests/python/python_tests.bzl | 25 +-- tests/toolchains/defs.bzl | 10 +- tests/toolchains/python_toolchain_test.py | 13 +- .../transitions/transitions_tests.bzl | 17 +- 7 files changed, 209 insertions(+), 35 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 57001ca44f..488f1054a1 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -55,6 +55,12 @@ END_UNRELEASED_TEMPLATE {#1-5-0-changed} ### Changed +* (toolchain) Bundled toolchain version updates: + * 3.9 now references 3.9.23 + * 3.10 now references 3.10.18 + * 3.11 now references 3.11.13 + * 3.12 now references 3.12.11 + * 3.13 now references 3.13.4 * (rules) On Windows, {obj}`--bootstrap_impl=system_python` is forced. This allows setting `--bootstrap_impl=script` in bazelrc for mixed-platform environments. @@ -92,6 +98,8 @@ END_UNRELEASED_TEMPLATE * (pypi) Correctly aggregate the sources when the hashes specified in the lockfile differ by platform even though the same version is used. Fixes [#2648](https://github.com/bazel-contrib/rules_python/issues/2648). * (pypi) `compile_pip_requirements` test rule works behind the proxy +* (toolchains) The hermetic toolchains now correctly statically advertise the + `releaselevel` and `serial` for pre-release hermetic toolchains ({gh-issue}`2837`). {#1-5-0-added} ### Added @@ -114,6 +122,7 @@ END_UNRELEASED_TEMPLATE * (rules) Added support for a using constraints files with `compile_pip_requirements`. Useful when an intermediate dependency needs to be upgraded to pull in security patches. +* (toolchains): 3.14.0b2 has been added as a preview. {#1-5-0-removed} ### Removed diff --git a/python/private/hermetic_runtime_repo_setup.bzl b/python/private/hermetic_runtime_repo_setup.bzl index f944b0b914..98adba51d0 100644 --- a/python/private/hermetic_runtime_repo_setup.bzl +++ b/python/private/hermetic_runtime_repo_setup.bzl @@ -195,6 +195,14 @@ def define_hermetic_runtime_toolchain_impl( values = {"collect_code_coverage": "true"}, visibility = ["//visibility:private"], ) + if not version_info.pre: + releaselevel = "final" + else: + releaselevel = { + "a": "alpha", + "b": "beta", + "rc": "candidate", + }.get(version_info.pre[0]) py_runtime( name = "py3_runtime", @@ -204,6 +212,8 @@ def define_hermetic_runtime_toolchain_impl( "major": str(version_info.release[0]), "micro": str(version_info.release[2]), "minor": str(version_info.release[1]), + "releaselevel": releaselevel, + "serial": str(version_info.pre[1]) if version_info.pre else "0", }, coverage_tool = select({ # Convert empty string to None diff --git a/python/versions.bzl b/python/versions.bzl index e712a2e126..44af7baf69 100644 --- a/python/versions.bzl +++ b/python/versions.bzl @@ -186,6 +186,21 @@ TOOL_VERSIONS = { }, "strip_prefix": "python", }, + "3.9.23": { + "url": "20250610/cpython-{python_version}+20250610-{platform}-{build}.tar.gz", + "sha256": { + "aarch64-apple-darwin": "f1a60528b6088ee8b8a34ca0e960998f4f664bed300ec0bbfe9d66ccbda74e50", + "aarch64-unknown-linux-gnu": "2871cf240bce3c021de829d73da04026febd7a775d1a1a1b37603ec6419fb6c1", + "ppc64le-unknown-linux-gnu": "2ba44a8e084a4661dbe50c0f0e3cf0a57227c6f1cff13fc2ae2f4d8ceae699fc", + "riscv64-unknown-linux-gnu": "7a735aebfc8b19a8af1f03e28babaf18a46cf8db0a931343dac1269376a1f693", + "s390x-unknown-linux-gnu": "27cfc030f782e2683c664e41dcef36051467c98676e133cbef04d4b7155ac4aa", + "x86_64-apple-darwin": "debd576badb6fdabb793ec9956512102f5a813c837449b1fe007c0af977db36c", + "x86_64-pc-windows-msvc": "28fbf2026929e00a300466220917c7029a69331700badb34b1691f1a99aa38e3", + "x86_64-unknown-linux-gnu": "21440e51aee78f3d92faf9375a90713542d8332e83d94c284f8f3d52c58eb5ca", + "x86_64-unknown-linux-musl": "7a881405a41cb4edf8c0d7c469c2f4759f601bc6f3c47978424a1ab1d0f1fada", + }, + "strip_prefix": "python", + }, "3.10.2": { "url": "20220227/cpython-{python_version}+20220227-{platform}-{build}.tar.gz", "sha256": { @@ -321,6 +336,21 @@ TOOL_VERSIONS = { }, "strip_prefix": "python", }, + "3.10.18": { + "url": "20250610/cpython-{python_version}+20250610-{platform}-{build}.tar.gz", + "sha256": { + "aarch64-apple-darwin": "a6590f71f670c7d121ac4f068dc83e271cf03309b80b1fa5890ee4875b7b691d", + "aarch64-unknown-linux-gnu": "b4d7cfb2cb5163da1ae5955ae8b33ac0b356780483d2993099899cf59efaea70", + "ppc64le-unknown-linux-gnu": "36aeae5cc61ff07c78b061f1b6aac628998a380ad45fadc82b8764185544fd7f", + "riscv64-unknown-linux-gnu": "2f6dd270598b655db5da5d98d1c43e560f6fb46c67a8fd68ff9b11ee9f6d79ff", + "s390x-unknown-linux-gnu": "616e56fe69c97a1d0ff13c00f337b2a91c972323c5d9a1828fdfc4d764b440fa", + "x86_64-apple-darwin": "4d72c1c1dcd2c4fe80055ef1b24fe4146f2de938aea1e3676faf91476f3f17e8", + "x86_64-pc-windows-msvc": "867b6dbcdb71d8ebb709ff54fbca8ad43d05cc21e5c157f39745c4dc44c1f8e2", + "x86_64-unknown-linux-gnu": "58f88ed6117078fdbc98976c9bc83b918f1f9c0c2ec21b80a582104f4839861c", + "x86_64-unknown-linux-musl": "d782c0569d6d7e21a5ed195ad7b41d0af8456b031e0814714d18cdeaa876f262", + }, + "strip_prefix": "python", + }, "3.11.1": { "url": "20230116/cpython-{python_version}+20230116-{platform}-{build}.tar.gz", "sha256": { @@ -436,18 +466,18 @@ TOOL_VERSIONS = { }, "strip_prefix": "python", }, - "3.11.11": { - "url": "20250317/cpython-{python_version}+20250317-{platform}-{build}.tar.gz", + "3.11.13": { + "url": "20250610/cpython-{python_version}+20250610-{platform}-{build}.tar.gz", "sha256": { - "aarch64-apple-darwin": "19b147c7e4b742656da4cb6ba35bc3ea2f15aa5f4d1bbbc38d09e2e85551e927", - "aarch64-unknown-linux-gnu": "7d52b5206afe617de2899af477f5a1d275ecbce80fb8300301b254ebf1da5a90", - "ppc64le-unknown-linux-gnu": "17c049f70ce719adc89dd0ae26f4e6a28f6aaedc63c2efef6bbb9c112ea4d692", - "riscv64-unknown-linux-gnu": "83ed50713409576756f5708e8f0549a15c17071bea22b71f15e11a7084f09481", - "s390x-unknown-linux-gnu": "298507f1f8d962b1bb98cb506c99e7e0d291a63eb9117e1521141e6b3825fd56", - "x86_64-apple-darwin": "a870cd965e7dded5100d13b1d34cab1c32a92811e000d10fbfe9bbdb36cdaa0e", - "x86_64-pc-windows-msvc": "1cf5760eea0a9df3308ca2c4111b5cc18fd638b2a912dbe07606193e3f9aa123", - "x86_64-unknown-linux-gnu": "51e47bc0d1b9f4bf68dd395f7a39f60c58a87cde854cab47264a859eb666bb69", - "x86_64-unknown-linux-musl": "ee4d84f992c6a1df42096e26b970fe5938fd6c1eadd245894bc94c5737ff9977", + "aarch64-apple-darwin": "365037494ba4f53563c22292e49a8e4d0d495bcb6534fca9666bdd1b474abf36", + "aarch64-unknown-linux-gnu": "a5954f147e87d9bff3d9733ebb3e74fe997eec5b38eaf5cb4429038228962a16", + "ppc64le-unknown-linux-gnu": "9214126866418f290fda88832fa3e244630f918ebc8a4a9ee15ba922e9c98afd", + "riscv64-unknown-linux-gnu": "fd99008c3123f50ec2ad407c5c1e17c1a86590daaf88dae8e6f1fd28f099b7c2", + "s390x-unknown-linux-gnu": "e27ab1fff8bf9e507677252a03ed524c685a8629b56475e26ab6dd0f88465179", + "x86_64-apple-darwin": "b49044115a545e67d73f5265a613a25da7c9523431281aa7b94691f1013355af", + "x86_64-pc-windows-msvc": "c0f89e3776211147817d54084fa046e2603571e18ff2ae4a4a8ff84ca4f7defc", + "x86_64-unknown-linux-gnu": "d93a7699505ee0ac7dec0f09324ffb19a31cce3066a287bb1fe95285ce3ea0c7", + "x86_64-unknown-linux-musl": "499121bb917e5baeeb954f76bdbce36bb63af579ff1530966ae2280e8d812c5b", }, "strip_prefix": "python", }, @@ -559,6 +589,21 @@ TOOL_VERSIONS = { }, "strip_prefix": "python", }, + "3.12.11": { + "url": "20250610/cpython-{python_version}+20250610-{platform}-{build}.tar.gz", + "sha256": { + "aarch64-apple-darwin": "9c5826a93ddc15e8aa08de1e6e65b3ae0d45ea8eb0c2e9547b80ff4121b870ce", + "aarch64-unknown-linux-gnu": "eb33bc5a87443daf2fd218109df811bc4e4ea5ef9aec4fad75aa55da0258b96f", + "ppc64le-unknown-linux-gnu": "7b90bc528c5ddf30579dec52926d68fa6d5c90b65e24fc185d5fe283fdf0cbd9", + "riscv64-unknown-linux-gnu": "0f3103675102e351762a8fe574eae20335552a246a45a006d2a9ca14ce0952f8", + "s390x-unknown-linux-gnu": "a7ff0432208450ccebd5d328f69b84cc7c25b4af54fbab44803ddb11a2da5028", + "x86_64-apple-darwin": "199631baa35f3747ddfa2f1e28fc062b97ccd15b94a60c9294d4d129a73c9e53", + "x86_64-pc-windows-msvc": "e05fa165841c416d60365ca2216cad570f05ae5d3d027b9ad3beaad0529dd8cc", + "x86_64-unknown-linux-gnu": "77ab3efe5c6637fe8da0fdfbff5de1730c3b824874fe1368917886908b4c517b", + "x86_64-unknown-linux-musl": "9dd768494c4a34abcec316bc4802e957db98ed283024b527c0c40dfefd08b6fe", + }, + "strip_prefix": "python", + }, "3.13.0": { "url": "20241016/cpython-{python_version}+20241016-{platform}-{build}.{ext}", "sha256": { @@ -674,16 +719,99 @@ TOOL_VERSIONS = { "x86_64-unknown-linux-gnu-freethreaded": "python/install", }, }, + "3.13.4": { + "url": "20250610/cpython-{python_version}+20250610-{platform}-{build}.{ext}", + "sha256": { + "aarch64-apple-darwin": "c2ce6601b2668c7bd1f799986af5ddfbff36e88795741864aba6e578cb02ed7f", + "aarch64-unknown-linux-gnu": "3c2596ece08ffe17e11bc1f27aeb4ce1195d2490a83d695d36ef4933d5c5ca53", + "ppc64le-unknown-linux-gnu": "b3cc13ee177b8db1d3e9b2eac413484e3c6a356f97d91dc59de8d3fd8cf79d6b", + "riscv64-unknown-linux-gnu": "d1b989e57a9ce29f6c945eeffe0e9750c222fdd09e99d2f8d6b0d8532a523053", + "s390x-unknown-linux-gnu": "d1d19fb01961ac6476712fdd6c5031f74c83666f6f11aa066207e9a158f7e3d8", + "x86_64-apple-darwin": "79feb6ca68f3921d07af52d9db06cf134e6f36916941ea850ab0bc20f5ff638b", + "x86_64-pc-windows-msvc": "29ac3585cc2dcfd79e3fe380c272d00e9d34351fc456e149403c86d3fea34057", + "x86_64-unknown-linux-gnu": "44e5477333ebca298a7a0a316985c6c3533b8645f92a83f7f73c44033832bf32", + "x86_64-unknown-linux-musl": "a3afbfa94b9ff4d9fc426b47eb3c8446cada535075b8d51b7bdc9d9ab9911fc2", + "aarch64-apple-darwin-freethreaded": "278dccade56b4bbeecb9a613b77012cf5c1433a5e9b8ef99230d5e61f31d9e02", + "aarch64-unknown-linux-gnu-freethreaded": "b1c1bd6ab9ef95b464d92a6a911cef1a8d9f0b0f6a192f694ef18ed15d882edf", + "ppc64le-unknown-linux-gnu-freethreaded": "ed66ae213a62b286b9b7338b816ccd2815f5248b7a28a185dc8159fe004149ae", + "riscv64-unknown-linux-gnu-freethreaded": "913264545215236660e4178bc3e5b57a20a444a8deb5c11680c95afc960b4016", + "s390x-unknown-linux-gnu-freethreaded": "7556a38ab5e507c1ec22bc38f9859982bc956cab7f4de05a2faac114feb306db", + "x86_64-apple-darwin-freethreaded": "64ab7ac8c88002d9ba20a92f72945bfa350268e944a7922500af75d20330574d", + "x86_64-pc-windows-msvc-freethreaded": "9457504547edb2e0156bf76b53c7e4941c7f61c0eff9fd5f4d816d3df51c58e3", + "x86_64-unknown-linux-gnu-freethreaded": "864df6e6819e8f8e855ce30f34410fdc5867d0616e904daeb9a40e5806e970d7", + }, + "strip_prefix": { + "aarch64-apple-darwin": "python", + "aarch64-unknown-linux-gnu": "python", + "ppc64le-unknown-linux-gnu": "python", + "s390x-unknown-linux-gnu": "python", + "riscv64-unknown-linux-gnu": "python", + "x86_64-apple-darwin": "python", + "x86_64-pc-windows-msvc": "python", + "x86_64-unknown-linux-gnu": "python", + "x86_64-unknown-linux-musl": "python", + "aarch64-apple-darwin-freethreaded": "python/install", + "aarch64-unknown-linux-gnu-freethreaded": "python/install", + "ppc64le-unknown-linux-gnu-freethreaded": "python/install", + "riscv64-unknown-linux-gnu-freethreaded": "python/install", + "s390x-unknown-linux-gnu-freethreaded": "python/install", + "x86_64-apple-darwin-freethreaded": "python/install", + "x86_64-pc-windows-msvc-freethreaded": "python/install", + "x86_64-unknown-linux-gnu-freethreaded": "python/install", + }, + }, + "3.14.0b2": { + "url": "20250610/cpython-{python_version}+20250610-{platform}-{build}.{ext}", + "sha256": { + "aarch64-apple-darwin": "6607351d140e83feb6e11dbde46ab5f99fa9fe039bdbaa12611d26bda0ed9343", + "aarch64-unknown-linux-gnu": "cc388d567f7c23921e0bef8dcae959dfab9ee24d10aeeb23688b21eac402817f", + "ppc64le-unknown-linux-gnu": "f9379ecc5dc71f9c58adf03d5524176ec36e1b40c788d29c260df54d09ad351c", + "riscv64-unknown-linux-gnu": "e6fbe4f7928ec606edee1506752659bf59216fdb208c744d268082ec79b16f42", + "s390x-unknown-linux-gnu": "1cf32c1173adc1cb70952bb47c92177a196f9e83b7a874f09599682e92ba0010", + "x86_64-apple-darwin": "a6d8196b174409e0ce67829c4e4ee5005c4be20a2efb41116e0521ad1fa1a717", + "x86_64-pc-windows-msvc": "0d88ec80c6c3e3ac462368850c19d3930bf2b1a1a5fe89da60c8534d0fac1a01", + "x86_64-unknown-linux-gnu": "93b29eea5214d19f0420ef8e459b007e15ea58349d60811122c78241fe51cb92", + "x86_64-unknown-linux-musl": "90e90a58ebff3416eb5a3f93ecb59b6eda945e2b706f5c13b0ba85f6b2bee130", + "aarch64-apple-darwin-freethreaded": "af0f34aa0dcd02bd3d960a1572a1ed8a17d55b373a22866f05041aaf16f8607d", + "aarch64-unknown-linux-gnu-freethreaded": "e76c7ab98e1c0f86a6996d1ec775ba8497bf46aa8ffa8c7b0f2e761f37305329", + "ppc64le-unknown-linux-gnu-freethreaded": "df2ae00827406e247f1aaaec76ffc7963b909c81075fc9940eee1ea9f753dd16", + "riscv64-unknown-linux-gnu-freethreaded": "09e347cb5f29e0eafd1eba73105ea9d853184b55fbaf4746cebec217430d6db5", + "s390x-unknown-linux-gnu-freethreaded": "f911605eee0eb7845a69acaf8bfb2e1811c76e9a5e3980d97fae93135df4b773", + "x86_64-apple-darwin-freethreaded": "dd27d519cf2a04917cb566366d6539477791d1b2f1fb42037d9179f469ff55a9", + "x86_64-pc-windows-msvc-freethreaded": "da966a17e434094d8f10b719d93c782d82eaf5207f2843cbaa58c3d91a8f0e32", + "x86_64-unknown-linux-gnu-freethreaded": "abd60d3a302e9d9c32ec78581fb3a9903079c56ec7a949ce658a7950423f350a", + }, + "strip_prefix": { + "aarch64-apple-darwin": "python", + "aarch64-unknown-linux-gnu": "python", + "ppc64le-unknown-linux-gnu": "python", + "s390x-unknown-linux-gnu": "python", + "riscv64-unknown-linux-gnu": "python", + "x86_64-apple-darwin": "python", + "x86_64-pc-windows-msvc": "python", + "x86_64-unknown-linux-gnu": "python", + "x86_64-unknown-linux-musl": "python", + "aarch64-apple-darwin-freethreaded": "python/install", + "aarch64-unknown-linux-gnu-freethreaded": "python/install", + "ppc64le-unknown-linux-gnu-freethreaded": "python/install", + "riscv64-unknown-linux-gnu-freethreaded": "python/install", + "s390x-unknown-linux-gnu-freethreaded": "python/install", + "x86_64-apple-darwin-freethreaded": "python/install", + "x86_64-pc-windows-msvc-freethreaded": "python/install", + "x86_64-unknown-linux-gnu-freethreaded": "python/install", + }, + }, } # buildifier: disable=unsorted-dict-items MINOR_MAPPING = { "3.8": "3.8.20", - "3.9": "3.9.21", - "3.10": "3.10.16", - "3.11": "3.11.11", - "3.12": "3.12.9", - "3.13": "3.13.2", + "3.9": "3.9.23", + "3.10": "3.10.18", + "3.11": "3.11.13", + "3.12": "3.12.11", + "3.13": "3.13.4", + "3.14": "3.14.0b2", } def _generate_platforms(): diff --git a/tests/python/python_tests.bzl b/tests/python/python_tests.bzl index 116afa76ad..f0dc4825ac 100644 --- a/tests/python/python_tests.bzl +++ b/tests/python/python_tests.bzl @@ -304,11 +304,11 @@ def _test_toolchain_ordering(env): toolchain = [ _toolchain("3.10"), _toolchain("3.10.15"), - _toolchain("3.10.16"), - _toolchain("3.10.11"), + _toolchain("3.10.18"), + _toolchain("3.10.13"), _toolchain("3.11.1"), _toolchain("3.11.10"), - _toolchain("3.11.11", is_default = True), + _toolchain("3.11.13", is_default = True), ], ), _mod(name = "rules_python", toolchain = [_toolchain("3.11")]), @@ -320,14 +320,15 @@ def _test_toolchain_ordering(env): for t in py.toolchains ] - env.expect.that_str(py.default_python_version).equals("3.11.11") + env.expect.that_str(py.default_python_version).equals("3.11.13") env.expect.that_dict(py.config.minor_mapping).contains_exactly({ - "3.10": "3.10.16", - "3.11": "3.11.11", - "3.12": "3.12.9", - "3.13": "3.13.2", + "3.10": "3.10.18", + "3.11": "3.11.13", + "3.12": "3.12.11", + "3.13": "3.13.4", + "3.14": "3.14.0b2", "3.8": "3.8.20", - "3.9": "3.9.21", + "3.9": "3.9.23", }) env.expect.that_collection(got_versions).contains_exactly([ # First the full-version toolchains that are in minor_mapping @@ -336,13 +337,13 @@ def _test_toolchain_ordering(env): # The default version is always set in the `python_version` flag, so know, that # the default match will be somewhere in the first bunch. "3.10", - "3.10.16", + "3.10.18", "3.11", - "3.11.11", + "3.11.13", # Next, the rest, where we will match things based on the `python_version` being # the same "3.10.15", - "3.10.11", + "3.10.13", "3.11.1", "3.11.10", ]).in_order() diff --git a/tests/toolchains/defs.bzl b/tests/toolchains/defs.bzl index a883b0af33..25863d18c4 100644 --- a/tests/toolchains/defs.bzl +++ b/tests/toolchains/defs.bzl @@ -15,6 +15,7 @@ "" load("//python:versions.bzl", "PLATFORMS", "TOOL_VERSIONS") +load("//python/private:version.bzl", "version") # buildifier: disable=bzl-visibility load("//tests/support:py_reconfig.bzl", "py_reconfig_test") def define_toolchain_tests(name): @@ -38,13 +39,20 @@ def define_toolchain_tests(name): is_platform = "_is_{}".format(platform_key) target_compatible_with[is_platform] = [] + parsed = version.parse(python_version, strict = True) + expect_python_version = "{0}.{1}.{2}".format(*parsed.release) + if parsed.pre: + expect_python_version = "{0}{1}{2}".format( + expect_python_version, + *parsed.pre + ) py_reconfig_test( name = "python_{}_test".format(python_version), srcs = ["python_toolchain_test.py"], main = "python_toolchain_test.py", python_version = python_version, env = { - "EXPECT_PYTHON_VERSION": python_version, + "EXPECT_PYTHON_VERSION": expect_python_version, }, deps = ["//python/runfiles"], data = ["//tests/support:current_build_settings"], diff --git a/tests/toolchains/python_toolchain_test.py b/tests/toolchains/python_toolchain_test.py index 591d7dbe8a..63ed42488f 100644 --- a/tests/toolchains/python_toolchain_test.py +++ b/tests/toolchains/python_toolchain_test.py @@ -27,7 +27,18 @@ def test_expected_toolchain_matches(self): ) self.assertIn(expected, settings["toolchain_label"], msg) - actual = "{v.major}.{v.minor}.{v.micro}".format(v=sys.version_info) + if sys.version_info.releaselevel == "final": + actual = "{v.major}.{v.minor}.{v.micro}".format(v=sys.version_info) + elif sys.version_info.releaselevel in ["beta"]: + actual = ( + "{v.major}.{v.minor}.{v.micro}{v.releaselevel[0]}{v.serial}".format( + v=sys.version_info + ) + ) + else: + raise NotImplementedError( + "Unsupported release level, please update the test" + ) self.assertEqual(actual, expect_version) diff --git a/tests/toolchains/transitions/transitions_tests.bzl b/tests/toolchains/transitions/transitions_tests.bzl index bddd1745f0..ef071188bb 100644 --- a/tests/toolchains/transitions/transitions_tests.bzl +++ b/tests/toolchains/transitions/transitions_tests.bzl @@ -56,14 +56,21 @@ def _impl(ctx): exec_tools = ctx.toolchains[EXEC_TOOLS_TOOLCHAIN_TYPE].exec_tools got_version = exec_tools.exec_interpreter[platform_common.ToolchainInfo].py3_runtime.interpreter_version_info + got = "{}.{}.{}".format( + got_version.major, + got_version.minor, + got_version.micro, + ) + if got_version.releaselevel != "final": + got = "{}{}{}".format( + got, + got_version.releaselevel[0], + got_version.serial, + ) return [ TestInfo( - got = "{}.{}.{}".format( - got_version.major, - got_version.minor, - got_version.micro, - ), + got = got, want = ctx.attr.want_version, ), ] From e225a1eddd6055b08cb832f7d4e73922d5f7d956 Mon Sep 17 00:00:00 2001 From: Douglas Thor Date: Wed, 11 Jun 2025 22:27:04 -0700 Subject: [PATCH 280/922] chore: Fixup some typos in BuildKite job names (#2977) --- .bazelci/presubmit.yml | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/.bazelci/presubmit.yml b/.bazelci/presubmit.yml index 7e9d4dea53..01af217924 100644 --- a/.bazelci/presubmit.yml +++ b/.bazelci/presubmit.yml @@ -90,7 +90,7 @@ tasks: gazelle_extension_min: <<: *common_workspace_flags_min_bazel <<: *minimum_supported_version - name: "Gazelle: workspace, minumum supported Bazel version" + name: "Gazelle: workspace, minimum supported Bazel version" platform: ubuntu2004 build_targets: ["//..."] test_targets: ["//..."] @@ -338,7 +338,7 @@ tasks: integration_test_bzlmod_build_file_generation_windows: <<: *reusable_build_test_all # coverage is not supported on Windows - name: "examples/bzlmod_build_file_generateion: Windows" + name: "examples/bzlmod_build_file_generation: Windows" working_directory: examples/bzlmod_build_file_generation platform: windows From f2fa07a56f575028cd84d4d4d169b734507c34d7 Mon Sep 17 00:00:00 2001 From: John Cater Date: Fri, 13 Jun 2025 12:31:51 -0400 Subject: [PATCH 281/922] refactor: Remove unused CC_TOOLCHAIN definition (#2981) Fixes #2979. The definition appears unused and helps advance the goal of entirely removing current_cc_toolchain: see https://github.com/bazelbuild/bazel/issues/26282. --- python/private/attributes.bzl | 6 ------ 1 file changed, 6 deletions(-) diff --git a/python/private/attributes.bzl b/python/private/attributes.bzl index c3b1cade91..641fa13a23 100644 --- a/python/private/attributes.bzl +++ b/python/private/attributes.bzl @@ -156,12 +156,6 @@ def copy_common_test_kwargs(kwargs): if key in kwargs } -CC_TOOLCHAIN = { - # NOTE: The `cc_helper.find_cpp_toolchain()` function expects the attribute - # name to be this name. - "_cc_toolchain": attr.label(default = "@bazel_tools//tools/cpp:current_cc_toolchain"), -} - # The common "data" attribute definition. DATA_ATTRS = { # NOTE: The "flags" attribute is deprecated, but there isn't an alternative From 94e08f7dfe61962fa50508f01ea05c624307d487 Mon Sep 17 00:00:00 2001 From: Keith Smiley Date: Fri, 13 Jun 2025 19:20:26 -0700 Subject: [PATCH 282/922] Fix argument name typo (#2984) ``` ERROR: Traceback (most recent call last): File ".../rules_python++pip+rules_mypy_pip_312_click/BUILD.bazel", line 5, column 20, in whl_library_targets( File ".../rules_python+/python/private/pypi/whl_library_targets.bzl", line 337, column 53, in whl_library_targets "//conditions:default": create_inits( File ".../rules_python+/python/private/pypi/namespace_pkgs.bzl", line 72, column 25, in create_inits for out in get_files(**kwargs): File ".../rules_python+/python/private/pypi/namespace_pkgs.bzl", line 20, column 5, in get_files def get_files(*, srcs, ignored_dirnames = [], root = None): Error: get_files() got unexpected keyword argument: ignore_dirnames (did you mean 'ignored_dirnames'?) ``` --- python/private/pypi/whl_library_targets.bzl | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/python/private/pypi/whl_library_targets.bzl b/python/private/pypi/whl_library_targets.bzl index 3529566c49..518d17163f 100644 --- a/python/private/pypi/whl_library_targets.bzl +++ b/python/private/pypi/whl_library_targets.bzl @@ -336,7 +336,7 @@ def whl_library_targets( Label("//python/config_settings:is_venvs_site_packages"): [], "//conditions:default": create_inits( srcs = srcs + data + pyi_srcs, - ignore_dirnames = [], # If you need to ignore certain folders, you can patch rules_python here to do so. + ignored_dirnames = [], # If you need to ignore certain folders, you can patch rules_python here to do so. root = "site-packages", ), }) From ca235368d04fb0ebf39fc9174acfd883ca3e3675 Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Tue, 17 Jun 2025 16:23:13 +0900 Subject: [PATCH 283/922] build(deps): bump certifi from 2025.1.31 to 2025.6.15 in /tools/publish (#2999) Bumps [certifi](https://github.com/certifi/python-certifi) from 2025.1.31 to 2025.6.15.
Commits
  • e767d59 2025.06.15 (#357)
  • 3e70765 Bump actions/setup-python from 5.5.0 to 5.6.0
  • 9afd2ff Bump actions/download-artifact from 4.2.1 to 4.3.0
  • d7c816c remove code that's no longer required that 3.7 is our minimum (#351)
  • 1899613 Declare setuptools as the build backend in pyproject.toml (#350)
  • c874142 update CI for ubuntu 20.04 deprecation (#348)
  • 275c9eb 2025.04.26 (#347)
  • 3788331 Bump actions/setup-python from 5.4.0 to 5.5.0 (#346)
  • 9d1f1b7 Bump actions/download-artifact from 4.1.9 to 4.2.1 (#344)
  • 96b97a5 Bump actions/upload-artifact from 4.6.1 to 4.6.2 (#343)
  • Additional commits viewable in compare view

[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=certifi&package-manager=pip&previous-version=2025.1.31&new-version=2025.6.15)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) ---
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot show ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> --- tools/publish/requirements_darwin.txt | 6 +++--- tools/publish/requirements_linux.txt | 6 +++--- tools/publish/requirements_universal.txt | 6 +++--- tools/publish/requirements_windows.txt | 6 +++--- 4 files changed, 12 insertions(+), 12 deletions(-) diff --git a/tools/publish/requirements_darwin.txt b/tools/publish/requirements_darwin.txt index 483f88444e..af5bad246d 100644 --- a/tools/publish/requirements_darwin.txt +++ b/tools/publish/requirements_darwin.txt @@ -6,9 +6,9 @@ backports-tarfile==1.2.0 \ --hash=sha256:77e284d754527b01fb1e6fa8a1afe577858ebe4e9dad8919e34c862cb399bc34 \ --hash=sha256:d75e02c268746e1b8144c278978b6e98e85de6ad16f8e4b0844a154557eca991 # via jaraco-context -certifi==2025.1.31 \ - --hash=sha256:3d5da6925056f6f18f119200434a4780a94263f10d1c21d032a6f6b2baa20651 \ - --hash=sha256:ca78db4565a652026a4db2bcdf68f2fb589ea80d0be70e03929ed730746b84fe +certifi==2025.6.15 \ + --hash=sha256:2e0c7ce7cb5d8f8634ca55d2ba7e6ec2689a2fd6537d8dec1296a477a4910057 \ + --hash=sha256:d747aa5a8b9bbbb1bb8c22bb13e22bd1f18e9796defa16bab421f7f7a317323b # via requests charset-normalizer==3.4.1 \ --hash=sha256:0167ddc8ab6508fe81860a57dd472b2ef4060e8d378f0cc555707126830f2537 \ diff --git a/tools/publish/requirements_linux.txt b/tools/publish/requirements_linux.txt index 62dbf1eb77..b2e9ccf5ab 100644 --- a/tools/publish/requirements_linux.txt +++ b/tools/publish/requirements_linux.txt @@ -6,9 +6,9 @@ backports-tarfile==1.2.0 \ --hash=sha256:77e284d754527b01fb1e6fa8a1afe577858ebe4e9dad8919e34c862cb399bc34 \ --hash=sha256:d75e02c268746e1b8144c278978b6e98e85de6ad16f8e4b0844a154557eca991 # via jaraco-context -certifi==2025.1.31 \ - --hash=sha256:3d5da6925056f6f18f119200434a4780a94263f10d1c21d032a6f6b2baa20651 \ - --hash=sha256:ca78db4565a652026a4db2bcdf68f2fb589ea80d0be70e03929ed730746b84fe +certifi==2025.6.15 \ + --hash=sha256:2e0c7ce7cb5d8f8634ca55d2ba7e6ec2689a2fd6537d8dec1296a477a4910057 \ + --hash=sha256:d747aa5a8b9bbbb1bb8c22bb13e22bd1f18e9796defa16bab421f7f7a317323b # via requests cffi==1.17.1 \ --hash=sha256:045d61c734659cc045141be4bae381a41d89b741f795af1dd018bfb532fd0df8 \ diff --git a/tools/publish/requirements_universal.txt b/tools/publish/requirements_universal.txt index e4e876b176..8a7426e517 100644 --- a/tools/publish/requirements_universal.txt +++ b/tools/publish/requirements_universal.txt @@ -6,9 +6,9 @@ backports-tarfile==1.2.0 ; python_full_version < '3.12' \ --hash=sha256:77e284d754527b01fb1e6fa8a1afe577858ebe4e9dad8919e34c862cb399bc34 \ --hash=sha256:d75e02c268746e1b8144c278978b6e98e85de6ad16f8e4b0844a154557eca991 # via jaraco-context -certifi==2025.1.31 \ - --hash=sha256:3d5da6925056f6f18f119200434a4780a94263f10d1c21d032a6f6b2baa20651 \ - --hash=sha256:ca78db4565a652026a4db2bcdf68f2fb589ea80d0be70e03929ed730746b84fe +certifi==2025.6.15 \ + --hash=sha256:2e0c7ce7cb5d8f8634ca55d2ba7e6ec2689a2fd6537d8dec1296a477a4910057 \ + --hash=sha256:d747aa5a8b9bbbb1bb8c22bb13e22bd1f18e9796defa16bab421f7f7a317323b # via requests cffi==1.17.1 ; platform_python_implementation != 'PyPy' and sys_platform == 'linux' \ --hash=sha256:045d61c734659cc045141be4bae381a41d89b741f795af1dd018bfb532fd0df8 \ diff --git a/tools/publish/requirements_windows.txt b/tools/publish/requirements_windows.txt index 043de9ecb1..11017aa4f9 100644 --- a/tools/publish/requirements_windows.txt +++ b/tools/publish/requirements_windows.txt @@ -6,9 +6,9 @@ backports-tarfile==1.2.0 \ --hash=sha256:77e284d754527b01fb1e6fa8a1afe577858ebe4e9dad8919e34c862cb399bc34 \ --hash=sha256:d75e02c268746e1b8144c278978b6e98e85de6ad16f8e4b0844a154557eca991 # via jaraco-context -certifi==2025.1.31 \ - --hash=sha256:3d5da6925056f6f18f119200434a4780a94263f10d1c21d032a6f6b2baa20651 \ - --hash=sha256:ca78db4565a652026a4db2bcdf68f2fb589ea80d0be70e03929ed730746b84fe +certifi==2025.6.15 \ + --hash=sha256:2e0c7ce7cb5d8f8634ca55d2ba7e6ec2689a2fd6537d8dec1296a477a4910057 \ + --hash=sha256:d747aa5a8b9bbbb1bb8c22bb13e22bd1f18e9796defa16bab421f7f7a317323b # via requests charset-normalizer==3.4.1 \ --hash=sha256:0167ddc8ab6508fe81860a57dd472b2ef4060e8d378f0cc555707126830f2537 \ From 60b48e2156574ed40e24df32f7dbef59f6f6c4f4 Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Tue, 17 Jun 2025 17:00:40 +0900 Subject: [PATCH 284/922] build(deps): bump certifi from 2025.1.31 to 2025.6.15 in /docs (#3000) Bumps [certifi](https://github.com/certifi/python-certifi) from 2025.1.31 to 2025.6.15.
Commits
  • e767d59 2025.06.15 (#357)
  • 3e70765 Bump actions/setup-python from 5.5.0 to 5.6.0
  • 9afd2ff Bump actions/download-artifact from 4.2.1 to 4.3.0
  • d7c816c remove code that's no longer required that 3.7 is our minimum (#351)
  • 1899613 Declare setuptools as the build backend in pyproject.toml (#350)
  • c874142 update CI for ubuntu 20.04 deprecation (#348)
  • 275c9eb 2025.04.26 (#347)
  • 3788331 Bump actions/setup-python from 5.4.0 to 5.5.0 (#346)
  • 9d1f1b7 Bump actions/download-artifact from 4.1.9 to 4.2.1 (#344)
  • 96b97a5 Bump actions/upload-artifact from 4.6.1 to 4.6.2 (#343)
  • Additional commits viewable in compare view

[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=certifi&package-manager=pip&previous-version=2025.1.31&new-version=2025.6.15)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) ---
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot show ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> --- docs/requirements.txt | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/requirements.txt b/docs/requirements.txt index 87c13aa8ba..b0a84d476b 100644 --- a/docs/requirements.txt +++ b/docs/requirements.txt @@ -17,9 +17,9 @@ babel==2.17.0 \ --hash=sha256:0c54cffb19f690cdcc52a3b50bcbf71e07a808d1c80d549f2459b9d2cf0afb9d \ --hash=sha256:4d0b53093fdfb4b21c92b5213dba5a1b23885afa8383709427046b21c366e5f2 # via sphinx -certifi==2025.1.31 \ - --hash=sha256:3d5da6925056f6f18f119200434a4780a94263f10d1c21d032a6f6b2baa20651 \ - --hash=sha256:ca78db4565a652026a4db2bcdf68f2fb589ea80d0be70e03929ed730746b84fe +certifi==2025.6.15 \ + --hash=sha256:2e0c7ce7cb5d8f8634ca55d2ba7e6ec2689a2fd6537d8dec1296a477a4910057 \ + --hash=sha256:d747aa5a8b9bbbb1bb8c22bb13e22bd1f18e9796defa16bab421f7f7a317323b # via requests charset-normalizer==3.4.1 \ --hash=sha256:0167ddc8ab6508fe81860a57dd472b2ef4060e8d378f0cc555707126830f2537 \ From be86f4acae5571c39c2d6a952e28c435cd722a91 Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Tue, 17 Jun 2025 17:38:57 +0900 Subject: [PATCH 285/922] build(deps): bump requests from 2.32.3 to 2.32.4 in /docs (#2965) Bumps [requests](https://github.com/psf/requests) from 2.32.3 to 2.32.4.
Release notes

Sourced from requests's releases.

v2.32.4

2.32.4 (2025-06-10)

Security

  • CVE-2024-47081 Fixed an issue where a maliciously crafted URL and trusted environment will retrieve credentials for the wrong hostname/machine from a netrc file. (#6965)

Improvements

  • Numerous documentation improvements

Deprecations

  • Added support for pypy 3.11 for Linux and macOS. (#6926)
  • Dropped support for pypy 3.9 following its end of support. (#6926)
Changelog

Sourced from requests's changelog.

2.32.4 (2025-06-10)

Security

  • CVE-2024-47081 Fixed an issue where a maliciously crafted URL and trusted environment will retrieve credentials for the wrong hostname/machine from a netrc file.

Improvements

  • Numerous documentation improvements

Deprecations

  • Added support for pypy 3.11 for Linux and macOS.
  • Dropped support for pypy 3.9 following its end of support.
Commits
  • 021dc72 Polish up release tooling for last manual release
  • 821770e Bump version and add release notes for v2.32.4
  • 59f8aa2 Add netrc file search information to authentication documentation (#6876)
  • 5b4b64c Add more tests to prevent regression of CVE 2024 47081
  • 7bc4587 Add new test to check netrc auth leak (#6962)
  • 96ba401 Only use hostname to do netrc lookup instead of netloc
  • 7341690 Merge pull request #6951 from tswast/patch-1
  • 6716d7c remove links
  • a7e1c74 Update docs/conf.py
  • c799b81 docs: fix dead links to kenreitz.org
  • Additional commits viewable in compare view

[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=requests&package-manager=pip&previous-version=2.32.3&new-version=2.32.4)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) ---
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot show ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) You can disable automated security fix PRs for this repo from the [Security Alerts page](https://github.com/bazel-contrib/rules_python/network/alerts).
Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> --- docs/requirements.txt | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/requirements.txt b/docs/requirements.txt index b0a84d476b..cfeb0cbf31 100644 --- a/docs/requirements.txt +++ b/docs/requirements.txt @@ -291,9 +291,9 @@ readthedocs-sphinx-ext==2.2.5 \ --hash=sha256:ee5fd5b99db9f0c180b2396cbce528aa36671951b9526bb0272dbfce5517bd27 \ --hash=sha256:f8c56184ea011c972dd45a90122568587cc85b0127bc9cf064d17c68bc809daa # via rules-python-docs (docs/pyproject.toml) -requests==2.32.3 \ - --hash=sha256:55365417734eb18255590a9ff9eb97e9e1da868d4ccd6402399eaf68af20a760 \ - --hash=sha256:70761cfe03c773ceb22aa2f671b4757976145175cdfca038c02654d061d6dcc6 +requests==2.32.4 \ + --hash=sha256:27babd3cda2a6d50b30443204ee89830707d396671944c998b5975b031ac2b2c \ + --hash=sha256:27d0316682c8a29834d3264820024b62a36942083d52caf2f14c0591336d3422 # via # readthedocs-sphinx-ext # sphinx From 107a8781cdd207c9079ecd733c0028d2706a49f2 Mon Sep 17 00:00:00 2001 From: Ignas Anikevicius <240938+aignas@users.noreply.github.com> Date: Wed, 18 Jun 2025 02:44:55 +0900 Subject: [PATCH 286/922] fix: use platform_info.target_settings in toolchain aliases (#3001) During the refactor we forgot one more place where the `flag_values` on the platform information was used. They were no longer populated and broke. The solution is to use `selects.config_setting_group` to maintain behaviour and in order to smoke test I have added a target to verify that the aliases work. Related to #2875 Fixes #2993 Co-authored-by: Richard Levasseur --- python/config_settings/BUILD.bazel | 12 ++++++---- python/private/config_settings.bzl | 2 +- .../private/hermetic_runtime_repo_setup.bzl | 19 ++++++++-------- python/private/pypi/config_settings.bzl | 22 +++++++++---------- python/private/toolchain_aliases.bzl | 12 +++++++--- tests/toolchains/BUILD.bazel | 8 +++++++ 6 files changed, 47 insertions(+), 28 deletions(-) diff --git a/python/config_settings/BUILD.bazel b/python/config_settings/BUILD.bazel index b11580c4cb..82a73cee6c 100644 --- a/python/config_settings/BUILD.bazel +++ b/python/config_settings/BUILD.bazel @@ -125,15 +125,19 @@ string_flag( visibility = ["//visibility:public"], ) -config_setting( +alias( name = "is_py_freethreaded", - flag_values = {":py_freethreaded": FreeThreadedFlag.YES}, + actual = ":_is_py_freethreaded_yes", + deprecation = "not actually public, please create your own config_setting using the flag that rules_python exposes", + tags = ["manual"], visibility = ["//visibility:public"], ) -config_setting( +alias( name = "is_py_non_freethreaded", - flag_values = {":py_freethreaded": FreeThreadedFlag.NO}, + actual = ":_is_py_freethreaded_no", + deprecation = "not actually public, please create your own config_setting using the flag that rules_python exposes", + tags = ["manual"], visibility = ["//visibility:public"], ) diff --git a/python/private/config_settings.bzl b/python/private/config_settings.bzl index aff5d016fb..3089b9c6cf 100644 --- a/python/private/config_settings.bzl +++ b/python/private/config_settings.bzl @@ -143,7 +143,7 @@ def construct_config_settings(*, name, default_version, versions, minor_mapping, ) native.config_setting( name = "_is_py_linux_libc_musl", - flag_values = {libc: "glibc"}, + flag_values = {libc: "musl"}, visibility = _NOT_ACTUALLY_PUBLIC, ) freethreaded = Label("//python/config_settings:py_freethreaded") diff --git a/python/private/hermetic_runtime_repo_setup.bzl b/python/private/hermetic_runtime_repo_setup.bzl index 98adba51d0..6910ea14a1 100644 --- a/python/private/hermetic_runtime_repo_setup.bzl +++ b/python/private/hermetic_runtime_repo_setup.bzl @@ -22,7 +22,8 @@ load(":glob_excludes.bzl", "glob_excludes") load(":py_exec_tools_toolchain.bzl", "py_exec_tools_toolchain") load(":version.bzl", "version") -_IS_FREETHREADED = Label("//python/config_settings:is_py_freethreaded") +_IS_FREETHREADED_YES = Label("//python/config_settings:_is_py_freethreaded_yes") +_IS_FREETHREADED_NO = Label("//python/config_settings:_is_py_freethreaded_no") def define_hermetic_runtime_toolchain_impl( *, @@ -87,16 +88,16 @@ def define_hermetic_runtime_toolchain_impl( cc_import( name = "interface", interface_library = select({ - _IS_FREETHREADED: "libs/python{major}{minor}t.lib".format(**version_dict), - "//conditions:default": "libs/python{major}{minor}.lib".format(**version_dict), + _IS_FREETHREADED_YES: "libs/python{major}{minor}t.lib".format(**version_dict), + _IS_FREETHREADED_NO: "libs/python{major}{minor}.lib".format(**version_dict), }), system_provided = True, ) cc_import( name = "abi3_interface", interface_library = select({ - _IS_FREETHREADED: "libs/python3t.lib", - "//conditions:default": "libs/python3.lib", + _IS_FREETHREADED_YES: "libs/python3t.lib", + _IS_FREETHREADED_NO: "libs/python3.lib", }), system_provided = True, ) @@ -115,10 +116,10 @@ def define_hermetic_runtime_toolchain_impl( includes = [ "include", ] + select({ - _IS_FREETHREADED: [ + _IS_FREETHREADED_YES: [ "include/python{major}.{minor}t".format(**version_dict), ], - "//conditions:default": [ + _IS_FREETHREADED_NO: [ "include/python{major}.{minor}".format(**version_dict), "include/python{major}.{minor}m".format(**version_dict), ], @@ -224,8 +225,8 @@ def define_hermetic_runtime_toolchain_impl( implementation_name = "cpython", # See https://peps.python.org/pep-3147/ for pyc tag infix format pyc_tag = select({ - _IS_FREETHREADED: "cpython-{major}{minor}t".format(**version_dict), - "//conditions:default": "cpython-{major}{minor}".format(**version_dict), + _IS_FREETHREADED_YES: "cpython-{major}{minor}t".format(**version_dict), + _IS_FREETHREADED_NO: "cpython-{major}{minor}".format(**version_dict), }), ) diff --git a/python/private/pypi/config_settings.bzl b/python/private/pypi/config_settings.bzl index d1b85d16c1..3e828e59f5 100644 --- a/python/private/pypi/config_settings.bzl +++ b/python/private/pypi/config_settings.bzl @@ -80,8 +80,8 @@ FLAGS = struct( "is_pip_whl_auto", "is_pip_whl_no", "is_pip_whl_only", - "is_py_freethreaded", - "is_py_non_freethreaded", + "_is_py_freethreaded_yes", + "_is_py_freethreaded_no", "pip_whl_glibc_version", "pip_whl_muslc_version", "pip_whl_osx_arch", @@ -205,12 +205,12 @@ def _dist_config_settings(*, suffix, plat_flag_values, python_version, **kwargs) for name, f, compatible_with in [ ("py_none", _flags.whl, None), ("py3_none", _flags.whl_py3, None), - ("py3_abi3", _flags.whl_py3_abi3, (FLAGS.is_py_non_freethreaded,)), + ("py3_abi3", _flags.whl_py3_abi3, (FLAGS._is_py_freethreaded_no,)), ("none", _flags.whl_pycp3x, None), - ("abi3", _flags.whl_pycp3x_abi3, (FLAGS.is_py_non_freethreaded,)), + ("abi3", _flags.whl_pycp3x_abi3, (FLAGS._is_py_freethreaded_no,)), # The below are not specializations of one another, they are variants - (cpv, _flags.whl_pycp3x_abicp, (FLAGS.is_py_non_freethreaded,)), - (cpv + "t", _flags.whl_pycp3x_abicp, (FLAGS.is_py_freethreaded,)), + (cpv, _flags.whl_pycp3x_abicp, (FLAGS._is_py_freethreaded_no,)), + (cpv + "t", _flags.whl_pycp3x_abicp, (FLAGS._is_py_freethreaded_yes,)), ]: if (f, compatible_with) in used_flags: # This should never happen as all of the different whls should have @@ -237,12 +237,12 @@ def _dist_config_settings(*, suffix, plat_flag_values, python_version, **kwargs) for name, f, compatible_with in [ ("py_none", _flags.whl_plat, None), ("py3_none", _flags.whl_plat_py3, None), - ("py3_abi3", _flags.whl_plat_py3_abi3, (FLAGS.is_py_non_freethreaded,)), + ("py3_abi3", _flags.whl_plat_py3_abi3, (FLAGS._is_py_freethreaded_no,)), ("none", _flags.whl_plat_pycp3x, None), - ("abi3", _flags.whl_plat_pycp3x_abi3, (FLAGS.is_py_non_freethreaded,)), + ("abi3", _flags.whl_plat_pycp3x_abi3, (FLAGS._is_py_freethreaded_no,)), # The below are not specializations of one another, they are variants - (cpv, _flags.whl_plat_pycp3x_abicp, (FLAGS.is_py_non_freethreaded,)), - (cpv + "t", _flags.whl_plat_pycp3x_abicp, (FLAGS.is_py_freethreaded,)), + (cpv, _flags.whl_plat_pycp3x_abicp, (FLAGS._is_py_freethreaded_no,)), + (cpv + "t", _flags.whl_plat_pycp3x_abicp, (FLAGS._is_py_freethreaded_yes,)), ]: if (f, compatible_with) in used_flags: # This should never happen as all of the different whls should have @@ -329,7 +329,7 @@ def _dist_config_setting(*, name, compatible_with = None, native = native, **kwa compatible_with: {type}`tuple[Label]` A collection of config settings that are compatible with the given dist config setting. For example, if only non-freethreaded python builds are allowed, add - FLAGS.is_py_non_freethreaded here. + FLAGS._is_py_freethreaded_no here. native (struct): The struct containing alias and config_setting rules to use for creating the objects. Can be overridden for unit tests reasons. diff --git a/python/private/toolchain_aliases.bzl b/python/private/toolchain_aliases.bzl index 31ac4a8fdf..092863260c 100644 --- a/python/private/toolchain_aliases.bzl +++ b/python/private/toolchain_aliases.bzl @@ -14,7 +14,8 @@ """Create toolchain alias targets.""" -load("@rules_python//python:versions.bzl", "PLATFORMS") +load("@bazel_skylib//lib:selects.bzl", "selects") +load("//python:versions.bzl", "PLATFORMS") def toolchain_aliases(*, name, platforms, visibility = None, native = native): """Create toolchain aliases for the python toolchains. @@ -30,12 +31,17 @@ def toolchain_aliases(*, name, platforms, visibility = None, native = native): if platform not in platforms: continue + _platform = "_" + platform native.config_setting( - name = platform, - flag_values = PLATFORMS[platform].flag_values, + name = _platform, constraint_values = PLATFORMS[platform].compatible_with, visibility = ["//visibility:private"], ) + selects.config_setting_group( + name = platform, + match_all = PLATFORMS[platform].target_settings + [_platform], + visibility = ["//visibility:private"], + ) prefix = name for name in [ diff --git a/tests/toolchains/BUILD.bazel b/tests/toolchains/BUILD.bazel index f346651d46..b9952865cb 100644 --- a/tests/toolchains/BUILD.bazel +++ b/tests/toolchains/BUILD.bazel @@ -12,6 +12,7 @@ # See the License for the specific language governing permissions and # limitations under the License. +load("@bazel_skylib//rules:build_test.bzl", "build_test") load("//python/private:bzlmod_enabled.bzl", "BZLMOD_ENABLED") # buildifier: disable=bzl-visibility load("//tests/support:sh_py_run_test.bzl", "py_reconfig_test") load(":defs.bzl", "define_toolchain_tests") @@ -30,3 +31,10 @@ py_reconfig_test( "@platforms//cpu:x86_64", ] if BZLMOD_ENABLED else ["@platforms//:incompatible"], ) + +build_test( + name = "build_test", + targets = [ + "@python_3_11//:python_headers", + ], +) From 175a33610e853388c83730d9e2b5b2ac3626649d Mon Sep 17 00:00:00 2001 From: yushan26 <107004874+yushan26@users.noreply.github.com> Date: Wed, 18 Jun 2025 09:18:35 -0700 Subject: [PATCH 287/922] refactor(gazelle) Types for exposed members of `python.ParserOutput` are now all public (#2959) Export the members of `python.ParserOutput` struct to make it publicly accessible. This allows other `py` extensions to leverage the Python resolver logic for resolving Python imports, instead of have to duplicate the resolving logic. --------- Co-authored-by: yushan --- CHANGELOG.md | 21 +++++++++++++++++++++ gazelle/python/file_parser.go | 16 ++++++++-------- gazelle/python/file_parser_test.go | 22 +++++++++++----------- gazelle/python/generate.go | 2 +- gazelle/python/parser.go | 14 +++++++------- gazelle/python/resolve.go | 4 ++-- gazelle/python/std_modules.go | 2 +- gazelle/python/std_modules_test.go | 6 +++--- gazelle/python/target.go | 4 ++-- 9 files changed, 56 insertions(+), 35 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 488f1054a1..bf3d25c792 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -47,6 +47,27 @@ BEGIN_UNRELEASED_TEMPLATE END_UNRELEASED_TEMPLATE --> +{#v0-0-0} +## Unreleased + +[0.0.0]: https://github.com/bazel-contrib/rules_python/releases/tag/0.0.0 + +{#v0-0-0-changed} +### Changed +* (gazelle) Types for exposed members of `python.ParserOutput` are now all public. + +{#v0-0-0-fixed} +### Fixed +* Nothing fixed. + +{#v0-0-0-added} +### Added +* Nothing added. + +{#v0-0-0-removed} +### Removed +* Nothing removed. + {#1-5-0} ## [1.5.0] - 2025-06-11 diff --git a/gazelle/python/file_parser.go b/gazelle/python/file_parser.go index c147984fc3..3f8363fbdf 100644 --- a/gazelle/python/file_parser.go +++ b/gazelle/python/file_parser.go @@ -41,8 +41,8 @@ const ( type ParserOutput struct { FileName string - Modules []module - Comments []comment + Modules []Module + Comments []Comment HasMain bool } @@ -127,24 +127,24 @@ func (p *FileParser) parseMain(ctx context.Context, node *sitter.Node) bool { return false } -// parseImportStatement parses a node for an import statement, returning a `module` and a boolean +// parseImportStatement parses a node for an import statement, returning a `Module` and a boolean // representing if the parse was OK or not. -func parseImportStatement(node *sitter.Node, code []byte) (module, bool) { +func parseImportStatement(node *sitter.Node, code []byte) (Module, bool) { switch node.Type() { case sitterNodeTypeDottedName: - return module{ + return Module{ Name: node.Content(code), LineNumber: node.StartPoint().Row + 1, }, true case sitterNodeTypeAliasedImport: return parseImportStatement(node.Child(0), code) case sitterNodeTypeWildcardImport: - return module{ + return Module{ Name: "*", LineNumber: node.StartPoint().Row + 1, }, true } - return module{}, false + return Module{}, false } // parseImportStatements parses a node for import statements, returning true if the node is @@ -188,7 +188,7 @@ func (p *FileParser) parseImportStatements(node *sitter.Node) bool { // It updates FileParser.output.Comments with the parsed comment. func (p *FileParser) parseComments(node *sitter.Node) bool { if node.Type() == sitterNodeTypeComment { - p.output.Comments = append(p.output.Comments, comment(node.Content(p.code))) + p.output.Comments = append(p.output.Comments, Comment(node.Content(p.code))) return true } return false diff --git a/gazelle/python/file_parser_test.go b/gazelle/python/file_parser_test.go index 3682cff753..20085f0e76 100644 --- a/gazelle/python/file_parser_test.go +++ b/gazelle/python/file_parser_test.go @@ -27,7 +27,7 @@ func TestParseImportStatements(t *testing.T) { name string code string filepath string - result []module + result []Module }{ { name: "not has import", @@ -39,7 +39,7 @@ func TestParseImportStatements(t *testing.T) { name: "has import", code: "import unittest\nimport os.path\nfrom foo.bar import abc.xyz", filepath: "abc.py", - result: []module{ + result: []Module{ { Name: "unittest", LineNumber: 1, @@ -66,7 +66,7 @@ func TestParseImportStatements(t *testing.T) { import unittest `, filepath: "abc.py", - result: []module{ + result: []Module{ { Name: "unittest", LineNumber: 2, @@ -79,7 +79,7 @@ func TestParseImportStatements(t *testing.T) { name: "invalid syntax", code: "import os\nimport", filepath: "abc.py", - result: []module{ + result: []Module{ { Name: "os", LineNumber: 1, @@ -92,7 +92,7 @@ func TestParseImportStatements(t *testing.T) { name: "import as", code: "import os as b\nfrom foo import bar as c# 123", filepath: "abc.py", - result: []module{ + result: []Module{ { Name: "os", LineNumber: 1, @@ -111,7 +111,7 @@ func TestParseImportStatements(t *testing.T) { { name: "complex import", code: "from unittest import *\nfrom foo import (bar as c, baz, qux as d)\nfrom . import abc", - result: []module{ + result: []Module{ { Name: "unittest.*", LineNumber: 1, @@ -152,7 +152,7 @@ func TestParseComments(t *testing.T) { units := []struct { name string code string - result []comment + result []Comment }{ { name: "not has comment", @@ -162,17 +162,17 @@ func TestParseComments(t *testing.T) { { name: "has comment", code: "# a = 1\n# b = 2", - result: []comment{"# a = 1", "# b = 2"}, + result: []Comment{"# a = 1", "# b = 2"}, }, { name: "has comment in if", code: "if True:\n # a = 1\n # b = 2", - result: []comment{"# a = 1", "# b = 2"}, + result: []Comment{"# a = 1", "# b = 2"}, }, { name: "has comment inline", code: "import os# 123\nfrom pathlib import Path as b#456", - result: []comment{"# 123", "#456"}, + result: []Comment{"# 123", "#456"}, }, } for _, u := range units { @@ -248,7 +248,7 @@ func TestParseFull(t *testing.T) { output, err := p.Parse(context.Background()) assert.NoError(t, err) assert.Equal(t, ParserOutput{ - Modules: []module{{Name: "bar.abc", LineNumber: 1, Filepath: "foo/a.py", From: "bar"}}, + Modules: []Module{{Name: "bar.abc", LineNumber: 1, Filepath: "foo/a.py", From: "bar"}}, Comments: nil, HasMain: false, FileName: "a.py", diff --git a/gazelle/python/generate.go b/gazelle/python/generate.go index 27930c1025..5eedbd9601 100644 --- a/gazelle/python/generate.go +++ b/gazelle/python/generate.go @@ -471,7 +471,7 @@ func (py *Python) GenerateRules(args language.GenerateArgs) language.GenerateRes for _, pyTestTarget := range pyTestTargets { if conftest != nil { - pyTestTarget.addModuleDependency(module{Name: strings.TrimSuffix(conftestFilename, ".py")}) + pyTestTarget.addModuleDependency(Module{Name: strings.TrimSuffix(conftestFilename, ".py")}) } pyTest := pyTestTarget.build() diff --git a/gazelle/python/parser.go b/gazelle/python/parser.go index 1b2a90dddf..cf80578220 100644 --- a/gazelle/python/parser.go +++ b/gazelle/python/parser.go @@ -145,9 +145,9 @@ func removeDupesFromStringTreeSetSlice(array []string) []string { return dedupe } -// module represents a fully-qualified, dot-separated, Python module as seen on +// Module represents a fully-qualified, dot-separated, Python module as seen on // the import statement, alongside the line number where it happened. -type module struct { +type Module struct { // The fully-qualified, dot-separated, Python module name as seen on import // statements. Name string `json:"name"` @@ -162,7 +162,7 @@ type module struct { // moduleComparator compares modules by name. func moduleComparator(a, b interface{}) int { - return godsutils.StringComparator(a.(module).Name, b.(module).Name) + return godsutils.StringComparator(a.(Module).Name, b.(Module).Name) } // annotationKind represents Gazelle annotation kinds. @@ -176,12 +176,12 @@ const ( annotationKindIncludeDep annotationKind = "include_dep" ) -// comment represents a Python comment. -type comment string +// Comment represents a Python comment. +type Comment string // asAnnotation returns an annotation object if the comment has the // annotationPrefix. -func (c *comment) asAnnotation() (*annotation, error) { +func (c *Comment) asAnnotation() (*annotation, error) { uncomment := strings.TrimLeft(string(*c), "# ") if !strings.HasPrefix(uncomment, annotationPrefix) { return nil, nil @@ -215,7 +215,7 @@ type annotations struct { // annotationsFromComments returns all the annotations parsed out of the // comments of a Python module. -func annotationsFromComments(comments []comment) (*annotations, error) { +func annotationsFromComments(comments []Comment) (*annotations, error) { ignore := make(map[string]struct{}) includeDeps := []string{} for _, comment := range comments { diff --git a/gazelle/python/resolve.go b/gazelle/python/resolve.go index 7a2ec3d68a..996cbbadc0 100644 --- a/gazelle/python/resolve.go +++ b/gazelle/python/resolve.go @@ -151,7 +151,7 @@ func (py *Resolver) Resolve( hasFatalError := false MODULES_LOOP: for it.Next() { - mod := it.Value().(module) + mod := it.Value().(Module) moduleParts := strings.Split(mod.Name, ".") possibleModules := []string{mod.Name} for len(moduleParts) > 1 { @@ -214,7 +214,7 @@ func (py *Resolver) Resolve( matches := ix.FindRulesByImportWithConfig(c, imp, languageName) if len(matches) == 0 { // Check if the imported module is part of the standard library. - if isStdModule(module{Name: moduleName}) { + if isStdModule(Module{Name: moduleName}) { continue MODULES_LOOP } else if cfg.ValidateImportStatements() { err := fmt.Errorf( diff --git a/gazelle/python/std_modules.go b/gazelle/python/std_modules.go index e10f87b6ea..ecb4f4c454 100644 --- a/gazelle/python/std_modules.go +++ b/gazelle/python/std_modules.go @@ -34,7 +34,7 @@ func init() { } } -func isStdModule(m module) bool { +func isStdModule(m Module) bool { _, ok := stdModules[m.Name] return ok } diff --git a/gazelle/python/std_modules_test.go b/gazelle/python/std_modules_test.go index bc22638e69..dbcd18c9d6 100644 --- a/gazelle/python/std_modules_test.go +++ b/gazelle/python/std_modules_test.go @@ -21,7 +21,7 @@ import ( ) func TestIsStdModule(t *testing.T) { - assert.True(t, isStdModule(module{Name: "unittest"})) - assert.True(t, isStdModule(module{Name: "os.path"})) - assert.False(t, isStdModule(module{Name: "foo"})) + assert.True(t, isStdModule(Module{Name: "unittest"})) + assert.True(t, isStdModule(Module{Name: "os.path"})) + assert.False(t, isStdModule(Module{Name: "foo"})) } diff --git a/gazelle/python/target.go b/gazelle/python/target.go index c40d6fb3b7..1fb9218656 100644 --- a/gazelle/python/target.go +++ b/gazelle/python/target.go @@ -69,7 +69,7 @@ func (t *targetBuilder) addSrcs(srcs *treeset.Set) *targetBuilder { } // addModuleDependency adds a single module dep to the target. -func (t *targetBuilder) addModuleDependency(dep module) *targetBuilder { +func (t *targetBuilder) addModuleDependency(dep Module) *targetBuilder { fileName := dep.Name + ".py" if dep.From != "" { fileName = dep.From + ".py" @@ -87,7 +87,7 @@ func (t *targetBuilder) addModuleDependency(dep module) *targetBuilder { func (t *targetBuilder) addModuleDependencies(deps *treeset.Set) *targetBuilder { it := deps.Iterator() for it.Next() { - t.addModuleDependency(it.Value().(module)) + t.addModuleDependency(it.Value().(Module)) } return t } From 5b1db075d0810d09db7b1411c273a968ee3e4be0 Mon Sep 17 00:00:00 2001 From: Ignas Anikevicius <240938+aignas@users.noreply.github.com> Date: Thu, 19 Jun 2025 15:49:03 +0900 Subject: [PATCH 288/922] feat(pypi): pip.defaults API for customizing pipstar 1/n (#2987) Parse env markers in pip.parse using starlark Summary: - Allow switching to the Starlark implementation of the marker evaluation function. - Add a way for users to modify the `env` for the marker evaluation when parsing the requirements. This can only be done by `rules_python` or the root module. - Limit the platform selection when parsing the requirements files. Work towards #2747 Work towards #2949 Split out from #2909 --------- Co-authored-by: Richard Levasseur --- CHANGELOG.md | 4 +- python/private/pypi/BUILD.bazel | 5 +- python/private/pypi/env_marker_info.bzl | 2 +- python/private/pypi/evaluate_markers.bzl | 19 +- python/private/pypi/extension.bzl | 258 ++++++++++++++++-- python/private/pypi/pep508_evaluate.bzl | 2 +- python/private/pypi/pip_repository.bzl | 10 + .../pypi/requirements_files_by_platform.bzl | 54 ++-- tests/pypi/extension/extension_tests.bzl | 111 +++++++- .../requirements_files_by_platform_tests.bzl | 41 ++- 10 files changed, 437 insertions(+), 69 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index bf3d25c792..9897dc9ec8 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -62,7 +62,9 @@ END_UNRELEASED_TEMPLATE {#v0-0-0-added} ### Added -* Nothing added. +* (pypi) To configure the environment for `requirements.txt` evaluation, use the newly added + developer preview of the `pip.default` tag class. Only `rules_python` and root modules can use + this feature. {#v0-0-0-removed} ### Removed diff --git a/python/private/pypi/BUILD.bazel b/python/private/pypi/BUILD.bazel index d89dc6c228..b569b2217c 100644 --- a/python/private/pypi/BUILD.bazel +++ b/python/private/pypi/BUILD.bazel @@ -97,10 +97,10 @@ bzl_library( name = "evaluate_markers_bzl", srcs = ["evaluate_markers.bzl"], deps = [ - ":pep508_env_bzl", + ":deps_bzl", ":pep508_evaluate_bzl", - ":pep508_platform_bzl", ":pep508_requirement_bzl", + ":pypi_repo_utils_bzl", ], ) @@ -113,6 +113,7 @@ bzl_library( ":hub_repository_bzl", ":parse_requirements_bzl", ":parse_whl_name_bzl", + ":pep508_env_bzl", ":pip_repository_attrs_bzl", ":simpleapi_download_bzl", ":whl_config_setting_bzl", diff --git a/python/private/pypi/env_marker_info.bzl b/python/private/pypi/env_marker_info.bzl index c3c5ec69ed..37eefb2a0f 100644 --- a/python/private/pypi/env_marker_info.bzl +++ b/python/private/pypi/env_marker_info.bzl @@ -17,7 +17,7 @@ The {obj}`--//python/config_settings:pip_env_marker_config` flag. The values to use for environment markers when evaluating an expression. The keys and values should be compatible with the [PyPA dependency specifiers -specification](https://packaging.python.org/en/latest/specifications/dependency-specifiers/) +specification](https://packaging.python.org/en/latest/specifications/dependency-specifiers/). Missing values will be set to the specification's defaults or computed using available toolchain information. diff --git a/python/private/pypi/evaluate_markers.bzl b/python/private/pypi/evaluate_markers.bzl index 191933596e..58a29a9181 100644 --- a/python/private/pypi/evaluate_markers.bzl +++ b/python/private/pypi/evaluate_markers.bzl @@ -15,9 +15,7 @@ """A simple function that evaluates markers using a python interpreter.""" load(":deps.bzl", "record_files") -load(":pep508_env.bzl", "env") load(":pep508_evaluate.bzl", "evaluate") -load(":pep508_platform.bzl", "platform_from_str") load(":pep508_requirement.bzl", "requirement") load(":pypi_repo_utils.bzl", "pypi_repo_utils") @@ -30,22 +28,27 @@ SRCS = [ Label("//python/private/pypi/whl_installer:platform.py"), ] -def evaluate_markers(requirements, python_version = None): +def evaluate_markers(*, requirements, platforms): """Return the list of supported platforms per requirements line. Args: requirements: {type}`dict[str, list[str]]` of the requirement file lines to evaluate. - python_version: {type}`str | None` the version that can be used when evaluating the markers. + platforms: {type}`dict[str, dict[str, str]]` The environments that we for each requirement + file to evaluate. The keys between the platforms and requirements should be shared. Returns: dict of string lists with target platforms """ ret = {} - for req_string, platforms in requirements.items(): + for req_string, platform_strings in requirements.items(): req = requirement(req_string) - for platform in platforms: - if evaluate(req.marker, env = env(platform_from_str(platform, python_version))): - ret.setdefault(req_string, []).append(platform) + for platform_str in platform_strings: + env = platforms.get(platform_str) + if not env: + fail("Please define platform: '{}'".format(platform_str)) + + if evaluate(req.marker, env = env): + ret.setdefault(req_string, []).append(platform_str) return ret diff --git a/python/private/pypi/extension.bzl b/python/private/pypi/extension.bzl index 867abe0898..97b6825e51 100644 --- a/python/private/pypi/extension.bzl +++ b/python/private/pypi/extension.bzl @@ -25,10 +25,11 @@ load("//python/private:repo_utils.bzl", "repo_utils") load("//python/private:version.bzl", "version") load("//python/private:version_label.bzl", "version_label") load(":attrs.bzl", "use_isolated") -load(":evaluate_markers.bzl", "evaluate_markers_py", EVALUATE_MARKERS_SRCS = "SRCS") +load(":evaluate_markers.bzl", "evaluate_markers_py", EVALUATE_MARKERS_SRCS = "SRCS", evaluate_markers_star = "evaluate_markers") load(":hub_repository.bzl", "hub_repository", "whl_config_settings_to_json") load(":parse_requirements.bzl", "parse_requirements") load(":parse_whl_name.bzl", "parse_whl_name") +load(":pep508_env.bzl", "env") load(":pip_repository_attrs.bzl", "ATTRS") load(":requirements_files_by_platform.bzl", "requirements_files_by_platform") load(":simpleapi_download.bzl", "simpleapi_download") @@ -65,22 +66,36 @@ def _whl_mods_impl(whl_mods_dict): whl_mods = whl_mods, ) +def _platforms(*, python_version, minor_mapping, config): + platforms = {} + python_version = full_version( + version = python_version, + minor_mapping = minor_mapping, + ) + abi = "cp3{}".format(python_version[2:]) + + for platform, values in config.platforms.items(): + key = "{}_{}".format(abi, platform) + platforms[key] = env(key) | values.env + return platforms + def _create_whl_repos( module_ctx, *, pip_attr, whl_overrides, + config, available_interpreters = INTERPRETER_LABELS, minor_mapping = MINOR_MAPPING, - evaluate_markers = evaluate_markers_py, - get_index_urls = None, - enable_pipstar = False): + evaluate_markers = None, + get_index_urls = None): """create all of the whl repositories Args: module_ctx: {type}`module_ctx`. pip_attr: {type}`struct` - the struct that comes from the tag class iteration. whl_overrides: {type}`dict[str, struct]` - per-wheel overrides. + config: The platform configuration. get_index_urls: A function used to get the index URLs available_interpreters: {type}`dict[str, Label]` The dictionary of available interpreters that have been registered using the `python` bzlmod extension. @@ -89,7 +104,6 @@ def _create_whl_repos( minor_mapping: {type}`dict[str, str]` The dictionary needed to resolve the full python version used to parse package METADATA files. evaluate_markers: the function used to evaluate the markers. - enable_pipstar: enable the pipstar feature. Returns a {type}`struct` with the following attributes: whl_map: {type}`dict[str, list[struct]]` the output is keyed by the @@ -160,23 +174,19 @@ def _create_whl_repos( whl_group_mapping = {} requirement_cycles = {} - requirements_by_platform = parse_requirements( - module_ctx, - requirements_by_platform = requirements_files_by_platform( - requirements_by_platform = pip_attr.requirements_by_platform, - requirements_linux = pip_attr.requirements_linux, - requirements_lock = pip_attr.requirements_lock, - requirements_osx = pip_attr.requirements_darwin, - requirements_windows = pip_attr.requirements_windows, - extra_pip_args = pip_attr.extra_pip_args, - python_version = full_version( - version = pip_attr.python_version, + if evaluate_markers: + # This is most likely unit tests + pass + elif config.enable_pipstar: + evaluate_markers = lambda _, requirements: evaluate_markers_star( + requirements = requirements, + platforms = _platforms( + python_version = pip_attr.python_version, minor_mapping = minor_mapping, + config = config, ), - logger = logger, - ), - extra_pip_args = pip_attr.extra_pip_args, - get_index_urls = get_index_urls, + ) + else: # NOTE @aignas 2024-08-02: , we will execute any interpreter that we find either # in the PATH or if specified as a label. We will configure the env # markers when evaluating the requirement lines based on the output @@ -191,14 +201,34 @@ def _create_whl_repos( # instances to perform this manipulation. This function should be executed # only once by the underlying code to minimize the overhead needed to # spin up a Python interpreter. - evaluate_markers = lambda module_ctx, requirements: evaluate_markers( + evaluate_markers = lambda module_ctx, requirements: evaluate_markers_py( module_ctx, requirements = requirements, python_interpreter = pip_attr.python_interpreter, python_interpreter_target = python_interpreter_target, srcs = pip_attr._evaluate_markers_srcs, logger = logger, + ) + + requirements_by_platform = parse_requirements( + module_ctx, + requirements_by_platform = requirements_files_by_platform( + requirements_by_platform = pip_attr.requirements_by_platform, + requirements_linux = pip_attr.requirements_linux, + requirements_lock = pip_attr.requirements_lock, + requirements_osx = pip_attr.requirements_darwin, + requirements_windows = pip_attr.requirements_windows, + extra_pip_args = pip_attr.extra_pip_args, + platforms = sorted(config.platforms), # here we only need keys + python_version = full_version( + version = pip_attr.python_version, + minor_mapping = minor_mapping, + ), + logger = logger, ), + extra_pip_args = pip_attr.extra_pip_args, + get_index_urls = get_index_urls, + evaluate_markers = evaluate_markers, logger = logger, ) @@ -233,7 +263,7 @@ def _create_whl_repos( for p, args in whl_overrides.get(whl.name, {}).items() }, ) - if not enable_pipstar: + if not config.enable_pipstar: maybe_args["experimental_target_platforms"] = pip_attr.experimental_target_platforms whl_library_args.update({k: v for k, v in maybe_args.items() if v}) @@ -258,7 +288,7 @@ def _create_whl_repos( auth_patterns = pip_attr.auth_patterns, python_version = major_minor, is_multiple_versions = whl.is_multiple_versions, - enable_pipstar = enable_pipstar, + enable_pipstar = config.enable_pipstar, ) repo_name = "{}_{}".format(pip_name, repo.repo_name) @@ -342,16 +372,85 @@ def _whl_repo(*, src, whl_library_args, is_multiple_versions, download_only, net ), ) +def _configure(config, *, platform, os_name, arch_name, override = False, env = {}): + """Set the value in the config if the value is provided""" + config.setdefault("platforms", {}) + if platform: + if not override and config.get("platforms", {}).get(platform): + return + + for key in env: + if key not in _SUPPORTED_PEP508_KEYS: + fail("Unsupported key in the PEP508 environment: {}".format(key)) + + config["platforms"][platform] = struct( + name = platform.replace("-", "_").lower(), + os_name = os_name, + arch_name = arch_name, + env = env, + ) + else: + config["platforms"].pop(platform) + +def _create_config(defaults): + if defaults["platforms"]: + return struct(**defaults) + + # NOTE: We have this so that it is easier to maintain unit tests assuming certain + # defaults + for cpu in [ + "x86_64", + "aarch64", + # TODO @aignas 2025-05-19: only leave tier 0-1 cpus when stabilizing the + # `pip.default` extension. i.e. drop the below values - users will have to + # define themselves if they need them. + "arm", + "ppc", + "s390x", + ]: + _configure( + defaults, + arch_name = cpu, + os_name = "linux", + platform = "linux_{}".format(cpu), + env = {"platform_version": "0"}, + ) + for cpu in [ + "aarch64", + "x86_64", + ]: + _configure( + defaults, + arch_name = cpu, + # We choose the oldest non-EOL version at the time when we release `rules_python`. + # See https://endoflife.date/macos + env = {"platform_version": "14.0"}, + os_name = "osx", + platform = "osx_{}".format(cpu), + ) + + _configure( + defaults, + arch_name = "x86_64", + env = {"platform_version": "0"}, + os_name = "windows", + platform = "windows_x86_64", + ) + return struct(**defaults) + def parse_modules( module_ctx, _fail = fail, simpleapi_download = simpleapi_download, + enable_pipstar = False, **kwargs): """Implementation of parsing the tag classes for the extension and return a struct for registering repositories. Args: module_ctx: {type}`module_ctx` module context. simpleapi_download: Used for testing overrides + enable_pipstar: {type}`bool` a flag to enable dropping Python dependency for + evaluation of the extension. _fail: {type}`function` the failure function, mainly for testing. **kwargs: Extra arguments passed to the layers below. @@ -389,6 +488,34 @@ You cannot use both the additive_build_content and additive_build_content_file a srcs_exclude_glob = whl_mod.srcs_exclude_glob, ) + defaults = { + "enable_pipstar": enable_pipstar, + "platforms": {}, + } + for mod in module_ctx.modules: + if not (mod.is_root or mod.name == "rules_python"): + continue + + for tag in mod.tags.default: + _configure( + defaults, + arch_name = tag.arch_name, + env = tag.env, + os_name = tag.os_name, + platform = tag.platform, + override = mod.is_root, + # TODO @aignas 2025-05-19: add more attr groups: + # * for AUTH - the default `netrc` usage could be configured through a common + # attribute. + # * for index/downloader config. This includes all of those attributes for + # overrides, etc. Index overrides per platform could be also used here. + # * for whl selection - selecting preferences of which `platform_tag`s we should use + # for what. We could also model the `cp313t` freethreaded as separate platforms. + ) + + config = _create_config(defaults) + + # TODO @aignas 2025-06-03: Merge override API with the builder? _overriden_whl_set = {} whl_overrides = {} for module in module_ctx.modules: @@ -498,11 +625,13 @@ You cannot use both the additive_build_content and additive_build_content_file a elif pip_attr.experimental_index_url_overrides: fail("'experimental_index_url_overrides' is a no-op unless 'experimental_index_url' is set") + # TODO @aignas 2025-05-19: express pip.parse as a series of configure calls out = _create_whl_repos( module_ctx, pip_attr = pip_attr, get_index_urls = get_index_urls, whl_overrides = whl_overrides, + config = config, **kwargs ) hub_whl_map.setdefault(hub_name, {}) @@ -651,6 +780,72 @@ def _pip_impl(module_ctx): else: return None +_default_attrs = { + "arch_name": attr.string( + doc = """\ +The CPU architecture name to be used. + +:::{note} +Either this or {attr}`env` `platform_machine` key should be specified. +::: +""", + ), + "os_name": attr.string( + doc = """\ +The OS name to be used. + +:::{note} +Either this or the appropriate `env` keys should be specified. +::: +""", + ), + "platform": attr.string( + doc = """\ +A platform identifier which will be used as the unique identifier within the extension evaluation. +If you are defining custom platforms in your project and don't want things to clash, use extension +[isolation] feature. + +[isolation]: https://bazel.build/rules/lib/globals/module#use_extension.isolate +""", + ), +} | { + "env": attr.string_dict( + doc = """\ +The values to use for environment markers when evaluating an expression. + +The keys and values should be compatible with the [PyPA dependency specifiers +specification](https://packaging.python.org/en/latest/specifications/dependency-specifiers/). + +Missing values will be set to the specification's defaults or computed using +available toolchain information. + +Supported keys: +* `implementation_name`, defaults to `cpython`. +* `os_name`, defaults to a value inferred from the {attr}`os_name`. +* `platform_machine`, defaults to a value inferred from the {attr}`arch_name`. +* `platform_release`, defaults to an empty value. +* `platform_system`, defaults to a value inferred from the {attr}`os_name`. +* `platform_version`, defaults to `0`. +* `sys_platform`, defaults to a value inferred from the {attr}`os_name`. + +::::{note} +This is only used if the {envvar}`RULES_PYTHON_ENABLE_PIPSTAR` is enabled. +:::: +""", + ), + # The values for PEP508 env marker evaluation during the lock file parsing +} + +_SUPPORTED_PEP508_KEYS = [ + "implementation_name", + "os_name", + "platform_machine", + "platform_release", + "platform_system", + "platform_version", + "sys_platform", +] + def _pip_parse_ext_attrs(**kwargs): """Get the attributes for the pip extension. @@ -907,6 +1102,23 @@ the BUILD files for wheels. """, implementation = _pip_impl, tag_classes = { + "default": tag_class( + attrs = _default_attrs, + doc = """\ +This tag class allows for more customization of how the configuration for the hub repositories is built. + + +:::{include} /_includes/experimtal_api.md +::: + +:::{seealso} +The [environment markers][environment_markers] specification for the explanation of the +terms used in this extension. + +[environment_markers]: https://packaging.python.org/en/latest/specifications/dependency-specifiers/#environment-markers +::: +""", + ), "override": _override_tag, "parse": tag_class( attrs = _pip_parse_ext_attrs(), diff --git a/python/private/pypi/pep508_evaluate.bzl b/python/private/pypi/pep508_evaluate.bzl index d4492a75bb..fe2cac965a 100644 --- a/python/private/pypi/pep508_evaluate.bzl +++ b/python/private/pypi/pep508_evaluate.bzl @@ -117,7 +117,7 @@ def evaluate(marker, *, env, strict = True, **kwargs): Args: marker: {type}`str` The string marker to evaluate. - env: {type}`dict` The environment to evaluate the marker against. + env: {type}`dict[str, str]` The environment to evaluate the marker against. strict: {type}`bool` A setting to not fail on missing values in the env. **kwargs: Extra kwargs to be passed to the expression evaluator. diff --git a/python/private/pypi/pip_repository.bzl b/python/private/pypi/pip_repository.bzl index 724fb6ddba..e63bd6c3d1 100644 --- a/python/private/pypi/pip_repository.bzl +++ b/python/private/pypi/pip_repository.bzl @@ -80,6 +80,16 @@ def _pip_repository_impl(rctx): requirements_osx = rctx.attr.requirements_darwin, requirements_windows = rctx.attr.requirements_windows, extra_pip_args = rctx.attr.extra_pip_args, + platforms = [ + "linux_aarch64", + "linux_arm", + "linux_ppc", + "linux_s390x", + "linux_x86_64", + "osx_aarch64", + "osx_x86_64", + "windows_x86_64", + ], ), extra_pip_args = rctx.attr.extra_pip_args, evaluate_markers = lambda rctx, requirements: evaluate_markers_py( diff --git a/python/private/pypi/requirements_files_by_platform.bzl b/python/private/pypi/requirements_files_by_platform.bzl index 9165c05bed..d8d3651461 100644 --- a/python/private/pypi/requirements_files_by_platform.bzl +++ b/python/private/pypi/requirements_files_by_platform.bzl @@ -16,20 +16,7 @@ load(":whl_target_platforms.bzl", "whl_target_platforms") -# TODO @aignas 2024-05-13: consider using the same platform tags as are used in -# the //python:versions.bzl -DEFAULT_PLATFORMS = [ - "linux_aarch64", - "linux_arm", - "linux_ppc", - "linux_s390x", - "linux_x86_64", - "osx_aarch64", - "osx_x86_64", - "windows_x86_64", -] - -def _default_platforms(*, filter): +def _default_platforms(*, filter, platforms): if not filter: fail("Must specific a filter string, got: {}".format(filter)) @@ -48,11 +35,13 @@ def _default_platforms(*, filter): fail("The filter can only contain '*' at the end of it") if not prefix: - return DEFAULT_PLATFORMS + return platforms - return [p for p in DEFAULT_PLATFORMS if p.startswith(prefix)] + match = [p for p in platforms if p.startswith(prefix)] else: - return [p for p in DEFAULT_PLATFORMS if filter in p] + match = [p for p in platforms if filter in p] + + return match def _platforms_from_args(extra_pip_args): platform_values = [] @@ -105,6 +94,7 @@ def requirements_files_by_platform( requirements_linux = None, requirements_lock = None, requirements_windows = None, + platforms, extra_pip_args = None, python_version = None, logger = None, @@ -123,6 +113,8 @@ def requirements_files_by_platform( be joined with args fined in files. python_version: str or None. This is needed when the get_index_urls is specified. It should be of the form "3.x.x", + platforms: {type}`list[str]` the list of human-friendly platform labels that should + be used for the evaluation. logger: repo_utils.logger or None, a simple struct to log diagnostic messages. fail_fn (Callable[[str], None]): A failure function used in testing failure cases. @@ -144,11 +136,13 @@ def requirements_files_by_platform( ) return None - platforms = _platforms_from_args(extra_pip_args) + platforms_from_args = _platforms_from_args(extra_pip_args) if logger: - logger.debug(lambda: "Platforms from pip args: {}".format(platforms)) + logger.debug(lambda: "Platforms from pip args: {}".format(platforms_from_args)) + + default_platforms = [_platform(p, python_version) for p in platforms] - if platforms: + if platforms_from_args: lock_files = [ f for f in [ @@ -168,7 +162,7 @@ def requirements_files_by_platform( return None files_by_platform = [ - (lock_files[0], platforms), + (lock_files[0], platforms_from_args), ] if logger: logger.debug(lambda: "Files by platform with the platform set in the args: {}".format(files_by_platform)) @@ -177,7 +171,7 @@ def requirements_files_by_platform( file: [ platform for filter_or_platform in specifier.split(",") - for platform in (_default_platforms(filter = filter_or_platform) if filter_or_platform.endswith("*") else [filter_or_platform]) + for platform in (_default_platforms(filter = filter_or_platform, platforms = platforms) if filter_or_platform.endswith("*") else [filter_or_platform]) ] for file, specifier in requirements_by_platform.items() }.items() @@ -188,9 +182,9 @@ def requirements_files_by_platform( for f in [ # If the users need a greater span of the platforms, they should consider # using the 'requirements_by_platform' attribute. - (requirements_linux, _default_platforms(filter = "linux_*")), - (requirements_osx, _default_platforms(filter = "osx_*")), - (requirements_windows, _default_platforms(filter = "windows_*")), + (requirements_linux, _default_platforms(filter = "linux_*", platforms = platforms)), + (requirements_osx, _default_platforms(filter = "osx_*", platforms = platforms)), + (requirements_windows, _default_platforms(filter = "windows_*", platforms = platforms)), (requirements_lock, None), ]: if f[0]: @@ -215,8 +209,7 @@ def requirements_files_by_platform( return None configured_platforms[p] = file - else: - default_platforms = [_platform(p, python_version) for p in DEFAULT_PLATFORMS] + elif plats == None: plats = [ p for p in default_platforms @@ -231,6 +224,13 @@ def requirements_files_by_platform( for p in plats: configured_platforms[p] = file + elif logger: + logger.warn(lambda: "File {} will be ignored because there are no configured platforms: {}".format( + file, + default_platforms, + )) + continue + if logger: logger.debug(lambda: "Configured platforms for file {} are {}".format(file, plats)) diff --git a/tests/pypi/extension/extension_tests.bzl b/tests/pypi/extension/extension_tests.bzl index 8e325724f4..3d205a23c4 100644 --- a/tests/pypi/extension/extension_tests.bzl +++ b/tests/pypi/extension/extension_tests.bzl @@ -49,23 +49,22 @@ simple==0.0.1 \ ], ) -def _mod(*, name, parse = [], override = [], whl_mods = [], is_root = True): +def _mod(*, name, default = [], parse = [], override = [], whl_mods = [], is_root = True): return struct( name = name, tags = struct( parse = parse, override = override, whl_mods = whl_mods, + default = default, ), is_root = is_root, ) -def _parse_modules(env, **kwargs): +def _parse_modules(env, enable_pipstar = 0, **kwargs): return env.expect.that_struct( parse_modules( - # TODO @aignas 2025-05-11: start integration testing the branch which - # includes this. - enable_pipstar = 0, + enable_pipstar = enable_pipstar, **kwargs ), attrs = dict( @@ -77,6 +76,26 @@ def _parse_modules(env, **kwargs): ), ) +def _default( + arch_name = None, + constraint_values = None, + os_name = None, + platform = None, + target_settings = None, + env = None, + whl_limit = None, + whl_platforms = None): + return struct( + arch_name = arch_name, + constraint_values = constraint_values, + os_name = os_name, + platform = platform, + target_settings = target_settings, + env = env or {}, + whl_platforms = whl_platforms, + whl_limit = whl_limit, + ) + def _parse( *, hub_name, @@ -1023,6 +1042,88 @@ optimum[onnxruntime-gpu]==1.17.1 ; sys_platform == 'linux' _tests.append(_test_optimum_sys_platform_extra) +def _test_pipstar_platforms(env): + pypi = _parse_modules( + env, + module_ctx = _mock_mctx( + _mod( + name = "rules_python", + default = [ + _default( + platform = "{}_{}".format(os, cpu), + ) + for os, cpu in [ + ("linux", "x86_64"), + ("osx", "aarch64"), + ] + ], + parse = [ + _parse( + hub_name = "pypi", + python_version = "3.15", + requirements_lock = "universal.txt", + ), + ], + ), + read = lambda x: { + "universal.txt": """\ +optimum[onnxruntime]==1.17.1 ; sys_platform == 'darwin' +optimum[onnxruntime-gpu]==1.17.1 ; sys_platform == 'linux' +""", + }[x], + ), + enable_pipstar = True, + available_interpreters = { + "python_3_15_host": "unit_test_interpreter_target", + }, + minor_mapping = {"3.15": "3.15.19"}, + ) + + pypi.exposed_packages().contains_exactly({"pypi": ["optimum"]}) + pypi.hub_group_map().contains_exactly({"pypi": {}}) + pypi.hub_whl_map().contains_exactly({ + "pypi": { + "optimum": { + "pypi_315_optimum_linux_x86_64": [ + whl_config_setting( + version = "3.15", + target_platforms = [ + "cp315_linux_x86_64", + ], + config_setting = None, + filename = None, + ), + ], + "pypi_315_optimum_osx_aarch64": [ + whl_config_setting( + version = "3.15", + target_platforms = [ + "cp315_osx_aarch64", + ], + config_setting = None, + filename = None, + ), + ], + }, + }, + }) + + pypi.whl_libraries().contains_exactly({ + "pypi_315_optimum_linux_x86_64": { + "dep_template": "@pypi//{name}:{target}", + "python_interpreter_target": "unit_test_interpreter_target", + "requirement": "optimum[onnxruntime-gpu]==1.17.1", + }, + "pypi_315_optimum_osx_aarch64": { + "dep_template": "@pypi//{name}:{target}", + "python_interpreter_target": "unit_test_interpreter_target", + "requirement": "optimum[onnxruntime]==1.17.1", + }, + }) + pypi.whl_mods().contains_exactly({}) + +_tests.append(_test_pipstar_platforms) + def extension_test_suite(name): """Create the test suite. diff --git a/tests/pypi/requirements_files_by_platform/requirements_files_by_platform_tests.bzl b/tests/pypi/requirements_files_by_platform/requirements_files_by_platform_tests.bzl index b729b0eaf0..6688d72ffe 100644 --- a/tests/pypi/requirements_files_by_platform/requirements_files_by_platform_tests.bzl +++ b/tests/pypi/requirements_files_by_platform/requirements_files_by_platform_tests.bzl @@ -15,10 +15,27 @@ "" load("@rules_testing//lib:test_suite.bzl", "test_suite") -load("//python/private/pypi:requirements_files_by_platform.bzl", "requirements_files_by_platform") # buildifier: disable=bzl-visibility +load("//python/private/pypi:requirements_files_by_platform.bzl", _sut = "requirements_files_by_platform") # buildifier: disable=bzl-visibility _tests = [] +requirements_files_by_platform = lambda **kwargs: _sut( + platforms = kwargs.pop( + "platforms", + [ + "linux_aarch64", + "linux_arm", + "linux_ppc", + "linux_s390x", + "linux_x86_64", + "osx_aarch64", + "osx_x86_64", + "windows_x86_64", + ], + ), + **kwargs +) + def _test_fail_no_requirements(env): errors = [] requirements_files_by_platform( @@ -86,6 +103,28 @@ def _test_simple(env): _tests.append(_test_simple) +def _test_simple_limited(env): + for got in [ + requirements_files_by_platform( + requirements_lock = "requirements_lock", + platforms = ["linux_x86_64", "osx_x86_64"], + ), + requirements_files_by_platform( + requirements_by_platform = { + "requirements_lock": "*", + }, + platforms = ["linux_x86_64", "osx_x86_64"], + ), + ]: + env.expect.that_dict(got).contains_exactly({ + "requirements_lock": [ + "linux_x86_64", + "osx_x86_64", + ], + }) + +_tests.append(_test_simple_limited) + def _test_simple_with_python_version(env): for got in [ requirements_files_by_platform( From b8d6fa3f135fa7da2eed0c857bc25a43517f21fa Mon Sep 17 00:00:00 2001 From: Ignas Anikevicius <240938+aignas@users.noreply.github.com> Date: Fri, 20 Jun 2025 09:46:07 +0900 Subject: [PATCH 289/922] feat(pypi): pip.defaults API for customizing repo selection 2/n (#2988) WIP: stacked on #2987 This is adding `constraint_values` attribute to `pip.configure` and is threading it all the way down to the generation of `BUILD.bazel` file of for config settings used in the hub repository. Out of scope: - Passing `flag_values` or target settings. I am torn about it - doing it in this PR would flesh out the design more, but at the same time it might become harder to review. - `whl_target_platforms` and `select_whls` is still unchanged, not sure if it is related to this attribute addition. Work towards #2747 Work towards #2548 Work towards #260 --------- Co-authored-by: Richard Levasseur --- CHANGELOG.md | 2 +- python/private/pypi/config_settings.bzl | 31 +++++++------- python/private/pypi/extension.bzl | 34 +++++++++++++-- python/private/pypi/hub_repository.bzl | 5 +++ python/private/pypi/render_pkg_aliases.bzl | 12 ++++-- .../config_settings/config_settings_tests.bzl | 39 +++++++++++++---- tests/pypi/extension/extension_tests.bzl | 4 ++ tests/pypi/pkg_aliases/pkg_aliases_test.bzl | 42 +++++++++++++++---- .../render_pkg_aliases_test.bzl | 13 +++++- 9 files changed, 140 insertions(+), 42 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 9897dc9ec8..da3dcc8efc 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -64,7 +64,7 @@ END_UNRELEASED_TEMPLATE ### Added * (pypi) To configure the environment for `requirements.txt` evaluation, use the newly added developer preview of the `pip.default` tag class. Only `rules_python` and root modules can use - this feature. + this feature. You can also configure `constraint_values` using `pip.default`. {#v0-0-0-removed} ### Removed diff --git a/python/private/pypi/config_settings.bzl b/python/private/pypi/config_settings.bzl index 3e828e59f5..7edc578d7a 100644 --- a/python/private/pypi/config_settings.bzl +++ b/python/private/pypi/config_settings.bzl @@ -111,8 +111,8 @@ def config_settings( glibc_versions = [], muslc_versions = [], osx_versions = [], - target_platforms = [], name = None, + platform_constraint_values = {}, **kwargs): """Generate all of the pip config settings. @@ -126,8 +126,10 @@ def config_settings( configure config settings for. osx_versions (list[str]): The list of OSX OS versions to configure config settings for. - target_platforms (list[str]): The list of "{os}_{cpu}" for deriving - constraint values for each condition. + platform_constraint_values: {type}`dict[str, list[str]]` the constraint + values to use instead of the default ones. Key are platform names + (a human-friendly platform string). Values are lists of + `constraint_value` label strings. **kwargs: Other args passed to the underlying implementations, such as {obj}`native`. """ @@ -135,22 +137,17 @@ def config_settings( glibc_versions = [""] + glibc_versions muslc_versions = [""] + muslc_versions osx_versions = [""] + osx_versions - target_platforms = [("", ""), ("osx", "universal2")] + [ - t.split("_", 1) - for t in target_platforms - ] + target_platforms = { + "": [], + # TODO @aignas 2025-06-15: allowing universal2 and platform specific wheels in one + # closure is making things maybe a little bit too complicated. + "osx_universal2": ["@platforms//os:osx"], + } | platform_constraint_values for python_version in python_versions: - for os, cpu in target_platforms: - constraint_values = [] - suffix = "" - if os: - constraint_values.append("@platforms//os:" + os) - suffix += "_" + os - if cpu: - suffix += "_" + cpu - if cpu != "universal2": - constraint_values.append("@platforms//cpu:" + cpu) + for platform_name, constraint_values in target_platforms.items(): + suffix = "_{}".format(platform_name) if platform_name else "" + os, _, cpu = platform_name.partition("_") _dist_config_settings( suffix = suffix, diff --git a/python/private/pypi/extension.bzl b/python/private/pypi/extension.bzl index 97b6825e51..78511b4c27 100644 --- a/python/private/pypi/extension.bzl +++ b/python/private/pypi/extension.bzl @@ -372,7 +372,7 @@ def _whl_repo(*, src, whl_library_args, is_multiple_versions, download_only, net ), ) -def _configure(config, *, platform, os_name, arch_name, override = False, env = {}): +def _configure(config, *, platform, os_name, arch_name, constraint_values, env = {}, override = False): """Set the value in the config if the value is provided""" config.setdefault("platforms", {}) if platform: @@ -387,6 +387,7 @@ def _configure(config, *, platform, os_name, arch_name, override = False, env = name = platform.replace("-", "_").lower(), os_name = os_name, arch_name = arch_name, + constraint_values = constraint_values, env = env, ) else: @@ -413,6 +414,10 @@ def _create_config(defaults): arch_name = cpu, os_name = "linux", platform = "linux_{}".format(cpu), + constraint_values = [ + "@platforms//os:linux", + "@platforms//cpu:{}".format(cpu), + ], env = {"platform_version": "0"}, ) for cpu in [ @@ -424,17 +429,25 @@ def _create_config(defaults): arch_name = cpu, # We choose the oldest non-EOL version at the time when we release `rules_python`. # See https://endoflife.date/macos - env = {"platform_version": "14.0"}, os_name = "osx", platform = "osx_{}".format(cpu), + constraint_values = [ + "@platforms//os:osx", + "@platforms//cpu:{}".format(cpu), + ], + env = {"platform_version": "14.0"}, ) _configure( defaults, arch_name = "x86_64", - env = {"platform_version": "0"}, os_name = "windows", platform = "windows_x86_64", + constraint_values = [ + "@platforms//os:windows", + "@platforms//cpu:x86_64", + ], + env = {"platform_version": "0"}, ) return struct(**defaults) @@ -500,6 +513,7 @@ You cannot use both the additive_build_content and additive_build_content_file a _configure( defaults, arch_name = tag.arch_name, + constraint_values = tag.constraint_values, env = tag.env, os_name = tag.os_name, platform = tag.platform, @@ -679,6 +693,13 @@ You cannot use both the additive_build_content and additive_build_content_file a } for hub_name, extra_whl_aliases in extra_aliases.items() }, + platform_constraint_values = { + hub_name: { + platform_name: sorted([str(Label(cv)) for cv in p.constraint_values]) + for platform_name, p in config.platforms.items() + } + for hub_name in hub_whl_map + }, whl_libraries = { k: dict(sorted(args.items())) for k, args in sorted(whl_libraries.items()) @@ -769,6 +790,7 @@ def _pip_impl(module_ctx): for key, values in whl_map.items() }, packages = mods.exposed_packages.get(hub_name, []), + platform_constraint_values = mods.platform_constraint_values.get(hub_name, {}), groups = mods.hub_group_map.get(hub_name), ) @@ -788,6 +810,12 @@ The CPU architecture name to be used. :::{note} Either this or {attr}`env` `platform_machine` key should be specified. ::: +""", + ), + "constraint_values": attr.label_list( + mandatory = True, + doc = """\ +The constraint_values to use in select statements. """, ), "os_name": attr.string( diff --git a/python/private/pypi/hub_repository.bzl b/python/private/pypi/hub_repository.bzl index 0dbc6c29c2..4398d7b597 100644 --- a/python/private/pypi/hub_repository.bzl +++ b/python/private/pypi/hub_repository.bzl @@ -34,6 +34,7 @@ def _impl(rctx): }, extra_hub_aliases = rctx.attr.extra_hub_aliases, requirement_cycles = rctx.attr.groups, + platform_constraint_values = rctx.attr.platform_constraint_values, ) for path, contents in aliases.items(): rctx.file(path, contents) @@ -83,6 +84,10 @@ hub_repository = repository_rule( The list of packages that will be exposed via all_*requirements macros. Defaults to whl_map keys. """, ), + "platform_constraint_values": attr.string_list_dict( + doc = "The constraint values for each platform name. The values are string canonical string Label representations", + mandatory = False, + ), "repo_name": attr.string( mandatory = True, doc = "The apparent name of the repo. This is needed because in bzlmod, the name attribute becomes the canonical name.", diff --git a/python/private/pypi/render_pkg_aliases.bzl b/python/private/pypi/render_pkg_aliases.bzl index 28f32edc78..267d7ce85d 100644 --- a/python/private/pypi/render_pkg_aliases.bzl +++ b/python/private/pypi/render_pkg_aliases.bzl @@ -155,12 +155,14 @@ def _major_minor_versions(python_versions): # Use a dict as a simple set return sorted({_major_minor(v): None for v in python_versions}) -def render_multiplatform_pkg_aliases(*, aliases, **kwargs): +def render_multiplatform_pkg_aliases(*, aliases, platform_constraint_values = {}, **kwargs): """Render the multi-platform pkg aliases. Args: aliases: dict[str, list(whl_config_setting)] A list of aliases that will be transformed from ones having `filename` to ones having `config_setting`. + platform_constraint_values: {type}`dict[str, list[str]]` contains all of the + target platforms and their appropriate `constraint_values`. **kwargs: extra arguments passed to render_pkg_aliases. Returns: @@ -187,18 +189,22 @@ def render_multiplatform_pkg_aliases(*, aliases, **kwargs): muslc_versions = flag_versions.get("muslc_versions", []), osx_versions = flag_versions.get("osx_versions", []), python_versions = _major_minor_versions(flag_versions.get("python_versions", [])), - target_platforms = flag_versions.get("target_platforms", []), + platform_constraint_values = platform_constraint_values, visibility = ["//:__subpackages__"], ) return contents -def _render_config_settings(**kwargs): +def _render_config_settings(platform_constraint_values, **kwargs): return """\ load("@rules_python//python/private/pypi:config_settings.bzl", "config_settings") {}""".format(render.call( "config_settings", name = repr("config_settings"), + platform_constraint_values = render.dict( + platform_constraint_values, + value_repr = render.list, + ), **_repr_dict(value_repr = render.list, **kwargs) )) diff --git a/tests/pypi/config_settings/config_settings_tests.bzl b/tests/pypi/config_settings/config_settings_tests.bzl index f111d0c55c..9551d42d10 100644 --- a/tests/pypi/config_settings/config_settings_tests.bzl +++ b/tests/pypi/config_settings/config_settings_tests.bzl @@ -657,13 +657,34 @@ def config_settings_test_suite(name): # buildifier: disable=function-docstring glibc_versions = [(2, 14), (2, 17)], muslc_versions = [(1, 1)], osx_versions = [(10, 9), (11, 0)], - target_platforms = [ - "windows_x86_64", - "windows_aarch64", - "linux_x86_64", - "linux_ppc", - "linux_aarch64", - "osx_x86_64", - "osx_aarch64", - ], + platform_constraint_values = { + "linux_aarch64": [ + "@platforms//cpu:aarch64", + "@platforms//os:linux", + ], + "linux_ppc": [ + "@platforms//cpu:ppc", + "@platforms//os:linux", + ], + "linux_x86_64": [ + "@platforms//cpu:x86_64", + "@platforms//os:linux", + ], + "osx_aarch64": [ + "@platforms//cpu:aarch64", + "@platforms//os:osx", + ], + "osx_x86_64": [ + "@platforms//cpu:x86_64", + "@platforms//os:osx", + ], + "windows_aarch64": [ + "@platforms//cpu:aarch64", + "@platforms//os:windows", + ], + "windows_x86_64": [ + "@platforms//cpu:x86_64", + "@platforms//os:windows", + ], + }, ) diff --git a/tests/pypi/extension/extension_tests.bzl b/tests/pypi/extension/extension_tests.bzl index 3d205a23c4..231e8cab41 100644 --- a/tests/pypi/extension/extension_tests.bzl +++ b/tests/pypi/extension/extension_tests.bzl @@ -1051,6 +1051,10 @@ def _test_pipstar_platforms(env): default = [ _default( platform = "{}_{}".format(os, cpu), + constraint_values = [ + "@platforms//os:{}".format(os), + "@platforms//cpu:{}".format(cpu), + ], ) for os, cpu in [ ("linux", "x86_64"), diff --git a/tests/pypi/pkg_aliases/pkg_aliases_test.bzl b/tests/pypi/pkg_aliases/pkg_aliases_test.bzl index 71ca811fee..0fbcd4e7a6 100644 --- a/tests/pypi/pkg_aliases/pkg_aliases_test.bzl +++ b/tests/pypi/pkg_aliases/pkg_aliases_test.bzl @@ -419,10 +419,16 @@ def _test_config_settings_exist_legacy(env): alias = _mock_alias(available_config_settings), config_setting = _mock_config_setting(available_config_settings), ), - target_platforms = [ - "linux_aarch64", - "linux_x86_64", - ], + platform_constraint_values = { + "linux_aarch64": [ + "@platforms//cpu:aarch64", + "@platforms//os:linux", + ], + "linux_x86_64": [ + "@platforms//cpu:x86_64", + "@platforms//os:linux", + ], + }, ) got_aliases = multiplatform_whl_aliases( @@ -448,19 +454,39 @@ def _test_config_settings_exist(env): "any": {}, "macosx_11_0_arm64": { "osx_versions": [(11, 0)], - "target_platforms": ["osx_aarch64"], + "platform_constraint_values": { + "osx_aarch64": [ + "@platforms//cpu:aarch64", + "@platforms//os:osx", + ], + }, }, "manylinux_2_17_x86_64": { "glibc_versions": [(2, 17), (2, 18)], - "target_platforms": ["linux_x86_64"], + "platform_constraint_values": { + "linux_x86_64": [ + "@platforms//cpu:x86_64", + "@platforms//os:linux", + ], + }, }, "manylinux_2_18_x86_64": { "glibc_versions": [(2, 17), (2, 18)], - "target_platforms": ["linux_x86_64"], + "platform_constraint_values": { + "linux_x86_64": [ + "@platforms//cpu:x86_64", + "@platforms//os:linux", + ], + }, }, "musllinux_1_1_aarch64": { "muslc_versions": [(1, 2), (1, 1), (1, 0)], - "target_platforms": ["linux_aarch64"], + "platform_constraint_values": { + "linux_aarch64": [ + "@platforms//cpu:aarch64", + "@platforms//os:linux", + ], + }, }, }.items(): aliases = { diff --git a/tests/pypi/render_pkg_aliases/render_pkg_aliases_test.bzl b/tests/pypi/render_pkg_aliases/render_pkg_aliases_test.bzl index 416d50bd80..c262ed6823 100644 --- a/tests/pypi/render_pkg_aliases/render_pkg_aliases_test.bzl +++ b/tests/pypi/render_pkg_aliases/render_pkg_aliases_test.bzl @@ -93,6 +93,12 @@ def _test_bzlmod_aliases(env): }, }, extra_hub_aliases = {"bar_baz": ["foo"]}, + platform_constraint_values = { + "linux_x86_64": [ + "@platforms//os:linux", + "@platforms//cpu:x86_64", + ], + }, ) want_key = "bar_baz/BUILD.bazel" @@ -130,8 +136,13 @@ load("@rules_python//python/private/pypi:config_settings.bzl", "config_settings" config_settings( name = "config_settings", + platform_constraint_values = { + "linux_x86_64": [ + "@platforms//os:linux", + "@platforms//cpu:x86_64", + ], + }, python_versions = ["3.2"], - target_platforms = ["linux_x86_64"], visibility = ["//:__subpackages__"], )""", ) From c4543cd193752d0248226dcd07cc027e63ed7b8b Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Thu, 19 Jun 2025 23:28:52 -0700 Subject: [PATCH 290/922] fix(toolchains): use posix-compatible exec -a alternative (#3010) The `exec -a` command doesn't work in dash, the default shell for Ubuntu/debian. To work around, use `sh -c`, which is posix and dash compatible. This allows changing the argv0 while invoking a different command. Also adds a test to verify the the runtime_env toolchain works with bootstrap script. Fixes https://github.com/bazel-contrib/rules_python/issues/3009 --- .../runtime_env_toolchain_interpreter.sh | 13 ++++++----- tests/runtime_env_toolchain/BUILD.bazel | 23 +++++++++++++++++++ .../toolchain_runs_test.py | 9 ++++++++ tests/support/sh_py_run_test.bzl | 6 +++++ 4 files changed, 45 insertions(+), 6 deletions(-) diff --git a/python/private/runtime_env_toolchain_interpreter.sh b/python/private/runtime_env_toolchain_interpreter.sh index 7b3ec598b2..dd4d648d12 100755 --- a/python/private/runtime_env_toolchain_interpreter.sh +++ b/python/private/runtime_env_toolchain_interpreter.sh @@ -71,14 +71,15 @@ if [ -e "$self_dir/pyvenv.cfg" ] || [ -e "$self_dir/../pyvenv.cfg" ]; then if [ ! -e "$PYTHON_BIN" ]; then die "ERROR: Python interpreter does not exist: $PYTHON_BIN" fi - # PYTHONEXECUTABLE is also used because `exec -a` doesn't fully trick the - # pyenv wrappers. + # PYTHONEXECUTABLE is also used because switching argv0 doesn't fully trick + # the pyenv wrappers. # NOTE: The PYTHONEXECUTABLE envvar only works for non-Mac starting in Python 3.11 export PYTHONEXECUTABLE="$venv_bin" - # Python looks at argv[0] to determine sys.executable, so use exec -a - # to make it think it's the venv's binary, not the actual one invoked. - # NOTE: exec -a isn't strictly posix-compatible, but very widespread - exec -a "$venv_bin" "$PYTHON_BIN" "$@" + # Python looks at argv[0] to determine sys.executable, so set that to the venv + # binary, not the actual one invoked. + # NOTE: exec -a would be simpler, but isn't posix-compatible, and dash shell + # (Ubuntu/debian default) doesn't support it; see #3009. + exec sh -c "$PYTHON_BIN \$@" "$venv_bin" "$@" else exec "$PYTHON_BIN" "$@" fi diff --git a/tests/runtime_env_toolchain/BUILD.bazel b/tests/runtime_env_toolchain/BUILD.bazel index 2f82d204ff..f1bda251f9 100644 --- a/tests/runtime_env_toolchain/BUILD.bazel +++ b/tests/runtime_env_toolchain/BUILD.bazel @@ -40,3 +40,26 @@ py_reconfig_test( tags = ["no-remote-exec"], deps = ["//python/runfiles"], ) + +py_reconfig_test( + name = "bootstrap_script_test", + srcs = ["toolchain_runs_test.py"], + bootstrap_impl = "script", + data = [ + "//tests/support:current_build_settings", + ], + extra_toolchains = [ + "//python/runtime_env_toolchains:all", + # Necessary for RBE CI + CC_TOOLCHAIN, + ], + main = "toolchain_runs_test.py", + # With bootstrap=script, the build version must match the runtime version + # because the venv has the version in the lib/site-packages dir name. + python_version = PYTHON_VERSION, + # Our RBE has Python 3.6, which is too old for the language features + # we use now. Using the runtime-env toolchain on RBE is pretty + # questionable anyways. + tags = ["no-remote-exec"], + deps = ["//python/runfiles"], +) diff --git a/tests/runtime_env_toolchain/toolchain_runs_test.py b/tests/runtime_env_toolchain/toolchain_runs_test.py index 7be2472e8b..c66b0bbd8a 100644 --- a/tests/runtime_env_toolchain/toolchain_runs_test.py +++ b/tests/runtime_env_toolchain/toolchain_runs_test.py @@ -1,6 +1,7 @@ import json import pathlib import platform +import sys import unittest from python.runfiles import runfiles @@ -23,6 +24,14 @@ def test_ran(self): settings["interpreter"]["short_path"], ) + if settings["bootstrap_impl"] == "script": + # Verify we're running in a venv + self.assertNotEqual(sys.prefix, sys.base_prefix) + # .venv/ occurs for a build-time venv. + # For a runtime created venv, it goes into a temp dir, so + # look for the /bin/ dir as an indicator. + self.assertRegex(sys.executable, r"[.]venv/|/bin/") + if __name__ == "__main__": unittest.main() diff --git a/tests/support/sh_py_run_test.bzl b/tests/support/sh_py_run_test.bzl index 69141fe8a4..49445ed304 100644 --- a/tests/support/sh_py_run_test.bzl +++ b/tests/support/sh_py_run_test.bzl @@ -135,6 +135,7 @@ def _current_build_settings_impl(ctx): ctx.actions.write( output = info, content = json.encode({ + "bootstrap_impl": ctx.attr._bootstrap_impl_flag[config_common.FeatureFlagInfo].value, "interpreter": { "short_path": runtime.interpreter.short_path if runtime.interpreter else None, }, @@ -153,6 +154,11 @@ Writes information about the current build config to JSON for testing. This is so tests can verify information about the build config used for them. """, implementation = _current_build_settings_impl, + attrs = { + "_bootstrap_impl_flag": attr.label( + default = "//python/config_settings:bootstrap_impl", + ), + }, toolchains = [ TARGET_TOOLCHAIN_TYPE, ], From b924c43e0fadc78fe8de7d91c318c5299c8ab68b Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Thu, 19 Jun 2025 23:56:13 -0700 Subject: [PATCH 291/922] build(deps): bump urllib3 from 2.4.0 to 2.5.0 in /tools/publish (#3008) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Bumps [urllib3](https://github.com/urllib3/urllib3) from 2.4.0 to 2.5.0.
Release notes

Sourced from urllib3's releases.

2.5.0

🚀 urllib3 is fundraising for HTTP/2 support

urllib3 is raising ~$40,000 USD to release HTTP/2 support and ensure long-term sustainable maintenance of the project after a sharp decline in financial support. If your company or organization uses Python and would benefit from HTTP/2 support in Requests, pip, cloud SDKs, and thousands of other projects please consider contributing financially to ensure HTTP/2 support is developed sustainably and maintained for the long-haul.

Thank you for your support.

Security issues

urllib3 2.5.0 fixes two moderate security issues:

  • Pool managers now properly control redirects when retries is passed — CVE-2025-50181 reported by @​sandumjacob (5.3 Medium, GHSA-pq67-6m6q-mj2v)
  • Redirects are now controlled by urllib3 in the Node.js runtime — CVE-2025-50182 (5.3 Medium, GHSA-48p4-8xcf-vxj5)

Features

  • Added support for the compression.zstd module that is new in Python 3.14. See PEP 784 for more information. (#3610)
  • Added support for version 0.5 of hatch-vcs (#3612)

Bugfixes

  • Raised exception for HTTPResponse.shutdown on a connection already released to the pool. (#3581)
  • Fixed incorrect CONNECT statement when using an IPv6 proxy with connection_from_host. Previously would not be wrapped in []. (#3615)
Changelog

Sourced from urllib3's changelog.

2.5.0 (2025-06-18)

Features

  • Added support for the compression.zstd module that is new in Python 3.14. See PEP 784 <https://peps.python.org/pep-0784/>_ for more information. ([#3610](https://github.com/urllib3/urllib3/issues/3610) <https://github.com/urllib3/urllib3/issues/3610>__)
  • Added support for version 0.5 of hatch-vcs ([#3612](https://github.com/urllib3/urllib3/issues/3612) <https://github.com/urllib3/urllib3/issues/3612>__)

Bugfixes

  • Fixed a security issue where restricting the maximum number of followed redirects at the urllib3.PoolManager level via the retries parameter did not work.
  • Made the Node.js runtime respect redirect parameters such as retries and redirects.
  • Raised exception for HTTPResponse.shutdown on a connection already released to the pool. ([#3581](https://github.com/urllib3/urllib3/issues/3581) <https://github.com/urllib3/urllib3/issues/3581>__)
  • Fixed incorrect CONNECT statement when using an IPv6 proxy with connection_from_host. Previously would not be wrapped in []. ([#3615](https://github.com/urllib3/urllib3/issues/3615) <https://github.com/urllib3/urllib3/issues/3615>__)
Commits
  • aaab4ec Release 2.5.0
  • 7eb4a2a Merge commit from fork
  • f05b132 Merge commit from fork
  • d03fe32 Fix HTTP tunneling with IPv6 in older Python versions
  • 11661e9 Bump github/codeql-action from 3.28.0 to 3.29.0 (#3624)
  • 6a0ecc6 Update v2 migration guide to 2.4.0 (#3621)
  • 8e32e60 Raise exception for shutdown on a connection already released to the pool (#3...
  • 9996e0f Fix emscripten CI for Chrome 137+ (#3599)
  • 4fd1a99 Bump RECENT_DATE (#3617)
  • c4b5917 Add support for the new compression.zstd module in Python 3.14 (#3611)
  • Additional commits viewable in compare view

[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=urllib3&package-manager=pip&previous-version=2.4.0&new-version=2.5.0)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) ---
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot show ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) You can disable automated security fix PRs for this repo from the [Security Alerts page](https://github.com/bazel-contrib/rules_python/network/alerts).
Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> --- tools/publish/requirements_darwin.txt | 6 +++--- tools/publish/requirements_linux.txt | 6 +++--- tools/publish/requirements_universal.txt | 6 +++--- tools/publish/requirements_windows.txt | 6 +++--- 4 files changed, 12 insertions(+), 12 deletions(-) diff --git a/tools/publish/requirements_darwin.txt b/tools/publish/requirements_darwin.txt index af5bad246d..58973acb6f 100644 --- a/tools/publish/requirements_darwin.txt +++ b/tools/publish/requirements_darwin.txt @@ -202,9 +202,9 @@ twine==5.1.1 \ --hash=sha256:215dbe7b4b94c2c50a7315c0275d2258399280fbb7d04182c7e55e24b5f93997 \ --hash=sha256:9aa0825139c02b3434d913545c7b847a21c835e11597f5255842d457da2322db # via -r tools/publish/requirements.in -urllib3==2.4.0 \ - --hash=sha256:414bc6535b787febd7567804cc015fee39daab8ad86268f1310a9250697de466 \ - --hash=sha256:4e16665048960a0900c702d4a66415956a584919c03361cac9f1df5c5dd7e813 +urllib3==2.5.0 \ + --hash=sha256:3fc47733c7e419d4bc3f6b3dc2b4f890bb743906a30d56ba4a5bfa4bbff92760 \ + --hash=sha256:e6b01673c0fa6a13e374b50871808eb3bf7046c4b125b216f6bf1cc604cff0dc # via # requests # twine diff --git a/tools/publish/requirements_linux.txt b/tools/publish/requirements_linux.txt index b2e9ccf5ab..73edfce02f 100644 --- a/tools/publish/requirements_linux.txt +++ b/tools/publish/requirements_linux.txt @@ -318,9 +318,9 @@ twine==5.1.1 \ --hash=sha256:215dbe7b4b94c2c50a7315c0275d2258399280fbb7d04182c7e55e24b5f93997 \ --hash=sha256:9aa0825139c02b3434d913545c7b847a21c835e11597f5255842d457da2322db # via -r tools/publish/requirements.in -urllib3==2.4.0 \ - --hash=sha256:414bc6535b787febd7567804cc015fee39daab8ad86268f1310a9250697de466 \ - --hash=sha256:4e16665048960a0900c702d4a66415956a584919c03361cac9f1df5c5dd7e813 +urllib3==2.5.0 \ + --hash=sha256:3fc47733c7e419d4bc3f6b3dc2b4f890bb743906a30d56ba4a5bfa4bbff92760 \ + --hash=sha256:e6b01673c0fa6a13e374b50871808eb3bf7046c4b125b216f6bf1cc604cff0dc # via # requests # twine diff --git a/tools/publish/requirements_universal.txt b/tools/publish/requirements_universal.txt index 8a7426e517..c080f1d7de 100644 --- a/tools/publish/requirements_universal.txt +++ b/tools/publish/requirements_universal.txt @@ -322,9 +322,9 @@ twine==5.1.1 \ --hash=sha256:215dbe7b4b94c2c50a7315c0275d2258399280fbb7d04182c7e55e24b5f93997 \ --hash=sha256:9aa0825139c02b3434d913545c7b847a21c835e11597f5255842d457da2322db # via -r tools/publish/requirements.in -urllib3==2.4.0 \ - --hash=sha256:414bc6535b787febd7567804cc015fee39daab8ad86268f1310a9250697de466 \ - --hash=sha256:4e16665048960a0900c702d4a66415956a584919c03361cac9f1df5c5dd7e813 +urllib3==2.5.0 \ + --hash=sha256:3fc47733c7e419d4bc3f6b3dc2b4f890bb743906a30d56ba4a5bfa4bbff92760 \ + --hash=sha256:e6b01673c0fa6a13e374b50871808eb3bf7046c4b125b216f6bf1cc604cff0dc # via # requests # twine diff --git a/tools/publish/requirements_windows.txt b/tools/publish/requirements_windows.txt index 11017aa4f9..a4d5e3e25d 100644 --- a/tools/publish/requirements_windows.txt +++ b/tools/publish/requirements_windows.txt @@ -206,9 +206,9 @@ twine==5.1.1 \ --hash=sha256:215dbe7b4b94c2c50a7315c0275d2258399280fbb7d04182c7e55e24b5f93997 \ --hash=sha256:9aa0825139c02b3434d913545c7b847a21c835e11597f5255842d457da2322db # via -r tools/publish/requirements.in -urllib3==2.4.0 \ - --hash=sha256:414bc6535b787febd7567804cc015fee39daab8ad86268f1310a9250697de466 \ - --hash=sha256:4e16665048960a0900c702d4a66415956a584919c03361cac9f1df5c5dd7e813 +urllib3==2.5.0 \ + --hash=sha256:3fc47733c7e419d4bc3f6b3dc2b4f890bb743906a30d56ba4a5bfa4bbff92760 \ + --hash=sha256:e6b01673c0fa6a13e374b50871808eb3bf7046c4b125b216f6bf1cc604cff0dc # via # requests # twine From 6fd4c0bdc9eca48449c1f2b77a44f59a62a88dde Mon Sep 17 00:00:00 2001 From: Ignas Anikevicius <240938+aignas@users.noreply.github.com> Date: Fri, 20 Jun 2025 16:10:13 +0900 Subject: [PATCH 292/922] feat: support arbitrary target_settings in our platforms 3/n (#2990) With this PR we can support arbitrary target settings instead of just plain `constraint_values`. We still have custom logic to ensure that all of the tests pass. However, the plan is to remove those tests once we have simplified the wheel selection mechanisms and the `pkg_aliases` macro. I.e. if we have at most 1 wheel per platform that the `pypi` bzlmod extension passes to the `pkg_aliases` macro, then we can just have a simple `selects.with_or` where we list out all of the target platform values. This PR may result in us creating more targets but that is the price that we have to pay if we want to do this incrementally. Work towards #2747 Work towards #2548 Work towards #260 Co-authored-by: Richard Levasseur --- CHANGELOG.md | 2 +- python/private/pypi/BUILD.bazel | 1 + python/private/pypi/config_settings.bzl | 41 ++++++++++++++++--- python/private/pypi/extension.bzl | 26 +++++++----- python/private/pypi/hub_repository.bzl | 4 +- python/private/pypi/render_pkg_aliases.bzl | 14 +++---- .../config_settings/config_settings_tests.bzl | 2 +- tests/pypi/extension/extension_tests.bzl | 8 ++-- tests/pypi/pkg_aliases/pkg_aliases_test.bzl | 23 +++++++---- .../render_pkg_aliases_test.bzl | 4 +- 10 files changed, 83 insertions(+), 42 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index da3dcc8efc..f2fa98f73f 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -64,7 +64,7 @@ END_UNRELEASED_TEMPLATE ### Added * (pypi) To configure the environment for `requirements.txt` evaluation, use the newly added developer preview of the `pip.default` tag class. Only `rules_python` and root modules can use - this feature. You can also configure `constraint_values` using `pip.default`. + this feature. You can also configure custom `config_settings` using `pip.default`. {#v0-0-0-removed} ### Removed diff --git a/python/private/pypi/BUILD.bazel b/python/private/pypi/BUILD.bazel index b569b2217c..2666197786 100644 --- a/python/private/pypi/BUILD.bazel +++ b/python/private/pypi/BUILD.bazel @@ -64,6 +64,7 @@ bzl_library( deps = [ ":flags_bzl", "//python/private:flags_bzl", + "@bazel_skylib//lib:selects", ], ) diff --git a/python/private/pypi/config_settings.bzl b/python/private/pypi/config_settings.bzl index 7edc578d7a..f4826007f8 100644 --- a/python/private/pypi/config_settings.bzl +++ b/python/private/pypi/config_settings.bzl @@ -70,6 +70,7 @@ suffix. ::: """ +load("@bazel_skylib//lib:selects.bzl", "selects") load("//python/private:flags.bzl", "LibcFlag") load(":flags.bzl", "INTERNAL_FLAGS", "UniversalWhlFlag") @@ -112,7 +113,7 @@ def config_settings( muslc_versions = [], osx_versions = [], name = None, - platform_constraint_values = {}, + platform_config_settings = {}, **kwargs): """Generate all of the pip config settings. @@ -126,7 +127,7 @@ def config_settings( configure config settings for. osx_versions (list[str]): The list of OSX OS versions to configure config settings for. - platform_constraint_values: {type}`dict[str, list[str]]` the constraint + platform_config_settings: {type}`dict[str, list[str]]` the constraint values to use instead of the default ones. Key are platform names (a human-friendly platform string). Values are lists of `constraint_value` label strings. @@ -142,13 +143,24 @@ def config_settings( # TODO @aignas 2025-06-15: allowing universal2 and platform specific wheels in one # closure is making things maybe a little bit too complicated. "osx_universal2": ["@platforms//os:osx"], - } | platform_constraint_values + } | platform_config_settings for python_version in python_versions: - for platform_name, constraint_values in target_platforms.items(): + for platform_name, config_settings in target_platforms.items(): suffix = "_{}".format(platform_name) if platform_name else "" os, _, cpu = platform_name.partition("_") + # We parse the target settings and if there is a "platforms//os" or + # "platforms//cpu" value in here, we also add it into the constraint_values + # + # this is to ensure that we can still pass all of the unit tests for config + # setting specialization. + constraint_values = [] + for setting in config_settings: + setting_label = Label(setting) + if setting_label.repo_name == "platforms" and setting_label.package in ["os", "cpu"]: + constraint_values.append(setting) + _dist_config_settings( suffix = suffix, plat_flag_values = _plat_flag_values( @@ -158,6 +170,7 @@ def config_settings( glibc_versions = glibc_versions, muslc_versions = muslc_versions, ), + config_settings = config_settings, constraint_values = constraint_values, python_version = python_version, **kwargs @@ -318,7 +331,7 @@ def _plat_flag_values(os, cpu, osx_versions, glibc_versions, muslc_versions): return ret -def _dist_config_setting(*, name, compatible_with = None, native = native, **kwargs): +def _dist_config_setting(*, name, compatible_with = None, selects = selects, native = native, config_settings = None, **kwargs): """A macro to create a target for matching Python binary and source distributions. Args: @@ -327,6 +340,12 @@ def _dist_config_setting(*, name, compatible_with = None, native = native, **kwa compatible with the given dist config setting. For example, if only non-freethreaded python builds are allowed, add FLAGS._is_py_freethreaded_no here. + config_settings: {type}`list[str | Label]` the list of target settings that must + be matched before we try to evaluate the config_setting that we may create in + this function. + selects (struct): The struct containing config_setting_group function + to use for creating config setting groups. Can be overridden for unit tests + reasons. native (struct): The struct containing alias and config_setting rules to use for creating the objects. Can be overridden for unit tests reasons. @@ -346,4 +365,14 @@ def _dist_config_setting(*, name, compatible_with = None, native = native, **kwa ) name = dist_config_setting_name - native.config_setting(name = name, **kwargs) + # first define the config setting that has all of the constraint values + _name = "_" + name + native.config_setting( + name = _name, + **kwargs + ) + selects.config_setting_group( + name = name, + match_all = config_settings + [_name], + visibility = kwargs.get("visibility"), + ) diff --git a/python/private/pypi/extension.bzl b/python/private/pypi/extension.bzl index 78511b4c27..a0095f8f15 100644 --- a/python/private/pypi/extension.bzl +++ b/python/private/pypi/extension.bzl @@ -372,7 +372,7 @@ def _whl_repo(*, src, whl_library_args, is_multiple_versions, download_only, net ), ) -def _configure(config, *, platform, os_name, arch_name, constraint_values, env = {}, override = False): +def _configure(config, *, platform, os_name, arch_name, config_settings, env = {}, override = False): """Set the value in the config if the value is provided""" config.setdefault("platforms", {}) if platform: @@ -387,7 +387,7 @@ def _configure(config, *, platform, os_name, arch_name, constraint_values, env = name = platform.replace("-", "_").lower(), os_name = os_name, arch_name = arch_name, - constraint_values = constraint_values, + config_settings = config_settings, env = env, ) else: @@ -414,7 +414,7 @@ def _create_config(defaults): arch_name = cpu, os_name = "linux", platform = "linux_{}".format(cpu), - constraint_values = [ + config_settings = [ "@platforms//os:linux", "@platforms//cpu:{}".format(cpu), ], @@ -431,7 +431,7 @@ def _create_config(defaults): # See https://endoflife.date/macos os_name = "osx", platform = "osx_{}".format(cpu), - constraint_values = [ + config_settings = [ "@platforms//os:osx", "@platforms//cpu:{}".format(cpu), ], @@ -443,7 +443,7 @@ def _create_config(defaults): arch_name = "x86_64", os_name = "windows", platform = "windows_x86_64", - constraint_values = [ + config_settings = [ "@platforms//os:windows", "@platforms//cpu:x86_64", ], @@ -513,7 +513,7 @@ You cannot use both the additive_build_content and additive_build_content_file a _configure( defaults, arch_name = tag.arch_name, - constraint_values = tag.constraint_values, + config_settings = tag.config_settings, env = tag.env, os_name = tag.os_name, platform = tag.platform, @@ -693,9 +693,9 @@ You cannot use both the additive_build_content and additive_build_content_file a } for hub_name, extra_whl_aliases in extra_aliases.items() }, - platform_constraint_values = { + platform_config_settings = { hub_name: { - platform_name: sorted([str(Label(cv)) for cv in p.constraint_values]) + platform_name: sorted([str(Label(cv)) for cv in p.config_settings]) for platform_name, p in config.platforms.items() } for hub_name in hub_whl_map @@ -790,7 +790,7 @@ def _pip_impl(module_ctx): for key, values in whl_map.items() }, packages = mods.exposed_packages.get(hub_name, []), - platform_constraint_values = mods.platform_constraint_values.get(hub_name, {}), + platform_config_settings = mods.platform_config_settings.get(hub_name, {}), groups = mods.hub_group_map.get(hub_name), ) @@ -812,10 +812,11 @@ Either this or {attr}`env` `platform_machine` key should be specified. ::: """, ), - "constraint_values": attr.label_list( + "config_settings": attr.label_list( mandatory = True, doc = """\ -The constraint_values to use in select statements. +The list of labels to `config_setting` targets that need to be matched for the platform to be +selected. """, ), "os_name": attr.string( @@ -1145,6 +1146,9 @@ terms used in this extension. [environment_markers]: https://packaging.python.org/en/latest/specifications/dependency-specifiers/#environment-markers ::: + +:::{versionadded} VERSION_NEXT_FEATURE +::: """, ), "override": _override_tag, diff --git a/python/private/pypi/hub_repository.bzl b/python/private/pypi/hub_repository.bzl index 4398d7b597..75f3ec98d7 100644 --- a/python/private/pypi/hub_repository.bzl +++ b/python/private/pypi/hub_repository.bzl @@ -34,7 +34,7 @@ def _impl(rctx): }, extra_hub_aliases = rctx.attr.extra_hub_aliases, requirement_cycles = rctx.attr.groups, - platform_constraint_values = rctx.attr.platform_constraint_values, + platform_config_settings = rctx.attr.platform_config_settings, ) for path, contents in aliases.items(): rctx.file(path, contents) @@ -84,7 +84,7 @@ hub_repository = repository_rule( The list of packages that will be exposed via all_*requirements macros. Defaults to whl_map keys. """, ), - "platform_constraint_values": attr.string_list_dict( + "platform_config_settings": attr.string_list_dict( doc = "The constraint values for each platform name. The values are string canonical string Label representations", mandatory = False, ), diff --git a/python/private/pypi/render_pkg_aliases.bzl b/python/private/pypi/render_pkg_aliases.bzl index 267d7ce85d..e743fc20f7 100644 --- a/python/private/pypi/render_pkg_aliases.bzl +++ b/python/private/pypi/render_pkg_aliases.bzl @@ -155,14 +155,14 @@ def _major_minor_versions(python_versions): # Use a dict as a simple set return sorted({_major_minor(v): None for v in python_versions}) -def render_multiplatform_pkg_aliases(*, aliases, platform_constraint_values = {}, **kwargs): +def render_multiplatform_pkg_aliases(*, aliases, platform_config_settings = {}, **kwargs): """Render the multi-platform pkg aliases. Args: aliases: dict[str, list(whl_config_setting)] A list of aliases that will be transformed from ones having `filename` to ones having `config_setting`. - platform_constraint_values: {type}`dict[str, list[str]]` contains all of the - target platforms and their appropriate `constraint_values`. + platform_config_settings: {type}`dict[str, list[str]]` contains all of the + target platforms and their appropriate `target_settings`. **kwargs: extra arguments passed to render_pkg_aliases. Returns: @@ -189,20 +189,20 @@ def render_multiplatform_pkg_aliases(*, aliases, platform_constraint_values = {} muslc_versions = flag_versions.get("muslc_versions", []), osx_versions = flag_versions.get("osx_versions", []), python_versions = _major_minor_versions(flag_versions.get("python_versions", [])), - platform_constraint_values = platform_constraint_values, + platform_config_settings = platform_config_settings, visibility = ["//:__subpackages__"], ) return contents -def _render_config_settings(platform_constraint_values, **kwargs): +def _render_config_settings(platform_config_settings, **kwargs): return """\ load("@rules_python//python/private/pypi:config_settings.bzl", "config_settings") {}""".format(render.call( "config_settings", name = repr("config_settings"), - platform_constraint_values = render.dict( - platform_constraint_values, + platform_config_settings = render.dict( + platform_config_settings, value_repr = render.list, ), **_repr_dict(value_repr = render.list, **kwargs) diff --git a/tests/pypi/config_settings/config_settings_tests.bzl b/tests/pypi/config_settings/config_settings_tests.bzl index 9551d42d10..a15f6b4d32 100644 --- a/tests/pypi/config_settings/config_settings_tests.bzl +++ b/tests/pypi/config_settings/config_settings_tests.bzl @@ -657,7 +657,7 @@ def config_settings_test_suite(name): # buildifier: disable=function-docstring glibc_versions = [(2, 14), (2, 17)], muslc_versions = [(1, 1)], osx_versions = [(10, 9), (11, 0)], - platform_constraint_values = { + platform_config_settings = { "linux_aarch64": [ "@platforms//cpu:aarch64", "@platforms//os:linux", diff --git a/tests/pypi/extension/extension_tests.bzl b/tests/pypi/extension/extension_tests.bzl index 231e8cab41..146293ee8d 100644 --- a/tests/pypi/extension/extension_tests.bzl +++ b/tests/pypi/extension/extension_tests.bzl @@ -78,19 +78,17 @@ def _parse_modules(env, enable_pipstar = 0, **kwargs): def _default( arch_name = None, - constraint_values = None, + config_settings = None, os_name = None, platform = None, - target_settings = None, env = None, whl_limit = None, whl_platforms = None): return struct( arch_name = arch_name, - constraint_values = constraint_values, os_name = os_name, platform = platform, - target_settings = target_settings, + config_settings = config_settings, env = env or {}, whl_platforms = whl_platforms, whl_limit = whl_limit, @@ -1051,7 +1049,7 @@ def _test_pipstar_platforms(env): default = [ _default( platform = "{}_{}".format(os, cpu), - constraint_values = [ + config_settings = [ "@platforms//os:{}".format(os), "@platforms//cpu:{}".format(cpu), ], diff --git a/tests/pypi/pkg_aliases/pkg_aliases_test.bzl b/tests/pypi/pkg_aliases/pkg_aliases_test.bzl index 0fbcd4e7a6..123ee725f8 100644 --- a/tests/pypi/pkg_aliases/pkg_aliases_test.bzl +++ b/tests/pypi/pkg_aliases/pkg_aliases_test.bzl @@ -392,6 +392,9 @@ _tests.append(_test_multiplatform_whl_aliases_filename_versioned) def _mock_alias(container): return lambda name, **kwargs: container.append(name) +def _mock_config_setting_group(container): + return lambda name, **kwargs: container.append(name) + def _mock_config_setting(container): def _inner(name, flag_values = None, constraint_values = None, **_): if flag_values or constraint_values: @@ -417,9 +420,12 @@ def _test_config_settings_exist_legacy(env): python_versions = ["3.11"], native = struct( alias = _mock_alias(available_config_settings), - config_setting = _mock_config_setting(available_config_settings), + config_setting = _mock_config_setting([]), ), - platform_constraint_values = { + selects = struct( + config_setting_group = _mock_config_setting_group(available_config_settings), + ), + platform_config_settings = { "linux_aarch64": [ "@platforms//cpu:aarch64", "@platforms//os:linux", @@ -454,7 +460,7 @@ def _test_config_settings_exist(env): "any": {}, "macosx_11_0_arm64": { "osx_versions": [(11, 0)], - "platform_constraint_values": { + "platform_config_settings": { "osx_aarch64": [ "@platforms//cpu:aarch64", "@platforms//os:osx", @@ -463,7 +469,7 @@ def _test_config_settings_exist(env): }, "manylinux_2_17_x86_64": { "glibc_versions": [(2, 17), (2, 18)], - "platform_constraint_values": { + "platform_config_settings": { "linux_x86_64": [ "@platforms//cpu:x86_64", "@platforms//os:linux", @@ -472,7 +478,7 @@ def _test_config_settings_exist(env): }, "manylinux_2_18_x86_64": { "glibc_versions": [(2, 17), (2, 18)], - "platform_constraint_values": { + "platform_config_settings": { "linux_x86_64": [ "@platforms//cpu:x86_64", "@platforms//os:linux", @@ -481,7 +487,7 @@ def _test_config_settings_exist(env): }, "musllinux_1_1_aarch64": { "muslc_versions": [(1, 2), (1, 1), (1, 0)], - "platform_constraint_values": { + "platform_config_settings": { "linux_aarch64": [ "@platforms//cpu:aarch64", "@platforms//os:linux", @@ -500,7 +506,10 @@ def _test_config_settings_exist(env): python_versions = ["3.11"], native = struct( alias = _mock_alias(available_config_settings), - config_setting = _mock_config_setting(available_config_settings), + config_setting = _mock_config_setting([]), + ), + selects = struct( + config_setting_group = _mock_config_setting_group(available_config_settings), ), **kwargs ) diff --git a/tests/pypi/render_pkg_aliases/render_pkg_aliases_test.bzl b/tests/pypi/render_pkg_aliases/render_pkg_aliases_test.bzl index c262ed6823..ad7f36aed6 100644 --- a/tests/pypi/render_pkg_aliases/render_pkg_aliases_test.bzl +++ b/tests/pypi/render_pkg_aliases/render_pkg_aliases_test.bzl @@ -93,7 +93,7 @@ def _test_bzlmod_aliases(env): }, }, extra_hub_aliases = {"bar_baz": ["foo"]}, - platform_constraint_values = { + platform_config_settings = { "linux_x86_64": [ "@platforms//os:linux", "@platforms//cpu:x86_64", @@ -136,7 +136,7 @@ load("@rules_python//python/private/pypi:config_settings.bzl", "config_settings" config_settings( name = "config_settings", - platform_constraint_values = { + platform_config_settings = { "linux_x86_64": [ "@platforms//os:linux", "@platforms//cpu:x86_64", From 8f8c5b9ba7c7f68f37b7687ebb22931cff075241 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Fri, 20 Jun 2025 00:42:42 -0700 Subject: [PATCH 293/922] docs: fix various typos and improve grammar (#3015) Used Jules to do some copy editing. It found a variety of typos. * Consistently use backticks for rules_python, WORKSPACE, and some other terms * Various simple typo and grammar fixes --------- Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com> --- docs/README.md | 16 +-- docs/_includes/py_console_script_binary.md | 25 ++-- docs/coverage.md | 4 +- docs/devguide.md | 28 ++-- docs/environment-variables.md | 24 ++-- docs/extending.md | 12 +- docs/gazelle.md | 4 +- docs/getting-started.md | 10 +- docs/glossary.md | 10 +- docs/index.md | 22 ++-- docs/precompiling.md | 40 +++--- docs/pypi/circular-dependencies.md | 16 +-- docs/pypi/download-workspace.md | 12 +- docs/pypi/download.md | 68 +++++----- docs/pypi/index.md | 6 +- docs/pypi/lock.md | 11 +- docs/pypi/patch.md | 4 +- docs/pypi/use.md | 42 +++--- docs/repl.md | 2 +- docs/support.md | 22 ++-- docs/toolchains.md | 142 ++++++++++----------- 21 files changed, 263 insertions(+), 257 deletions(-) diff --git a/docs/README.md b/docs/README.md index d98be41232..456f1cfd64 100644 --- a/docs/README.md +++ b/docs/README.md @@ -1,14 +1,14 @@ # rules_python Sphinx docs generation The docs for rules_python are generated using a combination of Sphinx, Bazel, -and Readthedocs.org. The Markdown files in source control are unlikely to render +and Read the Docs. The Markdown files in source control are unlikely to render properly without the Sphinx processing step because they rely on Sphinx and MyST-specific Markdown functionality. The actual sources that Sphinx consumes are in this directory, with Stardoc -generating additional sources or Sphinx. +generating additional sources for Sphinx. -Manually building the docs isn't necessary -- readthedocs.org will +Manually building the docs isn't necessary -- Read the Docs will automatically build and deploy them when commits are pushed to the repo. ## Generating docs for development @@ -31,8 +31,8 @@ equivalent bazel command if desired. ### Installing ibazel The `ibazel` tool can be used to automatically rebuild the docs as you -development them. See the [ibazel docs](https://github.com/bazelbuild/bazel-watcher) for -how to install it. The quick start for linux is: +develop them. See the [ibazel docs](https://github.com/bazelbuild/bazel-watcher) for +how to install it. The quick start for Linux is: ``` sudo apt install npm @@ -57,9 +57,9 @@ docs/. The Sphinx configuration is `docs/conf.py`. See https://www.sphinx-doc.org/ for details about the configuration file. -## Readthedocs configuration +## Read the Docs configuration -There's two basic parts to the readthedocs configuration: +There's two basic parts to the Read the Docs configuration: * `.readthedocs.yaml`: This configuration file controls most settings, such as the OS version used to build, Python version, dependencies, what Bazel @@ -69,4 +69,4 @@ There's two basic parts to the readthedocs configuration: controls additional settings such as permissions, what versions are published, when to publish changes, etc. -For more readthedocs configuration details, see docs.readthedocs.io. +For more Read the Docs configuration details, see docs.readthedocs.io. diff --git a/docs/_includes/py_console_script_binary.md b/docs/_includes/py_console_script_binary.md index d327091630..cae9f9f2f5 100644 --- a/docs/_includes/py_console_script_binary.md +++ b/docs/_includes/py_console_script_binary.md @@ -1,8 +1,8 @@ This rule is to make it easier to generate `console_script` entry points as per Python [specification]. -Generate a `py_binary` target for a particular console_script `entry_point` -from a PyPI package, e.g. for creating an executable `pylint` target use: +Generate a `py_binary` target for a particular `console_script` entry_point +from a PyPI package, e.g. for creating an executable `pylint` target, use: ```starlark load("@rules_python//python/entry_points:py_console_script_binary.bzl", "py_console_script_binary") @@ -12,11 +12,12 @@ py_console_script_binary( ) ``` -#### Specifying extra dependencies +#### Specifying extra dependencies You can also specify extra dependencies and the -exact script name you want to call. It is useful for tools like `flake8`, `pylint`, -`pytest`, which have plugin discovery methods and discover dependencies from the -PyPI packages available in the `PYTHONPATH`. +exact script name you want to call. This is useful for tools like `flake8`, +`pylint`, and `pytest`, which have plugin discovery methods and discover +dependencies from the PyPI packages available in the `PYTHONPATH`. + ```starlark load("@rules_python//python/entry_points:py_console_script_binary.bzl", "py_console_script_binary") @@ -44,13 +45,13 @@ load("@rules_python//python/entry_points:py_console_script_binary.bzl", "py_cons py_console_script_binary( name = "yamllint", pkg = "@pip//yamllint", - python_version = "3.9" + python_version = "3.9", ) ``` #### Adding a Shebang Line -You can specify a shebang line for the generated binary, useful for Unix-like +You can specify a shebang line for the generated binary. This is useful for Unix-like systems where the shebang line determines which interpreter is used to execute the script, per [PEP441]: @@ -70,12 +71,12 @@ Python interpreter is available in the environment. #### Using a specific Python Version directly from a Toolchain :::{deprecated} 1.1.0 -The toolchain specific `py_binary` and `py_test` symbols are aliases to the regular rules. -i.e. Deprecated `load("@python_versions//3.11:defs.bzl", "py_binary")` and `load("@python_versions//3.11:defs.bzl", "py_test")` +The toolchain-specific `py_binary` and `py_test` symbols are aliases to the regular rules. +For example, `load("@python_versions//3.11:defs.bzl", "py_binary")` and `load("@python_versions//3.11:defs.bzl", "py_test")` are deprecated. -You should instead specify the desired python version with `python_version`; see above example. +You should instead specify the desired Python version with `python_version`; see the example above. ::: -Alternatively, the [`py_console_script_binary.binary_rule`] arg can be passed +Alternatively, the {obj}`py_console_script_binary.binary_rule` arg can be passed the version-bound `py_binary` symbol, or any other `py_binary`-compatible rule of your choosing: ```starlark diff --git a/docs/coverage.md b/docs/coverage.md index 3e0e67368c..3c7d9e0cfc 100644 --- a/docs/coverage.md +++ b/docs/coverage.md @@ -9,7 +9,7 @@ when configuring toolchains. ## Enabling `rules_python` coverage support Enabling the coverage support bundled with `rules_python` just requires setting an -argument when registerting toolchains. +argument when registering toolchains. For Bzlmod: @@ -32,7 +32,7 @@ python_register_toolchains( This will implicitly add the version of `coverage` bundled with `rules_python` to the dependencies of `py_test` rules when `bazel coverage` is run. If a target already transitively depends on a different version of -`coverage`, then behavior is undefined -- it is undefined which version comes +`coverage`, then the behavior is undefined -- it is undefined which version comes first in the import path. If you find yourself in this situation, then you'll need to manually configure coverage (see below). ::: diff --git a/docs/devguide.md b/docs/devguide.md index f233611cad..345907b374 100644 --- a/docs/devguide.md +++ b/docs/devguide.md @@ -1,7 +1,7 @@ # Dev Guide -This document covers tips and guidance for working on the rules_python code -base. A primary audience for it is first time contributors. +This document covers tips and guidance for working on the `rules_python` code +base. Its primary audience is first-time contributors. ## Running tests @@ -12,8 +12,8 @@ bazel test //... ``` And it will run all the tests it can find. The first time you do this, it will -probably take long time because various dependencies will need to be downloaded -and setup. Subsequent runs will be faster, but there are many tests, and some of +probably take a long time because various dependencies will need to be downloaded +and set up. Subsequent runs will be faster, but there are many tests, and some of them are slow. If you're working on a particular area of code, you can run just the tests in those directories instead, which can speed up your edit-run cycle. @@ -22,14 +22,14 @@ the tests in those directories instead, which can speed up your edit-run cycle. Most code should have tests of some sort. This helps us have confidence that refactors didn't break anything and that releases won't have regressions. -We don't require 100% test coverage, testing certain Bazel functionality is +We don't require 100% test coverage; testing certain Bazel functionality is difficult, and some edge cases are simply too hard to test or not worth the extra complexity. We try to judiciously decide when not having tests is a good idea. Tests go under `tests/`. They are loosely organized into directories for the particular subsystem or functionality they are testing. If an existing directory -doesn't seem like a good match for the functionality being testing, then it's +doesn't seem like a good match for the functionality being tested, then it's fine to create a new directory. Re-usable test helpers and support code go in `tests/support`. Tests don't need @@ -72,9 +72,9 @@ the rule. To have it support setting a new flag: An integration test is one that runs a separate Bazel instance inside the test. These tests are discouraged unless absolutely necessary because they are slow, -require much memory and CPU, and are generally harder to debug. Integration -tests are reserved for things that simple can't be tested otherwise, or for -simple high level verification tests. +require a lot of memory and CPU, and are generally harder to debug. Integration +tests are reserved for things that simply can't be tested otherwise, or for +simple high-level verification tests. Integration tests live in `tests/integration`. When possible, add to an existing integration test. @@ -98,9 +98,9 @@ integration test. ## Updating tool dependencies -It's suggested to routinely update the tool versions within our repo - some of the -tools are using requirement files compiled by `uv` and others use other means. In order -to have everything self-documented, we have a special target - -`//private:requirements.update`, which uses `rules_multirun` to run in sequence all -of the requirement updating scripts in one go. This can be done once per release as +It's suggested to routinely update the tool versions within our repo. Some of the +tools are using requirement files compiled by `uv`, and others use other means. In order +to have everything self-documented, we have a special target, +`//private:requirements.update`, which uses `rules_multirun` to run all +of the requirement-updating scripts in sequence in one go. This can be done once per release as we prepare for releases. diff --git a/docs/environment-variables.md b/docs/environment-variables.md index 8a51bcbfd2..9a8c1dfe99 100644 --- a/docs/environment-variables.md +++ b/docs/environment-variables.md @@ -5,16 +5,16 @@ This variable allows for additional arguments to be provided to the Python interpreter at bootstrap time when the `bash` bootstrap is used. If `RULES_PYTHON_ADDITIONAL_INTERPRETER_ARGS` were provided as `-Xaaa`, then the command -would be; +would be: ``` python -Xaaa /path/to/file.py ``` This feature is likely to be useful for the integration of debuggers. For example, -it would be possible to configure the `RULES_PYTHON_ADDITIONAL_INTERPRETER_ARGS` to -be set to `/path/to/debugger.py --port 12344 --file` resulting -in the command executed being; +it would be possible to configure `RULES_PYTHON_ADDITIONAL_INTERPRETER_ARGS` to +be set to `/path/to/debugger.py --port 12344 --file`, resulting +in the command executed being: ``` python /path/to/debugger.py --port 12345 --file /path/to/file.py @@ -42,14 +42,14 @@ doing. This is mostly useful for development to debug errors. :::{envvar} RULES_PYTHON_DEPRECATION_WARNINGS -When `1`, the rules_python will warn users about deprecated functionality that will +When `1`, `rules_python` will warn users about deprecated functionality that will be removed in a subsequent major `rules_python` version. Defaults to `0` if unset. ::: ::::{envvar} RULES_PYTHON_ENABLE_PYSTAR -When `1`, the rules_python Starlark implementation of the core rules is used -instead of the Bazel-builtin rules. Note this requires Bazel 7+. Defaults +When `1`, the `rules_python` Starlark implementation of the core rules is used +instead of the Bazel-builtin rules. Note that this requires Bazel 7+. Defaults to `1`. :::{versionadded} 0.26.0 @@ -62,7 +62,7 @@ The default became `1` if unspecified ::::{envvar} RULES_PYTHON_ENABLE_PIPSTAR -When `1`, the rules_python Starlark implementation of the pypi/pip integration is used +When `1`, the `rules_python` Starlark implementation of the PyPI/pip integration is used instead of the legacy Python scripts. :::{versionadded} 1.5.0 @@ -95,8 +95,8 @@ exit. :::{envvar} RULES_PYTHON_GAZELLE_VERBOSE -When `1`, debug information from gazelle is printed to stderr. -::: +When `1`, debug information from Gazelle is printed to stderr. +:::: :::{envvar} RULES_PYTHON_PIP_ISOLATED @@ -125,9 +125,9 @@ Determines the verbosity of logging output for repo rules. Valid values: :::{envvar} RULES_PYTHON_REPO_TOOLCHAIN_VERSION_OS_ARCH -Determines the python interpreter platform to be used for a particular +Determines the Python interpreter platform to be used for a particular interpreter `(version, os, arch)` triple to be used in repository rules. -Replace the `VERSION_OS_ARCH` part with actual values when using, e.g. +Replace the `VERSION_OS_ARCH` part with actual values when using, e.g., `3_13_0_linux_x86_64`. The version values must have `_` instead of `.` and the os, arch values are the same as the ones mentioned in the `//python:versions.bzl` file. diff --git a/docs/extending.md b/docs/extending.md index 387310e6cf..00018fbd74 100644 --- a/docs/extending.md +++ b/docs/extending.md @@ -41,10 +41,10 @@ wrappers around the keyword arguments eventually passed to the `rule()` function. These builder APIs give access to the _entire_ rule definition and allow arbitrary modifications. -This is level of control is powerful, but also volatile. A rule definition +This level of control is powerful but also volatile. A rule definition contains many details that _must_ change as the implementation changes. What is more or less likely to change isn't known in advance, but some general -rules are: +rules of thumb are: * Additive behavior to public attributes will be less prone to breaking. * Internal attributes that directly support a public attribute are likely @@ -55,7 +55,7 @@ rules are: ## Example: validating a source file -In this example, we derive from `py_library` a custom rule that verifies source +In this example, we derive a custom rule from `py_library` that verifies source code contains the word "snakes". It does this by: * Adding an implicit dependency on a checker program @@ -111,7 +111,7 @@ has_snakes_library = create_has_snakes_rule() ## Example: adding transitions -In this example, we derive from `py_binary` to force building for a particular +In this example, we derive a custom rule from `py_binary` to force building for a particular platform. We do this by: * Adding an additional output to the rule's cfg @@ -136,8 +136,8 @@ def create_rule(): r.cfg.add_output("//command_line_option:platforms") return r.build() -py_linux_binary = create_linux_binary_rule() +py_linux_binary = create_rule() ``` -Users can then use `py_linux_binary` the same as a regular py_binary. It will +Users can then use `py_linux_binary` the same as a regular `py_binary`. It will act as if `--platforms=//my/platforms:linux` was specified when building it. diff --git a/docs/gazelle.md b/docs/gazelle.md index 89f26d67bb..60b46faf2c 100644 --- a/docs/gazelle.md +++ b/docs/gazelle.md @@ -3,7 +3,7 @@ [Gazelle](https://github.com/bazelbuild/bazel-gazelle) is a build file generator for Bazel projects. It can create new `BUILD.bazel` files for a project that follows language conventions and update existing build files to include new sources, dependencies, and options. -Bazel may run Gazelle using the Gazelle rule, or it may be installed and run as a command line tool. +Bazel may run Gazelle using the Gazelle rule, or Gazelle may be installed and run as a command line tool. -See the documentation for Gazelle with rules_python in the {gh-path}`gazelle` +See the documentation for Gazelle with `rules_python` in the {gh-path}`gazelle` directory. diff --git a/docs/getting-started.md b/docs/getting-started.md index 7e7b88aa8a..d81d72f590 100644 --- a/docs/getting-started.md +++ b/docs/getting-started.md @@ -1,14 +1,14 @@ # Getting started -This doc is a simplified guide to help get started quickly. It provides +This document is a simplified guide to help you get started quickly. It provides a simplified introduction to having a working Python program for both `bzlmod` and the older way of using `WORKSPACE`. It assumes you have a `requirements.txt` file with your PyPI dependencies. -For more details information about configuring `rules_python`, see: +For more detailed information about configuring `rules_python`, see: * [Configuring the runtime](configuring-toolchains) -* [Configuring third party dependencies (pip/pypi)](./pypi/index) +* [Configuring third-party dependencies (pip/PyPI)](./pypi/index) * [API docs](api/index) ## Including dependencies @@ -32,7 +32,7 @@ use_repo(pip, "pypi") ### Using a WORKSPACE file -Using WORKSPACE is deprecated, but still supported, and a bit more involved than +Using `WORKSPACE` is deprecated but still supported, and it's a bit more involved than using Bzlmod. Here is a simplified setup to download the prebuilt runtimes. ```starlark @@ -72,7 +72,7 @@ pip_parse( ## "Hello World" -Once you've imported the rule set using either Bzlmod or WORKSPACE, you can then +Once you've imported the rule set using either Bzlmod or `WORKSPACE`, you can then load the core rules in your `BUILD` files with the following: ```starlark diff --git a/docs/glossary.md b/docs/glossary.md index 9afbcffb92..c9bd03fd0e 100644 --- a/docs/glossary.md +++ b/docs/glossary.md @@ -5,7 +5,7 @@ common attributes : Every rule has a set of common attributes. See Bazel's [Common attributes](https://bazel.build/reference/be/common-definitions#common-attributes) - for a complete listing + for a complete listing. in-build runtime : An in-build runtime is one where the Python runtime, and all its files, are @@ -21,9 +21,9 @@ which can be a significant number of files. platform runtime : A platform runtime is a Python runtime that is assumed to be installed on the -system where a Python binary runs, whereever that may be. For example, using `/usr/bin/python3` +system where a Python binary runs, wherever that may be. For example, using `/usr/bin/python3` as the interpreter is a platform runtime -- it assumes that, wherever the binary -runs (your local machine, a remote worker, within a container, etc), that path +runs (your local machine, a remote worker, within a container, etc.), that path is available. Such runtimes are _not_ part of a binary's runfiles. The main advantage of platform runtimes is they are lightweight insofar as @@ -42,8 +42,8 @@ rule callable accepted; refer to the respective API accepting this type. simple label -: A `str` or `Label` object but not a _direct_ `select` object. These usually - mean a string manipulation is occuring, which can't be done on `select` + A `str` or `Label` object but not a _direct_ `select` object. This usually + means a string manipulation is occurring, which can't be done on `select` objects. Such attributes are usually still configurable if an alias is used, and a reference to the alias is passed instead. diff --git a/docs/index.md b/docs/index.md index 82023f3ad8..25b423c6c3 100644 --- a/docs/index.md +++ b/docs/index.md @@ -1,6 +1,6 @@ # Python Rules for Bazel -`rules_python` is the home for 4 major components with varying maturity levels. +`rules_python` is the home for four major components with varying maturity levels. :::{topic} Core rules @@ -9,8 +9,8 @@ The core Python rules -- `py_library`, `py_binary`, `py_test`, support in Bazel. When using Bazel 6 (or earlier), the core rules are bundled into the Bazel binary, and the symbols -in this repository are simple aliases. On Bazel 7 and above `rules_python` uses -a separate Starlark implementation, +in this repository are simple aliases. On Bazel 7 and above, `rules_python` uses +a separate Starlark implementation; see {ref}`Migrating from the Bundled Rules` below. This repository follows @@ -21,12 +21,12 @@ outlined in the [support](support) page. :::{topic} PyPI integration -Package installation rules for integrating with PyPI and other SimpleAPI +Package installation rules for integrating with PyPI and other Simple API- compatible indexes. These rules work and can be used in production, but the cross-platform building that supports pulling PyPI dependencies for a target platform that is different -from the host platform is still in beta and the APIs that are subject to potential +from the host platform is still in beta, and the APIs that are subject to potential change are marked as `experimental`. ::: @@ -36,9 +36,9 @@ change are marked as `experimental`. `sphinxdocs` rules allow users to generate documentation using Sphinx powered by Bazel, with additional functionality for documenting Starlark and Bazel code. -The functionality is exposed because other projects find it useful, but -it is available as is and **the semantic versioning and -compatibility policy used by `rules_python` does not apply**. +The functionality is exposed because other projects find it useful, but +it is available "as is", and **the semantic versioning and +compatibility policy used by `rules_python` does not apply**. ::: @@ -47,7 +47,7 @@ compatibility policy used by `rules_python` does not apply**. `gazelle` plugin for generating `BUILD.bazel` files based on Python source code. -This is available as is and the semantic versioning used by `rules_python` does +This is available "as is", and the semantic versioning used by `rules_python` does not apply. ::: @@ -78,7 +78,7 @@ appropriate `load()` statements and rewrite uses of `native.py_*`. buildifier --lint=fix --warnings=native-py ``` -Currently, the `WORKSPACE` file needs to be updated manually as per +Currently, the `WORKSPACE` file needs to be updated manually as per [Getting started](getting-started). Note that Starlark-defined bundled symbols underneath @@ -87,7 +87,7 @@ by buildifier. ## Migrating to bzlmod -See {gh-path}`Bzlmod support ` for any behaviour differences between +See {gh-path}`Bzlmod support ` for any behavioral differences between `bzlmod` and `WORKSPACE`. diff --git a/docs/precompiling.md b/docs/precompiling.md index a46608f77e..ea978cddce 100644 --- a/docs/precompiling.md +++ b/docs/precompiling.md @@ -1,6 +1,6 @@ # Precompiling -Precompiling is compiling Python source files (`.py` files) into byte code +Precompiling is compiling Python source files (`.py` files) into bytecode (`.pyc` files) at build time instead of runtime. Doing it at build time can improve performance by skipping that work at runtime. @@ -15,12 +15,12 @@ While precompiling helps runtime performance, it has two main costs: a `.pyc` file. Compiled files are generally around the same size as the source files, so it approximately doubles the disk usage. 2. Precompiling requires running an extra action at build time. While - compiling itself isn't that expensive, the overhead can become noticable + compiling itself isn't that expensive, the overhead can become noticeable as more files need to be compiled. ## Binary-level opt-in -Binary-level opt-in allows enabling precompiling on a per-target basic. This is +Binary-level opt-in allows enabling precompiling on a per-target basis. This is useful for situations such as: * Globally enabling precompiling in your `.bazelrc` isn't feasible. This may @@ -41,7 +41,7 @@ can use an opt-in or opt-out approach by setting its value: ## Pyc-only builds -A pyc-only build (aka "source less" builds) is when only `.pyc` files are +A pyc-only build (aka "sourceless" builds) is when only `.pyc` files are included; the source `.py` files are not included. To enable this, set @@ -55,8 +55,8 @@ The advantage of pyc-only builds are: The disadvantages are: * Error messages will be less precise because the precise line and offset - information isn't in an pyc file. -* pyc files are Python major-version specific. + information isn't in a pyc file. +* pyc files are Python major-version-specific. :::{note} pyc files are not a form of hiding source code. They are trivial to uncompile, @@ -75,11 +75,11 @@ mechanisms are available: the {bzl:attr}`precompiler` attribute. Arbitrary binaries are supported. * The execution requirements can be customized using `--@rules_python//tools/precompiler:execution_requirements`. This is a list - flag that can be repeated. Each entry is a key=value that is added to the + flag that can be repeated. Each entry is a `key=value` pair that is added to the execution requirements of the `PyCompile` action. Note that this flag - is specific to the rules_python precompiler. If a custom binary is used, + is specific to the `rules_python` precompiler. If a custom binary is used, this flag will have to be propagated from the custom binary using the - `testing.ExecutionInfo` provider; refer to the `py_interpreter_program` an + `testing.ExecutionInfo` provider; refer to the `py_interpreter_program` example. The default precompiler implementation is an asynchronous/concurrent implementation. If you find it has bugs or hangs, please report them. In the @@ -90,18 +90,18 @@ as well, but is less likely to have issues. The `execution_requirements` keys of most relevance are: * `supports-workers`: 1 or 0, to indicate if a regular persistent worker is desired. -* `supports-multiplex-workers`: 1 o 0, to indicate if a multiplexed persistent +* `supports-multiplex-workers`: `1` or `0`, to indicate if a multiplexed persistent worker is desired. -* `requires-worker-protocol`: json or proto; the rules_python precompiler - currently only supports json. -* `supports-multiplex-sandboxing`: 1 or 0, to indicate if sanboxing is of the +* `requires-worker-protocol`: `json` or `proto`; the `rules_python` precompiler + currently only supports `json`. +* `supports-multiplex-sandboxing`: `1` or `0`, to indicate if sandboxing of the worker is supported. -* `supports-worker-cancellation`: 1 or 1, to indicate if requests to the worker +* `supports-worker-cancellation`: `1` or `0`, to indicate if requests to the worker can be cancelled. Note that any execution requirements values can be specified in the flag. -## Known issues, caveats, and idiosyncracies +## Known issues, caveats, and idiosyncrasies * Precompiling requires Bazel 7+ with the Pystar rule implementation enabled. * Mixing rules_python PyInfo with Bazel builtin PyInfo will result in pyc files @@ -111,14 +111,14 @@ Note that any execution requirements values can be specified in the flag. causes the module to be found in the workspace source directory instead of within the binary's runfiles directory (where the pyc files are). This can usually be worked around by removing `sys.path[0]` (or otherwise ensuring the - runfiles directory comes before the repos source directory in `sys.path`). -* The pyc filename does not include the optimization level (e.g. - `foo.cpython-39.opt-2.pyc`). This works fine (it's all byte code), but also + runfiles directory comes before the repo's source directory in `sys.path`). +* The pyc filename does not include the optimization level (e.g., + `foo.cpython-39.opt-2.pyc`). This works fine (it's all bytecode), but also means the interpreter `-O` argument can't be used -- doing so will cause the interpreter to look for the non-existent `opt-N` named files. -* Targets with the same source files and different exec properites will result +* Targets with the same source files and different exec properties will result in action conflicts. This most commonly occurs when a `py_binary` and - `py_library` have the same source files. To fix, modify both targets so + a `py_library` have the same source files. To fix this, modify both targets so they have the same exec properties. If this is difficult because unsupported exec groups end up being passed to the Python rules, please file an issue to have those exec groups added to the Python rules. diff --git a/docs/pypi/circular-dependencies.md b/docs/pypi/circular-dependencies.md index d22f5b36a7..62613f489e 100644 --- a/docs/pypi/circular-dependencies.md +++ b/docs/pypi/circular-dependencies.md @@ -3,8 +3,8 @@ # Circular dependencies -Sometimes PyPi packages contain dependency cycles -- for instance a particular -version `sphinx` (this is no longer the case in the latest version as of +Sometimes PyPI packages contain dependency cycles. For instance, a particular +version of `sphinx` (this is no longer the case in the latest version as of 2024-06-02) depends on `sphinxcontrib-serializinghtml`. When using them as `requirement()`s, ala @@ -47,10 +47,10 @@ simultaneously. ) ``` -`pip_parse` supports fixing multiple cycles simultaneously, however cycles must -be distinct. `apache-airflow` for instance has dependency cycles with a number +`pip_parse` supports fixing multiple cycles simultaneously, however, cycles must +be distinct. `apache-airflow`, for instance, has dependency cycles with a number of its optional dependencies, which means those optional dependencies must all -be a part of the `airflow` cycle. For instance -- +be a part of the `airflow` cycle. For instance: ```starlark ... @@ -67,9 +67,9 @@ be a part of the `airflow` cycle. For instance -- Alternatively, one could resolve the cycle by removing one leg of it. -For example while `apache-airflow-providers-sqlite` is "baked into" the Airflow +For example, while `apache-airflow-providers-sqlite` is "baked into" the Airflow package, `apache-airflow-providers-postgres` is not and is an optional feature. -Rather than listing `apache-airflow[postgres]` in your `requirements.txt` which +Rather than listing `apache-airflow[postgres]` in your `requirements.txt`, which would expose a cycle via the extra, one could either _manually_ depend on `apache-airflow` and `apache-airflow-providers-postgres` separately as requirements. Bazel rules which need only `apache-airflow` can take it as a @@ -77,6 +77,6 @@ dependency, and rules which explicitly want to mix in `apache-airflow-providers-postgres` now can. Alternatively, one could use `rules_python`'s patching features to remove one -leg of the dependency manually. For instance by making +leg of the dependency manually, for instance, by making `apache-airflow-providers-postgres` not explicitly depend on `apache-airflow` or perhaps `apache-airflow-providers-common-sql`. diff --git a/docs/pypi/download-workspace.md b/docs/pypi/download-workspace.md index 48710095a4..5dfb0f257a 100644 --- a/docs/pypi/download-workspace.md +++ b/docs/pypi/download-workspace.md @@ -3,7 +3,7 @@ # Download (WORKSPACE) -This documentation page covers how to download the PyPI dependencies in the legacy `WORKSPACE` setup. +This documentation page covers how to download PyPI dependencies in the legacy `WORKSPACE` setup. To add pip dependencies to your `WORKSPACE`, load the `pip_parse` function and call it to create the central external repo and individual wheel external repos. @@ -27,7 +27,7 @@ install_deps() ## Interpreter selection -Note that pip parse runs before the Bazel before decides which Python toolchain to use, it cannot +Note that because `pip_parse` runs before Bazel decides which Python toolchain to use, it cannot enforce that the interpreter used to invoke `pip` matches the interpreter used to run `py_binary` targets. By default, `pip_parse` uses the system command `"python3"`. To override this, pass in the {attr}`pip_parse.python_interpreter` attribute or {attr}`pip_parse.python_interpreter_target`. @@ -44,9 +44,9 @@ your system `python` interpreter), you can force it to re-execute by running (per-os-arch-requirements)= ## Requirements for a specific OS/Architecture -In some cases you may need to use different requirements files for different OS, Arch combinations. +In some cases, you may need to use different requirements files for different OS and architecture combinations. This is enabled via the {attr}`pip_parse.requirements_by_platform` attribute. The keys of the -dictionary are labels to the file and the values are a list of comma separated target (os, arch) +dictionary are labels to the file, and the values are a list of comma-separated target (os, arch) tuples. For example: @@ -63,8 +63,8 @@ For example: requirements_lock = "requirements_lock.txt", ``` -In case of duplicate platforms, `rules_python` will raise an error as there has -to be unambiguous mapping of the requirement files to the (os, arch) tuples. +In case of duplicate platforms, `rules_python` will raise an error, as there has +to be an unambiguous mapping of the requirement files to the (os, arch) tuples. An alternative way is to use per-OS requirement attributes. ```starlark diff --git a/docs/pypi/download.md b/docs/pypi/download.md index 18d6699ab3..7f4e205d84 100644 --- a/docs/pypi/download.md +++ b/docs/pypi/download.md @@ -8,8 +8,8 @@ For WORKSPACE instructions see [here](./download-workspace). ::: To add PyPI dependencies to your `MODULE.bazel` file, use the `pip.parse` -extension, and call it to create the central external repo and individual wheel -external repos. Include in the `MODULE.bazel` the toolchain extension as shown +extension and call it to create the central external repo and individual wheel +external repos. Include the toolchain extension in the `MODULE.bazel` file as shown in the first bzlmod example above. ```starlark @@ -24,7 +24,7 @@ pip.parse( use_repo(pip, "my_deps") ``` -For more documentation, see the bzlmod examples under the {gh-path}`examples` folder or the documentation +For more documentation, see the Bzlmod examples under the {gh-path}`examples` folder or the documentation for the {obj}`@rules_python//python/extensions:pip.bzl` extension. :::note} @@ -42,7 +42,7 @@ difference. ## Interpreter selection -The {obj}`pip.parse` `bzlmod` extension by default uses the hermetic python toolchain for the host +The {obj}`pip.parse` `bzlmod` extension by default uses the hermetic Python toolchain for the host platform, but you can customize the interpreter using {attr}`pip.parse.python_interpreter` and {attr}`pip.parse.python_interpreter_target`. @@ -58,10 +58,10 @@ name]`. (per-os-arch-requirements)= ## Requirements for a specific OS/Architecture -In some cases you may need to use different requirements files for different OS, Arch combinations. -This is enabled via the `requirements_by_platform` attribute in `pip.parse` extension and the -{obj}`pip.parse` tag class. The keys of the dictionary are labels to the file and the values are a -list of comma separated target (os, arch) tuples. +In some cases, you may need to use different requirements files for different OS and architecture combinations. +This is enabled via the `requirements_by_platform` attribute in the `pip.parse` extension and the +{obj}`pip.parse` tag class. The keys of the dictionary are labels to the file, and the values are a +list of comma-separated target (os, arch) tuples. For example: ```starlark @@ -77,8 +77,8 @@ For example: requirements_lock = "requirements_lock.txt", ``` -In case of duplicate platforms, `rules_python` will raise an error as there has -to be unambiguous mapping of the requirement files to the (os, arch) tuples. +In case of duplicate platforms, `rules_python` will raise an error, as there has +to be an unambiguous mapping of the requirement files to the (os, arch) tuples. An alternative way is to use per-OS requirement attributes. ```starlark @@ -98,24 +98,24 @@ the lock file will be evaluated against, consider using the aforementioned ## Multi-platform support -Historically the {obj}`pip_parse` and {obj}`pip.parse` have been only downloading/building +Historically, the {obj}`pip_parse` and {obj}`pip.parse` have only been downloading/building Python dependencies for the host platform that the `bazel` commands are executed on. Over -the years people started needing support for building containers and usually that involves -fetching dependencies for a particular target platform that may be other than the host +the years, people started needing support for building containers, and usually, that involves +fetching dependencies for a particular target platform that may be different from the host platform. -Multi-platform support of cross-building the wheels can be done in two ways: +Multi-platform support for cross-building the wheels can be done in two ways: 1. using {attr}`experimental_index_url` for the {bzl:obj}`pip.parse` bzlmod tag class -2. using {attr}`pip.parse.download_only` setting. +2. using the {attr}`pip.parse.download_only` setting. :::{warning} -This will not for sdists with C extensions, but pure Python sdists may still work using the first +This will not work for sdists with C extensions, but pure Python sdists may still work using the first approach. ::: ### Using `download_only` attribute -Let's say you have 2 requirements files: +Let's say you have two requirements files: ``` # requirements.linux_x86_64.txt --platform=manylinux_2_17_x86_64 @@ -151,9 +151,9 @@ pip.parse( ) ``` -With this, the `pip.parse` will create a hub repository that is going to -support only two platforms - `cp39_osx_aarch64` and `cp39_linux_x86_64` and it -will only use `wheels` and ignore any sdists that it may find on the PyPI +With this, `pip.parse` will create a hub repository that is going to +support only two platforms - `cp39_osx_aarch64` and `cp39_linux_x86_64` - and it +will only use `wheels` and ignore any sdists that it may find on the PyPI- compatible indexes. :::{warning} @@ -162,7 +162,7 @@ multiple times. ::: :::{note} -This will only work for wheel-only setups, i.e. all of your dependencies need to have wheels +This will only work for wheel-only setups, i.e., all of your dependencies need to have wheels available on the PyPI index that you use. ::: @@ -173,9 +173,9 @@ Currently this is disabled by default, but you can turn it on using {envvar}`RULES_PYTHON_ENABLE_PIPSTAR` environment variable. ::: -In order to understand what dependencies to pull for a particular package +In order to understand what dependencies to pull for a particular package, `rules_python` parses the `whl` file [`METADATA`][metadata]. -Packages can express dependencies via `Requires-Dist` and they can add conditions using +Packages can express dependencies via `Requires-Dist`, and they can add conditions using "environment markers", which represent the Python version, OS, etc. While the PyPI integration provides reasonable defaults to support most @@ -198,8 +198,8 @@ additional keys, which become available during dependency evaluation. ### Bazel downloader and multi-platform wheel hub repository. :::{warning} -This is currently still experimental and whilst it has been proven to work in quite a few -environments, the APIs are still being finalized and there may be changes to the APIs for this +This is currently still experimental, and whilst it has been proven to work in quite a few +environments, the APIs are still being finalized, and there may be changes to the APIs for this feature without much notice. The issues that you can subscribe to for updates are: @@ -207,7 +207,7 @@ The issues that you can subscribe to for updates are: * {gh-issue}`1357` ::: -The {obj}`pip` extension supports pulling information from `PyPI` (or a compatible mirror) and it +The {obj}`pip` extension supports pulling information from `PyPI` (or a compatible mirror), and it will ensure that the [bazel downloader][bazel_downloader] is used for downloading the wheels. This provides the following benefits: @@ -222,7 +222,7 @@ To enable the feature specify {attr}`pip.parse.experimental_index_url` as shown the {gh-path}`examples/bzlmod/MODULE.bazel` example. Similar to [uv](https://docs.astral.sh/uv/configuration/indexes/), one can override the -index that is used for a single package. By default we first search in the index specified by +index that is used for a single package. By default, we first search in the index specified by {attr}`pip.parse.experimental_index_url`, then we iterate through the {attr}`pip.parse.experimental_extra_index_urls` unless there are overrides specified via {attr}`pip.parse.experimental_index_url_overrides`. @@ -235,12 +235,12 @@ Loading: 0 packages loaded ``` -This does not mean that `rules_python` is fetching the wheels eagerly, but it -rather means that it is calling the PyPI server to get the Simple API response +This does not mean that `rules_python` is fetching the wheels eagerly; rather, +it means that it is calling the PyPI server to get the Simple API response to get the list of all available source and wheel distributions. Once it has -got all of the available distributions, it will select the right ones depending +gotten all of the available distributions, it will select the right ones depending on the `sha256` values in your `requirements_lock.txt` file. If `sha256` hashes -are not present in the requirements file, we will fallback to matching by version +are not present in the requirements file, we will fall back to matching by version specified in the lock file. Fetching the distribution information from the PyPI allows `rules_python` to @@ -264,10 +264,10 @@ available flags: The [Bazel downloader](#bazel-downloader) usage allows for the Bazel [Credential Helper][cred-helper-design]. -Your python artifact registry may provide a credential helper for you. +Your Python artifact registry may provide a credential helper for you. Refer to your index's docs to see if one is provided. -The simplest form of a credential helper is a bash script that accepts an arg and spits out JSON to +The simplest form of a credential helper is a bash script that accepts an argument and spits out JSON to stdout. For a service like Google Artifact Registry that uses ['Basic' HTTP Auth][rfc7617] and does not provide a credential helper that conforms to the [spec][cred-helper-spec], the script might look like: @@ -285,7 +285,7 @@ echo ' }' echo '}' ``` -Configure Bazel to use this credential helper for your python index `example.com`: +Configure Bazel to use this credential helper for your Python index `example.com`: ``` # .bazelrc diff --git a/docs/pypi/index.md b/docs/pypi/index.md index c300124398..c32bafc609 100644 --- a/docs/pypi/index.md +++ b/docs/pypi/index.md @@ -3,11 +3,11 @@ # Using PyPI -Using PyPI packages (aka "pip install") involves the following main steps. +Using PyPI packages (aka "pip install") involves the following main steps: 1. [Generating requirements file](./lock) -2. Installing third party packages in [bzlmod](./download) or [WORKSPACE](./download-workspace). -3. [Using third party packages as dependencies](./use) +2. Installing third-party packages in [bzlmod](./download) or [WORKSPACE](./download-workspace). +3. [Using third-party packages as dependencies](./use) With the advanced topics covered separately: * Dealing with [circular dependencies](./circular-dependencies). diff --git a/docs/pypi/lock.md b/docs/pypi/lock.md index c9376036fb..db557fe594 100644 --- a/docs/pypi/lock.md +++ b/docs/pypi/lock.md @@ -11,9 +11,14 @@ Currently `rules_python` only supports `requirements.txt` format. ### pip compile -Generally, when working on a Python project, you'll have some dependencies that themselves have other dependencies. You might also specify dependency bounds instead of specific versions. So you'll need to generate a full list of all transitive dependencies and pinned versions for every dependency. - -Typically, you'd have your project dependencies specified in `pyproject.toml` or `requirements.in` and generate the full pinned list of dependencies in `requirements_lock.txt`, which you can manage with the {obj}`compile_pip_requirements`: +Generally, when working on a Python project, you'll have some dependencies that themselves have +other dependencies. You might also specify dependency bounds instead of specific versions. +So you'll need to generate a full list of all transitive dependencies and pinned versions +for every dependency. + +Typically, you'd have your project dependencies specified in `pyproject.toml` or `requirements.in` +and generate the full pinned list of dependencies in `requirements_lock.txt`, which you can +manage with {obj}`compile_pip_requirements`: ```starlark load("@rules_python//python:pip.bzl", "compile_pip_requirements") diff --git a/docs/pypi/patch.md b/docs/pypi/patch.md index f341bd1091..7e3cb41981 100644 --- a/docs/pypi/patch.md +++ b/docs/pypi/patch.md @@ -4,7 +4,7 @@ # Patching wheels Sometimes the wheels have to be patched to: -* Workaround the lack of a standard `site-packages` layout ({gh-issue}`2156`) -* Include certain PRs of your choice on top of wheels and avoid building from sdist, +* Workaround the lack of a standard `site-packages` layout ({gh-issue}`2156`). +* Include certain PRs of your choice on top of wheels and avoid building from sdist. You can patch the wheels by using the {attr}`pip.override.patches` attribute. diff --git a/docs/pypi/use.md b/docs/pypi/use.md index 7a16b7d9e9..6212097f86 100644 --- a/docs/pypi/use.md +++ b/docs/pypi/use.md @@ -3,10 +3,10 @@ # Use in BUILD.bazel files -Once you have setup the dependencies, you are ready to start using them in your `BUILD.bazel` -files. If you haven't done so yet, set it up by following the following docs: +Once you have set up the dependencies, you are ready to start using them in your `BUILD.bazel` +files. If you haven't done so yet, set it up by following these docs: 1. [WORKSPACE](./download-workspace) -1. [bzlmod](./download) +2. [bzlmod](./download) To refer to targets in a hub repo `pypi`, you can do one of two things: ```starlark @@ -29,19 +29,19 @@ py_library( ) ``` -Note, that the usage of the `requirement` helper is not advised and can be problematic. See the +Note that the usage of the `requirement` helper is not advised and can be problematic. See the [notes below](#requirement-helper). -Note, that the hub repo contains the following targets for each package: -* `@pypi//numpy` which is a shorthand for `@pypi//numpy:numpy`. This is an {obj}`alias` to +Note that the hub repo contains the following targets for each package: +* `@pypi//numpy` - shorthand for `@pypi//numpy:numpy`. This is an {obj}`alias` to `@pypi//numpy:pkg`. * `@pypi//numpy:pkg` - the {obj}`py_library` target automatically generated by the repository rules. -* `@pypi//numpy:data` - the {obj}`filegroup` that is for all of the extra files that are included +* `@pypi//numpy:data` - the {obj}`filegroup` for all of the extra files that are included as data in the `pkg` target. -* `@pypi//numpy:dist_info` - the {obj}`filegroup` that is for all of the files in the `.distinfo` directory. -* `@pypi//numpy:whl` - the {obj}`filegroup` that is the `.whl` file itself which includes all of - the transitive dependencies via the {attr}`filegroup.data` attribute. +* `@pypi//numpy:dist_info` - the {obj}`filegroup` for all of the files in the `.distinfo` directory. +* `@pypi//numpy:whl` - the {obj}`filegroup` that is the `.whl` file itself, which includes all + transitive dependencies via the {attr}`filegroup.data` attribute. ## Entry points @@ -52,14 +52,14 @@ which can help you create a `py_binary` target for a particular console script e ## 'Extras' dependencies -Any 'extras' specified in the requirements lock file will be automatically added +Any "extras" specified in the requirements lock file will be automatically added as transitive dependencies of the package. In the example above, you'd just put `requirement("useful_dep")` or `@pypi//useful_dep`. ## Consuming Wheel Dists Directly -If you need to depend on the wheel dists themselves, for instance, to pass them -to some other packaging tool, you can get a handle to them with the +If you need to depend on the wheel dists themselves (for instance, to pass them +to some other packaging tool), you can get a handle to them with the `whl_requirement` macro. For example: ```starlark @@ -77,7 +77,7 @@ filegroup( ## Creating a filegroup of files within a whl The rule {obj}`whl_filegroup` exists as an easy way to extract the necessary files -from a whl file without the need to modify the `BUILD.bazel` contents of the +from a whl file without needing to modify the `BUILD.bazel` contents of the whl repositories generated via `pip_repository`. Use it similarly to the `filegroup` above. See the API docs for more information. @@ -104,16 +104,16 @@ py_library( ) ``` -The reason `requirement()` exists is to insulate from +The reason `requirement()` exists is to insulate users from changes to the underlying repository and label strings. However, those -labels have become directly used, so aren't able to easily change regardless. +labels have become directly used, so they aren't able to easily change regardless. -On the other hand, using `requirement()` helper has several drawbacks: +On the other hand, using the `requirement()` helper has several drawbacks: -- It doesn't work with `buildifier` -- It doesn't work with `buildozer` -- It adds extra layer on top of normal mechanisms to refer to targets. -- It does not scale well as each type of target needs a new macro to be loaded and imported. +- It doesn't work with `buildifier`. +- It doesn't work with `buildozer`. +- It adds an extra layer on top of normal mechanisms to refer to targets. +- It does not scale well, as each type of target needs a new macro to be loaded and imported. If you don't want to use `requirement()`, you can use the library labels directly instead. For `pip_parse`, the labels are of the following form: diff --git a/docs/repl.md b/docs/repl.md index edcf37e811..1434097fdf 100644 --- a/docs/repl.md +++ b/docs/repl.md @@ -1,6 +1,6 @@ # Getting a REPL or Interactive Shell -rules_python provides a REPL to help with debugging and developing. The goal of +`rules_python` provides a REPL to help with debugging and developing. The goal of the REPL is to present an environment identical to what a {bzl:obj}`py_binary` creates for your code. diff --git a/docs/support.md b/docs/support.md index 5e6de57fcb..ad943b3845 100644 --- a/docs/support.md +++ b/docs/support.md @@ -8,7 +8,7 @@ page for information on our development workflow. ## Supported rules_python Versions In general, only the latest version is supported. Backporting changes is -done on a best effort basis based on severity, risk of regressions, and +done on a best-effort basis based on severity, risk of regressions, and the willingness of volunteers. If you want or need particular functionality backported, then the best way @@ -33,24 +33,24 @@ for what versions are the rolling, active, and prior releases. ## Supported Python versions -As a general rule we test all released non-EOL Python versions. Different +As a general rule, we test all released non-EOL Python versions. Different interpreter versions may work but are not guaranteed. We are interested in staying compatible with upcoming unreleased versions, so if you see that things stop working, please create tickets or, more preferably, pull requests. ## Supported Platforms -We only support the platforms that our continuous integration jobs run, which -is Linux, Mac, and Windows. +We only support the platforms that our continuous integration jobs run on, which +are Linux, Mac, and Windows. -In order to better describe different support levels, the below acts as a rough +In order to better describe different support levels, the following acts as a rough guideline for different platform tiers: -* Tier 0 - The platforms that our CI runs: `linux_x86_64`, `osx_x86_64`, `RBE linux_x86_64`. -* Tier 1 - The platforms that are similar enough to what the CI runs: `linux_aarch64`, `osx_arm64`. - What is more, `windows_x86_64` is in this list as we run tests in CI but - developing for Windows is more challenging and features may come later to +* Tier 0 - The platforms that our CI runs on: `linux_x86_64`, `osx_x86_64`, `RBE linux_x86_64`. +* Tier 1 - The platforms that are similar enough to what the CI runs on: `linux_aarch64`, `osx_arm64`. + What is more, `windows_x86_64` is in this list, as we run tests in CI, but + developing for Windows is more challenging, and features may come later to this platform. -* Tier 2 - The rest of the platforms that may have varying level of support, e.g. +* Tier 2 - The rest of the platforms that may have a varying level of support, e.g., `linux_s390x`, `linux_ppc64le`, `windows_arm64`. :::{note} @@ -75,7 +75,7 @@ a series of releases to so users can still incrementally upgrade. See the ## Experimental Features -An experimental features is functionality that may not be ready for general +An experimental feature is functionality that may not be ready for general use and may change quickly and/or significantly. Such features are denoted in their name or API docs as "experimental". They may have breaking changes made at any time. diff --git a/docs/toolchains.md b/docs/toolchains.md index 668a458156..de819cb515 100644 --- a/docs/toolchains.md +++ b/docs/toolchains.md @@ -4,13 +4,13 @@ (configuring-toolchains)= # Configuring Python toolchains and runtimes -This documents how to configure the Python toolchain and runtimes for different +This document explains how to configure the Python toolchain and runtimes for different use cases. ## Bzlmod MODULE configuration -How to configure `rules_python` in your MODULE.bazel file depends on how and why -you're using Python. There are 4 basic use cases: +How to configure `rules_python` in your `MODULE.bazel` file depends on how and why +you're using Python. There are four basic use cases: 1. A root module that always uses Python. For example, you're building a Python application. @@ -51,7 +51,7 @@ python.toolchain(python_version = "3.12") ### Library modules A library module is a module that can show up in arbitrary locations in the -bzlmod module graph -- it's unknown where in the breadth-first search order the +Bzlmod module graph -- it's unknown where in the breadth-first search order the module will be relative to other modules. For example, `rules_python` is a library module. @@ -84,9 +84,9 @@ used for the Python programs it runs isn't chosen by the module itself. Instead, it's up to the root module to pick an appropriate version of Python. For this case, configuration is simple: just depend on `rules_python` and use -the normal `//python:py_binary.bzl` et al rules. There is no need to call -`python.toolchain` -- rules_python ensures _some_ Python version is available, -but more often the root module will specify some version. +the normal `//python:py_binary.bzl` et al. rules. There is no need to call +`python.toolchain` -- `rules_python` ensures _some_ Python version is available, +but more often, the root module will specify some version. ``` # MODULE.bazel @@ -108,7 +108,7 @@ specific Python version be used with its tools. This has some pros/cons: * It has higher build overhead because additional runtimes and libraries need to be downloaded, and Bazel has to keep additional configuration state. -To configure this, request the Python versions needed in MODULE.bazel and use +To configure this, request the Python versions needed in `MODULE.bazel` and use the version-aware rules for `py_binary`. ``` @@ -132,7 +132,7 @@ is most useful for two cases: 1. For submodules to ensure they run with the appropriate Python version 2. To allow incremental, per-target, upgrading to newer Python versions, - typically in a mono-repo situation. + typically in a monorepo situation. To configure a submodule with the version-aware rules, request the particular version you need when defining the toolchain: @@ -147,7 +147,7 @@ python.toolchain( use_repo(python) ``` -Then use the `@rules_python` repo in your BUILD file to explicity pin the Python version when calling the rule: +Then use the `@rules_python` repo in your `BUILD` file to explicitly pin the Python version when calling the rule: ```starlark # BUILD.bazel @@ -202,29 +202,29 @@ The `python.toolchain()` call makes its contents available under a repo named `python_X_Y`, where X and Y are the major and minor versions. For example, `python.toolchain(python_version="3.11")` creates the repo `@python_3_11`. Remember to call `use_repo()` to make repos visible to your module: -`use_repo(python, "python_3_11")` +`use_repo(python, "python_3_11")`. :::{deprecated} 1.1.0 -The toolchain specific `py_binary` and `py_test` symbols are aliases to the regular rules. -i.e. Deprecated `load("@python_versions//3.11:defs.bzl", "py_binary")` & `load("@python_versions//3.11:defs.bzl", "py_test")` +The toolchain-specific `py_binary` and `py_test` symbols are aliases to the regular rules. +For example, `load("@python_versions//3.11:defs.bzl", "py_binary")` & `load("@python_versions//3.11:defs.bzl", "py_test")` are deprecated. -Usages of them should be changed to load the regular rules directly; -i.e. Use `load("@rules_python//python:py_binary.bzl", "py_binary")` & `load("@rules_python//python:py_test.bzl", "py_test")` and then specify the `python_version` when using the rules corresponding to the python version you defined in your toolchain. {ref}`Library modules with version constraints` +Usages of them should be changed to load the regular rules directly. +For example, use `load("@rules_python//python:py_binary.bzl", "py_binary")` & `load("@rules_python//python:py_test.bzl", "py_test")` and then specify the `python_version` when using the rules corresponding to the Python version you defined in your toolchain. {ref}`Library modules with version constraints` ::: #### Toolchain usage in other rules -Python toolchains can be utilized in other bazel rules, such as `genrule()`, by +Python toolchains can be utilized in other Bazel rules, such as `genrule()`, by adding the `toolchains=["@rules_python//python:current_py_toolchain"]` attribute. You can obtain the path to the Python interpreter using the `$(PYTHON2)` and `$(PYTHON3)` ["Make" Variables](https://bazel.build/reference/be/make-variables). See the {gh-path}`test_current_py_toolchain ` target -for an example. We also make available `$(PYTHON2_ROOTPATH)` and `$(PYTHON3_ROOTPATH)` +for an example. We also make available `$(PYTHON2_ROOTPATH)` and `$(PYTHON3_ROOTPATH)`, which are Make Variable equivalents of `$(PYTHON2)` and `$(PYTHON3)` but for runfiles -locations. These will be helpful if you need to set env vars of binary/test rules +locations. These will be helpful if you need to set environment variables of binary/test rules while using [`--nolegacy_external_runfiles`](https://bazel.build/reference/command-line-reference#flag--legacy_external_runfiles). The original make variables still work in exec contexts such as genrules. @@ -246,9 +246,9 @@ existing attributes: ### Registering custom runtimes Because the python-build-standalone project has _thousands_ of prebuilt runtimes -available, rules_python only includes popular runtimes in its built in +available, `rules_python` only includes popular runtimes in its built-in configurations. If you want to use a runtime that isn't already known to -rules_python then {obj}`single_version_platform_override()` can be used to do +`rules_python`, then {obj}`single_version_platform_override()` can be used to do so. In short, it allows specifying an arbitrary URL and using custom flags to control when a runtime is used. @@ -287,21 +287,21 @@ config_setting( ``` Notes: -- While any URL and archive can be used, it's assumed their content looks how - a python-build-standalone archive looks. -- A "version aware" toolchain is registered, which means the Python version flag - must also match (e.g. `--@rules_python//python/config_settings:python_version=3.13.3` +- While any URL and archive can be used, it's assumed their content looks like + a python-build-standalone archive. +- A "version-aware" toolchain is registered, which means the Python version flag + must also match (e.g., `--@rules_python//python/config_settings:python_version=3.13.3` must be set -- see `minor_mapping` and `is_default` for controls and docs about version matching and selection). - The `target_compatible_with` attribute can be used to entirely specify the - arg of the same name the toolchain uses. + argument of the same name that the toolchain uses. - The labels in `target_settings` must be absolute; `@@` refers to the main repo. - The `target_settings` are `config_setting` targets, which means you can customize how matching occurs. :::{seealso} -See {obj}`//python/config_settings` for flags rules_python already defines -that can be used with `target_settings`. Some particular ones of note are: +See {obj}`//python/config_settings` for flags `rules_python` already defines +that can be used with `target_settings`. Some particular ones of note are {flag}`--py_linux_libc` and {flag}`--py_freethreaded`, among others. ::: @@ -312,7 +312,7 @@ Added support for custom platform names, `target_compatible_with`, and ### Using defined toolchains from WORKSPACE -It is possible to use toolchains defined in `MODULE.bazel` in `WORKSPACE`. For example +It is possible to use toolchains defined in `MODULE.bazel` in `WORKSPACE`. For example, the following `MODULE.bazel` and `WORKSPACE` provides a working {bzl:obj}`pip_parse` setup: ```starlark # File: WORKSPACE @@ -343,16 +343,16 @@ python.toolchain(python_version = "3.10") use_repo(python, "python_3_10", "python_3_10_host") ``` -Note, the user has to import the `*_host` repository to use the python interpreter in the -{bzl:obj}`pip_parse` and `whl_library` repository rules and once that is done +Note, the user has to import the `*_host` repository to use the Python interpreter in the +{bzl:obj}`pip_parse` and `whl_library` repository rules, and once that is done, users should be able to ensure the setting of the default toolchain even during the transition period when some of the code is still defined in `WORKSPACE`. ## Workspace configuration -To import rules_python in your project, you first need to add it to your +To import `rules_python` in your project, you first need to add it to your `WORKSPACE` file, using the snippet provided in the -[release you choose](https://github.com/bazel-contrib/rules_python/releases) +[release you choose](https://github.com/bazel-contrib/rules_python/releases). To depend on a particular unreleased version, you can do the following: @@ -403,15 +403,15 @@ pip_parse( ``` After registration, your Python targets will use the toolchain's interpreter during execution, but a system-installed interpreter -is still used to 'bootstrap' Python targets (see https://github.com/bazel-contrib/rules_python/issues/691). +is still used to "bootstrap" Python targets (see https://github.com/bazel-contrib/rules_python/issues/691). You may also find some quirks while using this toolchain. Please refer to [python-build-standalone documentation's _Quirks_ section](https://gregoryszorc.com/docs/python-build-standalone/main/quirks.html). ## Local toolchain It's possible to use a locally installed Python runtime instead of the regular prebuilt, remotely downloaded ones. A local toolchain contains the Python -runtime metadata (Python version, headers, ABI flags, etc) that the regular -remotely downloaded runtimes contain, which makes it possible to build e.g. C +runtime metadata (Python version, headers, ABI flags, etc.) that the regular +remotely downloaded runtimes contain, which makes it possible to build, e.g., C extensions (unlike the autodetecting and runtime environment toolchains). For simple cases, the {obj}`local_runtime_repo` and @@ -420,10 +420,10 @@ Python installation and create an appropriate Bazel definition from it. To do this, three pieces need to be wired together: 1. Specify a path or command to a Python interpreter (multiple can be defined). -2. Create toolchains for the runtimes in (1) -3. Register the toolchains created by (2) +2. Create toolchains for the runtimes in (1). +3. Register the toolchains created by (2). -The below is an example that will use `python3` from PATH to find the +The following is an example that will use `python3` from `PATH` to find the interpreter, then introspect its installation to generate a full toolchain. ```starlark @@ -474,7 +474,7 @@ Python versions and/or platforms to be configured in a single `MODULE.bazel`. Note that `register_toolchains` will insert the local toolchain earlier in the toolchain ordering, so it will take precedence over other registered toolchains. To better control when the toolchain is used, see [Conditionally using local -toolchains] +toolchains]. ### Conditionally using local toolchains @@ -483,22 +483,22 @@ ordering, which means it will usually be used no matter what. This can be problematic for CI (where it shouldn't be used), expensive for CI (CI must initialize/download the repository to determine its Python version), and annoying for iterative development (enabling/disabling it requires modifying -MODULE.bazel). +`MODULE.bazel`). These behaviors can be mitigated, but it requires additional configuration -to avoid triggering the local toolchain repository to initialize (i.e. run +to avoid triggering the local toolchain repository to initialize (i.e., run local commands and perform downloads). The two settings to change are {obj}`local_runtime_toolchains_repo.target_compatible_with` and {obj}`local_runtime_toolchains_repo.target_settings`, which control how Bazel decides if a toolchain should match. By default, they point to targets *within* -the local runtime repository (trigger repo initialization). We have to override +the local runtime repository (triggering repo initialization). We have to override them to *not* reference the local runtime repository at all. In the example below, we reconfigure the local toolchains so they are only activated if the custom flag `--//:py=local` is set and the target platform -matches the Bazel host platform. The net effect is CI won't use the local +matches the Bazel host platform. The net effect is that CI won't use the local toolchain (nor initialize its repository), and developers can easily enable/disable the local toolchain with a command line flag. @@ -545,9 +545,9 @@ information about Python at build time. In particular, this means it is not able to build C extensions -- doing so requires knowing, at build time, what Python headers to use. -In effect, all it does is generate a small wrapper script that simply calls e.g. +In effect, all it does is generate a small wrapper script that simply calls, e.g., `/usr/bin/env python3` to run a program. This makes it easy to change what -Python is used to run a program, but also makes it easy to use a Python version +Python is used to run a program but also makes it easy to use a Python version that isn't compatible with build-time assumptions. ``` @@ -565,26 +565,26 @@ locally installed Python. ### Autodetecting toolchain The autodetecting toolchain is a deprecated toolchain that is built into Bazel. -**It's name is a bit misleading: it doesn't autodetect anything**. All it does is +**Its name is a bit misleading: it doesn't autodetect anything.** All it does is use `python3` from the environment a binary runs within. This provides extremely limited functionality to the rules (at build time, nothing is knowable about the Python runtime). Bazel itself automatically registers `@bazel_tools//tools/python:autodetecting_toolchain` -as the lowest priority toolchain. For WORKSPACE builds, if no other toolchain -is registered, that toolchain will be used. For bzlmod builds, rules_python +as the lowest priority toolchain. For `WORKSPACE` builds, if no other toolchain +is registered, that toolchain will be used. For Bzlmod builds, `rules_python` automatically registers a higher-priority toolchain; it won't be used unless there is a toolchain misconfiguration somewhere. -To aid migration off the Bazel-builtin toolchain, rules_python provides +To aid migration off the Bazel-builtin toolchain, `rules_python` provides {bzl:obj}`@rules_python//python/runtime_env_toolchains:all`. This is an equivalent -toolchain, but is implemented using rules_python's objects. +toolchain but is implemented using `rules_python`'s objects. ## Custom toolchains -While rules_python provides toolchains by default, it is not required to use +While `rules_python` provides toolchains by default, it is not required to use them, and you can define your own toolchains to use instead. This section -gives an introduction for how to define them yourself. +gives an introduction to how to define them yourself. :::{note} * Defining your own toolchains is an advanced feature. @@ -599,7 +599,7 @@ toolchains a "toolchain suite". One of the underlying design goals of the toolchains is to support complex and bespoke environments. Such environments may use an arbitrary combination of {bzl:obj}`RBE`, cross-platform building, multiple Python versions, -building Python from source, embeding Python (as opposed to building separate +building Python from source, embedding Python (as opposed to building separate interpreters), using prebuilt binaries, or using binaries built from source. To that end, many of the attributes they accept, and fields they provide, are optional. @@ -610,7 +610,7 @@ The target toolchain type is {obj}`//python:toolchain_type`, and it is for _target configuration_ runtime information, e.g., the Python version and interpreter binary that a program will use. -The is typically implemented using {obj}`py_runtime()`, which +This is typically implemented using {obj}`py_runtime()`, which provides the {obj}`PyRuntimeInfo` provider. For historical reasons from the Python 2 transition, `py_runtime` is wrapped in {obj}`py_runtime_pair`, which provides {obj}`ToolchainInfo` with the field `py3_runtime`, which is an @@ -625,7 +625,7 @@ set {external:bzl:obj}`toolchain.exec_compatible_with`. ### Python C toolchain type The Python C toolchain type ("py cc") is {obj}`//python/cc:toolchain_type`, and -it has C/C++ information for the _target configuration_, e.g. the C headers that +it has C/C++ information for the _target configuration_, e.g., the C headers that provide `Python.h`. This is typically implemented using {obj}`py_cc_toolchain()`, which provides @@ -642,7 +642,7 @@ set {external:bzl:obj}`toolchain.exec_compatible_with`. ### Exec tools toolchain type The exec tools toolchain type is {obj}`//python:exec_tools_toolchain_type`, -and it is for supporting tools for _building_ programs, e.g. the binary to +and it is for supporting tools for _building_ programs, e.g., the binary to precompile code at build time. This toolchain type is intended to hold only _exec configuration_ values -- @@ -661,7 +661,7 @@ target configuration (e.g. Python version), then for one to be chosen based on finding one compatible with the available host platforms to run the tool on. However, what `target_compatible_with`/`target_settings` and -`exec_compatible_with` values to use depend on details of the tools being used. +`exec_compatible_with` values to use depends on the details of the tools being used. For example: * If you had a precompiler that supported any version of Python, then putting the Python version in `target_settings` is unnecessary. @@ -672,9 +672,9 @@ This can work because, when the rules invoke these build tools, they pass along all necessary information so that the tool can be entirely independent of the target configuration being built for. -Alternatively, if you had a precompiler that only ran on linux, and only -produced valid output for programs intended to run on linux, then _both_ -`exec_compatible_with` and `target_compatible_with` must be set to linux. +Alternatively, if you had a precompiler that only ran on Linux and only +produced valid output for programs intended to run on Linux, then _both_ +`exec_compatible_with` and `target_compatible_with` must be set to Linux. ### Custom toolchain example @@ -684,9 +684,9 @@ Here, we show an example for a semi-complicated toolchain suite, one that is: * For Python version 3.12.0 * Using an in-build interpreter built from source * That only runs on Linux -* Using a prebuilt precompiler that only runs on Linux, and only produces byte - code valid for 3.12 -* With the exec tools interpreter disabled (unnecessary with a prebuild +* Using a prebuilt precompiler that only runs on Linux and only produces + bytecode valid for 3.12 +* With the exec tools interpreter disabled (unnecessary with a prebuilt precompiler) * Providing C headers and libraries @@ -748,13 +748,13 @@ toolchain( name = "runtime_toolchain", toolchain = "//toolchain_impl:runtime_pair", toolchain_type = "@rules_python//python:toolchain_type", - target_compatible_with = ["@platforms/os:linux"] + target_compatible_with = ["@platforms/os:linux"], ) toolchain( name = "py_cc_toolchain", toolchain = "//toolchain_impl:py_cc_toolchain_impl", toolchain_type = "@rules_python//python/cc:toolchain_type", - target_compatible_with = ["@platforms/os:linux"] + target_compatible_with = ["@platforms/os:linux"], ) toolchain( @@ -764,19 +764,19 @@ toolchain( target_settings = [ "@rules_python//python/config_settings:is_python_3.12", ], - exec_comaptible_with = ["@platforms/os:linux"] + exec_compatible_with = ["@platforms/os:linux"], ) # ----------------------------------------------- # File: MODULE.bazel or WORKSPACE.bazel -# These toolchains will considered before others. +# These toolchains will be considered before others. # ----------------------------------------------- register_toolchains("//toolchains:all") ``` -When registering custom toolchains, be aware of the the [toolchain registration +When registering custom toolchains, be aware of the [toolchain registration order](https://bazel.build/extending/toolchains#toolchain-resolution). In brief, -toolchain order is the BFS-order of the modules; see the bazel docs for a more +toolchain order is the BFS-order of the modules; see the Bazel docs for a more detailed description. :::{note} @@ -796,7 +796,7 @@ Currently the following flags are used to influence toolchain selection: To run the interpreter that Bazel will use, you can use the `@rules_python//python/bin:python` target. This is a binary target with -the executable pointing at the `python3` binary plus its relevent runfiles. +the executable pointing at the `python3` binary plus its relevant runfiles. ```console $ bazel run @rules_python//python/bin:python @@ -838,7 +838,7 @@ targets on its own. Please file a feature request if this is desired. The `//python/bin:python` target provides access to the underlying interpreter without any hermeticity guarantees. -The [`//python/bin:repl` target](repl) provides an environment indentical to +The [`//python/bin:repl` target](repl) provides an environment identical to what `py_binary` provides. That means it handles things like the [`PYTHONSAFEPATH`](https://docs.python.org/3/using/cmdline.html#envvar-PYTHONSAFEPATH) environment variable automatically. The `//python/bin:python` target will not. From 036e8c5af1258cf1a0b318a51c75f88ea4c93f11 Mon Sep 17 00:00:00 2001 From: yushan26 <107004874+yushan26@users.noreply.github.com> Date: Sat, 21 Jun 2025 19:01:39 -0700 Subject: [PATCH 294/922] feat(gazelle): For package mode, resolve dependencies when imports are relative to the package path (#2865) When `# gazelle:python_generation_mode package` is enabled, relative imports are currently not being added to the `deps` field of the generated target. For example, given the following Python code: ``` from .library import add as _add from .library import divide as _divide from .library import multiply as _multiply from .library import subtract as _subtract ``` The expected py_library rule should include a dependency on the local library package: ``` py_library( name = "py_default_library", srcs = ["__init__.py"], visibility = ["//visibility:public"], deps = [ "//example/library:py_default_library", ], ) ``` However, the actual generated rule is missing the deps entry: ``` py_library( name = "py_default_library", srcs = ["__init__.py"], visibility = ["//visibility:public"], ) ``` This change updates file_parser.go to ensure that relative imports (those starting with a .) are parsed and preserved. In `Resolve()`, logic is added to correctly interpret relative paths: A single dot (.) refers to the current package. Multiple dots (.., ..., etc.) traverse up parent directories. The relative import is resolved against the current label.Pkg path that imports the module and converted into an path relative to the root before dependency resolution. As a result, dependencies for relative imports are now correctly added to the deps field in package generation mode. Added a directive `# gazelle:experimental_allow_relative_imports true` to allow this feature to be opt in. --------- Co-authored-by: yushan Co-authored-by: Ignas Anikevicius <240938+aignas@users.noreply.github.com> Co-authored-by: Douglas Thor --- CHANGELOG.md | 3 ++ gazelle/README.md | 48 +++++++++++++++-- gazelle/python/configure.go | 8 +++ gazelle/python/file_parser.go | 4 +- gazelle/python/resolve.go | 53 ++++++++++++++++++- .../testdata/relative_imports/README.md | 4 -- .../relative_imports_package_mode/BUILD.in | 2 + .../relative_imports_package_mode/BUILD.out | 15 ++++++ .../relative_imports_package_mode/README.md | 6 +++ .../WORKSPACE | 0 .../relative_imports_package_mode/__main__.py | 5 ++ .../package1}/BUILD.in | 0 .../package1/BUILD.out | 11 ++++ .../package1/__init__.py | 2 + .../package1/module1.py | 0 .../package1/module2.py | 0 .../package1/my_library/BUILD.in | 7 +++ .../package1/my_library/BUILD.out | 7 +++ .../package1/my_library/__init__.py | 2 + .../package1/my_library/foo/BUILD.in | 0 .../package1/my_library/foo/BUILD.out | 7 +++ .../package1/my_library/foo/__init__.py | 2 + .../package1/subpackage1/BUILD.in | 10 ++++ .../package1/subpackage1/BUILD.out | 10 ++++ .../package1/subpackage1/__init__.py | 3 ++ .../package1/subpackage1/some_module.py | 3 ++ .../package1/subpackage1/subpackage2/BUILD.in | 0 .../subpackage1/subpackage2/BUILD.out | 16 ++++++ .../subpackage1/subpackage2/__init__.py | 0 .../subpackage1/subpackage2/library/BUILD.in | 0 .../subpackage1/subpackage2/library/BUILD.out | 7 +++ .../subpackage2/library/other_module.py | 0 .../subpackage1/subpackage2/script.py | 11 ++++ .../package2/BUILD.in | 0 .../package2/BUILD.out | 12 +++++ .../package2/__init__.py | 20 +++++++ .../package2/library/BUILD.in | 0 .../package2/library/BUILD.out | 7 +++ .../package2/library/__init__.py | 14 +++++ .../package2/module3.py | 5 ++ .../package2/module4.py | 2 + .../test.yaml | 0 .../BUILD.in | 1 + .../BUILD.out | 7 +-- .../relative_imports_project_mode/README.md | 5 ++ .../relative_imports_project_mode/WORKSPACE | 1 + .../__main__.py | 0 .../package1/module1.py | 19 +++++++ .../package1/module2.py | 17 ++++++ .../package2/BUILD.in | 0 .../package2/BUILD.out | 0 .../package2/__init__.py | 0 .../package2/module3.py | 0 .../package2/module4.py | 0 .../package2/subpackage1/module5.py | 0 .../relative_imports_project_mode/test.yaml | 15 ++++++ gazelle/pythonconfig/pythonconfig.go | 16 ++++++ 57 files changed, 373 insertions(+), 14 deletions(-) delete mode 100644 gazelle/python/testdata/relative_imports/README.md create mode 100644 gazelle/python/testdata/relative_imports_package_mode/BUILD.in create mode 100644 gazelle/python/testdata/relative_imports_package_mode/BUILD.out create mode 100644 gazelle/python/testdata/relative_imports_package_mode/README.md rename gazelle/python/testdata/{relative_imports => relative_imports_package_mode}/WORKSPACE (100%) create mode 100644 gazelle/python/testdata/relative_imports_package_mode/__main__.py rename gazelle/python/testdata/{relative_imports/package2 => relative_imports_package_mode/package1}/BUILD.in (100%) create mode 100644 gazelle/python/testdata/relative_imports_package_mode/package1/BUILD.out create mode 100644 gazelle/python/testdata/relative_imports_package_mode/package1/__init__.py rename gazelle/python/testdata/{relative_imports => relative_imports_package_mode}/package1/module1.py (100%) rename gazelle/python/testdata/{relative_imports => relative_imports_package_mode}/package1/module2.py (100%) create mode 100644 gazelle/python/testdata/relative_imports_package_mode/package1/my_library/BUILD.in create mode 100644 gazelle/python/testdata/relative_imports_package_mode/package1/my_library/BUILD.out create mode 100644 gazelle/python/testdata/relative_imports_package_mode/package1/my_library/__init__.py create mode 100644 gazelle/python/testdata/relative_imports_package_mode/package1/my_library/foo/BUILD.in create mode 100644 gazelle/python/testdata/relative_imports_package_mode/package1/my_library/foo/BUILD.out create mode 100644 gazelle/python/testdata/relative_imports_package_mode/package1/my_library/foo/__init__.py create mode 100644 gazelle/python/testdata/relative_imports_package_mode/package1/subpackage1/BUILD.in create mode 100644 gazelle/python/testdata/relative_imports_package_mode/package1/subpackage1/BUILD.out create mode 100644 gazelle/python/testdata/relative_imports_package_mode/package1/subpackage1/__init__.py create mode 100644 gazelle/python/testdata/relative_imports_package_mode/package1/subpackage1/some_module.py create mode 100644 gazelle/python/testdata/relative_imports_package_mode/package1/subpackage1/subpackage2/BUILD.in create mode 100644 gazelle/python/testdata/relative_imports_package_mode/package1/subpackage1/subpackage2/BUILD.out create mode 100644 gazelle/python/testdata/relative_imports_package_mode/package1/subpackage1/subpackage2/__init__.py create mode 100644 gazelle/python/testdata/relative_imports_package_mode/package1/subpackage1/subpackage2/library/BUILD.in create mode 100644 gazelle/python/testdata/relative_imports_package_mode/package1/subpackage1/subpackage2/library/BUILD.out create mode 100644 gazelle/python/testdata/relative_imports_package_mode/package1/subpackage1/subpackage2/library/other_module.py create mode 100644 gazelle/python/testdata/relative_imports_package_mode/package1/subpackage1/subpackage2/script.py create mode 100644 gazelle/python/testdata/relative_imports_package_mode/package2/BUILD.in create mode 100644 gazelle/python/testdata/relative_imports_package_mode/package2/BUILD.out create mode 100644 gazelle/python/testdata/relative_imports_package_mode/package2/__init__.py create mode 100644 gazelle/python/testdata/relative_imports_package_mode/package2/library/BUILD.in create mode 100644 gazelle/python/testdata/relative_imports_package_mode/package2/library/BUILD.out create mode 100644 gazelle/python/testdata/relative_imports_package_mode/package2/library/__init__.py create mode 100644 gazelle/python/testdata/relative_imports_package_mode/package2/module3.py create mode 100644 gazelle/python/testdata/relative_imports_package_mode/package2/module4.py rename gazelle/python/testdata/{relative_imports => relative_imports_package_mode}/test.yaml (100%) rename gazelle/python/testdata/{relative_imports => relative_imports_project_mode}/BUILD.in (61%) rename gazelle/python/testdata/{relative_imports => relative_imports_project_mode}/BUILD.out (70%) create mode 100644 gazelle/python/testdata/relative_imports_project_mode/README.md create mode 100644 gazelle/python/testdata/relative_imports_project_mode/WORKSPACE rename gazelle/python/testdata/{relative_imports => relative_imports_project_mode}/__main__.py (100%) create mode 100644 gazelle/python/testdata/relative_imports_project_mode/package1/module1.py create mode 100644 gazelle/python/testdata/relative_imports_project_mode/package1/module2.py create mode 100644 gazelle/python/testdata/relative_imports_project_mode/package2/BUILD.in rename gazelle/python/testdata/{relative_imports => relative_imports_project_mode}/package2/BUILD.out (100%) rename gazelle/python/testdata/{relative_imports => relative_imports_project_mode}/package2/__init__.py (100%) rename gazelle/python/testdata/{relative_imports => relative_imports_project_mode}/package2/module3.py (100%) rename gazelle/python/testdata/{relative_imports => relative_imports_project_mode}/package2/module4.py (100%) rename gazelle/python/testdata/{relative_imports => relative_imports_project_mode}/package2/subpackage1/module5.py (100%) create mode 100644 gazelle/python/testdata/relative_imports_project_mode/test.yaml diff --git a/CHANGELOG.md b/CHANGELOG.md index f2fa98f73f..ecdc129502 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -54,6 +54,9 @@ END_UNRELEASED_TEMPLATE {#v0-0-0-changed} ### Changed +* (gazelle) For package mode, resolve dependencies when imports are relative + to the package path. This is enabled via the + `# gazelle:experimental_allow_relative_imports` true directive ({gh-issue}`2203`). * (gazelle) Types for exposed members of `python.ParserOutput` are now all public. {#v0-0-0-fixed} diff --git a/gazelle/README.md b/gazelle/README.md index 89ebaef4cd..58ec55eb11 100644 --- a/gazelle/README.md +++ b/gazelle/README.md @@ -121,12 +121,12 @@ gazelle_python_manifest( requirements = "//:requirements_lock.txt", # include_stub_packages: bool (default: False) # If set to True, this flag automatically includes any corresponding type stub packages - # for the third-party libraries that are present and used. For example, if you have + # for the third-party libraries that are present and used. For example, if you have # `boto3` as a dependency, and this flag is enabled, the corresponding `boto3-stubs` # package will be automatically included in the BUILD file. # - # Enabling this feature helps ensure that type hints and stubs are readily available - # for tools like type checkers and IDEs, improving the development experience and + # Enabling this feature helps ensure that type hints and stubs are readily available + # for tools like type checkers and IDEs, improving the development experience and # reducing manual overhead in managing separate stub packages. include_stub_packages = True ) @@ -220,6 +220,8 @@ Python-specific directives are as follows: | Defines the format of the distribution name in labels to third-party deps. Useful for using Gazelle plugin with other rules with different repository conventions (e.g. `rules_pycross`). Full label is always prepended with (pip) repository name, e.g. `@pip//numpy`. | | `# gazelle:python_label_normalization` | `snake_case` | | Controls how distribution names in labels to third-party deps are normalized. Useful for using Gazelle plugin with other rules with different label conventions (e.g. `rules_pycross` uses PEP-503). Can be "snake_case", "none", or "pep503". | +| `# gazelle:experimental_allow_relative_imports` | `false` | +| Controls whether Gazelle resolves dependencies for import statements that use paths relative to the current package. Can be "true" or "false".| #### Directive: `python_root`: @@ -468,7 +470,7 @@ def py_test(name, main=None, **kwargs): name = "__test__", deps = ["@pip_pytest//:pkg"], # change this to the pytest target in your repo. ) - + deps.append(":__test__") main = ":__test__.py" @@ -581,6 +583,44 @@ deps = [ ] ``` +#### Directive: `experimental_allow_relative_imports` +Enables experimental support for resolving relative imports in +`python_generation_mode package`. + +By default, when `# gazelle:python_generation_mode package` is enabled, +relative imports (e.g., from .library import foo) are not added to the +deps field of the generated target. This results in incomplete py_library +rules that lack required dependencies on sibling packages. + +Example: +Given this Python file import: +```python +from .library import add as _add +from .library import subtract as _subtract +``` + +Expected BUILD file output: +```starlark +py_library( + name = "py_default_library", + srcs = ["__init__.py"], + deps = [ + "//example/library:py_default_library", + ], + visibility = ["//visibility:public"], +) +``` + +Actual output without this annotation: +```starlark +py_library( + name = "py_default_library", + srcs = ["__init__.py"], + visibility = ["//visibility:public"], +) +``` +If the directive is set to `true`, gazelle will resolve imports +that are relative to the current package. ### Libraries diff --git a/gazelle/python/configure.go b/gazelle/python/configure.go index a00b0ba0ba..ae0f7ee1d1 100644 --- a/gazelle/python/configure.go +++ b/gazelle/python/configure.go @@ -68,6 +68,7 @@ func (py *Configurer) KnownDirectives() []string { pythonconfig.TestFilePattern, pythonconfig.LabelConvention, pythonconfig.LabelNormalization, + pythonconfig.ExperimentalAllowRelativeImports, } } @@ -222,6 +223,13 @@ func (py *Configurer) Configure(c *config.Config, rel string, f *rule.File) { default: config.SetLabelNormalization(pythonconfig.DefaultLabelNormalizationType) } + case pythonconfig.ExperimentalAllowRelativeImports: + v, err := strconv.ParseBool(strings.TrimSpace(d.Value)) + if err != nil { + log.Printf("invalid value for gazelle:%s in %q: %q", + pythonconfig.ExperimentalAllowRelativeImports, rel, d.Value) + } + config.SetExperimentalAllowRelativeImports(v) } } diff --git a/gazelle/python/file_parser.go b/gazelle/python/file_parser.go index 3f8363fbdf..cb82cb93b4 100644 --- a/gazelle/python/file_parser.go +++ b/gazelle/python/file_parser.go @@ -165,7 +165,9 @@ func (p *FileParser) parseImportStatements(node *sitter.Node) bool { } } else if node.Type() == sitterNodeTypeImportFromStatement { from := node.Child(1).Content(p.code) - if strings.HasPrefix(from, ".") { + // If the import is from the current package, we don't need to add it to the modules i.e. from . import Class1. + // If the import is from a different relative package i.e. from .package1 import foo, we need to add it to the modules. + if from == "." { return true } for j := 3; j < int(node.ChildCount()); j++ { diff --git a/gazelle/python/resolve.go b/gazelle/python/resolve.go index 996cbbadc0..413e69b289 100644 --- a/gazelle/python/resolve.go +++ b/gazelle/python/resolve.go @@ -148,12 +148,61 @@ func (py *Resolver) Resolve( modules := modulesRaw.(*treeset.Set) it := modules.Iterator() explainDependency := os.Getenv("EXPLAIN_DEPENDENCY") + // Resolve relative paths for package generation + isPackageGeneration := !cfg.PerFileGeneration() && !cfg.CoarseGrainedGeneration() hasFatalError := false MODULES_LOOP: for it.Next() { mod := it.Value().(Module) - moduleParts := strings.Split(mod.Name, ".") - possibleModules := []string{mod.Name} + moduleName := mod.Name + // Transform relative imports `.` or `..foo.bar` into the package path from root. + if strings.HasPrefix(mod.From, ".") { + if !cfg.ExperimentalAllowRelativeImports() || !isPackageGeneration { + continue MODULES_LOOP + } + + // Count number of leading dots in mod.From (e.g., ".." = 2, "...foo.bar" = 3) + relativeDepth := strings.IndexFunc(mod.From, func(r rune) bool { return r != '.' }) + if relativeDepth == -1 { + relativeDepth = len(mod.From) + } + + // Extract final symbol (e.g., "some_function") from mod.Name + imported := mod.Name + if idx := strings.LastIndex(mod.Name, "."); idx >= 0 { + imported = mod.Name[idx+1:] + } + + // Optional subpath in 'from' clause, e.g. "from ...my_library.foo import x" + fromPath := strings.TrimLeft(mod.From, ".") + var fromParts []string + if fromPath != "" { + fromParts = strings.Split(fromPath, ".") + } + + // Current Bazel package as path segments + pkgParts := strings.Split(from.Pkg, "/") + + if relativeDepth-1 > len(pkgParts) { + log.Printf("ERROR: Invalid relative import %q in %q: exceeds package root.", mod.Name, mod.Filepath) + continue MODULES_LOOP + } + + // Go up relativeDepth - 1 levels + baseParts := pkgParts + if relativeDepth > 1 { + baseParts = pkgParts[:len(pkgParts)-(relativeDepth-1)] + } + // Build absolute module path + absParts := append([]string{}, baseParts...) // base path + absParts = append(absParts, fromParts...) // subpath from 'from' + absParts = append(absParts, imported) // actual imported symbol + + moduleName = strings.Join(absParts, ".") + } + + moduleParts := strings.Split(moduleName, ".") + possibleModules := []string{moduleName} for len(moduleParts) > 1 { // Iterate back through the possible imports until // a match is found. diff --git a/gazelle/python/testdata/relative_imports/README.md b/gazelle/python/testdata/relative_imports/README.md deleted file mode 100644 index 1937cbcf4a..0000000000 --- a/gazelle/python/testdata/relative_imports/README.md +++ /dev/null @@ -1,4 +0,0 @@ -# Relative imports - -This test case asserts that the generated targets handle relative imports in -Python correctly. diff --git a/gazelle/python/testdata/relative_imports_package_mode/BUILD.in b/gazelle/python/testdata/relative_imports_package_mode/BUILD.in new file mode 100644 index 0000000000..78ef0a7863 --- /dev/null +++ b/gazelle/python/testdata/relative_imports_package_mode/BUILD.in @@ -0,0 +1,2 @@ +# gazelle:python_generation_mode package +# gazelle:experimental_allow_relative_imports true diff --git a/gazelle/python/testdata/relative_imports_package_mode/BUILD.out b/gazelle/python/testdata/relative_imports_package_mode/BUILD.out new file mode 100644 index 0000000000..f51b516cab --- /dev/null +++ b/gazelle/python/testdata/relative_imports_package_mode/BUILD.out @@ -0,0 +1,15 @@ +load("@rules_python//python:defs.bzl", "py_binary") + +# gazelle:python_generation_mode package +# gazelle:experimental_allow_relative_imports true + +py_binary( + name = "relative_imports_package_mode_bin", + srcs = ["__main__.py"], + main = "__main__.py", + visibility = ["//:__subpackages__"], + deps = [ + "//package1", + "//package2", + ], +) diff --git a/gazelle/python/testdata/relative_imports_package_mode/README.md b/gazelle/python/testdata/relative_imports_package_mode/README.md new file mode 100644 index 0000000000..eb9f8c096c --- /dev/null +++ b/gazelle/python/testdata/relative_imports_package_mode/README.md @@ -0,0 +1,6 @@ +# Resolve deps for relative imports + +This test case verifies that the generated targets correctly handle relative imports in +Python. Specifically, when the Python generation mode is set to "package," it ensures +that relative import statements such as from .foo import X are properly resolved to +their corresponding modules. diff --git a/gazelle/python/testdata/relative_imports/WORKSPACE b/gazelle/python/testdata/relative_imports_package_mode/WORKSPACE similarity index 100% rename from gazelle/python/testdata/relative_imports/WORKSPACE rename to gazelle/python/testdata/relative_imports_package_mode/WORKSPACE diff --git a/gazelle/python/testdata/relative_imports_package_mode/__main__.py b/gazelle/python/testdata/relative_imports_package_mode/__main__.py new file mode 100644 index 0000000000..4fb887a803 --- /dev/null +++ b/gazelle/python/testdata/relative_imports_package_mode/__main__.py @@ -0,0 +1,5 @@ +from package1.module1 import function1 +from package2.module3 import function3 + +print(function1()) +print(function3()) diff --git a/gazelle/python/testdata/relative_imports/package2/BUILD.in b/gazelle/python/testdata/relative_imports_package_mode/package1/BUILD.in similarity index 100% rename from gazelle/python/testdata/relative_imports/package2/BUILD.in rename to gazelle/python/testdata/relative_imports_package_mode/package1/BUILD.in diff --git a/gazelle/python/testdata/relative_imports_package_mode/package1/BUILD.out b/gazelle/python/testdata/relative_imports_package_mode/package1/BUILD.out new file mode 100644 index 0000000000..c562ff07de --- /dev/null +++ b/gazelle/python/testdata/relative_imports_package_mode/package1/BUILD.out @@ -0,0 +1,11 @@ +load("@rules_python//python:defs.bzl", "py_library") + +py_library( + name = "package1", + srcs = [ + "__init__.py", + "module1.py", + "module2.py", + ], + visibility = ["//:__subpackages__"], +) diff --git a/gazelle/python/testdata/relative_imports_package_mode/package1/__init__.py b/gazelle/python/testdata/relative_imports_package_mode/package1/__init__.py new file mode 100644 index 0000000000..11ffb98647 --- /dev/null +++ b/gazelle/python/testdata/relative_imports_package_mode/package1/__init__.py @@ -0,0 +1,2 @@ +def some_function(): + pass diff --git a/gazelle/python/testdata/relative_imports/package1/module1.py b/gazelle/python/testdata/relative_imports_package_mode/package1/module1.py similarity index 100% rename from gazelle/python/testdata/relative_imports/package1/module1.py rename to gazelle/python/testdata/relative_imports_package_mode/package1/module1.py diff --git a/gazelle/python/testdata/relative_imports/package1/module2.py b/gazelle/python/testdata/relative_imports_package_mode/package1/module2.py similarity index 100% rename from gazelle/python/testdata/relative_imports/package1/module2.py rename to gazelle/python/testdata/relative_imports_package_mode/package1/module2.py diff --git a/gazelle/python/testdata/relative_imports_package_mode/package1/my_library/BUILD.in b/gazelle/python/testdata/relative_imports_package_mode/package1/my_library/BUILD.in new file mode 100644 index 0000000000..80a4a22348 --- /dev/null +++ b/gazelle/python/testdata/relative_imports_package_mode/package1/my_library/BUILD.in @@ -0,0 +1,7 @@ +load("@rules_python//python:defs.bzl", "py_library") + +py_library( + name = "my_library", + srcs = ["__init__.py"], + visibility = ["//:__subpackages__"], +) diff --git a/gazelle/python/testdata/relative_imports_package_mode/package1/my_library/BUILD.out b/gazelle/python/testdata/relative_imports_package_mode/package1/my_library/BUILD.out new file mode 100644 index 0000000000..80a4a22348 --- /dev/null +++ b/gazelle/python/testdata/relative_imports_package_mode/package1/my_library/BUILD.out @@ -0,0 +1,7 @@ +load("@rules_python//python:defs.bzl", "py_library") + +py_library( + name = "my_library", + srcs = ["__init__.py"], + visibility = ["//:__subpackages__"], +) diff --git a/gazelle/python/testdata/relative_imports_package_mode/package1/my_library/__init__.py b/gazelle/python/testdata/relative_imports_package_mode/package1/my_library/__init__.py new file mode 100644 index 0000000000..aaa161cd59 --- /dev/null +++ b/gazelle/python/testdata/relative_imports_package_mode/package1/my_library/__init__.py @@ -0,0 +1,2 @@ +def some_function(): + return "some_function" diff --git a/gazelle/python/testdata/relative_imports_package_mode/package1/my_library/foo/BUILD.in b/gazelle/python/testdata/relative_imports_package_mode/package1/my_library/foo/BUILD.in new file mode 100644 index 0000000000..e69de29bb2 diff --git a/gazelle/python/testdata/relative_imports_package_mode/package1/my_library/foo/BUILD.out b/gazelle/python/testdata/relative_imports_package_mode/package1/my_library/foo/BUILD.out new file mode 100644 index 0000000000..58498ee3b3 --- /dev/null +++ b/gazelle/python/testdata/relative_imports_package_mode/package1/my_library/foo/BUILD.out @@ -0,0 +1,7 @@ +load("@rules_python//python:defs.bzl", "py_library") + +py_library( + name = "foo", + srcs = ["__init__.py"], + visibility = ["//:__subpackages__"], +) diff --git a/gazelle/python/testdata/relative_imports_package_mode/package1/my_library/foo/__init__.py b/gazelle/python/testdata/relative_imports_package_mode/package1/my_library/foo/__init__.py new file mode 100644 index 0000000000..aaa161cd59 --- /dev/null +++ b/gazelle/python/testdata/relative_imports_package_mode/package1/my_library/foo/__init__.py @@ -0,0 +1,2 @@ +def some_function(): + return "some_function" diff --git a/gazelle/python/testdata/relative_imports_package_mode/package1/subpackage1/BUILD.in b/gazelle/python/testdata/relative_imports_package_mode/package1/subpackage1/BUILD.in new file mode 100644 index 0000000000..0a5b665c8d --- /dev/null +++ b/gazelle/python/testdata/relative_imports_package_mode/package1/subpackage1/BUILD.in @@ -0,0 +1,10 @@ +load("@rules_python//python:defs.bzl", "py_library") + +py_library( + name = "subpackage1", + srcs = [ + "__init__.py", + "some_module.py", + ], + visibility = ["//:__subpackages__"], +) diff --git a/gazelle/python/testdata/relative_imports_package_mode/package1/subpackage1/BUILD.out b/gazelle/python/testdata/relative_imports_package_mode/package1/subpackage1/BUILD.out new file mode 100644 index 0000000000..0a5b665c8d --- /dev/null +++ b/gazelle/python/testdata/relative_imports_package_mode/package1/subpackage1/BUILD.out @@ -0,0 +1,10 @@ +load("@rules_python//python:defs.bzl", "py_library") + +py_library( + name = "subpackage1", + srcs = [ + "__init__.py", + "some_module.py", + ], + visibility = ["//:__subpackages__"], +) diff --git a/gazelle/python/testdata/relative_imports_package_mode/package1/subpackage1/__init__.py b/gazelle/python/testdata/relative_imports_package_mode/package1/subpackage1/__init__.py new file mode 100644 index 0000000000..02feaeb848 --- /dev/null +++ b/gazelle/python/testdata/relative_imports_package_mode/package1/subpackage1/__init__.py @@ -0,0 +1,3 @@ + +def some_init(): + return "some_init" diff --git a/gazelle/python/testdata/relative_imports_package_mode/package1/subpackage1/some_module.py b/gazelle/python/testdata/relative_imports_package_mode/package1/subpackage1/some_module.py new file mode 100644 index 0000000000..3cae706242 --- /dev/null +++ b/gazelle/python/testdata/relative_imports_package_mode/package1/subpackage1/some_module.py @@ -0,0 +1,3 @@ + +def some_function(): + return "some_function" diff --git a/gazelle/python/testdata/relative_imports_package_mode/package1/subpackage1/subpackage2/BUILD.in b/gazelle/python/testdata/relative_imports_package_mode/package1/subpackage1/subpackage2/BUILD.in new file mode 100644 index 0000000000..e69de29bb2 diff --git a/gazelle/python/testdata/relative_imports_package_mode/package1/subpackage1/subpackage2/BUILD.out b/gazelle/python/testdata/relative_imports_package_mode/package1/subpackage1/subpackage2/BUILD.out new file mode 100644 index 0000000000..8c34081210 --- /dev/null +++ b/gazelle/python/testdata/relative_imports_package_mode/package1/subpackage1/subpackage2/BUILD.out @@ -0,0 +1,16 @@ +load("@rules_python//python:defs.bzl", "py_library") + +py_library( + name = "subpackage2", + srcs = [ + "__init__.py", + "script.py", + ], + visibility = ["//:__subpackages__"], + deps = [ + "//package1/my_library", + "//package1/my_library/foo", + "//package1/subpackage1", + "//package1/subpackage1/subpackage2/library", + ], +) diff --git a/gazelle/python/testdata/relative_imports_package_mode/package1/subpackage1/subpackage2/__init__.py b/gazelle/python/testdata/relative_imports_package_mode/package1/subpackage1/subpackage2/__init__.py new file mode 100644 index 0000000000..e69de29bb2 diff --git a/gazelle/python/testdata/relative_imports_package_mode/package1/subpackage1/subpackage2/library/BUILD.in b/gazelle/python/testdata/relative_imports_package_mode/package1/subpackage1/subpackage2/library/BUILD.in new file mode 100644 index 0000000000..e69de29bb2 diff --git a/gazelle/python/testdata/relative_imports_package_mode/package1/subpackage1/subpackage2/library/BUILD.out b/gazelle/python/testdata/relative_imports_package_mode/package1/subpackage1/subpackage2/library/BUILD.out new file mode 100644 index 0000000000..9fe2e3d1d7 --- /dev/null +++ b/gazelle/python/testdata/relative_imports_package_mode/package1/subpackage1/subpackage2/library/BUILD.out @@ -0,0 +1,7 @@ +load("@rules_python//python:defs.bzl", "py_library") + +py_library( + name = "library", + srcs = ["other_module.py"], + visibility = ["//:__subpackages__"], +) diff --git a/gazelle/python/testdata/relative_imports_package_mode/package1/subpackage1/subpackage2/library/other_module.py b/gazelle/python/testdata/relative_imports_package_mode/package1/subpackage1/subpackage2/library/other_module.py new file mode 100644 index 0000000000..e69de29bb2 diff --git a/gazelle/python/testdata/relative_imports_package_mode/package1/subpackage1/subpackage2/script.py b/gazelle/python/testdata/relative_imports_package_mode/package1/subpackage1/subpackage2/script.py new file mode 100644 index 0000000000..e93f07719a --- /dev/null +++ b/gazelle/python/testdata/relative_imports_package_mode/package1/subpackage1/subpackage2/script.py @@ -0,0 +1,11 @@ +from ...my_library import ( + some_function, +) # Import path should be package1.my_library.some_function +from ...my_library.foo import ( + some_function, +) # Import path should be package1.my_library.foo.some_function +from .library import ( + other_module, +) # Import path should be package1.subpackage1.subpackage2.library.other_module +from .. import some_module # Import path should be package1.subpackage1.some_module +from .. import some_function # Import path should be package1.subpackage1.some_function diff --git a/gazelle/python/testdata/relative_imports_package_mode/package2/BUILD.in b/gazelle/python/testdata/relative_imports_package_mode/package2/BUILD.in new file mode 100644 index 0000000000..e69de29bb2 diff --git a/gazelle/python/testdata/relative_imports_package_mode/package2/BUILD.out b/gazelle/python/testdata/relative_imports_package_mode/package2/BUILD.out new file mode 100644 index 0000000000..bd78108159 --- /dev/null +++ b/gazelle/python/testdata/relative_imports_package_mode/package2/BUILD.out @@ -0,0 +1,12 @@ +load("@rules_python//python:defs.bzl", "py_library") + +py_library( + name = "package2", + srcs = [ + "__init__.py", + "module3.py", + "module4.py", + ], + visibility = ["//:__subpackages__"], + deps = ["//package2/library"], +) diff --git a/gazelle/python/testdata/relative_imports_package_mode/package2/__init__.py b/gazelle/python/testdata/relative_imports_package_mode/package2/__init__.py new file mode 100644 index 0000000000..3d19d80e21 --- /dev/null +++ b/gazelle/python/testdata/relative_imports_package_mode/package2/__init__.py @@ -0,0 +1,20 @@ +from .library import add as _add +from .library import divide as _divide +from .library import multiply as _multiply +from .library import subtract as _subtract + + +def add(a, b): + return _add(a, b) + + +def divide(a, b): + return _divide(a, b) + + +def multiply(a, b): + return _multiply(a, b) + + +def subtract(a, b): + return _subtract(a, b) diff --git a/gazelle/python/testdata/relative_imports_package_mode/package2/library/BUILD.in b/gazelle/python/testdata/relative_imports_package_mode/package2/library/BUILD.in new file mode 100644 index 0000000000..e69de29bb2 diff --git a/gazelle/python/testdata/relative_imports_package_mode/package2/library/BUILD.out b/gazelle/python/testdata/relative_imports_package_mode/package2/library/BUILD.out new file mode 100644 index 0000000000..d704b7fe93 --- /dev/null +++ b/gazelle/python/testdata/relative_imports_package_mode/package2/library/BUILD.out @@ -0,0 +1,7 @@ +load("@rules_python//python:defs.bzl", "py_library") + +py_library( + name = "library", + srcs = ["__init__.py"], + visibility = ["//:__subpackages__"], +) diff --git a/gazelle/python/testdata/relative_imports_package_mode/package2/library/__init__.py b/gazelle/python/testdata/relative_imports_package_mode/package2/library/__init__.py new file mode 100644 index 0000000000..5f8fc62492 --- /dev/null +++ b/gazelle/python/testdata/relative_imports_package_mode/package2/library/__init__.py @@ -0,0 +1,14 @@ +def add(a, b): + return a + b + + +def divide(a, b): + return a / b + + +def multiply(a, b): + return a * b + + +def subtract(a, b): + return a - b diff --git a/gazelle/python/testdata/relative_imports_package_mode/package2/module3.py b/gazelle/python/testdata/relative_imports_package_mode/package2/module3.py new file mode 100644 index 0000000000..6b955cfda6 --- /dev/null +++ b/gazelle/python/testdata/relative_imports_package_mode/package2/module3.py @@ -0,0 +1,5 @@ +from .library import function5 + + +def function3(): + return "function3 " + function5() diff --git a/gazelle/python/testdata/relative_imports_package_mode/package2/module4.py b/gazelle/python/testdata/relative_imports_package_mode/package2/module4.py new file mode 100644 index 0000000000..6e69699985 --- /dev/null +++ b/gazelle/python/testdata/relative_imports_package_mode/package2/module4.py @@ -0,0 +1,2 @@ +def function4(): + return "function4" diff --git a/gazelle/python/testdata/relative_imports/test.yaml b/gazelle/python/testdata/relative_imports_package_mode/test.yaml similarity index 100% rename from gazelle/python/testdata/relative_imports/test.yaml rename to gazelle/python/testdata/relative_imports_package_mode/test.yaml diff --git a/gazelle/python/testdata/relative_imports/BUILD.in b/gazelle/python/testdata/relative_imports_project_mode/BUILD.in similarity index 61% rename from gazelle/python/testdata/relative_imports/BUILD.in rename to gazelle/python/testdata/relative_imports_project_mode/BUILD.in index c04b5e5434..1059942bfb 100644 --- a/gazelle/python/testdata/relative_imports/BUILD.in +++ b/gazelle/python/testdata/relative_imports_project_mode/BUILD.in @@ -1 +1,2 @@ # gazelle:resolve py resolved_package //package2:resolved_package +# gazelle:python_generation_mode project diff --git a/gazelle/python/testdata/relative_imports/BUILD.out b/gazelle/python/testdata/relative_imports_project_mode/BUILD.out similarity index 70% rename from gazelle/python/testdata/relative_imports/BUILD.out rename to gazelle/python/testdata/relative_imports_project_mode/BUILD.out index bf9524480a..acdc914541 100644 --- a/gazelle/python/testdata/relative_imports/BUILD.out +++ b/gazelle/python/testdata/relative_imports_project_mode/BUILD.out @@ -1,9 +1,10 @@ load("@rules_python//python:defs.bzl", "py_binary", "py_library") # gazelle:resolve py resolved_package //package2:resolved_package +# gazelle:python_generation_mode project py_library( - name = "relative_imports", + name = "relative_imports_project_mode", srcs = [ "package1/module1.py", "package1/module2.py", @@ -12,12 +13,12 @@ py_library( ) py_binary( - name = "relative_imports_bin", + name = "relative_imports_project_mode_bin", srcs = ["__main__.py"], main = "__main__.py", visibility = ["//:__subpackages__"], deps = [ - ":relative_imports", + ":relative_imports_project_mode", "//package2", ], ) diff --git a/gazelle/python/testdata/relative_imports_project_mode/README.md b/gazelle/python/testdata/relative_imports_project_mode/README.md new file mode 100644 index 0000000000..3c95a36e62 --- /dev/null +++ b/gazelle/python/testdata/relative_imports_project_mode/README.md @@ -0,0 +1,5 @@ +# Relative imports + +This test case asserts that the generated targets handle relative imports in +Python correctly. This tests that if python generation mode is project, +the relative paths are included in the subdirectories. diff --git a/gazelle/python/testdata/relative_imports_project_mode/WORKSPACE b/gazelle/python/testdata/relative_imports_project_mode/WORKSPACE new file mode 100644 index 0000000000..4959898cdd --- /dev/null +++ b/gazelle/python/testdata/relative_imports_project_mode/WORKSPACE @@ -0,0 +1 @@ +# This is a test data Bazel workspace. diff --git a/gazelle/python/testdata/relative_imports/__main__.py b/gazelle/python/testdata/relative_imports_project_mode/__main__.py similarity index 100% rename from gazelle/python/testdata/relative_imports/__main__.py rename to gazelle/python/testdata/relative_imports_project_mode/__main__.py diff --git a/gazelle/python/testdata/relative_imports_project_mode/package1/module1.py b/gazelle/python/testdata/relative_imports_project_mode/package1/module1.py new file mode 100644 index 0000000000..28502f1f84 --- /dev/null +++ b/gazelle/python/testdata/relative_imports_project_mode/package1/module1.py @@ -0,0 +1,19 @@ +# Copyright 2023 The Bazel Authors. All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +from .module2 import function2 + + +def function1(): + return "function1 " + function2() diff --git a/gazelle/python/testdata/relative_imports_project_mode/package1/module2.py b/gazelle/python/testdata/relative_imports_project_mode/package1/module2.py new file mode 100644 index 0000000000..0cbc5f0be0 --- /dev/null +++ b/gazelle/python/testdata/relative_imports_project_mode/package1/module2.py @@ -0,0 +1,17 @@ +# Copyright 2023 The Bazel Authors. All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + + +def function2(): + return "function2" diff --git a/gazelle/python/testdata/relative_imports_project_mode/package2/BUILD.in b/gazelle/python/testdata/relative_imports_project_mode/package2/BUILD.in new file mode 100644 index 0000000000..e69de29bb2 diff --git a/gazelle/python/testdata/relative_imports/package2/BUILD.out b/gazelle/python/testdata/relative_imports_project_mode/package2/BUILD.out similarity index 100% rename from gazelle/python/testdata/relative_imports/package2/BUILD.out rename to gazelle/python/testdata/relative_imports_project_mode/package2/BUILD.out diff --git a/gazelle/python/testdata/relative_imports/package2/__init__.py b/gazelle/python/testdata/relative_imports_project_mode/package2/__init__.py similarity index 100% rename from gazelle/python/testdata/relative_imports/package2/__init__.py rename to gazelle/python/testdata/relative_imports_project_mode/package2/__init__.py diff --git a/gazelle/python/testdata/relative_imports/package2/module3.py b/gazelle/python/testdata/relative_imports_project_mode/package2/module3.py similarity index 100% rename from gazelle/python/testdata/relative_imports/package2/module3.py rename to gazelle/python/testdata/relative_imports_project_mode/package2/module3.py diff --git a/gazelle/python/testdata/relative_imports/package2/module4.py b/gazelle/python/testdata/relative_imports_project_mode/package2/module4.py similarity index 100% rename from gazelle/python/testdata/relative_imports/package2/module4.py rename to gazelle/python/testdata/relative_imports_project_mode/package2/module4.py diff --git a/gazelle/python/testdata/relative_imports/package2/subpackage1/module5.py b/gazelle/python/testdata/relative_imports_project_mode/package2/subpackage1/module5.py similarity index 100% rename from gazelle/python/testdata/relative_imports/package2/subpackage1/module5.py rename to gazelle/python/testdata/relative_imports_project_mode/package2/subpackage1/module5.py diff --git a/gazelle/python/testdata/relative_imports_project_mode/test.yaml b/gazelle/python/testdata/relative_imports_project_mode/test.yaml new file mode 100644 index 0000000000..fcea77710f --- /dev/null +++ b/gazelle/python/testdata/relative_imports_project_mode/test.yaml @@ -0,0 +1,15 @@ +# Copyright 2023 The Bazel Authors. All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +--- diff --git a/gazelle/pythonconfig/pythonconfig.go b/gazelle/pythonconfig/pythonconfig.go index 866339d449..e0a2b8a469 100644 --- a/gazelle/pythonconfig/pythonconfig.go +++ b/gazelle/pythonconfig/pythonconfig.go @@ -91,6 +91,9 @@ const ( // names of labels to third-party dependencies are normalized. Supported values // are 'none', 'pep503' and 'snake_case' (default). See LabelNormalizationType. LabelNormalization = "python_label_normalization" + // ExperimentalAllowRelativeImports represents the directive that controls + // whether relative imports are allowed. + ExperimentalAllowRelativeImports = "experimental_allow_relative_imports" ) // GenerationModeType represents one of the generation modes for the Python @@ -177,6 +180,7 @@ type Config struct { testFilePattern []string labelConvention string labelNormalization LabelNormalizationType + experimentalAllowRelativeImports bool } type LabelNormalizationType int @@ -212,6 +216,7 @@ func New( testFilePattern: strings.Split(DefaultTestFilePatternString, ","), labelConvention: DefaultLabelConvention, labelNormalization: DefaultLabelNormalizationType, + experimentalAllowRelativeImports: false, } } @@ -244,6 +249,7 @@ func (c *Config) NewChild() *Config { testFilePattern: c.testFilePattern, labelConvention: c.labelConvention, labelNormalization: c.labelNormalization, + experimentalAllowRelativeImports: c.experimentalAllowRelativeImports, } } @@ -520,6 +526,16 @@ func (c *Config) LabelNormalization() LabelNormalizationType { return c.labelNormalization } +// SetExperimentalAllowRelativeImports sets whether relative imports are allowed. +func (c *Config) SetExperimentalAllowRelativeImports(allowRelativeImports bool) { + c.experimentalAllowRelativeImports = allowRelativeImports +} + +// ExperimentalAllowRelativeImports returns whether relative imports are allowed. +func (c *Config) ExperimentalAllowRelativeImports() bool { + return c.experimentalAllowRelativeImports +} + // FormatThirdPartyDependency returns a label to a third-party dependency performing all formating and normalization. func (c *Config) FormatThirdPartyDependency(repositoryName string, distributionName string) label.Label { conventionalDistributionName := strings.ReplaceAll(c.labelConvention, distributionNameLabelConventionSubstitution, distributionName) From f6feca1e00d9ae768243f05e677b7b636b9ad7ba Mon Sep 17 00:00:00 2001 From: armandomontanez Date: Sat, 21 Jun 2025 19:18:35 -0700 Subject: [PATCH 295/922] fix: Fix bazel vendor support for requirements with environment markers (#2997) Fixes `bazel vendor` support for requirements files that contain environment markers. During a vendored `bazel build`, when evaluate_markers_py() is run it needs PYTHONHOME set to properly find the home of the vendored libraries. Resolves #2996 --------- Co-authored-by: Ignas Anikevicius <240938+aignas@users.noreply.github.com> --- .bazelci/presubmit.yml | 9 +++++++++ CHANGELOG.md | 4 +++- examples/bzlmod/.bazelignore | 1 + examples/bzlmod/.gitignore | 1 + python/private/pypi/evaluate_markers.bzl | 13 ++++++++----- 5 files changed, 22 insertions(+), 6 deletions(-) diff --git a/.bazelci/presubmit.yml b/.bazelci/presubmit.yml index 01af217924..07ffa4eaac 100644 --- a/.bazelci/presubmit.yml +++ b/.bazelci/presubmit.yml @@ -272,6 +272,15 @@ tasks: working_directory: examples/bzlmod platform: debian11 bazel: 7.x + integration_test_bzlmod_ubuntu_vendor: + <<: *reusable_build_test_all + name: "examples/bzlmod: bazel vendor" + working_directory: examples/bzlmod + platform: ubuntu2004 + shell_commands: + - "bazel vendor --vendor_dir=./vendor //..." + - "bazel build --vendor_dir=./vendor //..." + - "rm -rf ./vendor" integration_test_bzlmod_macos: <<: *reusable_build_test_all <<: *coverage_targets_example_bzlmod diff --git a/CHANGELOG.md b/CHANGELOG.md index ecdc129502..4facff4917 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -61,7 +61,9 @@ END_UNRELEASED_TEMPLATE {#v0-0-0-fixed} ### Fixed -* Nothing fixed. +* (pypi) Fixes an issue where builds using a `bazel vendor` vendor directory + would fail if the constraints file contained environment markers. Fixes + [#2996](https://github.com/bazel-contrib/rules_python/issues/2996). {#v0-0-0-added} ### Added diff --git a/examples/bzlmod/.bazelignore b/examples/bzlmod/.bazelignore index 3927f8e910..536ded93a6 100644 --- a/examples/bzlmod/.bazelignore +++ b/examples/bzlmod/.bazelignore @@ -1,2 +1,3 @@ other_module py_proto_library/foo_external +vendor diff --git a/examples/bzlmod/.gitignore b/examples/bzlmod/.gitignore index ac51a054d2..0f6c6316dd 100644 --- a/examples/bzlmod/.gitignore +++ b/examples/bzlmod/.gitignore @@ -1 +1,2 @@ bazel-* +vendor/ diff --git a/python/private/pypi/evaluate_markers.bzl b/python/private/pypi/evaluate_markers.bzl index 58a29a9181..2b805c33e6 100644 --- a/python/private/pypi/evaluate_markers.bzl +++ b/python/private/pypi/evaluate_markers.bzl @@ -78,14 +78,16 @@ def evaluate_markers_py(mrctx, *, requirements, python_interpreter, python_inter out_file = mrctx.path("requirements_with_markers.out.json") mrctx.file(in_file, json.encode(requirements)) + interpreter = pypi_repo_utils.resolve_python_interpreter( + mrctx, + python_interpreter = python_interpreter, + python_interpreter_target = python_interpreter_target, + ) + pypi_repo_utils.execute_checked( mrctx, op = "ResolveRequirementEnvMarkers({})".format(in_file), - python = pypi_repo_utils.resolve_python_interpreter( - mrctx, - python_interpreter = python_interpreter, - python_interpreter_target = python_interpreter_target, - ), + python = interpreter, arguments = [ "-m", "python.private.pypi.requirements_parser.resolve_target_platforms", @@ -94,6 +96,7 @@ def evaluate_markers_py(mrctx, *, requirements, python_interpreter, python_inter ], srcs = srcs, environment = { + "PYTHONHOME": str(interpreter.dirname), "PYTHONPATH": [ Label("@pypi__packaging//:BUILD.bazel"), Label("//:BUILD.bazel"), From 49780276797ecdb22afeda0b8c72a680a3c0b41a Mon Sep 17 00:00:00 2001 From: Ignas Anikevicius <240938+aignas@users.noreply.github.com> Date: Wed, 25 Jun 2025 12:58:07 +0900 Subject: [PATCH 296/922] fix(pypi): namespace_pkgs should pass correct arguments (#3026) It seems that the only function that did not have unit tests have bugs and the integration tests did not catch it because we weren't creating namespacepkg `__init__.py` files. This change fixes the bug, adds a unit test for the remaining untested function. Fixes #3023 Co-authored-by: Richard Levasseur --- python/private/pypi/namespace_pkgs.bzl | 21 ++++++---- .../namespace_pkgs/namespace_pkgs_tests.bzl | 41 ++++++++++++++++++- 2 files changed, 54 insertions(+), 8 deletions(-) diff --git a/python/private/pypi/namespace_pkgs.bzl b/python/private/pypi/namespace_pkgs.bzl index bf4689a5ea..be6244efc7 100644 --- a/python/private/pypi/namespace_pkgs.bzl +++ b/python/private/pypi/namespace_pkgs.bzl @@ -59,25 +59,32 @@ def get_files(*, srcs, ignored_dirnames = [], root = None): return sorted([d for d in dirs if d not in ignored]) -def create_inits(**kwargs): +def create_inits(*, srcs, ignored_dirnames = [], root = None, copy_file = copy_file, **kwargs): """Create init files and return the list to be included `py_library` srcs. Args: - **kwargs: passed to {obj}`get_files`. + srcs: {type}`src` a list of files to be passed to {bzl:obj}`py_library` + as `srcs` and `data`. This is usually a result of a {obj}`glob`. + ignored_dirnames: {type}`str` a list of patterns to ignore. + root: {type}`str` the prefix to use as the root. + copy_file: the `copy_file` rule to copy files in build context. + **kwargs: passed to {obj}`copy_file`. Returns: {type}`list[str]` to be included as part of `py_library`. """ - srcs = [] - for out in get_files(**kwargs): + ret = [] + for i, out in enumerate(get_files(srcs = srcs, ignored_dirnames = ignored_dirnames, root = root)): src = "{}/__init__.py".format(out) - srcs.append(srcs) + ret.append(src) copy_file( - name = "_cp_{}_namespace".format(out), + # For the target name, use a number instead of trying to convert an output + # path into a valid label. + name = "_cp_{}_namespace".format(i), src = _TEMPLATE, out = src, **kwargs ) - return srcs + return ret diff --git a/tests/pypi/namespace_pkgs/namespace_pkgs_tests.bzl b/tests/pypi/namespace_pkgs/namespace_pkgs_tests.bzl index 7ac938ff17..9c382d070c 100644 --- a/tests/pypi/namespace_pkgs/namespace_pkgs_tests.bzl +++ b/tests/pypi/namespace_pkgs/namespace_pkgs_tests.bzl @@ -1,7 +1,7 @@ "" load("@rules_testing//lib:analysis_test.bzl", "test_suite") -load("//python/private/pypi:namespace_pkgs.bzl", "get_files") # buildifier: disable=bzl-visibility +load("//python/private/pypi:namespace_pkgs.bzl", "create_inits", "get_files") # buildifier: disable=bzl-visibility _tests = [] @@ -160,6 +160,45 @@ def test_skips_ignored_directories(env): _tests.append(test_skips_ignored_directories) +def _test_create_inits(env): + srcs = [ + "nested/root/foo/bar/biz.py", + "nested/root/foo/bee/boo.py", + "nested/root/foo/buu/__init__.py", + "nested/root/foo/buu/bii.py", + ] + copy_file_calls = [] + template = Label("//python/private/pypi:namespace_pkg_tmpl.py") + + got = create_inits( + srcs = srcs, + root = "nested/root", + copy_file = lambda **kwargs: copy_file_calls.append(kwargs), + ) + env.expect.that_collection(got).contains_exactly([ + call["out"] + for call in copy_file_calls + ]) + env.expect.that_collection(copy_file_calls).contains_exactly([ + { + "name": "_cp_0_namespace", + "out": "nested/root/foo/__init__.py", + "src": template, + }, + { + "name": "_cp_1_namespace", + "out": "nested/root/foo/bar/__init__.py", + "src": template, + }, + { + "name": "_cp_2_namespace", + "out": "nested/root/foo/bee/__init__.py", + "src": template, + }, + ]) + +_tests.append(_test_create_inits) + def namespace_pkgs_test_suite(name): test_suite( name = name, From aab2650a5687984668674a3d48f9bf17efba0a10 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Wed, 25 Jun 2025 18:16:10 -0700 Subject: [PATCH 297/922] fix: work around version parsing by only parsing if site-packages is enabled (#3031) There's a bug in the version string parser that doesn't handle local identifiers correctly. Thankfully, it's only activated in the experimental code path when site packages for libraries is eanbled. Moving the logic within that block works around it. Work around for https://github.com/bazel-contrib/rules_python/issues/3030 --- python/private/py_library.bzl | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/python/private/py_library.bzl b/python/private/py_library.bzl index 24adb5f3ca..ea2e608401 100644 --- a/python/private/py_library.bzl +++ b/python/private/py_library.bzl @@ -161,8 +161,7 @@ def py_library_impl(ctx, *, semantics): imports = [] venv_symlinks = [] - package, version_str = _get_package_and_version(ctx) - imports, venv_symlinks = _get_imports_and_venv_symlinks(ctx, semantics, package, version_str) + imports, venv_symlinks = _get_imports_and_venv_symlinks(ctx, semantics) cc_info = semantics.get_cc_info_for_library(ctx) py_info, deps_transitive_sources, builtins_py_info = create_py_info( @@ -241,10 +240,11 @@ def _get_package_and_version(ctx): version.normalize(version_str), # will have no dashes either ) -def _get_imports_and_venv_symlinks(ctx, semantics, package, version_str): +def _get_imports_and_venv_symlinks(ctx, semantics): imports = depset() venv_symlinks = [] if VenvsSitePackages.is_enabled(ctx): + package, version_str = _get_package_and_version(ctx) venv_symlinks = _get_venv_symlinks(ctx, package, version_str) else: imports = collect_imports(ctx, semantics) From 4ec1e805133019e3a00bb935beb6115146b5825c Mon Sep 17 00:00:00 2001 From: Douglas Thor Date: Thu, 26 Jun 2025 09:31:51 -0700 Subject: [PATCH 298/922] docs,tests: Clarify how py_wheel.strip_path_prefixes works; add test case (#3027) Include a minor change to `arcname_from` to support cases where the distribution_prefix is the empty string. Fixes #3017 --------- Co-authored-by: Richard Levasseur --- python/private/py_wheel.bzl | 10 +++++++- tests/tools/BUILD.bazel | 23 +++++++++++++++++ tests/tools/wheelmaker_test.py | 38 +++++++++++++++++++++++++++ tools/wheelmaker.py | 47 ++++++++++++++++++++++++---------- 4 files changed, 103 insertions(+), 15 deletions(-) create mode 100644 tests/tools/BUILD.bazel create mode 100644 tests/tools/wheelmaker_test.py diff --git a/python/private/py_wheel.bzl b/python/private/py_wheel.bzl index cfd4efdcda..e6352efcea 100644 --- a/python/private/py_wheel.bzl +++ b/python/private/py_wheel.bzl @@ -217,7 +217,15 @@ _other_attrs = { ), "strip_path_prefixes": attr.string_list( default = [], - doc = "path prefixes to strip from files added to the generated package", + doc = """\ +Path prefixes to strip from files added to the generated package. +Prefixes are checked **in order** and only the **first match** will be used. + +For example: ++ `["foo", "foo/bar/baz"]` will strip `"foo/bar/baz/file.py"` to `"bar/baz/file.py"` ++ `["foo/bar/baz", "foo"]` will strip `"foo/bar/baz/file.py"` to `"file.py"` and + `"foo/file2.py"` to `"file2.py"` +""", ), "summary": attr.string( doc = "A one-line summary of what the distribution does", diff --git a/tests/tools/BUILD.bazel b/tests/tools/BUILD.bazel new file mode 100644 index 0000000000..4d163f19f1 --- /dev/null +++ b/tests/tools/BUILD.bazel @@ -0,0 +1,23 @@ +# Copyright 2025 The Bazel Authors. All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +load("//python:py_test.bzl", "py_test") + +licenses(["notice"]) + +py_test( + name = "wheelmaker_test", + size = "small", + srcs = ["wheelmaker_test.py"], + deps = ["//tools:wheelmaker"], +) diff --git a/tests/tools/wheelmaker_test.py b/tests/tools/wheelmaker_test.py new file mode 100644 index 0000000000..0efe1c9fbc --- /dev/null +++ b/tests/tools/wheelmaker_test.py @@ -0,0 +1,38 @@ +import unittest + +import tools.wheelmaker as wheelmaker + + +class ArcNameFromTest(unittest.TestCase): + def test_arcname_from(self) -> None: + # (name, distribution_prefix, strip_path_prefixes, want) tuples + checks = [ + ("a/b/c/file.py", "", [], "a/b/c/file.py"), + ("a/b/c/file.py", "", ["a"], "/b/c/file.py"), + ("a/b/c/file.py", "", ["a/b/"], "c/file.py"), + # only first found is used and it's not cumulative. + ("a/b/c/file.py", "", ["a/", "b/"], "b/c/file.py"), + # Examples from docs + ("foo/bar/baz/file.py", "", ["foo", "foo/bar/baz"], "/bar/baz/file.py"), + ("foo/bar/baz/file.py", "", ["foo/bar/baz", "foo"], "/file.py"), + ("foo/file2.py", "", ["foo/bar/baz", "foo"], "/file2.py"), + # Files under the distribution prefix (eg mylib-1.0.0-dist-info) + # are unmodified + ("mylib-0.0.1-dist-info/WHEEL", "mylib", [], "mylib-0.0.1-dist-info/WHEEL"), + ("mylib/a/b/c/WHEEL", "mylib", ["mylib"], "mylib/a/b/c/WHEEL"), + ] + for name, prefix, strip, want in checks: + with self.subTest( + name=name, + distribution_prefix=prefix, + strip_path_prefixes=strip, + want=want, + ): + got = wheelmaker.arcname_from( + name=name, distribution_prefix=prefix, strip_path_prefixes=strip + ) + self.assertEqual(got, want) + + +if __name__ == "__main__": + unittest.main() diff --git a/tools/wheelmaker.py b/tools/wheelmaker.py index 8b775e1541..3401c749ed 100644 --- a/tools/wheelmaker.py +++ b/tools/wheelmaker.py @@ -24,6 +24,7 @@ import stat import sys import zipfile +from collections.abc import Iterable from pathlib import Path _ZIP_EPOCH = (1980, 1, 1, 0, 0, 0) @@ -98,6 +99,30 @@ def normalize_pep440(version): return str(packaging.version.Version(f"0+{sanitized}")) +def arcname_from( + name: str, distribution_prefix: str, strip_path_prefixes: Sequence[str] = () +) -> str: + """Return the within-archive name for a given file path name. + + Prefixes to strip are checked in order and only the first match will be used. + + Args: + name: The file path eg 'mylib/a/b/c/file.py' + distribution_prefix: The + strip_path_prefixes: Remove these prefixes from names. + """ + # Always use unix path separators. + normalized_arcname = name.replace(os.path.sep, "/") + # Don't manipulate names filenames in the .distinfo or .data directories. + if distribution_prefix and normalized_arcname.startswith(distribution_prefix): + return normalized_arcname + for prefix in strip_path_prefixes: + if normalized_arcname.startswith(prefix): + return normalized_arcname[len(prefix) :] + + return normalized_arcname + + class _WhlFile(zipfile.ZipFile): def __init__( self, @@ -126,18 +151,6 @@ def data_path(self, basename): def add_file(self, package_filename, real_filename): """Add given file to the distribution.""" - def arcname_from(name): - # Always use unix path separators. - normalized_arcname = name.replace(os.path.sep, "/") - # Don't manipulate names filenames in the .distinfo or .data directories. - if normalized_arcname.startswith(self._distribution_prefix): - return normalized_arcname - for prefix in self._strip_path_prefixes: - if normalized_arcname.startswith(prefix): - return normalized_arcname[len(prefix) :] - - return normalized_arcname - if os.path.isdir(real_filename): directory_contents = os.listdir(real_filename) for file_ in directory_contents: @@ -147,7 +160,11 @@ def arcname_from(name): ) return - arcname = arcname_from(package_filename) + arcname = arcname_from( + package_filename, + distribution_prefix=self._distribution_prefix, + strip_path_prefixes=self._strip_path_prefixes, + ) zinfo = self._zipinfo(arcname) # Write file to the zip archive while computing the hash and length @@ -569,7 +586,9 @@ def get_new_requirement_line(reqs_text, extra): else: return f"Requires-Dist: {req.name}{req_extra_deps}{req.specifier}; {req.marker}" else: - return f"Requires-Dist: {req.name}{req_extra_deps}{req.specifier}; {extra}".strip(" ;") + return f"Requires-Dist: {req.name}{req_extra_deps}{req.specifier}; {extra}".strip( + " ;" + ) for meta_line in metadata.splitlines(): if not meta_line.startswith("Requires-Dist: "): From e5ef69bdc9dfca67ddf04150bd69f2010715c48b Mon Sep 17 00:00:00 2001 From: Alex Martani Date: Thu, 26 Jun 2025 14:51:16 -0700 Subject: [PATCH 299/922] feat(gazelle): Add type-checking only dependencies to pyi_deps (#3014) https://github.com/bazel-contrib/rules_python/pull/2538 added the attribute `pyi_deps` to python rules, intended to be used for dependencies that are only used for type-checking purposes. This PR adds a new directive, `gazelle:python_generate_pyi_deps`, which, when enabled: - When a dependency is added only to satisfy type-checking only imports (in a `if TYPE_CHECKING:` block), the dependency is added to `pyi_deps` instead of `deps`; - Third-party stub packages (eg. `boto3-stubs`) are now added to `pyi_deps` instead of `deps`. --------- Co-authored-by: Douglas Thor --- CHANGELOG.md | 3 + gazelle/README.md | 2 + gazelle/python/configure.go | 7 +++ gazelle/python/file_parser.go | 45 ++++++++++++++- gazelle/python/file_parser_test.go | 37 ++++++++++++ gazelle/python/parser.go | 15 ++++- gazelle/python/resolve.go | 56 ++++++++++++++++--- gazelle/python/target.go | 6 +- .../testdata/add_type_stub_packages/BUILD.in | 1 + .../testdata/add_type_stub_packages/BUILD.out | 8 ++- .../testdata/add_type_stub_packages/README.md | 4 +- .../testdata/type_checking_imports/BUILD.in | 2 + .../testdata/type_checking_imports/BUILD.out | 33 +++++++++++ .../testdata/type_checking_imports/README.md | 5 ++ .../testdata/type_checking_imports/WORKSPACE | 1 + .../testdata/type_checking_imports/bar.py | 9 +++ .../testdata/type_checking_imports/baz.py | 23 ++++++++ .../testdata/type_checking_imports/foo.py | 21 +++++++ .../type_checking_imports/gazelle_python.yaml | 20 +++++++ .../testdata/type_checking_imports/test.yaml | 15 +++++ .../type_checking_imports_disabled/BUILD.in | 2 + .../type_checking_imports_disabled/BUILD.out | 35 ++++++++++++ .../type_checking_imports_disabled/README.md | 3 + .../type_checking_imports_disabled/WORKSPACE | 1 + .../type_checking_imports_disabled/bar.py | 9 +++ .../type_checking_imports_disabled/baz.py | 23 ++++++++ .../type_checking_imports_disabled/foo.py | 21 +++++++ .../gazelle_python.yaml | 20 +++++++ .../type_checking_imports_disabled/test.yaml | 15 +++++ .../type_checking_imports_package/BUILD.in | 2 + .../type_checking_imports_package/BUILD.out | 19 +++++++ .../type_checking_imports_package/README.md | 3 + .../type_checking_imports_package/WORKSPACE | 1 + .../type_checking_imports_package/bar.py | 9 +++ .../type_checking_imports_package/baz.py | 23 ++++++++ .../type_checking_imports_package/foo.py | 21 +++++++ .../gazelle_python.yaml | 20 +++++++ .../type_checking_imports_package/test.yaml | 15 +++++ .../type_checking_imports_project/BUILD.in | 2 + .../type_checking_imports_project/BUILD.out | 19 +++++++ .../type_checking_imports_project/README.md | 3 + .../type_checking_imports_project/WORKSPACE | 1 + .../type_checking_imports_project/bar.py | 9 +++ .../type_checking_imports_project/baz.py | 23 ++++++++ .../type_checking_imports_project/foo.py | 21 +++++++ .../gazelle_python.yaml | 20 +++++++ .../type_checking_imports_project/test.yaml | 15 +++++ gazelle/pythonconfig/pythonconfig.go | 19 +++++++ 48 files changed, 667 insertions(+), 20 deletions(-) create mode 100644 gazelle/python/testdata/type_checking_imports/BUILD.in create mode 100644 gazelle/python/testdata/type_checking_imports/BUILD.out create mode 100644 gazelle/python/testdata/type_checking_imports/README.md create mode 100644 gazelle/python/testdata/type_checking_imports/WORKSPACE create mode 100644 gazelle/python/testdata/type_checking_imports/bar.py create mode 100644 gazelle/python/testdata/type_checking_imports/baz.py create mode 100644 gazelle/python/testdata/type_checking_imports/foo.py create mode 100644 gazelle/python/testdata/type_checking_imports/gazelle_python.yaml create mode 100644 gazelle/python/testdata/type_checking_imports/test.yaml create mode 100644 gazelle/python/testdata/type_checking_imports_disabled/BUILD.in create mode 100644 gazelle/python/testdata/type_checking_imports_disabled/BUILD.out create mode 100644 gazelle/python/testdata/type_checking_imports_disabled/README.md create mode 100644 gazelle/python/testdata/type_checking_imports_disabled/WORKSPACE create mode 100644 gazelle/python/testdata/type_checking_imports_disabled/bar.py create mode 100644 gazelle/python/testdata/type_checking_imports_disabled/baz.py create mode 100644 gazelle/python/testdata/type_checking_imports_disabled/foo.py create mode 100644 gazelle/python/testdata/type_checking_imports_disabled/gazelle_python.yaml create mode 100644 gazelle/python/testdata/type_checking_imports_disabled/test.yaml create mode 100644 gazelle/python/testdata/type_checking_imports_package/BUILD.in create mode 100644 gazelle/python/testdata/type_checking_imports_package/BUILD.out create mode 100644 gazelle/python/testdata/type_checking_imports_package/README.md create mode 100644 gazelle/python/testdata/type_checking_imports_package/WORKSPACE create mode 100644 gazelle/python/testdata/type_checking_imports_package/bar.py create mode 100644 gazelle/python/testdata/type_checking_imports_package/baz.py create mode 100644 gazelle/python/testdata/type_checking_imports_package/foo.py create mode 100644 gazelle/python/testdata/type_checking_imports_package/gazelle_python.yaml create mode 100644 gazelle/python/testdata/type_checking_imports_package/test.yaml create mode 100644 gazelle/python/testdata/type_checking_imports_project/BUILD.in create mode 100644 gazelle/python/testdata/type_checking_imports_project/BUILD.out create mode 100644 gazelle/python/testdata/type_checking_imports_project/README.md create mode 100644 gazelle/python/testdata/type_checking_imports_project/WORKSPACE create mode 100644 gazelle/python/testdata/type_checking_imports_project/bar.py create mode 100644 gazelle/python/testdata/type_checking_imports_project/baz.py create mode 100644 gazelle/python/testdata/type_checking_imports_project/foo.py create mode 100644 gazelle/python/testdata/type_checking_imports_project/gazelle_python.yaml create mode 100644 gazelle/python/testdata/type_checking_imports_project/test.yaml diff --git a/CHANGELOG.md b/CHANGELOG.md index 4facff4917..78a3d1caf5 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -70,6 +70,9 @@ END_UNRELEASED_TEMPLATE * (pypi) To configure the environment for `requirements.txt` evaluation, use the newly added developer preview of the `pip.default` tag class. Only `rules_python` and root modules can use this feature. You can also configure custom `config_settings` using `pip.default`. +* (gazelle) New directive `gazelle:python_generate_pyi_deps`; when `true`, + dependencies added to satisfy type-only imports (`if TYPE_CHECKING`) and type + stub packages are added to `pyi_deps` instead of `deps`. {#v0-0-0-removed} ### Removed diff --git a/gazelle/README.md b/gazelle/README.md index 58ec55eb11..5c63e21762 100644 --- a/gazelle/README.md +++ b/gazelle/README.md @@ -222,6 +222,8 @@ Python-specific directives are as follows: | Controls how distribution names in labels to third-party deps are normalized. Useful for using Gazelle plugin with other rules with different label conventions (e.g. `rules_pycross` uses PEP-503). Can be "snake_case", "none", or "pep503". | | `# gazelle:experimental_allow_relative_imports` | `false` | | Controls whether Gazelle resolves dependencies for import statements that use paths relative to the current package. Can be "true" or "false".| +| `# gazelle:python_generate_pyi_deps` | `false` | +| Controls whether to generate a separate `pyi_deps` attribute for type-checking dependencies or merge them into the regular `deps` attribute. When `false` (default), type-checking dependencies are merged into `deps` for backward compatibility. When `true`, generates separate `pyi_deps`. Imports in blocks with the format `if typing.TYPE_CHECKING:`/`if TYPE_CHECKING:` and type-only stub packages (eg. boto3-stubs) are recognized as type-checking dependencies. | #### Directive: `python_root`: diff --git a/gazelle/python/configure.go b/gazelle/python/configure.go index ae0f7ee1d1..db80fc1a22 100644 --- a/gazelle/python/configure.go +++ b/gazelle/python/configure.go @@ -68,6 +68,7 @@ func (py *Configurer) KnownDirectives() []string { pythonconfig.TestFilePattern, pythonconfig.LabelConvention, pythonconfig.LabelNormalization, + pythonconfig.GeneratePyiDeps, pythonconfig.ExperimentalAllowRelativeImports, } } @@ -230,6 +231,12 @@ func (py *Configurer) Configure(c *config.Config, rel string, f *rule.File) { pythonconfig.ExperimentalAllowRelativeImports, rel, d.Value) } config.SetExperimentalAllowRelativeImports(v) + case pythonconfig.GeneratePyiDeps: + v, err := strconv.ParseBool(strings.TrimSpace(d.Value)) + if err != nil { + log.Fatal(err) + } + config.SetGeneratePyiDeps(v) } } diff --git a/gazelle/python/file_parser.go b/gazelle/python/file_parser.go index cb82cb93b4..aca925cbe7 100644 --- a/gazelle/python/file_parser.go +++ b/gazelle/python/file_parser.go @@ -47,9 +47,10 @@ type ParserOutput struct { } type FileParser struct { - code []byte - relFilepath string - output ParserOutput + code []byte + relFilepath string + output ParserOutput + inTypeCheckingBlock bool } func NewFileParser() *FileParser { @@ -158,6 +159,7 @@ func (p *FileParser) parseImportStatements(node *sitter.Node) bool { continue } m.Filepath = p.relFilepath + m.TypeCheckingOnly = p.inTypeCheckingBlock if strings.HasPrefix(m.Name, ".") { continue } @@ -178,6 +180,7 @@ func (p *FileParser) parseImportStatements(node *sitter.Node) bool { m.Filepath = p.relFilepath m.From = from m.Name = fmt.Sprintf("%s.%s", from, m.Name) + m.TypeCheckingOnly = p.inTypeCheckingBlock p.output.Modules = append(p.output.Modules, m) } } else { @@ -202,10 +205,43 @@ func (p *FileParser) SetCodeAndFile(code []byte, relPackagePath, filename string p.output.FileName = filename } +// isTypeCheckingBlock returns true if the given node is an `if TYPE_CHECKING:` block. +func (p *FileParser) isTypeCheckingBlock(node *sitter.Node) bool { + if node.Type() != sitterNodeTypeIfStatement || node.ChildCount() < 2 { + return false + } + + condition := node.Child(1) + + // Handle `if TYPE_CHECKING:` + if condition.Type() == sitterNodeTypeIdentifier && condition.Content(p.code) == "TYPE_CHECKING" { + return true + } + + // Handle `if typing.TYPE_CHECKING:` + if condition.Type() == "attribute" && condition.ChildCount() >= 3 { + object := condition.Child(0) + attr := condition.Child(2) + if object.Type() == sitterNodeTypeIdentifier && object.Content(p.code) == "typing" && + attr.Type() == sitterNodeTypeIdentifier && attr.Content(p.code) == "TYPE_CHECKING" { + return true + } + } + + return false +} + func (p *FileParser) parse(ctx context.Context, node *sitter.Node) { if node == nil { return } + + // Check if this is a TYPE_CHECKING block + wasInTypeCheckingBlock := p.inTypeCheckingBlock + if p.isTypeCheckingBlock(node) { + p.inTypeCheckingBlock = true + } + for i := 0; i < int(node.ChildCount()); i++ { if err := ctx.Err(); err != nil { return @@ -219,6 +255,9 @@ func (p *FileParser) parse(ctx context.Context, node *sitter.Node) { } p.parse(ctx, child) } + + // Restore the previous state + p.inTypeCheckingBlock = wasInTypeCheckingBlock } func (p *FileParser) Parse(ctx context.Context) (*ParserOutput, error) { diff --git a/gazelle/python/file_parser_test.go b/gazelle/python/file_parser_test.go index 20085f0e76..f4db1a316b 100644 --- a/gazelle/python/file_parser_test.go +++ b/gazelle/python/file_parser_test.go @@ -254,3 +254,40 @@ func TestParseFull(t *testing.T) { FileName: "a.py", }, *output) } + +func TestTypeCheckingImports(t *testing.T) { + code := ` +import sys +from typing import TYPE_CHECKING + +if TYPE_CHECKING: + import boto3 + from rest_framework import serializers + +def example_function(): + _ = sys.version_info +` + p := NewFileParser() + p.SetCodeAndFile([]byte(code), "", "test.py") + + result, err := p.Parse(context.Background()) + if err != nil { + t.Fatalf("Failed to parse: %v", err) + } + + // Check that we found the expected modules + expectedModules := map[string]bool{ + "sys": false, + "typing.TYPE_CHECKING": false, + "boto3": true, + "rest_framework.serializers": true, + } + + for _, mod := range result.Modules { + if expected, exists := expectedModules[mod.Name]; exists { + if mod.TypeCheckingOnly != expected { + t.Errorf("Module %s: expected TypeCheckingOnly=%v, got %v", mod.Name, expected, mod.TypeCheckingOnly) + } + } + } +} diff --git a/gazelle/python/parser.go b/gazelle/python/parser.go index cf80578220..11e01dbf51 100644 --- a/gazelle/python/parser.go +++ b/gazelle/python/parser.go @@ -112,9 +112,9 @@ func (p *python3Parser) parse(pyFilenames *treeset.Set) (*treeset.Set, map[strin continue } - modules.Add(m) + addModuleToTreeSet(modules, m) if res.HasMain { - mainModules[res.FileName].Add(m) + addModuleToTreeSet(mainModules[res.FileName], m) } } @@ -158,6 +158,8 @@ type Module struct { // If this was a from import, e.g. from foo import bar, From indicates the module // from which it is imported. From string `json:"from"` + // Whether this import is type-checking only (inside if TYPE_CHECKING block). + TypeCheckingOnly bool `json:"type_checking_only"` } // moduleComparator compares modules by name. @@ -165,6 +167,15 @@ func moduleComparator(a, b interface{}) int { return godsutils.StringComparator(a.(Module).Name, b.(Module).Name) } +// addModuleToTreeSet adds a module to a treeset.Set, ensuring that a TypeCheckingOnly=false module is +// prefered over a TypeCheckingOnly=true module. +func addModuleToTreeSet(set *treeset.Set, mod Module) { + if mod.TypeCheckingOnly && set.Contains(mod) { + return + } + set.Add(mod) +} + // annotationKind represents Gazelle annotation kinds. type annotationKind string diff --git a/gazelle/python/resolve.go b/gazelle/python/resolve.go index 413e69b289..88275e007c 100644 --- a/gazelle/python/resolve.go +++ b/gazelle/python/resolve.go @@ -123,6 +123,16 @@ func (py *Resolver) Embeds(r *rule.Rule, from label.Label) []label.Label { return make([]label.Label, 0) } +// addDependency adds a dependency to either the regular deps or pyiDeps set based on +// whether the module is type-checking only. +func addDependency(dep string, mod Module, deps, pyiDeps *treeset.Set) { + if mod.TypeCheckingOnly { + pyiDeps.Add(dep) + } else { + deps.Add(dep) + } +} + // Resolve translates imported libraries for a given rule into Bazel // dependencies. Information about imported libraries is returned for each // rule generated by language.GenerateRules in @@ -141,9 +151,11 @@ func (py *Resolver) Resolve( // join with the main Gazelle binary with other rules. It may conflict with // other generators that generate py_* targets. deps := treeset.NewWith(godsutils.StringComparator) + pyiDeps := treeset.NewWith(godsutils.StringComparator) + cfgs := c.Exts[languageName].(pythonconfig.Configs) + cfg := cfgs[from.Pkg] + if modulesRaw != nil { - cfgs := c.Exts[languageName].(pythonconfig.Configs) - cfg := cfgs[from.Pkg] pythonProjectRoot := cfg.PythonProjectRoot() modules := modulesRaw.(*treeset.Set) it := modules.Iterator() @@ -228,7 +240,7 @@ func (py *Resolver) Resolve( override.Repo = "" } dep := override.Rel(from.Repo, from.Pkg).String() - deps.Add(dep) + addDependency(dep, mod, deps, pyiDeps) if explainDependency == dep { log.Printf("Explaining dependency (%s): "+ "in the target %q, the file %q imports %q at line %d, "+ @@ -239,7 +251,7 @@ func (py *Resolver) Resolve( } } else { if dep, distributionName, ok := cfg.FindThirdPartyDependency(moduleName); ok { - deps.Add(dep) + addDependency(dep, mod, deps, pyiDeps) // Add the type and stub dependencies if they exist. modules := []string{ fmt.Sprintf("%s_stubs", strings.ToLower(distributionName)), @@ -249,7 +261,8 @@ func (py *Resolver) Resolve( } for _, module := range modules { if dep, _, ok := cfg.FindThirdPartyDependency(module); ok { - deps.Add(dep) + // Type stub packages always go to pyiDeps + pyiDeps.Add(dep) } } if explainDependency == dep { @@ -308,7 +321,7 @@ func (py *Resolver) Resolve( } matchLabel := filteredMatches[0].Label.Rel(from.Repo, from.Pkg) dep := matchLabel.String() - deps.Add(dep) + addDependency(dep, mod, deps, pyiDeps) if explainDependency == dep { log.Printf("Explaining dependency (%s): "+ "in the target %q, the file %q imports %q at line %d, "+ @@ -333,6 +346,34 @@ func (py *Resolver) Resolve( os.Exit(1) } } + + addResolvedDeps(r, deps) + + if cfg.GeneratePyiDeps() { + if !deps.Empty() { + r.SetAttr("deps", convertDependencySetToExpr(deps)) + } + if !pyiDeps.Empty() { + r.SetAttr("pyi_deps", convertDependencySetToExpr(pyiDeps)) + } + } else { + // When generate_pyi_deps is false, merge both deps and pyiDeps into deps + combinedDeps := treeset.NewWith(godsutils.StringComparator) + combinedDeps.Add(deps.Values()...) + combinedDeps.Add(pyiDeps.Values()...) + + if !combinedDeps.Empty() { + r.SetAttr("deps", convertDependencySetToExpr(combinedDeps)) + } + } +} + +// addResolvedDeps adds the pre-resolved dependencies from the rule's private attributes +// to the provided deps set. +func addResolvedDeps( + r *rule.Rule, + deps *treeset.Set, +) { resolvedDeps := r.PrivateAttr(resolvedDepsKey).(*treeset.Set) if !resolvedDeps.Empty() { it := resolvedDeps.Iterator() @@ -340,9 +381,6 @@ func (py *Resolver) Resolve( deps.Add(it.Value()) } } - if !deps.Empty() { - r.SetAttr("deps", convertDependencySetToExpr(deps)) - } } // targetListFromResults returns a string with the human-readable list of diff --git a/gazelle/python/target.go b/gazelle/python/target.go index 1fb9218656..06b653d915 100644 --- a/gazelle/python/target.go +++ b/gazelle/python/target.go @@ -15,11 +15,12 @@ package python import ( + "path/filepath" + "github.com/bazelbuild/bazel-gazelle/config" "github.com/bazelbuild/bazel-gazelle/rule" "github.com/emirpasic/gods/sets/treeset" godsutils "github.com/emirpasic/gods/utils" - "path/filepath" ) // targetBuilder builds targets to be generated by Gazelle. @@ -79,7 +80,8 @@ func (t *targetBuilder) addModuleDependency(dep Module) *targetBuilder { // dependency resolution easier dep.Name = importSpecFromSrc(t.pythonProjectRoot, t.bzlPackage, fileName).Imp } - t.deps.Add(dep) + + addModuleToTreeSet(t.deps, dep) return t } diff --git a/gazelle/python/testdata/add_type_stub_packages/BUILD.in b/gazelle/python/testdata/add_type_stub_packages/BUILD.in index e69de29bb2..99d122ad12 100644 --- a/gazelle/python/testdata/add_type_stub_packages/BUILD.in +++ b/gazelle/python/testdata/add_type_stub_packages/BUILD.in @@ -0,0 +1 @@ +# gazelle:python_generate_pyi_deps true diff --git a/gazelle/python/testdata/add_type_stub_packages/BUILD.out b/gazelle/python/testdata/add_type_stub_packages/BUILD.out index d30540f61a..1a5b640ac8 100644 --- a/gazelle/python/testdata/add_type_stub_packages/BUILD.out +++ b/gazelle/python/testdata/add_type_stub_packages/BUILD.out @@ -1,14 +1,18 @@ load("@rules_python//python:defs.bzl", "py_binary") +# gazelle:python_generate_pyi_deps true + py_binary( name = "add_type_stub_packages_bin", srcs = ["__main__.py"], main = "__main__.py", + pyi_deps = [ + "@gazelle_python_test//boto3_stubs", + "@gazelle_python_test//django_types", + ], visibility = ["//:__subpackages__"], deps = [ "@gazelle_python_test//boto3", - "@gazelle_python_test//boto3_stubs", "@gazelle_python_test//django", - "@gazelle_python_test//django_types", ], ) diff --git a/gazelle/python/testdata/add_type_stub_packages/README.md b/gazelle/python/testdata/add_type_stub_packages/README.md index c42e76f8be..e3a2afee81 100644 --- a/gazelle/python/testdata/add_type_stub_packages/README.md +++ b/gazelle/python/testdata/add_type_stub_packages/README.md @@ -1,4 +1,4 @@ # Add stubs to `deps` of `py_library` target -This test case asserts that -* if a package has the corresponding stub available, it is added to the `deps` of the `py_library` target. +This test case asserts that +* if a package has the corresponding stub available, it is added to the `pyi_deps` of the `py_library` target. diff --git a/gazelle/python/testdata/type_checking_imports/BUILD.in b/gazelle/python/testdata/type_checking_imports/BUILD.in new file mode 100644 index 0000000000..d4dce063ef --- /dev/null +++ b/gazelle/python/testdata/type_checking_imports/BUILD.in @@ -0,0 +1,2 @@ +# gazelle:python_generation_mode file +# gazelle:python_generate_pyi_deps true diff --git a/gazelle/python/testdata/type_checking_imports/BUILD.out b/gazelle/python/testdata/type_checking_imports/BUILD.out new file mode 100644 index 0000000000..690210682c --- /dev/null +++ b/gazelle/python/testdata/type_checking_imports/BUILD.out @@ -0,0 +1,33 @@ +load("@rules_python//python:defs.bzl", "py_library") + +# gazelle:python_generation_mode file +# gazelle:python_generate_pyi_deps true + +py_library( + name = "bar", + srcs = ["bar.py"], + pyi_deps = [":foo"], + visibility = ["//:__subpackages__"], + deps = [":baz"], +) + +py_library( + name = "baz", + srcs = ["baz.py"], + pyi_deps = [ + "@gazelle_python_test//boto3", + "@gazelle_python_test//boto3_stubs", + ], + visibility = ["//:__subpackages__"], +) + +py_library( + name = "foo", + srcs = ["foo.py"], + pyi_deps = [ + "@gazelle_python_test//boto3_stubs", + "@gazelle_python_test//djangorestframework", + ], + visibility = ["//:__subpackages__"], + deps = ["@gazelle_python_test//boto3"], +) diff --git a/gazelle/python/testdata/type_checking_imports/README.md b/gazelle/python/testdata/type_checking_imports/README.md new file mode 100644 index 0000000000..b09f442be3 --- /dev/null +++ b/gazelle/python/testdata/type_checking_imports/README.md @@ -0,0 +1,5 @@ +# Type Checking Imports + +Test that the Python gazelle correctly handles type-only imports inside `if TYPE_CHECKING:` blocks. + +Type-only imports should be added to the `pyi_deps` attribute instead of the regular `deps` attribute. diff --git a/gazelle/python/testdata/type_checking_imports/WORKSPACE b/gazelle/python/testdata/type_checking_imports/WORKSPACE new file mode 100644 index 0000000000..3e6e74e7f4 --- /dev/null +++ b/gazelle/python/testdata/type_checking_imports/WORKSPACE @@ -0,0 +1 @@ +workspace(name = "gazelle_python_test") diff --git a/gazelle/python/testdata/type_checking_imports/bar.py b/gazelle/python/testdata/type_checking_imports/bar.py new file mode 100644 index 0000000000..47c7d93d08 --- /dev/null +++ b/gazelle/python/testdata/type_checking_imports/bar.py @@ -0,0 +1,9 @@ +from typing import TYPE_CHECKING + +# foo should be added as a pyi_deps, since it is only imported in a type-checking context, but baz should be +# added as a deps. +from baz import X + +if TYPE_CHECKING: + import baz + import foo diff --git a/gazelle/python/testdata/type_checking_imports/baz.py b/gazelle/python/testdata/type_checking_imports/baz.py new file mode 100644 index 0000000000..1c69e25da4 --- /dev/null +++ b/gazelle/python/testdata/type_checking_imports/baz.py @@ -0,0 +1,23 @@ +# Copyright 2023 The Bazel Authors. All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + + +# While this format is not official, it is supported by most type checkers and +# is used in the wild to avoid importing the typing module. +TYPE_CHECKING = False +if TYPE_CHECKING: + # Both boto3 and boto3_stubs should be added to pyi_deps. + import boto3 + +X = 1 diff --git a/gazelle/python/testdata/type_checking_imports/foo.py b/gazelle/python/testdata/type_checking_imports/foo.py new file mode 100644 index 0000000000..655cb54675 --- /dev/null +++ b/gazelle/python/testdata/type_checking_imports/foo.py @@ -0,0 +1,21 @@ +# Copyright 2023 The Bazel Authors. All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +import typing + +# boto3 should be added to deps. boto3_stubs and djangorestframework should be added to pyi_deps. +import boto3 + +if typing.TYPE_CHECKING: + from rest_framework import serializers diff --git a/gazelle/python/testdata/type_checking_imports/gazelle_python.yaml b/gazelle/python/testdata/type_checking_imports/gazelle_python.yaml new file mode 100644 index 0000000000..a782354215 --- /dev/null +++ b/gazelle/python/testdata/type_checking_imports/gazelle_python.yaml @@ -0,0 +1,20 @@ +# Copyright 2023 The Bazel Authors. All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +manifest: + modules_mapping: + boto3: boto3 + boto3_stubs: boto3_stubs + rest_framework: djangorestframework + pip_deps_repository_name: gazelle_python_test diff --git a/gazelle/python/testdata/type_checking_imports/test.yaml b/gazelle/python/testdata/type_checking_imports/test.yaml new file mode 100644 index 0000000000..fcea77710f --- /dev/null +++ b/gazelle/python/testdata/type_checking_imports/test.yaml @@ -0,0 +1,15 @@ +# Copyright 2023 The Bazel Authors. All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +--- diff --git a/gazelle/python/testdata/type_checking_imports_disabled/BUILD.in b/gazelle/python/testdata/type_checking_imports_disabled/BUILD.in new file mode 100644 index 0000000000..ab6d30f5a7 --- /dev/null +++ b/gazelle/python/testdata/type_checking_imports_disabled/BUILD.in @@ -0,0 +1,2 @@ +# gazelle:python_generation_mode file +# gazelle:python_generate_pyi_deps false diff --git a/gazelle/python/testdata/type_checking_imports_disabled/BUILD.out b/gazelle/python/testdata/type_checking_imports_disabled/BUILD.out new file mode 100644 index 0000000000..bf23d28da9 --- /dev/null +++ b/gazelle/python/testdata/type_checking_imports_disabled/BUILD.out @@ -0,0 +1,35 @@ +load("@rules_python//python:defs.bzl", "py_library") + +# gazelle:python_generation_mode file +# gazelle:python_generate_pyi_deps false + +py_library( + name = "bar", + srcs = ["bar.py"], + visibility = ["//:__subpackages__"], + deps = [ + ":baz", + ":foo", + ], +) + +py_library( + name = "baz", + srcs = ["baz.py"], + visibility = ["//:__subpackages__"], + deps = [ + "@gazelle_python_test//boto3", + "@gazelle_python_test//boto3_stubs", + ], +) + +py_library( + name = "foo", + srcs = ["foo.py"], + visibility = ["//:__subpackages__"], + deps = [ + "@gazelle_python_test//boto3", + "@gazelle_python_test//boto3_stubs", + "@gazelle_python_test//djangorestframework", + ], +) diff --git a/gazelle/python/testdata/type_checking_imports_disabled/README.md b/gazelle/python/testdata/type_checking_imports_disabled/README.md new file mode 100644 index 0000000000..0e3b623614 --- /dev/null +++ b/gazelle/python/testdata/type_checking_imports_disabled/README.md @@ -0,0 +1,3 @@ +# Type Checking Imports (disabled) + +See `type_checking_imports`; this is the same test case, but with the directive disabled. diff --git a/gazelle/python/testdata/type_checking_imports_disabled/WORKSPACE b/gazelle/python/testdata/type_checking_imports_disabled/WORKSPACE new file mode 100644 index 0000000000..3e6e74e7f4 --- /dev/null +++ b/gazelle/python/testdata/type_checking_imports_disabled/WORKSPACE @@ -0,0 +1 @@ +workspace(name = "gazelle_python_test") diff --git a/gazelle/python/testdata/type_checking_imports_disabled/bar.py b/gazelle/python/testdata/type_checking_imports_disabled/bar.py new file mode 100644 index 0000000000..47c7d93d08 --- /dev/null +++ b/gazelle/python/testdata/type_checking_imports_disabled/bar.py @@ -0,0 +1,9 @@ +from typing import TYPE_CHECKING + +# foo should be added as a pyi_deps, since it is only imported in a type-checking context, but baz should be +# added as a deps. +from baz import X + +if TYPE_CHECKING: + import baz + import foo diff --git a/gazelle/python/testdata/type_checking_imports_disabled/baz.py b/gazelle/python/testdata/type_checking_imports_disabled/baz.py new file mode 100644 index 0000000000..1c69e25da4 --- /dev/null +++ b/gazelle/python/testdata/type_checking_imports_disabled/baz.py @@ -0,0 +1,23 @@ +# Copyright 2023 The Bazel Authors. All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + + +# While this format is not official, it is supported by most type checkers and +# is used in the wild to avoid importing the typing module. +TYPE_CHECKING = False +if TYPE_CHECKING: + # Both boto3 and boto3_stubs should be added to pyi_deps. + import boto3 + +X = 1 diff --git a/gazelle/python/testdata/type_checking_imports_disabled/foo.py b/gazelle/python/testdata/type_checking_imports_disabled/foo.py new file mode 100644 index 0000000000..655cb54675 --- /dev/null +++ b/gazelle/python/testdata/type_checking_imports_disabled/foo.py @@ -0,0 +1,21 @@ +# Copyright 2023 The Bazel Authors. All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +import typing + +# boto3 should be added to deps. boto3_stubs and djangorestframework should be added to pyi_deps. +import boto3 + +if typing.TYPE_CHECKING: + from rest_framework import serializers diff --git a/gazelle/python/testdata/type_checking_imports_disabled/gazelle_python.yaml b/gazelle/python/testdata/type_checking_imports_disabled/gazelle_python.yaml new file mode 100644 index 0000000000..a782354215 --- /dev/null +++ b/gazelle/python/testdata/type_checking_imports_disabled/gazelle_python.yaml @@ -0,0 +1,20 @@ +# Copyright 2023 The Bazel Authors. All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +manifest: + modules_mapping: + boto3: boto3 + boto3_stubs: boto3_stubs + rest_framework: djangorestframework + pip_deps_repository_name: gazelle_python_test diff --git a/gazelle/python/testdata/type_checking_imports_disabled/test.yaml b/gazelle/python/testdata/type_checking_imports_disabled/test.yaml new file mode 100644 index 0000000000..fcea77710f --- /dev/null +++ b/gazelle/python/testdata/type_checking_imports_disabled/test.yaml @@ -0,0 +1,15 @@ +# Copyright 2023 The Bazel Authors. All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +--- diff --git a/gazelle/python/testdata/type_checking_imports_package/BUILD.in b/gazelle/python/testdata/type_checking_imports_package/BUILD.in new file mode 100644 index 0000000000..8e6c1cbabb --- /dev/null +++ b/gazelle/python/testdata/type_checking_imports_package/BUILD.in @@ -0,0 +1,2 @@ +# gazelle:python_generation_mode package +# gazelle:python_generate_pyi_deps true diff --git a/gazelle/python/testdata/type_checking_imports_package/BUILD.out b/gazelle/python/testdata/type_checking_imports_package/BUILD.out new file mode 100644 index 0000000000..0091e9c5c9 --- /dev/null +++ b/gazelle/python/testdata/type_checking_imports_package/BUILD.out @@ -0,0 +1,19 @@ +load("@rules_python//python:defs.bzl", "py_library") + +# gazelle:python_generation_mode package +# gazelle:python_generate_pyi_deps true + +py_library( + name = "type_checking_imports_package", + srcs = [ + "bar.py", + "baz.py", + "foo.py", + ], + pyi_deps = [ + "@gazelle_python_test//boto3_stubs", + "@gazelle_python_test//djangorestframework", + ], + visibility = ["//:__subpackages__"], + deps = ["@gazelle_python_test//boto3"], +) diff --git a/gazelle/python/testdata/type_checking_imports_package/README.md b/gazelle/python/testdata/type_checking_imports_package/README.md new file mode 100644 index 0000000000..3e2cafe992 --- /dev/null +++ b/gazelle/python/testdata/type_checking_imports_package/README.md @@ -0,0 +1,3 @@ +# Type Checking Imports (package mode) + +See `type_checking_imports`; this is the same test case, but using the package generation mode. diff --git a/gazelle/python/testdata/type_checking_imports_package/WORKSPACE b/gazelle/python/testdata/type_checking_imports_package/WORKSPACE new file mode 100644 index 0000000000..3e6e74e7f4 --- /dev/null +++ b/gazelle/python/testdata/type_checking_imports_package/WORKSPACE @@ -0,0 +1 @@ +workspace(name = "gazelle_python_test") diff --git a/gazelle/python/testdata/type_checking_imports_package/bar.py b/gazelle/python/testdata/type_checking_imports_package/bar.py new file mode 100644 index 0000000000..47c7d93d08 --- /dev/null +++ b/gazelle/python/testdata/type_checking_imports_package/bar.py @@ -0,0 +1,9 @@ +from typing import TYPE_CHECKING + +# foo should be added as a pyi_deps, since it is only imported in a type-checking context, but baz should be +# added as a deps. +from baz import X + +if TYPE_CHECKING: + import baz + import foo diff --git a/gazelle/python/testdata/type_checking_imports_package/baz.py b/gazelle/python/testdata/type_checking_imports_package/baz.py new file mode 100644 index 0000000000..1c69e25da4 --- /dev/null +++ b/gazelle/python/testdata/type_checking_imports_package/baz.py @@ -0,0 +1,23 @@ +# Copyright 2023 The Bazel Authors. All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + + +# While this format is not official, it is supported by most type checkers and +# is used in the wild to avoid importing the typing module. +TYPE_CHECKING = False +if TYPE_CHECKING: + # Both boto3 and boto3_stubs should be added to pyi_deps. + import boto3 + +X = 1 diff --git a/gazelle/python/testdata/type_checking_imports_package/foo.py b/gazelle/python/testdata/type_checking_imports_package/foo.py new file mode 100644 index 0000000000..655cb54675 --- /dev/null +++ b/gazelle/python/testdata/type_checking_imports_package/foo.py @@ -0,0 +1,21 @@ +# Copyright 2023 The Bazel Authors. All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +import typing + +# boto3 should be added to deps. boto3_stubs and djangorestframework should be added to pyi_deps. +import boto3 + +if typing.TYPE_CHECKING: + from rest_framework import serializers diff --git a/gazelle/python/testdata/type_checking_imports_package/gazelle_python.yaml b/gazelle/python/testdata/type_checking_imports_package/gazelle_python.yaml new file mode 100644 index 0000000000..a782354215 --- /dev/null +++ b/gazelle/python/testdata/type_checking_imports_package/gazelle_python.yaml @@ -0,0 +1,20 @@ +# Copyright 2023 The Bazel Authors. All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +manifest: + modules_mapping: + boto3: boto3 + boto3_stubs: boto3_stubs + rest_framework: djangorestframework + pip_deps_repository_name: gazelle_python_test diff --git a/gazelle/python/testdata/type_checking_imports_package/test.yaml b/gazelle/python/testdata/type_checking_imports_package/test.yaml new file mode 100644 index 0000000000..fcea77710f --- /dev/null +++ b/gazelle/python/testdata/type_checking_imports_package/test.yaml @@ -0,0 +1,15 @@ +# Copyright 2023 The Bazel Authors. All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +--- diff --git a/gazelle/python/testdata/type_checking_imports_project/BUILD.in b/gazelle/python/testdata/type_checking_imports_project/BUILD.in new file mode 100644 index 0000000000..808e3e044e --- /dev/null +++ b/gazelle/python/testdata/type_checking_imports_project/BUILD.in @@ -0,0 +1,2 @@ +# gazelle:python_generation_mode project +# gazelle:python_generate_pyi_deps true diff --git a/gazelle/python/testdata/type_checking_imports_project/BUILD.out b/gazelle/python/testdata/type_checking_imports_project/BUILD.out new file mode 100644 index 0000000000..6d6ac3cef9 --- /dev/null +++ b/gazelle/python/testdata/type_checking_imports_project/BUILD.out @@ -0,0 +1,19 @@ +load("@rules_python//python:defs.bzl", "py_library") + +# gazelle:python_generation_mode project +# gazelle:python_generate_pyi_deps true + +py_library( + name = "type_checking_imports_project", + srcs = [ + "bar.py", + "baz.py", + "foo.py", + ], + pyi_deps = [ + "@gazelle_python_test//boto3_stubs", + "@gazelle_python_test//djangorestframework", + ], + visibility = ["//:__subpackages__"], + deps = ["@gazelle_python_test//boto3"], +) diff --git a/gazelle/python/testdata/type_checking_imports_project/README.md b/gazelle/python/testdata/type_checking_imports_project/README.md new file mode 100644 index 0000000000..ead09e1994 --- /dev/null +++ b/gazelle/python/testdata/type_checking_imports_project/README.md @@ -0,0 +1,3 @@ +# Type Checking Imports (project mode) + +See `type_checking_imports`; this is the same test case, but using the project generation mode. diff --git a/gazelle/python/testdata/type_checking_imports_project/WORKSPACE b/gazelle/python/testdata/type_checking_imports_project/WORKSPACE new file mode 100644 index 0000000000..3e6e74e7f4 --- /dev/null +++ b/gazelle/python/testdata/type_checking_imports_project/WORKSPACE @@ -0,0 +1 @@ +workspace(name = "gazelle_python_test") diff --git a/gazelle/python/testdata/type_checking_imports_project/bar.py b/gazelle/python/testdata/type_checking_imports_project/bar.py new file mode 100644 index 0000000000..47c7d93d08 --- /dev/null +++ b/gazelle/python/testdata/type_checking_imports_project/bar.py @@ -0,0 +1,9 @@ +from typing import TYPE_CHECKING + +# foo should be added as a pyi_deps, since it is only imported in a type-checking context, but baz should be +# added as a deps. +from baz import X + +if TYPE_CHECKING: + import baz + import foo diff --git a/gazelle/python/testdata/type_checking_imports_project/baz.py b/gazelle/python/testdata/type_checking_imports_project/baz.py new file mode 100644 index 0000000000..1c69e25da4 --- /dev/null +++ b/gazelle/python/testdata/type_checking_imports_project/baz.py @@ -0,0 +1,23 @@ +# Copyright 2023 The Bazel Authors. All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + + +# While this format is not official, it is supported by most type checkers and +# is used in the wild to avoid importing the typing module. +TYPE_CHECKING = False +if TYPE_CHECKING: + # Both boto3 and boto3_stubs should be added to pyi_deps. + import boto3 + +X = 1 diff --git a/gazelle/python/testdata/type_checking_imports_project/foo.py b/gazelle/python/testdata/type_checking_imports_project/foo.py new file mode 100644 index 0000000000..655cb54675 --- /dev/null +++ b/gazelle/python/testdata/type_checking_imports_project/foo.py @@ -0,0 +1,21 @@ +# Copyright 2023 The Bazel Authors. All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +import typing + +# boto3 should be added to deps. boto3_stubs and djangorestframework should be added to pyi_deps. +import boto3 + +if typing.TYPE_CHECKING: + from rest_framework import serializers diff --git a/gazelle/python/testdata/type_checking_imports_project/gazelle_python.yaml b/gazelle/python/testdata/type_checking_imports_project/gazelle_python.yaml new file mode 100644 index 0000000000..a782354215 --- /dev/null +++ b/gazelle/python/testdata/type_checking_imports_project/gazelle_python.yaml @@ -0,0 +1,20 @@ +# Copyright 2023 The Bazel Authors. All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +manifest: + modules_mapping: + boto3: boto3 + boto3_stubs: boto3_stubs + rest_framework: djangorestframework + pip_deps_repository_name: gazelle_python_test diff --git a/gazelle/python/testdata/type_checking_imports_project/test.yaml b/gazelle/python/testdata/type_checking_imports_project/test.yaml new file mode 100644 index 0000000000..fcea77710f --- /dev/null +++ b/gazelle/python/testdata/type_checking_imports_project/test.yaml @@ -0,0 +1,15 @@ +# Copyright 2023 The Bazel Authors. All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +--- diff --git a/gazelle/pythonconfig/pythonconfig.go b/gazelle/pythonconfig/pythonconfig.go index e0a2b8a469..8bf79cbc15 100644 --- a/gazelle/pythonconfig/pythonconfig.go +++ b/gazelle/pythonconfig/pythonconfig.go @@ -94,6 +94,10 @@ const ( // ExperimentalAllowRelativeImports represents the directive that controls // whether relative imports are allowed. ExperimentalAllowRelativeImports = "experimental_allow_relative_imports" + // GeneratePyiDeps represents the directive that controls whether to generate + // separate pyi_deps attribute or merge type-checking dependencies into deps. + // Defaults to false for backward compatibility. + GeneratePyiDeps = "python_generate_pyi_deps" ) // GenerationModeType represents one of the generation modes for the Python @@ -181,6 +185,7 @@ type Config struct { labelConvention string labelNormalization LabelNormalizationType experimentalAllowRelativeImports bool + generatePyiDeps bool } type LabelNormalizationType int @@ -217,6 +222,7 @@ func New( labelConvention: DefaultLabelConvention, labelNormalization: DefaultLabelNormalizationType, experimentalAllowRelativeImports: false, + generatePyiDeps: false, } } @@ -250,6 +256,7 @@ func (c *Config) NewChild() *Config { labelConvention: c.labelConvention, labelNormalization: c.labelNormalization, experimentalAllowRelativeImports: c.experimentalAllowRelativeImports, + generatePyiDeps: c.generatePyiDeps, } } @@ -536,6 +543,18 @@ func (c *Config) ExperimentalAllowRelativeImports() bool { return c.experimentalAllowRelativeImports } +// SetGeneratePyiDeps sets whether pyi_deps attribute should be generated separately +// or type-checking dependencies should be merged into the regular deps attribute. +func (c *Config) SetGeneratePyiDeps(generatePyiDeps bool) { + c.generatePyiDeps = generatePyiDeps +} + +// GeneratePyiDeps returns whether pyi_deps attribute should be generated separately +// or type-checking dependencies should be merged into the regular deps attribute. +func (c *Config) GeneratePyiDeps() bool { + return c.generatePyiDeps +} + // FormatThirdPartyDependency returns a label to a third-party dependency performing all formating and normalization. func (c *Config) FormatThirdPartyDependency(repositoryName string, distributionName string) label.Label { conventionalDistributionName := strings.ReplaceAll(c.labelConvention, distributionNameLabelConventionSubstitution, distributionName) From 77195299d59eb3ea4234da43895488277c80a87e Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Fri, 27 Jun 2025 09:00:48 -0700 Subject: [PATCH 300/922] fix: delete BUILD et al files from pypi sourced dependencies (#3029) Sometimes wheels mistakenly contain BUILD files (or other special Bazel files). When extracted, these interfere with how we expected the repo to look. In particular, globs no longer match correctly because BUILD files create a Bazel package boundary. To fix, delete these files. They aren't generally usable, since they can't know what version of Bazel, rule_python, or similar, is consuming them. Fixes https://github.com/bazel-contrib/rules_python/issues/2782 --- .bazelrc | 4 +- CHANGELOG.md | 3 ++ MODULE.bazel | 2 + python/private/internal_dev_deps.bzl | 13 +++++ python/private/pypi/whl_library.bzl | 21 +++++++- tests/support/support.bzl | 6 +++ tests/support/whl_from_dir/BUILD.bazel | 0 .../whl_from_dir/whl_from_dir_repo.bzl | 50 +++++++++++++++++++ tests/whl_with_build_files/BUILD.bazel | 9 ++++ tests/whl_with_build_files/testdata/BUILD | 0 .../whl_with_build_files/testdata/BUILD.bazel | 0 .../whl_with_build_files/testdata/REPO.bazel | 0 .../testdata/somepkg-1.0.dist-info/BUILD | 0 .../somepkg-1.0.dist-info/BUILD.bazel | 0 .../testdata/somepkg-1.0.dist-info/METADATA | 0 .../testdata/somepkg-1.0.dist-info/RECORD | 0 .../testdata/somepkg-1.0.dist-info/WHEEL | 1 + .../testdata/somepkg/BUILD | 0 .../testdata/somepkg/BUILD.bazel | 0 .../testdata/somepkg/__init__.py | 0 .../testdata/somepkg/a.py | 0 .../testdata/somepkg/subpkg/BUILD | 0 .../testdata/somepkg/subpkg/BUILD.bazel | 0 .../testdata/somepkg/subpkg/__init__.py | 0 .../testdata/somepkg/subpkg/b.py | 0 .../whl_with_build_files/verify_files_test.py | 17 +++++++ 26 files changed, 123 insertions(+), 3 deletions(-) create mode 100644 tests/support/whl_from_dir/BUILD.bazel create mode 100644 tests/support/whl_from_dir/whl_from_dir_repo.bzl create mode 100644 tests/whl_with_build_files/BUILD.bazel create mode 100644 tests/whl_with_build_files/testdata/BUILD create mode 100644 tests/whl_with_build_files/testdata/BUILD.bazel create mode 100644 tests/whl_with_build_files/testdata/REPO.bazel create mode 100644 tests/whl_with_build_files/testdata/somepkg-1.0.dist-info/BUILD create mode 100644 tests/whl_with_build_files/testdata/somepkg-1.0.dist-info/BUILD.bazel create mode 100644 tests/whl_with_build_files/testdata/somepkg-1.0.dist-info/METADATA create mode 100644 tests/whl_with_build_files/testdata/somepkg-1.0.dist-info/RECORD create mode 100644 tests/whl_with_build_files/testdata/somepkg-1.0.dist-info/WHEEL create mode 100644 tests/whl_with_build_files/testdata/somepkg/BUILD create mode 100644 tests/whl_with_build_files/testdata/somepkg/BUILD.bazel create mode 100644 tests/whl_with_build_files/testdata/somepkg/__init__.py create mode 100644 tests/whl_with_build_files/testdata/somepkg/a.py create mode 100644 tests/whl_with_build_files/testdata/somepkg/subpkg/BUILD create mode 100644 tests/whl_with_build_files/testdata/somepkg/subpkg/BUILD.bazel create mode 100644 tests/whl_with_build_files/testdata/somepkg/subpkg/__init__.py create mode 100644 tests/whl_with_build_files/testdata/somepkg/subpkg/b.py create mode 100644 tests/whl_with_build_files/verify_files_test.py diff --git a/.bazelrc b/.bazelrc index f7f31aed98..8997db9f91 100644 --- a/.bazelrc +++ b/.bazelrc @@ -4,8 +4,8 @@ # (Note, we cannot use `common --deleted_packages` because the bazel version command doesn't support it) # To update these lines, execute # `bazel run @rules_bazel_integration_test//tools:update_deleted_packages` -build --deleted_packages=examples/build_file_generation,examples/build_file_generation/random_number_generator,examples/bzlmod,examples/bzlmod_build_file_generation,examples/bzlmod_build_file_generation/other_module/other_module/pkg,examples/bzlmod_build_file_generation/runfiles,examples/bzlmod/entry_points,examples/bzlmod/entry_points/tests,examples/bzlmod/libs/my_lib,examples/bzlmod/other_module,examples/bzlmod/other_module/other_module/pkg,examples/bzlmod/patches,examples/bzlmod/py_proto_library,examples/bzlmod/py_proto_library/example.com/another_proto,examples/bzlmod/py_proto_library/example.com/proto,examples/bzlmod/runfiles,examples/bzlmod/tests,examples/bzlmod/tests/other_module,examples/bzlmod/whl_mods,examples/multi_python_versions/libs/my_lib,examples/multi_python_versions/requirements,examples/multi_python_versions/tests,examples/pip_parse,examples/pip_parse_vendored,examples/pip_repository_annotations,examples/py_proto_library,examples/py_proto_library/example.com/another_proto,examples/py_proto_library/example.com/proto,gazelle,gazelle/manifest,gazelle/manifest/generate,gazelle/manifest/hasher,gazelle/manifest/test,gazelle/modules_mapping,gazelle/python,gazelle/pythonconfig,gazelle/python/private,tests/integration/compile_pip_requirements,tests/integration/compile_pip_requirements_test_from_external_repo,tests/integration/custom_commands,tests/integration/ignore_root_user_error,tests/integration/ignore_root_user_error/submodule,tests/integration/local_toolchains,tests/integration/pip_parse,tests/integration/pip_parse/empty,tests/integration/py_cc_toolchain_registered,tests/modules/another_module,tests/modules/other,tests/modules/other/nspkg_delta,tests/modules/other/nspkg_gamma,tests/modules/other/nspkg_single,tests/modules/other/simple_v1,tests/modules/other/simple_v2,tests/modules/other/with_external_data -query --deleted_packages=examples/build_file_generation,examples/build_file_generation/random_number_generator,examples/bzlmod,examples/bzlmod_build_file_generation,examples/bzlmod_build_file_generation/other_module/other_module/pkg,examples/bzlmod_build_file_generation/runfiles,examples/bzlmod/entry_points,examples/bzlmod/entry_points/tests,examples/bzlmod/libs/my_lib,examples/bzlmod/other_module,examples/bzlmod/other_module/other_module/pkg,examples/bzlmod/patches,examples/bzlmod/py_proto_library,examples/bzlmod/py_proto_library/example.com/another_proto,examples/bzlmod/py_proto_library/example.com/proto,examples/bzlmod/runfiles,examples/bzlmod/tests,examples/bzlmod/tests/other_module,examples/bzlmod/whl_mods,examples/multi_python_versions/libs/my_lib,examples/multi_python_versions/requirements,examples/multi_python_versions/tests,examples/pip_parse,examples/pip_parse_vendored,examples/pip_repository_annotations,examples/py_proto_library,examples/py_proto_library/example.com/another_proto,examples/py_proto_library/example.com/proto,gazelle,gazelle/manifest,gazelle/manifest/generate,gazelle/manifest/hasher,gazelle/manifest/test,gazelle/modules_mapping,gazelle/python,gazelle/pythonconfig,gazelle/python/private,tests/integration/compile_pip_requirements,tests/integration/compile_pip_requirements_test_from_external_repo,tests/integration/custom_commands,tests/integration/ignore_root_user_error,tests/integration/ignore_root_user_error/submodule,tests/integration/local_toolchains,tests/integration/pip_parse,tests/integration/pip_parse/empty,tests/integration/py_cc_toolchain_registered,tests/modules/another_module,tests/modules/other,tests/modules/other/nspkg_delta,tests/modules/other/nspkg_gamma,tests/modules/other/nspkg_single,tests/modules/other/simple_v1,tests/modules/other/simple_v2,tests/modules/other/with_external_data +build --deleted_packages=examples/build_file_generation,examples/build_file_generation/random_number_generator,examples/bzlmod,examples/bzlmod_build_file_generation,examples/bzlmod_build_file_generation/other_module/other_module/pkg,examples/bzlmod_build_file_generation/runfiles,examples/bzlmod/entry_points,examples/bzlmod/entry_points/tests,examples/bzlmod/libs/my_lib,examples/bzlmod/other_module,examples/bzlmod/other_module/other_module/pkg,examples/bzlmod/patches,examples/bzlmod/py_proto_library,examples/bzlmod/py_proto_library/example.com/another_proto,examples/bzlmod/py_proto_library/example.com/proto,examples/bzlmod/runfiles,examples/bzlmod/tests,examples/bzlmod/tests/other_module,examples/bzlmod/whl_mods,examples/multi_python_versions/libs/my_lib,examples/multi_python_versions/requirements,examples/multi_python_versions/tests,examples/pip_parse,examples/pip_parse_vendored,examples/pip_repository_annotations,examples/py_proto_library,examples/py_proto_library/example.com/another_proto,examples/py_proto_library/example.com/proto,gazelle,gazelle/manifest,gazelle/manifest/generate,gazelle/manifest/hasher,gazelle/manifest/test,gazelle/modules_mapping,gazelle/python,gazelle/pythonconfig,gazelle/python/private,tests/integration/compile_pip_requirements,tests/integration/compile_pip_requirements_test_from_external_repo,tests/integration/custom_commands,tests/integration/ignore_root_user_error,tests/integration/ignore_root_user_error/submodule,tests/integration/local_toolchains,tests/integration/pip_parse,tests/integration/pip_parse/empty,tests/integration/py_cc_toolchain_registered,tests/modules/another_module,tests/modules/other,tests/modules/other/nspkg_delta,tests/modules/other/nspkg_gamma,tests/modules/other/nspkg_single,tests/modules/other/simple_v1,tests/modules/other/simple_v2,tests/modules/other/with_external_data,tests/whl_with_build_files/testdata,tests/whl_with_build_files/testdata/somepkg,tests/whl_with_build_files/testdata/somepkg-1.0.dist-info,tests/whl_with_build_files/testdata/somepkg/subpkg +query --deleted_packages=examples/build_file_generation,examples/build_file_generation/random_number_generator,examples/bzlmod,examples/bzlmod_build_file_generation,examples/bzlmod_build_file_generation/other_module/other_module/pkg,examples/bzlmod_build_file_generation/runfiles,examples/bzlmod/entry_points,examples/bzlmod/entry_points/tests,examples/bzlmod/libs/my_lib,examples/bzlmod/other_module,examples/bzlmod/other_module/other_module/pkg,examples/bzlmod/patches,examples/bzlmod/py_proto_library,examples/bzlmod/py_proto_library/example.com/another_proto,examples/bzlmod/py_proto_library/example.com/proto,examples/bzlmod/runfiles,examples/bzlmod/tests,examples/bzlmod/tests/other_module,examples/bzlmod/whl_mods,examples/multi_python_versions/libs/my_lib,examples/multi_python_versions/requirements,examples/multi_python_versions/tests,examples/pip_parse,examples/pip_parse_vendored,examples/pip_repository_annotations,examples/py_proto_library,examples/py_proto_library/example.com/another_proto,examples/py_proto_library/example.com/proto,gazelle,gazelle/manifest,gazelle/manifest/generate,gazelle/manifest/hasher,gazelle/manifest/test,gazelle/modules_mapping,gazelle/python,gazelle/pythonconfig,gazelle/python/private,tests/integration/compile_pip_requirements,tests/integration/compile_pip_requirements_test_from_external_repo,tests/integration/custom_commands,tests/integration/ignore_root_user_error,tests/integration/ignore_root_user_error/submodule,tests/integration/local_toolchains,tests/integration/pip_parse,tests/integration/pip_parse/empty,tests/integration/py_cc_toolchain_registered,tests/modules/another_module,tests/modules/other,tests/modules/other/nspkg_delta,tests/modules/other/nspkg_gamma,tests/modules/other/nspkg_single,tests/modules/other/simple_v1,tests/modules/other/simple_v2,tests/modules/other/with_external_data,tests/whl_with_build_files/testdata,tests/whl_with_build_files/testdata/somepkg,tests/whl_with_build_files/testdata/somepkg-1.0.dist-info,tests/whl_with_build_files/testdata/somepkg/subpkg test --test_output=errors diff --git a/CHANGELOG.md b/CHANGELOG.md index 78a3d1caf5..8cb5ca3f9f 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -64,6 +64,9 @@ END_UNRELEASED_TEMPLATE * (pypi) Fixes an issue where builds using a `bazel vendor` vendor directory would fail if the constraints file contained environment markers. Fixes [#2996](https://github.com/bazel-contrib/rules_python/issues/2996). +* (pypi) Wheels with BUILD.bazel (or other special Bazel files) no longer + result in missing files at runtime + ([#2782](https://github.com/bazel-contrib/rules_python/issues/2782)). {#v0-0-0-added} ### Added diff --git a/MODULE.bazel b/MODULE.bazel index 77fa12d113..b1d8711815 100644 --- a/MODULE.bazel +++ b/MODULE.bazel @@ -103,6 +103,8 @@ use_repo( internal_dev_deps, "buildkite_config", "rules_python_runtime_env_tc_info", + "somepkg_with_build_files", + "whl_with_build_files", ) # Add gazelle plugin so that we can run the gazelle example as an e2e integration diff --git a/python/private/internal_dev_deps.bzl b/python/private/internal_dev_deps.bzl index 600c934ace..bb7d76f56a 100644 --- a/python/private/internal_dev_deps.bzl +++ b/python/private/internal_dev_deps.bzl @@ -14,6 +14,8 @@ """Module extension for internal dev_dependency=True setup.""" load("@bazel_ci_rules//:rbe_repo.bzl", "rbe_preconfig") +load("//python/private/pypi:whl_library.bzl", "whl_library") +load("//tests/support/whl_from_dir:whl_from_dir_repo.bzl", "whl_from_dir_repo") load(":runtime_env_repo.bzl", "runtime_env_repo") def _internal_dev_deps_impl(mctx): @@ -28,6 +30,17 @@ def _internal_dev_deps_impl(mctx): ) runtime_env_repo(name = "rules_python_runtime_env_tc_info") + whl_from_dir_repo( + name = "whl_with_build_files", + root = "//tests/whl_with_build_files:testdata/BUILD.bazel", + output = "somepkg-1.0-any-none-any.whl", + ) + whl_library( + name = "somepkg_with_build_files", + whl_file = "@whl_with_build_files//:somepkg-1.0-any-none-any.whl", + requirement = "somepkg", + ) + internal_dev_deps = module_extension( implementation = _internal_dev_deps_impl, doc = "This extension creates internal rules_python dev dependencies.", diff --git a/python/private/pypi/whl_library.bzl b/python/private/pypi/whl_library.bzl index c271449b3d..de5fcb9f91 100644 --- a/python/private/pypi/whl_library.bzl +++ b/python/private/pypi/whl_library.bzl @@ -249,6 +249,7 @@ def _whl_library_impl(rctx): whl_path = None if rctx.attr.whl_file: + rctx.watch(rctx.attr.whl_file) whl_path = rctx.path(rctx.attr.whl_file) # Simulate the behaviour where the whl is present in the current directory. @@ -471,8 +472,26 @@ def _whl_library_impl(rctx): ], ) - rctx.file("BUILD.bazel", build_file_contents) + # Delete these in case the wheel had them. They generally don't cause + # a problem, but let's avoid the chance of that happening. + rctx.file("WORKSPACE") + rctx.file("WORKSPACE.bazel") + rctx.file("MODULE.bazel") + rctx.file("REPO.bazel") + + paths = list(rctx.path(".").readdir()) + for _ in range(10000000): + if not paths: + break + path = paths.pop() + + # BUILD files interfere with globbing and Bazel package boundaries. + if path.basename in ("BUILD", "BUILD.bazel"): + rctx.delete(path) + elif path.is_dir: + paths.extend(path.readdir()) + rctx.file("BUILD.bazel", build_file_contents) return def _generate_entry_point_contents( diff --git a/tests/support/support.bzl b/tests/support/support.bzl index 7bab263c66..adb8e75f71 100644 --- a/tests/support/support.bzl +++ b/tests/support/support.bzl @@ -19,6 +19,7 @@ # rules_testing or as config_setting values, which don't support Label in some # places. +load("//python/private:bzlmod_enabled.bzl", "BZLMOD_ENABLED") # buildifier: disable=bzl-visibility load("//python/private:util.bzl", "IS_BAZEL_7_OR_HIGHER") # buildifier: disable=bzl-visibility MAC = Label("//tests/support:mac") @@ -48,3 +49,8 @@ SUPPORTS_BOOTSTRAP_SCRIPT = select({ "@platforms//os:windows": ["@platforms//:incompatible"], "//conditions:default": [], }) if IS_BAZEL_7_OR_HIGHER else ["@platforms//:incompatible"] + +SUPPORTS_BZLMOD_UNIXY = select({ + "@platforms//os:windows": ["@platforms//:incompatible"], + "//conditions:default": [], +}) if BZLMOD_ENABLED else ["@platforms//:incompatible"] diff --git a/tests/support/whl_from_dir/BUILD.bazel b/tests/support/whl_from_dir/BUILD.bazel new file mode 100644 index 0000000000..e69de29bb2 diff --git a/tests/support/whl_from_dir/whl_from_dir_repo.bzl b/tests/support/whl_from_dir/whl_from_dir_repo.bzl new file mode 100644 index 0000000000..176525636c --- /dev/null +++ b/tests/support/whl_from_dir/whl_from_dir_repo.bzl @@ -0,0 +1,50 @@ +"""Creates a whl file from a directory tree. + +Used to test wheels. Avoids checking in prebuilt files and their associated +security risks. +""" + +load("//python/private:repo_utils.bzl", "repo_utils") # buildifier: disable=bzl-visibility + +def _whl_from_dir_repo(rctx): + root = rctx.path(rctx.attr.root).dirname + repo_utils.watch_tree(rctx, root) + + output = rctx.path(rctx.attr.output) + repo_utils.execute_checked( + rctx, + # cd to root so zip recursively takes everything there. + working_directory = str(root), + op = "WhlFromDir", + arguments = [ + "zip", + "-0", # Skip compressing + "-X", # Don't store file time or metadata + str(output), + "-r", + ".", + ], + ) + rctx.file("BUILD.bazel", 'exports_files(glob(["*"]))') + +whl_from_dir_repo = repository_rule( + implementation = _whl_from_dir_repo, + attrs = { + "output": attr.string( + doc = """ +Output file name to write. Should match the wheel filename format: +`pkg-version-pyversion-abi-platform.whl`. Typically a value like +`mypkg-1.0-any-none-any.whl` is whats used for testing. + +For the full format, see +https://packaging.python.org/en/latest/specifications/binary-distribution-format/#file-name-convention +""", + ), + "root": attr.label( + doc = """ +A file whose directory will be put into the output wheel. All files +are included verbatim. + """, + ), + }, +) diff --git a/tests/whl_with_build_files/BUILD.bazel b/tests/whl_with_build_files/BUILD.bazel new file mode 100644 index 0000000000..e26dc1c3a6 --- /dev/null +++ b/tests/whl_with_build_files/BUILD.bazel @@ -0,0 +1,9 @@ +load("//python:py_test.bzl", "py_test") +load("//tests/support:support.bzl", "SUPPORTS_BZLMOD_UNIXY") + +py_test( + name = "verify_files_test", + srcs = ["verify_files_test.py"], + target_compatible_with = SUPPORTS_BZLMOD_UNIXY, + deps = ["@somepkg_with_build_files//:pkg"], +) diff --git a/tests/whl_with_build_files/testdata/BUILD b/tests/whl_with_build_files/testdata/BUILD new file mode 100644 index 0000000000..e69de29bb2 diff --git a/tests/whl_with_build_files/testdata/BUILD.bazel b/tests/whl_with_build_files/testdata/BUILD.bazel new file mode 100644 index 0000000000..e69de29bb2 diff --git a/tests/whl_with_build_files/testdata/REPO.bazel b/tests/whl_with_build_files/testdata/REPO.bazel new file mode 100644 index 0000000000..e69de29bb2 diff --git a/tests/whl_with_build_files/testdata/somepkg-1.0.dist-info/BUILD b/tests/whl_with_build_files/testdata/somepkg-1.0.dist-info/BUILD new file mode 100644 index 0000000000..e69de29bb2 diff --git a/tests/whl_with_build_files/testdata/somepkg-1.0.dist-info/BUILD.bazel b/tests/whl_with_build_files/testdata/somepkg-1.0.dist-info/BUILD.bazel new file mode 100644 index 0000000000..e69de29bb2 diff --git a/tests/whl_with_build_files/testdata/somepkg-1.0.dist-info/METADATA b/tests/whl_with_build_files/testdata/somepkg-1.0.dist-info/METADATA new file mode 100644 index 0000000000..e69de29bb2 diff --git a/tests/whl_with_build_files/testdata/somepkg-1.0.dist-info/RECORD b/tests/whl_with_build_files/testdata/somepkg-1.0.dist-info/RECORD new file mode 100644 index 0000000000..e69de29bb2 diff --git a/tests/whl_with_build_files/testdata/somepkg-1.0.dist-info/WHEEL b/tests/whl_with_build_files/testdata/somepkg-1.0.dist-info/WHEEL new file mode 100644 index 0000000000..a64521a1cc --- /dev/null +++ b/tests/whl_with_build_files/testdata/somepkg-1.0.dist-info/WHEEL @@ -0,0 +1 @@ +Wheel-Version: 1.0 diff --git a/tests/whl_with_build_files/testdata/somepkg/BUILD b/tests/whl_with_build_files/testdata/somepkg/BUILD new file mode 100644 index 0000000000..e69de29bb2 diff --git a/tests/whl_with_build_files/testdata/somepkg/BUILD.bazel b/tests/whl_with_build_files/testdata/somepkg/BUILD.bazel new file mode 100644 index 0000000000..e69de29bb2 diff --git a/tests/whl_with_build_files/testdata/somepkg/__init__.py b/tests/whl_with_build_files/testdata/somepkg/__init__.py new file mode 100644 index 0000000000..e69de29bb2 diff --git a/tests/whl_with_build_files/testdata/somepkg/a.py b/tests/whl_with_build_files/testdata/somepkg/a.py new file mode 100644 index 0000000000..e69de29bb2 diff --git a/tests/whl_with_build_files/testdata/somepkg/subpkg/BUILD b/tests/whl_with_build_files/testdata/somepkg/subpkg/BUILD new file mode 100644 index 0000000000..e69de29bb2 diff --git a/tests/whl_with_build_files/testdata/somepkg/subpkg/BUILD.bazel b/tests/whl_with_build_files/testdata/somepkg/subpkg/BUILD.bazel new file mode 100644 index 0000000000..e69de29bb2 diff --git a/tests/whl_with_build_files/testdata/somepkg/subpkg/__init__.py b/tests/whl_with_build_files/testdata/somepkg/subpkg/__init__.py new file mode 100644 index 0000000000..e69de29bb2 diff --git a/tests/whl_with_build_files/testdata/somepkg/subpkg/b.py b/tests/whl_with_build_files/testdata/somepkg/subpkg/b.py new file mode 100644 index 0000000000..e69de29bb2 diff --git a/tests/whl_with_build_files/verify_files_test.py b/tests/whl_with_build_files/verify_files_test.py new file mode 100644 index 0000000000..cfbbaa3aff --- /dev/null +++ b/tests/whl_with_build_files/verify_files_test.py @@ -0,0 +1,17 @@ +import unittest + + +class VerifyFilestest(unittest.TestCase): + + def test_wheel_with_build_files_importable(self): + # If the BUILD files are present, then these imports should fail + # because globs won't pass package boundaries, and the necessary + # py files end up missing in runfiles. + import somepkg + import somepkg.a + import somepkg.subpkg + import somepkg.subpkg.b + + +if __name__ == "__main__": + unittest.main() From 57f819c69a1e0014273228c0d6f88e25d23c3de0 Mon Sep 17 00:00:00 2001 From: Alex Martani Date: Sun, 29 Jun 2025 15:44:53 -0700 Subject: [PATCH 301/922] fix(gazelle) Fix dependency added as both deps and pyi_deps (#3036) Fix an issue in https://github.com/bazel-contrib/rules_python/pull/3014 where a dependency may end up being added in both `deps` and `pyi_deps`, in cases where the regular and the type-checking import refer to different python modules on the same `py_library` target. Other cases are already deduplicated earlier on, but this case can only be deduplicated in the resolve phase. (No new changelog entry since this is a fix to an unreleased feature that is already in the changelog) --- gazelle/python/resolve.go | 22 +++++++++++-------- .../BUILD.in | 2 ++ .../BUILD.out | 2 ++ .../README.md | 6 +++++ .../WORKSPACE | 1 + .../a/BUILD.in | 0 .../a/BUILD.out | 10 +++++++++ .../a/bar.py | 0 .../a/foo.py | 0 .../b/BUILD.in | 0 .../b/BUILD.out | 8 +++++++ .../b/b.py | 6 +++++ .../test.yaml | 1 + 13 files changed, 49 insertions(+), 9 deletions(-) create mode 100644 gazelle/python/testdata/type_checking_imports_across_packages/BUILD.in create mode 100644 gazelle/python/testdata/type_checking_imports_across_packages/BUILD.out create mode 100644 gazelle/python/testdata/type_checking_imports_across_packages/README.md create mode 100644 gazelle/python/testdata/type_checking_imports_across_packages/WORKSPACE create mode 100644 gazelle/python/testdata/type_checking_imports_across_packages/a/BUILD.in create mode 100644 gazelle/python/testdata/type_checking_imports_across_packages/a/BUILD.out create mode 100644 gazelle/python/testdata/type_checking_imports_across_packages/a/bar.py create mode 100644 gazelle/python/testdata/type_checking_imports_across_packages/a/foo.py create mode 100644 gazelle/python/testdata/type_checking_imports_across_packages/b/BUILD.in create mode 100644 gazelle/python/testdata/type_checking_imports_across_packages/b/BUILD.out create mode 100644 gazelle/python/testdata/type_checking_imports_across_packages/b/b.py create mode 100644 gazelle/python/testdata/type_checking_imports_across_packages/test.yaml diff --git a/gazelle/python/resolve.go b/gazelle/python/resolve.go index 88275e007c..0dd80841d4 100644 --- a/gazelle/python/resolve.go +++ b/gazelle/python/resolve.go @@ -124,12 +124,16 @@ func (py *Resolver) Embeds(r *rule.Rule, from label.Label) []label.Label { } // addDependency adds a dependency to either the regular deps or pyiDeps set based on -// whether the module is type-checking only. -func addDependency(dep string, mod Module, deps, pyiDeps *treeset.Set) { - if mod.TypeCheckingOnly { - pyiDeps.Add(dep) +// whether the module is type-checking only. If a module is added as both +// non-type-checking and type-checking, it should end up in deps and not pyiDeps. +func addDependency(dep string, typeCheckingOnly bool, deps, pyiDeps *treeset.Set) { + if typeCheckingOnly { + if !deps.Contains(dep) { + pyiDeps.Add(dep) + } } else { deps.Add(dep) + pyiDeps.Remove(dep) } } @@ -240,7 +244,7 @@ func (py *Resolver) Resolve( override.Repo = "" } dep := override.Rel(from.Repo, from.Pkg).String() - addDependency(dep, mod, deps, pyiDeps) + addDependency(dep, mod.TypeCheckingOnly, deps, pyiDeps) if explainDependency == dep { log.Printf("Explaining dependency (%s): "+ "in the target %q, the file %q imports %q at line %d, "+ @@ -251,7 +255,7 @@ func (py *Resolver) Resolve( } } else { if dep, distributionName, ok := cfg.FindThirdPartyDependency(moduleName); ok { - addDependency(dep, mod, deps, pyiDeps) + addDependency(dep, mod.TypeCheckingOnly, deps, pyiDeps) // Add the type and stub dependencies if they exist. modules := []string{ fmt.Sprintf("%s_stubs", strings.ToLower(distributionName)), @@ -261,8 +265,8 @@ func (py *Resolver) Resolve( } for _, module := range modules { if dep, _, ok := cfg.FindThirdPartyDependency(module); ok { - // Type stub packages always go to pyiDeps - pyiDeps.Add(dep) + // Type stub packages are added as type-checking only. + addDependency(dep, true, deps, pyiDeps) } } if explainDependency == dep { @@ -321,7 +325,7 @@ func (py *Resolver) Resolve( } matchLabel := filteredMatches[0].Label.Rel(from.Repo, from.Pkg) dep := matchLabel.String() - addDependency(dep, mod, deps, pyiDeps) + addDependency(dep, mod.TypeCheckingOnly, deps, pyiDeps) if explainDependency == dep { log.Printf("Explaining dependency (%s): "+ "in the target %q, the file %q imports %q at line %d, "+ diff --git a/gazelle/python/testdata/type_checking_imports_across_packages/BUILD.in b/gazelle/python/testdata/type_checking_imports_across_packages/BUILD.in new file mode 100644 index 0000000000..8e6c1cbabb --- /dev/null +++ b/gazelle/python/testdata/type_checking_imports_across_packages/BUILD.in @@ -0,0 +1,2 @@ +# gazelle:python_generation_mode package +# gazelle:python_generate_pyi_deps true diff --git a/gazelle/python/testdata/type_checking_imports_across_packages/BUILD.out b/gazelle/python/testdata/type_checking_imports_across_packages/BUILD.out new file mode 100644 index 0000000000..8e6c1cbabb --- /dev/null +++ b/gazelle/python/testdata/type_checking_imports_across_packages/BUILD.out @@ -0,0 +1,2 @@ +# gazelle:python_generation_mode package +# gazelle:python_generate_pyi_deps true diff --git a/gazelle/python/testdata/type_checking_imports_across_packages/README.md b/gazelle/python/testdata/type_checking_imports_across_packages/README.md new file mode 100644 index 0000000000..75fb3aae56 --- /dev/null +++ b/gazelle/python/testdata/type_checking_imports_across_packages/README.md @@ -0,0 +1,6 @@ +# Overlapping deps and pyi_deps across packages + +This test reproduces a case where a dependency may be added to both `deps` and +`pyi_deps`. Package `b` imports `a.foo` normally and imports `a.bar` as a +type-checking only import. The dependency on package `a` should appear only in +`deps` (and not `pyi_deps`) of package `b`. diff --git a/gazelle/python/testdata/type_checking_imports_across_packages/WORKSPACE b/gazelle/python/testdata/type_checking_imports_across_packages/WORKSPACE new file mode 100644 index 0000000000..3e6e74e7f4 --- /dev/null +++ b/gazelle/python/testdata/type_checking_imports_across_packages/WORKSPACE @@ -0,0 +1 @@ +workspace(name = "gazelle_python_test") diff --git a/gazelle/python/testdata/type_checking_imports_across_packages/a/BUILD.in b/gazelle/python/testdata/type_checking_imports_across_packages/a/BUILD.in new file mode 100644 index 0000000000..e69de29bb2 diff --git a/gazelle/python/testdata/type_checking_imports_across_packages/a/BUILD.out b/gazelle/python/testdata/type_checking_imports_across_packages/a/BUILD.out new file mode 100644 index 0000000000..cf9be008b1 --- /dev/null +++ b/gazelle/python/testdata/type_checking_imports_across_packages/a/BUILD.out @@ -0,0 +1,10 @@ +load("@rules_python//python:defs.bzl", "py_library") + +py_library( + name = "a", + srcs = [ + "bar.py", + "foo.py", + ], + visibility = ["//:__subpackages__"], +) diff --git a/gazelle/python/testdata/type_checking_imports_across_packages/a/bar.py b/gazelle/python/testdata/type_checking_imports_across_packages/a/bar.py new file mode 100644 index 0000000000..e69de29bb2 diff --git a/gazelle/python/testdata/type_checking_imports_across_packages/a/foo.py b/gazelle/python/testdata/type_checking_imports_across_packages/a/foo.py new file mode 100644 index 0000000000..e69de29bb2 diff --git a/gazelle/python/testdata/type_checking_imports_across_packages/b/BUILD.in b/gazelle/python/testdata/type_checking_imports_across_packages/b/BUILD.in new file mode 100644 index 0000000000..e69de29bb2 diff --git a/gazelle/python/testdata/type_checking_imports_across_packages/b/BUILD.out b/gazelle/python/testdata/type_checking_imports_across_packages/b/BUILD.out new file mode 100644 index 0000000000..15f4d343e1 --- /dev/null +++ b/gazelle/python/testdata/type_checking_imports_across_packages/b/BUILD.out @@ -0,0 +1,8 @@ +load("@rules_python//python:defs.bzl", "py_library") + +py_library( + name = "b", + srcs = ["b.py"], + visibility = ["//:__subpackages__"], + deps = ["//a"], +) diff --git a/gazelle/python/testdata/type_checking_imports_across_packages/b/b.py b/gazelle/python/testdata/type_checking_imports_across_packages/b/b.py new file mode 100644 index 0000000000..93d09c0baa --- /dev/null +++ b/gazelle/python/testdata/type_checking_imports_across_packages/b/b.py @@ -0,0 +1,6 @@ +from typing import TYPE_CHECKING + +from a import foo + +if TYPE_CHECKING: + from a import bar diff --git a/gazelle/python/testdata/type_checking_imports_across_packages/test.yaml b/gazelle/python/testdata/type_checking_imports_across_packages/test.yaml new file mode 100644 index 0000000000..ed97d539c0 --- /dev/null +++ b/gazelle/python/testdata/type_checking_imports_across_packages/test.yaml @@ -0,0 +1 @@ +--- From 581cddcad8b83f4b2855ffe09992d9b669ad2d37 Mon Sep 17 00:00:00 2001 From: Alex Martani Date: Mon, 30 Jun 2025 09:27:27 -0700 Subject: [PATCH 302/922] fix(gazelle) Register pyi_deps as ResolveAttrs (#3037) Fix an issue in https://github.com/bazel-contrib/rules_python/pull/3014 where, when all type-checking dependencies are removed from a file, the corresponding target's `pyi_deps` doesn't get cleaned up. I traced this back to `ResolveAttrs`, though I'm not entirely sure of what other behaviors this may trigger. (Currently, removing `deps` from `ResolveAttrs` doesn't break any existing test case) (No new changelog entry since this is a fix to an unreleased feature that is already in the changelog) --- gazelle/python/kinds.go | 3 +++ gazelle/python/testdata/clear_out_deps/BUILD.in | 1 + gazelle/python/testdata/clear_out_deps/BUILD.out | 1 + gazelle/python/testdata/clear_out_deps/README.md | 9 +++++++++ gazelle/python/testdata/clear_out_deps/WORKSPACE | 1 + gazelle/python/testdata/clear_out_deps/a/BUILD.in | 9 +++++++++ gazelle/python/testdata/clear_out_deps/a/BUILD.out | 7 +++++++ gazelle/python/testdata/clear_out_deps/a/__init__.py | 0 gazelle/python/testdata/clear_out_deps/b/BUILD.in | 9 +++++++++ gazelle/python/testdata/clear_out_deps/b/BUILD.out | 7 +++++++ gazelle/python/testdata/clear_out_deps/b/__init__.py | 0 gazelle/python/testdata/clear_out_deps/c/BUILD.in | 9 +++++++++ gazelle/python/testdata/clear_out_deps/c/BUILD.out | 9 +++++++++ gazelle/python/testdata/clear_out_deps/c/__init__.py | 6 ++++++ gazelle/python/testdata/clear_out_deps/test.yaml | 2 ++ 15 files changed, 73 insertions(+) create mode 100644 gazelle/python/testdata/clear_out_deps/BUILD.in create mode 100644 gazelle/python/testdata/clear_out_deps/BUILD.out create mode 100644 gazelle/python/testdata/clear_out_deps/README.md create mode 100644 gazelle/python/testdata/clear_out_deps/WORKSPACE create mode 100644 gazelle/python/testdata/clear_out_deps/a/BUILD.in create mode 100644 gazelle/python/testdata/clear_out_deps/a/BUILD.out create mode 100644 gazelle/python/testdata/clear_out_deps/a/__init__.py create mode 100644 gazelle/python/testdata/clear_out_deps/b/BUILD.in create mode 100644 gazelle/python/testdata/clear_out_deps/b/BUILD.out create mode 100644 gazelle/python/testdata/clear_out_deps/b/__init__.py create mode 100644 gazelle/python/testdata/clear_out_deps/c/BUILD.in create mode 100644 gazelle/python/testdata/clear_out_deps/c/BUILD.out create mode 100644 gazelle/python/testdata/clear_out_deps/c/__init__.py create mode 100644 gazelle/python/testdata/clear_out_deps/test.yaml diff --git a/gazelle/python/kinds.go b/gazelle/python/kinds.go index 7a0639abd3..ff3f6ce829 100644 --- a/gazelle/python/kinds.go +++ b/gazelle/python/kinds.go @@ -46,6 +46,7 @@ var pyKinds = map[string]rule.KindInfo{ }, ResolveAttrs: map[string]bool{ "deps": true, + "pyi_deps": true, }, }, pyLibraryKind: { @@ -62,6 +63,7 @@ var pyKinds = map[string]rule.KindInfo{ }, ResolveAttrs: map[string]bool{ "deps": true, + "pyi_deps": true, }, }, pyTestKind: { @@ -78,6 +80,7 @@ var pyKinds = map[string]rule.KindInfo{ }, ResolveAttrs: map[string]bool{ "deps": true, + "pyi_deps": true, }, }, } diff --git a/gazelle/python/testdata/clear_out_deps/BUILD.in b/gazelle/python/testdata/clear_out_deps/BUILD.in new file mode 100644 index 0000000000..99d122ad12 --- /dev/null +++ b/gazelle/python/testdata/clear_out_deps/BUILD.in @@ -0,0 +1 @@ +# gazelle:python_generate_pyi_deps true diff --git a/gazelle/python/testdata/clear_out_deps/BUILD.out b/gazelle/python/testdata/clear_out_deps/BUILD.out new file mode 100644 index 0000000000..99d122ad12 --- /dev/null +++ b/gazelle/python/testdata/clear_out_deps/BUILD.out @@ -0,0 +1 @@ +# gazelle:python_generate_pyi_deps true diff --git a/gazelle/python/testdata/clear_out_deps/README.md b/gazelle/python/testdata/clear_out_deps/README.md new file mode 100644 index 0000000000..53b62a46d5 --- /dev/null +++ b/gazelle/python/testdata/clear_out_deps/README.md @@ -0,0 +1,9 @@ +# Clearing deps / pyi_deps + +This test case asserts that an existing `py_library` specifying `deps` and +`pyi_deps` have these attributes removed if the corresponding imports are +removed. + +`a/BUILD.in` declares `deps`/`pyi_deps` on non-existing libraries, `b/BUILD.in` declares dependency on `//a` +without a matching import, and `c/BUILD.in` declares both `deps` and `pyi_deps` as `["//a", "//b"]`, but +it should have only `//a` as `deps` and only `//b` as `pyi_deps`. diff --git a/gazelle/python/testdata/clear_out_deps/WORKSPACE b/gazelle/python/testdata/clear_out_deps/WORKSPACE new file mode 100644 index 0000000000..faff6af87a --- /dev/null +++ b/gazelle/python/testdata/clear_out_deps/WORKSPACE @@ -0,0 +1 @@ +# This is a Bazel workspace for the Gazelle test data. diff --git a/gazelle/python/testdata/clear_out_deps/a/BUILD.in b/gazelle/python/testdata/clear_out_deps/a/BUILD.in new file mode 100644 index 0000000000..832683b22a --- /dev/null +++ b/gazelle/python/testdata/clear_out_deps/a/BUILD.in @@ -0,0 +1,9 @@ +load("@rules_python//python:defs.bzl", "py_library") + +py_library( + name = "a", + srcs = ["__init__.py"], + pyi_deps = ["//:nonexistent_pyi_dep"], + visibility = ["//:__subpackages__"], + deps = ["//nonexistent_dep"], +) diff --git a/gazelle/python/testdata/clear_out_deps/a/BUILD.out b/gazelle/python/testdata/clear_out_deps/a/BUILD.out new file mode 100644 index 0000000000..2668e97c42 --- /dev/null +++ b/gazelle/python/testdata/clear_out_deps/a/BUILD.out @@ -0,0 +1,7 @@ +load("@rules_python//python:defs.bzl", "py_library") + +py_library( + name = "a", + srcs = ["__init__.py"], + visibility = ["//:__subpackages__"], +) diff --git a/gazelle/python/testdata/clear_out_deps/a/__init__.py b/gazelle/python/testdata/clear_out_deps/a/__init__.py new file mode 100644 index 0000000000..e69de29bb2 diff --git a/gazelle/python/testdata/clear_out_deps/b/BUILD.in b/gazelle/python/testdata/clear_out_deps/b/BUILD.in new file mode 100644 index 0000000000..14cce87498 --- /dev/null +++ b/gazelle/python/testdata/clear_out_deps/b/BUILD.in @@ -0,0 +1,9 @@ +load("@rules_python//python:defs.bzl", "py_library") + +py_library( + name = "b", + srcs = ["__init__.py"], + pyi_deps = ["//a"], + visibility = ["//:__subpackages__"], + deps = ["//a"], +) diff --git a/gazelle/python/testdata/clear_out_deps/b/BUILD.out b/gazelle/python/testdata/clear_out_deps/b/BUILD.out new file mode 100644 index 0000000000..7305850a2e --- /dev/null +++ b/gazelle/python/testdata/clear_out_deps/b/BUILD.out @@ -0,0 +1,7 @@ +load("@rules_python//python:defs.bzl", "py_library") + +py_library( + name = "b", + srcs = ["__init__.py"], + visibility = ["//:__subpackages__"], +) diff --git a/gazelle/python/testdata/clear_out_deps/b/__init__.py b/gazelle/python/testdata/clear_out_deps/b/__init__.py new file mode 100644 index 0000000000..e69de29bb2 diff --git a/gazelle/python/testdata/clear_out_deps/c/BUILD.in b/gazelle/python/testdata/clear_out_deps/c/BUILD.in new file mode 100644 index 0000000000..10ace67dd2 --- /dev/null +++ b/gazelle/python/testdata/clear_out_deps/c/BUILD.in @@ -0,0 +1,9 @@ +load("@rules_python//python:defs.bzl", "py_library") + +py_library( + name = "c", + srcs = ["__init__.py"], + pyi_deps = ["//a", "//b"], + visibility = ["//:__subpackages__"], + deps = ["//a", "//b"], +) diff --git a/gazelle/python/testdata/clear_out_deps/c/BUILD.out b/gazelle/python/testdata/clear_out_deps/c/BUILD.out new file mode 100644 index 0000000000..d1aa97e5aa --- /dev/null +++ b/gazelle/python/testdata/clear_out_deps/c/BUILD.out @@ -0,0 +1,9 @@ +load("@rules_python//python:defs.bzl", "py_library") + +py_library( + name = "c", + srcs = ["__init__.py"], + pyi_deps = ["//b"], + visibility = ["//:__subpackages__"], + deps = ["//a"], +) diff --git a/gazelle/python/testdata/clear_out_deps/c/__init__.py b/gazelle/python/testdata/clear_out_deps/c/__init__.py new file mode 100644 index 0000000000..32d017f28a --- /dev/null +++ b/gazelle/python/testdata/clear_out_deps/c/__init__.py @@ -0,0 +1,6 @@ +from typing import TYPE_CHECKING + +import a + +if TYPE_CHECKING: + import b diff --git a/gazelle/python/testdata/clear_out_deps/test.yaml b/gazelle/python/testdata/clear_out_deps/test.yaml new file mode 100644 index 0000000000..88a0cbf018 --- /dev/null +++ b/gazelle/python/testdata/clear_out_deps/test.yaml @@ -0,0 +1,2 @@ + +--- From cd6948a0f706e75fa0f3ebd35e485aeec3e299fc Mon Sep 17 00:00:00 2001 From: Jeff Klukas Date: Mon, 30 Jun 2025 13:47:43 -0400 Subject: [PATCH 303/922] docs: Typo in gazelle/README.md (#3040) --- gazelle/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/gazelle/README.md b/gazelle/README.md index 5c63e21762..3dc8e12a0a 100644 --- a/gazelle/README.md +++ b/gazelle/README.md @@ -24,7 +24,7 @@ The following documentation covers using bzlmod. ## Adding Gazelle to your project -First, you'll need to add Gazelle to your `MODULES.bazel` file. +First, you'll need to add Gazelle to your `MODULE.bazel` file. Get the current version of Gazelle from there releases here: https://github.com/bazelbuild/bazel-gazelle/releases/. From 83e8f4bc2d759efc6fe787148773dac813449651 Mon Sep 17 00:00:00 2001 From: yushan26 <107004874+yushan26@users.noreply.github.com> Date: Tue, 1 Jul 2025 11:15:58 -0700 Subject: [PATCH 304/922] feat(gazelle) Remove entry point file requirements when generating rules (#2998) Remove entry point file requirements when generating rules. Enable python rule generation as long as there are .py source files under the directory so all new packages will have python rules generated in the package. The extension used to require entrypoints for generation but: - entry point for tests (i.e., `__test__.py` ) is no longer required after https://github.com/bazel-contrib/rules_python/pull/999 and https://github.com/bazel-contrib/rules_python/pull/2044 - entry point for binaries (i.e., `__main__.py` ) is no longer required after https://github.com/bazel-contrib/rules_python/pull/1584 The entry point for libraries (`__init__.py` ) shouldn't be required either, especially for Python 3.3 and after when namespace packages are supported. --------- Co-authored-by: yushan Co-authored-by: Douglas Thor --- CHANGELOG.md | 2 ++ gazelle/python/generate.go | 7 +------ gazelle/python/testdata/subdir_sources/BUILD.in | 1 + gazelle/python/testdata/subdir_sources/BUILD.out | 3 +++ 4 files changed, 7 insertions(+), 6 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 8cb5ca3f9f..da59ecf8b5 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -58,6 +58,8 @@ END_UNRELEASED_TEMPLATE to the package path. This is enabled via the `# gazelle:experimental_allow_relative_imports` true directive ({gh-issue}`2203`). * (gazelle) Types for exposed members of `python.ParserOutput` are now all public. +* (gazelle) Removed the requirement for `__init__.py`, `__main__.py`, or `__test__.py` files to be + present in a directory to generate a `BUILD.bazel` file. {#v0-0-0-fixed} ### Fixed diff --git a/gazelle/python/generate.go b/gazelle/python/generate.go index 5eedbd9601..c1edec4731 100644 --- a/gazelle/python/generate.go +++ b/gazelle/python/generate.go @@ -85,8 +85,6 @@ func (py *Python) GenerateRules(args language.GenerateArgs) language.GenerateRes if parent != nil && parent.CoarseGrainedGeneration() { return language.GenerateResult{} } - } else if !hasEntrypointFile(args.Dir) { - return language.GenerateResult{} } } @@ -172,9 +170,6 @@ func (py *Python) GenerateRules(args language.GenerateArgs) language.GenerateRes // 2. The directory has a BUILD or BUILD.bazel files. Then // it doesn't matter at all what it has since it's a // separate Bazel package. - // 3. (only for package generation) The directory has an - // __init__.py, __main__.py or __test__.py, meaning a - // BUILD file will be generated. if cfg.PerFileGeneration() { return fs.SkipDir } @@ -184,7 +179,7 @@ func (py *Python) GenerateRules(args language.GenerateArgs) language.GenerateRes return nil } - if !cfg.CoarseGrainedGeneration() && hasEntrypointFile(path) { + if !cfg.CoarseGrainedGeneration() { return fs.SkipDir } diff --git a/gazelle/python/testdata/subdir_sources/BUILD.in b/gazelle/python/testdata/subdir_sources/BUILD.in index e69de29bb2..adfdefdc8a 100644 --- a/gazelle/python/testdata/subdir_sources/BUILD.in +++ b/gazelle/python/testdata/subdir_sources/BUILD.in @@ -0,0 +1 @@ +# gazelle:python_generation_mode project diff --git a/gazelle/python/testdata/subdir_sources/BUILD.out b/gazelle/python/testdata/subdir_sources/BUILD.out index d03a8f05ac..5d77890d4f 100644 --- a/gazelle/python/testdata/subdir_sources/BUILD.out +++ b/gazelle/python/testdata/subdir_sources/BUILD.out @@ -1,5 +1,8 @@ + load("@rules_python//python:defs.bzl", "py_binary") +# gazelle:python_generation_mode project + py_binary( name = "subdir_sources_bin", srcs = ["__main__.py"], From 4e22d2560b3bd4c0cea9ad0880d1ff08df110456 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Robin=20Lind=C3=A9n?= <_@robinlinden.eu> Date: Wed, 2 Jul 2025 09:26:11 +0200 Subject: [PATCH 305/922] fix: Don't let deprecated test targets get matched by '...' (#3045) This fixes "target '//foo_test' is deprecated: Use 'foo.test' instead. The '*_test' target will be removed in the next major release." being warned about once per `compile_pip_requirement` call when running `bazel test ...`. Work towards #2976 --- python/private/pypi/pip_compile.bzl | 1 + 1 file changed, 1 insertion(+) diff --git a/python/private/pypi/pip_compile.bzl b/python/private/pypi/pip_compile.bzl index 78b681b4ad..2e3e530153 100644 --- a/python/private/pypi/pip_compile.bzl +++ b/python/private/pypi/pip_compile.bzl @@ -196,4 +196,5 @@ def pip_compile( name = "{}_test".format(name), actual = ":{}.test".format(name), deprecation = "Use '{}.test' instead. The '*_test' target will be removed in the next major release.".format(name), + tags = ["manual"], ) From cbe6d38d01c14de46d90ea717d0f2090117533fa Mon Sep 17 00:00:00 2001 From: Aaron Sky Date: Wed, 2 Jul 2025 19:51:33 -0400 Subject: [PATCH 306/922] fix: add py.typed to runfiles py_wheel so it gets packaged (#3041) Per the guidance in #2503, this is a quick fix just to restore type-checking for the runfiles package. It does not address or investigate further whether py.typed data dependencies in direct `py_library` dependencies of `py_wheel` should be automatically included as inputs to the wheel. Fixes #2503 --------- Co-authored-by: Richard Levasseur --- CHANGELOG.md | 3 +++ python/runfiles/BUILD.bazel | 13 +++++++++++-- 2 files changed, 14 insertions(+), 2 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index da59ecf8b5..7b2dfc3908 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -69,6 +69,9 @@ END_UNRELEASED_TEMPLATE * (pypi) Wheels with BUILD.bazel (or other special Bazel files) no longer result in missing files at runtime ([#2782](https://github.com/bazel-contrib/rules_python/issues/2782)). +* (runfiles) The pypi runfiles package now includes `py.typed` to indicate it + supports type checking + ([#2503](https://github.com/bazel-contrib/rules_python/issues/2503)). {#v0-0-0-added} ### Added diff --git a/python/runfiles/BUILD.bazel b/python/runfiles/BUILD.bazel index 2040403b10..73663472dc 100644 --- a/python/runfiles/BUILD.bazel +++ b/python/runfiles/BUILD.bazel @@ -22,13 +22,19 @@ filegroup( visibility = ["//python:__pkg__"], ) +filegroup( + name = "py_typed", + # See PEP 561: py.typed is a special file that indicates the code supports type checking + srcs = ["py.typed"], +) + py_library( name = "runfiles", srcs = [ "__init__.py", "runfiles.py", ], - data = ["py.typed"], + data = [":py_typed"], imports = [ # Add the repo root so `import python.runfiles.runfiles` works. This makes it agnostic # to the --experimental_python_import_all_repositories setting. @@ -57,5 +63,8 @@ py_wheel( # this can be replaced by building with --stamp --embed_label=1.2.3 version = "{BUILD_EMBED_LABEL}", visibility = ["//visibility:public"], - deps = [":runfiles"], + deps = [ + ":py_typed", + ":runfiles", + ], ) From d2c7ba2669b448c16876cee66f933c9e0da533cc Mon Sep 17 00:00:00 2001 From: Ted Kaplan Date: Thu, 3 Jul 2025 12:41:11 -0700 Subject: [PATCH 307/922] docs: Add note about Python 3.9 to CHANGELOG.md (#3052) rules_python 1.5.0 upgraded its internal setuputils to 78.1.1 which has a minimum supported Python version of 3.9. Using this version with Python 3.8 leads to errors (see below) although for some reason, I only see them on Linux builds, not Mac. Since Python 3.8 is EOL, document that Python 3.8 will no longer work due to this setuptools version. Fixes #3050 --------- Co-authored-by: Richard Levasseur --- CHANGELOG.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 7b2dfc3908..ea76c5a6d3 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -109,7 +109,8 @@ END_UNRELEASED_TEMPLATE * (py_wheel) py_wheel always creates zip64-capable wheel zips * (providers) (experimental) {obj}`PyInfo.venv_symlinks` replaces `PyInfo.site_packages_symlinks` -* (deps) Updating setuptools to patch CVE-2025-47273. +* (deps) Updated setuptools to 78.1.1 to patch CVE-2025-47273. This effectively makes + Python 3.9 the minimum supported version for using `pip_parse`. {#1-5-0-fixed} ### Fixed From b0671ed548bbc77152d2ed502b87435aeb3b3f6e Mon Sep 17 00:00:00 2001 From: Aaron Levy Date: Thu, 3 Jul 2025 17:51:18 -0700 Subject: [PATCH 308/922] fix: Updating Python toolchains to patch CVE-2025-47273 (#3053) Updating to a slightly newer build (20250612 instead of 20250610) of several Python toolchains that includes a newer version of setuptools that is no longer vulnerable to CVE-2025-47273. Also added support for Python 3.13.5, since a patched toolchain build for 3.13.4 is not available (and since 3.13.5 was released). See https://github.com/astral-sh/python-build-standalone/commit/5cc924bf04b73004c57bf438476255c1b5b63e9f#diff-860ea5e06ac2e2191008bbf2de9b216ed11533780f270b5e0cdfc31b37b3b3df and https://github.com/astral-sh/python-build-standalone/releases/tag/20250612 --- CHANGELOG.md | 7 ++ python/versions.bzl | 159 +++++++++++++++++++++------------- tests/python/python_tests.bzl | 2 +- 3 files changed, 108 insertions(+), 60 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index ea76c5a6d3..d8dda48f88 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -60,6 +60,13 @@ END_UNRELEASED_TEMPLATE * (gazelle) Types for exposed members of `python.ParserOutput` are now all public. * (gazelle) Removed the requirement for `__init__.py`, `__main__.py`, or `__test__.py` files to be present in a directory to generate a `BUILD.bazel` file. +* (toolchain) Updated the following toolchains to build 20250612 to patch CVE-2025-47273: + * 3.9.23 + * 3.10.18 + * 3.11.13 + * 3.12.11 + * 3.14.0b2 +* (toolchain) Python 3.13 now references 3.13.5 {#v0-0-0-fixed} ### Fixed diff --git a/python/versions.bzl b/python/versions.bzl index 44af7baf69..72ff7c2253 100644 --- a/python/versions.bzl +++ b/python/versions.bzl @@ -187,17 +187,17 @@ TOOL_VERSIONS = { "strip_prefix": "python", }, "3.9.23": { - "url": "20250610/cpython-{python_version}+20250610-{platform}-{build}.tar.gz", + "url": "20250612/cpython-{python_version}+20250612-{platform}-{build}.tar.gz", "sha256": { - "aarch64-apple-darwin": "f1a60528b6088ee8b8a34ca0e960998f4f664bed300ec0bbfe9d66ccbda74e50", - "aarch64-unknown-linux-gnu": "2871cf240bce3c021de829d73da04026febd7a775d1a1a1b37603ec6419fb6c1", - "ppc64le-unknown-linux-gnu": "2ba44a8e084a4661dbe50c0f0e3cf0a57227c6f1cff13fc2ae2f4d8ceae699fc", - "riscv64-unknown-linux-gnu": "7a735aebfc8b19a8af1f03e28babaf18a46cf8db0a931343dac1269376a1f693", - "s390x-unknown-linux-gnu": "27cfc030f782e2683c664e41dcef36051467c98676e133cbef04d4b7155ac4aa", - "x86_64-apple-darwin": "debd576badb6fdabb793ec9956512102f5a813c837449b1fe007c0af977db36c", - "x86_64-pc-windows-msvc": "28fbf2026929e00a300466220917c7029a69331700badb34b1691f1a99aa38e3", - "x86_64-unknown-linux-gnu": "21440e51aee78f3d92faf9375a90713542d8332e83d94c284f8f3d52c58eb5ca", - "x86_64-unknown-linux-musl": "7a881405a41cb4edf8c0d7c469c2f4759f601bc6f3c47978424a1ab1d0f1fada", + "aarch64-apple-darwin": "75c2bcc055088e9d20109910c82960bfe4ec5c1ea481e2176002aad4d7049eab", + "aarch64-unknown-linux-gnu": "1925b9aa73cd11633daa01756e32f9c319340c25e5338b151477691e8d99494b", + "ppc64le-unknown-linux-gnu": "bf0ebbf8842aff64955ec2d9c8bdc4fef266ffd2a92cff13d2c761e7a0039331", + "riscv64-unknown-linux-gnu": "a1623c1a3f4a91e4e022c08a8efb2177195bcdfcf715e1eb1612930324c68e3f", + "s390x-unknown-linux-gnu": "39806ac64f2375e1b6e4b0f378d01add441f1d81953629f828224a9b874a640a", + "x86_64-apple-darwin": "6565c263f28ae466f1b81cb902ac002bfcad7b1b04863e3576baa6c968dbf83a", + "x86_64-pc-windows-msvc": "42a80636326ca998fadb8840de4cb50716f6df63f815a8e71a4c922d3d6c00d0", + "x86_64-unknown-linux-gnu": "110ddaca41601b431041db6b4778584f671ca109ca25ef19fe32796026678358", + "x86_64-unknown-linux-musl": "c3bdcc5ce8ee357d856b22f6aa72da3126dd400ac9a643e5df91625376efc23a", }, "strip_prefix": "python", }, @@ -337,17 +337,17 @@ TOOL_VERSIONS = { "strip_prefix": "python", }, "3.10.18": { - "url": "20250610/cpython-{python_version}+20250610-{platform}-{build}.tar.gz", + "url": "20250612/cpython-{python_version}+20250612-{platform}-{build}.tar.gz", "sha256": { - "aarch64-apple-darwin": "a6590f71f670c7d121ac4f068dc83e271cf03309b80b1fa5890ee4875b7b691d", - "aarch64-unknown-linux-gnu": "b4d7cfb2cb5163da1ae5955ae8b33ac0b356780483d2993099899cf59efaea70", - "ppc64le-unknown-linux-gnu": "36aeae5cc61ff07c78b061f1b6aac628998a380ad45fadc82b8764185544fd7f", - "riscv64-unknown-linux-gnu": "2f6dd270598b655db5da5d98d1c43e560f6fb46c67a8fd68ff9b11ee9f6d79ff", - "s390x-unknown-linux-gnu": "616e56fe69c97a1d0ff13c00f337b2a91c972323c5d9a1828fdfc4d764b440fa", - "x86_64-apple-darwin": "4d72c1c1dcd2c4fe80055ef1b24fe4146f2de938aea1e3676faf91476f3f17e8", - "x86_64-pc-windows-msvc": "867b6dbcdb71d8ebb709ff54fbca8ad43d05cc21e5c157f39745c4dc44c1f8e2", - "x86_64-unknown-linux-gnu": "58f88ed6117078fdbc98976c9bc83b918f1f9c0c2ec21b80a582104f4839861c", - "x86_64-unknown-linux-musl": "d782c0569d6d7e21a5ed195ad7b41d0af8456b031e0814714d18cdeaa876f262", + "aarch64-apple-darwin": "ff6c9dd7172f82064f8d39fd4cd5d6bec77895ccffe480d846ff4a9750d14093", + "aarch64-unknown-linux-gnu": "11cc65da5cb3a469bc67b6f91bac5ec00d2070394f462ef8867a4db8d0fc6903", + "ppc64le-unknown-linux-gnu": "9fa6a75eb527016b0731faf2c9238dc4958ba85c41806f4c89efa6e12608cf86", + "riscv64-unknown-linux-gnu": "723a026f2184b4785a55da22b52ed0c0612f938c28ac6400b314b61e1daf10de", + "s390x-unknown-linux-gnu": "c43782f3efe25e0a0c62376643bd1bcdbde05c988aa86cc497df8031d619364a", + "x86_64-apple-darwin": "92ecfbfb89e8137cc88cabc2f408d00758d67454d07c1691706d3dcccc8fc446", + "x86_64-pc-windows-msvc": "d26dba4ec86f49ecbc6800e55f72691b9873115fa7c00f254f28dc04a03e8c13", + "x86_64-unknown-linux-gnu": "c28f5698033f3ba47f0c0f054fcf6b9134ff5082b478663c7c7c25bb7e0c4422", + "x86_64-unknown-linux-musl": "1b5c269a5eb04681e475aec673b1783e5f939f37dce305cd2e96eb0df186e9a2", }, "strip_prefix": "python", }, @@ -467,17 +467,17 @@ TOOL_VERSIONS = { "strip_prefix": "python", }, "3.11.13": { - "url": "20250610/cpython-{python_version}+20250610-{platform}-{build}.tar.gz", + "url": "20250612/cpython-{python_version}+20250612-{platform}-{build}.tar.gz", "sha256": { - "aarch64-apple-darwin": "365037494ba4f53563c22292e49a8e4d0d495bcb6534fca9666bdd1b474abf36", - "aarch64-unknown-linux-gnu": "a5954f147e87d9bff3d9733ebb3e74fe997eec5b38eaf5cb4429038228962a16", - "ppc64le-unknown-linux-gnu": "9214126866418f290fda88832fa3e244630f918ebc8a4a9ee15ba922e9c98afd", - "riscv64-unknown-linux-gnu": "fd99008c3123f50ec2ad407c5c1e17c1a86590daaf88dae8e6f1fd28f099b7c2", - "s390x-unknown-linux-gnu": "e27ab1fff8bf9e507677252a03ed524c685a8629b56475e26ab6dd0f88465179", - "x86_64-apple-darwin": "b49044115a545e67d73f5265a613a25da7c9523431281aa7b94691f1013355af", - "x86_64-pc-windows-msvc": "c0f89e3776211147817d54084fa046e2603571e18ff2ae4a4a8ff84ca4f7defc", - "x86_64-unknown-linux-gnu": "d93a7699505ee0ac7dec0f09324ffb19a31cce3066a287bb1fe95285ce3ea0c7", - "x86_64-unknown-linux-musl": "499121bb917e5baeeb954f76bdbce36bb63af579ff1530966ae2280e8d812c5b", + "aarch64-apple-darwin": "e272f0baca8f5a3cef29cc9c7418b80d0316553062ad3235205a33992155043c", + "aarch64-unknown-linux-gnu": "c6959d0c17fc221a9acc56e4827f3fe7386b610402055950e4b767b3b6871a40", + "ppc64le-unknown-linux-gnu": "22ab07e9bd167e2a7852a7b11b31cd91d090f3658e2ffc5bc6428751942cb1b9", + "riscv64-unknown-linux-gnu": "4ca57a3e139cf47803909a88f4f3940d9ecfde42d8089a11f42074859bc9a122", + "s390x-unknown-linux-gnu": "23cbd87fe9549ddda635ba9fb36b3622b5c939a10a39b25cd8c2587bb65e62ef", + "x86_64-apple-darwin": "e2a3e2434ba140615f01ed9328e063076c8282a38c11cab983bdcd5d1bd582da", + "x86_64-pc-windows-msvc": "cc28397fa47d28b98e1dc880b98cb061b76c88116b1d6028e04443f7221b30da", + "x86_64-unknown-linux-gnu": "4dd2c710a828c8cfff384e0549141016a563a5e153d2819a7225ccc05a1a17c7", + "x86_64-unknown-linux-musl": "130c6b55b06c92b7f952271fabedcdcfc06ac4717c133e0985ba27f799ed76b6", }, "strip_prefix": "python", }, @@ -590,17 +590,17 @@ TOOL_VERSIONS = { "strip_prefix": "python", }, "3.12.11": { - "url": "20250610/cpython-{python_version}+20250610-{platform}-{build}.tar.gz", + "url": "20250612/cpython-{python_version}+20250612-{platform}-{build}.tar.gz", "sha256": { - "aarch64-apple-darwin": "9c5826a93ddc15e8aa08de1e6e65b3ae0d45ea8eb0c2e9547b80ff4121b870ce", - "aarch64-unknown-linux-gnu": "eb33bc5a87443daf2fd218109df811bc4e4ea5ef9aec4fad75aa55da0258b96f", - "ppc64le-unknown-linux-gnu": "7b90bc528c5ddf30579dec52926d68fa6d5c90b65e24fc185d5fe283fdf0cbd9", - "riscv64-unknown-linux-gnu": "0f3103675102e351762a8fe574eae20335552a246a45a006d2a9ca14ce0952f8", - "s390x-unknown-linux-gnu": "a7ff0432208450ccebd5d328f69b84cc7c25b4af54fbab44803ddb11a2da5028", - "x86_64-apple-darwin": "199631baa35f3747ddfa2f1e28fc062b97ccd15b94a60c9294d4d129a73c9e53", - "x86_64-pc-windows-msvc": "e05fa165841c416d60365ca2216cad570f05ae5d3d027b9ad3beaad0529dd8cc", - "x86_64-unknown-linux-gnu": "77ab3efe5c6637fe8da0fdfbff5de1730c3b824874fe1368917886908b4c517b", - "x86_64-unknown-linux-musl": "9dd768494c4a34abcec316bc4802e957db98ed283024b527c0c40dfefd08b6fe", + "aarch64-apple-darwin": "c6d4843e8af496f034176908ae3384556680284653a4bff45eff07e43fe4ae34", + "aarch64-unknown-linux-gnu": "19e8d91b8c5cdb41c485e0d7daa726db6dd64c9a459029f738d5e55ad8da7c6f", + "ppc64le-unknown-linux-gnu": "32f489b4142ced7a3b476e25ac91ada4dc8aada1e771718a3aa9a0c818500a45", + "riscv64-unknown-linux-gnu": "0c1a3e976a117bf40ce8d75ad4806166e503d554263a9051f7606dbeb01d91ee", + "s390x-unknown-linux-gnu": "ee1a8451aaf49af330884553e2850961539b0563404c26241265ab0f0c929001", + "x86_64-apple-darwin": "7e3468bde68650fb8f63b663a24c56d0bb3353abd16158939b1de0ad60dab195", + "x86_64-pc-windows-msvc": "7b93afa91931dbc37b307a81b8680b30193736b5ef29a44ef6452f702c306e7a", + "x86_64-unknown-linux-gnu": "8e8bb0dbc815fb0b3912e0d8fc0a4f4aaac002bfc1f6cb0fcd278f2888f11bcf", + "x86_64-unknown-linux-musl": "b7464442265092259ee5f2e258c09cace4958f6b8733cff5e32bf8d2d6556a2a", }, "strip_prefix": "python", }, @@ -760,26 +760,67 @@ TOOL_VERSIONS = { "x86_64-unknown-linux-gnu-freethreaded": "python/install", }, }, + "3.13.5": { + "url": "20250612/cpython-{python_version}+20250612-{platform}-{build}.{ext}", + "sha256": { + "aarch64-apple-darwin": "d7867270b8c7be69ec26a351afb6bf24802b1cd9818e8426bd69d439a619bf2d", + "aarch64-unknown-linux-gnu": "685971ded0af96d1685941243ae1853c70c482b6f858dd86818760776d9c3cb9", + "ppc64le-unknown-linux-gnu": "ee15fcf2b64034dba13127aa37992edacf2efe1b2bb3d62ffd45eb9bea7b2d83", + "riscv64-unknown-linux-gnu": "c0f160ef9ab39c0f0e5baa00b1ecc3fff322c4ccbf1f04646c74559274ad5fc1", + "s390x-unknown-linux-gnu": "49131a3d16c13aea76f9ef5ce57fc612a3062fc866f6fcf971e0de8f8a9b8a8f", + "x86_64-apple-darwin": "d881b0226f1bef59b480c713126c54430a93ea21e5b39394c66927a412dd9907", + "x86_64-pc-windows-msvc": "8f4d4c7d270406be1f8f93b9fd2fd13951e4da274ba59d170f411a20cb1725b3", + "x86_64-unknown-linux-gnu": "f50dc28cfe99eccdadd4e74c2384607f7d5f50fc47447a39a4e24a793c07a9eb", + "x86_64-unknown-linux-musl": "c4bc1cda684320455d41e56980adbacbda269c78527f3ee926711d5d0ff33834", + "aarch64-apple-darwin-freethreaded": "a29cb4ef8adcd343e0f5bc5c4371cbc859fc7ce6d8f1a3c8d0cd7e44c4b9b866", + "aarch64-unknown-linux-gnu-freethreaded": "0ef13d13e16b4e58f167694940c6db54591db50bbc7ba61be6901ed5a69ad27b", + "ppc64le-unknown-linux-gnu-freethreaded": "66545ad4b09385750529ef09a665fc0b0ce698f984df106d7b167e3f7d59eace", + "riscv64-unknown-linux-gnu-freethreaded": "a82a741abefa7db61b2aeef36426bd56da5c69dc9dac105d68fba7fe658943ca", + "s390x-unknown-linux-gnu-freethreaded": "403c5758428013d5aa472841294c7b6ec91a572bb7123d02b7f1de24af4b0e13", + "x86_64-apple-darwin-freethreaded": "52aeb1b4073fa3f180d74a0712ceabc86dd2b40be499599e2e170948fb22acde", + "x86_64-pc-windows-msvc-freethreaded": "9da2f02d81597340163174ee91d91a8733dad2af53fc1b7c79ecc45a739a89d5", + "x86_64-unknown-linux-gnu-freethreaded": "33fdd6c42258cdf0402297d9e06842b53d9413d70849cee61755b9b5fb619836", + }, + "strip_prefix": { + "aarch64-apple-darwin": "python", + "aarch64-unknown-linux-gnu": "python", + "ppc64le-unknown-linux-gnu": "python", + "s390x-unknown-linux-gnu": "python", + "riscv64-unknown-linux-gnu": "python", + "x86_64-apple-darwin": "python", + "x86_64-pc-windows-msvc": "python", + "x86_64-unknown-linux-gnu": "python", + "x86_64-unknown-linux-musl": "python", + "aarch64-apple-darwin-freethreaded": "python/install", + "aarch64-unknown-linux-gnu-freethreaded": "python/install", + "ppc64le-unknown-linux-gnu-freethreaded": "python/install", + "riscv64-unknown-linux-gnu-freethreaded": "python/install", + "s390x-unknown-linux-gnu-freethreaded": "python/install", + "x86_64-apple-darwin-freethreaded": "python/install", + "x86_64-pc-windows-msvc-freethreaded": "python/install", + "x86_64-unknown-linux-gnu-freethreaded": "python/install", + }, + }, "3.14.0b2": { - "url": "20250610/cpython-{python_version}+20250610-{platform}-{build}.{ext}", + "url": "20250612/cpython-{python_version}+20250612-{platform}-{build}.{ext}", "sha256": { - "aarch64-apple-darwin": "6607351d140e83feb6e11dbde46ab5f99fa9fe039bdbaa12611d26bda0ed9343", - "aarch64-unknown-linux-gnu": "cc388d567f7c23921e0bef8dcae959dfab9ee24d10aeeb23688b21eac402817f", - "ppc64le-unknown-linux-gnu": "f9379ecc5dc71f9c58adf03d5524176ec36e1b40c788d29c260df54d09ad351c", - "riscv64-unknown-linux-gnu": "e6fbe4f7928ec606edee1506752659bf59216fdb208c744d268082ec79b16f42", - "s390x-unknown-linux-gnu": "1cf32c1173adc1cb70952bb47c92177a196f9e83b7a874f09599682e92ba0010", - "x86_64-apple-darwin": "a6d8196b174409e0ce67829c4e4ee5005c4be20a2efb41116e0521ad1fa1a717", - "x86_64-pc-windows-msvc": "0d88ec80c6c3e3ac462368850c19d3930bf2b1a1a5fe89da60c8534d0fac1a01", - "x86_64-unknown-linux-gnu": "93b29eea5214d19f0420ef8e459b007e15ea58349d60811122c78241fe51cb92", - "x86_64-unknown-linux-musl": "90e90a58ebff3416eb5a3f93ecb59b6eda945e2b706f5c13b0ba85f6b2bee130", - "aarch64-apple-darwin-freethreaded": "af0f34aa0dcd02bd3d960a1572a1ed8a17d55b373a22866f05041aaf16f8607d", - "aarch64-unknown-linux-gnu-freethreaded": "e76c7ab98e1c0f86a6996d1ec775ba8497bf46aa8ffa8c7b0f2e761f37305329", - "ppc64le-unknown-linux-gnu-freethreaded": "df2ae00827406e247f1aaaec76ffc7963b909c81075fc9940eee1ea9f753dd16", - "riscv64-unknown-linux-gnu-freethreaded": "09e347cb5f29e0eafd1eba73105ea9d853184b55fbaf4746cebec217430d6db5", - "s390x-unknown-linux-gnu-freethreaded": "f911605eee0eb7845a69acaf8bfb2e1811c76e9a5e3980d97fae93135df4b773", - "x86_64-apple-darwin-freethreaded": "dd27d519cf2a04917cb566366d6539477791d1b2f1fb42037d9179f469ff55a9", - "x86_64-pc-windows-msvc-freethreaded": "da966a17e434094d8f10b719d93c782d82eaf5207f2843cbaa58c3d91a8f0e32", - "x86_64-unknown-linux-gnu-freethreaded": "abd60d3a302e9d9c32ec78581fb3a9903079c56ec7a949ce658a7950423f350a", + "aarch64-apple-darwin": "35c02e465af605eafd29d5931daadce724eeb8a3e7cc7156ac046991cb24f1c1", + "aarch64-unknown-linux-gnu": "8c877a1b50eb2a9b34ddac5d52d50867f11ddc817f257eba4cbbc999a9edf2ea", + "ppc64le-unknown-linux-gnu": "735bad9359eb36b55b76d9c6db122fe4357951d7850324c76e168055ca70e0a0", + "riscv64-unknown-linux-gnu": "d4140196c052ba5832a439f84f6ca5b136bb16bceb8c5a52f5167a2c3f8b73b1", + "s390x-unknown-linux-gnu": "2f440257e02d0a4fb4e93fcbb95b9066ec42bd56a2f03de05f55636e5afcb4b9", + "x86_64-apple-darwin": "5144890b991e63fb73e2714c162c901c3b6f289ae0ef742df3673ab9824c844a", + "x86_64-pc-windows-msvc": "903cfb0ae1766a572dcf62835ef24d3250a512974dcf785738ac0d6c06c9db5b", + "x86_64-unknown-linux-gnu": "1c73b90a8febbd36fc973d7361a1be562e88437d95570721b701f03e59835600", + "x86_64-unknown-linux-musl": "9cdd3983abfca2151661c25cb0fae50a30c8961e07d07ba643edab5be277ae09", + "aarch64-apple-darwin-freethreaded": "1ae31adfed2a8425f08a945869d3bfd910e97acd150465de257d3ae3da37dc7c", + "aarch64-unknown-linux-gnu-freethreaded": "f5fcf5e8310244ccd346aab2abdc2650ffb900a429cfb732c4884e238cba1782", + "ppc64le-unknown-linux-gnu-freethreaded": "c1177510c359494b6a70601d9c810cdfc662f834c1d686abd487eb89d7a577ef", + "riscv64-unknown-linux-gnu-freethreaded": "cb0f2d86b20f47c70a9c8647b01a35ab7d53cbcbde9ab89ffc8aacafb36cc2e4", + "s390x-unknown-linux-gnu-freethreaded": "f38f126b31a55f37829ee581979214a6d2ac8a985ed7915b42c99d52af329d9f", + "x86_64-apple-darwin-freethreaded": "4e022b8b7a1b2986aa5780fae34b5a89a1ac5ed11bea0c3349e674a6cb7e31c1", + "x86_64-pc-windows-msvc-freethreaded": "35abc125304ec81a7be0d7ac54f515e7addd7dcba912882210d37720eaab1d7e", + "x86_64-unknown-linux-gnu-freethreaded": "61383d43f639533a5105abad376bc497cc94dde8a1ed294f523d534c8cd99a8e", }, "strip_prefix": { "aarch64-apple-darwin": "python", @@ -810,7 +851,7 @@ MINOR_MAPPING = { "3.10": "3.10.18", "3.11": "3.11.13", "3.12": "3.12.11", - "3.13": "3.13.4", + "3.13": "3.13.5", "3.14": "3.14.0b2", } diff --git a/tests/python/python_tests.bzl b/tests/python/python_tests.bzl index f0dc4825ac..106cff27bb 100644 --- a/tests/python/python_tests.bzl +++ b/tests/python/python_tests.bzl @@ -325,7 +325,7 @@ def _test_toolchain_ordering(env): "3.10": "3.10.18", "3.11": "3.11.13", "3.12": "3.12.11", - "3.13": "3.13.4", + "3.13": "3.13.5", "3.14": "3.14.0b2", "3.8": "3.8.20", "3.9": "3.9.23", From 5af778abe3b1078de4c35f226f56ad3ca1f1f18e Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Thu, 3 Jul 2025 17:53:11 -0700 Subject: [PATCH 309/922] docs: doc expectations of ai-assisted contributions (#3051) A lot of this should go without saying, but I want to have a written reference we can refer to and so it's clear to potential contributors. The two basic points it makes is that AI-assisted contributions are allowed, but they're treated no different than regular contributions, so all the usual expectations apply. --------- Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com> --- CONTRIBUTING.md | 19 +++++++++++++++++++ 1 file changed, 19 insertions(+) diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 324801cfc3..8f985c551b 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -318,6 +318,25 @@ Not breaking changes: * Changing internal details, such as renaming an internal file. * Changing a rule to a macro. +## AI-assisted Contributions + +Contributions assisted by AI tools are allowed. However, the human author +submitting the pull request is responsible for the contributed code as if they +had written it entirely themselves. This means: + +* **Understanding the code:** You must be able to explain what the code does + and why it's implemented that way. This includes discussing its + implications, and any trade-offs made during its development, just as if you + had written it entirely yourself. +* **Vetting the correctness and functionality:** You are responsible for + thoroughly testing and verifying that the code is correct, functional, and + meets all project requirements and standards. + +If the human PR author cannot fulfill these responsibilities, the `rules_python` +maintainers will not spend time reviewing or merging the PR. The goal is to +ensure that all contributions, regardless of their origin, maintain the quality +and integrity of the project and do not place an undue burden on maintainers. + ## FAQ ### Installation errors when during `git commit` From be55942a16b49fbafa63d0e26ab445c0dd5ca2ca Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Thu, 3 Jul 2025 18:02:17 -0700 Subject: [PATCH 310/922] fix(local-toolchains): don't watch non-existent include directory (#3048) Apparently, Macs can mis-report their include directory. Since includes are only needed if C extensions are built, skip watching the directory if it doesn't exist. Work around for https://github.com/bazel-contrib/rules_python/issues/3043 --------- Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com> Co-authored-by: Ignas Anikevicius <240938+aignas@users.noreply.github.com> --- CHANGELOG.md | 3 +++ python/private/local_runtime_repo.bzl | 10 +++++++++- 2 files changed, 12 insertions(+), 1 deletion(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index d8dda48f88..da22192d2b 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -79,6 +79,9 @@ END_UNRELEASED_TEMPLATE * (runfiles) The pypi runfiles package now includes `py.typed` to indicate it supports type checking ([#2503](https://github.com/bazel-contrib/rules_python/issues/2503)). +* (toolchains) `local_runtime_repo` now checks if the include directory exists + before attempting to watch it, fixing issues on macOS with system Python + ({gh-issue}`3043`). {#v0-0-0-added} ### Added diff --git a/python/private/local_runtime_repo.bzl b/python/private/local_runtime_repo.bzl index ec0643e497..3b4b4c020d 100644 --- a/python/private/local_runtime_repo.bzl +++ b/python/private/local_runtime_repo.bzl @@ -99,7 +99,15 @@ def _local_runtime_repo_impl(rctx): interpreter_path = info["base_executable"] # NOTE: Keep in sync with recursive glob in define_local_runtime_toolchain_impl - repo_utils.watch_tree(rctx, rctx.path(info["include"])) + include_path = rctx.path(info["include"]) + + # The reported include path may not exist, and watching a non-existant + # path is an error. Silently skip, since includes are only necessary + # if C extensions are built. + if include_path.exists and include_path.is_dir: + repo_utils.watch_tree(rctx, include_path) + else: + pass # The cc_library.includes values have to be non-absolute paths, otherwise # the toolchain will give an error. Work around this error by making them From 2b5e6f54314d2110490724eb707436355b1938fc Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Fri, 4 Jul 2025 14:00:37 +0900 Subject: [PATCH 311/922] build(deps): bump urllib3 from 2.4.0 to 2.5.0 in /docs (#3042) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Bumps [urllib3](https://github.com/urllib3/urllib3) from 2.4.0 to 2.5.0.
Release notes

Sourced from urllib3's releases.

2.5.0

🚀 urllib3 is fundraising for HTTP/2 support

urllib3 is raising ~$40,000 USD to release HTTP/2 support and ensure long-term sustainable maintenance of the project after a sharp decline in financial support. If your company or organization uses Python and would benefit from HTTP/2 support in Requests, pip, cloud SDKs, and thousands of other projects please consider contributing financially to ensure HTTP/2 support is developed sustainably and maintained for the long-haul.

Thank you for your support.

Security issues

urllib3 2.5.0 fixes two moderate security issues:

  • Pool managers now properly control redirects when retries is passed — CVE-2025-50181 reported by @​sandumjacob (5.3 Medium, GHSA-pq67-6m6q-mj2v)
  • Redirects are now controlled by urllib3 in the Node.js runtime — CVE-2025-50182 (5.3 Medium, GHSA-48p4-8xcf-vxj5)

Features

  • Added support for the compression.zstd module that is new in Python 3.14. See PEP 784 for more information. (#3610)
  • Added support for version 0.5 of hatch-vcs (#3612)

Bugfixes

  • Raised exception for HTTPResponse.shutdown on a connection already released to the pool. (#3581)
  • Fixed incorrect CONNECT statement when using an IPv6 proxy with connection_from_host. Previously would not be wrapped in []. (#3615)
Changelog

Sourced from urllib3's changelog.

2.5.0 (2025-06-18)

Features

  • Added support for the compression.zstd module that is new in Python 3.14. See PEP 784 <https://peps.python.org/pep-0784/>_ for more information. ([#3610](https://github.com/urllib3/urllib3/issues/3610) <https://github.com/urllib3/urllib3/issues/3610>__)
  • Added support for version 0.5 of hatch-vcs ([#3612](https://github.com/urllib3/urllib3/issues/3612) <https://github.com/urllib3/urllib3/issues/3612>__)

Bugfixes

  • Fixed a security issue where restricting the maximum number of followed redirects at the urllib3.PoolManager level via the retries parameter did not work.
  • Made the Node.js runtime respect redirect parameters such as retries and redirects.
  • Raised exception for HTTPResponse.shutdown on a connection already released to the pool. ([#3581](https://github.com/urllib3/urllib3/issues/3581) <https://github.com/urllib3/urllib3/issues/3581>__)
  • Fixed incorrect CONNECT statement when using an IPv6 proxy with connection_from_host. Previously would not be wrapped in []. ([#3615](https://github.com/urllib3/urllib3/issues/3615) <https://github.com/urllib3/urllib3/issues/3615>__)
Commits
  • aaab4ec Release 2.5.0
  • 7eb4a2a Merge commit from fork
  • f05b132 Merge commit from fork
  • d03fe32 Fix HTTP tunneling with IPv6 in older Python versions
  • 11661e9 Bump github/codeql-action from 3.28.0 to 3.29.0 (#3624)
  • 6a0ecc6 Update v2 migration guide to 2.4.0 (#3621)
  • 8e32e60 Raise exception for shutdown on a connection already released to the pool (#3...
  • 9996e0f Fix emscripten CI for Chrome 137+ (#3599)
  • 4fd1a99 Bump RECENT_DATE (#3617)
  • c4b5917 Add support for the new compression.zstd module in Python 3.14 (#3611)
  • Additional commits viewable in compare view

[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=urllib3&package-manager=pip&previous-version=2.4.0&new-version=2.5.0)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) ---
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot show ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) You can disable automated security fix PRs for this repo from the [Security Alerts page](https://github.com/bazel-contrib/rules_python/network/alerts).
Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> --- docs/requirements.txt | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/requirements.txt b/docs/requirements.txt index cfeb0cbf31..d351e0e946 100644 --- a/docs/requirements.txt +++ b/docs/requirements.txt @@ -356,7 +356,7 @@ typing-extensions==4.13.2 \ # via # rules-python-docs (docs/pyproject.toml) # sphinx-autodoc2 -urllib3==2.4.0 \ - --hash=sha256:414bc6535b787febd7567804cc015fee39daab8ad86268f1310a9250697de466 \ - --hash=sha256:4e16665048960a0900c702d4a66415956a584919c03361cac9f1df5c5dd7e813 +urllib3==2.5.0 \ + --hash=sha256:3fc47733c7e419d4bc3f6b3dc2b4f890bb743906a30d56ba4a5bfa4bbff92760 \ + --hash=sha256:e6b01673c0fa6a13e374b50871808eb3bf7046c4b125b216f6bf1cc604cff0dc # via requests From 47c681b00a8ca75eb34053501f1119d4f6700a4d Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Sat, 5 Jul 2025 19:19:47 -0700 Subject: [PATCH 312/922] fix(pypi): only generate namespace package shims if implicit namespaces are disabled (#3059) The refactoring to move the pkgutil shim generation to build phase inverted the logic for when it should be activated. When `enable_implicit_namespace_pkgs=True`, it means to not generate the pkgutil shims ("respect the Python definition of the namespace package"). To fix, just invert the logic that activates it. A test will be added in a subsequent PR because the necessary helper isn't in the 1.5 branch. Fixes https://github.com/bazel-contrib/rules_python/issues/3038 --------- Co-authored-by: Ignas Anikevicius <240938+aignas@users.noreply.github.com> --- CHANGELOG.md | 12 ++++++++++++ python/private/pypi/whl_library_targets.bzl | 2 +- .../whl_library_targets_tests.bzl | 10 +++++++++- 3 files changed, 22 insertions(+), 2 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index da22192d2b..1822933c52 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -96,6 +96,18 @@ END_UNRELEASED_TEMPLATE ### Removed * Nothing removed. +{#1-5-1} +## [1.5.1] - 2025-07-06 + +[1.5.1]: https://github.com/bazel-contrib/rules_python/releases/tag/1.5.1 + +{#v1-5-1-fixed} +### Fixed + +* (pypi) Namespace packages work by default (pkgutil shims are generated + by default again) + ([#3038](https://github.com/bazel-contrib/rules_python/issues/3038)). + {#1-5-0} ## [1.5.0] - 2025-06-11 diff --git a/python/private/pypi/whl_library_targets.bzl b/python/private/pypi/whl_library_targets.bzl index 518d17163f..474f39a34d 100644 --- a/python/private/pypi/whl_library_targets.bzl +++ b/python/private/pypi/whl_library_targets.bzl @@ -331,7 +331,7 @@ def whl_library_targets( allow_empty = True, ) - if enable_implicit_namespace_pkgs: + if not enable_implicit_namespace_pkgs: srcs = srcs + getattr(native, "select", select)({ Label("//python/config_settings:is_venvs_site_packages"): [], "//conditions:default": create_inits( diff --git a/tests/pypi/whl_library_targets/whl_library_targets_tests.bzl b/tests/pypi/whl_library_targets/whl_library_targets_tests.bzl index f0e5f57ac0..22fe3ab7ca 100644 --- a/tests/pypi/whl_library_targets/whl_library_targets_tests.bzl +++ b/tests/pypi/whl_library_targets/whl_library_targets_tests.bzl @@ -16,10 +16,18 @@ load("@rules_testing//lib:test_suite.bzl", "test_suite") load("//python/private:glob_excludes.bzl", "glob_excludes") # buildifier: disable=bzl-visibility -load("//python/private/pypi:whl_library_targets.bzl", "whl_library_targets", "whl_library_targets_from_requires") # buildifier: disable=bzl-visibility +load("//python/private/pypi:whl_library_targets.bzl", _whl_library_targets = "whl_library_targets", _whl_library_targets_from_requires = "whl_library_targets_from_requires") # buildifier: disable=bzl-visibility _tests = [] +def whl_library_targets(**kwargs): + # Let's skip testing this for now + _whl_library_targets(enable_implicit_namespace_pkgs = True, **kwargs) + +def whl_library_targets_from_requires(**kwargs): + # Let's skip testing this for now + _whl_library_targets_from_requires(enable_implicit_namespace_pkgs = True, **kwargs) + def _test_filegroups(env): calls = [] From 29a7f6a0d5c8996f4d3f36e08269e0faf478927b Mon Sep 17 00:00:00 2001 From: Austin Schuh Date: Sun, 6 Jul 2025 00:22:08 -0700 Subject: [PATCH 313/922] feat: Add windows arm64 python toolchains (#3062) The changelog for astral-sh/python-build-standalone release 20250630 says: * Add ARM64 Windows builds for Python 3.11+ Lets use them! This is helpful when using rules_python on arm64 windows Work towards #2276 --------- Signed-off-by: Austin Schuh Co-authored-by: Ignas Anikevicius <240938+aignas@users.noreply.github.com> --- CHANGELOG.md | 9 +- python/versions.bzl | 175 +++++++++++++++++++--------------- tests/python/python_tests.bzl | 2 +- 3 files changed, 105 insertions(+), 81 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 1822933c52..81768af36a 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -60,12 +60,12 @@ END_UNRELEASED_TEMPLATE * (gazelle) Types for exposed members of `python.ParserOutput` are now all public. * (gazelle) Removed the requirement for `__init__.py`, `__main__.py`, or `__test__.py` files to be present in a directory to generate a `BUILD.bazel` file. -* (toolchain) Updated the following toolchains to build 20250612 to patch CVE-2025-47273: +* (toolchain) Updated the following toolchains to build 20250702 to patch CVE-2025-47273: * 3.9.23 * 3.10.18 * 3.11.13 * 3.12.11 - * 3.14.0b2 + * 3.14.0b3 * (toolchain) Python 3.13 now references 3.13.5 {#v0-0-0-fixed} @@ -91,6 +91,11 @@ END_UNRELEASED_TEMPLATE * (gazelle) New directive `gazelle:python_generate_pyi_deps`; when `true`, dependencies added to satisfy type-only imports (`if TYPE_CHECKING`) and type stub packages are added to `pyi_deps` instead of `deps`. +* (toolchain) Add toolchains for aarch64 windows for + * 3.11.13 + * 3.12.11 + * 3.13.5 + * 3.14.0b3 {#v0-0-0-removed} ### Removed diff --git a/python/versions.bzl b/python/versions.bzl index 72ff7c2253..50ddf2068e 100644 --- a/python/versions.bzl +++ b/python/versions.bzl @@ -187,17 +187,17 @@ TOOL_VERSIONS = { "strip_prefix": "python", }, "3.9.23": { - "url": "20250612/cpython-{python_version}+20250612-{platform}-{build}.tar.gz", + "url": "20250702/cpython-{python_version}+20250702-{platform}-{build}.tar.gz", "sha256": { - "aarch64-apple-darwin": "75c2bcc055088e9d20109910c82960bfe4ec5c1ea481e2176002aad4d7049eab", - "aarch64-unknown-linux-gnu": "1925b9aa73cd11633daa01756e32f9c319340c25e5338b151477691e8d99494b", - "ppc64le-unknown-linux-gnu": "bf0ebbf8842aff64955ec2d9c8bdc4fef266ffd2a92cff13d2c761e7a0039331", - "riscv64-unknown-linux-gnu": "a1623c1a3f4a91e4e022c08a8efb2177195bcdfcf715e1eb1612930324c68e3f", - "s390x-unknown-linux-gnu": "39806ac64f2375e1b6e4b0f378d01add441f1d81953629f828224a9b874a640a", - "x86_64-apple-darwin": "6565c263f28ae466f1b81cb902ac002bfcad7b1b04863e3576baa6c968dbf83a", - "x86_64-pc-windows-msvc": "42a80636326ca998fadb8840de4cb50716f6df63f815a8e71a4c922d3d6c00d0", - "x86_64-unknown-linux-gnu": "110ddaca41601b431041db6b4778584f671ca109ca25ef19fe32796026678358", - "x86_64-unknown-linux-musl": "c3bdcc5ce8ee357d856b22f6aa72da3126dd400ac9a643e5df91625376efc23a", + "aarch64-apple-darwin": "f9ce2f9f99a84108d3fde97c37b0cada6379b3f9d1d5ef1c8e940b9eaa811c18", + "aarch64-unknown-linux-gnu": "aa830b41391a2b57640636e9c172df8cf560777e0611fd098b2b5471c541a51e", + "ppc64le-unknown-linux-gnu": "97132753da44781c3a2fcd24503197844f4cce4ea0dd20290675f4020df377a0", + "riscv64-unknown-linux-gnu": "a6560df42a9afe6605cc578572b20cbf798c7fdf7381ef2dda0d3715124408d0", + "s390x-unknown-linux-gnu": "936e5e940a13c0189d29e4755ec20f10a70ba378dc9e739dc114d730a91a2ee5", + "x86_64-apple-darwin": "a82445abf3797bb699ce9f7371e3a6357ab3ec8fc6d25f36a88291b2cd495980", + "x86_64-pc-windows-msvc": "eb32d4fdd3c929ad9601f3fe9f944b038db430003bc5d5623db068da4edf7628", + "x86_64-unknown-linux-gnu": "c9bb5cb35f2c9fb05fbe9aec84d555f6d3c0773e07d42e74f92a27e866e15657", + "x86_64-unknown-linux-musl": "7d1dbd48c8e558555c4aad0d367831ca257edd625688d1d902d6f72f02c224f9", }, "strip_prefix": "python", }, @@ -337,17 +337,17 @@ TOOL_VERSIONS = { "strip_prefix": "python", }, "3.10.18": { - "url": "20250612/cpython-{python_version}+20250612-{platform}-{build}.tar.gz", + "url": "20250702/cpython-{python_version}+20250702-{platform}-{build}.tar.gz", "sha256": { - "aarch64-apple-darwin": "ff6c9dd7172f82064f8d39fd4cd5d6bec77895ccffe480d846ff4a9750d14093", - "aarch64-unknown-linux-gnu": "11cc65da5cb3a469bc67b6f91bac5ec00d2070394f462ef8867a4db8d0fc6903", - "ppc64le-unknown-linux-gnu": "9fa6a75eb527016b0731faf2c9238dc4958ba85c41806f4c89efa6e12608cf86", - "riscv64-unknown-linux-gnu": "723a026f2184b4785a55da22b52ed0c0612f938c28ac6400b314b61e1daf10de", - "s390x-unknown-linux-gnu": "c43782f3efe25e0a0c62376643bd1bcdbde05c988aa86cc497df8031d619364a", - "x86_64-apple-darwin": "92ecfbfb89e8137cc88cabc2f408d00758d67454d07c1691706d3dcccc8fc446", - "x86_64-pc-windows-msvc": "d26dba4ec86f49ecbc6800e55f72691b9873115fa7c00f254f28dc04a03e8c13", - "x86_64-unknown-linux-gnu": "c28f5698033f3ba47f0c0f054fcf6b9134ff5082b478663c7c7c25bb7e0c4422", - "x86_64-unknown-linux-musl": "1b5c269a5eb04681e475aec673b1783e5f939f37dce305cd2e96eb0df186e9a2", + "aarch64-apple-darwin": "8f9e5395e3571fbb891a0be6428b4516fbde4064799ce6bda4a3c8f4e7860bd4", + "aarch64-unknown-linux-gnu": "b2d09fab0e4340621edb30c769be8b29dddc2776dad820298592eb6aa1970ec1", + "ppc64le-unknown-linux-gnu": "eafbbb7edafbda87e2080e5677855373f8b21606050229733a7352822ee4d84e", + "riscv64-unknown-linux-gnu": "113eb95dbfe8a24756239007239e18ae59c7fc54e6af46f8353f290225a3f811", + "s390x-unknown-linux-gnu": "fcbfa04bc9f9da1af4751fa916e224956c410ee23033b4fddeca9d2c64830362", + "x86_64-apple-darwin": "9a890f21ecc9692cffec77901fd7a786a330dd461fa97ecb10359ee21ca2be79", + "x86_64-pc-windows-msvc": "59399253bb9f864da6858c0e0e940250ebfdfd2609796dadc201aa487633fe84", + "x86_64-unknown-linux-gnu": "4be698bff9f4197fdbb5a82c03d57f4ec5972960492ad045c82ca53a9480342a", + "x86_64-unknown-linux-musl": "20b0fcae6ece29c681b5fd8e1b740000b6f8b907e68ba5621d029dfaa234b23b", }, "strip_prefix": "python", }, @@ -467,17 +467,18 @@ TOOL_VERSIONS = { "strip_prefix": "python", }, "3.11.13": { - "url": "20250612/cpython-{python_version}+20250612-{platform}-{build}.tar.gz", + "url": "20250702/cpython-{python_version}+20250702-{platform}-{build}.tar.gz", "sha256": { - "aarch64-apple-darwin": "e272f0baca8f5a3cef29cc9c7418b80d0316553062ad3235205a33992155043c", - "aarch64-unknown-linux-gnu": "c6959d0c17fc221a9acc56e4827f3fe7386b610402055950e4b767b3b6871a40", - "ppc64le-unknown-linux-gnu": "22ab07e9bd167e2a7852a7b11b31cd91d090f3658e2ffc5bc6428751942cb1b9", - "riscv64-unknown-linux-gnu": "4ca57a3e139cf47803909a88f4f3940d9ecfde42d8089a11f42074859bc9a122", - "s390x-unknown-linux-gnu": "23cbd87fe9549ddda635ba9fb36b3622b5c939a10a39b25cd8c2587bb65e62ef", - "x86_64-apple-darwin": "e2a3e2434ba140615f01ed9328e063076c8282a38c11cab983bdcd5d1bd582da", - "x86_64-pc-windows-msvc": "cc28397fa47d28b98e1dc880b98cb061b76c88116b1d6028e04443f7221b30da", - "x86_64-unknown-linux-gnu": "4dd2c710a828c8cfff384e0549141016a563a5e153d2819a7225ccc05a1a17c7", - "x86_64-unknown-linux-musl": "130c6b55b06c92b7f952271fabedcdcfc06ac4717c133e0985ba27f799ed76b6", + "aarch64-apple-darwin": "01167ac2c7336ff48a96e8dba30d92f29822a98e5ef27959178498b5a0de61da", + "aarch64-unknown-linux-gnu": "42c99f013117255edcbe7a367694941f1ac096fd9e9a7d7c0d18d09551181930", + "ppc64le-unknown-linux-gnu": "154ad77f7f552ab5f2ae07446eaccf6651db85db7403388c4439c6e43139d05e", + "riscv64-unknown-linux-gnu": "e800cd1651bf2ce0be28541377228258fbe9a9a1fe87633d5fc8c6cb47262525", + "s390x-unknown-linux-gnu": "5c6ce40240d92d9a3af4d49364205ce57bd4e73ba5274abcd3f20b85a0a88df9", + "x86_64-apple-darwin": "b5955f7a951f8aa8755b35a1b3175968fc2b4bff54b9edffc6225c791305c4e6", + "x86_64-pc-windows-msvc": "b68b7314e15f5d479acce2e9385a47f6ed978edc838dbb104175db889b349818", + "aarch64-pc-windows-msvc": "ea81e436ac20b894f2070468f3323e69d4cb1a0e4e12bc14bb702a861f7a323d", + "x86_64-unknown-linux-gnu": "e04944e70637f9d82022c9a41ae31de306b0d5bbd3fb64b9eb3261b8b5e0b30c", + "x86_64-unknown-linux-musl": "69aeea0c21b994874d8481c39b9ba2683cbc7f6ec9cff964e1ea821f5ae4fc31", }, "strip_prefix": "python", }, @@ -590,17 +591,18 @@ TOOL_VERSIONS = { "strip_prefix": "python", }, "3.12.11": { - "url": "20250612/cpython-{python_version}+20250612-{platform}-{build}.tar.gz", + "url": "20250702/cpython-{python_version}+20250702-{platform}-{build}.tar.gz", "sha256": { - "aarch64-apple-darwin": "c6d4843e8af496f034176908ae3384556680284653a4bff45eff07e43fe4ae34", - "aarch64-unknown-linux-gnu": "19e8d91b8c5cdb41c485e0d7daa726db6dd64c9a459029f738d5e55ad8da7c6f", - "ppc64le-unknown-linux-gnu": "32f489b4142ced7a3b476e25ac91ada4dc8aada1e771718a3aa9a0c818500a45", - "riscv64-unknown-linux-gnu": "0c1a3e976a117bf40ce8d75ad4806166e503d554263a9051f7606dbeb01d91ee", - "s390x-unknown-linux-gnu": "ee1a8451aaf49af330884553e2850961539b0563404c26241265ab0f0c929001", - "x86_64-apple-darwin": "7e3468bde68650fb8f63b663a24c56d0bb3353abd16158939b1de0ad60dab195", - "x86_64-pc-windows-msvc": "7b93afa91931dbc37b307a81b8680b30193736b5ef29a44ef6452f702c306e7a", - "x86_64-unknown-linux-gnu": "8e8bb0dbc815fb0b3912e0d8fc0a4f4aaac002bfc1f6cb0fcd278f2888f11bcf", - "x86_64-unknown-linux-musl": "b7464442265092259ee5f2e258c09cace4958f6b8733cff5e32bf8d2d6556a2a", + "aarch64-apple-darwin": "5f8e9480d0981268961e63729de1c9b037cabfe030949943be293f0d3e3e7703", + "aarch64-unknown-linux-gnu": "a63c9d7d712ca33e2fc57d9bf3ebf98c8f574f23b3eeeed44faf3b4b08d8a9b8", + "aarch64-pc-windows-msvc": "4d3736640d8916da6d69060e90cad607903e4f1d8dc0f284fd475f04f312712e", + "ppc64le-unknown-linux-gnu": "76dc3accfc8515fe7e11b5f1af26734bc7c0a075890a9c85dc1c7b6d0421ebbc", + "riscv64-unknown-linux-gnu": "d80dd210da941583c3166ff5a762bfd3f3211ecb2968eee8ec497548ef970682", + "s390x-unknown-linux-gnu": "a7d0778ae32c1d882eb3354877c31298010cde2107ecf60b7b75dcabe7ddd8ad", + "x86_64-apple-darwin": "f7a7a70fc7199cc37fd04bc1375b4cd7f44fb05128965e72b589fe112029cab8", + "x86_64-pc-windows-msvc": "19bdfa7362faf6869c376976e0296b597ce2d70e68ea7b357c6f68c79ad9aa9e", + "x86_64-unknown-linux-gnu": "0919f8b5311765b4cf1342371724d7bf2a6eaf51f15f5cb2b9ad5fd0ee54271c", + "x86_64-unknown-linux-musl": "64308b6133ae57de6d7c84b9caf6b084d1ccabf4b617c8a88a08fa57da66df16", }, "strip_prefix": "python", }, @@ -761,25 +763,27 @@ TOOL_VERSIONS = { }, }, "3.13.5": { - "url": "20250612/cpython-{python_version}+20250612-{platform}-{build}.{ext}", + "url": "20250702/cpython-{python_version}+20250702-{platform}-{build}.{ext}", "sha256": { - "aarch64-apple-darwin": "d7867270b8c7be69ec26a351afb6bf24802b1cd9818e8426bd69d439a619bf2d", - "aarch64-unknown-linux-gnu": "685971ded0af96d1685941243ae1853c70c482b6f858dd86818760776d9c3cb9", - "ppc64le-unknown-linux-gnu": "ee15fcf2b64034dba13127aa37992edacf2efe1b2bb3d62ffd45eb9bea7b2d83", - "riscv64-unknown-linux-gnu": "c0f160ef9ab39c0f0e5baa00b1ecc3fff322c4ccbf1f04646c74559274ad5fc1", - "s390x-unknown-linux-gnu": "49131a3d16c13aea76f9ef5ce57fc612a3062fc866f6fcf971e0de8f8a9b8a8f", - "x86_64-apple-darwin": "d881b0226f1bef59b480c713126c54430a93ea21e5b39394c66927a412dd9907", - "x86_64-pc-windows-msvc": "8f4d4c7d270406be1f8f93b9fd2fd13951e4da274ba59d170f411a20cb1725b3", - "x86_64-unknown-linux-gnu": "f50dc28cfe99eccdadd4e74c2384607f7d5f50fc47447a39a4e24a793c07a9eb", - "x86_64-unknown-linux-musl": "c4bc1cda684320455d41e56980adbacbda269c78527f3ee926711d5d0ff33834", - "aarch64-apple-darwin-freethreaded": "a29cb4ef8adcd343e0f5bc5c4371cbc859fc7ce6d8f1a3c8d0cd7e44c4b9b866", - "aarch64-unknown-linux-gnu-freethreaded": "0ef13d13e16b4e58f167694940c6db54591db50bbc7ba61be6901ed5a69ad27b", - "ppc64le-unknown-linux-gnu-freethreaded": "66545ad4b09385750529ef09a665fc0b0ce698f984df106d7b167e3f7d59eace", - "riscv64-unknown-linux-gnu-freethreaded": "a82a741abefa7db61b2aeef36426bd56da5c69dc9dac105d68fba7fe658943ca", - "s390x-unknown-linux-gnu-freethreaded": "403c5758428013d5aa472841294c7b6ec91a572bb7123d02b7f1de24af4b0e13", - "x86_64-apple-darwin-freethreaded": "52aeb1b4073fa3f180d74a0712ceabc86dd2b40be499599e2e170948fb22acde", - "x86_64-pc-windows-msvc-freethreaded": "9da2f02d81597340163174ee91d91a8733dad2af53fc1b7c79ecc45a739a89d5", - "x86_64-unknown-linux-gnu-freethreaded": "33fdd6c42258cdf0402297d9e06842b53d9413d70849cee61755b9b5fb619836", + "aarch64-apple-darwin": "66577414e9f4b0caa116a8e15fa50306db91bce13d49278079bb22adaeefb1fa", + "aarch64-unknown-linux-gnu": "272a8817921856d7ac47f44c076fb62fbaf5649aa1d97b2d67a3a6adee969ff0", + "ppc64le-unknown-linux-gnu": "7bfa9fed4b3a1e37b4879d51d82bce521bd999ec450c91f7787188ce1cafd76c", + "riscv64-unknown-linux-gnu": "deebdf17f7c153708b88ef2ae8b643635a02a9e9bdf4f0435e8c6cd15b37b248", + "s390x-unknown-linux-gnu": "38c10133adfc9ebe9d2e74f7047ab6763b05c978be2dc772e1deb2978504084f", + "x86_64-apple-darwin": "0682afbb238b4762b8f5e383fe19cc52969c780871016c50d4cb7088a536167c", + "x86_64-pc-windows-msvc": "f11f915437250657019c71adb81ec523d2932c2c3ea4441b592aa3bdce0e7ef7", + "aarch64-pc-windows-msvc": "f2de020035f125a47aee320f722b0ced19862ba1e1412392791cffa9cb174d0c", + "aarch64-pc-windows-msvc-freethreaded": "97041594d903d6a1de1e55e9a3e5c613384aa7b900a93096f372732d9953f52a", + "x86_64-unknown-linux-gnu": "9f5d5260f333fcb5372ec681851d92ddac79a33362aa85626b6cc96ffe75eeef", + "x86_64-unknown-linux-musl": "7856fd505e311d1a4c24e429ac5ef0ff6ca7a2005c3a7eff1fe204524a6f45aa", + "aarch64-apple-darwin-freethreaded": "52e582cc89d654c565297b4ff9c3bd4bed5c3e81cad46f41c62485e700faf8bd", + "aarch64-unknown-linux-gnu-freethreaded": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855", + "ppc64le-unknown-linux-gnu-freethreaded": "c65c75edb450de830f724afdc774a215c2d3255097e0d670f709d2271fd6fd52", + "riscv64-unknown-linux-gnu-freethreaded": "716e6e3fad24fb9931b93005000152dd9da4c3343b88ca54b5c01a7ab879d734", + "s390x-unknown-linux-gnu-freethreaded": "27276aee426a51f4165fac49391aedc5a9e301ae217366c77b65826122bb30fc", + "x86_64-apple-darwin-freethreaded": "5aed6d5950514004149d514f81a1cd426ac549696a563b8e47d32f7eba3b4be3", + "x86_64-pc-windows-msvc-freethreaded": "39e19dcb823a2ed47d9510753a642ba468802f1c5e15771c6c22814f4acada94", + "x86_64-unknown-linux-gnu-freethreaded": "f5eb29604c0b7afa2097fca094a06eb7a1f3ca4e194264c34f342739cae78202", }, "strip_prefix": { "aarch64-apple-darwin": "python", @@ -798,29 +802,33 @@ TOOL_VERSIONS = { "s390x-unknown-linux-gnu-freethreaded": "python/install", "x86_64-apple-darwin-freethreaded": "python/install", "x86_64-pc-windows-msvc-freethreaded": "python/install", + "aarch64-pc-windows-msvc": "python/install", + "aarch64-pc-windows-msvc-freethreaded": "python/install", "x86_64-unknown-linux-gnu-freethreaded": "python/install", }, }, - "3.14.0b2": { - "url": "20250612/cpython-{python_version}+20250612-{platform}-{build}.{ext}", + "3.14.0b3": { + "url": "20250702/cpython-{python_version}+20250702-{platform}-{build}.{ext}", "sha256": { - "aarch64-apple-darwin": "35c02e465af605eafd29d5931daadce724eeb8a3e7cc7156ac046991cb24f1c1", - "aarch64-unknown-linux-gnu": "8c877a1b50eb2a9b34ddac5d52d50867f11ddc817f257eba4cbbc999a9edf2ea", - "ppc64le-unknown-linux-gnu": "735bad9359eb36b55b76d9c6db122fe4357951d7850324c76e168055ca70e0a0", - "riscv64-unknown-linux-gnu": "d4140196c052ba5832a439f84f6ca5b136bb16bceb8c5a52f5167a2c3f8b73b1", - "s390x-unknown-linux-gnu": "2f440257e02d0a4fb4e93fcbb95b9066ec42bd56a2f03de05f55636e5afcb4b9", - "x86_64-apple-darwin": "5144890b991e63fb73e2714c162c901c3b6f289ae0ef742df3673ab9824c844a", - "x86_64-pc-windows-msvc": "903cfb0ae1766a572dcf62835ef24d3250a512974dcf785738ac0d6c06c9db5b", - "x86_64-unknown-linux-gnu": "1c73b90a8febbd36fc973d7361a1be562e88437d95570721b701f03e59835600", - "x86_64-unknown-linux-musl": "9cdd3983abfca2151661c25cb0fae50a30c8961e07d07ba643edab5be277ae09", - "aarch64-apple-darwin-freethreaded": "1ae31adfed2a8425f08a945869d3bfd910e97acd150465de257d3ae3da37dc7c", - "aarch64-unknown-linux-gnu-freethreaded": "f5fcf5e8310244ccd346aab2abdc2650ffb900a429cfb732c4884e238cba1782", - "ppc64le-unknown-linux-gnu-freethreaded": "c1177510c359494b6a70601d9c810cdfc662f834c1d686abd487eb89d7a577ef", - "riscv64-unknown-linux-gnu-freethreaded": "cb0f2d86b20f47c70a9c8647b01a35ab7d53cbcbde9ab89ffc8aacafb36cc2e4", - "s390x-unknown-linux-gnu-freethreaded": "f38f126b31a55f37829ee581979214a6d2ac8a985ed7915b42c99d52af329d9f", - "x86_64-apple-darwin-freethreaded": "4e022b8b7a1b2986aa5780fae34b5a89a1ac5ed11bea0c3349e674a6cb7e31c1", - "x86_64-pc-windows-msvc-freethreaded": "35abc125304ec81a7be0d7ac54f515e7addd7dcba912882210d37720eaab1d7e", - "x86_64-unknown-linux-gnu-freethreaded": "61383d43f639533a5105abad376bc497cc94dde8a1ed294f523d534c8cd99a8e", + "aarch64-apple-darwin": "14af7a0c0a50f82cf75f79f4c02dc31c73c74032930a8337f83f3ae3bee4660f", + "aarch64-unknown-linux-gnu": "013e2081c3e7e61932210ede84c9f05a4f6533f807287bab141d8abe77087ffd", + "ppc64le-unknown-linux-gnu": "2118b6b9baad4f4283246b281183254620d18d8c95991dc5db810ab07ff41cee", + "riscv64-unknown-linux-gnu": "7d11ccad5bff3085d8b3e725179d7e1f93cc8e4fb83391cb49bc4b29cf877153", + "s390x-unknown-linux-gnu": "e3c90fb8cfe897ac96bb0b0d5de9f4512646b8ebd5c8b3123d9e31a96a0eac3c", + "x86_64-apple-darwin": "8e9d640e5e7c49f8c67dfd2330bdd814f4c5de685abefbe91c639c0e0844c2bd", + "x86_64-pc-windows-msvc": "cdab7856e2495ab4ed666354e9391435c8e45512e841ef8452da69a6e96caa96", + "aarch64-pc-windows-msvc": "000fbc010e844bcd64330badb295da7b5b08b427357f463afc7e600988f7ecc6", + "x86_64-unknown-linux-gnu": "00328c48cc07076a5b083575654761cdb07bc8b3bba864d3a225062722485bac", + "x86_64-unknown-linux-musl": "a2fed85bc3d5415d2318a2eeb0cb9e6effb81667870ae568a08756838ad4926e", + "aarch64-apple-darwin-freethreaded": "d19213021f5fd039d7021ccb41698cc99ca313064d7c1cc9b5ef8f831abb9961", + "aarch64-unknown-linux-gnu-freethreaded": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855", + "ppc64le-unknown-linux-gnu-freethreaded": "1f093e0c3532e27744e3fb73a8c738355910b6bfa195039e4f73b4f48c1bc4fc", + "riscv64-unknown-linux-gnu-freethreaded": "73162a5da31cc1e410d456496114f8e5ee7243bc7bbe0e087b1ea50f0fdc6774", + "s390x-unknown-linux-gnu-freethreaded": "045017e60f1298111e8ccfec6afbe47abe56f82997258c8754009269a5343736", + "x86_64-apple-darwin-freethreaded": "26ec6697bbb38c3fa6275e79e110854b2585914ca503c65916478e7ca8d0491b", + "x86_64-pc-windows-msvc-freethreaded": "8de6235b29396e3b25fc3ade166c49506171ec464cda46987ef9641dd9a44071", + "aarch64-pc-windows-msvc-freethreaded": "331816d79cd78eaadba5ae6cdd3a243771199d0ca07057e7a452158dd4a7edcc", + "x86_64-unknown-linux-gnu-freethreaded": "081f0147d8f4479764d6a3819f67275be3306003366eda9ecb9ee844f2f611be", }, "strip_prefix": { "aarch64-apple-darwin": "python", @@ -830,6 +838,7 @@ TOOL_VERSIONS = { "riscv64-unknown-linux-gnu": "python", "x86_64-apple-darwin": "python", "x86_64-pc-windows-msvc": "python", + "aarch64-pc-windows-msvc": "python", "x86_64-unknown-linux-gnu": "python", "x86_64-unknown-linux-musl": "python", "aarch64-apple-darwin-freethreaded": "python/install", @@ -839,6 +848,7 @@ TOOL_VERSIONS = { "s390x-unknown-linux-gnu-freethreaded": "python/install", "x86_64-apple-darwin-freethreaded": "python/install", "x86_64-pc-windows-msvc-freethreaded": "python/install", + "aarch64-pc-windows-msvc-freethreaded": "python/install", "x86_64-unknown-linux-gnu-freethreaded": "python/install", }, }, @@ -852,7 +862,7 @@ MINOR_MAPPING = { "3.11": "3.11.13", "3.12": "3.12.11", "3.13": "3.13.5", - "3.14": "3.14.0b2", + "3.14": "3.14.0b3", } def _generate_platforms(): @@ -868,6 +878,14 @@ def _generate_platforms(): os_name = MACOS_NAME, arch = "aarch64", ), + "aarch64-pc-windows-msvc": platform_info( + compatible_with = [ + "@platforms//os:windows", + "@platforms//cpu:aarch64", + ], + os_name = WINDOWS_NAME, + arch = "aarch64", + ), "aarch64-unknown-linux-gnu": platform_info( compatible_with = [ "@platforms//os:linux", @@ -1029,6 +1047,7 @@ def get_release_info(platform, python_version, base_url = DEFAULT_RELEASE_BASE_U FREETHREADED.lstrip("-"), { "aarch64-apple-darwin": "pgo+lto", + "aarch64-pc-windows-msvc": "pgo", "aarch64-unknown-linux-gnu": "lto", "ppc64le-unknown-linux-gnu": "lto", "riscv64-unknown-linux-gnu": "lto", diff --git a/tests/python/python_tests.bzl b/tests/python/python_tests.bzl index 106cff27bb..bd2d812f28 100644 --- a/tests/python/python_tests.bzl +++ b/tests/python/python_tests.bzl @@ -326,7 +326,7 @@ def _test_toolchain_ordering(env): "3.11": "3.11.13", "3.12": "3.12.11", "3.13": "3.13.5", - "3.14": "3.14.0b2", + "3.14": "3.14.0b3", "3.8": "3.8.20", "3.9": "3.9.23", }) From 2690e3fef1478a5449b6af6b69a1b77f5513773c Mon Sep 17 00:00:00 2001 From: Ignas Anikevicius <240938+aignas@users.noreply.github.com> Date: Mon, 7 Jul 2025 01:56:22 +0900 Subject: [PATCH 314/922] refactor(toolchains): better sha256 printing helper (#3028) Before this PR the toolchain sha256 values would be printed in a way that would require further text manipulation. Now we print the values that need to be just copy pasted. Whilst at it simplify the `curl` command to remove the conditional. Testing done: ``` $ bazel run //python/private:print_toolchains_checksums --//python/config_settings:python_version="" # And then paste all of the output into the inside of the TOOL_VERSIONS ``` Work towards #2704 --- python/private/BUILD.bazel | 2 +- python/private/print_toolchain_checksums.bzl | 92 ++++++++++++++++++++ python/versions.bzl | 54 +----------- 3 files changed, 96 insertions(+), 52 deletions(-) create mode 100644 python/private/print_toolchain_checksums.bzl diff --git a/python/private/BUILD.bazel b/python/private/BUILD.bazel index 8bcc6eaebe..6fc78efc25 100644 --- a/python/private/BUILD.bazel +++ b/python/private/BUILD.bazel @@ -16,7 +16,7 @@ load("@bazel_skylib//:bzl_library.bzl", "bzl_library") load("@bazel_skylib//rules:common_settings.bzl", "bool_setting") load("//python:py_binary.bzl", "py_binary") load("//python:py_library.bzl", "py_library") -load("//python:versions.bzl", "print_toolchains_checksums") +load(":print_toolchain_checksums.bzl", "print_toolchains_checksums") load(":py_exec_tools_toolchain.bzl", "current_interpreter_executable") load(":sentinel.bzl", "sentinel") load(":stamp.bzl", "stamp_build_setting") diff --git a/python/private/print_toolchain_checksums.bzl b/python/private/print_toolchain_checksums.bzl new file mode 100644 index 0000000000..eaaa5b9d75 --- /dev/null +++ b/python/private/print_toolchain_checksums.bzl @@ -0,0 +1,92 @@ +"""Print the toolchain versions. +""" + +load("//python:versions.bzl", "TOOL_VERSIONS", "get_release_info") +load("//python/private:text_util.bzl", "render") +load("//python/private:version.bzl", "version") + +def print_toolchains_checksums(name): + """A macro to print checksums for a particular Python interpreter version. + + Args: + name: {type}`str`: the name of the runnable target. + """ + by_version = {} + + for python_version, metadata in TOOL_VERSIONS.items(): + by_version[python_version] = _commands_for_version( + python_version = python_version, + metadata = metadata, + ) + + all_commands = sorted( + by_version.items(), + key = lambda x: version.key(version.parse(x[0], strict = True)), + ) + all_commands = [x[1] for x in all_commands] + + template = """\ +cat > "$@" <<'EOF' +#!/bin/bash + +set -o errexit -o nounset -o pipefail + +echo "Fetching hashes..." + +{commands} +EOF + """ + + native.genrule( + name = name, + srcs = [], + outs = ["print_toolchains_checksums.sh"], + cmd = select({ + "//python/config_settings:is_python_{}".format(version_str): template.format( + commands = commands, + ) + for version_str, commands in by_version.items() + } | { + "//conditions:default": template.format(commands = "\n".join(all_commands)), + }), + executable = True, + ) + +def _commands_for_version(*, python_version, metadata): + lines = [] + lines += [ + "cat < "$@" <<'EOF' -#!/bin/bash - -set -o errexit -o nounset -o pipefail - -echo "Fetching hashes..." - -{commands} -EOF - """ - - native.genrule( - name = name, - srcs = [], - outs = ["print_toolchains_checksums.sh"], - cmd = select({ - "//python/config_settings:is_python_{}".format(version): template.format( - commands = commands, - ) - for version, commands in by_version.items() - } | { - "//conditions:default": template.format(commands = "\n".join(all_commands)), - }), - executable = True, - ) - -def _commands_for_version(python_version): - return "\n".join([ - "echo \"{python_version}: {platform}: $$(curl --location --fail {release_url_sha256} 2>/dev/null || curl --location --fail {release_url} 2>/dev/null | shasum -a 256 | awk '{{ print $$1 }}')\"".format( - python_version = python_version, - platform = platform, - release_url = release_url, - release_url_sha256 = release_url + ".sha256", - ) - for platform in TOOL_VERSIONS[python_version]["sha256"].keys() - for release_url in get_release_info(platform, python_version)[1] - ]) - def gen_python_config_settings(name = ""): for platform in PLATFORMS.keys(): native.config_setting( From 41439033ca6bac88b9a501d75c95a8994e1580b6 Mon Sep 17 00:00:00 2001 From: Ignas Anikevicius <240938+aignas@users.noreply.github.com> Date: Mon, 7 Jul 2025 11:05:14 +0900 Subject: [PATCH 315/922] refactor(pypi): move the platform config to MODULE.bazel (#3064) This splits the configuration that we have for the `rules_python` and the defaults that we set for our users from the actual unit tests where we check that the extension is working correctly. With this we will be able to dog food the API and point users into the `MODULE.bazel` as the example snippet. Work towards #2949 Work towards #2747 --- MODULE.bazel | 56 ++++++++++ python/private/pypi/extension.bzl | 60 +---------- tests/pypi/extension/extension_tests.bzl | 132 ++++++++++------------- 3 files changed, 115 insertions(+), 133 deletions(-) diff --git a/MODULE.bazel b/MODULE.bazel index b1d8711815..a9b51951b2 100644 --- a/MODULE.bazel +++ b/MODULE.bazel @@ -60,6 +60,62 @@ register_toolchains("@pythons_hub//:all") # Install twine for our own runfiles wheel publishing and allow bzlmod users to use it. pip = use_extension("//python/extensions:pip.bzl", "pip") + +# NOTE @aignas 2025-07-06: we define these platforms to keep backwards compatibility with the +# current `experimental_index_url` implementation. Whilst we stabilize the API this list may be +# updated with a mention in the CHANGELOG. +[ + pip.default( + arch_name = cpu, + config_settings = [ + "@platforms//cpu:{}".format(cpu), + "@platforms//os:linux", + ], + env = {"platform_version": "0"}, + os_name = "linux", + platform = "linux_{}".format(cpu), + ) + for cpu in [ + "x86_64", + "aarch64", + # TODO @aignas 2025-05-19: only leave tier 0-1 cpus when stabilizing the + # `pip.default` extension. i.e. drop the below values - users will have to + # define themselves if they need them. + "arm", + "ppc", + "s390x", + ] +] + +[ + pip.default( + arch_name = cpu, + config_settings = [ + "@platforms//cpu:{}".format(cpu), + "@platforms//os:osx", + ], + # We choose the oldest non-EOL version at the time when we release `rules_python`. + # See https://endoflife.date/macos + env = {"platform_version": "14.0"}, + os_name = "osx", + platform = "osx_{}".format(cpu), + ) + for cpu in [ + "aarch64", + "x86_64", + ] +] + +pip.default( + arch_name = "x86_64", + config_settings = [ + "@platforms//cpu:x86_64", + "@platforms//os:windows", + ], + env = {"platform_version": "0"}, + os_name = "windows", + platform = "windows_x86_64", +) pip.parse( # NOTE @aignas 2024-10-26: We have an integration test that depends on us # being able to build sdists for this hub, so explicitly set this to False. diff --git a/python/private/pypi/extension.bzl b/python/private/pypi/extension.bzl index a0095f8f15..505458008f 100644 --- a/python/private/pypi/extension.bzl +++ b/python/private/pypi/extension.bzl @@ -393,64 +393,6 @@ def _configure(config, *, platform, os_name, arch_name, config_settings, env = { else: config["platforms"].pop(platform) -def _create_config(defaults): - if defaults["platforms"]: - return struct(**defaults) - - # NOTE: We have this so that it is easier to maintain unit tests assuming certain - # defaults - for cpu in [ - "x86_64", - "aarch64", - # TODO @aignas 2025-05-19: only leave tier 0-1 cpus when stabilizing the - # `pip.default` extension. i.e. drop the below values - users will have to - # define themselves if they need them. - "arm", - "ppc", - "s390x", - ]: - _configure( - defaults, - arch_name = cpu, - os_name = "linux", - platform = "linux_{}".format(cpu), - config_settings = [ - "@platforms//os:linux", - "@platforms//cpu:{}".format(cpu), - ], - env = {"platform_version": "0"}, - ) - for cpu in [ - "aarch64", - "x86_64", - ]: - _configure( - defaults, - arch_name = cpu, - # We choose the oldest non-EOL version at the time when we release `rules_python`. - # See https://endoflife.date/macos - os_name = "osx", - platform = "osx_{}".format(cpu), - config_settings = [ - "@platforms//os:osx", - "@platforms//cpu:{}".format(cpu), - ], - env = {"platform_version": "14.0"}, - ) - - _configure( - defaults, - arch_name = "x86_64", - os_name = "windows", - platform = "windows_x86_64", - config_settings = [ - "@platforms//os:windows", - "@platforms//cpu:x86_64", - ], - env = {"platform_version": "0"}, - ) - return struct(**defaults) - def parse_modules( module_ctx, _fail = fail, @@ -527,7 +469,7 @@ You cannot use both the additive_build_content and additive_build_content_file a # for what. We could also model the `cp313t` freethreaded as separate platforms. ) - config = _create_config(defaults) + config = struct(**defaults) # TODO @aignas 2025-06-03: Merge override API with the builder? _overriden_whl_set = {} diff --git a/tests/pypi/extension/extension_tests.bzl b/tests/pypi/extension/extension_tests.bzl index 146293ee8d..cf96d4005a 100644 --- a/tests/pypi/extension/extension_tests.bzl +++ b/tests/pypi/extension/extension_tests.bzl @@ -56,7 +56,23 @@ def _mod(*, name, default = [], parse = [], override = [], whl_mods = [], is_roo parse = parse, override = override, whl_mods = whl_mods, - default = default, + default = default or [ + _default( + platform = "{}_{}".format(os, cpu), + os_name = os, + arch_name = cpu, + config_settings = [ + "@platforms//os:{}".format(os), + "@platforms//cpu:{}".format(cpu), + ], + ) + for os, cpu in [ + ("linux", "x86_64"), + ("linux", "aarch64"), + ("osx", "aarch64"), + ("windows", "aarch64"), + ] + ], ), is_root = is_root, ) @@ -235,19 +251,18 @@ def _test_simple_multiple_requirements(env): pypi.hub_group_map().contains_exactly({"pypi": {}}) pypi.hub_whl_map().contains_exactly({"pypi": { "simple": { - "pypi_315_simple_osx_aarch64_osx_x86_64": [ + "pypi_315_simple_osx_aarch64": [ whl_config_setting( target_platforms = [ "cp315_osx_aarch64", - "cp315_osx_x86_64", ], version = "3.15", ), ], - "pypi_315_simple_windows_x86_64": [ + "pypi_315_simple_windows_aarch64": [ whl_config_setting( target_platforms = [ - "cp315_windows_x86_64", + "cp315_windows_aarch64", ], version = "3.15", ), @@ -255,12 +270,12 @@ def _test_simple_multiple_requirements(env): }, }}) pypi.whl_libraries().contains_exactly({ - "pypi_315_simple_osx_aarch64_osx_x86_64": { + "pypi_315_simple_osx_aarch64": { "dep_template": "@pypi//{name}:{target}", "python_interpreter_target": "unit_test_interpreter_target", "requirement": "simple==0.0.2 --hash=sha256:deadb00f", }, - "pypi_315_simple_windows_x86_64": { + "pypi_315_simple_windows_aarch64": { "dep_template": "@pypi//{name}:{target}", "python_interpreter_target": "unit_test_interpreter_target", "requirement": "simple==0.0.1 --hash=sha256:deadbeef", @@ -310,24 +325,20 @@ torch==2.4.1 ; platform_machine != 'x86_64' \ pypi.hub_group_map().contains_exactly({"pypi": {}}) pypi.hub_whl_map().contains_exactly({"pypi": { "torch": { - "pypi_315_torch_linux_aarch64_linux_arm_linux_ppc_linux_s390x_osx_aarch64": [ + "pypi_315_torch_linux_aarch64_osx_aarch64_windows_aarch64": [ whl_config_setting( target_platforms = [ "cp315_linux_aarch64", - "cp315_linux_arm", - "cp315_linux_ppc", - "cp315_linux_s390x", "cp315_osx_aarch64", + "cp315_windows_aarch64", ], version = "3.15", ), ], - "pypi_315_torch_linux_x86_64_osx_x86_64_windows_x86_64": [ + "pypi_315_torch_linux_x86_64": [ whl_config_setting( target_platforms = [ "cp315_linux_x86_64", - "cp315_osx_x86_64", - "cp315_windows_x86_64", ], version = "3.15", ), @@ -335,12 +346,12 @@ torch==2.4.1 ; platform_machine != 'x86_64' \ }, }}) pypi.whl_libraries().contains_exactly({ - "pypi_315_torch_linux_aarch64_linux_arm_linux_ppc_linux_s390x_osx_aarch64": { + "pypi_315_torch_linux_aarch64_osx_aarch64_windows_aarch64": { "dep_template": "@pypi//{name}:{target}", "python_interpreter_target": "unit_test_interpreter_target", "requirement": "torch==2.4.1 --hash=sha256:deadbeef", }, - "pypi_315_torch_linux_x86_64_osx_x86_64_windows_x86_64": { + "pypi_315_torch_linux_x86_64": { "dep_template": "@pypi//{name}:{target}", "python_interpreter_target": "unit_test_interpreter_target", "requirement": "torch==2.4.1+cpu", @@ -385,6 +396,23 @@ def _test_torch_experimental_index_url(env): module_ctx = _mock_mctx( _mod( name = "rules_python", + default = [ + _default( + platform = "{}_{}".format(os, cpu), + os_name = os, + arch_name = cpu, + config_settings = [ + "@platforms//os:{}".format(os), + "@platforms//cpu:{}".format(cpu), + ], + ) + for os, cpu in [ + ("linux", "aarch64"), + ("linux", "x86_64"), + ("osx", "aarch64"), + ("windows", "x86_64"), + ] + ], parse = [ _parse( hub_name = "pypi", @@ -444,34 +472,26 @@ torch==2.4.1+cpu ; platform_machine == 'x86_64' \ pypi.hub_whl_map().contains_exactly({"pypi": { "torch": { "pypi_312_torch_cp312_cp312_linux_x86_64_8800deef": [ - struct( - config_setting = None, + whl_config_setting( filename = "torch-2.4.1+cpu-cp312-cp312-linux_x86_64.whl", - target_platforms = None, version = "3.12", ), ], "pypi_312_torch_cp312_cp312_manylinux_2_17_aarch64_36109432": [ - struct( - config_setting = None, + whl_config_setting( filename = "torch-2.4.1-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", - target_platforms = None, version = "3.12", ), ], "pypi_312_torch_cp312_cp312_win_amd64_3a570e5c": [ - struct( - config_setting = None, + whl_config_setting( filename = "torch-2.4.1+cpu-cp312-cp312-win_amd64.whl", - target_platforms = None, version = "3.12", ), ], "pypi_312_torch_cp312_none_macosx_11_0_arm64_72b484d5": [ - struct( - config_setting = None, + whl_config_setting( filename = "torch-2.4.1-cp312-none-macosx_11_0_arm64.whl", - target_platforms = None, version = "3.12", ), ], @@ -482,7 +502,6 @@ torch==2.4.1+cpu ; platform_machine == 'x86_64' \ "dep_template": "@pypi//{name}:{target}", "experimental_target_platforms": [ "linux_x86_64", - "osx_x86_64", "windows_x86_64", ], "filename": "torch-2.4.1+cpu-cp312-cp312-linux_x86_64.whl", @@ -495,9 +514,6 @@ torch==2.4.1+cpu ; platform_machine == 'x86_64' \ "dep_template": "@pypi//{name}:{target}", "experimental_target_platforms": [ "linux_aarch64", - "linux_arm", - "linux_ppc", - "linux_s390x", "osx_aarch64", ], "filename": "torch-2.4.1-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", @@ -510,7 +526,6 @@ torch==2.4.1+cpu ; platform_machine == 'x86_64' \ "dep_template": "@pypi//{name}:{target}", "experimental_target_platforms": [ "linux_x86_64", - "osx_x86_64", "windows_x86_64", ], "filename": "torch-2.4.1+cpu-cp312-cp312-win_amd64.whl", @@ -523,9 +538,6 @@ torch==2.4.1+cpu ; platform_machine == 'x86_64' \ "dep_template": "@pypi//{name}:{target}", "experimental_target_platforms": [ "linux_aarch64", - "linux_arm", - "linux_ppc", - "linux_s390x", "osx_aarch64", ], "filename": "torch-2.4.1-cp312-none-macosx_11_0_arm64.whl", @@ -817,13 +829,9 @@ git_dep @ git+https://git.server/repo/project@deadbeefdeadbeef "dep_template": "@pypi//{name}:{target}", "experimental_target_platforms": [ "linux_aarch64", - "linux_arm", - "linux_ppc", - "linux_s390x", "linux_x86_64", "osx_aarch64", - "osx_x86_64", - "windows_x86_64", + "windows_aarch64", ], "extra_pip_args": ["--extra-args-for-sdist-building"], "filename": "any-name.tar.gz", @@ -836,13 +844,9 @@ git_dep @ git+https://git.server/repo/project@deadbeefdeadbeef "dep_template": "@pypi//{name}:{target}", "experimental_target_platforms": [ "linux_aarch64", - "linux_arm", - "linux_ppc", - "linux_s390x", "linux_x86_64", "osx_aarch64", - "osx_x86_64", - "windows_x86_64", + "windows_aarch64", ], "filename": "direct_without_sha-0.0.1-py3-none-any.whl", "python_interpreter_target": "unit_test_interpreter_target", @@ -866,13 +870,9 @@ git_dep @ git+https://git.server/repo/project@deadbeefdeadbeef "dep_template": "@pypi//{name}:{target}", "experimental_target_platforms": [ "linux_aarch64", - "linux_arm", - "linux_ppc", - "linux_s390x", "linux_x86_64", "osx_aarch64", - "osx_x86_64", - "windows_x86_64", + "windows_aarch64", ], "filename": "simple-0.0.1-py3-none-any.whl", "python_interpreter_target": "unit_test_interpreter_target", @@ -884,13 +884,9 @@ git_dep @ git+https://git.server/repo/project@deadbeefdeadbeef "dep_template": "@pypi//{name}:{target}", "experimental_target_platforms": [ "linux_aarch64", - "linux_arm", - "linux_ppc", - "linux_s390x", "linux_x86_64", "osx_aarch64", - "osx_x86_64", - "windows_x86_64", + "windows_aarch64", ], "extra_pip_args": ["--extra-args-for-sdist-building"], "filename": "simple-0.0.1.tar.gz", @@ -903,13 +899,9 @@ git_dep @ git+https://git.server/repo/project@deadbeefdeadbeef "dep_template": "@pypi//{name}:{target}", "experimental_target_platforms": [ "linux_aarch64", - "linux_arm", - "linux_ppc", - "linux_s390x", "linux_x86_64", "osx_aarch64", - "osx_x86_64", - "windows_x86_64", + "windows_aarch64", ], "filename": "some_pkg-0.0.1-py3-none-any.whl", "python_interpreter_target": "unit_test_interpreter_target", @@ -921,13 +913,9 @@ git_dep @ git+https://git.server/repo/project@deadbeefdeadbeef "dep_template": "@pypi//{name}:{target}", "experimental_target_platforms": [ "linux_aarch64", - "linux_arm", - "linux_ppc", - "linux_s390x", "linux_x86_64", "osx_aarch64", - "osx_x86_64", - "windows_x86_64", + "windows_aarch64", ], "filename": "some-other-pkg-0.0.1-py3-none-any.whl", "python_interpreter_target": "unit_test_interpreter_target", @@ -995,26 +983,22 @@ optimum[onnxruntime-gpu]==1.17.1 ; sys_platform == 'linux' pypi.hub_whl_map().contains_exactly({ "pypi": { "optimum": { - "pypi_315_optimum_linux_aarch64_linux_arm_linux_ppc_linux_s390x_linux_x86_64": [ + "pypi_315_optimum_linux_aarch64_linux_x86_64": [ whl_config_setting( version = "3.15", target_platforms = [ "cp315_linux_aarch64", - "cp315_linux_arm", - "cp315_linux_ppc", - "cp315_linux_s390x", "cp315_linux_x86_64", ], config_setting = None, filename = None, ), ], - "pypi_315_optimum_osx_aarch64_osx_x86_64": [ + "pypi_315_optimum_osx_aarch64": [ whl_config_setting( version = "3.15", target_platforms = [ "cp315_osx_aarch64", - "cp315_osx_x86_64", ], config_setting = None, filename = None, @@ -1025,12 +1009,12 @@ optimum[onnxruntime-gpu]==1.17.1 ; sys_platform == 'linux' }) pypi.whl_libraries().contains_exactly({ - "pypi_315_optimum_linux_aarch64_linux_arm_linux_ppc_linux_s390x_linux_x86_64": { + "pypi_315_optimum_linux_aarch64_linux_x86_64": { "dep_template": "@pypi//{name}:{target}", "python_interpreter_target": "unit_test_interpreter_target", "requirement": "optimum[onnxruntime-gpu]==1.17.1", }, - "pypi_315_optimum_osx_aarch64_osx_x86_64": { + "pypi_315_optimum_osx_aarch64": { "dep_template": "@pypi//{name}:{target}", "python_interpreter_target": "unit_test_interpreter_target", "requirement": "optimum[onnxruntime]==1.17.1", From 3d932740ac5b10b48061f1b84788b1b131ce0736 Mon Sep 17 00:00:00 2001 From: Ignas Anikevicius <240938+aignas@users.noreply.github.com> Date: Mon, 7 Jul 2025 11:15:02 +0900 Subject: [PATCH 316/922] fix(pypi): correctly handle custom names in pipstar platforms (#3054) Before it seems that we were relying on particular names in the pipstar platforms. This ensures that we rely on this less. Whilst at it fix a few typos and improve the formatting of the code. Work towards #2949 Work towards #2747 --- CHANGELOG.md | 2 ++ python/private/pypi/BUILD.bazel | 4 ---- python/private/pypi/evaluate_markers.bzl | 2 +- python/private/pypi/extension.bzl | 8 ++++++-- python/private/pypi/parse_requirements.bzl | 6 +++++- python/private/pypi/pep508_env.bzl | 5 ----- tests/pypi/extension/extension_tests.bzl | 20 +++++++++----------- tests/pypi/pep508/evaluate_tests.bzl | 3 ++- 8 files changed, 25 insertions(+), 25 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 81768af36a..2b57af606e 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -82,6 +82,8 @@ END_UNRELEASED_TEMPLATE * (toolchains) `local_runtime_repo` now checks if the include directory exists before attempting to watch it, fixing issues on macOS with system Python ({gh-issue}`3043`). +* (pypi) The pipstar `defaults` configuration now supports any custom platform + name. {#v0-0-0-added} ### Added diff --git a/python/private/pypi/BUILD.bazel b/python/private/pypi/BUILD.bazel index 2666197786..b098f29e94 100644 --- a/python/private/pypi/BUILD.bazel +++ b/python/private/pypi/BUILD.bazel @@ -252,10 +252,6 @@ bzl_library( bzl_library( name = "pep508_env_bzl", srcs = ["pep508_env.bzl"], - deps = [ - ":pep508_platform_bzl", - "//python/private:version_bzl", - ], ) bzl_library( diff --git a/python/private/pypi/evaluate_markers.bzl b/python/private/pypi/evaluate_markers.bzl index 2b805c33e6..6167cdbc96 100644 --- a/python/private/pypi/evaluate_markers.bzl +++ b/python/private/pypi/evaluate_markers.bzl @@ -57,7 +57,7 @@ def evaluate_markers_py(mrctx, *, requirements, python_interpreter, python_inter Args: mrctx: repository_ctx or module_ctx. - requirements: list[str] of the requirement file lines to evaluate. + requirements: {type}`dict[str, list[str]]` of the requirement file lines to evaluate. python_interpreter: str, path to the python_interpreter to use to evaluate the env markers in the given requirements files. It will be only called if the requirements files have env markers. This diff --git a/python/private/pypi/extension.bzl b/python/private/pypi/extension.bzl index 505458008f..2c1528e18d 100644 --- a/python/private/pypi/extension.bzl +++ b/python/private/pypi/extension.bzl @@ -76,7 +76,11 @@ def _platforms(*, python_version, minor_mapping, config): for platform, values in config.platforms.items(): key = "{}_{}".format(abi, platform) - platforms[key] = env(key) | values.env + platforms[key] = env(struct( + abi = abi, + os = values.os_name, + arch = values.arch_name, + )) | values.env return platforms def _create_whl_repos( @@ -348,7 +352,7 @@ def _whl_repo(*, src, whl_library_args, is_multiple_versions, download_only, net args["filename"] = src.filename if not enable_pipstar: args["experimental_target_platforms"] = [ - # Get rid of the version fot the target platforms because we are + # Get rid of the version for the target platforms because we are # passing the interpreter any way. Ideally we should search of ways # how to pass the target platforms through the hub repo. p.partition("_")[2] diff --git a/python/private/pypi/parse_requirements.bzl b/python/private/pypi/parse_requirements.bzl index e4a8b90acb..9c610f11d3 100644 --- a/python/private/pypi/parse_requirements.bzl +++ b/python/private/pypi/parse_requirements.bzl @@ -402,6 +402,10 @@ def _add_dists(*, requirement, index_urls, logger = None): ])) # Filter out the wheels that are incompatible with the target_platforms. - whls = select_whls(whls = whls, want_platforms = requirement.target_platforms, logger = logger) + whls = select_whls( + whls = whls, + want_platforms = requirement.target_platforms, + logger = logger, + ) return whls, sdist diff --git a/python/private/pypi/pep508_env.bzl b/python/private/pypi/pep508_env.bzl index a6efb3c50c..c2d404bc3e 100644 --- a/python/private/pypi/pep508_env.bzl +++ b/python/private/pypi/pep508_env.bzl @@ -15,8 +15,6 @@ """This module is for implementing PEP508 environment definition. """ -load(":pep508_platform.bzl", "platform_from_str") - # See https://stackoverflow.com/a/45125525 platform_machine_aliases = { # These pairs mean the same hardware, but different values may be used @@ -175,9 +173,6 @@ def env(target_platform, *, extra = None): if extra != None: env["extra"] = extra - if type(target_platform) == type(""): - target_platform = platform_from_str(target_platform, python_version = "") - if target_platform.abi: minor_version, _, micro_version = target_platform.abi[3:].partition(".") micro_version = micro_version or "0" diff --git a/tests/pypi/extension/extension_tests.bzl b/tests/pypi/extension/extension_tests.bzl index cf96d4005a..0303843e80 100644 --- a/tests/pypi/extension/extension_tests.bzl +++ b/tests/pypi/extension/extension_tests.bzl @@ -1032,7 +1032,9 @@ def _test_pipstar_platforms(env): name = "rules_python", default = [ _default( - platform = "{}_{}".format(os, cpu), + platform = "my{}_{}".format(os, cpu), + os_name = os, + arch_name = cpu, config_settings = [ "@platforms//os:{}".format(os), "@platforms//cpu:{}".format(cpu), @@ -1070,24 +1072,20 @@ optimum[onnxruntime-gpu]==1.17.1 ; sys_platform == 'linux' pypi.hub_whl_map().contains_exactly({ "pypi": { "optimum": { - "pypi_315_optimum_linux_x86_64": [ + "pypi_315_optimum_mylinux_x86_64": [ whl_config_setting( version = "3.15", target_platforms = [ - "cp315_linux_x86_64", + "cp315_mylinux_x86_64", ], - config_setting = None, - filename = None, ), ], - "pypi_315_optimum_osx_aarch64": [ + "pypi_315_optimum_myosx_aarch64": [ whl_config_setting( version = "3.15", target_platforms = [ - "cp315_osx_aarch64", + "cp315_myosx_aarch64", ], - config_setting = None, - filename = None, ), ], }, @@ -1095,12 +1093,12 @@ optimum[onnxruntime-gpu]==1.17.1 ; sys_platform == 'linux' }) pypi.whl_libraries().contains_exactly({ - "pypi_315_optimum_linux_x86_64": { + "pypi_315_optimum_mylinux_x86_64": { "dep_template": "@pypi//{name}:{target}", "python_interpreter_target": "unit_test_interpreter_target", "requirement": "optimum[onnxruntime-gpu]==1.17.1", }, - "pypi_315_optimum_osx_aarch64": { + "pypi_315_optimum_myosx_aarch64": { "dep_template": "@pypi//{name}:{target}", "python_interpreter_target": "unit_test_interpreter_target", "requirement": "optimum[onnxruntime]==1.17.1", diff --git a/tests/pypi/pep508/evaluate_tests.bzl b/tests/pypi/pep508/evaluate_tests.bzl index 7b6c064b94..cc867f346c 100644 --- a/tests/pypi/pep508/evaluate_tests.bzl +++ b/tests/pypi/pep508/evaluate_tests.bzl @@ -16,6 +16,7 @@ load("@rules_testing//lib:test_suite.bzl", "test_suite") load("//python/private/pypi:pep508_env.bzl", pep508_env = "env") # buildifier: disable=bzl-visibility load("//python/private/pypi:pep508_evaluate.bzl", "evaluate", "tokenize") # buildifier: disable=bzl-visibility +load("//python/private/pypi:pep508_platform.bzl", "platform_from_str") # buildifier: disable=bzl-visibility _tests = [] @@ -262,7 +263,7 @@ def _evaluate_with_aliases(env): }, }.items(): # buildifier: @unsorted-dict-items for input, want in tests.items(): - _check_evaluate(env, input, want, pep508_env(target_platform)) + _check_evaluate(env, input, want, pep508_env(platform_from_str(target_platform, ""))) _tests.append(_evaluate_with_aliases) From 466ac33a1cfa729c4f78dcb40b903fd91218b1af Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Sun, 6 Jul 2025 19:34:56 -0700 Subject: [PATCH 317/922] tests(pypi): add tests for namespace shims generation (#3066) This adds functional tests for the generated pkgutil namespace files. The test works by creating two wheels with the necessary structure: * An `__init__.py` file isn't in the wheel for the namespace package * They are both part of the namespace package. * The test verifies both are importable. These are the tests for https://github.com/bazel-contrib/rules_python/pull/3059 --- .bazelrc | 4 +- MODULE.bazel | 2 + python/private/internal_dev_deps.bzl | 26 ++ python/private/pypi/whl_library_targets.bzl | 5 +- tests/implicit_namespace_packages/BUILD.bazel | 12 + .../namespace_packages_test.py | 24 ++ .../ns-sub1/ns-sub1-1.0.dist-info/METADATA | 0 .../ns-sub1/ns-sub1-1.0.dist-info/RECORD | 0 .../ns-sub1/ns-sub1-1.0.dist-info/WHEEL | 1 + .../ns-sub1/nspkg/subpkg1/__init__.py | 1 + .../ns-sub1/nspkg/subpkg1/subpkgmod.py | 1 + .../ns-sub2/ns_sub2-1.0.dist-info/METADATA | 0 .../ns-sub2/ns_sub2-1.0.dist-info/RECORD | 0 .../ns-sub2/ns_sub2-1.0.dist-info/WHEEL | 1 + .../ns-sub2/nspkg/subpkg2/__init__.py | 1 + .../ns-sub2/nspkg/subpkg2/subpkgmod.py | 1 + .../whl_library_targets_tests.bzl | 268 +++++++++++------- 17 files changed, 236 insertions(+), 111 deletions(-) create mode 100644 tests/implicit_namespace_packages/BUILD.bazel create mode 100644 tests/implicit_namespace_packages/namespace_packages_test.py create mode 100644 tests/implicit_namespace_packages/testdata/ns-sub1/ns-sub1-1.0.dist-info/METADATA create mode 100644 tests/implicit_namespace_packages/testdata/ns-sub1/ns-sub1-1.0.dist-info/RECORD create mode 100644 tests/implicit_namespace_packages/testdata/ns-sub1/ns-sub1-1.0.dist-info/WHEEL create mode 100644 tests/implicit_namespace_packages/testdata/ns-sub1/nspkg/subpkg1/__init__.py create mode 100644 tests/implicit_namespace_packages/testdata/ns-sub1/nspkg/subpkg1/subpkgmod.py create mode 100644 tests/implicit_namespace_packages/testdata/ns-sub2/ns_sub2-1.0.dist-info/METADATA create mode 100644 tests/implicit_namespace_packages/testdata/ns-sub2/ns_sub2-1.0.dist-info/RECORD create mode 100644 tests/implicit_namespace_packages/testdata/ns-sub2/ns_sub2-1.0.dist-info/WHEEL create mode 100644 tests/implicit_namespace_packages/testdata/ns-sub2/nspkg/subpkg2/__init__.py create mode 100644 tests/implicit_namespace_packages/testdata/ns-sub2/nspkg/subpkg2/subpkgmod.py diff --git a/.bazelrc b/.bazelrc index 8997db9f91..d7e1771336 100644 --- a/.bazelrc +++ b/.bazelrc @@ -4,8 +4,8 @@ # (Note, we cannot use `common --deleted_packages` because the bazel version command doesn't support it) # To update these lines, execute # `bazel run @rules_bazel_integration_test//tools:update_deleted_packages` -build --deleted_packages=examples/build_file_generation,examples/build_file_generation/random_number_generator,examples/bzlmod,examples/bzlmod_build_file_generation,examples/bzlmod_build_file_generation/other_module/other_module/pkg,examples/bzlmod_build_file_generation/runfiles,examples/bzlmod/entry_points,examples/bzlmod/entry_points/tests,examples/bzlmod/libs/my_lib,examples/bzlmod/other_module,examples/bzlmod/other_module/other_module/pkg,examples/bzlmod/patches,examples/bzlmod/py_proto_library,examples/bzlmod/py_proto_library/example.com/another_proto,examples/bzlmod/py_proto_library/example.com/proto,examples/bzlmod/runfiles,examples/bzlmod/tests,examples/bzlmod/tests/other_module,examples/bzlmod/whl_mods,examples/multi_python_versions/libs/my_lib,examples/multi_python_versions/requirements,examples/multi_python_versions/tests,examples/pip_parse,examples/pip_parse_vendored,examples/pip_repository_annotations,examples/py_proto_library,examples/py_proto_library/example.com/another_proto,examples/py_proto_library/example.com/proto,gazelle,gazelle/manifest,gazelle/manifest/generate,gazelle/manifest/hasher,gazelle/manifest/test,gazelle/modules_mapping,gazelle/python,gazelle/pythonconfig,gazelle/python/private,tests/integration/compile_pip_requirements,tests/integration/compile_pip_requirements_test_from_external_repo,tests/integration/custom_commands,tests/integration/ignore_root_user_error,tests/integration/ignore_root_user_error/submodule,tests/integration/local_toolchains,tests/integration/pip_parse,tests/integration/pip_parse/empty,tests/integration/py_cc_toolchain_registered,tests/modules/another_module,tests/modules/other,tests/modules/other/nspkg_delta,tests/modules/other/nspkg_gamma,tests/modules/other/nspkg_single,tests/modules/other/simple_v1,tests/modules/other/simple_v2,tests/modules/other/with_external_data,tests/whl_with_build_files/testdata,tests/whl_with_build_files/testdata/somepkg,tests/whl_with_build_files/testdata/somepkg-1.0.dist-info,tests/whl_with_build_files/testdata/somepkg/subpkg -query --deleted_packages=examples/build_file_generation,examples/build_file_generation/random_number_generator,examples/bzlmod,examples/bzlmod_build_file_generation,examples/bzlmod_build_file_generation/other_module/other_module/pkg,examples/bzlmod_build_file_generation/runfiles,examples/bzlmod/entry_points,examples/bzlmod/entry_points/tests,examples/bzlmod/libs/my_lib,examples/bzlmod/other_module,examples/bzlmod/other_module/other_module/pkg,examples/bzlmod/patches,examples/bzlmod/py_proto_library,examples/bzlmod/py_proto_library/example.com/another_proto,examples/bzlmod/py_proto_library/example.com/proto,examples/bzlmod/runfiles,examples/bzlmod/tests,examples/bzlmod/tests/other_module,examples/bzlmod/whl_mods,examples/multi_python_versions/libs/my_lib,examples/multi_python_versions/requirements,examples/multi_python_versions/tests,examples/pip_parse,examples/pip_parse_vendored,examples/pip_repository_annotations,examples/py_proto_library,examples/py_proto_library/example.com/another_proto,examples/py_proto_library/example.com/proto,gazelle,gazelle/manifest,gazelle/manifest/generate,gazelle/manifest/hasher,gazelle/manifest/test,gazelle/modules_mapping,gazelle/python,gazelle/pythonconfig,gazelle/python/private,tests/integration/compile_pip_requirements,tests/integration/compile_pip_requirements_test_from_external_repo,tests/integration/custom_commands,tests/integration/ignore_root_user_error,tests/integration/ignore_root_user_error/submodule,tests/integration/local_toolchains,tests/integration/pip_parse,tests/integration/pip_parse/empty,tests/integration/py_cc_toolchain_registered,tests/modules/another_module,tests/modules/other,tests/modules/other/nspkg_delta,tests/modules/other/nspkg_gamma,tests/modules/other/nspkg_single,tests/modules/other/simple_v1,tests/modules/other/simple_v2,tests/modules/other/with_external_data,tests/whl_with_build_files/testdata,tests/whl_with_build_files/testdata/somepkg,tests/whl_with_build_files/testdata/somepkg-1.0.dist-info,tests/whl_with_build_files/testdata/somepkg/subpkg +build --deleted_packages=examples/build_file_generation,examples/build_file_generation/random_number_generator,examples/bzlmod,examples/bzlmod_build_file_generation,examples/bzlmod_build_file_generation/other_module/other_module/pkg,examples/bzlmod_build_file_generation/runfiles,examples/bzlmod/entry_points,examples/bzlmod/entry_points/tests,examples/bzlmod/libs/my_lib,examples/bzlmod/other_module,examples/bzlmod/other_module/other_module/pkg,examples/bzlmod/patches,examples/bzlmod/py_proto_library,examples/bzlmod/py_proto_library/example.com/another_proto,examples/bzlmod/py_proto_library/example.com/proto,examples/bzlmod/runfiles,examples/bzlmod/tests,examples/bzlmod/tests/other_module,examples/bzlmod/whl_mods,examples/multi_python_versions/libs/my_lib,examples/multi_python_versions/requirements,examples/multi_python_versions/tests,examples/pip_parse,examples/pip_parse_vendored,examples/pip_repository_annotations,examples/py_proto_library,examples/py_proto_library/example.com/another_proto,examples/py_proto_library/example.com/proto,gazelle,gazelle/manifest,gazelle/manifest/generate,gazelle/manifest/hasher,gazelle/manifest/test,gazelle/modules_mapping,gazelle/python,gazelle/pythonconfig,gazelle/python/private,rules_python-repro,tests/integration/compile_pip_requirements,tests/integration/compile_pip_requirements_test_from_external_repo,tests/integration/custom_commands,tests/integration/ignore_root_user_error,tests/integration/ignore_root_user_error/submodule,tests/integration/local_toolchains,tests/integration/pip_parse,tests/integration/pip_parse/empty,tests/integration/py_cc_toolchain_registered,tests/modules/another_module,tests/modules/other,tests/modules/other/nspkg_delta,tests/modules/other/nspkg_gamma,tests/modules/other/nspkg_single,tests/modules/other/simple_v1,tests/modules/other/simple_v2,tests/modules/other/with_external_data,tests/whl_with_build_files/testdata,tests/whl_with_build_files/testdata/somepkg,tests/whl_with_build_files/testdata/somepkg-1.0.dist-info,tests/whl_with_build_files/testdata/somepkg/subpkg +query --deleted_packages=examples/build_file_generation,examples/build_file_generation/random_number_generator,examples/bzlmod,examples/bzlmod_build_file_generation,examples/bzlmod_build_file_generation/other_module/other_module/pkg,examples/bzlmod_build_file_generation/runfiles,examples/bzlmod/entry_points,examples/bzlmod/entry_points/tests,examples/bzlmod/libs/my_lib,examples/bzlmod/other_module,examples/bzlmod/other_module/other_module/pkg,examples/bzlmod/patches,examples/bzlmod/py_proto_library,examples/bzlmod/py_proto_library/example.com/another_proto,examples/bzlmod/py_proto_library/example.com/proto,examples/bzlmod/runfiles,examples/bzlmod/tests,examples/bzlmod/tests/other_module,examples/bzlmod/whl_mods,examples/multi_python_versions/libs/my_lib,examples/multi_python_versions/requirements,examples/multi_python_versions/tests,examples/pip_parse,examples/pip_parse_vendored,examples/pip_repository_annotations,examples/py_proto_library,examples/py_proto_library/example.com/another_proto,examples/py_proto_library/example.com/proto,gazelle,gazelle/manifest,gazelle/manifest/generate,gazelle/manifest/hasher,gazelle/manifest/test,gazelle/modules_mapping,gazelle/python,gazelle/pythonconfig,gazelle/python/private,rules_python-repro,tests/integration/compile_pip_requirements,tests/integration/compile_pip_requirements_test_from_external_repo,tests/integration/custom_commands,tests/integration/ignore_root_user_error,tests/integration/ignore_root_user_error/submodule,tests/integration/local_toolchains,tests/integration/pip_parse,tests/integration/pip_parse/empty,tests/integration/py_cc_toolchain_registered,tests/modules/another_module,tests/modules/other,tests/modules/other/nspkg_delta,tests/modules/other/nspkg_gamma,tests/modules/other/nspkg_single,tests/modules/other/simple_v1,tests/modules/other/simple_v2,tests/modules/other/with_external_data,tests/whl_with_build_files/testdata,tests/whl_with_build_files/testdata/somepkg,tests/whl_with_build_files/testdata/somepkg-1.0.dist-info,tests/whl_with_build_files/testdata/somepkg/subpkg test --test_output=errors diff --git a/MODULE.bazel b/MODULE.bazel index a9b51951b2..9db287dc28 100644 --- a/MODULE.bazel +++ b/MODULE.bazel @@ -158,6 +158,8 @@ internal_dev_deps = use_extension( use_repo( internal_dev_deps, "buildkite_config", + "implicit_namespace_ns_sub1", + "implicit_namespace_ns_sub2", "rules_python_runtime_env_tc_info", "somepkg_with_build_files", "whl_with_build_files", diff --git a/python/private/internal_dev_deps.bzl b/python/private/internal_dev_deps.bzl index bb7d76f56a..ca34dc698a 100644 --- a/python/private/internal_dev_deps.bzl +++ b/python/private/internal_dev_deps.bzl @@ -30,6 +30,7 @@ def _internal_dev_deps_impl(mctx): ) runtime_env_repo(name = "rules_python_runtime_env_tc_info") + # Setup for //tests/whl_with_build_files whl_from_dir_repo( name = "whl_with_build_files", root = "//tests/whl_with_build_files:testdata/BUILD.bazel", @@ -41,6 +42,31 @@ def _internal_dev_deps_impl(mctx): requirement = "somepkg", ) + # Setup for //tests/implicit_namespace_packages + whl_from_dir_repo( + name = "implicit_namespace_ns_sub1_whl", + root = "//tests/implicit_namespace_packages:testdata/ns-sub1/BUILD.bazel", + output = "ns_sub1-1.0-any-none-any.whl", + ) + whl_library( + name = "implicit_namespace_ns_sub1", + whl_file = "@implicit_namespace_ns_sub1_whl//:ns_sub1-1.0-any-none-any.whl", + requirement = "ns-sub1", + enable_implicit_namespace_pkgs = False, + ) + + whl_from_dir_repo( + name = "implicit_namespace_ns_sub2_whl", + root = "//tests/implicit_namespace_packages:testdata/ns-sub2/BUILD.bazel", + output = "ns_sub2-1.0-any-none-any.whl", + ) + whl_library( + name = "implicit_namespace_ns_sub2", + whl_file = "@implicit_namespace_ns_sub2_whl//:ns_sub2-1.0-any-none-any.whl", + requirement = "ns-sub2", + enable_implicit_namespace_pkgs = False, + ) + internal_dev_deps = module_extension( implementation = _internal_dev_deps_impl, doc = "This extension creates internal rules_python dev dependencies.", diff --git a/python/private/pypi/whl_library_targets.bzl b/python/private/pypi/whl_library_targets.bzl index 474f39a34d..95c1f5e981 100644 --- a/python/private/pypi/whl_library_targets.bzl +++ b/python/private/pypi/whl_library_targets.bzl @@ -30,7 +30,7 @@ load( "WHEEL_FILE_IMPL_LABEL", "WHEEL_FILE_PUBLIC_LABEL", ) -load(":namespace_pkgs.bzl", "create_inits") +load(":namespace_pkgs.bzl", _create_inits = "create_inits") load(":pep508_deps.bzl", "deps") def whl_library_targets_from_requires( @@ -120,6 +120,7 @@ def whl_library_targets( py_binary = py_binary, py_library = py_library, env_marker_setting = env_marker_setting, + create_inits = _create_inits, )): """Create all of the whl_library targets. @@ -334,7 +335,7 @@ def whl_library_targets( if not enable_implicit_namespace_pkgs: srcs = srcs + getattr(native, "select", select)({ Label("//python/config_settings:is_venvs_site_packages"): [], - "//conditions:default": create_inits( + "//conditions:default": rules.create_inits( srcs = srcs + data + pyi_srcs, ignored_dirnames = [], # If you need to ignore certain folders, you can patch rules_python here to do so. root = "site-packages", diff --git a/tests/implicit_namespace_packages/BUILD.bazel b/tests/implicit_namespace_packages/BUILD.bazel new file mode 100644 index 0000000000..42aca9b97f --- /dev/null +++ b/tests/implicit_namespace_packages/BUILD.bazel @@ -0,0 +1,12 @@ +load("//python:py_test.bzl", "py_test") +load("//tests/support:support.bzl", "SUPPORTS_BZLMOD_UNIXY") + +py_test( + name = "namespace_packages_test", + srcs = ["namespace_packages_test.py"], + target_compatible_with = SUPPORTS_BZLMOD_UNIXY, + deps = [ + "@implicit_namespace_ns_sub1//:pkg", + "@implicit_namespace_ns_sub2//:pkg", + ], +) diff --git a/tests/implicit_namespace_packages/namespace_packages_test.py b/tests/implicit_namespace_packages/namespace_packages_test.py new file mode 100644 index 0000000000..ea47c08fd2 --- /dev/null +++ b/tests/implicit_namespace_packages/namespace_packages_test.py @@ -0,0 +1,24 @@ +import unittest + + +class NamespacePackagesTest(unittest.TestCase): + + def test_both_importable(self): + import nspkg + import nspkg.subpkg1 + import nspkg.subpkg1.subpkgmod + import nspkg.subpkg2.subpkgmod + + self.assertEqual("nspkg.subpkg1", nspkg.subpkg1.expected_name) + self.assertEqual( + "nspkg.subpkg1.subpkgmod", nspkg.subpkg1.subpkgmod.expected_name + ) + + self.assertEqual("nspkg.subpkg2", nspkg.subpkg2.expected_name) + self.assertEqual( + "nspkg.subpkg2.subpkgmod", nspkg.subpkg2.subpkgmod.expected_name + ) + + +if __name__ == "__main__": + unittest.main() diff --git a/tests/implicit_namespace_packages/testdata/ns-sub1/ns-sub1-1.0.dist-info/METADATA b/tests/implicit_namespace_packages/testdata/ns-sub1/ns-sub1-1.0.dist-info/METADATA new file mode 100644 index 0000000000..e69de29bb2 diff --git a/tests/implicit_namespace_packages/testdata/ns-sub1/ns-sub1-1.0.dist-info/RECORD b/tests/implicit_namespace_packages/testdata/ns-sub1/ns-sub1-1.0.dist-info/RECORD new file mode 100644 index 0000000000..e69de29bb2 diff --git a/tests/implicit_namespace_packages/testdata/ns-sub1/ns-sub1-1.0.dist-info/WHEEL b/tests/implicit_namespace_packages/testdata/ns-sub1/ns-sub1-1.0.dist-info/WHEEL new file mode 100644 index 0000000000..a64521a1cc --- /dev/null +++ b/tests/implicit_namespace_packages/testdata/ns-sub1/ns-sub1-1.0.dist-info/WHEEL @@ -0,0 +1 @@ +Wheel-Version: 1.0 diff --git a/tests/implicit_namespace_packages/testdata/ns-sub1/nspkg/subpkg1/__init__.py b/tests/implicit_namespace_packages/testdata/ns-sub1/nspkg/subpkg1/__init__.py new file mode 100644 index 0000000000..6657257dc6 --- /dev/null +++ b/tests/implicit_namespace_packages/testdata/ns-sub1/nspkg/subpkg1/__init__.py @@ -0,0 +1 @@ +expected_name = "nspkg.subpkg1" diff --git a/tests/implicit_namespace_packages/testdata/ns-sub1/nspkg/subpkg1/subpkgmod.py b/tests/implicit_namespace_packages/testdata/ns-sub1/nspkg/subpkg1/subpkgmod.py new file mode 100644 index 0000000000..b03bf39642 --- /dev/null +++ b/tests/implicit_namespace_packages/testdata/ns-sub1/nspkg/subpkg1/subpkgmod.py @@ -0,0 +1 @@ +expected_name = "nspkg.subpkg1.subpkgmod" diff --git a/tests/implicit_namespace_packages/testdata/ns-sub2/ns_sub2-1.0.dist-info/METADATA b/tests/implicit_namespace_packages/testdata/ns-sub2/ns_sub2-1.0.dist-info/METADATA new file mode 100644 index 0000000000..e69de29bb2 diff --git a/tests/implicit_namespace_packages/testdata/ns-sub2/ns_sub2-1.0.dist-info/RECORD b/tests/implicit_namespace_packages/testdata/ns-sub2/ns_sub2-1.0.dist-info/RECORD new file mode 100644 index 0000000000..e69de29bb2 diff --git a/tests/implicit_namespace_packages/testdata/ns-sub2/ns_sub2-1.0.dist-info/WHEEL b/tests/implicit_namespace_packages/testdata/ns-sub2/ns_sub2-1.0.dist-info/WHEEL new file mode 100644 index 0000000000..a64521a1cc --- /dev/null +++ b/tests/implicit_namespace_packages/testdata/ns-sub2/ns_sub2-1.0.dist-info/WHEEL @@ -0,0 +1 @@ +Wheel-Version: 1.0 diff --git a/tests/implicit_namespace_packages/testdata/ns-sub2/nspkg/subpkg2/__init__.py b/tests/implicit_namespace_packages/testdata/ns-sub2/nspkg/subpkg2/__init__.py new file mode 100644 index 0000000000..29bfb67066 --- /dev/null +++ b/tests/implicit_namespace_packages/testdata/ns-sub2/nspkg/subpkg2/__init__.py @@ -0,0 +1 @@ +expected_name = "nspkg.subpkg2" diff --git a/tests/implicit_namespace_packages/testdata/ns-sub2/nspkg/subpkg2/subpkgmod.py b/tests/implicit_namespace_packages/testdata/ns-sub2/nspkg/subpkg2/subpkgmod.py new file mode 100644 index 0000000000..45a28eb851 --- /dev/null +++ b/tests/implicit_namespace_packages/testdata/ns-sub2/nspkg/subpkg2/subpkgmod.py @@ -0,0 +1 @@ +expected_name = "nspkg.subpkg2.subpkgmod" diff --git a/tests/pypi/whl_library_targets/whl_library_targets_tests.bzl b/tests/pypi/whl_library_targets/whl_library_targets_tests.bzl index 22fe3ab7ca..bc58be9698 100644 --- a/tests/pypi/whl_library_targets/whl_library_targets_tests.bzl +++ b/tests/pypi/whl_library_targets/whl_library_targets_tests.bzl @@ -16,18 +16,14 @@ load("@rules_testing//lib:test_suite.bzl", "test_suite") load("//python/private:glob_excludes.bzl", "glob_excludes") # buildifier: disable=bzl-visibility -load("//python/private/pypi:whl_library_targets.bzl", _whl_library_targets = "whl_library_targets", _whl_library_targets_from_requires = "whl_library_targets_from_requires") # buildifier: disable=bzl-visibility +load( + "//python/private/pypi:whl_library_targets.bzl", + "whl_library_targets", + "whl_library_targets_from_requires", +) # buildifier: disable=bzl-visibility _tests = [] -def whl_library_targets(**kwargs): - # Let's skip testing this for now - _whl_library_targets(enable_implicit_namespace_pkgs = True, **kwargs) - -def whl_library_targets_from_requires(**kwargs): - # Let's skip testing this for now - _whl_library_targets_from_requires(enable_implicit_namespace_pkgs = True, **kwargs) - def _test_filegroups(env): calls = [] @@ -190,6 +186,12 @@ def _test_whl_and_library_deps_from_requires(env): py_library_calls = [] env_marker_setting_calls = [] + mock_glob = _mock_glob() + + mock_glob.results.append(["site-packages/foo/SRCS.py"]) + mock_glob.results.append(["site-packages/foo/DATA.txt"]) + mock_glob.results.append(["site-packages/foo/PYI.pyi"]) + whl_library_targets_from_requires( name = "foo-0-py3-none-any.whl", metadata_name = "Foo", @@ -208,12 +210,13 @@ def _test_whl_and_library_deps_from_requires(env): native = struct( filegroup = lambda **kwargs: filegroup_calls.append(kwargs), config_setting = lambda **_: None, - glob = _glob, + glob = mock_glob.glob, select = _select, ), rules = struct( py_library = lambda **kwargs: py_library_calls.append(kwargs), env_marker_setting = lambda **kwargs: env_marker_setting_calls.append(kwargs), + create_inits = lambda *args, **kwargs: ["_create_inits_target"], ), ) @@ -228,34 +231,51 @@ def _test_whl_and_library_deps_from_requires(env): "visibility": ["//visibility:public"], }, ]) # buildifier: @unsorted-dict-items - env.expect.that_collection(py_library_calls).contains_exactly([ - { - "name": "pkg", - "srcs": _glob( - ["site-packages/**/*.py"], - exclude = [], - allow_empty = True, - ), - "pyi_srcs": _glob(["site-packages/**/*.pyi"], allow_empty = True), - "data": [] + _glob( - ["site-packages/**/*"], - exclude = [ - "**/*.py", - "**/*.pyc", - "**/*.pyc.*", - "**/*.dist-info/RECORD", - ] + glob_excludes.version_dependent_exclusions(), - ), - "imports": ["site-packages"], - "deps": ["@pypi//bar:pkg"] + _select({ - ":is_include_bar_baz_true": ["@pypi//bar_baz:pkg"], - "//conditions:default": [], - }), - "tags": ["pypi_name=Foo", "pypi_version=0"], - "visibility": ["//visibility:public"], - "experimental_venvs_site_packages": Label("//python/config_settings:venvs_site_packages"), - }, - ]) # buildifier: @unsorted-dict-items + + env.expect.that_collection(py_library_calls).has_size(1) + if len(py_library_calls) != 1: + return + py_library_call = py_library_calls[0] + + env.expect.that_dict(py_library_call).contains_exactly({ + "name": "pkg", + "srcs": ["site-packages/foo/SRCS.py"] + _select({ + Label("//python/config_settings:is_venvs_site_packages"): [], + "//conditions:default": ["_create_inits_target"], + }), + "pyi_srcs": ["site-packages/foo/PYI.pyi"], + "data": ["site-packages/foo/DATA.txt"], + "imports": ["site-packages"], + "deps": ["@pypi//bar:pkg"] + _select({ + ":is_include_bar_baz_true": ["@pypi//bar_baz:pkg"], + "//conditions:default": [], + }), + "tags": ["pypi_name=Foo", "pypi_version=0"], + "visibility": ["//visibility:public"], + "experimental_venvs_site_packages": Label("//python/config_settings:venvs_site_packages"), + }) # buildifier: @unsorted-dict-items + + env.expect.that_collection(mock_glob.calls).contains_exactly([ + # srcs call + _glob_call( + ["site-packages/**/*.py"], + exclude = [], + allow_empty = True, + ), + # data call + _glob_call( + ["site-packages/**/*"], + exclude = [ + "**/*.py", + "**/*.pyc", + "**/*.pyc.*", + "**/*.dist-info/RECORD", + ] + glob_excludes.version_dependent_exclusions(), + ), + # pyi call + _glob_call(["site-packages/**/*.pyi"], allow_empty = True), + ]) + env.expect.that_collection(env_marker_setting_calls).contains_exactly([ { "name": "include_bar_baz", @@ -269,6 +289,10 @@ _tests.append(_test_whl_and_library_deps_from_requires) def _test_whl_and_library_deps(env): filegroup_calls = [] py_library_calls = [] + mock_glob = _mock_glob() + mock_glob.results.append(["site-packages/foo/SRCS.py"]) + mock_glob.results.append(["site-packages/foo/DATA.txt"]) + mock_glob.results.append(["site-packages/foo/PYI.pyi"]) whl_library_targets( name = "foo.whl", @@ -290,11 +314,12 @@ def _test_whl_and_library_deps(env): native = struct( filegroup = lambda **kwargs: filegroup_calls.append(kwargs), config_setting = lambda **_: None, - glob = _glob, + glob = mock_glob.glob, select = _select, ), rules = struct( py_library = lambda **kwargs: py_library_calls.append(kwargs), + create_inits = lambda **kwargs: ["_create_inits_target"], ), ) @@ -320,45 +345,38 @@ def _test_whl_and_library_deps(env): "visibility": ["//visibility:public"], }, ]) # buildifier: @unsorted-dict-items - env.expect.that_collection(py_library_calls).contains_exactly([ - { - "name": "pkg", - "srcs": _glob( - ["site-packages/**/*.py"], - exclude = [], - allow_empty = True, - ), - "pyi_srcs": _glob(["site-packages/**/*.pyi"], allow_empty = True), - "data": [] + _glob( - ["site-packages/**/*"], - exclude = [ - "**/*.py", - "**/*.pyc", - "**/*.pyc.*", - "**/*.dist-info/RECORD", - ] + glob_excludes.version_dependent_exclusions(), - ), - "imports": ["site-packages"], - "deps": [ - "@pypi_bar_baz//:pkg", - "@pypi_foo//:pkg", - ] + _select( - { - Label("//python/config_settings:is_python_3.9"): ["@pypi_py39_dep//:pkg"], - "@platforms//cpu:aarch64": ["@pypi_arm_dep//:pkg"], - "@platforms//os:windows": ["@pypi_win_dep//:pkg"], - ":is_python_3.10_linux_ppc64le": ["@pypi_py310_linux_ppc64le_dep//:pkg"], - ":is_python_3.9_anyos_aarch64": ["@pypi_py39_arm_dep//:pkg"], - ":is_python_3.9_linux_anyarch": ["@pypi_py39_linux_dep//:pkg"], - ":is_linux_x86_64": ["@pypi_linux_intel_dep//:pkg"], - "//conditions:default": [], - }, - ), - "tags": ["tag1", "tag2"], - "visibility": ["//visibility:public"], - "experimental_venvs_site_packages": Label("//python/config_settings:venvs_site_packages"), - }, - ]) # buildifier: @unsorted-dict-items + + env.expect.that_collection(py_library_calls).has_size(1) + if len(py_library_calls) != 1: + return + env.expect.that_dict(py_library_calls[0]).contains_exactly({ + "name": "pkg", + "srcs": ["site-packages/foo/SRCS.py"] + _select({ + Label("//python/config_settings:is_venvs_site_packages"): [], + "//conditions:default": ["_create_inits_target"], + }), + "pyi_srcs": ["site-packages/foo/PYI.pyi"], + "data": ["site-packages/foo/DATA.txt"], + "imports": ["site-packages"], + "deps": [ + "@pypi_bar_baz//:pkg", + "@pypi_foo//:pkg", + ] + _select( + { + Label("//python/config_settings:is_python_3.9"): ["@pypi_py39_dep//:pkg"], + "@platforms//cpu:aarch64": ["@pypi_arm_dep//:pkg"], + "@platforms//os:windows": ["@pypi_win_dep//:pkg"], + ":is_python_3.10_linux_ppc64le": ["@pypi_py310_linux_ppc64le_dep//:pkg"], + ":is_python_3.9_anyos_aarch64": ["@pypi_py39_arm_dep//:pkg"], + ":is_python_3.9_linux_anyarch": ["@pypi_py39_linux_dep//:pkg"], + ":is_linux_x86_64": ["@pypi_linux_intel_dep//:pkg"], + "//conditions:default": [], + }, + ), + "tags": ["tag1", "tag2"], + "visibility": ["//visibility:public"], + "experimental_venvs_site_packages": Label("//python/config_settings:venvs_site_packages"), + }) # buildifier: @unsorted-dict-items _tests.append(_test_whl_and_library_deps) @@ -366,6 +384,11 @@ def _test_group(env): alias_calls = [] py_library_calls = [] + mock_glob = _mock_glob() + mock_glob.results.append(["site-packages/foo/srcs.py"]) + mock_glob.results.append(["site-packages/foo/data.txt"]) + mock_glob.results.append(["site-packages/foo/pyi.pyi"]) + whl_library_targets( name = "foo.whl", dep_template = "@pypi_{name}//:{target}", @@ -384,12 +407,13 @@ def _test_group(env): filegroups = {}, native = struct( config_setting = lambda **_: None, - glob = _glob, + glob = mock_glob.glob, alias = lambda **kwargs: alias_calls.append(kwargs), select = _select, ), rules = struct( py_library = lambda **kwargs: py_library_calls.append(kwargs), + create_inits = lambda **kwargs: ["_create_inits_target"], ), ) @@ -397,39 +421,69 @@ def _test_group(env): {"name": "pkg", "actual": "@pypi__groups//:qux_pkg", "visibility": ["//visibility:public"]}, {"name": "whl", "actual": "@pypi__groups//:qux_whl", "visibility": ["//visibility:public"]}, ]) # buildifier: @unsorted-dict-items - env.expect.that_collection(py_library_calls).contains_exactly([ - { - "name": "_pkg", - "srcs": _glob(["site-packages/**/*.py"], exclude = [], allow_empty = True), - "pyi_srcs": _glob(["site-packages/**/*.pyi"], allow_empty = True), - "data": [] + _glob( - ["site-packages/**/*"], - exclude = [ - "**/*.py", - "**/*.pyc", - "**/*.pyc.*", - "**/*.dist-info/RECORD", - ] + glob_excludes.version_dependent_exclusions(), - ), - "imports": ["site-packages"], - "deps": ["@pypi_bar_baz//:pkg"] + _select({ - "@platforms//os:linux": ["@pypi_box//:pkg"], - ":is_linux_x86_64": ["@pypi_box//:pkg", "@pypi_box_amd64//:pkg"], - "//conditions:default": [], - }), - "tags": [], - "visibility": ["@pypi__groups//:__pkg__"], - "experimental_venvs_site_packages": Label("//python/config_settings:venvs_site_packages"), - }, - ]) # buildifier: @unsorted-dict-items + + env.expect.that_collection(py_library_calls).has_size(1) + if len(py_library_calls) != 1: + return + + py_library_call = py_library_calls[0] + env.expect.where(case = "verify py library call").that_dict( + py_library_call, + ).contains_exactly({ + "name": "_pkg", + "srcs": ["site-packages/foo/srcs.py"] + _select({ + Label("//python/config_settings:is_venvs_site_packages"): [], + "//conditions:default": ["_create_inits_target"], + }), + "pyi_srcs": ["site-packages/foo/pyi.pyi"], + "data": ["site-packages/foo/data.txt"], + "imports": ["site-packages"], + "deps": ["@pypi_bar_baz//:pkg"] + _select({ + "@platforms//os:linux": ["@pypi_box//:pkg"], + ":is_linux_x86_64": ["@pypi_box//:pkg", "@pypi_box_amd64//:pkg"], + "//conditions:default": [], + }), + "tags": [], + "visibility": ["@pypi__groups//:__pkg__"], + "experimental_venvs_site_packages": Label("//python/config_settings:venvs_site_packages"), + }) # buildifier: @unsorted-dict-items + + env.expect.that_collection(mock_glob.calls, expr = "glob calls").contains_exactly([ + _glob_call(["site-packages/**/*.py"], exclude = [], allow_empty = True), + _glob_call(["site-packages/**/*"], exclude = [ + "**/*.py", + "**/*.pyc", + "**/*.pyc.*", + "**/*.dist-info/RECORD", + ]), + _glob_call(["site-packages/**/*.pyi"], allow_empty = True), + ]) _tests.append(_test_group) -def _glob(*args, **kwargs): - return [struct( +def _glob_call(*args, **kwargs): + return struct( glob = args, kwargs = kwargs, - )] + ) + +def _mock_glob(): + # buildifier: disable=uninitialized + def glob(*args, **kwargs): + mock.calls.append(_glob_call(*args, **kwargs)) + if not mock.results: + fail("Mock glob missing for invocation: args={} kwargs={}".format( + args, + kwargs, + )) + return mock.results.pop(0) + + mock = struct( + calls = [], + results = [], + glob = glob, + ) + return mock def _select(*args, **kwargs): """We need to have this mock select because we still need to support bazel 6.""" From 66963b9d7a2a51fd797c8c2251ff424a77d0db90 Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Mon, 7 Jul 2025 03:30:42 +0000 Subject: [PATCH 318/922] build(deps): bump pygments from 2.19.1 to 2.19.2 in /docs (#3019) Bumps [pygments](https://github.com/pygments/pygments) from 2.19.1 to 2.19.2.
Release notes

Sourced from pygments's releases.

2.19.2

  • Lua: Fix regression introduced in 2.19.0 (#2882, #2839)
Changelog

Sourced from pygments's changelog.

Version 2.19.2

(released June 21st, 2025)

  • Lua: Fix regression introduced in 2.19.0 (#2882, #2839)
Commits

[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=pygments&package-manager=pip&previous-version=2.19.1&new-version=2.19.2)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) ---
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot show ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> --- docs/requirements.txt | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/requirements.txt b/docs/requirements.txt index d351e0e946..26ba49ecab 100644 --- a/docs/requirements.txt +++ b/docs/requirements.txt @@ -228,9 +228,9 @@ packaging==25.0 \ # via # readthedocs-sphinx-ext # sphinx -pygments==2.19.1 \ - --hash=sha256:61c16d2a8576dc0649d9f39e089b5f02bcd27fba10d8fb4dcc28173f7a45151f \ - --hash=sha256:9ea1544ad55cecf4b8242fab6dd35a93bbce657034b0611ee383099054ab6d8c +pygments==2.19.2 \ + --hash=sha256:636cb2477cec7f8952536970bc533bc43743542f70392ae026374600add5b887 \ + --hash=sha256:86540386c03d588bb81d44bc3928634ff26449851e99741617ecb9037ee5ec0b # via sphinx pyyaml==6.0.2 \ --hash=sha256:01179a4a8559ab5de078078f37e5c1a30d76bb88519906844fd7bdea1b7729ff \ From 998e22e68b3cc775b1cfd337ab9bad0550dab2cb Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Mon, 7 Jul 2025 03:31:04 +0000 Subject: [PATCH 319/922] build(deps): bump charset-normalizer from 3.4.1 to 3.4.2 in /docs (#3018) Bumps [charset-normalizer](https://github.com/jawah/charset_normalizer) from 3.4.1 to 3.4.2.
Release notes

Sourced from charset-normalizer's releases.

Version 3.4.2

3.4.2 (2025-05-02)

Fixed

  • Addressed the DeprecationWarning in our CLI regarding argparse.FileType by backporting the target class into the package. (#591)
  • Improved the overall reliability of the detector with CJK Ideographs. (#605) (#587)

Changed

  • Optional mypyc compilation upgraded to version 1.15 for Python >= 3.9
Changelog

Sourced from charset-normalizer's changelog.

3.4.2 (2025-05-02)

Fixed

  • Addressed the DeprecationWarning in our CLI regarding argparse.FileType by backporting the target class into the package. (#591)
  • Improved the overall reliability of the detector with CJK Ideographs. (#605) (#587)

Changed

  • Optional mypyc compilation upgraded to version 1.15 for Python >= 3.8
Commits
  • 6422af1 :pencil: update release date
  • 0e60ec1 :bookmark: Release 3.4.2 (#614)
  • f6630ce :arrow_up: Bump pypa/cibuildwheel from 2.23.2 to 2.23.3 (#617)
  • 677c999 :arrow_up: Bump actions/download-artifact from 4.2.1 to 4.3.0 (#618)
  • 960ab1e :arrow_up: Bump actions/setup-python from 5.5.0 to 5.6.0 (#619)
  • 6eb6325 :arrow_up: Bump github/codeql-action from 3.28.10 to 3.28.16 (#620)
  • c99c0f2 :arrow_up: Update coverage requirement from <7.7,>=7.2.7 to >=7.2.7,<7.9 (#606)
  • 270f28e :arrow_up: Bump actions/setup-python from 5.4.0 to 5.5.0 (#607)
  • d4d89a0 :arrow_up: Bump pypa/cibuildwheel from 2.22.0 to 2.23.2 (#608)
  • 905fcf5 :arrow_up: Bump slsa-framework/slsa-github-generator from 2.0.0 to 2.1.0 (#609)
  • Additional commits viewable in compare view

[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=charset-normalizer&package-manager=pip&previous-version=3.4.1&new-version=3.4.2)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) ---
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot show ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> --- docs/requirements.txt | 186 +++++++++++++++++++++--------------------- 1 file changed, 93 insertions(+), 93 deletions(-) diff --git a/docs/requirements.txt b/docs/requirements.txt index 26ba49ecab..7a32ff7716 100644 --- a/docs/requirements.txt +++ b/docs/requirements.txt @@ -21,99 +21,99 @@ certifi==2025.6.15 \ --hash=sha256:2e0c7ce7cb5d8f8634ca55d2ba7e6ec2689a2fd6537d8dec1296a477a4910057 \ --hash=sha256:d747aa5a8b9bbbb1bb8c22bb13e22bd1f18e9796defa16bab421f7f7a317323b # via requests -charset-normalizer==3.4.1 \ - --hash=sha256:0167ddc8ab6508fe81860a57dd472b2ef4060e8d378f0cc555707126830f2537 \ - --hash=sha256:01732659ba9b5b873fc117534143e4feefecf3b2078b0a6a2e925271bb6f4cfa \ - --hash=sha256:01ad647cdd609225c5350561d084b42ddf732f4eeefe6e678765636791e78b9a \ - --hash=sha256:04432ad9479fa40ec0f387795ddad4437a2b50417c69fa275e212933519ff294 \ - --hash=sha256:0907f11d019260cdc3f94fbdb23ff9125f6b5d1039b76003b5b0ac9d6a6c9d5b \ - --hash=sha256:0924e81d3d5e70f8126529951dac65c1010cdf117bb75eb02dd12339b57749dd \ - --hash=sha256:09b26ae6b1abf0d27570633b2b078a2a20419c99d66fb2823173d73f188ce601 \ - --hash=sha256:09b5e6733cbd160dcc09589227187e242a30a49ca5cefa5a7edd3f9d19ed53fd \ - --hash=sha256:0af291f4fe114be0280cdd29d533696a77b5b49cfde5467176ecab32353395c4 \ - --hash=sha256:0f55e69f030f7163dffe9fd0752b32f070566451afe180f99dbeeb81f511ad8d \ - --hash=sha256:1a2bc9f351a75ef49d664206d51f8e5ede9da246602dc2d2726837620ea034b2 \ - --hash=sha256:22e14b5d70560b8dd51ec22863f370d1e595ac3d024cb8ad7d308b4cd95f8313 \ - --hash=sha256:234ac59ea147c59ee4da87a0c0f098e9c8d169f4dc2a159ef720f1a61bbe27cd \ - --hash=sha256:2369eea1ee4a7610a860d88f268eb39b95cb588acd7235e02fd5a5601773d4fa \ - --hash=sha256:237bdbe6159cff53b4f24f397d43c6336c6b0b42affbe857970cefbb620911c8 \ - --hash=sha256:28bf57629c75e810b6ae989f03c0828d64d6b26a5e205535585f96093e405ed1 \ - --hash=sha256:2967f74ad52c3b98de4c3b32e1a44e32975e008a9cd2a8cc8966d6a5218c5cb2 \ - --hash=sha256:2a75d49014d118e4198bcee5ee0a6f25856b29b12dbf7cd012791f8a6cc5c496 \ - --hash=sha256:2bdfe3ac2e1bbe5b59a1a63721eb3b95fc9b6817ae4a46debbb4e11f6232428d \ - --hash=sha256:2d074908e1aecee37a7635990b2c6d504cd4766c7bc9fc86d63f9c09af3fa11b \ - --hash=sha256:2fb9bd477fdea8684f78791a6de97a953c51831ee2981f8e4f583ff3b9d9687e \ - --hash=sha256:311f30128d7d333eebd7896965bfcfbd0065f1716ec92bd5638d7748eb6f936a \ - --hash=sha256:329ce159e82018d646c7ac45b01a430369d526569ec08516081727a20e9e4af4 \ - --hash=sha256:345b0426edd4e18138d6528aed636de7a9ed169b4aaf9d61a8c19e39d26838ca \ - --hash=sha256:363e2f92b0f0174b2f8238240a1a30142e3db7b957a5dd5689b0e75fb717cc78 \ - --hash=sha256:3a3bd0dcd373514dcec91c411ddb9632c0d7d92aed7093b8c3bbb6d69ca74408 \ - --hash=sha256:3bed14e9c89dcb10e8f3a29f9ccac4955aebe93c71ae803af79265c9ca5644c5 \ - --hash=sha256:44251f18cd68a75b56585dd00dae26183e102cd5e0f9f1466e6df5da2ed64ea3 \ - --hash=sha256:44ecbf16649486d4aebafeaa7ec4c9fed8b88101f4dd612dcaf65d5e815f837f \ - --hash=sha256:4532bff1b8421fd0a320463030c7520f56a79c9024a4e88f01c537316019005a \ - --hash=sha256:49402233c892a461407c512a19435d1ce275543138294f7ef013f0b63d5d3765 \ - --hash=sha256:4c0907b1928a36d5a998d72d64d8eaa7244989f7aaaf947500d3a800c83a3fd6 \ - --hash=sha256:4d86f7aff21ee58f26dcf5ae81a9addbd914115cdebcbb2217e4f0ed8982e146 \ - --hash=sha256:5777ee0881f9499ed0f71cc82cf873d9a0ca8af166dfa0af8ec4e675b7df48e6 \ - --hash=sha256:5df196eb874dae23dcfb968c83d4f8fdccb333330fe1fc278ac5ceeb101003a9 \ - --hash=sha256:619a609aa74ae43d90ed2e89bdd784765de0a25ca761b93e196d938b8fd1dbbd \ - --hash=sha256:6e27f48bcd0957c6d4cb9d6fa6b61d192d0b13d5ef563e5f2ae35feafc0d179c \ - --hash=sha256:6ff8a4a60c227ad87030d76e99cd1698345d4491638dfa6673027c48b3cd395f \ - --hash=sha256:73d94b58ec7fecbc7366247d3b0b10a21681004153238750bb67bd9012414545 \ - --hash=sha256:7461baadb4dc00fd9e0acbe254e3d7d2112e7f92ced2adc96e54ef6501c5f176 \ - --hash=sha256:75832c08354f595c760a804588b9357d34ec00ba1c940c15e31e96d902093770 \ - --hash=sha256:7709f51f5f7c853f0fb938bcd3bc59cdfdc5203635ffd18bf354f6967ea0f824 \ - --hash=sha256:78baa6d91634dfb69ec52a463534bc0df05dbd546209b79a3880a34487f4b84f \ - --hash=sha256:7974a0b5ecd505609e3b19742b60cee7aa2aa2fb3151bc917e6e2646d7667dcf \ - --hash=sha256:7a4f97a081603d2050bfaffdefa5b02a9ec823f8348a572e39032caa8404a487 \ - --hash=sha256:7b1bef6280950ee6c177b326508f86cad7ad4dff12454483b51d8b7d673a2c5d \ - --hash=sha256:7d053096f67cd1241601111b698f5cad775f97ab25d81567d3f59219b5f1adbd \ - --hash=sha256:804a4d582ba6e5b747c625bf1255e6b1507465494a40a2130978bda7b932c90b \ - --hash=sha256:807f52c1f798eef6cf26beb819eeb8819b1622ddfeef9d0977a8502d4db6d534 \ - --hash=sha256:80ed5e856eb7f30115aaf94e4a08114ccc8813e6ed1b5efa74f9f82e8509858f \ - --hash=sha256:8417cb1f36cc0bc7eaba8ccb0e04d55f0ee52df06df3ad55259b9a323555fc8b \ - --hash=sha256:8436c508b408b82d87dc5f62496973a1805cd46727c34440b0d29d8a2f50a6c9 \ - --hash=sha256:89149166622f4db9b4b6a449256291dc87a99ee53151c74cbd82a53c8c2f6ccd \ - --hash=sha256:8bfa33f4f2672964266e940dd22a195989ba31669bd84629f05fab3ef4e2d125 \ - --hash=sha256:8c60ca7339acd497a55b0ea5d506b2a2612afb2826560416f6894e8b5770d4a9 \ - --hash=sha256:91b36a978b5ae0ee86c394f5a54d6ef44db1de0815eb43de826d41d21e4af3de \ - --hash=sha256:955f8851919303c92343d2f66165294848d57e9bba6cf6e3625485a70a038d11 \ - --hash=sha256:97f68b8d6831127e4787ad15e6757232e14e12060bec17091b85eb1486b91d8d \ - --hash=sha256:9b23ca7ef998bc739bf6ffc077c2116917eabcc901f88da1b9856b210ef63f35 \ - --hash=sha256:9f0b8b1c6d84c8034a44893aba5e767bf9c7a211e313a9605d9c617d7083829f \ - --hash=sha256:aabfa34badd18f1da5ec1bc2715cadc8dca465868a4e73a0173466b688f29dda \ - --hash=sha256:ab36c8eb7e454e34e60eb55ca5d241a5d18b2c6244f6827a30e451c42410b5f7 \ - --hash=sha256:b010a7a4fd316c3c484d482922d13044979e78d1861f0e0650423144c616a46a \ - --hash=sha256:b1ac5992a838106edb89654e0aebfc24f5848ae2547d22c2c3f66454daa11971 \ - --hash=sha256:b7b2d86dd06bfc2ade3312a83a5c364c7ec2e3498f8734282c6c3d4b07b346b8 \ - --hash=sha256:b97e690a2118911e39b4042088092771b4ae3fc3aa86518f84b8cf6888dbdb41 \ - --hash=sha256:bc2722592d8998c870fa4e290c2eec2c1569b87fe58618e67d38b4665dfa680d \ - --hash=sha256:c0429126cf75e16c4f0ad00ee0eae4242dc652290f940152ca8c75c3a4b6ee8f \ - --hash=sha256:c30197aa96e8eed02200a83fba2657b4c3acd0f0aa4bdc9f6c1af8e8962e0757 \ - --hash=sha256:c4c3e6da02df6fa1410a7680bd3f63d4f710232d3139089536310d027950696a \ - --hash=sha256:c75cb2a3e389853835e84a2d8fb2b81a10645b503eca9bcb98df6b5a43eb8886 \ - --hash=sha256:c96836c97b1238e9c9e3fe90844c947d5afbf4f4c92762679acfe19927d81d77 \ - --hash=sha256:d7f50a1f8c450f3925cb367d011448c39239bb3eb4117c36a6d354794de4ce76 \ - --hash=sha256:d973f03c0cb71c5ed99037b870f2be986c3c05e63622c017ea9816881d2dd247 \ - --hash=sha256:d98b1668f06378c6dbefec3b92299716b931cd4e6061f3c875a71ced1780ab85 \ - --hash=sha256:d9c3cdf5390dcd29aa8056d13e8e99526cda0305acc038b96b30352aff5ff2bb \ - --hash=sha256:dad3e487649f498dd991eeb901125411559b22e8d7ab25d3aeb1af367df5efd7 \ - --hash=sha256:dccbe65bd2f7f7ec22c4ff99ed56faa1e9f785482b9bbd7c717e26fd723a1d1e \ - --hash=sha256:dd78cfcda14a1ef52584dbb008f7ac81c1328c0f58184bf9a84c49c605002da6 \ - --hash=sha256:e218488cd232553829be0664c2292d3af2eeeb94b32bea483cf79ac6a694e037 \ - --hash=sha256:e358e64305fe12299a08e08978f51fc21fac060dcfcddd95453eabe5b93ed0e1 \ - --hash=sha256:ea0d8d539afa5eb2728aa1932a988a9a7af94f18582ffae4bc10b3fbdad0626e \ - --hash=sha256:eab677309cdb30d047996b36d34caeda1dc91149e4fdca0b1a039b3f79d9a807 \ - --hash=sha256:eb8178fe3dba6450a3e024e95ac49ed3400e506fd4e9e5c32d30adda88cbd407 \ - --hash=sha256:ecddf25bee22fe4fe3737a399d0d177d72bc22be6913acfab364b40bce1ba83c \ - --hash=sha256:eea6ee1db730b3483adf394ea72f808b6e18cf3cb6454b4d86e04fa8c4327a12 \ - --hash=sha256:f08ff5e948271dc7e18a35641d2f11a4cd8dfd5634f55228b691e62b37125eb3 \ - --hash=sha256:f30bf9fd9be89ecb2360c7d94a711f00c09b976258846efe40db3d05828e8089 \ - --hash=sha256:fa88b843d6e211393a37219e6a1c1df99d35e8fd90446f1118f4216e307e48cd \ - --hash=sha256:fc54db6c8593ef7d4b2a331b58653356cf04f67c960f584edb7c3d8c97e8f39e \ - --hash=sha256:fd4ec41f914fa74ad1b8304bbc634b3de73d2a0889bd32076342a573e0779e00 \ - --hash=sha256:ffc9202a29ab3920fa812879e95a9e78b2465fd10be7fcbd042899695d75e616 +charset-normalizer==3.4.2 \ + --hash=sha256:005fa3432484527f9732ebd315da8da8001593e2cf46a3d817669f062c3d9ed4 \ + --hash=sha256:046595208aae0120559a67693ecc65dd75d46f7bf687f159127046628178dc45 \ + --hash=sha256:0c29de6a1a95f24b9a1aa7aefd27d2487263f00dfd55a77719b530788f75cff7 \ + --hash=sha256:0c8c57f84ccfc871a48a47321cfa49ae1df56cd1d965a09abe84066f6853b9c0 \ + --hash=sha256:0f5d9ed7f254402c9e7d35d2f5972c9bbea9040e99cd2861bd77dc68263277c7 \ + --hash=sha256:18dd2e350387c87dabe711b86f83c9c78af772c748904d372ade190b5c7c9d4d \ + --hash=sha256:1b1bde144d98e446b056ef98e59c256e9294f6b74d7af6846bf5ffdafd687a7d \ + --hash=sha256:1c95a1e2902a8b722868587c0e1184ad5c55631de5afc0eb96bc4b0d738092c0 \ + --hash=sha256:1cad5f45b3146325bb38d6855642f6fd609c3f7cad4dbaf75549bf3b904d3184 \ + --hash=sha256:21b2899062867b0e1fde9b724f8aecb1af14f2778d69aacd1a5a1853a597a5db \ + --hash=sha256:24498ba8ed6c2e0b56d4acbf83f2d989720a93b41d712ebd4f4979660db4417b \ + --hash=sha256:25a23ea5c7edc53e0f29bae2c44fcb5a1aa10591aae107f2a2b2583a9c5cbc64 \ + --hash=sha256:289200a18fa698949d2b39c671c2cc7a24d44096784e76614899a7ccf2574b7b \ + --hash=sha256:28a1005facc94196e1fb3e82a3d442a9d9110b8434fc1ded7a24a2983c9888d8 \ + --hash=sha256:32fc0341d72e0f73f80acb0a2c94216bd704f4f0bce10aedea38f30502b271ff \ + --hash=sha256:36b31da18b8890a76ec181c3cf44326bf2c48e36d393ca1b72b3f484113ea344 \ + --hash=sha256:3c21d4fca343c805a52c0c78edc01e3477f6dd1ad7c47653241cf2a206d4fc58 \ + --hash=sha256:3fddb7e2c84ac87ac3a947cb4e66d143ca5863ef48e4a5ecb83bd48619e4634e \ + --hash=sha256:43e0933a0eff183ee85833f341ec567c0980dae57c464d8a508e1b2ceb336471 \ + --hash=sha256:4a476b06fbcf359ad25d34a057b7219281286ae2477cc5ff5e3f70a246971148 \ + --hash=sha256:4e594135de17ab3866138f496755f302b72157d115086d100c3f19370839dd3a \ + --hash=sha256:50bf98d5e563b83cc29471fa114366e6806bc06bc7a25fd59641e41445327836 \ + --hash=sha256:5a9979887252a82fefd3d3ed2a8e3b937a7a809f65dcb1e068b090e165bbe99e \ + --hash=sha256:5baececa9ecba31eff645232d59845c07aa030f0c81ee70184a90d35099a0e63 \ + --hash=sha256:5bf4545e3b962767e5c06fe1738f951f77d27967cb2caa64c28be7c4563e162c \ + --hash=sha256:6333b3aa5a12c26b2a4d4e7335a28f1475e0e5e17d69d55141ee3cab736f66d1 \ + --hash=sha256:65c981bdbd3f57670af8b59777cbfae75364b483fa8a9f420f08094531d54a01 \ + --hash=sha256:68a328e5f55ec37c57f19ebb1fdc56a248db2e3e9ad769919a58672958e8f366 \ + --hash=sha256:6a0289e4589e8bdfef02a80478f1dfcb14f0ab696b5a00e1f4b8a14a307a3c58 \ + --hash=sha256:6b66f92b17849b85cad91259efc341dce9c1af48e2173bf38a85c6329f1033e5 \ + --hash=sha256:6c9379d65defcab82d07b2a9dfbfc2e95bc8fe0ebb1b176a3190230a3ef0e07c \ + --hash=sha256:6fc1f5b51fa4cecaa18f2bd7a003f3dd039dd615cd69a2afd6d3b19aed6775f2 \ + --hash=sha256:70f7172939fdf8790425ba31915bfbe8335030f05b9913d7ae00a87d4395620a \ + --hash=sha256:721c76e84fe669be19c5791da68232ca2e05ba5185575086e384352e2c309597 \ + --hash=sha256:7222ffd5e4de8e57e03ce2cef95a4c43c98fcb72ad86909abdfc2c17d227fc1b \ + --hash=sha256:75d10d37a47afee94919c4fab4c22b9bc2a8bf7d4f46f87363bcf0573f3ff4f5 \ + --hash=sha256:76af085e67e56c8816c3ccf256ebd136def2ed9654525348cfa744b6802b69eb \ + --hash=sha256:770cab594ecf99ae64c236bc9ee3439c3f46be49796e265ce0cc8bc17b10294f \ + --hash=sha256:7a6ab32f7210554a96cd9e33abe3ddd86732beeafc7a28e9955cdf22ffadbab0 \ + --hash=sha256:7c48ed483eb946e6c04ccbe02c6b4d1d48e51944b6db70f697e089c193404941 \ + --hash=sha256:7f56930ab0abd1c45cd15be65cc741c28b1c9a34876ce8c17a2fa107810c0af0 \ + --hash=sha256:8075c35cd58273fee266c58c0c9b670947c19df5fb98e7b66710e04ad4e9ff86 \ + --hash=sha256:8272b73e1c5603666618805fe821edba66892e2870058c94c53147602eab29c7 \ + --hash=sha256:82d8fd25b7f4675d0c47cf95b594d4e7b158aca33b76aa63d07186e13c0e0ab7 \ + --hash=sha256:844da2b5728b5ce0e32d863af26f32b5ce61bc4273a9c720a9f3aa9df73b1455 \ + --hash=sha256:8755483f3c00d6c9a77f490c17e6ab0c8729e39e6390328e42521ef175380ae6 \ + --hash=sha256:915f3849a011c1f593ab99092f3cecfcb4d65d8feb4a64cf1bf2d22074dc0ec4 \ + --hash=sha256:926ca93accd5d36ccdabd803392ddc3e03e6d4cd1cf17deff3b989ab8e9dbcf0 \ + --hash=sha256:982bb1e8b4ffda883b3d0a521e23abcd6fd17418f6d2c4118d257a10199c0ce3 \ + --hash=sha256:98f862da73774290f251b9df8d11161b6cf25b599a66baf087c1ffe340e9bfd1 \ + --hash=sha256:9cbfacf36cb0ec2897ce0ebc5d08ca44213af24265bd56eca54bee7923c48fd6 \ + --hash=sha256:a370b3e078e418187da8c3674eddb9d983ec09445c99a3a263c2011993522981 \ + --hash=sha256:a955b438e62efdf7e0b7b52a64dc5c3396e2634baa62471768a64bc2adb73d5c \ + --hash=sha256:aa6af9e7d59f9c12b33ae4e9450619cf2488e2bbe9b44030905877f0b2324980 \ + --hash=sha256:aa88ca0b1932e93f2d961bf3addbb2db902198dca337d88c89e1559e066e7645 \ + --hash=sha256:aaeeb6a479c7667fbe1099af9617c83aaca22182d6cf8c53966491a0f1b7ffb7 \ + --hash=sha256:aaf27faa992bfee0264dc1f03f4c75e9fcdda66a519db6b957a3f826e285cf12 \ + --hash=sha256:b2680962a4848b3c4f155dc2ee64505a9c57186d0d56b43123b17ca3de18f0fa \ + --hash=sha256:b2d318c11350e10662026ad0eb71bb51c7812fc8590825304ae0bdd4ac283acd \ + --hash=sha256:b33de11b92e9f75a2b545d6e9b6f37e398d86c3e9e9653c4864eb7e89c5773ef \ + --hash=sha256:b3daeac64d5b371dea99714f08ffc2c208522ec6b06fbc7866a450dd446f5c0f \ + --hash=sha256:be1e352acbe3c78727a16a455126d9ff83ea2dfdcbc83148d2982305a04714c2 \ + --hash=sha256:bee093bf902e1d8fc0ac143c88902c3dfc8941f7ea1d6a8dd2bcb786d33db03d \ + --hash=sha256:c72fbbe68c6f32f251bdc08b8611c7b3060612236e960ef848e0a517ddbe76c5 \ + --hash=sha256:c9e36a97bee9b86ef9a1cf7bb96747eb7a15c2f22bdb5b516434b00f2a599f02 \ + --hash=sha256:cddf7bd982eaa998934a91f69d182aec997c6c468898efe6679af88283b498d3 \ + --hash=sha256:cf713fe9a71ef6fd5adf7a79670135081cd4431c2943864757f0fa3a65b1fafd \ + --hash=sha256:d11b54acf878eef558599658b0ffca78138c8c3655cf4f3a4a673c437e67732e \ + --hash=sha256:d41c4d287cfc69060fa91cae9683eacffad989f1a10811995fa309df656ec214 \ + --hash=sha256:d524ba3f1581b35c03cb42beebab4a13e6cdad7b36246bd22541fa585a56cccd \ + --hash=sha256:daac4765328a919a805fa5e2720f3e94767abd632ae410a9062dff5412bae65a \ + --hash=sha256:db4c7bf0e07fc3b7d89ac2a5880a6a8062056801b83ff56d8464b70f65482b6c \ + --hash=sha256:dc7039885fa1baf9be153a0626e337aa7ec8bf96b0128605fb0d77788ddc1681 \ + --hash=sha256:dccab8d5fa1ef9bfba0590ecf4d46df048d18ffe3eec01eeb73a42e0d9e7a8ba \ + --hash=sha256:dedb8adb91d11846ee08bec4c8236c8549ac721c245678282dcb06b221aab59f \ + --hash=sha256:e45ba65510e2647721e35323d6ef54c7974959f6081b58d4ef5d87c60c84919a \ + --hash=sha256:e53efc7c7cee4c1e70661e2e112ca46a575f90ed9ae3fef200f2a25e954f4b28 \ + --hash=sha256:e635b87f01ebc977342e2697d05b56632f5f879a4f15955dfe8cef2448b51691 \ + --hash=sha256:e70e990b2137b29dc5564715de1e12701815dacc1d056308e2b17e9095372a82 \ + --hash=sha256:e8082b26888e2f8b36a042a58307d5b917ef2b1cacab921ad3323ef91901c71a \ + --hash=sha256:e8323a9b031aa0393768b87f04b4164a40037fb2a3c11ac06a03ffecd3618027 \ + --hash=sha256:e92fca20c46e9f5e1bb485887d074918b13543b1c2a1185e69bb8d17ab6236a7 \ + --hash=sha256:eb30abc20df9ab0814b5a2524f23d75dcf83cde762c161917a2b4b7b55b1e518 \ + --hash=sha256:eba9904b0f38a143592d9fc0e19e2df0fa2e41c3c3745554761c5f6447eedabf \ + --hash=sha256:ef8de666d6179b009dce7bcb2ad4c4a779f113f12caf8dc77f0162c29d20490b \ + --hash=sha256:efd387a49825780ff861998cd959767800d54f8308936b21025326de4b5a42b9 \ + --hash=sha256:f0aa37f3c979cf2546b73e8222bbfa3dc07a641585340179d768068e3455e544 \ + --hash=sha256:f4074c5a429281bf056ddd4c5d3b740ebca4d43ffffe2ef4bf4d2d05114299da \ + --hash=sha256:f69a27e45c43520f5487f27627059b64aaf160415589230992cec34c5e18a509 \ + --hash=sha256:fb707f3e15060adf5b7ada797624a6c6e0138e2a26baa089df64c68ee98e040f \ + --hash=sha256:fcbe676a55d7445b22c10967bceaaf0ee69407fbe0ece4d032b6eb8d4565982a \ + --hash=sha256:fdb20a30fe1175ecabed17cbf7812f7b804b8a315a25f24678bcdf120a90077f # via requests colorama==0.4.6 ; sys_platform == 'win32' \ --hash=sha256:08695f5cb7ed6e0531a20572697297273c47b8cae5a63ffc6d6ed5c201be6e44 \ From b2c39269d3977e21b47cdfa69d0f9a0bec8f6482 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Mon, 7 Jul 2025 07:04:43 -0700 Subject: [PATCH 320/922] fix: parsing local version with digit followed by non-digits (#3032) When parsing the local identifier segment of `` the parser would give an error saying the letter was unexpected. What was happening was `accept_digits()` consumed up to the first non-digit, and considered this success, which prevented calling `accept_alnum()` to finish the parsing. To fix, only call `accept_alnum()`, then post-process the value to normalize an all-digit segment. I'm guessing `accept_digits()` stopping at the first non-digit is WAI because it expects to parse e.g. "3.14b", where the caller handles subsequent characters. Along the way, some minor doc improvements to the parser code. Fixes https://github.com/bazel-contrib/rules_python/issues/3030 --- python/private/version.bzl | 47 +++++++++++++++++++++++++++------- tests/version/version_test.bzl | 8 ++++++ 2 files changed, 46 insertions(+), 9 deletions(-) diff --git a/python/private/version.bzl b/python/private/version.bzl index f98165d391..8b5fef7b2a 100644 --- a/python/private/version.bzl +++ b/python/private/version.bzl @@ -44,7 +44,13 @@ def _in(reference): return lambda token: token in reference def _ctx(start): - return {"norm": "", "start": start} + """Creates a context, which is state for parsing (or sub-parsing).""" + return { + # The result value from parsing + "norm": "", + # Where in the parser's input string this context starts. + "start": start, + } def _open_context(self): """Open an new parsing ctx. @@ -60,7 +66,16 @@ def _open_context(self): return self.contexts[-1] def _accept(self, key = None): - """Close the current ctx successfully and merge the results.""" + """Close the current ctx successfully and merge the results. + + Args: + self: {type}`Parser} + key: {type}`str | None` the key to store the result in + the most recent context. If not set, the key is "norm". + + Returns: + {type}`bool` always True + """ finished = self.contexts.pop() self.contexts[-1]["norm"] += finished["norm"] if key: @@ -79,7 +94,14 @@ def _discard(self, key = None): return False def _new(input): - """Create a new normalizer""" + """Create a new parser + + Args: + input: {type}`str` input to parse + + Returns: + {type}`Parser` a struct for a parser object. + """ self = struct( input = input, contexts = [_ctx(0)], @@ -167,7 +189,7 @@ def accept_placeholder(parser): return parser.accept() def accept_digits(parser): - """Accept multiple digits (or placeholders). + """Accept multiple digits (or placeholders), up to a non-digit/placeholder. Args: parser: The normalizer. @@ -275,13 +297,20 @@ def accept_separator_alnum(parser): Returns: whether a separator and an alphanumeric string were accepted. """ - parser.open_context() + ctx = parser.open_context() # PEP 440: Local version segments - if ( - accept(parser, _in([".", "-", "_"]), ".") and - (accept_digits(parser) or accept_alnum(parser)) - ): + if not accept(parser, _in([".", "-", "_"]), "."): + return parser.discard() + + if accept_alnum(parser): + # First character is separator; skip it. + value = ctx["norm"][1:] + + # PEP 440: Integer Normalization + if value.isdigit(): + value = str(int(value)) + ctx["norm"] = ctx["norm"][0] + value return parser.accept() return parser.discard() diff --git a/tests/version/version_test.bzl b/tests/version/version_test.bzl index 589f9ac05d..7ddb6cc851 100644 --- a/tests/version/version_test.bzl +++ b/tests/version/version_test.bzl @@ -105,6 +105,14 @@ def _test_normalization(env): _tests.append(_test_normalization) +def _test_normalize_local(env): + # Verify a local with a [digit][non-digit] sequence parses ok + in_str = "0.1.0+brt.9e" + actual = version.normalize(in_str) + env.expect.that_str(actual).equals(in_str) + +_tests.append(_test_normalize_local) + def _test_ordering(env): want = [ # Taken from https://peps.python.org/pep-0440/#summary-of-permitted-suffixes-and-relative-ordering From cdf4f55badbaba432058e8fb0f097a25df6af656 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Mon, 7 Jul 2025 07:05:21 -0700 Subject: [PATCH 321/922] feat(pypi): generate filegroup with all extracted wheel files (#3011) Adds a filegroup with all the files that came from the extracted wheel. This has two benefits over using `whl_filegroup`: it avoids copying the wheel and makes the set of files directly visible to the analysis phase. Some wheels are multiple gigabytes in size (e.g. torch, cuda, tensorflow), so avoiding the copy and archive processing saves a decent amount of time. Knowing the specific files at analysis time is generally beneficial. The particular case I ran into was the CC rules were unhappy with a TreeArtifact of header files because they couldn't enforce some check about who was properly providing headers that were included (layering check?). Another example is using the unused_inputs_list optimization, which allows an action to ignore inputs that aren't actually used. e.g. an action could take all the wheel's files as inputs, only care about the headers, and then tell bazel all the non-header files aren't relevant, and thus changes to other files don't re-run the thing that only cares about headers. --------- Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com> --- CHANGELOG.md | 4 ++ docs/pypi/use.md | 7 +++ python/private/pypi/labels.bzl | 1 + python/private/pypi/pkg_aliases.bzl | 2 + python/private/pypi/whl_library.bzl | 5 +++ python/private/pypi/whl_library_targets.bzl | 45 +++++++++++++++---- .../private/whl_filegroup/whl_filegroup.bzl | 7 +++ tests/pypi/pkg_aliases/pkg_aliases_test.bzl | 5 +++ .../whl_library_targets_tests.bzl | 12 +++-- 9 files changed, 77 insertions(+), 11 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 2b57af606e..c1d3a43814 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -90,6 +90,10 @@ END_UNRELEASED_TEMPLATE * (pypi) To configure the environment for `requirements.txt` evaluation, use the newly added developer preview of the `pip.default` tag class. Only `rules_python` and root modules can use this feature. You can also configure custom `config_settings` using `pip.default`. +* (pypi) PyPI dependencies now expose an `:extracted_whl_files` filegroup target + of all the files extracted from the wheel. This can be used in lieu of + {obj}`whl_filegroup` to avoid copying/extracting wheel multiple times to + get a subset of their files. * (gazelle) New directive `gazelle:python_generate_pyi_deps`; when `true`, dependencies added to satisfy type-only imports (`if TYPE_CHECKING`) and type stub packages are added to `pyi_deps` instead of `deps`. diff --git a/docs/pypi/use.md b/docs/pypi/use.md index 6212097f86..a668167114 100644 --- a/docs/pypi/use.md +++ b/docs/pypi/use.md @@ -40,9 +40,16 @@ Note that the hub repo contains the following targets for each package: * `@pypi//numpy:data` - the {obj}`filegroup` for all of the extra files that are included as data in the `pkg` target. * `@pypi//numpy:dist_info` - the {obj}`filegroup` for all of the files in the `.distinfo` directory. +* `@pypi//numpy:extracted_whl_files` - a {obj}`filegroup` of all the files + extracted from the whl file. * `@pypi//numpy:whl` - the {obj}`filegroup` that is the `.whl` file itself, which includes all transitive dependencies via the {attr}`filegroup.data` attribute. +:::{versionadded} VERSION_NEXT_FEATURE + +The `:extracted_whl_files` target was added +::: + ## Entry points If you would like to access [entry points][whl_ep], see the `py_console_script_binary` rule documentation, diff --git a/python/private/pypi/labels.bzl b/python/private/pypi/labels.bzl index 73df07b2d2..22161b1496 100644 --- a/python/private/pypi/labels.bzl +++ b/python/private/pypi/labels.bzl @@ -14,6 +14,7 @@ """Constants used by parts of pip_repository for naming libraries and wheels.""" +EXTRACTED_WHEEL_FILES = "extracted_whl_files" WHEEL_FILE_PUBLIC_LABEL = "whl" WHEEL_FILE_IMPL_LABEL = "_whl" PY_LIBRARY_PUBLIC_LABEL = "pkg" diff --git a/python/private/pypi/pkg_aliases.bzl b/python/private/pypi/pkg_aliases.bzl index d71c37cb4b..4d3cc61590 100644 --- a/python/private/pypi/pkg_aliases.bzl +++ b/python/private/pypi/pkg_aliases.bzl @@ -79,6 +79,7 @@ load( ":labels.bzl", "DATA_LABEL", "DIST_INFO_LABEL", + "EXTRACTED_WHEEL_FILES", "PY_LIBRARY_IMPL_LABEL", "PY_LIBRARY_PUBLIC_LABEL", "WHEEL_FILE_IMPL_LABEL", @@ -151,6 +152,7 @@ def pkg_aliases( WHEEL_FILE_PUBLIC_LABEL: WHEEL_FILE_IMPL_LABEL if group_name else WHEEL_FILE_PUBLIC_LABEL, DATA_LABEL: DATA_LABEL, DIST_INFO_LABEL: DIST_INFO_LABEL, + EXTRACTED_WHEEL_FILES: EXTRACTED_WHEEL_FILES, } | { x: x for x in extra_aliases or [] diff --git a/python/private/pypi/whl_library.bzl b/python/private/pypi/whl_library.bzl index de5fcb9f91..15bb680fea 100644 --- a/python/private/pypi/whl_library.bzl +++ b/python/private/pypi/whl_library.bzl @@ -248,6 +248,7 @@ def _whl_library_impl(rctx): environment = _create_repository_execution_environment(rctx, python_interpreter, logger = logger) whl_path = None + sdist_filename = None if rctx.attr.whl_file: rctx.watch(rctx.attr.whl_file) whl_path = rctx.path(rctx.attr.whl_file) @@ -277,6 +278,8 @@ def _whl_library_impl(rctx): if filename.endswith(".whl"): whl_path = rctx.path(filename) else: + sdist_filename = filename + # It is an sdist and we need to tell PyPI to use a file in this directory # and, allow getting build dependencies from PYTHONPATH, which we # setup in this repository rule, but still download any necessary @@ -382,6 +385,7 @@ def _whl_library_impl(rctx): build_file_contents = generate_whl_library_build_bazel( name = whl_path.basename, + sdist_filename = sdist_filename, dep_template = rctx.attr.dep_template or "@{}{{name}}//:{{target}}".format(rctx.attr.repo_prefix), entry_points = entry_points, metadata_name = metadata.name, @@ -455,6 +459,7 @@ def _whl_library_impl(rctx): build_file_contents = generate_whl_library_build_bazel( name = whl_path.basename, + sdist_filename = sdist_filename, dep_template = rctx.attr.dep_template or "@{}{{name}}//:{{target}}".format(rctx.attr.repo_prefix), entry_points = entry_points, # TODO @aignas 2025-05-17: maybe have a build flag for this instead diff --git a/python/private/pypi/whl_library_targets.bzl b/python/private/pypi/whl_library_targets.bzl index 95c1f5e981..aed5bc74f5 100644 --- a/python/private/pypi/whl_library_targets.bzl +++ b/python/private/pypi/whl_library_targets.bzl @@ -24,6 +24,7 @@ load( ":labels.bzl", "DATA_LABEL", "DIST_INFO_LABEL", + "EXTRACTED_WHEEL_FILES", "PY_LIBRARY_IMPL_LABEL", "PY_LIBRARY_PUBLIC_LABEL", "WHEEL_ENTRY_POINT_PREFIX", @@ -33,6 +34,16 @@ load( load(":namespace_pkgs.bzl", _create_inits = "create_inits") load(":pep508_deps.bzl", "deps") +# Files that are special to the Bazel processing of things. +_BAZEL_REPO_FILE_GLOBS = [ + "BUILD", + "BUILD.bazel", + "REPO.bazel", + "WORKSPACE", + "WORKSPACE", + "WORKSPACE.bazel", +] + def whl_library_targets_from_requires( *, name, @@ -97,14 +108,12 @@ def whl_library_targets( *, name, dep_template, + sdist_filename = None, data_exclude = [], srcs_exclude = [], tags = [], - filegroups = { - DIST_INFO_LABEL: ["site-packages/*.dist-info/**"], - DATA_LABEL: ["data/**"], - }, dependencies = [], + filegroups = None, dependencies_by_platform = {}, dependencies_with_markers = {}, group_deps = [], @@ -129,14 +138,16 @@ def whl_library_targets( filegroup. This may be also parsed to generate extra metadata. dep_template: {type}`str` The dep_template to use for dependency interpolation. + sdist_filename: {type}`str | None` If the wheel was built from an sdist, + the filename of the sdist. tags: {type}`list[str]` The tags set on the `py_library`. dependencies: {type}`list[str]` A list of dependencies. dependencies_by_platform: {type}`dict[str, list[str]]` A list of dependencies by platform key. dependencies_with_markers: {type}`dict[str, str]` A marker to evaluate in order for the dep to be included. - filegroups: {type}`dict[str, list[str]]` A dictionary of the target - names and the glob matches. + filegroups: {type}`dict[str, list[str]] | None` A dictionary of the target + names and the glob matches. If `None`, defaults will be used. group_name: {type}`str` name of the dependency group (if any) which contains this library. If set, this library will behave as a shim to group implementation rules which will provide simultaneously @@ -169,10 +180,28 @@ def whl_library_targets( tags = sorted(tags) data = [] + data - for filegroup_name, glob in filegroups.items(): + if filegroups == None: + filegroups = { + EXTRACTED_WHEEL_FILES: dict( + include = ["**"], + exclude = ( + _BAZEL_REPO_FILE_GLOBS + + [sdist_filename] if sdist_filename else [] + ), + ), + DIST_INFO_LABEL: dict( + include = ["site-packages/*.dist-info/**"], + ), + DATA_LABEL: dict( + include = ["data/**"], + ), + } + + for filegroup_name, glob_kwargs in filegroups.items(): + glob_kwargs = {"allow_empty": True} | glob_kwargs native.filegroup( name = filegroup_name, - srcs = native.glob(glob, allow_empty = True), + srcs = native.glob(**glob_kwargs), visibility = ["//visibility:public"], ) diff --git a/python/private/whl_filegroup/whl_filegroup.bzl b/python/private/whl_filegroup/whl_filegroup.bzl index d2e6e43b91..c52211bfbc 100644 --- a/python/private/whl_filegroup/whl_filegroup.bzl +++ b/python/private/whl_filegroup/whl_filegroup.bzl @@ -42,7 +42,14 @@ cc_library( includes = ["numpy_includes/numpy/core/include"], deps = ["@rules_python//python/cc:current_py_cc_headers"], ) + ``` + +:::{seealso} + +The `:extracted_whl_files` target, which is a filegroup of all the files +from the already extracted whl file. +::: """, attrs = { "pattern": attr.string(default = "", doc = "Only file paths matching this regex pattern will be extracted."), diff --git a/tests/pypi/pkg_aliases/pkg_aliases_test.bzl b/tests/pypi/pkg_aliases/pkg_aliases_test.bzl index 123ee725f8..3fd08c393c 100644 --- a/tests/pypi/pkg_aliases/pkg_aliases_test.bzl +++ b/tests/pypi/pkg_aliases/pkg_aliases_test.bzl @@ -43,6 +43,7 @@ def _test_legacy_aliases(env): "whl": "@repo//:whl", "data": "@repo//:data", "dist_info": "@repo//:dist_info", + "extracted_whl_files": "@repo//:extracted_whl_files", "my_special": "@repo//:my_special", } @@ -242,6 +243,10 @@ def _test_group_aliases(env): "name": "dist_info", "actual": "@repo//:dist_info", }, + { + "name": "extracted_whl_files", + "actual": "@repo//:extracted_whl_files", + }, { "name": "pkg", "actual": "//_groups:my_group_pkg", diff --git a/tests/pypi/whl_library_targets/whl_library_targets_tests.bzl b/tests/pypi/whl_library_targets/whl_library_targets_tests.bzl index bc58be9698..ec7ca63832 100644 --- a/tests/pypi/whl_library_targets/whl_library_targets_tests.bzl +++ b/tests/pypi/whl_library_targets/whl_library_targets_tests.bzl @@ -27,9 +27,10 @@ _tests = [] def _test_filegroups(env): calls = [] - def glob(match, *, allow_empty): + def glob(include, *, exclude = [], allow_empty): + _ = exclude # @unused env.expect.that_bool(allow_empty).equals(True) - return match + return include whl_library_targets( name = "", @@ -41,7 +42,7 @@ def _test_filegroups(env): rules = struct(), ) - env.expect.that_collection(calls).contains_exactly([ + env.expect.that_collection(calls, expr = "filegroup calls").contains_exactly([ { "name": "dist_info", "srcs": ["site-packages/*.dist-info/**"], @@ -52,6 +53,11 @@ def _test_filegroups(env): "srcs": ["data/**"], "visibility": ["//visibility:public"], }, + { + "name": "extracted_whl_files", + "srcs": ["**"], + "visibility": ["//visibility:public"], + }, { "name": "whl", "srcs": [""], From dca801472f9d8a6c5be6dde3d5e7f7783034d9ba Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Mon, 7 Jul 2025 16:51:35 -0700 Subject: [PATCH 322/922] docs: add whl_from_dir to dev guide docs (#3067) Mentions the whl_from_dir repository rule in our dev guide. --- CONTRIBUTING.md | 4 +++- docs/devguide.md | 12 ++++++++++++ 2 files changed, 15 insertions(+), 1 deletion(-) diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 8f985c551b..e1bd11b81d 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -234,11 +234,13 @@ merged: ## Binary artifacts Checking in binary artifacts is not allowed. This is because they are extremely -problematic to verify and ensure they're safe +problematic to verify and ensure they're safe. This is true even in +test contexts. Examples include, but aren't limited to: prebuilt binaries, shared libraries, zip files, or wheels. +See the dev guide for utilities to help with testing. (breaking-changes)= ## Breaking Changes diff --git a/docs/devguide.md b/docs/devguide.md index 345907b374..43120bf2a1 100644 --- a/docs/devguide.md +++ b/docs/devguide.md @@ -37,6 +37,12 @@ to be perfectly factored and not every common thing a test does needs to be factored into a more generally reusable piece. Copying and pasting is fine. It's more important for tests to balance understandability and maintainability. +### Test utilities + +General code to support testing is in {gh-path}`tests/support`. It has a variety +of functions, constants, rules etc, to make testing easier. Below are some +common utilities that are frequently used. + ### sh_py_run_test The {gh-path}`sh_py_run_test Date: Tue, 8 Jul 2025 13:15:36 +0900 Subject: [PATCH 323/922] fix(toolchains): fix the URLs and sha256 values (#3070) It seems that in #3062 we did not notice that the latest builds for `linux-aarch64-freethreaded` actually needed a different build tag and this is the case only for the latest releases. What is more, the `windows-aarch64` build had a wrong strip-prefix. This is fixing all of these issues. #3028 will make the 404 error messages from `curl` more visible, so the first issue will be more easily caught during authoring, whereas the second issue is going to be more likely caught via code review, or a CI that is exercising the actual toolchains. --- python/versions.bzl | 12 +++++++----- 1 file changed, 7 insertions(+), 5 deletions(-) diff --git a/python/versions.bzl b/python/versions.bzl index 1b33db0621..f6cf121187 100644 --- a/python/versions.bzl +++ b/python/versions.bzl @@ -780,7 +780,7 @@ TOOL_VERSIONS = { "x86_64-unknown-linux-gnu": "9f5d5260f333fcb5372ec681851d92ddac79a33362aa85626b6cc96ffe75eeef", "x86_64-unknown-linux-musl": "7856fd505e311d1a4c24e429ac5ef0ff6ca7a2005c3a7eff1fe204524a6f45aa", "aarch64-apple-darwin-freethreaded": "52e582cc89d654c565297b4ff9c3bd4bed5c3e81cad46f41c62485e700faf8bd", - "aarch64-unknown-linux-gnu-freethreaded": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855", + "aarch64-unknown-linux-gnu-freethreaded": "461832e4fb5ec1d719dc40f6490f9a639414dfa6769158187fa85d4b424b57cd", "ppc64le-unknown-linux-gnu-freethreaded": "c65c75edb450de830f724afdc774a215c2d3255097e0d670f709d2271fd6fd52", "riscv64-unknown-linux-gnu-freethreaded": "716e6e3fad24fb9931b93005000152dd9da4c3343b88ca54b5c01a7ab879d734", "s390x-unknown-linux-gnu-freethreaded": "27276aee426a51f4165fac49391aedc5a9e301ae217366c77b65826122bb30fc", @@ -796,6 +796,7 @@ TOOL_VERSIONS = { "riscv64-unknown-linux-gnu": "python", "x86_64-apple-darwin": "python", "x86_64-pc-windows-msvc": "python", + "aarch64-pc-windows-msvc": "python", "x86_64-unknown-linux-gnu": "python", "x86_64-unknown-linux-musl": "python", "aarch64-apple-darwin-freethreaded": "python/install", @@ -805,7 +806,6 @@ TOOL_VERSIONS = { "s390x-unknown-linux-gnu-freethreaded": "python/install", "x86_64-apple-darwin-freethreaded": "python/install", "x86_64-pc-windows-msvc-freethreaded": "python/install", - "aarch64-pc-windows-msvc": "python/install", "aarch64-pc-windows-msvc-freethreaded": "python/install", "x86_64-unknown-linux-gnu-freethreaded": "python/install", }, @@ -824,7 +824,7 @@ TOOL_VERSIONS = { "x86_64-unknown-linux-gnu": "00328c48cc07076a5b083575654761cdb07bc8b3bba864d3a225062722485bac", "x86_64-unknown-linux-musl": "a2fed85bc3d5415d2318a2eeb0cb9e6effb81667870ae568a08756838ad4926e", "aarch64-apple-darwin-freethreaded": "d19213021f5fd039d7021ccb41698cc99ca313064d7c1cc9b5ef8f831abb9961", - "aarch64-unknown-linux-gnu-freethreaded": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855", + "aarch64-unknown-linux-gnu-freethreaded": "b01cc74173515cc3733f0af62b7d574364c1c68daf3ad748bca47e4328770cde", "ppc64le-unknown-linux-gnu-freethreaded": "1f093e0c3532e27744e3fb73a8c738355910b6bfa195039e4f73b4f48c1bc4fc", "riscv64-unknown-linux-gnu-freethreaded": "73162a5da31cc1e410d456496114f8e5ee7243bc7bbe0e087b1ea50f0fdc6774", "s390x-unknown-linux-gnu-freethreaded": "045017e60f1298111e8ccfec6afbe47abe56f82997258c8754009269a5343736", @@ -1045,13 +1045,15 @@ def get_release_info(platform, python_version, base_url = DEFAULT_RELEASE_BASE_U for u in url: p, _, _ = platform.partition(FREETHREADED) + release_id = int(u.split("/")[-2]) + if FREETHREADED.lstrip("-") in platform: build = "{}+{}-full".format( FREETHREADED.lstrip("-"), { "aarch64-apple-darwin": "pgo+lto", "aarch64-pc-windows-msvc": "pgo", - "aarch64-unknown-linux-gnu": "lto", + "aarch64-unknown-linux-gnu": "lto" if release_id < 20250702 else "pgo+lto", "ppc64le-unknown-linux-gnu": "lto", "riscv64-unknown-linux-gnu": "lto", "s390x-unknown-linux-gnu": "lto", @@ -1063,7 +1065,7 @@ def get_release_info(platform, python_version, base_url = DEFAULT_RELEASE_BASE_U else: build = INSTALL_ONLY - if WINDOWS_NAME in platform and int(u.split("/")[0]) < 20250317: + if WINDOWS_NAME in platform and release_id < 20250317: build = "shared-" + build release_filename = u.format( From 16c65cf9987724db0a6a529f3a5496f19bd25ed2 Mon Sep 17 00:00:00 2001 From: Douglas Thor Date: Tue, 8 Jul 2025 09:26:09 -0700 Subject: [PATCH 324/922] chore: Switch back to smacker/go-tree-sitter (#3069) Finally remove the dougthor42/go-tree-sitter fork, fixing #2630. Admittedly we could have done this sooner had I figured things out sooner... but c'est la vie. Instead of using the BUILD.bazel files in dougthor42/go-tree-sitter, we basically vendor the build file via http_archive. This is different than using patches because non-root Bazel modules can still make use of the BUILD.bazel files we make. Background: The reason we migrated to dougthor42/go-tree-sitter in the first place was to support python 3.12 grammar. smacker/go-tree-sitter supported for python 3.12, but made a change to their file structure that Gazelle was unable to handle. Specifically, the python/binding.go file indirectly requires a c header file found in a parent directory, and Gazelle doesn't know how to handle that for `go_repository` (WORKSPACE) and `go_deps.from_file` (bzlmod). So dougthor42/go-tree-sitter created our own BUILD.bazel files that included the required filegroups and whatnot, thus negating the need for Gazelle to generate BUILD.bazel files. Future Work: This still doesn't resolve the issues with bumping rules_go and go seen in #2962, but it does simplify that investigation a bit as it's just one fewer thing to account for. It also doesn't address the desire to migrate to the official tree-sitter/go-tree-sitter repo, but @jbedard found some perf issues with that anyway (https://github.com/tree-sitter/go-tree-sitter/issues/32). --- CHANGELOG.md | 2 + gazelle/MODULE.bazel | 11 +++- gazelle/deps.bzl | 13 +++-- gazelle/go.mod | 2 +- gazelle/go.sum | 6 +-- gazelle/internal/smacker_BUILD.bazel | 80 ++++++++++++++++++++++++++++ gazelle/python/BUILD.bazel | 4 +- gazelle/python/file_parser.go | 8 +-- 8 files changed, 105 insertions(+), 21 deletions(-) create mode 100644 gazelle/internal/smacker_BUILD.bazel diff --git a/CHANGELOG.md b/CHANGELOG.md index c1d3a43814..7f02c8bbb4 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -67,6 +67,8 @@ END_UNRELEASED_TEMPLATE * 3.12.11 * 3.14.0b3 * (toolchain) Python 3.13 now references 3.13.5 +* (gazelle) Switched back to smacker/go-tree-sitter, fixing + [#2630](https://github.com/bazel-contrib/rules_python/issues/2630) {#v0-0-0-fixed} ### Fixed diff --git a/gazelle/MODULE.bazel b/gazelle/MODULE.bazel index 6bbc74bc61..51352a0ba6 100644 --- a/gazelle/MODULE.bazel +++ b/gazelle/MODULE.bazel @@ -21,7 +21,6 @@ use_repo( go_deps, "com_github_bazelbuild_buildtools", "com_github_bmatcuk_doublestar_v4", - "com_github_dougthor42_go_tree_sitter", "com_github_emirpasic_gods", "com_github_ghodss_yaml", "com_github_stretchr_testify", @@ -29,6 +28,16 @@ use_repo( "org_golang_x_sync", ) +http_archive = use_repo_rule("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive") + +http_archive( + name = "com_github_smacker_go_tree_sitter", + build_file = "//:internal/smacker_BUILD.bazel", + integrity = "sha256-4AkDY4Rh5Auu9Kwzhj5XYSirMLlhmd6ClMWo/r0kmu4=", + strip_prefix = "go-tree-sitter-dd81d9e9be82a8cac96ed1d50c7389c5f1997c02", + url = "https://github.com/smacker/go-tree-sitter/archive/dd81d9e9be82a8cac96ed1d50c7389c5f1997c02.zip", +) + python_stdlib_list = use_extension("//python:extensions.bzl", "python_stdlib_list") use_repo( python_stdlib_list, diff --git a/gazelle/deps.bzl b/gazelle/deps.bzl index 7253ef8194..8c4c055e9b 100644 --- a/gazelle/deps.bzl +++ b/gazelle/deps.bzl @@ -113,7 +113,6 @@ def go_deps(): sum = "h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=", version = "v1.1.1", ) - go_repository( name = "com_github_emirpasic_gods", importpath = "github.com/emirpasic/gods", @@ -175,18 +174,18 @@ def go_deps(): sum = "h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=", version = "v1.0.0", ) - go_repository( name = "com_github_prometheus_client_model", importpath = "github.com/prometheus/client_model", sum = "h1:gQz4mCbXsO+nc9n1hCxHcGA3Zx3Eo+UHZoInFGUIXNM=", version = "v0.0.0-20190812154241-14fe0d1b01d4", ) - go_repository( - name = "com_github_dougthor42_go_tree_sitter", - importpath = "github.com/dougthor42/go-tree-sitter", - sum = "h1:b9s96BulIARx0konX36sJ5oZhWvAvjQBBntxp1eUukQ=", - version = "v0.0.0-20241210060307-2737e1d0de6b", + http_archive( + name = "com_github_smacker_go_tree_sitter", + build_file = Label("//:internal/smacker_BUILD.bazel"), + integrity = "sha256-4AkDY4Rh5Auu9Kwzhj5XYSirMLlhmd6ClMWo/r0kmu4=", + strip_prefix = "go-tree-sitter-dd81d9e9be82a8cac96ed1d50c7389c5f1997c02", + url = "https://github.com/smacker/go-tree-sitter/archive/dd81d9e9be82a8cac96ed1d50c7389c5f1997c02.zip", ) go_repository( name = "com_github_stretchr_objx", diff --git a/gazelle/go.mod b/gazelle/go.mod index 91d27fdd5a..6f65ffbc7e 100644 --- a/gazelle/go.mod +++ b/gazelle/go.mod @@ -7,9 +7,9 @@ require ( github.com/bazelbuild/buildtools v0.0.0-20231103205921-433ea8554e82 github.com/bazelbuild/rules_go v0.41.0 github.com/bmatcuk/doublestar/v4 v4.7.1 - github.com/dougthor42/go-tree-sitter v0.0.0-20241210060307-2737e1d0de6b github.com/emirpasic/gods v1.18.1 github.com/ghodss/yaml v1.0.0 + github.com/smacker/go-tree-sitter v0.0.0-20240827094217-dd81d9e9be82 github.com/stretchr/testify v1.9.0 golang.org/x/sync v0.2.0 gopkg.in/yaml.v2 v2.4.0 diff --git a/gazelle/go.sum b/gazelle/go.sum index 5acd4a6db5..0aaa186620 100644 --- a/gazelle/go.sum +++ b/gazelle/go.sum @@ -6,8 +6,6 @@ github.com/bazelbuild/buildtools v0.0.0-20231103205921-433ea8554e82 h1:HTepWP/jh github.com/bazelbuild/buildtools v0.0.0-20231103205921-433ea8554e82/go.mod h1:689QdV3hBP7Vo9dJMmzhoYIyo/9iMhEmHkJcnaPRCbo= github.com/bazelbuild/rules_go v0.41.0 h1:JzlRxsFNhlX+g4drDRPhIaU5H5LnI978wdMJ0vK4I+k= github.com/bazelbuild/rules_go v0.41.0/go.mod h1:TMHmtfpvyfsxaqfL9WnahCsXMWDMICTw7XeK9yVb+YU= -github.com/bmatcuk/doublestar/v4 v4.6.1 h1:FH9SifrbvJhnlQpztAx++wlkk70QBf0iBWDwNy7PA4I= -github.com/bmatcuk/doublestar/v4 v4.6.1/go.mod h1:xBQ8jztBU6kakFMg+8WGxn0c6z1fTSPVIjEY1Wr7jzc= github.com/bmatcuk/doublestar/v4 v4.7.1 h1:fdDeAqgT47acgwd9bd9HxJRDmc9UAmPpc+2m0CXv75Q= github.com/bmatcuk/doublestar/v4 v4.7.1/go.mod h1:xBQ8jztBU6kakFMg+8WGxn0c6z1fTSPVIjEY1Wr7jzc= github.com/census-instrumentation/opencensus-proto v0.2.1/go.mod h1:f6KPmirojxKA12rnyqOA5BBL4O983OfeGPqjHWSTneU= @@ -17,8 +15,6 @@ github.com/chzyer/test v0.0.0-20180213035817-a1ea475d72b1/go.mod h1:Q3SI9o4m/ZMn github.com/client9/misspell v0.3.4/go.mod h1:qj6jICC3Q7zFZvVWo7KLAzC3yx5G7kyvSDkc90ppPyw= github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c= github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38= -github.com/dougthor42/go-tree-sitter v0.0.0-20241210060307-2737e1d0de6b h1:b9s96BulIARx0konX36sJ5oZhWvAvjQBBntxp1eUukQ= -github.com/dougthor42/go-tree-sitter v0.0.0-20241210060307-2737e1d0de6b/go.mod h1:87UkDyPt18bTH/FvinLc/kj587VNYOdRKZT1la4T8Hg= github.com/emirpasic/gods v1.18.1 h1:FXtiHYKDGKCW2KzwZKx0iC0PQmdlorYgdFG9jPXJ1Bc= github.com/emirpasic/gods v1.18.1/go.mod h1:8tpGGwCnJ5H4r6BWwaV6OrWmMoPhUl5jm/FMNAnJvWQ= github.com/envoyproxy/go-control-plane v0.9.1-0.20191026205805-5f8ba28d4473/go.mod h1:YTl/9mNaCwkRvm6d1a2C3ymFceY/DCBVvsKhRF0iEA4= @@ -47,6 +43,8 @@ github.com/google/go-cmp v0.5.9/go.mod h1:17dUlkBOakJ0+DkrSSNjCkIjxS6bF9zb3elmeN github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM= github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4= github.com/prometheus/client_model v0.0.0-20190812154241-14fe0d1b01d4/go.mod h1:xMI15A0UPsDsEKsMN9yxemIoYk6Tm2C1GtYGdfGttqA= +github.com/smacker/go-tree-sitter v0.0.0-20240827094217-dd81d9e9be82 h1:6C8qej6f1bStuePVkLSFxoU22XBS165D3klxlzRg8F4= +github.com/smacker/go-tree-sitter v0.0.0-20240827094217-dd81d9e9be82/go.mod h1:xe4pgH49k4SsmkQq5OT8abwhWmnzkhpgnXeekbx2efw= github.com/stretchr/testify v1.9.0 h1:HtqpIVDClZ4nwg75+f6Lvsy/wHu+3BoSGCbBAcpTsTg= github.com/stretchr/testify v1.9.0/go.mod h1:r2ic/lqez/lEtzL7wO/rwa5dbSLXVDPFyf8C91i36aY= go.starlark.net v0.0.0-20210223155950-e043a3d3c984/go.mod h1:t3mmBBPzAVvK0L0n1drDmrQsJ8FoIx4INCqVMTr/Zo0= diff --git a/gazelle/internal/smacker_BUILD.bazel b/gazelle/internal/smacker_BUILD.bazel new file mode 100644 index 0000000000..3ec96760e8 --- /dev/null +++ b/gazelle/internal/smacker_BUILD.bazel @@ -0,0 +1,80 @@ +load("@io_bazel_rules_go//go:def.bzl", "go_library", "go_test") + +filegroup( + name = "common_libs", + srcs = [ + "alloc.h", + "api.h", + "array.h", + ], + visibility = [":__subpackages__"], +) + +go_library( + name = "go-tree-sitter", + srcs = [ + "alloc.c", + "alloc.h", + "api.h", + "array.h", + "atomic.h", + "bindings.c", + "bindings.go", + "bindings.h", + "bits.h", + "clock.h", + "error_costs.h", + "get_changed_ranges.c", + "get_changed_ranges.h", + "host.h", + "iter.go", + "language.c", + "language.h", + "length.h", + "lexer.c", + "lexer.h", + "node.c", + "parser.c", + "parser.h", + "point.h", + "ptypes.h", + "query.c", + "reduce_action.h", + "reusable_node.h", + "stack.c", + "stack.h", + "subtree.c", + "subtree.h", + "test_grammar.go", + "tree.c", + "tree.h", + "tree_cursor.c", + "tree_cursor.h", + "umachine.h", + "unicode.h", + "urename.h", + "utf.h", + "utf16.h", + "utf8.h", + "wasm_store.c", + "wasm_store.h", + ], + cgo = True, + importpath = "github.com/smacker/go-tree-sitter", + visibility = ["//visibility:public"], +) + +go_library( + name = "python", + srcs = [ + "python/binding.go", + "python/parser.c", + "python/parser.h", + "python/scanner.c", + ":common_libs", + ], + cgo = True, + importpath = "github.com/smacker/go-tree-sitter/python", + visibility = ["//visibility:public"], + deps = [":go-tree-sitter"], +) diff --git a/gazelle/python/BUILD.bazel b/gazelle/python/BUILD.bazel index eb2d72e5eb..8e8216ddd4 100644 --- a/gazelle/python/BUILD.bazel +++ b/gazelle/python/BUILD.bazel @@ -39,11 +39,11 @@ go_library( "@bazel_gazelle//rule:go_default_library", "@com_github_bazelbuild_buildtools//build:go_default_library", "@com_github_bmatcuk_doublestar_v4//:doublestar", - "@com_github_dougthor42_go_tree_sitter//:go-tree-sitter", - "@com_github_dougthor42_go_tree_sitter//python", "@com_github_emirpasic_gods//lists/singlylinkedlist", "@com_github_emirpasic_gods//sets/treeset", "@com_github_emirpasic_gods//utils", + "@com_github_smacker_go_tree_sitter//:go-tree-sitter", + "@com_github_smacker_go_tree_sitter//:python", "@org_golang_x_sync//errgroup", ], ) diff --git a/gazelle/python/file_parser.go b/gazelle/python/file_parser.go index aca925cbe7..31fce02712 100644 --- a/gazelle/python/file_parser.go +++ b/gazelle/python/file_parser.go @@ -22,8 +22,8 @@ import ( "path/filepath" "strings" - sitter "github.com/dougthor42/go-tree-sitter" - "github.com/dougthor42/go-tree-sitter/python" + sitter "github.com/smacker/go-tree-sitter" + "github.com/smacker/go-tree-sitter/python" ) const ( @@ -116,10 +116,6 @@ func (p *FileParser) parseMain(ctx context.Context, node *sitter.Node) bool { a, b = b, a } if a.Type() == sitterNodeTypeIdentifier && a.Content(p.code) == "__name__" && - // at github.com/dougthor42/go-tree-sitter@latest (after v0.0.0-20240422154435-0628b34cbf9c we used) - // "__main__" is the second child of b. But now, it isn't. - // we cannot use the latest go-tree-sitter because of the top level reference in scanner.c. - // https://github.com/dougthor42/go-tree-sitter/blob/04d6b33fe138a98075210f5b770482ded024dc0f/python/scanner.c#L1 b.Type() == sitterNodeTypeString && string(p.code[b.StartByte()+1:b.EndByte()-1]) == "__main__" { return true } From a959cfc56111c691b23f674abb7bd2e60b469217 Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Tue, 8 Jul 2025 16:01:40 -0700 Subject: [PATCH 325/922] build(deps): bump charset-normalizer from 3.4.1 to 3.4.2 in /tools/publish (#3020) Bumps [charset-normalizer](https://github.com/jawah/charset_normalizer) from 3.4.1 to 3.4.2.
Release notes

Sourced from charset-normalizer's releases.

Version 3.4.2

3.4.2 (2025-05-02)

Fixed

  • Addressed the DeprecationWarning in our CLI regarding argparse.FileType by backporting the target class into the package. (#591)
  • Improved the overall reliability of the detector with CJK Ideographs. (#605) (#587)

Changed

  • Optional mypyc compilation upgraded to version 1.15 for Python >= 3.9
Changelog

Sourced from charset-normalizer's changelog.

3.4.2 (2025-05-02)

Fixed

  • Addressed the DeprecationWarning in our CLI regarding argparse.FileType by backporting the target class into the package. (#591)
  • Improved the overall reliability of the detector with CJK Ideographs. (#605) (#587)

Changed

  • Optional mypyc compilation upgraded to version 1.15 for Python >= 3.8
Commits
  • 6422af1 :pencil: update release date
  • 0e60ec1 :bookmark: Release 3.4.2 (#614)
  • f6630ce :arrow_up: Bump pypa/cibuildwheel from 2.23.2 to 2.23.3 (#617)
  • 677c999 :arrow_up: Bump actions/download-artifact from 4.2.1 to 4.3.0 (#618)
  • 960ab1e :arrow_up: Bump actions/setup-python from 5.5.0 to 5.6.0 (#619)
  • 6eb6325 :arrow_up: Bump github/codeql-action from 3.28.10 to 3.28.16 (#620)
  • c99c0f2 :arrow_up: Update coverage requirement from <7.7,>=7.2.7 to >=7.2.7,<7.9 (#606)
  • 270f28e :arrow_up: Bump actions/setup-python from 5.4.0 to 5.5.0 (#607)
  • d4d89a0 :arrow_up: Bump pypa/cibuildwheel from 2.22.0 to 2.23.2 (#608)
  • 905fcf5 :arrow_up: Bump slsa-framework/slsa-github-generator from 2.0.0 to 2.1.0 (#609)
  • Additional commits viewable in compare view

[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=charset-normalizer&package-manager=pip&previous-version=3.4.1&new-version=3.4.2)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) ---
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot show ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> --- tools/publish/requirements_darwin.txt | 186 +++++++++++------------ tools/publish/requirements_linux.txt | 186 +++++++++++------------ tools/publish/requirements_universal.txt | 186 +++++++++++------------ tools/publish/requirements_windows.txt | 186 +++++++++++------------ 4 files changed, 372 insertions(+), 372 deletions(-) diff --git a/tools/publish/requirements_darwin.txt b/tools/publish/requirements_darwin.txt index 58973acb6f..afc2bae956 100644 --- a/tools/publish/requirements_darwin.txt +++ b/tools/publish/requirements_darwin.txt @@ -10,99 +10,99 @@ certifi==2025.6.15 \ --hash=sha256:2e0c7ce7cb5d8f8634ca55d2ba7e6ec2689a2fd6537d8dec1296a477a4910057 \ --hash=sha256:d747aa5a8b9bbbb1bb8c22bb13e22bd1f18e9796defa16bab421f7f7a317323b # via requests -charset-normalizer==3.4.1 \ - --hash=sha256:0167ddc8ab6508fe81860a57dd472b2ef4060e8d378f0cc555707126830f2537 \ - --hash=sha256:01732659ba9b5b873fc117534143e4feefecf3b2078b0a6a2e925271bb6f4cfa \ - --hash=sha256:01ad647cdd609225c5350561d084b42ddf732f4eeefe6e678765636791e78b9a \ - --hash=sha256:04432ad9479fa40ec0f387795ddad4437a2b50417c69fa275e212933519ff294 \ - --hash=sha256:0907f11d019260cdc3f94fbdb23ff9125f6b5d1039b76003b5b0ac9d6a6c9d5b \ - --hash=sha256:0924e81d3d5e70f8126529951dac65c1010cdf117bb75eb02dd12339b57749dd \ - --hash=sha256:09b26ae6b1abf0d27570633b2b078a2a20419c99d66fb2823173d73f188ce601 \ - --hash=sha256:09b5e6733cbd160dcc09589227187e242a30a49ca5cefa5a7edd3f9d19ed53fd \ - --hash=sha256:0af291f4fe114be0280cdd29d533696a77b5b49cfde5467176ecab32353395c4 \ - --hash=sha256:0f55e69f030f7163dffe9fd0752b32f070566451afe180f99dbeeb81f511ad8d \ - --hash=sha256:1a2bc9f351a75ef49d664206d51f8e5ede9da246602dc2d2726837620ea034b2 \ - --hash=sha256:22e14b5d70560b8dd51ec22863f370d1e595ac3d024cb8ad7d308b4cd95f8313 \ - --hash=sha256:234ac59ea147c59ee4da87a0c0f098e9c8d169f4dc2a159ef720f1a61bbe27cd \ - --hash=sha256:2369eea1ee4a7610a860d88f268eb39b95cb588acd7235e02fd5a5601773d4fa \ - --hash=sha256:237bdbe6159cff53b4f24f397d43c6336c6b0b42affbe857970cefbb620911c8 \ - --hash=sha256:28bf57629c75e810b6ae989f03c0828d64d6b26a5e205535585f96093e405ed1 \ - --hash=sha256:2967f74ad52c3b98de4c3b32e1a44e32975e008a9cd2a8cc8966d6a5218c5cb2 \ - --hash=sha256:2a75d49014d118e4198bcee5ee0a6f25856b29b12dbf7cd012791f8a6cc5c496 \ - --hash=sha256:2bdfe3ac2e1bbe5b59a1a63721eb3b95fc9b6817ae4a46debbb4e11f6232428d \ - --hash=sha256:2d074908e1aecee37a7635990b2c6d504cd4766c7bc9fc86d63f9c09af3fa11b \ - --hash=sha256:2fb9bd477fdea8684f78791a6de97a953c51831ee2981f8e4f583ff3b9d9687e \ - --hash=sha256:311f30128d7d333eebd7896965bfcfbd0065f1716ec92bd5638d7748eb6f936a \ - --hash=sha256:329ce159e82018d646c7ac45b01a430369d526569ec08516081727a20e9e4af4 \ - --hash=sha256:345b0426edd4e18138d6528aed636de7a9ed169b4aaf9d61a8c19e39d26838ca \ - --hash=sha256:363e2f92b0f0174b2f8238240a1a30142e3db7b957a5dd5689b0e75fb717cc78 \ - --hash=sha256:3a3bd0dcd373514dcec91c411ddb9632c0d7d92aed7093b8c3bbb6d69ca74408 \ - --hash=sha256:3bed14e9c89dcb10e8f3a29f9ccac4955aebe93c71ae803af79265c9ca5644c5 \ - --hash=sha256:44251f18cd68a75b56585dd00dae26183e102cd5e0f9f1466e6df5da2ed64ea3 \ - --hash=sha256:44ecbf16649486d4aebafeaa7ec4c9fed8b88101f4dd612dcaf65d5e815f837f \ - --hash=sha256:4532bff1b8421fd0a320463030c7520f56a79c9024a4e88f01c537316019005a \ - --hash=sha256:49402233c892a461407c512a19435d1ce275543138294f7ef013f0b63d5d3765 \ - --hash=sha256:4c0907b1928a36d5a998d72d64d8eaa7244989f7aaaf947500d3a800c83a3fd6 \ - --hash=sha256:4d86f7aff21ee58f26dcf5ae81a9addbd914115cdebcbb2217e4f0ed8982e146 \ - --hash=sha256:5777ee0881f9499ed0f71cc82cf873d9a0ca8af166dfa0af8ec4e675b7df48e6 \ - --hash=sha256:5df196eb874dae23dcfb968c83d4f8fdccb333330fe1fc278ac5ceeb101003a9 \ - --hash=sha256:619a609aa74ae43d90ed2e89bdd784765de0a25ca761b93e196d938b8fd1dbbd \ - --hash=sha256:6e27f48bcd0957c6d4cb9d6fa6b61d192d0b13d5ef563e5f2ae35feafc0d179c \ - --hash=sha256:6ff8a4a60c227ad87030d76e99cd1698345d4491638dfa6673027c48b3cd395f \ - --hash=sha256:73d94b58ec7fecbc7366247d3b0b10a21681004153238750bb67bd9012414545 \ - --hash=sha256:7461baadb4dc00fd9e0acbe254e3d7d2112e7f92ced2adc96e54ef6501c5f176 \ - --hash=sha256:75832c08354f595c760a804588b9357d34ec00ba1c940c15e31e96d902093770 \ - --hash=sha256:7709f51f5f7c853f0fb938bcd3bc59cdfdc5203635ffd18bf354f6967ea0f824 \ - --hash=sha256:78baa6d91634dfb69ec52a463534bc0df05dbd546209b79a3880a34487f4b84f \ - --hash=sha256:7974a0b5ecd505609e3b19742b60cee7aa2aa2fb3151bc917e6e2646d7667dcf \ - --hash=sha256:7a4f97a081603d2050bfaffdefa5b02a9ec823f8348a572e39032caa8404a487 \ - --hash=sha256:7b1bef6280950ee6c177b326508f86cad7ad4dff12454483b51d8b7d673a2c5d \ - --hash=sha256:7d053096f67cd1241601111b698f5cad775f97ab25d81567d3f59219b5f1adbd \ - --hash=sha256:804a4d582ba6e5b747c625bf1255e6b1507465494a40a2130978bda7b932c90b \ - --hash=sha256:807f52c1f798eef6cf26beb819eeb8819b1622ddfeef9d0977a8502d4db6d534 \ - --hash=sha256:80ed5e856eb7f30115aaf94e4a08114ccc8813e6ed1b5efa74f9f82e8509858f \ - --hash=sha256:8417cb1f36cc0bc7eaba8ccb0e04d55f0ee52df06df3ad55259b9a323555fc8b \ - --hash=sha256:8436c508b408b82d87dc5f62496973a1805cd46727c34440b0d29d8a2f50a6c9 \ - --hash=sha256:89149166622f4db9b4b6a449256291dc87a99ee53151c74cbd82a53c8c2f6ccd \ - --hash=sha256:8bfa33f4f2672964266e940dd22a195989ba31669bd84629f05fab3ef4e2d125 \ - --hash=sha256:8c60ca7339acd497a55b0ea5d506b2a2612afb2826560416f6894e8b5770d4a9 \ - --hash=sha256:91b36a978b5ae0ee86c394f5a54d6ef44db1de0815eb43de826d41d21e4af3de \ - --hash=sha256:955f8851919303c92343d2f66165294848d57e9bba6cf6e3625485a70a038d11 \ - --hash=sha256:97f68b8d6831127e4787ad15e6757232e14e12060bec17091b85eb1486b91d8d \ - --hash=sha256:9b23ca7ef998bc739bf6ffc077c2116917eabcc901f88da1b9856b210ef63f35 \ - --hash=sha256:9f0b8b1c6d84c8034a44893aba5e767bf9c7a211e313a9605d9c617d7083829f \ - --hash=sha256:aabfa34badd18f1da5ec1bc2715cadc8dca465868a4e73a0173466b688f29dda \ - --hash=sha256:ab36c8eb7e454e34e60eb55ca5d241a5d18b2c6244f6827a30e451c42410b5f7 \ - --hash=sha256:b010a7a4fd316c3c484d482922d13044979e78d1861f0e0650423144c616a46a \ - --hash=sha256:b1ac5992a838106edb89654e0aebfc24f5848ae2547d22c2c3f66454daa11971 \ - --hash=sha256:b7b2d86dd06bfc2ade3312a83a5c364c7ec2e3498f8734282c6c3d4b07b346b8 \ - --hash=sha256:b97e690a2118911e39b4042088092771b4ae3fc3aa86518f84b8cf6888dbdb41 \ - --hash=sha256:bc2722592d8998c870fa4e290c2eec2c1569b87fe58618e67d38b4665dfa680d \ - --hash=sha256:c0429126cf75e16c4f0ad00ee0eae4242dc652290f940152ca8c75c3a4b6ee8f \ - --hash=sha256:c30197aa96e8eed02200a83fba2657b4c3acd0f0aa4bdc9f6c1af8e8962e0757 \ - --hash=sha256:c4c3e6da02df6fa1410a7680bd3f63d4f710232d3139089536310d027950696a \ - --hash=sha256:c75cb2a3e389853835e84a2d8fb2b81a10645b503eca9bcb98df6b5a43eb8886 \ - --hash=sha256:c96836c97b1238e9c9e3fe90844c947d5afbf4f4c92762679acfe19927d81d77 \ - --hash=sha256:d7f50a1f8c450f3925cb367d011448c39239bb3eb4117c36a6d354794de4ce76 \ - --hash=sha256:d973f03c0cb71c5ed99037b870f2be986c3c05e63622c017ea9816881d2dd247 \ - --hash=sha256:d98b1668f06378c6dbefec3b92299716b931cd4e6061f3c875a71ced1780ab85 \ - --hash=sha256:d9c3cdf5390dcd29aa8056d13e8e99526cda0305acc038b96b30352aff5ff2bb \ - --hash=sha256:dad3e487649f498dd991eeb901125411559b22e8d7ab25d3aeb1af367df5efd7 \ - --hash=sha256:dccbe65bd2f7f7ec22c4ff99ed56faa1e9f785482b9bbd7c717e26fd723a1d1e \ - --hash=sha256:dd78cfcda14a1ef52584dbb008f7ac81c1328c0f58184bf9a84c49c605002da6 \ - --hash=sha256:e218488cd232553829be0664c2292d3af2eeeb94b32bea483cf79ac6a694e037 \ - --hash=sha256:e358e64305fe12299a08e08978f51fc21fac060dcfcddd95453eabe5b93ed0e1 \ - --hash=sha256:ea0d8d539afa5eb2728aa1932a988a9a7af94f18582ffae4bc10b3fbdad0626e \ - --hash=sha256:eab677309cdb30d047996b36d34caeda1dc91149e4fdca0b1a039b3f79d9a807 \ - --hash=sha256:eb8178fe3dba6450a3e024e95ac49ed3400e506fd4e9e5c32d30adda88cbd407 \ - --hash=sha256:ecddf25bee22fe4fe3737a399d0d177d72bc22be6913acfab364b40bce1ba83c \ - --hash=sha256:eea6ee1db730b3483adf394ea72f808b6e18cf3cb6454b4d86e04fa8c4327a12 \ - --hash=sha256:f08ff5e948271dc7e18a35641d2f11a4cd8dfd5634f55228b691e62b37125eb3 \ - --hash=sha256:f30bf9fd9be89ecb2360c7d94a711f00c09b976258846efe40db3d05828e8089 \ - --hash=sha256:fa88b843d6e211393a37219e6a1c1df99d35e8fd90446f1118f4216e307e48cd \ - --hash=sha256:fc54db6c8593ef7d4b2a331b58653356cf04f67c960f584edb7c3d8c97e8f39e \ - --hash=sha256:fd4ec41f914fa74ad1b8304bbc634b3de73d2a0889bd32076342a573e0779e00 \ - --hash=sha256:ffc9202a29ab3920fa812879e95a9e78b2465fd10be7fcbd042899695d75e616 +charset-normalizer==3.4.2 \ + --hash=sha256:005fa3432484527f9732ebd315da8da8001593e2cf46a3d817669f062c3d9ed4 \ + --hash=sha256:046595208aae0120559a67693ecc65dd75d46f7bf687f159127046628178dc45 \ + --hash=sha256:0c29de6a1a95f24b9a1aa7aefd27d2487263f00dfd55a77719b530788f75cff7 \ + --hash=sha256:0c8c57f84ccfc871a48a47321cfa49ae1df56cd1d965a09abe84066f6853b9c0 \ + --hash=sha256:0f5d9ed7f254402c9e7d35d2f5972c9bbea9040e99cd2861bd77dc68263277c7 \ + --hash=sha256:18dd2e350387c87dabe711b86f83c9c78af772c748904d372ade190b5c7c9d4d \ + --hash=sha256:1b1bde144d98e446b056ef98e59c256e9294f6b74d7af6846bf5ffdafd687a7d \ + --hash=sha256:1c95a1e2902a8b722868587c0e1184ad5c55631de5afc0eb96bc4b0d738092c0 \ + --hash=sha256:1cad5f45b3146325bb38d6855642f6fd609c3f7cad4dbaf75549bf3b904d3184 \ + --hash=sha256:21b2899062867b0e1fde9b724f8aecb1af14f2778d69aacd1a5a1853a597a5db \ + --hash=sha256:24498ba8ed6c2e0b56d4acbf83f2d989720a93b41d712ebd4f4979660db4417b \ + --hash=sha256:25a23ea5c7edc53e0f29bae2c44fcb5a1aa10591aae107f2a2b2583a9c5cbc64 \ + --hash=sha256:289200a18fa698949d2b39c671c2cc7a24d44096784e76614899a7ccf2574b7b \ + --hash=sha256:28a1005facc94196e1fb3e82a3d442a9d9110b8434fc1ded7a24a2983c9888d8 \ + --hash=sha256:32fc0341d72e0f73f80acb0a2c94216bd704f4f0bce10aedea38f30502b271ff \ + --hash=sha256:36b31da18b8890a76ec181c3cf44326bf2c48e36d393ca1b72b3f484113ea344 \ + --hash=sha256:3c21d4fca343c805a52c0c78edc01e3477f6dd1ad7c47653241cf2a206d4fc58 \ + --hash=sha256:3fddb7e2c84ac87ac3a947cb4e66d143ca5863ef48e4a5ecb83bd48619e4634e \ + --hash=sha256:43e0933a0eff183ee85833f341ec567c0980dae57c464d8a508e1b2ceb336471 \ + --hash=sha256:4a476b06fbcf359ad25d34a057b7219281286ae2477cc5ff5e3f70a246971148 \ + --hash=sha256:4e594135de17ab3866138f496755f302b72157d115086d100c3f19370839dd3a \ + --hash=sha256:50bf98d5e563b83cc29471fa114366e6806bc06bc7a25fd59641e41445327836 \ + --hash=sha256:5a9979887252a82fefd3d3ed2a8e3b937a7a809f65dcb1e068b090e165bbe99e \ + --hash=sha256:5baececa9ecba31eff645232d59845c07aa030f0c81ee70184a90d35099a0e63 \ + --hash=sha256:5bf4545e3b962767e5c06fe1738f951f77d27967cb2caa64c28be7c4563e162c \ + --hash=sha256:6333b3aa5a12c26b2a4d4e7335a28f1475e0e5e17d69d55141ee3cab736f66d1 \ + --hash=sha256:65c981bdbd3f57670af8b59777cbfae75364b483fa8a9f420f08094531d54a01 \ + --hash=sha256:68a328e5f55ec37c57f19ebb1fdc56a248db2e3e9ad769919a58672958e8f366 \ + --hash=sha256:6a0289e4589e8bdfef02a80478f1dfcb14f0ab696b5a00e1f4b8a14a307a3c58 \ + --hash=sha256:6b66f92b17849b85cad91259efc341dce9c1af48e2173bf38a85c6329f1033e5 \ + --hash=sha256:6c9379d65defcab82d07b2a9dfbfc2e95bc8fe0ebb1b176a3190230a3ef0e07c \ + --hash=sha256:6fc1f5b51fa4cecaa18f2bd7a003f3dd039dd615cd69a2afd6d3b19aed6775f2 \ + --hash=sha256:70f7172939fdf8790425ba31915bfbe8335030f05b9913d7ae00a87d4395620a \ + --hash=sha256:721c76e84fe669be19c5791da68232ca2e05ba5185575086e384352e2c309597 \ + --hash=sha256:7222ffd5e4de8e57e03ce2cef95a4c43c98fcb72ad86909abdfc2c17d227fc1b \ + --hash=sha256:75d10d37a47afee94919c4fab4c22b9bc2a8bf7d4f46f87363bcf0573f3ff4f5 \ + --hash=sha256:76af085e67e56c8816c3ccf256ebd136def2ed9654525348cfa744b6802b69eb \ + --hash=sha256:770cab594ecf99ae64c236bc9ee3439c3f46be49796e265ce0cc8bc17b10294f \ + --hash=sha256:7a6ab32f7210554a96cd9e33abe3ddd86732beeafc7a28e9955cdf22ffadbab0 \ + --hash=sha256:7c48ed483eb946e6c04ccbe02c6b4d1d48e51944b6db70f697e089c193404941 \ + --hash=sha256:7f56930ab0abd1c45cd15be65cc741c28b1c9a34876ce8c17a2fa107810c0af0 \ + --hash=sha256:8075c35cd58273fee266c58c0c9b670947c19df5fb98e7b66710e04ad4e9ff86 \ + --hash=sha256:8272b73e1c5603666618805fe821edba66892e2870058c94c53147602eab29c7 \ + --hash=sha256:82d8fd25b7f4675d0c47cf95b594d4e7b158aca33b76aa63d07186e13c0e0ab7 \ + --hash=sha256:844da2b5728b5ce0e32d863af26f32b5ce61bc4273a9c720a9f3aa9df73b1455 \ + --hash=sha256:8755483f3c00d6c9a77f490c17e6ab0c8729e39e6390328e42521ef175380ae6 \ + --hash=sha256:915f3849a011c1f593ab99092f3cecfcb4d65d8feb4a64cf1bf2d22074dc0ec4 \ + --hash=sha256:926ca93accd5d36ccdabd803392ddc3e03e6d4cd1cf17deff3b989ab8e9dbcf0 \ + --hash=sha256:982bb1e8b4ffda883b3d0a521e23abcd6fd17418f6d2c4118d257a10199c0ce3 \ + --hash=sha256:98f862da73774290f251b9df8d11161b6cf25b599a66baf087c1ffe340e9bfd1 \ + --hash=sha256:9cbfacf36cb0ec2897ce0ebc5d08ca44213af24265bd56eca54bee7923c48fd6 \ + --hash=sha256:a370b3e078e418187da8c3674eddb9d983ec09445c99a3a263c2011993522981 \ + --hash=sha256:a955b438e62efdf7e0b7b52a64dc5c3396e2634baa62471768a64bc2adb73d5c \ + --hash=sha256:aa6af9e7d59f9c12b33ae4e9450619cf2488e2bbe9b44030905877f0b2324980 \ + --hash=sha256:aa88ca0b1932e93f2d961bf3addbb2db902198dca337d88c89e1559e066e7645 \ + --hash=sha256:aaeeb6a479c7667fbe1099af9617c83aaca22182d6cf8c53966491a0f1b7ffb7 \ + --hash=sha256:aaf27faa992bfee0264dc1f03f4c75e9fcdda66a519db6b957a3f826e285cf12 \ + --hash=sha256:b2680962a4848b3c4f155dc2ee64505a9c57186d0d56b43123b17ca3de18f0fa \ + --hash=sha256:b2d318c11350e10662026ad0eb71bb51c7812fc8590825304ae0bdd4ac283acd \ + --hash=sha256:b33de11b92e9f75a2b545d6e9b6f37e398d86c3e9e9653c4864eb7e89c5773ef \ + --hash=sha256:b3daeac64d5b371dea99714f08ffc2c208522ec6b06fbc7866a450dd446f5c0f \ + --hash=sha256:be1e352acbe3c78727a16a455126d9ff83ea2dfdcbc83148d2982305a04714c2 \ + --hash=sha256:bee093bf902e1d8fc0ac143c88902c3dfc8941f7ea1d6a8dd2bcb786d33db03d \ + --hash=sha256:c72fbbe68c6f32f251bdc08b8611c7b3060612236e960ef848e0a517ddbe76c5 \ + --hash=sha256:c9e36a97bee9b86ef9a1cf7bb96747eb7a15c2f22bdb5b516434b00f2a599f02 \ + --hash=sha256:cddf7bd982eaa998934a91f69d182aec997c6c468898efe6679af88283b498d3 \ + --hash=sha256:cf713fe9a71ef6fd5adf7a79670135081cd4431c2943864757f0fa3a65b1fafd \ + --hash=sha256:d11b54acf878eef558599658b0ffca78138c8c3655cf4f3a4a673c437e67732e \ + --hash=sha256:d41c4d287cfc69060fa91cae9683eacffad989f1a10811995fa309df656ec214 \ + --hash=sha256:d524ba3f1581b35c03cb42beebab4a13e6cdad7b36246bd22541fa585a56cccd \ + --hash=sha256:daac4765328a919a805fa5e2720f3e94767abd632ae410a9062dff5412bae65a \ + --hash=sha256:db4c7bf0e07fc3b7d89ac2a5880a6a8062056801b83ff56d8464b70f65482b6c \ + --hash=sha256:dc7039885fa1baf9be153a0626e337aa7ec8bf96b0128605fb0d77788ddc1681 \ + --hash=sha256:dccab8d5fa1ef9bfba0590ecf4d46df048d18ffe3eec01eeb73a42e0d9e7a8ba \ + --hash=sha256:dedb8adb91d11846ee08bec4c8236c8549ac721c245678282dcb06b221aab59f \ + --hash=sha256:e45ba65510e2647721e35323d6ef54c7974959f6081b58d4ef5d87c60c84919a \ + --hash=sha256:e53efc7c7cee4c1e70661e2e112ca46a575f90ed9ae3fef200f2a25e954f4b28 \ + --hash=sha256:e635b87f01ebc977342e2697d05b56632f5f879a4f15955dfe8cef2448b51691 \ + --hash=sha256:e70e990b2137b29dc5564715de1e12701815dacc1d056308e2b17e9095372a82 \ + --hash=sha256:e8082b26888e2f8b36a042a58307d5b917ef2b1cacab921ad3323ef91901c71a \ + --hash=sha256:e8323a9b031aa0393768b87f04b4164a40037fb2a3c11ac06a03ffecd3618027 \ + --hash=sha256:e92fca20c46e9f5e1bb485887d074918b13543b1c2a1185e69bb8d17ab6236a7 \ + --hash=sha256:eb30abc20df9ab0814b5a2524f23d75dcf83cde762c161917a2b4b7b55b1e518 \ + --hash=sha256:eba9904b0f38a143592d9fc0e19e2df0fa2e41c3c3745554761c5f6447eedabf \ + --hash=sha256:ef8de666d6179b009dce7bcb2ad4c4a779f113f12caf8dc77f0162c29d20490b \ + --hash=sha256:efd387a49825780ff861998cd959767800d54f8308936b21025326de4b5a42b9 \ + --hash=sha256:f0aa37f3c979cf2546b73e8222bbfa3dc07a641585340179d768068e3455e544 \ + --hash=sha256:f4074c5a429281bf056ddd4c5d3b740ebca4d43ffffe2ef4bf4d2d05114299da \ + --hash=sha256:f69a27e45c43520f5487f27627059b64aaf160415589230992cec34c5e18a509 \ + --hash=sha256:fb707f3e15060adf5b7ada797624a6c6e0138e2a26baa089df64c68ee98e040f \ + --hash=sha256:fcbe676a55d7445b22c10967bceaaf0ee69407fbe0ece4d032b6eb8d4565982a \ + --hash=sha256:fdb20a30fe1175ecabed17cbf7812f7b804b8a315a25f24678bcdf120a90077f # via requests docutils==0.21.2 \ --hash=sha256:3a6b18732edf182daa3cd12775bbb338cf5691468f91eeeb109deff6ebfa986f \ diff --git a/tools/publish/requirements_linux.txt b/tools/publish/requirements_linux.txt index 73edfce02f..6e43dab96c 100644 --- a/tools/publish/requirements_linux.txt +++ b/tools/publish/requirements_linux.txt @@ -79,99 +79,99 @@ cffi==1.17.1 \ --hash=sha256:f7f5baafcc48261359e14bcd6d9bff6d4b28d9103847c9e136694cb0501aef87 \ --hash=sha256:fc48c783f9c87e60831201f2cce7f3b2e4846bf4d8728eabe54d60700b318a0b # via cryptography -charset-normalizer==3.4.1 \ - --hash=sha256:0167ddc8ab6508fe81860a57dd472b2ef4060e8d378f0cc555707126830f2537 \ - --hash=sha256:01732659ba9b5b873fc117534143e4feefecf3b2078b0a6a2e925271bb6f4cfa \ - --hash=sha256:01ad647cdd609225c5350561d084b42ddf732f4eeefe6e678765636791e78b9a \ - --hash=sha256:04432ad9479fa40ec0f387795ddad4437a2b50417c69fa275e212933519ff294 \ - --hash=sha256:0907f11d019260cdc3f94fbdb23ff9125f6b5d1039b76003b5b0ac9d6a6c9d5b \ - --hash=sha256:0924e81d3d5e70f8126529951dac65c1010cdf117bb75eb02dd12339b57749dd \ - --hash=sha256:09b26ae6b1abf0d27570633b2b078a2a20419c99d66fb2823173d73f188ce601 \ - --hash=sha256:09b5e6733cbd160dcc09589227187e242a30a49ca5cefa5a7edd3f9d19ed53fd \ - --hash=sha256:0af291f4fe114be0280cdd29d533696a77b5b49cfde5467176ecab32353395c4 \ - --hash=sha256:0f55e69f030f7163dffe9fd0752b32f070566451afe180f99dbeeb81f511ad8d \ - --hash=sha256:1a2bc9f351a75ef49d664206d51f8e5ede9da246602dc2d2726837620ea034b2 \ - --hash=sha256:22e14b5d70560b8dd51ec22863f370d1e595ac3d024cb8ad7d308b4cd95f8313 \ - --hash=sha256:234ac59ea147c59ee4da87a0c0f098e9c8d169f4dc2a159ef720f1a61bbe27cd \ - --hash=sha256:2369eea1ee4a7610a860d88f268eb39b95cb588acd7235e02fd5a5601773d4fa \ - --hash=sha256:237bdbe6159cff53b4f24f397d43c6336c6b0b42affbe857970cefbb620911c8 \ - --hash=sha256:28bf57629c75e810b6ae989f03c0828d64d6b26a5e205535585f96093e405ed1 \ - --hash=sha256:2967f74ad52c3b98de4c3b32e1a44e32975e008a9cd2a8cc8966d6a5218c5cb2 \ - --hash=sha256:2a75d49014d118e4198bcee5ee0a6f25856b29b12dbf7cd012791f8a6cc5c496 \ - --hash=sha256:2bdfe3ac2e1bbe5b59a1a63721eb3b95fc9b6817ae4a46debbb4e11f6232428d \ - --hash=sha256:2d074908e1aecee37a7635990b2c6d504cd4766c7bc9fc86d63f9c09af3fa11b \ - --hash=sha256:2fb9bd477fdea8684f78791a6de97a953c51831ee2981f8e4f583ff3b9d9687e \ - --hash=sha256:311f30128d7d333eebd7896965bfcfbd0065f1716ec92bd5638d7748eb6f936a \ - --hash=sha256:329ce159e82018d646c7ac45b01a430369d526569ec08516081727a20e9e4af4 \ - --hash=sha256:345b0426edd4e18138d6528aed636de7a9ed169b4aaf9d61a8c19e39d26838ca \ - --hash=sha256:363e2f92b0f0174b2f8238240a1a30142e3db7b957a5dd5689b0e75fb717cc78 \ - --hash=sha256:3a3bd0dcd373514dcec91c411ddb9632c0d7d92aed7093b8c3bbb6d69ca74408 \ - --hash=sha256:3bed14e9c89dcb10e8f3a29f9ccac4955aebe93c71ae803af79265c9ca5644c5 \ - --hash=sha256:44251f18cd68a75b56585dd00dae26183e102cd5e0f9f1466e6df5da2ed64ea3 \ - --hash=sha256:44ecbf16649486d4aebafeaa7ec4c9fed8b88101f4dd612dcaf65d5e815f837f \ - --hash=sha256:4532bff1b8421fd0a320463030c7520f56a79c9024a4e88f01c537316019005a \ - --hash=sha256:49402233c892a461407c512a19435d1ce275543138294f7ef013f0b63d5d3765 \ - --hash=sha256:4c0907b1928a36d5a998d72d64d8eaa7244989f7aaaf947500d3a800c83a3fd6 \ - --hash=sha256:4d86f7aff21ee58f26dcf5ae81a9addbd914115cdebcbb2217e4f0ed8982e146 \ - --hash=sha256:5777ee0881f9499ed0f71cc82cf873d9a0ca8af166dfa0af8ec4e675b7df48e6 \ - --hash=sha256:5df196eb874dae23dcfb968c83d4f8fdccb333330fe1fc278ac5ceeb101003a9 \ - --hash=sha256:619a609aa74ae43d90ed2e89bdd784765de0a25ca761b93e196d938b8fd1dbbd \ - --hash=sha256:6e27f48bcd0957c6d4cb9d6fa6b61d192d0b13d5ef563e5f2ae35feafc0d179c \ - --hash=sha256:6ff8a4a60c227ad87030d76e99cd1698345d4491638dfa6673027c48b3cd395f \ - --hash=sha256:73d94b58ec7fecbc7366247d3b0b10a21681004153238750bb67bd9012414545 \ - --hash=sha256:7461baadb4dc00fd9e0acbe254e3d7d2112e7f92ced2adc96e54ef6501c5f176 \ - --hash=sha256:75832c08354f595c760a804588b9357d34ec00ba1c940c15e31e96d902093770 \ - --hash=sha256:7709f51f5f7c853f0fb938bcd3bc59cdfdc5203635ffd18bf354f6967ea0f824 \ - --hash=sha256:78baa6d91634dfb69ec52a463534bc0df05dbd546209b79a3880a34487f4b84f \ - --hash=sha256:7974a0b5ecd505609e3b19742b60cee7aa2aa2fb3151bc917e6e2646d7667dcf \ - --hash=sha256:7a4f97a081603d2050bfaffdefa5b02a9ec823f8348a572e39032caa8404a487 \ - --hash=sha256:7b1bef6280950ee6c177b326508f86cad7ad4dff12454483b51d8b7d673a2c5d \ - --hash=sha256:7d053096f67cd1241601111b698f5cad775f97ab25d81567d3f59219b5f1adbd \ - --hash=sha256:804a4d582ba6e5b747c625bf1255e6b1507465494a40a2130978bda7b932c90b \ - --hash=sha256:807f52c1f798eef6cf26beb819eeb8819b1622ddfeef9d0977a8502d4db6d534 \ - --hash=sha256:80ed5e856eb7f30115aaf94e4a08114ccc8813e6ed1b5efa74f9f82e8509858f \ - --hash=sha256:8417cb1f36cc0bc7eaba8ccb0e04d55f0ee52df06df3ad55259b9a323555fc8b \ - --hash=sha256:8436c508b408b82d87dc5f62496973a1805cd46727c34440b0d29d8a2f50a6c9 \ - --hash=sha256:89149166622f4db9b4b6a449256291dc87a99ee53151c74cbd82a53c8c2f6ccd \ - --hash=sha256:8bfa33f4f2672964266e940dd22a195989ba31669bd84629f05fab3ef4e2d125 \ - --hash=sha256:8c60ca7339acd497a55b0ea5d506b2a2612afb2826560416f6894e8b5770d4a9 \ - --hash=sha256:91b36a978b5ae0ee86c394f5a54d6ef44db1de0815eb43de826d41d21e4af3de \ - --hash=sha256:955f8851919303c92343d2f66165294848d57e9bba6cf6e3625485a70a038d11 \ - --hash=sha256:97f68b8d6831127e4787ad15e6757232e14e12060bec17091b85eb1486b91d8d \ - --hash=sha256:9b23ca7ef998bc739bf6ffc077c2116917eabcc901f88da1b9856b210ef63f35 \ - --hash=sha256:9f0b8b1c6d84c8034a44893aba5e767bf9c7a211e313a9605d9c617d7083829f \ - --hash=sha256:aabfa34badd18f1da5ec1bc2715cadc8dca465868a4e73a0173466b688f29dda \ - --hash=sha256:ab36c8eb7e454e34e60eb55ca5d241a5d18b2c6244f6827a30e451c42410b5f7 \ - --hash=sha256:b010a7a4fd316c3c484d482922d13044979e78d1861f0e0650423144c616a46a \ - --hash=sha256:b1ac5992a838106edb89654e0aebfc24f5848ae2547d22c2c3f66454daa11971 \ - --hash=sha256:b7b2d86dd06bfc2ade3312a83a5c364c7ec2e3498f8734282c6c3d4b07b346b8 \ - --hash=sha256:b97e690a2118911e39b4042088092771b4ae3fc3aa86518f84b8cf6888dbdb41 \ - --hash=sha256:bc2722592d8998c870fa4e290c2eec2c1569b87fe58618e67d38b4665dfa680d \ - --hash=sha256:c0429126cf75e16c4f0ad00ee0eae4242dc652290f940152ca8c75c3a4b6ee8f \ - --hash=sha256:c30197aa96e8eed02200a83fba2657b4c3acd0f0aa4bdc9f6c1af8e8962e0757 \ - --hash=sha256:c4c3e6da02df6fa1410a7680bd3f63d4f710232d3139089536310d027950696a \ - --hash=sha256:c75cb2a3e389853835e84a2d8fb2b81a10645b503eca9bcb98df6b5a43eb8886 \ - --hash=sha256:c96836c97b1238e9c9e3fe90844c947d5afbf4f4c92762679acfe19927d81d77 \ - --hash=sha256:d7f50a1f8c450f3925cb367d011448c39239bb3eb4117c36a6d354794de4ce76 \ - --hash=sha256:d973f03c0cb71c5ed99037b870f2be986c3c05e63622c017ea9816881d2dd247 \ - --hash=sha256:d98b1668f06378c6dbefec3b92299716b931cd4e6061f3c875a71ced1780ab85 \ - --hash=sha256:d9c3cdf5390dcd29aa8056d13e8e99526cda0305acc038b96b30352aff5ff2bb \ - --hash=sha256:dad3e487649f498dd991eeb901125411559b22e8d7ab25d3aeb1af367df5efd7 \ - --hash=sha256:dccbe65bd2f7f7ec22c4ff99ed56faa1e9f785482b9bbd7c717e26fd723a1d1e \ - --hash=sha256:dd78cfcda14a1ef52584dbb008f7ac81c1328c0f58184bf9a84c49c605002da6 \ - --hash=sha256:e218488cd232553829be0664c2292d3af2eeeb94b32bea483cf79ac6a694e037 \ - --hash=sha256:e358e64305fe12299a08e08978f51fc21fac060dcfcddd95453eabe5b93ed0e1 \ - --hash=sha256:ea0d8d539afa5eb2728aa1932a988a9a7af94f18582ffae4bc10b3fbdad0626e \ - --hash=sha256:eab677309cdb30d047996b36d34caeda1dc91149e4fdca0b1a039b3f79d9a807 \ - --hash=sha256:eb8178fe3dba6450a3e024e95ac49ed3400e506fd4e9e5c32d30adda88cbd407 \ - --hash=sha256:ecddf25bee22fe4fe3737a399d0d177d72bc22be6913acfab364b40bce1ba83c \ - --hash=sha256:eea6ee1db730b3483adf394ea72f808b6e18cf3cb6454b4d86e04fa8c4327a12 \ - --hash=sha256:f08ff5e948271dc7e18a35641d2f11a4cd8dfd5634f55228b691e62b37125eb3 \ - --hash=sha256:f30bf9fd9be89ecb2360c7d94a711f00c09b976258846efe40db3d05828e8089 \ - --hash=sha256:fa88b843d6e211393a37219e6a1c1df99d35e8fd90446f1118f4216e307e48cd \ - --hash=sha256:fc54db6c8593ef7d4b2a331b58653356cf04f67c960f584edb7c3d8c97e8f39e \ - --hash=sha256:fd4ec41f914fa74ad1b8304bbc634b3de73d2a0889bd32076342a573e0779e00 \ - --hash=sha256:ffc9202a29ab3920fa812879e95a9e78b2465fd10be7fcbd042899695d75e616 +charset-normalizer==3.4.2 \ + --hash=sha256:005fa3432484527f9732ebd315da8da8001593e2cf46a3d817669f062c3d9ed4 \ + --hash=sha256:046595208aae0120559a67693ecc65dd75d46f7bf687f159127046628178dc45 \ + --hash=sha256:0c29de6a1a95f24b9a1aa7aefd27d2487263f00dfd55a77719b530788f75cff7 \ + --hash=sha256:0c8c57f84ccfc871a48a47321cfa49ae1df56cd1d965a09abe84066f6853b9c0 \ + --hash=sha256:0f5d9ed7f254402c9e7d35d2f5972c9bbea9040e99cd2861bd77dc68263277c7 \ + --hash=sha256:18dd2e350387c87dabe711b86f83c9c78af772c748904d372ade190b5c7c9d4d \ + --hash=sha256:1b1bde144d98e446b056ef98e59c256e9294f6b74d7af6846bf5ffdafd687a7d \ + --hash=sha256:1c95a1e2902a8b722868587c0e1184ad5c55631de5afc0eb96bc4b0d738092c0 \ + --hash=sha256:1cad5f45b3146325bb38d6855642f6fd609c3f7cad4dbaf75549bf3b904d3184 \ + --hash=sha256:21b2899062867b0e1fde9b724f8aecb1af14f2778d69aacd1a5a1853a597a5db \ + --hash=sha256:24498ba8ed6c2e0b56d4acbf83f2d989720a93b41d712ebd4f4979660db4417b \ + --hash=sha256:25a23ea5c7edc53e0f29bae2c44fcb5a1aa10591aae107f2a2b2583a9c5cbc64 \ + --hash=sha256:289200a18fa698949d2b39c671c2cc7a24d44096784e76614899a7ccf2574b7b \ + --hash=sha256:28a1005facc94196e1fb3e82a3d442a9d9110b8434fc1ded7a24a2983c9888d8 \ + --hash=sha256:32fc0341d72e0f73f80acb0a2c94216bd704f4f0bce10aedea38f30502b271ff \ + --hash=sha256:36b31da18b8890a76ec181c3cf44326bf2c48e36d393ca1b72b3f484113ea344 \ + --hash=sha256:3c21d4fca343c805a52c0c78edc01e3477f6dd1ad7c47653241cf2a206d4fc58 \ + --hash=sha256:3fddb7e2c84ac87ac3a947cb4e66d143ca5863ef48e4a5ecb83bd48619e4634e \ + --hash=sha256:43e0933a0eff183ee85833f341ec567c0980dae57c464d8a508e1b2ceb336471 \ + --hash=sha256:4a476b06fbcf359ad25d34a057b7219281286ae2477cc5ff5e3f70a246971148 \ + --hash=sha256:4e594135de17ab3866138f496755f302b72157d115086d100c3f19370839dd3a \ + --hash=sha256:50bf98d5e563b83cc29471fa114366e6806bc06bc7a25fd59641e41445327836 \ + --hash=sha256:5a9979887252a82fefd3d3ed2a8e3b937a7a809f65dcb1e068b090e165bbe99e \ + --hash=sha256:5baececa9ecba31eff645232d59845c07aa030f0c81ee70184a90d35099a0e63 \ + --hash=sha256:5bf4545e3b962767e5c06fe1738f951f77d27967cb2caa64c28be7c4563e162c \ + --hash=sha256:6333b3aa5a12c26b2a4d4e7335a28f1475e0e5e17d69d55141ee3cab736f66d1 \ + --hash=sha256:65c981bdbd3f57670af8b59777cbfae75364b483fa8a9f420f08094531d54a01 \ + --hash=sha256:68a328e5f55ec37c57f19ebb1fdc56a248db2e3e9ad769919a58672958e8f366 \ + --hash=sha256:6a0289e4589e8bdfef02a80478f1dfcb14f0ab696b5a00e1f4b8a14a307a3c58 \ + --hash=sha256:6b66f92b17849b85cad91259efc341dce9c1af48e2173bf38a85c6329f1033e5 \ + --hash=sha256:6c9379d65defcab82d07b2a9dfbfc2e95bc8fe0ebb1b176a3190230a3ef0e07c \ + --hash=sha256:6fc1f5b51fa4cecaa18f2bd7a003f3dd039dd615cd69a2afd6d3b19aed6775f2 \ + --hash=sha256:70f7172939fdf8790425ba31915bfbe8335030f05b9913d7ae00a87d4395620a \ + --hash=sha256:721c76e84fe669be19c5791da68232ca2e05ba5185575086e384352e2c309597 \ + --hash=sha256:7222ffd5e4de8e57e03ce2cef95a4c43c98fcb72ad86909abdfc2c17d227fc1b \ + --hash=sha256:75d10d37a47afee94919c4fab4c22b9bc2a8bf7d4f46f87363bcf0573f3ff4f5 \ + --hash=sha256:76af085e67e56c8816c3ccf256ebd136def2ed9654525348cfa744b6802b69eb \ + --hash=sha256:770cab594ecf99ae64c236bc9ee3439c3f46be49796e265ce0cc8bc17b10294f \ + --hash=sha256:7a6ab32f7210554a96cd9e33abe3ddd86732beeafc7a28e9955cdf22ffadbab0 \ + --hash=sha256:7c48ed483eb946e6c04ccbe02c6b4d1d48e51944b6db70f697e089c193404941 \ + --hash=sha256:7f56930ab0abd1c45cd15be65cc741c28b1c9a34876ce8c17a2fa107810c0af0 \ + --hash=sha256:8075c35cd58273fee266c58c0c9b670947c19df5fb98e7b66710e04ad4e9ff86 \ + --hash=sha256:8272b73e1c5603666618805fe821edba66892e2870058c94c53147602eab29c7 \ + --hash=sha256:82d8fd25b7f4675d0c47cf95b594d4e7b158aca33b76aa63d07186e13c0e0ab7 \ + --hash=sha256:844da2b5728b5ce0e32d863af26f32b5ce61bc4273a9c720a9f3aa9df73b1455 \ + --hash=sha256:8755483f3c00d6c9a77f490c17e6ab0c8729e39e6390328e42521ef175380ae6 \ + --hash=sha256:915f3849a011c1f593ab99092f3cecfcb4d65d8feb4a64cf1bf2d22074dc0ec4 \ + --hash=sha256:926ca93accd5d36ccdabd803392ddc3e03e6d4cd1cf17deff3b989ab8e9dbcf0 \ + --hash=sha256:982bb1e8b4ffda883b3d0a521e23abcd6fd17418f6d2c4118d257a10199c0ce3 \ + --hash=sha256:98f862da73774290f251b9df8d11161b6cf25b599a66baf087c1ffe340e9bfd1 \ + --hash=sha256:9cbfacf36cb0ec2897ce0ebc5d08ca44213af24265bd56eca54bee7923c48fd6 \ + --hash=sha256:a370b3e078e418187da8c3674eddb9d983ec09445c99a3a263c2011993522981 \ + --hash=sha256:a955b438e62efdf7e0b7b52a64dc5c3396e2634baa62471768a64bc2adb73d5c \ + --hash=sha256:aa6af9e7d59f9c12b33ae4e9450619cf2488e2bbe9b44030905877f0b2324980 \ + --hash=sha256:aa88ca0b1932e93f2d961bf3addbb2db902198dca337d88c89e1559e066e7645 \ + --hash=sha256:aaeeb6a479c7667fbe1099af9617c83aaca22182d6cf8c53966491a0f1b7ffb7 \ + --hash=sha256:aaf27faa992bfee0264dc1f03f4c75e9fcdda66a519db6b957a3f826e285cf12 \ + --hash=sha256:b2680962a4848b3c4f155dc2ee64505a9c57186d0d56b43123b17ca3de18f0fa \ + --hash=sha256:b2d318c11350e10662026ad0eb71bb51c7812fc8590825304ae0bdd4ac283acd \ + --hash=sha256:b33de11b92e9f75a2b545d6e9b6f37e398d86c3e9e9653c4864eb7e89c5773ef \ + --hash=sha256:b3daeac64d5b371dea99714f08ffc2c208522ec6b06fbc7866a450dd446f5c0f \ + --hash=sha256:be1e352acbe3c78727a16a455126d9ff83ea2dfdcbc83148d2982305a04714c2 \ + --hash=sha256:bee093bf902e1d8fc0ac143c88902c3dfc8941f7ea1d6a8dd2bcb786d33db03d \ + --hash=sha256:c72fbbe68c6f32f251bdc08b8611c7b3060612236e960ef848e0a517ddbe76c5 \ + --hash=sha256:c9e36a97bee9b86ef9a1cf7bb96747eb7a15c2f22bdb5b516434b00f2a599f02 \ + --hash=sha256:cddf7bd982eaa998934a91f69d182aec997c6c468898efe6679af88283b498d3 \ + --hash=sha256:cf713fe9a71ef6fd5adf7a79670135081cd4431c2943864757f0fa3a65b1fafd \ + --hash=sha256:d11b54acf878eef558599658b0ffca78138c8c3655cf4f3a4a673c437e67732e \ + --hash=sha256:d41c4d287cfc69060fa91cae9683eacffad989f1a10811995fa309df656ec214 \ + --hash=sha256:d524ba3f1581b35c03cb42beebab4a13e6cdad7b36246bd22541fa585a56cccd \ + --hash=sha256:daac4765328a919a805fa5e2720f3e94767abd632ae410a9062dff5412bae65a \ + --hash=sha256:db4c7bf0e07fc3b7d89ac2a5880a6a8062056801b83ff56d8464b70f65482b6c \ + --hash=sha256:dc7039885fa1baf9be153a0626e337aa7ec8bf96b0128605fb0d77788ddc1681 \ + --hash=sha256:dccab8d5fa1ef9bfba0590ecf4d46df048d18ffe3eec01eeb73a42e0d9e7a8ba \ + --hash=sha256:dedb8adb91d11846ee08bec4c8236c8549ac721c245678282dcb06b221aab59f \ + --hash=sha256:e45ba65510e2647721e35323d6ef54c7974959f6081b58d4ef5d87c60c84919a \ + --hash=sha256:e53efc7c7cee4c1e70661e2e112ca46a575f90ed9ae3fef200f2a25e954f4b28 \ + --hash=sha256:e635b87f01ebc977342e2697d05b56632f5f879a4f15955dfe8cef2448b51691 \ + --hash=sha256:e70e990b2137b29dc5564715de1e12701815dacc1d056308e2b17e9095372a82 \ + --hash=sha256:e8082b26888e2f8b36a042a58307d5b917ef2b1cacab921ad3323ef91901c71a \ + --hash=sha256:e8323a9b031aa0393768b87f04b4164a40037fb2a3c11ac06a03ffecd3618027 \ + --hash=sha256:e92fca20c46e9f5e1bb485887d074918b13543b1c2a1185e69bb8d17ab6236a7 \ + --hash=sha256:eb30abc20df9ab0814b5a2524f23d75dcf83cde762c161917a2b4b7b55b1e518 \ + --hash=sha256:eba9904b0f38a143592d9fc0e19e2df0fa2e41c3c3745554761c5f6447eedabf \ + --hash=sha256:ef8de666d6179b009dce7bcb2ad4c4a779f113f12caf8dc77f0162c29d20490b \ + --hash=sha256:efd387a49825780ff861998cd959767800d54f8308936b21025326de4b5a42b9 \ + --hash=sha256:f0aa37f3c979cf2546b73e8222bbfa3dc07a641585340179d768068e3455e544 \ + --hash=sha256:f4074c5a429281bf056ddd4c5d3b740ebca4d43ffffe2ef4bf4d2d05114299da \ + --hash=sha256:f69a27e45c43520f5487f27627059b64aaf160415589230992cec34c5e18a509 \ + --hash=sha256:fb707f3e15060adf5b7ada797624a6c6e0138e2a26baa089df64c68ee98e040f \ + --hash=sha256:fcbe676a55d7445b22c10967bceaaf0ee69407fbe0ece4d032b6eb8d4565982a \ + --hash=sha256:fdb20a30fe1175ecabed17cbf7812f7b804b8a315a25f24678bcdf120a90077f # via requests cryptography==44.0.1 \ --hash=sha256:00918d859aa4e57db8299607086f793fa7813ae2ff5a4637e318a25ef82730f7 \ diff --git a/tools/publish/requirements_universal.txt b/tools/publish/requirements_universal.txt index c080f1d7de..92addf96ac 100644 --- a/tools/publish/requirements_universal.txt +++ b/tools/publish/requirements_universal.txt @@ -79,99 +79,99 @@ cffi==1.17.1 ; platform_python_implementation != 'PyPy' and sys_platform == 'lin --hash=sha256:f7f5baafcc48261359e14bcd6d9bff6d4b28d9103847c9e136694cb0501aef87 \ --hash=sha256:fc48c783f9c87e60831201f2cce7f3b2e4846bf4d8728eabe54d60700b318a0b # via cryptography -charset-normalizer==3.4.1 \ - --hash=sha256:0167ddc8ab6508fe81860a57dd472b2ef4060e8d378f0cc555707126830f2537 \ - --hash=sha256:01732659ba9b5b873fc117534143e4feefecf3b2078b0a6a2e925271bb6f4cfa \ - --hash=sha256:01ad647cdd609225c5350561d084b42ddf732f4eeefe6e678765636791e78b9a \ - --hash=sha256:04432ad9479fa40ec0f387795ddad4437a2b50417c69fa275e212933519ff294 \ - --hash=sha256:0907f11d019260cdc3f94fbdb23ff9125f6b5d1039b76003b5b0ac9d6a6c9d5b \ - --hash=sha256:0924e81d3d5e70f8126529951dac65c1010cdf117bb75eb02dd12339b57749dd \ - --hash=sha256:09b26ae6b1abf0d27570633b2b078a2a20419c99d66fb2823173d73f188ce601 \ - --hash=sha256:09b5e6733cbd160dcc09589227187e242a30a49ca5cefa5a7edd3f9d19ed53fd \ - --hash=sha256:0af291f4fe114be0280cdd29d533696a77b5b49cfde5467176ecab32353395c4 \ - --hash=sha256:0f55e69f030f7163dffe9fd0752b32f070566451afe180f99dbeeb81f511ad8d \ - --hash=sha256:1a2bc9f351a75ef49d664206d51f8e5ede9da246602dc2d2726837620ea034b2 \ - --hash=sha256:22e14b5d70560b8dd51ec22863f370d1e595ac3d024cb8ad7d308b4cd95f8313 \ - --hash=sha256:234ac59ea147c59ee4da87a0c0f098e9c8d169f4dc2a159ef720f1a61bbe27cd \ - --hash=sha256:2369eea1ee4a7610a860d88f268eb39b95cb588acd7235e02fd5a5601773d4fa \ - --hash=sha256:237bdbe6159cff53b4f24f397d43c6336c6b0b42affbe857970cefbb620911c8 \ - --hash=sha256:28bf57629c75e810b6ae989f03c0828d64d6b26a5e205535585f96093e405ed1 \ - --hash=sha256:2967f74ad52c3b98de4c3b32e1a44e32975e008a9cd2a8cc8966d6a5218c5cb2 \ - --hash=sha256:2a75d49014d118e4198bcee5ee0a6f25856b29b12dbf7cd012791f8a6cc5c496 \ - --hash=sha256:2bdfe3ac2e1bbe5b59a1a63721eb3b95fc9b6817ae4a46debbb4e11f6232428d \ - --hash=sha256:2d074908e1aecee37a7635990b2c6d504cd4766c7bc9fc86d63f9c09af3fa11b \ - --hash=sha256:2fb9bd477fdea8684f78791a6de97a953c51831ee2981f8e4f583ff3b9d9687e \ - --hash=sha256:311f30128d7d333eebd7896965bfcfbd0065f1716ec92bd5638d7748eb6f936a \ - --hash=sha256:329ce159e82018d646c7ac45b01a430369d526569ec08516081727a20e9e4af4 \ - --hash=sha256:345b0426edd4e18138d6528aed636de7a9ed169b4aaf9d61a8c19e39d26838ca \ - --hash=sha256:363e2f92b0f0174b2f8238240a1a30142e3db7b957a5dd5689b0e75fb717cc78 \ - --hash=sha256:3a3bd0dcd373514dcec91c411ddb9632c0d7d92aed7093b8c3bbb6d69ca74408 \ - --hash=sha256:3bed14e9c89dcb10e8f3a29f9ccac4955aebe93c71ae803af79265c9ca5644c5 \ - --hash=sha256:44251f18cd68a75b56585dd00dae26183e102cd5e0f9f1466e6df5da2ed64ea3 \ - --hash=sha256:44ecbf16649486d4aebafeaa7ec4c9fed8b88101f4dd612dcaf65d5e815f837f \ - --hash=sha256:4532bff1b8421fd0a320463030c7520f56a79c9024a4e88f01c537316019005a \ - --hash=sha256:49402233c892a461407c512a19435d1ce275543138294f7ef013f0b63d5d3765 \ - --hash=sha256:4c0907b1928a36d5a998d72d64d8eaa7244989f7aaaf947500d3a800c83a3fd6 \ - --hash=sha256:4d86f7aff21ee58f26dcf5ae81a9addbd914115cdebcbb2217e4f0ed8982e146 \ - --hash=sha256:5777ee0881f9499ed0f71cc82cf873d9a0ca8af166dfa0af8ec4e675b7df48e6 \ - --hash=sha256:5df196eb874dae23dcfb968c83d4f8fdccb333330fe1fc278ac5ceeb101003a9 \ - --hash=sha256:619a609aa74ae43d90ed2e89bdd784765de0a25ca761b93e196d938b8fd1dbbd \ - --hash=sha256:6e27f48bcd0957c6d4cb9d6fa6b61d192d0b13d5ef563e5f2ae35feafc0d179c \ - --hash=sha256:6ff8a4a60c227ad87030d76e99cd1698345d4491638dfa6673027c48b3cd395f \ - --hash=sha256:73d94b58ec7fecbc7366247d3b0b10a21681004153238750bb67bd9012414545 \ - --hash=sha256:7461baadb4dc00fd9e0acbe254e3d7d2112e7f92ced2adc96e54ef6501c5f176 \ - --hash=sha256:75832c08354f595c760a804588b9357d34ec00ba1c940c15e31e96d902093770 \ - --hash=sha256:7709f51f5f7c853f0fb938bcd3bc59cdfdc5203635ffd18bf354f6967ea0f824 \ - --hash=sha256:78baa6d91634dfb69ec52a463534bc0df05dbd546209b79a3880a34487f4b84f \ - --hash=sha256:7974a0b5ecd505609e3b19742b60cee7aa2aa2fb3151bc917e6e2646d7667dcf \ - --hash=sha256:7a4f97a081603d2050bfaffdefa5b02a9ec823f8348a572e39032caa8404a487 \ - --hash=sha256:7b1bef6280950ee6c177b326508f86cad7ad4dff12454483b51d8b7d673a2c5d \ - --hash=sha256:7d053096f67cd1241601111b698f5cad775f97ab25d81567d3f59219b5f1adbd \ - --hash=sha256:804a4d582ba6e5b747c625bf1255e6b1507465494a40a2130978bda7b932c90b \ - --hash=sha256:807f52c1f798eef6cf26beb819eeb8819b1622ddfeef9d0977a8502d4db6d534 \ - --hash=sha256:80ed5e856eb7f30115aaf94e4a08114ccc8813e6ed1b5efa74f9f82e8509858f \ - --hash=sha256:8417cb1f36cc0bc7eaba8ccb0e04d55f0ee52df06df3ad55259b9a323555fc8b \ - --hash=sha256:8436c508b408b82d87dc5f62496973a1805cd46727c34440b0d29d8a2f50a6c9 \ - --hash=sha256:89149166622f4db9b4b6a449256291dc87a99ee53151c74cbd82a53c8c2f6ccd \ - --hash=sha256:8bfa33f4f2672964266e940dd22a195989ba31669bd84629f05fab3ef4e2d125 \ - --hash=sha256:8c60ca7339acd497a55b0ea5d506b2a2612afb2826560416f6894e8b5770d4a9 \ - --hash=sha256:91b36a978b5ae0ee86c394f5a54d6ef44db1de0815eb43de826d41d21e4af3de \ - --hash=sha256:955f8851919303c92343d2f66165294848d57e9bba6cf6e3625485a70a038d11 \ - --hash=sha256:97f68b8d6831127e4787ad15e6757232e14e12060bec17091b85eb1486b91d8d \ - --hash=sha256:9b23ca7ef998bc739bf6ffc077c2116917eabcc901f88da1b9856b210ef63f35 \ - --hash=sha256:9f0b8b1c6d84c8034a44893aba5e767bf9c7a211e313a9605d9c617d7083829f \ - --hash=sha256:aabfa34badd18f1da5ec1bc2715cadc8dca465868a4e73a0173466b688f29dda \ - --hash=sha256:ab36c8eb7e454e34e60eb55ca5d241a5d18b2c6244f6827a30e451c42410b5f7 \ - --hash=sha256:b010a7a4fd316c3c484d482922d13044979e78d1861f0e0650423144c616a46a \ - --hash=sha256:b1ac5992a838106edb89654e0aebfc24f5848ae2547d22c2c3f66454daa11971 \ - --hash=sha256:b7b2d86dd06bfc2ade3312a83a5c364c7ec2e3498f8734282c6c3d4b07b346b8 \ - --hash=sha256:b97e690a2118911e39b4042088092771b4ae3fc3aa86518f84b8cf6888dbdb41 \ - --hash=sha256:bc2722592d8998c870fa4e290c2eec2c1569b87fe58618e67d38b4665dfa680d \ - --hash=sha256:c0429126cf75e16c4f0ad00ee0eae4242dc652290f940152ca8c75c3a4b6ee8f \ - --hash=sha256:c30197aa96e8eed02200a83fba2657b4c3acd0f0aa4bdc9f6c1af8e8962e0757 \ - --hash=sha256:c4c3e6da02df6fa1410a7680bd3f63d4f710232d3139089536310d027950696a \ - --hash=sha256:c75cb2a3e389853835e84a2d8fb2b81a10645b503eca9bcb98df6b5a43eb8886 \ - --hash=sha256:c96836c97b1238e9c9e3fe90844c947d5afbf4f4c92762679acfe19927d81d77 \ - --hash=sha256:d7f50a1f8c450f3925cb367d011448c39239bb3eb4117c36a6d354794de4ce76 \ - --hash=sha256:d973f03c0cb71c5ed99037b870f2be986c3c05e63622c017ea9816881d2dd247 \ - --hash=sha256:d98b1668f06378c6dbefec3b92299716b931cd4e6061f3c875a71ced1780ab85 \ - --hash=sha256:d9c3cdf5390dcd29aa8056d13e8e99526cda0305acc038b96b30352aff5ff2bb \ - --hash=sha256:dad3e487649f498dd991eeb901125411559b22e8d7ab25d3aeb1af367df5efd7 \ - --hash=sha256:dccbe65bd2f7f7ec22c4ff99ed56faa1e9f785482b9bbd7c717e26fd723a1d1e \ - --hash=sha256:dd78cfcda14a1ef52584dbb008f7ac81c1328c0f58184bf9a84c49c605002da6 \ - --hash=sha256:e218488cd232553829be0664c2292d3af2eeeb94b32bea483cf79ac6a694e037 \ - --hash=sha256:e358e64305fe12299a08e08978f51fc21fac060dcfcddd95453eabe5b93ed0e1 \ - --hash=sha256:ea0d8d539afa5eb2728aa1932a988a9a7af94f18582ffae4bc10b3fbdad0626e \ - --hash=sha256:eab677309cdb30d047996b36d34caeda1dc91149e4fdca0b1a039b3f79d9a807 \ - --hash=sha256:eb8178fe3dba6450a3e024e95ac49ed3400e506fd4e9e5c32d30adda88cbd407 \ - --hash=sha256:ecddf25bee22fe4fe3737a399d0d177d72bc22be6913acfab364b40bce1ba83c \ - --hash=sha256:eea6ee1db730b3483adf394ea72f808b6e18cf3cb6454b4d86e04fa8c4327a12 \ - --hash=sha256:f08ff5e948271dc7e18a35641d2f11a4cd8dfd5634f55228b691e62b37125eb3 \ - --hash=sha256:f30bf9fd9be89ecb2360c7d94a711f00c09b976258846efe40db3d05828e8089 \ - --hash=sha256:fa88b843d6e211393a37219e6a1c1df99d35e8fd90446f1118f4216e307e48cd \ - --hash=sha256:fc54db6c8593ef7d4b2a331b58653356cf04f67c960f584edb7c3d8c97e8f39e \ - --hash=sha256:fd4ec41f914fa74ad1b8304bbc634b3de73d2a0889bd32076342a573e0779e00 \ - --hash=sha256:ffc9202a29ab3920fa812879e95a9e78b2465fd10be7fcbd042899695d75e616 +charset-normalizer==3.4.2 \ + --hash=sha256:005fa3432484527f9732ebd315da8da8001593e2cf46a3d817669f062c3d9ed4 \ + --hash=sha256:046595208aae0120559a67693ecc65dd75d46f7bf687f159127046628178dc45 \ + --hash=sha256:0c29de6a1a95f24b9a1aa7aefd27d2487263f00dfd55a77719b530788f75cff7 \ + --hash=sha256:0c8c57f84ccfc871a48a47321cfa49ae1df56cd1d965a09abe84066f6853b9c0 \ + --hash=sha256:0f5d9ed7f254402c9e7d35d2f5972c9bbea9040e99cd2861bd77dc68263277c7 \ + --hash=sha256:18dd2e350387c87dabe711b86f83c9c78af772c748904d372ade190b5c7c9d4d \ + --hash=sha256:1b1bde144d98e446b056ef98e59c256e9294f6b74d7af6846bf5ffdafd687a7d \ + --hash=sha256:1c95a1e2902a8b722868587c0e1184ad5c55631de5afc0eb96bc4b0d738092c0 \ + --hash=sha256:1cad5f45b3146325bb38d6855642f6fd609c3f7cad4dbaf75549bf3b904d3184 \ + --hash=sha256:21b2899062867b0e1fde9b724f8aecb1af14f2778d69aacd1a5a1853a597a5db \ + --hash=sha256:24498ba8ed6c2e0b56d4acbf83f2d989720a93b41d712ebd4f4979660db4417b \ + --hash=sha256:25a23ea5c7edc53e0f29bae2c44fcb5a1aa10591aae107f2a2b2583a9c5cbc64 \ + --hash=sha256:289200a18fa698949d2b39c671c2cc7a24d44096784e76614899a7ccf2574b7b \ + --hash=sha256:28a1005facc94196e1fb3e82a3d442a9d9110b8434fc1ded7a24a2983c9888d8 \ + --hash=sha256:32fc0341d72e0f73f80acb0a2c94216bd704f4f0bce10aedea38f30502b271ff \ + --hash=sha256:36b31da18b8890a76ec181c3cf44326bf2c48e36d393ca1b72b3f484113ea344 \ + --hash=sha256:3c21d4fca343c805a52c0c78edc01e3477f6dd1ad7c47653241cf2a206d4fc58 \ + --hash=sha256:3fddb7e2c84ac87ac3a947cb4e66d143ca5863ef48e4a5ecb83bd48619e4634e \ + --hash=sha256:43e0933a0eff183ee85833f341ec567c0980dae57c464d8a508e1b2ceb336471 \ + --hash=sha256:4a476b06fbcf359ad25d34a057b7219281286ae2477cc5ff5e3f70a246971148 \ + --hash=sha256:4e594135de17ab3866138f496755f302b72157d115086d100c3f19370839dd3a \ + --hash=sha256:50bf98d5e563b83cc29471fa114366e6806bc06bc7a25fd59641e41445327836 \ + --hash=sha256:5a9979887252a82fefd3d3ed2a8e3b937a7a809f65dcb1e068b090e165bbe99e \ + --hash=sha256:5baececa9ecba31eff645232d59845c07aa030f0c81ee70184a90d35099a0e63 \ + --hash=sha256:5bf4545e3b962767e5c06fe1738f951f77d27967cb2caa64c28be7c4563e162c \ + --hash=sha256:6333b3aa5a12c26b2a4d4e7335a28f1475e0e5e17d69d55141ee3cab736f66d1 \ + --hash=sha256:65c981bdbd3f57670af8b59777cbfae75364b483fa8a9f420f08094531d54a01 \ + --hash=sha256:68a328e5f55ec37c57f19ebb1fdc56a248db2e3e9ad769919a58672958e8f366 \ + --hash=sha256:6a0289e4589e8bdfef02a80478f1dfcb14f0ab696b5a00e1f4b8a14a307a3c58 \ + --hash=sha256:6b66f92b17849b85cad91259efc341dce9c1af48e2173bf38a85c6329f1033e5 \ + --hash=sha256:6c9379d65defcab82d07b2a9dfbfc2e95bc8fe0ebb1b176a3190230a3ef0e07c \ + --hash=sha256:6fc1f5b51fa4cecaa18f2bd7a003f3dd039dd615cd69a2afd6d3b19aed6775f2 \ + --hash=sha256:70f7172939fdf8790425ba31915bfbe8335030f05b9913d7ae00a87d4395620a \ + --hash=sha256:721c76e84fe669be19c5791da68232ca2e05ba5185575086e384352e2c309597 \ + --hash=sha256:7222ffd5e4de8e57e03ce2cef95a4c43c98fcb72ad86909abdfc2c17d227fc1b \ + --hash=sha256:75d10d37a47afee94919c4fab4c22b9bc2a8bf7d4f46f87363bcf0573f3ff4f5 \ + --hash=sha256:76af085e67e56c8816c3ccf256ebd136def2ed9654525348cfa744b6802b69eb \ + --hash=sha256:770cab594ecf99ae64c236bc9ee3439c3f46be49796e265ce0cc8bc17b10294f \ + --hash=sha256:7a6ab32f7210554a96cd9e33abe3ddd86732beeafc7a28e9955cdf22ffadbab0 \ + --hash=sha256:7c48ed483eb946e6c04ccbe02c6b4d1d48e51944b6db70f697e089c193404941 \ + --hash=sha256:7f56930ab0abd1c45cd15be65cc741c28b1c9a34876ce8c17a2fa107810c0af0 \ + --hash=sha256:8075c35cd58273fee266c58c0c9b670947c19df5fb98e7b66710e04ad4e9ff86 \ + --hash=sha256:8272b73e1c5603666618805fe821edba66892e2870058c94c53147602eab29c7 \ + --hash=sha256:82d8fd25b7f4675d0c47cf95b594d4e7b158aca33b76aa63d07186e13c0e0ab7 \ + --hash=sha256:844da2b5728b5ce0e32d863af26f32b5ce61bc4273a9c720a9f3aa9df73b1455 \ + --hash=sha256:8755483f3c00d6c9a77f490c17e6ab0c8729e39e6390328e42521ef175380ae6 \ + --hash=sha256:915f3849a011c1f593ab99092f3cecfcb4d65d8feb4a64cf1bf2d22074dc0ec4 \ + --hash=sha256:926ca93accd5d36ccdabd803392ddc3e03e6d4cd1cf17deff3b989ab8e9dbcf0 \ + --hash=sha256:982bb1e8b4ffda883b3d0a521e23abcd6fd17418f6d2c4118d257a10199c0ce3 \ + --hash=sha256:98f862da73774290f251b9df8d11161b6cf25b599a66baf087c1ffe340e9bfd1 \ + --hash=sha256:9cbfacf36cb0ec2897ce0ebc5d08ca44213af24265bd56eca54bee7923c48fd6 \ + --hash=sha256:a370b3e078e418187da8c3674eddb9d983ec09445c99a3a263c2011993522981 \ + --hash=sha256:a955b438e62efdf7e0b7b52a64dc5c3396e2634baa62471768a64bc2adb73d5c \ + --hash=sha256:aa6af9e7d59f9c12b33ae4e9450619cf2488e2bbe9b44030905877f0b2324980 \ + --hash=sha256:aa88ca0b1932e93f2d961bf3addbb2db902198dca337d88c89e1559e066e7645 \ + --hash=sha256:aaeeb6a479c7667fbe1099af9617c83aaca22182d6cf8c53966491a0f1b7ffb7 \ + --hash=sha256:aaf27faa992bfee0264dc1f03f4c75e9fcdda66a519db6b957a3f826e285cf12 \ + --hash=sha256:b2680962a4848b3c4f155dc2ee64505a9c57186d0d56b43123b17ca3de18f0fa \ + --hash=sha256:b2d318c11350e10662026ad0eb71bb51c7812fc8590825304ae0bdd4ac283acd \ + --hash=sha256:b33de11b92e9f75a2b545d6e9b6f37e398d86c3e9e9653c4864eb7e89c5773ef \ + --hash=sha256:b3daeac64d5b371dea99714f08ffc2c208522ec6b06fbc7866a450dd446f5c0f \ + --hash=sha256:be1e352acbe3c78727a16a455126d9ff83ea2dfdcbc83148d2982305a04714c2 \ + --hash=sha256:bee093bf902e1d8fc0ac143c88902c3dfc8941f7ea1d6a8dd2bcb786d33db03d \ + --hash=sha256:c72fbbe68c6f32f251bdc08b8611c7b3060612236e960ef848e0a517ddbe76c5 \ + --hash=sha256:c9e36a97bee9b86ef9a1cf7bb96747eb7a15c2f22bdb5b516434b00f2a599f02 \ + --hash=sha256:cddf7bd982eaa998934a91f69d182aec997c6c468898efe6679af88283b498d3 \ + --hash=sha256:cf713fe9a71ef6fd5adf7a79670135081cd4431c2943864757f0fa3a65b1fafd \ + --hash=sha256:d11b54acf878eef558599658b0ffca78138c8c3655cf4f3a4a673c437e67732e \ + --hash=sha256:d41c4d287cfc69060fa91cae9683eacffad989f1a10811995fa309df656ec214 \ + --hash=sha256:d524ba3f1581b35c03cb42beebab4a13e6cdad7b36246bd22541fa585a56cccd \ + --hash=sha256:daac4765328a919a805fa5e2720f3e94767abd632ae410a9062dff5412bae65a \ + --hash=sha256:db4c7bf0e07fc3b7d89ac2a5880a6a8062056801b83ff56d8464b70f65482b6c \ + --hash=sha256:dc7039885fa1baf9be153a0626e337aa7ec8bf96b0128605fb0d77788ddc1681 \ + --hash=sha256:dccab8d5fa1ef9bfba0590ecf4d46df048d18ffe3eec01eeb73a42e0d9e7a8ba \ + --hash=sha256:dedb8adb91d11846ee08bec4c8236c8549ac721c245678282dcb06b221aab59f \ + --hash=sha256:e45ba65510e2647721e35323d6ef54c7974959f6081b58d4ef5d87c60c84919a \ + --hash=sha256:e53efc7c7cee4c1e70661e2e112ca46a575f90ed9ae3fef200f2a25e954f4b28 \ + --hash=sha256:e635b87f01ebc977342e2697d05b56632f5f879a4f15955dfe8cef2448b51691 \ + --hash=sha256:e70e990b2137b29dc5564715de1e12701815dacc1d056308e2b17e9095372a82 \ + --hash=sha256:e8082b26888e2f8b36a042a58307d5b917ef2b1cacab921ad3323ef91901c71a \ + --hash=sha256:e8323a9b031aa0393768b87f04b4164a40037fb2a3c11ac06a03ffecd3618027 \ + --hash=sha256:e92fca20c46e9f5e1bb485887d074918b13543b1c2a1185e69bb8d17ab6236a7 \ + --hash=sha256:eb30abc20df9ab0814b5a2524f23d75dcf83cde762c161917a2b4b7b55b1e518 \ + --hash=sha256:eba9904b0f38a143592d9fc0e19e2df0fa2e41c3c3745554761c5f6447eedabf \ + --hash=sha256:ef8de666d6179b009dce7bcb2ad4c4a779f113f12caf8dc77f0162c29d20490b \ + --hash=sha256:efd387a49825780ff861998cd959767800d54f8308936b21025326de4b5a42b9 \ + --hash=sha256:f0aa37f3c979cf2546b73e8222bbfa3dc07a641585340179d768068e3455e544 \ + --hash=sha256:f4074c5a429281bf056ddd4c5d3b740ebca4d43ffffe2ef4bf4d2d05114299da \ + --hash=sha256:f69a27e45c43520f5487f27627059b64aaf160415589230992cec34c5e18a509 \ + --hash=sha256:fb707f3e15060adf5b7ada797624a6c6e0138e2a26baa089df64c68ee98e040f \ + --hash=sha256:fcbe676a55d7445b22c10967bceaaf0ee69407fbe0ece4d032b6eb8d4565982a \ + --hash=sha256:fdb20a30fe1175ecabed17cbf7812f7b804b8a315a25f24678bcdf120a90077f # via requests cryptography==44.0.1 ; sys_platform == 'linux' \ --hash=sha256:00918d859aa4e57db8299607086f793fa7813ae2ff5a4637e318a25ef82730f7 \ diff --git a/tools/publish/requirements_windows.txt b/tools/publish/requirements_windows.txt index a4d5e3e25d..16f12238b8 100644 --- a/tools/publish/requirements_windows.txt +++ b/tools/publish/requirements_windows.txt @@ -10,99 +10,99 @@ certifi==2025.6.15 \ --hash=sha256:2e0c7ce7cb5d8f8634ca55d2ba7e6ec2689a2fd6537d8dec1296a477a4910057 \ --hash=sha256:d747aa5a8b9bbbb1bb8c22bb13e22bd1f18e9796defa16bab421f7f7a317323b # via requests -charset-normalizer==3.4.1 \ - --hash=sha256:0167ddc8ab6508fe81860a57dd472b2ef4060e8d378f0cc555707126830f2537 \ - --hash=sha256:01732659ba9b5b873fc117534143e4feefecf3b2078b0a6a2e925271bb6f4cfa \ - --hash=sha256:01ad647cdd609225c5350561d084b42ddf732f4eeefe6e678765636791e78b9a \ - --hash=sha256:04432ad9479fa40ec0f387795ddad4437a2b50417c69fa275e212933519ff294 \ - --hash=sha256:0907f11d019260cdc3f94fbdb23ff9125f6b5d1039b76003b5b0ac9d6a6c9d5b \ - --hash=sha256:0924e81d3d5e70f8126529951dac65c1010cdf117bb75eb02dd12339b57749dd \ - --hash=sha256:09b26ae6b1abf0d27570633b2b078a2a20419c99d66fb2823173d73f188ce601 \ - --hash=sha256:09b5e6733cbd160dcc09589227187e242a30a49ca5cefa5a7edd3f9d19ed53fd \ - --hash=sha256:0af291f4fe114be0280cdd29d533696a77b5b49cfde5467176ecab32353395c4 \ - --hash=sha256:0f55e69f030f7163dffe9fd0752b32f070566451afe180f99dbeeb81f511ad8d \ - --hash=sha256:1a2bc9f351a75ef49d664206d51f8e5ede9da246602dc2d2726837620ea034b2 \ - --hash=sha256:22e14b5d70560b8dd51ec22863f370d1e595ac3d024cb8ad7d308b4cd95f8313 \ - --hash=sha256:234ac59ea147c59ee4da87a0c0f098e9c8d169f4dc2a159ef720f1a61bbe27cd \ - --hash=sha256:2369eea1ee4a7610a860d88f268eb39b95cb588acd7235e02fd5a5601773d4fa \ - --hash=sha256:237bdbe6159cff53b4f24f397d43c6336c6b0b42affbe857970cefbb620911c8 \ - --hash=sha256:28bf57629c75e810b6ae989f03c0828d64d6b26a5e205535585f96093e405ed1 \ - --hash=sha256:2967f74ad52c3b98de4c3b32e1a44e32975e008a9cd2a8cc8966d6a5218c5cb2 \ - --hash=sha256:2a75d49014d118e4198bcee5ee0a6f25856b29b12dbf7cd012791f8a6cc5c496 \ - --hash=sha256:2bdfe3ac2e1bbe5b59a1a63721eb3b95fc9b6817ae4a46debbb4e11f6232428d \ - --hash=sha256:2d074908e1aecee37a7635990b2c6d504cd4766c7bc9fc86d63f9c09af3fa11b \ - --hash=sha256:2fb9bd477fdea8684f78791a6de97a953c51831ee2981f8e4f583ff3b9d9687e \ - --hash=sha256:311f30128d7d333eebd7896965bfcfbd0065f1716ec92bd5638d7748eb6f936a \ - --hash=sha256:329ce159e82018d646c7ac45b01a430369d526569ec08516081727a20e9e4af4 \ - --hash=sha256:345b0426edd4e18138d6528aed636de7a9ed169b4aaf9d61a8c19e39d26838ca \ - --hash=sha256:363e2f92b0f0174b2f8238240a1a30142e3db7b957a5dd5689b0e75fb717cc78 \ - --hash=sha256:3a3bd0dcd373514dcec91c411ddb9632c0d7d92aed7093b8c3bbb6d69ca74408 \ - --hash=sha256:3bed14e9c89dcb10e8f3a29f9ccac4955aebe93c71ae803af79265c9ca5644c5 \ - --hash=sha256:44251f18cd68a75b56585dd00dae26183e102cd5e0f9f1466e6df5da2ed64ea3 \ - --hash=sha256:44ecbf16649486d4aebafeaa7ec4c9fed8b88101f4dd612dcaf65d5e815f837f \ - --hash=sha256:4532bff1b8421fd0a320463030c7520f56a79c9024a4e88f01c537316019005a \ - --hash=sha256:49402233c892a461407c512a19435d1ce275543138294f7ef013f0b63d5d3765 \ - --hash=sha256:4c0907b1928a36d5a998d72d64d8eaa7244989f7aaaf947500d3a800c83a3fd6 \ - --hash=sha256:4d86f7aff21ee58f26dcf5ae81a9addbd914115cdebcbb2217e4f0ed8982e146 \ - --hash=sha256:5777ee0881f9499ed0f71cc82cf873d9a0ca8af166dfa0af8ec4e675b7df48e6 \ - --hash=sha256:5df196eb874dae23dcfb968c83d4f8fdccb333330fe1fc278ac5ceeb101003a9 \ - --hash=sha256:619a609aa74ae43d90ed2e89bdd784765de0a25ca761b93e196d938b8fd1dbbd \ - --hash=sha256:6e27f48bcd0957c6d4cb9d6fa6b61d192d0b13d5ef563e5f2ae35feafc0d179c \ - --hash=sha256:6ff8a4a60c227ad87030d76e99cd1698345d4491638dfa6673027c48b3cd395f \ - --hash=sha256:73d94b58ec7fecbc7366247d3b0b10a21681004153238750bb67bd9012414545 \ - --hash=sha256:7461baadb4dc00fd9e0acbe254e3d7d2112e7f92ced2adc96e54ef6501c5f176 \ - --hash=sha256:75832c08354f595c760a804588b9357d34ec00ba1c940c15e31e96d902093770 \ - --hash=sha256:7709f51f5f7c853f0fb938bcd3bc59cdfdc5203635ffd18bf354f6967ea0f824 \ - --hash=sha256:78baa6d91634dfb69ec52a463534bc0df05dbd546209b79a3880a34487f4b84f \ - --hash=sha256:7974a0b5ecd505609e3b19742b60cee7aa2aa2fb3151bc917e6e2646d7667dcf \ - --hash=sha256:7a4f97a081603d2050bfaffdefa5b02a9ec823f8348a572e39032caa8404a487 \ - --hash=sha256:7b1bef6280950ee6c177b326508f86cad7ad4dff12454483b51d8b7d673a2c5d \ - --hash=sha256:7d053096f67cd1241601111b698f5cad775f97ab25d81567d3f59219b5f1adbd \ - --hash=sha256:804a4d582ba6e5b747c625bf1255e6b1507465494a40a2130978bda7b932c90b \ - --hash=sha256:807f52c1f798eef6cf26beb819eeb8819b1622ddfeef9d0977a8502d4db6d534 \ - --hash=sha256:80ed5e856eb7f30115aaf94e4a08114ccc8813e6ed1b5efa74f9f82e8509858f \ - --hash=sha256:8417cb1f36cc0bc7eaba8ccb0e04d55f0ee52df06df3ad55259b9a323555fc8b \ - --hash=sha256:8436c508b408b82d87dc5f62496973a1805cd46727c34440b0d29d8a2f50a6c9 \ - --hash=sha256:89149166622f4db9b4b6a449256291dc87a99ee53151c74cbd82a53c8c2f6ccd \ - --hash=sha256:8bfa33f4f2672964266e940dd22a195989ba31669bd84629f05fab3ef4e2d125 \ - --hash=sha256:8c60ca7339acd497a55b0ea5d506b2a2612afb2826560416f6894e8b5770d4a9 \ - --hash=sha256:91b36a978b5ae0ee86c394f5a54d6ef44db1de0815eb43de826d41d21e4af3de \ - --hash=sha256:955f8851919303c92343d2f66165294848d57e9bba6cf6e3625485a70a038d11 \ - --hash=sha256:97f68b8d6831127e4787ad15e6757232e14e12060bec17091b85eb1486b91d8d \ - --hash=sha256:9b23ca7ef998bc739bf6ffc077c2116917eabcc901f88da1b9856b210ef63f35 \ - --hash=sha256:9f0b8b1c6d84c8034a44893aba5e767bf9c7a211e313a9605d9c617d7083829f \ - --hash=sha256:aabfa34badd18f1da5ec1bc2715cadc8dca465868a4e73a0173466b688f29dda \ - --hash=sha256:ab36c8eb7e454e34e60eb55ca5d241a5d18b2c6244f6827a30e451c42410b5f7 \ - --hash=sha256:b010a7a4fd316c3c484d482922d13044979e78d1861f0e0650423144c616a46a \ - --hash=sha256:b1ac5992a838106edb89654e0aebfc24f5848ae2547d22c2c3f66454daa11971 \ - --hash=sha256:b7b2d86dd06bfc2ade3312a83a5c364c7ec2e3498f8734282c6c3d4b07b346b8 \ - --hash=sha256:b97e690a2118911e39b4042088092771b4ae3fc3aa86518f84b8cf6888dbdb41 \ - --hash=sha256:bc2722592d8998c870fa4e290c2eec2c1569b87fe58618e67d38b4665dfa680d \ - --hash=sha256:c0429126cf75e16c4f0ad00ee0eae4242dc652290f940152ca8c75c3a4b6ee8f \ - --hash=sha256:c30197aa96e8eed02200a83fba2657b4c3acd0f0aa4bdc9f6c1af8e8962e0757 \ - --hash=sha256:c4c3e6da02df6fa1410a7680bd3f63d4f710232d3139089536310d027950696a \ - --hash=sha256:c75cb2a3e389853835e84a2d8fb2b81a10645b503eca9bcb98df6b5a43eb8886 \ - --hash=sha256:c96836c97b1238e9c9e3fe90844c947d5afbf4f4c92762679acfe19927d81d77 \ - --hash=sha256:d7f50a1f8c450f3925cb367d011448c39239bb3eb4117c36a6d354794de4ce76 \ - --hash=sha256:d973f03c0cb71c5ed99037b870f2be986c3c05e63622c017ea9816881d2dd247 \ - --hash=sha256:d98b1668f06378c6dbefec3b92299716b931cd4e6061f3c875a71ced1780ab85 \ - --hash=sha256:d9c3cdf5390dcd29aa8056d13e8e99526cda0305acc038b96b30352aff5ff2bb \ - --hash=sha256:dad3e487649f498dd991eeb901125411559b22e8d7ab25d3aeb1af367df5efd7 \ - --hash=sha256:dccbe65bd2f7f7ec22c4ff99ed56faa1e9f785482b9bbd7c717e26fd723a1d1e \ - --hash=sha256:dd78cfcda14a1ef52584dbb008f7ac81c1328c0f58184bf9a84c49c605002da6 \ - --hash=sha256:e218488cd232553829be0664c2292d3af2eeeb94b32bea483cf79ac6a694e037 \ - --hash=sha256:e358e64305fe12299a08e08978f51fc21fac060dcfcddd95453eabe5b93ed0e1 \ - --hash=sha256:ea0d8d539afa5eb2728aa1932a988a9a7af94f18582ffae4bc10b3fbdad0626e \ - --hash=sha256:eab677309cdb30d047996b36d34caeda1dc91149e4fdca0b1a039b3f79d9a807 \ - --hash=sha256:eb8178fe3dba6450a3e024e95ac49ed3400e506fd4e9e5c32d30adda88cbd407 \ - --hash=sha256:ecddf25bee22fe4fe3737a399d0d177d72bc22be6913acfab364b40bce1ba83c \ - --hash=sha256:eea6ee1db730b3483adf394ea72f808b6e18cf3cb6454b4d86e04fa8c4327a12 \ - --hash=sha256:f08ff5e948271dc7e18a35641d2f11a4cd8dfd5634f55228b691e62b37125eb3 \ - --hash=sha256:f30bf9fd9be89ecb2360c7d94a711f00c09b976258846efe40db3d05828e8089 \ - --hash=sha256:fa88b843d6e211393a37219e6a1c1df99d35e8fd90446f1118f4216e307e48cd \ - --hash=sha256:fc54db6c8593ef7d4b2a331b58653356cf04f67c960f584edb7c3d8c97e8f39e \ - --hash=sha256:fd4ec41f914fa74ad1b8304bbc634b3de73d2a0889bd32076342a573e0779e00 \ - --hash=sha256:ffc9202a29ab3920fa812879e95a9e78b2465fd10be7fcbd042899695d75e616 +charset-normalizer==3.4.2 \ + --hash=sha256:005fa3432484527f9732ebd315da8da8001593e2cf46a3d817669f062c3d9ed4 \ + --hash=sha256:046595208aae0120559a67693ecc65dd75d46f7bf687f159127046628178dc45 \ + --hash=sha256:0c29de6a1a95f24b9a1aa7aefd27d2487263f00dfd55a77719b530788f75cff7 \ + --hash=sha256:0c8c57f84ccfc871a48a47321cfa49ae1df56cd1d965a09abe84066f6853b9c0 \ + --hash=sha256:0f5d9ed7f254402c9e7d35d2f5972c9bbea9040e99cd2861bd77dc68263277c7 \ + --hash=sha256:18dd2e350387c87dabe711b86f83c9c78af772c748904d372ade190b5c7c9d4d \ + --hash=sha256:1b1bde144d98e446b056ef98e59c256e9294f6b74d7af6846bf5ffdafd687a7d \ + --hash=sha256:1c95a1e2902a8b722868587c0e1184ad5c55631de5afc0eb96bc4b0d738092c0 \ + --hash=sha256:1cad5f45b3146325bb38d6855642f6fd609c3f7cad4dbaf75549bf3b904d3184 \ + --hash=sha256:21b2899062867b0e1fde9b724f8aecb1af14f2778d69aacd1a5a1853a597a5db \ + --hash=sha256:24498ba8ed6c2e0b56d4acbf83f2d989720a93b41d712ebd4f4979660db4417b \ + --hash=sha256:25a23ea5c7edc53e0f29bae2c44fcb5a1aa10591aae107f2a2b2583a9c5cbc64 \ + --hash=sha256:289200a18fa698949d2b39c671c2cc7a24d44096784e76614899a7ccf2574b7b \ + --hash=sha256:28a1005facc94196e1fb3e82a3d442a9d9110b8434fc1ded7a24a2983c9888d8 \ + --hash=sha256:32fc0341d72e0f73f80acb0a2c94216bd704f4f0bce10aedea38f30502b271ff \ + --hash=sha256:36b31da18b8890a76ec181c3cf44326bf2c48e36d393ca1b72b3f484113ea344 \ + --hash=sha256:3c21d4fca343c805a52c0c78edc01e3477f6dd1ad7c47653241cf2a206d4fc58 \ + --hash=sha256:3fddb7e2c84ac87ac3a947cb4e66d143ca5863ef48e4a5ecb83bd48619e4634e \ + --hash=sha256:43e0933a0eff183ee85833f341ec567c0980dae57c464d8a508e1b2ceb336471 \ + --hash=sha256:4a476b06fbcf359ad25d34a057b7219281286ae2477cc5ff5e3f70a246971148 \ + --hash=sha256:4e594135de17ab3866138f496755f302b72157d115086d100c3f19370839dd3a \ + --hash=sha256:50bf98d5e563b83cc29471fa114366e6806bc06bc7a25fd59641e41445327836 \ + --hash=sha256:5a9979887252a82fefd3d3ed2a8e3b937a7a809f65dcb1e068b090e165bbe99e \ + --hash=sha256:5baececa9ecba31eff645232d59845c07aa030f0c81ee70184a90d35099a0e63 \ + --hash=sha256:5bf4545e3b962767e5c06fe1738f951f77d27967cb2caa64c28be7c4563e162c \ + --hash=sha256:6333b3aa5a12c26b2a4d4e7335a28f1475e0e5e17d69d55141ee3cab736f66d1 \ + --hash=sha256:65c981bdbd3f57670af8b59777cbfae75364b483fa8a9f420f08094531d54a01 \ + --hash=sha256:68a328e5f55ec37c57f19ebb1fdc56a248db2e3e9ad769919a58672958e8f366 \ + --hash=sha256:6a0289e4589e8bdfef02a80478f1dfcb14f0ab696b5a00e1f4b8a14a307a3c58 \ + --hash=sha256:6b66f92b17849b85cad91259efc341dce9c1af48e2173bf38a85c6329f1033e5 \ + --hash=sha256:6c9379d65defcab82d07b2a9dfbfc2e95bc8fe0ebb1b176a3190230a3ef0e07c \ + --hash=sha256:6fc1f5b51fa4cecaa18f2bd7a003f3dd039dd615cd69a2afd6d3b19aed6775f2 \ + --hash=sha256:70f7172939fdf8790425ba31915bfbe8335030f05b9913d7ae00a87d4395620a \ + --hash=sha256:721c76e84fe669be19c5791da68232ca2e05ba5185575086e384352e2c309597 \ + --hash=sha256:7222ffd5e4de8e57e03ce2cef95a4c43c98fcb72ad86909abdfc2c17d227fc1b \ + --hash=sha256:75d10d37a47afee94919c4fab4c22b9bc2a8bf7d4f46f87363bcf0573f3ff4f5 \ + --hash=sha256:76af085e67e56c8816c3ccf256ebd136def2ed9654525348cfa744b6802b69eb \ + --hash=sha256:770cab594ecf99ae64c236bc9ee3439c3f46be49796e265ce0cc8bc17b10294f \ + --hash=sha256:7a6ab32f7210554a96cd9e33abe3ddd86732beeafc7a28e9955cdf22ffadbab0 \ + --hash=sha256:7c48ed483eb946e6c04ccbe02c6b4d1d48e51944b6db70f697e089c193404941 \ + --hash=sha256:7f56930ab0abd1c45cd15be65cc741c28b1c9a34876ce8c17a2fa107810c0af0 \ + --hash=sha256:8075c35cd58273fee266c58c0c9b670947c19df5fb98e7b66710e04ad4e9ff86 \ + --hash=sha256:8272b73e1c5603666618805fe821edba66892e2870058c94c53147602eab29c7 \ + --hash=sha256:82d8fd25b7f4675d0c47cf95b594d4e7b158aca33b76aa63d07186e13c0e0ab7 \ + --hash=sha256:844da2b5728b5ce0e32d863af26f32b5ce61bc4273a9c720a9f3aa9df73b1455 \ + --hash=sha256:8755483f3c00d6c9a77f490c17e6ab0c8729e39e6390328e42521ef175380ae6 \ + --hash=sha256:915f3849a011c1f593ab99092f3cecfcb4d65d8feb4a64cf1bf2d22074dc0ec4 \ + --hash=sha256:926ca93accd5d36ccdabd803392ddc3e03e6d4cd1cf17deff3b989ab8e9dbcf0 \ + --hash=sha256:982bb1e8b4ffda883b3d0a521e23abcd6fd17418f6d2c4118d257a10199c0ce3 \ + --hash=sha256:98f862da73774290f251b9df8d11161b6cf25b599a66baf087c1ffe340e9bfd1 \ + --hash=sha256:9cbfacf36cb0ec2897ce0ebc5d08ca44213af24265bd56eca54bee7923c48fd6 \ + --hash=sha256:a370b3e078e418187da8c3674eddb9d983ec09445c99a3a263c2011993522981 \ + --hash=sha256:a955b438e62efdf7e0b7b52a64dc5c3396e2634baa62471768a64bc2adb73d5c \ + --hash=sha256:aa6af9e7d59f9c12b33ae4e9450619cf2488e2bbe9b44030905877f0b2324980 \ + --hash=sha256:aa88ca0b1932e93f2d961bf3addbb2db902198dca337d88c89e1559e066e7645 \ + --hash=sha256:aaeeb6a479c7667fbe1099af9617c83aaca22182d6cf8c53966491a0f1b7ffb7 \ + --hash=sha256:aaf27faa992bfee0264dc1f03f4c75e9fcdda66a519db6b957a3f826e285cf12 \ + --hash=sha256:b2680962a4848b3c4f155dc2ee64505a9c57186d0d56b43123b17ca3de18f0fa \ + --hash=sha256:b2d318c11350e10662026ad0eb71bb51c7812fc8590825304ae0bdd4ac283acd \ + --hash=sha256:b33de11b92e9f75a2b545d6e9b6f37e398d86c3e9e9653c4864eb7e89c5773ef \ + --hash=sha256:b3daeac64d5b371dea99714f08ffc2c208522ec6b06fbc7866a450dd446f5c0f \ + --hash=sha256:be1e352acbe3c78727a16a455126d9ff83ea2dfdcbc83148d2982305a04714c2 \ + --hash=sha256:bee093bf902e1d8fc0ac143c88902c3dfc8941f7ea1d6a8dd2bcb786d33db03d \ + --hash=sha256:c72fbbe68c6f32f251bdc08b8611c7b3060612236e960ef848e0a517ddbe76c5 \ + --hash=sha256:c9e36a97bee9b86ef9a1cf7bb96747eb7a15c2f22bdb5b516434b00f2a599f02 \ + --hash=sha256:cddf7bd982eaa998934a91f69d182aec997c6c468898efe6679af88283b498d3 \ + --hash=sha256:cf713fe9a71ef6fd5adf7a79670135081cd4431c2943864757f0fa3a65b1fafd \ + --hash=sha256:d11b54acf878eef558599658b0ffca78138c8c3655cf4f3a4a673c437e67732e \ + --hash=sha256:d41c4d287cfc69060fa91cae9683eacffad989f1a10811995fa309df656ec214 \ + --hash=sha256:d524ba3f1581b35c03cb42beebab4a13e6cdad7b36246bd22541fa585a56cccd \ + --hash=sha256:daac4765328a919a805fa5e2720f3e94767abd632ae410a9062dff5412bae65a \ + --hash=sha256:db4c7bf0e07fc3b7d89ac2a5880a6a8062056801b83ff56d8464b70f65482b6c \ + --hash=sha256:dc7039885fa1baf9be153a0626e337aa7ec8bf96b0128605fb0d77788ddc1681 \ + --hash=sha256:dccab8d5fa1ef9bfba0590ecf4d46df048d18ffe3eec01eeb73a42e0d9e7a8ba \ + --hash=sha256:dedb8adb91d11846ee08bec4c8236c8549ac721c245678282dcb06b221aab59f \ + --hash=sha256:e45ba65510e2647721e35323d6ef54c7974959f6081b58d4ef5d87c60c84919a \ + --hash=sha256:e53efc7c7cee4c1e70661e2e112ca46a575f90ed9ae3fef200f2a25e954f4b28 \ + --hash=sha256:e635b87f01ebc977342e2697d05b56632f5f879a4f15955dfe8cef2448b51691 \ + --hash=sha256:e70e990b2137b29dc5564715de1e12701815dacc1d056308e2b17e9095372a82 \ + --hash=sha256:e8082b26888e2f8b36a042a58307d5b917ef2b1cacab921ad3323ef91901c71a \ + --hash=sha256:e8323a9b031aa0393768b87f04b4164a40037fb2a3c11ac06a03ffecd3618027 \ + --hash=sha256:e92fca20c46e9f5e1bb485887d074918b13543b1c2a1185e69bb8d17ab6236a7 \ + --hash=sha256:eb30abc20df9ab0814b5a2524f23d75dcf83cde762c161917a2b4b7b55b1e518 \ + --hash=sha256:eba9904b0f38a143592d9fc0e19e2df0fa2e41c3c3745554761c5f6447eedabf \ + --hash=sha256:ef8de666d6179b009dce7bcb2ad4c4a779f113f12caf8dc77f0162c29d20490b \ + --hash=sha256:efd387a49825780ff861998cd959767800d54f8308936b21025326de4b5a42b9 \ + --hash=sha256:f0aa37f3c979cf2546b73e8222bbfa3dc07a641585340179d768068e3455e544 \ + --hash=sha256:f4074c5a429281bf056ddd4c5d3b740ebca4d43ffffe2ef4bf4d2d05114299da \ + --hash=sha256:f69a27e45c43520f5487f27627059b64aaf160415589230992cec34c5e18a509 \ + --hash=sha256:fb707f3e15060adf5b7ada797624a6c6e0138e2a26baa089df64c68ee98e040f \ + --hash=sha256:fcbe676a55d7445b22c10967bceaaf0ee69407fbe0ece4d032b6eb8d4565982a \ + --hash=sha256:fdb20a30fe1175ecabed17cbf7812f7b804b8a315a25f24678bcdf120a90077f # via requests docutils==0.21.2 \ --hash=sha256:3a6b18732edf182daa3cd12775bbb338cf5691468f91eeeb109deff6ebfa986f \ From 5c68ff90030dd5cb8fb4873992a955ac57de7a83 Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Tue, 8 Jul 2025 16:01:59 -0700 Subject: [PATCH 326/922] build(deps): bump pygments from 2.18.0 to 2.19.2 in /tools/publish (#3021) [![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=pygments&package-manager=pip&previous-version=2.18.0&new-version=2.19.2)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) ---
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot show ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> --- tools/publish/requirements_darwin.txt | 6 +++--- tools/publish/requirements_linux.txt | 6 +++--- tools/publish/requirements_universal.txt | 6 +++--- tools/publish/requirements_windows.txt | 6 +++--- 4 files changed, 12 insertions(+), 12 deletions(-) diff --git a/tools/publish/requirements_darwin.txt b/tools/publish/requirements_darwin.txt index afc2bae956..dab86f3adc 100644 --- a/tools/publish/requirements_darwin.txt +++ b/tools/publish/requirements_darwin.txt @@ -170,9 +170,9 @@ pkginfo==1.10.0 \ --hash=sha256:5df73835398d10db79f8eecd5cd86b1f6d29317589ea70796994d49399af6297 \ --hash=sha256:889a6da2ed7ffc58ab5b900d888ddce90bce912f2d2de1dc1c26f4cb9fe65097 # via twine -pygments==2.18.0 \ - --hash=sha256:786ff802f32e91311bff3889f6e9a86e81505fe99f2735bb6d60ae0c5004f199 \ - --hash=sha256:b8e6aca0523f3ab76fee51799c488e38782ac06eafcf95e7ba832985c8e7b13a +pygments==2.19.2 \ + --hash=sha256:636cb2477cec7f8952536970bc533bc43743542f70392ae026374600add5b887 \ + --hash=sha256:86540386c03d588bb81d44bc3928634ff26449851e99741617ecb9037ee5ec0b # via # readme-renderer # rich diff --git a/tools/publish/requirements_linux.txt b/tools/publish/requirements_linux.txt index 6e43dab96c..c9d25eab58 100644 --- a/tools/publish/requirements_linux.txt +++ b/tools/publish/requirements_linux.txt @@ -282,9 +282,9 @@ pycparser==2.22 \ --hash=sha256:491c8be9c040f5390f5bf44a5b07752bd07f56edf992381b05c701439eec10f6 \ --hash=sha256:c3702b6d3dd8c7abc1afa565d7e63d53a1d0bd86cdc24edd75470f4de499cfcc # via cffi -pygments==2.18.0 \ - --hash=sha256:786ff802f32e91311bff3889f6e9a86e81505fe99f2735bb6d60ae0c5004f199 \ - --hash=sha256:b8e6aca0523f3ab76fee51799c488e38782ac06eafcf95e7ba832985c8e7b13a +pygments==2.19.2 \ + --hash=sha256:636cb2477cec7f8952536970bc533bc43743542f70392ae026374600add5b887 \ + --hash=sha256:86540386c03d588bb81d44bc3928634ff26449851e99741617ecb9037ee5ec0b # via # readme-renderer # rich diff --git a/tools/publish/requirements_universal.txt b/tools/publish/requirements_universal.txt index 92addf96ac..a642e9280d 100644 --- a/tools/publish/requirements_universal.txt +++ b/tools/publish/requirements_universal.txt @@ -282,9 +282,9 @@ pycparser==2.22 ; platform_python_implementation != 'PyPy' and sys_platform == ' --hash=sha256:491c8be9c040f5390f5bf44a5b07752bd07f56edf992381b05c701439eec10f6 \ --hash=sha256:c3702b6d3dd8c7abc1afa565d7e63d53a1d0bd86cdc24edd75470f4de499cfcc # via cffi -pygments==2.18.0 \ - --hash=sha256:786ff802f32e91311bff3889f6e9a86e81505fe99f2735bb6d60ae0c5004f199 \ - --hash=sha256:b8e6aca0523f3ab76fee51799c488e38782ac06eafcf95e7ba832985c8e7b13a +pygments==2.19.2 \ + --hash=sha256:636cb2477cec7f8952536970bc533bc43743542f70392ae026374600add5b887 \ + --hash=sha256:86540386c03d588bb81d44bc3928634ff26449851e99741617ecb9037ee5ec0b # via # readme-renderer # rich diff --git a/tools/publish/requirements_windows.txt b/tools/publish/requirements_windows.txt index 16f12238b8..d3944056c0 100644 --- a/tools/publish/requirements_windows.txt +++ b/tools/publish/requirements_windows.txt @@ -170,9 +170,9 @@ pkginfo==1.10.0 \ --hash=sha256:5df73835398d10db79f8eecd5cd86b1f6d29317589ea70796994d49399af6297 \ --hash=sha256:889a6da2ed7ffc58ab5b900d888ddce90bce912f2d2de1dc1c26f4cb9fe65097 # via twine -pygments==2.18.0 \ - --hash=sha256:786ff802f32e91311bff3889f6e9a86e81505fe99f2735bb6d60ae0c5004f199 \ - --hash=sha256:b8e6aca0523f3ab76fee51799c488e38782ac06eafcf95e7ba832985c8e7b13a +pygments==2.19.2 \ + --hash=sha256:636cb2477cec7f8952536970bc533bc43743542f70392ae026374600add5b887 \ + --hash=sha256:86540386c03d588bb81d44bc3928634ff26449851e99741617ecb9037ee5ec0b # via # readme-renderer # rich From c30980a9f4cd17ee090746afb8af6f7bd5c8397e Mon Sep 17 00:00:00 2001 From: Ignas Anikevicius <240938+aignas@users.noreply.github.com> Date: Mon, 14 Jul 2025 23:58:42 +0900 Subject: [PATCH 327/922] ci: use Ubuntu 22.04 (#3083) Update the CI configuration and start testing on Ubuntu 22.04. Fixes #3084 --- .bazelci/presubmit.yml | 64 ++++++++++++++-------------- CHANGELOG.md | 1 + WORKSPACE | 2 +- python/private/internal_dev_deps.bzl | 2 +- 4 files changed, 35 insertions(+), 34 deletions(-) diff --git a/.bazelci/presubmit.yml b/.bazelci/presubmit.yml index 07ffa4eaac..6457363ccd 100644 --- a/.bazelci/presubmit.yml +++ b/.bazelci/presubmit.yml @@ -91,20 +91,20 @@ tasks: <<: *common_workspace_flags_min_bazel <<: *minimum_supported_version name: "Gazelle: workspace, minimum supported Bazel version" - platform: ubuntu2004 + platform: ubuntu2204 build_targets: ["//..."] test_targets: ["//..."] working_directory: gazelle gazelle_extension_workspace: <<: *common_workspace_flags name: "Gazelle: workspace" - platform: ubuntu2004 + platform: ubuntu2204 build_targets: ["//..."] test_targets: ["//..."] working_directory: gazelle gazelle_extension: name: "Gazelle: default settings" - platform: ubuntu2004 + platform: ubuntu2204 build_targets: ["//..."] test_targets: ["//..."] working_directory: gazelle @@ -114,28 +114,28 @@ tasks: <<: *reusable_config <<: *common_workspace_flags_min_bazel name: "Default: Ubuntu, workspace, minimum Bazel" - platform: ubuntu2004 + platform: ubuntu2204 ubuntu_min_bzlmod: <<: *minimum_supported_version <<: *reusable_config name: "Default: Ubuntu, bzlmod, minimum Bazel" - platform: ubuntu2004 + platform: ubuntu2204 bazel: 7.x ubuntu: <<: *reusable_config name: "Default: Ubuntu" - platform: ubuntu2004 + platform: ubuntu2204 ubuntu_upcoming: <<: *reusable_config name: "Default: Ubuntu, upcoming Bazel" - platform: ubuntu2004 + platform: ubuntu2204 bazel: last_rc ubuntu_workspace: <<: *reusable_config <<: *common_workspace_flags name: "Default: Ubuntu, workspace" - platform: ubuntu2004 + platform: ubuntu2204 mac_workspace: <<: *reusable_config <<: *common_workspace_flags @@ -185,7 +185,7 @@ tasks: <<: *minimum_supported_version <<: *reusable_config name: "RBE: Ubuntu, minimum Bazel" - platform: rbe_ubuntu2004 + platform: rbe_ubuntu2204 build_flags: # BazelCI sets --action_env=BAZEL_DO_NOT_DETECT_CPP_TOOLCHAIN=1, # which prevents cc toolchain autodetection from working correctly @@ -203,7 +203,7 @@ tasks: rbe: <<: *reusable_config name: "RBE: Ubuntu" - platform: rbe_ubuntu2004 + platform: rbe_ubuntu2204 # TODO @aignas 2024-12-11: get the RBE working in CI for bazel 8.0 # See https://github.com/bazelbuild/rules_python/issues/2499 bazel: 7.x @@ -217,13 +217,13 @@ tasks: <<: *common_workspace_flags_min_bazel name: "examples/build_file_generation: Ubuntu, workspace, minimum Bazel" working_directory: examples/build_file_generation - platform: ubuntu2004 + platform: ubuntu2204 integration_test_build_file_generation_ubuntu_workspace: <<: *reusable_build_test_all <<: *common_workspace_flags name: "examples/build_file_generation: Ubuntu, workspace" working_directory: examples/build_file_generation - platform: ubuntu2004 + platform: ubuntu2204 integration_test_build_file_generation_debian_workspace: <<: *reusable_build_test_all <<: *common_workspace_flags @@ -249,21 +249,21 @@ tasks: coverage_targets: ["//:test"] name: "examples/bzlmod: Ubuntu, minimum Bazel" working_directory: examples/bzlmod - platform: ubuntu2004 + platform: ubuntu2204 bazel: 7.x integration_test_bzlmod_ubuntu: <<: *reusable_build_test_all <<: *coverage_targets_example_bzlmod name: "examples/bzlmod: Ubuntu" working_directory: examples/bzlmod - platform: ubuntu2004 + platform: ubuntu2204 bazel: 7.x integration_test_bzlmod_ubuntu_upcoming: <<: *reusable_build_test_all <<: *coverage_targets_example_bzlmod name: "examples/bzlmod: Ubuntu, upcoming Bazel" working_directory: examples/bzlmod - platform: ubuntu2004 + platform: ubuntu2204 bazel: last_rc integration_test_bzlmod_debian: <<: *reusable_build_test_all @@ -276,7 +276,7 @@ tasks: <<: *reusable_build_test_all name: "examples/bzlmod: bazel vendor" working_directory: examples/bzlmod - platform: ubuntu2004 + platform: ubuntu2204 shell_commands: - "bazel vendor --vendor_dir=./vendor //..." - "bazel build --vendor_dir=./vendor //..." @@ -316,19 +316,19 @@ tasks: <<: *coverage_targets_example_bzlmod_build_file_generation name: "examples/bzlmod_build_file_generation: Ubuntu, minimum Bazel" working_directory: examples/bzlmod_build_file_generation - platform: ubuntu2004 + platform: ubuntu2204 bazel: 7.x integration_test_bzlmod_generation_build_files_ubuntu: <<: *reusable_build_test_all <<: *coverage_targets_example_bzlmod_build_file_generation name: "examples/bzlmod_build_file_generation: Ubuntu" working_directory: examples/bzlmod_build_file_generation - platform: ubuntu2004 + platform: ubuntu2204 integration_test_bzlmod_generation_build_files_ubuntu_run: <<: *reusable_build_test_all name: "examples/bzlmod_build_file_generation: Ubuntu, Gazelle and pip" working_directory: examples/bzlmod_build_file_generation - platform: ubuntu2004 + platform: ubuntu2204 shell_commands: - "bazel run //:gazelle_python_manifest.update" - "bazel run //:gazelle -- update" @@ -357,7 +357,7 @@ tasks: <<: *coverage_targets_example_multi_python name: "examples/multi_python_versions: Ubuntu, workspace" working_directory: examples/multi_python_versions - platform: ubuntu2004 + platform: ubuntu2204 integration_test_multi_python_versions_debian_workspace: <<: *reusable_build_test_all <<: *common_workspace_flags @@ -386,19 +386,19 @@ tasks: <<: *reusable_build_test_all name: "examples/pip_parse: Ubuntu, workspace, minimum supported Bazel version" working_directory: examples/pip_parse - platform: ubuntu2004 + platform: ubuntu2204 integration_test_pip_parse_ubuntu_min_bzlmod: <<: *minimum_supported_version <<: *reusable_build_test_all name: "examples/pip_parse: Ubuntu, bzlmod, minimum supported Bazel version" working_directory: examples/pip_parse - platform: ubuntu2004 + platform: ubuntu2204 bazel: 7.x integration_test_pip_parse_ubuntu: <<: *reusable_build_test_all name: "examples/pip_parse: Ubuntu" working_directory: examples/pip_parse - platform: ubuntu2004 + platform: ubuntu2204 integration_test_pip_parse_debian: <<: *reusable_build_test_all name: "examples/pip_parse: Debian" @@ -421,13 +421,13 @@ tasks: <<: *reusable_build_test_all name: "examples/pip_parse_vendored: Ubuntu, workspace, minimum Bazel" working_directory: examples/pip_parse_vendored - platform: ubuntu2004 + platform: ubuntu2204 integration_test_pip_parse_vendored_ubuntu: <<: *reusable_build_test_all <<: *common_workspace_flags name: "examples/pip_parse_vendored: Ubuntu" working_directory: examples/pip_parse_vendored - platform: ubuntu2004 + platform: ubuntu2204 integration_test_pip_parse_vendored_debian: <<: *reusable_build_test_all <<: *common_workspace_flags @@ -450,7 +450,7 @@ tasks: <<: *common_workspace_flags name: "examples/py_proto_library: Ubuntu, workspace" working_directory: examples/py_proto_library - platform: ubuntu2004 + platform: ubuntu2204 integration_test_py_proto_library_debian_workspace: <<: *reusable_build_test_all <<: *common_workspace_flags @@ -475,7 +475,7 @@ tasks: <<: *common_workspace_flags name: "examples/pip_repository_annotations: Ubuntu, workspace" working_directory: examples/pip_repository_annotations - platform: ubuntu2004 + platform: ubuntu2204 integration_test_pip_repository_annotations_debian_workspace: <<: *reusable_build_test_all <<: *common_workspace_flags @@ -498,7 +498,7 @@ tasks: integration_test_bazelinbazel_ubuntu: <<: *common_bazelinbazel_config name: "tests/integration bazel-in-bazel: Ubuntu" - platform: ubuntu2004 + platform: ubuntu2204 integration_test_bazelinbazel_debian: <<: *common_bazelinbazel_config name: "tests/integration bazel-in-bazel: Debian" @@ -508,7 +508,7 @@ tasks: <<: *reusable_build_test_all name: "compile_pip_requirements: Ubuntu" working_directory: tests/integration/compile_pip_requirements - platform: ubuntu2004 + platform: ubuntu2204 shell_commands: # Make a change to the locked requirements and then assert that //:requirements.update does the # right thing. @@ -596,7 +596,7 @@ tasks: <<: *common_workspace_flags_min_bazel name: "compile_pip_requirements_test_from_external_repo: Ubuntu, workspace, minimum Bazel" working_directory: tests/integration/compile_pip_requirements_test_from_external_repo - platform: ubuntu2004 + platform: ubuntu2204 shell_commands: # Assert that @compile_pip_requirements//:requirements_test does the right thing. - "bazel test @compile_pip_requirements//..." @@ -604,7 +604,7 @@ tasks: <<: *minimum_supported_version name: "compile_pip_requirements_test_from_external_repo: Ubuntu, bzlmod, minimum Bazel" working_directory: tests/integration/compile_pip_requirements_test_from_external_repo - platform: ubuntu2004 + platform: ubuntu2204 bazel: 7.x shell_commands: # Assert that @compile_pip_requirements//:requirements_test does the right thing. @@ -612,7 +612,7 @@ tasks: integration_compile_pip_requirements_test_from_external_repo_ubuntu: name: "compile_pip_requirements_test_from_external_repo: Ubuntu" working_directory: tests/integration/compile_pip_requirements_test_from_external_repo - platform: ubuntu2004 + platform: ubuntu2204 shell_commands: # Assert that @compile_pip_requirements//:requirements_test does the right thing. - "bazel test @compile_pip_requirements//..." diff --git a/CHANGELOG.md b/CHANGELOG.md index 7f02c8bbb4..ad68669df2 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -69,6 +69,7 @@ END_UNRELEASED_TEMPLATE * (toolchain) Python 3.13 now references 3.13.5 * (gazelle) Switched back to smacker/go-tree-sitter, fixing [#2630](https://github.com/bazel-contrib/rules_python/issues/2630) +* (ci) We are now testing on Ubuntu 22.04 for RBE and non-RBE configurations. {#v0-0-0-fixed} ### Fixed diff --git a/WORKSPACE b/WORKSPACE index dddc5105ed..5c2136666d 100644 --- a/WORKSPACE +++ b/WORKSPACE @@ -95,7 +95,7 @@ load("@bazelci_rules//:rbe_repo.bzl", "rbe_preconfig") # otherwise refer to RBE docs. rbe_preconfig( name = "buildkite_config", - toolchain = "ubuntu1804-bazel-java11", + toolchain = "ubuntu2204", ) local_repository( diff --git a/python/private/internal_dev_deps.bzl b/python/private/internal_dev_deps.bzl index ca34dc698a..d621a5d941 100644 --- a/python/private/internal_dev_deps.bzl +++ b/python/private/internal_dev_deps.bzl @@ -26,7 +26,7 @@ def _internal_dev_deps_impl(mctx): # otherwise refer to RBE docs. rbe_preconfig( name = "buildkite_config", - toolchain = "ubuntu1804-bazel-java11", + toolchain = "ubuntu2204", ) runtime_env_repo(name = "rules_python_runtime_env_tc_info") From 6f27511a35baa7d0de302e504cb161cf0af2f2bf Mon Sep 17 00:00:00 2001 From: yushan26 <107004874+yushan26@users.noreply.github.com> Date: Mon, 14 Jul 2025 16:51:42 -0700 Subject: [PATCH 328/922] fix(gazelle) Update gazelle to properly process multi-line python imports (#3077) A python import may be imported as: ``` from foo.bar.application.\ pipeline.model import ( Baz ) ``` However, gazelle fails to resolve this import with the error: `line 30: "foo.bar.application.pipeline.model\\\n pipeline.mode.Baz" is an invalid dependency:` Clean up the imports such that whitespace and \n are removed from the import path. --------- Co-authored-by: yushan Co-authored-by: Douglas Thor --- CHANGELOG.md | 1 + gazelle/python/file_parser.go | 14 +++ gazelle/python/file_parser_test.go | 92 +++++++++++++++++++ .../import_nested_var/__init__.py | 6 +- 4 files changed, 112 insertions(+), 1 deletion(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index ad68669df2..834a2c1a39 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -87,6 +87,7 @@ END_UNRELEASED_TEMPLATE ({gh-issue}`3043`). * (pypi) The pipstar `defaults` configuration now supports any custom platform name. +* Multi-line python imports (e.g. with escaped newlines) are now correctly processed by Gazelle. {#v0-0-0-added} ### Added diff --git a/gazelle/python/file_parser.go b/gazelle/python/file_parser.go index 31fce02712..e129337e11 100644 --- a/gazelle/python/file_parser.go +++ b/gazelle/python/file_parser.go @@ -144,6 +144,16 @@ func parseImportStatement(node *sitter.Node, code []byte) (Module, bool) { return Module{}, false } +// cleanImportString removes backslashes and all whitespace from the string. +func cleanImportString(s string) string { + s = strings.ReplaceAll(s, "\r\n", "") + s = strings.ReplaceAll(s, "\\", "") + s = strings.ReplaceAll(s, " ", "") + s = strings.ReplaceAll(s, "\n", "") + s = strings.ReplaceAll(s, "\t", "") + return s +} + // parseImportStatements parses a node for import statements, returning true if the node is // an import statement. It updates FileParser.output.Modules with the `module` that the // import represents. @@ -154,6 +164,8 @@ func (p *FileParser) parseImportStatements(node *sitter.Node) bool { if !ok { continue } + m.From = cleanImportString(m.From) + m.Name = cleanImportString(m.Name) m.Filepath = p.relFilepath m.TypeCheckingOnly = p.inTypeCheckingBlock if strings.HasPrefix(m.Name, ".") { @@ -163,6 +175,7 @@ func (p *FileParser) parseImportStatements(node *sitter.Node) bool { } } else if node.Type() == sitterNodeTypeImportFromStatement { from := node.Child(1).Content(p.code) + from = cleanImportString(from) // If the import is from the current package, we don't need to add it to the modules i.e. from . import Class1. // If the import is from a different relative package i.e. from .package1 import foo, we need to add it to the modules. if from == "." { @@ -175,6 +188,7 @@ func (p *FileParser) parseImportStatements(node *sitter.Node) bool { } m.Filepath = p.relFilepath m.From = from + m.Name = cleanImportString(m.Name) m.Name = fmt.Sprintf("%s.%s", from, m.Name) m.TypeCheckingOnly = p.inTypeCheckingBlock p.output.Modules = append(p.output.Modules, m) diff --git a/gazelle/python/file_parser_test.go b/gazelle/python/file_parser_test.go index f4db1a316b..0a6fd1b4ab 100644 --- a/gazelle/python/file_parser_test.go +++ b/gazelle/python/file_parser_test.go @@ -291,3 +291,95 @@ def example_function(): } } } + +func TestParseImportStatements_MultilineWithBackslashAndWhitespace(t *testing.T) { + t.Parallel() + t.Run("multiline from import", func(t *testing.T) { + p := NewFileParser() + code := []byte(`from foo.bar.\ + baz import ( + Something, + AnotherThing +) + +from foo\ + .test import ( + Foo, + Bar +) +`) + p.SetCodeAndFile(code, "", "test.py") + output, err := p.Parse(context.Background()) + assert.NoError(t, err) + // Updated expected to match parser output + expected := []Module{ + { + Name: "foo.bar.baz.Something", + LineNumber: 3, + Filepath: "test.py", + From: "foo.bar.baz", + }, + { + Name: "foo.bar.baz.AnotherThing", + LineNumber: 4, + Filepath: "test.py", + From: "foo.bar.baz", + }, + { + Name: "foo.test.Foo", + LineNumber: 9, + Filepath: "test.py", + From: "foo.test", + }, + { + Name: "foo.test.Bar", + LineNumber: 10, + Filepath: "test.py", + From: "foo.test", + }, + } + assert.ElementsMatch(t, expected, output.Modules) + }) + t.Run("multiline import", func(t *testing.T) { + p := NewFileParser() + code := []byte(`import foo.bar.\ + baz +`) + p.SetCodeAndFile(code, "", "test.py") + output, err := p.Parse(context.Background()) + assert.NoError(t, err) + // Updated expected to match parser output + expected := []Module{ + { + Name: "foo.bar.baz", + LineNumber: 1, + Filepath: "test.py", + From: "", + }, + } + assert.ElementsMatch(t, expected, output.Modules) + }) + t.Run("windows line endings", func(t *testing.T) { + p := NewFileParser() + code := []byte("from foo.bar.\r\n baz import (\r\n Something,\r\n AnotherThing\r\n)\r\n") + p.SetCodeAndFile(code, "", "test.py") + output, err := p.Parse(context.Background()) + assert.NoError(t, err) + // Updated expected to match parser output + expected := []Module{ + { + Name: "foo.bar.baz.Something", + LineNumber: 3, + Filepath: "test.py", + From: "foo.bar.baz", + }, + { + Name: "foo.bar.baz.AnotherThing", + LineNumber: 4, + Filepath: "test.py", + From: "foo.bar.baz", + }, + } + assert.ElementsMatch(t, expected, output.Modules) + }) +} diff --git a/gazelle/python/testdata/from_imports/import_nested_var/__init__.py b/gazelle/python/testdata/from_imports/import_nested_var/__init__.py index d0f51c443c..20eda530e5 100644 --- a/gazelle/python/testdata/from_imports/import_nested_var/__init__.py +++ b/gazelle/python/testdata/from_imports/import_nested_var/__init__.py @@ -13,4 +13,8 @@ # limitations under the License. # baz is a variable in foo/bar/baz.py -from foo.bar.baz import baz +from foo\ + .bar.\ + baz import ( + baz + ) From dd6550f18477105f362e44cb4868d9007e340d83 Mon Sep 17 00:00:00 2001 From: Charles OuGuo Date: Mon, 14 Jul 2025 19:57:52 -0400 Subject: [PATCH 329/922] feat(gazelle): Gazelle plugin generates py_proto_library (#3057) Fixes https://github.com/bazel-contrib/rules_python/issues/2994. Please go over this with a fine-toothed comb! This is my first contribution to `rules_python` / the gazelle plugin, and while I've worked in Gazelle before, I'm pretty unfamiliar with the Python plugin's architecture. This adds support in the Gazelle plugin for generating `py_proto_library` rules automatically, if there are any `proto_library` rules detected in a given package. We do this via a new Gazelle directive, `python_generate_proto`, which defaults to `true`, and controls whether these rules are generated. See the tests in `testdata/directive_python_generate_proto` for examples. By default, we source the `py_proto_library` rule from the `@protobuf` repository. I think this the intended long-term home of the rule? Users are expected to use `gazelle:map_kind` to change this if need be. I haven't done anything here to support resolution of imports of `py_proto_library`. I think this is worth landing first, to save folks from having to maintain these by hand. But this should lay the foundation for resolving that in https://github.com/bazel-contrib/rules_python/issues/1703. --------- Co-authored-by: Douglas Thor --- CHANGELOG.md | 2 + examples/bzlmod/py_proto_library/BUILD.bazel | 4 +- .../example.com/another_proto/BUILD.bazel | 2 +- .../example.com/proto/BUILD.bazel | 2 +- examples/py_proto_library/BUILD.bazel | 4 +- .../example.com/another_proto/BUILD.bazel | 2 +- .../example.com/proto/BUILD.bazel | 2 +- gazelle/README.md | 37 ++++++++++++ gazelle/python/BUILD.bazel | 6 +- gazelle/python/configure.go | 7 +++ gazelle/python/generate.go | 52 ++++++++++++++++ gazelle/python/kinds.go | 60 +++++++++++++------ .../directive_python_generate_proto/README.md | 9 +++ .../directive_python_generate_proto/WORKSPACE | 1 + .../directive_python_generate_proto/test.yaml | 3 + .../test1_default_with_proto/BUILD.in | 9 +++ .../test1_default_with_proto/BUILD.out | 9 +++ .../test1_default_with_proto/foo.proto | 7 +++ .../test2_default_without_proto/BUILD.in | 1 + .../test2_default_without_proto/BUILD.out | 1 + .../test3_disabled_with_proto/BUILD.in | 9 +++ .../test3_disabled_with_proto/BUILD.out | 9 +++ .../test3_disabled_with_proto/foo.proto | 7 +++ .../test4_disabled_without_proto/BUILD.in | 1 + .../test4_disabled_without_proto/BUILD.out | 1 + .../test5_enabled_with_proto/BUILD.in | 9 +++ .../test5_enabled_with_proto/BUILD.out | 16 +++++ .../test5_enabled_with_proto/foo.proto | 7 +++ .../test6_enabled_without_proto/BUILD.in | 1 + .../test6_enabled_without_proto/BUILD.out | 1 + .../test7_removes_when_unnecessary/BUILD.in | 16 +++++ .../test7_removes_when_unnecessary/BUILD.out | 1 + .../BUILD.in | 16 +++++ .../BUILD.out | 16 +++++ .../foo.proto | 7 +++ .../BUILD.in | 9 +++ .../BUILD.out | 16 +++++ .../MODULE.bazel | 1 + .../README.md | 6 ++ .../WORKSPACE | 0 .../foo.proto | 7 +++ .../test.yaml | 3 + .../BUILD.in | 9 +++ .../BUILD.out | 16 +++++ .../MODULE.bazel | 1 + .../README.md | 7 +++ .../WORKSPACE | 0 .../foo.proto | 7 +++ .../test.yaml | 3 + gazelle/pythonconfig/pythonconfig.go | 16 +++++ 50 files changed, 412 insertions(+), 26 deletions(-) create mode 100644 gazelle/python/testdata/directive_python_generate_proto/README.md create mode 100644 gazelle/python/testdata/directive_python_generate_proto/WORKSPACE create mode 100644 gazelle/python/testdata/directive_python_generate_proto/test.yaml create mode 100644 gazelle/python/testdata/directive_python_generate_proto/test1_default_with_proto/BUILD.in create mode 100644 gazelle/python/testdata/directive_python_generate_proto/test1_default_with_proto/BUILD.out create mode 100644 gazelle/python/testdata/directive_python_generate_proto/test1_default_with_proto/foo.proto create mode 100644 gazelle/python/testdata/directive_python_generate_proto/test2_default_without_proto/BUILD.in create mode 100644 gazelle/python/testdata/directive_python_generate_proto/test2_default_without_proto/BUILD.out create mode 100644 gazelle/python/testdata/directive_python_generate_proto/test3_disabled_with_proto/BUILD.in create mode 100644 gazelle/python/testdata/directive_python_generate_proto/test3_disabled_with_proto/BUILD.out create mode 100644 gazelle/python/testdata/directive_python_generate_proto/test3_disabled_with_proto/foo.proto create mode 100644 gazelle/python/testdata/directive_python_generate_proto/test4_disabled_without_proto/BUILD.in create mode 100644 gazelle/python/testdata/directive_python_generate_proto/test4_disabled_without_proto/BUILD.out create mode 100644 gazelle/python/testdata/directive_python_generate_proto/test5_enabled_with_proto/BUILD.in create mode 100644 gazelle/python/testdata/directive_python_generate_proto/test5_enabled_with_proto/BUILD.out create mode 100644 gazelle/python/testdata/directive_python_generate_proto/test5_enabled_with_proto/foo.proto create mode 100644 gazelle/python/testdata/directive_python_generate_proto/test6_enabled_without_proto/BUILD.in create mode 100644 gazelle/python/testdata/directive_python_generate_proto/test6_enabled_without_proto/BUILD.out create mode 100644 gazelle/python/testdata/directive_python_generate_proto/test7_removes_when_unnecessary/BUILD.in create mode 100644 gazelle/python/testdata/directive_python_generate_proto/test7_removes_when_unnecessary/BUILD.out create mode 100644 gazelle/python/testdata/directive_python_generate_proto/test8_disabled_ignores_py_proto_library/BUILD.in create mode 100644 gazelle/python/testdata/directive_python_generate_proto/test8_disabled_ignores_py_proto_library/BUILD.out create mode 100644 gazelle/python/testdata/directive_python_generate_proto/test8_disabled_ignores_py_proto_library/foo.proto create mode 100644 gazelle/python/testdata/directive_python_generate_proto_bzlmod_protobuf/BUILD.in create mode 100644 gazelle/python/testdata/directive_python_generate_proto_bzlmod_protobuf/BUILD.out create mode 100644 gazelle/python/testdata/directive_python_generate_proto_bzlmod_protobuf/MODULE.bazel create mode 100644 gazelle/python/testdata/directive_python_generate_proto_bzlmod_protobuf/README.md create mode 100644 gazelle/python/testdata/directive_python_generate_proto_bzlmod_protobuf/WORKSPACE create mode 100644 gazelle/python/testdata/directive_python_generate_proto_bzlmod_protobuf/foo.proto create mode 100644 gazelle/python/testdata/directive_python_generate_proto_bzlmod_protobuf/test.yaml create mode 100644 gazelle/python/testdata/directive_python_generate_proto_bzlmod_protobuf_renamed/BUILD.in create mode 100644 gazelle/python/testdata/directive_python_generate_proto_bzlmod_protobuf_renamed/BUILD.out create mode 100644 gazelle/python/testdata/directive_python_generate_proto_bzlmod_protobuf_renamed/MODULE.bazel create mode 100644 gazelle/python/testdata/directive_python_generate_proto_bzlmod_protobuf_renamed/README.md create mode 100644 gazelle/python/testdata/directive_python_generate_proto_bzlmod_protobuf_renamed/WORKSPACE create mode 100644 gazelle/python/testdata/directive_python_generate_proto_bzlmod_protobuf_renamed/foo.proto create mode 100644 gazelle/python/testdata/directive_python_generate_proto_bzlmod_protobuf_renamed/test.yaml diff --git a/CHANGELOG.md b/CHANGELOG.md index 834a2c1a39..e74f14b1db 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -106,6 +106,8 @@ END_UNRELEASED_TEMPLATE * 3.12.11 * 3.13.5 * 3.14.0b3 +* (gazelle) New directive `gazelle:python_generate_proto`; when `true`, + Gazelle generates `py_proto_library` rules for `proto_library`. `false` by default. {#v0-0-0-removed} ### Removed diff --git a/examples/bzlmod/py_proto_library/BUILD.bazel b/examples/bzlmod/py_proto_library/BUILD.bazel index 969cb8e9f7..daea410365 100644 --- a/examples/bzlmod/py_proto_library/BUILD.bazel +++ b/examples/bzlmod/py_proto_library/BUILD.bazel @@ -6,7 +6,7 @@ py_test( srcs = ["test.py"], main = "test.py", deps = [ - "//py_proto_library/example.com/proto:pricetag_proto_py_pb2", + "//py_proto_library/example.com/proto:pricetag_py_pb2", ], ) @@ -14,7 +14,7 @@ py_test( name = "message_test", srcs = ["message_test.py"], deps = [ - "//py_proto_library/example.com/another_proto:message_proto_py_pb2", + "//py_proto_library/example.com/another_proto:message_py_pb2", ], ) diff --git a/examples/bzlmod/py_proto_library/example.com/another_proto/BUILD.bazel b/examples/bzlmod/py_proto_library/example.com/another_proto/BUILD.bazel index 785d90d01e..29f08c21ca 100644 --- a/examples/bzlmod/py_proto_library/example.com/another_proto/BUILD.bazel +++ b/examples/bzlmod/py_proto_library/example.com/another_proto/BUILD.bazel @@ -2,7 +2,7 @@ load("@com_google_protobuf//bazel:proto_library.bzl", "proto_library") load("@rules_python//python:proto.bzl", "py_proto_library") py_proto_library( - name = "message_proto_py_pb2", + name = "message_py_pb2", visibility = ["//visibility:public"], deps = [":message_proto"], ) diff --git a/examples/bzlmod/py_proto_library/example.com/proto/BUILD.bazel b/examples/bzlmod/py_proto_library/example.com/proto/BUILD.bazel index 72af672219..1f8e8f2818 100644 --- a/examples/bzlmod/py_proto_library/example.com/proto/BUILD.bazel +++ b/examples/bzlmod/py_proto_library/example.com/proto/BUILD.bazel @@ -2,7 +2,7 @@ load("@com_google_protobuf//bazel:proto_library.bzl", "proto_library") load("@rules_python//python:proto.bzl", "py_proto_library") py_proto_library( - name = "pricetag_proto_py_pb2", + name = "pricetag_py_pb2", visibility = ["//visibility:public"], deps = [":pricetag_proto"], ) diff --git a/examples/py_proto_library/BUILD.bazel b/examples/py_proto_library/BUILD.bazel index d782fb296d..b57c528511 100644 --- a/examples/py_proto_library/BUILD.bazel +++ b/examples/py_proto_library/BUILD.bazel @@ -5,7 +5,7 @@ py_test( srcs = ["test.py"], main = "test.py", deps = [ - "//example.com/proto:pricetag_proto_py_pb2", + "//example.com/proto:pricetag_py_pb2", ], ) @@ -13,6 +13,6 @@ py_test( name = "message_test", srcs = ["message_test.py"], deps = [ - "//example.com/another_proto:message_proto_py_pb2", + "//example.com/another_proto:message_py_pb2", ], ) diff --git a/examples/py_proto_library/example.com/another_proto/BUILD.bazel b/examples/py_proto_library/example.com/another_proto/BUILD.bazel index 3d841554e9..55e83a209a 100644 --- a/examples/py_proto_library/example.com/another_proto/BUILD.bazel +++ b/examples/py_proto_library/example.com/another_proto/BUILD.bazel @@ -2,7 +2,7 @@ load("@com_google_protobuf//bazel:proto_library.bzl", "proto_library") load("@rules_python//python:proto.bzl", "py_proto_library") py_proto_library( - name = "message_proto_py_pb2", + name = "message_py_pb2", visibility = ["//visibility:public"], deps = [":message_proto"], ) diff --git a/examples/py_proto_library/example.com/proto/BUILD.bazel b/examples/py_proto_library/example.com/proto/BUILD.bazel index f84454f531..fdf2e6fe32 100644 --- a/examples/py_proto_library/example.com/proto/BUILD.bazel +++ b/examples/py_proto_library/example.com/proto/BUILD.bazel @@ -2,7 +2,7 @@ load("@com_google_protobuf//bazel:proto_library.bzl", "proto_library") load("@rules_python//python:proto.bzl", "py_proto_library") py_proto_library( - name = "pricetag_proto_py_pb2", + name = "pricetag_py_pb2", visibility = ["//visibility:public"], deps = [":pricetag_proto"], ) diff --git a/gazelle/README.md b/gazelle/README.md index 3dc8e12a0a..35a1e4f701 100644 --- a/gazelle/README.md +++ b/gazelle/README.md @@ -224,6 +224,8 @@ Python-specific directives are as follows: | Controls whether Gazelle resolves dependencies for import statements that use paths relative to the current package. Can be "true" or "false".| | `# gazelle:python_generate_pyi_deps` | `false` | | Controls whether to generate a separate `pyi_deps` attribute for type-checking dependencies or merge them into the regular `deps` attribute. When `false` (default), type-checking dependencies are merged into `deps` for backward compatibility. When `true`, generates separate `pyi_deps`. Imports in blocks with the format `if typing.TYPE_CHECKING:`/`if TYPE_CHECKING:` and type-only stub packages (eg. boto3-stubs) are recognized as type-checking dependencies. | +| [`# gazelle:python_generate_proto`](#directive-python_generate_proto) | `false` | +| Controls whether to generate a `py_proto_library` for each `proto_library` in the package. By default we load this rule from the `@protobuf` repository; use `gazelle:map_kind` if you need to load this from somewhere else. | #### Directive: `python_root`: @@ -484,6 +486,41 @@ def py_test(name, main=None, **kwargs): ) ``` +#### Directive: `python_generate_proto`: + +When `# gazelle:python_generate_proto true`, Gazelle will generate one +`py_proto_library` for each `proto_library`, generating Python clients for +protobuf in each package. By default this is turned off. Gazelle will also +generate a load statement for the `py_proto_library` - attempting to detect +the configured name for the `@protobuf` / `@com_google_protobuf` repo in your +`MODULE.bazel`, and otherwise falling back to `@com_google_protobuf` for +compatibility with `WORKSPACE`. + +For example, in a package with `# gazelle:python_generate_proto true` and a +`foo.proto`, if you have both the proto extension and the Python extension +loaded into Gazelle, you'll get something like: + +```starlark +load("@protobuf//bazel:py_proto_library.bzl", "py_proto_library") +load("@rules_proto//proto:defs.bzl", "proto_library") + +# gazelle:python_generate_proto true + +proto_library( + name = "foo_proto", + srcs = ["foo.proto"], + visibility = ["//:__subpackages__"], +) + +py_proto_library( + name = "foo_py_pb2", + visibility = ["//:__subpackages__"], + deps = [":foo_proto"], +) +``` + +When `false`, Gazelle will ignore any `py_proto_library`, including previously-generated or hand-created rules. + ### Annotations *Annotations* refer to comments found _within Python files_ that configure how diff --git a/gazelle/python/BUILD.bazel b/gazelle/python/BUILD.bazel index 8e8216ddd4..1a7c54f4b2 100644 --- a/gazelle/python/BUILD.bazel +++ b/gazelle/python/BUILD.bazel @@ -34,6 +34,7 @@ go_library( "@bazel_gazelle//config:go_default_library", "@bazel_gazelle//label:go_default_library", "@bazel_gazelle//language:go_default_library", + "@bazel_gazelle//language/proto:go_default_library", "@bazel_gazelle//repo:go_default_library", "@bazel_gazelle//resolve:go_default_library", "@bazel_gazelle//rule:go_default_library", @@ -91,7 +92,10 @@ gazelle_test( gazelle_binary( name = "gazelle_binary", - languages = [":python"], + languages = [ + "@bazel_gazelle//language/proto", + ":python", + ], visibility = ["//visibility:public"], ) diff --git a/gazelle/python/configure.go b/gazelle/python/configure.go index db80fc1a22..7131be283d 100644 --- a/gazelle/python/configure.go +++ b/gazelle/python/configure.go @@ -70,6 +70,7 @@ func (py *Configurer) KnownDirectives() []string { pythonconfig.LabelNormalization, pythonconfig.GeneratePyiDeps, pythonconfig.ExperimentalAllowRelativeImports, + pythonconfig.GenerateProto, } } @@ -237,6 +238,12 @@ func (py *Configurer) Configure(c *config.Config, rel string, f *rule.File) { log.Fatal(err) } config.SetGeneratePyiDeps(v) + case pythonconfig.GenerateProto: + v, err := strconv.ParseBool(strings.TrimSpace(d.Value)) + if err != nil { + log.Fatal(err) + } + config.SetGenerateProto(v) } } diff --git a/gazelle/python/generate.go b/gazelle/python/generate.go index c1edec4731..343743559f 100644 --- a/gazelle/python/generate.go +++ b/gazelle/python/generate.go @@ -226,6 +226,10 @@ func (py *Python) GenerateRules(args language.GenerateArgs) language.GenerateRes var result language.GenerateResult result.Gen = make([]*rule.Rule, 0) + if cfg.GenerateProto() { + generateProtoLibraries(args, pythonProjectRoot, visibility, &result) + } + collisionErrors := singlylinkedlist.New() appendPyLibrary := func(srcs *treeset.Set, pyLibraryTargetName string) { @@ -551,3 +555,51 @@ func ensureNoCollision(file *rule.File, targetName, kind string) error { } return nil } + +func generateProtoLibraries(args language.GenerateArgs, pythonProjectRoot string, visibility []string, res *language.GenerateResult) { + // First, enumerate all the proto_library in this package. + var protoRuleNames []string + for _, r := range args.OtherGen { + if r.Kind() != "proto_library" { + continue + } + protoRuleNames = append(protoRuleNames, r.Name()) + } + sort.Strings(protoRuleNames) + + // Next, enumerate all the pre-existing py_proto_library in this package, so we can delete unnecessary rules later. + pyProtoRules := map[string]bool{} + if args.File != nil { + for _, r := range args.File.Rules { + if r.Kind() == "py_proto_library" { + pyProtoRules[r.Name()] = false + } + } + } + + emptySiblings := treeset.Set{} + // Generate a py_proto_library for each proto_library. + for _, protoRuleName := range protoRuleNames { + pyProtoLibraryName := strings.TrimSuffix(protoRuleName, "_proto") + "_py_pb2" + pyProtoLibrary := newTargetBuilder(pyProtoLibraryKind, pyProtoLibraryName, pythonProjectRoot, args.Rel, &emptySiblings). + addVisibility(visibility). + addResolvedDependency(":" + protoRuleName). + generateImportsAttribute().build() + + res.Gen = append(res.Gen, pyProtoLibrary) + res.Imports = append(res.Imports, pyProtoLibrary.PrivateAttr(config.GazelleImportsKey)) + pyProtoRules[pyProtoLibrary.Name()] = true + + } + + // Finally, emit an empty rule for each pre-existing py_proto_library that we didn't already generate. + for ruleName, generated := range pyProtoRules { + if generated { + continue + } + + emptyRule := newTargetBuilder(pyProtoLibraryKind, ruleName, pythonProjectRoot, args.Rel, &emptySiblings).build() + res.Empty = append(res.Empty, emptyRule) + } + +} diff --git a/gazelle/python/kinds.go b/gazelle/python/kinds.go index ff3f6ce829..a4ce572aaa 100644 --- a/gazelle/python/kinds.go +++ b/gazelle/python/kinds.go @@ -15,13 +15,16 @@ package python import ( + "fmt" + "github.com/bazelbuild/bazel-gazelle/rule" ) const ( - pyBinaryKind = "py_binary" - pyLibraryKind = "py_library" - pyTestKind = "py_test" + pyBinaryKind = "py_binary" + pyLibraryKind = "py_library" + pyProtoLibraryKind = "py_proto_library" + pyTestKind = "py_test" ) // Kinds returns a map that maps rule names (kinds) and information on how to @@ -32,7 +35,7 @@ func (*Python) Kinds() map[string]rule.KindInfo { var pyKinds = map[string]rule.KindInfo{ pyBinaryKind: { - MatchAny: false, + MatchAny: false, MatchAttrs: []string{"srcs"}, NonEmptyAttrs: map[string]bool{ "deps": true, @@ -45,7 +48,7 @@ var pyKinds = map[string]rule.KindInfo{ "srcs": true, }, ResolveAttrs: map[string]bool{ - "deps": true, + "deps": true, "pyi_deps": true, }, }, @@ -62,10 +65,16 @@ var pyKinds = map[string]rule.KindInfo{ "srcs": true, }, ResolveAttrs: map[string]bool{ - "deps": true, + "deps": true, "pyi_deps": true, }, }, + pyProtoLibraryKind: { + NonEmptyAttrs: map[string]bool{ + "deps": true, + }, + ResolveAttrs: map[string]bool{"deps": true}, + }, pyTestKind: { MatchAny: false, NonEmptyAttrs: map[string]bool{ @@ -79,26 +88,43 @@ var pyKinds = map[string]rule.KindInfo{ "srcs": true, }, ResolveAttrs: map[string]bool{ - "deps": true, + "deps": true, "pyi_deps": true, }, }, } +func (py *Python) Loads() []rule.LoadInfo { + panic("ApparentLoads should be called instead") +} + // Loads returns .bzl files and symbols they define. Every rule generated by // GenerateRules, now or in the past, should be loadable from one of these // files. -func (py *Python) Loads() []rule.LoadInfo { - return pyLoads +func (py *Python) ApparentLoads(moduleToApparentName func(string) string) []rule.LoadInfo { + return apparentLoads(moduleToApparentName) } -var pyLoads = []rule.LoadInfo{ - { - Name: "@rules_python//python:defs.bzl", - Symbols: []string{ - pyBinaryKind, - pyLibraryKind, - pyTestKind, +func apparentLoads(moduleToApparentName func(string) string) []rule.LoadInfo { + protobuf := moduleToApparentName("protobuf") + if protobuf == "" { + protobuf = "com_google_protobuf" + } + + return []rule.LoadInfo{ + { + Name: "@rules_python//python:defs.bzl", + Symbols: []string{ + pyBinaryKind, + pyLibraryKind, + pyTestKind, + }, }, - }, + { + Name: fmt.Sprintf("@%s//bazel:py_proto_library.bzl", protobuf), + Symbols: []string{ + pyProtoLibraryKind, + }, + }, + } } diff --git a/gazelle/python/testdata/directive_python_generate_proto/README.md b/gazelle/python/testdata/directive_python_generate_proto/README.md new file mode 100644 index 0000000000..54261f47ca --- /dev/null +++ b/gazelle/python/testdata/directive_python_generate_proto/README.md @@ -0,0 +1,9 @@ +# Directive: `python_generate_proto` + +This test case asserts that the `# gazelle:python_generate_proto` directive +correctly: + +1. Uses the default value when `python_generate_proto` is not set. +2. Generates (or not) `py_proto_library` when `python_generate_proto` is set, based on whether a proto is present. + +[gh-2994]: https://github.com/bazel-contrib/rules_python/issues/2994 diff --git a/gazelle/python/testdata/directive_python_generate_proto/WORKSPACE b/gazelle/python/testdata/directive_python_generate_proto/WORKSPACE new file mode 100644 index 0000000000..faff6af87a --- /dev/null +++ b/gazelle/python/testdata/directive_python_generate_proto/WORKSPACE @@ -0,0 +1 @@ +# This is a Bazel workspace for the Gazelle test data. diff --git a/gazelle/python/testdata/directive_python_generate_proto/test.yaml b/gazelle/python/testdata/directive_python_generate_proto/test.yaml new file mode 100644 index 0000000000..36dd656b39 --- /dev/null +++ b/gazelle/python/testdata/directive_python_generate_proto/test.yaml @@ -0,0 +1,3 @@ +--- +expect: + exit_code: 0 diff --git a/gazelle/python/testdata/directive_python_generate_proto/test1_default_with_proto/BUILD.in b/gazelle/python/testdata/directive_python_generate_proto/test1_default_with_proto/BUILD.in new file mode 100644 index 0000000000..9784aafc17 --- /dev/null +++ b/gazelle/python/testdata/directive_python_generate_proto/test1_default_with_proto/BUILD.in @@ -0,0 +1,9 @@ +load("@rules_proto//proto:defs.bzl", "proto_library") + +# python_generate_proto is not set, so py_proto_library is not generated. + +proto_library( + name = "foo_proto", + srcs = ["foo.proto"], + visibility = ["//:__subpackages__"], +) diff --git a/gazelle/python/testdata/directive_python_generate_proto/test1_default_with_proto/BUILD.out b/gazelle/python/testdata/directive_python_generate_proto/test1_default_with_proto/BUILD.out new file mode 100644 index 0000000000..9784aafc17 --- /dev/null +++ b/gazelle/python/testdata/directive_python_generate_proto/test1_default_with_proto/BUILD.out @@ -0,0 +1,9 @@ +load("@rules_proto//proto:defs.bzl", "proto_library") + +# python_generate_proto is not set, so py_proto_library is not generated. + +proto_library( + name = "foo_proto", + srcs = ["foo.proto"], + visibility = ["//:__subpackages__"], +) diff --git a/gazelle/python/testdata/directive_python_generate_proto/test1_default_with_proto/foo.proto b/gazelle/python/testdata/directive_python_generate_proto/test1_default_with_proto/foo.proto new file mode 100644 index 0000000000..fe2af27aa6 --- /dev/null +++ b/gazelle/python/testdata/directive_python_generate_proto/test1_default_with_proto/foo.proto @@ -0,0 +1,7 @@ +syntax = "proto3"; + +package foo; + +message Foo { + string bar = 1; +} diff --git a/gazelle/python/testdata/directive_python_generate_proto/test2_default_without_proto/BUILD.in b/gazelle/python/testdata/directive_python_generate_proto/test2_default_without_proto/BUILD.in new file mode 100644 index 0000000000..0a869d0fd5 --- /dev/null +++ b/gazelle/python/testdata/directive_python_generate_proto/test2_default_without_proto/BUILD.in @@ -0,0 +1 @@ +# python_generate_proto is not set, so py_proto_library is not generated. diff --git a/gazelle/python/testdata/directive_python_generate_proto/test2_default_without_proto/BUILD.out b/gazelle/python/testdata/directive_python_generate_proto/test2_default_without_proto/BUILD.out new file mode 100644 index 0000000000..0a869d0fd5 --- /dev/null +++ b/gazelle/python/testdata/directive_python_generate_proto/test2_default_without_proto/BUILD.out @@ -0,0 +1 @@ +# python_generate_proto is not set, so py_proto_library is not generated. diff --git a/gazelle/python/testdata/directive_python_generate_proto/test3_disabled_with_proto/BUILD.in b/gazelle/python/testdata/directive_python_generate_proto/test3_disabled_with_proto/BUILD.in new file mode 100644 index 0000000000..62fd4be661 --- /dev/null +++ b/gazelle/python/testdata/directive_python_generate_proto/test3_disabled_with_proto/BUILD.in @@ -0,0 +1,9 @@ +load("@rules_proto//proto:defs.bzl", "proto_library") + +# gazelle:python_generate_proto false + +proto_library( + name = "foo_proto", + srcs = ["foo.proto"], + visibility = ["//:__subpackages__"], +) diff --git a/gazelle/python/testdata/directive_python_generate_proto/test3_disabled_with_proto/BUILD.out b/gazelle/python/testdata/directive_python_generate_proto/test3_disabled_with_proto/BUILD.out new file mode 100644 index 0000000000..62fd4be661 --- /dev/null +++ b/gazelle/python/testdata/directive_python_generate_proto/test3_disabled_with_proto/BUILD.out @@ -0,0 +1,9 @@ +load("@rules_proto//proto:defs.bzl", "proto_library") + +# gazelle:python_generate_proto false + +proto_library( + name = "foo_proto", + srcs = ["foo.proto"], + visibility = ["//:__subpackages__"], +) diff --git a/gazelle/python/testdata/directive_python_generate_proto/test3_disabled_with_proto/foo.proto b/gazelle/python/testdata/directive_python_generate_proto/test3_disabled_with_proto/foo.proto new file mode 100644 index 0000000000..022e29ae69 --- /dev/null +++ b/gazelle/python/testdata/directive_python_generate_proto/test3_disabled_with_proto/foo.proto @@ -0,0 +1,7 @@ +syntax = "proto3"; + +package foo.bar; + +message Foo { + string bar = 1; +} diff --git a/gazelle/python/testdata/directive_python_generate_proto/test4_disabled_without_proto/BUILD.in b/gazelle/python/testdata/directive_python_generate_proto/test4_disabled_without_proto/BUILD.in new file mode 100644 index 0000000000..b283b5fb51 --- /dev/null +++ b/gazelle/python/testdata/directive_python_generate_proto/test4_disabled_without_proto/BUILD.in @@ -0,0 +1 @@ +# gazelle:python_generate_proto false diff --git a/gazelle/python/testdata/directive_python_generate_proto/test4_disabled_without_proto/BUILD.out b/gazelle/python/testdata/directive_python_generate_proto/test4_disabled_without_proto/BUILD.out new file mode 100644 index 0000000000..b283b5fb51 --- /dev/null +++ b/gazelle/python/testdata/directive_python_generate_proto/test4_disabled_without_proto/BUILD.out @@ -0,0 +1 @@ +# gazelle:python_generate_proto false diff --git a/gazelle/python/testdata/directive_python_generate_proto/test5_enabled_with_proto/BUILD.in b/gazelle/python/testdata/directive_python_generate_proto/test5_enabled_with_proto/BUILD.in new file mode 100644 index 0000000000..4713404b19 --- /dev/null +++ b/gazelle/python/testdata/directive_python_generate_proto/test5_enabled_with_proto/BUILD.in @@ -0,0 +1,9 @@ +load("@rules_proto//proto:defs.bzl", "proto_library") + +# gazelle:python_generate_proto true + +proto_library( + name = "foo_proto", + srcs = ["foo.proto"], + visibility = ["//:__subpackages__"], +) diff --git a/gazelle/python/testdata/directive_python_generate_proto/test5_enabled_with_proto/BUILD.out b/gazelle/python/testdata/directive_python_generate_proto/test5_enabled_with_proto/BUILD.out new file mode 100644 index 0000000000..686252f27c --- /dev/null +++ b/gazelle/python/testdata/directive_python_generate_proto/test5_enabled_with_proto/BUILD.out @@ -0,0 +1,16 @@ +load("@com_google_protobuf//bazel:py_proto_library.bzl", "py_proto_library") +load("@rules_proto//proto:defs.bzl", "proto_library") + +# gazelle:python_generate_proto true + +proto_library( + name = "foo_proto", + srcs = ["foo.proto"], + visibility = ["//:__subpackages__"], +) + +py_proto_library( + name = "foo_py_pb2", + visibility = ["//:__subpackages__"], + deps = [":foo_proto"], +) diff --git a/gazelle/python/testdata/directive_python_generate_proto/test5_enabled_with_proto/foo.proto b/gazelle/python/testdata/directive_python_generate_proto/test5_enabled_with_proto/foo.proto new file mode 100644 index 0000000000..fe2af27aa6 --- /dev/null +++ b/gazelle/python/testdata/directive_python_generate_proto/test5_enabled_with_proto/foo.proto @@ -0,0 +1,7 @@ +syntax = "proto3"; + +package foo; + +message Foo { + string bar = 1; +} diff --git a/gazelle/python/testdata/directive_python_generate_proto/test6_enabled_without_proto/BUILD.in b/gazelle/python/testdata/directive_python_generate_proto/test6_enabled_without_proto/BUILD.in new file mode 100644 index 0000000000..ce3eec6001 --- /dev/null +++ b/gazelle/python/testdata/directive_python_generate_proto/test6_enabled_without_proto/BUILD.in @@ -0,0 +1 @@ +# gazelle:python_generate_proto true diff --git a/gazelle/python/testdata/directive_python_generate_proto/test6_enabled_without_proto/BUILD.out b/gazelle/python/testdata/directive_python_generate_proto/test6_enabled_without_proto/BUILD.out new file mode 100644 index 0000000000..ce3eec6001 --- /dev/null +++ b/gazelle/python/testdata/directive_python_generate_proto/test6_enabled_without_proto/BUILD.out @@ -0,0 +1 @@ +# gazelle:python_generate_proto true diff --git a/gazelle/python/testdata/directive_python_generate_proto/test7_removes_when_unnecessary/BUILD.in b/gazelle/python/testdata/directive_python_generate_proto/test7_removes_when_unnecessary/BUILD.in new file mode 100644 index 0000000000..686252f27c --- /dev/null +++ b/gazelle/python/testdata/directive_python_generate_proto/test7_removes_when_unnecessary/BUILD.in @@ -0,0 +1,16 @@ +load("@com_google_protobuf//bazel:py_proto_library.bzl", "py_proto_library") +load("@rules_proto//proto:defs.bzl", "proto_library") + +# gazelle:python_generate_proto true + +proto_library( + name = "foo_proto", + srcs = ["foo.proto"], + visibility = ["//:__subpackages__"], +) + +py_proto_library( + name = "foo_py_pb2", + visibility = ["//:__subpackages__"], + deps = [":foo_proto"], +) diff --git a/gazelle/python/testdata/directive_python_generate_proto/test7_removes_when_unnecessary/BUILD.out b/gazelle/python/testdata/directive_python_generate_proto/test7_removes_when_unnecessary/BUILD.out new file mode 100644 index 0000000000..ce3eec6001 --- /dev/null +++ b/gazelle/python/testdata/directive_python_generate_proto/test7_removes_when_unnecessary/BUILD.out @@ -0,0 +1 @@ +# gazelle:python_generate_proto true diff --git a/gazelle/python/testdata/directive_python_generate_proto/test8_disabled_ignores_py_proto_library/BUILD.in b/gazelle/python/testdata/directive_python_generate_proto/test8_disabled_ignores_py_proto_library/BUILD.in new file mode 100644 index 0000000000..f14ed4fc2d --- /dev/null +++ b/gazelle/python/testdata/directive_python_generate_proto/test8_disabled_ignores_py_proto_library/BUILD.in @@ -0,0 +1,16 @@ +load("@com_google_protobuf//bazel:py_proto_library.bzl", "py_proto_library") +load("@rules_proto//proto:defs.bzl", "proto_library") + +# gazelle:python_generate_proto false + +proto_library( + name = "foo_proto", + srcs = ["foo.proto"], + visibility = ["//:__subpackages__"], +) + +py_proto_library( + name = "foo_py_pb2", + visibility = ["//:__subpackages__"], + deps = [":foo_proto"], +) diff --git a/gazelle/python/testdata/directive_python_generate_proto/test8_disabled_ignores_py_proto_library/BUILD.out b/gazelle/python/testdata/directive_python_generate_proto/test8_disabled_ignores_py_proto_library/BUILD.out new file mode 100644 index 0000000000..f14ed4fc2d --- /dev/null +++ b/gazelle/python/testdata/directive_python_generate_proto/test8_disabled_ignores_py_proto_library/BUILD.out @@ -0,0 +1,16 @@ +load("@com_google_protobuf//bazel:py_proto_library.bzl", "py_proto_library") +load("@rules_proto//proto:defs.bzl", "proto_library") + +# gazelle:python_generate_proto false + +proto_library( + name = "foo_proto", + srcs = ["foo.proto"], + visibility = ["//:__subpackages__"], +) + +py_proto_library( + name = "foo_py_pb2", + visibility = ["//:__subpackages__"], + deps = [":foo_proto"], +) diff --git a/gazelle/python/testdata/directive_python_generate_proto/test8_disabled_ignores_py_proto_library/foo.proto b/gazelle/python/testdata/directive_python_generate_proto/test8_disabled_ignores_py_proto_library/foo.proto new file mode 100644 index 0000000000..022e29ae69 --- /dev/null +++ b/gazelle/python/testdata/directive_python_generate_proto/test8_disabled_ignores_py_proto_library/foo.proto @@ -0,0 +1,7 @@ +syntax = "proto3"; + +package foo.bar; + +message Foo { + string bar = 1; +} diff --git a/gazelle/python/testdata/directive_python_generate_proto_bzlmod_protobuf/BUILD.in b/gazelle/python/testdata/directive_python_generate_proto_bzlmod_protobuf/BUILD.in new file mode 100644 index 0000000000..4713404b19 --- /dev/null +++ b/gazelle/python/testdata/directive_python_generate_proto_bzlmod_protobuf/BUILD.in @@ -0,0 +1,9 @@ +load("@rules_proto//proto:defs.bzl", "proto_library") + +# gazelle:python_generate_proto true + +proto_library( + name = "foo_proto", + srcs = ["foo.proto"], + visibility = ["//:__subpackages__"], +) diff --git a/gazelle/python/testdata/directive_python_generate_proto_bzlmod_protobuf/BUILD.out b/gazelle/python/testdata/directive_python_generate_proto_bzlmod_protobuf/BUILD.out new file mode 100644 index 0000000000..dab84a6777 --- /dev/null +++ b/gazelle/python/testdata/directive_python_generate_proto_bzlmod_protobuf/BUILD.out @@ -0,0 +1,16 @@ +load("@protobuf//bazel:py_proto_library.bzl", "py_proto_library") +load("@rules_proto//proto:defs.bzl", "proto_library") + +# gazelle:python_generate_proto true + +proto_library( + name = "foo_proto", + srcs = ["foo.proto"], + visibility = ["//:__subpackages__"], +) + +py_proto_library( + name = "foo_py_pb2", + visibility = ["//:__subpackages__"], + deps = [":foo_proto"], +) diff --git a/gazelle/python/testdata/directive_python_generate_proto_bzlmod_protobuf/MODULE.bazel b/gazelle/python/testdata/directive_python_generate_proto_bzlmod_protobuf/MODULE.bazel new file mode 100644 index 0000000000..66d64afe03 --- /dev/null +++ b/gazelle/python/testdata/directive_python_generate_proto_bzlmod_protobuf/MODULE.bazel @@ -0,0 +1 @@ +bazel_dep(name = "protobuf", version = "29.3") diff --git a/gazelle/python/testdata/directive_python_generate_proto_bzlmod_protobuf/README.md b/gazelle/python/testdata/directive_python_generate_proto_bzlmod_protobuf/README.md new file mode 100644 index 0000000000..2d91ccff56 --- /dev/null +++ b/gazelle/python/testdata/directive_python_generate_proto_bzlmod_protobuf/README.md @@ -0,0 +1,6 @@ +# Directive: `python_generate_proto` + +This test case asserts that the `# gazelle:python_generate_proto` directive +correctly reads the name of the protobuf repository when bzlmod is being used. + +[gh-2994]: https://github.com/bazel-contrib/rules_python/issues/2994 diff --git a/gazelle/python/testdata/directive_python_generate_proto_bzlmod_protobuf/WORKSPACE b/gazelle/python/testdata/directive_python_generate_proto_bzlmod_protobuf/WORKSPACE new file mode 100644 index 0000000000..e69de29bb2 diff --git a/gazelle/python/testdata/directive_python_generate_proto_bzlmod_protobuf/foo.proto b/gazelle/python/testdata/directive_python_generate_proto_bzlmod_protobuf/foo.proto new file mode 100644 index 0000000000..fe2af27aa6 --- /dev/null +++ b/gazelle/python/testdata/directive_python_generate_proto_bzlmod_protobuf/foo.proto @@ -0,0 +1,7 @@ +syntax = "proto3"; + +package foo; + +message Foo { + string bar = 1; +} diff --git a/gazelle/python/testdata/directive_python_generate_proto_bzlmod_protobuf/test.yaml b/gazelle/python/testdata/directive_python_generate_proto_bzlmod_protobuf/test.yaml new file mode 100644 index 0000000000..36dd656b39 --- /dev/null +++ b/gazelle/python/testdata/directive_python_generate_proto_bzlmod_protobuf/test.yaml @@ -0,0 +1,3 @@ +--- +expect: + exit_code: 0 diff --git a/gazelle/python/testdata/directive_python_generate_proto_bzlmod_protobuf_renamed/BUILD.in b/gazelle/python/testdata/directive_python_generate_proto_bzlmod_protobuf_renamed/BUILD.in new file mode 100644 index 0000000000..4713404b19 --- /dev/null +++ b/gazelle/python/testdata/directive_python_generate_proto_bzlmod_protobuf_renamed/BUILD.in @@ -0,0 +1,9 @@ +load("@rules_proto//proto:defs.bzl", "proto_library") + +# gazelle:python_generate_proto true + +proto_library( + name = "foo_proto", + srcs = ["foo.proto"], + visibility = ["//:__subpackages__"], +) diff --git a/gazelle/python/testdata/directive_python_generate_proto_bzlmod_protobuf_renamed/BUILD.out b/gazelle/python/testdata/directive_python_generate_proto_bzlmod_protobuf_renamed/BUILD.out new file mode 100644 index 0000000000..686252f27c --- /dev/null +++ b/gazelle/python/testdata/directive_python_generate_proto_bzlmod_protobuf_renamed/BUILD.out @@ -0,0 +1,16 @@ +load("@com_google_protobuf//bazel:py_proto_library.bzl", "py_proto_library") +load("@rules_proto//proto:defs.bzl", "proto_library") + +# gazelle:python_generate_proto true + +proto_library( + name = "foo_proto", + srcs = ["foo.proto"], + visibility = ["//:__subpackages__"], +) + +py_proto_library( + name = "foo_py_pb2", + visibility = ["//:__subpackages__"], + deps = [":foo_proto"], +) diff --git a/gazelle/python/testdata/directive_python_generate_proto_bzlmod_protobuf_renamed/MODULE.bazel b/gazelle/python/testdata/directive_python_generate_proto_bzlmod_protobuf_renamed/MODULE.bazel new file mode 100644 index 0000000000..9ab4c175aa --- /dev/null +++ b/gazelle/python/testdata/directive_python_generate_proto_bzlmod_protobuf_renamed/MODULE.bazel @@ -0,0 +1 @@ +bazel_dep(name = "protobuf", version = "29.3", repo_name = "com_google_protobuf") diff --git a/gazelle/python/testdata/directive_python_generate_proto_bzlmod_protobuf_renamed/README.md b/gazelle/python/testdata/directive_python_generate_proto_bzlmod_protobuf_renamed/README.md new file mode 100644 index 0000000000..7900d49084 --- /dev/null +++ b/gazelle/python/testdata/directive_python_generate_proto_bzlmod_protobuf_renamed/README.md @@ -0,0 +1,7 @@ +# Directive: `python_generate_proto` + +This test case asserts that the `# gazelle:python_generate_proto` directive +correctly reads the name of the protobuf repository when bzlmod is being used, +but the repository is renamed. + +[gh-2994]: https://github.com/bazel-contrib/rules_python/issues/2994 diff --git a/gazelle/python/testdata/directive_python_generate_proto_bzlmod_protobuf_renamed/WORKSPACE b/gazelle/python/testdata/directive_python_generate_proto_bzlmod_protobuf_renamed/WORKSPACE new file mode 100644 index 0000000000..e69de29bb2 diff --git a/gazelle/python/testdata/directive_python_generate_proto_bzlmod_protobuf_renamed/foo.proto b/gazelle/python/testdata/directive_python_generate_proto_bzlmod_protobuf_renamed/foo.proto new file mode 100644 index 0000000000..fe2af27aa6 --- /dev/null +++ b/gazelle/python/testdata/directive_python_generate_proto_bzlmod_protobuf_renamed/foo.proto @@ -0,0 +1,7 @@ +syntax = "proto3"; + +package foo; + +message Foo { + string bar = 1; +} diff --git a/gazelle/python/testdata/directive_python_generate_proto_bzlmod_protobuf_renamed/test.yaml b/gazelle/python/testdata/directive_python_generate_proto_bzlmod_protobuf_renamed/test.yaml new file mode 100644 index 0000000000..36dd656b39 --- /dev/null +++ b/gazelle/python/testdata/directive_python_generate_proto_bzlmod_protobuf_renamed/test.yaml @@ -0,0 +1,3 @@ +--- +expect: + exit_code: 0 diff --git a/gazelle/pythonconfig/pythonconfig.go b/gazelle/pythonconfig/pythonconfig.go index 8bf79cbc15..b76e1f92ec 100644 --- a/gazelle/pythonconfig/pythonconfig.go +++ b/gazelle/pythonconfig/pythonconfig.go @@ -98,6 +98,9 @@ const ( // separate pyi_deps attribute or merge type-checking dependencies into deps. // Defaults to false for backward compatibility. GeneratePyiDeps = "python_generate_pyi_deps" + // GenerateProto represents the directive that controls whether to generate + // python_generate_proto targets. + GenerateProto = "python_generate_proto" ) // GenerationModeType represents one of the generation modes for the Python @@ -186,6 +189,7 @@ type Config struct { labelNormalization LabelNormalizationType experimentalAllowRelativeImports bool generatePyiDeps bool + generateProto bool } type LabelNormalizationType int @@ -223,6 +227,7 @@ func New( labelNormalization: DefaultLabelNormalizationType, experimentalAllowRelativeImports: false, generatePyiDeps: false, + generateProto: false, } } @@ -257,6 +262,7 @@ func (c *Config) NewChild() *Config { labelNormalization: c.labelNormalization, experimentalAllowRelativeImports: c.experimentalAllowRelativeImports, generatePyiDeps: c.generatePyiDeps, + generateProto: c.generateProto, } } @@ -555,6 +561,16 @@ func (c *Config) GeneratePyiDeps() bool { return c.generatePyiDeps } +// SetGenerateProto sets whether py_proto_library should be generated for proto_library. +func (c *Config) SetGenerateProto(generateProto bool) { + c.generateProto = generateProto +} + +// GenerateProto returns whether py_proto_library should be generated for proto_library. +func (c *Config) GenerateProto() bool { + return c.generateProto +} + // FormatThirdPartyDependency returns a label to a third-party dependency performing all formating and normalization. func (c *Config) FormatThirdPartyDependency(repositoryName string, distributionName string) label.Label { conventionalDistributionName := strings.ReplaceAll(c.labelConvention, distributionNameLabelConventionSubstitution, distributionName) From a97b98ccbbab5070b088dd94efc485952ed8459e Mon Sep 17 00:00:00 2001 From: Douglas Thor Date: Mon, 14 Jul 2025 18:23:25 -0700 Subject: [PATCH 330/922] feat(gazelle): Add `include_pytest_conftest` annotation (#3080) Fixes #3076. Add a new gazelle annotation `include_pytest_conftest`. When unset or true, the gazelle behavior is unchanged. When false, gazelle will *not* inject the `:conftest` dependency to py_test targets. One of the refactorings that is done to support this is to pass around an `annotations` struct in `target.targetBuilder`. This will also open up support for other annotations in the future. --------- Co-authored-by: Ignas Anikevicius <240938+aignas@users.noreply.github.com> --- CHANGELOG.md | 5 ++ gazelle/README.md | 85 +++++++++++++++++++ gazelle/python/generate.go | 17 +++- gazelle/python/parser.go | 30 ++++++- gazelle/python/target.go | 9 ++ .../README.md | 25 ++++++ .../WORKSPACE | 0 .../test.yaml | 5 ++ .../with_conftest/BUILD.in | 0 .../with_conftest/BUILD.out | 68 +++++++++++++++ .../with_conftest/bad_value_test.py | 1 + .../with_conftest/binary.py | 3 + .../with_conftest/conftest.py | 0 .../with_conftest/conftest_imported_test.py | 3 + .../with_conftest/conftest_included_test.py | 2 + .../with_conftest/false_test.py | 1 + .../with_conftest/falsey_test.py | 1 + .../with_conftest/last_value_wins_test.py | 6 ++ .../with_conftest/library.py | 1 + .../with_conftest/true_test.py | 1 + .../with_conftest/unset_test.py | 0 .../without_conftest/BUILD.in | 0 .../without_conftest/BUILD.out | 16 ++++ .../without_conftest/false_test.py | 1 + .../without_conftest/true_test.py | 1 + .../without_conftest/unset_test.py | 0 26 files changed, 275 insertions(+), 6 deletions(-) create mode 100644 gazelle/python/testdata/annotation_include_pytest_conftest/README.md create mode 100644 gazelle/python/testdata/annotation_include_pytest_conftest/WORKSPACE create mode 100644 gazelle/python/testdata/annotation_include_pytest_conftest/test.yaml create mode 100644 gazelle/python/testdata/annotation_include_pytest_conftest/with_conftest/BUILD.in create mode 100644 gazelle/python/testdata/annotation_include_pytest_conftest/with_conftest/BUILD.out create mode 100644 gazelle/python/testdata/annotation_include_pytest_conftest/with_conftest/bad_value_test.py create mode 100644 gazelle/python/testdata/annotation_include_pytest_conftest/with_conftest/binary.py create mode 100644 gazelle/python/testdata/annotation_include_pytest_conftest/with_conftest/conftest.py create mode 100644 gazelle/python/testdata/annotation_include_pytest_conftest/with_conftest/conftest_imported_test.py create mode 100644 gazelle/python/testdata/annotation_include_pytest_conftest/with_conftest/conftest_included_test.py create mode 100644 gazelle/python/testdata/annotation_include_pytest_conftest/with_conftest/false_test.py create mode 100644 gazelle/python/testdata/annotation_include_pytest_conftest/with_conftest/falsey_test.py create mode 100644 gazelle/python/testdata/annotation_include_pytest_conftest/with_conftest/last_value_wins_test.py create mode 100644 gazelle/python/testdata/annotation_include_pytest_conftest/with_conftest/library.py create mode 100644 gazelle/python/testdata/annotation_include_pytest_conftest/with_conftest/true_test.py create mode 100644 gazelle/python/testdata/annotation_include_pytest_conftest/with_conftest/unset_test.py create mode 100644 gazelle/python/testdata/annotation_include_pytest_conftest/without_conftest/BUILD.in create mode 100644 gazelle/python/testdata/annotation_include_pytest_conftest/without_conftest/BUILD.out create mode 100644 gazelle/python/testdata/annotation_include_pytest_conftest/without_conftest/false_test.py create mode 100644 gazelle/python/testdata/annotation_include_pytest_conftest/without_conftest/true_test.py create mode 100644 gazelle/python/testdata/annotation_include_pytest_conftest/without_conftest/unset_test.py diff --git a/CHANGELOG.md b/CHANGELOG.md index e74f14b1db..b65a233f1e 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -106,6 +106,11 @@ END_UNRELEASED_TEMPLATE * 3.12.11 * 3.13.5 * 3.14.0b3 +* (gazelle): New annotation `gazelle:include_pytest_conftest`. When not set (the + default) or `true`, gazelle will inject any `conftest.py` file found in the same + directory as a {obj}`py_test` target to that {obj}`py_test` target's `deps`. + This behavior is unchanged from previous versions. When `false`, the `:conftest` + dep is not added to the {obj}`py_test` target. * (gazelle) New directive `gazelle:python_generate_proto`; when `true`, Gazelle generates `py_proto_library` rules for `proto_library`. `false` by default. diff --git a/gazelle/README.md b/gazelle/README.md index 35a1e4f701..cf91461e39 100644 --- a/gazelle/README.md +++ b/gazelle/README.md @@ -550,6 +550,8 @@ The annotations are: | Tells Gazelle to ignore import statements. `imports` is a comma-separated list of imports to ignore. | | | [`# gazelle:include_dep targets`](#annotation-include_dep) | N/A | | Tells Gazelle to include a set of dependencies, even if they are not imported in a Python module. `targets` is a comma-separated list of target names to include as dependencies. | | +| [`# gazelle:include_pytest_conftest bool`](#annotation-include_pytest_conftest) | N/A | +| Whether or not to include a sibling `:conftest` target in the deps of a `py_test` target. Default behaviour is to include `:conftest`. | | #### Annotation: `ignore` @@ -622,6 +624,89 @@ deps = [ ] ``` +#### Annotation: `include_pytest_conftest` + +Added in [#3080][gh3080]. + +[gh3080]: https://github.com/bazel-contrib/rules_python/pull/3080 + +This annotation accepts any string that can be parsed by go's +[`strconv.ParseBool`][ParseBool]. If an unparsable string is passed, the +annotation is ignored. + +[ParseBool]: https://pkg.go.dev/strconv#ParseBool + +Starting with [`rules_python` 0.14.0][rules-python-0.14.0] (specifically [PR #879][gh879]), +Gazelle will include a `:conftest` dependency to an `py_test` target that is in +the same directory as `conftest.py`. + +[rules-python-0.14.0]: https://github.com/bazel-contrib/rules_python/releases/tag/0.14.0 +[gh879]: https://github.com/bazel-contrib/rules_python/pull/879 + +This annotation allows users to adjust that behavior. To disable the behavior, set +the annotation value to "false": + +``` +# some_file_test.py +# gazelle:include_pytest_conftest false +``` + +Example: + +Given a directory tree like: + +``` +. +├── BUILD.bazel +├── conftest.py +└── some_file_test.py +``` + +The default Gazelle behavior would create: + +```starlark +py_library( + name = "conftest", + testonly = True, + srcs = ["conftest.py"], + visibility = ["//:__subpackages__"], +) + +py_test( + name = "some_file_test", + srcs = ["some_file_test.py"], + deps = [":conftest"], +) +``` + +When `# gazelle:include_pytest_conftest false` is found in `some_file_test.py` + +```python +# some_file_test.py +# gazelle:include_pytest_conftest false +``` + +Gazelle will generate: + +```starlark +py_library( + name = "conftest", + testonly = True, + srcs = ["conftest.py"], + visibility = ["//:__subpackages__"], +) + +py_test( + name = "some_file_test", + srcs = ["some_file_test.py"], +) +``` + +See [Issue #3076][gh3076] for more information. + +[gh3076]: https://github.com/bazel-contrib/rules_python/issues/3076 + + #### Directive: `experimental_allow_relative_imports` Enables experimental support for resolving relative imports in `python_generation_mode package`. diff --git a/gazelle/python/generate.go b/gazelle/python/generate.go index 343743559f..279bee6af7 100644 --- a/gazelle/python/generate.go +++ b/gazelle/python/generate.go @@ -264,7 +264,9 @@ func (py *Python) GenerateRules(args language.GenerateArgs) language.GenerateRes addSrc(filename). addModuleDependencies(mainModules[filename]). addResolvedDependencies(annotations.includeDeps). - generateImportsAttribute().build() + generateImportsAttribute(). + setAnnotations(*annotations). + build() result.Gen = append(result.Gen, pyBinary) result.Imports = append(result.Imports, pyBinary.PrivateAttr(config.GazelleImportsKey)) } @@ -305,6 +307,7 @@ func (py *Python) GenerateRules(args language.GenerateArgs) language.GenerateRes addModuleDependencies(allDeps). addResolvedDependencies(annotations.includeDeps). generateImportsAttribute(). + setAnnotations(*annotations). build() if pyLibrary.IsEmpty(py.Kinds()[pyLibrary.Kind()]) { @@ -357,6 +360,7 @@ func (py *Python) GenerateRules(args language.GenerateArgs) language.GenerateRes addSrc(pyBinaryEntrypointFilename). addModuleDependencies(deps). addResolvedDependencies(annotations.includeDeps). + setAnnotations(*annotations). generateImportsAttribute() pyBinary := pyBinaryTarget.build() @@ -387,6 +391,7 @@ func (py *Python) GenerateRules(args language.GenerateArgs) language.GenerateRes addSrc(conftestFilename). addModuleDependencies(deps). addResolvedDependencies(annotations.includeDeps). + setAnnotations(*annotations). addVisibility(visibility). setTestonly(). generateImportsAttribute() @@ -418,6 +423,7 @@ func (py *Python) GenerateRules(args language.GenerateArgs) language.GenerateRes addSrcs(srcs). addModuleDependencies(deps). addResolvedDependencies(annotations.includeDeps). + setAnnotations(*annotations). generateImportsAttribute() } if (!cfg.PerPackageGenerationRequireTestEntryPoint() || hasPyTestEntryPointFile || hasPyTestEntryPointTarget || cfg.CoarseGrainedGeneration()) && !cfg.PerFileGeneration() { @@ -470,7 +476,14 @@ func (py *Python) GenerateRules(args language.GenerateArgs) language.GenerateRes for _, pyTestTarget := range pyTestTargets { if conftest != nil { - pyTestTarget.addModuleDependency(Module{Name: strings.TrimSuffix(conftestFilename, ".py")}) + conftestModule := Module{Name: strings.TrimSuffix(conftestFilename, ".py")} + if pyTestTarget.annotations.includePytestConftest == nil { + // unset; default behavior + pyTestTarget.addModuleDependency(conftestModule) + } else if *pyTestTarget.annotations.includePytestConftest { + // set; add if true, do not add if false + pyTestTarget.addModuleDependency(conftestModule) + } } pyTest := pyTestTarget.build() diff --git a/gazelle/python/parser.go b/gazelle/python/parser.go index 11e01dbf51..3d0dbe7a5f 100644 --- a/gazelle/python/parser.go +++ b/gazelle/python/parser.go @@ -18,6 +18,8 @@ import ( "context" _ "embed" "fmt" + "log" + "strconv" "strings" "github.com/emirpasic/gods/sets/treeset" @@ -123,6 +125,7 @@ func (p *python3Parser) parse(pyFilenames *treeset.Set) (*treeset.Set, map[strin allAnnotations.ignore[k] = v } allAnnotations.includeDeps = append(allAnnotations.includeDeps, annotations.includeDeps...) + allAnnotations.includePytestConftest = annotations.includePytestConftest } allAnnotations.includeDeps = removeDupesFromStringTreeSetSlice(allAnnotations.includeDeps) @@ -183,8 +186,12 @@ const ( // The Gazelle annotation prefix. annotationPrefix string = "gazelle:" // The ignore annotation kind. E.g. '# gazelle:ignore '. - annotationKindIgnore annotationKind = "ignore" - annotationKindIncludeDep annotationKind = "include_dep" + annotationKindIgnore annotationKind = "ignore" + // Force a particular target to be added to `deps`. Multiple invocations are + // accumulated and the value can be comma separated. + // Eg: '# gazelle:include_dep //foo/bar:baz,@repo//:target + annotationKindIncludeDep annotationKind = "include_dep" + annotationKindIncludePytestConftest annotationKind = "include_pytest_conftest" ) // Comment represents a Python comment. @@ -222,6 +229,10 @@ type annotations struct { ignore map[string]struct{} // Labels that Gazelle should include as deps of the generated target. includeDeps []string + // Whether the conftest.py file, found in the same directory as the current + // python test file, should be added to the py_test target's `deps` attribute. + // A *bool is used so that we can handle the "not set" state. + includePytestConftest *bool } // annotationsFromComments returns all the annotations parsed out of the @@ -229,6 +240,7 @@ type annotations struct { func annotationsFromComments(comments []Comment) (*annotations, error) { ignore := make(map[string]struct{}) includeDeps := []string{} + var includePytestConftest *bool for _, comment := range comments { annotation, err := comment.asAnnotation() if err != nil { @@ -255,11 +267,21 @@ func annotationsFromComments(comments []Comment) (*annotations, error) { includeDeps = append(includeDeps, t) } } + if annotation.kind == annotationKindIncludePytestConftest { + val := annotation.value + parsedVal, err := strconv.ParseBool(val) + if err != nil { + log.Printf("WARNING: unable to cast %q to bool in %q. Ignoring annotation", val, comment) + continue + } + includePytestConftest = &parsedVal + } } } return &annotations{ - ignore: ignore, - includeDeps: includeDeps, + ignore: ignore, + includeDeps: includeDeps, + includePytestConftest: includePytestConftest, }, nil } diff --git a/gazelle/python/target.go b/gazelle/python/target.go index 06b653d915..6e6c3f4b14 100644 --- a/gazelle/python/target.go +++ b/gazelle/python/target.go @@ -37,6 +37,7 @@ type targetBuilder struct { main *string imports []string testonly bool + annotations *annotations } // newTargetBuilder constructs a new targetBuilder. @@ -51,6 +52,7 @@ func newTargetBuilder(kind, name, pythonProjectRoot, bzlPackage string, siblingS deps: treeset.NewWith(moduleComparator), resolvedDeps: treeset.NewWith(godsutils.StringComparator), visibility: treeset.NewWith(godsutils.StringComparator), + annotations: new(annotations), } } @@ -130,6 +132,13 @@ func (t *targetBuilder) setTestonly() *targetBuilder { return t } +// setAnnotations sets the annotations attribute on the target. +func (t *targetBuilder) setAnnotations(val annotations) *targetBuilder { + t.annotations = &val + return t +} + + // generateImportsAttribute generates the imports attribute. // These are a list of import directories to be added to the PYTHONPATH. In our // case, the value we add is on Bazel sub-packages to be able to perform imports diff --git a/gazelle/python/testdata/annotation_include_pytest_conftest/README.md b/gazelle/python/testdata/annotation_include_pytest_conftest/README.md new file mode 100644 index 0000000000..6a347d154e --- /dev/null +++ b/gazelle/python/testdata/annotation_include_pytest_conftest/README.md @@ -0,0 +1,25 @@ +# Annotation: Include Pytest Conftest + +Validate that the `# gazelle:include_pytest_conftest` annotation follows +this logic: + ++ When a `conftest.py` file does not exist: + + all values have no affect ++ When a `conftest.py` file does exist: + + Truthy values add `:conftest` to `deps`. + + Falsey values do not add `:conftest` to `deps`. + + Unset (no annotation) performs the default action. + +Additionally, we test that: + ++ invalid values (eg `foo`) print a warning and then act as if + the annotation was not present. ++ last annotation (highest line number) wins. ++ the annotation has no effect on non-test files/targets. ++ the `include_dep` can still inject `:conftest` even when `include_pytest_conftest` + is false. ++ `import conftest` will still add the dep even when `include_pytest_conftest` is + false. + +An annotation without a value is not tested, as that's part of the core +annotation framework and not specific to this annotation. diff --git a/gazelle/python/testdata/annotation_include_pytest_conftest/WORKSPACE b/gazelle/python/testdata/annotation_include_pytest_conftest/WORKSPACE new file mode 100644 index 0000000000..e69de29bb2 diff --git a/gazelle/python/testdata/annotation_include_pytest_conftest/test.yaml b/gazelle/python/testdata/annotation_include_pytest_conftest/test.yaml new file mode 100644 index 0000000000..e643d0e90c --- /dev/null +++ b/gazelle/python/testdata/annotation_include_pytest_conftest/test.yaml @@ -0,0 +1,5 @@ +--- +expect: + stderr: | + gazelle: WARNING: unable to cast "foo" to bool in "# gazelle:include_pytest_conftest foo". Ignoring annotation + exit_code: 0 diff --git a/gazelle/python/testdata/annotation_include_pytest_conftest/with_conftest/BUILD.in b/gazelle/python/testdata/annotation_include_pytest_conftest/with_conftest/BUILD.in new file mode 100644 index 0000000000..e69de29bb2 diff --git a/gazelle/python/testdata/annotation_include_pytest_conftest/with_conftest/BUILD.out b/gazelle/python/testdata/annotation_include_pytest_conftest/with_conftest/BUILD.out new file mode 100644 index 0000000000..60695352ca --- /dev/null +++ b/gazelle/python/testdata/annotation_include_pytest_conftest/with_conftest/BUILD.out @@ -0,0 +1,68 @@ +load("@rules_python//python:defs.bzl", "py_binary", "py_library", "py_test") + +py_binary( + name = "binary", + srcs = ["binary.py"], + visibility = ["//:__subpackages__"], +) + +py_library( + name = "with_conftest", + srcs = [ + "binary.py", + "library.py", + ], + visibility = ["//:__subpackages__"], +) + +py_library( + name = "conftest", + testonly = True, + srcs = ["conftest.py"], + visibility = ["//:__subpackages__"], +) + +py_test( + name = "bad_value_test", + srcs = ["bad_value_test.py"], + deps = [":conftest"], +) + +py_test( + name = "conftest_imported_test", + srcs = ["conftest_imported_test.py"], + deps = [":conftest"], +) + +py_test( + name = "conftest_included_test", + srcs = ["conftest_included_test.py"], + deps = [":conftest"], +) + +py_test( + name = "false_test", + srcs = ["false_test.py"], +) + +py_test( + name = "falsey_test", + srcs = ["falsey_test.py"], +) + +py_test( + name = "last_value_wins_test", + srcs = ["last_value_wins_test.py"], +) + +py_test( + name = "true_test", + srcs = ["true_test.py"], + deps = [":conftest"], +) + +py_test( + name = "unset_test", + srcs = ["unset_test.py"], + deps = [":conftest"], +) diff --git a/gazelle/python/testdata/annotation_include_pytest_conftest/with_conftest/bad_value_test.py b/gazelle/python/testdata/annotation_include_pytest_conftest/with_conftest/bad_value_test.py new file mode 100644 index 0000000000..af2e8c54e0 --- /dev/null +++ b/gazelle/python/testdata/annotation_include_pytest_conftest/with_conftest/bad_value_test.py @@ -0,0 +1 @@ +# gazelle:include_pytest_conftest foo diff --git a/gazelle/python/testdata/annotation_include_pytest_conftest/with_conftest/binary.py b/gazelle/python/testdata/annotation_include_pytest_conftest/with_conftest/binary.py new file mode 100644 index 0000000000..d6dc8413d4 --- /dev/null +++ b/gazelle/python/testdata/annotation_include_pytest_conftest/with_conftest/binary.py @@ -0,0 +1,3 @@ +# gazelle:include_pytest_conftest true +if __name__ == "__main__": + pass diff --git a/gazelle/python/testdata/annotation_include_pytest_conftest/with_conftest/conftest.py b/gazelle/python/testdata/annotation_include_pytest_conftest/with_conftest/conftest.py new file mode 100644 index 0000000000..e69de29bb2 diff --git a/gazelle/python/testdata/annotation_include_pytest_conftest/with_conftest/conftest_imported_test.py b/gazelle/python/testdata/annotation_include_pytest_conftest/with_conftest/conftest_imported_test.py new file mode 100644 index 0000000000..2c72ca4df1 --- /dev/null +++ b/gazelle/python/testdata/annotation_include_pytest_conftest/with_conftest/conftest_imported_test.py @@ -0,0 +1,3 @@ +import conftest + +# gazelle:include_pytest_conftest false diff --git a/gazelle/python/testdata/annotation_include_pytest_conftest/with_conftest/conftest_included_test.py b/gazelle/python/testdata/annotation_include_pytest_conftest/with_conftest/conftest_included_test.py new file mode 100644 index 0000000000..c942bfb1ab --- /dev/null +++ b/gazelle/python/testdata/annotation_include_pytest_conftest/with_conftest/conftest_included_test.py @@ -0,0 +1,2 @@ +# gazelle:include_dep :conftest +# gazelle:include_pytest_conftest false diff --git a/gazelle/python/testdata/annotation_include_pytest_conftest/with_conftest/false_test.py b/gazelle/python/testdata/annotation_include_pytest_conftest/with_conftest/false_test.py new file mode 100644 index 0000000000..ba71a2818b --- /dev/null +++ b/gazelle/python/testdata/annotation_include_pytest_conftest/with_conftest/false_test.py @@ -0,0 +1 @@ +# gazelle:include_pytest_conftest false diff --git a/gazelle/python/testdata/annotation_include_pytest_conftest/with_conftest/falsey_test.py b/gazelle/python/testdata/annotation_include_pytest_conftest/with_conftest/falsey_test.py new file mode 100644 index 0000000000..c4387b3a8c --- /dev/null +++ b/gazelle/python/testdata/annotation_include_pytest_conftest/with_conftest/falsey_test.py @@ -0,0 +1 @@ +# gazelle:include_pytest_conftest 0 diff --git a/gazelle/python/testdata/annotation_include_pytest_conftest/with_conftest/last_value_wins_test.py b/gazelle/python/testdata/annotation_include_pytest_conftest/with_conftest/last_value_wins_test.py new file mode 100644 index 0000000000..6ffc06f9c0 --- /dev/null +++ b/gazelle/python/testdata/annotation_include_pytest_conftest/with_conftest/last_value_wins_test.py @@ -0,0 +1,6 @@ +# gazelle:include_pytest_conftest true +# gazelle:include_pytest_conftest TRUE +# gazelle:include_pytest_conftest False +# gazelle:include_pytest_conftest 0 +# gazelle:include_pytest_conftest 1 +# gazelle:include_pytest_conftest F diff --git a/gazelle/python/testdata/annotation_include_pytest_conftest/with_conftest/library.py b/gazelle/python/testdata/annotation_include_pytest_conftest/with_conftest/library.py new file mode 100644 index 0000000000..b2d10359da --- /dev/null +++ b/gazelle/python/testdata/annotation_include_pytest_conftest/with_conftest/library.py @@ -0,0 +1 @@ +# gazelle:include_pytest_conftest true diff --git a/gazelle/python/testdata/annotation_include_pytest_conftest/with_conftest/true_test.py b/gazelle/python/testdata/annotation_include_pytest_conftest/with_conftest/true_test.py new file mode 100644 index 0000000000..b2d10359da --- /dev/null +++ b/gazelle/python/testdata/annotation_include_pytest_conftest/with_conftest/true_test.py @@ -0,0 +1 @@ +# gazelle:include_pytest_conftest true diff --git a/gazelle/python/testdata/annotation_include_pytest_conftest/with_conftest/unset_test.py b/gazelle/python/testdata/annotation_include_pytest_conftest/with_conftest/unset_test.py new file mode 100644 index 0000000000..e69de29bb2 diff --git a/gazelle/python/testdata/annotation_include_pytest_conftest/without_conftest/BUILD.in b/gazelle/python/testdata/annotation_include_pytest_conftest/without_conftest/BUILD.in new file mode 100644 index 0000000000..e69de29bb2 diff --git a/gazelle/python/testdata/annotation_include_pytest_conftest/without_conftest/BUILD.out b/gazelle/python/testdata/annotation_include_pytest_conftest/without_conftest/BUILD.out new file mode 100644 index 0000000000..01383344c5 --- /dev/null +++ b/gazelle/python/testdata/annotation_include_pytest_conftest/without_conftest/BUILD.out @@ -0,0 +1,16 @@ +load("@rules_python//python:defs.bzl", "py_test") + +py_test( + name = "false_test", + srcs = ["false_test.py"], +) + +py_test( + name = "true_test", + srcs = ["true_test.py"], +) + +py_test( + name = "unset_test", + srcs = ["unset_test.py"], +) diff --git a/gazelle/python/testdata/annotation_include_pytest_conftest/without_conftest/false_test.py b/gazelle/python/testdata/annotation_include_pytest_conftest/without_conftest/false_test.py new file mode 100644 index 0000000000..ba71a2818b --- /dev/null +++ b/gazelle/python/testdata/annotation_include_pytest_conftest/without_conftest/false_test.py @@ -0,0 +1 @@ +# gazelle:include_pytest_conftest false diff --git a/gazelle/python/testdata/annotation_include_pytest_conftest/without_conftest/true_test.py b/gazelle/python/testdata/annotation_include_pytest_conftest/without_conftest/true_test.py new file mode 100644 index 0000000000..b2d10359da --- /dev/null +++ b/gazelle/python/testdata/annotation_include_pytest_conftest/without_conftest/true_test.py @@ -0,0 +1 @@ +# gazelle:include_pytest_conftest true diff --git a/gazelle/python/testdata/annotation_include_pytest_conftest/without_conftest/unset_test.py b/gazelle/python/testdata/annotation_include_pytest_conftest/without_conftest/unset_test.py new file mode 100644 index 0000000000..e69de29bb2 From 65a1c8541b8fdf45700bae110f2ae7d7a311c9cd Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Mon, 14 Jul 2025 18:29:31 -0700 Subject: [PATCH 331/922] docs: tell how to emulate dependency groups with pip-compile (#3089) While pyproject.toml is supported by piptools, dependency groups aren't. They can be emulated by using multiple files. Explain how to do that in the docs and link to the upstream feature request (https://github.com/jazzband/pip-tools/issues/2062) Along the way, link to our own feature request for pylock.toml support in pip.parse. --------- Co-authored-by: Ignas Anikevicius <240938+aignas@users.noreply.github.com> --- docs/pypi/lock.md | 28 ++++++++++++++++++++++++++-- python/private/pypi/pip_compile.bzl | 7 +++++-- 2 files changed, 31 insertions(+), 4 deletions(-) diff --git a/docs/pypi/lock.md b/docs/pypi/lock.md index db557fe594..b5d8ec24f7 100644 --- a/docs/pypi/lock.md +++ b/docs/pypi/lock.md @@ -5,6 +5,8 @@ :::{note} Currently `rules_python` only supports `requirements.txt` format. + +#{gh-issue}`2787` tracks `pylock.toml` support. ::: ## requirements.txt @@ -37,11 +39,33 @@ This rule generates two targets: Once you generate this fully specified list of requirements, you can install the requirements ([bzlmod](./download)/[WORKSPACE](./download-workspace)). :::{warning} -If you're specifying dependencies in `pyproject.toml`, make sure to include the `[build-system]` configuration, with pinned dependencies. `compile_pip_requirements` will use the build system specified to read your project's metadata, and you might see non-hermetic behavior if you don't pin the build system. +If you're specifying dependencies in `pyproject.toml`, make sure to include the +`[build-system]` configuration, with pinned dependencies. +`compile_pip_requirements` will use the build system specified to read your +project's metadata, and you might see non-hermetic behavior if you don't pin the +build system. -Not specifying `[build-system]` at all will result in using a default `[build-system]` configuration, which uses unpinned versions ([ref](https://peps.python.org/pep-0518/#build-system-table)). +Not specifying `[build-system]` at all will result in using a default +`[build-system]` configuration, which uses unpinned versions +([ref](https://peps.python.org/pep-0518/#build-system-table)). ::: + +#### pip compile Dependency groups + +pip-compile doesn't yet support pyproject.toml dependency groups. Follow +[pip-tools #2062](https://github.com/jazzband/pip-tools/issues/2062) +to see the status of their support. + +In the meantime, support can be emulated by passing multiple files to `srcs`: + +```starlark +compile_pip_requirements( + srcs = ["pyproject.toml", "requirements-dev.in"] + ... +) +``` + ### uv pip compile (bzlmod only) We also have experimental setup for the `uv pip compile` way of generating lock files. diff --git a/python/private/pypi/pip_compile.bzl b/python/private/pypi/pip_compile.bzl index 2e3e530153..28923005df 100644 --- a/python/private/pypi/pip_compile.bzl +++ b/python/private/pypi/pip_compile.bzl @@ -40,7 +40,7 @@ def pip_compile( tags = None, constraints = [], **kwargs): - """Generates targets for managing pip dependencies with pip-compile. + """Generates targets for managing pip dependencies with pip-compile (piptools). By default this rules generates a filegroup named "[name]" which can be included in the data of some other compile_pip_requirements rule that references these requirements @@ -65,7 +65,10 @@ def pip_compile( * a requirements text file, usually named `requirements.in` * A `.toml` file, where the `project.dependencies` list is used as per [PEP621](https://peps.python.org/pep-0621/). - extra_args: passed to pip-compile. + extra_args: passed to pip-compile (aka `piptools`). See the + [pip-compile docs](https://pip-tools.readthedocs.io/en/latest/cli/pip-compile) + for args and meaning (passing `-h` and/or `--version` can help + inform what args are available) extra_deps: extra dependencies passed to pip-compile. generate_hashes: whether to put hashes in the requirements_txt file. py_binary: the py_binary rule to be used. From 9555ba8e9ce0902f3d2a315a6e19b8bebe79c0ce Mon Sep 17 00:00:00 2001 From: Ignas Anikevicius <240938+aignas@users.noreply.github.com> Date: Tue, 15 Jul 2025 11:49:48 +0900 Subject: [PATCH 332/922] chore: update python toolchains (#3074) - use the SHA256SUMS file instead of individual sha256sum files. This improves the speed of the tooling and also the old files just disappeared for the latest toolchain release. - update to the latest release. --- CHANGELOG.md | 6 +- python/private/print_toolchain_checksums.bzl | 41 +++-- python/versions.bzl | 168 +++++++++---------- tests/python/python_tests.bzl | 2 +- 4 files changed, 107 insertions(+), 110 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index b65a233f1e..c7a5a8fad5 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -60,12 +60,12 @@ END_UNRELEASED_TEMPLATE * (gazelle) Types for exposed members of `python.ParserOutput` are now all public. * (gazelle) Removed the requirement for `__init__.py`, `__main__.py`, or `__test__.py` files to be present in a directory to generate a `BUILD.bazel` file. -* (toolchain) Updated the following toolchains to build 20250702 to patch CVE-2025-47273: +* (toolchain) Updated the following toolchains to build 20250708 to patch CVE-2025-47273: * 3.9.23 * 3.10.18 * 3.11.13 * 3.12.11 - * 3.14.0b3 + * 3.14.0b4 * (toolchain) Python 3.13 now references 3.13.5 * (gazelle) Switched back to smacker/go-tree-sitter, fixing [#2630](https://github.com/bazel-contrib/rules_python/issues/2630) @@ -105,7 +105,7 @@ END_UNRELEASED_TEMPLATE * 3.11.13 * 3.12.11 * 3.13.5 - * 3.14.0b3 + * 3.14.0b4 * (gazelle): New annotation `gazelle:include_pytest_conftest`. When not set (the default) or `true`, gazelle will inject any `conftest.py` file found in the same directory as a {obj}`py_test` target to that {obj}`py_test` target's `deps`. diff --git a/python/private/print_toolchain_checksums.bzl b/python/private/print_toolchain_checksums.bzl index eaaa5b9d75..bd370baf10 100644 --- a/python/private/print_toolchain_checksums.bzl +++ b/python/private/print_toolchain_checksums.bzl @@ -28,6 +28,7 @@ def print_toolchains_checksums(name): template = """\ cat > "$@" <<'EOF' #!/bin/bash +set -euo pipefail set -o errexit -o nounset -o pipefail @@ -54,28 +55,9 @@ EOF def _commands_for_version(*, python_version, metadata): lines = [] - lines += [ - "cat < Date: Tue, 15 Jul 2025 14:24:31 +0200 Subject: [PATCH 333/922] feat: replace /bin/bash with /usr/bin/env bash (#3087) This allows system which don't have this location (looking at you NixOS) to run the scripts. --- CHANGELOG.md | 1 + addlicense.sh | 2 +- docs/readthedocs_build.sh | 2 +- python/private/interpreter_tmpl.sh | 2 +- python/private/print_toolchain_checksums.bzl | 2 +- python/private/stage1_bootstrap_template.sh | 2 +- python/uv/private/lock.sh | 2 +- sphinxdocs/private/sphinx_run_template.sh | 2 +- tests/bootstrap_impls/external_binary_test.sh | 2 +- tests/integration/bazel_from_env | 2 +- 10 files changed, 10 insertions(+), 9 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index c7a5a8fad5..7248e1f9f2 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -70,6 +70,7 @@ END_UNRELEASED_TEMPLATE * (gazelle) Switched back to smacker/go-tree-sitter, fixing [#2630](https://github.com/bazel-contrib/rules_python/issues/2630) * (ci) We are now testing on Ubuntu 22.04 for RBE and non-RBE configurations. +* (core) #!/usr/bin/env bash is now used as a shebang in the stage1 bootstrap template. {#v0-0-0-fixed} ### Fixed diff --git a/addlicense.sh b/addlicense.sh index 8cc8fb33bc..8dc82bbcc9 100755 --- a/addlicense.sh +++ b/addlicense.sh @@ -1,4 +1,4 @@ -#!/bin/bash +#!/usr/bin/env bash # Copyright 2023 The Bazel Authors. All rights reserved. # # Licensed under the Apache License, Version 2.0 (the "License"); diff --git a/docs/readthedocs_build.sh b/docs/readthedocs_build.sh index 3f67310197..ec5390bfc7 100755 --- a/docs/readthedocs_build.sh +++ b/docs/readthedocs_build.sh @@ -1,4 +1,4 @@ -#!/bin/bash +#!/usr/bin/env bash set -eou pipefail diff --git a/python/private/interpreter_tmpl.sh b/python/private/interpreter_tmpl.sh index cfe85ec1be..c4e87fbb43 100644 --- a/python/private/interpreter_tmpl.sh +++ b/python/private/interpreter_tmpl.sh @@ -1,4 +1,4 @@ -#!/bin/bash +#!/usr/bin/env bash # --- begin runfiles.bash initialization v3 --- # Copy-pasted from the Bazel Bash runfiles library v3. diff --git a/python/private/print_toolchain_checksums.bzl b/python/private/print_toolchain_checksums.bzl index bd370baf10..b4fa400221 100644 --- a/python/private/print_toolchain_checksums.bzl +++ b/python/private/print_toolchain_checksums.bzl @@ -27,7 +27,7 @@ def print_toolchains_checksums(name): template = """\ cat > "$@" <<'EOF' -#!/bin/bash +#!/usr/bin/env bash set -euo pipefail set -o errexit -o nounset -o pipefail diff --git a/python/private/stage1_bootstrap_template.sh b/python/private/stage1_bootstrap_template.sh index d992b55cae..9927d4faa7 100644 --- a/python/private/stage1_bootstrap_template.sh +++ b/python/private/stage1_bootstrap_template.sh @@ -1,4 +1,4 @@ -#!/bin/bash +#!/usr/bin/env bash set -e diff --git a/python/uv/private/lock.sh b/python/uv/private/lock.sh index b6ba0c6c48..ffb19b2bea 100755 --- a/python/uv/private/lock.sh +++ b/python/uv/private/lock.sh @@ -1,4 +1,4 @@ -#!/bin/bash +#!/usr/bin/env bash set -euo pipefail if [[ -n "${BUILD_WORKSPACE_DIRECTORY:-}" ]]; then diff --git a/sphinxdocs/private/sphinx_run_template.sh b/sphinxdocs/private/sphinx_run_template.sh index 4a1f1e4410..aa83757c1b 100644 --- a/sphinxdocs/private/sphinx_run_template.sh +++ b/sphinxdocs/private/sphinx_run_template.sh @@ -1,4 +1,4 @@ -#!/bin/bash +#!/usr/bin/env bash declare -a args %SETUP_ARGS% diff --git a/tests/bootstrap_impls/external_binary_test.sh b/tests/bootstrap_impls/external_binary_test.sh index e3516af18e..92799354d6 100755 --- a/tests/bootstrap_impls/external_binary_test.sh +++ b/tests/bootstrap_impls/external_binary_test.sh @@ -1,4 +1,4 @@ -#!/bin/bash +#!/usr/bin/env bash set -euxo pipefail tmpdir="${TEST_TMPDIR}/external_binary" diff --git a/tests/integration/bazel_from_env b/tests/integration/bazel_from_env index 96780b8156..a372736f32 100755 --- a/tests/integration/bazel_from_env +++ b/tests/integration/bazel_from_env @@ -1,4 +1,4 @@ -#!/bin/bash +#!/usr/bin/env bash # # A simple wrapper so rules_bazel_integration_test can use the # bazel version inherited from the environment. From 6fa4459a877aa1df1dc26320182c403d00e003c8 Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Tue, 15 Jul 2025 21:24:58 +0900 Subject: [PATCH 334/922] build(deps): bump certifi from 2025.6.15 to 2025.7.14 in /docs (#3092) [![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=certifi&package-manager=pip&previous-version=2025.6.15&new-version=2025.7.14)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) ---
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot show ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> --- docs/requirements.txt | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/requirements.txt b/docs/requirements.txt index 7a32ff7716..9e46724770 100644 --- a/docs/requirements.txt +++ b/docs/requirements.txt @@ -17,9 +17,9 @@ babel==2.17.0 \ --hash=sha256:0c54cffb19f690cdcc52a3b50bcbf71e07a808d1c80d549f2459b9d2cf0afb9d \ --hash=sha256:4d0b53093fdfb4b21c92b5213dba5a1b23885afa8383709427046b21c366e5f2 # via sphinx -certifi==2025.6.15 \ - --hash=sha256:2e0c7ce7cb5d8f8634ca55d2ba7e6ec2689a2fd6537d8dec1296a477a4910057 \ - --hash=sha256:d747aa5a8b9bbbb1bb8c22bb13e22bd1f18e9796defa16bab421f7f7a317323b +certifi==2025.7.14 \ + --hash=sha256:6b31f564a415d79ee77df69d757bb49a5bb53bd9f756cbbe24394ffd6fc1f4b2 \ + --hash=sha256:8ea99dbdfaaf2ba2f9bac77b9249ef62ec5218e7c2b2e903378ed5fccf765995 # via requests charset-normalizer==3.4.2 \ --hash=sha256:005fa3432484527f9732ebd315da8da8001593e2cf46a3d817669f062c3d9ed4 \ From cab415d82ebbfd1b8b2d81ef61baf3f9a274a1b2 Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Tue, 15 Jul 2025 21:25:26 +0900 Subject: [PATCH 335/922] build(deps): bump certifi from 2025.6.15 to 2025.7.14 in /tools/publish (#3095) [![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=certifi&package-manager=pip&previous-version=2025.6.15&new-version=2025.7.14)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) ---
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot show ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> --- tools/publish/requirements_darwin.txt | 6 +++--- tools/publish/requirements_linux.txt | 6 +++--- tools/publish/requirements_universal.txt | 6 +++--- tools/publish/requirements_windows.txt | 6 +++--- 4 files changed, 12 insertions(+), 12 deletions(-) diff --git a/tools/publish/requirements_darwin.txt b/tools/publish/requirements_darwin.txt index dab86f3adc..677cc6f7eb 100644 --- a/tools/publish/requirements_darwin.txt +++ b/tools/publish/requirements_darwin.txt @@ -6,9 +6,9 @@ backports-tarfile==1.2.0 \ --hash=sha256:77e284d754527b01fb1e6fa8a1afe577858ebe4e9dad8919e34c862cb399bc34 \ --hash=sha256:d75e02c268746e1b8144c278978b6e98e85de6ad16f8e4b0844a154557eca991 # via jaraco-context -certifi==2025.6.15 \ - --hash=sha256:2e0c7ce7cb5d8f8634ca55d2ba7e6ec2689a2fd6537d8dec1296a477a4910057 \ - --hash=sha256:d747aa5a8b9bbbb1bb8c22bb13e22bd1f18e9796defa16bab421f7f7a317323b +certifi==2025.7.14 \ + --hash=sha256:6b31f564a415d79ee77df69d757bb49a5bb53bd9f756cbbe24394ffd6fc1f4b2 \ + --hash=sha256:8ea99dbdfaaf2ba2f9bac77b9249ef62ec5218e7c2b2e903378ed5fccf765995 # via requests charset-normalizer==3.4.2 \ --hash=sha256:005fa3432484527f9732ebd315da8da8001593e2cf46a3d817669f062c3d9ed4 \ diff --git a/tools/publish/requirements_linux.txt b/tools/publish/requirements_linux.txt index c9d25eab58..98f119b3c9 100644 --- a/tools/publish/requirements_linux.txt +++ b/tools/publish/requirements_linux.txt @@ -6,9 +6,9 @@ backports-tarfile==1.2.0 \ --hash=sha256:77e284d754527b01fb1e6fa8a1afe577858ebe4e9dad8919e34c862cb399bc34 \ --hash=sha256:d75e02c268746e1b8144c278978b6e98e85de6ad16f8e4b0844a154557eca991 # via jaraco-context -certifi==2025.6.15 \ - --hash=sha256:2e0c7ce7cb5d8f8634ca55d2ba7e6ec2689a2fd6537d8dec1296a477a4910057 \ - --hash=sha256:d747aa5a8b9bbbb1bb8c22bb13e22bd1f18e9796defa16bab421f7f7a317323b +certifi==2025.7.14 \ + --hash=sha256:6b31f564a415d79ee77df69d757bb49a5bb53bd9f756cbbe24394ffd6fc1f4b2 \ + --hash=sha256:8ea99dbdfaaf2ba2f9bac77b9249ef62ec5218e7c2b2e903378ed5fccf765995 # via requests cffi==1.17.1 \ --hash=sha256:045d61c734659cc045141be4bae381a41d89b741f795af1dd018bfb532fd0df8 \ diff --git a/tools/publish/requirements_universal.txt b/tools/publish/requirements_universal.txt index a642e9280d..58625a4aad 100644 --- a/tools/publish/requirements_universal.txt +++ b/tools/publish/requirements_universal.txt @@ -6,9 +6,9 @@ backports-tarfile==1.2.0 ; python_full_version < '3.12' \ --hash=sha256:77e284d754527b01fb1e6fa8a1afe577858ebe4e9dad8919e34c862cb399bc34 \ --hash=sha256:d75e02c268746e1b8144c278978b6e98e85de6ad16f8e4b0844a154557eca991 # via jaraco-context -certifi==2025.6.15 \ - --hash=sha256:2e0c7ce7cb5d8f8634ca55d2ba7e6ec2689a2fd6537d8dec1296a477a4910057 \ - --hash=sha256:d747aa5a8b9bbbb1bb8c22bb13e22bd1f18e9796defa16bab421f7f7a317323b +certifi==2025.7.14 \ + --hash=sha256:6b31f564a415d79ee77df69d757bb49a5bb53bd9f756cbbe24394ffd6fc1f4b2 \ + --hash=sha256:8ea99dbdfaaf2ba2f9bac77b9249ef62ec5218e7c2b2e903378ed5fccf765995 # via requests cffi==1.17.1 ; platform_python_implementation != 'PyPy' and sys_platform == 'linux' \ --hash=sha256:045d61c734659cc045141be4bae381a41d89b741f795af1dd018bfb532fd0df8 \ diff --git a/tools/publish/requirements_windows.txt b/tools/publish/requirements_windows.txt index d3944056c0..374541d96f 100644 --- a/tools/publish/requirements_windows.txt +++ b/tools/publish/requirements_windows.txt @@ -6,9 +6,9 @@ backports-tarfile==1.2.0 \ --hash=sha256:77e284d754527b01fb1e6fa8a1afe577858ebe4e9dad8919e34c862cb399bc34 \ --hash=sha256:d75e02c268746e1b8144c278978b6e98e85de6ad16f8e4b0844a154557eca991 # via jaraco-context -certifi==2025.6.15 \ - --hash=sha256:2e0c7ce7cb5d8f8634ca55d2ba7e6ec2689a2fd6537d8dec1296a477a4910057 \ - --hash=sha256:d747aa5a8b9bbbb1bb8c22bb13e22bd1f18e9796defa16bab421f7f7a317323b +certifi==2025.7.14 \ + --hash=sha256:6b31f564a415d79ee77df69d757bb49a5bb53bd9f756cbbe24394ffd6fc1f4b2 \ + --hash=sha256:8ea99dbdfaaf2ba2f9bac77b9249ef62ec5218e7c2b2e903378ed5fccf765995 # via requests charset-normalizer==3.4.2 \ --hash=sha256:005fa3432484527f9732ebd315da8da8001593e2cf46a3d817669f062c3d9ed4 \ From 2c7187d6558ce39e1bd2fea4c0637722258f7790 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Tue, 15 Jul 2025 22:34:01 -0700 Subject: [PATCH 336/922] fix: support debian multiarch with local toolchains (#3100) Apparently, there is a "multiarch" style of Python installations, where the shared libraries can be in a sub-directory. The best docs I could find about this are https://wiki.debian.org/Python/MultiArch. To fix, looking at the `MULTIARCH` sysconfig var tells what subdirectory, if any should be looked in. Along the way, fix a changelog issue reference url. Fixes https://github.com/bazel-contrib/rules_python/issues/3099 --- CHANGELOG.md | 4 +++- python/private/get_local_runtime_info.py | 8 ++++++++ python/private/local_runtime_repo.bzl | 6 ++++++ 3 files changed, 17 insertions(+), 1 deletion(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 7248e1f9f2..b36ceafa27 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -85,10 +85,12 @@ END_UNRELEASED_TEMPLATE ([#2503](https://github.com/bazel-contrib/rules_python/issues/2503)). * (toolchains) `local_runtime_repo` now checks if the include directory exists before attempting to watch it, fixing issues on macOS with system Python - ({gh-issue}`3043`). + ([#3043](https://github.com/bazel-contrib/rules_python/issues/3043)). * (pypi) The pipstar `defaults` configuration now supports any custom platform name. * Multi-line python imports (e.g. with escaped newlines) are now correctly processed by Gazelle. +* (toolchains) `local_runtime_repo` works with multiarch Debian with Python 3.8 + ([#3099](https://github.com/bazel-contrib/rules_python/issues/3099)). {#v0-0-0-added} ### Added diff --git a/python/private/get_local_runtime_info.py b/python/private/get_local_runtime_info.py index 19db3a2935..c8371357c2 100644 --- a/python/private/get_local_runtime_info.py +++ b/python/private/get_local_runtime_info.py @@ -35,7 +35,15 @@ # of settings. # https://stackoverflow.com/questions/47423246/get-pythons-lib-path # For now, it seems LIBDIR has what is needed, so just use that. + # See also: MULTIARCH "LIBDIR", + # On Debian, with multiarch enabled, prior to Python 3.10, `LIBDIR` didn't + # tell the location of the libs, just the base directory. The `MULTIARCH` + # sysconfig variable tells the subdirectory within it with the libs. + # See: + # https://wiki.debian.org/Python/MultiArch + # https://git.launchpad.net/ubuntu/+source/python3.12/tree/debian/changelog#n842 + "MULTIARCH", # The versioned libpythonX.Y.so.N file. Usually? # It might be a static archive (.a) file instead. "INSTSONAME", diff --git a/python/private/local_runtime_repo.bzl b/python/private/local_runtime_repo.bzl index 3b4b4c020d..b8b7164b54 100644 --- a/python/private/local_runtime_repo.bzl +++ b/python/private/local_runtime_repo.bzl @@ -126,6 +126,7 @@ def _local_runtime_repo_impl(rctx): # In some cases, the same value is returned for multiple keys. Not clear why. shared_lib_names = {v: None for v in shared_lib_names}.keys() shared_lib_dir = info["LIBDIR"] + multiarch = info["MULTIARCH"] # The specific files are symlinked instead of the whole directory # because it can point to a directory that has more than just @@ -135,6 +136,11 @@ def _local_runtime_repo_impl(rctx): for name in shared_lib_names: origin = rctx.path("{}/{}".format(shared_lib_dir, name)) + # If the origin doesn't exist, try the multiarch location, in case + # it's an older Python / Debian release. + if not origin.exists and multiarch: + origin = rctx.path("{}/{}/{}".format(shared_lib_dir, multiarch, name)) + # The reported names don't always exist; it depends on the particulars # of the runtime installation. if origin.exists: From f02c9c72f5f0d63ce10da7b7393f58bc541e835a Mon Sep 17 00:00:00 2001 From: Alex Eagle Date: Thu, 17 Jul 2025 12:34:52 -0700 Subject: [PATCH 337/922] refactor(gazelle_manifest): print the wrong hash when encountered (#3103) Ideally developers should always re-run the manifest generator, which will update the hash for them. However I've got clients where the CI system is the only place that all the dependencies can resolve, either because of credentials needed to access the wheelhouse/PyPI, or because of disk exhaustion from massive wheels. Printing the difference allows a red PR to be greened up just by reading the CI log. --- gazelle/manifest/manifest.go | 3 +++ 1 file changed, 3 insertions(+) diff --git a/gazelle/manifest/manifest.go b/gazelle/manifest/manifest.go index 26b0dfb394..c5cd8a7d69 100644 --- a/gazelle/manifest/manifest.go +++ b/gazelle/manifest/manifest.go @@ -70,6 +70,9 @@ func (f *File) VerifyIntegrity(manifestGeneratorHashFile, requirements io.Reader return false, fmt.Errorf("failed to verify integrity: %w", err) } valid := (f.Integrity == fmt.Sprintf("%x", integrityBytes)) + if (!valid) { + fmt.Printf("WARN: Integrity hash was %v but expected %x\n", f.Integrity, integrityBytes) + } return valid, nil } From 004be45d2bf9924fc6805445d34885b0e1d6283d Mon Sep 17 00:00:00 2001 From: Charles OuGuo Date: Sun, 20 Jul 2025 16:54:06 -0400 Subject: [PATCH 338/922] feat(gazelle): `python_proto_naming_convention` directive controls `py_proto_library` naming (#3093) Closes https://github.com/bazel-contrib/rules_python/issues/3081. This adds support in the Gazelle plugin for controlling how the generated `py_proto_library` rules are named; support for these was originally added in https://github.com/bazel-contrib/rules_python/pull/3057. We do this via a new Gazelle directive, `python_proto_naming_convention`, which is similar to `python_library_naming_convention` and the like, except it interpolates `$proto_name$`, which is the `proto_library` rule minus any trailing `_proto`. We default to `$proto_name$_py_pb2`. For instance, for a `proto_library` named `foo_proto`, the default value would generate `foo_py_pb2`, aligning with [the convention stated in the Bazel docs.](https://bazel.build/reference/be/protocol-buffer#py_proto_library) --- CHANGELOG.md | 2 ++ gazelle/README.md | 27 +++++++++++++++++++ gazelle/python/configure.go | 3 +++ gazelle/python/generate.go | 17 +++++++++--- .../README.md | 9 +++++++ .../WORKSPACE | 1 + .../test.yaml | 3 +++ .../BUILD.in | 17 ++++++++++++ .../BUILD.out | 17 ++++++++++++ .../foo.proto | 7 +++++ .../BUILD.in | 9 +++++++ .../BUILD.out | 16 +++++++++++ .../foo.proto | 7 +++++ .../BUILD.in | 10 +++++++ .../BUILD.out | 17 ++++++++++++ .../foo.proto | 7 +++++ .../BUILD.in | 16 +++++++++++ .../BUILD.out | 17 ++++++++++++ .../foo.proto | 7 +++++ gazelle/pythonconfig/pythonconfig.go | 21 +++++++++++++++ 20 files changed, 227 insertions(+), 3 deletions(-) create mode 100644 gazelle/python/testdata/directive_python_proto_naming_convention/README.md create mode 100644 gazelle/python/testdata/directive_python_proto_naming_convention/WORKSPACE create mode 100644 gazelle/python/testdata/directive_python_proto_naming_convention/test.yaml create mode 100644 gazelle/python/testdata/directive_python_proto_naming_convention/test1_python_generation_disabled_does_nothing/BUILD.in create mode 100644 gazelle/python/testdata/directive_python_proto_naming_convention/test1_python_generation_disabled_does_nothing/BUILD.out create mode 100644 gazelle/python/testdata/directive_python_proto_naming_convention/test1_python_generation_disabled_does_nothing/foo.proto create mode 100644 gazelle/python/testdata/directive_python_proto_naming_convention/test2_python_generation_enabled_uses_default/BUILD.in create mode 100644 gazelle/python/testdata/directive_python_proto_naming_convention/test2_python_generation_enabled_uses_default/BUILD.out create mode 100644 gazelle/python/testdata/directive_python_proto_naming_convention/test2_python_generation_enabled_uses_default/foo.proto create mode 100644 gazelle/python/testdata/directive_python_proto_naming_convention/test3_python_generation_enabled_uses_value/BUILD.in create mode 100644 gazelle/python/testdata/directive_python_proto_naming_convention/test3_python_generation_enabled_uses_value/BUILD.out create mode 100644 gazelle/python/testdata/directive_python_proto_naming_convention/test3_python_generation_enabled_uses_value/foo.proto create mode 100644 gazelle/python/testdata/directive_python_proto_naming_convention/test4_python_generation_enabled_with_preexisting_keeps_intact/BUILD.in create mode 100644 gazelle/python/testdata/directive_python_proto_naming_convention/test4_python_generation_enabled_with_preexisting_keeps_intact/BUILD.out create mode 100644 gazelle/python/testdata/directive_python_proto_naming_convention/test4_python_generation_enabled_with_preexisting_keeps_intact/foo.proto diff --git a/CHANGELOG.md b/CHANGELOG.md index b36ceafa27..d54d3a8881 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -116,6 +116,8 @@ END_UNRELEASED_TEMPLATE dep is not added to the {obj}`py_test` target. * (gazelle) New directive `gazelle:python_generate_proto`; when `true`, Gazelle generates `py_proto_library` rules for `proto_library`. `false` by default. +* (gazelle) New directive `gazelle:python_proto_naming_convention`; controls + naming of `py_proto_library` rules. {#v0-0-0-removed} ### Removed diff --git a/gazelle/README.md b/gazelle/README.md index cf91461e39..222c1171ab 100644 --- a/gazelle/README.md +++ b/gazelle/README.md @@ -208,6 +208,8 @@ Python-specific directives are as follows: | Controls the `py_binary` naming convention. Follows the same interpolation rules as `python_library_naming_convention`. | | | `# gazelle:python_test_naming_convention` | `$package_name$_test` | | Controls the `py_test` naming convention. Follows the same interpolation rules as `python_library_naming_convention`. | | +| [`# gazelle:python_proto_naming_convention`](#directive-python_proto_naming_convention) | `$proto_name$_py_pb2` | +| Controls the `py_proto_library` naming convention. It interpolates `$proto_name$` with the proto_library rule name, minus any trailing _proto. E.g. if the proto_library name is `foo_proto`, setting this to `$proto_name$_my_lib` would render to `foo_my_lib`. | | | `# gazelle:resolve py ...` | n/a | | Instructs the plugin what target to add as a dependency to satisfy a given import statement. The syntax is `# gazelle:resolve py import-string label` where `import-string` is the symbol in the python `import` statement, and `label` is the Bazel label that Gazelle should write in `deps`. | | | [`# gazelle:python_default_visibility labels`](#directive-python_default_visibility) | | @@ -262,6 +264,31 @@ py_libary( [python-packaging-user-guide]: https://github.com/pypa/packaging.python.org/blob/4c86169a/source/tutorials/packaging-projects.rst +#### Directive: `python_proto_naming_convention`: + +Set this directive to a string pattern to control how the generated `py_proto_library` targets are named. When generating new `py_proto_library` rules, Gazelle will replace `$proto_name$` in the pattern with the name of the `proto_library` rule, stripping out a trailing `_proto`. For example: + +```starlark +# gazelle:python_generate_proto true +# gazelle:python_proto_naming_convention my_custom_$proto_name$_pattern + +proto_library( + name = "foo_proto", + srcs = ["foo.proto"], +) +``` + +produces the following `py_proto_library` rule: +```starlark +py_proto_library( + name = "my_custom_foo_pattern", + deps = [":foo_proto"], +) +``` + +The default naming convention is `$proto_name$_pb2_py`, so by default in the above example Gazelle would generate `foo_pb2_py`. Any pre-existing rules are left in place and not renamed. + +Note that the Python library will always be imported as `foo_pb2` in Python code, regardless of the naming convention. Also note that Gazelle is currently not able to map said imports, e.g. `import foo_pb2`, to fill in `py_proto_library` targets as dependencies of other rules. See [this issue](https://github.com/bazel-contrib/rules_python/issues/1703). #### Directive: `python_default_visibility`: diff --git a/gazelle/python/configure.go b/gazelle/python/configure.go index 7131be283d..079f1d84d4 100644 --- a/gazelle/python/configure.go +++ b/gazelle/python/configure.go @@ -63,6 +63,7 @@ func (py *Configurer) KnownDirectives() []string { pythonconfig.LibraryNamingConvention, pythonconfig.BinaryNamingConvention, pythonconfig.TestNamingConvention, + pythonconfig.ProtoNamingConvention, pythonconfig.DefaultVisibilty, pythonconfig.Visibility, pythonconfig.TestFilePattern, @@ -179,6 +180,8 @@ func (py *Configurer) Configure(c *config.Config, rel string, f *rule.File) { config.SetBinaryNamingConvention(strings.TrimSpace(d.Value)) case pythonconfig.TestNamingConvention: config.SetTestNamingConvention(strings.TrimSpace(d.Value)) + case pythonconfig.ProtoNamingConvention: + config.SetProtoNamingConvention(strings.TrimSpace(d.Value)) case pythonconfig.DefaultVisibilty: switch directiveArg := strings.TrimSpace(d.Value); directiveArg { case "NONE": diff --git a/gazelle/python/generate.go b/gazelle/python/generate.go index 279bee6af7..5b6ba79d69 100644 --- a/gazelle/python/generate.go +++ b/gazelle/python/generate.go @@ -227,7 +227,7 @@ func (py *Python) GenerateRules(args language.GenerateArgs) language.GenerateRes result.Gen = make([]*rule.Rule, 0) if cfg.GenerateProto() { - generateProtoLibraries(args, pythonProjectRoot, visibility, &result) + generateProtoLibraries(args, cfg, pythonProjectRoot, visibility, &result) } collisionErrors := singlylinkedlist.New() @@ -569,7 +569,7 @@ func ensureNoCollision(file *rule.File, targetName, kind string) error { return nil } -func generateProtoLibraries(args language.GenerateArgs, pythonProjectRoot string, visibility []string, res *language.GenerateResult) { +func generateProtoLibraries(args language.GenerateArgs, cfg *pythonconfig.Config, pythonProjectRoot string, visibility []string, res *language.GenerateResult) { // First, enumerate all the proto_library in this package. var protoRuleNames []string for _, r := range args.OtherGen { @@ -582,10 +582,16 @@ func generateProtoLibraries(args language.GenerateArgs, pythonProjectRoot string // Next, enumerate all the pre-existing py_proto_library in this package, so we can delete unnecessary rules later. pyProtoRules := map[string]bool{} + pyProtoRulesForProto := map[string]string{} if args.File != nil { for _, r := range args.File.Rules { if r.Kind() == "py_proto_library" { pyProtoRules[r.Name()] = false + + protos := r.AttrStrings("deps") + for _, proto := range protos { + pyProtoRulesForProto[strings.TrimPrefix(proto, ":")] = r.Name() + } } } } @@ -593,7 +599,12 @@ func generateProtoLibraries(args language.GenerateArgs, pythonProjectRoot string emptySiblings := treeset.Set{} // Generate a py_proto_library for each proto_library. for _, protoRuleName := range protoRuleNames { - pyProtoLibraryName := strings.TrimSuffix(protoRuleName, "_proto") + "_py_pb2" + pyProtoLibraryName := cfg.RenderProtoName(protoRuleName) + if ruleName, ok := pyProtoRulesForProto[protoRuleName]; ok { + // There exists a pre-existing py_proto_library for this proto. Keep this name. + pyProtoLibraryName = ruleName + } + pyProtoLibrary := newTargetBuilder(pyProtoLibraryKind, pyProtoLibraryName, pythonProjectRoot, args.Rel, &emptySiblings). addVisibility(visibility). addResolvedDependency(":" + protoRuleName). diff --git a/gazelle/python/testdata/directive_python_proto_naming_convention/README.md b/gazelle/python/testdata/directive_python_proto_naming_convention/README.md new file mode 100644 index 0000000000..594379cdfc --- /dev/null +++ b/gazelle/python/testdata/directive_python_proto_naming_convention/README.md @@ -0,0 +1,9 @@ +# Directive: `python_proto_naming_convention` + +This test case asserts that the `# gazelle:python_proto_naming_convention` directive +correctly: + +1. Has no effect on pre-existing `py_proto_library` when `gazelle:python_generate_proto` is disabled. +2. Uses the default value when proto generation is on and `python_proto_naming_convention` is not set. +3. Uses the provided naming convention when proto generation is on and `python_proto_naming_convention` is set. +4. With a pre-existing `py_proto_library` not following a given naming convention, keeps it intact and does not rename it. \ No newline at end of file diff --git a/gazelle/python/testdata/directive_python_proto_naming_convention/WORKSPACE b/gazelle/python/testdata/directive_python_proto_naming_convention/WORKSPACE new file mode 100644 index 0000000000..faff6af87a --- /dev/null +++ b/gazelle/python/testdata/directive_python_proto_naming_convention/WORKSPACE @@ -0,0 +1 @@ +# This is a Bazel workspace for the Gazelle test data. diff --git a/gazelle/python/testdata/directive_python_proto_naming_convention/test.yaml b/gazelle/python/testdata/directive_python_proto_naming_convention/test.yaml new file mode 100644 index 0000000000..36dd656b39 --- /dev/null +++ b/gazelle/python/testdata/directive_python_proto_naming_convention/test.yaml @@ -0,0 +1,3 @@ +--- +expect: + exit_code: 0 diff --git a/gazelle/python/testdata/directive_python_proto_naming_convention/test1_python_generation_disabled_does_nothing/BUILD.in b/gazelle/python/testdata/directive_python_proto_naming_convention/test1_python_generation_disabled_does_nothing/BUILD.in new file mode 100644 index 0000000000..2171d877f4 --- /dev/null +++ b/gazelle/python/testdata/directive_python_proto_naming_convention/test1_python_generation_disabled_does_nothing/BUILD.in @@ -0,0 +1,17 @@ +load("@com_google_protobuf//bazel:py_proto_library.bzl", "py_proto_library") +load("@rules_proto//proto:defs.bzl", "proto_library") + +# gazelle:python_generate_proto false +# gazelle:python_proto_naming_convention some_$proto_name$_value + +proto_library( + name = "foo_proto", + srcs = ["foo.proto"], + visibility = ["//:__subpackages__"], +) + +py_proto_library( + name = "foo_proto_custom_name", + visibility = ["//:__subpackages__"], + deps = [":foo_proto"], +) diff --git a/gazelle/python/testdata/directive_python_proto_naming_convention/test1_python_generation_disabled_does_nothing/BUILD.out b/gazelle/python/testdata/directive_python_proto_naming_convention/test1_python_generation_disabled_does_nothing/BUILD.out new file mode 100644 index 0000000000..2171d877f4 --- /dev/null +++ b/gazelle/python/testdata/directive_python_proto_naming_convention/test1_python_generation_disabled_does_nothing/BUILD.out @@ -0,0 +1,17 @@ +load("@com_google_protobuf//bazel:py_proto_library.bzl", "py_proto_library") +load("@rules_proto//proto:defs.bzl", "proto_library") + +# gazelle:python_generate_proto false +# gazelle:python_proto_naming_convention some_$proto_name$_value + +proto_library( + name = "foo_proto", + srcs = ["foo.proto"], + visibility = ["//:__subpackages__"], +) + +py_proto_library( + name = "foo_proto_custom_name", + visibility = ["//:__subpackages__"], + deps = [":foo_proto"], +) diff --git a/gazelle/python/testdata/directive_python_proto_naming_convention/test1_python_generation_disabled_does_nothing/foo.proto b/gazelle/python/testdata/directive_python_proto_naming_convention/test1_python_generation_disabled_does_nothing/foo.proto new file mode 100644 index 0000000000..022e29ae69 --- /dev/null +++ b/gazelle/python/testdata/directive_python_proto_naming_convention/test1_python_generation_disabled_does_nothing/foo.proto @@ -0,0 +1,7 @@ +syntax = "proto3"; + +package foo.bar; + +message Foo { + string bar = 1; +} diff --git a/gazelle/python/testdata/directive_python_proto_naming_convention/test2_python_generation_enabled_uses_default/BUILD.in b/gazelle/python/testdata/directive_python_proto_naming_convention/test2_python_generation_enabled_uses_default/BUILD.in new file mode 100644 index 0000000000..4713404b19 --- /dev/null +++ b/gazelle/python/testdata/directive_python_proto_naming_convention/test2_python_generation_enabled_uses_default/BUILD.in @@ -0,0 +1,9 @@ +load("@rules_proto//proto:defs.bzl", "proto_library") + +# gazelle:python_generate_proto true + +proto_library( + name = "foo_proto", + srcs = ["foo.proto"], + visibility = ["//:__subpackages__"], +) diff --git a/gazelle/python/testdata/directive_python_proto_naming_convention/test2_python_generation_enabled_uses_default/BUILD.out b/gazelle/python/testdata/directive_python_proto_naming_convention/test2_python_generation_enabled_uses_default/BUILD.out new file mode 100644 index 0000000000..686252f27c --- /dev/null +++ b/gazelle/python/testdata/directive_python_proto_naming_convention/test2_python_generation_enabled_uses_default/BUILD.out @@ -0,0 +1,16 @@ +load("@com_google_protobuf//bazel:py_proto_library.bzl", "py_proto_library") +load("@rules_proto//proto:defs.bzl", "proto_library") + +# gazelle:python_generate_proto true + +proto_library( + name = "foo_proto", + srcs = ["foo.proto"], + visibility = ["//:__subpackages__"], +) + +py_proto_library( + name = "foo_py_pb2", + visibility = ["//:__subpackages__"], + deps = [":foo_proto"], +) diff --git a/gazelle/python/testdata/directive_python_proto_naming_convention/test2_python_generation_enabled_uses_default/foo.proto b/gazelle/python/testdata/directive_python_proto_naming_convention/test2_python_generation_enabled_uses_default/foo.proto new file mode 100644 index 0000000000..fe2af27aa6 --- /dev/null +++ b/gazelle/python/testdata/directive_python_proto_naming_convention/test2_python_generation_enabled_uses_default/foo.proto @@ -0,0 +1,7 @@ +syntax = "proto3"; + +package foo; + +message Foo { + string bar = 1; +} diff --git a/gazelle/python/testdata/directive_python_proto_naming_convention/test3_python_generation_enabled_uses_value/BUILD.in b/gazelle/python/testdata/directive_python_proto_naming_convention/test3_python_generation_enabled_uses_value/BUILD.in new file mode 100644 index 0000000000..b68a9937dc --- /dev/null +++ b/gazelle/python/testdata/directive_python_proto_naming_convention/test3_python_generation_enabled_uses_value/BUILD.in @@ -0,0 +1,10 @@ +load("@rules_proto//proto:defs.bzl", "proto_library") + +# gazelle:python_generate_proto true +# gazelle:python_proto_naming_convention some_$proto_name$_value + +proto_library( + name = "foo_proto", + srcs = ["foo.proto"], + visibility = ["//:__subpackages__"], +) diff --git a/gazelle/python/testdata/directive_python_proto_naming_convention/test3_python_generation_enabled_uses_value/BUILD.out b/gazelle/python/testdata/directive_python_proto_naming_convention/test3_python_generation_enabled_uses_value/BUILD.out new file mode 100644 index 0000000000..f432e9a0c3 --- /dev/null +++ b/gazelle/python/testdata/directive_python_proto_naming_convention/test3_python_generation_enabled_uses_value/BUILD.out @@ -0,0 +1,17 @@ +load("@com_google_protobuf//bazel:py_proto_library.bzl", "py_proto_library") +load("@rules_proto//proto:defs.bzl", "proto_library") + +# gazelle:python_generate_proto true +# gazelle:python_proto_naming_convention some_$proto_name$_value + +proto_library( + name = "foo_proto", + srcs = ["foo.proto"], + visibility = ["//:__subpackages__"], +) + +py_proto_library( + name = "some_foo_value", + visibility = ["//:__subpackages__"], + deps = [":foo_proto"], +) diff --git a/gazelle/python/testdata/directive_python_proto_naming_convention/test3_python_generation_enabled_uses_value/foo.proto b/gazelle/python/testdata/directive_python_proto_naming_convention/test3_python_generation_enabled_uses_value/foo.proto new file mode 100644 index 0000000000..fe2af27aa6 --- /dev/null +++ b/gazelle/python/testdata/directive_python_proto_naming_convention/test3_python_generation_enabled_uses_value/foo.proto @@ -0,0 +1,7 @@ +syntax = "proto3"; + +package foo; + +message Foo { + string bar = 1; +} diff --git a/gazelle/python/testdata/directive_python_proto_naming_convention/test4_python_generation_enabled_with_preexisting_keeps_intact/BUILD.in b/gazelle/python/testdata/directive_python_proto_naming_convention/test4_python_generation_enabled_with_preexisting_keeps_intact/BUILD.in new file mode 100644 index 0000000000..cc7d120a7e --- /dev/null +++ b/gazelle/python/testdata/directive_python_proto_naming_convention/test4_python_generation_enabled_with_preexisting_keeps_intact/BUILD.in @@ -0,0 +1,16 @@ +load("@rules_proto//proto:defs.bzl", "proto_library") + +# gazelle:python_generate_proto true +# gazelle:python_proto_naming_convention $proto_name$_bar + +proto_library( + name = "foo_proto", + srcs = ["foo.proto"], + visibility = ["//:__subpackages__"], +) + +py_proto_library( + name = "foo_py_proto", + visibility = ["//:__subpackages__"], + deps = [":foo_proto"], +) diff --git a/gazelle/python/testdata/directive_python_proto_naming_convention/test4_python_generation_enabled_with_preexisting_keeps_intact/BUILD.out b/gazelle/python/testdata/directive_python_proto_naming_convention/test4_python_generation_enabled_with_preexisting_keeps_intact/BUILD.out new file mode 100644 index 0000000000..080b83f1fb --- /dev/null +++ b/gazelle/python/testdata/directive_python_proto_naming_convention/test4_python_generation_enabled_with_preexisting_keeps_intact/BUILD.out @@ -0,0 +1,17 @@ +load("@com_google_protobuf//bazel:py_proto_library.bzl", "py_proto_library") +load("@rules_proto//proto:defs.bzl", "proto_library") + +# gazelle:python_generate_proto true +# gazelle:python_proto_naming_convention $proto_name$_bar + +proto_library( + name = "foo_proto", + srcs = ["foo.proto"], + visibility = ["//:__subpackages__"], +) + +py_proto_library( + name = "foo_py_proto", + visibility = ["//:__subpackages__"], + deps = [":foo_proto"], +) diff --git a/gazelle/python/testdata/directive_python_proto_naming_convention/test4_python_generation_enabled_with_preexisting_keeps_intact/foo.proto b/gazelle/python/testdata/directive_python_proto_naming_convention/test4_python_generation_enabled_with_preexisting_keeps_intact/foo.proto new file mode 100644 index 0000000000..fe2af27aa6 --- /dev/null +++ b/gazelle/python/testdata/directive_python_proto_naming_convention/test4_python_generation_enabled_with_preexisting_keeps_intact/foo.proto @@ -0,0 +1,7 @@ +syntax = "proto3"; + +package foo; + +message Foo { + string bar = 1; +} diff --git a/gazelle/pythonconfig/pythonconfig.go b/gazelle/pythonconfig/pythonconfig.go index b76e1f92ec..001fd334a4 100644 --- a/gazelle/pythonconfig/pythonconfig.go +++ b/gazelle/pythonconfig/pythonconfig.go @@ -74,6 +74,12 @@ const ( // naming convention. See python_library_naming_convention for more info on // the package name interpolation. TestNamingConvention = "python_test_naming_convention" + // ProtoNamingConvention represents the directive that controls the + // py_proto_library naming convention. It interpolates $proto_name$ with + // the proto_library rule name, minus any trailing _proto. E.g. if the + // proto_library name is `foo_proto`, setting this to `$proto_name$_my_lib` + // would render to `foo_my_lib`. + ProtoNamingConvention = "python_proto_naming_convention" // DefaultVisibilty represents the directive that controls what visibility // labels are added to generated python targets. DefaultVisibilty = "python_default_visibility" @@ -121,6 +127,7 @@ const ( const ( packageNameNamingConventionSubstitution = "$package_name$" + protoNameNamingConventionSubstitution = "$proto_name$" distributionNameLabelConventionSubstitution = "$distribution_name$" ) @@ -182,6 +189,7 @@ type Config struct { libraryNamingConvention string binaryNamingConvention string testNamingConvention string + protoNamingConvention string defaultVisibility []string visibility []string testFilePattern []string @@ -220,6 +228,7 @@ func New( libraryNamingConvention: packageNameNamingConventionSubstitution, binaryNamingConvention: fmt.Sprintf("%s_bin", packageNameNamingConventionSubstitution), testNamingConvention: fmt.Sprintf("%s_test", packageNameNamingConventionSubstitution), + protoNamingConvention: fmt.Sprintf("%s_py_pb2", protoNameNamingConventionSubstitution), defaultVisibility: []string{fmt.Sprintf(DefaultVisibilityFmtString, "")}, visibility: []string{}, testFilePattern: strings.Split(DefaultTestFilePatternString, ","), @@ -255,6 +264,7 @@ func (c *Config) NewChild() *Config { libraryNamingConvention: c.libraryNamingConvention, binaryNamingConvention: c.binaryNamingConvention, testNamingConvention: c.testNamingConvention, + protoNamingConvention: c.protoNamingConvention, defaultVisibility: c.defaultVisibility, visibility: c.visibility, testFilePattern: c.testFilePattern, @@ -489,6 +499,17 @@ func (c *Config) RenderTestName(packageName string) string { return strings.ReplaceAll(c.testNamingConvention, packageNameNamingConventionSubstitution, packageName) } +// SetProtoNamingConvention sets the py_proto_library target naming convention. +func (c *Config) SetProtoNamingConvention(protoNamingConvention string) { + c.protoNamingConvention = protoNamingConvention +} + +// RenderProtoName returns the py_proto_library target name by performing all +// substitutions. +func (c *Config) RenderProtoName(protoName string) string { + return strings.ReplaceAll(c.protoNamingConvention, protoNameNamingConventionSubstitution, strings.TrimSuffix(protoName, "_proto")) +} + // AppendVisibility adds additional items to the target's visibility. func (c *Config) AppendVisibility(visibility string) { c.visibility = append(c.visibility, visibility) From aa602298a8bc2945ebf42d0820ec30bc974c9216 Mon Sep 17 00:00:00 2001 From: Jaemin Choi <1dotolee@gmail.com> Date: Mon, 21 Jul 2025 23:01:30 +0900 Subject: [PATCH 339/922] fix(pypi): expose pypi packages only common to all python versions (#3107) Closes #2921 For a single pypi repo with multiple python versions, `all_requirements` fail when a pypi package supports Python version A but not version B. In this case, the pypi package would be included only in requirements lock file for version A, not in one for version B. However, the failure occurs since the package is included in `all_requirements` even for Python version B. (Minimal reproduction: https://github.com/dotoleeoak/rules-python-2921-repro) This happens since `packages` parameter for `hub_repository` targets are including all packages across all requirement lock files. Instead of union of packages, intersection of packages for requirement files should be passed to `packages` and exposed to `all_requirements` macro, so that those packages are compatible with all Python versions. --------- Co-authored-by: Ignas Anikevicius <240938+aignas@users.noreply.github.com> --- CHANGELOG.md | 2 + python/private/pypi/extension.bzl | 10 ++- tests/pypi/extension/extension_tests.bzl | 98 ++++++++++++++++++++++++ 3 files changed, 109 insertions(+), 1 deletion(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index d54d3a8881..74a4409cbb 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -91,6 +91,8 @@ END_UNRELEASED_TEMPLATE * Multi-line python imports (e.g. with escaped newlines) are now correctly processed by Gazelle. * (toolchains) `local_runtime_repo` works with multiarch Debian with Python 3.8 ([#3099](https://github.com/bazel-contrib/rules_python/issues/3099)). +* (pypi) Expose pypi packages only common to all Python versions in `all_requirements` + ([#2921](https://github.com/bazel-contrib/rules_python/issues/2921)). {#v0-0-0-added} ### Added diff --git a/python/private/pypi/extension.bzl b/python/private/pypi/extension.bzl index 2c1528e18d..096256e4be 100644 --- a/python/private/pypi/extension.bzl +++ b/python/private/pypi/extension.bzl @@ -601,7 +601,15 @@ You cannot use both the additive_build_content and additive_build_content_file a extra_aliases.setdefault(hub_name, {}) for whl_name, aliases in out.extra_aliases.items(): extra_aliases[hub_name].setdefault(whl_name, {}).update(aliases) - exposed_packages.setdefault(hub_name, {}).update(out.exposed_packages) + if hub_name not in exposed_packages: + exposed_packages[hub_name] = out.exposed_packages + else: + intersection = {} + for pkg in out.exposed_packages: + if pkg not in exposed_packages[hub_name]: + continue + intersection[pkg] = None + exposed_packages[hub_name] = intersection whl_libraries.update(out.whl_libraries) # TODO @aignas 2024-04-05: how do we support different requirement diff --git a/tests/pypi/extension/extension_tests.bzl b/tests/pypi/extension/extension_tests.bzl index 0303843e80..52e0e29cb0 100644 --- a/tests/pypi/extension/extension_tests.bzl +++ b/tests/pypi/extension/extension_tests.bzl @@ -285,6 +285,104 @@ def _test_simple_multiple_requirements(env): _tests.append(_test_simple_multiple_requirements) +def _test_simple_multiple_python_versions(env): + pypi = _parse_modules( + env, + module_ctx = _mock_mctx( + _mod( + name = "rules_python", + parse = [ + _parse( + hub_name = "pypi", + python_version = "3.15", + requirements_lock = "requirements_3_15.txt", + ), + _parse( + hub_name = "pypi", + python_version = "3.16", + requirements_lock = "requirements_3_16.txt", + ), + ], + ), + read = lambda x: { + "requirements_3_15.txt": """ +simple==0.0.1 --hash=sha256:deadbeef +old-package==0.0.1 --hash=sha256:deadbaaf +""", + "requirements_3_16.txt": """ +simple==0.0.2 --hash=sha256:deadb00f +new-package==0.0.1 --hash=sha256:deadb00f2 +""", + }[x], + ), + available_interpreters = { + "python_3_15_host": "unit_test_interpreter_target", + "python_3_16_host": "unit_test_interpreter_target", + }, + minor_mapping = { + "3.15": "3.15.19", + "3.16": "3.16.9", + }, + ) + + pypi.exposed_packages().contains_exactly({"pypi": ["simple"]}) + pypi.hub_group_map().contains_exactly({"pypi": {}}) + pypi.hub_whl_map().contains_exactly({ + "pypi": { + "new_package": { + "pypi_316_new_package": [ + whl_config_setting( + version = "3.16", + ), + ], + }, + "old_package": { + "pypi_315_old_package": [ + whl_config_setting( + version = "3.15", + ), + ], + }, + "simple": { + "pypi_315_simple": [ + whl_config_setting( + version = "3.15", + ), + ], + "pypi_316_simple": [ + whl_config_setting( + version = "3.16", + ), + ], + }, + }, + }) + pypi.whl_libraries().contains_exactly({ + "pypi_315_old_package": { + "dep_template": "@pypi//{name}:{target}", + "python_interpreter_target": "unit_test_interpreter_target", + "requirement": "old-package==0.0.1 --hash=sha256:deadbaaf", + }, + "pypi_315_simple": { + "dep_template": "@pypi//{name}:{target}", + "python_interpreter_target": "unit_test_interpreter_target", + "requirement": "simple==0.0.1 --hash=sha256:deadbeef", + }, + "pypi_316_new_package": { + "dep_template": "@pypi//{name}:{target}", + "python_interpreter_target": "unit_test_interpreter_target", + "requirement": "new-package==0.0.1 --hash=sha256:deadb00f2", + }, + "pypi_316_simple": { + "dep_template": "@pypi//{name}:{target}", + "python_interpreter_target": "unit_test_interpreter_target", + "requirement": "simple==0.0.2 --hash=sha256:deadb00f", + }, + }) + pypi.whl_mods().contains_exactly({}) + +_tests.append(_test_simple_multiple_python_versions) + def _test_simple_with_markers(env): pypi = _parse_modules( env, From 5281261a97235099a65aaeb83b74b30d16409798 Mon Sep 17 00:00:00 2001 From: Jonathan Woodbury Date: Mon, 21 Jul 2025 20:33:25 -0400 Subject: [PATCH 340/922] fix: normalize stub_path in repl.bzl (#3104) When a REPL target is run from an external Bazel module, the `stub_path` can have path components in it (e.g. "/..") which get rejected by the `Rlocation()` function in `runfiles.py` for not being normalized. This commit normalizes the path before it's passed to `Rlocation()`. Fixes #3101 --------- Co-authored-by: Ignas Anikevicius <240938+aignas@users.noreply.github.com> --- CHANGELOG.md | 2 ++ python/private/repl_template.py | 4 +++- 2 files changed, 5 insertions(+), 1 deletion(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 74a4409cbb..5ad48bee3f 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -93,6 +93,8 @@ END_UNRELEASED_TEMPLATE ([#3099](https://github.com/bazel-contrib/rules_python/issues/3099)). * (pypi) Expose pypi packages only common to all Python versions in `all_requirements` ([#2921](https://github.com/bazel-contrib/rules_python/issues/2921)). +* (repl) Normalize the path for the `REPL` stub to make it possible to use the + default stub template from outside `rules_python` ({gh-issue}`3101`). {#v0-0-0-added} ### Added diff --git a/python/private/repl_template.py b/python/private/repl_template.py index 37f4529fbe..dd8beb9784 100644 --- a/python/private/repl_template.py +++ b/python/private/repl_template.py @@ -5,7 +5,9 @@ from python.runfiles import runfiles -STUB_PATH = "%stub_path%" +# runfiles.py will reject paths which aren't normalized, which can happen when the REPL rules are +# used from a remote module. +STUB_PATH = os.path.normpath("%stub_path%") def start_repl(): From e6bba92d4a3de943c77a50834c727318870bef48 Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Wed, 23 Jul 2025 13:34:19 +0900 Subject: [PATCH 341/922] build(deps): bump typing-extensions from 4.13.2 to 4.14.1 in /docs (#3094) [![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=typing-extensions&package-manager=pip&previous-version=4.13.2&new-version=4.14.1)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) ---
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot show ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> --- docs/requirements.txt | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/requirements.txt b/docs/requirements.txt index 9e46724770..cb8900bf95 100644 --- a/docs/requirements.txt +++ b/docs/requirements.txt @@ -350,9 +350,9 @@ sphinxcontrib-serializinghtml==2.0.0 \ --hash=sha256:6e2cb0eef194e10c27ec0023bfeb25badbbb5868244cf5bc5bdc04e4464bf331 \ --hash=sha256:e9d912827f872c029017a53f0ef2180b327c3f7fd23c87229f7a8e8b70031d4d # via sphinx -typing-extensions==4.13.2 \ - --hash=sha256:a439e7c04b49fec3e5d3e2beaa21755cadbbdc391694e28ccdd36ca4a1408f8c \ - --hash=sha256:e6c81219bd689f51865d9e372991c540bda33a0379d5573cddb9a3a23f7caaef +typing-extensions==4.14.1 \ + --hash=sha256:38b39f4aeeab64884ce9f74c94263ef78f3c22467c8724005483154c26648d36 \ + --hash=sha256:d1e1e3b58374dc93031d6eda2420a48ea44a36c2b4766a4fdeb3710755731d76 # via # rules-python-docs (docs/pyproject.toml) # sphinx-autodoc2 From ab3e3f790788e119bfb362a60819727d7450f0f3 Mon Sep 17 00:00:00 2001 From: Alex Martani Date: Mon, 28 Jul 2025 21:29:18 -0700 Subject: [PATCH 342/922] fix(gazelle): Do not resolve absolute imports to sibling modules (#3106) Currently, gazelle allows absolute imports to be resolved to sibling modules: an `import foo` statement will resolve to a `foo.py` file in the same folder if such file exists. This seems to be a Python 2 behavior (ie. pre-`from __future__ import absolute_import`), and doesn't work on the current rules_python setup. This behavior is explicitly tested in the [siblings_import](https://github.com/bazel-contrib/rules_python/tree/cbe6d38d01c14de46d90ea717d0f2090117533fa/gazelle/python/testdata/sibling_imports) test case. However, recreating the exact same repository layout from this test case and running `bazel test //pkg:unit_test`, the test fails with the import failing. This PR adds a new directive, `gazelle:python_resolve_sibling_imports`, to allow disabling such behavior. The actual changes are in 3 places: - In `gazelle/python/target.go`, the directive is added to `if t.siblingSrcs.Contains(fileName) && fileName != filepath.Base(dep.Filepath)`, which is where the import is converted to a full absolute import if it matches a sibling file; - In `gazelle/python/generate.go`, the handling of `conftest.py` was dependent on this behavior (ie. it added a dependency on the module `conftest`, assuming that it would be resolved to the relative module). That was modified to compute the full absolute module path instead. - In `gazelle/python/resolve.go`, resolve relative imports even when using file generation mode. I also explicitly added `gazelle:python_resolve_sibling_imports true` to any test that breaks if the default value of this directive is changed to `false`. --- CHANGELOG.md | 4 ++ gazelle/README.md | 2 + gazelle/python/configure.go | 7 +++ gazelle/python/generate.go | 16 +++--- gazelle/python/resolve.go | 10 ++-- gazelle/python/target.go | 53 ++++++++++--------- .../testdata/annotation_include_dep/BUILD.in | 1 + .../testdata/annotation_include_dep/BUILD.out | 1 + .../with_conftest/BUILD.in | 1 + .../with_conftest/BUILD.out | 2 + .../testdata/naming_convention/BUILD.in | 1 + .../testdata/naming_convention/BUILD.out | 1 + .../python/testdata/sibling_imports/README.md | 12 ++++- .../testdata/sibling_imports/pkg/BUILD.in | 1 + .../testdata/sibling_imports/pkg/BUILD.out | 3 +- .../sibling_imports_disabled/BUILD.in | 2 + .../sibling_imports_disabled/BUILD.out | 18 +++++++ .../sibling_imports_disabled/README.md | 22 ++++++++ .../sibling_imports_disabled/WORKSPACE | 1 + .../testdata/sibling_imports_disabled/a.py | 1 + .../testdata/sibling_imports_disabled/b.py | 3 ++ .../sibling_imports_disabled/pkg/BUILD.in | 0 .../sibling_imports_disabled/pkg/BUILD.out | 27 ++++++++++ .../sibling_imports_disabled/pkg/__init__.py | 0 .../sibling_imports_disabled/pkg/a.py | 0 .../sibling_imports_disabled/pkg/b.py | 2 + .../sibling_imports_disabled/pkg/test_util.py | 2 + .../sibling_imports_disabled/pkg/typing.py | 1 + .../sibling_imports_disabled/pkg/unit_test.py | 5 ++ .../sibling_imports_disabled/test.yaml | 1 + .../sibling_imports_disabled/test_util.py | 1 + .../BUILD.in | 3 ++ .../BUILD.out | 22 ++++++++ .../README.md | 22 ++++++++ .../WORKSPACE | 1 + .../sibling_imports_disabled_file_mode/a.py | 1 + .../sibling_imports_disabled_file_mode/b.py | 3 ++ .../pkg/BUILD.in | 0 .../pkg/BUILD.out | 38 +++++++++++++ .../pkg/__init__.py | 0 .../pkg/a.py | 0 .../pkg/b.py | 2 + .../pkg/test_util.py | 2 + .../pkg/typing.py | 1 + .../pkg/unit_test.py | 5 ++ .../test.yaml | 1 + .../test_util.py | 1 + .../simple_test_with_conftest/BUILD.in | 2 + .../simple_test_with_conftest/BUILD.out | 2 + .../BUILD.in | 3 ++ .../BUILD.out | 29 ++++++++++ .../README.md | 4 ++ .../WORKSPACE | 1 + .../__init__.py | 3 ++ .../__test__.py | 12 +++++ .../bar/BUILD.in | 1 + .../bar/BUILD.out | 27 ++++++++++ .../bar/__init__.py | 3 ++ .../bar/__test__.py | 12 +++++ .../bar/bar.py | 2 + .../bar/conftest.py | 0 .../conftest.py | 0 .../foo.py | 2 + .../test.yaml | 4 ++ .../python/testdata/subdir_sources/BUILD.in | 1 + .../python/testdata/subdir_sources/BUILD.out | 1 + gazelle/pythonconfig/pythonconfig.go | 18 +++++++ 67 files changed, 388 insertions(+), 42 deletions(-) create mode 100644 gazelle/python/testdata/sibling_imports_disabled/BUILD.in create mode 100644 gazelle/python/testdata/sibling_imports_disabled/BUILD.out create mode 100644 gazelle/python/testdata/sibling_imports_disabled/README.md create mode 100644 gazelle/python/testdata/sibling_imports_disabled/WORKSPACE create mode 100644 gazelle/python/testdata/sibling_imports_disabled/a.py create mode 100644 gazelle/python/testdata/sibling_imports_disabled/b.py create mode 100644 gazelle/python/testdata/sibling_imports_disabled/pkg/BUILD.in create mode 100644 gazelle/python/testdata/sibling_imports_disabled/pkg/BUILD.out create mode 100644 gazelle/python/testdata/sibling_imports_disabled/pkg/__init__.py create mode 100644 gazelle/python/testdata/sibling_imports_disabled/pkg/a.py create mode 100644 gazelle/python/testdata/sibling_imports_disabled/pkg/b.py create mode 100644 gazelle/python/testdata/sibling_imports_disabled/pkg/test_util.py create mode 100644 gazelle/python/testdata/sibling_imports_disabled/pkg/typing.py create mode 100644 gazelle/python/testdata/sibling_imports_disabled/pkg/unit_test.py create mode 100644 gazelle/python/testdata/sibling_imports_disabled/test.yaml create mode 100644 gazelle/python/testdata/sibling_imports_disabled/test_util.py create mode 100644 gazelle/python/testdata/sibling_imports_disabled_file_mode/BUILD.in create mode 100644 gazelle/python/testdata/sibling_imports_disabled_file_mode/BUILD.out create mode 100644 gazelle/python/testdata/sibling_imports_disabled_file_mode/README.md create mode 100644 gazelle/python/testdata/sibling_imports_disabled_file_mode/WORKSPACE create mode 100644 gazelle/python/testdata/sibling_imports_disabled_file_mode/a.py create mode 100644 gazelle/python/testdata/sibling_imports_disabled_file_mode/b.py create mode 100644 gazelle/python/testdata/sibling_imports_disabled_file_mode/pkg/BUILD.in create mode 100644 gazelle/python/testdata/sibling_imports_disabled_file_mode/pkg/BUILD.out create mode 100644 gazelle/python/testdata/sibling_imports_disabled_file_mode/pkg/__init__.py create mode 100644 gazelle/python/testdata/sibling_imports_disabled_file_mode/pkg/a.py create mode 100644 gazelle/python/testdata/sibling_imports_disabled_file_mode/pkg/b.py create mode 100644 gazelle/python/testdata/sibling_imports_disabled_file_mode/pkg/test_util.py create mode 100644 gazelle/python/testdata/sibling_imports_disabled_file_mode/pkg/typing.py create mode 100644 gazelle/python/testdata/sibling_imports_disabled_file_mode/pkg/unit_test.py create mode 100644 gazelle/python/testdata/sibling_imports_disabled_file_mode/test.yaml create mode 100644 gazelle/python/testdata/sibling_imports_disabled_file_mode/test_util.py create mode 100644 gazelle/python/testdata/simple_test_with_conftest_sibling_imports_disabled/BUILD.in create mode 100644 gazelle/python/testdata/simple_test_with_conftest_sibling_imports_disabled/BUILD.out create mode 100644 gazelle/python/testdata/simple_test_with_conftest_sibling_imports_disabled/README.md create mode 100644 gazelle/python/testdata/simple_test_with_conftest_sibling_imports_disabled/WORKSPACE create mode 100644 gazelle/python/testdata/simple_test_with_conftest_sibling_imports_disabled/__init__.py create mode 100644 gazelle/python/testdata/simple_test_with_conftest_sibling_imports_disabled/__test__.py create mode 100644 gazelle/python/testdata/simple_test_with_conftest_sibling_imports_disabled/bar/BUILD.in create mode 100644 gazelle/python/testdata/simple_test_with_conftest_sibling_imports_disabled/bar/BUILD.out create mode 100644 gazelle/python/testdata/simple_test_with_conftest_sibling_imports_disabled/bar/__init__.py create mode 100644 gazelle/python/testdata/simple_test_with_conftest_sibling_imports_disabled/bar/__test__.py create mode 100644 gazelle/python/testdata/simple_test_with_conftest_sibling_imports_disabled/bar/bar.py create mode 100644 gazelle/python/testdata/simple_test_with_conftest_sibling_imports_disabled/bar/conftest.py create mode 100644 gazelle/python/testdata/simple_test_with_conftest_sibling_imports_disabled/conftest.py create mode 100644 gazelle/python/testdata/simple_test_with_conftest_sibling_imports_disabled/foo.py create mode 100644 gazelle/python/testdata/simple_test_with_conftest_sibling_imports_disabled/test.yaml diff --git a/CHANGELOG.md b/CHANGELOG.md index 5ad48bee3f..c52481d52e 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -95,6 +95,10 @@ END_UNRELEASED_TEMPLATE ([#2921](https://github.com/bazel-contrib/rules_python/issues/2921)). * (repl) Normalize the path for the `REPL` stub to make it possible to use the default stub template from outside `rules_python` ({gh-issue}`3101`). +* (gazelle) Fixes gazelle adding sibling module dependencies to resolve + absolute imports (Python 2's behavior without `absolute_import`). Previous + behavior can be restored using the directive + `# gazelle:python_resolve_sibling_imports true` {#v0-0-0-added} ### Added diff --git a/gazelle/README.md b/gazelle/README.md index 222c1171ab..8b088a4e70 100644 --- a/gazelle/README.md +++ b/gazelle/README.md @@ -228,6 +228,8 @@ Python-specific directives are as follows: | Controls whether to generate a separate `pyi_deps` attribute for type-checking dependencies or merge them into the regular `deps` attribute. When `false` (default), type-checking dependencies are merged into `deps` for backward compatibility. When `true`, generates separate `pyi_deps`. Imports in blocks with the format `if typing.TYPE_CHECKING:`/`if TYPE_CHECKING:` and type-only stub packages (eg. boto3-stubs) are recognized as type-checking dependencies. | | [`# gazelle:python_generate_proto`](#directive-python_generate_proto) | `false` | | Controls whether to generate a `py_proto_library` for each `proto_library` in the package. By default we load this rule from the `@protobuf` repository; use `gazelle:map_kind` if you need to load this from somewhere else. | +| `# gazelle:python_resolve_sibling_imports` | `false` | +| Allows absolute imports to be resolved to sibling modules (Python 2's behavior without `absolute_import`). | #### Directive: `python_root`: diff --git a/gazelle/python/configure.go b/gazelle/python/configure.go index 079f1d84d4..13ba6477cd 100644 --- a/gazelle/python/configure.go +++ b/gazelle/python/configure.go @@ -72,6 +72,7 @@ func (py *Configurer) KnownDirectives() []string { pythonconfig.GeneratePyiDeps, pythonconfig.ExperimentalAllowRelativeImports, pythonconfig.GenerateProto, + pythonconfig.PythonResolveSiblingImports, } } @@ -247,6 +248,12 @@ func (py *Configurer) Configure(c *config.Config, rel string, f *rule.File) { log.Fatal(err) } config.SetGenerateProto(v) + case pythonconfig.PythonResolveSiblingImports: + v, err := strconv.ParseBool(strings.TrimSpace(d.Value)) + if err != nil { + log.Fatal(err) + } + config.SetResolveSiblingImports(v) } } diff --git a/gazelle/python/generate.go b/gazelle/python/generate.go index 5b6ba79d69..a180ec527d 100644 --- a/gazelle/python/generate.go +++ b/gazelle/python/generate.go @@ -259,7 +259,7 @@ func (py *Python) GenerateRules(args language.GenerateArgs) language.GenerateRes fqTarget.String(), actualPyBinaryKind, err) continue } - pyBinary := newTargetBuilder(pyBinaryKind, pyBinaryTargetName, pythonProjectRoot, args.Rel, pyFileNames). + pyBinary := newTargetBuilder(pyBinaryKind, pyBinaryTargetName, pythonProjectRoot, args.Rel, pyFileNames, cfg.ResolveSiblingImports()). addVisibility(visibility). addSrc(filename). addModuleDependencies(mainModules[filename]). @@ -301,7 +301,7 @@ func (py *Python) GenerateRules(args language.GenerateArgs) language.GenerateRes collisionErrors.Add(err) } - pyLibrary := newTargetBuilder(pyLibraryKind, pyLibraryTargetName, pythonProjectRoot, args.Rel, pyFileNames). + pyLibrary := newTargetBuilder(pyLibraryKind, pyLibraryTargetName, pythonProjectRoot, args.Rel, pyFileNames, cfg.ResolveSiblingImports()). addVisibility(visibility). addSrcs(srcs). addModuleDependencies(allDeps). @@ -354,7 +354,7 @@ func (py *Python) GenerateRules(args language.GenerateArgs) language.GenerateRes collisionErrors.Add(err) } - pyBinaryTarget := newTargetBuilder(pyBinaryKind, pyBinaryTargetName, pythonProjectRoot, args.Rel, pyFileNames). + pyBinaryTarget := newTargetBuilder(pyBinaryKind, pyBinaryTargetName, pythonProjectRoot, args.Rel, pyFileNames, cfg.ResolveSiblingImports()). setMain(pyBinaryEntrypointFilename). addVisibility(visibility). addSrc(pyBinaryEntrypointFilename). @@ -387,7 +387,7 @@ func (py *Python) GenerateRules(args language.GenerateArgs) language.GenerateRes collisionErrors.Add(err) } - conftestTarget := newTargetBuilder(pyLibraryKind, conftestTargetname, pythonProjectRoot, args.Rel, pyFileNames). + conftestTarget := newTargetBuilder(pyLibraryKind, conftestTargetname, pythonProjectRoot, args.Rel, pyFileNames, cfg.ResolveSiblingImports()). addSrc(conftestFilename). addModuleDependencies(deps). addResolvedDependencies(annotations.includeDeps). @@ -419,7 +419,7 @@ func (py *Python) GenerateRules(args language.GenerateArgs) language.GenerateRes fqTarget.String(), actualPyTestKind, err, pythonconfig.TestNamingConvention) collisionErrors.Add(err) } - return newTargetBuilder(pyTestKind, pyTestTargetName, pythonProjectRoot, args.Rel, pyFileNames). + return newTargetBuilder(pyTestKind, pyTestTargetName, pythonProjectRoot, args.Rel, pyFileNames, cfg.ResolveSiblingImports()). addSrcs(srcs). addModuleDependencies(deps). addResolvedDependencies(annotations.includeDeps). @@ -476,7 +476,7 @@ func (py *Python) GenerateRules(args language.GenerateArgs) language.GenerateRes for _, pyTestTarget := range pyTestTargets { if conftest != nil { - conftestModule := Module{Name: strings.TrimSuffix(conftestFilename, ".py")} + conftestModule := Module{Name: importSpecFromSrc(pythonProjectRoot, args.Rel, conftestFilename).Imp} if pyTestTarget.annotations.includePytestConftest == nil { // unset; default behavior pyTestTarget.addModuleDependency(conftestModule) @@ -605,7 +605,7 @@ func generateProtoLibraries(args language.GenerateArgs, cfg *pythonconfig.Config pyProtoLibraryName = ruleName } - pyProtoLibrary := newTargetBuilder(pyProtoLibraryKind, pyProtoLibraryName, pythonProjectRoot, args.Rel, &emptySiblings). + pyProtoLibrary := newTargetBuilder(pyProtoLibraryKind, pyProtoLibraryName, pythonProjectRoot, args.Rel, &emptySiblings, false). addVisibility(visibility). addResolvedDependency(":" + protoRuleName). generateImportsAttribute().build() @@ -622,7 +622,7 @@ func generateProtoLibraries(args language.GenerateArgs, cfg *pythonconfig.Config continue } - emptyRule := newTargetBuilder(pyProtoLibraryKind, ruleName, pythonProjectRoot, args.Rel, &emptySiblings).build() + emptyRule := newTargetBuilder(pyProtoLibraryKind, ruleName, pythonProjectRoot, args.Rel, &emptySiblings, false).build() res.Empty = append(res.Empty, emptyRule) } diff --git a/gazelle/python/resolve.go b/gazelle/python/resolve.go index 0dd80841d4..cc57180a49 100644 --- a/gazelle/python/resolve.go +++ b/gazelle/python/resolve.go @@ -164,8 +164,6 @@ func (py *Resolver) Resolve( modules := modulesRaw.(*treeset.Set) it := modules.Iterator() explainDependency := os.Getenv("EXPLAIN_DEPENDENCY") - // Resolve relative paths for package generation - isPackageGeneration := !cfg.PerFileGeneration() && !cfg.CoarseGrainedGeneration() hasFatalError := false MODULES_LOOP: for it.Next() { @@ -173,7 +171,7 @@ func (py *Resolver) Resolve( moduleName := mod.Name // Transform relative imports `.` or `..foo.bar` into the package path from root. if strings.HasPrefix(mod.From, ".") { - if !cfg.ExperimentalAllowRelativeImports() || !isPackageGeneration { + if !cfg.ExperimentalAllowRelativeImports() { continue MODULES_LOOP } @@ -210,9 +208,9 @@ func (py *Resolver) Resolve( baseParts = pkgParts[:len(pkgParts)-(relativeDepth-1)] } // Build absolute module path - absParts := append([]string{}, baseParts...) // base path - absParts = append(absParts, fromParts...) // subpath from 'from' - absParts = append(absParts, imported) // actual imported symbol + absParts := append([]string{}, baseParts...) // base path + absParts = append(absParts, fromParts...) // subpath from 'from' + absParts = append(absParts, imported) // actual imported symbol moduleName = strings.Join(absParts, ".") } diff --git a/gazelle/python/target.go b/gazelle/python/target.go index 6e6c3f4b14..3fe5819e00 100644 --- a/gazelle/python/target.go +++ b/gazelle/python/target.go @@ -25,34 +25,36 @@ import ( // targetBuilder builds targets to be generated by Gazelle. type targetBuilder struct { - kind string - name string - pythonProjectRoot string - bzlPackage string - srcs *treeset.Set - siblingSrcs *treeset.Set - deps *treeset.Set - resolvedDeps *treeset.Set - visibility *treeset.Set - main *string - imports []string - testonly bool - annotations *annotations + kind string + name string + pythonProjectRoot string + bzlPackage string + srcs *treeset.Set + siblingSrcs *treeset.Set + deps *treeset.Set + resolvedDeps *treeset.Set + visibility *treeset.Set + main *string + imports []string + testonly bool + annotations *annotations + resolveSiblingImports bool } // newTargetBuilder constructs a new targetBuilder. -func newTargetBuilder(kind, name, pythonProjectRoot, bzlPackage string, siblingSrcs *treeset.Set) *targetBuilder { +func newTargetBuilder(kind, name, pythonProjectRoot, bzlPackage string, siblingSrcs *treeset.Set, resolveSiblingImports bool) *targetBuilder { return &targetBuilder{ - kind: kind, - name: name, - pythonProjectRoot: pythonProjectRoot, - bzlPackage: bzlPackage, - srcs: treeset.NewWith(godsutils.StringComparator), - siblingSrcs: siblingSrcs, - deps: treeset.NewWith(moduleComparator), - resolvedDeps: treeset.NewWith(godsutils.StringComparator), - visibility: treeset.NewWith(godsutils.StringComparator), - annotations: new(annotations), + kind: kind, + name: name, + pythonProjectRoot: pythonProjectRoot, + bzlPackage: bzlPackage, + srcs: treeset.NewWith(godsutils.StringComparator), + siblingSrcs: siblingSrcs, + deps: treeset.NewWith(moduleComparator), + resolvedDeps: treeset.NewWith(godsutils.StringComparator), + visibility: treeset.NewWith(godsutils.StringComparator), + annotations: new(annotations), + resolveSiblingImports: resolveSiblingImports, } } @@ -77,7 +79,7 @@ func (t *targetBuilder) addModuleDependency(dep Module) *targetBuilder { if dep.From != "" { fileName = dep.From + ".py" } - if t.siblingSrcs.Contains(fileName) && fileName != filepath.Base(dep.Filepath) { + if t.resolveSiblingImports && t.siblingSrcs.Contains(fileName) && fileName != filepath.Base(dep.Filepath) { // importing another module from the same package, converting to absolute imports to make // dependency resolution easier dep.Name = importSpecFromSrc(t.pythonProjectRoot, t.bzlPackage, fileName).Imp @@ -138,7 +140,6 @@ func (t *targetBuilder) setAnnotations(val annotations) *targetBuilder { return t } - // generateImportsAttribute generates the imports attribute. // These are a list of import directories to be added to the PYTHONPATH. In our // case, the value we add is on Bazel sub-packages to be able to perform imports diff --git a/gazelle/python/testdata/annotation_include_dep/BUILD.in b/gazelle/python/testdata/annotation_include_dep/BUILD.in index af2c2cea4b..5131712aca 100644 --- a/gazelle/python/testdata/annotation_include_dep/BUILD.in +++ b/gazelle/python/testdata/annotation_include_dep/BUILD.in @@ -1 +1,2 @@ # gazelle:python_generation_mode file +# gazelle:python_resolve_sibling_imports true diff --git a/gazelle/python/testdata/annotation_include_dep/BUILD.out b/gazelle/python/testdata/annotation_include_dep/BUILD.out index 1cff8f4676..412bf456f5 100644 --- a/gazelle/python/testdata/annotation_include_dep/BUILD.out +++ b/gazelle/python/testdata/annotation_include_dep/BUILD.out @@ -1,6 +1,7 @@ load("@rules_python//python:defs.bzl", "py_binary", "py_library", "py_test") # gazelle:python_generation_mode file +# gazelle:python_resolve_sibling_imports true py_library( name = "__init__", diff --git a/gazelle/python/testdata/annotation_include_pytest_conftest/with_conftest/BUILD.in b/gazelle/python/testdata/annotation_include_pytest_conftest/with_conftest/BUILD.in index e69de29bb2..5c25b0d5a6 100644 --- a/gazelle/python/testdata/annotation_include_pytest_conftest/with_conftest/BUILD.in +++ b/gazelle/python/testdata/annotation_include_pytest_conftest/with_conftest/BUILD.in @@ -0,0 +1 @@ +# gazelle:python_resolve_sibling_imports true diff --git a/gazelle/python/testdata/annotation_include_pytest_conftest/with_conftest/BUILD.out b/gazelle/python/testdata/annotation_include_pytest_conftest/with_conftest/BUILD.out index 60695352ca..52b915208e 100644 --- a/gazelle/python/testdata/annotation_include_pytest_conftest/with_conftest/BUILD.out +++ b/gazelle/python/testdata/annotation_include_pytest_conftest/with_conftest/BUILD.out @@ -1,5 +1,7 @@ load("@rules_python//python:defs.bzl", "py_binary", "py_library", "py_test") +# gazelle:python_resolve_sibling_imports true + py_binary( name = "binary", srcs = ["binary.py"], diff --git a/gazelle/python/testdata/naming_convention/BUILD.in b/gazelle/python/testdata/naming_convention/BUILD.in index 7517848a92..fee53ba7ff 100644 --- a/gazelle/python/testdata/naming_convention/BUILD.in +++ b/gazelle/python/testdata/naming_convention/BUILD.in @@ -1,3 +1,4 @@ # gazelle:python_library_naming_convention my_$package_name$_library # gazelle:python_binary_naming_convention my_$package_name$_binary # gazelle:python_test_naming_convention my_$package_name$_test +# gazelle:python_resolve_sibling_imports true diff --git a/gazelle/python/testdata/naming_convention/BUILD.out b/gazelle/python/testdata/naming_convention/BUILD.out index e2f067489c..7392cfeb35 100644 --- a/gazelle/python/testdata/naming_convention/BUILD.out +++ b/gazelle/python/testdata/naming_convention/BUILD.out @@ -3,6 +3,7 @@ load("@rules_python//python:defs.bzl", "py_binary", "py_library", "py_test") # gazelle:python_library_naming_convention my_$package_name$_library # gazelle:python_binary_naming_convention my_$package_name$_binary # gazelle:python_test_naming_convention my_$package_name$_test +# gazelle:python_resolve_sibling_imports true py_library( name = "my_naming_convention_library", diff --git a/gazelle/python/testdata/sibling_imports/README.md b/gazelle/python/testdata/sibling_imports/README.md index e59be07634..d21a671b1c 100644 --- a/gazelle/python/testdata/sibling_imports/README.md +++ b/gazelle/python/testdata/sibling_imports/README.md @@ -1,3 +1,13 @@ # Sibling imports -This test case asserts that imports from sibling modules are resolved correctly. It covers 3 different types of imports in `pkg/unit_test.py` \ No newline at end of file +This test case asserts that imports from sibling modules are resolved correctly +when the `python_resolve_sibling_imports` directive is enabled (default +behavior). It covers 3 different types of imports in `pkg/unit_test.py`: + +- `import a` - resolves to the sibling `a.py` in the same package +- `import test_util` - resolves to the sibling `test_util.py` in the same + package +- `from b import run` - resolves to the sibling `b.py` in the same package + +When sibling imports are enabled, we allow them to be satisfied by sibling +modules (ie. modules in the same package). diff --git a/gazelle/python/testdata/sibling_imports/pkg/BUILD.in b/gazelle/python/testdata/sibling_imports/pkg/BUILD.in index e69de29bb2..5c25b0d5a6 100644 --- a/gazelle/python/testdata/sibling_imports/pkg/BUILD.in +++ b/gazelle/python/testdata/sibling_imports/pkg/BUILD.in @@ -0,0 +1 @@ +# gazelle:python_resolve_sibling_imports true diff --git a/gazelle/python/testdata/sibling_imports/pkg/BUILD.out b/gazelle/python/testdata/sibling_imports/pkg/BUILD.out index cae6c3f17a..e8c13098c2 100644 --- a/gazelle/python/testdata/sibling_imports/pkg/BUILD.out +++ b/gazelle/python/testdata/sibling_imports/pkg/BUILD.out @@ -1,5 +1,7 @@ load("@rules_python//python:defs.bzl", "py_library", "py_test") +# gazelle:python_resolve_sibling_imports true + py_library( name = "pkg", srcs = [ @@ -23,4 +25,3 @@ py_test( ":test_util", ], ) - diff --git a/gazelle/python/testdata/sibling_imports_disabled/BUILD.in b/gazelle/python/testdata/sibling_imports_disabled/BUILD.in new file mode 100644 index 0000000000..9509fd9727 --- /dev/null +++ b/gazelle/python/testdata/sibling_imports_disabled/BUILD.in @@ -0,0 +1,2 @@ +# gazelle:python_resolve_sibling_imports false +# gazelle:experimental_allow_relative_imports true diff --git a/gazelle/python/testdata/sibling_imports_disabled/BUILD.out b/gazelle/python/testdata/sibling_imports_disabled/BUILD.out new file mode 100644 index 0000000000..7568f38f50 --- /dev/null +++ b/gazelle/python/testdata/sibling_imports_disabled/BUILD.out @@ -0,0 +1,18 @@ +load("@rules_python//python:defs.bzl", "py_library", "py_test") + +# gazelle:python_resolve_sibling_imports false +# gazelle:experimental_allow_relative_imports true + +py_library( + name = "sibling_imports_disabled", + srcs = [ + "a.py", + "b.py", + ], + visibility = ["//:__subpackages__"], +) + +py_test( + name = "test_util", + srcs = ["test_util.py"], +) diff --git a/gazelle/python/testdata/sibling_imports_disabled/README.md b/gazelle/python/testdata/sibling_imports_disabled/README.md new file mode 100644 index 0000000000..d534a44bf1 --- /dev/null +++ b/gazelle/python/testdata/sibling_imports_disabled/README.md @@ -0,0 +1,22 @@ +# Sibling imports disabled + +This test case asserts that imports from sibling modules are NOT resolved as +absolute imports when the `python_resolve_sibling_imports` directive is +disabled. It covers different types of imports in `pkg/unit_test.py`: + +- `import a` - resolves to the root-level `a.py` instead of the sibling + `pkg/a.py` +- `from typing import Iterable` - resolves to the stdlib `typing` module + (not the sibling `typing.py`). +- `from .b import run` / `from .typing import A` - resolves to the sibling + `pkg/b.py` / `pkg/typing.py` (with + `gazelle:experimental_allow_relative_imports` enabled) +- `import test_util` - resolves to the root-level `test_util.py` instead of + the sibling `pkg/test_util.py` +- `from b import run` - resolves to the root-level `b.py` instead of the + sibling `pkg/b.py` + +When sibling imports are disabled with +`# gazelle:python_resolve_sibling_imports false`, the imports remain as-is +and follow standard Python resolution rules where absolute imports can't refer +to sibling modules. diff --git a/gazelle/python/testdata/sibling_imports_disabled/WORKSPACE b/gazelle/python/testdata/sibling_imports_disabled/WORKSPACE new file mode 100644 index 0000000000..faff6af87a --- /dev/null +++ b/gazelle/python/testdata/sibling_imports_disabled/WORKSPACE @@ -0,0 +1 @@ +# This is a Bazel workspace for the Gazelle test data. diff --git a/gazelle/python/testdata/sibling_imports_disabled/a.py b/gazelle/python/testdata/sibling_imports_disabled/a.py new file mode 100644 index 0000000000..fad4fb1ff9 --- /dev/null +++ b/gazelle/python/testdata/sibling_imports_disabled/a.py @@ -0,0 +1 @@ +# Root level a.py file for testing disabled sibling imports diff --git a/gazelle/python/testdata/sibling_imports_disabled/b.py b/gazelle/python/testdata/sibling_imports_disabled/b.py new file mode 100644 index 0000000000..a5eafc436f --- /dev/null +++ b/gazelle/python/testdata/sibling_imports_disabled/b.py @@ -0,0 +1,3 @@ +# Root level b.py file for testing disabled sibling imports +def run(): + pass diff --git a/gazelle/python/testdata/sibling_imports_disabled/pkg/BUILD.in b/gazelle/python/testdata/sibling_imports_disabled/pkg/BUILD.in new file mode 100644 index 0000000000..e69de29bb2 diff --git a/gazelle/python/testdata/sibling_imports_disabled/pkg/BUILD.out b/gazelle/python/testdata/sibling_imports_disabled/pkg/BUILD.out new file mode 100644 index 0000000000..e778ce1076 --- /dev/null +++ b/gazelle/python/testdata/sibling_imports_disabled/pkg/BUILD.out @@ -0,0 +1,27 @@ +load("@rules_python//python:defs.bzl", "py_library", "py_test") + +py_library( + name = "pkg", + srcs = [ + "__init__.py", + "a.py", + "b.py", + "typing.py", + ], + visibility = ["//:__subpackages__"], +) + +py_test( + name = "test_util", + srcs = ["test_util.py"], + deps = [":pkg"], +) + +py_test( + name = "unit_test", + srcs = ["unit_test.py"], + deps = [ + "//:sibling_imports_disabled", + "//:test_util", + ], +) diff --git a/gazelle/python/testdata/sibling_imports_disabled/pkg/__init__.py b/gazelle/python/testdata/sibling_imports_disabled/pkg/__init__.py new file mode 100644 index 0000000000..e69de29bb2 diff --git a/gazelle/python/testdata/sibling_imports_disabled/pkg/a.py b/gazelle/python/testdata/sibling_imports_disabled/pkg/a.py new file mode 100644 index 0000000000..e69de29bb2 diff --git a/gazelle/python/testdata/sibling_imports_disabled/pkg/b.py b/gazelle/python/testdata/sibling_imports_disabled/pkg/b.py new file mode 100644 index 0000000000..d04d423678 --- /dev/null +++ b/gazelle/python/testdata/sibling_imports_disabled/pkg/b.py @@ -0,0 +1,2 @@ +def run(): + pass diff --git a/gazelle/python/testdata/sibling_imports_disabled/pkg/test_util.py b/gazelle/python/testdata/sibling_imports_disabled/pkg/test_util.py new file mode 100644 index 0000000000..01cc15da86 --- /dev/null +++ b/gazelle/python/testdata/sibling_imports_disabled/pkg/test_util.py @@ -0,0 +1,2 @@ +from .b import run +from .typing import A diff --git a/gazelle/python/testdata/sibling_imports_disabled/pkg/typing.py b/gazelle/python/testdata/sibling_imports_disabled/pkg/typing.py new file mode 100644 index 0000000000..76f516f79f --- /dev/null +++ b/gazelle/python/testdata/sibling_imports_disabled/pkg/typing.py @@ -0,0 +1 @@ +A = 1 diff --git a/gazelle/python/testdata/sibling_imports_disabled/pkg/unit_test.py b/gazelle/python/testdata/sibling_imports_disabled/pkg/unit_test.py new file mode 100644 index 0000000000..3c551a1cb1 --- /dev/null +++ b/gazelle/python/testdata/sibling_imports_disabled/pkg/unit_test.py @@ -0,0 +1,5 @@ +from typing import Iterable + +import a +import test_util +from b import run diff --git a/gazelle/python/testdata/sibling_imports_disabled/test.yaml b/gazelle/python/testdata/sibling_imports_disabled/test.yaml new file mode 100644 index 0000000000..ed97d539c0 --- /dev/null +++ b/gazelle/python/testdata/sibling_imports_disabled/test.yaml @@ -0,0 +1 @@ +--- diff --git a/gazelle/python/testdata/sibling_imports_disabled/test_util.py b/gazelle/python/testdata/sibling_imports_disabled/test_util.py new file mode 100644 index 0000000000..f5fa1b34ea --- /dev/null +++ b/gazelle/python/testdata/sibling_imports_disabled/test_util.py @@ -0,0 +1 @@ +# Root level test_util.py file for testing disabled sibling imports diff --git a/gazelle/python/testdata/sibling_imports_disabled_file_mode/BUILD.in b/gazelle/python/testdata/sibling_imports_disabled_file_mode/BUILD.in new file mode 100644 index 0000000000..04494394c7 --- /dev/null +++ b/gazelle/python/testdata/sibling_imports_disabled_file_mode/BUILD.in @@ -0,0 +1,3 @@ +# gazelle:python_generation_mode file +# gazelle:python_resolve_sibling_imports false +# gazelle:experimental_allow_relative_imports true diff --git a/gazelle/python/testdata/sibling_imports_disabled_file_mode/BUILD.out b/gazelle/python/testdata/sibling_imports_disabled_file_mode/BUILD.out new file mode 100644 index 0000000000..da53e14864 --- /dev/null +++ b/gazelle/python/testdata/sibling_imports_disabled_file_mode/BUILD.out @@ -0,0 +1,22 @@ +load("@rules_python//python:defs.bzl", "py_library", "py_test") + +# gazelle:python_generation_mode file +# gazelle:python_resolve_sibling_imports false +# gazelle:experimental_allow_relative_imports true + +py_library( + name = "a", + srcs = ["a.py"], + visibility = ["//:__subpackages__"], +) + +py_library( + name = "b", + srcs = ["b.py"], + visibility = ["//:__subpackages__"], +) + +py_test( + name = "test_util", + srcs = ["test_util.py"], +) diff --git a/gazelle/python/testdata/sibling_imports_disabled_file_mode/README.md b/gazelle/python/testdata/sibling_imports_disabled_file_mode/README.md new file mode 100644 index 0000000000..0bfbcffb58 --- /dev/null +++ b/gazelle/python/testdata/sibling_imports_disabled_file_mode/README.md @@ -0,0 +1,22 @@ +# Sibling imports disabled (file generation mode) + +This test case asserts that imports from sibling modules are NOT resolved as +absolute imports when the `python_resolve_sibling_imports` directive is +disabled. It covers different types of imports in `pkg/unit_test.py`: + +- `import a` - resolves to the root-level `a.py` instead of the sibling + `pkg/a.py` +- `from typing import Iterable` - resolves to the stdlib `typing` module + (not the sibling `typing.py`). +- `from .b import run` / `from .typing import A` - resolves to the sibling + `pkg/b.py` / `pkg/typing.py` (with + `gazelle:experimental_allow_relative_imports` enabled) +- `import test_util` - resolves to the root-level `test_util.py` instead of + the sibling `pkg/test_util.py` +- `from b import run` - resolves to the root-level `b.py` instead of the + sibling `pkg/b.py` + +When sibling imports are disabled with +`# gazelle:python_resolve_sibling_imports false`, the imports remain as-is +and follow standard Python resolution rules where absolute imports can't refer +to sibling modules. diff --git a/gazelle/python/testdata/sibling_imports_disabled_file_mode/WORKSPACE b/gazelle/python/testdata/sibling_imports_disabled_file_mode/WORKSPACE new file mode 100644 index 0000000000..faff6af87a --- /dev/null +++ b/gazelle/python/testdata/sibling_imports_disabled_file_mode/WORKSPACE @@ -0,0 +1 @@ +# This is a Bazel workspace for the Gazelle test data. diff --git a/gazelle/python/testdata/sibling_imports_disabled_file_mode/a.py b/gazelle/python/testdata/sibling_imports_disabled_file_mode/a.py new file mode 100644 index 0000000000..fad4fb1ff9 --- /dev/null +++ b/gazelle/python/testdata/sibling_imports_disabled_file_mode/a.py @@ -0,0 +1 @@ +# Root level a.py file for testing disabled sibling imports diff --git a/gazelle/python/testdata/sibling_imports_disabled_file_mode/b.py b/gazelle/python/testdata/sibling_imports_disabled_file_mode/b.py new file mode 100644 index 0000000000..a5eafc436f --- /dev/null +++ b/gazelle/python/testdata/sibling_imports_disabled_file_mode/b.py @@ -0,0 +1,3 @@ +# Root level b.py file for testing disabled sibling imports +def run(): + pass diff --git a/gazelle/python/testdata/sibling_imports_disabled_file_mode/pkg/BUILD.in b/gazelle/python/testdata/sibling_imports_disabled_file_mode/pkg/BUILD.in new file mode 100644 index 0000000000..e69de29bb2 diff --git a/gazelle/python/testdata/sibling_imports_disabled_file_mode/pkg/BUILD.out b/gazelle/python/testdata/sibling_imports_disabled_file_mode/pkg/BUILD.out new file mode 100644 index 0000000000..ab161e135f --- /dev/null +++ b/gazelle/python/testdata/sibling_imports_disabled_file_mode/pkg/BUILD.out @@ -0,0 +1,38 @@ +load("@rules_python//python:defs.bzl", "py_library", "py_test") + +py_library( + name = "a", + srcs = ["a.py"], + visibility = ["//:__subpackages__"], +) + +py_library( + name = "b", + srcs = ["b.py"], + visibility = ["//:__subpackages__"], +) + +py_library( + name = "typing", + srcs = ["typing.py"], + visibility = ["//:__subpackages__"], +) + +py_test( + name = "test_util", + srcs = ["test_util.py"], + deps = [ + ":b", + ":typing", + ], +) + +py_test( + name = "unit_test", + srcs = ["unit_test.py"], + deps = [ + "//:a", + "//:b", + "//:test_util", + ], +) diff --git a/gazelle/python/testdata/sibling_imports_disabled_file_mode/pkg/__init__.py b/gazelle/python/testdata/sibling_imports_disabled_file_mode/pkg/__init__.py new file mode 100644 index 0000000000..e69de29bb2 diff --git a/gazelle/python/testdata/sibling_imports_disabled_file_mode/pkg/a.py b/gazelle/python/testdata/sibling_imports_disabled_file_mode/pkg/a.py new file mode 100644 index 0000000000..e69de29bb2 diff --git a/gazelle/python/testdata/sibling_imports_disabled_file_mode/pkg/b.py b/gazelle/python/testdata/sibling_imports_disabled_file_mode/pkg/b.py new file mode 100644 index 0000000000..d04d423678 --- /dev/null +++ b/gazelle/python/testdata/sibling_imports_disabled_file_mode/pkg/b.py @@ -0,0 +1,2 @@ +def run(): + pass diff --git a/gazelle/python/testdata/sibling_imports_disabled_file_mode/pkg/test_util.py b/gazelle/python/testdata/sibling_imports_disabled_file_mode/pkg/test_util.py new file mode 100644 index 0000000000..01cc15da86 --- /dev/null +++ b/gazelle/python/testdata/sibling_imports_disabled_file_mode/pkg/test_util.py @@ -0,0 +1,2 @@ +from .b import run +from .typing import A diff --git a/gazelle/python/testdata/sibling_imports_disabled_file_mode/pkg/typing.py b/gazelle/python/testdata/sibling_imports_disabled_file_mode/pkg/typing.py new file mode 100644 index 0000000000..76f516f79f --- /dev/null +++ b/gazelle/python/testdata/sibling_imports_disabled_file_mode/pkg/typing.py @@ -0,0 +1 @@ +A = 1 diff --git a/gazelle/python/testdata/sibling_imports_disabled_file_mode/pkg/unit_test.py b/gazelle/python/testdata/sibling_imports_disabled_file_mode/pkg/unit_test.py new file mode 100644 index 0000000000..3c551a1cb1 --- /dev/null +++ b/gazelle/python/testdata/sibling_imports_disabled_file_mode/pkg/unit_test.py @@ -0,0 +1,5 @@ +from typing import Iterable + +import a +import test_util +from b import run diff --git a/gazelle/python/testdata/sibling_imports_disabled_file_mode/test.yaml b/gazelle/python/testdata/sibling_imports_disabled_file_mode/test.yaml new file mode 100644 index 0000000000..ed97d539c0 --- /dev/null +++ b/gazelle/python/testdata/sibling_imports_disabled_file_mode/test.yaml @@ -0,0 +1 @@ +--- diff --git a/gazelle/python/testdata/sibling_imports_disabled_file_mode/test_util.py b/gazelle/python/testdata/sibling_imports_disabled_file_mode/test_util.py new file mode 100644 index 0000000000..f5fa1b34ea --- /dev/null +++ b/gazelle/python/testdata/sibling_imports_disabled_file_mode/test_util.py @@ -0,0 +1 @@ +# Root level test_util.py file for testing disabled sibling imports diff --git a/gazelle/python/testdata/simple_test_with_conftest/BUILD.in b/gazelle/python/testdata/simple_test_with_conftest/BUILD.in index 3f2beb3147..6dfab75442 100644 --- a/gazelle/python/testdata/simple_test_with_conftest/BUILD.in +++ b/gazelle/python/testdata/simple_test_with_conftest/BUILD.in @@ -1 +1,3 @@ load("@rules_python//python:defs.bzl", "py_library") + +# gazelle:python_resolve_sibling_imports true diff --git a/gazelle/python/testdata/simple_test_with_conftest/BUILD.out b/gazelle/python/testdata/simple_test_with_conftest/BUILD.out index 18079bf2f4..62e1c550e6 100644 --- a/gazelle/python/testdata/simple_test_with_conftest/BUILD.out +++ b/gazelle/python/testdata/simple_test_with_conftest/BUILD.out @@ -1,5 +1,7 @@ load("@rules_python//python:defs.bzl", "py_library", "py_test") +# gazelle:python_resolve_sibling_imports true + py_library( name = "simple_test_with_conftest", srcs = [ diff --git a/gazelle/python/testdata/simple_test_with_conftest_sibling_imports_disabled/BUILD.in b/gazelle/python/testdata/simple_test_with_conftest_sibling_imports_disabled/BUILD.in new file mode 100644 index 0000000000..f8a40fe26c --- /dev/null +++ b/gazelle/python/testdata/simple_test_with_conftest_sibling_imports_disabled/BUILD.in @@ -0,0 +1,3 @@ +load("@rules_python//python:defs.bzl", "py_library") + +# gazelle:python_resolve_sibling_imports false diff --git a/gazelle/python/testdata/simple_test_with_conftest_sibling_imports_disabled/BUILD.out b/gazelle/python/testdata/simple_test_with_conftest_sibling_imports_disabled/BUILD.out new file mode 100644 index 0000000000..b5a7066aff --- /dev/null +++ b/gazelle/python/testdata/simple_test_with_conftest_sibling_imports_disabled/BUILD.out @@ -0,0 +1,29 @@ +load("@rules_python//python:defs.bzl", "py_library", "py_test") + +# gazelle:python_resolve_sibling_imports false + +py_library( + name = "simple_test_with_conftest_sibling_imports_disabled", + srcs = [ + "__init__.py", + "foo.py", + ], + visibility = ["//:__subpackages__"], +) + +py_library( + name = "conftest", + testonly = True, + srcs = ["conftest.py"], + visibility = ["//:__subpackages__"], +) + +py_test( + name = "simple_test_with_conftest_sibling_imports_disabled_test", + srcs = ["__test__.py"], + main = "__test__.py", + deps = [ + ":conftest", + ":simple_test_with_conftest_sibling_imports_disabled", + ], +) diff --git a/gazelle/python/testdata/simple_test_with_conftest_sibling_imports_disabled/README.md b/gazelle/python/testdata/simple_test_with_conftest_sibling_imports_disabled/README.md new file mode 100644 index 0000000000..98793c23de --- /dev/null +++ b/gazelle/python/testdata/simple_test_with_conftest_sibling_imports_disabled/README.md @@ -0,0 +1,4 @@ +# Simple test with conftest.py (sibling imports disable) + +This test case asserts that a simple `py_test` is generated as expected when a +`conftest.py` is present with sibling imports disabled. diff --git a/gazelle/python/testdata/simple_test_with_conftest_sibling_imports_disabled/WORKSPACE b/gazelle/python/testdata/simple_test_with_conftest_sibling_imports_disabled/WORKSPACE new file mode 100644 index 0000000000..faff6af87a --- /dev/null +++ b/gazelle/python/testdata/simple_test_with_conftest_sibling_imports_disabled/WORKSPACE @@ -0,0 +1 @@ +# This is a Bazel workspace for the Gazelle test data. diff --git a/gazelle/python/testdata/simple_test_with_conftest_sibling_imports_disabled/__init__.py b/gazelle/python/testdata/simple_test_with_conftest_sibling_imports_disabled/__init__.py new file mode 100644 index 0000000000..6a49193fe4 --- /dev/null +++ b/gazelle/python/testdata/simple_test_with_conftest_sibling_imports_disabled/__init__.py @@ -0,0 +1,3 @@ +from foo import foo + +_ = foo diff --git a/gazelle/python/testdata/simple_test_with_conftest_sibling_imports_disabled/__test__.py b/gazelle/python/testdata/simple_test_with_conftest_sibling_imports_disabled/__test__.py new file mode 100644 index 0000000000..d6085a41b4 --- /dev/null +++ b/gazelle/python/testdata/simple_test_with_conftest_sibling_imports_disabled/__test__.py @@ -0,0 +1,12 @@ +import unittest + +from __init__ import foo + + +class FooTest(unittest.TestCase): + def test_foo(self): + self.assertEqual("foo", foo()) + + +if __name__ == "__main__": + unittest.main() diff --git a/gazelle/python/testdata/simple_test_with_conftest_sibling_imports_disabled/bar/BUILD.in b/gazelle/python/testdata/simple_test_with_conftest_sibling_imports_disabled/bar/BUILD.in new file mode 100644 index 0000000000..3f2beb3147 --- /dev/null +++ b/gazelle/python/testdata/simple_test_with_conftest_sibling_imports_disabled/bar/BUILD.in @@ -0,0 +1 @@ +load("@rules_python//python:defs.bzl", "py_library") diff --git a/gazelle/python/testdata/simple_test_with_conftest_sibling_imports_disabled/bar/BUILD.out b/gazelle/python/testdata/simple_test_with_conftest_sibling_imports_disabled/bar/BUILD.out new file mode 100644 index 0000000000..ef8591f199 --- /dev/null +++ b/gazelle/python/testdata/simple_test_with_conftest_sibling_imports_disabled/bar/BUILD.out @@ -0,0 +1,27 @@ +load("@rules_python//python:defs.bzl", "py_library", "py_test") + +py_library( + name = "bar", + srcs = [ + "__init__.py", + "bar.py", + ], + visibility = ["//:__subpackages__"], +) + +py_library( + name = "conftest", + testonly = True, + srcs = ["conftest.py"], + visibility = ["//:__subpackages__"], +) + +py_test( + name = "bar_test", + srcs = ["__test__.py"], + main = "__test__.py", + deps = [ + ":conftest", + "//:simple_test_with_conftest_sibling_imports_disabled", + ], +) diff --git a/gazelle/python/testdata/simple_test_with_conftest_sibling_imports_disabled/bar/__init__.py b/gazelle/python/testdata/simple_test_with_conftest_sibling_imports_disabled/bar/__init__.py new file mode 100644 index 0000000000..0c59205559 --- /dev/null +++ b/gazelle/python/testdata/simple_test_with_conftest_sibling_imports_disabled/bar/__init__.py @@ -0,0 +1,3 @@ +from bar import bar + +_ = bar diff --git a/gazelle/python/testdata/simple_test_with_conftest_sibling_imports_disabled/bar/__test__.py b/gazelle/python/testdata/simple_test_with_conftest_sibling_imports_disabled/bar/__test__.py new file mode 100644 index 0000000000..c3d4734eed --- /dev/null +++ b/gazelle/python/testdata/simple_test_with_conftest_sibling_imports_disabled/bar/__test__.py @@ -0,0 +1,12 @@ +import unittest + +from __init__ import bar + + +class BarTest(unittest.TestCase): + def test_bar(self): + self.assertEqual("bar", bar()) + + +if __name__ == "__main__": + unittest.main() diff --git a/gazelle/python/testdata/simple_test_with_conftest_sibling_imports_disabled/bar/bar.py b/gazelle/python/testdata/simple_test_with_conftest_sibling_imports_disabled/bar/bar.py new file mode 100644 index 0000000000..ee70a51f03 --- /dev/null +++ b/gazelle/python/testdata/simple_test_with_conftest_sibling_imports_disabled/bar/bar.py @@ -0,0 +1,2 @@ +def bar(): + return "bar" diff --git a/gazelle/python/testdata/simple_test_with_conftest_sibling_imports_disabled/bar/conftest.py b/gazelle/python/testdata/simple_test_with_conftest_sibling_imports_disabled/bar/conftest.py new file mode 100644 index 0000000000..e69de29bb2 diff --git a/gazelle/python/testdata/simple_test_with_conftest_sibling_imports_disabled/conftest.py b/gazelle/python/testdata/simple_test_with_conftest_sibling_imports_disabled/conftest.py new file mode 100644 index 0000000000..e69de29bb2 diff --git a/gazelle/python/testdata/simple_test_with_conftest_sibling_imports_disabled/foo.py b/gazelle/python/testdata/simple_test_with_conftest_sibling_imports_disabled/foo.py new file mode 100644 index 0000000000..cf68624419 --- /dev/null +++ b/gazelle/python/testdata/simple_test_with_conftest_sibling_imports_disabled/foo.py @@ -0,0 +1,2 @@ +def foo(): + return "foo" diff --git a/gazelle/python/testdata/simple_test_with_conftest_sibling_imports_disabled/test.yaml b/gazelle/python/testdata/simple_test_with_conftest_sibling_imports_disabled/test.yaml new file mode 100644 index 0000000000..8071ef4094 --- /dev/null +++ b/gazelle/python/testdata/simple_test_with_conftest_sibling_imports_disabled/test.yaml @@ -0,0 +1,4 @@ + +--- +expect: + exit_code: 0 diff --git a/gazelle/python/testdata/subdir_sources/BUILD.in b/gazelle/python/testdata/subdir_sources/BUILD.in index adfdefdc8a..e8f3827bd2 100644 --- a/gazelle/python/testdata/subdir_sources/BUILD.in +++ b/gazelle/python/testdata/subdir_sources/BUILD.in @@ -1 +1,2 @@ # gazelle:python_generation_mode project +# gazelle:python_resolve_sibling_imports true diff --git a/gazelle/python/testdata/subdir_sources/BUILD.out b/gazelle/python/testdata/subdir_sources/BUILD.out index 5d77890d4f..5b96ad7576 100644 --- a/gazelle/python/testdata/subdir_sources/BUILD.out +++ b/gazelle/python/testdata/subdir_sources/BUILD.out @@ -2,6 +2,7 @@ load("@rules_python//python:defs.bzl", "py_binary") # gazelle:python_generation_mode project +# gazelle:python_resolve_sibling_imports true py_binary( name = "subdir_sources_bin", diff --git a/gazelle/pythonconfig/pythonconfig.go b/gazelle/pythonconfig/pythonconfig.go index 001fd334a4..b3d56591ee 100644 --- a/gazelle/pythonconfig/pythonconfig.go +++ b/gazelle/pythonconfig/pythonconfig.go @@ -107,6 +107,11 @@ const ( // GenerateProto represents the directive that controls whether to generate // python_generate_proto targets. GenerateProto = "python_generate_proto" + // PythonResolveSiblingImports represents the directive that controls whether + // absolute imports can be solved to sibling modules. When enabled, imports + // like "import a" can be resolved to sibling modules. When disabled, they + // can only be resolved as an absolute import. + PythonResolveSiblingImports = "python_resolve_sibling_imports" ) // GenerationModeType represents one of the generation modes for the Python @@ -198,6 +203,7 @@ type Config struct { experimentalAllowRelativeImports bool generatePyiDeps bool generateProto bool + resolveSiblingImports bool } type LabelNormalizationType int @@ -237,6 +243,7 @@ func New( experimentalAllowRelativeImports: false, generatePyiDeps: false, generateProto: false, + resolveSiblingImports: false, } } @@ -273,6 +280,7 @@ func (c *Config) NewChild() *Config { experimentalAllowRelativeImports: c.experimentalAllowRelativeImports, generatePyiDeps: c.generatePyiDeps, generateProto: c.generateProto, + resolveSiblingImports: c.resolveSiblingImports, } } @@ -592,6 +600,16 @@ func (c *Config) GenerateProto() bool { return c.generateProto } +// SetResolveSiblingImports sets whether absolute imports can be resolved to sibling modules. +func (c *Config) SetResolveSiblingImports(resolveSiblingImports bool) { + c.resolveSiblingImports = resolveSiblingImports +} + +// ResolveSiblingImports returns whether absolute imports can be resolved to sibling modules. +func (c *Config) ResolveSiblingImports() bool { + return c.resolveSiblingImports +} + // FormatThirdPartyDependency returns a label to a third-party dependency performing all formating and normalization. func (c *Config) FormatThirdPartyDependency(repositoryName string, distributionName string) label.Label { conventionalDistributionName := strings.ReplaceAll(c.labelConvention, distributionNameLabelConventionSubstitution, distributionName) From 0d0ab5cbf6ebd4c7a7c1be44c74df56be12185c9 Mon Sep 17 00:00:00 2001 From: Jaemin Choi <1dotolee@gmail.com> Date: Sat, 2 Aug 2025 08:07:35 +0900 Subject: [PATCH 343/922] fix(pypi): show overridden index urls in pypi download error (#3130) Closes #2985 Suppose invalid `experimental_index_url_overrides` in `pip.parse` is set like below. ```bzl pip.parse( experimental_index_url = "https://pypi.org/simple", experimental_index_url_overrides = {"mypy": "https://invalid.com"}, hub_name = "pypi", requirements_lock = "//:requirements_lock.txt", ) ``` It fails as follows, showing only "pypi.org" as pypi index url, not "invalid.com" for for `mypy` package. ``` Error in fail: Failed to download metadata for ["mypy"] for from urls: ["https://pypi.org/simple"]. If you would like to skip downloading metadata for these packages please add 'simpleapi_skip=["mypy"]' to your 'pip.parse' call. ``` To show overridden url for each package, show url of each package that has been failed to download metadata. The error message with this PR is like below. ``` Error in fail: Failed to download metadata of the following packages from urls: { "mypy": "https://invalid.com", } If you would like to skip downloading metadata for these packages please add 'simpleapi_skip=["mypy"]' to your 'pip.parse' call. ``` --- CHANGELOG.md | 2 ++ python/private/pypi/simpleapi_download.bzl | 26 +++++++++++------ .../simpleapi_download_tests.bzl | 28 +++++++++++++++---- 3 files changed, 41 insertions(+), 15 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index c52481d52e..d21ebc7d36 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -99,6 +99,8 @@ END_UNRELEASED_TEMPLATE absolute imports (Python 2's behavior without `absolute_import`). Previous behavior can be restored using the directive `# gazelle:python_resolve_sibling_imports true` +* (pypi) Show overridden index URL of packages when downloading metadata have failed. + ([#2985](https://github.com/bazel-contrib/rules_python/issues/2985)). {#v0-0-0-added} ### Added diff --git a/python/private/pypi/simpleapi_download.bzl b/python/private/pypi/simpleapi_download.bzl index a3ba9691cd..52ff02a178 100644 --- a/python/private/pypi/simpleapi_download.bzl +++ b/python/private/pypi/simpleapi_download.bzl @@ -128,16 +128,24 @@ def simpleapi_download( failed_sources = [pkg for pkg in attr.sources if pkg not in found_on_index] if failed_sources: + pkg_index_urls = { + pkg: index_url_overrides.get( + normalize_name(pkg), + index_urls, + ) + for pkg in failed_sources + } + _fail( - "\n".join([ - "Failed to download metadata for {} for from urls: {}.".format( - failed_sources, - index_urls, - ), - "If you would like to skip downloading metadata for these packages please add 'simpleapi_skip={}' to your 'pip.parse' call.".format( - render.list(failed_sources), - ), - ]), + """ +Failed to download metadata of the following packages from urls: +{pkg_index_urls} + +If you would like to skip downloading metadata for these packages please add 'simpleapi_skip={failed_sources}' to your 'pip.parse' call. +""".format( + pkg_index_urls = render.dict(pkg_index_urls), + failed_sources = render.list(failed_sources), + ), ) return None diff --git a/tests/pypi/simpleapi_download/simpleapi_download_tests.bzl b/tests/pypi/simpleapi_download/simpleapi_download_tests.bzl index a96815c12c..8dc307235a 100644 --- a/tests/pypi/simpleapi_download/simpleapi_download_tests.bzl +++ b/tests/pypi/simpleapi_download/simpleapi_download_tests.bzl @@ -87,6 +87,11 @@ def _test_fail(env): output = "", success = False, ) + if "bar" in url: + return struct( + output = "", + success = False, + ) else: return struct( output = "data from {}".format(url), @@ -99,7 +104,9 @@ def _test_fail(env): report_progress = lambda _: None, ), attr = struct( - index_url_overrides = {}, + index_url_overrides = { + "foo": "invalid", + }, index_url = "main", extra_index_urls = ["extra"], sources = ["foo", "bar", "baz"], @@ -112,16 +119,25 @@ def _test_fail(env): ) env.expect.that_collection(fails).contains_exactly([ - """\ -Failed to download metadata for ["foo"] for from urls: ["main", "extra"]. -If you would like to skip downloading metadata for these packages please add 'simpleapi_skip=["foo"]' to your 'pip.parse' call.\ + """ +Failed to download metadata of the following packages from urls: +{ + "foo": "invalid", + "bar": ["main", "extra"], +} + +If you would like to skip downloading metadata for these packages please add 'simpleapi_skip=[ + "foo", + "bar", +]' to your 'pip.parse' call. """, ]) env.expect.that_collection(calls).contains_exactly([ - "extra/foo/", + "invalid/foo/", "main/bar/", "main/baz/", - "main/foo/", + "invalid/foo/", + "extra/bar/", ]) _tests.append(_test_fail) From 2c53bf6968e170850237cbd0779337b093f4f94f Mon Sep 17 00:00:00 2001 From: Douglas Thor Date: Fri, 1 Aug 2025 16:08:21 -0700 Subject: [PATCH 344/922] chore: Remove aliases in //docs (#3125) Fixes #2976. The rest of it was fixed in #3045. --- docs/BUILD.bazel | 36 ---------------------------------- sphinxdocs/private/BUILD.bazel | 2 +- 2 files changed, 1 insertion(+), 37 deletions(-) diff --git a/docs/BUILD.bazel b/docs/BUILD.bazel index 852c4d4fa6..c1009b7313 100644 --- a/docs/BUILD.bazel +++ b/docs/BUILD.bazel @@ -12,7 +12,6 @@ # See the License for the specific language governing permissions and # limitations under the License. -load("@bazel_skylib//:bzl_library.bzl", "bzl_library") load("@bazel_skylib//rules:build_test.bzl", "build_test") load("@dev_pip//:requirements.bzl", "requirement") load("//python/private:bzlmod_enabled.bzl", "BZLMOD_ENABLED") # buildifier: disable=bzl-visibility @@ -189,38 +188,3 @@ lock( ], visibility = ["//:__subpackages__"], ) - -# Temporary compatibility aliases for some other projects depending on the old -# bzl_library targets. -alias( - name = "defs", - actual = "//python:defs_bzl", - deprecation = "Use //python:defs_bzl instead; targets under //docs are internal.", - visibility = ["//visibility:public"], -) - -alias( - name = "bazel_repo_tools", - actual = "//python/private:bazel_tools_bzl", - deprecation = "Use @bazel_tools//tools:bzl_srcs instead; targets under //docs are internal.", - visibility = ["//visibility:public"], -) - -bzl_library( - name = "pip_install_bzl", - deprecation = "Use //python:pip_bzl or //python/pip_install:pip_repository_bzl instead; " + - "targets under //docs are internal.", - visibility = ["//visibility:public"], - deps = [ - "//python:pip_bzl", - "//python/pip_install:pip_repository_bzl", - ], -) - -alias( - name = "requirements_parser_bzl", - actual = "//python/pip_install:pip_repository_bzl", - deprecation = "Use //python/pip_install:pip_repository_bzl instead; Both the requirements " + - "parser and targets under //docs are internal", - visibility = ["//visibility:public"], -) diff --git a/sphinxdocs/private/BUILD.bazel b/sphinxdocs/private/BUILD.bazel index c4246ed0de..c707b4d1d8 100644 --- a/sphinxdocs/private/BUILD.bazel +++ b/sphinxdocs/private/BUILD.bazel @@ -13,7 +13,7 @@ # limitations under the License. load("@bazel_skylib//:bzl_library.bzl", "bzl_library") -load("//python:proto.bzl", "py_proto_library") +load("@com_google_protobuf//bazel:py_proto_library.bzl", "py_proto_library") load("//python:py_binary.bzl", "py_binary") load("//python:py_library.bzl", "py_library") From 9889d9f6b06fcd058ccbbea856fa1c90f3bbc216 Mon Sep 17 00:00:00 2001 From: Douglas Thor Date: Fri, 1 Aug 2025 16:16:43 -0700 Subject: [PATCH 345/922] fix(gazelle): Rename experimental_allow_relative_imports directive to follow convention (#3128) Prefix `experimental_allow_relative_imports` with `python_` to match the rest of the directives. 1.6.0 hasn't been released yet, so this is a non-breaking change. --- CHANGELOG.md | 2 +- gazelle/README.md | 4 ++-- .../python/testdata/relative_imports_package_mode/BUILD.in | 2 +- .../python/testdata/relative_imports_package_mode/BUILD.out | 2 +- gazelle/python/testdata/sibling_imports_disabled/BUILD.in | 2 +- gazelle/python/testdata/sibling_imports_disabled/BUILD.out | 2 +- gazelle/python/testdata/sibling_imports_disabled/README.md | 2 +- .../testdata/sibling_imports_disabled_file_mode/BUILD.in | 2 +- .../testdata/sibling_imports_disabled_file_mode/BUILD.out | 2 +- .../testdata/sibling_imports_disabled_file_mode/README.md | 2 +- gazelle/pythonconfig/pythonconfig.go | 2 +- 11 files changed, 12 insertions(+), 12 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index d21ebc7d36..f69e94ec65 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -56,7 +56,7 @@ END_UNRELEASED_TEMPLATE ### Changed * (gazelle) For package mode, resolve dependencies when imports are relative to the package path. This is enabled via the - `# gazelle:experimental_allow_relative_imports` true directive ({gh-issue}`2203`). + `# gazelle:python_experimental_allow_relative_imports` true directive ({gh-issue}`2203`). * (gazelle) Types for exposed members of `python.ParserOutput` are now all public. * (gazelle) Removed the requirement for `__init__.py`, `__main__.py`, or `__test__.py` files to be present in a directory to generate a `BUILD.bazel` file. diff --git a/gazelle/README.md b/gazelle/README.md index 8b088a4e70..83f341c49d 100644 --- a/gazelle/README.md +++ b/gazelle/README.md @@ -222,7 +222,7 @@ Python-specific directives are as follows: | Defines the format of the distribution name in labels to third-party deps. Useful for using Gazelle plugin with other rules with different repository conventions (e.g. `rules_pycross`). Full label is always prepended with (pip) repository name, e.g. `@pip//numpy`. | | `# gazelle:python_label_normalization` | `snake_case` | | Controls how distribution names in labels to third-party deps are normalized. Useful for using Gazelle plugin with other rules with different label conventions (e.g. `rules_pycross` uses PEP-503). Can be "snake_case", "none", or "pep503". | -| `# gazelle:experimental_allow_relative_imports` | `false` | +| `# gazelle:python_experimental_allow_relative_imports` | `false` | | Controls whether Gazelle resolves dependencies for import statements that use paths relative to the current package. Can be "true" or "false".| | `# gazelle:python_generate_pyi_deps` | `false` | | Controls whether to generate a separate `pyi_deps` attribute for type-checking dependencies or merge them into the regular `deps` attribute. When `false` (default), type-checking dependencies are merged into `deps` for backward compatibility. When `true`, generates separate `pyi_deps`. Imports in blocks with the format `if typing.TYPE_CHECKING:`/`if TYPE_CHECKING:` and type-only stub packages (eg. boto3-stubs) are recognized as type-checking dependencies. | @@ -736,7 +736,7 @@ See [Issue #3076][gh3076] for more information. [gh3076]: https://github.com/bazel-contrib/rules_python/issues/3076 -#### Directive: `experimental_allow_relative_imports` +#### Directive: `python_experimental_allow_relative_imports` Enables experimental support for resolving relative imports in `python_generation_mode package`. diff --git a/gazelle/python/testdata/relative_imports_package_mode/BUILD.in b/gazelle/python/testdata/relative_imports_package_mode/BUILD.in index 78ef0a7863..52bcb68600 100644 --- a/gazelle/python/testdata/relative_imports_package_mode/BUILD.in +++ b/gazelle/python/testdata/relative_imports_package_mode/BUILD.in @@ -1,2 +1,2 @@ # gazelle:python_generation_mode package -# gazelle:experimental_allow_relative_imports true +# gazelle:python_experimental_allow_relative_imports true diff --git a/gazelle/python/testdata/relative_imports_package_mode/BUILD.out b/gazelle/python/testdata/relative_imports_package_mode/BUILD.out index f51b516cab..8775c114ef 100644 --- a/gazelle/python/testdata/relative_imports_package_mode/BUILD.out +++ b/gazelle/python/testdata/relative_imports_package_mode/BUILD.out @@ -1,7 +1,7 @@ load("@rules_python//python:defs.bzl", "py_binary") # gazelle:python_generation_mode package -# gazelle:experimental_allow_relative_imports true +# gazelle:python_experimental_allow_relative_imports true py_binary( name = "relative_imports_package_mode_bin", diff --git a/gazelle/python/testdata/sibling_imports_disabled/BUILD.in b/gazelle/python/testdata/sibling_imports_disabled/BUILD.in index 9509fd9727..44f7406e58 100644 --- a/gazelle/python/testdata/sibling_imports_disabled/BUILD.in +++ b/gazelle/python/testdata/sibling_imports_disabled/BUILD.in @@ -1,2 +1,2 @@ # gazelle:python_resolve_sibling_imports false -# gazelle:experimental_allow_relative_imports true +# gazelle:python_experimental_allow_relative_imports true diff --git a/gazelle/python/testdata/sibling_imports_disabled/BUILD.out b/gazelle/python/testdata/sibling_imports_disabled/BUILD.out index 7568f38f50..d3d5c6bfab 100644 --- a/gazelle/python/testdata/sibling_imports_disabled/BUILD.out +++ b/gazelle/python/testdata/sibling_imports_disabled/BUILD.out @@ -1,7 +1,7 @@ load("@rules_python//python:defs.bzl", "py_library", "py_test") # gazelle:python_resolve_sibling_imports false -# gazelle:experimental_allow_relative_imports true +# gazelle:python_experimental_allow_relative_imports true py_library( name = "sibling_imports_disabled", diff --git a/gazelle/python/testdata/sibling_imports_disabled/README.md b/gazelle/python/testdata/sibling_imports_disabled/README.md index d534a44bf1..a39023e8a3 100644 --- a/gazelle/python/testdata/sibling_imports_disabled/README.md +++ b/gazelle/python/testdata/sibling_imports_disabled/README.md @@ -10,7 +10,7 @@ disabled. It covers different types of imports in `pkg/unit_test.py`: (not the sibling `typing.py`). - `from .b import run` / `from .typing import A` - resolves to the sibling `pkg/b.py` / `pkg/typing.py` (with - `gazelle:experimental_allow_relative_imports` enabled) + `gazelle:python_experimental_allow_relative_imports` enabled) - `import test_util` - resolves to the root-level `test_util.py` instead of the sibling `pkg/test_util.py` - `from b import run` - resolves to the root-level `b.py` instead of the diff --git a/gazelle/python/testdata/sibling_imports_disabled_file_mode/BUILD.in b/gazelle/python/testdata/sibling_imports_disabled_file_mode/BUILD.in index 04494394c7..32b0bec20f 100644 --- a/gazelle/python/testdata/sibling_imports_disabled_file_mode/BUILD.in +++ b/gazelle/python/testdata/sibling_imports_disabled_file_mode/BUILD.in @@ -1,3 +1,3 @@ # gazelle:python_generation_mode file # gazelle:python_resolve_sibling_imports false -# gazelle:experimental_allow_relative_imports true +# gazelle:python_experimental_allow_relative_imports true diff --git a/gazelle/python/testdata/sibling_imports_disabled_file_mode/BUILD.out b/gazelle/python/testdata/sibling_imports_disabled_file_mode/BUILD.out index da53e14864..d7a829e8ea 100644 --- a/gazelle/python/testdata/sibling_imports_disabled_file_mode/BUILD.out +++ b/gazelle/python/testdata/sibling_imports_disabled_file_mode/BUILD.out @@ -2,7 +2,7 @@ load("@rules_python//python:defs.bzl", "py_library", "py_test") # gazelle:python_generation_mode file # gazelle:python_resolve_sibling_imports false -# gazelle:experimental_allow_relative_imports true +# gazelle:python_experimental_allow_relative_imports true py_library( name = "a", diff --git a/gazelle/python/testdata/sibling_imports_disabled_file_mode/README.md b/gazelle/python/testdata/sibling_imports_disabled_file_mode/README.md index 0bfbcffb58..124e751b10 100644 --- a/gazelle/python/testdata/sibling_imports_disabled_file_mode/README.md +++ b/gazelle/python/testdata/sibling_imports_disabled_file_mode/README.md @@ -10,7 +10,7 @@ disabled. It covers different types of imports in `pkg/unit_test.py`: (not the sibling `typing.py`). - `from .b import run` / `from .typing import A` - resolves to the sibling `pkg/b.py` / `pkg/typing.py` (with - `gazelle:experimental_allow_relative_imports` enabled) + `gazelle:python_experimental_allow_relative_imports` enabled) - `import test_util` - resolves to the root-level `test_util.py` instead of the sibling `pkg/test_util.py` - `from b import run` - resolves to the root-level `b.py` instead of the diff --git a/gazelle/pythonconfig/pythonconfig.go b/gazelle/pythonconfig/pythonconfig.go index b3d56591ee..ed9b914e82 100644 --- a/gazelle/pythonconfig/pythonconfig.go +++ b/gazelle/pythonconfig/pythonconfig.go @@ -99,7 +99,7 @@ const ( LabelNormalization = "python_label_normalization" // ExperimentalAllowRelativeImports represents the directive that controls // whether relative imports are allowed. - ExperimentalAllowRelativeImports = "experimental_allow_relative_imports" + ExperimentalAllowRelativeImports = "python_experimental_allow_relative_imports" // GeneratePyiDeps represents the directive that controls whether to generate // separate pyi_deps attribute or merge type-checking dependencies into deps. // Defaults to false for backward compatibility. From 6819b844e6fa80c18ebd61090b1c9b0f7abf623b Mon Sep 17 00:00:00 2001 From: Philipp Schrader Date: Fri, 1 Aug 2025 16:19:56 -0700 Subject: [PATCH 346/922] test: Print REPL error message during test failures (#3124) I noticed that the CI error messages for #3114 are not useful. This patch aims to help with that by printing the output of the REPL code. If there's an exception during startup for example, then the test log will now contain the stack trace. --------- Co-authored-by: Ignas Anikevicius <240938+aignas@users.noreply.github.com> --- tests/repl/repl_test.py | 17 ++++++++++------- 1 file changed, 10 insertions(+), 7 deletions(-) diff --git a/tests/repl/repl_test.py b/tests/repl/repl_test.py index 37c9a37a0d..01d0442922 100644 --- a/tests/repl/repl_test.py +++ b/tests/repl/repl_test.py @@ -29,13 +29,16 @@ def setUp(self): def run_code_in_repl(self, lines: Iterable[str], *, env=None) -> str: """Runs the lines of code in the REPL and returns the text output.""" - return subprocess.check_output( - [self.repl], - text=True, - stderr=subprocess.STDOUT, - input="\n".join(lines), - env=env, - ).strip() + try: + return subprocess.check_output( + [self.repl], + text=True, + stderr=subprocess.STDOUT, + input="\n".join(lines), + env=env, + ).strip() + except subprocess.CalledProcessError as error: + raise RuntimeError(f"Failed to run the REPL:\n{error.stdout}") from error def test_repl_version(self): """Validates that we can successfully execute arbitrary code on the REPL.""" From 39703fa18135fae3bcc458490a0b8266ca85e513 Mon Sep 17 00:00:00 2001 From: Douglas Thor Date: Fri, 1 Aug 2025 21:58:28 -0700 Subject: [PATCH 347/922] docs(gazelle): Start migrating Gazelle docs to ReadTheDocs, part 1 of ~5 (#3129) Part of #3082 First of probably 5 PRs. + Set up the Bazel config so that the rules_python root docs can reference and use the gazelle/docs directory. + Replace the original `docs/gazelle.md` with `gazelle/docs/index.md` + Migrate general info from `gazelle/README.md` to `gazelle/docs/index.md` + Mechanical updates: + Wrap at ~80 chars + Use MyST directives and roles. + Also a drive-by update to building and running docs _without_ `ibazel`. --- docs/BUILD.bazel | 2 +- docs/README.md | 14 +++++++++++++ docs/gazelle.md | 9 -------- docs/index.md | 2 +- gazelle/README.md | 29 ++++---------------------- gazelle/docs/BUILD.bazel | 5 +++++ gazelle/docs/index.md | 45 ++++++++++++++++++++++++++++++++++++++++ 7 files changed, 70 insertions(+), 36 deletions(-) delete mode 100644 docs/gazelle.md create mode 100644 gazelle/docs/BUILD.bazel create mode 100644 gazelle/docs/index.md diff --git a/docs/BUILD.bazel b/docs/BUILD.bazel index c1009b7313..fdb74f9407 100644 --- a/docs/BUILD.bazel +++ b/docs/BUILD.bazel @@ -55,7 +55,7 @@ sphinx_docs( "_*", "*.inv*", ], - ), + ) + ["//gazelle/docs"], config = "conf.py", formats = [ "html", diff --git a/docs/README.md b/docs/README.md index 456f1cfd64..1316d733bb 100644 --- a/docs/README.md +++ b/docs/README.md @@ -28,6 +28,20 @@ changes and re-run the build process, and you can simply refresh your browser to see the changes. Using ibazel is not required; you can manually run the equivalent bazel command if desired. +An alternative to `ibazel` is using `inotify` on Linux systems: + +``` +inotifywait --event modify --monitor . --recursive --includei '^.*\.md$' | +while read -r dir events filename; do bazel build //docs:docs; done; +``` + +And lastly, a poor-man's `ibazel` and `inotify` is simply `watch` with +a reasonable interval like 10s: + +``` +watch --interval 10 bazel build //docs:docs +``` + ### Installing ibazel The `ibazel` tool can be used to automatically rebuild the docs as you diff --git a/docs/gazelle.md b/docs/gazelle.md deleted file mode 100644 index 60b46faf2c..0000000000 --- a/docs/gazelle.md +++ /dev/null @@ -1,9 +0,0 @@ -# Gazelle plugin - -[Gazelle](https://github.com/bazelbuild/bazel-gazelle) -is a build file generator for Bazel projects. It can create new `BUILD.bazel` files for a project that follows language conventions and update existing build files to include new sources, dependencies, and options. - -Bazel may run Gazelle using the Gazelle rule, or Gazelle may be installed and run as a command line tool. - -See the documentation for Gazelle with `rules_python` in the {gh-path}`gazelle` -directory. diff --git a/docs/index.md b/docs/index.md index 25b423c6c3..bdc6982ad5 100644 --- a/docs/index.md +++ b/docs/index.md @@ -99,7 +99,7 @@ pypi/index Toolchains coverage precompiling -gazelle +gazelle/docs/index REPL Extending Contributing diff --git a/gazelle/README.md b/gazelle/README.md index 83f341c49d..df3085bb37 100644 --- a/gazelle/README.md +++ b/gazelle/README.md @@ -1,19 +1,10 @@ # Python Gazelle plugin -[Gazelle](https://github.com/bazelbuild/bazel-gazelle) -is a build file generator for Bazel projects. It can create new BUILD.bazel files for a project that follows language conventions, and it can update existing build files to include new sources, dependencies, and options. +:::{note} +The gazelle plugin docs are being migrated to our primary documentation on +ReadTheDocs. Please see https://rules-python.readthedocs.io/gazelle/docs/index.html. +::: -Gazelle may be run by Bazel using the gazelle rule, or it may be installed and run as a command line tool. - -This directory contains a plugin for -[Gazelle](https://github.com/bazelbuild/bazel-gazelle) -that generates BUILD files content for Python code. When Gazelle is run as a command line tool with this plugin, it embeds a Python interpreter resolved during the plugin build. -The behavior of the plugin is slightly different with different version of the interpreter as the Python `stdlib` changes with every minor version release. -Distributors of Gazelle binaries should, therefore, build a Gazelle binary for each OS+CPU architecture+Minor Python version combination they are targeting. - -The following instructions are for when you use [bzlmod](https://docs.bazel.build/versions/5.0.0/bzlmod.html). -Please refer to older documentation that includes instructions on how to use Gazelle -without using bzlmod as your dependency manager. ## Example @@ -153,18 +144,6 @@ gazelle( That's it, now you can finally run `bazel run //:gazelle` anytime you edit Python code, and it should update your `BUILD` files correctly. -## Usage - -Gazelle is non-destructive. -It will try to leave your edits to BUILD files alone, only making updates to `py_*` targets. -However it will remove dependencies that appear to be unused, so it's a -good idea to check in your work before running Gazelle so you can easily -revert any changes it made. - -The rules_python extension assumes some conventions about your Python code. -These are noted below, and might require changes to your existing code. - -Note that the `gazelle` program has multiple commands. At present, only the `update` command (the default) does anything for Python code. ### Directives diff --git a/gazelle/docs/BUILD.bazel b/gazelle/docs/BUILD.bazel new file mode 100644 index 0000000000..7c6b6fd56e --- /dev/null +++ b/gazelle/docs/BUILD.bazel @@ -0,0 +1,5 @@ +filegroup( + name = "docs", + srcs = glob(["*.md"]), + visibility = ["//visibility:public"], +) diff --git a/gazelle/docs/index.md b/gazelle/docs/index.md new file mode 100644 index 0000000000..ea20e9c3e0 --- /dev/null +++ b/gazelle/docs/index.md @@ -0,0 +1,45 @@ +# Gazelle Plugin + +[Gazelle][gazelle] is a build file generator for Bazel projects. It can +create new `BUILD` or `BUILD.bazel` files for a project that +follows language conventions and update existing build files to include new +sources, dependencies, and options. + +[gazelle]: https://github.com/bazel-contrib/bazel-gazelle + +Bazel may run Gazelle using the Gazelle rule, or Gazelle may be installed and run +as a command line tool. + +The {gh-path}`gazelle` directory contains a plugin for Gazelle +that generates `BUILD` files content for Python code. When Gazelle is +run as a command line tool with this plugin, it embeds a Python interpreter +resolved during the plugin build. The behavior of the plugin is slightly +different with different version of the interpreter as the Python +`stdlib` changes with every minor version release. Distributors of Gazelle +binaries should, therefore, build a Gazelle binary for each OS+CPU +architecture+Minor Python version combination they are targeting. + +:::{note} +These instructions are for when you use [bzlmod][bzlmod]. Please refer to +older documentation that includes instructions on how to use Gazelle +without using bzlmod as your dependency manager. +::: + +[bzlmod]: https://bazel.build/external/module + +Gazelle is non-destructive. It will try to leave your edits to `BUILD` +files alone, only making updates to `py_*` targets. However it **will +remove** dependencies that appear to be unused, so it's a good idea to check +in your work before running Gazelle so you can easily revert any changes it made. + +The `rules_python` extension assumes some conventions about your Python code. +These are noted in the subsequent documents, and might require changes to your +existing code. + +Note that the `gazelle` program has multiple commands. At present, only +the `update` command (the default) does anything for Python code. + + +```{toctree} +:maxdepth: 1 +``` From 2fc5f1cc5635be08dbebd0e0c448afb67c3f251e Mon Sep 17 00:00:00 2001 From: Jonathan Woodbury Date: Sun, 3 Aug 2025 03:14:51 -0400 Subject: [PATCH 348/922] feat(repl): add tab completion on platforms with readline support (#3114) This adds tab completion to the default stub when using the REPL feature. However, the feature only works in environments with `readline` support, which means that with the bundled toolchains, Windows will not have tab completion. Work towards #3090 --------- Co-authored-by: Ignas Anikevicius <240938+aignas@users.noreply.github.com> --- CHANGELOG.md | 3 +++ python/bin/repl_stub.py | 44 +++++++++++++++++++++++++++++++++++++++++ 2 files changed, 47 insertions(+) diff --git a/CHANGELOG.md b/CHANGELOG.md index f69e94ec65..422e399026 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -104,6 +104,9 @@ END_UNRELEASED_TEMPLATE {#v0-0-0-added} ### Added +* (repl) Default stub now has tab completion, where `readline` support is available, + see ([#3114](https://github.com/bazel-contrib/rules_python/pull/3114)). + ([#3114](https://github.com/bazel-contrib/rules_python/pull/3114)). * (pypi) To configure the environment for `requirements.txt` evaluation, use the newly added developer preview of the `pip.default` tag class. Only `rules_python` and root modules can use this feature. You can also configure custom `config_settings` using `pip.default`. diff --git a/python/bin/repl_stub.py b/python/bin/repl_stub.py index 1e21b26dc3..f5b7c0aa4f 100644 --- a/python/bin/repl_stub.py +++ b/python/bin/repl_stub.py @@ -17,8 +17,28 @@ console_locals = globals().copy() import code +import rlcompleter import sys + +class DynamicCompleter(rlcompleter.Completer): + """ + A custom completer that dynamically updates its namespace to include new + imports made within the interactive session. + """ + + def __init__(self, namespace): + # Store a reference to the namespace, not a copy, so that changes to the namespace are + # reflected. + self.namespace = namespace + + def complete(self, text, state): + # Update the completer's internal namespace with the current interactive session's locals + # and globals. This is the key to making new imports discoverable. + rlcompleter.Completer.__init__(self, self.namespace) + return super().complete(text, state) + + if sys.stdin.isatty(): # Use the default options. exitmsg = None @@ -28,5 +48,29 @@ sys.ps1 = "" sys.ps2 = "" +# Set up tab completion. +try: + import readline + + completer = DynamicCompleter(console_locals) + readline.set_completer(completer.complete) + + # TODO(jpwoodbu): Use readline.backend instead of readline.__doc__ once we can depend on having + # Python >=3.13. + if "libedit" in readline.__doc__: # type: ignore + readline.parse_and_bind("bind ^I rl_complete") + elif "GNU readline" in readline.__doc__: # type: ignore + readline.parse_and_bind("tab: complete") + else: + print( + "Could not enable tab completion: " + "unable to determine readline backend" + ) +except ImportError: + print( + "Could not enable tab completion: " + "readline module not available on this platform" + ) + # We set the banner to an empty string because the repl_template.py file already prints the banner. code.interact(local=console_locals, banner="", exitmsg=exitmsg) From 54397d71031de37fbc810922244e0cc4788713fd Mon Sep 17 00:00:00 2001 From: Douglas Thor Date: Sun, 3 Aug 2025 21:11:20 -0700 Subject: [PATCH 349/922] docs(gazelle): Migrate Gazelle docs to ReadTheDocs, part 2/5: installation and usage (#3132) Part of #3082 2nd of probably 5 PRs. + Migrate installation and usage info from `gazelle/README.md` to `gazelle/docs/installation_and_usage.md` + Slight rewording and reformatting of the `Example` section. + Reorganized and modernized the `MODULE.bazel` and `BUILD.bazel` examples: + less details on rules_python as that's available in other docs + default to gazelle multilang support now that #3057 is merged. + Mechanical updates: + Wrap at ~80 chars + Use MyST directives and roles. --- gazelle/README.md | 139 ----------------------- gazelle/docs/index.md | 1 + gazelle/docs/installation_and_usage.md | 151 +++++++++++++++++++++++++ 3 files changed, 152 insertions(+), 139 deletions(-) create mode 100644 gazelle/docs/installation_and_usage.md diff --git a/gazelle/README.md b/gazelle/README.md index df3085bb37..11a9e5b2ba 100644 --- a/gazelle/README.md +++ b/gazelle/README.md @@ -6,145 +6,6 @@ ReadTheDocs. Please see https://rules-python.readthedocs.io/gazelle/docs/index.h ::: -## Example - -We have an example of using Gazelle with Python located [here](https://github.com/bazel-contrib/rules_python/tree/main/examples/bzlmod). -A fully-working example without using bzlmod is in [`examples/build_file_generation`](../examples/build_file_generation). - -The following documentation covers using bzlmod. - -## Adding Gazelle to your project - -First, you'll need to add Gazelle to your `MODULE.bazel` file. -Get the current version of Gazelle from there releases here: https://github.com/bazelbuild/bazel-gazelle/releases/. - - -See the installation `MODULE.bazel` snippet on the Releases page: -https://github.com/bazel-contrib/rules_python/releases in order to configure rules_python. - -You will also need to add the `bazel_dep` for configuration for `rules_python_gazelle_plugin`. - -Here is a snippet of a `MODULE.bazel` file. - -```starlark -# The following stanza defines the dependency rules_python. -bazel_dep(name = "rules_python", version = "0.22.0") - -# The following stanza defines the dependency rules_python_gazelle_plugin. -# For typical setups you set the version. -bazel_dep(name = "rules_python_gazelle_plugin", version = "0.22.0") - -# The following stanza defines the dependency gazelle. -bazel_dep(name = "gazelle", version = "0.31.0", repo_name = "bazel_gazelle") - -# Import the python repositories generated by the given module extension into the scope of the current module. -use_repo(python, "python3_9") -use_repo(python, "python3_9_toolchains") - -# Register an already-defined toolchain so that Bazel can use it during toolchain resolution. -register_toolchains( - "@python3_9_toolchains//:all", -) - -# Use the pip extension -pip = use_extension("@rules_python//python:extensions.bzl", "pip") - -# Use the extension to call the `pip_repository` rule that invokes `pip`, with `incremental` set. -# Accepts a locked/compiled requirements file and installs the dependencies listed within. -# Those dependencies become available in a generated `requirements.bzl` file. -# You can instead check this `requirements.bzl` file into your repo. -# Because this project has different requirements for windows vs other -# operating systems, we have requirements for each. -pip.parse( - name = "pip", - requirements_lock = "//:requirements_lock.txt", - requirements_windows = "//:requirements_windows.txt", -) - -# Imports the pip toolchain generated by the given module extension into the scope of the current module. -use_repo(pip, "pip") -``` -Next, we'll fetch metadata about your Python dependencies, so that gazelle can -determine which package a given import statement comes from. This is provided -by the `modules_mapping` rule. We'll make a target for consuming this -`modules_mapping`, and writing it as a manifest file for Gazelle to read. -This is checked into the repo for speed, as it takes some time to calculate -in a large monorepo. - -Gazelle will walk up the filesystem from a Python file to find this metadata, -looking for a file called `gazelle_python.yaml` in an ancestor folder of the Python code. -Create an empty file with this name. It might be next to your `requirements.txt` file. -(You can just use `touch` at this point, it just needs to exist.) - -To keep the metadata updated, put this in your `BUILD.bazel` file next to `gazelle_python.yaml`: - -```starlark -load("@pip//:requirements.bzl", "all_whl_requirements") -load("@rules_python_gazelle_plugin//manifest:defs.bzl", "gazelle_python_manifest") -load("@rules_python_gazelle_plugin//modules_mapping:def.bzl", "modules_mapping") - -# This rule fetches the metadata for python packages we depend on. That data is -# required for the gazelle_python_manifest rule to update our manifest file. -modules_mapping( - name = "modules_map", - wheels = all_whl_requirements, -) - -# Gazelle python extension needs a manifest file mapping from -# an import to the installed package that provides it. -# This macro produces two targets: -# - //:gazelle_python_manifest.update can be used with `bazel run` -# to recalculate the manifest -# - //:gazelle_python_manifest.test is a test target ensuring that -# the manifest doesn't need to be updated -gazelle_python_manifest( - name = "gazelle_python_manifest", - modules_mapping = ":modules_map", - # This is what we called our `pip_parse` rule, where third-party - # python libraries are loaded in BUILD files. - pip_repository_name = "pip", - # This should point to wherever we declare our python dependencies - # (the same as what we passed to the modules_mapping rule in WORKSPACE) - # This argument is optional. If provided, the `.test` target is very - # fast because it just has to check an integrity field. If not provided, - # the integrity field is not added to the manifest which can help avoid - # merge conflicts in large repos. - requirements = "//:requirements_lock.txt", - # include_stub_packages: bool (default: False) - # If set to True, this flag automatically includes any corresponding type stub packages - # for the third-party libraries that are present and used. For example, if you have - # `boto3` as a dependency, and this flag is enabled, the corresponding `boto3-stubs` - # package will be automatically included in the BUILD file. - # - # Enabling this feature helps ensure that type hints and stubs are readily available - # for tools like type checkers and IDEs, improving the development experience and - # reducing manual overhead in managing separate stub packages. - include_stub_packages = True -) -``` - -Finally, you create a target that you'll invoke to run the Gazelle tool -with the rules_python extension included. This typically goes in your root -`/BUILD.bazel` file: - -```starlark -load("@bazel_gazelle//:def.bzl", "gazelle") - -# Our gazelle target points to the python gazelle binary. -# This is the simple case where we only need one language supported. -# If you also had proto, go, or other gazelle-supported languages, -# you would also need a gazelle_binary rule. -# See https://github.com/bazelbuild/bazel-gazelle/blob/master/extend.rst#example -gazelle( - name = "gazelle", - gazelle = "@rules_python_gazelle_plugin//python:gazelle_binary", -) -``` - -That's it, now you can finally run `bazel run //:gazelle` anytime -you edit Python code, and it should update your `BUILD` files correctly. - - ### Directives You can configure the extension using directives, just like for other diff --git a/gazelle/docs/index.md b/gazelle/docs/index.md index ea20e9c3e0..c04efd6a41 100644 --- a/gazelle/docs/index.md +++ b/gazelle/docs/index.md @@ -42,4 +42,5 @@ the `update` command (the default) does anything for Python code. ```{toctree} :maxdepth: 1 +installation_and_usage ``` diff --git a/gazelle/docs/installation_and_usage.md b/gazelle/docs/installation_and_usage.md new file mode 100644 index 0000000000..e764957581 --- /dev/null +++ b/gazelle/docs/installation_and_usage.md @@ -0,0 +1,151 @@ +# Installation and Usage + +## Example + +Examples of using Gazelle with Python can be found in the `rules_python` +repo: + +* bzlmod: {gh-path}`examples/bzlmod_build_file_generation` +* WORKSPACE: {gh-path}`examples/build_file_generation` + +:::{note} +The following documentation covers using bzlmod. +::: + + +## Adding Gazelle to your project + +First, you'll need to add Gazelle to your `MODULE.bazel` file. Get the current +version of [Gazelle][bcr-gazelle] from the [Bazel Central Registry][bcr]. Then +do the same for [`rules_python`][bcr-rules-python] and +[`rules_python_gazelle_plugin`][bcr-rules-python-gazelle-plugin]. + +[bcr-gazelle]: https://registry.bazel.build/modules/gazelle +[bcr]: https://registry.bazel.build/ +[bcr-rules-python]: https://registry.bazel.build/modules/rules_python +[bcr-rules-python-gazelle-plugin]: https://registry.bazel.build/modules/rules_python_gazelle_plugin + +Here is a snippet of a `MODULE.bazel` file. Note that most of it is just +general config for `rules_python` itself - the Gazelle plugin is only two lines +at the end. + +```starlark +################################################ +## START rules_python CONFIG ## +## See the main rules_python docs for details ## +################################################ +bazel_dep(name = "rules_python", version = "1.5.1") + +python = use_extension("@rules_python//python/extensions:python.bzl", "python") +python.toolchain(python_version = "3.12.2") +use_repo(python, "python_3_12_2") + +pip = use_extension("@rules_python//python:extensions.bzl", "pip") +pip.parse( + hub_name = "pip", + requirements_lock = "//:requirements_lock.txt", + requirements_windows = "//:requirements_windows.txt", +) +use_repo(pip, "pip") + +############################################## +## START rules_python_gazelle_plugin CONFIG ## +############################################## + +# The Gazelle plugin depends on Gazelle. +bazel_dep(name = "gazelle", version = "0.33.0", repo_name = "bazel_gazelle") + +# Typically rules_python_gazelle_plugin is version matched to rules_python. +bazel_dep(name = "rules_python_gazelle_plugin", version = "1.5.1") +``` + +Next, we'll fetch metadata about your Python dependencies, so that gazelle can +determine which package a given import statement comes from. This is provided +by the `modules_mapping` rule. We'll make a target for consuming this +`modules_mapping`, and writing it as a manifest file for Gazelle to read. +This is checked into the repo for speed, as it takes some time to calculate +in a large monorepo. + +Gazelle will walk up the filesystem from a Python file to find this metadata, +looking for a file called `gazelle_python.yaml` in an ancestor folder +of the Python code. Create an empty file with this name. It might be next +to your `requirements.txt` file. (You can just use {command}`touch` at +this point, it just needs to exist.) + +To keep the metadata updated, put this in your `BUILD.bazel` file next +to `gazelle_python.yaml`: + +```starlark +# `@pip` is the hub_name from pip.parse in MODULE.bazel. +load("@pip//:requirements.bzl", "all_whl_requirements") +load("@rules_python_gazelle_plugin//manifest:defs.bzl", "gazelle_python_manifest") +load("@rules_python_gazelle_plugin//modules_mapping:def.bzl", "modules_mapping") + +# This rule fetches the metadata for python packages we depend on. That data is +# required for the gazelle_python_manifest rule to update our manifest file. +modules_mapping( + name = "modules_map", + wheels = all_whl_requirements, +) + +# Gazelle python extension needs a manifest file mapping from +# an import to the installed package that provides it. +# This macro produces two targets: +# - //:gazelle_python_manifest.update can be used with `bazel run` +# to recalculate the manifest +# - //:gazelle_python_manifest.test is a test target ensuring that +# the manifest doesn't need to be updated +gazelle_python_manifest( + name = "gazelle_python_manifest", + modules_mapping = ":modules_map", + + # This is what we called our `pip.parse` rule in MODULE.bazel, where third-party + # python libraries are loaded in BUILD files. + pip_repository_name = "pip", + + # This should point to wherever we declare our python dependencies + # (the same as what we passed to the modules_mapping rule in WORKSPACE) + # This argument is optional. If provided, the `.test` target is very + # fast because it just has to check an integrity field. If not provided, + # the integrity field is not added to the manifest which can help avoid + # merge conflicts in large repos. + requirements = "//:requirements_lock.txt", + + # include_stub_packages: bool (default: False) + # If set to True, this flag automatically includes any corresponding type stub packages + # for the third-party libraries that are present and used. For example, if you have + # `boto3` as a dependency, and this flag is enabled, the corresponding `boto3-stubs` + # package will be automatically included in the BUILD file. + # Enabling this feature helps ensure that type hints and stubs are readily available + # for tools like type checkers and IDEs, improving the development experience and + # reducing manual overhead in managing separate stub packages. + include_stub_packages = True +) +``` + +Finally, you create a target that you'll invoke to run the Gazelle tool +with the `rules_python` extension included. This typically goes in your root +`/BUILD.bazel` file: + +```starlark +load("@bazel_gazelle//:def.bzl", "gazelle", "gazelle_binary") + +gazelle_binary( + name = "gazelle_multilang", + languages = [ + # List of language plugins. + # If you want to generate py_proto_library targets PR #3057), then + # the proto language plugin _must_ come before the rules_python plugin. + #"@bazel_gazelle//lanugage/proto", + "@rules_python_gazelle_plugin//python", + ], +) + +gazelle( + name = "gazelle", + gazelle = ":gazelle_multilang", +) +``` + +That's it, now you can finally run `bazel run //:gazelle` anytime +you edit Python code, and it should update your `BUILD` files correctly. From 3c88a5bec26351da1daa1a331f094f11e68e7664 Mon Sep 17 00:00:00 2001 From: Douglas Thor Date: Mon, 4 Aug 2025 09:59:57 -0700 Subject: [PATCH 350/922] docs(gazelle): Migrate Gazelle docs to ReadTheDocs, part 3/5: annotations (#3137) Part of #3082 3rd of probably 5 PRs. + Migrate annotations docs from `gazelle/README.md` to `gazelle/docs/annotations.md` + Switch from table-based summary to bulleted lists + This will be much easier to maintain going forward. + Mechanical updates: + Wrap at ~80 chars + Use MyST directives and roles. --- gazelle/README.md | 185 ---------------------------------- gazelle/docs/annotations.md | 194 ++++++++++++++++++++++++++++++++++++ gazelle/docs/index.md | 1 + 3 files changed, 195 insertions(+), 185 deletions(-) create mode 100644 gazelle/docs/annotations.md diff --git a/gazelle/README.md b/gazelle/README.md index 11a9e5b2ba..efc7004eaf 100644 --- a/gazelle/README.md +++ b/gazelle/README.md @@ -390,191 +390,6 @@ py_proto_library( When `false`, Gazelle will ignore any `py_proto_library`, including previously-generated or hand-created rules. -### Annotations - -*Annotations* refer to comments found _within Python files_ that configure how -Gazelle acts for that particular file. - -Annotations have the form: - -```python -# gazelle:annotation_name value -``` - -and can reside anywhere within a Python file where comments are valid. For example: - -```python -import foo -# gazelle:annotation_name value - -def bar(): # gazelle:annotation_name value - pass -``` - -The annotations are: - -| **Annotation** | **Default value** | -|---------------------------------------------------------------|-------------------| -| [`# gazelle:ignore imports`](#annotation-ignore) | N/A | -| Tells Gazelle to ignore import statements. `imports` is a comma-separated list of imports to ignore. | | -| [`# gazelle:include_dep targets`](#annotation-include_dep) | N/A | -| Tells Gazelle to include a set of dependencies, even if they are not imported in a Python module. `targets` is a comma-separated list of target names to include as dependencies. | | -| [`# gazelle:include_pytest_conftest bool`](#annotation-include_pytest_conftest) | N/A | -| Whether or not to include a sibling `:conftest` target in the deps of a `py_test` target. Default behaviour is to include `:conftest`. | | - - -#### Annotation: `ignore` - -This annotation accepts a comma-separated string of values. Values are names of Python -imports that Gazelle should _not_ include in target dependencies. - -The annotation can be added multiple times, and all values are combined and -de-duplicated. - -For `python_generation_mode = "package"`, the `ignore` annotations -found across all files included in the generated target are removed from `deps`. - -Example: - -```python -import numpy # a pypi package - -# gazelle:ignore bar.baz.hello,foo -import bar.baz.hello -import foo - -# Ignore this import because _reasons_ -import baz # gazelle:ignore baz -``` - -will cause Gazelle to generate: - -```starlark -deps = ["@pypi//numpy"], -``` - - -#### Annotation: `include_dep` - -This annotation accepts a comma-separated string of values. Values _must_ -be Python targets, but _no validation is done_. If a value is not a Python -target, building will result in an error saying: - -``` - does not have mandatory providers: 'PyInfo' or 'CcInfo' or 'PyInfo'. -``` - -Adding non-Python targets to the generated target is a feature request being -tracked in [Issue #1865](https://github.com/bazel-contrib/rules_python/issues/1865). - -The annotation can be added multiple times, and all values are combined -and de-duplicated. - -For `python_generation_mode = "package"`, the `include_dep` annotations -found across all files included in the generated target are included in `deps`. - -Example: - -```python -# gazelle:include_dep //foo:bar,:hello_world,//:abc -# gazelle:include_dep //:def,//foo:bar -import numpy # a pypi package -``` - -will cause Gazelle to generate: - -```starlark -deps = [ - ":hello_world", - "//:abc", - "//:def", - "//foo:bar", - "@pypi//numpy", -] -``` - -#### Annotation: `include_pytest_conftest` - -Added in [#3080][gh3080]. - -[gh3080]: https://github.com/bazel-contrib/rules_python/pull/3080 - -This annotation accepts any string that can be parsed by go's -[`strconv.ParseBool`][ParseBool]. If an unparsable string is passed, the -annotation is ignored. - -[ParseBool]: https://pkg.go.dev/strconv#ParseBool - -Starting with [`rules_python` 0.14.0][rules-python-0.14.0] (specifically [PR #879][gh879]), -Gazelle will include a `:conftest` dependency to an `py_test` target that is in -the same directory as `conftest.py`. - -[rules-python-0.14.0]: https://github.com/bazel-contrib/rules_python/releases/tag/0.14.0 -[gh879]: https://github.com/bazel-contrib/rules_python/pull/879 - -This annotation allows users to adjust that behavior. To disable the behavior, set -the annotation value to "false": - -``` -# some_file_test.py -# gazelle:include_pytest_conftest false -``` - -Example: - -Given a directory tree like: - -``` -. -├── BUILD.bazel -├── conftest.py -└── some_file_test.py -``` - -The default Gazelle behavior would create: - -```starlark -py_library( - name = "conftest", - testonly = True, - srcs = ["conftest.py"], - visibility = ["//:__subpackages__"], -) - -py_test( - name = "some_file_test", - srcs = ["some_file_test.py"], - deps = [":conftest"], -) -``` - -When `# gazelle:include_pytest_conftest false` is found in `some_file_test.py` - -```python -# some_file_test.py -# gazelle:include_pytest_conftest false -``` - -Gazelle will generate: - -```starlark -py_library( - name = "conftest", - testonly = True, - srcs = ["conftest.py"], - visibility = ["//:__subpackages__"], -) - -py_test( - name = "some_file_test", - srcs = ["some_file_test.py"], -) -``` - -See [Issue #3076][gh3076] for more information. - -[gh3076]: https://github.com/bazel-contrib/rules_python/issues/3076 - #### Directive: `python_experimental_allow_relative_imports` Enables experimental support for resolving relative imports in diff --git a/gazelle/docs/annotations.md b/gazelle/docs/annotations.md new file mode 100644 index 0000000000..cc87543c29 --- /dev/null +++ b/gazelle/docs/annotations.md @@ -0,0 +1,194 @@ +# Annotations + +*Annotations* refer to comments found _within Python files_ that configure how +Gazelle acts for that particular file. + +Annotations have the form: + +```python +# gazelle:annotation_name value +``` + +and can reside anywhere within a Python file where comments are valid. For example: + +```python +import foo +# gazelle:annotation_name value + +def bar(): # gazelle:annotation_name value + pass +``` + +The annotations are: + +* [`# gazelle:ignore imports`](#ignore) + * Default: n/a + * Allowed Values: A comma-separated string of python package names + * Tells Gazelle to ignore import statements. `imports` is a comma-separated + list of imports to ignore. +* [`# gazelle:include_dep targets`](#include-dep) + * Default: n/a + * Allowed Values: A string + * Tells Gazelle to include a set of dependencies, even if they are not imported + in a Python module. `targets` is a comma-separated list of target names + to include as dependencies. +* [`# gazelle:include_pytest_conftest bool`](#include-pytest-conftest) + * Default: n/a + * Allowed Values: `true`, `false` + * Whether or not to include a sibling `:conftest` target in the `deps` + of a {bzl:obj}`py_test` target. The default behaviour is to include `:conftest` + (i.e.: `# gazelle:include_pytest_conftest true`). + + +## `ignore` + +This annotation accepts a comma-separated string of values. Values are names of +Python imports that Gazelle should _not_ include in target dependencies. + +The annotation can be added multiple times, and all values are combined and +de-duplicated. + +For `python_generation_mode = "package"`, the `ignore` annotations +found across all files included in the generated target are removed from +`deps`. + +### Example: + +```python +import numpy # a pypi package + +# gazelle:ignore bar.baz.hello,foo +import bar.baz.hello +import foo + +# Ignore this import because _reasons_ +import baz # gazelle:ignore baz +``` + +will cause Gazelle to generate: + +```starlark +deps = ["@pypi//numpy"], +``` + + +## `include_dep` + +This annotation accepts a comma-separated string of values. Values _must_ +be Python targets, but _no validation is done_. If a value is not a Python +target, building will result in an error saying: + +``` + does not have mandatory providers: 'PyInfo' or 'CcInfo' or 'PyInfo'. +``` + +Adding non-Python targets to the generated target is a feature request being +tracked in {gh-issue}`1865`. + +The annotation can be added multiple times, and all values are combined +and de-duplicated. + +For `python_generation_mode = "package"`, the `include_dep` annotations +found across all files included in the generated target are included in +`deps`. + +### Example: + +```python +# gazelle:include_dep //foo:bar,:hello_world,//:abc +# gazelle:include_dep //:def,//foo:bar +import numpy # a pypi package +``` + +will cause Gazelle to generate: + +```starlark +deps = [ + ":hello_world", + "//:abc", + "//:def", + "//foo:bar", + "@pypi//numpy", +] +``` + + +## `include_pytest_conftest` + +:::{versionadded} VERSION_NEXT_FEATURE +{gh-pr}`3080` +::: + +This annotation accepts any string that can be parsed by go's +[`strconv.ParseBool`][ParseBool]. If an unparsable string is passed, the +annotation is ignored. + +[ParseBool]: https://pkg.go.dev/strconv#ParseBool + +Starting with [`rules_python` 0.14.0][rules-python-0.14.0] (specifically +{gh-pr}`879`), Gazelle will include a `:conftest` dependency to a +{bzl:obj}`py_test` target that is in the same directory as `conftest.py`. + +[rules-python-0.14.0]: https://github.com/bazel-contrib/rules_python/releases/tag/0.14.0 + +This annotation allows users to adjust that behavior. To disable the behavior, +set the annotation value to `false`: + +``` +# some_file_test.py +# gazelle:include_pytest_conftest false +``` + +### Example: + +Given a directory tree like: + +``` +. +├── BUILD.bazel +├── conftest.py +└── some_file_test.py +``` + +The default Gazelle behavior would create: + +```starlark +py_library( + name = "conftest", + testonly = True, + srcs = ["conftest.py"], + visibility = ["//:__subpackages__"], +) + +py_test( + name = "some_file_test", + srcs = ["some_file_test.py"], + deps = [":conftest"], +) +``` + +When `# gazelle:include_pytest_conftest false` is found in +`some_file_test.py` + +```python +# some_file_test.py +# gazelle:include_pytest_conftest false +``` + +Gazelle will generate: + +```starlark +py_library( + name = "conftest", + testonly = True, + srcs = ["conftest.py"], + visibility = ["//:__subpackages__"], +) + +py_test( + name = "some_file_test", + srcs = ["some_file_test.py"], +) +``` + +See {gh-issue}`3076` for more information. diff --git a/gazelle/docs/index.md b/gazelle/docs/index.md index c04efd6a41..e262d7ff46 100644 --- a/gazelle/docs/index.md +++ b/gazelle/docs/index.md @@ -43,4 +43,5 @@ the `update` command (the default) does anything for Python code. ```{toctree} :maxdepth: 1 installation_and_usage +annotations ``` From 768292440b242887f5488d378594838fe633360d Mon Sep 17 00:00:00 2001 From: Douglas Thor Date: Mon, 4 Aug 2025 22:20:55 -0700 Subject: [PATCH 351/922] docs(gazelle): Migrate Gazelle docs to ReadTheDocs, part 4/5: directives (#3139) Part of #3082 4th of probably 5 PRs. + Migrate directive docs from `gazelle/README.md` to `gazelle/docs/directives.md` + Switch from table-based summary to bulleted lists + This will be much easier to maintain going forward. + Add dedicated sections for each directive + Though not filled out yet. I do plan on filling them out later, I just can't say when. + Mechanical updates: + Wrap at ~80 chars + Use MyST directives and roles. --- gazelle/README.md | 424 ------------------------ gazelle/docs/directives.md | 647 +++++++++++++++++++++++++++++++++++++ gazelle/docs/index.md | 1 + 3 files changed, 648 insertions(+), 424 deletions(-) create mode 100644 gazelle/docs/directives.md diff --git a/gazelle/README.md b/gazelle/README.md index efc7004eaf..4de2c3c0cd 100644 --- a/gazelle/README.md +++ b/gazelle/README.md @@ -6,430 +6,6 @@ ReadTheDocs. Please see https://rules-python.readthedocs.io/gazelle/docs/index.h ::: -### Directives - -You can configure the extension using directives, just like for other -languages. These are just comments in the `BUILD.bazel` file which -govern behavior of the extension when processing files under that -folder. - -See https://github.com/bazelbuild/bazel-gazelle#directives -for some general directives that may be useful. -In particular, the `resolve` directive is language-specific -and can be used with Python. -Examples of these directives in use can be found in the -/gazelle/testdata folder in the rules_python repo. - -Python-specific directives are as follows: - -| **Directive** | **Default value** | -|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|-------------------| -| `# gazelle:python_extension` | `enabled` | -| Controls whether the Python extension is enabled or not. Sub-packages inherit this value. Can be either "enabled" or "disabled". | | -| [`# gazelle:python_root`](#directive-python_root) | n/a | -| Sets a Bazel package as a Python root. This is used on monorepos with multiple Python projects that don't share the top-level of the workspace as the root. See [Directive: `python_root`](#directive-python_root) below. | | -| `# gazelle:python_manifest_file_name` | `gazelle_python.yaml` | -| Overrides the default manifest file name. | | -| `# gazelle:python_ignore_files` | n/a | -| Controls the files which are ignored from the generated targets. | | -| `# gazelle:python_ignore_dependencies` | n/a | -| Controls the ignored dependencies from the generated targets. | | -| `# gazelle:python_validate_import_statements` | `true` | -| Controls whether the Python import statements should be validated. Can be "true" or "false" | | -| `# gazelle:python_generation_mode` | `package` | -| Controls the target generation mode. Can be "file", "package", or "project" | | -| `# gazelle:python_generation_mode_per_file_include_init` | `false` | -| Controls whether `__init__.py` files are included as srcs in each generated target when target generation mode is "file". Can be "true", or "false" | | -| [`# gazelle:python_generation_mode_per_package_require_test_entry_point`](#directive-python_generation_mode_per_package_require_test_entry_point) | `true` | -| Controls whether a file called `__test__.py` or a target called `__test__` is required to generate one test target per package in package mode. || -| `# gazelle:python_library_naming_convention` | `$package_name$` | -| Controls the `py_library` naming convention. It interpolates `$package_name$` with the Bazel package name. E.g. if the Bazel package name is `foo`, setting this to `$package_name$_my_lib` would result in a generated target named `foo_my_lib`. | | -| `# gazelle:python_binary_naming_convention` | `$package_name$_bin` | -| Controls the `py_binary` naming convention. Follows the same interpolation rules as `python_library_naming_convention`. | | -| `# gazelle:python_test_naming_convention` | `$package_name$_test` | -| Controls the `py_test` naming convention. Follows the same interpolation rules as `python_library_naming_convention`. | | -| [`# gazelle:python_proto_naming_convention`](#directive-python_proto_naming_convention) | `$proto_name$_py_pb2` | -| Controls the `py_proto_library` naming convention. It interpolates `$proto_name$` with the proto_library rule name, minus any trailing _proto. E.g. if the proto_library name is `foo_proto`, setting this to `$proto_name$_my_lib` would render to `foo_my_lib`. | | -| `# gazelle:resolve py ...` | n/a | -| Instructs the plugin what target to add as a dependency to satisfy a given import statement. The syntax is `# gazelle:resolve py import-string label` where `import-string` is the symbol in the python `import` statement, and `label` is the Bazel label that Gazelle should write in `deps`. | | -| [`# gazelle:python_default_visibility labels`](#directive-python_default_visibility) | | -| Instructs gazelle to use these visibility labels on all python targets. `labels` is a comma-separated list of labels (without spaces). | `//$python_root$:__subpackages__` | -| [`# gazelle:python_visibility label`](#directive-python_visibility) | | -| Appends additional visibility labels to each generated target. This directive can be set multiple times. | | -| [`# gazelle:python_test_file_pattern`](#directive-python_test_file_pattern) | `*_test.py,test_*.py` | -| Filenames matching these comma-separated `glob`s will be mapped to `py_test` targets. | -| `# gazelle:python_label_convention` | `$distribution_name$` | -| Defines the format of the distribution name in labels to third-party deps. Useful for using Gazelle plugin with other rules with different repository conventions (e.g. `rules_pycross`). Full label is always prepended with (pip) repository name, e.g. `@pip//numpy`. | -| `# gazelle:python_label_normalization` | `snake_case` | -| Controls how distribution names in labels to third-party deps are normalized. Useful for using Gazelle plugin with other rules with different label conventions (e.g. `rules_pycross` uses PEP-503). Can be "snake_case", "none", or "pep503". | -| `# gazelle:python_experimental_allow_relative_imports` | `false` | -| Controls whether Gazelle resolves dependencies for import statements that use paths relative to the current package. Can be "true" or "false".| -| `# gazelle:python_generate_pyi_deps` | `false` | -| Controls whether to generate a separate `pyi_deps` attribute for type-checking dependencies or merge them into the regular `deps` attribute. When `false` (default), type-checking dependencies are merged into `deps` for backward compatibility. When `true`, generates separate `pyi_deps`. Imports in blocks with the format `if typing.TYPE_CHECKING:`/`if TYPE_CHECKING:` and type-only stub packages (eg. boto3-stubs) are recognized as type-checking dependencies. | -| [`# gazelle:python_generate_proto`](#directive-python_generate_proto) | `false` | -| Controls whether to generate a `py_proto_library` for each `proto_library` in the package. By default we load this rule from the `@protobuf` repository; use `gazelle:map_kind` if you need to load this from somewhere else. | -| `# gazelle:python_resolve_sibling_imports` | `false` | -| Allows absolute imports to be resolved to sibling modules (Python 2's behavior without `absolute_import`). | - -#### Directive: `python_root`: - -Set this directive within the Bazel package that you want to use as the Python root. -For example, if using a `src` dir (as recommended by the [Python Packaging User -Guide][python-packaging-user-guide]), then set this directive in `src/BUILD.bazel`: - -```starlark -# ./src/BUILD.bazel -# Tell gazelle that are python root is the same dir as this Bazel package. -# gazelle:python_root -``` - -Note that the directive does not have any arguments. - -Gazelle will then add the necessary `imports` attribute to all targets that it -generates: - -```starlark -# in ./src/foo/BUILD.bazel -py_libary( - ... - imports = [".."], # Gazelle adds this - ... -) - -# in ./src/foo/bar/BUILD.bazel -py_libary( - ... - imports = ["../.."], # Gazelle adds this - ... -) -``` - -[python-packaging-user-guide]: https://github.com/pypa/packaging.python.org/blob/4c86169a/source/tutorials/packaging-projects.rst - -#### Directive: `python_proto_naming_convention`: - -Set this directive to a string pattern to control how the generated `py_proto_library` targets are named. When generating new `py_proto_library` rules, Gazelle will replace `$proto_name$` in the pattern with the name of the `proto_library` rule, stripping out a trailing `_proto`. For example: - -```starlark -# gazelle:python_generate_proto true -# gazelle:python_proto_naming_convention my_custom_$proto_name$_pattern - -proto_library( - name = "foo_proto", - srcs = ["foo.proto"], -) -``` - -produces the following `py_proto_library` rule: -```starlark -py_proto_library( - name = "my_custom_foo_pattern", - deps = [":foo_proto"], -) -``` - -The default naming convention is `$proto_name$_pb2_py`, so by default in the above example Gazelle would generate `foo_pb2_py`. Any pre-existing rules are left in place and not renamed. - -Note that the Python library will always be imported as `foo_pb2` in Python code, regardless of the naming convention. Also note that Gazelle is currently not able to map said imports, e.g. `import foo_pb2`, to fill in `py_proto_library` targets as dependencies of other rules. See [this issue](https://github.com/bazel-contrib/rules_python/issues/1703). - -#### Directive: `python_default_visibility`: - -Instructs gazelle to use these visibility labels on all _python_ targets -(typically `py_*`, but can be modified via the `map_kind` directive). The arg -to this directive is a a comma-separated list (without spaces) of labels. - -For example: - -```starlark -# gazelle:python_default_visibility //:__subpackages__,//tests:__subpackages__ -``` - -produces the following visibility attribute: - -```starlark -py_library( - ..., - visibility = [ - "//:__subpackages__", - "//tests:__subpackages__", - ], - ..., -) -``` - -You can also inject the `python_root` value by using the exact string -`$python_root$`. All instances of this string will be replaced by the `python_root` -value. - -```starlark -# gazelle:python_default_visibility //$python_root$:__pkg__,//foo/$python_root$/tests:__subpackages__ - -# Assuming the "# gazelle:python_root" directive is set in ./py/src/BUILD.bazel, -# the results will be: -py_library( - ..., - visibility = [ - "//foo/py/src/tests:__subpackages__", # sorted alphabetically - "//py/src:__pkg__", - ], - ..., -) -``` - -Two special values are also accepted as an argument to the directive: - -+ `NONE`: This removes all default visibility. Labels added by the - `python_visibility` directive are still included. -+ `DEFAULT`: This resets the default visibility. - -For example: - -```starlark -# gazelle:python_default_visibility NONE - -py_library( - name = "...", - srcs = [...], -) -``` - -```starlark -# gazelle:python_default_visibility //foo:bar -# gazelle:python_default_visibility DEFAULT - -py_library( - ..., - visibility = ["//:__subpackages__"], - ..., -) -``` - -These special values can be useful for sub-packages. - - -#### Directive: `python_visibility`: - -Appends additional `visibility` labels to each generated target. - -This directive can be set multiple times. The generated `visibility` attribute -will include the default visibility and all labels defined by this directive. -All labels will be ordered alphabetically. - -```starlark -# ./BUILD.bazel -# gazelle:python_visibility //tests:__pkg__ -# gazelle:python_visibility //bar:baz - -py_library( - ... - visibility = [ - "//:__subpackages__", # default visibility - "//bar:baz", - "//tests:__pkg__", - ], - ... -) -``` - -Child Bazel packages inherit values from parents: - -```starlark -# ./bar/BUILD.bazel -# gazelle:python_visibility //tests:__subpackages__ - -py_library( - ... - visibility = [ - "//:__subpackages__", # default visibility - "//bar:baz", # defined in ../BUILD.bazel - "//tests:__pkg__", # defined in ../BUILD.bazel - "//tests:__subpackages__", # defined in this ./BUILD.bazel - ], - ... -) - -``` - -This directive also supports the `$python_root$` placeholder that -`# gazelle:python_default_visibility` supports. - -```starlark -# gazlle:python_visibility //$python_root$/foo:bar - -py_library( - ... - visibility = ["//this_is_my_python_root/foo:bar"], - ... -) -``` - - -#### Directive: `python_test_file_pattern`: - -This directive adjusts which python files will be mapped to the `py_test` rule. - -+ The default is `*_test.py,test_*.py`: both `test_*.py` and `*_test.py` files - will generate `py_test` targets. -+ This directive must have a value. If no value is given, an error will be raised. -+ It is recommended, though not necessary, to include the `.py` extension in - the `glob`s: `foo*.py,?at.py`. -+ Like most directives, it applies to the current Bazel package and all subpackages - until the directive is set again. -+ This directive accepts multiple `glob` patterns, separated by commas without spaces: - -```starlark -# gazelle:python_test_file_pattern foo*.py,?at - -py_library( - name = "mylib", - srcs = ["mylib.py"], -) - -py_test( - name = "foo_bar", - srcs = ["foo_bar.py"], -) - -py_test( - name = "cat", - srcs = ["cat.py"], -) - -py_test( - name = "hat", - srcs = ["hat.py"], -) -``` - - -##### Notes - -Resetting to the default value (such as in a subpackage) is manual. Set: - -```starlark -# gazelle:python_test_file_pattern *_test.py,test_*.py -``` - -There currently is no way to tell gazelle that _no_ files in a package should -be mapped to `py_test` targets (see [Issue #1826][issue-1826]). The workaround -is to set this directive to a pattern that will never match a `.py` file, such -as `foo.bar`: - -```starlark -# No files in this package should be mapped to py_test targets. -# gazelle:python_test_file_pattern foo.bar - -py_library( - name = "my_test", - srcs = ["my_test.py"], -) -``` - -[issue-1826]: https://github.com/bazel-contrib/rules_python/issues/1826 - -#### Directive: `python_generation_mode_per_package_require_test_entry_point`: -When `# gazelle:python_generation_mode package`, whether a file called `__test__.py` or a target called `__test__`, a.k.a., entry point, is required to generate one test target per package. If this is set to true but no entry point is found, Gazelle will fall back to file mode and generate one test target per file. Setting this directive to false forces Gazelle to generate one test target per package even without entry point. However, this means the `main` attribute of the `py_test` will not be set and the target will not be runnable unless either: -1. there happen to be a file in the `srcs` with the same name as the `py_test` target, or -2. a macro populating the `main` attribute of `py_test` is configured with `gazelle:map_kind` to replace `py_test` when Gazelle is generating Python test targets. For example, user can provide such a macro to Gazelle: - -```starlark -load("@rules_python//python:defs.bzl", _py_test="py_test") -load("@aspect_rules_py//py:defs.bzl", "py_pytest_main") - -def py_test(name, main=None, **kwargs): - deps = kwargs.pop("deps", []) - if not main: - py_pytest_main( - name = "__test__", - deps = ["@pip_pytest//:pkg"], # change this to the pytest target in your repo. - ) - - deps.append(":__test__") - main = ":__test__.py" - - _py_test( - name = name, - main = main, - deps = deps, - **kwargs, -) -``` - -#### Directive: `python_generate_proto`: - -When `# gazelle:python_generate_proto true`, Gazelle will generate one -`py_proto_library` for each `proto_library`, generating Python clients for -protobuf in each package. By default this is turned off. Gazelle will also -generate a load statement for the `py_proto_library` - attempting to detect -the configured name for the `@protobuf` / `@com_google_protobuf` repo in your -`MODULE.bazel`, and otherwise falling back to `@com_google_protobuf` for -compatibility with `WORKSPACE`. - -For example, in a package with `# gazelle:python_generate_proto true` and a -`foo.proto`, if you have both the proto extension and the Python extension -loaded into Gazelle, you'll get something like: - -```starlark -load("@protobuf//bazel:py_proto_library.bzl", "py_proto_library") -load("@rules_proto//proto:defs.bzl", "proto_library") - -# gazelle:python_generate_proto true - -proto_library( - name = "foo_proto", - srcs = ["foo.proto"], - visibility = ["//:__subpackages__"], -) - -py_proto_library( - name = "foo_py_pb2", - visibility = ["//:__subpackages__"], - deps = [":foo_proto"], -) -``` - -When `false`, Gazelle will ignore any `py_proto_library`, including previously-generated or hand-created rules. - - -#### Directive: `python_experimental_allow_relative_imports` -Enables experimental support for resolving relative imports in -`python_generation_mode package`. - -By default, when `# gazelle:python_generation_mode package` is enabled, -relative imports (e.g., from .library import foo) are not added to the -deps field of the generated target. This results in incomplete py_library -rules that lack required dependencies on sibling packages. - -Example: -Given this Python file import: -```python -from .library import add as _add -from .library import subtract as _subtract -``` - -Expected BUILD file output: -```starlark -py_library( - name = "py_default_library", - srcs = ["__init__.py"], - deps = [ - "//example/library:py_default_library", - ], - visibility = ["//visibility:public"], -) -``` - -Actual output without this annotation: -```starlark -py_library( - name = "py_default_library", - srcs = ["__init__.py"], - visibility = ["//visibility:public"], -) -``` -If the directive is set to `true`, gazelle will resolve imports -that are relative to the current package. - ### Libraries Python source files are those ending in `.py` but not ending in `_test.py`. diff --git a/gazelle/docs/directives.md b/gazelle/docs/directives.md new file mode 100644 index 0000000000..9221c60823 --- /dev/null +++ b/gazelle/docs/directives.md @@ -0,0 +1,647 @@ +# Directives + +You can configure the extension using directives, just like for other +languages. These are just comments in the `BUILD.bazel` file which +govern behavior of the extension when processing files under that +folder. + +See the [Gazelle docs on directives][gazelle-directives] for some general +directives that may be useful. In particular, the `resolve` directive +is language-specific and can be used with Python. Examples of these and +the Python-specific directives in use can be found in the +{gh-path}`gazelle/testdata` folder in the `rules_python` repo. + +[gazelle-directives]: https://github.com/bazelbuild/bazel-gazelle#directives + +The Python-specific directives are: + +* [`# gazelle:python_extension`](#python-extension) + * Default: `enabled` + * Allowed Values: `enabled`, `disabled` + * Controls whether the Python extension is enabled or not. Sub-packages + inherit this value. +* [`# gazelle:python_root`](#python-root) + * Default: n/a + * Allowed Values: None. This direcive does not consume values. + * Sets a Bazel package as a Python root. This is used on monorepos with + multiple Python projects that don't share the top-level of the workspace + as the root. +* [`# gazelle:python_manifest_file_name`](#python-manifest-file-name) + * Default: `gazelle_python.yaml` + * Allowed Values: A string + * Overrides the default manifest file name. +* [`# gazelle:python_ignore_files`](#python-ignore-files) + * Default: n/a + * Allowed Values: WIP + * Controls the files which are ignored from the generated targets. +* [`# gazelle:python_ignore_dependencies`](#python-ignore-dependencies) + * Default: n/a + * Allowed Values: WIP + * Controls the ignored dependencies from the generated targets. +* [`# gazelle:python_validate_import_statements`](#python-validate-import-statements) + * Default: `true` + * Allowed Values: `true`, `false` + * Controls whether the Python import statements should be validated. +* [`# gazelle:python_generation_mode`](#python-generation-mode) + * Default: `package` + * Allowed Values: `file`, `package`, `project` + * Controls the target generation mode. +* [`# gazelle:python_generation_mode_per_file_include_init`](#python-generation-mode-per-file-include-init) + * Default: `false` + * Allowed Values: `true`, `false` + * Controls whether `__init__.py` files are included as srcs in each + generated target when target generation mode is "file". +* [`# gazelle:python_generation_mode_per_package_require_test_entry_point`](python-generation-mode-per-package-require-test-entry-point) + * Default: `true` + * Allowed Values: `true`, `false` + * Controls whether a file called `__test__.py` or a target called + `__test__` is required to generate one test target per package in + package mode. +* [`# gazelle:python_library_naming_convention`](#python-library-naming-convention) + * Default: `$package_name$` + * Allowed Values: A string containing `"$package_name$"` + * Controls the {bzl:obj}`py_library` naming convention. It interpolates + `$package_name$` with the Bazel package name. E.g. if the Bazel package + name is `foo`, setting this to `$package_name$_my_lib` would result in a + generated target named `foo_my_lib`. +* [`# gazelle:python_binary_naming_convention`](#python-binary-naming-convention) + * Default: `$package_name$_bin` + * Allowed Values: A string containing `"$package_name$"` + * Controls the {bzl:obj}`py_binary` naming convention. Follows the same interpolation + rules as `python_library_naming_convention`. +* [`# gazelle:python_test_naming_convention`](#python-test-naming-convention) + * Default: `$package_name$_test` + * Allowed Values: A string containing `"$package_name$"` + * Controls the {bzl:obj}`py_test` naming convention. Follows the same interpolation + rules as `python_library_naming_convention`. +* [`# gazelle:python_proto_naming_convention`](#python-proto-naming-convention) + * Default: `$proto_name$_py_pb2` + * Allowed Values: A string containing `"$proto_name$"` + * Controls the {bzl:obj}`py_proto_library` naming convention. It interpolates + `$proto_name$` with the {bzl:obj}`proto_library` rule name, minus any trailing + `_proto`. E.g. if the {bzl:obj}`proto_library` name is `foo_proto`, setting this + to `$proto_name$_my_lib` would render to `foo_my_lib`. +* [`# gazelle:resolve py ...`](#resolve-py) + * Default: n/a + * Allowed Values: See the [bazel-gazelle docs][gazelle-directives] + * Instructs the plugin what target to add as a dependency to satisfy a given + import statement. The syntax is `# gazelle:resolve py import-string label` + where `import-string` is the symbol in the python `import` statement, + and `label` is the Bazel label that Gazelle should write in `deps`. +* [`# gazelle:python_default_visibility labels`](python-default-visibility) + * Default: `//$python_root$:__subpackages__` + * Allowed Values: A string + * Instructs gazelle to use these visibility labels on all python targets. + `labels` is a comma-separated list of labels (without spaces). +* [`# gazelle:python_visibility label`](python-visibility) + * Default: n/a + * Allowed Values: A string + * Appends additional visibility labels to each generated target. This r + directive can be set multiple times. +* [`# gazelle:python_test_file_pattern`](python-test-file-pattern) + * Default: `*_test.py,test_*.py` + * Allowed Values: A glob string + * Filenames matching these comma-separated {command}`glob`s will be mapped to + {bzl:obj}`py_test` targets. +* [`# gazelle:python_label_convention`](#python-label-convention) + * Default: `$distribution_name$` + * Allowed Values: A string + * Defines the format of the distribution name in labels to third-party deps. + Useful for using Gazelle plugin with other rules with different repository + conventions (e.g. `rules_pycross`). Full label is always prepended with + the `pip` repository name, e.g. `@pip//numpy` if your + `MODULE.bazel` has `use_repo(pip, "pip")` or `@pypi//numpy` + if your `MODULE.bazel` has `use_repo(pip, "pypi")`. +* [`# gazelle:python_label_normalization`](#python-label-normalization) + * Default: `snake_case` + * Allowed Values: `snake_case`, `none`, `pep503` + * Controls how distribution names in labels to third-party deps are + normalized. Useful for using Gazelle plugin with other rules with different + label conventions (e.g. `rules_pycross` uses PEP-503). +* [`# gazelle:python_experimental_allow_relative_imports`](#python-experimental-allow-relative-imports) + * Default: `false` + * Allowed Values: `true`, `false` + * Controls whether Gazelle resolves dependencies for import statements that + use paths relative to the current package. +* [`# gazelle:python_generate_pyi_deps`](#python-generate-pyi-deps) + * Default: `false` + * Allowed Values: `true`, `false` + * Controls whether to generate a separate `pyi_deps` attribute for + type-checking dependencies or merge them into the regular `deps` + attribute. When `false` (default), type-checking dependencies are + merged into `deps` for backward compatibility. When `true`, generates + separate `pyi_deps`. Imports in blocks with the format + `if typing.TYPE_CHECKING:` or `if TYPE_CHECKING:` and type-only stub + packages (eg. boto3-stubs) are recognized as type-checking dependencies. +* [`# gazelle:python_generate_proto`](#python-generate-proto) + * Default: `false` + * Allowed Values: `true`, `false` + * Controls whether to generate a {bzl:obj}`py_proto_library` for each + {bzl:obj}`proto_library` in the package. By default we load this rule from the + `@protobuf` repository; use `gazelle:map_kind` if you need to load this + from somewhere else. +* [`# gazelle:python_resolve_sibling_imports`](#python-resolve-sibling-imports) + * Default: `false` + * Allowed Values: `true`, `false` + * Allows absolute imports to be resolved to sibling modules (Python 2's + behavior without `absolute_import`). + + +## `python_extension` + +:::{error} +Detailed docs are not yet written. +::: + + +## `python_root` + +Set this directive within the Bazel package that you want to use as the Python root. +For example, if using a `src` dir (as recommended by the [Python Packaging User +Guide][python-packaging-user-guide]), then set this directive in `src/BUILD.bazel`: + +```starlark +# ./src/BUILD.bazel +# Tell gazelle that are python root is the same dir as this Bazel package. +# gazelle:python_root +``` + +Note that the directive does not have any arguments. + +Gazelle will then add the necessary `imports` attribute to all targets that it +generates: + +```starlark +# in ./src/foo/BUILD.bazel +py_libary( + ... + imports = [".."], # Gazelle adds this + ... +) + +# in ./src/foo/bar/BUILD.bazel +py_libary( + ... + imports = ["../.."], # Gazelle adds this + ... +) +``` + +[python-packaging-user-guide]: https://github.com/pypa/packaging.python.org/blob/4c86169a/source/tutorials/packaging-projects.rst + + +## `python_manifest_file_name` + +:::{error} +Detailed docs are not yet written. +::: + + +## `python_ignore_files` + +:::{error} +Detailed docs are not yet written. +::: + + +## `python_ignore_dependencies` + +:::{error} +Detailed docs are not yet written. +::: + + +## `python_validate_import_statements` + +:::{error} +Detailed docs are not yet written. +::: + + +## `python_generation_mode` + +:::{error} +Detailed docs are not yet written. +::: + + +## `python_generation_mode_per_file_include_init` + +:::{error} +Detailed docs are not yet written. +::: + + +## `python_generation_mode_per_package_require_test_entry_point` + +When `# gazelle:python_generation_mode package`, whether a file called +`__test__.py` or a target called `__test__`, a.k.a., entry point, is required +to generate one test target per package. If this is set to true but no entry +point is found, Gazelle will fall back to file mode and generate one test target +per file. Setting this directive to false forces Gazelle to generate one test +target per package even without entry point. However, this means the `main` +attribute of the {bzl:obj}`py_test` will not be set and the target will not be runnable +unless either: + +1. there happen to be a file in the `srcs` with the same name as the {bzl:obj}`py_test` + target, or +2. a macro populating the `main` attribute of {bzl:obj}`py_test` is configured with + `gazelle:map_kind` to replace {bzl:obj}`py_test` when Gazelle is generating Python + test targets. For example, user can provide such a macro to Gazelle: + +```starlark +load("@rules_python//python:defs.bzl", _py_test="py_test") +load("@aspect_rules_py//py:defs.bzl", "py_pytest_main") + +def py_test(name, main=None, **kwargs): + deps = kwargs.pop("deps", []) + if not main: + py_pytest_main( + name = "__test__", + deps = ["@pip_pytest//:pkg"], # change this to the pytest target in your repo. + ) + + deps.append(":__test__") + main = ":__test__.py" + + _py_test( + name = name, + main = main, + deps = deps, + **kwargs, +) +``` + + +## `python_library_naming_convention` + +:::{error} +Detailed docs are not yet written. +::: + + +## `python_binary_naming_convention` + +:::{error} +Detailed docs are not yet written. +::: + + +## `python_test_naming_convention` + +:::{error} +Detailed docs are not yet written. +::: + + +## `python_proto_naming_convention` + +Set this directive to a string pattern to control how the generated +{bzl:obj}`py_proto_library` targets are named. When generating new +{bzl:obj}`py_proto_library` rules, Gazelle will replace `$proto_name$` in the +pattern with the name of the {bzl:obj}`proto_library` rule, stripping out a +trailing `_proto`. For example: + +```starlark +# gazelle:python_generate_proto true +# gazelle:python_proto_naming_convention my_custom_$proto_name$_pattern + +proto_library( + name = "foo_proto", + srcs = ["foo.proto"], +) +``` + +produces the following {bzl:obj}`py_proto_library` rule: + +```starlark +py_proto_library( + name = "my_custom_foo_pattern", + deps = [":foo_proto"], +) +``` + +The default naming convention is `$proto_name$_pb2_py` in accordance with +the [Bazel `py_proto_library` convention][bazel-py-proto-library], so by default +in the above example Gazelle would generate `foo_pb2_py`. Any pre-existing +rules are left in place and not renamed. + +[bazel-py-proto-library]: https://bazel.build/reference/be/protocol-buffer#py_proto_library + +Note that the Python library will always be imported as `foo_pb2` in Python +code, regardless of the naming convention. Also note that Gazelle is currently +not able to map said imports, e.g. `import foo_pb2`, to fill in +{bzl:obj}`py_proto_library` targets as dependencies of other rules. See +{gh-issue}`1703`. + + +## `resolve py` + +:::{error} +Detailed docs are not yet written. +::: + + +## `python_default_visibility` + +Instructs gazelle to use these visibility labels on all _python_ targets +(typically `py_*`, but can be modified via the `map_kind` directive). The arg +to this directive is a comma-separated list (without spaces) of labels. + +For example: + +```starlark +# gazelle:python_default_visibility //:__subpackages__,//tests:__subpackages__ +``` + +produces the following visibility attribute: + +```starlark +py_library( + ..., + visibility = [ + "//:__subpackages__", + "//tests:__subpackages__", + ], + ..., +) +``` + +You can also inject the `python_root` value by using the exact string +`$python_root$`. All instances of this string will be replaced by the `python_root` +value. + +```starlark +# gazelle:python_default_visibility //$python_root$:__pkg__,//foo/$python_root$/tests:__subpackages__ + +# Assuming the "# gazelle:python_root" directive is set in ./py/src/BUILD.bazel, +# the results will be: +py_library( + ..., + visibility = [ + "//foo/py/src/tests:__subpackages__", # sorted alphabetically + "//py/src:__pkg__", + ], + ..., +) +``` + +Two special values are also accepted as an argument to the directive: + +* `NONE`: This removes all default visibility. Labels added by the + `python_visibility` directive are still included. +* `DEFAULT`: This resets the default visibility. + +For example: + +```starlark +# gazelle:python_default_visibility NONE + +py_library( + name = "...", + srcs = [...], +) +``` + +```starlark +# gazelle:python_default_visibility //foo:bar +# gazelle:python_default_visibility DEFAULT + +py_library( + ..., + visibility = ["//:__subpackages__"], + ..., +) +``` + +These special values can be useful for sub-packages. + + +## `python_visibility` + +Appends additional `visibility` labels to each generated target. + +This directive can be set multiple times. The generated `visibility` attribute +will include the default visibility and all labels defined by this directive. +All labels will be ordered alphabetically. + +```starlark +# ./BUILD.bazel +# gazelle:python_visibility //tests:__pkg__ +# gazelle:python_visibility //bar:baz + +py_library( + ... + visibility = [ + "//:__subpackages__", # default visibility + "//bar:baz", + "//tests:__pkg__", + ], + ... +) +``` + +Child Bazel packages inherit values from parents: + +```starlark +# ./bar/BUILD.bazel +# gazelle:python_visibility //tests:__subpackages__ + +py_library( + ... + visibility = [ + "//:__subpackages__", # default visibility + "//bar:baz", # defined in ../BUILD.bazel + "//tests:__pkg__", # defined in ../BUILD.bazel + "//tests:__subpackages__", # defined in this ./BUILD.bazel + ], + ... +) + +``` + +This directive also supports the `$python_root$` placeholder that +`# gazelle:python_default_visibility` supports. + +```starlark +# gazlle:python_visibility //$python_root$/foo:bar + +py_library( + ... + visibility = ["//this_is_my_python_root/foo:bar"], + ... +) +``` + + +## `python_test_file_pattern` + +This directive adjusts which python files will be mapped to the {bzl:obj}`py_test` rule. + ++ The default is `*_test.py,test_*.py`: both `test_*.py` and `*_test.py` files + will generate {bzl:obj}`py_test` targets. ++ This directive must have a value. If no value is given, an error will be raised. ++ It is recommended, though not necessary, to include the `.py` extension in + the {command}`glob`: `foo*.py,?at.py`. ++ Like most directives, it applies to the current Bazel package and all subpackages + until the directive is set again. ++ This directive accepts multiple {command}`glob` patterns, separated by commas without spaces: + +```starlark +# gazelle:python_test_file_pattern foo*.py,?at + +py_library( + name = "mylib", + srcs = ["mylib.py"], +) + +py_test( + name = "foo_bar", + srcs = ["foo_bar.py"], +) + +py_test( + name = "cat", + srcs = ["cat.py"], +) + +py_test( + name = "hat", + srcs = ["hat.py"], +) +``` + + +### Notes + +Resetting to the default value (such as in a subpackage) is manual. Set: + +```starlark +# gazelle:python_test_file_pattern *_test.py,test_*.py +``` + +There currently is no way to tell gazelle that _no_ files in a package should +be mapped to {bzl:obj}`py_test` targets (see {gh-issue}`1826`). The workaround +is to set this directive to a pattern that will never match a `.py` file, such +as `foo.bar`: + +```starlark +# No files in this package should be mapped to py_test targets. +# gazelle:python_test_file_pattern foo.bar + +py_library( + name = "my_test", + srcs = ["my_test.py"], +) +``` + + +## `python_label_convention` + +:::{error} +Detailed docs are not yet written. +::: + + +## `python_label_normalization` + +:::{error} +Detailed docs are not yet written. +::: + + +## `python_experimental_allow_relative_imports` + +Enables experimental support for resolving relative imports in +`python_generation_mode package`. + +By default, when `# gazelle:python_generation_mode package` is enabled, +relative imports (e.g., `from .library import foo`) are not added to the +deps field of the generated target. This results in incomplete {bzl:obj}`py_library` +rules that lack required dependencies on sibling packages. + +Example: + +Given this Python file import: + +```python +from .library import add as _add +from .library import subtract as _subtract +``` + +Expected BUILD file output: + +```starlark +py_library( + name = "py_default_library", + srcs = ["__init__.py"], + deps = [ + "//example/library:py_default_library", + ], + visibility = ["//visibility:public"], +) +``` + +Actual output without this annotation: + +```starlark +py_library( + name = "py_default_library", + srcs = ["__init__.py"], + visibility = ["//visibility:public"], +) +``` + +If the directive is set to `true`, gazelle will resolve imports +that are relative to the current package. + + +## `python_generate_pyi_deps` + +:::{error} +Detailed docs are not yet written. +::: + + +## `python_generate_proto` + +When `# gazelle:python_generate_proto true`, Gazelle will generate one +{bzl:obj}`py_proto_library` for each {bzl:obj}`proto_library`, generating Python clients for +protobuf in each package. By default this is turned off. Gazelle will also +generate a load statement for the {bzl:obj}`py_proto_library` - attempting to detect +the configured name for the `@protobuf` / `@com_google_protobuf` repo in your +`MODULE.bazel`, and otherwise falling back to `@com_google_protobuf` for +compatibility with `WORKSPACE`. + +For example, in a package with `# gazelle:python_generate_proto true` and a +`foo.proto`, if you have both the proto extension and the Python extension +loaded into Gazelle, you'll get something like: + +```starlark +load("@protobuf//bazel:py_proto_library.bzl", "py_proto_library") +load("@rules_proto//proto:defs.bzl", "proto_library") + +# gazelle:python_generate_proto true + +proto_library( + name = "foo_proto", + srcs = ["foo.proto"], + visibility = ["//:__subpackages__"], +) + +py_proto_library( + name = "foo_py_pb2", + visibility = ["//:__subpackages__"], + deps = [":foo_proto"], +) +``` + +When `false`, Gazelle will ignore any {bzl:obj}`py_proto_library`, including +previously-generated or hand-created rules. + + +## `python_resolve_sibling_imports` + +:::{error} +Detailed docs are not yet written. +::: diff --git a/gazelle/docs/index.md b/gazelle/docs/index.md index e262d7ff46..6758e11d81 100644 --- a/gazelle/docs/index.md +++ b/gazelle/docs/index.md @@ -43,5 +43,6 @@ the `update` command (the default) does anything for Python code. ```{toctree} :maxdepth: 1 installation_and_usage +directives annotations ``` From ae2fee13c192440be166fa2107e44b3555510100 Mon Sep 17 00:00:00 2001 From: Douglas Thor Date: Wed, 6 Aug 2025 09:12:42 -0700 Subject: [PATCH 352/922] docs(gazelle): Migrate Gazelle docs to ReadTheDocs, part 5/5: target types (#3147) Part of #3082 5th of ~~5~~ 6 PRs. + Migrate target types docs from `gazelle/README.md` to `gazelle/docs/installation_and_usage.md` + `Libraries` section: + Update wording + list for the generation mode types + `Binaries` section: + Slight rewording + `note` block instead of simple "Note that ..." + Mechanical updates: + Wrap at ~80 chars + Use MyST directives and roles. --- gazelle/README.md | 70 ---------------------- gazelle/docs/installation_and_usage.md | 83 ++++++++++++++++++++++++++ 2 files changed, 83 insertions(+), 70 deletions(-) diff --git a/gazelle/README.md b/gazelle/README.md index 4de2c3c0cd..30067b4c23 100644 --- a/gazelle/README.md +++ b/gazelle/README.md @@ -6,76 +6,6 @@ ReadTheDocs. Please see https://rules-python.readthedocs.io/gazelle/docs/index.h ::: -### Libraries - -Python source files are those ending in `.py` but not ending in `_test.py`. - -First, we look for the nearest ancestor BUILD file starting from the folder -containing the Python source file. - -In package generation mode, if there is no `py_library` in this BUILD file, one -is created using the package name as the target's name. This makes it the -default target in the package. Next, all source files are collected into the -`srcs` of the `py_library`. - -In project generation mode, all source files in subdirectories (that don't have -BUILD files) are also collected. - -In file generation mode, each file is given its own target. - -Finally, the `import` statements in the source files are parsed, and -dependencies are added to the `deps` attribute. - -### Unit Tests - -A `py_test` target is added to the BUILD file when gazelle encounters -a file named `__test__.py`. -Often, Python unit test files are named with the suffix `_test`. -For example, if we had a folder that is a package named "foo" we could have a Python file named `foo_test.py` -and gazelle would create a `py_test` block for the file. - -The following is an example of a `py_test` target that gazelle would add when -it encounters a file named `__test__.py`. - -```starlark -py_test( - name = "build_file_generation_test", - srcs = ["__test__.py"], - main = "__test__.py", - deps = [":build_file_generation"], -) -``` - -You can control the naming convention for test targets by adding a gazelle directive named -`# gazelle:python_test_naming_convention`. See the instructions in the section above that -covers directives. - -### Binaries - -When a `__main__.py` file is encountered, this indicates the entry point -of a Python program. A `py_binary` target will be created, named `[package]_bin`. - -When no such entry point exists, Gazelle will look for a line like this in the top level in every module: - -```python -if __name == "__main__": -``` - -Gazelle will create a `py_binary` target for every module with such a line, with -the target name the same as the module name. - -If `python_generation_mode` is set to `file`, then instead of one `py_binary` -target per module, Gazelle will create one `py_binary` target for each file with -such a line, and the name of the target will match the name of the script. - -Note that it's possible for another script to depend on a `py_binary` target and -import from the `py_binary`'s scripts. This can have possible negative effects on -Bazel analysis time and runfiles size compared to depending on a `py_library` -target. The simplest way to avoid these negative effects is to extract library -code into a separate script without a `main` line. Gazelle will then create a -`py_library` target for that library code, and other scripts can depend on that -`py_library` target. - ## Developer Notes Gazelle extensions are written in Go. diff --git a/gazelle/docs/installation_and_usage.md b/gazelle/docs/installation_and_usage.md index e764957581..123f30a068 100644 --- a/gazelle/docs/installation_and_usage.md +++ b/gazelle/docs/installation_and_usage.md @@ -149,3 +149,86 @@ gazelle( That's it, now you can finally run `bazel run //:gazelle` anytime you edit Python code, and it should update your `BUILD` files correctly. + + +## Target Types and How They're Generated + +### Libraries + +Python source files are those ending in `.py` that are not matched as a test +file via the `# gazelle:python_test_file_pattern` directive. By default, +python source files are all `*.py` files except for `*_test.py` and +`test_*.py`. + +First, we look for the nearest ancestor `BUILD(.bazel)` file starting from +the folder containing the Python source file. + ++ In `package` generation mode, if there is no {bzl:obj}`py_library` in this + `BUILD(.bazel)` file, one is created using the package name as the target's + name. This makes it the default target in the package. Next, all source + files are collected into the `srcs` of the {bzl:obj}`py_library`. ++ In `project` generation mode, all source files in subdirectories (that don't + have `BUILD(.bazel)` files) are also collected. ++ In `file` generation mode, each python source file is given its own target. + +Finally, the `import` statements in the source files are parsed and +dependencies are added to the `deps` attribute of the target. + + +### Tests + +A {bzl:obj}`py_test` target is added to the `BUILD(.bazel)` file when gazelle +encounters a file named `__test__.py` or when files matching the +`# gazelle:python_test_file_pattern` directive are found. + +For example, if we had a folder that is a package named "foo" we could have a +Python file named `foo_test.py` and gazelle would create a {bzl:obj}`py_test` +block for the file. + +The following is an example of a {bzl:obj}`py_test` target that gazelle would +add when it encounters a file named `__test__.py`. + +```starlark +py_test( + name = "build_file_generation_test", + srcs = ["__test__.py"], + main = "__test__.py", + deps = [":build_file_generation"], +) +``` + +You can control the naming convention for test targets using the +`# gazelle:python_test_naming_convention` directive. + + +### Binaries + +When a `__main__.py` file is encountered, this indicates the entry point +of a Python program. A {bzl:obj}`py_binary` target will be created, named +`[package]_bin`. + +When no such entry point exists, Gazelle will look for a line like this in +the top level in every module: + +```python +if __name == "__main__": +``` + +Gazelle will create a {bzl:obj}`py_binary` target for every module with such +a line, with the target name the same as the module name. + +If the `# gazelle:python_generation_mode` directive is set to `file`, then +instead of one {bzl:obj}`py_binary` target per module, Gazelle will create +one {bzl:obj}`py_binary` target for each file with such a line, and the name +of the target will match the name of the script. + +:::{note} +It's possible for another script to depend on a {bzl:obj}`py_binary` target +and import from the {bzl:obj}`py_binary`'s scripts. This can have possible +negative effects on Bazel analysis time and runfiles size compared to +depending on a {bzl:obj}`py_library` target. The simplest way to avoid these +negative effects is to extract library code into a separate script without a +`main` line. Gazelle will then create a {bzl:obj}`py_library` target for +that library code, and other scripts can depend on that {bzl:obj}`py_library` +target. +::: From e3655c2bee36225366a89554a5a2628237c84f00 Mon Sep 17 00:00:00 2001 From: Douglas Thor Date: Wed, 6 Aug 2025 09:31:06 -0700 Subject: [PATCH 353/922] docs(gazelle): Migrate Gazelle docs to ReadTheDocs, part 6/5: development (#3149) Fixes #3082 6th of 5 PRs. + Migrate gazelle development docs from `gazelle/README.md` to `gazelle/docs/development.md` + Add information on writing tests + Mechanical updates: + Wrap at ~80 chars + Use MyST directives and roles. --- gazelle/README.md | 18 +----------- gazelle/docs/development.md | 57 +++++++++++++++++++++++++++++++++++++ gazelle/docs/index.md | 1 + 3 files changed, 59 insertions(+), 17 deletions(-) create mode 100644 gazelle/docs/development.md diff --git a/gazelle/README.md b/gazelle/README.md index 30067b4c23..128fb1f583 100644 --- a/gazelle/README.md +++ b/gazelle/README.md @@ -1,22 +1,6 @@ # Python Gazelle plugin :::{note} -The gazelle plugin docs are being migrated to our primary documentation on +The gazelle plugin docs have been migrated to our primary documentation on ReadTheDocs. Please see https://rules-python.readthedocs.io/gazelle/docs/index.html. ::: - - -## Developer Notes - -Gazelle extensions are written in Go. -See the gazelle documentation https://github.com/bazelbuild/bazel-gazelle/blob/master/extend.md -for more information on extending Gazelle. - -If you add new Go dependencies to the plugin source code, you need to "tidy" the go.mod file. -After changing that file, run `go mod tidy` or `bazel run @go_sdk//:bin/go -- mod tidy` -to update the go.mod and go.sum files. Then run `bazel run //:gazelle_update_repos` to have gazelle -add the new dependenies to the deps.bzl file. The deps.bzl file is used as defined in our /WORKSPACE -to include the external repos Bazel loads Go dependencies from. - -Then after editing Go code, run `bazel run //:gazelle` to generate/update the rules in the -BUILD.bazel files in our repo. diff --git a/gazelle/docs/development.md b/gazelle/docs/development.md new file mode 100644 index 0000000000..29ac7a0605 --- /dev/null +++ b/gazelle/docs/development.md @@ -0,0 +1,57 @@ +# Development + +Gazelle extensions are written in Go. + +See the [Gazelle documentation][gazelle-extend] for more information on +extending Gazelle. + +[gazelle-extend]: https://github.com/bazel-contrib/bazel-gazelle/blob/master/extend.md + + +## Dependencies + +If you add new Go dependencies to the plugin source code, you need to "tidy" +the go.mod file. After changing that file, run `go mod tidy` or +`bazel run @go_sdk//:bin/go -- mod tidy` to update the `go.mod` and `go.sum` +files. Then run `bazel run //:gazelle_update_repos` to have gazelle add the +new dependencies to the `deps.bzl` file. The `deps.bzl` file is used as +defined in our `/WORKSPACE` to include the external repos Bazel loads Go +dependencies from. + +Then after editing Go code, run `bazel run //:gazelle` to generate/update +the rules in the `BUILD.bazel` files in our repo. + + +## Tests + +:::{seealso} +{gh-path}`gazelle/python/testdata/README.md` +::: + +To run tests, {command}`cd` into the {gh-path}`gazelle` directory and run +`bazel test //...`. + +Test cases are found at {gh-path}`gazelle/python/testdata`. To make a new +test case, create a directory in that folder with the following files: + ++ `README.md` with a short blurb describing the test case(s). ++ `test.yaml`, either empty (with just the docstart `---` line) or with + the expected `stderr` and exit codes of the test case. ++ and empty `WORKSPACE` file + +You will also need `BUILD.in` and `BUILD.out` files somewhere within the test +case directory. These can be in the test case root, in subdirectories, or +both. + ++ `BUILD.in` files are populated with the "before" information - typically + things like Gazelle directives or pre-existing targets. This is how the + `BUILD.bazel` file looks before running Gazelle. ++ `BUILD.out` files are the expected result after running Gazelle within + the test case. + +:::{tip} +The easiest way to create a new test is to look at one of the existing test +cases. +::: + +The source code for running tests is {gh-path}`gazelle/python/python_test.go`. diff --git a/gazelle/docs/index.md b/gazelle/docs/index.md index 6758e11d81..f276b0ca16 100644 --- a/gazelle/docs/index.md +++ b/gazelle/docs/index.md @@ -45,4 +45,5 @@ the `update` command (the default) does anything for Python code. installation_and_usage directives annotations +development ``` From 32d7a24d45eae7430b38733353e3ee77583d2da8 Mon Sep 17 00:00:00 2001 From: Malte Poll <1780588+malt3@users.noreply.github.com> Date: Thu, 7 Aug 2025 03:39:18 +0200 Subject: [PATCH 354/922] fix: use "command -v" to find interpreter in $PATH (#3150) In some environments, `which` doesn't work correctly under Bazel, while `command -v` does. I think the difference is that `command` is a shell builtin (and POSIX compliant), whereas `which` is not: ``` $ sh -c 'builtin command -v python3' /usr/bin/python3 $ sh -c 'builtin which python3' sh: line 1: builtin: which: not a shell builtin ``` While `command -v` performs fewer checks under the hood, it is more portable. --- CHANGELOG.md | 2 ++ python/private/runtime_env_toolchain_interpreter.sh | 12 +++++------- 2 files changed, 7 insertions(+), 7 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 422e399026..08991c6107 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -101,6 +101,8 @@ END_UNRELEASED_TEMPLATE `# gazelle:python_resolve_sibling_imports true` * (pypi) Show overridden index URL of packages when downloading metadata have failed. ([#2985](https://github.com/bazel-contrib/rules_python/issues/2985)). +* (toolchains) use "command -v" to find interpreter in `$PATH` + ([#3150](https://github.com/bazel-contrib/rules_python/pull/3150)). {#v0-0-0-added} ### Added diff --git a/python/private/runtime_env_toolchain_interpreter.sh b/python/private/runtime_env_toolchain_interpreter.sh index dd4d648d12..c78cfe1a9b 100755 --- a/python/private/runtime_env_toolchain_interpreter.sh +++ b/python/private/runtime_env_toolchain_interpreter.sh @@ -17,16 +17,14 @@ die() { exit 1 } -# We use `which` to locate the Python interpreter command on PATH. `command -v` -# is another option, but it doesn't check whether the file it finds has the -# executable bit. +# We use `command -v` to locate the Python interpreter command on PATH. # # A tricky situation happens when this wrapper is invoked as part of running a # tool, e.g. passing a py_binary target to `ctx.actions.run()`. Bazel will unset # the PATH variable. Then the shell will see there's no PATH and initialize its -# own, sometimes without exporting it. This causes `which` to fail and this +# own, sometimes without exporting it. This causes `command -v` to fail and this # script to think there's no Python interpreter installed. To avoid this we -# explicitly pass PATH to each `which` invocation. We can't just export PATH +# explicitly pass PATH to each `command -v` invocation. We can't just export PATH # because that would modify the environment seen by the final user Python # program. # @@ -37,9 +35,9 @@ die() { # https://github.com/bazelbuild/bazel/issues/8415 # Try the "python3" command name first, then fall back on "python". -PYTHON_BIN="$(PATH="$PATH" which python3 2> /dev/null)" +PYTHON_BIN="$(PATH="$PATH" command -v python3 2> /dev/null)" if [ -z "${PYTHON_BIN:-}" ]; then - PYTHON_BIN="$(PATH="$PATH" which python 2>/dev/null)" + PYTHON_BIN="$(PATH="$PATH" command -v python 2>/dev/null)" fi if [ -z "${PYTHON_BIN:-}" ]; then die "Neither 'python3' nor 'python' were found on the target \ From 4068ba197dc167730a2766c3621ff9b834db4782 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Thu, 7 Aug 2025 02:35:28 -0700 Subject: [PATCH 355/922] docs(pypi): clarify when extra_hub_aliases was added to workspace (#3152) The `extra_hub_aliases` feature was added for bzlmod and workspace in different versions. Bzlmod added it in 0.38, while workspace added it in 1.0 Co-authored-by: Ignas Anikevicius <240938+aignas@users.noreply.github.com> --- python/private/pypi/attrs.bzl | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/python/private/pypi/attrs.bzl b/python/private/pypi/attrs.bzl index 7ea19d106a..a122fc8479 100644 --- a/python/private/pypi/attrs.bzl +++ b/python/private/pypi/attrs.bzl @@ -159,6 +159,13 @@ Extra aliases to make for specific wheels in the hub repo. This is useful when paired with the {attr}`whl_modifications`. :::{versionadded} 0.38.0 + +For `pip.parse` with bzlmod +::: + +:::{versionadded} 1.0.0 + +For `pip_parse` with workspace. ::: """, mandatory = False, From 4a8cca8c74db1bf35d1ecad71d47395c85e217f6 Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Thu, 7 Aug 2025 22:00:52 +0900 Subject: [PATCH 356/922] build(deps): bump certifi from 2025.7.14 to 2025.8.3 in /docs (#3143) Bumps [certifi](https://github.com/certifi/python-certifi) from 2025.7.14 to 2025.8.3.
Commits

[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=certifi&package-manager=pip&previous-version=2025.7.14&new-version=2025.8.3)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) ---
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot show ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> --- docs/requirements.txt | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/requirements.txt b/docs/requirements.txt index cb8900bf95..f0abac5c30 100644 --- a/docs/requirements.txt +++ b/docs/requirements.txt @@ -17,9 +17,9 @@ babel==2.17.0 \ --hash=sha256:0c54cffb19f690cdcc52a3b50bcbf71e07a808d1c80d549f2459b9d2cf0afb9d \ --hash=sha256:4d0b53093fdfb4b21c92b5213dba5a1b23885afa8383709427046b21c366e5f2 # via sphinx -certifi==2025.7.14 \ - --hash=sha256:6b31f564a415d79ee77df69d757bb49a5bb53bd9f756cbbe24394ffd6fc1f4b2 \ - --hash=sha256:8ea99dbdfaaf2ba2f9bac77b9249ef62ec5218e7c2b2e903378ed5fccf765995 +certifi==2025.8.3 \ + --hash=sha256:e564105f78ded564e3ae7c923924435e1daa7463faeab5bb932bc53ffae63407 \ + --hash=sha256:f6c12493cfb1b06ba2ff328595af9350c65d6644968e5d3a2ffd78699af217a5 # via requests charset-normalizer==3.4.2 \ --hash=sha256:005fa3432484527f9732ebd315da8da8001593e2cf46a3d817669f062c3d9ed4 \ From 3a927ee6984d114dc5bebe0b36328ef13b3b5bed Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Thu, 7 Aug 2025 22:01:11 +0900 Subject: [PATCH 357/922] build(deps): bump nh3 from 0.2.18 to 0.3.0 in /tools/publish (#3141) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Bumps [nh3](https://github.com/messense/nh3) from 0.2.18 to 0.3.0.
Release notes

Sourced from nh3's releases.

v0.3.0

What's Changed

Full Changelog: https://github.com/messense/nh3/compare/v0.2.22...v0.3.0

v0.2.22

What's Changed

New Contributors

Full Changelog: https://github.com/messense/nh3/compare/v0.2.21...v0.2.22

v0.2.21

What's Changed

New Contributors

Full Changelog: https://github.com/messense/nh3/compare/v0.2.20...v0.2.21

v0.2.20

What's Changed

Full Changelog: https://github.com/messense/nh3/compare/v0.2.19...v0.2.20

v0.2.19

What's Changed

New Contributors

... (truncated)

Commits

[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=nh3&package-manager=pip&previous-version=0.2.18&new-version=0.3.0)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) ---
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot show ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> --- tools/publish/requirements_darwin.txt | 44 +++++++++++++++--------- tools/publish/requirements_linux.txt | 44 +++++++++++++++--------- tools/publish/requirements_universal.txt | 44 +++++++++++++++--------- tools/publish/requirements_windows.txt | 44 +++++++++++++++--------- 4 files changed, 108 insertions(+), 68 deletions(-) diff --git a/tools/publish/requirements_darwin.txt b/tools/publish/requirements_darwin.txt index 677cc6f7eb..ee6837a198 100644 --- a/tools/publish/requirements_darwin.txt +++ b/tools/publish/requirements_darwin.txt @@ -148,23 +148,33 @@ more-itertools==10.7.0 \ # via # jaraco-classes # jaraco-functools -nh3==0.2.18 \ - --hash=sha256:0411beb0589eacb6734f28d5497ca2ed379eafab8ad8c84b31bb5c34072b7164 \ - --hash=sha256:14c5a72e9fe82aea5fe3072116ad4661af5cf8e8ff8fc5ad3450f123e4925e86 \ - --hash=sha256:19aaba96e0f795bd0a6c56291495ff59364f4300d4a39b29a0abc9cb3774a84b \ - --hash=sha256:34c03fa78e328c691f982b7c03d4423bdfd7da69cd707fe572f544cf74ac23ad \ - --hash=sha256:36c95d4b70530b320b365659bb5034341316e6a9b30f0b25fa9c9eff4c27a204 \ - --hash=sha256:3a157ab149e591bb638a55c8c6bcb8cdb559c8b12c13a8affaba6cedfe51713a \ - --hash=sha256:42c64511469005058cd17cc1537578eac40ae9f7200bedcfd1fc1a05f4f8c200 \ - --hash=sha256:5f36b271dae35c465ef5e9090e1fdaba4a60a56f0bb0ba03e0932a66f28b9189 \ - --hash=sha256:6955369e4d9f48f41e3f238a9e60f9410645db7e07435e62c6a9ea6135a4907f \ - --hash=sha256:7b7c2a3c9eb1a827d42539aa64091640bd275b81e097cd1d8d82ef91ffa2e811 \ - --hash=sha256:8ce0f819d2f1933953fca255db2471ad58184a60508f03e6285e5114b6254844 \ - --hash=sha256:94a166927e53972a9698af9542ace4e38b9de50c34352b962f4d9a7d4c927af4 \ - --hash=sha256:a7f1b5b2c15866f2db413a3649a8fe4fd7b428ae58be2c0f6bca5eefd53ca2be \ - --hash=sha256:c8b3a1cebcba9b3669ed1a84cc65bf005728d2f0bc1ed2a6594a992e817f3a50 \ - --hash=sha256:de3ceed6e661954871d6cd78b410213bdcb136f79aafe22aa7182e028b8c7307 \ - --hash=sha256:f0eca9ca8628dbb4e916ae2491d72957fdd35f7a5d326b7032a345f111ac07fe +nh3==0.3.0 \ + --hash=sha256:0649464ac8eee018644aacbc103874ccbfac80e3035643c3acaab4287e36e7f5 \ + --hash=sha256:16f8670201f7e8e0e05ed1a590eb84bfa51b01a69dd5caf1d3ea57733de6a52f \ + --hash=sha256:1adeb1062a1c2974bc75b8d1ecb014c5fd4daf2df646bbe2831f7c23659793f9 \ + --hash=sha256:37d3003d98dedca6cd762bf88f2e70b67f05100f6b949ffe540e189cc06887f9 \ + --hash=sha256:389d93d59b8214d51c400fb5b07866c2a4f79e4e14b071ad66c92184fec3a392 \ + --hash=sha256:3f1b4f8a264a0c86ea01da0d0c390fe295ea0bcacc52c2103aca286f6884f518 \ + --hash=sha256:423201bbdf3164a9e09aa01e540adbb94c9962cc177d5b1cbb385f5e1e79216e \ + --hash=sha256:634e34e6162e0408e14fb61d5e69dbaea32f59e847cfcfa41b66100a6b796f62 \ + --hash=sha256:6d68fa277b4a3cf04e5c4b84dd0c6149ff7d56c12b3e3fab304c525b850f613d \ + --hash=sha256:7275fdffaab10cc5801bf026e3c089d8de40a997afc9e41b981f7ac48c5aa7d5 \ + --hash=sha256:7852f038a054e0096dac12b8141191e02e93e0b4608c4b993ec7d4ffafea4e49 \ + --hash=sha256:7c915060a2c8131bef6a29f78debc29ba40859b6dbe2362ef9e5fd44f11487c2 \ + --hash=sha256:80fe20171c6da69c7978ecba33b638e951b85fb92059259edd285ff108b82a6d \ + --hash=sha256:a537ece1bf513e5a88d8cff8a872e12fe8d0f42ef71dd15a5e7520fecd191bbb \ + --hash=sha256:af5aa8127f62bbf03d68f67a956627b1bd0469703a35b3dad28d0c1195e6c7fb \ + --hash=sha256:b0612ccf5de8a480cf08f047b08f9d3fecc12e63d2ee91769cb19d7290614c23 \ + --hash=sha256:ba0caa8aa184196daa6e574d997a33867d6d10234018012d35f86d46024a2a95 \ + --hash=sha256:bae63772408fd63ad836ec569a7c8f444dd32863d0c67f6e0b25ebbd606afa95 \ + --hash=sha256:c7a32a7f0d89f7d30cb8f4a84bdbd56d1eb88b78a2434534f62c71dac538c450 \ + --hash=sha256:ce5e7185599f89b0e391e2f29cc12dc2e206167380cea49b33beda4891be2fe1 \ + --hash=sha256:d8ba24cb31525492ea71b6aac11a4adac91d828aadeff7c4586541bf5dc34d2f \ + --hash=sha256:d97d3efd61404af7e5721a0e74d81cdbfc6e5f97e11e731bb6d090e30a7b62b2 \ + --hash=sha256:e90883f9f85288f423c77b3f5a6f4486375636f25f793165112679a7b6363b35 \ + --hash=sha256:e9e6a7e4d38f7e8dda9edd1433af5170c597336c1a74b4693c5cb75ab2b30f2a \ + --hash=sha256:ec6cfdd2e0399cb79ba4dcffb2332b94d9696c52272ff9d48a630c5dca5e325a \ + --hash=sha256:f416c35efee3e6a6c9ab7716d9e57aa0a49981be915963a82697952cba1353e1 # via readme-renderer pkginfo==1.10.0 \ --hash=sha256:5df73835398d10db79f8eecd5cd86b1f6d29317589ea70796994d49399af6297 \ diff --git a/tools/publish/requirements_linux.txt b/tools/publish/requirements_linux.txt index 98f119b3c9..529a6d68e0 100644 --- a/tools/publish/requirements_linux.txt +++ b/tools/publish/requirements_linux.txt @@ -256,23 +256,33 @@ more-itertools==10.7.0 \ # via # jaraco-classes # jaraco-functools -nh3==0.2.18 \ - --hash=sha256:0411beb0589eacb6734f28d5497ca2ed379eafab8ad8c84b31bb5c34072b7164 \ - --hash=sha256:14c5a72e9fe82aea5fe3072116ad4661af5cf8e8ff8fc5ad3450f123e4925e86 \ - --hash=sha256:19aaba96e0f795bd0a6c56291495ff59364f4300d4a39b29a0abc9cb3774a84b \ - --hash=sha256:34c03fa78e328c691f982b7c03d4423bdfd7da69cd707fe572f544cf74ac23ad \ - --hash=sha256:36c95d4b70530b320b365659bb5034341316e6a9b30f0b25fa9c9eff4c27a204 \ - --hash=sha256:3a157ab149e591bb638a55c8c6bcb8cdb559c8b12c13a8affaba6cedfe51713a \ - --hash=sha256:42c64511469005058cd17cc1537578eac40ae9f7200bedcfd1fc1a05f4f8c200 \ - --hash=sha256:5f36b271dae35c465ef5e9090e1fdaba4a60a56f0bb0ba03e0932a66f28b9189 \ - --hash=sha256:6955369e4d9f48f41e3f238a9e60f9410645db7e07435e62c6a9ea6135a4907f \ - --hash=sha256:7b7c2a3c9eb1a827d42539aa64091640bd275b81e097cd1d8d82ef91ffa2e811 \ - --hash=sha256:8ce0f819d2f1933953fca255db2471ad58184a60508f03e6285e5114b6254844 \ - --hash=sha256:94a166927e53972a9698af9542ace4e38b9de50c34352b962f4d9a7d4c927af4 \ - --hash=sha256:a7f1b5b2c15866f2db413a3649a8fe4fd7b428ae58be2c0f6bca5eefd53ca2be \ - --hash=sha256:c8b3a1cebcba9b3669ed1a84cc65bf005728d2f0bc1ed2a6594a992e817f3a50 \ - --hash=sha256:de3ceed6e661954871d6cd78b410213bdcb136f79aafe22aa7182e028b8c7307 \ - --hash=sha256:f0eca9ca8628dbb4e916ae2491d72957fdd35f7a5d326b7032a345f111ac07fe +nh3==0.3.0 \ + --hash=sha256:0649464ac8eee018644aacbc103874ccbfac80e3035643c3acaab4287e36e7f5 \ + --hash=sha256:16f8670201f7e8e0e05ed1a590eb84bfa51b01a69dd5caf1d3ea57733de6a52f \ + --hash=sha256:1adeb1062a1c2974bc75b8d1ecb014c5fd4daf2df646bbe2831f7c23659793f9 \ + --hash=sha256:37d3003d98dedca6cd762bf88f2e70b67f05100f6b949ffe540e189cc06887f9 \ + --hash=sha256:389d93d59b8214d51c400fb5b07866c2a4f79e4e14b071ad66c92184fec3a392 \ + --hash=sha256:3f1b4f8a264a0c86ea01da0d0c390fe295ea0bcacc52c2103aca286f6884f518 \ + --hash=sha256:423201bbdf3164a9e09aa01e540adbb94c9962cc177d5b1cbb385f5e1e79216e \ + --hash=sha256:634e34e6162e0408e14fb61d5e69dbaea32f59e847cfcfa41b66100a6b796f62 \ + --hash=sha256:6d68fa277b4a3cf04e5c4b84dd0c6149ff7d56c12b3e3fab304c525b850f613d \ + --hash=sha256:7275fdffaab10cc5801bf026e3c089d8de40a997afc9e41b981f7ac48c5aa7d5 \ + --hash=sha256:7852f038a054e0096dac12b8141191e02e93e0b4608c4b993ec7d4ffafea4e49 \ + --hash=sha256:7c915060a2c8131bef6a29f78debc29ba40859b6dbe2362ef9e5fd44f11487c2 \ + --hash=sha256:80fe20171c6da69c7978ecba33b638e951b85fb92059259edd285ff108b82a6d \ + --hash=sha256:a537ece1bf513e5a88d8cff8a872e12fe8d0f42ef71dd15a5e7520fecd191bbb \ + --hash=sha256:af5aa8127f62bbf03d68f67a956627b1bd0469703a35b3dad28d0c1195e6c7fb \ + --hash=sha256:b0612ccf5de8a480cf08f047b08f9d3fecc12e63d2ee91769cb19d7290614c23 \ + --hash=sha256:ba0caa8aa184196daa6e574d997a33867d6d10234018012d35f86d46024a2a95 \ + --hash=sha256:bae63772408fd63ad836ec569a7c8f444dd32863d0c67f6e0b25ebbd606afa95 \ + --hash=sha256:c7a32a7f0d89f7d30cb8f4a84bdbd56d1eb88b78a2434534f62c71dac538c450 \ + --hash=sha256:ce5e7185599f89b0e391e2f29cc12dc2e206167380cea49b33beda4891be2fe1 \ + --hash=sha256:d8ba24cb31525492ea71b6aac11a4adac91d828aadeff7c4586541bf5dc34d2f \ + --hash=sha256:d97d3efd61404af7e5721a0e74d81cdbfc6e5f97e11e731bb6d090e30a7b62b2 \ + --hash=sha256:e90883f9f85288f423c77b3f5a6f4486375636f25f793165112679a7b6363b35 \ + --hash=sha256:e9e6a7e4d38f7e8dda9edd1433af5170c597336c1a74b4693c5cb75ab2b30f2a \ + --hash=sha256:ec6cfdd2e0399cb79ba4dcffb2332b94d9696c52272ff9d48a630c5dca5e325a \ + --hash=sha256:f416c35efee3e6a6c9ab7716d9e57aa0a49981be915963a82697952cba1353e1 # via readme-renderer pkginfo==1.10.0 \ --hash=sha256:5df73835398d10db79f8eecd5cd86b1f6d29317589ea70796994d49399af6297 \ diff --git a/tools/publish/requirements_universal.txt b/tools/publish/requirements_universal.txt index 58625a4aad..7e67221c57 100644 --- a/tools/publish/requirements_universal.txt +++ b/tools/publish/requirements_universal.txt @@ -256,23 +256,33 @@ more-itertools==10.7.0 \ # via # jaraco-classes # jaraco-functools -nh3==0.2.18 \ - --hash=sha256:0411beb0589eacb6734f28d5497ca2ed379eafab8ad8c84b31bb5c34072b7164 \ - --hash=sha256:14c5a72e9fe82aea5fe3072116ad4661af5cf8e8ff8fc5ad3450f123e4925e86 \ - --hash=sha256:19aaba96e0f795bd0a6c56291495ff59364f4300d4a39b29a0abc9cb3774a84b \ - --hash=sha256:34c03fa78e328c691f982b7c03d4423bdfd7da69cd707fe572f544cf74ac23ad \ - --hash=sha256:36c95d4b70530b320b365659bb5034341316e6a9b30f0b25fa9c9eff4c27a204 \ - --hash=sha256:3a157ab149e591bb638a55c8c6bcb8cdb559c8b12c13a8affaba6cedfe51713a \ - --hash=sha256:42c64511469005058cd17cc1537578eac40ae9f7200bedcfd1fc1a05f4f8c200 \ - --hash=sha256:5f36b271dae35c465ef5e9090e1fdaba4a60a56f0bb0ba03e0932a66f28b9189 \ - --hash=sha256:6955369e4d9f48f41e3f238a9e60f9410645db7e07435e62c6a9ea6135a4907f \ - --hash=sha256:7b7c2a3c9eb1a827d42539aa64091640bd275b81e097cd1d8d82ef91ffa2e811 \ - --hash=sha256:8ce0f819d2f1933953fca255db2471ad58184a60508f03e6285e5114b6254844 \ - --hash=sha256:94a166927e53972a9698af9542ace4e38b9de50c34352b962f4d9a7d4c927af4 \ - --hash=sha256:a7f1b5b2c15866f2db413a3649a8fe4fd7b428ae58be2c0f6bca5eefd53ca2be \ - --hash=sha256:c8b3a1cebcba9b3669ed1a84cc65bf005728d2f0bc1ed2a6594a992e817f3a50 \ - --hash=sha256:de3ceed6e661954871d6cd78b410213bdcb136f79aafe22aa7182e028b8c7307 \ - --hash=sha256:f0eca9ca8628dbb4e916ae2491d72957fdd35f7a5d326b7032a345f111ac07fe +nh3==0.3.0 \ + --hash=sha256:0649464ac8eee018644aacbc103874ccbfac80e3035643c3acaab4287e36e7f5 \ + --hash=sha256:16f8670201f7e8e0e05ed1a590eb84bfa51b01a69dd5caf1d3ea57733de6a52f \ + --hash=sha256:1adeb1062a1c2974bc75b8d1ecb014c5fd4daf2df646bbe2831f7c23659793f9 \ + --hash=sha256:37d3003d98dedca6cd762bf88f2e70b67f05100f6b949ffe540e189cc06887f9 \ + --hash=sha256:389d93d59b8214d51c400fb5b07866c2a4f79e4e14b071ad66c92184fec3a392 \ + --hash=sha256:3f1b4f8a264a0c86ea01da0d0c390fe295ea0bcacc52c2103aca286f6884f518 \ + --hash=sha256:423201bbdf3164a9e09aa01e540adbb94c9962cc177d5b1cbb385f5e1e79216e \ + --hash=sha256:634e34e6162e0408e14fb61d5e69dbaea32f59e847cfcfa41b66100a6b796f62 \ + --hash=sha256:6d68fa277b4a3cf04e5c4b84dd0c6149ff7d56c12b3e3fab304c525b850f613d \ + --hash=sha256:7275fdffaab10cc5801bf026e3c089d8de40a997afc9e41b981f7ac48c5aa7d5 \ + --hash=sha256:7852f038a054e0096dac12b8141191e02e93e0b4608c4b993ec7d4ffafea4e49 \ + --hash=sha256:7c915060a2c8131bef6a29f78debc29ba40859b6dbe2362ef9e5fd44f11487c2 \ + --hash=sha256:80fe20171c6da69c7978ecba33b638e951b85fb92059259edd285ff108b82a6d \ + --hash=sha256:a537ece1bf513e5a88d8cff8a872e12fe8d0f42ef71dd15a5e7520fecd191bbb \ + --hash=sha256:af5aa8127f62bbf03d68f67a956627b1bd0469703a35b3dad28d0c1195e6c7fb \ + --hash=sha256:b0612ccf5de8a480cf08f047b08f9d3fecc12e63d2ee91769cb19d7290614c23 \ + --hash=sha256:ba0caa8aa184196daa6e574d997a33867d6d10234018012d35f86d46024a2a95 \ + --hash=sha256:bae63772408fd63ad836ec569a7c8f444dd32863d0c67f6e0b25ebbd606afa95 \ + --hash=sha256:c7a32a7f0d89f7d30cb8f4a84bdbd56d1eb88b78a2434534f62c71dac538c450 \ + --hash=sha256:ce5e7185599f89b0e391e2f29cc12dc2e206167380cea49b33beda4891be2fe1 \ + --hash=sha256:d8ba24cb31525492ea71b6aac11a4adac91d828aadeff7c4586541bf5dc34d2f \ + --hash=sha256:d97d3efd61404af7e5721a0e74d81cdbfc6e5f97e11e731bb6d090e30a7b62b2 \ + --hash=sha256:e90883f9f85288f423c77b3f5a6f4486375636f25f793165112679a7b6363b35 \ + --hash=sha256:e9e6a7e4d38f7e8dda9edd1433af5170c597336c1a74b4693c5cb75ab2b30f2a \ + --hash=sha256:ec6cfdd2e0399cb79ba4dcffb2332b94d9696c52272ff9d48a630c5dca5e325a \ + --hash=sha256:f416c35efee3e6a6c9ab7716d9e57aa0a49981be915963a82697952cba1353e1 # via readme-renderer pkginfo==1.10.0 \ --hash=sha256:5df73835398d10db79f8eecd5cd86b1f6d29317589ea70796994d49399af6297 \ diff --git a/tools/publish/requirements_windows.txt b/tools/publish/requirements_windows.txt index 374541d96f..4a4d1ca3ed 100644 --- a/tools/publish/requirements_windows.txt +++ b/tools/publish/requirements_windows.txt @@ -148,23 +148,33 @@ more-itertools==10.7.0 \ # via # jaraco-classes # jaraco-functools -nh3==0.2.18 \ - --hash=sha256:0411beb0589eacb6734f28d5497ca2ed379eafab8ad8c84b31bb5c34072b7164 \ - --hash=sha256:14c5a72e9fe82aea5fe3072116ad4661af5cf8e8ff8fc5ad3450f123e4925e86 \ - --hash=sha256:19aaba96e0f795bd0a6c56291495ff59364f4300d4a39b29a0abc9cb3774a84b \ - --hash=sha256:34c03fa78e328c691f982b7c03d4423bdfd7da69cd707fe572f544cf74ac23ad \ - --hash=sha256:36c95d4b70530b320b365659bb5034341316e6a9b30f0b25fa9c9eff4c27a204 \ - --hash=sha256:3a157ab149e591bb638a55c8c6bcb8cdb559c8b12c13a8affaba6cedfe51713a \ - --hash=sha256:42c64511469005058cd17cc1537578eac40ae9f7200bedcfd1fc1a05f4f8c200 \ - --hash=sha256:5f36b271dae35c465ef5e9090e1fdaba4a60a56f0bb0ba03e0932a66f28b9189 \ - --hash=sha256:6955369e4d9f48f41e3f238a9e60f9410645db7e07435e62c6a9ea6135a4907f \ - --hash=sha256:7b7c2a3c9eb1a827d42539aa64091640bd275b81e097cd1d8d82ef91ffa2e811 \ - --hash=sha256:8ce0f819d2f1933953fca255db2471ad58184a60508f03e6285e5114b6254844 \ - --hash=sha256:94a166927e53972a9698af9542ace4e38b9de50c34352b962f4d9a7d4c927af4 \ - --hash=sha256:a7f1b5b2c15866f2db413a3649a8fe4fd7b428ae58be2c0f6bca5eefd53ca2be \ - --hash=sha256:c8b3a1cebcba9b3669ed1a84cc65bf005728d2f0bc1ed2a6594a992e817f3a50 \ - --hash=sha256:de3ceed6e661954871d6cd78b410213bdcb136f79aafe22aa7182e028b8c7307 \ - --hash=sha256:f0eca9ca8628dbb4e916ae2491d72957fdd35f7a5d326b7032a345f111ac07fe +nh3==0.3.0 \ + --hash=sha256:0649464ac8eee018644aacbc103874ccbfac80e3035643c3acaab4287e36e7f5 \ + --hash=sha256:16f8670201f7e8e0e05ed1a590eb84bfa51b01a69dd5caf1d3ea57733de6a52f \ + --hash=sha256:1adeb1062a1c2974bc75b8d1ecb014c5fd4daf2df646bbe2831f7c23659793f9 \ + --hash=sha256:37d3003d98dedca6cd762bf88f2e70b67f05100f6b949ffe540e189cc06887f9 \ + --hash=sha256:389d93d59b8214d51c400fb5b07866c2a4f79e4e14b071ad66c92184fec3a392 \ + --hash=sha256:3f1b4f8a264a0c86ea01da0d0c390fe295ea0bcacc52c2103aca286f6884f518 \ + --hash=sha256:423201bbdf3164a9e09aa01e540adbb94c9962cc177d5b1cbb385f5e1e79216e \ + --hash=sha256:634e34e6162e0408e14fb61d5e69dbaea32f59e847cfcfa41b66100a6b796f62 \ + --hash=sha256:6d68fa277b4a3cf04e5c4b84dd0c6149ff7d56c12b3e3fab304c525b850f613d \ + --hash=sha256:7275fdffaab10cc5801bf026e3c089d8de40a997afc9e41b981f7ac48c5aa7d5 \ + --hash=sha256:7852f038a054e0096dac12b8141191e02e93e0b4608c4b993ec7d4ffafea4e49 \ + --hash=sha256:7c915060a2c8131bef6a29f78debc29ba40859b6dbe2362ef9e5fd44f11487c2 \ + --hash=sha256:80fe20171c6da69c7978ecba33b638e951b85fb92059259edd285ff108b82a6d \ + --hash=sha256:a537ece1bf513e5a88d8cff8a872e12fe8d0f42ef71dd15a5e7520fecd191bbb \ + --hash=sha256:af5aa8127f62bbf03d68f67a956627b1bd0469703a35b3dad28d0c1195e6c7fb \ + --hash=sha256:b0612ccf5de8a480cf08f047b08f9d3fecc12e63d2ee91769cb19d7290614c23 \ + --hash=sha256:ba0caa8aa184196daa6e574d997a33867d6d10234018012d35f86d46024a2a95 \ + --hash=sha256:bae63772408fd63ad836ec569a7c8f444dd32863d0c67f6e0b25ebbd606afa95 \ + --hash=sha256:c7a32a7f0d89f7d30cb8f4a84bdbd56d1eb88b78a2434534f62c71dac538c450 \ + --hash=sha256:ce5e7185599f89b0e391e2f29cc12dc2e206167380cea49b33beda4891be2fe1 \ + --hash=sha256:d8ba24cb31525492ea71b6aac11a4adac91d828aadeff7c4586541bf5dc34d2f \ + --hash=sha256:d97d3efd61404af7e5721a0e74d81cdbfc6e5f97e11e731bb6d090e30a7b62b2 \ + --hash=sha256:e90883f9f85288f423c77b3f5a6f4486375636f25f793165112679a7b6363b35 \ + --hash=sha256:e9e6a7e4d38f7e8dda9edd1433af5170c597336c1a74b4693c5cb75ab2b30f2a \ + --hash=sha256:ec6cfdd2e0399cb79ba4dcffb2332b94d9696c52272ff9d48a630c5dca5e325a \ + --hash=sha256:f416c35efee3e6a6c9ab7716d9e57aa0a49981be915963a82697952cba1353e1 # via readme-renderer pkginfo==1.10.0 \ --hash=sha256:5df73835398d10db79f8eecd5cd86b1f6d29317589ea70796994d49399af6297 \ From 0b4d9f67544f58cdd9c6e8d7e69f7753ce78d072 Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Thu, 7 Aug 2025 22:01:31 +0900 Subject: [PATCH 358/922] build(deps): bump certifi from 2025.7.14 to 2025.8.3 in /tools/publish (#3145) Bumps [certifi](https://github.com/certifi/python-certifi) from 2025.7.14 to 2025.8.3.
Commits

[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=certifi&package-manager=pip&previous-version=2025.7.14&new-version=2025.8.3)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) ---
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot show ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> --- tools/publish/requirements_darwin.txt | 6 +++--- tools/publish/requirements_linux.txt | 6 +++--- tools/publish/requirements_universal.txt | 6 +++--- tools/publish/requirements_windows.txt | 6 +++--- 4 files changed, 12 insertions(+), 12 deletions(-) diff --git a/tools/publish/requirements_darwin.txt b/tools/publish/requirements_darwin.txt index ee6837a198..11b1ddbea5 100644 --- a/tools/publish/requirements_darwin.txt +++ b/tools/publish/requirements_darwin.txt @@ -6,9 +6,9 @@ backports-tarfile==1.2.0 \ --hash=sha256:77e284d754527b01fb1e6fa8a1afe577858ebe4e9dad8919e34c862cb399bc34 \ --hash=sha256:d75e02c268746e1b8144c278978b6e98e85de6ad16f8e4b0844a154557eca991 # via jaraco-context -certifi==2025.7.14 \ - --hash=sha256:6b31f564a415d79ee77df69d757bb49a5bb53bd9f756cbbe24394ffd6fc1f4b2 \ - --hash=sha256:8ea99dbdfaaf2ba2f9bac77b9249ef62ec5218e7c2b2e903378ed5fccf765995 +certifi==2025.8.3 \ + --hash=sha256:e564105f78ded564e3ae7c923924435e1daa7463faeab5bb932bc53ffae63407 \ + --hash=sha256:f6c12493cfb1b06ba2ff328595af9350c65d6644968e5d3a2ffd78699af217a5 # via requests charset-normalizer==3.4.2 \ --hash=sha256:005fa3432484527f9732ebd315da8da8001593e2cf46a3d817669f062c3d9ed4 \ diff --git a/tools/publish/requirements_linux.txt b/tools/publish/requirements_linux.txt index 529a6d68e0..eee98b98e7 100644 --- a/tools/publish/requirements_linux.txt +++ b/tools/publish/requirements_linux.txt @@ -6,9 +6,9 @@ backports-tarfile==1.2.0 \ --hash=sha256:77e284d754527b01fb1e6fa8a1afe577858ebe4e9dad8919e34c862cb399bc34 \ --hash=sha256:d75e02c268746e1b8144c278978b6e98e85de6ad16f8e4b0844a154557eca991 # via jaraco-context -certifi==2025.7.14 \ - --hash=sha256:6b31f564a415d79ee77df69d757bb49a5bb53bd9f756cbbe24394ffd6fc1f4b2 \ - --hash=sha256:8ea99dbdfaaf2ba2f9bac77b9249ef62ec5218e7c2b2e903378ed5fccf765995 +certifi==2025.8.3 \ + --hash=sha256:e564105f78ded564e3ae7c923924435e1daa7463faeab5bb932bc53ffae63407 \ + --hash=sha256:f6c12493cfb1b06ba2ff328595af9350c65d6644968e5d3a2ffd78699af217a5 # via requests cffi==1.17.1 \ --hash=sha256:045d61c734659cc045141be4bae381a41d89b741f795af1dd018bfb532fd0df8 \ diff --git a/tools/publish/requirements_universal.txt b/tools/publish/requirements_universal.txt index 7e67221c57..85648b24e9 100644 --- a/tools/publish/requirements_universal.txt +++ b/tools/publish/requirements_universal.txt @@ -6,9 +6,9 @@ backports-tarfile==1.2.0 ; python_full_version < '3.12' \ --hash=sha256:77e284d754527b01fb1e6fa8a1afe577858ebe4e9dad8919e34c862cb399bc34 \ --hash=sha256:d75e02c268746e1b8144c278978b6e98e85de6ad16f8e4b0844a154557eca991 # via jaraco-context -certifi==2025.7.14 \ - --hash=sha256:6b31f564a415d79ee77df69d757bb49a5bb53bd9f756cbbe24394ffd6fc1f4b2 \ - --hash=sha256:8ea99dbdfaaf2ba2f9bac77b9249ef62ec5218e7c2b2e903378ed5fccf765995 +certifi==2025.8.3 \ + --hash=sha256:e564105f78ded564e3ae7c923924435e1daa7463faeab5bb932bc53ffae63407 \ + --hash=sha256:f6c12493cfb1b06ba2ff328595af9350c65d6644968e5d3a2ffd78699af217a5 # via requests cffi==1.17.1 ; platform_python_implementation != 'PyPy' and sys_platform == 'linux' \ --hash=sha256:045d61c734659cc045141be4bae381a41d89b741f795af1dd018bfb532fd0df8 \ diff --git a/tools/publish/requirements_windows.txt b/tools/publish/requirements_windows.txt index 4a4d1ca3ed..b2a01f474f 100644 --- a/tools/publish/requirements_windows.txt +++ b/tools/publish/requirements_windows.txt @@ -6,9 +6,9 @@ backports-tarfile==1.2.0 \ --hash=sha256:77e284d754527b01fb1e6fa8a1afe577858ebe4e9dad8919e34c862cb399bc34 \ --hash=sha256:d75e02c268746e1b8144c278978b6e98e85de6ad16f8e4b0844a154557eca991 # via jaraco-context -certifi==2025.7.14 \ - --hash=sha256:6b31f564a415d79ee77df69d757bb49a5bb53bd9f756cbbe24394ffd6fc1f4b2 \ - --hash=sha256:8ea99dbdfaaf2ba2f9bac77b9249ef62ec5218e7c2b2e903378ed5fccf765995 +certifi==2025.8.3 \ + --hash=sha256:e564105f78ded564e3ae7c923924435e1daa7463faeab5bb932bc53ffae63407 \ + --hash=sha256:f6c12493cfb1b06ba2ff328595af9350c65d6644968e5d3a2ffd78699af217a5 # via requests charset-normalizer==3.4.2 \ --hash=sha256:005fa3432484527f9732ebd315da8da8001593e2cf46a3d817669f062c3d9ed4 \ From babfc2b336ba9386f3a6c30ac5017969f7dae031 Mon Sep 17 00:00:00 2001 From: Omar Droubi Date: Thu, 7 Aug 2025 15:19:15 +0200 Subject: [PATCH 359/922] fix: Fix whl_library in bazel vendor mode (#3096) Resolves https://github.com/bazel-contrib/rules_python/issues/3079 - Added PYTHONHOME to whl_library execution environment. Without it, the python interpreter is getting confused where it's running from when bazel --vendor_dir is used - In _get_toolchain_unix_cflags, the real path for python is used instead of the symlink. --------- Co-authored-by: Omar Aldrroubi Co-authored-by: Ignas Anikevicius <240938+aignas@users.noreply.github.com> --- CHANGELOG.md | 1 + python/private/pypi/whl_library.bzl | 38 ++++++++++++++++++++++++++++- 2 files changed, 38 insertions(+), 1 deletion(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 08991c6107..1e7441beab 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -103,6 +103,7 @@ END_UNRELEASED_TEMPLATE ([#2985](https://github.com/bazel-contrib/rules_python/issues/2985)). * (toolchains) use "command -v" to find interpreter in `$PATH` ([#3150](https://github.com/bazel-contrib/rules_python/pull/3150)). +* (pypi) `bazel vendor` now works in `bzlmod` ({gh-issue}`3079`). {#v0-0-0-added} ### Added diff --git a/python/private/pypi/whl_library.bzl b/python/private/pypi/whl_library.bzl index 15bb680fea..b1aaf4f062 100644 --- a/python/private/pypi/whl_library.bzl +++ b/python/private/pypi/whl_library.bzl @@ -109,7 +109,11 @@ def _get_toolchain_unix_cflags(rctx, python_interpreter, logger = None): stdout = pypi_repo_utils.execute_checked_stdout( rctx, op = "GetPythonVersionForUnixCflags", - python = python_interpreter, + # python_interpreter by default points to a symlink, however when using bazel in vendor mode, + # and the vendored directory moves around, the execution of python fails, as it's getting confused + # where it's running from. More to the fact that we are executing it in isolated mode "-I", which + # results in PYTHONHOME being ignored. The solution is to run python from it's real directory. + python = python_interpreter.realpath, arguments = [ # Run the interpreter in isolated mode, this options implies -E, -P and -s. # Ensures environment variables are ignored that are set in userspace, such as PYTHONPATH, @@ -198,6 +202,37 @@ def _parse_optional_attrs(rctx, args, extra_pip_args = None): return args +def _get_python_home(rctx, python_interpreter, logger = None): + """Get the PYTHONHOME directory from the selected python interpretter + + Args: + rctx (repository_ctx): The repository context. + python_interpreter (path): The resolved python interpreter. + logger: Optional logger to use for operations. + Returns: + String of PYTHONHOME directory. + """ + + return pypi_repo_utils.execute_checked_stdout( + rctx, + op = "GetPythonHome", + # python_interpreter by default points to a symlink, however when using bazel in vendor mode, + # and the vendored directory moves around, the execution of python fails, as it's getting confused + # where it's running from. More to the fact that we are executing it in isolated mode "-I", which + # results in PYTHONHOME being ignored. The solution is to run python from it's real directory. + python = python_interpreter.realpath, + arguments = [ + # Run the interpreter in isolated mode, this options implies -E, -P and -s. + # Ensures environment variables are ignored that are set in userspace, such as PYTHONPATH, + # which may interfere with this invocation. + "-I", + "-c", + "import sys; print(f'{sys.prefix}', end='')", + ], + srcs = [], + logger = logger, + ) + def _create_repository_execution_environment(rctx, python_interpreter, logger = None): """Create a environment dictionary for processes we spawn with rctx.execute. @@ -210,6 +245,7 @@ def _create_repository_execution_environment(rctx, python_interpreter, logger = """ env = { + "PYTHONHOME": _get_python_home(rctx, python_interpreter, logger), "PYTHONPATH": pypi_repo_utils.construct_pythonpath( rctx, entries = rctx.attr._python_path_entries, From bc788d510401b7013a29d3a8ce3816754b987265 Mon Sep 17 00:00:00 2001 From: Douglas Thor Date: Fri, 8 Aug 2025 14:33:51 -0700 Subject: [PATCH 360/922] docs(gazelle): Use definition lists instead of bullets for Gazelle docs (#3154) Primary updates: + Use Definition List + Move the annotation/directive description to the definition line Secondary updates: + Add missing directive arguments. For example, `# gazelle:python_extension` becomes `# gazelle:python_extension value`. + Address WIP lines for "allowed values" + Link things with `{term}`. Fixes #3142. --- CHANGELOG.md | 9 +- gazelle/README.md | 2 +- gazelle/docs/annotations.md | 27 ++-- gazelle/docs/directives.md | 195 ++++++++++++++----------- gazelle/docs/installation_and_usage.md | 8 +- 5 files changed, 135 insertions(+), 106 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 1e7441beab..2535a0f1dc 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -70,7 +70,10 @@ END_UNRELEASED_TEMPLATE * (gazelle) Switched back to smacker/go-tree-sitter, fixing [#2630](https://github.com/bazel-contrib/rules_python/issues/2630) * (ci) We are now testing on Ubuntu 22.04 for RBE and non-RBE configurations. -* (core) #!/usr/bin/env bash is now used as a shebang in the stage1 bootstrap template. +* (core) `#!/usr/bin/env bash` is now used as a shebang in the stage1 bootstrap template. +* (gazelle:docs) The Gazelle docs have been migrated from {gh-path}`gazelle/README.md` to + {gh-path}`gazelle/docs` and are now available on the primary documentation site + at https://rules-python.readthedocs.io/en/latest/gazelle/docs/index.html {#v0-0-0-fixed} ### Fixed @@ -108,8 +111,8 @@ END_UNRELEASED_TEMPLATE {#v0-0-0-added} ### Added * (repl) Default stub now has tab completion, where `readline` support is available, - see ([#3114](https://github.com/bazel-contrib/rules_python/pull/3114)). - ([#3114](https://github.com/bazel-contrib/rules_python/pull/3114)). + see ([#3114](https://github.com/bazel-contrib/rules_python/pull/3114)). + ([#3114](https://github.com/bazel-contrib/rules_python/pull/3114)). * (pypi) To configure the environment for `requirements.txt` evaluation, use the newly added developer preview of the `pip.default` tag class. Only `rules_python` and root modules can use this feature. You can also configure custom `config_settings` using `pip.default`. diff --git a/gazelle/README.md b/gazelle/README.md index 128fb1f583..1cbef2d856 100644 --- a/gazelle/README.md +++ b/gazelle/README.md @@ -2,5 +2,5 @@ :::{note} The gazelle plugin docs have been migrated to our primary documentation on -ReadTheDocs. Please see https://rules-python.readthedocs.io/gazelle/docs/index.html. +ReadTheDocs. Please see https://rules-python.readthedocs.io/en/latest/gazelle/docs/index.html. ::: diff --git a/gazelle/docs/annotations.md b/gazelle/docs/annotations.md index cc87543c29..da6e58f7f8 100644 --- a/gazelle/docs/annotations.md +++ b/gazelle/docs/annotations.md @@ -21,23 +21,26 @@ def bar(): # gazelle:annotation_name value The annotations are: -* [`# gazelle:ignore imports`](#ignore) +{.glossary} +[`# gazelle:ignore imports`](#ignore) +: Tells Gazelle to ignore import statements. `imports` is a comma-separated + list of imports to ignore. * Default: n/a * Allowed Values: A comma-separated string of python package names - * Tells Gazelle to ignore import statements. `imports` is a comma-separated - list of imports to ignore. -* [`# gazelle:include_dep targets`](#include-dep) + +[`# gazelle:include_dep targets`](#include-dep) +: Tells Gazelle to include a set of dependencies, even if they are not imported + in a Python module. `targets` is a comma-separated list of target names + to include as dependencies. * Default: n/a - * Allowed Values: A string - * Tells Gazelle to include a set of dependencies, even if they are not imported - in a Python module. `targets` is a comma-separated list of target names - to include as dependencies. -* [`# gazelle:include_pytest_conftest bool`](#include-pytest-conftest) + * Allowed Values: A comma-separated string of targets + +[`# gazelle:include_pytest_conftest bool`](#include-pytest-conftest) +: Whether or not to include a sibling `:conftest` target in the `deps` + of a {bzl:obj}`py_test` target. The default behaviour is to include `:conftest` + (i.e.: `# gazelle:include_pytest_conftest true`). * Default: n/a * Allowed Values: `true`, `false` - * Whether or not to include a sibling `:conftest` target in the `deps` - of a {bzl:obj}`py_test` target. The default behaviour is to include `:conftest` - (i.e.: `# gazelle:include_pytest_conftest true`). ## `ignore` diff --git a/gazelle/docs/directives.md b/gazelle/docs/directives.md index 9221c60823..ecc30a93b5 100644 --- a/gazelle/docs/directives.md +++ b/gazelle/docs/directives.md @@ -15,136 +15,159 @@ the Python-specific directives in use can be found in the The Python-specific directives are: -* [`# gazelle:python_extension`](#python-extension) +{.glossary} +[`# gazelle:python_extension value`](#python-extension) +: Controls whether the Python extension is enabled or not. Sub-packages + inherit this value. * Default: `enabled` * Allowed Values: `enabled`, `disabled` - * Controls whether the Python extension is enabled or not. Sub-packages - inherit this value. -* [`# gazelle:python_root`](#python-root) + +[`# gazelle:python_root`](#python-root) +: Sets a Bazel package as a Python root. This is used on monorepos with + multiple Python projects that don't share the top-level of the workspace + as the root. * Default: n/a * Allowed Values: None. This direcive does not consume values. - * Sets a Bazel package as a Python root. This is used on monorepos with - multiple Python projects that don't share the top-level of the workspace - as the root. -* [`# gazelle:python_manifest_file_name`](#python-manifest-file-name) + +[`# gazelle:python_manifest_file_name value`](#python-manifest-file-name) +: Overrides the default manifest file name. * Default: `gazelle_python.yaml` * Allowed Values: A string - * Overrides the default manifest file name. -* [`# gazelle:python_ignore_files`](#python-ignore-files) + +[`# gazelle:python_ignore_files value`](#python-ignore-files) +: Controls the files which are ignored from the generated targets. * Default: n/a - * Allowed Values: WIP - * Controls the files which are ignored from the generated targets. -* [`# gazelle:python_ignore_dependencies`](#python-ignore-dependencies) + * Allowed Values: A comma-separated list of strings. + +[`# gazelle:python_ignore_dependencies value`](#python-ignore-dependencies) +: Controls the ignored dependencies from the generated targets. * Default: n/a - * Allowed Values: WIP - * Controls the ignored dependencies from the generated targets. -* [`# gazelle:python_validate_import_statements`](#python-validate-import-statements) + * Allowed Values: A comma-separated list of strings. + +[`# gazelle:python_validate_import_statements bool`](#python-validate-import-statements) +: Controls whether the Python import statements should be validated. * Default: `true` * Allowed Values: `true`, `false` - * Controls whether the Python import statements should be validated. -* [`# gazelle:python_generation_mode`](#python-generation-mode) + +[`# gazelle:python_generation_mode value`](#python-generation-mode) +: Controls the target generation mode. * Default: `package` * Allowed Values: `file`, `package`, `project` - * Controls the target generation mode. -* [`# gazelle:python_generation_mode_per_file_include_init`](#python-generation-mode-per-file-include-init) + +[`# gazelle:python_generation_mode_per_file_include_init bool`](#python-generation-mode-per-file-include-init) +: Controls whether `__init__.py` files are included as srcs in each + generated target when target generation mode is "file". * Default: `false` * Allowed Values: `true`, `false` - * Controls whether `__init__.py` files are included as srcs in each - generated target when target generation mode is "file". -* [`# gazelle:python_generation_mode_per_package_require_test_entry_point`](python-generation-mode-per-package-require-test-entry-point) + +[`# gazelle:python_generation_mode_per_package_require_test_entry_point bool`](python-generation-mode-per-package-require-test-entry-point) +: Controls whether a file called `__test__.py` or a target called + `__test__` is required to generate one test target per package in + package mode. * Default: `true` * Allowed Values: `true`, `false` - * Controls whether a file called `__test__.py` or a target called - `__test__` is required to generate one test target per package in - package mode. -* [`# gazelle:python_library_naming_convention`](#python-library-naming-convention) + +[`# gazelle:python_library_naming_convention value`](#python-library-naming-convention) +: Controls the {bzl:obj}`py_library` naming convention. It interpolates + `$package_name$` with the Bazel package name. E.g. if the Bazel package + name is `foo`, setting this to `$package_name$_my_lib` would result in a + generated target named `foo_my_lib`. * Default: `$package_name$` * Allowed Values: A string containing `"$package_name$"` - * Controls the {bzl:obj}`py_library` naming convention. It interpolates - `$package_name$` with the Bazel package name. E.g. if the Bazel package - name is `foo`, setting this to `$package_name$_my_lib` would result in a - generated target named `foo_my_lib`. -* [`# gazelle:python_binary_naming_convention`](#python-binary-naming-convention) + +[`# gazelle:python_binary_naming_convention value`](#python-binary-naming-convention) +: Controls the {bzl:obj}`py_binary` naming convention. Follows the same interpolation + rules as `python_library_naming_convention`. * Default: `$package_name$_bin` * Allowed Values: A string containing `"$package_name$"` - * Controls the {bzl:obj}`py_binary` naming convention. Follows the same interpolation - rules as `python_library_naming_convention`. -* [`# gazelle:python_test_naming_convention`](#python-test-naming-convention) + +[`# gazelle:python_test_naming_convention value`](#python-test-naming-convention) +: Controls the {bzl:obj}`py_test` naming convention. Follows the same interpolation + rules as `python_library_naming_convention`. * Default: `$package_name$_test` * Allowed Values: A string containing `"$package_name$"` - * Controls the {bzl:obj}`py_test` naming convention. Follows the same interpolation - rules as `python_library_naming_convention`. -* [`# gazelle:python_proto_naming_convention`](#python-proto-naming-convention) + +[`# gazelle:python_proto_naming_convention value`](#python-proto-naming-convention) +: Controls the {bzl:obj}`py_proto_library` naming convention. It interpolates + `$proto_name$` with the {bzl:obj}`proto_library` rule name, minus any trailing + `_proto`. E.g. if the {bzl:obj}`proto_library` name is `foo_proto`, setting this + to `$proto_name$_my_lib` would render to `foo_my_lib`. * Default: `$proto_name$_py_pb2` * Allowed Values: A string containing `"$proto_name$"` - * Controls the {bzl:obj}`py_proto_library` naming convention. It interpolates - `$proto_name$` with the {bzl:obj}`proto_library` rule name, minus any trailing - `_proto`. E.g. if the {bzl:obj}`proto_library` name is `foo_proto`, setting this - to `$proto_name$_my_lib` would render to `foo_my_lib`. -* [`# gazelle:resolve py ...`](#resolve-py) + +[`# gazelle:resolve py import-lang import-string label`](#resolve-py) +: Instructs the plugin what target to add as a dependency to satisfy a given + import statement. The syntax is `# gazelle:resolve py import-string label` + where `import-string` is the symbol in the python `import` statement, + and `label` is the Bazel label that Gazelle should write in `deps`. * Default: n/a * Allowed Values: See the [bazel-gazelle docs][gazelle-directives] - * Instructs the plugin what target to add as a dependency to satisfy a given - import statement. The syntax is `# gazelle:resolve py import-string label` - where `import-string` is the symbol in the python `import` statement, - and `label` is the Bazel label that Gazelle should write in `deps`. -* [`# gazelle:python_default_visibility labels`](python-default-visibility) + +[`# gazelle:python_default_visibility labels`](python-default-visibility) +: Instructs gazelle to use these visibility labels on all python targets. + `labels` is a comma-separated list of labels (without spaces). * Default: `//$python_root$:__subpackages__` * Allowed Values: A string - * Instructs gazelle to use these visibility labels on all python targets. - `labels` is a comma-separated list of labels (without spaces). -* [`# gazelle:python_visibility label`](python-visibility) + +[`# gazelle:python_visibility label`](python-visibility) +: Appends additional visibility labels to each generated target. This r + directive can be set multiple times. * Default: n/a * Allowed Values: A string - * Appends additional visibility labels to each generated target. This r - directive can be set multiple times. -* [`# gazelle:python_test_file_pattern`](python-test-file-pattern) + +[`# gazelle:python_test_file_pattern value`](python-test-file-pattern) +: Filenames matching these comma-separated {command}`glob`s will be mapped to + {bzl:obj}`py_test` targets. * Default: `*_test.py,test_*.py` * Allowed Values: A glob string - * Filenames matching these comma-separated {command}`glob`s will be mapped to - {bzl:obj}`py_test` targets. -* [`# gazelle:python_label_convention`](#python-label-convention) + +[`# gazelle:python_label_convention value`](#python-label-convention) +: Defines the format of the distribution name in labels to third-party deps. + Useful for using Gazelle plugin with other rules with different repository + conventions (e.g. `rules_pycross`). Full label is always prepended with + the `pip` repository name, e.g. `@pip//numpy` if your + `MODULE.bazel` has `use_repo(pip, "pip")` or `@pypi//numpy` + if your `MODULE.bazel` has `use_repo(pip, "pypi")`. * Default: `$distribution_name$` * Allowed Values: A string - * Defines the format of the distribution name in labels to third-party deps. - Useful for using Gazelle plugin with other rules with different repository - conventions (e.g. `rules_pycross`). Full label is always prepended with - the `pip` repository name, e.g. `@pip//numpy` if your - `MODULE.bazel` has `use_repo(pip, "pip")` or `@pypi//numpy` - if your `MODULE.bazel` has `use_repo(pip, "pypi")`. -* [`# gazelle:python_label_normalization`](#python-label-normalization) + +[`# gazelle:python_label_normalization value`](#python-label-normalization) +: Controls how distribution names in labels to third-party deps are + normalized. Useful for using Gazelle plugin with other rules with different + label conventions (e.g. `rules_pycross` uses PEP-503). * Default: `snake_case` * Allowed Values: `snake_case`, `none`, `pep503` - * Controls how distribution names in labels to third-party deps are - normalized. Useful for using Gazelle plugin with other rules with different - label conventions (e.g. `rules_pycross` uses PEP-503). -* [`# gazelle:python_experimental_allow_relative_imports`](#python-experimental-allow-relative-imports) + +[`# gazelle:python_experimental_allow_relative_imports bool`](#python-experimental-allow-relative-imports) +: Controls whether Gazelle resolves dependencies for import statements that + use paths relative to the current package. * Default: `false` * Allowed Values: `true`, `false` - * Controls whether Gazelle resolves dependencies for import statements that - use paths relative to the current package. -* [`# gazelle:python_generate_pyi_deps`](#python-generate-pyi-deps) + +[`# gazelle:python_generate_pyi_deps bool`](#python-generate-pyi-deps) +: Controls whether to generate a separate `pyi_deps` attribute for + type-checking dependencies or merge them into the regular `deps` + attribute. When `false` (default), type-checking dependencies are + merged into `deps` for backward compatibility. When `true`, generates + separate `pyi_deps`. Imports in blocks with the format + `if typing.TYPE_CHECKING:` or `if TYPE_CHECKING:` and type-only stub + packages (eg. boto3-stubs) are recognized as type-checking dependencies. * Default: `false` * Allowed Values: `true`, `false` - * Controls whether to generate a separate `pyi_deps` attribute for - type-checking dependencies or merge them into the regular `deps` - attribute. When `false` (default), type-checking dependencies are - merged into `deps` for backward compatibility. When `true`, generates - separate `pyi_deps`. Imports in blocks with the format - `if typing.TYPE_CHECKING:` or `if TYPE_CHECKING:` and type-only stub - packages (eg. boto3-stubs) are recognized as type-checking dependencies. -* [`# gazelle:python_generate_proto`](#python-generate-proto) + +[`# gazelle:python_generate_proto bool`](#python-generate-proto) +: Controls whether to generate a {bzl:obj}`py_proto_library` for each + {bzl:obj}`proto_library` in the package. By default we load this rule from the + `@protobuf` repository; use `gazelle:map_kind` if you need to load this + from somewhere else. * Default: `false` * Allowed Values: `true`, `false` - * Controls whether to generate a {bzl:obj}`py_proto_library` for each - {bzl:obj}`proto_library` in the package. By default we load this rule from the - `@protobuf` repository; use `gazelle:map_kind` if you need to load this - from somewhere else. -* [`# gazelle:python_resolve_sibling_imports`](#python-resolve-sibling-imports) + +[`# gazelle:python_resolve_sibling_imports bool`](#python-resolve-sibling-imports) +: Allows absolute imports to be resolved to sibling modules (Python 2's + behavior without `absolute_import`). * Default: `false` * Allowed Values: `true`, `false` - * Allows absolute imports to be resolved to sibling modules (Python 2's - behavior without `absolute_import`). ## `python_extension` diff --git a/gazelle/docs/installation_and_usage.md b/gazelle/docs/installation_and_usage.md index 123f30a068..1858f41bb9 100644 --- a/gazelle/docs/installation_and_usage.md +++ b/gazelle/docs/installation_and_usage.md @@ -156,7 +156,7 @@ you edit Python code, and it should update your `BUILD` files correctly. ### Libraries Python source files are those ending in `.py` that are not matched as a test -file via the `# gazelle:python_test_file_pattern` directive. By default, +file via the {term}`# gazelle:python_test_file_pattern value` directive. By default, python source files are all `*.py` files except for `*_test.py` and `test_*.py`. @@ -179,7 +179,7 @@ dependencies are added to the `deps` attribute of the target. A {bzl:obj}`py_test` target is added to the `BUILD(.bazel)` file when gazelle encounters a file named `__test__.py` or when files matching the -`# gazelle:python_test_file_pattern` directive are found. +{term}`# gazelle:python_test_file_pattern value` directive are found. For example, if we had a folder that is a package named "foo" we could have a Python file named `foo_test.py` and gazelle would create a {bzl:obj}`py_test` @@ -198,7 +198,7 @@ py_test( ``` You can control the naming convention for test targets using the -`# gazelle:python_test_naming_convention` directive. +{term}`# gazelle:python_test_naming_convention value` directive. ### Binaries @@ -217,7 +217,7 @@ if __name == "__main__": Gazelle will create a {bzl:obj}`py_binary` target for every module with such a line, with the target name the same as the module name. -If the `# gazelle:python_generation_mode` directive is set to `file`, then +If the {term}`# gazelle:python_generation_mode value` directive is set to `file`, then instead of one {bzl:obj}`py_binary` target per module, Gazelle will create one {bzl:obj}`py_binary` target for each file with such a line, and the name of the target will match the name of the script. From 89e0f63c576d152293d5c866394f31b08a569ad3 Mon Sep 17 00:00:00 2001 From: Ignas Anikevicius <240938+aignas@users.noreply.github.com> Date: Sun, 10 Aug 2025 11:08:08 +0900 Subject: [PATCH 361/922] chore(toolchains): start pulling toolchains from 20250808 release (#3116) This ships the latest versions of the toolchains fixing: * `bootstrap_impl=script` behaviour on `3.14`. * `matplotlib` UI with the latest Python toolchain versions. Fixes #2983 Fixes #2540 Related #3155 --- CHANGELOG.md | 12 +- .../private/hermetic_runtime_repo_setup.bzl | 1 + python/versions.bzl | 172 +++++++++--------- tests/python/python_tests.bzl | 4 +- tests/toolchains/python_toolchain_test.py | 18 +- .../transitions/transitions_tests.bzl | 2 +- 6 files changed, 104 insertions(+), 105 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 2535a0f1dc..04979e8c41 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -60,13 +60,13 @@ END_UNRELEASED_TEMPLATE * (gazelle) Types for exposed members of `python.ParserOutput` are now all public. * (gazelle) Removed the requirement for `__init__.py`, `__main__.py`, or `__test__.py` files to be present in a directory to generate a `BUILD.bazel` file. -* (toolchain) Updated the following toolchains to build 20250708 to patch CVE-2025-47273: +* (toolchain) Updated the following toolchains to build [20250808] to patch CVE-2025-47273: * 3.9.23 * 3.10.18 * 3.11.13 * 3.12.11 - * 3.14.0b4 -* (toolchain) Python 3.13 now references 3.13.5 + * 3.14.0rc1 +* (toolchain) Python 3.13 now references 3.13.6 * (gazelle) Switched back to smacker/go-tree-sitter, fixing [#2630](https://github.com/bazel-contrib/rules_python/issues/2630) * (ci) We are now testing on Ubuntu 22.04 for RBE and non-RBE configurations. @@ -75,6 +75,8 @@ END_UNRELEASED_TEMPLATE {gh-path}`gazelle/docs` and are now available on the primary documentation site at https://rules-python.readthedocs.io/en/latest/gazelle/docs/index.html +[20250808]: https://github.com/astral-sh/python-build-standalone/releases/tag/20250808 + {#v0-0-0-fixed} ### Fixed * (pypi) Fixes an issue where builds using a `bazel vendor` vendor directory @@ -126,8 +128,8 @@ END_UNRELEASED_TEMPLATE * (toolchain) Add toolchains for aarch64 windows for * 3.11.13 * 3.12.11 - * 3.13.5 - * 3.14.0b4 + * 3.13.6 + * 3.14.0rc1 * (gazelle): New annotation `gazelle:include_pytest_conftest`. When not set (the default) or `true`, gazelle will inject any `conftest.py` file found in the same directory as a {obj}`py_test` target to that {obj}`py_test` target's `deps`. diff --git a/python/private/hermetic_runtime_repo_setup.bzl b/python/private/hermetic_runtime_repo_setup.bzl index 6910ea14a1..a35cd6bae3 100644 --- a/python/private/hermetic_runtime_repo_setup.bzl +++ b/python/private/hermetic_runtime_repo_setup.bzl @@ -239,6 +239,7 @@ def define_hermetic_runtime_toolchain_impl( py_cc_toolchain( name = "py_cc_toolchain", headers = ":python_headers", + # TODO #3155: add libctl, libtk libs = ":libpython", python_version = python_version, ) diff --git a/python/versions.bzl b/python/versions.bzl index 3f9d6c57a8..30929f82fd 100644 --- a/python/versions.bzl +++ b/python/versions.bzl @@ -190,17 +190,17 @@ TOOL_VERSIONS = { "strip_prefix": "python", }, "3.9.23": { - "url": "20250708/cpython-{python_version}+20250708-{platform}-{build}.tar.gz", + "url": "20250808/cpython-{python_version}+20250808-{platform}-{build}.tar.gz", "sha256": { - "aarch64-apple-darwin": "e7653969f362c099158f4bd8daa06a7545871a1f50b7c088ac875c46e68481cc", - "aarch64-unknown-linux-gnu": "b6d1bb94972d79d21661d821621edd23be6e7fad258f394a895b392282eef71a", - "ppc64le-unknown-linux-gnu": "b90457324ab106fc5146388e418d08502bb1393c2bec25f15bc5cc2497b13fd2", - "riscv64-unknown-linux-gnu": "00a2e2e031f80731e5d812de62a66ff577692ec555c1e9b5798a448ae3671f81", - "s390x-unknown-linux-gnu": "6b1c749813d251460a7a2c0b5bc751cd6f25f82a1b31a01c3efbc5cbece7c55c", - "x86_64-apple-darwin": "cdb39a635a0b8e4487555935fecdbb6f6eaab9706e0e71a0cf61a5728b6b3819", - "x86_64-pc-windows-msvc": "94925d6fafa4d823336081f77f38912f9a6e76c27243e656a90d7480f8c5eceb", - "x86_64-unknown-linux-gnu": "a11d8d52587db34f370a2f56d7310b727de180973653d865f097b7880ead3e2d", - "x86_64-unknown-linux-musl": "dfdfbf35bf9d087398b6b9c5627f86d5921c4a9284cfeeef48e3f2980b4ad762", + "aarch64-apple-darwin": "d32da9eae3f516cc0bd8240bfef54dede757d6daf1d8cf605eacbc8a205884e8", + "aarch64-unknown-linux-gnu": "0318b6c9ad6fb229da8d40aa3671ee27eeb678530246a1b172b72071f76091bc", + "ppc64le-unknown-linux-gnu": "b40b3509dc72abb21f4310f0e94678b36ff73432dc84c41fea132a51c4017f79", + "riscv64-unknown-linux-gnu": "a7d847dc62177cf06237dfa26c317148b22418ded51aa89e8cf7242784293ad4", + "s390x-unknown-linux-gnu": "425abe5d3ec98e9b18c908209a4ffe239a283ee648e0eea65821e45f074689e7", + "x86_64-apple-darwin": "c1bfab90aea566ffaeff65299a20503a880ea93054bbd8bbed98f4f11e9e7383", + "x86_64-pc-windows-msvc": "fb400b25cbcbfed6aeaaca8d9a3cdf1a09b602bf5ed6d1ae7075cde40c1cd81e", + "x86_64-unknown-linux-gnu": "77fd3fa10abbb08949eda70ca7fb94f72e2f9e0016611be328a7b31c3aa9894d", + "x86_64-unknown-linux-musl": "a8a0df23bc1bc050ed8730c65d818382667cf37ba96a08fccd5bb12a689e6a1c", }, "strip_prefix": "python", }, @@ -340,17 +340,17 @@ TOOL_VERSIONS = { "strip_prefix": "python", }, "3.10.18": { - "url": "20250708/cpython-{python_version}+20250708-{platform}-{build}.tar.gz", + "url": "20250808/cpython-{python_version}+20250808-{platform}-{build}.tar.gz", "sha256": { - "aarch64-apple-darwin": "8b70988e7d7d930e18179f0c464b5a1fb595b64c7a282b78c5e8ff2c8fcc51f2", - "aarch64-unknown-linux-gnu": "eda9db45b2f1e4987559f7026fc655a8521d8974a6ae3a53b3d61e3f59dd0938", - "ppc64le-unknown-linux-gnu": "f247afed2d72cff4b3e5723f345ae4c07fd115985c188217a47c18e1249fed9a", - "riscv64-unknown-linux-gnu": "af2f9a619f2343627488e64428cd348d127e02723c53dd052eb66c6e8cebbf1d", - "s390x-unknown-linux-gnu": "e8db7743627e60ffcec4e1ac1e3098718f6e7aa52731e0d6b9b2856f90a7c338", - "x86_64-apple-darwin": "af2b6fb02e8f9a266c4ed5b173634c8bcfa38d6b282ecdfa137faec555eea971", - "x86_64-pc-windows-msvc": "841ca8f71be9a01bdf6c72e5fe8bee1a4988f8637ead801d6be095f5bedce0f5", - "x86_64-unknown-linux-gnu": "17476eee3a20d06dd4cd58594cedc4badfa29984ae727ff1df119ec8e206ab35", - "x86_64-unknown-linux-musl": "3b38bfa0ecce16b2f987a26602e05e332acec1864a6cc4b4753bdaaf12ce08ac", + "aarch64-apple-darwin": "a94c02b2d597cd6b075a713fe4e9a909cc97ca6a3b2b2ce86eda21be2062d48e", + "aarch64-unknown-linux-gnu": "ef7de3b715d519e246d98ff7856247f7f7b357068705f09c6f300b7e7b76c701", + "ppc64le-unknown-linux-gnu": "f580efed11cc54e1a221c052e8bc88bfbc12844d3ca8949da828351a1232386e", + "riscv64-unknown-linux-gnu": "0d7e460e30203a9225b6f417ae972f66415a1cc0e32b37ebc48d195816282669", + "s390x-unknown-linux-gnu": "d4ada974daadb08a0184c19232ee3b03b3137aa70609760e1a94aaf7b12989ef", + "x86_64-apple-darwin": "da96fe2ba841640215788ddb9f151f03629360e37fcb94d4f76e5095b87df0d4", + "x86_64-pc-windows-msvc": "a648f3c9d136985ccfe57a5507e73d9d0839f7fd09eebd7c247857f2feaecb2a", + "x86_64-unknown-linux-gnu": "0b310a73bb9e7a495dbcad5f685e508ca2e7b36ee8f29301a52285730c425789", + "x86_64-unknown-linux-musl": "9cecf6ea2effbe183faebcf7e1160425a4ee17a68e49f2eefe5e1c59c51fa7ee", }, "strip_prefix": "python", }, @@ -470,18 +470,18 @@ TOOL_VERSIONS = { "strip_prefix": "python", }, "3.11.13": { - "url": "20250708/cpython-{python_version}+20250708-{platform}-{build}.tar.gz", + "url": "20250808/cpython-{python_version}+20250808-{platform}-{build}.tar.gz", "sha256": { - "aarch64-apple-darwin": "baec549f2f9367993731d15f9bbed81394c381f8d66bacdee7d448e3a8adaa3b", - "aarch64-unknown-linux-gnu": "b0c5cc99ec81301c24872ff3f180d8e6828a7c2bde3ea5e7b06f71cbb4833293", - "ppc64le-unknown-linux-gnu": "34c9754e6a383ecc36e73ade5374bbc62ade75029efd0aa4651af5bc555984a0", - "riscv64-unknown-linux-gnu": "52e6d43ebfccf5fe7be3b819dc3193941116b1360e74cd3a3a8c568ce5d165c2", - "s390x-unknown-linux-gnu": "f309f3d994465f86d38b383b2d28e9c3e1eb09cffa9b4ca598eee68fd4bc7bbb", - "x86_64-apple-darwin": "34c386610791305b04f4f6bc13396453cbf95b9df7d12aaa03e81f5f86ae6e37", - "x86_64-pc-windows-msvc": "551ca09ea10e3e98fadc1ba63a4c486527d11eabc7345956238a3b4998e8a840", - "aarch64-pc-windows-msvc": "e1f0e3eeb2566d5ec7b234f4ecb46a739d17d0bb73cdd72b37cc06bd21f0d555", - "x86_64-unknown-linux-gnu": "a90c03e8d8128058d6680fa3edee4afb8c4ee3a863455d367b3f70a300c1b862", - "x86_64-unknown-linux-musl": "6f73c6887f1f308ee4088ccd86453df69c2c7bbef1f5c619764a0efc492b75e3", + "aarch64-apple-darwin": "d089bfd2c7b98a0942750a195e70d3172beda76d7747097b8afd87028b6e59b6", + "aarch64-unknown-linux-gnu": "bc57105f8a16acd57b71d926143c7f6ecf61729b40c8b4656f1b98bebd47c710", + "ppc64le-unknown-linux-gnu": "16a0165b0744940702b8fff80b8bf973ac914f78cb6fca28d389583f675e84de", + "riscv64-unknown-linux-gnu": "d8e62306be8f41c46bcd62ca68f91a1467f47adff632a35ff413dc1043ed56e8", + "s390x-unknown-linux-gnu": "4e302a4514a73baefdd9b327062bdafeb4115a799deec91c185f6ab45a857241", + "x86_64-apple-darwin": "d946d618f8bba8308b67e460a30612a71e2ccc309f85f6628aaae24e2b816981", + "x86_64-pc-windows-msvc": "ed963aee33d29ad8abfbb5fe63e42f57a2638a4a11a88e11d8bb66e61f20a6e5", + "aarch64-pc-windows-msvc": "a632857c966237e7fd38b44c47c350f6e30d8ec54dcad6c832865ad670f0f22f", + "x86_64-unknown-linux-gnu": "3ad988c702cbb017fef1208d47dea4138a2e85fd0f7f01ec5e1e335e597131b9", + "x86_64-unknown-linux-musl": "3a5810f0696f844289aa06d5c3a1efeab66eee999c25196b7d1954192a2c2100", }, "strip_prefix": "python", }, @@ -594,18 +594,18 @@ TOOL_VERSIONS = { "strip_prefix": "python", }, "3.12.11": { - "url": "20250708/cpython-{python_version}+20250708-{platform}-{build}.tar.gz", + "url": "20250808/cpython-{python_version}+20250808-{platform}-{build}.tar.gz", "sha256": { - "aarch64-apple-darwin": "d1e426dd70d4cef0344c838e84924b6901bdb25e06d8b5235ce94fe6d5e9f798", - "aarch64-unknown-linux-gnu": "415105aee82617f1ecf88d1f594eb5209f34109d90aeae860bc36f3a05a97dcd", - "aarch64-pc-windows-msvc": "83c655cb0b9805bbfd6062535329440e9635ff45b9f2d584df9de99635aaa6ed", - "ppc64le-unknown-linux-gnu": "b4dd82d30e9357a355f1e9d7960e2714d7b6c6eb95d5cabcd5afc33abb6ed0df", - "riscv64-unknown-linux-gnu": "3d220cdfa2fda11223b7c9f4f0d03a2b0d6f5d752544d08766d3450579cda490", - "s390x-unknown-linux-gnu": "5def9e4c9b00560d38120584b8878f30722ba50d7e26ca7b339ee7bea5e87709", - "x86_64-apple-darwin": "e5d587c50fdc7a872a32341fc47c710a0653d5269f7fd5bcf0dbc8d2330d4525", - "x86_64-pc-windows-msvc": "92fded0d45537d707c67904577af32cef16e6d69c94fea1da7b24da8b75629a2", - "x86_64-unknown-linux-gnu": "3b7802c8d99e9b3efd1e97de4155d0391e464b0ebd92233ede114f3b8a93bc7d", - "x86_64-unknown-linux-musl": "cb8f825d30dd6864a179fbd34b1592325bdcc2b211c01f94d7ed5f0fd790c3fb", + "aarch64-apple-darwin": "8792c4a84c364ab975feca0c27d3157a5435b7baab325a346ae56b223893b661", + "aarch64-unknown-linux-gnu": "4d7ba5314fab02130d6538f074961ffbf61310cade9180e59026074f9a8939cb", + "aarch64-pc-windows-msvc": "00bf7d7e8bcf5d1e9c4dfca0247d8e035147777cd57ee9d4c64dedca86b0a464", + "ppc64le-unknown-linux-gnu": "2c862eb40a81549d9c11e6bf5a7f07c3406310b14e6a4d16dcdf1c4763ef7090", + "riscv64-unknown-linux-gnu": "0bb729b95fabd49c7b495f7c44a9086e3970ea57daf66365741574bd36a17e81", + "s390x-unknown-linux-gnu": "99e465882d217d24ac90e99fac8f32e6a644d0340ac05ee510fb5cdf53f0cfb8", + "x86_64-apple-darwin": "e0c932709dafb05f00e528a7560ef8ee559ac82b75faca60dd1245bca1c1553f", + "x86_64-pc-windows-msvc": "81214ef71964a40ec269a79067ca490d45298c350583bc3af0e5781451a05c3c", + "x86_64-unknown-linux-gnu": "63d78840bf209af8da8f24e335d910f88387b892ca9187be571d481c071751bb", + "x86_64-unknown-linux-musl": "d633d070780590aa03ac5575cd9d7b9e17682d80f14b400313c009c387cf706b", }, "strip_prefix": "python", }, @@ -765,28 +765,28 @@ TOOL_VERSIONS = { "x86_64-unknown-linux-gnu-freethreaded": "python/install", }, }, - "3.13.5": { - "url": "20250708/cpython-{python_version}+20250708-{platform}-{build}.{ext}", + "3.13.6": { + "url": "20250808/cpython-{python_version}+20250808-{platform}-{build}.{ext}", "sha256": { - "aarch64-apple-darwin": "ac3708b0e11c9377210961ccfa7c9c497564723c2ceec09e1a96b43c4bb12c2c", - "aarch64-unknown-linux-gnu": "2c7ba8fb7311ab724e6176916cd6426b6517ca4d6b40b5e939b9fcefca72f888", - "ppc64le-unknown-linux-gnu": "3ae74c7a74d8d79c022e15bd9796c3b0a627b1a4a6c94f59b9f14b3e1b084c97", - "riscv64-unknown-linux-gnu": "01410a477681839a2c567bd17b6080937303fac3f8cc386650386862d5bc37b6", - "s390x-unknown-linux-gnu": "48c9e779826d25327f5a05b25be49da375538367e44c8a43bca3404c665f3138", - "x86_64-apple-darwin": "d8673b4616d19b75f15499d50a585eeb332ff47fad6387d88546f9b0515d7744", - "x86_64-pc-windows-msvc": "5a9a699c5314b9681d585c05d91bfa2e8cec79225e76abad0f3a8f9c6d7f014e", - "aarch64-pc-windows-msvc": "7510a28230535a1547edef9f15912cbd16574ec814ede20ae19a6d5b2ecb7a26", - "aarch64-pc-windows-msvc-freethreaded": "accb608c75ba9d6487fa3c611e1b8038873675cb058423a23fa7e30fc849cf69", - "x86_64-unknown-linux-gnu": "5b16ef64075d941933acf4e4ada7b0c7d5925ce5a2e053e905b5c148ada1bdfe", - "x86_64-unknown-linux-musl": "79f38f297eb91aca4ef165fa66ae91ca5d53f60db942658a877a71c9d8be5cb5", - "aarch64-apple-darwin-freethreaded": "b7764ec1b41a7018c67c83ce3c98f47b0eeac9c4039f3cd50b5bcde4e86bde96", - "aarch64-unknown-linux-gnu-freethreaded": "ced03b7ba62d2864df87ae86ecc50512fbfed66897602ae6f7aacbfb8d7eab38", - "ppc64le-unknown-linux-gnu-freethreaded": "9c943e130a9893c9f6f375c02b34c0b7e62d186d283fc7950d0ee20d7e2f6821", - "riscv64-unknown-linux-gnu-freethreaded": "8075ed7b5f8c8a7c7c65563d2a1d5c20622a46416fb2e5b8d746592527472ea7", - "s390x-unknown-linux-gnu-freethreaded": "a8dbcbe79f7603d82a3640dfd05f9dbff07264f14a6a9a616d277f19d113222c", - "x86_64-apple-darwin-freethreaded": "f15f0700b64fb3475c4dcc2a41540b47857da0c777544c10eb510f71f552e8ec", - "x86_64-pc-windows-msvc-freethreaded": "75acd65c9a44afae432abfd83db648256ac89122f31e21a59310b0c373b147f1", - "x86_64-unknown-linux-gnu-freethreaded": "e21a8d49749ffd40a439349f62fc59cb9e6424a22b40da0242bb8af6e964ba04", + "aarch64-apple-darwin": "8a1efa6af4e80f08e2c97dda822a3d6c24d6c98e518242f802c6a43ae8401488", + "aarch64-unknown-linux-gnu": "11fa0591ae2211c08a42ae54944260e36ddf88a1d5604ea0c49e2477be4e5388", + "ppc64le-unknown-linux-gnu": "8dcf34ae1a685fe1893b52917ae04f23328edadc4acae28499d43850c2bdd26c", + "riscv64-unknown-linux-gnu": "f8ed75aa6cc2011a046be00b629c3c8295267f34280324feaff34c73e7afce39", + "s390x-unknown-linux-gnu": "7707ee5d19a78bc64ef8a66751ec7f97b64ea06714c7b1b52e8b321c2923ead8", + "x86_64-apple-darwin": "27badce7201321a8363219e438a6205165e5b4884012b1046532203df2ec9379", + "x86_64-pc-windows-msvc": "af5cc733c33b9aa9f1d74c81a59351e9b27215486d8b6cdbc06d97646a58c953", + "aarch64-pc-windows-msvc": "8e1617bd407ec1a874499daab26ae95080d1e0267ae616d34490137a28705827", + "aarch64-pc-windows-msvc-freethreaded": "552cfabcc3b103f4b1c4036d2592d5f0373c9554a2c4d2b6631b04ef7e592067", + "x86_64-unknown-linux-gnu": "f844e8c8b6847628b472f7e97d8893a4e93acd5382a902b465776063668c4d64", + "x86_64-unknown-linux-musl": "70076dea0ff65b3c05aae1a97b4a556bf613cc73db30309e59134f9d318f4f7b", + "aarch64-apple-darwin-freethreaded": "f2143304012e021a603bf1807bf3e4ce163832e43ab9a9829e53cb136497f207", + "aarch64-unknown-linux-gnu-freethreaded": "d84a7d64c284be387386b9f5da273f6d05486eb6bd8f9e86e2575cb59604cb22", + "ppc64le-unknown-linux-gnu-freethreaded": "e76fcaf1bf80a615520dbe7f85ca0bb557fad96d132d836b0ac721e7cc1e2a37", + "riscv64-unknown-linux-gnu-freethreaded": "24e08a39ba4fc77753e61541e52eed39cc871f4a92a80a3c5dd495056bd8eff9", + "s390x-unknown-linux-gnu-freethreaded": "1609b223fd38a4a7a4d20e7173d7d9390fe2258f7dd9a15dc9ef0fa49613735d", + "x86_64-apple-darwin-freethreaded": "4360a1278dd0a96b526d108c8fd23498a9d2028dd7791e510fd51ff5ea3f462a", + "x86_64-pc-windows-msvc-freethreaded": "4e727cdbe4057b16a170f887c0fa4227a825ac59bcda84ae946c77cc932af78c", + "x86_64-unknown-linux-gnu-freethreaded": "e48c13c59cc3c01b79f63c8bccec27d2db6e97f64213b8731e2077b6ed8ed52c", }, "strip_prefix": { "aarch64-apple-darwin": "python", @@ -810,28 +810,28 @@ TOOL_VERSIONS = { "x86_64-unknown-linux-gnu-freethreaded": "python/install", }, }, - "3.14.0b4": { - "url": "20250708/cpython-{python_version}+20250708-{platform}-{build}.{ext}", + "3.14.0rc1": { + "url": "20250808/cpython-{python_version}+20250808-{platform}-{build}.{ext}", "sha256": { - "aarch64-apple-darwin": "fe6b2f1f2a7423d277d2ac247d8273fa52f82465a86d37835edfdd540835b2c9", - "aarch64-unknown-linux-gnu": "0320067c5d6bcb3fe7d5dee966021a680e7d8ffaa51300e25825b6d431fd7796", - "ppc64le-unknown-linux-gnu": "4b6cb9b78299f30aa07c62cb081a016df7ac2f77bbee00959bfa1d1a073c7728", - "riscv64-unknown-linux-gnu": "6974cbba97be68fbf05735692950203292997308a2595a167f34a168e5dfbd4a", - "s390x-unknown-linux-gnu": "51fd4370e40af33e891dd221a82e249aed7b080ef48948933cc9252b423f7c3d", - "x86_64-apple-darwin": "a5567f7efde6d70a7be518991e0968683cef64672778015b2203dca96e3e8d17", - "x86_64-pc-windows-msvc": "909664ce85ce6c3d5deeb8451242458e7c53d6c3a604c098386036c20d56f8c7", - "aarch64-pc-windows-msvc": "21017616e457d164b7262c0bf39794d5726c666b9482b152b664ae772bb8e9c6", - "x86_64-unknown-linux-gnu": "f029f9fa03cf1a2147dd03c043da033373300a3c6c38a97661641d2b45e18368", - "x86_64-unknown-linux-musl": "61af21a536f32b0bb88d5983262a8101498f7d573142db93abe2def013f17634", - "aarch64-apple-darwin-freethreaded": "f4a28e1d77003d6cd955f2a436a244ec03bb64f142a9afc79246634d3dec5da3", - "aarch64-unknown-linux-gnu-freethreaded": "2a92a108a3fbd5c439408fe9f3b62bf569ef06dbc2b5b657de301f14a537231a", - "ppc64le-unknown-linux-gnu-freethreaded": "5823a07c957162d6d675488d5306ac3f35a3f458e946cd74da6d1ac69bc97ce3", - "riscv64-unknown-linux-gnu-freethreaded": "f48843e0f1c13ddeaaf9180bc105475873d924638969bc9256a2ac170faeb933", - "s390x-unknown-linux-gnu-freethreaded": "a1e6f843d533c88e290d1e757d4c7953c4f4ccfb5380fef5405aceab938c6f57", - "x86_64-apple-darwin-freethreaded": "f1ea70b041fa5862124980b7fe34362987243a7ecc34fde881357503e47f32ab", - "x86_64-pc-windows-msvc-freethreaded": "5de7968ba0e344562fcff0f9f7c9454966279f1e274b6e701edee253b4a6b565", - "aarch64-pc-windows-msvc-freethreaded": "d7396bafafc82b7e817f0d16208d0f37a88a97c0a71d91e477cbadc5b9d55f6d", - "x86_64-unknown-linux-gnu-freethreaded": "7f5ab66a563f48f169bdb1d216eed8c4126698583d21fa191ab4d995ca8b5506", + "aarch64-apple-darwin": "016b9eb7c6c41d358a095f52203297812a566376b1e4372571b850f621dc720d", + "aarch64-unknown-linux-gnu": "bfa5cb2f56032f4ed2c105f5b3b59ea1809672cb74b453e4450399517a594137", + "ppc64le-unknown-linux-gnu": "eb8fade967732032be70d5129ed66ad28dfe57e2964550f0f6800dddc1fdcb80", + "riscv64-unknown-linux-gnu": "82313ee3c45ad0dc825044fef161840dd60277003bf5458da24d46206fff1e09", + "s390x-unknown-linux-gnu": "5af30600105c42e920e9709afc8deae07c309cd46959c22dc099a1fb45b73902", + "x86_64-apple-darwin": "a74da55830354eb13c5e1fd12bb9b0b624ed0daeafec48444eda86b21476e4c8", + "x86_64-pc-windows-msvc": "71d7fe086604835e5ebd38b45829c1e7ce38fb8f5399d287d219f930cd6efdc1", + "aarch64-pc-windows-msvc": "631c007e1b90dfc9f22d05d4f4e5ef9f70bfa01b675a2bd8590994369746f852", + "x86_64-unknown-linux-gnu": "644028c49cdd9d082274f7265857bc5b5bb4eea8c3e58187e5b3ebb74de9ad3a", + "x86_64-unknown-linux-musl": "af82c7e3ddaebbfb10967c5dd9751bc2fa7f735806a16ebde7600e519c34e587", + "aarch64-apple-darwin-freethreaded": "d611134bcf090db920d8192ec26e899433cdbae42476fd92ee07cc13b5e32d1f", + "aarch64-unknown-linux-gnu-freethreaded": "9fcbb8947c07421506187b3f605f34e94c68824d3fd362d84cd1dcdf13285ee3", + "ppc64le-unknown-linux-gnu-freethreaded": "8cb6937fb0804ca0d5d867af15b6e7fd05d79d0faa5a6e617e6f6280580a5f66", + "riscv64-unknown-linux-gnu-freethreaded": "7a235a6f5b814f5ae789fea65b097ad52f840619dfd054135d8ae8443fa8e362", + "s390x-unknown-linux-gnu-freethreaded": "fee35437df67782b348d57b32cb1acac61384504af28860d552b0d6aeb3ae19e", + "x86_64-apple-darwin-freethreaded": "07deb66e52c91c69e15a3d644ff1527dbd9e278c389ed58341af10872dc15ab5", + "x86_64-pc-windows-msvc-freethreaded": "7a4cdb4c213a2f486b5d6b1044970f6529b7e5365a2d5503ffa8224e62da9ccf", + "aarch64-pc-windows-msvc-freethreaded": "aa9f871afc67419e867535eb0d368e5ec7828138799985d21448443ff1185087", + "x86_64-unknown-linux-gnu-freethreaded": "b6237adf6cf3b8ae00238936d61045d33a6b45147e6540a91ae6e6696aeff23c", }, "strip_prefix": { "aarch64-apple-darwin": "python", @@ -864,8 +864,8 @@ MINOR_MAPPING = { "3.10": "3.10.18", "3.11": "3.11.13", "3.12": "3.12.11", - "3.13": "3.13.5", - "3.14": "3.14.0b4", + "3.13": "3.13.6", + "3.14": "3.14.0rc1", } def _generate_platforms(): diff --git a/tests/python/python_tests.bzl b/tests/python/python_tests.bzl index 136f90c519..9081a0e306 100644 --- a/tests/python/python_tests.bzl +++ b/tests/python/python_tests.bzl @@ -325,8 +325,8 @@ def _test_toolchain_ordering(env): "3.10": "3.10.18", "3.11": "3.11.13", "3.12": "3.12.11", - "3.13": "3.13.5", - "3.14": "3.14.0b4", + "3.13": "3.13.6", + "3.14": "3.14.0rc1", "3.8": "3.8.20", "3.9": "3.9.23", }) diff --git a/tests/toolchains/python_toolchain_test.py b/tests/toolchains/python_toolchain_test.py index 63ed42488f..ff45fc0863 100644 --- a/tests/toolchains/python_toolchain_test.py +++ b/tests/toolchains/python_toolchain_test.py @@ -27,18 +27,14 @@ def test_expected_toolchain_matches(self): ) self.assertIn(expected, settings["toolchain_label"], msg) - if sys.version_info.releaselevel == "final": - actual = "{v.major}.{v.minor}.{v.micro}".format(v=sys.version_info) - elif sys.version_info.releaselevel in ["beta"]: - actual = ( - "{v.major}.{v.minor}.{v.micro}{v.releaselevel[0]}{v.serial}".format( - v=sys.version_info - ) - ) - else: - raise NotImplementedError( - "Unsupported release level, please update the test" + actual = "{v.major}.{v.minor}.{v.micro}".format(v=sys.version_info) + if sys.version_info.releaselevel != "final": + release_prefix = ( + "rc" + if sys.version_info.releaselevel == "candidate" + else sys.version_info.releaselevel[0] ) + actual = f"{actual}{release_prefix}{sys.version_info.serial}" self.assertEqual(actual, expect_version) diff --git a/tests/toolchains/transitions/transitions_tests.bzl b/tests/toolchains/transitions/transitions_tests.bzl index ef071188bb..0f1db2eecd 100644 --- a/tests/toolchains/transitions/transitions_tests.bzl +++ b/tests/toolchains/transitions/transitions_tests.bzl @@ -64,7 +64,7 @@ def _impl(ctx): if got_version.releaselevel != "final": got = "{}{}{}".format( got, - got_version.releaselevel[0], + "rc" if got_version.releaselevel == "candidate" else got_version.releaselevel[0], got_version.serial, ) From 537fe302fd84a68d67976ff715b85662172c8342 Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Sun, 10 Aug 2025 23:18:30 +0900 Subject: [PATCH 362/922] build(deps): bump snowballstemmer from 2.2.0 to 3.0.1 in /docs (#3144) Bumps [snowballstemmer](https://github.com/snowballstem/snowball) from 2.2.0 to 3.0.1.
Changelog

Sourced from snowballstemmer's changelog.

Snowball 3.0.1 (2025-05-09)

Python

  • The init.py in 3.0.0 was incorrectly generated due to a missing build dependency and the list of algorithms was empty. First reported by laymonage. Thanks to Dmitry Shachnev, Henry Schreiner and Adam Turner for diagnosing and fixing. (#229, #230, #231)

  • Add trove classifiers for Armenian and Yiddish which have now been registered with PyPI. Thanks to Henry Schreiner and Dmitry Shachnev. (#228)

  • Update documented details of Python 2 support in old versions.

Snowball 3.0.0 (2025-05-08)

Ada

  • Bug fixes:

    • Fix invalid Ada code generated for Snowball loop (it was partly Pascal!) None of the stemmers shipped in previous releases triggered this bug, but the Turkish stemmer now does.

    • The Ada runtime was not tracking the current length of the string but instead used the current limit value or some other substitute, which manifested as various incorrect behaviours for code inside of setlimit.

    • size was incorrectly returning the difference between the limit and the backwards limit.

    • lenof or sizeof on a string variable generated Ada code that didn't even compile.

    • Fix incorrect preconditions on some methods in the runtime.

    • Fix bug in runtime code used by attach, insert, <- and string variable assignment when a (sub)string was replaced with a larger string. This bug was triggered by code in the Kraaij-Pohlmann Dutch stemmer implementation (which was previously not enabled by default but is now the standard Dutch stemmer).

    • Fix invalid code generated for insert, <- and string variable assignment. This bug was triggered by code in the Kraaij-Pohlmann Dutch stemmer implementation (which was previously not enabled by default but is now the standard Dutch stemmer).

... (truncated)

Commits
  • e4b3efb Update for 3.0.1
  • bbd3319 Protect empty languages dict
  • 298ff9f Update details of Python 2 support in old versions
  • 53fe098 python: Specify correct dependencies for $(python_output_dir)/__init__.py
  • 00a22de Stop excluding classifiers for Armenian and Yiddish
  • abd9adc Update for 3.0.0
  • d23d356 Back out incomplete ESM support for 3.0.0
  • ff42274 Update draft NEWS entry
  • cd61f01 tamil: remove_tense_suffix signals if ending removed
  • edfe576 nepali: Reformat amongs to be clearer
  • Additional commits viewable in compare view

[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=snowballstemmer&package-manager=pip&previous-version=2.2.0&new-version=3.0.1)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) ---
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot show ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Ignas Anikevicius <240938+aignas@users.noreply.github.com> --- docs/requirements.txt | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/requirements.txt b/docs/requirements.txt index f0abac5c30..5ef561f06e 100644 --- a/docs/requirements.txt +++ b/docs/requirements.txt @@ -297,9 +297,9 @@ requests==2.32.4 \ # via # readthedocs-sphinx-ext # sphinx -snowballstemmer==2.2.0 \ - --hash=sha256:09b16deb8547d3412ad7b590689584cd0fe25ec8db3be37788be3810cbf19cb1 \ - --hash=sha256:c8e1716e83cc398ae16824e5572ae04e0d9fc2c6b985fb0f900f5f0c96ecba1a +snowballstemmer==3.0.1 \ + --hash=sha256:6cd7b3897da8d6c9ffb968a6781fa6532dce9c3618a4b127d920dab764a19064 \ + --hash=sha256:6d5eeeec8e9f84d4d56b847692bacf79bc2c8e90c7f80ca4444ff8b6f2e52895 # via sphinx sphinx==8.1.3 \ --hash=sha256:09719015511837b76bf6e03e42eb7595ac8c2e41eeb9c29c5b755c6b677992a2 \ From acf75079fa5d7837ca4f3e45ff57b3b07adf4919 Mon Sep 17 00:00:00 2001 From: Ignas Anikevicius <240938+aignas@users.noreply.github.com> Date: Mon, 11 Aug 2025 12:12:42 +0900 Subject: [PATCH 363/922] fix(core): do not assume rules_python runtime (#3134) This change reverts the behaviour where we assume that particular attributes will be always present - if bazel is doing autoloading for WORKSPACE builds (7.6.1), then we will crash with attribute error. I could not think how to add a unit test, which would test this fix because it seems to only happen with a released version of rules_python where we are not using `local_repository` override. Fixes #3119 --------- Co-authored-by: Richard Levasseur --- CHANGELOG.md | 2 ++ python/private/py_executable.bzl | 8 ++++++-- 2 files changed, 8 insertions(+), 2 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 04979e8c41..b235989f6a 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -109,6 +109,8 @@ END_UNRELEASED_TEMPLATE * (toolchains) use "command -v" to find interpreter in `$PATH` ([#3150](https://github.com/bazel-contrib/rules_python/pull/3150)). * (pypi) `bazel vendor` now works in `bzlmod` ({gh-issue}`3079`). +* (core) builds work again on `7.x` `WORKSPACE` configurations + ([#3119](https://github.com/bazel-contrib/rules_python/issues/3119)). {#v0-0-0-added} ### Added diff --git a/python/private/py_executable.bzl b/python/private/py_executable.bzl index 7e50247e61..9927975aa8 100644 --- a/python/private/py_executable.bzl +++ b/python/private/py_executable.bzl @@ -796,6 +796,7 @@ def _create_stage1_bootstrap( is_for_zip, runtime_details, venv = None): + """Create a legacy bootstrap script that is written in Python.""" runtime = runtime_details.effective_runtime if venv: @@ -805,8 +806,11 @@ def _create_stage1_bootstrap( python_binary_actual = venv.interpreter_actual_path if venv else "" - # Runtime may be None on Windows due to the --python_path flag. - if runtime and runtime.supports_build_time_venv: + # Guard against the following: + # * Runtime may be None on Windows due to the --python_path flag. + # * Runtime may not have 'supports_build_time_venv' if a really old version is autoloaded + # on bazel 7.6.x. + if runtime and getattr(runtime, "supports_build_time_venv", False): resolve_python_binary_at_runtime = "0" else: resolve_python_binary_at_runtime = "1" From 673cd7608b706e91bf8b44f1786d9eb59a8b963a Mon Sep 17 00:00:00 2001 From: Ignas Anikevicius <240938+aignas@users.noreply.github.com> Date: Mon, 11 Aug 2025 12:26:29 +0900 Subject: [PATCH 364/922] chore(deps): upgrade bazel-skylib to 1.8.1 (#3118) With most recent bazel versions, older versions of rules_python started spewing a lot of warnings due to us using `bazel-skylib` for copying files around. The only solution is to bump the bazel-skylib version. Fixes #3113 --------- Co-authored-by: Richard Levasseur --- CHANGELOG.md | 3 +++ MODULE.bazel | 2 +- 2 files changed, 4 insertions(+), 1 deletion(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index b235989f6a..abeb174bf3 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -54,6 +54,9 @@ END_UNRELEASED_TEMPLATE {#v0-0-0-changed} ### Changed +* (deps) (bzlmod) Upgraded to `bazel-skylib` version + [1.8.1](https://github.com/bazelbuild/bazel-skylib/releases/tag/1.8.1) + to remove deprecation warnings. * (gazelle) For package mode, resolve dependencies when imports are relative to the package path. This is enabled via the `# gazelle:python_experimental_allow_relative_imports` true directive ({gh-issue}`2203`). diff --git a/MODULE.bazel b/MODULE.bazel index 9db287dc28..b8d8c16a0c 100644 --- a/MODULE.bazel +++ b/MODULE.bazel @@ -5,7 +5,7 @@ module( ) bazel_dep(name = "bazel_features", version = "1.21.0") -bazel_dep(name = "bazel_skylib", version = "1.7.1") +bazel_dep(name = "bazel_skylib", version = "1.8.1") bazel_dep(name = "rules_cc", version = "0.0.16") bazel_dep(name = "platforms", version = "0.0.11") From 5c09732b758482dcebdb8b78bc51c558b00f35af Mon Sep 17 00:00:00 2001 From: Ignas Anikevicius <240938+aignas@users.noreply.github.com> Date: Mon, 11 Aug 2025 12:32:47 +0900 Subject: [PATCH 365/922] fix(pypi): reuse select dicts for constructing the env (#3108) Before this PR we would be constructing slightly different environments when the `env_marker_setting` is doing it in the analysis phase and when we are doing it in the repo phase due to how the defaults are handled. In this change we simply reuse the same select statements and add an extra helper that is allowing us to process that. Work towards #2949 Prep for #3058 Co-authored-by: Richard Levasseur --- python/private/pypi/BUILD.bazel | 8 +-- python/private/pypi/extension.bzl | 7 +- python/private/pypi/pep508_env.bzl | 86 +++++++++++++++---------- python/private/pypi/pep508_platform.bzl | 57 ---------------- tests/pypi/pep508/BUILD.bazel | 5 ++ tests/pypi/pep508/env_tests.bzl | 69 ++++++++++++++++++++ tests/pypi/pep508/evaluate_tests.bzl | 22 +++++-- 7 files changed, 150 insertions(+), 104 deletions(-) delete mode 100644 python/private/pypi/pep508_platform.bzl create mode 100644 tests/pypi/pep508/env_tests.bzl diff --git a/python/private/pypi/BUILD.bazel b/python/private/pypi/BUILD.bazel index b098f29e94..e5a916be64 100644 --- a/python/private/pypi/BUILD.bazel +++ b/python/private/pypi/BUILD.bazel @@ -252,6 +252,9 @@ bzl_library( bzl_library( name = "pep508_env_bzl", srcs = ["pep508_env.bzl"], + deps = [ + "//python/private:version_bzl", + ], ) bzl_library( @@ -263,11 +266,6 @@ bzl_library( ], ) -bzl_library( - name = "pep508_platform_bzl", - srcs = ["pep508_platform.bzl"], -) - bzl_library( name = "pep508_requirement_bzl", srcs = ["pep508_requirement.bzl"], diff --git a/python/private/pypi/extension.bzl b/python/private/pypi/extension.bzl index 096256e4be..08e1af4d81 100644 --- a/python/private/pypi/extension.bzl +++ b/python/private/pypi/extension.bzl @@ -76,11 +76,12 @@ def _platforms(*, python_version, minor_mapping, config): for platform, values in config.platforms.items(): key = "{}_{}".format(abi, platform) - platforms[key] = env(struct( - abi = abi, + platforms[key] = env( + env = values.env, os = values.os_name, arch = values.arch_name, - )) | values.env + python_version = python_version, + ) return platforms def _create_whl_repos( diff --git a/python/private/pypi/pep508_env.bzl b/python/private/pypi/pep508_env.bzl index c2d404bc3e..5031ebae12 100644 --- a/python/private/pypi/pep508_env.bzl +++ b/python/private/pypi/pep508_env.bzl @@ -15,6 +15,20 @@ """This module is for implementing PEP508 environment definition. """ +load("//python/private:version.bzl", "version") + +_DEFAULT = "//conditions:default" + +# Here we store the aliases in the platform so that the users can specify any valid target in +# there. +_cpu_aliases = { + "arm": "aarch32", + "arm64": "aarch64", +} +_os_aliases = { + "macos": "osx", +} + # See https://stackoverflow.com/a/45125525 platform_machine_aliases = { # These pairs mean the same hardware, but different values may be used @@ -59,7 +73,7 @@ platform_machine_select_map = { "@platforms//cpu:x86_64": "x86_64", # The value is empty string if it cannot be determined: # https://docs.python.org/3/library/platform.html#platform.machine - "//conditions:default": "", + _DEFAULT: "", } # Platform system returns results from the `uname` call. @@ -73,7 +87,7 @@ _platform_system_values = { "linux": "Linux", "netbsd": "NetBSD", "openbsd": "OpenBSD", - "osx": "Darwin", + "osx": "Darwin", # NOTE: macos is an alias to osx, we handle it through _os_aliases "windows": "Windows", } @@ -83,7 +97,7 @@ platform_system_select_map = { } | { # The value is empty string if it cannot be determined: # https://docs.python.org/3/library/platform.html#platform.machine - "//conditions:default": "", + _DEFAULT: "", } # The copy of SO [answer](https://stackoverflow.com/a/13874620) containing @@ -123,18 +137,19 @@ _sys_platform_values = { "ios": "ios", "linux": "linux", "openbsd": "openbsd", - "osx": "darwin", + "osx": "darwin", # NOTE: macos is an alias to osx, we handle it through _os_aliases "wasi": "wasi", "windows": "win32", } sys_platform_select_map = { + # These values are decided by the sys.platform docs. "@platforms//os:{}".format(bazel_os): py_platform for bazel_os, py_platform in _sys_platform_values.items() } | { # For lack of a better option, use empty string. No standard doc/spec # about sys_platform value. - "//conditions:default": "", + _DEFAULT: "", } # The "java" value is documented, but with Jython defunct, @@ -142,53 +157,58 @@ sys_platform_select_map = { # The os.name value is technically a property of the runtime, not the # targetted runtime OS, but the distinction shouldn't matter if # things are properly configured. -_os_name_values = { - "linux": "posix", - "osx": "posix", - "windows": "nt", -} - os_name_select_map = { - "@platforms//os:{}".format(bazel_os): py_os - for bazel_os, py_os in _os_name_values.items() -} | { - "//conditions:default": "posix", + "@platforms//os:windows": "nt", + _DEFAULT: "posix", } -def env(target_platform, *, extra = None): +def _set_default(env, env_key, m, key): + """Set the default value in the env if it is not already set.""" + default = m.get(key, m[_DEFAULT]) + env.setdefault(env_key, default) + +def env(*, env = None, os, arch, python_version = "", extra = None): """Return an env target platform NOTE: This is for use during the loading phase. For the analysis phase, `env_marker_setting()` constructs the env dict. Args: - target_platform: {type}`str` the target platform identifier, e.g. - `cp33_linux_aarch64` + env: {type}`str` the environment. + os: {type}`str` the OS name. + arch: {type}`str` the CPU name. + python_version: {type}`str` the full python version. extra: {type}`str` the extra value to be added into the env. Returns: A dict that can be used as `env` in the marker evaluation. """ - env = create_env() + env = env or {} + env = env | create_env() if extra != None: env["extra"] = extra - if target_platform.abi: - minor_version, _, micro_version = target_platform.abi[3:].partition(".") - micro_version = micro_version or "0" - env = env | { - "implementation_version": "3.{}.{}".format(minor_version, micro_version), - "python_full_version": "3.{}.{}".format(minor_version, micro_version), - "python_version": "3.{}".format(minor_version), - } - if target_platform.os and target_platform.arch: - os = target_platform.os + if python_version: + v = version.parse(python_version) + major = v.release[0] + minor = v.release[1] + micro = v.release[2] if len(v.release) > 2 else 0 env = env | { - "os_name": _os_name_values.get(os, ""), - "platform_machine": target_platform.arch, - "platform_system": _platform_system_values.get(os, ""), - "sys_platform": _sys_platform_values.get(os, ""), + "implementation_version": "{}.{}.{}".format(major, minor, micro), + "python_full_version": "{}.{}.{}".format(major, minor, micro), + "python_version": "{}.{}".format(major, minor), } + + if os: + os = "@platforms//os:{}".format(_os_aliases.get(os, os)) + _set_default(env, "os_name", os_name_select_map, os) + _set_default(env, "platform_system", platform_system_select_map, os) + _set_default(env, "sys_platform", sys_platform_select_map, os) + + if arch: + arch = "@platforms//cpu:{}".format(_cpu_aliases.get(arch, arch)) + _set_default(env, "platform_machine", platform_machine_select_map, arch) + set_missing_env_defaults(env) return env diff --git a/python/private/pypi/pep508_platform.bzl b/python/private/pypi/pep508_platform.bzl deleted file mode 100644 index 381a8d7a08..0000000000 --- a/python/private/pypi/pep508_platform.bzl +++ /dev/null @@ -1,57 +0,0 @@ -# Copyright 2025 The Bazel Authors. All rights reserved. -# -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -"""The platform abstraction -""" - -def platform(*, abi = None, os = None, arch = None): - """platform returns a struct for the platform. - - Args: - abi: {type}`str | None` the target ABI, e.g. `"cp39"`. - os: {type}`str | None` the target os, e.g. `"linux"`. - arch: {type}`str | None` the target CPU, e.g. `"aarch64"`. - - Returns: - A struct. - """ - - # Note, this is used a lot as a key in dictionaries, so it cannot contain - # methods. - return struct( - abi = abi, - os = os, - arch = arch, - ) - -def platform_from_str(p, python_version): - """Return a platform from a string. - - Args: - p: {type}`str` the actual string. - python_version: {type}`str` the python version to add to platform if needed. - - Returns: - A struct that is returned by the `_platform` function. - """ - if p.startswith("cp"): - abi, _, p = p.partition("_") - elif python_version: - major, _, tail = python_version.partition(".") - abi = "cp{}{}".format(major, tail) - else: - abi = None - - os, _, arch = p.partition("_") - return platform(abi = abi, os = os or None, arch = arch or None) diff --git a/tests/pypi/pep508/BUILD.bazel b/tests/pypi/pep508/BUILD.bazel index 7eab2e096a..36fce0fa89 100644 --- a/tests/pypi/pep508/BUILD.bazel +++ b/tests/pypi/pep508/BUILD.bazel @@ -1,4 +1,5 @@ load(":deps_tests.bzl", "deps_test_suite") +load(":env_tests.bzl", "env_test_suite") load(":evaluate_tests.bzl", "evaluate_test_suite") load(":requirement_tests.bzl", "requirement_test_suite") @@ -6,6 +7,10 @@ deps_test_suite( name = "deps_tests", ) +env_test_suite( + name = "env_tests", +) + evaluate_test_suite( name = "evaluate_tests", ) diff --git a/tests/pypi/pep508/env_tests.bzl b/tests/pypi/pep508/env_tests.bzl new file mode 100644 index 0000000000..cfd94a1b01 --- /dev/null +++ b/tests/pypi/pep508/env_tests.bzl @@ -0,0 +1,69 @@ +"""Tests to check for env construction.""" + +load("@rules_testing//lib:test_suite.bzl", "test_suite") +load("//python/private/pypi:pep508_env.bzl", pep508_env = "env") # buildifier: disable=bzl-visibility + +_tests = [] + +def _test_env_defaults(env): + got = pep508_env(os = "exotic", arch = "exotic", python_version = "3.1.1") + got.pop("_aliases") + env.expect.that_dict(got).contains_exactly({ + "implementation_name": "cpython", + "implementation_version": "3.1.1", + "os_name": "posix", + "platform_machine": "", + "platform_python_implementation": "CPython", + "platform_release": "", + "platform_system": "", + "platform_version": "0", + "python_full_version": "3.1.1", + "python_version": "3.1", + "sys_platform": "", + }) + +_tests.append(_test_env_defaults) + +def _test_env_freebsd(env): + got = pep508_env(os = "freebsd", arch = "arm64", python_version = "3.1.1") + got.pop("_aliases") + env.expect.that_dict(got).contains_exactly({ + "implementation_name": "cpython", + "implementation_version": "3.1.1", + "os_name": "posix", + "platform_machine": "aarch64", + "platform_python_implementation": "CPython", + "platform_release": "", + "platform_system": "FreeBSD", + "platform_version": "0", + "python_full_version": "3.1.1", + "python_version": "3.1", + "sys_platform": "freebsd", + }) + +_tests.append(_test_env_freebsd) + +def _test_env_macos(env): + got = pep508_env(os = "macos", arch = "arm64", python_version = "3.1.1") + got.pop("_aliases") + env.expect.that_dict(got).contains_exactly({ + "implementation_name": "cpython", + "implementation_version": "3.1.1", + "os_name": "posix", + "platform_machine": "aarch64", + "platform_python_implementation": "CPython", + "platform_release": "", + "platform_system": "Darwin", + "platform_version": "0", + "python_full_version": "3.1.1", + "python_version": "3.1", + "sys_platform": "darwin", + }) + +_tests.append(_test_env_macos) + +def env_test_suite(name): # buildifier: disable=function-docstring + test_suite( + name = name, + basic_tests = _tests, + ) diff --git a/tests/pypi/pep508/evaluate_tests.bzl b/tests/pypi/pep508/evaluate_tests.bzl index cc867f346c..7843f88e89 100644 --- a/tests/pypi/pep508/evaluate_tests.bzl +++ b/tests/pypi/pep508/evaluate_tests.bzl @@ -16,7 +16,6 @@ load("@rules_testing//lib:test_suite.bzl", "test_suite") load("//python/private/pypi:pep508_env.bzl", pep508_env = "env") # buildifier: disable=bzl-visibility load("//python/private/pypi:pep508_evaluate.bzl", "evaluate", "tokenize") # buildifier: disable=bzl-visibility -load("//python/private/pypi:pep508_platform.bzl", "platform_from_str") # buildifier: disable=bzl-visibility _tests = [] @@ -244,26 +243,37 @@ _tests.append(_evaluate_partial_only_extra) def _evaluate_with_aliases(env): # When - for target_platform, tests in { + for (os, cpu), tests in { # buildifier: @unsorted-dict-items - "osx_aarch64": { + ("osx", "aarch64"): { "platform_system == 'Darwin' and platform_machine == 'arm64'": True, "platform_system == 'Darwin' and platform_machine == 'aarch64'": True, "platform_system == 'Darwin' and platform_machine == 'amd64'": False, }, - "osx_x86_64": { + ("osx", "x86_64"): { "platform_system == 'Darwin' and platform_machine == 'amd64'": True, "platform_system == 'Darwin' and platform_machine == 'x86_64'": True, }, - "osx_x86_32": { + ("osx", "x86_32"): { "platform_system == 'Darwin' and platform_machine == 'i386'": True, "platform_system == 'Darwin' and platform_machine == 'i686'": True, "platform_system == 'Darwin' and platform_machine == 'x86_32'": True, "platform_system == 'Darwin' and platform_machine == 'x86_64'": False, }, + ("freebsd", "x86_32"): { + "platform_system == 'FreeBSD' and platform_machine == 'i386'": True, + "platform_system == 'FreeBSD' and platform_machine == 'i686'": True, + "platform_system == 'FreeBSD' and platform_machine == 'x86_32'": True, + "platform_system == 'FreeBSD' and platform_machine == 'x86_64'": False, + "platform_system == 'FreeBSD' and os_name == 'posix'": True, + }, }.items(): # buildifier: @unsorted-dict-items for input, want in tests.items(): - _check_evaluate(env, input, want, pep508_env(platform_from_str(target_platform, ""))) + _check_evaluate(env, input, want, pep508_env( + os = os, + arch = cpu, + python_version = "3.2", + )) _tests.append(_evaluate_with_aliases) From f6dd386697fa1302dcb089c46c565a05146c0b68 Mon Sep 17 00:00:00 2001 From: Ignas Anikevicius <240938+aignas@users.noreply.github.com> Date: Mon, 11 Aug 2025 15:33:52 +0900 Subject: [PATCH 366/922] fix(pypi): support properly installing sdists via pypi without index (#3115) This fixes the subtle bug introduced in #2871, where we were dropping the URL from the requirement, because we can download the sdist directly. We cannot add --no-index because sdists in general may require extra build dependencies and we had already issues previously (see 0.36 release notes). Fixes #2363 Fixes #3131 --------- Co-authored-by: Richard Levasseur --- CHANGELOG.md | 2 ++ python/private/pypi/index_sources.bzl | 6 +++--- tests/pypi/extension/extension_tests.bzl | 2 +- .../index_sources/index_sources_tests.bzl | 5 ++++- .../parse_requirements_tests.bzl | 20 ++++++++++++++++--- 5 files changed, 27 insertions(+), 8 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index abeb174bf3..54eccb1b53 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -112,6 +112,8 @@ END_UNRELEASED_TEMPLATE * (toolchains) use "command -v" to find interpreter in `$PATH` ([#3150](https://github.com/bazel-contrib/rules_python/pull/3150)). * (pypi) `bazel vendor` now works in `bzlmod` ({gh-issue}`3079`). +* (pypi) Correctly pull `sdist` distributions using `pip` + ([#3131](https://github.com/bazel-contrib/rules_python/pull/3131)). * (core) builds work again on `7.x` `WORKSPACE` configurations ([#3119](https://github.com/bazel-contrib/rules_python/issues/3119)). diff --git a/python/private/pypi/index_sources.bzl b/python/private/pypi/index_sources.bzl index 803670c3e4..1998e4fb33 100644 --- a/python/private/pypi/index_sources.bzl +++ b/python/private/pypi/index_sources.bzl @@ -93,12 +93,12 @@ def index_sources(line): is_known_ext = True break - if is_known_ext: + requirement = requirement_line + if filename.endswith(".whl"): requirement = maybe_requirement.strip() - else: + elif not is_known_ext: # could not detect filename from the URL filename = "" - requirement = requirement_line return struct( requirement = requirement, diff --git a/tests/pypi/extension/extension_tests.bzl b/tests/pypi/extension/extension_tests.bzl index 52e0e29cb0..4949c0df85 100644 --- a/tests/pypi/extension/extension_tests.bzl +++ b/tests/pypi/extension/extension_tests.bzl @@ -934,7 +934,7 @@ git_dep @ git+https://git.server/repo/project@deadbeefdeadbeef "extra_pip_args": ["--extra-args-for-sdist-building"], "filename": "any-name.tar.gz", "python_interpreter_target": "unit_test_interpreter_target", - "requirement": "direct_sdist_without_sha", + "requirement": "direct_sdist_without_sha @ some-archive/any-name.tar.gz", "sha256": "", "urls": ["some-archive/any-name.tar.gz"], }, diff --git a/tests/pypi/index_sources/index_sources_tests.bzl b/tests/pypi/index_sources/index_sources_tests.bzl index d4062b47fe..7aa22d164a 100644 --- a/tests/pypi/index_sources/index_sources_tests.bzl +++ b/tests/pypi/index_sources/index_sources_tests.bzl @@ -73,7 +73,10 @@ def _test_no_simple_api_sources(env): filename = "package.whl", ), "foo[extra] @ https://example.org/foo-1.0.tar.gz --hash=sha256:deadbe0f": struct( - requirement = "foo[extra]", + # NOTE @aignas 2025-08-03: we need to ensure that sdists continue working + # when we are using pip to install them even if the experimental_index_url + # code path is used. + requirement = "foo[extra] @ https://example.org/foo-1.0.tar.gz --hash=sha256:deadbe0f", requirement_line = "foo[extra] @ https://example.org/foo-1.0.tar.gz --hash=sha256:deadbe0f", marker = "", url = "https://example.org/foo-1.0.tar.gz", diff --git a/tests/pypi/parse_requirements/parse_requirements_tests.bzl b/tests/pypi/parse_requirements/parse_requirements_tests.bzl index 82fdd0a051..b14467bc84 100644 --- a/tests/pypi/parse_requirements/parse_requirements_tests.bzl +++ b/tests/pypi/parse_requirements/parse_requirements_tests.bzl @@ -27,6 +27,9 @@ foo==0.0.1 \ """, "requirements_direct": """\ foo[extra] @ https://some-url/package.whl +""", + "requirements_direct_sdist": """ +foo @ https://github.com/org/foo/downloads/foo-1.1.tar.gz """, "requirements_extra_args": """\ --index-url=example.org @@ -131,22 +134,33 @@ def _test_direct_urls_integration(env): ctx = _mock_ctx(), requirements_by_platform = { "requirements_direct": ["linux_x86_64"], + "requirements_direct_sdist": ["osx_x86_64"], }, ) env.expect.that_collection(got).contains_exactly([ struct( name = "foo", is_exposed = True, - is_multiple_versions = False, + is_multiple_versions = True, srcs = [ struct( distribution = "foo", extra_pip_args = [], + filename = "foo-1.1.tar.gz", + requirement_line = "foo @ https://github.com/org/foo/downloads/foo-1.1.tar.gz", + sha256 = "", + target_platforms = ["osx_x86_64"], + url = "https://github.com/org/foo/downloads/foo-1.1.tar.gz", + yanked = False, + ), + struct( + distribution = "foo", + extra_pip_args = [], + filename = "package.whl", requirement_line = "foo[extra]", + sha256 = "", target_platforms = ["linux_x86_64"], url = "https://some-url/package.whl", - filename = "package.whl", - sha256 = "", yanked = False, ), ], From a36d002318407dc16e2be1fa4c80b7aeb8d2dda9 Mon Sep 17 00:00:00 2001 From: Ignas Anikevicius <240938+aignas@users.noreply.github.com> Date: Mon, 11 Aug 2025 15:38:46 +0900 Subject: [PATCH 367/922] feat(pypi): add a standards compliant python_tag creator (#3110) This will be needed when we start selecting wheels entirely in the bzlmod extension evaluation phase (#3058). This adds a few unit tests to just ensure that we conform to the spec even though the code is very simple. Work towards #2747 Work towards #2759 Work towards #2849 --- python/private/pypi/BUILD.bazel | 8 +++++ python/private/pypi/python_tag.bzl | 41 ++++++++++++++++++++++ tests/pypi/python_tag/BUILD.bazel | 3 ++ tests/pypi/python_tag/python_tag_tests.bzl | 34 ++++++++++++++++++ 4 files changed, 86 insertions(+) create mode 100644 python/private/pypi/python_tag.bzl create mode 100644 tests/pypi/python_tag/BUILD.bazel create mode 100644 tests/pypi/python_tag/python_tag_tests.bzl diff --git a/python/private/pypi/BUILD.bazel b/python/private/pypi/BUILD.bazel index e5a916be64..3a66170768 100644 --- a/python/private/pypi/BUILD.bazel +++ b/python/private/pypi/BUILD.bazel @@ -336,6 +336,14 @@ bzl_library( ], ) +bzl_library( + name = "python_tag_bzl", + srcs = ["python_tag.bzl"], + deps = [ + "//python/private:version_bzl", + ], +) + bzl_library( name = "render_pkg_aliases_bzl", srcs = ["render_pkg_aliases.bzl"], diff --git a/python/private/pypi/python_tag.bzl b/python/private/pypi/python_tag.bzl new file mode 100644 index 0000000000..224c5f96f0 --- /dev/null +++ b/python/private/pypi/python_tag.bzl @@ -0,0 +1,41 @@ +"A simple utility function to get the python_tag from the implementation name" + +load("//python/private:version.bzl", "version") + +# Taken from +# https://packaging.python.org/en/latest/specifications/platform-compatibility-tags/#python-tag +_PY_TAGS = { + # "py": Generic Python (does not require implementation-specific features) + "cpython": "cp", + "ironpython": "ip", + "jython": "jy", + "pypy": "pp", + "python": "py", +} +PY_TAG_GENERIC = "py" + +def python_tag(implementation_name, python_version = ""): + """Get the python_tag from the implementation_name. + + Args: + implementation_name: {type}`str` the implementation name, e.g. "cpython" + python_version: {type}`str` a version who can be parsed using PEP440 compliant + parser. + + Returns: + A {type}`str` that represents the python_tag with a version if the + python_version is given. + """ + if python_version: + v = version.parse(python_version, strict = True) + suffix = "{}{}".format( + v.release[0], + v.release[1] if len(v.release) > 1 else "", + ) + else: + suffix = "" + + return "{}{}".format( + _PY_TAGS.get(implementation_name, implementation_name), + suffix, + ) diff --git a/tests/pypi/python_tag/BUILD.bazel b/tests/pypi/python_tag/BUILD.bazel new file mode 100644 index 0000000000..d4b37cea16 --- /dev/null +++ b/tests/pypi/python_tag/BUILD.bazel @@ -0,0 +1,3 @@ +load(":python_tag_tests.bzl", "python_tag_test_suite") + +python_tag_test_suite(name = "python_tag_tests") diff --git a/tests/pypi/python_tag/python_tag_tests.bzl b/tests/pypi/python_tag/python_tag_tests.bzl new file mode 100644 index 0000000000..ca86575e5b --- /dev/null +++ b/tests/pypi/python_tag/python_tag_tests.bzl @@ -0,0 +1,34 @@ +"" + +load("@rules_testing//lib:test_suite.bzl", "test_suite") +load("//python/private/pypi:python_tag.bzl", "python_tag") # buildifier: disable=bzl-visibility + +_tests = [] + +def _test_without_version(env): + for give, expect in { + "cpython": "cp", + "ironpython": "ip", + "jython": "jy", + "pypy": "pp", + "python": "py", + "something_else": "something_else", + }.items(): + got = python_tag(give) + env.expect.that_str(got).equals(expect) + +_tests.append(_test_without_version) + +def _test_with_version(env): + got = python_tag("cpython", "3.1.15") + env.expect.that_str(got).equals("cp31") + +_tests.append(_test_with_version) + +def python_tag_test_suite(name): + """Create the test suite. + + Args: + name: the name of the test suite + """ + test_suite(name = name, basic_tests = _tests) From 119fa6a61ea5a384694c5863530aee4d780c447c Mon Sep 17 00:00:00 2001 From: Ignas Anikevicius <240938+aignas@users.noreply.github.com> Date: Tue, 12 Aug 2025 01:06:33 +0900 Subject: [PATCH 368/922] doc: changelog cherry-picks for 1.5.2 (#3158) Fixes #3135 The cherry-picks in the release/1.5 branch already include the fixes to the changelog, so this PR won't need to be cherry-picked itself. --- CHANGELOG.md | 25 ++++++++++++++++++------- 1 file changed, 18 insertions(+), 7 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 54eccb1b53..55c4659b39 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -54,9 +54,6 @@ END_UNRELEASED_TEMPLATE {#v0-0-0-changed} ### Changed -* (deps) (bzlmod) Upgraded to `bazel-skylib` version - [1.8.1](https://github.com/bazelbuild/bazel-skylib/releases/tag/1.8.1) - to remove deprecation warnings. * (gazelle) For package mode, resolve dependencies when imports are relative to the package path. This is enabled via the `# gazelle:python_experimental_allow_relative_imports` true directive ({gh-issue}`2203`). @@ -112,10 +109,6 @@ END_UNRELEASED_TEMPLATE * (toolchains) use "command -v" to find interpreter in `$PATH` ([#3150](https://github.com/bazel-contrib/rules_python/pull/3150)). * (pypi) `bazel vendor` now works in `bzlmod` ({gh-issue}`3079`). -* (pypi) Correctly pull `sdist` distributions using `pip` - ([#3131](https://github.com/bazel-contrib/rules_python/pull/3131)). -* (core) builds work again on `7.x` `WORKSPACE` configurations - ([#3119](https://github.com/bazel-contrib/rules_python/issues/3119)). {#v0-0-0-added} ### Added @@ -151,6 +144,24 @@ END_UNRELEASED_TEMPLATE ### Removed * Nothing removed. +{#1-5-2} +## [1.5.2] - 2025-08-11 + +[1.5.2]: https://github.com/bazel-contrib/rules_python/releases/tag/1.5.2 + +{#v1-5-2-changed} +### Changed +* (deps) (bzlmod) Upgraded to `bazel-skylib` version + [1.8.1](https://github.com/bazelbuild/bazel-skylib/releases/tag/1.8.1) + to remove deprecation warnings. + +{#v1-5-2-fixed} +### Fixed +* (pypi) Correctly pull `sdist` distributions using `pip` + ([#3131](https://github.com/bazel-contrib/rules_python/pull/3131)). +* (core) builds work again on `7.x` `WORKSPACE` configurations + ([#3119](https://github.com/bazel-contrib/rules_python/issues/3119)). + {#1-5-1} ## [1.5.1] - 2025-07-06 From 6038ac43659ba35cd5df8bacd36e39a42f7d9a50 Mon Sep 17 00:00:00 2001 From: honglooker Date: Mon, 11 Aug 2025 12:10:11 -0400 Subject: [PATCH 369/922] docs(toolchains): set dev_dependency=True on repo rule invocation (#3127) Move `dev_dependency = True` to the repo rule invocation. The `use_repo_rule` call doesn't support that arg. --------- Co-authored-by: Richard Levasseur --- docs/toolchains.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/toolchains.md b/docs/toolchains.md index de819cb515..52e619a120 100644 --- a/docs/toolchains.md +++ b/docs/toolchains.md @@ -432,13 +432,11 @@ interpreter, then introspect its installation to generate a full toolchain. local_runtime_repo = use_repo_rule( "@rules_python//python/local_toolchains:repos.bzl", "local_runtime_repo", - dev_dependency = True, ) local_runtime_toolchains_repo = use_repo_rule( "@rules_python//python/local_toolchains:repos.bzl", "local_runtime_toolchains_repo", - dev_dependency = True, ) # Step 1: Define the Python runtime @@ -446,6 +444,7 @@ local_runtime_repo( name = "local_python3", interpreter_path = "python3", on_failure = "fail", + dev_dependency = True ) # Step 2: Create toolchains for the runtimes @@ -454,6 +453,7 @@ local_runtime_toolchains_repo( runtimes = ["local_python3"], # TIP: The `target_settings` arg can be used to activate them based on # command line flags; see docs below. + dev_dependency = True ) # Step 3: Register the toolchains From 9382ed2125d6b99c6f699f84c5b610f6b39289c8 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Mon, 11 Aug 2025 09:40:57 -0700 Subject: [PATCH 370/922] docs: various howto guides (#3157) Create a "How to" section of docs that explain how to accomplish specific tasks. While these things can be inferred from the various API references, those are large and overwhelming, which can make it hard to figure it out. Instead, provide how to guides from tasks that we've seen users ask about. Fixes https://github.com/bazel-contrib/rules_python/issues/3044 --- docs/howto/build-a-wheel.md | 87 ++++++++++++++++++++++++ docs/howto/get-python-version.md | 57 ++++++++++++++++ docs/howto/index.md | 13 ++++ docs/howto/linking-libpython.md | 66 +++++++++++++++++++ docs/howto/pypi-headers.md | 109 +++++++++++++++++++++++++++++++ docs/howto/python-headers.md | 30 +++++++++ docs/index.md | 1 + 7 files changed, 363 insertions(+) create mode 100644 docs/howto/build-a-wheel.md create mode 100644 docs/howto/get-python-version.md create mode 100644 docs/howto/index.md create mode 100644 docs/howto/linking-libpython.md create mode 100644 docs/howto/pypi-headers.md create mode 100644 docs/howto/python-headers.md diff --git a/docs/howto/build-a-wheel.md b/docs/howto/build-a-wheel.md new file mode 100644 index 0000000000..0248ff802b --- /dev/null +++ b/docs/howto/build-a-wheel.md @@ -0,0 +1,87 @@ +:::{default-domain} bzl +::: + +# How to build a wheel + +This guide explains how to use the `py_wheel` rule to build a wheel +file from a `py_library`. + +## Basic usage + +The `py_wheel` rule takes any file-providing target as input and put its files +into a wheel. Because `py_library` provides its source files, simple cases can +pass `py_library` directly to `py_wheel`: + +```starlark +# BUILD.bazel + +py_library( + name = "my_project_lib", + srcs = glob(["my_project/**/*.py"]), + # ... +) + +py_wheel( + name = "my_project_wheel", + distribution = "my-project", + version = "0.1.0", + deps = [":my_project_lib"], +) +``` + +The above will include the *default outputs* of the `py_library`, which are the +direct `.py` files listed in the py library. It does **not** include transitive +dependencies. + +## Including and filtering transitive dependencies + + +Use the `py_package` rule to include and filter the transitive parts of +a `py_library` target. + +The `py_package` rule has a `packages` attribute that takes a list of dotted +Python package names to include. All files and dependencies of those packages +are included. + +Here is an example: + +```starlark +# BUILD.bazel + +py_library( + name = "my_project_lib", + srcs = glob(["my_project/**/*.py"]), + deps = ["@pypi//some_dep"], +) + +py_package( + name = "my_project_package", + # This will only include files for the "my_package" package; other files + # will be excluded. + packages = ["my_project"], +) + +py_wheel( + name = "my_project_wheel", + distribution = "my-project", + version = "0.1.0", + # The `py_wheel` rule takes the `py_package` target in the `deps` + # attribute. + deps = [":my_project_package"], +) +``` + +## Disabling `__init__.py` generation + +By default, Bazel automatically creates `__init__.py` files in directories to +make them importable. This can sometimes be undesirable when building wheels +because it interfers with namespace packages or makes directories importable +that shouldn't be importable. + +It's highly recommended to disable this behavior by setting a flag in your +`.bazelrc` file: + +``` +# .bazelrc +build --incompatible_default_to_explicit_init_py=true +``` \ No newline at end of file diff --git a/docs/howto/get-python-version.md b/docs/howto/get-python-version.md new file mode 100644 index 0000000000..92af433320 --- /dev/null +++ b/docs/howto/get-python-version.md @@ -0,0 +1,57 @@ +:::{default-domain} bzl +::: + +# How to get the current Python version + +This guide explains how to use a [toolchain](toolchains) to get the current Python +version and, as an example, write it to a file. + +You can create a simple rule that accesses the Python toolchain and retrieves +the version string. + +## The rule implementation + +Create a file named `my_rule.bzl`: + +```starlark +# my_rule.bzl +def _my_rule_impl(ctx): + toolchain = ctx.toolchains["@rules_python//python:toolchain_type"] + info = toolchain.py3_runtime.interpreter_version_info + python_version = str(info.major) + "." + str(info.minor) + "." + str(info.micro) + + output_file = ctx.actions.declare_file(ctx.attr.name + ".txt") + ctx.actions.write( + output = output_file, + content = python_version, + ) + + return [DefaultInfo(files = depset([output_file]))] + +my_rule = rule( + implementation = _my_rule_impl, + attrs = {}, + toolchains = ["@rules_python//python:toolchain_type"], +) +``` + +## Using the rule + +In your `BUILD.bazel` file, you can use the rule like this: + +```starlark +# BUILD.bazel +load(":my_rule.bzl", "my_rule") + +my_rule( + name = "show_python_version", +) +``` + +When you build this target, it will generate a file named +`show_python_version.txt` containing the Python version (e.g., `3.9`). + +```starlark +bazel build :show_python_version +cat bazel-bin/show_python_version.txt +``` diff --git a/docs/howto/index.md b/docs/howto/index.md new file mode 100644 index 0000000000..e6ddf70325 --- /dev/null +++ b/docs/howto/index.md @@ -0,0 +1,13 @@ +:::{default-domain} bzl +::: + +# How-to Guides + +This section contains a collection of how-to guides for accomplishing specific tasks with `rules_python`. + +```{toctree} +:maxdepth: 1 +:glob: + +* +``` \ No newline at end of file diff --git a/docs/howto/linking-libpython.md b/docs/howto/linking-libpython.md new file mode 100644 index 0000000000..4d70b322c9 --- /dev/null +++ b/docs/howto/linking-libpython.md @@ -0,0 +1,66 @@ +:::{default-domain} bzl +::: + +# How to link to libpython + +This guide explains how to use the Python [toolchain](toolchains) to get the linker +flags required for linking against `libpython`. This is often necessary when +embedding Python in a C/C++ application. + +Currently, the `:current_py_cc_libs` target does *not* include `-lpython` et al +linker flags. This is intentional because it forces dynamic linking (via the +dynamic linker processing `DT_NEEDED` entries), which prevents users who want +to load it in some more custom way. + +## Exposing linker flags in a rule + +You can create a rule that gets the Python version from the toolchain and +constructs the correct linker flag. This rule can then provide the flag to +other C/C++ rules via the `CcInfo` provider. + +Here's an example of a rule that creates the `-lpython` flag: + +```starlark +# python_libs.bzl +load("@bazel_tools//tools/cpp:toolchain_utils.bzl", "cc_common") + +def _python_libs_impl(ctx): + toolchain = ctx.toolchains["@rules_python//python:toolchain_type"] + info = toolchain.py3_runtime.interpreter_version_info + link_flag = "-lpython{}.{}".format(info.major, info.minor) + + cc_info = CcInfo( + linking_context = cc_common.create_linking_context( + user_link_flags = [link_flag], + ), + ) + return [cc_info] + +python_libs = rule( + implementation = _python_libs_impl, + toolchains = ["@rules_python//python:toolchain_type"], +) +``` + +## Using the rule + +In your `BUILD.bazel` file, define a target using this rule and add it to the +`deps` of your `cc_binary` or `cc_library`. + +```starlark +# BUILD.bazel +load(":python_libs.bzl", "python_libs") + +python_libs( + name = "py_libs", +) + +cc_binary( + name = "my_app", + srcs = ["my_app.c"], + deps = [ + ":py_libs", + # Other dependencies + ], +) +``` diff --git a/docs/howto/pypi-headers.md b/docs/howto/pypi-headers.md new file mode 100644 index 0000000000..3675031096 --- /dev/null +++ b/docs/howto/pypi-headers.md @@ -0,0 +1,109 @@ +:::{default-domain} bzl +::: + +# How to expose headers from a PyPI package + +When you depend on a PyPI package that includes C headers (like `numpy`), you +need to make those headers available to your `cc_library` or +`cc_binary` targets. + +The recommended way to do this is to inject a `BUILD.bazel` file into the +external repository for the package. This `BUILD` file will create +a `cc_library` target that exposes the header files. + +First, create a `.bzl` file that has the extra logic we'll inject. Putting it +in a separate bzl file avoids having to redownload and extract the whl file +when our logic changes. + +```bzl + +# pypi_extra_targets.bzl +load("@rules_cc//cc:cc_library.bzl", "cc_library") + +def extra_numpy_targets(): + cc_library( + name = "headers", + hdrs = glob(["**/*.h"]), + visibility = ["//visibility:public"], + ) +``` + +## Bzlmod setup + +In your `MODULE.bazel` file, use the `build_file_content` attribute of +`pip.parse` to inject the `BUILD` file content for the `numpy` package. + +```bazel +# MODULE.bazel +load("@rules_python//python/extensions:pip.bzl", "parse", "whl_mods") +pip = use_extension("@rules_python//python/extensions:pip.bzl", "pip") +whl_mods = use_extension("@rules_python//python/extensions:pip.bzl", "whl_mods") + + +# Define a specific modification for a wheel +whl_mods( + hub_name = "pypi_mods", + whl_name = "numpy-1.0.0-py3-none-any.whl", # The exact wheel filename + additive_build_content = """ +load("@//:pypi_extra_targets.bzl", "numpy_hdrs") + +extra_numpy_targets() +""", +) +pip.parse( + hub_name = "pypi", + wheel_name = "numpy", + requirements_lock = "//:requirements.txt", + whl_modifications = { + "@pypi_mods//:numpy.json": "numpy", + }, + extra_hub_aliases = { + "numpy": ["headers"], + } +) +``` + +## WORKSPACE setup + +In your `WORKSPACE` file, use the `annotations` attribute of `pip_parse` to +inject additional `BUILD` file content, then use `extra_hub_targets` to expose +that target in the `@pypi` hub repo. + +The {obj}`package_annotation` helper can be used to construct the value for the +`annotations` attribute. + +```starlark +# WORKSPACE +load("@rules_python//python:pip.bzl", "package_annotation", "pip_parse") + +pip_parse( + name = "pypi", + requirements_lock = "//:requirements.txt", + annotations = { + "numpy": package_annotation( + additive_build_content = """\ +load("@//:pypi_extra_targets.bzl", "numpy_hdrs") + +extra_numpy_targets() +""" + ), + }, + extra_hub_targets = { + "numpy": ["headers"], + }, +) +``` + +## Using the headers + +In your `BUILD.bazel` file, you can now depend on the generated `headers` +target. + +```bazel +# BUILD.bazel +cc_library( + name = "my_c_extension", + srcs = ["my_c_extension.c"], + deps = ["@pypi//numpy:headers"], +) +``` diff --git a/docs/howto/python-headers.md b/docs/howto/python-headers.md new file mode 100644 index 0000000000..f830febc81 --- /dev/null +++ b/docs/howto/python-headers.md @@ -0,0 +1,30 @@ +:::{default-domain} bzl +::: + +# How to get Python headers for C extensions + +When building a Python C extension, you need access to the Python header +files. This guide shows how to get the necessary include paths from the Python +[toolchain](toolchains). + +The recommended way to get the headers is to depend on the +`@rules_python//python/cc:current_py_cc_headers` target. This is a helper +target that uses toolchain resolution to find the correct headers for the +target platform. + +## Using the headers + +In your `BUILD.bazel` file, you can add `@rules_python//python/cc:current_py_cc_headers` +to the `deps` of a `cc_library` or `cc_binary` target. + +```bazel +# BUILD.bazel +cc_library( + name = "my_c_extension", + srcs = ["my_c_extension.c"], + deps = ["@rules_python//python/cc:current_py_cc_headers"], +) +``` + +This setup ensures that your C extension code can find and use the Python +headers during compilation. \ No newline at end of file diff --git a/docs/index.md b/docs/index.md index bdc6982ad5..7f03681b76 100644 --- a/docs/index.md +++ b/docs/index.md @@ -102,6 +102,7 @@ precompiling gazelle/docs/index REPL Extending +How-to Guides Contributing devguide support From 33da6af2e498746c772b4e6ad4b708aede9e9e2e Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Mon, 11 Aug 2025 17:12:46 -0700 Subject: [PATCH 371/922] docs: move changelog note to 1.5.3 section (#3163) Move the changelog note for the fix to #3043 into the 1.5.3 section, since that's the version it will be released with. --- CHANGELOG.md | 13 ++++++++++--- 1 file changed, 10 insertions(+), 3 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 55c4659b39..5f0505ed60 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -88,9 +88,6 @@ END_UNRELEASED_TEMPLATE * (runfiles) The pypi runfiles package now includes `py.typed` to indicate it supports type checking ([#2503](https://github.com/bazel-contrib/rules_python/issues/2503)). -* (toolchains) `local_runtime_repo` now checks if the include directory exists - before attempting to watch it, fixing issues on macOS with system Python - ([#3043](https://github.com/bazel-contrib/rules_python/issues/3043)). * (pypi) The pipstar `defaults` configuration now supports any custom platform name. * Multi-line python imports (e.g. with escaped newlines) are now correctly processed by Gazelle. @@ -144,6 +141,16 @@ END_UNRELEASED_TEMPLATE ### Removed * Nothing removed. +{#1-5-3} +## [1.5.3] - 2025-08-11 + +[1.5.3]: https://github.com/bazel-contrib/rules_python/releases/tag/1.5.3 + +### Fixed +* (toolchains) `local_runtime_repo` now checks if the include directory exists + before attempting to watch it, fixing issues on macOS with system Python + ([#3043](https://github.com/bazel-contrib/rules_python/issues/3043)). + {#1-5-2} ## [1.5.2] - 2025-08-11 From ecd395f47d44d6d72d67929007296f5214b178f8 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Tue, 12 Aug 2025 18:16:50 -0700 Subject: [PATCH 372/922] docs: link to PyRuntimeInfo and mention it has more than example shows (#3170) Have the example that gets the python version also link to the provider. Also mention that more than just the version is available. --- docs/howto/get-python-version.md | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/docs/howto/get-python-version.md b/docs/howto/get-python-version.md index 92af433320..8ff9cffd71 100644 --- a/docs/howto/get-python-version.md +++ b/docs/howto/get-python-version.md @@ -35,6 +35,10 @@ my_rule = rule( ) ``` +The `info` variable above is a {obj}`PyRuntimeInfo` object, which contains +information about the Python runtime. It contains more than just the version; +see the {obj}`PyRuntimeInfo` docs for its API documentation. + ## Using the rule In your `BUILD.bazel` file, you can use the rule like this: From 6a105019dc0e9595a300281175a410ba23408b0c Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Wed, 13 Aug 2025 22:28:34 +0900 Subject: [PATCH 373/922] build(deps): bump requests from 2.32.3 to 2.32.4 in /tools/publish in the pip group across 1 directory (#3169) Bumps the pip group with 1 update in the /tools/publish directory: [requests](https://github.com/psf/requests). Updates `requests` from 2.32.3 to 2.32.4
Release notes

Sourced from requests's releases.

v2.32.4

2.32.4 (2025-06-10)

Security

  • CVE-2024-47081 Fixed an issue where a maliciously crafted URL and trusted environment will retrieve credentials for the wrong hostname/machine from a netrc file. (#6965)

Improvements

  • Numerous documentation improvements

Deprecations

  • Added support for pypy 3.11 for Linux and macOS. (#6926)
  • Dropped support for pypy 3.9 following its end of support. (#6926)
Changelog

Sourced from requests's changelog.

2.32.4 (2025-06-10)

Security

  • CVE-2024-47081 Fixed an issue where a maliciously crafted URL and trusted environment will retrieve credentials for the wrong hostname/machine from a netrc file.

Improvements

  • Numerous documentation improvements

Deprecations

  • Added support for pypy 3.11 for Linux and macOS.
  • Dropped support for pypy 3.9 following its end of support.
Commits
  • 021dc72 Polish up release tooling for last manual release
  • 821770e Bump version and add release notes for v2.32.4
  • 59f8aa2 Add netrc file search information to authentication documentation (#6876)
  • 5b4b64c Add more tests to prevent regression of CVE 2024 47081
  • 7bc4587 Add new test to check netrc auth leak (#6962)
  • 96ba401 Only use hostname to do netrc lookup instead of netloc
  • 7341690 Merge pull request #6951 from tswast/patch-1
  • 6716d7c remove links
  • a7e1c74 Update docs/conf.py
  • c799b81 docs: fix dead links to kenreitz.org
  • Additional commits viewable in compare view

[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=requests&package-manager=pip&previous-version=2.32.3&new-version=2.32.4)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) ---
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot show ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore major version` will close this group update PR and stop Dependabot creating any more for the specific dependency's major version (unless you unignore this specific dependency's major version or upgrade to it yourself) - `@dependabot ignore minor version` will close this group update PR and stop Dependabot creating any more for the specific dependency's minor version (unless you unignore this specific dependency's minor version or upgrade to it yourself) - `@dependabot ignore ` will close this group update PR and stop Dependabot creating any more for the specific dependency (unless you unignore this specific dependency or upgrade to it yourself) - `@dependabot unignore ` will remove all of the ignore conditions of the specified dependency - `@dependabot unignore ` will remove the ignore condition of the specified dependency and ignore conditions You can disable automated security fix PRs for this repo from the [Security Alerts page](https://github.com/bazel-contrib/rules_python/network/alerts).
Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> --- tools/publish/requirements_darwin.txt | 6 +++--- tools/publish/requirements_linux.txt | 6 +++--- tools/publish/requirements_universal.txt | 6 +++--- tools/publish/requirements_windows.txt | 6 +++--- 4 files changed, 12 insertions(+), 12 deletions(-) diff --git a/tools/publish/requirements_darwin.txt b/tools/publish/requirements_darwin.txt index 11b1ddbea5..90cbd89793 100644 --- a/tools/publish/requirements_darwin.txt +++ b/tools/publish/requirements_darwin.txt @@ -190,9 +190,9 @@ readme-renderer==44.0 \ --hash=sha256:2fbca89b81a08526aadf1357a8c2ae889ec05fb03f5da67f9769c9a592166151 \ --hash=sha256:8712034eabbfa6805cacf1402b4eeb2a73028f72d1166d6f5cb7f9c047c5d1e1 # via twine -requests==2.32.3 \ - --hash=sha256:55365417734eb18255590a9ff9eb97e9e1da868d4ccd6402399eaf68af20a760 \ - --hash=sha256:70761cfe03c773ceb22aa2f671b4757976145175cdfca038c02654d061d6dcc6 +requests==2.32.4 \ + --hash=sha256:27babd3cda2a6d50b30443204ee89830707d396671944c998b5975b031ac2b2c \ + --hash=sha256:27d0316682c8a29834d3264820024b62a36942083d52caf2f14c0591336d3422 # via # requests-toolbelt # twine diff --git a/tools/publish/requirements_linux.txt b/tools/publish/requirements_linux.txt index eee98b98e7..448bbdf37a 100644 --- a/tools/publish/requirements_linux.txt +++ b/tools/publish/requirements_linux.txt @@ -302,9 +302,9 @@ readme-renderer==44.0 \ --hash=sha256:2fbca89b81a08526aadf1357a8c2ae889ec05fb03f5da67f9769c9a592166151 \ --hash=sha256:8712034eabbfa6805cacf1402b4eeb2a73028f72d1166d6f5cb7f9c047c5d1e1 # via twine -requests==2.32.3 \ - --hash=sha256:55365417734eb18255590a9ff9eb97e9e1da868d4ccd6402399eaf68af20a760 \ - --hash=sha256:70761cfe03c773ceb22aa2f671b4757976145175cdfca038c02654d061d6dcc6 +requests==2.32.4 \ + --hash=sha256:27babd3cda2a6d50b30443204ee89830707d396671944c998b5975b031ac2b2c \ + --hash=sha256:27d0316682c8a29834d3264820024b62a36942083d52caf2f14c0591336d3422 # via # requests-toolbelt # twine diff --git a/tools/publish/requirements_universal.txt b/tools/publish/requirements_universal.txt index 85648b24e9..9c8d017300 100644 --- a/tools/publish/requirements_universal.txt +++ b/tools/publish/requirements_universal.txt @@ -306,9 +306,9 @@ readme-renderer==44.0 \ --hash=sha256:2fbca89b81a08526aadf1357a8c2ae889ec05fb03f5da67f9769c9a592166151 \ --hash=sha256:8712034eabbfa6805cacf1402b4eeb2a73028f72d1166d6f5cb7f9c047c5d1e1 # via twine -requests==2.32.3 \ - --hash=sha256:55365417734eb18255590a9ff9eb97e9e1da868d4ccd6402399eaf68af20a760 \ - --hash=sha256:70761cfe03c773ceb22aa2f671b4757976145175cdfca038c02654d061d6dcc6 +requests==2.32.4 \ + --hash=sha256:27babd3cda2a6d50b30443204ee89830707d396671944c998b5975b031ac2b2c \ + --hash=sha256:27d0316682c8a29834d3264820024b62a36942083d52caf2f14c0591336d3422 # via # requests-toolbelt # twine diff --git a/tools/publish/requirements_windows.txt b/tools/publish/requirements_windows.txt index b2a01f474f..94e4962842 100644 --- a/tools/publish/requirements_windows.txt +++ b/tools/publish/requirements_windows.txt @@ -194,9 +194,9 @@ readme-renderer==44.0 \ --hash=sha256:2fbca89b81a08526aadf1357a8c2ae889ec05fb03f5da67f9769c9a592166151 \ --hash=sha256:8712034eabbfa6805cacf1402b4eeb2a73028f72d1166d6f5cb7f9c047c5d1e1 # via twine -requests==2.32.3 \ - --hash=sha256:55365417734eb18255590a9ff9eb97e9e1da868d4ccd6402399eaf68af20a760 \ - --hash=sha256:70761cfe03c773ceb22aa2f671b4757976145175cdfca038c02654d061d6dcc6 +requests==2.32.4 \ + --hash=sha256:27babd3cda2a6d50b30443204ee89830707d396671944c998b5975b031ac2b2c \ + --hash=sha256:27d0316682c8a29834d3264820024b62a36942083d52caf2f14c0591336d3422 # via # requests-toolbelt # twine From 282d9e842f1a019e82a338dc1145469b34c45f90 Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Wed, 13 Aug 2025 22:28:46 +0900 Subject: [PATCH 374/922] build(deps): bump absl-py from 2.2.2 to 2.3.1 in /docs (#3167) Bumps [absl-py](https://github.com/abseil/abseil-py) from 2.2.2 to 2.3.1.
Release notes

Sourced from absl-py's releases.

v2.3.1

Changed

  • (cleanup) Removed leftover code supporting Python < 3.8, as well as other references to older Python versions.

Fixed

  • (typechecking) Fixed typechecking errors that appeared under mypy release 1.16

v2.3.0

Added

  • (testing) Add extension point for letting TestLoader specify a custom sharding scheme.

Changed

  • Update package build and release process. Switched to using pyproject.toml, hatch, and GitHub Actions.
Changelog

Sourced from absl-py's changelog.

2.3.1 (2025-07-03)

Changed

  • (cleanup) Removed leftover code supporting Python < 3.8, as well as other references to older Python versions.

Fixed

  • (typechecking) Fixed typechecking errors that appeared under mypy release 1.16

2.3.0 (2025-05-26)

Added

  • (testing) Add extension point for letting TestLoader specify a custom sharding scheme.

Changed

  • Update package build and release process. Switched to using pyproject.toml, hatch, and GitHub Actions.
Commits
  • bdad52d Release Abseil-py 2.3.1
  • a2d0583 Clean up some references to older Python versions
  • 55c8f4d Fix typechecking errors that appeared under mypy release 1.16
  • aafb0d8 Add useful links to the abseil-py public files
  • 2f11045 Bump absl-py version to 2.3.0
  • 4d008a9 Update CHANGELOG
  • c31c4f6 Automatize package release process
  • 842bf09 Switch to pyproject.toml + hatchling
  • 369ce9b Fix help argument indentation in DEFINE_multi_enum_class function documen...
  • 71eb53d Add extension point for letting TestLoader specify a custom sharding scheme.
  • See full diff in compare view

[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=absl-py&package-manager=pip&previous-version=2.2.2&new-version=2.3.1)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) ---
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot show ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> --- docs/requirements.txt | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/requirements.txt b/docs/requirements.txt index 5ef561f06e..4feab0167a 100644 --- a/docs/requirements.txt +++ b/docs/requirements.txt @@ -1,9 +1,9 @@ # This file was autogenerated by uv via the following command: # bazel run //docs:requirements.update -absl-py==2.2.2 \ - --hash=sha256:bf25b2c2eed013ca456918c453d687eab4e8309fba81ee2f4c1a6aa2494175eb \ - --hash=sha256:e5797bc6abe45f64fd95dc06394ca3f2bedf3b5d895e9da691c9ee3397d70092 +absl-py==2.3.1 \ + --hash=sha256:a97820526f7fbfd2ec1bce83f3f25e3a14840dac0d8e02a0b71cd75db3f77fc9 \ + --hash=sha256:eeecf07f0c2a93ace0772c92e596ace6d3d3996c042b2128459aaae2a76de11d # via rules-python-docs (docs/pyproject.toml) alabaster==1.0.0 \ --hash=sha256:c00dca57bca26fa62a6d7d0a9fcce65f3e026e9bfe33e9c538fd3fbb2144fd9e \ From f86c8bc172f14c246c077819b5891a370b51aae5 Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Wed, 13 Aug 2025 13:29:07 +0000 Subject: [PATCH 375/922] build(deps): bump docutils from 0.21.2 to 0.22 in /tools/publish (#3168) Bumps [docutils](https://github.com/rtfd/recommonmark) from 0.21.2 to 0.22.
Commits

[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=docutils&package-manager=pip&previous-version=0.21.2&new-version=0.22)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) ---
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot show ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> --- tools/publish/requirements_darwin.txt | 6 +++--- tools/publish/requirements_linux.txt | 6 +++--- tools/publish/requirements_universal.txt | 6 +++--- tools/publish/requirements_windows.txt | 6 +++--- 4 files changed, 12 insertions(+), 12 deletions(-) diff --git a/tools/publish/requirements_darwin.txt b/tools/publish/requirements_darwin.txt index 90cbd89793..9b1e5a4258 100644 --- a/tools/publish/requirements_darwin.txt +++ b/tools/publish/requirements_darwin.txt @@ -104,9 +104,9 @@ charset-normalizer==3.4.2 \ --hash=sha256:fcbe676a55d7445b22c10967bceaaf0ee69407fbe0ece4d032b6eb8d4565982a \ --hash=sha256:fdb20a30fe1175ecabed17cbf7812f7b804b8a315a25f24678bcdf120a90077f # via requests -docutils==0.21.2 \ - --hash=sha256:3a6b18732edf182daa3cd12775bbb338cf5691468f91eeeb109deff6ebfa986f \ - --hash=sha256:dafca5b9e384f0e419294eb4d2ff9fa826435bf15f15b7bd45723e8ad76811b2 +docutils==0.22 \ + --hash=sha256:4ed966a0e96a0477d852f7af31bdcb3adc049fbb35ccba358c2ea8a03287615e \ + --hash=sha256:ba9d57750e92331ebe7c08a1bbf7a7f8143b86c476acd51528b042216a6aad0f # via readme-renderer idna==3.10 \ --hash=sha256:12f65c9b470abda6dc35cf8e63cc574b1c52b11df2c86030af0ac09b01b13ea9 \ diff --git a/tools/publish/requirements_linux.txt b/tools/publish/requirements_linux.txt index 448bbdf37a..80fb6a16e0 100644 --- a/tools/publish/requirements_linux.txt +++ b/tools/publish/requirements_linux.txt @@ -206,9 +206,9 @@ cryptography==44.0.1 \ --hash=sha256:f51f5705ab27898afda1aaa430f34ad90dc117421057782022edf0600bec5f14 \ --hash=sha256:fd0ee90072861e276b0ff08bd627abec29e32a53b2be44e41dbcdf87cbee2b00 # via secretstorage -docutils==0.21.2 \ - --hash=sha256:3a6b18732edf182daa3cd12775bbb338cf5691468f91eeeb109deff6ebfa986f \ - --hash=sha256:dafca5b9e384f0e419294eb4d2ff9fa826435bf15f15b7bd45723e8ad76811b2 +docutils==0.22 \ + --hash=sha256:4ed966a0e96a0477d852f7af31bdcb3adc049fbb35ccba358c2ea8a03287615e \ + --hash=sha256:ba9d57750e92331ebe7c08a1bbf7a7f8143b86c476acd51528b042216a6aad0f # via readme-renderer idna==3.10 \ --hash=sha256:12f65c9b470abda6dc35cf8e63cc574b1c52b11df2c86030af0ac09b01b13ea9 \ diff --git a/tools/publish/requirements_universal.txt b/tools/publish/requirements_universal.txt index 9c8d017300..3f1e2a756f 100644 --- a/tools/publish/requirements_universal.txt +++ b/tools/publish/requirements_universal.txt @@ -206,9 +206,9 @@ cryptography==44.0.1 ; sys_platform == 'linux' \ --hash=sha256:f51f5705ab27898afda1aaa430f34ad90dc117421057782022edf0600bec5f14 \ --hash=sha256:fd0ee90072861e276b0ff08bd627abec29e32a53b2be44e41dbcdf87cbee2b00 # via secretstorage -docutils==0.21.2 \ - --hash=sha256:3a6b18732edf182daa3cd12775bbb338cf5691468f91eeeb109deff6ebfa986f \ - --hash=sha256:dafca5b9e384f0e419294eb4d2ff9fa826435bf15f15b7bd45723e8ad76811b2 +docutils==0.22 \ + --hash=sha256:4ed966a0e96a0477d852f7af31bdcb3adc049fbb35ccba358c2ea8a03287615e \ + --hash=sha256:ba9d57750e92331ebe7c08a1bbf7a7f8143b86c476acd51528b042216a6aad0f # via readme-renderer idna==3.10 \ --hash=sha256:12f65c9b470abda6dc35cf8e63cc574b1c52b11df2c86030af0ac09b01b13ea9 \ diff --git a/tools/publish/requirements_windows.txt b/tools/publish/requirements_windows.txt index 94e4962842..e5d6eafd4c 100644 --- a/tools/publish/requirements_windows.txt +++ b/tools/publish/requirements_windows.txt @@ -104,9 +104,9 @@ charset-normalizer==3.4.2 \ --hash=sha256:fcbe676a55d7445b22c10967bceaaf0ee69407fbe0ece4d032b6eb8d4565982a \ --hash=sha256:fdb20a30fe1175ecabed17cbf7812f7b804b8a315a25f24678bcdf120a90077f # via requests -docutils==0.21.2 \ - --hash=sha256:3a6b18732edf182daa3cd12775bbb338cf5691468f91eeeb109deff6ebfa986f \ - --hash=sha256:dafca5b9e384f0e419294eb4d2ff9fa826435bf15f15b7bd45723e8ad76811b2 +docutils==0.22 \ + --hash=sha256:4ed966a0e96a0477d852f7af31bdcb3adc049fbb35ccba358c2ea8a03287615e \ + --hash=sha256:ba9d57750e92331ebe7c08a1bbf7a7f8143b86c476acd51528b042216a6aad0f # via readme-renderer idna==3.10 \ --hash=sha256:12f65c9b470abda6dc35cf8e63cc574b1c52b11df2c86030af0ac09b01b13ea9 \ From e8faf103d470e71d7fce0f4b6ac0c85654060ede Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Wed, 13 Aug 2025 13:29:18 +0000 Subject: [PATCH 376/922] build(deps): bump actions/checkout from 4 to 5 (#3165) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Bumps [actions/checkout](https://github.com/actions/checkout) from 4 to 5.
Release notes

Sourced from actions/checkout's releases.

v5.0.0

What's Changed

⚠️ Minimum Compatible Runner Version

v2.327.1
Release Notes

Make sure your runner is updated to this version or newer to use this release.

Full Changelog: https://github.com/actions/checkout/compare/v4...v5.0.0

v4.3.0

What's Changed

New Contributors

Full Changelog: https://github.com/actions/checkout/compare/v4...v4.3.0

v4.2.2

What's Changed

Full Changelog: https://github.com/actions/checkout/compare/v4.2.1...v4.2.2

v4.2.1

What's Changed

New Contributors

Full Changelog: https://github.com/actions/checkout/compare/v4.2.0...v4.2.1

... (truncated)

Changelog

Sourced from actions/checkout's changelog.

Changelog

V5.0.0

V4.3.0

v4.2.2

v4.2.1

v4.2.0

v4.1.7

v4.1.6

v4.1.5

v4.1.4

v4.1.3

... (truncated)

Commits

[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=actions/checkout&package-manager=github_actions&previous-version=4&new-version=5)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) ---
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot show ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> --- .github/workflows/mypy.yaml | 2 +- .github/workflows/release.yml | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/.github/workflows/mypy.yaml b/.github/workflows/mypy.yaml index e774b9b03b..b83b5d4b37 100644 --- a/.github/workflows/mypy.yaml +++ b/.github/workflows/mypy.yaml @@ -18,7 +18,7 @@ jobs: runs-on: ubuntu-latest steps: # Checkout the code - - uses: actions/checkout@v4 + - uses: actions/checkout@v5 - uses: jpetrucciani/mypy-check@master with: requirements: 1.6.0 diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index 436797e3ed..e13ab97fb6 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -25,7 +25,7 @@ jobs: runs-on: ubuntu-latest steps: - name: Checkout - uses: actions/checkout@v4 + uses: actions/checkout@v5 - name: Create release archive and notes run: .github/workflows/create_archive_and_notes.sh - name: Publish wheel dist From bba07596838c2f6bcf7a32335d0599182b43507e Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Wed, 13 Aug 2025 22:29:37 +0900 Subject: [PATCH 377/922] build(deps): bump astroid from 3.3.9 to 3.3.11 in /docs (#3164) Bumps [astroid](https://github.com/pylint-dev/astroid) from 3.3.9 to 3.3.11.
Release notes

Sourced from astroid's releases.

v3.3.11

  • Fix a crash when parsing an empty arbitrary expression with extract_node (extract_node("__()")).

    Closes #2734

  • Fix a crash when parsing a slice called in a decorator on a function that is also decorated with a known six decorator.

    Closes #2721

v3.3.10

  • Avoid importing submodules sharing names with standard library modules.

    Closes #2684

  • Fix bug where pylint code.custom_extension would analyze code.py or code.pyi instead if they existed.

    Closes pylint-dev/pylint#3631

Changelog

Sourced from astroid's changelog.

What's New in astroid 3.3.11?

Release date: 2025-07-13

  • Fix a crash when parsing an empty arbitrary expression with extract_node (extract_node("__()")).

    Closes #2734

  • Fix a crash when parsing a slice called in a decorator on a function that is also decorated with a known six decorator.

    Closes #2721

What's New in astroid 3.3.10?

Release date: 2025-05-10

  • Avoid importing submodules sharing names with standard library modules.

    Closes #2684

  • Fix bug where pylint code.custom_extension would analyze code.py or code.pyi instead if they existed.

    Closes pylint-dev/pylint#3631

Commits
  • fbea510 Bump astroid to 3.3.11, update changelog (#2777)
  • bf3977c Include subclasses of standard property classes as property decorators (#2735)
  • 18f9626 Use custom Github App to authenticate backport job (#2751) (#2752)
  • c1d9c73 Improve backport job permissions (#2750)
  • b1adb1c [Backport maintenance/3.3.x] Initial fixes for Python 3.14 (#2747) (#2748)
  • 0aaf213 [fix] Prevent crash on slice decorator for 'six' decorated function (#2738) (...
  • c8bd28a [fix] Crash when parsing an empty arbitrary expression with extract_node ...
  • a362368 Bump astroid to 3.3.10, update changelog (#2730)
  • d87efc6 Pick correct file if two files with the same name but with different extensio...
  • e29d726 [setuptools] Upgrade the license handling for latest setuptools
  • Additional commits viewable in compare view

[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=astroid&package-manager=pip&previous-version=3.3.9&new-version=3.3.11)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) ---
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot show ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> --- docs/requirements.txt | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/requirements.txt b/docs/requirements.txt index 4feab0167a..af691dfd21 100644 --- a/docs/requirements.txt +++ b/docs/requirements.txt @@ -9,9 +9,9 @@ alabaster==1.0.0 \ --hash=sha256:c00dca57bca26fa62a6d7d0a9fcce65f3e026e9bfe33e9c538fd3fbb2144fd9e \ --hash=sha256:fc6786402dc3fcb2de3cabd5fe455a2db534b371124f1f21de8731783dec828b # via sphinx -astroid==3.3.9 \ - --hash=sha256:622cc8e3048684aa42c820d9d218978021c3c3d174fb03a9f0d615921744f550 \ - --hash=sha256:d05bfd0acba96a7bd43e222828b7d9bc1e138aaeb0649707908d3702a9831248 +astroid==3.3.11 \ + --hash=sha256:1e5a5011af2920c7c67a53f65d536d65bfa7116feeaf2354d8b94f29573bb0ce \ + --hash=sha256:54c760ae8322ece1abd213057c4b5bba7c49818853fc901ef09719a60dbf9dec # via sphinx-autodoc2 babel==2.17.0 \ --hash=sha256:0c54cffb19f690cdcc52a3b50bcbf71e07a808d1c80d549f2459b9d2cf0afb9d \ From 3262233b2e37f8f28e9e78a16ba3d2cad8850e55 Mon Sep 17 00:00:00 2001 From: Ignas Anikevicius <240938+aignas@users.noreply.github.com> Date: Wed, 13 Aug 2025 22:35:15 +0900 Subject: [PATCH 378/922] feat(pypi): implement a new whl selection algorithm (#3111) This PR only implements the selection algorithm where instead of selecting all wheels that are compatible with the set of target platforms, we select a single wheel that is most specialized for a particular single target platform. What is more, compared to the existing algorithm it does not assume a particular list of supported platforms and just fully implements the spec. Work towards #2747 Work towards #2759 Work towards #2849 --- python/private/pypi/BUILD.bazel | 10 + python/private/pypi/select_whl.bzl | 237 +++++++++++ tests/pypi/select_whl/BUILD.bazel | 3 + tests/pypi/select_whl/select_whl_tests.bzl | 463 +++++++++++++++++++++ 4 files changed, 713 insertions(+) create mode 100644 python/private/pypi/select_whl.bzl create mode 100644 tests/pypi/select_whl/BUILD.bazel create mode 100644 tests/pypi/select_whl/select_whl_tests.bzl diff --git a/python/private/pypi/BUILD.bazel b/python/private/pypi/BUILD.bazel index 3a66170768..4b56b73284 100644 --- a/python/private/pypi/BUILD.bazel +++ b/python/private/pypi/BUILD.bazel @@ -365,6 +365,16 @@ bzl_library( ], ) +bzl_library( + name = "select_whl_bzl", + srcs = ["select_whl.bzl"], + deps = [ + ":parse_whl_name_bzl", + ":python_tag_bzl", + "//python/private:version_bzl", + ], +) + bzl_library( name = "simpleapi_download_bzl", srcs = ["simpleapi_download.bzl"], diff --git a/python/private/pypi/select_whl.bzl b/python/private/pypi/select_whl.bzl new file mode 100644 index 0000000000..e9db1886e7 --- /dev/null +++ b/python/private/pypi/select_whl.bzl @@ -0,0 +1,237 @@ +"Select a single wheel that fits the parameters of a target platform." + +load("//python/private:version.bzl", "version") +load(":parse_whl_name.bzl", "parse_whl_name") +load(":python_tag.bzl", "PY_TAG_GENERIC", "python_tag") + +_ANDROID = "android" +_IOS = "ios" +_MANYLINUX = "manylinux" +_MACOSX = "macosx" +_MUSLLINUX = "musllinux" + +def _value_priority(*, tag, values): + keys = [] + for priority, wp in enumerate(values): + if tag == wp: + keys.append(priority) + + return max(keys) if keys else None + +def _platform_tag_priority(*, tag, values): + # Implements matching platform tag + # https://packaging.python.org/en/latest/specifications/platform-compatibility-tags/ + + if not ( + tag.startswith(_ANDROID) or + tag.startswith(_IOS) or + tag.startswith(_MACOSX) or + tag.startswith(_MANYLINUX) or + tag.startswith(_MUSLLINUX) + ): + res = _value_priority(tag = tag, values = values) + if res == None: + return res + + return (res, (0, 0)) + + # Only android, ios, macosx, manylinux or musllinux platforms should be considered + + os, _, tail = tag.partition("_") + major, _, tail = tail.partition("_") + if not os.startswith(_ANDROID): + minor, _, arch = tail.partition("_") + else: + minor = "0" + arch = tail + version = (int(major), int(minor)) + + keys = [] + for priority, wp in enumerate(values): + want_os, sep, tail = wp.partition("_") + if not sep: + # if there is no `_` separator, then it means that we have something like `win32` or + # similar wheels that we are considering, this means that it should be discarded because + # we are dealing only with platforms that have `_`. + continue + + if want_os != os: + # os should always match exactly for us to match and assign a priority + continue + + want_major, _, tail = tail.partition("_") + if want_major == "*": + # the expected match is any version + want_major = "" + want_minor = "" + want_arch = tail + elif os.startswith(_ANDROID): + # we set it to `0` above, so setting the `want_minor` her to `0` will make things + # consistent. + want_minor = "0" + want_arch = tail + else: + # here we parse the values from the given platform + want_minor, _, want_arch = tail.partition("_") + + if want_arch != arch: + # the arch should match exactly + continue + + # if want_major is defined, then we know that we don't have a `*` in the matcher. + want_version = (int(want_major), int(want_minor)) if want_major else None + if not want_version or version <= want_version: + keys.append((priority, version)) + + return max(keys) if keys else None + +def _python_tag_priority(*, tag, implementation, py_version): + if tag.startswith(PY_TAG_GENERIC): + ver_str = tag[len(PY_TAG_GENERIC):] + elif tag.startswith(implementation): + ver_str = tag[len(implementation):] + else: + return None + + # Add a 0 at the end in case it is a single digit + ver_str = "{}.{}".format(ver_str[0], ver_str[1:] or "0") + + ver = version.parse(ver_str) + if not version.is_compatible(py_version, ver): + return None + + return ( + tag.startswith(implementation), + version.key(ver), + ) + +def _candidates_by_priority( + *, + whls, + implementation_name, + python_version, + whl_abi_tags, + whl_platform_tags, + logger): + """Calculate the priority of each wheel + + Returns: + A dictionary where keys are priority tuples which allows us to sort and pick the + last item. + """ + py_version = version.parse(python_version, strict = True) + implementation = python_tag(implementation_name) + + ret = {} + for whl in whls: + parsed = parse_whl_name(whl.filename) + priority = None + + # See https://packaging.python.org/en/latest/specifications/platform-compatibility-tags/#compressed-tag-sets + for platform in parsed.platform_tag.split("."): + platform = _platform_tag_priority(tag = platform, values = whl_platform_tags) + if platform == None: + logger.debug(lambda: "The platform_tag in '{}' does not match given list: {}".format( + whl.filename, + whl_platform_tags, + )) + continue + + for py in parsed.python_tag.split("."): + py = _python_tag_priority( + tag = py, + implementation = implementation, + py_version = py_version, + ) + if py == None: + logger.debug(lambda: "The python_tag in '{}' does not match implementation or version: {} {}".format( + whl.filename, + implementation, + py_version.string, + )) + continue + + for abi in parsed.abi_tag.split("."): + abi = _value_priority( + tag = abi, + values = whl_abi_tags, + ) + if abi == None: + logger.debug(lambda: "The abi_tag in '{}' does not match given list: {}".format( + whl.filename, + whl_abi_tags, + )) + continue + + # 1. Prefer platform wheels + # 2. Then prefer implementation/python version + # 3. Then prefer more specific ABI wheels + candidate = (platform, py, abi) + priority = priority or candidate + if candidate > priority: + priority = candidate + + if priority == None: + logger.debug(lambda: "The whl '{}' is incompatible".format( + whl.filename, + )) + continue + + ret[priority] = whl + + return ret + +def select_whl( + *, + whls, + python_version, + whl_platform_tags, + whl_abi_tags, + implementation_name = "cpython", + limit = 1, + logger): + """Select a whl that is the most suitable for the given platform. + + Args: + whls: {type}`list[struct]` a list of candidates which have a `filename` + attribute containing the `whl` filename. + python_version: {type}`str` the target python version. + implementation_name: {type}`str` the `implementation_name` from the target_platform env. + whl_abi_tags: {type}`list[str]` The whl abi tags to select from. The preference is + for wheels that have ABI values appearing later in the `whl_abi_tags` list. + whl_platform_tags: {type}`list[str]` The whl platform tags to select from. + The platform tag may contain `*` and this means that if the platform tag is + versioned (e.g. `manylinux`), then we will select the highest available + platform version, e.g. if `manylinux_2_17` and `manylinux_2_5` wheels are both + compatible, we will select `manylinux_2_17`. Otherwise for versioned platform + tags we select the highest *compatible* version, e.g. if `manylinux_2_6` + support is requested, then we would select `manylinux_2_5` in the previous + example. This allows us to pass the same filtering parameters when selecting + all of the whl dependencies irrespective of what actual platform tags they + contain. + limit: {type}`int` number of wheels to return. Defaults to 1. + logger: {type}`struct` the logger instance. + + Returns: + {type}`list[struct] | struct | None`, a single struct from the `whls` input + argument or `None` if a match is not found. If the `limit` is greater than + one, then we will return a list. + """ + candidates = _candidates_by_priority( + whls = whls, + implementation_name = implementation_name, + python_version = python_version, + whl_abi_tags = whl_abi_tags, + whl_platform_tags = whl_platform_tags, + logger = logger, + ) + + if not candidates: + return None + + res = [i[1] for i in sorted(candidates.items())] + logger.debug(lambda: "Sorted candidates:\n{}".format( + "\n".join([c.filename for c in res]), + )) + + return res[-1] if limit == 1 else res[-limit:] diff --git a/tests/pypi/select_whl/BUILD.bazel b/tests/pypi/select_whl/BUILD.bazel new file mode 100644 index 0000000000..0ad8cba0cd --- /dev/null +++ b/tests/pypi/select_whl/BUILD.bazel @@ -0,0 +1,3 @@ +load(":select_whl_tests.bzl", "select_whl_test_suite") + +select_whl_test_suite(name = "select_whl_tests") diff --git a/tests/pypi/select_whl/select_whl_tests.bzl b/tests/pypi/select_whl/select_whl_tests.bzl new file mode 100644 index 0000000000..28e17ba3b3 --- /dev/null +++ b/tests/pypi/select_whl/select_whl_tests.bzl @@ -0,0 +1,463 @@ +"" + +load("@rules_testing//lib:test_suite.bzl", "test_suite") +load("//python/private:repo_utils.bzl", "REPO_DEBUG_ENV_VAR", "REPO_VERBOSITY_ENV_VAR", "repo_utils") # buildifier: disable=bzl-visibility +load("//python/private/pypi:select_whl.bzl", "select_whl") # buildifier: disable=bzl-visibility + +WHL_LIST = [ + "pkg-0.0.1-cp311-cp311-macosx_10_9_universal2.whl", + "pkg-0.0.1-cp311-cp311-macosx_10_9_x86_64.whl", + "pkg-0.0.1-cp311-cp311-macosx_11_0_arm64.whl", + "pkg-0.0.1-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", + "pkg-0.0.1-cp311-cp311-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", + "pkg-0.0.1-cp311-cp311-manylinux_2_17_s390x.manylinux2014_s390x.whl", + "pkg-0.0.1-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", + "pkg-0.0.1-cp311-cp311-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", + "pkg-0.0.1-cp313-cp313t-musllinux_1_1_x86_64.whl", + "pkg-0.0.1-cp313-cp313-musllinux_1_1_x86_64.whl", + "pkg-0.0.1-cp313-abi3-musllinux_1_1_x86_64.whl", + "pkg-0.0.1-cp313-none-musllinux_1_1_x86_64.whl", + "pkg-0.0.1-cp311-cp311-musllinux_1_1_aarch64.whl", + "pkg-0.0.1-cp311-cp311-musllinux_1_1_i686.whl", + "pkg-0.0.1-cp311-cp311-musllinux_1_1_ppc64le.whl", + "pkg-0.0.1-cp311-cp311-musllinux_1_1_s390x.whl", + "pkg-0.0.1-cp311-cp311-musllinux_1_1_x86_64.whl", + "pkg-0.0.1-cp311-cp311-win32.whl", + "pkg-0.0.1-cp311-cp311-win_amd64.whl", + "pkg-0.0.1-cp37-cp37m-macosx_10_9_x86_64.whl", + "pkg-0.0.1-cp37-cp37m-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", + "pkg-0.0.1-cp37-cp37m-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", + "pkg-0.0.1-cp37-cp37m-manylinux_2_17_s390x.manylinux2014_s390x.whl", + "pkg-0.0.1-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", + "pkg-0.0.1-cp37-cp37m-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", + "pkg-0.0.1-cp37-cp37m-musllinux_1_1_aarch64.whl", + "pkg-0.0.1-cp37-cp37m-musllinux_1_1_i686.whl", + "pkg-0.0.1-cp37-cp37m-musllinux_1_1_ppc64le.whl", + "pkg-0.0.1-cp37-cp37m-musllinux_1_1_s390x.whl", + "pkg-0.0.1-cp37-cp37m-musllinux_1_1_x86_64.whl", + "pkg-0.0.1-cp37-cp37m-win32.whl", + "pkg-0.0.1-cp37-cp37m-win_amd64.whl", + "pkg-0.0.1-cp39-cp39-macosx_10_9_universal2.whl", + "pkg-0.0.1-cp39-cp39-macosx_10_9_x86_64.whl", + "pkg-0.0.1-cp39-cp39-macosx_11_0_arm64.whl", + "pkg-0.0.1-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", + "pkg-0.0.1-cp39-cp39-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", + "pkg-0.0.1-cp39-cp39-manylinux_2_17_s390x.manylinux2014_s390x.whl", + "pkg-0.0.1-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", + "pkg-0.0.1-cp39-cp39-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", + "pkg-0.0.1-cp39-cp39-musllinux_1_1_aarch64.whl", + "pkg-0.0.1-cp39-cp39-musllinux_1_1_i686.whl", + "pkg-0.0.1-cp39-cp39-musllinux_1_1_ppc64le.whl", + "pkg-0.0.1-cp39-cp39-musllinux_1_1_s390x.whl", + "pkg-0.0.1-cp39-cp39-musllinux_1_1_x86_64.whl", + "pkg-0.0.1-cp39-cp39-win32.whl", + "pkg-0.0.1-cp39-cp39-win_amd64.whl", + "pkg-0.0.1-cp39-abi3-any.whl", + "pkg-0.0.1-py310-abi3-any.whl", + "pkg-0.0.1-py3-abi3-any.whl", + "pkg-0.0.1-py3-none-any.whl", + # Extra examples that should be discarded + "pkg-0.0.1-py27-cp27mu-win_amd64.whl", + "pkg-0.0.1-pp310-pypy310_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", +] + +def _match(env, got, *want_filenames): + if not want_filenames: + env.expect.that_collection(got).has_size(len(want_filenames)) + return + + got = [g for g in got if g] + got_filenames = [g.filename for g in got] + env.expect.that_collection(got_filenames).contains_exactly(want_filenames).in_order() + + if got: + # Check that we pass the original structs + env.expect.that_str(got[0].other).equals("dummy") + +def _select_whl(whls, debug = False, **kwargs): + return select_whl( + whls = [ + struct( + filename = f, + other = "dummy", + ) + for f in whls + ], + logger = repo_utils.logger(struct( + os = struct( + environ = { + REPO_DEBUG_ENV_VAR: "1", + REPO_VERBOSITY_ENV_VAR: "TRACE" if debug else "INFO", + }, + ), + ), "unit-test"), + **kwargs + ) + +_tests = [] + +def _test_not_select_py2(env): + # Check we prefer platform specific wheels + got = _select_whl( + whls = [ + "pkg-0.0.1-py2-none-any.whl", + "pkg-0.0.1-py3-none-any.whl", + "pkg-0.0.1-py312-none-any.whl", + ], + whl_platform_tags = ["any"], + whl_abi_tags = ["none"], + python_version = "3.13", + limit = 2, + ) + _match( + env, + got, + "pkg-0.0.1-py3-none-any.whl", + "pkg-0.0.1-py312-none-any.whl", + ) + +_tests.append(_test_not_select_py2) + +def _test_not_select_abi3(env): + # Check we prefer platform specific wheels + got = _select_whl( + whls = [ + "pkg-0.0.1-py3-none-any.whl", + # the following should be ignored + "pkg-0.0.1-py3-abi3-any.whl", + "pkg-0.0.1-py3-abi3-p1.p2.p2.whl", + ], + whl_platform_tags = ["any", "p1"], + whl_abi_tags = ["none"], + python_version = "3.13", + limit = 2, + debug = True, + ) + _match( + env, + got, + "pkg-0.0.1-py3-none-any.whl", + ) + +_tests.append(_test_not_select_abi3) + +def _test_select_cp312(env): + # Check we prefer platform specific wheels + got = _select_whl( + whls = [ + "pkg-0.0.1-py2-none-any.whl", + "pkg-0.0.1-py3-none-any.whl", + "pkg-0.0.1-py312-none-any.whl", + "pkg-0.0.1-cp39-none-any.whl", + "pkg-0.0.1-cp312-none-any.whl", + "pkg-0.0.1-cp314-none-any.whl", + ], + whl_platform_tags = ["any"], + whl_abi_tags = ["none"], + python_version = "3.13", + limit = 5, + ) + _match( + env, + got, + "pkg-0.0.1-py3-none-any.whl", + "pkg-0.0.1-py312-none-any.whl", + "pkg-0.0.1-cp39-none-any.whl", + "pkg-0.0.1-cp312-none-any.whl", + ) + +_tests.append(_test_select_cp312) + +def _test_simplest(env): + whls = [ + "pkg-0.0.1-py2.py3-abi3-any.whl", + "pkg-0.0.1-py3-abi3-any.whl", + "pkg-0.0.1-py3-none-any.whl", + ] + + got = _select_whl( + whls = whls, + whl_platform_tags = ["any"], + whl_abi_tags = ["abi3"], + python_version = "3.0", + ) + _match( + env, + [got], + "pkg-0.0.1-py3-abi3-any.whl", + ) + +_tests.append(_test_simplest) + +def _test_select_by_supported_py_version(env): + whls = [ + "pkg-0.0.1-py2.py3-abi3-any.whl", + "pkg-0.0.1-py3-abi3-any.whl", + "pkg-0.0.1-py311-abi3-any.whl", + ] + + for minor_version, match in { + 8: "pkg-0.0.1-py3-abi3-any.whl", + 11: "pkg-0.0.1-py311-abi3-any.whl", + }.items(): + got = _select_whl( + whls = whls, + whl_platform_tags = ["any"], + whl_abi_tags = ["abi3"], + python_version = "3.{}".format(minor_version), + ) + _match(env, [got], match) + +_tests.append(_test_select_by_supported_py_version) + +def _test_select_by_supported_cp_version(env): + whls = [ + "pkg-0.0.1-py2.py3-abi3-any.whl", + "pkg-0.0.1-py3-abi3-any.whl", + "pkg-0.0.1-py311-abi3-any.whl", + "pkg-0.0.1-cp311-abi3-any.whl", + ] + + for minor_version, match in { + 11: "pkg-0.0.1-cp311-abi3-any.whl", + 8: "pkg-0.0.1-py3-abi3-any.whl", + }.items(): + got = _select_whl( + whls = whls, + whl_platform_tags = ["any"], + whl_abi_tags = ["abi3"], + python_version = "3.{}".format(minor_version), + ) + _match(env, [got], match) + +_tests.append(_test_select_by_supported_cp_version) + +def _test_supported_cp_version_manylinux(env): + whls = [ + "pkg-0.0.1-py2.py3-none-manylinux_1_1_x86_64.whl", + "pkg-0.0.1-py3-none-manylinux_1_1_x86_64.whl", + "pkg-0.0.1-py311-none-manylinux_1_1_x86_64.whl", + "pkg-0.0.1-cp311-none-manylinux_1_1_x86_64.whl", + ] + + for minor_version, match in { + 8: "pkg-0.0.1-py3-none-manylinux_1_1_x86_64.whl", + 11: "pkg-0.0.1-cp311-none-manylinux_1_1_x86_64.whl", + }.items(): + got = _select_whl( + whls = whls, + whl_platform_tags = ["manylinux_1_1_x86_64"], + whl_abi_tags = ["none"], + python_version = "3.{}".format(minor_version), + ) + _match(env, [got], match) + +_tests.append(_test_supported_cp_version_manylinux) + +def _test_ignore_unsupported(env): + whls = ["pkg-0.0.1-xx3-abi3-any.whl"] + got = _select_whl( + whls = whls, + whl_platform_tags = ["any"], + whl_abi_tags = ["none"], + python_version = "3.0", + ) + if got: + _match(env, [got], None) + +_tests.append(_test_ignore_unsupported) + +def _test_match_abi_and_not_py_version(env): + # Check we match the ABI and not the py version + whls = WHL_LIST + whl_platform_tags = [ + "musllinux_*_x86_64", + "manylinux_*_x86_64", + ] + got = _select_whl( + whls = whls, + whl_platform_tags = whl_platform_tags, + whl_abi_tags = ["abi3", "cp37m"], + python_version = "3.7", + ) + _match( + env, + [got], + "pkg-0.0.1-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", + ) + + got = _select_whl( + whls = whls, + whl_platform_tags = whl_platform_tags[::-1], + whl_abi_tags = ["abi3", "cp37m"], + python_version = "3.7", + ) + _match( + env, + [got], + "pkg-0.0.1-cp37-cp37m-musllinux_1_1_x86_64.whl", + ) + +_tests.append(_test_match_abi_and_not_py_version) + +def _test_select_filename_with_many_tags(env): + # Check we can select a filename with many platform tags + got = _select_whl( + whls = WHL_LIST, + whl_platform_tags = [ + "any", + "musllinux_*_i686", + "manylinux_*_i686", + ], + whl_abi_tags = ["none", "abi3", "cp39"], + python_version = "3.9", + limit = 5, + ) + _match( + env, + got, + "pkg-0.0.1-py3-none-any.whl", + "pkg-0.0.1-py3-abi3-any.whl", + "pkg-0.0.1-cp39-abi3-any.whl", + "pkg-0.0.1-cp39-cp39-musllinux_1_1_i686.whl", + "pkg-0.0.1-cp39-cp39-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", + ) + +_tests.append(_test_select_filename_with_many_tags) + +def _test_freethreaded_wheels(env): + # Check we prefer platform specific wheels + got = _select_whl( + whls = WHL_LIST, + whl_platform_tags = [ + "any", + "musllinux_*_x86_64", + ], + whl_abi_tags = ["none", "abi3", "cp313", "cp313t"], + python_version = "3.13", + limit = 8, + ) + _match( + env, + got, + # The last item has the most priority + "pkg-0.0.1-py3-none-any.whl", + "pkg-0.0.1-py3-abi3-any.whl", + "pkg-0.0.1-py310-abi3-any.whl", + "pkg-0.0.1-cp39-abi3-any.whl", + "pkg-0.0.1-cp313-none-musllinux_1_1_x86_64.whl", + "pkg-0.0.1-cp313-abi3-musllinux_1_1_x86_64.whl", + "pkg-0.0.1-cp313-cp313-musllinux_1_1_x86_64.whl", + "pkg-0.0.1-cp313-cp313t-musllinux_1_1_x86_64.whl", + ) + +_tests.append(_test_freethreaded_wheels) + +def _test_pytags_all_possible(env): + got = _select_whl( + whls = [ + "pkg-0.0.1-py2.py27.py3.py30.py31.py32.py33.py34.py35.py36.py37.py38.py39.py310.py311.py312.py313-none-win_amd64.whl", + ], + whl_platform_tags = ["win_amd64"], + whl_abi_tags = ["none"], + python_version = "3.12", + ) + _match( + env, + [got], + "pkg-0.0.1-py2.py27.py3.py30.py31.py32.py33.py34.py35.py36.py37.py38.py39.py310.py311.py312.py313-none-win_amd64.whl", + ) + +_tests.append(_test_pytags_all_possible) + +def _test_manylinx_musllinux_pref(env): + got = _select_whl( + whls = [ + "pkg-0.0.1-py3-none-manylinux_2_31_x86_64.musllinux_1_1_x86_64.whl", + ], + whl_platform_tags = [ + "manylinux_*_x86_64", + "musllinux_*_x86_64", + ], + whl_abi_tags = ["none"], + python_version = "3.12", + limit = 2, + ) + _match( + env, + got, + # there is only one wheel, just select that + "pkg-0.0.1-py3-none-manylinux_2_31_x86_64.musllinux_1_1_x86_64.whl", + ) + +_tests.append(_test_manylinx_musllinux_pref) + +def _test_multiple_musllinux(env): + got = _select_whl( + whls = [ + "pkg-0.0.1-py3-none-musllinux_1_2_x86_64.whl", + "pkg-0.0.1-py3-none-musllinux_1_1_x86_64.whl", + ], + whl_platform_tags = ["musllinux_*_x86_64"], + whl_abi_tags = ["none"], + python_version = "3.12", + limit = 2, + ) + _match( + env, + got, + # select the one with the highest version that is matching + "pkg-0.0.1-py3-none-musllinux_1_1_x86_64.whl", + "pkg-0.0.1-py3-none-musllinux_1_2_x86_64.whl", + ) + +_tests.append(_test_multiple_musllinux) + +def _test_multiple_musllinux_exact_params(env): + got = _select_whl( + whls = [ + "pkg-0.0.1-py3-none-musllinux_1_2_x86_64.whl", + "pkg-0.0.1-py3-none-musllinux_1_1_x86_64.whl", + ], + whl_platform_tags = ["musllinux_1_2_x86_64", "musllinux_1_1_x86_64"], + whl_abi_tags = ["none"], + python_version = "3.12", + limit = 2, + ) + _match( + env, + got, + # select the one with the lowest version, because of the input to the function + "pkg-0.0.1-py3-none-musllinux_1_2_x86_64.whl", + "pkg-0.0.1-py3-none-musllinux_1_1_x86_64.whl", + ) + +_tests.append(_test_multiple_musllinux_exact_params) + +def _test_android(env): + got = _select_whl( + whls = [ + "pkg-0.0.1-py3-none-android_4_x86_64.whl", + "pkg-0.0.1-py3-none-android_8_x86_64.whl", + ], + whl_platform_tags = ["android_5_x86_64"], + whl_abi_tags = ["none"], + python_version = "3.12", + limit = 2, + ) + _match( + env, + got, + # select the one with the highest version that is matching + "pkg-0.0.1-py3-none-android_4_x86_64.whl", + ) + +_tests.append(_test_android) + +def select_whl_test_suite(name): + """Create the test suite. + + Args: + name: the name of the test suite + """ + test_suite(name = name, basic_tests = _tests) From 2eacb709c17aa6394dd1caa7e014447f94669110 Mon Sep 17 00:00:00 2001 From: elsk Date: Wed, 13 Aug 2025 12:52:03 -0700 Subject: [PATCH 379/922] fix: bootstrapping script to not use multiline f-strings (#3175) With bootstrap_impl=system_python, after https://github.com/bazel-contrib/rules_python/pull/2607, host Python 3.6 and above is required because of the use of multiline f-strings. For maximum backwards compatibility (because this is a host dependency), do not use multiline f-strings. This issue naturally does not apply to bootstrap_impl=script because there are no host Python dependency. Link: https://github.com/bazel-contrib/rules_python/pull/2607 --- python/private/python_bootstrap_template.txt | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/python/private/python_bootstrap_template.txt b/python/private/python_bootstrap_template.txt index a979fd4422..1eaa0483df 100644 --- a/python/private/python_bootstrap_template.txt +++ b/python/private/python_bootstrap_template.txt @@ -436,11 +436,11 @@ def _RunForCoverage(python_program, main_filename, args, env, unique_id = uuid.uuid4() rcfile_name = os.path.join(os.environ['COVERAGE_DIR'], ".coveragerc_{}".format(unique_id)) with open(rcfile_name, "w") as rcfile: - rcfile.write(f'''[run] + rcfile.write('''[run] relative_files = True source = \t{source} -''') +'''.format(source=source)) PrintVerboseCoverage('Coverage entrypoint:', coverage_entrypoint) # First run the target Python file via coveragepy to create a .coverage # database file, from which we can later export lcov. From 9843447a4cf9bf5af41211b2ddb44a293425739e Mon Sep 17 00:00:00 2001 From: honglooker Date: Wed, 13 Aug 2025 22:35:56 -0400 Subject: [PATCH 380/922] docs: add 1.4.2 changelog (#3173) This documents the backport made in https://github.com/bazel-contrib/rules_python/pull/3174 --- CHANGELOG.md | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/CHANGELOG.md b/CHANGELOG.md index 5f0505ed60..37a8c1dbe1 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -263,6 +263,16 @@ END_UNRELEASED_TEMPLATE ### Removed * Nothing removed. +{#1-4-2} +## [1.4.2] - 2025-08-13 + +[1.4.2]: https://github.com/bazel-contrib/rules_python/releases/tag/1.4.2 + +### Fixed +* (toolchains) `local_runtime_repo` now checks if the include directory exists + before attempting to watch it, fixing issues on macOS with system Python + ([#3043](https://github.com/bazel-contrib/rules_python/issues/3043)). + {#1-4-1} ## [1.4.1] - 2025-05-08 From 3ade15cc6f1973c8f2ba24fac6a6ddb956021e1f Mon Sep 17 00:00:00 2001 From: Laramie Leavitt Date: Thu, 14 Aug 2025 12:54:55 -0700 Subject: [PATCH 381/922] fix(local_runtime): Improve local_runtime usability in macos / windows (#3148) local_runtime fails to handle many variations of python install on Windows and MacOS, such as: * LDLIBRARY on MacOS may refer to a file under PYTHONFRAMEWORKPREFIX, not LIBDIR * LDLIBRARY on Windows refers to pythonXY.dll, not the linkable pythonXY.lib * LIBDIR may not be correctly set on Windows. * On windows, interpreter_path needs to be normalized. Other paths also require this. * SHLIB_SUFFIX does not indicate the correct suffix. For examples, see: https://docs.python.org/3/extending/windows.html In order to resolve this the shared library resolution has been moved into get_local_runtime_info.py, which now does the following: * Constructs a list of paths to search based on LIBDIR, LIBPL, PYTHONFRAMEWORKPREFIX, and the executable directory. * Constructs a list of libraries to search based on INSTSONAME, LDLIBRARY, pythonXY.lib, etc. * Checks to see which files exist, partitioning the result into a list of "dynamic_libraries" and "static_libraries" On Windows and macOS, since SHLIB_SUFFIX does not always indicate the filenames needed searching, this has been removed from local_runtime_repo_setup and replaced with an explicit file. On Windows the interpreter_path and other search paths are now normalized (`\` converted to `/`). Additional logging added to local_runtime_repo. Fixes https://github.com/bazel-contrib/rules_python/issues/3055 Work towards https://github.com/bazel-contrib/rules_python/issues/824 --------- Co-authored-by: Richard Levasseur --- .bazelci/presubmit.yml | 19 ++ .gitignore | 2 + CHANGELOG.md | 6 + python/private/get_local_runtime_info.py | 199 +++++++++++++++--- python/private/local_runtime_repo.bzl | 124 ++++++----- python/private/local_runtime_repo_setup.bzl | 41 ++-- tests/integration/BUILD.bazel | 11 + .../integration/local_toolchains/BUILD.bazel | 29 +++ .../integration/local_toolchains/MODULE.bazel | 1 + tests/integration/local_toolchains/WORKSPACE | 31 +++ .../integration/local_toolchains/echo_ext.cc | 21 ++ .../integration/local_toolchains/echo_test.py | 9 + .../local_toolchains/py_extension.bzl | 154 ++++++++++++++ tests/integration/local_toolchains/test.py | 12 +- 14 files changed, 559 insertions(+), 100 deletions(-) create mode 100644 tests/integration/local_toolchains/echo_ext.cc create mode 100644 tests/integration/local_toolchains/echo_test.py create mode 100644 tests/integration/local_toolchains/py_extension.bzl diff --git a/.bazelci/presubmit.yml b/.bazelci/presubmit.yml index 6457363ccd..5889823d3d 100644 --- a/.bazelci/presubmit.yml +++ b/.bazelci/presubmit.yml @@ -57,6 +57,7 @@ buildifier: - "--enable_workspace" - "--build_tag_filters=-integration-test" bazel: 7.x +# NOTE: The Mac and Windows bazelinbazel jobs override parts of this config. .common_bazelinbazel_config: &common_bazelinbazel_config build_flags: - "--build_tag_filters=integration-test" @@ -503,6 +504,24 @@ tasks: <<: *common_bazelinbazel_config name: "tests/integration bazel-in-bazel: Debian" platform: debian11 + # The bazelinbazel tests were disabled on Mac to save CI jobs slots, and + # have bitrotted a bit. For now, just run a subset of what we're most + # interested in. + integration_test_bazelinbazel_macos: + <<: *common_bazelinbazel_config + name: "tests/integration bazel-in-bazel: macOS (subset)" + platform: macos + build_targets: ["//tests/integration:local_toolchains_test_bazel_self"] + test_targets: ["//tests/integration:local_toolchains_test_bazel_self"] + # The bazelinbazel tests were disabled on Windows to save CI jobs slots, and + # have bitrotted a bit. For now, just run a subset of what we're most + # interested in. + integration_test_bazelinbazel_windows: + <<: *common_bazelinbazel_config + name: "tests/integration bazel-in-bazel: Windows (subset)" + platform: windows + build_targets: ["//tests/integration:local_toolchains_test_bazel_self"] + test_targets: ["//tests/integration:local_toolchains_test_bazel_self"] integration_test_compile_pip_requirements_ubuntu: <<: *reusable_build_test_all diff --git a/.gitignore b/.gitignore index 863b0e9c3f..fb1b17e466 100644 --- a/.gitignore +++ b/.gitignore @@ -37,6 +37,8 @@ /bazel-genfiles /bazel-out /bazel-testlogs +**/bazel-* + user.bazelrc # vim swap files diff --git a/CHANGELOG.md b/CHANGELOG.md index 37a8c1dbe1..2dc235fbf6 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -106,6 +106,12 @@ END_UNRELEASED_TEMPLATE * (toolchains) use "command -v" to find interpreter in `$PATH` ([#3150](https://github.com/bazel-contrib/rules_python/pull/3150)). * (pypi) `bazel vendor` now works in `bzlmod` ({gh-issue}`3079`). +* (toolchains) `local_runtime_repo` now works on Windows + ([#3055](https://github.com/bazel-contrib/rules_python/issues/3055)). +* (toolchains) `local_runtime_repo` supports more types of Python + installations (Mac frameworks, missing dynamic libraries, and other + esoteric cases, see + [#3148](https://github.com/bazel-contrib/rules_python/pull/3148) for details). {#v0-0-0-added} ### Added diff --git a/python/private/get_local_runtime_info.py b/python/private/get_local_runtime_info.py index c8371357c2..ff3b0aeb01 100644 --- a/python/private/get_local_runtime_info.py +++ b/python/private/get_local_runtime_info.py @@ -12,47 +12,188 @@ # See the License for the specific language governing permissions and # limitations under the License. +"""Returns information about the local Python runtime as JSON.""" + import json +import os import sys import sysconfig -data = { - "major": sys.version_info.major, - "minor": sys.version_info.minor, - "micro": sys.version_info.micro, - "include": sysconfig.get_path("include"), - "implementation_name": sys.implementation.name, - "base_executable": sys._base_executable, -} +_IS_WINDOWS = sys.platform == "win32" +_IS_DARWIN = sys.platform == "darwin" + -config_vars = [ - # The libpythonX.Y.so file. Usually? - # It might be a static archive (.a) file instead. - "LDLIBRARY", - # The directory with library files. Supposedly. - # It's not entirely clear how to get the directory with libraries. +def _search_directories(get_config): + """Returns a list of library directories to search for shared libraries.""" # There's several types of libraries with different names and a plethora - # of settings. + # of settings, and many different config variables to check: + # + # LIBPL is used in python-config when shared library is not enabled: + # https://github.com/python/cpython/blob/v3.12.0/Misc/python-config.in#L63 + # + # LIBDIR may also be the python directory with library files. # https://stackoverflow.com/questions/47423246/get-pythons-lib-path - # For now, it seems LIBDIR has what is needed, so just use that. # See also: MULTIARCH - "LIBDIR", + # + # On MacOS, the LDLIBRARY may be a relative path under /Library/Frameworks, + # such as "Python.framework/Versions/3.12/Python", not a file under the + # LIBDIR/LIBPL directory, so include PYTHONFRAMEWORKPREFIX. + lib_dirs = [get_config(x) for x in ("PYTHONFRAMEWORKPREFIX", "LIBPL", "LIBDIR")] + # On Debian, with multiarch enabled, prior to Python 3.10, `LIBDIR` didn't # tell the location of the libs, just the base directory. The `MULTIARCH` # sysconfig variable tells the subdirectory within it with the libs. # See: # https://wiki.debian.org/Python/MultiArch # https://git.launchpad.net/ubuntu/+source/python3.12/tree/debian/changelog#n842 - "MULTIARCH", - # The versioned libpythonX.Y.so.N file. Usually? - # It might be a static archive (.a) file instead. - "INSTSONAME", - # The libpythonX.so file. Usually? - # It might be a static archive (a.) file instead. - "PY3LIBRARY", - # The platform-specific filename suffix for library files. - # Includes the dot, e.g. `.so` - "SHLIB_SUFFIX", -] -data.update(zip(config_vars, sysconfig.get_config_vars(*config_vars))) + multiarch = get_config("MULTIARCH") + if multiarch: + for x in ("LIBPL", "LIBDIR"): + config_value = get_config(x) + if config_value and not config_value.endswith(multiarch): + lib_dirs.append(os.path.join(config_value, multiarch)) + + if _IS_WINDOWS: + # On Windows DLLs go in the same directory as the executable, while .lib + # files live in the lib/ or libs/ subdirectory. + lib_dirs.append(get_config("BINDIR")) + lib_dirs.append(os.path.join(os.path.dirname(sys.executable))) + lib_dirs.append(os.path.join(os.path.dirname(sys.executable), "lib")) + lib_dirs.append(os.path.join(os.path.dirname(sys.executable), "libs")) + elif not _IS_DARWIN: + # On most systems the executable is in a bin/ directory and the libraries + # are in a sibling lib/ directory. + lib_dirs.append( + os.path.join(os.path.dirname(os.path.dirname(sys.executable)), "lib") + ) + + # Dedup and remove empty values, keeping the order. + lib_dirs = [v for v in lib_dirs if v] + return {k: None for k in lib_dirs}.keys() + + +def _search_library_names(get_config): + """Returns a list of library files to search for shared libraries.""" + # Quoting configure.ac in the cpython code base: + # "INSTSONAME is the name of the shared library that will be use to install + # on the system - some systems like version suffix, others don't."" + # + # A typical INSTSONAME is 'libpython3.8.so.1.0' on Linux, or + # 'Python.framework/Versions/3.9/Python' on MacOS. + # + # A typical LDLIBRARY is 'libpythonX.Y.so' on Linux, or 'pythonXY.dll' on + # Windows, or 'Python.framework/Versions/3.9/Python' on MacOS. + # + # A typical LIBRARY is 'libpythonX.Y.a' on Linux. + lib_names = [ + get_config(x) + for x in ( + "LDLIBRARY", + "INSTSONAME", + "PY3LIBRARY", + "LIBRARY", + "DLLLIBRARY", + ) + ] + + # Set the prefix and suffix to construct the library name used for linking. + # The suffix and version are set here to the default values for the OS, + # since they are used below to construct "default" library names. + if _IS_DARWIN: + suffix = ".dylib" + prefix = "lib" + elif _IS_WINDOWS: + suffix = ".dll" + prefix = "" + else: + suffix = get_config("SHLIB_SUFFIX") + prefix = "lib" + if not suffix: + suffix = ".so" + + version = get_config("VERSION") + + # Ensure that the pythonXY.dll files are included in the search. + lib_names.append(f"{prefix}python{version}{suffix}") + + # If there are ABIFLAGS, also add them to the python version lib search. + abiflags = get_config("ABIFLAGS") or get_config("abiflags") or "" + if abiflags: + lib_names.append(f"{prefix}python{version}{abiflags}{suffix}") + + # Dedup and remove empty values, keeping the order. + lib_names = [v for v in lib_names if v] + return {k: None for k in lib_names}.keys() + + +def _get_python_library_info(): + """Returns a dictionary with the static and dynamic python libraries.""" + config_vars = sysconfig.get_config_vars() + + # VERSION is X.Y in Linux/macOS and XY in Windows. This is used to + # construct library paths such as python3.12, so ensure it exists. + if not config_vars.get("VERSION"): + if sys.platform == "win32": + config_vars["VERSION"] = f"{sys.version_info.major}{sys.version_info.minor}" + else: + config_vars["VERSION"] = ( + f"{sys.version_info.major}.{sys.version_info.minor}" + ) + + search_directories = _search_directories(config_vars.get) + search_libnames = _search_library_names(config_vars.get) + + def _add_if_exists(target, path): + if os.path.exists(path) or os.path.isdir(path): + target[path] = None + + interface_libraries = {} + dynamic_libraries = {} + static_libraries = {} + for root_dir in search_directories: + for libname in search_libnames: + composed_path = os.path.join(root_dir, libname) + if libname.endswith(".a"): + _add_if_exists(static_libraries, composed_path) + continue + + _add_if_exists(dynamic_libraries, composed_path) + if libname.endswith(".dll"): + # On windows a .lib file may be an "import library" or a static library. + # The file could be inspected to determine which it is; typically python + # is used as a shared library. + # + # On Windows, extensions should link with the pythonXY.lib interface + # libraries. + # + # See: https://docs.python.org/3/extending/windows.html + # https://learn.microsoft.com/en-us/windows/win32/dlls/dynamic-link-library-creation + _add_if_exists( + interface_libraries, os.path.join(root_dir, libname[:-3] + "lib") + ) + elif libname.endswith(".so"): + # It's possible, though unlikely, that interface stubs (.ifso) exist. + _add_if_exists( + interface_libraries, os.path.join(root_dir, libname[:-2] + "ifso") + ) + + # When no libraries are found it's likely that the python interpreter is not + # configured to use shared or static libraries (minilinux). If this seems + # suspicious try running `uv tool run find_libpython --list-all -v` + return { + "dynamic_libraries": list(dynamic_libraries.keys()), + "static_libraries": list(static_libraries.keys()), + "interface_libraries": list(interface_libraries.keys()), + } + + +data = { + "major": sys.version_info.major, + "minor": sys.version_info.minor, + "micro": sys.version_info.micro, + "include": sysconfig.get_path("include"), + "implementation_name": sys.implementation.name, + "base_executable": sys._base_executable, +} +data.update(_get_python_library_info()) print(json.dumps(data)) diff --git a/python/private/local_runtime_repo.bzl b/python/private/local_runtime_repo.bzl index b8b7164b54..21bdfa627e 100644 --- a/python/private/local_runtime_repo.bzl +++ b/python/private/local_runtime_repo.bzl @@ -31,27 +31,67 @@ load("@rules_python//python/private:local_runtime_repo_setup.bzl", "define_local define_local_runtime_toolchain_impl( name = "local_runtime", - lib_ext = "{lib_ext}", major = "{major}", minor = "{minor}", micro = "{micro}", interpreter_path = "{interpreter_path}", + interface_library = {interface_library}, + libraries = {libraries}, implementation_name = "{implementation_name}", os = "{os}", ) """ +def _norm_path(path): + """Returns a path using '/' separators and no trailing slash.""" + path = path.replace("\\", "/") + if path[-1] == "/": + path = path[:-1] + return path + +def _symlink_first_library(rctx, logger, libraries): + """Symlinks the shared libraries into the lib/ directory. + + Args: + rctx: A repository_ctx object + logger: A repo_utils.logger object + libraries: A list of static library paths to potentially symlink. + Returns: + A single library path linked by the action. + """ + linked = None + for target in libraries: + origin = rctx.path(target) + if not origin.exists: + # The reported names don't always exist; it depends on the particulars + # of the runtime installation. + continue + if target.endswith("/Python"): + linked = "lib/{}.dylib".format(origin.basename) + else: + linked = "lib/{}".format(origin.basename) + logger.debug("Symlinking {} to {}".format(origin, linked)) + repo_utils.watch(rctx, origin) + rctx.symlink(origin, linked) + break + + return linked + def _local_runtime_repo_impl(rctx): logger = repo_utils.logger(rctx) on_failure = rctx.attr.on_failure - result = _resolve_interpreter_path(rctx) - if not result.resolved_path: + def _emit_log(msg): if on_failure == "fail": - fail("interpreter not found: {}".format(result.describe_failure())) + logger.fail(msg) + elif on_failure == "warn": + logger.warn(msg) + else: + logger.debug(msg) - if on_failure == "warn": - logger.warn(lambda: "interpreter not found: {}".format(result.describe_failure())) + result = _resolve_interpreter_path(rctx) + if not result.resolved_path: + _emit_log(lambda: "interpreter not found: {}".format(result.describe_failure())) # else, on_failure must be skip rctx.file("BUILD.bazel", _expand_incompatible_template()) @@ -72,10 +112,7 @@ def _local_runtime_repo_impl(rctx): logger = logger, ) if exec_result.return_code != 0: - if on_failure == "fail": - fail("GetPythonInfo failed: {}".format(exec_result.describe_failure())) - if on_failure == "warn": - logger.warn(lambda: "GetPythonInfo failed: {}".format(exec_result.describe_failure())) + _emit_log(lambda: "GetPythonInfo failed: {}".format(exec_result.describe_failure())) # else, on_failure must be skip rctx.file("BUILD.bazel", _expand_incompatible_template()) @@ -112,53 +149,37 @@ def _local_runtime_repo_impl(rctx): # The cc_library.includes values have to be non-absolute paths, otherwise # the toolchain will give an error. Work around this error by making them # appear as part of this repo. - rctx.symlink(info["include"], "include") - - shared_lib_names = [ - info["PY3LIBRARY"], - info["LDLIBRARY"], - info["INSTSONAME"], - ] - - # In some cases, the value may be empty. Not clear why. - shared_lib_names = [v for v in shared_lib_names if v] - - # In some cases, the same value is returned for multiple keys. Not clear why. - shared_lib_names = {v: None for v in shared_lib_names}.keys() - shared_lib_dir = info["LIBDIR"] - multiarch = info["MULTIARCH"] - - # The specific files are symlinked instead of the whole directory - # because it can point to a directory that has more than just - # the Python runtime shared libraries, e.g. /usr/lib, or a Python - # specific directory with pip-installed shared libraries. - rctx.report_progress("Symlinking external Python shared libraries") - for name in shared_lib_names: - origin = rctx.path("{}/{}".format(shared_lib_dir, name)) + rctx.symlink(include_path, "include") - # If the origin doesn't exist, try the multiarch location, in case - # it's an older Python / Debian release. - if not origin.exists and multiarch: - origin = rctx.path("{}/{}/{}".format(shared_lib_dir, multiarch, name)) - - # The reported names don't always exist; it depends on the particulars - # of the runtime installation. - if origin.exists: - repo_utils.watch(rctx, origin) - rctx.symlink(origin, "lib/" + name) + rctx.report_progress("Symlinking external Python shared libraries") + interface_library = _symlink_first_library(rctx, logger, info["interface_libraries"]) + shared_library = _symlink_first_library(rctx, logger, info["dynamic_libraries"]) + static_library = _symlink_first_library(rctx, logger, info["static_libraries"]) + + libraries = [] + if shared_library: + libraries.append(shared_library) + elif static_library: + libraries.append(static_library) + else: + logger.warn("No external python libraries found.") - rctx.file("WORKSPACE", "") - rctx.file("MODULE.bazel", "") - rctx.file("REPO.bazel", "") - rctx.file("BUILD.bazel", _TOOLCHAIN_IMPL_TEMPLATE.format( + build_bazel = _TOOLCHAIN_IMPL_TEMPLATE.format( major = info["major"], minor = info["minor"], micro = info["micro"], - interpreter_path = interpreter_path, - lib_ext = info["SHLIB_SUFFIX"], + interpreter_path = _norm_path(interpreter_path), + interface_library = repr(interface_library), + libraries = repr(libraries), implementation_name = info["implementation_name"], os = "@platforms//os:{}".format(repo_utils.get_platforms_os_name(rctx)), - )) + ) + logger.debug(lambda: "BUILD.bazel\n{}".format(build_bazel)) + + rctx.file("WORKSPACE", "") + rctx.file("MODULE.bazel", "") + rctx.file("REPO.bazel", "") + rctx.file("BUILD.bazel", build_bazel) local_runtime_repo = repository_rule( implementation = _local_runtime_repo_impl, @@ -218,7 +239,8 @@ def _expand_incompatible_template(): return _TOOLCHAIN_IMPL_TEMPLATE.format( interpreter_path = "/incompatible", implementation_name = "incompatible", - lib_ext = "incompatible", + interface_library = "None", + libraries = "[]", major = "0", minor = "0", micro = "0", diff --git a/python/private/local_runtime_repo_setup.bzl b/python/private/local_runtime_repo_setup.bzl index 37eab59575..5d3a781152 100644 --- a/python/private/local_runtime_repo_setup.bzl +++ b/python/private/local_runtime_repo_setup.bzl @@ -15,6 +15,7 @@ """Setup code called by the code generated by `local_runtime_repo`.""" load("@bazel_skylib//lib:selects.bzl", "selects") +load("@rules_cc//cc:cc_import.bzl", "cc_import") load("@rules_cc//cc:cc_library.bzl", "cc_library") load("@rules_python//python:py_runtime.bzl", "py_runtime") load("@rules_python//python:py_runtime_pair.bzl", "py_runtime_pair") @@ -25,11 +26,12 @@ _PYTHON_VERSION_FLAG = Label("@rules_python//python/config_settings:python_versi def define_local_runtime_toolchain_impl( name, - lib_ext, major, minor, micro, interpreter_path, + interface_library, + libraries, implementation_name, os): """Defines a toolchain implementation for a local Python runtime. @@ -45,11 +47,14 @@ def define_local_runtime_toolchain_impl( Args: name: `str` Only present to satisfy tooling - lib_ext: `str` The file extension for the `libpython` shared libraries major: `str` The major Python version, e.g. `3` of `3.9.1`. minor: `str` The minor Python version, e.g. `9` of `3.9.1`. micro: `str` The micro Python version, e.g. "1" of `3.9.1`. interpreter_path: `str` Absolute path to the interpreter. + interface_library: `str` Path to the interface library. + e.g. "lib/python312.lib" + libraries: `list[str]` Path[s] to the python libraries. + e.g. ["lib/python312.dll"] or ["lib/python312.so"] implementation_name: `str` The implementation name, as returned by `sys.implementation.name`. os: `str` A label to the OS constraint (e.g. `@platforms//os:linux`) for @@ -58,30 +63,36 @@ def define_local_runtime_toolchain_impl( major_minor = "{}.{}".format(major, minor) major_minor_micro = "{}.{}".format(major_minor, micro) + # To build Python C/C++ extension on Windows, we need to link to python import library pythonXY.lib + # See https://docs.python.org/3/extending/windows.html + # However not all python installations (such as manylinux) include shared or static libraries, + # so only create the import library when interface_library is set. + import_deps = [] + if interface_library: + cc_import( + name = "_python_interface_library", + interface_library = interface_library, + system_provided = 1, + ) + import_deps = [":_python_interface_library"] + cc_library( name = "_python_headers", # NOTE: Keep in sync with watch_tree() called in local_runtime_repo srcs = native.glob( - ["include/**/*.h"], - # A Python install may not have C headers - allow_empty = True, + include = ["include/**/*.h"], + exclude = ["include/numpy/**"], # numpy headers are handled separately + allow_empty = True, # A Python install may not have C headers ), + deps = import_deps, includes = ["include"], ) cc_library( name = "_libpython", - # Don't use a recursive glob because the lib/ directory usually contains - # a subdirectory of the stdlib -- lots of unrelated files - srcs = native.glob( - [ - "lib/*{}".format(lib_ext), # Match libpython*.so - "lib/*{}*".format(lib_ext), # Also match libpython*.so.1.0 - ], - # A Python install may not have shared libraries. - allow_empty = True, - ), hdrs = [":_python_headers"], + srcs = libraries, + deps = [], ) py_runtime( diff --git a/tests/integration/BUILD.bazel b/tests/integration/BUILD.bazel index d178e0f01c..df7fe15444 100644 --- a/tests/integration/BUILD.bazel +++ b/tests/integration/BUILD.bazel @@ -95,6 +95,17 @@ rules_python_integration_test( ], ) +rules_python_integration_test( + name = "local_toolchains_workspace_test", + bazel_versions = [ + version + for version in bazel_binaries.versions.all + if not version.startswith("6.") + ], + bzlmod = False, + workspace_path = "local_toolchains", +) + rules_python_integration_test( name = "pip_parse_test", ) diff --git a/tests/integration/local_toolchains/BUILD.bazel b/tests/integration/local_toolchains/BUILD.bazel index 6b731181a6..a0cb2b164d 100644 --- a/tests/integration/local_toolchains/BUILD.bazel +++ b/tests/integration/local_toolchains/BUILD.bazel @@ -13,7 +13,9 @@ # limitations under the License. load("@bazel_skylib//rules:common_settings.bzl", "string_flag") +load("@rules_cc//cc:cc_library.bzl", "cc_library") load("@rules_python//python:py_test.bzl", "py_test") +load(":py_extension.bzl", "py_extension") py_test( name = "test", @@ -35,3 +37,30 @@ string_flag( name = "py", build_setting_default = "", ) + +# Build rules to generate a python extension. +cc_library( + name = "echo_ext_cc", + testonly = True, + srcs = ["echo_ext.cc"], + deps = [ + "@rules_python//python/cc:current_py_cc_headers", + ], + alwayslink = True, +) + +py_extension( + name = "echo_ext", + testonly = True, + copts = select({ + "@rules_cc//cc/compiler:msvc-cl": [], + "//conditions:default": ["-fvisibility=hidden"], + }), + deps = [":echo_ext_cc"], +) + +py_test( + name = "echo_test", + srcs = ["echo_test.py"], + deps = [":echo_ext"], +) diff --git a/tests/integration/local_toolchains/MODULE.bazel b/tests/integration/local_toolchains/MODULE.bazel index 6c06909cd7..45afaafbc9 100644 --- a/tests/integration/local_toolchains/MODULE.bazel +++ b/tests/integration/local_toolchains/MODULE.bazel @@ -16,6 +16,7 @@ module(name = "module_under_test") bazel_dep(name = "rules_python", version = "0.0.0") bazel_dep(name = "bazel_skylib", version = "1.7.1") bazel_dep(name = "platforms", version = "0.0.11") +bazel_dep(name = "rules_cc", version = "0.0.16") local_path_override( module_name = "rules_python", diff --git a/tests/integration/local_toolchains/WORKSPACE b/tests/integration/local_toolchains/WORKSPACE index e69de29bb2..480cd2794a 100644 --- a/tests/integration/local_toolchains/WORKSPACE +++ b/tests/integration/local_toolchains/WORKSPACE @@ -0,0 +1,31 @@ +workspace( + name = "module_under_test", +) + +local_repository( + name = "rules_python", + path = "../../..", +) + +load("@rules_python//python:repositories.bzl", "py_repositories") + +py_repositories() + +load("@rules_python//python/local_toolchains:repos.bzl", "local_runtime_repo", "local_runtime_toolchains_repo") + +# Step 1: Define the python runtime. +local_runtime_repo( + name = "local_python3", + interpreter_path = "python3", + on_failure = "fail", + # or interpreter_path = "C:\\path\\to\\python.exe" +) + +# Step 2: Create toolchains for the runtimes +local_runtime_toolchains_repo( + name = "local_toolchains", + runtimes = ["local_python3"], +) + +# Step 3: Register the toolchains +register_toolchains("@local_toolchains//:all") diff --git a/tests/integration/local_toolchains/echo_ext.cc b/tests/integration/local_toolchains/echo_ext.cc new file mode 100644 index 0000000000..367d1a13b3 --- /dev/null +++ b/tests/integration/local_toolchains/echo_ext.cc @@ -0,0 +1,21 @@ +#include + +static PyObject *echoArgs(PyObject *self, PyObject *args) { return args; } + +static PyMethodDef echo_methods[] = { + { "echo", echoArgs, METH_VARARGS, "Returns a tuple of the input args" }, + { NULL, NULL, 0, NULL }, +}; + +extern "C" { + +PyMODINIT_FUNC PyInit_echo_ext(void) { + static struct PyModuleDef echo_module_def = { + // Module definition + PyModuleDef_HEAD_INIT, "echo_ext", "'echo_ext' module", -1, echo_methods + }; + + return PyModule_Create(&echo_module_def); +} + +} // extern "C" diff --git a/tests/integration/local_toolchains/echo_test.py b/tests/integration/local_toolchains/echo_test.py new file mode 100644 index 0000000000..4cc31ff759 --- /dev/null +++ b/tests/integration/local_toolchains/echo_test.py @@ -0,0 +1,9 @@ +import unittest + +import echo_ext + + +class ExtensionTest(unittest.TestCase): + + def test_echo_extension(self): + self.assertEqual(echo_ext.echo(42, "str"), tuple(42, "str")) diff --git a/tests/integration/local_toolchains/py_extension.bzl b/tests/integration/local_toolchains/py_extension.bzl new file mode 100644 index 0000000000..5d37fd7824 --- /dev/null +++ b/tests/integration/local_toolchains/py_extension.bzl @@ -0,0 +1,154 @@ +# Copyright 2025 The Bazel Authors. All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +"""Macro to build a python C/C++ extension. + +There are variants of py_extension in many other projects, such as: +* https://github.com/protocolbuffers/protobuf/tree/main/python/py_extension.bzl +* https://github.com/google/riegeli/blob/master/python/riegeli/py_extension.bzl +* https://github.com/pybind/pybind11_bazel/blob/master/build_defs.bzl + +The issue for a generic verion is: +* https://github.com/bazel-contrib/rules_python/issues/824 +""" + +load("@bazel_skylib//rules:copy_file.bzl", "copy_file") +load("@rules_cc//cc:cc_binary.bzl", "cc_binary") +load("@rules_python//python:defs.bzl", "py_library") + +def py_extension( + *, + name, + deps = None, + linkopts = None, + imports = None, + visibility = None, + **kwargs): + """Creates a Python module implemented in C++. + + A Python extension has 2 essential parts: + 1. An internal shared object / pyd package for the extension, `name.pyd`/`name.so` + 2. The py_library target for the extension.` + + Python modules can depend on a py_extension. + + Args: + name: `str`. Name for this target. This is typically the module name. + deps: `list`. Required. C++ libraries to link into the module. + linkopts: `list`. Linking options for the shared library. + imports: `list`. Additional imports for the py_library rule. + visibility: `str`. Visibility for target. + **kwargs: Additional options for the cc_library rule. + """ + if not name: + fail("py_extension requires a name") + if not deps: + fail("py_extension requires a non-empty deps attribute") + if "linkshared" in kwargs: + fail("py_extension attribute linkshared not allowed") + + if not linkopts: + linkopts = [] + + testonly = kwargs.get("testonly") + tags = kwargs.pop("tags", []) + + cc_binary_so_name = name + ".so" + cc_binary_dll_name = name + ".dll" + cc_binary_pyd_name = name + ".pyd" + linker_script_name = name + ".lds" + linker_script_name_rule = name + "_lds" + shared_objects_name = name + "__shared_objects" + + # On Unix, restrict symbol visibility. + exported_symbol = "PyInit_" + name + + # Generate linker script used on non-macOS unix platforms. + native.genrule( + name = linker_script_name_rule, + outs = [linker_script_name], + cmd = "\n".join([ + "cat <<'EOF' >$@", + "{", + " global: " + exported_symbol + ";", + " local: *;", + "};", + "EOF", + ]), + ) + + for cc_binary_name in [cc_binary_dll_name, cc_binary_so_name]: + cur_linkopts = linkopts + cur_deps = deps + if cc_binary_name == cc_binary_so_name: + cur_linkopts = linkopts + select({ + "@platforms//os:macos": [ + # Avoid undefined symbol errors for CPython symbols that + # will be resolved at runtime. + "-undefined", + "dynamic_lookup", + # On macOS, the linker does not support version scripts. Use + # the `-exported_symbol` option instead to restrict symbol + # visibility. + "-Wl,-exported_symbol", + # On macOS, the symbol starts with an underscore. + "-Wl,_" + exported_symbol, + ], + # On non-macOS unix, use a version script to restrict symbol + # visibility. + "//conditions:default": [ + "-Wl,--version-script", + "-Wl,$(location :" + linker_script_name + ")", + ], + }) + cur_deps = cur_deps + select({ + "@platforms//os:macos": [], + "//conditions:default": [linker_script_name], + }) + + cc_binary( + name = cc_binary_name, + linkshared = True, + visibility = ["//visibility:private"], + deps = cur_deps, + tags = tags + ["manual"], + linkopts = cur_linkopts, + **kwargs + ) + + copy_file( + name = cc_binary_pyd_name + "__pyd_copy", + src = ":" + cc_binary_dll_name, + out = cc_binary_pyd_name, + visibility = visibility, + tags = ["manual"], + testonly = testonly, + ) + + native.filegroup( + name = shared_objects_name, + data = select({ + "@platforms//os:windows": [":" + cc_binary_pyd_name], + "//conditions:default": [":" + cc_binary_so_name], + }), + testonly = testonly, + ) + py_library( + name = name, + data = [":" + shared_objects_name], + imports = imports, + tags = tags, + testonly = testonly, + visibility = visibility, + ) diff --git a/tests/integration/local_toolchains/test.py b/tests/integration/local_toolchains/test.py index 8e37fff652..0a0d6bedeb 100644 --- a/tests/integration/local_toolchains/test.py +++ b/tests/integration/local_toolchains/test.py @@ -20,18 +20,20 @@ def test_python_from_path_used(self): # things like pyenv: they install a shim that re-execs python. # The shim is e.g. /home/user/.pyenv/shims/python3, which then # runs e.g. /usr/bin/python3 - with tempfile.NamedTemporaryFile(suffix="_info.py", mode="w+") as f: - f.write( + with tempfile.TemporaryDirectory() as temp_dir: + file_path = os.path.join(temp_dir, "info.py") + with open(file_path, 'w') as f: + f.write( """ import sys print(sys.executable) print(sys._base_executable) """ - ) - f.flush() + ) + f.flush() output_lines = ( subprocess.check_output( - [shell_path, f.name], + [shell_path, file_path], text=True, ) .strip() From 6b5ecf76cbbc69fd2d10ce5d3019292306f77c3f Mon Sep 17 00:00:00 2001 From: Brandon Chinn Date: Fri, 15 Aug 2025 14:38:00 -0700 Subject: [PATCH 382/922] docs: Fix docs for gazelle usage (#3182) --- gazelle/docs/installation_and_usage.md | 24 ++++++++++++------------ 1 file changed, 12 insertions(+), 12 deletions(-) diff --git a/gazelle/docs/installation_and_usage.md b/gazelle/docs/installation_and_usage.md index 1858f41bb9..b151ade25e 100644 --- a/gazelle/docs/installation_and_usage.md +++ b/gazelle/docs/installation_and_usage.md @@ -86,6 +86,16 @@ load("@rules_python_gazelle_plugin//modules_mapping:def.bzl", "modules_mapping") modules_mapping( name = "modules_map", wheels = all_whl_requirements, + + # include_stub_packages: bool (default: False) + # If set to True, this flag automatically includes any corresponding type stub packages + # for the third-party libraries that are present and used. For example, if you have + # `boto3` as a dependency, and this flag is enabled, the corresponding `boto3-stubs` + # package will be automatically included in the BUILD file. + # Enabling this feature helps ensure that type hints and stubs are readily available + # for tools like type checkers and IDEs, improving the development experience and + # reducing manual overhead in managing separate stub packages. + include_stub_packages = True, ) # Gazelle python extension needs a manifest file mapping from @@ -110,16 +120,6 @@ gazelle_python_manifest( # the integrity field is not added to the manifest which can help avoid # merge conflicts in large repos. requirements = "//:requirements_lock.txt", - - # include_stub_packages: bool (default: False) - # If set to True, this flag automatically includes any corresponding type stub packages - # for the third-party libraries that are present and used. For example, if you have - # `boto3` as a dependency, and this flag is enabled, the corresponding `boto3-stubs` - # package will be automatically included in the BUILD file. - # Enabling this feature helps ensure that type hints and stubs are readily available - # for tools like type checkers and IDEs, improving the development experience and - # reducing manual overhead in managing separate stub packages. - include_stub_packages = True ) ``` @@ -134,9 +134,9 @@ gazelle_binary( name = "gazelle_multilang", languages = [ # List of language plugins. - # If you want to generate py_proto_library targets PR #3057), then + # If you want to generate py_proto_library targets (PR #3057), then # the proto language plugin _must_ come before the rules_python plugin. - #"@bazel_gazelle//lanugage/proto", + #"@bazel_gazelle//language/proto", "@rules_python_gazelle_plugin//python", ], ) From eb37df0fd0ad3ad0767702cd1ec64a4a206a4996 Mon Sep 17 00:00:00 2001 From: Ignas Anikevicius <240938+aignas@users.noreply.github.com> Date: Sat, 16 Aug 2025 11:45:00 +0900 Subject: [PATCH 383/922] feat(pypi): incrementally build platform configuration (#3112) Before this PR the configuration for platforms would be built non-incrementally, making it harder for users to override particular attributes of the already configured ones. With this PR the new features introduced in #3058 will be easier to override. Work towards #2747 --- python/private/pypi/extension.bzl | 113 ++++++++++++++--------- tests/pypi/extension/extension_tests.bzl | 62 ++++++++++++- 2 files changed, 132 insertions(+), 43 deletions(-) diff --git a/python/private/pypi/extension.bzl b/python/private/pypi/extension.bzl index 08e1af4d81..59e77d13e4 100644 --- a/python/private/pypi/extension.bzl +++ b/python/private/pypi/extension.bzl @@ -377,26 +377,80 @@ def _whl_repo(*, src, whl_library_args, is_multiple_versions, download_only, net ), ) -def _configure(config, *, platform, os_name, arch_name, config_settings, env = {}, override = False): - """Set the value in the config if the value is provided""" - config.setdefault("platforms", {}) - if platform: - if not override and config.get("platforms", {}).get(platform): - return +def _plat(*, name, arch_name, os_name, config_settings = [], env = {}): + return struct( + name = name, + arch_name = arch_name, + os_name = os_name, + config_settings = config_settings, + env = env, + ) +def _configure(config, *, override = False, **kwargs): + """Set the value in the config if the value is provided""" + env = kwargs.get("env") + if env: for key in env: if key not in _SUPPORTED_PEP508_KEYS: fail("Unsupported key in the PEP508 environment: {}".format(key)) - config["platforms"][platform] = struct( - name = platform.replace("-", "_").lower(), - os_name = os_name, - arch_name = arch_name, - config_settings = config_settings, - env = env, - ) - else: - config["platforms"].pop(platform) + for key, value in kwargs.items(): + if value and (override or key not in config): + config[key] = value + +def build_config( + *, + module_ctx, + enable_pipstar): + """Parse 'configure' and 'default' extension tags + + Args: + module_ctx: {type}`module_ctx` module context. + enable_pipstar: {type}`bool` a flag to enable dropping Python dependency for + evaluation of the extension. + + Returns: + A struct with the configuration. + """ + defaults = { + "platforms": {}, + } + for mod in module_ctx.modules: + if not (mod.is_root or mod.name == "rules_python"): + continue + + for tag in mod.tags.default: + platform = tag.platform + if platform: + specific_config = defaults["platforms"].setdefault(platform, {}) + _configure( + specific_config, + arch_name = tag.arch_name, + config_settings = tag.config_settings, + env = tag.env, + os_name = tag.os_name, + name = platform.replace("-", "_").lower(), + override = mod.is_root, + ) + + if platform and not (tag.arch_name or tag.config_settings or tag.env or tag.os_name): + defaults["platforms"].pop(platform) + + # TODO @aignas 2025-05-19: add more attr groups: + # * for AUTH - the default `netrc` usage could be configured through a common + # attribute. + # * for index/downloader config. This includes all of those attributes for + # overrides, etc. Index overrides per platform could be also used here. + # * for whl selection - selecting preferences of which `platform_tag`s we should use + # for what. We could also model the `cp313t` freethreaded as separate platforms. + + return struct( + platforms = { + name: _plat(**values) + for name, values in defaults["platforms"].items() + }, + enable_pipstar = enable_pipstar, + ) def parse_modules( module_ctx, @@ -448,33 +502,7 @@ You cannot use both the additive_build_content and additive_build_content_file a srcs_exclude_glob = whl_mod.srcs_exclude_glob, ) - defaults = { - "enable_pipstar": enable_pipstar, - "platforms": {}, - } - for mod in module_ctx.modules: - if not (mod.is_root or mod.name == "rules_python"): - continue - - for tag in mod.tags.default: - _configure( - defaults, - arch_name = tag.arch_name, - config_settings = tag.config_settings, - env = tag.env, - os_name = tag.os_name, - platform = tag.platform, - override = mod.is_root, - # TODO @aignas 2025-05-19: add more attr groups: - # * for AUTH - the default `netrc` usage could be configured through a common - # attribute. - # * for index/downloader config. This includes all of those attributes for - # overrides, etc. Index overrides per platform could be also used here. - # * for whl selection - selecting preferences of which `platform_tag`s we should use - # for what. We could also model the `cp313t` freethreaded as separate platforms. - ) - - config = struct(**defaults) + config = build_config(module_ctx = module_ctx, enable_pipstar = enable_pipstar) # TODO @aignas 2025-06-03: Merge override API with the builder? _overriden_whl_set = {} @@ -659,6 +687,7 @@ You cannot use both the additive_build_content and additive_build_content_file a k: dict(sorted(args.items())) for k, args in sorted(whl_libraries.items()) }, + config = config, ) def _pip_impl(module_ctx): diff --git a/tests/pypi/extension/extension_tests.bzl b/tests/pypi/extension/extension_tests.bzl index 4949c0df85..d115546b63 100644 --- a/tests/pypi/extension/extension_tests.bzl +++ b/tests/pypi/extension/extension_tests.bzl @@ -16,7 +16,7 @@ load("@rules_testing//lib:test_suite.bzl", "test_suite") load("@rules_testing//lib:truth.bzl", "subjects") -load("//python/private/pypi:extension.bzl", "parse_modules") # buildifier: disable=bzl-visibility +load("//python/private/pypi:extension.bzl", "build_config", "parse_modules") # buildifier: disable=bzl-visibility load("//python/private/pypi:parse_simpleapi_html.bzl", "parse_simpleapi_html") # buildifier: disable=bzl-visibility load("//python/private/pypi:whl_config_setting.bzl", "whl_config_setting") # buildifier: disable=bzl-visibility @@ -92,6 +92,18 @@ def _parse_modules(env, enable_pipstar = 0, **kwargs): ), ) +def _build_config(env, enable_pipstar = 0, **kwargs): + return env.expect.that_struct( + build_config( + enable_pipstar = enable_pipstar, + **kwargs + ), + attrs = dict( + platforms = subjects.dict, + enable_pipstar = subjects.bool, + ), + ) + def _default( arch_name = None, config_settings = None, @@ -1206,6 +1218,54 @@ optimum[onnxruntime-gpu]==1.17.1 ; sys_platform == 'linux' _tests.append(_test_pipstar_platforms) +def _test_build_pipstar_platform(env): + config = _build_config( + env, + module_ctx = _mock_mctx( + _mod( + name = "rules_python", + default = [ + _default( + platform = "myplat", + os_name = "linux", + arch_name = "x86_64", + config_settings = [ + "@platforms//os:linux", + "@platforms//cpu:x86_64", + ], + ), + _default(), + _default( + platform = "myplat2", + os_name = "linux", + arch_name = "x86_64", + config_settings = [ + "@platforms//os:linux", + "@platforms//cpu:x86_64", + ], + ), + _default(platform = "myplat2"), + ], + ), + ), + enable_pipstar = True, + ) + config.enable_pipstar().equals(True) + config.platforms().contains_exactly({ + "myplat": struct( + name = "myplat", + os_name = "linux", + arch_name = "x86_64", + config_settings = [ + "@platforms//os:linux", + "@platforms//cpu:x86_64", + ], + env = {}, + ), + }) + +_tests.append(_test_build_pipstar_platform) + def extension_test_suite(name): """Create the test suite. From cda58775c6fb1bfba93b3bbc55e8ce003a56960b Mon Sep 17 00:00:00 2001 From: Laramie Leavitt Date: Fri, 15 Aug 2025 21:02:03 -0700 Subject: [PATCH 384/922] fix(local_runtime): Search for libs in sys._base_executable when available. (#3178) Search directory for libraries should look in the same directory as sys._base_executable. Since sys._base_executable may be unset, fallback to sys.executable Found this when trying to build using a venv for [tensorstore](https://github.com/google/tensorstore) on Windows: * Github CI uses nuget to download Python. * Build sets up a Python venv. The venv does not include all the lib directories required to link an extension. Fixes https://github.com/bazel-contrib/rules_python/issues/3172 --------- Co-authored-by: Richard Levasseur --- python/private/get_local_runtime_info.py | 55 ++++++++++++++++-------- 1 file changed, 37 insertions(+), 18 deletions(-) diff --git a/python/private/get_local_runtime_info.py b/python/private/get_local_runtime_info.py index ff3b0aeb01..d176b1a7c6 100644 --- a/python/private/get_local_runtime_info.py +++ b/python/private/get_local_runtime_info.py @@ -23,7 +23,7 @@ _IS_DARWIN = sys.platform == "darwin" -def _search_directories(get_config): +def _search_directories(get_config, base_executable): """Returns a list of library directories to search for shared libraries.""" # There's several types of libraries with different names and a plethora # of settings, and many different config variables to check: @@ -53,19 +53,23 @@ def _search_directories(get_config): if config_value and not config_value.endswith(multiarch): lib_dirs.append(os.path.join(config_value, multiarch)) - if _IS_WINDOWS: - # On Windows DLLs go in the same directory as the executable, while .lib - # files live in the lib/ or libs/ subdirectory. - lib_dirs.append(get_config("BINDIR")) - lib_dirs.append(os.path.join(os.path.dirname(sys.executable))) - lib_dirs.append(os.path.join(os.path.dirname(sys.executable), "lib")) - lib_dirs.append(os.path.join(os.path.dirname(sys.executable), "libs")) - elif not _IS_DARWIN: - # On most systems the executable is in a bin/ directory and the libraries - # are in a sibling lib/ directory. - lib_dirs.append( - os.path.join(os.path.dirname(os.path.dirname(sys.executable)), "lib") - ) + if not _IS_DARWIN: + for exec_dir in ( + os.path.dirname(base_executable) if base_executable else None, + get_config("BINDIR"), + ): + if not exec_dir: + continue + if _IS_WINDOWS: + # On Windows DLLs go in the same directory as the executable, while .lib + # files live in the lib/ or libs/ subdirectory. + lib_dirs.append(exec_dir) + lib_dirs.append(os.path.join(exec_dir, "lib")) + lib_dirs.append(os.path.join(exec_dir, "libs")) + else: + # On most systems the executable is in a bin/ directory and the libraries + # are in a sibling lib/ directory. + lib_dirs.append(os.path.join(os.path.dirname(exec_dir), "lib")) # Dedup and remove empty values, keeping the order. lib_dirs = [v for v in lib_dirs if v] @@ -126,7 +130,7 @@ def _search_library_names(get_config): return {k: None for k in lib_names}.keys() -def _get_python_library_info(): +def _get_python_library_info(base_executable): """Returns a dictionary with the static and dynamic python libraries.""" config_vars = sysconfig.get_config_vars() @@ -140,7 +144,7 @@ def _get_python_library_info(): f"{sys.version_info.major}.{sys.version_info.minor}" ) - search_directories = _search_directories(config_vars.get) + search_directories = _search_directories(config_vars.get, base_executable) search_libnames = _search_library_names(config_vars.get) def _add_if_exists(target, path): @@ -187,13 +191,28 @@ def _add_if_exists(target, path): } +def _get_base_executable(): + """Returns the base executable path.""" + try: + if sys._base_executable: # pylint: disable=protected-access + return sys._base_executable # pylint: disable=protected-access + except AttributeError: + # Bug reports indicate sys._base_executable doesn't exist in some cases, + # but it's not clear why. + # See https://github.com/bazel-contrib/rules_python/issues/3172 + pass + # The normal sys.executable is the next-best guess if sys._base_executable + # is missing. + return sys.executable + + data = { "major": sys.version_info.major, "minor": sys.version_info.minor, "micro": sys.version_info.micro, "include": sysconfig.get_path("include"), "implementation_name": sys.implementation.name, - "base_executable": sys._base_executable, + "base_executable": _get_base_executable(), } -data.update(_get_python_library_info()) +data.update(_get_python_library_info(_get_base_executable())) print(json.dumps(data)) From d48286fdd0125cfae313bf782a58c54742444a77 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Sat, 16 Aug 2025 18:35:18 -0700 Subject: [PATCH 385/922] docs: tell how to do and request patch releases/backports (#3185) We've had several backport requests of late. It's tenable for users to do the more time consuming working of backporting the code, so give steps on how to do so. --- .../ISSUE_TEMPLATE/patch_release_request.md | 22 +++++++++++++ docs/devguide.md | 32 +++++++++++++++++++ docs/support.md | 16 ++++++++++ 3 files changed, 70 insertions(+) create mode 100644 .github/ISSUE_TEMPLATE/patch_release_request.md diff --git a/.github/ISSUE_TEMPLATE/patch_release_request.md b/.github/ISSUE_TEMPLATE/patch_release_request.md new file mode 100644 index 0000000000..7b38ec40df --- /dev/null +++ b/.github/ISSUE_TEMPLATE/patch_release_request.md @@ -0,0 +1,22 @@ +--- +name: "🏗️ Patch release or backport" +about: Request a patch release or backport of a fix to a release. +title: 'Patch release: MAJOR.MINOR.PATCH' +labels: 'type: process' +--- + + + +**What version of `rules_python` do you want to patch?** + + +**What pull requests do you want to backport?** + +Please provide a list of pull request numbers. + +- # +- # diff --git a/docs/devguide.md b/docs/devguide.md index 43120bf2a1..afb990588b 100644 --- a/docs/devguide.md +++ b/docs/devguide.md @@ -116,3 +116,35 @@ to have everything self-documented, we have a special target, `//private:requirements.update`, which uses `rules_multirun` to run all of the requirement-updating scripts in sequence in one go. This can be done once per release as we prepare for releases. + +## Creating Backport PRs + +The steps to create a backport PR are: + +1. Create an issue for the patch release; use the [patch relase + template][patch-release-issue]. +2. Create a fork of `rules_python`. +3. Checkout the `release/X.Y` branch. +4. Use `git cherry-pick -x` to cherry pick the desired fixes. +5. Update the release's `CHANGELOG.md` file: + * Add a Major.Minor.Patch section if one doesn't exist + * Copy the changelog text from `main` to the release's changelog. +6. Send a PR with the backport's changes. + * The title should be `backport: PR#N to Major.Minor` + * The body must preserve the original PR's number, commit hash, description, + and authorship. + Use the following format (`git cherry-pick` will use this format): + ``` + + + + (cherry picked from commit ) + ----- + Co-authored-by: + ``` + * If the PR contains multiple backport commits, separate each's description + with `-----`. +7. Send a PR to update the `main` branch's `CHANGELOG.md` to reflect the + changes done in the patched release. + +[patch-release-issue]: https://github.com/bazelbuild/rules_python/issues/new?template=patch_release.md diff --git a/docs/support.md b/docs/support.md index ad943b3845..8728540804 100644 --- a/docs/support.md +++ b/docs/support.md @@ -14,6 +14,22 @@ the willingness of volunteers. If you want or need particular functionality backported, then the best way is to open a PR to demonstrate the feasibility of the backport. +### Backports and Patch Releases + +Backports and patch releases are provided on a best-effort basis. Only fixes are +backported. Features are not backported. + +Backports can be done to older releases, but only if newer releases also have +the fix backported. For example, if the current release is 1.5, in order to +patch 1.4, version 1.5 must be patched first. + +Backports can be requested by [creating an issue with the patch release +template][patch-release-issue] or by sending a pull request performing the backport. +See the dev guide for [how to create a backport PR][backport-pr]. + +[patch-release-issue]: https://github.com/bazelbuild/rules_python/issues/new?template=patch_release_request.md +[backport-pr]: devguide.html#creating-backport-prs + ## Supported Bazel Versions The supported Bazel versions are: From 8c33aa6d64898fcf4513fae0de5982903a3b8ee8 Mon Sep 17 00:00:00 2001 From: Ignas Anikevicius <240938+aignas@users.noreply.github.com> Date: Sun, 17 Aug 2025 10:36:23 +0900 Subject: [PATCH 386/922] fix(pypi): pull fewer wheels with experimental_index_url (#3058) Before this we would pull all of the wheels that the user target configuration would be compatible with and that meant that it was not customizable. This also meant that there were a lot of footguns in the configuration where the select statements were not really foolproof. With this PR we select only those sources that need to be for the declared configurations. Freethreaded support should be done by defining extra freethreaded platforms using the new builder API. It is done as a followup in #3063. This is also changing the default platforms to be only the fully supported platforms. This makes the testing easier and avoids us running into compatibility issues during the rollout. Work towards #2747 Fixes #2759 Fixes #2849 --- CHANGELOG.md | 8 + MODULE.bazel | 69 ++-- examples/bzlmod/MODULE.bazel | 17 + python/private/pypi/BUILD.bazel | 5 +- python/private/pypi/evaluate_markers.bzl | 6 +- python/private/pypi/extension.bzl | 242 ++++++++++---- python/private/pypi/parse_requirements.bzl | 94 +++--- python/private/pypi/whl_target_platforms.bzl | 132 -------- tests/pypi/extension/extension_tests.bzl | 157 ++++----- .../parse_requirements_tests.bzl | 159 +++++++-- tests/pypi/whl_target_platforms/BUILD.bazel | 3 - .../whl_target_platforms/select_whl_tests.bzl | 314 ------------------ 12 files changed, 523 insertions(+), 683 deletions(-) delete mode 100644 tests/pypi/whl_target_platforms/select_whl_tests.bzl diff --git a/CHANGELOG.md b/CHANGELOG.md index 2dc235fbf6..9d45aa53c3 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -69,6 +69,10 @@ END_UNRELEASED_TEMPLATE * (toolchain) Python 3.13 now references 3.13.6 * (gazelle) Switched back to smacker/go-tree-sitter, fixing [#2630](https://github.com/bazel-contrib/rules_python/issues/2630) +* (pypi) From now on the list of default platforms only includes `linux_x86_64`, `linux_aarch64`, + `osx_x86_64`, `osx_aarch64` and `windows_x86_64`. If you are on other platforms, you need to + use the `pip.default` to configure it yourself. If you are interested in graduating the + platform, consider helping set us up CI for them and update the documentation. * (ci) We are now testing on Ubuntu 22.04 for RBE and non-RBE configurations. * (core) `#!/usr/bin/env bash` is now used as a shebang in the stage1 bootstrap template. * (gazelle:docs) The Gazelle docs have been migrated from {gh-path}`gazelle/README.md` to @@ -90,6 +94,10 @@ END_UNRELEASED_TEMPLATE ([#2503](https://github.com/bazel-contrib/rules_python/issues/2503)). * (pypi) The pipstar `defaults` configuration now supports any custom platform name. +* (pypi) The selection of the whls has been changed and should no longer result + in ambiguous select matches ({gh-issue}`2759`) and should be much more efficient + when running `bazel query` due to fewer repositories being included + ({gh-issue}`2849`). * Multi-line python imports (e.g. with escaped newlines) are now correctly processed by Gazelle. * (toolchains) `local_runtime_repo` works with multiarch Debian with Python 3.8 ([#3099](https://github.com/bazel-contrib/rules_python/issues/3099)). diff --git a/MODULE.bazel b/MODULE.bazel index b8d8c16a0c..66297b99a1 100644 --- a/MODULE.bazel +++ b/MODULE.bazel @@ -74,16 +74,18 @@ pip = use_extension("//python/extensions:pip.bzl", "pip") env = {"platform_version": "0"}, os_name = "linux", platform = "linux_{}".format(cpu), + whl_abi_tags = [ + "abi3", + "cp{major}{minor}", + ], + whl_platform_tags = [ + "linux_{}".format(cpu), + "manylinux_*_{}".format(cpu), + ], ) for cpu in [ "x86_64", "aarch64", - # TODO @aignas 2025-05-19: only leave tier 0-1 cpus when stabilizing the - # `pip.default` extension. i.e. drop the below values - users will have to - # define themselves if they need them. - "arm", - "ppc", - "s390x", ] ] @@ -99,26 +101,53 @@ pip = use_extension("//python/extensions:pip.bzl", "pip") env = {"platform_version": "14.0"}, os_name = "osx", platform = "osx_{}".format(cpu), + whl_abi_tags = [ + "abi3", + "cp{major}{minor}", + ], + whl_platform_tags = [ + "macosx_*_{}".format(suffix) + for suffix in platform_tag_cpus + ], ) - for cpu in [ - "aarch64", - "x86_64", - ] + for cpu, platform_tag_cpus in { + "aarch64": [ + "universal2", + "arm64", + ], + "x86_64": [ + "universal2", + "x86_64", + ], + }.items() +] + +[ + pip.default( + arch_name = cpu, + config_settings = [ + "@platforms//cpu:{}".format(cpu), + "@platforms//os:windows", + ], + env = {"platform_version": "0"}, + os_name = "windows", + platform = "windows_{}".format(cpu), + whl_abi_tags = [ + "abi3", + "cp{major}{minor}", + ], + whl_platform_tags = whl_platform_tags, + ) + for cpu, whl_platform_tags in { + "x86_64": ["win_amd64"], + }.items() ] -pip.default( - arch_name = "x86_64", - config_settings = [ - "@platforms//cpu:x86_64", - "@platforms//os:windows", - ], - env = {"platform_version": "0"}, - os_name = "windows", - platform = "windows_x86_64", -) pip.parse( # NOTE @aignas 2024-10-26: We have an integration test that depends on us # being able to build sdists for this hub, so explicitly set this to False. + # + # how do we test sdists? Maybe just worth adding a single sdist somewhere? download_only = False, experimental_index_url = "https://pypi.org/simple", hub_name = "rules_python_publish_deps", diff --git a/examples/bzlmod/MODULE.bazel b/examples/bzlmod/MODULE.bazel index 841c096dcf..95e1090f53 100644 --- a/examples/bzlmod/MODULE.bazel +++ b/examples/bzlmod/MODULE.bazel @@ -158,6 +158,23 @@ pip.whl_mods( ) use_repo(pip, "whl_mods_hub") +# Because below we are using `windows_aarch64` platform, we have to define various +# properties for it. +pip.default( + arch_name = "aarch64", + config_settings = [ + "@platforms//os:windows", + "@platforms//cpu:aarch64", + ], + env = { + "platform_version": "0", + }, + os_name = "windows", + platform = "windows_aarch64", + whl_abi_tags = [], # default to all ABIs + whl_platform_tags = ["win_amd64"], +) + # To fetch pip dependencies, use pip.parse. We can pass in various options, # but typically we pass requirements and the Python version. The Python # version must have been configured by a corresponding `python.toolchain()` diff --git a/python/private/pypi/BUILD.bazel b/python/private/pypi/BUILD.bazel index 4b56b73284..847c85d634 100644 --- a/python/private/pypi/BUILD.bazel +++ b/python/private/pypi/BUILD.bazel @@ -116,11 +116,11 @@ bzl_library( ":parse_whl_name_bzl", ":pep508_env_bzl", ":pip_repository_attrs_bzl", + ":python_tag_bzl", ":simpleapi_download_bzl", ":whl_config_setting_bzl", ":whl_library_bzl", ":whl_repo_name_bzl", - ":whl_target_platforms_bzl", "//python/private:full_version_bzl", "//python/private:normalize_name_bzl", "//python/private:version_bzl", @@ -209,7 +209,7 @@ bzl_library( ":parse_requirements_txt_bzl", ":pypi_repo_utils_bzl", ":requirements_files_by_platform_bzl", - ":whl_target_platforms_bzl", + ":select_whl_bzl", "//python/private:normalize_name_bzl", "//python/private:repo_utils_bzl", ], @@ -438,5 +438,4 @@ bzl_library( bzl_library( name = "whl_target_platforms_bzl", srcs = ["whl_target_platforms.bzl"], - deps = [":parse_whl_name_bzl"], ) diff --git a/python/private/pypi/evaluate_markers.bzl b/python/private/pypi/evaluate_markers.bzl index 6167cdbc96..4d6a39a1df 100644 --- a/python/private/pypi/evaluate_markers.bzl +++ b/python/private/pypi/evaluate_markers.bzl @@ -43,11 +43,11 @@ def evaluate_markers(*, requirements, platforms): for req_string, platform_strings in requirements.items(): req = requirement(req_string) for platform_str in platform_strings: - env = platforms.get(platform_str) - if not env: + plat = platforms.get(platform_str) + if not plat: fail("Please define platform: '{}'".format(platform_str)) - if evaluate(req.marker, env = env): + if evaluate(req.marker, env = plat.env): ret.setdefault(req_string, []).append(platform_str) return ret diff --git a/python/private/pypi/extension.bzl b/python/private/pypi/extension.bzl index 59e77d13e4..2c7aa8a0e5 100644 --- a/python/private/pypi/extension.bzl +++ b/python/private/pypi/extension.bzl @@ -31,6 +31,7 @@ load(":parse_requirements.bzl", "parse_requirements") load(":parse_whl_name.bzl", "parse_whl_name") load(":pep508_env.bzl", "env") load(":pip_repository_attrs.bzl", "ATTRS") +load(":python_tag.bzl", "python_tag") load(":requirements_files_by_platform.bzl", "requirements_files_by_platform") load(":simpleapi_download.bzl", "simpleapi_download") load(":whl_config_setting.bzl", "whl_config_setting") @@ -68,19 +69,40 @@ def _whl_mods_impl(whl_mods_dict): def _platforms(*, python_version, minor_mapping, config): platforms = {} - python_version = full_version( - version = python_version, - minor_mapping = minor_mapping, + python_version = version.parse( + full_version( + version = python_version, + minor_mapping = minor_mapping, + ), + strict = True, ) - abi = "cp3{}".format(python_version[2:]) for platform, values in config.platforms.items(): - key = "{}_{}".format(abi, platform) - platforms[key] = env( - env = values.env, - os = values.os_name, - arch = values.arch_name, - python_version = python_version, + # TODO @aignas 2025-07-07: this is probably doing the parsing of the version too + # many times. + key = "{}{}{}.{}_{}".format( + python_tag(values.env["implementation_name"]), + python_version.release[0], + python_version.release[1], + python_version.release[2], + platform, + ) + + platforms[key] = struct( + env = env( + env = values.env, + os = values.os_name, + arch = values.arch_name, + python_version = python_version.string, + ), + whl_abi_tags = [ + v.format( + major = python_version.release[0], + minor = python_version.release[1], + ) + for v in values.whl_abi_tags + ], + whl_platform_tags = values.whl_platform_tags, ) return platforms @@ -153,6 +175,8 @@ def _create_whl_repos( )) python_interpreter_target = available_interpreters[python_name] + # TODO @aignas 2025-06-29: we should not need the version in the pip_name if + # we are using pipstar and we are downloading the wheel using the downloader pip_name = "{}_{}".format( hub_name, version_label(pip_attr.python_version), @@ -231,12 +255,21 @@ def _create_whl_repos( ), logger = logger, ), + platforms = _platforms( + python_version = pip_attr.python_version, + minor_mapping = minor_mapping, + config = config, + ), extra_pip_args = pip_attr.extra_pip_args, get_index_urls = get_index_urls, evaluate_markers = evaluate_markers, logger = logger, ) + use_downloader = { + normalize_name(s): False + for s in pip_attr.simpleapi_skip + } exposed_packages = {} for whl in requirements_by_platform: if whl.is_exposed: @@ -290,11 +323,20 @@ def _create_whl_repos( whl_library_args = whl_library_args, download_only = pip_attr.download_only, netrc = pip_attr.netrc, + use_downloader = use_downloader.get( + whl.name, + get_index_urls != None, # defaults to True if the get_index_urls is defined + ), auth_patterns = pip_attr.auth_patterns, python_version = major_minor, is_multiple_versions = whl.is_multiple_versions, enable_pipstar = config.enable_pipstar, ) + if repo == None: + # NOTE @aignas 2025-07-07: we guard against an edge-case where there + # are more platforms defined than there are wheels for and users + # disallow building from sdist. + continue repo_name = "{}_{}".format(pip_name, repo.repo_name) if repo_name in whl_libraries: @@ -313,7 +355,17 @@ def _create_whl_repos( whl_libraries = whl_libraries, ) -def _whl_repo(*, src, whl_library_args, is_multiple_versions, download_only, netrc, auth_patterns, python_version, enable_pipstar = False): +def _whl_repo( + *, + src, + whl_library_args, + is_multiple_versions, + download_only, + netrc, + auth_patterns, + python_version, + use_downloader, + enable_pipstar = False): args = dict(whl_library_args) args["requirement"] = src.requirement_line is_whl = src.filename.endswith(".whl") @@ -326,19 +378,24 @@ def _whl_repo(*, src, whl_library_args, is_multiple_versions, download_only, net args["extra_pip_args"] = src.extra_pip_args if not src.url or (not is_whl and download_only): - # Fallback to a pip-installed wheel - target_platforms = src.target_platforms if is_multiple_versions else [] - return struct( - repo_name = pypi_repo_name( - normalize_name(src.distribution), - *target_platforms - ), - args = args, - config_setting = whl_config_setting( - version = python_version, - target_platforms = target_platforms or None, - ), - ) + if download_only and use_downloader: + # If the user did not allow using sdists and we are using the downloader + # and we are not using simpleapi_skip for this + return None + else: + # Fallback to a pip-installed wheel + target_platforms = src.target_platforms if is_multiple_versions else [] + return struct( + repo_name = pypi_repo_name( + normalize_name(src.distribution), + *target_platforms + ), + args = args, + config_setting = whl_config_setting( + version = python_version, + target_platforms = target_platforms or None, + ), + ) # This is no-op because pip is not used to download the wheel. args.pop("download_only", None) @@ -360,30 +417,37 @@ def _whl_repo(*, src, whl_library_args, is_multiple_versions, download_only, net for p in src.target_platforms ] - # Pure python wheels or sdists may need to have a platform here - target_platforms = None - if is_whl and not src.filename.endswith("-any.whl"): - pass - elif is_multiple_versions: - target_platforms = src.target_platforms - return struct( repo_name = whl_repo_name(src.filename, src.sha256), args = args, config_setting = whl_config_setting( version = python_version, - filename = src.filename, - target_platforms = target_platforms, + target_platforms = src.target_platforms, ), ) -def _plat(*, name, arch_name, os_name, config_settings = [], env = {}): +def _plat(*, name, arch_name, os_name, config_settings = [], env = {}, whl_abi_tags = [], whl_platform_tags = []): + # NOTE @aignas 2025-07-08: the least preferred is the first item in the list + if "any" not in whl_platform_tags: + # the lowest priority one needs to be the first one + whl_platform_tags = ["any"] + whl_platform_tags + + whl_abi_tags = whl_abi_tags or ["abi3", "cp{major}{minor}"] + if "none" not in whl_abi_tags: + # the lowest priority one needs to be the first one + whl_abi_tags = ["none"] + whl_abi_tags + return struct( name = name, arch_name = arch_name, os_name = os_name, config_settings = config_settings, - env = env, + env = { + # defaults for env + "implementation_name": "cpython", + } | env, + whl_abi_tags = whl_abi_tags, + whl_platform_tags = whl_platform_tags, ) def _configure(config, *, override = False, **kwargs): @@ -430,10 +494,12 @@ def build_config( env = tag.env, os_name = tag.os_name, name = platform.replace("-", "_").lower(), + whl_abi_tags = tag.whl_abi_tags, + whl_platform_tags = tag.whl_platform_tags, override = mod.is_root, ) - if platform and not (tag.arch_name or tag.config_settings or tag.env or tag.os_name): + if platform and not (tag.arch_name or tag.config_settings or tag.env or tag.os_name or tag.whl_abi_tags or tag.whl_platform_tags): defaults["platforms"].pop(platform) # TODO @aignas 2025-05-19: add more attr groups: @@ -441,8 +507,6 @@ def build_config( # attribute. # * for index/downloader config. This includes all of those attributes for # overrides, etc. Index overrides per platform could be also used here. - # * for whl selection - selecting preferences of which `platform_tag`s we should use - # for what. We could also model the `cp313t` freethreaded as separate platforms. return struct( platforms = { @@ -630,6 +694,7 @@ You cannot use both the additive_build_content and additive_build_content_file a extra_aliases.setdefault(hub_name, {}) for whl_name, aliases in out.extra_aliases.items(): extra_aliases[hub_name].setdefault(whl_name, {}).update(aliases) + if hub_name not in exposed_packages: exposed_packages[hub_name] = out.exposed_packages else: @@ -640,6 +705,14 @@ You cannot use both the additive_build_content and additive_build_content_file a intersection[pkg] = None exposed_packages[hub_name] = intersection whl_libraries.update(out.whl_libraries) + for whl_name, lib in out.whl_libraries.items(): + if enable_pipstar: + whl_libraries.setdefault(whl_name, lib) + elif whl_name in lib: + fail("'{}' already in created".format(whl_name)) + else: + # replicate whl_libraries.update(out.whl_libraries) + whl_libraries[whl_name] = lib # TODO @aignas 2024-04-05: how do we support different requirement # cycles for different abis/oses? For now we will need the users to @@ -790,6 +863,7 @@ _default_attrs = { "arch_name": attr.string( doc = """\ The CPU architecture name to be used. +You can use any cpu name from the `@platforms//cpu:` package. :::{note} Either this or {attr}`env` `platform_machine` key should be specified. @@ -803,25 +877,6 @@ The list of labels to `config_setting` targets that need to be matched for the p selected. """, ), - "os_name": attr.string( - doc = """\ -The OS name to be used. - -:::{note} -Either this or the appropriate `env` keys should be specified. -::: -""", - ), - "platform": attr.string( - doc = """\ -A platform identifier which will be used as the unique identifier within the extension evaluation. -If you are defining custom platforms in your project and don't want things to clash, use extension -[isolation] feature. - -[isolation]: https://bazel.build/rules/lib/globals/module#use_extension.isolate -""", - ), -} | { "env": attr.string_dict( doc = """\ The values to use for environment markers when evaluating an expression. @@ -847,6 +902,79 @@ This is only used if the {envvar}`RULES_PYTHON_ENABLE_PIPSTAR` is enabled. """, ), # The values for PEP508 env marker evaluation during the lock file parsing + "os_name": attr.string( + doc = """\ +The OS name to be used. +You can use any OS name from the `@platforms//os:` package. + +:::{note} +Either this or the appropriate `env` keys should be specified. +::: +""", + ), + "platform": attr.string( + doc = """\ +A platform identifier which will be used as the unique identifier within the extension evaluation. +If you are defining custom platforms in your project and don't want things to clash, use extension +[isolation] feature. + +[isolation]: https://bazel.build/rules/lib/globals/module#use_extension.isolate +""", + ), + "whl_abi_tags": attr.string_list( + doc = """\ +A list of ABIs to select wheels for. The values can be either strings or include template +parameters like `{major}` and `{minor}` which will be replaced with python version parts. e.g. +`cp{major}{minor}` will result in `cp313` given the full python version is `3.13.5`. +Will always include `"none"` even if it is not specified. + +:::{note} +We select a single wheel and the last match will take precedence. +::: + +:::{seealso} +See official [docs](https://packaging.python.org/en/latest/specifications/platform-compatibility-tags/#abi-tag) for more information. +::: +""", + ), + "whl_platform_tags": attr.string_list( + doc = """\ +A list of `platform_tag` matchers so that we can select the best wheel based on the user +preference. +Will always include `"any"` even if it is not specified. + +The items in this list can contain a single `*` character that is equivalent to matching the +latest available version component in the platform_tag. Note, if the wheel platform tag does not +have a version component, e.g. `linux_x86_64` or `win_amd64`, then `*` will act as a regular +character. + +We will always select the highest available `platform_tag` version that is compatible with the +target platform. + +:::{note} +We select a single wheel and the last match will take precedence, if the platform_tag that we +match has a version component (e.g. `android_x_arch`, then the version `x` will be used in the +matching algorithm). + +If the matcher you provide has `*`, then we will match a wheel with the highest available target platform, i.e. if `musllinux_1_1_arch` and `musllinux_1_2_arch` are both present, then we will select `musllinux_1_2_arch`. +Otherwise we will select the highest available version that is equal or lower to the specifier, i.e. if `manylinux_2_12` and `manylinux_2_17` wheels are present and the matcher is `manylinux_2_15`, then we will match `manylinux_2_12` but not `manylinux_2_17`. +::: + +:::{note} +The following tag prefixes should be used instead of the legacy equivalents: +* `manylinux_2_5` instead of `manylinux1` +* `manylinux_2_12` instead of `manylinux2010` +* `manylinux_2_17` instead of `manylinux2014` + +When parsing the whl filenames `rules_python` will automatically transform wheel filenames to the +latest format. +::: + +:::{seealso} +See official [docs](https://packaging.python.org/en/latest/specifications/platform-compatibility-tags/#platform-tag) for more information. +::: +""", + ), } _SUPPORTED_PEP508_KEYS = [ diff --git a/python/private/pypi/parse_requirements.bzl b/python/private/pypi/parse_requirements.bzl index 9c610f11d3..ebd447d95d 100644 --- a/python/private/pypi/parse_requirements.bzl +++ b/python/private/pypi/parse_requirements.bzl @@ -31,13 +31,14 @@ load("//python/private:repo_utils.bzl", "repo_utils") load(":index_sources.bzl", "index_sources") load(":parse_requirements_txt.bzl", "parse_requirements_txt") load(":pep508_requirement.bzl", "requirement") -load(":whl_target_platforms.bzl", "select_whls") +load(":select_whl.bzl", "select_whl") def parse_requirements( ctx, *, requirements_by_platform = {}, extra_pip_args = [], + platforms = {}, get_index_urls = None, evaluate_markers = None, extract_url_srcs = True, @@ -46,6 +47,7 @@ def parse_requirements( Args: ctx: A context that has .read function that would read contents from a label. + platforms: The target platform descriptions. requirements_by_platform (label_keyed_string_dict): a way to have different package versions (or different packages) for different os, arch combinations. @@ -88,7 +90,7 @@ def parse_requirements( requirements = {} for file, plats in requirements_by_platform.items(): if logger: - logger.debug(lambda: "Using {} for {}".format(file, plats)) + logger.trace(lambda: "Using {} for {}".format(file, plats)) contents = ctx.read(file) # Parse the requirements file directly in starlark to get the information @@ -161,7 +163,7 @@ def parse_requirements( # VCS package references. env_marker_target_platforms = evaluate_markers(ctx, reqs_with_env_markers) if logger: - logger.debug(lambda: "Evaluated env markers from:\n{}\n\nTo:\n{}".format( + logger.trace(lambda: "Evaluated env markers from:\n{}\n\nTo:\n{}".format( reqs_with_env_markers, env_marker_target_platforms, )) @@ -196,6 +198,7 @@ def parse_requirements( name = name, reqs = reqs, index_urls = index_urls, + platforms = platforms, env_marker_target_platforms = env_marker_target_platforms, extract_url_srcs = extract_url_srcs, logger = logger, @@ -203,7 +206,7 @@ def parse_requirements( ) ret.append(item) if not item.is_exposed and logger: - logger.debug(lambda: "Package '{}' will not be exposed because it is only present on a subset of platforms: {} out of {}".format( + logger.trace(lambda: "Package '{}' will not be exposed because it is only present on a subset of platforms: {} out of {}".format( name, sorted(requirement_target_platforms), sorted(requirements), @@ -219,38 +222,43 @@ def _package_srcs( name, reqs, index_urls, + platforms, logger, env_marker_target_platforms, extract_url_srcs): """A function to return sources for a particular package.""" srcs = {} for r in sorted(reqs.values(), key = lambda r: r.requirement_line): - whls, sdist = _add_dists( - requirement = r, - index_urls = index_urls.get(name), - logger = logger, - ) - target_platforms = env_marker_target_platforms.get(r.requirement_line, r.target_platforms) - target_platforms = sorted(target_platforms) + extra_pip_args = tuple(r.extra_pip_args) - all_dists = [] + whls - if sdist: - all_dists.append(sdist) + for target_platform in target_platforms: + if platforms and target_platform not in platforms: + fail("The target platform '{}' could not be found in {}".format( + target_platform, + platforms.keys(), + )) - if extract_url_srcs and all_dists: - req_line = r.srcs.requirement - else: - all_dists = [struct( - url = "", - filename = "", - sha256 = "", - yanked = False, - )] - req_line = r.srcs.requirement_line + dist = _add_dists( + requirement = r, + target_platform = platforms.get(target_platform), + index_urls = index_urls.get(name), + logger = logger, + ) + if logger: + logger.debug(lambda: "The whl dist is: {}".format(dist.filename if dist else dist)) + + if extract_url_srcs and dist: + req_line = r.srcs.requirement + else: + dist = struct( + url = "", + filename = "", + sha256 = "", + yanked = False, + ) + req_line = r.srcs.requirement_line - extra_pip_args = tuple(r.extra_pip_args) - for dist in all_dists: key = ( dist.filename, req_line, @@ -269,9 +277,9 @@ def _package_srcs( yanked = dist.yanked, ), ) - for p in target_platforms: - if p not in entry.target_platforms: - entry.target_platforms.append(p) + + if target_platform not in entry.target_platforms: + entry.target_platforms.append(target_platform) return srcs.values() @@ -325,7 +333,7 @@ def host_platform(ctx): repo_utils.get_platforms_cpu_name(ctx), ) -def _add_dists(*, requirement, index_urls, logger = None): +def _add_dists(*, requirement, index_urls, target_platform, logger = None): """Populate dists based on the information from the PyPI index. This function will modify the given requirements_by_platform data structure. @@ -333,6 +341,7 @@ def _add_dists(*, requirement, index_urls, logger = None): Args: requirement: The result of parse_requirements function. index_urls: The result of simpleapi_download. + target_platform: The target_platform information. logger: A logger for printing diagnostic info. """ @@ -342,7 +351,7 @@ def _add_dists(*, requirement, index_urls, logger = None): logger.debug(lambda: "Could not detect the filename from the URL, falling back to pip: {}".format( requirement.srcs.url, )) - return [], None + return None # Handle direct URLs in requirements dist = struct( @@ -353,12 +362,12 @@ def _add_dists(*, requirement, index_urls, logger = None): ) if dist.filename.endswith(".whl"): - return [dist], None + return dist else: - return [], dist + return dist if not index_urls: - return [], None + return None whls = [] sdist = None @@ -401,11 +410,16 @@ def _add_dists(*, requirement, index_urls, logger = None): for reason, dists in yanked.items() ])) - # Filter out the wheels that are incompatible with the target_platforms. - whls = select_whls( + if not target_platform: + # The pipstar platforms are undefined here, so we cannot do any matching + return sdist + + # Select a single wheel that can work on the target_platform + return select_whl( whls = whls, - want_platforms = requirement.target_platforms, + python_version = target_platform.env["python_full_version"], + implementation_name = target_platform.env["implementation_name"], + whl_abi_tags = target_platform.whl_abi_tags, + whl_platform_tags = target_platform.whl_platform_tags, logger = logger, - ) - - return whls, sdist + ) or sdist diff --git a/python/private/pypi/whl_target_platforms.bzl b/python/private/pypi/whl_target_platforms.bzl index 6ea3f120c3..6c3dd5da83 100644 --- a/python/private/pypi/whl_target_platforms.bzl +++ b/python/private/pypi/whl_target_platforms.bzl @@ -16,8 +16,6 @@ A starlark implementation of the wheel platform tag parsing to get the target platform. """ -load(":parse_whl_name.bzl", "parse_whl_name") - # The order of the dictionaries is to keep definitions with their aliases next to each # other _CPU_ALIASES = { @@ -46,136 +44,6 @@ _OS_PREFIXES = { "win": "windows", } # buildifier: disable=unsorted-dict-items -def select_whls(*, whls, want_platforms = [], logger = None): - """Select a subset of wheels suitable for target platforms from a list. - - Args: - whls(list[struct]): A list of candidates which have a `filename` - attribute containing the `whl` filename. - want_platforms(str): The platforms in "{abi}_{os}_{cpu}" or "{os}_{cpu}" format. - logger: A logger for printing diagnostic messages. - - Returns: - A filtered list of items from the `whls` arg where `filename` matches - the selected criteria. If no match is found, an empty list is returned. - """ - if not whls: - return [] - - want_abis = { - "abi3": None, - "none": None, - } - - _want_platforms = {} - version_limit = None - - for p in want_platforms: - if not p.startswith("cp3"): - fail("expected all platforms to start with ABI, but got: {}".format(p)) - - abi, _, os_cpu = p.partition("_") - abi, _, _ = abi.partition(".") - _want_platforms[os_cpu] = None - - # TODO @aignas 2025-04-20: add a test - _want_platforms["{}_{}".format(abi, os_cpu)] = None - - version_limit_candidate = int(abi[3:]) - if not version_limit: - version_limit = version_limit_candidate - if version_limit and version_limit != version_limit_candidate: - fail("Only a single python version is supported for now") - - # For some legacy implementations the wheels may target the `cp3xm` ABI - _want_platforms["{}m_{}".format(abi, os_cpu)] = None - want_abis[abi] = None - want_abis[abi + "m"] = None - - # Also add freethreaded wheels if we find them since we started supporting them - _want_platforms["{}t_{}".format(abi, os_cpu)] = None - want_abis[abi + "t"] = None - - want_platforms = sorted(_want_platforms) - - candidates = {} - for whl in whls: - parsed = parse_whl_name(whl.filename) - - if logger: - logger.trace(lambda: "Deciding whether to use '{}'".format(whl.filename)) - - supported_implementations = {} - whl_version_min = 0 - for tag in parsed.python_tag.split("."): - supported_implementations[tag[:2]] = None - - if tag.startswith("cp3") or tag.startswith("py3"): - version = int(tag[len("..3"):] or 0) - else: - # In this case it should be eithor "cp2" or "py2" and we will default - # to `whl_version_min` = 0 - continue - - if whl_version_min == 0 or version < whl_version_min: - whl_version_min = version - - if not ("cp" in supported_implementations or "py" in supported_implementations): - if logger: - logger.trace(lambda: "Discarding the whl because the whl does not support CPython, whl supported implementations are: {}".format(supported_implementations)) - continue - - if want_abis and parsed.abi_tag not in want_abis: - # Filter out incompatible ABIs - if logger: - logger.trace(lambda: "Discarding the whl because the whl abi did not match") - continue - - if whl_version_min > version_limit: - if logger: - logger.trace(lambda: "Discarding the whl because the whl supported python version is too high") - continue - - compatible = False - if parsed.platform_tag == "any": - compatible = True - else: - for p in whl_target_platforms(parsed.platform_tag, abi_tag = parsed.abi_tag.strip("m") if parsed.abi_tag.startswith("cp") else None): - if p.target_platform in want_platforms: - compatible = True - break - - if not compatible: - if logger: - logger.trace(lambda: "Discarding the whl because the whl does not support the desired platforms: {}".format(want_platforms)) - continue - - for implementation in supported_implementations: - candidates.setdefault( - ( - parsed.abi_tag, - parsed.platform_tag, - ), - {}, - ).setdefault( - ( - # prefer cp implementation - implementation == "cp", - # prefer higher versions - whl_version_min, - # prefer abi3 over none - parsed.abi_tag != "none", - # prefer cpx abi over abi3 - parsed.abi_tag != "abi3", - ), - [], - ).append(whl) - - return [ - candidates[key][sorted(v)[-1]][-1] - for key, v in candidates.items() - ] - def whl_target_platforms(platform_tag, abi_tag = ""): """Parse the wheel abi and platform tags and return (os, cpu) tuples. diff --git a/tests/pypi/extension/extension_tests.bzl b/tests/pypi/extension/extension_tests.bzl index d115546b63..72cbb61d81 100644 --- a/tests/pypi/extension/extension_tests.bzl +++ b/tests/pypi/extension/extension_tests.bzl @@ -65,13 +65,14 @@ def _mod(*, name, default = [], parse = [], override = [], whl_mods = [], is_roo "@platforms//os:{}".format(os), "@platforms//cpu:{}".format(cpu), ], + whl_platform_tags = whl_platform_tags, ) - for os, cpu in [ - ("linux", "x86_64"), - ("linux", "aarch64"), - ("osx", "aarch64"), - ("windows", "aarch64"), - ] + for (os, cpu), whl_platform_tags in { + ("linux", "x86_64"): ["linux_*_x86_64", "manylinux_*_x86_64"], + ("linux", "aarch64"): ["linux_*_aarch64", "manylinux_*_aarch64"], + ("osx", "aarch64"): ["macosx_*_arm64"], + ("windows", "aarch64"): ["win_arm64"], + }.items() ], ), is_root = is_root, @@ -105,21 +106,22 @@ def _build_config(env, enable_pipstar = 0, **kwargs): ) def _default( + *, arch_name = None, config_settings = None, os_name = None, platform = None, + whl_platform_tags = None, env = None, - whl_limit = None, - whl_platforms = None): + whl_abi_tags = None): return struct( arch_name = arch_name, os_name = os_name, platform = platform, + whl_platform_tags = whl_platform_tags or [], config_settings = config_settings, env = env or {}, - whl_platforms = whl_platforms, - whl_limit = whl_limit, + whl_abi_tags = whl_abi_tags or [], ) def _parse( @@ -515,18 +517,21 @@ def _test_torch_experimental_index_url(env): "@platforms//os:{}".format(os), "@platforms//cpu:{}".format(cpu), ], + whl_platform_tags = whl_platform_tags, ) - for os, cpu in [ - ("linux", "aarch64"), - ("linux", "x86_64"), - ("osx", "aarch64"), - ("windows", "x86_64"), - ] + for (os, cpu), whl_platform_tags in { + ("linux", "x86_64"): ["linux_x86_64", "manylinux_*_x86_64"], + ("linux", "aarch64"): ["linux_aarch64", "manylinux_*_aarch64"], + ("osx", "aarch64"): ["macosx_*_arm64"], + ("windows", "x86_64"): ["win_amd64"], + ("windows", "aarch64"): ["win_arm64"], # this should be ignored + }.items() ], parse = [ _parse( hub_name = "pypi", python_version = "3.12", + download_only = True, experimental_index_url = "https://torch.index", requirements_lock = "universal.txt", ), @@ -583,25 +588,25 @@ torch==2.4.1+cpu ; platform_machine == 'x86_64' \ "torch": { "pypi_312_torch_cp312_cp312_linux_x86_64_8800deef": [ whl_config_setting( - filename = "torch-2.4.1+cpu-cp312-cp312-linux_x86_64.whl", + target_platforms = ["cp312_linux_x86_64"], version = "3.12", ), ], "pypi_312_torch_cp312_cp312_manylinux_2_17_aarch64_36109432": [ whl_config_setting( - filename = "torch-2.4.1-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", + target_platforms = ["cp312_linux_aarch64"], version = "3.12", ), ], "pypi_312_torch_cp312_cp312_win_amd64_3a570e5c": [ whl_config_setting( - filename = "torch-2.4.1+cpu-cp312-cp312-win_amd64.whl", + target_platforms = ["cp312_windows_x86_64"], version = "3.12", ), ], "pypi_312_torch_cp312_none_macosx_11_0_arm64_72b484d5": [ whl_config_setting( - filename = "torch-2.4.1-cp312-none-macosx_11_0_arm64.whl", + target_platforms = ["cp312_osx_aarch64"], version = "3.12", ), ], @@ -610,10 +615,7 @@ torch==2.4.1+cpu ; platform_machine == 'x86_64' \ pypi.whl_libraries().contains_exactly({ "pypi_312_torch_cp312_cp312_linux_x86_64_8800deef": { "dep_template": "@pypi//{name}:{target}", - "experimental_target_platforms": [ - "linux_x86_64", - "windows_x86_64", - ], + "experimental_target_platforms": ["linux_x86_64"], "filename": "torch-2.4.1+cpu-cp312-cp312-linux_x86_64.whl", "python_interpreter_target": "unit_test_interpreter_target", "requirement": "torch==2.4.1+cpu", @@ -622,10 +624,7 @@ torch==2.4.1+cpu ; platform_machine == 'x86_64' \ }, "pypi_312_torch_cp312_cp312_manylinux_2_17_aarch64_36109432": { "dep_template": "@pypi//{name}:{target}", - "experimental_target_platforms": [ - "linux_aarch64", - "osx_aarch64", - ], + "experimental_target_platforms": ["linux_aarch64"], "filename": "torch-2.4.1-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", "python_interpreter_target": "unit_test_interpreter_target", "requirement": "torch==2.4.1", @@ -634,10 +633,7 @@ torch==2.4.1+cpu ; platform_machine == 'x86_64' \ }, "pypi_312_torch_cp312_cp312_win_amd64_3a570e5c": { "dep_template": "@pypi//{name}:{target}", - "experimental_target_platforms": [ - "linux_x86_64", - "windows_x86_64", - ], + "experimental_target_platforms": ["windows_x86_64"], "filename": "torch-2.4.1+cpu-cp312-cp312-win_amd64.whl", "python_interpreter_target": "unit_test_interpreter_target", "requirement": "torch==2.4.1+cpu", @@ -646,10 +642,7 @@ torch==2.4.1+cpu ; platform_machine == 'x86_64' \ }, "pypi_312_torch_cp312_none_macosx_11_0_arm64_72b484d5": { "dep_template": "@pypi//{name}:{target}", - "experimental_target_platforms": [ - "linux_aarch64", - "osx_aarch64", - ], + "experimental_target_platforms": ["osx_aarch64"], "filename": "torch-2.4.1-cp312-none-macosx_11_0_arm64.whl", "python_interpreter_target": "unit_test_interpreter_target", "requirement": "torch==2.4.1", @@ -856,78 +849,79 @@ git_dep @ git+https://git.server/repo/project@deadbeefdeadbeef "pypi": { "direct_sdist_without_sha": { "pypi_315_any_name": [ - struct( - config_setting = None, - filename = "any-name.tar.gz", - target_platforms = None, + whl_config_setting( + target_platforms = ( + "cp315_linux_aarch64", + "cp315_linux_x86_64", + "cp315_osx_aarch64", + "cp315_windows_aarch64", + ), version = "3.15", ), ], }, "direct_without_sha": { "pypi_315_direct_without_sha_0_0_1_py3_none_any": [ - struct( - config_setting = None, - filename = "direct_without_sha-0.0.1-py3-none-any.whl", - target_platforms = None, + whl_config_setting( + target_platforms = ( + "cp315_linux_aarch64", + "cp315_linux_x86_64", + "cp315_osx_aarch64", + "cp315_windows_aarch64", + ), version = "3.15", ), ], }, "git_dep": { "pypi_315_git_dep": [ - struct( - config_setting = None, - filename = None, - target_platforms = None, + whl_config_setting( version = "3.15", ), ], }, "pip_fallback": { "pypi_315_pip_fallback": [ - struct( - config_setting = None, - filename = None, - target_platforms = None, + whl_config_setting( version = "3.15", ), ], }, "simple": { "pypi_315_simple_py3_none_any_deadb00f": [ - struct( - config_setting = None, - filename = "simple-0.0.1-py3-none-any.whl", - target_platforms = None, - version = "3.15", - ), - ], - "pypi_315_simple_sdist_deadbeef": [ - struct( - config_setting = None, - filename = "simple-0.0.1.tar.gz", - target_platforms = None, + whl_config_setting( + target_platforms = ( + "cp315_linux_aarch64", + "cp315_linux_x86_64", + "cp315_osx_aarch64", + "cp315_windows_aarch64", + ), version = "3.15", ), ], }, "some_other_pkg": { "pypi_315_some_py3_none_any_deadb33f": [ - struct( - config_setting = None, - filename = "some-other-pkg-0.0.1-py3-none-any.whl", - target_platforms = None, + whl_config_setting( + target_platforms = ( + "cp315_linux_aarch64", + "cp315_linux_x86_64", + "cp315_osx_aarch64", + "cp315_windows_aarch64", + ), version = "3.15", ), ], }, "some_pkg": { "pypi_315_some_pkg_py3_none_any_deadbaaf": [ - struct( - config_setting = None, - filename = "some_pkg-0.0.1-py3-none-any.whl", - target_platforms = None, + whl_config_setting( + target_platforms = ( + "cp315_linux_aarch64", + "cp315_linux_x86_64", + "cp315_osx_aarch64", + "cp315_windows_aarch64", + ), version = "3.15", ), ], @@ -990,21 +984,6 @@ git_dep @ git+https://git.server/repo/project@deadbeefdeadbeef "sha256": "deadb00f", "urls": ["example2.org"], }, - "pypi_315_simple_sdist_deadbeef": { - "dep_template": "@pypi//{name}:{target}", - "experimental_target_platforms": [ - "linux_aarch64", - "linux_x86_64", - "osx_aarch64", - "windows_aarch64", - ], - "extra_pip_args": ["--extra-args-for-sdist-building"], - "filename": "simple-0.0.1.tar.gz", - "python_interpreter_target": "unit_test_interpreter_target", - "requirement": "simple==0.0.1", - "sha256": "deadbeef", - "urls": ["example.org"], - }, "pypi_315_some_pkg_py3_none_any_deadbaaf": { "dep_template": "@pypi//{name}:{target}", "experimental_target_platforms": [ @@ -1260,7 +1239,9 @@ def _test_build_pipstar_platform(env): "@platforms//os:linux", "@platforms//cpu:x86_64", ], - env = {}, + env = {"implementation_name": "cpython"}, + whl_abi_tags = ["none", "abi3", "cp{major}{minor}"], + whl_platform_tags = ["any"], ), }) diff --git a/tests/pypi/parse_requirements/parse_requirements_tests.bzl b/tests/pypi/parse_requirements/parse_requirements_tests.bzl index b14467bc84..249af90114 100644 --- a/tests/pypi/parse_requirements/parse_requirements_tests.bzl +++ b/tests/pypi/parse_requirements/parse_requirements_tests.bzl @@ -15,7 +15,10 @@ "" load("@rules_testing//lib:test_suite.bzl", "test_suite") -load("//python/private/pypi:parse_requirements.bzl", "parse_requirements", "select_requirement") # buildifier: disable=bzl-visibility +load("//python/private:repo_utils.bzl", "REPO_DEBUG_ENV_VAR", "REPO_VERBOSITY_ENV_VAR", "repo_utils") # buildifier: disable=bzl-visibility +load("//python/private/pypi:evaluate_markers.bzl", "evaluate_markers") # buildifier: disable=bzl-visibility +load("//python/private/pypi:parse_requirements.bzl", "select_requirement", _parse_requirements = "parse_requirements") # buildifier: disable=bzl-visibility +load("//python/private/pypi:pep508_env.bzl", pep508_env = "env") # buildifier: disable=bzl-visibility def _mock_ctx(): testdata = { @@ -64,6 +67,12 @@ foo[extra]==0.0.1 --hash=sha256:deadbeef "requirements_marker": """\ foo[extra]==0.0.1 ;marker --hash=sha256:deadbeef bar==0.0.1 --hash=sha256:deadbeef +""", + "requirements_multi_version": """\ +foo==0.0.1; python_full_version < '3.10.0' \ + --hash=sha256:deadbeef +foo==0.0.2; python_full_version >= '3.10.0' \ + --hash=sha256:deadb11f """, "requirements_optional_hash": """ foo==0.0.4 @ https://example.org/foo-0.0.4.whl @@ -96,9 +105,22 @@ bar==0.0.1 --hash=sha256:deadb00f _tests = [] +def parse_requirements(debug = False, **kwargs): + return _parse_requirements( + ctx = _mock_ctx(), + logger = repo_utils.logger(struct( + os = struct( + environ = { + REPO_DEBUG_ENV_VAR: "1", + REPO_VERBOSITY_ENV_VAR: "TRACE" if debug else "INFO", + }, + ), + ), "unit-test"), + **kwargs + ) + def _test_simple(env): got = parse_requirements( - ctx = _mock_ctx(), requirements_by_platform = { "requirements_lock": ["linux_x86_64", "windows_x86_64"], }, @@ -131,7 +153,6 @@ _tests.append(_test_simple) def _test_direct_urls_integration(env): """Check that we are using the filename from index_sources.""" got = parse_requirements( - ctx = _mock_ctx(), requirements_by_platform = { "requirements_direct": ["linux_x86_64"], "requirements_direct_sdist": ["osx_x86_64"], @@ -171,7 +192,6 @@ _tests.append(_test_direct_urls_integration) def _test_extra_pip_args(env): got = parse_requirements( - ctx = _mock_ctx(), requirements_by_platform = { "requirements_extra_args": ["linux_x86_64"], }, @@ -203,7 +223,6 @@ _tests.append(_test_extra_pip_args) def _test_dupe_requirements(env): got = parse_requirements( - ctx = _mock_ctx(), requirements_by_platform = { "requirements_lock_dupe": ["linux_x86_64"], }, @@ -232,7 +251,6 @@ _tests.append(_test_dupe_requirements) def _test_multi_os(env): got = parse_requirements( - ctx = _mock_ctx(), requirements_by_platform = { "requirements_linux": ["linux_x86_64"], "requirements_windows": ["windows_x86_64"], @@ -296,7 +314,6 @@ _tests.append(_test_multi_os) def _test_multi_os_legacy(env): got = parse_requirements( - ctx = _mock_ctx(), requirements_by_platform = { "requirements_linux_download_only": ["cp39_linux_x86_64"], "requirements_osx_download_only": ["cp39_osx_aarch64"], @@ -377,7 +394,6 @@ def _test_env_marker_resolution(env): return ret got = parse_requirements( - ctx = _mock_ctx(), requirements_by_platform = { "requirements_marker": ["cp311_linux_super_exotic", "cp311_windows_x86_64"], }, @@ -424,7 +440,6 @@ _tests.append(_test_env_marker_resolution) def _test_different_package_version(env): got = parse_requirements( - ctx = _mock_ctx(), requirements_by_platform = { "requirements_different_package_version": ["linux_x86_64"], }, @@ -463,7 +478,6 @@ _tests.append(_test_different_package_version) def _test_optional_hash(env): got = parse_requirements( - ctx = _mock_ctx(), requirements_by_platform = { "requirements_optional_hash": ["linux_x86_64"], }, @@ -502,7 +516,6 @@ _tests.append(_test_optional_hash) def _test_git_sources(env): got = parse_requirements( - ctx = _mock_ctx(), requirements_by_platform = { "requirements_git": ["linux_x86_64"], }, @@ -531,11 +544,30 @@ _tests.append(_test_git_sources) def _test_overlapping_shas_with_index_results(env): got = parse_requirements( - ctx = _mock_ctx(), requirements_by_platform = { "requirements_linux": ["cp39_linux_x86_64"], "requirements_osx": ["cp39_osx_x86_64"], }, + platforms = { + "cp39_linux_x86_64": struct( + env = pep508_env( + python_version = "3.9.0", + os = "linux", + arch = "x86_64", + ), + whl_abi_tags = ["none"], + whl_platform_tags = ["any"], + ), + "cp39_osx_x86_64": struct( + env = pep508_env( + python_version = "3.9.0", + os = "osx", + arch = "x86_64", + ), + whl_abi_tags = ["none"], + whl_platform_tags = ["macosx_*_x86_64"], + ), + }, get_index_urls = lambda _, __: { "foo": struct( sdists = { @@ -566,10 +598,9 @@ def _test_overlapping_shas_with_index_results(env): env.expect.that_collection(got).contains_exactly([ struct( - name = "foo", is_exposed = True, - # TODO @aignas 2025-05-25: how do we rename this? is_multiple_versions = True, + name = "foo", srcs = [ struct( distribution = "foo", @@ -577,27 +608,109 @@ def _test_overlapping_shas_with_index_results(env): filename = "foo-0.0.1-py3-none-any.whl", requirement_line = "foo==0.0.3", sha256 = "deadbaaf", - target_platforms = ["cp39_linux_x86_64", "cp39_osx_x86_64"], + target_platforms = ["cp39_linux_x86_64"], url = "super2", yanked = False, ), struct( distribution = "foo", extra_pip_args = [], - filename = "foo-0.0.1.tar.gz", + filename = "foo-0.0.1-py3-none-macosx_14_0_x86_64.whl", requirement_line = "foo==0.0.3", - sha256 = "5d15t", - target_platforms = ["cp39_linux_x86_64", "cp39_osx_x86_64"], - url = "sdist", + sha256 = "deadb11f", + target_platforms = ["cp39_osx_x86_64"], + url = "super2", yanked = False, ), + ], + ), + ]) + +_tests.append(_test_overlapping_shas_with_index_results) + +def _test_get_index_urls_different_versions(env): + got = parse_requirements( + requirements_by_platform = { + "requirements_multi_version": [ + "cp39_linux_x86_64", + "cp310_linux_x86_64", + ], + }, + platforms = { + "cp310_linux_x86_64": struct( + env = pep508_env( + python_version = "3.9.0", + os = "linux", + arch = "x86_64", + ), + whl_abi_tags = ["none"], + whl_platform_tags = ["any"], + ), + "cp39_linux_x86_64": struct( + env = pep508_env( + python_version = "3.9.0", + os = "linux", + arch = "x86_64", + ), + whl_abi_tags = ["none"], + whl_platform_tags = ["any"], + ), + }, + get_index_urls = lambda _, __: { + "foo": struct( + sdists = {}, + whls = { + "deadb11f": struct( + url = "super2", + sha256 = "deadb11f", + filename = "foo-0.0.2-py3-none-any.whl", + yanked = False, + ), + "deadbaaf": struct( + url = "super2", + sha256 = "deadbaaf", + filename = "foo-0.0.1-py3-none-any.whl", + yanked = False, + ), + }, + ), + }, + evaluate_markers = lambda _, requirements: evaluate_markers( + requirements = requirements, + platforms = { + "cp310_linux_x86_64": struct( + env = {"python_full_version": "3.10.0"}, + ), + "cp39_linux_x86_64": struct( + env = {"python_full_version": "3.9.0"}, + ), + }, + ), + ) + + env.expect.that_collection(got).contains_exactly([ + struct( + is_exposed = True, + is_multiple_versions = True, + name = "foo", + srcs = [ struct( distribution = "foo", extra_pip_args = [], - filename = "foo-0.0.1-py3-none-macosx_14_0_x86_64.whl", - requirement_line = "foo==0.0.3", + filename = "", + requirement_line = "foo==0.0.1 --hash=sha256:deadbeef", + sha256 = "", + target_platforms = ["cp39_linux_x86_64"], + url = "", + yanked = False, + ), + struct( + distribution = "foo", + extra_pip_args = [], + filename = "foo-0.0.2-py3-none-any.whl", + requirement_line = "foo==0.0.2", sha256 = "deadb11f", - target_platforms = ["cp39_osx_x86_64"], + target_platforms = ["cp310_linux_x86_64"], url = "super2", yanked = False, ), @@ -605,7 +718,7 @@ def _test_overlapping_shas_with_index_results(env): ), ]) -_tests.append(_test_overlapping_shas_with_index_results) +_tests.append(_test_get_index_urls_different_versions) def parse_requirements_test_suite(name): """Create the test suite. diff --git a/tests/pypi/whl_target_platforms/BUILD.bazel b/tests/pypi/whl_target_platforms/BUILD.bazel index 6c35b08d32..fec25af033 100644 --- a/tests/pypi/whl_target_platforms/BUILD.bazel +++ b/tests/pypi/whl_target_platforms/BUILD.bazel @@ -12,9 +12,6 @@ # See the License for the specific language governing permissions and # limitations under the License. -load(":select_whl_tests.bzl", "select_whl_test_suite") load(":whl_target_platforms_tests.bzl", "whl_target_platforms_test_suite") -select_whl_test_suite(name = "select_whl_tests") - whl_target_platforms_test_suite(name = "whl_target_platforms_tests") diff --git a/tests/pypi/whl_target_platforms/select_whl_tests.bzl b/tests/pypi/whl_target_platforms/select_whl_tests.bzl deleted file mode 100644 index 1674ac5ef2..0000000000 --- a/tests/pypi/whl_target_platforms/select_whl_tests.bzl +++ /dev/null @@ -1,314 +0,0 @@ -# Copyright 2024 The Bazel Authors. All rights reserved. -# -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -"" - -load("@rules_testing//lib:test_suite.bzl", "test_suite") -load("//python/private:repo_utils.bzl", "REPO_DEBUG_ENV_VAR", "REPO_VERBOSITY_ENV_VAR", "repo_utils") # buildifier: disable=bzl-visibility -load("//python/private/pypi:whl_target_platforms.bzl", "select_whls") # buildifier: disable=bzl-visibility - -WHL_LIST = [ - "pkg-0.0.1-cp311-cp311-macosx_10_9_universal2.whl", - "pkg-0.0.1-cp311-cp311-macosx_10_9_x86_64.whl", - "pkg-0.0.1-cp311-cp311-macosx_11_0_arm64.whl", - "pkg-0.0.1-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", - "pkg-0.0.1-cp311-cp311-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", - "pkg-0.0.1-cp311-cp311-manylinux_2_17_s390x.manylinux2014_s390x.whl", - "pkg-0.0.1-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", - "pkg-0.0.1-cp311-cp311-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", - "pkg-0.0.1-cp313-cp313t-musllinux_1_1_x86_64.whl", - "pkg-0.0.1-cp313-cp313-musllinux_1_1_x86_64.whl", - "pkg-0.0.1-cp313-abi3-musllinux_1_1_x86_64.whl", - "pkg-0.0.1-cp313-none-musllinux_1_1_x86_64.whl", - "pkg-0.0.1-cp311-cp311-musllinux_1_1_aarch64.whl", - "pkg-0.0.1-cp311-cp311-musllinux_1_1_i686.whl", - "pkg-0.0.1-cp311-cp311-musllinux_1_1_ppc64le.whl", - "pkg-0.0.1-cp311-cp311-musllinux_1_1_s390x.whl", - "pkg-0.0.1-cp311-cp311-musllinux_1_1_x86_64.whl", - "pkg-0.0.1-cp311-cp311-win32.whl", - "pkg-0.0.1-cp311-cp311-win_amd64.whl", - "pkg-0.0.1-cp37-cp37m-macosx_10_9_x86_64.whl", - "pkg-0.0.1-cp37-cp37m-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", - "pkg-0.0.1-cp37-cp37m-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", - "pkg-0.0.1-cp37-cp37m-manylinux_2_17_s390x.manylinux2014_s390x.whl", - "pkg-0.0.1-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", - "pkg-0.0.1-cp37-cp37m-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", - "pkg-0.0.1-cp37-cp37m-musllinux_1_1_aarch64.whl", - "pkg-0.0.1-cp37-cp37m-musllinux_1_1_i686.whl", - "pkg-0.0.1-cp37-cp37m-musllinux_1_1_ppc64le.whl", - "pkg-0.0.1-cp37-cp37m-musllinux_1_1_s390x.whl", - "pkg-0.0.1-cp37-cp37m-musllinux_1_1_x86_64.whl", - "pkg-0.0.1-cp37-cp37m-win32.whl", - "pkg-0.0.1-cp37-cp37m-win_amd64.whl", - "pkg-0.0.1-cp39-cp39-macosx_10_9_universal2.whl", - "pkg-0.0.1-cp39-cp39-macosx_10_9_x86_64.whl", - "pkg-0.0.1-cp39-cp39-macosx_11_0_arm64.whl", - "pkg-0.0.1-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", - "pkg-0.0.1-cp39-cp39-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", - "pkg-0.0.1-cp39-cp39-manylinux_2_17_s390x.manylinux2014_s390x.whl", - "pkg-0.0.1-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", - "pkg-0.0.1-cp39-cp39-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", - "pkg-0.0.1-cp39-cp39-musllinux_1_1_aarch64.whl", - "pkg-0.0.1-cp39-cp39-musllinux_1_1_i686.whl", - "pkg-0.0.1-cp39-cp39-musllinux_1_1_ppc64le.whl", - "pkg-0.0.1-cp39-cp39-musllinux_1_1_s390x.whl", - "pkg-0.0.1-cp39-cp39-musllinux_1_1_x86_64.whl", - "pkg-0.0.1-cp39-cp39-win32.whl", - "pkg-0.0.1-cp39-cp39-win_amd64.whl", - "pkg-0.0.1-cp39-abi3-any.whl", - "pkg-0.0.1-py310-abi3-any.whl", - "pkg-0.0.1-py3-abi3-any.whl", - "pkg-0.0.1-py3-none-any.whl", -] - -def _match(env, got, *want_filenames): - if not want_filenames: - env.expect.that_collection(got).has_size(len(want_filenames)) - return - - got_filenames = [g.filename for g in got] - env.expect.that_collection(got_filenames).contains_exactly(want_filenames) - - if got: - # Check that we pass the original structs - env.expect.that_str(got[0].other).equals("dummy") - -def _select_whls(whls, debug = False, **kwargs): - return select_whls( - whls = [ - struct( - filename = f, - other = "dummy", - ) - for f in whls - ], - logger = repo_utils.logger(struct( - os = struct( - environ = { - REPO_DEBUG_ENV_VAR: "1", - REPO_VERBOSITY_ENV_VAR: "TRACE" if debug else "INFO", - }, - ), - ), "unit-test"), - **kwargs - ) - -_tests = [] - -def _test_simplest(env): - got = _select_whls( - whls = [ - "pkg-0.0.1-py2.py3-abi3-any.whl", - "pkg-0.0.1-py3-abi3-any.whl", - "pkg-0.0.1-py3-none-any.whl", - ], - want_platforms = ["cp30_ignored"], - ) - _match( - env, - got, - "pkg-0.0.1-py3-abi3-any.whl", - "pkg-0.0.1-py3-none-any.whl", - ) - -_tests.append(_test_simplest) - -def _test_select_by_supported_py_version(env): - for minor_version, match in { - 8: "pkg-0.0.1-py3-abi3-any.whl", - 11: "pkg-0.0.1-py311-abi3-any.whl", - }.items(): - got = _select_whls( - whls = [ - "pkg-0.0.1-py2.py3-abi3-any.whl", - "pkg-0.0.1-py3-abi3-any.whl", - "pkg-0.0.1-py311-abi3-any.whl", - ], - want_platforms = ["cp3{}_ignored".format(minor_version)], - ) - _match(env, got, match) - -_tests.append(_test_select_by_supported_py_version) - -def _test_select_by_supported_cp_version(env): - for minor_version, match in { - 11: "pkg-0.0.1-cp311-abi3-any.whl", - 8: "pkg-0.0.1-py3-abi3-any.whl", - }.items(): - got = _select_whls( - whls = [ - "pkg-0.0.1-py2.py3-abi3-any.whl", - "pkg-0.0.1-py3-abi3-any.whl", - "pkg-0.0.1-py311-abi3-any.whl", - "pkg-0.0.1-cp311-abi3-any.whl", - ], - want_platforms = ["cp3{}_ignored".format(minor_version)], - ) - _match(env, got, match) - -_tests.append(_test_select_by_supported_cp_version) - -def _test_supported_cp_version_manylinux(env): - for minor_version, match in { - 8: "pkg-0.0.1-py3-none-manylinux_x86_64.whl", - 11: "pkg-0.0.1-cp311-none-manylinux_x86_64.whl", - }.items(): - got = _select_whls( - whls = [ - "pkg-0.0.1-py2.py3-none-manylinux_x86_64.whl", - "pkg-0.0.1-py3-none-manylinux_x86_64.whl", - "pkg-0.0.1-py311-none-manylinux_x86_64.whl", - "pkg-0.0.1-cp311-none-manylinux_x86_64.whl", - ], - want_platforms = ["cp3{}_linux_x86_64".format(minor_version)], - ) - _match(env, got, match) - -_tests.append(_test_supported_cp_version_manylinux) - -def _test_ignore_unsupported(env): - got = _select_whls( - whls = [ - "pkg-0.0.1-xx3-abi3-any.whl", - ], - want_platforms = ["cp30_ignored"], - ) - _match(env, got) - -_tests.append(_test_ignore_unsupported) - -def _test_match_abi_and_not_py_version(env): - # Check we match the ABI and not the py version - got = _select_whls(whls = WHL_LIST, want_platforms = ["cp37_linux_x86_64"]) - _match( - env, - got, - "pkg-0.0.1-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", - "pkg-0.0.1-cp37-cp37m-musllinux_1_1_x86_64.whl", - "pkg-0.0.1-py3-abi3-any.whl", - "pkg-0.0.1-py3-none-any.whl", - ) - -_tests.append(_test_match_abi_and_not_py_version) - -def _test_select_filename_with_many_tags(env): - # Check we can select a filename with many platform tags - got = _select_whls(whls = WHL_LIST, want_platforms = ["cp39_linux_x86_32"]) - _match( - env, - got, - "pkg-0.0.1-cp39-cp39-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", - "pkg-0.0.1-cp39-cp39-musllinux_1_1_i686.whl", - "pkg-0.0.1-cp39-abi3-any.whl", - "pkg-0.0.1-py3-none-any.whl", - ) - -_tests.append(_test_select_filename_with_many_tags) - -def _test_osx_prefer_arch_specific(env): - # Check that we prefer the specific wheel - got = _select_whls( - whls = WHL_LIST, - want_platforms = ["cp311_osx_x86_64", "cp311_osx_x86_32"], - ) - _match( - env, - got, - "pkg-0.0.1-cp311-cp311-macosx_10_9_universal2.whl", - "pkg-0.0.1-cp311-cp311-macosx_10_9_x86_64.whl", - "pkg-0.0.1-cp39-abi3-any.whl", - "pkg-0.0.1-py3-none-any.whl", - ) - - got = _select_whls(whls = WHL_LIST, want_platforms = ["cp311_osx_aarch64"]) - _match( - env, - got, - "pkg-0.0.1-cp311-cp311-macosx_10_9_universal2.whl", - "pkg-0.0.1-cp311-cp311-macosx_11_0_arm64.whl", - "pkg-0.0.1-cp39-abi3-any.whl", - "pkg-0.0.1-py3-none-any.whl", - ) - -_tests.append(_test_osx_prefer_arch_specific) - -def _test_osx_fallback_to_universal2(env): - # Check that we can use the universal2 if the arm wheel is not available - got = _select_whls( - whls = [w for w in WHL_LIST if "arm64" not in w], - want_platforms = ["cp311_osx_aarch64"], - ) - _match( - env, - got, - "pkg-0.0.1-cp311-cp311-macosx_10_9_universal2.whl", - "pkg-0.0.1-cp39-abi3-any.whl", - "pkg-0.0.1-py3-none-any.whl", - ) - -_tests.append(_test_osx_fallback_to_universal2) - -def _test_prefer_manylinux_wheels(env): - # Check we prefer platform specific wheels - got = _select_whls(whls = WHL_LIST, want_platforms = ["cp39_linux_x86_64"]) - _match( - env, - got, - "pkg-0.0.1-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", - "pkg-0.0.1-cp39-cp39-musllinux_1_1_x86_64.whl", - "pkg-0.0.1-cp39-abi3-any.whl", - "pkg-0.0.1-py3-none-any.whl", - ) - -_tests.append(_test_prefer_manylinux_wheels) - -def _test_freethreaded_wheels(env): - # Check we prefer platform specific wheels - got = _select_whls(whls = WHL_LIST, want_platforms = ["cp313_linux_x86_64"]) - _match( - env, - got, - "pkg-0.0.1-cp313-cp313t-musllinux_1_1_x86_64.whl", - "pkg-0.0.1-cp313-cp313-musllinux_1_1_x86_64.whl", - "pkg-0.0.1-cp313-abi3-musllinux_1_1_x86_64.whl", - "pkg-0.0.1-cp313-none-musllinux_1_1_x86_64.whl", - "pkg-0.0.1-cp39-abi3-any.whl", - "pkg-0.0.1-py3-none-any.whl", - ) - -_tests.append(_test_freethreaded_wheels) - -def _test_micro_version_freethreaded(env): - # Check we prefer platform specific wheels - got = _select_whls(whls = WHL_LIST, want_platforms = ["cp313.3_linux_x86_64"]) - _match( - env, - got, - "pkg-0.0.1-cp313-cp313t-musllinux_1_1_x86_64.whl", - "pkg-0.0.1-cp313-cp313-musllinux_1_1_x86_64.whl", - "pkg-0.0.1-cp313-abi3-musllinux_1_1_x86_64.whl", - "pkg-0.0.1-cp313-none-musllinux_1_1_x86_64.whl", - "pkg-0.0.1-cp39-abi3-any.whl", - "pkg-0.0.1-py3-none-any.whl", - ) - -_tests.append(_test_micro_version_freethreaded) - -def select_whl_test_suite(name): - """Create the test suite. - - Args: - name: the name of the test suite - """ - test_suite(name = name, basic_tests = _tests) From e2295aba9aae8c6aef60eae6be9597052349e8d0 Mon Sep 17 00:00:00 2001 From: Ignas Anikevicius <240938+aignas@users.noreply.github.com> Date: Sun, 17 Aug 2025 11:08:03 +0900 Subject: [PATCH 387/922] feat(pypi): builder for netrc and auth_patterns (#3136) With this we move closer towards starting playing with the API to fully replace `pip.parse` with `pip.configure` builder pattern for better expressiveness. Work towards #2747 --- CHANGELOG.md | 3 ++- python/private/pypi/extension.bzl | 23 +++++++++++++++-------- tests/pypi/extension/extension_tests.bzl | 22 +++++++++++++++++----- 3 files changed, 34 insertions(+), 14 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 9d45aa53c3..0e8ad65a5e 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -128,7 +128,8 @@ END_UNRELEASED_TEMPLATE ([#3114](https://github.com/bazel-contrib/rules_python/pull/3114)). * (pypi) To configure the environment for `requirements.txt` evaluation, use the newly added developer preview of the `pip.default` tag class. Only `rules_python` and root modules can use - this feature. You can also configure custom `config_settings` using `pip.default`. + this feature. You can also configure custom `config_settings` using `pip.default`. It + can also be used to set the global `netrc` or `auth_patterns` variables. * (pypi) PyPI dependencies now expose an `:extracted_whl_files` filegroup target of all the files extracted from the wheel. This can be used in lieu of {obj}`whl_filegroup` to avoid copying/extracting wheel multiple times to diff --git a/python/private/pypi/extension.bzl b/python/private/pypi/extension.bzl index 2c7aa8a0e5..0c06dea2ff 100644 --- a/python/private/pypi/extension.bzl +++ b/python/private/pypi/extension.bzl @@ -322,12 +322,12 @@ def _create_whl_repos( src = src, whl_library_args = whl_library_args, download_only = pip_attr.download_only, - netrc = pip_attr.netrc, + netrc = config.netrc or pip_attr.netrc, use_downloader = use_downloader.get( whl.name, get_index_urls != None, # defaults to True if the get_index_urls is defined ), - auth_patterns = pip_attr.auth_patterns, + auth_patterns = config.auth_patterns or pip_attr.auth_patterns, python_version = major_minor, is_multiple_versions = whl.is_multiple_versions, enable_pipstar = config.enable_pipstar, @@ -502,13 +502,20 @@ def build_config( if platform and not (tag.arch_name or tag.config_settings or tag.env or tag.os_name or tag.whl_abi_tags or tag.whl_platform_tags): defaults["platforms"].pop(platform) - # TODO @aignas 2025-05-19: add more attr groups: - # * for AUTH - the default `netrc` usage could be configured through a common - # attribute. - # * for index/downloader config. This includes all of those attributes for - # overrides, etc. Index overrides per platform could be also used here. + _configure( + defaults, + override = mod.is_root, + # extra values that we just add + auth_patterns = tag.auth_patterns, + netrc = tag.netrc, + # TODO @aignas 2025-05-19: add more attr groups: + # * for index/downloader config. This includes all of those attributes for + # overrides, etc. Index overrides per platform could be also used here. + ) return struct( + auth_patterns = defaults.get("auth_patterns", {}), + netrc = defaults.get("netrc", None), platforms = { name: _plat(**values) for name, values in defaults["platforms"].items() @@ -975,7 +982,7 @@ See official [docs](https://packaging.python.org/en/latest/specifications/platfo ::: """, ), -} +} | AUTH_ATTRS _SUPPORTED_PEP508_KEYS = [ "implementation_name", diff --git a/tests/pypi/extension/extension_tests.bzl b/tests/pypi/extension/extension_tests.bzl index 72cbb61d81..ab8362ef0c 100644 --- a/tests/pypi/extension/extension_tests.bzl +++ b/tests/pypi/extension/extension_tests.bzl @@ -100,28 +100,34 @@ def _build_config(env, enable_pipstar = 0, **kwargs): **kwargs ), attrs = dict( - platforms = subjects.dict, + auth_patterns = subjects.dict, enable_pipstar = subjects.bool, + netrc = subjects.str, + platforms = subjects.dict, ), ) def _default( *, arch_name = None, + auth_patterns = None, config_settings = None, + env = None, + netrc = None, os_name = None, platform = None, whl_platform_tags = None, - env = None, whl_abi_tags = None): return struct( arch_name = arch_name, - os_name = os_name, - platform = platform, - whl_platform_tags = whl_platform_tags or [], + auth_patterns = auth_patterns or {}, config_settings = config_settings, env = env or {}, + netrc = netrc, + os_name = os_name, + platform = platform, whl_abi_tags = whl_abi_tags or [], + whl_platform_tags = whl_platform_tags or [], ) def _parse( @@ -1224,11 +1230,17 @@ def _test_build_pipstar_platform(env): ], ), _default(platform = "myplat2"), + _default( + netrc = "my_netrc", + auth_patterns = {"foo": "bar"}, + ), ], ), ), enable_pipstar = True, ) + config.auth_patterns().contains_exactly({"foo": "bar"}) + config.netrc().equals("my_netrc") config.enable_pipstar().equals(True) config.platforms().contains_exactly({ "myplat": struct( From 8684dd98df009c321e7a5525f224b2eb61a8fa2d Mon Sep 17 00:00:00 2001 From: Ivo List Date: Sun, 17 Aug 2025 04:19:35 +0200 Subject: [PATCH 388/922] refactor: Use the linkstamps from linker_inputs instead of from cc_linking_context for to support upcoming CcInfo changes (#3075) This change allows Bazel to remove linkstamp from CcLinkingContext. The change is a no-op. Co-authored-by: Richard Levasseur Co-authored-by: Ignas Anikevicius <240938+aignas@users.noreply.github.com> --- python/private/py_executable.bzl | 11 ++++++----- 1 file changed, 6 insertions(+), 5 deletions(-) diff --git a/python/private/py_executable.bzl b/python/private/py_executable.bzl index 9927975aa8..30f18b5e64 100644 --- a/python/private/py_executable.bzl +++ b/python/private/py_executable.bzl @@ -1578,7 +1578,11 @@ def _create_shared_native_deps_dso( feature_configuration, requested_features, cc_toolchain): - linkstamps = py_internal.linking_context_linkstamps(cc_info.linking_context) + linkstamps = [ + py_internal.linkstamp_file(linkstamp) + for linker_input in cc_info.linking_context.linker_inputs.to_list() + for linkstamp in linker_input.linkstamps + ] partially_disabled_thin_lto = ( cc_common.is_enabled( @@ -1602,10 +1606,7 @@ def _create_shared_native_deps_dso( for input in cc_info.linking_context.linker_inputs.to_list() for flag in input.user_link_flags ], - linkstamps = [ - py_internal.linkstamp_file(linkstamp) - for linkstamp in linkstamps.to_list() - ], + linkstamps = linkstamps, build_info_artifacts = _get_build_info(ctx, cc_toolchain) if linkstamps else [], features = requested_features, is_test_target_partially_disabled_thin_lto = is_test and partially_disabled_thin_lto, From 51d9b98bf9f03804280a4c8e8a97ab0de5c2ae5f Mon Sep 17 00:00:00 2001 From: Ignas Anikevicius <240938+aignas@users.noreply.github.com> Date: Sun, 17 Aug 2025 11:21:27 +0900 Subject: [PATCH 389/922] chore(pypi): remove unused config setting code (#3065) Summary: - remove config settings generation - remove unnecessary params - remove BUILD.bazel generating unnecessary params --- python/private/pypi/BUILD.bazel | 4 - python/private/pypi/config_settings.bzl | 328 +---------- python/private/pypi/hub_repository.bzl | 4 - python/private/pypi/pkg_aliases.bzl | 243 +------- python/private/pypi/render_pkg_aliases.bzl | 53 +- python/private/pypi/whl_config_setting.bzl | 12 +- .../config_settings/config_settings_tests.bzl | 551 ------------------ tests/pypi/extension/extension_tests.bzl | 26 +- tests/pypi/integration/BUILD.bazel | 13 - tests/pypi/pkg_aliases/pkg_aliases_test.bzl | 198 +------ .../render_pkg_aliases_test.bzl | 263 +-------- 11 files changed, 72 insertions(+), 1623 deletions(-) diff --git a/python/private/pypi/BUILD.bazel b/python/private/pypi/BUILD.bazel index 847c85d634..cb3408a191 100644 --- a/python/private/pypi/BUILD.bazel +++ b/python/private/pypi/BUILD.bazel @@ -320,8 +320,6 @@ bzl_library( srcs = ["pkg_aliases.bzl"], deps = [ ":labels_bzl", - ":parse_whl_name_bzl", - ":whl_target_platforms_bzl", "//python/private:text_util_bzl", "@bazel_skylib//lib:selects", ], @@ -349,9 +347,7 @@ bzl_library( srcs = ["render_pkg_aliases.bzl"], deps = [ ":generate_group_library_build_bazel_bzl", - ":parse_whl_name_bzl", ":whl_config_setting_bzl", - ":whl_target_platforms_bzl", "//python/private:normalize_name_bzl", "//python/private:text_util_bzl", ], diff --git a/python/private/pypi/config_settings.bzl b/python/private/pypi/config_settings.bzl index f4826007f8..dcb6779d5b 100644 --- a/python/private/pypi/config_settings.bzl +++ b/python/private/pypi/config_settings.bzl @@ -17,101 +17,17 @@ The {obj}`config_settings` macro is used to create the config setting targets that can be used in the {obj}`pkg_aliases` macro for selecting the compatible repositories. -Bazel's selects work by selecting the most-specialized configuration setting -that matches the target platform, which is further described in [bazel documentation][docs]. -We can leverage this fact to ensure that the most specialized matches are used -by default with the users being able to configure string_flag values to select -the less specialized ones. - -[docs]: https://bazel.build/docs/configurable-attributes - -The config settings in the order from the least specialized to the most -specialized is as follows: -* `:is_cp3` -* `:is_cp3_sdist` -* `:is_cp3_py_none_any` -* `:is_cp3_py3_none_any` -* `:is_cp3_py3_abi3_any` -* `:is_cp3_none_any` -* `:is_cp3_any_any` -* `:is_cp3_cp3_any` and `:is_cp3_cp3t_any` -* `:is_cp3_py_none_` -* `:is_cp3_py3_none_` -* `:is_cp3_py3_abi3_` -* `:is_cp3_none_` -* `:is_cp3_abi3_` -* `:is_cp3_cp3_` and `:is_cp3_cp3t_` - -Optionally instead of `` there sometimes may be `.` used in order to fully specify the versions - -The specialization of free-threaded vs non-free-threaded wheels is the same as -they are just variants of each other. The same goes for the specialization of -`musllinux` vs `manylinux`. - -The goal of this macro is to provide config settings that provide unambigous -matches if any pair of them is used together for any target configuration -setting. We achieve this by using dummy internal `flag_values` keys to force the -items further down the list to appear to be more specialized than the ones above. - -What is more, the names of the config settings are as similar to the platform wheel -specification as possible. How the wheel names map to the config setting names defined -in here is described in {obj}`pkg_aliases` documentation. - -:::{note} -Right now the specialization of adjacent config settings where one is with -`constraint_values` and one is without is ambiguous. I.e. `py_none_any` and -`sdist_linux_x86_64` have the same specialization from bazel point of view -because one has one `flag_value` entry and `constraint_values` and the -other has 2 flag_value entries. And unfortunately there is no way to disambiguate -it, because we are essentially in two dimensions here (`flag_values` and -`constraint_values`). Hence, when using the `config_settings` from here, -either have all of them with empty `suffix` or all of them with a non-empty -suffix. -::: +The config settings are of the form `:is_cp3`. Suffix is a +normalized user provided value for the platform name. The goal of this macro is to +ensure that we can incorporate the user provided `config_setting` targets to create +our composite config_setting targets. """ load("@bazel_skylib//lib:selects.bzl", "selects") -load("//python/private:flags.bzl", "LibcFlag") -load(":flags.bzl", "INTERNAL_FLAGS", "UniversalWhlFlag") - -FLAGS = struct( - **{ - f: str(Label("//python/config_settings:" + f)) - for f in [ - "is_pip_whl_auto", - "is_pip_whl_no", - "is_pip_whl_only", - "_is_py_freethreaded_yes", - "_is_py_freethreaded_no", - "pip_whl_glibc_version", - "pip_whl_muslc_version", - "pip_whl_osx_arch", - "pip_whl_osx_version", - "py_linux_libc", - "python_version", - ] - } -) - -_DEFAULT = "//conditions:default" -_INCOMPATIBLE = "@platforms//:incompatible" - -# Here we create extra string flags that are just to work with the select -# selecting the most specialized match. We don't allow the user to change -# them. -_flags = struct( - **{ - f: str(Label("//python/config_settings:_internal_pip_" + f)) - for f in INTERNAL_FLAGS - } -) def config_settings( *, python_versions = [], - glibc_versions = [], - muslc_versions = [], - osx_versions = [], name = None, platform_config_settings = {}, **kwargs): @@ -121,12 +37,6 @@ def config_settings( name (str): Currently unused. python_versions (list[str]): The list of python versions to configure config settings for. - glibc_versions (list[str]): The list of glibc version of the wheels to - configure config settings for. - muslc_versions (list[str]): The list of musl version of the wheels to - configure config settings for. - osx_versions (list[str]): The list of OSX OS versions to configure - config settings for. platform_config_settings: {type}`dict[str, list[str]]` the constraint values to use instead of the default ones. Key are platform names (a human-friendly platform string). Values are lists of @@ -134,212 +44,46 @@ def config_settings( **kwargs: Other args passed to the underlying implementations, such as {obj}`native`. """ - - glibc_versions = [""] + glibc_versions - muslc_versions = [""] + muslc_versions - osx_versions = [""] + osx_versions target_platforms = { "": [], - # TODO @aignas 2025-06-15: allowing universal2 and platform specific wheels in one - # closure is making things maybe a little bit too complicated. - "osx_universal2": ["@platforms//os:osx"], } | platform_config_settings - for python_version in python_versions: - for platform_name, config_settings in target_platforms.items(): - suffix = "_{}".format(platform_name) if platform_name else "" - os, _, cpu = platform_name.partition("_") + for platform_name, config_settings in target_platforms.items(): + suffix = "_{}".format(platform_name) if platform_name else "" + + # We parse the target settings and if there is a "platforms//os" or + # "platforms//cpu" value in here, we also add it into the constraint_values + # + # this is to ensure that we can still pass all of the unit tests for config + # setting specialization. + # + # TODO @aignas 2025-07-23: is this the right way? Maybe we should drop these + # and remove the tests? + constraint_values = [] + for setting in config_settings: + setting_label = Label(setting) + if setting_label.repo_name == "platforms" and setting_label.package in ["os", "cpu"]: + constraint_values.append(setting) + + for python_version in python_versions: + cpv = "cp" + python_version.replace(".", "") + prefix = "is_{}".format(cpv) - # We parse the target settings and if there is a "platforms//os" or - # "platforms//cpu" value in here, we also add it into the constraint_values - # - # this is to ensure that we can still pass all of the unit tests for config - # setting specialization. - constraint_values = [] - for setting in config_settings: - setting_label = Label(setting) - if setting_label.repo_name == "platforms" and setting_label.package in ["os", "cpu"]: - constraint_values.append(setting) - - _dist_config_settings( - suffix = suffix, - plat_flag_values = _plat_flag_values( - os = os, - cpu = cpu, - osx_versions = osx_versions, - glibc_versions = glibc_versions, - muslc_versions = muslc_versions, - ), + _dist_config_setting( + name = prefix + suffix, + flag_values = { + Label("//python/config_settings:python_version_major_minor"): python_version, + }, config_settings = config_settings, constraint_values = constraint_values, - python_version = python_version, **kwargs ) -def _dist_config_settings(*, suffix, plat_flag_values, python_version, **kwargs): - flag_values = { - Label("//python/config_settings:python_version_major_minor"): python_version, - } - - cpv = "cp" + python_version.replace(".", "") - prefix = "is_{}".format(cpv) - - _dist_config_setting( - name = prefix + suffix, - flag_values = flag_values, - **kwargs - ) - - flag_values[_flags.dist] = "" - - # First create an sdist, we will be building upon the flag values, which - # will ensure that each sdist config setting is the least specialized of - # all. However, we need at least one flag value to cover the case where we - # have `sdist` for any platform, hence we have a non-empty `flag_values` - # here. - _dist_config_setting( - name = "{}_sdist{}".format(prefix, suffix), - flag_values = flag_values, - compatible_with = (FLAGS.is_pip_whl_no, FLAGS.is_pip_whl_auto), - **kwargs - ) - - used_flags = {} - - # NOTE @aignas 2024-12-01: the abi3 is not compatible with freethreaded - # builds as per PEP703 (https://peps.python.org/pep-0703/#backwards-compatibility) - # - # The discussion here also reinforces this notion: - # https://discuss.python.org/t/pep-703-making-the-global-interpreter-lock-optional-3-12-updates/26503/99 - - for name, f, compatible_with in [ - ("py_none", _flags.whl, None), - ("py3_none", _flags.whl_py3, None), - ("py3_abi3", _flags.whl_py3_abi3, (FLAGS._is_py_freethreaded_no,)), - ("none", _flags.whl_pycp3x, None), - ("abi3", _flags.whl_pycp3x_abi3, (FLAGS._is_py_freethreaded_no,)), - # The below are not specializations of one another, they are variants - (cpv, _flags.whl_pycp3x_abicp, (FLAGS._is_py_freethreaded_no,)), - (cpv + "t", _flags.whl_pycp3x_abicp, (FLAGS._is_py_freethreaded_yes,)), - ]: - if (f, compatible_with) in used_flags: - # This should never happen as all of the different whls should have - # unique flag values - fail("BUG: the flag {} is attempted to be added twice to the list".format(f)) - else: - flag_values[f] = "yes" if f == _flags.whl else "" - used_flags[(f, compatible_with)] = True - - _dist_config_setting( - name = "{}_{}_any{}".format(prefix, name, suffix), - flag_values = flag_values, - compatible_with = compatible_with, - **kwargs - ) - - generic_flag_values = flag_values - generic_used_flags = used_flags - - for (suffix, flag_values) in plat_flag_values: - used_flags = {(f, None): True for f in flag_values} | generic_used_flags - flag_values = flag_values | generic_flag_values - - for name, f, compatible_with in [ - ("py_none", _flags.whl_plat, None), - ("py3_none", _flags.whl_plat_py3, None), - ("py3_abi3", _flags.whl_plat_py3_abi3, (FLAGS._is_py_freethreaded_no,)), - ("none", _flags.whl_plat_pycp3x, None), - ("abi3", _flags.whl_plat_pycp3x_abi3, (FLAGS._is_py_freethreaded_no,)), - # The below are not specializations of one another, they are variants - (cpv, _flags.whl_plat_pycp3x_abicp, (FLAGS._is_py_freethreaded_no,)), - (cpv + "t", _flags.whl_plat_pycp3x_abicp, (FLAGS._is_py_freethreaded_yes,)), - ]: - if (f, compatible_with) in used_flags: - # This should never happen as all of the different whls should have - # unique flag values. - fail("BUG: the flag {} is attempted to be added twice to the list".format(f)) - else: - flag_values[f] = "" - used_flags[(f, compatible_with)] = True - - _dist_config_setting( - name = "{}_{}_{}".format(prefix, name, suffix), - flag_values = flag_values, - compatible_with = compatible_with, - **kwargs - ) - -def _to_version_string(version, sep = "."): - if not version: - return "" - - return "{}{}{}".format(version[0], sep, version[1]) - -def _plat_flag_values(os, cpu, osx_versions, glibc_versions, muslc_versions): - ret = [] - if os == "": - return [] - elif os == "windows": - ret.append(("{}_{}".format(os, cpu), {})) - elif os == "osx": - for osx_version in osx_versions: - flags = { - FLAGS.pip_whl_osx_version: _to_version_string(osx_version), - } - if cpu != "universal2": - flags[FLAGS.pip_whl_osx_arch] = UniversalWhlFlag.ARCH - - if not osx_version: - suffix = "{}_{}".format(os, cpu) - else: - suffix = "{}_{}_{}".format(os, _to_version_string(osx_version, "_"), cpu) - - ret.append((suffix, flags)) - - elif os == "linux": - for os_prefix, linux_libc in { - os: LibcFlag.GLIBC, - "many" + os: LibcFlag.GLIBC, - "musl" + os: LibcFlag.MUSL, - }.items(): - if linux_libc == LibcFlag.GLIBC: - libc_versions = glibc_versions - libc_flag = FLAGS.pip_whl_glibc_version - elif linux_libc == LibcFlag.MUSL: - libc_versions = muslc_versions - libc_flag = FLAGS.pip_whl_muslc_version - else: - fail("Unsupported libc type: {}".format(linux_libc)) - - for libc_version in libc_versions: - if libc_version and os_prefix == os: - continue - elif libc_version: - suffix = "{}_{}_{}".format(os_prefix, _to_version_string(libc_version, "_"), cpu) - else: - suffix = "{}_{}".format(os_prefix, cpu) - - ret.append(( - suffix, - { - FLAGS.py_linux_libc: linux_libc, - libc_flag: _to_version_string(libc_version), - }, - )) - else: - fail("Unsupported os: {}".format(os)) - - return ret - -def _dist_config_setting(*, name, compatible_with = None, selects = selects, native = native, config_settings = None, **kwargs): +def _dist_config_setting(*, name, selects = selects, native = native, config_settings = None, **kwargs): """A macro to create a target for matching Python binary and source distributions. Args: name: The name of the public target. - compatible_with: {type}`tuple[Label]` A collection of config settings that are - compatible with the given dist config setting. For example, if only - non-freethreaded python builds are allowed, add - FLAGS._is_py_freethreaded_no here. config_settings: {type}`list[str | Label]` the list of target settings that must be matched before we try to evaluate the config_setting that we may create in this function. @@ -352,18 +96,6 @@ def _dist_config_setting(*, name, compatible_with = None, selects = selects, nat **kwargs: The kwargs passed to the config_setting rule. Visibility of the main alias target is also taken from the kwargs. """ - if compatible_with: - dist_config_setting_name = "_" + name - native.alias( - name = name, - actual = select( - {setting: dist_config_setting_name for setting in compatible_with} | { - _DEFAULT: _INCOMPATIBLE, - }, - ), - visibility = kwargs.get("visibility"), - ) - name = dist_config_setting_name # first define the config setting that has all of the constraint values _name = "_" + name diff --git a/python/private/pypi/hub_repository.bzl b/python/private/pypi/hub_repository.bzl index 75f3ec98d7..1d572d09e2 100644 --- a/python/private/pypi/hub_repository.bzl +++ b/python/private/pypi/hub_repository.bzl @@ -142,10 +142,6 @@ def whl_config_settings_to_json(repo_mapping): def _whl_config_setting_dict(a): ret = {} - if a.config_setting: - ret["config_setting"] = a.config_setting - if a.filename: - ret["filename"] = a.filename if a.target_platforms: ret["target_platforms"] = a.target_platforms if a.version: diff --git a/python/private/pypi/pkg_aliases.bzl b/python/private/pypi/pkg_aliases.bzl index 4d3cc61590..67ce297466 100644 --- a/python/private/pypi/pkg_aliases.bzl +++ b/python/private/pypi/pkg_aliases.bzl @@ -23,54 +23,12 @@ Definitions: :suffix: Can be either empty or `__`, which is usually used to distinguish multiple versions used for different target platforms. :os: OS identifier that exists in `@platforms//os:`. :cpu: CPU architecture identifier that exists in `@platforms//cpu:`. -:python_tag: The Python tag as defined by the [Python Packaging Authority][packaging_spec]. E.g. `py2.py3`, `py3`, `py311`, `cp311`. -:abi_tag: The ABI tag as defined by the [Python Packaging Authority][packaging_spec]. E.g. `none`, `abi3`, `cp311`, `cp311t`. -:platform_tag: The Platform tag as defined by the [Python Packaging Authority][packaging_spec]. E.g. `manylinux_2_17_x86_64`. -:platform_suffix: is a derivative of the `platform_tag` and is used to implement selection based on `libc` or `osx` version. All of the config settings used by this macro are generated by {obj}`config_settings`, for more detailed documentation on what each config setting maps to and their precedence, refer to documentation on that page. -The first group of config settings that are as follows: - -* `//_config:is_cp3` is used to select legacy `pip` - based `whl` and `sdist` {obj}`whl_library` instances. Whereas other config - settings are created when {obj}`pip.parse.experimental_index_url` is used. -* `//_config:is_cp3_sdist` is for wheels built from - `sdist` in {obj}`whl_library`. -* `//_config:is_cp3_py__any` for wheels with - `py2.py3` `python_tag` value. -* `//_config:is_cp3_py3__any` for wheels with - `py3` `python_tag` value. -* `//_config:is_cp3__any` for any other wheels. -* `//_config:is_cp3_py__` for - platform-specific wheels with `py2.py3` `python_tag` value. -* `//_config:is_cp3_py3__` for - platform-specific wheels with `py3` `python_tag` value. -* `//_config:is_cp3__` for any other - platform-specific wheels. - -Note that wheels with `abi3` or `none` `abi_tag` values and `python_tag` values -other than `py2.py3` or `py3` are compatible with the python version that is -equal or higher than the one denoted in the `python_tag`. For example: `py37` -and `cp37` wheels are compatible with Python 3.7 and above and in the case of -the target python version being `3.11`, `rules_python` will use -`//_config:is_cp311__any` config settings. - -For platform-specific wheels, i.e. the ones that have their `platform_tag` as -something else than `any`, we treat them as below: -* `linux_` tags assume that the target `libc` flavour is `glibc`, so this - is in many ways equivalent to it being `manylinux`, but with an unspecified - `libc` version. -* For `osx` and `linux` OSes wheel filename will be mapped to multiple config settings: - * `osx_` and `osx___` where - `major_version` and `minor_version` are the compatible OSX versions. - * `linux_` and - `linux___` where the version - identifiers are the compatible libc versions. - -[packaging_spec]: https://packaging.python.org/en/latest/specifications/platform-compatibility-tags/ +`//_config:is_cp3` is used to select any target platforms. """ load("@bazel_skylib//lib:selects.bzl", "selects") @@ -85,14 +43,6 @@ load( "WHEEL_FILE_IMPL_LABEL", "WHEEL_FILE_PUBLIC_LABEL", ) -load(":parse_whl_name.bzl", "parse_whl_name") -load(":whl_target_platforms.bzl", "whl_target_platforms") - -# This value is used as sentinel value in the alias/config setting machinery -# for libc and osx versions. If we encounter this version in this part of the -# code, then it means that we have a bug in rules_python and that we should fix -# it. It is more of an internal consistency check. -_VERSION_NONE = (0, 0) _NO_MATCH_ERROR_TEMPLATE = """\ No matching wheel for current configuration's Python version. @@ -137,7 +87,7 @@ def pkg_aliases( to bazel skylib's `selects.with_or`, so they can be tuples as well. group_name: {type}`str` The group name that the pkg belongs to. extra_aliases: {type}`list[str]` The extra aliases to be created. - **kwargs: extra kwargs to pass to {bzl:obj}`get_filename_config_settings`. + **kwargs: extra kwargs to pass to {bzl:obj}`get_config_settings`. """ alias = kwargs.pop("native", native).alias select = kwargs.pop("select", selects.with_or) @@ -219,21 +169,9 @@ def pkg_aliases( actual = "//_groups:{}_whl".format(group_name), ) -def _normalize_versions(name, versions): - if not versions: - return [] - - if _VERSION_NONE in versions: - fail("a sentinel version found in '{}', check render_pkg_aliases for bugs".format(name)) - - return sorted(versions) - def multiplatform_whl_aliases( *, - aliases = [], - glibc_versions = [], - muslc_versions = [], - osx_versions = []): + aliases = []): """convert a list of aliases from filename to config_setting ones. Exposed only for unit tests. @@ -243,12 +181,6 @@ def multiplatform_whl_aliases( to process. Any aliases that have the filename set will be converted to a dict of config settings to repo names. The struct is created by {func}`whl_config_setting`. - glibc_versions: {type}`list[tuple[int, int]]` list of versions that can be - used in this hub repo. - muslc_versions: {type}`list[tuple[int, int]]` list of versions that can be - used in this hub repo. - osx_versions: {type}`list[tuple[int, int]]` list of versions that can be - used in this hub repo. Returns: A dict with of config setting labels to repo names or the repo name itself. @@ -258,207 +190,54 @@ def multiplatform_whl_aliases( # We don't have any aliases, this is a repo name return aliases - # TODO @aignas 2024-11-17: we might be able to use FeatureFlagInfo and some - # code gen to create a version_lt_x target, which would allow us to check - # if the libc version is in a particular range. - glibc_versions = _normalize_versions("glibc_versions", glibc_versions) - muslc_versions = _normalize_versions("muslc_versions", muslc_versions) - osx_versions = _normalize_versions("osx_versions", osx_versions) - ret = {} - versioned_additions = {} for alias, repo in aliases.items(): if type(alias) != "struct": ret[alias] = repo continue - elif not (alias.filename or alias.target_platforms): - # This is an internal consistency check - fail("Expected to have either 'filename' or 'target_platforms' set, got: {}".format(alias)) - config_settings, all_versioned_settings = get_filename_config_settings( - filename = alias.filename or "", + config_settings = get_config_settings( target_platforms = alias.target_platforms, python_version = alias.version, - # If we have multiple platforms but no wheel filename, lets use different - # config settings. - non_whl_prefix = "sdist" if alias.filename else "", - glibc_versions = glibc_versions, - muslc_versions = muslc_versions, - osx_versions = osx_versions, ) for setting in config_settings: ret["//_config" + setting] = repo - # Now for the versioned platform config settings, we need to select one - # that best fits the bill and if there are multiple wheels, e.g. - # manylinux_2_17_x86_64 and manylinux_2_28_x86_64, then we need to select - # the former when the glibc is in the range of [2.17, 2.28) and then chose - # the later if it is [2.28, ...). If the 2.28 wheel was not present in - # the hub, then we would need to use 2.17 for all the glibc version - # configurations. - # - # Here we add the version settings to a dict where we key the range of - # versions that the whl spans. If the wheel supports musl and glibc at - # the same time, we do this for each supported platform, hence the - # double dict. - for default_setting, versioned in all_versioned_settings.items(): - versions = sorted(versioned) - min_version = versions[0] - max_version = versions[-1] - - versioned_additions.setdefault(default_setting, {})[(min_version, max_version)] = struct( - repo = repo, - settings = versioned, - ) - - versioned = {} - for default_setting, candidates in versioned_additions.items(): - # Sort the candidates by the range of versions the span, so that we - # start with the lowest version. - for _, candidate in sorted(candidates.items()): - # Set the default with the first candidate, which gives us the highest - # compatibility. If the users want to use a higher-version than the default - # they can configure the glibc_version flag. - versioned.setdefault("//_config" + default_setting, candidate.repo) - - # We will be overwriting previously added entries, but that is intended. - for _, setting in candidate.settings.items(): - versioned["//_config" + setting] = candidate.repo - - ret.update(versioned) return ret -def get_filename_config_settings( +def get_config_settings( *, - filename, target_platforms, - python_version, - glibc_versions = None, - muslc_versions = None, - osx_versions = None, - non_whl_prefix = "sdist"): + python_version): """Get the filename config settings. Exposed only for unit tests. Args: - filename: the distribution filename (can be a whl or an sdist). target_platforms: list[str], target platforms in "{abi}_{os}_{cpu}" format. - glibc_versions: list[tuple[int, int]], list of versions. - muslc_versions: list[tuple[int, int]], list of versions. - osx_versions: list[tuple[int, int]], list of versions. python_version: the python version to generate the config_settings for. - non_whl_prefix: the prefix of the config setting when the whl we don't have - a filename ending with ".whl". Returns: A tuple: * A list of config settings that are generated by ./pip_config_settings.bzl * The list of default version settings. """ - prefixes = [] - suffixes = [] - setting_supported_versions = {} - - if filename.endswith(".whl"): - parsed = parse_whl_name(filename) - if parsed.python_tag == "py2.py3": - py = "py_" - elif parsed.python_tag == "py3": - py = "py3_" - elif parsed.python_tag.startswith("cp"): - py = "" - else: - py = "py3_" - abi = parsed.abi_tag - - # TODO @aignas 2025-04-20: test - abi, _, _ = abi.partition(".") - - if parsed.platform_tag == "any": - prefixes = ["{}{}_any".format(py, abi)] - else: - prefixes = ["{}{}".format(py, abi)] - suffixes = _whl_config_setting_suffixes( - platform_tag = parsed.platform_tag, - glibc_versions = glibc_versions, - muslc_versions = muslc_versions, - osx_versions = osx_versions, - setting_supported_versions = setting_supported_versions, - ) - else: - prefixes = [non_whl_prefix or ""] - - py = "cp{}".format(python_version).replace(".", "") prefixes = [ - "{}_{}".format(py, prefix) if prefix else py - for prefix in prefixes + "cp{}".format(python_version).replace(".", ""), ] - versioned = { - ":is_{}_{}".format(prefix, suffix): { - version: ":is_{}_{}".format(prefix, setting) - for version, setting in versions.items() - } - for prefix in prefixes - for suffix, versions in setting_supported_versions.items() - } - - if suffixes or target_platforms or versioned: + if target_platforms: target_platforms = target_platforms or [] - suffixes = suffixes or [_non_versioned_platform(p) for p in target_platforms] + suffixes = [_non_versioned_platform(p) for p in target_platforms] return [ ":is_{}_{}".format(prefix, suffix) for prefix in prefixes for suffix in suffixes - ], versioned + ] else: - return [":is_{}".format(p) for p in prefixes], setting_supported_versions - -def _whl_config_setting_suffixes( - platform_tag, - glibc_versions, - muslc_versions, - osx_versions, - setting_supported_versions): - suffixes = [] - for platform_tag in platform_tag.split("."): - for p in whl_target_platforms(platform_tag): - prefix = p.os - suffix = p.cpu - if "manylinux" in platform_tag: - prefix = "manylinux" - versions = glibc_versions - elif "musllinux" in platform_tag: - prefix = "musllinux" - versions = muslc_versions - elif p.os in ["linux", "windows"]: - versions = [(0, 0)] - elif p.os == "osx": - versions = osx_versions - if "universal2" in platform_tag: - suffix = "universal2" - else: - fail("Unsupported whl os: {}".format(p.os)) - - default_version_setting = "{}_{}".format(prefix, suffix) - supported_versions = {} - for v in versions: - if v == (0, 0): - suffixes.append(default_version_setting) - elif v >= p.version: - supported_versions[v] = "{}_{}_{}_{}".format( - prefix, - v[0], - v[1], - suffix, - ) - if supported_versions: - setting_supported_versions[default_version_setting] = supported_versions - - return suffixes + return [":is_{}".format(p) for p in prefixes] def _non_versioned_platform(p, *, strict = False): """A small utility function that converts 'cp311_linux_x86_64' to 'linux_x86_64'. diff --git a/python/private/pypi/render_pkg_aliases.bzl b/python/private/pypi/render_pkg_aliases.bzl index e743fc20f7..0a1c328491 100644 --- a/python/private/pypi/render_pkg_aliases.bzl +++ b/python/private/pypi/render_pkg_aliases.bzl @@ -22,8 +22,6 @@ load( ":generate_group_library_build_bazel.bzl", "generate_group_library_build_bazel", ) # buildifier: disable=bzl-visibility -load(":parse_whl_name.bzl", "parse_whl_name") -load(":whl_target_platforms.bzl", "whl_target_platforms") NO_MATCH_ERROR_MESSAGE_TEMPLATE = """\ No matching wheel for current configuration's Python version. @@ -48,19 +46,17 @@ def _repr_dict(*, value_repr = repr, **kwargs): return {k: value_repr(v) for k, v in kwargs.items() if v} def _repr_config_setting(alias): - if alias.filename or alias.target_platforms: + if alias.target_platforms: return render.call( "whl_config_setting", **_repr_dict( - filename = alias.filename, target_platforms = alias.target_platforms, - config_setting = alias.config_setting, version = alias.version, ) ) else: return repr( - alias.config_setting or "//_config:is_cp{}".format(alias.version.replace(".", "")), + "//_config:is_cp{}".format(alias.version.replace(".", "")), ) def _repr_actual(aliases): @@ -179,15 +175,9 @@ def render_multiplatform_pkg_aliases(*, aliases, platform_config_settings = {}, contents = render_pkg_aliases( aliases = aliases, - glibc_versions = flag_versions.get("glibc_versions", []), - muslc_versions = flag_versions.get("muslc_versions", []), - osx_versions = flag_versions.get("osx_versions", []), **kwargs ) contents["_config/BUILD.bazel"] = _render_config_settings( - glibc_versions = flag_versions.get("glibc_versions", []), - muslc_versions = flag_versions.get("muslc_versions", []), - osx_versions = flag_versions.get("osx_versions", []), python_versions = _major_minor_versions(flag_versions.get("python_versions", [])), platform_config_settings = platform_config_settings, visibility = ["//:__subpackages__"], @@ -219,54 +209,21 @@ def get_whl_flag_versions(settings): * python_versions """ python_versions = {} - glibc_versions = {} target_platforms = {} - muslc_versions = {} - osx_versions = {} for setting in settings: - if not setting.version and not setting.filename: + if not setting.version: continue if setting.version: python_versions[setting.version] = None - if setting.filename and setting.filename.endswith(".whl") and not setting.filename.endswith("-any.whl"): - parsed = parse_whl_name(setting.filename) - else: - for plat in setting.target_platforms or []: - target_platforms[_non_versioned_platform(plat)] = None - continue - - for platform_tag in parsed.platform_tag.split("."): - parsed = whl_target_platforms(platform_tag) - - for p in parsed: - target_platforms[p.target_platform] = None - - if platform_tag.startswith("win") or platform_tag.startswith("linux"): - continue - - head, _, tail = platform_tag.partition("_") - major, _, tail = tail.partition("_") - minor, _, tail = tail.partition("_") - if tail: - version = (int(major), int(minor)) - if "many" in head: - glibc_versions[version] = None - elif "musl" in head: - muslc_versions[version] = None - elif "mac" in head: - osx_versions[version] = None - else: - fail(platform_tag) + for plat in setting.target_platforms or []: + target_platforms[_non_versioned_platform(plat)] = None return { k: sorted(v) for k, v in { - "glibc_versions": glibc_versions, - "muslc_versions": muslc_versions, - "osx_versions": osx_versions, "python_versions": python_versions, "target_platforms": target_platforms, }.items() diff --git a/python/private/pypi/whl_config_setting.bzl b/python/private/pypi/whl_config_setting.bzl index 3b81e4694f..1d868b1b65 100644 --- a/python/private/pypi/whl_config_setting.bzl +++ b/python/private/pypi/whl_config_setting.bzl @@ -14,7 +14,7 @@ "A small function to create an alias for a whl distribution" -def whl_config_setting(*, version = None, config_setting = None, filename = None, target_platforms = None): +def whl_config_setting(*, version = None, target_platforms = None): """The bzl_packages value used by by the render_pkg_aliases function. This contains the minimum amount of information required to generate correct @@ -25,15 +25,17 @@ def whl_config_setting(*, version = None, config_setting = None, filename = None whl alias is for. If not set, then non-version aware aliases will be constructed. This is mainly used for better error messages when there is no match found during a select. - config_setting: {type}`str | Label | None` the config setting that we should use. Defaults - to "//_config:is_python_{version}". - filename: {type}`str | None` the distribution filename to derive the config_setting. target_platforms: {type}`list[str] | None` the list of target_platforms for this distribution. Returns: a struct with the validated and parsed values. """ + + # FIXME @aignas 2025-07-26: There is still a potential that there will be ambigous match + # if the user is trying to have different packages for 3.X.Y and 3.X.Z versions of python + # in the same hub repository. Consider just removing this processing as I am not sure it + # has much value. if target_platforms: target_platforms_input = target_platforms target_platforms = [] @@ -50,8 +52,6 @@ def whl_config_setting(*, version = None, config_setting = None, filename = None target_platforms.append("{}_{}".format(abi, tail)) return struct( - config_setting = config_setting, - filename = filename, # Make the struct hashable target_platforms = tuple(target_platforms) if target_platforms else None, version = version, diff --git a/tests/pypi/config_settings/config_settings_tests.bzl b/tests/pypi/config_settings/config_settings_tests.bzl index a15f6b4d32..b3e6ada9e8 100644 --- a/tests/pypi/config_settings/config_settings_tests.bzl +++ b/tests/pypi/config_settings/config_settings_tests.bzl @@ -97,554 +97,6 @@ def _test_legacy_with_constraint_values(name): _tests.append(_test_legacy_with_constraint_values) -# Tests when we only have an `sdist` present. - -def _test_sdist_default(name): - _analysis_test( - name = name, - dist = { - "is_cp37_sdist": "sdist", - }, - want = "sdist", - ) - -_tests.append(_test_sdist_default) - -def _test_legacy_less_specialized_than_sdist(name): - _analysis_test( - name = name, - dist = { - "is_cp37": "legacy", - "is_cp37_sdist": "sdist", - }, - want = "sdist", - ) - -_tests.append(_test_legacy_less_specialized_than_sdist) - -def _test_sdist_no_whl(name): - _analysis_test( - name = name, - dist = { - "is_cp37_sdist": "sdist", - }, - config_settings = [ - _flag.platform("linux_aarch64"), - _flag.pip_whl("no"), - ], - want = "sdist", - ) - -_tests.append(_test_sdist_no_whl) - -def _test_sdist_no_sdist(name): - _analysis_test( - name = name, - dist = { - "is_cp37_sdist": "sdist", - }, - config_settings = [ - _flag.platform("linux_aarch64"), - _flag.pip_whl("only"), - ], - # We will use `no_match_error` in the real case to indicate that `sdist` is not - # allowed to be used. - want = "no_match", - ) - -_tests.append(_test_sdist_no_sdist) - -def _test_basic_whl_default(name): - _analysis_test( - name = name, - dist = { - "is_cp37_py_none_any": "whl", - "is_cp37_sdist": "sdist", - }, - want = "whl", - ) - -_tests.append(_test_basic_whl_default) - -def _test_basic_whl_nowhl(name): - _analysis_test( - name = name, - dist = { - "is_cp37_py_none_any": "whl", - "is_cp37_sdist": "sdist", - }, - config_settings = [ - _flag.platform("linux_aarch64"), - _flag.pip_whl("no"), - ], - want = "sdist", - ) - -_tests.append(_test_basic_whl_nowhl) - -def _test_basic_whl_nosdist(name): - _analysis_test( - name = name, - dist = { - "is_cp37_py_none_any": "whl", - "is_cp37_sdist": "sdist", - }, - config_settings = [ - _flag.platform("linux_aarch64"), - _flag.pip_whl("only"), - ], - want = "whl", - ) - -_tests.append(_test_basic_whl_nosdist) - -def _test_whl_default(name): - _analysis_test( - name = name, - dist = { - "is_cp37_py3_none_any": "whl", - "is_cp37_py_none_any": "basic_whl", - }, - want = "whl", - ) - -_tests.append(_test_whl_default) - -def _test_whl_nowhl(name): - _analysis_test( - name = name, - dist = { - "is_cp37_py3_none_any": "whl", - "is_cp37_py_none_any": "basic_whl", - }, - config_settings = [ - _flag.platform("linux_aarch64"), - _flag.pip_whl("no"), - ], - want = "no_match", - ) - -_tests.append(_test_whl_nowhl) - -def _test_whl_nosdist(name): - _analysis_test( - name = name, - dist = { - "is_cp37_py3_none_any": "whl", - }, - config_settings = [ - _flag.platform("linux_aarch64"), - _flag.pip_whl("only"), - ], - want = "whl", - ) - -_tests.append(_test_whl_nosdist) - -def _test_abi_whl_is_prefered(name): - _analysis_test( - name = name, - dist = { - "is_cp37_py3_abi3_any": "abi_whl", - "is_cp37_py3_none_any": "whl", - }, - want = "abi_whl", - ) - -_tests.append(_test_abi_whl_is_prefered) - -def _test_whl_with_constraints_is_prefered(name): - _analysis_test( - name = name, - dist = { - "is_cp37_py3_none_any": "default_whl", - "is_cp37_py3_none_any_linux_aarch64": "whl", - "is_cp37_py3_none_any_linux_x86_64": "amd64_whl", - }, - want = "whl", - ) - -_tests.append(_test_whl_with_constraints_is_prefered) - -def _test_cp_whl_is_prefered_over_py3(name): - _analysis_test( - name = name, - dist = { - "is_cp37_none_any": "cp", - "is_cp37_py3_abi3_any": "py3_abi3", - "is_cp37_py3_none_any": "py3", - }, - want = "cp", - ) - -_tests.append(_test_cp_whl_is_prefered_over_py3) - -def _test_cp_abi_whl_is_prefered_over_py3(name): - _analysis_test( - name = name, - dist = { - "is_cp37_abi3_any": "cp", - "is_cp37_py3_abi3_any": "py3", - }, - want = "cp", - ) - -_tests.append(_test_cp_abi_whl_is_prefered_over_py3) - -def _test_cp_version_is_selected_when_python_version_is_specified(name): - _analysis_test( - name = name, - dist = { - "is_cp310_none_any": "cp310", - "is_cp38_none_any": "cp38", - "is_cp39_none_any": "cp39", - }, - want = "cp310", - config_settings = [ - _flag.python_version("3.10.9"), - _flag.platform("linux_aarch64"), - ], - ) - -_tests.append(_test_cp_version_is_selected_when_python_version_is_specified) - -def _test_py_none_any_versioned(name): - _analysis_test( - name = name, - dist = { - "is_cp310_py_none_any": "whl", - "is_cp39_py_none_any": "too-low", - }, - want = "whl", - config_settings = [ - _flag.python_version("3.10.9"), - _flag.platform("linux_aarch64"), - ], - ) - -_tests.append(_test_py_none_any_versioned) - -def _test_cp_whl_is_not_prefered_over_py3_non_freethreaded(name): - _analysis_test( - name = name, - dist = { - "is_cp37_abi3_any": "py3_abi3", - "is_cp37_cp37t_any": "cp", - "is_cp37_none_any": "py3", - }, - want = "py3_abi3", - config_settings = [ - _flag.py_freethreaded("no"), - ], - ) - -_tests.append(_test_cp_whl_is_not_prefered_over_py3_non_freethreaded) - -def _test_cp_whl_is_not_prefered_over_py3_freethreaded(name): - _analysis_test( - name = name, - dist = { - "is_cp37_abi3_any": "py3_abi3", - "is_cp37_cp37_any": "cp", - "is_cp37_none_any": "py3", - }, - want = "py3", - config_settings = [ - _flag.py_freethreaded("yes"), - ], - ) - -_tests.append(_test_cp_whl_is_not_prefered_over_py3_freethreaded) - -def _test_cp_cp_whl(name): - _analysis_test( - name = name, - dist = { - "is_cp310_cp310_linux_aarch64": "whl", - }, - want = "whl", - config_settings = [ - _flag.python_version("3.10.9"), - _flag.platform("linux_aarch64"), - ], - ) - -_tests.append(_test_cp_cp_whl) - -def _test_cp_version_sdist_is_selected(name): - _analysis_test( - name = name, - dist = { - "is_cp310_sdist": "sdist", - }, - want = "sdist", - config_settings = [ - _flag.python_version("3.10.9"), - _flag.platform("linux_aarch64"), - ], - ) - -_tests.append(_test_cp_version_sdist_is_selected) - -# NOTE: Right now there is no way to get the following behaviour without -# breaking other tests. We need to choose either ta have the correct -# specialization behaviour between `is_cp37_cp37_any` and -# `is_cp37_cp37_any_linux_aarch64` or this commented out test case. -# -# I think having this behaviour not working is fine because the `suffix` -# will be either present on all of config settings of the same platform -# or none, because we use it as a way to select a separate version of the -# wheel for a single platform only. -# -# If we can think of a better way to handle it, then we can lift this -# limitation. -# -# def _test_any_whl_with_suffix_specialization(name): -# _analysis_test( -# name = name, -# dist = { -# "is_cp37_abi3_any_linux_aarch64": "abi3", -# "is_cp37_cp37_any": "cp37", -# }, -# want = "cp37", -# ) -# -# _tests.append(_test_any_whl_with_suffix_specialization) - -def _test_platform_vs_any_with_suffix_specialization(name): - _analysis_test( - name = name, - dist = { - "is_cp37_cp37_any_linux_aarch64": "any", - "is_cp37_py3_none_linux_aarch64": "platform_whl", - }, - want = "platform_whl", - ) - -_tests.append(_test_platform_vs_any_with_suffix_specialization) - -def _test_platform_whl_is_prefered_over_any_whl_with_constraints(name): - _analysis_test( - name = name, - dist = { - "is_cp37_py3_abi3_any": "better_default_whl", - "is_cp37_py3_abi3_any_linux_aarch64": "better_default_any_whl", - "is_cp37_py3_none_any": "default_whl", - "is_cp37_py3_none_any_linux_aarch64": "whl", - "is_cp37_py3_none_linux_aarch64": "platform_whl", - }, - want = "platform_whl", - ) - -_tests.append(_test_platform_whl_is_prefered_over_any_whl_with_constraints) - -def _test_abi3_platform_whl_preference(name): - _analysis_test( - name = name, - dist = { - "is_cp37_py3_abi3_linux_aarch64": "abi3_platform", - "is_cp37_py3_none_linux_aarch64": "platform", - }, - want = "abi3_platform", - ) - -_tests.append(_test_abi3_platform_whl_preference) - -def _test_glibc(name): - _analysis_test( - name = name, - dist = { - "is_cp37_cp37_manylinux_aarch64": "glibc", - "is_cp37_py3_abi3_linux_aarch64": "abi3_platform", - }, - want = "glibc", - ) - -_tests.append(_test_glibc) - -def _test_glibc_versioned(name): - _analysis_test( - name = name, - dist = { - "is_cp37_cp37_manylinux_2_14_aarch64": "glibc", - "is_cp37_cp37_manylinux_2_17_aarch64": "glibc", - "is_cp37_py3_abi3_linux_aarch64": "abi3_platform", - }, - want = "glibc", - config_settings = [ - _flag.py_linux_libc("glibc"), - _flag.pip_whl_glibc_version("2.17"), - _flag.platform("linux_aarch64"), - ], - ) - -_tests.append(_test_glibc_versioned) - -def _test_glibc_compatible_exists(name): - _analysis_test( - name = name, - dist = { - # Code using the conditions will need to construct selects, which - # do the version matching correctly. - "is_cp37_cp37_manylinux_2_14_aarch64": "2_14_whl_via_2_14_branch", - "is_cp37_cp37_manylinux_2_17_aarch64": "2_14_whl_via_2_17_branch", - }, - want = "2_14_whl_via_2_17_branch", - config_settings = [ - _flag.py_linux_libc("glibc"), - _flag.pip_whl_glibc_version("2.17"), - _flag.platform("linux_aarch64"), - ], - ) - -_tests.append(_test_glibc_compatible_exists) - -def _test_musl(name): - _analysis_test( - name = name, - dist = { - "is_cp37_cp37_musllinux_aarch64": "musl", - }, - want = "musl", - config_settings = [ - _flag.py_linux_libc("musl"), - _flag.platform("linux_aarch64"), - ], - ) - -_tests.append(_test_musl) - -def _test_windows(name): - _analysis_test( - name = name, - dist = { - "is_cp37_cp37_windows_x86_64": "whl", - "is_cp37_cp37t_windows_x86_64": "whl_freethreaded", - }, - want = "whl", - config_settings = [ - _flag.platform("windows_x86_64"), - ], - ) - -_tests.append(_test_windows) - -def _test_windows_freethreaded(name): - _analysis_test( - name = name, - dist = { - "is_cp37_cp37_windows_x86_64": "whl", - "is_cp37_cp37t_windows_x86_64": "whl_freethreaded", - }, - want = "whl_freethreaded", - config_settings = [ - _flag.platform("windows_x86_64"), - _flag.py_freethreaded("yes"), - ], - ) - -_tests.append(_test_windows_freethreaded) - -def _test_osx(name): - _analysis_test( - name = name, - dist = { - # We prefer arch specific whls over universal - "is_cp37_cp37_osx_universal2": "universal_whl", - "is_cp37_cp37_osx_x86_64": "whl", - }, - want = "whl", - config_settings = [ - _flag.platform("mac_x86_64"), - ], - ) - -_tests.append(_test_osx) - -def _test_osx_universal_default(name): - _analysis_test( - name = name, - dist = { - # We default to universal if only that exists - "is_cp37_cp37_osx_universal2": "whl", - }, - want = "whl", - config_settings = [ - _flag.platform("mac_x86_64"), - ], - ) - -_tests.append(_test_osx_universal_default) - -def _test_osx_universal_only(name): - _analysis_test( - name = name, - dist = { - # If we prefer universal, then we use that - "is_cp37_cp37_osx_universal2": "universal", - "is_cp37_cp37_osx_x86_64": "whl", - }, - want = "universal", - config_settings = [ - _flag.pip_whl_osx_arch("universal"), - _flag.platform("mac_x86_64"), - ], - ) - -_tests.append(_test_osx_universal_only) - -def _test_osx_os_version(name): - _analysis_test( - name = name, - dist = { - # Similarly to the libc version, the user of the config settings will have to - # construct the select so that the version selection is correct. - "is_cp37_cp37_osx_10_9_x86_64": "whl", - }, - want = "whl", - config_settings = [ - _flag.pip_whl_osx_version("10.9"), - _flag.platform("mac_x86_64"), - ], - ) - -_tests.append(_test_osx_os_version) - -def _test_all(name): - _analysis_test( - name = name, - dist = { - "is_cp37_" + f: f - for f in [ - "{py}{abi}_{plat}".format(py = valid_py, abi = valid_abi, plat = valid_plat) - # we have py2.py3, py3, cp3 - for valid_py in ["py_", "py3_", ""] - # cp abi usually comes with a version and we only need one - # config setting variant for all of them because the python - # version will discriminate between different versions. - for valid_abi in ["none", "abi3", "cp37"] - for valid_plat in [ - "any", - "manylinux_2_17_x86_64", - "manylinux_2_17_aarch64", - "osx_x86_64", - "windows_x86_64", - ] - if not ( - valid_abi == "abi3" and valid_py == "py_" or - valid_abi == "cp37" and valid_py != "" - ) - ] - }, - want = "cp37_manylinux_2_17_x86_64", - config_settings = [ - _flag.pip_whl_glibc_version("2.17"), - _flag.platform("linux_x86_64"), - ], - ) - -_tests.append(_test_all) - def config_settings_test_suite(name): # buildifier: disable=function-docstring test_suite( name = name, @@ -654,9 +106,6 @@ def config_settings_test_suite(name): # buildifier: disable=function-docstring config_settings( name = "dummy", python_versions = ["3.7", "3.8", "3.9", "3.10"], - glibc_versions = [(2, 14), (2, 17)], - muslc_versions = [(1, 1)], - osx_versions = [(10, 9), (11, 0)], platform_config_settings = { "linux_aarch64": [ "@platforms//cpu:aarch64", diff --git a/tests/pypi/extension/extension_tests.bzl b/tests/pypi/extension/extension_tests.bzl index ab8362ef0c..b85414528d 100644 --- a/tests/pypi/extension/extension_tests.bzl +++ b/tests/pypi/extension/extension_tests.bzl @@ -594,25 +594,25 @@ torch==2.4.1+cpu ; platform_machine == 'x86_64' \ "torch": { "pypi_312_torch_cp312_cp312_linux_x86_64_8800deef": [ whl_config_setting( - target_platforms = ["cp312_linux_x86_64"], + target_platforms = ("cp312_linux_x86_64",), version = "3.12", ), ], "pypi_312_torch_cp312_cp312_manylinux_2_17_aarch64_36109432": [ whl_config_setting( - target_platforms = ["cp312_linux_aarch64"], + target_platforms = ("cp312_linux_aarch64",), version = "3.12", ), ], "pypi_312_torch_cp312_cp312_win_amd64_3a570e5c": [ whl_config_setting( - target_platforms = ["cp312_windows_x86_64"], + target_platforms = ("cp312_windows_x86_64",), version = "3.12", ), ], "pypi_312_torch_cp312_none_macosx_11_0_arm64_72b484d5": [ whl_config_setting( - target_platforms = ["cp312_osx_aarch64"], + target_platforms = ("cp312_osx_aarch64",), version = "3.12", ), ], @@ -1085,8 +1085,6 @@ optimum[onnxruntime-gpu]==1.17.1 ; sys_platform == 'linux' "cp315_linux_aarch64", "cp315_linux_x86_64", ], - config_setting = None, - filename = None, ), ], "pypi_315_optimum_osx_aarch64": [ @@ -1095,8 +1093,6 @@ optimum[onnxruntime-gpu]==1.17.1 ; sys_platform == 'linux' target_platforms = [ "cp315_osx_aarch64", ], - config_setting = None, - filename = None, ), ], }, @@ -1127,7 +1123,7 @@ def _test_pipstar_platforms(env): name = "rules_python", default = [ _default( - platform = "my{}_{}".format(os, cpu), + platform = "my{}{}".format(os, cpu), os_name = os, arch_name = cpu, config_settings = [ @@ -1167,19 +1163,19 @@ optimum[onnxruntime-gpu]==1.17.1 ; sys_platform == 'linux' pypi.hub_whl_map().contains_exactly({ "pypi": { "optimum": { - "pypi_315_optimum_mylinux_x86_64": [ + "pypi_315_optimum_mylinuxx86_64": [ whl_config_setting( version = "3.15", target_platforms = [ - "cp315_mylinux_x86_64", + "cp315_mylinuxx86_64", ], ), ], - "pypi_315_optimum_myosx_aarch64": [ + "pypi_315_optimum_myosxaarch64": [ whl_config_setting( version = "3.15", target_platforms = [ - "cp315_myosx_aarch64", + "cp315_myosxaarch64", ], ), ], @@ -1188,12 +1184,12 @@ optimum[onnxruntime-gpu]==1.17.1 ; sys_platform == 'linux' }) pypi.whl_libraries().contains_exactly({ - "pypi_315_optimum_mylinux_x86_64": { + "pypi_315_optimum_mylinuxx86_64": { "dep_template": "@pypi//{name}:{target}", "python_interpreter_target": "unit_test_interpreter_target", "requirement": "optimum[onnxruntime-gpu]==1.17.1", }, - "pypi_315_optimum_myosx_aarch64": { + "pypi_315_optimum_myosxaarch64": { "dep_template": "@pypi//{name}:{target}", "python_interpreter_target": "unit_test_interpreter_target", "requirement": "optimum[onnxruntime]==1.17.1", diff --git a/tests/pypi/integration/BUILD.bazel b/tests/pypi/integration/BUILD.bazel index 9ea8dcebe4..316abe8271 100644 --- a/tests/pypi/integration/BUILD.bazel +++ b/tests/pypi/integration/BUILD.bazel @@ -1,20 +1,7 @@ load("@bazel_skylib//rules:build_test.bzl", "build_test") load("@rules_python_publish_deps//:requirements.bzl", "all_requirements") -load(":transitions.bzl", "transition_rule") build_test( name = "all_requirements_build_test", targets = all_requirements, ) - -# Rule that transitions dependencies to be built from sdist -transition_rule( - name = "all_requirements_from_sdist", - testonly = True, - deps = all_requirements, -) - -build_test( - name = "all_requirements_from_sdist_build_test", - targets = ["all_requirements_from_sdist"], -) diff --git a/tests/pypi/pkg_aliases/pkg_aliases_test.bzl b/tests/pypi/pkg_aliases/pkg_aliases_test.bzl index 3fd08c393c..6248261ed4 100644 --- a/tests/pypi/pkg_aliases/pkg_aliases_test.bzl +++ b/tests/pypi/pkg_aliases/pkg_aliases_test.bzl @@ -159,18 +159,12 @@ def _test_multiplatform_whl_aliases(env): name = "bar_baz", actual = { whl_config_setting( - filename = "foo-0.0.0-py3-none-any.whl", version = "3.9", - ): "filename_repo", + ): "version_repo", whl_config_setting( - filename = "foo-0.0.0-py3-none-any.whl", version = "3.9", target_platforms = ["cp39_linux_x86_64"], - ): "filename_repo_for_platform", - whl_config_setting( - version = "3.9", - target_platforms = ["cp39_linux_x86_64"], - ): "bzlmod_repo_for_a_particular_platform", + ): "version_platform_repo", "//:my_config_setting": "bzlmod_repo", }, extra_aliases = [], @@ -178,18 +172,14 @@ def _test_multiplatform_whl_aliases(env): alias = lambda *, name, actual, visibility = None, tags = None: got.update({name: actual}), ), select = mock_select, - glibc_versions = [], - muslc_versions = [], - osx_versions = [], ) # buildifier: disable=unsorted-dict-items want = { "pkg": { "//:my_config_setting": "@bzlmod_repo//:pkg", - "//_config:is_cp39_linux_x86_64": "@bzlmod_repo_for_a_particular_platform//:pkg", - "//_config:is_cp39_py3_none_any": "@filename_repo//:pkg", - "//_config:is_cp39_py3_none_any_linux_x86_64": "@filename_repo_for_platform//:pkg", + "//_config:is_cp39": "@version_repo//:pkg", + "//_config:is_cp39_linux_x86_64": "@version_platform_repo//:pkg", "//conditions:default": "_no_matching_repository", }, } @@ -198,9 +188,8 @@ def _test_multiplatform_whl_aliases(env): env.expect.that_str(actual_no_match_error[0]).contains("""\ configuration settings: //:my_config_setting + //_config:is_cp39 //_config:is_cp39_linux_x86_64 - //_config:is_cp39_py3_none_any - //_config:is_cp39_py3_none_any_linux_x86_64 """) @@ -279,7 +268,6 @@ _tests.append(_test_multiplatform_whl_aliases_nofilename) def _test_multiplatform_whl_aliases_nofilename_target_platforms(env): aliases = { whl_config_setting( - config_setting = "//:ignored", version = "3.1", target_platforms = [ "cp31_linux_x86_64", @@ -298,102 +286,6 @@ def _test_multiplatform_whl_aliases_nofilename_target_platforms(env): _tests.append(_test_multiplatform_whl_aliases_nofilename_target_platforms) -def _test_multiplatform_whl_aliases_filename(env): - aliases = { - whl_config_setting( - filename = "foo-0.0.3-py3-none-any.whl", - version = "3.2", - ): "foo-py3-0.0.3", - whl_config_setting( - filename = "foo-0.0.1-py3-none-any.whl", - version = "3.1", - ): "foo-py3-0.0.1", - whl_config_setting( - filename = "foo-0.0.1-cp313-cp313-any.whl", - version = "3.13", - ): "foo-cp-0.0.1", - whl_config_setting( - filename = "foo-0.0.1-cp313-cp313t-any.whl", - version = "3.13", - ): "foo-cpt-0.0.1", - whl_config_setting( - filename = "foo-0.0.2-py3-none-any.whl", - version = "3.1", - target_platforms = [ - "cp31_linux_x86_64", - "cp31_linux_aarch64", - ], - ): "foo-0.0.2", - } - got = multiplatform_whl_aliases( - aliases = aliases, - glibc_versions = [], - muslc_versions = [], - osx_versions = [], - ) - want = { - "//_config:is_cp313_cp313_any": "foo-cp-0.0.1", - "//_config:is_cp313_cp313t_any": "foo-cpt-0.0.1", - "//_config:is_cp31_py3_none_any": "foo-py3-0.0.1", - "//_config:is_cp31_py3_none_any_linux_aarch64": "foo-0.0.2", - "//_config:is_cp31_py3_none_any_linux_x86_64": "foo-0.0.2", - "//_config:is_cp32_py3_none_any": "foo-py3-0.0.3", - } - env.expect.that_dict(got).contains_exactly(want) - -_tests.append(_test_multiplatform_whl_aliases_filename) - -def _test_multiplatform_whl_aliases_filename_versioned(env): - aliases = { - whl_config_setting( - filename = "foo-0.0.1-py3-none-manylinux_2_17_x86_64.whl", - version = "3.1", - ): "glibc-2.17", - whl_config_setting( - filename = "foo-0.0.1-py3-none-manylinux_2_18_x86_64.whl", - version = "3.1", - ): "glibc-2.18", - whl_config_setting( - filename = "foo-0.0.1-py3-none-musllinux_1_1_x86_64.whl", - version = "3.1", - ): "musl-1.1", - } - got = multiplatform_whl_aliases( - aliases = aliases, - glibc_versions = [(2, 17), (2, 18)], - muslc_versions = [(1, 1), (1, 2)], - osx_versions = [], - ) - want = { - # This could just work with: - # select({ - # "//_config:is_gt_eq_2.18": "//_config:is_cp3.1_py3_none_manylinux_x86_64", - # "//conditions:default": "//_config:is_gt_eq_2.18", - # }): "glibc-2.18", - # select({ - # "//_config:is_range_2.17_2.18": "//_config:is_cp3.1_py3_none_manylinux_x86_64", - # "//_config:is_glibc_default": "//_config:is_cp3.1_py3_none_manylinux_x86_64", - # "//conditions:default": "//_config:is_glibc_default", - # }): "glibc-2.17", - # ( - # "//_config:is_gt_musl_1.1": "musl-1.1", - # "//_config:is_musl_default": "musl-1.1", - # ): "musl-1.1", - # - # For this to fully work we need to have the pypi:config_settings.bzl to generate the - # extra targets that use the FeatureFlagInfo and this to generate extra aliases for the - # config settings. - "//_config:is_cp31_py3_none_manylinux_2_17_x86_64": "glibc-2.17", - "//_config:is_cp31_py3_none_manylinux_2_18_x86_64": "glibc-2.18", - "//_config:is_cp31_py3_none_manylinux_x86_64": "glibc-2.17", - "//_config:is_cp31_py3_none_musllinux_1_1_x86_64": "musl-1.1", - "//_config:is_cp31_py3_none_musllinux_1_2_x86_64": "musl-1.1", - "//_config:is_cp31_py3_none_musllinux_x86_64": "musl-1.1", - } - env.expect.that_dict(got).contains_exactly(want) - -_tests.append(_test_multiplatform_whl_aliases_filename_versioned) - def _mock_alias(container): return lambda name, **kwargs: container.append(name) @@ -451,86 +343,6 @@ def _test_config_settings_exist_legacy(env): _tests.append(_test_config_settings_exist_legacy) -def _test_config_settings_exist(env): - for py_tag in ["py2.py3", "py3", "py311", "cp311"]: - if py_tag == "py2.py3": - abis = ["none"] - elif py_tag.startswith("py"): - abis = ["none", "abi3"] - else: - abis = ["none", "abi3", "cp311"] - - for abi_tag in abis: - for platform_tag, kwargs in { - "any": {}, - "macosx_11_0_arm64": { - "osx_versions": [(11, 0)], - "platform_config_settings": { - "osx_aarch64": [ - "@platforms//cpu:aarch64", - "@platforms//os:osx", - ], - }, - }, - "manylinux_2_17_x86_64": { - "glibc_versions": [(2, 17), (2, 18)], - "platform_config_settings": { - "linux_x86_64": [ - "@platforms//cpu:x86_64", - "@platforms//os:linux", - ], - }, - }, - "manylinux_2_18_x86_64": { - "glibc_versions": [(2, 17), (2, 18)], - "platform_config_settings": { - "linux_x86_64": [ - "@platforms//cpu:x86_64", - "@platforms//os:linux", - ], - }, - }, - "musllinux_1_1_aarch64": { - "muslc_versions": [(1, 2), (1, 1), (1, 0)], - "platform_config_settings": { - "linux_aarch64": [ - "@platforms//cpu:aarch64", - "@platforms//os:linux", - ], - }, - }, - }.items(): - aliases = { - whl_config_setting( - filename = "foo-0.0.1-{}-{}-{}.whl".format(py_tag, abi_tag, platform_tag), - version = "3.11", - ): "repo", - } - available_config_settings = [] - config_settings( - python_versions = ["3.11"], - native = struct( - alias = _mock_alias(available_config_settings), - config_setting = _mock_config_setting([]), - ), - selects = struct( - config_setting_group = _mock_config_setting_group(available_config_settings), - ), - **kwargs - ) - - got_aliases = multiplatform_whl_aliases( - aliases = aliases, - glibc_versions = kwargs.get("glibc_versions", []), - muslc_versions = kwargs.get("muslc_versions", []), - osx_versions = kwargs.get("osx_versions", []), - ) - got = [a.partition(":")[-1] for a in got_aliases] - - env.expect.that_collection(available_config_settings).contains_at_least(got) - -_tests.append(_test_config_settings_exist) - def pkg_aliases_test_suite(name): """Create the test suite. diff --git a/tests/pypi/render_pkg_aliases/render_pkg_aliases_test.bzl b/tests/pypi/render_pkg_aliases/render_pkg_aliases_test.bzl index ad7f36aed6..9114f59279 100644 --- a/tests/pypi/render_pkg_aliases/render_pkg_aliases_test.bzl +++ b/tests/pypi/render_pkg_aliases/render_pkg_aliases_test.bzl @@ -15,10 +15,6 @@ """render_pkg_aliases tests""" load("@rules_testing//lib:test_suite.bzl", "test_suite") -load( - "//python/private/pypi:pkg_aliases.bzl", - "get_filename_config_settings", -) # buildifier: disable=bzl-visibility load( "//python/private/pypi:render_pkg_aliases.bzl", "get_whl_flag_versions", @@ -70,26 +66,13 @@ def _test_bzlmod_aliases(env): whl_config_setting( # Add one with micro version to mimic construction in the extension version = "3.2.2", - config_setting = "//:my_config_setting", ): "pypi_32_bar_baz", whl_config_setting( version = "3.2", - config_setting = "//:my_config_setting", target_platforms = [ "cp32_linux_x86_64", ], ): "pypi_32_bar_baz_linux_x86_64", - whl_config_setting( - version = "3.2", - filename = "foo-0.0.0-py3-none-any.whl", - ): "filename_repo", - whl_config_setting( - version = "3.2.2", - filename = "foo-0.0.0-py3-none-any.whl", - target_platforms = [ - "cp32.2_linux_x86_64", - ], - ): "filename_repo_linux_x86_64", }, }, extra_hub_aliases = {"bar_baz": ["foo"]}, @@ -111,21 +94,11 @@ package(default_visibility = ["//visibility:public"]) pkg_aliases( name = "bar_baz", actual = { - "//:my_config_setting": "pypi_32_bar_baz", + "//_config:is_cp322": "pypi_32_bar_baz", whl_config_setting( target_platforms = ("cp32_linux_x86_64",), - config_setting = "//:my_config_setting", version = "3.2", ): "pypi_32_bar_baz_linux_x86_64", - whl_config_setting( - filename = "foo-0.0.0-py3-none-any.whl", - version = "3.2", - ): "filename_repo", - whl_config_setting( - filename = "foo-0.0.0-py3-none-any.whl", - target_platforms = ("cp32_linux_x86_64",), - version = "3.2.2", - ): "filename_repo_linux_x86_64", }, extra_aliases = ["foo"], )""" @@ -256,252 +229,24 @@ def _test_get_python_versions_with_target_platforms(env): _tests.append(_test_get_python_versions_with_target_platforms) -def _test_get_python_versions_from_filenames(env): - got = get_whl_flag_versions( - settings = [ - whl_config_setting( - version = "3.3", - filename = "foo-0.0.0-py3-none-" + plat + ".whl", - ) - for plat in [ - "linux_x86_64", - "manylinux_2_17_x86_64", - "manylinux_2_14_aarch64.musllinux_1_1_aarch64", - "musllinux_1_0_x86_64", - "manylinux2014_x86_64.manylinux_2_17_x86_64", - "macosx_11_0_arm64", - "macosx_10_9_x86_64", - "macosx_10_9_universal2", - "windows_x86_64", - ] - ], - ) - want = { - "glibc_versions": [(2, 14), (2, 17)], - "muslc_versions": [(1, 0), (1, 1)], - "osx_versions": [(10, 9), (11, 0)], - "python_versions": ["3.3"], - "target_platforms": [ - "linux_aarch64", - "linux_x86_64", - "osx_aarch64", - "osx_x86_64", - "windows_x86_64", - ], - } - env.expect.that_dict(got).contains_exactly(want) - -_tests.append(_test_get_python_versions_from_filenames) - def _test_get_flag_versions_from_alias_target_platforms(env): got = get_whl_flag_versions( settings = [ + whl_config_setting(version = "3.3"), whl_config_setting( version = "3.3", - filename = "foo-0.0.0-py3-none-" + plat + ".whl", - ) - for plat in [ - "windows_x86_64", - ] - ] + [ - whl_config_setting( - version = "3.3", - filename = "foo-0.0.0-py3-none-any.whl", - target_platforms = [ - "cp33_linux_x86_64", - ], + target_platforms = ["cp33_linux_x86_64"], ), ], ) want = { "python_versions": ["3.3"], - "target_platforms": [ - "linux_x86_64", - "windows_x86_64", - ], + "target_platforms": ["linux_x86_64"], } env.expect.that_dict(got).contains_exactly(want) _tests.append(_test_get_flag_versions_from_alias_target_platforms) -def _test_config_settings( - env, - *, - filename, - want, - python_version, - want_versions = {}, - target_platforms = [], - glibc_versions = [], - muslc_versions = [], - osx_versions = []): - got, got_default_version_settings = get_filename_config_settings( - filename = filename, - target_platforms = target_platforms, - glibc_versions = glibc_versions, - muslc_versions = muslc_versions, - osx_versions = osx_versions, - python_version = python_version, - ) - env.expect.that_collection(got).contains_exactly(want) - env.expect.that_dict(got_default_version_settings).contains_exactly(want_versions) - -def _test_sdist(env): - # Do the first test for multiple extensions - for ext in [".tar.gz", ".zip"]: - _test_config_settings( - env, - filename = "foo-0.0.1" + ext, - python_version = "3.2", - want = [":is_cp32_sdist"], - ) - - ext = ".zip" - _test_config_settings( - env, - filename = "foo-0.0.1" + ext, - python_version = "3.2", - target_platforms = [ - "linux_aarch64", - "linux_x86_64", - ], - want = [ - ":is_cp32_sdist_linux_aarch64", - ":is_cp32_sdist_linux_x86_64", - ], - ) - -_tests.append(_test_sdist) - -def _test_py2_py3_none_any(env): - _test_config_settings( - env, - filename = "foo-0.0.1-py2.py3-none-any.whl", - python_version = "3.2", - want = [ - ":is_cp32_py_none_any", - ], - ) - - _test_config_settings( - env, - filename = "foo-0.0.1-py2.py3-none-any.whl", - python_version = "3.2", - target_platforms = [ - "osx_x86_64", - ], - want = [":is_cp32_py_none_any_osx_x86_64"], - ) - -_tests.append(_test_py2_py3_none_any) - -def _test_py3_none_any(env): - _test_config_settings( - env, - filename = "foo-0.0.1-py3-none-any.whl", - python_version = "3.1", - want = [":is_cp31_py3_none_any"], - ) - - _test_config_settings( - env, - filename = "foo-0.0.1-py3-none-any.whl", - python_version = "3.1", - target_platforms = ["linux_x86_64"], - want = [":is_cp31_py3_none_any_linux_x86_64"], - ) - -_tests.append(_test_py3_none_any) - -def _test_py3_none_macosx_10_9_universal2(env): - _test_config_settings( - env, - filename = "foo-0.0.1-py3-none-macosx_10_9_universal2.whl", - python_version = "3.1", - osx_versions = [ - (10, 9), - (11, 0), - ], - want = [], - want_versions = { - ":is_cp31_py3_none_osx_universal2": { - (10, 9): ":is_cp31_py3_none_osx_10_9_universal2", - (11, 0): ":is_cp31_py3_none_osx_11_0_universal2", - }, - }, - ) - -_tests.append(_test_py3_none_macosx_10_9_universal2) - -def _test_cp37_abi3_linux_x86_64(env): - _test_config_settings( - env, - filename = "foo-0.0.1-cp37-abi3-linux_x86_64.whl", - python_version = "3.7", - want = [":is_cp37_abi3_linux_x86_64"], - ) - -_tests.append(_test_cp37_abi3_linux_x86_64) - -def _test_cp37_abi3_windows_x86_64(env): - _test_config_settings( - env, - filename = "foo-0.0.1-cp37-abi3-windows_x86_64.whl", - python_version = "3.7", - want = [":is_cp37_abi3_windows_x86_64"], - ) - -_tests.append(_test_cp37_abi3_windows_x86_64) - -def _test_cp37_abi3_manylinux_2_17_x86_64(env): - _test_config_settings( - env, - filename = "foo-0.0.1-cp37-abi3-manylinux2014_x86_64.manylinux_2_17_x86_64.whl", - python_version = "3.7", - glibc_versions = [ - (2, 16), - (2, 17), - (2, 18), - ], - want = [], - want_versions = { - ":is_cp37_abi3_manylinux_x86_64": { - (2, 17): ":is_cp37_abi3_manylinux_2_17_x86_64", - (2, 18): ":is_cp37_abi3_manylinux_2_18_x86_64", - }, - }, - ) - -_tests.append(_test_cp37_abi3_manylinux_2_17_x86_64) - -def _test_cp37_abi3_manylinux_2_17_musllinux_1_1_aarch64(env): - # I've seen such a wheel being built for `uv` - _test_config_settings( - env, - filename = "foo-0.0.1-cp37-cp37-manylinux_2_17_arm64.musllinux_1_1_arm64.whl", - python_version = "3.7", - glibc_versions = [ - (2, 16), - (2, 17), - (2, 18), - ], - muslc_versions = [ - (1, 1), - ], - want = [], - want_versions = { - ":is_cp37_cp37_manylinux_aarch64": { - (2, 17): ":is_cp37_cp37_manylinux_2_17_aarch64", - (2, 18): ":is_cp37_cp37_manylinux_2_18_aarch64", - }, - ":is_cp37_cp37_musllinux_aarch64": { - (1, 1): ":is_cp37_cp37_musllinux_1_1_aarch64", - }, - }, - ) - -_tests.append(_test_cp37_abi3_manylinux_2_17_musllinux_1_1_aarch64) - def render_pkg_aliases_test_suite(name): """Create the test suite. From e09a6f434ab6634a857b5303d088246ad0f940c2 Mon Sep 17 00:00:00 2001 From: Jason Bedard Date: Sat, 16 Aug 2025 20:28:03 -0700 Subject: [PATCH 390/922] deps(gazelle): upgrade rules_go to remove patching of tree-sitter (#3179) Update rules_go to include https://github.com/bazel-contrib/rules_go/pull/4298 Update gazelle to align with the version the rules_go bzlmod will bring in, and ensure the go.mod version is the same as bzlmod version. Update go to 1.21 to include the `slices` library that some of the go.mod updates depend on. Fixes #2956. --------- Co-authored-by: Douglas Thor --- CHANGELOG.md | 3 + examples/build_file_generation/WORKSPACE | 14 +-- .../bzlmod_build_file_generation/MODULE.bazel | 2 +- gazelle/MODULE.bazel | 15 +-- gazelle/WORKSPACE | 14 +-- gazelle/deps.bzl | 109 ++++++++++-------- gazelle/go.mod | 18 +-- gazelle/go.sum | 32 ++--- gazelle/internal/smacker_BUILD.bazel | 80 ------------- gazelle/python/BUILD.bazel | 6 +- .../testdata/respect_kind_mapping/BUILD.out | 2 +- 11 files changed, 109 insertions(+), 186 deletions(-) delete mode 100644 gazelle/internal/smacker_BUILD.bazel diff --git a/CHANGELOG.md b/CHANGELOG.md index 0e8ad65a5e..0fa5e373ed 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -54,6 +54,9 @@ END_UNRELEASED_TEMPLATE {#v0-0-0-changed} ### Changed +* (gazelle) update minimum gazelle version to 0.36.0 - may cause BUILD file changes +* (gazelle) update minimum rules_go version to 0.55.1 +* (gazelle) remove custom go-tree-sitter module BUILD file * (gazelle) For package mode, resolve dependencies when imports are relative to the package path. This is enabled via the `# gazelle:python_experimental_allow_relative_imports` true directive ({gh-issue}`2203`). diff --git a/examples/build_file_generation/WORKSPACE b/examples/build_file_generation/WORKSPACE index 6681ad6861..27f6ec071c 100644 --- a/examples/build_file_generation/WORKSPACE +++ b/examples/build_file_generation/WORKSPACE @@ -20,20 +20,20 @@ load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive") http_archive( name = "io_bazel_rules_go", - sha256 = "278b7ff5a826f3dc10f04feaf0b70d48b68748ccd512d7f98bf442077f043fe3", + sha256 = "9d72f7b8904128afb98d46bbef82ad7223ec9ff3718d419afb355fddd9f9484a", urls = [ - "https://mirror.bazel.build/github.com/bazelbuild/rules_go/releases/download/v0.41.0/rules_go-v0.41.0.zip", - "https://github.com/bazelbuild/rules_go/releases/download/v0.41.0/rules_go-v0.41.0.zip", + "https://mirror.bazel.build/github.com/bazel-contrib/rules_go/releases/download/v0.55.1/rules_go-v0.55.1.zip", + "https://github.com/bazel-contrib/rules_go/releases/download/v0.55.1/rules_go-v0.55.1.zip", ], ) # Download the bazel_gazelle ruleset. http_archive( name = "bazel_gazelle", - sha256 = "d3fa66a39028e97d76f9e2db8f1b0c11c099e8e01bf363a923074784e451f809", + sha256 = "75df288c4b31c81eb50f51e2e14f4763cb7548daae126817247064637fd9ea62", urls = [ - "https://mirror.bazel.build/github.com/bazelbuild/bazel-gazelle/releases/download/v0.33.0/bazel-gazelle-v0.33.0.tar.gz", - "https://github.com/bazelbuild/bazel-gazelle/releases/download/v0.33.0/bazel-gazelle-v0.33.0.tar.gz", + "https://mirror.bazel.build/github.com/bazelbuild/bazel-gazelle/releases/download/v0.36.0/bazel-gazelle-v0.36.0.tar.gz", + "https://github.com/bazelbuild/bazel-gazelle/releases/download/v0.36.0/bazel-gazelle-v0.36.0.tar.gz", ], ) @@ -49,7 +49,7 @@ go_rules_dependencies() # go_rules_dependencies is a function that registers external dependencies # needed by the Go rules. # See: https://github.com/bazelbuild/rules_go/blob/master/go/dependencies.rst#go_rules_dependencies -go_register_toolchains(version = "1.19.4") +go_register_toolchains(version = "1.21.13") # The following call configured the gazelle dependencies, Go environment and Go SDK. gazelle_dependencies() diff --git a/examples/bzlmod_build_file_generation/MODULE.bazel b/examples/bzlmod_build_file_generation/MODULE.bazel index b9b428d365..3436fbf0af 100644 --- a/examples/bzlmod_build_file_generation/MODULE.bazel +++ b/examples/bzlmod_build_file_generation/MODULE.bazel @@ -38,7 +38,7 @@ local_path_override( # The following stanza defines the dependency for gazelle # See here https://github.com/bazelbuild/bazel-gazelle/releases/ for the # latest version. -bazel_dep(name = "gazelle", version = "0.30.0", repo_name = "bazel_gazelle") +bazel_dep(name = "gazelle", version = "0.36.0", repo_name = "bazel_gazelle") # The following stanze returns a proxy object representing a module extension; # its methods can be invoked to create module extension tags. diff --git a/gazelle/MODULE.bazel b/gazelle/MODULE.bazel index 51352a0ba6..1560e73d7b 100644 --- a/gazelle/MODULE.bazel +++ b/gazelle/MODULE.bazel @@ -6,8 +6,8 @@ module( bazel_dep(name = "bazel_skylib", version = "1.6.1") bazel_dep(name = "rules_python", version = "0.18.0") -bazel_dep(name = "rules_go", version = "0.41.0", repo_name = "io_bazel_rules_go") -bazel_dep(name = "gazelle", version = "0.33.0", repo_name = "bazel_gazelle") +bazel_dep(name = "rules_go", version = "0.55.1", repo_name = "io_bazel_rules_go") +bazel_dep(name = "gazelle", version = "0.36.0", repo_name = "bazel_gazelle") bazel_dep(name = "rules_cc", version = "0.0.16") local_path_override( @@ -23,21 +23,12 @@ use_repo( "com_github_bmatcuk_doublestar_v4", "com_github_emirpasic_gods", "com_github_ghodss_yaml", + "com_github_smacker_go_tree_sitter", "com_github_stretchr_testify", "in_gopkg_yaml_v2", "org_golang_x_sync", ) -http_archive = use_repo_rule("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive") - -http_archive( - name = "com_github_smacker_go_tree_sitter", - build_file = "//:internal/smacker_BUILD.bazel", - integrity = "sha256-4AkDY4Rh5Auu9Kwzhj5XYSirMLlhmd6ClMWo/r0kmu4=", - strip_prefix = "go-tree-sitter-dd81d9e9be82a8cac96ed1d50c7389c5f1997c02", - url = "https://github.com/smacker/go-tree-sitter/archive/dd81d9e9be82a8cac96ed1d50c7389c5f1997c02.zip", -) - python_stdlib_list = use_extension("//python:extensions.bzl", "python_stdlib_list") use_repo( python_stdlib_list, diff --git a/gazelle/WORKSPACE b/gazelle/WORKSPACE index ad428b10cd..ec0532c3f6 100644 --- a/gazelle/WORKSPACE +++ b/gazelle/WORKSPACE @@ -4,19 +4,19 @@ load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive") http_archive( name = "io_bazel_rules_go", - sha256 = "278b7ff5a826f3dc10f04feaf0b70d48b68748ccd512d7f98bf442077f043fe3", + sha256 = "9d72f7b8904128afb98d46bbef82ad7223ec9ff3718d419afb355fddd9f9484a", urls = [ - "https://mirror.bazel.build/github.com/bazelbuild/rules_go/releases/download/v0.41.0/rules_go-v0.41.0.zip", - "https://github.com/bazelbuild/rules_go/releases/download/v0.41.0/rules_go-v0.41.0.zip", + "https://mirror.bazel.build/github.com/bazel-contrib/rules_go/releases/download/v0.55.1/rules_go-v0.55.1.zip", + "https://github.com/bazel-contrib/rules_go/releases/download/v0.55.1/rules_go-v0.55.1.zip", ], ) http_archive( name = "bazel_gazelle", - sha256 = "29d5dafc2a5582995488c6735115d1d366fcd6a0fc2e2a153f02988706349825", + sha256 = "75df288c4b31c81eb50f51e2e14f4763cb7548daae126817247064637fd9ea62", urls = [ - "https://mirror.bazel.build/github.com/bazelbuild/bazel-gazelle/releases/download/v0.31.0/bazel-gazelle-v0.31.0.tar.gz", - "https://github.com/bazelbuild/bazel-gazelle/releases/download/v0.31.0/bazel-gazelle-v0.31.0.tar.gz", + "https://mirror.bazel.build/github.com/bazelbuild/bazel-gazelle/releases/download/v0.36.0/bazel-gazelle-v0.36.0.tar.gz", + "https://github.com/bazelbuild/bazel-gazelle/releases/download/v0.36.0/bazel-gazelle-v0.36.0.tar.gz", ], ) @@ -25,7 +25,7 @@ load("@io_bazel_rules_go//go:deps.bzl", "go_register_toolchains", "go_rules_depe go_rules_dependencies() -go_register_toolchains(version = "1.19.4") +go_register_toolchains(version = "1.21.13") gazelle_dependencies() diff --git a/gazelle/deps.bzl b/gazelle/deps.bzl index 8c4c055e9b..a7b5990c3b 100644 --- a/gazelle/deps.bzl +++ b/gazelle/deps.bzl @@ -46,31 +46,28 @@ def go_deps(): go_repository( name = "com_github_bazelbuild_bazel_gazelle", importpath = "github.com/bazelbuild/bazel-gazelle", - sum = "h1:ROyUyUHzoEdvoOs1e0haxJx1l5EjZX6AOqiKdVlaBbg=", - version = "v0.31.1", + sum = "h1:n41ODckCkU9D2BEwBxYN+xu5E92Vd0gaW6QmsIW9l00=", + version = "v0.36.0", ) - go_repository( name = "com_github_bazelbuild_buildtools", build_naming_convention = "go_default_library", importpath = "github.com/bazelbuild/buildtools", - sum = "h1:HTepWP/jhtWTC1gvK0RnvKCgjh4gLqiwaOwGozAXcbw=", - version = "v0.0.0-20231103205921-433ea8554e82", + sum = "h1:VNqmvOfFzn2Hrtoni8vqgXlIQ4C2Zt22fxeZ9gOOkp0=", + version = "v0.0.0-20240313121412-66c605173954", ) go_repository( name = "com_github_bazelbuild_rules_go", importpath = "github.com/bazelbuild/rules_go", - sum = "h1:JzlRxsFNhlX+g4drDRPhIaU5H5LnI978wdMJ0vK4I+k=", - version = "v0.41.0", + sum = "h1:cQYGcunY8myOB+0Ym6PGQRhc/milkRcNv0my3XgxaDU=", + version = "v0.55.1", ) - go_repository( name = "com_github_bmatcuk_doublestar_v4", importpath = "github.com/bmatcuk/doublestar/v4", sum = "h1:fdDeAqgT47acgwd9bd9HxJRDmc9UAmPpc+2m0CXv75Q=", version = "v4.7.1", ) - go_repository( name = "com_github_burntsushi_toml", importpath = "github.com/BurntSushi/toml", @@ -134,16 +131,21 @@ def go_deps(): go_repository( name = "com_github_fsnotify_fsnotify", importpath = "github.com/fsnotify/fsnotify", - sum = "h1:n+5WquG0fcWoWp6xPWfHdbskMCQaFnG6PfBrh1Ky4HY=", - version = "v1.6.0", + sum = "h1:8JEhPFa5W2WU7YfeZzPNqzMP6Lwt7L2715Ggo0nosvA=", + version = "v1.7.0", ) - go_repository( name = "com_github_ghodss_yaml", importpath = "github.com/ghodss/yaml", sum = "h1:wQHKEahhL6wmXdzwWG11gIVCkOv05bNOh+Rxn0yngAk=", version = "v1.0.0", ) + go_repository( + name = "com_github_gogo_protobuf", + importpath = "github.com/gogo/protobuf", + sum = "h1:Ov1cvc58UF3b5XjBnZv7+opcTcQFZebYjWzi34vdm4Q=", + version = "v1.3.2", + ) go_repository( name = "com_github_golang_glog", importpath = "github.com/golang/glog", @@ -153,20 +155,20 @@ def go_deps(): go_repository( name = "com_github_golang_mock", importpath = "github.com/golang/mock", - sum = "h1:ErTB+efbowRARo13NNdxyJji2egdxLGQhRaY+DUumQc=", - version = "v1.6.0", + sum = "h1:YojYx61/OLFsiv6Rw1Z96LpldJIy31o+UHmwAUMJ6/U=", + version = "v1.7.0-rc.1", ) go_repository( name = "com_github_golang_protobuf", importpath = "github.com/golang/protobuf", - sum = "h1:ROPKBNFfQgOUMifHyP+KYbvpjbdoFNs+aK7DXlji0Tw=", - version = "v1.5.2", + sum = "h1:i7eJL8qZTpSEXOPTxNKhASYpMn+8e5Q6AdndVa1dWek=", + version = "v1.5.4", ) go_repository( name = "com_github_google_go_cmp", importpath = "github.com/google/go-cmp", - sum = "h1:O2Tfq5qg4qc4AmwVlvv0oLiVAGB7enBSJ2x2DqQFi38=", - version = "v0.5.9", + sum = "h1:ofyhxvXcZhMsU5ulbFiLKl/XBFqE1GSq7atu8tAmTRI=", + version = "v0.6.0", ) go_repository( name = "com_github_pmezard_go_difflib", @@ -180,12 +182,11 @@ def go_deps(): sum = "h1:gQz4mCbXsO+nc9n1hCxHcGA3Zx3Eo+UHZoInFGUIXNM=", version = "v0.0.0-20190812154241-14fe0d1b01d4", ) - http_archive( + go_repository( name = "com_github_smacker_go_tree_sitter", - build_file = Label("//:internal/smacker_BUILD.bazel"), - integrity = "sha256-4AkDY4Rh5Auu9Kwzhj5XYSirMLlhmd6ClMWo/r0kmu4=", - strip_prefix = "go-tree-sitter-dd81d9e9be82a8cac96ed1d50c7389c5f1997c02", - url = "https://github.com/smacker/go-tree-sitter/archive/dd81d9e9be82a8cac96ed1d50c7389c5f1997c02.zip", + importpath = "github.com/smacker/go-tree-sitter", + sum = "h1:6C8qej6f1bStuePVkLSFxoU22XBS165D3klxlzRg8F4=", + version = "v0.0.0-20240827094217-dd81d9e9be82", ) go_repository( name = "com_github_stretchr_objx", @@ -199,13 +200,6 @@ def go_deps(): sum = "h1:HtqpIVDClZ4nwg75+f6Lvsy/wHu+3BoSGCbBAcpTsTg=", version = "v1.9.0", ) - - go_repository( - name = "com_github_yuin_goldmark", - importpath = "github.com/yuin/goldmark", - sum = "h1:fVcFKWvrslecOb/tg+Cc05dkeYx540o0FuFt3nUVDoE=", - version = "v1.4.13", - ) go_repository( name = "com_google_cloud_go", importpath = "cloud.google.com/go", @@ -230,7 +224,6 @@ def go_deps(): sum = "h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=", version = "v3.0.1", ) - go_repository( name = "net_starlark_go", importpath = "go.starlark.net", @@ -246,20 +239,32 @@ def go_deps(): go_repository( name = "org_golang_google_genproto", importpath = "google.golang.org/genproto", - sum = "h1:+kGHl1aib/qcwaRi1CbqBZ1rk19r85MNUf8HaBghugY=", - version = "v0.0.0-20200526211855-cb27e3aa2013", + sum = "h1:387Y+JbxF52bmesc8kq1NyYIp33dnxCw6eiA7JMsTmw=", + version = "v0.0.0-20250115164207-1a7da9e5054f", + ) + go_repository( + name = "org_golang_google_genproto_googleapis_rpc", + importpath = "google.golang.org/genproto/googleapis/rpc", + sum = "h1:3UsHvIr4Wc2aW4brOaSCmcxh9ksica6fHEr8P1XhkYw=", + version = "v0.0.0-20250106144421-5f5ef82da422", ) go_repository( name = "org_golang_google_grpc", importpath = "google.golang.org/grpc", - sum = "h1:fPVVDxY9w++VjTZsYvXWqEf9Rqar/e+9zYfxKK+W+YU=", - version = "v1.50.0", + sum = "h1:OgPcDAFKHnH8X3O4WcO4XUc8GRDeKsKReqbQtiCj7N8=", + version = "v1.67.3", + ) + go_repository( + name = "org_golang_google_grpc_cmd_protoc_gen_go_grpc", + importpath = "google.golang.org/grpc/cmd/protoc-gen-go-grpc", + sum = "h1:F29+wU6Ee6qgu9TddPgooOdaqsxTMunOoj8KA5yuS5A=", + version = "v1.5.1", ) go_repository( name = "org_golang_google_protobuf", importpath = "google.golang.org/protobuf", - sum = "h1:w43yiav+6bVFTBQFZX0r7ipe9JQ1QsbMgHwbBziscLw=", - version = "v1.28.0", + sum = "h1:82DV7MYdb8anAVi3qge1wSnMDrnKK7ebr+I0hHRN1BU=", + version = "v1.36.3", ) go_repository( name = "org_golang_x_crypto", @@ -282,14 +287,14 @@ def go_deps(): go_repository( name = "org_golang_x_mod", importpath = "golang.org/x/mod", - sum = "h1:lFO9qtOdlre5W1jxS3r/4szv2/6iXxScdzjoBMXNhYk=", - version = "v0.10.0", + sum = "h1:Zb7khfcRGKk+kqfxFaP5tZqCnDZMjC5VtUBs87Hr6QM=", + version = "v0.23.0", ) go_repository( name = "org_golang_x_net", importpath = "golang.org/x/net", - sum = "h1:X2//UzNDwYmtCLn7To6G58Wr6f5ahEAQgKNzv9Y951M=", - version = "v0.10.0", + sum = "h1:T5GQRQb2y08kTAByq9L4/bz8cipCdA8FbRTXewonqY8=", + version = "v0.35.0", ) go_repository( name = "org_golang_x_oauth2", @@ -300,20 +305,20 @@ def go_deps(): go_repository( name = "org_golang_x_sync", importpath = "golang.org/x/sync", - sum = "h1:PUR+T4wwASmuSTYdKjYHI5TD22Wy5ogLU5qZCOLxBrI=", - version = "v0.2.0", + sum = "h1:GGz8+XQP4FvTTrjZPzNKTMFtSXH80RAzG+5ghFPgK9w=", + version = "v0.11.0", ) go_repository( name = "org_golang_x_sys", importpath = "golang.org/x/sys", - sum = "h1:EBmGv8NaZBZTWvrbjNoL6HVt+IVy3QDQpJs7VRIw3tU=", - version = "v0.8.0", + sum = "h1:QjkSwP/36a20jFYWkSue1YwXzLmsV5Gfq7Eiy72C1uc=", + version = "v0.30.0", ) go_repository( name = "org_golang_x_text", importpath = "golang.org/x/text", - sum = "h1:cokOdA+Jmi5PJGXLlLllQSgYigAEfHXJAERHVMaCc2k=", - version = "v0.3.3", + sum = "h1:bofq7m3/HAFvbF51jz3Q9wLg3jkvSPuiZu/pD1XwgtM=", + version = "v0.22.0", ) go_repository( name = "org_golang_x_tools", @@ -321,8 +326,14 @@ def go_deps(): "gazelle:exclude **/testdata/**/*", ], importpath = "golang.org/x/tools", - sum = "h1:8WMNJAz3zrtPmnYC7ISf5dEn3MT0gY7jBJfw27yrrLo=", - version = "v0.9.1", + sum = "h1:BgcpHewrV5AUp2G9MebG4XPFI1E2W41zU1SaqVA9vJY=", + version = "v0.30.0", + ) + go_repository( + name = "org_golang_x_tools_go_vcs", + importpath = "golang.org/x/tools/go/vcs", + sum = "h1:cOIJqWBl99H1dH5LWizPa+0ImeeJq3t3cJjaeOWUAL4=", + version = "v0.1.0-deprecated", ) go_repository( name = "org_golang_x_xerrors", diff --git a/gazelle/go.mod b/gazelle/go.mod index 6f65ffbc7e..7623079af9 100644 --- a/gazelle/go.mod +++ b/gazelle/go.mod @@ -1,26 +1,26 @@ module github.com/bazel-contrib/rules_python/gazelle -go 1.19 +go 1.21.13 require ( - github.com/bazelbuild/bazel-gazelle v0.31.1 - github.com/bazelbuild/buildtools v0.0.0-20231103205921-433ea8554e82 - github.com/bazelbuild/rules_go v0.41.0 + github.com/bazelbuild/bazel-gazelle v0.36.0 + github.com/bazelbuild/buildtools v0.0.0-20240313121412-66c605173954 + github.com/bazelbuild/rules_go v0.55.1 github.com/bmatcuk/doublestar/v4 v4.7.1 github.com/emirpasic/gods v1.18.1 github.com/ghodss/yaml v1.0.0 github.com/smacker/go-tree-sitter v0.0.0-20240827094217-dd81d9e9be82 github.com/stretchr/testify v1.9.0 - golang.org/x/sync v0.2.0 + golang.org/x/sync v0.11.0 gopkg.in/yaml.v2 v2.4.0 ) require ( github.com/davecgh/go-spew v1.1.1 // indirect - github.com/google/go-cmp v0.5.9 // indirect + github.com/google/go-cmp v0.6.0 // indirect github.com/pmezard/go-difflib v1.0.0 // indirect - golang.org/x/mod v0.10.0 // indirect - golang.org/x/sys v0.8.0 // indirect - golang.org/x/tools v0.9.1 // indirect + golang.org/x/mod v0.23.0 // indirect + golang.org/x/sys v0.30.0 // indirect + golang.org/x/tools/go/vcs v0.1.0-deprecated // indirect gopkg.in/yaml.v3 v3.0.1 // indirect ) diff --git a/gazelle/go.sum b/gazelle/go.sum index 0aaa186620..5a4d42d46a 100644 --- a/gazelle/go.sum +++ b/gazelle/go.sum @@ -1,11 +1,11 @@ cloud.google.com/go v0.26.0/go.mod h1:aQUYkXzVsufM+DwF1aE+0xfcU+56JwCaLick0ClmMTw= github.com/BurntSushi/toml v0.3.1/go.mod h1:xHWCNGjB5oqiDr8zfno3MHue2Ht5sIBksp03qcyfWMU= -github.com/bazelbuild/bazel-gazelle v0.31.1 h1:ROyUyUHzoEdvoOs1e0haxJx1l5EjZX6AOqiKdVlaBbg= -github.com/bazelbuild/bazel-gazelle v0.31.1/go.mod h1:Ul0pqz50f5wxz0QNzsZ+mrEu4AVAVJZEB5xLnHgIG9c= -github.com/bazelbuild/buildtools v0.0.0-20231103205921-433ea8554e82 h1:HTepWP/jhtWTC1gvK0RnvKCgjh4gLqiwaOwGozAXcbw= -github.com/bazelbuild/buildtools v0.0.0-20231103205921-433ea8554e82/go.mod h1:689QdV3hBP7Vo9dJMmzhoYIyo/9iMhEmHkJcnaPRCbo= -github.com/bazelbuild/rules_go v0.41.0 h1:JzlRxsFNhlX+g4drDRPhIaU5H5LnI978wdMJ0vK4I+k= -github.com/bazelbuild/rules_go v0.41.0/go.mod h1:TMHmtfpvyfsxaqfL9WnahCsXMWDMICTw7XeK9yVb+YU= +github.com/bazelbuild/bazel-gazelle v0.36.0 h1:n41ODckCkU9D2BEwBxYN+xu5E92Vd0gaW6QmsIW9l00= +github.com/bazelbuild/bazel-gazelle v0.36.0/go.mod h1:5wGHbkRpDUdz4LxREtPYwXstrWfnkV+oDmOuxNAxW1s= +github.com/bazelbuild/buildtools v0.0.0-20240313121412-66c605173954 h1:VNqmvOfFzn2Hrtoni8vqgXlIQ4C2Zt22fxeZ9gOOkp0= +github.com/bazelbuild/buildtools v0.0.0-20240313121412-66c605173954/go.mod h1:689QdV3hBP7Vo9dJMmzhoYIyo/9iMhEmHkJcnaPRCbo= +github.com/bazelbuild/rules_go v0.55.1 h1:cQYGcunY8myOB+0Ym6PGQRhc/milkRcNv0my3XgxaDU= +github.com/bazelbuild/rules_go v0.55.1/go.mod h1:T90Gpyq4HDFlsrvtQa2CBdHNJ2P4rAu/uUTmQbanzf0= github.com/bmatcuk/doublestar/v4 v4.7.1 h1:fdDeAqgT47acgwd9bd9HxJRDmc9UAmPpc+2m0CXv75Q= github.com/bmatcuk/doublestar/v4 v4.7.1/go.mod h1:xBQ8jztBU6kakFMg+8WGxn0c6z1fTSPVIjEY1Wr7jzc= github.com/census-instrumentation/opencensus-proto v0.2.1/go.mod h1:f6KPmirojxKA12rnyqOA5BBL4O983OfeGPqjHWSTneU= @@ -38,8 +38,8 @@ github.com/google/go-cmp v0.3.1/go.mod h1:8QqcDgzrUqlUb/G2PQTWiueGozuR1884gddMyw github.com/google/go-cmp v0.4.0/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE= github.com/google/go-cmp v0.5.0/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE= github.com/google/go-cmp v0.5.1/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE= -github.com/google/go-cmp v0.5.9 h1:O2Tfq5qg4qc4AmwVlvv0oLiVAGB7enBSJ2x2DqQFi38= -github.com/google/go-cmp v0.5.9/go.mod h1:17dUlkBOakJ0+DkrSSNjCkIjxS6bF9zb3elmeNGIjoY= +github.com/google/go-cmp v0.6.0 h1:ofyhxvXcZhMsU5ulbFiLKl/XBFqE1GSq7atu8tAmTRI= +github.com/google/go-cmp v0.6.0/go.mod h1:17dUlkBOakJ0+DkrSSNjCkIjxS6bF9zb3elmeNGIjoY= github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM= github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4= github.com/prometheus/client_model v0.0.0-20190812154241-14fe0d1b01d4/go.mod h1:xMI15A0UPsDsEKsMN9yxemIoYk6Tm2C1GtYGdfGttqA= @@ -53,8 +53,8 @@ golang.org/x/exp v0.0.0-20190121172915-509febef88a4/go.mod h1:CJ0aWSM057203Lf6IL golang.org/x/lint v0.0.0-20181026193005-c67002cb31c3/go.mod h1:UVdnD1Gm6xHRNCYTkRU2/jEulfH38KcIWyp/GAMgvoE= golang.org/x/lint v0.0.0-20190227174305-5b3e6a55c961/go.mod h1:wehouNa3lNwaWXcvxsM5YxQ5yQlVC4a0KAMCusXpPoU= golang.org/x/lint v0.0.0-20190313153728-d0100b6bd8b3/go.mod h1:6SW0HCj/g11FgYtHlgUYUwCkIfeOF89ocIRzGO/8vkc= -golang.org/x/mod v0.10.0 h1:lFO9qtOdlre5W1jxS3r/4szv2/6iXxScdzjoBMXNhYk= -golang.org/x/mod v0.10.0/go.mod h1:iBbtSCu2XBx23ZKBPSOrRkjjQPZFPuis4dIYUhu/chs= +golang.org/x/mod v0.23.0 h1:Zb7khfcRGKk+kqfxFaP5tZqCnDZMjC5VtUBs87Hr6QM= +golang.org/x/mod v0.23.0/go.mod h1:6SkKJ3Xj0I0BrPOZoBy3bdMptDDU9oJrpohJ3eWZ1fY= golang.org/x/net v0.0.0-20180724234803-3673e40ba225/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4= golang.org/x/net v0.0.0-20180826012351-8a410e7b638d/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4= golang.org/x/net v0.0.0-20190213061140-3a22650c66bd/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4= @@ -63,20 +63,20 @@ golang.org/x/oauth2 v0.0.0-20180821212333-d2e6202438be/go.mod h1:N/0e6XlmueqKjAG golang.org/x/sync v0.0.0-20180314180146-1d60e4601c6f/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= golang.org/x/sync v0.0.0-20181108010431-42b317875d0f/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= golang.org/x/sync v0.0.0-20190423024810-112230192c58/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= -golang.org/x/sync v0.2.0 h1:PUR+T4wwASmuSTYdKjYHI5TD22Wy5ogLU5qZCOLxBrI= -golang.org/x/sync v0.2.0/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= +golang.org/x/sync v0.11.0 h1:GGz8+XQP4FvTTrjZPzNKTMFtSXH80RAzG+5ghFPgK9w= +golang.org/x/sync v0.11.0/go.mod h1:Czt+wKu1gCyEFDUtn0jG5QVvpJ6rzVqr5aXyt9drQfk= golang.org/x/sys v0.0.0-20180830151530-49385e6e1522/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= golang.org/x/sys v0.0.0-20200930185726-fdedc70b468f/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= -golang.org/x/sys v0.8.0 h1:EBmGv8NaZBZTWvrbjNoL6HVt+IVy3QDQpJs7VRIw3tU= -golang.org/x/sys v0.8.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= +golang.org/x/sys v0.30.0 h1:QjkSwP/36a20jFYWkSue1YwXzLmsV5Gfq7Eiy72C1uc= +golang.org/x/sys v0.30.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA= golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ= golang.org/x/tools v0.0.0-20190114222345-bf090417da8b/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ= golang.org/x/tools v0.0.0-20190226205152-f727befe758c/go.mod h1:9Yl7xja0Znq3iFh3HoIrodX9oNMXvdceNzlUR8zjMvY= golang.org/x/tools v0.0.0-20190311212946-11955173bddd/go.mod h1:LCzVGOaR6xXOjkQ3onu1FJEFr0SW1gC7cKk1uF8kGRs= golang.org/x/tools v0.0.0-20190524140312-2c0ae7006135/go.mod h1:RgjU9mgBXZiqYHBnxXauZ1Gv1EHHAz9KjViQ78xBX0Q= -golang.org/x/tools v0.9.1 h1:8WMNJAz3zrtPmnYC7ISf5dEn3MT0gY7jBJfw27yrrLo= -golang.org/x/tools v0.9.1/go.mod h1:owI94Op576fPu3cIGQeHs3joujW/2Oc6MtlxbF5dfNc= +golang.org/x/tools/go/vcs v0.1.0-deprecated h1:cOIJqWBl99H1dH5LWizPa+0ImeeJq3t3cJjaeOWUAL4= +golang.org/x/tools/go/vcs v0.1.0-deprecated/go.mod h1:zUrvATBAvEI9535oC0yWYsLsHIV4Z7g63sNPVMtuBy8= golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0= golang.org/x/xerrors v0.0.0-20200804184101-5ec99f83aff1/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0= google.golang.org/appengine v1.1.0/go.mod h1:EbEs0AVv82hx2wNQdGPgUI5lhzA/G0D9YwlJXL52JkM= diff --git a/gazelle/internal/smacker_BUILD.bazel b/gazelle/internal/smacker_BUILD.bazel deleted file mode 100644 index 3ec96760e8..0000000000 --- a/gazelle/internal/smacker_BUILD.bazel +++ /dev/null @@ -1,80 +0,0 @@ -load("@io_bazel_rules_go//go:def.bzl", "go_library", "go_test") - -filegroup( - name = "common_libs", - srcs = [ - "alloc.h", - "api.h", - "array.h", - ], - visibility = [":__subpackages__"], -) - -go_library( - name = "go-tree-sitter", - srcs = [ - "alloc.c", - "alloc.h", - "api.h", - "array.h", - "atomic.h", - "bindings.c", - "bindings.go", - "bindings.h", - "bits.h", - "clock.h", - "error_costs.h", - "get_changed_ranges.c", - "get_changed_ranges.h", - "host.h", - "iter.go", - "language.c", - "language.h", - "length.h", - "lexer.c", - "lexer.h", - "node.c", - "parser.c", - "parser.h", - "point.h", - "ptypes.h", - "query.c", - "reduce_action.h", - "reusable_node.h", - "stack.c", - "stack.h", - "subtree.c", - "subtree.h", - "test_grammar.go", - "tree.c", - "tree.h", - "tree_cursor.c", - "tree_cursor.h", - "umachine.h", - "unicode.h", - "urename.h", - "utf.h", - "utf16.h", - "utf8.h", - "wasm_store.c", - "wasm_store.h", - ], - cgo = True, - importpath = "github.com/smacker/go-tree-sitter", - visibility = ["//visibility:public"], -) - -go_library( - name = "python", - srcs = [ - "python/binding.go", - "python/parser.c", - "python/parser.h", - "python/scanner.c", - ":common_libs", - ], - cgo = True, - importpath = "github.com/smacker/go-tree-sitter/python", - visibility = ["//visibility:public"], - deps = [":go-tree-sitter"], -) diff --git a/gazelle/python/BUILD.bazel b/gazelle/python/BUILD.bazel index 1a7c54f4b2..b6ca8adef5 100644 --- a/gazelle/python/BUILD.bazel +++ b/gazelle/python/BUILD.bazel @@ -29,22 +29,20 @@ go_library( importpath = "github.com/bazel-contrib/rules_python/gazelle/python", visibility = ["//visibility:public"], deps = [ - "//manifest", "//pythonconfig", "@bazel_gazelle//config:go_default_library", "@bazel_gazelle//label:go_default_library", "@bazel_gazelle//language:go_default_library", - "@bazel_gazelle//language/proto:go_default_library", "@bazel_gazelle//repo:go_default_library", "@bazel_gazelle//resolve:go_default_library", "@bazel_gazelle//rule:go_default_library", - "@com_github_bazelbuild_buildtools//build:go_default_library", + "@com_github_bazelbuild_buildtools//build", "@com_github_bmatcuk_doublestar_v4//:doublestar", "@com_github_emirpasic_gods//lists/singlylinkedlist", "@com_github_emirpasic_gods//sets/treeset", "@com_github_emirpasic_gods//utils", "@com_github_smacker_go_tree_sitter//:go-tree-sitter", - "@com_github_smacker_go_tree_sitter//:python", + "@com_github_smacker_go_tree_sitter//python", "@org_golang_x_sync//errgroup", ], ) diff --git a/gazelle/python/testdata/respect_kind_mapping/BUILD.out b/gazelle/python/testdata/respect_kind_mapping/BUILD.out index 7c5fb0bd20..fa06e2af12 100644 --- a/gazelle/python/testdata/respect_kind_mapping/BUILD.out +++ b/gazelle/python/testdata/respect_kind_mapping/BUILD.out @@ -1,5 +1,5 @@ -load(":mytest.bzl", "my_test") load("@rules_python//python:defs.bzl", "py_library") +load(":mytest.bzl", "my_test") # gazelle:map_kind py_test my_test :mytest.bzl From 5e750070e3bcf952f10c9f32d1da468c3c784f3a Mon Sep 17 00:00:00 2001 From: Ignas Anikevicius <240938+aignas@users.noreply.github.com> Date: Sun, 17 Aug 2025 15:44:00 +0900 Subject: [PATCH 391/922] fix(pypi): correctly handle different package versions (#3186) After merging #3058, I wanted to try creating a `universal` requirements file for our `sphinx` setup and have an integration test in that way. It seems that `alabaster` has different versions for different python versions and we were not handling this correctly. In #3058 I have attempted adding a unit test for this but it escaped me that the only time where this bug manifests itself is when the `parse_requirements` is used multiple times, once per each `python_version`. With this I think it is safe to say that the #2797 is fixed, because we found a bug, where we were not skipping requirements. As an added counter measure, I have added an extra check elsewhere to catch a regression. Fixes #2797 --- CHANGELOG.md | 3 + MODULE.bazel | 34 ++- docs/BUILD.bazel | 3 + docs/requirements.txt | 271 +++++++++++------- python/private/pypi/extension.bzl | 12 +- python/private/pypi/parse_requirements.bzl | 40 ++- python/private/pypi/pip_repository.bzl | 4 +- .../parse_requirements_tests.bzl | 67 ++++- 8 files changed, 294 insertions(+), 140 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 0fa5e373ed..2a5380677b 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -123,6 +123,9 @@ END_UNRELEASED_TEMPLATE installations (Mac frameworks, missing dynamic libraries, and other esoteric cases, see [#3148](https://github.com/bazel-contrib/rules_python/pull/3148) for details). +* (pypi) Support `requirements.txt` files that use different versions of the same + package targeting different target platforms. + ([#2797](https://github.com/bazel-contrib/rules_python/issues/2797)). {#v0-0-0-added} ### Added diff --git a/MODULE.bazel b/MODULE.bazel index 66297b99a1..b0b31dd73d 100644 --- a/MODULE.bazel +++ b/MODULE.bazel @@ -241,21 +241,25 @@ dev_pip = use_extension( "pip", dev_dependency = True, ) -dev_pip.parse( - download_only = True, - experimental_index_url = "https://pypi.org/simple", - hub_name = "dev_pip", - parallel_download = False, - python_version = "3.11", - requirements_lock = "//docs:requirements.txt", -) -dev_pip.parse( - download_only = True, - experimental_index_url = "https://pypi.org/simple", - hub_name = "dev_pip", - python_version = "3.13", - requirements_lock = "//docs:requirements.txt", -) + +[ + dev_pip.parse( + download_only = True, + experimental_index_url = "https://pypi.org/simple", + hub_name = "dev_pip", + parallel_download = False, + python_version = python_version, + requirements_lock = "//docs:requirements.txt", + ) + for python_version in [ + "3.9", + "3.10", + "3.11", + "3.12", + "3.13", + ] +] + dev_pip.parse( download_only = True, experimental_index_url = "https://pypi.org/simple", diff --git a/docs/BUILD.bazel b/docs/BUILD.bazel index fdb74f9407..b6c48b0539 100644 --- a/docs/BUILD.bazel +++ b/docs/BUILD.bazel @@ -186,5 +186,8 @@ lock( "--universal", "--upgrade", ], + # NOTE @aignas 2025-08-17: here we select the lowest actively supported version so that the + # requirements file is generated to be compatible with Python version 3.9 or greater. + python_version = "3.9", visibility = ["//:__subpackages__"], ) diff --git a/docs/requirements.txt b/docs/requirements.txt index af691dfd21..fc786fa9d2 100644 --- a/docs/requirements.txt +++ b/docs/requirements.txt @@ -1,11 +1,16 @@ # This file was autogenerated by uv via the following command: # bazel run //docs:requirements.update +--index-url https://pypi.org/simple absl-py==2.3.1 \ --hash=sha256:a97820526f7fbfd2ec1bce83f3f25e3a14840dac0d8e02a0b71cd75db3f77fc9 \ --hash=sha256:eeecf07f0c2a93ace0772c92e596ace6d3d3996c042b2128459aaae2a76de11d # via rules-python-docs (docs/pyproject.toml) -alabaster==1.0.0 \ +alabaster==0.7.16 ; python_full_version < '3.10' \ + --hash=sha256:75a8b99c28a5dad50dd7f8ccdd447a121ddb3892da9e53d1ca5cca3106d58d65 \ + --hash=sha256:b46733c07dce03ae4e150330b975c75737fa60f0a7c591b6c8bf4928a28e2c92 + # via sphinx +alabaster==1.0.0 ; python_full_version >= '3.10' \ --hash=sha256:c00dca57bca26fa62a6d7d0a9fcce65f3e026e9bfe33e9c538fd3fbb2144fd9e \ --hash=sha256:fc6786402dc3fcb2de3cabd5fe455a2db534b371124f1f21de8731783dec828b # via sphinx @@ -21,99 +26,86 @@ certifi==2025.8.3 \ --hash=sha256:e564105f78ded564e3ae7c923924435e1daa7463faeab5bb932bc53ffae63407 \ --hash=sha256:f6c12493cfb1b06ba2ff328595af9350c65d6644968e5d3a2ffd78699af217a5 # via requests -charset-normalizer==3.4.2 \ - --hash=sha256:005fa3432484527f9732ebd315da8da8001593e2cf46a3d817669f062c3d9ed4 \ - --hash=sha256:046595208aae0120559a67693ecc65dd75d46f7bf687f159127046628178dc45 \ - --hash=sha256:0c29de6a1a95f24b9a1aa7aefd27d2487263f00dfd55a77719b530788f75cff7 \ - --hash=sha256:0c8c57f84ccfc871a48a47321cfa49ae1df56cd1d965a09abe84066f6853b9c0 \ - --hash=sha256:0f5d9ed7f254402c9e7d35d2f5972c9bbea9040e99cd2861bd77dc68263277c7 \ - --hash=sha256:18dd2e350387c87dabe711b86f83c9c78af772c748904d372ade190b5c7c9d4d \ - --hash=sha256:1b1bde144d98e446b056ef98e59c256e9294f6b74d7af6846bf5ffdafd687a7d \ - --hash=sha256:1c95a1e2902a8b722868587c0e1184ad5c55631de5afc0eb96bc4b0d738092c0 \ - --hash=sha256:1cad5f45b3146325bb38d6855642f6fd609c3f7cad4dbaf75549bf3b904d3184 \ - --hash=sha256:21b2899062867b0e1fde9b724f8aecb1af14f2778d69aacd1a5a1853a597a5db \ - --hash=sha256:24498ba8ed6c2e0b56d4acbf83f2d989720a93b41d712ebd4f4979660db4417b \ - --hash=sha256:25a23ea5c7edc53e0f29bae2c44fcb5a1aa10591aae107f2a2b2583a9c5cbc64 \ - --hash=sha256:289200a18fa698949d2b39c671c2cc7a24d44096784e76614899a7ccf2574b7b \ - --hash=sha256:28a1005facc94196e1fb3e82a3d442a9d9110b8434fc1ded7a24a2983c9888d8 \ - --hash=sha256:32fc0341d72e0f73f80acb0a2c94216bd704f4f0bce10aedea38f30502b271ff \ - --hash=sha256:36b31da18b8890a76ec181c3cf44326bf2c48e36d393ca1b72b3f484113ea344 \ - --hash=sha256:3c21d4fca343c805a52c0c78edc01e3477f6dd1ad7c47653241cf2a206d4fc58 \ - --hash=sha256:3fddb7e2c84ac87ac3a947cb4e66d143ca5863ef48e4a5ecb83bd48619e4634e \ - --hash=sha256:43e0933a0eff183ee85833f341ec567c0980dae57c464d8a508e1b2ceb336471 \ - --hash=sha256:4a476b06fbcf359ad25d34a057b7219281286ae2477cc5ff5e3f70a246971148 \ - --hash=sha256:4e594135de17ab3866138f496755f302b72157d115086d100c3f19370839dd3a \ - --hash=sha256:50bf98d5e563b83cc29471fa114366e6806bc06bc7a25fd59641e41445327836 \ - --hash=sha256:5a9979887252a82fefd3d3ed2a8e3b937a7a809f65dcb1e068b090e165bbe99e \ - --hash=sha256:5baececa9ecba31eff645232d59845c07aa030f0c81ee70184a90d35099a0e63 \ - --hash=sha256:5bf4545e3b962767e5c06fe1738f951f77d27967cb2caa64c28be7c4563e162c \ - --hash=sha256:6333b3aa5a12c26b2a4d4e7335a28f1475e0e5e17d69d55141ee3cab736f66d1 \ - --hash=sha256:65c981bdbd3f57670af8b59777cbfae75364b483fa8a9f420f08094531d54a01 \ - --hash=sha256:68a328e5f55ec37c57f19ebb1fdc56a248db2e3e9ad769919a58672958e8f366 \ - --hash=sha256:6a0289e4589e8bdfef02a80478f1dfcb14f0ab696b5a00e1f4b8a14a307a3c58 \ - --hash=sha256:6b66f92b17849b85cad91259efc341dce9c1af48e2173bf38a85c6329f1033e5 \ - --hash=sha256:6c9379d65defcab82d07b2a9dfbfc2e95bc8fe0ebb1b176a3190230a3ef0e07c \ - --hash=sha256:6fc1f5b51fa4cecaa18f2bd7a003f3dd039dd615cd69a2afd6d3b19aed6775f2 \ - --hash=sha256:70f7172939fdf8790425ba31915bfbe8335030f05b9913d7ae00a87d4395620a \ - --hash=sha256:721c76e84fe669be19c5791da68232ca2e05ba5185575086e384352e2c309597 \ - --hash=sha256:7222ffd5e4de8e57e03ce2cef95a4c43c98fcb72ad86909abdfc2c17d227fc1b \ - --hash=sha256:75d10d37a47afee94919c4fab4c22b9bc2a8bf7d4f46f87363bcf0573f3ff4f5 \ - --hash=sha256:76af085e67e56c8816c3ccf256ebd136def2ed9654525348cfa744b6802b69eb \ - --hash=sha256:770cab594ecf99ae64c236bc9ee3439c3f46be49796e265ce0cc8bc17b10294f \ - --hash=sha256:7a6ab32f7210554a96cd9e33abe3ddd86732beeafc7a28e9955cdf22ffadbab0 \ - --hash=sha256:7c48ed483eb946e6c04ccbe02c6b4d1d48e51944b6db70f697e089c193404941 \ - --hash=sha256:7f56930ab0abd1c45cd15be65cc741c28b1c9a34876ce8c17a2fa107810c0af0 \ - --hash=sha256:8075c35cd58273fee266c58c0c9b670947c19df5fb98e7b66710e04ad4e9ff86 \ - --hash=sha256:8272b73e1c5603666618805fe821edba66892e2870058c94c53147602eab29c7 \ - --hash=sha256:82d8fd25b7f4675d0c47cf95b594d4e7b158aca33b76aa63d07186e13c0e0ab7 \ - --hash=sha256:844da2b5728b5ce0e32d863af26f32b5ce61bc4273a9c720a9f3aa9df73b1455 \ - --hash=sha256:8755483f3c00d6c9a77f490c17e6ab0c8729e39e6390328e42521ef175380ae6 \ - --hash=sha256:915f3849a011c1f593ab99092f3cecfcb4d65d8feb4a64cf1bf2d22074dc0ec4 \ - --hash=sha256:926ca93accd5d36ccdabd803392ddc3e03e6d4cd1cf17deff3b989ab8e9dbcf0 \ - --hash=sha256:982bb1e8b4ffda883b3d0a521e23abcd6fd17418f6d2c4118d257a10199c0ce3 \ - --hash=sha256:98f862da73774290f251b9df8d11161b6cf25b599a66baf087c1ffe340e9bfd1 \ - --hash=sha256:9cbfacf36cb0ec2897ce0ebc5d08ca44213af24265bd56eca54bee7923c48fd6 \ - --hash=sha256:a370b3e078e418187da8c3674eddb9d983ec09445c99a3a263c2011993522981 \ - --hash=sha256:a955b438e62efdf7e0b7b52a64dc5c3396e2634baa62471768a64bc2adb73d5c \ - --hash=sha256:aa6af9e7d59f9c12b33ae4e9450619cf2488e2bbe9b44030905877f0b2324980 \ - --hash=sha256:aa88ca0b1932e93f2d961bf3addbb2db902198dca337d88c89e1559e066e7645 \ - --hash=sha256:aaeeb6a479c7667fbe1099af9617c83aaca22182d6cf8c53966491a0f1b7ffb7 \ - --hash=sha256:aaf27faa992bfee0264dc1f03f4c75e9fcdda66a519db6b957a3f826e285cf12 \ - --hash=sha256:b2680962a4848b3c4f155dc2ee64505a9c57186d0d56b43123b17ca3de18f0fa \ - --hash=sha256:b2d318c11350e10662026ad0eb71bb51c7812fc8590825304ae0bdd4ac283acd \ - --hash=sha256:b33de11b92e9f75a2b545d6e9b6f37e398d86c3e9e9653c4864eb7e89c5773ef \ - --hash=sha256:b3daeac64d5b371dea99714f08ffc2c208522ec6b06fbc7866a450dd446f5c0f \ - --hash=sha256:be1e352acbe3c78727a16a455126d9ff83ea2dfdcbc83148d2982305a04714c2 \ - --hash=sha256:bee093bf902e1d8fc0ac143c88902c3dfc8941f7ea1d6a8dd2bcb786d33db03d \ - --hash=sha256:c72fbbe68c6f32f251bdc08b8611c7b3060612236e960ef848e0a517ddbe76c5 \ - --hash=sha256:c9e36a97bee9b86ef9a1cf7bb96747eb7a15c2f22bdb5b516434b00f2a599f02 \ - --hash=sha256:cddf7bd982eaa998934a91f69d182aec997c6c468898efe6679af88283b498d3 \ - --hash=sha256:cf713fe9a71ef6fd5adf7a79670135081cd4431c2943864757f0fa3a65b1fafd \ - --hash=sha256:d11b54acf878eef558599658b0ffca78138c8c3655cf4f3a4a673c437e67732e \ - --hash=sha256:d41c4d287cfc69060fa91cae9683eacffad989f1a10811995fa309df656ec214 \ - --hash=sha256:d524ba3f1581b35c03cb42beebab4a13e6cdad7b36246bd22541fa585a56cccd \ - --hash=sha256:daac4765328a919a805fa5e2720f3e94767abd632ae410a9062dff5412bae65a \ - --hash=sha256:db4c7bf0e07fc3b7d89ac2a5880a6a8062056801b83ff56d8464b70f65482b6c \ - --hash=sha256:dc7039885fa1baf9be153a0626e337aa7ec8bf96b0128605fb0d77788ddc1681 \ - --hash=sha256:dccab8d5fa1ef9bfba0590ecf4d46df048d18ffe3eec01eeb73a42e0d9e7a8ba \ - --hash=sha256:dedb8adb91d11846ee08bec4c8236c8549ac721c245678282dcb06b221aab59f \ - --hash=sha256:e45ba65510e2647721e35323d6ef54c7974959f6081b58d4ef5d87c60c84919a \ - --hash=sha256:e53efc7c7cee4c1e70661e2e112ca46a575f90ed9ae3fef200f2a25e954f4b28 \ - --hash=sha256:e635b87f01ebc977342e2697d05b56632f5f879a4f15955dfe8cef2448b51691 \ - --hash=sha256:e70e990b2137b29dc5564715de1e12701815dacc1d056308e2b17e9095372a82 \ - --hash=sha256:e8082b26888e2f8b36a042a58307d5b917ef2b1cacab921ad3323ef91901c71a \ - --hash=sha256:e8323a9b031aa0393768b87f04b4164a40037fb2a3c11ac06a03ffecd3618027 \ - --hash=sha256:e92fca20c46e9f5e1bb485887d074918b13543b1c2a1185e69bb8d17ab6236a7 \ - --hash=sha256:eb30abc20df9ab0814b5a2524f23d75dcf83cde762c161917a2b4b7b55b1e518 \ - --hash=sha256:eba9904b0f38a143592d9fc0e19e2df0fa2e41c3c3745554761c5f6447eedabf \ - --hash=sha256:ef8de666d6179b009dce7bcb2ad4c4a779f113f12caf8dc77f0162c29d20490b \ - --hash=sha256:efd387a49825780ff861998cd959767800d54f8308936b21025326de4b5a42b9 \ - --hash=sha256:f0aa37f3c979cf2546b73e8222bbfa3dc07a641585340179d768068e3455e544 \ - --hash=sha256:f4074c5a429281bf056ddd4c5d3b740ebca4d43ffffe2ef4bf4d2d05114299da \ - --hash=sha256:f69a27e45c43520f5487f27627059b64aaf160415589230992cec34c5e18a509 \ - --hash=sha256:fb707f3e15060adf5b7ada797624a6c6e0138e2a26baa089df64c68ee98e040f \ - --hash=sha256:fcbe676a55d7445b22c10967bceaaf0ee69407fbe0ece4d032b6eb8d4565982a \ - --hash=sha256:fdb20a30fe1175ecabed17cbf7812f7b804b8a315a25f24678bcdf120a90077f +charset-normalizer==3.4.3 \ + --hash=sha256:00237675befef519d9af72169d8604a067d92755e84fe76492fef5441db05b91 \ + --hash=sha256:02425242e96bcf29a49711b0ca9f37e451da7c70562bc10e8ed992a5a7a25cc0 \ + --hash=sha256:027b776c26d38b7f15b26a5da1044f376455fb3766df8fc38563b4efbc515154 \ + --hash=sha256:07a0eae9e2787b586e129fdcbe1af6997f8d0e5abaa0bc98c0e20e124d67e601 \ + --hash=sha256:0cacf8f7297b0c4fcb74227692ca46b4a5852f8f4f24b3c766dd94a1075c4884 \ + --hash=sha256:0e78314bdc32fa80696f72fa16dc61168fda4d6a0c014e0380f9d02f0e5d8a07 \ + --hash=sha256:0f2be7e0cf7754b9a30eb01f4295cc3d4358a479843b31f328afd210e2c7598c \ + --hash=sha256:13faeacfe61784e2559e690fc53fa4c5ae97c6fcedb8eb6fb8d0a15b475d2c64 \ + --hash=sha256:14c2a87c65b351109f6abfc424cab3927b3bdece6f706e4d12faaf3d52ee5efe \ + --hash=sha256:1606f4a55c0fd363d754049cdf400175ee96c992b1f8018b993941f221221c5f \ + --hash=sha256:16a8770207946ac75703458e2c743631c79c59c5890c80011d536248f8eaa432 \ + --hash=sha256:18343b2d246dc6761a249ba1fb13f9ee9a2bcd95decc767319506056ea4ad4dc \ + --hash=sha256:18b97b8404387b96cdbd30ad660f6407799126d26a39ca65729162fd810a99aa \ + --hash=sha256:1bb60174149316da1c35fa5233681f7c0f9f514509b8e399ab70fea5f17e45c9 \ + --hash=sha256:1e8ac75d72fa3775e0b7cb7e4629cec13b7514d928d15ef8ea06bca03ef01cae \ + --hash=sha256:1ef99f0456d3d46a50945c98de1774da86f8e992ab5c77865ea8b8195341fc19 \ + --hash=sha256:2001a39612b241dae17b4687898843f254f8748b796a2e16f1051a17078d991d \ + --hash=sha256:23b6b24d74478dc833444cbd927c338349d6ae852ba53a0d02a2de1fce45b96e \ + --hash=sha256:252098c8c7a873e17dd696ed98bbe91dbacd571da4b87df3736768efa7a792e4 \ + --hash=sha256:257f26fed7d7ff59921b78244f3cd93ed2af1800ff048c33f624c87475819dd7 \ + --hash=sha256:2c322db9c8c89009a990ef07c3bcc9f011a3269bc06782f916cd3d9eed7c9312 \ + --hash=sha256:30a96e1e1f865f78b030d65241c1ee850cdf422d869e9028e2fc1d5e4db73b92 \ + --hash=sha256:30d006f98569de3459c2fc1f2acde170b7b2bd265dc1943e87e1a4efe1b67c31 \ + --hash=sha256:31a9a6f775f9bcd865d88ee350f0ffb0e25936a7f930ca98995c05abf1faf21c \ + --hash=sha256:320e8e66157cc4e247d9ddca8e21f427efc7a04bbd0ac8a9faf56583fa543f9f \ + --hash=sha256:34a7f768e3f985abdb42841e20e17b330ad3aaf4bb7e7aeeb73db2e70f077b99 \ + --hash=sha256:3653fad4fe3ed447a596ae8638b437f827234f01a8cd801842e43f3d0a6b281b \ + --hash=sha256:3cd35b7e8aedeb9e34c41385fda4f73ba609e561faedfae0a9e75e44ac558a15 \ + --hash=sha256:3cfb2aad70f2c6debfbcb717f23b7eb55febc0bb23dcffc0f076009da10c6392 \ + --hash=sha256:416175faf02e4b0810f1f38bcb54682878a4af94059a1cd63b8747244420801f \ + --hash=sha256:41d1fc408ff5fdfb910200ec0e74abc40387bccb3252f3f27c0676731df2b2c8 \ + --hash=sha256:42e5088973e56e31e4fa58eb6bd709e42fc03799c11c42929592889a2e54c491 \ + --hash=sha256:4ca4c094de7771a98d7fbd67d9e5dbf1eb73efa4f744a730437d8a3a5cf994f0 \ + --hash=sha256:511729f456829ef86ac41ca78c63a5cb55240ed23b4b737faca0eb1abb1c41bc \ + --hash=sha256:53cd68b185d98dde4ad8990e56a58dea83a4162161b1ea9272e5c9182ce415e0 \ + --hash=sha256:585f3b2a80fbd26b048a0be90c5aae8f06605d3c92615911c3a2b03a8a3b796f \ + --hash=sha256:5b413b0b1bfd94dbf4023ad6945889f374cd24e3f62de58d6bb102c4d9ae534a \ + --hash=sha256:5d8d01eac18c423815ed4f4a2ec3b439d654e55ee4ad610e153cf02faf67ea40 \ + --hash=sha256:6aab0f181c486f973bc7262a97f5aca3ee7e1437011ef0c2ec04b5a11d16c927 \ + --hash=sha256:6cf8fd4c04756b6b60146d98cd8a77d0cdae0e1ca20329da2ac85eed779b6849 \ + --hash=sha256:6fb70de56f1859a3f71261cbe41005f56a7842cc348d3aeb26237560bfa5e0ce \ + --hash=sha256:6fce4b8500244f6fcb71465d4a4930d132ba9ab8e71a7859e6a5d59851068d14 \ + --hash=sha256:70bfc5f2c318afece2f5838ea5e4c3febada0be750fcf4775641052bbba14d05 \ + --hash=sha256:73dc19b562516fc9bcf6e5d6e596df0b4eb98d87e4f79f3ae71840e6ed21361c \ + --hash=sha256:74d77e25adda8581ffc1c720f1c81ca082921329452eba58b16233ab1842141c \ + --hash=sha256:78deba4d8f9590fe4dae384aeff04082510a709957e968753ff3c48399f6f92a \ + --hash=sha256:86df271bf921c2ee3818f0522e9a5b8092ca2ad8b065ece5d7d9d0e9f4849bcc \ + --hash=sha256:88ab34806dea0671532d3f82d82b85e8fc23d7b2dd12fa837978dad9bb392a34 \ + --hash=sha256:8999f965f922ae054125286faf9f11bc6932184b93011d138925a1773830bbe9 \ + --hash=sha256:8dcfc373f888e4fb39a7bc57e93e3b845e7f462dacc008d9749568b1c4ece096 \ + --hash=sha256:939578d9d8fd4299220161fdd76e86c6a251987476f5243e8864a7844476ba14 \ + --hash=sha256:96b2b3d1a83ad55310de8c7b4a2d04d9277d5591f40761274856635acc5fcb30 \ + --hash=sha256:a2d08ac246bb48479170408d6c19f6385fa743e7157d716e144cad849b2dd94b \ + --hash=sha256:b256ee2e749283ef3ddcff51a675ff43798d92d746d1a6e4631bf8c707d22d0b \ + --hash=sha256:b5e3b2d152e74e100a9e9573837aba24aab611d39428ded46f4e4022ea7d1942 \ + --hash=sha256:b89bc04de1d83006373429975f8ef9e7932534b8cc9ca582e4db7d20d91816db \ + --hash=sha256:bd28b817ea8c70215401f657edef3a8aa83c29d447fb0b622c35403780ba11d5 \ + --hash=sha256:c60e092517a73c632ec38e290eba714e9627abe9d301c8c8a12ec32c314a2a4b \ + --hash=sha256:c6dbd0ccdda3a2ba7c2ecd9d77b37f3b5831687d8dc1b6ca5f56a4880cc7b7ce \ + --hash=sha256:c6e490913a46fa054e03699c70019ab869e990270597018cef1d8562132c2669 \ + --hash=sha256:c6f162aabe9a91a309510d74eeb6507fab5fff92337a15acbe77753d88d9dcf0 \ + --hash=sha256:c6fd51128a41297f5409deab284fecbe5305ebd7e5a1f959bee1c054622b7018 \ + --hash=sha256:cc34f233c9e71701040d772aa7490318673aa7164a0efe3172b2981218c26d93 \ + --hash=sha256:cc9370a2da1ac13f0153780040f465839e6cccb4a1e44810124b4e22483c93fe \ + --hash=sha256:ccf600859c183d70eb47e05a44cd80a4ce77394d1ac0f79dbd2dd90a69a3a049 \ + --hash=sha256:ce571ab16d890d23b5c278547ba694193a45011ff86a9162a71307ed9f86759a \ + --hash=sha256:cf1ebb7d78e1ad8ec2a8c4732c7be2e736f6e5123a4146c5b89c9d1f585f8cef \ + --hash=sha256:d0e909868420b7049dafd3a31d45125b31143eec59235311fc4c57ea26a4acd2 \ + --hash=sha256:d22dbedd33326a4a5190dd4fe9e9e693ef12160c77382d9e87919bce54f3d4ca \ + --hash=sha256:d716a916938e03231e86e43782ca7878fb602a125a91e7acb8b5112e2e96ac16 \ + --hash=sha256:d79c198e27580c8e958906f803e63cddb77653731be08851c7df0b1a14a8fc0f \ + --hash=sha256:d95bfb53c211b57198bb91c46dd5a2d8018b3af446583aab40074bf7988401cb \ + --hash=sha256:e28e334d3ff134e88989d90ba04b47d84382a828c061d0d1027b1b12a62b39b1 \ + --hash=sha256:ec557499516fc90fd374bf2e32349a2887a876fbf162c160e3c01b6849eaf557 \ + --hash=sha256:fb6fecfd65564f208cbf0fba07f107fb661bcd1a7c389edbced3f7a493f70e37 \ + --hash=sha256:fb731e5deb0c7ef82d698b0f4c5bb724633ee2a489401594c5c88b02e6cb15f7 \ + --hash=sha256:fb7f67a1bfa6e40b438170ebdc8158b78dc465a5a67b6dde178a46987b244a72 \ + --hash=sha256:fd10de089bcdcd1be95a2f73dbe6254798ec1bda9f450d5828c96f93e2536b9c \ + --hash=sha256:fdabf8315679312cfa71302f9bd509ded4f2f263fb5b765cf1433b39106c3cc9 # via requests colorama==0.4.6 ; sys_platform == 'win32' \ --hash=sha256:08695f5cb7ed6e0531a20572697297273c47b8cae5a63ffc6d6ed5c201be6e44 \ @@ -134,6 +126,10 @@ imagesize==1.4.1 \ --hash=sha256:0d8d18d08f840c19d0ee7ca1fd82490fdc3729b7ac93f49870406ddde8ef8d8b \ --hash=sha256:69150444affb9cb0d5cc5a92b3676f0b2fb7cd9ae39e947a5e11a36b4497cd4a # via sphinx +importlib-metadata==8.7.0 ; python_full_version < '3.10' \ + --hash=sha256:d13b81ad223b890aa16c5471f2ac3056cf76c5f10f82d6f9292f0b415f389000 \ + --hash=sha256:e5dd1551894c77868a30651cef00984d50e1002d06942a7101d34870c5f02afd + # via sphinx jinja2==3.1.6 \ --hash=sha256:0137fb05990d35f1275a587e9aee6d56da821fc83491a0fb838183be43f66d6d \ --hash=sha256:85ece4451f492d0c13c5dd7c13a64681a86afae63a5f347908daf103ce6d2f67 @@ -210,17 +206,25 @@ markupsafe==3.0.2 \ --hash=sha256:f8b3d067f2e40fe93e1ccdd6b2e1d16c43140e76f02fb1319a05cf2b79d99430 \ --hash=sha256:fcabf5ff6eea076f859677f5f0b6b5c1a51e70a376b0579e0eadef8db48c6b50 # via jinja2 -mdit-py-plugins==0.4.2 \ +mdit-py-plugins==0.4.2 ; python_full_version < '3.10' \ --hash=sha256:0c673c3f889399a33b95e88d2f0d111b4447bdfea7f237dab2d488f459835636 \ --hash=sha256:5f2cd1fdb606ddf152d37ec30e46101a60512bc0e5fa1a7002c36647b09e26b5 # via myst-parser +mdit-py-plugins==0.5.0 ; python_full_version >= '3.10' \ + --hash=sha256:07a08422fc1936a5d26d146759e9155ea466e842f5ab2f7d2266dd084c8dab1f \ + --hash=sha256:f4918cb50119f50446560513a8e311d574ff6aaed72606ddae6d35716fe809c6 + # via myst-parser mdurl==0.1.2 \ --hash=sha256:84008a41e51615a49fc9966191ff91509e3c40b939176e643fd50a5c2196b8f8 \ --hash=sha256:bb413d29f5eea38f31dd4754dd7377d4465116fb207585f97bf925588687c1ba # via markdown-it-py -myst-parser==4.0.0 \ - --hash=sha256:851c9dfb44e36e56d15d05e72f02b80da21a9e0d07cba96baf5e2d476bb91531 \ - --hash=sha256:b9317997552424448c6096c2558872fdb6f81d3ecb3a40ce84a7518798f3f28d +myst-parser==3.0.1 ; python_full_version < '3.10' \ + --hash=sha256:6457aaa33a5d474aca678b8ead9b3dc298e89c68e67012e73146ea6fd54babf1 \ + --hash=sha256:88f0cb406cb363b077d176b51c476f62d60604d68a8dcdf4832e080441301a87 + # via rules-python-docs (docs/pyproject.toml) +myst-parser==4.0.1 ; python_full_version >= '3.10' \ + --hash=sha256:5cfea715e4f3574138aecbf7d54132296bfd72bb614d31168f48c477a830a7c4 \ + --hash=sha256:9134e88959ec3b5780aedf8a99680ea242869d012e8821db3126d427edc9c95d # via rules-python-docs (docs/pyproject.toml) packaging==25.0 \ --hash=sha256:29572ef2b1f17581046b3a2227d5c611fb25ec70ca1ba8554b24b0e69331a484 \ @@ -297,11 +301,24 @@ requests==2.32.4 \ # via # readthedocs-sphinx-ext # sphinx +roman-numerals-py==3.1.0 ; python_full_version >= '3.11' \ + --hash=sha256:9da2ad2fb670bcf24e81070ceb3be72f6c11c440d73bd579fbeca1e9f330954c \ + --hash=sha256:be4bf804f083a4ce001b5eb7e3c0862479d10f94c936f6c4e5f250aa5ff5bd2d + # via sphinx snowballstemmer==3.0.1 \ --hash=sha256:6cd7b3897da8d6c9ffb968a6781fa6532dce9c3618a4b127d920dab764a19064 \ --hash=sha256:6d5eeeec8e9f84d4d56b847692bacf79bc2c8e90c7f80ca4444ff8b6f2e52895 # via sphinx -sphinx==8.1.3 \ +sphinx==7.4.7 ; python_full_version < '3.10' \ + --hash=sha256:242f92a7ea7e6c5b406fdc2615413890ba9f699114a9c09192d7dfead2ee9cfe \ + --hash=sha256:c2419e2135d11f1951cd994d6eb18a1835bd8fdd8429f9ca375dc1f3281bd239 + # via + # rules-python-docs (docs/pyproject.toml) + # myst-parser + # sphinx-reredirects + # sphinx-rtd-theme + # sphinxcontrib-jquery +sphinx==8.1.3 ; python_full_version == '3.10.*' \ --hash=sha256:09719015511837b76bf6e03e42eb7595ac8c2e41eeb9c29c5b755c6b677992a2 \ --hash=sha256:43c1911eecb0d3e161ad78611bc905d1ad0e523e4ddc202a58a821773dc4c927 # via @@ -310,14 +327,27 @@ sphinx==8.1.3 \ # sphinx-reredirects # sphinx-rtd-theme # sphinxcontrib-jquery +sphinx==8.2.3 ; python_full_version >= '3.11' \ + --hash=sha256:398ad29dee7f63a75888314e9424d40f52ce5a6a87ae88e7071e80af296ec348 \ + --hash=sha256:4405915165f13521d875a8c29c8970800a0141c14cc5416a38feca4ea5d9b9c3 + # via + # rules-python-docs (docs/pyproject.toml) + # myst-parser + # sphinx-reredirects + # sphinx-rtd-theme + # sphinxcontrib-jquery sphinx-autodoc2==0.5.0 \ --hash=sha256:7d76044aa81d6af74447080182b6868c7eb066874edc835e8ddf810735b6565a \ --hash=sha256:e867013b1512f9d6d7e6f6799f8b537d6884462acd118ef361f3f619a60b5c9e # via rules-python-docs (docs/pyproject.toml) -sphinx-reredirects==0.1.6 \ +sphinx-reredirects==0.1.6 ; python_full_version < '3.11' \ --hash=sha256:c491cba545f67be9697508727818d8626626366245ae64456fe29f37e9bbea64 \ --hash=sha256:efd50c766fbc5bf40cd5148e10c00f2c00d143027de5c5e48beece93cc40eeea # via rules-python-docs (docs/pyproject.toml) +sphinx-reredirects==1.0.0 ; python_full_version >= '3.11' \ + --hash=sha256:1d0102710a8f633c6c885f940f440f7195ada675c1739976f0135790747dea06 \ + --hash=sha256:7c9bada9f1330489fcf4c7297a2d6da2a49ca4877d3f42d1388ae1de1019bf5c + # via rules-python-docs (docs/pyproject.toml) sphinx-rtd-theme==3.0.2 \ --hash=sha256:422ccc750c3a3a311de4ae327e82affdaf59eb695ba4936538552f3b00f4ee13 \ --hash=sha256:b7457bc25dda723b20b086a670b9953c859eab60a2a03ee8eb2bb23e176e5f85 @@ -350,13 +380,54 @@ sphinxcontrib-serializinghtml==2.0.0 \ --hash=sha256:6e2cb0eef194e10c27ec0023bfeb25badbbb5868244cf5bc5bdc04e4464bf331 \ --hash=sha256:e9d912827f872c029017a53f0ef2180b327c3f7fd23c87229f7a8e8b70031d4d # via sphinx +tomli==2.2.1 ; python_full_version < '3.11' \ + --hash=sha256:023aa114dd824ade0100497eb2318602af309e5a55595f76b626d6d9f3b7b0a6 \ + --hash=sha256:02abe224de6ae62c19f090f68da4e27b10af2b93213d36cf44e6e1c5abd19fdd \ + --hash=sha256:286f0ca2ffeeb5b9bd4fcc8d6c330534323ec51b2f52da063b11c502da16f30c \ + --hash=sha256:2d0f2fdd22b02c6d81637a3c95f8cd77f995846af7414c5c4b8d0545afa1bc4b \ + --hash=sha256:33580bccab0338d00994d7f16f4c4ec25b776af3ffaac1ed74e0b3fc95e885a8 \ + --hash=sha256:400e720fe168c0f8521520190686ef8ef033fb19fc493da09779e592861b78c6 \ + --hash=sha256:40741994320b232529c802f8bc86da4e1aa9f413db394617b9a256ae0f9a7f77 \ + --hash=sha256:465af0e0875402f1d226519c9904f37254b3045fc5084697cefb9bdde1ff99ff \ + --hash=sha256:4a8f6e44de52d5e6c657c9fe83b562f5f4256d8ebbfe4ff922c495620a7f6cea \ + --hash=sha256:4e340144ad7ae1533cb897d406382b4b6fede8890a03738ff1683af800d54192 \ + --hash=sha256:678e4fa69e4575eb77d103de3df8a895e1591b48e740211bd1067378c69e8249 \ + --hash=sha256:6972ca9c9cc9f0acaa56a8ca1ff51e7af152a9f87fb64623e31d5c83700080ee \ + --hash=sha256:7fc04e92e1d624a4a63c76474610238576942d6b8950a2d7f908a340494e67e4 \ + --hash=sha256:889f80ef92701b9dbb224e49ec87c645ce5df3fa2cc548664eb8a25e03127a98 \ + --hash=sha256:8d57ca8095a641b8237d5b079147646153d22552f1c637fd3ba7f4b0b29167a8 \ + --hash=sha256:8dd28b3e155b80f4d54beb40a441d366adcfe740969820caf156c019fb5c7ec4 \ + --hash=sha256:9316dc65bed1684c9a98ee68759ceaed29d229e985297003e494aa825ebb0281 \ + --hash=sha256:a198f10c4d1b1375d7687bc25294306e551bf1abfa4eace6650070a5c1ae2744 \ + --hash=sha256:a38aa0308e754b0e3c67e344754dff64999ff9b513e691d0e786265c93583c69 \ + --hash=sha256:a92ef1a44547e894e2a17d24e7557a5e85a9e1d0048b0b5e7541f76c5032cb13 \ + --hash=sha256:ac065718db92ca818f8d6141b5f66369833d4a80a9d74435a268c52bdfa73140 \ + --hash=sha256:b82ebccc8c8a36f2094e969560a1b836758481f3dc360ce9a3277c65f374285e \ + --hash=sha256:c954d2250168d28797dd4e3ac5cf812a406cd5a92674ee4c8f123c889786aa8e \ + --hash=sha256:cb55c73c5f4408779d0cf3eef9f762b9c9f147a77de7b258bef0a5628adc85cc \ + --hash=sha256:cd45e1dc79c835ce60f7404ec8119f2eb06d38b1deba146f07ced3bbc44505ff \ + --hash=sha256:d3f5614314d758649ab2ab3a62d4f2004c825922f9e370b29416484086b264ec \ + --hash=sha256:d920f33822747519673ee656a4b6ac33e382eca9d331c87770faa3eef562aeb2 \ + --hash=sha256:db2b95f9de79181805df90bedc5a5ab4c165e6ec3fe99f970d0e302f384ad222 \ + --hash=sha256:e59e304978767a54663af13c07b3d1af22ddee3bb2fb0618ca1593e4f593a106 \ + --hash=sha256:e85e99945e688e32d5a35c1ff38ed0b3f41f43fad8df0bdf79f72b2ba7bc5272 \ + --hash=sha256:ece47d672db52ac607a3d9599a9d48dcb2f2f735c6c2d1f34130085bb12b112a \ + --hash=sha256:f4039b9cbc3048b2416cc57ab3bda989a6fcf9b36cf8937f01a6e731b64f80d7 + # via + # sphinx + # sphinx-autodoc2 typing-extensions==4.14.1 \ --hash=sha256:38b39f4aeeab64884ce9f74c94263ef78f3c22467c8724005483154c26648d36 \ --hash=sha256:d1e1e3b58374dc93031d6eda2420a48ea44a36c2b4766a4fdeb3710755731d76 # via # rules-python-docs (docs/pyproject.toml) + # astroid # sphinx-autodoc2 urllib3==2.5.0 \ --hash=sha256:3fc47733c7e419d4bc3f6b3dc2b4f890bb743906a30d56ba4a5bfa4bbff92760 \ --hash=sha256:e6b01673c0fa6a13e374b50871808eb3bf7046c4b125b216f6bf1cc604cff0dc # via requests +zipp==3.23.0 ; python_full_version < '3.10' \ + --hash=sha256:071652d6115ed432f5ce1d34c336c0adfd6a884660d1e9712a256d3d3bd4b14e \ + --hash=sha256:a07157588a12518c9d4034df3fbbee09c814741a33ff63c05fa29d26a2404166 + # via importlib-metadata diff --git a/python/private/pypi/extension.bzl b/python/private/pypi/extension.bzl index 0c06dea2ff..618682603c 100644 --- a/python/private/pypi/extension.bzl +++ b/python/private/pypi/extension.bzl @@ -346,7 +346,17 @@ def _create_whl_repos( )) whl_libraries[repo_name] = repo.args - whl_map.setdefault(whl.name, {})[repo.config_setting] = repo_name + mapping = whl_map.setdefault(whl.name, {}) + if repo.config_setting in mapping and mapping[repo.config_setting] != repo_name: + fail( + "attempting to override an existing repo '{}' for config setting '{}' with a new repo '{}'".format( + mapping[repo.config_setting], + repo.config_setting, + repo_name, + ), + ) + else: + mapping[repo.config_setting] = repo_name return struct( whl_map = whl_map, diff --git a/python/private/pypi/parse_requirements.bzl b/python/private/pypi/parse_requirements.bzl index ebd447d95d..acf3b0c6ae 100644 --- a/python/private/pypi/parse_requirements.bzl +++ b/python/private/pypi/parse_requirements.bzl @@ -42,7 +42,7 @@ def parse_requirements( get_index_urls = None, evaluate_markers = None, extract_url_srcs = True, - logger = None): + logger): """Get the requirements with platforms that the requirements apply to. Args: @@ -63,7 +63,7 @@ def parse_requirements( requirements line. extract_url_srcs: A boolean to enable extracting URLs from requirement lines to enable using bazel downloader. - logger: repo_utils.logger or None, a simple struct to log diagnostic messages. + logger: repo_utils.logger, a simple struct to log diagnostic messages. Returns: {type}`dict[str, list[struct]]` where the key is the distribution name and the struct @@ -89,8 +89,7 @@ def parse_requirements( options = {} requirements = {} for file, plats in requirements_by_platform.items(): - if logger: - logger.trace(lambda: "Using {} for {}".format(file, plats)) + logger.trace(lambda: "Using {} for {}".format(file, plats)) contents = ctx.read(file) # Parse the requirements file directly in starlark to get the information @@ -162,11 +161,10 @@ def parse_requirements( # URL of the files to download things from. This should be important for # VCS package references. env_marker_target_platforms = evaluate_markers(ctx, reqs_with_env_markers) - if logger: - logger.trace(lambda: "Evaluated env markers from:\n{}\n\nTo:\n{}".format( - reqs_with_env_markers, - env_marker_target_platforms, - )) + logger.trace(lambda: "Evaluated env markers from:\n{}\n\nTo:\n{}".format( + reqs_with_env_markers, + env_marker_target_platforms, + )) index_urls = {} if get_index_urls: @@ -212,8 +210,7 @@ def parse_requirements( sorted(requirements), )) - if logger: - logger.debug(lambda: "Will configure whl repos: {}".format([w.name for w in ret])) + logger.debug(lambda: "Will configure whl repos: {}".format([w.name for w in ret])) return ret @@ -229,7 +226,10 @@ def _package_srcs( """A function to return sources for a particular package.""" srcs = {} for r in sorted(reqs.values(), key = lambda r: r.requirement_line): - target_platforms = env_marker_target_platforms.get(r.requirement_line, r.target_platforms) + if ";" in r.requirement_line: + target_platforms = env_marker_target_platforms.get(r.requirement_line, []) + else: + target_platforms = r.target_platforms extra_pip_args = tuple(r.extra_pip_args) for target_platform in target_platforms: @@ -245,8 +245,7 @@ def _package_srcs( index_urls = index_urls.get(name), logger = logger, ) - if logger: - logger.debug(lambda: "The whl dist is: {}".format(dist.filename if dist else dist)) + logger.debug(lambda: "The whl dist is: {}".format(dist.filename if dist else dist)) if extract_url_srcs and dist: req_line = r.srcs.requirement @@ -347,10 +346,9 @@ def _add_dists(*, requirement, index_urls, target_platform, logger = None): if requirement.srcs.url: if not requirement.srcs.filename: - if logger: - logger.debug(lambda: "Could not detect the filename from the URL, falling back to pip: {}".format( - requirement.srcs.url, - )) + logger.debug(lambda: "Could not detect the filename from the URL, falling back to pip: {}".format( + requirement.srcs.url, + )) return None # Handle direct URLs in requirements @@ -377,8 +375,7 @@ def _add_dists(*, requirement, index_urls, target_platform, logger = None): if not shas_to_use: version = requirement.srcs.version shas_to_use = index_urls.sha256s_by_version.get(version, []) - if logger: - logger.warn(lambda: "requirement file has been generated without hashes, will use all hashes for the given version {} that could find on the index:\n {}".format(version, shas_to_use)) + logger.warn(lambda: "requirement file has been generated without hashes, will use all hashes for the given version {} that could find on the index:\n {}".format(version, shas_to_use)) for sha256 in shas_to_use: # For now if the artifact is marked as yanked we just ignore it. @@ -395,8 +392,7 @@ def _add_dists(*, requirement, index_urls, target_platform, logger = None): sdist = maybe_sdist continue - if logger: - logger.warn(lambda: "Could not find a whl or an sdist with sha256={}".format(sha256)) + logger.warn(lambda: "Could not find a whl or an sdist with sha256={}".format(sha256)) yanked = {} for dist in whls + [sdist]: diff --git a/python/private/pypi/pip_repository.bzl b/python/private/pypi/pip_repository.bzl index e63bd6c3d1..5ad388d8ea 100644 --- a/python/private/pypi/pip_repository.bzl +++ b/python/private/pypi/pip_repository.bzl @@ -16,7 +16,7 @@ load("@bazel_skylib//lib:sets.bzl", "sets") load("//python/private:normalize_name.bzl", "normalize_name") -load("//python/private:repo_utils.bzl", "REPO_DEBUG_ENV_VAR") +load("//python/private:repo_utils.bzl", "REPO_DEBUG_ENV_VAR", "repo_utils") load("//python/private:text_util.bzl", "render") load(":evaluate_markers.bzl", "evaluate_markers_py", EVALUATE_MARKERS_SRCS = "SRCS") load(":parse_requirements.bzl", "host_platform", "parse_requirements", "select_requirement") @@ -71,6 +71,7 @@ exports_files(["requirements.bzl"]) """ def _pip_repository_impl(rctx): + logger = repo_utils.logger(rctx) requirements_by_platform = parse_requirements( rctx, requirements_by_platform = requirements_files_by_platform( @@ -100,6 +101,7 @@ def _pip_repository_impl(rctx): srcs = rctx.attr._evaluate_markers_srcs, ), extract_url_srcs = False, + logger = logger, ) selected_requirements = {} options = None diff --git a/tests/pypi/parse_requirements/parse_requirements_tests.bzl b/tests/pypi/parse_requirements/parse_requirements_tests.bzl index 249af90114..bd0078bfa4 100644 --- a/tests/pypi/parse_requirements/parse_requirements_tests.bzl +++ b/tests/pypi/parse_requirements/parse_requirements_tests.bzl @@ -639,7 +639,7 @@ def _test_get_index_urls_different_versions(env): platforms = { "cp310_linux_x86_64": struct( env = pep508_env( - python_version = "3.9.0", + python_version = "3.10.0", os = "linux", arch = "x86_64", ), @@ -686,6 +686,7 @@ def _test_get_index_urls_different_versions(env): ), }, ), + debug = True, ) env.expect.that_collection(got).contains_exactly([ @@ -720,6 +721,70 @@ def _test_get_index_urls_different_versions(env): _tests.append(_test_get_index_urls_different_versions) +def _test_get_index_urls_single_py_version(env): + got = parse_requirements( + requirements_by_platform = { + "requirements_multi_version": [ + "cp310_linux_x86_64", + ], + }, + platforms = { + "cp310_linux_x86_64": struct( + env = pep508_env( + python_version = "3.10.0", + os = "linux", + arch = "x86_64", + ), + whl_abi_tags = ["none"], + whl_platform_tags = ["any"], + ), + }, + get_index_urls = lambda _, __: { + "foo": struct( + sdists = {}, + whls = { + "deadb11f": struct( + url = "super2", + sha256 = "deadb11f", + filename = "foo-0.0.2-py3-none-any.whl", + yanked = False, + ), + }, + ), + }, + evaluate_markers = lambda _, requirements: evaluate_markers( + requirements = requirements, + platforms = { + "cp310_linux_x86_64": struct( + env = {"python_full_version": "3.10.0"}, + ), + }, + ), + debug = True, + ) + + env.expect.that_collection(got).contains_exactly([ + struct( + is_exposed = True, + is_multiple_versions = True, + name = "foo", + srcs = [ + struct( + distribution = "foo", + extra_pip_args = [], + filename = "foo-0.0.2-py3-none-any.whl", + requirement_line = "foo==0.0.2", + sha256 = "deadb11f", + target_platforms = ["cp310_linux_x86_64"], + url = "super2", + yanked = False, + ), + ], + ), + ]) + +_tests.append(_test_get_index_urls_single_py_version) + def parse_requirements_test_suite(name): """Create the test suite. From 10d9ab91377483e24c991269e3d7dd2a0c56e727 Mon Sep 17 00:00:00 2001 From: Philipp Stephani Date: Mon, 18 Aug 2025 14:47:40 +0200 Subject: [PATCH 392/922] style: Print coverage return codes in verbose mode (#3190) --- python/private/python_bootstrap_template.txt | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/python/private/python_bootstrap_template.txt b/python/private/python_bootstrap_template.txt index 1eaa0483df..62ded87337 100644 --- a/python/private/python_bootstrap_template.txt +++ b/python/private/python_bootstrap_template.txt @@ -457,6 +457,7 @@ source = env=env, cwd=workspace ) + PrintVerboseCoverage('Return code of coverage run:', ret_code) output_filename = os.path.join(os.environ['COVERAGE_DIR'], 'pylcov.dat') PrintVerboseCoverage('Converting coveragepy database to lcov:', output_filename) @@ -470,10 +471,12 @@ source = kparams['stdout'] = sys.stderr kparams['stderr'] = sys.stderr - ret_code = subprocess.call( + lcov_ret_code = subprocess.call( params, **kparams - ) or ret_code + ) + PrintVerboseCoverage('Return code of coverage lcov:', lcov_ret_code) + ret_code = lcov_ret_code or ret_code try: os.unlink(rcfile_name) From bb2aad2d1e3f883c9cdc2264e0b4a2815233db57 Mon Sep 17 00:00:00 2001 From: Jeremy Nimmer Date: Mon, 18 Aug 2025 15:18:15 -0700 Subject: [PATCH 393/922] fix(py_wheel): add directories in deterministic order (#3194) A call to `os.listdir` does not return a deterministic result from one run to the next. So for example if the output of a skylib [copy_directory](https://github.com/bazelbuild/bazel-skylib/blob/main/docs/copy_directory_doc.md) is added to a wheel, the files will be in a random order. --- CHANGELOG.md | 1 + tools/wheelmaker.py | 2 +- 2 files changed, 2 insertions(+), 1 deletion(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 2a5380677b..37329e3fb8 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -126,6 +126,7 @@ END_UNRELEASED_TEMPLATE * (pypi) Support `requirements.txt` files that use different versions of the same package targeting different target platforms. ([#2797](https://github.com/bazel-contrib/rules_python/issues/2797)). +* (py_wheel) Add directories in deterministic order. {#v0-0-0-added} ### Added diff --git a/tools/wheelmaker.py b/tools/wheelmaker.py index 3401c749ed..de6b8f48af 100644 --- a/tools/wheelmaker.py +++ b/tools/wheelmaker.py @@ -152,7 +152,7 @@ def add_file(self, package_filename, real_filename): """Add given file to the distribution.""" if os.path.isdir(real_filename): - directory_contents = os.listdir(real_filename) + directory_contents = sorted(os.listdir(real_filename)) for file_ in directory_contents: self.add_file( "{}/{}".format(package_filename, file_), From 56c9a3499a312031a02d6fd65726098403fd87f5 Mon Sep 17 00:00:00 2001 From: Ignas Anikevicius <240938+aignas@users.noreply.github.com> Date: Fri, 22 Aug 2025 01:45:47 +0900 Subject: [PATCH 394/922] feat: freethreaded support for the builder API (#3063) This is a continuation of #3058 where we define freethreaded platforms. They need to be used only for particular python versions so I included an extra marker configuration attribute where we are using pipstar marker evaluation before using the platform. I think this in general will be a useful tool to configure only particular platforms for particular python versions Fixes #2548, since this shows how we can define custom platforms Work towards #2747 --- MODULE.bazel | 36 ++++++-- python/private/pypi/extension.bzl | 84 ++++++++++++++----- python/private/pypi/pip_repository.bzl | 7 +- .../pypi/requirements_files_by_platform.bzl | 8 +- .../resolve_target_platforms.py | 4 +- tests/pypi/extension/extension_tests.bzl | 33 +++++--- 6 files changed, 127 insertions(+), 45 deletions(-) diff --git a/MODULE.bazel b/MODULE.bazel index b0b31dd73d..4f442bacec 100644 --- a/MODULE.bazel +++ b/MODULE.bazel @@ -70,11 +70,15 @@ pip = use_extension("//python/extensions:pip.bzl", "pip") config_settings = [ "@platforms//cpu:{}".format(cpu), "@platforms//os:linux", + "//python/config_settings:_is_py_freethreaded_{}".format( + "yes" if freethreaded else "no", + ), ], env = {"platform_version": "0"}, + marker = "python_version >= '3.13'" if freethreaded else "", os_name = "linux", - platform = "linux_{}".format(cpu), - whl_abi_tags = [ + platform = "linux_{}{}".format(cpu, freethreaded), + whl_abi_tags = ["cp{major}{minor}t"] if freethreaded else [ "abi3", "cp{major}{minor}", ], @@ -87,6 +91,10 @@ pip = use_extension("//python/extensions:pip.bzl", "pip") "x86_64", "aarch64", ] + for freethreaded in [ + "", + "_freethreaded", + ] ] [ @@ -95,13 +103,17 @@ pip = use_extension("//python/extensions:pip.bzl", "pip") config_settings = [ "@platforms//cpu:{}".format(cpu), "@platforms//os:osx", + "//python/config_settings:_is_py_freethreaded_{}".format( + "yes" if freethreaded else "no", + ), ], # We choose the oldest non-EOL version at the time when we release `rules_python`. # See https://endoflife.date/macos env = {"platform_version": "14.0"}, + marker = "python_version >= '3.13'" if freethreaded else "", os_name = "osx", - platform = "osx_{}".format(cpu), - whl_abi_tags = [ + platform = "osx_{}{}".format(cpu, freethreaded), + whl_abi_tags = ["cp{major}{minor}t"] if freethreaded else [ "abi3", "cp{major}{minor}", ], @@ -120,6 +132,10 @@ pip = use_extension("//python/extensions:pip.bzl", "pip") "x86_64", ], }.items() + for freethreaded in [ + "", + "_freethreaded", + ] ] [ @@ -128,11 +144,15 @@ pip = use_extension("//python/extensions:pip.bzl", "pip") config_settings = [ "@platforms//cpu:{}".format(cpu), "@platforms//os:windows", + "//python/config_settings:_is_py_freethreaded_{}".format( + "yes" if freethreaded else "no", + ), ], env = {"platform_version": "0"}, + marker = "python_version >= '3.13'" if freethreaded else "", os_name = "windows", - platform = "windows_{}".format(cpu), - whl_abi_tags = [ + platform = "windows_{}{}".format(cpu, freethreaded), + whl_abi_tags = ["cp{major}{minor}t"] if freethreaded else [ "abi3", "cp{major}{minor}", ], @@ -141,6 +161,10 @@ pip = use_extension("//python/extensions:pip.bzl", "pip") for cpu, whl_platform_tags in { "x86_64": ["win_amd64"], }.items() + for freethreaded in [ + "", + "_freethreaded", + ] ] pip.parse( diff --git a/python/private/pypi/extension.bzl b/python/private/pypi/extension.bzl index 618682603c..331ecf2340 100644 --- a/python/private/pypi/extension.bzl +++ b/python/private/pypi/extension.bzl @@ -30,6 +30,7 @@ load(":hub_repository.bzl", "hub_repository", "whl_config_settings_to_json") load(":parse_requirements.bzl", "parse_requirements") load(":parse_whl_name.bzl", "parse_whl_name") load(":pep508_env.bzl", "env") +load(":pep508_evaluate.bzl", "evaluate") load(":pip_repository_attrs.bzl", "ATTRS") load(":python_tag.bzl", "python_tag") load(":requirements_files_by_platform.bzl", "requirements_files_by_platform") @@ -80,21 +81,27 @@ def _platforms(*, python_version, minor_mapping, config): for platform, values in config.platforms.items(): # TODO @aignas 2025-07-07: this is probably doing the parsing of the version too # many times. - key = "{}{}{}.{}_{}".format( + abi = "{}{}{}.{}".format( python_tag(values.env["implementation_name"]), python_version.release[0], python_version.release[1], python_version.release[2], - platform, ) + key = "{}_{}".format(abi, platform) + + env_ = env( + env = values.env, + os = values.os_name, + arch = values.arch_name, + python_version = python_version.string, + ) + + if values.marker and not evaluate(values.marker, env = env_): + continue platforms[key] = struct( - env = env( - env = values.env, - os = values.os_name, - arch = values.arch_name, - python_version = python_version.string, - ), + env = env_, + triple = "{}_{}_{}".format(abi, values.os_name, values.arch_name), whl_abi_tags = [ v.format( major = python_version.release[0], @@ -203,17 +210,19 @@ def _create_whl_repos( whl_group_mapping = {} requirement_cycles = {} + platforms = _platforms( + python_version = pip_attr.python_version, + minor_mapping = minor_mapping, + config = config, + ) + if evaluate_markers: # This is most likely unit tests pass elif config.enable_pipstar: evaluate_markers = lambda _, requirements: evaluate_markers_star( requirements = requirements, - platforms = _platforms( - python_version = pip_attr.python_version, - minor_mapping = minor_mapping, - config = config, - ), + platforms = platforms, ) else: # NOTE @aignas 2024-08-02: , we will execute any interpreter that we find either @@ -232,7 +241,13 @@ def _create_whl_repos( # spin up a Python interpreter. evaluate_markers = lambda module_ctx, requirements: evaluate_markers_py( module_ctx, - requirements = requirements, + requirements = { + k: { + p: platforms[p].triple + for p in plats + } + for k, plats in requirements.items() + }, python_interpreter = pip_attr.python_interpreter, python_interpreter_target = python_interpreter_target, srcs = pip_attr._evaluate_markers_srcs, @@ -248,18 +263,14 @@ def _create_whl_repos( requirements_osx = pip_attr.requirements_darwin, requirements_windows = pip_attr.requirements_windows, extra_pip_args = pip_attr.extra_pip_args, - platforms = sorted(config.platforms), # here we only need keys + platforms = sorted(platforms), # here we only need keys python_version = full_version( version = pip_attr.python_version, minor_mapping = minor_mapping, ), logger = logger, ), - platforms = _platforms( - python_version = pip_attr.python_version, - minor_mapping = minor_mapping, - config = config, - ), + platforms = platforms, extra_pip_args = pip_attr.extra_pip_args, get_index_urls = get_index_urls, evaluate_markers = evaluate_markers, @@ -344,8 +355,19 @@ def _create_whl_repos( repo_name, whl.name, )) - whl_libraries[repo_name] = repo.args + + if not config.enable_pipstar and "experimental_target_platforms" in repo.args: + whl_libraries[repo_name] |= { + "experimental_target_platforms": sorted({ + # TODO @aignas 2025-07-07: this should be solved in a better way + platforms[candidate].triple.partition("_")[-1]: None + for p in repo.args["experimental_target_platforms"] + for candidate in platforms + if candidate.endswith(p) + }), + } + mapping = whl_map.setdefault(whl.name, {}) if repo.config_setting in mapping and mapping[repo.config_setting] != repo_name: fail( @@ -436,7 +458,7 @@ def _whl_repo( ), ) -def _plat(*, name, arch_name, os_name, config_settings = [], env = {}, whl_abi_tags = [], whl_platform_tags = []): +def _plat(*, name, arch_name, os_name, config_settings = [], env = {}, marker = "", whl_abi_tags = [], whl_platform_tags = []): # NOTE @aignas 2025-07-08: the least preferred is the first item in the list if "any" not in whl_platform_tags: # the lowest priority one needs to be the first one @@ -456,6 +478,7 @@ def _plat(*, name, arch_name, os_name, config_settings = [], env = {}, whl_abi_t # defaults for env "implementation_name": "cpython", } | env, + marker = marker, whl_abi_tags = whl_abi_tags, whl_platform_tags = whl_platform_tags, ) @@ -503,13 +526,14 @@ def build_config( config_settings = tag.config_settings, env = tag.env, os_name = tag.os_name, + marker = tag.marker, name = platform.replace("-", "_").lower(), whl_abi_tags = tag.whl_abi_tags, whl_platform_tags = tag.whl_platform_tags, override = mod.is_root, ) - if platform and not (tag.arch_name or tag.config_settings or tag.env or tag.os_name or tag.whl_abi_tags or tag.whl_platform_tags): + if platform and not (tag.arch_name or tag.config_settings or tag.env or tag.os_name or tag.whl_abi_tags or tag.whl_platform_tags or tag.marker): defaults["platforms"].pop(platform) _configure( @@ -916,6 +940,20 @@ Supported keys: ::::{note} This is only used if the {envvar}`RULES_PYTHON_ENABLE_PIPSTAR` is enabled. :::: +""", + ), + "marker": attr.string( + doc = """\ +An environment marker expression that is used to enable/disable platforms for specific python +versions, operating systems or CPU architectures. + +If specified, the expression is evaluated during the `bzlmod` extension evaluation phase and if it +evaluates to `True`, then the platform will be used to construct the hub repositories, otherwise, it +will be skipped. + +This is especially useful for setting up freethreaded platform variants only for particular Python +versions for which the interpreter builds are available. However, this could be also used for other +things, such as setting up platforms for different `libc` variants. """, ), # The values for PEP508 env marker evaluation during the lock file parsing diff --git a/python/private/pypi/pip_repository.bzl b/python/private/pypi/pip_repository.bzl index 5ad388d8ea..6d539a5f24 100644 --- a/python/private/pypi/pip_repository.bzl +++ b/python/private/pypi/pip_repository.bzl @@ -95,7 +95,12 @@ def _pip_repository_impl(rctx): extra_pip_args = rctx.attr.extra_pip_args, evaluate_markers = lambda rctx, requirements: evaluate_markers_py( rctx, - requirements = requirements, + requirements = { + # NOTE @aignas 2025-07-07: because we don't distinguish between + # freethreaded and non-freethreaded, it is a 1:1 mapping. + req: {p: p for p in plats} + for req, plats in requirements.items() + }, python_interpreter = rctx.attr.python_interpreter, python_interpreter_target = rctx.attr.python_interpreter_target, srcs = rctx.attr._evaluate_markers_srcs, diff --git a/python/private/pypi/requirements_files_by_platform.bzl b/python/private/pypi/requirements_files_by_platform.bzl index d8d3651461..356bd4416e 100644 --- a/python/private/pypi/requirements_files_by_platform.bzl +++ b/python/private/pypi/requirements_files_by_platform.bzl @@ -37,7 +37,9 @@ def _default_platforms(*, filter, platforms): if not prefix: return platforms - match = [p for p in platforms if p.startswith(prefix)] + match = [p for p in platforms if p.startswith(prefix) or ( + p.startswith("cp") and p.partition("_")[-1].startswith(prefix) + )] else: match = [p for p in platforms if filter in p] @@ -140,7 +142,7 @@ def requirements_files_by_platform( if logger: logger.debug(lambda: "Platforms from pip args: {}".format(platforms_from_args)) - default_platforms = [_platform(p, python_version) for p in platforms] + default_platforms = platforms if platforms_from_args: lock_files = [ @@ -252,6 +254,6 @@ def requirements_files_by_platform( ret = {} for plat, file in requirements.items(): - ret.setdefault(file, []).append(plat) + ret.setdefault(file, []).append(_platform(plat, python_version = python_version)) return ret diff --git a/python/private/pypi/requirements_parser/resolve_target_platforms.py b/python/private/pypi/requirements_parser/resolve_target_platforms.py index c899a943cc..accacf5bfa 100755 --- a/python/private/pypi/requirements_parser/resolve_target_platforms.py +++ b/python/private/pypi/requirements_parser/resolve_target_platforms.py @@ -50,8 +50,8 @@ def main(): hashes = prefix + hashes req = Requirement(entry) - for p in target_platforms: - (platform,) = Platform.from_string(p) + for p, triple in target_platforms.items(): + (platform,) = Platform.from_string(triple) if not req.marker or req.marker.evaluate(platform.env_markers("")): response.setdefault(requirement_line, []).append(p) diff --git a/tests/pypi/extension/extension_tests.bzl b/tests/pypi/extension/extension_tests.bzl index b85414528d..55de99b7d9 100644 --- a/tests/pypi/extension/extension_tests.bzl +++ b/tests/pypi/extension/extension_tests.bzl @@ -58,20 +58,22 @@ def _mod(*, name, default = [], parse = [], override = [], whl_mods = [], is_roo whl_mods = whl_mods, default = default or [ _default( - platform = "{}_{}".format(os, cpu), + platform = "{}_{}{}".format(os, cpu, freethreaded), os_name = os, arch_name = cpu, config_settings = [ "@platforms//os:{}".format(os), "@platforms//cpu:{}".format(cpu), ], + whl_abi_tags = ["cp{major}{minor}t"] if freethreaded else ["abi3", "cp{major}{minor}"], whl_platform_tags = whl_platform_tags, ) - for (os, cpu), whl_platform_tags in { - ("linux", "x86_64"): ["linux_*_x86_64", "manylinux_*_x86_64"], - ("linux", "aarch64"): ["linux_*_aarch64", "manylinux_*_aarch64"], - ("osx", "aarch64"): ["macosx_*_arm64"], - ("windows", "aarch64"): ["win_arm64"], + for (os, cpu, freethreaded), whl_platform_tags in { + ("linux", "x86_64", ""): ["linux_x86_64", "manylinux_*_x86_64"], + ("linux", "x86_64", "_freethreaded"): ["linux_x86_64", "manylinux_*_x86_64"], + ("linux", "aarch64", ""): ["linux_aarch64", "manylinux_*_aarch64"], + ("osx", "aarch64", ""): ["macosx_*_arm64"], + ("windows", "aarch64", ""): ["win_arm64"], }.items() ], ), @@ -113,6 +115,7 @@ def _default( auth_patterns = None, config_settings = None, env = None, + marker = None, netrc = None, os_name = None, platform = None, @@ -123,6 +126,7 @@ def _default( auth_patterns = auth_patterns or {}, config_settings = config_settings, env = env or {}, + marker = marker or "", netrc = netrc, os_name = os_name, platform = platform, @@ -453,10 +457,11 @@ torch==2.4.1 ; platform_machine != 'x86_64' \ version = "3.15", ), ], - "pypi_315_torch_linux_x86_64": [ + "pypi_315_torch_linux_x86_64_linux_x86_64_freethreaded": [ whl_config_setting( target_platforms = [ "cp315_linux_x86_64", + "cp315_linux_x86_64_freethreaded", ], version = "3.15", ), @@ -469,7 +474,7 @@ torch==2.4.1 ; platform_machine != 'x86_64' \ "python_interpreter_target": "unit_test_interpreter_target", "requirement": "torch==2.4.1 --hash=sha256:deadbeef", }, - "pypi_315_torch_linux_x86_64": { + "pypi_315_torch_linux_x86_64_linux_x86_64_freethreaded": { "dep_template": "@pypi//{name}:{target}", "python_interpreter_target": "unit_test_interpreter_target", "requirement": "torch==2.4.1+cpu", @@ -859,6 +864,7 @@ git_dep @ git+https://git.server/repo/project@deadbeefdeadbeef target_platforms = ( "cp315_linux_aarch64", "cp315_linux_x86_64", + "cp315_linux_x86_64_freethreaded", "cp315_osx_aarch64", "cp315_windows_aarch64", ), @@ -872,6 +878,7 @@ git_dep @ git+https://git.server/repo/project@deadbeefdeadbeef target_platforms = ( "cp315_linux_aarch64", "cp315_linux_x86_64", + "cp315_linux_x86_64_freethreaded", "cp315_osx_aarch64", "cp315_windows_aarch64", ), @@ -899,6 +906,7 @@ git_dep @ git+https://git.server/repo/project@deadbeefdeadbeef target_platforms = ( "cp315_linux_aarch64", "cp315_linux_x86_64", + "cp315_linux_x86_64_freethreaded", "cp315_osx_aarch64", "cp315_windows_aarch64", ), @@ -912,6 +920,7 @@ git_dep @ git+https://git.server/repo/project@deadbeefdeadbeef target_platforms = ( "cp315_linux_aarch64", "cp315_linux_x86_64", + "cp315_linux_x86_64_freethreaded", "cp315_osx_aarch64", "cp315_windows_aarch64", ), @@ -925,6 +934,7 @@ git_dep @ git+https://git.server/repo/project@deadbeefdeadbeef target_platforms = ( "cp315_linux_aarch64", "cp315_linux_x86_64", + "cp315_linux_x86_64_freethreaded", "cp315_osx_aarch64", "cp315_windows_aarch64", ), @@ -1078,12 +1088,13 @@ optimum[onnxruntime-gpu]==1.17.1 ; sys_platform == 'linux' pypi.hub_whl_map().contains_exactly({ "pypi": { "optimum": { - "pypi_315_optimum_linux_aarch64_linux_x86_64": [ + "pypi_315_optimum_linux_aarch64_linux_x86_64_linux_x86_64_freethreaded": [ whl_config_setting( version = "3.15", target_platforms = [ "cp315_linux_aarch64", "cp315_linux_x86_64", + "cp315_linux_x86_64_freethreaded", ], ), ], @@ -1100,7 +1111,7 @@ optimum[onnxruntime-gpu]==1.17.1 ; sys_platform == 'linux' }) pypi.whl_libraries().contains_exactly({ - "pypi_315_optimum_linux_aarch64_linux_x86_64": { + "pypi_315_optimum_linux_aarch64_linux_x86_64_linux_x86_64_freethreaded": { "dep_template": "@pypi//{name}:{target}", "python_interpreter_target": "unit_test_interpreter_target", "requirement": "optimum[onnxruntime-gpu]==1.17.1", @@ -1126,6 +1137,7 @@ def _test_pipstar_platforms(env): platform = "my{}{}".format(os, cpu), os_name = os, arch_name = cpu, + marker = "python_version ~= \"3.13\"", config_settings = [ "@platforms//os:{}".format(os), "@platforms//cpu:{}".format(cpu), @@ -1248,6 +1260,7 @@ def _test_build_pipstar_platform(env): "@platforms//cpu:x86_64", ], env = {"implementation_name": "cpython"}, + marker = "", whl_abi_tags = ["none", "abi3", "cp{major}{minor}"], whl_platform_tags = ["any"], ), From 563c58510c785726c3c154c2332b52bf58ba2e3b Mon Sep 17 00:00:00 2001 From: honglooker Date: Thu, 21 Aug 2025 14:03:11 -0700 Subject: [PATCH 395/922] docs: correctly spell release in devguide (#3201) relase -> release --- docs/devguide.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/devguide.md b/docs/devguide.md index afb990588b..e7870b5733 100644 --- a/docs/devguide.md +++ b/docs/devguide.md @@ -121,7 +121,7 @@ we prepare for releases. The steps to create a backport PR are: -1. Create an issue for the patch release; use the [patch relase +1. Create an issue for the patch release; use the [patch release template][patch-release-issue]. 2. Create a fork of `rules_python`. 3. Checkout the `release/X.Y` branch. From fe45faabeb3dceab8766fb1a67131ec0cc1135dc Mon Sep 17 00:00:00 2001 From: Matt Pennig Date: Fri, 22 Aug 2025 17:32:49 -0500 Subject: [PATCH 396/922] fix(toolchains): Add Xcode repo env vars to local_runtime_repo for better cache invalidation (#3203) On macOS, if one writes a `local_runtime_repo` with `interpreter_path = "/usr/bin/python3"`, the path to python3 inside the selected _Xcode.app/Contents/Developer_ directory gets cached. If a developer changes that directory with `xcode-select --switch` that cached file with the old directory remains. Making the local_runtime_repo rule sensitive to DEVELOPER_DIR and XCODE_VERSION (two conventionally adopted env vars among the Bazel + Apple ecosystem) will ensure that if Xcode changes, so will the resolved python3 path. Fixes #3123 --- CHANGELOG.md | 3 +++ python/private/local_runtime_repo.bzl | 2 +- 2 files changed, 4 insertions(+), 1 deletion(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 37329e3fb8..0ab44208de 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -86,6 +86,9 @@ END_UNRELEASED_TEMPLATE {#v0-0-0-fixed} ### Fixed +* (toolchains) `local_runtime_repo` now respects changes to the `DEVELOPER_DIR` and `XCODE_VERSION` + repo env vars, fixing stale cache issues on macOS with system (i.e. Xcode-supplied) Python + ([#3123](https://github.com/bazel-contrib/rules_python/issues/3123)). * (pypi) Fixes an issue where builds using a `bazel vendor` vendor directory would fail if the constraints file contained environment markers. Fixes [#2996](https://github.com/bazel-contrib/rules_python/issues/2996). diff --git a/python/private/local_runtime_repo.bzl b/python/private/local_runtime_repo.bzl index 21bdfa627e..c053a03508 100644 --- a/python/private/local_runtime_repo.bzl +++ b/python/private/local_runtime_repo.bzl @@ -232,7 +232,7 @@ How to handle errors when trying to automatically determine settings. ), "_rule_name": attr.string(default = "local_runtime_repo"), }, - environ = ["PATH", REPO_DEBUG_ENV_VAR], + environ = ["PATH", REPO_DEBUG_ENV_VAR, "DEVELOPER_DIR", "XCODE_VERSION"], ) def _expand_incompatible_template(): From 24146a49cc34269d1dd7f7cd334fa80e0c8a2935 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Sat, 23 Aug 2025 21:41:57 -0700 Subject: [PATCH 397/922] docs: update for 1.6 release (#3205) Doc updates for 1.6 release Work towards https://github.com/bazel-contrib/rules_python/issues/3188 --- CHANGELOG.md | 14 +++++++------- docs/pypi/use.md | 2 +- gazelle/docs/annotations.md | 2 +- python/private/pypi/extension.bzl | 2 +- 4 files changed, 10 insertions(+), 10 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 0ab44208de..fc3d7bbce3 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -47,12 +47,12 @@ BEGIN_UNRELEASED_TEMPLATE END_UNRELEASED_TEMPLATE --> -{#v0-0-0} -## Unreleased +{#1-6-0} +## [1.6.0] - 2025-08-23 -[0.0.0]: https://github.com/bazel-contrib/rules_python/releases/tag/0.0.0 +[1.6.0]: https://github.com/bazel-contrib/rules_python/releases/tag/1.6.0 -{#v0-0-0-changed} +{#1-6-0-changed} ### Changed * (gazelle) update minimum gazelle version to 0.36.0 - may cause BUILD file changes * (gazelle) update minimum rules_go version to 0.55.1 @@ -84,7 +84,7 @@ END_UNRELEASED_TEMPLATE [20250808]: https://github.com/astral-sh/python-build-standalone/releases/tag/20250808 -{#v0-0-0-fixed} +{#1-6-0-fixed} ### Fixed * (toolchains) `local_runtime_repo` now respects changes to the `DEVELOPER_DIR` and `XCODE_VERSION` repo env vars, fixing stale cache issues on macOS with system (i.e. Xcode-supplied) Python @@ -131,7 +131,7 @@ END_UNRELEASED_TEMPLATE ([#2797](https://github.com/bazel-contrib/rules_python/issues/2797)). * (py_wheel) Add directories in deterministic order. -{#v0-0-0-added} +{#1-6-0-added} ### Added * (repl) Default stub now has tab completion, where `readline` support is available, see ([#3114](https://github.com/bazel-contrib/rules_python/pull/3114)). @@ -162,7 +162,7 @@ END_UNRELEASED_TEMPLATE * (gazelle) New directive `gazelle:python_proto_naming_convention`; controls naming of `py_proto_library` rules. -{#v0-0-0-removed} +{#1-6-0-removed} ### Removed * Nothing removed. diff --git a/docs/pypi/use.md b/docs/pypi/use.md index a668167114..9d0c54c4ab 100644 --- a/docs/pypi/use.md +++ b/docs/pypi/use.md @@ -45,7 +45,7 @@ Note that the hub repo contains the following targets for each package: * `@pypi//numpy:whl` - the {obj}`filegroup` that is the `.whl` file itself, which includes all transitive dependencies via the {attr}`filegroup.data` attribute. -:::{versionadded} VERSION_NEXT_FEATURE +:::{versionadded} 1.6.0 The `:extracted_whl_files` target was added ::: diff --git a/gazelle/docs/annotations.md b/gazelle/docs/annotations.md index da6e58f7f8..728027ffda 100644 --- a/gazelle/docs/annotations.md +++ b/gazelle/docs/annotations.md @@ -118,7 +118,7 @@ deps = [ ## `include_pytest_conftest` -:::{versionadded} VERSION_NEXT_FEATURE +:::{versionadded} 1.6.0 {gh-pr}`3080` ::: diff --git a/python/private/pypi/extension.bzl b/python/private/pypi/extension.bzl index 331ecf2340..03af863e1e 100644 --- a/python/private/pypi/extension.bzl +++ b/python/private/pypi/extension.bzl @@ -1314,7 +1314,7 @@ terms used in this extension. [environment_markers]: https://packaging.python.org/en/latest/specifications/dependency-specifiers/#environment-markers ::: -:::{versionadded} VERSION_NEXT_FEATURE +:::{versionadded} 1.6.0 ::: """, ), From 06eaaa29a908cf81ac14881de18799f1675beabf Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Sun, 24 Aug 2025 14:44:05 -0700 Subject: [PATCH 398/922] fix(bootstrap): handle when runfiles env vars don't point to current binary's runfiles (#3192) The stage1 bootstrap script had a bug in the find_runfiles_root function where it would unconditionally use the RUNFILES_DIR et al environment variables if they were set. This failed in a particular nested context: an outer binary calling an inner binary when the inner binary isn't a data dependency of the outer binary (i.e. the outer doesn't contain the inner in runfiles). This would cause the inner binary to incorrectly resolve its runfiles, leading to failures. Such a case can occur if a genrule calls the outer binary, which has the inner binary passed as an arg. This change adds a check to validate that the script's entry point exists within the inherited RUNFILES_DIR before using it. If the entry point is not found, it proceeds with other runfiles discovery methods. This matches the system_python runfiles discovery logic. Fixes https://github.com/bazel-contrib/rules_python/issues/3187 --- CHANGELOG.md | 25 +++++- python/private/python_bootstrap_template.txt | 3 + python/private/stage1_bootstrap_template.sh | 19 ++-- .../bootstrap_impls/bin_calls_bin/BUILD.bazel | 86 +++++++++++++++++++ tests/bootstrap_impls/bin_calls_bin/inner.py | 4 + tests/bootstrap_impls/bin_calls_bin/outer.py | 18 ++++ tests/bootstrap_impls/bin_calls_bin/verify.sh | 32 +++++++ .../bin_calls_bin/verify_script_python.sh | 5 ++ .../bin_calls_bin/verify_system_python.sh | 5 ++ tests/support/support.bzl | 5 ++ 10 files changed, 196 insertions(+), 6 deletions(-) create mode 100644 tests/bootstrap_impls/bin_calls_bin/BUILD.bazel create mode 100644 tests/bootstrap_impls/bin_calls_bin/inner.py create mode 100644 tests/bootstrap_impls/bin_calls_bin/outer.py create mode 100755 tests/bootstrap_impls/bin_calls_bin/verify.sh create mode 100755 tests/bootstrap_impls/bin_calls_bin/verify_script_python.sh create mode 100755 tests/bootstrap_impls/bin_calls_bin/verify_system_python.sh diff --git a/CHANGELOG.md b/CHANGELOG.md index fc3d7bbce3..03eccf881e 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -47,6 +47,29 @@ BEGIN_UNRELEASED_TEMPLATE END_UNRELEASED_TEMPLATE --> +{#v0-0-0} +## Unreleased + +[0.0.0]: https://github.com/bazel-contrib/rules_python/releases/tag/0.0.0 + +{#v0-0-0-changed} +### Changed +* Nothing changed. + +{#v0-0-0-fixed} +### Fixed +* (bootstrap) The stage1 bootstrap script now correctly handles nested `RUNFILES_DIR` + environments, fixing issues where a `py_binary` calls another `py_binary` + ([#3187](https://github.com/bazel-contrib/rules_python/issues/3187)). + +{#v0-0-0-added} +### Added +* Nothing added. + +{#v0-0-0-removed} +### Removed +* Nothing removed. + {#1-6-0} ## [1.6.0] - 2025-08-23 @@ -102,7 +125,7 @@ END_UNRELEASED_TEMPLATE name. * (pypi) The selection of the whls has been changed and should no longer result in ambiguous select matches ({gh-issue}`2759`) and should be much more efficient - when running `bazel query` due to fewer repositories being included + when running `bazel query` due to fewer repositories being included ({gh-issue}`2849`). * Multi-line python imports (e.g. with escaped newlines) are now correctly processed by Gazelle. * (toolchains) `local_runtime_repo` works with multiarch Debian with Python 3.8 diff --git a/python/private/python_bootstrap_template.txt b/python/private/python_bootstrap_template.txt index 62ded87337..495a52cfe9 100644 --- a/python/private/python_bootstrap_template.txt +++ b/python/private/python_bootstrap_template.txt @@ -516,6 +516,9 @@ def Main(): module_space = FindModuleSpace(main_rel_path) delete_module_space = False + if os.environ.get("RULES_PYTHON_TESTING_TELL_MODULE_SPACE"): + new_env["RULES_PYTHON_TESTING_MODULE_SPACE"] = module_space + python_imports = '%imports%' python_path_entries = CreatePythonPathEntries(python_imports, module_space) python_path_entries += GetRepositoriesImports(module_space, %import_all%) diff --git a/python/private/stage1_bootstrap_template.sh b/python/private/stage1_bootstrap_template.sh index 9927d4faa7..a984344647 100644 --- a/python/private/stage1_bootstrap_template.sh +++ b/python/private/stage1_bootstrap_template.sh @@ -61,14 +61,20 @@ if [[ "$IS_ZIPFILE" == "1" ]]; then else function find_runfiles_root() { + local maybe_root="" if [[ -n "${RUNFILES_DIR:-}" ]]; then - echo "$RUNFILES_DIR" - return 0 + maybe_root="$RUNFILES_DIR" elif [[ "${RUNFILES_MANIFEST_FILE:-}" = *".runfiles_manifest" ]]; then - echo "${RUNFILES_MANIFEST_FILE%%.runfiles_manifest}.runfiles" - return 0 + maybe_root="${RUNFILES_MANIFEST_FILE%%.runfiles_manifest}.runfiles" elif [[ "${RUNFILES_MANIFEST_FILE:-}" = *".runfiles/MANIFEST" ]]; then - echo "${RUNFILES_MANIFEST_FILE%%.runfiles/MANIFEST}.runfiles" + maybe_root="${RUNFILES_MANIFEST_FILE%%.runfiles/MANIFEST}.runfiles" + fi + + # The RUNFILES_DIR et al variables may misreport the runfiles directory + # if an outer binary invokes this binary when it isn't a data dependency. + # e.g. a genrule calls `bazel-bin/outer --inner=bazel-bin/inner` + if [[ -n "$maybe_root" && -e "$maybe_root/$STAGE2_BOOTSTRAP" ]]; then + echo "$maybe_root" return 0 fi @@ -99,6 +105,9 @@ else RUNFILES_DIR=$(find_runfiles_root $0) fi +if [[ -n "$RULES_PYTHON_TESTING_TELL_MODULE_SPACE" ]]; then + export RULES_PYTHON_TESTING_MODULE_SPACE="$RUNFILES_DIR" +fi function find_python_interpreter() { runfiles_root="$1" diff --git a/tests/bootstrap_impls/bin_calls_bin/BUILD.bazel b/tests/bootstrap_impls/bin_calls_bin/BUILD.bazel new file mode 100644 index 0000000000..02835fb77b --- /dev/null +++ b/tests/bootstrap_impls/bin_calls_bin/BUILD.bazel @@ -0,0 +1,86 @@ +load("@rules_shell//shell:sh_test.bzl", "sh_test") +load("//tests/support:py_reconfig.bzl", "py_reconfig_binary") +load("//tests/support:support.bzl", "NOT_WINDOWS", "SUPPORTS_BOOTSTRAP_SCRIPT") + +# ===== +# bootstrap_impl=system_python testing +# ===== +py_reconfig_binary( + name = "outer_bootstrap_system_python", + srcs = ["outer.py"], + bootstrap_impl = "system_python", + main = "outer.py", + tags = ["manual"], +) + +py_reconfig_binary( + name = "inner_bootstrap_system_python", + srcs = ["inner.py"], + bootstrap_impl = "system_python", + main = "inner.py", + tags = ["manual"], +) + +genrule( + name = "outer_calls_inner_system_python", + outs = ["outer_calls_inner_system_python.out"], + cmd = "RULES_PYTHON_TESTING_TELL_MODULE_SPACE=1 $(location :outer_bootstrap_system_python) $(location :inner_bootstrap_system_python) > $@", + tags = ["manual"], + tools = [ + ":inner_bootstrap_system_python", + ":outer_bootstrap_system_python", + ], +) + +sh_test( + name = "bootstrap_system_python_test", + srcs = ["verify_system_python.sh"], + data = [ + "verify.sh", + ":outer_calls_inner_system_python", + ], + # The way verify_system_python.sh loads verify.sh doesn't work + # with Windows for some annoying reason. Just skip windows for now; + # the logic being test isn't OS-specific, so this should be fine. + target_compatible_with = NOT_WINDOWS, +) + +# ===== +# bootstrap_impl=script testing +# ===== +py_reconfig_binary( + name = "inner_bootstrap_script", + srcs = ["inner.py"], + bootstrap_impl = "script", + main = "inner.py", + tags = ["manual"], +) + +py_reconfig_binary( + name = "outer_bootstrap_script", + srcs = ["outer.py"], + bootstrap_impl = "script", + main = "outer.py", + tags = ["manual"], +) + +genrule( + name = "outer_calls_inner_script_python", + outs = ["outer_calls_inner_script_python.out"], + cmd = "RULES_PYTHON_TESTING_TELL_MODULE_SPACE=1 $(location :outer_bootstrap_script) $(location :inner_bootstrap_script) > $@", + tags = ["manual"], + tools = [ + ":inner_bootstrap_script", + ":outer_bootstrap_script", + ], +) + +sh_test( + name = "bootstrap_script_python_test", + srcs = ["verify_script_python.sh"], + data = [ + "verify.sh", + ":outer_calls_inner_script_python", + ], + target_compatible_with = SUPPORTS_BOOTSTRAP_SCRIPT, +) diff --git a/tests/bootstrap_impls/bin_calls_bin/inner.py b/tests/bootstrap_impls/bin_calls_bin/inner.py new file mode 100644 index 0000000000..e67b31dda3 --- /dev/null +++ b/tests/bootstrap_impls/bin_calls_bin/inner.py @@ -0,0 +1,4 @@ +import os + +module_space = os.environ.get("RULES_PYTHON_TESTING_MODULE_SPACE") +print(f"inner: RULES_PYTHON_TESTING_MODULE_SPACE='{module_space}'") diff --git a/tests/bootstrap_impls/bin_calls_bin/outer.py b/tests/bootstrap_impls/bin_calls_bin/outer.py new file mode 100644 index 0000000000..19dac06eb7 --- /dev/null +++ b/tests/bootstrap_impls/bin_calls_bin/outer.py @@ -0,0 +1,18 @@ +import os +import subprocess +import sys + +if __name__ == "__main__": + module_space = os.environ.get("RULES_PYTHON_TESTING_MODULE_SPACE") + print(f"outer: RULES_PYTHON_TESTING_MODULE_SPACE='{module_space}'") + + inner_binary_path = sys.argv[1] + result = subprocess.run( + [inner_binary_path], + capture_output=True, + text=True, + check=True, + ) + print(result.stdout, end="") + if result.stderr: + print(result.stderr, end="", file=sys.stderr) diff --git a/tests/bootstrap_impls/bin_calls_bin/verify.sh b/tests/bootstrap_impls/bin_calls_bin/verify.sh new file mode 100755 index 0000000000..433704e9ab --- /dev/null +++ b/tests/bootstrap_impls/bin_calls_bin/verify.sh @@ -0,0 +1,32 @@ +#!/bin/bash +set -euo pipefail + +verify_output() { + local OUTPUT_FILE=$1 + + # Extract the RULES_PYTHON_TESTING_MODULE_SPACE values + local OUTER_MODULE_SPACE=$(grep "outer: RULES_PYTHON_TESTING_MODULE_SPACE" "$OUTPUT_FILE" | sed "s/outer: RULES_PYTHON_TESTING_MODULE_SPACE='\(.*\)'/\1/") + local INNER_MODULE_SPACE=$(grep "inner: RULES_PYTHON_TESTING_MODULE_SPACE" "$OUTPUT_FILE" | sed "s/inner: RULES_PYTHON_TESTING_MODULE_SPACE='\(.*\)'/\1/") + + echo "Outer module space: $OUTER_MODULE_SPACE" + echo "Inner module space: $INNER_MODULE_SPACE" + + # Check 1: The two values are different + if [ "$OUTER_MODULE_SPACE" == "$INNER_MODULE_SPACE" ]; then + echo "Error: Outer and Inner module spaces are the same." + exit 1 + fi + + # Check 2: Inner is not a subdirectory of Outer + case "$INNER_MODULE_SPACE" in + "$OUTER_MODULE_SPACE"/*) + echo "Error: Inner module space is a subdirectory of Outer's." + exit 1 + ;; + *) + # This is the success case + ;; + esac + + echo "Verification successful." +} diff --git a/tests/bootstrap_impls/bin_calls_bin/verify_script_python.sh b/tests/bootstrap_impls/bin_calls_bin/verify_script_python.sh new file mode 100755 index 0000000000..012daee05b --- /dev/null +++ b/tests/bootstrap_impls/bin_calls_bin/verify_script_python.sh @@ -0,0 +1,5 @@ +#!/bin/bash +set -euo pipefail + +source "$(dirname "$0")/verify.sh" +verify_output "$(dirname "$0")/outer_calls_inner_script_python.out" diff --git a/tests/bootstrap_impls/bin_calls_bin/verify_system_python.sh b/tests/bootstrap_impls/bin_calls_bin/verify_system_python.sh new file mode 100755 index 0000000000..460769fd04 --- /dev/null +++ b/tests/bootstrap_impls/bin_calls_bin/verify_system_python.sh @@ -0,0 +1,5 @@ +#!/bin/bash +set -euo pipefail + +source "$(dirname "$0")/verify.sh" +verify_output "$(dirname "$0")/outer_calls_inner_system_python.out" diff --git a/tests/support/support.bzl b/tests/support/support.bzl index adb8e75f71..f8694629c1 100644 --- a/tests/support/support.bzl +++ b/tests/support/support.bzl @@ -54,3 +54,8 @@ SUPPORTS_BZLMOD_UNIXY = select({ "@platforms//os:windows": ["@platforms//:incompatible"], "//conditions:default": [], }) if BZLMOD_ENABLED else ["@platforms//:incompatible"] + +NOT_WINDOWS = select({ + "@platforms//os:windows": ["@platforms//:incompatible"], + "//conditions:default": [], +}) From fb9b098f7f9aee57ef997392eab36a8a8debc138 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Sun, 24 Aug 2025 15:34:45 -0700 Subject: [PATCH 399/922] docs: fix a couple typos in the changelog (#3208) I ran Jules against the changelog to look for typos. It found a couple small ones. --------- Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com> --- CHANGELOG.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 03eccf881e..4bc14f20f7 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -610,7 +610,7 @@ END_UNRELEASED_TEMPLATE To select the free-threaded interpreter in the repo phase, please use the documented [env](environment-variables) variables. Fixes [#2386](https://github.com/bazel-contrib/rules_python/issues/2386). -* (toolchains) Use the latest astrahl-sh toolchain release [20241206] for Python versions: +* (toolchains) Use the latest astral-sh toolchain release [20241206] for Python versions: * 3.9.21 * 3.10.16 * 3.11.11 @@ -665,7 +665,7 @@ Other changes: * (binaries/tests) For {obj}`--bootstrap_impl=script`, a binary-specific (but otherwise empty) virtual env is used to customize `sys.path` initialization. * (deps) bazel_skylib 1.7.0 (workspace; bzlmod already specifying that version) -* (deps) bazel_features 1.21.0; necessary for compatiblity with Bazel 8 rc3 +* (deps) bazel_features 1.21.0; necessary for compatibility with Bazel 8 rc3 * (deps) stardoc 0.7.2 to support Bazel 8. {#v1-0-0-fixed} @@ -1573,7 +1573,7 @@ Other changes: * **BREAKING** Support for Bazel 5 has been officially dropped. This release was only partially tested with Bazel 5 and may or may not work with Bazel 5. - Subequent versions will no longer be tested under Bazel 5. + Subsequent versions will no longer be tested under Bazel 5. * (runfiles) `rules_python.python.runfiles` now directly implements type hints and drops support for python2 as a result. From d9fe62c11b11f70fdc47037f93d972794ce3c347 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Mon, 25 Aug 2025 20:31:51 -0700 Subject: [PATCH 400/922] chore: release helper tool (#3206) Right now, it just updates the changelog and replaces the version placeholders. --- RELEASING.md | 12 +- tests/tools/private/release/BUILD.bazel | 7 + tests/tools/private/release/release_test.py | 174 ++++++++++++++++++++ tools/private/release/BUILD.bazel | 9 + tools/private/release/release.py | 127 ++++++++++++++ 5 files changed, 321 insertions(+), 8 deletions(-) create mode 100644 tests/tools/private/release/BUILD.bazel create mode 100644 tests/tools/private/release/release_test.py create mode 100644 tools/private/release/BUILD.bazel create mode 100644 tools/private/release/release.py diff --git a/RELEASING.md b/RELEASING.md index c9d46c39f0..a99b7d8d00 100644 --- a/RELEASING.md +++ b/RELEASING.md @@ -13,14 +13,10 @@ These are the steps for a regularly scheduled release from HEAD. ### Steps 1. [Determine the next semantic version number](#determining-semantic-version). -1. Update CHANGELOG.md: replace the `v0-0-0` and `0.0.0` with `X.Y.0`. - ``` - awk -v version=X.Y.0 'BEGIN { hv=version; gsub(/\./, "-", hv) } /END_UNRELEASED_TEMPLATE/ { found_marker = 1 } found_marker { gsub(/v0-0-0/, hv, $0); gsub(/Unreleased/, "[" version "] - " strftime("%Y-%m-%d"), $0); gsub(/0.0.0/, version, $0); } { print } ' CHANGELOG.md > /tmp/changelog && cp /tmp/changelog CHANGELOG.md - ``` -1. Replace `VERSION_NEXT_*` strings with `X.Y.0`. - ``` - grep -l --exclude=CONTRIBUTING.md --exclude=RELEASING.md --exclude-dir=.* VERSION_NEXT_ -r \ - | xargs sed -i -e 's/VERSION_NEXT_FEATURE/X.Y.0/' -e 's/VERSION_NEXT_PATCH/X.Y.0/' +1. Update the changelog and replace the version placeholders by running the + release tool: + ```shell + bazel run //tools/private/release -- X.Y.Z ``` 1. Send these changes for review and get them merged. 1. Create a branch for the new release, named `release/X.Y` diff --git a/tests/tools/private/release/BUILD.bazel b/tests/tools/private/release/BUILD.bazel new file mode 100644 index 0000000000..3c9db2d4e9 --- /dev/null +++ b/tests/tools/private/release/BUILD.bazel @@ -0,0 +1,7 @@ +load("@rules_python//python:defs.bzl", "py_test") + +py_test( + name = "release_test", + srcs = ["release_test.py"], + deps = ["//tools/private/release"], +) diff --git a/tests/tools/private/release/release_test.py b/tests/tools/private/release/release_test.py new file mode 100644 index 0000000000..5f0446410b --- /dev/null +++ b/tests/tools/private/release/release_test.py @@ -0,0 +1,174 @@ +import datetime +import os +import pathlib +import shutil +import tempfile +import unittest + +from tools.private.release import release as releaser + +_UNRELEASED_TEMPLATE = """ + +""" + + +class ReleaserTest(unittest.TestCase): + def setUp(self): + self.tmpdir = pathlib.Path(tempfile.mkdtemp()) + self.original_cwd = os.getcwd() + self.addCleanup(shutil.rmtree, self.tmpdir) + + os.chdir(self.tmpdir) + # NOTE: On windows, this must be done before files are deleted. + self.addCleanup(os.chdir, self.original_cwd) + + def test_update_changelog(self): + changelog = f""" +# Changelog + +{_UNRELEASED_TEMPLATE} + +{{#v0-0-0}} +## Unreleased + +[0.0.0]: https://github.com/bazel-contrib/rules_python/releases/tag/0.0.0 + +{{#v0-0-0-changed}} +### Changed +* Nothing changed + +{{#v0-0-0-fixed}} +### Fixed +* Nothing fixed + +{{#v0-0-0-added}} +### Added +* Nothing added + +{{#v0-0-0-removed}} +### Removed +* Nothing removed. +""" + changelog_path = self.tmpdir / "CHANGELOG.md" + changelog_path.write_text(changelog) + + # Act + releaser.update_changelog( + "1.23.4", + "2025-01-01", + changelog_path=changelog_path, + ) + + # Assert + new_content = changelog_path.read_text() + + self.assertIn( + _UNRELEASED_TEMPLATE, new_content, msg=f"ACTUAL:\n\n{new_content}\n\n" + ) + self.assertIn(f"## [1.23.4] - 2025-01-01", new_content) + self.assertIn( + f"[1.23.4]: https://github.com/bazel-contrib/rules_python/releases/tag/1.23.4", + new_content, + ) + self.assertIn("{#v1-23-4}", new_content) + self.assertIn("{#v1-23-4-changed}", new_content) + self.assertIn("{#v1-23-4-fixed}", new_content) + self.assertIn("{#v1-23-4-added}", new_content) + self.assertIn("{#v1-23-4-removed}", new_content) + + def test_replace_version_next(self): + # Arrange + mock_file_content = """ +:::{versionadded} VERSION_NEXT_FEATURE +blabla +::: + +:::{versionchanged} VERSION_NEXT_PATCH +blabla +::: +""" + (self.tmpdir / "mock_file.bzl").write_text(mock_file_content) + + releaser.replace_version_next("0.28.0") + + new_content = (self.tmpdir / "mock_file.bzl").read_text() + + self.assertIn(":::{versionadded} 0.28.0", new_content) + self.assertIn(":::{versionadded} 0.28.0", new_content) + self.assertNotIn("VERSION_NEXT_FEATURE", new_content) + self.assertNotIn("VERSION_NEXT_PATCH", new_content) + + def test_replace_version_next_excludes_bazel_dirs(self): + # Arrange + mock_file_content = """ +:::{versionadded} VERSION_NEXT_FEATURE +blabla +::: +""" + bazel_dir = self.tmpdir / "bazel-rules_python" + bazel_dir.mkdir() + (bazel_dir / "mock_file.bzl").write_text(mock_file_content) + + tools_dir = self.tmpdir / "tools" / "private" / "release" + tools_dir.mkdir(parents=True) + (tools_dir / "mock_file.bzl").write_text(mock_file_content) + + tests_dir = self.tmpdir / "tests" / "tools" / "private" / "release" + tests_dir.mkdir(parents=True) + (tests_dir / "mock_file.bzl").write_text(mock_file_content) + + version = "0.28.0" + + # Act + releaser.replace_version_next(version) + + # Assert + new_content = (bazel_dir / "mock_file.bzl").read_text() + self.assertIn("VERSION_NEXT_FEATURE", new_content) + + new_content = (tools_dir / "mock_file.bzl").read_text() + self.assertIn("VERSION_NEXT_FEATURE", new_content) + + new_content = (tests_dir / "mock_file.bzl").read_text() + self.assertIn("VERSION_NEXT_FEATURE", new_content) + + def test_valid_version(self): + # These should not raise an exception + releaser.create_parser().parse_args(["0.28.0"]) + releaser.create_parser().parse_args(["1.0.0"]) + releaser.create_parser().parse_args(["1.2.3rc4"]) + + def test_invalid_version(self): + with self.assertRaises(SystemExit): + releaser.create_parser().parse_args(["0.28"]) + with self.assertRaises(SystemExit): + releaser.create_parser().parse_args(["a.b.c"]) + + +if __name__ == "__main__": + unittest.main() diff --git a/tools/private/release/BUILD.bazel b/tools/private/release/BUILD.bazel new file mode 100644 index 0000000000..9cd8ec2fba --- /dev/null +++ b/tools/private/release/BUILD.bazel @@ -0,0 +1,9 @@ +load("@rules_python//python:defs.bzl", "py_binary") + +package(default_visibility = ["//visibility:public"]) + +py_binary( + name = "release", + srcs = ["release.py"], + main = "release.py", +) diff --git a/tools/private/release/release.py b/tools/private/release/release.py new file mode 100644 index 0000000000..f37a5ff7de --- /dev/null +++ b/tools/private/release/release.py @@ -0,0 +1,127 @@ +"""A tool to perform release steps.""" + +import argparse +import datetime +import fnmatch +import os +import pathlib +import re + + +def update_changelog(version, release_date, changelog_path="CHANGELOG.md"): + """Performs the version replacements in CHANGELOG.md.""" + + header_version = version.replace(".", "-") + + changelog_path_obj = pathlib.Path(changelog_path) + lines = changelog_path_obj.read_text().splitlines() + + new_lines = [] + after_template = False + before_already_released = True + for line in lines: + if "END_UNRELEASED_TEMPLATE" in line: + after_template = True + if re.match("#v[1-9]-", line): + before_already_released = False + + if after_template and before_already_released: + line = line.replace("## Unreleased", f"## [{version}] - {release_date}") + line = line.replace("v0-0-0", f"v{header_version}") + line = line.replace("0.0.0", version) + + new_lines.append(line) + + changelog_path_obj.write_text("\n".join(new_lines)) + + +def replace_version_next(version): + """Replaces all VERSION_NEXT_* placeholders with the new version.""" + exclude_patterns = [ + "./.git/*", + "./.github/*", + "./.bazelci/*", + "./.bcr/*", + "./bazel-*/*", + "./CONTRIBUTING.md", + "./RELEASING.md", + "./tools/private/release/*", + "./tests/tools/private/release/*", + ] + + for root, dirs, files in os.walk(".", topdown=True): + # Filter directories + dirs[:] = [ + d + for d in dirs + if not any( + fnmatch.fnmatch(os.path.join(root, d), pattern) + for pattern in exclude_patterns + ) + ] + + for filename in files: + filepath = os.path.join(root, filename) + if any(fnmatch.fnmatch(filepath, pattern) for pattern in exclude_patterns): + continue + + try: + with open(filepath, "r") as f: + content = f.read() + except (IOError, UnicodeDecodeError): + # Ignore binary files or files with read errors + continue + + if "VERSION_NEXT_FEATURE" in content or "VERSION_NEXT_PATCH" in content: + new_content = content.replace("VERSION_NEXT_FEATURE", version) + new_content = new_content.replace("VERSION_NEXT_PATCH", version) + with open(filepath, "w") as f: + f.write(new_content) + + +def _semver_type(value): + if not re.match(r"^\d+\.\d+\.\d+(rc\d+)?$", value): + raise argparse.ArgumentTypeError( + f"'{value}' is not a valid semantic version (X.Y.Z or X.Y.ZrcN)" + ) + return value + + +def create_parser(): + """Creates the argument parser.""" + parser = argparse.ArgumentParser( + description="Automate release steps for rules_python." + ) + parser.add_argument( + "version", + help="The new release version (e.g., 0.28.0).", + type=_semver_type, + ) + return parser + + +def main(): + parser = create_parser() + args = parser.parse_args() + + if not re.match(r"^\d+\.\d+\.\d+(rc\d+)?$", args.version): + raise ValueError( + f"Version '{args.version}' is not a valid semantic version (X.Y.Z or X.Y.ZrcN)" + ) + + # Change to the workspace root so the script can be run from anywhere. + if "BUILD_WORKSPACE_DIRECTORY" in os.environ: + os.chdir(os.environ["BUILD_WORKSPACE_DIRECTORY"]) + + print("Updating changelog ...") + release_date = datetime.date.today().strftime("%Y-%m-%d") + update_changelog(args.version, release_date) + + print("Replacing VERSION_NEXT placeholders ...") + replace_version_next(args.version) + + print("Done") + + +if __name__ == "__main__": + main() From cebfc9d85c9397deb14340f4a5103ba8183dd144 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Tue, 26 Aug 2025 00:25:05 -0700 Subject: [PATCH 401/922] docs: fix changelog header anchors (#3207) It looks like back in v1.4 we copy/pasted incorrectly and forget to include the leading `v` in the anchors. The leading `v` is present because I found something (can't remember if it was Sphinx, MyST, or github) didn't like the anchors starting with numbers. Co-authored-by: Ignas Anikevicius <240938+aignas@users.noreply.github.com> --- CHANGELOG.md | 42 +++++++++++++++++++++--------------------- 1 file changed, 21 insertions(+), 21 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 4bc14f20f7..82a66eda7b 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -70,12 +70,12 @@ END_UNRELEASED_TEMPLATE ### Removed * Nothing removed. -{#1-6-0} +{#v1-6-0} ## [1.6.0] - 2025-08-23 [1.6.0]: https://github.com/bazel-contrib/rules_python/releases/tag/1.6.0 -{#1-6-0-changed} +{#v1-6-0-changed} ### Changed * (gazelle) update minimum gazelle version to 0.36.0 - may cause BUILD file changes * (gazelle) update minimum rules_go version to 0.55.1 @@ -107,7 +107,7 @@ END_UNRELEASED_TEMPLATE [20250808]: https://github.com/astral-sh/python-build-standalone/releases/tag/20250808 -{#1-6-0-fixed} +{#v1-6-0-fixed} ### Fixed * (toolchains) `local_runtime_repo` now respects changes to the `DEVELOPER_DIR` and `XCODE_VERSION` repo env vars, fixing stale cache issues on macOS with system (i.e. Xcode-supplied) Python @@ -154,7 +154,7 @@ END_UNRELEASED_TEMPLATE ([#2797](https://github.com/bazel-contrib/rules_python/issues/2797)). * (py_wheel) Add directories in deterministic order. -{#1-6-0-added} +{#v1-6-0-added} ### Added * (repl) Default stub now has tab completion, where `readline` support is available, see ([#3114](https://github.com/bazel-contrib/rules_python/pull/3114)). @@ -185,11 +185,11 @@ END_UNRELEASED_TEMPLATE * (gazelle) New directive `gazelle:python_proto_naming_convention`; controls naming of `py_proto_library` rules. -{#1-6-0-removed} +{#v1-6-0-removed} ### Removed * Nothing removed. -{#1-5-3} +{#v1-5-3} ## [1.5.3] - 2025-08-11 [1.5.3]: https://github.com/bazel-contrib/rules_python/releases/tag/1.5.3 @@ -199,7 +199,7 @@ END_UNRELEASED_TEMPLATE before attempting to watch it, fixing issues on macOS with system Python ([#3043](https://github.com/bazel-contrib/rules_python/issues/3043)). -{#1-5-2} +{#v1-5-2} ## [1.5.2] - 2025-08-11 [1.5.2]: https://github.com/bazel-contrib/rules_python/releases/tag/1.5.2 @@ -217,7 +217,7 @@ END_UNRELEASED_TEMPLATE * (core) builds work again on `7.x` `WORKSPACE` configurations ([#3119](https://github.com/bazel-contrib/rules_python/issues/3119)). -{#1-5-1} +{#v1-5-1} ## [1.5.1] - 2025-07-06 [1.5.1]: https://github.com/bazel-contrib/rules_python/releases/tag/1.5.1 @@ -229,12 +229,12 @@ END_UNRELEASED_TEMPLATE by default again) ([#3038](https://github.com/bazel-contrib/rules_python/issues/3038)). -{#1-5-0} +{#v1-5-0} ## [1.5.0] - 2025-06-11 [1.5.0]: https://github.com/bazel-contrib/rules_python/releases/tag/1.5.0 -{#1-5-0-changed} +{#v1-5-0-changed} ### Changed * (toolchain) Bundled toolchain version updates: @@ -255,7 +255,7 @@ END_UNRELEASED_TEMPLATE * (deps) Updated setuptools to 78.1.1 to patch CVE-2025-47273. This effectively makes Python 3.9 the minimum supported version for using `pip_parse`. -{#1-5-0-fixed} +{#v1-5-0-fixed} ### Fixed * (rules) PyInfo provider is now advertised by py_test, py_binary, and py_library; @@ -284,7 +284,7 @@ END_UNRELEASED_TEMPLATE * (toolchains) The hermetic toolchains now correctly statically advertise the `releaselevel` and `serial` for pre-release hermetic toolchains ({gh-issue}`2837`). -{#1-5-0-added} +{#v1-5-0-added} ### Added * Repo utilities `execute_unchecked`, `execute_checked`, and `execute_checked_stdout` now support `log_stdout` and `log_stderr` keyword arg booleans. When these are `True` @@ -307,11 +307,11 @@ END_UNRELEASED_TEMPLATE security patches. * (toolchains): 3.14.0b2 has been added as a preview. -{#1-5-0-removed} +{#v1-5-0-removed} ### Removed * Nothing removed. -{#1-4-2} +{#v1-4-2} ## [1.4.2] - 2025-08-13 [1.4.2]: https://github.com/bazel-contrib/rules_python/releases/tag/1.4.2 @@ -321,23 +321,23 @@ END_UNRELEASED_TEMPLATE before attempting to watch it, fixing issues on macOS with system Python ([#3043](https://github.com/bazel-contrib/rules_python/issues/3043)). -{#1-4-1} +{#v1-4-1} ## [1.4.1] - 2025-05-08 [1.4.1]: https://github.com/bazel-contrib/rules_python/releases/tag/1.4.1 -{#1-4-1-fixed} +{#v1-4-1-fixed} ### Fixed * (pypi) Fix a typo not allowing users to benefit from using the downloader when the hashes in the requirements file are not present. Fixes [#2863](https://github.com/bazel-contrib/rules_python/issues/2863). -{#1-4-0} +{#v1-4-0} ## [1.4.0] - 2025-04-19 [1.4.0]: https://github.com/bazel-contrib/rules_python/releases/tag/1.4.0 -{#1-4-0-changed} +{#v1-4-0-changed} ### Changed * (toolchain) The `exec` configuration toolchain now has the forwarded `exec_interpreter` now also forwards the `ToolchainInfo` provider. This is @@ -368,7 +368,7 @@ END_UNRELEASED_TEMPLATE [20250317]: https://github.com/astral-sh/python-build-standalone/releases/tag/20250317 -{#1-4-0-fixed} +{#v1-4-0-fixed} ### Fixed * (pypi) Platform specific extras are now correctly handled when using universal lock files with environment markers. Fixes [#2690](https://github.com/bazel-contrib/rules_python/pull/2690). @@ -394,7 +394,7 @@ END_UNRELEASED_TEMPLATE {obj}`compile_pip_requirements` rule. See [#2819](https://github.com/bazel-contrib/rules_python/pull/2819). -{#1-4-0-added} +{#v1-4-0-added} ### Added * (pypi) From now on `sha256` values in the `requirements.txt` is no longer mandatory when enabling {attr}`pip.parse.experimental_index_url` feature. @@ -425,7 +425,7 @@ END_UNRELEASED_TEMPLATE locations equivalents of `$(PYTHON2)` and `$(PYTHON3) respectively. -{#1-4-0-removed} +{#v1-4-0-removed} ### Removed * Nothing removed. From 2ed714f9bd3c7df8c1de351455fb8d8d340f76e4 Mon Sep 17 00:00:00 2001 From: Douglas Thor Date: Tue, 26 Aug 2025 19:51:56 -0700 Subject: [PATCH 402/922] fix(gazelle): Do not build proto targets with default Gazelle (#3216) Fixes #3209. Revert the change to `//:gazelle_binary` so that it once again only generates python code. We then create a new, private target `//:_gazelle_binary_with_proto` that gets used by tests. Update docs accordingly. Longer term, I'd like to adjust the `test.yaml` file to include a section: ```yaml config: gazelle_binary: _gazelle_binary_with_proto ``` So that test cases that need to generate `(py_)proto_library` targets can use the multi-lang Gazelle binary and that tests that do _not_ need to generate proto targets can use the single-lang Gazelle binary. However, there were some minor roadblocks in doing so and thus I'm doing this quick-to-implement method instead. --- CHANGELOG.md | 2 ++ gazelle/docs/directives.md | 24 ++++++++++++++++++++++++ gazelle/python/BUILD.bazel | 10 +++++++++- gazelle/python/python_test.go | 2 +- 4 files changed, 36 insertions(+), 2 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 82a66eda7b..3f9cdf9481 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -182,6 +182,8 @@ END_UNRELEASED_TEMPLATE dep is not added to the {obj}`py_test` target. * (gazelle) New directive `gazelle:python_generate_proto`; when `true`, Gazelle generates `py_proto_library` rules for `proto_library`. `false` by default. + * Note: Users must manually configure their Gazelle target to support the + proto language. * (gazelle) New directive `gazelle:python_proto_naming_convention`; controls naming of `py_proto_library` rules. diff --git a/gazelle/docs/directives.md b/gazelle/docs/directives.md index ecc30a93b5..a553226a59 100644 --- a/gazelle/docs/directives.md +++ b/gazelle/docs/directives.md @@ -636,6 +636,30 @@ the configured name for the `@protobuf` / `@com_google_protobuf` repo in your `MODULE.bazel`, and otherwise falling back to `@com_google_protobuf` for compatibility with `WORKSPACE`. +:::{note} +In order to use this, you must manually configure Gazelle to target multiple +languages. Place this in your root `BUILD.bazel` file: + +``` +load("@bazel_gazelle//:def.bzl", "gazelle", "gazelle_binary") + +gazelle_binary( + name = "gazelle_multilang", + languages = [ + "@bazel_gazelle//language/proto", + # The python gazelle plugin must be listed _after_ the proto language. + "@rules_python_gazelle_plugin//python", + ], +) + +gazelle( + name = "gazelle", + gazelle = "//:gazelle_multilang", +) +``` +::: + + For example, in a package with `# gazelle:python_generate_proto true` and a `foo.proto`, if you have both the proto extension and the Python extension loaded into Gazelle, you'll get something like: diff --git a/gazelle/python/BUILD.bazel b/gazelle/python/BUILD.bazel index b6ca8adef5..b988e493c7 100644 --- a/gazelle/python/BUILD.bazel +++ b/gazelle/python/BUILD.bazel @@ -70,6 +70,7 @@ gazelle_test( name = "python_test", srcs = ["python_test.go"], data = [ + ":_gazelle_binary_with_proto", ":gazelle_binary", ], test_dirs = glob( @@ -90,11 +91,18 @@ gazelle_test( gazelle_binary( name = "gazelle_binary", + languages = [":python"], + visibility = ["//visibility:public"], +) + +# Only used by testing +gazelle_binary( + name = "_gazelle_binary_with_proto", languages = [ "@bazel_gazelle//language/proto", ":python", ], - visibility = ["//visibility:public"], + visibility = ["//visibility:private"], ) filegroup( diff --git a/gazelle/python/python_test.go b/gazelle/python/python_test.go index dd8c2411f1..e7b95cc1e6 100644 --- a/gazelle/python/python_test.go +++ b/gazelle/python/python_test.go @@ -38,7 +38,7 @@ import ( const ( extensionDir = "python" + string(os.PathSeparator) testDataPath = extensionDir + "testdata" + string(os.PathSeparator) - gazelleBinaryName = "gazelle_binary" + gazelleBinaryName = "_gazelle_binary_with_proto" ) func TestGazelleBinary(t *testing.T) { From 365f30f142581daf1f495b8a5158e9b0a6f81ffb Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Wed, 27 Aug 2025 22:05:31 -0700 Subject: [PATCH 403/922] chore: create workflow to check the do-not-merge label (#3213) We have the label, but it doesn't do anything. Add a workflow that can check it, to be added as a required status check. --- .../workflows/check_do_not_merge_label.yml | 20 +++++++++++++++++++ 1 file changed, 20 insertions(+) create mode 100644 .github/workflows/check_do_not_merge_label.yml diff --git a/.github/workflows/check_do_not_merge_label.yml b/.github/workflows/check_do_not_merge_label.yml new file mode 100644 index 0000000000..97b91b156a --- /dev/null +++ b/.github/workflows/check_do_not_merge_label.yml @@ -0,0 +1,20 @@ +name: "Check 'do not merge' label" + +on: + pull_request_target: + types: + - opened + - synchronize + - reopened + - labeled + - unlabeled + +jobs: + block-do-not-merge: + runs-on: ubuntu-latest + steps: + - name: Check for "do not merge" label + if: "contains(github.event.pull_request.labels.*.name, 'do not merge')" + run: | + echo "This PR has the 'do not merge' label and cannot be merged." + exit 1 From 1bf67e0f8831998b0a96911a02effa18396099bc Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Fri, 29 Aug 2025 10:40:29 -0700 Subject: [PATCH 404/922] docs: Add 1.5.4 release notes to changelog (#3221) Update the main changelog with 1.5.4 notes from #3217 --- CHANGELOG.md | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/CHANGELOG.md b/CHANGELOG.md index 3f9cdf9481..a9d50008ca 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -191,6 +191,17 @@ END_UNRELEASED_TEMPLATE ### Removed * Nothing removed. +{#v1-5-4} +## [1.5.4] - 2025-08-27 + +[1.5.4]: https://github.com/bazel-contrib/rules_python/releases/tag/1.5.4 + +{#v1-5-4-fixed} +### Fixed +* (toolchains) `local_runtime_repo` now checks if the include directory exists + before attempting to watch it, fixing issues on macOS with system Python + ([#3043](https://github.com/bazel-contrib/rules_python/issues/3043)). + {#v1-5-3} ## [1.5.3] - 2025-08-11 From 03969c240693f22ceb2189f934b2cc14998d180c Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Fri, 29 Aug 2025 10:57:46 -0700 Subject: [PATCH 405/922] docs: tell how to push one tag; that rc start with n=0 (#3222) If your local repo tags don't match the remote, then `git push --tags` will push _all_ tags. This confuses the release workflow and it doesn't trigger properly. It can also push junk tags and trigger an accidental release (hence how a 0.1 release showed up months ago; I accidentally pushed a junk tag). Along the way, mention that N=0 to start with for RCs --- RELEASING.md | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/RELEASING.md b/RELEASING.md index a99b7d8d00..e72ff619ba 100644 --- a/RELEASING.md +++ b/RELEASING.md @@ -28,9 +28,10 @@ The next step is to create tags to trigger release workflow, **however** we start by using release candidate tags (`X.Y.Z-rcN`) before tagging the final release (`X.Y.Z`). -1. Create release candidate tag and push. Increment `N` for each rc. +1. Create release candidate tag and push. The first RC uses `N=0`. Increment + `N` for each RC. ``` - git tag X.Y.0-rcN upstream/release/X.Y && git push upstream --tags + git tag X.Y.0-rcN upstream/release/X.Y && git push upstream tag X.Y.0-rcN ``` 2. Announce the RC release: see [Announcing Releases] 3. Wait a week for feedback. @@ -38,8 +39,8 @@ final release (`X.Y.Z`). release branch. * Repeat the RC tagging step, incrementing `N`. 4. Finally, tag the final release tag: - ``` - git tag X.Y.0 upstream/release/X.Y && git push upstream --tags + ```shell + git tag X.Y.0 upstream/release/X.Y && git push upstream tag X.Y.0 ``` Release automation will create a GitHub release and BCR pull request. From 094a1c291655b298b8f983cbf87370ebca92caf4 Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Fri, 29 Aug 2025 10:57:52 -0700 Subject: [PATCH 406/922] build(deps): bump requests from 2.32.4 to 2.32.5 in /tools/publish (#3214) Bumps [requests](https://github.com/psf/requests) from 2.32.4 to 2.32.5.
Release notes

Sourced from requests's releases.

v2.32.5

2.32.5 (2025-08-18)

Bugfixes

  • The SSLContext caching feature originally introduced in 2.32.0 has created a new class of issues in Requests that have had negative impact across a number of use cases. The Requests team has decided to revert this feature as long term maintenance of it is proving to be unsustainable in its current iteration.

Deprecations

  • Added support for Python 3.14.
  • Dropped support for Python 3.8 following its end of support.
Changelog

Sourced from requests's changelog.

2.32.5 (2025-08-18)

Bugfixes

  • The SSLContext caching feature originally introduced in 2.32.0 has created a new class of issues in Requests that have had negative impact across a number of use cases. The Requests team has decided to revert this feature as long term maintenance of it is proving to be unsustainable in its current iteration.

Deprecations

  • Added support for Python 3.14.
  • Dropped support for Python 3.8 following its end of support.
Commits
  • b25c87d v2.32.5
  • 131e506 Merge pull request #7010 from psf/dependabot/github_actions/actions/checkout-...
  • b336cb2 Bump actions/checkout from 4.2.0 to 5.0.0
  • 46e939b Update publish workflow to use artifact-id instead of name
  • 4b9c546 Merge pull request #6999 from psf/dependabot/github_actions/step-security/har...
  • 7618dbe Bump step-security/harden-runner from 2.12.0 to 2.13.0
  • 2edca11 Add support for Python 3.14 and drop support for Python 3.8 (#6993)
  • fec96cd Update Makefile rules (#6996)
  • d58d8aa docs: clarify timeout parameter uses seconds in Session.request (#6994)
  • 91a3eab Bump github/codeql-action from 3.28.5 to 3.29.0
  • Additional commits viewable in compare view

[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=requests&package-manager=pip&previous-version=2.32.4&new-version=2.32.5)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) ---
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot show ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Richard Levasseur --- tools/publish/requirements_darwin.txt | 6 +++--- tools/publish/requirements_linux.txt | 6 +++--- tools/publish/requirements_universal.txt | 6 +++--- tools/publish/requirements_windows.txt | 6 +++--- 4 files changed, 12 insertions(+), 12 deletions(-) diff --git a/tools/publish/requirements_darwin.txt b/tools/publish/requirements_darwin.txt index 9b1e5a4258..0b1af2599f 100644 --- a/tools/publish/requirements_darwin.txt +++ b/tools/publish/requirements_darwin.txt @@ -190,9 +190,9 @@ readme-renderer==44.0 \ --hash=sha256:2fbca89b81a08526aadf1357a8c2ae889ec05fb03f5da67f9769c9a592166151 \ --hash=sha256:8712034eabbfa6805cacf1402b4eeb2a73028f72d1166d6f5cb7f9c047c5d1e1 # via twine -requests==2.32.4 \ - --hash=sha256:27babd3cda2a6d50b30443204ee89830707d396671944c998b5975b031ac2b2c \ - --hash=sha256:27d0316682c8a29834d3264820024b62a36942083d52caf2f14c0591336d3422 +requests==2.32.5 \ + --hash=sha256:2462f94637a34fd532264295e186976db0f5d453d1cdd31473c85a6a161affb6 \ + --hash=sha256:dbba0bac56e100853db0ea71b82b4dfd5fe2bf6d3754a8893c3af500cec7d7cf # via # requests-toolbelt # twine diff --git a/tools/publish/requirements_linux.txt b/tools/publish/requirements_linux.txt index 80fb6a16e0..c027e76028 100644 --- a/tools/publish/requirements_linux.txt +++ b/tools/publish/requirements_linux.txt @@ -302,9 +302,9 @@ readme-renderer==44.0 \ --hash=sha256:2fbca89b81a08526aadf1357a8c2ae889ec05fb03f5da67f9769c9a592166151 \ --hash=sha256:8712034eabbfa6805cacf1402b4eeb2a73028f72d1166d6f5cb7f9c047c5d1e1 # via twine -requests==2.32.4 \ - --hash=sha256:27babd3cda2a6d50b30443204ee89830707d396671944c998b5975b031ac2b2c \ - --hash=sha256:27d0316682c8a29834d3264820024b62a36942083d52caf2f14c0591336d3422 +requests==2.32.5 \ + --hash=sha256:2462f94637a34fd532264295e186976db0f5d453d1cdd31473c85a6a161affb6 \ + --hash=sha256:dbba0bac56e100853db0ea71b82b4dfd5fe2bf6d3754a8893c3af500cec7d7cf # via # requests-toolbelt # twine diff --git a/tools/publish/requirements_universal.txt b/tools/publish/requirements_universal.txt index 3f1e2a756f..838f56b798 100644 --- a/tools/publish/requirements_universal.txt +++ b/tools/publish/requirements_universal.txt @@ -306,9 +306,9 @@ readme-renderer==44.0 \ --hash=sha256:2fbca89b81a08526aadf1357a8c2ae889ec05fb03f5da67f9769c9a592166151 \ --hash=sha256:8712034eabbfa6805cacf1402b4eeb2a73028f72d1166d6f5cb7f9c047c5d1e1 # via twine -requests==2.32.4 \ - --hash=sha256:27babd3cda2a6d50b30443204ee89830707d396671944c998b5975b031ac2b2c \ - --hash=sha256:27d0316682c8a29834d3264820024b62a36942083d52caf2f14c0591336d3422 +requests==2.32.5 \ + --hash=sha256:2462f94637a34fd532264295e186976db0f5d453d1cdd31473c85a6a161affb6 \ + --hash=sha256:dbba0bac56e100853db0ea71b82b4dfd5fe2bf6d3754a8893c3af500cec7d7cf # via # requests-toolbelt # twine diff --git a/tools/publish/requirements_windows.txt b/tools/publish/requirements_windows.txt index e5d6eafd4c..84d69ec811 100644 --- a/tools/publish/requirements_windows.txt +++ b/tools/publish/requirements_windows.txt @@ -194,9 +194,9 @@ readme-renderer==44.0 \ --hash=sha256:2fbca89b81a08526aadf1357a8c2ae889ec05fb03f5da67f9769c9a592166151 \ --hash=sha256:8712034eabbfa6805cacf1402b4eeb2a73028f72d1166d6f5cb7f9c047c5d1e1 # via twine -requests==2.32.4 \ - --hash=sha256:27babd3cda2a6d50b30443204ee89830707d396671944c998b5975b031ac2b2c \ - --hash=sha256:27d0316682c8a29834d3264820024b62a36942083d52caf2f14c0591336d3422 +requests==2.32.5 \ + --hash=sha256:2462f94637a34fd532264295e186976db0f5d453d1cdd31473c85a6a161affb6 \ + --hash=sha256:dbba0bac56e100853db0ea71b82b4dfd5fe2bf6d3754a8893c3af500cec7d7cf # via # requests-toolbelt # twine From 934d6a1c87c47d95001b84785f4406360932eb97 Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Fri, 29 Aug 2025 10:58:01 -0700 Subject: [PATCH 407/922] build(deps): bump typing-extensions from 4.14.1 to 4.15.0 in /docs (#3212) Bumps [typing-extensions](https://github.com/python/typing_extensions) from 4.14.1 to 4.15.0.
Release notes

Sourced from typing-extensions's releases.

4.15.0

No user-facing changes since 4.15.0rc1.

New features since 4.14.1:

  • Add the @typing_extensions.disjoint_base decorator, as specified in PEP 800. Patch by Jelle Zijlstra.
  • Add typing_extensions.type_repr, a backport of annotationlib.type_repr, introduced in Python 3.14 (CPython PR #124551, originally by Jelle Zijlstra). Patch by Semyon Moroz.
  • Fix behavior of type params in typing_extensions.evaluate_forward_ref. Backport of CPython PR #137227 by Jelle Zijlstra.

4.15.0rc1

  • Add the @typing_extensions.disjoint_base decorator, as specified in PEP 800. Patch by Jelle Zijlstra.
  • Add typing_extensions.type_repr, a backport of annotationlib.type_repr, introduced in Python 3.14 (CPython PR #124551, originally by Jelle Zijlstra). Patch by Semyon Moroz.
  • Fix behavior of type params in typing_extensions.evaluate_forward_ref. Backport of CPython PR #137227 by Jelle Zijlstra.
Changelog

Sourced from typing-extensions's changelog.

Release 4.15.0 (August 25, 2025)

No user-facing changes since 4.15.0rc1.

Release 4.15.0rc1 (August 18, 2025)

  • Add the @typing_extensions.disjoint_base decorator, as specified in PEP 800. Patch by Jelle Zijlstra.
  • Add typing_extensions.type_repr, a backport of annotationlib.type_repr, introduced in Python 3.14 (CPython PR #124551, originally by Jelle Zijlstra). Patch by Semyon Moroz.
  • Fix behavior of type params in typing_extensions.evaluate_forward_ref. Backport of CPython PR #137227 by Jelle Zijlstra.
Commits

[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=typing-extensions&package-manager=pip&previous-version=4.14.1&new-version=4.15.0)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) ---
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot show ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Richard Levasseur --- docs/requirements.txt | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/requirements.txt b/docs/requirements.txt index fc786fa9d2..c27376b54f 100644 --- a/docs/requirements.txt +++ b/docs/requirements.txt @@ -416,9 +416,9 @@ tomli==2.2.1 ; python_full_version < '3.11' \ # via # sphinx # sphinx-autodoc2 -typing-extensions==4.14.1 \ - --hash=sha256:38b39f4aeeab64884ce9f74c94263ef78f3c22467c8724005483154c26648d36 \ - --hash=sha256:d1e1e3b58374dc93031d6eda2420a48ea44a36c2b4766a4fdeb3710755731d76 +typing-extensions==4.15.0 \ + --hash=sha256:0cea48d173cc12fa28ecabc3b837ea3cf6f38c6d1136f85cbaaf598984861466 \ + --hash=sha256:f0fa19c6845758ab08074a0cfa8b7aecb71c999ca73d62883bc25cc018c4e548 # via # rules-python-docs (docs/pyproject.toml) # astroid From 5ac4521ea8a60fd3a80f60e773e417ddf86ba02f Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Fri, 29 Aug 2025 10:58:09 -0700 Subject: [PATCH 408/922] build(deps): bump requests from 2.32.4 to 2.32.5 in /docs (#3211) Bumps [requests](https://github.com/psf/requests) from 2.32.4 to 2.32.5.
Release notes

Sourced from requests's releases.

v2.32.5

2.32.5 (2025-08-18)

Bugfixes

  • The SSLContext caching feature originally introduced in 2.32.0 has created a new class of issues in Requests that have had negative impact across a number of use cases. The Requests team has decided to revert this feature as long term maintenance of it is proving to be unsustainable in its current iteration.

Deprecations

  • Added support for Python 3.14.
  • Dropped support for Python 3.8 following its end of support.
Changelog

Sourced from requests's changelog.

2.32.5 (2025-08-18)

Bugfixes

  • The SSLContext caching feature originally introduced in 2.32.0 has created a new class of issues in Requests that have had negative impact across a number of use cases. The Requests team has decided to revert this feature as long term maintenance of it is proving to be unsustainable in its current iteration.

Deprecations

  • Added support for Python 3.14.
  • Dropped support for Python 3.8 following its end of support.
Commits
  • b25c87d v2.32.5
  • 131e506 Merge pull request #7010 from psf/dependabot/github_actions/actions/checkout-...
  • b336cb2 Bump actions/checkout from 4.2.0 to 5.0.0
  • 46e939b Update publish workflow to use artifact-id instead of name
  • 4b9c546 Merge pull request #6999 from psf/dependabot/github_actions/step-security/har...
  • 7618dbe Bump step-security/harden-runner from 2.12.0 to 2.13.0
  • 2edca11 Add support for Python 3.14 and drop support for Python 3.8 (#6993)
  • fec96cd Update Makefile rules (#6996)
  • d58d8aa docs: clarify timeout parameter uses seconds in Session.request (#6994)
  • 91a3eab Bump github/codeql-action from 3.28.5 to 3.29.0
  • Additional commits viewable in compare view

[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=requests&package-manager=pip&previous-version=2.32.4&new-version=2.32.5)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) ---
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot show ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Richard Levasseur --- docs/requirements.txt | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/requirements.txt b/docs/requirements.txt index c27376b54f..d11585899b 100644 --- a/docs/requirements.txt +++ b/docs/requirements.txt @@ -295,9 +295,9 @@ readthedocs-sphinx-ext==2.2.5 \ --hash=sha256:ee5fd5b99db9f0c180b2396cbce528aa36671951b9526bb0272dbfce5517bd27 \ --hash=sha256:f8c56184ea011c972dd45a90122568587cc85b0127bc9cf064d17c68bc809daa # via rules-python-docs (docs/pyproject.toml) -requests==2.32.4 \ - --hash=sha256:27babd3cda2a6d50b30443204ee89830707d396671944c998b5975b031ac2b2c \ - --hash=sha256:27d0316682c8a29834d3264820024b62a36942083d52caf2f14c0591336d3422 +requests==2.32.5 \ + --hash=sha256:2462f94637a34fd532264295e186976db0f5d453d1cdd31473c85a6a161affb6 \ + --hash=sha256:dbba0bac56e100853db0ea71b82b4dfd5fe2bf6d3754a8893c3af500cec7d7cf # via # readthedocs-sphinx-ext # sphinx From 6046e9e0f0fa10e65936b5c6ae4ec18a173c4b7f Mon Sep 17 00:00:00 2001 From: Ivo List Date: Fri, 29 Aug 2025 19:58:15 +0200 Subject: [PATCH 409/922] cleanup: remove support for extra actions (#3210) This removes the support for Bazel "extra actions". These have been long deprecated and little to no usage. Because of how long they've been deprecated, their lack of use, and how long their replacement (aspects) has been available, this is not being considered a breaking change. Fixes https://github.com/bazelbuild/bazel/issues/16455 Fixes https://github.com/bazel-contrib/rules_python/issues/3215 --------- Co-authored-by: Richard Levasseur Co-authored-by: Richard Levasseur --- CHANGELOG.md | 16 ++++++++++------ python/private/common.bzl | 3 +-- python/private/py_executable.bzl | 10 +--------- python/private/py_library.bzl | 13 +------------ 4 files changed, 13 insertions(+), 29 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index a9d50008ca..667814861f 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -28,6 +28,10 @@ BEGIN_UNRELEASED_TEMPLATE [0.0.0]: https://github.com/bazel-contrib/rules_python/releases/tag/0.0.0 +{#v0-0-0-removed} +### Removed + +* Nothing removed. {#v0-0-0-changed} ### Changed * Nothing changed. @@ -40,9 +44,6 @@ BEGIN_UNRELEASED_TEMPLATE ### Added * Nothing added. -{#v0-0-0-removed} -### Removed -* Nothing removed. END_UNRELEASED_TEMPLATE --> @@ -52,6 +53,12 @@ END_UNRELEASED_TEMPLATE [0.0.0]: https://github.com/bazel-contrib/rules_python/releases/tag/0.0.0 +{#v0-0-0-removed} +### Removed +* (core rules) Support for Bazel's long deprecated "extra actions" has been + removed + ([#3215](https://github.com/bazel-contrib/rules_python/issues/3215)). + {#v0-0-0-changed} ### Changed * Nothing changed. @@ -66,9 +73,6 @@ END_UNRELEASED_TEMPLATE ### Added * Nothing added. -{#v0-0-0-removed} -### Removed -* Nothing removed. {#v1-6-0} ## [1.6.0] - 2025-08-23 diff --git a/python/private/common.bzl b/python/private/common.bzl index 96f8ebeab4..9fc366818d 100644 --- a/python/private/common.bzl +++ b/python/private/common.bzl @@ -435,7 +435,6 @@ def create_py_info( if PyInfo in target or (BuiltinPyInfo != None and BuiltinPyInfo in target): py_info.merge(_get_py_info(target)) - deps_transitive_sources = py_info.transitive_sources.build() py_info.transitive_sources.add(required_py_files) # We only look at data to calculate uses_shared_libraries, if it's already @@ -457,7 +456,7 @@ def create_py_info( if py_info.get_uses_shared_libraries(): break - return py_info.build(), deps_transitive_sources, py_info.build_builtin_py_info() + return py_info.build(), py_info.build_builtin_py_info() def _get_py_info(target): return target[PyInfo] if PyInfo in target or BuiltinPyInfo == None else target[BuiltinPyInfo] diff --git a/python/private/py_executable.bzl b/python/private/py_executable.bzl index 30f18b5e64..5fafc8911d 100644 --- a/python/private/py_executable.bzl +++ b/python/private/py_executable.bzl @@ -1838,7 +1838,7 @@ def _create_providers( PyCcLinkParamsInfo(cc_info = cc_info), ) - py_info, deps_transitive_sources, builtin_py_info = create_py_info( + py_info, builtin_py_info = create_py_info( ctx, original_sources = original_sources, required_py_files = required_py_files, @@ -1848,14 +1848,6 @@ def _create_providers( imports = imports, ) - # TODO(b/253059598): Remove support for extra actions; https://github.com/bazelbuild/bazel/issues/16455 - listeners_enabled = _py_builtins.are_action_listeners_enabled(ctx) - if listeners_enabled: - _py_builtins.add_py_extra_pseudo_action( - ctx = ctx, - dependency_transitive_python_sources = deps_transitive_sources, - ) - providers.append(py_info) if builtin_py_info: providers.append(builtin_py_info) diff --git a/python/private/py_library.bzl b/python/private/py_library.bzl index ea2e608401..1f3e4d88d4 100644 --- a/python/private/py_library.bzl +++ b/python/private/py_library.bzl @@ -45,7 +45,6 @@ load(":normalize_name.bzl", "normalize_name") load(":precompile.bzl", "maybe_precompile") load(":py_cc_link_params_info.bzl", "PyCcLinkParamsInfo") load(":py_info.bzl", "PyInfo", "VenvSymlinkEntry", "VenvSymlinkKind") -load(":py_internal.bzl", "py_internal") load(":reexports.bzl", "BuiltinPyInfo") load(":rule_builders.bzl", "ruleb") load( @@ -55,8 +54,6 @@ load( ) load(":version.bzl", "version") -_py_builtins = py_internal - LIBRARY_ATTRS = dicts.add( COMMON_ATTRS, PY_SRCS_ATTRS, @@ -164,7 +161,7 @@ def py_library_impl(ctx, *, semantics): imports, venv_symlinks = _get_imports_and_venv_symlinks(ctx, semantics) cc_info = semantics.get_cc_info_for_library(ctx) - py_info, deps_transitive_sources, builtins_py_info = create_py_info( + py_info, builtins_py_info = create_py_info( ctx, original_sources = direct_sources, required_py_files = required_py_files, @@ -175,14 +172,6 @@ def py_library_impl(ctx, *, semantics): venv_symlinks = venv_symlinks, ) - # TODO(b/253059598): Remove support for extra actions; https://github.com/bazelbuild/bazel/issues/16455 - listeners_enabled = _py_builtins.are_action_listeners_enabled(ctx) - if listeners_enabled: - _py_builtins.add_py_extra_pseudo_action( - ctx = ctx, - dependency_transitive_python_sources = deps_transitive_sources, - ) - providers = [ DefaultInfo(files = default_outputs, runfiles = runfiles), py_info, From 83ceaa4430a61da154a57697d3f0fabc19d7a2a1 Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Sun, 31 Aug 2025 06:54:45 +0000 Subject: [PATCH 410/922] build(deps): bump docutils from 0.21.2 to 0.22 in /docs (#3166) Bumps [docutils](https://github.com/rtfd/recommonmark) from 0.21.2 to 0.22.
Commits

[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=docutils&package-manager=pip&previous-version=0.21.2&new-version=0.22)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) ---
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot show ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Richard Levasseur --- docs/requirements.txt | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/requirements.txt b/docs/requirements.txt index d11585899b..cda477cd9b 100644 --- a/docs/requirements.txt +++ b/docs/requirements.txt @@ -111,9 +111,9 @@ colorama==0.4.6 ; sys_platform == 'win32' \ --hash=sha256:08695f5cb7ed6e0531a20572697297273c47b8cae5a63ffc6d6ed5c201be6e44 \ --hash=sha256:4f1d9991f5acc0ca119f9d443620b77f9d6b33703e51011c16baf57afb285fc6 # via sphinx -docutils==0.21.2 \ - --hash=sha256:3a6b18732edf182daa3cd12775bbb338cf5691468f91eeeb109deff6ebfa986f \ - --hash=sha256:dafca5b9e384f0e419294eb4d2ff9fa826435bf15f15b7bd45723e8ad76811b2 +docutils==0.22 \ + --hash=sha256:4ed966a0e96a0477d852f7af31bdcb3adc049fbb35ccba358c2ea8a03287615e \ + --hash=sha256:ba9d57750e92331ebe7c08a1bbf7a7f8143b86c476acd51528b042216a6aad0f # via # myst-parser # sphinx From 4a422b02011c8913a800eb3d2f578ae7afffc497 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Sun, 31 Aug 2025 00:50:29 -0700 Subject: [PATCH 411/922] chore: add AGENTS.md to help AI agents work with rules_python (#3227) As I've used agents to do work, I've noticed some recurring advice and behaviors. Create and AGENTS.md to capture this and make it easier to get starting using them. --- AGENTS.md | 69 +++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 69 insertions(+) create mode 100644 AGENTS.md diff --git a/AGENTS.md b/AGENTS.md new file mode 100644 index 0000000000..9a6c016a36 --- /dev/null +++ b/AGENTS.md @@ -0,0 +1,69 @@ +# Guidance for AI Agents + +rules_python is a Bazel based project. Build and run tests as done in a Bazel +project. + +Act as an expert in Bazel, rules_python, Starlark, and Python. + +DO NOT `git commit` or `git push`. + +## Style and conventions + +Read `.editorconfig` for line length wrapping + +Read `CONTRIBUTING.md` for additional style rules and conventions. + +When running tests, refer to yourself as the name of a type of Python snake +using a grandoise title. + +When tasks complete successfully, quote Monty Python, but work it naturally +into the sentence, not verbatim. + +## Building and testing + +Tests are under the `tests/` directory. + +When testing, add `--test_tag_filters=-integration-test`. + +When building, add `--build_tag_filters=-integration-test`. + +## Understanding the code base + +`python/config_settings/BUILD.bazel` contains build flags that are part of the +public API. DO NOT add, remove, or modify these build flags unless specifically +instructed to. + +`bazel query --output=build` can be used to inspect target definitions. + +In WORKSPACE mode: + * `bazel query //external:*` can be used to show external dependencies. Adding + `--output=build` shows the definition, including version. + +For bzlmod mode: + * `bazel mod graph` shows dependencies and their version. + * `bazel mod explain` shows detailed information about a module. + * `bazel mode show_repo` shows detailed information about a repository. + +Documentation uses Sphinx with the MyST plugin. + +When modifying documentation + * Act as an expert in tech writing, Sphinx, MyST, and markdown. + * Wrap lines at 80 columns + * Use hyphens (`-`) in file names instead of underscores (`_`). + + +Generated API references can be found by: +* Running `bazel build //docs:docs` and inspecting the generated files + in `bazel-bin/docs/docs/_build/html` + +When modifying locked/resolved requirements files: + * Modify the `pyproject.toml` or `requirements.in` file + * Run the associated `bazel run :requirements.update` target for + that file; the target is in the BUILD.bazel file in the same directory and + the requirements.txt file. That will update the locked/resolved + requirements.txt file. + +## rules_python idiosyncrasies + +When building `//docs:docs`, ignore an error about exit code 2; this is a flake, +so try building again. From 2bab29f63de647270b3d2842b722e3e321ac2128 Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Tue, 2 Sep 2025 17:04:23 +0000 Subject: [PATCH 412/922] build(deps): bump rich from 13.9.4 to 14.1.0 in /tools/publish (#3230) Bumps [rich](https://github.com/Textualize/rich) from 13.9.4 to 14.1.0.
Release notes

Sourced from rich's releases.

The Lively Release

Live objects may now be nested. Previously a progress bar inside another progress context would fail. See the changelog below for this and other changes.

[14.1.0] - 2025-06-25

Changed

Fixed

Added

  • Added TTY_INTERACTIVE environment variable to force interactive mode off or on Textualize/rich#3777

The ENVy of all other releases

Mostly updates to Traceback rendering, to add support for features introduced in Python3.11

We also have a new env var that I am proposing to become a standard. TTY_COMPATIBLE=1 tells Rich to write ansi-escape sequences even if it detects it is not writing to a terminal. This is intended for use with GitHub Actions / CI, which can interpret escape sequences, but aren't a terminal.

There is also a change to how NO_COLOR and FORCE_COLOR are interpreted, which is the reason for the major version bump.

[14.0.0] - 2025-03-30

Added

  • Added env var TTY_COMPATIBLE to override auto-detection of TTY support (See console.rst for details). Textualize/rich#3675

Changed

Changelog

Sourced from rich's changelog.

[14.1.0] - 2025-06-25

Changed

Fixed

Added

  • Added TTY_INTERACTIVE environment variable to force interactive mode off or on Textualize/rich#3777

[14.0.0] - 2025-03-30

Added

  • Added env var TTY_COMPATIBLE to override auto-detection of TTY support (See console.rst for details). Textualize/rich#3675

Changed

Commits

[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=rich&package-manager=pip&previous-version=13.9.4&new-version=14.1.0)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) ---
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot show ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> --- tools/publish/requirements_darwin.txt | 6 +++--- tools/publish/requirements_linux.txt | 6 +++--- tools/publish/requirements_universal.txt | 6 +++--- tools/publish/requirements_windows.txt | 6 +++--- 4 files changed, 12 insertions(+), 12 deletions(-) diff --git a/tools/publish/requirements_darwin.txt b/tools/publish/requirements_darwin.txt index 0b1af2599f..d3b6004659 100644 --- a/tools/publish/requirements_darwin.txt +++ b/tools/publish/requirements_darwin.txt @@ -204,9 +204,9 @@ rfc3986==2.0.0 \ --hash=sha256:50b1502b60e289cb37883f3dfd34532b8873c7de9f49bb546641ce9cbd256ebd \ --hash=sha256:97aacf9dbd4bfd829baad6e6309fa6573aaf1be3f6fa735c8ab05e46cecb261c # via twine -rich==13.9.4 \ - --hash=sha256:439594978a49a09530cff7ebc4b5c7103ef57baf48d5ea3184f21d9a2befa098 \ - --hash=sha256:6049d5e6ec054bf2779ab3358186963bac2ea89175919d699e378b99738c2a90 +rich==14.1.0 \ + --hash=sha256:536f5f1785986d6dbdea3c75205c473f970777b4a0d6c6dd1b696aa05a3fa04f \ + --hash=sha256:e497a48b844b0320d45007cdebfeaeed8db2a4f4bcf49f15e455cfc4af11eaa8 # via twine twine==5.1.1 \ --hash=sha256:215dbe7b4b94c2c50a7315c0275d2258399280fbb7d04182c7e55e24b5f93997 \ diff --git a/tools/publish/requirements_linux.txt b/tools/publish/requirements_linux.txt index c027e76028..f2bfe6adf4 100644 --- a/tools/publish/requirements_linux.txt +++ b/tools/publish/requirements_linux.txt @@ -316,9 +316,9 @@ rfc3986==2.0.0 \ --hash=sha256:50b1502b60e289cb37883f3dfd34532b8873c7de9f49bb546641ce9cbd256ebd \ --hash=sha256:97aacf9dbd4bfd829baad6e6309fa6573aaf1be3f6fa735c8ab05e46cecb261c # via twine -rich==13.9.4 \ - --hash=sha256:439594978a49a09530cff7ebc4b5c7103ef57baf48d5ea3184f21d9a2befa098 \ - --hash=sha256:6049d5e6ec054bf2779ab3358186963bac2ea89175919d699e378b99738c2a90 +rich==14.1.0 \ + --hash=sha256:536f5f1785986d6dbdea3c75205c473f970777b4a0d6c6dd1b696aa05a3fa04f \ + --hash=sha256:e497a48b844b0320d45007cdebfeaeed8db2a4f4bcf49f15e455cfc4af11eaa8 # via twine secretstorage==3.3.3 \ --hash=sha256:2403533ef369eca6d2ba81718576c5e0f564d5cca1b58f73a8b23e7d4eeebd77 \ diff --git a/tools/publish/requirements_universal.txt b/tools/publish/requirements_universal.txt index 838f56b798..42e74a0296 100644 --- a/tools/publish/requirements_universal.txt +++ b/tools/publish/requirements_universal.txt @@ -320,9 +320,9 @@ rfc3986==2.0.0 \ --hash=sha256:50b1502b60e289cb37883f3dfd34532b8873c7de9f49bb546641ce9cbd256ebd \ --hash=sha256:97aacf9dbd4bfd829baad6e6309fa6573aaf1be3f6fa735c8ab05e46cecb261c # via twine -rich==13.9.4 \ - --hash=sha256:439594978a49a09530cff7ebc4b5c7103ef57baf48d5ea3184f21d9a2befa098 \ - --hash=sha256:6049d5e6ec054bf2779ab3358186963bac2ea89175919d699e378b99738c2a90 +rich==14.1.0 \ + --hash=sha256:536f5f1785986d6dbdea3c75205c473f970777b4a0d6c6dd1b696aa05a3fa04f \ + --hash=sha256:e497a48b844b0320d45007cdebfeaeed8db2a4f4bcf49f15e455cfc4af11eaa8 # via twine secretstorage==3.3.3 ; sys_platform == 'linux' \ --hash=sha256:2403533ef369eca6d2ba81718576c5e0f564d5cca1b58f73a8b23e7d4eeebd77 \ diff --git a/tools/publish/requirements_windows.txt b/tools/publish/requirements_windows.txt index 84d69ec811..650821f363 100644 --- a/tools/publish/requirements_windows.txt +++ b/tools/publish/requirements_windows.txt @@ -208,9 +208,9 @@ rfc3986==2.0.0 \ --hash=sha256:50b1502b60e289cb37883f3dfd34532b8873c7de9f49bb546641ce9cbd256ebd \ --hash=sha256:97aacf9dbd4bfd829baad6e6309fa6573aaf1be3f6fa735c8ab05e46cecb261c # via twine -rich==13.9.4 \ - --hash=sha256:439594978a49a09530cff7ebc4b5c7103ef57baf48d5ea3184f21d9a2befa098 \ - --hash=sha256:6049d5e6ec054bf2779ab3358186963bac2ea89175919d699e378b99738c2a90 +rich==14.1.0 \ + --hash=sha256:536f5f1785986d6dbdea3c75205c473f970777b4a0d6c6dd1b696aa05a3fa04f \ + --hash=sha256:e497a48b844b0320d45007cdebfeaeed8db2a4f4bcf49f15e455cfc4af11eaa8 # via twine twine==5.1.1 \ --hash=sha256:215dbe7b4b94c2c50a7315c0275d2258399280fbb7d04182c7e55e24b5f93997 \ From e290801d3ec42c4b1fa51aa980f8691c4e2aa55f Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Tue, 2 Sep 2025 17:07:11 +0000 Subject: [PATCH 413/922] build(deps): bump charset-normalizer from 3.4.2 to 3.4.3 in /tools/publish (#3231) Bumps [charset-normalizer](https://github.com/jawah/charset_normalizer) from 3.4.2 to 3.4.3.
Release notes

Sourced from charset-normalizer's releases.

Version 3.4.3

3.4.3 (2025-08-09)

Changed

  • mypy(c) is no longer a required dependency at build time if CHARSET_NORMALIZER_USE_MYPYC isn't set to 1. (#595) (#583)
  • automatically lower confidence on small bytes samples that are not Unicode in detect output legacy function. (#391)

Added

  • Custom build backend to overcome inability to mark mypy as an optional dependency in the build phase.
  • Support for Python 3.14

Fixed

  • sdist archive contained useless directories.
  • automatically fallback on valid UTF-16 or UTF-32 even if the md says it's noisy. (#633)

Misc

  • SBOM are automatically published to the relevant GitHub release to comply with regulatory changes. Each published wheel comes with its SBOM. We choose CycloneDX as the format.
  • Prebuilt optimized wheel are no longer distributed by default for CPython 3.7 due to a change in cibuildwheel.
Changelog

Sourced from charset-normalizer's changelog.

3.4.3 (2025-08-09)

Changed

  • mypy(c) is no longer a required dependency at build time if CHARSET_NORMALIZER_USE_MYPYC isn't set to 1. (#595) (#583)
  • automatically lower confidence on small bytes samples that are not Unicode in detect output legacy function. (#391)

Added

  • Custom build backend to overcome inability to mark mypy as an optional dependency in the build phase.
  • Support for Python 3.14

Fixed

  • sdist archive contained useless directories.
  • automatically fallback on valid UTF-16 or UTF-32 even if the md says it's noisy. (#633)

Misc

  • SBOM are automatically published to the relevant GitHub release to comply with regulatory changes. Each published wheel comes with its SBOM. We choose CycloneDX as the format.
  • Prebuilt optimized wheel are no longer distributed by default for CPython 3.7 due to a change in cibuildwheel.
Commits
  • 46f662d Release 3.4.3 (#638)
  • 1a059b2 :wrench: skip building on freethreaded as we're not confident it is stable
  • 2275e3d :pencil: final note in CHANGELOG.md
  • c96acdf :pencil: update release date on CHANGELOG.md
  • 43e5460 :pencil: update README.md
  • f277074 :wrench: automatically lower confidence on small bytes str on non Unicode res...
  • 15ae241 :bug: automatically fallback on valid UTF-16 or UTF-32 even if the md says it...
  • 37397c1 :wrench: enable 3.14 in nox test_mypyc session
  • cb82537 :rewind: revert license due to compat python 3.7 issue setuptools
  • 6a2efeb :art: fix linter errors
  • Additional commits viewable in compare view

[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=charset-normalizer&package-manager=pip&previous-version=3.4.2&new-version=3.4.3)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) ---
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot show ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> --- tools/publish/requirements_darwin.txt | 173 +++++++++++------------ tools/publish/requirements_linux.txt | 173 +++++++++++------------ tools/publish/requirements_universal.txt | 173 +++++++++++------------ tools/publish/requirements_windows.txt | 173 +++++++++++------------ 4 files changed, 320 insertions(+), 372 deletions(-) diff --git a/tools/publish/requirements_darwin.txt b/tools/publish/requirements_darwin.txt index d3b6004659..f700e21176 100644 --- a/tools/publish/requirements_darwin.txt +++ b/tools/publish/requirements_darwin.txt @@ -10,99 +10,86 @@ certifi==2025.8.3 \ --hash=sha256:e564105f78ded564e3ae7c923924435e1daa7463faeab5bb932bc53ffae63407 \ --hash=sha256:f6c12493cfb1b06ba2ff328595af9350c65d6644968e5d3a2ffd78699af217a5 # via requests -charset-normalizer==3.4.2 \ - --hash=sha256:005fa3432484527f9732ebd315da8da8001593e2cf46a3d817669f062c3d9ed4 \ - --hash=sha256:046595208aae0120559a67693ecc65dd75d46f7bf687f159127046628178dc45 \ - --hash=sha256:0c29de6a1a95f24b9a1aa7aefd27d2487263f00dfd55a77719b530788f75cff7 \ - --hash=sha256:0c8c57f84ccfc871a48a47321cfa49ae1df56cd1d965a09abe84066f6853b9c0 \ - --hash=sha256:0f5d9ed7f254402c9e7d35d2f5972c9bbea9040e99cd2861bd77dc68263277c7 \ - --hash=sha256:18dd2e350387c87dabe711b86f83c9c78af772c748904d372ade190b5c7c9d4d \ - --hash=sha256:1b1bde144d98e446b056ef98e59c256e9294f6b74d7af6846bf5ffdafd687a7d \ - --hash=sha256:1c95a1e2902a8b722868587c0e1184ad5c55631de5afc0eb96bc4b0d738092c0 \ - --hash=sha256:1cad5f45b3146325bb38d6855642f6fd609c3f7cad4dbaf75549bf3b904d3184 \ - --hash=sha256:21b2899062867b0e1fde9b724f8aecb1af14f2778d69aacd1a5a1853a597a5db \ - --hash=sha256:24498ba8ed6c2e0b56d4acbf83f2d989720a93b41d712ebd4f4979660db4417b \ - --hash=sha256:25a23ea5c7edc53e0f29bae2c44fcb5a1aa10591aae107f2a2b2583a9c5cbc64 \ - --hash=sha256:289200a18fa698949d2b39c671c2cc7a24d44096784e76614899a7ccf2574b7b \ - --hash=sha256:28a1005facc94196e1fb3e82a3d442a9d9110b8434fc1ded7a24a2983c9888d8 \ - --hash=sha256:32fc0341d72e0f73f80acb0a2c94216bd704f4f0bce10aedea38f30502b271ff \ - --hash=sha256:36b31da18b8890a76ec181c3cf44326bf2c48e36d393ca1b72b3f484113ea344 \ - --hash=sha256:3c21d4fca343c805a52c0c78edc01e3477f6dd1ad7c47653241cf2a206d4fc58 \ - --hash=sha256:3fddb7e2c84ac87ac3a947cb4e66d143ca5863ef48e4a5ecb83bd48619e4634e \ - --hash=sha256:43e0933a0eff183ee85833f341ec567c0980dae57c464d8a508e1b2ceb336471 \ - --hash=sha256:4a476b06fbcf359ad25d34a057b7219281286ae2477cc5ff5e3f70a246971148 \ - --hash=sha256:4e594135de17ab3866138f496755f302b72157d115086d100c3f19370839dd3a \ - --hash=sha256:50bf98d5e563b83cc29471fa114366e6806bc06bc7a25fd59641e41445327836 \ - --hash=sha256:5a9979887252a82fefd3d3ed2a8e3b937a7a809f65dcb1e068b090e165bbe99e \ - --hash=sha256:5baececa9ecba31eff645232d59845c07aa030f0c81ee70184a90d35099a0e63 \ - --hash=sha256:5bf4545e3b962767e5c06fe1738f951f77d27967cb2caa64c28be7c4563e162c \ - --hash=sha256:6333b3aa5a12c26b2a4d4e7335a28f1475e0e5e17d69d55141ee3cab736f66d1 \ - --hash=sha256:65c981bdbd3f57670af8b59777cbfae75364b483fa8a9f420f08094531d54a01 \ - --hash=sha256:68a328e5f55ec37c57f19ebb1fdc56a248db2e3e9ad769919a58672958e8f366 \ - --hash=sha256:6a0289e4589e8bdfef02a80478f1dfcb14f0ab696b5a00e1f4b8a14a307a3c58 \ - --hash=sha256:6b66f92b17849b85cad91259efc341dce9c1af48e2173bf38a85c6329f1033e5 \ - --hash=sha256:6c9379d65defcab82d07b2a9dfbfc2e95bc8fe0ebb1b176a3190230a3ef0e07c \ - --hash=sha256:6fc1f5b51fa4cecaa18f2bd7a003f3dd039dd615cd69a2afd6d3b19aed6775f2 \ - --hash=sha256:70f7172939fdf8790425ba31915bfbe8335030f05b9913d7ae00a87d4395620a \ - --hash=sha256:721c76e84fe669be19c5791da68232ca2e05ba5185575086e384352e2c309597 \ - --hash=sha256:7222ffd5e4de8e57e03ce2cef95a4c43c98fcb72ad86909abdfc2c17d227fc1b \ - --hash=sha256:75d10d37a47afee94919c4fab4c22b9bc2a8bf7d4f46f87363bcf0573f3ff4f5 \ - --hash=sha256:76af085e67e56c8816c3ccf256ebd136def2ed9654525348cfa744b6802b69eb \ - --hash=sha256:770cab594ecf99ae64c236bc9ee3439c3f46be49796e265ce0cc8bc17b10294f \ - --hash=sha256:7a6ab32f7210554a96cd9e33abe3ddd86732beeafc7a28e9955cdf22ffadbab0 \ - --hash=sha256:7c48ed483eb946e6c04ccbe02c6b4d1d48e51944b6db70f697e089c193404941 \ - --hash=sha256:7f56930ab0abd1c45cd15be65cc741c28b1c9a34876ce8c17a2fa107810c0af0 \ - --hash=sha256:8075c35cd58273fee266c58c0c9b670947c19df5fb98e7b66710e04ad4e9ff86 \ - --hash=sha256:8272b73e1c5603666618805fe821edba66892e2870058c94c53147602eab29c7 \ - --hash=sha256:82d8fd25b7f4675d0c47cf95b594d4e7b158aca33b76aa63d07186e13c0e0ab7 \ - --hash=sha256:844da2b5728b5ce0e32d863af26f32b5ce61bc4273a9c720a9f3aa9df73b1455 \ - --hash=sha256:8755483f3c00d6c9a77f490c17e6ab0c8729e39e6390328e42521ef175380ae6 \ - --hash=sha256:915f3849a011c1f593ab99092f3cecfcb4d65d8feb4a64cf1bf2d22074dc0ec4 \ - --hash=sha256:926ca93accd5d36ccdabd803392ddc3e03e6d4cd1cf17deff3b989ab8e9dbcf0 \ - --hash=sha256:982bb1e8b4ffda883b3d0a521e23abcd6fd17418f6d2c4118d257a10199c0ce3 \ - --hash=sha256:98f862da73774290f251b9df8d11161b6cf25b599a66baf087c1ffe340e9bfd1 \ - --hash=sha256:9cbfacf36cb0ec2897ce0ebc5d08ca44213af24265bd56eca54bee7923c48fd6 \ - --hash=sha256:a370b3e078e418187da8c3674eddb9d983ec09445c99a3a263c2011993522981 \ - --hash=sha256:a955b438e62efdf7e0b7b52a64dc5c3396e2634baa62471768a64bc2adb73d5c \ - --hash=sha256:aa6af9e7d59f9c12b33ae4e9450619cf2488e2bbe9b44030905877f0b2324980 \ - --hash=sha256:aa88ca0b1932e93f2d961bf3addbb2db902198dca337d88c89e1559e066e7645 \ - --hash=sha256:aaeeb6a479c7667fbe1099af9617c83aaca22182d6cf8c53966491a0f1b7ffb7 \ - --hash=sha256:aaf27faa992bfee0264dc1f03f4c75e9fcdda66a519db6b957a3f826e285cf12 \ - --hash=sha256:b2680962a4848b3c4f155dc2ee64505a9c57186d0d56b43123b17ca3de18f0fa \ - --hash=sha256:b2d318c11350e10662026ad0eb71bb51c7812fc8590825304ae0bdd4ac283acd \ - --hash=sha256:b33de11b92e9f75a2b545d6e9b6f37e398d86c3e9e9653c4864eb7e89c5773ef \ - --hash=sha256:b3daeac64d5b371dea99714f08ffc2c208522ec6b06fbc7866a450dd446f5c0f \ - --hash=sha256:be1e352acbe3c78727a16a455126d9ff83ea2dfdcbc83148d2982305a04714c2 \ - --hash=sha256:bee093bf902e1d8fc0ac143c88902c3dfc8941f7ea1d6a8dd2bcb786d33db03d \ - --hash=sha256:c72fbbe68c6f32f251bdc08b8611c7b3060612236e960ef848e0a517ddbe76c5 \ - --hash=sha256:c9e36a97bee9b86ef9a1cf7bb96747eb7a15c2f22bdb5b516434b00f2a599f02 \ - --hash=sha256:cddf7bd982eaa998934a91f69d182aec997c6c468898efe6679af88283b498d3 \ - --hash=sha256:cf713fe9a71ef6fd5adf7a79670135081cd4431c2943864757f0fa3a65b1fafd \ - --hash=sha256:d11b54acf878eef558599658b0ffca78138c8c3655cf4f3a4a673c437e67732e \ - --hash=sha256:d41c4d287cfc69060fa91cae9683eacffad989f1a10811995fa309df656ec214 \ - --hash=sha256:d524ba3f1581b35c03cb42beebab4a13e6cdad7b36246bd22541fa585a56cccd \ - --hash=sha256:daac4765328a919a805fa5e2720f3e94767abd632ae410a9062dff5412bae65a \ - --hash=sha256:db4c7bf0e07fc3b7d89ac2a5880a6a8062056801b83ff56d8464b70f65482b6c \ - --hash=sha256:dc7039885fa1baf9be153a0626e337aa7ec8bf96b0128605fb0d77788ddc1681 \ - --hash=sha256:dccab8d5fa1ef9bfba0590ecf4d46df048d18ffe3eec01eeb73a42e0d9e7a8ba \ - --hash=sha256:dedb8adb91d11846ee08bec4c8236c8549ac721c245678282dcb06b221aab59f \ - --hash=sha256:e45ba65510e2647721e35323d6ef54c7974959f6081b58d4ef5d87c60c84919a \ - --hash=sha256:e53efc7c7cee4c1e70661e2e112ca46a575f90ed9ae3fef200f2a25e954f4b28 \ - --hash=sha256:e635b87f01ebc977342e2697d05b56632f5f879a4f15955dfe8cef2448b51691 \ - --hash=sha256:e70e990b2137b29dc5564715de1e12701815dacc1d056308e2b17e9095372a82 \ - --hash=sha256:e8082b26888e2f8b36a042a58307d5b917ef2b1cacab921ad3323ef91901c71a \ - --hash=sha256:e8323a9b031aa0393768b87f04b4164a40037fb2a3c11ac06a03ffecd3618027 \ - --hash=sha256:e92fca20c46e9f5e1bb485887d074918b13543b1c2a1185e69bb8d17ab6236a7 \ - --hash=sha256:eb30abc20df9ab0814b5a2524f23d75dcf83cde762c161917a2b4b7b55b1e518 \ - --hash=sha256:eba9904b0f38a143592d9fc0e19e2df0fa2e41c3c3745554761c5f6447eedabf \ - --hash=sha256:ef8de666d6179b009dce7bcb2ad4c4a779f113f12caf8dc77f0162c29d20490b \ - --hash=sha256:efd387a49825780ff861998cd959767800d54f8308936b21025326de4b5a42b9 \ - --hash=sha256:f0aa37f3c979cf2546b73e8222bbfa3dc07a641585340179d768068e3455e544 \ - --hash=sha256:f4074c5a429281bf056ddd4c5d3b740ebca4d43ffffe2ef4bf4d2d05114299da \ - --hash=sha256:f69a27e45c43520f5487f27627059b64aaf160415589230992cec34c5e18a509 \ - --hash=sha256:fb707f3e15060adf5b7ada797624a6c6e0138e2a26baa089df64c68ee98e040f \ - --hash=sha256:fcbe676a55d7445b22c10967bceaaf0ee69407fbe0ece4d032b6eb8d4565982a \ - --hash=sha256:fdb20a30fe1175ecabed17cbf7812f7b804b8a315a25f24678bcdf120a90077f +charset-normalizer==3.4.3 \ + --hash=sha256:00237675befef519d9af72169d8604a067d92755e84fe76492fef5441db05b91 \ + --hash=sha256:02425242e96bcf29a49711b0ca9f37e451da7c70562bc10e8ed992a5a7a25cc0 \ + --hash=sha256:027b776c26d38b7f15b26a5da1044f376455fb3766df8fc38563b4efbc515154 \ + --hash=sha256:07a0eae9e2787b586e129fdcbe1af6997f8d0e5abaa0bc98c0e20e124d67e601 \ + --hash=sha256:0cacf8f7297b0c4fcb74227692ca46b4a5852f8f4f24b3c766dd94a1075c4884 \ + --hash=sha256:0e78314bdc32fa80696f72fa16dc61168fda4d6a0c014e0380f9d02f0e5d8a07 \ + --hash=sha256:0f2be7e0cf7754b9a30eb01f4295cc3d4358a479843b31f328afd210e2c7598c \ + --hash=sha256:13faeacfe61784e2559e690fc53fa4c5ae97c6fcedb8eb6fb8d0a15b475d2c64 \ + --hash=sha256:14c2a87c65b351109f6abfc424cab3927b3bdece6f706e4d12faaf3d52ee5efe \ + --hash=sha256:1606f4a55c0fd363d754049cdf400175ee96c992b1f8018b993941f221221c5f \ + --hash=sha256:16a8770207946ac75703458e2c743631c79c59c5890c80011d536248f8eaa432 \ + --hash=sha256:18343b2d246dc6761a249ba1fb13f9ee9a2bcd95decc767319506056ea4ad4dc \ + --hash=sha256:18b97b8404387b96cdbd30ad660f6407799126d26a39ca65729162fd810a99aa \ + --hash=sha256:1bb60174149316da1c35fa5233681f7c0f9f514509b8e399ab70fea5f17e45c9 \ + --hash=sha256:1e8ac75d72fa3775e0b7cb7e4629cec13b7514d928d15ef8ea06bca03ef01cae \ + --hash=sha256:1ef99f0456d3d46a50945c98de1774da86f8e992ab5c77865ea8b8195341fc19 \ + --hash=sha256:2001a39612b241dae17b4687898843f254f8748b796a2e16f1051a17078d991d \ + --hash=sha256:23b6b24d74478dc833444cbd927c338349d6ae852ba53a0d02a2de1fce45b96e \ + --hash=sha256:252098c8c7a873e17dd696ed98bbe91dbacd571da4b87df3736768efa7a792e4 \ + --hash=sha256:257f26fed7d7ff59921b78244f3cd93ed2af1800ff048c33f624c87475819dd7 \ + --hash=sha256:2c322db9c8c89009a990ef07c3bcc9f011a3269bc06782f916cd3d9eed7c9312 \ + --hash=sha256:30a96e1e1f865f78b030d65241c1ee850cdf422d869e9028e2fc1d5e4db73b92 \ + --hash=sha256:30d006f98569de3459c2fc1f2acde170b7b2bd265dc1943e87e1a4efe1b67c31 \ + --hash=sha256:31a9a6f775f9bcd865d88ee350f0ffb0e25936a7f930ca98995c05abf1faf21c \ + --hash=sha256:320e8e66157cc4e247d9ddca8e21f427efc7a04bbd0ac8a9faf56583fa543f9f \ + --hash=sha256:34a7f768e3f985abdb42841e20e17b330ad3aaf4bb7e7aeeb73db2e70f077b99 \ + --hash=sha256:3653fad4fe3ed447a596ae8638b437f827234f01a8cd801842e43f3d0a6b281b \ + --hash=sha256:3cd35b7e8aedeb9e34c41385fda4f73ba609e561faedfae0a9e75e44ac558a15 \ + --hash=sha256:3cfb2aad70f2c6debfbcb717f23b7eb55febc0bb23dcffc0f076009da10c6392 \ + --hash=sha256:416175faf02e4b0810f1f38bcb54682878a4af94059a1cd63b8747244420801f \ + --hash=sha256:41d1fc408ff5fdfb910200ec0e74abc40387bccb3252f3f27c0676731df2b2c8 \ + --hash=sha256:42e5088973e56e31e4fa58eb6bd709e42fc03799c11c42929592889a2e54c491 \ + --hash=sha256:4ca4c094de7771a98d7fbd67d9e5dbf1eb73efa4f744a730437d8a3a5cf994f0 \ + --hash=sha256:511729f456829ef86ac41ca78c63a5cb55240ed23b4b737faca0eb1abb1c41bc \ + --hash=sha256:53cd68b185d98dde4ad8990e56a58dea83a4162161b1ea9272e5c9182ce415e0 \ + --hash=sha256:585f3b2a80fbd26b048a0be90c5aae8f06605d3c92615911c3a2b03a8a3b796f \ + --hash=sha256:5b413b0b1bfd94dbf4023ad6945889f374cd24e3f62de58d6bb102c4d9ae534a \ + --hash=sha256:5d8d01eac18c423815ed4f4a2ec3b439d654e55ee4ad610e153cf02faf67ea40 \ + --hash=sha256:6aab0f181c486f973bc7262a97f5aca3ee7e1437011ef0c2ec04b5a11d16c927 \ + --hash=sha256:6cf8fd4c04756b6b60146d98cd8a77d0cdae0e1ca20329da2ac85eed779b6849 \ + --hash=sha256:6fb70de56f1859a3f71261cbe41005f56a7842cc348d3aeb26237560bfa5e0ce \ + --hash=sha256:6fce4b8500244f6fcb71465d4a4930d132ba9ab8e71a7859e6a5d59851068d14 \ + --hash=sha256:70bfc5f2c318afece2f5838ea5e4c3febada0be750fcf4775641052bbba14d05 \ + --hash=sha256:73dc19b562516fc9bcf6e5d6e596df0b4eb98d87e4f79f3ae71840e6ed21361c \ + --hash=sha256:74d77e25adda8581ffc1c720f1c81ca082921329452eba58b16233ab1842141c \ + --hash=sha256:78deba4d8f9590fe4dae384aeff04082510a709957e968753ff3c48399f6f92a \ + --hash=sha256:86df271bf921c2ee3818f0522e9a5b8092ca2ad8b065ece5d7d9d0e9f4849bcc \ + --hash=sha256:88ab34806dea0671532d3f82d82b85e8fc23d7b2dd12fa837978dad9bb392a34 \ + --hash=sha256:8999f965f922ae054125286faf9f11bc6932184b93011d138925a1773830bbe9 \ + --hash=sha256:8dcfc373f888e4fb39a7bc57e93e3b845e7f462dacc008d9749568b1c4ece096 \ + --hash=sha256:939578d9d8fd4299220161fdd76e86c6a251987476f5243e8864a7844476ba14 \ + --hash=sha256:96b2b3d1a83ad55310de8c7b4a2d04d9277d5591f40761274856635acc5fcb30 \ + --hash=sha256:a2d08ac246bb48479170408d6c19f6385fa743e7157d716e144cad849b2dd94b \ + --hash=sha256:b256ee2e749283ef3ddcff51a675ff43798d92d746d1a6e4631bf8c707d22d0b \ + --hash=sha256:b5e3b2d152e74e100a9e9573837aba24aab611d39428ded46f4e4022ea7d1942 \ + --hash=sha256:b89bc04de1d83006373429975f8ef9e7932534b8cc9ca582e4db7d20d91816db \ + --hash=sha256:bd28b817ea8c70215401f657edef3a8aa83c29d447fb0b622c35403780ba11d5 \ + --hash=sha256:c60e092517a73c632ec38e290eba714e9627abe9d301c8c8a12ec32c314a2a4b \ + --hash=sha256:c6dbd0ccdda3a2ba7c2ecd9d77b37f3b5831687d8dc1b6ca5f56a4880cc7b7ce \ + --hash=sha256:c6e490913a46fa054e03699c70019ab869e990270597018cef1d8562132c2669 \ + --hash=sha256:c6f162aabe9a91a309510d74eeb6507fab5fff92337a15acbe77753d88d9dcf0 \ + --hash=sha256:c6fd51128a41297f5409deab284fecbe5305ebd7e5a1f959bee1c054622b7018 \ + --hash=sha256:cc34f233c9e71701040d772aa7490318673aa7164a0efe3172b2981218c26d93 \ + --hash=sha256:cc9370a2da1ac13f0153780040f465839e6cccb4a1e44810124b4e22483c93fe \ + --hash=sha256:ccf600859c183d70eb47e05a44cd80a4ce77394d1ac0f79dbd2dd90a69a3a049 \ + --hash=sha256:ce571ab16d890d23b5c278547ba694193a45011ff86a9162a71307ed9f86759a \ + --hash=sha256:cf1ebb7d78e1ad8ec2a8c4732c7be2e736f6e5123a4146c5b89c9d1f585f8cef \ + --hash=sha256:d0e909868420b7049dafd3a31d45125b31143eec59235311fc4c57ea26a4acd2 \ + --hash=sha256:d22dbedd33326a4a5190dd4fe9e9e693ef12160c77382d9e87919bce54f3d4ca \ + --hash=sha256:d716a916938e03231e86e43782ca7878fb602a125a91e7acb8b5112e2e96ac16 \ + --hash=sha256:d79c198e27580c8e958906f803e63cddb77653731be08851c7df0b1a14a8fc0f \ + --hash=sha256:d95bfb53c211b57198bb91c46dd5a2d8018b3af446583aab40074bf7988401cb \ + --hash=sha256:e28e334d3ff134e88989d90ba04b47d84382a828c061d0d1027b1b12a62b39b1 \ + --hash=sha256:ec557499516fc90fd374bf2e32349a2887a876fbf162c160e3c01b6849eaf557 \ + --hash=sha256:fb6fecfd65564f208cbf0fba07f107fb661bcd1a7c389edbced3f7a493f70e37 \ + --hash=sha256:fb731e5deb0c7ef82d698b0f4c5bb724633ee2a489401594c5c88b02e6cb15f7 \ + --hash=sha256:fb7f67a1bfa6e40b438170ebdc8158b78dc465a5a67b6dde178a46987b244a72 \ + --hash=sha256:fd10de089bcdcd1be95a2f73dbe6254798ec1bda9f450d5828c96f93e2536b9c \ + --hash=sha256:fdabf8315679312cfa71302f9bd509ded4f2f263fb5b765cf1433b39106c3cc9 # via requests docutils==0.22 \ --hash=sha256:4ed966a0e96a0477d852f7af31bdcb3adc049fbb35ccba358c2ea8a03287615e \ diff --git a/tools/publish/requirements_linux.txt b/tools/publish/requirements_linux.txt index f2bfe6adf4..f8a065606c 100644 --- a/tools/publish/requirements_linux.txt +++ b/tools/publish/requirements_linux.txt @@ -79,99 +79,86 @@ cffi==1.17.1 \ --hash=sha256:f7f5baafcc48261359e14bcd6d9bff6d4b28d9103847c9e136694cb0501aef87 \ --hash=sha256:fc48c783f9c87e60831201f2cce7f3b2e4846bf4d8728eabe54d60700b318a0b # via cryptography -charset-normalizer==3.4.2 \ - --hash=sha256:005fa3432484527f9732ebd315da8da8001593e2cf46a3d817669f062c3d9ed4 \ - --hash=sha256:046595208aae0120559a67693ecc65dd75d46f7bf687f159127046628178dc45 \ - --hash=sha256:0c29de6a1a95f24b9a1aa7aefd27d2487263f00dfd55a77719b530788f75cff7 \ - --hash=sha256:0c8c57f84ccfc871a48a47321cfa49ae1df56cd1d965a09abe84066f6853b9c0 \ - --hash=sha256:0f5d9ed7f254402c9e7d35d2f5972c9bbea9040e99cd2861bd77dc68263277c7 \ - --hash=sha256:18dd2e350387c87dabe711b86f83c9c78af772c748904d372ade190b5c7c9d4d \ - --hash=sha256:1b1bde144d98e446b056ef98e59c256e9294f6b74d7af6846bf5ffdafd687a7d \ - --hash=sha256:1c95a1e2902a8b722868587c0e1184ad5c55631de5afc0eb96bc4b0d738092c0 \ - --hash=sha256:1cad5f45b3146325bb38d6855642f6fd609c3f7cad4dbaf75549bf3b904d3184 \ - --hash=sha256:21b2899062867b0e1fde9b724f8aecb1af14f2778d69aacd1a5a1853a597a5db \ - --hash=sha256:24498ba8ed6c2e0b56d4acbf83f2d989720a93b41d712ebd4f4979660db4417b \ - --hash=sha256:25a23ea5c7edc53e0f29bae2c44fcb5a1aa10591aae107f2a2b2583a9c5cbc64 \ - --hash=sha256:289200a18fa698949d2b39c671c2cc7a24d44096784e76614899a7ccf2574b7b \ - --hash=sha256:28a1005facc94196e1fb3e82a3d442a9d9110b8434fc1ded7a24a2983c9888d8 \ - --hash=sha256:32fc0341d72e0f73f80acb0a2c94216bd704f4f0bce10aedea38f30502b271ff \ - --hash=sha256:36b31da18b8890a76ec181c3cf44326bf2c48e36d393ca1b72b3f484113ea344 \ - --hash=sha256:3c21d4fca343c805a52c0c78edc01e3477f6dd1ad7c47653241cf2a206d4fc58 \ - --hash=sha256:3fddb7e2c84ac87ac3a947cb4e66d143ca5863ef48e4a5ecb83bd48619e4634e \ - --hash=sha256:43e0933a0eff183ee85833f341ec567c0980dae57c464d8a508e1b2ceb336471 \ - --hash=sha256:4a476b06fbcf359ad25d34a057b7219281286ae2477cc5ff5e3f70a246971148 \ - --hash=sha256:4e594135de17ab3866138f496755f302b72157d115086d100c3f19370839dd3a \ - --hash=sha256:50bf98d5e563b83cc29471fa114366e6806bc06bc7a25fd59641e41445327836 \ - --hash=sha256:5a9979887252a82fefd3d3ed2a8e3b937a7a809f65dcb1e068b090e165bbe99e \ - --hash=sha256:5baececa9ecba31eff645232d59845c07aa030f0c81ee70184a90d35099a0e63 \ - --hash=sha256:5bf4545e3b962767e5c06fe1738f951f77d27967cb2caa64c28be7c4563e162c \ - --hash=sha256:6333b3aa5a12c26b2a4d4e7335a28f1475e0e5e17d69d55141ee3cab736f66d1 \ - --hash=sha256:65c981bdbd3f57670af8b59777cbfae75364b483fa8a9f420f08094531d54a01 \ - --hash=sha256:68a328e5f55ec37c57f19ebb1fdc56a248db2e3e9ad769919a58672958e8f366 \ - --hash=sha256:6a0289e4589e8bdfef02a80478f1dfcb14f0ab696b5a00e1f4b8a14a307a3c58 \ - --hash=sha256:6b66f92b17849b85cad91259efc341dce9c1af48e2173bf38a85c6329f1033e5 \ - --hash=sha256:6c9379d65defcab82d07b2a9dfbfc2e95bc8fe0ebb1b176a3190230a3ef0e07c \ - --hash=sha256:6fc1f5b51fa4cecaa18f2bd7a003f3dd039dd615cd69a2afd6d3b19aed6775f2 \ - --hash=sha256:70f7172939fdf8790425ba31915bfbe8335030f05b9913d7ae00a87d4395620a \ - --hash=sha256:721c76e84fe669be19c5791da68232ca2e05ba5185575086e384352e2c309597 \ - --hash=sha256:7222ffd5e4de8e57e03ce2cef95a4c43c98fcb72ad86909abdfc2c17d227fc1b \ - --hash=sha256:75d10d37a47afee94919c4fab4c22b9bc2a8bf7d4f46f87363bcf0573f3ff4f5 \ - --hash=sha256:76af085e67e56c8816c3ccf256ebd136def2ed9654525348cfa744b6802b69eb \ - --hash=sha256:770cab594ecf99ae64c236bc9ee3439c3f46be49796e265ce0cc8bc17b10294f \ - --hash=sha256:7a6ab32f7210554a96cd9e33abe3ddd86732beeafc7a28e9955cdf22ffadbab0 \ - --hash=sha256:7c48ed483eb946e6c04ccbe02c6b4d1d48e51944b6db70f697e089c193404941 \ - --hash=sha256:7f56930ab0abd1c45cd15be65cc741c28b1c9a34876ce8c17a2fa107810c0af0 \ - --hash=sha256:8075c35cd58273fee266c58c0c9b670947c19df5fb98e7b66710e04ad4e9ff86 \ - --hash=sha256:8272b73e1c5603666618805fe821edba66892e2870058c94c53147602eab29c7 \ - --hash=sha256:82d8fd25b7f4675d0c47cf95b594d4e7b158aca33b76aa63d07186e13c0e0ab7 \ - --hash=sha256:844da2b5728b5ce0e32d863af26f32b5ce61bc4273a9c720a9f3aa9df73b1455 \ - --hash=sha256:8755483f3c00d6c9a77f490c17e6ab0c8729e39e6390328e42521ef175380ae6 \ - --hash=sha256:915f3849a011c1f593ab99092f3cecfcb4d65d8feb4a64cf1bf2d22074dc0ec4 \ - --hash=sha256:926ca93accd5d36ccdabd803392ddc3e03e6d4cd1cf17deff3b989ab8e9dbcf0 \ - --hash=sha256:982bb1e8b4ffda883b3d0a521e23abcd6fd17418f6d2c4118d257a10199c0ce3 \ - --hash=sha256:98f862da73774290f251b9df8d11161b6cf25b599a66baf087c1ffe340e9bfd1 \ - --hash=sha256:9cbfacf36cb0ec2897ce0ebc5d08ca44213af24265bd56eca54bee7923c48fd6 \ - --hash=sha256:a370b3e078e418187da8c3674eddb9d983ec09445c99a3a263c2011993522981 \ - --hash=sha256:a955b438e62efdf7e0b7b52a64dc5c3396e2634baa62471768a64bc2adb73d5c \ - --hash=sha256:aa6af9e7d59f9c12b33ae4e9450619cf2488e2bbe9b44030905877f0b2324980 \ - --hash=sha256:aa88ca0b1932e93f2d961bf3addbb2db902198dca337d88c89e1559e066e7645 \ - --hash=sha256:aaeeb6a479c7667fbe1099af9617c83aaca22182d6cf8c53966491a0f1b7ffb7 \ - --hash=sha256:aaf27faa992bfee0264dc1f03f4c75e9fcdda66a519db6b957a3f826e285cf12 \ - --hash=sha256:b2680962a4848b3c4f155dc2ee64505a9c57186d0d56b43123b17ca3de18f0fa \ - --hash=sha256:b2d318c11350e10662026ad0eb71bb51c7812fc8590825304ae0bdd4ac283acd \ - --hash=sha256:b33de11b92e9f75a2b545d6e9b6f37e398d86c3e9e9653c4864eb7e89c5773ef \ - --hash=sha256:b3daeac64d5b371dea99714f08ffc2c208522ec6b06fbc7866a450dd446f5c0f \ - --hash=sha256:be1e352acbe3c78727a16a455126d9ff83ea2dfdcbc83148d2982305a04714c2 \ - --hash=sha256:bee093bf902e1d8fc0ac143c88902c3dfc8941f7ea1d6a8dd2bcb786d33db03d \ - --hash=sha256:c72fbbe68c6f32f251bdc08b8611c7b3060612236e960ef848e0a517ddbe76c5 \ - --hash=sha256:c9e36a97bee9b86ef9a1cf7bb96747eb7a15c2f22bdb5b516434b00f2a599f02 \ - --hash=sha256:cddf7bd982eaa998934a91f69d182aec997c6c468898efe6679af88283b498d3 \ - --hash=sha256:cf713fe9a71ef6fd5adf7a79670135081cd4431c2943864757f0fa3a65b1fafd \ - --hash=sha256:d11b54acf878eef558599658b0ffca78138c8c3655cf4f3a4a673c437e67732e \ - --hash=sha256:d41c4d287cfc69060fa91cae9683eacffad989f1a10811995fa309df656ec214 \ - --hash=sha256:d524ba3f1581b35c03cb42beebab4a13e6cdad7b36246bd22541fa585a56cccd \ - --hash=sha256:daac4765328a919a805fa5e2720f3e94767abd632ae410a9062dff5412bae65a \ - --hash=sha256:db4c7bf0e07fc3b7d89ac2a5880a6a8062056801b83ff56d8464b70f65482b6c \ - --hash=sha256:dc7039885fa1baf9be153a0626e337aa7ec8bf96b0128605fb0d77788ddc1681 \ - --hash=sha256:dccab8d5fa1ef9bfba0590ecf4d46df048d18ffe3eec01eeb73a42e0d9e7a8ba \ - --hash=sha256:dedb8adb91d11846ee08bec4c8236c8549ac721c245678282dcb06b221aab59f \ - --hash=sha256:e45ba65510e2647721e35323d6ef54c7974959f6081b58d4ef5d87c60c84919a \ - --hash=sha256:e53efc7c7cee4c1e70661e2e112ca46a575f90ed9ae3fef200f2a25e954f4b28 \ - --hash=sha256:e635b87f01ebc977342e2697d05b56632f5f879a4f15955dfe8cef2448b51691 \ - --hash=sha256:e70e990b2137b29dc5564715de1e12701815dacc1d056308e2b17e9095372a82 \ - --hash=sha256:e8082b26888e2f8b36a042a58307d5b917ef2b1cacab921ad3323ef91901c71a \ - --hash=sha256:e8323a9b031aa0393768b87f04b4164a40037fb2a3c11ac06a03ffecd3618027 \ - --hash=sha256:e92fca20c46e9f5e1bb485887d074918b13543b1c2a1185e69bb8d17ab6236a7 \ - --hash=sha256:eb30abc20df9ab0814b5a2524f23d75dcf83cde762c161917a2b4b7b55b1e518 \ - --hash=sha256:eba9904b0f38a143592d9fc0e19e2df0fa2e41c3c3745554761c5f6447eedabf \ - --hash=sha256:ef8de666d6179b009dce7bcb2ad4c4a779f113f12caf8dc77f0162c29d20490b \ - --hash=sha256:efd387a49825780ff861998cd959767800d54f8308936b21025326de4b5a42b9 \ - --hash=sha256:f0aa37f3c979cf2546b73e8222bbfa3dc07a641585340179d768068e3455e544 \ - --hash=sha256:f4074c5a429281bf056ddd4c5d3b740ebca4d43ffffe2ef4bf4d2d05114299da \ - --hash=sha256:f69a27e45c43520f5487f27627059b64aaf160415589230992cec34c5e18a509 \ - --hash=sha256:fb707f3e15060adf5b7ada797624a6c6e0138e2a26baa089df64c68ee98e040f \ - --hash=sha256:fcbe676a55d7445b22c10967bceaaf0ee69407fbe0ece4d032b6eb8d4565982a \ - --hash=sha256:fdb20a30fe1175ecabed17cbf7812f7b804b8a315a25f24678bcdf120a90077f +charset-normalizer==3.4.3 \ + --hash=sha256:00237675befef519d9af72169d8604a067d92755e84fe76492fef5441db05b91 \ + --hash=sha256:02425242e96bcf29a49711b0ca9f37e451da7c70562bc10e8ed992a5a7a25cc0 \ + --hash=sha256:027b776c26d38b7f15b26a5da1044f376455fb3766df8fc38563b4efbc515154 \ + --hash=sha256:07a0eae9e2787b586e129fdcbe1af6997f8d0e5abaa0bc98c0e20e124d67e601 \ + --hash=sha256:0cacf8f7297b0c4fcb74227692ca46b4a5852f8f4f24b3c766dd94a1075c4884 \ + --hash=sha256:0e78314bdc32fa80696f72fa16dc61168fda4d6a0c014e0380f9d02f0e5d8a07 \ + --hash=sha256:0f2be7e0cf7754b9a30eb01f4295cc3d4358a479843b31f328afd210e2c7598c \ + --hash=sha256:13faeacfe61784e2559e690fc53fa4c5ae97c6fcedb8eb6fb8d0a15b475d2c64 \ + --hash=sha256:14c2a87c65b351109f6abfc424cab3927b3bdece6f706e4d12faaf3d52ee5efe \ + --hash=sha256:1606f4a55c0fd363d754049cdf400175ee96c992b1f8018b993941f221221c5f \ + --hash=sha256:16a8770207946ac75703458e2c743631c79c59c5890c80011d536248f8eaa432 \ + --hash=sha256:18343b2d246dc6761a249ba1fb13f9ee9a2bcd95decc767319506056ea4ad4dc \ + --hash=sha256:18b97b8404387b96cdbd30ad660f6407799126d26a39ca65729162fd810a99aa \ + --hash=sha256:1bb60174149316da1c35fa5233681f7c0f9f514509b8e399ab70fea5f17e45c9 \ + --hash=sha256:1e8ac75d72fa3775e0b7cb7e4629cec13b7514d928d15ef8ea06bca03ef01cae \ + --hash=sha256:1ef99f0456d3d46a50945c98de1774da86f8e992ab5c77865ea8b8195341fc19 \ + --hash=sha256:2001a39612b241dae17b4687898843f254f8748b796a2e16f1051a17078d991d \ + --hash=sha256:23b6b24d74478dc833444cbd927c338349d6ae852ba53a0d02a2de1fce45b96e \ + --hash=sha256:252098c8c7a873e17dd696ed98bbe91dbacd571da4b87df3736768efa7a792e4 \ + --hash=sha256:257f26fed7d7ff59921b78244f3cd93ed2af1800ff048c33f624c87475819dd7 \ + --hash=sha256:2c322db9c8c89009a990ef07c3bcc9f011a3269bc06782f916cd3d9eed7c9312 \ + --hash=sha256:30a96e1e1f865f78b030d65241c1ee850cdf422d869e9028e2fc1d5e4db73b92 \ + --hash=sha256:30d006f98569de3459c2fc1f2acde170b7b2bd265dc1943e87e1a4efe1b67c31 \ + --hash=sha256:31a9a6f775f9bcd865d88ee350f0ffb0e25936a7f930ca98995c05abf1faf21c \ + --hash=sha256:320e8e66157cc4e247d9ddca8e21f427efc7a04bbd0ac8a9faf56583fa543f9f \ + --hash=sha256:34a7f768e3f985abdb42841e20e17b330ad3aaf4bb7e7aeeb73db2e70f077b99 \ + --hash=sha256:3653fad4fe3ed447a596ae8638b437f827234f01a8cd801842e43f3d0a6b281b \ + --hash=sha256:3cd35b7e8aedeb9e34c41385fda4f73ba609e561faedfae0a9e75e44ac558a15 \ + --hash=sha256:3cfb2aad70f2c6debfbcb717f23b7eb55febc0bb23dcffc0f076009da10c6392 \ + --hash=sha256:416175faf02e4b0810f1f38bcb54682878a4af94059a1cd63b8747244420801f \ + --hash=sha256:41d1fc408ff5fdfb910200ec0e74abc40387bccb3252f3f27c0676731df2b2c8 \ + --hash=sha256:42e5088973e56e31e4fa58eb6bd709e42fc03799c11c42929592889a2e54c491 \ + --hash=sha256:4ca4c094de7771a98d7fbd67d9e5dbf1eb73efa4f744a730437d8a3a5cf994f0 \ + --hash=sha256:511729f456829ef86ac41ca78c63a5cb55240ed23b4b737faca0eb1abb1c41bc \ + --hash=sha256:53cd68b185d98dde4ad8990e56a58dea83a4162161b1ea9272e5c9182ce415e0 \ + --hash=sha256:585f3b2a80fbd26b048a0be90c5aae8f06605d3c92615911c3a2b03a8a3b796f \ + --hash=sha256:5b413b0b1bfd94dbf4023ad6945889f374cd24e3f62de58d6bb102c4d9ae534a \ + --hash=sha256:5d8d01eac18c423815ed4f4a2ec3b439d654e55ee4ad610e153cf02faf67ea40 \ + --hash=sha256:6aab0f181c486f973bc7262a97f5aca3ee7e1437011ef0c2ec04b5a11d16c927 \ + --hash=sha256:6cf8fd4c04756b6b60146d98cd8a77d0cdae0e1ca20329da2ac85eed779b6849 \ + --hash=sha256:6fb70de56f1859a3f71261cbe41005f56a7842cc348d3aeb26237560bfa5e0ce \ + --hash=sha256:6fce4b8500244f6fcb71465d4a4930d132ba9ab8e71a7859e6a5d59851068d14 \ + --hash=sha256:70bfc5f2c318afece2f5838ea5e4c3febada0be750fcf4775641052bbba14d05 \ + --hash=sha256:73dc19b562516fc9bcf6e5d6e596df0b4eb98d87e4f79f3ae71840e6ed21361c \ + --hash=sha256:74d77e25adda8581ffc1c720f1c81ca082921329452eba58b16233ab1842141c \ + --hash=sha256:78deba4d8f9590fe4dae384aeff04082510a709957e968753ff3c48399f6f92a \ + --hash=sha256:86df271bf921c2ee3818f0522e9a5b8092ca2ad8b065ece5d7d9d0e9f4849bcc \ + --hash=sha256:88ab34806dea0671532d3f82d82b85e8fc23d7b2dd12fa837978dad9bb392a34 \ + --hash=sha256:8999f965f922ae054125286faf9f11bc6932184b93011d138925a1773830bbe9 \ + --hash=sha256:8dcfc373f888e4fb39a7bc57e93e3b845e7f462dacc008d9749568b1c4ece096 \ + --hash=sha256:939578d9d8fd4299220161fdd76e86c6a251987476f5243e8864a7844476ba14 \ + --hash=sha256:96b2b3d1a83ad55310de8c7b4a2d04d9277d5591f40761274856635acc5fcb30 \ + --hash=sha256:a2d08ac246bb48479170408d6c19f6385fa743e7157d716e144cad849b2dd94b \ + --hash=sha256:b256ee2e749283ef3ddcff51a675ff43798d92d746d1a6e4631bf8c707d22d0b \ + --hash=sha256:b5e3b2d152e74e100a9e9573837aba24aab611d39428ded46f4e4022ea7d1942 \ + --hash=sha256:b89bc04de1d83006373429975f8ef9e7932534b8cc9ca582e4db7d20d91816db \ + --hash=sha256:bd28b817ea8c70215401f657edef3a8aa83c29d447fb0b622c35403780ba11d5 \ + --hash=sha256:c60e092517a73c632ec38e290eba714e9627abe9d301c8c8a12ec32c314a2a4b \ + --hash=sha256:c6dbd0ccdda3a2ba7c2ecd9d77b37f3b5831687d8dc1b6ca5f56a4880cc7b7ce \ + --hash=sha256:c6e490913a46fa054e03699c70019ab869e990270597018cef1d8562132c2669 \ + --hash=sha256:c6f162aabe9a91a309510d74eeb6507fab5fff92337a15acbe77753d88d9dcf0 \ + --hash=sha256:c6fd51128a41297f5409deab284fecbe5305ebd7e5a1f959bee1c054622b7018 \ + --hash=sha256:cc34f233c9e71701040d772aa7490318673aa7164a0efe3172b2981218c26d93 \ + --hash=sha256:cc9370a2da1ac13f0153780040f465839e6cccb4a1e44810124b4e22483c93fe \ + --hash=sha256:ccf600859c183d70eb47e05a44cd80a4ce77394d1ac0f79dbd2dd90a69a3a049 \ + --hash=sha256:ce571ab16d890d23b5c278547ba694193a45011ff86a9162a71307ed9f86759a \ + --hash=sha256:cf1ebb7d78e1ad8ec2a8c4732c7be2e736f6e5123a4146c5b89c9d1f585f8cef \ + --hash=sha256:d0e909868420b7049dafd3a31d45125b31143eec59235311fc4c57ea26a4acd2 \ + --hash=sha256:d22dbedd33326a4a5190dd4fe9e9e693ef12160c77382d9e87919bce54f3d4ca \ + --hash=sha256:d716a916938e03231e86e43782ca7878fb602a125a91e7acb8b5112e2e96ac16 \ + --hash=sha256:d79c198e27580c8e958906f803e63cddb77653731be08851c7df0b1a14a8fc0f \ + --hash=sha256:d95bfb53c211b57198bb91c46dd5a2d8018b3af446583aab40074bf7988401cb \ + --hash=sha256:e28e334d3ff134e88989d90ba04b47d84382a828c061d0d1027b1b12a62b39b1 \ + --hash=sha256:ec557499516fc90fd374bf2e32349a2887a876fbf162c160e3c01b6849eaf557 \ + --hash=sha256:fb6fecfd65564f208cbf0fba07f107fb661bcd1a7c389edbced3f7a493f70e37 \ + --hash=sha256:fb731e5deb0c7ef82d698b0f4c5bb724633ee2a489401594c5c88b02e6cb15f7 \ + --hash=sha256:fb7f67a1bfa6e40b438170ebdc8158b78dc465a5a67b6dde178a46987b244a72 \ + --hash=sha256:fd10de089bcdcd1be95a2f73dbe6254798ec1bda9f450d5828c96f93e2536b9c \ + --hash=sha256:fdabf8315679312cfa71302f9bd509ded4f2f263fb5b765cf1433b39106c3cc9 # via requests cryptography==44.0.1 \ --hash=sha256:00918d859aa4e57db8299607086f793fa7813ae2ff5a4637e318a25ef82730f7 \ diff --git a/tools/publish/requirements_universal.txt b/tools/publish/requirements_universal.txt index 42e74a0296..7d6b37c955 100644 --- a/tools/publish/requirements_universal.txt +++ b/tools/publish/requirements_universal.txt @@ -79,99 +79,86 @@ cffi==1.17.1 ; platform_python_implementation != 'PyPy' and sys_platform == 'lin --hash=sha256:f7f5baafcc48261359e14bcd6d9bff6d4b28d9103847c9e136694cb0501aef87 \ --hash=sha256:fc48c783f9c87e60831201f2cce7f3b2e4846bf4d8728eabe54d60700b318a0b # via cryptography -charset-normalizer==3.4.2 \ - --hash=sha256:005fa3432484527f9732ebd315da8da8001593e2cf46a3d817669f062c3d9ed4 \ - --hash=sha256:046595208aae0120559a67693ecc65dd75d46f7bf687f159127046628178dc45 \ - --hash=sha256:0c29de6a1a95f24b9a1aa7aefd27d2487263f00dfd55a77719b530788f75cff7 \ - --hash=sha256:0c8c57f84ccfc871a48a47321cfa49ae1df56cd1d965a09abe84066f6853b9c0 \ - --hash=sha256:0f5d9ed7f254402c9e7d35d2f5972c9bbea9040e99cd2861bd77dc68263277c7 \ - --hash=sha256:18dd2e350387c87dabe711b86f83c9c78af772c748904d372ade190b5c7c9d4d \ - --hash=sha256:1b1bde144d98e446b056ef98e59c256e9294f6b74d7af6846bf5ffdafd687a7d \ - --hash=sha256:1c95a1e2902a8b722868587c0e1184ad5c55631de5afc0eb96bc4b0d738092c0 \ - --hash=sha256:1cad5f45b3146325bb38d6855642f6fd609c3f7cad4dbaf75549bf3b904d3184 \ - --hash=sha256:21b2899062867b0e1fde9b724f8aecb1af14f2778d69aacd1a5a1853a597a5db \ - --hash=sha256:24498ba8ed6c2e0b56d4acbf83f2d989720a93b41d712ebd4f4979660db4417b \ - --hash=sha256:25a23ea5c7edc53e0f29bae2c44fcb5a1aa10591aae107f2a2b2583a9c5cbc64 \ - --hash=sha256:289200a18fa698949d2b39c671c2cc7a24d44096784e76614899a7ccf2574b7b \ - --hash=sha256:28a1005facc94196e1fb3e82a3d442a9d9110b8434fc1ded7a24a2983c9888d8 \ - --hash=sha256:32fc0341d72e0f73f80acb0a2c94216bd704f4f0bce10aedea38f30502b271ff \ - --hash=sha256:36b31da18b8890a76ec181c3cf44326bf2c48e36d393ca1b72b3f484113ea344 \ - --hash=sha256:3c21d4fca343c805a52c0c78edc01e3477f6dd1ad7c47653241cf2a206d4fc58 \ - --hash=sha256:3fddb7e2c84ac87ac3a947cb4e66d143ca5863ef48e4a5ecb83bd48619e4634e \ - --hash=sha256:43e0933a0eff183ee85833f341ec567c0980dae57c464d8a508e1b2ceb336471 \ - --hash=sha256:4a476b06fbcf359ad25d34a057b7219281286ae2477cc5ff5e3f70a246971148 \ - --hash=sha256:4e594135de17ab3866138f496755f302b72157d115086d100c3f19370839dd3a \ - --hash=sha256:50bf98d5e563b83cc29471fa114366e6806bc06bc7a25fd59641e41445327836 \ - --hash=sha256:5a9979887252a82fefd3d3ed2a8e3b937a7a809f65dcb1e068b090e165bbe99e \ - --hash=sha256:5baececa9ecba31eff645232d59845c07aa030f0c81ee70184a90d35099a0e63 \ - --hash=sha256:5bf4545e3b962767e5c06fe1738f951f77d27967cb2caa64c28be7c4563e162c \ - --hash=sha256:6333b3aa5a12c26b2a4d4e7335a28f1475e0e5e17d69d55141ee3cab736f66d1 \ - --hash=sha256:65c981bdbd3f57670af8b59777cbfae75364b483fa8a9f420f08094531d54a01 \ - --hash=sha256:68a328e5f55ec37c57f19ebb1fdc56a248db2e3e9ad769919a58672958e8f366 \ - --hash=sha256:6a0289e4589e8bdfef02a80478f1dfcb14f0ab696b5a00e1f4b8a14a307a3c58 \ - --hash=sha256:6b66f92b17849b85cad91259efc341dce9c1af48e2173bf38a85c6329f1033e5 \ - --hash=sha256:6c9379d65defcab82d07b2a9dfbfc2e95bc8fe0ebb1b176a3190230a3ef0e07c \ - --hash=sha256:6fc1f5b51fa4cecaa18f2bd7a003f3dd039dd615cd69a2afd6d3b19aed6775f2 \ - --hash=sha256:70f7172939fdf8790425ba31915bfbe8335030f05b9913d7ae00a87d4395620a \ - --hash=sha256:721c76e84fe669be19c5791da68232ca2e05ba5185575086e384352e2c309597 \ - --hash=sha256:7222ffd5e4de8e57e03ce2cef95a4c43c98fcb72ad86909abdfc2c17d227fc1b \ - --hash=sha256:75d10d37a47afee94919c4fab4c22b9bc2a8bf7d4f46f87363bcf0573f3ff4f5 \ - --hash=sha256:76af085e67e56c8816c3ccf256ebd136def2ed9654525348cfa744b6802b69eb \ - --hash=sha256:770cab594ecf99ae64c236bc9ee3439c3f46be49796e265ce0cc8bc17b10294f \ - --hash=sha256:7a6ab32f7210554a96cd9e33abe3ddd86732beeafc7a28e9955cdf22ffadbab0 \ - --hash=sha256:7c48ed483eb946e6c04ccbe02c6b4d1d48e51944b6db70f697e089c193404941 \ - --hash=sha256:7f56930ab0abd1c45cd15be65cc741c28b1c9a34876ce8c17a2fa107810c0af0 \ - --hash=sha256:8075c35cd58273fee266c58c0c9b670947c19df5fb98e7b66710e04ad4e9ff86 \ - --hash=sha256:8272b73e1c5603666618805fe821edba66892e2870058c94c53147602eab29c7 \ - --hash=sha256:82d8fd25b7f4675d0c47cf95b594d4e7b158aca33b76aa63d07186e13c0e0ab7 \ - --hash=sha256:844da2b5728b5ce0e32d863af26f32b5ce61bc4273a9c720a9f3aa9df73b1455 \ - --hash=sha256:8755483f3c00d6c9a77f490c17e6ab0c8729e39e6390328e42521ef175380ae6 \ - --hash=sha256:915f3849a011c1f593ab99092f3cecfcb4d65d8feb4a64cf1bf2d22074dc0ec4 \ - --hash=sha256:926ca93accd5d36ccdabd803392ddc3e03e6d4cd1cf17deff3b989ab8e9dbcf0 \ - --hash=sha256:982bb1e8b4ffda883b3d0a521e23abcd6fd17418f6d2c4118d257a10199c0ce3 \ - --hash=sha256:98f862da73774290f251b9df8d11161b6cf25b599a66baf087c1ffe340e9bfd1 \ - --hash=sha256:9cbfacf36cb0ec2897ce0ebc5d08ca44213af24265bd56eca54bee7923c48fd6 \ - --hash=sha256:a370b3e078e418187da8c3674eddb9d983ec09445c99a3a263c2011993522981 \ - --hash=sha256:a955b438e62efdf7e0b7b52a64dc5c3396e2634baa62471768a64bc2adb73d5c \ - --hash=sha256:aa6af9e7d59f9c12b33ae4e9450619cf2488e2bbe9b44030905877f0b2324980 \ - --hash=sha256:aa88ca0b1932e93f2d961bf3addbb2db902198dca337d88c89e1559e066e7645 \ - --hash=sha256:aaeeb6a479c7667fbe1099af9617c83aaca22182d6cf8c53966491a0f1b7ffb7 \ - --hash=sha256:aaf27faa992bfee0264dc1f03f4c75e9fcdda66a519db6b957a3f826e285cf12 \ - --hash=sha256:b2680962a4848b3c4f155dc2ee64505a9c57186d0d56b43123b17ca3de18f0fa \ - --hash=sha256:b2d318c11350e10662026ad0eb71bb51c7812fc8590825304ae0bdd4ac283acd \ - --hash=sha256:b33de11b92e9f75a2b545d6e9b6f37e398d86c3e9e9653c4864eb7e89c5773ef \ - --hash=sha256:b3daeac64d5b371dea99714f08ffc2c208522ec6b06fbc7866a450dd446f5c0f \ - --hash=sha256:be1e352acbe3c78727a16a455126d9ff83ea2dfdcbc83148d2982305a04714c2 \ - --hash=sha256:bee093bf902e1d8fc0ac143c88902c3dfc8941f7ea1d6a8dd2bcb786d33db03d \ - --hash=sha256:c72fbbe68c6f32f251bdc08b8611c7b3060612236e960ef848e0a517ddbe76c5 \ - --hash=sha256:c9e36a97bee9b86ef9a1cf7bb96747eb7a15c2f22bdb5b516434b00f2a599f02 \ - --hash=sha256:cddf7bd982eaa998934a91f69d182aec997c6c468898efe6679af88283b498d3 \ - --hash=sha256:cf713fe9a71ef6fd5adf7a79670135081cd4431c2943864757f0fa3a65b1fafd \ - --hash=sha256:d11b54acf878eef558599658b0ffca78138c8c3655cf4f3a4a673c437e67732e \ - --hash=sha256:d41c4d287cfc69060fa91cae9683eacffad989f1a10811995fa309df656ec214 \ - --hash=sha256:d524ba3f1581b35c03cb42beebab4a13e6cdad7b36246bd22541fa585a56cccd \ - --hash=sha256:daac4765328a919a805fa5e2720f3e94767abd632ae410a9062dff5412bae65a \ - --hash=sha256:db4c7bf0e07fc3b7d89ac2a5880a6a8062056801b83ff56d8464b70f65482b6c \ - --hash=sha256:dc7039885fa1baf9be153a0626e337aa7ec8bf96b0128605fb0d77788ddc1681 \ - --hash=sha256:dccab8d5fa1ef9bfba0590ecf4d46df048d18ffe3eec01eeb73a42e0d9e7a8ba \ - --hash=sha256:dedb8adb91d11846ee08bec4c8236c8549ac721c245678282dcb06b221aab59f \ - --hash=sha256:e45ba65510e2647721e35323d6ef54c7974959f6081b58d4ef5d87c60c84919a \ - --hash=sha256:e53efc7c7cee4c1e70661e2e112ca46a575f90ed9ae3fef200f2a25e954f4b28 \ - --hash=sha256:e635b87f01ebc977342e2697d05b56632f5f879a4f15955dfe8cef2448b51691 \ - --hash=sha256:e70e990b2137b29dc5564715de1e12701815dacc1d056308e2b17e9095372a82 \ - --hash=sha256:e8082b26888e2f8b36a042a58307d5b917ef2b1cacab921ad3323ef91901c71a \ - --hash=sha256:e8323a9b031aa0393768b87f04b4164a40037fb2a3c11ac06a03ffecd3618027 \ - --hash=sha256:e92fca20c46e9f5e1bb485887d074918b13543b1c2a1185e69bb8d17ab6236a7 \ - --hash=sha256:eb30abc20df9ab0814b5a2524f23d75dcf83cde762c161917a2b4b7b55b1e518 \ - --hash=sha256:eba9904b0f38a143592d9fc0e19e2df0fa2e41c3c3745554761c5f6447eedabf \ - --hash=sha256:ef8de666d6179b009dce7bcb2ad4c4a779f113f12caf8dc77f0162c29d20490b \ - --hash=sha256:efd387a49825780ff861998cd959767800d54f8308936b21025326de4b5a42b9 \ - --hash=sha256:f0aa37f3c979cf2546b73e8222bbfa3dc07a641585340179d768068e3455e544 \ - --hash=sha256:f4074c5a429281bf056ddd4c5d3b740ebca4d43ffffe2ef4bf4d2d05114299da \ - --hash=sha256:f69a27e45c43520f5487f27627059b64aaf160415589230992cec34c5e18a509 \ - --hash=sha256:fb707f3e15060adf5b7ada797624a6c6e0138e2a26baa089df64c68ee98e040f \ - --hash=sha256:fcbe676a55d7445b22c10967bceaaf0ee69407fbe0ece4d032b6eb8d4565982a \ - --hash=sha256:fdb20a30fe1175ecabed17cbf7812f7b804b8a315a25f24678bcdf120a90077f +charset-normalizer==3.4.3 \ + --hash=sha256:00237675befef519d9af72169d8604a067d92755e84fe76492fef5441db05b91 \ + --hash=sha256:02425242e96bcf29a49711b0ca9f37e451da7c70562bc10e8ed992a5a7a25cc0 \ + --hash=sha256:027b776c26d38b7f15b26a5da1044f376455fb3766df8fc38563b4efbc515154 \ + --hash=sha256:07a0eae9e2787b586e129fdcbe1af6997f8d0e5abaa0bc98c0e20e124d67e601 \ + --hash=sha256:0cacf8f7297b0c4fcb74227692ca46b4a5852f8f4f24b3c766dd94a1075c4884 \ + --hash=sha256:0e78314bdc32fa80696f72fa16dc61168fda4d6a0c014e0380f9d02f0e5d8a07 \ + --hash=sha256:0f2be7e0cf7754b9a30eb01f4295cc3d4358a479843b31f328afd210e2c7598c \ + --hash=sha256:13faeacfe61784e2559e690fc53fa4c5ae97c6fcedb8eb6fb8d0a15b475d2c64 \ + --hash=sha256:14c2a87c65b351109f6abfc424cab3927b3bdece6f706e4d12faaf3d52ee5efe \ + --hash=sha256:1606f4a55c0fd363d754049cdf400175ee96c992b1f8018b993941f221221c5f \ + --hash=sha256:16a8770207946ac75703458e2c743631c79c59c5890c80011d536248f8eaa432 \ + --hash=sha256:18343b2d246dc6761a249ba1fb13f9ee9a2bcd95decc767319506056ea4ad4dc \ + --hash=sha256:18b97b8404387b96cdbd30ad660f6407799126d26a39ca65729162fd810a99aa \ + --hash=sha256:1bb60174149316da1c35fa5233681f7c0f9f514509b8e399ab70fea5f17e45c9 \ + --hash=sha256:1e8ac75d72fa3775e0b7cb7e4629cec13b7514d928d15ef8ea06bca03ef01cae \ + --hash=sha256:1ef99f0456d3d46a50945c98de1774da86f8e992ab5c77865ea8b8195341fc19 \ + --hash=sha256:2001a39612b241dae17b4687898843f254f8748b796a2e16f1051a17078d991d \ + --hash=sha256:23b6b24d74478dc833444cbd927c338349d6ae852ba53a0d02a2de1fce45b96e \ + --hash=sha256:252098c8c7a873e17dd696ed98bbe91dbacd571da4b87df3736768efa7a792e4 \ + --hash=sha256:257f26fed7d7ff59921b78244f3cd93ed2af1800ff048c33f624c87475819dd7 \ + --hash=sha256:2c322db9c8c89009a990ef07c3bcc9f011a3269bc06782f916cd3d9eed7c9312 \ + --hash=sha256:30a96e1e1f865f78b030d65241c1ee850cdf422d869e9028e2fc1d5e4db73b92 \ + --hash=sha256:30d006f98569de3459c2fc1f2acde170b7b2bd265dc1943e87e1a4efe1b67c31 \ + --hash=sha256:31a9a6f775f9bcd865d88ee350f0ffb0e25936a7f930ca98995c05abf1faf21c \ + --hash=sha256:320e8e66157cc4e247d9ddca8e21f427efc7a04bbd0ac8a9faf56583fa543f9f \ + --hash=sha256:34a7f768e3f985abdb42841e20e17b330ad3aaf4bb7e7aeeb73db2e70f077b99 \ + --hash=sha256:3653fad4fe3ed447a596ae8638b437f827234f01a8cd801842e43f3d0a6b281b \ + --hash=sha256:3cd35b7e8aedeb9e34c41385fda4f73ba609e561faedfae0a9e75e44ac558a15 \ + --hash=sha256:3cfb2aad70f2c6debfbcb717f23b7eb55febc0bb23dcffc0f076009da10c6392 \ + --hash=sha256:416175faf02e4b0810f1f38bcb54682878a4af94059a1cd63b8747244420801f \ + --hash=sha256:41d1fc408ff5fdfb910200ec0e74abc40387bccb3252f3f27c0676731df2b2c8 \ + --hash=sha256:42e5088973e56e31e4fa58eb6bd709e42fc03799c11c42929592889a2e54c491 \ + --hash=sha256:4ca4c094de7771a98d7fbd67d9e5dbf1eb73efa4f744a730437d8a3a5cf994f0 \ + --hash=sha256:511729f456829ef86ac41ca78c63a5cb55240ed23b4b737faca0eb1abb1c41bc \ + --hash=sha256:53cd68b185d98dde4ad8990e56a58dea83a4162161b1ea9272e5c9182ce415e0 \ + --hash=sha256:585f3b2a80fbd26b048a0be90c5aae8f06605d3c92615911c3a2b03a8a3b796f \ + --hash=sha256:5b413b0b1bfd94dbf4023ad6945889f374cd24e3f62de58d6bb102c4d9ae534a \ + --hash=sha256:5d8d01eac18c423815ed4f4a2ec3b439d654e55ee4ad610e153cf02faf67ea40 \ + --hash=sha256:6aab0f181c486f973bc7262a97f5aca3ee7e1437011ef0c2ec04b5a11d16c927 \ + --hash=sha256:6cf8fd4c04756b6b60146d98cd8a77d0cdae0e1ca20329da2ac85eed779b6849 \ + --hash=sha256:6fb70de56f1859a3f71261cbe41005f56a7842cc348d3aeb26237560bfa5e0ce \ + --hash=sha256:6fce4b8500244f6fcb71465d4a4930d132ba9ab8e71a7859e6a5d59851068d14 \ + --hash=sha256:70bfc5f2c318afece2f5838ea5e4c3febada0be750fcf4775641052bbba14d05 \ + --hash=sha256:73dc19b562516fc9bcf6e5d6e596df0b4eb98d87e4f79f3ae71840e6ed21361c \ + --hash=sha256:74d77e25adda8581ffc1c720f1c81ca082921329452eba58b16233ab1842141c \ + --hash=sha256:78deba4d8f9590fe4dae384aeff04082510a709957e968753ff3c48399f6f92a \ + --hash=sha256:86df271bf921c2ee3818f0522e9a5b8092ca2ad8b065ece5d7d9d0e9f4849bcc \ + --hash=sha256:88ab34806dea0671532d3f82d82b85e8fc23d7b2dd12fa837978dad9bb392a34 \ + --hash=sha256:8999f965f922ae054125286faf9f11bc6932184b93011d138925a1773830bbe9 \ + --hash=sha256:8dcfc373f888e4fb39a7bc57e93e3b845e7f462dacc008d9749568b1c4ece096 \ + --hash=sha256:939578d9d8fd4299220161fdd76e86c6a251987476f5243e8864a7844476ba14 \ + --hash=sha256:96b2b3d1a83ad55310de8c7b4a2d04d9277d5591f40761274856635acc5fcb30 \ + --hash=sha256:a2d08ac246bb48479170408d6c19f6385fa743e7157d716e144cad849b2dd94b \ + --hash=sha256:b256ee2e749283ef3ddcff51a675ff43798d92d746d1a6e4631bf8c707d22d0b \ + --hash=sha256:b5e3b2d152e74e100a9e9573837aba24aab611d39428ded46f4e4022ea7d1942 \ + --hash=sha256:b89bc04de1d83006373429975f8ef9e7932534b8cc9ca582e4db7d20d91816db \ + --hash=sha256:bd28b817ea8c70215401f657edef3a8aa83c29d447fb0b622c35403780ba11d5 \ + --hash=sha256:c60e092517a73c632ec38e290eba714e9627abe9d301c8c8a12ec32c314a2a4b \ + --hash=sha256:c6dbd0ccdda3a2ba7c2ecd9d77b37f3b5831687d8dc1b6ca5f56a4880cc7b7ce \ + --hash=sha256:c6e490913a46fa054e03699c70019ab869e990270597018cef1d8562132c2669 \ + --hash=sha256:c6f162aabe9a91a309510d74eeb6507fab5fff92337a15acbe77753d88d9dcf0 \ + --hash=sha256:c6fd51128a41297f5409deab284fecbe5305ebd7e5a1f959bee1c054622b7018 \ + --hash=sha256:cc34f233c9e71701040d772aa7490318673aa7164a0efe3172b2981218c26d93 \ + --hash=sha256:cc9370a2da1ac13f0153780040f465839e6cccb4a1e44810124b4e22483c93fe \ + --hash=sha256:ccf600859c183d70eb47e05a44cd80a4ce77394d1ac0f79dbd2dd90a69a3a049 \ + --hash=sha256:ce571ab16d890d23b5c278547ba694193a45011ff86a9162a71307ed9f86759a \ + --hash=sha256:cf1ebb7d78e1ad8ec2a8c4732c7be2e736f6e5123a4146c5b89c9d1f585f8cef \ + --hash=sha256:d0e909868420b7049dafd3a31d45125b31143eec59235311fc4c57ea26a4acd2 \ + --hash=sha256:d22dbedd33326a4a5190dd4fe9e9e693ef12160c77382d9e87919bce54f3d4ca \ + --hash=sha256:d716a916938e03231e86e43782ca7878fb602a125a91e7acb8b5112e2e96ac16 \ + --hash=sha256:d79c198e27580c8e958906f803e63cddb77653731be08851c7df0b1a14a8fc0f \ + --hash=sha256:d95bfb53c211b57198bb91c46dd5a2d8018b3af446583aab40074bf7988401cb \ + --hash=sha256:e28e334d3ff134e88989d90ba04b47d84382a828c061d0d1027b1b12a62b39b1 \ + --hash=sha256:ec557499516fc90fd374bf2e32349a2887a876fbf162c160e3c01b6849eaf557 \ + --hash=sha256:fb6fecfd65564f208cbf0fba07f107fb661bcd1a7c389edbced3f7a493f70e37 \ + --hash=sha256:fb731e5deb0c7ef82d698b0f4c5bb724633ee2a489401594c5c88b02e6cb15f7 \ + --hash=sha256:fb7f67a1bfa6e40b438170ebdc8158b78dc465a5a67b6dde178a46987b244a72 \ + --hash=sha256:fd10de089bcdcd1be95a2f73dbe6254798ec1bda9f450d5828c96f93e2536b9c \ + --hash=sha256:fdabf8315679312cfa71302f9bd509ded4f2f263fb5b765cf1433b39106c3cc9 # via requests cryptography==44.0.1 ; sys_platform == 'linux' \ --hash=sha256:00918d859aa4e57db8299607086f793fa7813ae2ff5a4637e318a25ef82730f7 \ diff --git a/tools/publish/requirements_windows.txt b/tools/publish/requirements_windows.txt index 650821f363..18356503f5 100644 --- a/tools/publish/requirements_windows.txt +++ b/tools/publish/requirements_windows.txt @@ -10,99 +10,86 @@ certifi==2025.8.3 \ --hash=sha256:e564105f78ded564e3ae7c923924435e1daa7463faeab5bb932bc53ffae63407 \ --hash=sha256:f6c12493cfb1b06ba2ff328595af9350c65d6644968e5d3a2ffd78699af217a5 # via requests -charset-normalizer==3.4.2 \ - --hash=sha256:005fa3432484527f9732ebd315da8da8001593e2cf46a3d817669f062c3d9ed4 \ - --hash=sha256:046595208aae0120559a67693ecc65dd75d46f7bf687f159127046628178dc45 \ - --hash=sha256:0c29de6a1a95f24b9a1aa7aefd27d2487263f00dfd55a77719b530788f75cff7 \ - --hash=sha256:0c8c57f84ccfc871a48a47321cfa49ae1df56cd1d965a09abe84066f6853b9c0 \ - --hash=sha256:0f5d9ed7f254402c9e7d35d2f5972c9bbea9040e99cd2861bd77dc68263277c7 \ - --hash=sha256:18dd2e350387c87dabe711b86f83c9c78af772c748904d372ade190b5c7c9d4d \ - --hash=sha256:1b1bde144d98e446b056ef98e59c256e9294f6b74d7af6846bf5ffdafd687a7d \ - --hash=sha256:1c95a1e2902a8b722868587c0e1184ad5c55631de5afc0eb96bc4b0d738092c0 \ - --hash=sha256:1cad5f45b3146325bb38d6855642f6fd609c3f7cad4dbaf75549bf3b904d3184 \ - --hash=sha256:21b2899062867b0e1fde9b724f8aecb1af14f2778d69aacd1a5a1853a597a5db \ - --hash=sha256:24498ba8ed6c2e0b56d4acbf83f2d989720a93b41d712ebd4f4979660db4417b \ - --hash=sha256:25a23ea5c7edc53e0f29bae2c44fcb5a1aa10591aae107f2a2b2583a9c5cbc64 \ - --hash=sha256:289200a18fa698949d2b39c671c2cc7a24d44096784e76614899a7ccf2574b7b \ - --hash=sha256:28a1005facc94196e1fb3e82a3d442a9d9110b8434fc1ded7a24a2983c9888d8 \ - --hash=sha256:32fc0341d72e0f73f80acb0a2c94216bd704f4f0bce10aedea38f30502b271ff \ - --hash=sha256:36b31da18b8890a76ec181c3cf44326bf2c48e36d393ca1b72b3f484113ea344 \ - --hash=sha256:3c21d4fca343c805a52c0c78edc01e3477f6dd1ad7c47653241cf2a206d4fc58 \ - --hash=sha256:3fddb7e2c84ac87ac3a947cb4e66d143ca5863ef48e4a5ecb83bd48619e4634e \ - --hash=sha256:43e0933a0eff183ee85833f341ec567c0980dae57c464d8a508e1b2ceb336471 \ - --hash=sha256:4a476b06fbcf359ad25d34a057b7219281286ae2477cc5ff5e3f70a246971148 \ - --hash=sha256:4e594135de17ab3866138f496755f302b72157d115086d100c3f19370839dd3a \ - --hash=sha256:50bf98d5e563b83cc29471fa114366e6806bc06bc7a25fd59641e41445327836 \ - --hash=sha256:5a9979887252a82fefd3d3ed2a8e3b937a7a809f65dcb1e068b090e165bbe99e \ - --hash=sha256:5baececa9ecba31eff645232d59845c07aa030f0c81ee70184a90d35099a0e63 \ - --hash=sha256:5bf4545e3b962767e5c06fe1738f951f77d27967cb2caa64c28be7c4563e162c \ - --hash=sha256:6333b3aa5a12c26b2a4d4e7335a28f1475e0e5e17d69d55141ee3cab736f66d1 \ - --hash=sha256:65c981bdbd3f57670af8b59777cbfae75364b483fa8a9f420f08094531d54a01 \ - --hash=sha256:68a328e5f55ec37c57f19ebb1fdc56a248db2e3e9ad769919a58672958e8f366 \ - --hash=sha256:6a0289e4589e8bdfef02a80478f1dfcb14f0ab696b5a00e1f4b8a14a307a3c58 \ - --hash=sha256:6b66f92b17849b85cad91259efc341dce9c1af48e2173bf38a85c6329f1033e5 \ - --hash=sha256:6c9379d65defcab82d07b2a9dfbfc2e95bc8fe0ebb1b176a3190230a3ef0e07c \ - --hash=sha256:6fc1f5b51fa4cecaa18f2bd7a003f3dd039dd615cd69a2afd6d3b19aed6775f2 \ - --hash=sha256:70f7172939fdf8790425ba31915bfbe8335030f05b9913d7ae00a87d4395620a \ - --hash=sha256:721c76e84fe669be19c5791da68232ca2e05ba5185575086e384352e2c309597 \ - --hash=sha256:7222ffd5e4de8e57e03ce2cef95a4c43c98fcb72ad86909abdfc2c17d227fc1b \ - --hash=sha256:75d10d37a47afee94919c4fab4c22b9bc2a8bf7d4f46f87363bcf0573f3ff4f5 \ - --hash=sha256:76af085e67e56c8816c3ccf256ebd136def2ed9654525348cfa744b6802b69eb \ - --hash=sha256:770cab594ecf99ae64c236bc9ee3439c3f46be49796e265ce0cc8bc17b10294f \ - --hash=sha256:7a6ab32f7210554a96cd9e33abe3ddd86732beeafc7a28e9955cdf22ffadbab0 \ - --hash=sha256:7c48ed483eb946e6c04ccbe02c6b4d1d48e51944b6db70f697e089c193404941 \ - --hash=sha256:7f56930ab0abd1c45cd15be65cc741c28b1c9a34876ce8c17a2fa107810c0af0 \ - --hash=sha256:8075c35cd58273fee266c58c0c9b670947c19df5fb98e7b66710e04ad4e9ff86 \ - --hash=sha256:8272b73e1c5603666618805fe821edba66892e2870058c94c53147602eab29c7 \ - --hash=sha256:82d8fd25b7f4675d0c47cf95b594d4e7b158aca33b76aa63d07186e13c0e0ab7 \ - --hash=sha256:844da2b5728b5ce0e32d863af26f32b5ce61bc4273a9c720a9f3aa9df73b1455 \ - --hash=sha256:8755483f3c00d6c9a77f490c17e6ab0c8729e39e6390328e42521ef175380ae6 \ - --hash=sha256:915f3849a011c1f593ab99092f3cecfcb4d65d8feb4a64cf1bf2d22074dc0ec4 \ - --hash=sha256:926ca93accd5d36ccdabd803392ddc3e03e6d4cd1cf17deff3b989ab8e9dbcf0 \ - --hash=sha256:982bb1e8b4ffda883b3d0a521e23abcd6fd17418f6d2c4118d257a10199c0ce3 \ - --hash=sha256:98f862da73774290f251b9df8d11161b6cf25b599a66baf087c1ffe340e9bfd1 \ - --hash=sha256:9cbfacf36cb0ec2897ce0ebc5d08ca44213af24265bd56eca54bee7923c48fd6 \ - --hash=sha256:a370b3e078e418187da8c3674eddb9d983ec09445c99a3a263c2011993522981 \ - --hash=sha256:a955b438e62efdf7e0b7b52a64dc5c3396e2634baa62471768a64bc2adb73d5c \ - --hash=sha256:aa6af9e7d59f9c12b33ae4e9450619cf2488e2bbe9b44030905877f0b2324980 \ - --hash=sha256:aa88ca0b1932e93f2d961bf3addbb2db902198dca337d88c89e1559e066e7645 \ - --hash=sha256:aaeeb6a479c7667fbe1099af9617c83aaca22182d6cf8c53966491a0f1b7ffb7 \ - --hash=sha256:aaf27faa992bfee0264dc1f03f4c75e9fcdda66a519db6b957a3f826e285cf12 \ - --hash=sha256:b2680962a4848b3c4f155dc2ee64505a9c57186d0d56b43123b17ca3de18f0fa \ - --hash=sha256:b2d318c11350e10662026ad0eb71bb51c7812fc8590825304ae0bdd4ac283acd \ - --hash=sha256:b33de11b92e9f75a2b545d6e9b6f37e398d86c3e9e9653c4864eb7e89c5773ef \ - --hash=sha256:b3daeac64d5b371dea99714f08ffc2c208522ec6b06fbc7866a450dd446f5c0f \ - --hash=sha256:be1e352acbe3c78727a16a455126d9ff83ea2dfdcbc83148d2982305a04714c2 \ - --hash=sha256:bee093bf902e1d8fc0ac143c88902c3dfc8941f7ea1d6a8dd2bcb786d33db03d \ - --hash=sha256:c72fbbe68c6f32f251bdc08b8611c7b3060612236e960ef848e0a517ddbe76c5 \ - --hash=sha256:c9e36a97bee9b86ef9a1cf7bb96747eb7a15c2f22bdb5b516434b00f2a599f02 \ - --hash=sha256:cddf7bd982eaa998934a91f69d182aec997c6c468898efe6679af88283b498d3 \ - --hash=sha256:cf713fe9a71ef6fd5adf7a79670135081cd4431c2943864757f0fa3a65b1fafd \ - --hash=sha256:d11b54acf878eef558599658b0ffca78138c8c3655cf4f3a4a673c437e67732e \ - --hash=sha256:d41c4d287cfc69060fa91cae9683eacffad989f1a10811995fa309df656ec214 \ - --hash=sha256:d524ba3f1581b35c03cb42beebab4a13e6cdad7b36246bd22541fa585a56cccd \ - --hash=sha256:daac4765328a919a805fa5e2720f3e94767abd632ae410a9062dff5412bae65a \ - --hash=sha256:db4c7bf0e07fc3b7d89ac2a5880a6a8062056801b83ff56d8464b70f65482b6c \ - --hash=sha256:dc7039885fa1baf9be153a0626e337aa7ec8bf96b0128605fb0d77788ddc1681 \ - --hash=sha256:dccab8d5fa1ef9bfba0590ecf4d46df048d18ffe3eec01eeb73a42e0d9e7a8ba \ - --hash=sha256:dedb8adb91d11846ee08bec4c8236c8549ac721c245678282dcb06b221aab59f \ - --hash=sha256:e45ba65510e2647721e35323d6ef54c7974959f6081b58d4ef5d87c60c84919a \ - --hash=sha256:e53efc7c7cee4c1e70661e2e112ca46a575f90ed9ae3fef200f2a25e954f4b28 \ - --hash=sha256:e635b87f01ebc977342e2697d05b56632f5f879a4f15955dfe8cef2448b51691 \ - --hash=sha256:e70e990b2137b29dc5564715de1e12701815dacc1d056308e2b17e9095372a82 \ - --hash=sha256:e8082b26888e2f8b36a042a58307d5b917ef2b1cacab921ad3323ef91901c71a \ - --hash=sha256:e8323a9b031aa0393768b87f04b4164a40037fb2a3c11ac06a03ffecd3618027 \ - --hash=sha256:e92fca20c46e9f5e1bb485887d074918b13543b1c2a1185e69bb8d17ab6236a7 \ - --hash=sha256:eb30abc20df9ab0814b5a2524f23d75dcf83cde762c161917a2b4b7b55b1e518 \ - --hash=sha256:eba9904b0f38a143592d9fc0e19e2df0fa2e41c3c3745554761c5f6447eedabf \ - --hash=sha256:ef8de666d6179b009dce7bcb2ad4c4a779f113f12caf8dc77f0162c29d20490b \ - --hash=sha256:efd387a49825780ff861998cd959767800d54f8308936b21025326de4b5a42b9 \ - --hash=sha256:f0aa37f3c979cf2546b73e8222bbfa3dc07a641585340179d768068e3455e544 \ - --hash=sha256:f4074c5a429281bf056ddd4c5d3b740ebca4d43ffffe2ef4bf4d2d05114299da \ - --hash=sha256:f69a27e45c43520f5487f27627059b64aaf160415589230992cec34c5e18a509 \ - --hash=sha256:fb707f3e15060adf5b7ada797624a6c6e0138e2a26baa089df64c68ee98e040f \ - --hash=sha256:fcbe676a55d7445b22c10967bceaaf0ee69407fbe0ece4d032b6eb8d4565982a \ - --hash=sha256:fdb20a30fe1175ecabed17cbf7812f7b804b8a315a25f24678bcdf120a90077f +charset-normalizer==3.4.3 \ + --hash=sha256:00237675befef519d9af72169d8604a067d92755e84fe76492fef5441db05b91 \ + --hash=sha256:02425242e96bcf29a49711b0ca9f37e451da7c70562bc10e8ed992a5a7a25cc0 \ + --hash=sha256:027b776c26d38b7f15b26a5da1044f376455fb3766df8fc38563b4efbc515154 \ + --hash=sha256:07a0eae9e2787b586e129fdcbe1af6997f8d0e5abaa0bc98c0e20e124d67e601 \ + --hash=sha256:0cacf8f7297b0c4fcb74227692ca46b4a5852f8f4f24b3c766dd94a1075c4884 \ + --hash=sha256:0e78314bdc32fa80696f72fa16dc61168fda4d6a0c014e0380f9d02f0e5d8a07 \ + --hash=sha256:0f2be7e0cf7754b9a30eb01f4295cc3d4358a479843b31f328afd210e2c7598c \ + --hash=sha256:13faeacfe61784e2559e690fc53fa4c5ae97c6fcedb8eb6fb8d0a15b475d2c64 \ + --hash=sha256:14c2a87c65b351109f6abfc424cab3927b3bdece6f706e4d12faaf3d52ee5efe \ + --hash=sha256:1606f4a55c0fd363d754049cdf400175ee96c992b1f8018b993941f221221c5f \ + --hash=sha256:16a8770207946ac75703458e2c743631c79c59c5890c80011d536248f8eaa432 \ + --hash=sha256:18343b2d246dc6761a249ba1fb13f9ee9a2bcd95decc767319506056ea4ad4dc \ + --hash=sha256:18b97b8404387b96cdbd30ad660f6407799126d26a39ca65729162fd810a99aa \ + --hash=sha256:1bb60174149316da1c35fa5233681f7c0f9f514509b8e399ab70fea5f17e45c9 \ + --hash=sha256:1e8ac75d72fa3775e0b7cb7e4629cec13b7514d928d15ef8ea06bca03ef01cae \ + --hash=sha256:1ef99f0456d3d46a50945c98de1774da86f8e992ab5c77865ea8b8195341fc19 \ + --hash=sha256:2001a39612b241dae17b4687898843f254f8748b796a2e16f1051a17078d991d \ + --hash=sha256:23b6b24d74478dc833444cbd927c338349d6ae852ba53a0d02a2de1fce45b96e \ + --hash=sha256:252098c8c7a873e17dd696ed98bbe91dbacd571da4b87df3736768efa7a792e4 \ + --hash=sha256:257f26fed7d7ff59921b78244f3cd93ed2af1800ff048c33f624c87475819dd7 \ + --hash=sha256:2c322db9c8c89009a990ef07c3bcc9f011a3269bc06782f916cd3d9eed7c9312 \ + --hash=sha256:30a96e1e1f865f78b030d65241c1ee850cdf422d869e9028e2fc1d5e4db73b92 \ + --hash=sha256:30d006f98569de3459c2fc1f2acde170b7b2bd265dc1943e87e1a4efe1b67c31 \ + --hash=sha256:31a9a6f775f9bcd865d88ee350f0ffb0e25936a7f930ca98995c05abf1faf21c \ + --hash=sha256:320e8e66157cc4e247d9ddca8e21f427efc7a04bbd0ac8a9faf56583fa543f9f \ + --hash=sha256:34a7f768e3f985abdb42841e20e17b330ad3aaf4bb7e7aeeb73db2e70f077b99 \ + --hash=sha256:3653fad4fe3ed447a596ae8638b437f827234f01a8cd801842e43f3d0a6b281b \ + --hash=sha256:3cd35b7e8aedeb9e34c41385fda4f73ba609e561faedfae0a9e75e44ac558a15 \ + --hash=sha256:3cfb2aad70f2c6debfbcb717f23b7eb55febc0bb23dcffc0f076009da10c6392 \ + --hash=sha256:416175faf02e4b0810f1f38bcb54682878a4af94059a1cd63b8747244420801f \ + --hash=sha256:41d1fc408ff5fdfb910200ec0e74abc40387bccb3252f3f27c0676731df2b2c8 \ + --hash=sha256:42e5088973e56e31e4fa58eb6bd709e42fc03799c11c42929592889a2e54c491 \ + --hash=sha256:4ca4c094de7771a98d7fbd67d9e5dbf1eb73efa4f744a730437d8a3a5cf994f0 \ + --hash=sha256:511729f456829ef86ac41ca78c63a5cb55240ed23b4b737faca0eb1abb1c41bc \ + --hash=sha256:53cd68b185d98dde4ad8990e56a58dea83a4162161b1ea9272e5c9182ce415e0 \ + --hash=sha256:585f3b2a80fbd26b048a0be90c5aae8f06605d3c92615911c3a2b03a8a3b796f \ + --hash=sha256:5b413b0b1bfd94dbf4023ad6945889f374cd24e3f62de58d6bb102c4d9ae534a \ + --hash=sha256:5d8d01eac18c423815ed4f4a2ec3b439d654e55ee4ad610e153cf02faf67ea40 \ + --hash=sha256:6aab0f181c486f973bc7262a97f5aca3ee7e1437011ef0c2ec04b5a11d16c927 \ + --hash=sha256:6cf8fd4c04756b6b60146d98cd8a77d0cdae0e1ca20329da2ac85eed779b6849 \ + --hash=sha256:6fb70de56f1859a3f71261cbe41005f56a7842cc348d3aeb26237560bfa5e0ce \ + --hash=sha256:6fce4b8500244f6fcb71465d4a4930d132ba9ab8e71a7859e6a5d59851068d14 \ + --hash=sha256:70bfc5f2c318afece2f5838ea5e4c3febada0be750fcf4775641052bbba14d05 \ + --hash=sha256:73dc19b562516fc9bcf6e5d6e596df0b4eb98d87e4f79f3ae71840e6ed21361c \ + --hash=sha256:74d77e25adda8581ffc1c720f1c81ca082921329452eba58b16233ab1842141c \ + --hash=sha256:78deba4d8f9590fe4dae384aeff04082510a709957e968753ff3c48399f6f92a \ + --hash=sha256:86df271bf921c2ee3818f0522e9a5b8092ca2ad8b065ece5d7d9d0e9f4849bcc \ + --hash=sha256:88ab34806dea0671532d3f82d82b85e8fc23d7b2dd12fa837978dad9bb392a34 \ + --hash=sha256:8999f965f922ae054125286faf9f11bc6932184b93011d138925a1773830bbe9 \ + --hash=sha256:8dcfc373f888e4fb39a7bc57e93e3b845e7f462dacc008d9749568b1c4ece096 \ + --hash=sha256:939578d9d8fd4299220161fdd76e86c6a251987476f5243e8864a7844476ba14 \ + --hash=sha256:96b2b3d1a83ad55310de8c7b4a2d04d9277d5591f40761274856635acc5fcb30 \ + --hash=sha256:a2d08ac246bb48479170408d6c19f6385fa743e7157d716e144cad849b2dd94b \ + --hash=sha256:b256ee2e749283ef3ddcff51a675ff43798d92d746d1a6e4631bf8c707d22d0b \ + --hash=sha256:b5e3b2d152e74e100a9e9573837aba24aab611d39428ded46f4e4022ea7d1942 \ + --hash=sha256:b89bc04de1d83006373429975f8ef9e7932534b8cc9ca582e4db7d20d91816db \ + --hash=sha256:bd28b817ea8c70215401f657edef3a8aa83c29d447fb0b622c35403780ba11d5 \ + --hash=sha256:c60e092517a73c632ec38e290eba714e9627abe9d301c8c8a12ec32c314a2a4b \ + --hash=sha256:c6dbd0ccdda3a2ba7c2ecd9d77b37f3b5831687d8dc1b6ca5f56a4880cc7b7ce \ + --hash=sha256:c6e490913a46fa054e03699c70019ab869e990270597018cef1d8562132c2669 \ + --hash=sha256:c6f162aabe9a91a309510d74eeb6507fab5fff92337a15acbe77753d88d9dcf0 \ + --hash=sha256:c6fd51128a41297f5409deab284fecbe5305ebd7e5a1f959bee1c054622b7018 \ + --hash=sha256:cc34f233c9e71701040d772aa7490318673aa7164a0efe3172b2981218c26d93 \ + --hash=sha256:cc9370a2da1ac13f0153780040f465839e6cccb4a1e44810124b4e22483c93fe \ + --hash=sha256:ccf600859c183d70eb47e05a44cd80a4ce77394d1ac0f79dbd2dd90a69a3a049 \ + --hash=sha256:ce571ab16d890d23b5c278547ba694193a45011ff86a9162a71307ed9f86759a \ + --hash=sha256:cf1ebb7d78e1ad8ec2a8c4732c7be2e736f6e5123a4146c5b89c9d1f585f8cef \ + --hash=sha256:d0e909868420b7049dafd3a31d45125b31143eec59235311fc4c57ea26a4acd2 \ + --hash=sha256:d22dbedd33326a4a5190dd4fe9e9e693ef12160c77382d9e87919bce54f3d4ca \ + --hash=sha256:d716a916938e03231e86e43782ca7878fb602a125a91e7acb8b5112e2e96ac16 \ + --hash=sha256:d79c198e27580c8e958906f803e63cddb77653731be08851c7df0b1a14a8fc0f \ + --hash=sha256:d95bfb53c211b57198bb91c46dd5a2d8018b3af446583aab40074bf7988401cb \ + --hash=sha256:e28e334d3ff134e88989d90ba04b47d84382a828c061d0d1027b1b12a62b39b1 \ + --hash=sha256:ec557499516fc90fd374bf2e32349a2887a876fbf162c160e3c01b6849eaf557 \ + --hash=sha256:fb6fecfd65564f208cbf0fba07f107fb661bcd1a7c389edbced3f7a493f70e37 \ + --hash=sha256:fb731e5deb0c7ef82d698b0f4c5bb724633ee2a489401594c5c88b02e6cb15f7 \ + --hash=sha256:fb7f67a1bfa6e40b438170ebdc8158b78dc465a5a67b6dde178a46987b244a72 \ + --hash=sha256:fd10de089bcdcd1be95a2f73dbe6254798ec1bda9f450d5828c96f93e2536b9c \ + --hash=sha256:fdabf8315679312cfa71302f9bd509ded4f2f263fb5b765cf1433b39106c3cc9 # via requests docutils==0.22 \ --hash=sha256:4ed966a0e96a0477d852f7af31bdcb3adc049fbb35ccba358c2ea8a03287615e \ From 6610fd742ae804cf2d8374b98d5fc4a9d949d9bb Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Tue, 2 Sep 2025 17:03:34 -0700 Subject: [PATCH 414/922] chore: allow release workflow to be manually run and skip pypi upload (#3232) This makes it possible to manually invoke the release workflow and skip the pypi upload. This is useful if the release workflow was cancelled (or failed) after the pypi upload step. Work towards https://github.com/bazel-contrib/rules_python/issues/3188 --- .github/workflows/release.yml | 8 ++++++++ RELEASING.md | 19 +++++++++++++++++++ 2 files changed, 27 insertions(+) diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index e13ab97fb6..7a25c6eca0 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -19,6 +19,13 @@ on: push: tags: - "*.*.*" + workflow_dispatch: + inputs: + publish_to_pypi: + description: 'Publish to PyPI' + required: true + type: boolean + default: true jobs: build: @@ -29,6 +36,7 @@ jobs: - name: Create release archive and notes run: .github/workflows/create_archive_and_notes.sh - name: Publish wheel dist + if: github.event_name == 'push' || github.event.inputs.publish_to_pypi env: # This special value tells pypi that the user identity is supplied within the token TWINE_USERNAME: __token__ diff --git a/RELEASING.md b/RELEASING.md index e72ff619ba..3d58a9339e 100644 --- a/RELEASING.md +++ b/RELEASING.md @@ -45,6 +45,25 @@ final release (`X.Y.Z`). Release automation will create a GitHub release and BCR pull request. +### Manually triggering the release workflow + +The release workflow can be manually triggered using the GitHub CLI (`gh`). +This is useful for re-running a release or for creating a release from a +specific commit. + +To trigger the workflow, use the `gh workflow run` command: + +```shell +gh workflow run release.yml --ref +``` + +By default, the workflow will publish the wheel to PyPI. To skip this step, +you can set the `publish_to_pypi` input to `false`: + +```shell +gh workflow run release.yml --ref -f publish_to_pypi=false +``` + ### Determining Semantic Version **rules_python** uses [semantic version](https://semver.org), so releases with From 1e1748684e98042a3d52c6d49802c53a17cd3246 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Tue, 2 Sep 2025 17:04:59 -0700 Subject: [PATCH 415/922] chore: make release tool auto detect next version (#3219) This makes the release tool determine the next version automatically. It does so by searching for the VERSION_NEXT strings. If VERSION_NEXT_FEATURE is found, then it increments the minor version. If only patch placeholders are found, then it increments the patch version. When the latest version is an RC, an error is raised. This is to protect against accidentally running it when we're in the middle of the RC phase. --------- Co-authored-by: Ignas Anikevicius <240938+aignas@users.noreply.github.com> --- RELEASING.md | 11 +- tests/tools/private/release/BUILD.bazel | 5 +- tests/tools/private/release/release_test.py | 40 +++++ tools/private/release/BUILD.bazel | 3 + tools/private/release/release.py | 164 ++++++++++++++------ 5 files changed, 171 insertions(+), 52 deletions(-) diff --git a/RELEASING.md b/RELEASING.md index 3d58a9339e..e4cf738f3d 100644 --- a/RELEASING.md +++ b/RELEASING.md @@ -12,12 +12,14 @@ These are the steps for a regularly scheduled release from HEAD. ### Steps -1. [Determine the next semantic version number](#determining-semantic-version). 1. Update the changelog and replace the version placeholders by running the - release tool: + release tool. The next version number will by automatically determined + based on the presence of `VERSION_NEXT_*` placeholders and git tags. + ```shell - bazel run //tools/private/release -- X.Y.Z + bazel run //tools/private/release ``` + 1. Send these changes for review and get them merged. 1. Create a branch for the new release, named `release/X.Y` ``` @@ -70,7 +72,8 @@ gh workflow run release.yml --ref -f publish_to_pypi=false API changes and new features bump the minor, and those with only bug fixes and other minor changes bump the patch digit. -To find if there were any features added or incompatible changes made, review +The release tool will automatically determine the next version number. To find +if there were any features added or incompatible changes made, review [CHANGELOG.md](CHANGELOG.md) and the commit history. This can be done using github by going to the url: `https://github.com/bazel-contrib/rules_python/compare/...main`. diff --git a/tests/tools/private/release/BUILD.bazel b/tests/tools/private/release/BUILD.bazel index 3c9db2d4e9..9f3bc0542a 100644 --- a/tests/tools/private/release/BUILD.bazel +++ b/tests/tools/private/release/BUILD.bazel @@ -3,5 +3,8 @@ load("@rules_python//python:defs.bzl", "py_test") py_test( name = "release_test", srcs = ["release_test.py"], - deps = ["//tools/private/release"], + deps = [ + "//tools/private/release", + "@dev_pip//packaging", + ], ) diff --git a/tests/tools/private/release/release_test.py b/tests/tools/private/release/release_test.py index 5f0446410b..72a9a05cd6 100644 --- a/tests/tools/private/release/release_test.py +++ b/tests/tools/private/release/release_test.py @@ -4,6 +4,7 @@ import shutil import tempfile import unittest +from unittest.mock import patch from tools.private.release import release as releaser @@ -170,5 +171,44 @@ def test_invalid_version(self): releaser.create_parser().parse_args(["a.b.c"]) +class GetLatestVersionTest(unittest.TestCase): + @patch("tools.private.release.release._get_git_tags") + def test_get_latest_version_success(self, mock_get_tags): + mock_get_tags.return_value = ["0.1.0", "1.0.0", "0.2.0"] + self.assertEqual(releaser.get_latest_version(), "1.0.0") + + @patch("tools.private.release.release._get_git_tags") + def test_get_latest_version_rc_is_latest(self, mock_get_tags): + mock_get_tags.return_value = ["0.1.0", "1.0.0", "1.1.0rc0"] + with self.assertRaisesRegex( + ValueError, "The latest version is a pre-release version: 1.1.0rc0" + ): + releaser.get_latest_version() + + @patch("tools.private.release.release._get_git_tags") + def test_get_latest_version_no_tags(self, mock_get_tags): + mock_get_tags.return_value = [] + with self.assertRaisesRegex( + RuntimeError, "No git tags found matching X.Y.Z or X.Y.ZrcN format." + ): + releaser.get_latest_version() + + @patch("tools.private.release.release._get_git_tags") + def test_get_latest_version_no_matching_tags(self, mock_get_tags): + mock_get_tags.return_value = ["v1.0", "latest"] + with self.assertRaisesRegex( + RuntimeError, "No git tags found matching X.Y.Z or X.Y.ZrcN format." + ): + releaser.get_latest_version() + + @patch("tools.private.release.release._get_git_tags") + def test_get_latest_version_only_rc_tags(self, mock_get_tags): + mock_get_tags.return_value = ["1.0.0rc0", "1.1.0rc0"] + with self.assertRaisesRegex( + ValueError, "The latest version is a pre-release version: 1.1.0rc0" + ): + releaser.get_latest_version() + + if __name__ == "__main__": unittest.main() diff --git a/tools/private/release/BUILD.bazel b/tools/private/release/BUILD.bazel index 9cd8ec2fba..31cc3a0239 100644 --- a/tools/private/release/BUILD.bazel +++ b/tools/private/release/BUILD.bazel @@ -6,4 +6,7 @@ py_binary( name = "release", srcs = ["release.py"], main = "release.py", + deps = [ + "@dev_pip//packaging", + ], ) diff --git a/tools/private/release/release.py b/tools/private/release/release.py index f37a5ff7de..def6754347 100644 --- a/tools/private/release/release.py +++ b/tools/private/release/release.py @@ -6,6 +6,100 @@ import os import pathlib import re +import subprocess + +from packaging.version import parse as parse_version + +_EXCLUDE_PATTERNS = [ + "./.git/*", + "./.github/*", + "./.bazelci/*", + "./.bcr/*", + "./bazel-*/*", + "./CONTRIBUTING.md", + "./RELEASING.md", + "./tools/private/release/*", + "./tests/tools/private/release/*", +] + + +def _iter_version_placeholder_files(): + for root, dirs, files in os.walk(".", topdown=True): + # Filter directories + dirs[:] = [ + d + for d in dirs + if not any( + fnmatch.fnmatch(os.path.join(root, d), pattern) + for pattern in _EXCLUDE_PATTERNS + ) + ] + + for filename in files: + filepath = os.path.join(root, filename) + if any(fnmatch.fnmatch(filepath, pattern) for pattern in _EXCLUDE_PATTERNS): + continue + + yield filepath + + +def _get_git_tags(): + """Runs a git command and returns the output.""" + return subprocess.check_output(["git", "tag"]).decode("utf-8").splitlines() + + +def get_latest_version(): + """Gets the latest version from git tags.""" + tags = _get_git_tags() + # The packaging module can parse PEP440 versions, including RCs. + # It has a good understanding of version precedence. + versions = [ + (tag, parse_version(tag)) + for tag in tags + if re.match(r"^\d+\.\d+\.\d+(rc\d+)?$", tag.strip()) + ] + if not versions: + raise RuntimeError("No git tags found matching X.Y.Z or X.Y.ZrcN format.") + + versions.sort(key=lambda v: v[1]) + latest_tag, latest_version = versions[-1] + + if latest_version.is_prerelease: + raise ValueError(f"The latest version is a pre-release version: {latest_tag}") + + # After all that, we only want to consider stable versions for the release. + stable_versions = [tag for tag, version in versions if not version.is_prerelease] + if not stable_versions: + raise ValueError("No stable git tags found matching X.Y.Z format.") + + # The versions are already sorted, so the last one is the latest. + return stable_versions[-1] + + +def should_increment_minor(): + """Checks if the minor version should be incremented.""" + for filepath in _iter_version_placeholder_files(): + try: + with open(filepath, "r") as f: + content = f.read() + except (IOError, UnicodeDecodeError): + # Ignore binary files or files with read errors + continue + + if "VERSION_NEXT_FEATURE" in content: + return True + return False + + +def determine_next_version(): + """Determines the next version based on git tags and placeholders.""" + latest_version = get_latest_version() + major, minor, patch = [int(n) for n in latest_version.split(".")] + + if should_increment_minor(): + return f"{major}.{minor + 1}.0" + else: + return f"{major}.{minor}.{patch + 1}" def update_changelog(version, release_date, changelog_path="CHANGELOG.md"): @@ -37,46 +131,19 @@ def update_changelog(version, release_date, changelog_path="CHANGELOG.md"): def replace_version_next(version): """Replaces all VERSION_NEXT_* placeholders with the new version.""" - exclude_patterns = [ - "./.git/*", - "./.github/*", - "./.bazelci/*", - "./.bcr/*", - "./bazel-*/*", - "./CONTRIBUTING.md", - "./RELEASING.md", - "./tools/private/release/*", - "./tests/tools/private/release/*", - ] + for filepath in _iter_version_placeholder_files(): + try: + with open(filepath, "r") as f: + content = f.read() + except (IOError, UnicodeDecodeError): + # Ignore binary files or files with read errors + continue - for root, dirs, files in os.walk(".", topdown=True): - # Filter directories - dirs[:] = [ - d - for d in dirs - if not any( - fnmatch.fnmatch(os.path.join(root, d), pattern) - for pattern in exclude_patterns - ) - ] - - for filename in files: - filepath = os.path.join(root, filename) - if any(fnmatch.fnmatch(filepath, pattern) for pattern in exclude_patterns): - continue - - try: - with open(filepath, "r") as f: - content = f.read() - except (IOError, UnicodeDecodeError): - # Ignore binary files or files with read errors - continue - - if "VERSION_NEXT_FEATURE" in content or "VERSION_NEXT_PATCH" in content: - new_content = content.replace("VERSION_NEXT_FEATURE", version) - new_content = new_content.replace("VERSION_NEXT_PATCH", version) - with open(filepath, "w") as f: - f.write(new_content) + if "VERSION_NEXT_FEATURE" in content or "VERSION_NEXT_PATCH" in content: + new_content = content.replace("VERSION_NEXT_FEATURE", version) + new_content = new_content.replace("VERSION_NEXT_PATCH", version) + with open(filepath, "w") as f: + f.write(new_content) def _semver_type(value): @@ -94,8 +161,10 @@ def create_parser(): ) parser.add_argument( "version", - help="The new release version (e.g., 0.28.0).", + nargs="?", type=_semver_type, + help="The new release version (e.g., 0.28.0). If not provided, " + "it will be determined automatically.", ) return parser @@ -104,21 +173,22 @@ def main(): parser = create_parser() args = parser.parse_args() - if not re.match(r"^\d+\.\d+\.\d+(rc\d+)?$", args.version): - raise ValueError( - f"Version '{args.version}' is not a valid semantic version (X.Y.Z or X.Y.ZrcN)" - ) + version = args.version + if version is None: + print("No version provided, determining next version automatically...") + version = determine_next_version() + print(f"Determined next version: {version}") - # Change to the workspace root so the script can be run from anywhere. + # Change to the workspace root so the script can be run using `bazel run` if "BUILD_WORKSPACE_DIRECTORY" in os.environ: os.chdir(os.environ["BUILD_WORKSPACE_DIRECTORY"]) print("Updating changelog ...") release_date = datetime.date.today().strftime("%Y-%m-%d") - update_changelog(args.version, release_date) + update_changelog(version, release_date) print("Replacing VERSION_NEXT placeholders ...") - replace_version_next(args.version) + replace_version_next(version) print("Done") From 764712cd6d9b640cd048b5b85f9c479d3bdce230 Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Wed, 3 Sep 2025 08:40:27 -0700 Subject: [PATCH 416/922] build(deps): bump cryptography from 44.0.1 to 45.0.7 in /tools/publish (#3235) Bumps [cryptography](https://github.com/pyca/cryptography) from 44.0.1 to 45.0.7.
Changelog

Sourced from cryptography's changelog.

45.0.7 - 2025-09-01


* Added a function to support an upcoming ``pyOpenSSL`` release.

.. _v45-0-6:

45.0.6 - 2025-08-05

  • Updated Windows, macOS, and Linux wheels to be compiled with OpenSSL 3.5.2.

.. _v45-0-5:

45.0.5 - 2025-07-02


* Updated Windows, macOS, and Linux wheels to be compiled with OpenSSL
3.5.1.

.. _v45-0-4:

45.0.4 - 2025-06-09

  • Fixed decrypting PKCS#8 files encrypted with SHA1-RC4. (This is not considered secure, and is supported only for backwards compatibility.)

.. _v45-0-3:

45.0.3 - 2025-05-25


* Fixed decrypting PKCS#8 files encrypted with long salts (this impacts
keys
  encrypted by Bouncy Castle).
* Fixed decrypting PKCS#8 files encrypted with DES-CBC-MD5. While wildly
  insecure, this remains prevalent.

.. _v45-0-2:

45.0.2 - 2025-05-17

  • Fixed using mypy with cryptography on older versions of Python.

.. _v45-0-1:

45.0.1 - 2025-05-17


* Updated Windows, macOS, and Linux wheels to be compiled with OpenSSL
3.5.0.
</tr></table>

... (truncated)

Commits

[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=cryptography&package-manager=pip&previous-version=44.0.1&new-version=45.0.7)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) ---
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot show ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> --- tools/publish/requirements_linux.txt | 70 +++++++++++++----------- tools/publish/requirements_universal.txt | 70 +++++++++++++----------- 2 files changed, 76 insertions(+), 64 deletions(-) diff --git a/tools/publish/requirements_linux.txt b/tools/publish/requirements_linux.txt index f8a065606c..7e3d42f518 100644 --- a/tools/publish/requirements_linux.txt +++ b/tools/publish/requirements_linux.txt @@ -160,38 +160,44 @@ charset-normalizer==3.4.3 \ --hash=sha256:fd10de089bcdcd1be95a2f73dbe6254798ec1bda9f450d5828c96f93e2536b9c \ --hash=sha256:fdabf8315679312cfa71302f9bd509ded4f2f263fb5b765cf1433b39106c3cc9 # via requests -cryptography==44.0.1 \ - --hash=sha256:00918d859aa4e57db8299607086f793fa7813ae2ff5a4637e318a25ef82730f7 \ - --hash=sha256:1e8d181e90a777b63f3f0caa836844a1182f1f265687fac2115fcf245f5fbec3 \ - --hash=sha256:1f9a92144fa0c877117e9748c74501bea842f93d21ee00b0cf922846d9d0b183 \ - --hash=sha256:21377472ca4ada2906bc313168c9dc7b1d7ca417b63c1c3011d0c74b7de9ae69 \ - --hash=sha256:24979e9f2040c953a94bf3c6782e67795a4c260734e5264dceea65c8f4bae64a \ - --hash=sha256:2a46a89ad3e6176223b632056f321bc7de36b9f9b93b2cc1cccf935a3849dc62 \ - --hash=sha256:322eb03ecc62784536bc173f1483e76747aafeb69c8728df48537eb431cd1911 \ - --hash=sha256:436df4f203482f41aad60ed1813811ac4ab102765ecae7a2bbb1dbb66dcff5a7 \ - --hash=sha256:4f422e8c6a28cf8b7f883eb790695d6d45b0c385a2583073f3cec434cc705e1a \ - --hash=sha256:53f23339864b617a3dfc2b0ac8d5c432625c80014c25caac9082314e9de56f41 \ - --hash=sha256:5fed5cd6102bb4eb843e3315d2bf25fede494509bddadb81e03a859c1bc17b83 \ - --hash=sha256:610a83540765a8d8ce0f351ce42e26e53e1f774a6efb71eb1b41eb01d01c3d12 \ - --hash=sha256:6c8acf6f3d1f47acb2248ec3ea261171a671f3d9428e34ad0357148d492c7864 \ - --hash=sha256:6f76fdd6fd048576a04c5210d53aa04ca34d2ed63336d4abd306d0cbe298fddf \ - --hash=sha256:72198e2b5925155497a5a3e8c216c7fb3e64c16ccee11f0e7da272fa93b35c4c \ - --hash=sha256:887143b9ff6bad2b7570da75a7fe8bbf5f65276365ac259a5d2d5147a73775f2 \ - --hash=sha256:888fcc3fce0c888785a4876ca55f9f43787f4c5c1cc1e2e0da71ad481ff82c5b \ - --hash=sha256:8e6a85a93d0642bd774460a86513c5d9d80b5c002ca9693e63f6e540f1815ed0 \ - --hash=sha256:94f99f2b943b354a5b6307d7e8d19f5c423a794462bde2bf310c770ba052b1c4 \ - --hash=sha256:9b336599e2cb77b1008cb2ac264b290803ec5e8e89d618a5e978ff5eb6f715d9 \ - --hash=sha256:a2d8a7045e1ab9b9f803f0d9531ead85f90c5f2859e653b61497228b18452008 \ - --hash=sha256:b8272f257cf1cbd3f2e120f14c68bff2b6bdfcc157fafdee84a1b795efd72862 \ - --hash=sha256:bf688f615c29bfe9dfc44312ca470989279f0e94bb9f631f85e3459af8efc009 \ - --hash=sha256:d9c5b9f698a83c8bd71e0f4d3f9f839ef244798e5ffe96febfa9714717db7af7 \ - --hash=sha256:dd7c7e2d71d908dc0f8d2027e1604102140d84b155e658c20e8ad1304317691f \ - --hash=sha256:df978682c1504fc93b3209de21aeabf2375cb1571d4e61907b3e7a2540e83026 \ - --hash=sha256:e403f7f766ded778ecdb790da786b418a9f2394f36e8cc8b796cc056ab05f44f \ - --hash=sha256:eb3889330f2a4a148abead555399ec9a32b13b7c8ba969b72d8e500eb7ef84cd \ - --hash=sha256:f4daefc971c2d1f82f03097dc6f216744a6cd2ac0f04c68fb935ea2ba2a0d420 \ - --hash=sha256:f51f5705ab27898afda1aaa430f34ad90dc117421057782022edf0600bec5f14 \ - --hash=sha256:fd0ee90072861e276b0ff08bd627abec29e32a53b2be44e41dbcdf87cbee2b00 +cryptography==45.0.7 \ + --hash=sha256:06ce84dc14df0bf6ea84666f958e6080cdb6fe1231be2a51f3fc1267d9f3fb34 \ + --hash=sha256:16ede8a4f7929b4b7ff3642eba2bf79aa1d71f24ab6ee443935c0d269b6bc513 \ + --hash=sha256:18fcf70f243fe07252dcb1b268a687f2358025ce32f9f88028ca5c364b123ef5 \ + --hash=sha256:1993a1bb7e4eccfb922b6cd414f072e08ff5816702a0bdb8941c247a6b1b287c \ + --hash=sha256:1f3d56f73595376f4244646dd5c5870c14c196949807be39e79e7bd9bac3da63 \ + --hash=sha256:258e0dff86d1d891169b5af222d362468a9570e2532923088658aa866eb11130 \ + --hash=sha256:2f641b64acc00811da98df63df7d59fd4706c0df449da71cb7ac39a0732b40ae \ + --hash=sha256:3808e6b2e5f0b46d981c24d79648e5c25c35e59902ea4391a0dcb3e667bf7443 \ + --hash=sha256:3994c809c17fc570c2af12c9b840d7cea85a9fd3e5c0e0491f4fa3c029216d59 \ + --hash=sha256:3be4f21c6245930688bd9e162829480de027f8bf962ede33d4f8ba7d67a00cee \ + --hash=sha256:465ccac9d70115cd4de7186e60cfe989de73f7bb23e8a7aa45af18f7412e75bf \ + --hash=sha256:48c41a44ef8b8c2e80ca4527ee81daa4c527df3ecbc9423c41a420a9559d0e27 \ + --hash=sha256:4a862753b36620af6fc54209264f92c716367f2f0ff4624952276a6bbd18cbde \ + --hash=sha256:4b1654dfc64ea479c242508eb8c724044f1e964a47d1d1cacc5132292d851971 \ + --hash=sha256:4bd3e5c4b9682bc112d634f2c6ccc6736ed3635fc3319ac2bb11d768cc5a00d8 \ + --hash=sha256:577470e39e60a6cd7780793202e63536026d9b8641de011ed9d8174da9ca5339 \ + --hash=sha256:67285f8a611b0ebc0857ced2081e30302909f571a46bfa7a3cc0ad303fe015c6 \ + --hash=sha256:7285a89df4900ed3bfaad5679b1e668cb4b38a8de1ccbfc84b05f34512da0a90 \ + --hash=sha256:81823935e2f8d476707e85a78a405953a03ef7b7b4f55f93f7c2d9680e5e0691 \ + --hash=sha256:8978132287a9d3ad6b54fcd1e08548033cc09dc6aacacb6c004c73c3eb5d3ac3 \ + --hash=sha256:a20e442e917889d1a6b3c570c9e3fa2fdc398c20868abcea268ea33c024c4083 \ + --hash=sha256:a24ee598d10befaec178efdff6054bc4d7e883f615bfbcd08126a0f4931c83a6 \ + --hash=sha256:b04f85ac3a90c227b6e5890acb0edbaf3140938dbecf07bff618bf3638578cf1 \ + --hash=sha256:b6a0e535baec27b528cb07a119f321ac024592388c5681a5ced167ae98e9fff3 \ + --hash=sha256:bef32a5e327bd8e5af915d3416ffefdbe65ed975b646b3805be81b23580b57b8 \ + --hash=sha256:bfb4c801f65dd61cedfc61a83732327fafbac55a47282e6f26f073ca7a41c3b2 \ + --hash=sha256:c13b1e3afd29a5b3b2656257f14669ca8fa8d7956d509926f0b130b600b50ab7 \ + --hash=sha256:c987dad82e8c65ebc985f5dae5e74a3beda9d0a2a4daf8a1115f3772b59e5141 \ + --hash=sha256:ce7a453385e4c4693985b4a4a3533e041558851eae061a58a5405363b098fcd3 \ + --hash=sha256:d0c5c6bac22b177bf8da7435d9d27a6834ee130309749d162b26c3105c0795a9 \ + --hash=sha256:d97cf502abe2ab9eff8bd5e4aca274da8d06dd3ef08b759a8d6143f4ad65d4b4 \ + --hash=sha256:dad43797959a74103cb59c5dac71409f9c27d34c8a05921341fb64ea8ccb1dd4 \ + --hash=sha256:dd342f085542f6eb894ca00ef70236ea46070c8a13824c6bde0dfdcd36065b9b \ + --hash=sha256:de58755d723e86175756f463f2f0bddd45cc36fbd62601228a3f8761c9f58252 \ + --hash=sha256:f3df7b3d0f91b88b2106031fd995802a2e9ae13e02c36c1fc075b43f420f3a17 \ + --hash=sha256:f5414a788ecc6ee6bc58560e85ca624258a55ca434884445440a810796ea0e0b \ + --hash=sha256:fa26fa54c0a9384c27fcdc905a2fb7d60ac6e47d14bc2692145f2b3b1e2cfdbd # via secretstorage docutils==0.22 \ --hash=sha256:4ed966a0e96a0477d852f7af31bdcb3adc049fbb35ccba358c2ea8a03287615e \ diff --git a/tools/publish/requirements_universal.txt b/tools/publish/requirements_universal.txt index 7d6b37c955..c3217299c9 100644 --- a/tools/publish/requirements_universal.txt +++ b/tools/publish/requirements_universal.txt @@ -160,38 +160,44 @@ charset-normalizer==3.4.3 \ --hash=sha256:fd10de089bcdcd1be95a2f73dbe6254798ec1bda9f450d5828c96f93e2536b9c \ --hash=sha256:fdabf8315679312cfa71302f9bd509ded4f2f263fb5b765cf1433b39106c3cc9 # via requests -cryptography==44.0.1 ; sys_platform == 'linux' \ - --hash=sha256:00918d859aa4e57db8299607086f793fa7813ae2ff5a4637e318a25ef82730f7 \ - --hash=sha256:1e8d181e90a777b63f3f0caa836844a1182f1f265687fac2115fcf245f5fbec3 \ - --hash=sha256:1f9a92144fa0c877117e9748c74501bea842f93d21ee00b0cf922846d9d0b183 \ - --hash=sha256:21377472ca4ada2906bc313168c9dc7b1d7ca417b63c1c3011d0c74b7de9ae69 \ - --hash=sha256:24979e9f2040c953a94bf3c6782e67795a4c260734e5264dceea65c8f4bae64a \ - --hash=sha256:2a46a89ad3e6176223b632056f321bc7de36b9f9b93b2cc1cccf935a3849dc62 \ - --hash=sha256:322eb03ecc62784536bc173f1483e76747aafeb69c8728df48537eb431cd1911 \ - --hash=sha256:436df4f203482f41aad60ed1813811ac4ab102765ecae7a2bbb1dbb66dcff5a7 \ - --hash=sha256:4f422e8c6a28cf8b7f883eb790695d6d45b0c385a2583073f3cec434cc705e1a \ - --hash=sha256:53f23339864b617a3dfc2b0ac8d5c432625c80014c25caac9082314e9de56f41 \ - --hash=sha256:5fed5cd6102bb4eb843e3315d2bf25fede494509bddadb81e03a859c1bc17b83 \ - --hash=sha256:610a83540765a8d8ce0f351ce42e26e53e1f774a6efb71eb1b41eb01d01c3d12 \ - --hash=sha256:6c8acf6f3d1f47acb2248ec3ea261171a671f3d9428e34ad0357148d492c7864 \ - --hash=sha256:6f76fdd6fd048576a04c5210d53aa04ca34d2ed63336d4abd306d0cbe298fddf \ - --hash=sha256:72198e2b5925155497a5a3e8c216c7fb3e64c16ccee11f0e7da272fa93b35c4c \ - --hash=sha256:887143b9ff6bad2b7570da75a7fe8bbf5f65276365ac259a5d2d5147a73775f2 \ - --hash=sha256:888fcc3fce0c888785a4876ca55f9f43787f4c5c1cc1e2e0da71ad481ff82c5b \ - --hash=sha256:8e6a85a93d0642bd774460a86513c5d9d80b5c002ca9693e63f6e540f1815ed0 \ - --hash=sha256:94f99f2b943b354a5b6307d7e8d19f5c423a794462bde2bf310c770ba052b1c4 \ - --hash=sha256:9b336599e2cb77b1008cb2ac264b290803ec5e8e89d618a5e978ff5eb6f715d9 \ - --hash=sha256:a2d8a7045e1ab9b9f803f0d9531ead85f90c5f2859e653b61497228b18452008 \ - --hash=sha256:b8272f257cf1cbd3f2e120f14c68bff2b6bdfcc157fafdee84a1b795efd72862 \ - --hash=sha256:bf688f615c29bfe9dfc44312ca470989279f0e94bb9f631f85e3459af8efc009 \ - --hash=sha256:d9c5b9f698a83c8bd71e0f4d3f9f839ef244798e5ffe96febfa9714717db7af7 \ - --hash=sha256:dd7c7e2d71d908dc0f8d2027e1604102140d84b155e658c20e8ad1304317691f \ - --hash=sha256:df978682c1504fc93b3209de21aeabf2375cb1571d4e61907b3e7a2540e83026 \ - --hash=sha256:e403f7f766ded778ecdb790da786b418a9f2394f36e8cc8b796cc056ab05f44f \ - --hash=sha256:eb3889330f2a4a148abead555399ec9a32b13b7c8ba969b72d8e500eb7ef84cd \ - --hash=sha256:f4daefc971c2d1f82f03097dc6f216744a6cd2ac0f04c68fb935ea2ba2a0d420 \ - --hash=sha256:f51f5705ab27898afda1aaa430f34ad90dc117421057782022edf0600bec5f14 \ - --hash=sha256:fd0ee90072861e276b0ff08bd627abec29e32a53b2be44e41dbcdf87cbee2b00 +cryptography==45.0.7 ; sys_platform == 'linux' \ + --hash=sha256:06ce84dc14df0bf6ea84666f958e6080cdb6fe1231be2a51f3fc1267d9f3fb34 \ + --hash=sha256:16ede8a4f7929b4b7ff3642eba2bf79aa1d71f24ab6ee443935c0d269b6bc513 \ + --hash=sha256:18fcf70f243fe07252dcb1b268a687f2358025ce32f9f88028ca5c364b123ef5 \ + --hash=sha256:1993a1bb7e4eccfb922b6cd414f072e08ff5816702a0bdb8941c247a6b1b287c \ + --hash=sha256:1f3d56f73595376f4244646dd5c5870c14c196949807be39e79e7bd9bac3da63 \ + --hash=sha256:258e0dff86d1d891169b5af222d362468a9570e2532923088658aa866eb11130 \ + --hash=sha256:2f641b64acc00811da98df63df7d59fd4706c0df449da71cb7ac39a0732b40ae \ + --hash=sha256:3808e6b2e5f0b46d981c24d79648e5c25c35e59902ea4391a0dcb3e667bf7443 \ + --hash=sha256:3994c809c17fc570c2af12c9b840d7cea85a9fd3e5c0e0491f4fa3c029216d59 \ + --hash=sha256:3be4f21c6245930688bd9e162829480de027f8bf962ede33d4f8ba7d67a00cee \ + --hash=sha256:465ccac9d70115cd4de7186e60cfe989de73f7bb23e8a7aa45af18f7412e75bf \ + --hash=sha256:48c41a44ef8b8c2e80ca4527ee81daa4c527df3ecbc9423c41a420a9559d0e27 \ + --hash=sha256:4a862753b36620af6fc54209264f92c716367f2f0ff4624952276a6bbd18cbde \ + --hash=sha256:4b1654dfc64ea479c242508eb8c724044f1e964a47d1d1cacc5132292d851971 \ + --hash=sha256:4bd3e5c4b9682bc112d634f2c6ccc6736ed3635fc3319ac2bb11d768cc5a00d8 \ + --hash=sha256:577470e39e60a6cd7780793202e63536026d9b8641de011ed9d8174da9ca5339 \ + --hash=sha256:67285f8a611b0ebc0857ced2081e30302909f571a46bfa7a3cc0ad303fe015c6 \ + --hash=sha256:7285a89df4900ed3bfaad5679b1e668cb4b38a8de1ccbfc84b05f34512da0a90 \ + --hash=sha256:81823935e2f8d476707e85a78a405953a03ef7b7b4f55f93f7c2d9680e5e0691 \ + --hash=sha256:8978132287a9d3ad6b54fcd1e08548033cc09dc6aacacb6c004c73c3eb5d3ac3 \ + --hash=sha256:a20e442e917889d1a6b3c570c9e3fa2fdc398c20868abcea268ea33c024c4083 \ + --hash=sha256:a24ee598d10befaec178efdff6054bc4d7e883f615bfbcd08126a0f4931c83a6 \ + --hash=sha256:b04f85ac3a90c227b6e5890acb0edbaf3140938dbecf07bff618bf3638578cf1 \ + --hash=sha256:b6a0e535baec27b528cb07a119f321ac024592388c5681a5ced167ae98e9fff3 \ + --hash=sha256:bef32a5e327bd8e5af915d3416ffefdbe65ed975b646b3805be81b23580b57b8 \ + --hash=sha256:bfb4c801f65dd61cedfc61a83732327fafbac55a47282e6f26f073ca7a41c3b2 \ + --hash=sha256:c13b1e3afd29a5b3b2656257f14669ca8fa8d7956d509926f0b130b600b50ab7 \ + --hash=sha256:c987dad82e8c65ebc985f5dae5e74a3beda9d0a2a4daf8a1115f3772b59e5141 \ + --hash=sha256:ce7a453385e4c4693985b4a4a3533e041558851eae061a58a5405363b098fcd3 \ + --hash=sha256:d0c5c6bac22b177bf8da7435d9d27a6834ee130309749d162b26c3105c0795a9 \ + --hash=sha256:d97cf502abe2ab9eff8bd5e4aca274da8d06dd3ef08b759a8d6143f4ad65d4b4 \ + --hash=sha256:dad43797959a74103cb59c5dac71409f9c27d34c8a05921341fb64ea8ccb1dd4 \ + --hash=sha256:dd342f085542f6eb894ca00ef70236ea46070c8a13824c6bde0dfdcd36065b9b \ + --hash=sha256:de58755d723e86175756f463f2f0bddd45cc36fbd62601228a3f8761c9f58252 \ + --hash=sha256:f3df7b3d0f91b88b2106031fd995802a2e9ae13e02c36c1fc075b43f420f3a17 \ + --hash=sha256:f5414a788ecc6ee6bc58560e85ca624258a55ca434884445440a810796ea0e0b \ + --hash=sha256:fa26fa54c0a9384c27fcdc905a2fb7d60ac6e47d14bc2692145f2b3b1e2cfdbd # via secretstorage docutils==0.22 \ --hash=sha256:4ed966a0e96a0477d852f7af31bdcb3adc049fbb35ccba358c2ea8a03287615e \ From 2523c1e76d38586e9fe99498758381a03c29f8bc Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Wed, 3 Sep 2025 08:40:50 -0700 Subject: [PATCH 417/922] build(deps): bump jeepney from 0.8.0 to 0.9.0 in /tools/publish (#3234) Bumps [jeepney](https://gitlab.com/takluyver/jeepney) from 0.8.0 to 0.9.0.
Commits
  • bbd29d2 Merge branch 'changelog-0.9' into 'master'
  • 0e96cc2 Version number -> 0.9.0
  • ee71ce5 Add release notes for 0.9
  • a426b9f Merge branch 'attestations' into 'master'
  • 361bbd5 Only sign packages on tag
  • e79e0b1 Sign/attest packages before uploading
  • 0720488 Merge branch 'trusted-publish' into 'master'
  • e2356b3 Merge branch 'async-timeout-optional' into 'master'
  • 9173d08 Optionally depend on async_timeout in Python 3.11 and higher
  • 605f147 Merge branch 'matchrul'
  • Additional commits viewable in compare view

[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=jeepney&package-manager=pip&previous-version=0.8.0&new-version=0.9.0)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) ---
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot show ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> --- tools/publish/requirements_linux.txt | 6 +++--- tools/publish/requirements_universal.txt | 6 +++--- 2 files changed, 6 insertions(+), 6 deletions(-) diff --git a/tools/publish/requirements_linux.txt b/tools/publish/requirements_linux.txt index 7e3d42f518..1a381b2202 100644 --- a/tools/publish/requirements_linux.txt +++ b/tools/publish/requirements_linux.txt @@ -225,9 +225,9 @@ jaraco-functools==4.1.0 \ --hash=sha256:70f7e0e2ae076498e212562325e805204fc092d7b4c17e0e86c959e249701a9d \ --hash=sha256:ad159f13428bc4acbf5541ad6dec511f91573b90fba04df61dafa2a1231cf649 # via keyring -jeepney==0.8.0 \ - --hash=sha256:5efe48d255973902f6badc3ce55e2aa6c5c3b3bc642059ef3a91247bcfcc5806 \ - --hash=sha256:c0a454ad016ca575060802ee4d590dd912e35c122fa04e70306de3d076cce755 +jeepney==0.9.0 \ + --hash=sha256:97e5714520c16fc0a45695e5365a2e11b81ea79bba796e26f9f1d178cb182683 \ + --hash=sha256:cf0e9e845622b81e4a28df94c40345400256ec608d0e55bb8a3feaa9163f5732 # via # keyring # secretstorage diff --git a/tools/publish/requirements_universal.txt b/tools/publish/requirements_universal.txt index c3217299c9..c01f440d02 100644 --- a/tools/publish/requirements_universal.txt +++ b/tools/publish/requirements_universal.txt @@ -225,9 +225,9 @@ jaraco-functools==4.1.0 \ --hash=sha256:70f7e0e2ae076498e212562325e805204fc092d7b4c17e0e86c959e249701a9d \ --hash=sha256:ad159f13428bc4acbf5541ad6dec511f91573b90fba04df61dafa2a1231cf649 # via keyring -jeepney==0.8.0 ; sys_platform == 'linux' \ - --hash=sha256:5efe48d255973902f6badc3ce55e2aa6c5c3b3bc642059ef3a91247bcfcc5806 \ - --hash=sha256:c0a454ad016ca575060802ee4d590dd912e35c122fa04e70306de3d076cce755 +jeepney==0.9.0 ; sys_platform == 'linux' \ + --hash=sha256:97e5714520c16fc0a45695e5365a2e11b81ea79bba796e26f9f1d178cb182683 \ + --hash=sha256:cf0e9e845622b81e4a28df94c40345400256ec608d0e55bb8a3feaa9163f5732 # via # keyring # secretstorage From a9d4a8f90b295cbcb3c9ca18c1c43e2b4e39e6f6 Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Thu, 4 Sep 2025 00:19:45 -0700 Subject: [PATCH 418/922] build(deps): bump importlib-metadata from 8.5.0 to 8.7.0 in /tools/publish (#3237) Bumps [importlib-metadata](https://github.com/python/importlib_metadata) from 8.5.0 to 8.7.0.
Changelog

Sourced from importlib-metadata's changelog.

v8.7.0

Features

  • .metadata() (and Distribution.metadata) can now return None if the metadata directory exists but not metadata file is present. (#493)

Bugfixes

  • Raise consistent ValueError for invalid EntryPoint.value (#518)

v8.6.1

Bugfixes

  • Fixed indentation logic to also honor blank lines.

v8.6.0

Features

  • python/cpython#119650
Commits
  • 708dff4 Finalize
  • b3065f0 Merge pull request #519 from python/bugfix/493-metadata-missing
  • e4351c2 Add a new test capturing the new expectation.
  • 5a65705 Refactor the casting into a wrapper for brevity and to document its purpose.
  • 0830c39 Add news fragment.
  • 22bb567 Fix type errors where metadata could be None.
  • 57f31d7 Allow metadata to return None when there is no metadata present.
  • b9c4be4 Merge pull request #518 from python/bugfix/488-bad-ep-value
  • 9f8af01 Prefer a cached property, as the property is likely to be retrieved at least ...
  • f179e28 Also raise ValueError on construction if the value is invalid.
  • Additional commits viewable in compare view

[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=importlib-metadata&package-manager=pip&previous-version=8.5.0&new-version=8.7.0)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) ---
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot show ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> --- tools/publish/requirements_darwin.txt | 6 +++--- tools/publish/requirements_linux.txt | 6 +++--- tools/publish/requirements_universal.txt | 6 +++--- tools/publish/requirements_windows.txt | 6 +++--- 4 files changed, 12 insertions(+), 12 deletions(-) diff --git a/tools/publish/requirements_darwin.txt b/tools/publish/requirements_darwin.txt index f700e21176..d1c5aca6d3 100644 --- a/tools/publish/requirements_darwin.txt +++ b/tools/publish/requirements_darwin.txt @@ -99,9 +99,9 @@ idna==3.10 \ --hash=sha256:12f65c9b470abda6dc35cf8e63cc574b1c52b11df2c86030af0ac09b01b13ea9 \ --hash=sha256:946d195a0d259cbba61165e88e65941f16e9b36ea6ddb97f00452bae8b1287d3 # via requests -importlib-metadata==8.5.0 \ - --hash=sha256:45e54197d28b7a7f1559e60b95e7c567032b602131fbd588f1497f47880aa68b \ - --hash=sha256:71522656f0abace1d072b9e5481a48f07c138e00f079c38c8f883823f9c26bd7 +importlib-metadata==8.7.0 \ + --hash=sha256:d13b81ad223b890aa16c5471f2ac3056cf76c5f10f82d6f9292f0b415f389000 \ + --hash=sha256:e5dd1551894c77868a30651cef00984d50e1002d06942a7101d34870c5f02afd # via # keyring # twine diff --git a/tools/publish/requirements_linux.txt b/tools/publish/requirements_linux.txt index 1a381b2202..ea95036951 100644 --- a/tools/publish/requirements_linux.txt +++ b/tools/publish/requirements_linux.txt @@ -207,9 +207,9 @@ idna==3.10 \ --hash=sha256:12f65c9b470abda6dc35cf8e63cc574b1c52b11df2c86030af0ac09b01b13ea9 \ --hash=sha256:946d195a0d259cbba61165e88e65941f16e9b36ea6ddb97f00452bae8b1287d3 # via requests -importlib-metadata==8.5.0 \ - --hash=sha256:45e54197d28b7a7f1559e60b95e7c567032b602131fbd588f1497f47880aa68b \ - --hash=sha256:71522656f0abace1d072b9e5481a48f07c138e00f079c38c8f883823f9c26bd7 +importlib-metadata==8.7.0 \ + --hash=sha256:d13b81ad223b890aa16c5471f2ac3056cf76c5f10f82d6f9292f0b415f389000 \ + --hash=sha256:e5dd1551894c77868a30651cef00984d50e1002d06942a7101d34870c5f02afd # via # keyring # twine diff --git a/tools/publish/requirements_universal.txt b/tools/publish/requirements_universal.txt index c01f440d02..7df6b7b90e 100644 --- a/tools/publish/requirements_universal.txt +++ b/tools/publish/requirements_universal.txt @@ -207,9 +207,9 @@ idna==3.10 \ --hash=sha256:12f65c9b470abda6dc35cf8e63cc574b1c52b11df2c86030af0ac09b01b13ea9 \ --hash=sha256:946d195a0d259cbba61165e88e65941f16e9b36ea6ddb97f00452bae8b1287d3 # via requests -importlib-metadata==8.5.0 \ - --hash=sha256:45e54197d28b7a7f1559e60b95e7c567032b602131fbd588f1497f47880aa68b \ - --hash=sha256:71522656f0abace1d072b9e5481a48f07c138e00f079c38c8f883823f9c26bd7 +importlib-metadata==8.7.0 \ + --hash=sha256:d13b81ad223b890aa16c5471f2ac3056cf76c5f10f82d6f9292f0b415f389000 \ + --hash=sha256:e5dd1551894c77868a30651cef00984d50e1002d06942a7101d34870c5f02afd # via # keyring # twine diff --git a/tools/publish/requirements_windows.txt b/tools/publish/requirements_windows.txt index 18356503f5..c23911cd7d 100644 --- a/tools/publish/requirements_windows.txt +++ b/tools/publish/requirements_windows.txt @@ -99,9 +99,9 @@ idna==3.10 \ --hash=sha256:12f65c9b470abda6dc35cf8e63cc574b1c52b11df2c86030af0ac09b01b13ea9 \ --hash=sha256:946d195a0d259cbba61165e88e65941f16e9b36ea6ddb97f00452bae8b1287d3 # via requests -importlib-metadata==8.5.0 \ - --hash=sha256:45e54197d28b7a7f1559e60b95e7c567032b602131fbd588f1497f47880aa68b \ - --hash=sha256:71522656f0abace1d072b9e5481a48f07c138e00f079c38c8f883823f9c26bd7 +importlib-metadata==8.7.0 \ + --hash=sha256:d13b81ad223b890aa16c5471f2ac3056cf76c5f10f82d6f9292f0b415f389000 \ + --hash=sha256:e5dd1551894c77868a30651cef00984d50e1002d06942a7101d34870c5f02afd # via # keyring # twine From 1169eec93cae138a1c514bf7a8b6f537367b46ad Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Thu, 4 Sep 2025 00:20:09 -0700 Subject: [PATCH 419/922] build(deps): bump keyring from 25.5.0 to 25.6.0 in /tools/publish (#3236) Bumps [keyring](https://github.com/jaraco/keyring) from 25.5.0 to 25.6.0.
Changelog

Sourced from keyring's changelog.

v25.6.0

Features

  • Avoid logging a warning when config does not specify a backend. (#682)
Commits

[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=keyring&package-manager=pip&previous-version=25.5.0&new-version=25.6.0)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) ---
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot show ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> --- tools/publish/requirements_darwin.txt | 6 +++--- tools/publish/requirements_linux.txt | 6 +++--- tools/publish/requirements_universal.txt | 6 +++--- tools/publish/requirements_windows.txt | 6 +++--- 4 files changed, 12 insertions(+), 12 deletions(-) diff --git a/tools/publish/requirements_darwin.txt b/tools/publish/requirements_darwin.txt index d1c5aca6d3..2ecf5a0e51 100644 --- a/tools/publish/requirements_darwin.txt +++ b/tools/publish/requirements_darwin.txt @@ -117,9 +117,9 @@ jaraco-functools==4.1.0 \ --hash=sha256:70f7e0e2ae076498e212562325e805204fc092d7b4c17e0e86c959e249701a9d \ --hash=sha256:ad159f13428bc4acbf5541ad6dec511f91573b90fba04df61dafa2a1231cf649 # via keyring -keyring==25.5.0 \ - --hash=sha256:4c753b3ec91717fe713c4edd522d625889d8973a349b0e582622f49766de58e6 \ - --hash=sha256:e67f8ac32b04be4714b42fe84ce7dad9c40985b9ca827c592cc303e7c26d9741 +keyring==25.6.0 \ + --hash=sha256:0b39998aa941431eb3d9b0d4b2460bc773b9df6fed7621c2dfb291a7e0187a66 \ + --hash=sha256:552a3f7af126ece7ed5c89753650eec89c7eaae8617d0aa4d9ad2b75111266bd # via twine markdown-it-py==3.0.0 \ --hash=sha256:355216845c60bd96232cd8d8c40e8f9765cc86f46880e43a8fd22dc1a1a8cab1 \ diff --git a/tools/publish/requirements_linux.txt b/tools/publish/requirements_linux.txt index ea95036951..d5d7563f94 100644 --- a/tools/publish/requirements_linux.txt +++ b/tools/publish/requirements_linux.txt @@ -231,9 +231,9 @@ jeepney==0.9.0 \ # via # keyring # secretstorage -keyring==25.5.0 \ - --hash=sha256:4c753b3ec91717fe713c4edd522d625889d8973a349b0e582622f49766de58e6 \ - --hash=sha256:e67f8ac32b04be4714b42fe84ce7dad9c40985b9ca827c592cc303e7c26d9741 +keyring==25.6.0 \ + --hash=sha256:0b39998aa941431eb3d9b0d4b2460bc773b9df6fed7621c2dfb291a7e0187a66 \ + --hash=sha256:552a3f7af126ece7ed5c89753650eec89c7eaae8617d0aa4d9ad2b75111266bd # via twine markdown-it-py==3.0.0 \ --hash=sha256:355216845c60bd96232cd8d8c40e8f9765cc86f46880e43a8fd22dc1a1a8cab1 \ diff --git a/tools/publish/requirements_universal.txt b/tools/publish/requirements_universal.txt index 7df6b7b90e..aaff8bd59a 100644 --- a/tools/publish/requirements_universal.txt +++ b/tools/publish/requirements_universal.txt @@ -231,9 +231,9 @@ jeepney==0.9.0 ; sys_platform == 'linux' \ # via # keyring # secretstorage -keyring==25.5.0 \ - --hash=sha256:4c753b3ec91717fe713c4edd522d625889d8973a349b0e582622f49766de58e6 \ - --hash=sha256:e67f8ac32b04be4714b42fe84ce7dad9c40985b9ca827c592cc303e7c26d9741 +keyring==25.6.0 \ + --hash=sha256:0b39998aa941431eb3d9b0d4b2460bc773b9df6fed7621c2dfb291a7e0187a66 \ + --hash=sha256:552a3f7af126ece7ed5c89753650eec89c7eaae8617d0aa4d9ad2b75111266bd # via twine markdown-it-py==3.0.0 \ --hash=sha256:355216845c60bd96232cd8d8c40e8f9765cc86f46880e43a8fd22dc1a1a8cab1 \ diff --git a/tools/publish/requirements_windows.txt b/tools/publish/requirements_windows.txt index c23911cd7d..0a3139a17e 100644 --- a/tools/publish/requirements_windows.txt +++ b/tools/publish/requirements_windows.txt @@ -117,9 +117,9 @@ jaraco-functools==4.1.0 \ --hash=sha256:70f7e0e2ae076498e212562325e805204fc092d7b4c17e0e86c959e249701a9d \ --hash=sha256:ad159f13428bc4acbf5541ad6dec511f91573b90fba04df61dafa2a1231cf649 # via keyring -keyring==25.5.0 \ - --hash=sha256:4c753b3ec91717fe713c4edd522d625889d8973a349b0e582622f49766de58e6 \ - --hash=sha256:e67f8ac32b04be4714b42fe84ce7dad9c40985b9ca827c592cc303e7c26d9741 +keyring==25.6.0 \ + --hash=sha256:0b39998aa941431eb3d9b0d4b2460bc773b9df6fed7621c2dfb291a7e0187a66 \ + --hash=sha256:552a3f7af126ece7ed5c89753650eec89c7eaae8617d0aa4d9ad2b75111266bd # via twine markdown-it-py==3.0.0 \ --hash=sha256:355216845c60bd96232cd8d8c40e8f9765cc86f46880e43a8fd22dc1a1a8cab1 \ From 49772214d4f2c87e3b0968a373d4a694a147c1e7 Mon Sep 17 00:00:00 2001 From: Joshua Bronson Date: Thu, 4 Sep 2025 16:40:15 -0400 Subject: [PATCH 420/922] refactor(gazelle): report missing BUILD_WORKSPACE_DIRECTORY key more directly (#3240) Replace `os.environ.get("BUILD_WORKSPACE_DIRECTORY")` with `os.environ["BUILD_WORKSPACE_DIRECTORY"]`. The former may return None if the environment variable is not set, in which case the code will crash with a TypeError when the line is run since the result is concatenated with a `pathlib.Path` object, and is therefore making it impossible to use rules_python_gazelle_plugin along with rules_mypy: These changes allow rules_mypy users to also use rules_python_gazelle_plugin without having to work around the type error. Now if the environment variable is not set, the code will still crash, but now with an error that better indicates the failed precondition, namely `KeyError("BUILD_WORKSPACE_DIRECTORY")` rather than `TypeError("unsupported operand type(s) for /: 'PosixPath' and 'NoneType')`. --- gazelle/manifest/copy_to_source.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/gazelle/manifest/copy_to_source.py b/gazelle/manifest/copy_to_source.py index 4ebb958c3d..b897b1fcf3 100644 --- a/gazelle/manifest/copy_to_source.py +++ b/gazelle/manifest/copy_to_source.py @@ -20,7 +20,7 @@ def copy_to_source(generated_relative_path: Path, target_relative_path: Path) -> generated_absolute_path = Path.cwd() / generated_relative_path # Similarly, the target is relative to the source directory. - target_absolute_path = os.getenv("BUILD_WORKSPACE_DIRECTORY") / target_relative_path + target_absolute_path = os.environ["BUILD_WORKSPACE_DIRECTORY"] / target_relative_path print(f"Copying {generated_absolute_path} to {target_absolute_path}") target_absolute_path.parent.mkdir(parents=True, exist_ok=True) From 277089e6a4b2997d3722b519f9f058ce6a578dd6 Mon Sep 17 00:00:00 2001 From: Ignas Anikevicius <240938+aignas@users.noreply.github.com> Date: Fri, 5 Sep 2025 22:39:09 +0900 Subject: [PATCH 421/922] chore(deps): bump rules_cc to 0.1.5 (#3238) This fixes an issue compiling protobuf on windows due to c++ 17 support. In particular, it gets the fix in https://github.com/bazelbuild/rules_cc/commit/c7e5c8c9b6a53695b29766f7fcfe655ef2609b1d which adds `/std:c++17` for Window builds. Fixes #3122 --- CHANGELOG.md | 2 +- MODULE.bazel | 2 +- internal_dev_deps.bzl | 6 +++--- python/private/py_repositories.bzl | 6 +++--- tests/integration/local_toolchains/MODULE.bazel | 2 +- 5 files changed, 9 insertions(+), 9 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 667814861f..48dd26c846 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -61,7 +61,7 @@ END_UNRELEASED_TEMPLATE {#v0-0-0-changed} ### Changed -* Nothing changed. +* (deps) bumped rules_cc dependency to `0.1.5`. {#v0-0-0-fixed} ### Fixed diff --git a/MODULE.bazel b/MODULE.bazel index 4f442bacec..1dca3e91fa 100644 --- a/MODULE.bazel +++ b/MODULE.bazel @@ -6,7 +6,7 @@ module( bazel_dep(name = "bazel_features", version = "1.21.0") bazel_dep(name = "bazel_skylib", version = "1.8.1") -bazel_dep(name = "rules_cc", version = "0.0.16") +bazel_dep(name = "rules_cc", version = "0.1.5") bazel_dep(name = "platforms", version = "0.0.11") # Those are loaded only when using py_proto_library diff --git a/internal_dev_deps.bzl b/internal_dev_deps.bzl index e6ade4035c..e1a6562fe6 100644 --- a/internal_dev_deps.bzl +++ b/internal_dev_deps.bzl @@ -233,9 +233,9 @@ def rules_python_internal_deps(): http_archive( name = "rules_cc", - urls = ["https://github.com/bazelbuild/rules_cc/releases/download/0.0.16/rules_cc-0.0.16.tar.gz"], - sha256 = "bbf1ae2f83305b7053b11e4467d317a7ba3517a12cef608543c1b1c5bf48a4df", - strip_prefix = "rules_cc-0.0.16", + urls = ["https://github.com/bazelbuild/rules_cc/releases/download/0.1.5/rules_cc-0.1.5.tar.gz"], + sha256 = "b8b918a85f9144c01f6cfe0f45e4f2838c7413961a8ff23bc0c6cdf8bb07a3b6", + strip_prefix = "rules_cc-0.1.5", ) http_archive( diff --git a/python/private/py_repositories.bzl b/python/private/py_repositories.bzl index 10bc06630b..c09ba68361 100644 --- a/python/private/py_repositories.bzl +++ b/python/private/py_repositories.bzl @@ -59,9 +59,9 @@ def py_repositories(): ) http_archive( name = "rules_cc", - sha256 = "4b12149a041ddfb8306a8fd0e904e39d673552ce82e4296e96fac9cbf0780e59", - strip_prefix = "rules_cc-0.1.0", - urls = ["https://github.com/bazelbuild/rules_cc/releases/download/0.1.0/rules_cc-0.1.0.tar.gz"], + sha256 = "b8b918a85f9144c01f6cfe0f45e4f2838c7413961a8ff23bc0c6cdf8bb07a3b6", + strip_prefix = "rules_cc-0.1.5", + urls = ["https://github.com/bazelbuild/rules_cc/releases/download/0.1.5/rules_cc-0.1.5.tar.gz"], ) # Needed by rules_cc, triggered by @rules_java_prebuilt in Bazel by using @rules_cc//cc:defs.bzl diff --git a/tests/integration/local_toolchains/MODULE.bazel b/tests/integration/local_toolchains/MODULE.bazel index 45afaafbc9..e81c012c2d 100644 --- a/tests/integration/local_toolchains/MODULE.bazel +++ b/tests/integration/local_toolchains/MODULE.bazel @@ -16,7 +16,7 @@ module(name = "module_under_test") bazel_dep(name = "rules_python", version = "0.0.0") bazel_dep(name = "bazel_skylib", version = "1.7.1") bazel_dep(name = "platforms", version = "0.0.11") -bazel_dep(name = "rules_cc", version = "0.0.16") +bazel_dep(name = "rules_cc", version = "0.1.5") local_path_override( module_name = "rules_python", From 5cbb5b16c08f1832c569608126b27046e32e18ad Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Sat, 6 Sep 2025 01:27:28 -0700 Subject: [PATCH 422/922] fix(sphinxdocs): add retry logic when exit code 2 occurs (#3241) Running Sphinx multiple times in the same process sometimes results in an error ("exit code 2"). Digging in, this is likely a bug in the sphinx_bzl plugin in how it merges data when parallel or incremental builds are performed. Until that's fixed, work around the problem by internally retrying the Sphinx build when exit code 2 occurs. This is basically what we're doing today and should reduce the number of flakes for the RTD builds. Along the way, improve the error reporting to make it easier to diagnose the underlying failure. --- sphinxdocs/private/sphinx_build.py | 74 ++++++++++++++++++++++++------ 1 file changed, 60 insertions(+), 14 deletions(-) diff --git a/sphinxdocs/private/sphinx_build.py b/sphinxdocs/private/sphinx_build.py index e9711042f6..b438c89fe1 100644 --- a/sphinxdocs/private/sphinx_build.py +++ b/sphinxdocs/private/sphinx_build.py @@ -14,6 +14,13 @@ WorkRequest = object WorkResponse = object + +class SphinxMainError(Exception): + def __init__(self, message, exit_code): + super().__init__(message) + self.exit_code = exit_code + + logger = logging.getLogger("sphinxdocs_build") _WORKER_SPHINX_EXT_MODULE_NAME = "bazel_worker_sphinx_ext" @@ -58,7 +65,7 @@ def __init__( def __enter__(self): return self - def __exit__(self): + def __exit__(self, exc_type, exc_val, exc_tb): for worker_outdir in self._worker_outdirs: shutil.rmtree(worker_outdir, ignore_errors=True) @@ -75,6 +82,17 @@ def run(self) -> None: response = self._process_request(request) if response: self._send_response(response) + except SphinxMainError as e: + logger.error("Sphinx main returned failure: exit_code=%s request=%s", + request, e.exit_code) + request_id = 0 if not request else request.get("requestId", 0) + self._send_response( + { + "exitCode": e.exit_code, + "output": str(e), + "requestId": request_id, + } + ) except Exception: logger.exception("Unhandled error: request=%s", request) output = ( @@ -142,13 +160,10 @@ def _prepare_sphinx(self, request): @contextlib.contextmanager def _redirect_streams(self): - out = io.StringIO() - orig_stdout = sys.stdout - try: - sys.stdout = out - yield out - finally: - sys.stdout = orig_stdout + stdout = io.StringIO() + stderr = io.StringIO() + with contextlib.redirect_stdout(stdout), contextlib.redirect_stderr(stderr): + yield stdout, stderr def _process_request(self, request: "WorkRequest") -> "WorkResponse | None": logger.info("Request: %s", json.dumps(request, sort_keys=True, indent=2)) @@ -159,19 +174,50 @@ def _process_request(self, request: "WorkRequest") -> "WorkResponse | None": # Prevent anything from going to stdout because it breaks the worker # protocol. We have limited control over where Sphinx sends output. - with self._redirect_streams() as stdout: + with self._redirect_streams() as (stdout, stderr): logger.info("main args: %s", sphinx_args) exit_code = main(sphinx_args) + # Running Sphinx multiple times in a process can give spurious + # errors. An invocation after an error seems to work, though. + if exit_code == 2: + logger.warning("Sphinx main() returned exit_code=2, retrying...") + # Reset streams to capture output of the retry cleanly + stdout.seek(0) + stdout.truncate(0) + stderr.seek(0) + stderr.truncate(0) + exit_code = main(sphinx_args) if exit_code: - raise Exception( + stdout_output = stdout.getvalue().strip() + stderr_output = stderr.getvalue().strip() + if stdout_output: + stdout_output = ( + "========== STDOUT START ==========\n" + + stdout_output + + "\n" + + "========== STDOUT END ==========\n" + ) + else: + stdout_output = "========== STDOUT EMPTY ==========\n" + if stderr_output: + stderr_output = ( + "========== STDERR START ==========\n" + + stderr_output + + "\n" + + "========== STDERR END ==========\n" + ) + else: + stderr_output = "========== STDERR EMPTY ==========\n" + + message = ( "Sphinx main() returned failure: " + f" exit code: {exit_code}\n" - + "========== STDOUT START ==========\n" - + stdout.getvalue().rstrip("\n") - + "\n" - + "========== STDOUT END ==========\n" + + stdout_output + + stderr_output ) + raise SphinxMainError(message, exit_code) + # Copying is unfortunately necessary because Bazel doesn't know to # implicily bring along what the symlinks point to. From b8e32c454a1158cd78ce4ecaef809b99bef4e5da Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Sat, 6 Sep 2025 10:30:38 -0700 Subject: [PATCH 423/922] fix(system_python): write import paths to generated file instead of using PYTHONPATH (#3242) This changes the system_python bootstrap to use a 2-stage process like the script bootstrap does. Among other things, this means the import paths are written to a generated file (`bazel_site_init.py`, same as boostrap=script) and sys.path setup is performed by the Python code in stage 2. Since the PYTHONPATH environment variable isn't used, this fixes the problem on Windows where the value is too long. This also better unifies the system_python and script based bootstraps because the same stage 2 code and bazel_site_init code is used. Along the way, several other improvements: * Fixes path ordering for system_python. The order now matches venv ordering (stdlib, binary paths, runtime site packages). * Makes the venv-based solution work when the site module is disabled (`-S`). * Makes `interpreter_args` attribute and `RULES_PYTHON_ADDITIONAL_INTERPRETER_ARGS` env var work with system_python. * Makes `main_module` work with system_python. * Progress towards a supportable non-shell based bootstrap (a user requested this because their environment doesn't install any shells as a security precaution). Fixes https://github.com/bazel-contrib/rules_python/issues/2652 --- CHANGELOG.md | 16 +- docs/environment-variables.md | 4 + python/private/py_executable.bzl | 137 +++++--- python/private/python_bootstrap_template.txt | 324 ++++-------------- python/private/stage2_bootstrap_template.py | 59 +++- python/private/zip_main_template.py | 112 +++--- tests/base_rules/py_executable_base_tests.bzl | 5 +- tests/bootstrap_impls/sys_path_order_test.py | 28 +- 8 files changed, 280 insertions(+), 405 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 48dd26c846..55d0d3fa2f 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -62,16 +62,30 @@ END_UNRELEASED_TEMPLATE {#v0-0-0-changed} ### Changed * (deps) bumped rules_cc dependency to `0.1.5`. +* (bootstrap) For {obj}`--bootstrap_impl=system_python`, `PYTHONPATH` is no + longer used to add import paths. The sys.path order has changed from + `[app paths, stdlib, runtime site-packages]` to `[stdlib, app paths, runtime + site-packages]`. +* (bootstrap) For {obj}`--bootstrap_impl=system_python`, the sys.path order has + changed from `[app paths, stdlib, runtime site-packages]` to `[stdlib, app + paths, runtime site-packages]`. {#v0-0-0-fixed} ### Fixed * (bootstrap) The stage1 bootstrap script now correctly handles nested `RUNFILES_DIR` environments, fixing issues where a `py_binary` calls another `py_binary` ([#3187](https://github.com/bazel-contrib/rules_python/issues/3187)). +* (bootstrap) For Windows, having many dependencies no longer results in max + length errors due to too long environment variables. +* (bootstrap) {obj}`--bootstrap_impl=script` now supports the `-S` interpreter + setting. {#v0-0-0-added} ### Added -* Nothing added. +* (bootstrap) {obj}`--bootstrap_impl=system_python` now supports the + {obj}`main_module` attribute. +* (bootstrap) {obj}`--bootstrap_impl=system_python` now supports the + {any}`RULES_PYTHON_ADDITIONAL_INTERPRETER_ARGS` attribute. {#v1-6-0} diff --git a/docs/environment-variables.md b/docs/environment-variables.md index 9a8c1dfe99..4913e329e4 100644 --- a/docs/environment-variables.md +++ b/docs/environment-variables.md @@ -25,6 +25,10 @@ The {bzl:obj}`interpreter_args` attribute. ::: :::{versionadded} 1.3.0 +::: +:::{versionchanged} VERSION_NEXT_FEATURE +Support added for {obj}`--bootstrap_impl=system_python`. +::: :::: diff --git a/python/private/py_executable.bzl b/python/private/py_executable.bzl index 5fafc8911d..41938ebf78 100644 --- a/python/private/py_executable.bzl +++ b/python/private/py_executable.bzl @@ -140,6 +140,9 @@ This is mutually exclusive with {obj}`main`. :::{versionadded} 1.3.0 ::: +:::{versionchanged} VERSION_NEXT_FEATURE +Support added for {obj}`--bootstrap_impl=system_python`. +::: """, ), "pyc_collection": lambda: attrb.String( @@ -332,9 +335,10 @@ def _create_executable( # BuiltinPyRuntimeInfo providers, which is likely to come from # @bazel_tools//tools/python:autodetecting_toolchain, the toolchain used # for workspace builds when no rules_python toolchain is configured. - if (BootstrapImplFlag.get_value(ctx) == BootstrapImplFlag.SCRIPT and + if ( runtime_details.effective_runtime and - hasattr(runtime_details.effective_runtime, "stage2_bootstrap_template")): + hasattr(runtime_details.effective_runtime, "stage2_bootstrap_template") + ): venv = _create_venv( ctx, output_prefix = base_executable_name, @@ -351,7 +355,11 @@ def _create_executable( runtime_details = runtime_details, venv = venv, ) - extra_runfiles = ctx.runfiles([stage2_bootstrap] + venv.files_without_interpreter) + extra_runfiles = ctx.runfiles( + [stage2_bootstrap] + ( + venv.files_without_interpreter if venv else [] + ), + ) zip_main = _create_zip_main( ctx, stage2_bootstrap = stage2_bootstrap, @@ -460,7 +468,7 @@ def _create_executable( # The interpreter is added this late in the process so that it isn't # added to the zipped files. - if venv: + if venv and venv.interpreter: extra_runfiles = extra_runfiles.merge(ctx.runfiles([venv.interpreter])) return create_executable_result_struct( extra_files_to_build = depset(extra_files_to_build), @@ -469,7 +477,10 @@ def _create_executable( ) def _create_zip_main(ctx, *, stage2_bootstrap, runtime_details, venv): - python_binary = runfiles_root_path(ctx, venv.interpreter.short_path) + if venv.interpreter: + python_binary = runfiles_root_path(ctx, venv.interpreter.short_path) + else: + python_binary = "" python_binary_actual = venv.interpreter_actual_path # The location of this file doesn't really matter. It's added to @@ -529,13 +540,17 @@ def relative_path(from_, to): # * https://github.com/python/cpython/blob/main/Modules/getpath.py # * https://github.com/python/cpython/blob/main/Lib/site.py def _create_venv(ctx, output_prefix, imports, runtime_details): + create_full_venv = BootstrapImplFlag.get_value(ctx) == BootstrapImplFlag.SCRIPT venv = "_{}.venv".format(output_prefix.lstrip("_")) - # The pyvenv.cfg file must be present to trigger the venv site hooks. - # Because it's paths are expected to be absolute paths, we can't reliably - # put much in it. See https://github.com/python/cpython/issues/83650 - pyvenv_cfg = ctx.actions.declare_file("{}/pyvenv.cfg".format(venv)) - ctx.actions.write(pyvenv_cfg, "") + if create_full_venv: + # The pyvenv.cfg file must be present to trigger the venv site hooks. + # Because it's paths are expected to be absolute paths, we can't reliably + # put much in it. See https://github.com/python/cpython/issues/83650 + pyvenv_cfg = ctx.actions.declare_file("{}/pyvenv.cfg".format(venv)) + ctx.actions.write(pyvenv_cfg, "") + else: + pyvenv_cfg = None runtime = runtime_details.effective_runtime @@ -543,48 +558,48 @@ def _create_venv(ctx, output_prefix, imports, runtime_details): VenvsUseDeclareSymlinkFlag.get_value(ctx) == VenvsUseDeclareSymlinkFlag.YES ) recreate_venv_at_runtime = False - bin_dir = "{}/bin".format(venv) - - if not venvs_use_declare_symlink_enabled or not runtime.supports_build_time_venv: - recreate_venv_at_runtime = True - if runtime.interpreter: - interpreter_actual_path = runfiles_root_path(ctx, runtime.interpreter.short_path) - else: - interpreter_actual_path = runtime.interpreter_path - py_exe_basename = paths.basename(interpreter_actual_path) + if runtime.interpreter: + interpreter_actual_path = runfiles_root_path(ctx, runtime.interpreter.short_path) + else: + interpreter_actual_path = runtime.interpreter_path - # When the venv symlinks are disabled, the $venv/bin/python3 file isn't - # needed or used at runtime. However, the zip code uses the interpreter - # File object to figure out some paths. - interpreter = ctx.actions.declare_file("{}/{}".format(bin_dir, py_exe_basename)) - ctx.actions.write(interpreter, "actual:{}".format(interpreter_actual_path)) + bin_dir = "{}/bin".format(venv) - elif runtime.interpreter: + if create_full_venv: # Some wrappers around the interpreter (e.g. pyenv) use the program # name to decide what to do, so preserve the name. - py_exe_basename = paths.basename(runtime.interpreter.short_path) + py_exe_basename = paths.basename(interpreter_actual_path) - # Even though ctx.actions.symlink() is used, using - # declare_symlink() is required to ensure that the resulting file - # in runfiles is always a symlink. An RBE implementation, for example, - # may choose to write what symlink() points to instead. - interpreter = ctx.actions.declare_symlink("{}/{}".format(bin_dir, py_exe_basename)) + if not venvs_use_declare_symlink_enabled or not runtime.supports_build_time_venv: + recreate_venv_at_runtime = True - interpreter_actual_path = runfiles_root_path(ctx, runtime.interpreter.short_path) - rel_path = relative_path( - # dirname is necessary because a relative symlink is relative to - # the directory the symlink resides within. - from_ = paths.dirname(runfiles_root_path(ctx, interpreter.short_path)), - to = interpreter_actual_path, - ) + # When the venv symlinks are disabled, the $venv/bin/python3 file isn't + # needed or used at runtime. However, the zip code uses the interpreter + # File object to figure out some paths. + interpreter = ctx.actions.declare_file("{}/{}".format(bin_dir, py_exe_basename)) + ctx.actions.write(interpreter, "actual:{}".format(interpreter_actual_path)) - ctx.actions.symlink(output = interpreter, target_path = rel_path) + elif runtime.interpreter: + # Even though ctx.actions.symlink() is used, using + # declare_symlink() is required to ensure that the resulting file + # in runfiles is always a symlink. An RBE implementation, for example, + # may choose to write what symlink() points to instead. + interpreter = ctx.actions.declare_symlink("{}/{}".format(bin_dir, py_exe_basename)) + + rel_path = relative_path( + # dirname is necessary because a relative symlink is relative to + # the directory the symlink resides within. + from_ = paths.dirname(runfiles_root_path(ctx, interpreter.short_path)), + to = interpreter_actual_path, + ) + + ctx.actions.symlink(output = interpreter, target_path = rel_path) + else: + interpreter = ctx.actions.declare_symlink("{}/{}".format(bin_dir, py_exe_basename)) + ctx.actions.symlink(output = interpreter, target_path = runtime.interpreter_path) else: - py_exe_basename = paths.basename(runtime.interpreter_path) - interpreter = ctx.actions.declare_symlink("{}/{}".format(bin_dir, py_exe_basename)) - ctx.actions.symlink(output = interpreter, target_path = runtime.interpreter_path) - interpreter_actual_path = runtime.interpreter_path + interpreter = None if runtime.interpreter_version_info: version = "{}.{}".format( @@ -626,14 +641,29 @@ def _create_venv(ctx, output_prefix, imports, runtime_details): } venv_symlinks = _create_venv_symlinks(ctx, venv_dir_map) + files_without_interpreter = [pth, site_init] + venv_symlinks + if pyvenv_cfg: + files_without_interpreter.append(pyvenv_cfg) + return struct( + # File or None; the `bin/python3` executable in the venv. + # None if a full venv isn't created. interpreter = interpreter, + # bool; True if the venv should be recreated at runtime recreate_venv_at_runtime = recreate_venv_at_runtime, # Runfiles root relative path or absolute path interpreter_actual_path = interpreter_actual_path, - files_without_interpreter = [pyvenv_cfg, pth, site_init] + venv_symlinks, + files_without_interpreter = files_without_interpreter, # string; venv-relative path to the site-packages directory. venv_site_packages = venv_site_packages, + # string; runfiles-root relative path to venv root. + venv_root = runfiles_root_path( + ctx, + paths.join( + py_internal.get_label_repo_runfiles_path(ctx.label), + venv, + ), + ), ) def _create_venv_symlinks(ctx, venv_dir_map): @@ -746,7 +776,7 @@ def _create_stage2_bootstrap( main_py, imports, runtime_details, - venv = None): + venv): output = ctx.actions.declare_file( # Prepend with underscore to prevent pytest from trying to # process the bootstrap for files starting with `test_` @@ -758,17 +788,10 @@ def _create_stage2_bootstrap( template = runtime.stage2_bootstrap_template if main_py: - main_py_path = "{}/{}".format(ctx.workspace_name, main_py.short_path) + main_py_path = runfiles_root_path(ctx, main_py.short_path) else: main_py_path = "" - # The stage2 bootstrap uses the venv site-packages location to fix up issues - # that occur when the toolchain doesn't support the build-time venv. - if venv and not runtime.supports_build_time_venv: - venv_rel_site_packages = venv.venv_site_packages - else: - venv_rel_site_packages = "" - ctx.actions.expand_template( template = template, output = output, @@ -779,7 +802,8 @@ def _create_stage2_bootstrap( "%main%": main_py_path, "%main_module%": ctx.attr.main_module, "%target%": str(ctx.label), - "%venv_rel_site_packages%": venv_rel_site_packages, + "%venv_rel_site_packages%": venv.venv_site_packages, + "%venv_root%": venv.venv_root, "%workspace_name%": ctx.workspace_name, }, is_executable = True, @@ -800,7 +824,10 @@ def _create_stage1_bootstrap( runtime = runtime_details.effective_runtime if venv: - python_binary_path = runfiles_root_path(ctx, venv.interpreter.short_path) + if venv.interpreter: + python_binary_path = runfiles_root_path(ctx, venv.interpreter.short_path) + else: + python_binary_path = "" else: python_binary_path = runtime_details.executable_interpreter_path diff --git a/python/private/python_bootstrap_template.txt b/python/private/python_bootstrap_template.txt index 495a52cfe9..9717756036 100644 --- a/python/private/python_bootstrap_template.txt +++ b/python/private/python_bootstrap_template.txt @@ -1,4 +1,5 @@ %shebang% +# vim: syntax=python from __future__ import absolute_import from __future__ import division @@ -6,18 +7,42 @@ from __future__ import print_function import sys -# The Python interpreter unconditionally prepends the directory containing this -# script (following symlinks) to the import path. This is the cause of #9239, -# and is a special case of #7091. We therefore explicitly delete that entry. -# TODO(#7091): Remove this hack when no longer necessary. -del sys.path[0] - import os import subprocess import uuid +# runfiles-relative path +STAGE2_BOOTSTRAP="%stage2_bootstrap%" + +# runfiles-relative path to venv's python interpreter +# Empty string if a venv is not setup. +PYTHON_BINARY = '%python_binary%' + +# The path to the actual interpreter that is used. +# Typically PYTHON_BINARY is a symlink pointing to this. +# runfiles-relative path, absolute path, or single word. +# Used to create a venv at runtime, or when a venv isn't setup. +PYTHON_BINARY_ACTUAL = "%python_binary_actual%" + +# 0 or 1. +# 1 if this bootstrap was created for placement within a zipfile. 0 otherwise. +IS_ZIPFILE = "%is_zipfile%" == "1" +# 0 or 1. +# If 1, then a venv will be created at runtime that replicates what would have +# been the build-time structure. +RECREATE_VENV_AT_RUNTIME="%recreate_venv_at_runtime%" + +WORKSPACE_NAME = "%workspace_name%" + +# Target-specific interpreter args. +INTERPRETER_ARGS = [ +%interpreter_args% +] + +ADDITIONAL_INTERPRETER_ARGS = os.environ.get("RULES_PYTHON_ADDITIONAL_INTERPRETER_ARGS", "") + def IsRunningFromZip(): - return %is_zipfile% + return IS_ZIPFILE if IsRunningFromZip(): import shutil @@ -73,8 +98,7 @@ def GetWindowsPathWithUNCPrefix(path): def HasWindowsExecutableExtension(path): return path.endswith('.exe') or path.endswith('.com') or path.endswith('.bat') -PYTHON_BINARY = '%python_binary%' -if IsWindows() and not HasWindowsExecutableExtension(PYTHON_BINARY): +if PYTHON_BINARY and IsWindows() and not HasWindowsExecutableExtension(PYTHON_BINARY): PYTHON_BINARY = PYTHON_BINARY + '.exe' def SearchPath(name): @@ -89,14 +113,18 @@ def SearchPath(name): def FindPythonBinary(module_space): """Finds the real Python binary if it's not a normal absolute path.""" - return FindBinary(module_space, PYTHON_BINARY) + if PYTHON_BINARY: + return FindBinary(module_space, PYTHON_BINARY) + else: + return FindBinary(module_space, PYTHON_BINARY_ACTUAL) + def print_verbose(*args, mapping=None, values=None): if os.environ.get("RULES_PYTHON_BOOTSTRAP_VERBOSE"): if mapping is not None: for key, value in sorted((mapping or {}).items()): print( - "bootstrap:", + "bootstrap: stage 1: ", *(list(args) + ["{}={}".format(key, repr(value))]), file=sys.stderr, flush=True @@ -104,34 +132,13 @@ def print_verbose(*args, mapping=None, values=None): elif values is not None: for i, v in enumerate(values): print( - "bootstrap:", + "bootstrap: stage 1:", *(list(args) + ["[{}] {}".format(i, repr(v))]), file=sys.stderr, flush=True ) else: - print("bootstrap:", *args, file=sys.stderr, flush=True) - -def PrintVerboseCoverage(*args): - """Print output if VERBOSE_COVERAGE is non-empty in the environment.""" - if os.environ.get("VERBOSE_COVERAGE"): - print(*args, file=sys.stderr) - -def IsVerboseCoverage(): - """Returns True if VERBOSE_COVERAGE is non-empty in the environment.""" - return os.environ.get("VERBOSE_COVERAGE") - -def FindCoverageEntryPoint(module_space): - cov_tool = '%coverage_tool%' - if cov_tool: - PrintVerboseCoverage('Using toolchain coverage_tool %r' % cov_tool) - else: - cov_tool = os.environ.get('PYTHON_COVERAGE') - if cov_tool: - PrintVerboseCoverage('PYTHON_COVERAGE: %r' % cov_tool) - if cov_tool: - return FindBinary(module_space, cov_tool) - return None + print("bootstrap: stage 1:", *args, file=sys.stderr, flush=True) def FindBinary(module_space, bin_name): """Finds the real binary if it's not a normal absolute path.""" @@ -153,10 +160,6 @@ def FindBinary(module_space, bin_name): # Case 4: Path has to be looked up in the search path. return SearchPath(bin_name) -def CreatePythonPathEntries(python_imports, module_space): - parts = python_imports.split(':') - return [module_space] + ['%s/%s' % (module_space, path) for path in parts] - def FindModuleSpace(main_rel_path): """Finds the runfiles tree.""" # When the calling process used the runfiles manifest to resolve the @@ -240,14 +243,6 @@ def CreateModuleSpace(): # important that deletion code be in sync with this directory structure return os.path.join(temp_dir, 'runfiles') -# Returns repository roots to add to the import path. -def GetRepositoriesImports(module_space, import_all): - if import_all: - repo_dirs = [os.path.join(module_space, d) for d in os.listdir(module_space)] - repo_dirs.sort() - return [d for d in repo_dirs if os.path.isdir(d)] - return [os.path.join(module_space, '%workspace_name%')] - def RunfilesEnvvar(module_space): """Finds the runfiles manifest or the runfiles directory. @@ -290,63 +285,8 @@ def RunfilesEnvvar(module_space): return (None, None) -def Deduplicate(items): - """Efficiently filter out duplicates, keeping the first element only.""" - seen = set() - for it in items: - if it not in seen: - seen.add(it) - yield it - -def InstrumentedFilePaths(): - """Yields tuples of realpath of each instrumented file with the relative path.""" - manifest_filename = os.environ.get('COVERAGE_MANIFEST') - if not manifest_filename: - return - with open(manifest_filename, "r") as manifest: - for line in manifest: - filename = line.strip() - if not filename: - continue - try: - realpath = os.path.realpath(filename) - except OSError: - print( - "Could not find instrumented file {}".format(filename), - file=sys.stderr) - continue - if realpath != filename: - PrintVerboseCoverage("Fixing up {} -> {}".format(realpath, filename)) - yield (realpath, filename) - -def UnresolveSymlinks(output_filename): - # type: (str) -> None - """Replace realpath of instrumented files with the relative path in the lcov output. - - Though we are asking coveragepy to use relative file names, currently - ignore that for purposes of generating the lcov report (and other reports - which are not the XML report), so we need to go and fix up the report. - - This function is a workaround for that issue. Once that issue is fixed - upstream and the updated version is widely in use, this should be removed. - - See https://github.com/nedbat/coveragepy/issues/963. - """ - substitutions = list(InstrumentedFilePaths()) - if substitutions: - unfixed_file = output_filename + '.tmp' - os.rename(output_filename, unfixed_file) - with open(unfixed_file, "r") as unfixed: - with open(output_filename, "w") as output_file: - for line in unfixed: - if line.startswith('SF:'): - for (realpath, filename) in substitutions: - line = line.replace(realpath, filename) - output_file.write(line) - os.unlink(unfixed_file) - def ExecuteFile(python_program, main_filename, args, env, module_space, - coverage_entrypoint, workspace, delete_module_space): + workspace, delete_module_space): # type: (str, str, list[str], dict[str, str], str, str|None, str|None) -> ... """Executes the given Python file using the various environment settings. @@ -359,12 +299,19 @@ def ExecuteFile(python_program, main_filename, args, env, module_space, args: (list[str]) Additional args to pass to the Python file env: (dict[str, str]) A dict of environment variables to set for the execution module_space: (str) Path to the module space/runfiles tree directory - coverage_entrypoint: (str|None) Path to the coverage tool entry point file. workspace: (str|None) Name of the workspace to execute in. This is expected to be a directory under the runfiles tree. delete_module_space: (bool), True if the module space should be deleted after a successful (exit code zero) program run, False if not. """ + argv = [python_program] + argv.extend(INTERPRETER_ARGS) + additional_interpreter_args = os.environ.pop("RULES_PYTHON_ADDITIONAL_INTERPRETER_ARGS", "") + if additional_interpreter_args: + import shlex + argv.extend(shlex.split(additional_interpreter_args)) + argv.append(main_filename) + argv.extend(args) # We want to use os.execv instead of subprocess.call, which causes # problems with signal passing (making it difficult to kill # Bazel). However, these conditions force us to run via @@ -378,21 +325,15 @@ def ExecuteFile(python_program, main_filename, args, env, module_space, # - If we may need to emit a host config warning after execution, we # can't execv because we need control to return here. This only # happens for targets built in the host config. - # - For coverage targets, at least coveragepy requires running in - # two invocations, which also requires control to return here. # - if not (IsWindows() or workspace or coverage_entrypoint or delete_module_space): - _RunExecv(python_program, main_filename, args, env) + if not (IsWindows() or workspace or delete_module_space): + _RunExecv(python_program, argv, env) - if coverage_entrypoint is not None: - ret_code = _RunForCoverage(python_program, main_filename, args, env, - coverage_entrypoint, workspace) - else: - ret_code = subprocess.call( - [python_program, main_filename] + args, - env=env, - cwd=workspace - ) + ret_code = subprocess.call( + argv, + env=env, + cwd=workspace + ) if delete_module_space: # NOTE: dirname() is called because CreateModuleSpace() creates a @@ -401,94 +342,15 @@ def ExecuteFile(python_program, main_filename, args, env, module_space, shutil.rmtree(os.path.dirname(module_space), True) sys.exit(ret_code) -def _RunExecv(python_program, main_filename, args, env): - # type: (str, str, list[str], dict[str, str]) -> ... +def _RunExecv(python_program, argv, env): + # type: (str, list[str], dict[str, str]) -> ... """Executes the given Python file using the various environment settings.""" os.environ.update(env) print_verbose("RunExecv: environ:", mapping=os.environ) - argv = [python_program, main_filename] + args - print_verbose("RunExecv: argv:", python_program, argv) + print_verbose("RunExecv: python:", python_program) + print_verbose("RunExecv: argv:", values=argv) os.execv(python_program, argv) -def _RunForCoverage(python_program, main_filename, args, env, - coverage_entrypoint, workspace): - # type: (str, str, list[str], dict[str, str], str, str|None) -> int - """Collects coverage infomration for the given Python file. - - Args: - python_program: (str) Path to the Python binary to use for execution - main_filename: (str) The Python file to execute - args: (list[str]) Additional args to pass to the Python file - env: (dict[str, str]) A dict of environment variables to set for the execution - coverage_entrypoint: (str|None) Path to the coverage entry point to execute with. - workspace: (str|None) Name of the workspace to execute in. This is expected to be a - directory under the runfiles tree, and will recursively delete the - runfiles directory if set. - """ - instrumented_files = [abs_path for abs_path, _ in InstrumentedFilePaths()] - unique_dirs = {os.path.dirname(file) for file in instrumented_files} - source = "\n\t".join(unique_dirs) - - PrintVerboseCoverage("[coveragepy] Instrumented Files:\n" + "\n".join(instrumented_files)) - PrintVerboseCoverage("[coveragepy] Sources:\n" + "\n".join(unique_dirs)) - - # We need for coveragepy to use relative paths. This can only be configured - unique_id = uuid.uuid4() - rcfile_name = os.path.join(os.environ['COVERAGE_DIR'], ".coveragerc_{}".format(unique_id)) - with open(rcfile_name, "w") as rcfile: - rcfile.write('''[run] -relative_files = True -source = -\t{source} -'''.format(source=source)) - PrintVerboseCoverage('Coverage entrypoint:', coverage_entrypoint) - # First run the target Python file via coveragepy to create a .coverage - # database file, from which we can later export lcov. - ret_code = subprocess.call( - [ - python_program, - coverage_entrypoint, - "run", - "--rcfile=" + rcfile_name, - "--append", - "--branch", - main_filename - ] + args, - env=env, - cwd=workspace - ) - PrintVerboseCoverage('Return code of coverage run:', ret_code) - output_filename = os.path.join(os.environ['COVERAGE_DIR'], 'pylcov.dat') - - PrintVerboseCoverage('Converting coveragepy database to lcov:', output_filename) - # Run coveragepy again to convert its .coverage database file into lcov. - # Under normal conditions running lcov outputs to stdout/stderr, which causes problems for `coverage`. - params = [python_program, coverage_entrypoint, "lcov", "--rcfile=" + rcfile_name, "-o", output_filename, "--quiet"] - kparams = {"env": env, "cwd": workspace, "stdout": subprocess.DEVNULL, "stderr": subprocess.DEVNULL} - if IsVerboseCoverage(): - # reconnect stdout/stderr to lcov generation. Should be useful for debugging `coverage` issues. - params.remove("--quiet") - kparams['stdout'] = sys.stderr - kparams['stderr'] = sys.stderr - - lcov_ret_code = subprocess.call( - params, - **kparams - ) - PrintVerboseCoverage('Return code of coverage lcov:', lcov_ret_code) - ret_code = lcov_ret_code or ret_code - - try: - os.unlink(rcfile_name) - except OSError as err: - # It's possible that the profiled program might execute another Python - # binary through a wrapper that would then delete the rcfile. Not much - # we can do about that, besides ignore the failure here. - PrintVerboseCoverage('Error removing temporary coverage rc file:', err) - if os.path.isfile(output_filename): - UnresolveSymlinks(output_filename) - return ret_code - def Main(): print_verbose("initial argv:", values=sys.argv) print_verbose("initial cwd:", os.getcwd()) @@ -498,16 +360,12 @@ def Main(): new_env = {} - # The main Python source file. - # The magic string percent-main-percent is replaced with the runfiles-relative - # filename of the main file of the Python binary in BazelPythonSemantics.java. - main_rel_path = '%main%' # NOTE: We call normpath for two reasons: # 1. Transform Bazel `foo/bar` to Windows `foo\bar` # 2. Transform `_main/../foo/main.py` to simply `foo/main.py`, which # matters if `_main` doesn't exist (which can occur if a binary # is packaged and needs no artifacts from the main repo) - main_rel_path = os.path.normpath(main_rel_path) + main_rel_path = os.path.normpath(STAGE2_BOOTSTRAP) if IsRunningFromZip(): module_space = CreateModuleSpace() @@ -519,26 +377,6 @@ def Main(): if os.environ.get("RULES_PYTHON_TESTING_TELL_MODULE_SPACE"): new_env["RULES_PYTHON_TESTING_MODULE_SPACE"] = module_space - python_imports = '%imports%' - python_path_entries = CreatePythonPathEntries(python_imports, module_space) - python_path_entries += GetRepositoriesImports(module_space, %import_all%) - # Remove duplicates to avoid overly long PYTHONPATH (#10977). Preserve order, - # keep first occurrence only. - python_path_entries = [ - GetWindowsPathWithUNCPrefix(d) - for d in python_path_entries - ] - - old_python_path = os.environ.get('PYTHONPATH') - if old_python_path: - python_path_entries += old_python_path.split(os.pathsep) - - python_path = os.pathsep.join(Deduplicate(python_path_entries)) - - if IsWindows(): - python_path = python_path.replace('/', os.sep) - - new_env['PYTHONPATH'] = python_path runfiles_envkey, runfiles_envvalue = RunfilesEnvvar(module_space) if runfiles_envkey: new_env[runfiles_envkey] = runfiles_envvalue @@ -556,39 +394,7 @@ def Main(): program = python_program = FindPythonBinary(module_space) if python_program is None: - raise AssertionError('Could not find python binary: ' + PYTHON_BINARY) - - # COVERAGE_DIR is set if coverage is enabled and instrumentation is configured - # for something, though it could be another program executing this one or - # one executed by this one (e.g. an extension module). - if os.environ.get('COVERAGE_DIR'): - cov_tool = FindCoverageEntryPoint(module_space) - if cov_tool is None: - PrintVerboseCoverage('Coverage was enabled, but python coverage tool was not configured.') - else: - # Inhibit infinite recursion: - if 'PYTHON_COVERAGE' in os.environ: - del os.environ['PYTHON_COVERAGE'] - - if not os.path.exists(cov_tool): - raise EnvironmentError( - 'Python coverage tool %r not found. ' - 'Try running with VERBOSE_COVERAGE=1 to collect more information.' - % cov_tool - ) - - # coverage library expects sys.path[0] to contain the library, and replaces - # it with the directory of the program it starts. Our actual sys.path[0] is - # the runfiles directory, which must not be replaced. - # CoverageScript.do_execute() undoes this sys.path[0] setting. - # - # Update sys.path such that python finds the coverage package. The coverage - # entry point is coverage.coverage_main, so we need to do twice the dirname. - python_path_entries = new_env['PYTHONPATH'].split(os.pathsep) - python_path_entries.append(os.path.dirname(os.path.dirname(cov_tool))) - new_env['PYTHONPATH'] = os.pathsep.join(Deduplicate(python_path_entries)) - else: - cov_tool = None + raise AssertionError('Could not find python binary: ' + repr(PYTHON_BINARY)) # Some older Python versions on macOS (namely Python 3.7) may unintentionally # leave this environment variable set after starting the interpreter, which @@ -605,14 +411,14 @@ def Main(): # change directory to the right runfiles directory. # (So that the data files are accessible) if os.environ.get('RUN_UNDER_RUNFILES') == '1': - workspace = os.path.join(module_space, '%workspace_name%') + workspace = os.path.join(module_space, WORKSPACE_NAME) try: sys.stdout.flush() # NOTE: ExecuteFile may call execve() and lines after this will never run. ExecuteFile( python_program, main_filename, args, new_env, module_space, - cov_tool, workspace, + workspace, delete_module_space = delete_module_space, ) diff --git a/python/private/stage2_bootstrap_template.py b/python/private/stage2_bootstrap_template.py index 689602d3aa..4d98b03846 100644 --- a/python/private/stage2_bootstrap_template.py +++ b/python/private/stage2_bootstrap_template.py @@ -32,6 +32,9 @@ # Module name to execute. Empty if MAIN is used. MAIN_MODULE = "%main_module%" +# runfiles-root relative path to the root of the venv +VENV_ROOT = "%venv_root%" + # venv-relative path to the expected location of the binary's site-packages # directory. # Only set when the toolchain doesn't support the build-time venv. Empty @@ -66,7 +69,7 @@ def get_windows_path_with_unc_prefix(path): break except (ValueError, KeyError): pass - if win32_version and win32_version >= '10.0.14393': + if win32_version and win32_version >= "10.0.14393": return path # import sysconfig only now to maintain python 2.6 compatibility @@ -373,28 +376,33 @@ def _maybe_collect_coverage(enable): print_verbose_coverage("Error removing temporary coverage rc file:", err) +def _add_site_packages(site_packages): + first_global_offset = len(sys.path) + for i, p in enumerate(sys.path): + # We assume the first *-packages is the runtime's. + # *-packages is matched because Debian may use dist-packages + # instead of site-packages. + if p.endswith("-packages"): + first_global_offset = i + break + prev_len = len(sys.path) + import site + + site.addsitedir(site_packages) + added_dirs = sys.path[prev_len:] + del sys.path[prev_len:] + # Re-insert the binary specific paths so the order is + # (stdlib, binary specific, runtime site) + # This matches what a venv's ordering is like. + sys.path[first_global_offset:0] = added_dirs + + def main(): print_verbose("initial argv:", values=sys.argv) print_verbose("initial cwd:", os.getcwd()) print_verbose("initial environ:", mapping=os.environ) print_verbose("initial sys.path:", values=sys.path) - if VENV_SITE_PACKAGES: - site_packages = os.path.join(sys.prefix, VENV_SITE_PACKAGES) - if site_packages not in sys.path and os.path.exists(site_packages): - # NOTE: if this happens, it likely means we're running with a different - # Python version than was built with. Things may or may not work. - # Such a situation is likely due to the runtime_env toolchain, or some - # toolchain configuration. In any case, this better matches how the - # previous bootstrap=system_python bootstrap worked (using PYTHONPATH, - # which isn't version-specific). - print_verbose( - f"sys.path missing expected site-packages: adding {site_packages}" - ) - import site - - site.addsitedir(site_packages) - main_rel_path = None # todo: things happen to work because find_runfiles_root # ends up using stage2_bootstrap, and ends up computing the proper @@ -408,6 +416,23 @@ def main(): else: runfiles_root = find_runfiles_root("") + site_packages = os.path.join(runfiles_root, VENV_ROOT, VENV_SITE_PACKAGES) + if site_packages not in sys.path and os.path.exists(site_packages): + # This can happen in a few situations: + # 1. We're running with a different Python version than was built with. + # Things may or may not work. Such a situation is likely due to the + # runtime_env toolchain, or some toolchain configuration. In any + # case, this better matches how the previous bootstrap=system_python + # bootstrap worked (using PYTHONPATH, which isn't version-specific). + # 2. If site is disabled (`-S` interpreter arg). Some users do this to + # prevent interference from the system. + # 3. If running without a venv configured. This occurs with the + # system_python bootstrap. + print_verbose( + f"sys.path missing expected site-packages: adding {site_packages}" + ) + _add_site_packages(site_packages) + print_verbose("runfiles root:", runfiles_root) runfiles_envkey, runfiles_envvalue = runfiles_envvar(runfiles_root) diff --git a/python/private/zip_main_template.py b/python/private/zip_main_template.py index 5ec5ba07fa..d1489b46aa 100644 --- a/python/private/zip_main_template.py +++ b/python/private/zip_main_template.py @@ -25,13 +25,38 @@ # runfiles-relative path _STAGE2_BOOTSTRAP = "%stage2_bootstrap%" -# runfiles-relative path +# runfiles-relative path to venv's bin/python3. Empty if venv not being used. _PYTHON_BINARY = "%python_binary%" -# runfiles-relative path, absolute path, or single word +# runfiles-relative path, absolute path, or single word. The actual Python +# executable to use. _PYTHON_BINARY_ACTUAL = "%python_binary_actual%" _WORKSPACE_NAME = "%workspace_name%" +def print_verbose(*args, mapping=None, values=None): + if bool(os.environ.get("RULES_PYTHON_BOOTSTRAP_VERBOSE")): + if mapping is not None: + for key, value in sorted((mapping or {}).items()): + print( + "bootstrap: stage 1:", + *args, + f"{key}={value!r}", + file=sys.stderr, + flush=True, + ) + elif values is not None: + for i, v in enumerate(values): + print( + "bootstrap: stage 1:", + *args, + f"[{i}] {v!r}", + file=sys.stderr, + flush=True, + ) + else: + print("bootstrap: stage 1:", *args, file=sys.stderr, flush=True) + + # Return True if running on Windows def is_windows(): return os.name == "nt" @@ -76,7 +101,11 @@ def has_windows_executable_extension(path): return path.endswith(".exe") or path.endswith(".com") or path.endswith(".bat") -if is_windows() and not has_windows_executable_extension(_PYTHON_BINARY): +if ( + _PYTHON_BINARY + and is_windows() + and not has_windows_executable_extension(_PYTHON_BINARY) +): _PYTHON_BINARY = _PYTHON_BINARY + ".exe" @@ -93,31 +122,10 @@ def search_path(name): def find_python_binary(module_space): """Finds the real Python binary if it's not a normal absolute path.""" - return find_binary(module_space, _PYTHON_BINARY) - - -def print_verbose(*args, mapping=None, values=None): - if bool(os.environ.get("RULES_PYTHON_BOOTSTRAP_VERBOSE")): - if mapping is not None: - for key, value in sorted((mapping or {}).items()): - print( - "bootstrap: stage 1:", - *args, - f"{key}={value!r}", - file=sys.stderr, - flush=True, - ) - elif values is not None: - for i, v in enumerate(values): - print( - "bootstrap: stage 1:", - *args, - f"[{i}] {v!r}", - file=sys.stderr, - flush=True, - ) - else: - print("bootstrap: stage 1:", *args, file=sys.stderr, flush=True) + if _PYTHON_BINARY: + return find_binary(module_space, _PYTHON_BINARY) + else: + return find_binary(module_space, _PYTHON_BINARY_ACTUAL) def find_binary(module_space, bin_name): @@ -265,32 +273,34 @@ def main(): if python_program is None: raise AssertionError("Could not find python binary: " + _PYTHON_BINARY) - # The python interpreter should always be under runfiles, but double check. - # We don't want to accidentally create symlinks elsewhere. - if not python_program.startswith(module_space): - raise AssertionError( - "Program's venv binary not under runfiles: {python_program}" - ) - - if os.path.isabs(_PYTHON_BINARY_ACTUAL): - symlink_to = _PYTHON_BINARY_ACTUAL - elif "/" in _PYTHON_BINARY_ACTUAL: - symlink_to = os.path.join(module_space, _PYTHON_BINARY_ACTUAL) - else: - symlink_to = search_path(_PYTHON_BINARY_ACTUAL) - if not symlink_to: + # When a venv is used, the `bin/python3` symlink has to be recreated. + if _PYTHON_BINARY: + # The venv bin/python3 interpreter should always be under runfiles, but + # double check. We don't want to accidentally create symlinks elsewhere. + if not python_program.startswith(module_space): raise AssertionError( - f"Python interpreter to use not found on PATH: {_PYTHON_BINARY_ACTUAL}" + "Program's venv binary not under runfiles: {python_program}" ) - # The bin/ directory may not exist if it is empty. - os.makedirs(os.path.dirname(python_program), exist_ok=True) - try: - os.symlink(symlink_to, python_program) - except OSError as e: - raise Exception( - f"Unable to create venv python interpreter symlink: {python_program} -> {symlink_to}" - ) from e + if os.path.isabs(_PYTHON_BINARY_ACTUAL): + symlink_to = _PYTHON_BINARY_ACTUAL + elif "/" in _PYTHON_BINARY_ACTUAL: + symlink_to = os.path.join(module_space, _PYTHON_BINARY_ACTUAL) + else: + symlink_to = search_path(_PYTHON_BINARY_ACTUAL) + if not symlink_to: + raise AssertionError( + f"Python interpreter to use not found on PATH: {_PYTHON_BINARY_ACTUAL}" + ) + + # The bin/ directory may not exist if it is empty. + os.makedirs(os.path.dirname(python_program), exist_ok=True) + try: + os.symlink(symlink_to, python_program) + except OSError as e: + raise Exception( + f"Unable to create venv python interpreter symlink: {python_program} -> {symlink_to}" + ) from e # Some older Python versions on macOS (namely Python 3.7) may unintentionally # leave this environment variable set after starting the interpreter, which diff --git a/tests/base_rules/py_executable_base_tests.bzl b/tests/base_rules/py_executable_base_tests.bzl index 49cbb1586c..2b96451e35 100644 --- a/tests/base_rules/py_executable_base_tests.bzl +++ b/tests/base_rules/py_executable_base_tests.bzl @@ -359,12 +359,11 @@ def _test_main_module_bootstrap_system_python(name, config): "//command_line_option:extra_execution_platforms": ["@bazel_tools//tools:host_platform", LINUX_X86_64], "//command_line_option:platforms": [LINUX_X86_64], }, - expect_failure = True, ) def _test_main_module_bootstrap_system_python_impl(env, target): - env.expect.that_target(target).failures().contains_predicate( - matching.str_matches("mandatory*srcs"), + env.expect.that_target(target).default_outputs().contains( + "{package}/{test_name}_subject", ) _tests.append(_test_main_module_bootstrap_system_python) diff --git a/tests/bootstrap_impls/sys_path_order_test.py b/tests/bootstrap_impls/sys_path_order_test.py index 97c62a6be5..9ae03bb129 100644 --- a/tests/bootstrap_impls/sys_path_order_test.py +++ b/tests/bootstrap_impls/sys_path_order_test.py @@ -73,25 +73,15 @@ def test_sys_path_order(self): + f"for sys.path:\n{sys_path_str}" ) - if os.environ["BOOTSTRAP"] == "script": - self.assertTrue( - last_stdlib < first_user < first_runtime_site, - "Expected overall order to be (stdlib, user imports, runtime site) " - + f"with {last_stdlib=} < {first_user=} < {first_runtime_site=}\n" - + f"for sys.prefix={sys.prefix}\n" - + f"for sys.exec_prefix={sys.exec_prefix}\n" - + f"for sys.base_prefix={sys.base_prefix}\n" - + f"for sys.path:\n{sys_path_str}", - ) - else: - self.assertTrue( - first_user < last_stdlib < first_runtime_site, - f"Expected {first_user=} < {last_stdlib=} < {first_runtime_site=}\n" - + f"for sys.prefix={sys.prefix}\n" - + f"for sys.exec_prefix={sys.exec_prefix}\n" - + f"for sys.base_prefix={sys.base_prefix}\n" - + f"for sys.path:\n{sys_path_str}", - ) + self.assertTrue( + last_stdlib < first_user < first_runtime_site, + "Expected overall order to be (stdlib, user imports, runtime site) " + + f"with {last_stdlib=} < {first_user=} < {first_runtime_site=}\n" + + f"for sys.prefix={sys.prefix}\n" + + f"for sys.exec_prefix={sys.exec_prefix}\n" + + f"for sys.base_prefix={sys.base_prefix}\n" + + f"for sys.path:\n{sys_path_str}", + ) if __name__ == "__main__": From 7b88c87aaab1e4711a8b61d2a47f445052ed6e9a Mon Sep 17 00:00:00 2001 From: Ignas Anikevicius <240938+aignas@users.noreply.github.com> Date: Sun, 7 Sep 2025 02:57:35 +0900 Subject: [PATCH 424/922] refactor(pypi): split out a hub_builder helper from the extension code (#3243) This is a somewhat tedious refactor, where I am just moving code around (and sometimes renaming various parameters). I am not modifying and/or fixing any bugs other than more error messages in one place since I noticed there was a lack of validation. The main idea is to create a `hub_builder` so that we could also use it for `pip.configure` calls and/or use it for `py.lock` file parsing and reuse code. I hope that moving it to a separate file makes it a little bit more obvious what pieces are used to create a hub repository. What is more, since the pip extension is reproducible, I have removed some code that was sorting the output. Work towards #2747 --------- Co-authored-by: Richard Levasseur --- python/private/pypi/BUILD.bazel | 32 +- python/private/pypi/extension.bzl | 573 +++------------------------ python/private/pypi/hub_builder.bzl | 581 ++++++++++++++++++++++++++++ 3 files changed, 650 insertions(+), 536 deletions(-) create mode 100644 python/private/pypi/hub_builder.bzl diff --git a/python/private/pypi/BUILD.bazel b/python/private/pypi/BUILD.bazel index cb3408a191..fd850857e9 100644 --- a/python/private/pypi/BUILD.bazel +++ b/python/private/pypi/BUILD.bazel @@ -109,22 +109,17 @@ bzl_library( name = "extension_bzl", srcs = ["extension.bzl"], deps = [ - ":attrs_bzl", ":evaluate_markers_bzl", + ":hub_builder_bzl", ":hub_repository_bzl", - ":parse_requirements_bzl", ":parse_whl_name_bzl", ":pep508_env_bzl", ":pip_repository_attrs_bzl", - ":python_tag_bzl", ":simpleapi_download_bzl", - ":whl_config_setting_bzl", ":whl_library_bzl", - ":whl_repo_name_bzl", - "//python/private:full_version_bzl", + "//python/private:auth_bzl", "//python/private:normalize_name_bzl", - "//python/private:version_bzl", - "//python/private:version_label_bzl", + "//python/private:repo_utils_bzl", "@bazel_features//:features", "@pythons_hub//:interpreters_bzl", "@pythons_hub//:versions_bzl", @@ -167,6 +162,27 @@ bzl_library( ], ) +bzl_library( + name = "hub_builder_bzl", + srcs = ["hub_builder.bzl"], + visibility = ["//:__subpackages__"], + deps = [ + ":attrs_bzl", + ":evaluate_markers_bzl", + ":parse_requirements_bzl", + ":pep508_env_bzl", + ":pep508_evaluate_bzl", + ":python_tag_bzl", + ":requirements_files_by_platform_bzl", + ":whl_config_setting_bzl", + ":whl_repo_name_bzl", + "//python/private:full_version_bzl", + "//python/private:normalize_name_bzl", + "//python/private:version_bzl", + "//python/private:version_label_bzl", + ], +) + bzl_library( name = "hub_repository_bzl", srcs = ["hub_repository.bzl"], diff --git a/python/private/pypi/extension.bzl b/python/private/pypi/extension.bzl index 03af863e1e..c73e88ac0d 100644 --- a/python/private/pypi/extension.bzl +++ b/python/private/pypi/extension.bzl @@ -19,29 +19,16 @@ load("@pythons_hub//:interpreters.bzl", "INTERPRETER_LABELS") load("@pythons_hub//:versions.bzl", "MINOR_MAPPING") load("@rules_python_internal//:rules_python_config.bzl", rp_config = "config") load("//python/private:auth.bzl", "AUTH_ATTRS") -load("//python/private:full_version.bzl", "full_version") load("//python/private:normalize_name.bzl", "normalize_name") load("//python/private:repo_utils.bzl", "repo_utils") -load("//python/private:version.bzl", "version") -load("//python/private:version_label.bzl", "version_label") -load(":attrs.bzl", "use_isolated") -load(":evaluate_markers.bzl", "evaluate_markers_py", EVALUATE_MARKERS_SRCS = "SRCS", evaluate_markers_star = "evaluate_markers") +load(":evaluate_markers.bzl", EVALUATE_MARKERS_SRCS = "SRCS") +load(":hub_builder.bzl", "hub_builder") load(":hub_repository.bzl", "hub_repository", "whl_config_settings_to_json") -load(":parse_requirements.bzl", "parse_requirements") load(":parse_whl_name.bzl", "parse_whl_name") load(":pep508_env.bzl", "env") -load(":pep508_evaluate.bzl", "evaluate") load(":pip_repository_attrs.bzl", "ATTRS") -load(":python_tag.bzl", "python_tag") -load(":requirements_files_by_platform.bzl", "requirements_files_by_platform") load(":simpleapi_download.bzl", "simpleapi_download") -load(":whl_config_setting.bzl", "whl_config_setting") load(":whl_library.bzl", "whl_library") -load(":whl_repo_name.bzl", "pypi_repo_name", "whl_repo_name") - -def _major_minor_version(version_str): - ver = version.parse(version_str) - return "{}.{}".format(ver.release[0], ver.release[1]) def _whl_mods_impl(whl_mods_dict): """Implementation of the pip.whl_mods tag class. @@ -68,396 +55,6 @@ def _whl_mods_impl(whl_mods_dict): whl_mods = whl_mods, ) -def _platforms(*, python_version, minor_mapping, config): - platforms = {} - python_version = version.parse( - full_version( - version = python_version, - minor_mapping = minor_mapping, - ), - strict = True, - ) - - for platform, values in config.platforms.items(): - # TODO @aignas 2025-07-07: this is probably doing the parsing of the version too - # many times. - abi = "{}{}{}.{}".format( - python_tag(values.env["implementation_name"]), - python_version.release[0], - python_version.release[1], - python_version.release[2], - ) - key = "{}_{}".format(abi, platform) - - env_ = env( - env = values.env, - os = values.os_name, - arch = values.arch_name, - python_version = python_version.string, - ) - - if values.marker and not evaluate(values.marker, env = env_): - continue - - platforms[key] = struct( - env = env_, - triple = "{}_{}_{}".format(abi, values.os_name, values.arch_name), - whl_abi_tags = [ - v.format( - major = python_version.release[0], - minor = python_version.release[1], - ) - for v in values.whl_abi_tags - ], - whl_platform_tags = values.whl_platform_tags, - ) - return platforms - -def _create_whl_repos( - module_ctx, - *, - pip_attr, - whl_overrides, - config, - available_interpreters = INTERPRETER_LABELS, - minor_mapping = MINOR_MAPPING, - evaluate_markers = None, - get_index_urls = None): - """create all of the whl repositories - - Args: - module_ctx: {type}`module_ctx`. - pip_attr: {type}`struct` - the struct that comes from the tag class iteration. - whl_overrides: {type}`dict[str, struct]` - per-wheel overrides. - config: The platform configuration. - get_index_urls: A function used to get the index URLs - available_interpreters: {type}`dict[str, Label]` The dictionary of available - interpreters that have been registered using the `python` bzlmod extension. - The keys are in the form `python_{snake_case_version}_host`. This is to be - used during the `repository_rule` and must be always compatible with the host. - minor_mapping: {type}`dict[str, str]` The dictionary needed to resolve the full - python version used to parse package METADATA files. - evaluate_markers: the function used to evaluate the markers. - - Returns a {type}`struct` with the following attributes: - whl_map: {type}`dict[str, list[struct]]` the output is keyed by the - normalized package name and the values are the instances of the - {bzl:obj}`whl_config_setting` return values. - exposed_packages: {type}`dict[str, Any]` this is just a way to - represent a set of string values. - whl_libraries: {type}`dict[str, dict[str, Any]]` the keys are the - aparent repository names for the hub repo and the values are the - arguments that will be passed to {bzl:obj}`whl_library` repository - rule. - """ - logger = repo_utils.logger(module_ctx, "pypi:create_whl_repos") - python_interpreter_target = pip_attr.python_interpreter_target - - # containers to aggregate outputs from this function - whl_map = {} - extra_aliases = { - whl_name: {alias: True for alias in aliases} - for whl_name, aliases in pip_attr.extra_hub_aliases.items() - } - whl_libraries = {} - - # if we do not have the python_interpreter set in the attributes - # we programmatically find it. - hub_name = pip_attr.hub_name - if python_interpreter_target == None and not pip_attr.python_interpreter: - python_name = "python_{}_host".format( - pip_attr.python_version.replace(".", "_"), - ) - if python_name not in available_interpreters: - fail(( - "Unable to find interpreter for pip hub '{hub_name}' for " + - "python_version={version}: Make sure a corresponding " + - '`python.toolchain(python_version="{version}")` call exists.' + - "Expected to find {python_name} among registered versions:\n {labels}" - ).format( - hub_name = hub_name, - version = pip_attr.python_version, - python_name = python_name, - labels = " \n".join(available_interpreters), - )) - python_interpreter_target = available_interpreters[python_name] - - # TODO @aignas 2025-06-29: we should not need the version in the pip_name if - # we are using pipstar and we are downloading the wheel using the downloader - pip_name = "{}_{}".format( - hub_name, - version_label(pip_attr.python_version), - ) - major_minor = _major_minor_version(pip_attr.python_version) - - whl_modifications = {} - if pip_attr.whl_modifications != None: - for mod, whl_name in pip_attr.whl_modifications.items(): - whl_modifications[normalize_name(whl_name)] = mod - - if pip_attr.experimental_requirement_cycles: - requirement_cycles = { - name: [normalize_name(whl_name) for whl_name in whls] - for name, whls in pip_attr.experimental_requirement_cycles.items() - } - - whl_group_mapping = { - whl_name: group_name - for group_name, group_whls in requirement_cycles.items() - for whl_name in group_whls - } - else: - whl_group_mapping = {} - requirement_cycles = {} - - platforms = _platforms( - python_version = pip_attr.python_version, - minor_mapping = minor_mapping, - config = config, - ) - - if evaluate_markers: - # This is most likely unit tests - pass - elif config.enable_pipstar: - evaluate_markers = lambda _, requirements: evaluate_markers_star( - requirements = requirements, - platforms = platforms, - ) - else: - # NOTE @aignas 2024-08-02: , we will execute any interpreter that we find either - # in the PATH or if specified as a label. We will configure the env - # markers when evaluating the requirement lines based on the output - # from the `requirements_files_by_platform` which should have something - # similar to: - # { - # "//:requirements.txt": ["cp311_linux_x86_64", ...] - # } - # - # We know the target python versions that we need to evaluate the - # markers for and thus we don't need to use multiple python interpreter - # instances to perform this manipulation. This function should be executed - # only once by the underlying code to minimize the overhead needed to - # spin up a Python interpreter. - evaluate_markers = lambda module_ctx, requirements: evaluate_markers_py( - module_ctx, - requirements = { - k: { - p: platforms[p].triple - for p in plats - } - for k, plats in requirements.items() - }, - python_interpreter = pip_attr.python_interpreter, - python_interpreter_target = python_interpreter_target, - srcs = pip_attr._evaluate_markers_srcs, - logger = logger, - ) - - requirements_by_platform = parse_requirements( - module_ctx, - requirements_by_platform = requirements_files_by_platform( - requirements_by_platform = pip_attr.requirements_by_platform, - requirements_linux = pip_attr.requirements_linux, - requirements_lock = pip_attr.requirements_lock, - requirements_osx = pip_attr.requirements_darwin, - requirements_windows = pip_attr.requirements_windows, - extra_pip_args = pip_attr.extra_pip_args, - platforms = sorted(platforms), # here we only need keys - python_version = full_version( - version = pip_attr.python_version, - minor_mapping = minor_mapping, - ), - logger = logger, - ), - platforms = platforms, - extra_pip_args = pip_attr.extra_pip_args, - get_index_urls = get_index_urls, - evaluate_markers = evaluate_markers, - logger = logger, - ) - - use_downloader = { - normalize_name(s): False - for s in pip_attr.simpleapi_skip - } - exposed_packages = {} - for whl in requirements_by_platform: - if whl.is_exposed: - exposed_packages[whl.name] = None - - group_name = whl_group_mapping.get(whl.name) - group_deps = requirement_cycles.get(group_name, []) - - # Construct args separately so that the lock file can be smaller and does not include unused - # attrs. - whl_library_args = dict( - dep_template = "@{}//{{name}}:{{target}}".format(hub_name), - ) - maybe_args = dict( - # The following values are safe to omit if they have false like values - add_libdir_to_library_search_path = pip_attr.add_libdir_to_library_search_path, - annotation = whl_modifications.get(whl.name), - download_only = pip_attr.download_only, - enable_implicit_namespace_pkgs = pip_attr.enable_implicit_namespace_pkgs, - environment = pip_attr.environment, - envsubst = pip_attr.envsubst, - group_deps = group_deps, - group_name = group_name, - pip_data_exclude = pip_attr.pip_data_exclude, - python_interpreter = pip_attr.python_interpreter, - python_interpreter_target = python_interpreter_target, - whl_patches = { - p: json.encode(args) - for p, args in whl_overrides.get(whl.name, {}).items() - }, - ) - if not config.enable_pipstar: - maybe_args["experimental_target_platforms"] = pip_attr.experimental_target_platforms - - whl_library_args.update({k: v for k, v in maybe_args.items() if v}) - maybe_args_with_default = dict( - # The following values have defaults next to them - isolated = (use_isolated(module_ctx, pip_attr), True), - quiet = (pip_attr.quiet, True), - timeout = (pip_attr.timeout, 600), - ) - whl_library_args.update({ - k: v - for k, (v, default) in maybe_args_with_default.items() - if v != default - }) - - for src in whl.srcs: - repo = _whl_repo( - src = src, - whl_library_args = whl_library_args, - download_only = pip_attr.download_only, - netrc = config.netrc or pip_attr.netrc, - use_downloader = use_downloader.get( - whl.name, - get_index_urls != None, # defaults to True if the get_index_urls is defined - ), - auth_patterns = config.auth_patterns or pip_attr.auth_patterns, - python_version = major_minor, - is_multiple_versions = whl.is_multiple_versions, - enable_pipstar = config.enable_pipstar, - ) - if repo == None: - # NOTE @aignas 2025-07-07: we guard against an edge-case where there - # are more platforms defined than there are wheels for and users - # disallow building from sdist. - continue - - repo_name = "{}_{}".format(pip_name, repo.repo_name) - if repo_name in whl_libraries: - fail("attempting to create a duplicate library {} for {}".format( - repo_name, - whl.name, - )) - whl_libraries[repo_name] = repo.args - - if not config.enable_pipstar and "experimental_target_platforms" in repo.args: - whl_libraries[repo_name] |= { - "experimental_target_platforms": sorted({ - # TODO @aignas 2025-07-07: this should be solved in a better way - platforms[candidate].triple.partition("_")[-1]: None - for p in repo.args["experimental_target_platforms"] - for candidate in platforms - if candidate.endswith(p) - }), - } - - mapping = whl_map.setdefault(whl.name, {}) - if repo.config_setting in mapping and mapping[repo.config_setting] != repo_name: - fail( - "attempting to override an existing repo '{}' for config setting '{}' with a new repo '{}'".format( - mapping[repo.config_setting], - repo.config_setting, - repo_name, - ), - ) - else: - mapping[repo.config_setting] = repo_name - - return struct( - whl_map = whl_map, - exposed_packages = exposed_packages, - extra_aliases = extra_aliases, - whl_libraries = whl_libraries, - ) - -def _whl_repo( - *, - src, - whl_library_args, - is_multiple_versions, - download_only, - netrc, - auth_patterns, - python_version, - use_downloader, - enable_pipstar = False): - args = dict(whl_library_args) - args["requirement"] = src.requirement_line - is_whl = src.filename.endswith(".whl") - - if src.extra_pip_args and not is_whl: - # pip is not used to download wheels and the python - # `whl_library` helpers are only extracting things, however - # for sdists, they will be built by `pip`, so we still - # need to pass the extra args there, so only pop this for whls - args["extra_pip_args"] = src.extra_pip_args - - if not src.url or (not is_whl and download_only): - if download_only and use_downloader: - # If the user did not allow using sdists and we are using the downloader - # and we are not using simpleapi_skip for this - return None - else: - # Fallback to a pip-installed wheel - target_platforms = src.target_platforms if is_multiple_versions else [] - return struct( - repo_name = pypi_repo_name( - normalize_name(src.distribution), - *target_platforms - ), - args = args, - config_setting = whl_config_setting( - version = python_version, - target_platforms = target_platforms or None, - ), - ) - - # This is no-op because pip is not used to download the wheel. - args.pop("download_only", None) - - if netrc: - args["netrc"] = netrc - if auth_patterns: - args["auth_patterns"] = auth_patterns - - args["urls"] = [src.url] - args["sha256"] = src.sha256 - args["filename"] = src.filename - if not enable_pipstar: - args["experimental_target_platforms"] = [ - # Get rid of the version for the target platforms because we are - # passing the interpreter any way. Ideally we should search of ways - # how to pass the target platforms through the hub repo. - p.partition("_")[2] - for p in src.target_platforms - ] - - return struct( - repo_name = whl_repo_name(src.filename, src.sha256), - args = args, - config_setting = whl_config_setting( - version = python_version, - target_platforms = src.target_platforms, - ), - ) - def _plat(*, name, arch_name, os_name, config_settings = [], env = {}, marker = "", whl_abi_tags = [], whl_platform_tags = []): # NOTE @aignas 2025-07-08: the least preferred is the first item in the list if "any" not in whl_platform_tags: @@ -571,7 +168,7 @@ def parse_modules( enable_pipstar: {type}`bool` a flag to enable dropping Python dependency for evaluation of the extension. _fail: {type}`function` the failure function, mainly for testing. - **kwargs: Extra arguments passed to the layers below. + **kwargs: Extra arguments passed to the hub_builder. Returns: A struct with the following attributes: @@ -645,23 +242,24 @@ You cannot use both the additive_build_content and additive_build_content_file a pip_hub_map = {} simpleapi_cache = {} - # Keeps track of all the hub's whl repos across the different versions. - # dict[hub, dict[whl, dict[version, str pip]]] - # Where hub, whl, and pip are the repo names - hub_whl_map = {} - hub_group_map = {} - exposed_packages = {} - extra_aliases = {} - whl_libraries = {} - for mod in module_ctx.modules: for pip_attr in mod.tags.parse: hub_name = pip_attr.hub_name if hub_name not in pip_hub_map: - pip_hub_map[pip_attr.hub_name] = struct( + builder = hub_builder( + name = hub_name, module_name = mod.name, - python_versions = [pip_attr.python_version], + config = config, + whl_overrides = whl_overrides, + simpleapi_download_fn = simpleapi_download, + simpleapi_cache = simpleapi_cache, + # TODO @aignas 2025-09-06: do not use kwargs + minor_mapping = kwargs.get("minor_mapping", MINOR_MAPPING), + evaluate_markers_fn = kwargs.get("evaluate_markers", None), + available_interpreters = kwargs.get("available_interpreters", INTERPRETER_LABELS), + logger = repo_utils.logger(module_ctx, "pypi:hub:" + hub_name), ) + pip_hub_map[pip_attr.hub_name] = builder elif pip_hub_map[hub_name].module_name != mod.name: # We cannot have two hubs with the same name in different # modules. @@ -676,120 +274,44 @@ You cannot use both the additive_build_content and additive_build_content_file a second_module = mod.name, )) - elif pip_attr.python_version in pip_hub_map[hub_name].python_versions: - fail(( - "Duplicate pip python version '{version}' for hub " + - "'{hub}' in module '{module}': the Python versions " + - "used for a hub must be unique" - ).format( - hub = hub_name, - module = mod.name, - version = pip_attr.python_version, - )) else: - pip_hub_map[pip_attr.hub_name].python_versions.append(pip_attr.python_version) - - get_index_urls = None - if pip_attr.experimental_index_url: - skip_sources = [ - normalize_name(s) - for s in pip_attr.simpleapi_skip - ] - get_index_urls = lambda ctx, distributions: simpleapi_download( - ctx, - attr = struct( - index_url = pip_attr.experimental_index_url, - extra_index_urls = pip_attr.experimental_extra_index_urls or [], - index_url_overrides = pip_attr.experimental_index_url_overrides or {}, - sources = [ - d - for d in distributions - if normalize_name(d) not in skip_sources - ], - envsubst = pip_attr.envsubst, - # Auth related info - netrc = pip_attr.netrc, - auth_patterns = pip_attr.auth_patterns, - ), - cache = simpleapi_cache, - parallel_download = pip_attr.parallel_download, - ) - elif pip_attr.experimental_extra_index_urls: - fail("'experimental_extra_index_urls' is a no-op unless 'experimental_index_url' is set") - elif pip_attr.experimental_index_url_overrides: - fail("'experimental_index_url_overrides' is a no-op unless 'experimental_index_url' is set") + builder = pip_hub_map[pip_attr.hub_name] - # TODO @aignas 2025-05-19: express pip.parse as a series of configure calls - out = _create_whl_repos( + builder.pip_parse( module_ctx, pip_attr = pip_attr, - get_index_urls = get_index_urls, - whl_overrides = whl_overrides, - config = config, - **kwargs ) - hub_whl_map.setdefault(hub_name, {}) - for key, settings in out.whl_map.items(): - for setting, repo in settings.items(): - hub_whl_map[hub_name].setdefault(key, {}).setdefault(repo, []).append(setting) - extra_aliases.setdefault(hub_name, {}) - for whl_name, aliases in out.extra_aliases.items(): - extra_aliases[hub_name].setdefault(whl_name, {}).update(aliases) - - if hub_name not in exposed_packages: - exposed_packages[hub_name] = out.exposed_packages + + # Keeps track of all the hub's whl repos across the different versions. + # dict[hub, dict[whl, dict[version, str pip]]] + # Where hub, whl, and pip are the repo names + hub_whl_map = {} + hub_group_map = {} + exposed_packages = {} + extra_aliases = {} + whl_libraries = {} + for hub in pip_hub_map.values(): + out = hub.build() + + for whl_name, lib in out.whl_libraries.items(): + if whl_name in whl_libraries: + fail("'{}' already in created".format(whl_name)) else: - intersection = {} - for pkg in out.exposed_packages: - if pkg not in exposed_packages[hub_name]: - continue - intersection[pkg] = None - exposed_packages[hub_name] = intersection - whl_libraries.update(out.whl_libraries) - for whl_name, lib in out.whl_libraries.items(): - if enable_pipstar: - whl_libraries.setdefault(whl_name, lib) - elif whl_name in lib: - fail("'{}' already in created".format(whl_name)) - else: - # replicate whl_libraries.update(out.whl_libraries) - whl_libraries[whl_name] = lib - - # TODO @aignas 2024-04-05: how do we support different requirement - # cycles for different abis/oses? For now we will need the users to - # assume the same groups across all versions/platforms until we start - # using an alternative cycle resolution strategy. - hub_group_map[hub_name] = pip_attr.experimental_requirement_cycles + whl_libraries[whl_name] = lib + + exposed_packages[hub.name] = out.exposed_packages + extra_aliases[hub.name] = out.extra_aliases + hub_group_map[hub.name] = out.group_map + hub_whl_map[hub.name] = out.whl_map return struct( - # We sort so that the lock-file remains the same no matter the order of how the - # args are manipulated in the code going before. - whl_mods = dict(sorted(whl_mods.items())), - hub_whl_map = { - hub_name: { - whl_name: dict(settings) - for whl_name, settings in sorted(whl_map.items()) - } - for hub_name, whl_map in sorted(hub_whl_map.items()) - }, - hub_group_map = { - hub_name: { - key: sorted(values) - for key, values in sorted(group_map.items()) - } - for hub_name, group_map in sorted(hub_group_map.items()) - }, - exposed_packages = { - k: sorted(v) - for k, v in sorted(exposed_packages.items()) - }, - extra_aliases = { - hub_name: { - whl_name: sorted(aliases) - for whl_name, aliases in extra_whl_aliases.items() - } - for hub_name, extra_whl_aliases in extra_aliases.items() - }, + config = config, + exposed_packages = exposed_packages, + extra_aliases = extra_aliases, + hub_group_map = hub_group_map, + hub_whl_map = hub_whl_map, + whl_libraries = whl_libraries, + whl_mods = whl_mods, platform_config_settings = { hub_name: { platform_name: sorted([str(Label(cv)) for cv in p.config_settings]) @@ -797,11 +319,6 @@ You cannot use both the additive_build_content and additive_build_content_file a } for hub_name in hub_whl_map }, - whl_libraries = { - k: dict(sorted(args.items())) - for k, args in sorted(whl_libraries.items()) - }, - config = config, ) def _pip_impl(module_ctx): diff --git a/python/private/pypi/hub_builder.bzl b/python/private/pypi/hub_builder.bzl new file mode 100644 index 0000000000..b6088e4ded --- /dev/null +++ b/python/private/pypi/hub_builder.bzl @@ -0,0 +1,581 @@ +"""A hub repository builder for incrementally building the hub configuration.""" + +load("//python/private:full_version.bzl", "full_version") +load("//python/private:normalize_name.bzl", "normalize_name") +load("//python/private:version.bzl", "version") +load("//python/private:version_label.bzl", "version_label") +load(":attrs.bzl", "use_isolated") +load(":evaluate_markers.bzl", "evaluate_markers_py", evaluate_markers_star = "evaluate_markers") +load(":parse_requirements.bzl", "parse_requirements") +load(":pep508_env.bzl", "env") +load(":pep508_evaluate.bzl", "evaluate") +load(":python_tag.bzl", "python_tag") +load(":requirements_files_by_platform.bzl", "requirements_files_by_platform") +load(":whl_config_setting.bzl", "whl_config_setting") +load(":whl_repo_name.bzl", "pypi_repo_name", "whl_repo_name") + +def _major_minor_version(version_str): + ver = version.parse(version_str) + return "{}.{}".format(ver.release[0], ver.release[1]) + +def hub_builder( + *, + name, + module_name, + config, + whl_overrides, + minor_mapping, + available_interpreters, + simpleapi_download_fn, + evaluate_markers_fn, + logger, + simpleapi_cache = {}): + """Return a hub builder instance + + Args: + name: {type}`str`, the name of the hub. + module_name: {type}`str`, the module name that has created the hub. + config: The platform configuration. + whl_overrides: {type}`dict[str, struct]` - per-wheel overrides. + minor_mapping: {type}`dict[str, str]` the mapping between minor and full versions. + evaluate_markers_fn: the override function used to evaluate the markers. + available_interpreters: {type}`dict[str, Label]` The dictionary of available + interpreters that have been registered using the `python` bzlmod extension. + The keys are in the form `python_{snake_case_version}_host`. This is to be + used during the `repository_rule` and must be always compatible with the host. + simpleapi_download_fn: the function used to download from SimpleAPI. + simpleapi_cache: the cache for the download results. + logger: the logger for this builder. + """ + + # buildifier: disable=uninitialized + self = struct( + name = name, + module_name = module_name, + + # public methods, keep sorted and to minimum + build = lambda: _build(self), + pip_parse = lambda *a, **k: _pip_parse(self, *a, **k), + + # build output + _exposed_packages = {}, # modified by _add_exposed_packages + _extra_aliases = {}, # modified by _add_extra_aliases + _group_map = {}, # modified by _add_group_map + _whl_libraries = {}, # modified by _add_whl_library + _whl_map = {}, # modified by _add_whl_library + # internal + _platforms = {}, + _group_name_by_whl = {}, + _get_index_urls = {}, + _use_downloader = {}, + _simpleapi_cache = simpleapi_cache, + # instance constants + _config = config, + _whl_overrides = whl_overrides, + _evaluate_markers_fn = evaluate_markers_fn, + _logger = logger, + _minor_mapping = minor_mapping, + _available_interpreters = available_interpreters, + _simpleapi_download_fn = simpleapi_download_fn, + ) + + # buildifier: enable=uninitialized + return self + +### PUBLIC methods + +def _build(self): + whl_map = {} + for key, settings in self._whl_map.items(): + for setting, repo in settings.items(): + whl_map.setdefault(key, {}).setdefault(repo, []).append(setting) + + return struct( + whl_map = whl_map, + group_map = self._group_map, + extra_aliases = { + whl: sorted(aliases) + for whl, aliases in self._extra_aliases.items() + }, + exposed_packages = sorted(self._exposed_packages), + whl_libraries = self._whl_libraries, + ) + +def _pip_parse(self, module_ctx, pip_attr): + python_version = pip_attr.python_version + if python_version in self._platforms: + fail(( + "Duplicate pip python version '{version}' for hub " + + "'{hub}' in module '{module}': the Python versions " + + "used for a hub must be unique" + ).format( + hub = self.name, + module = self.module_name, + version = python_version, + )) + + self._platforms[python_version] = _platforms( + python_version = python_version, + minor_mapping = self._minor_mapping, + config = self._config, + ) + _set_get_index_urls(self, pip_attr) + _add_group_map(self, pip_attr.experimental_requirement_cycles) + _add_extra_aliases(self, pip_attr.extra_hub_aliases) + _create_whl_repos( + self, + module_ctx, + pip_attr = pip_attr, + ) + +### end of PUBLIC methods +### setters for build outputs + +def _add_exposed_packages(self, exposed_packages): + if self._exposed_packages: + intersection = {} + for pkg in exposed_packages: + if pkg not in self._exposed_packages: + continue + intersection[pkg] = None + self._exposed_packages.clear() + exposed_packages = intersection + + self._exposed_packages.update(exposed_packages) + +def _add_group_map(self, group_map): + # TODO @aignas 2024-04-05: how do we support different requirement + # cycles for different abis/oses? For now we will need the users to + # assume the same groups across all versions/platforms until we start + # using an alternative cycle resolution strategy. + group_map = { + name: [normalize_name(whl_name) for whl_name in whls] + for name, whls in group_map.items() + } + self._group_map.clear() + self._group_name_by_whl.clear() + + self._group_map.update(group_map) + self._group_name_by_whl.update({ + whl_name: group_name + for group_name, group_whls in self._group_map.items() + for whl_name in group_whls + }) + +def _add_extra_aliases(self, extra_hub_aliases): + for whl_name, aliases in extra_hub_aliases.items(): + self._extra_aliases.setdefault(whl_name, {}).update( + {alias: True for alias in aliases}, + ) + +def _add_whl_library(self, *, python_version, whl, repo): + if repo == None: + # NOTE @aignas 2025-07-07: we guard against an edge-case where there + # are more platforms defined than there are wheels for and users + # disallow building from sdist. + return + + platforms = self._platforms[python_version] + + # TODO @aignas 2025-06-29: we should not need the version in the repo_name if + # we are using pipstar and we are downloading the wheel using the downloader + repo_name = "{}_{}_{}".format(self.name, version_label(python_version), repo.repo_name) + + if repo_name in self._whl_libraries: + fail("attempting to create a duplicate library {} for {}".format( + repo_name, + whl.name, + )) + self._whl_libraries[repo_name] = repo.args + + if not self._config.enable_pipstar and "experimental_target_platforms" in repo.args: + self._whl_libraries[repo_name] |= { + "experimental_target_platforms": sorted({ + # TODO @aignas 2025-07-07: this should be solved in a better way + platforms[candidate].triple.partition("_")[-1]: None + for p in repo.args["experimental_target_platforms"] + for candidate in platforms + if candidate.endswith(p) + }), + } + + mapping = self._whl_map.setdefault(whl.name, {}) + if repo.config_setting in mapping and mapping[repo.config_setting] != repo_name: + fail( + "attempting to override an existing repo '{}' for config setting '{}' with a new repo '{}'".format( + mapping[repo.config_setting], + repo.config_setting, + repo_name, + ), + ) + else: + mapping[repo.config_setting] = repo_name + +### end of setters, below we have various functions to implement the public methods + +def _set_get_index_urls(self, pip_attr): + if not pip_attr.experimental_index_url: + if pip_attr.experimental_extra_index_urls: + fail("'experimental_extra_index_urls' is a no-op unless 'experimental_index_url' is set") + elif pip_attr.experimental_index_url_overrides: + fail("'experimental_index_url_overrides' is a no-op unless 'experimental_index_url' is set") + elif pip_attr.simpleapi_skip: + fail("'simpleapi_skip' is a no-op unless 'experimental_index_url' is set") + elif pip_attr.netrc: + fail("'netrc' is a no-op unless 'experimental_index_url' is set") + elif pip_attr.auth_patterns: + fail("'auth_patterns' is a no-op unless 'experimental_index_url' is set") + + # parallel_download is set to True by default, so we are not checking/validating it + # here + return + + python_version = pip_attr.python_version + self._use_downloader.setdefault(python_version, {}).update({ + normalize_name(s): False + for s in pip_attr.simpleapi_skip + }) + self._get_index_urls[python_version] = lambda ctx, distributions: self._simpleapi_download_fn( + ctx, + attr = struct( + index_url = pip_attr.experimental_index_url, + extra_index_urls = pip_attr.experimental_extra_index_urls or [], + index_url_overrides = pip_attr.experimental_index_url_overrides or {}, + sources = [ + d + for d in distributions + if _use_downloader(self, python_version, d) + ], + envsubst = pip_attr.envsubst, + # Auth related info + netrc = pip_attr.netrc, + auth_patterns = pip_attr.auth_patterns, + ), + cache = self._simpleapi_cache, + parallel_download = pip_attr.parallel_download, + ) + +def _detect_interpreter(self, pip_attr): + python_interpreter_target = pip_attr.python_interpreter_target + if python_interpreter_target == None and not pip_attr.python_interpreter: + python_name = "python_{}_host".format( + pip_attr.python_version.replace(".", "_"), + ) + if python_name not in self._available_interpreters: + fail(( + "Unable to find interpreter for pip hub '{hub_name}' for " + + "python_version={version}: Make sure a corresponding " + + '`python.toolchain(python_version="{version}")` call exists.' + + "Expected to find {python_name} among registered versions:\n {labels}" + ).format( + hub_name = self.name, + version = pip_attr.python_version, + python_name = python_name, + labels = " \n".join(self._available_interpreters), + )) + python_interpreter_target = self._available_interpreters[python_name] + + return struct( + target = python_interpreter_target, + path = pip_attr.python_interpreter, + ) + +def _platforms(*, python_version, minor_mapping, config): + platforms = {} + python_version = version.parse( + full_version( + version = python_version, + minor_mapping = minor_mapping, + ), + strict = True, + ) + + for platform, values in config.platforms.items(): + # TODO @aignas 2025-07-07: this is probably doing the parsing of the version too + # many times. + abi = "{}{}{}.{}".format( + python_tag(values.env["implementation_name"]), + python_version.release[0], + python_version.release[1], + python_version.release[2], + ) + key = "{}_{}".format(abi, platform) + + env_ = env( + env = values.env, + os = values.os_name, + arch = values.arch_name, + python_version = python_version.string, + ) + + if values.marker and not evaluate(values.marker, env = env_): + continue + + platforms[key] = struct( + env = env_, + triple = "{}_{}_{}".format(abi, values.os_name, values.arch_name), + whl_abi_tags = [ + v.format( + major = python_version.release[0], + minor = python_version.release[1], + ) + for v in values.whl_abi_tags + ], + whl_platform_tags = values.whl_platform_tags, + ) + return platforms + +def _evaluate_markers(self, pip_attr): + if self._evaluate_markers_fn: + return self._evaluate_markers_fn + + if self._config.enable_pipstar: + return lambda _, requirements: evaluate_markers_star( + requirements = requirements, + platforms = self._platforms[pip_attr.python_version], + ) + + interpreter = _detect_interpreter(self, pip_attr) + + # NOTE @aignas 2024-08-02: , we will execute any interpreter that we find either + # in the PATH or if specified as a label. We will configure the env + # markers when evaluating the requirement lines based on the output + # from the `requirements_files_by_platform` which should have something + # similar to: + # { + # "//:requirements.txt": ["cp311_linux_x86_64", ...] + # } + # + # We know the target python versions that we need to evaluate the + # markers for and thus we don't need to use multiple python interpreter + # instances to perform this manipulation. This function should be executed + # only once by the underlying code to minimize the overhead needed to + # spin up a Python interpreter. + return lambda module_ctx, requirements: evaluate_markers_py( + module_ctx, + requirements = { + k: { + p: self._platforms[pip_attr.python_version][p].triple + for p in plats + } + for k, plats in requirements.items() + }, + python_interpreter = interpreter.path, + python_interpreter_target = interpreter.target, + srcs = pip_attr._evaluate_markers_srcs, + logger = self._logger, + ) + +def _create_whl_repos( + self, + module_ctx, + *, + pip_attr): + """create all of the whl repositories + + Args: + self: the builder. + module_ctx: {type}`module_ctx`. + pip_attr: {type}`struct` - the struct that comes from the tag class iteration. + """ + logger = self._logger + platforms = self._platforms[pip_attr.python_version] + requirements_by_platform = parse_requirements( + module_ctx, + requirements_by_platform = requirements_files_by_platform( + requirements_by_platform = pip_attr.requirements_by_platform, + requirements_linux = pip_attr.requirements_linux, + requirements_lock = pip_attr.requirements_lock, + requirements_osx = pip_attr.requirements_darwin, + requirements_windows = pip_attr.requirements_windows, + extra_pip_args = pip_attr.extra_pip_args, + platforms = sorted(platforms), # here we only need keys + python_version = full_version( + version = pip_attr.python_version, + minor_mapping = self._minor_mapping, + ), + logger = logger, + ), + platforms = platforms, + extra_pip_args = pip_attr.extra_pip_args, + get_index_urls = self._get_index_urls.get(pip_attr.python_version), + evaluate_markers = _evaluate_markers(self, pip_attr), + logger = logger, + ) + + _add_exposed_packages(self, { + whl.name: None + for whl in requirements_by_platform + if whl.is_exposed + }) + + whl_modifications = {} + if pip_attr.whl_modifications != None: + for mod, whl_name in pip_attr.whl_modifications.items(): + whl_modifications[normalize_name(whl_name)] = mod + + common_args = _common_args( + self, + module_ctx, + pip_attr = pip_attr, + ) + for whl in requirements_by_platform: + whl_library_args = common_args | _whl_library_args( + self, + whl = whl, + whl_modifications = whl_modifications, + ) + for src in whl.srcs: + repo = _whl_repo( + src = src, + whl_library_args = whl_library_args, + download_only = pip_attr.download_only, + netrc = self._config.netrc or pip_attr.netrc, + use_downloader = _use_downloader(self, pip_attr.python_version, whl.name), + auth_patterns = self._config.auth_patterns or pip_attr.auth_patterns, + python_version = _major_minor_version(pip_attr.python_version), + is_multiple_versions = whl.is_multiple_versions, + enable_pipstar = self._config.enable_pipstar, + ) + _add_whl_library( + self, + python_version = pip_attr.python_version, + whl = whl, + repo = repo, + ) + +def _common_args(self, module_ctx, *, pip_attr): + interpreter = _detect_interpreter(self, pip_attr) + + # Construct args separately so that the lock file can be smaller and does not include unused + # attrs. + whl_library_args = dict( + dep_template = "@{}//{{name}}:{{target}}".format(self.name), + ) + maybe_args = dict( + # The following values are safe to omit if they have false like values + add_libdir_to_library_search_path = pip_attr.add_libdir_to_library_search_path, + download_only = pip_attr.download_only, + enable_implicit_namespace_pkgs = pip_attr.enable_implicit_namespace_pkgs, + environment = pip_attr.environment, + envsubst = pip_attr.envsubst, + pip_data_exclude = pip_attr.pip_data_exclude, + python_interpreter = interpreter.path, + python_interpreter_target = interpreter.target, + ) + if not self._config.enable_pipstar: + maybe_args["experimental_target_platforms"] = pip_attr.experimental_target_platforms + + whl_library_args.update({k: v for k, v in maybe_args.items() if v}) + maybe_args_with_default = dict( + # The following values have defaults next to them + isolated = (use_isolated(module_ctx, pip_attr), True), + quiet = (pip_attr.quiet, True), + timeout = (pip_attr.timeout, 600), + ) + whl_library_args.update({ + k: v + for k, (v, default) in maybe_args_with_default.items() + if v != default + }) + return whl_library_args + +def _whl_library_args(self, *, whl, whl_modifications): + group_name = self._group_name_by_whl.get(whl.name) + group_deps = self._group_map.get(group_name, []) + + # Construct args separately so that the lock file can be smaller and does not include unused + # attrs. + whl_library_args = dict( + dep_template = "@{}//{{name}}:{{target}}".format(self.name), + ) + maybe_args = dict( + # The following values are safe to omit if they have false like values + annotation = whl_modifications.get(whl.name), + group_deps = group_deps, + group_name = group_name, + whl_patches = { + p: json.encode(args) + for p, args in self._whl_overrides.get(whl.name, {}).items() + }, + ) + + whl_library_args.update({k: v for k, v in maybe_args.items() if v}) + return whl_library_args + +def _whl_repo( + *, + src, + whl_library_args, + is_multiple_versions, + download_only, + netrc, + auth_patterns, + python_version, + use_downloader, + enable_pipstar = False): + args = dict(whl_library_args) + args["requirement"] = src.requirement_line + is_whl = src.filename.endswith(".whl") + + if src.extra_pip_args and not is_whl: + # pip is not used to download wheels and the python + # `whl_library` helpers are only extracting things, however + # for sdists, they will be built by `pip`, so we still + # need to pass the extra args there, so only pop this for whls + args["extra_pip_args"] = src.extra_pip_args + + if not src.url or (not is_whl and download_only): + if download_only and use_downloader: + # If the user did not allow using sdists and we are using the downloader + # and we are not using simpleapi_skip for this + return None + else: + # Fallback to a pip-installed wheel + target_platforms = src.target_platforms if is_multiple_versions else [] + return struct( + repo_name = pypi_repo_name( + normalize_name(src.distribution), + *target_platforms + ), + args = args, + config_setting = whl_config_setting( + version = python_version, + target_platforms = target_platforms or None, + ), + ) + + # This is no-op because pip is not used to download the wheel. + args.pop("download_only", None) + + if netrc: + args["netrc"] = netrc + if auth_patterns: + args["auth_patterns"] = auth_patterns + + args["urls"] = [src.url] + args["sha256"] = src.sha256 + args["filename"] = src.filename + if not enable_pipstar: + args["experimental_target_platforms"] = [ + # Get rid of the version for the target platforms because we are + # passing the interpreter any way. Ideally we should search of ways + # how to pass the target platforms through the hub repo. + p.partition("_")[2] + for p in src.target_platforms + ] + + return struct( + repo_name = whl_repo_name(src.filename, src.sha256), + args = args, + config_setting = whl_config_setting( + version = python_version, + target_platforms = src.target_platforms, + ), + ) + +def _use_downloader(self, python_version, whl_name): + return self._use_downloader.get(python_version, {}).get( + normalize_name(whl_name), + self._get_index_urls.get(python_version) != None, + ) From e8d9cabbaaf4d1dabee9359c786b1dd1536013f5 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Sat, 6 Sep 2025 15:07:25 -0700 Subject: [PATCH 425/922] chore: add GEMINI.md, have it load AGENTS.md (#3246) Apparently, Gemini doesn't automatically process AGENTS.md files. This can be worked around by creating GEMINI.md and telling it to read the AGENTS.md file. --------- Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> --- GEMINI.md | 1 + 1 file changed, 1 insertion(+) create mode 100644 GEMINI.md diff --git a/GEMINI.md b/GEMINI.md new file mode 100644 index 0000000000..285e0f5b36 --- /dev/null +++ b/GEMINI.md @@ -0,0 +1 @@ +@./AGENTS.md From 5467ed6ae811e2e296ab960165e36f7285127465 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Sat, 6 Sep 2025 19:22:24 -0700 Subject: [PATCH 426/922] docs: fix pr doc builds by removing external_version_warning plugin (#3244) Doc builds for PR were failing because the readthedocs_ext.external_version_warning plugin wasn't handling something correctly. Activating it manually was originally done to get the warning banners to appear, but it looks like RTD now displays a warning banner without this special plugin being needed. Since it's now unnecessary, remove the code that can activate it. --- docs/conf.py | 20 -------------------- 1 file changed, 20 deletions(-) diff --git a/docs/conf.py b/docs/conf.py index 8537d9996c..47ab378cfb 100644 --- a/docs/conf.py +++ b/docs/conf.py @@ -102,26 +102,6 @@ # to the original conf.py template comments extensions.insert(0, "readthedocs_ext.readthedocs") - if os.environ.get("READTHEDOCS_VERSION_TYPE") == "external": - # Insert after the main extension - extensions.insert(1, "readthedocs_ext.external_version_warning") - readthedocs_vcs_url = ( - "http://github.com/bazel-contrib/rules_python/pull/{}".format( - os.environ.get("READTHEDOCS_VERSION", "") - ) - ) - # The build id isn't directly available, but it appears to be encoded - # into the host name, so we can parse it from that. The format appears - # to be `build-X-project-Y-Z`, where: - # * X is an integer build id - # * Y is an integer project id - # * Z is the project name - _build_id = os.environ.get("HOSTNAME", "build-0-project-0-rules-python") - _build_id = _build_id.split("-")[1] - readthedocs_build_url = ( - f"https://readthedocs.org/projects/rules-python/builds/{_build_id}" - ) - exclude_patterns = ["_includes/*"] templates_path = ["_templates"] primary_domain = None # The default is 'py', which we don't make much use of From 9ba8c127a111f9695087ce22cb00c78cf75f5ec1 Mon Sep 17 00:00:00 2001 From: Ignas Anikevicius <240938+aignas@users.noreply.github.com> Date: Mon, 8 Sep 2025 17:22:31 +0900 Subject: [PATCH 427/922] refactor: migrate tests to use hub_builder instead of full integration (#3247) This PR migrates some of the tests that we had testing the full extension parsing to just test the hub builder. I ran out of time to migrate everything and there is a little bit of copy pasted code. The goal is to make the assertions easier to understand because the nesting of the dictionaries will be not as large. Later we can add tests that are testing individual `hub_builder` methods, which will be needed when we start implementing the `pip.configure` tag class or we implement a `py.lock` parsing. Work towards #2747 --------- Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> --- python/private/pypi/BUILD.bazel | 6 + python/private/pypi/extension.bzl | 26 +- python/private/pypi/platform.bzl | 45 + tests/pypi/extension/extension_tests.bzl | 1036 +---------------- tests/pypi/extension/pip_parse.bzl | 70 ++ tests/pypi/hub_builder/BUILD.bazel | 3 + tests/pypi/hub_builder/hub_builder_tests.bzl | 1082 ++++++++++++++++++ 7 files changed, 1208 insertions(+), 1060 deletions(-) create mode 100644 python/private/pypi/platform.bzl create mode 100644 tests/pypi/extension/pip_parse.bzl create mode 100644 tests/pypi/hub_builder/BUILD.bazel create mode 100644 tests/pypi/hub_builder/hub_builder_tests.bzl diff --git a/python/private/pypi/BUILD.bazel b/python/private/pypi/BUILD.bazel index fd850857e9..c7a74ee306 100644 --- a/python/private/pypi/BUILD.bazel +++ b/python/private/pypi/BUILD.bazel @@ -115,6 +115,7 @@ bzl_library( ":parse_whl_name_bzl", ":pep508_env_bzl", ":pip_repository_attrs_bzl", + ":platform_bzl", ":simpleapi_download_bzl", ":whl_library_bzl", "//python/private:auth_bzl", @@ -341,6 +342,11 @@ bzl_library( ], ) +bzl_library( + name = "platform_bzl", + srcs = ["platform.bzl"], +) + bzl_library( name = "pypi_repo_utils_bzl", srcs = ["pypi_repo_utils.bzl"], diff --git a/python/private/pypi/extension.bzl b/python/private/pypi/extension.bzl index c73e88ac0d..4708c8e53a 100644 --- a/python/private/pypi/extension.bzl +++ b/python/private/pypi/extension.bzl @@ -27,6 +27,7 @@ load(":hub_repository.bzl", "hub_repository", "whl_config_settings_to_json") load(":parse_whl_name.bzl", "parse_whl_name") load(":pep508_env.bzl", "env") load(":pip_repository_attrs.bzl", "ATTRS") +load(":platform.bzl", _plat = "platform") load(":simpleapi_download.bzl", "simpleapi_download") load(":whl_library.bzl", "whl_library") @@ -55,31 +56,6 @@ def _whl_mods_impl(whl_mods_dict): whl_mods = whl_mods, ) -def _plat(*, name, arch_name, os_name, config_settings = [], env = {}, marker = "", whl_abi_tags = [], whl_platform_tags = []): - # NOTE @aignas 2025-07-08: the least preferred is the first item in the list - if "any" not in whl_platform_tags: - # the lowest priority one needs to be the first one - whl_platform_tags = ["any"] + whl_platform_tags - - whl_abi_tags = whl_abi_tags or ["abi3", "cp{major}{minor}"] - if "none" not in whl_abi_tags: - # the lowest priority one needs to be the first one - whl_abi_tags = ["none"] + whl_abi_tags - - return struct( - name = name, - arch_name = arch_name, - os_name = os_name, - config_settings = config_settings, - env = { - # defaults for env - "implementation_name": "cpython", - } | env, - marker = marker, - whl_abi_tags = whl_abi_tags, - whl_platform_tags = whl_platform_tags, - ) - def _configure(config, *, override = False, **kwargs): """Set the value in the config if the value is provided""" env = kwargs.get("env") diff --git a/python/private/pypi/platform.bzl b/python/private/pypi/platform.bzl new file mode 100644 index 0000000000..e8f36a980b --- /dev/null +++ b/python/private/pypi/platform.bzl @@ -0,0 +1,45 @@ +"""A common platform structure for using internally.""" + +def platform(*, name, arch_name, os_name, config_settings = [], env = {}, marker = "", whl_abi_tags = [], whl_platform_tags = []): + """A platform structure for using internally. + + Args: + name: {type}`str` the human friendly name of the platform. + arch_name: {type}`str` the @platforms//cpu: value. + os_name: {type}`str` the @platforms//os: value. + config_settings: {type}`list[Label|str]` The list of labels for selecting the + platform. + env: {type}`dict[str, str]` the PEP508 environment for marker evaluation. + marker: {type}`str` the env marker expression that is evaluated to determine if we + should use the platform. This is useful to turn on certain platforms for + particular python versions. + whl_abi_tags: {type}`list[str]` A list of values for matching abi tags. + whl_platform_tags: {type}`list[str]` A list of values for matching platform tags. + + Returns: + struct with the necessary values for pipstar implementation. + """ + + # NOTE @aignas 2025-07-08: the least preferred is the first item in the list + if "any" not in whl_platform_tags: + # the lowest priority one needs to be the first one + whl_platform_tags = ["any"] + whl_platform_tags + + whl_abi_tags = whl_abi_tags or ["abi3", "cp{major}{minor}"] + if "none" not in whl_abi_tags: + # the lowest priority one needs to be the first one + whl_abi_tags = ["none"] + whl_abi_tags + + return struct( + name = name, + arch_name = arch_name, + os_name = os_name, + config_settings = config_settings, + env = { + # defaults for env + "implementation_name": "cpython", + } | env, + marker = marker, + whl_abi_tags = whl_abi_tags, + whl_platform_tags = whl_platform_tags, + ) diff --git a/tests/pypi/extension/extension_tests.bzl b/tests/pypi/extension/extension_tests.bzl index 55de99b7d9..0514e1d95b 100644 --- a/tests/pypi/extension/extension_tests.bzl +++ b/tests/pypi/extension/extension_tests.bzl @@ -17,8 +17,8 @@ load("@rules_testing//lib:test_suite.bzl", "test_suite") load("@rules_testing//lib:truth.bzl", "subjects") load("//python/private/pypi:extension.bzl", "build_config", "parse_modules") # buildifier: disable=bzl-visibility -load("//python/private/pypi:parse_simpleapi_html.bzl", "parse_simpleapi_html") # buildifier: disable=bzl-visibility load("//python/private/pypi:whl_config_setting.bzl", "whl_config_setting") # buildifier: disable=bzl-visibility +load(":pip_parse.bzl", _parse = "pip_parse") _tests = [] @@ -134,74 +134,6 @@ def _default( whl_platform_tags = whl_platform_tags or [], ) -def _parse( - *, - hub_name, - python_version, - add_libdir_to_library_search_path = False, - auth_patterns = {}, - download_only = False, - enable_implicit_namespace_pkgs = False, - environment = {}, - envsubst = {}, - experimental_index_url = "", - experimental_requirement_cycles = {}, - experimental_target_platforms = [], - extra_hub_aliases = {}, - extra_pip_args = [], - isolated = True, - netrc = None, - parse_all_requirements_files = True, - pip_data_exclude = None, - python_interpreter = None, - python_interpreter_target = None, - quiet = True, - requirements_by_platform = {}, - requirements_darwin = None, - requirements_linux = None, - requirements_lock = None, - requirements_windows = None, - simpleapi_skip = [], - timeout = 600, - whl_modifications = {}, - **kwargs): - return struct( - auth_patterns = auth_patterns, - add_libdir_to_library_search_path = add_libdir_to_library_search_path, - download_only = download_only, - enable_implicit_namespace_pkgs = enable_implicit_namespace_pkgs, - environment = environment, - envsubst = envsubst, - experimental_index_url = experimental_index_url, - experimental_requirement_cycles = experimental_requirement_cycles, - experimental_target_platforms = experimental_target_platforms, - extra_hub_aliases = extra_hub_aliases, - extra_pip_args = extra_pip_args, - hub_name = hub_name, - isolated = isolated, - netrc = netrc, - parse_all_requirements_files = parse_all_requirements_files, - pip_data_exclude = pip_data_exclude, - python_interpreter = python_interpreter, - python_interpreter_target = python_interpreter_target, - python_version = python_version, - quiet = quiet, - requirements_by_platform = requirements_by_platform, - requirements_darwin = requirements_darwin, - requirements_linux = requirements_linux, - requirements_lock = requirements_lock, - requirements_windows = requirements_windows, - timeout = timeout, - whl_modifications = whl_modifications, - # The following are covered by other unit tests - experimental_extra_index_urls = [], - parallel_download = False, - experimental_index_url_overrides = {}, - simpleapi_skip = simpleapi_skip, - _evaluate_markers_srcs = [], - **kwargs - ) - def _test_simple(env): pypi = _parse_modules( env, @@ -245,972 +177,6 @@ def _test_simple(env): _tests.append(_test_simple) -def _test_simple_multiple_requirements(env): - pypi = _parse_modules( - env, - module_ctx = _mock_mctx( - _mod( - name = "rules_python", - parse = [ - _parse( - hub_name = "pypi", - python_version = "3.15", - requirements_darwin = "darwin.txt", - requirements_windows = "win.txt", - ), - ], - ), - read = lambda x: { - "darwin.txt": "simple==0.0.2 --hash=sha256:deadb00f", - "win.txt": "simple==0.0.1 --hash=sha256:deadbeef", - }[x], - ), - available_interpreters = { - "python_3_15_host": "unit_test_interpreter_target", - }, - minor_mapping = {"3.15": "3.15.19"}, - ) - - pypi.exposed_packages().contains_exactly({"pypi": ["simple"]}) - pypi.hub_group_map().contains_exactly({"pypi": {}}) - pypi.hub_whl_map().contains_exactly({"pypi": { - "simple": { - "pypi_315_simple_osx_aarch64": [ - whl_config_setting( - target_platforms = [ - "cp315_osx_aarch64", - ], - version = "3.15", - ), - ], - "pypi_315_simple_windows_aarch64": [ - whl_config_setting( - target_platforms = [ - "cp315_windows_aarch64", - ], - version = "3.15", - ), - ], - }, - }}) - pypi.whl_libraries().contains_exactly({ - "pypi_315_simple_osx_aarch64": { - "dep_template": "@pypi//{name}:{target}", - "python_interpreter_target": "unit_test_interpreter_target", - "requirement": "simple==0.0.2 --hash=sha256:deadb00f", - }, - "pypi_315_simple_windows_aarch64": { - "dep_template": "@pypi//{name}:{target}", - "python_interpreter_target": "unit_test_interpreter_target", - "requirement": "simple==0.0.1 --hash=sha256:deadbeef", - }, - }) - pypi.whl_mods().contains_exactly({}) - -_tests.append(_test_simple_multiple_requirements) - -def _test_simple_multiple_python_versions(env): - pypi = _parse_modules( - env, - module_ctx = _mock_mctx( - _mod( - name = "rules_python", - parse = [ - _parse( - hub_name = "pypi", - python_version = "3.15", - requirements_lock = "requirements_3_15.txt", - ), - _parse( - hub_name = "pypi", - python_version = "3.16", - requirements_lock = "requirements_3_16.txt", - ), - ], - ), - read = lambda x: { - "requirements_3_15.txt": """ -simple==0.0.1 --hash=sha256:deadbeef -old-package==0.0.1 --hash=sha256:deadbaaf -""", - "requirements_3_16.txt": """ -simple==0.0.2 --hash=sha256:deadb00f -new-package==0.0.1 --hash=sha256:deadb00f2 -""", - }[x], - ), - available_interpreters = { - "python_3_15_host": "unit_test_interpreter_target", - "python_3_16_host": "unit_test_interpreter_target", - }, - minor_mapping = { - "3.15": "3.15.19", - "3.16": "3.16.9", - }, - ) - - pypi.exposed_packages().contains_exactly({"pypi": ["simple"]}) - pypi.hub_group_map().contains_exactly({"pypi": {}}) - pypi.hub_whl_map().contains_exactly({ - "pypi": { - "new_package": { - "pypi_316_new_package": [ - whl_config_setting( - version = "3.16", - ), - ], - }, - "old_package": { - "pypi_315_old_package": [ - whl_config_setting( - version = "3.15", - ), - ], - }, - "simple": { - "pypi_315_simple": [ - whl_config_setting( - version = "3.15", - ), - ], - "pypi_316_simple": [ - whl_config_setting( - version = "3.16", - ), - ], - }, - }, - }) - pypi.whl_libraries().contains_exactly({ - "pypi_315_old_package": { - "dep_template": "@pypi//{name}:{target}", - "python_interpreter_target": "unit_test_interpreter_target", - "requirement": "old-package==0.0.1 --hash=sha256:deadbaaf", - }, - "pypi_315_simple": { - "dep_template": "@pypi//{name}:{target}", - "python_interpreter_target": "unit_test_interpreter_target", - "requirement": "simple==0.0.1 --hash=sha256:deadbeef", - }, - "pypi_316_new_package": { - "dep_template": "@pypi//{name}:{target}", - "python_interpreter_target": "unit_test_interpreter_target", - "requirement": "new-package==0.0.1 --hash=sha256:deadb00f2", - }, - "pypi_316_simple": { - "dep_template": "@pypi//{name}:{target}", - "python_interpreter_target": "unit_test_interpreter_target", - "requirement": "simple==0.0.2 --hash=sha256:deadb00f", - }, - }) - pypi.whl_mods().contains_exactly({}) - -_tests.append(_test_simple_multiple_python_versions) - -def _test_simple_with_markers(env): - pypi = _parse_modules( - env, - module_ctx = _mock_mctx( - _mod( - name = "rules_python", - parse = [ - _parse( - hub_name = "pypi", - python_version = "3.15", - requirements_lock = "universal.txt", - ), - ], - ), - read = lambda x: { - "universal.txt": """\ -torch==2.4.1+cpu ; platform_machine == 'x86_64' -torch==2.4.1 ; platform_machine != 'x86_64' \ - --hash=sha256:deadbeef -""", - }[x], - ), - available_interpreters = { - "python_3_15_host": "unit_test_interpreter_target", - }, - minor_mapping = {"3.15": "3.15.19"}, - evaluate_markers = lambda _, requirements, **__: { - key: [ - platform - for platform in platforms - if ("x86_64" in platform and "platform_machine ==" in key) or ("x86_64" not in platform and "platform_machine !=" in key) - ] - for key, platforms in requirements.items() - }, - ) - - pypi.exposed_packages().contains_exactly({"pypi": ["torch"]}) - pypi.hub_group_map().contains_exactly({"pypi": {}}) - pypi.hub_whl_map().contains_exactly({"pypi": { - "torch": { - "pypi_315_torch_linux_aarch64_osx_aarch64_windows_aarch64": [ - whl_config_setting( - target_platforms = [ - "cp315_linux_aarch64", - "cp315_osx_aarch64", - "cp315_windows_aarch64", - ], - version = "3.15", - ), - ], - "pypi_315_torch_linux_x86_64_linux_x86_64_freethreaded": [ - whl_config_setting( - target_platforms = [ - "cp315_linux_x86_64", - "cp315_linux_x86_64_freethreaded", - ], - version = "3.15", - ), - ], - }, - }}) - pypi.whl_libraries().contains_exactly({ - "pypi_315_torch_linux_aarch64_osx_aarch64_windows_aarch64": { - "dep_template": "@pypi//{name}:{target}", - "python_interpreter_target": "unit_test_interpreter_target", - "requirement": "torch==2.4.1 --hash=sha256:deadbeef", - }, - "pypi_315_torch_linux_x86_64_linux_x86_64_freethreaded": { - "dep_template": "@pypi//{name}:{target}", - "python_interpreter_target": "unit_test_interpreter_target", - "requirement": "torch==2.4.1+cpu", - }, - }) - pypi.whl_mods().contains_exactly({}) - -_tests.append(_test_simple_with_markers) - -def _test_torch_experimental_index_url(env): - def mocksimpleapi_download(*_, **__): - return { - "torch": parse_simpleapi_html( - url = "https://torch.index", - content = """\ - torch-2.4.1+cpu-cp310-cp310-linux_x86_64.whl
- torch-2.4.1+cpu-cp310-cp310-win_amd64.whl
- torch-2.4.1+cpu-cp311-cp311-linux_x86_64.whl
- torch-2.4.1+cpu-cp311-cp311-win_amd64.whl
- torch-2.4.1+cpu-cp312-cp312-linux_x86_64.whl
- torch-2.4.1+cpu-cp312-cp312-win_amd64.whl
- torch-2.4.1+cpu-cp38-cp38-linux_x86_64.whl
- torch-2.4.1+cpu-cp38-cp38-win_amd64.whl
- torch-2.4.1+cpu-cp39-cp39-linux_x86_64.whl
- torch-2.4.1+cpu-cp39-cp39-win_amd64.whl
- torch-2.4.1-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
- torch-2.4.1-cp310-none-macosx_11_0_arm64.whl
- torch-2.4.1-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
- torch-2.4.1-cp311-none-macosx_11_0_arm64.whl
- torch-2.4.1-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
- torch-2.4.1-cp312-none-macosx_11_0_arm64.whl
- torch-2.4.1-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
- torch-2.4.1-cp38-none-macosx_11_0_arm64.whl
- torch-2.4.1-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
- torch-2.4.1-cp39-none-macosx_11_0_arm64.whl
-""", - ), - } - - pypi = _parse_modules( - env, - module_ctx = _mock_mctx( - _mod( - name = "rules_python", - default = [ - _default( - platform = "{}_{}".format(os, cpu), - os_name = os, - arch_name = cpu, - config_settings = [ - "@platforms//os:{}".format(os), - "@platforms//cpu:{}".format(cpu), - ], - whl_platform_tags = whl_platform_tags, - ) - for (os, cpu), whl_platform_tags in { - ("linux", "x86_64"): ["linux_x86_64", "manylinux_*_x86_64"], - ("linux", "aarch64"): ["linux_aarch64", "manylinux_*_aarch64"], - ("osx", "aarch64"): ["macosx_*_arm64"], - ("windows", "x86_64"): ["win_amd64"], - ("windows", "aarch64"): ["win_arm64"], # this should be ignored - }.items() - ], - parse = [ - _parse( - hub_name = "pypi", - python_version = "3.12", - download_only = True, - experimental_index_url = "https://torch.index", - requirements_lock = "universal.txt", - ), - ], - ), - read = lambda x: { - "universal.txt": """\ -torch==2.4.1 ; platform_machine != 'x86_64' \ - --hash=sha256:1495132f30f722af1a091950088baea383fe39903db06b20e6936fd99402803e \ - --hash=sha256:30be2844d0c939161a11073bfbaf645f1c7cb43f62f46cc6e4df1c119fb2a798 \ - --hash=sha256:36109432b10bd7163c9b30ce896f3c2cca1b86b9765f956a1594f0ff43091e2a \ - --hash=sha256:56ad2a760b7a7882725a1eebf5657abbb3b5144eb26bcb47b52059357463c548 \ - --hash=sha256:5fc1d4d7ed265ef853579caf272686d1ed87cebdcd04f2a498f800ffc53dab71 \ - --hash=sha256:72b484d5b6cec1a735bf3fa5a1c4883d01748698c5e9cfdbeb4ffab7c7987e0d \ - --hash=sha256:a38de2803ee6050309aac032676536c3d3b6a9804248537e38e098d0e14817ec \ - --hash=sha256:d36a8ef100f5bff3e9c3cea934b9e0d7ea277cb8210c7152d34a9a6c5830eadd \ - --hash=sha256:ddddbd8b066e743934a4200b3d54267a46db02106876d21cf31f7da7a96f98ea \ - --hash=sha256:fa27b048d32198cda6e9cff0bf768e8683d98743903b7e5d2b1f5098ded1d343 - # via -r requirements.in -torch==2.4.1+cpu ; platform_machine == 'x86_64' \ - --hash=sha256:0c0a7cc4f7c74ff024d5a5e21230a01289b65346b27a626f6c815d94b4b8c955 \ - --hash=sha256:1dd062d296fb78aa7cfab8690bf03704995a821b5ef69cfc807af5c0831b4202 \ - --hash=sha256:2b03e20f37557d211d14e3fb3f71709325336402db132a1e0dd8b47392185baf \ - --hash=sha256:330e780f478707478f797fdc82c2a96e9b8c5f60b6f1f57bb6ad1dd5b1e7e97e \ - --hash=sha256:3a570e5c553415cdbddfe679207327b3a3806b21c6adea14fba77684d1619e97 \ - --hash=sha256:3c99506980a2fb4b634008ccb758f42dd82f93ae2830c1e41f64536e310bf562 \ - --hash=sha256:76a6fe7b10491b650c630bc9ae328df40f79a948296b41d3b087b29a8a63cbad \ - --hash=sha256:833490a28ac156762ed6adaa7c695879564fa2fd0dc51bcf3fdb2c7b47dc55e6 \ - --hash=sha256:8800deef0026011d502c0c256cc4b67d002347f63c3a38cd8e45f1f445c61364 \ - --hash=sha256:c4f2c3c026e876d4dad7629170ec14fff48c076d6c2ae0e354ab3fdc09024f00 - # via -r requirements.in -""", - }[x], - ), - available_interpreters = { - "python_3_12_host": "unit_test_interpreter_target", - }, - minor_mapping = {"3.12": "3.12.19"}, - simpleapi_download = mocksimpleapi_download, - evaluate_markers = lambda _, requirements, **__: { - # todo once 2692 is merged, this is going to be easier to test. - key: [ - platform - for platform in platforms - if ("x86_64" in platform and "platform_machine ==" in key) or ("x86_64" not in platform and "platform_machine !=" in key) - ] - for key, platforms in requirements.items() - }, - ) - - pypi.exposed_packages().contains_exactly({"pypi": ["torch"]}) - pypi.hub_group_map().contains_exactly({"pypi": {}}) - pypi.hub_whl_map().contains_exactly({"pypi": { - "torch": { - "pypi_312_torch_cp312_cp312_linux_x86_64_8800deef": [ - whl_config_setting( - target_platforms = ("cp312_linux_x86_64",), - version = "3.12", - ), - ], - "pypi_312_torch_cp312_cp312_manylinux_2_17_aarch64_36109432": [ - whl_config_setting( - target_platforms = ("cp312_linux_aarch64",), - version = "3.12", - ), - ], - "pypi_312_torch_cp312_cp312_win_amd64_3a570e5c": [ - whl_config_setting( - target_platforms = ("cp312_windows_x86_64",), - version = "3.12", - ), - ], - "pypi_312_torch_cp312_none_macosx_11_0_arm64_72b484d5": [ - whl_config_setting( - target_platforms = ("cp312_osx_aarch64",), - version = "3.12", - ), - ], - }, - }}) - pypi.whl_libraries().contains_exactly({ - "pypi_312_torch_cp312_cp312_linux_x86_64_8800deef": { - "dep_template": "@pypi//{name}:{target}", - "experimental_target_platforms": ["linux_x86_64"], - "filename": "torch-2.4.1+cpu-cp312-cp312-linux_x86_64.whl", - "python_interpreter_target": "unit_test_interpreter_target", - "requirement": "torch==2.4.1+cpu", - "sha256": "8800deef0026011d502c0c256cc4b67d002347f63c3a38cd8e45f1f445c61364", - "urls": ["https://torch.index/whl/cpu/torch-2.4.1%2Bcpu-cp312-cp312-linux_x86_64.whl"], - }, - "pypi_312_torch_cp312_cp312_manylinux_2_17_aarch64_36109432": { - "dep_template": "@pypi//{name}:{target}", - "experimental_target_platforms": ["linux_aarch64"], - "filename": "torch-2.4.1-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", - "python_interpreter_target": "unit_test_interpreter_target", - "requirement": "torch==2.4.1", - "sha256": "36109432b10bd7163c9b30ce896f3c2cca1b86b9765f956a1594f0ff43091e2a", - "urls": ["https://torch.index/whl/cpu/torch-2.4.1-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl"], - }, - "pypi_312_torch_cp312_cp312_win_amd64_3a570e5c": { - "dep_template": "@pypi//{name}:{target}", - "experimental_target_platforms": ["windows_x86_64"], - "filename": "torch-2.4.1+cpu-cp312-cp312-win_amd64.whl", - "python_interpreter_target": "unit_test_interpreter_target", - "requirement": "torch==2.4.1+cpu", - "sha256": "3a570e5c553415cdbddfe679207327b3a3806b21c6adea14fba77684d1619e97", - "urls": ["https://torch.index/whl/cpu/torch-2.4.1%2Bcpu-cp312-cp312-win_amd64.whl"], - }, - "pypi_312_torch_cp312_none_macosx_11_0_arm64_72b484d5": { - "dep_template": "@pypi//{name}:{target}", - "experimental_target_platforms": ["osx_aarch64"], - "filename": "torch-2.4.1-cp312-none-macosx_11_0_arm64.whl", - "python_interpreter_target": "unit_test_interpreter_target", - "requirement": "torch==2.4.1", - "sha256": "72b484d5b6cec1a735bf3fa5a1c4883d01748698c5e9cfdbeb4ffab7c7987e0d", - "urls": ["https://torch.index/whl/cpu/torch-2.4.1-cp312-none-macosx_11_0_arm64.whl"], - }, - }) - pypi.whl_mods().contains_exactly({}) - -_tests.append(_test_torch_experimental_index_url) - -def _test_download_only_multiple(env): - pypi = _parse_modules( - env, - module_ctx = _mock_mctx( - _mod( - name = "rules_python", - parse = [ - _parse( - hub_name = "pypi", - python_version = "3.15", - download_only = True, - requirements_by_platform = { - "requirements.linux_x86_64.txt": "linux_x86_64", - "requirements.osx_aarch64.txt": "osx_aarch64", - }, - ), - ], - ), - read = lambda x: { - "requirements.linux_x86_64.txt": """\ ---platform=manylinux_2_17_x86_64 ---python-version=315 ---implementation=cp ---abi=cp315 - -simple==0.0.1 \ - --hash=sha256:deadbeef -extra==0.0.1 \ - --hash=sha256:deadb00f -""", - "requirements.osx_aarch64.txt": """\ ---platform=macosx_10_9_arm64 ---python-version=315 ---implementation=cp ---abi=cp315 - -simple==0.0.3 \ - --hash=sha256:deadbaaf -""", - }[x], - ), - available_interpreters = { - "python_3_15_host": "unit_test_interpreter_target", - }, - minor_mapping = {"3.15": "3.15.19"}, - ) - - pypi.exposed_packages().contains_exactly({"pypi": ["simple"]}) - pypi.hub_group_map().contains_exactly({"pypi": {}}) - pypi.hub_whl_map().contains_exactly({"pypi": { - "extra": { - "pypi_315_extra": [ - whl_config_setting(version = "3.15"), - ], - }, - "simple": { - "pypi_315_simple_linux_x86_64": [ - whl_config_setting( - target_platforms = ["cp315_linux_x86_64"], - version = "3.15", - ), - ], - "pypi_315_simple_osx_aarch64": [ - whl_config_setting( - target_platforms = ["cp315_osx_aarch64"], - version = "3.15", - ), - ], - }, - }}) - pypi.whl_libraries().contains_exactly({ - "pypi_315_extra": { - "dep_template": "@pypi//{name}:{target}", - "download_only": True, - # TODO @aignas 2025-04-20: ensure that this is in the hub repo - # "experimental_target_platforms": ["cp315_linux_x86_64"], - "extra_pip_args": ["--platform=manylinux_2_17_x86_64", "--python-version=315", "--implementation=cp", "--abi=cp315"], - "python_interpreter_target": "unit_test_interpreter_target", - "requirement": "extra==0.0.1 --hash=sha256:deadb00f", - }, - "pypi_315_simple_linux_x86_64": { - "dep_template": "@pypi//{name}:{target}", - "download_only": True, - "extra_pip_args": ["--platform=manylinux_2_17_x86_64", "--python-version=315", "--implementation=cp", "--abi=cp315"], - "python_interpreter_target": "unit_test_interpreter_target", - "requirement": "simple==0.0.1 --hash=sha256:deadbeef", - }, - "pypi_315_simple_osx_aarch64": { - "dep_template": "@pypi//{name}:{target}", - "download_only": True, - "extra_pip_args": ["--platform=macosx_10_9_arm64", "--python-version=315", "--implementation=cp", "--abi=cp315"], - "python_interpreter_target": "unit_test_interpreter_target", - "requirement": "simple==0.0.3 --hash=sha256:deadbaaf", - }, - }) - pypi.whl_mods().contains_exactly({}) - -_tests.append(_test_download_only_multiple) - -def _test_simple_get_index(env): - got_simpleapi_download_args = [] - got_simpleapi_download_kwargs = {} - - def mocksimpleapi_download(*args, **kwargs): - got_simpleapi_download_args.extend(args) - got_simpleapi_download_kwargs.update(kwargs) - return { - "simple": struct( - whls = { - "deadb00f": struct( - yanked = False, - filename = "simple-0.0.1-py3-none-any.whl", - sha256 = "deadb00f", - url = "example2.org", - ), - }, - sdists = { - "deadbeef": struct( - yanked = False, - filename = "simple-0.0.1.tar.gz", - sha256 = "deadbeef", - url = "example.org", - ), - }, - ), - "some_other_pkg": struct( - whls = { - "deadb33f": struct( - yanked = False, - filename = "some-other-pkg-0.0.1-py3-none-any.whl", - sha256 = "deadb33f", - url = "example2.org/index/some_other_pkg/", - ), - }, - sdists = {}, - sha256s_by_version = { - "0.0.1": ["deadb33f"], - "0.0.3": ["deadbeef"], - }, - ), - } - - pypi = _parse_modules( - env, - module_ctx = _mock_mctx( - _mod( - name = "rules_python", - parse = [ - _parse( - hub_name = "pypi", - python_version = "3.15", - requirements_lock = "requirements.txt", - experimental_index_url = "pypi.org", - extra_pip_args = [ - "--extra-args-for-sdist-building", - ], - ), - ], - ), - read = lambda x: { - "requirements.txt": """ -simple==0.0.1 \ - --hash=sha256:deadbeef \ - --hash=sha256:deadb00f -some_pkg==0.0.1 @ example-direct.org/some_pkg-0.0.1-py3-none-any.whl \ - --hash=sha256:deadbaaf -direct_without_sha==0.0.1 @ example-direct.org/direct_without_sha-0.0.1-py3-none-any.whl -some_other_pkg==0.0.1 -pip_fallback==0.0.1 -direct_sdist_without_sha @ some-archive/any-name.tar.gz -git_dep @ git+https://git.server/repo/project@deadbeefdeadbeef -""", - }[x], - ), - available_interpreters = { - "python_3_15_host": "unit_test_interpreter_target", - }, - minor_mapping = {"3.15": "3.15.19"}, - simpleapi_download = mocksimpleapi_download, - ) - - pypi.exposed_packages().contains_exactly({"pypi": [ - "direct_sdist_without_sha", - "direct_without_sha", - "git_dep", - "pip_fallback", - "simple", - "some_other_pkg", - "some_pkg", - ]}) - pypi.hub_group_map().contains_exactly({"pypi": {}}) - pypi.hub_whl_map().contains_exactly({ - "pypi": { - "direct_sdist_without_sha": { - "pypi_315_any_name": [ - whl_config_setting( - target_platforms = ( - "cp315_linux_aarch64", - "cp315_linux_x86_64", - "cp315_linux_x86_64_freethreaded", - "cp315_osx_aarch64", - "cp315_windows_aarch64", - ), - version = "3.15", - ), - ], - }, - "direct_without_sha": { - "pypi_315_direct_without_sha_0_0_1_py3_none_any": [ - whl_config_setting( - target_platforms = ( - "cp315_linux_aarch64", - "cp315_linux_x86_64", - "cp315_linux_x86_64_freethreaded", - "cp315_osx_aarch64", - "cp315_windows_aarch64", - ), - version = "3.15", - ), - ], - }, - "git_dep": { - "pypi_315_git_dep": [ - whl_config_setting( - version = "3.15", - ), - ], - }, - "pip_fallback": { - "pypi_315_pip_fallback": [ - whl_config_setting( - version = "3.15", - ), - ], - }, - "simple": { - "pypi_315_simple_py3_none_any_deadb00f": [ - whl_config_setting( - target_platforms = ( - "cp315_linux_aarch64", - "cp315_linux_x86_64", - "cp315_linux_x86_64_freethreaded", - "cp315_osx_aarch64", - "cp315_windows_aarch64", - ), - version = "3.15", - ), - ], - }, - "some_other_pkg": { - "pypi_315_some_py3_none_any_deadb33f": [ - whl_config_setting( - target_platforms = ( - "cp315_linux_aarch64", - "cp315_linux_x86_64", - "cp315_linux_x86_64_freethreaded", - "cp315_osx_aarch64", - "cp315_windows_aarch64", - ), - version = "3.15", - ), - ], - }, - "some_pkg": { - "pypi_315_some_pkg_py3_none_any_deadbaaf": [ - whl_config_setting( - target_platforms = ( - "cp315_linux_aarch64", - "cp315_linux_x86_64", - "cp315_linux_x86_64_freethreaded", - "cp315_osx_aarch64", - "cp315_windows_aarch64", - ), - version = "3.15", - ), - ], - }, - }, - }) - pypi.whl_libraries().contains_exactly({ - "pypi_315_any_name": { - "dep_template": "@pypi//{name}:{target}", - "experimental_target_platforms": [ - "linux_aarch64", - "linux_x86_64", - "osx_aarch64", - "windows_aarch64", - ], - "extra_pip_args": ["--extra-args-for-sdist-building"], - "filename": "any-name.tar.gz", - "python_interpreter_target": "unit_test_interpreter_target", - "requirement": "direct_sdist_without_sha @ some-archive/any-name.tar.gz", - "sha256": "", - "urls": ["some-archive/any-name.tar.gz"], - }, - "pypi_315_direct_without_sha_0_0_1_py3_none_any": { - "dep_template": "@pypi//{name}:{target}", - "experimental_target_platforms": [ - "linux_aarch64", - "linux_x86_64", - "osx_aarch64", - "windows_aarch64", - ], - "filename": "direct_without_sha-0.0.1-py3-none-any.whl", - "python_interpreter_target": "unit_test_interpreter_target", - "requirement": "direct_without_sha==0.0.1", - "sha256": "", - "urls": ["example-direct.org/direct_without_sha-0.0.1-py3-none-any.whl"], - }, - "pypi_315_git_dep": { - "dep_template": "@pypi//{name}:{target}", - "extra_pip_args": ["--extra-args-for-sdist-building"], - "python_interpreter_target": "unit_test_interpreter_target", - "requirement": "git_dep @ git+https://git.server/repo/project@deadbeefdeadbeef", - }, - "pypi_315_pip_fallback": { - "dep_template": "@pypi//{name}:{target}", - "extra_pip_args": ["--extra-args-for-sdist-building"], - "python_interpreter_target": "unit_test_interpreter_target", - "requirement": "pip_fallback==0.0.1", - }, - "pypi_315_simple_py3_none_any_deadb00f": { - "dep_template": "@pypi//{name}:{target}", - "experimental_target_platforms": [ - "linux_aarch64", - "linux_x86_64", - "osx_aarch64", - "windows_aarch64", - ], - "filename": "simple-0.0.1-py3-none-any.whl", - "python_interpreter_target": "unit_test_interpreter_target", - "requirement": "simple==0.0.1", - "sha256": "deadb00f", - "urls": ["example2.org"], - }, - "pypi_315_some_pkg_py3_none_any_deadbaaf": { - "dep_template": "@pypi//{name}:{target}", - "experimental_target_platforms": [ - "linux_aarch64", - "linux_x86_64", - "osx_aarch64", - "windows_aarch64", - ], - "filename": "some_pkg-0.0.1-py3-none-any.whl", - "python_interpreter_target": "unit_test_interpreter_target", - "requirement": "some_pkg==0.0.1", - "sha256": "deadbaaf", - "urls": ["example-direct.org/some_pkg-0.0.1-py3-none-any.whl"], - }, - "pypi_315_some_py3_none_any_deadb33f": { - "dep_template": "@pypi//{name}:{target}", - "experimental_target_platforms": [ - "linux_aarch64", - "linux_x86_64", - "osx_aarch64", - "windows_aarch64", - ], - "filename": "some-other-pkg-0.0.1-py3-none-any.whl", - "python_interpreter_target": "unit_test_interpreter_target", - "requirement": "some_other_pkg==0.0.1", - "sha256": "deadb33f", - "urls": ["example2.org/index/some_other_pkg/"], - }, - }) - pypi.whl_mods().contains_exactly({}) - env.expect.that_dict(got_simpleapi_download_kwargs).contains_exactly( - { - "attr": struct( - auth_patterns = {}, - envsubst = {}, - extra_index_urls = [], - index_url = "pypi.org", - index_url_overrides = {}, - netrc = None, - sources = ["simple", "pip_fallback", "some_other_pkg"], - ), - "cache": {}, - "parallel_download": False, - }, - ) - -_tests.append(_test_simple_get_index) - -def _test_optimum_sys_platform_extra(env): - pypi = _parse_modules( - env, - module_ctx = _mock_mctx( - _mod( - name = "rules_python", - parse = [ - _parse( - hub_name = "pypi", - python_version = "3.15", - requirements_lock = "universal.txt", - ), - ], - ), - read = lambda x: { - "universal.txt": """\ -optimum[onnxruntime]==1.17.1 ; sys_platform == 'darwin' -optimum[onnxruntime-gpu]==1.17.1 ; sys_platform == 'linux' -""", - }[x], - ), - available_interpreters = { - "python_3_15_host": "unit_test_interpreter_target", - }, - minor_mapping = {"3.15": "3.15.19"}, - evaluate_markers = lambda _, requirements, **__: { - key: [ - platform - for platform in platforms - if ("darwin" in key and "osx" in platform) or ("linux" in key and "linux" in platform) - ] - for key, platforms in requirements.items() - }, - ) - - pypi.exposed_packages().contains_exactly({"pypi": []}) - pypi.hub_group_map().contains_exactly({"pypi": {}}) - pypi.hub_whl_map().contains_exactly({ - "pypi": { - "optimum": { - "pypi_315_optimum_linux_aarch64_linux_x86_64_linux_x86_64_freethreaded": [ - whl_config_setting( - version = "3.15", - target_platforms = [ - "cp315_linux_aarch64", - "cp315_linux_x86_64", - "cp315_linux_x86_64_freethreaded", - ], - ), - ], - "pypi_315_optimum_osx_aarch64": [ - whl_config_setting( - version = "3.15", - target_platforms = [ - "cp315_osx_aarch64", - ], - ), - ], - }, - }, - }) - - pypi.whl_libraries().contains_exactly({ - "pypi_315_optimum_linux_aarch64_linux_x86_64_linux_x86_64_freethreaded": { - "dep_template": "@pypi//{name}:{target}", - "python_interpreter_target": "unit_test_interpreter_target", - "requirement": "optimum[onnxruntime-gpu]==1.17.1", - }, - "pypi_315_optimum_osx_aarch64": { - "dep_template": "@pypi//{name}:{target}", - "python_interpreter_target": "unit_test_interpreter_target", - "requirement": "optimum[onnxruntime]==1.17.1", - }, - }) - pypi.whl_mods().contains_exactly({}) - -_tests.append(_test_optimum_sys_platform_extra) - -def _test_pipstar_platforms(env): - pypi = _parse_modules( - env, - module_ctx = _mock_mctx( - _mod( - name = "rules_python", - default = [ - _default( - platform = "my{}{}".format(os, cpu), - os_name = os, - arch_name = cpu, - marker = "python_version ~= \"3.13\"", - config_settings = [ - "@platforms//os:{}".format(os), - "@platforms//cpu:{}".format(cpu), - ], - ) - for os, cpu in [ - ("linux", "x86_64"), - ("osx", "aarch64"), - ] - ], - parse = [ - _parse( - hub_name = "pypi", - python_version = "3.15", - requirements_lock = "universal.txt", - ), - ], - ), - read = lambda x: { - "universal.txt": """\ -optimum[onnxruntime]==1.17.1 ; sys_platform == 'darwin' -optimum[onnxruntime-gpu]==1.17.1 ; sys_platform == 'linux' -""", - }[x], - ), - enable_pipstar = True, - available_interpreters = { - "python_3_15_host": "unit_test_interpreter_target", - }, - minor_mapping = {"3.15": "3.15.19"}, - ) - - pypi.exposed_packages().contains_exactly({"pypi": ["optimum"]}) - pypi.hub_group_map().contains_exactly({"pypi": {}}) - pypi.hub_whl_map().contains_exactly({ - "pypi": { - "optimum": { - "pypi_315_optimum_mylinuxx86_64": [ - whl_config_setting( - version = "3.15", - target_platforms = [ - "cp315_mylinuxx86_64", - ], - ), - ], - "pypi_315_optimum_myosxaarch64": [ - whl_config_setting( - version = "3.15", - target_platforms = [ - "cp315_myosxaarch64", - ], - ), - ], - }, - }, - }) - - pypi.whl_libraries().contains_exactly({ - "pypi_315_optimum_mylinuxx86_64": { - "dep_template": "@pypi//{name}:{target}", - "python_interpreter_target": "unit_test_interpreter_target", - "requirement": "optimum[onnxruntime-gpu]==1.17.1", - }, - "pypi_315_optimum_myosxaarch64": { - "dep_template": "@pypi//{name}:{target}", - "python_interpreter_target": "unit_test_interpreter_target", - "requirement": "optimum[onnxruntime]==1.17.1", - }, - }) - pypi.whl_mods().contains_exactly({}) - -_tests.append(_test_pipstar_platforms) - def _test_build_pipstar_platform(env): config = _build_config( env, diff --git a/tests/pypi/extension/pip_parse.bzl b/tests/pypi/extension/pip_parse.bzl new file mode 100644 index 0000000000..21569cf04e --- /dev/null +++ b/tests/pypi/extension/pip_parse.bzl @@ -0,0 +1,70 @@ +"""A simple test helper""" + +def pip_parse( + *, + hub_name, + python_version, + add_libdir_to_library_search_path = False, + auth_patterns = {}, + download_only = False, + enable_implicit_namespace_pkgs = False, + environment = {}, + envsubst = {}, + experimental_index_url = "", + experimental_requirement_cycles = {}, + experimental_target_platforms = [], + extra_hub_aliases = {}, + extra_pip_args = [], + isolated = True, + netrc = None, + parse_all_requirements_files = True, + pip_data_exclude = None, + python_interpreter = None, + python_interpreter_target = None, + quiet = True, + requirements_by_platform = {}, + requirements_darwin = None, + requirements_linux = None, + requirements_lock = None, + requirements_windows = None, + simpleapi_skip = [], + timeout = 600, + whl_modifications = {}, + **kwargs): + """A simple helper for testing to simulate the PyPI extension parse tag class""" + return struct( + auth_patterns = auth_patterns, + add_libdir_to_library_search_path = add_libdir_to_library_search_path, + download_only = download_only, + enable_implicit_namespace_pkgs = enable_implicit_namespace_pkgs, + environment = environment, + envsubst = envsubst, + experimental_index_url = experimental_index_url, + experimental_requirement_cycles = experimental_requirement_cycles, + experimental_target_platforms = experimental_target_platforms, + extra_hub_aliases = extra_hub_aliases, + extra_pip_args = extra_pip_args, + hub_name = hub_name, + isolated = isolated, + netrc = netrc, + parse_all_requirements_files = parse_all_requirements_files, + pip_data_exclude = pip_data_exclude, + python_interpreter = python_interpreter, + python_interpreter_target = python_interpreter_target, + python_version = python_version, + quiet = quiet, + requirements_by_platform = requirements_by_platform, + requirements_darwin = requirements_darwin, + requirements_linux = requirements_linux, + requirements_lock = requirements_lock, + requirements_windows = requirements_windows, + timeout = timeout, + whl_modifications = whl_modifications, + # The following are covered by other unit tests + experimental_extra_index_urls = [], + parallel_download = False, + experimental_index_url_overrides = {}, + simpleapi_skip = simpleapi_skip, + _evaluate_markers_srcs = [], + **kwargs + ) diff --git a/tests/pypi/hub_builder/BUILD.bazel b/tests/pypi/hub_builder/BUILD.bazel new file mode 100644 index 0000000000..eb52fff01c --- /dev/null +++ b/tests/pypi/hub_builder/BUILD.bazel @@ -0,0 +1,3 @@ +load(":hub_builder_tests.bzl", "hub_builder_test_suite") + +hub_builder_test_suite(name = "hub_builder_tests") diff --git a/tests/pypi/hub_builder/hub_builder_tests.bzl b/tests/pypi/hub_builder/hub_builder_tests.bzl new file mode 100644 index 0000000000..9f6ee6720d --- /dev/null +++ b/tests/pypi/hub_builder/hub_builder_tests.bzl @@ -0,0 +1,1082 @@ +# Copyright 2024 The Bazel Authors. All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +"" + +load("@rules_testing//lib:test_suite.bzl", "test_suite") +load("@rules_testing//lib:truth.bzl", "subjects") +load("//python/private:repo_utils.bzl", "REPO_DEBUG_ENV_VAR", "REPO_VERBOSITY_ENV_VAR", "repo_utils") # buildifier: disable=bzl-visibility +load("//python/private/pypi:hub_builder.bzl", _hub_builder = "hub_builder") # buildifier: disable=bzl-visibility +load("//python/private/pypi:parse_simpleapi_html.bzl", "parse_simpleapi_html") # buildifier: disable=bzl-visibility +load("//python/private/pypi:platform.bzl", _plat = "platform") # buildifier: disable=bzl-visibility +load("//python/private/pypi:whl_config_setting.bzl", "whl_config_setting") # buildifier: disable=bzl-visibility +load("//tests/pypi/extension:pip_parse.bzl", _parse = "pip_parse") + +_tests = [] + +def _mock_mctx(environ = {}, read = None): + return struct( + os = struct( + environ = environ, + name = "unittest", + arch = "exotic", + ), + read = read or (lambda _: """\ +simple==0.0.1 \ + --hash=sha256:deadbeef \ + --hash=sha256:deadbaaf"""), + ) + +def hub_builder( + env, + enable_pipstar = False, + debug = False, + config = None, + minor_mapping = {}, + evaluate_markers_fn = None, + simpleapi_download_fn = None, + available_interpreters = {}): + builder = _hub_builder( + name = "pypi", + module_name = "unit_test", + config = config or struct( + # no need to evaluate the markers with the interpreter + enable_pipstar = enable_pipstar, + platforms = { + "{}_{}{}".format(os, cpu, freethreaded): _plat( + name = "{}_{}{}".format(os, cpu, freethreaded), + os_name = os, + arch_name = cpu, + config_settings = [ + "@platforms//os:{}".format(os), + "@platforms//cpu:{}".format(cpu), + ], + whl_abi_tags = ["cp{major}{minor}t"] if freethreaded else ["abi3", "cp{major}{minor}"], + whl_platform_tags = whl_platform_tags, + ) + for (os, cpu, freethreaded), whl_platform_tags in { + ("linux", "x86_64", ""): ["linux_x86_64", "manylinux_*_x86_64"], + ("linux", "x86_64", "_freethreaded"): ["linux_x86_64", "manylinux_*_x86_64"], + ("linux", "aarch64", ""): ["linux_aarch64", "manylinux_*_aarch64"], + ("osx", "aarch64", ""): ["macosx_*_arm64"], + ("windows", "aarch64", ""): ["win_arm64"], + }.items() + }, + netrc = None, + auth_patterns = None, + ), + whl_overrides = {}, + minor_mapping = minor_mapping or {"3.15": "3.15.19"}, + available_interpreters = available_interpreters or { + "python_3_15_host": "unit_test_interpreter_target", + }, + simpleapi_download_fn = simpleapi_download_fn or (lambda *a, **k: {}), + evaluate_markers_fn = evaluate_markers_fn, + logger = repo_utils.logger( + struct( + os = struct( + environ = { + REPO_DEBUG_ENV_VAR: "1", + REPO_VERBOSITY_ENV_VAR: "TRACE" if debug else "FAIL", + }, + ), + ), + "unit-test", + ), + ) + self = struct( + build = lambda: env.expect.that_struct( + builder.build(), + attrs = dict( + exposed_packages = subjects.collection, + group_map = subjects.dict, + whl_map = subjects.dict, + whl_libraries = subjects.dict, + extra_aliases = subjects.dict, + ), + ), + pip_parse = builder.pip_parse, + ) + return self + +def _test_simple(env): + builder = hub_builder(env) + builder.pip_parse( + _mock_mctx(), + _parse( + hub_name = "pypi", + python_version = "3.15", + requirements_lock = "requirements.txt", + ), + ) + pypi = builder.build() + + pypi.exposed_packages().contains_exactly(["simple"]) + pypi.group_map().contains_exactly({}) + pypi.whl_map().contains_exactly({ + "simple": { + "pypi_315_simple": [ + whl_config_setting( + version = "3.15", + ), + ], + }, + }) + pypi.whl_libraries().contains_exactly({ + "pypi_315_simple": { + "dep_template": "@pypi//{name}:{target}", + "python_interpreter_target": "unit_test_interpreter_target", + "requirement": "simple==0.0.1 --hash=sha256:deadbeef --hash=sha256:deadbaaf", + }, + }) + pypi.extra_aliases().contains_exactly({}) + +_tests.append(_test_simple) + +def _test_simple_multiple_requirements(env): + builder = hub_builder(env) + builder.pip_parse( + _mock_mctx( + read = lambda x: { + "darwin.txt": "simple==0.0.2 --hash=sha256:deadb00f", + "win.txt": "simple==0.0.1 --hash=sha256:deadbeef", + }[x], + ), + _parse( + hub_name = "pypi", + python_version = "3.15", + requirements_darwin = "darwin.txt", + requirements_windows = "win.txt", + ), + ) + pypi = builder.build() + + pypi.exposed_packages().contains_exactly(["simple"]) + pypi.group_map().contains_exactly({}) + pypi.whl_map().contains_exactly({ + "simple": { + "pypi_315_simple_osx_aarch64": [ + whl_config_setting( + target_platforms = [ + "cp315_osx_aarch64", + ], + version = "3.15", + ), + ], + "pypi_315_simple_windows_aarch64": [ + whl_config_setting( + target_platforms = [ + "cp315_windows_aarch64", + ], + version = "3.15", + ), + ], + }, + }) + pypi.whl_libraries().contains_exactly({ + "pypi_315_simple_osx_aarch64": { + "dep_template": "@pypi//{name}:{target}", + "python_interpreter_target": "unit_test_interpreter_target", + "requirement": "simple==0.0.2 --hash=sha256:deadb00f", + }, + "pypi_315_simple_windows_aarch64": { + "dep_template": "@pypi//{name}:{target}", + "python_interpreter_target": "unit_test_interpreter_target", + "requirement": "simple==0.0.1 --hash=sha256:deadbeef", + }, + }) + pypi.extra_aliases().contains_exactly({}) + +_tests.append(_test_simple_multiple_requirements) + +def _test_simple_multiple_python_versions(env): + builder = hub_builder( + env, + available_interpreters = { + "python_3_15_host": "unit_test_interpreter_target", + "python_3_16_host": "unit_test_interpreter_target", + }, + minor_mapping = { + "3.15": "3.15.19", + "3.16": "3.16.9", + }, + ) + builder.pip_parse( + _mock_mctx( + read = lambda x: { + "requirements_3_15.txt": """ +simple==0.0.1 --hash=sha256:deadbeef +old-package==0.0.1 --hash=sha256:deadbaaf +""", + }[x], + ), + _parse( + hub_name = "pypi", + python_version = "3.15", + requirements_lock = "requirements_3_15.txt", + ), + ) + builder.pip_parse( + _mock_mctx( + read = lambda x: { + "requirements_3_16.txt": """ +simple==0.0.2 --hash=sha256:deadb00f +new-package==0.0.1 --hash=sha256:deadb00f2 +""", + }[x], + ), + _parse( + hub_name = "pypi", + python_version = "3.16", + requirements_lock = "requirements_3_16.txt", + ), + ) + pypi = builder.build() + + pypi.exposed_packages().contains_exactly(["simple"]) + pypi.group_map().contains_exactly({}) + pypi.whl_map().contains_exactly({ + "new_package": { + "pypi_316_new_package": [ + whl_config_setting( + version = "3.16", + ), + ], + }, + "old_package": { + "pypi_315_old_package": [ + whl_config_setting( + version = "3.15", + ), + ], + }, + "simple": { + "pypi_315_simple": [ + whl_config_setting( + version = "3.15", + ), + ], + "pypi_316_simple": [ + whl_config_setting( + version = "3.16", + ), + ], + }, + }) + pypi.whl_libraries().contains_exactly({ + "pypi_315_old_package": { + "dep_template": "@pypi//{name}:{target}", + "python_interpreter_target": "unit_test_interpreter_target", + "requirement": "old-package==0.0.1 --hash=sha256:deadbaaf", + }, + "pypi_315_simple": { + "dep_template": "@pypi//{name}:{target}", + "python_interpreter_target": "unit_test_interpreter_target", + "requirement": "simple==0.0.1 --hash=sha256:deadbeef", + }, + "pypi_316_new_package": { + "dep_template": "@pypi//{name}:{target}", + "python_interpreter_target": "unit_test_interpreter_target", + "requirement": "new-package==0.0.1 --hash=sha256:deadb00f2", + }, + "pypi_316_simple": { + "dep_template": "@pypi//{name}:{target}", + "python_interpreter_target": "unit_test_interpreter_target", + "requirement": "simple==0.0.2 --hash=sha256:deadb00f", + }, + }) + pypi.extra_aliases().contains_exactly({}) + +_tests.append(_test_simple_multiple_python_versions) + +def _test_simple_with_markers(env): + builder = hub_builder( + env, + evaluate_markers_fn = lambda _, requirements, **__: { + key: [ + platform + for platform in platforms + if ("x86_64" in platform and "platform_machine ==" in key) or ("x86_64" not in platform and "platform_machine !=" in key) + ] + for key, platforms in requirements.items() + }, + ) + builder.pip_parse( + _mock_mctx( + read = lambda x: { + "universal.txt": """\ +torch==2.4.1+cpu ; platform_machine == 'x86_64' +torch==2.4.1 ; platform_machine != 'x86_64' \ + --hash=sha256:deadbeef +""", + }[x], + ), + _parse( + hub_name = "pypi", + python_version = "3.15", + requirements_lock = "universal.txt", + ), + ) + pypi = builder.build() + + pypi.exposed_packages().contains_exactly(["torch"]) + pypi.group_map().contains_exactly({}) + pypi.whl_map().contains_exactly({ + "torch": { + "pypi_315_torch_linux_aarch64_osx_aarch64_windows_aarch64": [ + whl_config_setting( + target_platforms = [ + "cp315_linux_aarch64", + "cp315_osx_aarch64", + "cp315_windows_aarch64", + ], + version = "3.15", + ), + ], + "pypi_315_torch_linux_x86_64_linux_x86_64_freethreaded": [ + whl_config_setting( + target_platforms = [ + "cp315_linux_x86_64", + "cp315_linux_x86_64_freethreaded", + ], + version = "3.15", + ), + ], + }, + }) + pypi.whl_libraries().contains_exactly({ + "pypi_315_torch_linux_aarch64_osx_aarch64_windows_aarch64": { + "dep_template": "@pypi//{name}:{target}", + "python_interpreter_target": "unit_test_interpreter_target", + "requirement": "torch==2.4.1 --hash=sha256:deadbeef", + }, + "pypi_315_torch_linux_x86_64_linux_x86_64_freethreaded": { + "dep_template": "@pypi//{name}:{target}", + "python_interpreter_target": "unit_test_interpreter_target", + "requirement": "torch==2.4.1+cpu", + }, + }) + pypi.extra_aliases().contains_exactly({}) + +_tests.append(_test_simple_with_markers) + +def _test_torch_experimental_index_url(env): + def mocksimpleapi_download(*_, **__): + return { + "torch": parse_simpleapi_html( + url = "https://torch.index", + content = """\ + torch-2.4.1+cpu-cp310-cp310-linux_x86_64.whl
+ torch-2.4.1+cpu-cp310-cp310-win_amd64.whl
+ torch-2.4.1+cpu-cp311-cp311-linux_x86_64.whl
+ torch-2.4.1+cpu-cp311-cp311-win_amd64.whl
+ torch-2.4.1+cpu-cp312-cp312-linux_x86_64.whl
+ torch-2.4.1+cpu-cp312-cp312-win_amd64.whl
+ torch-2.4.1+cpu-cp38-cp38-linux_x86_64.whl
+ torch-2.4.1+cpu-cp38-cp38-win_amd64.whl
+ torch-2.4.1+cpu-cp39-cp39-linux_x86_64.whl
+ torch-2.4.1+cpu-cp39-cp39-win_amd64.whl
+ torch-2.4.1-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
+ torch-2.4.1-cp310-none-macosx_11_0_arm64.whl
+ torch-2.4.1-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
+ torch-2.4.1-cp311-none-macosx_11_0_arm64.whl
+ torch-2.4.1-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
+ torch-2.4.1-cp312-none-macosx_11_0_arm64.whl
+ torch-2.4.1-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
+ torch-2.4.1-cp38-none-macosx_11_0_arm64.whl
+ torch-2.4.1-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
+ torch-2.4.1-cp39-none-macosx_11_0_arm64.whl
+""", + ), + } + + builder = hub_builder( + env, + config = struct( + netrc = None, + enable_pipstar = False, + auth_patterns = {}, + platforms = { + "{}_{}".format(os, cpu): _plat( + name = "{}_{}".format(os, cpu), + os_name = os, + arch_name = cpu, + config_settings = [ + "@platforms//os:{}".format(os), + "@platforms//cpu:{}".format(cpu), + ], + whl_platform_tags = whl_platform_tags, + ) + for (os, cpu), whl_platform_tags in { + ("linux", "x86_64"): ["linux_x86_64", "manylinux_*_x86_64"], + ("linux", "aarch64"): ["linux_aarch64", "manylinux_*_aarch64"], + ("osx", "aarch64"): ["macosx_*_arm64"], + ("windows", "x86_64"): ["win_amd64"], + ("windows", "aarch64"): ["win_arm64"], # this should be ignored + }.items() + }, + ), + available_interpreters = { + "python_3_12_host": "unit_test_interpreter_target", + }, + minor_mapping = {"3.12": "3.12.19"}, + evaluate_markers_fn = lambda _, requirements, **__: { + # todo once 2692 is merged, this is going to be easier to test. + key: [ + platform + for platform in platforms + if ("x86_64" in platform and "platform_machine ==" in key) or ("x86_64" not in platform and "platform_machine !=" in key) + ] + for key, platforms in requirements.items() + }, + simpleapi_download_fn = mocksimpleapi_download, + ) + builder.pip_parse( + _mock_mctx( + read = lambda x: { + "universal.txt": """\ +torch==2.4.1 ; platform_machine != 'x86_64' \ + --hash=sha256:1495132f30f722af1a091950088baea383fe39903db06b20e6936fd99402803e \ + --hash=sha256:30be2844d0c939161a11073bfbaf645f1c7cb43f62f46cc6e4df1c119fb2a798 \ + --hash=sha256:36109432b10bd7163c9b30ce896f3c2cca1b86b9765f956a1594f0ff43091e2a \ + --hash=sha256:56ad2a760b7a7882725a1eebf5657abbb3b5144eb26bcb47b52059357463c548 \ + --hash=sha256:5fc1d4d7ed265ef853579caf272686d1ed87cebdcd04f2a498f800ffc53dab71 \ + --hash=sha256:72b484d5b6cec1a735bf3fa5a1c4883d01748698c5e9cfdbeb4ffab7c7987e0d \ + --hash=sha256:a38de2803ee6050309aac032676536c3d3b6a9804248537e38e098d0e14817ec \ + --hash=sha256:d36a8ef100f5bff3e9c3cea934b9e0d7ea277cb8210c7152d34a9a6c5830eadd \ + --hash=sha256:ddddbd8b066e743934a4200b3d54267a46db02106876d21cf31f7da7a96f98ea \ + --hash=sha256:fa27b048d32198cda6e9cff0bf768e8683d98743903b7e5d2b1f5098ded1d343 + # via -r requirements.in +torch==2.4.1+cpu ; platform_machine == 'x86_64' \ + --hash=sha256:0c0a7cc4f7c74ff024d5a5e21230a01289b65346b27a626f6c815d94b4b8c955 \ + --hash=sha256:1dd062d296fb78aa7cfab8690bf03704995a821b5ef69cfc807af5c0831b4202 \ + --hash=sha256:2b03e20f37557d211d14e3fb3f71709325336402db132a1e0dd8b47392185baf \ + --hash=sha256:330e780f478707478f797fdc82c2a96e9b8c5f60b6f1f57bb6ad1dd5b1e7e97e \ + --hash=sha256:3a570e5c553415cdbddfe679207327b3a3806b21c6adea14fba77684d1619e97 \ + --hash=sha256:3c99506980a2fb4b634008ccb758f42dd82f93ae2830c1e41f64536e310bf562 \ + --hash=sha256:76a6fe7b10491b650c630bc9ae328df40f79a948296b41d3b087b29a8a63cbad \ + --hash=sha256:833490a28ac156762ed6adaa7c695879564fa2fd0dc51bcf3fdb2c7b47dc55e6 \ + --hash=sha256:8800deef0026011d502c0c256cc4b67d002347f63c3a38cd8e45f1f445c61364 \ + --hash=sha256:c4f2c3c026e876d4dad7629170ec14fff48c076d6c2ae0e354ab3fdc09024f00 + # via -r requirements.in +""", + }[x], + ), + _parse( + hub_name = "pypi", + python_version = "3.12", + download_only = True, + experimental_index_url = "https://torch.index", + requirements_lock = "universal.txt", + ), + ) + pypi = builder.build() + + pypi.exposed_packages().contains_exactly(["torch"]) + pypi.group_map().contains_exactly({}) + pypi.whl_map().contains_exactly({ + "torch": { + "pypi_312_torch_cp312_cp312_linux_x86_64_8800deef": [ + whl_config_setting( + target_platforms = ("cp312_linux_x86_64",), + version = "3.12", + ), + ], + "pypi_312_torch_cp312_cp312_manylinux_2_17_aarch64_36109432": [ + whl_config_setting( + target_platforms = ("cp312_linux_aarch64",), + version = "3.12", + ), + ], + "pypi_312_torch_cp312_cp312_win_amd64_3a570e5c": [ + whl_config_setting( + target_platforms = ("cp312_windows_x86_64",), + version = "3.12", + ), + ], + "pypi_312_torch_cp312_none_macosx_11_0_arm64_72b484d5": [ + whl_config_setting( + target_platforms = ("cp312_osx_aarch64",), + version = "3.12", + ), + ], + }, + }) + pypi.whl_libraries().contains_exactly({ + "pypi_312_torch_cp312_cp312_linux_x86_64_8800deef": { + "dep_template": "@pypi//{name}:{target}", + "experimental_target_platforms": ["linux_x86_64"], + "filename": "torch-2.4.1+cpu-cp312-cp312-linux_x86_64.whl", + "python_interpreter_target": "unit_test_interpreter_target", + "requirement": "torch==2.4.1+cpu", + "sha256": "8800deef0026011d502c0c256cc4b67d002347f63c3a38cd8e45f1f445c61364", + "urls": ["https://torch.index/whl/cpu/torch-2.4.1%2Bcpu-cp312-cp312-linux_x86_64.whl"], + }, + "pypi_312_torch_cp312_cp312_manylinux_2_17_aarch64_36109432": { + "dep_template": "@pypi//{name}:{target}", + "experimental_target_platforms": ["linux_aarch64"], + "filename": "torch-2.4.1-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", + "python_interpreter_target": "unit_test_interpreter_target", + "requirement": "torch==2.4.1", + "sha256": "36109432b10bd7163c9b30ce896f3c2cca1b86b9765f956a1594f0ff43091e2a", + "urls": ["https://torch.index/whl/cpu/torch-2.4.1-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl"], + }, + "pypi_312_torch_cp312_cp312_win_amd64_3a570e5c": { + "dep_template": "@pypi//{name}:{target}", + "experimental_target_platforms": ["windows_x86_64"], + "filename": "torch-2.4.1+cpu-cp312-cp312-win_amd64.whl", + "python_interpreter_target": "unit_test_interpreter_target", + "requirement": "torch==2.4.1+cpu", + "sha256": "3a570e5c553415cdbddfe679207327b3a3806b21c6adea14fba77684d1619e97", + "urls": ["https://torch.index/whl/cpu/torch-2.4.1%2Bcpu-cp312-cp312-win_amd64.whl"], + }, + "pypi_312_torch_cp312_none_macosx_11_0_arm64_72b484d5": { + "dep_template": "@pypi//{name}:{target}", + "experimental_target_platforms": ["osx_aarch64"], + "filename": "torch-2.4.1-cp312-none-macosx_11_0_arm64.whl", + "python_interpreter_target": "unit_test_interpreter_target", + "requirement": "torch==2.4.1", + "sha256": "72b484d5b6cec1a735bf3fa5a1c4883d01748698c5e9cfdbeb4ffab7c7987e0d", + "urls": ["https://torch.index/whl/cpu/torch-2.4.1-cp312-none-macosx_11_0_arm64.whl"], + }, + }) + pypi.extra_aliases().contains_exactly({}) + +_tests.append(_test_torch_experimental_index_url) + +def _test_download_only_multiple(env): + builder = hub_builder(env) + builder.pip_parse( + _mock_mctx( + read = lambda x: { + "requirements.linux_x86_64.txt": """\ +--platform=manylinux_2_17_x86_64 +--python-version=315 +--implementation=cp +--abi=cp315 + +simple==0.0.1 \ + --hash=sha256:deadbeef +extra==0.0.1 \ + --hash=sha256:deadb00f +""", + "requirements.osx_aarch64.txt": """\ +--platform=macosx_10_9_arm64 +--python-version=315 +--implementation=cp +--abi=cp315 + +simple==0.0.3 \ + --hash=sha256:deadbaaf +""", + }[x], + ), + _parse( + hub_name = "pypi", + python_version = "3.15", + download_only = True, + requirements_by_platform = { + "requirements.linux_x86_64.txt": "linux_x86_64", + "requirements.osx_aarch64.txt": "osx_aarch64", + }, + ), + ) + pypi = builder.build() + + pypi.exposed_packages().contains_exactly(["simple"]) + pypi.group_map().contains_exactly({}) + pypi.whl_map().contains_exactly({ + "extra": { + "pypi_315_extra": [ + whl_config_setting(version = "3.15"), + ], + }, + "simple": { + "pypi_315_simple_linux_x86_64": [ + whl_config_setting( + target_platforms = ["cp315_linux_x86_64"], + version = "3.15", + ), + ], + "pypi_315_simple_osx_aarch64": [ + whl_config_setting( + target_platforms = ["cp315_osx_aarch64"], + version = "3.15", + ), + ], + }, + }) + pypi.whl_libraries().contains_exactly({ + "pypi_315_extra": { + "dep_template": "@pypi//{name}:{target}", + "download_only": True, + # TODO @aignas 2025-04-20: ensure that this is in the hub repo + # "experimental_target_platforms": ["cp315_linux_x86_64"], + "extra_pip_args": ["--platform=manylinux_2_17_x86_64", "--python-version=315", "--implementation=cp", "--abi=cp315"], + "python_interpreter_target": "unit_test_interpreter_target", + "requirement": "extra==0.0.1 --hash=sha256:deadb00f", + }, + "pypi_315_simple_linux_x86_64": { + "dep_template": "@pypi//{name}:{target}", + "download_only": True, + "extra_pip_args": ["--platform=manylinux_2_17_x86_64", "--python-version=315", "--implementation=cp", "--abi=cp315"], + "python_interpreter_target": "unit_test_interpreter_target", + "requirement": "simple==0.0.1 --hash=sha256:deadbeef", + }, + "pypi_315_simple_osx_aarch64": { + "dep_template": "@pypi//{name}:{target}", + "download_only": True, + "extra_pip_args": ["--platform=macosx_10_9_arm64", "--python-version=315", "--implementation=cp", "--abi=cp315"], + "python_interpreter_target": "unit_test_interpreter_target", + "requirement": "simple==0.0.3 --hash=sha256:deadbaaf", + }, + }) + pypi.extra_aliases().contains_exactly({}) + +_tests.append(_test_download_only_multiple) + +def _test_simple_get_index(env): + got_simpleapi_download_args = [] + got_simpleapi_download_kwargs = {} + + def mocksimpleapi_download(*args, **kwargs): + got_simpleapi_download_args.extend(args) + got_simpleapi_download_kwargs.update(kwargs) + return { + "simple": struct( + whls = { + "deadb00f": struct( + yanked = False, + filename = "simple-0.0.1-py3-none-any.whl", + sha256 = "deadb00f", + url = "example2.org", + ), + }, + sdists = { + "deadbeef": struct( + yanked = False, + filename = "simple-0.0.1.tar.gz", + sha256 = "deadbeef", + url = "example.org", + ), + }, + ), + "some_other_pkg": struct( + whls = { + "deadb33f": struct( + yanked = False, + filename = "some-other-pkg-0.0.1-py3-none-any.whl", + sha256 = "deadb33f", + url = "example2.org/index/some_other_pkg/", + ), + }, + sdists = {}, + sha256s_by_version = { + "0.0.1": ["deadb33f"], + "0.0.3": ["deadbeef"], + }, + ), + } + + builder = hub_builder( + env, + simpleapi_download_fn = mocksimpleapi_download, + ) + builder.pip_parse( + _mock_mctx( + read = lambda x: { + "requirements.txt": """ +simple==0.0.1 \ + --hash=sha256:deadbeef \ + --hash=sha256:deadb00f +some_pkg==0.0.1 @ example-direct.org/some_pkg-0.0.1-py3-none-any.whl \ + --hash=sha256:deadbaaf +direct_without_sha==0.0.1 @ example-direct.org/direct_without_sha-0.0.1-py3-none-any.whl +some_other_pkg==0.0.1 +pip_fallback==0.0.1 +direct_sdist_without_sha @ some-archive/any-name.tar.gz +git_dep @ git+https://git.server/repo/project@deadbeefdeadbeef +""", + }[x], + ), + _parse( + hub_name = "pypi", + python_version = "3.15", + requirements_lock = "requirements.txt", + experimental_index_url = "pypi.org", + extra_pip_args = [ + "--extra-args-for-sdist-building", + ], + ), + ) + pypi = builder.build() + + pypi.exposed_packages().contains_exactly([ + "direct_sdist_without_sha", + "direct_without_sha", + "git_dep", + "pip_fallback", + "simple", + "some_other_pkg", + "some_pkg", + ]) + pypi.group_map().contains_exactly({}) + pypi.whl_map().contains_exactly({ + "direct_sdist_without_sha": { + "pypi_315_any_name": [ + whl_config_setting( + target_platforms = ( + "cp315_linux_aarch64", + "cp315_linux_x86_64", + "cp315_linux_x86_64_freethreaded", + "cp315_osx_aarch64", + "cp315_windows_aarch64", + ), + version = "3.15", + ), + ], + }, + "direct_without_sha": { + "pypi_315_direct_without_sha_0_0_1_py3_none_any": [ + whl_config_setting( + target_platforms = ( + "cp315_linux_aarch64", + "cp315_linux_x86_64", + "cp315_linux_x86_64_freethreaded", + "cp315_osx_aarch64", + "cp315_windows_aarch64", + ), + version = "3.15", + ), + ], + }, + "git_dep": { + "pypi_315_git_dep": [ + whl_config_setting( + version = "3.15", + ), + ], + }, + "pip_fallback": { + "pypi_315_pip_fallback": [ + whl_config_setting( + version = "3.15", + ), + ], + }, + "simple": { + "pypi_315_simple_py3_none_any_deadb00f": [ + whl_config_setting( + target_platforms = ( + "cp315_linux_aarch64", + "cp315_linux_x86_64", + "cp315_linux_x86_64_freethreaded", + "cp315_osx_aarch64", + "cp315_windows_aarch64", + ), + version = "3.15", + ), + ], + }, + "some_other_pkg": { + "pypi_315_some_py3_none_any_deadb33f": [ + whl_config_setting( + target_platforms = ( + "cp315_linux_aarch64", + "cp315_linux_x86_64", + "cp315_linux_x86_64_freethreaded", + "cp315_osx_aarch64", + "cp315_windows_aarch64", + ), + version = "3.15", + ), + ], + }, + "some_pkg": { + "pypi_315_some_pkg_py3_none_any_deadbaaf": [ + whl_config_setting( + target_platforms = ( + "cp315_linux_aarch64", + "cp315_linux_x86_64", + "cp315_linux_x86_64_freethreaded", + "cp315_osx_aarch64", + "cp315_windows_aarch64", + ), + version = "3.15", + ), + ], + }, + }) + pypi.whl_libraries().contains_exactly({ + "pypi_315_any_name": { + "dep_template": "@pypi//{name}:{target}", + "experimental_target_platforms": [ + "linux_aarch64", + "linux_x86_64", + "osx_aarch64", + "windows_aarch64", + ], + "extra_pip_args": ["--extra-args-for-sdist-building"], + "filename": "any-name.tar.gz", + "python_interpreter_target": "unit_test_interpreter_target", + "requirement": "direct_sdist_without_sha @ some-archive/any-name.tar.gz", + "sha256": "", + "urls": ["some-archive/any-name.tar.gz"], + }, + "pypi_315_direct_without_sha_0_0_1_py3_none_any": { + "dep_template": "@pypi//{name}:{target}", + "experimental_target_platforms": [ + "linux_aarch64", + "linux_x86_64", + "osx_aarch64", + "windows_aarch64", + ], + "filename": "direct_without_sha-0.0.1-py3-none-any.whl", + "python_interpreter_target": "unit_test_interpreter_target", + "requirement": "direct_without_sha==0.0.1", + "sha256": "", + "urls": ["example-direct.org/direct_without_sha-0.0.1-py3-none-any.whl"], + }, + "pypi_315_git_dep": { + "dep_template": "@pypi//{name}:{target}", + "extra_pip_args": ["--extra-args-for-sdist-building"], + "python_interpreter_target": "unit_test_interpreter_target", + "requirement": "git_dep @ git+https://git.server/repo/project@deadbeefdeadbeef", + }, + "pypi_315_pip_fallback": { + "dep_template": "@pypi//{name}:{target}", + "extra_pip_args": ["--extra-args-for-sdist-building"], + "python_interpreter_target": "unit_test_interpreter_target", + "requirement": "pip_fallback==0.0.1", + }, + "pypi_315_simple_py3_none_any_deadb00f": { + "dep_template": "@pypi//{name}:{target}", + "experimental_target_platforms": [ + "linux_aarch64", + "linux_x86_64", + "osx_aarch64", + "windows_aarch64", + ], + "filename": "simple-0.0.1-py3-none-any.whl", + "python_interpreter_target": "unit_test_interpreter_target", + "requirement": "simple==0.0.1", + "sha256": "deadb00f", + "urls": ["example2.org"], + }, + "pypi_315_some_pkg_py3_none_any_deadbaaf": { + "dep_template": "@pypi//{name}:{target}", + "experimental_target_platforms": [ + "linux_aarch64", + "linux_x86_64", + "osx_aarch64", + "windows_aarch64", + ], + "filename": "some_pkg-0.0.1-py3-none-any.whl", + "python_interpreter_target": "unit_test_interpreter_target", + "requirement": "some_pkg==0.0.1", + "sha256": "deadbaaf", + "urls": ["example-direct.org/some_pkg-0.0.1-py3-none-any.whl"], + }, + "pypi_315_some_py3_none_any_deadb33f": { + "dep_template": "@pypi//{name}:{target}", + "experimental_target_platforms": [ + "linux_aarch64", + "linux_x86_64", + "osx_aarch64", + "windows_aarch64", + ], + "filename": "some-other-pkg-0.0.1-py3-none-any.whl", + "python_interpreter_target": "unit_test_interpreter_target", + "requirement": "some_other_pkg==0.0.1", + "sha256": "deadb33f", + "urls": ["example2.org/index/some_other_pkg/"], + }, + }) + pypi.extra_aliases().contains_exactly({}) + env.expect.that_dict(got_simpleapi_download_kwargs).contains_exactly( + { + "attr": struct( + auth_patterns = {}, + envsubst = {}, + extra_index_urls = [], + index_url = "pypi.org", + index_url_overrides = {}, + netrc = None, + sources = ["simple", "pip_fallback", "some_other_pkg"], + ), + "cache": {}, + "parallel_download": False, + }, + ) + +_tests.append(_test_simple_get_index) + +def _test_optimum_sys_platform_extra(env): + builder = hub_builder( + env, + evaluate_markers_fn = lambda _, requirements, **__: { + key: [ + platform + for platform in platforms + if ("darwin" in key and "osx" in platform) or ("linux" in key and "linux" in platform) + ] + for key, platforms in requirements.items() + }, + ) + builder.pip_parse( + _mock_mctx( + read = lambda x: { + "universal.txt": """\ +optimum[onnxruntime]==1.17.1 ; sys_platform == 'darwin' +optimum[onnxruntime-gpu]==1.17.1 ; sys_platform == 'linux' +""", + }[x], + ), + _parse( + hub_name = "pypi", + python_version = "3.15", + requirements_lock = "universal.txt", + ), + ) + pypi = builder.build() + + # FIXME @aignas 2025-09-07: we should expose the `optimum` package + pypi.exposed_packages().contains_exactly([]) + pypi.group_map().contains_exactly({}) + pypi.whl_map().contains_exactly({ + "optimum": { + "pypi_315_optimum_linux_aarch64_linux_x86_64_linux_x86_64_freethreaded": [ + whl_config_setting( + version = "3.15", + target_platforms = [ + "cp315_linux_aarch64", + "cp315_linux_x86_64", + "cp315_linux_x86_64_freethreaded", + ], + ), + ], + "pypi_315_optimum_osx_aarch64": [ + whl_config_setting( + version = "3.15", + target_platforms = [ + "cp315_osx_aarch64", + ], + ), + ], + }, + }) + pypi.whl_libraries().contains_exactly({ + "pypi_315_optimum_linux_aarch64_linux_x86_64_linux_x86_64_freethreaded": { + "dep_template": "@pypi//{name}:{target}", + "python_interpreter_target": "unit_test_interpreter_target", + "requirement": "optimum[onnxruntime-gpu]==1.17.1", + }, + "pypi_315_optimum_osx_aarch64": { + "dep_template": "@pypi//{name}:{target}", + "python_interpreter_target": "unit_test_interpreter_target", + "requirement": "optimum[onnxruntime]==1.17.1", + }, + }) + pypi.extra_aliases().contains_exactly({}) + +_tests.append(_test_optimum_sys_platform_extra) + +def _test_pipstar_platforms(env): + builder = hub_builder( + env, + enable_pipstar = True, + config = struct( + enable_pipstar = True, + netrc = None, + auth_patterns = {}, + platforms = { + "my{}{}".format(os, cpu): _plat( + name = "my{}{}".format(os, cpu), + os_name = os, + arch_name = cpu, + marker = "python_version ~= \"3.13\"", + config_settings = [ + "@platforms//os:{}".format(os), + "@platforms//cpu:{}".format(cpu), + ], + ) + for os, cpu in [ + ("linux", "x86_64"), + ("osx", "aarch64"), + ] + }, + ), + ) + builder.pip_parse( + _mock_mctx( + read = lambda x: { + "universal.txt": """\ +optimum[onnxruntime]==1.17.1 ; sys_platform == 'darwin' +optimum[onnxruntime-gpu]==1.17.1 ; sys_platform == 'linux' +""", + }[x], + ), + _parse( + hub_name = "pypi", + python_version = "3.15", + requirements_lock = "universal.txt", + ), + ) + pypi = builder.build() + + pypi.exposed_packages().contains_exactly(["optimum"]) + pypi.group_map().contains_exactly({}) + pypi.whl_map().contains_exactly({ + "optimum": { + "pypi_315_optimum_mylinuxx86_64": [ + whl_config_setting( + version = "3.15", + target_platforms = [ + "cp315_mylinuxx86_64", + ], + ), + ], + "pypi_315_optimum_myosxaarch64": [ + whl_config_setting( + version = "3.15", + target_platforms = [ + "cp315_myosxaarch64", + ], + ), + ], + }, + }) + pypi.whl_libraries().contains_exactly({ + "pypi_315_optimum_mylinuxx86_64": { + "dep_template": "@pypi//{name}:{target}", + "python_interpreter_target": "unit_test_interpreter_target", + "requirement": "optimum[onnxruntime-gpu]==1.17.1", + }, + "pypi_315_optimum_myosxaarch64": { + "dep_template": "@pypi//{name}:{target}", + "python_interpreter_target": "unit_test_interpreter_target", + "requirement": "optimum[onnxruntime]==1.17.1", + }, + }) + pypi.extra_aliases().contains_exactly({}) + +_tests.append(_test_pipstar_platforms) + +def hub_builder_test_suite(name): + """Create the test suite. + + Args: + name: the name of the test suite + """ + test_suite(name = name, basic_tests = _tests) From 43c3013a086cc140b795f17cd224118f650305d1 Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Mon, 8 Sep 2025 19:46:16 -0700 Subject: [PATCH 428/922] build(deps): bump zipp from 3.20.2 to 3.23.0 in /tools/publish (#3253) Bumps [zipp](https://github.com/jaraco/zipp) from 3.20.2 to 3.23.0.
Changelog

Sourced from zipp's changelog.

v3.23.0

Features

  • Add a compatibility shim for Python 3.13 and earlier. (#145)

v3.22.0

Features

Bugfixes

  • Fixed .name, .stem, and other basename-based properties on Windows when working with a zipfile on disk. (#133)

v3.21.0

Features

  • Improve performances of :meth:zipfile.Path.open for non-reading modes. (1a1928d)
  • Rely on cached_property to cache values on the instance.
  • Rely on save_method_args to save method args.
Commits

[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=zipp&package-manager=pip&previous-version=3.20.2&new-version=3.23.0)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) ---
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot show ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> --- tools/publish/requirements_darwin.txt | 6 +++--- tools/publish/requirements_linux.txt | 6 +++--- tools/publish/requirements_universal.txt | 6 +++--- tools/publish/requirements_windows.txt | 6 +++--- 4 files changed, 12 insertions(+), 12 deletions(-) diff --git a/tools/publish/requirements_darwin.txt b/tools/publish/requirements_darwin.txt index 2ecf5a0e51..7e0acb9ecf 100644 --- a/tools/publish/requirements_darwin.txt +++ b/tools/publish/requirements_darwin.txt @@ -205,7 +205,7 @@ urllib3==2.5.0 \ # via # requests # twine -zipp==3.20.2 \ - --hash=sha256:a817ac80d6cf4b23bf7f2828b7cabf326f15a001bea8b1f9b49631780ba28350 \ - --hash=sha256:bc9eb26f4506fda01b81bcde0ca78103b6e62f991b381fec825435c836edbc29 +zipp==3.23.0 \ + --hash=sha256:071652d6115ed432f5ce1d34c336c0adfd6a884660d1e9712a256d3d3bd4b14e \ + --hash=sha256:a07157588a12518c9d4034df3fbbee09c814741a33ff63c05fa29d26a2404166 # via importlib-metadata diff --git a/tools/publish/requirements_linux.txt b/tools/publish/requirements_linux.txt index d5d7563f94..aedb3c4c97 100644 --- a/tools/publish/requirements_linux.txt +++ b/tools/publish/requirements_linux.txt @@ -327,7 +327,7 @@ urllib3==2.5.0 \ # via # requests # twine -zipp==3.20.2 \ - --hash=sha256:a817ac80d6cf4b23bf7f2828b7cabf326f15a001bea8b1f9b49631780ba28350 \ - --hash=sha256:bc9eb26f4506fda01b81bcde0ca78103b6e62f991b381fec825435c836edbc29 +zipp==3.23.0 \ + --hash=sha256:071652d6115ed432f5ce1d34c336c0adfd6a884660d1e9712a256d3d3bd4b14e \ + --hash=sha256:a07157588a12518c9d4034df3fbbee09c814741a33ff63c05fa29d26a2404166 # via importlib-metadata diff --git a/tools/publish/requirements_universal.txt b/tools/publish/requirements_universal.txt index aaff8bd59a..79bc359451 100644 --- a/tools/publish/requirements_universal.txt +++ b/tools/publish/requirements_universal.txt @@ -331,7 +331,7 @@ urllib3==2.5.0 \ # via # requests # twine -zipp==3.20.2 \ - --hash=sha256:a817ac80d6cf4b23bf7f2828b7cabf326f15a001bea8b1f9b49631780ba28350 \ - --hash=sha256:bc9eb26f4506fda01b81bcde0ca78103b6e62f991b381fec825435c836edbc29 +zipp==3.23.0 \ + --hash=sha256:071652d6115ed432f5ce1d34c336c0adfd6a884660d1e9712a256d3d3bd4b14e \ + --hash=sha256:a07157588a12518c9d4034df3fbbee09c814741a33ff63c05fa29d26a2404166 # via importlib-metadata diff --git a/tools/publish/requirements_windows.txt b/tools/publish/requirements_windows.txt index 0a3139a17e..3799652b3d 100644 --- a/tools/publish/requirements_windows.txt +++ b/tools/publish/requirements_windows.txt @@ -209,7 +209,7 @@ urllib3==2.5.0 \ # via # requests # twine -zipp==3.20.2 \ - --hash=sha256:a817ac80d6cf4b23bf7f2828b7cabf326f15a001bea8b1f9b49631780ba28350 \ - --hash=sha256:bc9eb26f4506fda01b81bcde0ca78103b6e62f991b381fec825435c836edbc29 +zipp==3.23.0 \ + --hash=sha256:071652d6115ed432f5ce1d34c336c0adfd6a884660d1e9712a256d3d3bd4b14e \ + --hash=sha256:a07157588a12518c9d4034df3fbbee09c814741a33ff63c05fa29d26a2404166 # via importlib-metadata From 6df5cbb68b15b70ecff20d7054fb051edad41864 Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Mon, 8 Sep 2025 19:47:24 -0700 Subject: [PATCH 429/922] build(deps): bump more-itertools from 10.7.0 to 10.8.0 in /tools/publish (#3254) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Bumps [more-itertools](https://github.com/more-itertools/more-itertools) from 10.7.0 to 10.8.0.
Release notes

Sourced from more-itertools's releases.

Version 10.8.0

What's Changed

... (truncated)

Commits
  • 8c1a6ef Merge pull request #1071 from more-itertools/version-10.8.0
  • 24be440 Add note for issue 1054
  • 3dd5980 Add a note for issue 1063
  • 2ce52d1 Update docs for 10.8.0
  • eae9156 Bump version: 10.7.0 → 10.8.0
  • a80f1c5 Merge pull request #1068 from rhettinger/cleanup_tail
  • 5701589 Merge pull request #1067 from rhettinger/reshape_beautification
  • 58e0331 Merge pull request #1069 from rhettinger/derangements_doc
  • 9a3d7e3 Clarify how derangements treats duplicate inputs
  • c509b14 Clean-up tail(). Prefer try/except over the Sized ABC.
  • Additional commits viewable in compare view

[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=more-itertools&package-manager=pip&previous-version=10.7.0&new-version=10.8.0)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) ---
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot show ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> --- tools/publish/requirements_darwin.txt | 6 +++--- tools/publish/requirements_linux.txt | 6 +++--- tools/publish/requirements_universal.txt | 6 +++--- tools/publish/requirements_windows.txt | 6 +++--- 4 files changed, 12 insertions(+), 12 deletions(-) diff --git a/tools/publish/requirements_darwin.txt b/tools/publish/requirements_darwin.txt index 7e0acb9ecf..05f18f99ae 100644 --- a/tools/publish/requirements_darwin.txt +++ b/tools/publish/requirements_darwin.txt @@ -129,9 +129,9 @@ mdurl==0.1.2 \ --hash=sha256:84008a41e51615a49fc9966191ff91509e3c40b939176e643fd50a5c2196b8f8 \ --hash=sha256:bb413d29f5eea38f31dd4754dd7377d4465116fb207585f97bf925588687c1ba # via markdown-it-py -more-itertools==10.7.0 \ - --hash=sha256:9fddd5403be01a94b204faadcff459ec3568cf110265d3c54323e1e866ad29d3 \ - --hash=sha256:d43980384673cb07d2f7d2d918c616b30c659c089ee23953f601d6609c67510e +more-itertools==10.8.0 \ + --hash=sha256:52d4362373dcf7c52546bc4af9a86ee7c4579df9a8dc268be0a2f949d376cc9b \ + --hash=sha256:f638ddf8a1a0d134181275fb5d58b086ead7c6a72429ad725c67503f13ba30bd # via # jaraco-classes # jaraco-functools diff --git a/tools/publish/requirements_linux.txt b/tools/publish/requirements_linux.txt index aedb3c4c97..75a125e8f1 100644 --- a/tools/publish/requirements_linux.txt +++ b/tools/publish/requirements_linux.txt @@ -243,9 +243,9 @@ mdurl==0.1.2 \ --hash=sha256:84008a41e51615a49fc9966191ff91509e3c40b939176e643fd50a5c2196b8f8 \ --hash=sha256:bb413d29f5eea38f31dd4754dd7377d4465116fb207585f97bf925588687c1ba # via markdown-it-py -more-itertools==10.7.0 \ - --hash=sha256:9fddd5403be01a94b204faadcff459ec3568cf110265d3c54323e1e866ad29d3 \ - --hash=sha256:d43980384673cb07d2f7d2d918c616b30c659c089ee23953f601d6609c67510e +more-itertools==10.8.0 \ + --hash=sha256:52d4362373dcf7c52546bc4af9a86ee7c4579df9a8dc268be0a2f949d376cc9b \ + --hash=sha256:f638ddf8a1a0d134181275fb5d58b086ead7c6a72429ad725c67503f13ba30bd # via # jaraco-classes # jaraco-functools diff --git a/tools/publish/requirements_universal.txt b/tools/publish/requirements_universal.txt index 79bc359451..65d70a4d25 100644 --- a/tools/publish/requirements_universal.txt +++ b/tools/publish/requirements_universal.txt @@ -243,9 +243,9 @@ mdurl==0.1.2 \ --hash=sha256:84008a41e51615a49fc9966191ff91509e3c40b939176e643fd50a5c2196b8f8 \ --hash=sha256:bb413d29f5eea38f31dd4754dd7377d4465116fb207585f97bf925588687c1ba # via markdown-it-py -more-itertools==10.7.0 \ - --hash=sha256:9fddd5403be01a94b204faadcff459ec3568cf110265d3c54323e1e866ad29d3 \ - --hash=sha256:d43980384673cb07d2f7d2d918c616b30c659c089ee23953f601d6609c67510e +more-itertools==10.8.0 \ + --hash=sha256:52d4362373dcf7c52546bc4af9a86ee7c4579df9a8dc268be0a2f949d376cc9b \ + --hash=sha256:f638ddf8a1a0d134181275fb5d58b086ead7c6a72429ad725c67503f13ba30bd # via # jaraco-classes # jaraco-functools diff --git a/tools/publish/requirements_windows.txt b/tools/publish/requirements_windows.txt index 3799652b3d..6dd7ffe978 100644 --- a/tools/publish/requirements_windows.txt +++ b/tools/publish/requirements_windows.txt @@ -129,9 +129,9 @@ mdurl==0.1.2 \ --hash=sha256:84008a41e51615a49fc9966191ff91509e3c40b939176e643fd50a5c2196b8f8 \ --hash=sha256:bb413d29f5eea38f31dd4754dd7377d4465116fb207585f97bf925588687c1ba # via markdown-it-py -more-itertools==10.7.0 \ - --hash=sha256:9fddd5403be01a94b204faadcff459ec3568cf110265d3c54323e1e866ad29d3 \ - --hash=sha256:d43980384673cb07d2f7d2d918c616b30c659c089ee23953f601d6609c67510e +more-itertools==10.8.0 \ + --hash=sha256:52d4362373dcf7c52546bc4af9a86ee7c4579df9a8dc268be0a2f949d376cc9b \ + --hash=sha256:f638ddf8a1a0d134181275fb5d58b086ead7c6a72429ad725c67503f13ba30bd # via # jaraco-classes # jaraco-functools From 37cb91a33fecc10597c67fef6fe0c35011cf7e67 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Mon, 8 Sep 2025 21:41:19 -0700 Subject: [PATCH 430/922] feat: allow registering arbitrary settings for py_binary transitions (#3248) This implements the ability for users to add additional settings that py_binary, py_test, and py_wheel can transition on. There were three main use cases motivating this feature: 1. Making it easier to have multiple pypi dependency closures and shared dependencies. 2. Making it easier to override flags for `py_wheel`. 3. Making it easier to have per-target setting of things like bootstrap_impl, venv site packages, etc. It also adds most of our config settings to the the transition inputs/outputs for those rules, which allows users to per-target force particular settings without having to use e.g. `with_cfg` to wrap a target with the desired transition settings. It also lets use avoid adding dozens of attributes (one per setting); today there are about 17 flags. Under the hood, this works by having a bzlmod api that users can pass labels to. These labels are put into a generated bzl file, which the rules load and add to their list of transition inputs/outputs. On the target level, the `config_settings` attribute, which is a `dict[label, str]`, can be set to change the particular flags of interest. Along the way... * Create a common_labels.bzl file for the shared label strings * Remove the defunct py_reconfig code in sh_py_run_test. --------- Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> --- MODULE.bazel | 29 +++++- WORKSPACE | 27 +++++- .../common-deps-with-multipe-pypi-versions.md | 91 +++++++++++++++++++ internal_dev_deps.bzl | 7 +- python/extensions/BUILD.bazel | 9 ++ python/extensions/config.bzl | 53 +++++++++++ python/private/BUILD.bazel | 18 ++++ python/private/attr_builders.bzl | 3 + python/private/attributes.bzl | 50 ++++++++++ python/private/builders_util.bzl | 39 +++++++- python/private/common_labels.bzl | 27 ++++++ python/private/internal_config_repo.bzl | 47 +++++++++- python/private/internal_deps.bzl | 22 ----- python/private/py_executable.bzl | 14 +-- python/private/py_repositories.bzl | 11 ++- python/private/py_wheel.bzl | 17 +++- python/private/rule_builders.bzl | 17 +++- python/private/transition_labels.bzl | 32 +++++++ tests/builders/rule_builders_tests.bzl | 4 +- tests/multi_pypi/BUILD.bazel | 29 ++++++ tests/multi_pypi/alpha/BUILD.bazel | 7 ++ tests/multi_pypi/alpha/pyproject.toml | 6 ++ tests/multi_pypi/alpha/requirements.txt | 6 ++ tests/multi_pypi/beta/BUILD.bazel | 7 ++ tests/multi_pypi/beta/pyproject.toml | 6 ++ tests/multi_pypi/beta/requirements.txt | 6 ++ tests/multi_pypi/pypi_alpha/BUILD.bazel | 11 +++ .../multi_pypi/pypi_alpha/pypi_alpha_test.py | 8 ++ tests/multi_pypi/pypi_beta/BUILD.bazel | 11 +++ tests/multi_pypi/pypi_beta/pypi_beta_test.py | 8 ++ tests/py_wheel/py_wheel_tests.bzl | 41 ++++++++- tests/support/py_reconfig.bzl | 27 +++--- tests/support/sh_py_run_test.bzl | 79 +--------------- tests/support/support.bzl | 1 + tests/toolchains/BUILD.bazel | 4 +- 35 files changed, 638 insertions(+), 136 deletions(-) create mode 100644 docs/howto/common-deps-with-multipe-pypi-versions.md create mode 100644 python/extensions/config.bzl create mode 100644 python/private/common_labels.bzl delete mode 100644 python/private/internal_deps.bzl create mode 100644 python/private/transition_labels.bzl create mode 100644 tests/multi_pypi/BUILD.bazel create mode 100644 tests/multi_pypi/alpha/BUILD.bazel create mode 100644 tests/multi_pypi/alpha/pyproject.toml create mode 100644 tests/multi_pypi/alpha/requirements.txt create mode 100644 tests/multi_pypi/beta/BUILD.bazel create mode 100644 tests/multi_pypi/beta/pyproject.toml create mode 100644 tests/multi_pypi/beta/requirements.txt create mode 100644 tests/multi_pypi/pypi_alpha/BUILD.bazel create mode 100644 tests/multi_pypi/pypi_alpha/pypi_alpha_test.py create mode 100644 tests/multi_pypi/pypi_beta/BUILD.bazel create mode 100644 tests/multi_pypi/pypi_beta/pypi_beta_test.py diff --git a/MODULE.bazel b/MODULE.bazel index 1dca3e91fa..6251ed4c3c 100644 --- a/MODULE.bazel +++ b/MODULE.bazel @@ -13,9 +13,9 @@ bazel_dep(name = "platforms", version = "0.0.11") # Use py_proto_library directly from protobuf repository bazel_dep(name = "protobuf", version = "29.0-rc2", repo_name = "com_google_protobuf") -internal_deps = use_extension("//python/private:internal_deps.bzl", "internal_deps") +rules_python_config = use_extension("//python/extensions:config.bzl", "config") use_repo( - internal_deps, + rules_python_config, "pypi__build", "pypi__click", "pypi__colorama", @@ -218,6 +218,19 @@ use_repo( "whl_with_build_files", ) +dev_rules_python_config = use_extension( + "//python/extensions:config.bzl", + "config", + dev_dependency = True, +) +dev_rules_python_config.add_transition_setting( + # Intentionally add a setting already present for testing + setting = "//python/config_settings:python_version", +) +dev_rules_python_config.add_transition_setting( + setting = "//tests/multi_pypi:external_deps_name", +) + # Add gazelle plugin so that we can run the gazelle example as an e2e integration # test and include the distribution files. local_path_override( @@ -291,7 +304,17 @@ dev_pip.parse( python_version = "3.11", requirements_lock = "//examples/wheel:requirements_server.txt", ) -use_repo(dev_pip, "dev_pip", "pypiserver") +dev_pip.parse( + hub_name = "pypi_alpha", + python_version = "3.11", + requirements_lock = "//tests/multi_pypi/alpha:requirements.txt", +) +dev_pip.parse( + hub_name = "pypi_beta", + python_version = "3.11", + requirements_lock = "//tests/multi_pypi/beta:requirements.txt", +) +use_repo(dev_pip, "dev_pip", "pypi_alpha", "pypi_beta", "pypiserver") # Bazel integration test setup below diff --git a/WORKSPACE b/WORKSPACE index 5c2136666d..077ddb5e68 100644 --- a/WORKSPACE +++ b/WORKSPACE @@ -69,7 +69,9 @@ load("//:internal_dev_setup.bzl", "rules_python_internal_setup") rules_python_internal_setup() load("@pythons_hub//:versions.bzl", "PYTHON_VERSIONS") -load("//python:repositories.bzl", "python_register_multi_toolchains") +load("//python:repositories.bzl", "py_repositories", "python_register_multi_toolchains") + +py_repositories() python_register_multi_toolchains( name = "python", @@ -155,3 +157,26 @@ pip_parse( load("@dev_pip//:requirements.bzl", docs_install_deps = "install_deps") docs_install_deps() + +##################### +# Pypi repos for //tests/multi_pypi + +pip_parse( + name = "pypi_alpha", + python_interpreter_target = interpreter, + requirements_lock = "//tests/multi_pypi/alpha:requirements.txt", +) + +load("@pypi_alpha//:requirements.bzl", pypi_alpha_install_deps = "install_deps") + +pypi_alpha_install_deps() + +pip_parse( + name = "pypi_beta", + python_interpreter_target = interpreter, + requirements_lock = "//tests/multi_pypi/beta:requirements.txt", +) + +load("@pypi_beta//:requirements.bzl", pypi_beta_install_deps = "install_deps") + +pypi_beta_install_deps() diff --git a/docs/howto/common-deps-with-multipe-pypi-versions.md b/docs/howto/common-deps-with-multipe-pypi-versions.md new file mode 100644 index 0000000000..ba3568682f --- /dev/null +++ b/docs/howto/common-deps-with-multipe-pypi-versions.md @@ -0,0 +1,91 @@ +# How to use a common set of dependencies with multiple PyPI versions + +In this guide, we show how to handle a situation common to monorepos +that extensively share code: How does a common library refer to the correct +`@pypi_` hub when binaries may have their own requirements (and thus +PyPI hub name)? Stated as code, this situation: + +```bzl + +py_binary( + name = "bin_alpha", + deps = ["@pypi_alpha//requests", ":common"], +) +py_binary( + name = "bin_beta", + deps = ["@pypi_beta//requests", ":common"], +) + +py_library( + name = "common", + deps = ["@pypi_???//more_itertools"] # <-- Which @pypi repo? +) +``` + +## Using flags to pick a hub + +The basic trick to make `:common` pick the appropriate `@pypi_` is to use +`select()` to choose one based on build flags. To help this process, `py_binary` +et al allow forcing particular build flags to be used, and custom flags can be +registered to allow `py_binary` et al to set them. + +In this example, we create a custom string flag named `//:pypi_hub`, +register it to allow using it with `py_binary` directly, then use `select()` +to pick different dependencies. + +```bzl +# File: MODULE.bazel + +rules_python_config.add_transition_setting( + setting = "//:pypi_hub", +) + +# File: BUILD.bazel + +```bzl + +load("@bazel_skylib//rules:common_settings.bzl", "string_flag") + +string_flag( + name = "pypi_hub", +) + +config_setting( + name = "is_pypi_alpha", + flag_values = {"//:pypi_hub": "alpha"}, +) + +config_setting( + name = "is_pypi_beta", + flag_values = {"//:pypi_hub": "beta"} +) + +py_binary( + name = "bin_alpha", + srcs = ["bin_alpha.py"], + config_settings = { + "//:pypi_hub": "alpha", + }, + deps = ["@pypi_alpha//requests", ":common"], +) +py_binary( + name = "bin_beta", + srcs = ["bin_beta.py"], + config_settings = { + "//:pypi_hub": "beta", + }, + deps = ["@pypi_beta//requests", ":common"], +) +py_library( + name = "common", + deps = select({ + ":is_pypi_alpha": ["@pypi_alpha//more_itertools"], + ":is_pypi_beta": ["@pypi_beta//more_itertools"], + }), +) +``` + +When `bin_alpha` and `bin_beta` are built, they will have the `pypi_hub` +flag force to their respective value. When `:common` is evaluated, it sees +the flag value of the binary that is consuming it, and the `select()` resolves +appropriately. diff --git a/internal_dev_deps.bzl b/internal_dev_deps.bzl index e1a6562fe6..91f5defd3e 100644 --- a/internal_dev_deps.bzl +++ b/internal_dev_deps.bzl @@ -41,7 +41,12 @@ def rules_python_internal_deps(): For dependencies needed by *users* of rules_python, see python/private/py_repositories.bzl. """ - internal_config_repo(name = "rules_python_internal") + internal_config_repo( + name = "rules_python_internal", + transition_settings = [ + str(Label("//tests/multi_pypi:external_deps_name")), + ], + ) local_repository( name = "other", diff --git a/python/extensions/BUILD.bazel b/python/extensions/BUILD.bazel index e8a63d6d5b..e6c876c76f 100644 --- a/python/extensions/BUILD.bazel +++ b/python/extensions/BUILD.bazel @@ -39,3 +39,12 @@ bzl_library( "//python/private:python_bzl", ], ) + +bzl_library( + name = "config_bzl", + srcs = ["config.bzl"], + visibility = ["//:__subpackages__"], + deps = [ + "//python/private:internal_config_repo_bzl", + ], +) diff --git a/python/extensions/config.bzl b/python/extensions/config.bzl new file mode 100644 index 0000000000..2667b2a4fb --- /dev/null +++ b/python/extensions/config.bzl @@ -0,0 +1,53 @@ +"""Extension for configuring global settings of rules_python.""" + +load("//python/private:internal_config_repo.bzl", "internal_config_repo") +load("//python/private/pypi:deps.bzl", "pypi_deps") + +_add_transition_setting = tag_class( + doc = """ +Specify a build setting that terminal rules transition on by default. + +Terminal rules are rules such as py_binary, py_test, py_wheel, or similar +rules that represent some deployable unit. Settings added here can +then be used a keys with the {obj}`config_settings` attribute. + +:::{note} +This adds the label as a dependency of the Python rules. Take care to not refer +to repositories that are expensive to create or invalidate frequently. +::: +""", + attrs = { + "setting": attr.label(doc = "The build setting to add."), + }, +) + +def _config_impl(mctx): + transition_setting_generators = {} + transition_settings = [] + for mod in mctx.modules: + for tag in mod.tags.add_transition_setting: + setting = str(tag.setting) + if setting not in transition_setting_generators: + transition_setting_generators[setting] = [] + transition_settings.append(setting) + transition_setting_generators[setting].append(mod.name) + + internal_config_repo( + name = "rules_python_internal", + transition_setting_generators = transition_setting_generators, + transition_settings = transition_settings, + ) + + pypi_deps() + +config = module_extension( + doc = """Global settings for rules_python. + +:::{versionadded} VERSION_NEXT_FEATURE +::: +""", + implementation = _config_impl, + tag_classes = { + "add_transition_setting": _add_transition_setting, + }, +) diff --git a/python/private/BUILD.bazel b/python/private/BUILD.bazel index 6fc78efc25..f31b56ec50 100644 --- a/python/private/BUILD.bazel +++ b/python/private/BUILD.bazel @@ -106,6 +106,7 @@ bzl_library( name = "builders_util_bzl", srcs = ["builders_util.bzl"], deps = [ + ":bzlmod_enabled_bzl", "@bazel_skylib//lib:types", ], ) @@ -135,6 +136,11 @@ bzl_library( ], ) +bzl_library( + name = "common_labels_bzl", + srcs = ["common_labels.bzl"], +) + bzl_library( name = "config_settings_bzl", srcs = ["config_settings.bzl"], @@ -408,6 +414,7 @@ bzl_library( ":py_runtime_info_bzl", ":rules_cc_srcs_bzl", ":toolchain_types_bzl", + ":transition_labels_bzl", "@bazel_skylib//lib:dicts", "@bazel_skylib//lib:paths", "@bazel_skylib//lib:structs", @@ -583,6 +590,7 @@ bzl_library( deps = [ ":py_package_bzl", ":stamp_bzl", + ":transition_labels_bzl", ], ) @@ -649,6 +657,16 @@ bzl_library( srcs = ["toolchain_types.bzl"], ) +bzl_library( + name = "transition_labels_bzl", + srcs = ["transition_labels.bzl"], + deps = [ + "common_labels_bzl", + "@bazel_skylib//lib:collections", + "@rules_python_internal//:extra_transition_settings_bzl", + ], +) + bzl_library( name = "util_bzl", srcs = ["util.bzl"], diff --git a/python/private/attr_builders.bzl b/python/private/attr_builders.bzl index be9fa22138..ecfc570a2b 100644 --- a/python/private/attr_builders.bzl +++ b/python/private/attr_builders.bzl @@ -31,6 +31,7 @@ load( "kwargs_setter", "kwargs_setter_doc", "kwargs_setter_mandatory", + "normalize_transition_in_out_values", "to_label_maybe", ) @@ -167,6 +168,8 @@ def _AttrCfg_new( } kwargs_set_default_list(state, _INPUTS) kwargs_set_default_list(state, _OUTPUTS) + normalize_transition_in_out_values("input", state[_INPUTS]) + normalize_transition_in_out_values("output", state[_OUTPUTS]) # buildifier: disable=uninitialized self = struct( diff --git a/python/private/attributes.bzl b/python/private/attributes.bzl index 641fa13a23..0ff92e31ee 100644 --- a/python/private/attributes.bzl +++ b/python/private/attributes.bzl @@ -405,8 +405,58 @@ COVERAGE_ATTRS = { # Attributes specific to Python executable-equivalent rules. Such rules may not # accept Python sources (e.g. some packaged-version of a py_test/py_binary), but # still accept Python source-agnostic settings. +CONFIG_SETTINGS_ATTR = { + "config_settings": lambda: attrb.LabelKeyedStringDict( + doc = """ +Config settings to change for this target. + +The keys are labels for settings, and the values are strings for the new value +to use. Pass `Label` objects or canonical label strings for the keys to ensure +they resolve as expected (canonical labels start with `@@` and can be +obtained by calling `str(Label(...))`). + +Most `@rules_python//python/config_setting` settings can be used here, which +allows, for example, making only a certain `py_binary` use +{obj}`--boostrap_impl=script`. + +Additional or custom config settings can be registered using the +{obj}`add_transition_setting` API. This allows, for example, forcing a +particular CPU, or defining a custom setting that `select()` uses elsewhere +to pick between `pip.parse` hubs. See the [How to guide on multiple +versions of a library] for a more concrete example. + +:::{note} +These values are transitioned on, so will affect the analysis graph and the +associated memory overhead. The more unique configurations in your overall +build, the more memory and (often unnecessary) re-analysis and re-building +can occur. See +https://bazel.build/extending/config#memory-performance-considerations for +more information about risks and considerations. +::: + +:::{versionadded} VERSION_NEXT_FEATURE +::: +""", + ), +} + +def apply_config_settings_attr(settings, attr): + """Applies the config_settings attribute to the settings. + + Args: + settings: The settings dict to modify in-place. + attr: The rule attributes struct. + + Returns: + {type}`dict[str, object]` the input `settings` value. + """ + for key, value in attr.config_settings.items(): + settings[str(key)] = value + return settings + AGNOSTIC_EXECUTABLE_ATTRS = dicts.add( DATA_ATTRS, + CONFIG_SETTINGS_ATTR, { "env": lambda: attrb.StringDict( doc = """\ diff --git a/python/private/builders_util.bzl b/python/private/builders_util.bzl index 139084f79a..7710383cb1 100644 --- a/python/private/builders_util.bzl +++ b/python/private/builders_util.bzl @@ -15,6 +15,41 @@ """Utilities for builders.""" load("@bazel_skylib//lib:types.bzl", "types") +load(":bzlmod_enabled.bzl", "BZLMOD_ENABLED") + +def normalize_transition_in_out_values(arg_name, values): + """Normalize transition inputs/outputs to canonical label strings.""" + for i, value in enumerate(values): + values[i] = normalize_transition_in_out_value(arg_name, value) + +def normalize_transition_in_out_value(arg_name, value): + """Normalize a transition input/output value to a canonical label string. + + Args: + arg_name: {type}`str` the transition arg name, "input" or "output" + value: A label-like value to normalize. + + Returns: + {type}`str` the canonical label string. + """ + if is_label(value): + return str(value) + elif types.is_string(value): + if value.startswith("//command_line_option:"): + return value + if value.startswith("@@" if BZLMOD_ENABLED else "@"): + return value + else: + fail("transition {arg_name} invalid: non-canonical string '{value}'".format( + arg_name = arg_name, + value = value, + )) + else: + fail("transition {arg_name} invalid: ({type}) {value}".format( + arg_name = arg_name, + type = type(value), + value = repr(value), + )) def to_label_maybe(value): """Converts `value` to a `Label`, maybe. @@ -100,7 +135,7 @@ def kwargs_setter_mandatory(kwargs): """Creates a `kwargs_setter` for the `mandatory` key.""" return kwargs_setter(kwargs, "mandatory") -def list_add_unique(add_to, others): +def list_add_unique(add_to, others, convert = None): """Bulk add values to a list if not already present. Args: @@ -108,9 +143,11 @@ def list_add_unique(add_to, others): in-place. others: {type}`collection[collection[T]]` collection of collections of the values to add. + convert: {type}`callable | None` function to convert the values to add. """ existing = {v: None for v in add_to} for values in others: for value in values: + value = convert(value) if convert else value if value not in existing: add_to.append(value) diff --git a/python/private/common_labels.bzl b/python/private/common_labels.bzl new file mode 100644 index 0000000000..a55b594706 --- /dev/null +++ b/python/private/common_labels.bzl @@ -0,0 +1,27 @@ +"""Constants for common labels used in the codebase.""" + +# NOTE: str() is called because some APIs don't accept Label objects +# (e.g. transition inputs/outputs or the transition settings return dict) + +labels = struct( + # keep sorted + ADD_SRCS_TO_RUNFILES = str(Label("//python/config_settings:add_srcs_to_runfiles")), + BOOTSTRAP_IMPL = str(Label("//python/config_settings:bootstrap_impl")), + EXEC_TOOLS_TOOLCHAIN = str(Label("//python/config_settings:exec_tools_toolchain")), + PIP_ENV_MARKER_CONFIG = str(Label("//python/config_settings:pip_env_marker_config")), + PIP_WHL_MUSLC_VERSION = str(Label("//python/config_settings:pip_whl_muslc_version")), + PIP_WHL = str(Label("//python/config_settings:pip_whl")), + PIP_WHL_GLIBC_VERSION = str(Label("//python/config_settings:pip_whl_glibc_version")), + PIP_WHL_OSX_ARCH = str(Label("//python/config_settings:pip_whl_osx_arch")), + PIP_WHL_OSX_VERSION = str(Label("//python/config_settings:pip_whl_osx_version")), + PRECOMPILE = str(Label("//python/config_settings:precompile")), + PRECOMPILE_SOURCE_RETENTION = str(Label("//python/config_settings:precompile_source_retention")), + PYTHON_SRC = str(Label("//python/bin:python_src")), + PYTHON_VERSION = str(Label("//python/config_settings:python_version")), + PYTHON_VERSION_MAJOR_MINOR = str(Label("//python/config_settings:python_version_major_minor")), + PY_FREETHREADED = str(Label("//python/config_settings:py_freethreaded")), + PY_LINUX_LIBC = str(Label("//python/config_settings:py_linux_libc")), + REPL_DEP = str(Label("//python/bin:repl_dep")), + VENVS_SITE_PACKAGES = str(Label("//python/config_settings:venvs_site_packages")), + VENVS_USE_DECLARE_SYMLINK = str(Label("//python/config_settings:venvs_use_declare_symlink")), +) diff --git a/python/private/internal_config_repo.bzl b/python/private/internal_config_repo.bzl index cfe2fdfd77..b57275b672 100644 --- a/python/private/internal_config_repo.bzl +++ b/python/private/internal_config_repo.bzl @@ -18,6 +18,7 @@ such as globals available to Bazel versions, or propagating user environment settings for rules to later use. """ +load("//python/private:text_util.bzl", "render") load(":repo_utils.bzl", "repo_utils") _ENABLE_PIPSTAR_ENVVAR_NAME = "RULES_PYTHON_ENABLE_PIPSTAR" @@ -27,7 +28,7 @@ _ENABLE_PYSTAR_DEFAULT = "1" _ENABLE_DEPRECATION_WARNINGS_ENVVAR_NAME = "RULES_PYTHON_DEPRECATION_WARNINGS" _ENABLE_DEPRECATION_WARNINGS_DEFAULT = "0" -_CONFIG_TEMPLATE = """\ +_CONFIG_TEMPLATE = """ config = struct( enable_pystar = {enable_pystar}, enable_pipstar = {enable_pipstar}, @@ -40,12 +41,12 @@ config = struct( # The py_internal symbol is only accessible from within @rules_python, so we have to # load it from there and re-export it so that rules_python can later load it. -_PY_INTERNAL_SHIM = """\ +_PY_INTERNAL_SHIM = """ load("@rules_python//tools/build_defs/python/private:py_internal_renamed.bzl", "py_internal_renamed") py_internal_impl = py_internal_renamed """ -ROOT_BUILD_TEMPLATE = """\ +ROOT_BUILD_TEMPLATE = """ load("@bazel_skylib//:bzl_library.bzl", "bzl_library") package( @@ -64,6 +65,26 @@ bzl_library( srcs = ["py_internal.bzl"], deps = [{py_internal_dep}], ) + +bzl_library( + name = "extra_transition_settings_bzl", + srcs = ["extra_transition_settings.bzl"], +) +""" + +_EXTRA_TRANSITIONS_TEMPLATE = """ +# Generated by @rules_python//python/private:internal_config_repo.bzl +# +# For a list of what modules added what labels, see +# transition_settings_debug.txt + +EXTRA_TRANSITION_SETTINGS = {labels} +""" + +_TRANSITION_SETTINGS_DEBUG_TEMPLATE = """ +# Generated by @rules_python//python/private:internal_config_repo.bzl + +{lines} """ def _internal_config_repo_impl(rctx): @@ -113,12 +134,32 @@ def _internal_config_repo_impl(rctx): visibility = visibility, )) rctx.file("py_internal.bzl", shim_content) + + rctx.file( + "extra_transition_settings.bzl", + _EXTRA_TRANSITIONS_TEMPLATE.format( + labels = render.list(rctx.attr.transition_settings), + ), + ) + debug_lines = [ + "{} added by modules: {}".format(setting, ", ".join(sorted(requesters))) + for setting, requesters in rctx.attr.transition_setting_generators.items() + ] + rctx.file( + "transition_settings_debug.txt", + _TRANSITION_SETTINGS_DEBUG_TEMPLATE.format(lines = "\n".join(debug_lines)), + ) + return None internal_config_repo = repository_rule( implementation = _internal_config_repo_impl, configure = True, environ = [_ENABLE_PYSTAR_ENVVAR_NAME], + attrs = { + "transition_setting_generators": attr.string_list_dict(), + "transition_settings": attr.string_list(), + }, ) def _bool_from_environ(rctx, key, default): diff --git a/python/private/internal_deps.bzl b/python/private/internal_deps.bzl deleted file mode 100644 index 6ea3fa40c7..0000000000 --- a/python/private/internal_deps.bzl +++ /dev/null @@ -1,22 +0,0 @@ -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -"Python toolchain module extension for internal rule use" - -load("@bazel_skylib//lib:modules.bzl", "modules") -load("//python/private/pypi:deps.bzl", "pypi_deps") -load(":internal_config_repo.bzl", "internal_config_repo") - -def _internal_deps(): - internal_config_repo(name = "rules_python_internal") - pypi_deps() - -internal_deps = modules.as_extension( - _internal_deps, - doc = "This extension registers internal rules_python dependencies.", -) diff --git a/python/private/py_executable.bzl b/python/private/py_executable.bzl index 41938ebf78..98dbc7f284 100644 --- a/python/private/py_executable.bzl +++ b/python/private/py_executable.bzl @@ -29,6 +29,7 @@ load( "PrecompileAttr", "PycCollectionAttr", "REQUIRED_EXEC_GROUP_BUILDERS", + "apply_config_settings_attr", ) load(":builders.bzl", "builders") load(":cc_helper.bzl", "cc_helper") @@ -65,6 +66,7 @@ load( "TARGET_TOOLCHAIN_TYPE", TOOLCHAIN_TYPE = "TARGET_TOOLCHAIN_TYPE", ) +load(":transition_labels.bzl", "TRANSITION_LABELS") _py_builtins = py_internal _EXTERNAL_PATH_PREFIX = "external" @@ -1902,10 +1904,10 @@ def _create_run_environment_info(ctx, inherited_environment): inherited_environment = inherited_environment, ) -def _transition_executable_impl(input_settings, attr): - settings = { - _PYTHON_VERSION_FLAG: input_settings[_PYTHON_VERSION_FLAG], - } +def _transition_executable_impl(settings, attr): + settings = dict(settings) + apply_config_settings_attr(settings, attr) + if attr.python_version and attr.python_version not in ("PY2", "PY3"): settings[_PYTHON_VERSION_FLAG] = attr.python_version return settings @@ -1958,8 +1960,8 @@ def create_executable_rule_builder(implementation, **kwargs): ], cfg = dict( implementation = _transition_executable_impl, - inputs = [_PYTHON_VERSION_FLAG], - outputs = [_PYTHON_VERSION_FLAG], + inputs = TRANSITION_LABELS + [_PYTHON_VERSION_FLAG], + outputs = TRANSITION_LABELS + [_PYTHON_VERSION_FLAG], ), **kwargs ) diff --git a/python/private/py_repositories.bzl b/python/private/py_repositories.bzl index c09ba68361..3ad2a97214 100644 --- a/python/private/py_repositories.bzl +++ b/python/private/py_repositories.bzl @@ -24,15 +24,24 @@ load(":pythons_hub.bzl", "hub_repo") def http_archive(**kwargs): maybe(_http_archive, **kwargs) -def py_repositories(): +def py_repositories(transition_settings = []): """Runtime dependencies that users must install. This function should be loaded and called in the user's `WORKSPACE`. With `bzlmod` enabled, this function is not needed since `MODULE.bazel` handles transitive deps. + + Args: + transition_settings: A list of labels that terminal rules transition on + by default. """ + + # NOTE: The @rules_python_internal repo is special cased by Bazel: it + # has autoloading disabled. This allows the rules to load from it + # without triggering recursion. maybe( internal_config_repo, name = "rules_python_internal", + transition_settings = transition_settings, ) maybe( hub_repo, diff --git a/python/private/py_wheel.bzl b/python/private/py_wheel.bzl index e6352efcea..8202fa015a 100644 --- a/python/private/py_wheel.bzl +++ b/python/private/py_wheel.bzl @@ -14,9 +14,12 @@ "Implementation of py_wheel rule" +load(":attributes.bzl", "CONFIG_SETTINGS_ATTR", "apply_config_settings_attr") load(":py_info.bzl", "PyInfo") load(":py_package.bzl", "py_package_lib") +load(":rule_builders.bzl", "ruleb") load(":stamp.bzl", "is_stamping_enabled") +load(":transition_labels.bzl", "TRANSITION_LABELS") load(":version.bzl", "version") PyWheelInfo = provider( @@ -577,10 +580,15 @@ tries to locate `.runfiles` directory which is not packaged in the wheel. _requirement_attrs, _entrypoint_attrs, _other_attrs, + CONFIG_SETTINGS_ATTR, ), ) -py_wheel = rule( +def _transition_wheel_impl(settings, attr): + """Transition for py_wheel.""" + return apply_config_settings_attr(dict(settings), attr) + +py_wheel = ruleb.Rule( implementation = py_wheel_lib.implementation, doc = """\ Internal rule used by the [py_wheel macro](#py_wheel). @@ -590,4 +598,9 @@ For example, a `bazel query` for a user's `py_wheel` macro expands to `py_wheel` in the way they expect. """, attrs = py_wheel_lib.attrs, -) + cfg = transition( + implementation = _transition_wheel_impl, + inputs = TRANSITION_LABELS, + outputs = TRANSITION_LABELS, + ), +).build() diff --git a/python/private/rule_builders.bzl b/python/private/rule_builders.bzl index 360503b21b..876ca2bf97 100644 --- a/python/private/rule_builders.bzl +++ b/python/private/rule_builders.bzl @@ -108,6 +108,8 @@ load( "kwargs_setter", "kwargs_setter_doc", "list_add_unique", + "normalize_transition_in_out_value", + "normalize_transition_in_out_values", ) # Various string constants for kwarg key names used across two or more @@ -314,6 +316,9 @@ def _RuleCfg_new(rule_cfg_arg): kwargs_set_default_list(state, _INPUTS) kwargs_set_default_list(state, _OUTPUTS) + normalize_transition_in_out_values("input", state[_INPUTS]) + normalize_transition_in_out_values("output", state[_OUTPUTS]) + # buildifier: disable=uninitialized self = struct( add_inputs = lambda *a, **k: _RuleCfg_add_inputs(self, *a, **k), @@ -398,7 +403,11 @@ def _RuleCfg_update_inputs(self, *others): `Label`, not `str`, should be passed to ensure different apparent labels can be properly de-duplicated. """ - list_add_unique(self._state[_INPUTS], others) + list_add_unique( + self._state[_INPUTS], + others, + convert = lambda v: normalize_transition_in_out_value("input", v), + ) def _RuleCfg_update_outputs(self, *others): """Add a collection of values to outputs. @@ -410,7 +419,11 @@ def _RuleCfg_update_outputs(self, *others): `Label`, not `str`, should be passed to ensure different apparent labels can be properly de-duplicated. """ - list_add_unique(self._state[_OUTPUTS], others) + list_add_unique( + self._state[_OUTPUTS], + others, + convert = lambda v: normalize_transition_in_out_value("output", v), + ) # buildifier: disable=name-conventions RuleCfg = struct( diff --git a/python/private/transition_labels.bzl b/python/private/transition_labels.bzl new file mode 100644 index 0000000000..b2cf6d7d88 --- /dev/null +++ b/python/private/transition_labels.bzl @@ -0,0 +1,32 @@ +"""Flags that terminal rules should allow transitioning on by default. + +Terminal rules are e.g. py_binary, py_test, or packaging rules. +""" + +load("@bazel_skylib//lib:collections.bzl", "collections") +load("@rules_python_internal//:extra_transition_settings.bzl", "EXTRA_TRANSITION_SETTINGS") +load(":common_labels.bzl", "labels") + +_BASE_TRANSITION_LABELS = [ + labels.ADD_SRCS_TO_RUNFILES, + labels.BOOTSTRAP_IMPL, + labels.EXEC_TOOLS_TOOLCHAIN, + labels.PIP_ENV_MARKER_CONFIG, + labels.PIP_WHL_MUSLC_VERSION, + labels.PIP_WHL, + labels.PIP_WHL_GLIBC_VERSION, + labels.PIP_WHL_OSX_ARCH, + labels.PIP_WHL_OSX_VERSION, + labels.PRECOMPILE, + labels.PRECOMPILE_SOURCE_RETENTION, + labels.PYTHON_SRC, + labels.PYTHON_VERSION, + labels.PY_FREETHREADED, + labels.PY_LINUX_LIBC, + labels.VENVS_SITE_PACKAGES, + labels.VENVS_USE_DECLARE_SYMLINK, +] + +TRANSITION_LABELS = collections.uniq( + _BASE_TRANSITION_LABELS + EXTRA_TRANSITION_SETTINGS, +) diff --git a/tests/builders/rule_builders_tests.bzl b/tests/builders/rule_builders_tests.bzl index 9a91ceb062..3f14832d80 100644 --- a/tests/builders/rule_builders_tests.bzl +++ b/tests/builders/rule_builders_tests.bzl @@ -153,11 +153,11 @@ def _test_rule_api(env): expect.that_bool(subject.cfg.implementation()).equals(impl) subject.cfg.add_inputs(Label("//some:input")) expect.that_collection(subject.cfg.inputs()).contains_exactly([ - Label("//some:input"), + str(Label("//some:input")), ]) subject.cfg.add_outputs(Label("//some:output")) expect.that_collection(subject.cfg.outputs()).contains_exactly([ - Label("//some:output"), + str(Label("//some:output")), ]) _basic_tests.append(_test_rule_api) diff --git a/tests/multi_pypi/BUILD.bazel b/tests/multi_pypi/BUILD.bazel new file mode 100644 index 0000000000..a119ebe116 --- /dev/null +++ b/tests/multi_pypi/BUILD.bazel @@ -0,0 +1,29 @@ +load("@bazel_skylib//rules:common_settings.bzl", "string_flag") +load("//python:defs.bzl", "py_library") + +string_flag( + name = "external_deps_name", + build_setting_default = "", + visibility = ["//visibility:public"], +) + +py_library( + name = "common", + srcs = [], + visibility = ["//visibility:public"], + deps = select({ + ":is_external_alpha": ["@pypi_alpha//more_itertools"], + ":is_external_beta": ["@pypi_beta//more_itertools"], + "//conditions:default": [], + }), +) + +config_setting( + name = "is_external_alpha", + flag_values = {"//tests/multi_pypi:external_deps_name": "alpha"}, +) + +config_setting( + name = "is_external_beta", + flag_values = {"//tests/multi_pypi:external_deps_name": "beta"}, +) diff --git a/tests/multi_pypi/alpha/BUILD.bazel b/tests/multi_pypi/alpha/BUILD.bazel new file mode 100644 index 0000000000..7b56e0a547 --- /dev/null +++ b/tests/multi_pypi/alpha/BUILD.bazel @@ -0,0 +1,7 @@ +load("//python/uv:lock.bzl", "lock") + +lock( + name = "requirements", + srcs = ["pyproject.toml"], + out = "requirements.txt", +) diff --git a/tests/multi_pypi/alpha/pyproject.toml b/tests/multi_pypi/alpha/pyproject.toml new file mode 100644 index 0000000000..8f99cd08fc --- /dev/null +++ b/tests/multi_pypi/alpha/pyproject.toml @@ -0,0 +1,6 @@ +[project] +name = "multi-pypi-test-alpha" +version = "0.1.0" +dependencies = [ + "more-itertools==9.1.0" +] diff --git a/tests/multi_pypi/alpha/requirements.txt b/tests/multi_pypi/alpha/requirements.txt new file mode 100644 index 0000000000..febb6b72ae --- /dev/null +++ b/tests/multi_pypi/alpha/requirements.txt @@ -0,0 +1,6 @@ +# This file was autogenerated by uv via the following command: +# bazel run //tests/multi_pypi/alpha:requirements.update +more-itertools==9.1.0 \ + --hash=sha256:cabaa341ad0389ea83c17a94566a53ae4c9d07349861ecb14dc6d0345cf9ac5d \ + --hash=sha256:d2bc7f02446e86a68911e58ded76d6561eea00cddfb2a91e7019bbb586c799f3 + # via multi-pypi-test-alpha (tests/multi_pypi/alpha/pyproject.toml) diff --git a/tests/multi_pypi/beta/BUILD.bazel b/tests/multi_pypi/beta/BUILD.bazel new file mode 100644 index 0000000000..7b56e0a547 --- /dev/null +++ b/tests/multi_pypi/beta/BUILD.bazel @@ -0,0 +1,7 @@ +load("//python/uv:lock.bzl", "lock") + +lock( + name = "requirements", + srcs = ["pyproject.toml"], + out = "requirements.txt", +) diff --git a/tests/multi_pypi/beta/pyproject.toml b/tests/multi_pypi/beta/pyproject.toml new file mode 100644 index 0000000000..02a510ffa2 --- /dev/null +++ b/tests/multi_pypi/beta/pyproject.toml @@ -0,0 +1,6 @@ +[project] +name = "multi-pypi-test-beta" +version = "0.1.0" +dependencies = [ + "more-itertools==9.0.0" +] diff --git a/tests/multi_pypi/beta/requirements.txt b/tests/multi_pypi/beta/requirements.txt new file mode 100644 index 0000000000..de05f6dc9a --- /dev/null +++ b/tests/multi_pypi/beta/requirements.txt @@ -0,0 +1,6 @@ +# This file was autogenerated by uv via the following command: +# bazel run //tests/multi_pypi/beta:requirements.update +more-itertools==9.0.0 \ + --hash=sha256:250e83d7e81d0c87ca6bd942e6aeab8cc9daa6096d12c5308f3f92fa5e5c1f41 \ + --hash=sha256:5a6257e40878ef0520b1803990e3e22303a41b5714006c32a3fd8304b26ea1ab + # via multi-pypi-test-beta (tests/multi_pypi/beta/pyproject.toml) diff --git a/tests/multi_pypi/pypi_alpha/BUILD.bazel b/tests/multi_pypi/pypi_alpha/BUILD.bazel new file mode 100644 index 0000000000..47e3b2fa88 --- /dev/null +++ b/tests/multi_pypi/pypi_alpha/BUILD.bazel @@ -0,0 +1,11 @@ +load("//tests/support:py_reconfig.bzl", "py_reconfig_test") + +py_reconfig_test( + name = "pypi_alpha_test", + srcs = ["pypi_alpha_test.py"], + config_settings = { + "//tests/multi_pypi:external_deps_name": "alpha", + }, + main = "pypi_alpha_test.py", + deps = ["//tests/multi_pypi:common"], +) diff --git a/tests/multi_pypi/pypi_alpha/pypi_alpha_test.py b/tests/multi_pypi/pypi_alpha/pypi_alpha_test.py new file mode 100644 index 0000000000..0521327563 --- /dev/null +++ b/tests/multi_pypi/pypi_alpha/pypi_alpha_test.py @@ -0,0 +1,8 @@ +import sys + +from more_itertools import __version__ + +if __name__ == "__main__": + expected_version = "9.1.0" + if __version__ != expected_version: + sys.exit(f"Expected version {expected_version}, got {__version__}") diff --git a/tests/multi_pypi/pypi_beta/BUILD.bazel b/tests/multi_pypi/pypi_beta/BUILD.bazel new file mode 100644 index 0000000000..077d87bdf0 --- /dev/null +++ b/tests/multi_pypi/pypi_beta/BUILD.bazel @@ -0,0 +1,11 @@ +load("//tests/support:py_reconfig.bzl", "py_reconfig_test") + +py_reconfig_test( + name = "pypi_beta_test", + srcs = ["pypi_beta_test.py"], + config_settings = { + "//tests/multi_pypi:external_deps_name": "beta", + }, + main = "pypi_beta_test.py", + deps = ["//tests/multi_pypi:common"], +) diff --git a/tests/multi_pypi/pypi_beta/pypi_beta_test.py b/tests/multi_pypi/pypi_beta/pypi_beta_test.py new file mode 100644 index 0000000000..8c34de0735 --- /dev/null +++ b/tests/multi_pypi/pypi_beta/pypi_beta_test.py @@ -0,0 +1,8 @@ +import sys + +from more_itertools import __version__ + +if __name__ == "__main__": + expected_version = "9.0.0" + if __version__ != expected_version: + sys.exit(f"Expected version {expected_version}, got {__version__}") diff --git a/tests/py_wheel/py_wheel_tests.bzl b/tests/py_wheel/py_wheel_tests.bzl index 43c068e597..75fef3a622 100644 --- a/tests/py_wheel/py_wheel_tests.bzl +++ b/tests/py_wheel/py_wheel_tests.bzl @@ -14,9 +14,10 @@ """Test for py_wheel.""" load("@rules_testing//lib:analysis_test.bzl", "analysis_test", "test_suite") -load("@rules_testing//lib:truth.bzl", "matching") +load("@rules_testing//lib:truth.bzl", "matching", "subjects") load("@rules_testing//lib:util.bzl", rt_util = "util") load("//python:packaging.bzl", "py_wheel") +load("//python/private:common_labels.bzl", "labels") # buildifier: disable=bzl-visibility _basic_tests = [] _tests = [] @@ -167,6 +168,44 @@ def _test_content_type_from_description_impl(env, target): _tests.append(_test_content_type_from_description) +def _test_config_settings(name): + rt_util.helper_target( + native.config_setting, + name = "is_py_39", + flag_values = { + labels.PYTHON_VERSION_MAJOR_MINOR: "3.9", + }, + ) + rt_util.helper_target( + py_wheel, + name = name + "_subject", + distribution = "mydist_" + name, + version = select({ + ":is_py_39": "3.9", + "//conditions:default": "not-3.9", + }), + config_settings = { + labels.PYTHON_VERSION: "3.9", + }, + ) + analysis_test( + name = name, + impl = _test_config_settings_impl, + target = name + "_subject", + config_settings = { + # Ensure a different value than the target under test. + labels.PYTHON_VERSION: "3.11", + }, + ) + +def _test_config_settings_impl(env, target): + env.expect.that_target(target).attr( + "version", + factory = subjects.str, + ).equals("3.9") + +_tests.append(_test_config_settings) + def py_wheel_test_suite(name): test_suite( name = name, diff --git a/tests/support/py_reconfig.bzl b/tests/support/py_reconfig.bzl index b33f679e77..38d53667fd 100644 --- a/tests/support/py_reconfig.bzl +++ b/tests/support/py_reconfig.bzl @@ -18,11 +18,12 @@ without the overhead of a bazel-in-bazel integration test. """ load("//python/private:attr_builders.bzl", "attrb") # buildifier: disable=bzl-visibility +load("//python/private:common_labels.bzl", "labels") # buildifier: disable=bzl-visibility load("//python/private:py_binary_macro.bzl", "py_binary_macro") # buildifier: disable=bzl-visibility load("//python/private:py_binary_rule.bzl", "create_py_binary_rule_builder") # buildifier: disable=bzl-visibility load("//python/private:py_test_macro.bzl", "py_test_macro") # buildifier: disable=bzl-visibility load("//python/private:py_test_rule.bzl", "create_py_test_rule_builder") # buildifier: disable=bzl-visibility -load("//tests/support:support.bzl", "VISIBLE_FOR_TESTING") +load("//tests/support:support.bzl", "CUSTOM_RUNTIME", "VISIBLE_FOR_TESTING") def _perform_transition_impl(input_settings, attr, base_impl): settings = {k: input_settings[k] for k in _RECONFIG_INHERITED_OUTPUTS if k in input_settings} @@ -31,26 +32,29 @@ def _perform_transition_impl(input_settings, attr, base_impl): settings[VISIBLE_FOR_TESTING] = True settings["//command_line_option:build_python_zip"] = attr.build_python_zip if attr.bootstrap_impl: - settings["//python/config_settings:bootstrap_impl"] = attr.bootstrap_impl + settings[labels.BOOTSTRAP_IMPL] = attr.bootstrap_impl if attr.extra_toolchains: settings["//command_line_option:extra_toolchains"] = attr.extra_toolchains if attr.python_src: - settings["//python/bin:python_src"] = attr.python_src + settings[labels.PYTHON_SRC] = attr.python_src if attr.repl_dep: - settings["//python/bin:repl_dep"] = attr.repl_dep + settings[labels.REPL_DEP] = attr.repl_dep if attr.venvs_use_declare_symlink: - settings["//python/config_settings:venvs_use_declare_symlink"] = attr.venvs_use_declare_symlink + settings[labels.VENVS_USE_DECLARE_SYMLINK] = attr.venvs_use_declare_symlink if attr.venvs_site_packages: - settings["//python/config_settings:venvs_site_packages"] = attr.venvs_site_packages + settings[labels.VENVS_SITE_PACKAGES] = attr.venvs_site_packages + for key, value in attr.config_settings.items(): + settings[str(key)] = value return settings _RECONFIG_INPUTS = [ - "//python/config_settings:bootstrap_impl", - "//python/bin:python_src", - "//python/bin:repl_dep", "//command_line_option:extra_toolchains", - "//python/config_settings:venvs_use_declare_symlink", - "//python/config_settings:venvs_site_packages", + CUSTOM_RUNTIME, + labels.BOOTSTRAP_IMPL, + labels.PYTHON_SRC, + labels.REPL_DEP, + labels.VENVS_SITE_PACKAGES, + labels.VENVS_USE_DECLARE_SYMLINK, ] _RECONFIG_OUTPUTS = _RECONFIG_INPUTS + [ "//command_line_option:build_python_zip", @@ -61,6 +65,7 @@ _RECONFIG_INHERITED_OUTPUTS = [v for v in _RECONFIG_OUTPUTS if v in _RECONFIG_IN _RECONFIG_ATTRS = { "bootstrap_impl": attrb.String(), "build_python_zip": attrb.String(default = "auto"), + "config_settings": attrb.LabelKeyedStringDict(), "extra_toolchains": attrb.StringList( doc = """ Value for the --extra_toolchains flag. diff --git a/tests/support/sh_py_run_test.bzl b/tests/support/sh_py_run_test.bzl index 49445ed304..83ac2c814b 100644 --- a/tests/support/sh_py_run_test.bzl +++ b/tests/support/sh_py_run_test.bzl @@ -18,85 +18,8 @@ without the overhead of a bazel-in-bazel integration test. """ load("@rules_shell//shell:sh_test.bzl", "sh_test") -load("//python/private:attr_builders.bzl", "attrb") # buildifier: disable=bzl-visibility -load("//python/private:py_binary_macro.bzl", "py_binary_macro") # buildifier: disable=bzl-visibility -load("//python/private:py_binary_rule.bzl", "create_py_binary_rule_builder") # buildifier: disable=bzl-visibility -load("//python/private:py_test_macro.bzl", "py_test_macro") # buildifier: disable=bzl-visibility -load("//python/private:py_test_rule.bzl", "create_py_test_rule_builder") # buildifier: disable=bzl-visibility load("//python/private:toolchain_types.bzl", "TARGET_TOOLCHAIN_TYPE") # buildifier: disable=bzl-visibility -load("//tests/support:support.bzl", "VISIBLE_FOR_TESTING") - -def _perform_transition_impl(input_settings, attr, base_impl): - settings = {k: input_settings[k] for k in _RECONFIG_INHERITED_OUTPUTS if k in input_settings} - settings.update(base_impl(input_settings, attr)) - - settings[VISIBLE_FOR_TESTING] = True - settings["//command_line_option:build_python_zip"] = attr.build_python_zip - - for attr_name, setting_label in _RECONFIG_ATTR_SETTING_MAP.items(): - if getattr(attr, attr_name): - settings[setting_label] = getattr(attr, attr_name) - return settings - -# Attributes that, if non-falsey (`if attr.`), will copy their -# value into the output settings -_RECONFIG_ATTR_SETTING_MAP = { - "bootstrap_impl": "//python/config_settings:bootstrap_impl", - "custom_runtime": "//tests/support:custom_runtime", - "extra_toolchains": "//command_line_option:extra_toolchains", - "python_src": "//python/bin:python_src", - "venvs_site_packages": "//python/config_settings:venvs_site_packages", - "venvs_use_declare_symlink": "//python/config_settings:venvs_use_declare_symlink", -} - -_RECONFIG_INPUTS = _RECONFIG_ATTR_SETTING_MAP.values() -_RECONFIG_OUTPUTS = _RECONFIG_INPUTS + [ - "//command_line_option:build_python_zip", - VISIBLE_FOR_TESTING, -] -_RECONFIG_INHERITED_OUTPUTS = [v for v in _RECONFIG_OUTPUTS if v in _RECONFIG_INPUTS] - -_RECONFIG_ATTRS = { - "bootstrap_impl": attrb.String(), - "build_python_zip": attrb.String(default = "auto"), - "custom_runtime": attrb.String(), - "extra_toolchains": attrb.StringList( - doc = """ -Value for the --extra_toolchains flag. - -NOTE: You'll likely have to also specify //tests/support/cc_toolchains:all (or some CC toolchain) -to make the RBE presubmits happy, which disable auto-detection of a CC -toolchain. -""", - ), - "python_src": attrb.Label(), - "venvs_site_packages": attrb.String(), - "venvs_use_declare_symlink": attrb.String(), -} - -def _create_reconfig_rule(builder): - builder.attrs.update(_RECONFIG_ATTRS) - - base_cfg_impl = builder.cfg.implementation() - builder.cfg.set_implementation(lambda *args: _perform_transition_impl(base_impl = base_cfg_impl, *args)) - builder.cfg.update_inputs(_RECONFIG_INPUTS) - builder.cfg.update_outputs(_RECONFIG_OUTPUTS) - return builder.build() - -_py_reconfig_binary = _create_reconfig_rule(create_py_binary_rule_builder()) - -_py_reconfig_test = _create_reconfig_rule(create_py_test_rule_builder()) - -def py_reconfig_test(**kwargs): - """Create a py_test with customized build settings for testing. - - Args: - **kwargs: kwargs to pass along to _py_reconfig_test. - """ - py_test_macro(_py_reconfig_test, **kwargs) - -def py_reconfig_binary(**kwargs): - py_binary_macro(_py_reconfig_binary, **kwargs) +load(":py_reconfig.bzl", "py_reconfig_binary") def sh_py_run_test(*, name, sh_src, py_src, **kwargs): """Run a py_binary within a sh_test. diff --git a/tests/support/support.bzl b/tests/support/support.bzl index f8694629c1..28cab0dcbf 100644 --- a/tests/support/support.bzl +++ b/tests/support/support.bzl @@ -44,6 +44,7 @@ PRECOMPILE_SOURCE_RETENTION = str(Label("//python/config_settings:precompile_sou PYC_COLLECTION = str(Label("//python/config_settings:pyc_collection")) PYTHON_VERSION = str(Label("//python/config_settings:python_version")) VISIBLE_FOR_TESTING = str(Label("//python/private:visible_for_testing")) +CUSTOM_RUNTIME = str(Label("//tests/support:custom_runtime")) SUPPORTS_BOOTSTRAP_SCRIPT = select({ "@platforms//os:windows": ["@platforms//:incompatible"], diff --git a/tests/toolchains/BUILD.bazel b/tests/toolchains/BUILD.bazel index b9952865cb..f32ab6f056 100644 --- a/tests/toolchains/BUILD.bazel +++ b/tests/toolchains/BUILD.bazel @@ -14,7 +14,7 @@ load("@bazel_skylib//rules:build_test.bzl", "build_test") load("//python/private:bzlmod_enabled.bzl", "BZLMOD_ENABLED") # buildifier: disable=bzl-visibility -load("//tests/support:sh_py_run_test.bzl", "py_reconfig_test") +load("//tests/support:py_reconfig.bzl", "py_reconfig_test") load(":defs.bzl", "define_toolchain_tests") define_toolchain_tests( @@ -24,7 +24,7 @@ define_toolchain_tests( py_reconfig_test( name = "custom_platform_toolchain_test", srcs = ["custom_platform_toolchain_test.py"], - custom_runtime = "linux-x86-install-only-stripped", + config_settings = {"//tests/support:custom_runtime": "linux-x86-install-only-stripped"}, python_version = "3.13.1", target_compatible_with = [ "@platforms//os:linux", From d91401ce196a6855c2856f211cc6f6c218a501db Mon Sep 17 00:00:00 2001 From: Ed Schouten Date: Tue, 9 Sep 2025 16:59:47 +0200 Subject: [PATCH 431/922] fix: ensure the stage1 bootstrap is executable (#3258) Bazel tends to make files executable, even if ctx.actions.write() or ctx.actions.expand_template() is called without is_executable = True. However, in an analysis tool of mine that is a bit more pedantic than Bazel this leads to the issue that py_binary() targets can't be executed due to them not having a +x bit. Considering that the stage2 bootstrap is marked executable, let's mark is_executable for consistency. --- python/private/py_executable.bzl | 1 + 1 file changed, 1 insertion(+) diff --git a/python/private/py_executable.bzl b/python/private/py_executable.bzl index 98dbc7f284..fa80ea5105 100644 --- a/python/private/py_executable.bzl +++ b/python/private/py_executable.bzl @@ -894,6 +894,7 @@ def _create_stage1_bootstrap( template = template, output = output, substitutions = subs, + is_executable = True, ) def _create_windows_exe_launcher( From cdd933879e9b2b172dde6360eff119485dd2f88a Mon Sep 17 00:00:00 2001 From: Ed Schouten Date: Tue, 9 Sep 2025 17:28:47 +0200 Subject: [PATCH 432/922] fix: don't call Args.add() with an integer (#3259) The documentation for Bazel's Args states that standard conversion rules are only specified for strings, Files, and Labels. For all other types the conversion to a string is done in an unspecified manner, which is why it should be avoided. Let's stay away from this unspecified behaviour by explicitly converting the precompile optimization level to a string before calling Args.add(). --- python/private/precompile.bzl | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/python/private/precompile.bzl b/python/private/precompile.bzl index 23e8f81426..c12882bf82 100644 --- a/python/private/precompile.bzl +++ b/python/private/precompile.bzl @@ -182,7 +182,7 @@ def _precompile(ctx, src, *, use_pycache): # have the repo name, which is likely to contain extraneous info. precompile_request_args.add("--src_name", src.short_path) precompile_request_args.add("--pyc", pyc) - precompile_request_args.add("--optimize", ctx.attr.precompile_optimize_level) + precompile_request_args.add("--optimize", str(ctx.attr.precompile_optimize_level)) version_info = target_toolchain.interpreter_version_info python_version = "{}.{}".format(version_info.major, version_info.minor) From b67b9b6f3a993ce901a8862b1dc8a25d7e0c4253 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Tue, 9 Sep 2025 13:33:06 -0700 Subject: [PATCH 433/922] docs: update changelog for config_settings attribute (#3257) Add the config_settings and bzlmod/workspace apis to changelog. Along the way, fix the filename for the common deps with pypi guide. --------- Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> --- CHANGELOG.md | 9 ++++++++- ...ons.md => common-deps-with-multiple-pypi-versions.md} | 1 + 2 files changed, 9 insertions(+), 1 deletion(-) rename docs/howto/{common-deps-with-multipe-pypi-versions.md => common-deps-with-multiple-pypi-versions.md} (98%) diff --git a/CHANGELOG.md b/CHANGELOG.md index 55d0d3fa2f..abbd5f5cf1 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -85,7 +85,14 @@ END_UNRELEASED_TEMPLATE * (bootstrap) {obj}`--bootstrap_impl=system_python` now supports the {obj}`main_module` attribute. * (bootstrap) {obj}`--bootstrap_impl=system_python` now supports the - {any}`RULES_PYTHON_ADDITIONAL_INTERPRETER_ARGS` attribute. + {any}`RULES_PYTHON_ADDITIONAL_INTERPRETER_ARGS` environment variable. +* (rules) The `py_binary`, `py_test`, and `py_wheel` rules now have a + {obj}`config_settings` attribute to control build flags within the build graph. + Custom settings can be added using {obj}`config.add_transition_setting` in + `MODULE.bazel` files, or {obj}`py_repositories(transition_settings=...)` in + `WORKSPACE` files. See the + {ref}`common-deps-with-multiple-pypi-versions` guide on using common + dependencies with multiple PyPI versions` for an example. {#v1-6-0} diff --git a/docs/howto/common-deps-with-multipe-pypi-versions.md b/docs/howto/common-deps-with-multiple-pypi-versions.md similarity index 98% rename from docs/howto/common-deps-with-multipe-pypi-versions.md rename to docs/howto/common-deps-with-multiple-pypi-versions.md index ba3568682f..3b933d22f4 100644 --- a/docs/howto/common-deps-with-multipe-pypi-versions.md +++ b/docs/howto/common-deps-with-multiple-pypi-versions.md @@ -1,3 +1,4 @@ +(common-deps-with-multiple-pypi-versions)= # How to use a common set of dependencies with multiple PyPI versions In this guide, we show how to handle a situation common to monorepos From 35adf5c2379170eaf2e9529c8015fb9fba5770eb Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Wed, 10 Sep 2025 19:30:24 -0700 Subject: [PATCH 434/922] chore: add agents guidance for creating bzl_library targets (#3264) I found it was doing a very poor job, so some guidance is needed. --------- Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> --- AGENTS.md | 30 ++++++++++++++++++++++++++++++ 1 file changed, 30 insertions(+) diff --git a/AGENTS.md b/AGENTS.md index 9a6c016a36..e21b15bd03 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -19,6 +19,28 @@ using a grandoise title. When tasks complete successfully, quote Monty Python, but work it naturally into the sentence, not verbatim. +### bzl_library targets for bzl source files + +* `.bzl` files should have `bzl_library` defined for them. +* They should have a single `srcs` file and be named after the file with `_bzl` + appended. +* Their deps should be based on the `load()` statements in the source file. +* `bzl_library()` targets should be kept in alphabetical order by name. + +Example: + +``` +bzl_library( + name = "alpha_bzl", + srcs = ["alpha.bzl"], + deps = [":beta_bzl"], +) +bzl_library( + name = "beta_bzl", + srcs = ["beta.bzl"] +) +``` + ## Building and testing Tests are under the `tests/` directory. @@ -67,3 +89,11 @@ When modifying locked/resolved requirements files: When building `//docs:docs`, ignore an error about exit code 2; this is a flake, so try building again. + +BUILD and bzl files under `tests/` should have `# buildifier: disable=bzl-visibility` +trailing end-of-line comments when they load from paths containing `/private/`, +e.g. + +``` +load("//python/private:foo.bzl", "foo") # buildifier: disable=bzl-visibility +``` From 029a4dc45cb34384d8ee2ceb57e7306eeec9f6dd Mon Sep 17 00:00:00 2001 From: Ben Axelrod Date: Thu, 11 Sep 2025 15:56:51 -0400 Subject: [PATCH 435/922] docs: improve whl_library documentation (#3266) The documentation for the `whl_patches` argument of `whl_library` contained an error. I fixed it and also elaborated the text in places that tripped me up. --------- Co-authored-by: Richard Levasseur --- python/private/pypi/whl_library.bzl | 31 ++++++++++++++++++++++++----- 1 file changed, 26 insertions(+), 5 deletions(-) diff --git a/python/private/pypi/whl_library.bzl b/python/private/pypi/whl_library.bzl index b1aaf4f062..5cc53d84c6 100644 --- a/python/private/pypi/whl_library.bzl +++ b/python/private/pypi/whl_library.bzl @@ -577,6 +577,9 @@ whl_library_attrs = dict({ The dep template to use for referencing the dependencies. It should have `{name}` and `{target}` tokens that will be replaced with the normalized distribution name and the target that we need respectively. + +For example if your whl depends on `numpy` and your Python package repo is named +`pip` so that you would normally do `@pip//numpy`, then this should be: `@pip//{name}`. """, ), "filename": attr.string( @@ -615,11 +618,29 @@ attr makes `extra_pip_args` and `download_only` ignored.""", doc = "The whl file that should be used instead of downloading or building the whl.", ), "whl_patches": attr.label_keyed_string_dict( - doc = """a label-keyed-string dict that has - json.encode(struct([whl_file], patch_strip]) as values. This - is to maintain flexibility and correct bzlmod extension interface - until we have a better way to define whl_library and move whl - patching to a separate place. INTERNAL USE ONLY.""", + doc = """ +A label-keyed-string dict with patch files as keys and json-strings as values. + +The keys are labels to the patch file to apply. + +The values describe what to apply the patch to and how to apply it. +It is encoded as `json.encode(struct([whls], patch_strip])`, +where `whls` is a `list[str`] of wheel filenames, and `patch_strip` +is a number. + +So it will look something like this: +``` +"//path/to/package:my.patch": json.encode(struct( + whls = ["something-2.7.1-py3-none-any.whl"], + patch_strip = 1, +)), +``` +The patch is applied within the scope of the .whl file. +I.e. you should create the patch from the same place you unziped the wheel. + + +This is to maintain flexibility and correct bzlmod extension interface until we have a better +way to define whl_library and move whl patching to a separate place. INTERNAL USE ONLY.""", ), "_python_path_entries": attr.label_list( # Get the root directory of these rules and keep them as a default attribute From 668a551e0253ef205924e2bcecfc74468fae4983 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Sun, 14 Sep 2025 20:12:22 -0700 Subject: [PATCH 436/922] refactor: use common_labels.bzl for labels used across files (#3263) Cleanup after the PR introducing the common labels file. Testing related labels (those starting with `//tests`) are left in `tests/support/support.bzl`. Only labels that are used in two or more files are moved into common_labels. This avoids obscuring otherwise simple assignments like defaults for attributes. It also acts as a signal that, if something is in common_labels.bzl, be ware it's used in multiple places. Only non-testing related labels (those not under `//tests`) are moved into common_labels. --- python/BUILD.bazel | 6 ++---- python/extensions/BUILD.bazel | 1 + python/private/BUILD.bazel | 11 +++++++++- python/private/attributes.bzl | 5 +++-- python/private/common_labels.bzl | 5 ++++- python/private/py_cc_toolchain_rule.bzl | 3 ++- python/private/py_exec_tools_toolchain.bzl | 3 ++- python/private/py_executable.bzl | 21 +++++++------------ python/private/py_library.bzl | 3 ++- python/private/py_runtime_pair_rule.bzl | 3 ++- python/private/py_runtime_rule.bzl | 5 +++-- python/private/pypi/BUILD.bazel | 3 +++ python/private/pypi/env_marker_setting.bzl | 7 ++++--- python/private/pypi/flags.bzl | 7 ++++--- python/private/pypi/pkg_aliases.bzl | 3 ++- python/uv/private/BUILD.bazel | 2 ++ python/uv/private/lock.bzl | 11 +++++----- python/uv/private/uv.bzl | 3 ++- .../precompile/precompile_tests.bzl | 16 +++++++------- tests/base_rules/py_executable_base_tests.bzl | 7 ++++--- tests/base_rules/py_test/py_test_tests.bzl | 4 ++-- tests/builders/attr_builders_tests.bzl | 3 ++- tests/builders/rule_builders_tests.bzl | 5 +++-- .../exec_toolchain_matching_tests.bzl | 5 +++-- tests/py_runtime/py_runtime_tests.bzl | 4 ++-- .../env_marker_setting_tests.bzl | 8 +++---- tests/pypi/pkg_aliases/pkg_aliases_test.bzl | 3 ++- .../runtime_env_toolchain_tests.bzl | 7 ++++--- tests/support/py_reconfig.bzl | 6 +++--- tests/support/sh_py_run_test.bzl | 3 ++- tests/support/support.bzl | 9 -------- .../transitions/transitions_tests.bzl | 10 ++++----- tests/uv/uv/uv_tests.bzl | 3 ++- 33 files changed, 106 insertions(+), 89 deletions(-) diff --git a/python/BUILD.bazel b/python/BUILD.bazel index 58cff5b99d..76fa5dde6e 100644 --- a/python/BUILD.bazel +++ b/python/BUILD.bazel @@ -56,6 +56,7 @@ filegroup( bzl_library( name = "current_py_toolchain_bzl", srcs = ["current_py_toolchain.bzl"], + deps = ["//python/private:toolchain_types_bzl"], ) bzl_library( @@ -91,11 +92,9 @@ bzl_library( deps = [ ":py_binary_bzl", "//python/private:bzlmod_enabled_bzl", - "//python/private:py_package.bzl", + "//python/private:py_package_bzl", "//python/private:py_wheel_bzl", - "//python/private:stamp_bzl", "//python/private:util_bzl", - "//python/private:version.bzl", "@bazel_skylib//rules:native_binary", ], ) @@ -215,7 +214,6 @@ bzl_library( deps = [ "//python/private:py_runtime_info_bzl", "//python/private:reexports_bzl", - "//python/private:util_bzl", "@rules_python_internal//:rules_python_config_bzl", ], ) diff --git a/python/extensions/BUILD.bazel b/python/extensions/BUILD.bazel index e6c876c76f..12c0f248fe 100644 --- a/python/extensions/BUILD.bazel +++ b/python/extensions/BUILD.bazel @@ -46,5 +46,6 @@ bzl_library( visibility = ["//:__subpackages__"], deps = [ "//python/private:internal_config_repo_bzl", + "//python/private/pypi:deps_bzl", ], ) diff --git a/python/private/BUILD.bazel b/python/private/BUILD.bazel index f31b56ec50..916c14f9f2 100644 --- a/python/private/BUILD.bazel +++ b/python/private/BUILD.bazel @@ -66,6 +66,7 @@ bzl_library( deps = [ ":attr_builders_bzl", ":common_bzl", + ":common_labels_bzl", ":enum_bzl", ":flags_bzl", ":py_info_bzl", @@ -356,6 +357,7 @@ bzl_library( name = "py_cc_toolchain_rule_bzl", srcs = ["py_cc_toolchain_rule.bzl"], deps = [ + ":common_labels.bzl", ":py_cc_toolchain_info_bzl", ":rules_cc_srcs_bzl", ":util_bzl", @@ -390,6 +392,7 @@ bzl_library( srcs = ["py_exec_tools_toolchain.bzl"], deps = [ ":common_bzl", + ":common_labels_bzl", ":py_exec_tools_info_bzl", ":sentinel_bzl", ":toolchain_types_bzl", @@ -405,6 +408,7 @@ bzl_library( ":attributes_bzl", ":cc_helper_bzl", ":common_bzl", + ":common_labels_bzl", ":flags_bzl", ":precompile_bzl", ":py_cc_link_params_info_bzl", @@ -456,6 +460,7 @@ bzl_library( deps = [ ":attributes_bzl", ":common_bzl", + ":common_labels_bzl", ":flags_bzl", ":normalize_name_bzl", ":precompile_bzl", @@ -479,6 +484,7 @@ bzl_library( name = "py_library_rule_bzl", srcs = ["py_library_rule.bzl"], deps = [ + ":common_labels_bzl", ":py_library_bzl", ], ) @@ -522,6 +528,7 @@ bzl_library( srcs = ["py_runtime_rule.bzl"], deps = [ ":attributes_bzl", + ":common_labels_bzl", ":flags_bzl", ":py_internal_bzl", ":py_runtime_info_bzl", @@ -545,6 +552,7 @@ bzl_library( name = "py_runtime_pair_rule_bzl", srcs = ["py_runtime_pair_rule.bzl"], deps = [ + ":common_labels_bzl", "//python:py_runtime_bzl", "//python:py_runtime_info_bzl", "@bazel_skylib//rules:common_settings", @@ -591,6 +599,7 @@ bzl_library( ":py_package_bzl", ":stamp_bzl", ":transition_labels_bzl", + ":version_bzl", ], ) @@ -661,7 +670,7 @@ bzl_library( name = "transition_labels_bzl", srcs = ["transition_labels.bzl"], deps = [ - "common_labels_bzl", + ":common_labels_bzl", "@bazel_skylib//lib:collections", "@rules_python_internal//:extra_transition_settings_bzl", ], diff --git a/python/private/attributes.bzl b/python/private/attributes.bzl index 0ff92e31ee..8151a30fad 100644 --- a/python/private/attributes.bzl +++ b/python/private/attributes.bzl @@ -17,6 +17,7 @@ load("@bazel_skylib//lib:dicts.bzl", "dicts") load("@bazel_skylib//rules:common_settings.bzl", "BuildSettingInfo") load("@rules_cc//cc/common:cc_info.bzl", "CcInfo") load(":attr_builders.bzl", "attrb") +load(":common_labels.bzl", "labels") load(":enum.bzl", "enum") load(":flags.bzl", "PrecompileFlag", "PrecompileSourceRetentionFlag") load(":py_info.bzl", "PyInfo") @@ -370,11 +371,11 @@ files that may be needed at run time belong in `data`. doc = "Defunct, unused, does nothing.", ), "_precompile_flag": lambda: attrb.Label( - default = "//python/config_settings:precompile", + default = labels.PRECOMPILE, providers = [BuildSettingInfo], ), "_precompile_source_retention_flag": lambda: attrb.Label( - default = "//python/config_settings:precompile_source_retention", + default = labels.PRECOMPILE_SOURCE_RETENTION, providers = [BuildSettingInfo], ), # Force enabling auto exec groups, see diff --git a/python/private/common_labels.bzl b/python/private/common_labels.bzl index a55b594706..4a6f6d3f0f 100644 --- a/python/private/common_labels.bzl +++ b/python/private/common_labels.bzl @@ -9,13 +9,15 @@ labels = struct( BOOTSTRAP_IMPL = str(Label("//python/config_settings:bootstrap_impl")), EXEC_TOOLS_TOOLCHAIN = str(Label("//python/config_settings:exec_tools_toolchain")), PIP_ENV_MARKER_CONFIG = str(Label("//python/config_settings:pip_env_marker_config")), - PIP_WHL_MUSLC_VERSION = str(Label("//python/config_settings:pip_whl_muslc_version")), + NONE = str(Label("//python:none")), PIP_WHL = str(Label("//python/config_settings:pip_whl")), PIP_WHL_GLIBC_VERSION = str(Label("//python/config_settings:pip_whl_glibc_version")), + PIP_WHL_MUSLC_VERSION = str(Label("//python/config_settings:pip_whl_muslc_version")), PIP_WHL_OSX_ARCH = str(Label("//python/config_settings:pip_whl_osx_arch")), PIP_WHL_OSX_VERSION = str(Label("//python/config_settings:pip_whl_osx_version")), PRECOMPILE = str(Label("//python/config_settings:precompile")), PRECOMPILE_SOURCE_RETENTION = str(Label("//python/config_settings:precompile_source_retention")), + PYC_COLLECTION = str(Label("//python/config_settings:pyc_collection")), PYTHON_SRC = str(Label("//python/bin:python_src")), PYTHON_VERSION = str(Label("//python/config_settings:python_version")), PYTHON_VERSION_MAJOR_MINOR = str(Label("//python/config_settings:python_version_major_minor")), @@ -24,4 +26,5 @@ labels = struct( REPL_DEP = str(Label("//python/bin:repl_dep")), VENVS_SITE_PACKAGES = str(Label("//python/config_settings:venvs_site_packages")), VENVS_USE_DECLARE_SYMLINK = str(Label("//python/config_settings:venvs_use_declare_symlink")), + VISIBLE_FOR_TESTING = str(Label("//python/private:visible_for_testing")), ) diff --git a/python/private/py_cc_toolchain_rule.bzl b/python/private/py_cc_toolchain_rule.bzl index f12933e245..8adf73c25f 100644 --- a/python/private/py_cc_toolchain_rule.bzl +++ b/python/private/py_cc_toolchain_rule.bzl @@ -20,6 +20,7 @@ https://github.com/bazel-contrib/rules_python/issues/824 is considered done. load("@bazel_skylib//rules:common_settings.bzl", "BuildSettingInfo") load("@rules_cc//cc/common:cc_info.bzl", "CcInfo") +load(":common_labels.bzl", "labels") load(":py_cc_toolchain_info.bzl", "PyCcToolchainInfo") def _py_cc_toolchain_impl(ctx): @@ -70,7 +71,7 @@ py_cc_toolchain = rule( mandatory = True, ), "_visible_for_testing": attr.label( - default = "//python/private:visible_for_testing", + default = labels.VISIBLE_FOR_TESTING, ), }, doc = """\ diff --git a/python/private/py_exec_tools_toolchain.bzl b/python/private/py_exec_tools_toolchain.bzl index 332570b26b..00ad8072f6 100644 --- a/python/private/py_exec_tools_toolchain.bzl +++ b/python/private/py_exec_tools_toolchain.bzl @@ -16,6 +16,7 @@ load("@bazel_skylib//lib:paths.bzl", "paths") load("@bazel_skylib//rules:common_settings.bzl", "BuildSettingInfo") +load(":common_labels.bzl", "labels") load(":py_exec_tools_info.bzl", "PyExecToolsInfo") load(":sentinel.bzl", "SentinelInfo") load(":toolchain_types.bzl", "TARGET_TOOLCHAIN_TYPE") @@ -89,7 +90,7 @@ so that the toolchain `py_runtime` field can be correctly forwarded. doc = "See {obj}`PyExecToolsInfo.precompiler`", ), "_visible_for_testing": attr.label( - default = "//python/private:visible_for_testing", + default = labels.VISIBLE_FOR_TESTING, ), }, ) diff --git a/python/private/py_executable.bzl b/python/private/py_executable.bzl index fa80ea5105..59800da566 100644 --- a/python/private/py_executable.bzl +++ b/python/private/py_executable.bzl @@ -51,6 +51,7 @@ load( "runfiles_root_path", "target_platform_has_any_constraint", ) +load(":common_labels.bzl", "labels") load(":flags.bzl", "BootstrapImplFlag", "VenvsUseDeclareSymlinkFlag") load(":precompile.bzl", "maybe_precompile") load(":py_cc_link_params_info.bzl", "PyCcLinkParamsInfo") @@ -60,18 +61,12 @@ load(":py_internal.bzl", "py_internal") load(":py_runtime_info.bzl", "DEFAULT_STUB_SHEBANG", "PyRuntimeInfo") load(":reexports.bzl", "BuiltinPyInfo", "BuiltinPyRuntimeInfo") load(":rule_builders.bzl", "ruleb") -load( - ":toolchain_types.bzl", - "EXEC_TOOLS_TOOLCHAIN_TYPE", - "TARGET_TOOLCHAIN_TYPE", - TOOLCHAIN_TYPE = "TARGET_TOOLCHAIN_TYPE", -) +load(":toolchain_types.bzl", "EXEC_TOOLS_TOOLCHAIN_TYPE", "TARGET_TOOLCHAIN_TYPE", TOOLCHAIN_TYPE = "TARGET_TOOLCHAIN_TYPE") load(":transition_labels.bzl", "TRANSITION_LABELS") _py_builtins = py_internal _EXTERNAL_PATH_PREFIX = "external" _ZIP_RUNFILES_DIRECTORY_NAME = "runfiles" -_PYTHON_VERSION_FLAG = str(Label("//python/config_settings:python_version")) # Non-Google-specific attributes for executables # These attributes are for rules that accept Python sources. @@ -192,7 +187,7 @@ accepting arbitrary Python versions. default = "@bazel_tools//tools/allowlists/function_transition_allowlist", ), "_bootstrap_impl_flag": lambda: attrb.Label( - default = "//python/config_settings:bootstrap_impl", + default = labels.BOOTSTRAP_IMPL, providers = [BuildSettingInfo], ), "_bootstrap_template": lambda: attrb.Label( @@ -222,10 +217,10 @@ accepting arbitrary Python versions. default = TARGET_TOOLCHAIN_TYPE, ), "_python_version_flag": lambda: attrb.Label( - default = "//python/config_settings:python_version", + default = labels.PYTHON_VERSION, ), "_venvs_use_declare_symlink_flag": lambda: attrb.Label( - default = "//python/config_settings:venvs_use_declare_symlink", + default = labels.VENVS_USE_DECLARE_SYMLINK, providers = [BuildSettingInfo], ), "_windows_constraints": lambda: attrb.LabelList( @@ -1910,7 +1905,7 @@ def _transition_executable_impl(settings, attr): apply_config_settings_attr(settings, attr) if attr.python_version and attr.python_version not in ("PY2", "PY3"): - settings[_PYTHON_VERSION_FLAG] = attr.python_version + settings[labels.PYTHON_VERSION] = attr.python_version return settings def create_executable_rule(*, attrs, **kwargs): @@ -1961,8 +1956,8 @@ def create_executable_rule_builder(implementation, **kwargs): ], cfg = dict( implementation = _transition_executable_impl, - inputs = TRANSITION_LABELS + [_PYTHON_VERSION_FLAG], - outputs = TRANSITION_LABELS + [_PYTHON_VERSION_FLAG], + inputs = TRANSITION_LABELS + [labels.PYTHON_VERSION], + outputs = TRANSITION_LABELS + [labels.PYTHON_VERSION], ), **kwargs ) diff --git a/python/private/py_library.bzl b/python/private/py_library.bzl index 1f3e4d88d4..fc8e5839a0 100644 --- a/python/private/py_library.bzl +++ b/python/private/py_library.bzl @@ -40,6 +40,7 @@ load( "get_imports", "runfiles_root_path", ) +load(":common_labels.bzl", "labels") load(":flags.bzl", "AddSrcsToRunfilesFlag", "PrecompileFlag", "VenvsSitePackages") load(":normalize_name.bzl", "normalize_name") load(":precompile.bzl", "maybe_precompile") @@ -102,7 +103,7 @@ and that only one package version will be included. """, ), "_add_srcs_to_runfiles_flag": lambda: attrb.Label( - default = "//python/config_settings:add_srcs_to_runfiles", + default = labels.ADD_SRCS_TO_RUNFILES, ), }, ) diff --git a/python/private/py_runtime_pair_rule.bzl b/python/private/py_runtime_pair_rule.bzl index b3b7a4e5f8..775d53a0b8 100644 --- a/python/private/py_runtime_pair_rule.bzl +++ b/python/private/py_runtime_pair_rule.bzl @@ -16,6 +16,7 @@ load("@bazel_skylib//rules:common_settings.bzl", "BuildSettingInfo") load("//python:py_runtime_info.bzl", "PyRuntimeInfo") +load(":common_labels.bzl", "labels") load(":reexports.bzl", "BuiltinPyRuntimeInfo") load(":util.bzl", "IS_BAZEL_7_OR_HIGHER") @@ -94,7 +95,7 @@ The runtime to use for Python 3 targets. Must have `python_version` set to """, ), "_visible_for_testing": attr.label( - default = "//python/private:visible_for_testing", + default = labels.VISIBLE_FOR_TESTING, ), }, fragments = ["py"], diff --git a/python/private/py_runtime_rule.bzl b/python/private/py_runtime_rule.bzl index 861014e117..a511a0e1a5 100644 --- a/python/private/py_runtime_rule.bzl +++ b/python/private/py_runtime_rule.bzl @@ -17,6 +17,7 @@ load("@bazel_skylib//lib:dicts.bzl", "dicts") load("@bazel_skylib//lib:paths.bzl", "paths") load("@bazel_skylib//rules:common_settings.bzl", "BuildSettingInfo") load(":attributes.bzl", "NATIVE_RULES_ALLOWLIST_ATTRS") +load(":common_labels.bzl", "labels") load(":flags.bzl", "FreeThreadedFlag") load(":py_internal.bzl", "py_internal") load(":py_runtime_info.bzl", "DEFAULT_STUB_SHEBANG", "PyRuntimeInfo") @@ -379,10 +380,10 @@ The {obj}`PyRuntimeInfo.zip_main_template` field. """, ), "_py_freethreaded_flag": attr.label( - default = "//python/config_settings:py_freethreaded", + default = labels.PY_FREETHREADED, ), "_python_version_flag": attr.label( - default = "//python/config_settings:python_version", + default = labels.PYTHON_VERSION, ), }, ), diff --git a/python/private/pypi/BUILD.bazel b/python/private/pypi/BUILD.bazel index c7a74ee306..0d2f73fb0b 100644 --- a/python/private/pypi/BUILD.bazel +++ b/python/private/pypi/BUILD.bazel @@ -63,6 +63,7 @@ bzl_library( srcs = ["config_settings.bzl"], deps = [ ":flags_bzl", + "//python/private:common_labels_bzl", "//python/private:flags_bzl", "@bazel_skylib//lib:selects", ], @@ -89,6 +90,7 @@ bzl_library( ":env_marker_info_bzl", ":pep508_env_bzl", ":pep508_evaluate_bzl", + "//python/private:common_labels_bzl", "//python/private:toolchain_types_bzl", "@bazel_skylib//rules:common_settings", ], @@ -337,6 +339,7 @@ bzl_library( srcs = ["pkg_aliases.bzl"], deps = [ ":labels_bzl", + "//python/private:common_labels_bzl", "//python/private:text_util_bzl", "@bazel_skylib//lib:selects", ], diff --git a/python/private/pypi/env_marker_setting.bzl b/python/private/pypi/env_marker_setting.bzl index 2bfdf42ef0..71c6b410ed 100644 --- a/python/private/pypi/env_marker_setting.bzl +++ b/python/private/pypi/env_marker_setting.bzl @@ -1,6 +1,7 @@ """Implement a flag for matching the dependency specifiers at analysis time.""" load("@bazel_skylib//rules:common_settings.bzl", "BuildSettingInfo") +load("//python/private:common_labels.bzl", "labels") load("//python/private:toolchain_types.bzl", "TARGET_TOOLCHAIN_TYPE") load(":env_marker_info.bzl", "EnvMarkerInfo") load(":pep508_env.bzl", "create_env", "set_missing_env_defaults") @@ -85,15 +86,15 @@ for the specification of behavior. doc = "Environment marker expression to evaluate.", ), "_env_marker_config_flag": attr.label( - default = "//python/config_settings:pip_env_marker_config", + default = labels.PIP_ENV_MARKER_CONFIG, providers = [EnvMarkerInfo], ), "_python_full_version_flag": attr.label( - default = "//python/config_settings:python_version", + default = labels.PYTHON_VERSION, providers = [config_common.FeatureFlagInfo], ), "_python_version_major_minor_flag": attr.label( - default = "//python/config_settings:python_version_major_minor", + default = labels.PYTHON_VERSION_MAJOR_MINOR, providers = [config_common.FeatureFlagInfo], ), }, diff --git a/python/private/pypi/flags.bzl b/python/private/pypi/flags.bzl index 037383910e..f88690d843 100644 --- a/python/private/pypi/flags.bzl +++ b/python/private/pypi/flags.bzl @@ -19,6 +19,7 @@ unnecessary files when all that are needed are flag definitions. """ load("@bazel_skylib//rules:common_settings.bzl", "BuildSettingInfo", "string_flag") +load("//python/private:common_labels.bzl", "labels") load("//python/private:enum.bzl", "enum") load(":env_marker_info.bzl", "EnvMarkerInfo") load( @@ -103,9 +104,9 @@ def _allow_wheels_flag_impl(ctx): _allow_wheels_flag = rule( implementation = _allow_wheels_flag_impl, attrs = { - "_setting": attr.label(default = "//python/config_settings:pip_whl"), + "_setting": attr.label(default = labels.PIP_WHL), }, - doc = """\ + doc = """ This rule allows us to greatly reduce the number of config setting targets at no cost even if we are duplicating some of the functionality of the `native.config_setting`. """, @@ -153,7 +154,7 @@ _env_marker_config = rule( "platform_system": attr.string(), "sys_platform": attr.string(), "_pip_whl_osx_version_flag": attr.label( - default = "//python/config_settings:pip_whl_osx_version", + default = labels.PIP_WHL_OSX_VERSION, providers = [[BuildSettingInfo], [config_common.FeatureFlagInfo]], ), }, diff --git a/python/private/pypi/pkg_aliases.bzl b/python/private/pypi/pkg_aliases.bzl index 67ce297466..ac063fac48 100644 --- a/python/private/pypi/pkg_aliases.bzl +++ b/python/private/pypi/pkg_aliases.bzl @@ -32,6 +32,7 @@ setting maps to and their precedence, refer to documentation on that page. """ load("@bazel_skylib//lib:selects.bzl", "selects") +load("//python/private:common_labels.bzl", "labels") load("//python/private:text_util.bzl", "render") load( ":labels.bzl", @@ -63,7 +64,7 @@ build to make it a failure instead by running the build with: However, the command above will hide the `bazel config ` message. """ -_LABEL_NONE = Label("//python:none") +_LABEL_NONE = labels.NONE _LABEL_CURRENT_CONFIG = Label("//python/config_settings:current_config") _LABEL_CURRENT_CONFIG_NO_MATCH = Label("//python/config_settings:is_not_matching_current_config") _INCOMPATIBLE = "_no_matching_repository" diff --git a/python/uv/private/BUILD.bazel b/python/uv/private/BUILD.bazel index a07d8591ad..3e4d6c7baa 100644 --- a/python/uv/private/BUILD.bazel +++ b/python/uv/private/BUILD.bazel @@ -43,6 +43,7 @@ bzl_library( ":toolchain_types_bzl", "//python:py_binary_bzl", "//python/private:bzlmod_enabled_bzl", + "//python/private:common_labels_bzl", "//python/private:toolchain_types_bzl", "@bazel_skylib//lib:shell", ], @@ -63,6 +64,7 @@ bzl_library( ":uv_repository_bzl", ":uv_toolchains_repo_bzl", "//python/private:auth_bzl", + "//python/private:common_labels_bzl", ], ) diff --git a/python/uv/private/lock.bzl b/python/uv/private/lock.bzl index 2731d6b009..281a0decc0 100644 --- a/python/uv/private/lock.bzl +++ b/python/uv/private/lock.bzl @@ -18,13 +18,12 @@ load("@bazel_skylib//lib:shell.bzl", "shell") load("//python:py_binary.bzl", "py_binary") load("//python/private:bzlmod_enabled.bzl", "BZLMOD_ENABLED") # buildifier: disable=bzl-visibility +load("//python/private:common_labels.bzl", "labels") load("//python/private:toolchain_types.bzl", "EXEC_TOOLS_TOOLCHAIN_TYPE") # buildifier: disable=bzl-visibility load(":toolchain_types.bzl", "UV_TOOLCHAIN_TYPE") visibility(["//..."]) -_PYTHON_VERSION_FLAG = "//python/config_settings:python_version" - _RunLockInfo = provider( doc = "", fields = { @@ -161,16 +160,16 @@ def _lock_impl(ctx): def _transition_impl(input_settings, attr): settings = { - _PYTHON_VERSION_FLAG: input_settings[_PYTHON_VERSION_FLAG], + labels.PYTHON_VERSION: input_settings[labels.PYTHON_VERSION], } if attr.python_version: - settings[_PYTHON_VERSION_FLAG] = attr.python_version + settings[labels.PYTHON_VERSION] = attr.python_version return settings _python_version_transition = transition( implementation = _transition_impl, - inputs = [_PYTHON_VERSION_FLAG], - outputs = [_PYTHON_VERSION_FLAG], + inputs = [labels.PYTHON_VERSION], + outputs = [labels.PYTHON_VERSION], ) _lock = rule( diff --git a/python/uv/private/uv.bzl b/python/uv/private/uv.bzl index 2cc2df1b21..fe0911e3ea 100644 --- a/python/uv/private/uv.bzl +++ b/python/uv/private/uv.bzl @@ -19,6 +19,7 @@ A module extension for working with uv. """ load("//python/private:auth.bzl", "AUTH_ATTRS", "get_auth") +load("//python/private:common_labels.bzl", "labels") load(":toolchain_types.bzl", "UV_TOOLCHAIN_TYPE") load(":uv_repository.bzl", "uv_repository") load(":uv_toolchains_repo.bzl", "uv_toolchains_repo") @@ -288,7 +289,7 @@ def process_modules( toolchain_names = ["none"], toolchain_implementations = { # NOTE @aignas 2025-02-24: the label to the toolchain can be anything - "none": str(Label("//python:none")), + "none": labels.NONE, }, toolchain_compatible_with = { "none": ["@platforms//:incompatible"], diff --git a/tests/base_rules/precompile/precompile_tests.bzl b/tests/base_rules/precompile/precompile_tests.bzl index 895f2d3156..fe5c165648 100644 --- a/tests/base_rules/precompile/precompile_tests.bzl +++ b/tests/base_rules/precompile/precompile_tests.bzl @@ -23,13 +23,11 @@ load("//python:py_binary.bzl", "py_binary") load("//python:py_info.bzl", "PyInfo") load("//python:py_library.bzl", "py_library") load("//python:py_test.bzl", "py_test") +load("//python/private:common_labels.bzl", "labels") # buildifier: disable=bzl-visibility load("//tests/support:py_info_subject.bzl", "py_info_subject") load( "//tests/support:support.bzl", - "ADD_SRCS_TO_RUNFILES", "CC_TOOLCHAIN", - "EXEC_TOOLS_TOOLCHAIN", - "PRECOMPILE", "PY_TOOLCHAINS", ) @@ -38,7 +36,7 @@ _COMMON_CONFIG_SETTINGS = { # it for conformity. "//command_line_option:allow_unresolved_symlinks": True, "//command_line_option:extra_toolchains": [PY_TOOLCHAINS, CC_TOOLCHAIN], - EXEC_TOOLS_TOOLCHAIN: "enabled", + labels.EXEC_TOOLS_TOOLCHAIN: "enabled", } _tests = [] @@ -150,7 +148,7 @@ def _test_precompile_enabled_py_library_add_to_runfiles_disabled(name): name = name, impl = _test_precompile_enabled_py_library_add_to_runfiles_disabled_impl, config_settings = { - ADD_SRCS_TO_RUNFILES: "disabled", + labels.ADD_SRCS_TO_RUNFILES: "disabled", }, ) @@ -166,7 +164,7 @@ def _test_precompile_enabled_py_library_add_to_runfiles_enabled(name): name = name, impl = _test_precompile_enabled_py_library_add_to_runfiles_enabled_impl, config_settings = { - ADD_SRCS_TO_RUNFILES: "enabled", + labels.ADD_SRCS_TO_RUNFILES: "enabled", }, ) @@ -203,7 +201,7 @@ def _test_pyc_only(name): name = name, impl = _test_pyc_only_impl, config_settings = _COMMON_CONFIG_SETTINGS | { - PRECOMPILE: "enabled", + labels.PRECOMPILE: "enabled", }, target = name + "_subject", ) @@ -310,7 +308,7 @@ def _setup_precompile_flag_pyc_collection_attr_interaction( impl = test_impl, target = name + "_bin", config_settings = _COMMON_CONFIG_SETTINGS | { - PRECOMPILE: precompile_flag, + labels.PRECOMPILE: precompile_flag, }, ) @@ -531,7 +529,7 @@ def _test_precompile_attr_inherit_pyc_collection_disabled_precompile_flag_enable impl = _test_precompile_attr_inherit_pyc_collection_disabled_precompile_flag_enabled_impl, target = name + "_subject", config_settings = _COMMON_CONFIG_SETTINGS | { - PRECOMPILE: "enabled", + labels.PRECOMPILE: "enabled", }, ) diff --git a/tests/base_rules/py_executable_base_tests.bzl b/tests/base_rules/py_executable_base_tests.bzl index 2b96451e35..4e451289dc 100644 --- a/tests/base_rules/py_executable_base_tests.bzl +++ b/tests/base_rules/py_executable_base_tests.bzl @@ -19,12 +19,13 @@ load("@rules_testing//lib:analysis_test.bzl", "analysis_test") load("@rules_testing//lib:truth.bzl", "matching") load("@rules_testing//lib:util.bzl", rt_util = "util") load("//python:py_executable_info.bzl", "PyExecutableInfo") +load("//python/private:common_labels.bzl", "labels") # buildifier: disable=bzl-visibility load("//python/private:reexports.bzl", "BuiltinPyRuntimeInfo") # buildifier: disable=bzl-visibility load("//python/private:util.bzl", "IS_BAZEL_7_OR_HIGHER") # buildifier: disable=bzl-visibility load("//tests/base_rules:base_tests.bzl", "create_base_tests") load("//tests/base_rules:util.bzl", "WINDOWS_ATTR", pt_util = "util") load("//tests/support:py_executable_info_subject.bzl", "PyExecutableInfoSubject") -load("//tests/support:support.bzl", "BOOTSTRAP_IMPL", "CC_TOOLCHAIN", "CROSSTOOL_TOP", "LINUX_X86_64", "WINDOWS_X86_64") +load("//tests/support:support.bzl", "CC_TOOLCHAIN", "CROSSTOOL_TOP", "LINUX_X86_64", "WINDOWS_X86_64") _tests = [] @@ -355,7 +356,7 @@ def _test_main_module_bootstrap_system_python(name, config): impl = _test_main_module_bootstrap_system_python_impl, target = name + "_subject", config_settings = { - BOOTSTRAP_IMPL: "system_python", + labels.BOOTSTRAP_IMPL: "system_python", "//command_line_option:extra_execution_platforms": ["@bazel_tools//tools:host_platform", LINUX_X86_64], "//command_line_option:platforms": [LINUX_X86_64], }, @@ -379,7 +380,7 @@ def _test_main_module_bootstrap_script(name, config): impl = _test_main_module_bootstrap_script_impl, target = name + "_subject", config_settings = { - BOOTSTRAP_IMPL: "script", + labels.BOOTSTRAP_IMPL: "script", "//command_line_option:extra_execution_platforms": ["@bazel_tools//tools:host_platform", LINUX_X86_64], "//command_line_option:platforms": [LINUX_X86_64], }, diff --git a/tests/base_rules/py_test/py_test_tests.bzl b/tests/base_rules/py_test/py_test_tests.bzl index c51aa53a95..1ec1dc428f 100644 --- a/tests/base_rules/py_test/py_test_tests.bzl +++ b/tests/base_rules/py_test/py_test_tests.bzl @@ -60,7 +60,7 @@ def _test_mac_requires_darwin_for_execution(name, config): "//command_line_option:cpu": "darwin_x86_64", "//command_line_option:crosstool_top": CROSSTOOL_TOP, "//command_line_option:extra_execution_platforms": [MAC_X86_64], - "//command_line_option:extra_toolchains": CC_TOOLCHAIN, + "//command_line_option:extra_toolchains": [CC_TOOLCHAIN], "//command_line_option:platforms": [MAC_X86_64], }, attr_values = _SKIP_WINDOWS, @@ -94,7 +94,7 @@ def _test_non_mac_doesnt_require_darwin_for_execution(name, config): "//command_line_option:cpu": "k8", "//command_line_option:crosstool_top": CROSSTOOL_TOP, "//command_line_option:extra_execution_platforms": [LINUX_X86_64], - "//command_line_option:extra_toolchains": CC_TOOLCHAIN, + "//command_line_option:extra_toolchains": [CC_TOOLCHAIN], "//command_line_option:platforms": [LINUX_X86_64], }, attr_values = _SKIP_WINDOWS, diff --git a/tests/builders/attr_builders_tests.bzl b/tests/builders/attr_builders_tests.bzl index e92ba2ae0a..3a771afde5 100644 --- a/tests/builders/attr_builders_tests.bzl +++ b/tests/builders/attr_builders_tests.bzl @@ -18,6 +18,7 @@ load("@rules_testing//lib:analysis_test.bzl", "analysis_test") load("@rules_testing//lib:test_suite.bzl", "test_suite") load("@rules_testing//lib:truth.bzl", "truth") load("//python/private:attr_builders.bzl", "attrb") # buildifier: disable=bzl-visibility +load("//python/private:common_labels.bzl", "labels") # buildifier: disable=bzl-visibility def _expect_cfg_defaults(expect, cfg): expect.where(expr = "cfg.outputs").that_collection(cfg.outputs()).contains_exactly([]) @@ -41,7 +42,7 @@ def _report_failures(name, env): analysis_test( name = name, - target = "//python:none", + target = labels.NONE, impl = _report_failures_impl, ) diff --git a/tests/builders/rule_builders_tests.bzl b/tests/builders/rule_builders_tests.bzl index 3f14832d80..a8ac31f4bf 100644 --- a/tests/builders/rule_builders_tests.bzl +++ b/tests/builders/rule_builders_tests.bzl @@ -18,6 +18,7 @@ load("@rules_testing//lib:analysis_test.bzl", "analysis_test") load("@rules_testing//lib:test_suite.bzl", "test_suite") load("@rules_testing//lib:util.bzl", "TestingAspectInfo") load("//python/private:attr_builders.bzl", "attrb") # buildifier: disable=bzl-visibility +load("//python/private:common_labels.bzl", "labels") # buildifier: disable=bzl-visibility load("//python/private:rule_builders.bzl", "ruleb") # buildifier: disable=bzl-visibility RuleInfo = provider(doc = "test provider", fields = []) @@ -49,7 +50,7 @@ def _test_fruit_rule(name): flavors = ["spicy", "sweet"], organic = True, size = 5, - origin = "//python:none", + origin = labels.NONE, fertilizers = [ "nitrogen.txt", "phosphorus.txt", @@ -169,7 +170,7 @@ def _test_exec_group(env): env.expect.that_collection(subject.exec_compatible_with()).contains_exactly([]) env.expect.that_str(str(subject.build())).contains("ExecGroup") - subject.toolchains().append(ruleb.ToolchainType("//python:none")) + subject.toolchains().append(ruleb.ToolchainType(labels.NONE)) subject.exec_compatible_with().append("//some:constraint") env.expect.that_str(str(subject.build())).contains("ExecGroup") diff --git a/tests/exec_toolchain_matching/exec_toolchain_matching_tests.bzl b/tests/exec_toolchain_matching/exec_toolchain_matching_tests.bzl index f6eae5ad5f..43a9717314 100644 --- a/tests/exec_toolchain_matching/exec_toolchain_matching_tests.bzl +++ b/tests/exec_toolchain_matching/exec_toolchain_matching_tests.bzl @@ -18,9 +18,10 @@ load("@rules_testing//lib:test_suite.bzl", "test_suite") load("@rules_testing//lib:util.bzl", rt_util = "util") load("//python:py_runtime.bzl", "py_runtime") load("//python:py_runtime_pair.bzl", "py_runtime_pair") +load("//python/private:common_labels.bzl", "labels") # buildifier: disable=bzl-visibility load("//python/private:toolchain_types.bzl", "EXEC_TOOLS_TOOLCHAIN_TYPE", "TARGET_TOOLCHAIN_TYPE") # buildifier: disable=bzl-visibility load("//python/private:util.bzl", "IS_BAZEL_7_OR_HIGHER") # buildifier: disable=bzl-visibility -load("//tests/support:support.bzl", "LINUX", "MAC", "PYTHON_VERSION") +load("//tests/support:support.bzl", "LINUX", "MAC") _LookupInfo = provider() # buildifier: disable=provider-params @@ -129,7 +130,7 @@ def _test_exec_matches_target_python_version(name): "//command_line_option:extra_execution_platforms": [str(MAC)], "//command_line_option:extra_toolchains": ["//tests/exec_toolchain_matching:all"], "//command_line_option:platforms": [str(LINUX)], - PYTHON_VERSION: "3.12", + labels.PYTHON_VERSION: "3.12", }, ) diff --git a/tests/py_runtime/py_runtime_tests.bzl b/tests/py_runtime/py_runtime_tests.bzl index d5a6076153..4ec7590ab2 100644 --- a/tests/py_runtime/py_runtime_tests.bzl +++ b/tests/py_runtime/py_runtime_tests.bzl @@ -20,9 +20,9 @@ load("@rules_testing//lib:truth.bzl", "matching") load("@rules_testing//lib:util.bzl", rt_util = "util") load("//python:py_runtime.bzl", "py_runtime") load("//python:py_runtime_info.bzl", "PyRuntimeInfo") +load("//python/private:common_labels.bzl", "labels") # buildifier: disable=bzl-visibility load("//tests/base_rules:util.bzl", br_util = "util") load("//tests/support:py_runtime_info_subject.bzl", "py_runtime_info_subject") -load("//tests/support:support.bzl", "PYTHON_VERSION") _tests = [] @@ -543,7 +543,7 @@ def _test_version_info_from_flag(name): target = name + "_subject", impl = _test_version_info_from_flag_impl, config_settings = { - PYTHON_VERSION: "3.12", + labels.PYTHON_VERSION: "3.12", }, ) diff --git a/tests/pypi/env_marker_setting/env_marker_setting_tests.bzl b/tests/pypi/env_marker_setting/env_marker_setting_tests.bzl index e16f2c8ef6..c5b3f72d8c 100644 --- a/tests/pypi/env_marker_setting/env_marker_setting_tests.bzl +++ b/tests/pypi/env_marker_setting/env_marker_setting_tests.bzl @@ -3,9 +3,9 @@ load("@rules_testing//lib:analysis_test.bzl", "analysis_test") load("@rules_testing//lib:test_suite.bzl", "test_suite") load("@rules_testing//lib:util.bzl", "TestingAspectInfo") +load("//python/private:common_labels.bzl", "labels") # buildifier: disable=bzl-visibility load("//python/private/pypi:env_marker_info.bzl", "EnvMarkerInfo") # buildifier: disable=bzl-visibility load("//python/private/pypi:env_marker_setting.bzl", "env_marker_setting") # buildifier: disable=bzl-visibility -load("//tests/support:support.bzl", "PIP_ENV_MARKER_CONFIG", "PYTHON_VERSION") def _custom_env_markers_impl(ctx): _ = ctx # @unused @@ -37,7 +37,7 @@ def _test_custom_env_markers(name): impl = _impl, target = name + "_subject", config_settings = { - PIP_ENV_MARKER_CONFIG: str(Label(name + "_env")), + labels.PIP_ENV_MARKER_CONFIG: str(Label(name + "_env")), }, ) @@ -56,14 +56,14 @@ def _test_expr(name): cases = { "python_full_version_lt_negative": { "config_settings": { - PYTHON_VERSION: "3.12.0", + labels.PYTHON_VERSION: "3.12.0", }, "expected": "FALSE", "expression": "python_full_version < '3.8'", }, "python_version_gte": { "config_settings": { - PYTHON_VERSION: "3.12.0", + labels.PYTHON_VERSION: "3.12.0", }, "expected": "TRUE", "expression": "python_version >= '3.12.0'", diff --git a/tests/pypi/pkg_aliases/pkg_aliases_test.bzl b/tests/pypi/pkg_aliases/pkg_aliases_test.bzl index 6248261ed4..32be6ba47c 100644 --- a/tests/pypi/pkg_aliases/pkg_aliases_test.bzl +++ b/tests/pypi/pkg_aliases/pkg_aliases_test.bzl @@ -15,6 +15,7 @@ """pkg_aliases tests""" load("@rules_testing//lib:test_suite.bzl", "test_suite") +load("//python/private:common_labels.bzl", "labels") # buildifier: disable=bzl-visibility load("//python/private/pypi:config_settings.bzl", "config_settings") # buildifier: disable=bzl-visibility load( "//python/private/pypi:pkg_aliases.bzl", @@ -81,7 +82,7 @@ def _test_config_setting_aliases(env): }, # This will be printing the current config values and will make sure we # have an error. - "_no_matching_repository": {Label("//python/config_settings:is_not_matching_current_config"): Label("//python:none")}, + "_no_matching_repository": {Label("//python/config_settings:is_not_matching_current_config"): labels.NONE}, } env.expect.that_dict(got).contains_at_least(want) env.expect.that_collection(actual_no_match_error).has_size(1) diff --git a/tests/runtime_env_toolchain/runtime_env_toolchain_tests.bzl b/tests/runtime_env_toolchain/runtime_env_toolchain_tests.bzl index 9885a1ef9b..aa4d1c793b 100644 --- a/tests/runtime_env_toolchain/runtime_env_toolchain_tests.bzl +++ b/tests/runtime_env_toolchain/runtime_env_toolchain_tests.bzl @@ -17,6 +17,7 @@ load("@rules_testing//lib:analysis_test.bzl", "analysis_test") load("@rules_testing//lib:test_suite.bzl", "test_suite") load("@rules_testing//lib:util.bzl", rt_util = "util") +load("//python/private:common_labels.bzl", "labels") # buildifier: disable=bzl-visibility load( "//python/private:toolchain_types.bzl", "EXEC_TOOLS_TOOLCHAIN_TYPE", @@ -24,7 +25,7 @@ load( "TARGET_TOOLCHAIN_TYPE", ) # buildifier: disable=bzl-visibility load("//python/private:util.bzl", "IS_BAZEL_7_OR_HIGHER") # buildifier: disable=bzl-visibility -load("//tests/support:support.bzl", "CC_TOOLCHAIN", "EXEC_TOOLS_TOOLCHAIN", "VISIBLE_FOR_TESTING") +load("//tests/support:support.bzl", "CC_TOOLCHAIN") _LookupInfo = provider() # buildifier: disable=provider-params @@ -79,8 +80,8 @@ def _test_runtime_env_toolchain_matches(name): target = name + "_subject", config_settings = { "//command_line_option:extra_toolchains": extra_toolchains, - EXEC_TOOLS_TOOLCHAIN: "enabled", - VISIBLE_FOR_TESTING: True, + labels.EXEC_TOOLS_TOOLCHAIN: "enabled", + labels.VISIBLE_FOR_TESTING: True, }, ) diff --git a/tests/support/py_reconfig.bzl b/tests/support/py_reconfig.bzl index 38d53667fd..d52cc5dd95 100644 --- a/tests/support/py_reconfig.bzl +++ b/tests/support/py_reconfig.bzl @@ -23,13 +23,13 @@ load("//python/private:py_binary_macro.bzl", "py_binary_macro") # buildifier: d load("//python/private:py_binary_rule.bzl", "create_py_binary_rule_builder") # buildifier: disable=bzl-visibility load("//python/private:py_test_macro.bzl", "py_test_macro") # buildifier: disable=bzl-visibility load("//python/private:py_test_rule.bzl", "create_py_test_rule_builder") # buildifier: disable=bzl-visibility -load("//tests/support:support.bzl", "CUSTOM_RUNTIME", "VISIBLE_FOR_TESTING") +load("//tests/support:support.bzl", "CUSTOM_RUNTIME") def _perform_transition_impl(input_settings, attr, base_impl): settings = {k: input_settings[k] for k in _RECONFIG_INHERITED_OUTPUTS if k in input_settings} settings.update(base_impl(input_settings, attr)) - settings[VISIBLE_FOR_TESTING] = True + settings[labels.VISIBLE_FOR_TESTING] = True settings["//command_line_option:build_python_zip"] = attr.build_python_zip if attr.bootstrap_impl: settings[labels.BOOTSTRAP_IMPL] = attr.bootstrap_impl @@ -58,7 +58,7 @@ _RECONFIG_INPUTS = [ ] _RECONFIG_OUTPUTS = _RECONFIG_INPUTS + [ "//command_line_option:build_python_zip", - VISIBLE_FOR_TESTING, + labels.VISIBLE_FOR_TESTING, ] _RECONFIG_INHERITED_OUTPUTS = [v for v in _RECONFIG_OUTPUTS if v in _RECONFIG_INPUTS] diff --git a/tests/support/sh_py_run_test.bzl b/tests/support/sh_py_run_test.bzl index 83ac2c814b..7dff6673ce 100644 --- a/tests/support/sh_py_run_test.bzl +++ b/tests/support/sh_py_run_test.bzl @@ -18,6 +18,7 @@ without the overhead of a bazel-in-bazel integration test. """ load("@rules_shell//shell:sh_test.bzl", "sh_test") +load("//python/private:common_labels.bzl", "labels") # buildifier: disable=bzl-visibility load("//python/private:toolchain_types.bzl", "TARGET_TOOLCHAIN_TYPE") # buildifier: disable=bzl-visibility load(":py_reconfig.bzl", "py_reconfig_binary") @@ -79,7 +80,7 @@ This is so tests can verify information about the build config used for them. implementation = _current_build_settings_impl, attrs = { "_bootstrap_impl_flag": attr.label( - default = "//python/config_settings:bootstrap_impl", + default = labels.BOOTSTRAP_IMPL, ), }, toolchains = [ diff --git a/tests/support/support.bzl b/tests/support/support.bzl index 28cab0dcbf..37d3488316 100644 --- a/tests/support/support.bzl +++ b/tests/support/support.bzl @@ -35,15 +35,6 @@ CROSSTOOL_TOP = Label("//tests/support/cc_toolchains:cc_toolchain_suite") # str() around Label() is necessary because rules_testing's config_settings # doesn't accept yet Label objects. -ADD_SRCS_TO_RUNFILES = str(Label("//python/config_settings:add_srcs_to_runfiles")) -BOOTSTRAP_IMPL = str(Label("//python/config_settings:bootstrap_impl")) -EXEC_TOOLS_TOOLCHAIN = str(Label("//python/config_settings:exec_tools_toolchain")) -PIP_ENV_MARKER_CONFIG = str(Label("//python/config_settings:pip_env_marker_config")) -PRECOMPILE = str(Label("//python/config_settings:precompile")) -PRECOMPILE_SOURCE_RETENTION = str(Label("//python/config_settings:precompile_source_retention")) -PYC_COLLECTION = str(Label("//python/config_settings:pyc_collection")) -PYTHON_VERSION = str(Label("//python/config_settings:python_version")) -VISIBLE_FOR_TESTING = str(Label("//python/private:visible_for_testing")) CUSTOM_RUNTIME = str(Label("//tests/support:custom_runtime")) SUPPORTS_BOOTSTRAP_SCRIPT = select({ diff --git a/tests/toolchains/transitions/transitions_tests.bzl b/tests/toolchains/transitions/transitions_tests.bzl index 0f1db2eecd..0cd79b373b 100644 --- a/tests/toolchains/transitions/transitions_tests.bzl +++ b/tests/toolchains/transitions/transitions_tests.bzl @@ -20,9 +20,9 @@ load("@rules_testing//lib:test_suite.bzl", "test_suite") load("@rules_testing//lib:util.bzl", rt_util = "util") load("//python:versions.bzl", "TOOL_VERSIONS") load("//python/private:bzlmod_enabled.bzl", "BZLMOD_ENABLED") # buildifier: disable=bzl-visibility +load("//python/private:common_labels.bzl", "labels") # buildifier: disable=bzl-visibility load("//python/private:full_version.bzl", "full_version") # buildifier: disable=bzl-visibility load("//python/private:toolchain_types.bzl", "EXEC_TOOLS_TOOLCHAIN_TYPE") # buildifier: disable=bzl-visibility -load("//tests/support:support.bzl", "PYTHON_VERSION") _analysis_tests = [] @@ -33,16 +33,16 @@ def _transition_impl(input_settings, attr): for their own rule. """ settings = { - PYTHON_VERSION: input_settings[PYTHON_VERSION], + labels.PYTHON_VERSION: input_settings[labels.PYTHON_VERSION], } if attr.python_version: - settings[PYTHON_VERSION] = attr.python_version + settings[labels.PYTHON_VERSION] = attr.python_version return settings _python_version_transition = transition( implementation = _transition_impl, - inputs = [PYTHON_VERSION], - outputs = [PYTHON_VERSION], + inputs = [labels.PYTHON_VERSION], + outputs = [labels.PYTHON_VERSION], ) TestInfo = provider( diff --git a/tests/uv/uv/uv_tests.bzl b/tests/uv/uv/uv_tests.bzl index b464dab55c..8009405cec 100644 --- a/tests/uv/uv/uv_tests.bzl +++ b/tests/uv/uv/uv_tests.bzl @@ -17,6 +17,7 @@ load("@rules_testing//lib:analysis_test.bzl", "analysis_test") load("@rules_testing//lib:test_suite.bzl", "test_suite") load("@rules_testing//lib:truth.bzl", "subjects") +load("//python/private:common_labels.bzl", "labels") # buildifier: disable=bzl-visibility load("//python/uv:uv_toolchain_info.bzl", "UvToolchainInfo") load("//python/uv/private:uv.bzl", "process_modules") # buildifier: disable=bzl-visibility load("//python/uv/private:uv_toolchain.bzl", "uv_toolchain") # buildifier: disable=bzl-visibility @@ -166,7 +167,7 @@ def _test_only_defaults(env): "none", ]) uv.implementations().contains_exactly({ - "none": str(Label("//python:none")), + "none": labels.NONE, }) uv.compatible_with().contains_exactly({ "none": ["@platforms//:incompatible"], From 5fa1a87cd9d477311deaa6eb29e936c6ba7fd5fb Mon Sep 17 00:00:00 2001 From: Ignas Anikevicius <240938+aignas@users.noreply.github.com> Date: Tue, 16 Sep 2025 11:55:46 +0900 Subject: [PATCH 437/922] fix(pypi): select the lowest available libc version by default (#3255) The #3058 PR has subtly changed the default behaviour of `experimental_index_url` code path and I think in order to make things easier by default for our users we should go back to that behaviour. And in addition to this we are starting to make use of the Minimal Version Selection algorithm for the platforms. This in general allows users to configure the upper platform version for a particular wheel. This meant that we had to change the semantics of the API a little: 1. Use MVS for each platform platform tag. 2. Make it such that earlier entries are overridden by later ones, i.e. `["musllinux_*_x86_64", "musllinux_1_2_x86_64"]` is effectively the same as just `["musllinux_1_2_x86_64"]`. A remaining thing that will be left as a followup for #2747 will be to figure out how to allow users to ignore certain platform tags. Fixes #3250 --------- Co-authored-by: Richard Levasseur Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> --- CHANGELOG.md | 7 ++ python/private/pypi/extension.bzl | 42 ++++---- python/private/pypi/select_whl.bzl | 79 +++++++++++++-- tests/pypi/select_whl/select_whl_tests.bzl | 106 ++++++++++++++++++++- 4 files changed, 199 insertions(+), 35 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index abbd5f5cf1..abd89e51b5 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -79,6 +79,13 @@ END_UNRELEASED_TEMPLATE length errors due to too long environment variables. * (bootstrap) {obj}`--bootstrap_impl=script` now supports the `-S` interpreter setting. +* (pypi) We now use the Minimal Version Selection (MVS) algorithm to select + the right wheel when there are multiple wheels for the target platform + (e.g. `musllinux_1_1_x86_64` and `musllinux_1_2_x86_64`). If the user + wants to set the minimum version for the selection algorithm, use the + {attr}`pip.defaults.whl_platform_tags` attribute to configure that. If + `musllinux_*_x86_64` is specified, we will chose the lowest available + wheel version. Fixes [#3250](https://github.com/bazel-contrib/rules_python/issues/3250). {#v0-0-0-added} ### Added diff --git a/python/private/pypi/extension.bzl b/python/private/pypi/extension.bzl index 4708c8e53a..c14912c2c9 100644 --- a/python/private/pypi/extension.bzl +++ b/python/private/pypi/extension.bzl @@ -492,35 +492,37 @@ preference. Will always include `"any"` even if it is not specified. The items in this list can contain a single `*` character that is equivalent to matching the -latest available version component in the platform_tag. Note, if the wheel platform tag does not -have a version component, e.g. `linux_x86_64` or `win_amd64`, then `*` will act as a regular -character. - -We will always select the highest available `platform_tag` version that is compatible with the -target platform. +lowest available version component in the platform_tag. If the wheel platform tag does not +have a version component, e.g. `linux_x86_64` or `win_amd64`, then `*` will act as a regular character. :::{note} +Normally, the `*` in the matcher means that we will target the lowest platform version that we can +and will give preference to whls built targeting the older versions of the platform. If you +specify the version, then we will use the MVS (Minimal Version Selection) algorithm to select the +compatible wheel. As such, you need to keep in mind how to configure the target platforms to +select a particular wheel of your preference. + We select a single wheel and the last match will take precedence, if the platform_tag that we match has a version component (e.g. `android_x_arch`, then the version `x` will be used in the -matching algorithm). - -If the matcher you provide has `*`, then we will match a wheel with the highest available target platform, i.e. if `musllinux_1_1_arch` and `musllinux_1_2_arch` are both present, then we will select `musllinux_1_2_arch`. -Otherwise we will select the highest available version that is equal or lower to the specifier, i.e. if `manylinux_2_12` and `manylinux_2_17` wheels are present and the matcher is `manylinux_2_15`, then we will match `manylinux_2_12` but not `manylinux_2_17`. -::: - -:::{note} -The following tag prefixes should be used instead of the legacy equivalents: -* `manylinux_2_5` instead of `manylinux1` -* `manylinux_2_12` instead of `manylinux2010` -* `manylinux_2_17` instead of `manylinux2014` - -When parsing the whl filenames `rules_python` will automatically transform wheel filenames to the -latest format. +MVS matching algorithm). + +Common patterns: +* To select any versioned wheel for an ``, ``, use `_*_`, e.g. + `manylinux_2_17_x86_64`. +* To exclude versions up to `X.Y` - **submit a PR supporting this feature**. +* To exclude versions above `X.Y`, provide the full platform tag specifier, e.g. + `musllinux_1_2_x86_64`, which will ensure that no wheels with `musllinux_1_3_x86_64` or higher + are selected. ::: :::{seealso} See official [docs](https://packaging.python.org/en/latest/specifications/platform-compatibility-tags/#platform-tag) for more information. ::: +:::{versionchanged} VERSION_NEXT_FEATURE +The matching of versioned platforms have been switched to MVS (Minimal Version Selection) +algorithm for easier evaluation logic and fewer surprises. The legacy platform tags are +supported from this version without extra handling from the user. +::: """, ), } | AUTH_ATTRS diff --git a/python/private/pypi/select_whl.bzl b/python/private/pypi/select_whl.bzl index e9db1886e7..b32fc68f01 100644 --- a/python/private/pypi/select_whl.bzl +++ b/python/private/pypi/select_whl.bzl @@ -10,6 +10,21 @@ _MANYLINUX = "manylinux" _MACOSX = "macosx" _MUSLLINUX = "musllinux" +# Taken from https://peps.python.org/pep-0600/ +_LEGACY_ALIASES = { + "manylinux1_i686": "manylinux_2_5_i686", + "manylinux1_x86_64": "manylinux_2_5_x86_64", + "manylinux2010_i686": "manylinux_2_12_i686", + "manylinux2010_x86_64": "manylinux_2_12_x86_64", + "manylinux2014_aarch64": "manylinux_2_17_aarch64", + "manylinux2014_armv7l": "manylinux_2_17_armv7l", + "manylinux2014_i686": "manylinux_2_17_i686", + "manylinux2014_ppc64": "manylinux_2_17_ppc64", + "manylinux2014_ppc64le": "manylinux_2_17_ppc64le", + "manylinux2014_s390x": "manylinux_2_17_s390x", + "manylinux2014_x86_64": "manylinux_2_17_x86_64", +} + def _value_priority(*, tag, values): keys = [] for priority, wp in enumerate(values): @@ -18,17 +33,61 @@ def _value_priority(*, tag, values): return max(keys) if keys else None -def _platform_tag_priority(*, tag, values): - # Implements matching platform tag - # https://packaging.python.org/en/latest/specifications/platform-compatibility-tags/ - - if not ( +def _is_platform_tag_versioned(tag): + return ( tag.startswith(_ANDROID) or tag.startswith(_IOS) or tag.startswith(_MACOSX) or tag.startswith(_MANYLINUX) or tag.startswith(_MUSLLINUX) - ): + ) + +def _parse_platform_tags(tags): + """A helper function that parses all of the platform tags. + + The main idea is to make this more robust and have better debug messages about which will + is compatible and which is not with the target platform. + """ + ret = [] + replacements = {} + for tag in tags: + tag = _LEGACY_ALIASES.get(tag, tag) + + if not _is_platform_tag_versioned(tag): + ret.append(tag) + continue + + want_os, sep, tail = tag.partition("_") + if not sep: + fail("could not parse the tag: {}".format(tag)) + + want_major, _, tail = tail.partition("_") + if want_major == "*": + # the expected match is any version + want_arch = tail + elif want_os.startswith(_ANDROID): + want_arch = tail + else: + # drop the minor version segment + _, _, want_arch = tail.partition("_") + + placeholder = "{}_*_{}".format(want_os, want_arch) + replacements[placeholder] = tag + if placeholder in ret: + ret.remove(placeholder) + + ret.append(placeholder) + + return [ + replacements.get(p, p) + for p in ret + ] + +def _platform_tag_priority(*, tag, values): + # Implements matching platform tag + # https://packaging.python.org/en/latest/specifications/platform-compatibility-tags/ + + if not _is_platform_tag_versioned(tag): res = _value_priority(tag = tag, values = values) if res == None: return res @@ -39,7 +98,7 @@ def _platform_tag_priority(*, tag, values): os, _, tail = tag.partition("_") major, _, tail = tail.partition("_") - if not os.startswith(_ANDROID): + if not tag.startswith(_ANDROID): minor, _, arch = tail.partition("_") else: minor = "0" @@ -65,7 +124,7 @@ def _platform_tag_priority(*, tag, values): want_major = "" want_minor = "" want_arch = tail - elif os.startswith(_ANDROID): + elif tag.startswith(_ANDROID): # we set it to `0` above, so setting the `want_minor` her to `0` will make things # consistent. want_minor = "0" @@ -81,7 +140,7 @@ def _platform_tag_priority(*, tag, values): # if want_major is defined, then we know that we don't have a `*` in the matcher. want_version = (int(want_major), int(want_minor)) if want_major else None if not want_version or version <= want_version: - keys.append((priority, version)) + keys.append((priority, (-version[0], -version[1]))) return max(keys) if keys else None @@ -222,7 +281,7 @@ def select_whl( implementation_name = implementation_name, python_version = python_version, whl_abi_tags = whl_abi_tags, - whl_platform_tags = whl_platform_tags, + whl_platform_tags = _parse_platform_tags(whl_platform_tags), logger = logger, ) diff --git a/tests/pypi/select_whl/select_whl_tests.bzl b/tests/pypi/select_whl/select_whl_tests.bzl index 28e17ba3b3..1c28fcca5f 100644 --- a/tests/pypi/select_whl/select_whl_tests.bzl +++ b/tests/pypi/select_whl/select_whl_tests.bzl @@ -131,7 +131,6 @@ def _test_not_select_abi3(env): whl_abi_tags = ["none"], python_version = "3.13", limit = 2, - debug = True, ) _match( env, @@ -232,6 +231,34 @@ def _test_select_by_supported_cp_version(env): _tests.append(_test_select_by_supported_cp_version) +def _test_legacy_manylinux(env): + for legacy, replacement in { + "manylinux1": "manylinux_2_5", + "manylinux2010": "manylinux_2_12", + "manylinux2014": "manylinux_2_17", + }.items(): + for plat in [legacy, replacement]: + whls = [ + "pkg-0.0.1-py3-none-{}_x86_64.whl".format(plat), + "pkg-0.0.1-py3-none-any.whl", + ] + + got = _select_whl( + whls = whls, + whl_platform_tags = ["{}_x86_64".format(legacy)], + whl_abi_tags = ["none"], + python_version = "3.10", + ) + want = _select_whl( + whls = whls, + whl_platform_tags = ["{}_x86_64".format(replacement)], + whl_abi_tags = ["none"], + python_version = "3.10", + ) + _match(env, [got], want.filename) + +_tests.append(_test_legacy_manylinux) + def _test_supported_cp_version_manylinux(env): whls = [ "pkg-0.0.1-py2.py3-none-manylinux_1_1_x86_64.whl", @@ -406,9 +433,10 @@ def _test_multiple_musllinux(env): _match( env, got, - # select the one with the highest version that is matching - "pkg-0.0.1-py3-none-musllinux_1_1_x86_64.whl", + # select the one with the lowest version that is matching because we want to + # increase the compatibility "pkg-0.0.1-py3-none-musllinux_1_2_x86_64.whl", + "pkg-0.0.1-py3-none-musllinux_1_1_x86_64.whl", ) _tests.append(_test_multiple_musllinux) @@ -423,17 +451,85 @@ def _test_multiple_musllinux_exact_params(env): whl_abi_tags = ["none"], python_version = "3.12", limit = 2, + debug = True, ) _match( env, got, - # select the one with the lowest version, because of the input to the function - "pkg-0.0.1-py3-none-musllinux_1_2_x86_64.whl", + # 1.2 is not within the candidates because it is not compatible "pkg-0.0.1-py3-none-musllinux_1_1_x86_64.whl", ) _tests.append(_test_multiple_musllinux_exact_params) +def _test_multiple_mvs_match(env): + got = _select_whl( + whls = [ + "pkg-0.0.1-py3-none-musllinux_1_4_x86_64.whl", + "pkg-0.0.1-py3-none-musllinux_1_2_x86_64.whl", + "pkg-0.0.1-py3-none-musllinux_1_1_x86_64.whl", + ], + whl_platform_tags = ["musllinux_1_3_x86_64"], + whl_abi_tags = ["none"], + python_version = "3.12", + limit = 2, + ) + _match( + env, + got, + # select the one with the lowest version + "pkg-0.0.1-py3-none-musllinux_1_2_x86_64.whl", + "pkg-0.0.1-py3-none-musllinux_1_1_x86_64.whl", + ) + +_tests.append(_test_multiple_mvs_match) + +def _test_multiple_mvs_match_override_more_specific(env): + got = _select_whl( + whls = [ + "pkg-0.0.1-py3-none-musllinux_1_4_x86_64.whl", + "pkg-0.0.1-py3-none-musllinux_1_2_x86_64.whl", + "pkg-0.0.1-py3-none-musllinux_1_1_x86_64.whl", + ], + whl_platform_tags = [ + "musllinux_*_x86_64", # default to something + "musllinux_1_3_x86_64", # override the previous + ], + whl_abi_tags = ["none"], + python_version = "3.12", + limit = 2, + ) + _match( + env, + got, + # Should be the same as without the `*` match + "pkg-0.0.1-py3-none-musllinux_1_2_x86_64.whl", + "pkg-0.0.1-py3-none-musllinux_1_1_x86_64.whl", + ) + +_tests.append(_test_multiple_mvs_match_override_more_specific) + +def _test_multiple_mvs_match_override_less_specific(env): + got = _select_whl( + whls = [ + "pkg-0.0.1-py3-none-musllinux_1_4_x86_64.whl", + ], + whl_platform_tags = [ + "musllinux_1_3_x86_64", # default to 1.3 + "musllinux_*_x86_64", # then override to something less specific + ], + whl_abi_tags = ["none"], + python_version = "3.12", + limit = 2, + ) + _match( + env, + got, + "pkg-0.0.1-py3-none-musllinux_1_4_x86_64.whl", + ) + +_tests.append(_test_multiple_mvs_match_override_less_specific) + def _test_android(env): got = _select_whl( whls = [ From 77cf48db6914186efbaae05f6b9ae5941e760c7c Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Mon, 15 Sep 2025 22:05:38 -0700 Subject: [PATCH 438/922] tests: add non-blocking ci config for bazel rolling (#3272) With some core Bazel changes to flags coming that will affect us, and the difficulty it is to keep the bazel-at-head-and-downstream pipeline green, I figured it'd be a good idea to add a CI job that uses the weekly Bazel release so we can identify problems sooner and more obviously. The CI job only run on Ubuntu to save CI slots and is won't block merges if it has failures. --- .bazelci/presubmit.yml | 14 ++++++++++++++ 1 file changed, 14 insertions(+) diff --git a/.bazelci/presubmit.yml b/.bazelci/presubmit.yml index 5889823d3d..119ad498b0 100644 --- a/.bazelci/presubmit.yml +++ b/.bazelci/presubmit.yml @@ -132,6 +132,20 @@ tasks: name: "Default: Ubuntu, upcoming Bazel" platform: ubuntu2204 bazel: last_rc + ubuntu_rolling: + name: "Default: Ubuntu, rolling Bazel" + platform: ubuntu2204 + bazel: rolling + # This is an advisory job; doesn't block merges + soft_fail: + - exit_status: 1 + - exit_status: 3 + test_targets: + - "--" + - "//tests/..." + test_flags: + - "--keep_going" + - "--test_tag_filters=-integration-test" ubuntu_workspace: <<: *reusable_config <<: *common_workspace_flags From cbc8774681a3c9cf70cf71ddf1a485e06cf20f59 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Mon, 15 Sep 2025 23:25:19 -0700 Subject: [PATCH 439/922] fix: venv site packages with pkgutil packages (#3268) Currently, an error occurs if one packages files are intended to go into a sub-directory of another package's directory. This can happen when pkgutil-style namespace packages are used, which results in multiple distributions wanting to install the same files (pkgutil `__init__.py` files) into the same top-level directories. This eventually results in a Bazel error because Bazel detects that the one output is the prefix of another. To fix, detect when distributions overlap in their paths and merge their files manually. Internally, entries are sorted from shorted venv path to longest, however, that's just an implementation detail. Along the way, give agents better advice for bzl_library targets. Fixes https://github.com/bazel-contrib/rules_python/issues/3204 --------- Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> --- AGENTS.md | 10 +- CHANGELOG.md | 3 + python/private/BUILD.bazel | 12 + python/private/common.bzl | 34 ++ python/private/py_executable.bzl | 125 +------ python/private/py_info.bzl | 8 + python/private/py_library.bzl | 131 ++------ python/private/venv_runfiles.bzl | 318 ++++++++++++++++++ .../venv_relative_path_tests.bzl | 2 +- tests/venv_site_packages_libs/BUILD.bazel | 2 + .../app_files_building/BUILD.bazel | 5 + .../app_files_building_tests.bzl | 247 ++++++++++++++ tests/venv_site_packages_libs/bin.py | 12 + .../pkgutil_top/BUILD.bazel | 10 + .../site-packages/pkgutil_top/__init__.py | 2 + .../site-packages/pkgutil_top/top.py | 0 .../pkgutil_top_sub/BUILD.bazel | 10 + .../site-packages/pkgutil_top/sub/__init__.py | 1 + .../site-packages/pkgutil_top/sub/suba.py | 0 19 files changed, 697 insertions(+), 235 deletions(-) create mode 100644 python/private/venv_runfiles.bzl create mode 100644 tests/venv_site_packages_libs/app_files_building/BUILD.bazel create mode 100644 tests/venv_site_packages_libs/app_files_building/app_files_building_tests.bzl create mode 100644 tests/venv_site_packages_libs/pkgutil_top/BUILD.bazel create mode 100644 tests/venv_site_packages_libs/pkgutil_top/site-packages/pkgutil_top/__init__.py create mode 100644 tests/venv_site_packages_libs/pkgutil_top/site-packages/pkgutil_top/top.py create mode 100644 tests/venv_site_packages_libs/pkgutil_top_sub/BUILD.bazel create mode 100644 tests/venv_site_packages_libs/pkgutil_top_sub/site-packages/pkgutil_top/sub/__init__.py create mode 100644 tests/venv_site_packages_libs/pkgutil_top_sub/site-packages/pkgutil_top/sub/suba.py diff --git a/AGENTS.md b/AGENTS.md index e21b15bd03..3d5219c2bc 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -21,10 +21,16 @@ into the sentence, not verbatim. ### bzl_library targets for bzl source files -* `.bzl` files should have `bzl_library` defined for them. +* A `bzl_library` target should be defined for every `.bzl` file outside + of the `tests/` directory. * They should have a single `srcs` file and be named after the file with `_bzl` appended. -* Their deps should be based on the `load()` statements in the source file. +* Their deps should be based on the `load()` statements in the source file + and refer to the `bzl_library` target containing the loaded file. + * For files in rules_python: replace `.bzl` with `_bzl`. + e.g. given `load("//foo:bar.bzl", ...)`, the target is `//foo:bar_bzl`. + * For files outside rules_python: remove the `.bzl` suffix. e.g. given + `load("@foo//foo:bar.bzl", ...)`, the target is `@foo//foo:bar`. * `bzl_library()` targets should be kept in alphabetical order by name. Example: diff --git a/CHANGELOG.md b/CHANGELOG.md index abd89e51b5..382096f826 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -86,6 +86,9 @@ END_UNRELEASED_TEMPLATE {attr}`pip.defaults.whl_platform_tags` attribute to configure that. If `musllinux_*_x86_64` is specified, we will chose the lowest available wheel version. Fixes [#3250](https://github.com/bazel-contrib/rules_python/issues/3250). +* (venvs) {obj}`--vens_site_packages=yes` no longer errors when packages with + overlapping files or directories are used together. + ([#3204](https://github.com/bazel-contrib/rules_python/issues/3204)). {#v0-0-0-added} ### Added diff --git a/python/private/BUILD.bazel b/python/private/BUILD.bazel index 916c14f9f2..5e2043c0c5 100644 --- a/python/private/BUILD.bazel +++ b/python/private/BUILD.bazel @@ -419,6 +419,7 @@ bzl_library( ":rules_cc_srcs_bzl", ":toolchain_types_bzl", ":transition_labels_bzl", + ":venv_runfiles_bzl", "@bazel_skylib//lib:dicts", "@bazel_skylib//lib:paths", "@bazel_skylib//lib:structs", @@ -468,6 +469,7 @@ bzl_library( ":py_internal_bzl", ":rule_builders_bzl", ":toolchain_types_bzl", + ":venv_runfiles_bzl", ":version_bzl", "@bazel_skylib//lib:dicts", "@bazel_skylib//rules:common_settings", @@ -727,6 +729,16 @@ bzl_library( ], ) +bzl_library( + name = "venv_runfiles_bzl", + srcs = ["venv_runfiles.bzl"], + deps = [ + ":common_bzl", + ":py_info.bzl", + "@bazel_skylib//lib:paths", + ], +) + # Needed to define bzl_library targets for docgen. (We don't define the # bzl_library target here because it'd give our users a transitive dependency # on Skylib.) diff --git a/python/private/common.bzl b/python/private/common.bzl index 9fc366818d..33b175c247 100644 --- a/python/private/common.bzl +++ b/python/private/common.bzl @@ -495,6 +495,9 @@ _BOOL_TYPE = type(True) def is_bool(v): return type(v) == _BOOL_TYPE +def is_file(v): + return type(v) == "File" + def target_platform_has_any_constraint(ctx, constraints): """Check if target platform has any of a list of constraints. @@ -511,6 +514,37 @@ def target_platform_has_any_constraint(ctx, constraints): return True return False +def relative_path(from_, to): + """Compute a relative path from one path to another. + + Args: + from_: {type}`str` the starting directory. Note that it should be + a directory because relative-symlinks are relative to the + directory the symlink resides in. + to: {type}`str` the path that `from_` wants to point to + + Returns: + {type}`str` a relative path + """ + from_parts = from_.split("/") + to_parts = to.split("/") + + # Strip common leading parts from both paths + n = min(len(from_parts), len(to_parts)) + for _ in range(n): + if from_parts[0] == to_parts[0]: + from_parts.pop(0) + to_parts.pop(0) + else: + break + + # Impossible to compute a relative path without knowing what ".." is + if from_parts and from_parts[0] == "..": + fail("cannot compute relative path from '%s' to '%s'", from_, to) + + parts = ([".."] * len(from_parts)) + to_parts + return paths.join(*parts) + def runfiles_root_path(ctx, short_path): """Compute a runfiles-root relative path from `File.short_path` diff --git a/python/private/py_executable.bzl b/python/private/py_executable.bzl index 59800da566..bef5934729 100644 --- a/python/private/py_executable.bzl +++ b/python/private/py_executable.bzl @@ -48,6 +48,7 @@ load( "filter_to_py_srcs", "get_imports", "is_bool", + "relative_path", "runfiles_root_path", "target_platform_has_any_constraint", ) @@ -63,6 +64,7 @@ load(":reexports.bzl", "BuiltinPyInfo", "BuiltinPyRuntimeInfo") load(":rule_builders.bzl", "ruleb") load(":toolchain_types.bzl", "EXEC_TOOLS_TOOLCHAIN_TYPE", "TARGET_TOOLCHAIN_TYPE", TOOLCHAIN_TYPE = "TARGET_TOOLCHAIN_TYPE") load(":transition_labels.bzl", "TRANSITION_LABELS") +load(":venv_runfiles.bzl", "create_venv_app_files") _py_builtins = py_internal _EXTERNAL_PATH_PREFIX = "external" @@ -499,37 +501,6 @@ def _create_zip_main(ctx, *, stage2_bootstrap, runtime_details, venv): ) return output -def relative_path(from_, to): - """Compute a relative path from one path to another. - - Args: - from_: {type}`str` the starting directory. Note that it should be - a directory because relative-symlinks are relative to the - directory the symlink resides in. - to: {type}`str` the path that `from_` wants to point to - - Returns: - {type}`str` a relative path - """ - from_parts = from_.split("/") - to_parts = to.split("/") - - # Strip common leading parts from both paths - n = min(len(from_parts), len(to_parts)) - for _ in range(n): - if from_parts[0] == to_parts[0]: - from_parts.pop(0) - to_parts.pop(0) - else: - break - - # Impossible to compute a relative path without knowing what ".." is - if from_parts and from_parts[0] == "..": - fail("cannot compute relative path from '%s' to '%s'", from_, to) - - parts = ([".."] * len(from_parts)) + to_parts - return paths.join(*parts) - # Create a venv the executable can use. # For venv details and the venv startup process, see: # * https://docs.python.org/3/library/venv.html @@ -636,9 +607,9 @@ def _create_venv(ctx, output_prefix, imports, runtime_details): VenvSymlinkKind.BIN: bin_dir, VenvSymlinkKind.LIB: site_packages, } - venv_symlinks = _create_venv_symlinks(ctx, venv_dir_map) + venv_app_files = create_venv_app_files(ctx, ctx.attr.deps, venv_dir_map) - files_without_interpreter = [pth, site_init] + venv_symlinks + files_without_interpreter = [pth, site_init] + venv_app_files if pyvenv_cfg: files_without_interpreter.append(pyvenv_cfg) @@ -663,94 +634,6 @@ def _create_venv(ctx, output_prefix, imports, runtime_details): ), ) -def _create_venv_symlinks(ctx, venv_dir_map): - """Creates symlinks within the venv. - - Args: - ctx: current rule ctx - venv_dir_map: mapping of VenvSymlinkKind constants to the - venv path. - - Returns: - {type}`list[File]` list of the File symlink objects created. - """ - - # maps venv-relative path to the runfiles path it should point to - entries = depset( - transitive = [ - dep[PyInfo].venv_symlinks - for dep in ctx.attr.deps - if PyInfo in dep - ], - ).to_list() - - link_map = _build_link_map(entries) - venv_files = [] - for kind, kind_map in link_map.items(): - base = venv_dir_map[kind] - for venv_path, link_to in kind_map.items(): - venv_link = ctx.actions.declare_symlink(paths.join(base, venv_path)) - venv_link_rf_path = runfiles_root_path(ctx, venv_link.short_path) - rel_path = relative_path( - # dirname is necessary because a relative symlink is relative to - # the directory the symlink resides within. - from_ = paths.dirname(venv_link_rf_path), - to = link_to, - ) - ctx.actions.symlink(output = venv_link, target_path = rel_path) - venv_files.append(venv_link) - - return venv_files - -def _build_link_map(entries): - # dict[str package, dict[str kind, dict[str rel_path, str link_to_path]]] - pkg_link_map = {} - - # dict[str package, str version] - version_by_pkg = {} - - for entry in entries: - link_map = pkg_link_map.setdefault(entry.package, {}) - kind_map = link_map.setdefault(entry.kind, {}) - - if version_by_pkg.setdefault(entry.package, entry.version) != entry.version: - # We ignore duplicates by design. - continue - elif entry.venv_path in kind_map: - # We ignore duplicates by design. - continue - else: - kind_map[entry.venv_path] = entry.link_to_path - - # An empty link_to value means to not create the site package symlink. Because of the - # ordering, this allows binaries to remove entries by having an earlier dependency produce - # empty link_to values. - for link_map in pkg_link_map.values(): - for kind, kind_map in link_map.items(): - for dir_path, link_to in kind_map.items(): - if not link_to: - kind_map.pop(dir_path) - - # dict[str kind, dict[str rel_path, str link_to_path]] - keep_link_map = {} - - # Remove entries that would be a child path of a created symlink. - # Earlier entries have precedence to match how exact matches are handled. - for link_map in pkg_link_map.values(): - for kind, kind_map in link_map.items(): - keep_kind_map = keep_link_map.setdefault(kind, {}) - for _ in range(len(kind_map)): - if not kind_map: - break - dirname, value = kind_map.popitem() - keep_kind_map[dirname] = value - prefix = dirname + "/" # Add slash to prevent /X matching /XY - for maybe_suffix in kind_map.keys(): - maybe_suffix += "/" # Add slash to prevent /X matching /XY - if maybe_suffix.startswith(prefix) or prefix.startswith(maybe_suffix): - kind_map.pop(maybe_suffix) - return keep_link_map - def _map_each_identity(v): return v diff --git a/python/private/py_info.bzl b/python/private/py_info.bzl index 31df5cfbde..f96dec554b 100644 --- a/python/private/py_info.bzl +++ b/python/private/py_info.bzl @@ -56,6 +56,14 @@ VenvSymlinkEntry = provider( An entry in `PyInfo.venv_symlinks` """, fields = { + "files": """ +:type: depset[File] + +Files under `link_to_path`. + +This is only used when multiple targets have overlapping `venv_path` paths. e.g. +if one adds files to `venv_path=a/` and another adds files to `venv_path=a/b/`. +""", "kind": """ :type: str diff --git a/python/private/py_library.bzl b/python/private/py_library.bzl index fc8e5839a0..b2a9fdd3be 100644 --- a/python/private/py_library.bzl +++ b/python/private/py_library.bzl @@ -28,7 +28,6 @@ load( load(":builders.bzl", "builders") load( ":common.bzl", - "PYTHON_FILE_EXTENSIONS", "collect_cc_info", "collect_imports", "collect_runfiles", @@ -38,14 +37,13 @@ load( "create_py_info", "filter_to_py_srcs", "get_imports", - "runfiles_root_path", ) load(":common_labels.bzl", "labels") load(":flags.bzl", "AddSrcsToRunfilesFlag", "PrecompileFlag", "VenvsSitePackages") load(":normalize_name.bzl", "normalize_name") load(":precompile.bzl", "maybe_precompile") load(":py_cc_link_params_info.bzl", "PyCcLinkParamsInfo") -load(":py_info.bzl", "PyInfo", "VenvSymlinkEntry", "VenvSymlinkKind") +load(":py_info.bzl", "PyInfo") load(":reexports.bzl", "BuiltinPyInfo") load(":rule_builders.bzl", "ruleb") load( @@ -53,6 +51,7 @@ load( "EXEC_TOOLS_TOOLCHAIN_TYPE", TOOLCHAIN_TYPE = "TARGET_TOOLCHAIN_TYPE", ) +load(":venv_runfiles.bzl", "get_venv_symlinks") load(":version.bzl", "version") LIBRARY_ATTRS = dicts.add( @@ -235,117 +234,27 @@ def _get_imports_and_venv_symlinks(ctx, semantics): venv_symlinks = [] if VenvsSitePackages.is_enabled(ctx): package, version_str = _get_package_and_version(ctx) - venv_symlinks = _get_venv_symlinks(ctx, package, version_str) - else: - imports = collect_imports(ctx, semantics) - return imports, venv_symlinks - -def _get_venv_symlinks(ctx, package, version_str): - imports = ctx.attr.imports - if len(imports) == 0: - fail("When venvs_site_packages is enabled, exactly one `imports` " + - "value must be specified, got 0") - elif len(imports) > 1: - fail("When venvs_site_packages is enabled, exactly one `imports` " + - "value must be specified, got {}".format(imports)) - else: - site_packages_root = imports[0] - - if site_packages_root.endswith("/"): - fail("The site packages root value from `imports` cannot end in " + - "slash, got {}".format(site_packages_root)) - if site_packages_root.startswith("/"): - fail("The site packages root value from `imports` cannot start with " + - "slash, got {}".format(site_packages_root)) - - # Append slash to prevent incorrectly prefix-string matches - site_packages_root += "/" - - # We have to build a list of (runfiles path, site-packages path) pairs of the files to - # create in the consuming binary's venv site-packages directory. To minimize the number of - # files to create, we just return the paths to the directories containing the code of - # interest. - # - # However, namespace packages complicate matters: multiple distributions install in the - # same directory in site-packages. This works out because they don't overlap in their - # files. Typically, they install to different directories within the namespace package - # directory. We also need to ensure that we can handle a case where the main package (e.g. - # airflow) has directories only containing data files and then namespace packages coming - # along and being next to it. - # - # Lastly we have to assume python modules just being `.py` files (e.g. typing-extensions) - # is just a single Python file. - - dir_symlinks = {} # dirname -> runfile path - venv_symlinks = [] - for src in ctx.files.srcs + ctx.files.data + ctx.files.pyi_srcs: - path = _repo_relative_short_path(src.short_path) - if not path.startswith(site_packages_root): - continue - path = path.removeprefix(site_packages_root) - dir_name, _, filename = path.rpartition("/") - if dir_name in dir_symlinks: - # we already have this dir, this allows us to short-circuit since most of the - # ctx.files.data might share the same directories as ctx.files.srcs - continue - - runfiles_dir_name, _, _ = runfiles_root_path(ctx, src.short_path).partition("/") - if dir_name: - # This can be either: - # * a directory with libs (e.g. numpy.libs, created by auditwheel) - # * a directory with `__init__.py` file that potentially also needs to be - # symlinked. - # * `.dist-info` directory - # - # This could be also regular files, that just need to be symlinked, so we will - # add the directory here. - dir_symlinks[dir_name] = runfiles_dir_name - elif src.extension in PYTHON_FILE_EXTENSIONS: - # This would be files that do not have directories and we just need to add - # direct symlinks to them as is, we only allow Python files in here - entry = VenvSymlinkEntry( - kind = VenvSymlinkKind.LIB, - link_to_path = paths.join(runfiles_dir_name, site_packages_root, filename), - package = package, - version = version_str, - venv_path = filename, - ) - venv_symlinks.append(entry) - - # Sort so that we encounter `foo` before `foo/bar`. This ensures we - # see the top-most explicit package first. - dirnames = sorted(dir_symlinks.keys()) - first_level_explicit_packages = [] - for d in dirnames: - is_sub_package = False - for existing in first_level_explicit_packages: - # Suffix with / to prevent foo matching foobar - if d.startswith(existing + "/"): - is_sub_package = True - break - if not is_sub_package: - first_level_explicit_packages.append(d) - - for dirname in first_level_explicit_packages: - prefix = dir_symlinks[dirname] - entry = VenvSymlinkEntry( - kind = VenvSymlinkKind.LIB, - link_to_path = paths.join(prefix, site_packages_root, dirname), - package = package, - version = version_str, - venv_path = dirname, + # NOTE: Already a list, but buildifier thinks its a depset and + # adds to_list() calls later. + imports = list(ctx.attr.imports) + if len(imports) == 0: + fail("When venvs_site_packages is enabled, exactly one `imports` " + + "value must be specified, got 0") + elif len(imports) > 1: + fail("When venvs_site_packages is enabled, exactly one `imports` " + + "value must be specified, got {}".format(imports)) + + venv_symlinks = get_venv_symlinks( + ctx, + ctx.files.srcs + ctx.files.data + ctx.files.pyi_srcs, + package, + version_str, + site_packages_root = imports[0], ) - venv_symlinks.append(entry) - - return venv_symlinks - -def _repo_relative_short_path(short_path): - # Convert `../+pypi+foo/some/file.py` to `some/file.py` - if short_path.startswith("../"): - return short_path[3:].partition("/")[2] else: - return short_path + imports = collect_imports(ctx, semantics) + return imports, venv_symlinks _MaybeBuiltinPyInfo = [BuiltinPyInfo] if BuiltinPyInfo != None else [] diff --git a/python/private/venv_runfiles.bzl b/python/private/venv_runfiles.bzl new file mode 100644 index 0000000000..291920b848 --- /dev/null +++ b/python/private/venv_runfiles.bzl @@ -0,0 +1,318 @@ +"""Code for constructing venvs.""" + +load("@bazel_skylib//lib:paths.bzl", "paths") +load( + ":common.bzl", + "PYTHON_FILE_EXTENSIONS", + "is_file", + "relative_path", + "runfiles_root_path", +) +load( + ":py_info.bzl", + "PyInfo", + "VenvSymlinkEntry", + "VenvSymlinkKind", +) + +def create_venv_app_files(ctx, deps, venv_dir_map): + """Creates the tree of app-specific files for a venv for a binary. + + App specific files are the files that come from dependencies. + + Args: + ctx: {type}`ctx` current ctx. + deps: {type}`list[Target]` the targets whose venv information + to put into the returned venv files. + venv_dir_map: mapping of VenvSymlinkKind constants to the + venv path. This tells the directory name of + platform/configuration-dependent directories. The values are + paths within the current ctx's venv (e.g. `_foo.venv/bin`). + + Returns: + {type}`list[File]` of the files that were created. + """ + + # maps venv-relative path to the runfiles path it should point to + entries = depset( + transitive = [ + dep[PyInfo].venv_symlinks + for dep in deps + if PyInfo in dep + ], + ).to_list() + + link_map = build_link_map(ctx, entries) + venv_files = [] + for kind, kind_map in link_map.items(): + base = venv_dir_map[kind] + for venv_path, link_to in kind_map.items(): + bin_venv_path = paths.join(base, venv_path) + if is_file(link_to): + if link_to.is_directory: + venv_link = ctx.actions.declare_directory(bin_venv_path) + else: + venv_link = ctx.actions.declare_file(bin_venv_path) + ctx.actions.symlink(output = venv_link, target_file = link_to) + else: + venv_link = ctx.actions.declare_symlink(bin_venv_path) + venv_link_rf_path = runfiles_root_path(ctx, venv_link.short_path) + rel_path = relative_path( + # dirname is necessary because a relative symlink is relative to + # the directory the symlink resides within. + from_ = paths.dirname(venv_link_rf_path), + to = link_to, + ) + ctx.actions.symlink(output = venv_link, target_path = rel_path) + venv_files.append(venv_link) + + return venv_files + +# Visible for testing +def build_link_map(ctx, entries): + """Compute the mapping of venv paths to their backing objects. + + + Args: + ctx: {type}`ctx` current ctx. + entries: {type}`list[VenvSymlinkEntry]` the entries that describe the + venv-relative + + Returns: + {type}`dict[str, dict[str, str|File]]` Mappings of venv paths to their + backing files. The first key is a `VenvSymlinkKind` value. + The inner dict keys are venv paths relative to the kind's diretory. The + inner dict values are strings or Files to link to. + """ + + version_by_pkg = {} # dict[str pkg, str version] + entries_by_kind = {} # dict[str kind, list[entry]] + + # Group by path kind and reduce to a single package's version of entries + for entry in entries: + entries_by_kind.setdefault(entry.kind, []) + if not entry.package: + entries_by_kind[entry.kind].append(entry) + continue + if entry.package not in version_by_pkg: + version_by_pkg[entry.package] = entry.version + entries_by_kind[entry.kind].append(entry) + continue + if entry.version == version_by_pkg[entry.package]: + entries_by_kind[entry.kind].append(entry) + continue + + # else: ignore it; not the selected version + + # final paths to keep, grouped by kind + keep_link_map = {} # dict[str kind, dict[path, str|File]] + for kind, entries in entries_by_kind.items(): + # dict[str kind-relative path, str|File link_to] + keep_kind_link_map = {} + + groups = _group_venv_path_entries(entries) + + for group in groups: + # If there's just one group, we can symlink to the directory + if len(group) == 1: + entry = group[0] + keep_kind_link_map[entry.venv_path] = entry.link_to_path + else: + # Merge a group of overlapping prefixes + _merge_venv_path_group(ctx, group, keep_kind_link_map) + + keep_link_map[kind] = keep_kind_link_map + + return keep_link_map + +def _group_venv_path_entries(entries): + """Group entries by VenvSymlinkEntry.venv_path overlap. + + This does an initial grouping by the top-level venv path an entry wants. + Entries that are underneath another entry are put into the same group. + + Returns: + {type}`list[list[VenvSymlinkEntry]]` The inner list is the entries under + a common venv path. The inner list is ordered from shortest to longest + path. + """ + + # Sort so order is top-down, ensuring grouping by short common prefix + entries = sorted(entries, key = lambda e: e.venv_path) + + groups = [] + current_group = None + current_group_prefix = None + for entry in entries: + prefix = entry.venv_path + anchored_prefix = prefix + "/" + if (current_group_prefix == None or + not anchored_prefix.startswith(current_group_prefix)): + current_group_prefix = anchored_prefix + current_group = [entry] + groups.append(current_group) + else: + current_group.append(entry) + + return groups + +def _merge_venv_path_group(ctx, group, keep_map): + """Merges a group of overlapping prefixes. + + Args: + ctx: {type}`ctx` current ctx. + group: {type}`list[VenvSymlinkEntry]` a group of entries with overlapping + `venv_path` prefixes, ordered from shortest to longest path. + keep_map: {type}`dict[str, str|File]` files kept after merging are + populated into this map. + """ + + # TODO: Compute the minimum number of entries to create. This can't avoid + # flattening the files depset, but can lower the number of materialized + # files significantly. Usually overlaps are limited to a small number + # of directories. + for entry in group: + prefix = entry.venv_path + for file in entry.files.to_list(): + # Compute the file-specific venv path. i.e. the relative + # path of the file under entry.venv_path, joined with + # entry.venv_path + rf_root_path = runfiles_root_path(ctx, file.short_path) + if not rf_root_path.startswith(entry.link_to_path): + # This generally shouldn't occur in practice, but just + # in case, skip them, for lack of a better option. + continue + venv_path = "{}/{}".format( + prefix, + rf_root_path.removeprefix(entry.link_to_path + "/"), + ) + + # For lack of a better option, first added wins. We happen to + # go in top-down prefix order, so the highest level namespace + # package typically wins. + if venv_path not in keep_map: + keep_map[venv_path] = file + +def get_venv_symlinks(ctx, files, package, version_str, site_packages_root): + """Compute the VenvSymlinkEntry objects for a library. + + Args: + ctx: {type}`ctx` the current ctx. + files: {type}`list[File]` the underlying files that are under + `site_packages_root` and intended to be part of the venv + contents. + package: {type}`str` the Python distribution name. + version_str: {type}`str` the distribution's version. + site_packages_root: {type}`str` prefix under which files are + considered to be part of the installed files. + + Returns: + {type}`list[VenvSymlinkEntry]` the entries that describe how + to map the files into a venv. + """ + if site_packages_root.endswith("/"): + fail("The `site_packages_root` value cannot end in " + + "slash, got {}".format(site_packages_root)) + if site_packages_root.startswith("/"): + fail("The `site_packages_root` cannot start with " + + "slash, got {}".format(site_packages_root)) + + # Append slash to prevent incorrect prefix-string matches + site_packages_root += "/" + + # We have to build a list of (runfiles path, site-packages path) pairs of the files to + # create in the consuming binary's venv site-packages directory. To minimize the number of + # files to create, we just return the paths to the directories containing the code of + # interest. + # + # However, namespace packages complicate matters: multiple distributions install in the + # same directory in site-packages. This works out because they don't overlap in their + # files. Typically, they install to different directories within the namespace package + # directory. We also need to ensure that we can handle a case where the main package (e.g. + # airflow) has directories only containing data files and then namespace packages coming + # along and being next to it. + # + # Lastly we have to assume python modules just being `.py` files (e.g. typing-extensions) + # is just a single Python file. + + dir_symlinks = {} # dirname -> runfile path + venv_symlinks = [] + + # Sort so order is top-down + all_files = sorted(files, key = lambda f: f.short_path) + + for src in all_files: + path = _repo_relative_short_path(src.short_path) + if not path.startswith(site_packages_root): + continue + path = path.removeprefix(site_packages_root) + dir_name, _, filename = path.rpartition("/") + + if dir_name in dir_symlinks: + # we already have this dir, this allows us to short-circuit since most of the + # ctx.files.data might share the same directories as ctx.files.srcs + continue + + runfiles_dir_name, _, _ = runfiles_root_path(ctx, src.short_path).partition("/") + if dir_name: + # This can be either: + # * a directory with libs (e.g. numpy.libs, created by auditwheel) + # * a directory with `__init__.py` file that potentially also needs to be + # symlinked. + # * `.dist-info` directory + # + # This could be also regular files, that just need to be symlinked, so we will + # add the directory here. + dir_symlinks[dir_name] = runfiles_dir_name + elif src.extension in PYTHON_FILE_EXTENSIONS: + # This would be files that do not have directories and we just need to add + # direct symlinks to them as is, we only allow Python files in here + entry = VenvSymlinkEntry( + kind = VenvSymlinkKind.LIB, + link_to_path = paths.join(runfiles_dir_name, site_packages_root, filename), + package = package, + version = version_str, + venv_path = filename, + files = depset([src]), + ) + venv_symlinks.append(entry) + + # Sort so that we encounter `foo` before `foo/bar`. This ensures we + # see the top-most explicit package first. + dirnames = sorted(dir_symlinks.keys()) + first_level_explicit_packages = [] + for d in dirnames: + is_sub_package = False + for existing in first_level_explicit_packages: + # Suffix with / to prevent foo matching foobar + if d.startswith(existing + "/"): + is_sub_package = True + break + if not is_sub_package: + first_level_explicit_packages.append(d) + + for dirname in first_level_explicit_packages: + prefix = dir_symlinks[dirname] + link_to_path = paths.join(prefix, site_packages_root, dirname) + entry = VenvSymlinkEntry( + kind = VenvSymlinkKind.LIB, + link_to_path = link_to_path, + package = package, + version = version_str, + venv_path = dirname, + files = depset([ + f + for f in all_files + if runfiles_root_path(ctx, f.short_path).startswith(link_to_path + "/") + ]), + ) + venv_symlinks.append(entry) + + return venv_symlinks + +def _repo_relative_short_path(short_path): + # Convert `../+pypi+foo/some/file.py` to `some/file.py` + if short_path.startswith("../"): + return short_path[3:].partition("/")[2] + else: + return short_path diff --git a/tests/bootstrap_impls/venv_relative_path_tests.bzl b/tests/bootstrap_impls/venv_relative_path_tests.bzl index ad4870fe08..72b5012809 100644 --- a/tests/bootstrap_impls/venv_relative_path_tests.bzl +++ b/tests/bootstrap_impls/venv_relative_path_tests.bzl @@ -15,7 +15,7 @@ "Unit tests for relative_path computation" load("@rules_testing//lib:test_suite.bzl", "test_suite") -load("//python/private:py_executable.bzl", "relative_path") # buildifier: disable=bzl-visibility +load("//python/private:common.bzl", "relative_path") # buildifier: disable=bzl-visibility _tests = [] diff --git a/tests/venv_site_packages_libs/BUILD.bazel b/tests/venv_site_packages_libs/BUILD.bazel index e64299e1ad..92d5dec6d3 100644 --- a/tests/venv_site_packages_libs/BUILD.bazel +++ b/tests/venv_site_packages_libs/BUILD.bazel @@ -26,6 +26,8 @@ py_reconfig_test( ":closer_lib", "//tests/venv_site_packages_libs/nspkg_alpha", "//tests/venv_site_packages_libs/nspkg_beta", + "//tests/venv_site_packages_libs/pkgutil_top", + "//tests/venv_site_packages_libs/pkgutil_top_sub", "@other//nspkg_delta", "@other//nspkg_gamma", "@other//nspkg_single", diff --git a/tests/venv_site_packages_libs/app_files_building/BUILD.bazel b/tests/venv_site_packages_libs/app_files_building/BUILD.bazel new file mode 100644 index 0000000000..60afd34c38 --- /dev/null +++ b/tests/venv_site_packages_libs/app_files_building/BUILD.bazel @@ -0,0 +1,5 @@ +load(":app_files_building_tests.bzl", "app_files_building_test_suite") + +app_files_building_test_suite( + name = "app_files_building_tests", +) diff --git a/tests/venv_site_packages_libs/app_files_building/app_files_building_tests.bzl b/tests/venv_site_packages_libs/app_files_building/app_files_building_tests.bzl new file mode 100644 index 0000000000..0a0265eb8c --- /dev/null +++ b/tests/venv_site_packages_libs/app_files_building/app_files_building_tests.bzl @@ -0,0 +1,247 @@ +"" + +load("@bazel_skylib//lib:paths.bzl", "paths") +load("@rules_testing//lib:analysis_test.bzl", "analysis_test") +load("@rules_testing//lib:test_suite.bzl", "test_suite") +load("//python/private:py_info.bzl", "VenvSymlinkEntry", "VenvSymlinkKind") # buildifier: disable=bzl-visibility +load("//python/private:venv_runfiles.bzl", "build_link_map") # buildifier: disable=bzl-visibility + +_tests = [] + +def _ctx(workspace_name = "_main"): + return struct( + workspace_name = workspace_name, + ) + +def _file(short_path): + return struct( + short_path = short_path, + ) + +def _entry(venv_path, link_to_path, files = [], **kwargs): + kwargs.setdefault("kind", VenvSymlinkKind.LIB) + kwargs.setdefault("package", None) + kwargs.setdefault("version", None) + + def short_pathify(path): + path = paths.join(link_to_path, path) + + # In tests, `../` is used to step out of the link_to_path scope. + path = paths.normalize(path) + + # Treat paths starting with "+" as external references. This matches + # how bzlmod names things. + if link_to_path.startswith("+"): + # File.short_path to external repos have `../` prefixed + path = paths.join("../", path) + else: + # File.short_path in main repo is main-repo relative + _, _, path = path.partition("/") + return path + + return VenvSymlinkEntry( + venv_path = venv_path, + link_to_path = link_to_path, + files = depset([ + _file(short_pathify(f)) + for f in files + ]), + **kwargs + ) + +def _test_conflict_merging(name): + analysis_test( + name = name, + impl = _test_conflict_merging_impl, + target = "//python:none", + ) + +_tests.append(_test_conflict_merging) + +def _test_conflict_merging_impl(env, _): + entries = [ + _entry("a", "+pypi_a/site-packages/a", ["a.txt"]), + _entry("a/b", "+pypi_a_b/site-packages/a/b", ["b.txt"]), + _entry("x", "_main/src/x", ["x.txt"]), + _entry("x/p", "_main/src-dev/x/p", ["p.txt"]), + _entry("duplicate", "+dupe_a/site-packages/duplicate", ["d.py"]), + # This entry also provides a/x.py, but since the "a" entry is shorter + # and comes first, its version of x.py should win. + _entry("duplicate", "+dupe_b/site-packages/duplicate", ["d.py"]), + ] + + actual = build_link_map(_ctx(), entries) + expected_libs = { + "a/a.txt": _file("../+pypi_a/site-packages/a/a.txt"), + "a/b/b.txt": _file("../+pypi_a_b/site-packages/a/b/b.txt"), + "duplicate/d.py": _file("../+dupe_a/site-packages/duplicate/d.py"), + "x/p/p.txt": _file("src-dev/x/p/p.txt"), + "x/x.txt": _file("src/x/x.txt"), + } + env.expect.that_dict(actual[VenvSymlinkKind.LIB]).contains_exactly(expected_libs) + env.expect.that_dict(actual).keys().contains_exactly([VenvSymlinkKind.LIB]) + +def _test_package_version_filtering(name): + analysis_test( + name = name, + impl = _test_package_version_filtering_impl, + target = "//python:none", + ) + +_tests.append(_test_package_version_filtering) + +def _test_package_version_filtering_impl(env, _): + entries = [ + _entry("foo", "+pypi_v1/site-packages/foo", ["foo.txt"], package = "foo", version = "1.0"), + _entry("foo", "+pypi_v2/site-packages/foo", ["bar.txt"], package = "foo", version = "2.0"), + ] + + actual = build_link_map(_ctx(), entries) + + expected_libs = { + "foo": "+pypi_v1/site-packages/foo", + } + env.expect.that_dict(actual[VenvSymlinkKind.LIB]).contains_exactly(expected_libs) + +def _test_malformed_entry(name): + analysis_test( + name = name, + impl = _test_malformed_entry_impl, + target = "//python:none", + ) + +_tests.append(_test_malformed_entry) + +def _test_malformed_entry_impl(env, _): + entries = [ + _entry( + "a", + "+pypi_a/site-packages/a", + # This file is outside the link_to_path, so it should be ignored. + ["../outside.txt"], + ), + # A second, conflicting, entry is added to force merging of the known + # files. Without this, there's no conflict, so files is never + # considered. + _entry( + "a", + "+pypi_b/site-packages/a", + ["../outside.txt"], + ), + ] + + actual = build_link_map(_ctx(), entries) + env.expect.that_dict(actual).contains_exactly({ + VenvSymlinkKind.LIB: {}, + }) + +def _test_complex_namespace_packages(name): + analysis_test( + name = name, + impl = _test_complex_namespace_packages_impl, + target = "//python:none", + ) + +_tests.append(_test_complex_namespace_packages) + +def _test_complex_namespace_packages_impl(env, _): + entries = [ + _entry("a/b", "+pypi_a_b/site-packages/a/b", ["b.txt"]), + _entry("a/c", "+pypi_a_c/site-packages/a/c", ["c.txt"]), + _entry("x/y/z", "+pypi_x_y_z/site-packages/x/y/z", ["z.txt"]), + _entry("foo", "+pypi_foo/site-packages/foo", ["foo.txt"]), + _entry("foobar", "+pypi_foobar/site-packages/foobar", ["foobar.txt"]), + ] + + actual = build_link_map(_ctx(), entries) + expected_libs = { + "a/b": "+pypi_a_b/site-packages/a/b", + "a/c": "+pypi_a_c/site-packages/a/c", + "foo": "+pypi_foo/site-packages/foo", + "foobar": "+pypi_foobar/site-packages/foobar", + "x/y/z": "+pypi_x_y_z/site-packages/x/y/z", + } + env.expect.that_dict(actual[VenvSymlinkKind.LIB]).contains_exactly(expected_libs) + +def _test_empty_and_trivial_inputs(name): + analysis_test( + name = name, + impl = _test_empty_and_trivial_inputs_impl, + target = "//python:none", + ) + +_tests.append(_test_empty_and_trivial_inputs) + +def _test_empty_and_trivial_inputs_impl(env, _): + # Test with empty list of entries + actual = build_link_map(_ctx(), []) + env.expect.that_dict(actual).contains_exactly({}) + + # Test with an entry with no files + entries = [_entry("a", "+pypi_a/site-packages/a", [])] + actual = build_link_map(_ctx(), entries) + env.expect.that_dict(actual).contains_exactly({ + VenvSymlinkKind.LIB: {"a": "+pypi_a/site-packages/a"}, + }) + +def _test_multiple_venv_symlink_kinds(name): + analysis_test( + name = name, + impl = _test_multiple_venv_symlink_kinds_impl, + target = "//python:none", + ) + +_tests.append(_test_multiple_venv_symlink_kinds) + +def _test_multiple_venv_symlink_kinds_impl(env, _): + entries = [ + _entry( + "libfile", + "+pypi_lib/site-packages/libfile", + ["lib.txt"], + kind = + VenvSymlinkKind.LIB, + ), + _entry( + "binfile", + "+pypi_bin/bin/binfile", + ["bin.txt"], + kind = VenvSymlinkKind.BIN, + ), + _entry( + "includefile", + "+pypi_include/include/includefile", + ["include.h"], + kind = + VenvSymlinkKind.INCLUDE, + ), + ] + + actual = build_link_map(_ctx(), entries) + + expected_libs = { + "libfile": "+pypi_lib/site-packages/libfile", + } + env.expect.that_dict(actual[VenvSymlinkKind.LIB]).contains_exactly(expected_libs) + + expected_bins = { + "binfile": "+pypi_bin/bin/binfile", + } + env.expect.that_dict(actual[VenvSymlinkKind.BIN]).contains_exactly(expected_bins) + + expected_includes = { + "includefile": "+pypi_include/include/includefile", + } + env.expect.that_dict(actual[VenvSymlinkKind.INCLUDE]).contains_exactly(expected_includes) + + env.expect.that_dict(actual).keys().contains_exactly([ + VenvSymlinkKind.LIB, + VenvSymlinkKind.BIN, + VenvSymlinkKind.INCLUDE, + ]) + +def app_files_building_test_suite(name): + test_suite( + name = name, + tests = _tests, + ) diff --git a/tests/venv_site_packages_libs/bin.py b/tests/venv_site_packages_libs/bin.py index 7e5838d2c2..772925f00e 100644 --- a/tests/venv_site_packages_libs/bin.py +++ b/tests/venv_site_packages_libs/bin.py @@ -14,14 +14,26 @@ def setUp(self): def assert_imported_from_venv(self, module_name): module = importlib.import_module(module_name) self.assertEqual(module.__name__, module_name) + self.assertIsNotNone( + module.__file__, + f"Expected module {module_name!r} to have" + + f"__file__ set, but got None. {module=}", + ) self.assertTrue( module.__file__.startswith(self.venv), f"\n{module_name} was imported, but not from the venv.\n" + f"venv : {self.venv}\n" + f"actual: {module.__file__}", ) + return module def test_imported_from_venv(self): + m = self.assert_imported_from_venv("pkgutil_top") + self.assertEqual(m.WHOAMI, "pkgutil_top") + + m = self.assert_imported_from_venv("pkgutil_top.sub") + self.assertEqual(m.WHOAMI, "pkgutil_top.sub") + self.assert_imported_from_venv("nspkg.subnspkg.alpha") self.assert_imported_from_venv("nspkg.subnspkg.beta") self.assert_imported_from_venv("nspkg.subnspkg.gamma") diff --git a/tests/venv_site_packages_libs/pkgutil_top/BUILD.bazel b/tests/venv_site_packages_libs/pkgutil_top/BUILD.bazel new file mode 100644 index 0000000000..c805b1ad53 --- /dev/null +++ b/tests/venv_site_packages_libs/pkgutil_top/BUILD.bazel @@ -0,0 +1,10 @@ +load("//python:py_library.bzl", "py_library") + +package(default_visibility = ["//visibility:public"]) + +py_library( + name = "pkgutil_top", + srcs = glob(["site-packages/**/*.py"]), + experimental_venvs_site_packages = "//python/config_settings:venvs_site_packages", + imports = [package_name() + "/site-packages"], +) diff --git a/tests/venv_site_packages_libs/pkgutil_top/site-packages/pkgutil_top/__init__.py b/tests/venv_site_packages_libs/pkgutil_top/site-packages/pkgutil_top/__init__.py new file mode 100644 index 0000000000..e1809a325a --- /dev/null +++ b/tests/venv_site_packages_libs/pkgutil_top/site-packages/pkgutil_top/__init__.py @@ -0,0 +1,2 @@ +WHOAMI = "pkgutil_top" +__path__ = __import__("pkgutil").extend_path(__path__, __name__) diff --git a/tests/venv_site_packages_libs/pkgutil_top/site-packages/pkgutil_top/top.py b/tests/venv_site_packages_libs/pkgutil_top/site-packages/pkgutil_top/top.py new file mode 100644 index 0000000000..e69de29bb2 diff --git a/tests/venv_site_packages_libs/pkgutil_top_sub/BUILD.bazel b/tests/venv_site_packages_libs/pkgutil_top_sub/BUILD.bazel new file mode 100644 index 0000000000..9d771628a0 --- /dev/null +++ b/tests/venv_site_packages_libs/pkgutil_top_sub/BUILD.bazel @@ -0,0 +1,10 @@ +load("//python:py_library.bzl", "py_library") + +package(default_visibility = ["//visibility:public"]) + +py_library( + name = "pkgutil_top_sub", + srcs = glob(["site-packages/**/*.py"]), + experimental_venvs_site_packages = "//python/config_settings:venvs_site_packages", + imports = [package_name() + "/site-packages"], +) diff --git a/tests/venv_site_packages_libs/pkgutil_top_sub/site-packages/pkgutil_top/sub/__init__.py b/tests/venv_site_packages_libs/pkgutil_top_sub/site-packages/pkgutil_top/sub/__init__.py new file mode 100644 index 0000000000..e7fb2340ea --- /dev/null +++ b/tests/venv_site_packages_libs/pkgutil_top_sub/site-packages/pkgutil_top/sub/__init__.py @@ -0,0 +1 @@ +WHOAMI = "pkgutil_top.sub" diff --git a/tests/venv_site_packages_libs/pkgutil_top_sub/site-packages/pkgutil_top/sub/suba.py b/tests/venv_site_packages_libs/pkgutil_top_sub/site-packages/pkgutil_top/sub/suba.py new file mode 100644 index 0000000000..e69de29bb2 From df8e93fc251133499f73bcfb5a876bdd51a81a42 Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Mon, 15 Sep 2025 23:31:11 -0700 Subject: [PATCH 440/922] build(deps): bump pycparser from 2.22 to 2.23 in /tools/publish (#3271) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Bumps [pycparser](https://github.com/eliben/pycparser) from 2.22 to 2.23.
Release notes

Sourced from pycparser's releases.

release_v2.23

What's Changed

New Contributors

Full Changelog: https://github.com/eliben/pycparser/compare/release_v2.22...release_v2.23

Commits

[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=pycparser&package-manager=pip&previous-version=2.22&new-version=2.23)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) ---
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot show ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> --- tools/publish/requirements_linux.txt | 6 +++--- tools/publish/requirements_universal.txt | 2 +- 2 files changed, 4 insertions(+), 4 deletions(-) diff --git a/tools/publish/requirements_linux.txt b/tools/publish/requirements_linux.txt index 75a125e8f1..f9686dda55 100644 --- a/tools/publish/requirements_linux.txt +++ b/tools/publish/requirements_linux.txt @@ -281,9 +281,9 @@ pkginfo==1.10.0 \ --hash=sha256:5df73835398d10db79f8eecd5cd86b1f6d29317589ea70796994d49399af6297 \ --hash=sha256:889a6da2ed7ffc58ab5b900d888ddce90bce912f2d2de1dc1c26f4cb9fe65097 # via twine -pycparser==2.22 \ - --hash=sha256:491c8be9c040f5390f5bf44a5b07752bd07f56edf992381b05c701439eec10f6 \ - --hash=sha256:c3702b6d3dd8c7abc1afa565d7e63d53a1d0bd86cdc24edd75470f4de499cfcc +pycparser==2.23 \ + --hash=sha256:78816d4f24add8f10a06d6f05b4d424ad9e96cfebf68a4ddc99c65c0720d00c2 \ + --hash=sha256:e5c6e8d3fbad53479cab09ac03729e0a9faf2bee3db8208a550daf5af81a5934 # via cffi pygments==2.19.2 \ --hash=sha256:636cb2477cec7f8952536970bc533bc43743542f70392ae026374600add5b887 \ diff --git a/tools/publish/requirements_universal.txt b/tools/publish/requirements_universal.txt index 65d70a4d25..9ff4ca059a 100644 --- a/tools/publish/requirements_universal.txt +++ b/tools/publish/requirements_universal.txt @@ -281,7 +281,7 @@ pkginfo==1.10.0 \ --hash=sha256:5df73835398d10db79f8eecd5cd86b1f6d29317589ea70796994d49399af6297 \ --hash=sha256:889a6da2ed7ffc58ab5b900d888ddce90bce912f2d2de1dc1c26f4cb9fe65097 # via twine -pycparser==2.22 ; platform_python_implementation != 'PyPy' and sys_platform == 'linux' \ +pycparser==2.23 ; platform_python_implementation != 'PyPy' and sys_platform == 'linux' \ --hash=sha256:491c8be9c040f5390f5bf44a5b07752bd07f56edf992381b05c701439eec10f6 \ --hash=sha256:c3702b6d3dd8c7abc1afa565d7e63d53a1d0bd86cdc24edd75470f4de499cfcc # via cffi From ea4b0404abbe762b6a3fada53f4539b6ee8e4afd Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Mon, 15 Sep 2025 23:31:33 -0700 Subject: [PATCH 441/922] build(deps): bump cffi from 1.17.1 to 2.0.0 in /tools/publish (#3270) Bumps [cffi](https://github.com/python-cffi/cffi) from 1.17.1 to 2.0.0.
Release notes

Sourced from cffi's releases.

v2.0.0

What's Changed

  • Add Python 3.14 support.
  • Add CPython free-threaded support (3.14t+ only) - huge thanks to the folks at Quansight Labs for all the work to get this one sorted!
  • Drop Python <= 3.8 support.
  • Fix order dependency affecting nested type size calculation (#148).

Full Changelog: https://github.com/python-cffi/cffi/compare/v1.17.1...v2.0.0

v2.0.0b1

What's Changed

  • Add Python 3.14 support.
  • Add CPython free-threaded support (3.14t+ only).
  • Drop Python <= 3.8 support.
  • Fix order dependency affecting nested type size calculation (#148).

Full Changelog: https://github.com/python-cffi/cffi/compare/v1.17.1...v2.0.0b1

Commits
  • 6366c01 release 2.0.0 (#196)
  • 95c8476 2.0.0 post beta backports (#195)
  • 195cbda Release 2.0.0b1 (#183)
  • b4bbe79 fix version test to support beta
  • 7ed073d Add support for the free-threaded build (#178)
  • 67a170d Change the license from MIT to MIT-no-attribution, which is the same without ...
  • 92645ec Add Python 3.14 support/testing (#177)
  • 2b81170 doc: update test commands in Section Testing/development tips (#158)
  • 25172b8 doc: update year (#153)
  • b57a92c issue 147: force-compute nested structs before parent structs. Occurs mainly...
  • Additional commits viewable in compare view

[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=cffi&package-manager=pip&previous-version=1.17.1&new-version=2.0.0)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) ---
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot show ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> --- tools/publish/requirements_linux.txt | 153 +++++++++++++---------- tools/publish/requirements_universal.txt | 2 +- 2 files changed, 86 insertions(+), 69 deletions(-) diff --git a/tools/publish/requirements_linux.txt b/tools/publish/requirements_linux.txt index f9686dda55..80abfbf4c6 100644 --- a/tools/publish/requirements_linux.txt +++ b/tools/publish/requirements_linux.txt @@ -10,74 +10,91 @@ certifi==2025.8.3 \ --hash=sha256:e564105f78ded564e3ae7c923924435e1daa7463faeab5bb932bc53ffae63407 \ --hash=sha256:f6c12493cfb1b06ba2ff328595af9350c65d6644968e5d3a2ffd78699af217a5 # via requests -cffi==1.17.1 \ - --hash=sha256:045d61c734659cc045141be4bae381a41d89b741f795af1dd018bfb532fd0df8 \ - --hash=sha256:0984a4925a435b1da406122d4d7968dd861c1385afe3b45ba82b750f229811e2 \ - --hash=sha256:0e2b1fac190ae3ebfe37b979cc1ce69c81f4e4fe5746bb401dca63a9062cdaf1 \ - --hash=sha256:0f048dcf80db46f0098ccac01132761580d28e28bc0f78ae0d58048063317e15 \ - --hash=sha256:1257bdabf294dceb59f5e70c64a3e2f462c30c7ad68092d01bbbfb1c16b1ba36 \ - --hash=sha256:1c39c6016c32bc48dd54561950ebd6836e1670f2ae46128f67cf49e789c52824 \ - --hash=sha256:1d599671f396c4723d016dbddb72fe8e0397082b0a77a4fab8028923bec050e8 \ - --hash=sha256:28b16024becceed8c6dfbc75629e27788d8a3f9030691a1dbf9821a128b22c36 \ - --hash=sha256:2bb1a08b8008b281856e5971307cc386a8e9c5b625ac297e853d36da6efe9c17 \ - --hash=sha256:30c5e0cb5ae493c04c8b42916e52ca38079f1b235c2f8ae5f4527b963c401caf \ - --hash=sha256:31000ec67d4221a71bd3f67df918b1f88f676f1c3b535a7eb473255fdc0b83fc \ - --hash=sha256:386c8bf53c502fff58903061338ce4f4950cbdcb23e2902d86c0f722b786bbe3 \ - --hash=sha256:3edc8d958eb099c634dace3c7e16560ae474aa3803a5df240542b305d14e14ed \ - --hash=sha256:45398b671ac6d70e67da8e4224a065cec6a93541bb7aebe1b198a61b58c7b702 \ - --hash=sha256:46bf43160c1a35f7ec506d254e5c890f3c03648a4dbac12d624e4490a7046cd1 \ - --hash=sha256:4ceb10419a9adf4460ea14cfd6bc43d08701f0835e979bf821052f1805850fe8 \ - --hash=sha256:51392eae71afec0d0c8fb1a53b204dbb3bcabcb3c9b807eedf3e1e6ccf2de903 \ - --hash=sha256:5da5719280082ac6bd9aa7becb3938dc9f9cbd57fac7d2871717b1feb0902ab6 \ - --hash=sha256:610faea79c43e44c71e1ec53a554553fa22321b65fae24889706c0a84d4ad86d \ - --hash=sha256:636062ea65bd0195bc012fea9321aca499c0504409f413dc88af450b57ffd03b \ - --hash=sha256:6883e737d7d9e4899a8a695e00ec36bd4e5e4f18fabe0aca0efe0a4b44cdb13e \ - --hash=sha256:6b8b4a92e1c65048ff98cfe1f735ef8f1ceb72e3d5f0c25fdb12087a23da22be \ - --hash=sha256:6f17be4345073b0a7b8ea599688f692ac3ef23ce28e5df79c04de519dbc4912c \ - --hash=sha256:706510fe141c86a69c8ddc029c7910003a17353970cff3b904ff0686a5927683 \ - --hash=sha256:72e72408cad3d5419375fc87d289076ee319835bdfa2caad331e377589aebba9 \ - --hash=sha256:733e99bc2df47476e3848417c5a4540522f234dfd4ef3ab7fafdf555b082ec0c \ - --hash=sha256:7596d6620d3fa590f677e9ee430df2958d2d6d6de2feeae5b20e82c00b76fbf8 \ - --hash=sha256:78122be759c3f8a014ce010908ae03364d00a1f81ab5c7f4a7a5120607ea56e1 \ - --hash=sha256:805b4371bf7197c329fcb3ead37e710d1bca9da5d583f5073b799d5c5bd1eee4 \ - --hash=sha256:85a950a4ac9c359340d5963966e3e0a94a676bd6245a4b55bc43949eee26a655 \ - --hash=sha256:8f2cdc858323644ab277e9bb925ad72ae0e67f69e804f4898c070998d50b1a67 \ - --hash=sha256:9755e4345d1ec879e3849e62222a18c7174d65a6a92d5b346b1863912168b595 \ - --hash=sha256:98e3969bcff97cae1b2def8ba499ea3d6f31ddfdb7635374834cf89a1a08ecf0 \ - --hash=sha256:a08d7e755f8ed21095a310a693525137cfe756ce62d066e53f502a83dc550f65 \ - --hash=sha256:a1ed2dd2972641495a3ec98445e09766f077aee98a1c896dcb4ad0d303628e41 \ - --hash=sha256:a24ed04c8ffd54b0729c07cee15a81d964e6fee0e3d4d342a27b020d22959dc6 \ - --hash=sha256:a45e3c6913c5b87b3ff120dcdc03f6131fa0065027d0ed7ee6190736a74cd401 \ - --hash=sha256:a9b15d491f3ad5d692e11f6b71f7857e7835eb677955c00cc0aefcd0669adaf6 \ - --hash=sha256:ad9413ccdeda48c5afdae7e4fa2192157e991ff761e7ab8fdd8926f40b160cc3 \ - --hash=sha256:b2ab587605f4ba0bf81dc0cb08a41bd1c0a5906bd59243d56bad7668a6fc6c16 \ - --hash=sha256:b62ce867176a75d03a665bad002af8e6d54644fad99a3c70905c543130e39d93 \ - --hash=sha256:c03e868a0b3bc35839ba98e74211ed2b05d2119be4e8a0f224fba9384f1fe02e \ - --hash=sha256:c59d6e989d07460165cc5ad3c61f9fd8f1b4796eacbd81cee78957842b834af4 \ - --hash=sha256:c7eac2ef9b63c79431bc4b25f1cd649d7f061a28808cbc6c47b534bd789ef964 \ - --hash=sha256:c9c3d058ebabb74db66e431095118094d06abf53284d9c81f27300d0e0d8bc7c \ - --hash=sha256:ca74b8dbe6e8e8263c0ffd60277de77dcee6c837a3d0881d8c1ead7268c9e576 \ - --hash=sha256:caaf0640ef5f5517f49bc275eca1406b0ffa6aa184892812030f04c2abf589a0 \ - --hash=sha256:cdf5ce3acdfd1661132f2a9c19cac174758dc2352bfe37d98aa7512c6b7178b3 \ - --hash=sha256:d016c76bdd850f3c626af19b0542c9677ba156e4ee4fccfdd7848803533ef662 \ - --hash=sha256:d01b12eeeb4427d3110de311e1774046ad344f5b1a7403101878976ecd7a10f3 \ - --hash=sha256:d63afe322132c194cf832bfec0dc69a99fb9bb6bbd550f161a49e9e855cc78ff \ - --hash=sha256:da95af8214998d77a98cc14e3a3bd00aa191526343078b530ceb0bd710fb48a5 \ - --hash=sha256:dd398dbc6773384a17fe0d3e7eeb8d1a21c2200473ee6806bb5e6a8e62bb73dd \ - --hash=sha256:de2ea4b5833625383e464549fec1bc395c1bdeeb5f25c4a3a82b5a8c756ec22f \ - --hash=sha256:de55b766c7aa2e2a3092c51e0483d700341182f08e67c63630d5b6f200bb28e5 \ - --hash=sha256:df8b1c11f177bc2313ec4b2d46baec87a5f3e71fc8b45dab2ee7cae86d9aba14 \ - --hash=sha256:e03eab0a8677fa80d646b5ddece1cbeaf556c313dcfac435ba11f107ba117b5d \ - --hash=sha256:e221cf152cff04059d011ee126477f0d9588303eb57e88923578ace7baad17f9 \ - --hash=sha256:e31ae45bc2e29f6b2abd0de1cc3b9d5205aa847cafaecb8af1476a609a2f6eb7 \ - --hash=sha256:edae79245293e15384b51f88b00613ba9f7198016a5948b5dddf4917d4d26382 \ - --hash=sha256:f1e22e8c4419538cb197e4dd60acc919d7696e5ef98ee4da4e01d3f8cfa4cc5a \ - --hash=sha256:f3a2b4222ce6b60e2e8b337bb9596923045681d71e5a082783484d845390938e \ - --hash=sha256:f6a16c31041f09ead72d69f583767292f750d24913dadacf5756b966aacb3f1a \ - --hash=sha256:f75c7ab1f9e4aca5414ed4d8e5c0e303a34f4421f8a0d47a4d019ceff0ab6af4 \ - --hash=sha256:f79fc4fc25f1c8698ff97788206bb3c2598949bfe0fef03d299eb1b5356ada99 \ - --hash=sha256:f7f5baafcc48261359e14bcd6d9bff6d4b28d9103847c9e136694cb0501aef87 \ - --hash=sha256:fc48c783f9c87e60831201f2cce7f3b2e4846bf4d8728eabe54d60700b318a0b +cffi==2.0.0 \ + --hash=sha256:00bdf7acc5f795150faa6957054fbbca2439db2f775ce831222b66f192f03beb \ + --hash=sha256:07b271772c100085dd28b74fa0cd81c8fb1a3ba18b21e03d7c27f3436a10606b \ + --hash=sha256:087067fa8953339c723661eda6b54bc98c5625757ea62e95eb4898ad5e776e9f \ + --hash=sha256:0a1527a803f0a659de1af2e1fd700213caba79377e27e4693648c2923da066f9 \ + --hash=sha256:0cf2d91ecc3fcc0625c2c530fe004f82c110405f101548512cce44322fa8ac44 \ + --hash=sha256:0f6084a0ea23d05d20c3edcda20c3d006f9b6f3fefeac38f59262e10cef47ee2 \ + --hash=sha256:12873ca6cb9b0f0d3a0da705d6086fe911591737a59f28b7936bdfed27c0d47c \ + --hash=sha256:19f705ada2530c1167abacb171925dd886168931e0a7b78f5bffcae5c6b5be75 \ + --hash=sha256:1cd13c99ce269b3ed80b417dcd591415d3372bcac067009b6e0f59c7d4015e65 \ + --hash=sha256:1e3a615586f05fc4065a8b22b8152f0c1b00cdbc60596d187c2a74f9e3036e4e \ + --hash=sha256:1f72fb8906754ac8a2cc3f9f5aaa298070652a0ffae577e0ea9bd480dc3c931a \ + --hash=sha256:1fc9ea04857caf665289b7a75923f2c6ed559b8298a1b8c49e59f7dd95c8481e \ + --hash=sha256:203a48d1fb583fc7d78a4c6655692963b860a417c0528492a6bc21f1aaefab25 \ + --hash=sha256:2081580ebb843f759b9f617314a24ed5738c51d2aee65d31e02f6f7a2b97707a \ + --hash=sha256:21d1152871b019407d8ac3985f6775c079416c282e431a4da6afe7aefd2bccbe \ + --hash=sha256:24b6f81f1983e6df8db3adc38562c83f7d4a0c36162885ec7f7b77c7dcbec97b \ + --hash=sha256:256f80b80ca3853f90c21b23ee78cd008713787b1b1e93eae9f3d6a7134abd91 \ + --hash=sha256:28a3a209b96630bca57cce802da70c266eb08c6e97e5afd61a75611ee6c64592 \ + --hash=sha256:2c8f814d84194c9ea681642fd164267891702542f028a15fc97d4674b6206187 \ + --hash=sha256:2de9a304e27f7596cd03d16f1b7c72219bd944e99cc52b84d0145aefb07cbd3c \ + --hash=sha256:38100abb9d1b1435bc4cc340bb4489635dc2f0da7456590877030c9b3d40b0c1 \ + --hash=sha256:3925dd22fa2b7699ed2617149842d2e6adde22b262fcbfada50e3d195e4b3a94 \ + --hash=sha256:3e17ed538242334bf70832644a32a7aae3d83b57567f9fd60a26257e992b79ba \ + --hash=sha256:3e837e369566884707ddaf85fc1744b47575005c0a229de3327f8f9a20f4efeb \ + --hash=sha256:3f4d46d8b35698056ec29bca21546e1551a205058ae1a181d871e278b0b28165 \ + --hash=sha256:44d1b5909021139fe36001ae048dbdde8214afa20200eda0f64c068cac5d5529 \ + --hash=sha256:45d5e886156860dc35862657e1494b9bae8dfa63bf56796f2fb56e1679fc0bca \ + --hash=sha256:4647afc2f90d1ddd33441e5b0e85b16b12ddec4fca55f0d9671fef036ecca27c \ + --hash=sha256:4671d9dd5ec934cb9a73e7ee9676f9362aba54f7f34910956b84d727b0d73fb6 \ + --hash=sha256:53f77cbe57044e88bbd5ed26ac1d0514d2acf0591dd6bb02a3ae37f76811b80c \ + --hash=sha256:5eda85d6d1879e692d546a078b44251cdd08dd1cfb98dfb77b670c97cee49ea0 \ + --hash=sha256:5fed36fccc0612a53f1d4d9a816b50a36702c28a2aa880cb8a122b3466638743 \ + --hash=sha256:61d028e90346df14fedc3d1e5441df818d095f3b87d286825dfcbd6459b7ef63 \ + --hash=sha256:66f011380d0e49ed280c789fbd08ff0d40968ee7b665575489afa95c98196ab5 \ + --hash=sha256:6824f87845e3396029f3820c206e459ccc91760e8fa24422f8b0c3d1731cbec5 \ + --hash=sha256:6c6c373cfc5c83a975506110d17457138c8c63016b563cc9ed6e056a82f13ce4 \ + --hash=sha256:6d02d6655b0e54f54c4ef0b94eb6be0607b70853c45ce98bd278dc7de718be5d \ + --hash=sha256:6d50360be4546678fc1b79ffe7a66265e28667840010348dd69a314145807a1b \ + --hash=sha256:730cacb21e1bdff3ce90babf007d0a0917cc3e6492f336c2f0134101e0944f93 \ + --hash=sha256:737fe7d37e1a1bffe70bd5754ea763a62a066dc5913ca57e957824b72a85e205 \ + --hash=sha256:74a03b9698e198d47562765773b4a8309919089150a0bb17d829ad7b44b60d27 \ + --hash=sha256:7553fb2090d71822f02c629afe6042c299edf91ba1bf94951165613553984512 \ + --hash=sha256:7a66c7204d8869299919db4d5069a82f1561581af12b11b3c9f48c584eb8743d \ + --hash=sha256:7cc09976e8b56f8cebd752f7113ad07752461f48a58cbba644139015ac24954c \ + --hash=sha256:81afed14892743bbe14dacb9e36d9e0e504cd204e0b165062c488942b9718037 \ + --hash=sha256:8941aaadaf67246224cee8c3803777eed332a19d909b47e29c9842ef1e79ac26 \ + --hash=sha256:89472c9762729b5ae1ad974b777416bfda4ac5642423fa93bd57a09204712322 \ + --hash=sha256:8ea985900c5c95ce9db1745f7933eeef5d314f0565b27625d9a10ec9881e1bfb \ + --hash=sha256:8eca2a813c1cb7ad4fb74d368c2ffbbb4789d377ee5bb8df98373c2cc0dee76c \ + --hash=sha256:92b68146a71df78564e4ef48af17551a5ddd142e5190cdf2c5624d0c3ff5b2e8 \ + --hash=sha256:9332088d75dc3241c702d852d4671613136d90fa6881da7d770a483fd05248b4 \ + --hash=sha256:94698a9c5f91f9d138526b48fe26a199609544591f859c870d477351dc7b2414 \ + --hash=sha256:9a67fc9e8eb39039280526379fb3a70023d77caec1852002b4da7e8b270c4dd9 \ + --hash=sha256:9de40a7b0323d889cf8d23d1ef214f565ab154443c42737dfe52ff82cf857664 \ + --hash=sha256:a05d0c237b3349096d3981b727493e22147f934b20f6f125a3eba8f994bec4a9 \ + --hash=sha256:afb8db5439b81cf9c9d0c80404b60c3cc9c3add93e114dcae767f1477cb53775 \ + --hash=sha256:b18a3ed7d5b3bd8d9ef7a8cb226502c6bf8308df1525e1cc676c3680e7176739 \ + --hash=sha256:b1e74d11748e7e98e2f426ab176d4ed720a64412b6a15054378afdb71e0f37dc \ + --hash=sha256:b21e08af67b8a103c71a250401c78d5e0893beff75e28c53c98f4de42f774062 \ + --hash=sha256:b4c854ef3adc177950a8dfc81a86f5115d2abd545751a304c5bcf2c2c7283cfe \ + --hash=sha256:b882b3df248017dba09d6b16defe9b5c407fe32fc7c65a9c69798e6175601be9 \ + --hash=sha256:baf5215e0ab74c16e2dd324e8ec067ef59e41125d3eade2b863d294fd5035c92 \ + --hash=sha256:c649e3a33450ec82378822b3dad03cc228b8f5963c0c12fc3b1e0ab940f768a5 \ + --hash=sha256:c654de545946e0db659b3400168c9ad31b5d29593291482c43e3564effbcee13 \ + --hash=sha256:c6638687455baf640e37344fe26d37c404db8b80d037c3d29f58fe8d1c3b194d \ + --hash=sha256:c8d3b5532fc71b7a77c09192b4a5a200ea992702734a2e9279a37f2478236f26 \ + --hash=sha256:cb527a79772e5ef98fb1d700678fe031e353e765d1ca2d409c92263c6d43e09f \ + --hash=sha256:cf364028c016c03078a23b503f02058f1814320a56ad535686f90565636a9495 \ + --hash=sha256:d48a880098c96020b02d5a1f7d9251308510ce8858940e6fa99ece33f610838b \ + --hash=sha256:d68b6cef7827e8641e8ef16f4494edda8b36104d79773a334beaa1e3521430f6 \ + --hash=sha256:d9b29c1f0ae438d5ee9acb31cadee00a58c46cc9c0b2f9038c6b0b3470877a8c \ + --hash=sha256:d9b97165e8aed9272a6bb17c01e3cc5871a594a446ebedc996e2397a1c1ea8ef \ + --hash=sha256:da68248800ad6320861f129cd9c1bf96ca849a2771a59e0344e88681905916f5 \ + --hash=sha256:da902562c3e9c550df360bfa53c035b2f241fed6d9aef119048073680ace4a18 \ + --hash=sha256:dbd5c7a25a7cb98f5ca55d258b103a2054f859a46ae11aaf23134f9cc0d356ad \ + --hash=sha256:dd4f05f54a52fb558f1ba9f528228066954fee3ebe629fc1660d874d040ae5a3 \ + --hash=sha256:de8dad4425a6ca6e4e5e297b27b5c824ecc7581910bf9aee86cb6835e6812aa7 \ + --hash=sha256:e11e82b744887154b182fd3e7e8512418446501191994dbf9c9fc1f32cc8efd5 \ + --hash=sha256:e6e73b9e02893c764e7e8d5bb5ce277f1a009cd5243f8228f75f842bf937c534 \ + --hash=sha256:f73b96c41e3b2adedc34a7356e64c8eb96e03a3782b535e043a986276ce12a49 \ + --hash=sha256:f93fd8e5c8c0a4aa1f424d6173f14a892044054871c771f8566e4008eaa359d2 \ + --hash=sha256:fc33c5141b55ed366cfaad382df24fe7dcbc686de5be719b207bb248e3053dc5 \ + --hash=sha256:fc7de24befaeae77ba923797c7c87834c73648a05a4bde34b3b7e5588973a453 \ + --hash=sha256:fe562eb1a64e67dd297ccc4f5addea2501664954f2692b69a76449ec7913ecbf # via cryptography charset-normalizer==3.4.3 \ --hash=sha256:00237675befef519d9af72169d8604a067d92755e84fe76492fef5441db05b91 \ diff --git a/tools/publish/requirements_universal.txt b/tools/publish/requirements_universal.txt index 9ff4ca059a..9c44c89d75 100644 --- a/tools/publish/requirements_universal.txt +++ b/tools/publish/requirements_universal.txt @@ -10,7 +10,7 @@ certifi==2025.8.3 \ --hash=sha256:e564105f78ded564e3ae7c923924435e1daa7463faeab5bb932bc53ffae63407 \ --hash=sha256:f6c12493cfb1b06ba2ff328595af9350c65d6644968e5d3a2ffd78699af217a5 # via requests -cffi==1.17.1 ; platform_python_implementation != 'PyPy' and sys_platform == 'linux' \ +cffi==2.0.0 ; platform_python_implementation != 'PyPy' and sys_platform == 'linux' \ --hash=sha256:045d61c734659cc045141be4bae381a41d89b741f795af1dd018bfb532fd0df8 \ --hash=sha256:0984a4925a435b1da406122d4d7968dd861c1385afe3b45ba82b750f229811e2 \ --hash=sha256:0e2b1fac190ae3ebfe37b979cc1ce69c81f4e4fe5746bb401dca63a9062cdaf1 \ From 15b5da7d79b6eaa31f65fb771b2ac238f6254a8b Mon Sep 17 00:00:00 2001 From: Mai Hussien <70515749+mai93@users.noreply.github.com> Date: Tue, 16 Sep 2025 17:16:02 -0700 Subject: [PATCH 442/922] build: remove no-op _native_rules_allowlist (#3275) Remove `_native_rules_allowlist` attribute as it is no-op This is needed to allow removing the deprecated flag `--native_rules_allowlist` from bazel. Work towards: https://github.com/bazel-contrib/rules_python/issues/3252 --- python/private/attributes.bzl | 27 --------------------------- python/private/py_runtime_rule.bzl | 2 -- 2 files changed, 29 deletions(-) diff --git a/python/private/attributes.bzl b/python/private/attributes.bzl index 8151a30fad..6d08c3d926 100644 --- a/python/private/attributes.bzl +++ b/python/private/attributes.bzl @@ -21,12 +21,9 @@ load(":common_labels.bzl", "labels") load(":enum.bzl", "enum") load(":flags.bzl", "PrecompileFlag", "PrecompileSourceRetentionFlag") load(":py_info.bzl", "PyInfo") -load(":py_internal.bzl", "py_internal") load(":reexports.bzl", "BuiltinPyInfo") load(":rule_builders.bzl", "ruleb") -_PackageSpecificationInfo = getattr(py_internal, "PackageSpecificationInfo", None) - # Due to how the common exec_properties attribute works, rules must add exec # groups even if they don't actually use them. This is due to two interactions: # 1. Rules give an error if users pass an unsupported exec group. @@ -174,33 +171,9 @@ This is because Python has a concept of runtime resources. ), } -def _create_native_rules_allowlist_attrs(): - if py_internal: - # The fragment and name are validated when configuration_field is called - default = configuration_field( - fragment = "py", - name = "native_rules_allowlist", - ) - - # A None provider isn't allowed - providers = [_PackageSpecificationInfo] - else: - default = None - providers = [] - - return { - "_native_rules_allowlist": lambda: attrb.Label( - default = default, - providers = providers, - ), - } - -NATIVE_RULES_ALLOWLIST_ATTRS = _create_native_rules_allowlist_attrs() - # Attributes common to all rules. COMMON_ATTRS = dicts.add( DATA_ATTRS, - NATIVE_RULES_ALLOWLIST_ATTRS, # buildifier: disable=attr-licenses { # NOTE: This attribute is deprecated and slated for removal. diff --git a/python/private/py_runtime_rule.bzl b/python/private/py_runtime_rule.bzl index a511a0e1a5..ba1a390ef3 100644 --- a/python/private/py_runtime_rule.bzl +++ b/python/private/py_runtime_rule.bzl @@ -16,7 +16,6 @@ load("@bazel_skylib//lib:dicts.bzl", "dicts") load("@bazel_skylib//lib:paths.bzl", "paths") load("@bazel_skylib//rules:common_settings.bzl", "BuildSettingInfo") -load(":attributes.bzl", "NATIVE_RULES_ALLOWLIST_ATTRS") load(":common_labels.bzl", "labels") load(":flags.bzl", "FreeThreadedFlag") load(":py_internal.bzl", "py_internal") @@ -191,7 +190,6 @@ py_runtime( """, fragments = ["py"], attrs = dicts.add( - {k: v().build() for k, v in NATIVE_RULES_ALLOWLIST_ATTRS.items()}, { "abi_flags": attr.string( default = "", From ac177a6cc8840cc3f2c020d49eabc0186e25b122 Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Tue, 16 Sep 2025 19:21:44 -0700 Subject: [PATCH 443/922] build(deps): bump pkginfo from 1.10.0 to 1.12.1.2 in /tools/publish (#3229) Bumps [pkginfo](https://code.launchpad.net/~tseaver/pkginfo/trunk) from 1.10.0 to 1.12.1.2. [![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=pkginfo&package-manager=pip&previous-version=1.10.0&new-version=1.12.1.2)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) ---
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot show ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> --- tools/publish/requirements_darwin.txt | 6 +++--- tools/publish/requirements_linux.txt | 6 +++--- tools/publish/requirements_universal.txt | 6 +++--- tools/publish/requirements_windows.txt | 6 +++--- 4 files changed, 12 insertions(+), 12 deletions(-) diff --git a/tools/publish/requirements_darwin.txt b/tools/publish/requirements_darwin.txt index 05f18f99ae..bc7e1c609d 100644 --- a/tools/publish/requirements_darwin.txt +++ b/tools/publish/requirements_darwin.txt @@ -163,9 +163,9 @@ nh3==0.3.0 \ --hash=sha256:ec6cfdd2e0399cb79ba4dcffb2332b94d9696c52272ff9d48a630c5dca5e325a \ --hash=sha256:f416c35efee3e6a6c9ab7716d9e57aa0a49981be915963a82697952cba1353e1 # via readme-renderer -pkginfo==1.10.0 \ - --hash=sha256:5df73835398d10db79f8eecd5cd86b1f6d29317589ea70796994d49399af6297 \ - --hash=sha256:889a6da2ed7ffc58ab5b900d888ddce90bce912f2d2de1dc1c26f4cb9fe65097 +pkginfo==1.12.1.2 \ + --hash=sha256:5cd957824ac36f140260964eba3c6be6442a8359b8c48f4adf90210f33a04b7b \ + --hash=sha256:c783ac885519cab2c34927ccfa6bf64b5a704d7c69afaea583dd9b7afe969343 # via twine pygments==2.19.2 \ --hash=sha256:636cb2477cec7f8952536970bc533bc43743542f70392ae026374600add5b887 \ diff --git a/tools/publish/requirements_linux.txt b/tools/publish/requirements_linux.txt index 80abfbf4c6..522a5f3da2 100644 --- a/tools/publish/requirements_linux.txt +++ b/tools/publish/requirements_linux.txt @@ -294,9 +294,9 @@ nh3==0.3.0 \ --hash=sha256:ec6cfdd2e0399cb79ba4dcffb2332b94d9696c52272ff9d48a630c5dca5e325a \ --hash=sha256:f416c35efee3e6a6c9ab7716d9e57aa0a49981be915963a82697952cba1353e1 # via readme-renderer -pkginfo==1.10.0 \ - --hash=sha256:5df73835398d10db79f8eecd5cd86b1f6d29317589ea70796994d49399af6297 \ - --hash=sha256:889a6da2ed7ffc58ab5b900d888ddce90bce912f2d2de1dc1c26f4cb9fe65097 +pkginfo==1.12.1.2 \ + --hash=sha256:5cd957824ac36f140260964eba3c6be6442a8359b8c48f4adf90210f33a04b7b \ + --hash=sha256:c783ac885519cab2c34927ccfa6bf64b5a704d7c69afaea583dd9b7afe969343 # via twine pycparser==2.23 \ --hash=sha256:78816d4f24add8f10a06d6f05b4d424ad9e96cfebf68a4ddc99c65c0720d00c2 \ diff --git a/tools/publish/requirements_universal.txt b/tools/publish/requirements_universal.txt index 9c44c89d75..e8d1c747f6 100644 --- a/tools/publish/requirements_universal.txt +++ b/tools/publish/requirements_universal.txt @@ -277,9 +277,9 @@ nh3==0.3.0 \ --hash=sha256:ec6cfdd2e0399cb79ba4dcffb2332b94d9696c52272ff9d48a630c5dca5e325a \ --hash=sha256:f416c35efee3e6a6c9ab7716d9e57aa0a49981be915963a82697952cba1353e1 # via readme-renderer -pkginfo==1.10.0 \ - --hash=sha256:5df73835398d10db79f8eecd5cd86b1f6d29317589ea70796994d49399af6297 \ - --hash=sha256:889a6da2ed7ffc58ab5b900d888ddce90bce912f2d2de1dc1c26f4cb9fe65097 +pkginfo==1.12.1.2 \ + --hash=sha256:5cd957824ac36f140260964eba3c6be6442a8359b8c48f4adf90210f33a04b7b \ + --hash=sha256:c783ac885519cab2c34927ccfa6bf64b5a704d7c69afaea583dd9b7afe969343 # via twine pycparser==2.23 ; platform_python_implementation != 'PyPy' and sys_platform == 'linux' \ --hash=sha256:491c8be9c040f5390f5bf44a5b07752bd07f56edf992381b05c701439eec10f6 \ diff --git a/tools/publish/requirements_windows.txt b/tools/publish/requirements_windows.txt index 6dd7ffe978..38d854eb5c 100644 --- a/tools/publish/requirements_windows.txt +++ b/tools/publish/requirements_windows.txt @@ -163,9 +163,9 @@ nh3==0.3.0 \ --hash=sha256:ec6cfdd2e0399cb79ba4dcffb2332b94d9696c52272ff9d48a630c5dca5e325a \ --hash=sha256:f416c35efee3e6a6c9ab7716d9e57aa0a49981be915963a82697952cba1353e1 # via readme-renderer -pkginfo==1.10.0 \ - --hash=sha256:5df73835398d10db79f8eecd5cd86b1f6d29317589ea70796994d49399af6297 \ - --hash=sha256:889a6da2ed7ffc58ab5b900d888ddce90bce912f2d2de1dc1c26f4cb9fe65097 +pkginfo==1.12.1.2 \ + --hash=sha256:5cd957824ac36f140260964eba3c6be6442a8359b8c48f4adf90210f33a04b7b \ + --hash=sha256:c783ac885519cab2c34927ccfa6bf64b5a704d7c69afaea583dd9b7afe969343 # via twine pygments==2.19.2 \ --hash=sha256:636cb2477cec7f8952536970bc533bc43743542f70392ae026374600add5b887 \ From 0cd9bfafc7ebf4cde2e4f84cab6ae756753f9660 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Wed, 17 Sep 2025 22:27:29 -0700 Subject: [PATCH 444/922] tests: make py_cc_toolchain test of headers/includes work with Bazel 9 (#3276) Bazel 9 has a small change in cc_library behavior: the includes attribute populates the `.includes` instead of `.system_includes`. This is OK for rules_python, since both result in the includes being added as system include paths. To fix, change the test to look at both; as long as the include paths are in one, then its OK. Fixes https://github.com/bazel-contrib/rules_python/issues/3239 --- .../current_py_cc_headers_tests.bzl | 30 ++++++++++++------- .../py_cc_toolchain/py_cc_toolchain_tests.bzl | 22 +++++++------- tests/support/cc_info_subject.bzl | 2 ++ tests/support/cc_toolchains/BUILD.bazel | 20 ++++++++----- tests/support/cc_toolchains/py_header.h | 0 .../cc_toolchains/py_include/py_include.h | 0 6 files changed, 45 insertions(+), 29 deletions(-) create mode 100644 tests/support/cc_toolchains/py_header.h create mode 100644 tests/support/cc_toolchains/py_include/py_include.h diff --git a/tests/cc/current_py_cc_headers/current_py_cc_headers_tests.bzl b/tests/cc/current_py_cc_headers/current_py_cc_headers_tests.bzl index d07d08ac61..818f7ff092 100644 --- a/tests/cc/current_py_cc_headers/current_py_cc_headers_tests.bzl +++ b/tests/cc/current_py_cc_headers/current_py_cc_headers_tests.bzl @@ -31,9 +31,11 @@ def _test_current_toolchain_headers(name): "//command_line_option:extra_toolchains": [CC_TOOLCHAIN], }, attrs = { - "header": attr.label( - default = "//tests/support/cc_toolchains:fake_header.h", - allow_single_file = True, + "header_files": attr.label_list( + default = [ + "//tests/support/cc_toolchains:py_header_files", + ], + allow_files = True, ), }, ) @@ -44,17 +46,23 @@ def _test_current_toolchain_headers_impl(env, target): CcInfo, factory = cc_info_subject, ).compilation_context() - compilation_context.direct_headers().contains_exactly([ - env.ctx.file.header, - ]) - compilation_context.direct_public_headers().contains_exactly([ - env.ctx.file.header, - ]) + compilation_context.direct_headers().contains_exactly( + env.ctx.files.header_files, + ) + compilation_context.direct_public_headers().contains_exactly( + env.ctx.files.header_files, + ) + + # NOTE: Bazel 8 and lower put cc_library.includes into `.system_includes`, + # while Bazel 9 put it in `.includes`. Both result in the includes being + # added as system includes, so either is acceptable for the expected + # `#include ` to work. + includes = compilation_context.actual.includes.to_list() + compilation_context.actual.system_includes.to_list() # NOTE: The include dir gets added twice, once for the source path, # and once for the config-specific path. - compilation_context.system_includes().contains_at_least_predicates([ - matching.str_matches("*/fake_include"), + env.expect.that_collection(includes).contains_at_least_predicates([ + matching.str_matches("*/py_include"), ]) # Check that the forward DefaultInfo looks correct diff --git a/tests/cc/py_cc_toolchain/py_cc_toolchain_tests.bzl b/tests/cc/py_cc_toolchain/py_cc_toolchain_tests.bzl index 0419a04a45..ba8e089cbb 100644 --- a/tests/cc/py_cc_toolchain/py_cc_toolchain_tests.bzl +++ b/tests/cc/py_cc_toolchain/py_cc_toolchain_tests.bzl @@ -28,9 +28,9 @@ def _test_py_cc_toolchain(name): impl = _test_py_cc_toolchain_impl, target = "//tests/support/cc_toolchains:fake_py_cc_toolchain_impl", attrs = { - "header": attr.label( - default = "//tests/support/cc_toolchains:fake_header.h", - allow_single_file = True, + "header_files": attr.label_list( + default = ["//tests/support/cc_toolchains:py_header_files"], + allow_files = True, ), }, ) @@ -50,17 +50,17 @@ def _test_py_cc_toolchain_impl(env, target): cc_info = headers_providers.get("CcInfo", factory = cc_info_subject) compilation_context = cc_info.compilation_context() - compilation_context.direct_headers().contains_exactly([ - env.ctx.file.header, - ]) - compilation_context.direct_public_headers().contains_exactly([ - env.ctx.file.header, - ]) + compilation_context.direct_headers().contains_exactly( + env.ctx.files.header_files, + ) + compilation_context.direct_public_headers().contains_exactly( + env.ctx.files.header_files, + ) # NOTE: The include dir gets added twice, once for the source path, # and once for the config-specific path, but we don't care about that. compilation_context.system_includes().contains_at_least_predicates([ - matching.str_matches("*/fake_include"), + matching.str_matches("*/py_include"), ]) default_info = headers_providers.get("DefaultInfo", factory = subjects.default_info) @@ -87,7 +87,7 @@ def _test_libs_optional(name): py_cc_toolchain( name = name + "_subject", libs = None, - headers = "//tests/support/cc_toolchains:fake_headers", + headers = "//tests/support/cc_toolchains:py_headers", python_version = "4.5", ) analysis_test( diff --git a/tests/support/cc_info_subject.bzl b/tests/support/cc_info_subject.bzl index e33ccb8262..433a3da72d 100644 --- a/tests/support/cc_info_subject.bzl +++ b/tests/support/cc_info_subject.bzl @@ -81,6 +81,7 @@ def _compilation_context_subject_new(info, *, meta): # buildifier: disable=uninitialized public = struct( + actual = info, # go/keep-sorted start direct_headers = lambda *a, **k: _compilation_context_subject_direct_headers(self, *a, **k), direct_public_headers = lambda *a, **k: _compilation_context_subject_direct_public_headers(self, *a, **k), @@ -156,6 +157,7 @@ def _linking_context_subject_new(info, meta): # buildifier: disable=uninitialized public = struct( + actual = info, # go/keep-sorted start linker_inputs = lambda *a, **k: _linking_context_subject_linker_inputs(self, *a, **k), # go/keep-sorted end diff --git a/tests/support/cc_toolchains/BUILD.bazel b/tests/support/cc_toolchains/BUILD.bazel index f6e6654d09..afa88dc836 100644 --- a/tests/support/cc_toolchains/BUILD.bazel +++ b/tests/support/cc_toolchains/BUILD.bazel @@ -12,6 +12,7 @@ # See the License for the specific language governing permissions and # limitations under the License. +load("@rules_cc//cc:cc_library.bzl", "cc_library") load("@rules_cc//cc/toolchains:cc_toolchain.bzl", "cc_toolchain") load("@rules_cc//cc/toolchains:cc_toolchain_suite.bzl", "cc_toolchain_suite") load("@rules_testing//lib:util.bzl", "PREVENT_IMPLICIT_BUILDING_TAGS") @@ -20,7 +21,14 @@ load(":fake_cc_toolchain_config.bzl", "fake_cc_toolchain_config") package(default_visibility = ["//:__subpackages__"]) -exports_files(["fake_header.h"]) +# Factored out for testing +filegroup( + name = "py_header_files", + srcs = [ + "py_header.h", + "py_include/py_include.h", + ], +) filegroup( name = "libpython", @@ -37,22 +45,20 @@ toolchain( py_cc_toolchain( name = "fake_py_cc_toolchain_impl", - headers = ":fake_headers", + headers = ":py_headers", libs = ":fake_libs", python_version = "3.999", tags = PREVENT_IMPLICIT_BUILDING_TAGS, ) -# buildifier: disable=native-cc cc_library( - name = "fake_headers", - hdrs = ["fake_header.h"], + name = "py_headers", + hdrs = [":py_header_files"], data = ["data.txt"], - includes = ["fake_include"], + includes = ["py_include"], tags = PREVENT_IMPLICIT_BUILDING_TAGS, ) -# buildifier: disable=native-cc cc_library( name = "fake_libs", srcs = ["libpython3.so"], diff --git a/tests/support/cc_toolchains/py_header.h b/tests/support/cc_toolchains/py_header.h new file mode 100644 index 0000000000..e69de29bb2 diff --git a/tests/support/cc_toolchains/py_include/py_include.h b/tests/support/cc_toolchains/py_include/py_include.h new file mode 100644 index 0000000000..e69de29bb2 From 5c6acb0f03a1d2d536281e6041c71eb6b75962a0 Mon Sep 17 00:00:00 2001 From: Alex Eagle Date: Sat, 20 Sep 2025 04:27:53 -0700 Subject: [PATCH 445/922] chore(docs): remove duplicate bzlmod guidance (#3278) It's out of date with https://github.com/bazel-contrib/rules_python/blob/main/BZLMOD_SUPPORT.md which says the feature is GA now --- README.md | 3 --- 1 file changed, 3 deletions(-) diff --git a/README.md b/README.md index d890d702d6..a7399fff9c 100644 --- a/README.md +++ b/README.md @@ -25,7 +25,4 @@ For detailed documentation, see ## Bzlmod support -- Status: Beta -- Full Feature Parity: No - See [Bzlmod support](BZLMOD_SUPPORT.md) for more details. From df9e94ece96336a007a879285ca37eef3f548e63 Mon Sep 17 00:00:00 2001 From: Greg Date: Sat, 20 Sep 2025 15:03:19 -0400 Subject: [PATCH 446/922] chore: remove non-toolchain runtime resolution logic. (#3280) Finding the Python runtime via toolchain resolution has been enabled by default for many years. Toolchains are the preferred mechanism for specifying Python runtime info. Usage of the non-toolchain ways should be approximately zero, and they'd be considered unsupported anyways. Hence this is not considered a breaking change. This PR removes support for - "If toolchains disabled" logic (`--incompatible_use_python_toolchains`) - `configuration_field` on `python_top` (`--python_top` flag) - `_py_interpreter` attribute (plumbing for python_top flag) Work towards https://github.com/bazel-contrib/rules_python/issues/3252. --------- Co-authored-by: Richard Levasseur Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> --- python/private/py_executable.bzl | 68 +++++++++++--------------------- 1 file changed, 23 insertions(+), 45 deletions(-) diff --git a/python/private/py_executable.bzl b/python/private/py_executable.bzl index bef5934729..5993a4f003 100644 --- a/python/private/py_executable.bzl +++ b/python/private/py_executable.bzl @@ -59,7 +59,7 @@ load(":py_cc_link_params_info.bzl", "PyCcLinkParamsInfo") load(":py_executable_info.bzl", "PyExecutableInfo") load(":py_info.bzl", "PyInfo", "VenvSymlinkKind") load(":py_internal.bzl", "py_internal") -load(":py_runtime_info.bzl", "DEFAULT_STUB_SHEBANG", "PyRuntimeInfo") +load(":py_runtime_info.bzl", "DEFAULT_STUB_SHEBANG") load(":reexports.bzl", "BuiltinPyInfo", "BuiltinPyRuntimeInfo") load(":rule_builders.bzl", "ruleb") load(":toolchain_types.bzl", "EXEC_TOOLS_TOOLCHAIN_TYPE", "TARGET_TOOLCHAIN_TYPE", TOOLCHAIN_TYPE = "TARGET_TOOLCHAIN_TYPE") @@ -203,15 +203,6 @@ accepting arbitrary Python versions. # empty target for other platforms. default = "//tools/launcher:launcher", ), - "_py_interpreter": lambda: attrb.Label( - # The configuration_field args are validated when called; - # we use the precense of py_internal to indicate this Bazel - # build has that fragment and name. - default = configuration_field( - fragment = "bazel_py", - name = "python_top", - ) if py_internal else None, - ), # TODO: This appears to be vestigial. It's only added because # GraphlessQueryTest.testLabelsOperator relies on it to test for # query behavior of implicit dependencies. @@ -1202,41 +1193,28 @@ def _maybe_get_runtime_from_ctx(ctx): Returns: 2-tuple of toolchain_runtime, effective_runtime """ - if ctx.fragments.py.use_toolchains: - toolchain = ctx.toolchains[TOOLCHAIN_TYPE] - - if not hasattr(toolchain, "py3_runtime"): - fail("Python toolchain field 'py3_runtime' is missing") - if not toolchain.py3_runtime: - fail("Python toolchain missing py3_runtime") - py3_runtime = toolchain.py3_runtime - - # Hack around the fact that the autodetecting Python toolchain, which is - # automatically registered, does not yet support Windows. In this case, - # we want to return null so that _get_interpreter_path falls back on - # --python_path. See tools/python/toolchain.bzl. - # TODO(#7844): Remove this hack when the autodetecting toolchain has a - # Windows implementation. - if py3_runtime.interpreter_path == "/_magic_pyruntime_sentinel_do_not_use": - return None, None - - if py3_runtime.python_version != "PY3": - fail("Python toolchain py3_runtime must be python_version=PY3, got {}".format( - py3_runtime.python_version, - )) - toolchain_runtime = toolchain.py3_runtime - effective_runtime = toolchain_runtime - else: - toolchain_runtime = None - attr_target = ctx.attr._py_interpreter - - # In Bazel, --python_top is null by default. - if attr_target and PyRuntimeInfo in attr_target: - effective_runtime = attr_target[PyRuntimeInfo] - else: - return None, None - - return toolchain_runtime, effective_runtime + toolchain = ctx.toolchains[TOOLCHAIN_TYPE] + + if not hasattr(toolchain, "py3_runtime"): + fail("Python toolchain field 'py3_runtime' is missing") + if not toolchain.py3_runtime: + fail("Python toolchain missing py3_runtime") + py3_runtime = toolchain.py3_runtime + + # Hack around the fact that the autodetecting Python toolchain, which is + # automatically registered, does not yet support Windows. In this case, + # we want to return null so that _get_interpreter_path falls back on + # --python_path. See tools/python/toolchain.bzl. + # TODO(#7844): Remove this hack when the autodetecting toolchain has a + # Windows implementation. + if py3_runtime.interpreter_path == "/_magic_pyruntime_sentinel_do_not_use": + return None, None + + if py3_runtime.python_version != "PY3": + fail("Python toolchain py3_runtime must be python_version=PY3, got {}".format( + py3_runtime.python_version, + )) + return py3_runtime, py3_runtime def _get_base_runfiles_for_binary( ctx, From 3b48bf81263526f28a41f07176859397033c2160 Mon Sep 17 00:00:00 2001 From: Ignas Anikevicius <240938+aignas@users.noreply.github.com> Date: Sun, 21 Sep 2025 22:18:44 +0900 Subject: [PATCH 447/922] docs: move 1.6.2 related changelog Fixes #3273 --- CHANGELOG.md | 22 +++++++++++++++------- 1 file changed, 15 insertions(+), 7 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 382096f826..7e5598afcf 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -79,13 +79,6 @@ END_UNRELEASED_TEMPLATE length errors due to too long environment variables. * (bootstrap) {obj}`--bootstrap_impl=script` now supports the `-S` interpreter setting. -* (pypi) We now use the Minimal Version Selection (MVS) algorithm to select - the right wheel when there are multiple wheels for the target platform - (e.g. `musllinux_1_1_x86_64` and `musllinux_1_2_x86_64`). If the user - wants to set the minimum version for the selection algorithm, use the - {attr}`pip.defaults.whl_platform_tags` attribute to configure that. If - `musllinux_*_x86_64` is specified, we will chose the lowest available - wheel version. Fixes [#3250](https://github.com/bazel-contrib/rules_python/issues/3250). * (venvs) {obj}`--vens_site_packages=yes` no longer errors when packages with overlapping files or directories are used together. ([#3204](https://github.com/bazel-contrib/rules_python/issues/3204)). @@ -104,6 +97,21 @@ END_UNRELEASED_TEMPLATE {ref}`common-deps-with-multiple-pypi-versions` guide on using common dependencies with multiple PyPI versions` for an example. +{#v1-6-2} +## [1.6.2] - 2025-09-21 + +[1.6.2]: https://github.com/bazel-contrib/rules_python/releases/tag/1.6.2 + +{#v1-6-2-fixed} +### Fixed + +* (pypi) We now use the Minimal Version Selection (MVS) algorithm to select + the right wheel when there are multiple wheels for the target platform + (e.g. `musllinux_1_1_x86_64` and `musllinux_1_2_x86_64`). If the user + wants to set the minimum version for the selection algorithm, use the + {attr}`pip.defaults.whl_platform_tags` attribute to configure that. If + `musllinux_*_x86_64` is specified, we will chose the lowest available + wheel version. Fixes [#3250](https://github.com/bazel-contrib/rules_python/issues/3250). {#v1-6-0} ## [1.6.0] - 2025-08-23 From cdfa93ea7994cfcb1f45e579efb645fb65b84cd4 Mon Sep 17 00:00:00 2001 From: Nicholas Junge Date: Sun, 21 Sep 2025 21:27:56 +0200 Subject: [PATCH 448/922] feat(toolchains): ABI3 Python headers target (#3274) Until now, we silently link extensions with both stable and unstable ABI libs, with the latter taking precedence in symbol resolution, because it appears first in the linker command AND, crucially, contains all CPython symbols present in the stable ABI library, thus overriding them. This has the effect that stable ABI extensions on Windows are usable only with the Python distribution that they were built on. To fix, a separate ABI3 header target is introduced, and should be used for C++ extensions on Windows if stable ABI builds are requested. Idea as formulated by `@dgrunwald-qt` in https://github.com/nicholasjng/nanobind-bazel/issues/72#issuecomment-3249959583. This is motivated by https://github.com/nicholasjng/nanobind-bazel/issues/72. This change shifts stable ABI selection on Windows to the extension developer, where it has arguably always been (they had to set the `Py_LIMITED_API` macro). An upside of this approach is that with a separate target, the question "stable ABI or not" can be decided on an extension-by-extension basis, giving maximum flexibility to developers. This should not influence the wheel platform target, because a wheel is marked ABI3 if and only if all of its extensions are marked as ABI3. --------- Co-authored-by: Richard Levasseur Co-authored-by: Richard Levasseur --- AGENTS.md | 20 +++++- CHANGELOG.md | 9 +++ docs/api/rules_python/python/cc/index.md | 27 +++++++- docs/howto/python-headers.md | 32 ++++++++-- docs/pyproject.toml | 3 +- docs/requirements.txt | 10 ++- python/cc/BUILD.bazel | 13 +++- python/features.bzl | 11 ++++ python/private/BUILD.bazel | 2 + python/private/cc/BUILD.bazel | 20 ++++++ python/private/current_py_cc_headers.bzl | 47 ++++++++++++-- .../private/hermetic_runtime_repo_setup.bzl | 12 +++- python/private/local_runtime_repo_setup.bzl | 17 +++-- python/private/py_cc_toolchain_info.bzl | 31 ++++++++- python/private/py_cc_toolchain_rule.bzl | 28 +++++++- python/private/runtime_env_toolchain.bzl | 5 +- python/private/visibility.bzl | 7 ++ tests/cc/current_py_cc_headers/BUILD.bazel | 24 +++++++ .../abi3_headers_linkage_test.py | 28 ++++++++ tests/cc/current_py_cc_headers/bin_abi3.cc | 17 +++++ .../current_py_cc_headers_tests.bzl | 64 +++++++++++++------ tests/cc/current_py_cc_libs/BUILD.bazel | 8 +-- .../py_cc_toolchain/py_cc_toolchain_tests.bzl | 43 ++++++++++++- tests/support/cc_toolchains/BUILD.bazel | 23 ++++++- tests/support/cc_toolchains/py_abi3_header.h | 0 .../support/py_cc_toolchain_info_subject.bzl | 10 +++ 26 files changed, 456 insertions(+), 55 deletions(-) create mode 100644 python/private/cc/BUILD.bazel create mode 100644 python/private/visibility.bzl create mode 100644 tests/cc/current_py_cc_headers/abi3_headers_linkage_test.py create mode 100644 tests/cc/current_py_cc_headers/bin_abi3.cc create mode 100644 tests/support/cc_toolchains/py_abi3_header.h diff --git a/AGENTS.md b/AGENTS.md index 3d5219c2bc..671b85c6bd 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -19,6 +19,14 @@ using a grandoise title. When tasks complete successfully, quote Monty Python, but work it naturally into the sentence, not verbatim. +When adding `{versionadded}` or `{versionchanged}` sections, add them add the +end of the documentation text. + +### Starlark style + +For doc strings, using triple quoted strings when the doc string is more than +three lines. Do not use a trailing backslack (`\`) for the opening triple-quote. + ### bzl_library targets for bzl source files * A `bzl_library` target should be defined for every `.bzl` file outside @@ -78,7 +86,17 @@ When modifying documentation * Act as an expert in tech writing, Sphinx, MyST, and markdown. * Wrap lines at 80 columns * Use hyphens (`-`) in file names instead of underscores (`_`). - + * In Sphinx MyST markup, outer directives must have more colons than inner + directives. For example: + ``` + ::::{outerdirective} + outer text + + :::{innertdirective} + inner text + ::: + :::: + ``` Generated API references can be found by: * Running `bazel build //docs:docs` and inspecting the generated files diff --git a/CHANGELOG.md b/CHANGELOG.md index 7e5598afcf..02f1df6ae0 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -96,6 +96,15 @@ END_UNRELEASED_TEMPLATE `WORKSPACE` files. See the {ref}`common-deps-with-multiple-pypi-versions` guide on using common dependencies with multiple PyPI versions` for an example. +* (toolchains) Stable ABI headers support added. To use, depend on + {obj}`//python/cc:current_py_cc_headers_abi3`. This allows Windows builds + a way to depend on headers without the potentially Python unstable ABI + objects from the regular {obj}`//python/cc:current_py_cc_headers` target + being included. + * Adds {obj}`//python/cc:current_py_cc_headers_abi3`, + {obj}`py_cc_toolchain.headers_abi3`, and {obj}`PyCcToolchainInfo.headers_abi3`. + * {obj}`//python:features.bzl%features.headers_abi3` can be used to + feature-detect the presense of the above. {#v1-6-2} ## [1.6.2] - 2025-09-21 diff --git a/docs/api/rules_python/python/cc/index.md b/docs/api/rules_python/python/cc/index.md index 82c59343be..2f4e3ae171 100644 --- a/docs/api/rules_python/python/cc/index.md +++ b/docs/api/rules_python/python/cc/index.md @@ -4,7 +4,7 @@ ::: # //python/cc -:::{bzl:target} current_py_cc_headers +::::{bzl:target} current_py_cc_headers A convenience target that provides the Python headers. It uses toolchain resolution to find the headers for the Python runtime matching the interpreter @@ -14,7 +14,32 @@ that will be used. This basically forwards the underlying This target provides: * `CcInfo`: The C++ information about the Python headers. + +:::{seealso} + +The {obj}`:current_py_cc_headers_abi3` target for explicitly using the +stable ABI. +::: + +:::: + +::::{bzl:target} current_py_cc_headers_abi3 + +A convenience target that provides the Python ABI3 headers (stable ABI headers). +It uses toolchain resolution to find the headers for the Python runtime matching +the interpreter that will be used. This basically forwards the underlying +`cc_library(name="python_headers_abi3")` target defined in the `@python_X_Y` +repo. + +This target provides: + +* `CcInfo`: The C++ information about the Python ABI3 headers. + +:::{versionadded} VERSION_NEXT_FEATURE +The {obj}`features.headers_abi3` attribute can be used to detect if this target +is available or not. ::: +:::: :::{bzl:target} current_py_cc_libs diff --git a/docs/howto/python-headers.md b/docs/howto/python-headers.md index f830febc81..fa8c2cece5 100644 --- a/docs/howto/python-headers.md +++ b/docs/howto/python-headers.md @@ -8,9 +8,10 @@ files. This guide shows how to get the necessary include paths from the Python [toolchain](toolchains). The recommended way to get the headers is to depend on the -`@rules_python//python/cc:current_py_cc_headers` target. This is a helper -target that uses toolchain resolution to find the correct headers for the -target platform. +{obj}`@rules_python//python/cc:current_py_cc_headers` or +{obj}`@rules_python//python/cc:current_py_cc_headers_abi3` +targets. These are convenience targets that use toolchain resolution to find +the correct headers for the target platform. ## Using the headers @@ -27,4 +28,27 @@ cc_library( ``` This setup ensures that your C extension code can find and use the Python -headers during compilation. \ No newline at end of file +headers during compilation. + +:::{note} +The `:current_py_cc_headers` target provides all the Python headers. This _may_ +include ABI-specific information. +::: + +## Using the stable ABI headers + +If you're building for the [Python stable ABI](https://docs.python.org/3/c-api/stable.html), +then depend on {obj}`@rules_python//python/cc:current_py_cc_headers_abi3`. This +target contains only objects relevant to the Python stable ABI. Remember to +define +[`Py_LIMITED_API`](https://docs.python.org/3/c-api/stable.html#c.Py_LIMITED_API) +when building such extensions. + +```bazel +# BUILD.bazel +cc_library( + name = "my_stable_abi_extension", + srcs = ["my_stable_abi_extension.c"], + deps = ["@rules_python//python/cc:current_py_cc_headers_abi3"], +) +``` diff --git a/docs/pyproject.toml b/docs/pyproject.toml index 2bcb31bfc2..9a089df59c 100644 --- a/docs/pyproject.toml +++ b/docs/pyproject.toml @@ -12,5 +12,6 @@ dependencies = [ "readthedocs-sphinx-ext", "absl-py", "typing-extensions", - "sphinx-reredirects" + "sphinx-reredirects", + "pefile" ] diff --git a/docs/requirements.txt b/docs/requirements.txt index cda477cd9b..5929e73785 100644 --- a/docs/requirements.txt +++ b/docs/requirements.txt @@ -111,9 +111,9 @@ colorama==0.4.6 ; sys_platform == 'win32' \ --hash=sha256:08695f5cb7ed6e0531a20572697297273c47b8cae5a63ffc6d6ed5c201be6e44 \ --hash=sha256:4f1d9991f5acc0ca119f9d443620b77f9d6b33703e51011c16baf57afb285fc6 # via sphinx -docutils==0.22 \ - --hash=sha256:4ed966a0e96a0477d852f7af31bdcb3adc049fbb35ccba358c2ea8a03287615e \ - --hash=sha256:ba9d57750e92331ebe7c08a1bbf7a7f8143b86c476acd51528b042216a6aad0f +docutils==0.21.2 \ + --hash=sha256:3a6b18732edf182daa3cd12775bbb338cf5691468f91eeeb109deff6ebfa986f \ + --hash=sha256:dafca5b9e384f0e419294eb4d2ff9fa826435bf15f15b7bd45723e8ad76811b2 # via # myst-parser # sphinx @@ -232,6 +232,10 @@ packaging==25.0 \ # via # readthedocs-sphinx-ext # sphinx +pefile==2024.8.26 \ + --hash=sha256:3ff6c5d8b43e8c37bb6e6dd5085658d658a7a0bdcd20b6a07b1fcfc1c4e9d632 \ + --hash=sha256:76f8b485dcd3b1bb8166f1128d395fa3d87af26360c2358fb75b80019b957c6f + # via rules-python-docs (docs/pyproject.toml) pygments==2.19.2 \ --hash=sha256:636cb2477cec7f8952536970bc533bc43743542f70392ae026374600add5b887 \ --hash=sha256:86540386c03d588bb81d44bc3928634ff26449851e99741617ecb9037ee5ec0b diff --git a/python/cc/BUILD.bazel b/python/cc/BUILD.bazel index f4e4aeb00f..f7686c41f6 100644 --- a/python/cc/BUILD.bazel +++ b/python/cc/BUILD.bazel @@ -2,7 +2,7 @@ load("@bazel_skylib//:bzl_library.bzl", "bzl_library") load("//python/private:bzlmod_enabled.bzl", "BZLMOD_ENABLED") -load("//python/private:current_py_cc_headers.bzl", "current_py_cc_headers") +load("//python/private:current_py_cc_headers.bzl", "current_py_cc_headers", "current_py_cc_headers_abi3") load("//python/private:current_py_cc_libs.bzl", "current_py_cc_libs") package( @@ -20,6 +20,17 @@ current_py_cc_headers( visibility = ["//visibility:public"], ) +# This target provides the C ABI3 headers for whatever the current toolchain is +# for the consuming rule. It basically acts like a cc_library by forwarding +# on the providers for the underlying cc_library that the toolchain is using. +current_py_cc_headers_abi3( + name = "current_py_cc_headers_abi3", + # Building this directly will fail unless a py cc toolchain is registered, + # and it's only under bzlmod that one is registered by default. + tags = [] if BZLMOD_ENABLED else ["manual"], + visibility = ["//visibility:public"], +) + # This target provides the C libraries for whatever the current toolchain is for # the consuming rule. It basically acts like a cc_library by forwarding on the # providers for the underlying cc_library that the toolchain is using. diff --git a/python/features.bzl b/python/features.bzl index e3d1ffdf61..21ff588dca 100644 --- a/python/features.bzl +++ b/python/features.bzl @@ -22,6 +22,16 @@ _VERSION_PRIVATE = "$Format:%(describe:tags=true)$" def _features_typedef(): """Information about features rules_python has implemented. + ::::{field} headers_abi3 + :type: bool + + True if the {obj}`@rules_python//python/cc:current_py_cc_headers_abi3` + target is available. + + :::{versionadded} VERSION_NEXT_FEATURE + ::: + :::: + ::::{field} precompile :type: bool @@ -60,6 +70,7 @@ def _features_typedef(): features = struct( TYPEDEF = _features_typedef, # keep sorted + headers_abi3 = True, precompile = True, py_info_venv_symlinks = True, uses_builtin_rules = not config.enable_pystar, diff --git a/python/private/BUILD.bazel b/python/private/BUILD.bazel index 5e2043c0c5..0c8ccdea99 100644 --- a/python/private/BUILD.bazel +++ b/python/private/BUILD.bazel @@ -31,6 +31,7 @@ filegroup( name = "distribution", srcs = glob(["**"]) + [ "//python/private/api:distribution", + "//python/private/cc:distribution", "//python/private/pypi:distribution", "//python/private/whl_filegroup:distribution", "//tools/build_defs/python/private:distribution", @@ -360,6 +361,7 @@ bzl_library( ":common_labels.bzl", ":py_cc_toolchain_info_bzl", ":rules_cc_srcs_bzl", + ":sentinel_bzl", ":util_bzl", "@bazel_skylib//rules:common_settings", ], diff --git a/python/private/cc/BUILD.bazel b/python/private/cc/BUILD.bazel new file mode 100644 index 0000000000..8f4fb468a4 --- /dev/null +++ b/python/private/cc/BUILD.bazel @@ -0,0 +1,20 @@ +load("@rules_cc//cc:cc_library.bzl", "cc_library") +load("//python/private:visibility.bzl", "NOT_ACTUALLY_PUBLIC") + +package( + default_visibility = ["//:__subpackages__"], +) + +licenses(["notice"]) + +filegroup( + name = "distribution", + srcs = glob(["**"]), +) + +# An empty cc target for use when a cc target is needed to satisfy +# Bazel, but its contents don't matter. +cc_library( + name = "empty", + visibility = NOT_ACTUALLY_PUBLIC, +) diff --git a/python/private/current_py_cc_headers.bzl b/python/private/current_py_cc_headers.bzl index 217904c22f..ef646317a6 100644 --- a/python/private/current_py_cc_headers.bzl +++ b/python/private/current_py_cc_headers.bzl @@ -12,19 +12,21 @@ # See the License for the specific language governing permissions and # limitations under the License. -"""Implementation of current_py_cc_headers rule.""" +"""Implementation of current_py_cc_headers and current_py_cc_headers_abi3 rules. +""" load("@rules_cc//cc/common:cc_info.bzl", "CcInfo") +load("//python/private:toolchain_types.bzl", "PY_CC_TOOLCHAIN_TYPE") def _current_py_cc_headers_impl(ctx): - py_cc_toolchain = ctx.toolchains["//python/cc:toolchain_type"].py_cc_toolchain + py_cc_toolchain = ctx.toolchains[PY_CC_TOOLCHAIN_TYPE].py_cc_toolchain return py_cc_toolchain.headers.providers_map.values() current_py_cc_headers = rule( implementation = _current_py_cc_headers_impl, - toolchains = ["//python/cc:toolchain_type"], + toolchains = [PY_CC_TOOLCHAIN_TYPE], provides = [CcInfo], - doc = """\ + doc = """ Provides the currently active Python toolchain's C headers. This is a wrapper around the underlying `cc_library()` for the @@ -41,3 +43,40 @@ cc_library( ``` """, ) + +def _current_py_cc_headers_abi3_impl(ctx): + py_cc_toolchain = ctx.toolchains[PY_CC_TOOLCHAIN_TYPE].py_cc_toolchain + if not py_cc_toolchain.headers_abi3: + fail(( + "The resolved {} toolchain does not provide abi3 headers. " + + "Verify the toolchain sets `.headers_abi3`, or use the " + + "`:current_py_cc_headers` target." + ).format( + PY_CC_TOOLCHAIN_TYPE, + )) + return py_cc_toolchain.headers_abi3.providers_map.values() + +current_py_cc_headers_abi3 = rule( + implementation = _current_py_cc_headers_abi3_impl, + toolchains = [PY_CC_TOOLCHAIN_TYPE], + provides = [CcInfo], + doc = """ +Provides the currently active Python toolchain's C ABI3 headers. + +This is a wrapper around the underlying `cc_library()` for the +C ABI3 headers for the consuming target's currently active Python toolchain. + +To use, simply depend on this target where you would have wanted the +toolchain's underlying `:python_headers_abi3` target: + +```starlark +cc_library( + name = "foo", + deps = ["@rules_python//python/cc:current_py_cc_headers_abi3"] +) +``` + +:::{versionadded} VERSION_NEXT_FEATURE +::: +""", +) diff --git a/python/private/hermetic_runtime_repo_setup.bzl b/python/private/hermetic_runtime_repo_setup.bzl index a35cd6bae3..a35ce8ae7d 100644 --- a/python/private/hermetic_runtime_repo_setup.bzl +++ b/python/private/hermetic_runtime_repo_setup.bzl @@ -107,9 +107,9 @@ def define_hermetic_runtime_toolchain_impl( srcs = native.glob(["include/**/*.h"]), ) cc_library( - name = "python_headers", + name = "python_headers_abi3", deps = select({ - "@bazel_tools//src/conditions:windows": [":interface", ":abi3_interface"], + "@bazel_tools//src/conditions:windows": [":abi3_interface"], "//conditions:default": None, }), hdrs = [":includes"], @@ -125,6 +125,13 @@ def define_hermetic_runtime_toolchain_impl( ], }), ) + cc_library( + name = "python_headers", + deps = [":python_headers_abi3"] + select({ + "@bazel_tools//src/conditions:windows": [":interface"], + "//conditions:default": [], + }), + ) native.config_setting( name = "is_freethreaded_linux", flag_values = { @@ -239,6 +246,7 @@ def define_hermetic_runtime_toolchain_impl( py_cc_toolchain( name = "py_cc_toolchain", headers = ":python_headers", + headers_abi3 = ":python_headers_abi3", # TODO #3155: add libctl, libtk libs = ":libpython", python_version = python_version, diff --git a/python/private/local_runtime_repo_setup.bzl b/python/private/local_runtime_repo_setup.bzl index 5d3a781152..6cff1aea43 100644 --- a/python/private/local_runtime_repo_setup.bzl +++ b/python/private/local_runtime_repo_setup.bzl @@ -67,26 +67,34 @@ def define_local_runtime_toolchain_impl( # See https://docs.python.org/3/extending/windows.html # However not all python installations (such as manylinux) include shared or static libraries, # so only create the import library when interface_library is set. - import_deps = [] + full_abi_deps = [] + abi3_deps = [] if interface_library: cc_import( name = "_python_interface_library", interface_library = interface_library, system_provided = 1, ) - import_deps = [":_python_interface_library"] + if interface_library.endswith("{}.lib".format(major)): + abi3_deps = [":_python_interface_library"] + else: + full_abi_deps = [":_python_interface_library"] cc_library( - name = "_python_headers", + name = "_python_headers_abi3", # NOTE: Keep in sync with watch_tree() called in local_runtime_repo srcs = native.glob( include = ["include/**/*.h"], exclude = ["include/numpy/**"], # numpy headers are handled separately allow_empty = True, # A Python install may not have C headers ), - deps = import_deps, + deps = abi3_deps, includes = ["include"], ) + cc_library( + name = "_python_headers", + deps = [":_python_headers_abi3"] + full_abi_deps, + ) cc_library( name = "_libpython", @@ -123,6 +131,7 @@ def define_local_runtime_toolchain_impl( py_cc_toolchain( name = "py_cc_toolchain", headers = ":_python_headers", + headers_abi3 = ":_python_headers_abi3", libs = ":_libpython", python_version = major_minor_micro, visibility = ["//visibility:public"], diff --git a/python/private/py_cc_toolchain_info.bzl b/python/private/py_cc_toolchain_info.bzl index c5cdbd9d84..8cb3680b59 100644 --- a/python/private/py_cc_toolchain_info.bzl +++ b/python/private/py_cc_toolchain_info.bzl @@ -40,7 +40,36 @@ Information about the header files, struct with fields: e.g. `:current_py_cc_headers` to act as the underlying headers target it represents). """, - "libs": """\ + "headers_abi3": """ +:type: struct | None + +If available, information about ABI3 (stable ABI) header files, struct with +fields: + * providers_map: a dict of string to provider instances. The key should be + a fully qualified name (e.g. `@rules_foo//bar:baz.bzl#MyInfo`) of the + provider to uniquely identify its type. + + The following keys are always present: + * CcInfo: the CcInfo provider instance for the headers. + * DefaultInfo: the DefaultInfo provider instance for the headers. + + A map is used to allow additional providers from the originating headers + target (typically a `cc_library`) to be propagated to consumers (directly + exposing a Target object can cause memory issues and is an anti-pattern). + + When consuming this map, it's suggested to use `providers_map.values()` to + return all providers; or copy the map and filter out or replace keys as + appropriate. Note that any keys beginning with `_` (underscore) are + considered private and should be forward along as-is (this better allows + e.g. `:current_py_cc_headers` to act as the underlying headers target it + represents). + +:::{versionadded} VERSION_NEXT_FEATURE +The {obj}`features.headers_abi3` attribute can be used to detect if this +attribute is available or not. +::: +""", + "libs": """ :type: struct | None If available, information about C libraries, struct with fields: diff --git a/python/private/py_cc_toolchain_rule.bzl b/python/private/py_cc_toolchain_rule.bzl index 8adf73c25f..b5c997ea6e 100644 --- a/python/private/py_cc_toolchain_rule.bzl +++ b/python/private/py_cc_toolchain_rule.bzl @@ -22,6 +22,7 @@ load("@bazel_skylib//rules:common_settings.bzl", "BuildSettingInfo") load("@rules_cc//cc/common:cc_info.bzl", "CcInfo") load(":common_labels.bzl", "labels") load(":py_cc_toolchain_info.bzl", "PyCcToolchainInfo") +load(":sentinel.bzl", "SentinelInfo") def _py_cc_toolchain_impl(ctx): if ctx.attr.libs: @@ -34,6 +35,16 @@ def _py_cc_toolchain_impl(ctx): else: libs = None + if ctx.attr.headers_abi3 and SentinelInfo not in ctx.attr.headers_abi3: + headers_abi3 = struct( + providers_map = { + "CcInfo": ctx.attr.headers_abi3[CcInfo], + "DefaultInfo": ctx.attr.headers_abi3[DefaultInfo], + }, + ) + else: + headers_abi3 = None + py_cc_toolchain = PyCcToolchainInfo( headers = struct( providers_map = { @@ -41,6 +52,7 @@ def _py_cc_toolchain_impl(ctx): "DefaultInfo": ctx.attr.headers[DefaultInfo], }, ), + headers_abi3 = headers_abi3, libs = libs, python_version = ctx.attr.python_version, ) @@ -61,6 +73,20 @@ py_cc_toolchain = rule( providers = [CcInfo], mandatory = True, ), + "headers_abi3": attr.label( + doc = """ +Target that provides the Python ABI3 (stable abi) headers. + +Typically this is a cc_library target. + +:::{versionadded} VERSION_NEXT_FEATURE +The {obj}`features.headers_abi3` attribute can be used to detect if this +attribute is available or not. +::: +""", + default = "//python:none", + providers = [[SentinelInfo], [CcInfo]], + ), "libs": attr.label( doc = ("Target that provides the Python runtime libraries for linking. " + "Typically this is a cc_library target of `.so` files."), @@ -74,7 +100,7 @@ py_cc_toolchain = rule( default = labels.VISIBLE_FOR_TESTING, ), }, - doc = """\ + doc = """ A toolchain for a Python runtime's C/C++ information (e.g. headers) This rule carries information about the C/C++ side of a Python runtime, e.g. diff --git a/python/private/runtime_env_toolchain.bzl b/python/private/runtime_env_toolchain.bzl index 1956ad5e95..de74900750 100644 --- a/python/private/runtime_env_toolchain.bzl +++ b/python/private/runtime_env_toolchain.bzl @@ -107,8 +107,9 @@ def define_runtime_env_toolchain(name): ) py_cc_toolchain( name = "_runtime_env_py_cc_toolchain_impl", - headers = ":_empty_cc_lib", - libs = ":_empty_cc_lib", + headers = "//python/private/cc:empty", + headers_abi3 = "//python/private/cc:empty", + libs = "//python/private/cc:empty", python_version = "0.0", tags = ["manual"], ) diff --git a/python/private/visibility.bzl b/python/private/visibility.bzl new file mode 100644 index 0000000000..3883e23638 --- /dev/null +++ b/python/private/visibility.bzl @@ -0,0 +1,7 @@ +"""Shared code for use with visibility specs.""" + +# Use when a target isn't actually public, but needs public +# visibility to keep Bazel happy. +# Such cases are typically for defaults of rule attributes or macro args that +# get used outside of rules_python itself. +NOT_ACTUALLY_PUBLIC = ["//visibility:public"] diff --git a/tests/cc/current_py_cc_headers/BUILD.bazel b/tests/cc/current_py_cc_headers/BUILD.bazel index e2d6a1b521..21723b59c4 100644 --- a/tests/cc/current_py_cc_headers/BUILD.bazel +++ b/tests/cc/current_py_cc_headers/BUILD.bazel @@ -12,6 +12,30 @@ # See the License for the specific language governing permissions and # limitations under the License. +load("@rules_cc//cc:cc_binary.bzl", "cc_binary") +load("//python:py_test.bzl", "py_test") load(":current_py_cc_headers_tests.bzl", "current_py_cc_headers_test_suite") current_py_cc_headers_test_suite(name = "current_py_cc_headers_tests") + +cc_binary( + name = "bin_abi3", + srcs = ["bin_abi3.cc"], + defines = ["Py_LIMITED_API=0x030A0000"], + linkshared = True, + deps = [ + "//python/cc:current_py_cc_headers_abi3", + ], +) + +py_test( + name = "abi3_headers_linkage_test_py", + srcs = ["abi3_headers_linkage_test.py"], + data = [":bin_abi3"], + main = "abi3_headers_linkage_test.py", + target_compatible_with = ["@platforms//os:windows"], + deps = [ + "//python/runfiles", + "@dev_pip//pefile", + ], +) diff --git a/tests/cc/current_py_cc_headers/abi3_headers_linkage_test.py b/tests/cc/current_py_cc_headers/abi3_headers_linkage_test.py new file mode 100644 index 0000000000..6c337653b1 --- /dev/null +++ b/tests/cc/current_py_cc_headers/abi3_headers_linkage_test.py @@ -0,0 +1,28 @@ +import os.path +import pathlib +import sys +import unittest + +import pefile + +from python.runfiles import runfiles + + +class CheckLinkageTest(unittest.TestCase): + @unittest.skipUnless(sys.platform.startswith("win"), "requires windows") + def test_linkage_windows(self): + rf = runfiles.Create() + dll_path = rf.Rlocation("rules_python/tests/cc/current_py_cc_headers/bin_abi3.dll") + pe = pefile.PE(dll_path) + if not hasattr(pe, "DIRECTORY_ENTRY_IMPORT"): + self.fail("No import directory found.") + + imported_dlls = [ + entry.dll.decode("utf-8").lower() for entry in pe.DIRECTORY_ENTRY_IMPORT + ] + python_dlls = [dll for dll in imported_dlls if dll.startswith("python3")] + self.assertEqual(python_dlls, ["python3.dll"]) + + +if __name__ == "__main__": + unittest.main() diff --git a/tests/cc/current_py_cc_headers/bin_abi3.cc b/tests/cc/current_py_cc_headers/bin_abi3.cc new file mode 100644 index 0000000000..0211dbbfc6 --- /dev/null +++ b/tests/cc/current_py_cc_headers/bin_abi3.cc @@ -0,0 +1,17 @@ +#include + +int SomeFunction() { + // Early return to prevent the broken code below from running. + if (true) { + return 0; + } + + // The below code won't actually run. We just reference some Python + // symbols so the compiler and linker do some work to verify they are + // able to resolve the symbols. + // To make it actually run, more custom initialization is necessary. + // See https://docs.python.org/3/c-api/intro.html#embedding-python + Py_Initialize(); + Py_Finalize(); + return 0; +} diff --git a/tests/cc/current_py_cc_headers/current_py_cc_headers_tests.bzl b/tests/cc/current_py_cc_headers/current_py_cc_headers_tests.bzl index 818f7ff092..f52d93e75f 100644 --- a/tests/cc/current_py_cc_headers/current_py_cc_headers_tests.bzl +++ b/tests/cc/current_py_cc_headers/current_py_cc_headers_tests.bzl @@ -22,25 +22,7 @@ load("//tests/support:support.bzl", "CC_TOOLCHAIN") _tests = [] -def _test_current_toolchain_headers(name): - analysis_test( - name = name, - impl = _test_current_toolchain_headers_impl, - target = "//python/cc:current_py_cc_headers", - config_settings = { - "//command_line_option:extra_toolchains": [CC_TOOLCHAIN], - }, - attrs = { - "header_files": attr.label_list( - default = [ - "//tests/support/cc_toolchains:py_header_files", - ], - allow_files = True, - ), - }, - ) - -def _test_current_toolchain_headers_impl(env, target): +def _verify_headers_target(env, target): # Check that the forwarded CcInfo looks vaguely correct. compilation_context = env.expect.that_target(target).provider( CcInfo, @@ -70,6 +52,27 @@ def _test_current_toolchain_headers_impl(env, target): matching.str_matches("*/cc_toolchains/data.txt"), ) +def _test_current_toolchain_headers(name): + analysis_test( + name = name, + impl = _test_current_toolchain_headers_impl, + target = "//python/cc:current_py_cc_headers", + config_settings = { + "//command_line_option:extra_toolchains": [CC_TOOLCHAIN], + }, + attrs = { + "header_files": attr.label_list( + default = [ + "//tests/support/cc_toolchains:py_headers_files", + ], + allow_files = True, + ), + }, + ) + +def _test_current_toolchain_headers_impl(env, target): + _verify_headers_target(env, target) + _tests.append(_test_current_toolchain_headers) def _test_toolchain_is_registered_by_default(name): @@ -84,6 +87,29 @@ def _test_toolchain_is_registered_by_default_impl(env, target): _tests.append(_test_toolchain_is_registered_by_default) +def _test_current_toolchain_headers_abi3(name): + analysis_test( + name = name, + impl = _test_current_toolchain_headers_abi3_impl, + target = "//python/cc:current_py_cc_headers_abi3", + config_settings = { + "//command_line_option:extra_toolchains": [CC_TOOLCHAIN], + }, + attrs = { + "header_files": attr.label_list( + default = [ + "//tests/support/cc_toolchains:py_headers_abi3_files", + ], + allow_files = True, + ), + }, + ) + +def _test_current_toolchain_headers_abi3_impl(env, target): + _verify_headers_target(env, target) + +_tests.append(_test_current_toolchain_headers_abi3) + def current_py_cc_headers_test_suite(name): test_suite( name = name, diff --git a/tests/cc/current_py_cc_libs/BUILD.bazel b/tests/cc/current_py_cc_libs/BUILD.bazel index 9269553a3f..6b4e80fc6a 100644 --- a/tests/cc/current_py_cc_libs/BUILD.bazel +++ b/tests/cc/current_py_cc_libs/BUILD.bazel @@ -12,11 +12,11 @@ # See the License for the specific language governing permissions and # limitations under the License. +load("@rules_cc//cc:cc_test.bzl", "cc_test") load(":current_py_cc_libs_tests.bzl", "current_py_cc_libs_test_suite") current_py_cc_libs_test_suite(name = "current_py_cc_libs_tests") -# buildifier: disable=native-cc cc_test( name = "python_libs_linking_test", srcs = ["python_libs_linking_test.cc"], @@ -31,14 +31,12 @@ cc_test( # the expected Windows libraries are all present in the expected location. # Since we define the Py_LIMITED_API macro, we expect the linker to go search # for libs/python3.lib. -# buildifier: disable=native-cc cc_test( - name = "python_abi3_libs_linking_windows_test", + name = "python_abi3_libs_linking_test", srcs = ["python_libs_linking_test.cc"], defines = ["Py_LIMITED_API=0x030A0000"], - target_compatible_with = ["@platforms//os:windows"], deps = [ - "@rules_python//python/cc:current_py_cc_headers", + "@rules_python//python/cc:current_py_cc_headers_abi3", "@rules_python//python/cc:current_py_cc_libs", ], ) diff --git a/tests/cc/py_cc_toolchain/py_cc_toolchain_tests.bzl b/tests/cc/py_cc_toolchain/py_cc_toolchain_tests.bzl index ba8e089cbb..975f0d56b5 100644 --- a/tests/cc/py_cc_toolchain/py_cc_toolchain_tests.bzl +++ b/tests/cc/py_cc_toolchain/py_cc_toolchain_tests.bzl @@ -28,8 +28,12 @@ def _test_py_cc_toolchain(name): impl = _test_py_cc_toolchain_impl, target = "//tests/support/cc_toolchains:fake_py_cc_toolchain_impl", attrs = { + "header_abi3_files": attr.label_list( + default = ["//tests/support/cc_toolchains:py_headers_abi3_files"], + allow_files = True, + ), "header_files": attr.label_list( - default = ["//tests/support/cc_toolchains:py_header_files"], + default = ["//tests/support/cc_toolchains:py_headers_files"], allow_files = True, ), }, @@ -44,6 +48,7 @@ def _test_py_cc_toolchain_impl(env, target): ) toolchain.python_version().equals("3.999") + # ===== Verify headers info ===== headers_providers = toolchain.headers().providers_map() headers_providers.keys().contains_exactly(["CcInfo", "DefaultInfo"]) @@ -57,9 +62,15 @@ def _test_py_cc_toolchain_impl(env, target): env.ctx.files.header_files, ) + # NOTE: Bazel 8 and lower put cc_library.includes into `.system_includes`, + # while Bazel 9 put it in `.includes`. Both result in the includes being + # added as system includes, so either is acceptable for the expected + # `#include ` to work. + includes = compilation_context.actual.includes.to_list() + compilation_context.actual.system_includes.to_list() + # NOTE: The include dir gets added twice, once for the source path, - # and once for the config-specific path, but we don't care about that. - compilation_context.system_includes().contains_at_least_predicates([ + # and once for the config-specific path. + env.expect.that_collection(includes).contains_at_least_predicates([ matching.str_matches("*/py_include"), ]) @@ -68,6 +79,32 @@ def _test_py_cc_toolchain_impl(env, target): matching.str_matches("*/cc_toolchains/data.txt"), ) + # ===== Verify headers_abi3 info ===== + headers_abi3_providers = toolchain.headers_abi3().providers_map() + headers_abi3_providers.keys().contains_exactly(["CcInfo", "DefaultInfo"]) + + cc_info = headers_abi3_providers.get("CcInfo", factory = cc_info_subject) + + compilation_context = cc_info.compilation_context() + compilation_context.direct_headers().contains_exactly( + env.ctx.files.header_abi3_files, + ) + compilation_context.direct_public_headers().contains_exactly( + env.ctx.files.header_abi3_files, + ) + + # NOTE: Bazel 8 and lower put cc_library.includes into `.system_includes`, + # while Bazel 9 put it in `.includes`. Both result in the includes being + # added as system includes, so either is acceptable for the expected + # `#include ` to work. + includes = compilation_context.actual.includes.to_list() + compilation_context.actual.system_includes.to_list() + + default_info = headers_abi3_providers.get("DefaultInfo", factory = subjects.default_info) + default_info.runfiles().contains_predicate( + matching.str_matches("*/cc_toolchains/data.txt"), + ) + + # ===== Verify libs info ===== libs_providers = toolchain.libs().providers_map() libs_providers.keys().contains_exactly(["CcInfo", "DefaultInfo"]) diff --git a/tests/support/cc_toolchains/BUILD.bazel b/tests/support/cc_toolchains/BUILD.bazel index afa88dc836..1c1a714626 100644 --- a/tests/support/cc_toolchains/BUILD.bazel +++ b/tests/support/cc_toolchains/BUILD.bazel @@ -23,9 +23,18 @@ package(default_visibility = ["//:__subpackages__"]) # Factored out for testing filegroup( - name = "py_header_files", + name = "py_headers_files", srcs = [ "py_header.h", + ":py_headers_abi3_files", + ], +) + +# Factored out for testing +filegroup( + name = "py_headers_abi3_files", + srcs = [ + "py_abi3_header.h", "py_include/py_include.h", ], ) @@ -46,19 +55,27 @@ toolchain( py_cc_toolchain( name = "fake_py_cc_toolchain_impl", headers = ":py_headers", + headers_abi3 = ":py_headers_abi3", libs = ":fake_libs", python_version = "3.999", tags = PREVENT_IMPLICIT_BUILDING_TAGS, ) cc_library( - name = "py_headers", - hdrs = [":py_header_files"], + name = "py_headers_abi3", + hdrs = [":py_headers_abi3_files"], data = ["data.txt"], includes = ["py_include"], tags = PREVENT_IMPLICIT_BUILDING_TAGS, ) +cc_library( + name = "py_headers", + hdrs = [":py_headers_files"], + tags = PREVENT_IMPLICIT_BUILDING_TAGS, + deps = [":py_headers_abi3"], +) + cc_library( name = "fake_libs", srcs = ["libpython3.so"], diff --git a/tests/support/cc_toolchains/py_abi3_header.h b/tests/support/cc_toolchains/py_abi3_header.h new file mode 100644 index 0000000000..e69de29bb2 diff --git a/tests/support/py_cc_toolchain_info_subject.bzl b/tests/support/py_cc_toolchain_info_subject.bzl index 4d3647c53e..3820e04e90 100644 --- a/tests/support/py_cc_toolchain_info_subject.bzl +++ b/tests/support/py_cc_toolchain_info_subject.bzl @@ -19,6 +19,7 @@ def _py_cc_toolchain_info_subject_new(info, *, meta): # buildifier: disable=uninitialized public = struct( headers = lambda *a, **k: _py_cc_toolchain_info_subject_headers(self, *a, **k), + headers_abi3 = lambda *a, **k: _py_cc_toolchain_info_subject_headers_abi3(self, *a, **k), libs = lambda *a, **k: _py_cc_toolchain_info_subject_libs(self, *a, **k), python_version = lambda *a, **k: _py_cc_toolchain_info_subject_python_version(self, *a, **k), actual = info, @@ -35,6 +36,15 @@ def _py_cc_toolchain_info_subject_headers(self): ), ) +def _py_cc_toolchain_info_subject_headers_abi3(self): + return subjects.struct( + self.actual.headers_abi3, + meta = self.meta.derive("headers_abi3()"), + attrs = dict( + providers_map = subjects.dict, + ), + ) + def _py_cc_toolchain_info_subject_libs(self): return subjects.struct( self.actual.libs, From fec87e4f987f6bed7344d03cb4277c50d33a2d71 Mon Sep 17 00:00:00 2001 From: Ignas Anikevicius <240938+aignas@users.noreply.github.com> Date: Mon, 22 Sep 2025 11:21:05 +0900 Subject: [PATCH 449/922] docs: move 1.6.3 related changelog (#3284) This fixes the changelog and manually changes the version updated tag. The changes have been also cherry-picked to the release branch as I saw release fail in the [workflow]. Hence this has become a 1.6.3 instead. [workflow]: https://github.com/bazel-contrib/rules_python/actions/runs/17894155072/job/50878515841 Fixes #3273 --------- Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> --- CHANGELOG.md | 10 +++++----- python/private/pypi/extension.bzl | 2 +- 2 files changed, 6 insertions(+), 6 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 02f1df6ae0..3d06a68935 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -106,12 +106,12 @@ END_UNRELEASED_TEMPLATE * {obj}`//python:features.bzl%features.headers_abi3` can be used to feature-detect the presense of the above. -{#v1-6-2} -## [1.6.2] - 2025-09-21 +{#v1-6-3} +## [1.6.3] - 2025-09-21 -[1.6.2]: https://github.com/bazel-contrib/rules_python/releases/tag/1.6.2 +[1.6.3]: https://github.com/bazel-contrib/rules_python/releases/tag/1.6.3 -{#v1-6-2-fixed} +{#v1-6-3-fixed} ### Fixed * (pypi) We now use the Minimal Version Selection (MVS) algorithm to select @@ -119,7 +119,7 @@ END_UNRELEASED_TEMPLATE (e.g. `musllinux_1_1_x86_64` and `musllinux_1_2_x86_64`). If the user wants to set the minimum version for the selection algorithm, use the {attr}`pip.defaults.whl_platform_tags` attribute to configure that. If - `musllinux_*_x86_64` is specified, we will chose the lowest available + `musllinux_*_x86_64` is specified, we will choose the lowest available wheel version. Fixes [#3250](https://github.com/bazel-contrib/rules_python/issues/3250). {#v1-6-0} diff --git a/python/private/pypi/extension.bzl b/python/private/pypi/extension.bzl index c14912c2c9..be1a8e4d03 100644 --- a/python/private/pypi/extension.bzl +++ b/python/private/pypi/extension.bzl @@ -518,7 +518,7 @@ Common patterns: :::{seealso} See official [docs](https://packaging.python.org/en/latest/specifications/platform-compatibility-tags/#platform-tag) for more information. ::: -:::{versionchanged} VERSION_NEXT_FEATURE +:::{versionchanged} 1.6.3 The matching of versioned platforms have been switched to MVS (Minimal Version Selection) algorithm for easier evaluation logic and fewer surprises. The legacy platform tags are supported from this version without extra handling from the user. From 45762fc5ee7eb3c3f8943f9a2a47303233a25800 Mon Sep 17 00:00:00 2001 From: Greg Date: Mon, 22 Sep 2025 22:40:01 -0400 Subject: [PATCH 450/922] refactor: read migrated native flags through a centralized accessor function (#3290) This lets Python logic support either Starlark- or native-defined flags, based on Bazel support: - Pre-Bazel 9.0: always use native flags - Bazel 9.0+: use Starlark flags if [incompatible change](https://github.com/bazelbuild/bazel/pull/27056) that disables `ctx.fragments.py` and `ctx.fragments.bazel_py` is set - Include a developer override to trigger Starlark versions while testing Starlarkification. Developers enable this by updating .bzl code in their local workspace. This can be removed as soon as Starlarkification is complete Also check in a Starlark version of `--experimental_python_import_all_repositories` for testing. **This is a no-op. New flag sources of truth don't take effect without a supporting bazel version that sets appropriate incompatible flags.** **Caveats:** - May not work with `configuration_field` as implemented. We can loosen the incompatible flag lockdown if that's an issue. For https://github.com/bazel-contrib/rules_python/issues/3252 TODO list * [ ] Add docs to docs/api/rules_python/python/config_settings/index.md for starlarkified flags --- python/config_settings/BUILD.bazel | 8 +++- python/private/flags.bzl | 53 +++++++++++++++++++++++++ python/private/py_executable.bzl | 14 +++---- python/private/py_runtime_pair_rule.bzl | 3 +- 4 files changed, 69 insertions(+), 9 deletions(-) diff --git a/python/config_settings/BUILD.bazel b/python/config_settings/BUILD.bazel index 82a73cee6c..cc5c472fe7 100644 --- a/python/config_settings/BUILD.bazel +++ b/python/config_settings/BUILD.bazel @@ -1,4 +1,4 @@ -load("@bazel_skylib//rules:common_settings.bzl", "string_flag") +load("@bazel_skylib//rules:common_settings.bzl", "bool_flag", "string_flag") load("@pythons_hub//:versions.bzl", "DEFAULT_PYTHON_VERSION", "MINOR_MAPPING", "PYTHON_VERSIONS") load( "//python/private:flags.bzl", @@ -240,3 +240,9 @@ label_flag( # NOTE: Only public because it is used in pip hub repos. visibility = ["//visibility:public"], ) + +bool_flag( + name = "experimental_python_import_all_repositories", + build_setting_default = True, + visibility = ["//visibility:public"], +) diff --git a/python/private/flags.bzl b/python/private/flags.bzl index 710402ba68..82ec83294b 100644 --- a/python/private/flags.bzl +++ b/python/private/flags.bzl @@ -21,6 +21,59 @@ unnecessary files when all that are needed are flag definitions. load("@bazel_skylib//rules:common_settings.bzl", "BuildSettingInfo") load(":enum.bzl", "FlagEnum", "enum") +# Maps "--myflag" to a tuple of: +# +# - the flag's ctx.fragments native API accessor +# -"native|starlark": which definition to use if the flag is available both +# from ctx.fragments and Starlark +# +# Builds that set --incompatible_remove_ctx_py_fragment or +# --incompatible_remove_ctx_bazel_py_fragment disable ctx.fragments. These +# builds assume flags are solely defined in Starlark. +# +# The "native|starlark" override is only for devs who are testing flag +# Starlarkification. If ctx.fragments.[py|bazel_py] is available and +# a flag is set to "starlark", we exclusively read its starlark version. +# +# See https://github.com/bazel-contrib/rules_python/issues/3252. +_POSSIBLY_NATIVE_FLAGS = { + "build_python_zip": (lambda ctx: ctx.fragments.py.build_python_zip, "native"), + "default_to_explicit_init_py": (lambda ctx: ctx.fragments.py.default_to_explicit_init_py, "native"), + "disable_py2": (lambda ctx: ctx.fragments.py.disable_py2, "native"), + "python_import_all_repositories": (lambda ctx: ctx.fragments.bazel_py.python_import_all_repositories, "native"), + "python_path": (lambda ctx: ctx.fragments.bazel_py.python_path, "native"), +} + +def read_possibly_native_flag(ctx, flag_name): + """ + Canonical API for reading a Python build flag. + + Flags might be defined in Starlark or native-Bazel. This function reasd flags + from tbe correct source based on supporting Bazel version and --incompatible* + flags that disable native references. + + Args: + ctx: Rule's configuration context. + flag_name: Name of the flag to read, without preceding "--". + + Returns: + The flag's value. + """ + + # Bazel 9.0+ can disable these fragments with --incompatible_remove_ctx_py_fragment and + # --incompatible_remove_ctx_bazel_py_fragment. Disabling them means bazel expects + # Python to read Starlark flags. + use_native_def = hasattr(ctx.fragments, "py") and hasattr(ctx.fragments, "bazel_py") + + # Developer override to force the Starlark definition for testing. + if _POSSIBLY_NATIVE_FLAGS[flag_name][1] == "starlark": + use_native_def = False + if use_native_def: + return _POSSIBLY_NATIVE_FLAGS[flag_name][0](ctx) + else: + # Starlark definition of "--foo" is assumed to be a label dependency named "_foo". + return getattr(ctx.attr, "_" + flag_name)[BuildSettingInfo].value + def _AddSrcsToRunfilesFlag_is_enabled(ctx): value = ctx.attr._add_srcs_to_runfiles_flag[BuildSettingInfo].value if value == AddSrcsToRunfilesFlag.AUTO: diff --git a/python/private/py_executable.bzl b/python/private/py_executable.bzl index 5993a4f003..dd0a1a1d6e 100644 --- a/python/private/py_executable.bzl +++ b/python/private/py_executable.bzl @@ -53,7 +53,7 @@ load( "target_platform_has_any_constraint", ) load(":common_labels.bzl", "labels") -load(":flags.bzl", "BootstrapImplFlag", "VenvsUseDeclareSymlinkFlag") +load(":flags.bzl", "BootstrapImplFlag", "VenvsUseDeclareSymlinkFlag", "read_possibly_native_flag") load(":precompile.bzl", "maybe_precompile") load(":py_cc_link_params_info.bzl", "PyCcLinkParamsInfo") load(":py_executable_info.bzl", "PyExecutableInfo") @@ -293,7 +293,7 @@ def _get_stamp_flag(ctx): def _should_create_init_files(ctx): if ctx.attr.legacy_create_init == -1: - return not ctx.fragments.py.default_to_explicit_init_py + return not read_possibly_native_flag(ctx, "default_to_explicit_init_py") else: return bool(ctx.attr.legacy_create_init) @@ -381,7 +381,7 @@ def _create_executable( extra_files_to_build = [] # NOTE: --build_python_zip defaults to true on Windows - build_zip_enabled = ctx.fragments.py.build_python_zip + build_zip_enabled = read_possibly_native_flag(ctx, "build_python_zip") # When --build_python_zip is enabled, then the zip file becomes # one of the default outputs. @@ -587,7 +587,7 @@ def _create_venv(ctx, output_prefix, imports, runtime_details): output = site_init, substitutions = { "%coverage_tool%": _get_coverage_tool_runfiles_path(ctx, runtime), - "%import_all%": "True" if ctx.fragments.bazel_py.python_import_all_repositories else "False", + "%import_all%": "True" if read_possibly_native_flag(ctx, "python_import_all_repositories") else "False", "%site_init_runfiles_path%": "{}/{}".format(ctx.workspace_name, site_init.short_path), "%workspace_name%": ctx.workspace_name, }, @@ -668,7 +668,7 @@ def _create_stage2_bootstrap( output = output, substitutions = { "%coverage_tool%": _get_coverage_tool_runfiles_path(ctx, runtime), - "%import_all%": "True" if ctx.fragments.bazel_py.python_import_all_repositories else "False", + "%import_all%": "True" if read_possibly_native_flag(ctx, "python_import_all_repositories") else "False", "%imports%": ":".join(imports.to_list()), "%main%": main_py_path, "%main_module%": ctx.attr.main_module, @@ -755,7 +755,7 @@ def _create_stage1_bootstrap( template = ctx.file._bootstrap_template subs["%coverage_tool%"] = coverage_tool_runfiles_path - subs["%import_all%"] = ("True" if ctx.fragments.bazel_py.python_import_all_repositories else "False") + subs["%import_all%"] = ("True" if read_possibly_native_flag(ctx, "python_import_all_repositories") else "False") subs["%imports%"] = ":".join(imports.to_list()) subs["%main%"] = "{}/{}".format(ctx.workspace_name, main_py.short_path) @@ -1135,7 +1135,7 @@ def _get_runtime_details(ctx, semantics): # # TOOD(bazelbuild/bazel#7901): Remove this once --python_path flag is removed. - flag_interpreter_path = ctx.fragments.bazel_py.python_path + flag_interpreter_path = read_possibly_native_flag(ctx, "python_path") toolchain_runtime, effective_runtime = _maybe_get_runtime_from_ctx(ctx) if not effective_runtime: # Clear these just in case diff --git a/python/private/py_runtime_pair_rule.bzl b/python/private/py_runtime_pair_rule.bzl index 775d53a0b8..61cbdcd6f4 100644 --- a/python/private/py_runtime_pair_rule.bzl +++ b/python/private/py_runtime_pair_rule.bzl @@ -17,6 +17,7 @@ load("@bazel_skylib//rules:common_settings.bzl", "BuildSettingInfo") load("//python:py_runtime_info.bzl", "PyRuntimeInfo") load(":common_labels.bzl", "labels") +load(":flags.bzl", "read_possibly_native_flag") load(":reexports.bzl", "BuiltinPyRuntimeInfo") load(":util.bzl", "IS_BAZEL_7_OR_HIGHER") @@ -69,7 +70,7 @@ def _is_py2_disabled(ctx): # TODO: Remove this once all supported Balze versions have this flag. if not hasattr(ctx.fragments.py, "disable_py"): return False - return ctx.fragments.py.disable_py2 + return read_possibly_native_flag(ctx, "disable_py2") _MaybeBuiltinPyRuntimeInfo = [[BuiltinPyRuntimeInfo]] if BuiltinPyRuntimeInfo != None else [] From 5b5e58f4f6fe5e350826f34666570bf7d364e78d Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Mon, 22 Sep 2025 19:41:22 -0700 Subject: [PATCH 451/922] build(deps): bump docutils from 0.21.2 to 0.22.2 in /docs (#3287) Bumps [docutils](https://github.com/rtfd/recommonmark) from 0.21.2 to 0.22.2.
Commits

[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=docutils&package-manager=pip&previous-version=0.21.2&new-version=0.22.2)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) ---
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot show ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> --- docs/requirements.txt | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/requirements.txt b/docs/requirements.txt index 5929e73785..c05554aeeb 100644 --- a/docs/requirements.txt +++ b/docs/requirements.txt @@ -111,9 +111,9 @@ colorama==0.4.6 ; sys_platform == 'win32' \ --hash=sha256:08695f5cb7ed6e0531a20572697297273c47b8cae5a63ffc6d6ed5c201be6e44 \ --hash=sha256:4f1d9991f5acc0ca119f9d443620b77f9d6b33703e51011c16baf57afb285fc6 # via sphinx -docutils==0.21.2 \ - --hash=sha256:3a6b18732edf182daa3cd12775bbb338cf5691468f91eeeb109deff6ebfa986f \ - --hash=sha256:dafca5b9e384f0e419294eb4d2ff9fa826435bf15f15b7bd45723e8ad76811b2 +docutils==0.22.2 \ + --hash=sha256:9fdb771707c8784c8f2728b67cb2c691305933d68137ef95a75db5f4dfbc213d \ + --hash=sha256:b0e98d679283fc3bb0ead8a5da7f501baa632654e7056e9c5846842213d674d8 # via # myst-parser # sphinx From 05735c8fa4ab81077a73baca2c9c6ec18091609a Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Tue, 23 Sep 2025 11:25:14 -0700 Subject: [PATCH 452/922] build(deps): bump jaraco-functools from 4.1.0 to 4.3.0 in /tools/publish (#3288) Bumps [jaraco-functools](https://github.com/jaraco/jaraco.functools) from 4.1.0 to 4.3.0.
Changelog

Sourced from jaraco-functools's changelog.

v4.3.0

Features

  • Add none_as function.

v4.2.1

No significant changes.

v4.2.0

Features

  • Add 'passthrough' function.

Bugfixes

  • Added missing splat in stubs -- by :user:Avasam (#29)
Commits

[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=jaraco-functools&package-manager=pip&previous-version=4.1.0&new-version=4.3.0)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) ---
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot show ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> --- tools/publish/requirements_darwin.txt | 6 +++--- tools/publish/requirements_linux.txt | 6 +++--- tools/publish/requirements_universal.txt | 6 +++--- tools/publish/requirements_windows.txt | 6 +++--- 4 files changed, 12 insertions(+), 12 deletions(-) diff --git a/tools/publish/requirements_darwin.txt b/tools/publish/requirements_darwin.txt index bc7e1c609d..39248d4675 100644 --- a/tools/publish/requirements_darwin.txt +++ b/tools/publish/requirements_darwin.txt @@ -113,9 +113,9 @@ jaraco-context==6.0.1 \ --hash=sha256:9bae4ea555cf0b14938dc0aee7c9f32ed303aa20a3b73e7dc80111628792d1b3 \ --hash=sha256:f797fc481b490edb305122c9181830a3a5b76d84ef6d1aef2fb9b47ab956f9e4 # via keyring -jaraco-functools==4.1.0 \ - --hash=sha256:70f7e0e2ae076498e212562325e805204fc092d7b4c17e0e86c959e249701a9d \ - --hash=sha256:ad159f13428bc4acbf5541ad6dec511f91573b90fba04df61dafa2a1231cf649 +jaraco-functools==4.3.0 \ + --hash=sha256:227ff8ed6f7b8f62c56deff101545fa7543cf2c8e7b82a7c2116e672f29c26e8 \ + --hash=sha256:cfd13ad0dd2c47a3600b439ef72d8615d482cedcff1632930d6f28924d92f294 # via keyring keyring==25.6.0 \ --hash=sha256:0b39998aa941431eb3d9b0d4b2460bc773b9df6fed7621c2dfb291a7e0187a66 \ diff --git a/tools/publish/requirements_linux.txt b/tools/publish/requirements_linux.txt index 522a5f3da2..c078a0ed61 100644 --- a/tools/publish/requirements_linux.txt +++ b/tools/publish/requirements_linux.txt @@ -238,9 +238,9 @@ jaraco-context==6.0.1 \ --hash=sha256:9bae4ea555cf0b14938dc0aee7c9f32ed303aa20a3b73e7dc80111628792d1b3 \ --hash=sha256:f797fc481b490edb305122c9181830a3a5b76d84ef6d1aef2fb9b47ab956f9e4 # via keyring -jaraco-functools==4.1.0 \ - --hash=sha256:70f7e0e2ae076498e212562325e805204fc092d7b4c17e0e86c959e249701a9d \ - --hash=sha256:ad159f13428bc4acbf5541ad6dec511f91573b90fba04df61dafa2a1231cf649 +jaraco-functools==4.3.0 \ + --hash=sha256:227ff8ed6f7b8f62c56deff101545fa7543cf2c8e7b82a7c2116e672f29c26e8 \ + --hash=sha256:cfd13ad0dd2c47a3600b439ef72d8615d482cedcff1632930d6f28924d92f294 # via keyring jeepney==0.9.0 \ --hash=sha256:97e5714520c16fc0a45695e5365a2e11b81ea79bba796e26f9f1d178cb182683 \ diff --git a/tools/publish/requirements_universal.txt b/tools/publish/requirements_universal.txt index e8d1c747f6..a3a3f23a51 100644 --- a/tools/publish/requirements_universal.txt +++ b/tools/publish/requirements_universal.txt @@ -221,9 +221,9 @@ jaraco-context==6.0.1 \ --hash=sha256:9bae4ea555cf0b14938dc0aee7c9f32ed303aa20a3b73e7dc80111628792d1b3 \ --hash=sha256:f797fc481b490edb305122c9181830a3a5b76d84ef6d1aef2fb9b47ab956f9e4 # via keyring -jaraco-functools==4.1.0 \ - --hash=sha256:70f7e0e2ae076498e212562325e805204fc092d7b4c17e0e86c959e249701a9d \ - --hash=sha256:ad159f13428bc4acbf5541ad6dec511f91573b90fba04df61dafa2a1231cf649 +jaraco-functools==4.3.0 \ + --hash=sha256:227ff8ed6f7b8f62c56deff101545fa7543cf2c8e7b82a7c2116e672f29c26e8 \ + --hash=sha256:cfd13ad0dd2c47a3600b439ef72d8615d482cedcff1632930d6f28924d92f294 # via keyring jeepney==0.9.0 ; sys_platform == 'linux' \ --hash=sha256:97e5714520c16fc0a45695e5365a2e11b81ea79bba796e26f9f1d178cb182683 \ diff --git a/tools/publish/requirements_windows.txt b/tools/publish/requirements_windows.txt index 38d854eb5c..b4eb83d1f1 100644 --- a/tools/publish/requirements_windows.txt +++ b/tools/publish/requirements_windows.txt @@ -113,9 +113,9 @@ jaraco-context==6.0.1 \ --hash=sha256:9bae4ea555cf0b14938dc0aee7c9f32ed303aa20a3b73e7dc80111628792d1b3 \ --hash=sha256:f797fc481b490edb305122c9181830a3a5b76d84ef6d1aef2fb9b47ab956f9e4 # via keyring -jaraco-functools==4.1.0 \ - --hash=sha256:70f7e0e2ae076498e212562325e805204fc092d7b4c17e0e86c959e249701a9d \ - --hash=sha256:ad159f13428bc4acbf5541ad6dec511f91573b90fba04df61dafa2a1231cf649 +jaraco-functools==4.3.0 \ + --hash=sha256:227ff8ed6f7b8f62c56deff101545fa7543cf2c8e7b82a7c2116e672f29c26e8 \ + --hash=sha256:cfd13ad0dd2c47a3600b439ef72d8615d482cedcff1632930d6f28924d92f294 # via keyring keyring==25.6.0 \ --hash=sha256:0b39998aa941431eb3d9b0d4b2460bc773b9df6fed7621c2dfb291a7e0187a66 \ From c5662583fed5d0418cfbc0f514935752ce950fae Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Fri, 26 Sep 2025 16:25:36 -0700 Subject: [PATCH 453/922] docs: add example for a complex multi-platform pypi configuration (#3292) The core PyPI docs and API reference docs have the basics for setting up a multi-platform Bazel build, but there's a lot of cross-referencing and reading between the lines necessary. Create a how to guide specifically on how to do it to better explain the nuances. --------- Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> --- docs/howto/multi-platform-pypi-deps.md | 194 +++++++++++++++++++++++++ docs/pypi/index.md | 4 + python/private/pypi/pip_repository.bzl | 3 + 3 files changed, 201 insertions(+) create mode 100644 docs/howto/multi-platform-pypi-deps.md diff --git a/docs/howto/multi-platform-pypi-deps.md b/docs/howto/multi-platform-pypi-deps.md new file mode 100644 index 0000000000..6cc7f842ea --- /dev/null +++ b/docs/howto/multi-platform-pypi-deps.md @@ -0,0 +1,194 @@ +:::{default-domain} bzl +::: + +# How-to: Multi-Platform PyPI Dependencies + +When developing applications that need to run on a wide variety of platforms, +managing PyPI dependencies can become complex. You might need different sets of +dependencies for different combinations of Python version, threading model, +operating system, CPU architecture, libc, and even hardware accelerators like +GPUs. + +This guide demonstrates how to manage this complexity using `rules_python` with +bzlmod. If you prefer to learn by example, complete example code is provided at +the end. + +In this how to guide, we configure for using 4 requirements files, each +for a different variation using Python 3.14 on Linux: + +* Regular (non-freethreaded) Python +* Freethreaded Python +* Regular Python for CUDA 12.9 +* Freethreaded Python for ARM and Musl + +## Mapping requirements files to Bazel configuration settings + +Unfortunately, a requirements file doesn't tell what it's compatible with, +so we have to manually specify the Bazel configuration settings for it. To do +that using rules_python, there are two steps: defining a platform, then +associating a requirements file with the platform. + +### Defining a platform + +First, we define a "platform" using {obj}`pip.default`. This associates an +arbitrary name with a list of Bazel {obj}`config_setting` targets. While any +name can be used for a platform (its name has no inherent semantic meaning), it +should encode all the relevant dimensions that distinguish a requirements file. +For example, if a requirements file is specifically for the combination of CUDA +12.9 and NumPy 2.0, then the platform name should represent that. + +The convention is to follow the format of `{os}_{cpu}{threading}`, where: + +* `{os}` is the operating system (`linux`, `osx`, `windows`). +* `{cpu}` is the architecture (`x86_64`, `aarch64`). +* `{threading}` is `_freethreaded` for a freethreaded Python runtime, or an + empty string for the regular runtime. + +Additional dimensions should be appended and separated with an underscore (e.g. +`linux_x86_64_musl_cuda12.9_numpy2`). + +The platform name should not include the Python version. That is handled by +`pip.parse.python_version` separately. + +:::{note} +The term _platform_ here has nothing to do with Bazel's `platform()` rule. +::: + +#### Defining custom settings + +Because {obj}`pip.parse.config_settings` is a list of arbitrary `config_setting` +targets, you can define your own flags or implement custom config matching +logic. This allows you to model settings that aren't inherently part of +rules_python. + +This is typically done using [bazel_skylib flags](https://bazel.build/extending/config), but any [Starlark +defined build setting](https://bazel.build/extending/config) can be used. Just +remember to use `config_setting()` to match a particular value of the flag. + +In our example below, we define a custom flag for CUDA version. + +#### Predefined and common build settings + +rules_python has some predefined build settings you can use. Commonly used ones +are: + +* {obj}`@rules_python//python/config_settings:py_linux_libc` +* {obj}`@rules_python//python/config_settings:py_freethreaded` + +Additionally, [Bazel @platforms](https://github.com/bazelbuild/platforms) +contains commonly used settings for OS and CPU: + +* `@platforms//os:windows` +* `@platforms//os:linux` +* `@platforms//os:osx` +* `@platforms//cpu:x86_64` +* `@platforms//cpu:aarch64` + +Note that these are the raw flag names. In order to use them with `pip.default`, +you must use {obj}`config_setting()` to match a particular value for them. + +### Associating Requirements to Platforms + +Next, we associate a requirements file with a platform using +{obj}`pip.parse.requirements_by_platform`. This is a dictionary attribute where +the keys are requirements files and the value is a platform name. The platform +value can use a trailing or leading `*` to match multiple platforms. It can also +specify multiple platform names using commas to separate them. + +Note that the Python version is _not_ part of the platform name. + +Under the hood, `pip.parse` merges all the requirements (for a `hub_name`) and +constructs `select()` expressions to route to the appropriate dependencies. + +### Using it in practice + +Finally, to make use of what we've configured, perform a build and set +command line flags to the appropriate values. + +```shell +# Build for CUDA +bazel build --//:cuda_version=12.9 //:binary + +# Build for ARM with musl +bazel build --@rules_python//python/config_settings:py_linux_libc=musl \ + --cpu=aarch64 //:binary + +# Build for freethreaded +bazel build --@rules_python//python/config_settings:py_freethreaded=yes //:binary +``` + +Note that certain combinations of flags may result in an error or undefined +behavior. For example, trying to set both freethreaded and CUDA at the same +time would result in an error because no requirements file was registered +to match that combination. + +## Multiple Python Versions + +Having multiple Python versions is fully supported. Simply add a `pip.parse()` +call and set `python_version` appropriately. + +## Multiple hubs + +Having multiple `pip.parse` calls with different `hub_name` values is fully +supported. Each hub only contains the requirements registered to it. + +## Complete Example + +Here is a complete example that puts all the pieces together. + +```starlark +# File: BUILD.bazel +load("@bazel_skylib//rules:common_settings.bzl", "string_flag") + +# A custom flag for controlling the CUDA version +string_flag( + name = "cuda_version", + build_setting_default = "none", +) + +config_setting( + name = "is_cuda_12_9", + flag_values = {":cuda_version": "12.9"}, +) + +# A config_setting that uses the built-in libc flag from rules_python +config_setting( + name = "is_musl", + flag_values = {"@rules_python//python/config_settings:py_linux_libc": "muslc"}, +) + +# File: MODULE.bazel +pip = use_extension("@rules_python//python/extensions:pip.bzl", "pip") + +# A custom platform for CUDA on glibc linux +pip.default( + platform = "linux_x86_64_cuda12.9", + os = "linux", + cpu = "x86_64", + config_settings = ["@//:is_cuda_12_9"], +) + +# A custom platform for musl on linux +pip.default( + platform = "linux_aarch64_musl", + os = "linux", + cpu = "aarch64", + config_settings = ["@//:is_musl"], +) + +pip.parse( + hub_name = "my_deps", + python_version = "3.14", + requirements_by_platform = { + # Map to default platform names + "//:py3.14-regular-linux-x86-glibc-cpu.txt": "linux_x86_64", + "//:py3.14-freethreaded-linux-x86-glibc-cpu.txt": "linux_x86_64_freethreaded", + + # Map to our custom platform names + "//:py3.14-regular-linux-x86-glibc-cuda12.9.txt": "linux_x86_64_cuda12.9", + "//:py3.14-freethreaded-linux-arm-musl-cpu.txt": "linux_aarch64_musl", + }, +) + +use_repo(pip, "my_deps") +``` diff --git a/docs/pypi/index.md b/docs/pypi/index.md index c32bafc609..17928898c5 100644 --- a/docs/pypi/index.md +++ b/docs/pypi/index.md @@ -11,6 +11,7 @@ Using PyPI packages (aka "pip install") involves the following main steps: With the advanced topics covered separately: * Dealing with [circular dependencies](./circular-dependencies). +* Handling [multi-platform dependencies](../howto/multi-platform-pypi-deps). ```{toctree} lock @@ -22,6 +23,9 @@ use ## Advanced topics ```{toctree} +:maxdepth: 1 + circular-dependencies patch +../howto/multi-platform-pypi-deps ``` diff --git a/python/private/pypi/pip_repository.bzl b/python/private/pypi/pip_repository.bzl index 6d539a5f24..2cf20cd5a7 100644 --- a/python/private/pypi/pip_repository.bzl +++ b/python/private/pypi/pip_repository.bzl @@ -266,6 +266,9 @@ code will be re-evaluated when any of files in the default changes. Those dependencies become available in a generated `requirements.bzl` file. You can instead check this `requirements.bzl` file into your repo, see the "vendoring" section below. +For advanced use-cases, such as handling multi-platform dependencies, see the +[How-to: Multi-Platform PyPI Dependencies guide](/howto/multi-platform-pypi-deps). + In your WORKSPACE file: ```starlark From f2668295be2f73623ea0ed62f2942ecfd448c8d2 Mon Sep 17 00:00:00 2001 From: Jeff Klukas Date: Fri, 26 Sep 2025 19:27:48 -0400 Subject: [PATCH 454/922] feat(runfiles): support for --incompatible_compact_repo_mapping_manifest (#3277) Under bzlmod, the repo mapping can become quite large (i.e. tens of megabytes) because its size scales as a factor of the number of repos in the transitive dependencies. To address this, the --incompatible_compact_repo_mapping_manifest flag was introduced. This changes the repo mapping formation to use prefixes (instead of exact repo names) for mapping things. To make this work with the runfiles library, the code has to be updated to handle these prefixes instead of just exact strings. Fixes #3022. --------- Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> Co-authored-by: Richard Levasseur --- CHANGELOG.md | 2 + python/runfiles/runfiles.py | 162 +++++++++++++++++++++++++------- tests/runfiles/runfiles_test.py | 160 +++++++++++++++++++++++++++++++ 3 files changed, 292 insertions(+), 32 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 3d06a68935..469e9d3612 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -85,6 +85,8 @@ END_UNRELEASED_TEMPLATE {#v0-0-0-added} ### Added +* (runfiles) The Python runfiles library now supports Bazel's + `--incompatible_compact_repo_mapping_manifest` flag. * (bootstrap) {obj}`--bootstrap_impl=system_python` now supports the {obj}`main_module` attribute. * (bootstrap) {obj}`--bootstrap_impl=system_python` now supports the diff --git a/python/runfiles/runfiles.py b/python/runfiles/runfiles.py index 3943be5646..58f59c5406 100644 --- a/python/runfiles/runfiles.py +++ b/python/runfiles/runfiles.py @@ -15,14 +15,131 @@ """Runfiles lookup library for Bazel-built Python binaries and tests. See @rules_python//python/runfiles/README.md for usage instructions. + +:::{versionadded} VERSION_NEXT_FEATURE +Support for Bazel's `--incompatible_compact_repo_mapping_manifest` flag was added. +This enables prefix-based repository mappings to reduce memory usage for large +dependency graphs under bzlmod. +::: """ +import collections.abc import inspect import os import posixpath import sys +from collections import defaultdict from typing import Dict, Optional, Tuple, Union +class _RepositoryMapping: + """Repository mapping for resolving apparent repository names to canonical ones. + + Handles both exact mappings and prefix-based mappings introduced by the + --incompatible_compact_repo_mapping_manifest flag. + """ + + def __init__( + self, + exact_mappings: Dict[Tuple[str, str], str], + prefixed_mappings: Dict[Tuple[str, str], str], + ) -> None: + """Initialize repository mapping with exact and prefixed mappings. + + Args: + exact_mappings: Dict mapping (source_canonical, target_apparent) -> target_canonical + prefixed_mappings: Dict mapping (source_prefix, target_apparent) -> target_canonical + """ + self._exact_mappings = exact_mappings + + # Group prefixed mappings by target_apparent for faster lookups + self._grouped_prefixed_mappings = defaultdict(list) + for ( + prefix_source, + target_app, + ), target_canonical in prefixed_mappings.items(): + self._grouped_prefixed_mappings[target_app].append( + (prefix_source, target_canonical) + ) + + @staticmethod + def create_from_file(repo_mapping_path: Optional[str]) -> "_RepositoryMapping": + """Create RepositoryMapping from a repository mapping manifest file. + + Args: + repo_mapping_path: Path to the repository mapping file, or None if not available + + Returns: + RepositoryMapping instance with parsed mappings + """ + # If the repository mapping file can't be found, that is not an error: We + # might be running without Bzlmod enabled or there may not be any runfiles. + # In this case, just apply empty repo mappings. + if not repo_mapping_path: + return _RepositoryMapping({}, {}) + + try: + with open(repo_mapping_path, "r", encoding="utf-8", newline="\n") as f: + content = f.read() + except FileNotFoundError: + return _RepositoryMapping({}, {}) + + exact_mappings = {} + prefixed_mappings = {} + for line in content.splitlines(): + source_canonical, target_apparent, target_canonical = line.split(",") + if source_canonical.endswith("*"): + # This is a prefixed mapping - remove the '*' for prefix matching + prefix = source_canonical[:-1] + prefixed_mappings[(prefix, target_apparent)] = target_canonical + else: + # This is an exact mapping + exact_mappings[(source_canonical, target_apparent)] = target_canonical + + return _RepositoryMapping(exact_mappings, prefixed_mappings) + + def lookup(self, source_repo: Optional[str], target_apparent: str) -> Optional[str]: + """Look up repository mapping for the given source and target. + + This handles both exact mappings and prefix-based mappings introduced by the + --incompatible_compact_repo_mapping_manifest flag. Exact mappings are tried + first, followed by prefix-based mappings where order matters. + + Args: + source_repo: Source canonical repository name + target_apparent: Target apparent repository name + + Returns: + target_canonical repository name, or None if no mapping exists + """ + if source_repo is None: + return None + + key = (source_repo, target_apparent) + + # Try exact mapping first + if key in self._exact_mappings: + return self._exact_mappings[key] + + # Try prefixed mapping if no exact match found + if target_apparent in self._grouped_prefixed_mappings: + for prefix_source, target_canonical in self._grouped_prefixed_mappings[ + target_apparent + ]: + if source_repo.startswith(prefix_source): + return target_canonical + + # No mapping found + return None + + def is_empty(self) -> bool: + """Check if this repository mapping is empty (no exact or prefixed mappings). + + Returns: + True if there are no mappings, False otherwise + """ + return len(self._exact_mappings) == 0 and len(self._grouped_prefixed_mappings) == 0 + + class _ManifestBased: """`Runfiles` strategy that parses a runfiles-manifest to look up runfiles.""" @@ -130,7 +247,7 @@ class Runfiles: def __init__(self, strategy: Union[_ManifestBased, _DirectoryBased]) -> None: self._strategy = strategy self._python_runfiles_root = _FindPythonRunfilesRoot() - self._repo_mapping = _ParseRepoMapping( + self._repo_mapping = _RepositoryMapping.create_from_file( strategy.RlocationChecked("_repo_mapping") ) @@ -179,7 +296,7 @@ def Rlocation(self, path: str, source_repo: Optional[str] = None) -> Optional[st if os.path.isabs(path): return path - if source_repo is None and self._repo_mapping: + if source_repo is None and not self._repo_mapping.is_empty(): # Look up runfiles using the repository mapping of the caller of the # current method. If the repo mapping is empty, determining this # name is not necessary. @@ -188,7 +305,8 @@ def Rlocation(self, path: str, source_repo: Optional[str] = None) -> Optional[st # Split off the first path component, which contains the repository # name (apparent or canonical). target_repo, _, remainder = path.partition("/") - if not remainder or (source_repo, target_repo) not in self._repo_mapping: + target_canonical = self._repo_mapping.lookup(source_repo, target_repo) + if not remainder or target_canonical is None: # One of the following is the case: # - not using Bzlmod, so the repository mapping is empty and # apparent and canonical repository names are the same @@ -202,11 +320,15 @@ def Rlocation(self, path: str, source_repo: Optional[str] = None) -> Optional[st source_repo is not None ), "BUG: if the `source_repo` is None, we should never go past the `if` statement above" - # target_repo is an apparent repository name. Look up the corresponding - # canonical repository name with respect to the current repository, - # identified by its canonical name. - target_canonical = self._repo_mapping[(source_repo, target_repo)] - return self._strategy.RlocationChecked(target_canonical + "/" + remainder) + # Look up the target repository using the repository mapping + if target_canonical is not None: + return self._strategy.RlocationChecked( + target_canonical + "/" + remainder + ) + + # No mapping found - assume target_repo is already canonical or + # we're not using Bzlmod + return self._strategy.RlocationChecked(path) def EnvVars(self) -> Dict[str, str]: """Returns environment variables for subprocesses. @@ -359,30 +481,6 @@ def _FindPythonRunfilesRoot() -> str: return root -def _ParseRepoMapping(repo_mapping_path: Optional[str]) -> Dict[Tuple[str, str], str]: - """Parses the repository mapping manifest.""" - # If the repository mapping file can't be found, that is not an error: We - # might be running without Bzlmod enabled or there may not be any runfiles. - # In this case, just apply an empty repo mapping. - if not repo_mapping_path: - return {} - try: - with open(repo_mapping_path, "r", encoding="utf-8", newline="\n") as f: - content = f.read() - except FileNotFoundError: - return {} - - repo_mapping = {} - for line in content.split("\n"): - if not line: - # Empty line following the last line break - break - current_canonical, target_local, target_canonical = line.split(",") - repo_mapping[(current_canonical, target_local)] = target_canonical - - return repo_mapping - - def CreateManifestBased(manifest_path: str) -> Runfiles: return Runfiles.CreateManifestBased(manifest_path) diff --git a/tests/runfiles/runfiles_test.py b/tests/runfiles/runfiles_test.py index a3837ac842..b8a3d5f7b7 100644 --- a/tests/runfiles/runfiles_test.py +++ b/tests/runfiles/runfiles_test.py @@ -18,6 +18,7 @@ from typing import Any, List, Optional from python.runfiles import runfiles +from python.runfiles.runfiles import _RepositoryMapping class RunfilesTest(unittest.TestCase): @@ -525,6 +526,165 @@ def testDirectoryBasedRlocationWithRepoMappingFromOtherRepo(self) -> None: r.Rlocation("config.json", "protobuf~3.19.2"), dir + "/config.json" ) + def testDirectoryBasedRlocationWithCompactRepoMappingFromMain(self) -> None: + """Test repository mapping with prefix-based entries (compact format).""" + with _MockFile( + name="_repo_mapping", + contents=[ + # Exact mappings (no asterisk) + "_,config.json,config.json~1.2.3", + ",my_module,_main", + ",my_workspace,_main", + # Prefixed mappings (with asterisk) - these apply to any repo starting with the prefix + "deps+*,external_dep,external_dep~1.0.0", + "test_deps+*,test_lib,test_lib~2.1.0", + ], + ) as rm: + dir = os.path.dirname(rm.Path()) + r = runfiles.CreateDirectoryBased(dir) + + # Test exact mappings still work + self.assertEqual( + r.Rlocation("my_module/bar/runfile", ""), dir + "/_main/bar/runfile" + ) + self.assertEqual( + r.Rlocation("my_workspace/bar/runfile", ""), dir + "/_main/bar/runfile" + ) + + # Test prefixed mappings - should match any repo starting with "deps+" + self.assertEqual( + r.Rlocation("external_dep/foo/file", "deps+dep1"), + dir + "/external_dep~1.0.0/foo/file", + ) + self.assertEqual( + r.Rlocation("external_dep/bar/file", "deps+dep2"), + dir + "/external_dep~1.0.0/bar/file", + ) + self.assertEqual( + r.Rlocation("external_dep/nested/path/file", "deps+some_long_dep_name"), + dir + "/external_dep~1.0.0/nested/path/file", + ) + + # Test that prefixed mappings work for test_deps+ prefix too + self.assertEqual( + r.Rlocation("test_lib/test/file", "test_deps+junit"), + dir + "/test_lib~2.1.0/test/file", + ) + + # Test that non-matching prefixes don't match + self.assertEqual( + r.Rlocation("external_dep/foo/file", "other_prefix"), + dir + "/external_dep/foo/file", # No mapping applied, use as-is + ) + + def testDirectoryBasedRlocationWithCompactRepoMappingPrecedence(self) -> None: + """Test that exact mappings take precedence over prefixed mappings.""" + with _MockFile( + name="_repo_mapping", + contents=[ + # Exact mapping for a specific source repo + "deps+specific_repo,external_dep,external_dep~exact", + # Prefixed mapping for repos starting with "deps+" + "deps+*,external_dep,external_dep~prefix", + # Another prefixed mapping with different prefix + "other+*,external_dep,external_dep~other", + ], + ) as rm: + dir = os.path.dirname(rm.Path()) + r = runfiles.CreateDirectoryBased(dir) + + # Exact mapping should take precedence over prefix + self.assertEqual( + r.Rlocation("external_dep/foo/file", "deps+specific_repo"), + dir + "/external_dep~exact/foo/file", + ) + + # Other repos with deps+ prefix should use the prefixed mapping + self.assertEqual( + r.Rlocation("external_dep/foo/file", "deps+other_repo"), + dir + "/external_dep~prefix/foo/file", + ) + + # Different prefix should use its own mapping + self.assertEqual( + r.Rlocation("external_dep/foo/file", "other+some_repo"), + dir + "/external_dep~other/foo/file", + ) + + def testDirectoryBasedRlocationWithCompactRepoMappingOrderMatters(self) -> None: + """Test that order matters for prefixed mappings (first match wins).""" + with _MockFile( + name="_repo_mapping", + contents=[ + # More specific prefix comes first + "deps+specific+*,lib,lib~specific", + # More general prefix comes second + "deps+*,lib,lib~general", + ], + ) as rm: + dir = os.path.dirname(rm.Path()) + r = runfiles.CreateDirectoryBased(dir) + + # Should match the more specific prefix first + self.assertEqual( + r.Rlocation("lib/foo/file", "deps+specific+repo"), + dir + "/lib~specific/foo/file", + ) + + # Should match the general prefix for non-specific repos + self.assertEqual( + r.Rlocation("lib/foo/file", "deps+other_repo"), + dir + "/lib~general/foo/file", + ) + + def testRepositoryMappingLookup(self) -> None: + """Test _RepositoryMapping.lookup() method for both exact and prefix-based mappings.""" + exact_mappings = { + ("", "my_workspace"): "_main", + ("", "config_lib"): "config_lib~1.0.0", + ("deps+specific_repo", "external_dep"): "external_dep~exact", + } + prefixed_mappings = { + ("deps+", "external_dep"): "external_dep~prefix", + ("test_deps+", "test_lib"): "test_lib~2.1.0", + } + + repo_mapping = _RepositoryMapping(exact_mappings, prefixed_mappings) + + # Test exact lookups + self.assertEqual(repo_mapping.lookup("", "my_workspace"), "_main") + self.assertEqual(repo_mapping.lookup("", "config_lib"), "config_lib~1.0.0") + self.assertEqual( + repo_mapping.lookup("deps+specific_repo", "external_dep"), + "external_dep~exact", + ) + + # Test prefix-based lookups + self.assertEqual( + repo_mapping.lookup("deps+some_repo", "external_dep"), "external_dep~prefix" + ) + self.assertEqual( + repo_mapping.lookup("test_deps+another_repo", "test_lib"), "test_lib~2.1.0" + ) + + # Test that exact takes precedence over prefix + self.assertEqual( + repo_mapping.lookup("deps+specific_repo", "external_dep"), + "external_dep~exact", + ) + + # Test non-existent mapping + self.assertIsNone(repo_mapping.lookup("nonexistent", "repo")) + self.assertIsNone(repo_mapping.lookup("unknown+repo", "missing")) + + # Test empty mapping + empty_mapping = _RepositoryMapping({}, {}) + self.assertIsNone(empty_mapping.lookup("any", "repo")) + + # Test is_empty() method + self.assertFalse(repo_mapping.is_empty()) # Should have mappings + self.assertTrue(empty_mapping.is_empty()) # Should be empty + def testCurrentRepository(self) -> None: # Under bzlmod, the current repository name is the empty string instead # of the name in the workspace file. From 726ffa27b6698a7544e4c6825f0626bbbb07a983 Mon Sep 17 00:00:00 2001 From: Ignas Anikevicius <240938+aignas@users.noreply.github.com> Date: Sun, 28 Sep 2025 10:24:39 +0900 Subject: [PATCH 455/922] chore: cleanup bazel flags related to bazel 6 or below (#3282) Summary: * refactor: use rules_shell runfiles lib * refactor: remove watch helpers * refactor: remove usage of select helper * refactor: make enable_pystar fixed and cleanup code * refactor: remove migration tag helper * refactor: remove is_bazel_6_or_higher * refactor: remove is_bazel_6_4_or_higher * refactor: remove is_bazel_7_or_greater * remove: is_bazel_7_4_or_greater * fix: pipstar env var is now respected * chore: drop bazel 5 support code * chore: add an override for bzlmod example * chore: remove version specific globs, since the supported versions support spaces in filenames and the file becomes redundant. --- docs/BUILD.bazel | 7 +- examples/bzlmod/other_module/MODULE.bazel | 18 +-- python/BUILD.bazel | 17 --- python/features.bzl | 4 +- python/private/BUILD.bazel | 12 -- python/private/glob_excludes.bzl | 32 ------ .../private/hermetic_runtime_repo_setup.bzl | 3 +- python/private/internal_config_repo.bzl | 33 +----- python/private/local_runtime_repo.bzl | 6 +- python/private/py_cc_link_params_info.bzl | 3 +- python/private/py_info.bzl | 31 ++--- python/private/py_package.bzl | 9 +- python/private/py_runtime_info.bzl | 4 +- python/private/py_runtime_pair_rule.bzl | 3 +- python/private/py_runtime_rule.bzl | 10 +- python/private/pypi/BUILD.bazel | 3 +- python/private/pypi/deps.bzl | 3 +- python/private/pypi/pypi_repo_utils.bzl | 5 +- python/private/pypi/whl_library_targets.bzl | 15 +-- python/private/python.bzl | 11 +- python/private/python_register_toolchains.bzl | 15 --- python/private/repo_utils.bzl | 22 +--- python/private/util.bzl | 50 --------- python/py_binary.bzl | 6 +- python/py_cc_link_params_info.bzl | 7 +- python/py_info.bzl | 4 +- python/py_library.bzl | 6 +- python/py_runtime.bzl | 5 +- python/py_runtime_info.bzl | 4 +- python/py_runtime_pair.bzl | 4 +- python/py_test.bzl | 6 +- .../tests/proto_to_markdown/BUILD.bazel | 2 - sphinxdocs/tests/sphinx_docs/BUILD.bazel | 3 +- sphinxdocs/tests/sphinx_stardoc/BUILD.bazel | 3 +- tests/api/py_common/py_common_tests.bzl | 10 +- .../precompile/precompile_tests.bzl | 34 ------ tests/base_rules/py_executable_base_tests.bzl | 73 ++++-------- tests/base_rules/py_info/py_info_tests.bzl | 17 +-- tests/base_rules/py_test/py_test_tests.bzl | 15 --- tests/base_rules/util.bzl | 5 - tests/bootstrap_impls/a/b/c/BUILD.bazel | 3 +- .../transition/multi_version_tests.bzl | 50 ++++----- .../exec_toolchain_matching_tests.bzl | 8 +- tests/integration/integration_test.bzl | 6 - tests/py_runtime/py_runtime_tests.bzl | 106 ++++-------------- .../py_runtime_info/py_runtime_info_tests.bzl | 7 +- .../whl_library_targets_tests.bzl | 29 ++--- .../runtime_env_toolchain_tests.bzl | 22 +--- tests/support/support.bzl | 3 +- .../whl_from_dir/whl_from_dir_repo.bzl | 2 +- 50 files changed, 146 insertions(+), 610 deletions(-) delete mode 100644 python/private/glob_excludes.bzl diff --git a/docs/BUILD.bazel b/docs/BUILD.bazel index b6c48b0539..c36ed5722a 100644 --- a/docs/BUILD.bazel +++ b/docs/BUILD.bazel @@ -15,7 +15,6 @@ load("@bazel_skylib//rules:build_test.bzl", "build_test") load("@dev_pip//:requirements.bzl", "requirement") load("//python/private:bzlmod_enabled.bzl", "BZLMOD_ENABLED") # buildifier: disable=bzl-visibility -load("//python/private:util.bzl", "IS_BAZEL_7_OR_HIGHER") # buildifier: disable=bzl-visibility load("//python/uv:lock.bzl", "lock") # buildifier: disable=bzl-visibility load("//sphinxdocs:readthedocs.bzl", "readthedocs_install") load("//sphinxdocs:sphinx.bzl", "sphinx_build_binary", "sphinx_docs") @@ -107,6 +106,7 @@ sphinx_stardocs( "//python/cc:py_cc_toolchain_bzl", "//python/cc:py_cc_toolchain_info_bzl", "//python/entry_points:py_console_script_binary_bzl", + "//python/extensions:python_bzl", "//python/local_toolchains:repos_bzl", "//python/private:attr_builders_bzl", "//python/private:builders_util_bzl", @@ -128,12 +128,9 @@ sphinx_stardocs( "//python/uv:uv_toolchain_bzl", "//python/uv:uv_toolchain_info_bzl", ] + ([ - # Bazel 6 + Stardoc isn't able to parse something about the python bzlmod extension - "//python/extensions:python_bzl", - ] if IS_BAZEL_7_OR_HIGHER else []) + ([ # This depends on @pythons_hub, which is only created under bzlmod, "//python/extensions:pip_bzl", - ] if IS_BAZEL_7_OR_HIGHER and BZLMOD_ENABLED else []), + ] if BZLMOD_ENABLED else []), prefix = "api/rules_python/", tags = ["docs"], target_compatible_with = _TARGET_COMPATIBLE_WITH, diff --git a/examples/bzlmod/other_module/MODULE.bazel b/examples/bzlmod/other_module/MODULE.bazel index f9d6706120..7b88bd73ff 100644 --- a/examples/bzlmod/other_module/MODULE.bazel +++ b/examples/bzlmod/other_module/MODULE.bazel @@ -5,20 +5,10 @@ module( # This module is using the same version of rules_python # that the parent module uses. bazel_dep(name = "rules_python", version = "") - -# The story behind this commented out override: -# This override is necessary to generate/update the requirements file -# for this module. This is because running it via the outer -# module doesn't work -- the `requirements.update` target can't find -# the correct file to update. -# Running in the submodule itself works, but submodules using overrides -# is considered an error until Bazel 6.3, which prevents the outer module -# from depending on this module. -# So until 6.3 and higher is the minimum, we leave this commented out. -# local_path_override( -# module_name = "rules_python", -# path = "../../..", -# ) +local_path_override( + module_name = "rules_python", + path = "../../..", +) PYTHON_NAME_39 = "python_3_9" diff --git a/python/BUILD.bazel b/python/BUILD.bazel index 76fa5dde6e..5fc35f8357 100644 --- a/python/BUILD.bazel +++ b/python/BUILD.bazel @@ -81,9 +81,6 @@ bzl_library( bzl_library( name = "features_bzl", srcs = ["features.bzl"], - deps = [ - "@rules_python_internal//:rules_python_config_bzl", - ], ) bzl_library( @@ -130,8 +127,6 @@ bzl_library( deps = [ "//python/private:py_binary_macro_bzl", "//python/private:register_extension_info_bzl", - "//python/private:util_bzl", - "@rules_python_internal//:rules_python_config_bzl", ], ) @@ -140,7 +135,6 @@ bzl_library( srcs = ["py_cc_link_params_info.bzl"], deps = [ "//python/private:py_cc_link_params_info_bzl", - "@rules_python_internal//:rules_python_config_bzl", ], ) @@ -173,8 +167,6 @@ bzl_library( srcs = ["py_info.bzl"], deps = [ "//python/private:py_info_bzl", - "//python/private:reexports_bzl", - "@rules_python_internal//:rules_python_config_bzl", ], ) @@ -184,8 +176,6 @@ bzl_library( deps = [ "//python/private:py_library_macro_bzl", "//python/private:register_extension_info_bzl", - "//python/private:util_bzl", - "@rules_python_internal//:rules_python_config_bzl", ], ) @@ -194,7 +184,6 @@ bzl_library( srcs = ["py_runtime.bzl"], deps = [ "//python/private:py_runtime_macro_bzl", - "//python/private:util_bzl", ], ) @@ -202,9 +191,7 @@ bzl_library( name = "py_runtime_pair_bzl", srcs = ["py_runtime_pair.bzl"], deps = [ - "//python/private:bazel_tools_bzl", "//python/private:py_runtime_pair_macro_bzl", - "//python/private:util_bzl", ], ) @@ -213,8 +200,6 @@ bzl_library( srcs = ["py_runtime_info.bzl"], deps = [ "//python/private:py_runtime_info_bzl", - "//python/private:reexports_bzl", - "@rules_python_internal//:rules_python_config_bzl", ], ) @@ -224,8 +209,6 @@ bzl_library( deps = [ "//python/private:py_test_macro_bzl", "//python/private:register_extension_info_bzl", - "//python/private:util_bzl", - "@rules_python_internal//:rules_python_config_bzl", ], ) diff --git a/python/features.bzl b/python/features.bzl index 21ff588dca..00bc1a7817 100644 --- a/python/features.bzl +++ b/python/features.bzl @@ -13,8 +13,6 @@ # limitations under the License. """Allows detecting of rules_python features that aren't easily detected.""" -load("@rules_python_internal//:rules_python_config.bzl", "config") - # This is a magic string expanded by `git archive`, as set by `.gitattributes` # See https://git-scm.com/docs/git-archive/2.29.0#Documentation/git-archive.txt-export-subst _VERSION_PRIVATE = "$Format:%(describe:tags=true)$" @@ -73,6 +71,6 @@ features = struct( headers_abi3 = True, precompile = True, py_info_venv_symlinks = True, - uses_builtin_rules = not config.enable_pystar, + uses_builtin_rules = False, version = _VERSION_PRIVATE if "$Format" not in _VERSION_PRIVATE else "", ) diff --git a/python/private/BUILD.bazel b/python/private/BUILD.bazel index 0c8ccdea99..1bcd0f678f 100644 --- a/python/private/BUILD.bazel +++ b/python/private/BUILD.bazel @@ -194,12 +194,6 @@ bzl_library( srcs = ["full_version.bzl"], ) -bzl_library( - name = "glob_excludes_bzl", - srcs = ["glob_excludes.bzl"], - deps = [":util_bzl"], -) - bzl_library( name = "internal_config_repo_bzl", srcs = ["internal_config_repo.bzl"], @@ -264,7 +258,6 @@ bzl_library( ":pythons_hub_bzl", ":repo_utils_bzl", ":toolchains_repo_bzl", - ":util_bzl", ":version_bzl", "@bazel_features//:features", ], @@ -440,8 +433,6 @@ bzl_library( deps = [ ":builders_bzl", ":reexports_bzl", - ":util_bzl", - "@rules_python_internal//:rules_python_config_bzl", ], ) @@ -506,7 +497,6 @@ bzl_library( bzl_library( name = "py_runtime_info_bzl", srcs = ["py_runtime_info.bzl"], - deps = [":util_bzl"], ) bzl_library( @@ -538,7 +528,6 @@ bzl_library( ":py_runtime_info_bzl", ":reexports_bzl", ":rule_builders_bzl", - ":util_bzl", "@bazel_skylib//lib:dicts", "@bazel_skylib//lib:paths", "@bazel_skylib//rules:common_settings", @@ -688,7 +677,6 @@ bzl_library( ], deps = [ "@bazel_skylib//lib:types", - "@rules_python_internal//:rules_python_config_bzl", ], ) diff --git a/python/private/glob_excludes.bzl b/python/private/glob_excludes.bzl deleted file mode 100644 index c98afe0ae2..0000000000 --- a/python/private/glob_excludes.bzl +++ /dev/null @@ -1,32 +0,0 @@ -# Copyright 2024 The Bazel Authors. All rights reserved. -# -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -"Utilities for glob exclusions." - -load(":util.bzl", "IS_BAZEL_7_4_OR_HIGHER") - -def _version_dependent_exclusions(): - """Returns glob exclusions that are sensitive to Bazel version. - - Returns: - a list of glob exclusion patterns - """ - if IS_BAZEL_7_4_OR_HIGHER: - return [] - else: - return ["**/* *"] - -glob_excludes = struct( - version_dependent_exclusions = _version_dependent_exclusions, -) diff --git a/python/private/hermetic_runtime_repo_setup.bzl b/python/private/hermetic_runtime_repo_setup.bzl index a35ce8ae7d..46495e49c0 100644 --- a/python/private/hermetic_runtime_repo_setup.bzl +++ b/python/private/hermetic_runtime_repo_setup.bzl @@ -18,7 +18,6 @@ load("@rules_cc//cc:cc_library.bzl", "cc_library") load("//python:py_runtime.bzl", "py_runtime") load("//python:py_runtime_pair.bzl", "py_runtime_pair") load("//python/cc:py_cc_toolchain.bzl", "py_cc_toolchain") -load(":glob_excludes.bzl", "glob_excludes") load(":py_exec_tools_toolchain.bzl", "py_exec_tools_toolchain") load(":version.bzl", "version") @@ -82,7 +81,7 @@ def define_hermetic_runtime_toolchain_impl( "lib/python{major}.{minor}*/**/tests/**".format(**version_dict), # During pyc creation, temp files named *.pyc.NNN are created "**/__pycache__/*.pyc.*", - ] + glob_excludes.version_dependent_exclusions() + extra_files_glob_exclude, + ] + extra_files_glob_exclude, ), ) cc_import( diff --git a/python/private/internal_config_repo.bzl b/python/private/internal_config_repo.bzl index b57275b672..109e68a8a1 100644 --- a/python/private/internal_config_repo.bzl +++ b/python/private/internal_config_repo.bzl @@ -23,14 +23,12 @@ load(":repo_utils.bzl", "repo_utils") _ENABLE_PIPSTAR_ENVVAR_NAME = "RULES_PYTHON_ENABLE_PIPSTAR" _ENABLE_PIPSTAR_DEFAULT = "0" -_ENABLE_PYSTAR_ENVVAR_NAME = "RULES_PYTHON_ENABLE_PYSTAR" -_ENABLE_PYSTAR_DEFAULT = "1" _ENABLE_DEPRECATION_WARNINGS_ENVVAR_NAME = "RULES_PYTHON_DEPRECATION_WARNINGS" _ENABLE_DEPRECATION_WARNINGS_DEFAULT = "0" _CONFIG_TEMPLATE = """ config = struct( - enable_pystar = {enable_pystar}, + enable_pystar = True, enable_pipstar = {enable_pipstar}, enable_deprecation_warnings = {enable_deprecation_warnings}, BuiltinPyInfo = getattr(getattr(native, "legacy_globals", None), "PyInfo", {builtin_py_info_symbol}), @@ -88,15 +86,6 @@ _TRANSITION_SETTINGS_DEBUG_TEMPLATE = """ """ def _internal_config_repo_impl(rctx): - pystar_requested = _bool_from_environ(rctx, _ENABLE_PYSTAR_ENVVAR_NAME, _ENABLE_PYSTAR_DEFAULT) - - # Bazel 7+ (dev and later) has native.starlark_doc_extract, and thus the - # py_internal global, which are necessary for the pystar implementation. - if pystar_requested and hasattr(native, "starlark_doc_extract"): - enable_pystar = pystar_requested - else: - enable_pystar = False - if not native.bazel_version or int(native.bazel_version.split(".")[0]) >= 8: builtin_py_info_symbol = "None" builtin_py_runtime_info_symbol = "None" @@ -107,7 +96,6 @@ def _internal_config_repo_impl(rctx): builtin_py_cc_link_params_provider = "PyCcLinkParamsProvider" rctx.file("rules_python_config.bzl", _CONFIG_TEMPLATE.format( - enable_pystar = enable_pystar, enable_pipstar = _bool_from_environ(rctx, _ENABLE_PIPSTAR_ENVVAR_NAME, _ENABLE_PIPSTAR_DEFAULT), enable_deprecation_warnings = _bool_from_environ(rctx, _ENABLE_DEPRECATION_WARNINGS_ENVVAR_NAME, _ENABLE_DEPRECATION_WARNINGS_DEFAULT), builtin_py_info_symbol = builtin_py_info_symbol, @@ -115,23 +103,12 @@ def _internal_config_repo_impl(rctx): builtin_py_cc_link_params_provider = builtin_py_cc_link_params_provider, )) - if enable_pystar: - shim_content = _PY_INTERNAL_SHIM - py_internal_dep = '"@rules_python//tools/build_defs/python/private:py_internal_renamed_bzl"' - else: - shim_content = "py_internal_impl = None\n" - py_internal_dep = "" - - # Bazel 5 doesn't support repository visibility, so just use public - # as a stand-in - if native.bazel_version.startswith("5."): - visibility = "//visibility:public" - else: - visibility = "@rules_python//:__subpackages__" + shim_content = _PY_INTERNAL_SHIM + py_internal_dep = '"@rules_python//tools/build_defs/python/private:py_internal_renamed_bzl"' rctx.file("BUILD", ROOT_BUILD_TEMPLATE.format( py_internal_dep = py_internal_dep, - visibility = visibility, + visibility = "@rules_python//:__subpackages__", )) rctx.file("py_internal.bzl", shim_content) @@ -155,7 +132,7 @@ def _internal_config_repo_impl(rctx): internal_config_repo = repository_rule( implementation = _internal_config_repo_impl, configure = True, - environ = [_ENABLE_PYSTAR_ENVVAR_NAME], + environ = [_ENABLE_PIPSTAR_ENVVAR_NAME], attrs = { "transition_setting_generators": attr.string_list_dict(), "transition_settings": attr.string_list(), diff --git a/python/private/local_runtime_repo.bzl b/python/private/local_runtime_repo.bzl index c053a03508..27c90b1bc9 100644 --- a/python/private/local_runtime_repo.bzl +++ b/python/private/local_runtime_repo.bzl @@ -71,7 +71,7 @@ def _symlink_first_library(rctx, logger, libraries): else: linked = "lib/{}".format(origin.basename) logger.debug("Symlinking {} to {}".format(origin, linked)) - repo_utils.watch(rctx, origin) + rctx.watch(origin) rctx.symlink(origin, linked) break @@ -142,7 +142,7 @@ def _local_runtime_repo_impl(rctx): # path is an error. Silently skip, since includes are only necessary # if C extensions are built. if include_path.exists and include_path.is_dir: - repo_utils.watch_tree(rctx, include_path) + rctx.watch_tree(include_path) else: pass @@ -268,7 +268,7 @@ def _resolve_interpreter_path(rctx): resolved_path = result.binary describe_failure = result.describe_failure else: - repo_utils.watch(rctx, rctx.attr.interpreter_path) + rctx.watch(rctx.attr.interpreter_path) resolved_path = rctx.path(rctx.attr.interpreter_path) if not resolved_path.exists: describe_failure = lambda: "Path not found: {}".format(repr(rctx.attr.interpreter_path)) diff --git a/python/private/py_cc_link_params_info.bzl b/python/private/py_cc_link_params_info.bzl index 35919a04e2..2fbd255eed 100644 --- a/python/private/py_cc_link_params_info.bzl +++ b/python/private/py_cc_link_params_info.bzl @@ -14,7 +14,6 @@ """Providers for Python rules.""" load("@rules_cc//cc/common:cc_info.bzl", "CcInfo") -load(":util.bzl", "define_bazel_6_provider") def _PyCcLinkParamsInfo_init(cc_info): return { @@ -22,7 +21,7 @@ def _PyCcLinkParamsInfo_init(cc_info): } # buildifier: disable=name-conventions -PyCcLinkParamsInfo, _unused_raw_py_cc_link_params_provider_ctor = define_bazel_6_provider( +PyCcLinkParamsInfo, _unused_raw_py_cc_link_params_provider_ctor = provider( doc = ("Python-wrapper to forward {obj}`CcInfo.linking_context`. This is to " + "allow Python targets to propagate C++ linking information, but " + "without the Python target appearing to be a valid C++ rule dependency"), diff --git a/python/private/py_info.bzl b/python/private/py_info.bzl index f96dec554b..4059b30c63 100644 --- a/python/private/py_info.bzl +++ b/python/private/py_info.bzl @@ -13,10 +13,8 @@ # limitations under the License. """Implementation of PyInfo provider and PyInfo-specific utilities.""" -load("@rules_python_internal//:rules_python_config.bzl", "config") load(":builders.bzl", "builders") load(":reexports.bzl", "BuiltinPyInfo") -load(":util.bzl", "define_bazel_6_provider") def _VenvSymlinkKind_typedef(): """An enum of types of venv directories. @@ -160,7 +158,7 @@ def _PyInfo_init( "venv_symlinks": venv_symlinks, } -PyInfo, _unused_raw_py_info_ctor = define_bazel_6_provider( +PyInfo, _unused_raw_py_info_ctor = provider( doc = """Encapsulates information provided by the Python rules. Instead of creating this object directly, use {obj}`PyInfoBuilder` and @@ -327,7 +325,7 @@ This field is currently unused in Bazel and may go away in the future. ) # The "effective" PyInfo is what the canonical //python:py_info.bzl%PyInfo symbol refers to -_EffectivePyInfo = PyInfo if (config.enable_pystar or BuiltinPyInfo == None) else BuiltinPyInfo +_EffectivePyInfo = PyInfo def _PyInfoBuilder_typedef(): """Builder for PyInfo. @@ -630,28 +628,21 @@ def _PyInfoBuilder_build(self): Returns: {type}`PyInfo` """ - if config.enable_pystar: - kwargs = dict( - direct_original_sources = self.direct_original_sources.build(), - direct_pyc_files = self.direct_pyc_files.build(), - direct_pyi_files = self.direct_pyi_files.build(), - transitive_implicit_pyc_files = self.transitive_implicit_pyc_files.build(), - transitive_implicit_pyc_source_files = self.transitive_implicit_pyc_source_files.build(), - transitive_original_sources = self.transitive_original_sources.build(), - transitive_pyc_files = self.transitive_pyc_files.build(), - transitive_pyi_files = self.transitive_pyi_files.build(), - venv_symlinks = self.venv_symlinks.build(), - ) - else: - kwargs = {} - return _EffectivePyInfo( has_py2_only_sources = self._has_py2_only_sources[0], has_py3_only_sources = self._has_py3_only_sources[0], imports = self.imports.build(), transitive_sources = self.transitive_sources.build(), uses_shared_libraries = self._uses_shared_libraries[0], - **kwargs + direct_original_sources = self.direct_original_sources.build(), + direct_pyc_files = self.direct_pyc_files.build(), + direct_pyi_files = self.direct_pyi_files.build(), + transitive_implicit_pyc_files = self.transitive_implicit_pyc_files.build(), + transitive_implicit_pyc_source_files = self.transitive_implicit_pyc_source_files.build(), + transitive_original_sources = self.transitive_original_sources.build(), + transitive_pyc_files = self.transitive_pyc_files.build(), + transitive_pyi_files = self.transitive_pyi_files.build(), + venv_symlinks = self.venv_symlinks.build(), ) def _PyInfoBuilder_build_builtin_py_info(self): diff --git a/python/private/py_package.bzl b/python/private/py_package.bzl index adf2b6deef..d23276a53b 100644 --- a/python/private/py_package.bzl +++ b/python/private/py_package.bzl @@ -41,13 +41,8 @@ def _py_package_impl(ctx): py_info.merge_target(dep) py_info = py_info.build() inputs.add(py_info.transitive_sources) - - # Remove conditional once Bazel 6 support dropped. - if hasattr(py_info, "transitive_pyc_files"): - inputs.add(py_info.transitive_pyc_files) - - if hasattr(py_info, "transitive_pyi_files"): - inputs.add(py_info.transitive_pyi_files) + inputs.add(py_info.transitive_pyc_files) + inputs.add(py_info.transitive_pyi_files) inputs = inputs.build() diff --git a/python/private/py_runtime_info.bzl b/python/private/py_runtime_info.bzl index efe14b2c06..af4e7f0596 100644 --- a/python/private/py_runtime_info.bzl +++ b/python/private/py_runtime_info.bzl @@ -13,8 +13,6 @@ # limitations under the License. """Providers for Python rules.""" -load(":util.bzl", "define_bazel_6_provider") - DEFAULT_STUB_SHEBANG = "#!/usr/bin/env python3" _PYTHON_VERSION_VALUES = ["PY2", "PY3"] @@ -124,7 +122,7 @@ def _PyRuntimeInfo_init( "zip_main_template": zip_main_template, } -PyRuntimeInfo, _unused_raw_py_runtime_info_ctor = define_bazel_6_provider( +PyRuntimeInfo, _unused_raw_py_runtime_info_ctor = provider( doc = """Contains information about a Python runtime, as returned by the `py_runtime` rule. diff --git a/python/private/py_runtime_pair_rule.bzl b/python/private/py_runtime_pair_rule.bzl index 61cbdcd6f4..203e5d4df7 100644 --- a/python/private/py_runtime_pair_rule.bzl +++ b/python/private/py_runtime_pair_rule.bzl @@ -19,7 +19,6 @@ load("//python:py_runtime_info.bzl", "PyRuntimeInfo") load(":common_labels.bzl", "labels") load(":flags.bzl", "read_possibly_native_flag") load(":reexports.bzl", "BuiltinPyRuntimeInfo") -load(":util.bzl", "IS_BAZEL_7_OR_HIGHER") def _py_runtime_pair_impl(ctx): if ctx.attr.py2_runtime != None: @@ -58,7 +57,7 @@ def _get_py_runtime_info(target): # py_binary (implemented in Java) performs a type check on the provider # value to verify it is an instance of the Java-implemented PyRuntimeInfo # class. - if (IS_BAZEL_7_OR_HIGHER and PyRuntimeInfo in target) or BuiltinPyRuntimeInfo == None: + if (PyRuntimeInfo in target) or BuiltinPyRuntimeInfo == None: return target[PyRuntimeInfo] else: return target[BuiltinPyRuntimeInfo] diff --git a/python/private/py_runtime_rule.bzl b/python/private/py_runtime_rule.bzl index ba1a390ef3..5020d7ad9b 100644 --- a/python/private/py_runtime_rule.bzl +++ b/python/private/py_runtime_rule.bzl @@ -21,7 +21,6 @@ load(":flags.bzl", "FreeThreadedFlag") load(":py_internal.bzl", "py_internal") load(":py_runtime_info.bzl", "DEFAULT_STUB_SHEBANG", "PyRuntimeInfo") load(":reexports.bzl", "BuiltinPyRuntimeInfo") -load(":util.bzl", "IS_BAZEL_7_OR_HIGHER") _py_builtins = py_internal @@ -133,9 +132,6 @@ def _py_runtime_impl(ctx): supports_build_time_venv = ctx.attr.supports_build_time_venv, )) - if not IS_BAZEL_7_OR_HIGHER: - builtin_py_runtime_info_kwargs.pop("bootstrap_template") - providers = [ PyRuntimeInfo(**py_runtime_info_kwargs), DefaultInfo( @@ -388,11 +384,7 @@ The {obj}`PyRuntimeInfo.zip_main_template` field. ) def _is_singleton_depset(files): - # Bazel 6 doesn't have this helper to optimize detecting singleton depsets. - if _py_builtins: - return _py_builtins.is_singleton_depset(files) - else: - return len(files.to_list()) == 1 + return _py_builtins.is_singleton_depset(files) def _interpreter_version_info_from_version_str(version_str): parts = version_str.split(".") diff --git a/python/private/pypi/BUILD.bazel b/python/private/pypi/BUILD.bazel index 0d2f73fb0b..b9650c8152 100644 --- a/python/private/pypi/BUILD.bazel +++ b/python/private/pypi/BUILD.bazel @@ -74,7 +74,6 @@ bzl_library( srcs = ["deps.bzl"], deps = [ "//python/private:bazel_tools_bzl", - "//python/private:glob_excludes_bzl", ], ) @@ -126,6 +125,7 @@ bzl_library( "@bazel_features//:features", "@pythons_hub//:interpreters_bzl", "@pythons_hub//:versions_bzl", + "@rules_python_internal//:rules_python_config_bzl", ], ) @@ -439,6 +439,7 @@ bzl_library( "//python/private:envsubst_bzl", "//python/private:is_standalone_interpreter_bzl", "//python/private:repo_utils_bzl", + "@rules_python_internal//:rules_python_config_bzl", ], ) diff --git a/python/private/pypi/deps.bzl b/python/private/pypi/deps.bzl index 73b30c69ee..5379343d62 100644 --- a/python/private/pypi/deps.bzl +++ b/python/private/pypi/deps.bzl @@ -101,7 +101,6 @@ _GENERIC_WHEEL = """\ package(default_visibility = ["//visibility:public"]) load("@rules_python//python:py_library.bzl", "py_library") -load("@rules_python//python/private:glob_excludes.bzl", "glob_excludes") py_library( name = "lib", @@ -115,7 +114,7 @@ py_library( "**/*.dist-info/RECORD", "BUILD", "WORKSPACE", - ] + glob_excludes.version_dependent_exclusions()), + ]), # This makes this directory a top-level in the python import # search path for anything that depends on this. imports = ["."], diff --git a/python/private/pypi/pypi_repo_utils.bzl b/python/private/pypi/pypi_repo_utils.bzl index bb2acc850a..04c9b5d685 100644 --- a/python/private/pypi/pypi_repo_utils.bzl +++ b/python/private/pypi/pypi_repo_utils.bzl @@ -107,9 +107,8 @@ def _construct_pypath(mrctx, *, entries): def _execute_prep(mrctx, *, python, srcs, **kwargs): for src in srcs: # This will ensure that we will re-evaluate the bzlmod extension or - # refetch the repository_rule when the srcs change. This should work on - # Bazel versions without `mrctx.watch` as well. - repo_utils.watch(mrctx, mrctx.path(src)) + # refetch the repository_rule when the srcs change. + mrctx.watch(mrctx.path(src)) environment = kwargs.pop("environment", {}) pythonpath = environment.get("PYTHONPATH", "") diff --git a/python/private/pypi/whl_library_targets.bzl b/python/private/pypi/whl_library_targets.bzl index aed5bc74f5..89c1d348b3 100644 --- a/python/private/pypi/whl_library_targets.bzl +++ b/python/private/pypi/whl_library_targets.bzl @@ -17,7 +17,6 @@ load("@bazel_skylib//rules:copy_file.bzl", "copy_file") load("//python:py_binary.bzl", "py_binary") load("//python:py_library.bzl", "py_library") -load("//python/private:glob_excludes.bzl", "glob_excludes") load("//python/private:normalize_name.bzl", "normalize_name") load(":env_marker_setting.bzl", "env_marker_setting") load( @@ -315,13 +314,6 @@ def whl_library_targets( deps_by_platform = dependencies_by_platform, deps_conditional = deps_conditional, tmpl = dep_template.format(name = "{}", target = WHEEL_FILE_PUBLIC_LABEL), - # NOTE @aignas 2024-10-28: Actually, `select` is not part of - # `native`, but in order to support bazel 6.4 in unit tests, I - # have to somehow pass the `select` implementation in the unit - # tests and I chose this to be routed through the `native` - # struct. So, tests` will be successful in `getattr` and the - # real code will use the fallback provided here. - select = getattr(native, "select", select), ), visibility = impl_vis, ) @@ -346,7 +338,7 @@ def whl_library_targets( # of generated files produced when wheels are installed. The file is ignored to avoid # Bazel caching issues. "**/*.dist-info/RECORD", - ] + glob_excludes.version_dependent_exclusions() + ] for item in data_exclude: if item not in _data_exclude: _data_exclude.append(item) @@ -362,7 +354,7 @@ def whl_library_targets( ) if not enable_implicit_namespace_pkgs: - srcs = srcs + getattr(native, "select", select)({ + srcs = srcs + select({ Label("//python/config_settings:is_venvs_site_packages"): [], "//conditions:default": rules.create_inits( srcs = srcs + data + pyi_srcs, @@ -384,7 +376,6 @@ def whl_library_targets( deps_by_platform = dependencies_by_platform, deps_conditional = deps_conditional, tmpl = dep_template.format(name = "{}", target = PY_LIBRARY_PUBLIC_LABEL), - select = getattr(native, "select", select), ), tags = tags, visibility = impl_vis, @@ -455,7 +446,7 @@ def _plat_label(plat): else: return ":is_" + plat.replace("cp3", "python_3.") -def _deps(deps, deps_by_platform, deps_conditional, tmpl, select = select): +def _deps(deps, deps_by_platform, deps_conditional, tmpl): deps = [tmpl.format(d) for d in sorted(deps)] for dep, setting in deps_conditional.items(): diff --git a/python/private/python.bzl b/python/private/python.bzl index 6eb8a3742e..faad53fab4 100644 --- a/python/private/python.bzl +++ b/python/private/python.bzl @@ -29,7 +29,6 @@ load( "sorted_host_platform_names", "sorted_host_platforms", ) -load(":util.bzl", "IS_BAZEL_6_4_OR_HIGHER") load(":version.bzl", "version") def parse_modules(*, module_ctx, logger, _fail = fail): @@ -932,14 +931,6 @@ def _create_toolchain_attrs_struct(*, tag = None, python_version = None, toolcha ignore_root_user_error = getattr(tag, "ignore_root_user_error", True), ) -def _get_bazel_version_specific_kwargs(): - kwargs = {} - - if IS_BAZEL_6_4_OR_HIGHER: - kwargs["environ"] = ["RULES_PYTHON_BZLMOD_DEBUG"] - - return kwargs - _defaults = tag_class( doc = """Tag class to specify the default Python version.""", attrs = { @@ -1356,7 +1347,7 @@ python = module_extension( "single_version_platform_override": _single_version_platform_override, "toolchain": _toolchain, }, - **_get_bazel_version_specific_kwargs() + environ = ["RULES_PYTHON_BZLMOD_DEBUG"], ) _DEBUG_BUILD_CONTENT = """ diff --git a/python/private/python_register_toolchains.bzl b/python/private/python_register_toolchains.bzl index 2e0748deb0..9e75c41978 100644 --- a/python/private/python_register_toolchains.bzl +++ b/python/private/python_register_toolchains.bzl @@ -97,21 +97,6 @@ def python_register_toolchains( toolchain_repo_name = "{name}_toolchains".format(name = name) - # When using unreleased Bazel versions, the version is an empty string - if native.bazel_version: - bazel_major = int(native.bazel_version.split(".")[0]) - if bazel_major < 6: - if register_coverage_tool: - # buildifier: disable=print - print(( - "WARNING: ignoring register_coverage_tool=True when " + - "registering @{name}: Bazel 6+ required, got {version}" - ).format( - name = name, - version = native.bazel_version, - )) - register_coverage_tool = False - # list[str] of the platform names that were used loaded_platforms = [] diff --git a/python/private/repo_utils.bzl b/python/private/repo_utils.bzl index 32a5b70e15..77eac55c16 100644 --- a/python/private/repo_utils.bzl +++ b/python/private/repo_utils.bzl @@ -291,7 +291,7 @@ def _which_unchecked(mrctx, binary_name): """ binary = mrctx.which(binary_name) if binary: - _watch(mrctx, binary) + mrctx.watch(binary) describe_failure = None else: path = _getenv(mrctx, "PATH", "") @@ -429,24 +429,6 @@ def _get_platforms_cpu_name(mrctx): return "riscv64" return arch -# TODO: Remove after Bazel 6 support dropped -def _watch(mrctx, *args, **kwargs): - """Calls mrctx.watch, if available.""" - if not args and not kwargs: - fail("'watch' needs at least a single argument.") - - if hasattr(mrctx, "watch"): - mrctx.watch(*args, **kwargs) - -# TODO: Remove after Bazel 6 support dropped -def _watch_tree(mrctx, *args, **kwargs): - """Calls mrctx.watch_tree, if available.""" - if not args and not kwargs: - fail("'watch_tree' needs at least a single argument.") - - if hasattr(mrctx, "watch_tree"): - mrctx.watch_tree(*args, **kwargs) - repo_utils = struct( # keep sorted execute_checked = _execute_checked, @@ -457,8 +439,6 @@ repo_utils = struct( getenv = _getenv, is_repo_debug_enabled = _is_repo_debug_enabled, logger = _logger, - watch = _watch, - watch_tree = _watch_tree, which_checked = _which_checked, which_unchecked = _which_unchecked, ) diff --git a/python/private/util.bzl b/python/private/util.bzl index 4d2da57760..d3053fe626 100644 --- a/python/private/util.bzl +++ b/python/private/util.bzl @@ -15,7 +15,6 @@ """Functionality shared by multiple pieces of code.""" load("@bazel_skylib//lib:types.bzl", "types") -load("@rules_python_internal//:rules_python_config.bzl", "config") def copy_propagating_kwargs(from_kwargs, into_kwargs = None): """Copies args that must be compatible between two targets with a dependency relationship. @@ -50,21 +49,6 @@ def copy_propagating_kwargs(from_kwargs, into_kwargs = None): # The implementation of the macros and tagging mechanism follows the example # set by rules_cc and rules_java. -_MIGRATION_TAG = "__PYTHON_RULES_MIGRATION_DO_NOT_USE_WILL_BREAK__" - -def add_migration_tag(attrs): - """Add a special tag to `attrs` to aid migration off native rles. - - Args: - attrs: dict of keyword args. The `tags` key will be modified in-place. - - Returns: - The same `attrs` object, but modified. - """ - if not config.enable_pystar: - add_tag(attrs, _MIGRATION_TAG) - return attrs - def add_tag(attrs, tag): """Adds `tag` to `attrs["tags"]`. @@ -85,37 +69,3 @@ def add_tag(attrs, tag): attrs["tags"] = tags + [tag] else: attrs["tags"] = [tag] - -# Helper to make the provider definitions not crash under Bazel 5.4: -# Bazel 5.4 doesn't support the `init` arg of `provider()`, so we have to -# not pass that when using Bazel 5.4. But, not passing the `init` arg -# changes the return value from a two-tuple to a single value, which then -# breaks Bazel 6+ code. -# This isn't actually used under Bazel 5.4, so just stub out the values -# to get past the loading phase. -def define_bazel_6_provider(doc, fields, **kwargs): - """Define a provider, or a stub for pre-Bazel 7.""" - if not IS_BAZEL_6_OR_HIGHER: - return provider("Stub, not used", fields = []), None - return provider(doc = doc, fields = fields, **kwargs) - -IS_BAZEL_7_4_OR_HIGHER = hasattr(native, "legacy_globals") - -IS_BAZEL_7_OR_HIGHER = hasattr(native, "starlark_doc_extract") - -# Bazel 5.4 has a bug where every access of testing.ExecutionInfo is a -# different object that isn't equal to any other. This is fixed in bazel 6+. -IS_BAZEL_6_OR_HIGHER = testing.ExecutionInfo == testing.ExecutionInfo - -_marker_rule_to_detect_bazel_6_4_or_higher = rule(implementation = lambda ctx: None) - -# Bazel 6.4 and higher have a bug fix where rule names show up in the str() -# of a rule. See -# https://github.com/bazelbuild/bazel/commit/002490b9a2376f0b2ea4a37102c5e94fc50a65ba -# https://github.com/bazelbuild/bazel/commit/443cbcb641e17f7337ccfdecdfa5e69bc16cae55 -# This technique is done instead of using native.bazel_version because, -# under stardoc, the native.bazel_version attribute is entirely missing, which -# prevents doc generation from being able to correctly generate docs. -IS_BAZEL_6_4_OR_HIGHER = "_marker_rule_to_detect_bazel_6_4_or_higher" in str( - _marker_rule_to_detect_bazel_6_4_or_higher, -) diff --git a/python/py_binary.bzl b/python/py_binary.bzl index 48ea768948..4e26a29af2 100644 --- a/python/py_binary.bzl +++ b/python/py_binary.bzl @@ -14,13 +14,11 @@ """Public entry point for py_binary.""" -load("@rules_python_internal//:rules_python_config.bzl", "config") load("//python/private:py_binary_macro.bzl", _starlark_py_binary = "py_binary") load("//python/private:register_extension_info.bzl", "register_extension_info") -load("//python/private:util.bzl", "add_migration_tag") # buildifier: disable=native-python -_py_binary_impl = _starlark_py_binary if config.enable_pystar else native.py_binary +_py_binary_impl = _starlark_py_binary def py_binary(**attrs): """Creates an executable Python program. @@ -42,7 +40,7 @@ def py_binary(**attrs): if attrs.get("srcs_version") in ("PY2", "PY2ONLY"): fail("Python 2 is no longer supported: https://github.com/bazel-contrib/rules_python/issues/886") - _py_binary_impl(**add_migration_tag(attrs)) + _py_binary_impl(**attrs) register_extension_info( extension = py_binary, diff --git a/python/py_cc_link_params_info.bzl b/python/py_cc_link_params_info.bzl index 02eff71c4d..6c510d6c8e 100644 --- a/python/py_cc_link_params_info.bzl +++ b/python/py_cc_link_params_info.bzl @@ -1,10 +1,5 @@ """Public entry point for PyCcLinkParamsInfo.""" -load("@rules_python_internal//:rules_python_config.bzl", "config") load("//python/private:py_cc_link_params_info.bzl", _starlark_PyCcLinkParamsInfo = "PyCcLinkParamsInfo") -PyCcLinkParamsInfo = ( - _starlark_PyCcLinkParamsInfo if ( - config.enable_pystar or config.BuiltinPyCcLinkParamsProvider == None - ) else config.BuiltinPyCcLinkParamsProvider -) +PyCcLinkParamsInfo = _starlark_PyCcLinkParamsInfo diff --git a/python/py_info.bzl b/python/py_info.bzl index 5697f58419..5582d3b491 100644 --- a/python/py_info.bzl +++ b/python/py_info.bzl @@ -14,8 +14,6 @@ """Public entry point for PyInfo.""" -load("@rules_python_internal//:rules_python_config.bzl", "config") load("//python/private:py_info.bzl", _starlark_PyInfo = "PyInfo") -load("//python/private:reexports.bzl", "BuiltinPyInfo") -PyInfo = _starlark_PyInfo if config.enable_pystar or BuiltinPyInfo == None else BuiltinPyInfo +PyInfo = _starlark_PyInfo diff --git a/python/py_library.bzl b/python/py_library.bzl index 8b8d46870b..4b79d8f0eb 100644 --- a/python/py_library.bzl +++ b/python/py_library.bzl @@ -14,13 +14,11 @@ """Public entry point for py_library.""" -load("@rules_python_internal//:rules_python_config.bzl", "config") load("//python/private:py_library_macro.bzl", _starlark_py_library = "py_library") load("//python/private:register_extension_info.bzl", "register_extension_info") -load("//python/private:util.bzl", "add_migration_tag") # buildifier: disable=native-python -_py_library_impl = _starlark_py_library if config.enable_pystar else native.py_library +_py_library_impl = _starlark_py_library def py_library(**attrs): """Creates an executable Python program. @@ -39,7 +37,7 @@ def py_library(**attrs): if attrs.get("srcs_version") in ("PY2", "PY2ONLY"): fail("Python 2 is no longer supported: https://github.com/bazel-contrib/rules_python/issues/886") - _py_library_impl(**add_migration_tag(attrs)) + _py_library_impl(**attrs) register_extension_info( extension = py_library, diff --git a/python/py_runtime.bzl b/python/py_runtime.bzl index dad2965cf5..8c3cee2eb7 100644 --- a/python/py_runtime.bzl +++ b/python/py_runtime.bzl @@ -15,10 +15,9 @@ """Public entry point for py_runtime.""" load("//python/private:py_runtime_macro.bzl", _starlark_py_runtime = "py_runtime") -load("//python/private:util.bzl", "IS_BAZEL_6_OR_HIGHER", "add_migration_tag") # buildifier: disable=native-python -_py_runtime_impl = _starlark_py_runtime if IS_BAZEL_6_OR_HIGHER else native.py_runtime +_py_runtime_impl = _starlark_py_runtime def py_runtime(**attrs): """Creates an executable Python program. @@ -39,4 +38,4 @@ def py_runtime(**attrs): if attrs.get("python_version") == "PY2": fail("Python 2 is no longer supported: see https://github.com/bazel-contrib/rules_python/issues/886") - _py_runtime_impl(**add_migration_tag(attrs)) + _py_runtime_impl(**attrs) diff --git a/python/py_runtime_info.bzl b/python/py_runtime_info.bzl index 3a31c0f2f4..082a9b0f19 100644 --- a/python/py_runtime_info.bzl +++ b/python/py_runtime_info.bzl @@ -14,8 +14,6 @@ """Public entry point for PyRuntimeInfo.""" -load("@rules_python_internal//:rules_python_config.bzl", "config") load("//python/private:py_runtime_info.bzl", _starlark_PyRuntimeInfo = "PyRuntimeInfo") -load("//python/private:reexports.bzl", "BuiltinPyRuntimeInfo") -PyRuntimeInfo = _starlark_PyRuntimeInfo if config.enable_pystar else BuiltinPyRuntimeInfo +PyRuntimeInfo = _starlark_PyRuntimeInfo diff --git a/python/py_runtime_pair.bzl b/python/py_runtime_pair.bzl index 26d378fce2..97cc4f5f18 100644 --- a/python/py_runtime_pair.bzl +++ b/python/py_runtime_pair.bzl @@ -14,11 +14,9 @@ """Public entry point for py_runtime_pair.""" -load("@bazel_tools//tools/python:toolchain.bzl", _bazel_tools_impl = "py_runtime_pair") load("//python/private:py_runtime_pair_macro.bzl", _starlark_impl = "py_runtime_pair") -load("//python/private:util.bzl", "IS_BAZEL_6_OR_HIGHER") -_py_runtime_pair = _starlark_impl if IS_BAZEL_6_OR_HIGHER else _bazel_tools_impl +_py_runtime_pair = _starlark_impl # NOTE: This doc is copy/pasted from the builtin py_runtime_pair rule so our # doc generator gives useful API docs. diff --git a/python/py_test.bzl b/python/py_test.bzl index b5657730b7..5b8ad31725 100644 --- a/python/py_test.bzl +++ b/python/py_test.bzl @@ -14,13 +14,11 @@ """Public entry point for py_test.""" -load("@rules_python_internal//:rules_python_config.bzl", "config") load("//python/private:py_test_macro.bzl", _starlark_py_test = "py_test") load("//python/private:register_extension_info.bzl", "register_extension_info") -load("//python/private:util.bzl", "add_migration_tag") # buildifier: disable=native-python -_py_test_impl = _starlark_py_test if config.enable_pystar else native.py_test +_py_test_impl = _starlark_py_test def py_test(**attrs): """Creates an executable Python program. @@ -43,7 +41,7 @@ def py_test(**attrs): fail("Python 2 is no longer supported: https://github.com/bazel-contrib/rules_python/issues/886") # buildifier: disable=native-python - _py_test_impl(**add_migration_tag(attrs)) + _py_test_impl(**attrs) register_extension_info( extension = py_test, diff --git a/sphinxdocs/tests/proto_to_markdown/BUILD.bazel b/sphinxdocs/tests/proto_to_markdown/BUILD.bazel index 09f537472c..2964785eed 100644 --- a/sphinxdocs/tests/proto_to_markdown/BUILD.bazel +++ b/sphinxdocs/tests/proto_to_markdown/BUILD.bazel @@ -13,12 +13,10 @@ # limitations under the License. load("//python:py_test.bzl", "py_test") -load("//python/private:util.bzl", "IS_BAZEL_7_OR_HIGHER") # buildifier: disable=bzl-visibility py_test( name = "proto_to_markdown_test", srcs = ["proto_to_markdown_test.py"], - target_compatible_with = [] if IS_BAZEL_7_OR_HIGHER else ["@platforms//:incompatible"], deps = [ "//sphinxdocs/private:proto_to_markdown_lib", "@dev_pip//absl_py", diff --git a/sphinxdocs/tests/sphinx_docs/BUILD.bazel b/sphinxdocs/tests/sphinx_docs/BUILD.bazel index f9c82967c1..33b98ec585 100644 --- a/sphinxdocs/tests/sphinx_docs/BUILD.bazel +++ b/sphinxdocs/tests/sphinx_docs/BUILD.bazel @@ -1,5 +1,4 @@ load("@bazel_skylib//rules:build_test.bzl", "build_test") -load("//python/private:util.bzl", "IS_BAZEL_7_OR_HIGHER") # buildifier: disable=bzl-visibility load("//sphinxdocs:sphinx.bzl", "sphinx_build_binary", "sphinx_docs") load(":defs.bzl", "gen_directory") @@ -13,7 +12,7 @@ _TARGET_COMPATIBLE_WITH = select({ "@platforms//os:linux": [], "@platforms//os:macos": [], "//conditions:default": ["@platforms//:incompatible"], -}) if IS_BAZEL_7_OR_HIGHER else ["@platforms//:incompatible"] +}) sphinx_docs( name = "docs", diff --git a/sphinxdocs/tests/sphinx_stardoc/BUILD.bazel b/sphinxdocs/tests/sphinx_stardoc/BUILD.bazel index e3a68ea225..af9af30886 100644 --- a/sphinxdocs/tests/sphinx_stardoc/BUILD.bazel +++ b/sphinxdocs/tests/sphinx_stardoc/BUILD.bazel @@ -1,7 +1,6 @@ load("@bazel_skylib//:bzl_library.bzl", "bzl_library") load("@bazel_skylib//rules:build_test.bzl", "build_test") load("//python:py_test.bzl", "py_test") -load("//python/private:util.bzl", "IS_BAZEL_7_OR_HIGHER") # buildifier: disable=bzl-visibility load("//sphinxdocs:sphinx.bzl", "sphinx_build_binary", "sphinx_docs") load("//sphinxdocs:sphinx_stardoc.bzl", "sphinx_stardoc", "sphinx_stardocs") @@ -15,7 +14,7 @@ _TARGET_COMPATIBLE_WITH = select({ "@platforms//os:linux": [], "@platforms//os:macos": [], "//conditions:default": ["@platforms//:incompatible"], -}) if IS_BAZEL_7_OR_HIGHER else ["@platforms//:incompatible"] +}) sphinx_docs( name = "docs", diff --git a/tests/api/py_common/py_common_tests.bzl b/tests/api/py_common/py_common_tests.bzl index 572028b2a6..028da6cc37 100644 --- a/tests/api/py_common/py_common_tests.bzl +++ b/tests/api/py_common/py_common_tests.bzl @@ -13,7 +13,6 @@ # limitations under the License. """py_common tests.""" -load("@rules_python_internal//:rules_python_config.bzl", "config") load("@rules_testing//lib:analysis_test.bzl", "analysis_test") load("@rules_testing//lib:test_suite.bzl", "test_suite") load("@rules_testing//lib:util.bzl", rt_util = "util") @@ -41,13 +40,11 @@ def _test_merge_py_infos_impl(env, target): py_common = _py_common.get(env.ctx) py1 = py_common.PyInfoBuilder() - if config.enable_pystar: - py1.direct_pyc_files.add(f1_pyc) + py1.direct_pyc_files.add(f1_pyc) py1.transitive_sources.add(f1_py) py2 = py_common.PyInfoBuilder() - if config.enable_pystar: - py1.direct_pyc_files.add(f2_pyc) + py1.direct_pyc_files.add(f2_pyc) py2.transitive_sources.add(f2_py) actual = py_info_subject( @@ -56,8 +53,7 @@ def _test_merge_py_infos_impl(env, target): ) actual.transitive_sources().contains_exactly([f1_py.path, f2_py.path]) - if config.enable_pystar: - actual.direct_pyc_files().contains_exactly([f1_pyc.path, f2_pyc.path]) + actual.direct_pyc_files().contains_exactly([f1_pyc.path, f2_pyc.path]) _tests.append(_test_merge_py_infos) diff --git a/tests/base_rules/precompile/precompile_tests.bzl b/tests/base_rules/precompile/precompile_tests.bzl index fe5c165648..bff994aa1a 100644 --- a/tests/base_rules/precompile/precompile_tests.bzl +++ b/tests/base_rules/precompile/precompile_tests.bzl @@ -14,7 +14,6 @@ """Tests for precompiling behavior.""" -load("@rules_python_internal//:rules_python_config.bzl", rp_config = "config") load("@rules_testing//lib:analysis_test.bzl", "analysis_test") load("@rules_testing//lib:test_suite.bzl", "test_suite") load("@rules_testing//lib:truth.bzl", "matching") @@ -42,9 +41,6 @@ _COMMON_CONFIG_SETTINGS = { _tests = [] def _test_executable_precompile_attr_enabled_setup(name, py_rule, **kwargs): - if not rp_config.enable_pystar: - rt_util.skip_test(name = name) - return rt_util.helper_target( py_rule, name = name + "_subject", @@ -112,9 +108,6 @@ def _test_precompile_enabled_py_test(name): _tests.append(_test_precompile_enabled_py_test) def _test_precompile_enabled_py_library_setup(name, impl, config_settings): - if not rp_config.enable_pystar: - rt_util.skip_test(name = name) - return rt_util.helper_target( py_library, name = name + "_subject", @@ -178,9 +171,6 @@ def _test_precompile_enabled_py_library_add_to_runfiles_enabled_impl(env, target _tests.append(_test_precompile_enabled_py_library_add_to_runfiles_enabled) def _test_pyc_only(name): - if not rp_config.enable_pystar: - rt_util.skip_test(name = name) - return rt_util.helper_target( py_binary, name = name + "_subject", @@ -231,9 +221,6 @@ def _test_pyc_only_impl(env, target): ) def _test_precompiler_action(name): - if not rp_config.enable_pystar: - rt_util.skip_test(name = name) - return rt_util.helper_target( py_binary, name = name + "_subject", @@ -325,9 +312,6 @@ def _verify_runfiles(contains_patterns, not_contains_patterns): return _verify_runfiles_impl def _test_precompile_flag_enabled_pyc_collection_attr_include_pyc(name): - if not rp_config.enable_pystar: - rt_util.skip_test(name = name) - return _setup_precompile_flag_pyc_collection_attr_interaction( name = name, precompile_flag = "enabled", @@ -351,9 +335,6 @@ def _test_precompile_flag_enabled_pyc_collection_attr_disabled(name): """Verify that a binary can opt-out of using implicit pycs even when precompiling is enabled by default. """ - if not rp_config.enable_pystar: - rt_util.skip_test(name = name) - return _setup_precompile_flag_pyc_collection_attr_interaction( name = name, precompile_flag = "enabled", @@ -376,9 +357,6 @@ _tests.append(_test_precompile_flag_enabled_pyc_collection_attr_disabled) def _test_precompile_flag_disabled_pyc_collection_attr_include_pyc(name): """Verify that a binary can opt-in to using pycs even when precompiling is disabled by default.""" - if not rp_config.enable_pystar: - rt_util.skip_test(name = name) - return _setup_precompile_flag_pyc_collection_attr_interaction( name = name, precompile_flag = "disabled", @@ -398,9 +376,6 @@ def _test_precompile_flag_disabled_pyc_collection_attr_include_pyc(name): _tests.append(_test_precompile_flag_disabled_pyc_collection_attr_include_pyc) def _test_precompile_flag_disabled_pyc_collection_attr_disabled(name): - if not rp_config.enable_pystar: - rt_util.skip_test(name = name) - return _setup_precompile_flag_pyc_collection_attr_interaction( name = name, precompile_flag = "disabled", @@ -424,9 +399,6 @@ def _test_pyc_collection_disabled_library_omit_source(name): """Verify that, when a binary doesn't include implicit pyc files, libraries that set omit_source still have the py source file included. """ - if not rp_config.enable_pystar: - rt_util.skip_test(name = name) - return rt_util.helper_target( py_binary, name = name + "_subject", @@ -469,9 +441,6 @@ def _test_pyc_collection_disabled_library_omit_source_impl(env, target): _tests.append(_test_pyc_collection_disabled_library_omit_source) def _test_pyc_collection_include_dep_omit_source(name): - if not rp_config.enable_pystar: - rt_util.skip_test(name = name) - return rt_util.helper_target( py_binary, name = name + "_subject", @@ -513,9 +482,6 @@ def _test_pyc_collection_include_dep_omit_source_impl(env, target): _tests.append(_test_pyc_collection_include_dep_omit_source) def _test_precompile_attr_inherit_pyc_collection_disabled_precompile_flag_enabled(name): - if not rp_config.enable_pystar: - rt_util.skip_test(name = name) - return rt_util.helper_target( py_binary, name = name + "_subject", diff --git a/tests/base_rules/py_executable_base_tests.bzl b/tests/base_rules/py_executable_base_tests.bzl index 4e451289dc..c7723be54a 100644 --- a/tests/base_rules/py_executable_base_tests.bzl +++ b/tests/base_rules/py_executable_base_tests.bzl @@ -14,14 +14,12 @@ """Tests common to py_binary and py_test (executable rules).""" load("@rules_python//python:py_runtime_info.bzl", RulesPythonPyRuntimeInfo = "PyRuntimeInfo") -load("@rules_python_internal//:rules_python_config.bzl", rp_config = "config") load("@rules_testing//lib:analysis_test.bzl", "analysis_test") load("@rules_testing//lib:truth.bzl", "matching") load("@rules_testing//lib:util.bzl", rt_util = "util") load("//python:py_executable_info.bzl", "PyExecutableInfo") load("//python/private:common_labels.bzl", "labels") # buildifier: disable=bzl-visibility load("//python/private:reexports.bzl", "BuiltinPyRuntimeInfo") # buildifier: disable=bzl-visibility -load("//python/private:util.bzl", "IS_BAZEL_7_OR_HIGHER") # buildifier: disable=bzl-visibility load("//tests/base_rules:base_tests.bzl", "create_base_tests") load("//tests/base_rules:util.bzl", "WINDOWS_ATTR", pt_util = "util") load("//tests/support:py_executable_info_subject.bzl", "PyExecutableInfoSubject") @@ -30,10 +28,6 @@ load("//tests/support:support.bzl", "CC_TOOLCHAIN", "CROSSTOOL_TOP", "LINUX_X86_ _tests = [] def _test_basic_windows(name, config): - if rp_config.enable_pystar: - target_compatible_with = [] - else: - target_compatible_with = ["@platforms//:incompatible"] rt_util.helper_target( config.rule, name = name + "_subject", @@ -56,7 +50,7 @@ def _test_basic_windows(name, config): "//command_line_option:extra_toolchains": [CC_TOOLCHAIN], "//command_line_option:platforms": [WINDOWS_X86_64], }, - attr_values = {"target_compatible_with": target_compatible_with}, + attr_values = {}, ) def _test_basic_windows_impl(env, target): @@ -72,14 +66,11 @@ def _test_basic_windows_impl(env, target): _tests.append(_test_basic_windows) def _test_basic_zip(name, config): - if rp_config.enable_pystar: - target_compatible_with = select({ - # Disable the new test on windows because we have _test_basic_windows. - "@platforms//os:windows": ["@platforms//:incompatible"], - "//conditions:default": [], - }) - else: - target_compatible_with = ["@platforms//:incompatible"] + target_compatible_with = select({ + # Disable the new test on windows because we have _test_basic_windows. + "@platforms//os:windows": ["@platforms//:incompatible"], + "//conditions:default": [], + }) rt_util.helper_target( config.rule, name = name + "_subject", @@ -140,14 +131,13 @@ def _test_executable_in_runfiles_impl(env, target): "{workspace}/{package}/{test_name}_subject" + exe, ]) - if rp_config.enable_pystar: - py_exec_info = env.expect.that_target(target).provider(PyExecutableInfo, factory = PyExecutableInfoSubject.new) - py_exec_info.main().path().contains("_subject.py") - py_exec_info.interpreter_path().contains("python") - py_exec_info.runfiles_without_exe().contains_none_of([ - "{workspace}/{package}/{test_name}_subject" + exe, - "{workspace}/{package}/{test_name}_subject", - ]) + py_exec_info = env.expect.that_target(target).provider(PyExecutableInfo, factory = PyExecutableInfoSubject.new) + py_exec_info.main().path().contains("_subject.py") + py_exec_info.interpreter_path().contains("python") + py_exec_info.runfiles_without_exe().contains_none_of([ + "{workspace}/{package}/{test_name}_subject" + exe, + "{workspace}/{package}/{test_name}_subject", + ]) def _test_default_main_can_be_generated(name, config): rt_util.helper_target( @@ -188,11 +178,6 @@ def _test_default_main_can_have_multiple_path_segments_impl(env, target): ) def _test_default_main_must_be_in_srcs(name, config): - # Bazel 5 will crash with a Java stacktrace when the native Python - # rules have an error. - if not pt_util.is_bazel_6_or_higher(): - rt_util.skip_test(name = name) - return rt_util.helper_target( config.rule, name = name + "_subject", @@ -213,11 +198,6 @@ def _test_default_main_must_be_in_srcs_impl(env, target): ) def _test_default_main_cannot_be_ambiguous(name, config): - # Bazel 5 will crash with a Java stacktrace when the native Python - # rules have an error. - if not pt_util.is_bazel_6_or_higher(): - rt_util.skip_test(name = name) - return rt_util.helper_target( config.rule, name = name + "_subject", @@ -260,11 +240,6 @@ def _test_explicit_main_impl(env, target): ) def _test_explicit_main_cannot_be_ambiguous(name, config): - # Bazel 5 will crash with a Java stacktrace when the native Python - # rules have an error. - if not pt_util.is_bazel_6_or_higher(): - rt_util.skip_test(name = name) - return rt_util.helper_target( config.rule, name = name + "_subject", @@ -310,22 +285,16 @@ def _test_files_to_build_impl(env, target): "{package}/{test_name}_subject.py", ]) - if IS_BAZEL_7_OR_HIGHER: - # As of Bazel 7, the first default output is the executable, so - # verify that is the case. rules_testing - # DepsetFileSubject.contains_exactly doesn't provide an in_order() - # call, nor access to the underlying depset, so we have to do things - # manually. - first_default_output = target[DefaultInfo].files.to_list()[0] - executable = target[DefaultInfo].files_to_run.executable - env.expect.that_file(first_default_output).equals(executable) + # As of Bazel 7, the first default output is the executable, so + # verify that is the case. rules_testing + # DepsetFileSubject.contains_exactly doesn't provide an in_order() + # call, nor access to the underlying depset, so we have to do things + # manually. + first_default_output = target[DefaultInfo].files.to_list()[0] + executable = target[DefaultInfo].files_to_run.executable + env.expect.that_file(first_default_output).equals(executable) def _test_name_cannot_end_in_py(name, config): - # Bazel 5 will crash with a Java stacktrace when the native Python - # rules have an error. - if not pt_util.is_bazel_6_or_higher(): - rt_util.skip_test(name = name) - return rt_util.helper_target( config.rule, name = name + "_subject.py", diff --git a/tests/base_rules/py_info/py_info_tests.bzl b/tests/base_rules/py_info/py_info_tests.bzl index aa252a2937..623594807a 100644 --- a/tests/base_rules/py_info/py_info_tests.bzl +++ b/tests/base_rules/py_info/py_info_tests.bzl @@ -13,7 +13,6 @@ # limitations under the License. """Tests for py_info.""" -load("@rules_python_internal//:rules_python_config.bzl", "config") load("@rules_testing//lib:analysis_test.bzl", "analysis_test") load("@rules_testing//lib:test_suite.bzl", "test_suite") load("@rules_testing//lib:util.bzl", rt_util = "util") @@ -39,11 +38,10 @@ def _provide_py_info_impl(ctx): kwargs["has_py2_only_sources"] = bool(ctx.attr.has_py2_only_sources) providers = [] - if config.enable_pystar: - providers.append(PyInfo(**kwargs)) + providers.append(PyInfo(**kwargs)) # Handle Bazel 6 or if Bazel autoloading is enabled - if not config.enable_pystar or (BuiltinPyInfo and PyInfo != BuiltinPyInfo): + if BuiltinPyInfo and PyInfo != BuiltinPyInfo: providers.append(BuiltinPyInfo(**{ k: kwargs[k] for k in ( @@ -95,10 +93,8 @@ def _test_py_info_create_impl(env, target): imports = depset(["import-path"]), transitive_sources = depset([trans_py]), uses_shared_libraries = True, - **(dict( - direct_pyc_files = depset([direct_pyc]), - transitive_pyc_files = depset([trans_pyc]), - ) if config.enable_pystar else {}) + direct_pyc_files = depset([direct_pyc]), + transitive_pyc_files = depset([trans_pyc]), ) subject = py_info_subject(actual, meta = env.expect.meta) @@ -107,9 +103,8 @@ def _test_py_info_create_impl(env, target): subject.has_py3_only_sources().equals(True) subject.transitive_sources().contains_exactly(["tests/base_rules/py_info/trans.py"]) subject.imports().contains_exactly(["import-path"]) - if config.enable_pystar: - subject.direct_pyc_files().contains_exactly(["tests/base_rules/py_info/direct.pyc"]) - subject.transitive_pyc_files().contains_exactly(["tests/base_rules/py_info/trans.pyc"]) + subject.direct_pyc_files().contains_exactly(["tests/base_rules/py_info/direct.pyc"]) + subject.transitive_pyc_files().contains_exactly(["tests/base_rules/py_info/trans.pyc"]) _tests.append(_test_py_info_create) diff --git a/tests/base_rules/py_test/py_test_tests.bzl b/tests/base_rules/py_test/py_test_tests.bzl index 1ec1dc428f..c28eec4346 100644 --- a/tests/base_rules/py_test/py_test_tests.bzl +++ b/tests/base_rules/py_test/py_test_tests.bzl @@ -39,14 +39,6 @@ _SKIP_WINDOWS = { _tests = [] def _test_mac_requires_darwin_for_execution(name, config): - # Bazel 5.4 has a bug where every access of testing.ExecutionInfo is - # a different object that isn't equal to any other, which prevents - # rules_testing from detecting it properly and fails with an error. - # This is fixed in Bazel 6+. - if not pt_util.is_bazel_6_or_higher(): - rt_util.skip_test(name = name) - return - rt_util.helper_target( config.rule, name = name + "_subject", @@ -74,13 +66,6 @@ def _test_mac_requires_darwin_for_execution_impl(env, target): _tests.append(_test_mac_requires_darwin_for_execution) def _test_non_mac_doesnt_require_darwin_for_execution(name, config): - # Bazel 5.4 has a bug where every access of testing.ExecutionInfo is - # a different object that isn't equal to any other, which prevents - # rules_testing from detecting it properly and fails with an error. - # This is fixed in Bazel 6+. - if not pt_util.is_bazel_6_or_higher(): - rt_util.skip_test(name = name) - return rt_util.helper_target( config.rule, name = name + "_subject", diff --git a/tests/base_rules/util.bzl b/tests/base_rules/util.bzl index a02cafa992..9fb66d7eb3 100644 --- a/tests/base_rules/util.bzl +++ b/tests/base_rules/util.bzl @@ -14,7 +14,6 @@ """Helpers and utilities multiple tests re-use.""" load("@bazel_skylib//lib:structs.bzl", "structs") -load("//python/private:util.bzl", "IS_BAZEL_6_OR_HIGHER") # buildifier: disable=bzl-visibility # Use this with is_windows() WINDOWS_ATTR = {"windows": attr.label(default = "@platforms//os:windows")} @@ -53,9 +52,6 @@ def _struct_with(s, **kwargs): struct_dict.update(kwargs) return struct(**struct_dict) -def _is_bazel_6_or_higher(): - return IS_BAZEL_6_OR_HIGHER - def _is_windows(env): """Tell if the target platform is windows. @@ -72,6 +68,5 @@ def _is_windows(env): util = struct( create_tests = _create_tests, struct_with = _struct_with, - is_bazel_6_or_higher = _is_bazel_6_or_higher, is_windows = _is_windows, ) diff --git a/tests/bootstrap_impls/a/b/c/BUILD.bazel b/tests/bootstrap_impls/a/b/c/BUILD.bazel index 1659ef25bc..1c4b1e7b6b 100644 --- a/tests/bootstrap_impls/a/b/c/BUILD.bazel +++ b/tests/bootstrap_impls/a/b/c/BUILD.bazel @@ -1,10 +1,9 @@ -load("//python/private:util.bzl", "IS_BAZEL_7_OR_HIGHER") # buildifier: disable=bzl-visibility load("//tests/support:py_reconfig.bzl", "py_reconfig_test") _SUPPORTS_BOOTSTRAP_SCRIPT = select({ "@platforms//os:windows": ["@platforms//:incompatible"], "//conditions:default": [], -}) if IS_BAZEL_7_OR_HIGHER else ["@platforms//:incompatible"] +}) py_reconfig_test( name = "nested_dir_test", diff --git a/tests/config_settings/transition/multi_version_tests.bzl b/tests/config_settings/transition/multi_version_tests.bzl index 93f6efd728..b2564a3fb3 100644 --- a/tests/config_settings/transition/multi_version_tests.bzl +++ b/tests/config_settings/transition/multi_version_tests.bzl @@ -16,12 +16,11 @@ load("@pythons_hub//:versions.bzl", "DEFAULT_PYTHON_VERSION") load("@rules_testing//lib:analysis_test.bzl", "analysis_test") load("@rules_testing//lib:test_suite.bzl", "test_suite") -load("@rules_testing//lib:util.bzl", "TestingAspectInfo", rt_util = "util") +load("@rules_testing//lib:util.bzl", rt_util = "util") load("//python:py_binary.bzl", "py_binary") load("//python:py_info.bzl", "PyInfo") load("//python:py_test.bzl", "py_test") load("//python/private:reexports.bzl", "BuiltinPyInfo") # buildifier: disable=bzl-visibility -load("//python/private:util.bzl", "IS_BAZEL_7_OR_HIGHER") # buildifier: disable=bzl-visibility load("//tests/support:support.bzl", "CC_TOOLCHAIN") # NOTE @aignas 2024-06-04: we are using here something that is registered in the MODULE.Bazel @@ -106,20 +105,15 @@ def _test_py_binary_windows_build_python_zip_false(name): def _test_py_binary_windows_build_python_zip_false_impl(env, target): default_outputs = env.expect.that_target(target).default_outputs() - if IS_BAZEL_7_OR_HIGHER: - # TODO: These outputs aren't correct. The outputs shouldn't - # have the "_" prefix on them (those are coming from the underlying - # wrapped binary). - env.expect.that_target(target).default_outputs().contains_exactly([ - "{package}/{test_name}_subject.exe", - "{package}/{test_name}_subject", - "{package}/{test_name}_subject.py", - ]) - else: - inner_exe = target[TestingAspectInfo].attrs.target[DefaultInfo].files_to_run.executable - default_outputs.contains_at_least([ - inner_exe.short_path, - ]) + + # TODO: These outputs aren't correct. The outputs shouldn't + # have the "_" prefix on them (those are coming from the underlying + # wrapped binary). + default_outputs.contains_exactly([ + "{package}/{test_name}_subject.exe", + "{package}/{test_name}_subject", + "{package}/{test_name}_subject.py", + ]) _tests.append(_test_py_binary_windows_build_python_zip_false) @@ -132,21 +126,15 @@ def _test_py_binary_windows_build_python_zip_true(name): def _test_py_binary_windows_build_python_zip_true_impl(env, target): default_outputs = env.expect.that_target(target).default_outputs() - if IS_BAZEL_7_OR_HIGHER: - # TODO: These outputs aren't correct. The outputs shouldn't - # have the "_" prefix on them (those are coming from the underlying - # wrapped binary). - default_outputs.contains_exactly([ - "{package}/{test_name}_subject.exe", - "{package}/{test_name}_subject.py", - "{package}/{test_name}_subject.zip", - ]) - else: - inner_exe = target[TestingAspectInfo].attrs.target[DefaultInfo].files_to_run.executable - default_outputs.contains_at_least([ - "{package}/{test_name}_subject.zip", - inner_exe.short_path, - ]) + + # TODO: These outputs aren't correct. The outputs shouldn't + # have the "_" prefix on them (those are coming from the underlying + # wrapped binary). + default_outputs.contains_exactly([ + "{package}/{test_name}_subject.exe", + "{package}/{test_name}_subject.py", + "{package}/{test_name}_subject.zip", + ]) _tests.append(_test_py_binary_windows_build_python_zip_true) diff --git a/tests/exec_toolchain_matching/exec_toolchain_matching_tests.bzl b/tests/exec_toolchain_matching/exec_toolchain_matching_tests.bzl index 43a9717314..b3ff294b6f 100644 --- a/tests/exec_toolchain_matching/exec_toolchain_matching_tests.bzl +++ b/tests/exec_toolchain_matching/exec_toolchain_matching_tests.bzl @@ -20,7 +20,6 @@ load("//python:py_runtime.bzl", "py_runtime") load("//python:py_runtime_pair.bzl", "py_runtime_pair") load("//python/private:common_labels.bzl", "labels") # buildifier: disable=bzl-visibility load("//python/private:toolchain_types.bzl", "EXEC_TOOLS_TOOLCHAIN_TYPE", "TARGET_TOOLCHAIN_TYPE") # buildifier: disable=bzl-visibility -load("//python/private:util.bzl", "IS_BAZEL_7_OR_HIGHER") # buildifier: disable=bzl-visibility load("//tests/support:support.bzl", "LINUX", "MAC") _LookupInfo = provider() # buildifier: disable=provider-params @@ -143,11 +142,10 @@ def _test_exec_matches_target_python_version_impl(env, target): env.expect.that_str(target_runtime.interpreter_path).equals("/linux/python3.12") env.expect.that_str(exec_runtime.interpreter_path).equals("/mac/python3.12") - if IS_BAZEL_7_OR_HIGHER: - target_version = target_runtime.interpreter_version_info - exec_version = exec_runtime.interpreter_version_info + target_version = target_runtime.interpreter_version_info + exec_version = exec_runtime.interpreter_version_info - env.expect.that_bool(target_version == exec_version) + env.expect.that_bool(target_version == exec_version) def exec_toolchain_matching_test_suite(name): test_suite(name = name, tests = _tests) diff --git a/tests/integration/integration_test.bzl b/tests/integration/integration_test.bzl index c437953319..90cc4a3fb7 100644 --- a/tests/integration/integration_test.bzl +++ b/tests/integration/integration_test.bzl @@ -35,12 +35,6 @@ def _test_runner(*, name, bazel_version, py_main, bzlmod, gazelle_plugin): ) return test_runner - if bazel_version.startswith("6") and not bzlmod: - if gazelle_plugin: - return "//tests/integration:bazel_6_4_workspace_test_runner_gazelle_plugin" - else: - return "//tests/integration:bazel_6_4_workspace_test_runner" - if bzlmod and gazelle_plugin: return "//tests/integration:test_runner_gazelle_plugin" elif bzlmod: diff --git a/tests/py_runtime/py_runtime_tests.bzl b/tests/py_runtime/py_runtime_tests.bzl index 4ec7590ab2..b8aa1f3fa6 100644 --- a/tests/py_runtime/py_runtime_tests.bzl +++ b/tests/py_runtime/py_runtime_tests.bzl @@ -13,7 +13,6 @@ # limitations under the License. """Starlark tests for py_runtime rule.""" -load("@rules_python_internal//:rules_python_config.bzl", "config") load("@rules_testing//lib:analysis_test.bzl", "analysis_test") load("@rules_testing//lib:test_suite.bzl", "test_suite") load("@rules_testing//lib:truth.bzl", "matching") @@ -21,15 +20,10 @@ load("@rules_testing//lib:util.bzl", rt_util = "util") load("//python:py_runtime.bzl", "py_runtime") load("//python:py_runtime_info.bzl", "PyRuntimeInfo") load("//python/private:common_labels.bzl", "labels") # buildifier: disable=bzl-visibility -load("//tests/base_rules:util.bzl", br_util = "util") load("//tests/support:py_runtime_info_subject.bzl", "py_runtime_info_subject") _tests = [] -_SKIP_TEST = { - "target_compatible_with": ["@platforms//:incompatible"], -} - def _simple_binary_impl(ctx): executable = ctx.actions.declare_file(ctx.label.name) ctx.actions.write(executable, "", is_executable = True) @@ -49,27 +43,18 @@ _simple_binary = rule( ) def _test_bootstrap_template(name): - # The bootstrap_template arg isn't present in older Bazel versions, so - # we have to conditionally pass the arg and mark the test incompatible. - if config.enable_pystar: - py_runtime_kwargs = {"bootstrap_template": "bootstrap.txt"} - attr_values = {} - else: - py_runtime_kwargs = {} - attr_values = _SKIP_TEST - rt_util.helper_target( py_runtime, name = name + "_subject", interpreter_path = "/py", python_version = "PY3", - **py_runtime_kwargs + bootstrap_template = "bootstrap.txt", ) analysis_test( name = name, target = name + "_subject", impl = _test_bootstrap_template_impl, - attr_values = attr_values, + attr_values = {}, ) def _test_bootstrap_template_impl(env, target): @@ -81,29 +66,19 @@ def _test_bootstrap_template_impl(env, target): _tests.append(_test_bootstrap_template) def _test_cannot_have_both_inbuild_and_system_interpreter(name): - if br_util.is_bazel_6_or_higher(): - py_runtime_kwargs = { - "interpreter": "fake_interpreter", - "interpreter_path": "/some/path", - } - attr_values = {} - else: - py_runtime_kwargs = { - "interpreter_path": "/some/path", - } - attr_values = _SKIP_TEST rt_util.helper_target( py_runtime, name = name + "_subject", python_version = "PY3", - **py_runtime_kwargs + interpreter = "fake_interpreter", + interpreter_path = "/some/path", ) analysis_test( name = name, target = name + "_subject", impl = _test_cannot_have_both_inbuild_and_system_interpreter_impl, expect_failure = True, - attr_values = attr_values, + attr_values = {}, ) def _test_cannot_have_both_inbuild_and_system_interpreter_impl(env, target): @@ -114,25 +89,19 @@ def _test_cannot_have_both_inbuild_and_system_interpreter_impl(env, target): _tests.append(_test_cannot_have_both_inbuild_and_system_interpreter) def _test_cannot_specify_files_for_system_interpreter(name): - if br_util.is_bazel_6_or_higher(): - py_runtime_kwargs = {"files": ["foo.txt"]} - attr_values = {} - else: - py_runtime_kwargs = {} - attr_values = _SKIP_TEST rt_util.helper_target( py_runtime, name = name + "_subject", interpreter_path = "/foo", python_version = "PY3", - **py_runtime_kwargs + files = ["foo.txt"], ) analysis_test( name = name, target = name + "_subject", impl = _test_cannot_specify_files_for_system_interpreter_impl, expect_failure = True, - attr_values = attr_values, + attr_values = {}, ) def _test_cannot_specify_files_for_system_interpreter_impl(env, target): @@ -143,21 +112,12 @@ def _test_cannot_specify_files_for_system_interpreter_impl(env, target): _tests.append(_test_cannot_specify_files_for_system_interpreter) def _test_coverage_tool_executable(name): - if br_util.is_bazel_6_or_higher(): - py_runtime_kwargs = { - "coverage_tool": name + "_coverage_tool", - } - attr_values = {} - else: - py_runtime_kwargs = {} - attr_values = _SKIP_TEST - rt_util.helper_target( py_runtime, name = name + "_subject", python_version = "PY3", interpreter_path = "/bogus", - **py_runtime_kwargs + coverage_tool = name + "_coverage_tool", ) rt_util.helper_target( _simple_binary, @@ -168,7 +128,7 @@ def _test_coverage_tool_executable(name): name = name, target = name + "_subject", impl = _test_coverage_tool_executable_impl, - attr_values = attr_values, + attr_values = {}, ) def _test_coverage_tool_executable_impl(env, target): @@ -183,14 +143,10 @@ def _test_coverage_tool_executable_impl(env, target): _tests.append(_test_coverage_tool_executable) def _test_coverage_tool_plain_files(name): - if br_util.is_bazel_6_or_higher(): - py_runtime_kwargs = { - "coverage_tool": name + "_coverage_tool", - } - attr_values = {} - else: - py_runtime_kwargs = {} - attr_values = _SKIP_TEST + py_runtime_kwargs = { + "coverage_tool": name + "_coverage_tool", + } + attr_values = {} rt_util.helper_target( py_runtime, name = name + "_subject", @@ -334,14 +290,8 @@ def _test_interpreter_binary_with_single_output_and_runfiles_impl(env, target): _tests.append(_test_interpreter_binary_with_single_output_and_runfiles) def _test_must_have_either_inbuild_or_system_interpreter(name): - if br_util.is_bazel_6_or_higher(): - py_runtime_kwargs = {} - attr_values = {} - else: - py_runtime_kwargs = { - "interpreter_path": "/some/path", - } - attr_values = _SKIP_TEST + py_runtime_kwargs = {} + attr_values = {} rt_util.helper_target( py_runtime, name = name + "_subject", @@ -385,14 +335,8 @@ def _test_system_interpreter_impl(env, target): _tests.append(_test_system_interpreter) def _test_system_interpreter_must_be_absolute(name): - # Bazel 5.4 will entirely crash when an invalid interpreter_path - # is given. - if br_util.is_bazel_6_or_higher(): - py_runtime_kwargs = {"interpreter_path": "relative/path"} - attr_values = {} - else: - py_runtime_kwargs = {"interpreter_path": "/junk/value/for/bazel5.4"} - attr_values = _SKIP_TEST + py_runtime_kwargs = {"interpreter_path": "relative/path"} + attr_values = {} rt_util.helper_target( py_runtime, name = name + "_subject", @@ -415,28 +359,19 @@ def _test_system_interpreter_must_be_absolute_impl(env, target): _tests.append(_test_system_interpreter_must_be_absolute) def _interpreter_version_info_test(name, interpreter_version_info, impl, expect_failure = True): - if config.enable_pystar: - py_runtime_kwargs = { - "interpreter_version_info": interpreter_version_info, - } - attr_values = {} - else: - py_runtime_kwargs = {} - attr_values = _SKIP_TEST - rt_util.helper_target( py_runtime, name = name + "_subject", python_version = "PY3", interpreter_path = "/py", - **py_runtime_kwargs + interpreter_version_info = interpreter_version_info, ) analysis_test( name = name, target = name + "_subject", impl = impl, expect_failure = expect_failure, - attr_values = attr_values, + attr_values = {}, ) def _test_interpreter_version_info_must_define_major_and_minor_only_major(name): @@ -530,9 +465,6 @@ def _test_interpreter_version_info_parses_values_to_struct_impl(env, target): _tests.append(_test_interpreter_version_info_parses_values_to_struct) def _test_version_info_from_flag(name): - if not config.enable_pystar: - rt_util.skip_test(name) - return py_runtime( name = name + "_subject", interpreter_version_info = None, diff --git a/tests/py_runtime_info/py_runtime_info_tests.bzl b/tests/py_runtime_info/py_runtime_info_tests.bzl index 9acf541683..a44fb60c2f 100644 --- a/tests/py_runtime_info/py_runtime_info_tests.bzl +++ b/tests/py_runtime_info/py_runtime_info_tests.bzl @@ -16,18 +16,13 @@ load("@rules_testing//lib:analysis_test.bzl", "analysis_test") load("@rules_testing//lib:test_suite.bzl", "test_suite") load("//python:py_runtime_info.bzl", "PyRuntimeInfo") -load("//python/private:util.bzl", "IS_BAZEL_7_OR_HIGHER") # buildifier: disable=bzl-visibility def _create_py_runtime_info_without_interpreter_version_info_impl(ctx): - kwargs = {} - if IS_BAZEL_7_OR_HIGHER: - kwargs["bootstrap_template"] = ctx.attr.bootstrap_template - return [PyRuntimeInfo( interpreter = ctx.file.interpreter, files = depset(ctx.files.files), python_version = "PY3", - **kwargs + bootstrap_template = ctx.attr.bootstrap_template, )] _create_py_runtime_info_without_interpreter_version_info = rule( diff --git a/tests/pypi/whl_library_targets/whl_library_targets_tests.bzl b/tests/pypi/whl_library_targets/whl_library_targets_tests.bzl index ec7ca63832..615358f35d 100644 --- a/tests/pypi/whl_library_targets/whl_library_targets_tests.bzl +++ b/tests/pypi/whl_library_targets/whl_library_targets_tests.bzl @@ -15,7 +15,6 @@ "" load("@rules_testing//lib:test_suite.bzl", "test_suite") -load("//python/private:glob_excludes.bzl", "glob_excludes") # buildifier: disable=bzl-visibility load( "//python/private/pypi:whl_library_targets.bzl", "whl_library_targets", @@ -217,7 +216,6 @@ def _test_whl_and_library_deps_from_requires(env): filegroup = lambda **kwargs: filegroup_calls.append(kwargs), config_setting = lambda **_: None, glob = mock_glob.glob, - select = _select, ), rules = struct( py_library = lambda **kwargs: py_library_calls.append(kwargs), @@ -230,7 +228,7 @@ def _test_whl_and_library_deps_from_requires(env): { "name": "whl", "srcs": ["foo-0-py3-none-any.whl"], - "data": ["@pypi//bar:whl"] + _select({ + "data": ["@pypi//bar:whl"] + select({ ":is_include_bar_baz_true": ["@pypi//bar_baz:whl"], "//conditions:default": [], }), @@ -245,14 +243,14 @@ def _test_whl_and_library_deps_from_requires(env): env.expect.that_dict(py_library_call).contains_exactly({ "name": "pkg", - "srcs": ["site-packages/foo/SRCS.py"] + _select({ + "srcs": ["site-packages/foo/SRCS.py"] + select({ Label("//python/config_settings:is_venvs_site_packages"): [], "//conditions:default": ["_create_inits_target"], }), "pyi_srcs": ["site-packages/foo/PYI.pyi"], "data": ["site-packages/foo/DATA.txt"], "imports": ["site-packages"], - "deps": ["@pypi//bar:pkg"] + _select({ + "deps": ["@pypi//bar:pkg"] + select({ ":is_include_bar_baz_true": ["@pypi//bar_baz:pkg"], "//conditions:default": [], }), @@ -276,7 +274,7 @@ def _test_whl_and_library_deps_from_requires(env): "**/*.pyc", "**/*.pyc.*", "**/*.dist-info/RECORD", - ] + glob_excludes.version_dependent_exclusions(), + ], ), # pyi call _glob_call(["site-packages/**/*.pyi"], allow_empty = True), @@ -321,7 +319,6 @@ def _test_whl_and_library_deps(env): filegroup = lambda **kwargs: filegroup_calls.append(kwargs), config_setting = lambda **_: None, glob = mock_glob.glob, - select = _select, ), rules = struct( py_library = lambda **kwargs: py_library_calls.append(kwargs), @@ -336,7 +333,7 @@ def _test_whl_and_library_deps(env): "data": [ "@pypi_bar_baz//:whl", "@pypi_foo//:whl", - ] + _select( + ] + select( { Label("//python/config_settings:is_python_3.9"): ["@pypi_py39_dep//:whl"], "@platforms//cpu:aarch64": ["@pypi_arm_dep//:whl"], @@ -357,7 +354,7 @@ def _test_whl_and_library_deps(env): return env.expect.that_dict(py_library_calls[0]).contains_exactly({ "name": "pkg", - "srcs": ["site-packages/foo/SRCS.py"] + _select({ + "srcs": ["site-packages/foo/SRCS.py"] + select({ Label("//python/config_settings:is_venvs_site_packages"): [], "//conditions:default": ["_create_inits_target"], }), @@ -367,7 +364,7 @@ def _test_whl_and_library_deps(env): "deps": [ "@pypi_bar_baz//:pkg", "@pypi_foo//:pkg", - ] + _select( + ] + select( { Label("//python/config_settings:is_python_3.9"): ["@pypi_py39_dep//:pkg"], "@platforms//cpu:aarch64": ["@pypi_arm_dep//:pkg"], @@ -415,7 +412,6 @@ def _test_group(env): config_setting = lambda **_: None, glob = mock_glob.glob, alias = lambda **kwargs: alias_calls.append(kwargs), - select = _select, ), rules = struct( py_library = lambda **kwargs: py_library_calls.append(kwargs), @@ -437,14 +433,14 @@ def _test_group(env): py_library_call, ).contains_exactly({ "name": "_pkg", - "srcs": ["site-packages/foo/srcs.py"] + _select({ + "srcs": ["site-packages/foo/srcs.py"] + select({ Label("//python/config_settings:is_venvs_site_packages"): [], "//conditions:default": ["_create_inits_target"], }), "pyi_srcs": ["site-packages/foo/pyi.pyi"], "data": ["site-packages/foo/data.txt"], "imports": ["site-packages"], - "deps": ["@pypi_bar_baz//:pkg"] + _select({ + "deps": ["@pypi_bar_baz//:pkg"] + select({ "@platforms//os:linux": ["@pypi_box//:pkg"], ":is_linux_x86_64": ["@pypi_box//:pkg", "@pypi_box_amd64//:pkg"], "//conditions:default": [], @@ -491,13 +487,6 @@ def _mock_glob(): ) return mock -def _select(*args, **kwargs): - """We need to have this mock select because we still need to support bazel 6.""" - return [struct( - select = args, - kwargs = kwargs, - )] - def whl_library_targets_test_suite(name): """create the test suite. diff --git a/tests/runtime_env_toolchain/runtime_env_toolchain_tests.bzl b/tests/runtime_env_toolchain/runtime_env_toolchain_tests.bzl index aa4d1c793b..527448ebbc 100644 --- a/tests/runtime_env_toolchain/runtime_env_toolchain_tests.bzl +++ b/tests/runtime_env_toolchain/runtime_env_toolchain_tests.bzl @@ -24,7 +24,6 @@ load( "PY_CC_TOOLCHAIN_TYPE", "TARGET_TOOLCHAIN_TYPE", ) # buildifier: disable=bzl-visibility -load("//python/private:util.bzl", "IS_BAZEL_7_OR_HIGHER") # buildifier: disable=bzl-visibility load("//tests/support:support.bzl", "CC_TOOLCHAIN") _LookupInfo = provider() # buildifier: disable=provider-params @@ -55,25 +54,14 @@ def _test_runtime_env_toolchain_matches(name): name = name + "_subject", ) extra_toolchains = [ + # We have to add a cc toolchain because py_cc toolchain depends on it. + # However, that package also defines a different fake py_cc toolchain we + # don't want to use, so we need to ensure the runtime_env toolchain has + # higher precendence. + CC_TOOLCHAIN, str(Label("//python/runtime_env_toolchains:all")), ] - # We have to add a cc toolchain because py_cc toolchain depends on it. - # However, that package also defines a different fake py_cc toolchain we - # don't want to use, so we need to ensure the runtime_env toolchain has - # higher precendence. - # However, Bazel 6 and Bazel 7 process --extra_toolchains in different - # orders: - # * Bazel 6 goes left to right - # * Bazel 7 goes right to left - # We could just put our preferred toolchain before *and* after - # the undesired toolchain... - # However, Bazel 7 has a bug where *duplicate* entries are ignored, - # and only the *first* entry is respected. - if IS_BAZEL_7_OR_HIGHER: - extra_toolchains.insert(0, CC_TOOLCHAIN) - else: - extra_toolchains.append(CC_TOOLCHAIN) analysis_test( name = name, impl = _test_runtime_env_toolchain_matches_impl, diff --git a/tests/support/support.bzl b/tests/support/support.bzl index 37d3488316..96c6ad902a 100644 --- a/tests/support/support.bzl +++ b/tests/support/support.bzl @@ -20,7 +20,6 @@ # places. load("//python/private:bzlmod_enabled.bzl", "BZLMOD_ENABLED") # buildifier: disable=bzl-visibility -load("//python/private:util.bzl", "IS_BAZEL_7_OR_HIGHER") # buildifier: disable=bzl-visibility MAC = Label("//tests/support:mac") MAC_X86_64 = Label("//tests/support:mac_x86_64") @@ -40,7 +39,7 @@ CUSTOM_RUNTIME = str(Label("//tests/support:custom_runtime")) SUPPORTS_BOOTSTRAP_SCRIPT = select({ "@platforms//os:windows": ["@platforms//:incompatible"], "//conditions:default": [], -}) if IS_BAZEL_7_OR_HIGHER else ["@platforms//:incompatible"] +}) SUPPORTS_BZLMOD_UNIXY = select({ "@platforms//os:windows": ["@platforms//:incompatible"], diff --git a/tests/support/whl_from_dir/whl_from_dir_repo.bzl b/tests/support/whl_from_dir/whl_from_dir_repo.bzl index 176525636c..4e16e8ee4a 100644 --- a/tests/support/whl_from_dir/whl_from_dir_repo.bzl +++ b/tests/support/whl_from_dir/whl_from_dir_repo.bzl @@ -8,7 +8,7 @@ load("//python/private:repo_utils.bzl", "repo_utils") # buildifier: disable=bzl def _whl_from_dir_repo(rctx): root = rctx.path(rctx.attr.root).dirname - repo_utils.watch_tree(rctx, root) + rctx.watch_tree(root) output = rctx.path(rctx.attr.output) repo_utils.execute_checked( From 65f4c6e08f11afc218b48907daaa942af43a77c5 Mon Sep 17 00:00:00 2001 From: Ignas Anikevicius <240938+aignas@users.noreply.github.com> Date: Sun, 28 Sep 2025 12:09:48 +0900 Subject: [PATCH 456/922] refactor: rename symbols in re-exports (#3300) A followup to #3282 to finish up the cleanup and remove the unnecessary `starlark` usage in naming. --- python/py_binary.bzl | 7 ++----- python/py_cc_link_params_info.bzl | 4 ++-- python/py_info.bzl | 4 ++-- python/py_library.bzl | 7 ++----- python/py_runtime.bzl | 7 ++----- python/py_runtime_info.bzl | 4 ++-- python/py_runtime_pair.bzl | 4 +--- python/py_test.bzl | 7 ++----- 8 files changed, 15 insertions(+), 29 deletions(-) diff --git a/python/py_binary.bzl b/python/py_binary.bzl index 4e26a29af2..68541ab842 100644 --- a/python/py_binary.bzl +++ b/python/py_binary.bzl @@ -14,12 +14,9 @@ """Public entry point for py_binary.""" -load("//python/private:py_binary_macro.bzl", _starlark_py_binary = "py_binary") +load("//python/private:py_binary_macro.bzl", _py_binary = "py_binary") load("//python/private:register_extension_info.bzl", "register_extension_info") -# buildifier: disable=native-python -_py_binary_impl = _starlark_py_binary - def py_binary(**attrs): """Creates an executable Python program. @@ -40,7 +37,7 @@ def py_binary(**attrs): if attrs.get("srcs_version") in ("PY2", "PY2ONLY"): fail("Python 2 is no longer supported: https://github.com/bazel-contrib/rules_python/issues/886") - _py_binary_impl(**attrs) + _py_binary(**attrs) register_extension_info( extension = py_binary, diff --git a/python/py_cc_link_params_info.bzl b/python/py_cc_link_params_info.bzl index 6c510d6c8e..9f25f17f4b 100644 --- a/python/py_cc_link_params_info.bzl +++ b/python/py_cc_link_params_info.bzl @@ -1,5 +1,5 @@ """Public entry point for PyCcLinkParamsInfo.""" -load("//python/private:py_cc_link_params_info.bzl", _starlark_PyCcLinkParamsInfo = "PyCcLinkParamsInfo") +load("//python/private:py_cc_link_params_info.bzl", _PyCcLinkParamsInfo = "PyCcLinkParamsInfo") -PyCcLinkParamsInfo = _starlark_PyCcLinkParamsInfo +PyCcLinkParamsInfo = _PyCcLinkParamsInfo diff --git a/python/py_info.bzl b/python/py_info.bzl index 5582d3b491..350a4dbd9e 100644 --- a/python/py_info.bzl +++ b/python/py_info.bzl @@ -14,6 +14,6 @@ """Public entry point for PyInfo.""" -load("//python/private:py_info.bzl", _starlark_PyInfo = "PyInfo") +load("//python/private:py_info.bzl", _PyInfo = "PyInfo") -PyInfo = _starlark_PyInfo +PyInfo = _PyInfo diff --git a/python/py_library.bzl b/python/py_library.bzl index 4b79d8f0eb..277593b0c5 100644 --- a/python/py_library.bzl +++ b/python/py_library.bzl @@ -14,12 +14,9 @@ """Public entry point for py_library.""" -load("//python/private:py_library_macro.bzl", _starlark_py_library = "py_library") +load("//python/private:py_library_macro.bzl", _py_library = "py_library") load("//python/private:register_extension_info.bzl", "register_extension_info") -# buildifier: disable=native-python -_py_library_impl = _starlark_py_library - def py_library(**attrs): """Creates an executable Python program. @@ -37,7 +34,7 @@ def py_library(**attrs): if attrs.get("srcs_version") in ("PY2", "PY2ONLY"): fail("Python 2 is no longer supported: https://github.com/bazel-contrib/rules_python/issues/886") - _py_library_impl(**attrs) + _py_library(**attrs) register_extension_info( extension = py_library, diff --git a/python/py_runtime.bzl b/python/py_runtime.bzl index 8c3cee2eb7..86f644a770 100644 --- a/python/py_runtime.bzl +++ b/python/py_runtime.bzl @@ -14,10 +14,7 @@ """Public entry point for py_runtime.""" -load("//python/private:py_runtime_macro.bzl", _starlark_py_runtime = "py_runtime") - -# buildifier: disable=native-python -_py_runtime_impl = _starlark_py_runtime +load("//python/private:py_runtime_macro.bzl", _py_runtime = "py_runtime") def py_runtime(**attrs): """Creates an executable Python program. @@ -38,4 +35,4 @@ def py_runtime(**attrs): if attrs.get("python_version") == "PY2": fail("Python 2 is no longer supported: see https://github.com/bazel-contrib/rules_python/issues/886") - _py_runtime_impl(**attrs) + _py_runtime(**attrs) diff --git a/python/py_runtime_info.bzl b/python/py_runtime_info.bzl index 082a9b0f19..ac7b1b0fb6 100644 --- a/python/py_runtime_info.bzl +++ b/python/py_runtime_info.bzl @@ -14,6 +14,6 @@ """Public entry point for PyRuntimeInfo.""" -load("//python/private:py_runtime_info.bzl", _starlark_PyRuntimeInfo = "PyRuntimeInfo") +load("//python/private:py_runtime_info.bzl", _PyRuntimeInfo = "PyRuntimeInfo") -PyRuntimeInfo = _starlark_PyRuntimeInfo +PyRuntimeInfo = _PyRuntimeInfo diff --git a/python/py_runtime_pair.bzl b/python/py_runtime_pair.bzl index 97cc4f5f18..f44c722d54 100644 --- a/python/py_runtime_pair.bzl +++ b/python/py_runtime_pair.bzl @@ -14,9 +14,7 @@ """Public entry point for py_runtime_pair.""" -load("//python/private:py_runtime_pair_macro.bzl", _starlark_impl = "py_runtime_pair") - -_py_runtime_pair = _starlark_impl +load("//python/private:py_runtime_pair_macro.bzl", _py_runtime_pair = "py_runtime_pair") # NOTE: This doc is copy/pasted from the builtin py_runtime_pair rule so our # doc generator gives useful API docs. diff --git a/python/py_test.bzl b/python/py_test.bzl index 5b8ad31725..70f8ff5d09 100644 --- a/python/py_test.bzl +++ b/python/py_test.bzl @@ -14,12 +14,9 @@ """Public entry point for py_test.""" -load("//python/private:py_test_macro.bzl", _starlark_py_test = "py_test") +load("//python/private:py_test_macro.bzl", _py_test = "py_test") load("//python/private:register_extension_info.bzl", "register_extension_info") -# buildifier: disable=native-python -_py_test_impl = _starlark_py_test - def py_test(**attrs): """Creates an executable Python program. @@ -41,7 +38,7 @@ def py_test(**attrs): fail("Python 2 is no longer supported: https://github.com/bazel-contrib/rules_python/issues/886") # buildifier: disable=native-python - _py_test_impl(**attrs) + _py_test(**attrs) register_extension_info( extension = py_test, From 5dfd1993ab797b179e5e85064edd2638263dab73 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Sun, 28 Sep 2025 14:56:02 -0700 Subject: [PATCH 457/922] chore: use python.defaults to set rules_python default python version (#3301) python.defaults is the modern way to set the default. Setting it this way also helps avoid a bug where if a root module has a single `python.toolchain()` call (which are implicitly treated as `is_default=True`) and also sets the default using `python.defaults()`, some validation logic gives an error about using both ways to set a default. --- MODULE.bazel | 6 +- python/private/python.bzl | 145 +++++++++++++++++++++------------- tests/python/python_tests.bzl | 134 ++++++++++++++++++++++--------- 3 files changed, 191 insertions(+), 94 deletions(-) diff --git a/MODULE.bazel b/MODULE.bazel index 6251ed4c3c..a29a898772 100644 --- a/MODULE.bazel +++ b/MODULE.bazel @@ -42,8 +42,12 @@ python = use_extension("//python/extensions:python.bzl", "python") # NOTE: This is not a stable version. It is provided for convenience, but will # change frequently to track the most recent Python version. # NOTE: The root module can override this. +# NOTE: There must be a corresponding `python.toolchain()` call for the version +# specified here. +python.defaults( + python_version = "3.11", +) python.toolchain( - is_default = True, python_version = "3.11", ) use_repo( diff --git a/python/private/python.bzl b/python/private/python.bzl index faad53fab4..a1fe80e0ce 100644 --- a/python/private/python.bzl +++ b/python/private/python.bzl @@ -85,46 +85,7 @@ def parse_modules(*, module_ctx, logger, _fail = fail): config = _get_toolchain_config(modules = module_ctx.modules, _fail = _fail) - default_python_version = None - for mod in module_ctx.modules: - defaults_attr_structs = _create_defaults_attr_structs(mod = mod) - default_python_version_env = None - default_python_version_file = None - - # Only the root module and rules_python are allowed to specify the default - # toolchain for a couple reasons: - # * It prevents submodules from specifying different defaults and only - # one of them winning. - # * rules_python needs to set a soft default in case the root module doesn't, - # e.g. if the root module doesn't use Python itself. - # * The root module is allowed to override the rules_python default. - if mod.is_root or (mod.name == "rules_python" and not default_python_version): - for defaults_attr in defaults_attr_structs: - default_python_version = _one_or_the_same( - default_python_version, - defaults_attr.python_version, - onerror = _fail_multiple_defaults_python_version, - ) - default_python_version_env = _one_or_the_same( - default_python_version_env, - defaults_attr.python_version_env, - onerror = _fail_multiple_defaults_python_version_env, - ) - default_python_version_file = _one_or_the_same( - default_python_version_file, - defaults_attr.python_version_file, - onerror = _fail_multiple_defaults_python_version_file, - ) - if default_python_version_file: - default_python_version = _one_or_the_same( - default_python_version, - module_ctx.read(default_python_version_file, watch = "yes").strip(), - ) - if default_python_version_env: - default_python_version = module_ctx.getenv( - default_python_version_env, - default_python_version, - ) + default_python_version = _compute_default_python_version(module_ctx) seen_versions = {} for mod in module_ctx.modules: @@ -152,13 +113,7 @@ def parse_modules(*, module_ctx, logger, _fail = fail): # * rules_python needs to set a soft default in case the root module doesn't, # e.g. if the root module doesn't use Python itself. # * The root module is allowed to override the rules_python default. - if default_python_version: - is_default = default_python_version == toolchain_version - if toolchain_attr.is_default and not is_default: - fail("The 'is_default' attribute doesn't work if you set " + - "the default Python version with the `defaults` tag.") - else: - is_default = toolchain_attr.is_default + is_default = default_python_version == toolchain_version # Also only the root module should be able to decide ignore_root_user_error. # Modules being depended upon don't know the final environment, so they aren't @@ -169,15 +124,15 @@ def parse_modules(*, module_ctx, logger, _fail = fail): fail("Toolchains in the root module must have consistent 'ignore_root_user_error' attributes") ignore_root_user_error = toolchain_attr.ignore_root_user_error - elif mod.name == "rules_python" and not default_toolchain and not default_python_version: - # We don't do the len() check because we want the default that rules_python - # sets to be clearly visible. - is_default = toolchain_attr.is_default + elif mod.name == "rules_python" and not default_toolchain: + # This branch handles when the root module doesn't declare a + # Python toolchain + is_default = default_python_version == toolchain_version else: is_default = False if is_default and default_toolchain != None: - _fail_multiple_default_toolchains( + _fail_multiple_default_toolchains_chosen( first = default_toolchain.name, second = toolchain_name, ) @@ -577,14 +532,24 @@ def _fail_multiple_defaults_python_version_env(first, second): second = second, )) -def _fail_multiple_default_toolchains(first, second): +def _fail_multiple_default_toolchains_chosen(first, second): fail(("Multiple default toolchains: only one toolchain " + - "can have is_default=True. First default " + + "can be chosen as a default. First default " + "was toolchain '{first}'. Second was '{second}'").format( first = first, second = second, )) +def _fail_multiple_default_toolchains_in_module(mod, toolchain_attrs): + fail(("Multiple default toolchains: only one toolchain " + + "can have is_default=True.\n" + + "Module '{module}' contains {count} toolchains with " + + "is_default=True: {versions}").format( + module = mod.name, + count = len(toolchain_attrs), + versions = ", ".join(sorted([v.python_version for v in toolchain_attrs])), + )) + def _validate_version(version_str, *, _fail = fail): v = version.parse(version_str, strict = True, _fail = _fail) if v == None: @@ -880,6 +845,72 @@ def _get_toolchain_config(*, modules, _fail = fail): register_all_versions = register_all_versions, ) +def _compute_default_python_version(mctx): + default_python_version = None + for mod in mctx.modules: + # Only the root module and rules_python are allowed to specify the default + # toolchain for a couple reasons: + # * It prevents submodules from specifying different defaults and only + # one of them winning. + # * rules_python needs to set a soft default in case the root module doesn't, + # e.g. if the root module doesn't use Python itself. + # * The root module is allowed to override the rules_python default. + if not (mod.is_root or mod.name == "rules_python"): + continue + + defaults_attr_structs = _create_defaults_attr_structs(mod = mod) + default_python_version_env = None + default_python_version_file = None + + for defaults_attr in defaults_attr_structs: + default_python_version = _one_or_the_same( + default_python_version, + defaults_attr.python_version, + onerror = _fail_multiple_defaults_python_version, + ) + default_python_version_env = _one_or_the_same( + default_python_version_env, + defaults_attr.python_version_env, + onerror = _fail_multiple_defaults_python_version_env, + ) + default_python_version_file = _one_or_the_same( + default_python_version_file, + defaults_attr.python_version_file, + onerror = _fail_multiple_defaults_python_version_file, + ) + if default_python_version_file: + default_python_version = _one_or_the_same( + default_python_version, + mctx.read(default_python_version_file, watch = "yes").strip(), + ) + if default_python_version_env: + default_python_version = mctx.getenv( + default_python_version_env, + default_python_version, + ) + + if default_python_version: + break + + # Otherwise, look at legacy python.toolchain() calls for a default + toolchain_attrs = mod.tags.toolchain + + # Convenience: if one python.toolchain() call exists, treat it as + # the default. + if len(toolchain_attrs) == 1: + default_python_version = toolchain_attrs[0].python_version + else: + sets_default = [v for v in toolchain_attrs if v.is_default] + if len(sets_default) == 1: + default_python_version = sets_default[0].python_version + elif len(sets_default) > 1: + _fail_multiple_default_toolchains_in_module(mod, toolchain_attrs) + + if default_python_version: + break + + return default_python_version + def _create_defaults_attr_structs(*, mod): arg_structs = [] @@ -915,7 +946,11 @@ def _create_toolchain_attr_structs(*, mod, config, seen_versions): return arg_structs -def _create_toolchain_attrs_struct(*, tag = None, python_version = None, toolchain_tag_count = None): +def _create_toolchain_attrs_struct( + *, + tag = None, + python_version = None, + toolchain_tag_count = None): if tag and python_version: fail("Only one of tag and python version can be specified") if tag: diff --git a/tests/python/python_tests.bzl b/tests/python/python_tests.bzl index 9081a0e306..96d78d13df 100644 --- a/tests/python/python_tests.bzl +++ b/tests/python/python_tests.bzl @@ -12,7 +12,7 @@ # See the License for the specific language governing permissions and # limitations under the License. -"" +"""Unit tests for //python/extensions:python.bzl bzlmod extension.""" load("@pythons_hub//:versions.bzl", "MINOR_MAPPING") load("@rules_testing//lib:test_suite.bzl", "test_suite") @@ -43,7 +43,8 @@ def _mock_mctx(*modules, environ = {}, mocked_files = {}): ], ) -def _mod(*, name, defaults = [], toolchain = [], override = [], single_version_override = [], single_version_platform_override = [], is_root = True): +# todo: change is_root to false by default. most modules aren't root +def _mod(*, name, defaults = [], toolchain = [], override = [], single_version_override = [], single_version_platform_override = [], is_root = False): return struct( name = name, tags = struct( @@ -88,6 +89,15 @@ def _override( register_all_versions = register_all_versions, ) +def _rules_python_module(is_root = False): + """A mock of what the real rules_python MODULE.bazel looks like.""" + return _mod( + name = "rules_python", + defaults = [_defaults(python_version = "3.11")], + toolchain = [_toolchain("3.11")], + is_root = is_root, + ) + def _single_version_override( python_version = "", sha256 = {}, @@ -138,10 +148,11 @@ def _single_version_platform_override( arch = "", ) -def _test_default(env): +def _test_default_from_rules_python_when_rules_python_is_root(env): + """Verify that rules_python (as root module) default is applied.""" py = parse_modules( module_ctx = _mock_mctx( - _mod(name = "rules_python", toolchain = [_toolchain("3.11")]), + _rules_python_module(is_root = True), ), logger = repo_utils.logger(verbosity_level = 0, name = "python"), ) @@ -167,12 +178,13 @@ def _test_default(env): ) env.expect.that_collection(py.toolchains).contains_exactly([want_toolchain]) -_tests.append(_test_default) +_tests.append(_test_default_from_rules_python_when_rules_python_is_root) -def _test_default_some_module(env): +def _test_default_from_rules_python_when_rules_python_is_not_root(env): + """Verify that rules_python default applies when rules_python is not the root module.""" py = parse_modules( module_ctx = _mock_mctx( - _mod(name = "rules_python", toolchain = [_toolchain("3.11")], is_root = False), + _rules_python_module(), ), logger = repo_utils.logger(verbosity_level = 0, name = "python"), ) @@ -186,12 +198,13 @@ def _test_default_some_module(env): ) env.expect.that_collection(py.toolchains).contains_exactly([want_toolchain]) -_tests.append(_test_default_some_module) +_tests.append(_test_default_from_rules_python_when_rules_python_is_not_root) def _test_default_with_patch_version(env): py = parse_modules( module_ctx = _mock_mctx( - _mod(name = "rules_python", toolchain = [_toolchain("3.11.2")]), + _mod(name = "alpha", toolchain = [_toolchain("3.11.2")], is_root = True), + _rules_python_module(is_root = True), ), logger = repo_utils.logger(verbosity_level = 0, name = "python"), ) @@ -203,39 +216,19 @@ def _test_default_with_patch_version(env): python_version = "3.11.2", register_coverage_tool = False, ) - env.expect.that_collection(py.toolchains).contains_exactly([want_toolchain]) + env.expect.that_collection(py.toolchains).contains_at_least([want_toolchain]) _tests.append(_test_default_with_patch_version) -def _test_default_non_rules_python(env): - py = parse_modules( - module_ctx = _mock_mctx( - # NOTE @aignas 2024-09-06: the first item in the module_ctx.modules - # could be a non-root module, which is the case if the root module - # does not make any calls to the extension. - _mod(name = "rules_python", toolchain = [_toolchain("3.11")], is_root = False), - ), - logger = repo_utils.logger(verbosity_level = 0, name = "python"), - ) - - env.expect.that_str(py.default_python_version).equals("3.11") - rules_python_toolchain = struct( - name = "python_3_11", - python_version = "3.11", - register_coverage_tool = False, - ) - env.expect.that_collection(py.toolchains).contains_exactly([rules_python_toolchain]) - -_tests.append(_test_default_non_rules_python) - def _test_default_non_rules_python_ignore_root_user_error(env): py = parse_modules( module_ctx = _mock_mctx( _mod( name = "my_module", toolchain = [_toolchain("3.12", ignore_root_user_error = False)], + is_root = True, ), - _mod(name = "rules_python", toolchain = [_toolchain("3.11")]), + _rules_python_module(), ), logger = repo_utils.logger(verbosity_level = 0, name = "python"), ) @@ -261,11 +254,12 @@ def _test_default_non_rules_python_ignore_root_user_error(env): _tests.append(_test_default_non_rules_python_ignore_root_user_error) def _test_default_non_rules_python_ignore_root_user_error_non_root_module(env): + """Verify a non-root intermediate module has its ignore_root_user_error setting ignored.""" py = parse_modules( module_ctx = _mock_mctx( - _mod(name = "my_module", toolchain = [_toolchain("3.13")]), + _mod(name = "my_module", is_root = True, toolchain = [_toolchain("3.13")]), _mod(name = "some_module", toolchain = [_toolchain("3.12", ignore_root_user_error = False)]), - _mod(name = "rules_python", toolchain = [_toolchain("3.11")]), + _rules_python_module(), ), logger = repo_utils.logger(verbosity_level = 0, name = "python"), ) @@ -310,8 +304,9 @@ def _test_toolchain_ordering(env): _toolchain("3.11.10"), _toolchain("3.11.13", is_default = True), ], + is_root = True, ), - _mod(name = "rules_python", toolchain = [_toolchain("3.11")]), + _rules_python_module(), ), logger = repo_utils.logger(verbosity_level = 0, name = "python"), ) @@ -433,12 +428,68 @@ def _test_default_from_defaults_file(env): _tests.append(_test_default_from_defaults_file) +def _test_default_from_single_toolchain(env): + py = parse_modules( + module_ctx = _mock_mctx( + _mod( + name = "my_root_module", + toolchain = [_toolchain("3.12")], + is_root = True, + ), + _rules_python_module(), + ), + logger = repo_utils.logger(verbosity_level = 0, name = "python"), + ) + env.expect.that_str(py.default_python_version).equals("3.12") + +_tests.append(_test_default_from_single_toolchain) + +def _test_defaults_overrides_single_toolchain(env): + py = parse_modules( + module_ctx = _mock_mctx( + _mod( + name = "my_root_module", + defaults = [ + # This relies on rules_python registering 3.11 + _defaults(python_version = "3.11"), + ], + toolchain = [_toolchain("3.12")], + is_root = True, + ), + _rules_python_module(), + ), + logger = repo_utils.logger(verbosity_level = 0, name = "python"), + ) + env.expect.that_str(py.default_python_version).equals("3.11") + +_tests.append(_test_defaults_overrides_single_toolchain) + +def _test_defaults_overrides_toolchains_setting_is_default(env): + py = parse_modules( + module_ctx = _mock_mctx( + _mod( + name = "my_root_module", + defaults = [_defaults(python_version = "3.13")], + toolchain = [ + _toolchain("3.13"), + _toolchain("3.12", is_default = True), + ], + is_root = True, + ), + _rules_python_module(), + ), + logger = repo_utils.logger(verbosity_level = 0, name = "python"), + ) + env.expect.that_str(py.default_python_version).equals("3.13") + +_tests.append(_test_defaults_overrides_toolchains_setting_is_default) + def _test_first_occurance_of_the_toolchain_wins(env): py = parse_modules( module_ctx = _mock_mctx( - _mod(name = "my_module", toolchain = [_toolchain("3.12")]), + _mod(name = "my_module", is_root = True, toolchain = [_toolchain("3.12")]), _mod(name = "some_module", toolchain = [_toolchain("3.12", configure_coverage_tool = True)]), - _mod(name = "rules_python", toolchain = [_toolchain("3.11")]), + _rules_python_module(), environ = { "RULES_PYTHON_BZLMOD_DEBUG": "1", }, @@ -486,8 +537,9 @@ def _test_auth_overrides(env): auth_patterns = {"foo": "bar"}, ), ], + is_root = True, ), - _mod(name = "rules_python", toolchain = [_toolchain("3.11")]), + _rules_python_module(), ), logger = repo_utils.logger(verbosity_level = 0, name = "python"), ) @@ -521,6 +573,7 @@ def _test_add_new_version(env): module_ctx = _mock_mctx( _mod( name = "my_module", + is_root = True, toolchain = [_toolchain("3.13")], single_version_override = [ _single_version_override( @@ -601,6 +654,7 @@ def _test_register_all_versions(env): module_ctx = _mock_mctx( _mod( name = "my_module", + is_root = True, toolchain = [_toolchain("3.13")], single_version_override = [ _single_version_override( @@ -666,6 +720,7 @@ def _test_add_patches(env): module_ctx = _mock_mctx( _mod( name = "my_module", + is_root = True, toolchain = [_toolchain("3.13")], single_version_override = [ _single_version_override( @@ -744,6 +799,7 @@ def _test_fail_two_overrides(env): module_ctx = _mock_mctx( _mod( name = "my_module", + is_root = True, toolchain = [_toolchain("3.13")], override = [ _override(base_url = "foo"), @@ -775,6 +831,7 @@ def _test_single_version_override_errors(env): module_ctx = _mock_mctx( _mod( name = "my_module", + is_root = True, toolchain = [_toolchain("3.13")], single_version_override = test.overrides, ), @@ -815,6 +872,7 @@ def _test_single_version_platform_override_errors(env): name = "my_module", toolchain = [_toolchain("3.13")], single_version_platform_override = test.overrides, + is_root = True, ), ), _fail = lambda *a: errors.append(" ".join(a)), From 08bd45101a5887a8b1449f39ff29764b3fe19d79 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Sun, 28 Sep 2025 18:38:01 -0700 Subject: [PATCH 458/922] docs: fix spelling of venvs_site_packages flag in changelog (#3302) The flag name for the `--venvs_site_packages` flag was misspelled in the changelog. --- CHANGELOG.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 469e9d3612..2912700e31 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -79,7 +79,7 @@ END_UNRELEASED_TEMPLATE length errors due to too long environment variables. * (bootstrap) {obj}`--bootstrap_impl=script` now supports the `-S` interpreter setting. -* (venvs) {obj}`--vens_site_packages=yes` no longer errors when packages with +* (venvs) {obj}`--venvs_site_packages=yes` no longer errors when packages with overlapping files or directories are used together. ([#3204](https://github.com/bazel-contrib/rules_python/issues/3204)). From 3ad6cb464e926609015632a9c5ded1232de34b5b Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Sun, 28 Sep 2025 18:39:44 -0700 Subject: [PATCH 459/922] fix(uv): make uv lock rule work with platform python runtime (#3303) A platform runtime is when the interpreter is provided as an absolute path to Python instead of bundled in the runfiles. The uv lock rule almost worked, it just didn't handle the runtime files being empty in such a case. To fix, use an empty depset to satisfy the subsequent APIs that use it. --- CHANGELOG.md | 2 ++ python/uv/private/lock.bzl | 2 +- 2 files changed, 3 insertions(+), 1 deletion(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 2912700e31..ff2257a06b 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -82,6 +82,8 @@ END_UNRELEASED_TEMPLATE * (venvs) {obj}`--venvs_site_packages=yes` no longer errors when packages with overlapping files or directories are used together. ([#3204](https://github.com/bazel-contrib/rules_python/issues/3204)). +* (uv) {obj}`//python/uv:lock.bzl%lock` now works with a local platform + runtime. {#v0-0-0-added} ### Added diff --git a/python/uv/private/lock.bzl b/python/uv/private/lock.bzl index 281a0decc0..b007baf9c1 100644 --- a/python/uv/private/lock.bzl +++ b/python/uv/private/lock.bzl @@ -106,7 +106,7 @@ def _lock_impl(ctx): exec_tools = ctx.toolchains[EXEC_TOOLS_TOOLCHAIN_TYPE].exec_tools runtime = exec_tools.exec_interpreter[platform_common.ToolchainInfo].py3_runtime python = runtime.interpreter or runtime.interpreter_path - python_files = runtime.files + python_files = runtime.files or depset() args.add("--python", python) args.add_all(srcs) From c4935b1f3566a2793fae7d01f85ebe15a3addf71 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Sun, 28 Sep 2025 22:17:58 -0700 Subject: [PATCH 460/922] docs: add config bzlmod extension to docs (#3305) The original PR omitted the target from the docs, so it was absent. Along the way, fix some incorrect deps for its target so docs build. --- docs/BUILD.bazel | 1 + python/private/BUILD.bazel | 5 ++++- 2 files changed, 5 insertions(+), 1 deletion(-) diff --git a/docs/BUILD.bazel b/docs/BUILD.bazel index c36ed5722a..ffe800a72c 100644 --- a/docs/BUILD.bazel +++ b/docs/BUILD.bazel @@ -106,6 +106,7 @@ sphinx_stardocs( "//python/cc:py_cc_toolchain_bzl", "//python/cc:py_cc_toolchain_info_bzl", "//python/entry_points:py_console_script_binary_bzl", + "//python/extensions:config_bzl", "//python/extensions:python_bzl", "//python/local_toolchains:repos_bzl", "//python/private:attr_builders_bzl", diff --git a/python/private/BUILD.bazel b/python/private/BUILD.bazel index 1bcd0f678f..c77417892b 100644 --- a/python/private/BUILD.bazel +++ b/python/private/BUILD.bazel @@ -197,7 +197,10 @@ bzl_library( bzl_library( name = "internal_config_repo_bzl", srcs = ["internal_config_repo.bzl"], - deps = [":bzlmod_enabled_bzl"], + deps = [ + ":repo_utils_bzl", + ":text_util_bzl", + ], ) bzl_library( From 87906f060a4d68cf223476a27e117ba65fd1ea03 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Mon, 29 Sep 2025 01:08:28 -0700 Subject: [PATCH 461/922] fix: handle urls without release id format (#3306) Sometimes when users customize the URLs for release, they use urls that don't have the same format the python-build-standalone URLs. Namely, they may not have the release_id component of the url. This would result in an error parsing such urls. To fix, refine the url parsing to check if a component is a valid numeric release id. Fixes https://github.com/bazel-contrib/rules_python/issues/3285 --- python/versions.bzl | 7 ++- tests/get_release_info/BUILD.bazel | 24 ++++++++ .../get_release_info_tests.bzl | 56 +++++++++++++++++++ 3 files changed, 86 insertions(+), 1 deletion(-) create mode 100644 tests/get_release_info/BUILD.bazel create mode 100644 tests/get_release_info/get_release_info_tests.bzl diff --git a/python/versions.bzl b/python/versions.bzl index 30929f82fd..34cffc5664 100644 --- a/python/versions.bzl +++ b/python/versions.bzl @@ -1045,7 +1045,12 @@ def get_release_info(platform, python_version, base_url = DEFAULT_RELEASE_BASE_U for u in url: p, _, _ = platform.partition(FREETHREADED) - release_id = int(u.split("/")[-2]) + # Assume an unknown release_id is a newer url format + release_id = 99999999 + url_parts = u.split("/") + if len(url_parts) >= 2 and url_parts[-2].isdigit(): + maybe_release_id = url_parts[-2] + release_id = int(maybe_release_id) if FREETHREADED.lstrip("-") in platform: build = "{}+{}-full".format( diff --git a/tests/get_release_info/BUILD.bazel b/tests/get_release_info/BUILD.bazel new file mode 100644 index 0000000000..26517e6dec --- /dev/null +++ b/tests/get_release_info/BUILD.bazel @@ -0,0 +1,24 @@ +# Copyright 2024 The Bazel Authors. All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +load(":get_release_info_tests.bzl", "get_release_info_test_suite") + +package( + default_testonly = True, + default_visibility = ["//:__subpackages__"], +) + +licenses(["notice"]) + +get_release_info_test_suite(name = "get_release_info") diff --git a/tests/get_release_info/get_release_info_tests.bzl b/tests/get_release_info/get_release_info_tests.bzl new file mode 100644 index 0000000000..ca553a3c4b --- /dev/null +++ b/tests/get_release_info/get_release_info_tests.bzl @@ -0,0 +1,56 @@ +# Copyright 2024 The Bazel Authors. All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +"""Tests for get_release_info.""" + +load("@rules_testing//lib:test_suite.bzl", "test_suite") +load("//python:versions.bzl", "get_release_info") # buildifier: disable=bzl-visibility + +_tests = [] + +def _test_file_url(env): + """Tests that a file:/// url is handled correctly.""" + tool_versions = { + "3.11.5": { + "sha256": { + "x86_64-unknown-linux-gnu": "fbed6f7694b2faae5d7c401a856219c945397f772eea5ca50c6eb825cbc9d1e1", + }, + "strip_prefix": "python", + "url": "file:///tmp/cpython-3.11.5.tar.gz", + }, + } + + expected_url = "file:///tmp/cpython-3.11.5.tar.gz" + expected_filename = "file:///tmp/cpython-3.11.5.tar.gz" + + filename, urls, strip_prefix, patches, patch_strip = get_release_info( + platform = "x86_64-unknown-linux-gnu", + python_version = "3.11.5", + tool_versions = tool_versions, + ) + + env.expect.that_str(filename).equals(expected_filename) + env.expect.that_collection(urls).contains_exactly([expected_url]) + env.expect.that_str(strip_prefix).equals("python") + env.expect.that_collection(patches).has_size(0) + env.expect.that_bool(patch_strip == None).equals(True) + +_tests.append(_test_file_url) + +def get_release_info_test_suite(name): + """Defines the test suite for get_release_info.""" + test_suite( + name = name, + basic_tests = _tests, + ) From e89605a2de41766c7855ef7b09551ba2f98868f3 Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Tue, 30 Sep 2025 17:25:32 +0900 Subject: [PATCH 462/922] build(deps): bump pyyaml from 6.0.2 to 6.0.3 in /docs (#3308) Bumps [pyyaml](https://github.com/yaml/pyyaml) from 6.0.2 to 6.0.3.
Release notes

Sourced from pyyaml's releases.

6.0.3

What's Changed

  • Support for Python 3.14 and free-threading (experimental).

Full Changelog: https://github.com/yaml/pyyaml/compare/6.0.2...6.0.3

Changelog

Sourced from pyyaml's changelog.

6.0.3 (2025-09-25)

  • yaml/pyyaml#864 -- Support for Python 3.14 and free-threading (experimental)
Commits

[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=pyyaml&package-manager=pip&previous-version=6.0.2&new-version=6.0.3)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) ---
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot show ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> --- docs/requirements.txt | 128 ++++++++++++++++++++++++------------------ 1 file changed, 74 insertions(+), 54 deletions(-) diff --git a/docs/requirements.txt b/docs/requirements.txt index c05554aeeb..a83e3197ba 100644 --- a/docs/requirements.txt +++ b/docs/requirements.txt @@ -240,60 +240,80 @@ pygments==2.19.2 \ --hash=sha256:636cb2477cec7f8952536970bc533bc43743542f70392ae026374600add5b887 \ --hash=sha256:86540386c03d588bb81d44bc3928634ff26449851e99741617ecb9037ee5ec0b # via sphinx -pyyaml==6.0.2 \ - --hash=sha256:01179a4a8559ab5de078078f37e5c1a30d76bb88519906844fd7bdea1b7729ff \ - --hash=sha256:0833f8694549e586547b576dcfaba4a6b55b9e96098b36cdc7ebefe667dfed48 \ - --hash=sha256:0a9a2848a5b7feac301353437eb7d5957887edbf81d56e903999a75a3d743086 \ - --hash=sha256:0b69e4ce7a131fe56b7e4d770c67429700908fc0752af059838b1cfb41960e4e \ - --hash=sha256:0ffe8360bab4910ef1b9e87fb812d8bc0a308b0d0eef8c8f44e0254ab3b07133 \ - --hash=sha256:11d8f3dd2b9c1207dcaf2ee0bbbfd5991f571186ec9cc78427ba5bd32afae4b5 \ - --hash=sha256:17e311b6c678207928d649faa7cb0d7b4c26a0ba73d41e99c4fff6b6c3276484 \ - --hash=sha256:1e2120ef853f59c7419231f3bf4e7021f1b936f6ebd222406c3b60212205d2ee \ - --hash=sha256:1f71ea527786de97d1a0cc0eacd1defc0985dcf6b3f17bb77dcfc8c34bec4dc5 \ - --hash=sha256:23502f431948090f597378482b4812b0caae32c22213aecf3b55325e049a6c68 \ - --hash=sha256:24471b829b3bf607e04e88d79542a9d48bb037c2267d7927a874e6c205ca7e9a \ - --hash=sha256:29717114e51c84ddfba879543fb232a6ed60086602313ca38cce623c1d62cfbf \ - --hash=sha256:2e99c6826ffa974fe6e27cdb5ed0021786b03fc98e5ee3c5bfe1fd5015f42b99 \ - --hash=sha256:39693e1f8320ae4f43943590b49779ffb98acb81f788220ea932a6b6c51004d8 \ - --hash=sha256:3ad2a3decf9aaba3d29c8f537ac4b243e36bef957511b4766cb0057d32b0be85 \ - --hash=sha256:3b1fdb9dc17f5a7677423d508ab4f243a726dea51fa5e70992e59a7411c89d19 \ - --hash=sha256:41e4e3953a79407c794916fa277a82531dd93aad34e29c2a514c2c0c5fe971cc \ - --hash=sha256:43fa96a3ca0d6b1812e01ced1044a003533c47f6ee8aca31724f78e93ccc089a \ - --hash=sha256:50187695423ffe49e2deacb8cd10510bc361faac997de9efef88badc3bb9e2d1 \ - --hash=sha256:5ac9328ec4831237bec75defaf839f7d4564be1e6b25ac710bd1a96321cc8317 \ - --hash=sha256:5d225db5a45f21e78dd9358e58a98702a0302f2659a3c6cd320564b75b86f47c \ - --hash=sha256:6395c297d42274772abc367baaa79683958044e5d3835486c16da75d2a694631 \ - --hash=sha256:688ba32a1cffef67fd2e9398a2efebaea461578b0923624778664cc1c914db5d \ - --hash=sha256:68ccc6023a3400877818152ad9a1033e3db8625d899c72eacb5a668902e4d652 \ - --hash=sha256:70b189594dbe54f75ab3a1acec5f1e3faa7e8cf2f1e08d9b561cb41b845f69d5 \ - --hash=sha256:797b4f722ffa07cc8d62053e4cff1486fa6dc094105d13fea7b1de7d8bf71c9e \ - --hash=sha256:7c36280e6fb8385e520936c3cb3b8042851904eba0e58d277dca80a5cfed590b \ - --hash=sha256:7e7401d0de89a9a855c839bc697c079a4af81cf878373abd7dc625847d25cbd8 \ - --hash=sha256:80bab7bfc629882493af4aa31a4cfa43a4c57c83813253626916b8c7ada83476 \ - --hash=sha256:82d09873e40955485746739bcb8b4586983670466c23382c19cffecbf1fd8706 \ - --hash=sha256:8388ee1976c416731879ac16da0aff3f63b286ffdd57cdeb95f3f2e085687563 \ - --hash=sha256:8824b5a04a04a047e72eea5cec3bc266db09e35de6bdfe34c9436ac5ee27d237 \ - --hash=sha256:8b9c7197f7cb2738065c481a0461e50ad02f18c78cd75775628afb4d7137fb3b \ - --hash=sha256:9056c1ecd25795207ad294bcf39f2db3d845767be0ea6e6a34d856f006006083 \ - --hash=sha256:936d68689298c36b53b29f23c6dbb74de12b4ac12ca6cfe0e047bedceea56180 \ - --hash=sha256:9b22676e8097e9e22e36d6b7bda33190d0d400f345f23d4065d48f4ca7ae0425 \ - --hash=sha256:a4d3091415f010369ae4ed1fc6b79def9416358877534caf6a0fdd2146c87a3e \ - --hash=sha256:a8786accb172bd8afb8be14490a16625cbc387036876ab6ba70912730faf8e1f \ - --hash=sha256:a9f8c2e67970f13b16084e04f134610fd1d374bf477b17ec1599185cf611d725 \ - --hash=sha256:bc2fa7c6b47d6bc618dd7fb02ef6fdedb1090ec036abab80d4681424b84c1183 \ - --hash=sha256:c70c95198c015b85feafc136515252a261a84561b7b1d51e3384e0655ddf25ab \ - --hash=sha256:cc1c1159b3d456576af7a3e4d1ba7e6924cb39de8f67111c735f6fc832082774 \ - --hash=sha256:ce826d6ef20b1bc864f0a68340c8b3287705cae2f8b4b1d932177dcc76721725 \ - --hash=sha256:d584d9ec91ad65861cc08d42e834324ef890a082e591037abe114850ff7bbc3e \ - --hash=sha256:d7fded462629cfa4b685c5416b949ebad6cec74af5e2d42905d41e257e0869f5 \ - --hash=sha256:d84a1718ee396f54f3a086ea0a66d8e552b2ab2017ef8b420e92edbc841c352d \ - --hash=sha256:d8e03406cac8513435335dbab54c0d385e4a49e4945d2909a581c83647ca0290 \ - --hash=sha256:e10ce637b18caea04431ce14fabcf5c64a1c61ec9c56b071a4b7ca131ca52d44 \ - --hash=sha256:ec031d5d2feb36d1d1a24380e4db6d43695f3748343d99434e6f5f9156aaa2ed \ - --hash=sha256:ef6107725bd54b262d6dedcc2af448a266975032bc85ef0172c5f059da6325b4 \ - --hash=sha256:efdca5630322a10774e8e98e1af481aad470dd62c3170801852d752aa7a783ba \ - --hash=sha256:f753120cb8181e736c57ef7636e83f31b9c0d1722c516f7e86cf15b7aa57ff12 \ - --hash=sha256:ff3824dc5261f50c9b0dfb3be22b4567a6f938ccce4587b38952d85fd9e9afe4 +pyyaml==6.0.3 \ + --hash=sha256:00c4bdeba853cc34e7dd471f16b4114f4162dc03e6b7afcc2128711f0eca823c \ + --hash=sha256:0150219816b6a1fa26fb4699fb7daa9caf09eb1999f3b70fb6e786805e80375a \ + --hash=sha256:02893d100e99e03eda1c8fd5c441d8c60103fd175728e23e431db1b589cf5ab3 \ + --hash=sha256:02ea2dfa234451bbb8772601d7b8e426c2bfa197136796224e50e35a78777956 \ + --hash=sha256:0f29edc409a6392443abf94b9cf89ce99889a1dd5376d94316ae5145dfedd5d6 \ + --hash=sha256:10892704fc220243f5305762e276552a0395f7beb4dbf9b14ec8fd43b57f126c \ + --hash=sha256:16249ee61e95f858e83976573de0f5b2893b3677ba71c9dd36b9cf8be9ac6d65 \ + --hash=sha256:1d37d57ad971609cf3c53ba6a7e365e40660e3be0e5175fa9f2365a379d6095a \ + --hash=sha256:1ebe39cb5fc479422b83de611d14e2c0d3bb2a18bbcb01f229ab3cfbd8fee7a0 \ + --hash=sha256:214ed4befebe12df36bcc8bc2b64b396ca31be9304b8f59e25c11cf94a4c033b \ + --hash=sha256:2283a07e2c21a2aa78d9c4442724ec1eb15f5e42a723b99cb3d822d48f5f7ad1 \ + --hash=sha256:22ba7cfcad58ef3ecddc7ed1db3409af68d023b7f940da23c6c2a1890976eda6 \ + --hash=sha256:27c0abcb4a5dac13684a37f76e701e054692a9b2d3064b70f5e4eb54810553d7 \ + --hash=sha256:28c8d926f98f432f88adc23edf2e6d4921ac26fb084b028c733d01868d19007e \ + --hash=sha256:2e71d11abed7344e42a8849600193d15b6def118602c4c176f748e4583246007 \ + --hash=sha256:34d5fcd24b8445fadc33f9cf348c1047101756fd760b4dacb5c3e99755703310 \ + --hash=sha256:37503bfbfc9d2c40b344d06b2199cf0e96e97957ab1c1b546fd4f87e53e5d3e4 \ + --hash=sha256:3c5677e12444c15717b902a5798264fa7909e41153cdf9ef7ad571b704a63dd9 \ + --hash=sha256:3ff07ec89bae51176c0549bc4c63aa6202991da2d9a6129d7aef7f1407d3f295 \ + --hash=sha256:41715c910c881bc081f1e8872880d3c650acf13dfa8214bad49ed4cede7c34ea \ + --hash=sha256:418cf3f2111bc80e0933b2cd8cd04f286338bb88bdc7bc8e6dd775ebde60b5e0 \ + --hash=sha256:44edc647873928551a01e7a563d7452ccdebee747728c1080d881d68af7b997e \ + --hash=sha256:4a2e8cebe2ff6ab7d1050ecd59c25d4c8bd7e6f400f5f82b96557ac0abafd0ac \ + --hash=sha256:4ad1906908f2f5ae4e5a8ddfce73c320c2a1429ec52eafd27138b7f1cbe341c9 \ + --hash=sha256:501a031947e3a9025ed4405a168e6ef5ae3126c59f90ce0cd6f2bfc477be31b7 \ + --hash=sha256:5190d403f121660ce8d1d2c1bb2ef1bd05b5f68533fc5c2ea899bd15f4399b35 \ + --hash=sha256:5498cd1645aa724a7c71c8f378eb29ebe23da2fc0d7a08071d89469bf1d2defb \ + --hash=sha256:5cf4e27da7e3fbed4d6c3d8e797387aaad68102272f8f9752883bc32d61cb87b \ + --hash=sha256:5e0b74767e5f8c593e8c9b5912019159ed0533c70051e9cce3e8b6aa699fcd69 \ + --hash=sha256:5ed875a24292240029e4483f9d4a4b8a1ae08843b9c54f43fcc11e404532a8a5 \ + --hash=sha256:5fcd34e47f6e0b794d17de1b4ff496c00986e1c83f7ab2fb8fcfe9616ff7477b \ + --hash=sha256:5fdec68f91a0c6739b380c83b951e2c72ac0197ace422360e6d5a959d8d97b2c \ + --hash=sha256:6344df0d5755a2c9a276d4473ae6b90647e216ab4757f8426893b5dd2ac3f369 \ + --hash=sha256:64386e5e707d03a7e172c0701abfb7e10f0fb753ee1d773128192742712a98fd \ + --hash=sha256:652cb6edd41e718550aad172851962662ff2681490a8a711af6a4d288dd96824 \ + --hash=sha256:66291b10affd76d76f54fad28e22e51719ef9ba22b29e1d7d03d6777a9174198 \ + --hash=sha256:66e1674c3ef6f541c35191caae2d429b967b99e02040f5ba928632d9a7f0f065 \ + --hash=sha256:6adc77889b628398debc7b65c073bcb99c4a0237b248cacaf3fe8a557563ef6c \ + --hash=sha256:79005a0d97d5ddabfeeea4cf676af11e647e41d81c9a7722a193022accdb6b7c \ + --hash=sha256:7c6610def4f163542a622a73fb39f534f8c101d690126992300bf3207eab9764 \ + --hash=sha256:7f047e29dcae44602496db43be01ad42fc6f1cc0d8cd6c83d342306c32270196 \ + --hash=sha256:8098f252adfa6c80ab48096053f512f2321f0b998f98150cea9bd23d83e1467b \ + --hash=sha256:850774a7879607d3a6f50d36d04f00ee69e7fc816450e5f7e58d7f17f1ae5c00 \ + --hash=sha256:8d1fab6bb153a416f9aeb4b8763bc0f22a5586065f86f7664fc23339fc1c1fac \ + --hash=sha256:8da9669d359f02c0b91ccc01cac4a67f16afec0dac22c2ad09f46bee0697eba8 \ + --hash=sha256:8dc52c23056b9ddd46818a57b78404882310fb473d63f17b07d5c40421e47f8e \ + --hash=sha256:9149cad251584d5fb4981be1ecde53a1ca46c891a79788c0df828d2f166bda28 \ + --hash=sha256:93dda82c9c22deb0a405ea4dc5f2d0cda384168e466364dec6255b293923b2f3 \ + --hash=sha256:96b533f0e99f6579b3d4d4995707cf36df9100d67e0c8303a0c55b27b5f99bc5 \ + --hash=sha256:9c57bb8c96f6d1808c030b1687b9b5fb476abaa47f0db9c0101f5e9f394e97f4 \ + --hash=sha256:9c7708761fccb9397fe64bbc0395abcae8c4bf7b0eac081e12b809bf47700d0b \ + --hash=sha256:9f3bfb4965eb874431221a3ff3fdcddc7e74e3b07799e0e84ca4a0f867d449bf \ + --hash=sha256:a33284e20b78bd4a18c8c2282d549d10bc8408a2a7ff57653c0cf0b9be0afce5 \ + --hash=sha256:a80cb027f6b349846a3bf6d73b5e95e782175e52f22108cfa17876aaeff93702 \ + --hash=sha256:b30236e45cf30d2b8e7b3e85881719e98507abed1011bf463a8fa23e9c3e98a8 \ + --hash=sha256:b3bc83488de33889877a0f2543ade9f70c67d66d9ebb4ac959502e12de895788 \ + --hash=sha256:b865addae83924361678b652338317d1bd7e79b1f4596f96b96c77a5a34b34da \ + --hash=sha256:b8bb0864c5a28024fac8a632c443c87c5aa6f215c0b126c449ae1a150412f31d \ + --hash=sha256:ba1cc08a7ccde2d2ec775841541641e4548226580ab850948cbfda66a1befcdc \ + --hash=sha256:bdb2c67c6c1390b63c6ff89f210c8fd09d9a1217a465701eac7316313c915e4c \ + --hash=sha256:c1ff362665ae507275af2853520967820d9124984e0f7466736aea23d8611fba \ + --hash=sha256:c2514fceb77bc5e7a2f7adfaa1feb2fb311607c9cb518dbc378688ec73d8292f \ + --hash=sha256:c3355370a2c156cffb25e876646f149d5d68f5e0a3ce86a5084dd0b64a994917 \ + --hash=sha256:c458b6d084f9b935061bc36216e8a69a7e293a2f1e68bf956dcd9e6cbcd143f5 \ + --hash=sha256:d0eae10f8159e8fdad514efdc92d74fd8d682c933a6dd088030f3834bc8e6b26 \ + --hash=sha256:d76623373421df22fb4cf8817020cbb7ef15c725b9d5e45f17e189bfc384190f \ + --hash=sha256:ebc55a14a21cb14062aa4162f906cd962b28e2e9ea38f9b4391244cd8de4ae0b \ + --hash=sha256:eda16858a3cab07b80edaf74336ece1f986ba330fdb8ee0d6c0d68fe82bc96be \ + --hash=sha256:ee2922902c45ae8ccada2c5b501ab86c36525b883eff4255313a253a3160861c \ + --hash=sha256:efd7b85f94a6f21e4932043973a7ba2613b059c4a000551892ac9f1d11f5baf3 \ + --hash=sha256:f7057c9a337546edc7973c0d3ba84ddcdf0daa14533c2065749c9075001090e6 \ + --hash=sha256:fa160448684b4e94d80416c0fa4aac48967a969efe22931448d853ada8baf926 \ + --hash=sha256:fc09d0aa354569bc501d4e787133afc08552722d3ab34836a80547331bb5d4a0 # via myst-parser readthedocs-sphinx-ext==2.2.5 \ --hash=sha256:ee5fd5b99db9f0c180b2396cbce528aa36671951b9526bb0272dbfce5517bd27 \ From b913add58acb7c1932ca648a2fa1feb6983ba52e Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Tue, 30 Sep 2025 08:26:10 +0000 Subject: [PATCH 463/922] build(deps): bump markupsafe from 3.0.2 to 3.0.3 in /docs (#3307) Bumps [markupsafe](https://github.com/pallets/markupsafe) from 3.0.2 to 3.0.3.
Release notes

Sourced from markupsafe's releases.

3.0.3

This is the MarkupSafe 3.0.3 fix release, which fixes bugs but does not otherwise change behavior and should not result in breaking changes compared to the latest feature release.

PyPI: https://pypi.org/project/MarkupSafe/3.0.3/ Changes: https://markupsafe.palletsprojects.com/page/changes/#version-3-0-3 Milestone: https://github.com/pallets/markupsafe/milestone/15?closed=1

  • __version__ raises DeprecationWarning instead of UserWarning. #487
  • Adopt multi-phase initialization PEP 489 for the C extension. #494
  • Build Windows ARM64 wheels. #485
  • Build Python 3.14 wheels. #503
  • Build riscv64 wheels. #505
Changelog

Sourced from markupsafe's changelog.

Version 3.0.3

Released 2025-09-27

  • __version__ raises DeprecationWarning instead of UserWarning. :issue:487
  • Adopt multi-phase initialisation (:pep:489) for the C extension. :issue:494
  • Build Windows ARM64 wheels. :issue:485
  • Build Python 3.14 wheels. :issue:503
  • Build riscv64 wheels. :issue:505
Commits

[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=markupsafe&package-manager=pip&previous-version=3.0.2&new-version=3.0.3)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) ---
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot show ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> --- docs/requirements.txt | 152 +++++++++++++++++++++++++----------------- 1 file changed, 90 insertions(+), 62 deletions(-) diff --git a/docs/requirements.txt b/docs/requirements.txt index a83e3197ba..c40b2960b5 100644 --- a/docs/requirements.txt +++ b/docs/requirements.txt @@ -143,68 +143,96 @@ markdown-it-py==3.0.0 \ # via # mdit-py-plugins # myst-parser -markupsafe==3.0.2 \ - --hash=sha256:0bff5e0ae4ef2e1ae4fdf2dfd5b76c75e5c2fa4132d05fc1b0dabcd20c7e28c4 \ - --hash=sha256:0f4ca02bea9a23221c0182836703cbf8930c5e9454bacce27e767509fa286a30 \ - --hash=sha256:1225beacc926f536dc82e45f8a4d68502949dc67eea90eab715dea3a21c1b5f0 \ - --hash=sha256:131a3c7689c85f5ad20f9f6fb1b866f402c445b220c19fe4308c0b147ccd2ad9 \ - --hash=sha256:15ab75ef81add55874e7ab7055e9c397312385bd9ced94920f2802310c930396 \ - --hash=sha256:1a9d3f5f0901fdec14d8d2f66ef7d035f2157240a433441719ac9a3fba440b13 \ - --hash=sha256:1c99d261bd2d5f6b59325c92c73df481e05e57f19837bdca8413b9eac4bd8028 \ - --hash=sha256:1e084f686b92e5b83186b07e8a17fc09e38fff551f3602b249881fec658d3eca \ - --hash=sha256:2181e67807fc2fa785d0592dc2d6206c019b9502410671cc905d132a92866557 \ - --hash=sha256:2cb8438c3cbb25e220c2ab33bb226559e7afb3baec11c4f218ffa7308603c832 \ - --hash=sha256:3169b1eefae027567d1ce6ee7cae382c57fe26e82775f460f0b2778beaad66c0 \ - --hash=sha256:3809ede931876f5b2ec92eef964286840ed3540dadf803dd570c3b7e13141a3b \ - --hash=sha256:38a9ef736c01fccdd6600705b09dc574584b89bea478200c5fbf112a6b0d5579 \ - --hash=sha256:3d79d162e7be8f996986c064d1c7c817f6df3a77fe3d6859f6f9e7be4b8c213a \ - --hash=sha256:444dcda765c8a838eaae23112db52f1efaf750daddb2d9ca300bcae1039adc5c \ - --hash=sha256:48032821bbdf20f5799ff537c7ac3d1fba0ba032cfc06194faffa8cda8b560ff \ - --hash=sha256:4aa4e5faecf353ed117801a068ebab7b7e09ffb6e1d5e412dc852e0da018126c \ - --hash=sha256:52305740fe773d09cffb16f8ed0427942901f00adedac82ec8b67752f58a1b22 \ - --hash=sha256:569511d3b58c8791ab4c2e1285575265991e6d8f8700c7be0e88f86cb0672094 \ - --hash=sha256:57cb5a3cf367aeb1d316576250f65edec5bb3be939e9247ae594b4bcbc317dfb \ - --hash=sha256:5b02fb34468b6aaa40dfc198d813a641e3a63b98c2b05a16b9f80b7ec314185e \ - --hash=sha256:6381026f158fdb7c72a168278597a5e3a5222e83ea18f543112b2662a9b699c5 \ - --hash=sha256:6af100e168aa82a50e186c82875a5893c5597a0c1ccdb0d8b40240b1f28b969a \ - --hash=sha256:6c89876f41da747c8d3677a2b540fb32ef5715f97b66eeb0c6b66f5e3ef6f59d \ - --hash=sha256:6e296a513ca3d94054c2c881cc913116e90fd030ad1c656b3869762b754f5f8a \ - --hash=sha256:70a87b411535ccad5ef2f1df5136506a10775d267e197e4cf531ced10537bd6b \ - --hash=sha256:7e94c425039cde14257288fd61dcfb01963e658efbc0ff54f5306b06054700f8 \ - --hash=sha256:846ade7b71e3536c4e56b386c2a47adf5741d2d8b94ec9dc3e92e5e1ee1e2225 \ - --hash=sha256:88416bd1e65dcea10bc7569faacb2c20ce071dd1f87539ca2ab364bf6231393c \ - --hash=sha256:88b49a3b9ff31e19998750c38e030fc7bb937398b1f78cfa599aaef92d693144 \ - --hash=sha256:8c4e8c3ce11e1f92f6536ff07154f9d49677ebaaafc32db9db4620bc11ed480f \ - --hash=sha256:8e06879fc22a25ca47312fbe7c8264eb0b662f6db27cb2d3bbbc74b1df4b9b87 \ - --hash=sha256:9025b4018f3a1314059769c7bf15441064b2207cb3f065e6ea1e7359cb46db9d \ - --hash=sha256:93335ca3812df2f366e80509ae119189886b0f3c2b81325d39efdb84a1e2ae93 \ - --hash=sha256:9778bd8ab0a994ebf6f84c2b949e65736d5575320a17ae8984a77fab08db94cf \ - --hash=sha256:9e2d922824181480953426608b81967de705c3cef4d1af983af849d7bd619158 \ - --hash=sha256:a123e330ef0853c6e822384873bef7507557d8e4a082961e1defa947aa59ba84 \ - --hash=sha256:a904af0a6162c73e3edcb969eeeb53a63ceeb5d8cf642fade7d39e7963a22ddb \ - --hash=sha256:ad10d3ded218f1039f11a75f8091880239651b52e9bb592ca27de44eed242a48 \ - --hash=sha256:b424c77b206d63d500bcb69fa55ed8d0e6a3774056bdc4839fc9298a7edca171 \ - --hash=sha256:b5a6b3ada725cea8a5e634536b1b01c30bcdcd7f9c6fff4151548d5bf6b3a36c \ - --hash=sha256:ba8062ed2cf21c07a9e295d5b8a2a5ce678b913b45fdf68c32d95d6c1291e0b6 \ - --hash=sha256:ba9527cdd4c926ed0760bc301f6728ef34d841f405abf9d4f959c478421e4efd \ - --hash=sha256:bbcb445fa71794da8f178f0f6d66789a28d7319071af7a496d4d507ed566270d \ - --hash=sha256:bcf3e58998965654fdaff38e58584d8937aa3096ab5354d493c77d1fdd66d7a1 \ - --hash=sha256:c0ef13eaeee5b615fb07c9a7dadb38eac06a0608b41570d8ade51c56539e509d \ - --hash=sha256:cabc348d87e913db6ab4aa100f01b08f481097838bdddf7c7a84b7575b7309ca \ - --hash=sha256:cdb82a876c47801bb54a690c5ae105a46b392ac6099881cdfb9f6e95e4014c6a \ - --hash=sha256:cfad01eed2c2e0c01fd0ecd2ef42c492f7f93902e39a42fc9ee1692961443a29 \ - --hash=sha256:d16a81a06776313e817c951135cf7340a3e91e8c1ff2fac444cfd75fffa04afe \ - --hash=sha256:d8213e09c917a951de9d09ecee036d5c7d36cb6cb7dbaece4c71a60d79fb9798 \ - --hash=sha256:e07c3764494e3776c602c1e78e298937c3315ccc9043ead7e685b7f2b8d47b3c \ - --hash=sha256:e17c96c14e19278594aa4841ec148115f9c7615a47382ecb6b82bd8fea3ab0c8 \ - --hash=sha256:e444a31f8db13eb18ada366ab3cf45fd4b31e4db1236a4448f68778c1d1a5a2f \ - --hash=sha256:e6a2a455bd412959b57a172ce6328d2dd1f01cb2135efda2e4576e8a23fa3b0f \ - --hash=sha256:eaa0a10b7f72326f1372a713e73c3f739b524b3af41feb43e4921cb529f5929a \ - --hash=sha256:eb7972a85c54febfb25b5c4b4f3af4dcc731994c7da0d8a0b4a6eb0640e1d178 \ - --hash=sha256:ee55d3edf80167e48ea11a923c7386f4669df67d7994554387f84e7d8b0a2bf0 \ - --hash=sha256:f3818cb119498c0678015754eba762e0d61e5b52d34c8b13d770f0719f7b1d79 \ - --hash=sha256:f8b3d067f2e40fe93e1ccdd6b2e1d16c43140e76f02fb1319a05cf2b79d99430 \ - --hash=sha256:fcabf5ff6eea076f859677f5f0b6b5c1a51e70a376b0579e0eadef8db48c6b50 +markupsafe==3.0.3 \ + --hash=sha256:0303439a41979d9e74d18ff5e2dd8c43ed6c6001fd40e5bf2e43f7bd9bbc523f \ + --hash=sha256:068f375c472b3e7acbe2d5318dea141359e6900156b5b2ba06a30b169086b91a \ + --hash=sha256:0bf2a864d67e76e5c9a34dc26ec616a66b9888e25e7b9460e1c76d3293bd9dbf \ + --hash=sha256:0db14f5dafddbb6d9208827849fad01f1a2609380add406671a26386cdf15a19 \ + --hash=sha256:0eb9ff8191e8498cca014656ae6b8d61f39da5f95b488805da4bb029cccbfbaf \ + --hash=sha256:0f4b68347f8c5eab4a13419215bdfd7f8c9b19f2b25520968adfad23eb0ce60c \ + --hash=sha256:1085e7fbddd3be5f89cc898938f42c0b3c711fdcb37d75221de2666af647c175 \ + --hash=sha256:116bb52f642a37c115f517494ea5feb03889e04df47eeff5b130b1808ce7c219 \ + --hash=sha256:12c63dfb4a98206f045aa9563db46507995f7ef6d83b2f68eda65c307c6829eb \ + --hash=sha256:133a43e73a802c5562be9bbcd03d090aa5a1fe899db609c29e8c8d815c5f6de6 \ + --hash=sha256:1353ef0c1b138e1907ae78e2f6c63ff67501122006b0f9abad68fda5f4ffc6ab \ + --hash=sha256:15d939a21d546304880945ca1ecb8a039db6b4dc49b2c5a400387cdae6a62e26 \ + --hash=sha256:177b5253b2834fe3678cb4a5f0059808258584c559193998be2601324fdeafb1 \ + --hash=sha256:1872df69a4de6aead3491198eaf13810b565bdbeec3ae2dc8780f14458ec73ce \ + --hash=sha256:1b4b79e8ebf6b55351f0d91fe80f893b4743f104bff22e90697db1590e47a218 \ + --hash=sha256:1b52b4fb9df4eb9ae465f8d0c228a00624de2334f216f178a995ccdcf82c4634 \ + --hash=sha256:1ba88449deb3de88bd40044603fafffb7bc2b055d626a330323a9ed736661695 \ + --hash=sha256:1cc7ea17a6824959616c525620e387f6dd30fec8cb44f649e31712db02123dad \ + --hash=sha256:218551f6df4868a8d527e3062d0fb968682fe92054e89978594c28e642c43a73 \ + --hash=sha256:26a5784ded40c9e318cfc2bdb30fe164bdb8665ded9cd64d500a34fb42067b1c \ + --hash=sha256:2713baf880df847f2bece4230d4d094280f4e67b1e813eec43b4c0e144a34ffe \ + --hash=sha256:2a15a08b17dd94c53a1da0438822d70ebcd13f8c3a95abe3a9ef9f11a94830aa \ + --hash=sha256:2f981d352f04553a7171b8e44369f2af4055f888dfb147d55e42d29e29e74559 \ + --hash=sha256:32001d6a8fc98c8cb5c947787c5d08b0a50663d139f1305bac5885d98d9b40fa \ + --hash=sha256:3524b778fe5cfb3452a09d31e7b5adefeea8c5be1d43c4f810ba09f2ceb29d37 \ + --hash=sha256:3537e01efc9d4dccdf77221fb1cb3b8e1a38d5428920e0657ce299b20324d758 \ + --hash=sha256:35add3b638a5d900e807944a078b51922212fb3dedb01633a8defc4b01a3c85f \ + --hash=sha256:38664109c14ffc9e7437e86b4dceb442b0096dfe3541d7864d9cbe1da4cf36c8 \ + --hash=sha256:3a7e8ae81ae39e62a41ec302f972ba6ae23a5c5396c8e60113e9066ef893da0d \ + --hash=sha256:3b562dd9e9ea93f13d53989d23a7e775fdfd1066c33494ff43f5418bc8c58a5c \ + --hash=sha256:457a69a9577064c05a97c41f4e65148652db078a3a509039e64d3467b9e7ef97 \ + --hash=sha256:4bd4cd07944443f5a265608cc6aab442e4f74dff8088b0dfc8238647b8f6ae9a \ + --hash=sha256:4e885a3d1efa2eadc93c894a21770e4bc67899e3543680313b09f139e149ab19 \ + --hash=sha256:4faffd047e07c38848ce017e8725090413cd80cbc23d86e55c587bf979e579c9 \ + --hash=sha256:509fa21c6deb7a7a273d629cf5ec029bc209d1a51178615ddf718f5918992ab9 \ + --hash=sha256:5678211cb9333a6468fb8d8be0305520aa073f50d17f089b5b4b477ea6e67fdc \ + --hash=sha256:591ae9f2a647529ca990bc681daebdd52c8791ff06c2bfa05b65163e28102ef2 \ + --hash=sha256:5a7d5dc5140555cf21a6fefbdbf8723f06fcd2f63ef108f2854de715e4422cb4 \ + --hash=sha256:69c0b73548bc525c8cb9a251cddf1931d1db4d2258e9599c28c07ef3580ef354 \ + --hash=sha256:6b5420a1d9450023228968e7e6a9ce57f65d148ab56d2313fcd589eee96a7a50 \ + --hash=sha256:722695808f4b6457b320fdc131280796bdceb04ab50fe1795cd540799ebe1698 \ + --hash=sha256:729586769a26dbceff69f7a7dbbf59ab6572b99d94576a5592625d5b411576b9 \ + --hash=sha256:77f0643abe7495da77fb436f50f8dab76dbc6e5fd25d39589a0f1fe6548bfa2b \ + --hash=sha256:795e7751525cae078558e679d646ae45574b47ed6e7771863fcc079a6171a0fc \ + --hash=sha256:7be7b61bb172e1ed687f1754f8e7484f1c8019780f6f6b0786e76bb01c2ae115 \ + --hash=sha256:7c3fb7d25180895632e5d3148dbdc29ea38ccb7fd210aa27acbd1201a1902c6e \ + --hash=sha256:7e68f88e5b8799aa49c85cd116c932a1ac15caaa3f5db09087854d218359e485 \ + --hash=sha256:83891d0e9fb81a825d9a6d61e3f07550ca70a076484292a70fde82c4b807286f \ + --hash=sha256:8485f406a96febb5140bfeca44a73e3ce5116b2501ac54fe953e488fb1d03b12 \ + --hash=sha256:8709b08f4a89aa7586de0aadc8da56180242ee0ada3999749b183aa23df95025 \ + --hash=sha256:8f71bc33915be5186016f675cd83a1e08523649b0e33efdb898db577ef5bb009 \ + --hash=sha256:915c04ba3851909ce68ccc2b8e2cd691618c4dc4c4232fb7982bca3f41fd8c3d \ + --hash=sha256:949b8d66bc381ee8b007cd945914c721d9aba8e27f71959d750a46f7c282b20b \ + --hash=sha256:94c6f0bb423f739146aec64595853541634bde58b2135f27f61c1ffd1cd4d16a \ + --hash=sha256:9a1abfdc021a164803f4d485104931fb8f8c1efd55bc6b748d2f5774e78b62c5 \ + --hash=sha256:9b79b7a16f7fedff2495d684f2b59b0457c3b493778c9eed31111be64d58279f \ + --hash=sha256:a320721ab5a1aba0a233739394eb907f8c8da5c98c9181d1161e77a0c8e36f2d \ + --hash=sha256:a4afe79fb3de0b7097d81da19090f4df4f8d3a2b3adaa8764138aac2e44f3af1 \ + --hash=sha256:ad2cf8aa28b8c020ab2fc8287b0f823d0a7d8630784c31e9ee5edea20f406287 \ + --hash=sha256:b8512a91625c9b3da6f127803b166b629725e68af71f8184ae7e7d54686a56d6 \ + --hash=sha256:bc51efed119bc9cfdf792cdeaa4d67e8f6fcccab66ed4bfdd6bde3e59bfcbb2f \ + --hash=sha256:bdc919ead48f234740ad807933cdf545180bfbe9342c2bb451556db2ed958581 \ + --hash=sha256:bdd37121970bfd8be76c5fb069c7751683bdf373db1ed6c010162b2a130248ed \ + --hash=sha256:be8813b57049a7dc738189df53d69395eba14fb99345e0a5994914a3864c8a4b \ + --hash=sha256:c0c0b3ade1c0b13b936d7970b1d37a57acde9199dc2aecc4c336773e1d86049c \ + --hash=sha256:c47a551199eb8eb2121d4f0f15ae0f923d31350ab9280078d1e5f12b249e0026 \ + --hash=sha256:c4ffb7ebf07cfe8931028e3e4c85f0357459a3f9f9490886198848f4fa002ec8 \ + --hash=sha256:ccfcd093f13f0f0b7fdd0f198b90053bf7b2f02a3927a30e63f3ccc9df56b676 \ + --hash=sha256:d2ee202e79d8ed691ceebae8e0486bd9a2cd4794cec4824e1c99b6f5009502f6 \ + --hash=sha256:d53197da72cc091b024dd97249dfc7794d6a56530370992a5e1a08983ad9230e \ + --hash=sha256:d6dd0be5b5b189d31db7cda48b91d7e0a9795f31430b7f271219ab30f1d3ac9d \ + --hash=sha256:d88b440e37a16e651bda4c7c2b930eb586fd15ca7406cb39e211fcff3bf3017d \ + --hash=sha256:de8a88e63464af587c950061a5e6a67d3632e36df62b986892331d4620a35c01 \ + --hash=sha256:df2449253ef108a379b8b5d6b43f4b1a8e81a061d6537becd5582fba5f9196d7 \ + --hash=sha256:e1c1493fb6e50ab01d20a22826e57520f1284df32f2d8601fdd90b6304601419 \ + --hash=sha256:e1cf1972137e83c5d4c136c43ced9ac51d0e124706ee1c8aa8532c1287fa8795 \ + --hash=sha256:e2103a929dfa2fcaf9bb4e7c091983a49c9ac3b19c9061b6d5427dd7d14d81a1 \ + --hash=sha256:e56b7d45a839a697b5eb268c82a71bd8c7f6c94d6fd50c3d577fa39a9f1409f5 \ + --hash=sha256:e8afc3f2ccfa24215f8cb28dcf43f0113ac3c37c2f0f0806d8c70e4228c5cf4d \ + --hash=sha256:e8fc20152abba6b83724d7ff268c249fa196d8259ff481f3b1476383f8f24e42 \ + --hash=sha256:eaa9599de571d72e2daf60164784109f19978b327a3910d3e9de8c97b5b70cfe \ + --hash=sha256:ec15a59cf5af7be74194f7ab02d0f59a62bdcf1a537677ce67a2537c9b87fcda \ + --hash=sha256:f190daf01f13c72eac4efd5c430a8de82489d9cff23c364c3ea822545032993e \ + --hash=sha256:f34c41761022dd093b4b6896d4810782ffbabe30f2d443ff5f083e0cbbb8c737 \ + --hash=sha256:f3e98bb3798ead92273dc0e5fd0f31ade220f59a266ffd8a4f6065e0a3ce0523 \ + --hash=sha256:f42d0984e947b8adf7dd6dde396e720934d12c506ce84eea8476409563607591 \ + --hash=sha256:f71a396b3bf33ecaa1626c255855702aca4d3d9fea5e051b41ac59a9c1c41edc \ + --hash=sha256:f9e130248f4462aaa8e2552d547f36ddadbeaa573879158d721bbd33dfe4743a \ + --hash=sha256:fed51ac40f757d41b7c48425901843666a6677e3e8eb0abcff09e4ba6e664f50 # via jinja2 mdit-py-plugins==0.4.2 ; python_full_version < '3.10' \ --hash=sha256:0c673c3f889399a33b95e88d2f0d111b4447bdfea7f237dab2d488f459835636 \ From ecc339001eed1455a5ba2f85a6c31e7661f6e556 Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Tue, 30 Sep 2025 17:28:30 +0900 Subject: [PATCH 464/922] build(deps): bump docutils from 0.22 to 0.22.2 in /tools/publish (#3289) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit [//]: # (dependabot-start) ⚠️ **Dependabot is rebasing this PR** ⚠️ Rebasing might not happen immediately, so don't worry if this takes some time. Note: if you make any changes to this PR yourself, they will take precedence over the rebase. --- [//]: # (dependabot-end) [//]: # (dependabot-start) ⚠️ **Dependabot is rebasing this PR** ⚠️ Rebasing might not happen immediately, so don't worry if this takes some time. Note: if you make any changes to this PR yourself, they will take precedence over the rebase. --- [//]: # (dependabot-end) Bumps [docutils](https://github.com/rtfd/recommonmark) from 0.22 to 0.22.2.
Commits

[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=docutils&package-manager=pip&previous-version=0.22&new-version=0.22.2)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) ---
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot show ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> --- tools/publish/requirements_darwin.txt | 6 +++--- tools/publish/requirements_linux.txt | 6 +++--- tools/publish/requirements_universal.txt | 6 +++--- tools/publish/requirements_windows.txt | 6 +++--- 4 files changed, 12 insertions(+), 12 deletions(-) diff --git a/tools/publish/requirements_darwin.txt b/tools/publish/requirements_darwin.txt index 39248d4675..f24f3dc655 100644 --- a/tools/publish/requirements_darwin.txt +++ b/tools/publish/requirements_darwin.txt @@ -91,9 +91,9 @@ charset-normalizer==3.4.3 \ --hash=sha256:fd10de089bcdcd1be95a2f73dbe6254798ec1bda9f450d5828c96f93e2536b9c \ --hash=sha256:fdabf8315679312cfa71302f9bd509ded4f2f263fb5b765cf1433b39106c3cc9 # via requests -docutils==0.22 \ - --hash=sha256:4ed966a0e96a0477d852f7af31bdcb3adc049fbb35ccba358c2ea8a03287615e \ - --hash=sha256:ba9d57750e92331ebe7c08a1bbf7a7f8143b86c476acd51528b042216a6aad0f +docutils==0.22.2 \ + --hash=sha256:9fdb771707c8784c8f2728b67cb2c691305933d68137ef95a75db5f4dfbc213d \ + --hash=sha256:b0e98d679283fc3bb0ead8a5da7f501baa632654e7056e9c5846842213d674d8 # via readme-renderer idna==3.10 \ --hash=sha256:12f65c9b470abda6dc35cf8e63cc574b1c52b11df2c86030af0ac09b01b13ea9 \ diff --git a/tools/publish/requirements_linux.txt b/tools/publish/requirements_linux.txt index c078a0ed61..76426fa019 100644 --- a/tools/publish/requirements_linux.txt +++ b/tools/publish/requirements_linux.txt @@ -216,9 +216,9 @@ cryptography==45.0.7 \ --hash=sha256:f5414a788ecc6ee6bc58560e85ca624258a55ca434884445440a810796ea0e0b \ --hash=sha256:fa26fa54c0a9384c27fcdc905a2fb7d60ac6e47d14bc2692145f2b3b1e2cfdbd # via secretstorage -docutils==0.22 \ - --hash=sha256:4ed966a0e96a0477d852f7af31bdcb3adc049fbb35ccba358c2ea8a03287615e \ - --hash=sha256:ba9d57750e92331ebe7c08a1bbf7a7f8143b86c476acd51528b042216a6aad0f +docutils==0.22.2 \ + --hash=sha256:9fdb771707c8784c8f2728b67cb2c691305933d68137ef95a75db5f4dfbc213d \ + --hash=sha256:b0e98d679283fc3bb0ead8a5da7f501baa632654e7056e9c5846842213d674d8 # via readme-renderer idna==3.10 \ --hash=sha256:12f65c9b470abda6dc35cf8e63cc574b1c52b11df2c86030af0ac09b01b13ea9 \ diff --git a/tools/publish/requirements_universal.txt b/tools/publish/requirements_universal.txt index a3a3f23a51..21b3288461 100644 --- a/tools/publish/requirements_universal.txt +++ b/tools/publish/requirements_universal.txt @@ -199,9 +199,9 @@ cryptography==45.0.7 ; sys_platform == 'linux' \ --hash=sha256:f5414a788ecc6ee6bc58560e85ca624258a55ca434884445440a810796ea0e0b \ --hash=sha256:fa26fa54c0a9384c27fcdc905a2fb7d60ac6e47d14bc2692145f2b3b1e2cfdbd # via secretstorage -docutils==0.22 \ - --hash=sha256:4ed966a0e96a0477d852f7af31bdcb3adc049fbb35ccba358c2ea8a03287615e \ - --hash=sha256:ba9d57750e92331ebe7c08a1bbf7a7f8143b86c476acd51528b042216a6aad0f +docutils==0.22.2 \ + --hash=sha256:9fdb771707c8784c8f2728b67cb2c691305933d68137ef95a75db5f4dfbc213d \ + --hash=sha256:b0e98d679283fc3bb0ead8a5da7f501baa632654e7056e9c5846842213d674d8 # via readme-renderer idna==3.10 \ --hash=sha256:12f65c9b470abda6dc35cf8e63cc574b1c52b11df2c86030af0ac09b01b13ea9 \ diff --git a/tools/publish/requirements_windows.txt b/tools/publish/requirements_windows.txt index b4eb83d1f1..e21cb32fd3 100644 --- a/tools/publish/requirements_windows.txt +++ b/tools/publish/requirements_windows.txt @@ -91,9 +91,9 @@ charset-normalizer==3.4.3 \ --hash=sha256:fd10de089bcdcd1be95a2f73dbe6254798ec1bda9f450d5828c96f93e2536b9c \ --hash=sha256:fdabf8315679312cfa71302f9bd509ded4f2f263fb5b765cf1433b39106c3cc9 # via requests -docutils==0.22 \ - --hash=sha256:4ed966a0e96a0477d852f7af31bdcb3adc049fbb35ccba358c2ea8a03287615e \ - --hash=sha256:ba9d57750e92331ebe7c08a1bbf7a7f8143b86c476acd51528b042216a6aad0f +docutils==0.22.2 \ + --hash=sha256:9fdb771707c8784c8f2728b67cb2c691305933d68137ef95a75db5f4dfbc213d \ + --hash=sha256:b0e98d679283fc3bb0ead8a5da7f501baa632654e7056e9c5846842213d674d8 # via readme-renderer idna==3.10 \ --hash=sha256:12f65c9b470abda6dc35cf8e63cc574b1c52b11df2c86030af0ac09b01b13ea9 \ From fb06c86322fc46a230ee2758dbae75f1c5eba97e Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Tue, 30 Sep 2025 22:50:03 -0700 Subject: [PATCH 465/922] feat(toolchains): let local toolchains point to a label (#3304) Currently, the local toolchain code requires using a path (or program name) to find the Python interpreter. This comes up short when using Bazel to download an arbitrary runtime (or otherwise manage the creation of it, e.g. downloading Python and building it from source in a repo rule). In such cases, the file system location of the interpreter isn't known (it'll be in some Bazel cache directory). To fix, add the `interpreter_target` attribute to `local_runtime_repo`, which it looks up the path for, then continues on as normal. As an example, the test uses a custom repository rule to download a particular version of Python appropriate to the OS. --- CHANGELOG.md | 1 + docs/toolchains.md | 4 + python/private/local_runtime_repo.bzl | 88 ++++++++++++++++--- .../integration/local_toolchains/BUILD.bazel | 22 ++++- .../integration/local_toolchains/MODULE.bazel | 46 +++++++++- tests/integration/local_toolchains/WORKSPACE | 49 ++++++++++- .../{test.py => local_runtime_test.py} | 0 .../local_toolchains/pbs_archive.bzl | 53 +++++++++++ .../local_toolchains/repo_runtime_test.py | 19 ++++ 9 files changed, 264 insertions(+), 18 deletions(-) rename tests/integration/local_toolchains/{test.py => local_runtime_test.py} (100%) create mode 100644 tests/integration/local_toolchains/pbs_archive.bzl create mode 100644 tests/integration/local_toolchains/repo_runtime_test.py diff --git a/CHANGELOG.md b/CHANGELOG.md index ff2257a06b..ed859d3fea 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -109,6 +109,7 @@ END_UNRELEASED_TEMPLATE {obj}`py_cc_toolchain.headers_abi3`, and {obj}`PyCcToolchainInfo.headers_abi3`. * {obj}`//python:features.bzl%features.headers_abi3` can be used to feature-detect the presense of the above. +* (toolchains) Local toolchains can use a label for the interpreter to use. {#v1-6-3} ## [1.6.3] - 2025-09-21 diff --git a/docs/toolchains.md b/docs/toolchains.md index 52e619a120..186ad11e73 100644 --- a/docs/toolchains.md +++ b/docs/toolchains.md @@ -460,6 +460,10 @@ local_runtime_toolchains_repo( register_toolchains("@local_toolchains//:all", dev_dependency = True) ``` +In the example above, `interpreter_path` is used to find Python via `PATH` +lookups. Alternatively, {obj}`interpreter_target` can be set, which can +refer to a Python in an arbitrary Bazel repository. + :::{important} Be sure to set `dev_dependency = True`. Using a local toolchain only makes sense for the root module. diff --git a/python/private/local_runtime_repo.bzl b/python/private/local_runtime_repo.bzl index 27c90b1bc9..583926b15f 100644 --- a/python/private/local_runtime_repo.bzl +++ b/python/private/local_runtime_repo.bzl @@ -200,13 +200,37 @@ a system having the necessary Python installed. doc = """ An absolute path or program name on the `PATH` env var. +*Mutually exclusive with `interpreter_target`.* + Values with slashes are assumed to be the path to a program. Otherwise, it is treated as something to search for on `PATH` Note that, when a plain program name is used, the path to the interpreter is resolved at repository evalution time, not runtime of any resulting binaries. + +If not set, defaults to `python3`. + +:::{seealso} +The {obj}`interpreter_target` attribute for getting the interpreter from +a label +::: +""", + default = "", + ), + "interpreter_target": attr.label( + doc = """ +A label to a Python interpreter executable. + +*Mutually exclusive with `interpreter_path`.* + +On Windows, if the path doesn't exist, various suffixes will be tried to +find a usable path. + +:::{seealso} +The {obj}`interpreter_path` attribute for getting the interpreter from +a path or PATH environment lookup. +::: """, - default = "python3", ), "on_failure": attr.string( default = _OnFailure.SKIP, @@ -247,6 +271,37 @@ def _expand_incompatible_template(): os = "@platforms//:incompatible", ) +def _find_python_exe_from_target(rctx): + base_path = rctx.path(rctx.attr.interpreter_target) + if base_path.exists: + return base_path, None + attempted_paths = [base_path] + + # Try to convert a unix-y path to a Windows path. On Linux/Mac, + # the path is usually `bin/python3`. On Windows, it's simply + # `python.exe`. + basename = base_path.basename.rstrip("3") + path = base_path.dirname.dirname.get_child(basename) + path = rctx.path("{}.exe".format(path)) + if path.exists: + return path, None + attempted_paths.append(path) + + # Try adding .exe to the base path + path = rctx.path("{}.exe".format(base_path)) + if path.exists: + return path, None + attempted_paths.append(path) + + describe_failure = lambda: ( + "Target '{target}' could not be resolved to a valid path. " + + "Attempted paths: {paths}" + ).format( + target = rctx.attr.interpreter_target, + paths = "\n".join([str(p) for p in attempted_paths]), + ) + return None, describe_failure + def _resolve_interpreter_path(rctx): """Find the absolute path for an interpreter. @@ -260,20 +315,27 @@ def _resolve_interpreter_path(rctx): returns a description of why it couldn't be resolved A path object or None. The path may not exist. """ - if "/" not in rctx.attr.interpreter_path and "\\" not in rctx.attr.interpreter_path: - # Provide a bit nicer integration with pyenv: recalculate the runtime if the - # user changes the python version using e.g. `pyenv shell` - repo_utils.getenv(rctx, "PYENV_VERSION") - result = repo_utils.which_unchecked(rctx, rctx.attr.interpreter_path) - resolved_path = result.binary - describe_failure = result.describe_failure + if rctx.attr.interpreter_path and rctx.attr.interpreter_target: + fail("interpreter_path and interpreter_target are mutually exclusive") + + if rctx.attr.interpreter_target: + resolved_path, describe_failure = _find_python_exe_from_target(rctx) else: - rctx.watch(rctx.attr.interpreter_path) - resolved_path = rctx.path(rctx.attr.interpreter_path) - if not resolved_path.exists: - describe_failure = lambda: "Path not found: {}".format(repr(rctx.attr.interpreter_path)) + interpreter_path = rctx.attr.interpreter_path or "python3" + if "/" not in interpreter_path and "\\" not in interpreter_path: + # Provide a bit nicer integration with pyenv: recalculate the runtime if the + # user changes the python version using e.g. `pyenv shell` + repo_utils.getenv(rctx, "PYENV_VERSION") + result = repo_utils.which_unchecked(rctx, interpreter_path) + resolved_path = result.binary + describe_failure = result.describe_failure else: - describe_failure = None + rctx.watch(interpreter_path) + resolved_path = rctx.path(interpreter_path) + if not resolved_path.exists: + describe_failure = lambda: "Path not found: {}".format(repr(interpreter_path)) + else: + describe_failure = None return struct( resolved_path = resolved_path, diff --git a/tests/integration/local_toolchains/BUILD.bazel b/tests/integration/local_toolchains/BUILD.bazel index a0cb2b164d..bf47316027 100644 --- a/tests/integration/local_toolchains/BUILD.bazel +++ b/tests/integration/local_toolchains/BUILD.bazel @@ -18,12 +18,23 @@ load("@rules_python//python:py_test.bzl", "py_test") load(":py_extension.bzl", "py_extension") py_test( - name = "test", - srcs = ["test.py"], + name = "local_runtime_test", + srcs = ["local_runtime_test.py"], + config_settings = { + "//:py": "local", + }, # Make this test better respect pyenv env_inherit = ["PYENV_VERSION"], ) +py_test( + name = "repo_runtime_test", + srcs = ["repo_runtime_test.py"], + config_settings = { + "//:py": "repo", + }, +) + config_setting( name = "is_py_local", flag_values = { @@ -31,6 +42,13 @@ config_setting( }, ) +config_setting( + name = "is_py_repo", + flag_values = { + ":py": "repo", + }, +) + # Set `--//:py=local` to use the local toolchain # (This is set in this example's .bazelrc) string_flag( diff --git a/tests/integration/local_toolchains/MODULE.bazel b/tests/integration/local_toolchains/MODULE.bazel index e81c012c2d..6c821c5bb0 100644 --- a/tests/integration/local_toolchains/MODULE.bazel +++ b/tests/integration/local_toolchains/MODULE.bazel @@ -23,32 +23,76 @@ local_path_override( path = "../../..", ) +# Step 1: Define the python runtime local_runtime_repo = use_repo_rule("@rules_python//python/local_toolchains:repos.bzl", "local_runtime_repo") local_runtime_toolchains_repo = use_repo_rule("@rules_python//python/local_toolchains:repos.bzl", "local_runtime_toolchains_repo") +# This will use `python3` from the environment local_runtime_repo( name = "local_python3", interpreter_path = "python3", on_failure = "fail", ) +pbs_archive = use_repo_rule("//:pbs_archive.bzl", "pbs_archive") + +pbs_archive( + name = "pbs_runtime", + sha256 = { + "linux": "0a01bad99fd4a165a11335c29eb43015dfdb8bd5ba8e305538ebb54f3bf3146d", + "mac os x": "4fb42ffc8aad2a42ca7646715b8926bc6b2e0d31f13d2fec25943dc236a6fd60", + "windows": "005cb2abf4cfa4aaa48fb10ce4e33fe4335ea4d1f55202dbe4e20c852e45e0f9", + }, + urls = { + "linux": "https://github.com/astral-sh/python-build-standalone/releases/download/20250918/cpython-3.13.7+20250918-x86_64-unknown-linux-gnu-install_only.tar.gz", + "mac os x": "https://github.com/astral-sh/python-build-standalone/releases/download/20250918/cpython-3.13.7+20250918-x86_64-apple-darwin-install_only.tar.gz", + "windows server 2022": "https://github.com/astral-sh/python-build-standalone/releases/download/20250918/cpython-3.13.7+20250918-x86_64-pc-windows-msvc-install_only.tar.gz", + }, +) + +# This will use Python from the `pbs_runtime` repository. +# The pbs_runtime is just an example; the repo just needs to be a valid Python +# installation. +local_runtime_repo( + name = "repo_python3", + interpreter_target = "@pbs_runtime//:python/bin/python", + on_failure = "fail", +) + +# Step 2: Create toolchains for the runtimes +# Below, we configure them to only activate if the `//:py` flag has particular +# values. local_runtime_toolchains_repo( name = "local_toolchains", - runtimes = ["local_python3"], + runtimes = [ + "local_python3", + "repo_python3", + ], target_compatible_with = { "local_python3": [ "HOST_CONSTRAINTS", ], + "repo_python3": [ + "HOST_CONSTRAINTS", + ], }, target_settings = { "local_python3": [ "@//:is_py_local", ], + "repo_python3": [ + "@//:is_py_repo", + ], }, ) +config = use_extension("@rules_python//python/extensions:config.bzl", "config") +config.add_transition_setting(setting = "//:py") + python = use_extension("@rules_python//python/extensions:python.bzl", "python") +python.toolchain(python_version = "3.13") use_repo(python, "rules_python_bzlmod_debug") +# Step 3: Register the toolchains register_toolchains("@local_toolchains//:all") diff --git a/tests/integration/local_toolchains/WORKSPACE b/tests/integration/local_toolchains/WORKSPACE index 480cd2794a..159f16deab 100644 --- a/tests/integration/local_toolchains/WORKSPACE +++ b/tests/integration/local_toolchains/WORKSPACE @@ -9,7 +9,11 @@ local_repository( load("@rules_python//python:repositories.bzl", "py_repositories") -py_repositories() +py_repositories( + transition_settings = [ + "@//:py", + ], +) load("@rules_python//python/local_toolchains:repos.bzl", "local_runtime_repo", "local_runtime_toolchains_repo") @@ -21,10 +25,51 @@ local_runtime_repo( # or interpreter_path = "C:\\path\\to\\python.exe" ) +load("//:pbs_archive.bzl", "pbs_archive") + +pbs_archive( + name = "pbs_runtime", + sha256 = { + "linux": "0a01bad99fd4a165a11335c29eb43015dfdb8bd5ba8e305538ebb54f3bf3146d", + "mac os x": "4fb42ffc8aad2a42ca7646715b8926bc6b2e0d31f13d2fec25943dc236a6fd60", + "windows": "005cb2abf4cfa4aaa48fb10ce4e33fe4335ea4d1f55202dbe4e20c852e45e0f9", + }, + urls = { + "linux": "https://github.com/astral-sh/python-build-standalone/releases/download/20250918/cpython-3.13.7+20250918-x86_64-unknown-linux-gnu-install_only.tar.gz", + "mac os x": "https://github.com/astral-sh/python-build-standalone/releases/download/20250918/cpython-3.13.7+20250918-x86_64-apple-darwin-install_only.tar.gz", + "windows server 2022": "https://github.com/astral-sh/python-build-standalone/releases/download/20250918/cpython-3.13.7+20250918-x86_64-pc-windows-msvc-install_only.tar.gz", + }, +) + +local_runtime_repo( + name = "repo_python3", + interpreter_target = "@pbs_runtime//:python/bin/python3", + on_failure = "fail", +) + # Step 2: Create toolchains for the runtimes local_runtime_toolchains_repo( name = "local_toolchains", - runtimes = ["local_python3"], + runtimes = [ + "local_python3", + "repo_python3", + ], + target_compatible_with = { + "local_python3": [ + "HOST_CONSTRAINTS", + ], + "repo_python3": [ + "HOST_CONSTRAINTS", + ], + }, + target_settings = { + "local_python3": [ + "@//:is_py_local", + ], + "repo_python3": [ + "@//:is_py_repo", + ], + }, ) # Step 3: Register the toolchains diff --git a/tests/integration/local_toolchains/test.py b/tests/integration/local_toolchains/local_runtime_test.py similarity index 100% rename from tests/integration/local_toolchains/test.py rename to tests/integration/local_toolchains/local_runtime_test.py diff --git a/tests/integration/local_toolchains/pbs_archive.bzl b/tests/integration/local_toolchains/pbs_archive.bzl new file mode 100644 index 0000000000..8bd0c1eb10 --- /dev/null +++ b/tests/integration/local_toolchains/pbs_archive.bzl @@ -0,0 +1,53 @@ +"""A repository rule to download and extract a Python runtime archive.""" + +BUILD_BAZEL = """ +# Generated by pbs_archive.bzl + +package( + default_visibility = ["//visibility:public"], +) + +exports_files(glob(["**"])) +""" + +def _pbs_archive_impl(repository_ctx): + """Implementation of the python_build_standalone_archive rule.""" + os_name = repository_ctx.os.name.lower() + urls = repository_ctx.attr.urls + sha256s = repository_ctx.attr.sha256 + + if os_name not in urls: + fail("Unsupported OS: '{}'. Available OSs are: {}".format( + os_name, + ", ".join(urls.keys()), + )) + + url = urls[os_name] + sha256 = sha256s.get(os_name, "") + + repository_ctx.download_and_extract( + url = url, + sha256 = sha256, + ) + + repository_ctx.file("BUILD.bazel", BUILD_BAZEL) + +pbs_archive = repository_rule( + implementation = _pbs_archive_impl, + attrs = { + "sha256": attr.string_dict( + doc = "A dictionary of SHA256 checksums for the archives, keyed by OS name.", + mandatory = True, + ), + "urls": attr.string_dict( + doc = "A dictionary of URLs to the runtime archives, keyed by OS name (e.g., 'linux', 'windows').", + mandatory = True, + ), + }, + doc = """ +Downloads and extracts a Python runtime archive for the current OS. + +This rule selects a URL from the `urls` attribute based on the host OS, +downloads the archive, and extracts it. +""", +) diff --git a/tests/integration/local_toolchains/repo_runtime_test.py b/tests/integration/local_toolchains/repo_runtime_test.py new file mode 100644 index 0000000000..4614407c4e --- /dev/null +++ b/tests/integration/local_toolchains/repo_runtime_test.py @@ -0,0 +1,19 @@ +import os.path +import shutil +import subprocess +import sys +import tempfile +import unittest + + +class RepoToolchainTest(unittest.TestCase): + maxDiff = None + + def test_python_from_repo_used(self): + actual = os.path.realpath(sys._base_executable.lower()) + # Normalize case: Windows may have case differences + self.assertIn("pbs_runtime", actual.lower()) + + +if __name__ == "__main__": + unittest.main() From f5ab3bcc7e89df2ac439027a5c0116d8dc03a49b Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Tue, 30 Sep 2025 22:51:12 -0700 Subject: [PATCH 466/922] fix(toolchains): add musl list of freethreaded runtimes (workspace) (#3310) The musl entry got lost as part of a refactoring. Add it back to the platforms that generate freethreaded variants. Fixes https://github.com/bazel-contrib/rules_python/issues/3286 --- python/versions.bzl | 1 + 1 file changed, 1 insertion(+) diff --git a/python/versions.bzl b/python/versions.bzl index 34cffc5664..1b20290b17 100644 --- a/python/versions.bzl +++ b/python/versions.bzl @@ -1065,6 +1065,7 @@ def get_release_info(platform, python_version, base_url = DEFAULT_RELEASE_BASE_U "x86_64-apple-darwin": "pgo+lto", "x86_64-pc-windows-msvc": "pgo", "x86_64-unknown-linux-gnu": "pgo+lto", + "x86_64-unknown-linux-musl": "pgo+lto", }[p], ) else: From 394dda20df8e70367d9f3cc0278cfe13d93f6eea Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Wed, 1 Oct 2025 21:21:49 -0700 Subject: [PATCH 467/922] tests: add tests to verify toolchain registration (#3313) This adds tests that verify toolchains are registered and resolving correctly for the different variants and platforms for the runtimes. This also shows that workspace mode isn't registering musl or freethreaded builds correctly, so use them isn't as easy as simply setting the build flags. For now, the tests skip those in workspace mode. Along the way, fix a bug where py_runtime would crash if it got the python version from the flag, and the version contained more than micro (e.g. "3.14.0rc0") --- python/private/BUILD.bazel | 1 + python/private/py_runtime_rule.bzl | 5 +- python/private/version.bzl | 5 +- tests/base_rules/py_executable_base_tests.bzl | 19 +- tests/base_rules/py_test/py_test_tests.bzl | 11 +- .../transition/multi_version_tests.bzl | 3 +- .../exec_toolchain_matching_tests.bzl | 6 +- .../config_settings/config_settings_tests.bzl | 2 +- tests/support/BUILD.bazel | 69 ------- tests/support/platforms/BUILD.bazel | 77 ++++++++ tests/support/platforms/platforms.bzl | 13 ++ tests/support/support.bzl | 7 - .../multi_platform_resolution/BUILD.bazel | 3 + .../resolution_tests.bzl | 186 ++++++++++++++++++ tests/uv/uv/uv_tests.bzl | 3 +- 15 files changed, 311 insertions(+), 99 deletions(-) create mode 100644 tests/support/platforms/BUILD.bazel create mode 100644 tests/support/platforms/platforms.bzl create mode 100644 tests/toolchains/multi_platform_resolution/BUILD.bazel create mode 100644 tests/toolchains/multi_platform_resolution/resolution_tests.bzl diff --git a/python/private/BUILD.bazel b/python/private/BUILD.bazel index c77417892b..dd66c9ddfd 100644 --- a/python/private/BUILD.bazel +++ b/python/private/BUILD.bazel @@ -531,6 +531,7 @@ bzl_library( ":py_runtime_info_bzl", ":reexports_bzl", ":rule_builders_bzl", + ":version_bzl", "@bazel_skylib//lib:dicts", "@bazel_skylib//lib:paths", "@bazel_skylib//rules:common_settings", diff --git a/python/private/py_runtime_rule.bzl b/python/private/py_runtime_rule.bzl index 5020d7ad9b..f8182e73da 100644 --- a/python/private/py_runtime_rule.bzl +++ b/python/private/py_runtime_rule.bzl @@ -21,6 +21,7 @@ load(":flags.bzl", "FreeThreadedFlag") load(":py_internal.bzl", "py_internal") load(":py_runtime_info.bzl", "DEFAULT_STUB_SHEBANG", "PyRuntimeInfo") load(":reexports.bzl", "BuiltinPyRuntimeInfo") +load(":version.bzl", "version") _py_builtins = py_internal @@ -387,11 +388,11 @@ def _is_singleton_depset(files): return _py_builtins.is_singleton_depset(files) def _interpreter_version_info_from_version_str(version_str): - parts = version_str.split(".") + v = version.parse(version_str) version_info = {} + parts = list(v.release) for key in ("major", "minor", "micro"): if not parts: break version_info[key] = parts.pop(0) - return version_info diff --git a/python/private/version.bzl b/python/private/version.bzl index 8b5fef7b2a..c41524a9e3 100644 --- a/python/private/version.bzl +++ b/python/private/version.bzl @@ -622,7 +622,8 @@ def parse(version_str, strict = False, _fail = fail): # https://peps.python.org/pep-0440/#public-version-identifiers return None - return struct( + # buildifier: disable=uninitialized + self = struct( epoch = _parse_epoch(parts["epoch"], _fail), release = _parse_release(parts["release"]), pre = _parse_pre(parts["pre"]), @@ -631,7 +632,9 @@ def parse(version_str, strict = False, _fail = fail): local = _parse_local(parts["local"], _fail), string = parts["norm"], is_prefix = parts["is_prefix"], + key = lambda *a, **k: _version_key(self, *a, **k), ) + return self def _parse_epoch(value, fail): if not value: diff --git a/tests/base_rules/py_executable_base_tests.bzl b/tests/base_rules/py_executable_base_tests.bzl index c7723be54a..e86a94990a 100644 --- a/tests/base_rules/py_executable_base_tests.bzl +++ b/tests/base_rules/py_executable_base_tests.bzl @@ -23,7 +23,8 @@ load("//python/private:reexports.bzl", "BuiltinPyRuntimeInfo") # buildifier: di load("//tests/base_rules:base_tests.bzl", "create_base_tests") load("//tests/base_rules:util.bzl", "WINDOWS_ATTR", pt_util = "util") load("//tests/support:py_executable_info_subject.bzl", "PyExecutableInfoSubject") -load("//tests/support:support.bzl", "CC_TOOLCHAIN", "CROSSTOOL_TOP", "LINUX_X86_64", "WINDOWS_X86_64") +load("//tests/support:support.bzl", "CC_TOOLCHAIN", "CROSSTOOL_TOP") +load("//tests/support/platforms:platforms.bzl", "platform_targets") _tests = [] @@ -46,9 +47,9 @@ def _test_basic_windows(name, config): "//command_line_option:build_python_zip": "true", "//command_line_option:cpu": "windows_x86_64", "//command_line_option:crosstool_top": CROSSTOOL_TOP, - "//command_line_option:extra_execution_platforms": [WINDOWS_X86_64], + "//command_line_option:extra_execution_platforms": [platform_targets.WINDOWS_X86_64], "//command_line_option:extra_toolchains": [CC_TOOLCHAIN], - "//command_line_option:platforms": [WINDOWS_X86_64], + "//command_line_option:platforms": [platform_targets.WINDOWS_X86_64], }, attr_values = {}, ) @@ -89,9 +90,9 @@ def _test_basic_zip(name, config): "//command_line_option:build_python_zip": "true", "//command_line_option:cpu": "linux_x86_64", "//command_line_option:crosstool_top": CROSSTOOL_TOP, - "//command_line_option:extra_execution_platforms": [LINUX_X86_64], + "//command_line_option:extra_execution_platforms": [platform_targets.LINUX_X86_64], "//command_line_option:extra_toolchains": [CC_TOOLCHAIN], - "//command_line_option:platforms": [LINUX_X86_64], + "//command_line_option:platforms": [platform_targets.LINUX_X86_64], }, attr_values = {"target_compatible_with": target_compatible_with}, ) @@ -326,8 +327,8 @@ def _test_main_module_bootstrap_system_python(name, config): target = name + "_subject", config_settings = { labels.BOOTSTRAP_IMPL: "system_python", - "//command_line_option:extra_execution_platforms": ["@bazel_tools//tools:host_platform", LINUX_X86_64], - "//command_line_option:platforms": [LINUX_X86_64], + "//command_line_option:extra_execution_platforms": ["@bazel_tools//tools:host_platform", platform_targets.LINUX_X86_64], + "//command_line_option:platforms": [platform_targets.LINUX_X86_64], }, ) @@ -350,8 +351,8 @@ def _test_main_module_bootstrap_script(name, config): target = name + "_subject", config_settings = { labels.BOOTSTRAP_IMPL: "script", - "//command_line_option:extra_execution_platforms": ["@bazel_tools//tools:host_platform", LINUX_X86_64], - "//command_line_option:platforms": [LINUX_X86_64], + "//command_line_option:extra_execution_platforms": ["@bazel_tools//tools:host_platform", platform_targets.LINUX_X86_64], + "//command_line_option:platforms": [platform_targets.LINUX_X86_64], }, ) diff --git a/tests/base_rules/py_test/py_test_tests.bzl b/tests/base_rules/py_test/py_test_tests.bzl index c28eec4346..fd284beffd 100644 --- a/tests/base_rules/py_test/py_test_tests.bzl +++ b/tests/base_rules/py_test/py_test_tests.bzl @@ -21,7 +21,8 @@ load( "create_executable_tests", ) load("//tests/base_rules:util.bzl", pt_util = "util") -load("//tests/support:support.bzl", "CC_TOOLCHAIN", "CROSSTOOL_TOP", "LINUX_X86_64", "MAC_X86_64") +load("//tests/support:support.bzl", "CC_TOOLCHAIN", "CROSSTOOL_TOP") +load("//tests/support/platforms:platforms.bzl", "platform_targets") # The Windows CI currently runs as root, which breaks when # the analysis tests try to install (but not use, because @@ -51,9 +52,9 @@ def _test_mac_requires_darwin_for_execution(name, config): config_settings = { "//command_line_option:cpu": "darwin_x86_64", "//command_line_option:crosstool_top": CROSSTOOL_TOP, - "//command_line_option:extra_execution_platforms": [MAC_X86_64], + "//command_line_option:extra_execution_platforms": [platform_targets.MAC_X86_64], "//command_line_option:extra_toolchains": [CC_TOOLCHAIN], - "//command_line_option:platforms": [MAC_X86_64], + "//command_line_option:platforms": [platform_targets.MAC_X86_64], }, attr_values = _SKIP_WINDOWS, ) @@ -78,9 +79,9 @@ def _test_non_mac_doesnt_require_darwin_for_execution(name, config): config_settings = { "//command_line_option:cpu": "k8", "//command_line_option:crosstool_top": CROSSTOOL_TOP, - "//command_line_option:extra_execution_platforms": [LINUX_X86_64], + "//command_line_option:extra_execution_platforms": [platform_targets.LINUX_X86_64], "//command_line_option:extra_toolchains": [CC_TOOLCHAIN], - "//command_line_option:platforms": [LINUX_X86_64], + "//command_line_option:platforms": [platform_targets.LINUX_X86_64], }, attr_values = _SKIP_WINDOWS, ) diff --git a/tests/config_settings/transition/multi_version_tests.bzl b/tests/config_settings/transition/multi_version_tests.bzl index b2564a3fb3..dfe2bf9981 100644 --- a/tests/config_settings/transition/multi_version_tests.bzl +++ b/tests/config_settings/transition/multi_version_tests.bzl @@ -22,6 +22,7 @@ load("//python:py_info.bzl", "PyInfo") load("//python:py_test.bzl", "py_test") load("//python/private:reexports.bzl", "BuiltinPyInfo") # buildifier: disable=bzl-visibility load("//tests/support:support.bzl", "CC_TOOLCHAIN") +load("//tests/support/platforms:platforms.bzl", "platform_targets") # NOTE @aignas 2024-06-04: we are using here something that is registered in the MODULE.Bazel # and if you find tests failing, it could be because of the toolchain resolution issues here. @@ -92,7 +93,7 @@ def _setup_py_binary_windows(name, *, impl, build_python_zip): config_settings = { "//command_line_option:build_python_zip": build_python_zip, "//command_line_option:extra_toolchains": CC_TOOLCHAIN, - "//command_line_option:platforms": str(Label("//tests/support:windows_x86_64")), + "//command_line_option:platforms": str(platform_targets.WINDOWS_X86_64), }, ) diff --git a/tests/exec_toolchain_matching/exec_toolchain_matching_tests.bzl b/tests/exec_toolchain_matching/exec_toolchain_matching_tests.bzl index b3ff294b6f..a26e4f5f6e 100644 --- a/tests/exec_toolchain_matching/exec_toolchain_matching_tests.bzl +++ b/tests/exec_toolchain_matching/exec_toolchain_matching_tests.bzl @@ -20,7 +20,7 @@ load("//python:py_runtime.bzl", "py_runtime") load("//python:py_runtime_pair.bzl", "py_runtime_pair") load("//python/private:common_labels.bzl", "labels") # buildifier: disable=bzl-visibility load("//python/private:toolchain_types.bzl", "EXEC_TOOLS_TOOLCHAIN_TYPE", "TARGET_TOOLCHAIN_TYPE") # buildifier: disable=bzl-visibility -load("//tests/support:support.bzl", "LINUX", "MAC") +load("//tests/support/platforms:platforms.bzl", "platform_targets") _LookupInfo = provider() # buildifier: disable=provider-params @@ -126,9 +126,9 @@ def _test_exec_matches_target_python_version(name): target = name + "_subject", impl = _test_exec_matches_target_python_version_impl, config_settings = { - "//command_line_option:extra_execution_platforms": [str(MAC)], + "//command_line_option:extra_execution_platforms": [str(platform_targets.MAC)], "//command_line_option:extra_toolchains": ["//tests/exec_toolchain_matching:all"], - "//command_line_option:platforms": [str(LINUX)], + "//command_line_option:platforms": [str(platform_targets.LINUX)], labels.PYTHON_VERSION: "3.12", }, ) diff --git a/tests/pypi/config_settings/config_settings_tests.bzl b/tests/pypi/config_settings/config_settings_tests.bzl index b3e6ada9e8..ed95bd4877 100644 --- a/tests/pypi/config_settings/config_settings_tests.bzl +++ b/tests/pypi/config_settings/config_settings_tests.bzl @@ -31,7 +31,7 @@ _subject = rule( ) _flag = struct( - platform = lambda x: ("//command_line_option:platforms", str(Label("//tests/support:" + x))), + platform = lambda x: ("//command_line_option:platforms", str(Label("//tests/support/platforms:" + x))), pip_whl = lambda x: (str(Label("//python/config_settings:pip_whl")), str(x)), pip_whl_glibc_version = lambda x: (str(Label("//python/config_settings:pip_whl_glibc_version")), str(x)), pip_whl_muslc_version = lambda x: (str(Label("//python/config_settings:pip_whl_muslc_version")), str(x)), diff --git a/tests/support/BUILD.bazel b/tests/support/BUILD.bazel index 303dbafbdf..45f43c89e2 100644 --- a/tests/support/BUILD.bazel +++ b/tests/support/BUILD.bazel @@ -12,12 +12,6 @@ # See the License for the specific language governing permissions and # limitations under the License. -# ==================== -# NOTE: You probably want to use the constants in test_platforms.bzl -# Otherwise, you'll probably have to manually call Label() on these targets -# to force them to resolve in the proper context. -# ==================== - load("@bazel_skylib//rules:common_settings.bzl", "string_flag") load(":sh_py_run_test.bzl", "current_build_settings") @@ -25,69 +19,6 @@ package( default_visibility = ["//:__subpackages__"], ) -platform( - name = "mac", - constraint_values = [ - "@platforms//os:macos", - ], -) - -platform( - name = "linux", - constraint_values = [ - "@platforms//os:linux", - ], -) - -platform( - name = "windows", - constraint_values = [ - "@platforms//os:windows", - ], -) - -# Used when testing downloading of toolchains for a different platform - -platform( - name = "linux_x86_64", - constraint_values = [ - "@platforms//cpu:x86_64", - "@platforms//os:linux", - ], -) - -platform( - name = "linux_aarch64", - constraint_values = [ - "@platforms//cpu:aarch64", - "@platforms//os:linux", - ], -) - -platform( - name = "mac_x86_64", - constraint_values = [ - "@platforms//cpu:x86_64", - "@platforms//os:macos", - ], -) - -platform( - name = "windows_x86_64", - constraint_values = [ - "@platforms//cpu:x86_64", - "@platforms//os:windows", - ], -) - -platform( - name = "win_aarch64", - constraint_values = [ - "@platforms//os:windows", - "@platforms//cpu:aarch64", - ], -) - current_build_settings( name = "current_build_settings", ) diff --git a/tests/support/platforms/BUILD.bazel b/tests/support/platforms/BUILD.bazel new file mode 100644 index 0000000000..41d7936394 --- /dev/null +++ b/tests/support/platforms/BUILD.bazel @@ -0,0 +1,77 @@ +package( + default_visibility = ["//:__subpackages__"], +) + +# ==================== +# NOTE: You probably want to use the constants in test_platforms.bzl +# Otherwise, you'll probably have to manually call Label() on these targets +# to force them to resolve in the proper context. +# ==================== +platform( + name = "mac", + constraint_values = [ + "@platforms//os:macos", + ], +) + +platform( + name = "linux", + constraint_values = [ + "@platforms//os:linux", + ], +) + +platform( + name = "windows", + constraint_values = [ + "@platforms//os:windows", + ], +) + +platform( + name = "linux_x86_64", + constraint_values = [ + "@platforms//cpu:x86_64", + "@platforms//os:linux", + ], +) + +platform( + name = "linux_aarch64", + constraint_values = [ + "@platforms//cpu:aarch64", + "@platforms//os:linux", + ], +) + +platform( + name = "mac_x86_64", + constraint_values = [ + "@platforms//cpu:x86_64", + "@platforms//os:macos", + ], +) + +platform( + name = "mac_aarch64", + constraint_values = [ + "@platforms//cpu:aarch64", + "@platforms//os:macos", + ], +) + +platform( + name = "windows_x86_64", + constraint_values = [ + "@platforms//cpu:x86_64", + "@platforms//os:windows", + ], +) + +platform( + name = "windows_aarch64", + constraint_values = [ + "@platforms//os:windows", + "@platforms//cpu:aarch64", + ], +) diff --git a/tests/support/platforms/platforms.bzl b/tests/support/platforms/platforms.bzl new file mode 100644 index 0000000000..af049f202c --- /dev/null +++ b/tests/support/platforms/platforms.bzl @@ -0,0 +1,13 @@ +"""Constants and utilities for platforms used for testing.""" + +platform_targets = struct( + LINUX = Label("//tests/support/platforms:linux"), + LINUX_AARCH64 = Label("//tests/support/platforms:linux_aarch64"), + LINUX_X86_64 = Label("//tests/support/platforms:linux_x86_64"), + MAC = Label("//tests/support/platforms:mac"), + MAC_X86_64 = Label("//tests/support/platforms:mac_x86_64"), + MAC_AARCH64 = Label("//tests/support/platforms:mac_aarch64"), + WINDOWS = Label("//tests/support/platforms:windows"), + WINDOWS_AARCH64 = Label("//tests/support/platforms:windows_aarch64"), + WINDOWS_X86_64 = Label("//tests/support/platforms:windows_x86_64"), +) diff --git a/tests/support/support.bzl b/tests/support/support.bzl index 96c6ad902a..c6997e35d1 100644 --- a/tests/support/support.bzl +++ b/tests/support/support.bzl @@ -21,13 +21,6 @@ load("//python/private:bzlmod_enabled.bzl", "BZLMOD_ENABLED") # buildifier: disable=bzl-visibility -MAC = Label("//tests/support:mac") -MAC_X86_64 = Label("//tests/support:mac_x86_64") -LINUX = Label("//tests/support:linux") -LINUX_X86_64 = Label("//tests/support:linux_x86_64") -WINDOWS = Label("//tests/support:windows") -WINDOWS_X86_64 = Label("//tests/support:windows_x86_64") - PY_TOOLCHAINS = str(Label("//tests/support/py_toolchains:all")) CC_TOOLCHAIN = str(Label("//tests/support/cc_toolchains:all")) CROSSTOOL_TOP = Label("//tests/support/cc_toolchains:cc_toolchain_suite") diff --git a/tests/toolchains/multi_platform_resolution/BUILD.bazel b/tests/toolchains/multi_platform_resolution/BUILD.bazel new file mode 100644 index 0000000000..35f18e98ed --- /dev/null +++ b/tests/toolchains/multi_platform_resolution/BUILD.bazel @@ -0,0 +1,3 @@ +load(":resolution_tests.bzl", "resolution_test_suite") + +resolution_test_suite(name = "resolution_tests") diff --git a/tests/toolchains/multi_platform_resolution/resolution_tests.bzl b/tests/toolchains/multi_platform_resolution/resolution_tests.bzl new file mode 100644 index 0000000000..609fa3fe53 --- /dev/null +++ b/tests/toolchains/multi_platform_resolution/resolution_tests.bzl @@ -0,0 +1,186 @@ +"""Tests to verify toolchain resolution of different config variants. + +NOTE: This test relies on the project toolchain configuration. This is +intentional because it wants to verify that, using the toolchains as +rules_python configures them, the different implementations can be used +by setting the appropriate flags. +""" + +load("@bazel_skylib//lib:structs.bzl", "structs") +load("@rules_testing//lib:analysis_test.bzl", "analysis_test") +load("@rules_testing//lib:test_suite.bzl", "test_suite") +load("//python:versions.bzl", "TOOL_VERSIONS") +load("//python/private:bzlmod_enabled.bzl", "BZLMOD_ENABLED") # buildifier: disable=bzl-visibility +load("//python/private:common_labels.bzl", "labels") # buildifier: disable=bzl-visibility +load("//python/private:toolchain_types.bzl", "TARGET_TOOLCHAIN_TYPE") # buildifier: disable=bzl-visibility +load("//python/private:version.bzl", "version") # buildifier: disable=bzl-visibility +load("//tests/support/platforms:platforms.bzl", "platform_targets") + +_PLATFORM_TARGET_MAP = { + "linux": { + "aarch64": platform_targets.LINUX_AARCH64, + "x86_64": platform_targets.LINUX_X86_64, + }, + "osx": { + "aarch64": platform_targets.MAC_AARCH64, + "x86_64": platform_targets.MAC_X86_64, + }, + "windows": { + "aarch64": platform_targets.WINDOWS_AARCH64, + "x86_64": platform_targets.WINDOWS_X86_64, + }, +} +_PLATFORM_TRIPLES = { + ("linux", "glibc"): "unknown_linux_gnu", + ("linux", "musl"): "unknown_linux_musl", + "osx": "apple_darwin", + "windows": "pc_windows_msvc", +} + +_ResolvedToolchainsInfo = provider( + doc = "Tell what toolchain was found", + fields = { + "target": "ToolchainInfo for //python:toolchain_type", + }, +) + +def _current_toolchain_impl(ctx): + # todo: also return current settings for various config flags + # to help identify state + return [_ResolvedToolchainsInfo( + target = ctx.toolchains[TARGET_TOOLCHAIN_TYPE], + )] + +_current_toolchain = rule( + implementation = _current_toolchain_impl, + toolchains = [ + TARGET_TOOLCHAIN_TYPE, + ], +) + +def _platform(os, arch, libc, *, ft): + if os == "linux": + platform_triple = _PLATFORM_TRIPLES[os, libc] + else: + platform_triple = _PLATFORM_TRIPLES[os] + return struct( + arch = arch, + freethreaded = ft, + libc = libc, + os = os, + platform_triple = platform_triple, + platform_target = _PLATFORM_TARGET_MAP[os][arch], + ) + +# There's many exceptions to the full `os x arch x libc x threading` matrix, +# so just list the specific combinations that are supported. +# We also omit some more esoteric archs to reduce the matrix size (and +# thus how many runtimes get downloaded). +# The important quality is to have at least 2 for every dimension +_PLATFORMS = [ + _platform("linux", "aarch64", "glibc", ft = "no"), + _platform("linux", "aarch64", "glibc", ft = "yes"), + _platform("linux", "x86_64", "glibc", ft = "no"), + _platform("linux", "x86_64", "glibc", ft = "yes"), + _platform("linux", "x86_64", "musl", ft = "no"), + _platform("osx", "aarch64", None, ft = "no"), + _platform("osx", "aarch64", None, ft = "yes"), + _platform("osx", "x86_64", None, ft = "no"), + _platform("osx", "x86_64", None, ft = "yes"), + _platform("windows", "aarch64", None, ft = "no"), + _platform("windows", "aarch64", None, ft = "yes"), + _platform("windows", "x86_64", None, ft = "no"), + _platform("windows", "x86_64", None, ft = "yes"), +] + +def _compute_runtimes(): + runtimes = [] + + # Limit to the two most recent versions. This helps ensure that multiple + # versions are matching correctly. Limit to two because the disk/download + # isn't worth the marginal coverage improvment. + selected_versions = sorted( + TOOL_VERSIONS.keys(), + key = lambda v: version.parse(v).key(), + )[-2:] + + for python_version in selected_versions: + for platform in _PLATFORMS: + runtimes.append(struct( + name = "{python_version}_{arch}_{triple}{threading}".format( + python_version = python_version.replace(".", "_"), + arch = platform.arch, + triple = platform.platform_triple, + threading = "_freethreaded" if platform.freethreaded == "yes" else "", + ), + python_version = python_version, + **structs.to_dict(platform) + )) + + return sorted(runtimes, key = lambda v: v.name) + +def _test_toolchains_impl(env, target): + target_tc = target[_ResolvedToolchainsInfo].target + toolchain_str = str(target_tc.toolchain_label).replace("-", "_") + env.expect.that_str(toolchain_str).contains(env.ctx.attr.expected_toolchain_name) + +def _test_toolchains(name): + _current_toolchain( + name = name + "_current_toolchain", + ) + test_names = [] + for runtime in _compute_runtimes(): + test_name = "test_{}".format(runtime.name) + test_names.append(test_name) + config_settings = { + "//command_line_option:platforms": [runtime.platform_target], + labels.VISIBLE_FOR_TESTING: True, + labels.PY_FREETHREADED: runtime.freethreaded, + labels.PYTHON_VERSION: runtime.python_version, + } + if runtime.libc: + config_settings[labels.PY_LINUX_LIBC] = runtime.libc + + # TODO: Workspace isn't correctly registering musl and freethreaded + # toolchains, so skip them for now. + target_compatible_with = [] + if not BZLMOD_ENABLED and (runtime.libc == "musl" or + runtime.freethreaded == "yes"): + target_compatible_with = ["@platforms//:incompatible"] + + analysis_test( + name = test_name, + target = name + "_current_toolchain", + impl = _test_toolchains_impl, + config_settings = config_settings, + attrs = {"expected_toolchain_name": attr.string()}, + attr_values = { + "expected_toolchain_name": runtime.name, + # A lot of tests are generated, so set tags to make selecting + # subsets easier + "tags": [ + "python-version={}".format(runtime.python_version), + "libc={}".format(runtime.libc), + "freethreaded={}".format(runtime.freethreaded), + "os={}".format(runtime.os), + "arch={}".format(runtime.arch), + ], + "target_compatible_with": target_compatible_with, + }, + ) + + # We have to return a target for `name`. + native.test_suite( + name = name, + tests = test_names, + ) + +_tests = [ + _test_toolchains, +] + +def resolution_test_suite(name): + test_suite( + name = name, + tests = _tests, + ) diff --git a/tests/uv/uv/uv_tests.bzl b/tests/uv/uv/uv_tests.bzl index 8009405cec..d82e5cb385 100644 --- a/tests/uv/uv/uv_tests.bzl +++ b/tests/uv/uv/uv_tests.bzl @@ -21,6 +21,7 @@ load("//python/private:common_labels.bzl", "labels") # buildifier: disable=bzl- load("//python/uv:uv_toolchain_info.bzl", "UvToolchainInfo") load("//python/uv/private:uv.bzl", "process_modules") # buildifier: disable=bzl-visibility load("//python/uv/private:uv_toolchain.bzl", "uv_toolchain") # buildifier: disable=bzl-visibility +load("//tests/support/platforms:platforms.bzl", "platform_targets") _tests = [] @@ -575,7 +576,7 @@ def _test_toolchain_precedence(name): "//command_line_option:extra_toolchains": [ str(Label("//tests/uv/uv_toolchains:all")), ], - "//command_line_option:platforms": str(Label("//tests/support:linux_aarch64")), + "//command_line_option:platforms": str(platform_targets.LINUX_AARCH64), }, ) From e9b4dfcc7f03e633e6c6e6808c358138356d02d3 Mon Sep 17 00:00:00 2001 From: Oleh Prypin Date: Thu, 2 Oct 2025 17:16:03 +0200 Subject: [PATCH 468/922] refactor: remove Google-specific stubs (#3316) These were needed only to avoid patches inside Google. --- python/BUILD.bazel | 3 --- python/private/BUILD.bazel | 5 ----- python/private/attributes.bzl | 6 ------ python/private/py_executable.bzl | 4 ---- python/private/register_extension_info.bzl | 18 ------------------ python/py_binary.bzl | 6 ------ python/py_library.bzl | 6 ------ python/py_test.bzl | 6 ------ 8 files changed, 54 deletions(-) delete mode 100644 python/private/register_extension_info.bzl diff --git a/python/BUILD.bazel b/python/BUILD.bazel index 5fc35f8357..5c4626ee55 100644 --- a/python/BUILD.bazel +++ b/python/BUILD.bazel @@ -126,7 +126,6 @@ bzl_library( srcs = ["py_binary.bzl"], deps = [ "//python/private:py_binary_macro_bzl", - "//python/private:register_extension_info_bzl", ], ) @@ -175,7 +174,6 @@ bzl_library( srcs = ["py_library.bzl"], deps = [ "//python/private:py_library_macro_bzl", - "//python/private:register_extension_info_bzl", ], ) @@ -208,7 +206,6 @@ bzl_library( srcs = ["py_test.bzl"], deps = [ "//python/private:py_test_macro_bzl", - "//python/private:register_extension_info_bzl", ], ) diff --git a/python/private/BUILD.bazel b/python/private/BUILD.bazel index dd66c9ddfd..7b8c82de8f 100644 --- a/python/private/BUILD.bazel +++ b/python/private/BUILD.bazel @@ -612,11 +612,6 @@ bzl_library( ], ) -bzl_library( - name = "register_extension_info_bzl", - srcs = ["register_extension_info.bzl"], -) - bzl_library( name = "repo_utils_bzl", srcs = ["repo_utils.bzl"], diff --git a/python/private/attributes.bzl b/python/private/attributes.bzl index 6d08c3d926..8fef1bbe2c 100644 --- a/python/private/attributes.bzl +++ b/python/private/attributes.bzl @@ -44,12 +44,6 @@ REQUIRED_EXEC_GROUP_BUILDERS = { "py_precompile": lambda: ruleb.ExecGroup(), } -# Backwards compatibility symbol for Google. -REQUIRED_EXEC_GROUPS = { - k: v().build() - for k, v in REQUIRED_EXEC_GROUP_BUILDERS.items() -} - _STAMP_VALUES = [-1, 0, 1] def _precompile_attr_get_effective_value(ctx): diff --git a/python/private/py_executable.bzl b/python/private/py_executable.bzl index dd0a1a1d6e..0df04a96d9 100644 --- a/python/private/py_executable.bzl +++ b/python/private/py_executable.bzl @@ -1859,7 +1859,3 @@ def cc_configure_features( feature_configuration = feature_configuration, requested_features = requested_features, ) - -only_exposed_for_google_internal_reason = struct( - create_runfiles_with_build_data = _create_runfiles_with_build_data, -) diff --git a/python/private/register_extension_info.bzl b/python/private/register_extension_info.bzl deleted file mode 100644 index 408df6261e..0000000000 --- a/python/private/register_extension_info.bzl +++ /dev/null @@ -1,18 +0,0 @@ -# Copyright 2023 The Bazel Authors. All rights reserved. -# -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. -"""Stub implementation to make patching easier.""" - -# buildifier: disable=unused-variable -def register_extension_info(**kwargs): - """A no-op stub to make Google patching easier.""" diff --git a/python/py_binary.bzl b/python/py_binary.bzl index 68541ab842..80d371fd4c 100644 --- a/python/py_binary.bzl +++ b/python/py_binary.bzl @@ -15,7 +15,6 @@ """Public entry point for py_binary.""" load("//python/private:py_binary_macro.bzl", _py_binary = "py_binary") -load("//python/private:register_extension_info.bzl", "register_extension_info") def py_binary(**attrs): """Creates an executable Python program. @@ -38,8 +37,3 @@ def py_binary(**attrs): fail("Python 2 is no longer supported: https://github.com/bazel-contrib/rules_python/issues/886") _py_binary(**attrs) - -register_extension_info( - extension = py_binary, - label_regex_for_dep = "{extension_name}", -) diff --git a/python/py_library.bzl b/python/py_library.bzl index 277593b0c5..c5cae241f6 100644 --- a/python/py_library.bzl +++ b/python/py_library.bzl @@ -15,7 +15,6 @@ """Public entry point for py_library.""" load("//python/private:py_library_macro.bzl", _py_library = "py_library") -load("//python/private:register_extension_info.bzl", "register_extension_info") def py_library(**attrs): """Creates an executable Python program. @@ -35,8 +34,3 @@ def py_library(**attrs): fail("Python 2 is no longer supported: https://github.com/bazel-contrib/rules_python/issues/886") _py_library(**attrs) - -register_extension_info( - extension = py_library, - label_regex_for_dep = "{extension_name}", -) diff --git a/python/py_test.bzl b/python/py_test.bzl index 70f8ff5d09..2f31e0865e 100644 --- a/python/py_test.bzl +++ b/python/py_test.bzl @@ -15,7 +15,6 @@ """Public entry point for py_test.""" load("//python/private:py_test_macro.bzl", _py_test = "py_test") -load("//python/private:register_extension_info.bzl", "register_extension_info") def py_test(**attrs): """Creates an executable Python program. @@ -39,8 +38,3 @@ def py_test(**attrs): # buildifier: disable=native-python _py_test(**attrs) - -register_extension_info( - extension = py_test, - label_regex_for_dep = "{extension_name}", -) From 997d6f7afc565e45e4c5120613d4d9369bb6d6b8 Mon Sep 17 00:00:00 2001 From: Oleh Prypin Date: Thu, 2 Oct 2025 17:17:27 +0200 Subject: [PATCH 469/922] fix: wrong value of `has_py3_only_sources` in the PyInfo constructor (#3315) The has_py3_only_sources field was being set to the has_py2_only_sources variable --- CHANGELOG.md | 2 ++ python/private/py_info.bzl | 2 +- tests/base_rules/py_info/py_info_tests.bzl | 2 +- 3 files changed, 4 insertions(+), 2 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index ed859d3fea..af11d33803 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -72,6 +72,8 @@ END_UNRELEASED_TEMPLATE {#v0-0-0-fixed} ### Fixed +* (rules) The `PyInfo` constructor was setting the wrong value for + `has_py3_only_sources` - this is now fixed. * (bootstrap) The stage1 bootstrap script now correctly handles nested `RUNFILES_DIR` environments, fixing issues where a `py_binary` calls another `py_binary` ([#3187](https://github.com/bazel-contrib/rules_python/issues/3187)). diff --git a/python/private/py_info.bzl b/python/private/py_info.bzl index 4059b30c63..9318347819 100644 --- a/python/private/py_info.bzl +++ b/python/private/py_info.bzl @@ -146,7 +146,7 @@ def _PyInfo_init( "direct_pyc_files": direct_pyc_files, "direct_pyi_files": direct_pyi_files, "has_py2_only_sources": has_py2_only_sources, - "has_py3_only_sources": has_py2_only_sources, + "has_py3_only_sources": has_py3_only_sources, "imports": imports, "transitive_implicit_pyc_files": transitive_implicit_pyc_files, "transitive_implicit_pyc_source_files": transitive_implicit_pyc_source_files, diff --git a/tests/base_rules/py_info/py_info_tests.bzl b/tests/base_rules/py_info/py_info_tests.bzl index 623594807a..273959b957 100644 --- a/tests/base_rules/py_info/py_info_tests.bzl +++ b/tests/base_rules/py_info/py_info_tests.bzl @@ -35,7 +35,7 @@ def _provide_py_info_impl(ctx): if ctx.attr.has_py2_only_sources != -1: kwargs["has_py2_only_sources"] = bool(ctx.attr.has_py2_only_sources) if ctx.attr.has_py3_only_sources != -1: - kwargs["has_py2_only_sources"] = bool(ctx.attr.has_py2_only_sources) + kwargs["has_py3_only_sources"] = bool(ctx.attr.has_py3_only_sources) providers = [] providers.append(PyInfo(**kwargs)) From 852e95419c603f4e34bc5f0e813daff48d1f9f54 Mon Sep 17 00:00:00 2001 From: Ignas Anikevicius <240938+aignas@users.noreply.github.com> Date: Sat, 4 Oct 2025 00:12:45 +0900 Subject: [PATCH 470/922] fix(pip): do not use experimental_index_url for publish_deps (#3311) Currently, experimental_index_url will trigger network access (by the bzlmod extension logic) as it figures out the wheels. This is problematic because this network access always occurs, even if a user isn't using the publish tool. Additionally, failed network accesses can get Bazel into stuck state requiring `bazel shutdown` to fix it. To fix, don't use experimental_index_url for the publish tool deps. Once bazelbuild/bazel#24777 is merged, we can roll it back, but then we may need to check-in the `MODULE.bazel.lock` file into git. This should be no-op for most users because the publisher is usually run from with `host == exec`, but there is a small chance that people will hit #2241. Work towards #2937 --- CHANGELOG.md | 2 ++ MODULE.bazel | 6 ------ 2 files changed, 2 insertions(+), 6 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index af11d33803..3f85127a7b 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -69,6 +69,8 @@ END_UNRELEASED_TEMPLATE * (bootstrap) For {obj}`--bootstrap_impl=system_python`, the sys.path order has changed from `[app paths, stdlib, runtime site-packages]` to `[stdlib, app paths, runtime site-packages]`. +* (pip) Publishing deps are no longer pulled via `experimental_index_url`. + ([#2937](https://github.com/bazel-contrib/rules_python/issues/2937)). {#v0-0-0-fixed} ### Fixed diff --git a/MODULE.bazel b/MODULE.bazel index a29a898772..481e13e39f 100644 --- a/MODULE.bazel +++ b/MODULE.bazel @@ -172,12 +172,6 @@ pip = use_extension("//python/extensions:pip.bzl", "pip") ] pip.parse( - # NOTE @aignas 2024-10-26: We have an integration test that depends on us - # being able to build sdists for this hub, so explicitly set this to False. - # - # how do we test sdists? Maybe just worth adding a single sdist somewhere? - download_only = False, - experimental_index_url = "https://pypi.org/simple", hub_name = "rules_python_publish_deps", python_version = "3.11", requirements_by_platform = { From 309e93e32a12fc1105c8935acea6d75f810c34bc Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Fri, 3 Oct 2025 08:17:09 -0700 Subject: [PATCH 471/922] fix(toolchains): correctly register musl/freethreaded toolchains for workspace (#3314) The musl/freethreaded runtimes weren't being activated when the flags were set. This was because the toolchains weren't having `target_settings` set, which means extra settings, such as musl/freethreaded-ness were ignored when matching. The net result is the regular toolchain, because it's registered earlier, would always match earlier. To fix, set the target_settings in the toolchain() call. This matches the bzlmod behavior. Also update the toolchain resolution tests to verify resolution. Fixes https://github.com/bazel-contrib/rules_python/issues/3262 --- CHANGELOG.md | 4 ++++ python/private/toolchains_repo.bzl | 2 +- .../multi_platform_resolution/resolution_tests.bzl | 9 --------- 3 files changed, 5 insertions(+), 10 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 3f85127a7b..2955d5f268 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -88,6 +88,10 @@ END_UNRELEASED_TEMPLATE ([#3204](https://github.com/bazel-contrib/rules_python/issues/3204)). * (uv) {obj}`//python/uv:lock.bzl%lock` now works with a local platform runtime. +* (toolchains) WORKSPACE builds now correctly register musl and freethreaded + variants. Setting {obj}`--py_linux_libc=musl` and `--py_freethreaded=yes` now + activate them, respectively. + ([#3262](https://github.com/bazel-contrib/rules_python/issues/3262)). {#v0-0-0-added} ### Added diff --git a/python/private/toolchains_repo.bzl b/python/private/toolchains_repo.bzl index 93bbb52108..f7ff19c30e 100644 --- a/python/private/toolchains_repo.bzl +++ b/python/private/toolchains_repo.bzl @@ -214,7 +214,7 @@ def python_toolchain_build_file_content( user_repository_name = "{}_{}".format(user_repository_name, platform), python_version = python_version, set_python_version_constraint = set_python_version_constraint, - target_settings = [], + target_settings = meta.target_settings, )) return "\n\n".join(entries) diff --git a/tests/toolchains/multi_platform_resolution/resolution_tests.bzl b/tests/toolchains/multi_platform_resolution/resolution_tests.bzl index 609fa3fe53..a48d815789 100644 --- a/tests/toolchains/multi_platform_resolution/resolution_tests.bzl +++ b/tests/toolchains/multi_platform_resolution/resolution_tests.bzl @@ -10,7 +10,6 @@ load("@bazel_skylib//lib:structs.bzl", "structs") load("@rules_testing//lib:analysis_test.bzl", "analysis_test") load("@rules_testing//lib:test_suite.bzl", "test_suite") load("//python:versions.bzl", "TOOL_VERSIONS") -load("//python/private:bzlmod_enabled.bzl", "BZLMOD_ENABLED") # buildifier: disable=bzl-visibility load("//python/private:common_labels.bzl", "labels") # buildifier: disable=bzl-visibility load("//python/private:toolchain_types.bzl", "TARGET_TOOLCHAIN_TYPE") # buildifier: disable=bzl-visibility load("//python/private:version.bzl", "version") # buildifier: disable=bzl-visibility @@ -141,13 +140,6 @@ def _test_toolchains(name): if runtime.libc: config_settings[labels.PY_LINUX_LIBC] = runtime.libc - # TODO: Workspace isn't correctly registering musl and freethreaded - # toolchains, so skip them for now. - target_compatible_with = [] - if not BZLMOD_ENABLED and (runtime.libc == "musl" or - runtime.freethreaded == "yes"): - target_compatible_with = ["@platforms//:incompatible"] - analysis_test( name = test_name, target = name + "_current_toolchain", @@ -165,7 +157,6 @@ def _test_toolchains(name): "os={}".format(runtime.os), "arch={}".format(runtime.arch), ], - "target_compatible_with": target_compatible_with, }, ) From 79f654686646e2ab213741bc4774ea9d69895b54 Mon Sep 17 00:00:00 2001 From: Mai Hussien <70515749+mai93@users.noreply.github.com> Date: Fri, 3 Oct 2025 15:37:07 -0700 Subject: [PATCH 472/922] chore: reject py2 runtimes and remove usages of ctx.fragments.py.disable_py2 (#3319) This removes the usages of the disable_py2 fragment attribute. This also makes py2 runtimes rejected (both when set on py_runtime and PyRuntimeInfo) Because Python 2 support was dropped long ago and other parts of the code reject, ignore, or don't work with Python 2 already, this is not considered a breaking change. Work towards: https://github.com/bazel-contrib/rules_python/issues/3252 --------- Co-authored-by: Richard Levasseur --- CHANGELOG.md | 3 +++ python/private/flags.bzl | 1 - python/private/py_runtime_pair_rule.bzl | 17 +++-------------- python/private/py_runtime_rule.bzl | 7 +++---- 4 files changed, 9 insertions(+), 19 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 2955d5f268..34f658cb87 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -71,6 +71,9 @@ END_UNRELEASED_TEMPLATE paths, runtime site-packages]`. * (pip) Publishing deps are no longer pulled via `experimental_index_url`. ([#2937](https://github.com/bazel-contrib/rules_python/issues/2937)). +* (toolchains) `py_runtime` and `PyRuntimeInfo` reject Python 2 settings. + Setting `py_runtime.python_version = "PY2"` or non-None + `PyRuntimeInfo.py2_runtime` is an error. {#v0-0-0-fixed} ### Fixed diff --git a/python/private/flags.bzl b/python/private/flags.bzl index 82ec83294b..35181e9f96 100644 --- a/python/private/flags.bzl +++ b/python/private/flags.bzl @@ -39,7 +39,6 @@ load(":enum.bzl", "FlagEnum", "enum") _POSSIBLY_NATIVE_FLAGS = { "build_python_zip": (lambda ctx: ctx.fragments.py.build_python_zip, "native"), "default_to_explicit_init_py": (lambda ctx: ctx.fragments.py.default_to_explicit_init_py, "native"), - "disable_py2": (lambda ctx: ctx.fragments.py.disable_py2, "native"), "python_import_all_repositories": (lambda ctx: ctx.fragments.bazel_py.python_import_all_repositories, "native"), "python_path": (lambda ctx: ctx.fragments.bazel_py.python_path, "native"), } diff --git a/python/private/py_runtime_pair_rule.bzl b/python/private/py_runtime_pair_rule.bzl index 203e5d4df7..c6c4c34d19 100644 --- a/python/private/py_runtime_pair_rule.bzl +++ b/python/private/py_runtime_pair_rule.bzl @@ -17,7 +17,6 @@ load("@bazel_skylib//rules:common_settings.bzl", "BuildSettingInfo") load("//python:py_runtime_info.bzl", "PyRuntimeInfo") load(":common_labels.bzl", "labels") -load(":flags.bzl", "read_possibly_native_flag") load(":reexports.bzl", "BuiltinPyRuntimeInfo") def _py_runtime_pair_impl(ctx): @@ -37,10 +36,9 @@ def _py_runtime_pair_impl(ctx): else: py3_runtime = None - # TODO: Uncomment this after --incompatible_python_disable_py2 defaults to true - # if _is_py2_disabled(ctx) and py2_runtime != None: - # fail("Using Python 2 is not supported and disabled; see " + - # "https://github.com/bazelbuild/bazel/issues/15684") + if py2_runtime != None: + fail("Using Python 2 is not supported and disabled; see " + + "https://github.com/bazelbuild/bazel/issues/15684") extra_kwargs = {} if ctx.attr._visible_for_testing[BuildSettingInfo].value: @@ -62,15 +60,6 @@ def _get_py_runtime_info(target): else: return target[BuiltinPyRuntimeInfo] -# buildifier: disable=unused-variable -def _is_py2_disabled(ctx): - # Because this file isn't bundled with Bazel, so we have to conditionally - # check for this flag. - # TODO: Remove this once all supported Balze versions have this flag. - if not hasattr(ctx.fragments.py, "disable_py"): - return False - return read_possibly_native_flag(ctx, "disable_py2") - _MaybeBuiltinPyRuntimeInfo = [[BuiltinPyRuntimeInfo]] if BuiltinPyRuntimeInfo != None else [] py_runtime_pair = rule( diff --git a/python/private/py_runtime_rule.bzl b/python/private/py_runtime_rule.bzl index f8182e73da..3bcee4cfd7 100644 --- a/python/private/py_runtime_rule.bzl +++ b/python/private/py_runtime_rule.bzl @@ -87,10 +87,9 @@ def _py_runtime_impl(ctx): if python_version_flag: interpreter_version_info = _interpreter_version_info_from_version_str(python_version_flag) - # TODO: Uncomment this after --incompatible_python_disable_py2 defaults to true - # if ctx.fragments.py.disable_py2 and python_version == "PY2": - # fail("Using Python 2 is not supported and disabled; see " + - # "https://github.com/bazelbuild/bazel/issues/15684") + if python_version == "PY2": + fail("Using Python 2 is not supported and disabled; see " + + "https://github.com/bazelbuild/bazel/issues/15684") pyc_tag = ctx.attr.pyc_tag if not pyc_tag and (ctx.attr.implementation_name and From 1ebc93f19de1420ea12c59e3bfb315fec92e54f5 Mon Sep 17 00:00:00 2001 From: Ignas Anikevicius <240938+aignas@users.noreply.github.com> Date: Sun, 5 Oct 2025 13:26:45 +0900 Subject: [PATCH 473/922] feat(pypi): enable pipstar by default (#3225) Before this PR we were using our python helpers to parse the whl METADATA for a particular host (or experimentally - target) platform and the fixed dependency list would be written to the `BUILD.bazel` files materialized by the `whl_library` repository rule. This PR adds extra plumbing to leverage the Starlark implementation of the METADATA file parsing (a.k.a. pipstar) which moves the evaluation to analysis phase as we will get the target platform details via the `env_marker_config_setting` from the `toolchain` used for the dependencies. Since users are normally working with a subset of platforms that the METADATA would pull in, we are writing the list of packages from the requirements file to a `bzl` file so that `pipstar` knows which packages to consider when parsing the METADATA. This will ensure that `bazel query` continues to work. Since just passing the list of packages to the `whl_library` would cause refetches of all of the `whl_library` dependencies when we add or remove a package, we pass a path where to load the symbol called `packages` from, which means that only the analysis phase will be affected because the macros will be re-evaluated if one adds or removes a package. We still need to download the correct platform-specific wheels for the right platform in order to fully resolve the referenced tickets. With this PR we are deprecating the `experimental_target_platforms` attribute since it has no longer any effect. Fixes #2949 Work towards #260 Work towards #2241 --------- Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> --- CHANGELOG.md | 8 ++ examples/pip_parse_vendored/requirements.bzl | 14 +++- python/pip_install/BUILD.bazel | 2 +- python/pip_install/pip_repository.bzl | 5 +- python/private/internal_config_repo.bzl | 2 +- python/private/pypi/BUILD.bazel | 17 ++-- ...config.bzl.tmpl.bzlmod => config.bzl.tmpl} | 4 +- .../pypi/generate_whl_library_build_bazel.bzl | 19 ++--- python/private/pypi/group_library.bzl | 40 ---------- python/private/pypi/hub_builder.bzl | 1 + python/private/pypi/hub_repository.bzl | 4 +- python/private/pypi/pip_repository.bzl | 41 +++++++--- .../pypi/requirements.bzl.tmpl.workspace | 4 +- python/private/pypi/whl_config_repo.bzl | 46 +++++++++++ python/private/pypi/whl_library.bzl | 15 +++- python/private/pypi/whl_library_targets.bzl | 14 +++- python/private/pypi/whl_metadata.bzl | 6 +- .../ns-sub1/ns-sub1-1.0.dist-info/METADATA | 2 + .../ns-sub2/ns_sub2-1.0.dist-info/METADATA | 2 + tests/pypi/extension/extension_tests.bzl | 1 + ...generate_whl_library_build_bazel_tests.bzl | 79 +++++++++++++++++-- tests/pypi/hub_builder/hub_builder_tests.bzl | 67 +++++++--------- .../whl_library_targets_tests.bzl | 6 +- .../testdata/somepkg-1.0.dist-info/METADATA | 2 + 24 files changed, 267 insertions(+), 134 deletions(-) rename python/private/pypi/{config.bzl.tmpl.bzlmod => config.bzl.tmpl} (79%) delete mode 100644 python/private/pypi/group_library.bzl create mode 100644 python/private/pypi/whl_config_repo.bzl diff --git a/CHANGELOG.md b/CHANGELOG.md index 34f658cb87..cc59e387ee 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -74,6 +74,14 @@ END_UNRELEASED_TEMPLATE * (toolchains) `py_runtime` and `PyRuntimeInfo` reject Python 2 settings. Setting `py_runtime.python_version = "PY2"` or non-None `PyRuntimeInfo.py2_runtime` is an error. +* (pypi) `pipstar` flag has been flipped to be enabled by default, to turn it + off use `RULES_PYTHON_ENABLE_PIPSTAR=0` environment variable. If you do, please + add a comment to + [#2949](https://github.com/bazel-contrib/rules_python/issues/2949). + With this release we are deprecating {obj}`pip.parse.experimental_target_platforms` and + {obj}`pip_repository.experimental_target_platforms`. For users using `WORKSPACE` and + vendoring the `requirements.bzl` file, please re-vendor so that downstream is unaffected + when the APIs get removed. {#v0-0-0-fixed} ### Fixed diff --git a/examples/pip_parse_vendored/requirements.bzl b/examples/pip_parse_vendored/requirements.bzl index ead5c49b26..f5551573fb 100644 --- a/examples/pip_parse_vendored/requirements.bzl +++ b/examples/pip_parse_vendored/requirements.bzl @@ -4,7 +4,7 @@ """ load("@rules_python//python:pip.bzl", "pip_utils") -load("@rules_python//python/pip_install:pip_repository.bzl", "group_library", "whl_library") +load("@rules_python//python/pip_install:pip_repository.bzl", "whl_config_repo", "whl_library") all_requirements = [ "@my_project_pip_deps_vendored_certifi//:pkg", @@ -91,12 +91,17 @@ def install_deps(**whl_library_kwargs): for requirement in group_requirements } - group_repo = "my_project_pip_deps_vendored__groups" - group_library( - name = group_repo, + config_repo = "my_project_pip_deps_vendored__config" + whl_config_repo( + name = "my_project_pip_deps_vendored__config", repo_prefix = "my_project_pip_deps_vendored_", groups = all_requirement_groups, + whl_map = { + p: "" + for p in all_whl_requirements_by_package + }, ) + config_load = "@{}//:config.bzl".format(config_repo) # Install wheels which may be participants in a group whl_config = dict(_config) @@ -112,5 +117,6 @@ def install_deps(**whl_library_kwargs): group_name = group_name, group_deps = group_deps, annotation = _get_annotation(requirement), + config_load = config_load, **whl_config ) diff --git a/python/pip_install/BUILD.bazel b/python/pip_install/BUILD.bazel index 09bc46eea7..665375cc5b 100644 --- a/python/pip_install/BUILD.bazel +++ b/python/pip_install/BUILD.bazel @@ -22,9 +22,9 @@ bzl_library( name = "pip_repository_bzl", srcs = ["pip_repository.bzl"], deps = [ - "//python/private/pypi:group_library_bzl", "//python/private/pypi:package_annotation_bzl", "//python/private/pypi:pip_repository_bzl", + "//python/private/pypi:whl_config_repo_bzl", "//python/private/pypi:whl_library_bzl", ], ) diff --git a/python/pip_install/pip_repository.bzl b/python/pip_install/pip_repository.bzl index 18deee1993..f9c3c9fb56 100644 --- a/python/pip_install/pip_repository.bzl +++ b/python/pip_install/pip_repository.bzl @@ -14,13 +14,14 @@ "" -load("//python/private/pypi:group_library.bzl", _group_library = "group_library") load("//python/private/pypi:package_annotation.bzl", _package_annotation = "package_annotation") load("//python/private/pypi:pip_repository.bzl", _pip_repository = "pip_repository") +load("//python/private/pypi:whl_config_repo.bzl", _whl_config_repo = "whl_config_repo") load("//python/private/pypi:whl_library.bzl", _whl_library = "whl_library") # Re-exports for backwards compatibility -group_library = _group_library +group_library = _whl_config_repo pip_repository = _pip_repository whl_library = _whl_library +whl_config_repo = _whl_config_repo package_annotation = _package_annotation diff --git a/python/private/internal_config_repo.bzl b/python/private/internal_config_repo.bzl index 109e68a8a1..0c6210696e 100644 --- a/python/private/internal_config_repo.bzl +++ b/python/private/internal_config_repo.bzl @@ -22,7 +22,7 @@ load("//python/private:text_util.bzl", "render") load(":repo_utils.bzl", "repo_utils") _ENABLE_PIPSTAR_ENVVAR_NAME = "RULES_PYTHON_ENABLE_PIPSTAR" -_ENABLE_PIPSTAR_DEFAULT = "0" +_ENABLE_PIPSTAR_DEFAULT = "1" _ENABLE_DEPRECATION_WARNINGS_ENVVAR_NAME = "RULES_PYTHON_DEPRECATION_WARNINGS" _ENABLE_DEPRECATION_WARNINGS_DEFAULT = "0" diff --git a/python/private/pypi/BUILD.bazel b/python/private/pypi/BUILD.bazel index b9650c8152..7d5314dd62 100644 --- a/python/private/pypi/BUILD.bazel +++ b/python/private/pypi/BUILD.bazel @@ -157,14 +157,6 @@ bzl_library( ], ) -bzl_library( - name = "group_library_bzl", - srcs = ["group_library.bzl"], - deps = [ - ":generate_group_library_build_bazel_bzl", - ], -) - bzl_library( name = "hub_builder_bzl", srcs = ["hub_builder.bzl"], @@ -408,6 +400,15 @@ bzl_library( ], ) +bzl_library( + name = "whl_config_repo_bzl", + srcs = ["whl_config_repo.bzl"], + deps = [ + ":generate_group_library_build_bazel_bzl", + "//python/private:text_util_bzl", + ], +) + bzl_library( name = "whl_config_setting_bzl", srcs = ["whl_config_setting.bzl"], diff --git a/python/private/pypi/config.bzl.tmpl.bzlmod b/python/private/pypi/config.bzl.tmpl similarity index 79% rename from python/private/pypi/config.bzl.tmpl.bzlmod rename to python/private/pypi/config.bzl.tmpl index c3ada70d27..1037153cac 100644 --- a/python/private/pypi/config.bzl.tmpl.bzlmod +++ b/python/private/pypi/config.bzl.tmpl @@ -2,8 +2,6 @@ NOTE: This is internal `rules_python` API and if you would like to depend on it, please raise an issue with your usecase. This may change in between rules_python versions without any notice. - -@generated by rules_python pip.parse bzlmod extension. """ -whl_map = %%WHL_MAP%% +packages = %%PACKAGES%% diff --git a/python/private/pypi/generate_whl_library_build_bazel.bzl b/python/private/pypi/generate_whl_library_build_bazel.bzl index 3764e720c0..e207f6d2f5 100644 --- a/python/private/pypi/generate_whl_library_build_bazel.bzl +++ b/python/private/pypi/generate_whl_library_build_bazel.bzl @@ -72,6 +72,7 @@ def generate_whl_library_build_bazel( "requires", "metadata_name", "metadata_version", + "packages", "include", ] else: @@ -82,17 +83,13 @@ def generate_whl_library_build_bazel( "target_platforms", "default_python_version", ] - dep_template = kwargs.get("dep_template") - loads.append( - """load("{}", "{}")""".format( - dep_template.format( - name = "", - target = "config.bzl", - ), - "whl_map", - ), - ) - kwargs["include"] = "whl_map" + packages_load = kwargs.pop("config_load") + if not kwargs.get("requires_dist"): + # no deps, we can leave the extra loads out + pass + else: + loads.append("""load("{}", "{}")""".format(packages_load, "packages")) + kwargs["include"] = "packages" for arg in unsupported_args: if kwargs.get(arg): diff --git a/python/private/pypi/group_library.bzl b/python/private/pypi/group_library.bzl deleted file mode 100644 index ff800e2f18..0000000000 --- a/python/private/pypi/group_library.bzl +++ /dev/null @@ -1,40 +0,0 @@ -# Copyright 2024 The Bazel Authors. All rights reserved. -# -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -"""group_library implementation for WORKSPACE setups.""" - -load(":generate_group_library_build_bazel.bzl", "generate_group_library_build_bazel") - -def _group_library_impl(rctx): - build_file_contents = generate_group_library_build_bazel( - repo_prefix = rctx.attr.repo_prefix, - groups = rctx.attr.groups, - ) - rctx.file("BUILD.bazel", build_file_contents) - -group_library = repository_rule( - attrs = { - "groups": attr.string_list_dict( - doc = "A mapping of group names to requirements within that group.", - ), - "repo_prefix": attr.string( - doc = "Prefix used for the whl_library created components of each group", - ), - }, - implementation = _group_library_impl, - doc = """ -Create a package containing only wrapper py_library and whl_library rules for implementing dependency groups. -This is an implementation detail of dependency groups and should not be used alone. - """, -) diff --git a/python/private/pypi/hub_builder.bzl b/python/private/pypi/hub_builder.bzl index b6088e4ded..58d35f2681 100644 --- a/python/private/pypi/hub_builder.bzl +++ b/python/private/pypi/hub_builder.bzl @@ -451,6 +451,7 @@ def _common_args(self, module_ctx, *, pip_attr): # attrs. whl_library_args = dict( dep_template = "@{}//{{name}}:{{target}}".format(self.name), + config_load = "@{}//:config.bzl".format(self.name), ) maybe_args = dict( # The following values are safe to omit if they have false like values diff --git a/python/private/pypi/hub_repository.bzl b/python/private/pypi/hub_repository.bzl index 1d572d09e2..f915aa1c77 100644 --- a/python/private/pypi/hub_repository.bzl +++ b/python/private/pypi/hub_repository.bzl @@ -50,7 +50,7 @@ def _impl(rctx): "config.bzl", rctx.attr._config_template, substitutions = { - "%%WHL_MAP%%": render.dict(rctx.attr.whl_map, value_repr = lambda x: "None"), + "%%PACKAGES%%": render.dict(rctx.attr.whl_map, value_repr = lambda x: "None"), }, ) rctx.template("requirements.bzl", rctx.attr._requirements_bzl_template, substitutions = { @@ -100,7 +100,7 @@ in the pip.parse tag class. """, ), "_config_template": attr.label( - default = ":config.bzl.tmpl.bzlmod", + default = ":config.bzl.tmpl", ), "_requirements_bzl_template": attr.label( default = ":requirements.bzl.tmpl.bzlmod", diff --git a/python/private/pypi/pip_repository.bzl b/python/private/pypi/pip_repository.bzl index 2cf20cd5a7..e9a4c44da3 100644 --- a/python/private/pypi/pip_repository.bzl +++ b/python/private/pypi/pip_repository.bzl @@ -24,6 +24,17 @@ load(":pip_repository_attrs.bzl", "ATTRS") load(":render_pkg_aliases.bzl", "render_pkg_aliases") load(":requirements_files_by_platform.bzl", "requirements_files_by_platform") +_CONFIG_REPO_TEMPLATE = """"{name}__config" + whl_config_repo( + name = "{name}__config", + repo_prefix = "{name}_", + groups = all_requirement_groups, + whl_map = {{ + p: "" + for p in all_whl_requirements_by_package + }}, + )""" + def _get_python_interpreter_attr(rctx): """A helper function for getting the `python_interpreter` attribute or it's default @@ -156,7 +167,7 @@ def _pip_repository_impl(rctx): imports = [ # NOTE: Maintain the order consistent with `buildifier` 'load("@rules_python//python:pip.bzl", "pip_utils")', - 'load("@rules_python//python/pip_install:pip_repository.bzl", "group_library", "whl_library")', + 'load("@rules_python//python/pip_install:pip_repository.bzl", "whl_config_repo", "whl_library")', ] annotations = {} @@ -193,7 +204,7 @@ def _pip_repository_impl(rctx): aliases = render_pkg_aliases( aliases = { pkg: rctx.attr.name + "_" + pkg - for pkg in bzl_packages or [] + for pkg in bzl_packages }, extra_hub_aliases = rctx.attr.extra_hub_aliases, requirement_cycles = requirement_cycles, @@ -202,14 +213,22 @@ def _pip_repository_impl(rctx): rctx.file(path, contents) rctx.file("BUILD.bazel", _BUILD_FILE_CONTENTS) + if rctx.attr.use_hub_alias_dependencies: + rctx.template( + "config.bzl", + rctx.attr._config_template, + substitutions = { + "%%PACKAGES%%": render.dict({ + pkg: None + for pkg in bzl_packages + }, value_repr = lambda x: "None"), + }, + ) + config_repo_template = repr(rctx.attr.name) + else: + config_repo_template = _CONFIG_REPO_TEMPLATE.format(name = rctx.attr.name) + rctx.template("requirements.bzl", rctx.attr._template, substitutions = { - " # %%GROUP_LIBRARY%%": """\ - group_repo = "{name}__groups" - group_library( - name = group_repo, - repo_prefix = "{name}_", - groups = all_requirement_groups, - )""".format(name = rctx.attr.name) if not rctx.attr.use_hub_alias_dependencies else "", "%%ALL_DATA_REQUIREMENTS%%": render.list([ macro_tmpl.format(p, "data") for p in bzl_packages @@ -225,6 +244,7 @@ def _pip_repository_impl(rctx): }), "%%ANNOTATIONS%%": render.dict(dict(sorted(annotations.items()))), "%%CONFIG%%": render.dict(dict(sorted(config.items()))), + "%%CONFIG_REPO%%": config_repo_template, "%%EXTRA_PIP_ARGS%%": json.encode(options), "%%IMPORTS%%": "\n".join(imports), "%%MACRO_TMPL%%": macro_tmpl, @@ -249,6 +269,9 @@ generated using the `package_name` macro. For example usage, see [this WORKSPACE file](https://github.com/bazel-contrib/rules_python/blob/main/examples/pip_repository_annotations/WORKSPACE). """, ), + _config_template = attr.label( + default = ":config.bzl.tmpl", + ), _template = attr.label( default = ":requirements.bzl.tmpl.workspace", ), diff --git a/python/private/pypi/requirements.bzl.tmpl.workspace b/python/private/pypi/requirements.bzl.tmpl.workspace index 2f4bcd6916..f61a5271f6 100644 --- a/python/private/pypi/requirements.bzl.tmpl.workspace +++ b/python/private/pypi/requirements.bzl.tmpl.workspace @@ -52,7 +52,8 @@ def install_deps(**whl_library_kwargs): for requirement in group_requirements } - # %%GROUP_LIBRARY%% + config_repo = %%CONFIG_REPO%% + config_load = "@{}//:config.bzl".format(config_repo) # Install wheels which may be participants in a group whl_config = dict(_config) @@ -68,5 +69,6 @@ def install_deps(**whl_library_kwargs): group_name = group_name, group_deps = group_deps, annotation = _get_annotation(requirement), + config_load = config_load, **whl_config ) diff --git a/python/private/pypi/whl_config_repo.bzl b/python/private/pypi/whl_config_repo.bzl new file mode 100644 index 0000000000..b7cea5a8b7 --- /dev/null +++ b/python/private/pypi/whl_config_repo.bzl @@ -0,0 +1,46 @@ +"""whl_config_library implementation for WORKSPACE setups.""" + +load("//python/private:text_util.bzl", "render") +load(":generate_group_library_build_bazel.bzl", "generate_group_library_build_bazel") + +def _whl_config_repo_impl(rctx): + build_file_contents = generate_group_library_build_bazel( + repo_prefix = rctx.attr.repo_prefix, + groups = rctx.attr.groups, + ) + rctx.file("_groups/BUILD.bazel", build_file_contents) + rctx.file("BUILD.bazel", "") + rctx.template( + "config.bzl", + rctx.attr._config_template, + substitutions = { + "%%PACKAGES%%": render.dict(rctx.attr.whl_map or {}, value_repr = lambda x: "None"), + }, + ) + +whl_config_repo = repository_rule( + attrs = { + "groups": attr.string_list_dict( + doc = "A mapping of group names to requirements within that group.", + ), + "repo_prefix": attr.string( + doc = "Prefix used for the whl_library created components of each group", + ), + "whl_map": attr.string_dict( + doc = """\ +The wheel map where values are json.encoded strings of the whl_map constructed +in the pip.parse tag class. +""", + ), + "_config_template": attr.label( + default = ":config.bzl.tmpl", + ), + }, + doc = """ +Create a package containing only wrapper py_library and whl_library rules for implementing dependency groups. +This is an implementation detail of dependency groups and should not be used alone. + +PRIVATE USE ONLY, only used in WORKSPACE. + """, + implementation = _whl_config_repo_impl, +) diff --git a/python/private/pypi/whl_library.bzl b/python/private/pypi/whl_library.bzl index 5cc53d84c6..fe3308ad3c 100644 --- a/python/private/pypi/whl_library.bzl +++ b/python/private/pypi/whl_library.bzl @@ -369,7 +369,12 @@ def _whl_library_impl(rctx): timeout = rctx.attr.timeout, ) - if rp_config.enable_pipstar: + # NOTE @aignas 2025-09-28: if someone has an old vendored file that does not have the + # dep_template set or the packages is not set either, we should still not break, best to + # disable pipstar for that particular case. + # + # Remove non-pipstar and config_load check when we release rules_python 2. + if rp_config.enable_pipstar and rctx.attr.config_load: pypi_repo_utils.execute_checked( rctx, op = "whl_library.ExtractWheel({}, {})".format(rctx.attr.name, whl_path), @@ -422,7 +427,10 @@ def _whl_library_impl(rctx): build_file_contents = generate_whl_library_build_bazel( name = whl_path.basename, sdist_filename = sdist_filename, - dep_template = rctx.attr.dep_template or "@{}{{name}}//:{{target}}".format(rctx.attr.repo_prefix), + dep_template = rctx.attr.dep_template or "@{}{{name}}//:{{target}}".format( + rctx.attr.repo_prefix, + ), + config_load = rctx.attr.config_load, entry_points = entry_points, metadata_name = metadata.name, metadata_version = metadata.version, @@ -572,6 +580,9 @@ whl_library_attrs = dict({ ), allow_files = True, ), + "config_load": attr.string( + doc = "The load location for configuration for pipstar.", + ), "dep_template": attr.string( doc = """ The dep template to use for referencing the dependencies. It should have `{name}` diff --git a/python/private/pypi/whl_library_targets.bzl b/python/private/pypi/whl_library_targets.bzl index 89c1d348b3..a2d77daf4c 100644 --- a/python/private/pypi/whl_library_targets.bzl +++ b/python/private/pypi/whl_library_targets.bzl @@ -273,13 +273,23 @@ def whl_library_targets( # implementation. if group_name and "//:" in dep_template: # This is the legacy behaviour where the group library is outside the hub repo + # + # It is expected to disappear when we drop WORKSPACE or drop the vendoring of + # pip_parse `requirements.bzl` in WORKSPACE. The alternative would be to add + # another argument to the macro, but it is already full of arguments. label_tmpl = dep_template.format( - name = "_groups", + name = "_config", target = normalize_name(group_name) + "_{}", + ).replace( + "//:", + "//_groups:", ) impl_vis = [dep_template.format( - name = "_groups", + name = "_config", target = "__pkg__", + ).replace( + "//:", + "//_groups:", )] native.alias( diff --git a/python/private/pypi/whl_metadata.bzl b/python/private/pypi/whl_metadata.bzl index cf2d51afda..a56aac5782 100644 --- a/python/private/pypi/whl_metadata.bzl +++ b/python/private/pypi/whl_metadata.bzl @@ -26,7 +26,11 @@ def whl_metadata(*, install_dir, read_fn, logger): result = parse_whl_metadata(contents) if not (result.name and result.version): - logger.fail("Failed to parsed the wheel METADATA file:\n{}".format(contents)) + logger.fail("Failed to parse the wheel METADATA file:\n{}\n{}\n{}".format( + 80 * "=", + contents.rstrip("\n"), + 80 * "=", + )) return None return result diff --git a/tests/implicit_namespace_packages/testdata/ns-sub1/ns-sub1-1.0.dist-info/METADATA b/tests/implicit_namespace_packages/testdata/ns-sub1/ns-sub1-1.0.dist-info/METADATA index e69de29bb2..ecec6086ba 100644 --- a/tests/implicit_namespace_packages/testdata/ns-sub1/ns-sub1-1.0.dist-info/METADATA +++ b/tests/implicit_namespace_packages/testdata/ns-sub1/ns-sub1-1.0.dist-info/METADATA @@ -0,0 +1,2 @@ +Name: ns-sub1 +Version: 1.0 diff --git a/tests/implicit_namespace_packages/testdata/ns-sub2/ns_sub2-1.0.dist-info/METADATA b/tests/implicit_namespace_packages/testdata/ns-sub2/ns_sub2-1.0.dist-info/METADATA index e69de29bb2..92cbb8ec2e 100644 --- a/tests/implicit_namespace_packages/testdata/ns-sub2/ns_sub2-1.0.dist-info/METADATA +++ b/tests/implicit_namespace_packages/testdata/ns-sub2/ns_sub2-1.0.dist-info/METADATA @@ -0,0 +1,2 @@ +Name: ns-sub2 +Version: 1.0 diff --git a/tests/pypi/extension/extension_tests.bzl b/tests/pypi/extension/extension_tests.bzl index 0514e1d95b..b1e363bc7b 100644 --- a/tests/pypi/extension/extension_tests.bzl +++ b/tests/pypi/extension/extension_tests.bzl @@ -168,6 +168,7 @@ def _test_simple(env): }}) pypi.whl_libraries().contains_exactly({ "pypi_315_simple": { + "config_load": "@pypi//:config.bzl", "dep_template": "@pypi//{name}:{target}", "python_interpreter_target": "unit_test_interpreter_target", "requirement": "simple==0.0.1 --hash=sha256:deadbeef --hash=sha256:deadbaaf", diff --git a/tests/pypi/generate_whl_library_build_bazel/generate_whl_library_build_bazel_tests.bzl b/tests/pypi/generate_whl_library_build_bazel/generate_whl_library_build_bazel_tests.bzl index 225b296ebf..39c2eb4379 100644 --- a/tests/pypi/generate_whl_library_build_bazel/generate_whl_library_build_bazel_tests.bzl +++ b/tests/pypi/generate_whl_library_build_bazel/generate_whl_library_build_bazel_tests.bzl @@ -37,7 +37,7 @@ whl_library_targets( "exclude_via_attr", "data_exclude_all", ], - dep_template = "@pypi//{name}:{target}", + dep_template = "@pypi_{name}//:{target}", dependencies = ["foo"], dependencies_by_platform = { "baz": ["bar"], @@ -59,7 +59,7 @@ whl_library_targets( # SOMETHING SPECIAL AT THE END """ actual = generate_whl_library_build_bazel( - dep_template = "@pypi//{name}:{target}", + dep_template = "@pypi_{name}//:{target}", name = "foo.whl", dependencies = ["foo"], dependencies_by_platform = {"baz": ["bar"]}, @@ -83,9 +83,74 @@ whl_library_targets( _tests.append(_test_all_legacy) +def _test_all_workspace(env): + want = """\ +load("@pypi//:config.bzl", "packages") +load("@rules_python//python/private/pypi:whl_library_targets.bzl", "whl_library_targets_from_requires") + +package(default_visibility = ["//visibility:public"]) + +whl_library_targets_from_requires( + copy_executables = { + "exec_src": "exec_dest", + }, + copy_files = { + "file_src": "file_dest", + }, + data = ["extra_target"], + data_exclude = [ + "exclude_via_attr", + "data_exclude_all", + ], + dep_template = "@pypi//{name}:{target}", + entry_points = { + "foo": "bar.py", + }, + group_deps = [ + "foo", + "fox", + "qux", + ], + group_name = "qux", + include = packages, + name = "foo.whl", + requires_dist = [ + "foo", + "bar-baz", + "qux", + ], + srcs_exclude = ["srcs_exclude_all"], +) + +# SOMETHING SPECIAL AT THE END +""" + actual = generate_whl_library_build_bazel( + dep_template = "@pypi//{name}:{target}", + name = "foo.whl", + requires_dist = ["foo", "bar-baz", "qux"], + entry_points = { + "foo": "bar.py", + }, + data_exclude = ["exclude_via_attr"], + annotation = struct( + copy_files = {"file_src": "file_dest"}, + copy_executables = {"exec_src": "exec_dest"}, + data = ["extra_target"], + data_exclude_glob = ["data_exclude_all"], + srcs_exclude_glob = ["srcs_exclude_all"], + additive_build_content = """# SOMETHING SPECIAL AT THE END""", + ), + config_load = "@pypi//:config.bzl", + group_name = "qux", + group_deps = ["foo", "fox", "qux"], + ) + env.expect.that_str(actual.replace("@@", "@")).equals(want) + +_tests.append(_test_all_workspace) + def _test_all(env): want = """\ -load("@pypi//:config.bzl", "whl_map") +load("@pypi//:config.bzl", "packages") load("@rules_python//python/private/pypi:whl_library_targets.bzl", "whl_library_targets_from_requires") package(default_visibility = ["//visibility:public"]) @@ -112,7 +177,7 @@ whl_library_targets_from_requires( "qux", ], group_name = "qux", - include = whl_map, + include = packages, name = "foo.whl", requires_dist = [ "foo", @@ -140,6 +205,7 @@ whl_library_targets_from_requires( srcs_exclude_glob = ["srcs_exclude_all"], additive_build_content = """# SOMETHING SPECIAL AT THE END""", ), + config_load = "@pypi//:config.bzl", group_name = "qux", group_deps = ["foo", "fox", "qux"], ) @@ -149,7 +215,7 @@ _tests.append(_test_all) def _test_all_with_loads(env): want = """\ -load("@pypi//:config.bzl", "whl_map") +load("@pypi//:config.bzl", "packages") load("@rules_python//python/private/pypi:whl_library_targets.bzl", "whl_library_targets_from_requires") package(default_visibility = ["//visibility:public"]) @@ -176,7 +242,7 @@ whl_library_targets_from_requires( "qux", ], group_name = "qux", - include = whl_map, + include = packages, name = "foo.whl", requires_dist = [ "foo", @@ -205,6 +271,7 @@ whl_library_targets_from_requires( additive_build_content = """# SOMETHING SPECIAL AT THE END""", ), group_name = "qux", + config_load = "@pypi//:config.bzl", group_deps = ["foo", "fox", "qux"], ) env.expect.that_str(actual.replace("@@", "@")).equals(want) diff --git a/tests/pypi/hub_builder/hub_builder_tests.bzl b/tests/pypi/hub_builder/hub_builder_tests.bzl index 9f6ee6720d..ee6200a70a 100644 --- a/tests/pypi/hub_builder/hub_builder_tests.bzl +++ b/tests/pypi/hub_builder/hub_builder_tests.bzl @@ -40,7 +40,7 @@ simple==0.0.1 \ def hub_builder( env, - enable_pipstar = False, + enable_pipstar = True, debug = False, config = None, minor_mapping = {}, @@ -135,6 +135,7 @@ def _test_simple(env): }) pypi.whl_libraries().contains_exactly({ "pypi_315_simple": { + "config_load": "@pypi//:config.bzl", "dep_template": "@pypi//{name}:{target}", "python_interpreter_target": "unit_test_interpreter_target", "requirement": "simple==0.0.1 --hash=sha256:deadbeef --hash=sha256:deadbaaf", @@ -186,11 +187,13 @@ def _test_simple_multiple_requirements(env): }) pypi.whl_libraries().contains_exactly({ "pypi_315_simple_osx_aarch64": { + "config_load": "@pypi//:config.bzl", "dep_template": "@pypi//{name}:{target}", "python_interpreter_target": "unit_test_interpreter_target", "requirement": "simple==0.0.2 --hash=sha256:deadb00f", }, "pypi_315_simple_windows_aarch64": { + "config_load": "@pypi//:config.bzl", "dep_template": "@pypi//{name}:{target}", "python_interpreter_target": "unit_test_interpreter_target", "requirement": "simple==0.0.1 --hash=sha256:deadbeef", @@ -276,21 +279,25 @@ new-package==0.0.1 --hash=sha256:deadb00f2 }) pypi.whl_libraries().contains_exactly({ "pypi_315_old_package": { + "config_load": "@pypi//:config.bzl", "dep_template": "@pypi//{name}:{target}", "python_interpreter_target": "unit_test_interpreter_target", "requirement": "old-package==0.0.1 --hash=sha256:deadbaaf", }, "pypi_315_simple": { + "config_load": "@pypi//:config.bzl", "dep_template": "@pypi//{name}:{target}", "python_interpreter_target": "unit_test_interpreter_target", "requirement": "simple==0.0.1 --hash=sha256:deadbeef", }, "pypi_316_new_package": { + "config_load": "@pypi//:config.bzl", "dep_template": "@pypi//{name}:{target}", "python_interpreter_target": "unit_test_interpreter_target", "requirement": "new-package==0.0.1 --hash=sha256:deadb00f2", }, "pypi_316_simple": { + "config_load": "@pypi//:config.bzl", "dep_template": "@pypi//{name}:{target}", "python_interpreter_target": "unit_test_interpreter_target", "requirement": "simple==0.0.2 --hash=sha256:deadb00f", @@ -357,11 +364,13 @@ torch==2.4.1 ; platform_machine != 'x86_64' \ }) pypi.whl_libraries().contains_exactly({ "pypi_315_torch_linux_aarch64_osx_aarch64_windows_aarch64": { + "config_load": "@pypi//:config.bzl", "dep_template": "@pypi//{name}:{target}", "python_interpreter_target": "unit_test_interpreter_target", "requirement": "torch==2.4.1 --hash=sha256:deadbeef", }, "pypi_315_torch_linux_x86_64_linux_x86_64_freethreaded": { + "config_load": "@pypi//:config.bzl", "dep_template": "@pypi//{name}:{target}", "python_interpreter_target": "unit_test_interpreter_target", "requirement": "torch==2.4.1+cpu", @@ -405,7 +414,7 @@ def _test_torch_experimental_index_url(env): env, config = struct( netrc = None, - enable_pipstar = False, + enable_pipstar = True, auth_patterns = {}, platforms = { "{}_{}".format(os, cpu): _plat( @@ -515,8 +524,8 @@ torch==2.4.1+cpu ; platform_machine == 'x86_64' \ }) pypi.whl_libraries().contains_exactly({ "pypi_312_torch_cp312_cp312_linux_x86_64_8800deef": { + "config_load": "@pypi//:config.bzl", "dep_template": "@pypi//{name}:{target}", - "experimental_target_platforms": ["linux_x86_64"], "filename": "torch-2.4.1+cpu-cp312-cp312-linux_x86_64.whl", "python_interpreter_target": "unit_test_interpreter_target", "requirement": "torch==2.4.1+cpu", @@ -524,8 +533,8 @@ torch==2.4.1+cpu ; platform_machine == 'x86_64' \ "urls": ["https://torch.index/whl/cpu/torch-2.4.1%2Bcpu-cp312-cp312-linux_x86_64.whl"], }, "pypi_312_torch_cp312_cp312_manylinux_2_17_aarch64_36109432": { + "config_load": "@pypi//:config.bzl", "dep_template": "@pypi//{name}:{target}", - "experimental_target_platforms": ["linux_aarch64"], "filename": "torch-2.4.1-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", "python_interpreter_target": "unit_test_interpreter_target", "requirement": "torch==2.4.1", @@ -533,8 +542,8 @@ torch==2.4.1+cpu ; platform_machine == 'x86_64' \ "urls": ["https://torch.index/whl/cpu/torch-2.4.1-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl"], }, "pypi_312_torch_cp312_cp312_win_amd64_3a570e5c": { + "config_load": "@pypi//:config.bzl", "dep_template": "@pypi//{name}:{target}", - "experimental_target_platforms": ["windows_x86_64"], "filename": "torch-2.4.1+cpu-cp312-cp312-win_amd64.whl", "python_interpreter_target": "unit_test_interpreter_target", "requirement": "torch==2.4.1+cpu", @@ -542,8 +551,8 @@ torch==2.4.1+cpu ; platform_machine == 'x86_64' \ "urls": ["https://torch.index/whl/cpu/torch-2.4.1%2Bcpu-cp312-cp312-win_amd64.whl"], }, "pypi_312_torch_cp312_none_macosx_11_0_arm64_72b484d5": { + "config_load": "@pypi//:config.bzl", "dep_template": "@pypi//{name}:{target}", - "experimental_target_platforms": ["osx_aarch64"], "filename": "torch-2.4.1-cp312-none-macosx_11_0_arm64.whl", "python_interpreter_target": "unit_test_interpreter_target", "requirement": "torch==2.4.1", @@ -619,15 +628,15 @@ simple==0.0.3 \ }) pypi.whl_libraries().contains_exactly({ "pypi_315_extra": { + "config_load": "@pypi//:config.bzl", "dep_template": "@pypi//{name}:{target}", "download_only": True, - # TODO @aignas 2025-04-20: ensure that this is in the hub repo - # "experimental_target_platforms": ["cp315_linux_x86_64"], "extra_pip_args": ["--platform=manylinux_2_17_x86_64", "--python-version=315", "--implementation=cp", "--abi=cp315"], "python_interpreter_target": "unit_test_interpreter_target", "requirement": "extra==0.0.1 --hash=sha256:deadb00f", }, "pypi_315_simple_linux_x86_64": { + "config_load": "@pypi//:config.bzl", "dep_template": "@pypi//{name}:{target}", "download_only": True, "extra_pip_args": ["--platform=manylinux_2_17_x86_64", "--python-version=315", "--implementation=cp", "--abi=cp315"], @@ -635,6 +644,7 @@ simple==0.0.3 \ "requirement": "simple==0.0.1 --hash=sha256:deadbeef", }, "pypi_315_simple_osx_aarch64": { + "config_load": "@pypi//:config.bzl", "dep_template": "@pypi//{name}:{target}", "download_only": True, "extra_pip_args": ["--platform=macosx_10_9_arm64", "--python-version=315", "--implementation=cp", "--abi=cp315"], @@ -820,13 +830,8 @@ git_dep @ git+https://git.server/repo/project@deadbeefdeadbeef }) pypi.whl_libraries().contains_exactly({ "pypi_315_any_name": { + "config_load": "@pypi//:config.bzl", "dep_template": "@pypi//{name}:{target}", - "experimental_target_platforms": [ - "linux_aarch64", - "linux_x86_64", - "osx_aarch64", - "windows_aarch64", - ], "extra_pip_args": ["--extra-args-for-sdist-building"], "filename": "any-name.tar.gz", "python_interpreter_target": "unit_test_interpreter_target", @@ -835,13 +840,8 @@ git_dep @ git+https://git.server/repo/project@deadbeefdeadbeef "urls": ["some-archive/any-name.tar.gz"], }, "pypi_315_direct_without_sha_0_0_1_py3_none_any": { + "config_load": "@pypi//:config.bzl", "dep_template": "@pypi//{name}:{target}", - "experimental_target_platforms": [ - "linux_aarch64", - "linux_x86_64", - "osx_aarch64", - "windows_aarch64", - ], "filename": "direct_without_sha-0.0.1-py3-none-any.whl", "python_interpreter_target": "unit_test_interpreter_target", "requirement": "direct_without_sha==0.0.1", @@ -849,25 +849,22 @@ git_dep @ git+https://git.server/repo/project@deadbeefdeadbeef "urls": ["example-direct.org/direct_without_sha-0.0.1-py3-none-any.whl"], }, "pypi_315_git_dep": { + "config_load": "@pypi//:config.bzl", "dep_template": "@pypi//{name}:{target}", "extra_pip_args": ["--extra-args-for-sdist-building"], "python_interpreter_target": "unit_test_interpreter_target", "requirement": "git_dep @ git+https://git.server/repo/project@deadbeefdeadbeef", }, "pypi_315_pip_fallback": { + "config_load": "@pypi//:config.bzl", "dep_template": "@pypi//{name}:{target}", "extra_pip_args": ["--extra-args-for-sdist-building"], "python_interpreter_target": "unit_test_interpreter_target", "requirement": "pip_fallback==0.0.1", }, "pypi_315_simple_py3_none_any_deadb00f": { + "config_load": "@pypi//:config.bzl", "dep_template": "@pypi//{name}:{target}", - "experimental_target_platforms": [ - "linux_aarch64", - "linux_x86_64", - "osx_aarch64", - "windows_aarch64", - ], "filename": "simple-0.0.1-py3-none-any.whl", "python_interpreter_target": "unit_test_interpreter_target", "requirement": "simple==0.0.1", @@ -875,13 +872,8 @@ git_dep @ git+https://git.server/repo/project@deadbeefdeadbeef "urls": ["example2.org"], }, "pypi_315_some_pkg_py3_none_any_deadbaaf": { + "config_load": "@pypi//:config.bzl", "dep_template": "@pypi//{name}:{target}", - "experimental_target_platforms": [ - "linux_aarch64", - "linux_x86_64", - "osx_aarch64", - "windows_aarch64", - ], "filename": "some_pkg-0.0.1-py3-none-any.whl", "python_interpreter_target": "unit_test_interpreter_target", "requirement": "some_pkg==0.0.1", @@ -889,13 +881,8 @@ git_dep @ git+https://git.server/repo/project@deadbeefdeadbeef "urls": ["example-direct.org/some_pkg-0.0.1-py3-none-any.whl"], }, "pypi_315_some_py3_none_any_deadb33f": { + "config_load": "@pypi//:config.bzl", "dep_template": "@pypi//{name}:{target}", - "experimental_target_platforms": [ - "linux_aarch64", - "linux_x86_64", - "osx_aarch64", - "windows_aarch64", - ], "filename": "some-other-pkg-0.0.1-py3-none-any.whl", "python_interpreter_target": "unit_test_interpreter_target", "requirement": "some_other_pkg==0.0.1", @@ -978,11 +965,13 @@ optimum[onnxruntime-gpu]==1.17.1 ; sys_platform == 'linux' }) pypi.whl_libraries().contains_exactly({ "pypi_315_optimum_linux_aarch64_linux_x86_64_linux_x86_64_freethreaded": { + "config_load": "@pypi//:config.bzl", "dep_template": "@pypi//{name}:{target}", "python_interpreter_target": "unit_test_interpreter_target", "requirement": "optimum[onnxruntime-gpu]==1.17.1", }, "pypi_315_optimum_osx_aarch64": { + "config_load": "@pypi//:config.bzl", "dep_template": "@pypi//{name}:{target}", "python_interpreter_target": "unit_test_interpreter_target", "requirement": "optimum[onnxruntime]==1.17.1", @@ -1059,11 +1048,13 @@ optimum[onnxruntime-gpu]==1.17.1 ; sys_platform == 'linux' }) pypi.whl_libraries().contains_exactly({ "pypi_315_optimum_mylinuxx86_64": { + "config_load": "@pypi//:config.bzl", "dep_template": "@pypi//{name}:{target}", "python_interpreter_target": "unit_test_interpreter_target", "requirement": "optimum[onnxruntime-gpu]==1.17.1", }, "pypi_315_optimum_myosxaarch64": { + "config_load": "@pypi//:config.bzl", "dep_template": "@pypi//{name}:{target}", "python_interpreter_target": "unit_test_interpreter_target", "requirement": "optimum[onnxruntime]==1.17.1", diff --git a/tests/pypi/whl_library_targets/whl_library_targets_tests.bzl b/tests/pypi/whl_library_targets/whl_library_targets_tests.bzl index 615358f35d..b7fb9094d7 100644 --- a/tests/pypi/whl_library_targets/whl_library_targets_tests.bzl +++ b/tests/pypi/whl_library_targets/whl_library_targets_tests.bzl @@ -420,8 +420,8 @@ def _test_group(env): ) env.expect.that_collection(alias_calls).contains_exactly([ - {"name": "pkg", "actual": "@pypi__groups//:qux_pkg", "visibility": ["//visibility:public"]}, - {"name": "whl", "actual": "@pypi__groups//:qux_whl", "visibility": ["//visibility:public"]}, + {"name": "pkg", "actual": "@pypi__config//_groups:qux_pkg", "visibility": ["//visibility:public"]}, + {"name": "whl", "actual": "@pypi__config//_groups:qux_whl", "visibility": ["//visibility:public"]}, ]) # buildifier: @unsorted-dict-items env.expect.that_collection(py_library_calls).has_size(1) @@ -446,7 +446,7 @@ def _test_group(env): "//conditions:default": [], }), "tags": [], - "visibility": ["@pypi__groups//:__pkg__"], + "visibility": ["@pypi__config//_groups:__pkg__"], "experimental_venvs_site_packages": Label("//python/config_settings:venvs_site_packages"), }) # buildifier: @unsorted-dict-items diff --git a/tests/whl_with_build_files/testdata/somepkg-1.0.dist-info/METADATA b/tests/whl_with_build_files/testdata/somepkg-1.0.dist-info/METADATA index e69de29bb2..0829d1f756 100644 --- a/tests/whl_with_build_files/testdata/somepkg-1.0.dist-info/METADATA +++ b/tests/whl_with_build_files/testdata/somepkg-1.0.dist-info/METADATA @@ -0,0 +1,2 @@ +Name: somepkg +Version: 1.0 From 3076f90a7dcce55eff4f63b1969d33f55a6be37e Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Mon, 6 Oct 2025 16:10:41 -0700 Subject: [PATCH 474/922] build(deps): bump certifi from 2025.8.3 to 2025.10.5 in /tools/publish (#3328) Bumps [certifi](https://github.com/certifi/python-certifi) from 2025.8.3 to 2025.10.5.
Commits
  • fb14ac4 2025.10.05 (#371)
  • 2c7c7ee Add Python 3.14 classifier in setup.py
  • 1a5cb7b Bump actions/setup-python from 5.6.0 to 6.0.0 (#367)
  • dea5960 Bump pypa/gh-action-pypi-publish from 1.12.4 to 1.13.0 (#366)
  • 83566b7 Bump actions/checkout from 4.2.2 to 5.0.0
  • ca2e121 Bump actions/download-artifact from 4.3.0 to 5.0.0
  • See full diff in compare view

[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=certifi&package-manager=pip&previous-version=2025.8.3&new-version=2025.10.5)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) ---
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot show ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> --- tools/publish/requirements_darwin.txt | 6 +++--- tools/publish/requirements_linux.txt | 6 +++--- tools/publish/requirements_universal.txt | 6 +++--- tools/publish/requirements_windows.txt | 6 +++--- 4 files changed, 12 insertions(+), 12 deletions(-) diff --git a/tools/publish/requirements_darwin.txt b/tools/publish/requirements_darwin.txt index f24f3dc655..b746d6362e 100644 --- a/tools/publish/requirements_darwin.txt +++ b/tools/publish/requirements_darwin.txt @@ -6,9 +6,9 @@ backports-tarfile==1.2.0 \ --hash=sha256:77e284d754527b01fb1e6fa8a1afe577858ebe4e9dad8919e34c862cb399bc34 \ --hash=sha256:d75e02c268746e1b8144c278978b6e98e85de6ad16f8e4b0844a154557eca991 # via jaraco-context -certifi==2025.8.3 \ - --hash=sha256:e564105f78ded564e3ae7c923924435e1daa7463faeab5bb932bc53ffae63407 \ - --hash=sha256:f6c12493cfb1b06ba2ff328595af9350c65d6644968e5d3a2ffd78699af217a5 +certifi==2025.10.5 \ + --hash=sha256:0f212c2744a9bb6de0c56639a6f68afe01ecd92d91f14ae897c4fe7bbeeef0de \ + --hash=sha256:47c09d31ccf2acf0be3f701ea53595ee7e0b8fa08801c6624be771df09ae7b43 # via requests charset-normalizer==3.4.3 \ --hash=sha256:00237675befef519d9af72169d8604a067d92755e84fe76492fef5441db05b91 \ diff --git a/tools/publish/requirements_linux.txt b/tools/publish/requirements_linux.txt index 76426fa019..bc8e396fdb 100644 --- a/tools/publish/requirements_linux.txt +++ b/tools/publish/requirements_linux.txt @@ -6,9 +6,9 @@ backports-tarfile==1.2.0 \ --hash=sha256:77e284d754527b01fb1e6fa8a1afe577858ebe4e9dad8919e34c862cb399bc34 \ --hash=sha256:d75e02c268746e1b8144c278978b6e98e85de6ad16f8e4b0844a154557eca991 # via jaraco-context -certifi==2025.8.3 \ - --hash=sha256:e564105f78ded564e3ae7c923924435e1daa7463faeab5bb932bc53ffae63407 \ - --hash=sha256:f6c12493cfb1b06ba2ff328595af9350c65d6644968e5d3a2ffd78699af217a5 +certifi==2025.10.5 \ + --hash=sha256:0f212c2744a9bb6de0c56639a6f68afe01ecd92d91f14ae897c4fe7bbeeef0de \ + --hash=sha256:47c09d31ccf2acf0be3f701ea53595ee7e0b8fa08801c6624be771df09ae7b43 # via requests cffi==2.0.0 \ --hash=sha256:00bdf7acc5f795150faa6957054fbbca2439db2f775ce831222b66f192f03beb \ diff --git a/tools/publish/requirements_universal.txt b/tools/publish/requirements_universal.txt index 21b3288461..955b4cd276 100644 --- a/tools/publish/requirements_universal.txt +++ b/tools/publish/requirements_universal.txt @@ -6,9 +6,9 @@ backports-tarfile==1.2.0 ; python_full_version < '3.12' \ --hash=sha256:77e284d754527b01fb1e6fa8a1afe577858ebe4e9dad8919e34c862cb399bc34 \ --hash=sha256:d75e02c268746e1b8144c278978b6e98e85de6ad16f8e4b0844a154557eca991 # via jaraco-context -certifi==2025.8.3 \ - --hash=sha256:e564105f78ded564e3ae7c923924435e1daa7463faeab5bb932bc53ffae63407 \ - --hash=sha256:f6c12493cfb1b06ba2ff328595af9350c65d6644968e5d3a2ffd78699af217a5 +certifi==2025.10.5 \ + --hash=sha256:0f212c2744a9bb6de0c56639a6f68afe01ecd92d91f14ae897c4fe7bbeeef0de \ + --hash=sha256:47c09d31ccf2acf0be3f701ea53595ee7e0b8fa08801c6624be771df09ae7b43 # via requests cffi==2.0.0 ; platform_python_implementation != 'PyPy' and sys_platform == 'linux' \ --hash=sha256:045d61c734659cc045141be4bae381a41d89b741f795af1dd018bfb532fd0df8 \ diff --git a/tools/publish/requirements_windows.txt b/tools/publish/requirements_windows.txt index e21cb32fd3..6283dda046 100644 --- a/tools/publish/requirements_windows.txt +++ b/tools/publish/requirements_windows.txt @@ -6,9 +6,9 @@ backports-tarfile==1.2.0 \ --hash=sha256:77e284d754527b01fb1e6fa8a1afe577858ebe4e9dad8919e34c862cb399bc34 \ --hash=sha256:d75e02c268746e1b8144c278978b6e98e85de6ad16f8e4b0844a154557eca991 # via jaraco-context -certifi==2025.8.3 \ - --hash=sha256:e564105f78ded564e3ae7c923924435e1daa7463faeab5bb932bc53ffae63407 \ - --hash=sha256:f6c12493cfb1b06ba2ff328595af9350c65d6644968e5d3a2ffd78699af217a5 +certifi==2025.10.5 \ + --hash=sha256:0f212c2744a9bb6de0c56639a6f68afe01ecd92d91f14ae897c4fe7bbeeef0de \ + --hash=sha256:47c09d31ccf2acf0be3f701ea53595ee7e0b8fa08801c6624be771df09ae7b43 # via requests charset-normalizer==3.4.3 \ --hash=sha256:00237675befef519d9af72169d8604a067d92755e84fe76492fef5441db05b91 \ From 42dfe57ec85b0ac6dace2f7e797618e5b0911f90 Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Mon, 6 Oct 2025 16:12:57 -0700 Subject: [PATCH 475/922] build(deps): bump markdown-it-py from 3.0.0 to 4.0.0 in /tools/publish (#3329) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Bumps [markdown-it-py](https://github.com/executablebooks/markdown-it-py) from 3.0.0 to 4.0.0.
Release notes

Sourced from markdown-it-py's releases.

v4.0.0

What's Changed

This primarily drops support for Python 3.8 and 3.9, adds support for Python 3.13, and updates the parser to comply with Commonmark 0.31.2 and Markdown-It v14.1.0.

Upgrades

Improvements

Bug fixes

Maintenance

Documentation

... (truncated)

Changelog

Sourced from markdown-it-py's changelog.

4.0.0 - 2024-08-10

This primarily drops support for Python 3.9, adds support for Python 3.13, and updates the parser to comply with Commonmark 0.31.2 and Markdown-It v14.1.0.

  • ⬆️ Drop support for Python 3.9 in #360
  • ⬆️ Comply with Commonmark 0.31.2 in #362
  • 👌 Improve performance of "text" inline rule in #347
  • 👌 Use str.removesuffix in #348
  • 👌 limit the number of autocompleted cells in a table in #364
  • 👌 fix quadratic complexity in reference parser in #367
  • 🐛 Fix emphasis inside raw links bugs in #320

Full Changelog: https://github.com/executablebooks/markdown-it-py/compare/v3.0.0...v4.0.0

Commits

[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=markdown-it-py&package-manager=pip&previous-version=3.0.0&new-version=4.0.0)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) ---
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot show ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> --- tools/publish/requirements_darwin.txt | 6 +++--- tools/publish/requirements_linux.txt | 6 +++--- tools/publish/requirements_universal.txt | 6 +++--- tools/publish/requirements_windows.txt | 6 +++--- 4 files changed, 12 insertions(+), 12 deletions(-) diff --git a/tools/publish/requirements_darwin.txt b/tools/publish/requirements_darwin.txt index b746d6362e..5cccf1607f 100644 --- a/tools/publish/requirements_darwin.txt +++ b/tools/publish/requirements_darwin.txt @@ -121,9 +121,9 @@ keyring==25.6.0 \ --hash=sha256:0b39998aa941431eb3d9b0d4b2460bc773b9df6fed7621c2dfb291a7e0187a66 \ --hash=sha256:552a3f7af126ece7ed5c89753650eec89c7eaae8617d0aa4d9ad2b75111266bd # via twine -markdown-it-py==3.0.0 \ - --hash=sha256:355216845c60bd96232cd8d8c40e8f9765cc86f46880e43a8fd22dc1a1a8cab1 \ - --hash=sha256:e3f60a94fa066dc52ec76661e37c851cb232d92f9886b15cb560aaada2df8feb +markdown-it-py==4.0.0 \ + --hash=sha256:87327c59b172c5011896038353a81343b6754500a08cd7a4973bb48c6d578147 \ + --hash=sha256:cb0a2b4aa34f932c007117b194e945bd74e0ec24133ceb5bac59009cda1cb9f3 # via rich mdurl==0.1.2 \ --hash=sha256:84008a41e51615a49fc9966191ff91509e3c40b939176e643fd50a5c2196b8f8 \ diff --git a/tools/publish/requirements_linux.txt b/tools/publish/requirements_linux.txt index bc8e396fdb..4adc3339bf 100644 --- a/tools/publish/requirements_linux.txt +++ b/tools/publish/requirements_linux.txt @@ -252,9 +252,9 @@ keyring==25.6.0 \ --hash=sha256:0b39998aa941431eb3d9b0d4b2460bc773b9df6fed7621c2dfb291a7e0187a66 \ --hash=sha256:552a3f7af126ece7ed5c89753650eec89c7eaae8617d0aa4d9ad2b75111266bd # via twine -markdown-it-py==3.0.0 \ - --hash=sha256:355216845c60bd96232cd8d8c40e8f9765cc86f46880e43a8fd22dc1a1a8cab1 \ - --hash=sha256:e3f60a94fa066dc52ec76661e37c851cb232d92f9886b15cb560aaada2df8feb +markdown-it-py==4.0.0 \ + --hash=sha256:87327c59b172c5011896038353a81343b6754500a08cd7a4973bb48c6d578147 \ + --hash=sha256:cb0a2b4aa34f932c007117b194e945bd74e0ec24133ceb5bac59009cda1cb9f3 # via rich mdurl==0.1.2 \ --hash=sha256:84008a41e51615a49fc9966191ff91509e3c40b939176e643fd50a5c2196b8f8 \ diff --git a/tools/publish/requirements_universal.txt b/tools/publish/requirements_universal.txt index 955b4cd276..fc765f72f7 100644 --- a/tools/publish/requirements_universal.txt +++ b/tools/publish/requirements_universal.txt @@ -235,9 +235,9 @@ keyring==25.6.0 \ --hash=sha256:0b39998aa941431eb3d9b0d4b2460bc773b9df6fed7621c2dfb291a7e0187a66 \ --hash=sha256:552a3f7af126ece7ed5c89753650eec89c7eaae8617d0aa4d9ad2b75111266bd # via twine -markdown-it-py==3.0.0 \ - --hash=sha256:355216845c60bd96232cd8d8c40e8f9765cc86f46880e43a8fd22dc1a1a8cab1 \ - --hash=sha256:e3f60a94fa066dc52ec76661e37c851cb232d92f9886b15cb560aaada2df8feb +markdown-it-py==4.0.0 \ + --hash=sha256:87327c59b172c5011896038353a81343b6754500a08cd7a4973bb48c6d578147 \ + --hash=sha256:cb0a2b4aa34f932c007117b194e945bd74e0ec24133ceb5bac59009cda1cb9f3 # via rich mdurl==0.1.2 \ --hash=sha256:84008a41e51615a49fc9966191ff91509e3c40b939176e643fd50a5c2196b8f8 \ diff --git a/tools/publish/requirements_windows.txt b/tools/publish/requirements_windows.txt index 6283dda046..f18a51e6f1 100644 --- a/tools/publish/requirements_windows.txt +++ b/tools/publish/requirements_windows.txt @@ -121,9 +121,9 @@ keyring==25.6.0 \ --hash=sha256:0b39998aa941431eb3d9b0d4b2460bc773b9df6fed7621c2dfb291a7e0187a66 \ --hash=sha256:552a3f7af126ece7ed5c89753650eec89c7eaae8617d0aa4d9ad2b75111266bd # via twine -markdown-it-py==3.0.0 \ - --hash=sha256:355216845c60bd96232cd8d8c40e8f9765cc86f46880e43a8fd22dc1a1a8cab1 \ - --hash=sha256:e3f60a94fa066dc52ec76661e37c851cb232d92f9886b15cb560aaada2df8feb +markdown-it-py==4.0.0 \ + --hash=sha256:87327c59b172c5011896038353a81343b6754500a08cd7a4973bb48c6d578147 \ + --hash=sha256:cb0a2b4aa34f932c007117b194e945bd74e0ec24133ceb5bac59009cda1cb9f3 # via rich mdurl==0.1.2 \ --hash=sha256:84008a41e51615a49fc9966191ff91509e3c40b939176e643fd50a5c2196b8f8 \ From 37c629bfc12e2f9c282fe3bc6ef2195807e56688 Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Mon, 6 Oct 2025 16:13:16 -0700 Subject: [PATCH 476/922] build(deps): bump certifi from 2025.8.3 to 2025.10.5 in /docs (#3327) Bumps [certifi](https://github.com/certifi/python-certifi) from 2025.8.3 to 2025.10.5.
Commits
  • fb14ac4 2025.10.05 (#371)
  • 2c7c7ee Add Python 3.14 classifier in setup.py
  • 1a5cb7b Bump actions/setup-python from 5.6.0 to 6.0.0 (#367)
  • dea5960 Bump pypa/gh-action-pypi-publish from 1.12.4 to 1.13.0 (#366)
  • 83566b7 Bump actions/checkout from 4.2.2 to 5.0.0
  • ca2e121 Bump actions/download-artifact from 4.3.0 to 5.0.0
  • See full diff in compare view

[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=certifi&package-manager=pip&previous-version=2025.8.3&new-version=2025.10.5)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) ---
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot show ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> --- docs/requirements.txt | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/requirements.txt b/docs/requirements.txt index c40b2960b5..4270d7808d 100644 --- a/docs/requirements.txt +++ b/docs/requirements.txt @@ -22,9 +22,9 @@ babel==2.17.0 \ --hash=sha256:0c54cffb19f690cdcc52a3b50bcbf71e07a808d1c80d549f2459b9d2cf0afb9d \ --hash=sha256:4d0b53093fdfb4b21c92b5213dba5a1b23885afa8383709427046b21c366e5f2 # via sphinx -certifi==2025.8.3 \ - --hash=sha256:e564105f78ded564e3ae7c923924435e1daa7463faeab5bb932bc53ffae63407 \ - --hash=sha256:f6c12493cfb1b06ba2ff328595af9350c65d6644968e5d3a2ffd78699af217a5 +certifi==2025.10.5 \ + --hash=sha256:0f212c2744a9bb6de0c56639a6f68afe01ecd92d91f14ae897c4fe7bbeeef0de \ + --hash=sha256:47c09d31ccf2acf0be3f701ea53595ee7e0b8fa08801c6624be771df09ae7b43 # via requests charset-normalizer==3.4.3 \ --hash=sha256:00237675befef519d9af72169d8604a067d92755e84fe76492fef5441db05b91 \ From 83cea7772914db849a0159e775134b75f4d9fd87 Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Mon, 6 Oct 2025 16:13:45 -0700 Subject: [PATCH 477/922] build(deps): bump markdown-it-py from 3.0.0 to 4.0.0 in /docs (#3326) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Bumps [markdown-it-py](https://github.com/executablebooks/markdown-it-py) from 3.0.0 to 4.0.0.
Release notes

Sourced from markdown-it-py's releases.

v4.0.0

What's Changed

This primarily drops support for Python 3.8 and 3.9, adds support for Python 3.13, and updates the parser to comply with Commonmark 0.31.2 and Markdown-It v14.1.0.

Upgrades

Improvements

Bug fixes

Maintenance

Documentation

... (truncated)

Changelog

Sourced from markdown-it-py's changelog.

4.0.0 - 2024-08-10

This primarily drops support for Python 3.9, adds support for Python 3.13, and updates the parser to comply with Commonmark 0.31.2 and Markdown-It v14.1.0.

  • ⬆️ Drop support for Python 3.9 in #360
  • ⬆️ Comply with Commonmark 0.31.2 in #362
  • 👌 Improve performance of "text" inline rule in #347
  • 👌 Use str.removesuffix in #348
  • 👌 limit the number of autocompleted cells in a table in #364
  • 👌 fix quadratic complexity in reference parser in #367
  • 🐛 Fix emphasis inside raw links bugs in #320

Full Changelog: https://github.com/executablebooks/markdown-it-py/compare/v3.0.0...v4.0.0

Commits

[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=markdown-it-py&package-manager=pip&previous-version=3.0.0&new-version=4.0.0)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) ---
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot show ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> --- docs/requirements.txt | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/requirements.txt b/docs/requirements.txt index 4270d7808d..290113c1b9 100644 --- a/docs/requirements.txt +++ b/docs/requirements.txt @@ -137,9 +137,9 @@ jinja2==3.1.6 \ # myst-parser # readthedocs-sphinx-ext # sphinx -markdown-it-py==3.0.0 \ - --hash=sha256:355216845c60bd96232cd8d8c40e8f9765cc86f46880e43a8fd22dc1a1a8cab1 \ - --hash=sha256:e3f60a94fa066dc52ec76661e37c851cb232d92f9886b15cb560aaada2df8feb +markdown-it-py==4.0.0 \ + --hash=sha256:87327c59b172c5011896038353a81343b6754500a08cd7a4973bb48c6d578147 \ + --hash=sha256:cb0a2b4aa34f932c007117b194e945bd74e0ec24133ceb5bac59009cda1cb9f3 # via # mdit-py-plugins # myst-parser From edd5b1eb41efe8d86a4f3fdb73f3456ed5873a55 Mon Sep 17 00:00:00 2001 From: Ignas Anikevicius <240938+aignas@users.noreply.github.com> Date: Tue, 7 Oct 2025 08:16:05 +0900 Subject: [PATCH 478/922] docs(pipstar): add more docs to make it ready for release (#3323) Followup to #3225 to add more documentation. --- CHANGELOG.md | 3 ++- docs/environment-variables.md | 3 +++ docs/pypi/download.md | 5 ----- 3 files changed, 5 insertions(+), 6 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index cc59e387ee..002a57db2a 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -81,7 +81,8 @@ END_UNRELEASED_TEMPLATE With this release we are deprecating {obj}`pip.parse.experimental_target_platforms` and {obj}`pip_repository.experimental_target_platforms`. For users using `WORKSPACE` and vendoring the `requirements.bzl` file, please re-vendor so that downstream is unaffected - when the APIs get removed. + when the APIs get removed. If you need to customize the way the dependencies get + evaluated, see [our docs](/pypi/download.html#customizing-requires-dist-resolution) on customizing `Requires-Dist` resolution. {#v0-0-0-fixed} ### Fixed diff --git a/docs/environment-variables.md b/docs/environment-variables.md index 4913e329e4..f0cf777a56 100644 --- a/docs/environment-variables.md +++ b/docs/environment-variables.md @@ -71,6 +71,9 @@ instead of the legacy Python scripts. :::{versionadded} 1.5.0 ::: +:::{versionchanged} VERSION_NEXT_FEATURE +Flipped to be enabled by default. +::: :::: ::::{envvar} RULES_PYTHON_EXTRACT_ROOT diff --git a/docs/pypi/download.md b/docs/pypi/download.md index 7f4e205d84..c40f2d4347 100644 --- a/docs/pypi/download.md +++ b/docs/pypi/download.md @@ -168,11 +168,6 @@ available on the PyPI index that you use. ### Customizing `Requires-Dist` resolution -:::{note} -Currently this is disabled by default, but you can turn it on using -{envvar}`RULES_PYTHON_ENABLE_PIPSTAR` environment variable. -::: - In order to understand what dependencies to pull for a particular package, `rules_python` parses the `whl` file [`METADATA`][metadata]. Packages can express dependencies via `Requires-Dist`, and they can add conditions using From ec1df016325f1ac5a949aa7db90b056f558eb6c4 Mon Sep 17 00:00:00 2001 From: Alex Trotta <44127594+Ahajha@users.noreply.github.com> Date: Tue, 7 Oct 2025 14:34:27 -0400 Subject: [PATCH 479/922] feat(toolchains): Add 3.14.0 (#3330) Per the title: 3.14.0 was released today. --- CHANGELOG.md | 1 + MODULE.bazel | 1 + python/versions.bzl | 44 +++++++++++++++++------------------ tests/python/python_tests.bzl | 2 +- 4 files changed, 25 insertions(+), 23 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 002a57db2a..281653aa5b 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -83,6 +83,7 @@ END_UNRELEASED_TEMPLATE vendoring the `requirements.bzl` file, please re-vendor so that downstream is unaffected when the APIs get removed. If you need to customize the way the dependencies get evaluated, see [our docs](/pypi/download.html#customizing-requires-dist-resolution) on customizing `Requires-Dist` resolution. +* (toolchains) Added Python version 3.14.0. {#v0-0-0-fixed} ### Fixed diff --git a/MODULE.bazel b/MODULE.bazel index 481e13e39f..4ad6758455 100644 --- a/MODULE.bazel +++ b/MODULE.bazel @@ -292,6 +292,7 @@ dev_pip = use_extension( "3.11", "3.12", "3.13", + "3.14", ] ] diff --git a/python/versions.bzl b/python/versions.bzl index 1b20290b17..f43e6b6dba 100644 --- a/python/versions.bzl +++ b/python/versions.bzl @@ -810,28 +810,28 @@ TOOL_VERSIONS = { "x86_64-unknown-linux-gnu-freethreaded": "python/install", }, }, - "3.14.0rc1": { - "url": "20250808/cpython-{python_version}+20250808-{platform}-{build}.{ext}", + "3.14.0": { + "url": "20251007/cpython-{python_version}+20251007-{platform}-{build}.{ext}", "sha256": { - "aarch64-apple-darwin": "016b9eb7c6c41d358a095f52203297812a566376b1e4372571b850f621dc720d", - "aarch64-unknown-linux-gnu": "bfa5cb2f56032f4ed2c105f5b3b59ea1809672cb74b453e4450399517a594137", - "ppc64le-unknown-linux-gnu": "eb8fade967732032be70d5129ed66ad28dfe57e2964550f0f6800dddc1fdcb80", - "riscv64-unknown-linux-gnu": "82313ee3c45ad0dc825044fef161840dd60277003bf5458da24d46206fff1e09", - "s390x-unknown-linux-gnu": "5af30600105c42e920e9709afc8deae07c309cd46959c22dc099a1fb45b73902", - "x86_64-apple-darwin": "a74da55830354eb13c5e1fd12bb9b0b624ed0daeafec48444eda86b21476e4c8", - "x86_64-pc-windows-msvc": "71d7fe086604835e5ebd38b45829c1e7ce38fb8f5399d287d219f930cd6efdc1", - "aarch64-pc-windows-msvc": "631c007e1b90dfc9f22d05d4f4e5ef9f70bfa01b675a2bd8590994369746f852", - "x86_64-unknown-linux-gnu": "644028c49cdd9d082274f7265857bc5b5bb4eea8c3e58187e5b3ebb74de9ad3a", - "x86_64-unknown-linux-musl": "af82c7e3ddaebbfb10967c5dd9751bc2fa7f735806a16ebde7600e519c34e587", - "aarch64-apple-darwin-freethreaded": "d611134bcf090db920d8192ec26e899433cdbae42476fd92ee07cc13b5e32d1f", - "aarch64-unknown-linux-gnu-freethreaded": "9fcbb8947c07421506187b3f605f34e94c68824d3fd362d84cd1dcdf13285ee3", - "ppc64le-unknown-linux-gnu-freethreaded": "8cb6937fb0804ca0d5d867af15b6e7fd05d79d0faa5a6e617e6f6280580a5f66", - "riscv64-unknown-linux-gnu-freethreaded": "7a235a6f5b814f5ae789fea65b097ad52f840619dfd054135d8ae8443fa8e362", - "s390x-unknown-linux-gnu-freethreaded": "fee35437df67782b348d57b32cb1acac61384504af28860d552b0d6aeb3ae19e", - "x86_64-apple-darwin-freethreaded": "07deb66e52c91c69e15a3d644ff1527dbd9e278c389ed58341af10872dc15ab5", - "x86_64-pc-windows-msvc-freethreaded": "7a4cdb4c213a2f486b5d6b1044970f6529b7e5365a2d5503ffa8224e62da9ccf", - "aarch64-pc-windows-msvc-freethreaded": "aa9f871afc67419e867535eb0d368e5ec7828138799985d21448443ff1185087", - "x86_64-unknown-linux-gnu-freethreaded": "b6237adf6cf3b8ae00238936d61045d33a6b45147e6540a91ae6e6696aeff23c", + "aarch64-apple-darwin": "41c502cf32d650673bfbee35f73c9140897dd26c43b97da1177cee00f40033fb", + "aarch64-unknown-linux-gnu": "7b4fc36ee88ec693fcf7ac696bc018a8254a1f166f4cd5f6a352d5432cb5836a", + "ppc64le-unknown-linux-gnu": "e5df0738e3f7da9977d6b789fad0b3e8ccc117a3337bf6d4de673cd6472239c8", + "riscv64-unknown-linux-gnu": "cfff02bd9b3d6c64e2eacf725557599ce17f65e30776f41c0643613cbcf2042e", + "s390x-unknown-linux-gnu": "e2cbe581954685ae0a77206c8318c351e3a9d99b28924e3527610e76487c6201", + "x86_64-apple-darwin": "543accfe71df014a08295a4bbaa4e4cf2b80ab2977ec362e38be24c36076d7fe", + "x86_64-pc-windows-msvc": "77cd2c0e167726e0476e35c7e483cf2f05172dff2326e1c4bf9887aff8353b2f", + "aarch64-pc-windows-msvc": "52434459d376f3fc272596d7b5f97b2248e51362a6157091f9d64e630ddd8fdd", + "x86_64-unknown-linux-gnu": "8203b9355b605ad80be6f1aa467226cfbd55b9839063c173c494de5e69c4a722", + "x86_64-unknown-linux-musl": "6a0350e642dddc6c54f568c08239ca7af08cf8621d5797afc6a0df7c40b8eb7b", + "aarch64-apple-darwin-freethreaded": "72475196f0092d29bcd2fca298fe198cad135762118e8470083789a3e86cc30f", + "aarch64-unknown-linux-gnu-freethreaded": "c9f4550cdfe4d72c526a3aead8ff1f63a6f0e46cde3d64093177fa1b1944b662", + "ppc64le-unknown-linux-gnu-freethreaded": "2b39b7074a26d44f98275bfa6ea4128e691cc02409edc830dc1b8c19da38ec0f", + "riscv64-unknown-linux-gnu-freethreaded": "16a91fcf2b434c0ba48580aeccf61dfe682efbae5c05b21d0a7780b2cf20cd01", + "s390x-unknown-linux-gnu-freethreaded": "2e42043598543ccf92a5e58f55083ed12156f71cbfe4b2698d4f66dbe3864530", + "x86_64-apple-darwin-freethreaded": "7afbad6cc08072268ad9286dc16be5a04add68af2e3fbef69a429f0a223c275d", + "x86_64-pc-windows-msvc-freethreaded": "7ce62b9445d6d8a8518963e43eb655f5b9f7d08d084d7efc7164b1212fe13d16", + "aarch64-pc-windows-msvc-freethreaded": "87a9c334d1b591ad8561e74d70208eee4b86e23215af031ad7b445a694a45326", + "x86_64-unknown-linux-gnu-freethreaded": "254b71ac6c8557165d88fca355ca8861e303c726bd4ce100eead45d7fb59fb8a", }, "strip_prefix": { "aarch64-apple-darwin": "python", @@ -865,7 +865,7 @@ MINOR_MAPPING = { "3.11": "3.11.13", "3.12": "3.12.11", "3.13": "3.13.6", - "3.14": "3.14.0rc1", + "3.14": "3.14.0", } def _generate_platforms(): diff --git a/tests/python/python_tests.bzl b/tests/python/python_tests.bzl index 96d78d13df..da48981e18 100644 --- a/tests/python/python_tests.bzl +++ b/tests/python/python_tests.bzl @@ -321,7 +321,7 @@ def _test_toolchain_ordering(env): "3.11": "3.11.13", "3.12": "3.12.11", "3.13": "3.13.6", - "3.14": "3.14.0rc1", + "3.14": "3.14.0", "3.8": "3.8.20", "3.9": "3.9.23", }) From 33fd322c7dec57fdff24311e655e03f274f8b2a5 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Thu, 9 Oct 2025 17:19:37 -0700 Subject: [PATCH 480/922] fix(venv): group venv prefixes by path component, not raw path (#3333) When files overlap between packages, and dist-info directories are present, it results in a prefix list like `foo foo-bar foo/bar`. When sorted as raw strings, hyphen sorts before slash, so the continuity of path prefixes is violated and they are grouped separately. An error then occurs because both `foo/` and `foo/bar` are created, but the latter is a sub-path of the former. To fix, change the sort key to a tuple of path components. This makes `foo foo-bar foo/bar` sort as `(foo,) (foo, bar), (foo-bar, )`, resulting in the correct order. Fixes https://github.com/bazel-contrib/rules_python/issues/3204 --- python/private/venv_runfiles.bzl | 4 +++- .../app_files_building/app_files_building_tests.bzl | 2 ++ 2 files changed, 5 insertions(+), 1 deletion(-) diff --git a/python/private/venv_runfiles.bzl b/python/private/venv_runfiles.bzl index 291920b848..9fbe97a52e 100644 --- a/python/private/venv_runfiles.bzl +++ b/python/private/venv_runfiles.bzl @@ -138,7 +138,9 @@ def _group_venv_path_entries(entries): """ # Sort so order is top-down, ensuring grouping by short common prefix - entries = sorted(entries, key = lambda e: e.venv_path) + # Split it into path components so `foo foo-bar foo/bar` sorts as + # `foo foo/bar foo-bar` + entries = sorted(entries, key = lambda e: tuple(e.venv_path.split("/"))) groups = [] current_group = None diff --git a/tests/venv_site_packages_libs/app_files_building/app_files_building_tests.bzl b/tests/venv_site_packages_libs/app_files_building/app_files_building_tests.bzl index 0a0265eb8c..68e17160e7 100644 --- a/tests/venv_site_packages_libs/app_files_building/app_files_building_tests.bzl +++ b/tests/venv_site_packages_libs/app_files_building/app_files_building_tests.bzl @@ -61,6 +61,7 @@ _tests.append(_test_conflict_merging) def _test_conflict_merging_impl(env, _): entries = [ _entry("a", "+pypi_a/site-packages/a", ["a.txt"]), + _entry("a-1.0.dist-info", "+pypi_a/site-packages/a-1.0.dist-info", ["METADATA"]), _entry("a/b", "+pypi_a_b/site-packages/a/b", ["b.txt"]), _entry("x", "_main/src/x", ["x.txt"]), _entry("x/p", "_main/src-dev/x/p", ["p.txt"]), @@ -72,6 +73,7 @@ def _test_conflict_merging_impl(env, _): actual = build_link_map(_ctx(), entries) expected_libs = { + "a-1.0.dist-info": "+pypi_a/site-packages/a-1.0.dist-info", "a/a.txt": _file("../+pypi_a/site-packages/a/a.txt"), "a/b/b.txt": _file("../+pypi_a_b/site-packages/a/b/b.txt"), "duplicate/d.py": _file("../+dupe_a/site-packages/duplicate/d.py"), From 5c13539708a80560729424e341603ac59d6ecece Mon Sep 17 00:00:00 2001 From: Ignas Anikevicius <240938+aignas@users.noreply.github.com> Date: Fri, 10 Oct 2025 09:28:26 +0900 Subject: [PATCH 481/922] feat(pypi): support aarch64 windows on pipstar (#3226) This adds the necessary bits for Windows ARM64 support when evaluating env markers in requirements files and selecting the right wheels in `experimental_index_url`. Related #2276 --- CHANGELOG.md | 3 + MODULE.bazel | 29 + examples/BUILD.bazel | 25 + examples/bzlmod/MODULE.bazel | 43 +- examples/bzlmod/requirements_lock_3_11.txt | 530 ++++++++++++++++++ examples/bzlmod/requirements_windows_3_11.txt | 530 ++++++++++++++++++ 6 files changed, 1138 insertions(+), 22 deletions(-) create mode 100644 examples/bzlmod/requirements_lock_3_11.txt create mode 100644 examples/bzlmod/requirements_windows_3_11.txt diff --git a/CHANGELOG.md b/CHANGELOG.md index 281653aa5b..a04ed04ffa 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -131,6 +131,9 @@ END_UNRELEASED_TEMPLATE * {obj}`//python:features.bzl%features.headers_abi3` can be used to feature-detect the presense of the above. * (toolchains) Local toolchains can use a label for the interpreter to use. +* (pypi) Support for environment marker handling and `experimental_index_url` handling for + Windows ARM64 for Python 3.11 and later + ([#2276](https://github.com/bazel-contrib/rules_python/issues/2276)). {#v1-6-3} ## [1.6.3] - 2025-09-21 diff --git a/MODULE.bazel b/MODULE.bazel index 4ad6758455..36135bbb8b 100644 --- a/MODULE.bazel +++ b/MODULE.bazel @@ -171,6 +171,35 @@ pip = use_extension("//python/extensions:pip.bzl", "pip") ] ] +[ + pip.default( + arch_name = cpu, + config_settings = [ + "@platforms//cpu:{}".format(cpu), + "@platforms//os:windows", + "//python/config_settings:_is_py_freethreaded_{}".format( + "yes" if freethreaded else "no", + ), + ], + env = {"platform_version": "0"}, + marker = "python_version >= '3.13'" if freethreaded else "python_version >= '3.11'", + os_name = "windows", + platform = "windows_{}{}".format(cpu, freethreaded), + whl_abi_tags = ["cp{major}{minor}t"] if freethreaded else [ + "abi3", + "cp{major}{minor}", + ], + whl_platform_tags = whl_platform_tags, + ) + for cpu, whl_platform_tags in { + "aarch64": ["win_arm64"], + }.items() + for freethreaded in [ + "", + "_freethreaded", + ] +] + pip.parse( hub_name = "rules_python_publish_deps", python_version = "3.11", diff --git a/examples/BUILD.bazel b/examples/BUILD.bazel index d2fddc44c5..716cb9a20a 100644 --- a/examples/BUILD.bazel +++ b/examples/BUILD.bazel @@ -28,3 +28,28 @@ lock( ], python_version = "3.9.19", ) + +lock( + name = "bzlmod_requirements_3_11", + srcs = ["bzlmod/requirements.in"], + out = "bzlmod/requirements_lock_3_11.txt", + args = [ + "--emit-index-url", + "--universal", + "--python-version=3.11", + ], + python_version = "3.11", +) + +lock( + name = "bzlmod_requirements_3_11_windows", + srcs = ["bzlmod/requirements.in"], + out = "bzlmod/requirements_windows_3_11.txt", + args = [ + "--emit-index-url", + "--python-platform", + "windows", + "--python-version=3.11", + ], + python_version = "3.11", +) diff --git a/examples/bzlmod/MODULE.bazel b/examples/bzlmod/MODULE.bazel index 95e1090f53..8bd6718387 100644 --- a/examples/bzlmod/MODULE.bazel +++ b/examples/bzlmod/MODULE.bazel @@ -169,10 +169,13 @@ pip.default( env = { "platform_version": "0", }, + # Windows ARM64 support has been added only on 3.11 and above, hence, constrain + # the availability of the platform for those python versions. + marker = "python_version >= '3.11'", os_name = "windows", platform = "windows_aarch64", whl_abi_tags = [], # default to all ABIs - whl_platform_tags = ["win_amd64"], + whl_platform_tags = ["win_arm64"], ) # To fetch pip dependencies, use pip.parse. We can pass in various options, @@ -206,14 +209,6 @@ pip.parse( "sphinxcontrib-serializinghtml", ], }, - # You can use one of the values below to specify the target platform - # to generate the dependency graph for. - experimental_target_platforms = [ - # Specifying the target platforms explicitly - "cp39_linux_x86_64", - "cp39_linux_*", - "cp39_*", - ], extra_hub_aliases = { "wheel": ["generated_file"], }, @@ -239,30 +234,34 @@ pip.parse( "sphinxcontrib-serializinghtml", ], }, - # You can use one of the values below to specify the target platform - # to generate the dependency graph for. - experimental_target_platforms = [ - # Using host python version - "linux_*", - "osx_*", - "windows_*", - # Or specifying an exact platform - "linux_x86_64", - # Or the following to get the `host` platform only - "host", - ], hub_name = "pip", python_version = "3.10", # The requirements files for each platform that we want to support. requirements_by_platform = { # Default requirements file for needs to explicitly provide the platforms "//:requirements_lock_3_10.txt": "linux_*,osx_*", + "//:requirements_windows_3_10.txt": "windows_x86_64", + }, + # These modifications were created above and we + # are providing pip.parse with the label of the mod + # and the name of the wheel. + whl_modifications = { + "@whl_mods_hub//:requests.json": "requests", + "@whl_mods_hub//:wheel.json": "wheel", + }, +) +pip.parse( + hub_name = "pip", + python_version = "3.11", + requirements_by_platform = { + # Default requirements file for needs to explicitly provide the platforms + "//:requirements_lock_3_11.txt": "linux_*,osx_*", # This API allows one to specify additional platforms that the users # configure the toolchains for themselves. In this example we add # `windows_aarch64` to illustrate that `rules_python` won't fail to # process the value, but it does not mean that this example will work # on Windows ARM. - "//:requirements_windows_3_10.txt": "windows_x86_64,windows_aarch64", + "//:requirements_windows_3_11.txt": "windows_x86_64,windows_aarch64", }, # These modifications were created above and we # are providing pip.parse with the label of the mod diff --git a/examples/bzlmod/requirements_lock_3_11.txt b/examples/bzlmod/requirements_lock_3_11.txt new file mode 100644 index 0000000000..dd6ec4d29e --- /dev/null +++ b/examples/bzlmod/requirements_lock_3_11.txt @@ -0,0 +1,530 @@ +# This file was autogenerated by uv via the following command: +# bazel run //examples:bzlmod_requirements_3_11.update +--index-url https://pypi.org/simple +--extra-index-url https://pypi.org/simple/ + +alabaster==0.7.16 \ + --hash=sha256:75a8b99c28a5dad50dd7f8ccdd447a121ddb3892da9e53d1ca5cca3106d58d65 \ + --hash=sha256:b46733c07dce03ae4e150330b975c75737fa60f0a7c591b6c8bf4928a28e2c92 + # via sphinx +astroid==2.13.5 \ + --hash=sha256:6891f444625b6edb2ac798829b689e95297e100ddf89dbed5a8c610e34901501 \ + --hash=sha256:df164d5ac811b9f44105a72b8f9d5edfb7b5b2d7e979b04ea377a77b3229114a + # via pylint +babel==2.17.0 \ + --hash=sha256:0c54cffb19f690cdcc52a3b50bcbf71e07a808d1c80d549f2459b9d2cf0afb9d \ + --hash=sha256:4d0b53093fdfb4b21c92b5213dba5a1b23885afa8383709427046b21c366e5f2 + # via sphinx +certifi==2025.10.5 \ + --hash=sha256:0f212c2744a9bb6de0c56639a6f68afe01ecd92d91f14ae897c4fe7bbeeef0de \ + --hash=sha256:47c09d31ccf2acf0be3f701ea53595ee7e0b8fa08801c6624be771df09ae7b43 + # via requests +chardet==4.0.0 \ + --hash=sha256:0d6f53a15db4120f2b08c94f11e7d93d2c911ee118b6b30a04ec3ee8310179fa \ + --hash=sha256:f864054d66fd9118f2e67044ac8981a54775ec5b67aed0441892edb553d21da5 + # via requests +colorama==0.4.6 \ + --hash=sha256:08695f5cb7ed6e0531a20572697297273c47b8cae5a63ffc6d6ed5c201be6e44 \ + --hash=sha256:4f1d9991f5acc0ca119f9d443620b77f9d6b33703e51011c16baf57afb285fc6 + # via + # -r examples/bzlmod/requirements.in + # pylint + # sphinx +dill==0.4.0 \ + --hash=sha256:0633f1d2df477324f53a895b02c901fb961bdbf65a17122586ea7019292cbcf0 \ + --hash=sha256:44f54bf6412c2c8464c14e8243eb163690a9800dbe2c367330883b19c7561049 + # via pylint +docutils==0.21.2 \ + --hash=sha256:3a6b18732edf182daa3cd12775bbb338cf5691468f91eeeb109deff6ebfa986f \ + --hash=sha256:dafca5b9e384f0e419294eb4d2ff9fa826435bf15f15b7bd45723e8ad76811b2 + # via sphinx +idna==2.10 \ + --hash=sha256:b307872f855b18632ce0c21c5e45be78c0ea7ae4c15c828c20788b26921eb3f6 \ + --hash=sha256:b97d804b1e9b523befed77c48dacec60e6dcb0b5391d57af6a65a312a90648c0 + # via requests +imagesize==1.4.1 \ + --hash=sha256:0d8d18d08f840c19d0ee7ca1fd82490fdc3729b7ac93f49870406ddde8ef8d8b \ + --hash=sha256:69150444affb9cb0d5cc5a92b3676f0b2fb7cd9ae39e947a5e11a36b4497cd4a + # via sphinx +isort==5.13.2 \ + --hash=sha256:48fdfcb9face5d58a4f6dde2e72a1fb8dcaf8ab26f95ab49fab84c2ddefb0109 \ + --hash=sha256:8ca5e72a8d85860d5a3fa69b8745237f2939afe12dbf656afbcb47fe72d947a6 + # via pylint +jinja2==3.1.6 \ + --hash=sha256:0137fb05990d35f1275a587e9aee6d56da821fc83491a0fb838183be43f66d6d \ + --hash=sha256:85ece4451f492d0c13c5dd7c13a64681a86afae63a5f347908daf103ce6d2f67 + # via sphinx +lazy-object-proxy==1.12.0 \ + --hash=sha256:029d2b355076710505c9545aef5ab3f750d89779310e26ddf2b7b23f6ea03cd8 \ + --hash=sha256:08c465fb5cd23527512f9bd7b4c7ba6cec33e28aad36fbbe46bf7b858f9f3f7f \ + --hash=sha256:0a83c6f7a6b2bfc11ef3ed67f8cbe99f8ff500b05655d8e7df9aab993a6abc95 \ + --hash=sha256:1192e8c2f1031a6ff453ee40213afa01ba765b3dc861302cd91dbdb2e2660b00 \ + --hash=sha256:14e348185adbd03ec17d051e169ec45686dcd840a3779c9d4c10aabe2ca6e1c0 \ + --hash=sha256:15400b18893f345857b9e18b9bd87bd06aba84af6ed086187add70aeaa3f93f1 \ + --hash=sha256:1cf69cd1a6c7fe2dbcc3edaa017cf010f4192e53796538cc7d5e1fedbfa4bcff \ + --hash=sha256:1f5a462d92fd0cfb82f1fab28b51bfb209fabbe6aabf7f0d51472c0c124c0c61 \ + --hash=sha256:256262384ebd2a77b023ad02fbcc9326282bcfd16484d5531154b02bc304f4c5 \ + --hash=sha256:31020c84005d3daa4cc0fa5a310af2066efe6b0d82aeebf9ab199292652ff036 \ + --hash=sha256:338ab2f132276203e404951205fe80c3fd59429b3a724e7b662b2eb539bb1be9 \ + --hash=sha256:3605b632e82a1cbc32a1e5034278a64db555b3496e0795723ee697006b980508 \ + --hash=sha256:3d3964fbd326578bcdfffd017ef101b6fb0484f34e731fe060ba9b8816498c36 \ + --hash=sha256:424a8ab6695400845c39f13c685050eab69fa0bbac5790b201cd27375e5e41d7 \ + --hash=sha256:4a79b909aa16bde8ae606f06e6bbc9d3219d2e57fb3e0076e17879072b742c65 \ + --hash=sha256:4ab2c584e3cc8be0dfca422e05ad30a9abe3555ce63e9ab7a559f62f8dbc6ff9 \ + --hash=sha256:53c7fd99eb156bbb82cbc5d5188891d8fdd805ba6c1e3b92b90092da2a837073 \ + --hash=sha256:563d2ec8e4d4b68ee7848c5ab4d6057a6d703cb7963b342968bb8758dda33a23 \ + --hash=sha256:61d5e3310a4aa5792c2b599a7a78ccf8687292c8eb09cf187cca8f09cf6a7519 \ + --hash=sha256:6763941dbf97eea6b90f5b06eb4da9418cc088fce0e3883f5816090f9afcde4a \ + --hash=sha256:67f07ab742f1adfb3966c40f630baaa7902be4222a17941f3d85fd1dae5565ff \ + --hash=sha256:717484c309df78cedf48396e420fa57fc8a2b1f06ea889df7248fdd156e58847 \ + --hash=sha256:75ba769017b944fcacbf6a80c18b2761a1795b03f8899acdad1f1c39db4409be \ + --hash=sha256:7601ec171c7e8584f8ff3f4e440aa2eebf93e854f04639263875b8c2971f819f \ + --hash=sha256:7b22c2bbfb155706b928ac4d74c1a63ac8552a55ba7fff4445155523ea4067e1 \ + --hash=sha256:800f32b00a47c27446a2b767df7538e6c66a3488632c402b4fb2224f9794f3c0 \ + --hash=sha256:81d1852fb30fab81696f93db1b1e55a5d1ff7940838191062f5f56987d5fcc3e \ + --hash=sha256:86fd61cb2ba249b9f436d789d1356deae69ad3231dc3c0f17293ac535162672e \ + --hash=sha256:8c40b3c9faee2e32bfce0df4ae63f4e73529766893258eca78548bac801c8f66 \ + --hash=sha256:8ee0d6027b760a11cc18281e702c0309dd92da458a74b4c15025d7fc490deede \ + --hash=sha256:997b1d6e10ecc6fb6fe0f2c959791ae59599f41da61d652f6c903d1ee58b7370 \ + --hash=sha256:a61095f5d9d1a743e1e20ec6d6db6c2ca511961777257ebd9b288951b23b44fa \ + --hash=sha256:a6b7ea5ea1ffe15059eb44bcbcb258f97bcb40e139b88152c40d07b1a1dfc9ac \ + --hash=sha256:ae575ad9b674d0029fc077c5231b3bc6b433a3d1a62a8c363df96974b5534728 \ + --hash=sha256:be5fe974e39ceb0d6c9db0663c0464669cf866b2851c73971409b9566e880eab \ + --hash=sha256:be9045646d83f6c2664c1330904b245ae2371b5c57a3195e4028aedc9f999655 \ + --hash=sha256:c1ca33565f698ac1aece152a10f432415d1a2aa9a42dfe23e5ba2bc255ab91f6 \ + --hash=sha256:c3b2e0af1f7f77c4263759c4824316ce458fabe0fceadcd24ef8ca08b2d1e402 \ + --hash=sha256:c4fcbe74fb85df8ba7825fa05eddca764138da752904b378f0ae5ab33a36c308 \ + --hash=sha256:c9defba70ab943f1df98a656247966d7729da2fe9c2d5d85346464bf320820a3 \ + --hash=sha256:cc6e3614eca88b1c8a625fc0a47d0d745e7c3255b21dac0e30b3037c5e3deeb8 \ + --hash=sha256:d01c7819a410f7c255b20799b65d36b414379a30c6f1684c7bd7eb6777338c1b \ + --hash=sha256:efff4375a8c52f55a145dc8487a2108c2140f0bec4151ab4e1843e52eb9987ad \ + --hash=sha256:fdc70d81235fc586b9e3d1aeef7d1553259b62ecaae9db2167a5d2550dcc391a + # via astroid +markupsafe==3.0.3 \ + --hash=sha256:0303439a41979d9e74d18ff5e2dd8c43ed6c6001fd40e5bf2e43f7bd9bbc523f \ + --hash=sha256:068f375c472b3e7acbe2d5318dea141359e6900156b5b2ba06a30b169086b91a \ + --hash=sha256:0bf2a864d67e76e5c9a34dc26ec616a66b9888e25e7b9460e1c76d3293bd9dbf \ + --hash=sha256:0db14f5dafddbb6d9208827849fad01f1a2609380add406671a26386cdf15a19 \ + --hash=sha256:0eb9ff8191e8498cca014656ae6b8d61f39da5f95b488805da4bb029cccbfbaf \ + --hash=sha256:0f4b68347f8c5eab4a13419215bdfd7f8c9b19f2b25520968adfad23eb0ce60c \ + --hash=sha256:1085e7fbddd3be5f89cc898938f42c0b3c711fdcb37d75221de2666af647c175 \ + --hash=sha256:116bb52f642a37c115f517494ea5feb03889e04df47eeff5b130b1808ce7c219 \ + --hash=sha256:12c63dfb4a98206f045aa9563db46507995f7ef6d83b2f68eda65c307c6829eb \ + --hash=sha256:133a43e73a802c5562be9bbcd03d090aa5a1fe899db609c29e8c8d815c5f6de6 \ + --hash=sha256:1353ef0c1b138e1907ae78e2f6c63ff67501122006b0f9abad68fda5f4ffc6ab \ + --hash=sha256:15d939a21d546304880945ca1ecb8a039db6b4dc49b2c5a400387cdae6a62e26 \ + --hash=sha256:177b5253b2834fe3678cb4a5f0059808258584c559193998be2601324fdeafb1 \ + --hash=sha256:1872df69a4de6aead3491198eaf13810b565bdbeec3ae2dc8780f14458ec73ce \ + --hash=sha256:1b4b79e8ebf6b55351f0d91fe80f893b4743f104bff22e90697db1590e47a218 \ + --hash=sha256:1b52b4fb9df4eb9ae465f8d0c228a00624de2334f216f178a995ccdcf82c4634 \ + --hash=sha256:1ba88449deb3de88bd40044603fafffb7bc2b055d626a330323a9ed736661695 \ + --hash=sha256:1cc7ea17a6824959616c525620e387f6dd30fec8cb44f649e31712db02123dad \ + --hash=sha256:218551f6df4868a8d527e3062d0fb968682fe92054e89978594c28e642c43a73 \ + --hash=sha256:26a5784ded40c9e318cfc2bdb30fe164bdb8665ded9cd64d500a34fb42067b1c \ + --hash=sha256:2713baf880df847f2bece4230d4d094280f4e67b1e813eec43b4c0e144a34ffe \ + --hash=sha256:2a15a08b17dd94c53a1da0438822d70ebcd13f8c3a95abe3a9ef9f11a94830aa \ + --hash=sha256:2f981d352f04553a7171b8e44369f2af4055f888dfb147d55e42d29e29e74559 \ + --hash=sha256:32001d6a8fc98c8cb5c947787c5d08b0a50663d139f1305bac5885d98d9b40fa \ + --hash=sha256:3524b778fe5cfb3452a09d31e7b5adefeea8c5be1d43c4f810ba09f2ceb29d37 \ + --hash=sha256:3537e01efc9d4dccdf77221fb1cb3b8e1a38d5428920e0657ce299b20324d758 \ + --hash=sha256:35add3b638a5d900e807944a078b51922212fb3dedb01633a8defc4b01a3c85f \ + --hash=sha256:38664109c14ffc9e7437e86b4dceb442b0096dfe3541d7864d9cbe1da4cf36c8 \ + --hash=sha256:3a7e8ae81ae39e62a41ec302f972ba6ae23a5c5396c8e60113e9066ef893da0d \ + --hash=sha256:3b562dd9e9ea93f13d53989d23a7e775fdfd1066c33494ff43f5418bc8c58a5c \ + --hash=sha256:457a69a9577064c05a97c41f4e65148652db078a3a509039e64d3467b9e7ef97 \ + --hash=sha256:4bd4cd07944443f5a265608cc6aab442e4f74dff8088b0dfc8238647b8f6ae9a \ + --hash=sha256:4e885a3d1efa2eadc93c894a21770e4bc67899e3543680313b09f139e149ab19 \ + --hash=sha256:4faffd047e07c38848ce017e8725090413cd80cbc23d86e55c587bf979e579c9 \ + --hash=sha256:509fa21c6deb7a7a273d629cf5ec029bc209d1a51178615ddf718f5918992ab9 \ + --hash=sha256:5678211cb9333a6468fb8d8be0305520aa073f50d17f089b5b4b477ea6e67fdc \ + --hash=sha256:591ae9f2a647529ca990bc681daebdd52c8791ff06c2bfa05b65163e28102ef2 \ + --hash=sha256:5a7d5dc5140555cf21a6fefbdbf8723f06fcd2f63ef108f2854de715e4422cb4 \ + --hash=sha256:69c0b73548bc525c8cb9a251cddf1931d1db4d2258e9599c28c07ef3580ef354 \ + --hash=sha256:6b5420a1d9450023228968e7e6a9ce57f65d148ab56d2313fcd589eee96a7a50 \ + --hash=sha256:722695808f4b6457b320fdc131280796bdceb04ab50fe1795cd540799ebe1698 \ + --hash=sha256:729586769a26dbceff69f7a7dbbf59ab6572b99d94576a5592625d5b411576b9 \ + --hash=sha256:77f0643abe7495da77fb436f50f8dab76dbc6e5fd25d39589a0f1fe6548bfa2b \ + --hash=sha256:795e7751525cae078558e679d646ae45574b47ed6e7771863fcc079a6171a0fc \ + --hash=sha256:7be7b61bb172e1ed687f1754f8e7484f1c8019780f6f6b0786e76bb01c2ae115 \ + --hash=sha256:7c3fb7d25180895632e5d3148dbdc29ea38ccb7fd210aa27acbd1201a1902c6e \ + --hash=sha256:7e68f88e5b8799aa49c85cd116c932a1ac15caaa3f5db09087854d218359e485 \ + --hash=sha256:83891d0e9fb81a825d9a6d61e3f07550ca70a076484292a70fde82c4b807286f \ + --hash=sha256:8485f406a96febb5140bfeca44a73e3ce5116b2501ac54fe953e488fb1d03b12 \ + --hash=sha256:8709b08f4a89aa7586de0aadc8da56180242ee0ada3999749b183aa23df95025 \ + --hash=sha256:8f71bc33915be5186016f675cd83a1e08523649b0e33efdb898db577ef5bb009 \ + --hash=sha256:915c04ba3851909ce68ccc2b8e2cd691618c4dc4c4232fb7982bca3f41fd8c3d \ + --hash=sha256:949b8d66bc381ee8b007cd945914c721d9aba8e27f71959d750a46f7c282b20b \ + --hash=sha256:94c6f0bb423f739146aec64595853541634bde58b2135f27f61c1ffd1cd4d16a \ + --hash=sha256:9a1abfdc021a164803f4d485104931fb8f8c1efd55bc6b748d2f5774e78b62c5 \ + --hash=sha256:9b79b7a16f7fedff2495d684f2b59b0457c3b493778c9eed31111be64d58279f \ + --hash=sha256:a320721ab5a1aba0a233739394eb907f8c8da5c98c9181d1161e77a0c8e36f2d \ + --hash=sha256:a4afe79fb3de0b7097d81da19090f4df4f8d3a2b3adaa8764138aac2e44f3af1 \ + --hash=sha256:ad2cf8aa28b8c020ab2fc8287b0f823d0a7d8630784c31e9ee5edea20f406287 \ + --hash=sha256:b8512a91625c9b3da6f127803b166b629725e68af71f8184ae7e7d54686a56d6 \ + --hash=sha256:bc51efed119bc9cfdf792cdeaa4d67e8f6fcccab66ed4bfdd6bde3e59bfcbb2f \ + --hash=sha256:bdc919ead48f234740ad807933cdf545180bfbe9342c2bb451556db2ed958581 \ + --hash=sha256:bdd37121970bfd8be76c5fb069c7751683bdf373db1ed6c010162b2a130248ed \ + --hash=sha256:be8813b57049a7dc738189df53d69395eba14fb99345e0a5994914a3864c8a4b \ + --hash=sha256:c0c0b3ade1c0b13b936d7970b1d37a57acde9199dc2aecc4c336773e1d86049c \ + --hash=sha256:c47a551199eb8eb2121d4f0f15ae0f923d31350ab9280078d1e5f12b249e0026 \ + --hash=sha256:c4ffb7ebf07cfe8931028e3e4c85f0357459a3f9f9490886198848f4fa002ec8 \ + --hash=sha256:ccfcd093f13f0f0b7fdd0f198b90053bf7b2f02a3927a30e63f3ccc9df56b676 \ + --hash=sha256:d2ee202e79d8ed691ceebae8e0486bd9a2cd4794cec4824e1c99b6f5009502f6 \ + --hash=sha256:d53197da72cc091b024dd97249dfc7794d6a56530370992a5e1a08983ad9230e \ + --hash=sha256:d6dd0be5b5b189d31db7cda48b91d7e0a9795f31430b7f271219ab30f1d3ac9d \ + --hash=sha256:d88b440e37a16e651bda4c7c2b930eb586fd15ca7406cb39e211fcff3bf3017d \ + --hash=sha256:de8a88e63464af587c950061a5e6a67d3632e36df62b986892331d4620a35c01 \ + --hash=sha256:df2449253ef108a379b8b5d6b43f4b1a8e81a061d6537becd5582fba5f9196d7 \ + --hash=sha256:e1c1493fb6e50ab01d20a22826e57520f1284df32f2d8601fdd90b6304601419 \ + --hash=sha256:e1cf1972137e83c5d4c136c43ced9ac51d0e124706ee1c8aa8532c1287fa8795 \ + --hash=sha256:e2103a929dfa2fcaf9bb4e7c091983a49c9ac3b19c9061b6d5427dd7d14d81a1 \ + --hash=sha256:e56b7d45a839a697b5eb268c82a71bd8c7f6c94d6fd50c3d577fa39a9f1409f5 \ + --hash=sha256:e8afc3f2ccfa24215f8cb28dcf43f0113ac3c37c2f0f0806d8c70e4228c5cf4d \ + --hash=sha256:e8fc20152abba6b83724d7ff268c249fa196d8259ff481f3b1476383f8f24e42 \ + --hash=sha256:eaa9599de571d72e2daf60164784109f19978b327a3910d3e9de8c97b5b70cfe \ + --hash=sha256:ec15a59cf5af7be74194f7ab02d0f59a62bdcf1a537677ce67a2537c9b87fcda \ + --hash=sha256:f190daf01f13c72eac4efd5c430a8de82489d9cff23c364c3ea822545032993e \ + --hash=sha256:f34c41761022dd093b4b6896d4810782ffbabe30f2d443ff5f083e0cbbb8c737 \ + --hash=sha256:f3e98bb3798ead92273dc0e5fd0f31ade220f59a266ffd8a4f6065e0a3ce0523 \ + --hash=sha256:f42d0984e947b8adf7dd6dde396e720934d12c506ce84eea8476409563607591 \ + --hash=sha256:f71a396b3bf33ecaa1626c255855702aca4d3d9fea5e051b41ac59a9c1c41edc \ + --hash=sha256:f9e130248f4462aaa8e2552d547f36ddadbeaa573879158d721bbd33dfe4743a \ + --hash=sha256:fed51ac40f757d41b7c48425901843666a6677e3e8eb0abcff09e4ba6e664f50 + # via jinja2 +mccabe==0.7.0 \ + --hash=sha256:348e0240c33b60bbdf4e523192ef919f28cb2c3d7d5c7794f74009290f236325 \ + --hash=sha256:6c2d30ab6be0e4a46919781807b4f0d834ebdd6c6e3dca0bda5a15f863427b6e + # via pylint +packaging==25.0 \ + --hash=sha256:29572ef2b1f17581046b3a2227d5c611fb25ec70ca1ba8554b24b0e69331a484 \ + --hash=sha256:d443872c98d677bf60f6a1f2f8c1cb748e8fe762d2bf9d3148b5599295b0fc4f + # via sphinx +pathspec==0.12.1 \ + --hash=sha256:a0d503e138a4c123b27490a4f7beda6a01c6f288df0e4a8b79c7eb0dc7b4cc08 \ + --hash=sha256:a482d51503a1ab33b1c67a6c3813a26953dbdc71c31dacaef9a838c4e29f5712 + # via yamllint +platformdirs==4.4.0 \ + --hash=sha256:abd01743f24e5287cd7a5db3752faf1a2d65353f38ec26d98e25a6db65958c85 \ + --hash=sha256:ca753cf4d81dc309bc67b0ea38fd15dc97bc30ce419a7f58d13eb3bf14c4febf + # via pylint +pygments==2.19.2 \ + --hash=sha256:636cb2477cec7f8952536970bc533bc43743542f70392ae026374600add5b887 \ + --hash=sha256:86540386c03d588bb81d44bc3928634ff26449851e99741617ecb9037ee5ec0b + # via sphinx +pylint==2.15.10 \ + --hash=sha256:9df0d07e8948a1c3ffa3b6e2d7e6e63d9fb457c5da5b961ed63106594780cc7e \ + --hash=sha256:b3dc5ef7d33858f297ac0d06cc73862f01e4f2e74025ec3eff347ce0bc60baf5 + # via + # -r examples/bzlmod/requirements.in + # pylint-print +pylint-print==1.0.1 \ + --hash=sha256:30aa207e9718ebf4ceb47fb87012092e6d8743aab932aa07aa14a73e750ad3d0 \ + --hash=sha256:a2b2599e7887b93e551db2624c523c1e6e9e58c3be8416cd98d41e4427e2669b + # via -r examples/bzlmod/requirements.in +python-dateutil==2.9.0.post0 \ + --hash=sha256:37dd54208da7e1cd875388217d5e00ebd4179249f90fb72437e91a35459a0ad3 \ + --hash=sha256:a8b2bc7bffae282281c8140a97d3aa9c14da0b136dfe83f850eea9a5f7470427 + # via + # -r examples/bzlmod/requirements.in + # s3cmd +python-magic==0.4.27 \ + --hash=sha256:c1ba14b08e4a5f5c31a302b7721239695b2f0f058d125bd5ce1ee36b9d9d3c3b \ + --hash=sha256:c212960ad306f700aa0d01e5d7a325d20548ff97eb9920dcd29513174f0294d3 + # via s3cmd +pyyaml==6.0.3 \ + --hash=sha256:00c4bdeba853cc34e7dd471f16b4114f4162dc03e6b7afcc2128711f0eca823c \ + --hash=sha256:0150219816b6a1fa26fb4699fb7daa9caf09eb1999f3b70fb6e786805e80375a \ + --hash=sha256:02893d100e99e03eda1c8fd5c441d8c60103fd175728e23e431db1b589cf5ab3 \ + --hash=sha256:02ea2dfa234451bbb8772601d7b8e426c2bfa197136796224e50e35a78777956 \ + --hash=sha256:0f29edc409a6392443abf94b9cf89ce99889a1dd5376d94316ae5145dfedd5d6 \ + --hash=sha256:10892704fc220243f5305762e276552a0395f7beb4dbf9b14ec8fd43b57f126c \ + --hash=sha256:16249ee61e95f858e83976573de0f5b2893b3677ba71c9dd36b9cf8be9ac6d65 \ + --hash=sha256:1d37d57ad971609cf3c53ba6a7e365e40660e3be0e5175fa9f2365a379d6095a \ + --hash=sha256:1ebe39cb5fc479422b83de611d14e2c0d3bb2a18bbcb01f229ab3cfbd8fee7a0 \ + --hash=sha256:214ed4befebe12df36bcc8bc2b64b396ca31be9304b8f59e25c11cf94a4c033b \ + --hash=sha256:2283a07e2c21a2aa78d9c4442724ec1eb15f5e42a723b99cb3d822d48f5f7ad1 \ + --hash=sha256:22ba7cfcad58ef3ecddc7ed1db3409af68d023b7f940da23c6c2a1890976eda6 \ + --hash=sha256:27c0abcb4a5dac13684a37f76e701e054692a9b2d3064b70f5e4eb54810553d7 \ + --hash=sha256:28c8d926f98f432f88adc23edf2e6d4921ac26fb084b028c733d01868d19007e \ + --hash=sha256:2e71d11abed7344e42a8849600193d15b6def118602c4c176f748e4583246007 \ + --hash=sha256:34d5fcd24b8445fadc33f9cf348c1047101756fd760b4dacb5c3e99755703310 \ + --hash=sha256:37503bfbfc9d2c40b344d06b2199cf0e96e97957ab1c1b546fd4f87e53e5d3e4 \ + --hash=sha256:3c5677e12444c15717b902a5798264fa7909e41153cdf9ef7ad571b704a63dd9 \ + --hash=sha256:3ff07ec89bae51176c0549bc4c63aa6202991da2d9a6129d7aef7f1407d3f295 \ + --hash=sha256:41715c910c881bc081f1e8872880d3c650acf13dfa8214bad49ed4cede7c34ea \ + --hash=sha256:418cf3f2111bc80e0933b2cd8cd04f286338bb88bdc7bc8e6dd775ebde60b5e0 \ + --hash=sha256:44edc647873928551a01e7a563d7452ccdebee747728c1080d881d68af7b997e \ + --hash=sha256:4a2e8cebe2ff6ab7d1050ecd59c25d4c8bd7e6f400f5f82b96557ac0abafd0ac \ + --hash=sha256:4ad1906908f2f5ae4e5a8ddfce73c320c2a1429ec52eafd27138b7f1cbe341c9 \ + --hash=sha256:501a031947e3a9025ed4405a168e6ef5ae3126c59f90ce0cd6f2bfc477be31b7 \ + --hash=sha256:5190d403f121660ce8d1d2c1bb2ef1bd05b5f68533fc5c2ea899bd15f4399b35 \ + --hash=sha256:5498cd1645aa724a7c71c8f378eb29ebe23da2fc0d7a08071d89469bf1d2defb \ + --hash=sha256:5cf4e27da7e3fbed4d6c3d8e797387aaad68102272f8f9752883bc32d61cb87b \ + --hash=sha256:5e0b74767e5f8c593e8c9b5912019159ed0533c70051e9cce3e8b6aa699fcd69 \ + --hash=sha256:5ed875a24292240029e4483f9d4a4b8a1ae08843b9c54f43fcc11e404532a8a5 \ + --hash=sha256:5fcd34e47f6e0b794d17de1b4ff496c00986e1c83f7ab2fb8fcfe9616ff7477b \ + --hash=sha256:5fdec68f91a0c6739b380c83b951e2c72ac0197ace422360e6d5a959d8d97b2c \ + --hash=sha256:6344df0d5755a2c9a276d4473ae6b90647e216ab4757f8426893b5dd2ac3f369 \ + --hash=sha256:64386e5e707d03a7e172c0701abfb7e10f0fb753ee1d773128192742712a98fd \ + --hash=sha256:652cb6edd41e718550aad172851962662ff2681490a8a711af6a4d288dd96824 \ + --hash=sha256:66291b10affd76d76f54fad28e22e51719ef9ba22b29e1d7d03d6777a9174198 \ + --hash=sha256:66e1674c3ef6f541c35191caae2d429b967b99e02040f5ba928632d9a7f0f065 \ + --hash=sha256:6adc77889b628398debc7b65c073bcb99c4a0237b248cacaf3fe8a557563ef6c \ + --hash=sha256:79005a0d97d5ddabfeeea4cf676af11e647e41d81c9a7722a193022accdb6b7c \ + --hash=sha256:7c6610def4f163542a622a73fb39f534f8c101d690126992300bf3207eab9764 \ + --hash=sha256:7f047e29dcae44602496db43be01ad42fc6f1cc0d8cd6c83d342306c32270196 \ + --hash=sha256:8098f252adfa6c80ab48096053f512f2321f0b998f98150cea9bd23d83e1467b \ + --hash=sha256:850774a7879607d3a6f50d36d04f00ee69e7fc816450e5f7e58d7f17f1ae5c00 \ + --hash=sha256:8d1fab6bb153a416f9aeb4b8763bc0f22a5586065f86f7664fc23339fc1c1fac \ + --hash=sha256:8da9669d359f02c0b91ccc01cac4a67f16afec0dac22c2ad09f46bee0697eba8 \ + --hash=sha256:8dc52c23056b9ddd46818a57b78404882310fb473d63f17b07d5c40421e47f8e \ + --hash=sha256:9149cad251584d5fb4981be1ecde53a1ca46c891a79788c0df828d2f166bda28 \ + --hash=sha256:93dda82c9c22deb0a405ea4dc5f2d0cda384168e466364dec6255b293923b2f3 \ + --hash=sha256:96b533f0e99f6579b3d4d4995707cf36df9100d67e0c8303a0c55b27b5f99bc5 \ + --hash=sha256:9c57bb8c96f6d1808c030b1687b9b5fb476abaa47f0db9c0101f5e9f394e97f4 \ + --hash=sha256:9c7708761fccb9397fe64bbc0395abcae8c4bf7b0eac081e12b809bf47700d0b \ + --hash=sha256:9f3bfb4965eb874431221a3ff3fdcddc7e74e3b07799e0e84ca4a0f867d449bf \ + --hash=sha256:a33284e20b78bd4a18c8c2282d549d10bc8408a2a7ff57653c0cf0b9be0afce5 \ + --hash=sha256:a80cb027f6b349846a3bf6d73b5e95e782175e52f22108cfa17876aaeff93702 \ + --hash=sha256:b30236e45cf30d2b8e7b3e85881719e98507abed1011bf463a8fa23e9c3e98a8 \ + --hash=sha256:b3bc83488de33889877a0f2543ade9f70c67d66d9ebb4ac959502e12de895788 \ + --hash=sha256:b865addae83924361678b652338317d1bd7e79b1f4596f96b96c77a5a34b34da \ + --hash=sha256:b8bb0864c5a28024fac8a632c443c87c5aa6f215c0b126c449ae1a150412f31d \ + --hash=sha256:ba1cc08a7ccde2d2ec775841541641e4548226580ab850948cbfda66a1befcdc \ + --hash=sha256:bdb2c67c6c1390b63c6ff89f210c8fd09d9a1217a465701eac7316313c915e4c \ + --hash=sha256:c1ff362665ae507275af2853520967820d9124984e0f7466736aea23d8611fba \ + --hash=sha256:c2514fceb77bc5e7a2f7adfaa1feb2fb311607c9cb518dbc378688ec73d8292f \ + --hash=sha256:c3355370a2c156cffb25e876646f149d5d68f5e0a3ce86a5084dd0b64a994917 \ + --hash=sha256:c458b6d084f9b935061bc36216e8a69a7e293a2f1e68bf956dcd9e6cbcd143f5 \ + --hash=sha256:d0eae10f8159e8fdad514efdc92d74fd8d682c933a6dd088030f3834bc8e6b26 \ + --hash=sha256:d76623373421df22fb4cf8817020cbb7ef15c725b9d5e45f17e189bfc384190f \ + --hash=sha256:ebc55a14a21cb14062aa4162f906cd962b28e2e9ea38f9b4391244cd8de4ae0b \ + --hash=sha256:eda16858a3cab07b80edaf74336ece1f986ba330fdb8ee0d6c0d68fe82bc96be \ + --hash=sha256:ee2922902c45ae8ccada2c5b501ab86c36525b883eff4255313a253a3160861c \ + --hash=sha256:efd7b85f94a6f21e4932043973a7ba2613b059c4a000551892ac9f1d11f5baf3 \ + --hash=sha256:f7057c9a337546edc7973c0d3ba84ddcdf0daa14533c2065749c9075001090e6 \ + --hash=sha256:fa160448684b4e94d80416c0fa4aac48967a969efe22931448d853ada8baf926 \ + --hash=sha256:fc09d0aa354569bc501d4e787133afc08552722d3ab34836a80547331bb5d4a0 + # via yamllint +requests==2.25.1 \ + --hash=sha256:27973dd4a904a4f13b263a19c866c13b92a39ed1c964655f025f3f8d3d75b804 \ + --hash=sha256:c210084e36a42ae6b9219e00e48287def368a26d03a048ddad7bfee44f75871e + # via + # -r examples/bzlmod/requirements.in + # sphinx +s3cmd==2.1.0 \ + --hash=sha256:49cd23d516b17974b22b611a95ce4d93fe326feaa07320bd1d234fed68cbccfa \ + --hash=sha256:966b0a494a916fc3b4324de38f089c86c70ee90e8e1cae6d59102103a4c0cc03 + # via -r examples/bzlmod/requirements.in +six==1.17.0 \ + --hash=sha256:4721f391ed90541fddacab5acf947aa0d3dc7d27b2e1e8eda2be8970586c3274 \ + --hash=sha256:ff70335d468e7eb6ec65b95b99d3a2836546063f63acc5171de367e834932a81 + # via python-dateutil +snowballstemmer==3.0.1 \ + --hash=sha256:6cd7b3897da8d6c9ffb968a6781fa6532dce9c3618a4b127d920dab764a19064 \ + --hash=sha256:6d5eeeec8e9f84d4d56b847692bacf79bc2c8e90c7f80ca4444ff8b6f2e52895 + # via sphinx +sphinx==7.3.7 \ + --hash=sha256:413f75440be4cacf328f580b4274ada4565fb2187d696a84970c23f77b64d8c3 \ + --hash=sha256:a4a7db75ed37531c05002d56ed6948d4c42f473a36f46e1382b0bd76ca9627bc + # via -r examples/bzlmod/requirements.in +sphinxcontrib-applehelp==2.0.0 \ + --hash=sha256:2f29ef331735ce958efa4734873f084941970894c6090408b079c61b2e1c06d1 \ + --hash=sha256:4cd3f0ec4ac5dd9c17ec65e9ab272c9b867ea77425228e68ecf08d6b28ddbdb5 + # via sphinx +sphinxcontrib-devhelp==2.0.0 \ + --hash=sha256:411f5d96d445d1d73bb5d52133377b4248ec79db5c793ce7dbe59e074b4dd1ad \ + --hash=sha256:aefb8b83854e4b0998877524d1029fd3e6879210422ee3780459e28a1f03a8a2 + # via sphinx +sphinxcontrib-htmlhelp==2.1.0 \ + --hash=sha256:166759820b47002d22914d64a075ce08f4c46818e17cfc9470a9786b759b19f8 \ + --hash=sha256:c9e2916ace8aad64cc13a0d233ee22317f2b9025b9cf3295249fa985cc7082e9 + # via sphinx +sphinxcontrib-jsmath==1.0.1 \ + --hash=sha256:2ec2eaebfb78f3f2078e73666b1415417a116cc848b72e5172e596c871103178 \ + --hash=sha256:a9925e4a4587247ed2191a22df5f6970656cb8ca2bd6284309578f2153e0c4b8 + # via sphinx +sphinxcontrib-qthelp==2.0.0 \ + --hash=sha256:4fe7d0ac8fc171045be623aba3e2a8f613f8682731f9153bb2e40ece16b9bbab \ + --hash=sha256:b18a828cdba941ccd6ee8445dbe72ffa3ef8cbe7505d8cd1fa0d42d3f2d5f3eb + # via sphinx +sphinxcontrib-serializinghtml==2.0.0 \ + --hash=sha256:6e2cb0eef194e10c27ec0023bfeb25badbbb5868244cf5bc5bdc04e4464bf331 \ + --hash=sha256:e9d912827f872c029017a53f0ef2180b327c3f7fd23c87229f7a8e8b70031d4d + # via + # -r examples/bzlmod/requirements.in + # sphinx +tabulate==0.9.0 \ + --hash=sha256:0095b12bf5966de529c0feb1fa08671671b3368eec77d7ef7ab114be2c068b3c \ + --hash=sha256:024ca478df22e9340661486f85298cff5f6dcdba14f3813e8830015b9ed1948f + # via -r examples/bzlmod/requirements.in +tomlkit==0.13.3 \ + --hash=sha256:430cf247ee57df2b94ee3fbe588e71d362a941ebb545dec29b53961d61add2a1 \ + --hash=sha256:c89c649d79ee40629a9fda55f8ace8c6a1b42deb912b2a8fd8d942ddadb606b0 + # via pylint +urllib3==1.26.20 \ + --hash=sha256:0ed14ccfbf1c30a9072c7ca157e4319b70d65f623e91e7b32fadb2853431016e \ + --hash=sha256:40c2dc0c681e47eb8f90e7e27bf6ff7df2e677421fd46756da1161c39ca70d32 + # via requests +websockets==15.0.1 \ + --hash=sha256:0701bc3cfcb9164d04a14b149fd74be7347a530ad3bbf15ab2c678a2cd3dd9a2 \ + --hash=sha256:0a34631031a8f05657e8e90903e656959234f3a04552259458aac0b0f9ae6fd9 \ + --hash=sha256:0af68c55afbd5f07986df82831c7bff04846928ea8d1fd7f30052638788bc9b5 \ + --hash=sha256:0c9e74d766f2818bb95f84c25be4dea09841ac0f734d1966f415e4edfc4ef1c3 \ + --hash=sha256:0f3c1e2ab208db911594ae5b4f79addeb3501604a165019dd221c0bdcabe4db8 \ + --hash=sha256:0fdfe3e2a29e4db3659dbd5bbf04560cea53dd9610273917799f1cde46aa725e \ + --hash=sha256:1009ee0c7739c08a0cd59de430d6de452a55e42d6b522de7aa15e6f67db0b8e1 \ + --hash=sha256:1234d4ef35db82f5446dca8e35a7da7964d02c127b095e172e54397fb6a6c256 \ + --hash=sha256:16b6c1b3e57799b9d38427dda63edcbe4926352c47cf88588c0be4ace18dac85 \ + --hash=sha256:2034693ad3097d5355bfdacfffcbd3ef5694f9718ab7f29c29689a9eae841880 \ + --hash=sha256:21c1fa28a6a7e3cbdc171c694398b6df4744613ce9b36b1a498e816787e28123 \ + --hash=sha256:229cf1d3ca6c1804400b0a9790dc66528e08a6a1feec0d5040e8b9eb14422375 \ + --hash=sha256:27ccee0071a0e75d22cb35849b1db43f2ecd3e161041ac1ee9d2352ddf72f065 \ + --hash=sha256:363c6f671b761efcb30608d24925a382497c12c506b51661883c3e22337265ed \ + --hash=sha256:39c1fec2c11dc8d89bba6b2bf1556af381611a173ac2b511cf7231622058af41 \ + --hash=sha256:3b1ac0d3e594bf121308112697cf4b32be538fb1444468fb0a6ae4feebc83411 \ + --hash=sha256:3be571a8b5afed347da347bfcf27ba12b069d9d7f42cb8c7028b5e98bbb12597 \ + --hash=sha256:3c714d2fc58b5ca3e285461a4cc0c9a66bd0e24c5da9911e30158286c9b5be7f \ + --hash=sha256:3d00075aa65772e7ce9e990cab3ff1de702aa09be3940d1dc88d5abf1ab8a09c \ + --hash=sha256:3e90baa811a5d73f3ca0bcbf32064d663ed81318ab225ee4f427ad4e26e5aff3 \ + --hash=sha256:47819cea040f31d670cc8d324bb6435c6f133b8c7a19ec3d61634e62f8d8f9eb \ + --hash=sha256:47b099e1f4fbc95b701b6e85768e1fcdaf1630f3cbe4765fa216596f12310e2e \ + --hash=sha256:4a9fac8e469d04ce6c25bb2610dc535235bd4aa14996b4e6dbebf5e007eba5ee \ + --hash=sha256:4b826973a4a2ae47ba357e4e82fa44a463b8f168e1ca775ac64521442b19e87f \ + --hash=sha256:4c2529b320eb9e35af0fa3016c187dffb84a3ecc572bcee7c3ce302bfeba52bf \ + --hash=sha256:54479983bd5fb469c38f2f5c7e3a24f9a4e70594cd68cd1fa6b9340dadaff7cf \ + --hash=sha256:558d023b3df0bffe50a04e710bc87742de35060580a293c2a984299ed83bc4e4 \ + --hash=sha256:5756779642579d902eed757b21b0164cd6fe338506a8083eb58af5c372e39d9a \ + --hash=sha256:592f1a9fe869c778694f0aa806ba0374e97648ab57936f092fd9d87f8bc03665 \ + --hash=sha256:595b6c3969023ecf9041b2936ac3827e4623bfa3ccf007575f04c5a6aa318c22 \ + --hash=sha256:5a939de6b7b4e18ca683218320fc67ea886038265fd1ed30173f5ce3f8e85675 \ + --hash=sha256:5d54b09eba2bada6011aea5375542a157637b91029687eb4fdb2dab11059c1b4 \ + --hash=sha256:5df592cd503496351d6dc14f7cdad49f268d8e618f80dce0cd5a36b93c3fc08d \ + --hash=sha256:5f4c04ead5aed67c8a1a20491d54cdfba5884507a48dd798ecaf13c74c4489f5 \ + --hash=sha256:64dee438fed052b52e4f98f76c5790513235efaa1ef7f3f2192c392cd7c91b65 \ + --hash=sha256:66dd88c918e3287efc22409d426c8f729688d89a0c587c88971a0faa2c2f3792 \ + --hash=sha256:678999709e68425ae2593acf2e3ebcbcf2e69885a5ee78f9eb80e6e371f1bf57 \ + --hash=sha256:67f2b6de947f8c757db2db9c71527933ad0019737ec374a8a6be9a956786aaf9 \ + --hash=sha256:693f0192126df6c2327cce3baa7c06f2a117575e32ab2308f7f8216c29d9e2e3 \ + --hash=sha256:746ee8dba912cd6fc889a8147168991d50ed70447bf18bcda7039f7d2e3d9151 \ + --hash=sha256:756c56e867a90fb00177d530dca4b097dd753cde348448a1012ed6c5131f8b7d \ + --hash=sha256:76d1f20b1c7a2fa82367e04982e708723ba0e7b8d43aa643d3dcd404d74f1475 \ + --hash=sha256:7f493881579c90fc262d9cdbaa05a6b54b3811c2f300766748db79f098db9940 \ + --hash=sha256:823c248b690b2fd9303ba00c4f66cd5e2d8c3ba4aa968b2779be9532a4dad431 \ + --hash=sha256:82544de02076bafba038ce055ee6412d68da13ab47f0c60cab827346de828dee \ + --hash=sha256:8dd8327c795b3e3f219760fa603dcae1dcc148172290a8ab15158cf85a953413 \ + --hash=sha256:8fdc51055e6ff4adeb88d58a11042ec9a5eae317a0a53d12c062c8a8865909e8 \ + --hash=sha256:a625e06551975f4b7ea7102bc43895b90742746797e2e14b70ed61c43a90f09b \ + --hash=sha256:abdc0c6c8c648b4805c5eacd131910d2a7f6455dfd3becab248ef108e89ab16a \ + --hash=sha256:ac017dd64572e5c3bd01939121e4d16cf30e5d7e110a119399cf3133b63ad054 \ + --hash=sha256:ac1e5c9054fe23226fb11e05a6e630837f074174c4c2f0fe442996112a6de4fb \ + --hash=sha256:ac60e3b188ec7574cb761b08d50fcedf9d77f1530352db4eef1707fe9dee7205 \ + --hash=sha256:b359ed09954d7c18bbc1680f380c7301f92c60bf924171629c5db97febb12f04 \ + --hash=sha256:b7643a03db5c95c799b89b31c036d5f27eeb4d259c798e878d6937d71832b1e4 \ + --hash=sha256:ba9e56e8ceeeedb2e080147ba85ffcd5cd0711b89576b83784d8605a7df455fa \ + --hash=sha256:c338ffa0520bdb12fbc527265235639fb76e7bc7faafbb93f6ba80d9c06578a9 \ + --hash=sha256:cad21560da69f4ce7658ca2cb83138fb4cf695a2ba3e475e0559e05991aa8122 \ + --hash=sha256:d08eb4c2b7d6c41da6ca0600c077e93f5adcfd979cd777d747e9ee624556da4b \ + --hash=sha256:d50fd1ee42388dcfb2b3676132c78116490976f1300da28eb629272d5d93e905 \ + --hash=sha256:d591f8de75824cbb7acad4e05d2d710484f15f29d4a915092675ad3456f11770 \ + --hash=sha256:d5f6b181bb38171a8ad1d6aa58a67a6aa9d4b38d0f8c5f496b9e42561dfc62fe \ + --hash=sha256:d63efaa0cd96cf0c5fe4d581521d9fa87744540d4bc999ae6e08595a1014b45b \ + --hash=sha256:d99e5546bf73dbad5bf3547174cd6cb8ba7273062a23808ffea025ecb1cf8562 \ + --hash=sha256:e09473f095a819042ecb2ab9465aee615bd9c2028e4ef7d933600a8401c79561 \ + --hash=sha256:e8b56bdcdb4505c8078cb6c7157d9811a85790f2f2b3632c7d1462ab5783d215 \ + --hash=sha256:ee443ef070bb3b6ed74514f5efaa37a252af57c90eb33b956d35c8e9c10a1931 \ + --hash=sha256:f29d80eb9a9263b8d109135351caf568cc3f80b9928bccde535c235de55c22d9 \ + --hash=sha256:f7a866fbc1e97b5c617ee4116daaa09b722101d4a3c170c787450ba409f9736f \ + --hash=sha256:fcd5cf9e305d7b8338754470cf69cf81f420459dbae8a3b40cee57417f4614a7 + # via -r examples/bzlmod/requirements.in +wheel==0.45.1 \ + --hash=sha256:661e1abd9198507b1409a20c02106d9670b2576e916d58f520316666abca6729 \ + --hash=sha256:708e7481cc80179af0e556bbf0cc00b8444c7321e2700b8d8580231d13017248 + # via -r examples/bzlmod/requirements.in +wrapt==1.17.3 \ + --hash=sha256:02b551d101f31694fc785e58e0720ef7d9a10c4e62c1c9358ce6f63f23e30a56 \ + --hash=sha256:042ec3bb8f319c147b1301f2393bc19dba6e176b7da446853406d041c36c7828 \ + --hash=sha256:0610b46293c59a3adbae3dee552b648b984176f8562ee0dba099a56cfbe4df1f \ + --hash=sha256:0b02e424deef65c9f7326d8c19220a2c9040c51dc165cddb732f16198c168396 \ + --hash=sha256:0b1831115c97f0663cb77aa27d381237e73ad4f721391a9bfb2fe8bc25fa6e77 \ + --hash=sha256:0ed61b7c2d49cee3c027372df5809a59d60cf1b6c2f81ee980a091f3afed6a2d \ + --hash=sha256:0f5f51a6466667a5a356e6381d362d259125b57f059103dd9fdc8c0cf1d14139 \ + --hash=sha256:16ecf15d6af39246fe33e507105d67e4b81d8f8d2c6598ff7e3ca1b8a37213f7 \ + --hash=sha256:1f0b2f40cf341ee8cc1a97d51ff50dddb9fcc73241b9143ec74b30fc4f44f6cb \ + --hash=sha256:1f23fa283f51c890eda8e34e4937079114c74b4c81d2b2f1f1d94948f5cc3d7f \ + --hash=sha256:223db574bb38637e8230eb14b185565023ab624474df94d2af18f1cdb625216f \ + --hash=sha256:249f88ed15503f6492a71f01442abddd73856a0032ae860de6d75ca62eed8067 \ + --hash=sha256:24c2ed34dc222ed754247a2702b1e1e89fdbaa4016f324b4b8f1a802d4ffe87f \ + --hash=sha256:273a736c4645e63ac582c60a56b0acb529ef07f78e08dc6bfadf6a46b19c0da7 \ + --hash=sha256:281262213373b6d5e4bb4353bc36d1ba4084e6d6b5d242863721ef2bf2c2930b \ + --hash=sha256:30ce38e66630599e1193798285706903110d4f057aab3168a34b7fdc85569afc \ + --hash=sha256:33486899acd2d7d3066156b03465b949da3fd41a5da6e394ec49d271baefcf05 \ + --hash=sha256:343e44b2a8e60e06a7e0d29c1671a0d9951f59174f3709962b5143f60a2a98bd \ + --hash=sha256:373342dd05b1d07d752cecbec0c41817231f29f3a89aa8b8843f7b95992ed0c7 \ + --hash=sha256:3af60380ba0b7b5aeb329bc4e402acd25bd877e98b3727b0135cb5c2efdaefe9 \ + --hash=sha256:3e62d15d3cfa26e3d0788094de7b64efa75f3a53875cdbccdf78547aed547a81 \ + --hash=sha256:41b1d2bc74c2cac6f9074df52b2efbef2b30bdfe5f40cb78f8ca22963bc62977 \ + --hash=sha256:423ed5420ad5f5529db9ce89eac09c8a2f97da18eb1c870237e84c5a5c2d60aa \ + --hash=sha256:46acc57b331e0b3bcb3e1ca3b421d65637915cfcd65eb783cb2f78a511193f9b \ + --hash=sha256:4da9f45279fff3543c371d5ababc57a0384f70be244de7759c85a7f989cb4ebe \ + --hash=sha256:507553480670cab08a800b9463bdb881b2edeed77dc677b0a5915e6106e91a58 \ + --hash=sha256:53e5e39ff71b3fc484df8a522c933ea2b7cdd0d5d15ae82e5b23fde87d44cbd8 \ + --hash=sha256:54a30837587c6ee3cd1a4d1c2ec5d24e77984d44e2f34547e2323ddb4e22eb77 \ + --hash=sha256:5531d911795e3f935a9c23eb1c8c03c211661a5060aab167065896bbf62a5f85 \ + --hash=sha256:55cbbc356c2842f39bcc553cf695932e8b30e30e797f961860afb308e6b1bb7c \ + --hash=sha256:59923aa12d0157f6b82d686c3fd8e1166fa8cdfb3e17b42ce3b6147ff81528df \ + --hash=sha256:5a03a38adec8066d5a37bea22f2ba6bbf39fcdefbe2d91419ab864c3fb515454 \ + --hash=sha256:5a7b3c1ee8265eb4c8f1b7d29943f195c00673f5ab60c192eba2d4a7eae5f46a \ + --hash=sha256:5d4478d72eb61c36e5b446e375bbc49ed002430d17cdec3cecb36993398e1a9e \ + --hash=sha256:5ea5eb3c0c071862997d6f3e02af1d055f381b1d25b286b9d6644b79db77657c \ + --hash=sha256:604d076c55e2fdd4c1c03d06dc1a31b95130010517b5019db15365ec4a405fc6 \ + --hash=sha256:656873859b3b50eeebe6db8b1455e99d90c26ab058db8e427046dbc35c3140a5 \ + --hash=sha256:65d1d00fbfb3ea5f20add88bbc0f815150dbbde3b026e6c24759466c8b5a9ef9 \ + --hash=sha256:6b538e31eca1a7ea4605e44f81a48aa24c4632a277431a6ed3f328835901f4fd \ + --hash=sha256:6fd1ad24dc235e4ab88cda009e19bf347aabb975e44fd5c2fb22a3f6e4141277 \ + --hash=sha256:70d86fa5197b8947a2fa70260b48e400bf2ccacdcab97bb7de47e3d1e6312225 \ + --hash=sha256:7171ae35d2c33d326ac19dd8facb1e82e5fd04ef8c6c0e394d7af55a55051c22 \ + --hash=sha256:73d496de46cd2cdbdbcce4ae4bcdb4afb6a11234a1df9c085249d55166b95116 \ + --hash=sha256:7425ac3c54430f5fc5e7b6f41d41e704db073309acfc09305816bc6a0b26bb16 \ + --hash=sha256:74afa28374a3c3a11b3b5e5fca0ae03bef8450d6aa3ab3a1e2c30e3a75d023dc \ + --hash=sha256:758895b01d546812d1f42204bd443b8c433c44d090248bf22689df673ccafe00 \ + --hash=sha256:79573c24a46ce11aab457b472efd8d125e5a51da2d1d24387666cd85f54c05b2 \ + --hash=sha256:7e18f01b0c3e4a07fe6dfdb00e29049ba17eadbc5e7609a2a3a4af83ab7d710a \ + --hash=sha256:88547535b787a6c9ce4086917b6e1d291aa8ed914fdd3a838b3539dc95c12804 \ + --hash=sha256:88bbae4d40d5a46142e70d58bf664a89b6b4befaea7b2ecc14e03cedb8e06c04 \ + --hash=sha256:8cccf4f81371f257440c88faed6b74f1053eef90807b77e31ca057b2db74edb1 \ + --hash=sha256:9baa544e6acc91130e926e8c802a17f3b16fbea0fd441b5a60f5cf2cc5c3deba \ + --hash=sha256:a36692b8491d30a8c75f1dfee65bef119d6f39ea84ee04d9f9311f83c5ad9390 \ + --hash=sha256:a47681378a0439215912ef542c45a783484d4dd82bac412b71e59cf9c0e1cea0 \ + --hash=sha256:a7c06742645f914f26c7f1fa47b8bc4c91d222f76ee20116c43d5ef0912bba2d \ + --hash=sha256:a9a2203361a6e6404f80b99234fe7fb37d1fc73487b5a78dc1aa5b97201e0f22 \ + --hash=sha256:ab232e7fdb44cdfbf55fc3afa31bcdb0d8980b9b95c38b6405df2acb672af0e0 \ + --hash=sha256:ad85e269fe54d506b240d2d7b9f5f2057c2aa9a2ea5b32c66f8902f768117ed2 \ + --hash=sha256:af338aa93554be859173c39c85243970dc6a289fa907402289eeae7543e1ae18 \ + --hash=sha256:afd964fd43b10c12213574db492cb8f73b2f0826c8df07a68288f8f19af2ebe6 \ + --hash=sha256:b32888aad8b6e68f83a8fdccbf3165f5469702a7544472bdf41f582970ed3311 \ + --hash=sha256:c31eebe420a9a5d2887b13000b043ff6ca27c452a9a22fa71f35f118e8d4bf89 \ + --hash=sha256:caea3e9c79d5f0d2c6d9ab96111601797ea5da8e6d0723f77eabb0d4068d2b2f \ + --hash=sha256:cf30f6e3c077c8e6a9a7809c94551203c8843e74ba0c960f4a98cd80d4665d39 \ + --hash=sha256:d40770d7c0fd5cbed9d84b2c3f2e156431a12c9a37dc6284060fb4bec0b7ffd4 \ + --hash=sha256:d8a210b158a34164de8bb68b0e7780041a903d7b00c87e906fb69928bf7890d5 \ + --hash=sha256:dc4a8d2b25efb6681ecacad42fca8859f88092d8732b170de6a5dddd80a1c8fa \ + --hash=sha256:df7d30371a2accfe4013e90445f6388c570f103d61019b6b7c57e0265250072a \ + --hash=sha256:e01375f275f010fcbf7f643b4279896d04e571889b8a5b3f848423d91bf07050 \ + --hash=sha256:e1a4120ae5705f673727d3253de3ed0e016f7cd78dc463db1b31e2463e1f3cf6 \ + --hash=sha256:e228514a06843cae89621384cfe3a80418f3c04aadf8a3b14e46a7be704e4235 \ + --hash=sha256:e405adefb53a435f01efa7ccdec012c016b5a1d3f35459990afc39b6be4d5056 \ + --hash=sha256:e6b13af258d6a9ad602d57d889f83b9d5543acd471eee12eb51f5b01f8eb1bc2 \ + --hash=sha256:e6f40a8aa5a92f150bdb3e1c44b7e98fb7113955b2e5394122fa5532fec4b418 \ + --hash=sha256:e71d5c6ebac14875668a1e90baf2ea0ef5b7ac7918355850c0908ae82bcb297c \ + --hash=sha256:ed7c635ae45cfbc1a7371f708727bf74690daedc49b4dba310590ca0bd28aa8a \ + --hash=sha256:f38e60678850c42461d4202739f9bf1e3a737c7ad283638251e79cc49effb6b6 \ + --hash=sha256:f66eb08feaa410fe4eebd17f2a2c8e2e46d3476e9f8c783daa8e09e0faa666d0 \ + --hash=sha256:f9b2601381be482f70e5d1051a5965c25fb3625455a2bf520b5a077b22afb775 \ + --hash=sha256:fbd3c8319de8e1dc79d346929cd71d523622da527cca14e0c1d257e31c2b8b10 \ + --hash=sha256:fd341868a4b6714a5962c1af0bd44f7c404ef78720c7de4892901e540417111c + # via astroid +yamllint==1.37.1 \ + --hash=sha256:364f0d79e81409f591e323725e6a9f4504c8699ddf2d7263d8d2b539cd66a583 \ + --hash=sha256:81f7c0c5559becc8049470d86046b36e96113637bcbe4753ecef06977c00245d + # via -r examples/bzlmod/requirements.in diff --git a/examples/bzlmod/requirements_windows_3_11.txt b/examples/bzlmod/requirements_windows_3_11.txt new file mode 100644 index 0000000000..fd7895b273 --- /dev/null +++ b/examples/bzlmod/requirements_windows_3_11.txt @@ -0,0 +1,530 @@ +# This file was autogenerated by uv via the following command: +# bazel run //examples:bzlmod_requirements_3_11_windows.update +--index-url https://pypi.org/simple +--extra-index-url https://pypi.org/simple/ + +alabaster==0.7.16 \ + --hash=sha256:75a8b99c28a5dad50dd7f8ccdd447a121ddb3892da9e53d1ca5cca3106d58d65 \ + --hash=sha256:b46733c07dce03ae4e150330b975c75737fa60f0a7c591b6c8bf4928a28e2c92 + # via sphinx +astroid==2.13.5 \ + --hash=sha256:6891f444625b6edb2ac798829b689e95297e100ddf89dbed5a8c610e34901501 \ + --hash=sha256:df164d5ac811b9f44105a72b8f9d5edfb7b5b2d7e979b04ea377a77b3229114a + # via pylint +babel==2.17.0 \ + --hash=sha256:0c54cffb19f690cdcc52a3b50bcbf71e07a808d1c80d549f2459b9d2cf0afb9d \ + --hash=sha256:4d0b53093fdfb4b21c92b5213dba5a1b23885afa8383709427046b21c366e5f2 + # via sphinx +certifi==2025.10.5 \ + --hash=sha256:0f212c2744a9bb6de0c56639a6f68afe01ecd92d91f14ae897c4fe7bbeeef0de \ + --hash=sha256:47c09d31ccf2acf0be3f701ea53595ee7e0b8fa08801c6624be771df09ae7b43 + # via requests +chardet==4.0.0 \ + --hash=sha256:0d6f53a15db4120f2b08c94f11e7d93d2c911ee118b6b30a04ec3ee8310179fa \ + --hash=sha256:f864054d66fd9118f2e67044ac8981a54775ec5b67aed0441892edb553d21da5 + # via requests +colorama==0.4.6 \ + --hash=sha256:08695f5cb7ed6e0531a20572697297273c47b8cae5a63ffc6d6ed5c201be6e44 \ + --hash=sha256:4f1d9991f5acc0ca119f9d443620b77f9d6b33703e51011c16baf57afb285fc6 + # via + # -r examples/bzlmod/requirements.in + # pylint + # sphinx +dill==0.4.0 \ + --hash=sha256:0633f1d2df477324f53a895b02c901fb961bdbf65a17122586ea7019292cbcf0 \ + --hash=sha256:44f54bf6412c2c8464c14e8243eb163690a9800dbe2c367330883b19c7561049 + # via pylint +docutils==0.21.2 \ + --hash=sha256:3a6b18732edf182daa3cd12775bbb338cf5691468f91eeeb109deff6ebfa986f \ + --hash=sha256:dafca5b9e384f0e419294eb4d2ff9fa826435bf15f15b7bd45723e8ad76811b2 + # via sphinx +idna==2.10 \ + --hash=sha256:b307872f855b18632ce0c21c5e45be78c0ea7ae4c15c828c20788b26921eb3f6 \ + --hash=sha256:b97d804b1e9b523befed77c48dacec60e6dcb0b5391d57af6a65a312a90648c0 + # via requests +imagesize==1.4.1 \ + --hash=sha256:0d8d18d08f840c19d0ee7ca1fd82490fdc3729b7ac93f49870406ddde8ef8d8b \ + --hash=sha256:69150444affb9cb0d5cc5a92b3676f0b2fb7cd9ae39e947a5e11a36b4497cd4a + # via sphinx +isort==5.13.2 \ + --hash=sha256:48fdfcb9face5d58a4f6dde2e72a1fb8dcaf8ab26f95ab49fab84c2ddefb0109 \ + --hash=sha256:8ca5e72a8d85860d5a3fa69b8745237f2939afe12dbf656afbcb47fe72d947a6 + # via pylint +jinja2==3.1.6 \ + --hash=sha256:0137fb05990d35f1275a587e9aee6d56da821fc83491a0fb838183be43f66d6d \ + --hash=sha256:85ece4451f492d0c13c5dd7c13a64681a86afae63a5f347908daf103ce6d2f67 + # via sphinx +lazy-object-proxy==1.12.0 \ + --hash=sha256:029d2b355076710505c9545aef5ab3f750d89779310e26ddf2b7b23f6ea03cd8 \ + --hash=sha256:08c465fb5cd23527512f9bd7b4c7ba6cec33e28aad36fbbe46bf7b858f9f3f7f \ + --hash=sha256:0a83c6f7a6b2bfc11ef3ed67f8cbe99f8ff500b05655d8e7df9aab993a6abc95 \ + --hash=sha256:1192e8c2f1031a6ff453ee40213afa01ba765b3dc861302cd91dbdb2e2660b00 \ + --hash=sha256:14e348185adbd03ec17d051e169ec45686dcd840a3779c9d4c10aabe2ca6e1c0 \ + --hash=sha256:15400b18893f345857b9e18b9bd87bd06aba84af6ed086187add70aeaa3f93f1 \ + --hash=sha256:1cf69cd1a6c7fe2dbcc3edaa017cf010f4192e53796538cc7d5e1fedbfa4bcff \ + --hash=sha256:1f5a462d92fd0cfb82f1fab28b51bfb209fabbe6aabf7f0d51472c0c124c0c61 \ + --hash=sha256:256262384ebd2a77b023ad02fbcc9326282bcfd16484d5531154b02bc304f4c5 \ + --hash=sha256:31020c84005d3daa4cc0fa5a310af2066efe6b0d82aeebf9ab199292652ff036 \ + --hash=sha256:338ab2f132276203e404951205fe80c3fd59429b3a724e7b662b2eb539bb1be9 \ + --hash=sha256:3605b632e82a1cbc32a1e5034278a64db555b3496e0795723ee697006b980508 \ + --hash=sha256:3d3964fbd326578bcdfffd017ef101b6fb0484f34e731fe060ba9b8816498c36 \ + --hash=sha256:424a8ab6695400845c39f13c685050eab69fa0bbac5790b201cd27375e5e41d7 \ + --hash=sha256:4a79b909aa16bde8ae606f06e6bbc9d3219d2e57fb3e0076e17879072b742c65 \ + --hash=sha256:4ab2c584e3cc8be0dfca422e05ad30a9abe3555ce63e9ab7a559f62f8dbc6ff9 \ + --hash=sha256:53c7fd99eb156bbb82cbc5d5188891d8fdd805ba6c1e3b92b90092da2a837073 \ + --hash=sha256:563d2ec8e4d4b68ee7848c5ab4d6057a6d703cb7963b342968bb8758dda33a23 \ + --hash=sha256:61d5e3310a4aa5792c2b599a7a78ccf8687292c8eb09cf187cca8f09cf6a7519 \ + --hash=sha256:6763941dbf97eea6b90f5b06eb4da9418cc088fce0e3883f5816090f9afcde4a \ + --hash=sha256:67f07ab742f1adfb3966c40f630baaa7902be4222a17941f3d85fd1dae5565ff \ + --hash=sha256:717484c309df78cedf48396e420fa57fc8a2b1f06ea889df7248fdd156e58847 \ + --hash=sha256:75ba769017b944fcacbf6a80c18b2761a1795b03f8899acdad1f1c39db4409be \ + --hash=sha256:7601ec171c7e8584f8ff3f4e440aa2eebf93e854f04639263875b8c2971f819f \ + --hash=sha256:7b22c2bbfb155706b928ac4d74c1a63ac8552a55ba7fff4445155523ea4067e1 \ + --hash=sha256:800f32b00a47c27446a2b767df7538e6c66a3488632c402b4fb2224f9794f3c0 \ + --hash=sha256:81d1852fb30fab81696f93db1b1e55a5d1ff7940838191062f5f56987d5fcc3e \ + --hash=sha256:86fd61cb2ba249b9f436d789d1356deae69ad3231dc3c0f17293ac535162672e \ + --hash=sha256:8c40b3c9faee2e32bfce0df4ae63f4e73529766893258eca78548bac801c8f66 \ + --hash=sha256:8ee0d6027b760a11cc18281e702c0309dd92da458a74b4c15025d7fc490deede \ + --hash=sha256:997b1d6e10ecc6fb6fe0f2c959791ae59599f41da61d652f6c903d1ee58b7370 \ + --hash=sha256:a61095f5d9d1a743e1e20ec6d6db6c2ca511961777257ebd9b288951b23b44fa \ + --hash=sha256:a6b7ea5ea1ffe15059eb44bcbcb258f97bcb40e139b88152c40d07b1a1dfc9ac \ + --hash=sha256:ae575ad9b674d0029fc077c5231b3bc6b433a3d1a62a8c363df96974b5534728 \ + --hash=sha256:be5fe974e39ceb0d6c9db0663c0464669cf866b2851c73971409b9566e880eab \ + --hash=sha256:be9045646d83f6c2664c1330904b245ae2371b5c57a3195e4028aedc9f999655 \ + --hash=sha256:c1ca33565f698ac1aece152a10f432415d1a2aa9a42dfe23e5ba2bc255ab91f6 \ + --hash=sha256:c3b2e0af1f7f77c4263759c4824316ce458fabe0fceadcd24ef8ca08b2d1e402 \ + --hash=sha256:c4fcbe74fb85df8ba7825fa05eddca764138da752904b378f0ae5ab33a36c308 \ + --hash=sha256:c9defba70ab943f1df98a656247966d7729da2fe9c2d5d85346464bf320820a3 \ + --hash=sha256:cc6e3614eca88b1c8a625fc0a47d0d745e7c3255b21dac0e30b3037c5e3deeb8 \ + --hash=sha256:d01c7819a410f7c255b20799b65d36b414379a30c6f1684c7bd7eb6777338c1b \ + --hash=sha256:efff4375a8c52f55a145dc8487a2108c2140f0bec4151ab4e1843e52eb9987ad \ + --hash=sha256:fdc70d81235fc586b9e3d1aeef7d1553259b62ecaae9db2167a5d2550dcc391a + # via astroid +markupsafe==3.0.3 \ + --hash=sha256:0303439a41979d9e74d18ff5e2dd8c43ed6c6001fd40e5bf2e43f7bd9bbc523f \ + --hash=sha256:068f375c472b3e7acbe2d5318dea141359e6900156b5b2ba06a30b169086b91a \ + --hash=sha256:0bf2a864d67e76e5c9a34dc26ec616a66b9888e25e7b9460e1c76d3293bd9dbf \ + --hash=sha256:0db14f5dafddbb6d9208827849fad01f1a2609380add406671a26386cdf15a19 \ + --hash=sha256:0eb9ff8191e8498cca014656ae6b8d61f39da5f95b488805da4bb029cccbfbaf \ + --hash=sha256:0f4b68347f8c5eab4a13419215bdfd7f8c9b19f2b25520968adfad23eb0ce60c \ + --hash=sha256:1085e7fbddd3be5f89cc898938f42c0b3c711fdcb37d75221de2666af647c175 \ + --hash=sha256:116bb52f642a37c115f517494ea5feb03889e04df47eeff5b130b1808ce7c219 \ + --hash=sha256:12c63dfb4a98206f045aa9563db46507995f7ef6d83b2f68eda65c307c6829eb \ + --hash=sha256:133a43e73a802c5562be9bbcd03d090aa5a1fe899db609c29e8c8d815c5f6de6 \ + --hash=sha256:1353ef0c1b138e1907ae78e2f6c63ff67501122006b0f9abad68fda5f4ffc6ab \ + --hash=sha256:15d939a21d546304880945ca1ecb8a039db6b4dc49b2c5a400387cdae6a62e26 \ + --hash=sha256:177b5253b2834fe3678cb4a5f0059808258584c559193998be2601324fdeafb1 \ + --hash=sha256:1872df69a4de6aead3491198eaf13810b565bdbeec3ae2dc8780f14458ec73ce \ + --hash=sha256:1b4b79e8ebf6b55351f0d91fe80f893b4743f104bff22e90697db1590e47a218 \ + --hash=sha256:1b52b4fb9df4eb9ae465f8d0c228a00624de2334f216f178a995ccdcf82c4634 \ + --hash=sha256:1ba88449deb3de88bd40044603fafffb7bc2b055d626a330323a9ed736661695 \ + --hash=sha256:1cc7ea17a6824959616c525620e387f6dd30fec8cb44f649e31712db02123dad \ + --hash=sha256:218551f6df4868a8d527e3062d0fb968682fe92054e89978594c28e642c43a73 \ + --hash=sha256:26a5784ded40c9e318cfc2bdb30fe164bdb8665ded9cd64d500a34fb42067b1c \ + --hash=sha256:2713baf880df847f2bece4230d4d094280f4e67b1e813eec43b4c0e144a34ffe \ + --hash=sha256:2a15a08b17dd94c53a1da0438822d70ebcd13f8c3a95abe3a9ef9f11a94830aa \ + --hash=sha256:2f981d352f04553a7171b8e44369f2af4055f888dfb147d55e42d29e29e74559 \ + --hash=sha256:32001d6a8fc98c8cb5c947787c5d08b0a50663d139f1305bac5885d98d9b40fa \ + --hash=sha256:3524b778fe5cfb3452a09d31e7b5adefeea8c5be1d43c4f810ba09f2ceb29d37 \ + --hash=sha256:3537e01efc9d4dccdf77221fb1cb3b8e1a38d5428920e0657ce299b20324d758 \ + --hash=sha256:35add3b638a5d900e807944a078b51922212fb3dedb01633a8defc4b01a3c85f \ + --hash=sha256:38664109c14ffc9e7437e86b4dceb442b0096dfe3541d7864d9cbe1da4cf36c8 \ + --hash=sha256:3a7e8ae81ae39e62a41ec302f972ba6ae23a5c5396c8e60113e9066ef893da0d \ + --hash=sha256:3b562dd9e9ea93f13d53989d23a7e775fdfd1066c33494ff43f5418bc8c58a5c \ + --hash=sha256:457a69a9577064c05a97c41f4e65148652db078a3a509039e64d3467b9e7ef97 \ + --hash=sha256:4bd4cd07944443f5a265608cc6aab442e4f74dff8088b0dfc8238647b8f6ae9a \ + --hash=sha256:4e885a3d1efa2eadc93c894a21770e4bc67899e3543680313b09f139e149ab19 \ + --hash=sha256:4faffd047e07c38848ce017e8725090413cd80cbc23d86e55c587bf979e579c9 \ + --hash=sha256:509fa21c6deb7a7a273d629cf5ec029bc209d1a51178615ddf718f5918992ab9 \ + --hash=sha256:5678211cb9333a6468fb8d8be0305520aa073f50d17f089b5b4b477ea6e67fdc \ + --hash=sha256:591ae9f2a647529ca990bc681daebdd52c8791ff06c2bfa05b65163e28102ef2 \ + --hash=sha256:5a7d5dc5140555cf21a6fefbdbf8723f06fcd2f63ef108f2854de715e4422cb4 \ + --hash=sha256:69c0b73548bc525c8cb9a251cddf1931d1db4d2258e9599c28c07ef3580ef354 \ + --hash=sha256:6b5420a1d9450023228968e7e6a9ce57f65d148ab56d2313fcd589eee96a7a50 \ + --hash=sha256:722695808f4b6457b320fdc131280796bdceb04ab50fe1795cd540799ebe1698 \ + --hash=sha256:729586769a26dbceff69f7a7dbbf59ab6572b99d94576a5592625d5b411576b9 \ + --hash=sha256:77f0643abe7495da77fb436f50f8dab76dbc6e5fd25d39589a0f1fe6548bfa2b \ + --hash=sha256:795e7751525cae078558e679d646ae45574b47ed6e7771863fcc079a6171a0fc \ + --hash=sha256:7be7b61bb172e1ed687f1754f8e7484f1c8019780f6f6b0786e76bb01c2ae115 \ + --hash=sha256:7c3fb7d25180895632e5d3148dbdc29ea38ccb7fd210aa27acbd1201a1902c6e \ + --hash=sha256:7e68f88e5b8799aa49c85cd116c932a1ac15caaa3f5db09087854d218359e485 \ + --hash=sha256:83891d0e9fb81a825d9a6d61e3f07550ca70a076484292a70fde82c4b807286f \ + --hash=sha256:8485f406a96febb5140bfeca44a73e3ce5116b2501ac54fe953e488fb1d03b12 \ + --hash=sha256:8709b08f4a89aa7586de0aadc8da56180242ee0ada3999749b183aa23df95025 \ + --hash=sha256:8f71bc33915be5186016f675cd83a1e08523649b0e33efdb898db577ef5bb009 \ + --hash=sha256:915c04ba3851909ce68ccc2b8e2cd691618c4dc4c4232fb7982bca3f41fd8c3d \ + --hash=sha256:949b8d66bc381ee8b007cd945914c721d9aba8e27f71959d750a46f7c282b20b \ + --hash=sha256:94c6f0bb423f739146aec64595853541634bde58b2135f27f61c1ffd1cd4d16a \ + --hash=sha256:9a1abfdc021a164803f4d485104931fb8f8c1efd55bc6b748d2f5774e78b62c5 \ + --hash=sha256:9b79b7a16f7fedff2495d684f2b59b0457c3b493778c9eed31111be64d58279f \ + --hash=sha256:a320721ab5a1aba0a233739394eb907f8c8da5c98c9181d1161e77a0c8e36f2d \ + --hash=sha256:a4afe79fb3de0b7097d81da19090f4df4f8d3a2b3adaa8764138aac2e44f3af1 \ + --hash=sha256:ad2cf8aa28b8c020ab2fc8287b0f823d0a7d8630784c31e9ee5edea20f406287 \ + --hash=sha256:b8512a91625c9b3da6f127803b166b629725e68af71f8184ae7e7d54686a56d6 \ + --hash=sha256:bc51efed119bc9cfdf792cdeaa4d67e8f6fcccab66ed4bfdd6bde3e59bfcbb2f \ + --hash=sha256:bdc919ead48f234740ad807933cdf545180bfbe9342c2bb451556db2ed958581 \ + --hash=sha256:bdd37121970bfd8be76c5fb069c7751683bdf373db1ed6c010162b2a130248ed \ + --hash=sha256:be8813b57049a7dc738189df53d69395eba14fb99345e0a5994914a3864c8a4b \ + --hash=sha256:c0c0b3ade1c0b13b936d7970b1d37a57acde9199dc2aecc4c336773e1d86049c \ + --hash=sha256:c47a551199eb8eb2121d4f0f15ae0f923d31350ab9280078d1e5f12b249e0026 \ + --hash=sha256:c4ffb7ebf07cfe8931028e3e4c85f0357459a3f9f9490886198848f4fa002ec8 \ + --hash=sha256:ccfcd093f13f0f0b7fdd0f198b90053bf7b2f02a3927a30e63f3ccc9df56b676 \ + --hash=sha256:d2ee202e79d8ed691ceebae8e0486bd9a2cd4794cec4824e1c99b6f5009502f6 \ + --hash=sha256:d53197da72cc091b024dd97249dfc7794d6a56530370992a5e1a08983ad9230e \ + --hash=sha256:d6dd0be5b5b189d31db7cda48b91d7e0a9795f31430b7f271219ab30f1d3ac9d \ + --hash=sha256:d88b440e37a16e651bda4c7c2b930eb586fd15ca7406cb39e211fcff3bf3017d \ + --hash=sha256:de8a88e63464af587c950061a5e6a67d3632e36df62b986892331d4620a35c01 \ + --hash=sha256:df2449253ef108a379b8b5d6b43f4b1a8e81a061d6537becd5582fba5f9196d7 \ + --hash=sha256:e1c1493fb6e50ab01d20a22826e57520f1284df32f2d8601fdd90b6304601419 \ + --hash=sha256:e1cf1972137e83c5d4c136c43ced9ac51d0e124706ee1c8aa8532c1287fa8795 \ + --hash=sha256:e2103a929dfa2fcaf9bb4e7c091983a49c9ac3b19c9061b6d5427dd7d14d81a1 \ + --hash=sha256:e56b7d45a839a697b5eb268c82a71bd8c7f6c94d6fd50c3d577fa39a9f1409f5 \ + --hash=sha256:e8afc3f2ccfa24215f8cb28dcf43f0113ac3c37c2f0f0806d8c70e4228c5cf4d \ + --hash=sha256:e8fc20152abba6b83724d7ff268c249fa196d8259ff481f3b1476383f8f24e42 \ + --hash=sha256:eaa9599de571d72e2daf60164784109f19978b327a3910d3e9de8c97b5b70cfe \ + --hash=sha256:ec15a59cf5af7be74194f7ab02d0f59a62bdcf1a537677ce67a2537c9b87fcda \ + --hash=sha256:f190daf01f13c72eac4efd5c430a8de82489d9cff23c364c3ea822545032993e \ + --hash=sha256:f34c41761022dd093b4b6896d4810782ffbabe30f2d443ff5f083e0cbbb8c737 \ + --hash=sha256:f3e98bb3798ead92273dc0e5fd0f31ade220f59a266ffd8a4f6065e0a3ce0523 \ + --hash=sha256:f42d0984e947b8adf7dd6dde396e720934d12c506ce84eea8476409563607591 \ + --hash=sha256:f71a396b3bf33ecaa1626c255855702aca4d3d9fea5e051b41ac59a9c1c41edc \ + --hash=sha256:f9e130248f4462aaa8e2552d547f36ddadbeaa573879158d721bbd33dfe4743a \ + --hash=sha256:fed51ac40f757d41b7c48425901843666a6677e3e8eb0abcff09e4ba6e664f50 + # via jinja2 +mccabe==0.7.0 \ + --hash=sha256:348e0240c33b60bbdf4e523192ef919f28cb2c3d7d5c7794f74009290f236325 \ + --hash=sha256:6c2d30ab6be0e4a46919781807b4f0d834ebdd6c6e3dca0bda5a15f863427b6e + # via pylint +packaging==25.0 \ + --hash=sha256:29572ef2b1f17581046b3a2227d5c611fb25ec70ca1ba8554b24b0e69331a484 \ + --hash=sha256:d443872c98d677bf60f6a1f2f8c1cb748e8fe762d2bf9d3148b5599295b0fc4f + # via sphinx +pathspec==0.12.1 \ + --hash=sha256:a0d503e138a4c123b27490a4f7beda6a01c6f288df0e4a8b79c7eb0dc7b4cc08 \ + --hash=sha256:a482d51503a1ab33b1c67a6c3813a26953dbdc71c31dacaef9a838c4e29f5712 + # via yamllint +platformdirs==4.4.0 \ + --hash=sha256:abd01743f24e5287cd7a5db3752faf1a2d65353f38ec26d98e25a6db65958c85 \ + --hash=sha256:ca753cf4d81dc309bc67b0ea38fd15dc97bc30ce419a7f58d13eb3bf14c4febf + # via pylint +pygments==2.19.2 \ + --hash=sha256:636cb2477cec7f8952536970bc533bc43743542f70392ae026374600add5b887 \ + --hash=sha256:86540386c03d588bb81d44bc3928634ff26449851e99741617ecb9037ee5ec0b + # via sphinx +pylint==2.15.10 \ + --hash=sha256:9df0d07e8948a1c3ffa3b6e2d7e6e63d9fb457c5da5b961ed63106594780cc7e \ + --hash=sha256:b3dc5ef7d33858f297ac0d06cc73862f01e4f2e74025ec3eff347ce0bc60baf5 + # via + # -r examples/bzlmod/requirements.in + # pylint-print +pylint-print==1.0.1 \ + --hash=sha256:30aa207e9718ebf4ceb47fb87012092e6d8743aab932aa07aa14a73e750ad3d0 \ + --hash=sha256:a2b2599e7887b93e551db2624c523c1e6e9e58c3be8416cd98d41e4427e2669b + # via -r examples/bzlmod/requirements.in +python-dateutil==2.9.0.post0 \ + --hash=sha256:37dd54208da7e1cd875388217d5e00ebd4179249f90fb72437e91a35459a0ad3 \ + --hash=sha256:a8b2bc7bffae282281c8140a97d3aa9c14da0b136dfe83f850eea9a5f7470427 + # via + # -r examples/bzlmod/requirements.in + # s3cmd +python-magic==0.4.27 \ + --hash=sha256:c1ba14b08e4a5f5c31a302b7721239695b2f0f058d125bd5ce1ee36b9d9d3c3b \ + --hash=sha256:c212960ad306f700aa0d01e5d7a325d20548ff97eb9920dcd29513174f0294d3 + # via s3cmd +pyyaml==6.0.3 \ + --hash=sha256:00c4bdeba853cc34e7dd471f16b4114f4162dc03e6b7afcc2128711f0eca823c \ + --hash=sha256:0150219816b6a1fa26fb4699fb7daa9caf09eb1999f3b70fb6e786805e80375a \ + --hash=sha256:02893d100e99e03eda1c8fd5c441d8c60103fd175728e23e431db1b589cf5ab3 \ + --hash=sha256:02ea2dfa234451bbb8772601d7b8e426c2bfa197136796224e50e35a78777956 \ + --hash=sha256:0f29edc409a6392443abf94b9cf89ce99889a1dd5376d94316ae5145dfedd5d6 \ + --hash=sha256:10892704fc220243f5305762e276552a0395f7beb4dbf9b14ec8fd43b57f126c \ + --hash=sha256:16249ee61e95f858e83976573de0f5b2893b3677ba71c9dd36b9cf8be9ac6d65 \ + --hash=sha256:1d37d57ad971609cf3c53ba6a7e365e40660e3be0e5175fa9f2365a379d6095a \ + --hash=sha256:1ebe39cb5fc479422b83de611d14e2c0d3bb2a18bbcb01f229ab3cfbd8fee7a0 \ + --hash=sha256:214ed4befebe12df36bcc8bc2b64b396ca31be9304b8f59e25c11cf94a4c033b \ + --hash=sha256:2283a07e2c21a2aa78d9c4442724ec1eb15f5e42a723b99cb3d822d48f5f7ad1 \ + --hash=sha256:22ba7cfcad58ef3ecddc7ed1db3409af68d023b7f940da23c6c2a1890976eda6 \ + --hash=sha256:27c0abcb4a5dac13684a37f76e701e054692a9b2d3064b70f5e4eb54810553d7 \ + --hash=sha256:28c8d926f98f432f88adc23edf2e6d4921ac26fb084b028c733d01868d19007e \ + --hash=sha256:2e71d11abed7344e42a8849600193d15b6def118602c4c176f748e4583246007 \ + --hash=sha256:34d5fcd24b8445fadc33f9cf348c1047101756fd760b4dacb5c3e99755703310 \ + --hash=sha256:37503bfbfc9d2c40b344d06b2199cf0e96e97957ab1c1b546fd4f87e53e5d3e4 \ + --hash=sha256:3c5677e12444c15717b902a5798264fa7909e41153cdf9ef7ad571b704a63dd9 \ + --hash=sha256:3ff07ec89bae51176c0549bc4c63aa6202991da2d9a6129d7aef7f1407d3f295 \ + --hash=sha256:41715c910c881bc081f1e8872880d3c650acf13dfa8214bad49ed4cede7c34ea \ + --hash=sha256:418cf3f2111bc80e0933b2cd8cd04f286338bb88bdc7bc8e6dd775ebde60b5e0 \ + --hash=sha256:44edc647873928551a01e7a563d7452ccdebee747728c1080d881d68af7b997e \ + --hash=sha256:4a2e8cebe2ff6ab7d1050ecd59c25d4c8bd7e6f400f5f82b96557ac0abafd0ac \ + --hash=sha256:4ad1906908f2f5ae4e5a8ddfce73c320c2a1429ec52eafd27138b7f1cbe341c9 \ + --hash=sha256:501a031947e3a9025ed4405a168e6ef5ae3126c59f90ce0cd6f2bfc477be31b7 \ + --hash=sha256:5190d403f121660ce8d1d2c1bb2ef1bd05b5f68533fc5c2ea899bd15f4399b35 \ + --hash=sha256:5498cd1645aa724a7c71c8f378eb29ebe23da2fc0d7a08071d89469bf1d2defb \ + --hash=sha256:5cf4e27da7e3fbed4d6c3d8e797387aaad68102272f8f9752883bc32d61cb87b \ + --hash=sha256:5e0b74767e5f8c593e8c9b5912019159ed0533c70051e9cce3e8b6aa699fcd69 \ + --hash=sha256:5ed875a24292240029e4483f9d4a4b8a1ae08843b9c54f43fcc11e404532a8a5 \ + --hash=sha256:5fcd34e47f6e0b794d17de1b4ff496c00986e1c83f7ab2fb8fcfe9616ff7477b \ + --hash=sha256:5fdec68f91a0c6739b380c83b951e2c72ac0197ace422360e6d5a959d8d97b2c \ + --hash=sha256:6344df0d5755a2c9a276d4473ae6b90647e216ab4757f8426893b5dd2ac3f369 \ + --hash=sha256:64386e5e707d03a7e172c0701abfb7e10f0fb753ee1d773128192742712a98fd \ + --hash=sha256:652cb6edd41e718550aad172851962662ff2681490a8a711af6a4d288dd96824 \ + --hash=sha256:66291b10affd76d76f54fad28e22e51719ef9ba22b29e1d7d03d6777a9174198 \ + --hash=sha256:66e1674c3ef6f541c35191caae2d429b967b99e02040f5ba928632d9a7f0f065 \ + --hash=sha256:6adc77889b628398debc7b65c073bcb99c4a0237b248cacaf3fe8a557563ef6c \ + --hash=sha256:79005a0d97d5ddabfeeea4cf676af11e647e41d81c9a7722a193022accdb6b7c \ + --hash=sha256:7c6610def4f163542a622a73fb39f534f8c101d690126992300bf3207eab9764 \ + --hash=sha256:7f047e29dcae44602496db43be01ad42fc6f1cc0d8cd6c83d342306c32270196 \ + --hash=sha256:8098f252adfa6c80ab48096053f512f2321f0b998f98150cea9bd23d83e1467b \ + --hash=sha256:850774a7879607d3a6f50d36d04f00ee69e7fc816450e5f7e58d7f17f1ae5c00 \ + --hash=sha256:8d1fab6bb153a416f9aeb4b8763bc0f22a5586065f86f7664fc23339fc1c1fac \ + --hash=sha256:8da9669d359f02c0b91ccc01cac4a67f16afec0dac22c2ad09f46bee0697eba8 \ + --hash=sha256:8dc52c23056b9ddd46818a57b78404882310fb473d63f17b07d5c40421e47f8e \ + --hash=sha256:9149cad251584d5fb4981be1ecde53a1ca46c891a79788c0df828d2f166bda28 \ + --hash=sha256:93dda82c9c22deb0a405ea4dc5f2d0cda384168e466364dec6255b293923b2f3 \ + --hash=sha256:96b533f0e99f6579b3d4d4995707cf36df9100d67e0c8303a0c55b27b5f99bc5 \ + --hash=sha256:9c57bb8c96f6d1808c030b1687b9b5fb476abaa47f0db9c0101f5e9f394e97f4 \ + --hash=sha256:9c7708761fccb9397fe64bbc0395abcae8c4bf7b0eac081e12b809bf47700d0b \ + --hash=sha256:9f3bfb4965eb874431221a3ff3fdcddc7e74e3b07799e0e84ca4a0f867d449bf \ + --hash=sha256:a33284e20b78bd4a18c8c2282d549d10bc8408a2a7ff57653c0cf0b9be0afce5 \ + --hash=sha256:a80cb027f6b349846a3bf6d73b5e95e782175e52f22108cfa17876aaeff93702 \ + --hash=sha256:b30236e45cf30d2b8e7b3e85881719e98507abed1011bf463a8fa23e9c3e98a8 \ + --hash=sha256:b3bc83488de33889877a0f2543ade9f70c67d66d9ebb4ac959502e12de895788 \ + --hash=sha256:b865addae83924361678b652338317d1bd7e79b1f4596f96b96c77a5a34b34da \ + --hash=sha256:b8bb0864c5a28024fac8a632c443c87c5aa6f215c0b126c449ae1a150412f31d \ + --hash=sha256:ba1cc08a7ccde2d2ec775841541641e4548226580ab850948cbfda66a1befcdc \ + --hash=sha256:bdb2c67c6c1390b63c6ff89f210c8fd09d9a1217a465701eac7316313c915e4c \ + --hash=sha256:c1ff362665ae507275af2853520967820d9124984e0f7466736aea23d8611fba \ + --hash=sha256:c2514fceb77bc5e7a2f7adfaa1feb2fb311607c9cb518dbc378688ec73d8292f \ + --hash=sha256:c3355370a2c156cffb25e876646f149d5d68f5e0a3ce86a5084dd0b64a994917 \ + --hash=sha256:c458b6d084f9b935061bc36216e8a69a7e293a2f1e68bf956dcd9e6cbcd143f5 \ + --hash=sha256:d0eae10f8159e8fdad514efdc92d74fd8d682c933a6dd088030f3834bc8e6b26 \ + --hash=sha256:d76623373421df22fb4cf8817020cbb7ef15c725b9d5e45f17e189bfc384190f \ + --hash=sha256:ebc55a14a21cb14062aa4162f906cd962b28e2e9ea38f9b4391244cd8de4ae0b \ + --hash=sha256:eda16858a3cab07b80edaf74336ece1f986ba330fdb8ee0d6c0d68fe82bc96be \ + --hash=sha256:ee2922902c45ae8ccada2c5b501ab86c36525b883eff4255313a253a3160861c \ + --hash=sha256:efd7b85f94a6f21e4932043973a7ba2613b059c4a000551892ac9f1d11f5baf3 \ + --hash=sha256:f7057c9a337546edc7973c0d3ba84ddcdf0daa14533c2065749c9075001090e6 \ + --hash=sha256:fa160448684b4e94d80416c0fa4aac48967a969efe22931448d853ada8baf926 \ + --hash=sha256:fc09d0aa354569bc501d4e787133afc08552722d3ab34836a80547331bb5d4a0 + # via yamllint +requests==2.25.1 \ + --hash=sha256:27973dd4a904a4f13b263a19c866c13b92a39ed1c964655f025f3f8d3d75b804 \ + --hash=sha256:c210084e36a42ae6b9219e00e48287def368a26d03a048ddad7bfee44f75871e + # via + # -r examples/bzlmod/requirements.in + # sphinx +s3cmd==2.1.0 \ + --hash=sha256:49cd23d516b17974b22b611a95ce4d93fe326feaa07320bd1d234fed68cbccfa \ + --hash=sha256:966b0a494a916fc3b4324de38f089c86c70ee90e8e1cae6d59102103a4c0cc03 + # via -r examples/bzlmod/requirements.in +six==1.17.0 \ + --hash=sha256:4721f391ed90541fddacab5acf947aa0d3dc7d27b2e1e8eda2be8970586c3274 \ + --hash=sha256:ff70335d468e7eb6ec65b95b99d3a2836546063f63acc5171de367e834932a81 + # via python-dateutil +snowballstemmer==3.0.1 \ + --hash=sha256:6cd7b3897da8d6c9ffb968a6781fa6532dce9c3618a4b127d920dab764a19064 \ + --hash=sha256:6d5eeeec8e9f84d4d56b847692bacf79bc2c8e90c7f80ca4444ff8b6f2e52895 + # via sphinx +sphinx==7.3.7 \ + --hash=sha256:413f75440be4cacf328f580b4274ada4565fb2187d696a84970c23f77b64d8c3 \ + --hash=sha256:a4a7db75ed37531c05002d56ed6948d4c42f473a36f46e1382b0bd76ca9627bc + # via -r examples/bzlmod/requirements.in +sphinxcontrib-applehelp==2.0.0 \ + --hash=sha256:2f29ef331735ce958efa4734873f084941970894c6090408b079c61b2e1c06d1 \ + --hash=sha256:4cd3f0ec4ac5dd9c17ec65e9ab272c9b867ea77425228e68ecf08d6b28ddbdb5 + # via sphinx +sphinxcontrib-devhelp==2.0.0 \ + --hash=sha256:411f5d96d445d1d73bb5d52133377b4248ec79db5c793ce7dbe59e074b4dd1ad \ + --hash=sha256:aefb8b83854e4b0998877524d1029fd3e6879210422ee3780459e28a1f03a8a2 + # via sphinx +sphinxcontrib-htmlhelp==2.1.0 \ + --hash=sha256:166759820b47002d22914d64a075ce08f4c46818e17cfc9470a9786b759b19f8 \ + --hash=sha256:c9e2916ace8aad64cc13a0d233ee22317f2b9025b9cf3295249fa985cc7082e9 + # via sphinx +sphinxcontrib-jsmath==1.0.1 \ + --hash=sha256:2ec2eaebfb78f3f2078e73666b1415417a116cc848b72e5172e596c871103178 \ + --hash=sha256:a9925e4a4587247ed2191a22df5f6970656cb8ca2bd6284309578f2153e0c4b8 + # via sphinx +sphinxcontrib-qthelp==2.0.0 \ + --hash=sha256:4fe7d0ac8fc171045be623aba3e2a8f613f8682731f9153bb2e40ece16b9bbab \ + --hash=sha256:b18a828cdba941ccd6ee8445dbe72ffa3ef8cbe7505d8cd1fa0d42d3f2d5f3eb + # via sphinx +sphinxcontrib-serializinghtml==2.0.0 \ + --hash=sha256:6e2cb0eef194e10c27ec0023bfeb25badbbb5868244cf5bc5bdc04e4464bf331 \ + --hash=sha256:e9d912827f872c029017a53f0ef2180b327c3f7fd23c87229f7a8e8b70031d4d + # via + # -r examples/bzlmod/requirements.in + # sphinx +tabulate==0.9.0 \ + --hash=sha256:0095b12bf5966de529c0feb1fa08671671b3368eec77d7ef7ab114be2c068b3c \ + --hash=sha256:024ca478df22e9340661486f85298cff5f6dcdba14f3813e8830015b9ed1948f + # via -r examples/bzlmod/requirements.in +tomlkit==0.13.3 \ + --hash=sha256:430cf247ee57df2b94ee3fbe588e71d362a941ebb545dec29b53961d61add2a1 \ + --hash=sha256:c89c649d79ee40629a9fda55f8ace8c6a1b42deb912b2a8fd8d942ddadb606b0 + # via pylint +urllib3==1.26.20 \ + --hash=sha256:0ed14ccfbf1c30a9072c7ca157e4319b70d65f623e91e7b32fadb2853431016e \ + --hash=sha256:40c2dc0c681e47eb8f90e7e27bf6ff7df2e677421fd46756da1161c39ca70d32 + # via requests +websockets==15.0.1 \ + --hash=sha256:0701bc3cfcb9164d04a14b149fd74be7347a530ad3bbf15ab2c678a2cd3dd9a2 \ + --hash=sha256:0a34631031a8f05657e8e90903e656959234f3a04552259458aac0b0f9ae6fd9 \ + --hash=sha256:0af68c55afbd5f07986df82831c7bff04846928ea8d1fd7f30052638788bc9b5 \ + --hash=sha256:0c9e74d766f2818bb95f84c25be4dea09841ac0f734d1966f415e4edfc4ef1c3 \ + --hash=sha256:0f3c1e2ab208db911594ae5b4f79addeb3501604a165019dd221c0bdcabe4db8 \ + --hash=sha256:0fdfe3e2a29e4db3659dbd5bbf04560cea53dd9610273917799f1cde46aa725e \ + --hash=sha256:1009ee0c7739c08a0cd59de430d6de452a55e42d6b522de7aa15e6f67db0b8e1 \ + --hash=sha256:1234d4ef35db82f5446dca8e35a7da7964d02c127b095e172e54397fb6a6c256 \ + --hash=sha256:16b6c1b3e57799b9d38427dda63edcbe4926352c47cf88588c0be4ace18dac85 \ + --hash=sha256:2034693ad3097d5355bfdacfffcbd3ef5694f9718ab7f29c29689a9eae841880 \ + --hash=sha256:21c1fa28a6a7e3cbdc171c694398b6df4744613ce9b36b1a498e816787e28123 \ + --hash=sha256:229cf1d3ca6c1804400b0a9790dc66528e08a6a1feec0d5040e8b9eb14422375 \ + --hash=sha256:27ccee0071a0e75d22cb35849b1db43f2ecd3e161041ac1ee9d2352ddf72f065 \ + --hash=sha256:363c6f671b761efcb30608d24925a382497c12c506b51661883c3e22337265ed \ + --hash=sha256:39c1fec2c11dc8d89bba6b2bf1556af381611a173ac2b511cf7231622058af41 \ + --hash=sha256:3b1ac0d3e594bf121308112697cf4b32be538fb1444468fb0a6ae4feebc83411 \ + --hash=sha256:3be571a8b5afed347da347bfcf27ba12b069d9d7f42cb8c7028b5e98bbb12597 \ + --hash=sha256:3c714d2fc58b5ca3e285461a4cc0c9a66bd0e24c5da9911e30158286c9b5be7f \ + --hash=sha256:3d00075aa65772e7ce9e990cab3ff1de702aa09be3940d1dc88d5abf1ab8a09c \ + --hash=sha256:3e90baa811a5d73f3ca0bcbf32064d663ed81318ab225ee4f427ad4e26e5aff3 \ + --hash=sha256:47819cea040f31d670cc8d324bb6435c6f133b8c7a19ec3d61634e62f8d8f9eb \ + --hash=sha256:47b099e1f4fbc95b701b6e85768e1fcdaf1630f3cbe4765fa216596f12310e2e \ + --hash=sha256:4a9fac8e469d04ce6c25bb2610dc535235bd4aa14996b4e6dbebf5e007eba5ee \ + --hash=sha256:4b826973a4a2ae47ba357e4e82fa44a463b8f168e1ca775ac64521442b19e87f \ + --hash=sha256:4c2529b320eb9e35af0fa3016c187dffb84a3ecc572bcee7c3ce302bfeba52bf \ + --hash=sha256:54479983bd5fb469c38f2f5c7e3a24f9a4e70594cd68cd1fa6b9340dadaff7cf \ + --hash=sha256:558d023b3df0bffe50a04e710bc87742de35060580a293c2a984299ed83bc4e4 \ + --hash=sha256:5756779642579d902eed757b21b0164cd6fe338506a8083eb58af5c372e39d9a \ + --hash=sha256:592f1a9fe869c778694f0aa806ba0374e97648ab57936f092fd9d87f8bc03665 \ + --hash=sha256:595b6c3969023ecf9041b2936ac3827e4623bfa3ccf007575f04c5a6aa318c22 \ + --hash=sha256:5a939de6b7b4e18ca683218320fc67ea886038265fd1ed30173f5ce3f8e85675 \ + --hash=sha256:5d54b09eba2bada6011aea5375542a157637b91029687eb4fdb2dab11059c1b4 \ + --hash=sha256:5df592cd503496351d6dc14f7cdad49f268d8e618f80dce0cd5a36b93c3fc08d \ + --hash=sha256:5f4c04ead5aed67c8a1a20491d54cdfba5884507a48dd798ecaf13c74c4489f5 \ + --hash=sha256:64dee438fed052b52e4f98f76c5790513235efaa1ef7f3f2192c392cd7c91b65 \ + --hash=sha256:66dd88c918e3287efc22409d426c8f729688d89a0c587c88971a0faa2c2f3792 \ + --hash=sha256:678999709e68425ae2593acf2e3ebcbcf2e69885a5ee78f9eb80e6e371f1bf57 \ + --hash=sha256:67f2b6de947f8c757db2db9c71527933ad0019737ec374a8a6be9a956786aaf9 \ + --hash=sha256:693f0192126df6c2327cce3baa7c06f2a117575e32ab2308f7f8216c29d9e2e3 \ + --hash=sha256:746ee8dba912cd6fc889a8147168991d50ed70447bf18bcda7039f7d2e3d9151 \ + --hash=sha256:756c56e867a90fb00177d530dca4b097dd753cde348448a1012ed6c5131f8b7d \ + --hash=sha256:76d1f20b1c7a2fa82367e04982e708723ba0e7b8d43aa643d3dcd404d74f1475 \ + --hash=sha256:7f493881579c90fc262d9cdbaa05a6b54b3811c2f300766748db79f098db9940 \ + --hash=sha256:823c248b690b2fd9303ba00c4f66cd5e2d8c3ba4aa968b2779be9532a4dad431 \ + --hash=sha256:82544de02076bafba038ce055ee6412d68da13ab47f0c60cab827346de828dee \ + --hash=sha256:8dd8327c795b3e3f219760fa603dcae1dcc148172290a8ab15158cf85a953413 \ + --hash=sha256:8fdc51055e6ff4adeb88d58a11042ec9a5eae317a0a53d12c062c8a8865909e8 \ + --hash=sha256:a625e06551975f4b7ea7102bc43895b90742746797e2e14b70ed61c43a90f09b \ + --hash=sha256:abdc0c6c8c648b4805c5eacd131910d2a7f6455dfd3becab248ef108e89ab16a \ + --hash=sha256:ac017dd64572e5c3bd01939121e4d16cf30e5d7e110a119399cf3133b63ad054 \ + --hash=sha256:ac1e5c9054fe23226fb11e05a6e630837f074174c4c2f0fe442996112a6de4fb \ + --hash=sha256:ac60e3b188ec7574cb761b08d50fcedf9d77f1530352db4eef1707fe9dee7205 \ + --hash=sha256:b359ed09954d7c18bbc1680f380c7301f92c60bf924171629c5db97febb12f04 \ + --hash=sha256:b7643a03db5c95c799b89b31c036d5f27eeb4d259c798e878d6937d71832b1e4 \ + --hash=sha256:ba9e56e8ceeeedb2e080147ba85ffcd5cd0711b89576b83784d8605a7df455fa \ + --hash=sha256:c338ffa0520bdb12fbc527265235639fb76e7bc7faafbb93f6ba80d9c06578a9 \ + --hash=sha256:cad21560da69f4ce7658ca2cb83138fb4cf695a2ba3e475e0559e05991aa8122 \ + --hash=sha256:d08eb4c2b7d6c41da6ca0600c077e93f5adcfd979cd777d747e9ee624556da4b \ + --hash=sha256:d50fd1ee42388dcfb2b3676132c78116490976f1300da28eb629272d5d93e905 \ + --hash=sha256:d591f8de75824cbb7acad4e05d2d710484f15f29d4a915092675ad3456f11770 \ + --hash=sha256:d5f6b181bb38171a8ad1d6aa58a67a6aa9d4b38d0f8c5f496b9e42561dfc62fe \ + --hash=sha256:d63efaa0cd96cf0c5fe4d581521d9fa87744540d4bc999ae6e08595a1014b45b \ + --hash=sha256:d99e5546bf73dbad5bf3547174cd6cb8ba7273062a23808ffea025ecb1cf8562 \ + --hash=sha256:e09473f095a819042ecb2ab9465aee615bd9c2028e4ef7d933600a8401c79561 \ + --hash=sha256:e8b56bdcdb4505c8078cb6c7157d9811a85790f2f2b3632c7d1462ab5783d215 \ + --hash=sha256:ee443ef070bb3b6ed74514f5efaa37a252af57c90eb33b956d35c8e9c10a1931 \ + --hash=sha256:f29d80eb9a9263b8d109135351caf568cc3f80b9928bccde535c235de55c22d9 \ + --hash=sha256:f7a866fbc1e97b5c617ee4116daaa09b722101d4a3c170c787450ba409f9736f \ + --hash=sha256:fcd5cf9e305d7b8338754470cf69cf81f420459dbae8a3b40cee57417f4614a7 + # via -r examples/bzlmod/requirements.in +wheel==0.45.1 \ + --hash=sha256:661e1abd9198507b1409a20c02106d9670b2576e916d58f520316666abca6729 \ + --hash=sha256:708e7481cc80179af0e556bbf0cc00b8444c7321e2700b8d8580231d13017248 + # via -r examples/bzlmod/requirements.in +wrapt==1.17.3 \ + --hash=sha256:02b551d101f31694fc785e58e0720ef7d9a10c4e62c1c9358ce6f63f23e30a56 \ + --hash=sha256:042ec3bb8f319c147b1301f2393bc19dba6e176b7da446853406d041c36c7828 \ + --hash=sha256:0610b46293c59a3adbae3dee552b648b984176f8562ee0dba099a56cfbe4df1f \ + --hash=sha256:0b02e424deef65c9f7326d8c19220a2c9040c51dc165cddb732f16198c168396 \ + --hash=sha256:0b1831115c97f0663cb77aa27d381237e73ad4f721391a9bfb2fe8bc25fa6e77 \ + --hash=sha256:0ed61b7c2d49cee3c027372df5809a59d60cf1b6c2f81ee980a091f3afed6a2d \ + --hash=sha256:0f5f51a6466667a5a356e6381d362d259125b57f059103dd9fdc8c0cf1d14139 \ + --hash=sha256:16ecf15d6af39246fe33e507105d67e4b81d8f8d2c6598ff7e3ca1b8a37213f7 \ + --hash=sha256:1f0b2f40cf341ee8cc1a97d51ff50dddb9fcc73241b9143ec74b30fc4f44f6cb \ + --hash=sha256:1f23fa283f51c890eda8e34e4937079114c74b4c81d2b2f1f1d94948f5cc3d7f \ + --hash=sha256:223db574bb38637e8230eb14b185565023ab624474df94d2af18f1cdb625216f \ + --hash=sha256:249f88ed15503f6492a71f01442abddd73856a0032ae860de6d75ca62eed8067 \ + --hash=sha256:24c2ed34dc222ed754247a2702b1e1e89fdbaa4016f324b4b8f1a802d4ffe87f \ + --hash=sha256:273a736c4645e63ac582c60a56b0acb529ef07f78e08dc6bfadf6a46b19c0da7 \ + --hash=sha256:281262213373b6d5e4bb4353bc36d1ba4084e6d6b5d242863721ef2bf2c2930b \ + --hash=sha256:30ce38e66630599e1193798285706903110d4f057aab3168a34b7fdc85569afc \ + --hash=sha256:33486899acd2d7d3066156b03465b949da3fd41a5da6e394ec49d271baefcf05 \ + --hash=sha256:343e44b2a8e60e06a7e0d29c1671a0d9951f59174f3709962b5143f60a2a98bd \ + --hash=sha256:373342dd05b1d07d752cecbec0c41817231f29f3a89aa8b8843f7b95992ed0c7 \ + --hash=sha256:3af60380ba0b7b5aeb329bc4e402acd25bd877e98b3727b0135cb5c2efdaefe9 \ + --hash=sha256:3e62d15d3cfa26e3d0788094de7b64efa75f3a53875cdbccdf78547aed547a81 \ + --hash=sha256:41b1d2bc74c2cac6f9074df52b2efbef2b30bdfe5f40cb78f8ca22963bc62977 \ + --hash=sha256:423ed5420ad5f5529db9ce89eac09c8a2f97da18eb1c870237e84c5a5c2d60aa \ + --hash=sha256:46acc57b331e0b3bcb3e1ca3b421d65637915cfcd65eb783cb2f78a511193f9b \ + --hash=sha256:4da9f45279fff3543c371d5ababc57a0384f70be244de7759c85a7f989cb4ebe \ + --hash=sha256:507553480670cab08a800b9463bdb881b2edeed77dc677b0a5915e6106e91a58 \ + --hash=sha256:53e5e39ff71b3fc484df8a522c933ea2b7cdd0d5d15ae82e5b23fde87d44cbd8 \ + --hash=sha256:54a30837587c6ee3cd1a4d1c2ec5d24e77984d44e2f34547e2323ddb4e22eb77 \ + --hash=sha256:5531d911795e3f935a9c23eb1c8c03c211661a5060aab167065896bbf62a5f85 \ + --hash=sha256:55cbbc356c2842f39bcc553cf695932e8b30e30e797f961860afb308e6b1bb7c \ + --hash=sha256:59923aa12d0157f6b82d686c3fd8e1166fa8cdfb3e17b42ce3b6147ff81528df \ + --hash=sha256:5a03a38adec8066d5a37bea22f2ba6bbf39fcdefbe2d91419ab864c3fb515454 \ + --hash=sha256:5a7b3c1ee8265eb4c8f1b7d29943f195c00673f5ab60c192eba2d4a7eae5f46a \ + --hash=sha256:5d4478d72eb61c36e5b446e375bbc49ed002430d17cdec3cecb36993398e1a9e \ + --hash=sha256:5ea5eb3c0c071862997d6f3e02af1d055f381b1d25b286b9d6644b79db77657c \ + --hash=sha256:604d076c55e2fdd4c1c03d06dc1a31b95130010517b5019db15365ec4a405fc6 \ + --hash=sha256:656873859b3b50eeebe6db8b1455e99d90c26ab058db8e427046dbc35c3140a5 \ + --hash=sha256:65d1d00fbfb3ea5f20add88bbc0f815150dbbde3b026e6c24759466c8b5a9ef9 \ + --hash=sha256:6b538e31eca1a7ea4605e44f81a48aa24c4632a277431a6ed3f328835901f4fd \ + --hash=sha256:6fd1ad24dc235e4ab88cda009e19bf347aabb975e44fd5c2fb22a3f6e4141277 \ + --hash=sha256:70d86fa5197b8947a2fa70260b48e400bf2ccacdcab97bb7de47e3d1e6312225 \ + --hash=sha256:7171ae35d2c33d326ac19dd8facb1e82e5fd04ef8c6c0e394d7af55a55051c22 \ + --hash=sha256:73d496de46cd2cdbdbcce4ae4bcdb4afb6a11234a1df9c085249d55166b95116 \ + --hash=sha256:7425ac3c54430f5fc5e7b6f41d41e704db073309acfc09305816bc6a0b26bb16 \ + --hash=sha256:74afa28374a3c3a11b3b5e5fca0ae03bef8450d6aa3ab3a1e2c30e3a75d023dc \ + --hash=sha256:758895b01d546812d1f42204bd443b8c433c44d090248bf22689df673ccafe00 \ + --hash=sha256:79573c24a46ce11aab457b472efd8d125e5a51da2d1d24387666cd85f54c05b2 \ + --hash=sha256:7e18f01b0c3e4a07fe6dfdb00e29049ba17eadbc5e7609a2a3a4af83ab7d710a \ + --hash=sha256:88547535b787a6c9ce4086917b6e1d291aa8ed914fdd3a838b3539dc95c12804 \ + --hash=sha256:88bbae4d40d5a46142e70d58bf664a89b6b4befaea7b2ecc14e03cedb8e06c04 \ + --hash=sha256:8cccf4f81371f257440c88faed6b74f1053eef90807b77e31ca057b2db74edb1 \ + --hash=sha256:9baa544e6acc91130e926e8c802a17f3b16fbea0fd441b5a60f5cf2cc5c3deba \ + --hash=sha256:a36692b8491d30a8c75f1dfee65bef119d6f39ea84ee04d9f9311f83c5ad9390 \ + --hash=sha256:a47681378a0439215912ef542c45a783484d4dd82bac412b71e59cf9c0e1cea0 \ + --hash=sha256:a7c06742645f914f26c7f1fa47b8bc4c91d222f76ee20116c43d5ef0912bba2d \ + --hash=sha256:a9a2203361a6e6404f80b99234fe7fb37d1fc73487b5a78dc1aa5b97201e0f22 \ + --hash=sha256:ab232e7fdb44cdfbf55fc3afa31bcdb0d8980b9b95c38b6405df2acb672af0e0 \ + --hash=sha256:ad85e269fe54d506b240d2d7b9f5f2057c2aa9a2ea5b32c66f8902f768117ed2 \ + --hash=sha256:af338aa93554be859173c39c85243970dc6a289fa907402289eeae7543e1ae18 \ + --hash=sha256:afd964fd43b10c12213574db492cb8f73b2f0826c8df07a68288f8f19af2ebe6 \ + --hash=sha256:b32888aad8b6e68f83a8fdccbf3165f5469702a7544472bdf41f582970ed3311 \ + --hash=sha256:c31eebe420a9a5d2887b13000b043ff6ca27c452a9a22fa71f35f118e8d4bf89 \ + --hash=sha256:caea3e9c79d5f0d2c6d9ab96111601797ea5da8e6d0723f77eabb0d4068d2b2f \ + --hash=sha256:cf30f6e3c077c8e6a9a7809c94551203c8843e74ba0c960f4a98cd80d4665d39 \ + --hash=sha256:d40770d7c0fd5cbed9d84b2c3f2e156431a12c9a37dc6284060fb4bec0b7ffd4 \ + --hash=sha256:d8a210b158a34164de8bb68b0e7780041a903d7b00c87e906fb69928bf7890d5 \ + --hash=sha256:dc4a8d2b25efb6681ecacad42fca8859f88092d8732b170de6a5dddd80a1c8fa \ + --hash=sha256:df7d30371a2accfe4013e90445f6388c570f103d61019b6b7c57e0265250072a \ + --hash=sha256:e01375f275f010fcbf7f643b4279896d04e571889b8a5b3f848423d91bf07050 \ + --hash=sha256:e1a4120ae5705f673727d3253de3ed0e016f7cd78dc463db1b31e2463e1f3cf6 \ + --hash=sha256:e228514a06843cae89621384cfe3a80418f3c04aadf8a3b14e46a7be704e4235 \ + --hash=sha256:e405adefb53a435f01efa7ccdec012c016b5a1d3f35459990afc39b6be4d5056 \ + --hash=sha256:e6b13af258d6a9ad602d57d889f83b9d5543acd471eee12eb51f5b01f8eb1bc2 \ + --hash=sha256:e6f40a8aa5a92f150bdb3e1c44b7e98fb7113955b2e5394122fa5532fec4b418 \ + --hash=sha256:e71d5c6ebac14875668a1e90baf2ea0ef5b7ac7918355850c0908ae82bcb297c \ + --hash=sha256:ed7c635ae45cfbc1a7371f708727bf74690daedc49b4dba310590ca0bd28aa8a \ + --hash=sha256:f38e60678850c42461d4202739f9bf1e3a737c7ad283638251e79cc49effb6b6 \ + --hash=sha256:f66eb08feaa410fe4eebd17f2a2c8e2e46d3476e9f8c783daa8e09e0faa666d0 \ + --hash=sha256:f9b2601381be482f70e5d1051a5965c25fb3625455a2bf520b5a077b22afb775 \ + --hash=sha256:fbd3c8319de8e1dc79d346929cd71d523622da527cca14e0c1d257e31c2b8b10 \ + --hash=sha256:fd341868a4b6714a5962c1af0bd44f7c404ef78720c7de4892901e540417111c + # via astroid +yamllint==1.37.1 \ + --hash=sha256:364f0d79e81409f591e323725e6a9f4504c8699ddf2d7263d8d2b539cd66a583 \ + --hash=sha256:81f7c0c5559becc8049470d86046b36e96113637bcbe4753ecef06977c00245d + # via -r examples/bzlmod/requirements.in From 7eebd602432332e4c5b11439a6d85cb600dc3baf Mon Sep 17 00:00:00 2001 From: Alex Trotta <44127594+Ahajha@users.noreply.github.com> Date: Sat, 11 Oct 2025 00:20:56 -0400 Subject: [PATCH 482/922] feat(toolchains): Add latest Python versions (#3336) Per request in https://github.com/bazel-contrib/rules_python/pull/3330#discussion_r2412231508 --------- Co-authored-by: Ignas Anikevicius <240938+aignas@users.noreply.github.com> --- CHANGELOG.md | 5 +- python/versions.bzl | 157 ++++++++++++++++++++++++++++------ tests/python/python_tests.bzl | 20 ++--- 3 files changed, 142 insertions(+), 40 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index a04ed04ffa..a86dd01a9f 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -83,7 +83,10 @@ END_UNRELEASED_TEMPLATE vendoring the `requirements.bzl` file, please re-vendor so that downstream is unaffected when the APIs get removed. If you need to customize the way the dependencies get evaluated, see [our docs](/pypi/download.html#customizing-requires-dist-resolution) on customizing `Requires-Dist` resolution. -* (toolchains) Added Python version 3.14.0. +* (toolchains) Added Python versions 3.14.0, 3.13.8, 3.12.12, 3.11.14, 3.10.19, and 3.9.24 + from the [20251010] release. + +[20251010]: https://github.com/astral-sh/python-build-standalone/releases/tag/20251010 {#v0-0-0-fixed} ### Fixed diff --git a/python/versions.bzl b/python/versions.bzl index f43e6b6dba..ebd4d7f718 100644 --- a/python/versions.bzl +++ b/python/versions.bzl @@ -204,6 +204,21 @@ TOOL_VERSIONS = { }, "strip_prefix": "python", }, + "3.9.24": { + "url": "20251010/cpython-{python_version}+20251010-{platform}-{build}.tar.gz", + "sha256": { + "aarch64-apple-darwin": "03af9b83cdda23c8b82537f720cc37ddc0d5635b282c8ea8326382f3f0221141", + "aarch64-unknown-linux-gnu": "e1aff69545ab15dc22ffe9f28c8d162af0e4be9a9cf57c0a50a0916508964ccb", + "ppc64le-unknown-linux-gnu": "c70891539abf23746dcfb32043bf91c6f2dfcd97106ff1ed75b744f9e00b5551", + "riscv64-unknown-linux-gnu": "ce180e819d992fda0c523fba88db79b9d9bc9b12a52d548479d84c7b86d5bcda", + "s390x-unknown-linux-gnu": "9a830f9b8fa62962e22f5aa105f12fde9d40d7969d701e9600d580e4f0cb7ac6", + "x86_64-apple-darwin": "aabb592ba83217c092744f53c14da10b6e3dab6f96e0eefd346e6ac0c0723dd2", + "x86_64-pc-windows-msvc": "28fd634e9e20f2a94e29e8c8c8a5384a2882091e9d0af981e7e99cc156be1648", + "x86_64-unknown-linux-gnu": "fe63b51f543011e0582a98259d98fbc0033bc2009398404446f350fe9e531aa3", + "x86_64-unknown-linux-musl": "844db0ca41ccf897f58fbefd8f5df7ef18ea374788e6ce4cb87989f997c3f21a", + }, + "strip_prefix": "python", + }, "3.10.2": { "url": "20220227/cpython-{python_version}+20220227-{platform}-{build}.tar.gz", "sha256": { @@ -354,6 +369,21 @@ TOOL_VERSIONS = { }, "strip_prefix": "python", }, + "3.10.19": { + "url": "20251010/cpython-{python_version}+20251010-{platform}-{build}.tar.gz", + "sha256": { + "aarch64-apple-darwin": "4918c7b7dd9bdeb81b2a30b5305d85e04c6f322ed715b598bf7afbfd03b39f23", + "aarch64-unknown-linux-gnu": "356fce3bed7d4c416c552d054cf647886b3825b285a06b4b8782440dccc5f5c1", + "ppc64le-unknown-linux-gnu": "b129fcc438aee69429c1d90dd251579393eaf690d5cf9f327a41c79448ff062f", + "riscv64-unknown-linux-gnu": "118e4444dc8a98d7f52cca86b1f9a4636c45eba3d759689bd86f68c579fe03d0", + "s390x-unknown-linux-gnu": "d62e65e77199f35d54cae383977ccbf30fca1ba683f96e6ffc04a17d5d31e959", + "x86_64-apple-darwin": "90665ea564d409de944e7e985e9ddeecc1d61980616983d26b06bc721cfa8469", + "x86_64-pc-windows-msvc": "1e56b702e080723a76d92ba04d0f8b6e49986113b575c7026797d1de53a5b89b", + "x86_64-unknown-linux-gnu": "4f1c6812961ed57e408f2385a8b16745d3eb62b7c2e6e0eef7c05d37e4ba806e", + "x86_64-unknown-linux-musl": "e90c84c95440212e1c98d0bd7a54196746e886fdc8512ae7e8d6a7d46f058ef1", + }, + "strip_prefix": "python", + }, "3.11.1": { "url": "20230116/cpython-{python_version}+20230116-{platform}-{build}.tar.gz", "sha256": { @@ -485,6 +515,22 @@ TOOL_VERSIONS = { }, "strip_prefix": "python", }, + "3.11.14": { + "url": "20251010/cpython-{python_version}+20251010-{platform}-{build}.tar.gz", + "sha256": { + "aarch64-apple-darwin": "29ec457de1b5765eeade189efbe27e2b1f8c7a96e4b79471b7d41a18094b1870", + "aarch64-unknown-linux-gnu": "d46c18f9da8a673cc55de55d8cfb8ed3164849eac50edc222985b60a9eda3be3", + "ppc64le-unknown-linux-gnu": "fd395aa11d82a48bfe3cfdfcd41759ee4b65b3d1de67466329aaef284164650b", + "riscv64-unknown-linux-gnu": "c3b529408c176a222c863c1810ee6a635a7e9deb5f2c73c425181a2383d7da2a", + "s390x-unknown-linux-gnu": "ade377b4668e4a03bd0c2a5316c079b8c0d2d63db7ecd05f24715309c4efb298", + "x86_64-apple-darwin": "ae375cd49fecfc3aecbf942544c34a1c5251f2c2f9b19e0e2b889a9113ccfa62", + "x86_64-pc-windows-msvc": "58a2571b5268fc7891e28cab01f23c3561ef5bc146f1314e32ba32a06754442b", + "aarch64-pc-windows-msvc": "67600a84ba1cf43a826a31c1dbb144b987a6b4627f6e5fb3d34a54511c356187", + "x86_64-unknown-linux-gnu": "848789e630ada4012e64ac744a3f4a8342b975d69f9608c460bdf9f370fa1d30", + "x86_64-unknown-linux-musl": "fed38a53fdff4327295f052a2970692ca194e53444843b5f91bd478c7ffed1ba", + }, + "strip_prefix": "python", + }, "3.12.0": { "url": "20231002/cpython-{python_version}+20231002-{platform}-{build}.tar.gz", "sha256": { @@ -609,6 +655,22 @@ TOOL_VERSIONS = { }, "strip_prefix": "python", }, + "3.12.12": { + "url": "20251010/cpython-{python_version}+20251010-{platform}-{build}.tar.gz", + "sha256": { + "aarch64-apple-darwin": "2577a2629c89b3ff40dc16271cc8826d9ae20217e5a3189bbc7646b496e77687", + "aarch64-unknown-linux-gnu": "21bcf71dccb56ef611f50543b04e63e6585ac063463f2d248cb4ec28118d264d", + "aarch64-pc-windows-msvc": "a36719b442c22488f1ef7caea68943cf0c8dd367330fa5baac3cab4168a8e66a", + "ppc64le-unknown-linux-gnu": "782fbe38b63216cce3fdc4c7a2da86337ccef615fa9384acba7457ef17ad96a2", + "riscv64-unknown-linux-gnu": "c9bbb36a75466386497a38db0f9f707e63aaa8bae38c64d55683242d9b5b56a6", + "s390x-unknown-linux-gnu": "90e3f010692e65425cb503ae789e0ffefb3b5962b83e21d7c642d34cf056d48b", + "x86_64-apple-darwin": "c9fb9ab36c742f14388fd5b6a67cab5e6e1726b52624dafbd37d0176ad1752e1", + "x86_64-pc-windows-msvc": "c110c11de4299b0273caa99a1c7b895427c9441a231b3124bf5228a2e463ef43", + "x86_64-unknown-linux-gnu": "fbf55136d0f955ca2f93aeb7830f993d93acf1b9a729a15c3b6b5220a36bc835", + "x86_64-unknown-linux-musl": "bffd28435d28d55b33e0b90a5521b0220fe38c5ff67b2706cbc9ceb32af0e4a7", + }, + "strip_prefix": "python", + }, "3.13.0": { "url": "20241016/cpython-{python_version}+20241016-{platform}-{build}.{ext}", "sha256": { @@ -810,28 +872,73 @@ TOOL_VERSIONS = { "x86_64-unknown-linux-gnu-freethreaded": "python/install", }, }, + "3.13.8": { + "url": "20251010/cpython-{python_version}+20251010-{platform}-{build}.{ext}", + "sha256": { + "aarch64-apple-darwin": "0dc9061b9d8e02a9344aa569eecb795f41f16ac8bb215f973d8db9179700e296", + "aarch64-unknown-linux-gnu": "1b9af3c628cdbef3b9eb4c61df628bbc54f4684ace147f3dc9850c336c44c125", + "ppc64le-unknown-linux-gnu": "c3bae08423519b5224bfc87323b96381bdb9c589fbdc09cd3ce289f6c1ea4ed3", + "riscv64-unknown-linux-gnu": "e8be781bfaf5ad6b83094db96dd60d7dfcd4519f6550740354b4ea57acfbfa21", + "s390x-unknown-linux-gnu": "3c9faf91356d06b4cc993487a71915e7a0585aa592691f6afe8841188e024653", + "x86_64-apple-darwin": "e3b692599bb3c247a2f95c21577f8b85b70924a5f2d672c9e0005608d7b9c907", + "x86_64-pc-windows-msvc": "1dde7aab47a52e81b6bd3f7d1bc5fa2f3c9e428eb5f54f51e8b92ae0a3e2409f", + "aarch64-pc-windows-msvc": "430d9073f22c744d6ea8c224d7b458a6be3620f9bd2389068908c214ea4423f8", + "aarch64-pc-windows-msvc-freethreaded": "187b962d84af18d22f67e069776dff61a223271ae11f44507cfd8693430c03c4", + "x86_64-unknown-linux-gnu": "12dd8995e8ec2df68cd1301b053f415c7884b8aae9d3459a2ac1448f781dbbbc", + "x86_64-unknown-linux-musl": "fbf17b5acd1a33a7a0a6bcabab7065f3adc3de77efbc230555a5019843070b1d", + "aarch64-apple-darwin-freethreaded": "0b7a80716c2557800d8e3133744b9e913476d259cea0160dd8597e58284a5a1c", + "aarch64-unknown-linux-gnu-freethreaded": "5cf655065b59493d39235fec0e30c33a5196bb01cf74cef50907a4f18c8ef02a", + "ppc64le-unknown-linux-gnu-freethreaded": "dbae65b6e747537939c3f2804077bb4e49f528850e68800d54b0e8c4b7768fab", + "riscv64-unknown-linux-gnu-freethreaded": "10390b394f76cb14c808b5c87de37189bc72796e55a55e6e5aca83e4615dd5ca", + "s390x-unknown-linux-gnu-freethreaded": "7e9d32fae045cefe950d8f95ddf1824edc232d1c67b4cdd55ecec5042f05727d", + "x86_64-apple-darwin-freethreaded": "1112caf275e374fbe629c3fde265f1e7675b21553f1607085cedc821d184a4aa", + "x86_64-pc-windows-msvc-freethreaded": "d4b83250fe9fea9563b2fb79111d4af072279b690507ddb14df11f617fd57f7e", + "x86_64-unknown-linux-gnu-freethreaded": "714eec8f42eb023d0c9ed289115d52e96ce73dac7464a5d242b0988eeda90b56", + }, + "strip_prefix": { + "aarch64-apple-darwin": "python", + "aarch64-unknown-linux-gnu": "python", + "ppc64le-unknown-linux-gnu": "python", + "s390x-unknown-linux-gnu": "python", + "riscv64-unknown-linux-gnu": "python", + "x86_64-apple-darwin": "python", + "x86_64-pc-windows-msvc": "python", + "aarch64-pc-windows-msvc": "python", + "x86_64-unknown-linux-gnu": "python", + "x86_64-unknown-linux-musl": "python", + "aarch64-apple-darwin-freethreaded": "python/install", + "aarch64-unknown-linux-gnu-freethreaded": "python/install", + "ppc64le-unknown-linux-gnu-freethreaded": "python/install", + "riscv64-unknown-linux-gnu-freethreaded": "python/install", + "s390x-unknown-linux-gnu-freethreaded": "python/install", + "x86_64-apple-darwin-freethreaded": "python/install", + "x86_64-pc-windows-msvc-freethreaded": "python/install", + "aarch64-pc-windows-msvc-freethreaded": "python/install", + "x86_64-unknown-linux-gnu-freethreaded": "python/install", + }, + }, "3.14.0": { - "url": "20251007/cpython-{python_version}+20251007-{platform}-{build}.{ext}", + "url": "20251010/cpython-{python_version}+20251010-{platform}-{build}.{ext}", "sha256": { - "aarch64-apple-darwin": "41c502cf32d650673bfbee35f73c9140897dd26c43b97da1177cee00f40033fb", - "aarch64-unknown-linux-gnu": "7b4fc36ee88ec693fcf7ac696bc018a8254a1f166f4cd5f6a352d5432cb5836a", - "ppc64le-unknown-linux-gnu": "e5df0738e3f7da9977d6b789fad0b3e8ccc117a3337bf6d4de673cd6472239c8", - "riscv64-unknown-linux-gnu": "cfff02bd9b3d6c64e2eacf725557599ce17f65e30776f41c0643613cbcf2042e", - "s390x-unknown-linux-gnu": "e2cbe581954685ae0a77206c8318c351e3a9d99b28924e3527610e76487c6201", - "x86_64-apple-darwin": "543accfe71df014a08295a4bbaa4e4cf2b80ab2977ec362e38be24c36076d7fe", - "x86_64-pc-windows-msvc": "77cd2c0e167726e0476e35c7e483cf2f05172dff2326e1c4bf9887aff8353b2f", - "aarch64-pc-windows-msvc": "52434459d376f3fc272596d7b5f97b2248e51362a6157091f9d64e630ddd8fdd", - "x86_64-unknown-linux-gnu": "8203b9355b605ad80be6f1aa467226cfbd55b9839063c173c494de5e69c4a722", - "x86_64-unknown-linux-musl": "6a0350e642dddc6c54f568c08239ca7af08cf8621d5797afc6a0df7c40b8eb7b", - "aarch64-apple-darwin-freethreaded": "72475196f0092d29bcd2fca298fe198cad135762118e8470083789a3e86cc30f", - "aarch64-unknown-linux-gnu-freethreaded": "c9f4550cdfe4d72c526a3aead8ff1f63a6f0e46cde3d64093177fa1b1944b662", - "ppc64le-unknown-linux-gnu-freethreaded": "2b39b7074a26d44f98275bfa6ea4128e691cc02409edc830dc1b8c19da38ec0f", - "riscv64-unknown-linux-gnu-freethreaded": "16a91fcf2b434c0ba48580aeccf61dfe682efbae5c05b21d0a7780b2cf20cd01", - "s390x-unknown-linux-gnu-freethreaded": "2e42043598543ccf92a5e58f55083ed12156f71cbfe4b2698d4f66dbe3864530", - "x86_64-apple-darwin-freethreaded": "7afbad6cc08072268ad9286dc16be5a04add68af2e3fbef69a429f0a223c275d", - "x86_64-pc-windows-msvc-freethreaded": "7ce62b9445d6d8a8518963e43eb655f5b9f7d08d084d7efc7164b1212fe13d16", - "aarch64-pc-windows-msvc-freethreaded": "87a9c334d1b591ad8561e74d70208eee4b86e23215af031ad7b445a694a45326", - "x86_64-unknown-linux-gnu-freethreaded": "254b71ac6c8557165d88fca355ca8861e303c726bd4ce100eead45d7fb59fb8a", + "aarch64-apple-darwin": "e97387b1fe9048b949cb6616af2984581e84cb068198f73e96d646e3efa33a53", + "aarch64-unknown-linux-gnu": "7464a4086f40ca91f015f147156698734fc8d9185b3d7a5c7d83877a7c182cf1", + "ppc64le-unknown-linux-gnu": "2383189e07675207733b4b155dcc89892d33b92d36e52cce9426d827c05e9b05", + "riscv64-unknown-linux-gnu": "842e1bf032f467cfba267c7db2e0c9344ccf9a58612d81437f8da70538b0d3aa", + "s390x-unknown-linux-gnu": "7b93d636e05aa3c42b971706280d3c2540eed3b18b10968809b1731d0969d6ed", + "x86_64-apple-darwin": "3f8ee5c2087e355188cb443250d4a8c1bffb3b5e751b61b423ac195bfd8dd87f", + "x86_64-pc-windows-msvc": "3d8b2cc6f554f3ad7e54837be51b9dbb1f228900e9acc098a713f6191d33e649", + "aarch64-pc-windows-msvc": "d63fa3939ffa91b73712ba276ca0235183f35fa896e689bae6e37f5d53a702d3", + "x86_64-unknown-linux-gnu": "cf872c369be9ed424092be50d4228ffc8b46469f18307e869023770dd3011aa3", + "x86_64-unknown-linux-musl": "a8b923c7a77e6b9e14018d2774663d538fe87c9da7ba0e081061d651f9802c19", + "aarch64-apple-darwin-freethreaded": "0f0c33fd55475e7192d08c6a37575412c4de886be438e4f359004994c83fc907", + "aarch64-unknown-linux-gnu-freethreaded": "531635fd1c064d03cd14f24b82c531a664de1b8998677eae4c9401b3c9330e26", + "ppc64le-unknown-linux-gnu-freethreaded": "9ea379e46fe0e9befccc24b9b244be9aef7c0589c857654cdb586e3d3ef62464", + "riscv64-unknown-linux-gnu-freethreaded": "63d6ba87237bc6804cb4ca5d87bb1c4267759b5211e434f0187babc5d1a6b12f", + "s390x-unknown-linux-gnu-freethreaded": "2e877903f2f266ec60b661e3b0b074908c8e1699449f907d403eb551e289fa48", + "x86_64-apple-darwin-freethreaded": "3ca690d329ec7d7b950a3430c859969dec6af9290c36d5107990bcbc04635d7c", + "x86_64-pc-windows-msvc-freethreaded": "60240e379461ac802cb6fc3e06a700557a22c616337458dedb94099eee8d2353", + "aarch64-pc-windows-msvc-freethreaded": "4d388a28cd002a3d5d4929b2a40d6341351fbdbbb5811f9bf07c8b3cc5812101", + "x86_64-unknown-linux-gnu-freethreaded": "ddab9e3f5da84f7a330df813d5736802beb0e7179aeaa20e7c2bc4623ec0db5c", }, "strip_prefix": { "aarch64-apple-darwin": "python", @@ -860,11 +967,11 @@ TOOL_VERSIONS = { # buildifier: disable=unsorted-dict-items MINOR_MAPPING = { "3.8": "3.8.20", - "3.9": "3.9.23", - "3.10": "3.10.18", - "3.11": "3.11.13", - "3.12": "3.12.11", - "3.13": "3.13.6", + "3.9": "3.9.24", + "3.10": "3.10.19", + "3.11": "3.11.14", + "3.12": "3.12.12", + "3.13": "3.13.8", "3.14": "3.14.0", } diff --git a/tests/python/python_tests.bzl b/tests/python/python_tests.bzl index da48981e18..f2e87274f8 100644 --- a/tests/python/python_tests.bzl +++ b/tests/python/python_tests.bzl @@ -298,11 +298,11 @@ def _test_toolchain_ordering(env): toolchain = [ _toolchain("3.10"), _toolchain("3.10.15"), - _toolchain("3.10.18"), + _toolchain(MINOR_MAPPING["3.10"]), _toolchain("3.10.13"), _toolchain("3.11.1"), _toolchain("3.11.10"), - _toolchain("3.11.13", is_default = True), + _toolchain(MINOR_MAPPING["3.11"], is_default = True), ], is_root = True, ), @@ -315,16 +315,8 @@ def _test_toolchain_ordering(env): for t in py.toolchains ] - env.expect.that_str(py.default_python_version).equals("3.11.13") - env.expect.that_dict(py.config.minor_mapping).contains_exactly({ - "3.10": "3.10.18", - "3.11": "3.11.13", - "3.12": "3.12.11", - "3.13": "3.13.6", - "3.14": "3.14.0", - "3.8": "3.8.20", - "3.9": "3.9.23", - }) + env.expect.that_str(py.default_python_version).equals(MINOR_MAPPING["3.11"]) + env.expect.that_dict(py.config.minor_mapping).contains_exactly(MINOR_MAPPING) env.expect.that_collection(got_versions).contains_exactly([ # First the full-version toolchains that are in minor_mapping # so that they get matched first if only the `python_version` is in MINOR_MAPPING @@ -332,9 +324,9 @@ def _test_toolchain_ordering(env): # The default version is always set in the `python_version` flag, so know, that # the default match will be somewhere in the first bunch. "3.10", - "3.10.18", + MINOR_MAPPING["3.10"], "3.11", - "3.11.13", + MINOR_MAPPING["3.11"], # Next, the rest, where we will match things based on the `python_version` being # the same "3.10.15", From a2f14eb4f7349bd18530d8057e96f093570fd103 Mon Sep 17 00:00:00 2001 From: Jan Winkler <45763961+janwinkler1@users.noreply.github.com> Date: Sat, 11 Oct 2025 06:59:01 +0200 Subject: [PATCH 483/922] fix(rules): make `py_console_script_binary` compatible with symbolic macros (#3195) Fixes target naming to follow symbolic macro conventions introduced in Bazel 8.0. --------- Co-authored-by: Ignas Anikevicius <240938+aignas@users.noreply.github.com> --- CHANGELOG.md | 2 ++ python/private/py_console_script_binary.bzl | 7 +++++-- 2 files changed, 7 insertions(+), 2 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index a86dd01a9f..04e01a9c03 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -108,6 +108,8 @@ END_UNRELEASED_TEMPLATE variants. Setting {obj}`--py_linux_libc=musl` and `--py_freethreaded=yes` now activate them, respectively. ([#3262](https://github.com/bazel-contrib/rules_python/issues/3262)). +* (rules) {obj}`py_console_script_binary` is now compatible with symbolic macros + ([#3195](https://github.com/bazel-contrib/rules_python/pull/3195)). {#v0-0-0-added} ### Added diff --git a/python/private/py_console_script_binary.bzl b/python/private/py_console_script_binary.bzl index d98457dbe1..0dda44e7b9 100644 --- a/python/private/py_console_script_binary.bzl +++ b/python/private/py_console_script_binary.bzl @@ -53,6 +53,7 @@ def py_console_script_binary( script = None, binary_rule = py_binary, shebang = "", + main = None, **kwargs): """Generate a py_binary for a console_script entry_point. @@ -66,6 +67,8 @@ def py_console_script_binary( package as the `pkg` Label. script: {type}`str`, The console script name that the py_binary is going to be generated for. Defaults to the normalized name attribute. + main: {type}`str`, the python file to be generated, defaults to `_entry_point.py` to + be compatible with symbolic macros. binary_rule: {type}`callable`, The rule/macro to use to instantiate the target. It's expected to behave like {obj}`py_binary`. Defaults to {obj}`py_binary`. @@ -73,13 +76,13 @@ def py_console_script_binary( Defaults to empty string. **kwargs: Extra parameters forwarded to `binary_rule`. """ - main = "rules_python_entry_point_{}.py".format(name) + main = main or name + "_entry_point.py" if kwargs.pop("srcs", None): fail("passing 'srcs' attribute to py_console_script_binary is unsupported") py_console_script_gen( - name = "_{}_gen".format(name), + name = name + "_gen__", entry_points_txt = entry_points_txt or _dist_info(pkg), out = main, console_script = script, From f84640bec471629c25d76052ed858eb3c0dd3a14 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Sat, 11 Oct 2025 02:35:48 -0700 Subject: [PATCH 484/922] fix(venv): symlink shared libraries directly (#3331) It seems `$ORIGIN` resolves prior to symlink resolution. This makes it resolve differently depending on if the directory or file itself is symlinked. To fix, special case shared libraries and have them symlinked directly. Since an explicit file is the target, `VenvSymlinkEntry.link_to_file` is added to hold the File object that will be linked to. An unfortunate side-effect of this logic is any package with `lib*.so` files will be more expensive to build (depset flattened at analysis time, more files symlinked), but it beats not working at all. Optimizing that can be done in another change. Tests added to generate libraries that look like what something from PyPI does. Manually verified a case using jax and jax plugins. Fixes https://github.com/bazel-contrib/rules_python/issues/3228 --- CHANGELOG.md | 3 + docs/pyproject.toml | 4 +- docs/requirements.txt | 100 +++++++++------ python/private/py_info.bzl | 20 ++- python/private/venv_runfiles.bzl | 44 ++++++- tests/support/copy_file.bzl | 33 +++++ tests/venv_site_packages_libs/BUILD.bazel | 20 ++- .../app_files_building_tests.bzl | 77 +++++++++++- .../ext_with_libs/BUILD.bazel | 94 ++++++++++++++ .../ext_with_libs/adder.c | 15 +++ .../ext_with_libs/increment.c | 3 + .../ext_with_libs/increment.h | 6 + .../site-packages/ext_with_libs/__init__.py | 2 + .../shared_lib_loading_test.py | 117 ++++++++++++++++++ 14 files changed, 489 insertions(+), 49 deletions(-) create mode 100644 tests/support/copy_file.bzl create mode 100644 tests/venv_site_packages_libs/ext_with_libs/BUILD.bazel create mode 100644 tests/venv_site_packages_libs/ext_with_libs/adder.c create mode 100644 tests/venv_site_packages_libs/ext_with_libs/increment.c create mode 100644 tests/venv_site_packages_libs/ext_with_libs/increment.h create mode 100644 tests/venv_site_packages_libs/ext_with_libs/site-packages/ext_with_libs/__init__.py create mode 100644 tests/venv_site_packages_libs/shared_lib_loading_test.py diff --git a/CHANGELOG.md b/CHANGELOG.md index 04e01a9c03..7782454fbd 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -102,6 +102,9 @@ END_UNRELEASED_TEMPLATE * (venvs) {obj}`--venvs_site_packages=yes` no longer errors when packages with overlapping files or directories are used together. ([#3204](https://github.com/bazel-contrib/rules_python/issues/3204)). +* (venvs) {obj}`--venvs_site_packages=yes` works for packages that dynamically + link to shared libraries + ([#3228](https://github.com/bazel-contrib/rules_python/issues/3228)). * (uv) {obj}`//python/uv:lock.bzl%lock` now works with a local platform runtime. * (toolchains) WORKSPACE builds now correctly register musl and freethreaded diff --git a/docs/pyproject.toml b/docs/pyproject.toml index 9a089df59c..f4bbbaf35a 100644 --- a/docs/pyproject.toml +++ b/docs/pyproject.toml @@ -13,5 +13,7 @@ dependencies = [ "absl-py", "typing-extensions", "sphinx-reredirects", - "pefile" + "pefile", + "pyelftools", + "macholib", ] diff --git a/docs/requirements.txt b/docs/requirements.txt index 290113c1b9..c5a5feaae0 100644 --- a/docs/requirements.txt +++ b/docs/requirements.txt @@ -14,6 +14,10 @@ alabaster==1.0.0 ; python_full_version >= '3.10' \ --hash=sha256:c00dca57bca26fa62a6d7d0a9fcce65f3e026e9bfe33e9c538fd3fbb2144fd9e \ --hash=sha256:fc6786402dc3fcb2de3cabd5fe455a2db534b371124f1f21de8731783dec828b # via sphinx +altgraph==0.17.4 \ + --hash=sha256:1b5afbb98f6c4dcadb2e2ae6ab9fa994bbb8c1d75f4fa96d340f9437ae454406 \ + --hash=sha256:642743b4750de17e655e6711601b077bc6598dbfa3ba5fa2b2a35ce12b508dff + # via macholib astroid==3.3.11 \ --hash=sha256:1e5a5011af2920c7c67a53f65d536d65bfa7116feeaf2354d8b94f29573bb0ce \ --hash=sha256:54c760ae8322ece1abd213057c4b5bba7c49818853fc901ef09719a60dbf9dec @@ -111,9 +115,9 @@ colorama==0.4.6 ; sys_platform == 'win32' \ --hash=sha256:08695f5cb7ed6e0531a20572697297273c47b8cae5a63ffc6d6ed5c201be6e44 \ --hash=sha256:4f1d9991f5acc0ca119f9d443620b77f9d6b33703e51011c16baf57afb285fc6 # via sphinx -docutils==0.22.2 \ - --hash=sha256:9fdb771707c8784c8f2728b67cb2c691305933d68137ef95a75db5f4dfbc213d \ - --hash=sha256:b0e98d679283fc3bb0ead8a5da7f501baa632654e7056e9c5846842213d674d8 +docutils==0.21.2 \ + --hash=sha256:3a6b18732edf182daa3cd12775bbb338cf5691468f91eeeb109deff6ebfa986f \ + --hash=sha256:dafca5b9e384f0e419294eb4d2ff9fa826435bf15f15b7bd45723e8ad76811b2 # via # myst-parser # sphinx @@ -137,9 +141,13 @@ jinja2==3.1.6 \ # myst-parser # readthedocs-sphinx-ext # sphinx -markdown-it-py==4.0.0 \ - --hash=sha256:87327c59b172c5011896038353a81343b6754500a08cd7a4973bb48c6d578147 \ - --hash=sha256:cb0a2b4aa34f932c007117b194e945bd74e0ec24133ceb5bac59009cda1cb9f3 +macholib==1.16.3 \ + --hash=sha256:07ae9e15e8e4cd9a788013d81f5908b3609aa76f9b1421bae9c4d7606ec86a30 \ + --hash=sha256:0e315d7583d38b8c77e815b1ecbdbf504a8258d8b3e17b61165c6feb60d18f2c + # via rules-python-docs (docs/pyproject.toml) +markdown-it-py==3.0.0 \ + --hash=sha256:355216845c60bd96232cd8d8c40e8f9765cc86f46880e43a8fd22dc1a1a8cab1 \ + --hash=sha256:e3f60a94fa066dc52ec76661e37c851cb232d92f9886b15cb560aaada2df8feb # via # mdit-py-plugins # myst-parser @@ -264,6 +272,10 @@ pefile==2024.8.26 \ --hash=sha256:3ff6c5d8b43e8c37bb6e6dd5085658d658a7a0bdcd20b6a07b1fcfc1c4e9d632 \ --hash=sha256:76f8b485dcd3b1bb8166f1128d395fa3d87af26360c2358fb75b80019b957c6f # via rules-python-docs (docs/pyproject.toml) +pyelftools==0.32 \ + --hash=sha256:013df952a006db5e138b1edf6d8a68ecc50630adbd0d83a2d41e7f846163d738 \ + --hash=sha256:6de90ee7b8263e740c8715a925382d4099b354f29ac48ea40d840cf7aa14ace5 + # via rules-python-docs (docs/pyproject.toml) pygments==2.19.2 \ --hash=sha256:636cb2477cec7f8952536970bc533bc43743542f70392ae026374600add5b887 \ --hash=sha256:86540386c03d588bb81d44bc3928634ff26449851e99741617ecb9037ee5ec0b @@ -432,39 +444,49 @@ sphinxcontrib-serializinghtml==2.0.0 \ --hash=sha256:6e2cb0eef194e10c27ec0023bfeb25badbbb5868244cf5bc5bdc04e4464bf331 \ --hash=sha256:e9d912827f872c029017a53f0ef2180b327c3f7fd23c87229f7a8e8b70031d4d # via sphinx -tomli==2.2.1 ; python_full_version < '3.11' \ - --hash=sha256:023aa114dd824ade0100497eb2318602af309e5a55595f76b626d6d9f3b7b0a6 \ - --hash=sha256:02abe224de6ae62c19f090f68da4e27b10af2b93213d36cf44e6e1c5abd19fdd \ - --hash=sha256:286f0ca2ffeeb5b9bd4fcc8d6c330534323ec51b2f52da063b11c502da16f30c \ - --hash=sha256:2d0f2fdd22b02c6d81637a3c95f8cd77f995846af7414c5c4b8d0545afa1bc4b \ - --hash=sha256:33580bccab0338d00994d7f16f4c4ec25b776af3ffaac1ed74e0b3fc95e885a8 \ - --hash=sha256:400e720fe168c0f8521520190686ef8ef033fb19fc493da09779e592861b78c6 \ - --hash=sha256:40741994320b232529c802f8bc86da4e1aa9f413db394617b9a256ae0f9a7f77 \ - --hash=sha256:465af0e0875402f1d226519c9904f37254b3045fc5084697cefb9bdde1ff99ff \ - --hash=sha256:4a8f6e44de52d5e6c657c9fe83b562f5f4256d8ebbfe4ff922c495620a7f6cea \ - --hash=sha256:4e340144ad7ae1533cb897d406382b4b6fede8890a03738ff1683af800d54192 \ - --hash=sha256:678e4fa69e4575eb77d103de3df8a895e1591b48e740211bd1067378c69e8249 \ - --hash=sha256:6972ca9c9cc9f0acaa56a8ca1ff51e7af152a9f87fb64623e31d5c83700080ee \ - --hash=sha256:7fc04e92e1d624a4a63c76474610238576942d6b8950a2d7f908a340494e67e4 \ - --hash=sha256:889f80ef92701b9dbb224e49ec87c645ce5df3fa2cc548664eb8a25e03127a98 \ - --hash=sha256:8d57ca8095a641b8237d5b079147646153d22552f1c637fd3ba7f4b0b29167a8 \ - --hash=sha256:8dd28b3e155b80f4d54beb40a441d366adcfe740969820caf156c019fb5c7ec4 \ - --hash=sha256:9316dc65bed1684c9a98ee68759ceaed29d229e985297003e494aa825ebb0281 \ - --hash=sha256:a198f10c4d1b1375d7687bc25294306e551bf1abfa4eace6650070a5c1ae2744 \ - --hash=sha256:a38aa0308e754b0e3c67e344754dff64999ff9b513e691d0e786265c93583c69 \ - --hash=sha256:a92ef1a44547e894e2a17d24e7557a5e85a9e1d0048b0b5e7541f76c5032cb13 \ - --hash=sha256:ac065718db92ca818f8d6141b5f66369833d4a80a9d74435a268c52bdfa73140 \ - --hash=sha256:b82ebccc8c8a36f2094e969560a1b836758481f3dc360ce9a3277c65f374285e \ - --hash=sha256:c954d2250168d28797dd4e3ac5cf812a406cd5a92674ee4c8f123c889786aa8e \ - --hash=sha256:cb55c73c5f4408779d0cf3eef9f762b9c9f147a77de7b258bef0a5628adc85cc \ - --hash=sha256:cd45e1dc79c835ce60f7404ec8119f2eb06d38b1deba146f07ced3bbc44505ff \ - --hash=sha256:d3f5614314d758649ab2ab3a62d4f2004c825922f9e370b29416484086b264ec \ - --hash=sha256:d920f33822747519673ee656a4b6ac33e382eca9d331c87770faa3eef562aeb2 \ - --hash=sha256:db2b95f9de79181805df90bedc5a5ab4c165e6ec3fe99f970d0e302f384ad222 \ - --hash=sha256:e59e304978767a54663af13c07b3d1af22ddee3bb2fb0618ca1593e4f593a106 \ - --hash=sha256:e85e99945e688e32d5a35c1ff38ed0b3f41f43fad8df0bdf79f72b2ba7bc5272 \ - --hash=sha256:ece47d672db52ac607a3d9599a9d48dcb2f2f735c6c2d1f34130085bb12b112a \ - --hash=sha256:f4039b9cbc3048b2416cc57ab3bda989a6fcf9b36cf8937f01a6e731b64f80d7 +tomli==2.3.0 ; python_full_version < '3.11' \ + --hash=sha256:00b5f5d95bbfc7d12f91ad8c593a1659b6387b43f054104cda404be6bda62456 \ + --hash=sha256:0a154a9ae14bfcf5d8917a59b51ffd5a3ac1fd149b71b47a3a104ca4edcfa845 \ + --hash=sha256:0c95ca56fbe89e065c6ead5b593ee64b84a26fca063b5d71a1122bf26e533999 \ + --hash=sha256:0eea8cc5c5e9f89c9b90c4896a8deefc74f518db5927d0e0e8d4a80953d774d0 \ + --hash=sha256:1cb4ed918939151a03f33d4242ccd0aa5f11b3547d0cf30f7c74a408a5b99878 \ + --hash=sha256:4021923f97266babc6ccab9f5068642a0095faa0a51a246a6a02fccbb3514eaf \ + --hash=sha256:4c2ef0244c75aba9355561272009d934953817c49f47d768070c3c94355c2aa3 \ + --hash=sha256:4dc4ce8483a5d429ab602f111a93a6ab1ed425eae3122032db7e9acf449451be \ + --hash=sha256:4f195fe57ecceac95a66a75ac24d9d5fbc98ef0962e09b2eddec5d39375aae52 \ + --hash=sha256:5192f562738228945d7b13d4930baffda67b69425a7f0da96d360b0a3888136b \ + --hash=sha256:5e01decd096b1530d97d5d85cb4dff4af2d8347bd35686654a004f8dea20fc67 \ + --hash=sha256:64be704a875d2a59753d80ee8a533c3fe183e3f06807ff7dc2232938ccb01549 \ + --hash=sha256:70a251f8d4ba2d9ac2542eecf008b3c8a9fc5c3f9f02c56a9d7952612be2fdba \ + --hash=sha256:73ee0b47d4dad1c5e996e3cd33b8a76a50167ae5f96a2607cbe8cc773506ab22 \ + --hash=sha256:74bf8464ff93e413514fefd2be591c3b0b23231a77f901db1eb30d6f712fc42c \ + --hash=sha256:792262b94d5d0a466afb5bc63c7daa9d75520110971ee269152083270998316f \ + --hash=sha256:7b0882799624980785240ab732537fcfc372601015c00f7fc367c55308c186f6 \ + --hash=sha256:883b1c0d6398a6a9d29b508c331fa56adbcdff647f6ace4dfca0f50e90dfd0ba \ + --hash=sha256:88bd15eb972f3664f5ed4b57c1634a97153b4bac4479dcb6a495f41921eb7f45 \ + --hash=sha256:8a35dd0e643bb2610f156cca8db95d213a90015c11fee76c946aa62b7ae7e02f \ + --hash=sha256:940d56ee0410fa17ee1f12b817b37a4d4e4dc4d27340863cc67236c74f582e77 \ + --hash=sha256:97d5eec30149fd3294270e889b4234023f2c69747e555a27bd708828353ab606 \ + --hash=sha256:a0e285d2649b78c0d9027570d4da3425bdb49830a6156121360b3f8511ea3441 \ + --hash=sha256:a1f7f282fe248311650081faafa5f4732bdbfef5d45fe3f2e702fbc6f2d496e0 \ + --hash=sha256:a4ea38c40145a357d513bffad0ed869f13c1773716cf71ccaa83b0fa0cc4e42f \ + --hash=sha256:a56212bdcce682e56b0aaf79e869ba5d15a6163f88d5451cbde388d48b13f530 \ + --hash=sha256:ad805ea85eda330dbad64c7ea7a4556259665bdf9d2672f5dccc740eb9d3ca05 \ + --hash=sha256:b273fcbd7fc64dc3600c098e39136522650c49bca95df2d11cf3b626422392c8 \ + --hash=sha256:b5870b50c9db823c595983571d1296a6ff3e1b88f734a4c8f6fc6188397de005 \ + --hash=sha256:b74a0e59ec5d15127acdabd75ea17726ac4c5178ae51b85bfe39c4f8a278e879 \ + --hash=sha256:be71c93a63d738597996be9528f4abe628d1adf5e6eb11607bc8fe1a510b5dae \ + --hash=sha256:c22a8bf253bacc0cf11f35ad9808b6cb75ada2631c2d97c971122583b129afbc \ + --hash=sha256:c4665508bcbac83a31ff8ab08f424b665200c0e1e645d2bd9ab3d3e557b6185b \ + --hash=sha256:c5f3ffd1e098dfc032d4d3af5c0ac64f6d286d98bc148698356847b80fa4de1b \ + --hash=sha256:cebc6fe843e0733ee827a282aca4999b596241195f43b4cc371d64fc6639da9e \ + --hash=sha256:d1381caf13ab9f300e30dd8feadb3de072aeb86f1d34a8569453ff32a7dea4bf \ + --hash=sha256:d7d86942e56ded512a594786a5ba0a5e521d02529b3826e7761a05138341a2ac \ + --hash=sha256:e31d432427dcbf4d86958c184b9bfd1e96b5b71f8eb17e6d02531f434fd335b8 \ + --hash=sha256:e95b1af3c5b07d9e643909b5abbec77cd9f1217e6d0bca72b0234736b9fb1f1b \ + --hash=sha256:f85209946d1fe94416debbb88d00eb92ce9cd5266775424ff81bc959e001acaf \ + --hash=sha256:feb0dacc61170ed7ab602d3d972a58f14ee3ee60494292d384649a3dc38ef463 \ + --hash=sha256:ff72b71b5d10d22ecb084d345fc26f42b5143c5533db5e2eaba7d2d335358876 # via # sphinx # sphinx-autodoc2 diff --git a/python/private/py_info.bzl b/python/private/py_info.bzl index 9318347819..95b739dff2 100644 --- a/python/private/py_info.bzl +++ b/python/private/py_info.bzl @@ -47,12 +47,17 @@ VenvSymlinkKind = struct( INCLUDE = "INCLUDE", ) +def _VenvSymlinkEntry_init(**kwargs): + kwargs.setdefault("link_to_file", None) + return kwargs + # A provider is used for memory efficiency. # buildifier: disable=name-conventions -VenvSymlinkEntry = provider( +VenvSymlinkEntry, _ = provider( doc = """ An entry in `PyInfo.venv_symlinks` """, + init = _VenvSymlinkEntry_init, fields = { "files": """ :type: depset[File] @@ -67,12 +72,21 @@ if one adds files to `venv_path=a/` and another adds files to `venv_path=a/b/`. One of the {obj}`VenvSymlinkKind` values. It represents which directory within the venv to create the path under. +""", + "link_to_file": """ +:type: File | None + +A file that `venv_path` should point to. The file to link to should also be in +`files`. + +:::{versionadded} VERSION_NEXT_FEATURE +::: """, "link_to_path": """ :type: str | None -A runfiles-root relative path that `venv_path` will symlink to. If `None`, -it means to not create a symlink. +A runfiles-root relative path that `venv_path` will symlink to (if +`link_to_file` is `None`). If `None`, it means to not create it in the venv. """, "package": """ :type: str | None diff --git a/python/private/venv_runfiles.bzl b/python/private/venv_runfiles.bzl index 9fbe97a52e..9bdacf833e 100644 --- a/python/private/venv_runfiles.bzl +++ b/python/private/venv_runfiles.bzl @@ -81,7 +81,7 @@ def build_link_map(ctx, entries): Returns: {type}`dict[str, dict[str, str|File]]` Mappings of venv paths to their backing files. The first key is a `VenvSymlinkKind` value. - The inner dict keys are venv paths relative to the kind's diretory. The + The inner dict keys are venv paths relative to the kind's directory. The inner dict values are strings or Files to link to. """ @@ -116,7 +116,10 @@ def build_link_map(ctx, entries): # If there's just one group, we can symlink to the directory if len(group) == 1: entry = group[0] - keep_kind_link_map[entry.venv_path] = entry.link_to_path + if entry.link_to_file: + keep_kind_link_map[entry.venv_path] = entry.link_to_file + else: + keep_kind_link_map[entry.venv_path] = entry.link_to_path else: # Merge a group of overlapping prefixes _merge_venv_path_group(ctx, group, keep_kind_link_map) @@ -172,7 +175,9 @@ def _merge_venv_path_group(ctx, group, keep_map): # TODO: Compute the minimum number of entries to create. This can't avoid # flattening the files depset, but can lower the number of materialized # files significantly. Usually overlaps are limited to a small number - # of directories. + # of directories. Note that, when doing so, shared libraries need to + # be symlinked directly, not the directory containing them, due to + # dynamic linker symlink resolution semantics on Linux. for entry in group: prefix = entry.venv_path for file in entry.files.to_list(): @@ -249,13 +254,26 @@ def get_venv_symlinks(ctx, files, package, version_str, site_packages_root): continue path = path.removeprefix(site_packages_root) dir_name, _, filename = path.rpartition("/") + runfiles_dir_name, _, _ = runfiles_root_path(ctx, src.short_path).partition("/") + + if _is_linker_loaded_library(filename): + entry = VenvSymlinkEntry( + kind = VenvSymlinkKind.LIB, + link_to_path = paths.join(runfiles_dir_name, site_packages_root, filename), + link_to_file = src, + package = package, + version = version_str, + venv_path = path, + files = depset([src]), + ) + venv_symlinks.append(entry) + continue if dir_name in dir_symlinks: # we already have this dir, this allows us to short-circuit since most of the # ctx.files.data might share the same directories as ctx.files.srcs continue - runfiles_dir_name, _, _ = runfiles_root_path(ctx, src.short_path).partition("/") if dir_name: # This can be either: # * a directory with libs (e.g. numpy.libs, created by auditwheel) @@ -312,6 +330,24 @@ def get_venv_symlinks(ctx, files, package, version_str, site_packages_root): return venv_symlinks +def _is_linker_loaded_library(filename): + """Tells if a filename is one that `dlopen()` or the runtime linker handles. + + This should return true for regular C libraries, but false for Python + C extension modules. + + Python extensions: .so (linux, mac), .pyd (windows) + + C libraries: lib*.so (linux), lib*.so.* (linux), lib*.dylib (mac), .dll (windows) + """ + if filename.endswith(".dll"): + return True + if filename.startswith("lib") and ( + filename.endswith((".so", ".dylib")) or ".so." in filename + ): + return True + return False + def _repo_relative_short_path(short_path): # Convert `../+pypi+foo/some/file.py` to `some/file.py` if short_path.startswith("../"): diff --git a/tests/support/copy_file.bzl b/tests/support/copy_file.bzl new file mode 100644 index 0000000000..bd9bb218f3 --- /dev/null +++ b/tests/support/copy_file.bzl @@ -0,0 +1,33 @@ +"""Copies a file to a directory.""" + +def _copy_file_to_dir_impl(ctx): + out_file = ctx.actions.declare_file( + "{}/{}".format(ctx.attr.out_dir, ctx.file.src.basename), + ) + ctx.actions.run_shell( + inputs = [ctx.file.src], + outputs = [out_file], + arguments = [ctx.file.src.path, out_file.path], + # Perform a copy to better match how a file install from + # a repo-phase (e.g. whl extraction) looks. + command = 'cp -f "$1" "$2"', + progress_message = "Copying %{input} to %{output}", + ) + return [DefaultInfo(files = depset([out_file]))] + +copy_file_to_dir = rule( + implementation = _copy_file_to_dir_impl, + doc = """ +This allows copying a file whose name is platform-dependent to a directory. + +While bazel_skylib has a copy_file rule, you must statically specify the +output file name. +""", + attrs = { + "out_dir": attr.string(mandatory = True), + "src": attr.label( + allow_single_file = True, + mandatory = True, + ), + }, +) diff --git a/tests/venv_site_packages_libs/BUILD.bazel b/tests/venv_site_packages_libs/BUILD.bazel index 92d5dec6d3..2ce4ad9d22 100644 --- a/tests/venv_site_packages_libs/BUILD.bazel +++ b/tests/venv_site_packages_libs/BUILD.bazel @@ -1,6 +1,10 @@ load("//python:py_library.bzl", "py_library") load("//tests/support:py_reconfig.bzl", "py_reconfig_test") -load("//tests/support:support.bzl", "SUPPORTS_BOOTSTRAP_SCRIPT") +load( + "//tests/support:support.bzl", + "NOT_WINDOWS", + "SUPPORTS_BOOTSTRAP_SCRIPT", +) py_library( name = "user_lib", @@ -34,3 +38,17 @@ py_reconfig_test( "@other//with_external_data", ], ) + +py_reconfig_test( + name = "shared_lib_loading_test", + srcs = ["shared_lib_loading_test.py"], + bootstrap_impl = "script", + main = "shared_lib_loading_test.py", + target_compatible_with = NOT_WINDOWS, + venvs_site_packages = "yes", + deps = [ + "//tests/venv_site_packages_libs/ext_with_libs", + "@dev_pip//macholib", + "@dev_pip//pyelftools", + ], +) diff --git a/tests/venv_site_packages_libs/app_files_building/app_files_building_tests.bzl b/tests/venv_site_packages_libs/app_files_building/app_files_building_tests.bzl index 68e17160e7..31c720a986 100644 --- a/tests/venv_site_packages_libs/app_files_building/app_files_building_tests.bzl +++ b/tests/venv_site_packages_libs/app_files_building/app_files_building_tests.bzl @@ -4,7 +4,25 @@ load("@bazel_skylib//lib:paths.bzl", "paths") load("@rules_testing//lib:analysis_test.bzl", "analysis_test") load("@rules_testing//lib:test_suite.bzl", "test_suite") load("//python/private:py_info.bzl", "VenvSymlinkEntry", "VenvSymlinkKind") # buildifier: disable=bzl-visibility -load("//python/private:venv_runfiles.bzl", "build_link_map") # buildifier: disable=bzl-visibility +load("//python/private:venv_runfiles.bzl", "build_link_map", "get_venv_symlinks") # buildifier: disable=bzl-visibility + +def _empty_files_impl(ctx): + files = [] + for p in ctx.attr.paths: + f = ctx.actions.declare_file(p) + ctx.actions.write(output = f, content = "") + files.append(f) + return [DefaultInfo(files = depset(files))] + +empty_files = rule( + implementation = _empty_files_impl, + attrs = { + "paths": attr.string_list( + doc = "A list of paths to create as files.", + mandatory = True, + ), + }, +) _tests = [] @@ -242,6 +260,63 @@ def _test_multiple_venv_symlink_kinds_impl(env, _): VenvSymlinkKind.INCLUDE, ]) +def _test_shared_library_symlinking(name): + empty_files( + name = name + "_files", + # NOTE: Test relies upon order + paths = [ + "site-packages/bar/libs/liby.so", + "site-packages/bar/x.py", + "site-packages/bar/y.so", + "site-packages/foo.libs/libx.so", + "site-packages/foo/a.py", + "site-packages/foo/b.so", + ], + ) + analysis_test( + name = name, + impl = _test_shared_library_symlinking_impl, + target = name + "_files", + ) + +_tests.append(_test_shared_library_symlinking) + +def _test_shared_library_symlinking_impl(env, target): + srcs = target.files.to_list() + actual_entries = get_venv_symlinks( + _ctx(), + srcs, + package = "foo", + version_str = "1.0", + site_packages_root = env.ctx.label.package + "/site-packages", + ) + + actual = [e for e in actual_entries if e.venv_path == "foo.libs/libx.so"] + if not actual: + fail("Did not find VenvSymlinkEntry with venv_path equal to foo.libs/libx.so. " + + "Found: {}".format(actual_entries)) + elif len(actual) > 1: + fail("Found multiple entries with venv_path=foo.libs/libx.so. " + + "Found: {}".format(actual_entries)) + actual = actual[0] + + actual_files = actual.files.to_list() + expected_lib_dso = [f for f in srcs if f.basename == "libx.so"] + env.expect.that_collection(actual_files).contains_exactly(expected_lib_dso) + + entries = actual_entries + actual = build_link_map(_ctx(), entries) + + # The important condition is that each lib*.so file is linked directly. + expected_libs = { + "bar/libs/liby.so": srcs[0], + "bar/x.py": srcs[1], + "bar/y.so": srcs[2], + "foo": "_main/tests/venv_site_packages_libs/app_files_building/site-packages/foo", + "foo.libs/libx.so": srcs[3], + } + env.expect.that_dict(actual[VenvSymlinkKind.LIB]).contains_exactly(expected_libs) + def app_files_building_test_suite(name): test_suite( name = name, diff --git a/tests/venv_site_packages_libs/ext_with_libs/BUILD.bazel b/tests/venv_site_packages_libs/ext_with_libs/BUILD.bazel new file mode 100644 index 0000000000..8f161ee17c --- /dev/null +++ b/tests/venv_site_packages_libs/ext_with_libs/BUILD.bazel @@ -0,0 +1,94 @@ +load("@rules_cc//cc:cc_library.bzl", "cc_library") +load("@rules_cc//cc:cc_shared_library.bzl", "cc_shared_library") +load("//python:py_library.bzl", "py_library") +load("//tests/support:copy_file.bzl", "copy_file_to_dir") + +package( + default_visibility = ["//visibility:public"], +) + +cc_library( + name = "increment_impl", + srcs = ["increment.c"], + deps = [":increment_headers"], +) + +cc_library( + name = "increment_headers", + hdrs = ["increment.h"], +) + +cc_shared_library( + name = "increment", + user_link_flags = select({ + "@platforms//os:osx": [ + # Needed so that DT_NEEDED=libincrement.dylib can find + # this shared library + "-Wl,-install_name,@rpath/libincrement.dylib", + ], + "//conditions:default": [], + }), + deps = [":increment_impl"], +) + +cc_library( + name = "adder_impl", + srcs = ["adder.c"], + deps = [ + ":increment_headers", + "@rules_python//python/cc:current_py_cc_headers", + ], +) + +cc_shared_library( + name = "adder", + # Necessary for several reasons: + # 1. Ensures the output doesn't include increment itself (avoids ODRs) + # 2. Adds -lincrement (DT_NEEDED for libincrement.so) + # 3. Ensures libincrement.so is available at link time to satisfy (2) + dynamic_deps = [":increment"], + shared_lib_name = "adder.so", + tags = ["manual"], + # NOTE: cc_shared_library adds Bazelized rpath entries, too. + user_link_flags = [ + ] + select({ + "@platforms//os:osx": [ + "-Wl,-rpath,@loader_path/libs", + "-undefined", + "dynamic_lookup", + "-Wl,-exported_symbol", + "-Wl,_PyInit_adder", + ], + # Assume linux default + "//conditions:default": [ + "-Wl,-rpath,$ORIGIN/libs", + ], + }), + deps = [":adder_impl"], +) + +copy_file_to_dir( + name = "relocate_adder", + src = ":adder", + out_dir = "site-packages/ext_with_libs", + tags = ["manual"], +) + +copy_file_to_dir( + name = "relocate_increment", + src = ":increment", + out_dir = "site-packages/ext_with_libs/libs", + tags = ["manual"], +) + +py_library( + name = "ext_with_libs", + srcs = glob(["site-packages/**/*.py"]), + data = [ + ":relocate_adder", + ":relocate_increment", + ], + experimental_venvs_site_packages = "//python/config_settings:venvs_site_packages", + imports = [package_name() + "/site-packages"], + tags = ["manual"], +) diff --git a/tests/venv_site_packages_libs/ext_with_libs/adder.c b/tests/venv_site_packages_libs/ext_with_libs/adder.c new file mode 100644 index 0000000000..8b04b1721f --- /dev/null +++ b/tests/venv_site_packages_libs/ext_with_libs/adder.c @@ -0,0 +1,15 @@ +#include + +#include "increment.h" + +static PyObject *do_add(PyObject *self, PyObject *Py_UNUSED(args)) { + return PyLong_FromLong(increment(1)); +} + +static PyMethodDef AdderMethods[] = { + {"do_add", do_add, METH_NOARGS, "Add one"}, {NULL, NULL, 0, NULL}}; + +static struct PyModuleDef addermodule = {PyModuleDef_HEAD_INIT, "adder", NULL, + -1, AdderMethods}; + +PyMODINIT_FUNC PyInit_adder(void) { return PyModule_Create(&addermodule); } diff --git a/tests/venv_site_packages_libs/ext_with_libs/increment.c b/tests/venv_site_packages_libs/ext_with_libs/increment.c new file mode 100644 index 0000000000..b194325ac7 --- /dev/null +++ b/tests/venv_site_packages_libs/ext_with_libs/increment.c @@ -0,0 +1,3 @@ +#include "increment.h" + +int increment(int val) { return val + 1; } diff --git a/tests/venv_site_packages_libs/ext_with_libs/increment.h b/tests/venv_site_packages_libs/ext_with_libs/increment.h new file mode 100644 index 0000000000..8a13bf5621 --- /dev/null +++ b/tests/venv_site_packages_libs/ext_with_libs/increment.h @@ -0,0 +1,6 @@ +#ifndef TESTS_VENV_SITE_PACKAGES_LIBS_EXT_WITH_LIBS_INCREMENT_H_ +#define TESTS_VENV_SITE_PACKAGES_LIBS_EXT_WITH_LIBS_INCREMENT_H_ + +int increment(int); + +#endif // TESTS_VENV_SITE_PACKAGES_LIBS_EXT_WITH_LIBS_INCREMENT_H_ diff --git a/tests/venv_site_packages_libs/ext_with_libs/site-packages/ext_with_libs/__init__.py b/tests/venv_site_packages_libs/ext_with_libs/site-packages/ext_with_libs/__init__.py new file mode 100644 index 0000000000..ea0485cb1b --- /dev/null +++ b/tests/venv_site_packages_libs/ext_with_libs/site-packages/ext_with_libs/__init__.py @@ -0,0 +1,2 @@ +# This just marks the directory as a Pyton package. Python C extension modules +# and C libraries are populated in this directory at build time. diff --git a/tests/venv_site_packages_libs/shared_lib_loading_test.py b/tests/venv_site_packages_libs/shared_lib_loading_test.py new file mode 100644 index 0000000000..2b58f8571c --- /dev/null +++ b/tests/venv_site_packages_libs/shared_lib_loading_test.py @@ -0,0 +1,117 @@ +import importlib.util +import os +import unittest + +from elftools.elf.elffile import ELFFile +from macholib import mach_o +from macholib.MachO import MachO + +ELF_MAGIC = b"\x7fELF" +MACHO_MAGICS = ( + b"\xce\xfa\xed\xfe", # 32-bit big-endian + b"\xcf\xfa\xed\xfe", # 64-bit big-endian + b"\xfe\xed\xfa\xce", # 32-bit little-endian + b"\xfe\xed\xfa\xcf", # 64-bit little-endian +) + + +class SharedLibLoadingTest(unittest.TestCase): + def test_shared_library_linking(self): + try: + import ext_with_libs.adder + except ImportError as e: + spec = importlib.util.find_spec("ext_with_libs.adder") + if not spec or not spec.origin: + self.fail(f"Import failed and could not find module spec: {e}") + + info = self._get_linking_info(spec.origin) + + # Give a useful error message for debugging. + self.fail( + f"Failed to import adder extension.\n" + f"Original error: {e}\n" + f"Linking info for {spec.origin}:\n" + f" RPATHs: {info.get('rpaths', 'N/A')}\n" + f" Needed libs: {info.get('needed', 'N/A')}" + ) + + # Check that the module was loaded from the venv. + self.assertIn(".venv/", ext_with_libs.adder.__file__) + + adder_path = os.path.realpath(ext_with_libs.adder.__file__) + + with open(adder_path, "rb") as f: + magic_bytes = f.read(4) + + if magic_bytes == ELF_MAGIC: + self._assert_elf_linking(adder_path) + elif magic_bytes in MACHO_MAGICS: + self._assert_macho_linking(adder_path) + else: + self.fail(f"Unsupported file format for adder: magic bytes {magic_bytes!r}") + + # Check the function works regardless of format. + self.assertEqual(ext_with_libs.adder.do_add(), 2) + + def _get_linking_info(self, path): + """Parses a shared library and returns its rpaths and dependencies.""" + path = os.path.realpath(path) + with open(path, "rb") as f: + magic_bytes = f.read(4) + + if magic_bytes == ELF_MAGIC: + return self._get_elf_info(path) + elif magic_bytes in MACHO_MAGICS: + return self._get_macho_info(path) + return {} + + def _get_elf_info(self, path): + """Extracts linking information from an ELF file.""" + info = {"rpaths": [], "needed": [], "undefined_symbols": []} + with open(path, "rb") as f: + elf = ELFFile(f) + dynamic = elf.get_section_by_name(".dynamic") + if dynamic: + for tag in dynamic.iter_tags(): + if tag.entry.d_tag == "DT_NEEDED": + info["needed"].append(tag.needed) + elif tag.entry.d_tag == "DT_RPATH": + info["rpaths"].append(tag.rpath) + elif tag.entry.d_tag == "DT_RUNPATH": + info["rpaths"].append(tag.runpath) + + dynsym = elf.get_section_by_name(".dynsym") + if dynsym: + info["undefined_symbols"] = [ + s.name + for s in dynsym.iter_symbols() + if s.entry["st_shndx"] == "SHN_UNDEF" + ] + return info + + def _get_macho_info(self, path): + """Extracts linking information from a Mach-O file.""" + info = {"rpaths": [], "needed": []} + macho = MachO(path) + for header in macho.headers: + for cmd_load, cmd, data in header.commands: + if cmd_load.cmd == mach_o.LC_LOAD_DYLIB: + info["needed"].append(data.decode().strip("\x00")) + elif cmd_load.cmd == mach_o.LC_RPATH: + info["rpaths"].append(data.decode().strip("\x00")) + return info + + def _assert_elf_linking(self, path): + """Asserts dynamic linking properties for an ELF file.""" + info = self._get_elf_info(path) + self.assertIn("libincrement.so", info["needed"]) + self.assertIn("increment", info["undefined_symbols"]) + + def _assert_macho_linking(self, path): + """Asserts dynamic linking properties for a Mach-O file.""" + info = self._get_macho_info(path) + self.assertIn("@rpath/libincrement.dylib", info["needed"]) + + +if __name__ == "__main__": + unittest.main() From f3e83e7119532f90ca58d9b1af91a50d4d1ea417 Mon Sep 17 00:00:00 2001 From: Ignas Anikevicius <240938+aignas@users.noreply.github.com> Date: Sun, 12 Oct 2025 01:16:59 +0900 Subject: [PATCH 485/922] fix(venv): include pth files at the root of the site-packages folder (#3340) Before this PR we would not include `pth` files at the root of the `site-packages` folder, but we would include if they are further down the tree. Fixes #3339 Fixes #2071 --- CHANGELOG.md | 3 +++ python/private/common.bzl | 1 + python/private/venv_runfiles.bzl | 3 ++- .../app_files_building/app_files_building_tests.bzl | 6 ++++++ 4 files changed, 12 insertions(+), 1 deletion(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 7782454fbd..aebcfccc15 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -105,6 +105,9 @@ END_UNRELEASED_TEMPLATE * (venvs) {obj}`--venvs_site_packages=yes` works for packages that dynamically link to shared libraries ([#3228](https://github.com/bazel-contrib/rules_python/issues/3228)). +* (venvs) {obj}`--venvs_site_packages=yes` includes `pth` files at the root of the + site-packages folder + ([#3339](https://github.com/bazel-contrib/rules_python/issues/3339)). * (uv) {obj}`//python/uv:lock.bzl%lock` now works with a local platform runtime. * (toolchains) WORKSPACE builds now correctly register musl and freethreaded diff --git a/python/private/common.bzl b/python/private/common.bzl index 33b175c247..ddeea6ed2d 100644 --- a/python/private/common.bzl +++ b/python/private/common.bzl @@ -36,6 +36,7 @@ PYTHON_FILE_EXTENSIONS = [ "dylib", # Python C modules, Mac specific "py", "pyc", + "pth", # import 'pth' files "pyi", "so", # Python C modules, usually Linux ] diff --git a/python/private/venv_runfiles.bzl b/python/private/venv_runfiles.bzl index 9bdacf833e..05dc296e15 100644 --- a/python/private/venv_runfiles.bzl +++ b/python/private/venv_runfiles.bzl @@ -290,9 +290,10 @@ def get_venv_symlinks(ctx, files, package, version_str, site_packages_root): entry = VenvSymlinkEntry( kind = VenvSymlinkKind.LIB, link_to_path = paths.join(runfiles_dir_name, site_packages_root, filename), + link_to_file = src, package = package, version = version_str, - venv_path = filename, + venv_path = path, files = depset([src]), ) venv_symlinks.append(entry) diff --git a/tests/venv_site_packages_libs/app_files_building/app_files_building_tests.bzl b/tests/venv_site_packages_libs/app_files_building/app_files_building_tests.bzl index 31c720a986..db2f21c7e7 100644 --- a/tests/venv_site_packages_libs/app_files_building/app_files_building_tests.bzl +++ b/tests/venv_site_packages_libs/app_files_building/app_files_building_tests.bzl @@ -271,6 +271,9 @@ def _test_shared_library_symlinking(name): "site-packages/foo.libs/libx.so", "site-packages/foo/a.py", "site-packages/foo/b.so", + "site-packages/root.pth", + "site-packages/root.py", + "site-packages/root.so", ], ) analysis_test( @@ -314,6 +317,9 @@ def _test_shared_library_symlinking_impl(env, target): "bar/y.so": srcs[2], "foo": "_main/tests/venv_site_packages_libs/app_files_building/site-packages/foo", "foo.libs/libx.so": srcs[3], + "root.pth": srcs[-3], + "root.py": srcs[-2], + "root.so": srcs[-1], } env.expect.that_dict(actual[VenvSymlinkKind.LIB]).contains_exactly(expected_libs) From 8069fa5916bc8e9ef1d282f5882ff2bdf11ad229 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Sun, 12 Oct 2025 00:50:44 -0700 Subject: [PATCH 486/922] internal: make release to chdir before looking for version markers (#3342) The release tool wasn't determining the next version correctly because it was in the wrong directory when that logic ran. To fix, chdir before that logic kicks off. --- tests/tools/private/release/release_test.py | 57 +++++++++++++++++++++ tools/private/release/release.py | 8 +-- 2 files changed, 61 insertions(+), 4 deletions(-) diff --git a/tests/tools/private/release/release_test.py b/tests/tools/private/release/release_test.py index 72a9a05cd6..676a898440 100644 --- a/tests/tools/private/release/release_test.py +++ b/tests/tools/private/release/release_test.py @@ -210,5 +210,62 @@ def test_get_latest_version_only_rc_tags(self, mock_get_tags): releaser.get_latest_version() +class DetermineNextVersionTest(unittest.TestCase): + def setUp(self): + self.tmpdir = pathlib.Path(tempfile.mkdtemp()) + self.original_cwd = os.getcwd() + self.addCleanup(shutil.rmtree, self.tmpdir) + + os.chdir(self.tmpdir) + # NOTE: On windows, this must be done before files are deleted. + self.addCleanup(os.chdir, self.original_cwd) + + self.mock_get_latest_version = patch( + "tools.private.release.release.get_latest_version" + ).start() + self.addCleanup(patch.stopall) + + def test_no_markers(self): + (self.tmpdir / "mock_file.bzl").write_text("no markers here") + self.mock_get_latest_version.return_value = "1.2.3" + + next_version = releaser.determine_next_version() + + self.assertEqual(next_version, "1.2.4") + + def test_only_patch(self): + (self.tmpdir / "mock_file.bzl").write_text( + ":::{versionchanged} VERSION_NEXT_PATCH" + ) + self.mock_get_latest_version.return_value = "1.2.3" + + next_version = releaser.determine_next_version() + + self.assertEqual(next_version, "1.2.4") + + def test_only_feature(self): + (self.tmpdir / "mock_file.bzl").write_text( + ":::{versionadded} VERSION_NEXT_FEATURE" + ) + self.mock_get_latest_version.return_value = "1.2.3" + + next_version = releaser.determine_next_version() + + self.assertEqual(next_version, "1.3.0") + + def test_both_markers(self): + (self.tmpdir / "mock_file_patch.bzl").write_text( + ":::{versionchanged} VERSION_NEXT_PATCH" + ) + (self.tmpdir / "mock_file_feature.bzl").write_text( + ":::{versionadded} VERSION_NEXT_FEATURE" + ) + self.mock_get_latest_version.return_value = "1.2.3" + + next_version = releaser.determine_next_version() + + self.assertEqual(next_version, "1.3.0") + + if __name__ == "__main__": unittest.main() diff --git a/tools/private/release/release.py b/tools/private/release/release.py index def6754347..6fce0ff3b0 100644 --- a/tools/private/release/release.py +++ b/tools/private/release/release.py @@ -170,6 +170,10 @@ def create_parser(): def main(): + # Change to the workspace root so the script can be run using `bazel run` + if "BUILD_WORKSPACE_DIRECTORY" in os.environ: + os.chdir(os.environ["BUILD_WORKSPACE_DIRECTORY"]) + parser = create_parser() args = parser.parse_args() @@ -179,10 +183,6 @@ def main(): version = determine_next_version() print(f"Determined next version: {version}") - # Change to the workspace root so the script can be run using `bazel run` - if "BUILD_WORKSPACE_DIRECTORY" in os.environ: - os.chdir(os.environ["BUILD_WORKSPACE_DIRECTORY"]) - print("Updating changelog ...") release_date = datetime.date.today().strftime("%Y-%m-%d") update_changelog(version, release_date) From 43a5acf8cedfce07fd4f933c9165f75e0b4d88c9 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Sun, 12 Oct 2025 00:51:44 -0700 Subject: [PATCH 487/922] chore: release 1.7 prep (#3341) * Update changelog * Update version markers Work towards https://github.com/bazel-contrib/rules_python/issues/3338 --- CHANGELOG.md | 16 ++++++++-------- docs/api/rules_python/python/cc/index.md | 2 +- docs/environment-variables.md | 4 ++-- python/extensions/config.bzl | 2 +- python/features.bzl | 2 +- python/private/attributes.bzl | 2 +- python/private/current_py_cc_headers.bzl | 2 +- python/private/py_cc_toolchain_info.bzl | 2 +- python/private/py_cc_toolchain_rule.bzl | 2 +- python/private/py_executable.bzl | 2 +- python/private/py_info.bzl | 2 +- python/runfiles/runfiles.py | 2 +- 12 files changed, 20 insertions(+), 20 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index aebcfccc15..d7c480582a 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -48,18 +48,18 @@ BEGIN_UNRELEASED_TEMPLATE END_UNRELEASED_TEMPLATE --> -{#v0-0-0} -## Unreleased +{#v1-7-0} +## [1.7.0] - 2025-10-11 -[0.0.0]: https://github.com/bazel-contrib/rules_python/releases/tag/0.0.0 +[1.7.0]: https://github.com/bazel-contrib/rules_python/releases/tag/1.7.0 -{#v0-0-0-removed} +{#v1-7-0-removed} ### Removed * (core rules) Support for Bazel's long deprecated "extra actions" has been removed ([#3215](https://github.com/bazel-contrib/rules_python/issues/3215)). -{#v0-0-0-changed} +{#v1-7-0-changed} ### Changed * (deps) bumped rules_cc dependency to `0.1.5`. * (bootstrap) For {obj}`--bootstrap_impl=system_python`, `PYTHONPATH` is no @@ -88,7 +88,7 @@ END_UNRELEASED_TEMPLATE [20251010]: https://github.com/astral-sh/python-build-standalone/releases/tag/20251010 -{#v0-0-0-fixed} +{#v1-7-0-fixed} ### Fixed * (rules) The `PyInfo` constructor was setting the wrong value for `has_py3_only_sources` - this is now fixed. @@ -117,7 +117,7 @@ END_UNRELEASED_TEMPLATE * (rules) {obj}`py_console_script_binary` is now compatible with symbolic macros ([#3195](https://github.com/bazel-contrib/rules_python/pull/3195)). -{#v0-0-0-added} +{#v1-7-0-added} ### Added * (runfiles) The Python runfiles library now supports Bazel's `--incompatible_compact_repo_mapping_manifest` flag. @@ -1980,4 +1980,4 @@ Breaking changes: * (pip) Create all_data_requirements alias * Expose Python C headers through the toolchain. -[0.24.0]: https://github.com/bazel-contrib/rules_python/releases/tag/0.24.0 +[0.24.0]: https://github.com/bazel-contrib/rules_python/releases/tag/0.24.0 \ No newline at end of file diff --git a/docs/api/rules_python/python/cc/index.md b/docs/api/rules_python/python/cc/index.md index 2f4e3ae171..98d68dacd5 100644 --- a/docs/api/rules_python/python/cc/index.md +++ b/docs/api/rules_python/python/cc/index.md @@ -35,7 +35,7 @@ This target provides: * `CcInfo`: The C++ information about the Python ABI3 headers. -:::{versionadded} VERSION_NEXT_FEATURE +:::{versionadded} 1.7.0 The {obj}`features.headers_abi3` attribute can be used to detect if this target is available or not. ::: diff --git a/docs/environment-variables.md b/docs/environment-variables.md index f0cf777a56..fba876f859 100644 --- a/docs/environment-variables.md +++ b/docs/environment-variables.md @@ -26,7 +26,7 @@ The {bzl:obj}`interpreter_args` attribute. :::{versionadded} 1.3.0 ::: -:::{versionchanged} VERSION_NEXT_FEATURE +:::{versionchanged} 1.7.0 Support added for {obj}`--bootstrap_impl=system_python`. ::: @@ -71,7 +71,7 @@ instead of the legacy Python scripts. :::{versionadded} 1.5.0 ::: -:::{versionchanged} VERSION_NEXT_FEATURE +:::{versionchanged} 1.7.0 Flipped to be enabled by default. ::: :::: diff --git a/python/extensions/config.bzl b/python/extensions/config.bzl index 2667b2a4fb..d8e621031e 100644 --- a/python/extensions/config.bzl +++ b/python/extensions/config.bzl @@ -43,7 +43,7 @@ def _config_impl(mctx): config = module_extension( doc = """Global settings for rules_python. -:::{versionadded} VERSION_NEXT_FEATURE +:::{versionadded} 1.7.0 ::: """, implementation = _config_impl, diff --git a/python/features.bzl b/python/features.bzl index 00bc1a7817..291de33d1a 100644 --- a/python/features.bzl +++ b/python/features.bzl @@ -26,7 +26,7 @@ def _features_typedef(): True if the {obj}`@rules_python//python/cc:current_py_cc_headers_abi3` target is available. - :::{versionadded} VERSION_NEXT_FEATURE + :::{versionadded} 1.7.0 ::: :::: diff --git a/python/private/attributes.bzl b/python/private/attributes.bzl index 8fef1bbe2c..0e0872fbf5 100644 --- a/python/private/attributes.bzl +++ b/python/private/attributes.bzl @@ -402,7 +402,7 @@ https://bazel.build/extending/config#memory-performance-considerations for more information about risks and considerations. ::: -:::{versionadded} VERSION_NEXT_FEATURE +:::{versionadded} 1.7.0 ::: """, ), diff --git a/python/private/current_py_cc_headers.bzl b/python/private/current_py_cc_headers.bzl index ef646317a6..f7fcd8d738 100644 --- a/python/private/current_py_cc_headers.bzl +++ b/python/private/current_py_cc_headers.bzl @@ -76,7 +76,7 @@ cc_library( ) ``` -:::{versionadded} VERSION_NEXT_FEATURE +:::{versionadded} 1.7.0 ::: """, ) diff --git a/python/private/py_cc_toolchain_info.bzl b/python/private/py_cc_toolchain_info.bzl index 8cb3680b59..34d4acf305 100644 --- a/python/private/py_cc_toolchain_info.bzl +++ b/python/private/py_cc_toolchain_info.bzl @@ -64,7 +64,7 @@ fields: e.g. `:current_py_cc_headers` to act as the underlying headers target it represents). -:::{versionadded} VERSION_NEXT_FEATURE +:::{versionadded} 1.7.0 The {obj}`features.headers_abi3` attribute can be used to detect if this attribute is available or not. ::: diff --git a/python/private/py_cc_toolchain_rule.bzl b/python/private/py_cc_toolchain_rule.bzl index b5c997ea6e..b89ea0e6b0 100644 --- a/python/private/py_cc_toolchain_rule.bzl +++ b/python/private/py_cc_toolchain_rule.bzl @@ -79,7 +79,7 @@ Target that provides the Python ABI3 (stable abi) headers. Typically this is a cc_library target. -:::{versionadded} VERSION_NEXT_FEATURE +:::{versionadded} 1.7.0 The {obj}`features.headers_abi3` attribute can be used to detect if this attribute is available or not. ::: diff --git a/python/private/py_executable.bzl b/python/private/py_executable.bzl index 0df04a96d9..ad1afa91cd 100644 --- a/python/private/py_executable.bzl +++ b/python/private/py_executable.bzl @@ -139,7 +139,7 @@ This is mutually exclusive with {obj}`main`. :::{versionadded} 1.3.0 ::: -:::{versionchanged} VERSION_NEXT_FEATURE +:::{versionchanged} 1.7.0 Support added for {obj}`--bootstrap_impl=system_python`. ::: """, diff --git a/python/private/py_info.bzl b/python/private/py_info.bzl index 95b739dff2..8868b9d3b4 100644 --- a/python/private/py_info.bzl +++ b/python/private/py_info.bzl @@ -79,7 +79,7 @@ the venv to create the path under. A file that `venv_path` should point to. The file to link to should also be in `files`. -:::{versionadded} VERSION_NEXT_FEATURE +:::{versionadded} 1.7.0 ::: """, "link_to_path": """ diff --git a/python/runfiles/runfiles.py b/python/runfiles/runfiles.py index 58f59c5406..fc794272c9 100644 --- a/python/runfiles/runfiles.py +++ b/python/runfiles/runfiles.py @@ -16,7 +16,7 @@ See @rules_python//python/runfiles/README.md for usage instructions. -:::{versionadded} VERSION_NEXT_FEATURE +:::{versionadded} 1.7.0 Support for Bazel's `--incompatible_compact_repo_mapping_manifest` flag was added. This enables prefix-based repository mappings to reduce memory usage for large dependency graphs under bzlmod. From 8f64e2c50f0d0581b8f2931ec30e71fd00fc7bcb Mon Sep 17 00:00:00 2001 From: Ignas Anikevicius <240938+aignas@users.noreply.github.com> Date: Wed, 15 Oct 2025 07:47:52 +0900 Subject: [PATCH 488/922] test(venv): functional test for pth files (#3343) As requested in #3340, this adds a functional test for it all. Tests for #3339 --------- Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> --- tests/venv_site_packages_libs/BUILD.bazel | 1 + tests/venv_site_packages_libs/bin.py | 2 ++ .../nested_with_pth/BUILD.bazel | 11 +++++++++++ .../nested_with_pth/site-packages/nested.pth | 1 + .../nested_sdk/nested_with_pth/__init__.py | 1 + 5 files changed, 16 insertions(+) create mode 100644 tests/venv_site_packages_libs/nested_with_pth/BUILD.bazel create mode 100644 tests/venv_site_packages_libs/nested_with_pth/site-packages/nested.pth create mode 100644 tests/venv_site_packages_libs/nested_with_pth/site-packages/nested_sdk/nested_with_pth/__init__.py diff --git a/tests/venv_site_packages_libs/BUILD.bazel b/tests/venv_site_packages_libs/BUILD.bazel index 2ce4ad9d22..2eb9678838 100644 --- a/tests/venv_site_packages_libs/BUILD.bazel +++ b/tests/venv_site_packages_libs/BUILD.bazel @@ -28,6 +28,7 @@ py_reconfig_test( venvs_site_packages = "yes", deps = [ ":closer_lib", + "//tests/venv_site_packages_libs/nested_with_pth", "//tests/venv_site_packages_libs/nspkg_alpha", "//tests/venv_site_packages_libs/nspkg_beta", "//tests/venv_site_packages_libs/pkgutil_top", diff --git a/tests/venv_site_packages_libs/bin.py b/tests/venv_site_packages_libs/bin.py index 772925f00e..c075f4fc65 100644 --- a/tests/venv_site_packages_libs/bin.py +++ b/tests/venv_site_packages_libs/bin.py @@ -40,6 +40,8 @@ def test_imported_from_venv(self): self.assert_imported_from_venv("nspkg.subnspkg.delta") self.assert_imported_from_venv("single_file") self.assert_imported_from_venv("simple") + m = self.assert_imported_from_venv("nested_with_pth") + self.assertEqual(m.WHOAMI, "nested_with_pth") def test_data_is_included(self): self.assert_imported_from_venv("simple") diff --git a/tests/venv_site_packages_libs/nested_with_pth/BUILD.bazel b/tests/venv_site_packages_libs/nested_with_pth/BUILD.bazel new file mode 100644 index 0000000000..68c16cfde9 --- /dev/null +++ b/tests/venv_site_packages_libs/nested_with_pth/BUILD.bazel @@ -0,0 +1,11 @@ +load("//python:py_library.bzl", "py_library") + +package(default_visibility = ["//visibility:public"]) + +py_library( + name = "nested_with_pth", + srcs = glob(["site-packages/**/*.py"]), + data = glob(["site-packages/*.pth"]), + experimental_venvs_site_packages = "//python/config_settings:venvs_site_packages", + imports = [package_name() + "/site-packages"], +) diff --git a/tests/venv_site_packages_libs/nested_with_pth/site-packages/nested.pth b/tests/venv_site_packages_libs/nested_with_pth/site-packages/nested.pth new file mode 100644 index 0000000000..924e0dccc8 --- /dev/null +++ b/tests/venv_site_packages_libs/nested_with_pth/site-packages/nested.pth @@ -0,0 +1 @@ +nested_sdk diff --git a/tests/venv_site_packages_libs/nested_with_pth/site-packages/nested_sdk/nested_with_pth/__init__.py b/tests/venv_site_packages_libs/nested_with_pth/site-packages/nested_sdk/nested_with_pth/__init__.py new file mode 100644 index 0000000000..adde64d3ef --- /dev/null +++ b/tests/venv_site_packages_libs/nested_with_pth/site-packages/nested_sdk/nested_with_pth/__init__.py @@ -0,0 +1 @@ +WHOAMI = "nested_with_pth" From 587f6e916d6bdd097b1b59b8c7a7b8128ed94dcb Mon Sep 17 00:00:00 2001 From: Josh Cannon Date: Tue, 14 Oct 2025 21:29:53 -0500 Subject: [PATCH 489/922] docs: Fix GitHub PR links (#3346) Fix the `gh-pr` special markup to use the right URL. It was using `/pulls/` which is a search, when it should be `/pull/` --- docs/conf.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/conf.py b/docs/conf.py index 47ab378cfb..671cf23fba 100644 --- a/docs/conf.py +++ b/docs/conf.py @@ -123,7 +123,7 @@ extlinks = { "gh-issue": (f"https://github.com/bazel-contrib/rules_python/issues/%s", "#%s issue"), "gh-path": (f"https://github.com/bazel-contrib/rules_python/tree/main/%s", "%s"), - "gh-pr": (f"https://github.com/bazel-contrib/rules_python/pulls/%s", "#%s PR"), + "gh-pr": (f"https://github.com/bazel-contrib/rules_python/pull/%s", "#%s PR"), } # --- MyST configuration From 7cf098e9897ecada15955dce3a45332208a739a1 Mon Sep 17 00:00:00 2001 From: Mai Hussien <70515749+mai93@users.noreply.github.com> Date: Wed, 15 Oct 2025 11:05:01 -0700 Subject: [PATCH 490/922] build: Starlarkify python flags (#3334) Add starlark flags for `--python_path`, `--build_python_zip` and `--incompatible_default_to_explicit_init_py`. - The transitions logic is updated to set both the native and starlark versions of the flags to allow alternating between them until the native ones are removed. - `--build_python_zip` is changed to `boolean` instead of `Tristate` with default value set to `True` on `windows` and `False` otherwise. - `scope = universal` attribute is added to the starlark flags so they can be propagated to exec config on Bazel 9. This required upgrading `bazel_skylib` version to have `scope` attribute defined. Work towards: https://github.com/bazel-contrib/rules_python/issues/3252 cc @gregestren --------- Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> Co-authored-by: Richard Levasseur --- .bazelrc | 1 + CHANGELOG.md | 2 + MODULE.bazel | 2 +- .../python/config_settings/index.md | 82 ++++++++++++++++ examples/build_file_generation/WORKSPACE | 96 +++++++++---------- gazelle/.bazelrc | 1 + gazelle/MODULE.bazel | 2 +- gazelle/WORKSPACE | 13 +++ internal_dev_deps.bzl | 6 +- python/config_settings/BUILD.bazel | 24 +++++ python/private/common_labels.bzl | 2 + python/private/flags.bzl | 2 +- python/private/internal_config_repo.bzl | 2 + python/private/py_executable.bzl | 10 ++ python/private/py_repositories.bzl | 6 +- tests/base_rules/py_executable_base_tests.bzl | 6 ++ tests/bootstrap_impls/BUILD.bazel | 10 +- .../transition/multi_version_tests.bzl | 8 +- tests/integration/custom_commands_test.py | 2 +- .../ignore_root_user_error/WORKSPACE | 15 --- tests/support/py_reconfig.bzl | 7 +- 21 files changed, 216 insertions(+), 83 deletions(-) diff --git a/.bazelrc b/.bazelrc index d7e1771336..801b963ad5 100644 --- a/.bazelrc +++ b/.bazelrc @@ -16,6 +16,7 @@ test --test_output=errors # creating (possibly empty) __init__.py files and adding them to the srcs of # Python targets as required. build --incompatible_default_to_explicit_init_py +build --//python/config_settings:incompatible_default_to_explicit_init_py=True # Ensure ongoing compatibility with this flag. common --incompatible_disallow_struct_provider_syntax diff --git a/CHANGELOG.md b/CHANGELOG.md index d7c480582a..4f9ac020d7 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -85,6 +85,8 @@ END_UNRELEASED_TEMPLATE evaluated, see [our docs](/pypi/download.html#customizing-requires-dist-resolution) on customizing `Requires-Dist` resolution. * (toolchains) Added Python versions 3.14.0, 3.13.8, 3.12.12, 3.11.14, 3.10.19, and 3.9.24 from the [20251010] release. +* (deps) (bzlmod) Upgraded to `bazel-skylib` version + [1.8.2](https://github.com/bazelbuild/bazel-skylib/releases/tag/1.8.2) [20251010]: https://github.com/astral-sh/python-build-standalone/releases/tag/20251010 diff --git a/MODULE.bazel b/MODULE.bazel index 36135bbb8b..5854595bed 100644 --- a/MODULE.bazel +++ b/MODULE.bazel @@ -5,7 +5,7 @@ module( ) bazel_dep(name = "bazel_features", version = "1.21.0") -bazel_dep(name = "bazel_skylib", version = "1.8.1") +bazel_dep(name = "bazel_skylib", version = "1.8.2") bazel_dep(name = "rules_cc", version = "0.1.5") bazel_dep(name = "platforms", version = "0.0.11") diff --git a/docs/api/rules_python/python/config_settings/index.md b/docs/api/rules_python/python/config_settings/index.md index 989ebf1128..78a74c3f37 100644 --- a/docs/api/rules_python/python/config_settings/index.md +++ b/docs/api/rules_python/python/config_settings/index.md @@ -24,6 +24,65 @@ This is a transition flag and will be removed in a subsequent release. :::: ::: +::::{bzl:flag} build_python_zip +Controls if a `py_binary/py_test` output is a self-executable zipapp. + +When enabled, the output of `py_binary` or `py_test` targets will be a +self-executable zipapp. + +:::{note} +This affects _all_ `py_binary` and `py_test` targets in the build, not +only the target(s) specified on the command line. +::: + +Values: +* `true` +* `false` + +This flag replaces the Bazel builtin `--build_python_zip` flag. + +:::{versionadded} VERSION_NEXT_FEATURE +::: +:::: + +::::{bzl:flag} experimental_python_import_all_repositories +Controls whether repository directories are added to the import path. + +When enabled, the top-level directories in the runfiles root directory (which +are presumbed to be repository directories) are added to the Python import +search path. + +It's recommended to set this to **`false`** to avoid external dependencies +unexpectedly interferring with import searching. + +Values; +* `true` (default) +* `false` + +This flag replaces the Bazel builtin +`--experimental_python_import_all_repositories` flag. + +:::{versionadded} VERSION_NEXT_FEATURE +::: +:::: + +::::{bzl:flag} python_path +A fallback path to use for Python for particular legacy Windows-specific code paths. + +Deprecated, do not use. This flag is largely a no-op and was replaced by +toolchains. It only remains for some legacy Windows code-paths that will +be removed. + +This flag replaces the Bazel builtin `--python_path` flag. + +:::{deprecated} VERSION_NEXT_FEATURE +Use toolchains instead. +::: + +:::{versionadded} VERSION_NEXT_FEATURE +::: +:::: + :::{bzl:flag} python_version Determines the default hermetic Python toolchain version. This can be set to one of the values that `rules_python` maintains. @@ -33,6 +92,29 @@ one of the values that `rules_python` maintains. Parses the value of the `python_version` and transforms it into a `X.Y` value. ::: +::::{bzl:flag} incompatible_default_to_explicit_init_py +Controls if missing `__init__.py` files are generated or not. + +If false, `py_binary` and `py_test` will, for every `*.py` and `*.so` file, +create `__init__.py` files for the containing directory, and all parent +directories, that do not already have an `__init__.py` file. If true, this +behavior is disabled. + +It's recommended to disable this behavior to avoid surprising import effects +from directories being importable when they otherwise wouldn't be, and for +how it can interfere with implicit namespace packages. + +Values: +* `true`: do not generate missing `__init__.py` files +* `false` (default): generate missing `__init__.py` files + +This flag replaces the Bazel builtin +`--incompatible_default_to_explicit_init_py` flag. + +:::{versionadded} VERSION_NEXT_FEATURE +::: +:::: + :::{bzl:target} is_python_* config_settings to match Python versions diff --git a/examples/build_file_generation/WORKSPACE b/examples/build_file_generation/WORKSPACE index 27f6ec071c..27d0d13b7c 100644 --- a/examples/build_file_generation/WORKSPACE +++ b/examples/build_file_generation/WORKSPACE @@ -7,54 +7,7 @@ workspace(name = "build_file_generation_example") # file. When the symbol is loaded you can use the rule. load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive") -###################################################################### -# We need rules_go and bazel_gazelle, to build the gazelle plugin from source. -# Setup instructions for this section are at -# https://github.com/bazelbuild/bazel-gazelle#running-gazelle-with-bazel -# You may need to update the version of the rule, which is listed in the above -# documentation. -###################################################################### - -# Define an http_archive rule that will download the below ruleset, -# test the sha, and extract the ruleset to you local bazel cache. - -http_archive( - name = "io_bazel_rules_go", - sha256 = "9d72f7b8904128afb98d46bbef82ad7223ec9ff3718d419afb355fddd9f9484a", - urls = [ - "https://mirror.bazel.build/github.com/bazel-contrib/rules_go/releases/download/v0.55.1/rules_go-v0.55.1.zip", - "https://github.com/bazel-contrib/rules_go/releases/download/v0.55.1/rules_go-v0.55.1.zip", - ], -) - -# Download the bazel_gazelle ruleset. -http_archive( - name = "bazel_gazelle", - sha256 = "75df288c4b31c81eb50f51e2e14f4763cb7548daae126817247064637fd9ea62", - urls = [ - "https://mirror.bazel.build/github.com/bazelbuild/bazel-gazelle/releases/download/v0.36.0/bazel-gazelle-v0.36.0.tar.gz", - "https://github.com/bazelbuild/bazel-gazelle/releases/download/v0.36.0/bazel-gazelle-v0.36.0.tar.gz", - ], -) - -# Load rules_go ruleset and expose the toolchain and dep rules. -load("@bazel_gazelle//:deps.bzl", "gazelle_dependencies") -load("@io_bazel_rules_go//go:deps.bzl", "go_register_toolchains", "go_rules_dependencies") - -# go_rules_dependencies is a function that registers external dependencies -# needed by the Go rules. -# See: https://github.com/bazelbuild/rules_go/blob/master/go/dependencies.rst#go_rules_dependencies -go_rules_dependencies() - -# go_rules_dependencies is a function that registers external dependencies -# needed by the Go rules. -# See: https://github.com/bazelbuild/rules_go/blob/master/go/dependencies.rst#go_rules_dependencies -go_register_toolchains(version = "1.21.13") - -# The following call configured the gazelle dependencies, Go environment and Go SDK. -gazelle_dependencies() - -# Remaining setup is for rules_python. +# Setup rules_python. # DON'T COPY_PASTE THIS. # Our example uses `local_repository` to point to the HEAD version of rules_python. @@ -124,6 +77,53 @@ load("@pip//:requirements.bzl", "install_deps") # Initialize repositories for all packages in requirements_lock.txt. install_deps() +###################################################################### +# We need rules_go and bazel_gazelle, to build the gazelle plugin from source. +# Setup instructions for this section are at +# https://github.com/bazelbuild/bazel-gazelle#running-gazelle-with-bazel +# You may need to update the version of the rule, which is listed in the above +# documentation. +###################################################################### + +# Define an http_archive rule that will download the below ruleset, +# test the sha, and extract the ruleset to you local bazel cache. + +http_archive( + name = "io_bazel_rules_go", + sha256 = "9d72f7b8904128afb98d46bbef82ad7223ec9ff3718d419afb355fddd9f9484a", + urls = [ + "https://mirror.bazel.build/github.com/bazel-contrib/rules_go/releases/download/v0.55.1/rules_go-v0.55.1.zip", + "https://github.com/bazel-contrib/rules_go/releases/download/v0.55.1/rules_go-v0.55.1.zip", + ], +) + +# Download the bazel_gazelle ruleset. +http_archive( + name = "bazel_gazelle", + sha256 = "75df288c4b31c81eb50f51e2e14f4763cb7548daae126817247064637fd9ea62", + urls = [ + "https://mirror.bazel.build/github.com/bazelbuild/bazel-gazelle/releases/download/v0.36.0/bazel-gazelle-v0.36.0.tar.gz", + "https://github.com/bazelbuild/bazel-gazelle/releases/download/v0.36.0/bazel-gazelle-v0.36.0.tar.gz", + ], +) + +# Load rules_go ruleset and expose the toolchain and dep rules. +load("@bazel_gazelle//:deps.bzl", "gazelle_dependencies") +load("@io_bazel_rules_go//go:deps.bzl", "go_register_toolchains", "go_rules_dependencies") + +# go_rules_dependencies is a function that registers external dependencies +# needed by the Go rules. +# See: https://github.com/bazelbuild/rules_go/blob/master/go/dependencies.rst#go_rules_dependencies +go_rules_dependencies() + +# go_rules_dependencies is a function that registers external dependencies +# needed by the Go rules. +# See: https://github.com/bazelbuild/rules_go/blob/master/go/dependencies.rst#go_rules_dependencies +go_register_toolchains(version = "1.21.13") + +# The following call configured the gazelle dependencies, Go environment and Go SDK. +gazelle_dependencies() + # The rules_python gazelle extension has some third-party go dependencies # which we need to fetch in order to compile it. load("@rules_python_gazelle_plugin//:deps.bzl", _py_gazelle_deps = "gazelle_deps") diff --git a/gazelle/.bazelrc b/gazelle/.bazelrc index 97040903a6..791b93912a 100644 --- a/gazelle/.bazelrc +++ b/gazelle/.bazelrc @@ -7,6 +7,7 @@ test --test_output=errors # creating (possibly empty) __init__.py files and adding them to the srcs of # Python targets as required. build --incompatible_default_to_explicit_init_py +build --@rules_python//python/config_settings:incompatible_default_to_explicit_init_py=True # Windows makes use of runfiles for some rules build --enable_runfiles diff --git a/gazelle/MODULE.bazel b/gazelle/MODULE.bazel index 1560e73d7b..add5986903 100644 --- a/gazelle/MODULE.bazel +++ b/gazelle/MODULE.bazel @@ -4,7 +4,7 @@ module( compatibility_level = 1, ) -bazel_dep(name = "bazel_skylib", version = "1.6.1") +bazel_dep(name = "bazel_skylib", version = "1.8.2") bazel_dep(name = "rules_python", version = "0.18.0") bazel_dep(name = "rules_go", version = "0.55.1", repo_name = "io_bazel_rules_go") bazel_dep(name = "gazelle", version = "0.36.0", repo_name = "bazel_gazelle") diff --git a/gazelle/WORKSPACE b/gazelle/WORKSPACE index ec0532c3f6..f4a3abf36c 100644 --- a/gazelle/WORKSPACE +++ b/gazelle/WORKSPACE @@ -2,6 +2,19 @@ workspace(name = "rules_python_gazelle_plugin") load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive") +http_archive( + name = "bazel_skylib", + sha256 = "6e78f0e57de26801f6f564fa7c4a48dc8b36873e416257a92bbb0937eeac8446", + urls = [ + "https://mirror.bazel.build/github.com/bazelbuild/bazel-skylib/releases/download/1.8.2/bazel-skylib-1.8.2.tar.gz", + "https://github.com/bazelbuild/bazel-skylib/releases/download/1.8.2/bazel-skylib-1.8.2.tar.gz", + ], +) + +load("@bazel_skylib//:workspace.bzl", "bazel_skylib_workspace") + +bazel_skylib_workspace() + http_archive( name = "io_bazel_rules_go", sha256 = "9d72f7b8904128afb98d46bbef82ad7223ec9ff3718d419afb355fddd9f9484a", diff --git a/internal_dev_deps.bzl b/internal_dev_deps.bzl index 91f5defd3e..811240a06a 100644 --- a/internal_dev_deps.bzl +++ b/internal_dev_deps.bzl @@ -60,10 +60,10 @@ def rules_python_internal_deps(): http_archive( name = "bazel_skylib", - sha256 = "bc283cdfcd526a52c3201279cda4bc298652efa898b10b4db0837dc51652756f", + sha256 = "6e78f0e57de26801f6f564fa7c4a48dc8b36873e416257a92bbb0937eeac8446", urls = [ - "https://mirror.bazel.build/github.com/bazelbuild/bazel-skylib/releases/download/1.7.1/bazel-skylib-1.7.1.tar.gz", - "https://github.com/bazelbuild/bazel-skylib/releases/download/1.7.1/bazel-skylib-1.7.1.tar.gz", + "https://mirror.bazel.build/github.com/bazelbuild/bazel-skylib/releases/download/1.8.2/bazel-skylib-1.8.2.tar.gz", + "https://github.com/bazelbuild/bazel-skylib/releases/download/1.8.2/bazel-skylib-1.8.2.tar.gz", ], ) diff --git a/python/config_settings/BUILD.bazel b/python/config_settings/BUILD.bazel index cc5c472fe7..369989eb1e 100644 --- a/python/config_settings/BUILD.bazel +++ b/python/config_settings/BUILD.bazel @@ -1,5 +1,6 @@ load("@bazel_skylib//rules:common_settings.bzl", "bool_flag", "string_flag") load("@pythons_hub//:versions.bzl", "DEFAULT_PYTHON_VERSION", "MINOR_MAPPING", "PYTHON_VERSIONS") +load("@rules_python_internal//:rules_python_config.bzl", "config") load( "//python/private:flags.bzl", "AddSrcsToRunfilesFlag", @@ -244,5 +245,28 @@ label_flag( bool_flag( name = "experimental_python_import_all_repositories", build_setting_default = True, + scope = "universal", + visibility = ["//visibility:public"], +) + +bool_flag( + name = "build_python_zip", + build_setting_default = config.build_python_zip_default, + help = "Build python executable zip. Defaults to on on Windows, off on other platforms", + scope = "universal", + visibility = ["//visibility:public"], +) + +bool_flag( + name = "incompatible_default_to_explicit_init_py", + build_setting_default = False, + scope = "universal", + visibility = ["//visibility:public"], +) + +string_flag( + name = "python_path", + build_setting_default = "python", + scope = "universal", visibility = ["//visibility:public"], ) diff --git a/python/private/common_labels.bzl b/python/private/common_labels.bzl index 4a6f6d3f0f..e90679eb6f 100644 --- a/python/private/common_labels.bzl +++ b/python/private/common_labels.bzl @@ -7,6 +7,7 @@ labels = struct( # keep sorted ADD_SRCS_TO_RUNFILES = str(Label("//python/config_settings:add_srcs_to_runfiles")), BOOTSTRAP_IMPL = str(Label("//python/config_settings:bootstrap_impl")), + BUILD_PYTHON_ZIP = str(Label("//python/config_settings:build_python_zip")), EXEC_TOOLS_TOOLCHAIN = str(Label("//python/config_settings:exec_tools_toolchain")), PIP_ENV_MARKER_CONFIG = str(Label("//python/config_settings:pip_env_marker_config")), NONE = str(Label("//python:none")), @@ -18,6 +19,7 @@ labels = struct( PRECOMPILE = str(Label("//python/config_settings:precompile")), PRECOMPILE_SOURCE_RETENTION = str(Label("//python/config_settings:precompile_source_retention")), PYC_COLLECTION = str(Label("//python/config_settings:pyc_collection")), + PYTHON_IMPORT_ALL_REPOSITORIES = str(Label("//python/config_settings:experimental_python_import_all_repositories")), PYTHON_SRC = str(Label("//python/bin:python_src")), PYTHON_VERSION = str(Label("//python/config_settings:python_version")), PYTHON_VERSION_MAJOR_MINOR = str(Label("//python/config_settings:python_version_major_minor")), diff --git a/python/private/flags.bzl b/python/private/flags.bzl index 35181e9f96..d9e3aa41c3 100644 --- a/python/private/flags.bzl +++ b/python/private/flags.bzl @@ -71,7 +71,7 @@ def read_possibly_native_flag(ctx, flag_name): return _POSSIBLY_NATIVE_FLAGS[flag_name][0](ctx) else: # Starlark definition of "--foo" is assumed to be a label dependency named "_foo". - return getattr(ctx.attr, "_" + flag_name)[BuildSettingInfo].value + return getattr(ctx.attr, "_" + flag_name + "_flag")[BuildSettingInfo].value def _AddSrcsToRunfilesFlag_is_enabled(ctx): value = ctx.attr._add_srcs_to_runfiles_flag[BuildSettingInfo].value diff --git a/python/private/internal_config_repo.bzl b/python/private/internal_config_repo.bzl index 0c6210696e..dac6d741a5 100644 --- a/python/private/internal_config_repo.bzl +++ b/python/private/internal_config_repo.bzl @@ -28,6 +28,7 @@ _ENABLE_DEPRECATION_WARNINGS_DEFAULT = "0" _CONFIG_TEMPLATE = """ config = struct( + build_python_zip_default = {build_python_zip_default}, enable_pystar = True, enable_pipstar = {enable_pipstar}, enable_deprecation_warnings = {enable_deprecation_warnings}, @@ -96,6 +97,7 @@ def _internal_config_repo_impl(rctx): builtin_py_cc_link_params_provider = "PyCcLinkParamsProvider" rctx.file("rules_python_config.bzl", _CONFIG_TEMPLATE.format( + build_python_zip_default = repo_utils.get_platforms_os_name(rctx) == "windows", enable_pipstar = _bool_from_environ(rctx, _ENABLE_PIPSTAR_ENVVAR_NAME, _ENABLE_PIPSTAR_DEFAULT), enable_deprecation_warnings = _bool_from_environ(rctx, _ENABLE_DEPRECATION_WARNINGS_ENVVAR_NAME, _ENABLE_DEPRECATION_WARNINGS_DEFAULT), builtin_py_info_symbol = builtin_py_info_symbol, diff --git a/python/private/py_executable.bzl b/python/private/py_executable.bzl index ad1afa91cd..1a5ad4c3c6 100644 --- a/python/private/py_executable.bzl +++ b/python/private/py_executable.bzl @@ -77,6 +77,13 @@ EXECUTABLE_ATTRS = dicts.add( AGNOSTIC_EXECUTABLE_ATTRS, PY_SRCS_ATTRS, IMPORTS_ATTRS, + # starlark flags attributes + { + "_build_python_zip_flag": attr.label(default = "//python/config_settings:build_python_zip"), + "_default_to_explicit_init_py_flag": attr.label(default = "//python/config_settings:incompatible_default_to_explicit_init_py"), + "_python_import_all_repositories_flag": attr.label(default = "//python/config_settings:experimental_python_import_all_repositories"), + "_python_path_flag": attr.label(default = "//python/config_settings:python_path"), + }, { "interpreter_args": lambda: attrb.StringList( doc = """ @@ -1136,6 +1143,9 @@ def _get_runtime_details(ctx, semantics): # TOOD(bazelbuild/bazel#7901): Remove this once --python_path flag is removed. flag_interpreter_path = read_possibly_native_flag(ctx, "python_path") + if not flag_interpreter_path.startswith("python") and not paths.is_absolute(flag_interpreter_path): + fail("'python_path' must be an absolute path or a name to be resolved from the system PATH (e.g., 'python', 'python3').") + toolchain_runtime, effective_runtime = _maybe_get_runtime_from_ctx(ctx) if not effective_runtime: # Clear these just in case diff --git a/python/private/py_repositories.bzl b/python/private/py_repositories.bzl index 3ad2a97214..e3ab11c561 100644 --- a/python/private/py_repositories.bzl +++ b/python/private/py_repositories.bzl @@ -60,10 +60,10 @@ def py_repositories(transition_settings = []): ) http_archive( name = "bazel_skylib", - sha256 = "d00f1389ee20b60018e92644e0948e16e350a7707219e7a390fb0a99b6ec9262", + sha256 = "6e78f0e57de26801f6f564fa7c4a48dc8b36873e416257a92bbb0937eeac8446", urls = [ - "https://mirror.bazel.build/github.com/bazelbuild/bazel-skylib/releases/download/1.7.0/bazel-skylib-1.7.0.tar.gz", - "https://github.com/bazelbuild/bazel-skylib/releases/download/1.7.0/bazel-skylib-1.7.0.tar.gz", + "https://mirror.bazel.build/github.com/bazelbuild/bazel-skylib/releases/download/1.8.2/bazel-skylib-1.8.2.tar.gz", + "https://github.com/bazelbuild/bazel-skylib/releases/download/1.8.2/bazel-skylib-1.8.2.tar.gz", ], ) http_archive( diff --git a/tests/base_rules/py_executable_base_tests.bzl b/tests/base_rules/py_executable_base_tests.bzl index e86a94990a..e41bc2c022 100644 --- a/tests/base_rules/py_executable_base_tests.bzl +++ b/tests/base_rules/py_executable_base_tests.bzl @@ -44,7 +44,10 @@ def _test_basic_windows(name, config): # the target platform. For windows, it defaults to true, so force # it to that to match behavior when this test runs on other # platforms. + # Pass value to both native and starlark versions of the flag until + # the native one is removed. "//command_line_option:build_python_zip": "true", + labels.BUILD_PYTHON_ZIP: True, "//command_line_option:cpu": "windows_x86_64", "//command_line_option:crosstool_top": CROSSTOOL_TOP, "//command_line_option:extra_execution_platforms": [platform_targets.WINDOWS_X86_64], @@ -87,7 +90,10 @@ def _test_basic_zip(name, config): # the target platform. For windows, it defaults to true, so force # it to that to match behavior when this test runs on other # platforms. + # Pass value to both native and starlark versions of the flag until + # the native one is removed. "//command_line_option:build_python_zip": "true", + labels.BUILD_PYTHON_ZIP: True, "//command_line_option:cpu": "linux_x86_64", "//command_line_option:crosstool_top": CROSSTOOL_TOP, "//command_line_option:extra_execution_platforms": [platform_targets.LINUX_X86_64], diff --git a/tests/bootstrap_impls/BUILD.bazel b/tests/bootstrap_impls/BUILD.bazel index c3d44df240..dcc27514f7 100644 --- a/tests/bootstrap_impls/BUILD.bazel +++ b/tests/bootstrap_impls/BUILD.bazel @@ -23,7 +23,7 @@ py_reconfig_binary( srcs = ["bin.py"], bootstrap_impl = "script", # Force it to not be self-executable - build_python_zip = "no", + build_python_zip = False, main = "bin.py", target_compatible_with = SUPPORTS_BOOTSTRAP_SCRIPT, ) @@ -50,14 +50,14 @@ sh_test( sh_py_run_test( name = "run_binary_zip_no_test", - build_python_zip = "no", + build_python_zip = False, py_src = "bin.py", sh_src = "run_binary_zip_no_test.sh", ) sh_py_run_test( name = "run_binary_zip_yes_test", - build_python_zip = "yes", + build_python_zip = True, py_src = "bin.py", sh_src = "run_binary_zip_yes_test.sh", ) @@ -81,7 +81,7 @@ sh_py_run_test( sh_py_run_test( name = "run_binary_bootstrap_script_zip_yes_test", bootstrap_impl = "script", - build_python_zip = "yes", + build_python_zip = True, py_src = "bin.py", sh_src = "run_binary_zip_yes_test.sh", target_compatible_with = SUPPORTS_BOOTSTRAP_SCRIPT, @@ -90,7 +90,7 @@ sh_py_run_test( sh_py_run_test( name = "run_binary_bootstrap_script_zip_no_test", bootstrap_impl = "script", - build_python_zip = "no", + build_python_zip = False, py_src = "bin.py", sh_src = "run_binary_zip_no_test.sh", target_compatible_with = SUPPORTS_BOOTSTRAP_SCRIPT, diff --git a/tests/config_settings/transition/multi_version_tests.bzl b/tests/config_settings/transition/multi_version_tests.bzl index dfe2bf9981..05f010562c 100644 --- a/tests/config_settings/transition/multi_version_tests.bzl +++ b/tests/config_settings/transition/multi_version_tests.bzl @@ -20,6 +20,7 @@ load("@rules_testing//lib:util.bzl", rt_util = "util") load("//python:py_binary.bzl", "py_binary") load("//python:py_info.bzl", "PyInfo") load("//python:py_test.bzl", "py_test") +load("//python/private:common_labels.bzl", "labels") # buildifier: disable=bzl-visibility load("//python/private:reexports.bzl", "BuiltinPyInfo") # buildifier: disable=bzl-visibility load("//tests/support:support.bzl", "CC_TOOLCHAIN") load("//tests/support/platforms:platforms.bzl", "platform_targets") @@ -91,7 +92,8 @@ def _setup_py_binary_windows(name, *, impl, build_python_zip): target = name + "_subject", impl = impl, config_settings = { - "//command_line_option:build_python_zip": build_python_zip, + "//command_line_option:build_python_zip": str(build_python_zip), + labels.BUILD_PYTHON_ZIP: build_python_zip, "//command_line_option:extra_toolchains": CC_TOOLCHAIN, "//command_line_option:platforms": str(platform_targets.WINDOWS_X86_64), }, @@ -100,7 +102,7 @@ def _setup_py_binary_windows(name, *, impl, build_python_zip): def _test_py_binary_windows_build_python_zip_false(name): _setup_py_binary_windows( name, - build_python_zip = "false", + build_python_zip = False, impl = _test_py_binary_windows_build_python_zip_false_impl, ) @@ -121,7 +123,7 @@ _tests.append(_test_py_binary_windows_build_python_zip_false) def _test_py_binary_windows_build_python_zip_true(name): _setup_py_binary_windows( name, - build_python_zip = "true", + build_python_zip = True, impl = _test_py_binary_windows_build_python_zip_true_impl, ) diff --git a/tests/integration/custom_commands_test.py b/tests/integration/custom_commands_test.py index 2e9cb741b0..288a4e7a91 100644 --- a/tests/integration/custom_commands_test.py +++ b/tests/integration/custom_commands_test.py @@ -21,7 +21,7 @@ class CustomCommandsTest(runner.TestCase): # Regression test for https://github.com/bazel-contrib/rules_python/issues/1840 def test_run_build_python_zip_false(self): - result = self.run_bazel("run", "--build_python_zip=false", "//:bin") + result = self.run_bazel("run", "--build_python_zip=false", "--@rules_python//python/config_settings:build_python_zip=false", "//:bin") self.assert_result_matches(result, "bazel-out") diff --git a/tests/integration/ignore_root_user_error/WORKSPACE b/tests/integration/ignore_root_user_error/WORKSPACE index 0a25819ecd..7ac0a609eb 100644 --- a/tests/integration/ignore_root_user_error/WORKSPACE +++ b/tests/integration/ignore_root_user_error/WORKSPACE @@ -1,5 +1,3 @@ -load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive") - local_repository( name = "rules_python", path = "../../..", @@ -14,16 +12,3 @@ python_register_toolchains( ignore_root_user_error = True, python_version = "3.9", ) - -http_archive( - name = "bazel_skylib", - sha256 = "c6966ec828da198c5d9adbaa94c05e3a1c7f21bd012a0b29ba8ddbccb2c93b0d", - urls = [ - "https://mirror.bazel.build/github.com/bazelbuild/bazel-skylib/releases/download/1.1.1/bazel-skylib-1.1.1.tar.gz", - "https://github.com/bazelbuild/bazel-skylib/releases/download/1.1.1/bazel-skylib-1.1.1.tar.gz", - ], -) - -load("@bazel_skylib//:workspace.bzl", "bazel_skylib_workspace") - -bazel_skylib_workspace() diff --git a/tests/support/py_reconfig.bzl b/tests/support/py_reconfig.bzl index d52cc5dd95..efcb0e7a38 100644 --- a/tests/support/py_reconfig.bzl +++ b/tests/support/py_reconfig.bzl @@ -17,6 +17,7 @@ This facilitates verify running binaries with different configuration settings without the overhead of a bazel-in-bazel integration test. """ +load("@rules_python_internal//:rules_python_config.bzl", "config") load("//python/private:attr_builders.bzl", "attrb") # buildifier: disable=bzl-visibility load("//python/private:common_labels.bzl", "labels") # buildifier: disable=bzl-visibility load("//python/private:py_binary_macro.bzl", "py_binary_macro") # buildifier: disable=bzl-visibility @@ -30,7 +31,8 @@ def _perform_transition_impl(input_settings, attr, base_impl): settings.update(base_impl(input_settings, attr)) settings[labels.VISIBLE_FOR_TESTING] = True - settings["//command_line_option:build_python_zip"] = attr.build_python_zip + settings["//command_line_option:build_python_zip"] = str(attr.build_python_zip) + settings[labels.BUILD_PYTHON_ZIP] = attr.build_python_zip if attr.bootstrap_impl: settings[labels.BOOTSTRAP_IMPL] = attr.bootstrap_impl if attr.extra_toolchains: @@ -58,13 +60,14 @@ _RECONFIG_INPUTS = [ ] _RECONFIG_OUTPUTS = _RECONFIG_INPUTS + [ "//command_line_option:build_python_zip", + labels.BUILD_PYTHON_ZIP, labels.VISIBLE_FOR_TESTING, ] _RECONFIG_INHERITED_OUTPUTS = [v for v in _RECONFIG_OUTPUTS if v in _RECONFIG_INPUTS] _RECONFIG_ATTRS = { "bootstrap_impl": attrb.String(), - "build_python_zip": attrb.String(default = "auto"), + "build_python_zip": attrb.Bool(default = config.build_python_zip_default), "config_settings": attrb.LabelKeyedStringDict(), "extra_toolchains": attrb.StringList( doc = """ From 46167f5008fb1e4c90783efadef1c8293d599554 Mon Sep 17 00:00:00 2001 From: Kristian Hartikainen Date: Wed, 15 Oct 2025 14:07:42 -0400 Subject: [PATCH 491/922] doc: Fix `pip.default` arguments in multi-platform example (#3358) I was playing around with the [multi-platform PyPI example](https://rules-python.readthedocs.io/en/latest/howto/multi-platform-pypi-deps.html) and noticed that the `pip.default` arguments were wrong. Co-authored-by: Richard Levasseur --- docs/howto/multi-platform-pypi-deps.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/docs/howto/multi-platform-pypi-deps.md b/docs/howto/multi-platform-pypi-deps.md index 6cc7f842ea..61f3f40580 100644 --- a/docs/howto/multi-platform-pypi-deps.md +++ b/docs/howto/multi-platform-pypi-deps.md @@ -163,16 +163,16 @@ pip = use_extension("@rules_python//python/extensions:pip.bzl", "pip") # A custom platform for CUDA on glibc linux pip.default( platform = "linux_x86_64_cuda12.9", - os = "linux", - cpu = "x86_64", + arch_name = "x86_64", + os_name = "linux", config_settings = ["@//:is_cuda_12_9"], ) # A custom platform for musl on linux pip.default( platform = "linux_aarch64_musl", - os = "linux", - cpu = "aarch64", + os_name = "linux", + arch_name = "aarch64", config_settings = ["@//:is_musl"], ) From f1e585fc9927aeaf87cf43845e1d37289f881e99 Mon Sep 17 00:00:00 2001 From: Ignas Anikevicius <240938+aignas@users.noreply.github.com> Date: Thu, 16 Oct 2025 14:02:09 +0900 Subject: [PATCH 492/922] chore(toolchain): use the last build and add 3.15.0a1 (#3357) Updating the toolchain versions to fix a bug in `3.13.8`. This will make the release better to everyone. --------- Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> --- CHANGELOG.md | 8 +- python/versions.bzl | 214 +++++++++++++++++++++++++++----------------- 2 files changed, 134 insertions(+), 88 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 4f9ac020d7..23e05f02a5 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -83,12 +83,12 @@ END_UNRELEASED_TEMPLATE vendoring the `requirements.bzl` file, please re-vendor so that downstream is unaffected when the APIs get removed. If you need to customize the way the dependencies get evaluated, see [our docs](/pypi/download.html#customizing-requires-dist-resolution) on customizing `Requires-Dist` resolution. -* (toolchains) Added Python versions 3.14.0, 3.13.8, 3.12.12, 3.11.14, 3.10.19, and 3.9.24 - from the [20251010] release. +* (toolchains) Added Python versions 3.15.0a1, 3.14.0, 3.13.9, 3.12.12, 3.11.14, 3.10.19, and 3.9.24 + from the [20251014] release. * (deps) (bzlmod) Upgraded to `bazel-skylib` version [1.8.2](https://github.com/bazelbuild/bazel-skylib/releases/tag/1.8.2) -[20251010]: https://github.com/astral-sh/python-build-standalone/releases/tag/20251010 +[20251014]: https://github.com/astral-sh/python-build-standalone/releases/tag/20251014 {#v1-7-0-fixed} ### Fixed @@ -1982,4 +1982,4 @@ Breaking changes: * (pip) Create all_data_requirements alias * Expose Python C headers through the toolchain. -[0.24.0]: https://github.com/bazel-contrib/rules_python/releases/tag/0.24.0 \ No newline at end of file +[0.24.0]: https://github.com/bazel-contrib/rules_python/releases/tag/0.24.0 diff --git a/python/versions.bzl b/python/versions.bzl index ebd4d7f718..6de09ca72d 100644 --- a/python/versions.bzl +++ b/python/versions.bzl @@ -205,17 +205,17 @@ TOOL_VERSIONS = { "strip_prefix": "python", }, "3.9.24": { - "url": "20251010/cpython-{python_version}+20251010-{platform}-{build}.tar.gz", + "url": "20251014/cpython-{python_version}+20251014-{platform}-{build}.tar.gz", "sha256": { - "aarch64-apple-darwin": "03af9b83cdda23c8b82537f720cc37ddc0d5635b282c8ea8326382f3f0221141", - "aarch64-unknown-linux-gnu": "e1aff69545ab15dc22ffe9f28c8d162af0e4be9a9cf57c0a50a0916508964ccb", - "ppc64le-unknown-linux-gnu": "c70891539abf23746dcfb32043bf91c6f2dfcd97106ff1ed75b744f9e00b5551", - "riscv64-unknown-linux-gnu": "ce180e819d992fda0c523fba88db79b9d9bc9b12a52d548479d84c7b86d5bcda", - "s390x-unknown-linux-gnu": "9a830f9b8fa62962e22f5aa105f12fde9d40d7969d701e9600d580e4f0cb7ac6", - "x86_64-apple-darwin": "aabb592ba83217c092744f53c14da10b6e3dab6f96e0eefd346e6ac0c0723dd2", - "x86_64-pc-windows-msvc": "28fd634e9e20f2a94e29e8c8c8a5384a2882091e9d0af981e7e99cc156be1648", - "x86_64-unknown-linux-gnu": "fe63b51f543011e0582a98259d98fbc0033bc2009398404446f350fe9e531aa3", - "x86_64-unknown-linux-musl": "844db0ca41ccf897f58fbefd8f5df7ef18ea374788e6ce4cb87989f997c3f21a", + "aarch64-apple-darwin": "6b65213e639e91eb8072db80ed9c140d769af1d5e0386efd8f153449c3694714", + "aarch64-unknown-linux-gnu": "d840efd9d81ad557019ebd0d435828fc32101cd01be82046087b4aee463dca0c", + "ppc64le-unknown-linux-gnu": "e6501df1f32cc9cbfa8bb625b4d5a88ad9e83452525c1989ad50334a16a5d9a6", + "riscv64-unknown-linux-gnu": "811f0f3966f42186a59ae9112b8faf92bbe88fae8dae725f072fee116b628b2a", + "s390x-unknown-linux-gnu": "7bf6bb7a95527419379c94b5d3181f7000f47e7c5a828cde58b0f7cfe9421347", + "x86_64-apple-darwin": "14beda9465feb6991f73d6f6cb9e69afc576c5cac8c185bd729f491aa4305bfb", + "x86_64-pc-windows-msvc": "a2fdaf290361386396bbfaa08e13fc2b88e1149f870adf18836e262c609406db", + "x86_64-unknown-linux-gnu": "866745efbee219a3f9b9d54ee1477ebf92542bb9ff9f6591a7e5a3643a0d4214", + "x86_64-unknown-linux-musl": "ee1dec977925293be46cecc5f7e9034394f0f8cc736afc92528689e59d6f19db", }, "strip_prefix": "python", }, @@ -370,17 +370,17 @@ TOOL_VERSIONS = { "strip_prefix": "python", }, "3.10.19": { - "url": "20251010/cpython-{python_version}+20251010-{platform}-{build}.tar.gz", + "url": "20251014/cpython-{python_version}+20251014-{platform}-{build}.tar.gz", "sha256": { - "aarch64-apple-darwin": "4918c7b7dd9bdeb81b2a30b5305d85e04c6f322ed715b598bf7afbfd03b39f23", - "aarch64-unknown-linux-gnu": "356fce3bed7d4c416c552d054cf647886b3825b285a06b4b8782440dccc5f5c1", - "ppc64le-unknown-linux-gnu": "b129fcc438aee69429c1d90dd251579393eaf690d5cf9f327a41c79448ff062f", - "riscv64-unknown-linux-gnu": "118e4444dc8a98d7f52cca86b1f9a4636c45eba3d759689bd86f68c579fe03d0", - "s390x-unknown-linux-gnu": "d62e65e77199f35d54cae383977ccbf30fca1ba683f96e6ffc04a17d5d31e959", - "x86_64-apple-darwin": "90665ea564d409de944e7e985e9ddeecc1d61980616983d26b06bc721cfa8469", - "x86_64-pc-windows-msvc": "1e56b702e080723a76d92ba04d0f8b6e49986113b575c7026797d1de53a5b89b", - "x86_64-unknown-linux-gnu": "4f1c6812961ed57e408f2385a8b16745d3eb62b7c2e6e0eef7c05d37e4ba806e", - "x86_64-unknown-linux-musl": "e90c84c95440212e1c98d0bd7a54196746e886fdc8512ae7e8d6a7d46f058ef1", + "aarch64-apple-darwin": "06cfdfa8966dfd86204d45c6a241dd37cb0b3ede90986591fc0b0dbe576848de", + "aarch64-unknown-linux-gnu": "c4c760f49dbba10a0f91b2fd52c847dd50cbe7cb8cb19bb7598c4dc38a358e9c", + "ppc64le-unknown-linux-gnu": "8d32d9c85ac6ac71f6996313f87d50da34a159e037d3795bbb745f1c39d7b62f", + "riscv64-unknown-linux-gnu": "636d0001877c1d2566e3bd4be61c6df08ba55eefd06414cb72a22e154432c22a", + "s390x-unknown-linux-gnu": "1b15c9c090114c063a5802e005ea35c61a3c4e83efb8e8ce687d77f47060f8ed", + "x86_64-apple-darwin": "b4e0c82f350f18a8fb1b1982f03c1c90aaba5d9ab74fe6ede9896306f64a287c", + "x86_64-pc-windows-msvc": "e2d9193b2d2fd99fac3fb90eda216100b64cd7cf14f291d9425436ea9b1eaa04", + "x86_64-unknown-linux-gnu": "85c96114de83d783db18137f3858bcd3b5a9c4cbe9053f0072d7b5f52154a8c9", + "x86_64-unknown-linux-musl": "0d0f2b1f8bb014018dc4c24b6680f17f48017dafe25e380cefc2490e4b90e1ae", }, "strip_prefix": "python", }, @@ -516,18 +516,18 @@ TOOL_VERSIONS = { "strip_prefix": "python", }, "3.11.14": { - "url": "20251010/cpython-{python_version}+20251010-{platform}-{build}.tar.gz", + "url": "20251014/cpython-{python_version}+20251014-{platform}-{build}.tar.gz", "sha256": { - "aarch64-apple-darwin": "29ec457de1b5765eeade189efbe27e2b1f8c7a96e4b79471b7d41a18094b1870", - "aarch64-unknown-linux-gnu": "d46c18f9da8a673cc55de55d8cfb8ed3164849eac50edc222985b60a9eda3be3", - "ppc64le-unknown-linux-gnu": "fd395aa11d82a48bfe3cfdfcd41759ee4b65b3d1de67466329aaef284164650b", - "riscv64-unknown-linux-gnu": "c3b529408c176a222c863c1810ee6a635a7e9deb5f2c73c425181a2383d7da2a", - "s390x-unknown-linux-gnu": "ade377b4668e4a03bd0c2a5316c079b8c0d2d63db7ecd05f24715309c4efb298", - "x86_64-apple-darwin": "ae375cd49fecfc3aecbf942544c34a1c5251f2c2f9b19e0e2b889a9113ccfa62", - "x86_64-pc-windows-msvc": "58a2571b5268fc7891e28cab01f23c3561ef5bc146f1314e32ba32a06754442b", - "aarch64-pc-windows-msvc": "67600a84ba1cf43a826a31c1dbb144b987a6b4627f6e5fb3d34a54511c356187", - "x86_64-unknown-linux-gnu": "848789e630ada4012e64ac744a3f4a8342b975d69f9608c460bdf9f370fa1d30", - "x86_64-unknown-linux-musl": "fed38a53fdff4327295f052a2970692ca194e53444843b5f91bd478c7ffed1ba", + "aarch64-apple-darwin": "99d98bf73d9906d18a9184054a328288ede2cb4a2d245a05411a28e8d023aab6", + "aarch64-unknown-linux-gnu": "8b033614f3a6969d86c20f9b823277ee8e1f72788307c082a44d2ad4cc856e2b", + "ppc64le-unknown-linux-gnu": "3936f10e39f3ceeb422514f996de7f4ad095241be22df3f5db007c92f6ae1ac7", + "riscv64-unknown-linux-gnu": "790247290650896b40b7a1ca9e47b6951ac3d0750850b356033386ebf05edf80", + "s390x-unknown-linux-gnu": "459989097b6ac89c7b940ae8eb2f3508ea4f12d6c1ff192b4dbc1bb47e95ad2a", + "x86_64-apple-darwin": "d234fa6518634daf3aa812895ec757d0e0b1fea3335fd0c5038d4e2bcc5d7ee5", + "x86_64-pc-windows-msvc": "80022423ca581c88d5bb7beb889f10c12d3d8d2e5cc6422fd2b060b52e45aa05", + "aarch64-pc-windows-msvc": "94958c60345574c1cfdee7e57925642cdf2eb2008b64a0018ca9c3b509ce16b0", + "x86_64-unknown-linux-gnu": "d0623c777fb89b904b56cd5aba51af29cbb34b1f9d45f0672f90f6dce30fa93e", + "x86_64-unknown-linux-musl": "0ce7c9f584fa51860f79f4f6c7fe22a6bbd986d324acb23ad8c9f237c8af964a", }, "strip_prefix": "python", }, @@ -656,18 +656,18 @@ TOOL_VERSIONS = { "strip_prefix": "python", }, "3.12.12": { - "url": "20251010/cpython-{python_version}+20251010-{platform}-{build}.tar.gz", + "url": "20251014/cpython-{python_version}+20251014-{platform}-{build}.tar.gz", "sha256": { - "aarch64-apple-darwin": "2577a2629c89b3ff40dc16271cc8826d9ae20217e5a3189bbc7646b496e77687", - "aarch64-unknown-linux-gnu": "21bcf71dccb56ef611f50543b04e63e6585ac063463f2d248cb4ec28118d264d", - "aarch64-pc-windows-msvc": "a36719b442c22488f1ef7caea68943cf0c8dd367330fa5baac3cab4168a8e66a", - "ppc64le-unknown-linux-gnu": "782fbe38b63216cce3fdc4c7a2da86337ccef615fa9384acba7457ef17ad96a2", - "riscv64-unknown-linux-gnu": "c9bbb36a75466386497a38db0f9f707e63aaa8bae38c64d55683242d9b5b56a6", - "s390x-unknown-linux-gnu": "90e3f010692e65425cb503ae789e0ffefb3b5962b83e21d7c642d34cf056d48b", - "x86_64-apple-darwin": "c9fb9ab36c742f14388fd5b6a67cab5e6e1726b52624dafbd37d0176ad1752e1", - "x86_64-pc-windows-msvc": "c110c11de4299b0273caa99a1c7b895427c9441a231b3124bf5228a2e463ef43", - "x86_64-unknown-linux-gnu": "fbf55136d0f955ca2f93aeb7830f993d93acf1b9a729a15c3b6b5220a36bc835", - "x86_64-unknown-linux-musl": "bffd28435d28d55b33e0b90a5521b0220fe38c5ff67b2706cbc9ceb32af0e4a7", + "aarch64-apple-darwin": "6ceba34fe78802853a30bde6f303a0a54f71f6ab07a673da34e90c0aa06c786e", + "aarch64-unknown-linux-gnu": "d32487b853d6f5709019a471770be5e5d3e6bd2ac507e5629e2d6825565d3e71", + "aarch64-pc-windows-msvc": "d708734581e8cb03f4cf95f39f17ea331bc4761dfdad99b6b738a245444c9c54", + "ppc64le-unknown-linux-gnu": "951d2d4fb4d6bee3e9e100c06215cd7621ef9b4e70651870b1efb9e14caa3dd0", + "riscv64-unknown-linux-gnu": "b0b5e1d48cc5d1612a316bd59dc6179efad9644affff41a0820c4791151bb802", + "s390x-unknown-linux-gnu": "e290368e5d0f1e393733f26f4d05f666b36140c38b83b8e66182756940a396de", + "x86_64-apple-darwin": "9b8589eefb153cbe7cb652993d0ecc94aeb2fa13c1a2e8bc240f5f74f23bb21b", + "x86_64-pc-windows-msvc": "2d670beb3b930d30e3a13cc909923a001dbdfcb5537692d5da40b6b41643ce1c", + "x86_64-unknown-linux-gnu": "1ab2b6594d1c3d76cbebea09d6bc3e6ba68d8eb3b6322080375c4cc3dd188f34", + "x86_64-unknown-linux-musl": "d3395c3267617f49363f9114999685b865b2731804e3954e89b681254f62da4c", }, "strip_prefix": "python", }, @@ -872,28 +872,28 @@ TOOL_VERSIONS = { "x86_64-unknown-linux-gnu-freethreaded": "python/install", }, }, - "3.13.8": { - "url": "20251010/cpython-{python_version}+20251010-{platform}-{build}.{ext}", + "3.13.9": { + "url": "20251014/cpython-{python_version}+20251014-{platform}-{build}.{ext}", "sha256": { - "aarch64-apple-darwin": "0dc9061b9d8e02a9344aa569eecb795f41f16ac8bb215f973d8db9179700e296", - "aarch64-unknown-linux-gnu": "1b9af3c628cdbef3b9eb4c61df628bbc54f4684ace147f3dc9850c336c44c125", - "ppc64le-unknown-linux-gnu": "c3bae08423519b5224bfc87323b96381bdb9c589fbdc09cd3ce289f6c1ea4ed3", - "riscv64-unknown-linux-gnu": "e8be781bfaf5ad6b83094db96dd60d7dfcd4519f6550740354b4ea57acfbfa21", - "s390x-unknown-linux-gnu": "3c9faf91356d06b4cc993487a71915e7a0585aa592691f6afe8841188e024653", - "x86_64-apple-darwin": "e3b692599bb3c247a2f95c21577f8b85b70924a5f2d672c9e0005608d7b9c907", - "x86_64-pc-windows-msvc": "1dde7aab47a52e81b6bd3f7d1bc5fa2f3c9e428eb5f54f51e8b92ae0a3e2409f", - "aarch64-pc-windows-msvc": "430d9073f22c744d6ea8c224d7b458a6be3620f9bd2389068908c214ea4423f8", - "aarch64-pc-windows-msvc-freethreaded": "187b962d84af18d22f67e069776dff61a223271ae11f44507cfd8693430c03c4", - "x86_64-unknown-linux-gnu": "12dd8995e8ec2df68cd1301b053f415c7884b8aae9d3459a2ac1448f781dbbbc", - "x86_64-unknown-linux-musl": "fbf17b5acd1a33a7a0a6bcabab7065f3adc3de77efbc230555a5019843070b1d", - "aarch64-apple-darwin-freethreaded": "0b7a80716c2557800d8e3133744b9e913476d259cea0160dd8597e58284a5a1c", - "aarch64-unknown-linux-gnu-freethreaded": "5cf655065b59493d39235fec0e30c33a5196bb01cf74cef50907a4f18c8ef02a", - "ppc64le-unknown-linux-gnu-freethreaded": "dbae65b6e747537939c3f2804077bb4e49f528850e68800d54b0e8c4b7768fab", - "riscv64-unknown-linux-gnu-freethreaded": "10390b394f76cb14c808b5c87de37189bc72796e55a55e6e5aca83e4615dd5ca", - "s390x-unknown-linux-gnu-freethreaded": "7e9d32fae045cefe950d8f95ddf1824edc232d1c67b4cdd55ecec5042f05727d", - "x86_64-apple-darwin-freethreaded": "1112caf275e374fbe629c3fde265f1e7675b21553f1607085cedc821d184a4aa", - "x86_64-pc-windows-msvc-freethreaded": "d4b83250fe9fea9563b2fb79111d4af072279b690507ddb14df11f617fd57f7e", - "x86_64-unknown-linux-gnu-freethreaded": "714eec8f42eb023d0c9ed289115d52e96ce73dac7464a5d242b0988eeda90b56", + "aarch64-apple-darwin": "931db8f735e18700d4eab9ee39dbbd0b4c114d7d039dd2707b2d932ded039698", + "aarch64-unknown-linux-gnu": "c86606a45fb6540b1b66d9c52c6f5466fba8affb29acb9ab6a0b7f5ad54e588a", + "ppc64le-unknown-linux-gnu": "80218541bb73f7ccf7fe82660b403b6c35edfe91fc58052392f738a94dbd27ae", + "riscv64-unknown-linux-gnu": "8b210482f6fc46ae2b75fa21ba2e8edce3d11e5c27aa5d841acfc6b95778edb6", + "s390x-unknown-linux-gnu": "8e8cc90192da6ae59c6f26a084fb6f63ef228686643aad983f6183184881babd", + "x86_64-apple-darwin": "9f6bc3c15e2f9e2c9c90db2c8b3ee94598e777789f8aea6e36b69ae55d007d01", + "x86_64-pc-windows-msvc": "8b0efc2674bb293ce2d423d59765b1ca3a2d80dc0ca6168f6279cb569e72b55e", + "aarch64-pc-windows-msvc": "d4de66a7ad3f7c9acaf2db41148097f303985ff7f712795d436d21550ab5ff76", + "aarch64-pc-windows-msvc-freethreaded": "9510f4f9790aa800e6e1163eea450523a5be47a348051b31365a685143b3e17e", + "x86_64-unknown-linux-gnu": "b4b0204658930337c85c321b49ed2585fe544097a72bc76dcf0b77e49fff8473", + "x86_64-unknown-linux-musl": "1e227f10d59c197111c3cea81e352b9f13a136f44cf7bae87368987c48127055", + "aarch64-apple-darwin-freethreaded": "9e78bb28a4ef9d8195caa08586ded2468d575814af6806a9c34fe175614fd3c9", + "aarch64-unknown-linux-gnu-freethreaded": "3f13ad9d0f026e1c0cefe13415b0b965eff3a91c43a7e0c63d8f26fde2382f86", + "ppc64le-unknown-linux-gnu-freethreaded": "323a197e31c966f144bd0e94d8f8c0ee20775190f8b2a91efb191c612d6e94cb", + "riscv64-unknown-linux-gnu-freethreaded": "62d7dbd8ff4c64aeca2aa895c46ab0102433b44bf31b7971d48e3655e9d94688", + "s390x-unknown-linux-gnu-freethreaded": "4b54fe09739628b97aece3231f2ed4e2553ee0b41d0921dfef81fe50968f9afd", + "x86_64-apple-darwin-freethreaded": "405bbf1e443d12e48959ffc7c32674468226dff2c163b75f486686af9f8f7be4", + "x86_64-pc-windows-msvc-freethreaded": "50c5830e814eb057fed984b15dad250c62fda2e54a18ee9789ee2ba89e1951af", + "x86_64-unknown-linux-gnu-freethreaded": "515b92ab30010596ab239dad848c88af88703a054a04b70b5cf0ad22f107c75e", }, "strip_prefix": { "aarch64-apple-darwin": "python", @@ -918,27 +918,72 @@ TOOL_VERSIONS = { }, }, "3.14.0": { - "url": "20251010/cpython-{python_version}+20251010-{platform}-{build}.{ext}", + "url": "20251014/cpython-{python_version}+20251014-{platform}-{build}.{ext}", "sha256": { - "aarch64-apple-darwin": "e97387b1fe9048b949cb6616af2984581e84cb068198f73e96d646e3efa33a53", - "aarch64-unknown-linux-gnu": "7464a4086f40ca91f015f147156698734fc8d9185b3d7a5c7d83877a7c182cf1", - "ppc64le-unknown-linux-gnu": "2383189e07675207733b4b155dcc89892d33b92d36e52cce9426d827c05e9b05", - "riscv64-unknown-linux-gnu": "842e1bf032f467cfba267c7db2e0c9344ccf9a58612d81437f8da70538b0d3aa", - "s390x-unknown-linux-gnu": "7b93d636e05aa3c42b971706280d3c2540eed3b18b10968809b1731d0969d6ed", - "x86_64-apple-darwin": "3f8ee5c2087e355188cb443250d4a8c1bffb3b5e751b61b423ac195bfd8dd87f", - "x86_64-pc-windows-msvc": "3d8b2cc6f554f3ad7e54837be51b9dbb1f228900e9acc098a713f6191d33e649", - "aarch64-pc-windows-msvc": "d63fa3939ffa91b73712ba276ca0235183f35fa896e689bae6e37f5d53a702d3", - "x86_64-unknown-linux-gnu": "cf872c369be9ed424092be50d4228ffc8b46469f18307e869023770dd3011aa3", - "x86_64-unknown-linux-musl": "a8b923c7a77e6b9e14018d2774663d538fe87c9da7ba0e081061d651f9802c19", - "aarch64-apple-darwin-freethreaded": "0f0c33fd55475e7192d08c6a37575412c4de886be438e4f359004994c83fc907", - "aarch64-unknown-linux-gnu-freethreaded": "531635fd1c064d03cd14f24b82c531a664de1b8998677eae4c9401b3c9330e26", - "ppc64le-unknown-linux-gnu-freethreaded": "9ea379e46fe0e9befccc24b9b244be9aef7c0589c857654cdb586e3d3ef62464", - "riscv64-unknown-linux-gnu-freethreaded": "63d6ba87237bc6804cb4ca5d87bb1c4267759b5211e434f0187babc5d1a6b12f", - "s390x-unknown-linux-gnu-freethreaded": "2e877903f2f266ec60b661e3b0b074908c8e1699449f907d403eb551e289fa48", - "x86_64-apple-darwin-freethreaded": "3ca690d329ec7d7b950a3430c859969dec6af9290c36d5107990bcbc04635d7c", - "x86_64-pc-windows-msvc-freethreaded": "60240e379461ac802cb6fc3e06a700557a22c616337458dedb94099eee8d2353", - "aarch64-pc-windows-msvc-freethreaded": "4d388a28cd002a3d5d4929b2a40d6341351fbdbbb5811f9bf07c8b3cc5812101", - "x86_64-unknown-linux-gnu-freethreaded": "ddab9e3f5da84f7a330df813d5736802beb0e7179aeaa20e7c2bc4623ec0db5c", + "aarch64-apple-darwin": "1333ce2807fbea673eb242edbf4997ea1e2f6cbc01cd80dec1f9d19de2cd63ed", + "aarch64-unknown-linux-gnu": "e613f44e60227b3423a994698426698569e055c24447c10dd9c1c022cf511f05", + "ppc64le-unknown-linux-gnu": "91d164d5480015c7e6c441255cf4bcd182e0c4124e028716e58e1efef6418936", + "riscv64-unknown-linux-gnu": "11c9807fc52bae34a81ba8bd7cb35f5360be428b867377710a69c793e3917725", + "s390x-unknown-linux-gnu": "2e35106929e6f5a8d568f890522cdc7ad4382f696ad266537035753fb4916626", + "x86_64-apple-darwin": "0a4cc33ca56830b92545950aacdde8925c9d4259e4f00ceda04fedf853f70679", + "x86_64-pc-windows-msvc": "d90e97fe69b819f0a776cd665d06fef6526a4259211d11f00e501688659f1c0e", + "aarch64-pc-windows-msvc": "1359d52eaa584da8a76decbe4255f89dae47d81757b9f422b91467824ecfdd7f", + "x86_64-unknown-linux-gnu": "74d4516a64abc63ae4bcbffb35482879a85b7faa187fcfa47c1ca8f00faebf5f", + "x86_64-unknown-linux-musl": "7e603f71788edb5e6a4d92273eafb4d609972cd45f330032b6728c0d9753c37e", + "aarch64-apple-darwin-freethreaded": "1c61fa9c9979cfe74f992dd2f15cfed644ee9feec78e12c894ae446044186f74", + "aarch64-unknown-linux-gnu-freethreaded": "c2c5e0be76d7151b6a1c0fdd4ef58e0b81d36902580311c1c8c2b4b075ed3190", + "ppc64le-unknown-linux-gnu-freethreaded": "a1b9dc2130017208b04551a3b7e502e740691e0987f911ed392c8e6d77d611f9", + "riscv64-unknown-linux-gnu-freethreaded": "099f8e056f17f09dcb137c15aa162fe390055d4ad17d2115a8f7adbb4e768ec1", + "s390x-unknown-linux-gnu-freethreaded": "8e71db4558557315e56051fb59a0b2ed2701f7f9240339f867dc7b6440a72209", + "x86_64-apple-darwin-freethreaded": "a57a872d96f2711909181cf7da7b7a86e3bec293621ac54de3f89ea8d3fbb3bd", + "x86_64-pc-windows-msvc-freethreaded": "730449333b24fae53ce6872d8ade13564773f1fc652f926ca641a6a228e71dd6", + "aarch64-pc-windows-msvc-freethreaded": "dd062cff01d7c55c2fc5596c387423420155bafd03dae5fa9ecd66ee89df6695", + "x86_64-unknown-linux-gnu-freethreaded": "56ef2dbc787a0f75d63ab38b4f6b1a0b1f35ce1f710b68e8080aee9d6c1c7453", + }, + "strip_prefix": { + "aarch64-apple-darwin": "python", + "aarch64-unknown-linux-gnu": "python", + "ppc64le-unknown-linux-gnu": "python", + "s390x-unknown-linux-gnu": "python", + "riscv64-unknown-linux-gnu": "python", + "x86_64-apple-darwin": "python", + "x86_64-pc-windows-msvc": "python", + "aarch64-pc-windows-msvc": "python", + "x86_64-unknown-linux-gnu": "python", + "x86_64-unknown-linux-musl": "python", + "aarch64-apple-darwin-freethreaded": "python/install", + "aarch64-unknown-linux-gnu-freethreaded": "python/install", + "ppc64le-unknown-linux-gnu-freethreaded": "python/install", + "riscv64-unknown-linux-gnu-freethreaded": "python/install", + "s390x-unknown-linux-gnu-freethreaded": "python/install", + "x86_64-apple-darwin-freethreaded": "python/install", + "x86_64-pc-windows-msvc-freethreaded": "python/install", + "aarch64-pc-windows-msvc-freethreaded": "python/install", + "x86_64-unknown-linux-gnu-freethreaded": "python/install", + }, + }, + "3.15.0a1": { + "url": "20251014/cpython-{python_version}+20251014-{platform}-{build}.{ext}", + "sha256": { + "aarch64-apple-darwin": "b17d1c8dd0ee32004124345a1944891a3e11c3549c0c2575c192e785dc0ca452", + "aarch64-unknown-linux-gnu": "5b82e1cd640e6249794de367e6154682836a6919ea96b9b15309a624d293724d", + "ppc64le-unknown-linux-gnu": "68bc72f8f960d497002035f0ecfa5b22d866467e1d11e2bc56c441b3f63d50a5", + "riscv64-unknown-linux-gnu": "b535892e6f7f28856802d235198044facd129c4031ad8b1d2a00952e5b7f1c00", + "s390x-unknown-linux-gnu": "6a82a1b0490c5bca2ec69f0accead17bf86f60ac5de90335bf68d942e87e6bc3", + "x86_64-apple-darwin": "a8cddf0b4974be662dc157364360606af66ffe56d5b95e6b6c9d06e76b8cad16", + "x86_64-pc-windows-msvc": "3a9bea65091cbbd4b6db1ecbd99ca4da8d0ffe32360b953345f24eaba4a89fc8", + "aarch64-pc-windows-msvc": "9ed03d369562bfd6900dcc5b503193355388ab0c1c93268a68671ca5b6e8ae2e", + "x86_64-unknown-linux-gnu": "5fb9150d98c4e4d153bee6e5e5626882901b77d00e1fb7e481f4aa36d4b57c8d", + "x86_64-unknown-linux-musl": "f79b24cd6c9952c43f16d7f1812ecb99b5339385b479d60975f96eb212033f3e", + "aarch64-apple-darwin-freethreaded": "c66c98b7257f568510a8a988fa22a369ddb4fd2b031768a9e65aca43a3dc575a", + "aarch64-unknown-linux-gnu-freethreaded": "3a78b661e488e2e1fd9b614901af659bce295c9eee307313636bb358b8f11b6e", + "ppc64le-unknown-linux-gnu-freethreaded": "3a1b7c0e9c055ed4fc26d1029fc262ece9947db9e7346e7bf354b18f4ba7b9f1", + "riscv64-unknown-linux-gnu-freethreaded": "c0a3ff7053bf98398bd9595bc04b04e4c2dd03eaae607f57a74541abac494edc", + "s390x-unknown-linux-gnu-freethreaded": "95f337e46e0ec5266b00ff93235675cfdf88da583168da277f7dc67804268926", + "x86_64-apple-darwin-freethreaded": "6512751c57469ccdba1309f90a756feb6704b3af03ceeef2a1ec8e8f4a30554d", + "x86_64-pc-windows-msvc-freethreaded": "60636fc054223d3f83c387dccd084933fc5ab4d7182d9e47df05469d1ba05595", + "aarch64-pc-windows-msvc-freethreaded": "d624349224906d1653fb1c5338a931492767d21436530941172116209848d62d", + "x86_64-unknown-linux-gnu-freethreaded": "003c2125829b0859b1cbce74351f089963eec33ad071f89befb9a373005e8a24", }, "strip_prefix": { "aarch64-apple-darwin": "python", @@ -971,8 +1016,9 @@ MINOR_MAPPING = { "3.10": "3.10.19", "3.11": "3.11.14", "3.12": "3.12.12", - "3.13": "3.13.8", + "3.13": "3.13.9", "3.14": "3.14.0", + "3.15": "3.15.0a1", } def _generate_platforms(): From cd49a1cab10a1953c6228063bbb12948307ba44c Mon Sep 17 00:00:00 2001 From: Ignas Anikevicius <240938+aignas@users.noreply.github.com> Date: Thu, 16 Oct 2025 23:07:10 +0900 Subject: [PATCH 493/922] fix(doc): fix the release notes for the starlarkification of the flags (#3361) It seems that the changelog got merged for `1.7.0`, but the `VERSION_NEXT_FEATURE` got added to the docs instead of `1.7.0`. This is fixing that so that we can merge/cherry-pick again to the release branch. --- docs/api/rules_python/python/config_settings/index.md | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/docs/api/rules_python/python/config_settings/index.md b/docs/api/rules_python/python/config_settings/index.md index 78a74c3f37..3092326d6f 100644 --- a/docs/api/rules_python/python/config_settings/index.md +++ b/docs/api/rules_python/python/config_settings/index.md @@ -41,7 +41,7 @@ Values: This flag replaces the Bazel builtin `--build_python_zip` flag. -:::{versionadded} VERSION_NEXT_FEATURE +:::{versionadded} 1.7.0 ::: :::: @@ -62,7 +62,7 @@ Values; This flag replaces the Bazel builtin `--experimental_python_import_all_repositories` flag. -:::{versionadded} VERSION_NEXT_FEATURE +:::{versionadded} 1.7.0 ::: :::: @@ -75,11 +75,11 @@ be removed. This flag replaces the Bazel builtin `--python_path` flag. -:::{deprecated} VERSION_NEXT_FEATURE +:::{deprecated} 1.7.0 Use toolchains instead. ::: -:::{versionadded} VERSION_NEXT_FEATURE +:::{versionadded} 1.7.0 ::: :::: @@ -111,7 +111,7 @@ Values: This flag replaces the Bazel builtin `--incompatible_default_to_explicit_init_py` flag. -:::{versionadded} VERSION_NEXT_FEATURE +:::{versionadded} 1.7.0 ::: :::: From b9ec06fc498f8b01a9779120100cc009a2c83b33 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Thu, 16 Oct 2025 16:16:49 -0700 Subject: [PATCH 494/922] chore: switch to use publish-to-bcr workflow (#3359) The GitHub App is deprecated, so switch to the more modern workflow. This will also allow us to eventually use attestations. Along the way... * Split up the release workflow into some different sub-jobs * Run PyPI upload last, as its the most irrevocable step of the process. --- .github/workflows/publish.yml | 35 +++++++++++++++++++++++ .github/workflows/release.yml | 52 +++++++++++++++++++++++------------ 2 files changed, 69 insertions(+), 18 deletions(-) create mode 100644 .github/workflows/publish.yml diff --git a/.github/workflows/publish.yml b/.github/workflows/publish.yml new file mode 100644 index 0000000000..f03e02168f --- /dev/null +++ b/.github/workflows/publish.yml @@ -0,0 +1,35 @@ +# See https://github.com/bazel-contrib/publish-to-bcr +name: Publish to BCR + +on: + # Run the publish workflow after a successful release + # Can be triggered from the release.yaml workflow + workflow_call: + inputs: + tag_name: + required: true + type: string + secrets: + BCR_PUBLISH_TOKEN: + required: true + # In case of problems, let release engineers retry by manually dispatching + # the workflow from the GitHub UI + workflow_dispatch: + inputs: + tag_name: + required: true + type: string + +jobs: + publish: + uses: bazel-contrib/publish-to-bcr/.github/workflows/publish.yaml@v1.0.0 + with: + draft: false + tag_name: ${{ inputs.tag_name }} + # GitHub repository which is a fork of the upstream where the Pull Request will be opened. + registry_fork: bazel-contrib/bazel-central-registry + attest: false + permissions: + contents: write + secrets: + publish_token: ${{ secrets.BCR_PUBLISH_TOKEN }} diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index 7a25c6eca0..0ed4992ccd 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -35,21 +35,37 @@ jobs: uses: actions/checkout@v5 - name: Create release archive and notes run: .github/workflows/create_archive_and_notes.sh - - name: Publish wheel dist - if: github.event_name == 'push' || github.event.inputs.publish_to_pypi - env: - # This special value tells pypi that the user identity is supplied within the token - TWINE_USERNAME: __token__ - # Note, the PYPI_API_TOKEN is for the rules-python pypi user, added by @rickylev on - # https://github.com/bazel-contrib/rules_python/settings/secrets/actions - TWINE_PASSWORD: ${{ secrets.PYPI_API_TOKEN }} - run: bazel run --stamp --embed_label=${{ github.ref_name }} //python/runfiles:wheel.publish - - name: Release - uses: softprops/action-gh-release@v2 - with: - # Use GH feature to populate the changelog automatically - generate_release_notes: true - body_path: release_notes.txt - prerelease: ${{ contains(github.ref, '-rc') }} - fail_on_unmatched_files: true - files: rules_python-*.tar.gz + + release: + name: Release + uses: softprops/action-gh-release@v2 + with: + # Use GH feature to populate the changelog automatically + generate_release_notes: true + body_path: release_notes.txt + prerelease: ${{ contains(github.ref, '-rc') }} + fail_on_unmatched_files: true + files: rules_python-*.tar.gz + + publish_bcr: + name: Publish to BCR + needs: release + uses: .github/workflows/publish.yaml + with: + tag_name: ${{ github.ref_name }} + secrets: + BCR_PUBLISH_TOKEN: ${{ secrets.BCR_PUBLISH_TOKEN }} + + publish_pypi: + # We just want publish_pypi last, since once uploaded, it can't be changed. + name: Publish runfiles to PyPI + needs: publish_bcr + runs-on: ubuntu-latest + if: github.event_name == 'push' || github.event.inputs.publish_to_pypi + env: + # This special value tells pypi that the user identity is supplied within the token + TWINE_USERNAME: __token__ + # Note, the PYPI_API_TOKEN is for the rules-python pypi user, added by @rickylev on + # https://github.com/bazel-contrib/rules_python/settings/secrets/actions + TWINE_PASSWORD: ${{ secrets.PYPI_API_TOKEN }} + run: bazel run --stamp --embed_label=${{ github.ref_name }} //python/runfiles:wheel.publish From e487b6dddd2c3b233f94e4a343d9cf0cb32d60ec Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Thu, 16 Oct 2025 20:49:21 -0700 Subject: [PATCH 495/922] chore: fix create_archive_and_notes to ignore release tool markers (#3355) The create_archive_and_notes.sh script is incorrectly detecting the version markers in the tool meant to rewrite them. To fix, ignore those files. --- .github/workflows/create_archive_and_notes.sh | 14 ++++++++++++-- 1 file changed, 12 insertions(+), 2 deletions(-) diff --git a/.github/workflows/create_archive_and_notes.sh b/.github/workflows/create_archive_and_notes.sh index a21585f866..b53c49aa09 100755 --- a/.github/workflows/create_archive_and_notes.sh +++ b/.github/workflows/create_archive_and_notes.sh @@ -13,12 +13,22 @@ # See the License for the specific language governing permissions and # limitations under the License. -set -o errexit -o nounset -o pipefail +set -o nounset +set -o pipefail +set -o errexit +set -x # Exclude dot directories, specifically, this file so that we don't # find the substring we're looking for in our own file. # Exclude CONTRIBUTING.md, RELEASING.md because they document how to use these strings. -if grep --exclude=CONTRIBUTING.md --exclude=RELEASING.md --exclude-dir=.* VERSION_NEXT_ -r; then +grep --exclude=CONTRIBUTING.md \ + --exclude=RELEASING.md \ + --exclude=release.py \ + --exclude=release_test.py \ + --exclude-dir=.* \ + VERSION_NEXT_ -r || grep_exit_code=$? + +if [[ $grep_exit_code -eq 0 ]]; then echo echo "Found VERSION_NEXT markers indicating version needs to be specified" exit 1 From f92ad7136ff411755ec8d8361af8263fd8efe6f2 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Fri, 17 Oct 2025 16:09:48 -0700 Subject: [PATCH 496/922] chore: make gazelle bcr tests compatible with bcr presubmit environment (#3365) BCR recently changed how it extracts archives in its presubmits: instead of extracting the whole archive, only the specified portion (`gazelle/` in this case) is extracted. This broke the gazelle tests because they reference files above the gazelle directory. To fix, move the module it runs as a test under the gazelle directory. Because the test module also refers to rules_python, which is above the gazelle directory, the bcr presubmit has disable that override using `--override_module`. This means, going forward, the gazelle module, when bcr tests it, will use the version in the MODULE file (rather than the vendored copy). Fixes https://github.com/bazel-contrib/rules_python/issues/3364 --- .bazelci/presubmit.yml | 24 +++++++++---------- .bazelignore | 4 ++++ .bazelrc | 3 +++ .bcr/gazelle/presubmit.yml | 4 +++- .../other_module/MODULE.bazel | 5 ---- gazelle/.bazelignore | 8 +++++++ gazelle/.bazelrc | 3 +++ .../bzlmod_build_file_generation/.bazelignore | 0 .../bzlmod_build_file_generation/.bazelrc | 0 .../bzlmod_build_file_generation/.gitignore | 0 .../bzlmod_build_file_generation/BUILD.bazel | 0 .../bzlmod_build_file_generation/MODULE.bazel | 12 ++++++---- .../bzlmod_build_file_generation/README.md | 0 .../bzlmod_build_file_generation/WORKSPACE | 0 .../bzlmod_build_file_generation/__main__.py | 0 .../bzlmod_build_file_generation/__test__.py | 0 .../gazelle_python.yaml | 0 .../gazelle_python_with_types.yaml | 0 .../bzlmod_build_file_generation/lib.py | 0 .../other_module/MODULE.bazel | 7 ++++++ .../other_module/WORKSPACE | 0 .../other_module/other_module/pkg/BUILD.bazel | 0 .../other_module/pkg/data/data.txt | 0 .../other_module/other_module/pkg/lib.py | 0 .../requirements.in | 0 .../requirements_lock.txt | 0 .../requirements_windows.txt | 0 .../runfiles/BUILD.bazel | 0 .../runfiles/data/data.txt | 0 .../runfiles/runfiles_test.py | 0 30 files changed, 47 insertions(+), 23 deletions(-) delete mode 100644 examples/bzlmod_build_file_generation/other_module/MODULE.bazel create mode 100644 gazelle/.bazelignore rename {examples => gazelle/examples}/bzlmod_build_file_generation/.bazelignore (100%) rename {examples => gazelle/examples}/bzlmod_build_file_generation/.bazelrc (100%) rename {examples => gazelle/examples}/bzlmod_build_file_generation/.gitignore (100%) rename {examples => gazelle/examples}/bzlmod_build_file_generation/BUILD.bazel (100%) rename {examples => gazelle/examples}/bzlmod_build_file_generation/MODULE.bazel (91%) rename {examples => gazelle/examples}/bzlmod_build_file_generation/README.md (100%) rename {examples => gazelle/examples}/bzlmod_build_file_generation/WORKSPACE (100%) rename {examples => gazelle/examples}/bzlmod_build_file_generation/__main__.py (100%) rename {examples => gazelle/examples}/bzlmod_build_file_generation/__test__.py (100%) rename {examples => gazelle/examples}/bzlmod_build_file_generation/gazelle_python.yaml (100%) rename {examples => gazelle/examples}/bzlmod_build_file_generation/gazelle_python_with_types.yaml (100%) rename {examples => gazelle/examples}/bzlmod_build_file_generation/lib.py (100%) create mode 100644 gazelle/examples/bzlmod_build_file_generation/other_module/MODULE.bazel rename {examples => gazelle/examples}/bzlmod_build_file_generation/other_module/WORKSPACE (100%) rename {examples => gazelle/examples}/bzlmod_build_file_generation/other_module/other_module/pkg/BUILD.bazel (100%) rename {examples => gazelle/examples}/bzlmod_build_file_generation/other_module/other_module/pkg/data/data.txt (100%) rename {examples => gazelle/examples}/bzlmod_build_file_generation/other_module/other_module/pkg/lib.py (100%) rename {examples => gazelle/examples}/bzlmod_build_file_generation/requirements.in (100%) rename {examples => gazelle/examples}/bzlmod_build_file_generation/requirements_lock.txt (100%) rename {examples => gazelle/examples}/bzlmod_build_file_generation/requirements_windows.txt (100%) rename {examples => gazelle/examples}/bzlmod_build_file_generation/runfiles/BUILD.bazel (100%) rename {examples => gazelle/examples}/bzlmod_build_file_generation/runfiles/data/data.txt (100%) rename {examples => gazelle/examples}/bzlmod_build_file_generation/runfiles/runfiles_test.py (100%) diff --git a/.bazelci/presubmit.yml b/.bazelci/presubmit.yml index 119ad498b0..6ed93b083d 100644 --- a/.bazelci/presubmit.yml +++ b/.bazelci/presubmit.yml @@ -329,20 +329,20 @@ tasks: <<: *minimum_supported_version <<: *reusable_build_test_all <<: *coverage_targets_example_bzlmod_build_file_generation - name: "examples/bzlmod_build_file_generation: Ubuntu, minimum Bazel" - working_directory: examples/bzlmod_build_file_generation + name: "gazelle/examples/bzlmod_build_file_generation: Ubuntu, minimum Bazel" + working_directory: gazelle/examples/bzlmod_build_file_generation platform: ubuntu2204 bazel: 7.x integration_test_bzlmod_generation_build_files_ubuntu: <<: *reusable_build_test_all <<: *coverage_targets_example_bzlmod_build_file_generation - name: "examples/bzlmod_build_file_generation: Ubuntu" - working_directory: examples/bzlmod_build_file_generation + name: "gazelle/examples/bzlmod_build_file_generation: Ubuntu" + working_directory: gazelle/examples/bzlmod_build_file_generation platform: ubuntu2204 integration_test_bzlmod_generation_build_files_ubuntu_run: <<: *reusable_build_test_all - name: "examples/bzlmod_build_file_generation: Ubuntu, Gazelle and pip" - working_directory: examples/bzlmod_build_file_generation + name: "gazelle/examples/bzlmod_build_file_generation: Ubuntu, Gazelle and pip" + working_directory: gazelle/examples/bzlmod_build_file_generation platform: ubuntu2204 shell_commands: - "bazel run //:gazelle_python_manifest.update" @@ -350,20 +350,20 @@ tasks: integration_test_bzlmod_build_file_generation_debian: <<: *reusable_build_test_all <<: *coverage_targets_example_bzlmod_build_file_generation - name: "examples/bzlmod_build_file_generation: Debian" - working_directory: examples/bzlmod_build_file_generation + name: "gazelle/examples/bzlmod_build_file_generation: Debian" + working_directory: gazelle/examples/bzlmod_build_file_generation platform: debian11 integration_test_bzlmod_build_file_generation_macos: <<: *reusable_build_test_all <<: *coverage_targets_example_bzlmod_build_file_generation - name: "examples/bzlmod_build_file_generation: MacOS" - working_directory: examples/bzlmod_build_file_generation + name: "gazelle/examples/bzlmod_build_file_generation: MacOS" + working_directory: gazelle/examples/bzlmod_build_file_generation platform: macos integration_test_bzlmod_build_file_generation_windows: <<: *reusable_build_test_all # coverage is not supported on Windows - name: "examples/bzlmod_build_file_generation: Windows" - working_directory: examples/bzlmod_build_file_generation + name: "gazelle/examples/bzlmod_build_file_generation: Windows" + working_directory: gazelle/examples/bzlmod_build_file_generation platform: windows integration_test_multi_python_versions_ubuntu_workspace: diff --git a/.bazelignore b/.bazelignore index fb999097f5..dd58b79e3c 100644 --- a/.bazelignore +++ b/.bazelignore @@ -26,6 +26,10 @@ examples/pip_parse_vendored/bazel-pip_parse_vendored examples/pip_repository_annotations/bazel-pip_repository_annotations examples/py_proto_library/bazel-py_proto_library gazelle/bazel-gazelle +gazelle/examples/bzlmod_build_file_generation/bazel-bin +gazelle/examples/bzlmod_build_file_generation/bazel-bzlmod_build_file_generation +gazelle/examples/bzlmod_build_file_generation/bazel-out +gazelle/examples/bzlmod_build_file_generation/bazel-testlog tests/integration/compile_pip_requirements/bazel-compile_pip_requirements tests/integration/ignore_root_user_error/bazel-ignore_root_user_error tests/integration/local_toolchains/bazel-local_toolchains diff --git a/.bazelrc b/.bazelrc index 801b963ad5..b5c9c7c1e2 100644 --- a/.bazelrc +++ b/.bazelrc @@ -9,6 +9,9 @@ query --deleted_packages=examples/build_file_generation,examples/build_file_gene test --test_output=errors +common --deleted_packages=gazelle/examples/bzlmod_build_file_generation +common --deleted_packages=gazelle/examples/bzlmod_build_file_generation/runfiles + # Do NOT implicitly create empty __init__.py files in the runfiles tree. # By default, these are created in every directory containing Python source code # or shared libraries, and every parent directory of those directories, diff --git a/.bcr/gazelle/presubmit.yml b/.bcr/gazelle/presubmit.yml index bceed4f9e1..ff1c9e7d58 100644 --- a/.bcr/gazelle/presubmit.yml +++ b/.bcr/gazelle/presubmit.yml @@ -13,7 +13,7 @@ # limitations under the License. bcr_test_module: - module_path: "../examples/bzlmod_build_file_generation" + module_path: "examples/bzlmod_build_file_generation" matrix: platform: ["debian11", "macos", "ubuntu2004", "windows"] # last_rc is to get latest 8.x release. Replace with 8.x when available. @@ -23,6 +23,8 @@ bcr_test_module: name: "Run test module" platform: ${{ platform }} bazel: ${{ bazel }} + shell_commands: + - "echo 'common --override_module=rules_python=' >> .bazelrc" build_targets: - "//..." - ":modules_map" diff --git a/examples/bzlmod_build_file_generation/other_module/MODULE.bazel b/examples/bzlmod_build_file_generation/other_module/MODULE.bazel deleted file mode 100644 index 992e120760..0000000000 --- a/examples/bzlmod_build_file_generation/other_module/MODULE.bazel +++ /dev/null @@ -1,5 +0,0 @@ -module( - name = "other_module", -) - -bazel_dep(name = "rules_python", version = "") diff --git a/gazelle/.bazelignore b/gazelle/.bazelignore new file mode 100644 index 0000000000..5930a06190 --- /dev/null +++ b/gazelle/.bazelignore @@ -0,0 +1,8 @@ +bazel-bin +bazel-gazelle +bazel-out +bazel-testlogs +examples/bzlmod_build_file_generation/bazel-bin +examples/bzlmod_build_file_generation/bazel-bzlmod_build_file_generation +examples/bzlmod_build_file_generation/bazel-out +examples/bzlmod_build_file_generation/bazel-testlog diff --git a/gazelle/.bazelrc b/gazelle/.bazelrc index 791b93912a..9a38133e9d 100644 --- a/gazelle/.bazelrc +++ b/gazelle/.bazelrc @@ -1,3 +1,6 @@ +common --deleted_packages=examples/bzlmod_build_file_generation +common --deleted_packages=examples/bzlmod_build_file_generation/runfiles + test --test_output=errors # Do NOT implicitly create empty __init__.py files in the runfiles tree. diff --git a/examples/bzlmod_build_file_generation/.bazelignore b/gazelle/examples/bzlmod_build_file_generation/.bazelignore similarity index 100% rename from examples/bzlmod_build_file_generation/.bazelignore rename to gazelle/examples/bzlmod_build_file_generation/.bazelignore diff --git a/examples/bzlmod_build_file_generation/.bazelrc b/gazelle/examples/bzlmod_build_file_generation/.bazelrc similarity index 100% rename from examples/bzlmod_build_file_generation/.bazelrc rename to gazelle/examples/bzlmod_build_file_generation/.bazelrc diff --git a/examples/bzlmod_build_file_generation/.gitignore b/gazelle/examples/bzlmod_build_file_generation/.gitignore similarity index 100% rename from examples/bzlmod_build_file_generation/.gitignore rename to gazelle/examples/bzlmod_build_file_generation/.gitignore diff --git a/examples/bzlmod_build_file_generation/BUILD.bazel b/gazelle/examples/bzlmod_build_file_generation/BUILD.bazel similarity index 100% rename from examples/bzlmod_build_file_generation/BUILD.bazel rename to gazelle/examples/bzlmod_build_file_generation/BUILD.bazel diff --git a/examples/bzlmod_build_file_generation/MODULE.bazel b/gazelle/examples/bzlmod_build_file_generation/MODULE.bazel similarity index 91% rename from examples/bzlmod_build_file_generation/MODULE.bazel rename to gazelle/examples/bzlmod_build_file_generation/MODULE.bazel index 3436fbf0af..5ace7f3d3a 100644 --- a/examples/bzlmod_build_file_generation/MODULE.bazel +++ b/gazelle/examples/bzlmod_build_file_generation/MODULE.bazel @@ -13,26 +13,28 @@ module( # For typical setups you set the version. # See the releases page for available versions. # https://github.com/bazel-contrib/rules_python/releases -bazel_dep(name = "rules_python", version = "0.0.0") +bazel_dep(name = "rules_python", version = "1.0.0") +# NOTE: This override is removed for BCR presubmits and the version +# specified by bazel_dep() is used instead. # The following loads rules_python from the file system. # For usual setups you should remove this local_path_override block. local_path_override( module_name = "rules_python", - path = "../..", + path = "../../..", ) # The following stanza defines the dependency rules_python_gazelle_plugin. # For typical setups you set the version. # See the releases page for available versions. # https://github.com/bazel-contrib/rules_python/releases -bazel_dep(name = "rules_python_gazelle_plugin", version = "0.0.0") +bazel_dep(name = "rules_python_gazelle_plugin", version = "1.5.0") # The following starlark loads the gazelle plugin from the file system. # For usual setups you should remove this local_path_override block. local_path_override( module_name = "rules_python_gazelle_plugin", - path = "../../gazelle", + path = "../..", ) # The following stanza defines the dependency for gazelle @@ -84,7 +86,7 @@ use_repo(pip, "pip") # This project includes a different module that is on the local file system. # Add the module to this parent project. -bazel_dep(name = "other_module", version = "", repo_name = "our_other_module") +bazel_dep(name = "other_module", version = "0.0.0", repo_name = "our_other_module") local_path_override( module_name = "other_module", path = "other_module", diff --git a/examples/bzlmod_build_file_generation/README.md b/gazelle/examples/bzlmod_build_file_generation/README.md similarity index 100% rename from examples/bzlmod_build_file_generation/README.md rename to gazelle/examples/bzlmod_build_file_generation/README.md diff --git a/examples/bzlmod_build_file_generation/WORKSPACE b/gazelle/examples/bzlmod_build_file_generation/WORKSPACE similarity index 100% rename from examples/bzlmod_build_file_generation/WORKSPACE rename to gazelle/examples/bzlmod_build_file_generation/WORKSPACE diff --git a/examples/bzlmod_build_file_generation/__main__.py b/gazelle/examples/bzlmod_build_file_generation/__main__.py similarity index 100% rename from examples/bzlmod_build_file_generation/__main__.py rename to gazelle/examples/bzlmod_build_file_generation/__main__.py diff --git a/examples/bzlmod_build_file_generation/__test__.py b/gazelle/examples/bzlmod_build_file_generation/__test__.py similarity index 100% rename from examples/bzlmod_build_file_generation/__test__.py rename to gazelle/examples/bzlmod_build_file_generation/__test__.py diff --git a/examples/bzlmod_build_file_generation/gazelle_python.yaml b/gazelle/examples/bzlmod_build_file_generation/gazelle_python.yaml similarity index 100% rename from examples/bzlmod_build_file_generation/gazelle_python.yaml rename to gazelle/examples/bzlmod_build_file_generation/gazelle_python.yaml diff --git a/examples/bzlmod_build_file_generation/gazelle_python_with_types.yaml b/gazelle/examples/bzlmod_build_file_generation/gazelle_python_with_types.yaml similarity index 100% rename from examples/bzlmod_build_file_generation/gazelle_python_with_types.yaml rename to gazelle/examples/bzlmod_build_file_generation/gazelle_python_with_types.yaml diff --git a/examples/bzlmod_build_file_generation/lib.py b/gazelle/examples/bzlmod_build_file_generation/lib.py similarity index 100% rename from examples/bzlmod_build_file_generation/lib.py rename to gazelle/examples/bzlmod_build_file_generation/lib.py diff --git a/gazelle/examples/bzlmod_build_file_generation/other_module/MODULE.bazel b/gazelle/examples/bzlmod_build_file_generation/other_module/MODULE.bazel new file mode 100644 index 0000000000..3deeeb6ccc --- /dev/null +++ b/gazelle/examples/bzlmod_build_file_generation/other_module/MODULE.bazel @@ -0,0 +1,7 @@ +module( + name = "other_module", +) + +# Version doesn't matter because the root module overrides it, +# but Bazel requires it exist in the registry. +bazel_dep(name = "rules_python", version = "1.0.0") diff --git a/examples/bzlmod_build_file_generation/other_module/WORKSPACE b/gazelle/examples/bzlmod_build_file_generation/other_module/WORKSPACE similarity index 100% rename from examples/bzlmod_build_file_generation/other_module/WORKSPACE rename to gazelle/examples/bzlmod_build_file_generation/other_module/WORKSPACE diff --git a/examples/bzlmod_build_file_generation/other_module/other_module/pkg/BUILD.bazel b/gazelle/examples/bzlmod_build_file_generation/other_module/other_module/pkg/BUILD.bazel similarity index 100% rename from examples/bzlmod_build_file_generation/other_module/other_module/pkg/BUILD.bazel rename to gazelle/examples/bzlmod_build_file_generation/other_module/other_module/pkg/BUILD.bazel diff --git a/examples/bzlmod_build_file_generation/other_module/other_module/pkg/data/data.txt b/gazelle/examples/bzlmod_build_file_generation/other_module/other_module/pkg/data/data.txt similarity index 100% rename from examples/bzlmod_build_file_generation/other_module/other_module/pkg/data/data.txt rename to gazelle/examples/bzlmod_build_file_generation/other_module/other_module/pkg/data/data.txt diff --git a/examples/bzlmod_build_file_generation/other_module/other_module/pkg/lib.py b/gazelle/examples/bzlmod_build_file_generation/other_module/other_module/pkg/lib.py similarity index 100% rename from examples/bzlmod_build_file_generation/other_module/other_module/pkg/lib.py rename to gazelle/examples/bzlmod_build_file_generation/other_module/other_module/pkg/lib.py diff --git a/examples/bzlmod_build_file_generation/requirements.in b/gazelle/examples/bzlmod_build_file_generation/requirements.in similarity index 100% rename from examples/bzlmod_build_file_generation/requirements.in rename to gazelle/examples/bzlmod_build_file_generation/requirements.in diff --git a/examples/bzlmod_build_file_generation/requirements_lock.txt b/gazelle/examples/bzlmod_build_file_generation/requirements_lock.txt similarity index 100% rename from examples/bzlmod_build_file_generation/requirements_lock.txt rename to gazelle/examples/bzlmod_build_file_generation/requirements_lock.txt diff --git a/examples/bzlmod_build_file_generation/requirements_windows.txt b/gazelle/examples/bzlmod_build_file_generation/requirements_windows.txt similarity index 100% rename from examples/bzlmod_build_file_generation/requirements_windows.txt rename to gazelle/examples/bzlmod_build_file_generation/requirements_windows.txt diff --git a/examples/bzlmod_build_file_generation/runfiles/BUILD.bazel b/gazelle/examples/bzlmod_build_file_generation/runfiles/BUILD.bazel similarity index 100% rename from examples/bzlmod_build_file_generation/runfiles/BUILD.bazel rename to gazelle/examples/bzlmod_build_file_generation/runfiles/BUILD.bazel diff --git a/examples/bzlmod_build_file_generation/runfiles/data/data.txt b/gazelle/examples/bzlmod_build_file_generation/runfiles/data/data.txt similarity index 100% rename from examples/bzlmod_build_file_generation/runfiles/data/data.txt rename to gazelle/examples/bzlmod_build_file_generation/runfiles/data/data.txt diff --git a/examples/bzlmod_build_file_generation/runfiles/runfiles_test.py b/gazelle/examples/bzlmod_build_file_generation/runfiles/runfiles_test.py similarity index 100% rename from examples/bzlmod_build_file_generation/runfiles/runfiles_test.py rename to gazelle/examples/bzlmod_build_file_generation/runfiles/runfiles_test.py From 6f80d342b1a0711dbb36eeff05b900733be7bf39 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Sun, 19 Oct 2025 16:07:47 -0700 Subject: [PATCH 497/922] chore: fix release workflow (#3366) My original change to use the bazel-contrib BCR publishing workflow was horribly broken. This fixes a variety of issues. * Fix the workflow call secrets variable name * Allow getting the BCR publishing token from the workflow arg (the release workflow calls it) * Fix the overall syntax of the workflows. It was just entirely invalid in several regards. * Fix the yaml -> yml file name reference. NGL, it took me longer than I'd like to admit to see that, eh. Along the way, some cleanups and improvements * Allow specifying a particular tag to release, while using workflow state from a different commit. This allows us to make fixes on main, and then use it to manually re-trigger a tag to be released. * Add descriptions for workflow inputs * Allow bcr token to be passed to release. This will allow e.g. automatically scheduled releases. * Quote shell variables because its good practice. --- .github/workflows/create_archive_and_notes.sh | 18 ++++-- .github/workflows/publish.yml | 13 ++-- .github/workflows/release.yml | 59 +++++++++++-------- 3 files changed, 53 insertions(+), 37 deletions(-) diff --git a/.github/workflows/create_archive_and_notes.sh b/.github/workflows/create_archive_and_notes.sh index b53c49aa09..a3cf8280a2 100755 --- a/.github/workflows/create_archive_and_notes.sh +++ b/.github/workflows/create_archive_and_notes.sh @@ -18,6 +18,17 @@ set -o pipefail set -o errexit set -x + +TAG=$1 +if [ -z "$TAG" ]; then + echo "ERROR: TAG env var must be set" + exit 1 +fi +# If the workflow checks out one commit, but is releasing another +git fetch origin tag "$TAG" +# Update our local state so the grep command below searches what we expect +git checkout "$TAG" + # Exclude dot directories, specifically, this file so that we don't # find the substring we're looking for in our own file. # Exclude CONTRIBUTING.md, RELEASING.md because they document how to use these strings. @@ -34,14 +45,11 @@ if [[ $grep_exit_code -eq 0 ]]; then exit 1 fi -# Set by GH actions, see -# https://docs.github.com/en/actions/learn-github-actions/environment-variables#default-environment-variables -TAG=${GITHUB_REF_NAME} # A prefix is added to better match the GitHub generated archives. PREFIX="rules_python-${TAG}" ARCHIVE="rules_python-$TAG.tar.gz" -git archive --format=tar --prefix=${PREFIX}/ ${TAG} | gzip > $ARCHIVE -SHA=$(shasum -a 256 $ARCHIVE | awk '{print $1}') +git archive --format=tar "--prefix=${PREFIX}/" "$TAG" | gzip > "$ARCHIVE" +SHA=$(shasum -a 256 "$ARCHIVE" | awk '{print $1}') cat > release_notes.txt << EOF diff --git a/.github/workflows/publish.yml b/.github/workflows/publish.yml index f03e02168f..9ad5308968 100644 --- a/.github/workflows/publish.yml +++ b/.github/workflows/publish.yml @@ -1,30 +1,28 @@ -# See https://github.com/bazel-contrib/publish-to-bcr +# Publish new releases to Bazel Central Registry. name: Publish to BCR - on: # Run the publish workflow after a successful release - # Can be triggered from the release.yaml workflow + # Will be triggered from the release.yaml workflow workflow_call: inputs: tag_name: required: true type: string secrets: - BCR_PUBLISH_TOKEN: + publish_token: required: true # In case of problems, let release engineers retry by manually dispatching # the workflow from the GitHub UI workflow_dispatch: inputs: tag_name: + description: git tag being released required: true type: string - jobs: publish: uses: bazel-contrib/publish-to-bcr/.github/workflows/publish.yaml@v1.0.0 with: - draft: false tag_name: ${{ inputs.tag_name }} # GitHub repository which is a fork of the upstream where the Pull Request will be opened. registry_fork: bazel-contrib/bazel-central-registry @@ -32,4 +30,5 @@ jobs: permissions: contents: write secrets: - publish_token: ${{ secrets.BCR_PUBLISH_TOKEN }} + # Necessary to push to the BCR fork, and to open a pull request against a registry + publish_token: ${{ secrets.publish_token || secrets.BCR_PUBLISH_TOKEN }} diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index 0ed4992ccd..0d24d7913b 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -21,51 +21,60 @@ on: - "*.*.*" workflow_dispatch: inputs: + tag_name: + description: "release tag: tag that will be released" + required: true + type: string publish_to_pypi: description: 'Publish to PyPI' required: true type: boolean default: true + secrets: + publish_token: + required: false jobs: - build: + release: + name: Release runs-on: ubuntu-latest steps: - name: Checkout uses: actions/checkout@v5 + with: + ref: ${{ github.ref_name }} - name: Create release archive and notes - run: .github/workflows/create_archive_and_notes.sh - - release: - name: Release - uses: softprops/action-gh-release@v2 - with: - # Use GH feature to populate the changelog automatically - generate_release_notes: true - body_path: release_notes.txt - prerelease: ${{ contains(github.ref, '-rc') }} - fail_on_unmatched_files: true - files: rules_python-*.tar.gz + run: .github/workflows/create_archive_and_notes.sh ${{ inputs.tag_name || github.ref_name }} + - name: Release + uses: softprops/action-gh-release@v2 + with: + # Use GH feature to populate the changelog automatically + generate_release_notes: true + body_path: release_notes.txt + prerelease: ${{ contains( (inputs.tag_name || github.ref), '-rc') }} + fail_on_unmatched_files: true + files: rules_python-*.tar.gz + tag_name: ${{ inputs.tag_name || github.ref_name }} publish_bcr: - name: Publish to BCR needs: release - uses: .github/workflows/publish.yaml + uses: ./.github/workflows/publish.yml with: - tag_name: ${{ github.ref_name }} + tag_name: ${{ inputs.tag_name || github.ref_name }} secrets: - BCR_PUBLISH_TOKEN: ${{ secrets.BCR_PUBLISH_TOKEN }} + publish_token: ${{ secrets.publish_token || secrets.BCR_PUBLISH_TOKEN }} publish_pypi: # We just want publish_pypi last, since once uploaded, it can't be changed. name: Publish runfiles to PyPI needs: publish_bcr runs-on: ubuntu-latest - if: github.event_name == 'push' || github.event.inputs.publish_to_pypi - env: - # This special value tells pypi that the user identity is supplied within the token - TWINE_USERNAME: __token__ - # Note, the PYPI_API_TOKEN is for the rules-python pypi user, added by @rickylev on - # https://github.com/bazel-contrib/rules_python/settings/secrets/actions - TWINE_PASSWORD: ${{ secrets.PYPI_API_TOKEN }} - run: bazel run --stamp --embed_label=${{ github.ref_name }} //python/runfiles:wheel.publish + steps: + - if: github.event_name == 'push' || github.event.inputs.publish_to_pypi + env: + # This special value tells pypi that the user identity is supplied within the token + TWINE_USERNAME: __token__ + # Note, the PYPI_API_TOKEN is for the rules-python pypi user, added by @rickylev on + # https://github.com/bazel-contrib/rules_python/settings/secrets/actions + TWINE_PASSWORD: ${{ secrets.PYPI_API_TOKEN }} + run: bazel run --stamp --embed_label=${{ inputs.tag_name || github.ref_name }} //python/runfiles:wheel.publish From 891e647ef7642f2cc69a573e50c268a36122fb22 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Mon, 20 Oct 2025 06:21:46 -0700 Subject: [PATCH 498/922] chore: remove bcr app settings (#3370) The fixedRelease setting is only relevant to the BCR App, which we don't use anymore. --- .bcr/config.yml | 3 --- 1 file changed, 3 deletions(-) diff --git a/.bcr/config.yml b/.bcr/config.yml index 7672aa554d..f103e35bf7 100644 --- a/.bcr/config.yml +++ b/.bcr/config.yml @@ -12,7 +12,4 @@ # See the License for the specific language governing permissions and # limitations under the License. -fixedReleaser: - login: f0rmiga - email: 3149049+f0rmiga@users.noreply.github.com moduleRoots: [".", "gazelle"] From cf594f780c91f13d48e77faad34df48ac57398da Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Mon, 20 Oct 2025 06:29:19 -0700 Subject: [PATCH 499/922] chore: make pypi release workflow perform checkout (#3371) The pypi workflow step is now a separate job, so it doesn't start with the code checkout done already. Run the checkout action so it is available so the bazel upload command can run. --- .github/workflows/release.yml | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index 0d24d7913b..94c4d82561 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -70,6 +70,10 @@ jobs: needs: publish_bcr runs-on: ubuntu-latest steps: + - name: Checkout + uses: actions/checkout@v5 + with: + ref: ${{ github.tag_name || github.ref_name }} - if: github.event_name == 'push' || github.event.inputs.publish_to_pypi env: # This special value tells pypi that the user identity is supplied within the token From 39bd4d8fc0c793aea1ab6b8108ce6eca2bfa1140 Mon Sep 17 00:00:00 2001 From: yushan26 <107004874+yushan26@users.noreply.github.com> Date: Tue, 21 Oct 2025 12:12:29 -0700 Subject: [PATCH 500/922] fix(gazelle) Delete python targets with invalid srcs (#3046) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit When running Gazelle, it generated the following target: ``` py_binary( name = "remove_py_binary", srcs = ["__main__.py"], main = "__main__.py", visibility = ["//visibility:public"], ) ``` After `__main__.py` was deleted and the change committed, re-running Gazelle did not remove the file from the srcs list. This change introduces logic to check whether all entries in a Python target’s srcs attribute correspond to valid files. If none of them exist, the target is added to result.Empty to signal that it should be cleaned up. This cleanup behavior applies to when python_generation mode is package or file, as all `srcs` are expected to reside directly within the current directory. --------- Co-authored-by: yushan Co-authored-by: Douglas Thor --- CHANGELOG.md | 3 +- gazelle/python/generate.go | 45 ++++++++++++++++++- gazelle/python/kinds.go | 1 + .../testdata/remove_invalid_binary/BUILD.in | 18 ++++++++ .../testdata/remove_invalid_binary/BUILD.out | 6 +++ .../testdata/remove_invalid_binary/README.md | 3 ++ .../testdata/remove_invalid_binary/WORKSPACE | 0 .../keep_binary/BUILD.in | 13 ++++++ .../keep_binary/BUILD.out | 13 ++++++ .../remove_invalid_binary/keep_binary/foo.py | 2 + .../testdata/remove_invalid_binary/test.yaml | 0 .../testdata/respect_kind_mapping/BUILD.in | 6 +++ .../testdata/respect_kind_mapping/BUILD.out | 1 + 13 files changed, 109 insertions(+), 2 deletions(-) create mode 100644 gazelle/python/testdata/remove_invalid_binary/BUILD.in create mode 100644 gazelle/python/testdata/remove_invalid_binary/BUILD.out create mode 100644 gazelle/python/testdata/remove_invalid_binary/README.md create mode 100644 gazelle/python/testdata/remove_invalid_binary/WORKSPACE create mode 100644 gazelle/python/testdata/remove_invalid_binary/keep_binary/BUILD.in create mode 100644 gazelle/python/testdata/remove_invalid_binary/keep_binary/BUILD.out create mode 100644 gazelle/python/testdata/remove_invalid_binary/keep_binary/foo.py create mode 100644 gazelle/python/testdata/remove_invalid_binary/test.yaml diff --git a/CHANGELOG.md b/CHANGELOG.md index 23e05f02a5..4576926406 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -38,7 +38,8 @@ BEGIN_UNRELEASED_TEMPLATE {#v0-0-0-fixed} ### Fixed -* Nothing fixed. +* (gazelle) Remove {obj}`py_binary` targets with invalid `srcs`. This includes files + that are not generated or regular files. {#v0-0-0-added} ### Added diff --git a/gazelle/python/generate.go b/gazelle/python/generate.go index a180ec527d..cbceea4693 100644 --- a/gazelle/python/generate.go +++ b/gazelle/python/generate.go @@ -231,9 +231,14 @@ func (py *Python) GenerateRules(args language.GenerateArgs) language.GenerateRes } collisionErrors := singlylinkedlist.New() + // Create a validFilesMap of mainModules to validate if python macros have valid srcs. + validFilesMap := make(map[string]struct{}) appendPyLibrary := func(srcs *treeset.Set, pyLibraryTargetName string) { allDeps, mainModules, annotations, err := parser.parse(srcs) + for name := range mainModules { + validFilesMap[name] = struct{}{} + } if err != nil { log.Fatalf("ERROR: %v\n", err) } @@ -363,6 +368,7 @@ func (py *Python) GenerateRules(args language.GenerateArgs) language.GenerateRes setAnnotations(*annotations). generateImportsAttribute() + pyBinary := pyBinaryTarget.build() result.Gen = append(result.Gen, pyBinary) @@ -490,7 +496,8 @@ func (py *Python) GenerateRules(args language.GenerateArgs) language.GenerateRes result.Gen = append(result.Gen, pyTest) result.Imports = append(result.Imports, pyTest.PrivateAttr(config.GazelleImportsKey)) } - + emptyRules := py.getRulesWithInvalidSrcs(args, validFilesMap) + result.Empty = append(result.Empty, emptyRules...) if !collisionErrors.Empty() { it := collisionErrors.Iterator() for it.Next() { @@ -502,6 +509,42 @@ func (py *Python) GenerateRules(args language.GenerateArgs) language.GenerateRes return result } +// getRulesWithInvalidSrcs checks existing Python rules in the BUILD file and return the rules with invalid source files. +// Invalid source files are files that do not exist or not a target. +func (py *Python) getRulesWithInvalidSrcs(args language.GenerateArgs, validFilesMap map[string]struct{}) (invalidRules []*rule.Rule) { + if args.File == nil { + return + } + for _, file := range args.GenFiles { + validFilesMap[file] = struct{}{} + } + + isTarget := func(src string) bool { + return strings.HasPrefix(src, "@") || strings.HasPrefix(src, "//") || strings.HasPrefix(src, ":") + } + for _, existingRule := range args.File.Rules { + actualPyBinaryKind := GetActualKindName(pyBinaryKind, args) + if existingRule.Kind() != actualPyBinaryKind { + continue + } + var hasValidSrcs bool + for _, src := range existingRule.AttrStrings("srcs") { + if isTarget(src) { + hasValidSrcs = true + break + } + if _, ok := validFilesMap[src]; ok { + hasValidSrcs = true + break + } + } + if !hasValidSrcs { + invalidRules = append(invalidRules, newTargetBuilder(pyBinaryKind, existingRule.Name(), "", "", nil, false).build()) + } + } + return invalidRules +} + // isBazelPackage determines if the directory is a Bazel package by probing for // the existence of a known BUILD file name. func isBazelPackage(dir string) bool { diff --git a/gazelle/python/kinds.go b/gazelle/python/kinds.go index a4ce572aaa..4fe8090445 100644 --- a/gazelle/python/kinds.go +++ b/gazelle/python/kinds.go @@ -46,6 +46,7 @@ var pyKinds = map[string]rule.KindInfo{ SubstituteAttrs: map[string]bool{}, MergeableAttrs: map[string]bool{ "srcs": true, + "imports": true, }, ResolveAttrs: map[string]bool{ "deps": true, diff --git a/gazelle/python/testdata/remove_invalid_binary/BUILD.in b/gazelle/python/testdata/remove_invalid_binary/BUILD.in new file mode 100644 index 0000000000..87d357139b --- /dev/null +++ b/gazelle/python/testdata/remove_invalid_binary/BUILD.in @@ -0,0 +1,18 @@ +load("@rules_python//python:defs.bzl", "py_binary", "py_library") + +py_library( + name = "keep_library", + deps = ["//keep_binary:foo"], +) +py_binary( + name = "remove_invalid_binary", + srcs = ["__main__.py"], + data = ["testdata/test.txt"], + visibility = ["//:__subpackages__"], +) + +py_binary( + name = "another_removed_binary", + srcs = ["foo.py"], # eg a now-deleted file that used to have `if __name__` block + imports = ["."], +) diff --git a/gazelle/python/testdata/remove_invalid_binary/BUILD.out b/gazelle/python/testdata/remove_invalid_binary/BUILD.out new file mode 100644 index 0000000000..069188f5ca --- /dev/null +++ b/gazelle/python/testdata/remove_invalid_binary/BUILD.out @@ -0,0 +1,6 @@ +load("@rules_python//python:defs.bzl", "py_library") + +py_library( + name = "keep_library", + deps = ["//keep_binary:foo"], +) diff --git a/gazelle/python/testdata/remove_invalid_binary/README.md b/gazelle/python/testdata/remove_invalid_binary/README.md new file mode 100644 index 0000000000..be8a894420 --- /dev/null +++ b/gazelle/python/testdata/remove_invalid_binary/README.md @@ -0,0 +1,3 @@ +# Remove invalid binary + +This test case asserts that `py_binary` should be deleted if invalid (no source files). diff --git a/gazelle/python/testdata/remove_invalid_binary/WORKSPACE b/gazelle/python/testdata/remove_invalid_binary/WORKSPACE new file mode 100644 index 0000000000..e69de29bb2 diff --git a/gazelle/python/testdata/remove_invalid_binary/keep_binary/BUILD.in b/gazelle/python/testdata/remove_invalid_binary/keep_binary/BUILD.in new file mode 100644 index 0000000000..0036c6797a --- /dev/null +++ b/gazelle/python/testdata/remove_invalid_binary/keep_binary/BUILD.in @@ -0,0 +1,13 @@ +load("@rules_python//python:defs.bzl", "py_binary", "py_library") + +py_binary( + name = "foo", + srcs = ["foo.py"], + visibility = ["//:__subpackages__"], +) + +py_library( + name = "keep_binary", + srcs = ["foo.py"], + visibility = ["//:__subpackages__"], +) diff --git a/gazelle/python/testdata/remove_invalid_binary/keep_binary/BUILD.out b/gazelle/python/testdata/remove_invalid_binary/keep_binary/BUILD.out new file mode 100644 index 0000000000..0036c6797a --- /dev/null +++ b/gazelle/python/testdata/remove_invalid_binary/keep_binary/BUILD.out @@ -0,0 +1,13 @@ +load("@rules_python//python:defs.bzl", "py_binary", "py_library") + +py_binary( + name = "foo", + srcs = ["foo.py"], + visibility = ["//:__subpackages__"], +) + +py_library( + name = "keep_binary", + srcs = ["foo.py"], + visibility = ["//:__subpackages__"], +) diff --git a/gazelle/python/testdata/remove_invalid_binary/keep_binary/foo.py b/gazelle/python/testdata/remove_invalid_binary/keep_binary/foo.py new file mode 100644 index 0000000000..d3b51eea8e --- /dev/null +++ b/gazelle/python/testdata/remove_invalid_binary/keep_binary/foo.py @@ -0,0 +1,2 @@ +if __name__ == "__main__": + print("foo") diff --git a/gazelle/python/testdata/remove_invalid_binary/test.yaml b/gazelle/python/testdata/remove_invalid_binary/test.yaml new file mode 100644 index 0000000000..e69de29bb2 diff --git a/gazelle/python/testdata/respect_kind_mapping/BUILD.in b/gazelle/python/testdata/respect_kind_mapping/BUILD.in index 6a06737623..3c6ec07014 100644 --- a/gazelle/python/testdata/respect_kind_mapping/BUILD.in +++ b/gazelle/python/testdata/respect_kind_mapping/BUILD.in @@ -1,6 +1,7 @@ load("@rules_python//python:defs.bzl", "py_library") # gazelle:map_kind py_test my_test :mytest.bzl +# gazelle:map_kind py_binary my_bin :mytest.bzl py_library( name = "respect_kind_mapping", @@ -13,3 +14,8 @@ my_test( main = "__test__.py", deps = [":respect_kind_mapping"], ) + +my_bin( + name = "my_bin_gets_removed", + srcs = ["__main__.py"], +) diff --git a/gazelle/python/testdata/respect_kind_mapping/BUILD.out b/gazelle/python/testdata/respect_kind_mapping/BUILD.out index fa06e2af12..eb6894f1c8 100644 --- a/gazelle/python/testdata/respect_kind_mapping/BUILD.out +++ b/gazelle/python/testdata/respect_kind_mapping/BUILD.out @@ -2,6 +2,7 @@ load("@rules_python//python:defs.bzl", "py_library") load(":mytest.bzl", "my_test") # gazelle:map_kind py_test my_test :mytest.bzl +# gazelle:map_kind py_binary my_bin :mytest.bzl py_library( name = "respect_kind_mapping", From 3c3e0be43fce2cdf279fa42252fc958c05eb0779 Mon Sep 17 00:00:00 2001 From: Ignas Anikevicius <240938+aignas@users.noreply.github.com> Date: Fri, 24 Oct 2025 16:51:05 +0900 Subject: [PATCH 501/922] revert(pypi): revert the default for pipstar (#3373) A user mentioned that flipping the pipstar default caused a regression in their setup in [this comment]. Until we have a better understanding and a regression test for that, we should revert the flip. Adjusted the CHANGELOG notes for this. Related to #2949 [this comment]: https://github.com/bazel-contrib/rules_python/issues/2949#issuecomment-3417501543 --- CHANGELOG.md | 6 +++--- python/private/internal_config_repo.bzl | 2 +- 2 files changed, 4 insertions(+), 4 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 4576926406..6cb571681d 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -75,10 +75,10 @@ END_UNRELEASED_TEMPLATE * (toolchains) `py_runtime` and `PyRuntimeInfo` reject Python 2 settings. Setting `py_runtime.python_version = "PY2"` or non-None `PyRuntimeInfo.py2_runtime` is an error. -* (pypi) `pipstar` flag has been flipped to be enabled by default, to turn it - off use `RULES_PYTHON_ENABLE_PIPSTAR=0` environment variable. If you do, please +* (pypi) `pipstar` flag has been implemented for `WORKSPACE` and can be flipped to be enabled using `RULES_PYTHON_ENABLE_PIPSTAR=1` environment variable. If you do, please add a comment to - [#2949](https://github.com/bazel-contrib/rules_python/issues/2949). + [#2949](https://github.com/bazel-contrib/rules_python/issues/2949) if you run into any + problems. With this release we are deprecating {obj}`pip.parse.experimental_target_platforms` and {obj}`pip_repository.experimental_target_platforms`. For users using `WORKSPACE` and vendoring the `requirements.bzl` file, please re-vendor so that downstream is unaffected diff --git a/python/private/internal_config_repo.bzl b/python/private/internal_config_repo.bzl index dac6d741a5..d5192ec44b 100644 --- a/python/private/internal_config_repo.bzl +++ b/python/private/internal_config_repo.bzl @@ -22,7 +22,7 @@ load("//python/private:text_util.bzl", "render") load(":repo_utils.bzl", "repo_utils") _ENABLE_PIPSTAR_ENVVAR_NAME = "RULES_PYTHON_ENABLE_PIPSTAR" -_ENABLE_PIPSTAR_DEFAULT = "1" +_ENABLE_PIPSTAR_DEFAULT = "0" _ENABLE_DEPRECATION_WARNINGS_ENVVAR_NAME = "RULES_PYTHON_DEPRECATION_WARNINGS" _ENABLE_DEPRECATION_WARNINGS_DEFAULT = "0" From 793dddf2458403e7f0ce8a923584ec5b770d3f68 Mon Sep 17 00:00:00 2001 From: Levi Zim Date: Sun, 26 Oct 2025 12:35:53 +0800 Subject: [PATCH 502/922] fix: Add linux_riscv64 to _pip_repository_impl (#3350) Add `linux_riscv64` support for pulling pip dependencies. This is not adding any hermetic toolchain support - user has to provide a working toolchain. Fix #2729 --------- Co-authored-by: Ignas Anikevicius <240938+aignas@users.noreply.github.com> --- CHANGELOG.md | 2 ++ python/private/pypi/pip_repository.bzl | 1 + python/private/pypi/whl_installer/platform.py | 3 +++ python/private/pypi/whl_target_platforms.bzl | 1 + tests/pypi/whl_installer/platform_test.py | 6 +++--- .../whl_target_platforms/whl_target_platforms_tests.bzl | 8 ++++++++ 6 files changed, 18 insertions(+), 3 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 6cb571681d..ad8f1c93a7 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -113,6 +113,8 @@ END_UNRELEASED_TEMPLATE ([#3339](https://github.com/bazel-contrib/rules_python/issues/3339)). * (uv) {obj}`//python/uv:lock.bzl%lock` now works with a local platform runtime. +* (pypi) `linux_riscv64` is added to the platforms list in `_pip_repository_impl`, + which fixes [a build issue for tensorflow on riscv64](https://github.com/bazel-contrib/rules_python/discussions/2729). * (toolchains) WORKSPACE builds now correctly register musl and freethreaded variants. Setting {obj}`--py_linux_libc=musl` and `--py_freethreaded=yes` now activate them, respectively. diff --git a/python/private/pypi/pip_repository.bzl b/python/private/pypi/pip_repository.bzl index e9a4c44da3..d635651039 100644 --- a/python/private/pypi/pip_repository.bzl +++ b/python/private/pypi/pip_repository.bzl @@ -96,6 +96,7 @@ def _pip_repository_impl(rctx): "linux_aarch64", "linux_arm", "linux_ppc", + "linux_riscv64", "linux_s390x", "linux_x86_64", "osx_aarch64", diff --git a/python/private/pypi/whl_installer/platform.py b/python/private/pypi/whl_installer/platform.py index ff267fe4aa..0757d86990 100644 --- a/python/private/pypi/whl_installer/platform.py +++ b/python/private/pypi/whl_installer/platform.py @@ -45,6 +45,7 @@ class Arch(Enum): ppc64le = 5 s390x = 6 arm = 7 + riscv64 = 8 amd64 = x86_64 arm64 = aarch64 i386 = x86_32 @@ -269,6 +270,8 @@ def platform_machine(self) -> str: return "ppc" elif self.arch == Arch.ppc64le: return "ppc64le" + elif self.arch == Arch.riscv64: + return "riscv64" elif self.arch == Arch.s390x: return "s390x" else: diff --git a/python/private/pypi/whl_target_platforms.bzl b/python/private/pypi/whl_target_platforms.bzl index 6c3dd5da83..28547c679c 100644 --- a/python/private/pypi/whl_target_platforms.bzl +++ b/python/private/pypi/whl_target_platforms.bzl @@ -30,6 +30,7 @@ _CPU_ALIASES = { "ppc": "ppc", "ppc64": "ppc", "ppc64le": "ppc64le", + "riscv64": "riscv64", "s390x": "s390x", "arm": "arm", "armv6l": "arm", diff --git a/tests/pypi/whl_installer/platform_test.py b/tests/pypi/whl_installer/platform_test.py index ad65650779..0d944bb196 100644 --- a/tests/pypi/whl_installer/platform_test.py +++ b/tests/pypi/whl_installer/platform_test.py @@ -38,17 +38,17 @@ def test_can_get_specific_from_string(self): def test_can_get_all_for_py_version(self): cp39 = Platform.all(minor_version=9, micro_version=0) - self.assertEqual(21, len(cp39), f"Got {cp39}") + self.assertEqual(24, len(cp39), f"Got {cp39}") self.assertEqual(cp39, Platform.from_string("cp39.0_*")) def test_can_get_all_for_os(self): linuxes = Platform.all(OS.linux, minor_version=9) - self.assertEqual(7, len(linuxes)) + self.assertEqual(8, len(linuxes)) self.assertEqual(linuxes, Platform.from_string("cp39_linux_*")) def test_can_get_all_for_os_for_host_python(self): linuxes = Platform.all(OS.linux) - self.assertEqual(7, len(linuxes)) + self.assertEqual(8, len(linuxes)) self.assertEqual(linuxes, Platform.from_string("linux_*")) def test_platform_sort(self): diff --git a/tests/pypi/whl_target_platforms/whl_target_platforms_tests.bzl b/tests/pypi/whl_target_platforms/whl_target_platforms_tests.bzl index a976a0cf95..6bec26c10c 100644 --- a/tests/pypi/whl_target_platforms/whl_target_platforms_tests.bzl +++ b/tests/pypi/whl_target_platforms/whl_target_platforms_tests.bzl @@ -34,6 +34,9 @@ def _test_simple(env): "musllinux_1_1_ppc64le": [ struct(os = "linux", cpu = "ppc64le", abi = None, target_platform = "linux_ppc64le", version = (1, 1)), ], + "musllinux_1_2_riscv64": [ + struct(os = "linux", cpu = "riscv64", abi = None, target_platform = "linux_riscv64", version = (1, 2)), + ], "win_amd64": [ struct(os = "windows", cpu = "x86_64", abi = None, target_platform = "windows_x86_64", version = (0, 0)), ], @@ -66,6 +69,9 @@ def _test_with_abi(env): "musllinux_1_1_ppc64le": [ struct(os = "linux", cpu = "ppc64le", abi = "cp311", target_platform = "cp311_linux_ppc64le", version = (1, 1)), ], + "musllinux_1_2_riscv64": [ + struct(os = "linux", cpu = "riscv64", abi = "cp311", target_platform = "cp311_linux_riscv64", version = (1, 2)), + ], "win_amd64": [ struct(os = "windows", cpu = "x86_64", abi = "cp311", target_platform = "cp311_windows_x86_64", version = (0, 0)), ], @@ -103,6 +109,7 @@ def _can_parse_existing_tags(env): "manylinux_11_12_i686": 1, "manylinux_11_12_ppc64": 1, "manylinux_11_12_ppc64le": 1, + "manylinux_11_12_riscv64": 1, "manylinux_11_12_s390x": 1, "manylinux_11_12_x86_64": 1, "manylinux_1_2_aarch64": 1, @@ -111,6 +118,7 @@ def _can_parse_existing_tags(env): "musllinux_11_12_armv7l": 1, "musllinux_11_12_i686": 1, "musllinux_11_12_ppc64le": 1, + "musllinux_11_12_riscv64": 1, "musllinux_11_12_s390x": 1, "musllinux_11_12_x86_64": 1, "win32": 1, From 74d7698eafcc337ecf0b6208f2a9fbc7e75fe2a5 Mon Sep 17 00:00:00 2001 From: Laurenz Date: Sun, 26 Oct 2025 05:37:01 +0100 Subject: [PATCH 503/922] feat: Add libpython QNX platform support (#3372) Add libpython QNX platform support. Note that the qnx py toolchain must come from somewhere else, this doesn't add an interpreter --------- Co-authored-by: Ignas Anikevicius <240938+aignas@users.noreply.github.com> --- python/private/hermetic_runtime_repo_setup.bzl | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/python/private/hermetic_runtime_repo_setup.bzl b/python/private/hermetic_runtime_repo_setup.bzl index 46495e49c0..4bcc1c1512 100644 --- a/python/private/hermetic_runtime_repo_setup.bzl +++ b/python/private/hermetic_runtime_repo_setup.bzl @@ -179,10 +179,6 @@ def define_hermetic_runtime_toolchain_impl( "libs/python{major}{minor}t.lib".format(**version_dict), "libs/python3t.lib", ], - "@platforms//os:linux": [ - "lib/libpython{major}.{minor}.so".format(**version_dict), - "lib/libpython{major}.{minor}.so.1.0".format(**version_dict), - ], "@platforms//os:macos": ["lib/libpython{major}.{minor}.dylib".format(**version_dict)], "@platforms//os:windows": [ "python3.dll", @@ -190,6 +186,10 @@ def define_hermetic_runtime_toolchain_impl( "libs/python{major}{minor}.lib".format(**version_dict), "libs/python3.lib", ], + "//conditions:default": [ + "lib/libpython{major}.{minor}.so".format(**version_dict), + "lib/libpython{major}.{minor}.so.1.0".format(**version_dict), + ], }), ) From caab0fae80927e308e7aa18186ebf8c2b811b79e Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Mon, 27 Oct 2025 22:56:10 -0700 Subject: [PATCH 504/922] build: change gazelle rules_python override to use bazelrc (#3382) This is because the `--override_module` flag in BCR presubmit configs can't affect local_path_override in MODULE.bazel. Per https://github.com/bazelbuild/bazel-central-registry/pull/6241#issuecomment-3431422637 --- gazelle/examples/bzlmod_build_file_generation/.bazelrc | 7 +++++++ .../examples/bzlmod_build_file_generation/MODULE.bazel | 9 --------- 2 files changed, 7 insertions(+), 9 deletions(-) diff --git a/gazelle/examples/bzlmod_build_file_generation/.bazelrc b/gazelle/examples/bzlmod_build_file_generation/.bazelrc index 0289886d4d..d58665596b 100644 --- a/gazelle/examples/bzlmod_build_file_generation/.bazelrc +++ b/gazelle/examples/bzlmod_build_file_generation/.bazelrc @@ -7,3 +7,10 @@ common --experimental_enable_bzlmod coverage --java_runtime_version=remotejdk_11 common:bazel7.x --incompatible_python_disallow_native_rules + +# NOTE: This override is specific to the development of gazelle itself +# and the testing of it during its BCR release presubmits. +# In development of gazelle itself, we override it to the development +# rules_python code. In the BCR presubmits, this override is removed +# and the bazel_dep version of rules_python is used. +common --override_module=rules_python=../../../ diff --git a/gazelle/examples/bzlmod_build_file_generation/MODULE.bazel b/gazelle/examples/bzlmod_build_file_generation/MODULE.bazel index 5ace7f3d3a..d93e606d09 100644 --- a/gazelle/examples/bzlmod_build_file_generation/MODULE.bazel +++ b/gazelle/examples/bzlmod_build_file_generation/MODULE.bazel @@ -15,15 +15,6 @@ module( # https://github.com/bazel-contrib/rules_python/releases bazel_dep(name = "rules_python", version = "1.0.0") -# NOTE: This override is removed for BCR presubmits and the version -# specified by bazel_dep() is used instead. -# The following loads rules_python from the file system. -# For usual setups you should remove this local_path_override block. -local_path_override( - module_name = "rules_python", - path = "../../..", -) - # The following stanza defines the dependency rules_python_gazelle_plugin. # For typical setups you set the version. # See the releases page for available versions. From d08cf537541ffc99bbd98dd71949a507516a4ff8 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Wed, 29 Oct 2025 15:58:30 -0700 Subject: [PATCH 505/922] gazelle: set min rules_python version as 1.4 (#3383) rules_python 1.4 is when the python.defaults tag class was introduced, which is used in the example's module file. Fixes BCR presubmit failures in https://github.com/bazelbuild/bazel-central-registry/pull/6330 and https://github.com/bazelbuild/bazel-central-registry/pull/6241 --- gazelle/examples/bzlmod_build_file_generation/MODULE.bazel | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/gazelle/examples/bzlmod_build_file_generation/MODULE.bazel b/gazelle/examples/bzlmod_build_file_generation/MODULE.bazel index d93e606d09..ce779627f5 100644 --- a/gazelle/examples/bzlmod_build_file_generation/MODULE.bazel +++ b/gazelle/examples/bzlmod_build_file_generation/MODULE.bazel @@ -13,7 +13,7 @@ module( # For typical setups you set the version. # See the releases page for available versions. # https://github.com/bazel-contrib/rules_python/releases -bazel_dep(name = "rules_python", version = "1.0.0") +bazel_dep(name = "rules_python", version = "1.4.0") # The following stanza defines the dependency rules_python_gazelle_plugin. # For typical setups you set the version. From e85728f5df89408ed78450ee43e66b351f2e9ce9 Mon Sep 17 00:00:00 2001 From: Ignas Anikevicius <240938+aignas@users.noreply.github.com> Date: Sat, 1 Nov 2025 12:35:04 +0900 Subject: [PATCH 506/922] chore(toolchain): drop all but the latest 3.9 toolchain (#3377) Summary: - drop the `3.9` versions one by one. - add the latest astral toolchain builds. - add a disclaimer to our changeleg about `3.9`. - update examples to use the latest `3.9` build (separate PR) will switch to a different python version. Work towards #2704 --- CHANGELOG.md | 9 +- examples/BUILD.bazel | 2 +- examples/bzlmod/MODULE.bazel | 3 +- examples/pip_parse/MODULE.bazel | 6 +- examples/pip_parse/WORKSPACE | 2 +- python/versions.bzl | 347 ++++++++++---------------------- 6 files changed, 116 insertions(+), 253 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index ad8f1c93a7..bae98230e0 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -30,11 +30,13 @@ BEGIN_UNRELEASED_TEMPLATE {#v0-0-0-removed} ### Removed +* (toolchain) Remove all of the python 3.9 toolchain versions except for the `3.9.25`. + This version has reached EOL and will no longer receive any security fixes, please update to + `3.10` or above. -* Nothing removed. {#v0-0-0-changed} ### Changed -* Nothing changed. +* (toolchains) Use toolchains from the [20251031] release. {#v0-0-0-fixed} ### Fixed @@ -43,8 +45,9 @@ BEGIN_UNRELEASED_TEMPLATE {#v0-0-0-added} ### Added -* Nothing added. +* (toolchains) `3.9.25` Python toolchain from [20251031] release. +[20251031]: https://github.com/astral-sh/python-build-standalone/releases/tag/20251031 END_UNRELEASED_TEMPLATE --> diff --git a/examples/BUILD.bazel b/examples/BUILD.bazel index 716cb9a20a..a08e5e64ae 100644 --- a/examples/BUILD.bazel +++ b/examples/BUILD.bazel @@ -26,7 +26,7 @@ lock( "--universal", "--python-version=3.9", ], - python_version = "3.9.19", + python_version = "3.9", ) lock( diff --git a/examples/bzlmod/MODULE.bazel b/examples/bzlmod/MODULE.bazel index 8bd6718387..0a505aa5e4 100644 --- a/examples/bzlmod/MODULE.bazel +++ b/examples/bzlmod/MODULE.bazel @@ -55,7 +55,6 @@ python.override( # require versions not listed here. # available_python_versions = [ # "3.10.9", - # "3.9.18", # "3.9.19", # # The following is used by the `other_module` and we need to include it here # # as well. @@ -65,7 +64,7 @@ python.override( # instead of rules_python's defaulting to the latest available version, # controls what full version is used when `3.x` is requested. minor_mapping = { - "3.9": "3.9.19", + "3.9": "3.9.25", }, ) diff --git a/examples/pip_parse/MODULE.bazel b/examples/pip_parse/MODULE.bazel index f9ca90833f..ead5a06e29 100644 --- a/examples/pip_parse/MODULE.bazel +++ b/examples/pip_parse/MODULE.bazel @@ -9,14 +9,14 @@ local_path_override( python = use_extension("@rules_python//python/extensions:python.bzl", "python") python.toolchain( # We can specify the exact version. - python_version = "3.9.13", + python_version = "3.9.25", ) # You can use this repo mapping to ensure that your BUILD.bazel files don't need # to be updated when the python version changes to a different `3.9` version. use_repo( python, - python_3_9 = "python_3_9_13", + python_3_9 = "python_3_9_25", ) pip = use_extension("@rules_python//python/extensions:pip.bzl", "pip") @@ -34,7 +34,7 @@ pip.parse( }, hub_name = "pypi", # We need to use the same version here as in the `python.toolchain` call. - python_version = "3.9.13", + python_version = "3.9.25", requirements_lock = "//:requirements_lock.txt", requirements_windows = "//:requirements_windows.txt", ) diff --git a/examples/pip_parse/WORKSPACE b/examples/pip_parse/WORKSPACE index bb4714d941..e0d60af9ff 100644 --- a/examples/pip_parse/WORKSPACE +++ b/examples/pip_parse/WORKSPACE @@ -11,7 +11,7 @@ py_repositories() python_register_toolchains( name = "python_3_9", - python_version = "3.9.13", + python_version = "3.9.25", ) load("@rules_python//python:pip.bzl", "pip_parse") diff --git a/python/versions.bzl b/python/versions.bzl index 6de09ca72d..7e1b36b207 100644 --- a/python/versions.bzl +++ b/python/versions.bzl @@ -65,157 +65,18 @@ TOOL_VERSIONS = { }, "strip_prefix": "python", }, - "3.9.10": { - "url": "20220227/cpython-{python_version}+20220227-{platform}-{build}.tar.gz", - "sha256": { - "aarch64-apple-darwin": "ad66c2a3e7263147e046a32694de7b897a46fb0124409d29d3a93ede631c8aee", - "aarch64-unknown-linux-gnu": "12dd1f125762f47975990ec744532a1cf3db74ad60f4dfb476ca42deb7f78ca4", - "x86_64-apple-darwin": "fdaf594142446029e314a9beb91f1ac75af866320b50b8b968181e592550cd68", - "x86_64-pc-windows-msvc": "c145d9d8143ce163670af124b623d7a2405143a3708b033b4d33eed355e61b24", - "x86_64-unknown-linux-gnu": "455089cc576bd9a58db45e919d1fc867ecdbb0208067dffc845cc9bbf0701b70", - }, - "strip_prefix": "python", - }, - "3.9.12": { - "url": "20220502/cpython-{python_version}+20220502-{platform}-{build}.tar.gz", - "sha256": { - "aarch64-apple-darwin": "8dee06c07cc6429df34b6abe091a4684a86f7cec76f5d1ccc1c3ce2bd11168df", - "aarch64-unknown-linux-gnu": "2ee1426c181e65133e57dc55c6a685cb1fb5e63ef02d684b8a667d5c031c4203", - "x86_64-apple-darwin": "2453ba7f76b3df3310353b48c881d6cff622ba06e30d2b6ae91588b2bc9e481a", - "x86_64-pc-windows-msvc": "3024147fd987d9e1b064a3d94932178ff8e0fe98cfea955704213c0762fee8df", - "x86_64-unknown-linux-gnu": "ccca12f698b3b810d79c52f007078f520d588232a36bc12ede944ec3ea417816", - }, - "strip_prefix": "python", - }, - "3.9.13": { - "url": "20220802/cpython-{python_version}+20220802-{platform}-{build}.tar.gz", - "sha256": { - "aarch64-apple-darwin": "d9603edc296a2dcbc59d7ada780fd12527f05c3e0b99f7545112daf11636d6e5", - "aarch64-unknown-linux-gnu": "80415aac1b96255b9211f6a4c300f31e9940c7e07a23d0dec12b53aa52c0d25e", - "x86_64-apple-darwin": "9540a7efb7c8a54a48aff1cb9480e49588d9c0a3f934ad53f5b167338174afa3", - "x86_64-pc-windows-msvc": "b538127025a467c64b3351babca2e4d2ea7bdfb7867d5febb3529c34456cdcd4", - "x86_64-unknown-linux-gnu": "ce1cfca2715e7e646dd618a8cb9baff93000e345ccc979b801fc6ccde7ce97df", - }, - "strip_prefix": "python", - }, - "3.9.15": { - "url": "20221106/cpython-{python_version}+20221106-{platform}-{build}.tar.gz", - "sha256": { - "aarch64-apple-darwin": "64dc7e1013481c9864152c3dd806c41144c79d5e9cd3140e185c6a5060bdc9ab", - "aarch64-unknown-linux-gnu": "52a8c0a67fb919f80962d992da1bddb511cdf92faf382701ce7673e10a8ff98f", - "x86_64-apple-darwin": "f2bcade6fc976c472f18f2b3204d67202d43ae55cf6f9e670f95e488f780da08", - "x86_64-pc-windows-msvc": "022daacab215679b87f0d200d08b9068a721605fa4721ebeda38220fc641ccf6", - "x86_64-unknown-linux-gnu": "cdc3a4cfddcd63b6cebdd75b14970e02d8ef0ac5be4d350e57ab5df56c19e85e", - }, - "strip_prefix": "python", - }, - "3.9.16": { - "url": "20230507/cpython-{python_version}+20230507-{platform}-{build}.tar.gz", - "sha256": { - "aarch64-apple-darwin": "c1de1d854717a6245f45262ef1bb17b09e2c587590e7e3f406593c143ff875bd", - "aarch64-unknown-linux-gnu": "f629b75ebfcafe9ceee2e796b7e4df5cf8dbd14f3c021afca078d159ab797acf", - "ppc64le-unknown-linux-gnu": "ff3ac35c58f67839aff9b5185a976abd3d1abbe61af02089f7105e876c1fe284", - "x86_64-apple-darwin": "3abc4d5fbbc80f5f848f280927ac5d13de8dc03aabb6ae65d8247cbb68e6f6bf", - "x86_64-pc-windows-msvc": "cdabb47204e96ce7ea31fbd0b5ed586114dd7d8f8eddf60a509a7f70b48a1c5e", - "x86_64-unknown-linux-gnu": "2b6e146234a4ef2a8946081fc3fbfffe0765b80b690425a49ebe40b47c33445b", - }, - "strip_prefix": "python", - }, - "3.9.17": { - "url": "20230726/cpython-{python_version}+20230726-{platform}-{build}.tar.gz", - "sha256": { - "aarch64-apple-darwin": "73dbe2d702210b566221da9265acc274ba15275c5d0d1fa327f44ad86cde9aa1", - "aarch64-unknown-linux-gnu": "b77012ddaf7e0673e4aa4b1c5085275a06eee2d66f33442b5c54a12b62b96cbe", - "ppc64le-unknown-linux-gnu": "c591a28d943dce5cf9833e916125fdfbeb3120270c4866ee214493ccb5b83c3c", - "s390x-unknown-linux-gnu": "01454d7cc7c9c2fccde42ba868c4f372eaaafa48049d49dd94c9cf2875f497e6", - "x86_64-apple-darwin": "dfe1bea92c94b9cb779288b0b06e39157c5ff7e465cdd24032ac147c2af485c0", - "x86_64-pc-windows-msvc": "9b9a1e21eff29dcf043cea38180cf8ca3604b90117d00062a7b31605d4157714", - "x86_64-unknown-linux-gnu": "26c4a712b4b8e11ed5c027db5654eb12927c02da4857b777afb98f7a930ce637", - }, - "strip_prefix": "python", - }, - "3.9.18": { - "url": "20240224/cpython-{python_version}+20240224-{platform}-{build}.tar.gz", - "sha256": { - "aarch64-apple-darwin": "2548f911a6e316575c303ba42bb51540dc9b47a9f76a06a2a37460d93b177aa2", - "aarch64-unknown-linux-gnu": "e5bc5196baa603d635ee6b0cd141e359752ad3e8ea76127eb9141a3155c51200", - "ppc64le-unknown-linux-gnu": "d6b18df7a25fe034fd5ce4e64216df2cc78b2d4d908d2a1c94058ae700d73d22", - "s390x-unknown-linux-gnu": "15d059507c7e900e9665f31e8d903e5a24a68ceed24f9a1c5ac06ab42a354f3f", - "x86_64-apple-darwin": "171d8b472fce0295be0e28bb702c43d5a2a39feccb3e72efe620ac3843c3e402", - "x86_64-pc-windows-msvc": "a9bdbd728ed4c353a4157ecf74386117fb2a2769a9353f491c528371cfe7f6cd", - "x86_64-unknown-linux-gnu": "0e5663025121186bd17d331538a44f48b41baff247891d014f3f962cbe2716b4", - }, - "strip_prefix": "python", - }, - "3.9.19": { - "url": "20240726/cpython-{python_version}+20240726-{platform}-{build}.tar.gz", - "sha256": { - "aarch64-apple-darwin": "0e5a7aae57c53d7a849bc7f67764a947b626e3fe8d4d41a8eed11d9e4be0b1c6", - "aarch64-unknown-linux-gnu": "05ec896db9a9d4fe8004b4e4b6a6fdc588a015fedbddb475490885b0d9c7d9b3", - "ppc64le-unknown-linux-gnu": "bfff0e3d536b2f0c315e85926cc317b7b756701b6de781a8972cefbdbc991ca2", - "s390x-unknown-linux-gnu": "059ec97080b205ea5f1ddf71c18e22b691e8d68192bd37d13ad8f4359915299d", - "x86_64-apple-darwin": "f2ae9fcac044a329739b8c1676245e8cb6b3094416220e71823d2673bdea0bdb", - "x86_64-pc-windows-msvc": "a8df6a00140055c9accb0be632e7add951d587bbe3d63c40827bbd5145d8f557", - "x86_64-unknown-linux-gnu": "cbf94cb1c9d4b5501d9b3652f6e8400c2cab7c41dfea48d344d9e7f29692b91b", - }, - "strip_prefix": "python", - }, - "3.9.20": { - "url": "20241016/cpython-{python_version}+20241016-{platform}-{build}.tar.gz", - "sha256": { - "aarch64-apple-darwin": "34ab2bc4c51502145e1a624b4e4ea06877e3d1934a88cc73ac2e0fd5fd439b75", - "aarch64-unknown-linux-gnu": "1e486c054a4e86666cf24e04f5e29456324ba9c2b95bf1cae1805be90d3da154", - "ppc64le-unknown-linux-gnu": "9a24ccdbfc7f67545d859128f02a3150a160ea6c2fc134b0773bf56f2d90b397", - "s390x-unknown-linux-gnu": "2cee381069bf344fb20eba609af92dfe7ba67eb75bea08eeccf11048a2c380c0", - "x86_64-apple-darwin": "193dc7f0284e4917d52b17a077924474882ee172872f2257cfe3375d6d468ed9", - "x86_64-pc-windows-msvc": "5069008a237b90f6f7a86956903f2a0221b90d471daa6e4a94831eaa399e3993", - "x86_64-unknown-linux-gnu": "c20ee831f7f46c58fa57919b75a40eb2b6a31e03fd29aaa4e8dab4b9c4b60d5d", - "x86_64-unknown-linux-musl": "5c1cc348e317fe7af1acd6a7f665b46eccb554b20d6533f0e76c53f44d4556cc", - }, - "strip_prefix": "python", - }, - "3.9.21": { - "url": "20250317/cpython-{python_version}+20250317-{platform}-{build}.tar.gz", - "sha256": { - "aarch64-apple-darwin": "2a7d83db10c082ce59e9c4b8bd6c5790310198fb759a7c94aceebac1d93676d3", - "aarch64-unknown-linux-gnu": "758ebbc4d60b3ca26cf21720232043ad626373fbeb6632122e5db622a1f55465", - "ppc64le-unknown-linux-gnu": "3c7c0cc16468659049ac2f843ffba29144dd987869c943b83c2730569b7f57bd", - "riscv64-unknown-linux-gnu": "ef1463ad5349419309060854a5f942b0bd7bd0b9245b53980129836187e68ad9", - "s390x-unknown-linux-gnu": "e66e52dcbe3e20153e7d5844451bf58a69f41b858348e0f59c547444bfe191ee", - "x86_64-apple-darwin": "786ebd91e4dd0920acf60aa3428a627a937342d2455f7eb5e9a491517c32db3d", - "x86_64-pc-windows-msvc": "5392cee2ef7cd20b34128384d0b31864fb3c02bdb7a8ae6995cfec621bb657bc", - "x86_64-unknown-linux-gnu": "6f426b5494e90701ffa2753e229252e8b3ac61151a09c8cd6c0a649512df8ab2", - "x86_64-unknown-linux-musl": "6113c6c5f88d295bb26279b8a49d74126ee12db137854e0d8c3077051a4eddc4", - }, - "strip_prefix": "python", - }, - "3.9.23": { - "url": "20250808/cpython-{python_version}+20250808-{platform}-{build}.tar.gz", - "sha256": { - "aarch64-apple-darwin": "d32da9eae3f516cc0bd8240bfef54dede757d6daf1d8cf605eacbc8a205884e8", - "aarch64-unknown-linux-gnu": "0318b6c9ad6fb229da8d40aa3671ee27eeb678530246a1b172b72071f76091bc", - "ppc64le-unknown-linux-gnu": "b40b3509dc72abb21f4310f0e94678b36ff73432dc84c41fea132a51c4017f79", - "riscv64-unknown-linux-gnu": "a7d847dc62177cf06237dfa26c317148b22418ded51aa89e8cf7242784293ad4", - "s390x-unknown-linux-gnu": "425abe5d3ec98e9b18c908209a4ffe239a283ee648e0eea65821e45f074689e7", - "x86_64-apple-darwin": "c1bfab90aea566ffaeff65299a20503a880ea93054bbd8bbed98f4f11e9e7383", - "x86_64-pc-windows-msvc": "fb400b25cbcbfed6aeaaca8d9a3cdf1a09b602bf5ed6d1ae7075cde40c1cd81e", - "x86_64-unknown-linux-gnu": "77fd3fa10abbb08949eda70ca7fb94f72e2f9e0016611be328a7b31c3aa9894d", - "x86_64-unknown-linux-musl": "a8a0df23bc1bc050ed8730c65d818382667cf37ba96a08fccd5bb12a689e6a1c", - }, - "strip_prefix": "python", - }, - "3.9.24": { - "url": "20251014/cpython-{python_version}+20251014-{platform}-{build}.tar.gz", + "3.9.25": { + "url": "20251031/cpython-{python_version}+20251031-{platform}-{build}.tar.gz", "sha256": { - "aarch64-apple-darwin": "6b65213e639e91eb8072db80ed9c140d769af1d5e0386efd8f153449c3694714", - "aarch64-unknown-linux-gnu": "d840efd9d81ad557019ebd0d435828fc32101cd01be82046087b4aee463dca0c", - "ppc64le-unknown-linux-gnu": "e6501df1f32cc9cbfa8bb625b4d5a88ad9e83452525c1989ad50334a16a5d9a6", - "riscv64-unknown-linux-gnu": "811f0f3966f42186a59ae9112b8faf92bbe88fae8dae725f072fee116b628b2a", - "s390x-unknown-linux-gnu": "7bf6bb7a95527419379c94b5d3181f7000f47e7c5a828cde58b0f7cfe9421347", - "x86_64-apple-darwin": "14beda9465feb6991f73d6f6cb9e69afc576c5cac8c185bd729f491aa4305bfb", - "x86_64-pc-windows-msvc": "a2fdaf290361386396bbfaa08e13fc2b88e1149f870adf18836e262c609406db", - "x86_64-unknown-linux-gnu": "866745efbee219a3f9b9d54ee1477ebf92542bb9ff9f6591a7e5a3643a0d4214", - "x86_64-unknown-linux-musl": "ee1dec977925293be46cecc5f7e9034394f0f8cc736afc92528689e59d6f19db", + "aarch64-apple-darwin": "87275619c2706affa4d1090d2ca3dad354b6d69f8b85dbfafe38785870751b9a", + "aarch64-unknown-linux-gnu": "6112d46355857680b81849764a6cf9f38cc4cd0d1cf29d432bc12fe5aeedf9d0", + "ppc64le-unknown-linux-gnu": "828364b6f54fa45ac2dc91f8e45d5b74306372af374a9ef16eeb2ea81253ed3f", + "riscv64-unknown-linux-gnu": "17467e0158e5ad04453c447d6773c23b044172276441e22e23058fd3ea053e27", + "s390x-unknown-linux-gnu": "3e9539f83e67faa813fd06171199b2d33c89821dfa9a33bf6e27ad67f1b6932d", + "x86_64-apple-darwin": "ace63cfe27a9487c4d72e1cb518be01c1d985271da0b2158e813801f7d3e5503", + "x86_64-pc-windows-msvc": "4fb1b416482ce94d73cfa140317a670c596c830671d137b07c26afe8c461768a", + "x86_64-unknown-linux-gnu": "42834f61eb6df43432c3dd6ab9ca3fdf8c06d10a404ebdb53d6902e6b9570b08", + "x86_64-unknown-linux-musl": "76593e8c889e81e82db5fe117fe15b69466f85100ab2ec0e4035aa86242b4e93", }, "strip_prefix": "python", }, @@ -370,17 +231,17 @@ TOOL_VERSIONS = { "strip_prefix": "python", }, "3.10.19": { - "url": "20251014/cpython-{python_version}+20251014-{platform}-{build}.tar.gz", + "url": "20251031/cpython-{python_version}+20251031-{platform}-{build}.tar.gz", "sha256": { - "aarch64-apple-darwin": "06cfdfa8966dfd86204d45c6a241dd37cb0b3ede90986591fc0b0dbe576848de", - "aarch64-unknown-linux-gnu": "c4c760f49dbba10a0f91b2fd52c847dd50cbe7cb8cb19bb7598c4dc38a358e9c", - "ppc64le-unknown-linux-gnu": "8d32d9c85ac6ac71f6996313f87d50da34a159e037d3795bbb745f1c39d7b62f", - "riscv64-unknown-linux-gnu": "636d0001877c1d2566e3bd4be61c6df08ba55eefd06414cb72a22e154432c22a", - "s390x-unknown-linux-gnu": "1b15c9c090114c063a5802e005ea35c61a3c4e83efb8e8ce687d77f47060f8ed", - "x86_64-apple-darwin": "b4e0c82f350f18a8fb1b1982f03c1c90aaba5d9ab74fe6ede9896306f64a287c", - "x86_64-pc-windows-msvc": "e2d9193b2d2fd99fac3fb90eda216100b64cd7cf14f291d9425436ea9b1eaa04", - "x86_64-unknown-linux-gnu": "85c96114de83d783db18137f3858bcd3b5a9c4cbe9053f0072d7b5f52154a8c9", - "x86_64-unknown-linux-musl": "0d0f2b1f8bb014018dc4c24b6680f17f48017dafe25e380cefc2490e4b90e1ae", + "aarch64-apple-darwin": "43bda24c2fc073bc308bf631203b917a72640d59b59fdad4ba14503d84727012", + "aarch64-unknown-linux-gnu": "f77a8a8aa77f3f943126fa9215a25309da4bf20398fc8f4b4eec54b5fc7570ef", + "ppc64le-unknown-linux-gnu": "1c55d160fc4c3b93528cd6aaa2bb4ca6018a99e5a45919d33dc761a43a69f860", + "riscv64-unknown-linux-gnu": "21134d35721cdad4c881f35d0957cc19df9a45d194afb38a099faded3c1cfb4d", + "s390x-unknown-linux-gnu": "df0db070f1eb73ab4e371eea32213ddb3500737ea5560a6f0ffd65c82af64ddc", + "x86_64-apple-darwin": "76c12e633c09c2a790f8a958a55df4495527e0718d1875310c836e757c0c7b55", + "x86_64-pc-windows-msvc": "cfa08a4caf2df1b43551b843c052d6a8814e2ea0c97268b021f0423646c244c3", + "x86_64-unknown-linux-gnu": "fb1caac917d7b6497bb6f5950da5f1e48d05c43a498948dd97f85760c4382d9f", + "x86_64-unknown-linux-musl": "ba85013ed5ac7733fc6840168cc33ed19e9959b363dc80227d54f8fd9c92c0f4", }, "strip_prefix": "python", }, @@ -516,18 +377,18 @@ TOOL_VERSIONS = { "strip_prefix": "python", }, "3.11.14": { - "url": "20251014/cpython-{python_version}+20251014-{platform}-{build}.tar.gz", + "url": "20251031/cpython-{python_version}+20251031-{platform}-{build}.tar.gz", "sha256": { - "aarch64-apple-darwin": "99d98bf73d9906d18a9184054a328288ede2cb4a2d245a05411a28e8d023aab6", - "aarch64-unknown-linux-gnu": "8b033614f3a6969d86c20f9b823277ee8e1f72788307c082a44d2ad4cc856e2b", - "ppc64le-unknown-linux-gnu": "3936f10e39f3ceeb422514f996de7f4ad095241be22df3f5db007c92f6ae1ac7", - "riscv64-unknown-linux-gnu": "790247290650896b40b7a1ca9e47b6951ac3d0750850b356033386ebf05edf80", - "s390x-unknown-linux-gnu": "459989097b6ac89c7b940ae8eb2f3508ea4f12d6c1ff192b4dbc1bb47e95ad2a", - "x86_64-apple-darwin": "d234fa6518634daf3aa812895ec757d0e0b1fea3335fd0c5038d4e2bcc5d7ee5", - "x86_64-pc-windows-msvc": "80022423ca581c88d5bb7beb889f10c12d3d8d2e5cc6422fd2b060b52e45aa05", - "aarch64-pc-windows-msvc": "94958c60345574c1cfdee7e57925642cdf2eb2008b64a0018ca9c3b509ce16b0", - "x86_64-unknown-linux-gnu": "d0623c777fb89b904b56cd5aba51af29cbb34b1f9d45f0672f90f6dce30fa93e", - "x86_64-unknown-linux-musl": "0ce7c9f584fa51860f79f4f6c7fe22a6bbd986d324acb23ad8c9f237c8af964a", + "aarch64-apple-darwin": "6de5572b33c65af1c9b7caf00ec593fb04cffb7e14fa393a98261bb9bc464713", + "aarch64-unknown-linux-gnu": "510edb027527413c4249256194cb8ad2590b52dd93f7123b4cb341aff5d05894", + "ppc64le-unknown-linux-gnu": "4e0bc6a818e0c6a9d7d3ebe1a95591fd84440520577aa837facc96a4b7a80e35", + "riscv64-unknown-linux-gnu": "16519e69297144f81b2421333bc9e0b6466cf3c84749b216b695cfb4c9deb32f", + "s390x-unknown-linux-gnu": "5f9c1b203cdf34c8bff1aef69b63bbf11309bd16ca6e429d8c3651eaa2b3d080", + "x86_64-apple-darwin": "4891cbf34e8652b7bd1054b9502395e4b7e048e2e517c040fbf6c8297cb954d6", + "x86_64-pc-windows-msvc": "5223b83ed9e2aa5e9e17d2ebcf767956e998876339b9cde1980a47e9d4655fb6", + "aarch64-pc-windows-msvc": "38d0d1466561e15965e8d2c20f5e5be649598f55c761ecab553d087fbd217337", + "x86_64-unknown-linux-gnu": "60f0bd473d861cc45d3401d9914e47ccb9fa037f88a91879ed517a62042b8477", + "x86_64-unknown-linux-musl": "25e82d1e85b90a8ab724ee633a1811b1921797f5c25ee69c6595052371b91a87", }, "strip_prefix": "python", }, @@ -656,18 +517,18 @@ TOOL_VERSIONS = { "strip_prefix": "python", }, "3.12.12": { - "url": "20251014/cpython-{python_version}+20251014-{platform}-{build}.tar.gz", + "url": "20251031/cpython-{python_version}+20251031-{platform}-{build}.tar.gz", "sha256": { - "aarch64-apple-darwin": "6ceba34fe78802853a30bde6f303a0a54f71f6ab07a673da34e90c0aa06c786e", - "aarch64-unknown-linux-gnu": "d32487b853d6f5709019a471770be5e5d3e6bd2ac507e5629e2d6825565d3e71", - "aarch64-pc-windows-msvc": "d708734581e8cb03f4cf95f39f17ea331bc4761dfdad99b6b738a245444c9c54", - "ppc64le-unknown-linux-gnu": "951d2d4fb4d6bee3e9e100c06215cd7621ef9b4e70651870b1efb9e14caa3dd0", - "riscv64-unknown-linux-gnu": "b0b5e1d48cc5d1612a316bd59dc6179efad9644affff41a0820c4791151bb802", - "s390x-unknown-linux-gnu": "e290368e5d0f1e393733f26f4d05f666b36140c38b83b8e66182756940a396de", - "x86_64-apple-darwin": "9b8589eefb153cbe7cb652993d0ecc94aeb2fa13c1a2e8bc240f5f74f23bb21b", - "x86_64-pc-windows-msvc": "2d670beb3b930d30e3a13cc909923a001dbdfcb5537692d5da40b6b41643ce1c", - "x86_64-unknown-linux-gnu": "1ab2b6594d1c3d76cbebea09d6bc3e6ba68d8eb3b6322080375c4cc3dd188f34", - "x86_64-unknown-linux-musl": "d3395c3267617f49363f9114999685b865b2731804e3954e89b681254f62da4c", + "aarch64-apple-darwin": "5e110cb821d2eb8246065d3b46faa655180c976c4e17250f7883c634a629bc63", + "aarch64-unknown-linux-gnu": "81b644d166e0bfb918615af8a2363f8fcf26eccdcc60a5334b6a62c088470bac", + "aarch64-pc-windows-msvc": "b190fed7c2b0f6e1010f554a0d1fd191c0754c4c0718e69d9d795ae559613780", + "ppc64le-unknown-linux-gnu": "024f5e5678c9768d45cc24d37a8e9d265aae86c4a4602352dee3d7deba367052", + "riscv64-unknown-linux-gnu": "b13c57fc372c131e667a99b9680f41c0b4da571cf99ed412103c2fe9ad5ed1fb", + "s390x-unknown-linux-gnu": "2bf05bdd56cdf5ea4fd9f2faf151ea4211be96a0d1f4230b85f5dcae620d6400", + "x86_64-apple-darwin": "687052a046d33be49dc95dd671816709067cf6176ed36c93ea61b1fe0b883b0f", + "x86_64-pc-windows-msvc": "cff398b3f520c442a1b085dd347126c10c1b03f01ccc0decd8c897a687e893f1", + "x86_64-unknown-linux-gnu": "80c3882f14e15cef8260ef5257d198e8f4371ca265887431d939e0d561de3253", + "x86_64-unknown-linux-musl": "0a461330b9b89f2ea3088dde10d7a3f96aa65897b7c5ce2404fa3b5c4b8daa14", }, "strip_prefix": "python", }, @@ -873,27 +734,27 @@ TOOL_VERSIONS = { }, }, "3.13.9": { - "url": "20251014/cpython-{python_version}+20251014-{platform}-{build}.{ext}", + "url": "20251031/cpython-{python_version}+20251031-{platform}-{build}.{ext}", "sha256": { - "aarch64-apple-darwin": "931db8f735e18700d4eab9ee39dbbd0b4c114d7d039dd2707b2d932ded039698", - "aarch64-unknown-linux-gnu": "c86606a45fb6540b1b66d9c52c6f5466fba8affb29acb9ab6a0b7f5ad54e588a", - "ppc64le-unknown-linux-gnu": "80218541bb73f7ccf7fe82660b403b6c35edfe91fc58052392f738a94dbd27ae", - "riscv64-unknown-linux-gnu": "8b210482f6fc46ae2b75fa21ba2e8edce3d11e5c27aa5d841acfc6b95778edb6", - "s390x-unknown-linux-gnu": "8e8cc90192da6ae59c6f26a084fb6f63ef228686643aad983f6183184881babd", - "x86_64-apple-darwin": "9f6bc3c15e2f9e2c9c90db2c8b3ee94598e777789f8aea6e36b69ae55d007d01", - "x86_64-pc-windows-msvc": "8b0efc2674bb293ce2d423d59765b1ca3a2d80dc0ca6168f6279cb569e72b55e", - "aarch64-pc-windows-msvc": "d4de66a7ad3f7c9acaf2db41148097f303985ff7f712795d436d21550ab5ff76", - "aarch64-pc-windows-msvc-freethreaded": "9510f4f9790aa800e6e1163eea450523a5be47a348051b31365a685143b3e17e", - "x86_64-unknown-linux-gnu": "b4b0204658930337c85c321b49ed2585fe544097a72bc76dcf0b77e49fff8473", - "x86_64-unknown-linux-musl": "1e227f10d59c197111c3cea81e352b9f13a136f44cf7bae87368987c48127055", - "aarch64-apple-darwin-freethreaded": "9e78bb28a4ef9d8195caa08586ded2468d575814af6806a9c34fe175614fd3c9", - "aarch64-unknown-linux-gnu-freethreaded": "3f13ad9d0f026e1c0cefe13415b0b965eff3a91c43a7e0c63d8f26fde2382f86", - "ppc64le-unknown-linux-gnu-freethreaded": "323a197e31c966f144bd0e94d8f8c0ee20775190f8b2a91efb191c612d6e94cb", - "riscv64-unknown-linux-gnu-freethreaded": "62d7dbd8ff4c64aeca2aa895c46ab0102433b44bf31b7971d48e3655e9d94688", - "s390x-unknown-linux-gnu-freethreaded": "4b54fe09739628b97aece3231f2ed4e2553ee0b41d0921dfef81fe50968f9afd", - "x86_64-apple-darwin-freethreaded": "405bbf1e443d12e48959ffc7c32674468226dff2c163b75f486686af9f8f7be4", - "x86_64-pc-windows-msvc-freethreaded": "50c5830e814eb057fed984b15dad250c62fda2e54a18ee9789ee2ba89e1951af", - "x86_64-unknown-linux-gnu-freethreaded": "515b92ab30010596ab239dad848c88af88703a054a04b70b5cf0ad22f107c75e", + "aarch64-apple-darwin": "1f3568d17383426d52350c2ef7c93c1a5a043198b860cb05e5d19b35f9c25cef", + "aarch64-unknown-linux-gnu": "0a56d11b0fb1662e67f892b9d5d1717aef06f24dbb8362bc25b8f784e620d44e", + "ppc64le-unknown-linux-gnu": "99492123902bd5e9a6b1a30135061e93a2e6a11d25107a741d5a756e91054448", + "riscv64-unknown-linux-gnu": "b3dce3e4ef508773521e1ee1be989fff6118f8fd1fbbd0491d7ff7dfbc98ef06", + "s390x-unknown-linux-gnu": "f10e34aaa856c1b8a69c2ea4a9a6723d520443d1a957bf66dc55491334ca0c1e", + "x86_64-apple-darwin": "48c0f3ca5d31e90658ef99138dc21865bb62f388ab97a1ce72cac176da194ab0", + "x86_64-pc-windows-msvc": "874593f641f31ea101440c70f81768c35d4d7d6df111fde63094db67465ef787", + "aarch64-pc-windows-msvc": "20db43873d3c4c2175d866806545e4ad4ec6bb72ca95e60082a4df6c24567e8c", + "aarch64-pc-windows-msvc-freethreaded": "743ff69935ef28834621647dab30f032dfcd80315732917531eea333210941c7", + "x86_64-unknown-linux-gnu": "6f05b91ee8c7e6dd0f9c60b95bb29130e2d623961de6578b643e80ddd83f96b6", + "x86_64-unknown-linux-musl": "ad987197034185e628715da504a50613af213dc21ba6d5ccaeab3db2c464aa6c", + "aarch64-apple-darwin-freethreaded": "eae1272a72ccce601590a10a9ca2a58199b5fcdf022aa603a527e3e2a04de9bc", + "aarch64-unknown-linux-gnu-freethreaded": "a6e72f9de5d9b46cf6968d6a492f2401a919f9b959f8da2d87f43484b80169ee", + "ppc64le-unknown-linux-gnu-freethreaded": "0ed5c65437f875c58ba1bee2b8d261d18698d3d0347a2e66f8902fce022a2cda", + "riscv64-unknown-linux-gnu-freethreaded": "584e481d9b5225ffaf02f158fb26d2818207e65fc3c6dc21a6d500277f739220", + "s390x-unknown-linux-gnu-freethreaded": "7fa7fb912ca989ceac026a332d56a2c7d6d16ab0e94d89e690de5aade26103e2", + "x86_64-apple-darwin-freethreaded": "e2bf5fa6a3ef443ade362e08b0a19bbc172f7bfe34dabe933ccaad31d53af5da", + "x86_64-pc-windows-msvc-freethreaded": "318a9a1e43dd52054327de3bccc0c5b7afde7b7f2a398ccb4d38e03d28b05386", + "x86_64-unknown-linux-gnu-freethreaded": "dcc29b069d0588fbd4ea29c6df840c8d1207d2a3bce8cd5cd57d1b85373b6048", }, "strip_prefix": { "aarch64-apple-darwin": "python", @@ -918,27 +779,27 @@ TOOL_VERSIONS = { }, }, "3.14.0": { - "url": "20251014/cpython-{python_version}+20251014-{platform}-{build}.{ext}", + "url": "20251031/cpython-{python_version}+20251031-{platform}-{build}.{ext}", "sha256": { - "aarch64-apple-darwin": "1333ce2807fbea673eb242edbf4997ea1e2f6cbc01cd80dec1f9d19de2cd63ed", - "aarch64-unknown-linux-gnu": "e613f44e60227b3423a994698426698569e055c24447c10dd9c1c022cf511f05", - "ppc64le-unknown-linux-gnu": "91d164d5480015c7e6c441255cf4bcd182e0c4124e028716e58e1efef6418936", - "riscv64-unknown-linux-gnu": "11c9807fc52bae34a81ba8bd7cb35f5360be428b867377710a69c793e3917725", - "s390x-unknown-linux-gnu": "2e35106929e6f5a8d568f890522cdc7ad4382f696ad266537035753fb4916626", - "x86_64-apple-darwin": "0a4cc33ca56830b92545950aacdde8925c9d4259e4f00ceda04fedf853f70679", - "x86_64-pc-windows-msvc": "d90e97fe69b819f0a776cd665d06fef6526a4259211d11f00e501688659f1c0e", - "aarch64-pc-windows-msvc": "1359d52eaa584da8a76decbe4255f89dae47d81757b9f422b91467824ecfdd7f", - "x86_64-unknown-linux-gnu": "74d4516a64abc63ae4bcbffb35482879a85b7faa187fcfa47c1ca8f00faebf5f", - "x86_64-unknown-linux-musl": "7e603f71788edb5e6a4d92273eafb4d609972cd45f330032b6728c0d9753c37e", - "aarch64-apple-darwin-freethreaded": "1c61fa9c9979cfe74f992dd2f15cfed644ee9feec78e12c894ae446044186f74", - "aarch64-unknown-linux-gnu-freethreaded": "c2c5e0be76d7151b6a1c0fdd4ef58e0b81d36902580311c1c8c2b4b075ed3190", - "ppc64le-unknown-linux-gnu-freethreaded": "a1b9dc2130017208b04551a3b7e502e740691e0987f911ed392c8e6d77d611f9", - "riscv64-unknown-linux-gnu-freethreaded": "099f8e056f17f09dcb137c15aa162fe390055d4ad17d2115a8f7adbb4e768ec1", - "s390x-unknown-linux-gnu-freethreaded": "8e71db4558557315e56051fb59a0b2ed2701f7f9240339f867dc7b6440a72209", - "x86_64-apple-darwin-freethreaded": "a57a872d96f2711909181cf7da7b7a86e3bec293621ac54de3f89ea8d3fbb3bd", - "x86_64-pc-windows-msvc-freethreaded": "730449333b24fae53ce6872d8ade13564773f1fc652f926ca641a6a228e71dd6", - "aarch64-pc-windows-msvc-freethreaded": "dd062cff01d7c55c2fc5596c387423420155bafd03dae5fa9ecd66ee89df6695", - "x86_64-unknown-linux-gnu-freethreaded": "56ef2dbc787a0f75d63ab38b4f6b1a0b1f35ce1f710b68e8080aee9d6c1c7453", + "aarch64-apple-darwin": "b4bcd3c6c24cab32ae99e1b05c89312b783b4d69431d702e5012fe1fdcad4087", + "aarch64-unknown-linux-gnu": "128a9cbfb9645d5237ec01704d9d1d2ac5f084464cc43c37a4cd96aa9c3b1ad5", + "ppc64le-unknown-linux-gnu": "e16ca51f018e99a609faf953bd3a3aea31f45ee84262d1a517fb3abd98f1f4af", + "riscv64-unknown-linux-gnu": "fca340d8fb7a05cd90e216ce601b25d492ed8c1a3b6a6d77703e0f15ab3711a7", + "s390x-unknown-linux-gnu": "c5803644970eee931bb0581b3b64511d1a8612f67bc98951a7f7ab5581a9ed04", + "x86_64-apple-darwin": "4e71a3ce973be377ef18637826648bb936e2f9490f64a9e4f33a49bcc431d344", + "x86_64-pc-windows-msvc": "39acfcb3857d83eab054a3de11756ffc16b3d49c31393b9800dd2704d1f07fdf", + "aarch64-pc-windows-msvc": "599a8b7e12439cd95a201dbdfe95cf363146b1ff91f379555dafd86b170caab9", + "x86_64-unknown-linux-gnu": "3dec1ab70758a3467ac3313bbcdabf7a9b3016db5c072c4537e3cf0a9e6290f6", + "x86_64-unknown-linux-musl": "d0a2a6d3b1bb00dce2105377fda8aa79675d187f8d6d7010a42f651af25018dc", + "aarch64-apple-darwin-freethreaded": "d9c7b430b25bd3837dbb03f945dbe6b7bc526c5940ca96f5db7cdc42f6b2b801", + "aarch64-unknown-linux-gnu-freethreaded": "f383ef50d1da6ca511212e5ae601923b56636b87351fd5fc847e0ea0a19fa9b3", + "ppc64le-unknown-linux-gnu-freethreaded": "cb0e4ff781b856a47f0f461ceb41c78c7eeff65effd0957857ec4702ef1e1bd3", + "riscv64-unknown-linux-gnu-freethreaded": "929223470d11a55cd75f880ac3bd4969e42407e2cdf08d4e7e38ba721cf4abec", + "s390x-unknown-linux-gnu-freethreaded": "613fb1f7b249f798b52af957d181305244e936c8e5c94c84688fcdf93fe14253", + "x86_64-apple-darwin-freethreaded": "b3196f6b57bbb3dc2ee07f348f1d51117ffa376979eceafbf50c15f0f7980bf8", + "x86_64-pc-windows-msvc-freethreaded": "b81de5fc9e783ea6dfcf1098c28a278c874999c71afbb0309f6a8b4276c769d0", + "aarch64-pc-windows-msvc-freethreaded": "40266e60f655e49cd1d5303295255909a4b593b08b88be6e6a55b2c9fe6ed13d", + "x86_64-unknown-linux-gnu-freethreaded": "f4acbef0fbfaf7ab31ac63986da1d93dfa1c5cb797de1dcdc1a988aa18670120", }, "strip_prefix": { "aarch64-apple-darwin": "python", @@ -963,27 +824,27 @@ TOOL_VERSIONS = { }, }, "3.15.0a1": { - "url": "20251014/cpython-{python_version}+20251014-{platform}-{build}.{ext}", + "url": "20251031/cpython-{python_version}+20251031-{platform}-{build}.{ext}", "sha256": { - "aarch64-apple-darwin": "b17d1c8dd0ee32004124345a1944891a3e11c3549c0c2575c192e785dc0ca452", - "aarch64-unknown-linux-gnu": "5b82e1cd640e6249794de367e6154682836a6919ea96b9b15309a624d293724d", - "ppc64le-unknown-linux-gnu": "68bc72f8f960d497002035f0ecfa5b22d866467e1d11e2bc56c441b3f63d50a5", - "riscv64-unknown-linux-gnu": "b535892e6f7f28856802d235198044facd129c4031ad8b1d2a00952e5b7f1c00", - "s390x-unknown-linux-gnu": "6a82a1b0490c5bca2ec69f0accead17bf86f60ac5de90335bf68d942e87e6bc3", - "x86_64-apple-darwin": "a8cddf0b4974be662dc157364360606af66ffe56d5b95e6b6c9d06e76b8cad16", - "x86_64-pc-windows-msvc": "3a9bea65091cbbd4b6db1ecbd99ca4da8d0ffe32360b953345f24eaba4a89fc8", - "aarch64-pc-windows-msvc": "9ed03d369562bfd6900dcc5b503193355388ab0c1c93268a68671ca5b6e8ae2e", - "x86_64-unknown-linux-gnu": "5fb9150d98c4e4d153bee6e5e5626882901b77d00e1fb7e481f4aa36d4b57c8d", - "x86_64-unknown-linux-musl": "f79b24cd6c9952c43f16d7f1812ecb99b5339385b479d60975f96eb212033f3e", - "aarch64-apple-darwin-freethreaded": "c66c98b7257f568510a8a988fa22a369ddb4fd2b031768a9e65aca43a3dc575a", - "aarch64-unknown-linux-gnu-freethreaded": "3a78b661e488e2e1fd9b614901af659bce295c9eee307313636bb358b8f11b6e", - "ppc64le-unknown-linux-gnu-freethreaded": "3a1b7c0e9c055ed4fc26d1029fc262ece9947db9e7346e7bf354b18f4ba7b9f1", - "riscv64-unknown-linux-gnu-freethreaded": "c0a3ff7053bf98398bd9595bc04b04e4c2dd03eaae607f57a74541abac494edc", - "s390x-unknown-linux-gnu-freethreaded": "95f337e46e0ec5266b00ff93235675cfdf88da583168da277f7dc67804268926", - "x86_64-apple-darwin-freethreaded": "6512751c57469ccdba1309f90a756feb6704b3af03ceeef2a1ec8e8f4a30554d", - "x86_64-pc-windows-msvc-freethreaded": "60636fc054223d3f83c387dccd084933fc5ab4d7182d9e47df05469d1ba05595", - "aarch64-pc-windows-msvc-freethreaded": "d624349224906d1653fb1c5338a931492767d21436530941172116209848d62d", - "x86_64-unknown-linux-gnu-freethreaded": "003c2125829b0859b1cbce74351f089963eec33ad071f89befb9a373005e8a24", + "aarch64-apple-darwin": "3acf7aa3559b746498b18929456c5cacb84bae4e09249834cbc818970d71de87", + "aarch64-unknown-linux-gnu": "d55c2aeece827e6bec83fd18515ee281d9ea0efaa3e2d20130db8f1c7cbb71c6", + "ppc64le-unknown-linux-gnu": "c28beda791c499b16f06256339522f0002a3e9acba003e6b8374755d7be1def2", + "riscv64-unknown-linux-gnu": "36619f576b8154e4b56643c5c4a85c352f152df2989c4e602cbbe9c2b7ded870", + "s390x-unknown-linux-gnu": "5ea47be2a3a563ddd87ff510dae26b7aa7f3855ca00c5f1056ff8114c067c4e4", + "x86_64-apple-darwin": "0ab19d3ac25f99da438b088751e5ec2421f9f6aa4292fd2dc0f8e49eb3e16bdf", + "x86_64-pc-windows-msvc": "5f5d6bec2b381cfc771c49972d2a6f7b7e7ab6a1651d8fb6ef3983f3571722b3", + "aarch64-pc-windows-msvc": "1508bcd7195008479ed156aad3afbb3a3793097ed530690f0304a8107f0e53e8", + "x86_64-unknown-linux-gnu": "1f356288c2b2713619cb7a4e453d33bf8882f812af2987e21e01e7ae382fefba", + "x86_64-unknown-linux-musl": "caf5311f333eef082dd69a669ca65aceba09a08fc1e78aad602ad649106f294c", + "aarch64-apple-darwin-freethreaded": "12f1b16be4017181ad67904caf9e59e525b9b5d62f49105017d837e27b832959", + "aarch64-unknown-linux-gnu-freethreaded": "981fe8dfc6e7e1d0ffefa945a18d5c4c759bbe21722acf3a5cc7e62f16aa5f3c", + "ppc64le-unknown-linux-gnu-freethreaded": "088400dec25139f38eeecb48f090ff2ce06a96a1dd79fa8f1dfec1cd1786f5ef", + "riscv64-unknown-linux-gnu-freethreaded": "938061a0a31a06672526885de36037ddefd8c4acdb09424691b7000a8c8f8d01", + "s390x-unknown-linux-gnu-freethreaded": "2003e7e40bb44b3db7bca81087bfb738fe6af40e5db61cda8e23b59bf55d409e", + "x86_64-apple-darwin-freethreaded": "64fc29e6c7a2f02a18645d968f1b3fc1d00d12a5ef3fcbb0d077fa8c62c08904", + "x86_64-pc-windows-msvc-freethreaded": "34abc5603e1b4131f753d29b7deac865b9277912b851cbed5a149cf3e6745d3d", + "aarch64-pc-windows-msvc-freethreaded": "54ca78dae455ece6fefbd7f5f287cc55d5ce197caf51921f6d871d15069d9489", + "x86_64-unknown-linux-gnu-freethreaded": "0e0272186d9f5169394dbc4d4d72a3f4a5762a04c2e5ac2ab1e23aa41fc8538a", }, "strip_prefix": { "aarch64-apple-darwin": "python", @@ -1012,7 +873,7 @@ TOOL_VERSIONS = { # buildifier: disable=unsorted-dict-items MINOR_MAPPING = { "3.8": "3.8.20", - "3.9": "3.9.24", + "3.9": "3.9.25", "3.10": "3.10.19", "3.11": "3.11.14", "3.12": "3.12.12", From 49b66308af877661f318d2f7b22f5c59847695eb Mon Sep 17 00:00:00 2001 From: Joel Sing Date: Thu, 6 Nov 2025 12:43:19 +1100 Subject: [PATCH 507/922] fix: make CI pass with the next version of Bazel (9.0.0rc1) (#3393) Explicitly load CcToolchainConfigInfo in the fake cc toolchain config. For examples/bzlmod, bump rules_rust to a newer version so that it pulls in dependencies that work correctly with Bazel 9.0.0rc1. Work towards #3392 --- examples/bzlmod/MODULE.bazel | 2 +- tests/support/cc_toolchains/fake_cc_toolchain_config.bzl | 1 + 2 files changed, 2 insertions(+), 1 deletion(-) diff --git a/examples/bzlmod/MODULE.bazel b/examples/bzlmod/MODULE.bazel index 0a505aa5e4..f5311fb4d1 100644 --- a/examples/bzlmod/MODULE.bazel +++ b/examples/bzlmod/MODULE.bazel @@ -23,7 +23,7 @@ bazel_dep(name = "rules_java", version = "8.3.1") # MODULE.bazel.lock is cross-platform friendly, and there are transitive # dependencies on rules_rust, so we need rules_rust 0.54.1+ where such issues # were fixed. -bazel_dep(name = "rules_rust", version = "0.54.1") +bazel_dep(name = "rules_rust", version = "0.67.0") # We next initialize the python toolchain using the extension. # You can set different Python versions in this block. diff --git a/tests/support/cc_toolchains/fake_cc_toolchain_config.bzl b/tests/support/cc_toolchains/fake_cc_toolchain_config.bzl index 8240f09e04..b70c79cfd8 100644 --- a/tests/support/cc_toolchains/fake_cc_toolchain_config.bzl +++ b/tests/support/cc_toolchains/fake_cc_toolchain_config.bzl @@ -15,6 +15,7 @@ """Fake for providing CcToolchainConfigInfo.""" load("@rules_cc//cc/common:cc_common.bzl", "cc_common") +load("@rules_cc//cc/toolchains:cc_toolchain_config_info.bzl", "CcToolchainConfigInfo") def _impl(ctx): return cc_common.create_cc_toolchain_config_info( From c181f93c9f473cf73bb1522720d12280d8f46b2c Mon Sep 17 00:00:00 2001 From: Yun Peng Date: Thu, 6 Nov 2025 02:54:56 +0100 Subject: [PATCH 508/922] Add batch_commands to presubmit.yml (#3389) Fix bcr presubmit build on windows --- .bcr/gazelle/presubmit.yml | 2 ++ 1 file changed, 2 insertions(+) diff --git a/.bcr/gazelle/presubmit.yml b/.bcr/gazelle/presubmit.yml index ff1c9e7d58..e4db2a8a93 100644 --- a/.bcr/gazelle/presubmit.yml +++ b/.bcr/gazelle/presubmit.yml @@ -25,6 +25,8 @@ bcr_test_module: bazel: ${{ bazel }} shell_commands: - "echo 'common --override_module=rules_python=' >> .bazelrc" + batch_commands: + - "echo common --override_module=rules_python= >> .bazelrc" build_targets: - "//..." - ":modules_map" From a506d774dd60eedf6fce94dcccd2c233553b06d4 Mon Sep 17 00:00:00 2001 From: Joel Sing Date: Thu, 6 Nov 2025 15:35:38 +1100 Subject: [PATCH 509/922] fix(runfiles): correct Python runfiles path assumption (#3086) The current _FindPythonRunfilesRoot() implementation assumes that the Python module has been unpacked four levels below the runfiles directory. This is not the case in multiple situations, for example when rules_pycross is in use and has installed the module via pypi (in which case it is five levels below runfiles). Both strategies already know where the runfiles directory exists - implement _GetRunfilesDir() on the _DirectoryBased strategy, then call _GetRunfilesDir() in order to populate self._python_runfiles_dir. Stop passing a bogus path to runfiles.Create() in testCurrentRepository(), such that the test actually uses the appropriate runfiles path. Fixes #3085 --------- Co-authored-by: Ignas Anikevicius <240938+aignas@users.noreply.github.com> --- CHANGELOG.md | 4 ++++ python/runfiles/runfiles.py | 17 ++++------------- tests/runfiles/BUILD.bazel | 3 +++ tests/runfiles/runfiles_test.py | 14 +++++++++++++- .../toolchain_runs_test.py | 16 +++++++++++++--- 5 files changed, 37 insertions(+), 17 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index bae98230e0..9c6a6b62c8 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -42,6 +42,10 @@ BEGIN_UNRELEASED_TEMPLATE ### Fixed * (gazelle) Remove {obj}`py_binary` targets with invalid `srcs`. This includes files that are not generated or regular files. +* (runfiles) Fix incorrect Python runfiles path assumption - the existing + implementation assumes that it is always four levels below the runfiles + directory, leading to incorrect path checks + ([#3085](https://github.com/bazel-contrib/rules_python/issues/3085)). {#v0-0-0-added} ### Added diff --git a/python/runfiles/runfiles.py b/python/runfiles/runfiles.py index fc794272c9..bfa9d0d053 100644 --- a/python/runfiles/runfiles.py +++ b/python/runfiles/runfiles.py @@ -229,6 +229,9 @@ def RlocationChecked(self, path: str) -> str: # runfiles strategy on those platforms. return posixpath.join(self._runfiles_root, path) + def _GetRunfilesDir(self) -> str: + return self._runfiles_root + def EnvVars(self) -> Dict[str, str]: return { "RUNFILES_DIR": self._runfiles_root, @@ -246,7 +249,7 @@ class Runfiles: def __init__(self, strategy: Union[_ManifestBased, _DirectoryBased]) -> None: self._strategy = strategy - self._python_runfiles_root = _FindPythonRunfilesRoot() + self._python_runfiles_root = strategy._GetRunfilesDir() self._repo_mapping = _RepositoryMapping.create_from_file( strategy.RlocationChecked("_repo_mapping") ) @@ -469,18 +472,6 @@ def Create(env: Optional[Dict[str, str]] = None) -> Optional["Runfiles"]: _Runfiles = Runfiles -def _FindPythonRunfilesRoot() -> str: - """Finds the root of the Python runfiles tree.""" - root = __file__ - # Walk up our own runfiles path to the root of the runfiles tree from which - # the current file is being run. This path coincides with what the Bazel - # Python stub sets up as sys.path[0]. Since that entry can be changed at - # runtime, we rederive it here. - for _ in range("rules_python/python/runfiles/runfiles.py".count("/") + 1): - root = os.path.dirname(root) - return root - - def CreateManifestBased(manifest_path: str) -> Runfiles: return Runfiles.CreateManifestBased(manifest_path) diff --git a/tests/runfiles/BUILD.bazel b/tests/runfiles/BUILD.bazel index 5c92026082..84602d2bd6 100644 --- a/tests/runfiles/BUILD.bazel +++ b/tests/runfiles/BUILD.bazel @@ -5,6 +5,9 @@ load("@rules_python//python/private:bzlmod_enabled.bzl", "BZLMOD_ENABLED") # bu py_test( name = "runfiles_test", srcs = ["runfiles_test.py"], + data = [ + "//tests/support:current_build_settings", + ], env = { "BZLMOD_ENABLED": "1" if BZLMOD_ENABLED else "0", }, diff --git a/tests/runfiles/runfiles_test.py b/tests/runfiles/runfiles_test.py index b8a3d5f7b7..165ab8c8a9 100644 --- a/tests/runfiles/runfiles_test.py +++ b/tests/runfiles/runfiles_test.py @@ -12,7 +12,9 @@ # See the License for the specific language governing permissions and # limitations under the License. +import json import os +import pathlib import tempfile import unittest from typing import Any, List, Optional @@ -63,6 +65,16 @@ def testRlocationArgumentValidation(self) -> None: lambda: r.Rlocation("\\foo"), ) + def testRlocationWithData(self) -> None: + r = runfiles.Create() + assert r is not None # mypy doesn't understand the unittest api. + settings_path = r.Rlocation( + "rules_python/tests/support/current_build_settings.json" + ) + assert settings_path is not None + settings = json.loads(pathlib.Path(settings_path).read_text()) + self.assertIn("bootstrap_impl", settings) + def testCreatesManifestBasedRunfiles(self) -> None: with _MockFile(contents=["a/b c/d"]) as mf: r = runfiles.Create( @@ -692,7 +704,7 @@ def testCurrentRepository(self) -> None: expected = "" else: expected = "rules_python" - r = runfiles.Create({"RUNFILES_DIR": "whatever"}) + r = runfiles.Create() assert r is not None # mypy doesn't understand the unittest api. self.assertEqual(r.CurrentRepository(), expected) diff --git a/tests/runtime_env_toolchain/toolchain_runs_test.py b/tests/runtime_env_toolchain/toolchain_runs_test.py index c66b0bbd8a..6f0948feff 100644 --- a/tests/runtime_env_toolchain/toolchain_runs_test.py +++ b/tests/runtime_env_toolchain/toolchain_runs_test.py @@ -10,10 +10,20 @@ class RunTest(unittest.TestCase): def test_ran(self): rf = runfiles.Create() - settings_path = rf.Rlocation( - "rules_python/tests/support/current_build_settings.json" - ) + try: + settings_path = rf.Rlocation( + "rules_python/tests/support/current_build_settings.json" + ) + except ValueError as e: + # The current toolchain being used has a buggy zip file bootstrap, which + # leaves RUNFILES_DIR pointing at the first stage path and not the module + # path. + if platform.system() != "Windows" or "does not lie under the runfiles root" not in str(e): + raise e + settings_path = "./tests/support/current_build_settings.json" + settings = json.loads(pathlib.Path(settings_path).read_text()) + if platform.system() == "Windows": self.assertEqual( "/_magic_pyruntime_sentinel_do_not_use", settings["interpreter_path"] From 3241428be75d3ec94d1a3cb80908b415eeea7051 Mon Sep 17 00:00:00 2001 From: Alex Eagle Date: Wed, 5 Nov 2025 20:45:04 -0800 Subject: [PATCH 510/922] chore: Bazel 9 is rc now (#3394) Work towards #3392 --------- Co-authored-by: Ignas Anikevicius <240938+aignas@users.noreply.github.com> --- MODULE.bazel | 6 ++---- tests/integration/BUILD.bazel | 23 ++++++++++++----------- 2 files changed, 14 insertions(+), 15 deletions(-) diff --git a/MODULE.bazel b/MODULE.bazel index 5854595bed..6e9b725c53 100644 --- a/MODULE.bazel +++ b/MODULE.bazel @@ -359,10 +359,7 @@ bazel_binaries.local( ) bazel_binaries.download(version = "7.4.1") bazel_binaries.download(version = "8.0.0") - -# For now, don't test with rolling, because that's Bazel 9, which is a ways -# away. -# bazel_binaries.download(version = "rolling") +bazel_binaries.download(version = "9.0.0rc1") use_repo( bazel_binaries, "bazel_binaries", @@ -371,6 +368,7 @@ use_repo( "bazel_binaries_bazelisk", "build_bazel_bazel_7_4_1", "build_bazel_bazel_8_0_0", + "build_bazel_bazel_9_0_0rc1", # "build_bazel_bazel_rolling", "build_bazel_bazel_self", ) diff --git a/tests/integration/BUILD.bazel b/tests/integration/BUILD.bazel index df7fe15444..0e47faf91a 100644 --- a/tests/integration/BUILD.bazel +++ b/tests/integration/BUILD.bazel @@ -12,7 +12,6 @@ # See the License for the specific language governing permissions and # limitations under the License. -load("@bazel_binaries//:defs.bzl", "bazel_binaries") load("@rules_bazel_integration_test//bazel_integration_test:defs.bzl", "default_test_runner") load("//python:py_library.bzl", "py_library") load(":integration_test.bzl", "rules_python_integration_test") @@ -78,31 +77,33 @@ rules_python_integration_test( rules_python_integration_test( name = "ignore_root_user_error_test", + env = { + "RULES_PYTHON_BZLMOD_DEBUG": "1", + }, ) rules_python_integration_test( name = "ignore_root_user_error_workspace_test", bzlmod = False, + env = { + "RULES_PYTHON_BZLMOD_DEBUG": "1", + }, workspace_path = "ignore_root_user_error", ) rules_python_integration_test( name = "local_toolchains_test", - bazel_versions = [ - version - for version in bazel_binaries.versions.all - if not version.startswith("6.") - ], + env = { + "RULES_PYTHON_BZLMOD_DEBUG": "1", + }, ) rules_python_integration_test( name = "local_toolchains_workspace_test", - bazel_versions = [ - version - for version in bazel_binaries.versions.all - if not version.startswith("6.") - ], bzlmod = False, + env = { + "RULES_PYTHON_BZLMOD_DEBUG": "1", + }, workspace_path = "local_toolchains", ) From fda82b0a99b8ec85b4aca0cac871fc72581c4636 Mon Sep 17 00:00:00 2001 From: Ignas Anikevicius <240938+aignas@users.noreply.github.com> Date: Mon, 10 Nov 2025 04:43:18 +0900 Subject: [PATCH 511/922] chore: update to latest buildifier (#3386) Upgrade to the latest buildifier and fix an example. --- .bazelci/presubmit.yml | 2 +- .pre-commit-config.yaml | 2 +- examples/multi_python_versions/MODULE.bazel | 1 + examples/pip_parse_vendored/BUILD.bazel | 1 + examples/pip_parse_vendored/WORKSPACE | 16 ++++++++++++++++ 5 files changed, 20 insertions(+), 2 deletions(-) diff --git a/.bazelci/presubmit.yml b/.bazelci/presubmit.yml index 6ed93b083d..1cb3a01365 100644 --- a/.bazelci/presubmit.yml +++ b/.bazelci/presubmit.yml @@ -16,7 +16,7 @@ buildifier: # keep these arguments in sync with .pre-commit-config.yaml # Use a specific version to avoid skew issues when new versions are released. - version: 6.1.0 + version: 8.2.1 warnings: "all" # NOTE: Minimum supported version is 7.x .minimum_supported_version: &minimum_supported_version diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml index 67a02fc6c0..91e449f950 100644 --- a/.pre-commit-config.yaml +++ b/.pre-commit-config.yaml @@ -21,7 +21,7 @@ repos: hooks: - id: check-merge-conflict - repo: https://github.com/keith/pre-commit-buildifier - rev: 6.1.0 + rev: 8.2.1 hooks: - id: buildifier args: &args diff --git a/examples/multi_python_versions/MODULE.bazel b/examples/multi_python_versions/MODULE.bazel index 4e4a0473c2..eeb1dfc83e 100644 --- a/examples/multi_python_versions/MODULE.bazel +++ b/examples/multi_python_versions/MODULE.bazel @@ -35,6 +35,7 @@ use_repo( pip = use_extension("@rules_python//python/extensions:pip.bzl", "pip") use_repo(pip, "pypi") + pip.parse( hub_name = "pypi", python_version = "3.9", diff --git a/examples/pip_parse_vendored/BUILD.bazel b/examples/pip_parse_vendored/BUILD.bazel index 8d81e4ba8b..74b9286359 100644 --- a/examples/pip_parse_vendored/BUILD.bazel +++ b/examples/pip_parse_vendored/BUILD.bazel @@ -3,6 +3,7 @@ load("@bazel_skylib//rules:diff_test.bzl", "diff_test") load("@bazel_skylib//rules:write_file.bzl", "write_file") load("@rules_python//python:pip.bzl", "compile_pip_requirements") load("@rules_python//python:py_test.bzl", "py_test") +load("@rules_shell//shell:sh_binary.bzl", "sh_binary") load("//:requirements.bzl", "all_data_requirements", "all_requirements", "all_whl_requirements", "requirement") # This rule adds a convenient way to update the requirements.txt diff --git a/examples/pip_parse_vendored/WORKSPACE b/examples/pip_parse_vendored/WORKSPACE index d7a11ea596..5e80b4116b 100644 --- a/examples/pip_parse_vendored/WORKSPACE +++ b/examples/pip_parse_vendored/WORKSPACE @@ -39,3 +39,19 @@ pip_parse( load("//:requirements.bzl", "install_deps") install_deps() + +load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive") + +# See https://github.com/bazelbuild/rules_shell/releases/tag/v0.2.0 +http_archive( + name = "rules_shell", + sha256 = "410e8ff32e018b9efd2743507e7595c26e2628567c42224411ff533b57d27c28", + strip_prefix = "rules_shell-0.2.0", + url = "https://github.com/bazelbuild/rules_shell/releases/download/v0.2.0/rules_shell-v0.2.0.tar.gz", +) + +load("@rules_shell//shell:repositories.bzl", "rules_shell_dependencies", "rules_shell_toolchains") + +rules_shell_dependencies() + +rules_shell_toolchains() From bc196f57aa495843053cd059890b76a77e796b63 Mon Sep 17 00:00:00 2001 From: Ignas Anikevicius <240938+aignas@users.noreply.github.com> Date: Mon, 10 Nov 2025 04:43:48 +0900 Subject: [PATCH 512/922] chore: switch bcr to 8.x instead of last_rc (#3395) It seems that we forgot to complete this TODO item. Work towards #3392 --- .bcr/gazelle/presubmit.yml | 3 +-- .bcr/presubmit.yml | 3 +-- 2 files changed, 2 insertions(+), 4 deletions(-) diff --git a/.bcr/gazelle/presubmit.yml b/.bcr/gazelle/presubmit.yml index e4db2a8a93..3300f67f29 100644 --- a/.bcr/gazelle/presubmit.yml +++ b/.bcr/gazelle/presubmit.yml @@ -16,8 +16,7 @@ bcr_test_module: module_path: "examples/bzlmod_build_file_generation" matrix: platform: ["debian11", "macos", "ubuntu2004", "windows"] - # last_rc is to get latest 8.x release. Replace with 8.x when available. - bazel: [7.x, last_rc] + bazel: [7.x, 8.x] tasks: run_tests: name: "Run test module" diff --git a/.bcr/presubmit.yml b/.bcr/presubmit.yml index e1ddb7a1aa..b016dc9d6f 100644 --- a/.bcr/presubmit.yml +++ b/.bcr/presubmit.yml @@ -16,8 +16,7 @@ bcr_test_module: module_path: "examples/bzlmod" matrix: platform: ["debian11", "macos", "ubuntu2004", "windows"] - # last_rc is to get latest 8.x release. Replace with 8.x when available. - bazel: [7.x, last_rc] + bazel: [7.x, 8.x] tasks: run_tests: name: "Run test module" From c037d83aaad33808abc5f1cce5a797ca1550cf0c Mon Sep 17 00:00:00 2001 From: Laramie Leavitt Date: Sun, 9 Nov 2025 12:14:40 -0800 Subject: [PATCH 513/922] fix(local): Fix local_runtime use with free-threaded python (#3399) * Return abi_flags from get_local_runtime_info and pass it into the py3_runtime * Rework how shared-libraries are links are constructed to better meet @rules_cc cc_library.srcs requirements This improves runtime detection for macos when using a python3.14t framework runtime. --------- Co-authored-by: Richard Levasseur Co-authored-by: Richard Levasseur --- CHANGELOG.md | 31 +++++++- python/private/get_local_runtime_info.py | 87 +++++++++++++-------- python/private/local_runtime_repo.bzl | 23 +++--- python/private/local_runtime_repo_setup.bzl | 5 +- 4 files changed, 97 insertions(+), 49 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 9c6a6b62c8..8c472c08d8 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -23,6 +23,31 @@ A brief description of the categories of changes: + + {#v0-0-0} ## Unreleased @@ -46,16 +71,14 @@ BEGIN_UNRELEASED_TEMPLATE implementation assumes that it is always four levels below the runfiles directory, leading to incorrect path checks ([#3085](https://github.com/bazel-contrib/rules_python/issues/3085)). +* (toolchains) local toolchains now tell the `sys.abiflags` value of the + underlying runtime. {#v0-0-0-added} ### Added * (toolchains) `3.9.25` Python toolchain from [20251031] release. [20251031]: https://github.com/astral-sh/python-build-standalone/releases/tag/20251031 - -END_UNRELEASED_TEMPLATE ---> - {#v1-7-0} ## [1.7.0] - 2025-10-11 diff --git a/python/private/get_local_runtime_info.py b/python/private/get_local_runtime_info.py index d176b1a7c6..b20c159cfc 100644 --- a/python/private/get_local_runtime_info.py +++ b/python/private/get_local_runtime_info.py @@ -11,7 +11,6 @@ # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # See the License for the specific language governing permissions and # limitations under the License. - """Returns information about the local Python runtime as JSON.""" import json @@ -38,7 +37,9 @@ def _search_directories(get_config, base_executable): # On MacOS, the LDLIBRARY may be a relative path under /Library/Frameworks, # such as "Python.framework/Versions/3.12/Python", not a file under the # LIBDIR/LIBPL directory, so include PYTHONFRAMEWORKPREFIX. - lib_dirs = [get_config(x) for x in ("PYTHONFRAMEWORKPREFIX", "LIBPL", "LIBDIR")] + lib_dirs = [ + get_config(x) for x in ("PYTHONFRAMEWORKPREFIX", "LIBPL", "LIBDIR") + ] # On Debian, with multiarch enabled, prior to Python 3.10, `LIBDIR` didn't # tell the location of the libs, just the base directory. The `MULTIARCH` @@ -55,8 +56,8 @@ def _search_directories(get_config, base_executable): if not _IS_DARWIN: for exec_dir in ( - os.path.dirname(base_executable) if base_executable else None, - get_config("BINDIR"), + os.path.dirname(base_executable) if base_executable else None, + get_config("BINDIR"), ): if not exec_dir: continue @@ -67,8 +68,8 @@ def _search_directories(get_config, base_executable): lib_dirs.append(os.path.join(exec_dir, "lib")) lib_dirs.append(os.path.join(exec_dir, "libs")) else: - # On most systems the executable is in a bin/ directory and the libraries - # are in a sibling lib/ directory. + # On most non-windows systems the executable is in a bin/ directory and + # the libraries are in a sibling lib/ directory. lib_dirs.append(os.path.join(os.path.dirname(exec_dir), "lib")) # Dedup and remove empty values, keeping the order. @@ -76,7 +77,19 @@ def _search_directories(get_config, base_executable): return {k: None for k in lib_dirs}.keys() -def _search_library_names(get_config): +def _get_shlib_suffix(get_config) -> str: + """Returns the suffix for shared libraries.""" + if _IS_DARWIN: + return ".dylib" + if _IS_WINDOWS: + return ".dll" + suffix = get_config("SHLIB_SUFFIX") + if not suffix: + suffix = ".so" + return suffix + + +def _search_library_names(get_config, shlib_suffix): """Returns a list of library files to search for shared libraries.""" # Quoting configure.ac in the cpython code base: # "INSTSONAME is the name of the shared library that will be use to install @@ -90,8 +103,7 @@ def _search_library_names(get_config): # # A typical LIBRARY is 'libpythonX.Y.a' on Linux. lib_names = [ - get_config(x) - for x in ( + get_config(x) for x in ( "LDLIBRARY", "INSTSONAME", "PY3LIBRARY", @@ -104,26 +116,24 @@ def _search_library_names(get_config): # The suffix and version are set here to the default values for the OS, # since they are used below to construct "default" library names. if _IS_DARWIN: - suffix = ".dylib" prefix = "lib" elif _IS_WINDOWS: - suffix = ".dll" prefix = "" else: - suffix = get_config("SHLIB_SUFFIX") prefix = "lib" - if not suffix: - suffix = ".so" version = get_config("VERSION") # Ensure that the pythonXY.dll files are included in the search. - lib_names.append(f"{prefix}python{version}{suffix}") + lib_names.append(f"{prefix}python{version}{shlib_suffix}") # If there are ABIFLAGS, also add them to the python version lib search. abiflags = get_config("ABIFLAGS") or get_config("abiflags") or "" if abiflags: - lib_names.append(f"{prefix}python{version}{abiflags}{suffix}") + lib_names.append(f"{prefix}python{version}{abiflags}{shlib_suffix}") + + # Add the abi-version includes to the search list. + lib_names.append(f"{prefix}python{sys.version_info.major}{shlib_suffix}") # Dedup and remove empty values, keeping the order. lib_names = [v for v in lib_names if v] @@ -138,30 +148,31 @@ def _get_python_library_info(base_executable): # construct library paths such as python3.12, so ensure it exists. if not config_vars.get("VERSION"): if sys.platform == "win32": - config_vars["VERSION"] = f"{sys.version_info.major}{sys.version_info.minor}" + config_vars["VERSION"] = ( + f"{sys.version_info.major}{sys.version_info.minor}") else: config_vars["VERSION"] = ( - f"{sys.version_info.major}.{sys.version_info.minor}" - ) + f"{sys.version_info.major}.{sys.version_info.minor}") + shlib_suffix = _get_shlib_suffix(config_vars.get) search_directories = _search_directories(config_vars.get, base_executable) - search_libnames = _search_library_names(config_vars.get) - - def _add_if_exists(target, path): - if os.path.exists(path) or os.path.isdir(path): - target[path] = None + search_libnames = _search_library_names(config_vars.get, shlib_suffix) interface_libraries = {} dynamic_libraries = {} static_libraries = {} + for root_dir in search_directories: for libname in search_libnames: + # Check whether the library exists. composed_path = os.path.join(root_dir, libname) - if libname.endswith(".a"): - _add_if_exists(static_libraries, composed_path) - continue + if os.path.exists(composed_path) or os.path.isdir(composed_path): + if libname.endswith(".a"): + static_libraries[composed_path] = None + else: + dynamic_libraries[composed_path] = None - _add_if_exists(dynamic_libraries, composed_path) + interface_path = None if libname.endswith(".dll"): # On windows a .lib file may be an "import library" or a static library. # The file could be inspected to determine which it is; typically python @@ -172,14 +183,20 @@ def _add_if_exists(target, path): # # See: https://docs.python.org/3/extending/windows.html # https://learn.microsoft.com/en-us/windows/win32/dlls/dynamic-link-library-creation - _add_if_exists( - interface_libraries, os.path.join(root_dir, libname[:-3] + "lib") - ) + interface_path = os.path.join(root_dir, libname[:-3] + "lib") elif libname.endswith(".so"): # It's possible, though unlikely, that interface stubs (.ifso) exist. - _add_if_exists( - interface_libraries, os.path.join(root_dir, libname[:-2] + "ifso") - ) + interface_path = os.path.join(root_dir, libname[:-2] + "ifso") + + # Check whether an interface library exists. + if interface_path and os.path.exists(interface_path): + interface_libraries[interface_path] = None + + # Non-windows typically has abiflags. + if hasattr(sys, "abiflags"): + abiflags = sys.abiflags + else: + abiflags = "" # When no libraries are found it's likely that the python interpreter is not # configured to use shared or static libraries (minilinux). If this seems @@ -188,6 +205,8 @@ def _add_if_exists(target, path): "dynamic_libraries": list(dynamic_libraries.keys()), "static_libraries": list(static_libraries.keys()), "interface_libraries": list(interface_libraries.keys()), + "shlib_suffix": "" if _IS_WINDOWS else shlib_suffix, + "abi_flags": abiflags, } diff --git a/python/private/local_runtime_repo.bzl b/python/private/local_runtime_repo.bzl index 583926b15f..024f7c5e8a 100644 --- a/python/private/local_runtime_repo.bzl +++ b/python/private/local_runtime_repo.bzl @@ -39,6 +39,7 @@ define_local_runtime_toolchain_impl( libraries = {libraries}, implementation_name = "{implementation_name}", os = "{os}", + abi_flags = "{abi_flags}", ) """ @@ -49,33 +50,33 @@ def _norm_path(path): path = path[:-1] return path -def _symlink_first_library(rctx, logger, libraries): +def _symlink_first_library(rctx, logger, libraries, shlib_suffix): """Symlinks the shared libraries into the lib/ directory. Args: rctx: A repository_ctx object logger: A repo_utils.logger object libraries: A list of static library paths to potentially symlink. + shlib_suffix: A suffix only provided for shared libraries to ensure + that the srcs restriction of cc_library targets are met. Returns: A single library path linked by the action. """ - linked = None for target in libraries: origin = rctx.path(target) if not origin.exists: # The reported names don't always exist; it depends on the particulars # of the runtime installation. continue - if target.endswith("/Python"): - linked = "lib/{}.dylib".format(origin.basename) + if shlib_suffix and not target.endswith(shlib_suffix): + linked = "lib/{}{}".format(origin.basename, shlib_suffix) else: linked = "lib/{}".format(origin.basename) logger.debug("Symlinking {} to {}".format(origin, linked)) rctx.watch(origin) rctx.symlink(origin, linked) - break - - return linked + return linked + return None def _local_runtime_repo_impl(rctx): logger = repo_utils.logger(rctx) @@ -152,9 +153,9 @@ def _local_runtime_repo_impl(rctx): rctx.symlink(include_path, "include") rctx.report_progress("Symlinking external Python shared libraries") - interface_library = _symlink_first_library(rctx, logger, info["interface_libraries"]) - shared_library = _symlink_first_library(rctx, logger, info["dynamic_libraries"]) - static_library = _symlink_first_library(rctx, logger, info["static_libraries"]) + interface_library = _symlink_first_library(rctx, logger, info["interface_libraries"], None) + shared_library = _symlink_first_library(rctx, logger, info["dynamic_libraries"], info["shlib_suffix"]) + static_library = _symlink_first_library(rctx, logger, info["static_libraries"], None) libraries = [] if shared_library: @@ -173,6 +174,7 @@ def _local_runtime_repo_impl(rctx): libraries = repr(libraries), implementation_name = info["implementation_name"], os = "@platforms//os:{}".format(repo_utils.get_platforms_os_name(rctx)), + abi_flags = info["abi_flags"], ) logger.debug(lambda: "BUILD.bazel\n{}".format(build_bazel)) @@ -269,6 +271,7 @@ def _expand_incompatible_template(): minor = "0", micro = "0", os = "@platforms//:incompatible", + abi_flags = "", ) def _find_python_exe_from_target(rctx): diff --git a/python/private/local_runtime_repo_setup.bzl b/python/private/local_runtime_repo_setup.bzl index 6cff1aea43..0ce1d4d764 100644 --- a/python/private/local_runtime_repo_setup.bzl +++ b/python/private/local_runtime_repo_setup.bzl @@ -33,7 +33,8 @@ def define_local_runtime_toolchain_impl( interface_library, libraries, implementation_name, - os): + os, + abi_flags): """Defines a toolchain implementation for a local Python runtime. Generates public targets: @@ -59,6 +60,7 @@ def define_local_runtime_toolchain_impl( `sys.implementation.name`. os: `str` A label to the OS constraint (e.g. `@platforms//os:linux`) for this runtime. + abi_flags: `str` Str. Flags provided by sys.abiflags for the runtime. """ major_minor = "{}.{}".format(major, minor) major_minor_micro = "{}.{}".format(major_minor, micro) @@ -113,6 +115,7 @@ def define_local_runtime_toolchain_impl( "minor": minor, }, implementation_name = implementation_name, + abi_flags = abi_flags, ) py_runtime_pair( From 4fb634ebb9d07d50fd708d007cbf75d59e8094cc Mon Sep 17 00:00:00 2001 From: Toby Harradine Date: Mon, 10 Nov 2025 11:43:36 +1100 Subject: [PATCH 514/922] refactor: defer zip manifest building to execution phase to improve analysis phase performance (#3381) When py_binary/py_test were being built, they were flattening the runfiles depsets at analysis time in order to create the zip file mapping manifest for their implicit zipapp outputs. This flattening was necessary because they had to filter out the original main executable from the runfiles that didn't belong in the zipapp. This flattening is expensive for large builds, in some cases adding over 400 seconds of time and significant memory overhead. To fix, have the zip file manifest use the `runfiles_with_exe` object, which is the runfiles, but pre-filtered for the files zip building doesn't want. This then allows passing the depsets directly to `Args.add_all` and using map_each to transform them. Additionally, pass `runfiles.empty_filenames` using a lambda. Accessing that attribute implicitly flattens the runfiles. Finally, because the original profiles indicated `str.format()` was a non-trivial amount of time (46 seconds / 15% of build time), switch to using `+` instead. This is a more incremental alternative to #3380 which achieves _most_ of the same optimization with only Starlark changes, as opposed to introducing an external script written in C++. [Profile of a large build](https://github.com/user-attachments/assets/e90ae699-a04d-44df-b53c-1156aa890af5), which shows a Starlark CPU profile. It shows an overall build time of 305 seconds. 46 seconds (15%) are spent in `map_zip_runfiles`, half of which is in `str.startswith()` and the other half in `str.format()`. --------- Co-authored-by: Richard Levasseur --- CHANGELOG.md | 2 ++ python/private/py_executable.bzl | 57 ++++++++++++++++++-------------- 2 files changed, 35 insertions(+), 24 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 8c472c08d8..d7403b4f1a 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -73,6 +73,8 @@ END_UNRELEASED_TEMPLATE ([#3085](https://github.com/bazel-contrib/rules_python/issues/3085)). * (toolchains) local toolchains now tell the `sys.abiflags` value of the underlying runtime. +* (performance) 90% reduction in py_binary/py_test analysis phase cost. + ([#3381](https://github.com/bazel-contrib/rules_python/pull/3381)). {#v0-0-0-added} ### Added diff --git a/python/private/py_executable.bzl b/python/private/py_executable.bzl index 1a5ad4c3c6..d5c0fa5388 100644 --- a/python/private/py_executable.bzl +++ b/python/private/py_executable.bzl @@ -380,9 +380,8 @@ def _create_executable( _create_zip_file( ctx, output = zip_file, - original_nonzip_executable = executable, zip_main = zip_main, - runfiles = runfiles_details.default_runfiles.merge(extra_runfiles), + runfiles = runfiles_details.runfiles_without_exe.merge(extra_runfiles), ) extra_files_to_build = [] @@ -803,7 +802,7 @@ def _create_windows_exe_launcher( use_default_shell_env = True, ) -def _create_zip_file(ctx, *, output, original_nonzip_executable, zip_main, runfiles): +def _create_zip_file(ctx, *, output, zip_main, runfiles): """Create a Python zipapp (zip with __main__.py entry point).""" workspace_name = ctx.workspace_name legacy_external_runfiles = _py_builtins.get_legacy_external_runfiles(ctx) @@ -819,17 +818,27 @@ def _create_zip_file(ctx, *, output, original_nonzip_executable, zip_main, runfi _get_zip_runfiles_path("__init__.py", workspace_name, legacy_external_runfiles), ), ) - for path in runfiles.empty_filenames.to_list(): - manifest.add("{}=".format(_get_zip_runfiles_path(path, workspace_name, legacy_external_runfiles))) + + def map_zip_empty_filenames(list_paths_cb): + return [ + _get_zip_runfiles_path(path, workspace_name, legacy_external_runfiles) + "=" + for path in list_paths_cb().to_list() + ] + + manifest.add_all( + # NOTE: Accessing runfiles.empty_filenames implicitly flattens the runfiles. + # Smuggle a lambda in via a list to defer that flattening. + [lambda: runfiles.empty_filenames], + map_each = map_zip_empty_filenames, + allow_closure = True, + ) def map_zip_runfiles(file): - if file != original_nonzip_executable and file != output: - return "{}={}".format( - _get_zip_runfiles_path(file.short_path, workspace_name, legacy_external_runfiles), - file.path, - ) - else: - return None + return ( + # NOTE: Use "+" for performance + _get_zip_runfiles_path(file.short_path, workspace_name, legacy_external_runfiles) + + "=" + file.path + ) manifest.add_all(runfiles.files, map_each = map_zip_runfiles, allow_closure = True) @@ -850,13 +859,6 @@ def _create_zip_file(ctx, *, output, original_nonzip_executable, zip_main, runfi )) inputs.append(zip_repo_mapping_manifest) - for artifact in runfiles.files.to_list(): - # Don't include the original executable because it isn't used by the - # zip file, so no need to build it for the action. - # Don't include the zipfile itself because it's an output. - if artifact != original_nonzip_executable and artifact != output: - inputs.append(artifact) - zip_cli_args = ctx.actions.args() zip_cli_args.add("cC") zip_cli_args.add(output) @@ -864,7 +866,7 @@ def _create_zip_file(ctx, *, output, original_nonzip_executable, zip_main, runfi ctx.actions.run( executable = ctx.executable._zipper, arguments = [zip_cli_args, manifest], - inputs = depset(inputs), + inputs = depset(inputs, transitive = [runfiles.files]), outputs = [output], use_default_shell_env = True, mnemonic = "PythonZipper", @@ -872,15 +874,22 @@ def _create_zip_file(ctx, *, output, original_nonzip_executable, zip_main, runfi ) def _get_zip_runfiles_path(path, workspace_name, legacy_external_runfiles): + maybe_workspace = "" if legacy_external_runfiles and path.startswith(_EXTERNAL_PATH_PREFIX): - zip_runfiles_path = paths.relativize(path, _EXTERNAL_PATH_PREFIX) + zip_runfiles_path = path.removeprefix(_EXTERNAL_PATH_PREFIX) else: # NOTE: External runfiles (artifacts in other repos) will have a leading # path component of "../" so that they refer outside the main workspace - # directory and into the runfiles root. By normalizing, we simplify e.g. + # directory and into the runfiles root. So we simplify it, e.g. # "workspace/../foo/bar" to simply "foo/bar". - zip_runfiles_path = paths.normalize("{}/{}".format(workspace_name, path)) - return "{}/{}".format(_ZIP_RUNFILES_DIRECTORY_NAME, zip_runfiles_path) + if path.startswith("../"): + zip_runfiles_path = path[3:] + else: + zip_runfiles_path = path + maybe_workspace = workspace_name + "/" + + # NOTE: Use "+" for performance + return _ZIP_RUNFILES_DIRECTORY_NAME + "/" + maybe_workspace + zip_runfiles_path def _create_executable_zip_file( ctx, From e132012693f078d2bcbc359ae9b58a4d6d35b2ad Mon Sep 17 00:00:00 2001 From: Ignas Anikevicius <240938+aignas@users.noreply.github.com> Date: Mon, 10 Nov 2025 10:34:59 +0900 Subject: [PATCH 515/922] fix(pip): allow for different extras for different target platforms (#3385) With this PR we first evaluate the markers in the requirements files before going any further to aggregate them and process further. This makes the separation of logic a little bit more clear. I wanted to do this before I add more tests to after debugging the failures observed when enabling pipstar. Whilst cleaning up further I realized that I can fix the handling of packages where some platforms may end up not needing extras whilst others do. This is achieved by reusing the same code that allows us to have different versions per platform. Work towards #2949 Fixes #3374 --- python/private/pypi/hub_builder.bzl | 7 +- python/private/pypi/parse_requirements.bzl | 81 +++++---- python/private/pypi/whl_repo_name.bzl | 6 +- tests/pypi/hub_builder/hub_builder_tests.bzl | 160 ++++++++++++++++-- .../parse_requirements_tests.bzl | 81 +++++++-- 5 files changed, 261 insertions(+), 74 deletions(-) diff --git a/python/private/pypi/hub_builder.bzl b/python/private/pypi/hub_builder.bzl index 58d35f2681..7cf60ff85f 100644 --- a/python/private/pypi/hub_builder.bzl +++ b/python/private/pypi/hub_builder.bzl @@ -566,8 +566,13 @@ def _whl_repo( for p in src.target_platforms ] + # TODO @aignas 2025-11-02: once we have pipstar enabled we can add extra + # targets to each hub for each extra combination and solve this more cleanly as opposed to + # duplicating whl_library repositories. + target_platforms = src.target_platforms if is_multiple_versions else [] + return struct( - repo_name = whl_repo_name(src.filename, src.sha256), + repo_name = whl_repo_name(src.filename, src.sha256, *target_platforms), args = args, config_setting = whl_config_setting( version = python_version, diff --git a/python/private/pypi/parse_requirements.bzl b/python/private/pypi/parse_requirements.bzl index acf3b0c6ae..7d210abbaa 100644 --- a/python/private/pypi/parse_requirements.bzl +++ b/python/private/pypi/parse_requirements.bzl @@ -88,6 +88,7 @@ def parse_requirements( evaluate_markers = evaluate_markers or (lambda _ctx, _requirements: {}) options = {} requirements = {} + reqs_with_env_markers = {} for file, plats in requirements_by_platform.items(): logger.trace(lambda: "Using {} for {}".format(file, plats)) contents = ctx.read(file) @@ -96,16 +97,41 @@ def parse_requirements( # needed for the whl_library declarations later. parse_result = parse_requirements_txt(contents) + tokenized_options = [] + for opt in parse_result.options: + for p in opt.split(" "): + tokenized_options.append(p) + + pip_args = tokenized_options + extra_pip_args + for plat in plats: + requirements[plat] = parse_result.requirements + for entry in parse_result.requirements: + requirement_line = entry[1] + + # output all of the requirement lines that have a marker + if ";" in requirement_line: + reqs_with_env_markers.setdefault(requirement_line, []).append(plat) + options[plat] = pip_args + + # This may call to Python, so execute it early (before calling to the + # internet below) and ensure that we call it only once. + resolved_marker_platforms = evaluate_markers(ctx, reqs_with_env_markers) + logger.trace(lambda: "Evaluated env markers from:\n{}\n\nTo:\n{}".format( + reqs_with_env_markers, + resolved_marker_platforms, + )) + + requirements_by_platform = {} + for plat, parse_results in requirements.items(): # Replicate a surprising behavior that WORKSPACE builds allowed: # Defining a repo with the same name multiple times, but only the last # definition is respected. # The requirement lines might have duplicate names because lines for extras # are returned as just the base package name. e.g., `foo[bar]` results # in an entry like `("foo", "foo[bar] == 1.0 ...")`. - # Lines with different markers are not condidered duplicates. requirements_dict = {} for entry in sorted( - parse_result.requirements, + parse_results, # Get the longest match and fallback to original WORKSPACE sorting, # which should get us the entry with most extras. # @@ -114,33 +140,22 @@ def parse_requirements( # should do this now. key = lambda x: (len(x[1].partition("==")[0]), x), ): - req = requirement(entry[1]) - requirements_dict[(req.name, req.version, req.marker)] = entry + req_line = entry[1] + req = requirement(req_line) - tokenized_options = [] - for opt in parse_result.options: - for p in opt.split(" "): - tokenized_options.append(p) + if req.marker and plat not in resolved_marker_platforms.get(req_line, []): + continue - pip_args = tokenized_options + extra_pip_args - for plat in plats: - requirements[plat] = requirements_dict.values() - options[plat] = pip_args + requirements_dict[req.name] = entry - requirements_by_platform = {} - reqs_with_env_markers = {} - for target_platform, reqs_ in requirements.items(): - extra_pip_args = options[target_platform] + extra_pip_args = options[plat] - for distribution, requirement_line in reqs_: + for distribution, requirement_line in requirements_dict.values(): for_whl = requirements_by_platform.setdefault( normalize_name(distribution), {}, ) - if ";" in requirement_line: - reqs_with_env_markers.setdefault(requirement_line, []).append(target_platform) - for_req = for_whl.setdefault( (requirement_line, ",".join(extra_pip_args)), struct( @@ -151,20 +166,7 @@ def parse_requirements( extra_pip_args = extra_pip_args, ), ) - for_req.target_platforms.append(target_platform) - - # This may call to Python, so execute it early (before calling to the - # internet below) and ensure that we call it only once. - # - # NOTE @aignas 2024-07-13: in the future, if this is something that we want - # to do, we could use Python to parse the requirement lines and infer the - # URL of the files to download things from. This should be important for - # VCS package references. - env_marker_target_platforms = evaluate_markers(ctx, reqs_with_env_markers) - logger.trace(lambda: "Evaluated env markers from:\n{}\n\nTo:\n{}".format( - reqs_with_env_markers, - env_marker_target_platforms, - )) + for_req.target_platforms.append(plat) index_urls = {} if get_index_urls: @@ -183,8 +185,7 @@ def parse_requirements( for name, reqs in sorted(requirements_by_platform.items()): requirement_target_platforms = {} for r in reqs.values(): - target_platforms = env_marker_target_platforms.get(r.requirement_line, r.target_platforms) - for p in target_platforms: + for p in r.target_platforms: requirement_target_platforms[p] = None item = struct( @@ -197,7 +198,6 @@ def parse_requirements( reqs = reqs, index_urls = index_urls, platforms = platforms, - env_marker_target_platforms = env_marker_target_platforms, extract_url_srcs = extract_url_srcs, logger = logger, ), @@ -221,18 +221,13 @@ def _package_srcs( index_urls, platforms, logger, - env_marker_target_platforms, extract_url_srcs): """A function to return sources for a particular package.""" srcs = {} for r in sorted(reqs.values(), key = lambda r: r.requirement_line): - if ";" in r.requirement_line: - target_platforms = env_marker_target_platforms.get(r.requirement_line, []) - else: - target_platforms = r.target_platforms extra_pip_args = tuple(r.extra_pip_args) - for target_platform in target_platforms: + for target_platform in r.target_platforms: if platforms and target_platform not in platforms: fail("The target platform '{}' could not be found in {}".format( target_platform, diff --git a/python/private/pypi/whl_repo_name.bzl b/python/private/pypi/whl_repo_name.bzl index 2b3b5418aa..29d774c361 100644 --- a/python/private/pypi/whl_repo_name.bzl +++ b/python/private/pypi/whl_repo_name.bzl @@ -18,12 +18,14 @@ load("//python/private:normalize_name.bzl", "normalize_name") load(":parse_whl_name.bzl", "parse_whl_name") -def whl_repo_name(filename, sha256): +def whl_repo_name(filename, sha256, *target_platforms): """Return a valid whl_library repo name given a distribution filename. Args: filename: {type}`str` the filename of the distribution. sha256: {type}`str` the sha256 of the distribution. + *target_platforms: {type}`list[str]` the extra suffixes to append. + Only used when we need to support different extras per version. Returns: a string that can be used in {obj}`whl_library`. @@ -59,6 +61,8 @@ def whl_repo_name(filename, sha256): elif version: parts.insert(1, version) + parts.extend([p.partition("_")[-1] for p in target_platforms]) + return "_".join(parts) def pypi_repo_name(whl_name, *target_platforms): diff --git a/tests/pypi/hub_builder/hub_builder_tests.bzl b/tests/pypi/hub_builder/hub_builder_tests.bzl index ee6200a70a..6d061f4d56 100644 --- a/tests/pypi/hub_builder/hub_builder_tests.bzl +++ b/tests/pypi/hub_builder/hub_builder_tests.bzl @@ -203,6 +203,142 @@ def _test_simple_multiple_requirements(env): _tests.append(_test_simple_multiple_requirements) +def _test_simple_extras_vs_no_extras(env): + builder = hub_builder(env) + builder.pip_parse( + _mock_mctx( + read = lambda x: { + "darwin.txt": "simple[foo]==0.0.1 --hash=sha256:deadbeef", + "win.txt": "simple==0.0.1 --hash=sha256:deadbeef", + }[x], + ), + _parse( + hub_name = "pypi", + python_version = "3.15", + requirements_darwin = "darwin.txt", + requirements_windows = "win.txt", + ), + ) + pypi = builder.build() + + pypi.exposed_packages().contains_exactly(["simple"]) + pypi.group_map().contains_exactly({}) + pypi.whl_map().contains_exactly({ + "simple": { + "pypi_315_simple_osx_aarch64": [ + whl_config_setting( + target_platforms = [ + "cp315_osx_aarch64", + ], + version = "3.15", + ), + ], + "pypi_315_simple_windows_aarch64": [ + whl_config_setting( + target_platforms = [ + "cp315_windows_aarch64", + ], + version = "3.15", + ), + ], + }, + }) + pypi.whl_libraries().contains_exactly({ + "pypi_315_simple_osx_aarch64": { + "config_load": "@pypi//:config.bzl", + "dep_template": "@pypi//{name}:{target}", + "python_interpreter_target": "unit_test_interpreter_target", + "requirement": "simple[foo]==0.0.1 --hash=sha256:deadbeef", + }, + "pypi_315_simple_windows_aarch64": { + "config_load": "@pypi//:config.bzl", + "dep_template": "@pypi//{name}:{target}", + "python_interpreter_target": "unit_test_interpreter_target", + "requirement": "simple==0.0.1 --hash=sha256:deadbeef", + }, + }) + pypi.extra_aliases().contains_exactly({}) + +_tests.append(_test_simple_extras_vs_no_extras) + +def _test_simple_extras_vs_no_extras_simpleapi(env): + def mocksimpleapi_download(*_, **__): + return { + "simple": parse_simpleapi_html( + url = "https://example.com", + content = """\ + simple-0.0.1-py3-none-any.whl
+""", + ), + } + + builder = hub_builder( + env, + simpleapi_download_fn = mocksimpleapi_download, + ) + builder.pip_parse( + _mock_mctx( + read = lambda x: { + "darwin.txt": "simple[foo]==0.0.1 --hash=sha256:deadbeef", + "win.txt": "simple==0.0.1 --hash=sha256:deadbeef", + }[x], + ), + _parse( + hub_name = "pypi", + python_version = "3.15", + requirements_darwin = "darwin.txt", + requirements_windows = "win.txt", + experimental_index_url = "example.com", + ), + ) + pypi = builder.build() + + pypi.exposed_packages().contains_exactly(["simple"]) + pypi.group_map().contains_exactly({}) + pypi.whl_map().contains_exactly({ + "simple": { + "pypi_315_simple_py3_none_any_deadbeef_osx_aarch64": [ + whl_config_setting( + target_platforms = [ + "cp315_osx_aarch64", + ], + version = "3.15", + ), + ], + "pypi_315_simple_py3_none_any_deadbeef_windows_aarch64": [ + whl_config_setting( + target_platforms = [ + "cp315_windows_aarch64", + ], + version = "3.15", + ), + ], + }, + }) + pypi.whl_libraries().contains_exactly({ + "pypi_315_simple_py3_none_any_deadbeef_osx_aarch64": { + "config_load": "@pypi//:config.bzl", + "dep_template": "@pypi//{name}:{target}", + "filename": "simple-0.0.1-py3-none-any.whl", + "python_interpreter_target": "unit_test_interpreter_target", + "requirement": "simple[foo]==0.0.1", + "sha256": "deadbeef", + "urls": ["https://example.com/simple-0.0.1-py3-none-any.whl"], + }, + "pypi_315_simple_py3_none_any_deadbeef_windows_aarch64": { + "config_load": "@pypi//:config.bzl", + "dep_template": "@pypi//{name}:{target}", + "filename": "simple-0.0.1-py3-none-any.whl", + "python_interpreter_target": "unit_test_interpreter_target", + "requirement": "simple==0.0.1", + "sha256": "deadbeef", + "urls": ["https://example.com/simple-0.0.1-py3-none-any.whl"], + }, + }) + pypi.extra_aliases().contains_exactly({}) + +_tests.append(_test_simple_extras_vs_no_extras_simpleapi) + def _test_simple_multiple_python_versions(env): builder = hub_builder( env, @@ -496,34 +632,34 @@ torch==2.4.1+cpu ; platform_machine == 'x86_64' \ pypi.group_map().contains_exactly({}) pypi.whl_map().contains_exactly({ "torch": { - "pypi_312_torch_cp312_cp312_linux_x86_64_8800deef": [ + "pypi_312_torch_cp312_cp312_linux_x86_64_8800deef_linux_x86_64": [ whl_config_setting( - target_platforms = ("cp312_linux_x86_64",), + target_platforms = ["cp312_linux_x86_64"], version = "3.12", ), ], - "pypi_312_torch_cp312_cp312_manylinux_2_17_aarch64_36109432": [ + "pypi_312_torch_cp312_cp312_manylinux_2_17_aarch64_36109432_linux_aarch64": [ whl_config_setting( - target_platforms = ("cp312_linux_aarch64",), + target_platforms = ["cp312_linux_aarch64"], version = "3.12", ), ], - "pypi_312_torch_cp312_cp312_win_amd64_3a570e5c": [ + "pypi_312_torch_cp312_cp312_win_amd64_3a570e5c_windows_x86_64": [ whl_config_setting( - target_platforms = ("cp312_windows_x86_64",), + target_platforms = ["cp312_windows_x86_64"], version = "3.12", ), ], - "pypi_312_torch_cp312_none_macosx_11_0_arm64_72b484d5": [ + "pypi_312_torch_cp312_none_macosx_11_0_arm64_72b484d5_osx_aarch64": [ whl_config_setting( - target_platforms = ("cp312_osx_aarch64",), + target_platforms = ["cp312_osx_aarch64"], version = "3.12", ), ], }, }) pypi.whl_libraries().contains_exactly({ - "pypi_312_torch_cp312_cp312_linux_x86_64_8800deef": { + "pypi_312_torch_cp312_cp312_linux_x86_64_8800deef_linux_x86_64": { "config_load": "@pypi//:config.bzl", "dep_template": "@pypi//{name}:{target}", "filename": "torch-2.4.1+cpu-cp312-cp312-linux_x86_64.whl", @@ -532,7 +668,7 @@ torch==2.4.1+cpu ; platform_machine == 'x86_64' \ "sha256": "8800deef0026011d502c0c256cc4b67d002347f63c3a38cd8e45f1f445c61364", "urls": ["https://torch.index/whl/cpu/torch-2.4.1%2Bcpu-cp312-cp312-linux_x86_64.whl"], }, - "pypi_312_torch_cp312_cp312_manylinux_2_17_aarch64_36109432": { + "pypi_312_torch_cp312_cp312_manylinux_2_17_aarch64_36109432_linux_aarch64": { "config_load": "@pypi//:config.bzl", "dep_template": "@pypi//{name}:{target}", "filename": "torch-2.4.1-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", @@ -541,7 +677,7 @@ torch==2.4.1+cpu ; platform_machine == 'x86_64' \ "sha256": "36109432b10bd7163c9b30ce896f3c2cca1b86b9765f956a1594f0ff43091e2a", "urls": ["https://torch.index/whl/cpu/torch-2.4.1-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl"], }, - "pypi_312_torch_cp312_cp312_win_amd64_3a570e5c": { + "pypi_312_torch_cp312_cp312_win_amd64_3a570e5c_windows_x86_64": { "config_load": "@pypi//:config.bzl", "dep_template": "@pypi//{name}:{target}", "filename": "torch-2.4.1+cpu-cp312-cp312-win_amd64.whl", @@ -550,7 +686,7 @@ torch==2.4.1+cpu ; platform_machine == 'x86_64' \ "sha256": "3a570e5c553415cdbddfe679207327b3a3806b21c6adea14fba77684d1619e97", "urls": ["https://torch.index/whl/cpu/torch-2.4.1%2Bcpu-cp312-cp312-win_amd64.whl"], }, - "pypi_312_torch_cp312_none_macosx_11_0_arm64_72b484d5": { + "pypi_312_torch_cp312_none_macosx_11_0_arm64_72b484d5_osx_aarch64": { "config_load": "@pypi//:config.bzl", "dep_template": "@pypi//{name}:{target}", "filename": "torch-2.4.1-cp312-none-macosx_11_0_arm64.whl", diff --git a/tests/pypi/parse_requirements/parse_requirements_tests.bzl b/tests/pypi/parse_requirements/parse_requirements_tests.bzl index bd0078bfa4..63755d2edd 100644 --- a/tests/pypi/parse_requirements/parse_requirements_tests.bzl +++ b/tests/pypi/parse_requirements/parse_requirements_tests.bzl @@ -22,12 +22,6 @@ load("//python/private/pypi:pep508_env.bzl", pep508_env = "env") # buildifier: def _mock_ctx(): testdata = { - "requirements_different_package_version": """\ -foo==0.0.1+local \ - --hash=sha256:deadbeef -foo==0.0.1 \ - --hash=sha256:deadb00f -""", "requirements_direct": """\ foo[extra] @ https://some-url/package.whl """, @@ -39,6 +33,14 @@ foo @ https://github.com/org/foo/downloads/foo-1.1.tar.gz foo[extra]==0.0.1 \ --hash=sha256:deadbeef +""", + "requirements_foo": """\ +foo==0.0.1 \ + --hash=sha256:deadb00f +""", + "requirements_foo_local": """\ +foo==0.0.1+local \ + --hash=sha256:deadbeef """, "requirements_git": """ foo @ git+https://github.com/org/foo.git@deadbeef @@ -75,7 +77,7 @@ foo==0.0.2; python_full_version >= '3.10.0' \ --hash=sha256:deadb11f """, "requirements_optional_hash": """ -foo==0.0.4 @ https://example.org/foo-0.0.4.whl +bar==0.0.4 @ https://example.org/bar-0.0.4.whl foo==0.0.5 @ https://example.org/foo-0.0.5.whl --hash=sha256:deadbeef """, "requirements_osx": """\ @@ -441,7 +443,8 @@ _tests.append(_test_env_marker_resolution) def _test_different_package_version(env): got = parse_requirements( requirements_by_platform = { - "requirements_different_package_version": ["linux_x86_64"], + "requirements_foo": ["linux_aarch64"], + "requirements_foo_local": ["linux_x86_64"], }, ) env.expect.that_collection(got).contains_exactly([ @@ -454,7 +457,7 @@ def _test_different_package_version(env): distribution = "foo", extra_pip_args = [], requirement_line = "foo==0.0.1 --hash=sha256:deadb00f", - target_platforms = ["linux_x86_64"], + target_platforms = ["linux_aarch64"], url = "", filename = "", sha256 = "", @@ -476,10 +479,11 @@ def _test_different_package_version(env): _tests.append(_test_different_package_version) -def _test_optional_hash(env): +def _test_different_package_extras(env): got = parse_requirements( requirements_by_platform = { - "requirements_optional_hash": ["linux_x86_64"], + "requirements_foo": ["linux_aarch64"], + "requirements_lock": ["linux_x86_64"], }, ) env.expect.that_collection(got).contains_exactly([ @@ -491,13 +495,58 @@ def _test_optional_hash(env): struct( distribution = "foo", extra_pip_args = [], - requirement_line = "foo==0.0.4", + requirement_line = "foo==0.0.1 --hash=sha256:deadb00f", + target_platforms = ["linux_aarch64"], + url = "", + filename = "", + sha256 = "", + yanked = False, + ), + struct( + distribution = "foo", + extra_pip_args = [], + requirement_line = "foo[extra]==0.0.1 --hash=sha256:deadbeef", target_platforms = ["linux_x86_64"], - url = "https://example.org/foo-0.0.4.whl", - filename = "foo-0.0.4.whl", + url = "", + filename = "", sha256 = "", yanked = False, ), + ], + ), + ]) + +_tests.append(_test_different_package_extras) + +def _test_optional_hash(env): + got = parse_requirements( + requirements_by_platform = { + "requirements_optional_hash": ["linux_x86_64"], + }, + ) + env.expect.that_collection(got).contains_exactly([ + struct( + name = "bar", + is_exposed = True, + is_multiple_versions = False, + srcs = [ + struct( + distribution = "bar", + extra_pip_args = [], + requirement_line = "bar==0.0.4", + target_platforms = ["linux_x86_64"], + url = "https://example.org/bar-0.0.4.whl", + filename = "bar-0.0.4.whl", + sha256 = "", + yanked = False, + ), + ], + ), + struct( + name = "foo", + is_exposed = True, + is_multiple_versions = False, + srcs = [ struct( distribution = "foo", extra_pip_args = [], @@ -686,7 +735,6 @@ def _test_get_index_urls_different_versions(env): ), }, ), - debug = True, ) env.expect.that_collection(got).contains_exactly([ @@ -760,13 +808,12 @@ def _test_get_index_urls_single_py_version(env): ), }, ), - debug = True, ) env.expect.that_collection(got).contains_exactly([ struct( is_exposed = True, - is_multiple_versions = True, + is_multiple_versions = False, name = "foo", srcs = [ struct( From 4dca7e543b6355947cb1a1d309c610450fa70abf Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Sun, 9 Nov 2025 20:33:14 -0800 Subject: [PATCH 516/922] chore: make doc building use bootstrap script and venv site packages (#3403) Making the doc build use it seems like a good way to get some real usage of the feature. --- docs/BUILD.bazel | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/docs/BUILD.bazel b/docs/BUILD.bazel index ffe800a72c..d7748d35a4 100644 --- a/docs/BUILD.bazel +++ b/docs/BUILD.bazel @@ -15,6 +15,7 @@ load("@bazel_skylib//rules:build_test.bzl", "build_test") load("@dev_pip//:requirements.bzl", "requirement") load("//python/private:bzlmod_enabled.bzl", "BZLMOD_ENABLED") # buildifier: disable=bzl-visibility +load("//python/private:common_labels.bzl", "labels") # buildifier: disable=bzl-visibility load("//python/uv:lock.bzl", "lock") # buildifier: disable=bzl-visibility load("//sphinxdocs:readthedocs.bzl", "readthedocs_install") load("//sphinxdocs:sphinx.bzl", "sphinx_build_binary", "sphinx_docs") @@ -161,6 +162,10 @@ readthedocs_install( sphinx_build_binary( name = "sphinx-build", + config_settings = { + labels.BOOTSTRAP_IMPL: "script", + labels.VENVS_SITE_PACKAGES: "yes", + }, target_compatible_with = _TARGET_COMPATIBLE_WITH, deps = [ requirement("sphinx"), From e40b6093525feee99dc47db87ee7fb3b7da3f9ed Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Sun, 9 Nov 2025 20:42:10 -0800 Subject: [PATCH 517/922] fix: use runfiles symlinks for venv symlink creation to reduce action count (#3402) When the venv files are materialized, it can result in many symlink actions being created. Rather than register them as regular symlink actions, batch them into the runfiles object, which can probably handle large numbers of them more efficiently. Work towards https://github.com/bazel-contrib/rules_python/issues/3401 --- python/private/py_executable.bzl | 9 +++++++-- python/private/venv_runfiles.bzl | 24 ++++++++++++++++-------- 2 files changed, 23 insertions(+), 10 deletions(-) diff --git a/python/private/py_executable.bzl b/python/private/py_executable.bzl index d5c0fa5388..669951e172 100644 --- a/python/private/py_executable.bzl +++ b/python/private/py_executable.bzl @@ -356,7 +356,7 @@ def _create_executable( [stage2_bootstrap] + ( venv.files_without_interpreter if venv else [] ), - ) + ).merge(venv.lib_runfiles) zip_main = _create_zip_main( ctx, stage2_bootstrap = stage2_bootstrap, @@ -606,7 +606,7 @@ def _create_venv(ctx, output_prefix, imports, runtime_details): } venv_app_files = create_venv_app_files(ctx, ctx.attr.deps, venv_dir_map) - files_without_interpreter = [pth, site_init] + venv_app_files + files_without_interpreter = [pth, site_init] + venv_app_files.venv_files if pyvenv_cfg: files_without_interpreter.append(pyvenv_cfg) @@ -629,6 +629,11 @@ def _create_venv(ctx, output_prefix, imports, runtime_details): venv, ), ), + # venv files for user library dependencies (files that are specific + # to the executable bootstrap and python runtime aren't here). + lib_runfiles = ctx.runfiles( + symlinks = venv_app_files.runfiles_symlinks, + ), ) def _map_each_identity(v): diff --git a/python/private/venv_runfiles.bzl b/python/private/venv_runfiles.bzl index 05dc296e15..eeedda4555 100644 --- a/python/private/venv_runfiles.bzl +++ b/python/private/venv_runfiles.bzl @@ -30,7 +30,12 @@ def create_venv_app_files(ctx, deps, venv_dir_map): paths within the current ctx's venv (e.g. `_foo.venv/bin`). Returns: - {type}`list[File]` of the files that were created. + {type}`struct` with the following attributes: + * {type}`list[File]` `venv_files` additional files created for + the venv. + * {type}`dict[str, File]` `runfiles_symlinks` map intended for + the `runfiles.symlinks` argument. A map of main-repo + relative paths to File. """ # maps venv-relative path to the runfiles path it should point to @@ -44,16 +49,16 @@ def create_venv_app_files(ctx, deps, venv_dir_map): link_map = build_link_map(ctx, entries) venv_files = [] + runfiles_symlinks = {} + for kind, kind_map in link_map.items(): base = venv_dir_map[kind] for venv_path, link_to in kind_map.items(): bin_venv_path = paths.join(base, venv_path) if is_file(link_to): - if link_to.is_directory: - venv_link = ctx.actions.declare_directory(bin_venv_path) - else: - venv_link = ctx.actions.declare_file(bin_venv_path) - ctx.actions.symlink(output = venv_link, target_file = link_to) + symlink_from = "{}/{}".format(ctx.label.package, bin_venv_path) + runfiles_symlinks[symlink_from] = link_to + else: venv_link = ctx.actions.declare_symlink(bin_venv_path) venv_link_rf_path = runfiles_root_path(ctx, venv_link.short_path) @@ -64,9 +69,12 @@ def create_venv_app_files(ctx, deps, venv_dir_map): to = link_to, ) ctx.actions.symlink(output = venv_link, target_path = rel_path) - venv_files.append(venv_link) + venv_files.append(venv_link) - return venv_files + return struct( + venv_files = venv_files, + runfiles_symlinks = runfiles_symlinks, + ) # Visible for testing def build_link_map(ctx, entries): From 179e2cbf6a441159444d7e9d6c123b5590066249 Mon Sep 17 00:00:00 2001 From: Tim Date: Tue, 11 Nov 2025 17:07:06 -0800 Subject: [PATCH 518/922] fix(gazelle): correct runfiles path handling in gazelle_python_manifest test (#3398) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit ## Summary This PR fixes the `gazelle_python_manifest.test` failure on Linux CI by correcting the runfiles path handling in both the Bazel rule definition and the Go test code. Fixes #3397 ## Problem The test was failing on Linux but passing on macOS due to inconsistent file path handling: - Used `$(rootpath)` instead of `$(rlocationpath)` in the Bazel rule - Resolved runfiles paths but then didn't use the resolved values See issue #3397 for full technical details. ## Changes ### `gazelle/manifest/defs.bzl` - Line 120: Changed `$(rootpath)` to `$(rlocationpath)` for `_TEST_MANIFEST` - Line 122: Changed `$(rootpath)` to `$(rlocationpath)` for `_TEST_REQUIREMENTS` This makes them consistent with the existing `_TEST_MANIFEST_GENERATOR_HASH` which already used `$(rlocationpath)`. ### `gazelle/manifest/test/test.go` - Line 53: Use `manifestPathResolved` instead of `manifestPath` in `manifestFile.Decode()` - Line 73: Use `requirementsPathResolved` instead of `requirementsPath` in `os.Open()` - Lines 84-86: Use `manifestPathResolved` instead of `manifestPath` in error handling The test was already calling `runfiles.Rlocation()` to resolve the paths, but then wasn't using the resolved values. ## Testing Tested on Linux by running: ```bash cd gazelle/examples/bzlmod_build_file_generation bazel test //:gazelle_python_manifest.test ``` Result: ✅ **PASSED** ## Notes - This fix aligns with Bazel's recommended runfiles handling practices - All changes follow the existing pattern used for `_TEST_MANIFEST_GENERATOR_HASH` - Some code generation was assisted by Claude AI, but the human author has reviewed, tested, and takes full responsibility for all changes per the [contribution guidelines](https://github.com/bazel-contrib/rules_python/blob/main/CONTRIBUTING.md#ai-assisted-contributions) **Disclaimer**: This is my first contribution to this project, so I'm not entirely certain this is the correct contribution method. Please let me know if any changes to the PR process are needed. Happy to make adjustments! --- CHANGELOG.md | 2 ++ gazelle/manifest/defs.bzl | 4 ++-- gazelle/manifest/test/test.go | 43 +++++++++++++++++++---------------- 3 files changed, 28 insertions(+), 21 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index d7403b4f1a..5a71e5f972 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -75,6 +75,8 @@ END_UNRELEASED_TEMPLATE underlying runtime. * (performance) 90% reduction in py_binary/py_test analysis phase cost. ([#3381](https://github.com/bazel-contrib/rules_python/pull/3381)). +* (gazelle) Fix `gazelle_python_manifest.test` so that it accesses manifest files via `runfile` path handling rather than directly ([#3397](https://github.com/bazel-contrib/rules_python/issues/3397)). + {#v0-0-0-added} ### Added diff --git a/gazelle/manifest/defs.bzl b/gazelle/manifest/defs.bzl index 45fdb32e7d..b615c4efc1 100644 --- a/gazelle/manifest/defs.bzl +++ b/gazelle/manifest/defs.bzl @@ -117,9 +117,9 @@ def gazelle_python_manifest( if requirements: attrs = { "env": { - "_TEST_MANIFEST": "$(rootpath {})".format(manifest), + "_TEST_MANIFEST": "$(rlocationpath {})".format(manifest), "_TEST_MANIFEST_GENERATOR_HASH": "$(rlocationpath {})".format(manifest_generator_hash), - "_TEST_REQUIREMENTS": "$(rootpath {})".format(requirements), + "_TEST_REQUIREMENTS": "$(rlocationpath {})".format(requirements), }, "size": "small", } diff --git a/gazelle/manifest/test/test.go b/gazelle/manifest/test/test.go index 5804a7102e..77b354b495 100644 --- a/gazelle/manifest/test/test.go +++ b/gazelle/manifest/test/test.go @@ -26,23 +26,32 @@ import ( "path/filepath" "testing" - "github.com/bazelbuild/rules_go/go/runfiles" "github.com/bazel-contrib/rules_python/gazelle/manifest" + "github.com/bazelbuild/rules_go/go/runfiles" ) -func TestGazelleManifestIsUpdated(t *testing.T) { - requirementsPath := os.Getenv("_TEST_REQUIREMENTS") - if requirementsPath == "" { - t.Fatal("_TEST_REQUIREMENTS must be set") +// getResolvedRunfile resolves an environment variable to a runfiles path. +// It handles getting the env var, checking it's set, and resolving it through +// the runfiles mechanism, providing detailed error messages if anything fails. +func getResolvedRunfile(t *testing.T, envVar string) string { + t.Helper() + path := os.Getenv(envVar) + if path == "" { + t.Fatalf("%s must be set", envVar) } - - manifestPath := os.Getenv("_TEST_MANIFEST") - if manifestPath == "" { - t.Fatal("_TEST_MANIFEST must be set") + resolvedPath, err := runfiles.Rlocation(path) + if err != nil { + t.Fatalf("failed to resolve runfiles path for %s (%q): %v", envVar, path, err) } + return resolvedPath +} + +func TestGazelleManifestIsUpdated(t *testing.T) { + requirementsPathResolved := getResolvedRunfile(t, "_TEST_REQUIREMENTS") + manifestPathResolved := getResolvedRunfile(t, "_TEST_MANIFEST") manifestFile := new(manifest.File) - if err := manifestFile.Decode(manifestPath); err != nil { + if err := manifestFile.Decode(manifestPathResolved); err != nil { t.Fatalf("decoding manifest file: %v", err) } @@ -50,11 +59,7 @@ func TestGazelleManifestIsUpdated(t *testing.T) { t.Fatal("failed to find the Gazelle manifest file integrity") } - manifestGeneratorHashPath, err := runfiles.Rlocation( - os.Getenv("_TEST_MANIFEST_GENERATOR_HASH")) - if err != nil { - t.Fatalf("failed to resolve runfiles path of manifest: %v", err) - } + manifestGeneratorHashPath := getResolvedRunfile(t, "_TEST_MANIFEST_GENERATOR_HASH") manifestGeneratorHash, err := os.Open(manifestGeneratorHashPath) if err != nil { @@ -62,9 +67,9 @@ func TestGazelleManifestIsUpdated(t *testing.T) { } defer manifestGeneratorHash.Close() - requirements, err := os.Open(requirementsPath) + requirements, err := os.Open(requirementsPathResolved) if err != nil { - t.Fatalf("opening %q: %v", requirementsPath, err) + t.Fatalf("opening %q: %v", requirementsPathResolved, err) } defer requirements.Close() @@ -73,9 +78,9 @@ func TestGazelleManifestIsUpdated(t *testing.T) { t.Fatalf("verifying integrity: %v", err) } if !valid { - manifestRealpath, err := filepath.EvalSymlinks(manifestPath) + manifestRealpath, err := filepath.EvalSymlinks(manifestPathResolved) if err != nil { - t.Fatalf("evaluating symlink %q: %v", manifestPath, err) + t.Fatalf("evaluating symlink %q: %v", manifestPathResolved, err) } t.Errorf( "%q is out-of-date. Follow the update instructions in that file to resolve this", From 7c6b109db7e7fdece78561dc62ef8f7c92185734 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Wed, 12 Nov 2025 22:03:17 -0800 Subject: [PATCH 519/922] tests: set --windows_enable_symlinks in bzlmod example (#3409) The `//tests:version_test_binary_*` targets fail when run on Bazel 8 with Windows with a "permission denied" error with the Python executable. The notable thing about these tests is they're shell tests with a data dependency on the Python program, so a symlink is created to refer to the Python program. As best I can tell, Bazel 8 isn't creating the symlink quite right, so Windows considers the file non-executable. Setting `--windows_enable_symlinks` fixes this, but it isn't clear why. Presumably it's telling Bazel to create symlinks more properly. --- examples/bzlmod/.bazelrc | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/examples/bzlmod/.bazelrc b/examples/bzlmod/.bazelrc index ca83047ccc..7c92800d76 100644 --- a/examples/bzlmod/.bazelrc +++ b/examples/bzlmod/.bazelrc @@ -1,3 +1,8 @@ +# Starting with Bazel 8, Windows requires this flag in order +# for symlinks to work properly (namely, so that sh_test with +# py_binary as a data dependency gets symlinks that are executable) +startup --windows_enable_symlinks + common --enable_bzlmod common --lockfile_mode=update From 3440572f5d8bcb45df8db93ecc102475d6508bee Mon Sep 17 00:00:00 2001 From: Martin Medler <36563496+martis42@users.noreply.github.com> Date: Mon, 17 Nov 2025 05:29:36 +0100 Subject: [PATCH 520/922] docs: Explain why the `lock` rule has no implicit test target (#3411) Fixes https://github.com/bazel-contrib/rules_python/issues/3400 --- python/uv/lock.bzl | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-) diff --git a/python/uv/lock.bzl b/python/uv/lock.bzl index 82b00bc2d2..7bcca780a0 100644 --- a/python/uv/lock.bzl +++ b/python/uv/lock.bzl @@ -21,8 +21,13 @@ Differences with the legacy {obj}`compile_pip_requirements` rule: - This does not error out if the output file does not exist yet. - Supports transitions out of the box. -Note, this does not provide a `test` target, if you would like to add a test -target that always does the locking automatically to ensure that the +Note, this does not provide a test target like {obj}`compile_pip_requirements` does. +The `uv pip compile` command is not hermetic and thus a test based on it would most likely be flaky: +- It may require auth injected into it, so most likely it requires a local tag added so that the bazel action runs without sandboxing. +- It requires network access. + +Given those points, a test target should be an explicit and properly documented target and not a hidden implicit target. +If, you would like to add a test target that always does the locking automatically to ensure that the `requirements.txt` file is up-to-date, add something similar to: ```starlark From da23db5b4dc095c597a0fc792f04e7ff50d37142 Mon Sep 17 00:00:00 2001 From: Josh Cannon Date: Mon, 17 Nov 2025 23:13:13 -0600 Subject: [PATCH 521/922] refactor(gazelle): Generate a modules map per wheel, then merge (#3415) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit This change internally splits modules mapping generation to be per-wheel, with a final quick "merge" action at the end. The idea is to make this process both concurrent and cached (courtesy of Bazel), which can be ideal for codebases with a large set of requirements (as many monorepos end up doing) Note that the `generator.py` interface changed. This seemed internal, so I didn't mark it breaking (but this change could actually just leave the generator alone, since the current implementation is fine with 1 wheel). I ran this on the work repo and saw no change in output (but as I edited a single requirement, the overall process was fast ⚡ ) --- CHANGELOG.md | 1 + gazelle/modules_mapping/BUILD.bazel | 18 ++++++++ gazelle/modules_mapping/def.bzl | 45 +++++++++++++------ gazelle/modules_mapping/generator.py | 32 +++++++------- gazelle/modules_mapping/merger.py | 45 +++++++++++++++++++ gazelle/modules_mapping/test_merger.py | 61 ++++++++++++++++++++++++++ 6 files changed, 174 insertions(+), 28 deletions(-) create mode 100644 gazelle/modules_mapping/merger.py create mode 100644 gazelle/modules_mapping/test_merger.py diff --git a/CHANGELOG.md b/CHANGELOG.md index 5a71e5f972..7d3723c7af 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -62,6 +62,7 @@ END_UNRELEASED_TEMPLATE {#v0-0-0-changed} ### Changed * (toolchains) Use toolchains from the [20251031] release. +* (gazelle) Internally split modules mapping generation to be per-wheel for concurrency and caching. {#v0-0-0-fixed} ### Fixed diff --git a/gazelle/modules_mapping/BUILD.bazel b/gazelle/modules_mapping/BUILD.bazel index 3a9a8a47f3..3423f34e51 100644 --- a/gazelle/modules_mapping/BUILD.bazel +++ b/gazelle/modules_mapping/BUILD.bazel @@ -9,6 +9,12 @@ py_binary( visibility = ["//visibility:public"], ) +py_binary( + name = "merger", + srcs = ["merger.py"], + visibility = ["//visibility:public"], +) + copy_file( name = "pytest_wheel", src = "@pytest//file", @@ -33,6 +39,18 @@ py_test( deps = [":generator"], ) +py_test( + name = "test_merger", + srcs = ["test_merger.py"], + data = [ + "django_types_wheel", + "pytest_wheel", + ], + imports = ["."], + main = "test_merger.py", + deps = [":merger"], +) + filegroup( name = "distribution", srcs = glob(["**"]), diff --git a/gazelle/modules_mapping/def.bzl b/gazelle/modules_mapping/def.bzl index 48a5477b93..74d3c9ef35 100644 --- a/gazelle/modules_mapping/def.bzl +++ b/gazelle/modules_mapping/def.bzl @@ -30,25 +30,39 @@ def _modules_mapping_impl(ctx): transitive = [dep[DefaultInfo].files for dep in ctx.attr.wheels] + [dep[DefaultInfo].data_runfiles.files for dep in ctx.attr.wheels], ) - args = ctx.actions.args() + # Run the generator once per-wheel (to leverage caching) + per_wheel_outputs = [] + for idx, whl in enumerate(all_wheels.to_list()): + wheel_modules_mapping = ctx.actions.declare_file("{}.{}".format(modules_mapping.short_path, idx)) + args = ctx.actions.args() + args.add("--output_file", wheel_modules_mapping.path) + if ctx.attr.include_stub_packages: + args.add("--include_stub_packages") + args.add_all("--exclude_patterns", ctx.attr.exclude_patterns) + args.add("--wheel", whl.path) - # Spill parameters to a file prefixed with '@'. Note, the '@' prefix is the same - # prefix as used in the `generator.py` in `fromfile_prefix_chars` attribute. - args.use_param_file(param_file_arg = "@%s") - args.set_param_file_format(format = "multiline") - if ctx.attr.include_stub_packages: - args.add("--include_stub_packages") - args.add("--output_file", modules_mapping) - args.add_all("--exclude_patterns", ctx.attr.exclude_patterns) - args.add_all("--wheels", all_wheels) + ctx.actions.run( + inputs = [whl], + outputs = [wheel_modules_mapping], + executable = ctx.executable._generator, + arguments = [args], + use_default_shell_env = False, + ) + per_wheel_outputs.append(wheel_modules_mapping) + + # Then merge the individual JSONs together + merge_args = ctx.actions.args() + merge_args.add("--output", modules_mapping.path) + merge_args.add_all("--inputs", [f.path for f in per_wheel_outputs]) ctx.actions.run( - inputs = all_wheels, + inputs = per_wheel_outputs, outputs = [modules_mapping], - executable = ctx.executable._generator, - arguments = [args], + executable = ctx.executable._merger, + arguments = [merge_args], use_default_shell_env = False, ) + return [DefaultInfo(files = depset([modules_mapping]))] modules_mapping = rule( @@ -79,6 +93,11 @@ modules_mapping = rule( default = "//modules_mapping:generator", executable = True, ), + "_merger": attr.label( + cfg = "exec", + default = "//modules_mapping:merger", + executable = True, + ), }, doc = "Creates a modules_mapping.json file for mapping module names to wheel distribution names.", ) diff --git a/gazelle/modules_mapping/generator.py b/gazelle/modules_mapping/generator.py index ea11f3e236..611910c669 100644 --- a/gazelle/modules_mapping/generator.py +++ b/gazelle/modules_mapping/generator.py @@ -96,8 +96,7 @@ def module_for_path(self, path, whl): ext = "".join(pathlib.Path(root).suffixes) module = root[: -len(ext)].replace("/", ".") if not self.is_excluded(module): - if not self.is_excluded(module): - self.mapping[module] = wheel_name + self.mapping[module] = wheel_name def is_excluded(self, module): for pattern in self.excluded_patterns: @@ -105,14 +104,20 @@ def is_excluded(self, module): return True return False - # run is the entrypoint for the generator. - def run(self, wheels): - for whl in wheels: - try: - self.dig_wheel(whl) - except AssertionError as error: - print(error, file=self.stderr) - return 1 + def run(self, wheel: pathlib.Path) -> int: + """ + Entrypoint for the generator. + + Args: + wheel: The path to the wheel file (`.whl`) + Returns: + Exit code (for `sys.exit`) + """ + try: + self.dig_wheel(wheel) + except AssertionError as error: + print(error, file=self.stderr) + return 1 self.simplify() mapping_json = json.dumps(self.mapping) with open(self.output_file, "w") as f: @@ -152,16 +157,13 @@ def data_has_purelib_or_platlib(path): parser = argparse.ArgumentParser( prog="generator", description="Generates the modules mapping used by the Gazelle manifest.", - # Automatically read parameters from a file. Note, the '@' is the same prefix - # as set in the 'args.use_param_file' in the bazel rule. - fromfile_prefix_chars="@", ) parser.add_argument("--output_file", type=str) parser.add_argument("--include_stub_packages", action="store_true") parser.add_argument("--exclude_patterns", nargs="+", default=[]) - parser.add_argument("--wheels", nargs="+", default=[]) + parser.add_argument("--wheel", type=pathlib.Path) args = parser.parse_args() generator = Generator( sys.stderr, args.output_file, args.exclude_patterns, args.include_stub_packages ) - sys.exit(generator.run(args.wheels)) + sys.exit(generator.run(args.wheel)) diff --git a/gazelle/modules_mapping/merger.py b/gazelle/modules_mapping/merger.py new file mode 100644 index 0000000000..deb0cb2666 --- /dev/null +++ b/gazelle/modules_mapping/merger.py @@ -0,0 +1,45 @@ +#!/usr/bin/env python3 +"""Merges multiple modules_mapping.json files into a single file.""" + +import argparse +import json +from pathlib import Path + + +def merge_modules_mappings(input_files: list[Path], output_file: Path) -> None: + """Merge multiple modules_mapping.json files into one. + + Args: + input_files: List of paths to input JSON files to merge + output_file: Path where the merged output should be written + """ + merged_mapping = {} + for input_file in input_files: + mapping = json.loads(input_file.read_text()) + # Merge the mappings, with later files overwriting earlier ones + # if there are conflicts + merged_mapping.update(mapping) + + output_file.write_text(json.dumps(merged_mapping)) + + +if __name__ == "__main__": + parser = argparse.ArgumentParser( + description="Merge multiple modules_mapping.json files" + ) + parser.add_argument( + "--output", + required=True, + type=Path, + help="Output file path for merged mapping", + ) + parser.add_argument( + "--inputs", + required=True, + nargs="+", + type=Path, + help="Input JSON files to merge", + ) + + args = parser.parse_args() + merge_modules_mappings(args.inputs, args.output) diff --git a/gazelle/modules_mapping/test_merger.py b/gazelle/modules_mapping/test_merger.py new file mode 100644 index 0000000000..6260fdd6ff --- /dev/null +++ b/gazelle/modules_mapping/test_merger.py @@ -0,0 +1,61 @@ +import pathlib +import unittest +import json +import tempfile + +from merger import merge_modules_mappings + + +class MergerTest(unittest.TestCase): + _tmpdir: tempfile.TemporaryDirectory + + def setUp(self) -> None: + super().setUp() + self._tmpdir = tempfile.TemporaryDirectory() + + def tearDown(self) -> None: + super().tearDown() + self._tmpdir.cleanup() + del self._tmpdir + + @property + def tmppath(self) -> pathlib.Path: + return pathlib.Path(self._tmpdir.name) + + def make_input(self, mapping: dict[str, str]) -> pathlib.Path: + _fd, file = tempfile.mkstemp(suffix=".json", dir=self._tmpdir.name) + path = pathlib.Path(file) + path.write_text(json.dumps(mapping)) + return path + + def test_merger(self): + output_path = self.tmppath / "output.json" + merge_modules_mappings( + [ + self.make_input( + { + "_pytest": "pytest", + "_pytest.__init__": "pytest", + "_pytest._argcomplete": "pytest", + "_pytest.config.argparsing": "pytest", + } + ), + self.make_input({"django_types": "django_types"}), + ], + output_path, + ) + + self.assertEqual( + { + "_pytest": "pytest", + "_pytest.__init__": "pytest", + "_pytest._argcomplete": "pytest", + "_pytest.config.argparsing": "pytest", + "django_types": "django_types", + }, + json.loads(output_path.read_text()), + ) + + +if __name__ == "__main__": + unittest.main() From 1f6cc5cb610fd431e034b537d72d73db2d39ebe3 Mon Sep 17 00:00:00 2001 From: Alexey Preobrazhenskiy Date: Wed, 19 Nov 2025 00:53:59 +0100 Subject: [PATCH 522/922] docs: fix markdown (#3417) There's an extra code block open tag that gets highlighted as an error on readthedocs. Screenshot 2025-11-18 at 14 05 02 --- docs/howto/common-deps-with-multiple-pypi-versions.md | 2 -- 1 file changed, 2 deletions(-) diff --git a/docs/howto/common-deps-with-multiple-pypi-versions.md b/docs/howto/common-deps-with-multiple-pypi-versions.md index 3b933d22f4..eb45d72b52 100644 --- a/docs/howto/common-deps-with-multiple-pypi-versions.md +++ b/docs/howto/common-deps-with-multiple-pypi-versions.md @@ -43,8 +43,6 @@ rules_python_config.add_transition_setting( # File: BUILD.bazel -```bzl - load("@bazel_skylib//rules:common_settings.bzl", "string_flag") string_flag( From 846dfd0c5699da8cc2e42067e8ccd0f5a1704328 Mon Sep 17 00:00:00 2001 From: Ignas Anikevicius <240938+aignas@users.noreply.github.com> Date: Thu, 20 Nov 2025 15:27:34 +0900 Subject: [PATCH 523/922] feat(toolchain): drop 3.8 and print info level messages about it (#3387) Before this PR we had to have at least one 3.8 toolchain to not break things. With this we should be good to drop it. Any python_version 3.8 registrations will be dropped if there are no actual URLs configured, which means that 3.8 will not be selected. The same with pip.parse, we will just ignore it and won't add it to the hub. In order to ensure that `is_python_3.x` flags continue working, we just alias them to `@platforms//:incompatible`. No deprecation message is printed. Work towards #2704 Next step for anyone interested and who has more time than me these days: - [ ] Remove the 3.9 URLs and add them individually to our examples to show that one can do that. - [ ] Update the examples to no longer use 3.9, because it is a maintenance burden. --- python/private/config_settings.bzl | 40 ++++++++++++---- python/private/full_version.bzl | 5 +- python/private/pypi/hub_builder.bzl | 31 ++++++++++--- python/private/python.bzl | 11 +++++ python/versions.bzl | 12 ----- tests/python/python_tests.bzl | 71 +++++++++++++++++++++++++++++ 6 files changed, 142 insertions(+), 28 deletions(-) diff --git a/python/private/config_settings.bzl b/python/private/config_settings.bzl index 3089b9c6cf..91fbbba8cb 100644 --- a/python/private/config_settings.bzl +++ b/python/private/config_settings.bzl @@ -35,7 +35,14 @@ If the value is missing, then the default value is being used, see documentation # access it, but it's not intended for general public usage. _NOT_ACTUALLY_PUBLIC = ["//visibility:public"] -def construct_config_settings(*, name, default_version, versions, minor_mapping, documented_flags): # buildifier: disable=function-docstring +def construct_config_settings( + *, + name, + default_version, + versions, + minor_mapping, + compat_lowest_version = "3.8", + documented_flags): # buildifier: disable=function-docstring """Create a 'python_version' config flag and construct all config settings used in rules_python. This mainly includes the targets that are used in the toolchain and pip hub @@ -46,6 +53,8 @@ def construct_config_settings(*, name, default_version, versions, minor_mapping, default_version: {type}`str` the default value for the `python_version` flag. versions: {type}`list[str]` A list of versions to build constraint settings for. minor_mapping: {type}`dict[str, str]` A mapping from `X.Y` to `X.Y.Z` python versions. + compat_lowest_version: {type}`str` The version that we should use as the lowest available + version for `is_python_3.X` flags. documented_flags: {type}`list[str]` The labels of the documented settings that affect build configuration. """ @@ -69,21 +78,21 @@ def construct_config_settings(*, name, default_version, versions, minor_mapping, ) _reverse_minor_mapping = {full: minor for minor, full in minor_mapping.items()} - for version in versions: - minor_version = _reverse_minor_mapping.get(version) + for ver in versions: + minor_version = _reverse_minor_mapping.get(ver) if not minor_version: native.config_setting( - name = "is_python_{}".format(version), - flag_values = {":python_version": version}, + name = "is_python_{}".format(ver), + flag_values = {":python_version": ver}, visibility = ["//visibility:public"], ) continue # Also need to match the minor version when using - name = "is_python_{}".format(version) + name = "is_python_{}".format(ver) native.config_setting( name = "_" + name, - flag_values = {":python_version": version}, + flag_values = {":python_version": ver}, visibility = ["//visibility:public"], ) @@ -94,7 +103,7 @@ def construct_config_settings(*, name, default_version, versions, minor_mapping, selects.config_setting_group( name = "_{}_group".format(name), match_any = [ - ":_is_python_{}".format(version), + ":_is_python_{}".format(ver), ":is_python_{}".format(minor_version), ], visibility = ["//visibility:private"], @@ -109,13 +118,28 @@ def construct_config_settings(*, name, default_version, versions, minor_mapping, # It's private because matching the concept of e.g. "3.8" value is done # using the `is_python_X.Y` config setting group, which is aware of the # minor versions that could match instead. + first_minor = None for minor in minor_mapping.keys(): + ver = version.parse(minor) + if first_minor == None or version.is_lt(ver, first_minor): + first_minor = ver + native.config_setting( name = "is_python_{}".format(minor), flag_values = {_PYTHON_VERSION_MAJOR_MINOR_FLAG: minor}, visibility = ["//visibility:public"], ) + # This is a compatibility layer to ensure that `select` statements don't break out right + # when the toolchains for EOL minor versions are no longer registered. + compat_lowest_version = version.parse(compat_lowest_version) + for minor in range(compat_lowest_version.release[-1], first_minor.release[-1]): + native.alias( + name = "is_python_3.{}".format(minor), + actual = "@platforms//:incompatible", + visibility = ["//visibility:public"], + ) + _current_config( name = "current_config", build_setting_default = "", diff --git a/python/private/full_version.bzl b/python/private/full_version.bzl index 0292d6c77d..0be5b44daf 100644 --- a/python/private/full_version.bzl +++ b/python/private/full_version.bzl @@ -14,12 +14,13 @@ """A small helper to ensure that we are working with full versions.""" -def full_version(*, version, minor_mapping): +def full_version(*, version, minor_mapping, fail_on_err = True): """Return a full version. Args: version: {type}`str` the version in `X.Y` or `X.Y.Z` format. minor_mapping: {type}`dict[str, str]` mapping between `X.Y` to `X.Y.Z` format. + fail_on_err: {type}`bool` whether to fail on error or return `None` instead. Returns: a full version given the version string. If the string is already a @@ -31,6 +32,8 @@ def full_version(*, version, minor_mapping): parts = version.split(".") if len(parts) == 3: return version + elif not fail_on_err: + return None elif len(parts) == 2: fail( "Unknown Python version '{}', available values are: {}".format( diff --git a/python/private/pypi/hub_builder.bzl b/python/private/pypi/hub_builder.bzl index 7cf60ff85f..bd6008128b 100644 --- a/python/private/pypi/hub_builder.bzl +++ b/python/private/pypi/hub_builder.bzl @@ -114,9 +114,29 @@ def _pip_parse(self, module_ctx, pip_attr): version = python_version, )) - self._platforms[python_version] = _platforms( - python_version = python_version, + full_python_version = full_version( + version = python_version, minor_mapping = self._minor_mapping, + fail_on_err = False, + ) + if not full_python_version: + # NOTE @aignas 2025-11-18: If the python version is not present in our + # minor_mapping, then we will not register any packages and then the + # select in the hub repository will fail, which will prompt the user to + # configure the toolchain correctly and move forward. + self._logger.info(lambda: ( + "Ignoring pip python version '{version}' for hub " + + "'{hub}' in module '{module}' because there is no registered " + + "toolchain for it." + ).format( + hub = self.name, + module = self.module_name, + version = python_version, + )) + return + + self._platforms[python_version] = _platforms( + python_version = full_python_version, config = self._config, ) _set_get_index_urls(self, pip_attr) @@ -280,13 +300,10 @@ def _detect_interpreter(self, pip_attr): path = pip_attr.python_interpreter, ) -def _platforms(*, python_version, minor_mapping, config): +def _platforms(*, python_version, config): platforms = {} python_version = version.parse( - full_version( - version = python_version, - minor_mapping = minor_mapping, - ), + python_version, strict = True, ) diff --git a/python/private/python.bzl b/python/private/python.bzl index a1fe80e0ce..22f4753a62 100644 --- a/python/private/python.bzl +++ b/python/private/python.bzl @@ -268,7 +268,18 @@ def _python_impl(module_ctx): full_python_version = full_version( version = toolchain_info.python_version, minor_mapping = py.config.minor_mapping, + fail_on_err = False, ) + if not full_python_version: + logger.info(lambda: ( + "The actual toolchain for python_version '{version}' " + + "has not been registered, but was requested, please configure a toolchain " + + "to be actually downloaded and setup" + ).format( + version = toolchain_info.python_version, + )) + continue + kwargs = { "python_version": full_python_version, "register_coverage_tool": toolchain_info.register_coverage_tool, diff --git a/python/versions.bzl b/python/versions.bzl index 7e1b36b207..842fb39658 100644 --- a/python/versions.bzl +++ b/python/versions.bzl @@ -54,17 +54,6 @@ DEFAULT_RELEASE_BASE_URL = "https://github.com/astral-sh/python-build-standalone # # buildifier: disable=unsorted-dict-items TOOL_VERSIONS = { - "3.8.20": { - "url": "20241002/cpython-{python_version}+20241002-{platform}-{build}.tar.gz", - "sha256": { - "aarch64-apple-darwin": "2ddfc04bdb3e240f30fb782fa1deec6323799d0e857e0b63fa299218658fd3d4", - "aarch64-unknown-linux-gnu": "9d8798f9e79e0fc0f36fcb95bfa28a1023407d51a8ea5944b4da711f1f75f1ed", - "x86_64-apple-darwin": "68d060cd373255d2ca5b8b3441363d5aa7cc45b0c11bbccf52b1717c2b5aa8bb", - "x86_64-pc-windows-msvc": "41b6709fec9c56419b7de1940d1f87fa62045aff81734480672dcb807eedc47e", - "x86_64-unknown-linux-gnu": "285e141c36f88b2e9357654c5f77d1f8fb29cc25132698fe35bb30d787f38e87", - }, - "strip_prefix": "python", - }, "3.9.25": { "url": "20251031/cpython-{python_version}+20251031-{platform}-{build}.tar.gz", "sha256": { @@ -872,7 +861,6 @@ TOOL_VERSIONS = { # buildifier: disable=unsorted-dict-items MINOR_MAPPING = { - "3.8": "3.8.20", "3.9": "3.9.25", "3.10": "3.10.19", "3.11": "3.11.14", diff --git a/tests/python/python_tests.bzl b/tests/python/python_tests.bzl index f2e87274f8..ff02cc859e 100644 --- a/tests/python/python_tests.bzl +++ b/tests/python/python_tests.bzl @@ -707,6 +707,77 @@ def _test_register_all_versions(env): _tests.append(_test_register_all_versions) +def _test_ignore_unsupported_versions(env): + py = parse_modules( + module_ctx = _mock_mctx( + _mod( + name = "my_module", + is_root = True, + toolchain = [ + _toolchain("3.11"), + _toolchain("3.12"), + _toolchain("3.13", is_default = True), + ], + single_version_override = [ + _single_version_override( + python_version = "3.13.0", + sha256 = { + "aarch64-unknown-linux-gnu": "deadbeef", + }, + urls = ["example.org"], + ), + ], + single_version_platform_override = [ + _single_version_platform_override( + sha256 = "deadb00f", + urls = ["something.org"], + platform = "aarch64-unknown-linux-gnu", + python_version = "3.13.99", + ), + ], + override = [ + _override( + base_url = "", + available_python_versions = ["3.12.4", "3.13.0", "3.13.1"], + minor_mapping = { + "3.12": "3.12.4", + "3.13": "3.13.1", + }, + ), + ], + ), + ), + logger = repo_utils.logger(verbosity_level = 0, name = "python"), + ) + + env.expect.that_str(py.default_python_version).equals("3.13") + env.expect.that_collection(py.config.default["tool_versions"].keys()).contains_exactly([ + "3.12.4", + "3.13.0", + "3.13.1", + ]) + env.expect.that_dict(py.config.minor_mapping).contains_exactly({ + # The mapping is calculated automatically + "3.12": "3.12.4", + "3.13": "3.13.1", + }) + env.expect.that_collection(py.toolchains).contains_exactly([ + struct( + name = name, + python_version = version, + register_coverage_tool = False, + ) + for name, version in { + # NOTE: that '3.11' wont be actually registered and present in the + # `tool_versions` above. + "python_3_11": "3.11", + "python_3_12": "3.12", + "python_3_13": "3.13", + }.items() + ]) + +_tests.append(_test_ignore_unsupported_versions) + def _test_add_patches(env): py = parse_modules( module_ctx = _mock_mctx( From c2ff89f00f4e860cc79d169db07b3b8bedcf384f Mon Sep 17 00:00:00 2001 From: Fabian Meumertzheim Date: Sat, 22 Nov 2025 03:47:38 +0100 Subject: [PATCH 524/922] fix: Avoid C++ toolchain requirement if possible (#2919) By making use of the new `launcher_maker_toolchain` in Bazel 9, rules_python can avoid the requirement for a C++ toolchain targeting the target platform if that platform isn't Windows. For example, this makes it possible to cross-compile pure Python targets from one Unix to another. Since Java targets have a dependency on Python targets through the `proguard_allowlister`, this also allows Java targets to be built without any C++ toolchain. --- python/private/internal_config_repo.bzl | 7 +++++- python/private/py_executable.bzl | 25 +++++++++++++------ tests/base_rules/py_executable_base_tests.bzl | 24 ++++++++++++++++++ tests/support/platforms/BUILD.bazel | 8 ++++++ tests/support/platforms/platforms.bzl | 4 +++ 5 files changed, 60 insertions(+), 8 deletions(-) diff --git a/python/private/internal_config_repo.bzl b/python/private/internal_config_repo.bzl index d5192ec44b..b208037c13 100644 --- a/python/private/internal_config_repo.bzl +++ b/python/private/internal_config_repo.bzl @@ -32,6 +32,7 @@ config = struct( enable_pystar = True, enable_pipstar = {enable_pipstar}, enable_deprecation_warnings = {enable_deprecation_warnings}, + bazel_9_or_later = {bazel_9_or_later}, BuiltinPyInfo = getattr(getattr(native, "legacy_globals", None), "PyInfo", {builtin_py_info_symbol}), BuiltinPyRuntimeInfo = getattr(getattr(native, "legacy_globals", None), "PyRuntimeInfo", {builtin_py_runtime_info_symbol}), BuiltinPyCcLinkParamsProvider = getattr(getattr(native, "legacy_globals", None), "PyCcLinkParamsProvider", {builtin_py_cc_link_params_provider}), @@ -87,7 +88,10 @@ _TRANSITION_SETTINGS_DEBUG_TEMPLATE = """ """ def _internal_config_repo_impl(rctx): - if not native.bazel_version or int(native.bazel_version.split(".")[0]) >= 8: + # An empty version signifies a development build, which is treated as + # the latest version. + bazel_major_version = int(native.bazel_version.split(".")[0]) if native.bazel_version else 99999 + if bazel_major_version >= 8: builtin_py_info_symbol = "None" builtin_py_runtime_info_symbol = "None" builtin_py_cc_link_params_provider = "None" @@ -103,6 +107,7 @@ def _internal_config_repo_impl(rctx): builtin_py_info_symbol = builtin_py_info_symbol, builtin_py_runtime_info_symbol = builtin_py_runtime_info_symbol, builtin_py_cc_link_params_provider = builtin_py_cc_link_params_provider, + bazel_9_or_later = str(bazel_major_version >= 9), )) shim_content = _PY_INTERNAL_SHIM diff --git a/python/private/py_executable.bzl b/python/private/py_executable.bzl index 669951e172..99a3dffb49 100644 --- a/python/private/py_executable.bzl +++ b/python/private/py_executable.bzl @@ -18,6 +18,7 @@ load("@bazel_skylib//lib:paths.bzl", "paths") load("@bazel_skylib//lib:structs.bzl", "structs") load("@bazel_skylib//rules:common_settings.bzl", "BuildSettingInfo") load("@rules_cc//cc/common:cc_common.bzl", "cc_common") +load("@rules_python_internal//:rules_python_config.bzl", rp_config = "config") load(":attr_builders.bzl", "attrb") load( ":attributes.bzl", @@ -69,6 +70,7 @@ load(":venv_runfiles.bzl", "create_venv_app_files") _py_builtins = py_internal _EXTERNAL_PATH_PREFIX = "external" _ZIP_RUNFILES_DIRECTORY_NAME = "runfiles" +_LAUNCHER_MAKER_TOOLCHAIN_TYPE = "@bazel_tools//tools/launcher:launcher_maker_toolchain_type" # Non-Google-specific attributes for executables # These attributes are for rules that accept Python sources. @@ -228,17 +230,19 @@ accepting arbitrary Python versions. "@platforms//os:windows", ], ), - "_windows_launcher_maker": lambda: attrb.Label( - default = "@bazel_tools//tools/launcher:launcher_maker", - cfg = "exec", - executable = True, - ), "_zipper": lambda: attrb.Label( cfg = "exec", executable = True, default = "@bazel_tools//tools/zip:zipper", ), }, + { + "_windows_launcher_maker": lambda: attrb.Label( + default = "@bazel_tools//tools/launcher:launcher_maker", + cfg = "exec", + executable = True, + ), + } if not rp_config.bazel_9_or_later else {}, ) def convert_legacy_create_init_to_int(kwargs): @@ -777,6 +781,11 @@ def _create_stage1_bootstrap( is_executable = True, ) +def _find_launcher_maker(ctx): + if rp_config.bazel_9_or_later: + return (ctx.toolchains[_LAUNCHER_MAKER_TOOLCHAIN_TYPE].binary, _LAUNCHER_MAKER_TOOLCHAIN_TYPE) + return (ctx.executable._windows_launcher_maker, None) + def _create_windows_exe_launcher( ctx, *, @@ -796,8 +805,9 @@ def _create_windows_exe_launcher( launch_info.add("1" if use_zip_file else "0", format = "use_zip_file=%s") launcher = ctx.attr._launcher[DefaultInfo].files_to_run.executable + executable, toolchain = _find_launcher_maker(ctx) ctx.actions.run( - executable = ctx.executable._windows_launcher_maker, + executable = executable, arguments = [launcher.path, launch_info, output.path], inputs = [launcher], outputs = [output], @@ -805,6 +815,7 @@ def _create_windows_exe_launcher( progress_message = "Creating launcher for %{label}", # Needed to inherit PATH when using non-MSVC compilers like MinGW use_default_shell_env = True, + toolchain = toolchain, ) def _create_zip_file(ctx, *, output, zip_main, runfiles): @@ -1838,7 +1849,7 @@ def create_executable_rule_builder(implementation, **kwargs): ruleb.ToolchainType(TOOLCHAIN_TYPE), ruleb.ToolchainType(EXEC_TOOLS_TOOLCHAIN_TYPE, mandatory = False), ruleb.ToolchainType("@bazel_tools//tools/cpp:toolchain_type", mandatory = False), - ], + ] + ([ruleb.ToolchainType(_LAUNCHER_MAKER_TOOLCHAIN_TYPE)] if rp_config.bazel_9_or_later else []), cfg = dict( implementation = _transition_executable_impl, inputs = TRANSITION_LABELS + [labels.PYTHON_VERSION], diff --git a/tests/base_rules/py_executable_base_tests.bzl b/tests/base_rules/py_executable_base_tests.bzl index e41bc2c022..ed1a55021d 100644 --- a/tests/base_rules/py_executable_base_tests.bzl +++ b/tests/base_rules/py_executable_base_tests.bzl @@ -14,6 +14,7 @@ """Tests common to py_binary and py_test (executable rules).""" load("@rules_python//python:py_runtime_info.bzl", RulesPythonPyRuntimeInfo = "PyRuntimeInfo") +load("@rules_python_internal//:rules_python_config.bzl", rp_config = "config") load("@rules_testing//lib:analysis_test.bzl", "analysis_test") load("@rules_testing//lib:truth.bzl", "matching") load("@rules_testing//lib:util.bzl", rt_util = "util") @@ -114,6 +115,29 @@ def _test_basic_zip_impl(env, target): _tests.append(_test_basic_zip) +def _test_cross_compile_to_unix(name, config): + rt_util.helper_target( + config.rule, + name = name + "_subject", + main_module = "dummy", + ) + analysis_test( + name = name, + impl = _test_cross_compile_to_unix_impl, + target = name + "_subject", + # Cross-compilation of py_test fails since the default test toolchain + # requires an execution platform that matches the target platform. + config_settings = { + "//command_line_option:platforms": [platform_targets.EXOTIC_UNIX], + } if rp_config.bazel_9_or_later and not "py_test" in str(config.rule) else {}, + expect_failure = True, + ) + +def _test_cross_compile_to_unix_impl(_env, _target): + pass + +_tests.append(_test_cross_compile_to_unix) + def _test_executable_in_runfiles(name, config): rt_util.helper_target( config.rule, diff --git a/tests/support/platforms/BUILD.bazel b/tests/support/platforms/BUILD.bazel index 41d7936394..eeb7ccb597 100644 --- a/tests/support/platforms/BUILD.bazel +++ b/tests/support/platforms/BUILD.bazel @@ -75,3 +75,11 @@ platform( "@platforms//cpu:aarch64", ], ) + +platform( + name = "exotic_unix", + constraint_values = [ + "@platforms//os:linux", + "@platforms//cpu:s390x", + ], +) diff --git a/tests/support/platforms/platforms.bzl b/tests/support/platforms/platforms.bzl index af049f202c..92a1d61844 100644 --- a/tests/support/platforms/platforms.bzl +++ b/tests/support/platforms/platforms.bzl @@ -10,4 +10,8 @@ platform_targets = struct( WINDOWS = Label("//tests/support/platforms:windows"), WINDOWS_AARCH64 = Label("//tests/support/platforms:windows_aarch64"), WINDOWS_X86_64 = Label("//tests/support/platforms:windows_x86_64"), + + # Unspecified Unix platform that is unlikely to be the host platform in CI, + # but still provides a Python toolchain. + EXOTIC_UNIX = Label("//tests/support/platforms:exotic_unix"), ) From b57eba907eb10de98954ef915be82d1558b62f56 Mon Sep 17 00:00:00 2001 From: Ignas Anikevicius <240938+aignas@users.noreply.github.com> Date: Sat, 22 Nov 2025 12:06:36 +0900 Subject: [PATCH 525/922] doc: add documentation to the changelog about the removal of 3.8 (#3418) Add changelog for #3387 --- CHANGELOG.md | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 7d3723c7af..17aa502149 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -55,9 +55,13 @@ END_UNRELEASED_TEMPLATE {#v0-0-0-removed} ### Removed +* (toolchain) Remove all of the python 3.8 toolchain support out of the box. Users need + to pass the `TOOL_VERSIONS` that include 3.8 toolchains or use the `bzlmod` APIs to add + them back. This means any hub `pip.parse` calls that target `3.8` will be ignored from + now on. ([#2704](https://github.com/bazel-contrib/rules_python/issues/2704)) * (toolchain) Remove all of the python 3.9 toolchain versions except for the `3.9.25`. This version has reached EOL and will no longer receive any security fixes, please update to - `3.10` or above. + `3.10` or above. ([#2704](https://github.com/bazel-contrib/rules_python/issues/2704)) {#v0-0-0-changed} ### Changed From 23e605cb1efb8a6f3c6831e2efaffacd71a215f0 Mon Sep 17 00:00:00 2001 From: Keith Smiley Date: Fri, 21 Nov 2025 19:35:49 -0800 Subject: [PATCH 526/922] fix: make python_headers targets compatible with layering checks (#3420) Previously the headers from this target were available because it depends on python_headers_abi3, but that fails downstream layering_checks since the headers weren't direct. Now it re-exports the same headers as that underlying target to satisfy those checks. Inspectable with: ``` bazel cquery --output=starlark --starlark:expr 'providers(target)["@@rules_cc+//cc/private:cc_info.bzl%CcInfo"].compilation_context.direct_public_headers' @rules_python//python/cc:current_py_cc_headers ``` --------- Co-authored-by: Richard Levasseur Co-authored-by: Richard Levasseur --- CHANGELOG.md | 2 ++ python/private/hermetic_runtime_repo_setup.bzl | 1 + 2 files changed, 3 insertions(+) diff --git a/CHANGELOG.md b/CHANGELOG.md index 17aa502149..4f1911f99a 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -78,6 +78,8 @@ END_UNRELEASED_TEMPLATE ([#3085](https://github.com/bazel-contrib/rules_python/issues/3085)). * (toolchains) local toolchains now tell the `sys.abiflags` value of the underlying runtime. +* (toolchains) The `python_headers` target is now compatible with + layering_check. * (performance) 90% reduction in py_binary/py_test analysis phase cost. ([#3381](https://github.com/bazel-contrib/rules_python/pull/3381)). * (gazelle) Fix `gazelle_python_manifest.test` so that it accesses manifest files via `runfile` path handling rather than directly ([#3397](https://github.com/bazel-contrib/rules_python/issues/3397)). diff --git a/python/private/hermetic_runtime_repo_setup.bzl b/python/private/hermetic_runtime_repo_setup.bzl index 4bcc1c1512..c3c275546d 100644 --- a/python/private/hermetic_runtime_repo_setup.bzl +++ b/python/private/hermetic_runtime_repo_setup.bzl @@ -126,6 +126,7 @@ def define_hermetic_runtime_toolchain_impl( ) cc_library( name = "python_headers", + hdrs = [":includes"], deps = [":python_headers_abi3"] + select({ "@bazel_tools//src/conditions:windows": [":interface"], "//conditions:default": [], From 7ea47060104e053ec272c11909f90e639a67e87d Mon Sep 17 00:00:00 2001 From: Laramie Leavitt Date: Fri, 21 Nov 2025 20:02:42 -0800 Subject: [PATCH 527/922] fix(local) Add api3 targets and additional defines. (#3408) Propagate defines and additional dll requirements for local python installs. In get_local_runtime_info.py: * detect abi3 vs. full abi libraries. * Ensure that returned libraries are unique. * Add additional dlls required by pythonXY.dll / pythonX.dll on windows. * Add default defines for Py_GIL_DISABLED when the local python is a freethreaded install. * Add defines (windows) for Py_NO_LINK_LIB to avoid #pragma comment(lib ...) macros In local_runtime_repo_setup.bzl * More closely match hermetic_runtime_repo_setup * Add abi3 header targets. In local_runtime_repo.bzl * rework linking to local repository directories to handl abi3 and extra dlls. * Update parameters passed into local_runtime_repo_setup.bzl Before these changes, some bazel builds using local Python fail to link properly. This happens due to a mismatch in the interpreter and the python GIL DISABLED mode, or (on Windows), where both freethreaded and non-freethreaded libraries may attempt to be linked at the same time. --------- Co-authored-by: Richard Levasseur Co-authored-by: Richard Levasseur --- CHANGELOG.md | 3 + python/private/get_local_runtime_info.py | 189 ++++++++++++-------- python/private/local_runtime_repo.bzl | 117 +++++++----- python/private/local_runtime_repo_setup.bzl | 101 +++++++---- 4 files changed, 258 insertions(+), 152 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 4f1911f99a..6709a9dff1 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -78,6 +78,9 @@ END_UNRELEASED_TEMPLATE ([#3085](https://github.com/bazel-contrib/rules_python/issues/3085)). * (toolchains) local toolchains now tell the `sys.abiflags` value of the underlying runtime. +* (toolchains) various local toolchain fixes: add abi3 header targets, + fixes to linking, Windows DLL detection, and defines for free threaded + runtimes. * (toolchains) The `python_headers` target is now compatible with layering_check. * (performance) 90% reduction in py_binary/py_test analysis phase cost. diff --git a/python/private/get_local_runtime_info.py b/python/private/get_local_runtime_info.py index b20c159cfc..a59e17a012 100644 --- a/python/private/get_local_runtime_info.py +++ b/python/private/get_local_runtime_info.py @@ -13,16 +13,27 @@ # limitations under the License. """Returns information about the local Python runtime as JSON.""" +import glob import json import os import sys import sysconfig +from typing import Any _IS_WINDOWS = sys.platform == "win32" _IS_DARWIN = sys.platform == "darwin" -def _search_directories(get_config, base_executable): +def _get_abi_flags(get_config) -> str: + """Returns the ABI flags for the Python runtime.""" + # sys.abiflags may not exist, but it still may be set in the config. + abi_flags = getattr(sys, "abiflags", None) + if abi_flags is None: + abi_flags = get_config("ABIFLAGS") or get_config("abiflags") or "" + return abi_flags + + +def _search_directories(get_config, base_executable) -> list[str]: """Returns a list of library directories to search for shared libraries.""" # There's several types of libraries with different names and a plethora # of settings, and many different config variables to check: @@ -73,23 +84,31 @@ def _search_directories(get_config, base_executable): lib_dirs.append(os.path.join(os.path.dirname(exec_dir), "lib")) # Dedup and remove empty values, keeping the order. - lib_dirs = [v for v in lib_dirs if v] - return {k: None for k in lib_dirs}.keys() + return list(dict.fromkeys(d for d in lib_dirs if d)) -def _get_shlib_suffix(get_config) -> str: - """Returns the suffix for shared libraries.""" - if _IS_DARWIN: - return ".dylib" +def _default_library_names(version, abi_flags) -> tuple[str, ...]: + """Returns a list of default library files to search for shared libraries.""" if _IS_WINDOWS: - return ".dll" - suffix = get_config("SHLIB_SUFFIX") - if not suffix: - suffix = ".so" - return suffix + return ( + f"python{version}{abi_flags}.dll", + f"python{version}.dll", + ) + elif _IS_DARWIN: + return ( + f"libpython{version}{abi_flags}.dylib", + f"libpython{version}.dylib", + ) + else: + return ( + f"libpython{version}{abi_flags}.so", + f"libpython{version}.so", + f"libpython{version}{abi_flags}.so.1.0", + f"libpython{version}.so.1.0", + ) -def _search_library_names(get_config, shlib_suffix): +def _search_library_names(get_config, version, abi_flags) -> list[str]: """Returns a list of library files to search for shared libraries.""" # Quoting configure.ac in the cpython code base: # "INSTSONAME is the name of the shared library that will be use to install @@ -112,71 +131,75 @@ def _search_library_names(get_config, shlib_suffix): ) ] - # Set the prefix and suffix to construct the library name used for linking. - # The suffix and version are set here to the default values for the OS, - # since they are used below to construct "default" library names. - if _IS_DARWIN: - prefix = "lib" - elif _IS_WINDOWS: - prefix = "" - else: - prefix = "lib" - - version = get_config("VERSION") - - # Ensure that the pythonXY.dll files are included in the search. - lib_names.append(f"{prefix}python{version}{shlib_suffix}") + # Include the default libraries for the system. + lib_names.extend(_default_library_names(version, abi_flags)) - # If there are ABIFLAGS, also add them to the python version lib search. - abiflags = get_config("ABIFLAGS") or get_config("abiflags") or "" - if abiflags: - lib_names.append(f"{prefix}python{version}{abiflags}{shlib_suffix}") + # Also include the abi3 libraries for the system. + lib_names.extend(_default_library_names(sys.version_info.major, abi_flags)) - # Add the abi-version includes to the search list. - lib_names.append(f"{prefix}python{sys.version_info.major}{shlib_suffix}") - - # Dedup and remove empty values, keeping the order. - lib_names = [v for v in lib_names if v] - return {k: None for k in lib_names}.keys() + # Uniqify, preserving order. + return list(dict.fromkeys(k for k in lib_names if k)) -def _get_python_library_info(base_executable): +def _get_python_library_info(base_executable) -> dict[str, Any]: """Returns a dictionary with the static and dynamic python libraries.""" config_vars = sysconfig.get_config_vars() # VERSION is X.Y in Linux/macOS and XY in Windows. This is used to # construct library paths such as python3.12, so ensure it exists. - if not config_vars.get("VERSION"): - if sys.platform == "win32": - config_vars["VERSION"] = ( - f"{sys.version_info.major}{sys.version_info.minor}") + version = config_vars.get("VERSION") + if not version: + if _IS_WINDOWS: + version = f"{sys.version_info.major}{sys.version_info.minor}" else: - config_vars["VERSION"] = ( - f"{sys.version_info.major}.{sys.version_info.minor}") + version = f"{sys.version_info.major}.{sys.version_info.minor}" + + defines = [] + if config_vars.get("Py_GIL_DISABLED", "0") == "1": + defines.append("Py_GIL_DISABLED") + + # Avoid automatically linking the libraries on windows via pydefine.h + # pragma comment(lib ...) + if _IS_WINDOWS: + defines.append("Py_NO_LINK_LIB") + + # sys.abiflags may not exist, but it still may be set in the config. + abi_flags = _get_abi_flags(config_vars.get) - shlib_suffix = _get_shlib_suffix(config_vars.get) search_directories = _search_directories(config_vars.get, base_executable) - search_libnames = _search_library_names(config_vars.get, shlib_suffix) + search_libnames = _search_library_names(config_vars.get, version, + abi_flags) + + # Used to test whether the library is an abi3 library or a full api library. + abi3_libraries = _default_library_names(sys.version_info.major, abi_flags) - interface_libraries = {} - dynamic_libraries = {} - static_libraries = {} + # Found libraries + static_libraries: dict[str, None] = {} + dynamic_libraries: dict[str, None] = {} + interface_libraries: dict[str, None] = {} + abi_dynamic_libraries: dict[str, None] = {} + abi_interface_libraries: dict[str, None] = {} for root_dir in search_directories: for libname in search_libnames: - # Check whether the library exists. composed_path = os.path.join(root_dir, libname) + is_abi3_file = os.path.basename(composed_path) in abi3_libraries + + # Check whether the library exists and add it to the appropriate list. if os.path.exists(composed_path) or os.path.isdir(composed_path): - if libname.endswith(".a"): + if is_abi3_file: + if not libname.endswith(".a"): + abi_dynamic_libraries[composed_path] = None + elif libname.endswith(".a"): static_libraries[composed_path] = None else: dynamic_libraries[composed_path] = None interface_path = None if libname.endswith(".dll"): - # On windows a .lib file may be an "import library" or a static library. - # The file could be inspected to determine which it is; typically python - # is used as a shared library. + # On windows a .lib file may be an "import library" or a static + # library. The file could be inspected to determine which it is; + # typically python is used as a shared library. # # On Windows, extensions should link with the pythonXY.lib interface # libraries. @@ -190,39 +213,51 @@ def _get_python_library_info(base_executable): # Check whether an interface library exists. if interface_path and os.path.exists(interface_path): - interface_libraries[interface_path] = None + if is_abi3_file: + abi_interface_libraries[interface_path] = None + else: + interface_libraries[interface_path] = None - # Non-windows typically has abiflags. - if hasattr(sys, "abiflags"): - abiflags = sys.abiflags - else: - abiflags = "" + # Additional DLLs are needed on Windows to link properly. + dlls = [] + if _IS_WINDOWS: + dlls.extend( + glob.glob(os.path.join(os.path.dirname(base_executable), "*.dll"))) + dlls = [ + x for x in dlls + if x not in dynamic_libraries and x not in abi_dynamic_libraries + ] + + def _unique_basenames(inputs: dict[str, None]) -> list[str]: + """Returns a list of paths, keeping only the first path for each basename.""" + result = [] + seen = set() + for k in inputs: + b = os.path.basename(k) + if b not in seen: + seen.add(b) + result.append(k) + return result # When no libraries are found it's likely that the python interpreter is not # configured to use shared or static libraries (minilinux). If this seems # suspicious try running `uv tool run find_libpython --list-all -v` return { - "dynamic_libraries": list(dynamic_libraries.keys()), - "static_libraries": list(static_libraries.keys()), - "interface_libraries": list(interface_libraries.keys()), - "shlib_suffix": "" if _IS_WINDOWS else shlib_suffix, - "abi_flags": abiflags, + "dynamic_libraries": _unique_basenames(dynamic_libraries), + "static_libraries": _unique_basenames(static_libraries), + "interface_libraries": _unique_basenames(interface_libraries), + "abi_dynamic_libraries": _unique_basenames(abi_dynamic_libraries), + "abi_interface_libraries": _unique_basenames(abi_interface_libraries), + "abi_flags": abi_flags, + "shlib_suffix": ".dylib" if _IS_DARWIN else "", + "additional_dlls": dlls, + "defines": defines, } -def _get_base_executable(): +def _get_base_executable() -> str: """Returns the base executable path.""" - try: - if sys._base_executable: # pylint: disable=protected-access - return sys._base_executable # pylint: disable=protected-access - except AttributeError: - # Bug reports indicate sys._base_executable doesn't exist in some cases, - # but it's not clear why. - # See https://github.com/bazel-contrib/rules_python/issues/3172 - pass - # The normal sys.executable is the next-best guess if sys._base_executable - # is missing. - return sys.executable + return getattr(sys, "_base_executable", None) or sys.executable data = { diff --git a/python/private/local_runtime_repo.bzl b/python/private/local_runtime_repo.bzl index 024f7c5e8a..df27c74950 100644 --- a/python/private/local_runtime_repo.bzl +++ b/python/private/local_runtime_repo.bzl @@ -34,15 +34,36 @@ define_local_runtime_toolchain_impl( major = "{major}", minor = "{minor}", micro = "{micro}", + abi_flags = "{abi_flags}", + os = "{os}", + implementation_name = "{implementation_name}", interpreter_path = "{interpreter_path}", interface_library = {interface_library}, libraries = {libraries}, - implementation_name = "{implementation_name}", - os = "{os}", - abi_flags = "{abi_flags}", + defines = {defines}, + abi3_interface_library = {abi3_interface_library}, + abi3_libraries = {abi3_libraries}, + additional_dlls = {additional_dlls}, ) """ +def _expand_incompatible_template(): + return _TOOLCHAIN_IMPL_TEMPLATE.format( + major = "0", + minor = "0", + micro = "0", + abi_flags = "", + os = "@platforms//:incompatible", + implementation_name = "incompatible", + interpreter_path = "/incompatible", + interface_library = "None", + libraries = "[]", + defines = "[]", + abi3_interface_library = "None", + abi3_libraries = "[]", + additional_dlls = "[]", + ) + def _norm_path(path): """Returns a path using '/' separators and no trailing slash.""" path = path.replace("\\", "/") @@ -50,33 +71,39 @@ def _norm_path(path): path = path[:-1] return path -def _symlink_first_library(rctx, logger, libraries, shlib_suffix): +def _symlink_libraries(rctx, logger, libraries, shlib_suffix): """Symlinks the shared libraries into the lib/ directory. Args: rctx: A repository_ctx object logger: A repo_utils.logger object - libraries: A list of static library paths to potentially symlink. - shlib_suffix: A suffix only provided for shared libraries to ensure - that the srcs restriction of cc_library targets are met. + libraries: paths to libraries to attempt to symlink. + shlib_suffix: Optional. Ensure that the generated symlinks end with this suffix. Returns: - A single library path linked by the action. + A list of library paths (under lib/) linked by the action. + + Individual files are symlinked instead of the whole directory because + shared_lib_dirs contains multiple search paths for the shared libraries, + and the python files may be missing from any of those directories, and + any of those directories may include non-python runtime libraries, + as would be the case if LIBDIR were, for example, /usr/lib. """ - for target in libraries: - origin = rctx.path(target) + result = [] + for source in libraries: + origin = rctx.path(source) if not origin.exists: # The reported names don't always exist; it depends on the particulars # of the runtime installation. continue - if shlib_suffix and not target.endswith(shlib_suffix): - linked = "lib/{}{}".format(origin.basename, shlib_suffix) + if shlib_suffix and not origin.basename.endswith(shlib_suffix): + target = "lib/{}{}".format(origin.basename, shlib_suffix) else: - linked = "lib/{}".format(origin.basename) - logger.debug("Symlinking {} to {}".format(origin, linked)) + target = "lib/{}".format(origin.basename) + logger.debug(lambda: "Symlinking {} to {}".format(origin, target)) rctx.watch(origin) - rctx.symlink(origin, linked) - return linked - return None + rctx.symlink(origin, target) + result.append(target) + return result def _local_runtime_repo_impl(rctx): logger = repo_utils.logger(rctx) @@ -150,31 +177,48 @@ def _local_runtime_repo_impl(rctx): # The cc_library.includes values have to be non-absolute paths, otherwise # the toolchain will give an error. Work around this error by making them # appear as part of this repo. + logger.debug(lambda: "Symlinking {} to include".format(include_path)) rctx.symlink(include_path, "include") rctx.report_progress("Symlinking external Python shared libraries") - interface_library = _symlink_first_library(rctx, logger, info["interface_libraries"], None) - shared_library = _symlink_first_library(rctx, logger, info["dynamic_libraries"], info["shlib_suffix"]) - static_library = _symlink_first_library(rctx, logger, info["static_libraries"], None) - - libraries = [] - if shared_library: - libraries.append(shared_library) - elif static_library: - libraries.append(static_library) + + interface_library = None + if info["dynamic_libraries"]: + libraries = _symlink_libraries(rctx, logger, info["dynamic_libraries"][:1], info["shlib_suffix"]) + symlinked = _symlink_libraries(rctx, logger, info["interface_libraries"][:1], None) + if symlinked: + interface_library = symlinked[0] else: - logger.warn("No external python libraries found.") + libraries = _symlink_libraries(rctx, logger, info["static_libraries"], None) + if not libraries: + logger.info("No python libraries found.") + + abi3_interface_library = None + if info["abi_dynamic_libraries"]: + abi3_libraries = _symlink_libraries(rctx, logger, info["abi_dynamic_libraries"][:1], info["shlib_suffix"]) + symlinked = _symlink_libraries(rctx, logger, info["abi_interface_libraries"][:1], None) + if symlinked: + abi3_interface_library = symlinked[0] + else: + abi3_libraries = [] + logger.info("No abi3 python libraries found.") + + additional_dlls = _symlink_libraries(rctx, logger, info["additional_dlls"], None) build_bazel = _TOOLCHAIN_IMPL_TEMPLATE.format( major = info["major"], minor = info["minor"], micro = info["micro"], + abi_flags = info["abi_flags"], + os = "@platforms//os:{}".format(repo_utils.get_platforms_os_name(rctx)), + implementation_name = info["implementation_name"], interpreter_path = _norm_path(interpreter_path), interface_library = repr(interface_library), libraries = repr(libraries), - implementation_name = info["implementation_name"], - os = "@platforms//os:{}".format(repo_utils.get_platforms_os_name(rctx)), - abi_flags = info["abi_flags"], + defines = repr(info["defines"]), + abi3_interface_library = repr(abi3_interface_library), + abi3_libraries = repr(abi3_libraries), + additional_dlls = repr(additional_dlls), ) logger.debug(lambda: "BUILD.bazel\n{}".format(build_bazel)) @@ -261,19 +305,6 @@ How to handle errors when trying to automatically determine settings. environ = ["PATH", REPO_DEBUG_ENV_VAR, "DEVELOPER_DIR", "XCODE_VERSION"], ) -def _expand_incompatible_template(): - return _TOOLCHAIN_IMPL_TEMPLATE.format( - interpreter_path = "/incompatible", - implementation_name = "incompatible", - interface_library = "None", - libraries = "[]", - major = "0", - minor = "0", - micro = "0", - os = "@platforms//:incompatible", - abi_flags = "", - ) - def _find_python_exe_from_target(rctx): base_path = rctx.path(rctx.attr.interpreter_target) if base_path.exists: diff --git a/python/private/local_runtime_repo_setup.bzl b/python/private/local_runtime_repo_setup.bzl index 0ce1d4d764..0922181ffe 100644 --- a/python/private/local_runtime_repo_setup.bzl +++ b/python/private/local_runtime_repo_setup.bzl @@ -11,7 +11,6 @@ # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # See the License for the specific language governing permissions and # limitations under the License. - """Setup code called by the code generated by `local_runtime_repo`.""" load("@bazel_skylib//lib:selects.bzl", "selects") @@ -29,12 +28,16 @@ def define_local_runtime_toolchain_impl( major, minor, micro, + abi_flags, + os, + implementation_name, interpreter_path, interface_library, libraries, - implementation_name, - os, - abi_flags): + defines, + abi3_interface_library, + abi3_libraries, + additional_dlls): """Defines a toolchain implementation for a local Python runtime. Generates public targets: @@ -51,16 +54,23 @@ def define_local_runtime_toolchain_impl( major: `str` The major Python version, e.g. `3` of `3.9.1`. minor: `str` The minor Python version, e.g. `9` of `3.9.1`. micro: `str` The micro Python version, e.g. "1" of `3.9.1`. + abi_flags: `str` The abi flags, as returned by `sys.abiflags`. + os: `str` A label to the OS constraint (e.g. `@platforms//os:linux`) for + this runtime. + implementation_name: `str` The implementation name, as returned by + `sys.implementation.name`. interpreter_path: `str` Absolute path to the interpreter. interface_library: `str` Path to the interface library. e.g. "lib/python312.lib" libraries: `list[str]` Path[s] to the python libraries. e.g. ["lib/python312.dll"] or ["lib/python312.so"] - implementation_name: `str` The implementation name, as returned by - `sys.implementation.name`. - os: `str` A label to the OS constraint (e.g. `@platforms//os:linux`) for - this runtime. - abi_flags: `str` Str. Flags provided by sys.abiflags for the runtime. + defines: `list[str]` List of additional defines. + abi3_interface_library: `str` Path to the interface library. + e.g. "lib/python3.lib" + abi3_libraries: `list[str]` Path[s] to the python libraries. + e.g. ["lib/python3.dll"] or ["lib/python3.so"] + additional_dlls: `list[str]` Path[s] to additional DLLs. + e.g. ["lib/msvcrt123.dll"] """ major_minor = "{}.{}".format(major, minor) major_minor_micro = "{}.{}".format(major_minor, micro) @@ -69,44 +79,70 @@ def define_local_runtime_toolchain_impl( # See https://docs.python.org/3/extending/windows.html # However not all python installations (such as manylinux) include shared or static libraries, # so only create the import library when interface_library is set. - full_abi_deps = [] - abi3_deps = [] if interface_library: cc_import( - name = "_python_interface_library", + name = "interface", interface_library = interface_library, - system_provided = 1, + system_provided = True, ) - if interface_library.endswith("{}.lib".format(major)): - abi3_deps = [":_python_interface_library"] - else: - full_abi_deps = [":_python_interface_library"] - cc_library( - name = "_python_headers_abi3", - # NOTE: Keep in sync with watch_tree() called in local_runtime_repo + if abi3_interface_library: + cc_import( + name = "abi3_interface", + interface_library = abi3_interface_library, + system_provided = True, + ) + + native.filegroup( + name = "includes", srcs = native.glob( include = ["include/**/*.h"], exclude = ["include/numpy/**"], # numpy headers are handled separately allow_empty = True, # A Python install may not have C headers ), - deps = abi3_deps, + ) + + # header libraries. + cc_library( + name = "python_headers_abi3", + hdrs = [":includes"], + includes = ["include"], + defines = defines, # NOTE: Users should define Py_LIMITED_API=3 + deps = select({ + "@bazel_tools//src/conditions:windows": [":abi3_interface"], + "//conditions:default": [], + }), + ) + + cc_library( + name = "python_headers", + hdrs = [":includes"], includes = ["include"], + defines = defines, + deps = select({ + "@bazel_tools//src/conditions:windows": [":interface"], + "//conditions:default": [], + }), ) + + # python libraries cc_library( - name = "_python_headers", - deps = [":_python_headers_abi3"] + full_abi_deps, + name = "libpython_abi3", + hdrs = [":includes"], + defines = defines, # NOTE: Users should define Py_LIMITED_API=3 + srcs = abi3_libraries + additional_dlls, ) cc_library( - name = "_libpython", - hdrs = [":_python_headers"], - srcs = libraries, - deps = [], + name = "libpython", + hdrs = [":includes"], + defines = defines, + srcs = libraries + additional_dlls, ) + # runtime configuration py_runtime( - name = "_py3_runtime", + name = "py3_runtime", interpreter_path = interpreter_path, python_version = "PY3", interpreter_version_info = { @@ -116,12 +152,13 @@ def define_local_runtime_toolchain_impl( }, implementation_name = implementation_name, abi_flags = abi_flags, + pyc_tag = "{}-{}{}{}".format(implementation_name, major, minor, abi_flags), ) py_runtime_pair( name = "python_runtimes", py2_runtime = None, - py3_runtime = ":_py3_runtime", + py3_runtime = ":py3_runtime", visibility = ["//visibility:public"], ) @@ -133,9 +170,9 @@ def define_local_runtime_toolchain_impl( py_cc_toolchain( name = "py_cc_toolchain", - headers = ":_python_headers", - headers_abi3 = ":_python_headers_abi3", - libs = ":_libpython", + headers = ":python_headers", + headers_abi3 = ":python_headers_abi3", + libs = ":libpython", python_version = major_minor_micro, visibility = ["//visibility:public"], ) From 728cce72fc55e1f1a65ab5bb24aefe5f9fe8a0bb Mon Sep 17 00:00:00 2001 From: Ignas Anikevicius <240938+aignas@users.noreply.github.com> Date: Sat, 22 Nov 2025 15:18:59 +0900 Subject: [PATCH 528/922] chore: start cleaning up 3.9 usage from examples (#3419) Summary: - **example: stop using 3.9 in multi-python-versions** - **chore: cleanup unused code** There is more to go, but this is a good start remaining of #2704 --------- Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> --- .bazelci/presubmit.yml | 8 +- examples/bzlmod/other_module/MODULE.bazel | 14 +- examples/multi_python_versions/MODULE.bazel | 36 +++-- examples/multi_python_versions/WORKSPACE | 14 +- .../requirements/BUILD.bazel | 28 +++- ...ock_3_9.txt => requirements_lock_3_12.txt} | 4 +- .../requirements/requirements_lock_3_13.txt | 78 +++++++++++ .../requirements/requirements_lock_3_14.txt | 78 +++++++++++ .../multi_python_versions/tests/BUILD.bazel | 128 ++++++++++++------ 9 files changed, 310 insertions(+), 78 deletions(-) rename examples/multi_python_versions/requirements/{requirements_lock_3_9.txt => requirements_lock_3_12.txt} (98%) create mode 100644 examples/multi_python_versions/requirements/requirements_lock_3_13.txt create mode 100644 examples/multi_python_versions/requirements/requirements_lock_3_14.txt diff --git a/.bazelci/presubmit.yml b/.bazelci/presubmit.yml index 1cb3a01365..256961bf36 100644 --- a/.bazelci/presubmit.yml +++ b/.bazelci/presubmit.yml @@ -81,11 +81,15 @@ buildifier: coverage_targets: - //tests:my_lib_3_10_test - //tests:my_lib_3_11_test - - //tests:my_lib_3_9_test + - //tests:my_lib_3_12_test + - //tests:my_lib_3_13_test + - //tests:my_lib_3_14_test - //tests:my_lib_default_test - //tests:version_3_10_test - //tests:version_3_11_test - - //tests:version_3_9_test + - //tests:version_3_12_test + - //tests:version_3_13_test + - //tests:version_3_14_test - //tests:version_default_test tasks: gazelle_extension_min: diff --git a/examples/bzlmod/other_module/MODULE.bazel b/examples/bzlmod/other_module/MODULE.bazel index 7b88bd73ff..a128c39ca0 100644 --- a/examples/bzlmod/other_module/MODULE.bazel +++ b/examples/bzlmod/other_module/MODULE.bazel @@ -10,10 +10,6 @@ local_path_override( path = "../../..", ) -PYTHON_NAME_39 = "python_3_9" - -PYTHON_NAME_311 = "python_3_11" - python = use_extension("@rules_python//python/extensions:python.bzl", "python") python.defaults( # In a submodule this is ignored @@ -21,21 +17,13 @@ python.defaults( ) python.toolchain( configure_coverage_tool = True, - python_version = "3.9", + python_version = "3.12", ) python.toolchain( configure_coverage_tool = True, python_version = "3.11", ) -# created by the above python.toolchain calls. -use_repo( - python, - "python_versions", - PYTHON_NAME_39, - PYTHON_NAME_311, -) - pip = use_extension("@rules_python//python/extensions:pip.bzl", "pip") pip.parse( hub_name = "other_module_pip", diff --git a/examples/multi_python_versions/MODULE.bazel b/examples/multi_python_versions/MODULE.bazel index eeb1dfc83e..2ef09ade3e 100644 --- a/examples/multi_python_versions/MODULE.bazel +++ b/examples/multi_python_versions/MODULE.bazel @@ -12,20 +12,28 @@ local_path_override( python = use_extension("@rules_python//python/extensions:python.bzl", "python") python.defaults( # The environment variable takes precedence if set. - python_version = "3.9", + python_version = "3.10", python_version_env = "BAZEL_PYTHON_VERSION", ) python.toolchain( configure_coverage_tool = True, - python_version = "3.9", + python_version = "3.10", ) python.toolchain( configure_coverage_tool = True, - python_version = "3.10", + python_version = "3.11", ) python.toolchain( configure_coverage_tool = True, - python_version = "3.11", + python_version = "3.12", +) +python.toolchain( + configure_coverage_tool = True, + python_version = "3.13", +) +python.toolchain( + configure_coverage_tool = True, + python_version = "3.14", ) use_repo( python, @@ -36,11 +44,6 @@ use_repo( pip = use_extension("@rules_python//python/extensions:pip.bzl", "pip") use_repo(pip, "pypi") -pip.parse( - hub_name = "pypi", - python_version = "3.9", - requirements_lock = "//requirements:requirements_lock_3_9.txt", -) pip.parse( hub_name = "pypi", python_version = "3.10", @@ -51,6 +54,21 @@ pip.parse( python_version = "3.11", requirements_lock = "//requirements:requirements_lock_3_11.txt", ) +pip.parse( + hub_name = "pypi", + python_version = "3.12", + requirements_lock = "//requirements:requirements_lock_3_12.txt", +) +pip.parse( + hub_name = "pypi", + python_version = "3.13", + requirements_lock = "//requirements:requirements_lock_3_13.txt", +) +pip.parse( + hub_name = "pypi", + python_version = "3.14", + requirements_lock = "//requirements:requirements_lock_3_14.txt", +) # example test dependencies bazel_dep(name = "rules_shell", version = "0.2.0", dev_dependency = True) diff --git a/examples/multi_python_versions/WORKSPACE b/examples/multi_python_versions/WORKSPACE index 6b69e0a891..0b6b8a0cbf 100644 --- a/examples/multi_python_versions/WORKSPACE +++ b/examples/multi_python_versions/WORKSPACE @@ -9,15 +9,17 @@ load("@rules_python//python:repositories.bzl", "py_repositories", "python_regist py_repositories() -default_python_version = "3.9" +default_python_version = "3.10" python_register_multi_toolchains( name = "python", default_version = default_python_version, python_versions = [ - "3.9", "3.10", "3.11", + "3.12", + "3.13", + "3.14", ], register_coverage_tool = True, ) @@ -30,12 +32,16 @@ multi_pip_parse( python_interpreter_target = { "3.10": "@python_3_10_host//:python", "3.11": "@python_3_11_host//:python", - "3.9": "@python_3_9_host//:python", + "3.12": "@python_3_12_host//:python", + "3.13": "@python_3_13_host//:python", + "3.14": "@python_3_14_host//:python", }, requirements_lock = { "3.10": "//requirements:requirements_lock_3_10.txt", "3.11": "//requirements:requirements_lock_3_11.txt", - "3.9": "//requirements:requirements_lock_3_9.txt", + "3.12": "//requirements:requirements_lock_3_12.txt", + "3.13": "//requirements:requirements_lock_3_13.txt", + "3.14": "//requirements:requirements_lock_3_14.txt", }, ) diff --git a/examples/multi_python_versions/requirements/BUILD.bazel b/examples/multi_python_versions/requirements/BUILD.bazel index 516a378df8..ee8ff029f8 100644 --- a/examples/multi_python_versions/requirements/BUILD.bazel +++ b/examples/multi_python_versions/requirements/BUILD.bazel @@ -1,12 +1,5 @@ load("@rules_python//python:pip.bzl", "compile_pip_requirements") -compile_pip_requirements( - name = "requirements_3_9", - src = "requirements.in", - python_version = "3.9", - requirements_txt = "requirements_lock_3_9.txt", -) - compile_pip_requirements( name = "requirements_3_10", src = "requirements.in", @@ -20,3 +13,24 @@ compile_pip_requirements( python_version = "3.11", requirements_txt = "requirements_lock_3_11.txt", ) + +compile_pip_requirements( + name = "requirements_3_12", + src = "requirements.in", + python_version = "3.12", + requirements_txt = "requirements_lock_3_12.txt", +) + +compile_pip_requirements( + name = "requirements_3_13", + src = "requirements.in", + python_version = "3.13", + requirements_txt = "requirements_lock_3_13.txt", +) + +compile_pip_requirements( + name = "requirements_3_14", + src = "requirements.in", + python_version = "3.14", + requirements_txt = "requirements_lock_3_14.txt", +) diff --git a/examples/multi_python_versions/requirements/requirements_lock_3_9.txt b/examples/multi_python_versions/requirements/requirements_lock_3_12.txt similarity index 98% rename from examples/multi_python_versions/requirements/requirements_lock_3_9.txt rename to examples/multi_python_versions/requirements/requirements_lock_3_12.txt index 3c696a865e..818b20e14c 100644 --- a/examples/multi_python_versions/requirements/requirements_lock_3_9.txt +++ b/examples/multi_python_versions/requirements/requirements_lock_3_12.txt @@ -1,8 +1,8 @@ # -# This file is autogenerated by pip-compile with Python 3.9 +# This file is autogenerated by pip-compile with Python 3.12 # by the following command: # -# bazel run //requirements:requirements_3_9.update +# bazel run //requirements:requirements_3_12.update # websockets==11.0.3 ; python_full_version > "3.9.1" \ --hash=sha256:01f5567d9cf6f502d655151645d4e8b72b453413d3819d2b6f1185abc23e82dd \ diff --git a/examples/multi_python_versions/requirements/requirements_lock_3_13.txt b/examples/multi_python_versions/requirements/requirements_lock_3_13.txt new file mode 100644 index 0000000000..8dc44b8a07 --- /dev/null +++ b/examples/multi_python_versions/requirements/requirements_lock_3_13.txt @@ -0,0 +1,78 @@ +# +# This file is autogenerated by pip-compile with Python 3.13 +# by the following command: +# +# bazel run //requirements:requirements_3_13.update +# +websockets==11.0.3 ; python_full_version > "3.9.1" \ + --hash=sha256:01f5567d9cf6f502d655151645d4e8b72b453413d3819d2b6f1185abc23e82dd \ + --hash=sha256:03aae4edc0b1c68498f41a6772d80ac7c1e33c06c6ffa2ac1c27a07653e79d6f \ + --hash=sha256:0ac56b661e60edd453585f4bd68eb6a29ae25b5184fd5ba51e97652580458998 \ + --hash=sha256:0ee68fe502f9031f19d495dae2c268830df2760c0524cbac5d759921ba8c8e82 \ + --hash=sha256:1553cb82942b2a74dd9b15a018dce645d4e68674de2ca31ff13ebc2d9f283788 \ + --hash=sha256:1a073fc9ab1c8aff37c99f11f1641e16da517770e31a37265d2755282a5d28aa \ + --hash=sha256:1d2256283fa4b7f4c7d7d3e84dc2ece74d341bce57d5b9bf385df109c2a1a82f \ + --hash=sha256:1d5023a4b6a5b183dc838808087033ec5df77580485fc533e7dab2567851b0a4 \ + --hash=sha256:1fdf26fa8a6a592f8f9235285b8affa72748dc12e964a5518c6c5e8f916716f7 \ + --hash=sha256:2529338a6ff0eb0b50c7be33dc3d0e456381157a31eefc561771ee431134a97f \ + --hash=sha256:279e5de4671e79a9ac877427f4ac4ce93751b8823f276b681d04b2156713b9dd \ + --hash=sha256:2d903ad4419f5b472de90cd2d40384573b25da71e33519a67797de17ef849b69 \ + --hash=sha256:332d126167ddddec94597c2365537baf9ff62dfcc9db4266f263d455f2f031cb \ + --hash=sha256:34fd59a4ac42dff6d4681d8843217137f6bc85ed29722f2f7222bd619d15e95b \ + --hash=sha256:3580dd9c1ad0701169e4d6fc41e878ffe05e6bdcaf3c412f9d559389d0c9e016 \ + --hash=sha256:3ccc8a0c387629aec40f2fc9fdcb4b9d5431954f934da3eaf16cdc94f67dbfac \ + --hash=sha256:41f696ba95cd92dc047e46b41b26dd24518384749ed0d99bea0a941ca87404c4 \ + --hash=sha256:42cc5452a54a8e46a032521d7365da775823e21bfba2895fb7b77633cce031bb \ + --hash=sha256:4841ed00f1026dfbced6fca7d963c4e7043aa832648671b5138008dc5a8f6d99 \ + --hash=sha256:4b253869ea05a5a073ebfdcb5cb3b0266a57c3764cf6fe114e4cd90f4bfa5f5e \ + --hash=sha256:54c6e5b3d3a8936a4ab6870d46bdd6ec500ad62bde9e44462c32d18f1e9a8e54 \ + --hash=sha256:619d9f06372b3a42bc29d0cd0354c9bb9fb39c2cbc1a9c5025b4538738dbffaf \ + --hash=sha256:6505c1b31274723ccaf5f515c1824a4ad2f0d191cec942666b3d0f3aa4cb4007 \ + --hash=sha256:660e2d9068d2bedc0912af508f30bbeb505bbbf9774d98def45f68278cea20d3 \ + --hash=sha256:6681ba9e7f8f3b19440921e99efbb40fc89f26cd71bf539e45d8c8a25c976dc6 \ + --hash=sha256:68b977f21ce443d6d378dbd5ca38621755f2063d6fdb3335bda981d552cfff86 \ + --hash=sha256:69269f3a0b472e91125b503d3c0b3566bda26da0a3261c49f0027eb6075086d1 \ + --hash=sha256:6f1a3f10f836fab6ca6efa97bb952300b20ae56b409414ca85bff2ad241d2a61 \ + --hash=sha256:7622a89d696fc87af8e8d280d9b421db5133ef5b29d3f7a1ce9f1a7bf7fcfa11 \ + --hash=sha256:777354ee16f02f643a4c7f2b3eff8027a33c9861edc691a2003531f5da4f6bc8 \ + --hash=sha256:84d27a4832cc1a0ee07cdcf2b0629a8a72db73f4cf6de6f0904f6661227f256f \ + --hash=sha256:8531fdcad636d82c517b26a448dcfe62f720e1922b33c81ce695d0edb91eb931 \ + --hash=sha256:86d2a77fd490ae3ff6fae1c6ceaecad063d3cc2320b44377efdde79880e11526 \ + --hash=sha256:88fc51d9a26b10fc331be344f1781224a375b78488fc343620184e95a4b27016 \ + --hash=sha256:8a34e13a62a59c871064dfd8ffb150867e54291e46d4a7cf11d02c94a5275bae \ + --hash=sha256:8c82f11964f010053e13daafdc7154ce7385ecc538989a354ccc7067fd7028fd \ + --hash=sha256:92b2065d642bf8c0a82d59e59053dd2fdde64d4ed44efe4870fa816c1232647b \ + --hash=sha256:97b52894d948d2f6ea480171a27122d77af14ced35f62e5c892ca2fae9344311 \ + --hash=sha256:9d9acd80072abcc98bd2c86c3c9cd4ac2347b5a5a0cae7ed5c0ee5675f86d9af \ + --hash=sha256:9f59a3c656fef341a99e3d63189852be7084c0e54b75734cde571182c087b152 \ + --hash=sha256:aa5003845cdd21ac0dc6c9bf661c5beddd01116f6eb9eb3c8e272353d45b3288 \ + --hash=sha256:b16fff62b45eccb9c7abb18e60e7e446998093cdcb50fed33134b9b6878836de \ + --hash=sha256:b30c6590146e53149f04e85a6e4fcae068df4289e31e4aee1fdf56a0dead8f97 \ + --hash=sha256:b58cbf0697721120866820b89f93659abc31c1e876bf20d0b3d03cef14faf84d \ + --hash=sha256:b67c6f5e5a401fc56394f191f00f9b3811fe843ee93f4a70df3c389d1adf857d \ + --hash=sha256:bceab846bac555aff6427d060f2fcfff71042dba6f5fca7dc4f75cac815e57ca \ + --hash=sha256:bee9fcb41db2a23bed96c6b6ead6489702c12334ea20a297aa095ce6d31370d0 \ + --hash=sha256:c114e8da9b475739dde229fd3bc6b05a6537a88a578358bc8eb29b4030fac9c9 \ + --hash=sha256:c1f0524f203e3bd35149f12157438f406eff2e4fb30f71221c8a5eceb3617b6b \ + --hash=sha256:c792ea4eabc0159535608fc5658a74d1a81020eb35195dd63214dcf07556f67e \ + --hash=sha256:c7f3cb904cce8e1be667c7e6fef4516b98d1a6a0635a58a57528d577ac18a128 \ + --hash=sha256:d67ac60a307f760c6e65dad586f556dde58e683fab03323221a4e530ead6f74d \ + --hash=sha256:dcacf2c7a6c3a84e720d1bb2b543c675bf6c40e460300b628bab1b1efc7c034c \ + --hash=sha256:de36fe9c02995c7e6ae6efe2e205816f5f00c22fd1fbf343d4d18c3d5ceac2f5 \ + --hash=sha256:def07915168ac8f7853812cc593c71185a16216e9e4fa886358a17ed0fd9fcf6 \ + --hash=sha256:df41b9bc27c2c25b486bae7cf42fccdc52ff181c8c387bfd026624a491c2671b \ + --hash=sha256:e052b8467dd07d4943936009f46ae5ce7b908ddcac3fda581656b1b19c083d9b \ + --hash=sha256:e063b1865974611313a3849d43f2c3f5368093691349cf3c7c8f8f75ad7cb280 \ + --hash=sha256:e1459677e5d12be8bbc7584c35b992eea142911a6236a3278b9b5ce3326f282c \ + --hash=sha256:e1a99a7a71631f0efe727c10edfba09ea6bee4166a6f9c19aafb6c0b5917d09c \ + --hash=sha256:e590228200fcfc7e9109509e4d9125eace2042fd52b595dd22bbc34bb282307f \ + --hash=sha256:e6316827e3e79b7b8e7d8e3b08f4e331af91a48e794d5d8b099928b6f0b85f20 \ + --hash=sha256:e7837cb169eca3b3ae94cc5787c4fed99eef74c0ab9506756eea335e0d6f3ed8 \ + --hash=sha256:e848f46a58b9fcf3d06061d17be388caf70ea5b8cc3466251963c8345e13f7eb \ + --hash=sha256:ed058398f55163a79bb9f06a90ef9ccc063b204bb346c4de78efc5d15abfe602 \ + --hash=sha256:f2e58f2c36cc52d41f2659e4c0cbf7353e28c8c9e63e30d8c6d3494dc9fdedcf \ + --hash=sha256:f467ba0050b7de85016b43f5a22b46383ef004c4f672148a8abf32bc999a87f0 \ + --hash=sha256:f61bdb1df43dc9c131791fbc2355535f9024b9a04398d3bd0684fc16ab07df74 \ + --hash=sha256:fb06eea71a00a7af0ae6aefbb932fb8a7df3cb390cc217d51a9ad7343de1b8d0 \ + --hash=sha256:ffd7dcaf744f25f82190856bc26ed81721508fc5cbf2a330751e135ff1283564 + # via -r requirements/requirements.in diff --git a/examples/multi_python_versions/requirements/requirements_lock_3_14.txt b/examples/multi_python_versions/requirements/requirements_lock_3_14.txt new file mode 100644 index 0000000000..f0aaaa90af --- /dev/null +++ b/examples/multi_python_versions/requirements/requirements_lock_3_14.txt @@ -0,0 +1,78 @@ +# +# This file is autogenerated by pip-compile with Python 3.14 +# by the following command: +# +# bazel run //requirements:requirements_3_14.update +# +websockets==11.0.3 ; python_full_version > "3.9.1" \ + --hash=sha256:01f5567d9cf6f502d655151645d4e8b72b453413d3819d2b6f1185abc23e82dd \ + --hash=sha256:03aae4edc0b1c68498f41a6772d80ac7c1e33c06c6ffa2ac1c27a07653e79d6f \ + --hash=sha256:0ac56b661e60edd453585f4bd68eb6a29ae25b5184fd5ba51e97652580458998 \ + --hash=sha256:0ee68fe502f9031f19d495dae2c268830df2760c0524cbac5d759921ba8c8e82 \ + --hash=sha256:1553cb82942b2a74dd9b15a018dce645d4e68674de2ca31ff13ebc2d9f283788 \ + --hash=sha256:1a073fc9ab1c8aff37c99f11f1641e16da517770e31a37265d2755282a5d28aa \ + --hash=sha256:1d2256283fa4b7f4c7d7d3e84dc2ece74d341bce57d5b9bf385df109c2a1a82f \ + --hash=sha256:1d5023a4b6a5b183dc838808087033ec5df77580485fc533e7dab2567851b0a4 \ + --hash=sha256:1fdf26fa8a6a592f8f9235285b8affa72748dc12e964a5518c6c5e8f916716f7 \ + --hash=sha256:2529338a6ff0eb0b50c7be33dc3d0e456381157a31eefc561771ee431134a97f \ + --hash=sha256:279e5de4671e79a9ac877427f4ac4ce93751b8823f276b681d04b2156713b9dd \ + --hash=sha256:2d903ad4419f5b472de90cd2d40384573b25da71e33519a67797de17ef849b69 \ + --hash=sha256:332d126167ddddec94597c2365537baf9ff62dfcc9db4266f263d455f2f031cb \ + --hash=sha256:34fd59a4ac42dff6d4681d8843217137f6bc85ed29722f2f7222bd619d15e95b \ + --hash=sha256:3580dd9c1ad0701169e4d6fc41e878ffe05e6bdcaf3c412f9d559389d0c9e016 \ + --hash=sha256:3ccc8a0c387629aec40f2fc9fdcb4b9d5431954f934da3eaf16cdc94f67dbfac \ + --hash=sha256:41f696ba95cd92dc047e46b41b26dd24518384749ed0d99bea0a941ca87404c4 \ + --hash=sha256:42cc5452a54a8e46a032521d7365da775823e21bfba2895fb7b77633cce031bb \ + --hash=sha256:4841ed00f1026dfbced6fca7d963c4e7043aa832648671b5138008dc5a8f6d99 \ + --hash=sha256:4b253869ea05a5a073ebfdcb5cb3b0266a57c3764cf6fe114e4cd90f4bfa5f5e \ + --hash=sha256:54c6e5b3d3a8936a4ab6870d46bdd6ec500ad62bde9e44462c32d18f1e9a8e54 \ + --hash=sha256:619d9f06372b3a42bc29d0cd0354c9bb9fb39c2cbc1a9c5025b4538738dbffaf \ + --hash=sha256:6505c1b31274723ccaf5f515c1824a4ad2f0d191cec942666b3d0f3aa4cb4007 \ + --hash=sha256:660e2d9068d2bedc0912af508f30bbeb505bbbf9774d98def45f68278cea20d3 \ + --hash=sha256:6681ba9e7f8f3b19440921e99efbb40fc89f26cd71bf539e45d8c8a25c976dc6 \ + --hash=sha256:68b977f21ce443d6d378dbd5ca38621755f2063d6fdb3335bda981d552cfff86 \ + --hash=sha256:69269f3a0b472e91125b503d3c0b3566bda26da0a3261c49f0027eb6075086d1 \ + --hash=sha256:6f1a3f10f836fab6ca6efa97bb952300b20ae56b409414ca85bff2ad241d2a61 \ + --hash=sha256:7622a89d696fc87af8e8d280d9b421db5133ef5b29d3f7a1ce9f1a7bf7fcfa11 \ + --hash=sha256:777354ee16f02f643a4c7f2b3eff8027a33c9861edc691a2003531f5da4f6bc8 \ + --hash=sha256:84d27a4832cc1a0ee07cdcf2b0629a8a72db73f4cf6de6f0904f6661227f256f \ + --hash=sha256:8531fdcad636d82c517b26a448dcfe62f720e1922b33c81ce695d0edb91eb931 \ + --hash=sha256:86d2a77fd490ae3ff6fae1c6ceaecad063d3cc2320b44377efdde79880e11526 \ + --hash=sha256:88fc51d9a26b10fc331be344f1781224a375b78488fc343620184e95a4b27016 \ + --hash=sha256:8a34e13a62a59c871064dfd8ffb150867e54291e46d4a7cf11d02c94a5275bae \ + --hash=sha256:8c82f11964f010053e13daafdc7154ce7385ecc538989a354ccc7067fd7028fd \ + --hash=sha256:92b2065d642bf8c0a82d59e59053dd2fdde64d4ed44efe4870fa816c1232647b \ + --hash=sha256:97b52894d948d2f6ea480171a27122d77af14ced35f62e5c892ca2fae9344311 \ + --hash=sha256:9d9acd80072abcc98bd2c86c3c9cd4ac2347b5a5a0cae7ed5c0ee5675f86d9af \ + --hash=sha256:9f59a3c656fef341a99e3d63189852be7084c0e54b75734cde571182c087b152 \ + --hash=sha256:aa5003845cdd21ac0dc6c9bf661c5beddd01116f6eb9eb3c8e272353d45b3288 \ + --hash=sha256:b16fff62b45eccb9c7abb18e60e7e446998093cdcb50fed33134b9b6878836de \ + --hash=sha256:b30c6590146e53149f04e85a6e4fcae068df4289e31e4aee1fdf56a0dead8f97 \ + --hash=sha256:b58cbf0697721120866820b89f93659abc31c1e876bf20d0b3d03cef14faf84d \ + --hash=sha256:b67c6f5e5a401fc56394f191f00f9b3811fe843ee93f4a70df3c389d1adf857d \ + --hash=sha256:bceab846bac555aff6427d060f2fcfff71042dba6f5fca7dc4f75cac815e57ca \ + --hash=sha256:bee9fcb41db2a23bed96c6b6ead6489702c12334ea20a297aa095ce6d31370d0 \ + --hash=sha256:c114e8da9b475739dde229fd3bc6b05a6537a88a578358bc8eb29b4030fac9c9 \ + --hash=sha256:c1f0524f203e3bd35149f12157438f406eff2e4fb30f71221c8a5eceb3617b6b \ + --hash=sha256:c792ea4eabc0159535608fc5658a74d1a81020eb35195dd63214dcf07556f67e \ + --hash=sha256:c7f3cb904cce8e1be667c7e6fef4516b98d1a6a0635a58a57528d577ac18a128 \ + --hash=sha256:d67ac60a307f760c6e65dad586f556dde58e683fab03323221a4e530ead6f74d \ + --hash=sha256:dcacf2c7a6c3a84e720d1bb2b543c675bf6c40e460300b628bab1b1efc7c034c \ + --hash=sha256:de36fe9c02995c7e6ae6efe2e205816f5f00c22fd1fbf343d4d18c3d5ceac2f5 \ + --hash=sha256:def07915168ac8f7853812cc593c71185a16216e9e4fa886358a17ed0fd9fcf6 \ + --hash=sha256:df41b9bc27c2c25b486bae7cf42fccdc52ff181c8c387bfd026624a491c2671b \ + --hash=sha256:e052b8467dd07d4943936009f46ae5ce7b908ddcac3fda581656b1b19c083d9b \ + --hash=sha256:e063b1865974611313a3849d43f2c3f5368093691349cf3c7c8f8f75ad7cb280 \ + --hash=sha256:e1459677e5d12be8bbc7584c35b992eea142911a6236a3278b9b5ce3326f282c \ + --hash=sha256:e1a99a7a71631f0efe727c10edfba09ea6bee4166a6f9c19aafb6c0b5917d09c \ + --hash=sha256:e590228200fcfc7e9109509e4d9125eace2042fd52b595dd22bbc34bb282307f \ + --hash=sha256:e6316827e3e79b7b8e7d8e3b08f4e331af91a48e794d5d8b099928b6f0b85f20 \ + --hash=sha256:e7837cb169eca3b3ae94cc5787c4fed99eef74c0ab9506756eea335e0d6f3ed8 \ + --hash=sha256:e848f46a58b9fcf3d06061d17be388caf70ea5b8cc3466251963c8345e13f7eb \ + --hash=sha256:ed058398f55163a79bb9f06a90ef9ccc063b204bb346c4de78efc5d15abfe602 \ + --hash=sha256:f2e58f2c36cc52d41f2659e4c0cbf7353e28c8c9e63e30d8c6d3494dc9fdedcf \ + --hash=sha256:f467ba0050b7de85016b43f5a22b46383ef004c4f672148a8abf32bc999a87f0 \ + --hash=sha256:f61bdb1df43dc9c131791fbc2355535f9024b9a04398d3bd0684fc16ab07df74 \ + --hash=sha256:fb06eea71a00a7af0ae6aefbb932fb8a7df3cb390cc217d51a9ad7343de1b8d0 \ + --hash=sha256:ffd7dcaf744f25f82190856bc26ed81721508fc5cbf2a330751e135ff1283564 + # via -r requirements/requirements.in diff --git a/examples/multi_python_versions/tests/BUILD.bazel b/examples/multi_python_versions/tests/BUILD.bazel index 11fb98ca61..607058d992 100644 --- a/examples/multi_python_versions/tests/BUILD.bazel +++ b/examples/multi_python_versions/tests/BUILD.bazel @@ -23,24 +23,38 @@ py_binary( ) py_binary( - name = "version_3_9", + name = "version_3_10", srcs = ["version.py"], main = "version.py", - python_version = "3.9", + python_version = "3.10", ) py_binary( - name = "version_3_10", + name = "version_3_11", srcs = ["version.py"], main = "version.py", - python_version = "3.10", + python_version = "3.11", ) py_binary( - name = "version_3_11", + name = "version_3_12", srcs = ["version.py"], main = "version.py", - python_version = "3.11", + python_version = "3.12", +) + +py_binary( + name = "version_3_13", + srcs = ["version.py"], + main = "version.py", + python_version = "3.13", +) + +py_binary( + name = "version_3_14", + srcs = ["version.py"], + main = "version.py", + python_version = "3.14", ) py_test( @@ -51,26 +65,42 @@ py_test( ) py_test( - name = "my_lib_3_9_test", + name = "my_lib_3_10_test", + srcs = ["my_lib_test.py"], + main = "my_lib_test.py", + python_version = "3.10", + deps = ["//libs/my_lib"], +) + +py_test( + name = "my_lib_3_11_test", srcs = ["my_lib_test.py"], main = "my_lib_test.py", - python_version = "3.9", + python_version = "3.11", deps = ["//libs/my_lib"], ) py_test( - name = "my_lib_3_10_test", + name = "my_lib_3_12_test", srcs = ["my_lib_test.py"], main = "my_lib_test.py", - python_version = "3.10", + python_version = "3.12", deps = ["//libs/my_lib"], ) py_test( - name = "my_lib_3_11_test", + name = "my_lib_3_13_test", srcs = ["my_lib_test.py"], main = "my_lib_test.py", - python_version = "3.11", + python_version = "3.13", + deps = ["//libs/my_lib"], +) + +py_test( + name = "my_lib_3_14_test", + srcs = ["my_lib_test.py"], + main = "my_lib_test.py", + python_version = "3.14", deps = ["//libs/my_lib"], ) @@ -84,15 +114,7 @@ copy_file( py_test( name = "version_default_test", srcs = ["version_default_test.py"], - env = {"VERSION_CHECK": "3.9"}, # The default defined in the WORKSPACE. -) - -py_test( - name = "version_3_9_test", - srcs = ["version_test.py"], - env = {"VERSION_CHECK": "3.9"}, - main = "version_test.py", - python_version = "3.9", + env = {"VERSION_CHECK": "3.10"}, # The default defined in the WORKSPACE/MODULE ) py_test( @@ -112,28 +134,52 @@ py_test( ) py_test( - name = "version_default_takes_3_10_subprocess_test", + name = "version_3_12_test", + srcs = ["version_test.py"], + env = {"VERSION_CHECK": "3.12"}, + main = "version_test.py", + python_version = "3.12", +) + +py_test( + name = "version_3_13_test", + srcs = ["version_test.py"], + env = {"VERSION_CHECK": "3.13"}, + main = "version_test.py", + python_version = "3.13", +) + +py_test( + name = "version_3_14_test", + srcs = ["version_test.py"], + env = {"VERSION_CHECK": "3.14"}, + main = "version_test.py", + python_version = "3.14", +) + +py_test( + name = "version_default_takes_3_11_subprocess_test", srcs = ["cross_version_test.py"], - data = [":version_3_10"], + data = [":version_3_11"], env = { - "SUBPROCESS_VERSION_CHECK": "3.10", - "SUBPROCESS_VERSION_PY_BINARY": "$(rootpaths :version_3_10)", - "VERSION_CHECK": "3.9", + "SUBPROCESS_VERSION_CHECK": "3.11", + "SUBPROCESS_VERSION_PY_BINARY": "$(rootpaths :version_3_11)", + "VERSION_CHECK": "3.10", }, main = "cross_version_test.py", ) py_test( - name = "version_3_10_takes_3_9_subprocess_test", + name = "version_3_11_takes_3_10_subprocess_test", srcs = ["cross_version_test.py"], - data = [":version_3_9"], + data = [":version_3_10"], env = { - "SUBPROCESS_VERSION_CHECK": "3.9", - "SUBPROCESS_VERSION_PY_BINARY": "$(rootpaths :version_3_9)", - "VERSION_CHECK": "3.10", + "SUBPROCESS_VERSION_CHECK": "3.10", + "SUBPROCESS_VERSION_PY_BINARY": "$(rootpaths :version_3_10)", + "VERSION_CHECK": "3.11", }, main = "cross_version_test.py", - python_version = "3.10", + python_version = "3.11", ) sh_test( @@ -141,28 +187,28 @@ sh_test( srcs = ["version_test.sh"], data = [":version_default"], env = { - "VERSION_CHECK": "3.9", # The default defined in the WORKSPACE. + "VERSION_CHECK": "3.10", # The default defined in the WORKSPACE/MODULE. "VERSION_PY_BINARY": "$(rootpaths :version_default)", }, ) sh_test( - name = "version_test_binary_3_9", + name = "version_test_binary_3_10", srcs = ["version_test.sh"], - data = [":version_3_9"], + data = [":version_3_10"], env = { - "VERSION_CHECK": "3.9", - "VERSION_PY_BINARY": "$(rootpaths :version_3_9)", + "VERSION_CHECK": "3.10", + "VERSION_PY_BINARY": "$(rootpaths :version_3_10)", }, ) sh_test( - name = "version_test_binary_3_10", + name = "version_test_binary_3_11", srcs = ["version_test.sh"], - data = [":version_3_10"], + data = [":version_3_11"], env = { - "VERSION_CHECK": "3.10", - "VERSION_PY_BINARY": "$(rootpaths :version_3_10)", + "VERSION_CHECK": "3.11", + "VERSION_PY_BINARY": "$(rootpaths :version_3_11)", }, ) From 44e7723acfd79afd32fcd0637abe3ed7c0bc48d8 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Fri, 21 Nov 2025 23:22:04 -0800 Subject: [PATCH 529/922] chore: enable disk cache for faster local builds (#3424) The disk cache can greatly speed up local builds, in particular, if docs are built which requires building protobuf. --- .bazelrc | 3 +++ 1 file changed, 3 insertions(+) diff --git a/.bazelrc b/.bazelrc index b5c9c7c1e2..f9e00fe531 100644 --- a/.bazelrc +++ b/.bazelrc @@ -30,6 +30,9 @@ build --enable_runfiles # Make Bazel 7 use bzlmod by default common --enable_bzlmod +# Local disk cache greatly speeds up builds if the regular cache is lost +common --disk_cache=~/.cache/bazel/bazel-disk-cache + # Additional config to use for readthedocs builds. # See .readthedocs.yml for additional flags that can only be determined from # the runtime environment. From 877555589865bed20e874311040b7a12df75d2c7 Mon Sep 17 00:00:00 2001 From: Ignas Anikevicius <240938+aignas@users.noreply.github.com> Date: Sun, 23 Nov 2025 00:30:04 +0900 Subject: [PATCH 530/922] chore(bazelrc): fix the update-deleted-packages script (#3425) Before this we were relying on an upstream script, but because how we are building our docs we need to customize how it is working. What is more, the format was hard to maintain and prone to human errors. This splits out the deleted packages code into a separate rc file that is much easier to work with. Split out of #3421 --- .bazelrc | 9 ++--- .bazelrc.deleted_packages | 56 ++++++++++++++++++++++++++++++++ .pre-commit-config.yaml | 2 +- tools/update_deleted_packages.sh | 14 ++++---- 4 files changed, 67 insertions(+), 14 deletions(-) create mode 100644 .bazelrc.deleted_packages diff --git a/.bazelrc b/.bazelrc index f9e00fe531..6473f10231 100644 --- a/.bazelrc +++ b/.bazelrc @@ -2,16 +2,11 @@ # Trick bazel into treating BUILD files under examples/* as being regular files # This lets us glob() up all the files inside the examples to make them inputs to tests # (Note, we cannot use `common --deleted_packages` because the bazel version command doesn't support it) -# To update these lines, execute -# `bazel run @rules_bazel_integration_test//tools:update_deleted_packages` -build --deleted_packages=examples/build_file_generation,examples/build_file_generation/random_number_generator,examples/bzlmod,examples/bzlmod_build_file_generation,examples/bzlmod_build_file_generation/other_module/other_module/pkg,examples/bzlmod_build_file_generation/runfiles,examples/bzlmod/entry_points,examples/bzlmod/entry_points/tests,examples/bzlmod/libs/my_lib,examples/bzlmod/other_module,examples/bzlmod/other_module/other_module/pkg,examples/bzlmod/patches,examples/bzlmod/py_proto_library,examples/bzlmod/py_proto_library/example.com/another_proto,examples/bzlmod/py_proto_library/example.com/proto,examples/bzlmod/runfiles,examples/bzlmod/tests,examples/bzlmod/tests/other_module,examples/bzlmod/whl_mods,examples/multi_python_versions/libs/my_lib,examples/multi_python_versions/requirements,examples/multi_python_versions/tests,examples/pip_parse,examples/pip_parse_vendored,examples/pip_repository_annotations,examples/py_proto_library,examples/py_proto_library/example.com/another_proto,examples/py_proto_library/example.com/proto,gazelle,gazelle/manifest,gazelle/manifest/generate,gazelle/manifest/hasher,gazelle/manifest/test,gazelle/modules_mapping,gazelle/python,gazelle/pythonconfig,gazelle/python/private,rules_python-repro,tests/integration/compile_pip_requirements,tests/integration/compile_pip_requirements_test_from_external_repo,tests/integration/custom_commands,tests/integration/ignore_root_user_error,tests/integration/ignore_root_user_error/submodule,tests/integration/local_toolchains,tests/integration/pip_parse,tests/integration/pip_parse/empty,tests/integration/py_cc_toolchain_registered,tests/modules/another_module,tests/modules/other,tests/modules/other/nspkg_delta,tests/modules/other/nspkg_gamma,tests/modules/other/nspkg_single,tests/modules/other/simple_v1,tests/modules/other/simple_v2,tests/modules/other/with_external_data,tests/whl_with_build_files/testdata,tests/whl_with_build_files/testdata/somepkg,tests/whl_with_build_files/testdata/somepkg-1.0.dist-info,tests/whl_with_build_files/testdata/somepkg/subpkg -query --deleted_packages=examples/build_file_generation,examples/build_file_generation/random_number_generator,examples/bzlmod,examples/bzlmod_build_file_generation,examples/bzlmod_build_file_generation/other_module/other_module/pkg,examples/bzlmod_build_file_generation/runfiles,examples/bzlmod/entry_points,examples/bzlmod/entry_points/tests,examples/bzlmod/libs/my_lib,examples/bzlmod/other_module,examples/bzlmod/other_module/other_module/pkg,examples/bzlmod/patches,examples/bzlmod/py_proto_library,examples/bzlmod/py_proto_library/example.com/another_proto,examples/bzlmod/py_proto_library/example.com/proto,examples/bzlmod/runfiles,examples/bzlmod/tests,examples/bzlmod/tests/other_module,examples/bzlmod/whl_mods,examples/multi_python_versions/libs/my_lib,examples/multi_python_versions/requirements,examples/multi_python_versions/tests,examples/pip_parse,examples/pip_parse_vendored,examples/pip_repository_annotations,examples/py_proto_library,examples/py_proto_library/example.com/another_proto,examples/py_proto_library/example.com/proto,gazelle,gazelle/manifest,gazelle/manifest/generate,gazelle/manifest/hasher,gazelle/manifest/test,gazelle/modules_mapping,gazelle/python,gazelle/pythonconfig,gazelle/python/private,rules_python-repro,tests/integration/compile_pip_requirements,tests/integration/compile_pip_requirements_test_from_external_repo,tests/integration/custom_commands,tests/integration/ignore_root_user_error,tests/integration/ignore_root_user_error/submodule,tests/integration/local_toolchains,tests/integration/pip_parse,tests/integration/pip_parse/empty,tests/integration/py_cc_toolchain_registered,tests/modules/another_module,tests/modules/other,tests/modules/other/nspkg_delta,tests/modules/other/nspkg_gamma,tests/modules/other/nspkg_single,tests/modules/other/simple_v1,tests/modules/other/simple_v2,tests/modules/other/with_external_data,tests/whl_with_build_files/testdata,tests/whl_with_build_files/testdata/somepkg,tests/whl_with_build_files/testdata/somepkg-1.0.dist-info,tests/whl_with_build_files/testdata/somepkg/subpkg +# To update the file, execute +import %workspace%/.bazelrc.deleted_packages test --test_output=errors -common --deleted_packages=gazelle/examples/bzlmod_build_file_generation -common --deleted_packages=gazelle/examples/bzlmod_build_file_generation/runfiles - # Do NOT implicitly create empty __init__.py files in the runfiles tree. # By default, these are created in every directory containing Python source code # or shared libraries, and every parent directory of those directories, diff --git a/.bazelrc.deleted_packages b/.bazelrc.deleted_packages new file mode 100644 index 0000000000..45e433a881 --- /dev/null +++ b/.bazelrc.deleted_packages @@ -0,0 +1,56 @@ +# Generated via './tools/update_deleted_packages.sh' +common --deleted_packages=examples/build_file_generation +common --deleted_packages=examples/build_file_generation/random_number_generator +common --deleted_packages=examples/bzlmod +common --deleted_packages=examples/bzlmod/entry_points +common --deleted_packages=examples/bzlmod/entry_points/tests +common --deleted_packages=examples/bzlmod/libs/my_lib +common --deleted_packages=examples/bzlmod/other_module +common --deleted_packages=examples/bzlmod/other_module/other_module/pkg +common --deleted_packages=examples/bzlmod/patches +common --deleted_packages=examples/bzlmod/py_proto_library +common --deleted_packages=examples/bzlmod/py_proto_library/example.com/another_proto +common --deleted_packages=examples/bzlmod/py_proto_library/example.com/proto +common --deleted_packages=examples/bzlmod/py_proto_library/foo_external +common --deleted_packages=examples/bzlmod/runfiles +common --deleted_packages=examples/bzlmod/tests +common --deleted_packages=examples/bzlmod/tests/other_module +common --deleted_packages=examples/bzlmod/whl_mods +common --deleted_packages=examples/multi_python_versions/libs/my_lib +common --deleted_packages=examples/multi_python_versions/requirements +common --deleted_packages=examples/multi_python_versions/tests +common --deleted_packages=examples/pip_parse +common --deleted_packages=examples/pip_parse_vendored +common --deleted_packages=examples/pip_repository_annotations +common --deleted_packages=examples/py_proto_library +common --deleted_packages=examples/py_proto_library/example.com/another_proto +common --deleted_packages=examples/py_proto_library/example.com/proto +common --deleted_packages=gazelle +common --deleted_packages=gazelle/examples/bzlmod_build_file_generation +common --deleted_packages=gazelle/examples/bzlmod_build_file_generation/other_module/other_module/pkg +common --deleted_packages=gazelle/examples/bzlmod_build_file_generation/runfiles +common --deleted_packages=gazelle/manifest +common --deleted_packages=gazelle/manifest/generate +common --deleted_packages=gazelle/manifest/hasher +common --deleted_packages=gazelle/manifest/test +common --deleted_packages=gazelle/modules_mapping +common --deleted_packages=gazelle/python +common --deleted_packages=gazelle/pythonconfig +common --deleted_packages=gazelle/python/private +common --deleted_packages=tests/integration/compile_pip_requirements +common --deleted_packages=tests/integration/compile_pip_requirements_test_from_external_repo +common --deleted_packages=tests/integration/custom_commands +common --deleted_packages=tests/integration/ignore_root_user_error +common --deleted_packages=tests/integration/ignore_root_user_error/submodule +common --deleted_packages=tests/integration/local_toolchains +common --deleted_packages=tests/integration/pip_parse +common --deleted_packages=tests/integration/pip_parse/empty +common --deleted_packages=tests/integration/py_cc_toolchain_registered +common --deleted_packages=tests/modules/another_module +common --deleted_packages=tests/modules/other +common --deleted_packages=tests/modules/other/nspkg_delta +common --deleted_packages=tests/modules/other/nspkg_gamma +common --deleted_packages=tests/modules/other/nspkg_single +common --deleted_packages=tests/modules/other/simple_v1 +common --deleted_packages=tests/modules/other/simple_v2 +common --deleted_packages=tests/modules/other/with_external_data diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml index 91e449f950..57d31f5f5f 100644 --- a/.pre-commit-config.yaml +++ b/.pre-commit-config.yaml @@ -48,6 +48,6 @@ repos: language: system # 7.x is necessary until https://github.com/bazel-contrib/rules_bazel_integration_test/pull/414 # is merged and released - entry: env USE_BAZEL_VERSION=7.x bazel run @rules_bazel_integration_test//tools:update_deleted_packages + entry: ./tools/update_deleted_packages.sh files: ^((examples|tests)/.*/(MODULE.bazel|WORKSPACE|WORKSPACE.bzlmod|BUILD.bazel)|.bazelrc)$ pass_filenames: false diff --git a/tools/update_deleted_packages.sh b/tools/update_deleted_packages.sh index 17e33d182a..83bed91d16 100755 --- a/tools/update_deleted_packages.sh +++ b/tools/update_deleted_packages.sh @@ -23,17 +23,19 @@ # 2. For each of the directories, get all directories that contains a BUILD.bazel file. # 3. Sort and remove duplicates. -set -euxo pipefail +set -euo pipefail DIR="$(dirname $0)/.." cd $DIR # The sed -i.bak pattern is compatible between macos and linux -sed -i.bak "/^[^#].*--deleted_packages/s#=.*#=$(\ - find examples/*/* tests/*/* \( -name WORKSPACE -or -name MODULE.bazel \) | +{ + echo "# Generated via './tools/update_deleted_packages.sh'" + find examples tests gazelle \( -name WORKSPACE -or -name MODULE.bazel \) | xargs -n 1 dirname | - xargs -n 1 -I{} find {} \( -name BUILD -or -name BUILD.bazel \) | + xargs -I{} find {} \( -name BUILD -or -name BUILD.bazel \) | xargs -n 1 dirname | + grep -v "gazelle/docs" | sort -u | - paste -sd, -\ -)#" $DIR/.bazelrc && rm .bazelrc.bak + sed 's/^/common --deleted_packages=/g' +} | tee "$DIR"/.bazelrc.deleted_packages From 23397d50c638af74419f43fff300551f0085fc49 Mon Sep 17 00:00:00 2001 From: Ignas Anikevicius <240938+aignas@users.noreply.github.com> Date: Sun, 23 Nov 2025 09:28:49 +0900 Subject: [PATCH 531/922] chore(toolchain): remove chmod and disable ignore_root_error (#3421) As discussed in the #2024 messaged we decided to remove the chmoding and simplify the setup for all of our users. This in effect unifies how the toolchains are created across all of the platforms reducing the need for some of the integration tests. Summary: - `python_repository`: stop chmoding - `python_repository`: make `ignore_root_user_error` noop - `python(bzlmod)`: stop using `ignore_root_user_error`. - `tests`: remove `ignore_root_user_error` tests. Fixes #2016 Fixes #2053 Closes #2024 --- .bazelci/presubmit.yml | 13 --- .bazelignore | 1 - .bazelrc.deleted_packages | 2 - CHANGELOG.md | 5 ++ python/private/python.bzl | 34 +------- python/private/python_repository.bzl | 62 +++------------ tests/integration/BUILD.bazel | 16 ---- .../ignore_root_user_error/.bazelrc | 7 -- .../ignore_root_user_error/.gitignore | 1 - .../ignore_root_user_error/BUILD.bazel | 32 -------- .../ignore_root_user_error/MODULE.bazel | 20 ----- .../ignore_root_user_error/README.md | 2 - .../ignore_root_user_error/WORKSPACE | 14 ---- .../ignore_root_user_error/bzlmod_test.py | 40 ---------- .../ignore_root_user_error/foo_test.py | 13 --- .../submodule/BUILD.bazel | 0 .../submodule/MODULE.bazel | 9 --- .../submodule/WORKSPACE | 0 tests/python/python_tests.bzl | 79 +------------------ 19 files changed, 20 insertions(+), 330 deletions(-) delete mode 100644 tests/integration/ignore_root_user_error/.bazelrc delete mode 100644 tests/integration/ignore_root_user_error/.gitignore delete mode 100644 tests/integration/ignore_root_user_error/BUILD.bazel delete mode 100644 tests/integration/ignore_root_user_error/MODULE.bazel delete mode 100644 tests/integration/ignore_root_user_error/README.md delete mode 100644 tests/integration/ignore_root_user_error/WORKSPACE delete mode 100644 tests/integration/ignore_root_user_error/bzlmod_test.py delete mode 100644 tests/integration/ignore_root_user_error/foo_test.py delete mode 100644 tests/integration/ignore_root_user_error/submodule/BUILD.bazel delete mode 100644 tests/integration/ignore_root_user_error/submodule/MODULE.bazel delete mode 100644 tests/integration/ignore_root_user_error/submodule/WORKSPACE diff --git a/.bazelci/presubmit.yml b/.bazelci/presubmit.yml index 256961bf36..548d1e98f6 100644 --- a/.bazelci/presubmit.yml +++ b/.bazelci/presubmit.yml @@ -615,19 +615,6 @@ tasks: - "git diff --exit-code" - integration_test_ignore_root_user_error_macos_workspace: - <<: *reusable_build_test_all - <<: *common_workspace_flags - name: "ignore_root_user_error: macOS, workspace" - working_directory: tests/integration/ignore_root_user_error - platform: macos - integration_test_ignore_root_user_error_windows_workspace: - <<: *reusable_build_test_all - <<: *common_workspace_flags - name: "ignore_root_user_error: Windows, workspace" - working_directory: tests/integration/ignore_root_user_error - platform: windows - integration_compile_pip_requirements_test_from_external_repo_ubuntu_min_workspace: <<: *minimum_supported_version <<: *common_workspace_flags_min_bazel diff --git a/.bazelignore b/.bazelignore index dd58b79e3c..2f50cc2c52 100644 --- a/.bazelignore +++ b/.bazelignore @@ -31,6 +31,5 @@ gazelle/examples/bzlmod_build_file_generation/bazel-bzlmod_build_file_generation gazelle/examples/bzlmod_build_file_generation/bazel-out gazelle/examples/bzlmod_build_file_generation/bazel-testlog tests/integration/compile_pip_requirements/bazel-compile_pip_requirements -tests/integration/ignore_root_user_error/bazel-ignore_root_user_error tests/integration/local_toolchains/bazel-local_toolchains tests/integration/py_cc_toolchain_registered/bazel-py_cc_toolchain_registered diff --git a/.bazelrc.deleted_packages b/.bazelrc.deleted_packages index 45e433a881..d11f96d664 100644 --- a/.bazelrc.deleted_packages +++ b/.bazelrc.deleted_packages @@ -40,8 +40,6 @@ common --deleted_packages=gazelle/python/private common --deleted_packages=tests/integration/compile_pip_requirements common --deleted_packages=tests/integration/compile_pip_requirements_test_from_external_repo common --deleted_packages=tests/integration/custom_commands -common --deleted_packages=tests/integration/ignore_root_user_error -common --deleted_packages=tests/integration/ignore_root_user_error/submodule common --deleted_packages=tests/integration/local_toolchains common --deleted_packages=tests/integration/pip_parse common --deleted_packages=tests/integration/pip_parse/empty diff --git a/CHANGELOG.md b/CHANGELOG.md index 6709a9dff1..edcab7a357 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -62,6 +62,11 @@ END_UNRELEASED_TEMPLATE * (toolchain) Remove all of the python 3.9 toolchain versions except for the `3.9.25`. This version has reached EOL and will no longer receive any security fixes, please update to `3.10` or above. ([#2704](https://github.com/bazel-contrib/rules_python/issues/2704)) +* (toolchain) `ignore_root_user_error` has now been flipped to be always enabled and + the `chmod` of the python toolchain directories have been removed. From now on `rules_python` + always adds the `pyc` files to the glob excludes and in order to avoid any problems when using + the toolchains in the repository phase, ensure that you pass `-B` to the python interpreter. + ([#2016](https://github.com/bazel-contrib/rules_python/issues/2016)) {#v0-0-0-changed} ### Changed diff --git a/python/private/python.bzl b/python/private/python.bzl index 22f4753a62..80f2afac53 100644 --- a/python/private/python.bzl +++ b/python/private/python.bzl @@ -76,13 +76,6 @@ def parse_modules(*, module_ctx, logger, _fail = fail): # Map of string Major.Minor or Major.Minor.Patch to the toolchain_info struct global_toolchain_versions = {} - ignore_root_user_error = None - - # if the root module does not register any toolchain then the - # ignore_root_user_error takes its default value: True - if not module_ctx.modules[0].tags.toolchain: - ignore_root_user_error = True - config = _get_toolchain_config(modules = module_ctx.modules, _fail = _fail) default_python_version = _compute_default_python_version(module_ctx) @@ -115,15 +108,6 @@ def parse_modules(*, module_ctx, logger, _fail = fail): # * The root module is allowed to override the rules_python default. is_default = default_python_version == toolchain_version - # Also only the root module should be able to decide ignore_root_user_error. - # Modules being depended upon don't know the final environment, so they aren't - # in the right position to know or decide what the correct setting is. - - # If an inconsistency in the ignore_root_user_error among multiple toolchains is detected, fail. - if ignore_root_user_error != None and toolchain_attr.ignore_root_user_error != ignore_root_user_error: - fail("Toolchains in the root module must have consistent 'ignore_root_user_error' attributes") - - ignore_root_user_error = toolchain_attr.ignore_root_user_error elif mod.name == "rules_python" and not default_toolchain: # This branch handles when the root module doesn't declare a # Python toolchain @@ -166,7 +150,6 @@ def parse_modules(*, module_ctx, logger, _fail = fail): global_toolchain_versions[toolchain_version] = toolchain_info if debug_info: debug_info["toolchains_registered"].append({ - "ignore_root_user_error": ignore_root_user_error, "module": {"is_root": mod.is_root, "name": mod.name}, "name": toolchain_name, }) @@ -185,8 +168,6 @@ def parse_modules(*, module_ctx, logger, _fail = fail): elif toolchain_info: toolchains.append(toolchain_info) - config.default.setdefault("ignore_root_user_error", ignore_root_user_error) - # A default toolchain is required so that the non-version-specific rules # are able to match a toolchain. if default_toolchain == None: @@ -722,7 +703,6 @@ def _process_global_overrides(*, tag, default, _fail = fail): default["minor_mapping"] = tag.minor_mapping forwarded_attrs = sorted(AUTH_ATTRS) + [ - "ignore_root_user_error", "base_url", "register_all_versions", ] @@ -974,7 +954,6 @@ def _create_toolchain_attrs_struct( is_default = is_default, python_version = python_version if python_version else tag.python_version, configure_coverage_tool = getattr(tag, "configure_coverage_tool", False), - ignore_root_user_error = getattr(tag, "ignore_root_user_error", True), ) _defaults = tag_class( @@ -1086,16 +1065,9 @@ Then the python interpreter will be available as `my_python_name`. "ignore_root_user_error": attr.bool( default = True, doc = """\ -The Python runtime installation is made read only. This improves the ability for -Bazel to cache it by preventing the interpreter from creating `.pyc` files for -the standard library dynamically at runtime as they are loaded (this often leads -to spurious cache misses or build failures). - -However, if the user is running Bazel as root, this read-onlyness is not -respected. Bazel will print a warning message when it detects that the runtime -installation is writable despite being made read only (i.e. it's running with -root access) while this attribute is set `False`, however this messaging can be ignored by setting -this to `False`. +:::{versionchanged} VERSION_NEXT_FEATURE +Noop, will be removed in the next major release. +::: """, mandatory = False, ), diff --git a/python/private/python_repository.bzl b/python/private/python_repository.bzl index cb0731e6eb..16c522f398 100644 --- a/python/private/python_repository.bzl +++ b/python/private/python_repository.bzl @@ -123,45 +123,6 @@ def _python_repository_impl(rctx): logger = logger, ) - # Make the Python installation read-only. This is to prevent issues due to - # pycs being generated at runtime: - # * The pycs are not deterministic (they contain timestamps) - # * Multiple processes trying to write the same pycs can result in errors. - # - # Note, when on Windows the `chmod` may not work - if "windows" not in platform and "windows" != repo_utils.get_platforms_os_name(rctx): - repo_utils.execute_checked( - rctx, - op = "python_repository.MakeReadOnly", - arguments = [repo_utils.which_checked(rctx, "chmod"), "-R", "ugo-w", "lib"], - logger = logger, - ) - - # If the user is not ignoring the warnings, then proceed to run a check, - # otherwise these steps can be skipped, as they both result in some warning. - if not rctx.attr.ignore_root_user_error: - exec_result = repo_utils.execute_unchecked( - rctx, - op = "python_repository.TestReadOnly", - arguments = [repo_utils.which_checked(rctx, "touch"), "lib/.test"], - logger = logger, - ) - - # The issue with running as root is the installation is no longer - # read-only, so the problems due to pyc can resurface. - if exec_result.return_code == 0: - stdout = repo_utils.execute_checked_stdout( - rctx, - op = "python_repository.GetUserId", - arguments = [repo_utils.which_checked(rctx, "id"), "-u"], - logger = logger, - ) - uid = int(stdout.strip()) - if uid == 0: - logger.warn("The current user is root, which can cause spurious cache misses or build failures with the hermetic Python interpreter. See https://github.com/bazel-contrib/rules_python/pull/713.") - else: - logger.warn("The current user has CAP_DAC_OVERRIDE set, which can cause spurious cache misses or build failures with the hermetic Python interpreter. See https://github.com/bazel-contrib/rules_python/pull/713.") - python_bin = "python.exe" if ("windows" in platform) else "bin/python3" if "linux" in platform: @@ -186,17 +147,15 @@ def _python_repository_impl(rctx): break glob_include = [] - glob_exclude = [] - if rctx.attr.ignore_root_user_error or "windows" in platform: - glob_exclude += [ - # These pycache files are created on first use of the associated python files. - # Exclude them from the glob because otherwise between the first time and second time a python toolchain is used," - # the definition of this filegroup will change, and depending rules will get invalidated." - # See https://github.com/bazel-contrib/rules_python/issues/1008 for unconditionally adding these to toolchains so we can stop ignoring them." - # pyc* is ignored because pyc creation creates temporary .pyc.NNNN files - "**/__pycache__/*.pyc*", - "**/__pycache__/*.pyo*", - ] + glob_exclude = [ + # These pycache files are created on first use of the associated python files. + # Exclude them from the glob because otherwise between the first time and second time a python toolchain is used," + # the definition of this filegroup will change, and depending rules will get invalidated." + # See https://github.com/bazel-contrib/rules_python/issues/1008 for unconditionally adding these to toolchains so we can stop ignoring them." + # pyc* is ignored because pyc creation creates temporary .pyc.NNNN files + "**/__pycache__/*.pyc*", + "**/__pycache__/*.pyo*", + ] if "windows" in platform: glob_include += [ @@ -249,7 +208,6 @@ define_hermetic_runtime_toolchain_impl( "coverage_tool": rctx.attr.coverage_tool, "distutils": rctx.attr.distutils, "distutils_content": rctx.attr.distutils_content, - "ignore_root_user_error": rctx.attr.ignore_root_user_error, "name": rctx.attr.name, "netrc": rctx.attr.netrc, "patch_strip": rctx.attr.patch_strip, @@ -299,7 +257,7 @@ For more information see {attr}`py_runtime.coverage_tool`. ), "ignore_root_user_error": attr.bool( default = True, - doc = "Whether the check for root should be ignored or not. This causes cache misses with .pyc files.", + doc = "Noop, will be removed in the next major release", mandatory = False, ), "netrc": attr.string( diff --git a/tests/integration/BUILD.bazel b/tests/integration/BUILD.bazel index 0e47faf91a..673312903d 100644 --- a/tests/integration/BUILD.bazel +++ b/tests/integration/BUILD.bazel @@ -75,22 +75,6 @@ rules_python_integration_test( workspace_path = "compile_pip_requirements", ) -rules_python_integration_test( - name = "ignore_root_user_error_test", - env = { - "RULES_PYTHON_BZLMOD_DEBUG": "1", - }, -) - -rules_python_integration_test( - name = "ignore_root_user_error_workspace_test", - bzlmod = False, - env = { - "RULES_PYTHON_BZLMOD_DEBUG": "1", - }, - workspace_path = "ignore_root_user_error", -) - rules_python_integration_test( name = "local_toolchains_test", env = { diff --git a/tests/integration/ignore_root_user_error/.bazelrc b/tests/integration/ignore_root_user_error/.bazelrc deleted file mode 100644 index bb7b5742cd..0000000000 --- a/tests/integration/ignore_root_user_error/.bazelrc +++ /dev/null @@ -1,7 +0,0 @@ -common --action_env=RULES_PYTHON_BZLMOD_DEBUG=1 -common --lockfile_mode=off -test --test_output=errors - -# Windows requires these for multi-python support: -build --enable_runfiles -common:bazel7.x --incompatible_python_disallow_native_rules diff --git a/tests/integration/ignore_root_user_error/.gitignore b/tests/integration/ignore_root_user_error/.gitignore deleted file mode 100644 index ac51a054d2..0000000000 --- a/tests/integration/ignore_root_user_error/.gitignore +++ /dev/null @@ -1 +0,0 @@ -bazel-* diff --git a/tests/integration/ignore_root_user_error/BUILD.bazel b/tests/integration/ignore_root_user_error/BUILD.bazel deleted file mode 100644 index 6e3b7b9d24..0000000000 --- a/tests/integration/ignore_root_user_error/BUILD.bazel +++ /dev/null @@ -1,32 +0,0 @@ -# Copyright 2024 The Bazel Authors. All rights reserved. -# -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -load("@rules_python//python:py_test.bzl", "py_test") -load("@rules_python//python/private:bzlmod_enabled.bzl", "BZLMOD_ENABLED") # buildifier: disable=bzl-visibility - -py_test( - name = "foo_test", - srcs = ["foo_test.py"], - visibility = ["//visibility:public"], -) - -py_test( - name = "bzlmod_test", - srcs = ["bzlmod_test.py"], - data = [ - "@rules_python//python/runfiles", - "@rules_python_bzlmod_debug//:debug_info.json", - ], - target_compatible_with = [] if BZLMOD_ENABLED else ["@platforms//:incompatible"], -) diff --git a/tests/integration/ignore_root_user_error/MODULE.bazel b/tests/integration/ignore_root_user_error/MODULE.bazel deleted file mode 100644 index 15c37c4388..0000000000 --- a/tests/integration/ignore_root_user_error/MODULE.bazel +++ /dev/null @@ -1,20 +0,0 @@ -module(name = "ignore_root_user_error") - -bazel_dep(name = "rules_python", version = "0.0.0") -local_path_override( - module_name = "rules_python", - path = "../../..", -) - -bazel_dep(name = "submodule") -local_path_override( - module_name = "submodule", - path = "submodule", -) - -python = use_extension("@rules_python//python/extensions:python.bzl", "python") -python.toolchain( - ignore_root_user_error = True, - python_version = "3.11", -) -use_repo(python, "rules_python_bzlmod_debug") diff --git a/tests/integration/ignore_root_user_error/README.md b/tests/integration/ignore_root_user_error/README.md deleted file mode 100644 index 47da5eb9ad..0000000000 --- a/tests/integration/ignore_root_user_error/README.md +++ /dev/null @@ -1,2 +0,0 @@ -# ignore_root_user_errors -There are cases when we have to run Python targets with root, e.g., in Docker containers, requiring setting `ignore_root_user_error = True` when registering Python toolchain. This test makes sure that rules_python works in this case. \ No newline at end of file diff --git a/tests/integration/ignore_root_user_error/WORKSPACE b/tests/integration/ignore_root_user_error/WORKSPACE deleted file mode 100644 index 7ac0a609eb..0000000000 --- a/tests/integration/ignore_root_user_error/WORKSPACE +++ /dev/null @@ -1,14 +0,0 @@ -local_repository( - name = "rules_python", - path = "../../..", -) - -load("@rules_python//python:repositories.bzl", "py_repositories", "python_register_toolchains") - -py_repositories() - -python_register_toolchains( - name = "python39", - ignore_root_user_error = True, - python_version = "3.9", -) diff --git a/tests/integration/ignore_root_user_error/bzlmod_test.py b/tests/integration/ignore_root_user_error/bzlmod_test.py deleted file mode 100644 index a1d6dc0630..0000000000 --- a/tests/integration/ignore_root_user_error/bzlmod_test.py +++ /dev/null @@ -1,40 +0,0 @@ -# Copyright 2024 The Bazel Authors. All rights reserved. -# -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -import json -import pathlib -import unittest - -from python.runfiles import runfiles - - -class BzlmodTest(unittest.TestCase): - def test_ignore_root_user_error_true_for_all_toolchains(self): - rf = runfiles.Create() - debug_path = pathlib.Path( - rf.Rlocation("rules_python_bzlmod_debug/debug_info.json") - ) - debug_info = json.loads(debug_path.read_bytes()) - actual = debug_info["toolchains_registered"] - # Because the root module set ignore_root_user_error=True, that should - # be the default for all other toolchains. - for entry in actual: - self.assertTrue( - entry["ignore_root_user_error"], - msg=f"Expected ignore_root_user_error=True, but got: {entry}", - ) - - -if __name__ == "__main__": - unittest.main() diff --git a/tests/integration/ignore_root_user_error/foo_test.py b/tests/integration/ignore_root_user_error/foo_test.py deleted file mode 100644 index 724cdcb69a..0000000000 --- a/tests/integration/ignore_root_user_error/foo_test.py +++ /dev/null @@ -1,13 +0,0 @@ -# Copyright 2019 The Bazel Authors. All rights reserved. -# -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. diff --git a/tests/integration/ignore_root_user_error/submodule/BUILD.bazel b/tests/integration/ignore_root_user_error/submodule/BUILD.bazel deleted file mode 100644 index e69de29bb2..0000000000 diff --git a/tests/integration/ignore_root_user_error/submodule/MODULE.bazel b/tests/integration/ignore_root_user_error/submodule/MODULE.bazel deleted file mode 100644 index f12870963c..0000000000 --- a/tests/integration/ignore_root_user_error/submodule/MODULE.bazel +++ /dev/null @@ -1,9 +0,0 @@ -module(name = "submodule") - -bazel_dep(name = "rules_python", version = "0.0.0") - -python = use_extension("@rules_python//python/extensions:python.bzl", "python") -python.toolchain( - ignore_root_user_error = False, - python_version = "3.10", -) diff --git a/tests/integration/ignore_root_user_error/submodule/WORKSPACE b/tests/integration/ignore_root_user_error/submodule/WORKSPACE deleted file mode 100644 index e69de29bb2..0000000000 diff --git a/tests/python/python_tests.bzl b/tests/python/python_tests.bzl index ff02cc859e..53cfd3b09c 100644 --- a/tests/python/python_tests.bzl +++ b/tests/python/python_tests.bzl @@ -75,7 +75,6 @@ def _override( auth_patterns = {}, available_python_versions = [], base_url = "", - ignore_root_user_error = True, minor_mapping = {}, netrc = "", register_all_versions = False): @@ -83,7 +82,6 @@ def _override( auth_patterns = auth_patterns, available_python_versions = available_python_versions, base_url = base_url, - ignore_root_user_error = ignore_root_user_error, minor_mapping = minor_mapping, netrc = netrc, register_all_versions = register_all_versions, @@ -164,11 +162,9 @@ def _test_default_from_rules_python_when_rules_python_is_root(env): env.expect.that_collection(py.config.kwargs).has_size(0) env.expect.that_collection(py.config.default.keys()).contains_exactly([ "base_url", - "ignore_root_user_error", "tool_versions", "platforms", ]) - env.expect.that_bool(py.config.default["ignore_root_user_error"]).equals(True) env.expect.that_str(py.default_python_version).equals("3.11") want_toolchain = struct( @@ -220,76 +216,6 @@ def _test_default_with_patch_version(env): _tests.append(_test_default_with_patch_version) -def _test_default_non_rules_python_ignore_root_user_error(env): - py = parse_modules( - module_ctx = _mock_mctx( - _mod( - name = "my_module", - toolchain = [_toolchain("3.12", ignore_root_user_error = False)], - is_root = True, - ), - _rules_python_module(), - ), - logger = repo_utils.logger(verbosity_level = 0, name = "python"), - ) - - env.expect.that_bool(py.config.default["ignore_root_user_error"]).equals(False) - env.expect.that_str(py.default_python_version).equals("3.12") - - my_module_toolchain = struct( - name = "python_3_12", - python_version = "3.12", - register_coverage_tool = False, - ) - rules_python_toolchain = struct( - name = "python_3_11", - python_version = "3.11", - register_coverage_tool = False, - ) - env.expect.that_collection(py.toolchains).contains_exactly([ - rules_python_toolchain, - my_module_toolchain, - ]).in_order() - -_tests.append(_test_default_non_rules_python_ignore_root_user_error) - -def _test_default_non_rules_python_ignore_root_user_error_non_root_module(env): - """Verify a non-root intermediate module has its ignore_root_user_error setting ignored.""" - py = parse_modules( - module_ctx = _mock_mctx( - _mod(name = "my_module", is_root = True, toolchain = [_toolchain("3.13")]), - _mod(name = "some_module", toolchain = [_toolchain("3.12", ignore_root_user_error = False)]), - _rules_python_module(), - ), - logger = repo_utils.logger(verbosity_level = 0, name = "python"), - ) - - env.expect.that_str(py.default_python_version).equals("3.13") - env.expect.that_bool(py.config.default["ignore_root_user_error"]).equals(True) - - my_module_toolchain = struct( - name = "python_3_13", - python_version = "3.13", - register_coverage_tool = False, - ) - some_module_toolchain = struct( - name = "python_3_12", - python_version = "3.12", - register_coverage_tool = False, - ) - rules_python_toolchain = struct( - name = "python_3_11", - python_version = "3.11", - register_coverage_tool = False, - ) - env.expect.that_collection(py.toolchains).contains_exactly([ - some_module_toolchain, - rules_python_toolchain, - my_module_toolchain, # this was the only toolchain, default to that - ]).in_order() - -_tests.append(_test_default_non_rules_python_ignore_root_user_error_non_root_module) - def _test_toolchain_ordering(env): py = parse_modules( module_ctx = _mock_mctx( @@ -510,8 +436,8 @@ def _test_first_occurance_of_the_toolchain_wins(env): env.expect.that_dict(py.debug_info).contains_exactly({ "toolchains_registered": [ - {"ignore_root_user_error": True, "module": {"is_root": True, "name": "my_module"}, "name": "python_3_12"}, - {"ignore_root_user_error": True, "module": {"is_root": False, "name": "rules_python"}, "name": "python_3_11"}, + {"module": {"is_root": True, "name": "my_module"}, "name": "python_3_12"}, + {"module": {"is_root": False, "name": "rules_python"}, "name": "python_3_11"}, ], }) @@ -538,7 +464,6 @@ def _test_auth_overrides(env): env.expect.that_dict(py.config.default).contains_at_least({ "auth_patterns": {"foo": "bar"}, - "ignore_root_user_error": True, "netrc": "/my/netrc", }) env.expect.that_str(py.default_python_version).equals("3.12") From 3b2dd422d2bdaf9dd2f4558cb44490da3696c544 Mon Sep 17 00:00:00 2001 From: Ignas Anikevicius <240938+aignas@users.noreply.github.com> Date: Sun, 23 Nov 2025 12:58:42 +0900 Subject: [PATCH 532/922] ci: switch our jobs to mac arm64 (#3426) The arm64 macs have been in the wild for a while and it makes more sense to test against that platform instead of the Intel macs that very few people in our user-base are using. At the same time clarify the supported platforms. --------- Co-authored-by: Richard Levasseur --- .bazelci/presubmit.yml | 31 +++++++++---------- docs/support.md | 5 +-- .../integration/local_toolchains/MODULE.bazel | 4 +-- 3 files changed, 20 insertions(+), 20 deletions(-) diff --git a/.bazelci/presubmit.yml b/.bazelci/presubmit.yml index 548d1e98f6..e6ee33558a 100644 --- a/.bazelci/presubmit.yml +++ b/.bazelci/presubmit.yml @@ -159,7 +159,7 @@ tasks: <<: *reusable_config <<: *common_workspace_flags name: "Default: Mac, workspace" - platform: macos + platform: macos_arm64 windows_workspace: <<: *reusable_config <<: *common_workspace_flags @@ -190,10 +190,10 @@ tasks: <<: *reusable_config name: "Default: Debian" platform: debian11 - macos: + macos_arm64: <<: *reusable_config name: "Default: MacOS" - platform: macos + platform: macos_arm64 windows: <<: *reusable_config name: "Default: Windows" @@ -254,7 +254,7 @@ tasks: <<: *common_workspace_flags name: "examples/build_file_generation: macOS, workspace" working_directory: examples/build_file_generation - platform: macos + platform: macos_arm64 integration_test_build_file_generation_windows_workspace: <<: *reusable_build_test_all <<: *common_workspace_flags @@ -305,14 +305,14 @@ tasks: <<: *coverage_targets_example_bzlmod name: "examples/bzlmod: macOS" working_directory: examples/bzlmod - platform: macos + platform: macos_arm64 bazel: 7.x integration_test_bzlmod_macos_upcoming: <<: *reusable_build_test_all <<: *coverage_targets_example_bzlmod name: "examples/bzlmod: macOS, upcoming Bazel" working_directory: examples/bzlmod - platform: macos + platform: macos_arm64 bazel: last_rc integration_test_bzlmod_windows: <<: *reusable_build_test_all @@ -362,7 +362,7 @@ tasks: <<: *coverage_targets_example_bzlmod_build_file_generation name: "gazelle/examples/bzlmod_build_file_generation: MacOS" working_directory: gazelle/examples/bzlmod_build_file_generation - platform: macos + platform: macos_arm64 integration_test_bzlmod_build_file_generation_windows: <<: *reusable_build_test_all # coverage is not supported on Windows @@ -390,7 +390,7 @@ tasks: <<: *coverage_targets_example_multi_python name: "examples/multi_python_versions: MacOS, workspace" working_directory: examples/multi_python_versions - platform: macos + platform: macos_arm64 integration_test_multi_python_versions_windows_workspace: <<: *reusable_build_test_all <<: *common_workspace_flags @@ -427,7 +427,7 @@ tasks: <<: *reusable_build_test_all name: "examples/pip_parse: MacOS" working_directory: examples/pip_parse - platform: macos + platform: macos_arm64 integration_test_pip_parse_windows: <<: *reusable_build_test_all name: "examples/pip_parse: Windows" @@ -458,7 +458,7 @@ tasks: <<: *common_workspace_flags name: "examples/pip_parse_vendored: MacOS" working_directory: examples/pip_parse_vendored - platform: macos + platform: macos_arm64 # We don't run pip_parse_vendored under Windows as the file checked in is # generated from a repository rule containing OS-specific rendered paths. @@ -481,7 +481,7 @@ tasks: <<: *common_workspace_flags name: "examples/py_proto_library: MacOS, workspace" working_directory: examples/py_proto_library - platform: macos + platform: macos_arm64 integration_test_py_proto_library_windows_workspace: <<: *reusable_build_test_all <<: *common_workspace_flags @@ -506,7 +506,7 @@ tasks: <<: *common_workspace_flags name: "examples/pip_repository_annotations: macOS, workspace" working_directory: examples/pip_repository_annotations - platform: macos + platform: macos_arm64 integration_test_pip_repository_annotations_windows_workspace: <<: *reusable_build_test_all <<: *common_workspace_flags @@ -528,7 +528,7 @@ tasks: integration_test_bazelinbazel_macos: <<: *common_bazelinbazel_config name: "tests/integration bazel-in-bazel: macOS (subset)" - platform: macos + platform: macos_arm64 build_targets: ["//tests/integration:local_toolchains_test_bazel_self"] test_targets: ["//tests/integration:local_toolchains_test_bazel_self"] # The bazelinbazel tests were disabled on Windows to save CI jobs slots, and @@ -581,7 +581,7 @@ tasks: <<: *reusable_build_test_all name: "compile_pip_requirements: MacOS" working_directory: tests/integration/compile_pip_requirements - platform: macos + platform: macos_arm64 shell_commands: # Make a change to the locked requirements and then assert that //:requirements.update does the # right thing. @@ -614,7 +614,6 @@ tasks: - "bazel run //:os_specific_requirements.update" - "git diff --exit-code" - integration_compile_pip_requirements_test_from_external_repo_ubuntu_min_workspace: <<: *minimum_supported_version <<: *common_workspace_flags_min_bazel @@ -650,7 +649,7 @@ tasks: integration_compile_pip_requirements_test_from_external_repo_macos: name: "compile_pip_requirements_test_from_external_repo: macOS" working_directory: tests/integration/compile_pip_requirements_test_from_external_repo - platform: macos + platform: macos_arm64 shell_commands: # Assert that @compile_pip_requirements//:requirements_test does the right thing. - "bazel test @compile_pip_requirements//..." diff --git a/docs/support.md b/docs/support.md index 8728540804..08147f2a15 100644 --- a/docs/support.md +++ b/docs/support.md @@ -61,8 +61,9 @@ are Linux, Mac, and Windows. In order to better describe different support levels, the following acts as a rough guideline for different platform tiers: -* Tier 0 - The platforms that our CI runs on: `linux_x86_64`, `osx_x86_64`, `RBE linux_x86_64`. -* Tier 1 - The platforms that are similar enough to what the CI runs on: `linux_aarch64`, `osx_arm64`. +* Tier 0 - The platforms that our CI runs on: `linux_x86_64`, `osx_arm64`, `RBE linux_x86_64`. +* Tier 1 - The platforms that are similar enough to what the CI runs on: `linux_aarch64`, + `osx_x86_64`. What is more, `windows_x86_64` is in this list, as we run tests in CI, but developing for Windows is more challenging, and features may come later to this platform. diff --git a/tests/integration/local_toolchains/MODULE.bazel b/tests/integration/local_toolchains/MODULE.bazel index 6c821c5bb0..c818942748 100644 --- a/tests/integration/local_toolchains/MODULE.bazel +++ b/tests/integration/local_toolchains/MODULE.bazel @@ -41,12 +41,12 @@ pbs_archive( name = "pbs_runtime", sha256 = { "linux": "0a01bad99fd4a165a11335c29eb43015dfdb8bd5ba8e305538ebb54f3bf3146d", - "mac os x": "4fb42ffc8aad2a42ca7646715b8926bc6b2e0d31f13d2fec25943dc236a6fd60", + "mac os x": "7f5ec658219bdb1d1142c6abab89680322166c78350a017fb0af3c869dceee41", "windows": "005cb2abf4cfa4aaa48fb10ce4e33fe4335ea4d1f55202dbe4e20c852e45e0f9", }, urls = { "linux": "https://github.com/astral-sh/python-build-standalone/releases/download/20250918/cpython-3.13.7+20250918-x86_64-unknown-linux-gnu-install_only.tar.gz", - "mac os x": "https://github.com/astral-sh/python-build-standalone/releases/download/20250918/cpython-3.13.7+20250918-x86_64-apple-darwin-install_only.tar.gz", + "mac os x": "https://github.com/astral-sh/python-build-standalone/releases/download/20250918/cpython-3.13.7+20250918-aarch64-apple-darwin-install_only.tar.gz", "windows server 2022": "https://github.com/astral-sh/python-build-standalone/releases/download/20250918/cpython-3.13.7+20250918-x86_64-pc-windows-msvc-install_only.tar.gz", }, ) From 5bc7ba363a81da9c6cb03058bc3741b3ec84a5d3 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Sun, 23 Nov 2025 04:15:05 -0800 Subject: [PATCH 533/922] docs: enable pipstar for doc building (#3427) Since we want to enable pipstar by default, enable it for our docs builds as a way to try it out consistently. --- docs/readthedocs_build.sh | 3 +++ 1 file changed, 3 insertions(+) diff --git a/docs/readthedocs_build.sh b/docs/readthedocs_build.sh index ec5390bfc7..06ac7698d2 100755 --- a/docs/readthedocs_build.sh +++ b/docs/readthedocs_build.sh @@ -12,7 +12,10 @@ done < <(env -0) # In order to get the build number, we extract it from the host name extra_env+=("--//sphinxdocs:extra_env=HOSTNAME=$HOSTNAME") +export RULES_PYTHON_ENABLE_PIPSTAR=1 + set -x +export RULES_PYTHON_ENABLE_PIPSTAR=1 bazel run \ --config=rtd \ "--//sphinxdocs:extra_defines=version=$READTHEDOCS_VERSION" \ From 1b85ec8f89fb8968349cbc1c6354e88d5bb493fb Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Sun, 23 Nov 2025 07:51:19 -0800 Subject: [PATCH 534/922] fix: add runfiles root for system_python bootstrap (#3423) When the system_bootstrap code was changed to using the site init for adding to sys.path, importing `bazel_tools.tools.python.runfiles` stopped working. This is because the runfiles root was no longer being added to sys.path. This is somewhat WAI because: 1. Always adding the runfiles root to sys.path is a deprecated legacy behavior because it can interfere with Python imports (a repo name can mask a legitimate import). 2. Under bzlmod, repo directory names aren't importable Python names, so having the runfiles root on sys.path doesn't do much. An exception to (2) is bazel_tools: this is special cased to use the directory name `bazel_tools` in runfiles. This is where the legacy runfiles library for Python is. In any case, forgetting the runfiles root on sys.path for the system_python bootstrap was an oversight, as the intention was to be a more no-op refactoring. Fixes https://github.com/bazel-contrib/rules_python/issues/3422 --------- Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> --- CHANGELOG.md | 3 ++- python/private/py_executable.bzl | 6 +++++- python/private/site_init_template.py | 18 ++++++++++++----- tests/bootstrap_impls/BUILD.bazel | 15 ++++++++++++++ .../bazel_tools_importable_test.py | 20 +++++++++++++++++++ 5 files changed, 55 insertions(+), 7 deletions(-) create mode 100644 tests/bootstrap_impls/bazel_tools_importable_test.py diff --git a/CHANGELOG.md b/CHANGELOG.md index edcab7a357..94daf07fe8 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -91,7 +91,8 @@ END_UNRELEASED_TEMPLATE * (performance) 90% reduction in py_binary/py_test analysis phase cost. ([#3381](https://github.com/bazel-contrib/rules_python/pull/3381)). * (gazelle) Fix `gazelle_python_manifest.test` so that it accesses manifest files via `runfile` path handling rather than directly ([#3397](https://github.com/bazel-contrib/rules_python/issues/3397)). - +* (core rules) For the system_python bootstrap, the runfiles root is added to + sys.path. {#v0-0-0-added} ### Added diff --git a/python/private/py_executable.bzl b/python/private/py_executable.bzl index 99a3dffb49..0a16e7690e 100644 --- a/python/private/py_executable.bzl +++ b/python/private/py_executable.bzl @@ -345,6 +345,9 @@ def _create_executable( output_prefix = base_executable_name, imports = imports, runtime_details = runtime_details, + add_runfiles_root_to_sys_path = ( + "1" if BootstrapImplFlag.get_value(ctx) == BootstrapImplFlag.SYSTEM_PYTHON else "0" + ), ) stage2_bootstrap = _create_stage2_bootstrap( @@ -508,7 +511,7 @@ def _create_zip_main(ctx, *, stage2_bootstrap, runtime_details, venv): # * https://snarky.ca/how-virtual-environments-work/ # * https://github.com/python/cpython/blob/main/Modules/getpath.py # * https://github.com/python/cpython/blob/main/Lib/site.py -def _create_venv(ctx, output_prefix, imports, runtime_details): +def _create_venv(ctx, output_prefix, imports, runtime_details, add_runfiles_root_to_sys_path): create_full_venv = BootstrapImplFlag.get_value(ctx) == BootstrapImplFlag.SCRIPT venv = "_{}.venv".format(output_prefix.lstrip("_")) @@ -596,6 +599,7 @@ def _create_venv(ctx, output_prefix, imports, runtime_details): template = runtime.site_init_template, output = site_init, substitutions = { + "%add_runfiles_root_to_sys_path%": add_runfiles_root_to_sys_path, "%coverage_tool%": _get_coverage_tool_runfiles_path(ctx, runtime), "%import_all%": "True" if read_possibly_native_flag(ctx, "python_import_all_repositories") else "False", "%site_init_runfiles_path%": "{}/{}".format(ctx.workspace_name, site_init.short_path), diff --git a/python/private/site_init_template.py b/python/private/site_init_template.py index a87a0d2a8f..97d16b71c2 100644 --- a/python/private/site_init_template.py +++ b/python/private/site_init_template.py @@ -26,6 +26,8 @@ _SELF_RUNFILES_RELATIVE_PATH = "%site_init_runfiles_path%" # Runfiles-relative path to the coverage tool entry point, if any. _COVERAGE_TOOL = "%coverage_tool%" +# True if the runfiles root should be added to sys.path +_ADD_RUNFILES_ROOT_TO_SYS_PATH = "%add_runfiles_root_to_sys_path%" == "1" def _is_verbose(): @@ -127,11 +129,6 @@ def _search_path(name): def _setup_sys_path(): """Perform Bazel/binary specific sys.path setup. - NOTE: We do not add _RUNFILES_ROOT to sys.path for two reasons: - 1. Under workspace, it makes every external repository importable. If a Bazel - repository matches a Python import name, they conflict. - 2. Under bzlmod, the repo names in the runfiles directory aren't importable - Python names, so there's no point in adding the runfiles root to sys.path. """ seen = set(sys.path) python_path_entries = [] @@ -147,6 +144,17 @@ def _maybe_add_path(path): sys.path.append(path) seen.add(path) + # Adding the runfiles root to sys.path is a legacy behavior that will be + # removed. We don't want to add it to sys.path for two reasons: + # 1. Under workspace, it makes every external repository importable. If a Bazel + # repository matches a Python import name, they conflict. + # 2. Under bzlmod, the repo names in the runfiles directory aren't importable + # Python names, so there's no point in adding the runfiles root to sys.path. + # For temporary compatibility with the original system_python bootstrap + # behavior, it is conditionally added for that boostrap mode. + if _ADD_RUNFILES_ROOT_TO_SYS_PATH: + _maybe_add_path(_RUNFILES_ROOT) + for rel_path in _IMPORTS_STR.split(":"): abs_path = os.path.join(_RUNFILES_ROOT, rel_path) _maybe_add_path(abs_path) diff --git a/tests/bootstrap_impls/BUILD.bazel b/tests/bootstrap_impls/BUILD.bazel index dcc27514f7..5f7e5afd95 100644 --- a/tests/bootstrap_impls/BUILD.bazel +++ b/tests/bootstrap_impls/BUILD.bazel @@ -104,6 +104,18 @@ sh_py_run_test( target_compatible_with = SUPPORTS_BOOTSTRAP_SCRIPT, ) +py_reconfig_test( + name = "bazel_tools_importable_system_python_test", + srcs = ["bazel_tools_importable_test.py"], + bootstrap_impl = "system_python", + # Necessary because bazel_tools doesn't have __init__.py files. + legacy_create_init = True, + main = "bazel_tools_importable_test.py", + deps = [ + "@bazel_tools//tools/python/runfiles", + ], +) + py_reconfig_test( name = "sys_path_order_bootstrap_script_test", srcs = ["sys_path_order_test.py"], @@ -121,6 +133,9 @@ py_reconfig_test( env = {"BOOTSTRAP": "system_python"}, imports = ["./site-packages"], main = "sys_path_order_test.py", + deps = [ + "@bazel_tools//tools/python/runfiles", + ], ) py_reconfig_test( diff --git a/tests/bootstrap_impls/bazel_tools_importable_test.py b/tests/bootstrap_impls/bazel_tools_importable_test.py new file mode 100644 index 0000000000..ad753bc03d --- /dev/null +++ b/tests/bootstrap_impls/bazel_tools_importable_test.py @@ -0,0 +1,20 @@ +import sys +import unittest + + +class BazelToolsImportableTest(unittest.TestCase): + def test_bazel_tools_importable(self): + try: + import bazel_tools + import bazel_tools.tools.python + import bazel_tools.tools.python.runfiles + except ImportError as exc: + raise AssertionError( + "Failed to import bazel_tools.python.runfiles\n" + + "sys.path:\n" + + "\n".join(f"{i}: {v}" for i, v in enumerate(sys.path)) + ) from exc + + +if __name__ == "__main__": + unittest.main() From 235cc00cda5212af1d8349b0e9e269ac5bdc501d Mon Sep 17 00:00:00 2001 From: Ignas Anikevicius <240938+aignas@users.noreply.github.com> Date: Mon, 24 Nov 2025 03:27:20 +0900 Subject: [PATCH 535/922] refactor(pypi): parse entry_points without Python (#3429) This is a small utility function to get us Python free when wheels are extracted in the repository phase. Next is to extract the wheel using `repository_ctx.extract` (#3430). Whereas patching the wheel after extracting is more involved to be done without Python because we need to rezip the wheel and that has to be done with Python for a few reasons (to stay sane). If we want to remove this, then we would have to create a `whl` file in the build phase, which could work, but will need to be an exercise for the reader. Nevertheless, this moves us towards removing any side-effects from Python interpreter, so changing the default interpreter would not cause us to refetch everything. --------- Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> --- .../pypi/whl_installer/wheel_installer.py | 22 ++++--- python/private/pypi/whl_library.bzl | 21 +++---- python/private/pypi/whl_metadata.bzl | 59 ++++++++++++++++++- .../pypi/whl_metadata/whl_metadata_tests.bzl | 49 +++++++++++++++ 4 files changed, 125 insertions(+), 26 deletions(-) diff --git a/python/private/pypi/whl_installer/wheel_installer.py b/python/private/pypi/whl_installer/wheel_installer.py index a6a9dd0429..aae24ff4c7 100644 --- a/python/private/pypi/whl_installer/wheel_installer.py +++ b/python/private/pypi/whl_installer/wheel_installer.py @@ -96,7 +96,17 @@ def _extract_wheel( whl = wheel.Wheel(wheel_file) whl.unzip(installation_dir) + if enable_pipstar: + return + + extras_requested = extras[whl.name] if whl.name in extras else set() + dependencies = whl.dependencies(extras_requested, platforms) + metadata = { + "name": whl.name, + "version": whl.version, + "deps": dependencies.deps, + "deps_by_platform": dependencies.deps_select, "entry_points": [ { "name": name, @@ -106,18 +116,6 @@ def _extract_wheel( for name, (module, attribute) in sorted(whl.entry_points().items()) ], } - if not enable_pipstar: - extras_requested = extras[whl.name] if whl.name in extras else set() - dependencies = whl.dependencies(extras_requested, platforms) - - metadata.update( - { - "name": whl.name, - "version": whl.version, - "deps": dependencies.deps, - "deps_by_platform": dependencies.deps_select, - } - ) with open(os.path.join(installation_dir, "metadata.json"), "w") as f: json.dump(metadata, f) diff --git a/python/private/pypi/whl_library.bzl b/python/private/pypi/whl_library.bzl index fe3308ad3c..65888a4b4f 100644 --- a/python/private/pypi/whl_library.bzl +++ b/python/private/pypi/whl_library.bzl @@ -391,18 +391,21 @@ def _whl_library_impl(rctx): logger = logger, ) - metadata = json.decode(rctx.read("metadata.json")) - rctx.delete("metadata.json") + metadata = whl_metadata( + install_dir = whl_path.dirname.get_child("site-packages"), + read_fn = rctx.read, + logger = logger, + ) # NOTE @aignas 2024-06-22: this has to live on until we stop supporting # passing `twine` as a `:pkg` library via the `WORKSPACE` builds. # # See ../../packaging.bzl line 190 entry_points = {} - for item in metadata["entry_points"]: - name = item["name"] - module = item["module"] - attribute = item["attribute"] + for item in metadata.entry_points: + name = item.name + module = item.module + attribute = item.attribute # There is an extreme edge-case with entry_points that end with `.py` # See: https://github.com/bazelbuild/bazel/blob/09c621e4cf5b968f4c6cdf905ab142d5961f9ddc/src/test/java/com/google/devtools/build/lib/rules/python/PyBinaryConfiguredTargetTest.java#L174 @@ -418,12 +421,6 @@ def _whl_library_impl(rctx): ) entry_points[entry_point_without_py] = entry_point_script_name - metadata = whl_metadata( - install_dir = whl_path.dirname.get_child("site-packages"), - read_fn = rctx.read, - logger = logger, - ) - build_file_contents = generate_whl_library_build_bazel( name = whl_path.basename, sdist_filename = sdist_filename, diff --git a/python/private/pypi/whl_metadata.bzl b/python/private/pypi/whl_metadata.bzl index a56aac5782..0d3a14ab54 100644 --- a/python/private/pypi/whl_metadata.bzl +++ b/python/private/pypi/whl_metadata.bzl @@ -4,6 +4,7 @@ _NAME = "Name: " _PROVIDES_EXTRA = "Provides-Extra: " _REQUIRES_DIST = "Requires-Dist: " _VERSION = "Version: " +_CONSOLE_SCRIPTS = "[console_scripts]" def whl_metadata(*, install_dir, read_fn, logger): """Find and parse the METADATA file in the extracted whl contents dir. @@ -23,7 +24,13 @@ def whl_metadata(*, install_dir, read_fn, logger): """ metadata_file = find_whl_metadata(install_dir = install_dir, logger = logger) contents = read_fn(metadata_file) - result = parse_whl_metadata(contents) + entry_points_file = metadata_file.dirname.get_child("entry_points.txt") + if entry_points_file.exists: + entry_points_contents = read_fn(entry_points_file) + else: + entry_points_contents = "" + + result = parse_whl_metadata(contents, entry_points_contents) if not (result.name and result.version): logger.fail("Failed to parse the wheel METADATA file:\n{}\n{}\n{}".format( @@ -35,11 +42,12 @@ def whl_metadata(*, install_dir, read_fn, logger): return result -def parse_whl_metadata(contents): +def parse_whl_metadata(contents, entry_points_contents = ""): """Parse .whl METADATA file Args: contents: {type}`str` the contents of the file. + entry_points_contents: {type}`str` the contents of the `entry_points.txt` file if it exists. Returns: A struct with parsed values: @@ -48,6 +56,8 @@ def parse_whl_metadata(contents): * `requires_dist`: {type}`list[str]` the list of requirements. * `provides_extra`: {type}`list[str]` the list of extras that this package provides. + * `entry_points`: {type}`list[struct]` the list of + entry_point metadata. """ parsed = { "name": "", @@ -79,6 +89,7 @@ def parse_whl_metadata(contents): provides_extra = parsed["provides_extra"], requires_dist = parsed["requires_dist"], version = parsed["version"], + entry_points = _parse_entry_points(entry_points_contents), ) def find_whl_metadata(*, install_dir, logger): @@ -110,3 +121,47 @@ def find_whl_metadata(*, install_dir, logger): else: logger.fail("The '*.dist-info' directory could not be found in '{}'".format(install_dir.basename)) return None + +def _parse_entry_points(contents): + """parse the entry_points.txt file. + + Args: + contents: {type}`str` The contents of the file + + Returns: + A list of console_script entry point metadata. + """ + start = False + ret = [] + for line in contents.split("\n"): + line = line.rstrip() + + if line == _CONSOLE_SCRIPTS: + start = True + continue + + if not start: + continue + + if start and line.startswith("["): + break + + line, _, _comment = line.partition("#") + line = line.strip() + if not line: + continue + + name, _, tail = line.partition("=") + + # importable.module:object.attr + py_import, _, extras = tail.strip().partition(" ") + module, _, attribute = py_import.partition(":") + + ret.append(struct( + name = name.strip(), + module = module.strip(), + attribute = attribute.strip(), + extras = extras.replace(" ", ""), + )) + + return ret diff --git a/tests/pypi/whl_metadata/whl_metadata_tests.bzl b/tests/pypi/whl_metadata/whl_metadata_tests.bzl index 329423a26c..1d78611901 100644 --- a/tests/pypi/whl_metadata/whl_metadata_tests.bzl +++ b/tests/pypi/whl_metadata/whl_metadata_tests.bzl @@ -81,12 +81,14 @@ def _parse_whl_metadata(env, **kwargs): version = result.version, requires_dist = result.requires_dist, provides_extra = result.provides_extra, + entry_points = result.entry_points, ), attrs = dict( name = subjects.str, version = subjects.str, requires_dist = subjects.collection, provides_extra = subjects.collection, + entry_points = subjects.collection, ), ) @@ -171,6 +173,53 @@ Requires-Dist: this will be ignored _tests.append(_test_parse_metadata_multiline_license) +def _test_parse_entry_points_txt(env): + got = _parse_whl_metadata( + env, + contents = """\ +Name: foo +Version: 0.0.1 +""", + entry_points_contents = """\ +[something] +interesting # with comments + +[console_scripts] +foo = foomod:main +# One which depends on extras: +foobar = importable.foomod:main_bar [bar, baz] + + # With a comment at the end +foobarbaz = foomod:main.attr # comment + +[something else] +not very much interesting + +""", + ) + got.entry_points().contains_exactly([ + struct( + attribute = "main", + extras = "", + module = "foomod", + name = "foo", + ), + struct( + attribute = "main_bar", + extras = "[bar,baz]", + module = "importable.foomod", + name = "foobar", + ), + struct( + attribute = "main.attr", + extras = "", + module = "foomod", + name = "foobarbaz", + ), + ]) + +_tests.append(_test_parse_entry_points_txt) + def whl_metadata_test_suite(name): # buildifier: disable=function-docstring test_suite( name = name, From a9056b1ff23063f4605d77acc3aa362cbd6103d1 Mon Sep 17 00:00:00 2001 From: Ignas Anikevicius <240938+aignas@users.noreply.github.com> Date: Mon, 24 Nov 2025 04:04:42 +0900 Subject: [PATCH 536/922] chore: enable pipstar for experimental_index_url users (#3428) `bazel query` may break if users right now rely on the behaviour of: > only host platform packages are present in the whl_library instances When we enabled `pipstar` by default, one of our user reported this breakage because: 1. The `triton` dependency is only used on `linux` and it is in one of the `requirements` files. 2. On mac, this dependency did not have a `whl_library` repository materialized and the bazel query failed because of this reason. Hence we enable `pipstar` in a different way - we only enable it when whl files are downloaded via the bazel downloader and `bazel query` is less likely to fail if you can use the bazel downloader for all of your deps. Work towards #2949 --------- Co-authored-by: Richard Levasseur --- CHANGELOG.md | 3 +++ python/private/pypi/hub_builder.bzl | 23 ++++++++++++++--------- python/private/pypi/whl_library.bzl | 4 +++- 3 files changed, 20 insertions(+), 10 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 94daf07fe8..4e9514352d 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -72,6 +72,9 @@ END_UNRELEASED_TEMPLATE ### Changed * (toolchains) Use toolchains from the [20251031] release. * (gazelle) Internally split modules mapping generation to be per-wheel for concurrency and caching. +* (pip) `pipstar` has been enabled for all `whl_library` instances where the whl + is passed through a label or downloaded using the bazel downloader + ([#2949](https://github.com/bazel-contrib/rules_python/issues/2949)). {#v0-0-0-fixed} ### Fixed diff --git a/python/private/pypi/hub_builder.bzl b/python/private/pypi/hub_builder.bzl index bd6008128b..69359bcf8a 100644 --- a/python/private/pypi/hub_builder.bzl +++ b/python/private/pypi/hub_builder.bzl @@ -146,6 +146,7 @@ def _pip_parse(self, module_ctx, pip_attr): self, module_ctx, pip_attr = pip_attr, + enable_pipstar = self._config.enable_pipstar or self._get_index_urls.get(pip_attr.python_version), ) ### end of PUBLIC methods @@ -188,7 +189,7 @@ def _add_extra_aliases(self, extra_hub_aliases): {alias: True for alias in aliases}, ) -def _add_whl_library(self, *, python_version, whl, repo): +def _add_whl_library(self, *, python_version, whl, repo, enable_pipstar): if repo == None: # NOTE @aignas 2025-07-07: we guard against an edge-case where there # are more platforms defined than there are wheels for and users @@ -208,7 +209,7 @@ def _add_whl_library(self, *, python_version, whl, repo): )) self._whl_libraries[repo_name] = repo.args - if not self._config.enable_pipstar and "experimental_target_platforms" in repo.args: + if not enable_pipstar and "experimental_target_platforms" in repo.args: self._whl_libraries[repo_name] |= { "experimental_target_platforms": sorted({ # TODO @aignas 2025-07-07: this should be solved in a better way @@ -342,11 +343,11 @@ def _platforms(*, python_version, config): ) return platforms -def _evaluate_markers(self, pip_attr): +def _evaluate_markers(self, pip_attr, enable_pipstar): if self._evaluate_markers_fn: return self._evaluate_markers_fn - if self._config.enable_pipstar: + if enable_pipstar: return lambda _, requirements: evaluate_markers_star( requirements = requirements, platforms = self._platforms[pip_attr.python_version], @@ -387,13 +388,15 @@ def _create_whl_repos( self, module_ctx, *, - pip_attr): + pip_attr, + enable_pipstar = False): """create all of the whl repositories Args: self: the builder. module_ctx: {type}`module_ctx`. pip_attr: {type}`struct` - the struct that comes from the tag class iteration. + enable_pipstar: {type}`bool` - enable the pipstar or not. """ logger = self._logger platforms = self._platforms[pip_attr.python_version] @@ -416,7 +419,7 @@ def _create_whl_repos( platforms = platforms, extra_pip_args = pip_attr.extra_pip_args, get_index_urls = self._get_index_urls.get(pip_attr.python_version), - evaluate_markers = _evaluate_markers(self, pip_attr), + evaluate_markers = _evaluate_markers(self, pip_attr, enable_pipstar), logger = logger, ) @@ -435,6 +438,7 @@ def _create_whl_repos( self, module_ctx, pip_attr = pip_attr, + enable_pipstar = enable_pipstar, ) for whl in requirements_by_platform: whl_library_args = common_args | _whl_library_args( @@ -452,16 +456,17 @@ def _create_whl_repos( auth_patterns = self._config.auth_patterns or pip_attr.auth_patterns, python_version = _major_minor_version(pip_attr.python_version), is_multiple_versions = whl.is_multiple_versions, - enable_pipstar = self._config.enable_pipstar, + enable_pipstar = enable_pipstar, ) _add_whl_library( self, python_version = pip_attr.python_version, whl = whl, repo = repo, + enable_pipstar = enable_pipstar, ) -def _common_args(self, module_ctx, *, pip_attr): +def _common_args(self, module_ctx, *, pip_attr, enable_pipstar): interpreter = _detect_interpreter(self, pip_attr) # Construct args separately so that the lock file can be smaller and does not include unused @@ -481,7 +486,7 @@ def _common_args(self, module_ctx, *, pip_attr): python_interpreter = interpreter.path, python_interpreter_target = interpreter.target, ) - if not self._config.enable_pipstar: + if not enable_pipstar: maybe_args["experimental_target_platforms"] = pip_attr.experimental_target_platforms whl_library_args.update({k: v for k, v in maybe_args.items() if v}) diff --git a/python/private/pypi/whl_library.bzl b/python/private/pypi/whl_library.bzl index 65888a4b4f..5db7bc49a1 100644 --- a/python/private/pypi/whl_library.bzl +++ b/python/private/pypi/whl_library.bzl @@ -324,6 +324,8 @@ def _whl_library_impl(rctx): args = _parse_optional_attrs(rctx, args, extra_pip_args) + # also enable pipstar for any whls that are downloaded without `pip` + enable_pipstar = (rp_config.enable_pipstar or whl_path) and rctx.attr.config_load if not whl_path: if rctx.attr.urls: op_tmpl = "whl_library.BuildWheelFromSource({name}, {requirement})" @@ -374,7 +376,7 @@ def _whl_library_impl(rctx): # disable pipstar for that particular case. # # Remove non-pipstar and config_load check when we release rules_python 2. - if rp_config.enable_pipstar and rctx.attr.config_load: + if enable_pipstar: pypi_repo_utils.execute_checked( rctx, op = "whl_library.ExtractWheel({}, {})".format(rctx.attr.name, whl_path), From 45821b8603cd8b1f0791322de1c223bf5713c24d Mon Sep 17 00:00:00 2001 From: Ignas Anikevicius <240938+aignas@users.noreply.github.com> Date: Tue, 25 Nov 2025 08:51:39 +0900 Subject: [PATCH 537/922] fix(pip): do not add a pip-fallback when there is no sdist (#3432) Before this PR the user would have to specify `download = True` in order to not fallback to pip, which is admittedly an odd interface design. With this PR we correctly do not add a `pip` fallback if there is no `sdist` to be used. With this in place we are better placed to enable the `experimental_index_url` by default. What is more we can more reliably detect when we should use a special repository rule to prepare for building from sdist. Summary: - Add a test that shows the problem and then adjust the torch experimental URL test. - Cleanup unused marker stubs in the tests. - Fix the code to better handle the case when there is no sdist. Work towards #260 Work towards #2410 --- python/private/pypi/parse_requirements.bzl | 66 ++++++++++++++------ tests/pypi/hub_builder/hub_builder_tests.bzl | 59 +++++++++++------ 2 files changed, 88 insertions(+), 37 deletions(-) diff --git a/python/private/pypi/parse_requirements.bzl b/python/private/pypi/parse_requirements.bzl index 7d210abbaa..5c05c753fd 100644 --- a/python/private/pypi/parse_requirements.bzl +++ b/python/private/pypi/parse_requirements.bzl @@ -188,19 +188,35 @@ def parse_requirements( for p in r.target_platforms: requirement_target_platforms[p] = None + package_srcs = _package_srcs( + name = name, + reqs = reqs, + index_urls = index_urls, + platforms = platforms, + extract_url_srcs = extract_url_srcs, + logger = logger, + ) + + # FIXME @aignas 2025-11-24: we can get the list of target platforms here + # + # However it is likely that we may stop exposing packages like torch in here + # which do not have wheels for all osx platforms. + # + # If users specify the target platforms accurately, then it is a different + # (better) story, but we may not be able to guarantee this + # + # target_platforms = [ + # p + # for dist in package_srcs + # for p in dist.target_platforms + # ] + item = struct( # Return normalized names name = normalize_name(name), is_exposed = len(requirement_target_platforms) == len(requirements), is_multiple_versions = len(reqs.values()) > 1, - srcs = _package_srcs( - name = name, - reqs = reqs, - index_urls = index_urls, - platforms = platforms, - extract_url_srcs = extract_url_srcs, - logger = logger, - ), + srcs = package_srcs, ) ret.append(item) if not item.is_exposed and logger: @@ -234,7 +250,7 @@ def _package_srcs( platforms.keys(), )) - dist = _add_dists( + dist, can_fallback = _add_dists( requirement = r, target_platform = platforms.get(target_platform), index_urls = index_urls.get(name), @@ -244,7 +260,7 @@ def _package_srcs( if extract_url_srcs and dist: req_line = r.srcs.requirement - else: + elif can_fallback: dist = struct( url = "", filename = "", @@ -252,6 +268,8 @@ def _package_srcs( yanked = False, ) req_line = r.srcs.requirement_line + else: + continue key = ( dist.filename, @@ -337,6 +355,14 @@ def _add_dists(*, requirement, index_urls, target_platform, logger = None): index_urls: The result of simpleapi_download. target_platform: The target_platform information. logger: A logger for printing diagnostic info. + + Returns: + (dist, can_fallback_to_pip): a struct with distribution details and how to fetch + it and a boolean flag to tell the other layers if we should add an entry to + fallback for pip if there are no supported whls found - if there is an sdist, we + can attempt the fallback, otherwise better to not, because the pip command will + fail and the error message will be confusing. What is more that would lead to + breakage of the bazel query. """ if requirement.srcs.url: @@ -344,7 +370,7 @@ def _add_dists(*, requirement, index_urls, target_platform, logger = None): logger.debug(lambda: "Could not detect the filename from the URL, falling back to pip: {}".format( requirement.srcs.url, )) - return None + return None, True # Handle direct URLs in requirements dist = struct( @@ -354,13 +380,10 @@ def _add_dists(*, requirement, index_urls, target_platform, logger = None): yanked = False, ) - if dist.filename.endswith(".whl"): - return dist - else: - return dist + return dist, False if not index_urls: - return None + return None, True whls = [] sdist = None @@ -403,7 +426,14 @@ def _add_dists(*, requirement, index_urls, target_platform, logger = None): if not target_platform: # The pipstar platforms are undefined here, so we cannot do any matching - return sdist + return sdist, True + + if not whls and not sdist: + # If there are no suitable wheels to handle for now allow fallback to pip, it + # may be a little bit more helpful when debugging? Most likely something is + # going a bit wrong here, should we raise an error because the sha256 have most + # likely mismatched? We are already printing a warning above. + return None, True # Select a single wheel that can work on the target_platform return select_whl( @@ -413,4 +443,4 @@ def _add_dists(*, requirement, index_urls, target_platform, logger = None): whl_abi_tags = target_platform.whl_abi_tags, whl_platform_tags = target_platform.whl_platform_tags, logger = logger, - ) or sdist + ) or sdist, sdist != None diff --git a/tests/pypi/hub_builder/hub_builder_tests.bzl b/tests/pypi/hub_builder/hub_builder_tests.bzl index 6d061f4d56..a0ab919d68 100644 --- a/tests/pypi/hub_builder/hub_builder_tests.bzl +++ b/tests/pypi/hub_builder/hub_builder_tests.bzl @@ -566,6 +566,9 @@ def _test_torch_experimental_index_url(env): for (os, cpu), whl_platform_tags in { ("linux", "x86_64"): ["linux_x86_64", "manylinux_*_x86_64"], ("linux", "aarch64"): ["linux_aarch64", "manylinux_*_aarch64"], + # this should be ignored as well because there is no sdist and no whls + # for intel Macs + ("osx", "x86_64"): ["macosx_*_x86_64"], ("osx", "aarch64"): ["macosx_*_arm64"], ("windows", "x86_64"): ["win_amd64"], ("windows", "aarch64"): ["win_arm64"], # this should be ignored @@ -576,15 +579,6 @@ def _test_torch_experimental_index_url(env): "python_3_12_host": "unit_test_interpreter_target", }, minor_mapping = {"3.12": "3.12.19"}, - evaluate_markers_fn = lambda _, requirements, **__: { - # todo once 2692 is merged, this is going to be easier to test. - key: [ - platform - for platform in platforms - if ("x86_64" in platform and "platform_machine ==" in key) or ("x86_64" not in platform and "platform_machine !=" in key) - ] - for key, platforms in requirements.items() - }, simpleapi_download_fn = mocksimpleapi_download, ) builder.pip_parse( @@ -621,7 +615,6 @@ torch==2.4.1+cpu ; platform_machine == 'x86_64' \ _parse( hub_name = "pypi", python_version = "3.12", - download_only = True, experimental_index_url = "https://torch.index", requirements_lock = "universal.txt", ), @@ -800,6 +793,20 @@ def _test_simple_get_index(env): got_simpleapi_download_args.extend(args) got_simpleapi_download_kwargs.update(kwargs) return { + "plat_pkg": struct( + whls = { + "deadb44f": struct( + yanked = False, + filename = "plat-pkg-0.0.4-py3-none-linux_x86_64.whl", + sha256 = "deadb44f", + url = "example2.org/index/plat_pkg/", + ), + }, + sdists = {}, + sha256s_by_version = { + "0.0.4": ["deadb44f"], + }, + ), "simple": struct( whls = { "deadb00f": struct( @@ -850,6 +857,7 @@ some_pkg==0.0.1 @ example-direct.org/some_pkg-0.0.1-py3-none-any.whl \ --hash=sha256:deadbaaf direct_without_sha==0.0.1 @ example-direct.org/direct_without_sha-0.0.1-py3-none-any.whl some_other_pkg==0.0.1 +plat_pkg==0.0.4 pip_fallback==0.0.1 direct_sdist_without_sha @ some-archive/any-name.tar.gz git_dep @ git+https://git.server/repo/project@deadbeefdeadbeef @@ -873,6 +881,7 @@ git_dep @ git+https://git.server/repo/project@deadbeefdeadbeef "direct_without_sha", "git_dep", "pip_fallback", + "plat_pkg", "simple", "some_other_pkg", "some_pkg", @@ -921,6 +930,17 @@ git_dep @ git+https://git.server/repo/project@deadbeefdeadbeef ), ], }, + "plat_pkg": { + "pypi_315_plat_py3_none_linux_x86_64_deadb44f": [ + whl_config_setting( + target_platforms = [ + "cp315_linux_x86_64", + "cp315_linux_x86_64_freethreaded", + ], + version = "3.15", + ), + ], + }, "simple": { "pypi_315_simple_py3_none_any_deadb00f": [ whl_config_setting( @@ -998,6 +1018,15 @@ git_dep @ git+https://git.server/repo/project@deadbeefdeadbeef "python_interpreter_target": "unit_test_interpreter_target", "requirement": "pip_fallback==0.0.1", }, + "pypi_315_plat_py3_none_linux_x86_64_deadb44f": { + "config_load": "@pypi//:config.bzl", + "dep_template": "@pypi//{name}:{target}", + "filename": "plat-pkg-0.0.4-py3-none-linux_x86_64.whl", + "python_interpreter_target": "unit_test_interpreter_target", + "requirement": "plat_pkg==0.0.4", + "sha256": "deadb44f", + "urls": ["example2.org/index/plat_pkg/"], + }, "pypi_315_simple_py3_none_any_deadb00f": { "config_load": "@pypi//:config.bzl", "dep_template": "@pypi//{name}:{target}", @@ -1036,7 +1065,7 @@ git_dep @ git+https://git.server/repo/project@deadbeefdeadbeef index_url = "pypi.org", index_url_overrides = {}, netrc = None, - sources = ["simple", "pip_fallback", "some_other_pkg"], + sources = ["simple", "plat_pkg", "pip_fallback", "some_other_pkg"], ), "cache": {}, "parallel_download": False, @@ -1048,14 +1077,6 @@ _tests.append(_test_simple_get_index) def _test_optimum_sys_platform_extra(env): builder = hub_builder( env, - evaluate_markers_fn = lambda _, requirements, **__: { - key: [ - platform - for platform in platforms - if ("darwin" in key and "osx" in platform) or ("linux" in key and "linux" in platform) - ] - for key, platforms in requirements.items() - }, ) builder.pip_parse( _mock_mctx( From 0737a0bc24e0f9c5a767f34648372e26a14dc25e Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Tue, 25 Nov 2025 09:37:29 +0900 Subject: [PATCH 538/922] build(deps): bump actions/checkout from 5 to 6 (#3433) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Bumps [actions/checkout](https://github.com/actions/checkout) from 5 to 6.
Release notes

Sourced from actions/checkout's releases.

v6.0.0

What's Changed

Full Changelog: https://github.com/actions/checkout/compare/v5.0.0...v6.0.0

v6-beta

What's Changed

Updated persist-credentials to store the credentials under $RUNNER_TEMP instead of directly in the local git config.

This requires a minimum Actions Runner version of v2.329.0 to access the persisted credentials for Docker container action scenarios.

v5.0.1

What's Changed

Full Changelog: https://github.com/actions/checkout/compare/v5...v5.0.1

Changelog

Sourced from actions/checkout's changelog.

Changelog

V6.0.0

V5.0.1

V5.0.0

V4.3.1

V4.3.0

v4.2.2

v4.2.1

v4.2.0

v4.1.7

v4.1.6

v4.1.5

... (truncated)

Commits

[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=actions/checkout&package-manager=github_actions&previous-version=5&new-version=6)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) ---
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot show ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> --- .github/workflows/mypy.yaml | 2 +- .github/workflows/release.yml | 4 ++-- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/.github/workflows/mypy.yaml b/.github/workflows/mypy.yaml index b83b5d4b37..a22119e118 100644 --- a/.github/workflows/mypy.yaml +++ b/.github/workflows/mypy.yaml @@ -18,7 +18,7 @@ jobs: runs-on: ubuntu-latest steps: # Checkout the code - - uses: actions/checkout@v5 + - uses: actions/checkout@v6 - uses: jpetrucciani/mypy-check@master with: requirements: 1.6.0 diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index 94c4d82561..c565b03fa0 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -40,7 +40,7 @@ jobs: runs-on: ubuntu-latest steps: - name: Checkout - uses: actions/checkout@v5 + uses: actions/checkout@v6 with: ref: ${{ github.ref_name }} - name: Create release archive and notes @@ -71,7 +71,7 @@ jobs: runs-on: ubuntu-latest steps: - name: Checkout - uses: actions/checkout@v5 + uses: actions/checkout@v6 with: ref: ${{ github.tag_name || github.ref_name }} - if: github.event_name == 'push' || github.event.inputs.publish_to_pypi From 89fedb796afaca1304faa56937ae22650538de42 Mon Sep 17 00:00:00 2001 From: Ignas Anikevicius <240938+aignas@users.noreply.github.com> Date: Tue, 25 Nov 2025 12:08:52 +0900 Subject: [PATCH 539/922] refactor(pypi): extract the wheel without python (#3430) With this we start extracting the wheel without Python and it becomes a requirement only when patching (we will extract the wheel without Python, patch it and then re-compress it which makes a very inefficient process). This should result in much faster executions because we can start extracting wheels even before we fetch the entire Python toolchain and we don't need to fetch it in a wheel-only setup until we are actually building/executing tests. What is more bazel is faster in extracting everything. Work towards #2948 --- python/private/internal_config_repo.bzl | 2 ++ python/private/pypi/hub_builder.bzl | 15 ++++++++---- python/private/pypi/patch_whl.bzl | 2 ++ python/private/pypi/whl_library.bzl | 24 ++++++++------------ tests/pypi/hub_builder/hub_builder_tests.bzl | 22 +++++++++--------- 5 files changed, 36 insertions(+), 29 deletions(-) diff --git a/python/private/internal_config_repo.bzl b/python/private/internal_config_repo.bzl index b208037c13..91f786c64e 100644 --- a/python/private/internal_config_repo.bzl +++ b/python/private/internal_config_repo.bzl @@ -32,6 +32,7 @@ config = struct( enable_pystar = True, enable_pipstar = {enable_pipstar}, enable_deprecation_warnings = {enable_deprecation_warnings}, + bazel_8_or_later = {bazel_8_or_later}, bazel_9_or_later = {bazel_9_or_later}, BuiltinPyInfo = getattr(getattr(native, "legacy_globals", None), "PyInfo", {builtin_py_info_symbol}), BuiltinPyRuntimeInfo = getattr(getattr(native, "legacy_globals", None), "PyRuntimeInfo", {builtin_py_runtime_info_symbol}), @@ -107,6 +108,7 @@ def _internal_config_repo_impl(rctx): builtin_py_info_symbol = builtin_py_info_symbol, builtin_py_runtime_info_symbol = builtin_py_runtime_info_symbol, builtin_py_cc_link_params_provider = builtin_py_cc_link_params_provider, + bazel_8_or_later = str(bazel_major_version >= 8), bazel_9_or_later = str(bazel_major_version >= 9), )) diff --git a/python/private/pypi/hub_builder.bzl b/python/private/pypi/hub_builder.bzl index 69359bcf8a..1378e2f122 100644 --- a/python/private/pypi/hub_builder.bzl +++ b/python/private/pypi/hub_builder.bzl @@ -440,6 +440,9 @@ def _create_whl_repos( pip_attr = pip_attr, enable_pipstar = enable_pipstar, ) + + interpreter = _detect_interpreter(self, pip_attr) + for whl in requirements_by_platform: whl_library_args = common_args | _whl_library_args( self, @@ -456,6 +459,7 @@ def _create_whl_repos( auth_patterns = self._config.auth_patterns or pip_attr.auth_patterns, python_version = _major_minor_version(pip_attr.python_version), is_multiple_versions = whl.is_multiple_versions, + interpreter = interpreter, enable_pipstar = enable_pipstar, ) _add_whl_library( @@ -467,8 +471,6 @@ def _create_whl_repos( ) def _common_args(self, module_ctx, *, pip_attr, enable_pipstar): - interpreter = _detect_interpreter(self, pip_attr) - # Construct args separately so that the lock file can be smaller and does not include unused # attrs. whl_library_args = dict( @@ -483,8 +485,6 @@ def _common_args(self, module_ctx, *, pip_attr, enable_pipstar): environment = pip_attr.environment, envsubst = pip_attr.envsubst, pip_data_exclude = pip_attr.pip_data_exclude, - python_interpreter = interpreter.path, - python_interpreter_target = interpreter.target, ) if not enable_pipstar: maybe_args["experimental_target_platforms"] = pip_attr.experimental_target_platforms @@ -536,6 +536,7 @@ def _whl_repo( auth_patterns, python_version, use_downloader, + interpreter, enable_pipstar = False): args = dict(whl_library_args) args["requirement"] = src.requirement_line @@ -548,6 +549,12 @@ def _whl_repo( # need to pass the extra args there, so only pop this for whls args["extra_pip_args"] = src.extra_pip_args + if "whl_patches" in args or not (enable_pipstar and is_whl): + if interpreter.path: + args["python_interpreter"] = interpreter.path + if interpreter.target: + args["python_interpreter_target"] = interpreter.target + if not src.url or (not is_whl and download_only): if download_only and use_downloader: # If the user did not allow using sdists and we are using the downloader diff --git a/python/private/pypi/patch_whl.bzl b/python/private/pypi/patch_whl.bzl index 7af9c4da2f..e315989dd9 100644 --- a/python/private/pypi/patch_whl.bzl +++ b/python/private/pypi/patch_whl.bzl @@ -87,6 +87,8 @@ def patch_whl(rctx, *, python_interpreter, whl_path, patches, **kwargs): # symlink to a zip file to use bazel's extract so that we can use bazel's # repository_ctx patch implementation. The whl file may be in a different # external repository. + # + # TODO @aignas 2025-11-24: remove this symlinking workaround when we drop support for bazel 7 whl_file_zip = whl_input.basename + ".zip" rctx.symlink(whl_input, whl_file_zip) rctx.extract(whl_file_zip) diff --git a/python/private/pypi/whl_library.bzl b/python/private/pypi/whl_library.bzl index 5db7bc49a1..6b515a56a4 100644 --- a/python/private/pypi/whl_library.bzl +++ b/python/private/pypi/whl_library.bzl @@ -377,21 +377,17 @@ def _whl_library_impl(rctx): # # Remove non-pipstar and config_load check when we release rules_python 2. if enable_pipstar: - pypi_repo_utils.execute_checked( - rctx, - op = "whl_library.ExtractWheel({}, {})".format(rctx.attr.name, whl_path), - python = python_interpreter, - arguments = args + [ - "--whl-file", - whl_path, - "--enable-pipstar", - ], - srcs = rctx.attr._python_srcs, - environment = environment, - quiet = rctx.attr.quiet, - timeout = rctx.attr.timeout, - logger = logger, + if rp_config.bazel_8_or_later: + extract_path = whl_path + else: + extract_path = rctx.path(whl_path.basename + ".zip") + rctx.symlink(whl_path, extract_path) + rctx.extract( + archive = extract_path, + output = "site-packages", ) + if not rp_config.bazel_8_or_later: + rctx.delete(extract_path) metadata = whl_metadata( install_dir = whl_path.dirname.get_child("site-packages"), diff --git a/tests/pypi/hub_builder/hub_builder_tests.bzl b/tests/pypi/hub_builder/hub_builder_tests.bzl index a0ab919d68..414ad1250e 100644 --- a/tests/pypi/hub_builder/hub_builder_tests.bzl +++ b/tests/pypi/hub_builder/hub_builder_tests.bzl @@ -44,6 +44,7 @@ def hub_builder( debug = False, config = None, minor_mapping = {}, + whl_overrides = {}, evaluate_markers_fn = None, simpleapi_download_fn = None, available_interpreters = {}): @@ -76,7 +77,7 @@ def hub_builder( netrc = None, auth_patterns = None, ), - whl_overrides = {}, + whl_overrides = whl_overrides, minor_mapping = minor_mapping or {"3.15": "3.15.19"}, available_interpreters = available_interpreters or { "python_3_15_host": "unit_test_interpreter_target", @@ -320,7 +321,6 @@ def _test_simple_extras_vs_no_extras_simpleapi(env): "config_load": "@pypi//:config.bzl", "dep_template": "@pypi//{name}:{target}", "filename": "simple-0.0.1-py3-none-any.whl", - "python_interpreter_target": "unit_test_interpreter_target", "requirement": "simple[foo]==0.0.1", "sha256": "deadbeef", "urls": ["https://example.com/simple-0.0.1-py3-none-any.whl"], @@ -329,7 +329,6 @@ def _test_simple_extras_vs_no_extras_simpleapi(env): "config_load": "@pypi//:config.bzl", "dep_template": "@pypi//{name}:{target}", "filename": "simple-0.0.1-py3-none-any.whl", - "python_interpreter_target": "unit_test_interpreter_target", "requirement": "simple==0.0.1", "sha256": "deadbeef", "urls": ["https://example.com/simple-0.0.1-py3-none-any.whl"], @@ -656,7 +655,6 @@ torch==2.4.1+cpu ; platform_machine == 'x86_64' \ "config_load": "@pypi//:config.bzl", "dep_template": "@pypi//{name}:{target}", "filename": "torch-2.4.1+cpu-cp312-cp312-linux_x86_64.whl", - "python_interpreter_target": "unit_test_interpreter_target", "requirement": "torch==2.4.1+cpu", "sha256": "8800deef0026011d502c0c256cc4b67d002347f63c3a38cd8e45f1f445c61364", "urls": ["https://torch.index/whl/cpu/torch-2.4.1%2Bcpu-cp312-cp312-linux_x86_64.whl"], @@ -665,7 +663,6 @@ torch==2.4.1+cpu ; platform_machine == 'x86_64' \ "config_load": "@pypi//:config.bzl", "dep_template": "@pypi//{name}:{target}", "filename": "torch-2.4.1-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", - "python_interpreter_target": "unit_test_interpreter_target", "requirement": "torch==2.4.1", "sha256": "36109432b10bd7163c9b30ce896f3c2cca1b86b9765f956a1594f0ff43091e2a", "urls": ["https://torch.index/whl/cpu/torch-2.4.1-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl"], @@ -674,7 +671,6 @@ torch==2.4.1+cpu ; platform_machine == 'x86_64' \ "config_load": "@pypi//:config.bzl", "dep_template": "@pypi//{name}:{target}", "filename": "torch-2.4.1+cpu-cp312-cp312-win_amd64.whl", - "python_interpreter_target": "unit_test_interpreter_target", "requirement": "torch==2.4.1+cpu", "sha256": "3a570e5c553415cdbddfe679207327b3a3806b21c6adea14fba77684d1619e97", "urls": ["https://torch.index/whl/cpu/torch-2.4.1%2Bcpu-cp312-cp312-win_amd64.whl"], @@ -683,7 +679,6 @@ torch==2.4.1+cpu ; platform_machine == 'x86_64' \ "config_load": "@pypi//:config.bzl", "dep_template": "@pypi//{name}:{target}", "filename": "torch-2.4.1-cp312-none-macosx_11_0_arm64.whl", - "python_interpreter_target": "unit_test_interpreter_target", "requirement": "torch==2.4.1", "sha256": "72b484d5b6cec1a735bf3fa5a1c4883d01748698c5e9cfdbeb4ffab7c7987e0d", "urls": ["https://torch.index/whl/cpu/torch-2.4.1-cp312-none-macosx_11_0_arm64.whl"], @@ -845,6 +840,11 @@ def _test_simple_get_index(env): builder = hub_builder( env, simpleapi_download_fn = mocksimpleapi_download, + whl_overrides = { + "direct_without_sha": { + "my_patch": 1, + }, + }, ) builder.pip_parse( _mock_mctx( @@ -1003,6 +1003,10 @@ git_dep @ git+https://git.server/repo/project@deadbeefdeadbeef "requirement": "direct_without_sha==0.0.1", "sha256": "", "urls": ["example-direct.org/direct_without_sha-0.0.1-py3-none-any.whl"], + # NOTE @aignas 2025-11-24: any patching still requires the python interpreter from the + # hermetic toolchain or the system. This is so that we can rezip it back to a wheel and + # verify the metadata so that it is installable by any installer out there. + "whl_patches": {"my_patch": "1"}, }, "pypi_315_git_dep": { "config_load": "@pypi//:config.bzl", @@ -1022,7 +1026,6 @@ git_dep @ git+https://git.server/repo/project@deadbeefdeadbeef "config_load": "@pypi//:config.bzl", "dep_template": "@pypi//{name}:{target}", "filename": "plat-pkg-0.0.4-py3-none-linux_x86_64.whl", - "python_interpreter_target": "unit_test_interpreter_target", "requirement": "plat_pkg==0.0.4", "sha256": "deadb44f", "urls": ["example2.org/index/plat_pkg/"], @@ -1031,7 +1034,6 @@ git_dep @ git+https://git.server/repo/project@deadbeefdeadbeef "config_load": "@pypi//:config.bzl", "dep_template": "@pypi//{name}:{target}", "filename": "simple-0.0.1-py3-none-any.whl", - "python_interpreter_target": "unit_test_interpreter_target", "requirement": "simple==0.0.1", "sha256": "deadb00f", "urls": ["example2.org"], @@ -1040,7 +1042,6 @@ git_dep @ git+https://git.server/repo/project@deadbeefdeadbeef "config_load": "@pypi//:config.bzl", "dep_template": "@pypi//{name}:{target}", "filename": "some_pkg-0.0.1-py3-none-any.whl", - "python_interpreter_target": "unit_test_interpreter_target", "requirement": "some_pkg==0.0.1", "sha256": "deadbaaf", "urls": ["example-direct.org/some_pkg-0.0.1-py3-none-any.whl"], @@ -1049,7 +1050,6 @@ git_dep @ git+https://git.server/repo/project@deadbeefdeadbeef "config_load": "@pypi//:config.bzl", "dep_template": "@pypi//{name}:{target}", "filename": "some-other-pkg-0.0.1-py3-none-any.whl", - "python_interpreter_target": "unit_test_interpreter_target", "requirement": "some_other_pkg==0.0.1", "sha256": "deadb33f", "urls": ["example2.org/index/some_other_pkg/"], From 1771677d90e372bf722e13992c6ee22f036ccc12 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Tue, 25 Nov 2025 16:47:48 -0800 Subject: [PATCH 540/922] chore: remove extraneous dep from sys_path_order test (#3435) The bazel_tools runfiles dep was mistakenly also added to another test when the test for importing bazel_tools was added. --- tests/bootstrap_impls/BUILD.bazel | 3 --- 1 file changed, 3 deletions(-) diff --git a/tests/bootstrap_impls/BUILD.bazel b/tests/bootstrap_impls/BUILD.bazel index 5f7e5afd95..e1f60f5b40 100644 --- a/tests/bootstrap_impls/BUILD.bazel +++ b/tests/bootstrap_impls/BUILD.bazel @@ -133,9 +133,6 @@ py_reconfig_test( env = {"BOOTSTRAP": "system_python"}, imports = ["./site-packages"], main = "sys_path_order_test.py", - deps = [ - "@bazel_tools//tools/python/runfiles", - ], ) py_reconfig_test( From 411b937484b7a690fad2bacf2d03fec60bed765b Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Tue, 25 Nov 2025 23:46:18 -0800 Subject: [PATCH 541/922] refactor: remove gazelle plugin as dev dependency (#3436) Remove the gazelle pluging as a dev-time dependency of rules_python itself. This is to avoid gazelle-related build issues that affect the main build. The gazelle code paths in tests/integration are removed because they were just test runners being defined, but weren't actually used with any tests. --- BUILD.bazel | 1 - MODULE.bazel | 9 --------- tests/integration/BUILD.bazel | 28 -------------------------- tests/integration/integration_test.bzl | 11 ++-------- 4 files changed, 2 insertions(+), 47 deletions(-) diff --git a/BUILD.bazel b/BUILD.bazel index 5e85c27b3c..aa2642d43f 100644 --- a/BUILD.bazel +++ b/BUILD.bazel @@ -45,7 +45,6 @@ filegroup( "version.bzl", "//python:distribution", "//tools:distribution", - "@rules_python_gazelle_plugin//:distribution", ], visibility = [ "//:__subpackages__", diff --git a/MODULE.bazel b/MODULE.bazel index 6e9b725c53..b909124d11 100644 --- a/MODULE.bazel +++ b/MODULE.bazel @@ -227,8 +227,6 @@ bazel_dep(name = "another_module", version = "0", dev_dependency = True) # Extra gazelle plugin deps so that WORKSPACE.bzlmod can continue including it for e2e tests. # We use `WORKSPACE.bzlmod` because it is impossible to have dev-only local overrides. bazel_dep(name = "rules_go", version = "0.41.0", dev_dependency = True, repo_name = "io_bazel_rules_go") -bazel_dep(name = "rules_python_gazelle_plugin", version = "0", dev_dependency = True) -bazel_dep(name = "gazelle", version = "0.40.0", dev_dependency = True, repo_name = "bazel_gazelle") internal_dev_deps = use_extension( "//python/private:internal_dev_deps.bzl", @@ -258,13 +256,6 @@ dev_rules_python_config.add_transition_setting( setting = "//tests/multi_pypi:external_deps_name", ) -# Add gazelle plugin so that we can run the gazelle example as an e2e integration -# test and include the distribution files. -local_path_override( - module_name = "rules_python_gazelle_plugin", - path = "gazelle", -) - local_path_override( module_name = "other", path = "tests/modules/other", diff --git a/tests/integration/BUILD.bazel b/tests/integration/BUILD.bazel index 673312903d..f0f58daa3a 100644 --- a/tests/integration/BUILD.bazel +++ b/tests/integration/BUILD.bazel @@ -23,14 +23,6 @@ _WORKSPACE_FLAGS = [ "--enable_workspace", ] -_WORKSPACE_GAZELLE_PLUGIN_FLAGS = [ - "--override_repository=rules_python_gazelle_plugin=../../../rules_python_gazelle_plugin", -] - -_GAZELLE_PLUGIN_FLAGS = [ - "--override_module=rules_python_gazelle_plugin=../../../rules_python_gazelle_plugin", -] - default_test_runner( name = "workspace_test_runner", bazel_cmds = [ @@ -40,31 +32,11 @@ default_test_runner( visibility = ["//visibility:public"], ) -default_test_runner( - name = "workspace_test_runner_gazelle_plugin", - bazel_cmds = [ - "info {}".format(" ".join(_WORKSPACE_FLAGS + _WORKSPACE_GAZELLE_PLUGIN_FLAGS)), - "test {} //...".format(" ".join(_WORKSPACE_FLAGS + _WORKSPACE_GAZELLE_PLUGIN_FLAGS)), - ], - visibility = ["//visibility:public"], -) - default_test_runner( name = "test_runner", visibility = ["//visibility:public"], ) -default_test_runner( - name = "test_runner_gazelle_plugin", - bazel_cmds = [ - "info {}".format(" ".join(_GAZELLE_PLUGIN_FLAGS)), - "test {} //...".format(" ".join(_GAZELLE_PLUGIN_FLAGS)), - ], - visibility = ["//visibility:public"], -) - -# TODO: add compile_pip_requirements_test_from_external_repo - rules_python_integration_test( name = "compile_pip_requirements_test", ) diff --git a/tests/integration/integration_test.bzl b/tests/integration/integration_test.bzl index 90cc4a3fb7..771976d037 100644 --- a/tests/integration/integration_test.bzl +++ b/tests/integration/integration_test.bzl @@ -21,7 +21,7 @@ load( ) load("//python:py_test.bzl", "py_test") -def _test_runner(*, name, bazel_version, py_main, bzlmod, gazelle_plugin): +def _test_runner(*, name, bazel_version, py_main, bzlmod): if py_main: test_runner = "{}_bazel_{}_py_runner".format(name, bazel_version) py_test( @@ -35,12 +35,8 @@ def _test_runner(*, name, bazel_version, py_main, bzlmod, gazelle_plugin): ) return test_runner - if bzlmod and gazelle_plugin: - return "//tests/integration:test_runner_gazelle_plugin" - elif bzlmod: + if bzlmod: return "//tests/integration:test_runner" - elif gazelle_plugin: - return "//tests/integration:workspace_test_runner_gazelle_plugin" else: return "//tests/integration:workspace_test_runner" @@ -48,7 +44,6 @@ def rules_python_integration_test( name, workspace_path = None, bzlmod = True, - gazelle_plugin = False, tags = None, py_main = None, bazel_versions = None, @@ -61,7 +56,6 @@ def rules_python_integration_test( `_test` suffix. bzlmod: bool, default True. If true, run with bzlmod enabled, otherwise disable bzlmod. - gazelle_plugin: Whether the test uses the gazelle plugin. tags: Test tags. py_main: Optional `.py` file to run tests using. When specified, a python based test runner is used, and this source file is the main @@ -98,7 +92,6 @@ def rules_python_integration_test( bazel_version = bazel_version, py_main = py_main, bzlmod = bzlmod, - gazelle_plugin = gazelle_plugin, ) bazel_integration_test( name = "{}_bazel_{}".format(name, bazel_version), From 759f5da9ade00f60729eff85915bf173597c4767 Mon Sep 17 00:00:00 2001 From: Ignas Anikevicius <240938+aignas@users.noreply.github.com> Date: Thu, 4 Dec 2025 17:12:01 +0900 Subject: [PATCH 542/922] ci: add ci config to test 7 and 8 for bcr like setup (#3404) Run the examples that BCR uses as tests in a more similar way to BCR. Unfortunately, this breaks the gazelle plugin on Windows due to #3416, so testing of it is removed. Work towards #3392 --------- Co-authored-by: Richard Levasseur --- .bazelci/presubmit.yml | 160 +++++++----------- .bcr/gazelle/presubmit.yml | 15 +- .bcr/presubmit.yml | 12 +- CHANGELOG.md | 8 + examples/bzlmod/.bazelrc | 11 ++ examples/bzlmod/MODULE.bazel | 2 +- examples/multi_python_versions/MODULE.bazel | 2 +- gazelle/MODULE.bazel | 4 +- .../bzlmod_build_file_generation/.bazelrc | 2 +- .../bzlmod_build_file_generation/MODULE.bazel | 4 +- 10 files changed, 105 insertions(+), 115 deletions(-) diff --git a/.bazelci/presubmit.yml b/.bazelci/presubmit.yml index e6ee33558a..daa8f87ea3 100644 --- a/.bazelci/presubmit.yml +++ b/.bazelci/presubmit.yml @@ -91,7 +91,63 @@ buildifier: - //tests:version_3_13_test - //tests:version_3_14_test - //tests:version_default_test + +# Keep in sync with .bcr/gazelle/presubmit.yml +.gazelle_common_bcr: &gazelle_common_bcr + bazel: ${{ bazel }} + working_directory: gazelle/examples/bzlmod_build_file_generation + shell_commands: + - "echo 'common --override_module=rules_python=' >> .bazelrc" + - "bazel run //:gazelle_python_manifest.update" + - "bazel run //:gazelle -- update" + batch_commands: + - "echo common --override_module=rules_python= >> .bazelrc " + - " bazel run //:gazelle_python_manifest.update " + - " bazel run //:gazelle -- update" + build_targets: + - "//..." + - ":modules_map" + test_targets: + - "//..." + + +matrix: + platform: + - ubuntu2204 + - debian11 + - macos_arm64 + - windows + bazel: [7.*, 8.*] + tasks: + # Keep in sync with .bcr/presubmit.yml + bcr_test: + name: "BCR: Bazel {bazel}" + platform: ${{ platform }} + working_directory: examples/bzlmod + bazel: ${{ bazel }} + build_flags: + - "--keep_going" + test_flags: + - "--keep_going" + build_targets: + - "//..." + test_targets: + - "//..." + + gazelle_bcr_ubuntu: + <<: *gazelle_common_bcr + name: "Gazelle: BCR, Bazel {bazel}" + platform: ubuntu2204 + gazelle_bcr_debian11: + <<: *gazelle_common_bcr + name: "Gazelle: BCR, Bazel {bazel}" + platform: debian11 + gazelle_bcr_macos_arm64: + <<: *gazelle_common_bcr + name: "Gazelle: BCR, Bazel {bazel}" + platform: macos_arm64 + gazelle_extension_min: <<: *common_workspace_flags_min_bazel <<: *minimum_supported_version @@ -262,113 +318,15 @@ tasks: working_directory: examples/build_file_generation platform: windows - integration_test_bzlmod_ubuntu_min: - <<: *minimum_supported_version - <<: *reusable_build_test_all - coverage_targets: ["//:test"] - name: "examples/bzlmod: Ubuntu, minimum Bazel" - working_directory: examples/bzlmod - platform: ubuntu2204 - bazel: 7.x - integration_test_bzlmod_ubuntu: - <<: *reusable_build_test_all - <<: *coverage_targets_example_bzlmod - name: "examples/bzlmod: Ubuntu" - working_directory: examples/bzlmod - platform: ubuntu2204 - bazel: 7.x - integration_test_bzlmod_ubuntu_upcoming: - <<: *reusable_build_test_all - <<: *coverage_targets_example_bzlmod - name: "examples/bzlmod: Ubuntu, upcoming Bazel" - working_directory: examples/bzlmod - platform: ubuntu2204 - bazel: last_rc - integration_test_bzlmod_debian: - <<: *reusable_build_test_all - <<: *coverage_targets_example_bzlmod - name: "examples/bzlmod: Debian" - working_directory: examples/bzlmod - platform: debian11 - bazel: 7.x integration_test_bzlmod_ubuntu_vendor: <<: *reusable_build_test_all name: "examples/bzlmod: bazel vendor" working_directory: examples/bzlmod platform: ubuntu2204 shell_commands: - - "bazel vendor --vendor_dir=./vendor //..." - - "bazel build --vendor_dir=./vendor //..." - - "rm -rf ./vendor" - integration_test_bzlmod_macos: - <<: *reusable_build_test_all - <<: *coverage_targets_example_bzlmod - name: "examples/bzlmod: macOS" - working_directory: examples/bzlmod - platform: macos_arm64 - bazel: 7.x - integration_test_bzlmod_macos_upcoming: - <<: *reusable_build_test_all - <<: *coverage_targets_example_bzlmod - name: "examples/bzlmod: macOS, upcoming Bazel" - working_directory: examples/bzlmod - platform: macos_arm64 - bazel: last_rc - integration_test_bzlmod_windows: - <<: *reusable_build_test_all - # coverage is not supported on Windows - name: "examples/bzlmod: Windows" - working_directory: examples/bzlmod - platform: windows - bazel: 7.x - integration_test_bzlmod_windows_upcoming: - <<: *reusable_build_test_all - # coverage is not supported on Windows - name: "examples/bzlmod: Windows, upcoming Bazel" - working_directory: examples/bzlmod - platform: windows - bazel: last_rc - - integration_test_bzlmod_generate_build_file_generation_ubuntu_min: - <<: *minimum_supported_version - <<: *reusable_build_test_all - <<: *coverage_targets_example_bzlmod_build_file_generation - name: "gazelle/examples/bzlmod_build_file_generation: Ubuntu, minimum Bazel" - working_directory: gazelle/examples/bzlmod_build_file_generation - platform: ubuntu2204 - bazel: 7.x - integration_test_bzlmod_generation_build_files_ubuntu: - <<: *reusable_build_test_all - <<: *coverage_targets_example_bzlmod_build_file_generation - name: "gazelle/examples/bzlmod_build_file_generation: Ubuntu" - working_directory: gazelle/examples/bzlmod_build_file_generation - platform: ubuntu2204 - integration_test_bzlmod_generation_build_files_ubuntu_run: - <<: *reusable_build_test_all - name: "gazelle/examples/bzlmod_build_file_generation: Ubuntu, Gazelle and pip" - working_directory: gazelle/examples/bzlmod_build_file_generation - platform: ubuntu2204 - shell_commands: - - "bazel run //:gazelle_python_manifest.update" - - "bazel run //:gazelle -- update" - integration_test_bzlmod_build_file_generation_debian: - <<: *reusable_build_test_all - <<: *coverage_targets_example_bzlmod_build_file_generation - name: "gazelle/examples/bzlmod_build_file_generation: Debian" - working_directory: gazelle/examples/bzlmod_build_file_generation - platform: debian11 - integration_test_bzlmod_build_file_generation_macos: - <<: *reusable_build_test_all - <<: *coverage_targets_example_bzlmod_build_file_generation - name: "gazelle/examples/bzlmod_build_file_generation: MacOS" - working_directory: gazelle/examples/bzlmod_build_file_generation - platform: macos_arm64 - integration_test_bzlmod_build_file_generation_windows: - <<: *reusable_build_test_all - # coverage is not supported on Windows - name: "gazelle/examples/bzlmod_build_file_generation: Windows" - working_directory: gazelle/examples/bzlmod_build_file_generation - platform: windows + - "bazel vendor --vendor_dir=./vendor //..." + - "bazel build --vendor_dir=./vendor //..." + - "rm -rf ./vendor" integration_test_multi_python_versions_ubuntu_workspace: <<: *reusable_build_test_all diff --git a/.bcr/gazelle/presubmit.yml b/.bcr/gazelle/presubmit.yml index 3300f67f29..7e35ce5648 100644 --- a/.bcr/gazelle/presubmit.yml +++ b/.bcr/gazelle/presubmit.yml @@ -15,8 +15,15 @@ bcr_test_module: module_path: "examples/bzlmod_build_file_generation" matrix: - platform: ["debian11", "macos", "ubuntu2004", "windows"] - bazel: [7.x, 8.x] + platform: [ + "debian11", + "macos", + "ubuntu2204", + ] + bazel: [ + 7.*, + 8.*, + ] tasks: run_tests: name: "Run test module" @@ -24,8 +31,12 @@ bcr_test_module: bazel: ${{ bazel }} shell_commands: - "echo 'common --override_module=rules_python=' >> .bazelrc" + - "bazel run //:gazelle_python_manifest.update" + - "bazel run //:gazelle -- update" batch_commands: - "echo common --override_module=rules_python= >> .bazelrc" + - "bazel run //:gazelle_python_manifest.update" + - "bazel run //:gazelle -- update" build_targets: - "//..." - ":modules_map" diff --git a/.bcr/presubmit.yml b/.bcr/presubmit.yml index b016dc9d6f..6217ad7ba2 100644 --- a/.bcr/presubmit.yml +++ b/.bcr/presubmit.yml @@ -15,17 +15,19 @@ bcr_test_module: module_path: "examples/bzlmod" matrix: - platform: ["debian11", "macos", "ubuntu2004", "windows"] - bazel: [7.x, 8.x] + platform: ["debian11", "macos", "ubuntu2204", "windows"] + bazel: [7.*, 8.*] tasks: run_tests: name: "Run test module" platform: ${{ platform }} bazel: ${{ bazel }} test_flags: + # Minimum bazel supported C++ - "--keep_going" - # Without these cxxopts, BCR's Mac builds fail - - '--cxxopt=-std=c++14' - - '--host_cxxopt=-std=c++14' + - '--cxxopt=-std=c++17' + - '--host_cxxopt=-std=c++17' + build_targets: + - "//..." test_targets: - "//..." diff --git a/CHANGELOG.md b/CHANGELOG.md index 4e9514352d..9776a7f4c2 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -53,6 +53,12 @@ END_UNRELEASED_TEMPLATE [0.0.0]: https://github.com/bazel-contrib/rules_python/releases/tag/0.0.0 +{#v0-0-0-known-issues} +### Known Issues +* (gazelle) Windows support for the Gazelle plugin may be broken. See + [#3416](https://github.com/bazel-contrib/rules_python/issues/3416) for + details and possible workarounds. + {#v0-0-0-removed} ### Removed * (toolchain) Remove all of the python 3.8 toolchain support out of the box. Users need @@ -75,6 +81,8 @@ END_UNRELEASED_TEMPLATE * (pip) `pipstar` has been enabled for all `whl_library` instances where the whl is passed through a label or downloaded using the bazel downloader ([#2949](https://github.com/bazel-contrib/rules_python/issues/2949)). +* (gazelle deps) rules_go bumped from 0.55.1 to 0.59.0 +* (gazelle deps) gazelle bumped from 0.36.0 to 0.47.0 {#v0-0-0-fixed} ### Fixed diff --git a/examples/bzlmod/.bazelrc b/examples/bzlmod/.bazelrc index 7c92800d76..28a44a7523 100644 --- a/examples/bzlmod/.bazelrc +++ b/examples/bzlmod/.bazelrc @@ -5,6 +5,17 @@ startup --windows_enable_symlinks common --enable_bzlmod common --lockfile_mode=update +# This adds an implicit --config= +# See docs for osname values +# https://bazel.build/reference/command-line-reference#common_options-flag--enable_platform_specific_config +common --enable_platform_specific_config + +common:windows --cxxopt=/std:c++17 +common:windows --host_cxxopt=/std:c++17 +common:linux --cxxopt=-std=c++17 +common:linux --host_cxxopt=-std=c++17 +common:macos --cxxopt=-std=c++17 +common:macos --host_cxxopt=-std=c++17 coverage --java_runtime_version=remotejdk_11 diff --git a/examples/bzlmod/MODULE.bazel b/examples/bzlmod/MODULE.bazel index f5311fb4d1..14c490cb0d 100644 --- a/examples/bzlmod/MODULE.bazel +++ b/examples/bzlmod/MODULE.bazel @@ -17,7 +17,7 @@ bazel_dep(name = "protobuf", version = "27.0", repo_name = "com_google_protobuf" # Only needed to make rules_python's CI happy. rules_java 8.3.0+ is needed so # that --java_runtime_version=remotejdk_11 works with Bazel 8. -bazel_dep(name = "rules_java", version = "8.3.1") +bazel_dep(name = "rules_java", version = "8.16.1") # Only needed to make rules_python's CI happy. A test verifies that # MODULE.bazel.lock is cross-platform friendly, and there are transitive diff --git a/examples/multi_python_versions/MODULE.bazel b/examples/multi_python_versions/MODULE.bazel index 2ef09ade3e..82faaf8214 100644 --- a/examples/multi_python_versions/MODULE.bazel +++ b/examples/multi_python_versions/MODULE.bazel @@ -75,4 +75,4 @@ bazel_dep(name = "rules_shell", version = "0.2.0", dev_dependency = True) # Only needed to make rules_python's CI happy. rules_java 8.3.0+ is needed so # that --java_runtime_version=remotejdk_11 works with Bazel 8. -bazel_dep(name = "rules_java", version = "8.3.1") +bazel_dep(name = "rules_java", version = "8.16.1") diff --git a/gazelle/MODULE.bazel b/gazelle/MODULE.bazel index add5986903..cff6341a2b 100644 --- a/gazelle/MODULE.bazel +++ b/gazelle/MODULE.bazel @@ -6,8 +6,8 @@ module( bazel_dep(name = "bazel_skylib", version = "1.8.2") bazel_dep(name = "rules_python", version = "0.18.0") -bazel_dep(name = "rules_go", version = "0.55.1", repo_name = "io_bazel_rules_go") -bazel_dep(name = "gazelle", version = "0.36.0", repo_name = "bazel_gazelle") +bazel_dep(name = "rules_go", version = "0.59.0", repo_name = "io_bazel_rules_go") +bazel_dep(name = "gazelle", version = "0.47.0", repo_name = "bazel_gazelle") bazel_dep(name = "rules_cc", version = "0.0.16") local_path_override( diff --git a/gazelle/examples/bzlmod_build_file_generation/.bazelrc b/gazelle/examples/bzlmod_build_file_generation/.bazelrc index d58665596b..31097b41de 100644 --- a/gazelle/examples/bzlmod_build_file_generation/.bazelrc +++ b/gazelle/examples/bzlmod_build_file_generation/.bazelrc @@ -3,7 +3,7 @@ test --test_output=errors --enable_runfiles # Windows requires these for multi-python support: build --enable_runfiles -common --experimental_enable_bzlmod +common --enable_bzlmod coverage --java_runtime_version=remotejdk_11 common:bazel7.x --incompatible_python_disallow_native_rules diff --git a/gazelle/examples/bzlmod_build_file_generation/MODULE.bazel b/gazelle/examples/bzlmod_build_file_generation/MODULE.bazel index ce779627f5..1f92ea3826 100644 --- a/gazelle/examples/bzlmod_build_file_generation/MODULE.bazel +++ b/gazelle/examples/bzlmod_build_file_generation/MODULE.bazel @@ -31,7 +31,7 @@ local_path_override( # The following stanza defines the dependency for gazelle # See here https://github.com/bazelbuild/bazel-gazelle/releases/ for the # latest version. -bazel_dep(name = "gazelle", version = "0.36.0", repo_name = "bazel_gazelle") +bazel_dep(name = "gazelle", version = "0.47.0", repo_name = "bazel_gazelle") # The following stanze returns a proxy object representing a module extension; # its methods can be invoked to create module extension tags. @@ -84,4 +84,4 @@ local_path_override( ) # Only needed to make rules_python's CI happy -bazel_dep(name = "rules_java", version = "8.3.1") +bazel_dep(name = "rules_java", version = "8.16.1") From 4d9c2b1ea112964d22e16428262c968470c1c1b4 Mon Sep 17 00:00:00 2001 From: Ignas Anikevicius <240938+aignas@users.noreply.github.com> Date: Sun, 7 Dec 2025 09:27:34 +0900 Subject: [PATCH 543/922] feat(pip.parse): limit the target platforms we parse requirements for (#3441) Up until now the users can configure which requirements files to be used for specific platforms, however, what they cannot configure is what target platforms should actually be set up. The difference in the problems is: 1. I want my `bazel build` to work on `osx aarch64` and `linux x86_64`. 1. I want my `bazel build` to build for `linux x86_64` on `osx aarch64`. With the newly introduced `target_platforms` attribute users can finally specify their target platforms. To ensure that this also allows users to specify that they want to support `freethreaded` and `non-freethreaded` platforms at the same time we support `{os}` and `{arch}` templating in the strings. This should fix the `genquery` usage pattern breakage when we previously enabled `RULES_PYTHON_ENABLE_PIPSTAR=1`. Work towards #2949 Work towards #3434 --------- Co-authored-by: Richard Levasseur --- CHANGELOG.md | 8 ++ python/private/pypi/extension.bzl | 20 +++++ python/private/pypi/hub_builder.bzl | 23 +++++- .../pypi/requirements_files_by_platform.bzl | 9 ++- tests/pypi/extension/pip_parse.bzl | 3 + tests/pypi/hub_builder/hub_builder_tests.bzl | 77 ++++++++++++++++++- .../requirements_files_by_platform_tests.bzl | 17 ++++ 7 files changed, 148 insertions(+), 9 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 9776a7f4c2..a41ac20103 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -108,6 +108,14 @@ END_UNRELEASED_TEMPLATE {#v0-0-0-added} ### Added * (toolchains) `3.9.25` Python toolchain from [20251031] release. +* (pypi) API to tell `pip.parse` which platforms users care about. This is very useful to ensure + that when users do `bazel query` for their deps, they don't have to download all of the + dependencies for all of the available wheels. Torch wheels can be up of 1GB and it takes a lot + of time to download those, which is unnecessary if only the host platform builds are necessary + to be performed. This is mainly for backwards/forwards compatibility whilst rolling out + `RULES_PYTHON_ENABLE_PIPSTAR=1` by default. Users of `experimental_index_url` that perform + cross-builds should add {obj}`target_platforms` to their `pip.parse` invocations, which will + become mandatory if any cross-builds are required from the next release. [20251031]: https://github.com/astral-sh/python-build-standalone/releases/tag/20251031 {#v1-7-0} diff --git a/python/private/pypi/extension.bzl b/python/private/pypi/extension.bzl index be1a8e4d03..eaa6c0d428 100644 --- a/python/private/pypi/extension.bzl +++ b/python/private/pypi/extension.bzl @@ -667,6 +667,26 @@ EXPERIMENTAL: this may be removed without notice. :::{versionadded} 1.4.0 ::: +""", + ), + "target_platforms": attr.string_list( + default = ["{os}_{arch}"], + doc = """\ +The list of platforms for which we would evaluate the requirements files. If you need to be able to +only evaluate for a particular platform (e.g. "linux_x86_64"), then put it in here. + +If you want `freethreaded` variant, then you can use `_freethreaded` suffix as `rules_python` is +defining target platforms for these variants in its `MODULE.bazel` file. The identifiers for this +function in general are the same as used in the {obj}`pip.default.platform` attribute. + +If you only care for the host platform and do not have a usecase to cross-build, then you can put in +a string `"{os}_{arch}"` as the value here. You could also use `"{os}_{arch}_freethreaded"` as well. + +:::{include} /_includes/experimental_api.md +::: + +:::{versionadded} VERSION_NEXT_FEATURE +::: """, ), "whl_modifications": attr.label_keyed_string_dict( diff --git a/python/private/pypi/hub_builder.bzl b/python/private/pypi/hub_builder.bzl index 1378e2f122..3a1a3b07fe 100644 --- a/python/private/pypi/hub_builder.bzl +++ b/python/private/pypi/hub_builder.bzl @@ -2,6 +2,7 @@ load("//python/private:full_version.bzl", "full_version") load("//python/private:normalize_name.bzl", "normalize_name") +load("//python/private:repo_utils.bzl", "repo_utils") load("//python/private:version.bzl", "version") load("//python/private:version_label.bzl", "version_label") load(":attrs.bzl", "use_isolated") @@ -135,11 +136,15 @@ def _pip_parse(self, module_ctx, pip_attr): )) return + default_cross_setup = _set_get_index_urls(self, pip_attr) self._platforms[python_version] = _platforms( + module_ctx, python_version = full_python_version, config = self._config, + # FIXME @aignas 2025-12-06: should we have this behaviour? + # TODO @aignas 2025-12-06: use target_platforms always even when the get_index_urls is set. + target_platforms = [] if default_cross_setup else pip_attr.target_platforms, ) - _set_get_index_urls(self, pip_attr) _add_group_map(self, pip_attr.experimental_requirement_cycles) _add_extra_aliases(self, pip_attr.extra_hub_aliases) _create_whl_repos( @@ -249,7 +254,7 @@ def _set_get_index_urls(self, pip_attr): # parallel_download is set to True by default, so we are not checking/validating it # here - return + return False python_version = pip_attr.python_version self._use_downloader.setdefault(python_version, {}).update({ @@ -275,6 +280,7 @@ def _set_get_index_urls(self, pip_attr): cache = self._simpleapi_cache, parallel_download = pip_attr.parallel_download, ) + return True def _detect_interpreter(self, pip_attr): python_interpreter_target = pip_attr.python_interpreter_target @@ -301,14 +307,25 @@ def _detect_interpreter(self, pip_attr): path = pip_attr.python_interpreter, ) -def _platforms(*, python_version, config): +def _platforms(module_ctx, *, python_version, config, target_platforms): platforms = {} python_version = version.parse( python_version, strict = True, ) + target_platforms = sorted({ + p.format( + os = repo_utils.get_platforms_os_name(module_ctx), + arch = repo_utils.get_platforms_cpu_name(module_ctx), + ): None + for p in target_platforms + }) + for platform, values in config.platforms.items(): + if target_platforms and platform not in target_platforms: + continue + # TODO @aignas 2025-07-07: this is probably doing the parsing of the version too # many times. abi = "{}{}{}.{}".format( diff --git a/python/private/pypi/requirements_files_by_platform.bzl b/python/private/pypi/requirements_files_by_platform.bzl index 356bd4416e..2027b41594 100644 --- a/python/private/pypi/requirements_files_by_platform.bzl +++ b/python/private/pypi/requirements_files_by_platform.bzl @@ -140,9 +140,10 @@ def requirements_files_by_platform( platforms_from_args = _platforms_from_args(extra_pip_args) if logger: - logger.debug(lambda: "Platforms from pip args: {}".format(platforms_from_args)) + logger.debug(lambda: "Platforms from pip args: {} (from {})".format(platforms_from_args, extra_pip_args)) - default_platforms = platforms + input_platforms = platforms + default_platforms = [_platform(p, python_version) for p in platforms] if platforms_from_args: lock_files = [ @@ -174,6 +175,7 @@ def requirements_files_by_platform( platform for filter_or_platform in specifier.split(",") for platform in (_default_platforms(filter = filter_or_platform, platforms = platforms) if filter_or_platform.endswith("*") else [filter_or_platform]) + if _platform(platform, python_version) in default_platforms ] for file, specifier in requirements_by_platform.items() }.items() @@ -227,9 +229,10 @@ def requirements_files_by_platform( configured_platforms[p] = file elif logger: - logger.warn(lambda: "File {} will be ignored because there are no configured platforms: {}".format( + logger.info(lambda: "File {} will be ignored because there are no configured platforms: {} out of {}".format( file, default_platforms, + input_platforms, )) continue diff --git a/tests/pypi/extension/pip_parse.bzl b/tests/pypi/extension/pip_parse.bzl index 21569cf04e..edac12e344 100644 --- a/tests/pypi/extension/pip_parse.bzl +++ b/tests/pypi/extension/pip_parse.bzl @@ -27,6 +27,7 @@ def pip_parse( requirements_linux = None, requirements_lock = None, requirements_windows = None, + target_platforms = [], simpleapi_skip = [], timeout = 600, whl_modifications = {}, @@ -41,7 +42,9 @@ def pip_parse( envsubst = envsubst, experimental_index_url = experimental_index_url, experimental_requirement_cycles = experimental_requirement_cycles, + # TODO @aignas 2025-12-02: decide on a single attr - should we reuse this? experimental_target_platforms = experimental_target_platforms, + target_platforms = target_platforms, extra_hub_aliases = extra_hub_aliases, extra_pip_args = extra_pip_args, hub_name = hub_name, diff --git a/tests/pypi/hub_builder/hub_builder_tests.bzl b/tests/pypi/hub_builder/hub_builder_tests.bzl index 414ad1250e..e267f4ca34 100644 --- a/tests/pypi/hub_builder/hub_builder_tests.bzl +++ b/tests/pypi/hub_builder/hub_builder_tests.bzl @@ -25,12 +25,12 @@ load("//tests/pypi/extension:pip_parse.bzl", _parse = "pip_parse") _tests = [] -def _mock_mctx(environ = {}, read = None): +def _mock_mctx(os = "unittest", arch = "exotic", environ = {}, read = None): return struct( os = struct( environ = environ, - name = "unittest", - arch = "exotic", + name = os, + arch = arch, ), read = read or (lambda _: """\ simple==0.0.1 \ @@ -723,6 +723,10 @@ simple==0.0.3 \ "requirements.linux_x86_64.txt": "linux_x86_64", "requirements.osx_aarch64.txt": "osx_aarch64", }, + target_platforms = [ + "linux_x86_64", + "osx_aarch64", + ], ), ) pypi = builder.build() @@ -1221,6 +1225,73 @@ optimum[onnxruntime-gpu]==1.17.1 ; sys_platform == 'linux' _tests.append(_test_pipstar_platforms) +def _test_pipstar_platforms_limit(env): + builder = hub_builder( + env, + enable_pipstar = True, + config = struct( + enable_pipstar = True, + netrc = None, + auth_patterns = {}, + platforms = { + "my{}{}".format(os, cpu): _plat( + name = "my{}{}".format(os, cpu), + os_name = os, + arch_name = cpu, + marker = "python_version ~= \"3.13\"", + config_settings = [ + "@platforms//os:{}".format(os), + "@platforms//cpu:{}".format(cpu), + ], + ) + for os, cpu in [ + ("linux", "x86_64"), + ("osx", "aarch64"), + ] + }, + ), + ) + builder.pip_parse( + _mock_mctx( + os = "linux", + arch = "amd64", + read = lambda x: { + "universal.txt": """\ +optimum[onnxruntime]==1.17.1 ; sys_platform == 'darwin' +optimum[onnxruntime-gpu]==1.17.1 ; sys_platform == 'linux' +""", + }[x], + ), + _parse( + hub_name = "pypi", + python_version = "3.15", + requirements_lock = "universal.txt", + target_platforms = ["my{os}{arch}"], + ), + ) + pypi = builder.build() + + pypi.exposed_packages().contains_exactly(["optimum"]) + pypi.group_map().contains_exactly({}) + pypi.whl_map().contains_exactly({ + "optimum": { + "pypi_315_optimum": [ + whl_config_setting(version = "3.15"), + ], + }, + }) + pypi.whl_libraries().contains_exactly({ + "pypi_315_optimum": { + "config_load": "@pypi//:config.bzl", + "dep_template": "@pypi//{name}:{target}", + "python_interpreter_target": "unit_test_interpreter_target", + "requirement": "optimum[onnxruntime-gpu]==1.17.1", + }, + }) + pypi.extra_aliases().contains_exactly({}) + +_tests.append(_test_pipstar_platforms_limit) + def hub_builder_test_suite(name): """Create the test suite. diff --git a/tests/pypi/requirements_files_by_platform/requirements_files_by_platform_tests.bzl b/tests/pypi/requirements_files_by_platform/requirements_files_by_platform_tests.bzl index 6688d72ffe..d6aaf3ca99 100644 --- a/tests/pypi/requirements_files_by_platform/requirements_files_by_platform_tests.bzl +++ b/tests/pypi/requirements_files_by_platform/requirements_files_by_platform_tests.bzl @@ -115,6 +115,12 @@ def _test_simple_limited(env): }, platforms = ["linux_x86_64", "osx_x86_64"], ), + requirements_files_by_platform( + requirements_by_platform = { + "requirements_lock": "linux_x86_64,osx_aarch64,osx_x86_64", + }, + platforms = ["linux_x86_64", "osx_x86_64", "windows_x86_64"], + ), ]: env.expect.that_dict(got).contains_exactly({ "requirements_lock": [ @@ -219,6 +225,17 @@ def _test_os_arch_requirements_with_default(env): "requirements_linux": "linux_x86_64,linux_aarch64", }, requirements_lock = "requirements_lock", + platforms = [ + "linux_super_exotic", + "linux_x86_64", + "linux_aarch64", + "linux_arm", + "linux_ppc", + "linux_s390x", + "osx_aarch64", + "osx_x86_64", + "windows_x86_64", + ], ) env.expect.that_dict(got).contains_exactly({ "requirements_exotic": ["linux_super_exotic"], From 9559b2001ae3020bee303c412bc043f72090d476 Mon Sep 17 00:00:00 2001 From: Ignas Anikevicius <240938+aignas@users.noreply.github.com> Date: Sun, 7 Dec 2025 13:12:41 +0900 Subject: [PATCH 544/922] refactor(core): get_zip_runfiles_path should call startswith less (#3442) Looking at the investigation in #3380, it seems that we are calling the startswith many times and I wanted to see if it would be possible to optimize how it is done. I also realized that no matter what target we have, we will be calling the function once with a `__init__.py` path and we can inline this case as a separate if statement checking for equality instead, which Starlark optimizer should understand better. Before this PR for every executable target we would go through the `legacy_external_runfiles and "__init__.py".startswith("external")` and this PR eliminates this. Related to #3380 and #3381 --- python/private/py_executable.bzl | 40 ++++++++++++++++++-------------- 1 file changed, 23 insertions(+), 17 deletions(-) diff --git a/python/private/py_executable.bzl b/python/private/py_executable.bzl index 0a16e7690e..9084454c65 100644 --- a/python/private/py_executable.bzl +++ b/python/private/py_executable.bzl @@ -70,6 +70,7 @@ load(":venv_runfiles.bzl", "create_venv_app_files") _py_builtins = py_internal _EXTERNAL_PATH_PREFIX = "external" _ZIP_RUNFILES_DIRECTORY_NAME = "runfiles" +_INIT_PY = "__init__.py" _LAUNCHER_MAKER_TOOLCHAIN_TYPE = "@bazel_tools//tools/launcher:launcher_maker_toolchain_type" # Non-Google-specific attributes for executables @@ -834,14 +835,14 @@ def _create_zip_file(ctx, *, output, zip_main, runfiles): manifest.add("__main__.py={}".format(zip_main.path)) manifest.add("__init__.py=") manifest.add( - "{}=".format( - _get_zip_runfiles_path("__init__.py", workspace_name, legacy_external_runfiles), - ), + "{}=".format(_get_zip_runfiles_path(_INIT_PY, workspace_name)), ) def map_zip_empty_filenames(list_paths_cb): return [ - _get_zip_runfiles_path(path, workspace_name, legacy_external_runfiles) + "=" + # FIXME @aignas 2025-12-06: what kind of paths do we expect here? Will they + # ever start with `../` or `external`? + _get_zip_runfiles_path_legacy(path, workspace_name, legacy_external_runfiles) + "=" for path in list_paths_cb().to_list() ] @@ -856,7 +857,7 @@ def _create_zip_file(ctx, *, output, zip_main, runfiles): def map_zip_runfiles(file): return ( # NOTE: Use "+" for performance - _get_zip_runfiles_path(file.short_path, workspace_name, legacy_external_runfiles) + + _get_zip_runfiles_path_legacy(file.short_path, workspace_name, legacy_external_runfiles) + "=" + file.path ) @@ -893,23 +894,28 @@ def _create_zip_file(ctx, *, output, zip_main, runfiles): progress_message = "Building Python zip: %{label}", ) -def _get_zip_runfiles_path(path, workspace_name, legacy_external_runfiles): - maybe_workspace = "" - if legacy_external_runfiles and path.startswith(_EXTERNAL_PATH_PREFIX): - zip_runfiles_path = path.removeprefix(_EXTERNAL_PATH_PREFIX) +def _get_zip_runfiles_path(path, workspace_name = ""): + # NOTE @aignas 2025-12-06: This is to avoid the prefix checking in the very + # trivial case that is always happening once per this function call + + # NOTE: Use "+" for performance + if workspace_name: + # NOTE: Use "+" for performance + return _ZIP_RUNFILES_DIRECTORY_NAME + "/" + workspace_name + "/" + path else: + return _ZIP_RUNFILES_DIRECTORY_NAME + "/" + path + +def _get_zip_runfiles_path_legacy(path, workspace_name, legacy_external_runfiles): + if legacy_external_runfiles and path.startswith(_EXTERNAL_PATH_PREFIX): + return _get_zip_runfiles_path(path.removeprefix(_EXTERNAL_PATH_PREFIX)) + elif path.startswith("../"): # NOTE: External runfiles (artifacts in other repos) will have a leading # path component of "../" so that they refer outside the main workspace # directory and into the runfiles root. So we simplify it, e.g. # "workspace/../foo/bar" to simply "foo/bar". - if path.startswith("../"): - zip_runfiles_path = path[3:] - else: - zip_runfiles_path = path - maybe_workspace = workspace_name + "/" - - # NOTE: Use "+" for performance - return _ZIP_RUNFILES_DIRECTORY_NAME + "/" + maybe_workspace + zip_runfiles_path + return _get_zip_runfiles_path(path[3:]) + else: + return _get_zip_runfiles_path(path, workspace_name) def _create_executable_zip_file( ctx, From 2a2e136c2a4e393f5c84d90684680e01690278e3 Mon Sep 17 00:00:00 2001 From: Ignas Anikevicius <240938+aignas@users.noreply.github.com> Date: Sun, 7 Dec 2025 13:13:23 +0900 Subject: [PATCH 545/922] ci: start testing BCR tests with bazel 9 (#3443) It seems that the CI is passing now with the latest bazel 9 release. Hence, start testing it in our CI and when submitting new versions to BCR. Fix #3392 --- .bazelci/presubmit.yml | 3 ++- .bcr/gazelle/presubmit.yml | 5 +---- .bcr/presubmit.yml | 2 +- 3 files changed, 4 insertions(+), 6 deletions(-) diff --git a/.bazelci/presubmit.yml b/.bazelci/presubmit.yml index daa8f87ea3..6a5102d4f6 100644 --- a/.bazelci/presubmit.yml +++ b/.bazelci/presubmit.yml @@ -117,7 +117,7 @@ matrix: - debian11 - macos_arm64 - windows - bazel: [7.*, 8.*] + bazel: [7.*, 8.*, 9.*] tasks: # Keep in sync with .bcr/presubmit.yml @@ -135,6 +135,7 @@ tasks: test_targets: - "//..." + # Keep in sync with .bcr/gazelle/presubmit.yml gazelle_bcr_ubuntu: <<: *gazelle_common_bcr name: "Gazelle: BCR, Bazel {bazel}" diff --git a/.bcr/gazelle/presubmit.yml b/.bcr/gazelle/presubmit.yml index 7e35ce5648..99647cac6f 100644 --- a/.bcr/gazelle/presubmit.yml +++ b/.bcr/gazelle/presubmit.yml @@ -20,10 +20,7 @@ bcr_test_module: "macos", "ubuntu2204", ] - bazel: [ - 7.*, - 8.*, - ] + bazel: [7.*, 8.*, 9.*] tasks: run_tests: name: "Run test module" diff --git a/.bcr/presubmit.yml b/.bcr/presubmit.yml index 6217ad7ba2..1ad61c7f5a 100644 --- a/.bcr/presubmit.yml +++ b/.bcr/presubmit.yml @@ -16,7 +16,7 @@ bcr_test_module: module_path: "examples/bzlmod" matrix: platform: ["debian11", "macos", "ubuntu2204", "windows"] - bazel: [7.*, 8.*] + bazel: [7.*, 8.*, 9.*] tasks: run_tests: name: "Run test module" From 2828f69ebf23087b17fe0ebc8ee213f0da11af19 Mon Sep 17 00:00:00 2001 From: Ignas Anikevicius <240938+aignas@users.noreply.github.com> Date: Mon, 8 Dec 2025 07:45:25 +0900 Subject: [PATCH 546/922] doc: target_platforms (#3445) Followup to #3441 adding the documentation on how to use the new attribute. Work towards #2949 --------- Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> --- docs/pypi/download.md | 15 +++++++++++++++ 1 file changed, 15 insertions(+) diff --git a/docs/pypi/download.md b/docs/pypi/download.md index c40f2d4347..258f211301 100644 --- a/docs/pypi/download.md +++ b/docs/pypi/download.md @@ -113,6 +113,21 @@ This will not work for sdists with C extensions, but pure Python sdists may stil approach. ::: +By default, `rules_python` selects the host `{os}_{arch}` platform from its `MODULE.bazel` +file. This means that `rules_python` by default does not provide cross-platform building support +because some packages have very large wheels and users should be able to use `bazel query` with +minimal overhead. As a result, users should configure their `pip.parse` +calls and select which platforms they want to target via the +{attr}`pip.parse.target_platforms` attribute: +```starlark + # Example of enabling free threaded and non-freethreaded switching on the host platform: + target_platforms = ["{os}_{arch}", "{os}_{arch}_freethreaded"], + + # As another example, to enable building for `linux_x86_64` containers and the host platform: + # target_platforms = ["{os}_{arch}", "linux_x86_64"], +) +``` + ### Using `download_only` attribute Let's say you have two requirements files: From 89c8f204daeb588bfb34e142555b2a322a3a7126 Mon Sep 17 00:00:00 2001 From: Ignas Anikevicius <240938+aignas@users.noreply.github.com> Date: Wed, 10 Dec 2025 16:24:24 +0900 Subject: [PATCH 547/922] chore(bzlmod): assume that we can always mark the extension as reproducible (#3444) As part of the previous bazel 6 and bazel 7 support cleanup, we probably should have done this as well. All of the supported bazel versions allow us to mark the extension as reproducible. --- python/private/pypi/extension.bzl | 11 +++-------- 1 file changed, 3 insertions(+), 8 deletions(-) diff --git a/python/private/pypi/extension.bzl b/python/private/pypi/extension.bzl index eaa6c0d428..2a6d43f837 100644 --- a/python/private/pypi/extension.bzl +++ b/python/private/pypi/extension.bzl @@ -14,7 +14,6 @@ "pip module extension for use with bzlmod" -load("@bazel_features//:features.bzl", "bazel_features") load("@pythons_hub//:interpreters.bzl", "INTERPRETER_LABELS") load("@pythons_hub//:versions.bzl", "MINOR_MAPPING") load("@rules_python_internal//:rules_python_config.bzl", rp_config = "config") @@ -385,13 +384,9 @@ def _pip_impl(module_ctx): groups = mods.hub_group_map.get(hub_name), ) - if bazel_features.external_deps.extension_metadata_has_reproducible: - # NOTE @aignas 2025-04-15: this is set to be reproducible, because the - # results after calling the PyPI index should be reproducible on each - # machine. - return module_ctx.extension_metadata(reproducible = True) - else: - return None + return module_ctx.extension_metadata( + reproducible = True, + ) _default_attrs = { "arch_name": attr.string( From 41f91e96e5804e5fbe5fa83de965cd64219a8f28 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Wed, 10 Dec 2025 08:36:51 -0800 Subject: [PATCH 548/922] refactor: avoid conflict merging when shared libraries are present (#3448) Today, when a shared library is present, an extra VenvSymlinkEntry is generated so that it is linked directly. Unfortunately, this will always have a path overlap conflict with the rest of the venv symlinks, which triggers the conflict merge logic later in py_executable. That logic is expensive, as it must flatten all the files and then link each file individually (essentially doubling the number of files materialized). For large packages like torch (10k+ files), this can dramatically increase overhead. To fix, generate VenvSymlinkEntries that don't overlap. The basic logic for how this works is to identify paths that *must* be directly linked, marking all their parent directories as not being able to be directly linked, and then grouping what remains into the highest directly-linkable path. Along the way, drop the logic that only considers code files and special cases `__init__.py` files and implicit packages. This is simplify the code and more correctly map the extracted wheel into the venv. --- python/private/venv_runfiles.bzl | 199 +++++++++-------- .../app_files_building_tests.bzl | 206 ++++++++++++++++-- 2 files changed, 289 insertions(+), 116 deletions(-) diff --git a/python/private/venv_runfiles.bzl b/python/private/venv_runfiles.bzl index eeedda4555..43fcab6192 100644 --- a/python/private/venv_runfiles.bzl +++ b/python/private/venv_runfiles.bzl @@ -3,7 +3,6 @@ load("@bazel_skylib//lib:paths.bzl", "paths") load( ":common.bzl", - "PYTHON_FILE_EXTENSIONS", "is_file", "relative_path", "runfiles_root_path", @@ -58,7 +57,6 @@ def create_venv_app_files(ctx, deps, venv_dir_map): if is_file(link_to): symlink_from = "{}/{}".format(ctx.label.package, bin_venv_path) runfiles_symlinks[symlink_from] = link_to - else: venv_link = ctx.actions.declare_symlink(bin_venv_path) venv_link_rf_path = runfiles_root_path(ctx, venv_link.short_path) @@ -77,14 +75,16 @@ def create_venv_app_files(ctx, deps, venv_dir_map): ) # Visible for testing -def build_link_map(ctx, entries): +def build_link_map(ctx, entries, return_conflicts = False): """Compute the mapping of venv paths to their backing objects. - Args: ctx: {type}`ctx` current ctx. entries: {type}`list[VenvSymlinkEntry]` the entries that describe the venv-relative + return_conflicts: {type}`bool`. Only present for testing. If True, + also return a list of the groups that had overlapping paths and had + to be resolved and merged. Returns: {type}`dict[str, dict[str, str|File]]` Mappings of venv paths to their @@ -114,6 +114,7 @@ def build_link_map(ctx, entries): # final paths to keep, grouped by kind keep_link_map = {} # dict[str kind, dict[path, str|File]] + conflicts = [] if return_conflicts else None for kind, entries in entries_by_kind.items(): # dict[str kind-relative path, str|File link_to] keep_kind_link_map = {} @@ -129,12 +130,17 @@ def build_link_map(ctx, entries): else: keep_kind_link_map[entry.venv_path] = entry.link_to_path else: + if return_conflicts: + conflicts.append(group) + # Merge a group of overlapping prefixes _merge_venv_path_group(ctx, group, keep_kind_link_map) keep_link_map[kind] = keep_kind_link_map - - return keep_link_map + if return_conflicts: + return keep_link_map, conflicts + else: + return keep_link_map def _group_venv_path_entries(entries): """Group entries by VenvSymlinkEntry.venv_path overlap. @@ -235,109 +241,115 @@ def get_venv_symlinks(ctx, files, package, version_str, site_packages_root): # Append slash to prevent incorrect prefix-string matches site_packages_root += "/" - # We have to build a list of (runfiles path, site-packages path) pairs of the files to - # create in the consuming binary's venv site-packages directory. To minimize the number of - # files to create, we just return the paths to the directories containing the code of - # interest. - # - # However, namespace packages complicate matters: multiple distributions install in the - # same directory in site-packages. This works out because they don't overlap in their - # files. Typically, they install to different directories within the namespace package - # directory. We also need to ensure that we can handle a case where the main package (e.g. - # airflow) has directories only containing data files and then namespace packages coming - # along and being next to it. - # - # Lastly we have to assume python modules just being `.py` files (e.g. typing-extensions) - # is just a single Python file. + all_files = sorted(files, key = lambda f: f.short_path) - dir_symlinks = {} # dirname -> runfile path - venv_symlinks = [] + # venv paths that cannot be directly linked. Dict acting as set. + cannot_be_linked_directly = {} - # Sort so order is top-down - all_files = sorted(files, key = lambda f: f.short_path) + # dict[str path, VenvSymlinkEntry] + # Where path is the venv path (i.e. relative to site_packages_prefix) + venv_symlinks = {} + # List of (File, str venv_path) tuples + files_left_to_link = [] + + # We want to minimize the number of files symlinked. Ideally, only the + # top-level directories are symlinked. Unfortunately, shared libraries + # complicate matters: if a shared library's directory is linked, then the + # dynamic linker computes the wrong search path. + # + # To fix, we have to directly link shared libraries. This then means that + # all the parent directories of the shared library can't be linked + # directly. for src in all_files: - path = _repo_relative_short_path(src.short_path) - if not path.startswith(site_packages_root): + rf_root_path = runfiles_root_path(ctx, src.short_path) + _, _, repo_rel_path = rf_root_path.partition("/") + head, found_sp_root, venv_path = repo_rel_path.partition(site_packages_root) + if head or not found_sp_root: + # If head is set, then the path didn't start with site_packages_root + # if found_sp_root is empty, then it means it wasn't found at all. continue - path = path.removeprefix(site_packages_root) - dir_name, _, filename = path.rpartition("/") - runfiles_dir_name, _, _ = runfiles_root_path(ctx, src.short_path).partition("/") + filename = paths.basename(venv_path) if _is_linker_loaded_library(filename): - entry = VenvSymlinkEntry( + venv_symlinks[venv_path] = VenvSymlinkEntry( kind = VenvSymlinkKind.LIB, - link_to_path = paths.join(runfiles_dir_name, site_packages_root, filename), + link_to_path = rf_root_path, link_to_file = src, package = package, version = version_str, - venv_path = path, files = depset([src]), + venv_path = venv_path, ) - venv_symlinks.append(entry) - continue - - if dir_name in dir_symlinks: - # we already have this dir, this allows us to short-circuit since most of the - # ctx.files.data might share the same directories as ctx.files.srcs + parent = paths.dirname(venv_path) + for _ in range(len(venv_path) + 1): # Iterate enough times to traverse up + if not parent: + break + if cannot_be_linked_directly.get(parent, False): + # Already seen + break + cannot_be_linked_directly[parent] = True + parent = paths.dirname(parent) + else: + files_left_to_link.append((src, venv_path)) + + # At this point, venv_symlinks has entries for the shared libraries + # and cannot_be_linked_directly has the directories that cannot be + # directly linked. Next, we loop over the remaining files and group + # them into the highest level directory that can be linked. + + # dict[str venv_path, list[File]] + optimized_groups = {} + + for src, venv_path in files_left_to_link: + parent = paths.dirname(venv_path) + if not parent: + # File in root, must be linked directly + optimized_groups.setdefault(venv_path, []) + optimized_groups[venv_path].append(src) continue - if dir_name: - # This can be either: - # * a directory with libs (e.g. numpy.libs, created by auditwheel) - # * a directory with `__init__.py` file that potentially also needs to be - # symlinked. - # * `.dist-info` directory - # - # This could be also regular files, that just need to be symlinked, so we will - # add the directory here. - dir_symlinks[dir_name] = runfiles_dir_name - elif src.extension in PYTHON_FILE_EXTENSIONS: - # This would be files that do not have directories and we just need to add - # direct symlinks to them as is, we only allow Python files in here - entry = VenvSymlinkEntry( - kind = VenvSymlinkKind.LIB, - link_to_path = paths.join(runfiles_dir_name, site_packages_root, filename), - link_to_file = src, - package = package, - version = version_str, - venv_path = path, - files = depset([src]), - ) - venv_symlinks.append(entry) - - # Sort so that we encounter `foo` before `foo/bar`. This ensures we - # see the top-most explicit package first. - dirnames = sorted(dir_symlinks.keys()) - first_level_explicit_packages = [] - for d in dirnames: - is_sub_package = False - for existing in first_level_explicit_packages: - # Suffix with / to prevent foo matching foobar - if d.startswith(existing + "/"): - is_sub_package = True - break - if not is_sub_package: - first_level_explicit_packages.append(d) - - for dirname in first_level_explicit_packages: - prefix = dir_symlinks[dirname] - link_to_path = paths.join(prefix, site_packages_root, dirname) - entry = VenvSymlinkEntry( + if parent in cannot_be_linked_directly: + # File in a directory that cannot be directly linked, + # so link the file directly + optimized_groups.setdefault(venv_path, []) + optimized_groups[venv_path].append(src) + else: + # This path can be grouped. Find the highest-level directory to link. + venv_path = parent + next_parent = paths.dirname(parent) + for _ in range(len(venv_path) + 1): # Iterate enough times + if next_parent: + if next_parent not in cannot_be_linked_directly: + venv_path = next_parent + next_parent = paths.dirname(next_parent) + else: + break + else: + break + + optimized_groups.setdefault(venv_path, []) + optimized_groups[venv_path].append(src) + + # Finally, for each group, we create the VenvSymlinkEntry objects + for venv_path, files in optimized_groups.items(): + link_to_path = ( + _get_label_runfiles_repo(ctx, files[0].owner) + + "/" + + site_packages_root + + venv_path + ) + venv_symlinks[venv_path] = VenvSymlinkEntry( kind = VenvSymlinkKind.LIB, link_to_path = link_to_path, + link_to_file = None, package = package, version = version_str, - venv_path = dirname, - files = depset([ - f - for f in all_files - if runfiles_root_path(ctx, f.short_path).startswith(link_to_path + "/") - ]), + venv_path = venv_path, + files = depset(files), ) - venv_symlinks.append(entry) - return venv_symlinks + return venv_symlinks.values() def _is_linker_loaded_library(filename): """Tells if a filename is one that `dlopen()` or the runtime linker handles. @@ -357,9 +369,10 @@ def _is_linker_loaded_library(filename): return True return False -def _repo_relative_short_path(short_path): - # Convert `../+pypi+foo/some/file.py` to `some/file.py` - if short_path.startswith("../"): - return short_path[3:].partition("/")[2] +def _get_label_runfiles_repo(ctx, label): + repo = label.repo_name + if repo: + return repo else: - return short_path + # For files, empty repo means the main repo + return ctx.workspace_name diff --git a/tests/venv_site_packages_libs/app_files_building/app_files_building_tests.bzl b/tests/venv_site_packages_libs/app_files_building/app_files_building_tests.bzl index db2f21c7e7..fc0b5d0bf3 100644 --- a/tests/venv_site_packages_libs/app_files_building/app_files_building_tests.bzl +++ b/tests/venv_site_packages_libs/app_files_building/app_files_building_tests.bzl @@ -26,6 +26,11 @@ empty_files = rule( _tests = [] +# NOTE: In bzlmod, the workspace name is always "_main". +# Under workspace, the workspace name is the name configured in WORKSPACE, +# or "__main__" if was unspecified. +# NOTE: ctx.workspace_name is always the root workspace, not the workspace +# of the target being processed (ctx.label). def _ctx(workspace_name = "_main"): return struct( workspace_name = workspace_name, @@ -36,6 +41,23 @@ def _file(short_path): short_path = short_path, ) +def _venv_symlink(venv_path, *, link_to_path = None, files = []): + return struct( + link_to_path = link_to_path, + venv_path = venv_path, + files = files, + ) + +def _venv_symlinks_from_entries(entries): + result = [] + for symlink_entry in entries: + result.append(struct( + venv_path = symlink_entry.venv_path, + link_to_path = symlink_entry.link_to_path, + files = [f.short_path for f in symlink_entry.files.to_list()], + )) + return sorted(result, key = lambda e: (e.link_to_path, e.venv_path)) + def _entry(venv_path, link_to_path, files = [], **kwargs): kwargs.setdefault("kind", VenvSymlinkKind.LIB) kwargs.setdefault("package", None) @@ -89,7 +111,7 @@ def _test_conflict_merging_impl(env, _): _entry("duplicate", "+dupe_b/site-packages/duplicate", ["d.py"]), ] - actual = build_link_map(_ctx(), entries) + actual, conflicts = build_link_map(_ctx(), entries, return_conflicts = True) expected_libs = { "a-1.0.dist-info": "+pypi_a/site-packages/a-1.0.dist-info", "a/a.txt": _file("../+pypi_a/site-packages/a/a.txt"), @@ -101,6 +123,146 @@ def _test_conflict_merging_impl(env, _): env.expect.that_dict(actual[VenvSymlinkKind.LIB]).contains_exactly(expected_libs) env.expect.that_dict(actual).keys().contains_exactly([VenvSymlinkKind.LIB]) + env.expect.that_int(len(conflicts)).is_greater_than(0) + +def _test_optimized_grouping_complex(name): + empty_files( + name = name + "_files", + paths = [ + "site-packages/pkg1/a.txt", + "site-packages/pkg1/b/b_mod.so", + "site-packages/pkg1/c/c1.txt", + "site-packages/pkg1/c/c2.txt", + "site-packages/pkg1/d/d1.txt", + "site-packages/pkg1/dd/dd1.txt", + "site-packages/pkg1/q1/q1.txt", + "site-packages/pkg1/q1/q2a/libq.so", + "site-packages/pkg1/q1/q2a/q2.txt", + "site-packages/pkg1/q1/q2a/q3/q3a.txt", + "site-packages/pkg1/q1/q2a/q3/q3b.txt", + "site-packages/pkg1/q1/q2b/q2b.txt", + ], + ) + analysis_test( + name = name, + impl = _test_optimized_grouping_complex_impl, + target = name + "_files", + ) + +_tests.append(_test_optimized_grouping_complex) + +def _test_optimized_grouping_complex_impl(env, target): + test_ctx = _ctx(workspace_name = env.ctx.workspace_name) + entries = get_venv_symlinks( + test_ctx, + target.files.to_list(), + package = "pkg1", + version_str = "1.0", + site_packages_root = env.ctx.label.package + "/site-packages", + ) + actual = _venv_symlinks_from_entries(entries) + + rr = "{}/{}/site-packages/".format(test_ctx.workspace_name, env.ctx.label.package) + expected = [ + _venv_symlink( + "pkg1/a.txt", + link_to_path = rr + "pkg1/a.txt", + files = [ + "tests/venv_site_packages_libs/app_files_building/site-packages/pkg1/a.txt", + ], + ), + _venv_symlink( + "pkg1/b", + link_to_path = rr + "pkg1/b", + files = [ + "tests/venv_site_packages_libs/app_files_building/site-packages/pkg1/b/b_mod.so", + ], + ), + _venv_symlink("pkg1/c", link_to_path = rr + "pkg1/c", files = [ + "tests/venv_site_packages_libs/app_files_building/site-packages/pkg1/c/c1.txt", + "tests/venv_site_packages_libs/app_files_building/site-packages/pkg1/c/c2.txt", + ]), + _venv_symlink("pkg1/d", link_to_path = rr + "pkg1/d", files = [ + "tests/venv_site_packages_libs/app_files_building/site-packages/pkg1/d/d1.txt", + ]), + _venv_symlink("pkg1/dd", link_to_path = rr + "pkg1/dd", files = [ + "tests/venv_site_packages_libs/app_files_building/site-packages/pkg1/dd/dd1.txt", + ]), + _venv_symlink("pkg1/q1/q1.txt", link_to_path = rr + "pkg1/q1/q1.txt", files = [ + "tests/venv_site_packages_libs/app_files_building/site-packages/pkg1/q1/q1.txt", + ]), + _venv_symlink("pkg1/q1/q2a/libq.so", link_to_path = rr + "pkg1/q1/q2a/libq.so", files = [ + "tests/venv_site_packages_libs/app_files_building/site-packages/pkg1/q1/q2a/libq.so", + ]), + _venv_symlink("pkg1/q1/q2a/q2.txt", link_to_path = rr + "pkg1/q1/q2a/q2.txt", files = [ + "tests/venv_site_packages_libs/app_files_building/site-packages/pkg1/q1/q2a/q2.txt", + ]), + _venv_symlink("pkg1/q1/q2a/q3", link_to_path = rr + "pkg1/q1/q2a/q3", files = [ + "tests/venv_site_packages_libs/app_files_building/site-packages/pkg1/q1/q2a/q3/q3a.txt", + "tests/venv_site_packages_libs/app_files_building/site-packages/pkg1/q1/q2a/q3/q3b.txt", + ]), + _venv_symlink("pkg1/q1/q2b", link_to_path = rr + "pkg1/q1/q2b", files = [ + "tests/venv_site_packages_libs/app_files_building/site-packages/pkg1/q1/q2b/q2b.txt", + ]), + ] + expected = sorted(expected, key = lambda e: (e.link_to_path, e.venv_path)) + env.expect.that_collection( + actual, + ).contains_exactly(expected) + _, conflicts = build_link_map(test_ctx, entries, return_conflicts = True) + + # The point of the optimization is to avoid having to merge conflicts. + env.expect.that_collection(conflicts).contains_exactly([]) + +def _test_optimized_grouping_single_toplevel(name): + empty_files( + name = name + "_files", + paths = [ + "site-packages/pkg2/a.txt", + "site-packages/pkg2/b_mod.so", + ], + ) + analysis_test( + name = name, + impl = _test_optimized_grouping_single_toplevel_impl, + target = name + "_files", + ) + +_tests.append(_test_optimized_grouping_single_toplevel) + +def _test_optimized_grouping_single_toplevel_impl(env, target): + test_ctx = _ctx(workspace_name = env.ctx.workspace_name) + entries = get_venv_symlinks( + test_ctx, + target.files.to_list(), + package = "pkg2", + version_str = "1.0", + site_packages_root = env.ctx.label.package + "/site-packages", + ) + actual = _venv_symlinks_from_entries(entries) + + rr = "{}/{}/site-packages/".format(test_ctx.workspace_name, env.ctx.label.package) + expected = [ + _venv_symlink( + "pkg2", + link_to_path = rr + "pkg2", + files = [ + "tests/venv_site_packages_libs/app_files_building/site-packages/pkg2/a.txt", + "tests/venv_site_packages_libs/app_files_building/site-packages/pkg2/b_mod.so", + ], + ), + ] + expected = sorted(expected, key = lambda e: (e.link_to_path, e.venv_path)) + + env.expect.that_collection( + actual, + ).contains_exactly(expected) + + _, conflicts = build_link_map(test_ctx, entries, return_conflicts = True) + + # The point of the optimization is to avoid having to merge conflicts. + env.expect.that_collection(conflicts).contains_exactly([]) + def _test_package_version_filtering(name): analysis_test( name = name, @@ -219,8 +381,7 @@ def _test_multiple_venv_symlink_kinds_impl(env, _): "libfile", "+pypi_lib/site-packages/libfile", ["lib.txt"], - kind = - VenvSymlinkKind.LIB, + kind = VenvSymlinkKind.LIB, ), _entry( "binfile", @@ -294,34 +455,33 @@ def _test_shared_library_symlinking_impl(env, target): site_packages_root = env.ctx.label.package + "/site-packages", ) - actual = [e for e in actual_entries if e.venv_path == "foo.libs/libx.so"] - if not actual: - fail("Did not find VenvSymlinkEntry with venv_path equal to foo.libs/libx.so. " + - "Found: {}".format(actual_entries)) - elif len(actual) > 1: - fail("Found multiple entries with venv_path=foo.libs/libx.so. " + - "Found: {}".format(actual_entries)) - actual = actual[0] + actual = _venv_symlinks_from_entries(actual_entries) - actual_files = actual.files.to_list() - expected_lib_dso = [f for f in srcs if f.basename == "libx.so"] - env.expect.that_collection(actual_files).contains_exactly(expected_lib_dso) + env.expect.that_collection(actual).contains_at_least([ + _venv_symlink( + "bar/libs/liby.so", + link_to_path = "_main/tests/venv_site_packages_libs/app_files_building/site-packages/bar/libs/liby.so", + files = [ + "tests/venv_site_packages_libs/app_files_building/site-packages/bar/libs/liby.so", + ], + ), + _venv_symlink( + "foo.libs/libx.so", + link_to_path = "_main/tests/venv_site_packages_libs/app_files_building/site-packages/foo.libs/libx.so", + files = [ + "tests/venv_site_packages_libs/app_files_building/site-packages/foo.libs/libx.so", + ], + ), + ]) - entries = actual_entries - actual = build_link_map(_ctx(), entries) + actual = build_link_map(_ctx(), actual_entries) # The important condition is that each lib*.so file is linked directly. expected_libs = { "bar/libs/liby.so": srcs[0], - "bar/x.py": srcs[1], - "bar/y.so": srcs[2], - "foo": "_main/tests/venv_site_packages_libs/app_files_building/site-packages/foo", "foo.libs/libx.so": srcs[3], - "root.pth": srcs[-3], - "root.py": srcs[-2], - "root.so": srcs[-1], } - env.expect.that_dict(actual[VenvSymlinkKind.LIB]).contains_exactly(expected_libs) + env.expect.that_dict(actual[VenvSymlinkKind.LIB]).contains_at_least(expected_libs) def app_files_building_test_suite(name): test_suite( From d3ff1d12421c91c64236a08f6f6641ddf11bd131 Mon Sep 17 00:00:00 2001 From: Tyler French Date: Wed, 10 Dec 2025 12:03:14 -0500 Subject: [PATCH 549/922] refactor: add mnemonics to some gazelle and sphinxdocs actions (#3449) Having mnemonics allows actions to be better identified in output, logs, selected by flags, etc --------- Co-authored-by: Richard Levasseur --- examples/wheel/private/wheel_utils.bzl | 1 + gazelle/manifest/defs.bzl | 1 + gazelle/modules_mapping/def.bzl | 2 ++ sphinxdocs/private/sphinx.bzl | 1 + 4 files changed, 5 insertions(+) diff --git a/examples/wheel/private/wheel_utils.bzl b/examples/wheel/private/wheel_utils.bzl index 037fed0175..4cd776a98a 100644 --- a/examples/wheel/private/wheel_utils.bzl +++ b/examples/wheel/private/wheel_utils.bzl @@ -28,6 +28,7 @@ def _directory_writer_impl(ctx): ctx.actions.run( outputs = [output], + mnemonic = "PyDirWriter", arguments = [args], executable = ctx.executable._writer, ) diff --git a/gazelle/manifest/defs.bzl b/gazelle/manifest/defs.bzl index b615c4efc1..71a4d57156 100644 --- a/gazelle/manifest/defs.bzl +++ b/gazelle/manifest/defs.bzl @@ -184,6 +184,7 @@ def _sources_hash_impl(ctx): args.add_all(all_srcs) ctx.actions.run( outputs = [hash_file], + mnemonic = "PyGazelleManifestHash", inputs = all_srcs, arguments = [args], executable = ctx.executable._hasher, diff --git a/gazelle/modules_mapping/def.bzl b/gazelle/modules_mapping/def.bzl index 74d3c9ef35..8466edd39a 100644 --- a/gazelle/modules_mapping/def.bzl +++ b/gazelle/modules_mapping/def.bzl @@ -43,6 +43,7 @@ def _modules_mapping_impl(ctx): ctx.actions.run( inputs = [whl], + mnemonic = "PyGazelleModMapGen", outputs = [wheel_modules_mapping], executable = ctx.executable._generator, arguments = [args], @@ -57,6 +58,7 @@ def _modules_mapping_impl(ctx): ctx.actions.run( inputs = per_wheel_outputs, + mnemonic = "PyGazelleModMapMerge", outputs = [modules_mapping], executable = ctx.executable._merger, arguments = [merge_args], diff --git a/sphinxdocs/private/sphinx.bzl b/sphinxdocs/private/sphinx.bzl index c1efda3508..e444429233 100644 --- a/sphinxdocs/private/sphinx.bzl +++ b/sphinxdocs/private/sphinx.bzl @@ -498,6 +498,7 @@ def _sphinx_inventory_impl(ctx): args.add(output) ctx.actions.run( executable = ctx.executable._builder, + mnemonic = "SphinxInventoryBuilder", arguments = [args], inputs = depset([ctx.file.src]), outputs = [output], From b3d9b4207d558e84ab768739bf981035924962b4 Mon Sep 17 00:00:00 2001 From: armandomontanez Date: Wed, 10 Dec 2025 21:57:40 -0800 Subject: [PATCH 550/922] fix(pip): Only directly extract .whl files in Bazel >9 (#3452) Directly extracting .whl files is only supported in Bazel >=8.3.0 (see https://github.com/bazelbuild/bazel/pull/26323). This change applies the workaround for Bazel <8.0.0 to all versions <9.0.0 for broader compatibility. Work towards #2948 --- python/private/pypi/whl_library.bzl | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/python/private/pypi/whl_library.bzl b/python/private/pypi/whl_library.bzl index 6b515a56a4..d8f32ffef1 100644 --- a/python/private/pypi/whl_library.bzl +++ b/python/private/pypi/whl_library.bzl @@ -377,7 +377,10 @@ def _whl_library_impl(rctx): # # Remove non-pipstar and config_load check when we release rules_python 2. if enable_pipstar: - if rp_config.bazel_8_or_later: + # Extracting .whl files requires Bazel 8.3.0 or later, so require a + # minimum of Bazel 9.0.0 to ensure compatibilty with earlier versions + # of Bazel 8. + if rp_config.bazel_9_or_later: extract_path = whl_path else: extract_path = rctx.path(whl_path.basename + ".zip") @@ -386,7 +389,7 @@ def _whl_library_impl(rctx): archive = extract_path, output = "site-packages", ) - if not rp_config.bazel_8_or_later: + if not rp_config.bazel_9_or_later: rctx.delete(extract_path) metadata = whl_metadata( From 7223eb393cc46f7d7c054df6c19c5d27d10ee1c2 Mon Sep 17 00:00:00 2001 From: peter woodman Date: Sat, 13 Dec 2025 21:21:27 -0500 Subject: [PATCH 551/922] feat(toolchains): Add 3.13.10, 3.13.11, 3.14.1, 3.14.2, 3.15.0a2 (#3451) --- CHANGELOG.md | 5 + python/versions.bzl | 231 +++++++++++++++++++++++++++++++++++++++++++- 2 files changed, 233 insertions(+), 3 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index a41ac20103..87ae3f5ac4 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -108,6 +108,8 @@ END_UNRELEASED_TEMPLATE {#v0-0-0-added} ### Added * (toolchains) `3.9.25` Python toolchain from [20251031] release. +* (toolchains) `3.13.10`, `3.14.1` Python toolchain from [20251202] release. +* (toolchains) `3.13.11`, `3.14.2`, `3.15.0a2` Python toolchains from [20251209] release. * (pypi) API to tell `pip.parse` which platforms users care about. This is very useful to ensure that when users do `bazel query` for their deps, they don't have to download all of the dependencies for all of the available wheels. Torch wheels can be up of 1GB and it takes a lot @@ -118,6 +120,9 @@ END_UNRELEASED_TEMPLATE become mandatory if any cross-builds are required from the next release. [20251031]: https://github.com/astral-sh/python-build-standalone/releases/tag/20251031 +[20251202]: https://github.com/astral-sh/python-build-standalone/releases/tag/20251202 +[20251209]: https://github.com/astral-sh/python-build-standalone/releases/tag/20251209 + {#v1-7-0} ## [1.7.0] - 2025-10-11 diff --git a/python/versions.bzl b/python/versions.bzl index 842fb39658..3299a9f16b 100644 --- a/python/versions.bzl +++ b/python/versions.bzl @@ -767,6 +767,96 @@ TOOL_VERSIONS = { "x86_64-unknown-linux-gnu-freethreaded": "python/install", }, }, + "3.13.10": { + "url": "20251202/cpython-{python_version}+20251202-{platform}-{build}.{ext}", + "sha256": { + "aarch64-apple-darwin": "37afe4e77ab62ac50f197b1cb1f3bc02c82735c6be893da0996afcde5dc41048", + "aarch64-unknown-linux-gnu": "c68280591cda1c9515a04809fa6926020177e8e5892300206e0496ea1d10290e", + "ppc64le-unknown-linux-gnu": "1507e5528bd88131dc742a2941176aceea1838bc09860c21f179285b7865133b", + "riscv64-unknown-linux-gnu": "70169e916860b2e5b34c37c302d699eb2b8f24f28090968881942a37aeb7ed08", + "s390x-unknown-linux-gnu": "c5448863b64aacae62f3a213a6e6cf94ec63f96ee4d518491cd62fd3c81d952f", + "x86_64-apple-darwin": "a02761a4f189f71c0512e88df7ca2843696d61da659e47f8a5c8a9bd2c0d16f4", + "x86_64-pc-windows-msvc": "8b00014c7c35f9ad4cb1c565f067500bacc4125c8bc30e4389ee0be9fd6ffa3d", + "aarch64-pc-windows-msvc": "9060d644bd32ac0e0af970d0b21e207e6ff416b7c4dc26ffc4f9b043fb45b463", + "aarch64-pc-windows-msvc-freethreaded": "cdb7141327bdc244715b25752593e2c9eeb3cc2764f37dfe81cfbc92db9d6d57", + "x86_64-unknown-linux-gnu": "0cac1495fff920219904b1d573aaec0df54d549c226cb45f5c60cb6d2c72727a", + "x86_64-unknown-linux-musl": "04108190972ac98e13098abd972ec3f4f8b0880f83c0bb68249ce1a6164fa041", + "aarch64-apple-darwin-freethreaded": "3c9fdd76447c1549a0d3bc2a70c63f1daec997ab034206ac0260a03237166dbb", + "aarch64-unknown-linux-gnu-freethreaded": "6d277221fa4b172e00b29c7158ca9661917bc8db9a0084b1a0ff5c3a0ba8b648", + "ppc64le-unknown-linux-gnu-freethreaded": "d265d8d1c51e25ed70279540223589f79cf99ad00b50d28b6150c2658c973885", + "riscv64-unknown-linux-gnu-freethreaded": "ec411b4a2d167c3be0a9aeb3905e045d62c8e3c3db0caeade5d47d5f60b98dd0", + "s390x-unknown-linux-gnu-freethreaded": "4fc6443948bf5b729481ea02cc5c68e80cd0da42631f6936587a2b8fd45bc62c", + "x86_64-apple-darwin-freethreaded": "6ce608684df0f90350c7a1742e9685a7782d9b26ec99d1bd9d55c8cf9a405040", + "x86_64-pc-windows-msvc-freethreaded": "6a8b0372ded655e0d55318089fbce3122a446e69bcd120c79aaadfe9b017299c", + "x86_64-unknown-linux-gnu-freethreaded": "e39127fbe8d2ae7d86099f18b4da0918f9b60ce73ed491774d6dcfaa42b5c9ae", + }, + "strip_prefix": { + "aarch64-apple-darwin": "python", + "aarch64-unknown-linux-gnu": "python", + "ppc64le-unknown-linux-gnu": "python", + "s390x-unknown-linux-gnu": "python", + "riscv64-unknown-linux-gnu": "python", + "x86_64-apple-darwin": "python", + "x86_64-pc-windows-msvc": "python", + "aarch64-pc-windows-msvc": "python", + "x86_64-unknown-linux-gnu": "python", + "x86_64-unknown-linux-musl": "python", + "aarch64-apple-darwin-freethreaded": "python/install", + "aarch64-unknown-linux-gnu-freethreaded": "python/install", + "ppc64le-unknown-linux-gnu-freethreaded": "python/install", + "riscv64-unknown-linux-gnu-freethreaded": "python/install", + "s390x-unknown-linux-gnu-freethreaded": "python/install", + "x86_64-apple-darwin-freethreaded": "python/install", + "x86_64-pc-windows-msvc-freethreaded": "python/install", + "aarch64-pc-windows-msvc-freethreaded": "python/install", + "x86_64-unknown-linux-gnu-freethreaded": "python/install", + }, + }, + "3.13.11": { + "url": "20251209/cpython-{python_version}+20251209-{platform}-{build}.{ext}", + "sha256": { + "aarch64-apple-darwin": "295a9f7bc899ea1cc08baf60bbf511bdd1e4a29b2dd7e5f59b48f18bfa6bf585", + "aarch64-unknown-linux-gnu": "ea1e678e6e82301bb32bf3917732125949b6e46d541504465972024a3f165343", + "ppc64le-unknown-linux-gnu": "7660e53aad9d35ee256913c6d98427f81f078699962035c5fa8b5c3138695109", + "riscv64-unknown-linux-gnu": "763fa1548e6a432e9402916e690c74ea30f26dcd2e131893dd506f72b87c27c9", + "s390x-unknown-linux-gnu": "ffb6af51fbfabfc6fbc4e7379bdec70c2f51e972b1d2f45c053493b9da3a1bbe", + "x86_64-apple-darwin": "dac4a0a0a9b71f6b02a8b0886547fa22814474239bffb948e3e77185406ea136", + "x86_64-pc-windows-msvc": "87822417007045a28a7eccc47fe67b8c61265b99b10dbbfa24d231a3622b1c27", + "aarch64-pc-windows-msvc": "ba646d0c3b7dd7bdfb770d9b2ebd6cd2df02a37fda90c9c79a7cf59c7df6f165", + "aarch64-pc-windows-msvc-freethreaded": "6daf6d092c7294cfe68c4c7bf2698ac134235489c874b3bf796c7972b9dbba30", + "x86_64-unknown-linux-gnu": "1ffa06d714a44aea14c0c54c30656413e5955a6c92074b4b3cb4351dcc28b63b", + "x86_64-unknown-linux-musl": "969fe24017380b987c4e3ce15e9edf82a4618c1e61672b2cc9b021a1c98eae78", + "aarch64-apple-darwin-freethreaded": "4213058b7fcd875596c12b58cd46a399358b0a87ecde4b349cbdd00cf87ed79a", + "aarch64-unknown-linux-gnu-freethreaded": "290ca3bd0007db9e551f90b08dfcb6c1b2d62c33b2fc3e9a43e77d385d94f569", + "ppc64le-unknown-linux-gnu-freethreaded": "09d4b50f8abb443f7e3af858c920aa61c2430b0954df465e861caa7078e55e69", + "riscv64-unknown-linux-gnu-freethreaded": "5406f2a7cacafbd2aac3ce2de066a0929aab55423824276c36e04cb83babc36c", + "s390x-unknown-linux-gnu-freethreaded": "3984b67c4292892eaccdd1c094c7ec788884c4c9b3534ab6995f6be96d5ed51d", + "x86_64-apple-darwin-freethreaded": "d6f489464045d6895ae68b0a04a9e16477e74fe3185a75f3a9a0af8ccd25eade", + "x86_64-pc-windows-msvc-freethreaded": "bb9a29a7ba8f179273b79971da6aaa7be592d78c606a63f99eff3e4c12fb0fae", + "x86_64-unknown-linux-gnu-freethreaded": "33f89c957d986d525529b8a980103735776f4d20cf52f55960a057c760188ac3", + }, + "strip_prefix": { + "aarch64-apple-darwin": "python", + "aarch64-unknown-linux-gnu": "python", + "ppc64le-unknown-linux-gnu": "python", + "s390x-unknown-linux-gnu": "python", + "riscv64-unknown-linux-gnu": "python", + "x86_64-apple-darwin": "python", + "x86_64-pc-windows-msvc": "python", + "aarch64-pc-windows-msvc": "python", + "x86_64-unknown-linux-gnu": "python", + "x86_64-unknown-linux-musl": "python", + "aarch64-apple-darwin-freethreaded": "python/install", + "aarch64-unknown-linux-gnu-freethreaded": "python/install", + "ppc64le-unknown-linux-gnu-freethreaded": "python/install", + "riscv64-unknown-linux-gnu-freethreaded": "python/install", + "s390x-unknown-linux-gnu-freethreaded": "python/install", + "x86_64-apple-darwin-freethreaded": "python/install", + "x86_64-pc-windows-msvc-freethreaded": "python/install", + "aarch64-pc-windows-msvc-freethreaded": "python/install", + "x86_64-unknown-linux-gnu-freethreaded": "python/install", + }, + }, "3.14.0": { "url": "20251031/cpython-{python_version}+20251031-{platform}-{build}.{ext}", "sha256": { @@ -812,6 +902,96 @@ TOOL_VERSIONS = { "x86_64-unknown-linux-gnu-freethreaded": "python/install", }, }, + "3.14.1": { + "url": "20251202/cpython-{python_version}+20251202-{platform}-{build}.{ext}", + "sha256": { + "aarch64-apple-darwin": "cdf1ba0789f529fa34bb5b5619c5da9757ac1067d6b8dd0ee8b78e50078fc561", + "aarch64-unknown-linux-gnu": "5dde7dba0b8ef34c0d5cb8a721254b1e11028bfc09ff06664879c245fe8df73f", + "ppc64le-unknown-linux-gnu": "d2774701d53e2ac06f8c8c8e52dfa4ff346890de9b417c9a7664195443a4c766", + "riscv64-unknown-linux-gnu": "af840506efbcd5026d9140c0a0230e45e46bb1f339a65c10a22875930b2c0159", + "s390x-unknown-linux-gnu": "43f8f79bf4c66689d2019f193671d1df3e5e5dbb293382036285e8ce55fc55bb", + "x86_64-apple-darwin": "f25ce050e1d370f9c05c9623b769ffa4b269a6ae17e611b435fd2b8b09972a88", + "x86_64-pc-windows-msvc": "cb478a5a37eb93ce4d3c27ae64d211d6a5a42475ae53f666a8d1570e71fcf409", + "aarch64-pc-windows-msvc": "19129cf8b4d68c4e64c25bae43bca139d871267b59cf7f02b9dcf25f0bf59497", + "x86_64-unknown-linux-gnu": "a72f313bad49846e5e9671af2be7476033a877c80831cf47f431400ccb520090", + "x86_64-unknown-linux-musl": "15d50b15713097c38c67b1a06a0498ad102377f9b3999e98e4eefd6bf91bd82d", + "aarch64-apple-darwin-freethreaded": "61f38e947449cf00f32f0838e813358f6bf61025d0797531e5b8b8b175c617f0", + "aarch64-unknown-linux-gnu-freethreaded": "1a88a1fe21eb443d280999464b1a397605a7ca950d8ab73813ca6868835439a2", + "ppc64le-unknown-linux-gnu-freethreaded": "7207b736ed2569f307649ffd4b615a5346631bc244730b8702babee377cef528", + "riscv64-unknown-linux-gnu-freethreaded": "d1356ccd279920edc31bf0350674d966beb9522f9503846ed7855dbb109ccc14", + "s390x-unknown-linux-gnu-freethreaded": "477758eabc06dbc7e5e5d16e97c4672478acd409f420dd2e1b84d3452c0668d1", + "x86_64-apple-darwin-freethreaded": "c2cb2a9b44285fbc13c3c9b7eea813db6ed8d94909406b059db7afd39b32e786", + "x86_64-pc-windows-msvc-freethreaded": "8ef7048315cac6d26bdbef18512a87b1a24fffa21cec86e32f9a9425f2af9bf6", + "aarch64-pc-windows-msvc-freethreaded": "ddb10b645de2b1f6f2832a80b115a9cd34a4a760249983027efe46618a8efc48", + "x86_64-unknown-linux-gnu-freethreaded": "c5d5b89aab7de683e465e36de2477a131435076badda775ef6e9ea21109c1c32", + }, + "strip_prefix": { + "aarch64-apple-darwin": "python", + "aarch64-unknown-linux-gnu": "python", + "ppc64le-unknown-linux-gnu": "python", + "s390x-unknown-linux-gnu": "python", + "riscv64-unknown-linux-gnu": "python", + "x86_64-apple-darwin": "python", + "x86_64-pc-windows-msvc": "python", + "aarch64-pc-windows-msvc": "python", + "x86_64-unknown-linux-gnu": "python", + "x86_64-unknown-linux-musl": "python", + "aarch64-apple-darwin-freethreaded": "python/install", + "aarch64-unknown-linux-gnu-freethreaded": "python/install", + "ppc64le-unknown-linux-gnu-freethreaded": "python/install", + "riscv64-unknown-linux-gnu-freethreaded": "python/install", + "s390x-unknown-linux-gnu-freethreaded": "python/install", + "x86_64-apple-darwin-freethreaded": "python/install", + "x86_64-pc-windows-msvc-freethreaded": "python/install", + "aarch64-pc-windows-msvc-freethreaded": "python/install", + "x86_64-unknown-linux-gnu-freethreaded": "python/install", + }, + }, + "3.14.2": { + "url": "20251209/cpython-{python_version}+20251209-{platform}-{build}.{ext}", + "sha256": { + "aarch64-apple-darwin": "2f74bd26bd16487aca357c879d11f7b16c0521328e5148a1930ab6357bcb89fe", + "aarch64-unknown-linux-gnu": "869af31b2963194e8a2ecfadc36027c4c1c86a10f4960baec36dadb41b2acf02", + "ppc64le-unknown-linux-gnu": "86129976403fb5d64cf576329f94148f28cf6f82834e94df81ff31e9d5f404e0", + "riscv64-unknown-linux-gnu": "318dceecf119ea903aef1fb03a552cc592ecd61c08da891b68f5755e21e13511", + "s390x-unknown-linux-gnu": "53875c849a14194344ead1d9cd1e128cadd42a4b83c35eeb212417909ef05a6a", + "x86_64-apple-darwin": "58fa3e17d13ab956fd11055fb774c98ecfddcdf3b588e5f2369bdbc14ef9d76a", + "x86_64-pc-windows-msvc": "0d660bba9f58cb552e7e99e1f96a9c67b41618c9b8d29f9f3515fe2b5ad1966e", + "aarch64-pc-windows-msvc": "0be0d2557d73efa7f6f3f99679f05252d57fe2aad2d81cac3cad410a9b1eacbd", + "x86_64-unknown-linux-gnu": "121c3249bef497adf601df76a4d89aed6053fc5ec2f8c0ec656b86f0142e8ddd", + "x86_64-unknown-linux-musl": "71639cc5d1fb79840467531c5b53ca77170a58edd3f7e2d29330dd736e477469", + "aarch64-apple-darwin-freethreaded": "d6d17b8ef28326552cdeb2a7541c8a0cb711b378df9b93ebdb461dca065edfea", + "aarch64-unknown-linux-gnu-freethreaded": "adfcb90f3a7e1b3fbc6a99f9c8c8dce1f2e26ea72b724bbe4e9fa39e81e2b0db", + "ppc64le-unknown-linux-gnu-freethreaded": "2b1ce0c5a5f5e5add7e4f934f5bd35ac41660895a30b3098db7f7303d6952a4f", + "riscv64-unknown-linux-gnu-freethreaded": "4efb610fa07a6ee2639d14d78fc3b6ecb47431c14e1e4bda03c7f7dd60a5c1e5", + "s390x-unknown-linux-gnu-freethreaded": "e62f3bb3e66dac6c459690f9e9cd8cc2f6fe1dcf8bfed452af4c3df24cd7874f", + "x86_64-apple-darwin-freethreaded": "1fd76c79f7fc1753e8d2ed2f71406c0b65776c75f3e95ed99ffde8c95af2adc1", + "x86_64-pc-windows-msvc-freethreaded": "9927951e3997c186d2813ca1a0f4a8f5a2f771463f7f8ad0752fd3d2be2b74e4", + "aarch64-pc-windows-msvc-freethreaded": "43aac5bb4cdba71fc6775d26f47348d573a0b1210911438be71d7d96f4b18b51", + "x86_64-unknown-linux-gnu-freethreaded": "3728872ffd74989a7b4bbf3f0c629ae8fe821cda2bd6544012c1b92b9f5d5a5b", + }, + "strip_prefix": { + "aarch64-apple-darwin": "python", + "aarch64-unknown-linux-gnu": "python", + "ppc64le-unknown-linux-gnu": "python", + "s390x-unknown-linux-gnu": "python", + "riscv64-unknown-linux-gnu": "python", + "x86_64-apple-darwin": "python", + "x86_64-pc-windows-msvc": "python", + "aarch64-pc-windows-msvc": "python", + "x86_64-unknown-linux-gnu": "python", + "x86_64-unknown-linux-musl": "python", + "aarch64-apple-darwin-freethreaded": "python/install", + "aarch64-unknown-linux-gnu-freethreaded": "python/install", + "ppc64le-unknown-linux-gnu-freethreaded": "python/install", + "riscv64-unknown-linux-gnu-freethreaded": "python/install", + "s390x-unknown-linux-gnu-freethreaded": "python/install", + "x86_64-apple-darwin-freethreaded": "python/install", + "x86_64-pc-windows-msvc-freethreaded": "python/install", + "aarch64-pc-windows-msvc-freethreaded": "python/install", + "x86_64-unknown-linux-gnu-freethreaded": "python/install", + }, + }, "3.15.0a1": { "url": "20251031/cpython-{python_version}+20251031-{platform}-{build}.{ext}", "sha256": { @@ -857,6 +1037,51 @@ TOOL_VERSIONS = { "x86_64-unknown-linux-gnu-freethreaded": "python/install", }, }, + "3.15.0a2": { + "url": "20251209/cpython-{python_version}+20251209-{platform}-{build}.{ext}", + "sha256": { + "aarch64-apple-darwin": "5851f3744fbd39e3e323844cf4f68d7763fb25546aa5ffbb71b1b5ab69c56616", + "aarch64-unknown-linux-gnu": "17ba65d669be3052524e03b4d1426c072ef38df2a9065ff4525d1f4d1bc9f82c", + "ppc64le-unknown-linux-gnu": "5585bd7c5eefe28b9bf544d902cad9a2f81f33c618f2a1d3c006cbfcdec77abc", + "riscv64-unknown-linux-gnu": "bb7252edaffd422bd1c044a4764dfcf83a5d7159942f445abbef524e54ea79a0", + "s390x-unknown-linux-gnu": "03a90ffa9f92d4cf4caeefb9d15f0b39c05c1e60ade6688f32165f957db4f8f3", + "x86_64-apple-darwin": "cee576de4919cd422dbc31eb85d3c145ee82acec84f651daaf32dc669b5149c9", + "x86_64-pc-windows-msvc": "e538475ee249eacf63bfdae0e70af73e9c47360e6dd3d6825e7a35107e177de5", + "aarch64-pc-windows-msvc": "39bc2fcac13aeba7d650f76badf63350a81c86167a62174cb092eab7a749f4a5", + "x86_64-unknown-linux-gnu": "58addaabfab2de422180d32543fb3878ffc984c8a2e4005ff658a5cd83b31fc7", + "x86_64-unknown-linux-musl": "dcf844400dc2e7f5f3604e994532e4d49db45f4deefe9afdf6809ca1bc6532ee", + "aarch64-apple-darwin-freethreaded": "5b34488580df13df051a2e84e43cfca2ab28fdd7a61052f35988eb8b481b894a", + "aarch64-unknown-linux-gnu-freethreaded": "0c2c83236f6e28c103e2660a82be94b2459ee8cfdd90f5dd82f0d503ca2aec09", + "ppc64le-unknown-linux-gnu-freethreaded": "216842df2377fd032f279ded7fd23d7bdbd92d4c1fa7619523bc0dbdef5bd212", + "riscv64-unknown-linux-gnu-freethreaded": "2a8b56f318d2e21b01b54909554c53d81871b9bb05d23ea7808dde9acec4dc7e", + "s390x-unknown-linux-gnu-freethreaded": "06c4ca3983aad20723f68786e3663ab49fee1bf09326f341649205ed79d34fc6", + "x86_64-apple-darwin-freethreaded": "4d8102b70ea9fe726ee3ae9ad9e9bc4cbe0b6ed18f7989c81aef81de578f0163", + "x86_64-pc-windows-msvc-freethreaded": "6ff71bac78d650ce621fe6db49f06290e48bcceb61f69cccc7728584f70b6346", + "aarch64-pc-windows-msvc-freethreaded": "3d99152b4e29b947fb1cfc8d035d1d511e50aeed72886ff4a5fd0a3694bd0b51", + "x86_64-unknown-linux-gnu-freethreaded": "70f552e213734c0e260a57603bee504dd7ed0e78a10558b591e724ea8730fef5", + }, + "strip_prefix": { + "aarch64-apple-darwin": "python", + "aarch64-unknown-linux-gnu": "python", + "ppc64le-unknown-linux-gnu": "python", + "s390x-unknown-linux-gnu": "python", + "riscv64-unknown-linux-gnu": "python", + "x86_64-apple-darwin": "python", + "x86_64-pc-windows-msvc": "python", + "aarch64-pc-windows-msvc": "python", + "x86_64-unknown-linux-gnu": "python", + "x86_64-unknown-linux-musl": "python", + "aarch64-apple-darwin-freethreaded": "python/install", + "aarch64-unknown-linux-gnu-freethreaded": "python/install", + "ppc64le-unknown-linux-gnu-freethreaded": "python/install", + "riscv64-unknown-linux-gnu-freethreaded": "python/install", + "s390x-unknown-linux-gnu-freethreaded": "python/install", + "x86_64-apple-darwin-freethreaded": "python/install", + "x86_64-pc-windows-msvc-freethreaded": "python/install", + "aarch64-pc-windows-msvc-freethreaded": "python/install", + "x86_64-unknown-linux-gnu-freethreaded": "python/install", + }, + }, } # buildifier: disable=unsorted-dict-items @@ -865,9 +1090,9 @@ MINOR_MAPPING = { "3.10": "3.10.19", "3.11": "3.11.14", "3.12": "3.12.12", - "3.13": "3.13.9", - "3.14": "3.14.0", - "3.15": "3.15.0a1", + "3.13": "3.13.11", + "3.14": "3.14.2", + "3.15": "3.15.0a2", } def _generate_platforms(): From fd7a2e9611c280314df3c42296cca4bb94509c45 Mon Sep 17 00:00:00 2001 From: armandomontanez Date: Sat, 13 Dec 2025 18:23:34 -0800 Subject: [PATCH 552/922] chore(pip): Check for whl extract compatibility in internal_config_repo.bzl (#3456) Makes the check for whl extraction support slightly more granular, and moves it into internal_config_repo.bzl. --- python/private/internal_config_repo.bzl | 16 +++++++++++++++- python/private/pypi/whl_library.bzl | 7 ++----- 2 files changed, 17 insertions(+), 6 deletions(-) diff --git a/python/private/internal_config_repo.bzl b/python/private/internal_config_repo.bzl index 91f786c64e..fc1f8d3bbe 100644 --- a/python/private/internal_config_repo.bzl +++ b/python/private/internal_config_repo.bzl @@ -29,6 +29,7 @@ _ENABLE_DEPRECATION_WARNINGS_DEFAULT = "0" _CONFIG_TEMPLATE = """ config = struct( build_python_zip_default = {build_python_zip_default}, + supports_whl_extraction = {supports_whl_extraction}, enable_pystar = True, enable_pipstar = {enable_pipstar}, enable_deprecation_warnings = {enable_deprecation_warnings}, @@ -91,8 +92,20 @@ _TRANSITION_SETTINGS_DEBUG_TEMPLATE = """ def _internal_config_repo_impl(rctx): # An empty version signifies a development build, which is treated as # the latest version. - bazel_major_version = int(native.bazel_version.split(".")[0]) if native.bazel_version else 99999 + if native.bazel_version: + version_parts = native.bazel_version.split(".") + bazel_major_version = int(version_parts[0]) + bazel_minor_version = int(version_parts[1]) + else: + bazel_major_version = 99999 + bazel_minor_version = 99999 + + supports_whl_extraction = False if bazel_major_version >= 8: + # Extracting .whl files requires Bazel 8.3.0 or later. + if bazel_major_version > 8 or bazel_minor_version >= 3: + supports_whl_extraction = True + builtin_py_info_symbol = "None" builtin_py_runtime_info_symbol = "None" builtin_py_cc_link_params_provider = "None" @@ -107,6 +120,7 @@ def _internal_config_repo_impl(rctx): enable_deprecation_warnings = _bool_from_environ(rctx, _ENABLE_DEPRECATION_WARNINGS_ENVVAR_NAME, _ENABLE_DEPRECATION_WARNINGS_DEFAULT), builtin_py_info_symbol = builtin_py_info_symbol, builtin_py_runtime_info_symbol = builtin_py_runtime_info_symbol, + supports_whl_extraction = str(supports_whl_extraction), builtin_py_cc_link_params_provider = builtin_py_cc_link_params_provider, bazel_8_or_later = str(bazel_major_version >= 8), bazel_9_or_later = str(bazel_major_version >= 9), diff --git a/python/private/pypi/whl_library.bzl b/python/private/pypi/whl_library.bzl index d8f32ffef1..044d18a0df 100644 --- a/python/private/pypi/whl_library.bzl +++ b/python/private/pypi/whl_library.bzl @@ -377,10 +377,7 @@ def _whl_library_impl(rctx): # # Remove non-pipstar and config_load check when we release rules_python 2. if enable_pipstar: - # Extracting .whl files requires Bazel 8.3.0 or later, so require a - # minimum of Bazel 9.0.0 to ensure compatibilty with earlier versions - # of Bazel 8. - if rp_config.bazel_9_or_later: + if rp_config.supports_whl_extraction: extract_path = whl_path else: extract_path = rctx.path(whl_path.basename + ".zip") @@ -389,7 +386,7 @@ def _whl_library_impl(rctx): archive = extract_path, output = "site-packages", ) - if not rp_config.bazel_9_or_later: + if not rp_config.supports_whl_extraction: rctx.delete(extract_path) metadata = whl_metadata( From 0816a1ff1e425f5177d10433a50fc756ca79ccb1 Mon Sep 17 00:00:00 2001 From: Ignas Anikevicius <240938+aignas@users.noreply.github.com> Date: Mon, 15 Dec 2025 04:25:29 +0900 Subject: [PATCH 553/922] fix(pip): set better defaults for the new target_platforms attr (#3447) Fixup the default values for the `target_platforms` so that the users can actually switch it as per docs. I have also taken the liberty to update all of the tests to better reflect how we set things up for legacy and index-url setups. As it is now, legacy `bzlmod` whl_library layout is much more similar to WORKSPACE, which makes the transition easier. Work towards #2949 --- MODULE.bazel | 5 + docs/BUILD.bazel | 2 + python/private/pypi/extension.bzl | 2 +- python/private/pypi/hub_builder.bzl | 7 +- .../pypi/requirements_files_by_platform.bzl | 2 + tests/pypi/extension/extension_tests.bzl | 8 +- tests/pypi/hub_builder/hub_builder_tests.bzl | 444 ++++++++---------- 7 files changed, 215 insertions(+), 255 deletions(-) diff --git a/MODULE.bazel b/MODULE.bazel index b909124d11..ef7499eb0c 100644 --- a/MODULE.bazel +++ b/MODULE.bazel @@ -305,6 +305,11 @@ dev_pip = use_extension( parallel_download = False, python_version = python_version, requirements_lock = "//docs:requirements.txt", + # Ensure that we are setting up the following platforms + target_platforms = [ + "{os}_{arch}", + "{os}_{arch}_freethreaded", + ], ) for python_version in [ "3.9", diff --git a/docs/BUILD.bazel b/docs/BUILD.bazel index d7748d35a4..55d8f75a74 100644 --- a/docs/BUILD.bazel +++ b/docs/BUILD.bazel @@ -165,6 +165,8 @@ sphinx_build_binary( config_settings = { labels.BOOTSTRAP_IMPL: "script", labels.VENVS_SITE_PACKAGES: "yes", + labels.PY_FREETHREADED: "yes", + labels.PYTHON_VERSION: "3.14", }, target_compatible_with = _TARGET_COMPATIBLE_WITH, deps = [ diff --git a/python/private/pypi/extension.bzl b/python/private/pypi/extension.bzl index 2a6d43f837..3d7985ab3f 100644 --- a/python/private/pypi/extension.bzl +++ b/python/private/pypi/extension.bzl @@ -665,7 +665,7 @@ EXPERIMENTAL: this may be removed without notice. """, ), "target_platforms": attr.string_list( - default = ["{os}_{arch}"], + default = [], doc = """\ The list of platforms for which we would evaluate the requirements files. If you need to be able to only evaluate for a particular platform (e.g. "linux_x86_64"), then put it in here. diff --git a/python/private/pypi/hub_builder.bzl b/python/private/pypi/hub_builder.bzl index 3a1a3b07fe..97e0a111b2 100644 --- a/python/private/pypi/hub_builder.bzl +++ b/python/private/pypi/hub_builder.bzl @@ -141,9 +141,10 @@ def _pip_parse(self, module_ctx, pip_attr): module_ctx, python_version = full_python_version, config = self._config, - # FIXME @aignas 2025-12-06: should we have this behaviour? - # TODO @aignas 2025-12-06: use target_platforms always even when the get_index_urls is set. - target_platforms = [] if default_cross_setup else pip_attr.target_platforms, + # TODO @aignas 2025-12-09: flip or part to default to 'os_arch' after + # VERSION_NEXT_FEATURE is released and set the default of the `target_platforms` attribute + # to `{os}_{arch}`. + target_platforms = pip_attr.target_platforms or ([] if default_cross_setup else ["{os}_{arch}"]), ) _add_group_map(self, pip_attr.experimental_requirement_cycles) _add_extra_aliases(self, pip_attr.extra_hub_aliases) diff --git a/python/private/pypi/requirements_files_by_platform.bzl b/python/private/pypi/requirements_files_by_platform.bzl index 2027b41594..c0fc93a0fe 100644 --- a/python/private/pypi/requirements_files_by_platform.bzl +++ b/python/private/pypi/requirements_files_by_platform.bzl @@ -144,6 +144,8 @@ def requirements_files_by_platform( input_platforms = platforms default_platforms = [_platform(p, python_version) for p in platforms] + if logger: + logger.debug(lambda: "Input platforms: {}".format(input_platforms)) if platforms_from_args: lock_files = [ diff --git a/tests/pypi/extension/extension_tests.bzl b/tests/pypi/extension/extension_tests.bzl index b1e363bc7b..924796c703 100644 --- a/tests/pypi/extension/extension_tests.bzl +++ b/tests/pypi/extension/extension_tests.bzl @@ -22,12 +22,12 @@ load(":pip_parse.bzl", _parse = "pip_parse") _tests = [] -def _mock_mctx(*modules, environ = {}, read = None): +def _mock_mctx(*modules, os_name = "unittest", arch_name = "exotic", environ = {}, read = None): return struct( os = struct( environ = environ, - name = "unittest", - arch = "exotic", + name = os_name, + arch = arch_name, ), read = read or (lambda _: """\ simple==0.0.1 \ @@ -148,6 +148,8 @@ def _test_simple(env): ), ], ), + os_name = "linux", + arch_name = "x86_64", ), available_interpreters = { "python_3_15_host": "unit_test_interpreter_target", diff --git a/tests/pypi/hub_builder/hub_builder_tests.bzl b/tests/pypi/hub_builder/hub_builder_tests.bzl index e267f4ca34..bf21dcacfa 100644 --- a/tests/pypi/hub_builder/hub_builder_tests.bzl +++ b/tests/pypi/hub_builder/hub_builder_tests.bzl @@ -25,12 +25,12 @@ load("//tests/pypi/extension:pip_parse.bzl", _parse = "pip_parse") _tests = [] -def _mock_mctx(os = "unittest", arch = "exotic", environ = {}, read = None): +def _mock_mctx(os_name = "unittest", arch_name = "exotic", environ = {}, read = None): return struct( os = struct( environ = environ, - name = os, - arch = arch, + name = os_name, + arch = arch_name, ), read = read or (lambda _: """\ simple==0.0.1 \ @@ -114,7 +114,10 @@ def hub_builder( def _test_simple(env): builder = hub_builder(env) builder.pip_parse( - _mock_mctx(), + _mock_mctx( + os_name = "osx", + arch_name = "aarch64", + ), _parse( hub_name = "pypi", python_version = "3.15", @@ -147,118 +150,94 @@ def _test_simple(env): _tests.append(_test_simple) def _test_simple_multiple_requirements(env): - builder = hub_builder(env) - builder.pip_parse( - _mock_mctx( - read = lambda x: { - "darwin.txt": "simple==0.0.2 --hash=sha256:deadb00f", - "win.txt": "simple==0.0.1 --hash=sha256:deadbeef", - }[x], - ), - _parse( - hub_name = "pypi", - python_version = "3.15", - requirements_darwin = "darwin.txt", - requirements_windows = "win.txt", - ), - ) - pypi = builder.build() + sub_tests = { + ("osx", "aarch64"): "simple==0.0.2 --hash=sha256:deadb00f", + ("windows", "aarch64"): "simple==0.0.1 --hash=sha256:deadbeef", + } + for (host_os, host_arch), want_requirement in sub_tests.items(): + builder = hub_builder(env) + builder.pip_parse( + _mock_mctx( + read = lambda x: { + "darwin.txt": "simple==0.0.2 --hash=sha256:deadb00f", + "win.txt": "simple==0.0.1 --hash=sha256:deadbeef", + }[x], + os_name = host_os, + arch_name = host_arch, + ), + _parse( + hub_name = "pypi", + python_version = "3.15", + requirements_darwin = "darwin.txt", + requirements_windows = "win.txt", + ), + ) + pypi = builder.build() - pypi.exposed_packages().contains_exactly(["simple"]) - pypi.group_map().contains_exactly({}) - pypi.whl_map().contains_exactly({ - "simple": { - "pypi_315_simple_osx_aarch64": [ - whl_config_setting( - target_platforms = [ - "cp315_osx_aarch64", - ], - version = "3.15", - ), - ], - "pypi_315_simple_windows_aarch64": [ - whl_config_setting( - target_platforms = [ - "cp315_windows_aarch64", - ], - version = "3.15", - ), - ], - }, - }) - pypi.whl_libraries().contains_exactly({ - "pypi_315_simple_osx_aarch64": { - "config_load": "@pypi//:config.bzl", - "dep_template": "@pypi//{name}:{target}", - "python_interpreter_target": "unit_test_interpreter_target", - "requirement": "simple==0.0.2 --hash=sha256:deadb00f", - }, - "pypi_315_simple_windows_aarch64": { - "config_load": "@pypi//:config.bzl", - "dep_template": "@pypi//{name}:{target}", - "python_interpreter_target": "unit_test_interpreter_target", - "requirement": "simple==0.0.1 --hash=sha256:deadbeef", - }, - }) - pypi.extra_aliases().contains_exactly({}) + pypi.exposed_packages().contains_exactly(["simple"]) + pypi.group_map().contains_exactly({}) + pypi.whl_map().contains_exactly({ + "simple": { + "pypi_315_simple": [ + whl_config_setting(version = "3.15"), + ], + }, + }) + pypi.whl_libraries().contains_exactly({ + "pypi_315_simple": { + "config_load": "@pypi//:config.bzl", + "dep_template": "@pypi//{name}:{target}", + "python_interpreter_target": "unit_test_interpreter_target", + "requirement": want_requirement, + }, + }) + pypi.extra_aliases().contains_exactly({}) _tests.append(_test_simple_multiple_requirements) def _test_simple_extras_vs_no_extras(env): - builder = hub_builder(env) - builder.pip_parse( - _mock_mctx( - read = lambda x: { - "darwin.txt": "simple[foo]==0.0.1 --hash=sha256:deadbeef", - "win.txt": "simple==0.0.1 --hash=sha256:deadbeef", - }[x], - ), - _parse( - hub_name = "pypi", - python_version = "3.15", - requirements_darwin = "darwin.txt", - requirements_windows = "win.txt", - ), - ) - pypi = builder.build() + sub_tests = { + ("osx", "aarch64"): "simple[foo]==0.0.1 --hash=sha256:deadbeef", + ("windows", "aarch64"): "simple==0.0.1 --hash=sha256:deadbeef", + } + for (host_os, host_arch), want_requirement in sub_tests.items(): + builder = hub_builder(env) + builder.pip_parse( + _mock_mctx( + read = lambda x: { + "darwin.txt": "simple[foo]==0.0.1 --hash=sha256:deadbeef", + "win.txt": "simple==0.0.1 --hash=sha256:deadbeef", + }[x], + os_name = host_os, + arch_name = host_arch, + ), + _parse( + hub_name = "pypi", + python_version = "3.15", + requirements_darwin = "darwin.txt", + requirements_windows = "win.txt", + ), + ) + pypi = builder.build() - pypi.exposed_packages().contains_exactly(["simple"]) - pypi.group_map().contains_exactly({}) - pypi.whl_map().contains_exactly({ - "simple": { - "pypi_315_simple_osx_aarch64": [ - whl_config_setting( - target_platforms = [ - "cp315_osx_aarch64", - ], - version = "3.15", - ), - ], - "pypi_315_simple_windows_aarch64": [ - whl_config_setting( - target_platforms = [ - "cp315_windows_aarch64", - ], - version = "3.15", - ), - ], - }, - }) - pypi.whl_libraries().contains_exactly({ - "pypi_315_simple_osx_aarch64": { - "config_load": "@pypi//:config.bzl", - "dep_template": "@pypi//{name}:{target}", - "python_interpreter_target": "unit_test_interpreter_target", - "requirement": "simple[foo]==0.0.1 --hash=sha256:deadbeef", - }, - "pypi_315_simple_windows_aarch64": { - "config_load": "@pypi//:config.bzl", - "dep_template": "@pypi//{name}:{target}", - "python_interpreter_target": "unit_test_interpreter_target", - "requirement": "simple==0.0.1 --hash=sha256:deadbeef", - }, - }) - pypi.extra_aliases().contains_exactly({}) + pypi.exposed_packages().contains_exactly(["simple"]) + pypi.group_map().contains_exactly({}) + pypi.whl_map().contains_exactly({ + "simple": { + "pypi_315_simple": [ + whl_config_setting(version = "3.15"), + ], + }, + }) + pypi.whl_libraries().contains_exactly({ + "pypi_315_simple": { + "config_load": "@pypi//:config.bzl", + "dep_template": "@pypi//{name}:{target}", + "python_interpreter_target": "unit_test_interpreter_target", + "requirement": want_requirement, + }, + }) + pypi.extra_aliases().contains_exactly({}) _tests.append(_test_simple_extras_vs_no_extras) @@ -358,6 +337,8 @@ simple==0.0.1 --hash=sha256:deadbeef old-package==0.0.1 --hash=sha256:deadbaaf """, }[x], + os_name = "linux", + arch_name = "amd64", ), _parse( hub_name = "pypi", @@ -373,6 +354,8 @@ simple==0.0.2 --hash=sha256:deadb00f new-package==0.0.1 --hash=sha256:deadb00f2 """, }[x], + os_name = "linux", + arch_name = "amd64", ), _parse( hub_name = "pypi", @@ -387,28 +370,20 @@ new-package==0.0.1 --hash=sha256:deadb00f2 pypi.whl_map().contains_exactly({ "new_package": { "pypi_316_new_package": [ - whl_config_setting( - version = "3.16", - ), + whl_config_setting(version = "3.16"), ], }, "old_package": { "pypi_315_old_package": [ - whl_config_setting( - version = "3.15", - ), + whl_config_setting(version = "3.15"), ], }, "simple": { "pypi_315_simple": [ - whl_config_setting( - version = "3.15", - ), + whl_config_setting(version = "3.15"), ], "pypi_316_simple": [ - whl_config_setting( - version = "3.16", - ), + whl_config_setting(version = "3.16"), ], }, }) @@ -443,75 +418,62 @@ new-package==0.0.1 --hash=sha256:deadb00f2 _tests.append(_test_simple_multiple_python_versions) def _test_simple_with_markers(env): - builder = hub_builder( - env, - evaluate_markers_fn = lambda _, requirements, **__: { - key: [ - platform - for platform in platforms - if ("x86_64" in platform and "platform_machine ==" in key) or ("x86_64" not in platform and "platform_machine !=" in key) - ] - for key, platforms in requirements.items() - }, - ) - builder.pip_parse( - _mock_mctx( - read = lambda x: { - "universal.txt": """\ -torch==2.4.1+cpu ; platform_machine == 'x86_64' -torch==2.4.1 ; platform_machine != 'x86_64' \ - --hash=sha256:deadbeef -""", - }[x], - ), - _parse( - hub_name = "pypi", - python_version = "3.15", - requirements_lock = "universal.txt", - ), - ) - pypi = builder.build() + sub_tests = { + ("osx", "aarch64"): "torch==2.4.1 --hash=sha256:deadbeef", + ("linux", "x86_64"): "torch==2.4.1+cpu", + } + for (host_os, host_arch), want_requirement in sub_tests.items(): + builder = hub_builder( + env, + evaluate_markers_fn = lambda _, requirements, **__: { + key: [ + platform + for platform in platforms + if ("x86_64" in platform and "platform_machine ==" in key) or ("x86_64" not in platform and "platform_machine !=" in key) + ] + for key, platforms in requirements.items() + }, + ) + builder.pip_parse( + _mock_mctx( + read = lambda x: { + "universal.txt": """\ + torch==2.4.1+cpu ; platform_machine == 'x86_64' + torch==2.4.1 ; platform_machine != 'x86_64' \ + --hash=sha256:deadbeef + """, + }[x], + os_name = host_os, + arch_name = host_arch, + ), + _parse( + hub_name = "pypi", + python_version = "3.15", + requirements_lock = "universal.txt", + ), + ) + pypi = builder.build() - pypi.exposed_packages().contains_exactly(["torch"]) - pypi.group_map().contains_exactly({}) - pypi.whl_map().contains_exactly({ - "torch": { - "pypi_315_torch_linux_aarch64_osx_aarch64_windows_aarch64": [ - whl_config_setting( - target_platforms = [ - "cp315_linux_aarch64", - "cp315_osx_aarch64", - "cp315_windows_aarch64", - ], - version = "3.15", - ), - ], - "pypi_315_torch_linux_x86_64_linux_x86_64_freethreaded": [ - whl_config_setting( - target_platforms = [ - "cp315_linux_x86_64", - "cp315_linux_x86_64_freethreaded", - ], - version = "3.15", - ), - ], - }, - }) - pypi.whl_libraries().contains_exactly({ - "pypi_315_torch_linux_aarch64_osx_aarch64_windows_aarch64": { - "config_load": "@pypi//:config.bzl", - "dep_template": "@pypi//{name}:{target}", - "python_interpreter_target": "unit_test_interpreter_target", - "requirement": "torch==2.4.1 --hash=sha256:deadbeef", - }, - "pypi_315_torch_linux_x86_64_linux_x86_64_freethreaded": { - "config_load": "@pypi//:config.bzl", - "dep_template": "@pypi//{name}:{target}", - "python_interpreter_target": "unit_test_interpreter_target", - "requirement": "torch==2.4.1+cpu", - }, - }) - pypi.extra_aliases().contains_exactly({}) + pypi.exposed_packages().contains_exactly(["torch"]) + pypi.group_map().contains_exactly({}) + pypi.whl_map().contains_exactly({ + "torch": { + "pypi_315_torch": [ + whl_config_setting( + version = "3.15", + ), + ], + }, + }) + pypi.whl_libraries().contains_exactly({ + "pypi_315_torch": { + "config_load": "@pypi//:config.bzl", + "dep_template": "@pypi//{name}:{target}", + "python_interpreter_target": "unit_test_interpreter_target", + "requirement": want_requirement, + }, + }) + pypi.extra_aliases().contains_exactly({}) _tests.append(_test_simple_with_markers) @@ -1079,66 +1041,51 @@ git_dep @ git+https://git.server/repo/project@deadbeefdeadbeef _tests.append(_test_simple_get_index) def _test_optimum_sys_platform_extra(env): - builder = hub_builder( - env, - ) - builder.pip_parse( - _mock_mctx( - read = lambda x: { - "universal.txt": """\ + sub_tests = { + ("osx", "aarch64"): "optimum[onnxruntime]==1.17.1", + ("linux", "aarch64"): "optimum[onnxruntime-gpu]==1.17.1", + } + for (host_os, host_arch), want_requirement in sub_tests.items(): + builder = hub_builder( + env, + ) + builder.pip_parse( + _mock_mctx( + read = lambda x: { + "universal.txt": """\ optimum[onnxruntime]==1.17.1 ; sys_platform == 'darwin' optimum[onnxruntime-gpu]==1.17.1 ; sys_platform == 'linux' """, - }[x], - ), - _parse( - hub_name = "pypi", - python_version = "3.15", - requirements_lock = "universal.txt", - ), - ) - pypi = builder.build() + }[x], + os_name = host_os, + arch_name = host_arch, + ), + _parse( + hub_name = "pypi", + python_version = "3.15", + requirements_lock = "universal.txt", + ), + ) + pypi = builder.build() - # FIXME @aignas 2025-09-07: we should expose the `optimum` package - pypi.exposed_packages().contains_exactly([]) - pypi.group_map().contains_exactly({}) - pypi.whl_map().contains_exactly({ - "optimum": { - "pypi_315_optimum_linux_aarch64_linux_x86_64_linux_x86_64_freethreaded": [ - whl_config_setting( - version = "3.15", - target_platforms = [ - "cp315_linux_aarch64", - "cp315_linux_x86_64", - "cp315_linux_x86_64_freethreaded", - ], - ), - ], - "pypi_315_optimum_osx_aarch64": [ - whl_config_setting( - version = "3.15", - target_platforms = [ - "cp315_osx_aarch64", - ], - ), - ], - }, - }) - pypi.whl_libraries().contains_exactly({ - "pypi_315_optimum_linux_aarch64_linux_x86_64_linux_x86_64_freethreaded": { - "config_load": "@pypi//:config.bzl", - "dep_template": "@pypi//{name}:{target}", - "python_interpreter_target": "unit_test_interpreter_target", - "requirement": "optimum[onnxruntime-gpu]==1.17.1", - }, - "pypi_315_optimum_osx_aarch64": { - "config_load": "@pypi//:config.bzl", - "dep_template": "@pypi//{name}:{target}", - "python_interpreter_target": "unit_test_interpreter_target", - "requirement": "optimum[onnxruntime]==1.17.1", - }, - }) - pypi.extra_aliases().contains_exactly({}) + pypi.exposed_packages().contains_exactly(["optimum"]) + pypi.group_map().contains_exactly({}) + pypi.whl_map().contains_exactly({ + "optimum": { + "pypi_315_optimum": [ + whl_config_setting(version = "3.15"), + ], + }, + }) + pypi.whl_libraries().contains_exactly({ + "pypi_315_optimum": { + "config_load": "@pypi//:config.bzl", + "dep_template": "@pypi//{name}:{target}", + "python_interpreter_target": "unit_test_interpreter_target", + "requirement": want_requirement, + }, + }) + pypi.extra_aliases().contains_exactly({}) _tests.append(_test_optimum_sys_platform_extra) @@ -1181,6 +1128,7 @@ optimum[onnxruntime-gpu]==1.17.1 ; sys_platform == 'linux' hub_name = "pypi", python_version = "3.15", requirements_lock = "universal.txt", + target_platforms = ["mylinuxx86_64", "myosxaarch64"], ), ) pypi = builder.build() @@ -1253,8 +1201,8 @@ def _test_pipstar_platforms_limit(env): ) builder.pip_parse( _mock_mctx( - os = "linux", - arch = "amd64", + os_name = "linux", + arch_name = "amd64", read = lambda x: { "universal.txt": """\ optimum[onnxruntime]==1.17.1 ; sys_platform == 'darwin' From e05cf00465763d0798c9a2cae116dbadcc13c701 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Sun, 14 Dec 2025 12:29:35 -0800 Subject: [PATCH 554/922] fix: correctly merge conflicting paths when files (instead of dirs) are being linked (#3458) When a file (instead of directory) was duplicated, it was incorrectly be treated as a directory. The conflicting path was turned into a directory and all the files were made child paths of it. In practice, this usually worked out OK still because the conflicting file was an `__init__.py` file for a pkgutil-style namespace package, which effectively converted it to an implicit namespace package. I think this was introduced by https://github.com/bazel-contrib/rules_python/pull/3448, but am not 100% sure. To fix, first check for exact path equality when merging conflicting paths. If the candidate file has the same link_to_path as the grouping, then just link to that file and skip any remaining files in the group. The rest of the group's files are skipped because none of them can contribute links anymore: * If they have the same path, then they are ignored (first set wins) * If they are a sub-path, then it violates the preconditions of the group (all files in the group should be exact matches or sub-paths of a common prefix) * If they aren't under the common prefix, then it violates the preconditions of the group and are skipped. --------- Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> --- python/private/venv_runfiles.bzl | 55 ++++++---- .../app_files_building_tests.bzl | 101 ++++++++++-------- 2 files changed, 94 insertions(+), 62 deletions(-) diff --git a/python/private/venv_runfiles.bzl b/python/private/venv_runfiles.bzl index 43fcab6192..0c79ea82a1 100644 --- a/python/private/venv_runfiles.bzl +++ b/python/private/venv_runfiles.bzl @@ -163,8 +163,10 @@ def _group_venv_path_entries(entries): current_group = None current_group_prefix = None for entry in entries: - prefix = entry.venv_path - anchored_prefix = prefix + "/" + # NOTE: When a file is being directly linked, the anchored prefix can look + # odd, e.g. "foo/__init__.py/". This is OK; it's just used to prevent + # incorrect prefix substring matching. + anchored_prefix = entry.venv_path + "/" if (current_group_prefix == None or not anchored_prefix.startswith(current_group_prefix)): current_group_prefix = anchored_prefix @@ -193,26 +195,41 @@ def _merge_venv_path_group(ctx, group, keep_map): # be symlinked directly, not the directory containing them, due to # dynamic linker symlink resolution semantics on Linux. for entry in group: - prefix = entry.venv_path + root_venv_path = entry.venv_path + anchored_link_to_path = entry.link_to_path + "/" for file in entry.files.to_list(): - # Compute the file-specific venv path. i.e. the relative - # path of the file under entry.venv_path, joined with - # entry.venv_path rf_root_path = runfiles_root_path(ctx, file.short_path) - if not rf_root_path.startswith(entry.link_to_path): - # This generally shouldn't occur in practice, but just - # in case, skip them, for lack of a better option. - continue - venv_path = "{}/{}".format( - prefix, - rf_root_path.removeprefix(entry.link_to_path + "/"), - ) - # For lack of a better option, first added wins. We happen to - # go in top-down prefix order, so the highest level namespace - # package typically wins. - if venv_path not in keep_map: - keep_map[venv_path] = file + # It's a file (or directory) being directly linked and + # must be directly linked. + if rf_root_path == entry.link_to_path: + # For lack of a better option, first added wins. + if entry.venv_path not in keep_map: + keep_map[entry.venv_path] = file + + # Skip anything remaining: anything left is either + # the same path (first set wins), a suffix (violates + # preconditions and can't link anyways), or not under + # the prefix (violates preconditions). + break + else: + # Compute the file-specific venv path. i.e. the relative + # path of the file under entry.venv_path, joined with + # entry.venv_path + head, match, rel_venv_path = rf_root_path.partition(anchored_link_to_path) + if not match or head: + # If link_to_path didn't match, then obviously skip. + # If head is non-empty, it means link_to_path wasn't + # found at the start + # This shouldn't occur in practice, but guard against it + # just in case + continue + + venv_path = paths.join(root_venv_path, rel_venv_path) + + # For lack of a better option, first added wins. + if venv_path not in keep_map: + keep_map[venv_path] = file def get_venv_symlinks(ctx, files, package, version_str, site_packages_root): """Compute the VenvSymlinkEntry objects for a library. diff --git a/tests/venv_site_packages_libs/app_files_building/app_files_building_tests.bzl b/tests/venv_site_packages_libs/app_files_building/app_files_building_tests.bzl index fc0b5d0bf3..486293b38d 100644 --- a/tests/venv_site_packages_libs/app_files_building/app_files_building_tests.bzl +++ b/tests/venv_site_packages_libs/app_files_building/app_files_building_tests.bzl @@ -1,6 +1,5 @@ "" -load("@bazel_skylib//lib:paths.bzl", "paths") load("@rules_testing//lib:analysis_test.bzl", "analysis_test") load("@rules_testing//lib:test_suite.bzl", "test_suite") load("//python/private:py_info.bzl", "VenvSymlinkEntry", "VenvSymlinkKind") # buildifier: disable=bzl-visibility @@ -58,34 +57,16 @@ def _venv_symlinks_from_entries(entries): )) return sorted(result, key = lambda e: (e.link_to_path, e.venv_path)) -def _entry(venv_path, link_to_path, files = [], **kwargs): +def _entry(venv_path, link_to_path, files, **kwargs): kwargs.setdefault("kind", VenvSymlinkKind.LIB) kwargs.setdefault("package", None) kwargs.setdefault("version", None) - - def short_pathify(path): - path = paths.join(link_to_path, path) - - # In tests, `../` is used to step out of the link_to_path scope. - path = paths.normalize(path) - - # Treat paths starting with "+" as external references. This matches - # how bzlmod names things. - if link_to_path.startswith("+"): - # File.short_path to external repos have `../` prefixed - path = paths.join("../", path) - else: - # File.short_path in main repo is main-repo relative - _, _, path = path.partition("/") - return path + kwargs.setdefault("link_to_file", None) return VenvSymlinkEntry( venv_path = venv_path, link_to_path = link_to_path, - files = depset([ - _file(short_pathify(f)) - for f in files - ]), + files = depset(files), **kwargs ) @@ -100,15 +81,34 @@ _tests.append(_test_conflict_merging) def _test_conflict_merging_impl(env, _): entries = [ - _entry("a", "+pypi_a/site-packages/a", ["a.txt"]), - _entry("a-1.0.dist-info", "+pypi_a/site-packages/a-1.0.dist-info", ["METADATA"]), - _entry("a/b", "+pypi_a_b/site-packages/a/b", ["b.txt"]), - _entry("x", "_main/src/x", ["x.txt"]), - _entry("x/p", "_main/src-dev/x/p", ["p.txt"]), - _entry("duplicate", "+dupe_a/site-packages/duplicate", ["d.py"]), - # This entry also provides a/x.py, but since the "a" entry is shorter - # and comes first, its version of x.py should win. - _entry("duplicate", "+dupe_b/site-packages/duplicate", ["d.py"]), + _entry("a", "+pypi_a/site-packages/a", [ + _file("../+pypi_a/site-packages/a/a.txt"), + ]), + _entry("a-1.0.dist-info", "+pypi_a/site-packages/a-1.0.dist-info", [ + _file("../+pypi_a/site-packages/a-1.0.dist-info/METADATA"), + ]), + _entry("a/b", "+pypi_a_b/site-packages/a/b", [ + _file("../+pypi_a_b/site-packages/a/b/b.txt"), + ]), + _entry("x", "_main/src/x", [ + _file("src/x/x.txt"), + ]), + _entry("x/p", "_main/src-dev/x/p", [ + _file("src-dev/x/p/p.txt"), + ]), + _entry("duplicate", "+dupe_a/site-packages/duplicate", [ + _file("../+dupe_a/site-packages/duplicate/d.py"), + ]), + _entry("duplicate", "+dupe_b/site-packages/duplicate", [ + _file("../+dupe_b/site-packages/duplicate/d.py"), + ]), + # Case: two distributions provide the same file (instead of directory) + _entry("ff/fmod.py", "+ff_a/site-packages/ff/fmod.py", [ + _file("../+ff_a/site-packages/ff/fmod.py"), + ]), + _entry("ff/fmod.py", "+ff_b/site-packages/ff/fmod.py", [ + _file("../+ff_b/site-packages/ff/fmod.py"), + ]), ] actual, conflicts = build_link_map(_ctx(), entries, return_conflicts = True) @@ -117,6 +117,7 @@ def _test_conflict_merging_impl(env, _): "a/a.txt": _file("../+pypi_a/site-packages/a/a.txt"), "a/b/b.txt": _file("../+pypi_a_b/site-packages/a/b/b.txt"), "duplicate/d.py": _file("../+dupe_a/site-packages/duplicate/d.py"), + "ff/fmod.py": _file("../+ff_a/site-packages/ff/fmod.py"), "x/p/p.txt": _file("src-dev/x/p/p.txt"), "x/x.txt": _file("src/x/x.txt"), } @@ -274,8 +275,12 @@ _tests.append(_test_package_version_filtering) def _test_package_version_filtering_impl(env, _): entries = [ - _entry("foo", "+pypi_v1/site-packages/foo", ["foo.txt"], package = "foo", version = "1.0"), - _entry("foo", "+pypi_v2/site-packages/foo", ["bar.txt"], package = "foo", version = "2.0"), + _entry("foo", "+pypi_v1/site-packages/foo", [ + _file("../+pypi_v1/site-packages/foo/foo.txt"), + ], package = "foo", version = "1.0"), + _entry("foo", "+pypi_v2/site-packages/foo", [ + _file("../+pypi_v2/site-packages/foo/bar.txt"), + ], package = "foo", version = "2.0"), ] actual = build_link_map(_ctx(), entries) @@ -300,7 +305,7 @@ def _test_malformed_entry_impl(env, _): "a", "+pypi_a/site-packages/a", # This file is outside the link_to_path, so it should be ignored. - ["../outside.txt"], + [_file("../+pypi_a/site-packages/outside.txt")], ), # A second, conflicting, entry is added to force merging of the known # files. Without this, there's no conflict, so files is never @@ -308,7 +313,7 @@ def _test_malformed_entry_impl(env, _): _entry( "a", "+pypi_b/site-packages/a", - ["../outside.txt"], + [_file("../+pypi_b/site-packages/outside.txt")], ), ] @@ -328,11 +333,21 @@ _tests.append(_test_complex_namespace_packages) def _test_complex_namespace_packages_impl(env, _): entries = [ - _entry("a/b", "+pypi_a_b/site-packages/a/b", ["b.txt"]), - _entry("a/c", "+pypi_a_c/site-packages/a/c", ["c.txt"]), - _entry("x/y/z", "+pypi_x_y_z/site-packages/x/y/z", ["z.txt"]), - _entry("foo", "+pypi_foo/site-packages/foo", ["foo.txt"]), - _entry("foobar", "+pypi_foobar/site-packages/foobar", ["foobar.txt"]), + _entry("a/b", "+pypi_a_b/site-packages/a/b", [ + _file("../+pypi_a_b/site-packages/a/b/b.txt"), + ]), + _entry("a/c", "+pypi_a_c/site-packages/a/c", [ + _file("../+pypi_a_c/site-packages/a/cc.txt"), + ]), + _entry("x/y/z", "+pypi_x_y_z/site-packages/x/y/z", [ + _file("../+pypi_x_y_z/site-packages/x/y/z/z.txt"), + ]), + _entry("foo", "+pypi_foo/site-packages/foo", [ + _file("../+pypi_foo/site-packages/foo/foo.txt"), + ]), + _entry("foobar", "+pypi_foobar/site-packages/foobar", [ + _file("../+pypi_foobar/site-packages/foobar/foobar.txt"), + ]), ] actual = build_link_map(_ctx(), entries) @@ -380,19 +395,19 @@ def _test_multiple_venv_symlink_kinds_impl(env, _): _entry( "libfile", "+pypi_lib/site-packages/libfile", - ["lib.txt"], + [_file("../+pypi_lib/site-packages/libfile/lib.txt")], kind = VenvSymlinkKind.LIB, ), _entry( "binfile", "+pypi_bin/bin/binfile", - ["bin.txt"], + [_file("../+pypi_bin/bin/binfile/bin.txt")], kind = VenvSymlinkKind.BIN, ), _entry( "includefile", "+pypi_include/include/includefile", - ["include.h"], + [_file("../+pypi_include/include/includefile/include.h")], kind = VenvSymlinkKind.INCLUDE, ), From 4558ffb0c6f97044dc2fb9731454a4797e549fcf Mon Sep 17 00:00:00 2001 From: Ignas Anikevicius <240938+aignas@users.noreply.github.com> Date: Mon, 15 Dec 2025 07:39:22 +0900 Subject: [PATCH 555/922] refactor(repo_utils): create a helper for extracting files (#3459) This just moves common code to the same place, so that we can better maintain extracting from the whls easier. Work towards #2945 --- python/private/pypi/BUILD.bazel | 1 + python/private/pypi/patch_whl.bzl | 17 +++++++---------- python/private/pypi/whl_library.bzl | 13 ++++--------- python/private/repo_utils.bzl | 23 +++++++++++++++++++++++ 4 files changed, 35 insertions(+), 19 deletions(-) diff --git a/python/private/pypi/BUILD.bazel b/python/private/pypi/BUILD.bazel index 7d5314dd62..dd86dcfbc6 100644 --- a/python/private/pypi/BUILD.bazel +++ b/python/private/pypi/BUILD.bazel @@ -247,6 +247,7 @@ bzl_library( deps = [ ":parse_whl_name_bzl", "//python/private:repo_utils_bzl", + "@rules_python_internal//:rules_python_config_bzl", ], ) diff --git a/python/private/pypi/patch_whl.bzl b/python/private/pypi/patch_whl.bzl index e315989dd9..71b46f6e7c 100644 --- a/python/private/pypi/patch_whl.bzl +++ b/python/private/pypi/patch_whl.bzl @@ -27,6 +27,8 @@ other patches ensures that the users have overview on exactly what has changed within the wheel. """ +load("@rules_python_internal//:rules_python_config.bzl", rp_config = "config") +load("//python/private:repo_utils.bzl", "repo_utils") load(":parse_whl_name.bzl", "parse_whl_name") load(":pypi_repo_utils.bzl", "pypi_repo_utils") @@ -84,16 +86,11 @@ def patch_whl(rctx, *, python_interpreter, whl_path, patches, **kwargs): # does not support patching in another directory. whl_input = rctx.path(whl_path) - # symlink to a zip file to use bazel's extract so that we can use bazel's - # repository_ctx patch implementation. The whl file may be in a different - # external repository. - # - # TODO @aignas 2025-11-24: remove this symlinking workaround when we drop support for bazel 7 - whl_file_zip = whl_input.basename + ".zip" - rctx.symlink(whl_input, whl_file_zip) - rctx.extract(whl_file_zip) - if not rctx.delete(whl_file_zip): - fail("Failed to remove the symlink after extracting") + repo_utils.extract( + rctx, + archive = whl_input, + supports_whl_extraction = rp_config.supports_whl_extraction, + ) if not patches: fail("Trying to patch wheel without any patches") diff --git a/python/private/pypi/whl_library.bzl b/python/private/pypi/whl_library.bzl index 044d18a0df..fdb3f93231 100644 --- a/python/private/pypi/whl_library.bzl +++ b/python/private/pypi/whl_library.bzl @@ -377,17 +377,12 @@ def _whl_library_impl(rctx): # # Remove non-pipstar and config_load check when we release rules_python 2. if enable_pipstar: - if rp_config.supports_whl_extraction: - extract_path = whl_path - else: - extract_path = rctx.path(whl_path.basename + ".zip") - rctx.symlink(whl_path, extract_path) - rctx.extract( - archive = extract_path, + repo_utils.extract( + rctx, + archive = whl_path, output = "site-packages", + supports_whl_extraction = rp_config.supports_whl_extraction, ) - if not rp_config.supports_whl_extraction: - rctx.delete(extract_path) metadata = whl_metadata( install_dir = whl_path.dirname.get_child("site-packages"), diff --git a/python/private/repo_utils.bzl b/python/private/repo_utils.bzl index 77eac55c16..1abff36a04 100644 --- a/python/private/repo_utils.bzl +++ b/python/private/repo_utils.bzl @@ -429,11 +429,34 @@ def _get_platforms_cpu_name(mrctx): return "riscv64" return arch +def _extract(mrctx, *, archive, supports_whl_extraction = False, **kwargs): + """Extract an archive + + TODO: remove when the earliest supported bazel version is at least 8.3. + + Note, we are using the parameter here because there is very little ways how we can detect + whether we can support just extracting the whl. + """ + archive_original = None + if not supports_whl_extraction and archive.basename.endswith(".whl"): + archive_original = archive + archive = mrctx.path(archive.basename + ".zip") + mrctx.symlink(archive_original, archive) + + mrctx.extract( + archive = archive, + **kwargs + ) + if archive_original: + if not mrctx.delete(archive): + fail("Failed to remove the symlink after extracting") + repo_utils = struct( # keep sorted execute_checked = _execute_checked, execute_checked_stdout = _execute_checked_stdout, execute_unchecked = _execute_unchecked, + extract = _extract, get_platforms_cpu_name = _get_platforms_cpu_name, get_platforms_os_name = _get_platforms_os_name, getenv = _getenv, From 1198422c8986c10af8e9313b2570e8a4dd2dc2b1 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Sun, 14 Dec 2025 15:27:50 -0800 Subject: [PATCH 556/922] refactor: optimize venv building for namespace packages (#3454) When implicit namespace packages are used, it's common for multiple distributions to install into the same directory, triggering the expensive conflict merging logic. This can be observed wit our doc builds, where `sphinxcontrib` is a namespace package that 7 distributions install into. To fix, treat top-level directories that have an importable name and don't have an `__init__` looking file as implicit namespace packages and mark them as disallowed from being directly linked. The importable name check is to exclude dist-info directories. --------- Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> --- python/private/venv_runfiles.bzl | 42 ++++++++++++ .../app_files_building_tests.bzl | 66 +++++++++++++++++++ 2 files changed, 108 insertions(+) diff --git a/python/private/venv_runfiles.bzl b/python/private/venv_runfiles.bzl index 0c79ea82a1..7ff5c8512c 100644 --- a/python/private/venv_runfiles.bzl +++ b/python/private/venv_runfiles.bzl @@ -13,6 +13,7 @@ load( "VenvSymlinkEntry", "VenvSymlinkKind", ) +load(":py_internal.bzl", "py_internal") def create_venv_app_files(ctx, deps, venv_dir_map): """Creates the tree of app-specific files for a venv for a binary. @@ -231,6 +232,21 @@ def _merge_venv_path_group(ctx, group, keep_map): if venv_path not in keep_map: keep_map[venv_path] = file +def _is_importable_name(name): + # Requires Bazel 8+ + if hasattr(py_internal, "regex_match"): + # ?U means activates unicode matching (Python allows most unicode + # in module names / identifiers). + # \w matches alphanumeric and underscore. + # NOTE: regex_match has an implicit ^ and $ + return py_internal.regex_match(name, "(?U)\\w+") + else: + # Otherwise, use a rough hueristic that should catch most cases. + return ( + "." not in name and + "-" not in name + ) + def get_venv_symlinks(ctx, files, package, version_str, site_packages_root): """Compute the VenvSymlinkEntry objects for a library. @@ -270,6 +286,9 @@ def get_venv_symlinks(ctx, files, package, version_str, site_packages_root): # List of (File, str venv_path) tuples files_left_to_link = [] + # dict[str dirname, bool is_namespace_package] + namespace_package_dirs = {} + # We want to minimize the number of files symlinked. Ideally, only the # top-level directories are symlinked. Unfortunately, shared libraries # complicate matters: if a shared library's directory is linked, then the @@ -310,6 +329,29 @@ def get_venv_symlinks(ctx, files, package, version_str, site_packages_root): else: files_left_to_link.append((src, venv_path)) + top_level_dirname, _, tail = venv_path.partition("/") + if ( + # If it's already not directly linkable, nothing to do + not cannot_be_linked_directly.get(top_level_dirname, False) and + # If its already known to be non-implicit namespace, then skip + namespace_package_dirs.get(top_level_dirname, True) and + # It must be an importable name to be an implicit namespace package + _is_importable_name(top_level_dirname) + ): + namespace_package_dirs.setdefault(top_level_dirname, True) + + # Looking for `__init__.` isn't 100% correct, as it'll match e.g. + # `__init__.pyi`, but it's close enough. + if "/" not in tail and tail.startswith("__init__."): + namespace_package_dirs[top_level_dirname] = False + + # We treat namespace packages as a hint that other distributions may + # install into the same directory. As such, we avoid linking them directly + # to avoid conflict merging later. + for dirname, is_namespace_package in namespace_package_dirs.items(): + if is_namespace_package: + cannot_be_linked_directly[dirname] = True + # At this point, venv_symlinks has entries for the shared libraries # and cannot_be_linked_directly has the directories that cannot be # directly linked. Next, we loop over the remaining files and group diff --git a/tests/venv_site_packages_libs/app_files_building/app_files_building_tests.bzl b/tests/venv_site_packages_libs/app_files_building/app_files_building_tests.bzl index 486293b38d..e92c0aaf5a 100644 --- a/tests/venv_site_packages_libs/app_files_building/app_files_building_tests.bzl +++ b/tests/venv_site_packages_libs/app_files_building/app_files_building_tests.bzl @@ -219,6 +219,7 @@ def _test_optimized_grouping_single_toplevel(name): empty_files( name = name + "_files", paths = [ + "site-packages/pkg2/__init__.py", "site-packages/pkg2/a.txt", "site-packages/pkg2/b_mod.so", ], @@ -248,6 +249,7 @@ def _test_optimized_grouping_single_toplevel_impl(env, target): "pkg2", link_to_path = rr + "pkg2", files = [ + "tests/venv_site_packages_libs/app_files_building/site-packages/pkg2/__init__.py", "tests/venv_site_packages_libs/app_files_building/site-packages/pkg2/a.txt", "tests/venv_site_packages_libs/app_files_building/site-packages/pkg2/b_mod.so", ], @@ -264,6 +266,70 @@ def _test_optimized_grouping_single_toplevel_impl(env, target): # The point of the optimization is to avoid having to merge conflicts. env.expect.that_collection(conflicts).contains_exactly([]) +def _test_optimized_grouping_implicit_namespace_packages(name): + empty_files( + name = name + "_files", + paths = [ + # NOTE: An alphanumeric name with underscores is used to verify + # name matching is correct. + "site-packages/name_space9/part1/foo.py", + "site-packages/name_space9/part2/bar.py", + "site-packages/name_space9-1.0.dist-info/METADATA", + ], + ) + analysis_test( + name = name, + impl = _test_optimized_grouping_implicit_namespace_packages_impl, + target = name + "_files", + ) + +_tests.append(_test_optimized_grouping_implicit_namespace_packages) + +def _test_optimized_grouping_implicit_namespace_packages_impl(env, target): + test_ctx = _ctx(workspace_name = env.ctx.workspace_name) + entries = get_venv_symlinks( + test_ctx, + target.files.to_list(), + package = "pkg3", + version_str = "1.0", + site_packages_root = env.ctx.label.package + "/site-packages", + ) + actual = _venv_symlinks_from_entries(entries) + + rr = "{}/{}/site-packages/".format(test_ctx.workspace_name, env.ctx.label.package) + expected = [ + _venv_symlink( + "name_space9/part1", + link_to_path = rr + "name_space9/part1", + files = [ + "tests/venv_site_packages_libs/app_files_building/site-packages/name_space9/part1/foo.py", + ], + ), + _venv_symlink( + "name_space9/part2", + link_to_path = rr + "name_space9/part2", + files = [ + "tests/venv_site_packages_libs/app_files_building/site-packages/name_space9/part2/bar.py", + ], + ), + _venv_symlink( + "name_space9-1.0.dist-info", + link_to_path = rr + "name_space9-1.0.dist-info", + files = [ + "tests/venv_site_packages_libs/app_files_building/site-packages/name_space9-1.0.dist-info/METADATA", + ], + ), + ] + expected = sorted(expected, key = lambda e: (e.link_to_path, e.venv_path)) + env.expect.that_collection( + actual, + ).contains_exactly(expected) + + _, conflicts = build_link_map(test_ctx, entries, return_conflicts = True) + + # The point of the optimization is to avoid having to merge conflicts. + env.expect.that_collection(conflicts).contains_exactly([]) + def _test_package_version_filtering(name): analysis_test( name = name, From b8d2e9b6f6286bf9a6f295bca532fb1504d3c61d Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Sun, 14 Dec 2025 16:58:52 -0800 Subject: [PATCH 557/922] chore: add missing py_internal dep to venv_runfiles (#3462) Add the missing py_internal.bzl dep to venv_runfiles.bzl --- python/private/BUILD.bazel | 1 + 1 file changed, 1 insertion(+) diff --git a/python/private/BUILD.bazel b/python/private/BUILD.bazel index 7b8c82de8f..44048578c6 100644 --- a/python/private/BUILD.bazel +++ b/python/private/BUILD.bazel @@ -724,6 +724,7 @@ bzl_library( deps = [ ":common_bzl", ":py_info.bzl", + ":py_internal.bzl", "@bazel_skylib//lib:paths", ], ) From 1090b6cbf3a1ce2261e8ab2ef2f61df072245dbf Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Sun, 14 Dec 2025 17:29:41 -0800 Subject: [PATCH 558/922] fix: allow pypi packages with empty data attribute (#3463) An empty glob under site-packages (after excluding code and pyi files) is a bit strange, but not actually incorrect. If such a situation occurs, it would cause a failure because Bazel has since changed to disallow empty globs by default. To fix, just set allow_empty=True on the data glob. --- python/private/pypi/whl_library_targets.bzl | 1 + tests/pypi/whl_library_targets/whl_library_targets_tests.bzl | 3 ++- 2 files changed, 3 insertions(+), 1 deletion(-) diff --git a/python/private/pypi/whl_library_targets.bzl b/python/private/pypi/whl_library_targets.bzl index a2d77daf4c..39b1cccd5e 100644 --- a/python/private/pypi/whl_library_targets.bzl +++ b/python/private/pypi/whl_library_targets.bzl @@ -356,6 +356,7 @@ def whl_library_targets( data = data + native.glob( ["site-packages/**/*"], exclude = _data_exclude, + allow_empty = True, ) pyi_srcs = native.glob( diff --git a/tests/pypi/whl_library_targets/whl_library_targets_tests.bzl b/tests/pypi/whl_library_targets/whl_library_targets_tests.bzl index b7fb9094d7..1d80340a13 100644 --- a/tests/pypi/whl_library_targets/whl_library_targets_tests.bzl +++ b/tests/pypi/whl_library_targets/whl_library_targets_tests.bzl @@ -275,6 +275,7 @@ def _test_whl_and_library_deps_from_requires(env): "**/*.pyc.*", "**/*.dist-info/RECORD", ], + allow_empty = True, ), # pyi call _glob_call(["site-packages/**/*.pyi"], allow_empty = True), @@ -457,7 +458,7 @@ def _test_group(env): "**/*.pyc", "**/*.pyc.*", "**/*.dist-info/RECORD", - ]), + ], allow_empty = True), _glob_call(["site-packages/**/*.pyi"], allow_empty = True), ]) From fe9fd2d24a9f7710738a39accd7af3154cf87db5 Mon Sep 17 00:00:00 2001 From: Will Stranton <2659963+willstranton@users.noreply.github.com> Date: Mon, 15 Dec 2025 13:09:52 +0000 Subject: [PATCH 559/922] docs: Fix broken links to bzlmod build_file_generation example (#3464) The `examples/bzlmod_build_file_generation` directory was moved under the gazelle directory in f92ad7136ff411755ec8d8361af8263fd8efe6f2 This fixes some outdated links/references to that old directory --- .bazelignore | 1 - BZLMOD_SUPPORT.md | 2 +- examples/build_file_generation/.bazelrc | 2 +- gazelle/docs/installation_and_usage.md | 2 +- 4 files changed, 3 insertions(+), 4 deletions(-) diff --git a/.bazelignore b/.bazelignore index 2f50cc2c52..0384d0746e 100644 --- a/.bazelignore +++ b/.bazelignore @@ -19,7 +19,6 @@ examples/bzlmod/other_module/bazel-other_module examples/bzlmod/other_module/bazel-out examples/bzlmod/other_module/bazel-testlogs examples/bzlmod/py_proto_library/foo_external -examples/bzlmod_build_file_generation/bazel-bzlmod_build_file_generation examples/multi_python_versions/bazel-multi_python_versions examples/pip_parse/bazel-pip_parse examples/pip_parse_vendored/bazel-pip_parse_vendored diff --git a/BZLMOD_SUPPORT.md b/BZLMOD_SUPPORT.md index 73fde463b7..1e238da774 100644 --- a/BZLMOD_SUPPORT.md +++ b/BZLMOD_SUPPORT.md @@ -27,7 +27,7 @@ We have two examples that demonstrate how to configure `bzlmod`. The first example is in [examples/bzlmod](examples/bzlmod), and it demonstrates basic bzlmod configuration. A user does not use `local_path_override` stanza and would define the version in the `bazel_dep` line. -A second example, in [examples/bzlmod_build_file_generation](examples/bzlmod_build_file_generation) demonstrates the use of `bzlmod` to configure `gazelle` support for `rules_python`. +A second example, in [gazelle/examples/bzlmod_build_file_generation](gazelle/examples/bzlmod_build_file_generation) demonstrates the use of `bzlmod` to configure `gazelle` support for `rules_python`. ## Differences in behavior from WORKSPACE diff --git a/examples/build_file_generation/.bazelrc b/examples/build_file_generation/.bazelrc index 306954d7be..f1ae44fac8 100644 --- a/examples/build_file_generation/.bazelrc +++ b/examples/build_file_generation/.bazelrc @@ -3,7 +3,7 @@ test --test_output=errors --enable_runfiles # Windows requires these for multi-python support: build --enable_runfiles -# The bzlmod version of this example is in examples/bzlmod_build_file_generation +# The bzlmod version of this example is in gazelle/examples/bzlmod_build_file_generation # Once WORKSPACE support is dropped, this example can be entirely deleted. common --noenable_bzlmod common --enable_workspace diff --git a/gazelle/docs/installation_and_usage.md b/gazelle/docs/installation_and_usage.md index b151ade25e..c3a004310c 100644 --- a/gazelle/docs/installation_and_usage.md +++ b/gazelle/docs/installation_and_usage.md @@ -5,7 +5,7 @@ Examples of using Gazelle with Python can be found in the `rules_python` repo: -* bzlmod: {gh-path}`examples/bzlmod_build_file_generation` +* bzlmod: {gh-path}`gazelle/examples/bzlmod_build_file_generation` * WORKSPACE: {gh-path}`examples/build_file_generation` :::{note} From 10b28efc376d5810a08afa0d3ba873d89f7436a8 Mon Sep 17 00:00:00 2001 From: Will Stranton <2659963+willstranton@users.noreply.github.com> Date: Tue, 16 Dec 2025 10:36:18 +0000 Subject: [PATCH 560/922] docs: fix link to releases page (#3467) Previously, the link was an internal absolute link, which is relative to the repository's root directory, but the releases page isn't part of the source tree. --- BZLMOD_SUPPORT.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/BZLMOD_SUPPORT.md b/BZLMOD_SUPPORT.md index 1e238da774..49010b838e 100644 --- a/BZLMOD_SUPPORT.md +++ b/BZLMOD_SUPPORT.md @@ -11,7 +11,7 @@ In general `bzlmod` has more features than `WORKSPACE` and users are encouraged ## Configuration -The releases page will give you the latest version number, and a basic example. The release page is located [here](/bazel-contrib/rules_python/releases). +The releases page will give you the latest version number, and a basic example. The release page is located [here](https://github.com/bazel-contrib/rules_python/releases). ## What is bzlmod? From ca2c5b2273bcda68c6e2f5ab653501ac849b719b Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Wed, 17 Dec 2025 00:31:23 -0800 Subject: [PATCH 561/922] refactor: optimize venv creation for nvidia and pkgutil style namespace packages (#3460) When pkgutil style namespace packages are used, multiple distributions provide the same venv path (e.g. `foo/__init__.py`). The venv symlink logic then tries to symlink the `foo/` directory as it looks like the highest linkable directory. When conflict merging logic runs later, it then has to flatten a depset with all the files in the conflicting distributions. To fix, have whl_library() try to guess when a file is a pkgutil namespace package. These are then pass onto py_library's venv building logic so it can treat the directories as not directly linkable. A conflict still occurs, but it only contains the single `__init__.py` file. Along the way, special case the "nvidia" package name and always treat it as a namespace package. This is because nvidia packages aren't strictly correct: each has a blank `__init__.py` file (which marks it as a regular package, not namespace package). Special casing like this is undesirable, but it greatly reduces the number of conflicts if e.g. torch is installed, and I couldn't find any other metadata to indicate it's a namespace package. Along the way, add some hints to AGENTS.md so they understand repository rules better. Fixes https://github.com/bazel-contrib/rules_python/issues/3401 --------- Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> --- .bazelrc | 2 + AGENTS.md | 62 ++++++ CHANGELOG.md | 2 + MODULE.bazel | 2 + python/private/BUILD.bazel | 1 + python/private/internal_dev_deps.bzl | 24 +++ python/private/py_library.bzl | 15 ++ python/private/pypi/BUILD.bazel | 4 +- python/private/pypi/pypi_repo_utils.bzl | 36 ++++ python/private/pypi/whl_library.bzl | 8 +- python/private/pypi/whl_library_targets.bzl | 8 +- python/private/util.bzl | 16 ++ python/private/venv_runfiles.bzl | 91 ++++++--- .../whl_library_targets_tests.bzl | 12 ++ tests/repos/BUILD.bazel | 0 tests/repos/pkgutil_nspkg1/BUILD.bazel | 0 tests/repos/pkgutil_nspkg1/nspkg/__init__.py | 2 + tests/repos/pkgutil_nspkg1/nspkg/one/a.txt | 1 + .../pkgutil_nspkg1-1.0.dist-info/METADATA | 3 + .../pkgutil_nspkg1-1.0.dist-info/RECORD | 5 + .../pkgutil_nspkg1-1.0.dist-info/WHEEL | 4 + tests/repos/pkgutil_nspkg2/BUILD.bazel | 0 tests/repos/pkgutil_nspkg2/nspkg/__init__.py | 2 + tests/repos/pkgutil_nspkg2/nspkg/two/b.txt | 1 + .../pkgutil_nspkg2-1.0.dist-info/METADATA | 3 + .../pkgutil_nspkg2-1.0.dist-info/RECORD | 5 + .../pkgutil_nspkg2-1.0.dist-info/WHEEL | 4 + .../app_files_building_tests.bzl | 177 ++++++++++++++++++ 28 files changed, 458 insertions(+), 32 deletions(-) create mode 100644 tests/repos/BUILD.bazel create mode 100644 tests/repos/pkgutil_nspkg1/BUILD.bazel create mode 100644 tests/repos/pkgutil_nspkg1/nspkg/__init__.py create mode 100644 tests/repos/pkgutil_nspkg1/nspkg/one/a.txt create mode 100644 tests/repos/pkgutil_nspkg1/pkgutil_nspkg1-1.0.dist-info/METADATA create mode 100644 tests/repos/pkgutil_nspkg1/pkgutil_nspkg1-1.0.dist-info/RECORD create mode 100644 tests/repos/pkgutil_nspkg1/pkgutil_nspkg1-1.0.dist-info/WHEEL create mode 100644 tests/repos/pkgutil_nspkg2/BUILD.bazel create mode 100644 tests/repos/pkgutil_nspkg2/nspkg/__init__.py create mode 100644 tests/repos/pkgutil_nspkg2/nspkg/two/b.txt create mode 100644 tests/repos/pkgutil_nspkg2/pkgutil_nspkg2-1.0.dist-info/METADATA create mode 100644 tests/repos/pkgutil_nspkg2/pkgutil_nspkg2-1.0.dist-info/RECORD create mode 100644 tests/repos/pkgutil_nspkg2/pkgutil_nspkg2-1.0.dist-info/WHEEL diff --git a/.bazelrc b/.bazelrc index 6473f10231..718f83080c 100644 --- a/.bazelrc +++ b/.bazelrc @@ -18,6 +18,8 @@ build --//python/config_settings:incompatible_default_to_explicit_init_py=True # Ensure ongoing compatibility with this flag. common --incompatible_disallow_struct_provider_syntax +# Makes Bazel 7 act more like Bazel 8 +common --incompatible_use_plus_in_repo_names # Windows makes use of runfiles for some rules build --enable_runfiles diff --git a/AGENTS.md b/AGENTS.md index 671b85c6bd..c1c9f7902b 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -27,6 +27,68 @@ end of the documentation text. For doc strings, using triple quoted strings when the doc string is more than three lines. Do not use a trailing backslack (`\`) for the opening triple-quote. +### Starlark Code + +Starlark does not support recursion. Use iterative algorithms instead. + +Starlark does not support `while` loops. Use `for` loop with an appropriately +sized iterable instead. + +#### Starlark testing + +For Starlark tests: + +* Use `rules_testing`, not `bazel_skylib`. +* See https://rules-testing.readthedocs.io/en/latest/analysis_tests.html for + examples on using rules_testing. +* See `tests/builders/builders_tests.bzl` for an example of using it in + this project. + +A test is defined in two parts: + * A setup function, e.g. `def _test_foo(name)`. This defines targets + and calls `analysis_test`. + * An implementation function, e.g. `def _test_foo_impl(env, target)`. This + contains asserts. + +Example: + +``` +# File: foo_tests.bzl + +load("@rules_testing//lib:analysis_test.bzl", "analysis_test") +load("@rules_testing//lib:test_suite.bzl", "test_suite") + +_tests = [] + +def _test_foo(name): + foo_library( + name = name + "_subject", + ) + analysis_test( + name = name, + impl = _test_foo_impl, + target = name + "_subject", + ) +_tests.append(_test_foo) + +def _test_foo_impl(env, target): + env.expect.that_whatever(target[SomeInfo].whatever).equals(expected) + +def foo_test_suite(name): + test_suite(name=name, tests=_tests) +``` + + +#### Repository rules + +The function argument `rctx` is a hint that the function is a repository rule, +or used by a repository rule. + +The function argument `mrctx` is a hint that the function can be used by a +repository rule or module extension. + +The `repository_ctx` API docs are at: https://bazel.build/rules/lib/builtins/repository_ctx + ### bzl_library targets for bzl source files * A `bzl_library` target should be defined for every `.bzl` file outside diff --git a/CHANGELOG.md b/CHANGELOG.md index 87ae3f5ac4..69e159c6c7 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -118,6 +118,8 @@ END_UNRELEASED_TEMPLATE `RULES_PYTHON_ENABLE_PIPSTAR=1` by default. Users of `experimental_index_url` that perform cross-builds should add {obj}`target_platforms` to their `pip.parse` invocations, which will become mandatory if any cross-builds are required from the next release. +* (py_library) Attribute {obj}`namespace_package_files` added. It is a hint for + optimizing venv creation. [20251031]: https://github.com/astral-sh/python-build-standalone/releases/tag/20251031 [20251202]: https://github.com/astral-sh/python-build-standalone/releases/tag/20251202 diff --git a/MODULE.bazel b/MODULE.bazel index ef7499eb0c..80c7ab1d99 100644 --- a/MODULE.bazel +++ b/MODULE.bazel @@ -238,6 +238,8 @@ use_repo( "buildkite_config", "implicit_namespace_ns_sub1", "implicit_namespace_ns_sub2", + "pkgutil_nspkg1", + "pkgutil_nspkg2", "rules_python_runtime_env_tc_info", "somepkg_with_build_files", "whl_with_build_files", diff --git a/python/private/BUILD.bazel b/python/private/BUILD.bazel index 44048578c6..e92c45dad4 100644 --- a/python/private/BUILD.bazel +++ b/python/private/BUILD.bazel @@ -675,6 +675,7 @@ bzl_library( "//:__subpackages__", ], deps = [ + ":py_internal_bzl", "@bazel_skylib//lib:types", ], ) diff --git a/python/private/internal_dev_deps.bzl b/python/private/internal_dev_deps.bzl index d621a5d941..0e21d8b8a5 100644 --- a/python/private/internal_dev_deps.bzl +++ b/python/private/internal_dev_deps.bzl @@ -67,6 +67,30 @@ def _internal_dev_deps_impl(mctx): enable_implicit_namespace_pkgs = False, ) + whl_from_dir_repo( + name = "pkgutil_nspkg1_whl", + root = "//tests/repos/pkgutil_nspkg1:BUILD.bazel", + output = "pkgutil_nspkg1-1.0-any-none-any.whl", + ) + whl_library( + name = "pkgutil_nspkg1", + whl_file = "@pkgutil_nspkg1_whl//:pkgutil_nspkg1-1.0-any-none-any.whl", + requirement = "pkgutil_nspkg1", + enable_implicit_namespace_pkgs = False, + ) + + whl_from_dir_repo( + name = "pkgutil_nspkg2_whl", + root = "//tests/repos/pkgutil_nspkg2:BUILD.bazel", + output = "pkgutil_nspkg2-1.0-any-none-any.whl", + ) + whl_library( + name = "pkgutil_nspkg2", + whl_file = "@pkgutil_nspkg2_whl//:pkgutil_nspkg2-1.0-any-none-any.whl", + requirement = "pkgutil_nspkg2", + enable_implicit_namespace_pkgs = False, + ) + internal_dev_deps = module_extension( implementation = _internal_dev_deps_impl, doc = "This extension creates internal rules_python dev dependencies.", diff --git a/python/private/py_library.bzl b/python/private/py_library.bzl index b2a9fdd3be..7e9a59f277 100644 --- a/python/private/py_library.bzl +++ b/python/private/py_library.bzl @@ -99,6 +99,20 @@ The topological order has been removed and if 2 different versions of the same P package are observed, the behaviour has no guarantees except that it is deterministic and that only one package version will be included. ::: +""", + ), + "namespace_package_files": lambda: attrb.LabelList( + allow_empty = True, + allow_files = True, + doc = """ +Files whose directories are namespace packages. + +When {obj}`--venv_site_packages=yes` is set, this helps inform which directories should be +treated as namespace packages and expect files from other targets to be contributed. +This allows optimizing the generation of symlinks to be cheaper at analysis time. + +:::{versionadded} VERSION_NEXT_FEATURE +::: """, ), "_add_srcs_to_runfiles_flag": lambda: attrb.Label( @@ -251,6 +265,7 @@ def _get_imports_and_venv_symlinks(ctx, semantics): package, version_str, site_packages_root = imports[0], + namespace_package_files = ctx.files.namespace_package_files, ) else: imports = collect_imports(ctx, semantics) diff --git a/python/private/pypi/BUILD.bazel b/python/private/pypi/BUILD.bazel index dd86dcfbc6..18caa53d7f 100644 --- a/python/private/pypi/BUILD.bazel +++ b/python/private/pypi/BUILD.bazel @@ -431,16 +431,16 @@ bzl_library( ":attrs_bzl", ":deps_bzl", ":generate_whl_library_build_bazel_bzl", + ":parse_whl_name_bzl", ":patch_whl_bzl", - ":pep508_requirement_bzl", ":pypi_repo_utils_bzl", ":whl_metadata_bzl", ":whl_target_platforms_bzl", "//python/private:auth_bzl", - "//python/private:bzlmod_enabled_bzl", "//python/private:envsubst_bzl", "//python/private:is_standalone_interpreter_bzl", "//python/private:repo_utils_bzl", + "//python/private:util_bzl", "@rules_python_internal//:rules_python_config_bzl", ], ) diff --git a/python/private/pypi/pypi_repo_utils.bzl b/python/private/pypi/pypi_repo_utils.bzl index 04c9b5d685..d8e320014f 100644 --- a/python/private/pypi/pypi_repo_utils.bzl +++ b/python/private/pypi/pypi_repo_utils.bzl @@ -16,6 +16,7 @@ load("@bazel_skylib//lib:types.bzl", "types") load("//python/private:repo_utils.bzl", "repo_utils") +load("//python/private:util.bzl", "is_importable_name") def _get_python_interpreter_attr(mrctx, *, python_interpreter = None): """A helper function for getting the `python_interpreter` attribute or it's default @@ -161,9 +162,44 @@ def _execute_checked_stdout(mrctx, *, python, srcs, **kwargs): **_execute_prep(mrctx, python = python, srcs = srcs, **kwargs) ) +def _find_namespace_package_files(rctx, install_dir): + """Finds all `__init__.py` files that belong to namespace packages. + + A `__init__.py` file belongs to a namespace package if it contains `__path__ =`, + `pkgutil`, and `extend_path(`. + + Args: + rctx (repository_ctx): The repository context. + install_dir (path): The path to the install directory. + + Returns: + list[str]: A list of relative paths to `__init__.py` files that belong + to namespace packages. + """ + + repo_root = str(rctx.path(".")) + "/" + namespace_package_files = [] + for top_level_dir in install_dir.readdir(): + if not is_importable_name(top_level_dir.basename): + continue + init_py = top_level_dir.get_child("__init__.py") + if not init_py.exists: + continue + content = rctx.read(init_py) + + # Look for code resembling the pkgutil namespace setup code: + # __path__ = __import__("pkgutil").extend_path(__path__, __name__) + if ("__path__ =" in content and + "pkgutil" in content and + "extend_path(" in content): + namespace_package_files.append(str(init_py).removeprefix(repo_root)) + + return namespace_package_files + pypi_repo_utils = struct( construct_pythonpath = _construct_pypath, execute_checked = _execute_checked, execute_checked_stdout = _execute_checked_stdout, + find_namespace_package_files = _find_namespace_package_files, resolve_python_interpreter = _resolve_python_interpreter, ) diff --git a/python/private/pypi/whl_library.bzl b/python/private/pypi/whl_library.bzl index fdb3f93231..201ef59b89 100644 --- a/python/private/pypi/whl_library.bzl +++ b/python/private/pypi/whl_library.bzl @@ -384,11 +384,13 @@ def _whl_library_impl(rctx): supports_whl_extraction = rp_config.supports_whl_extraction, ) + install_dir_path = whl_path.dirname.get_child("site-packages") metadata = whl_metadata( - install_dir = whl_path.dirname.get_child("site-packages"), + install_dir = install_dir_path, read_fn = rctx.read, logger = logger, ) + namespace_package_files = pypi_repo_utils.find_namespace_package_files(rctx, install_dir_path) # NOTE @aignas 2024-06-22: this has to live on until we stop supporting # passing `twine` as a `:pkg` library via the `WORKSPACE` builds. @@ -432,6 +434,7 @@ def _whl_library_impl(rctx): data_exclude = rctx.attr.pip_data_exclude, group_deps = rctx.attr.group_deps, group_name = rctx.attr.group_name, + namespace_package_files = namespace_package_files, ) else: target_platforms = rctx.attr.experimental_target_platforms or [] @@ -491,6 +494,8 @@ def _whl_library_impl(rctx): ) entry_points[entry_point_without_py] = entry_point_script_name + namespace_package_files = pypi_repo_utils.find_namespace_package_files(rctx, rctx.path("site-packages")) + build_file_contents = generate_whl_library_build_bazel( name = whl_path.basename, sdist_filename = sdist_filename, @@ -509,6 +514,7 @@ def _whl_library_impl(rctx): "pypi_name={}".format(metadata["name"]), "pypi_version={}".format(metadata["version"]), ], + namespace_package_files = namespace_package_files, ) # Delete these in case the wheel had them. They generally don't cause diff --git a/python/private/pypi/whl_library_targets.bzl b/python/private/pypi/whl_library_targets.bzl index 39b1cccd5e..0fe2c52d9f 100644 --- a/python/private/pypi/whl_library_targets.bzl +++ b/python/private/pypi/whl_library_targets.bzl @@ -123,6 +123,7 @@ def whl_library_targets( entry_points = {}, native = native, enable_implicit_namespace_pkgs = False, + namespace_package_files = [], rules = struct( copy_file = copy_file, py_binary = py_binary, @@ -169,6 +170,8 @@ def whl_library_targets( enable_implicit_namespace_pkgs: {type}`boolean` generate __init__.py files for namespace pkgs. native: {type}`native` The native struct for overriding in tests. + namespace_package_files: {type}`list[str]` A list of labels of files whose + directories are namespace packages. rules: {type}`struct` A struct with references to rules for creating targets. """ dependencies = sorted([normalize_name(d) for d in dependencies]) @@ -365,7 +368,7 @@ def whl_library_targets( ) if not enable_implicit_namespace_pkgs: - srcs = srcs + select({ + generated_namespace_package_files = select({ Label("//python/config_settings:is_venvs_site_packages"): [], "//conditions:default": rules.create_inits( srcs = srcs + data + pyi_srcs, @@ -373,6 +376,8 @@ def whl_library_targets( root = "site-packages", ), }) + namespace_package_files += generated_namespace_package_files + srcs = srcs + generated_namespace_package_files rules.py_library( name = py_library_label, @@ -391,6 +396,7 @@ def whl_library_targets( tags = tags, visibility = impl_vis, experimental_venvs_site_packages = Label("@rules_python//python/config_settings:venvs_site_packages"), + namespace_package_files = namespace_package_files, ) def _config_settings(dependencies_by_platform, dependencies_with_markers, rules, native = native, **kwargs): diff --git a/python/private/util.bzl b/python/private/util.bzl index d3053fe626..31f317fedf 100644 --- a/python/private/util.bzl +++ b/python/private/util.bzl @@ -15,6 +15,7 @@ """Functionality shared by multiple pieces of code.""" load("@bazel_skylib//lib:types.bzl", "types") +load("//python/private:py_internal.bzl", "py_internal") def copy_propagating_kwargs(from_kwargs, into_kwargs = None): """Copies args that must be compatible between two targets with a dependency relationship. @@ -69,3 +70,18 @@ def add_tag(attrs, tag): attrs["tags"] = tags + [tag] else: attrs["tags"] = [tag] + +def is_importable_name(name): + # Requires Bazel 8+ + if hasattr(py_internal, "regex_match"): + # ?U means activates unicode matching (Python allows most unicode + # in module names / identifiers). + # \w matches alphanumeric and underscore. + # NOTE: regex_match has an implicit ^ and $ + return py_internal.regex_match(name, "(?U)\\w+") + else: + # Otherwise, use a rough hueristic that should catch most cases. + return ( + "." not in name and + "-" not in name + ) diff --git a/python/private/venv_runfiles.bzl b/python/private/venv_runfiles.bzl index 7ff5c8512c..851d7015c6 100644 --- a/python/private/venv_runfiles.bzl +++ b/python/private/venv_runfiles.bzl @@ -13,7 +13,15 @@ load( "VenvSymlinkEntry", "VenvSymlinkKind", ) -load(":py_internal.bzl", "py_internal") +load(":util.bzl", "is_importable_name") + +# List of top-level package names that are known to be namespace +# packages, but cannot be detected as such automatically. +_WELL_KNOWN_NAMESPACE_PACKAGES = [ + # nvidia wheels incorrectly use an empty `__init__.py` file, even + # though multiple distributions install into the directory. + "nvidia", +] def create_venv_app_files(ctx, deps, venv_dir_map): """Creates the tree of app-specific files for a venv for a binary. @@ -232,22 +240,34 @@ def _merge_venv_path_group(ctx, group, keep_map): if venv_path not in keep_map: keep_map[venv_path] = file -def _is_importable_name(name): - # Requires Bazel 8+ - if hasattr(py_internal, "regex_match"): - # ?U means activates unicode matching (Python allows most unicode - # in module names / identifiers). - # \w matches alphanumeric and underscore. - # NOTE: regex_match has an implicit ^ and $ - return py_internal.regex_match(name, "(?U)\\w+") - else: - # Otherwise, use a rough hueristic that should catch most cases. - return ( - "." not in name and - "-" not in name - ) +def _get_file_venv_path(ctx, f, site_packages_root): + """Computes a file's venv_path if it's under the site_packages_root. + + Args: + ctx: The current ctx. + f: The file to compute the venv_path for. + site_packages_root: The site packages root path. -def get_venv_symlinks(ctx, files, package, version_str, site_packages_root): + Returns: + A tuple `(venv_path, rf_root_path)` if the file is under + `site_packages_root`, otherwise `(None, None)`. + """ + rf_root_path = runfiles_root_path(ctx, f.short_path) + _, _, repo_rel_path = rf_root_path.partition("/") + head, found_sp_root, venv_path = repo_rel_path.partition(site_packages_root) + if head or not found_sp_root: + # If head is set, then the path didn't start with site_packages_root + # if found_sp_root is empty, then it means it wasn't found at all. + return (None, None) + return (venv_path, rf_root_path) + +def get_venv_symlinks( + ctx, + files, + package, + version_str, + site_packages_root, + namespace_package_files = []): """Compute the VenvSymlinkEntry objects for a library. Args: @@ -259,6 +279,9 @@ def get_venv_symlinks(ctx, files, package, version_str, site_packages_root): version_str: {type}`str` the distribution's version. site_packages_root: {type}`str` prefix under which files are considered to be part of the installed files. + namespace_package_files: {type}`list[File]` a list of files + that are pkgutil-style namespace packages and cannot be + directly linked. Returns: {type}`list[VenvSymlinkEntry]` the entries that describe how @@ -276,8 +299,24 @@ def get_venv_symlinks(ctx, files, package, version_str, site_packages_root): all_files = sorted(files, key = lambda f: f.short_path) + # dict[str venv-relative dirname, bool is_namespace_package] + namespace_package_dirs = { + ns: True + for ns in _WELL_KNOWN_NAMESPACE_PACKAGES + } + # venv paths that cannot be directly linked. Dict acting as set. - cannot_be_linked_directly = {} + cannot_be_linked_directly = { + dirname: True + for dirname in namespace_package_dirs.keys() + } + for f in namespace_package_files: + venv_path, _ = _get_file_venv_path(ctx, f, site_packages_root) + if venv_path == None: + continue + ns_dir = paths.dirname(venv_path) + namespace_package_dirs[ns_dir] = True + cannot_be_linked_directly[ns_dir] = True # dict[str path, VenvSymlinkEntry] # Where path is the venv path (i.e. relative to site_packages_prefix) @@ -286,9 +325,6 @@ def get_venv_symlinks(ctx, files, package, version_str, site_packages_root): # List of (File, str venv_path) tuples files_left_to_link = [] - # dict[str dirname, bool is_namespace_package] - namespace_package_dirs = {} - # We want to minimize the number of files symlinked. Ideally, only the # top-level directories are symlinked. Unfortunately, shared libraries # complicate matters: if a shared library's directory is linked, then the @@ -298,12 +334,8 @@ def get_venv_symlinks(ctx, files, package, version_str, site_packages_root): # all the parent directories of the shared library can't be linked # directly. for src in all_files: - rf_root_path = runfiles_root_path(ctx, src.short_path) - _, _, repo_rel_path = rf_root_path.partition("/") - head, found_sp_root, venv_path = repo_rel_path.partition(site_packages_root) - if head or not found_sp_root: - # If head is set, then the path didn't start with site_packages_root - # if found_sp_root is empty, then it means it wasn't found at all. + venv_path, rf_root_path = _get_file_venv_path(ctx, src, site_packages_root) + if venv_path == None: continue filename = paths.basename(venv_path) @@ -336,7 +368,7 @@ def get_venv_symlinks(ctx, files, package, version_str, site_packages_root): # If its already known to be non-implicit namespace, then skip namespace_package_dirs.get(top_level_dirname, True) and # It must be an importable name to be an implicit namespace package - _is_importable_name(top_level_dirname) + is_importable_name(top_level_dirname) ): namespace_package_dirs.setdefault(top_level_dirname, True) @@ -350,7 +382,10 @@ def get_venv_symlinks(ctx, files, package, version_str, site_packages_root): # to avoid conflict merging later. for dirname, is_namespace_package in namespace_package_dirs.items(): if is_namespace_package: - cannot_be_linked_directly[dirname] = True + # If it's already in cannot_be_linked_directly due to pkgutil_namespace_packages + # then we should not unset it. + if not cannot_be_linked_directly.get(dirname, False): + cannot_be_linked_directly[dirname] = True # At this point, venv_symlinks has entries for the shared libraries # and cannot_be_linked_directly has the directories that cannot be diff --git a/tests/pypi/whl_library_targets/whl_library_targets_tests.bzl b/tests/pypi/whl_library_targets/whl_library_targets_tests.bzl index 1d80340a13..9b574039b2 100644 --- a/tests/pypi/whl_library_targets/whl_library_targets_tests.bzl +++ b/tests/pypi/whl_library_targets/whl_library_targets_tests.bzl @@ -257,6 +257,10 @@ def _test_whl_and_library_deps_from_requires(env): "tags": ["pypi_name=Foo", "pypi_version=0"], "visibility": ["//visibility:public"], "experimental_venvs_site_packages": Label("//python/config_settings:venvs_site_packages"), + "namespace_package_files": [] + select({ + Label("//python/config_settings:is_venvs_site_packages"): [], + "//conditions:default": ["_create_inits_target"], + }), }) # buildifier: @unsorted-dict-items env.expect.that_collection(mock_glob.calls).contains_exactly([ @@ -380,6 +384,10 @@ def _test_whl_and_library_deps(env): "tags": ["tag1", "tag2"], "visibility": ["//visibility:public"], "experimental_venvs_site_packages": Label("//python/config_settings:venvs_site_packages"), + "namespace_package_files": [] + select({ + Label("//python/config_settings:is_venvs_site_packages"): [], + "//conditions:default": ["_create_inits_target"], + }), }) # buildifier: @unsorted-dict-items _tests.append(_test_whl_and_library_deps) @@ -449,6 +457,10 @@ def _test_group(env): "tags": [], "visibility": ["@pypi__config//_groups:__pkg__"], "experimental_venvs_site_packages": Label("//python/config_settings:venvs_site_packages"), + "namespace_package_files": [] + select({ + Label("//python/config_settings:is_venvs_site_packages"): [], + "//conditions:default": ["_create_inits_target"], + }), }) # buildifier: @unsorted-dict-items env.expect.that_collection(mock_glob.calls, expr = "glob calls").contains_exactly([ diff --git a/tests/repos/BUILD.bazel b/tests/repos/BUILD.bazel new file mode 100644 index 0000000000..e69de29bb2 diff --git a/tests/repos/pkgutil_nspkg1/BUILD.bazel b/tests/repos/pkgutil_nspkg1/BUILD.bazel new file mode 100644 index 0000000000..e69de29bb2 diff --git a/tests/repos/pkgutil_nspkg1/nspkg/__init__.py b/tests/repos/pkgutil_nspkg1/nspkg/__init__.py new file mode 100644 index 0000000000..c4da2cf0b7 --- /dev/null +++ b/tests/repos/pkgutil_nspkg1/nspkg/__init__.py @@ -0,0 +1,2 @@ +# __init__.py +__path__ = __import__("pkgutil").extend_path(__path__, __name__) diff --git a/tests/repos/pkgutil_nspkg1/nspkg/one/a.txt b/tests/repos/pkgutil_nspkg1/nspkg/one/a.txt new file mode 100644 index 0000000000..f4dbe63934 --- /dev/null +++ b/tests/repos/pkgutil_nspkg1/nspkg/one/a.txt @@ -0,0 +1 @@ +dummy content \ No newline at end of file diff --git a/tests/repos/pkgutil_nspkg1/pkgutil_nspkg1-1.0.dist-info/METADATA b/tests/repos/pkgutil_nspkg1/pkgutil_nspkg1-1.0.dist-info/METADATA new file mode 100644 index 0000000000..7ffb60b701 --- /dev/null +++ b/tests/repos/pkgutil_nspkg1/pkgutil_nspkg1-1.0.dist-info/METADATA @@ -0,0 +1,3 @@ +Metadata-Version: 2.1 +Name: pkgutil-nspkg1 +Version: 1.0 diff --git a/tests/repos/pkgutil_nspkg1/pkgutil_nspkg1-1.0.dist-info/RECORD b/tests/repos/pkgutil_nspkg1/pkgutil_nspkg1-1.0.dist-info/RECORD new file mode 100644 index 0000000000..e039fee1ae --- /dev/null +++ b/tests/repos/pkgutil_nspkg1/pkgutil_nspkg1-1.0.dist-info/RECORD @@ -0,0 +1,5 @@ +nspkg/__init__.py,sha256=d10f14d9ce938ae14416c3c1f4b516c4f40dfe0c9bf973a833e4ef40517ba7d0,81 +nspkg/one/a.txt,sha256=bf0ecbdb9b814248d086c9b69cf26182d9d4138f2ad3d0637c4555fc8cbf68e5,13 +pkgutil_nspkg1-1.0.dist-info/METADATA,sha256=49525c3e6f1fc8f46d9c92c996e35f81327f9fe417df2d9d10784fa4e9cfe84b,59 +pkgutil_nspkg1-1.0.dist-info/WHEEL,sha256=d652ec50af6f144788dc1ffef052e8833b6704e98818e6cf78d80a625ad498fb,105 +pkgutil_nspkg1-1.0.dist-info/RECORD,, diff --git a/tests/repos/pkgutil_nspkg1/pkgutil_nspkg1-1.0.dist-info/WHEEL b/tests/repos/pkgutil_nspkg1/pkgutil_nspkg1-1.0.dist-info/WHEEL new file mode 100644 index 0000000000..dad2b72544 --- /dev/null +++ b/tests/repos/pkgutil_nspkg1/pkgutil_nspkg1-1.0.dist-info/WHEEL @@ -0,0 +1,4 @@ +Wheel-Version: 1.0 +Generator: rules_python_whl_from_dir_repo +Root-Is-Purelib: true +Tag: py3-none-any diff --git a/tests/repos/pkgutil_nspkg2/BUILD.bazel b/tests/repos/pkgutil_nspkg2/BUILD.bazel new file mode 100644 index 0000000000..e69de29bb2 diff --git a/tests/repos/pkgutil_nspkg2/nspkg/__init__.py b/tests/repos/pkgutil_nspkg2/nspkg/__init__.py new file mode 100644 index 0000000000..c4da2cf0b7 --- /dev/null +++ b/tests/repos/pkgutil_nspkg2/nspkg/__init__.py @@ -0,0 +1,2 @@ +# __init__.py +__path__ = __import__("pkgutil").extend_path(__path__, __name__) diff --git a/tests/repos/pkgutil_nspkg2/nspkg/two/b.txt b/tests/repos/pkgutil_nspkg2/nspkg/two/b.txt new file mode 100644 index 0000000000..f4dbe63934 --- /dev/null +++ b/tests/repos/pkgutil_nspkg2/nspkg/two/b.txt @@ -0,0 +1 @@ +dummy content \ No newline at end of file diff --git a/tests/repos/pkgutil_nspkg2/pkgutil_nspkg2-1.0.dist-info/METADATA b/tests/repos/pkgutil_nspkg2/pkgutil_nspkg2-1.0.dist-info/METADATA new file mode 100644 index 0000000000..368e64ce19 --- /dev/null +++ b/tests/repos/pkgutil_nspkg2/pkgutil_nspkg2-1.0.dist-info/METADATA @@ -0,0 +1,3 @@ +Metadata-Version: 2.1 +Name: pkgutil-nspkg2 +Version: 1.0 diff --git a/tests/repos/pkgutil_nspkg2/pkgutil_nspkg2-1.0.dist-info/RECORD b/tests/repos/pkgutil_nspkg2/pkgutil_nspkg2-1.0.dist-info/RECORD new file mode 100644 index 0000000000..c93970cb26 --- /dev/null +++ b/tests/repos/pkgutil_nspkg2/pkgutil_nspkg2-1.0.dist-info/RECORD @@ -0,0 +1,5 @@ +nspkg/__init__.py,sha256=d10f14d9ce938ae14416c3c1f4b516c4f40dfe0c9bf973a833e4ef40517ba7d0,81 +nspkg/two/b.txt,sha256=bf0ecbdb9b814248d086c9b69cf26182d9d4138f2ad3d0637c4555fc8cbf68e5,13 +pkgutil_nspkg2-1.0.dist-info/METADATA,sha256=9a72654b480f17c55df07fe27d026c4ea461c493babcc675f26fbcfce40828b6,59 +pkgutil_nspkg2-1.0.dist-info/WHEEL,sha256=d652ec50af6f144788dc1ffef052e8833b6704e98818e6cf78d80a625ad498fb,105 +pkgutil_nspkg2-1.0.dist-info/RECORD,, diff --git a/tests/repos/pkgutil_nspkg2/pkgutil_nspkg2-1.0.dist-info/WHEEL b/tests/repos/pkgutil_nspkg2/pkgutil_nspkg2-1.0.dist-info/WHEEL new file mode 100644 index 0000000000..dad2b72544 --- /dev/null +++ b/tests/repos/pkgutil_nspkg2/pkgutil_nspkg2-1.0.dist-info/WHEEL @@ -0,0 +1,4 @@ +Wheel-Version: 1.0 +Generator: rules_python_whl_from_dir_repo +Root-Is-Purelib: true +Tag: py3-none-any diff --git a/tests/venv_site_packages_libs/app_files_building/app_files_building_tests.bzl b/tests/venv_site_packages_libs/app_files_building/app_files_building_tests.bzl index e92c0aaf5a..f85508dd3d 100644 --- a/tests/venv_site_packages_libs/app_files_building/app_files_building_tests.bzl +++ b/tests/venv_site_packages_libs/app_files_building/app_files_building_tests.bzl @@ -2,8 +2,12 @@ load("@rules_testing//lib:analysis_test.bzl", "analysis_test") load("@rules_testing//lib:test_suite.bzl", "test_suite") +load("//python:py_info.bzl", "PyInfo") +load("//python:py_library.bzl", "py_library") +load("//python/private:common_labels.bzl", "labels") # buildifier: disable=bzl-visibility load("//python/private:py_info.bzl", "VenvSymlinkEntry", "VenvSymlinkKind") # buildifier: disable=bzl-visibility load("//python/private:venv_runfiles.bzl", "build_link_map", "get_venv_symlinks") # buildifier: disable=bzl-visibility +load("//tests/support:support.bzl", "SUPPORTS_BZLMOD_UNIXY") def _empty_files_impl(ctx): files = [] @@ -33,6 +37,7 @@ _tests = [] def _ctx(workspace_name = "_main"): return struct( workspace_name = workspace_name, + label = Label("@@FAKE-CTX//:fake_ctx"), ) def _file(short_path): @@ -330,6 +335,178 @@ def _test_optimized_grouping_implicit_namespace_packages_impl(env, target): # The point of the optimization is to avoid having to merge conflicts. env.expect.that_collection(conflicts).contains_exactly([]) +def _test_optimized_grouping_pkgutil_namespace_packages(name): + empty_files( + name = name + "_files", + paths = [ + "site-packages/pkgutilns/__init__.py", + "site-packages/pkgutilns/foo.py", + # Special cases: These dirnames under site-packages are always + # treated as namespace packages + "site-packages/nvidia/whatever/w.py", + ], + ) + analysis_test( + name = name, + impl = _test_optimized_grouping_pkgutil_namespace_packages_impl, + target = name + "_files", + ) + +_tests.append(_test_optimized_grouping_pkgutil_namespace_packages) + +def _test_optimized_grouping_pkgutil_namespace_packages_impl(env, target): + test_ctx = _ctx(workspace_name = env.ctx.workspace_name) + files = target.files.to_list() + ns_inits = [f for f in files if f.basename == "__init__.py"] + + entries = get_venv_symlinks( + test_ctx, + files, + package = "pkgutilns", + version_str = "1.0", + site_packages_root = env.ctx.label.package + "/site-packages", + namespace_package_files = ns_inits, + ) + actual = _venv_symlinks_from_entries(entries) + + rr = "{}/{}/site-packages/".format(test_ctx.workspace_name, env.ctx.label.package) + expected = [ + _venv_symlink( + "pkgutilns/__init__.py", + link_to_path = rr + "pkgutilns/__init__.py", + files = [ + "tests/venv_site_packages_libs/app_files_building/site-packages/pkgutilns/__init__.py", + ], + ), + _venv_symlink( + "pkgutilns/foo.py", + link_to_path = rr + "pkgutilns/foo.py", + files = [ + "tests/venv_site_packages_libs/app_files_building/site-packages/pkgutilns/foo.py", + ], + ), + _venv_symlink( + "nvidia/whatever", + link_to_path = rr + "nvidia/whatever", + files = [ + "tests/venv_site_packages_libs/app_files_building/site-packages/nvidia/whatever/w.py", + ], + ), + ] + expected = sorted(expected, key = lambda e: (e.link_to_path, e.venv_path)) + env.expect.that_collection( + actual, + ).contains_exactly(expected) + + _, conflicts = build_link_map(test_ctx, entries, return_conflicts = True) + + # The point of the optimization is to avoid having to merge conflicts. + env.expect.that_collection(conflicts).contains_exactly([]) + +def _test_optimized_grouping_pkgutil_whls(name): + """Verify that the whl_library pkgutli style detection logic works.""" + py_library( + name = name + "_lib", + deps = [ + "@pkgutil_nspkg1//:pkg", + "@pkgutil_nspkg2//:pkg", + ], + target_compatible_with = SUPPORTS_BZLMOD_UNIXY, + ) + analysis_test( + name = name, + impl = _test_optimized_grouping_pkgutil_whls_impl, + target = name + "_lib", + config_settings = { + labels.VENVS_SITE_PACKAGES: "yes", + }, + attr_values = dict( + target_compatible_with = SUPPORTS_BZLMOD_UNIXY, + ), + ) + +_tests.append(_test_optimized_grouping_pkgutil_whls) + +def _test_optimized_grouping_pkgutil_whls_impl(env, target): + test_ctx = _ctx(workspace_name = env.ctx.workspace_name) + actual_raw_entries = target[PyInfo].venv_symlinks.to_list() + + actual = _venv_symlinks_from_entries(actual_raw_entries) + + # The important condition is that the top-level 'nspkg' directory + # is NOT linked because it's a pkgutil namespace package. + env.expect.that_collection(actual).contains_exactly([ + # Entries from pkgutil_ns1 + _venv_symlink( + "nspkg/__init__.py", + link_to_path = "+internal_dev_deps+pkgutil_nspkg1/site-packages/nspkg/__init__.py", + files = [ + "../+internal_dev_deps+pkgutil_nspkg1/site-packages/nspkg/__init__.py", + ], + ), + _venv_symlink( + "nspkg/one", + link_to_path = "+internal_dev_deps+pkgutil_nspkg1/site-packages/nspkg/one", + files = [ + "../+internal_dev_deps+pkgutil_nspkg1/site-packages/nspkg/one/a.txt", + ], + ), + _venv_symlink( + "pkgutil_nspkg1-1.0.dist-info", + link_to_path = "+internal_dev_deps+pkgutil_nspkg1/site-packages/pkgutil_nspkg1-1.0.dist-info", + files = [ + "../+internal_dev_deps+pkgutil_nspkg1/site-packages/pkgutil_nspkg1-1.0.dist-info/INSTALLER", + "../+internal_dev_deps+pkgutil_nspkg1/site-packages/pkgutil_nspkg1-1.0.dist-info/METADATA", + "../+internal_dev_deps+pkgutil_nspkg1/site-packages/pkgutil_nspkg1-1.0.dist-info/WHEEL", + ], + ), + # Entries from pkgutil_ns2 + _venv_symlink( + "nspkg/__init__.py", + link_to_path = "+internal_dev_deps+pkgutil_nspkg2/site-packages/nspkg/__init__.py", + files = [ + "../+internal_dev_deps+pkgutil_nspkg2/site-packages/nspkg/__init__.py", + ], + ), + _venv_symlink( + "nspkg/two", + link_to_path = "+internal_dev_deps+pkgutil_nspkg2/site-packages/nspkg/two", + files = [ + "../+internal_dev_deps+pkgutil_nspkg2/site-packages/nspkg/two/b.txt", + ], + ), + _venv_symlink( + "pkgutil_nspkg2-1.0.dist-info", + link_to_path = "+internal_dev_deps+pkgutil_nspkg2/site-packages/pkgutil_nspkg2-1.0.dist-info", + files = [ + "../+internal_dev_deps+pkgutil_nspkg2/site-packages/pkgutil_nspkg2-1.0.dist-info/INSTALLER", + "../+internal_dev_deps+pkgutil_nspkg2/site-packages/pkgutil_nspkg2-1.0.dist-info/METADATA", + "../+internal_dev_deps+pkgutil_nspkg2/site-packages/pkgutil_nspkg2-1.0.dist-info/WHEEL", + ], + ), + ]) + + # Verifying that the expected VenvSymlink structure is processed with minimal number + # of conflicts (Just the single pkgutil style __init__.py file) + _, conflicts = build_link_map(test_ctx, actual_raw_entries, return_conflicts = True) + env.expect.that_collection(_venv_symlinks_from_entries(conflicts[0])).contains_exactly([ + _venv_symlink( + "nspkg/__init__.py", + link_to_path = "+internal_dev_deps+pkgutil_nspkg1/site-packages/nspkg/__init__.py", + files = [ + "../+internal_dev_deps+pkgutil_nspkg1/site-packages/nspkg/__init__.py", + ], + ), + _venv_symlink( + "nspkg/__init__.py", + link_to_path = "+internal_dev_deps+pkgutil_nspkg2/site-packages/nspkg/__init__.py", + files = [ + "../+internal_dev_deps+pkgutil_nspkg2/site-packages/nspkg/__init__.py", + ], + ), + ]) + env.expect.that_collection(conflicts).has_size(1) + def _test_package_version_filtering(name): analysis_test( name = name, From 50fb48e6cc968ce7791433142c96d979709c60c8 Mon Sep 17 00:00:00 2001 From: Ignas Anikevicius <240938+aignas@users.noreply.github.com> Date: Thu, 18 Dec 2025 00:46:41 +0900 Subject: [PATCH 562/922] fix(pipstar): actually pass the extras down the call stack (#3468) It seems that the extras were not correctly passed down the call stack. The upstream code was handling everything correctly and we had unit tests for the downstream code. Fixes #3352 --- python/private/pypi/BUILD.bazel | 1 + python/private/pypi/whl_library.bzl | 2 ++ 2 files changed, 3 insertions(+) diff --git a/python/private/pypi/BUILD.bazel b/python/private/pypi/BUILD.bazel index 18caa53d7f..aa96cf86a5 100644 --- a/python/private/pypi/BUILD.bazel +++ b/python/private/pypi/BUILD.bazel @@ -433,6 +433,7 @@ bzl_library( ":generate_whl_library_build_bazel_bzl", ":parse_whl_name_bzl", ":patch_whl_bzl", + ":pep508_requirement_bzl", ":pypi_repo_utils_bzl", ":whl_metadata_bzl", ":whl_target_platforms_bzl", diff --git a/python/private/pypi/whl_library.bzl b/python/private/pypi/whl_library.bzl index 201ef59b89..9f04252356 100644 --- a/python/private/pypi/whl_library.bzl +++ b/python/private/pypi/whl_library.bzl @@ -24,6 +24,7 @@ load(":deps.bzl", "all_repo_names", "record_files") load(":generate_whl_library_build_bazel.bzl", "generate_whl_library_build_bazel") load(":parse_whl_name.bzl", "parse_whl_name") load(":patch_whl.bzl", "patch_whl") +load(":pep508_requirement.bzl", "requirement") load(":pypi_repo_utils.bzl", "pypi_repo_utils") load(":whl_metadata.bzl", "whl_metadata") load(":whl_target_platforms.bzl", "whl_target_platforms") @@ -435,6 +436,7 @@ def _whl_library_impl(rctx): group_deps = rctx.attr.group_deps, group_name = rctx.attr.group_name, namespace_package_files = namespace_package_files, + extras = requirement(rctx.attr.requirement).extras, ) else: target_platforms = rctx.attr.experimental_target_platforms or [] From fc2a9f987e629d6dbedec089d58386c836524eb3 Mon Sep 17 00:00:00 2001 From: Ignas Anikevicius <240938+aignas@users.noreply.github.com> Date: Thu, 18 Dec 2025 08:30:59 +0900 Subject: [PATCH 563/922] fix(pipstar): fix whl extraction and flip pipstar=true (#3461) Attempt number 2. This should be smoother this time and should not cause any breakage because we are not enabling any cross-building by default and only the host wheels will be present. Because we also started extracting using starlark APIs, some extra fixups where needed because some wheels require extracting `.data` files into correct paths. This also adds the `INSTALLER` file after extracting files to signify that `pipstar` has installed the file. Because we have stopped passing hermetic interpreter to the `whl_library` if pipstar is enabled, we also needed to ensure that the code path is only enabled if the extraction with pipstar is supported (i.e. bazel >= 8). Fixes #2949 --------- Co-authored-by: Richard Levasseur Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> --- CHANGELOG.md | 4 + python/private/internal_config_repo.bzl | 2 +- python/private/pypi/extension.bzl | 13 +- python/private/pypi/hub_builder.bzl | 11 +- python/private/pypi/whl_library.bzl | 126 +++++++++++++------ tests/pypi/extension/extension_tests.bzl | 1 + tests/pypi/hub_builder/hub_builder_tests.bzl | 5 + 7 files changed, 117 insertions(+), 45 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 69e159c6c7..fcf56348fc 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -81,6 +81,10 @@ END_UNRELEASED_TEMPLATE * (pip) `pipstar` has been enabled for all `whl_library` instances where the whl is passed through a label or downloaded using the bazel downloader ([#2949](https://github.com/bazel-contrib/rules_python/issues/2949)). +* (pypi) `pipstar` flag default has been flipped to be on by default. + It can be disabled through `RULES_PYTHON_ENABLE_PIPSTAR=0` environment variable. + If you do need to disable it, please add a comment to + [#2949](https://github.com/bazel-contrib/rules_python/issues/2949). * (gazelle deps) rules_go bumped from 0.55.1 to 0.59.0 * (gazelle deps) gazelle bumped from 0.36.0 to 0.47.0 diff --git a/python/private/internal_config_repo.bzl b/python/private/internal_config_repo.bzl index fc1f8d3bbe..9fc301cd2e 100644 --- a/python/private/internal_config_repo.bzl +++ b/python/private/internal_config_repo.bzl @@ -22,7 +22,7 @@ load("//python/private:text_util.bzl", "render") load(":repo_utils.bzl", "repo_utils") _ENABLE_PIPSTAR_ENVVAR_NAME = "RULES_PYTHON_ENABLE_PIPSTAR" -_ENABLE_PIPSTAR_DEFAULT = "0" +_ENABLE_PIPSTAR_DEFAULT = "1" _ENABLE_DEPRECATION_WARNINGS_ENVVAR_NAME = "RULES_PYTHON_DEPRECATION_WARNINGS" _ENABLE_DEPRECATION_WARNINGS_DEFAULT = "0" diff --git a/python/private/pypi/extension.bzl b/python/private/pypi/extension.bzl index 3d7985ab3f..e9033996c6 100644 --- a/python/private/pypi/extension.bzl +++ b/python/private/pypi/extension.bzl @@ -70,13 +70,16 @@ def _configure(config, *, override = False, **kwargs): def build_config( *, module_ctx, - enable_pipstar): + enable_pipstar, + enable_pipstar_extract): """Parse 'configure' and 'default' extension tags Args: module_ctx: {type}`module_ctx` module context. enable_pipstar: {type}`bool` a flag to enable dropping Python dependency for evaluation of the extension. + enable_pipstar_extract: {type}`bool | None` a flag to also not pass Python + interpreter to `whl_library` when possible. Returns: A struct with the configuration. @@ -127,6 +130,7 @@ def build_config( for name, values in defaults["platforms"].items() }, enable_pipstar = enable_pipstar, + enable_pipstar_extract = enable_pipstar_extract, ) def parse_modules( @@ -134,6 +138,7 @@ def parse_modules( _fail = fail, simpleapi_download = simpleapi_download, enable_pipstar = False, + enable_pipstar_extract = False, **kwargs): """Implementation of parsing the tag classes for the extension and return a struct for registering repositories. @@ -142,6 +147,8 @@ def parse_modules( simpleapi_download: Used for testing overrides enable_pipstar: {type}`bool` a flag to enable dropping Python dependency for evaluation of the extension. + enable_pipstar_extract: {type}`bool` a flag to enable dropping Python dependency for + extracting wheels. _fail: {type}`function` the failure function, mainly for testing. **kwargs: Extra arguments passed to the hub_builder. @@ -179,7 +186,7 @@ You cannot use both the additive_build_content and additive_build_content_file a srcs_exclude_glob = whl_mod.srcs_exclude_glob, ) - config = build_config(module_ctx = module_ctx, enable_pipstar = enable_pipstar) + config = build_config(module_ctx = module_ctx, enable_pipstar = enable_pipstar, enable_pipstar_extract = enable_pipstar_extract) # TODO @aignas 2025-06-03: Merge override API with the builder? _overriden_whl_set = {} @@ -362,7 +369,7 @@ def _pip_impl(module_ctx): module_ctx: module contents """ - mods = parse_modules(module_ctx, enable_pipstar = rp_config.enable_pipstar) + mods = parse_modules(module_ctx, enable_pipstar = rp_config.enable_pipstar, enable_pipstar_extract = rp_config.enable_pipstar and rp_config.bazel_8_or_later) # Build all of the wheel modifications if the tag class is called. _whl_mods_impl(mods.whl_mods) diff --git a/python/private/pypi/hub_builder.bzl b/python/private/pypi/hub_builder.bzl index 97e0a111b2..95e007aa69 100644 --- a/python/private/pypi/hub_builder.bzl +++ b/python/private/pypi/hub_builder.bzl @@ -153,6 +153,7 @@ def _pip_parse(self, module_ctx, pip_attr): module_ctx, pip_attr = pip_attr, enable_pipstar = self._config.enable_pipstar or self._get_index_urls.get(pip_attr.python_version), + enable_pipstar_extract = self._config.enable_pipstar_extract or self._get_index_urls.get(pip_attr.python_version), ) ### end of PUBLIC methods @@ -407,7 +408,8 @@ def _create_whl_repos( module_ctx, *, pip_attr, - enable_pipstar = False): + enable_pipstar = False, + enable_pipstar_extract = False): """create all of the whl repositories Args: @@ -415,6 +417,7 @@ def _create_whl_repos( module_ctx: {type}`module_ctx`. pip_attr: {type}`struct` - the struct that comes from the tag class iteration. enable_pipstar: {type}`bool` - enable the pipstar or not. + enable_pipstar_extract: {type}`bool` - enable the pipstar extraction or not. """ logger = self._logger platforms = self._platforms[pip_attr.python_version] @@ -479,6 +482,7 @@ def _create_whl_repos( is_multiple_versions = whl.is_multiple_versions, interpreter = interpreter, enable_pipstar = enable_pipstar, + enable_pipstar_extract = enable_pipstar_extract, ) _add_whl_library( self, @@ -555,7 +559,8 @@ def _whl_repo( python_version, use_downloader, interpreter, - enable_pipstar = False): + enable_pipstar = False, + enable_pipstar_extract = False): args = dict(whl_library_args) args["requirement"] = src.requirement_line is_whl = src.filename.endswith(".whl") @@ -567,7 +572,7 @@ def _whl_repo( # need to pass the extra args there, so only pop this for whls args["extra_pip_args"] = src.extra_pip_args - if "whl_patches" in args or not (enable_pipstar and is_whl): + if "whl_patches" in args or not (enable_pipstar_extract and is_whl): if interpreter.path: args["python_interpreter"] = interpreter.path if interpreter.target: diff --git a/python/private/pypi/whl_library.bzl b/python/private/pypi/whl_library.bzl index 9f04252356..c368dea733 100644 --- a/python/private/pypi/whl_library.bzl +++ b/python/private/pypi/whl_library.bzl @@ -26,7 +26,7 @@ load(":parse_whl_name.bzl", "parse_whl_name") load(":patch_whl.bzl", "patch_whl") load(":pep508_requirement.bzl", "requirement") load(":pypi_repo_utils.bzl", "pypi_repo_utils") -load(":whl_metadata.bzl", "whl_metadata") +load(":whl_metadata.bzl", "find_whl_metadata", "whl_metadata") load(":whl_target_platforms.bzl", "whl_target_platforms") _CPPFLAGS = "CPPFLAGS" @@ -265,6 +265,79 @@ def _create_repository_execution_environment(rctx, python_interpreter, logger = env[_CPPFLAGS] = " ".join(cppflags) return env +def _extract_whl_star(rctx, *, whl_path, logger): + install_dir_path = whl_path.dirname.get_child("site-packages") + repo_utils.extract( + rctx, + archive = whl_path, + output = install_dir_path, + supports_whl_extraction = rp_config.supports_whl_extraction, + ) + metadata_file = find_whl_metadata( + install_dir = install_dir_path, + logger = logger, + ) + + # Get the .dist_info dir name + dist_info_dir = metadata_file.dirname + rctx.file( + dist_info_dir.get_child("INSTALLER"), + "https://github.com/bazel-contrib/rules_python#pipstar", + ) + repo_root_dir = whl_path.dirname + + # Get the .dist_info dir name + data_dir = dist_info_dir.dirname.get_child(dist_info_dir.basename[:-len(".dist-info")] + ".data") + if data_dir.exists: + for prefix, dest in { + # https://docs.python.org/3/library/sysconfig.html#posix-prefix + # We are taking this from the legacy whl installer config + "data": "data", + "headers": "include", + "platlib": "site-packages", + "purelib": "site-packages", + "scripts": "bin", + }.items(): + src = data_dir.get_child(prefix) + dest = repo_root_dir.get_child(dest) + if src.exists: + rctx.rename(src, dest) + + # TODO @aignas 2025-12-16: when moving scripts to `bin`, rewrite the #!python + # shebang to be something else, for inspiration look at the hermetic + # toolchain wrappers + +def _extract_whl_py(rctx, *, python_interpreter, args, whl_path, environment, logger): + target_platforms = rctx.attr.experimental_target_platforms or [] + if target_platforms: + parsed_whl = parse_whl_name(whl_path.basename) + + # NOTE @aignas 2023-12-04: if the wheel is a platform specific wheel, we + # only include deps for that target platform + if parsed_whl.platform_tag != "any": + target_platforms = [ + p.target_platform + for p in whl_target_platforms( + platform_tag = parsed_whl.platform_tag, + abi_tag = parsed_whl.abi_tag.strip("tm"), + ) + ] + + pypi_repo_utils.execute_checked( + rctx, + op = "whl_library.ExtractWheel({}, {})".format(rctx.attr.name, whl_path), + python = python_interpreter, + arguments = args + [ + "--whl-file", + whl_path, + ] + ["--platform={}".format(p) for p in target_platforms], + srcs = rctx.attr._python_srcs, + environment = environment, + quiet = rctx.attr.quiet, + timeout = rctx.attr.timeout, + logger = logger, + ) + def _whl_library_impl(rctx): logger = repo_utils.logger(rctx) python_interpreter = pypi_repo_utils.resolve_python_interpreter( @@ -327,6 +400,8 @@ def _whl_library_impl(rctx): # also enable pipstar for any whls that are downloaded without `pip` enable_pipstar = (rp_config.enable_pipstar or whl_path) and rctx.attr.config_load + enable_pipstar_extract = (rp_config.enable_pipstar and rp_config.bazel_8_or_later) and rctx.attr.config_load + if not whl_path: if rctx.attr.urls: op_tmpl = "whl_library.BuildWheelFromSource({name}, {requirement})" @@ -372,19 +447,24 @@ def _whl_library_impl(rctx): timeout = rctx.attr.timeout, ) + if enable_pipstar_extract: + _extract_whl_star(rctx, whl_path = whl_path, logger = logger) + else: + _extract_whl_py( + rctx, + python_interpreter = python_interpreter, + args = args, + whl_path = whl_path, + environment = environment, + logger = logger, + ) + # NOTE @aignas 2025-09-28: if someone has an old vendored file that does not have the # dep_template set or the packages is not set either, we should still not break, best to # disable pipstar for that particular case. # # Remove non-pipstar and config_load check when we release rules_python 2. if enable_pipstar: - repo_utils.extract( - rctx, - archive = whl_path, - output = "site-packages", - supports_whl_extraction = rp_config.supports_whl_extraction, - ) - install_dir_path = whl_path.dirname.get_child("site-packages") metadata = whl_metadata( install_dir = install_dir_path, @@ -439,36 +519,6 @@ def _whl_library_impl(rctx): extras = requirement(rctx.attr.requirement).extras, ) else: - target_platforms = rctx.attr.experimental_target_platforms or [] - if target_platforms: - parsed_whl = parse_whl_name(whl_path.basename) - - # NOTE @aignas 2023-12-04: if the wheel is a platform specific wheel, we - # only include deps for that target platform - if parsed_whl.platform_tag != "any": - target_platforms = [ - p.target_platform - for p in whl_target_platforms( - platform_tag = parsed_whl.platform_tag, - abi_tag = parsed_whl.abi_tag.strip("tm"), - ) - ] - - pypi_repo_utils.execute_checked( - rctx, - op = "whl_library.ExtractWheel({}, {})".format(rctx.attr.name, whl_path), - python = python_interpreter, - arguments = args + [ - "--whl-file", - whl_path, - ] + ["--platform={}".format(p) for p in target_platforms], - srcs = rctx.attr._python_srcs, - environment = environment, - quiet = rctx.attr.quiet, - timeout = rctx.attr.timeout, - logger = logger, - ) - metadata = json.decode(rctx.read("metadata.json")) rctx.delete("metadata.json") diff --git a/tests/pypi/extension/extension_tests.bzl b/tests/pypi/extension/extension_tests.bzl index 924796c703..90723c487d 100644 --- a/tests/pypi/extension/extension_tests.bzl +++ b/tests/pypi/extension/extension_tests.bzl @@ -99,6 +99,7 @@ def _build_config(env, enable_pipstar = 0, **kwargs): return env.expect.that_struct( build_config( enable_pipstar = enable_pipstar, + enable_pipstar_extract = True, **kwargs ), attrs = dict( diff --git a/tests/pypi/hub_builder/hub_builder_tests.bzl b/tests/pypi/hub_builder/hub_builder_tests.bzl index bf21dcacfa..42c65ae8f7 100644 --- a/tests/pypi/hub_builder/hub_builder_tests.bzl +++ b/tests/pypi/hub_builder/hub_builder_tests.bzl @@ -41,6 +41,7 @@ simple==0.0.1 \ def hub_builder( env, enable_pipstar = True, + enable_pipstar_extract = True, debug = False, config = None, minor_mapping = {}, @@ -54,6 +55,7 @@ def hub_builder( config = config or struct( # no need to evaluate the markers with the interpreter enable_pipstar = enable_pipstar, + enable_pipstar_extract = enable_pipstar_extract, platforms = { "{}_{}{}".format(os, cpu, freethreaded): _plat( name = "{}_{}{}".format(os, cpu, freethreaded), @@ -512,6 +514,7 @@ def _test_torch_experimental_index_url(env): config = struct( netrc = None, enable_pipstar = True, + enable_pipstar_extract = True, auth_patterns = {}, platforms = { "{}_{}".format(os, cpu): _plat( @@ -1095,6 +1098,7 @@ def _test_pipstar_platforms(env): enable_pipstar = True, config = struct( enable_pipstar = True, + enable_pipstar_extract = True, netrc = None, auth_patterns = {}, platforms = { @@ -1179,6 +1183,7 @@ def _test_pipstar_platforms_limit(env): enable_pipstar = True, config = struct( enable_pipstar = True, + enable_pipstar_extract = True, netrc = None, auth_patterns = {}, platforms = { From 7f1fc2a353d8d44c76c03bad1dc2116c3dbf8c4d Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Sat, 20 Dec 2025 00:04:17 +0900 Subject: [PATCH 564/922] build(deps): bump bazel-contrib/publish-to-bcr/.github/workflows/publish.yaml from 1.0.0 to 1.1.0 (#3465) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Bumps [bazel-contrib/publish-to-bcr/.github/workflows/publish.yaml](https://github.com/bazel-contrib/publish-to-bcr) from 1.0.0 to 1.1.0.
Release notes

Sourced from bazel-contrib/publish-to-bcr/.github/workflows/publish.yaml's releases.

v1.1.0

What's Changed

New Contributors

Full Changelog: https://github.com/bazel-contrib/publish-to-bcr/compare/v1.0.0...v1.1.0

Commits
  • 0bd40ad chore(deps): update bazel-contrib/publish-to-bcr digest to ff2c5e3 (#283)
  • ff2c5e3 feat(workflow): support deployment environments (#310)
  • 7fcb100 fix: mismatched source and output bundles (#311)
  • aa49ba0 fix: show a helpful error when module name missing (#302)
  • 97e76f5 chore(deps): update dependency babel-plugin-transform-import-meta to v2.3.3 (...
  • 6acdd42 chore(deps): update dependency gcp-metadata to v6.1.1 (#307)
  • 08d0ade chore(deps): update dependency @​types/nodemailer to v6.4.21 (#303)
  • 9b0c9d4 chore(deps): update dependency @​types/yargs to v17.0.35 (#304)
  • 6fc1de9 chore(deps): update dependency @​types/mailparser to v3.4.6 (#292)
  • f120aa5 chore(deps): update dependency @​types/node to v18.19.130 (#301)
  • Additional commits viewable in compare view

[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=bazel-contrib/publish-to-bcr/.github/workflows/publish.yaml&package-manager=github_actions&previous-version=1.0.0&new-version=1.1.0)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) ---
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot show ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> --- .github/workflows/publish.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.github/workflows/publish.yml b/.github/workflows/publish.yml index 9ad5308968..fb46168fc1 100644 --- a/.github/workflows/publish.yml +++ b/.github/workflows/publish.yml @@ -21,7 +21,7 @@ on: type: string jobs: publish: - uses: bazel-contrib/publish-to-bcr/.github/workflows/publish.yaml@v1.0.0 + uses: bazel-contrib/publish-to-bcr/.github/workflows/publish.yaml@v1.1.0 with: tag_name: ${{ inputs.tag_name }} # GitHub repository which is a fork of the upstream where the Pull Request will be opened. From 9a20fc243a23399511023d78209590f776b36e38 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Fri, 19 Dec 2025 20:02:33 -0800 Subject: [PATCH 565/922] chore: update version markers for 1.8 release (#3472) Update version markers for the 1.8 release. Work towards https://github.com/bazel-contrib/rules_python/issues/3466 --- CHANGELOG.md | 18 +++++++++--------- python/private/py_library.bzl | 2 +- python/private/pypi/extension.bzl | 2 +- python/private/pypi/hub_builder.bzl | 2 +- python/private/python.bzl | 2 +- 5 files changed, 13 insertions(+), 13 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index fcf56348fc..aceccbbb85 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -48,18 +48,18 @@ END_UNRELEASED_TEMPLATE --> -{#v0-0-0} -## Unreleased +{#v1-8-0} +## [1.8.0] - 2025-12-19 -[0.0.0]: https://github.com/bazel-contrib/rules_python/releases/tag/0.0.0 +[1.8.0]: https://github.com/bazel-contrib/rules_python/releases/tag/1.8.0 -{#v0-0-0-known-issues} +{#v1-8-0-known-issues} ### Known Issues * (gazelle) Windows support for the Gazelle plugin may be broken. See [#3416](https://github.com/bazel-contrib/rules_python/issues/3416) for details and possible workarounds. -{#v0-0-0-removed} +{#v1-8-0-removed} ### Removed * (toolchain) Remove all of the python 3.8 toolchain support out of the box. Users need to pass the `TOOL_VERSIONS` that include 3.8 toolchains or use the `bzlmod` APIs to add @@ -74,7 +74,7 @@ END_UNRELEASED_TEMPLATE the toolchains in the repository phase, ensure that you pass `-B` to the python interpreter. ([#2016](https://github.com/bazel-contrib/rules_python/issues/2016)) -{#v0-0-0-changed} +{#v1-8-0-changed} ### Changed * (toolchains) Use toolchains from the [20251031] release. * (gazelle) Internally split modules mapping generation to be per-wheel for concurrency and caching. @@ -88,7 +88,7 @@ END_UNRELEASED_TEMPLATE * (gazelle deps) rules_go bumped from 0.55.1 to 0.59.0 * (gazelle deps) gazelle bumped from 0.36.0 to 0.47.0 -{#v0-0-0-fixed} +{#v1-8-0-fixed} ### Fixed * (gazelle) Remove {obj}`py_binary` targets with invalid `srcs`. This includes files that are not generated or regular files. @@ -109,7 +109,7 @@ END_UNRELEASED_TEMPLATE * (core rules) For the system_python bootstrap, the runfiles root is added to sys.path. -{#v0-0-0-added} +{#v1-8-0-added} ### Added * (toolchains) `3.9.25` Python toolchain from [20251031] release. * (toolchains) `3.13.10`, `3.14.1` Python toolchain from [20251202] release. @@ -2065,4 +2065,4 @@ Breaking changes: * (pip) Create all_data_requirements alias * Expose Python C headers through the toolchain. -[0.24.0]: https://github.com/bazel-contrib/rules_python/releases/tag/0.24.0 +[0.24.0]: https://github.com/bazel-contrib/rules_python/releases/tag/0.24.0 \ No newline at end of file diff --git a/python/private/py_library.bzl b/python/private/py_library.bzl index 7e9a59f277..6edb25abae 100644 --- a/python/private/py_library.bzl +++ b/python/private/py_library.bzl @@ -111,7 +111,7 @@ When {obj}`--venv_site_packages=yes` is set, this helps inform which directories treated as namespace packages and expect files from other targets to be contributed. This allows optimizing the generation of symlinks to be cheaper at analysis time. -:::{versionadded} VERSION_NEXT_FEATURE +:::{versionadded} 1.8.0 ::: """, ), diff --git a/python/private/pypi/extension.bzl b/python/private/pypi/extension.bzl index e9033996c6..3927f61c00 100644 --- a/python/private/pypi/extension.bzl +++ b/python/private/pypi/extension.bzl @@ -687,7 +687,7 @@ a string `"{os}_{arch}"` as the value here. You could also use `"{os}_{arch}_fre :::{include} /_includes/experimental_api.md ::: -:::{versionadded} VERSION_NEXT_FEATURE +:::{versionadded} 1.8.0 ::: """, ), diff --git a/python/private/pypi/hub_builder.bzl b/python/private/pypi/hub_builder.bzl index 95e007aa69..700f22e2c0 100644 --- a/python/private/pypi/hub_builder.bzl +++ b/python/private/pypi/hub_builder.bzl @@ -142,7 +142,7 @@ def _pip_parse(self, module_ctx, pip_attr): python_version = full_python_version, config = self._config, # TODO @aignas 2025-12-09: flip or part to default to 'os_arch' after - # VERSION_NEXT_FEATURE is released and set the default of the `target_platforms` attribute + # 1.8.0 is released and set the default of the `target_platforms` attribute # to `{os}_{arch}`. target_platforms = pip_attr.target_platforms or ([] if default_cross_setup else ["{os}_{arch}"]), ) diff --git a/python/private/python.bzl b/python/private/python.bzl index 80f2afac53..399743c18d 100644 --- a/python/private/python.bzl +++ b/python/private/python.bzl @@ -1065,7 +1065,7 @@ Then the python interpreter will be available as `my_python_name`. "ignore_root_user_error": attr.bool( default = True, doc = """\ -:::{versionchanged} VERSION_NEXT_FEATURE +:::{versionchanged} 1.8.0 Noop, will be removed in the next major release. ::: """, From e58f3962b1ebd8e089efe45c0ed3e8610cec24b1 Mon Sep 17 00:00:00 2001 From: Shayan Hoshyari <108962133+shayanhoshyari@users.noreply.github.com> Date: Mon, 22 Dec 2025 12:23:20 -0800 Subject: [PATCH 566/922] fix(venv): Fix all .so files missing when py_binary lives at //:BUILD (#3474) When `.label.package` was "" (empty string), doing `"{}/{}".format(ctx.label.package, bin_venv_path)` created an absolute path, which caused files to be ignored later, as they no longer looked to be part of the correct prefix. An empty package name occurs when the target is in `//:BUILD` with venv on. To fix, use skylib's `paths` instead, which handles it correctly and won't add the `/` prefix. Fixes https://github.com/bazel-contrib/rules_python/issues/3470 Co-authored-by: Shayan Hoshyari --- python/private/venv_runfiles.bzl | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/python/private/venv_runfiles.bzl b/python/private/venv_runfiles.bzl index 851d7015c6..02b18e57e8 100644 --- a/python/private/venv_runfiles.bzl +++ b/python/private/venv_runfiles.bzl @@ -64,7 +64,8 @@ def create_venv_app_files(ctx, deps, venv_dir_map): for venv_path, link_to in kind_map.items(): bin_venv_path = paths.join(base, venv_path) if is_file(link_to): - symlink_from = "{}/{}".format(ctx.label.package, bin_venv_path) + # use paths.join to handle ctx.label.package = "" + symlink_from = paths.join(ctx.label.package, bin_venv_path) runfiles_symlinks[symlink_from] = link_to else: venv_link = ctx.actions.declare_symlink(bin_venv_path) From da822a8ab5a00d3f8f039f6966690c02aa7e13a1 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Tue, 23 Dec 2025 18:53:49 -0800 Subject: [PATCH 567/922] refactor: remove most of semantics (#3475) This removes the majority of the "semantics" plugin objects. These are a holdover from making the code more amenable towards being extended by the Google implementation. py_library semantics are completely removed. Most of py_binary has had semantics removed. I've left the debugger, build data, and cc deps hooks in for now, since that functionality can still be folded into the code. --- python/private/common.bzl | 83 +----------------------------- python/private/py_executable.bzl | 88 +++++++------------------------- python/private/py_library.bzl | 27 +++------- 3 files changed, 28 insertions(+), 170 deletions(-) diff --git a/python/private/common.bzl b/python/private/common.bzl index ddeea6ed2d..19f2f39215 100644 --- a/python/private/common.bzl +++ b/python/private/common.bzl @@ -43,22 +43,10 @@ PYTHON_FILE_EXTENSIONS = [ def create_binary_semantics_struct( *, - create_executable, - get_cc_details_for_binary, get_central_uncachable_version_file, - get_coverage_deps, get_debugger_deps, - get_extra_common_runfiles_for_binary, - get_extra_providers, - get_extra_write_build_data_env, - get_interpreter_path, - get_imports, get_native_deps_dso_name, - get_native_deps_user_link_flags, - get_stamp_flag, - maybe_precompile, should_build_native_deps_dso, - should_create_init_files, should_include_build_data): """Helper to ensure a semantics struct has all necessary fields. @@ -66,42 +54,15 @@ def create_binary_semantics_struct( the necessary functions are being correctly provided. Args: - create_executable: Callable; creates a binary's executable output. See - py_executable.bzl#py_executable_base_impl for details. - get_cc_details_for_binary: Callable that returns a `CcDetails` struct; see - `create_cc_detail_struct`. get_central_uncachable_version_file: Callable that returns an optional Artifact; this artifact is special: it is never cached and is a copy of `ctx.version_file`; see py_builtins.copy_without_caching - get_coverage_deps: Callable that returns a list of Targets for making - coverage work; only called if coverage is enabled. get_debugger_deps: Callable that returns a list of Targets that provide custom debugger support; only called for target-configuration. - get_extra_common_runfiles_for_binary: Callable that returns a runfiles - object of extra runfiles a binary should include. - get_extra_providers: Callable that returns extra providers; see - py_executable.bzl#_create_providers for details. - get_extra_write_build_data_env: Callable that returns a dict[str, str] - of additional environment variable to pass to build data generation. - get_interpreter_path: Callable that returns an optional string, which is - the path to the Python interpreter to use for running the binary. - get_imports: Callable that returns a list of the target's import - paths (from the `imports` attribute, so just the target's own import - path strings, not from dependencies). get_native_deps_dso_name: Callable that returns a string, which is the basename (with extension) of the native deps DSO library. - get_native_deps_user_link_flags: Callable that returns a list of strings, - which are any extra linker flags to pass onto the native deps DSO - linking action. - get_stamp_flag: Callable that returns bool of if the --stamp flag was - enabled or not. - maybe_precompile: Callable that may optional precompile the input `.py` - sources and returns the full set of desired outputs derived from - the source files (e.g., both py and pyc, only one of them, etc). should_build_native_deps_dso: Callable that returns bool; True if building a native deps DSO is supported, False if not. - should_create_init_files: Callable that returns bool; True if - `__init__.py` files should be generated, False if not. should_include_build_data: Callable that returns bool; True if build data should be generated, False if not. Returns: @@ -109,50 +70,13 @@ def create_binary_semantics_struct( """ return struct( # keep-sorted - create_executable = create_executable, - get_cc_details_for_binary = get_cc_details_for_binary, get_central_uncachable_version_file = get_central_uncachable_version_file, - get_coverage_deps = get_coverage_deps, get_debugger_deps = get_debugger_deps, - get_extra_common_runfiles_for_binary = get_extra_common_runfiles_for_binary, - get_extra_providers = get_extra_providers, - get_extra_write_build_data_env = get_extra_write_build_data_env, - get_imports = get_imports, - get_interpreter_path = get_interpreter_path, get_native_deps_dso_name = get_native_deps_dso_name, - get_native_deps_user_link_flags = get_native_deps_user_link_flags, - get_stamp_flag = get_stamp_flag, - maybe_precompile = maybe_precompile, should_build_native_deps_dso = should_build_native_deps_dso, - should_create_init_files = should_create_init_files, should_include_build_data = should_include_build_data, ) -def create_library_semantics_struct( - *, - get_cc_info_for_library, - get_imports, - maybe_precompile): - """Create a `LibrarySemantics` struct. - - Call this instead of a raw call to `struct(...)`; it'll help ensure all - the necessary functions are being correctly provided. - - Args: - get_cc_info_for_library: Callable that returns a CcInfo for the library; - see py_library_impl for arg details. - get_imports: Callable; see create_binary_semantics_struct. - maybe_precompile: Callable; see create_binary_semantics_struct. - Returns: - a `LibrarySemantics` struct. - """ - return struct( - # keep sorted - get_cc_info_for_library = get_cc_info_for_library, - get_imports = get_imports, - maybe_precompile = maybe_precompile, - ) - def create_cc_details_struct( *, cc_info_for_propagating, @@ -255,12 +179,11 @@ def collect_cc_info(ctx, extra_deps = []): return cc_common.merge_cc_infos(cc_infos = cc_infos) -def collect_imports(ctx, semantics): +def collect_imports(ctx): """Collect the direct and transitive `imports` strings. Args: ctx: {type}`ctx` the current target ctx - semantics: semantics object for fetching direct imports. Returns: {type}`depset[str]` of import paths @@ -271,13 +194,11 @@ def collect_imports(ctx, semantics): transitive.append(dep[PyInfo].imports) if BuiltinPyInfo != None and BuiltinPyInfo in dep: transitive.append(dep[BuiltinPyInfo].imports) - return depset(direct = semantics.get_imports(ctx), transitive = transitive) + return depset(direct = get_imports(ctx), transitive = transitive) def get_imports(ctx): """Gets the imports from a rule's `imports` attribute. - See create_binary_semantics_struct for details about this function. - Args: ctx: Rule ctx. diff --git a/python/private/py_executable.bzl b/python/private/py_executable.bzl index 9084454c65..2e167b99ab 100644 --- a/python/private/py_executable.bzl +++ b/python/private/py_executable.bzl @@ -47,7 +47,6 @@ load( "create_py_info", "csv", "filter_to_py_srcs", - "get_imports", "is_bool", "relative_path", "runfiles_root_path", @@ -267,42 +266,18 @@ def py_executable_impl(ctx, *, is_test, inherited_environment): def create_binary_semantics(): return create_binary_semantics_struct( # keep-sorted start - create_executable = _create_executable, - get_cc_details_for_binary = _get_cc_details_for_binary, get_central_uncachable_version_file = lambda ctx: None, - get_coverage_deps = _get_coverage_deps, get_debugger_deps = _get_debugger_deps, - get_extra_common_runfiles_for_binary = lambda ctx: ctx.runfiles(), - get_extra_providers = _get_extra_providers, - get_extra_write_build_data_env = lambda ctx: {}, - get_imports = get_imports, - get_interpreter_path = _get_interpreter_path, get_native_deps_dso_name = _get_native_deps_dso_name, - get_native_deps_user_link_flags = _get_native_deps_user_link_flags, - get_stamp_flag = _get_stamp_flag, - maybe_precompile = maybe_precompile, should_build_native_deps_dso = lambda ctx: False, - should_create_init_files = _should_create_init_files, should_include_build_data = lambda ctx: False, # keep-sorted end ) -def _get_coverage_deps(ctx, runtime_details): - _ = ctx, runtime_details # @unused - return [] - def _get_debugger_deps(ctx, runtime_details): _ = ctx, runtime_details # @unused return [] -def _get_extra_providers(ctx, main_py, runtime_details): - _ = ctx, main_py, runtime_details # @unused - return [] - -def _get_stamp_flag(ctx): - # NOTE: Undocumented API; private to builtins - return ctx.configuration.stamp_binaries - def _should_create_init_files(ctx): if ctx.attr.legacy_create_init == -1: return not read_possibly_native_flag(ctx, "default_to_explicit_init_py") @@ -994,10 +969,6 @@ def _get_native_deps_dso_name(ctx): _ = ctx # @unused fail("Building native deps DSO not supported.") -def _get_native_deps_user_link_flags(ctx): - _ = ctx # @unused - fail("Building native deps DSO not supported.") - def py_executable_base_impl(ctx, *, semantics, is_test, inherited_environment = []): """Base rule implementation for a Python executable. @@ -1022,7 +993,7 @@ def py_executable_base_impl(ctx, *, semantics, is_test, inherited_environment = else: main_py = None direct_sources = filter_to_py_srcs(ctx.files.srcs) - precompile_result = semantics.maybe_precompile(ctx, direct_sources) + precompile_result = maybe_precompile(ctx, direct_sources) required_py_files = precompile_result.keep_srcs required_pyc_files = [] @@ -1046,20 +1017,17 @@ def py_executable_base_impl(ctx, *, semantics, is_test, inherited_environment = default_outputs.add(precompile_result.keep_srcs) default_outputs.add(required_pyc_files) - imports = collect_imports(ctx, semantics) + imports = collect_imports(ctx) - runtime_details = _get_runtime_details(ctx, semantics) - if ctx.configuration.coverage_enabled: - extra_deps = semantics.get_coverage_deps(ctx, runtime_details) - else: - extra_deps = [] + runtime_details = _get_runtime_details(ctx) + extra_deps = [] # The debugger dependency should be prevented by select() config elsewhere, # but just to be safe, also guard against adding it to the output here. if not _is_tool_config(ctx): extra_deps.extend(semantics.get_debugger_deps(ctx, runtime_details)) - cc_details = semantics.get_cc_details_for_binary(ctx, extra_deps = extra_deps) + cc_details = _get_cc_details_for_binary(ctx, extra_deps = extra_deps) native_deps_details = _get_native_deps_details( ctx, semantics = semantics, @@ -1078,11 +1046,10 @@ def py_executable_base_impl(ctx, *, semantics, is_test, inherited_environment = runtime_details.runfiles, cc_details.extra_runfiles, native_deps_details.runfiles, - semantics.get_extra_common_runfiles_for_binary(ctx), ], semantics = semantics, ) - exec_result = semantics.create_executable( + exec_result = _create_executable( ctx, executable = executable, main_py = main_py, @@ -1122,7 +1089,6 @@ def py_executable_base_impl(ctx, *, semantics, is_test, inherited_environment = runtime_details = runtime_details, cc_info = cc_details.cc_info_for_propagating, inherited_environment = inherited_environment, - semantics = semantics, output_groups = exec_result.output_groups, ) @@ -1154,7 +1120,7 @@ def _declare_executable_file(ctx): return executable -def _get_runtime_details(ctx, semantics): +def _get_runtime_details(ctx): """Gets various information about the Python runtime to use. While most information comes from the toolchain, various legacy and @@ -1162,7 +1128,6 @@ def _get_runtime_details(ctx, semantics): Args: ctx: Rule ctx - semantics: A `BinarySemantics` struct; see `create_binary_semantics_struct` Returns: A struct; see inline-field comments of the return value for details. @@ -1203,7 +1168,7 @@ def _get_runtime_details(ctx, semantics): else: runtime_files = depset() - executable_interpreter_path = semantics.get_interpreter_path( + executable_interpreter_path = _get_interpreter_path( ctx, runtime = effective_runtime, flag_interpreter_path = flag_interpreter_path, @@ -1336,7 +1301,7 @@ def _get_base_runfiles_for_binary( common_runfiles = common_runfiles.build(ctx) - if semantics.should_create_init_files(ctx): + if _should_create_init_files(ctx): common_runfiles = _py_builtins.merge_runfiles_with_generated_inits_empty_files_supplier( ctx = ctx, runfiles = common_runfiles, @@ -1352,11 +1317,10 @@ def _get_base_runfiles_for_binary( # removed and another way found to locate the underlying build data file. data_runfiles = runfiles_with_exe - if is_stamping_enabled(ctx, semantics) and semantics.should_include_build_data(ctx): + if is_stamping_enabled(ctx) and semantics.should_include_build_data(ctx): build_data_file, build_data_runfiles = _create_runfiles_with_build_data( ctx, semantics.get_central_uncachable_version_file(ctx), - semantics.get_extra_write_build_data_env(ctx), ) default_runfiles = runfiles_with_exe.merge(build_data_runfiles) else: @@ -1372,19 +1336,17 @@ def _get_base_runfiles_for_binary( def _create_runfiles_with_build_data( ctx, - central_uncachable_version_file, - extra_write_build_data_env): + central_uncachable_version_file): build_data_file = _write_build_data( ctx, central_uncachable_version_file, - extra_write_build_data_env, ) build_data_runfiles = ctx.runfiles(files = [ build_data_file, ]) return build_data_file, build_data_runfiles -def _write_build_data(ctx, central_uncachable_version_file, extra_write_build_data_env): +def _write_build_data(ctx, central_uncachable_version_file): # TODO: Remove this logic when a central file is always available if not central_uncachable_version_file: version_file = ctx.actions.declare_file(ctx.label.name + "-uncachable_version_file.txt") @@ -1430,7 +1392,7 @@ def _write_build_data(ctx, central_uncachable_version_file, extra_write_build_da ctx.actions.run( executable = ctx.executable._build_data_gen, - env = dicts.add({ + env = { # NOTE: ctx.info_file is undocumented; see # https://github.com/bazelbuild/bazel/issues/9363 "INFO_FILE": ctx.info_file.path, @@ -1438,7 +1400,7 @@ def _write_build_data(ctx, central_uncachable_version_file, extra_write_build_da "PLATFORM": cc_helper.find_cpp_toolchain(ctx).toolchain_id, "TARGET": str(ctx.label), "VERSION_FILE": version_file.path, - }, extra_write_build_data_env), + }, inputs = depset( direct = direct_inputs, ), @@ -1489,12 +1451,9 @@ def _get_native_deps_details(ctx, *, semantics, cc_details, is_test): feature_configuration = cc_feature_config.feature_configuration, cc_toolchain = cc_details.cc_toolchain, test_only_target = is_test, # private - stamp = 1 if is_stamping_enabled(ctx, semantics) else 0, + stamp = 1 if is_stamping_enabled(ctx) else 0, main_output = linked_lib, # private use_shareable_artifact_factory = True, # private - # NOTE: Only flags not captured by cc_info.linking_context need to - # be manually passed - user_link_flags = semantics.get_native_deps_user_link_flags(ctx), ) return struct( dso = dso, @@ -1634,12 +1593,11 @@ def _path_endswith(path, endswith): # "ab/c.py".endswith("b/c.py") from incorrectly matching. return ("/" + path).endswith("/" + endswith) -def is_stamping_enabled(ctx, semantics): +def is_stamping_enabled(ctx): """Tells if stamping is enabled or not. Args: ctx: The rule ctx - semantics: a semantics struct (see create_semantics_struct). Returns: bool; True if stamping is enabled, False if not. """ @@ -1652,7 +1610,8 @@ def is_stamping_enabled(ctx, semantics): elif stamp == 0: return False elif stamp == -1: - return semantics.get_stamp_flag(ctx) + # NOTE: Undocumented API; private to builtins + return ctx.configuration.stamp_binaries else: fail("Unsupported `stamp` value: {}".format(stamp)) @@ -1678,8 +1637,7 @@ def _create_providers( cc_info, inherited_environment, runtime_details, - output_groups, - semantics): + output_groups): """Creates the providers an executable should return. Args: @@ -1708,7 +1666,6 @@ def _create_providers( is run within. runtime_details: struct of runtime information; see _get_runtime_details() output_groups: dict[str, depset[File]]; used to create OutputGroupInfo - semantics: BinarySemantics struct; see create_binary_semantics() Returns: A list of modern providers. @@ -1783,13 +1740,6 @@ def _create_providers( if builtin_py_info: providers.append(builtin_py_info) providers.append(create_output_group_info(py_info.transitive_sources, output_groups)) - - extra_providers = semantics.get_extra_providers( - ctx, - main_py = main_py, - runtime_details = runtime_details, - ) - providers.extend(extra_providers) return providers def _create_run_environment_info(ctx, inherited_environment): diff --git a/python/private/py_library.bzl b/python/private/py_library.bzl index 6edb25abae..de73a53720 100644 --- a/python/private/py_library.bzl +++ b/python/private/py_library.bzl @@ -32,11 +32,9 @@ load( "collect_imports", "collect_runfiles", "create_instrumented_files_info", - "create_library_semantics_struct", "create_output_group_info", "create_py_info", "filter_to_py_srcs", - "get_imports", ) load(":common_labels.bzl", "labels") load(":flags.bzl", "AddSrcsToRunfilesFlag", "PrecompileFlag", "VenvsSitePackages") @@ -121,29 +119,18 @@ This allows optimizing the generation of symlinks to be cheaper at analysis time }, ) -def _py_library_impl_with_semantics(ctx): - return py_library_impl( - ctx, - semantics = create_library_semantics_struct( - get_imports = get_imports, - maybe_precompile = maybe_precompile, - get_cc_info_for_library = collect_cc_info, - ), - ) - -def py_library_impl(ctx, *, semantics): +def py_library_impl(ctx): """Abstract implementation of py_library rule. Args: ctx: The rule ctx - semantics: A `LibrarySemantics` struct; see `create_library_semantics_struct` Returns: A list of modern providers to propagate. """ direct_sources = filter_to_py_srcs(ctx.files.srcs) - precompile_result = semantics.maybe_precompile(ctx, direct_sources) + precompile_result = maybe_precompile(ctx, direct_sources) required_py_files = precompile_result.keep_srcs required_pyc_files = [] @@ -172,9 +159,9 @@ def py_library_impl(ctx, *, semantics): imports = [] venv_symlinks = [] - imports, venv_symlinks = _get_imports_and_venv_symlinks(ctx, semantics) + imports, venv_symlinks = _get_imports_and_venv_symlinks(ctx) - cc_info = semantics.get_cc_info_for_library(ctx) + cc_info = collect_cc_info(ctx) py_info, builtins_py_info = create_py_info( ctx, original_sources = direct_sources, @@ -243,7 +230,7 @@ def _get_package_and_version(ctx): version.normalize(version_str), # will have no dashes either ) -def _get_imports_and_venv_symlinks(ctx, semantics): +def _get_imports_and_venv_symlinks(ctx): imports = depset() venv_symlinks = [] if VenvsSitePackages.is_enabled(ctx): @@ -268,7 +255,7 @@ def _get_imports_and_venv_symlinks(ctx, semantics): namespace_package_files = ctx.files.namespace_package_files, ) else: - imports = collect_imports(ctx, semantics) + imports = collect_imports(ctx) return imports, venv_symlinks _MaybeBuiltinPyInfo = [BuiltinPyInfo] if BuiltinPyInfo != None else [] @@ -288,7 +275,7 @@ def create_py_library_rule_builder(): for creating a `py_library` rule. """ builder = ruleb.Rule( - implementation = _py_library_impl_with_semantics, + implementation = py_library_impl, doc = _DEFAULT_PY_LIBRARY_DOC, exec_groups = dict(REQUIRED_EXEC_GROUP_BUILDERS), attrs = LIBRARY_ATTRS, From a94bd0fdde426bf30efed7c819422d74b404cc18 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Tue, 23 Dec 2025 23:15:03 -0800 Subject: [PATCH 568/922] feat: add --debugger flag (#3478) The --debugger flag is useful for injecting a user-specified dependency without having to modify the binary or test. Similarly, tests now implicitly inherit the `PYTHONBREAKPOINT` environment variable. The dependency is only added for the target config because build tools can't be intercepted for debugging. --------- Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> --- CHANGELOG.md | 23 ++++++- .../python/config_settings/index.md | 21 ++++++ docs/howto/debuggers.md | 66 +++++++++++++++++++ python/config_settings/BUILD.bazel | 6 ++ python/private/BUILD.bazel | 18 +++++ python/private/bazel_config_mode.bzl | 12 ++++ python/private/common.bzl | 4 -- python/private/common_labels.bzl | 1 + python/private/py_executable.bzl | 13 ++-- python/private/transition_labels.bzl | 1 + python/py_binary.bzl | 5 ++ tests/base_rules/py_executable_base_tests.bzl | 39 +++++++++++ 12 files changed, 198 insertions(+), 11 deletions(-) create mode 100644 docs/howto/debuggers.md create mode 100644 python/private/bazel_config_mode.bzl diff --git a/CHANGELOG.md b/CHANGELOG.md index aceccbbb85..e92b3c2737 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -47,6 +47,27 @@ BEGIN_UNRELEASED_TEMPLATE END_UNRELEASED_TEMPLATE --> +{#v0-0-0} +## Unreleased + +[0.0.0]: https://github.com/bazel-contrib/rules_python/releases/tag/0.0.0 + +{#v0-0-0-removed} +### Removed +* Nothing removed. + +{#v0-0-0-changed} +### Changed +* (binaries/tests) The `PYTHONBREAKPOINT` environment variable is automatically inherited + +{#v0-0-0-fixed} +### Fixed +* Nothing fixed. + +{#v0-0-0-added} +### Added +* (binaries/tests) {obj}`--debugger`: allows specifying an extra dependency + to add to binaries/tests for custom debuggers. {#v1-8-0} ## [1.8.0] - 2025-12-19 @@ -2065,4 +2086,4 @@ Breaking changes: * (pip) Create all_data_requirements alias * Expose Python C headers through the toolchain. -[0.24.0]: https://github.com/bazel-contrib/rules_python/releases/tag/0.24.0 \ No newline at end of file +[0.24.0]: https://github.com/bazel-contrib/rules_python/releases/tag/0.24.0 diff --git a/docs/api/rules_python/python/config_settings/index.md b/docs/api/rules_python/python/config_settings/index.md index 3092326d6f..d92e7d404f 100644 --- a/docs/api/rules_python/python/config_settings/index.md +++ b/docs/api/rules_python/python/config_settings/index.md @@ -45,6 +45,27 @@ This flag replaces the Bazel builtin `--build_python_zip` flag. ::: :::: +::::{bzl:flag} debugger +A target for providing a custom debugger dependency. + +This flag is roughly equivalent to putting a target in `deps`. It allows +injecting a dependency into executables (`py_binary`, `py_test`) without having +to modify their deps. The expectation is it points to a target that provides an +alternative debugger (pudb, winpdb, debugpy, etc). + +* Must provide {obj}`PyInfo`. +* This dependency is only used for the target config, i.e. build tools don't + have it added. + +:::{note} +Setting this flag adds the debugger dependency, but doesn't automatically set +`PYTHONBREAKPOINT` to change `breakpoint()` behavior. +::: + +:::{versionadded} VERSION_NEXT_FEATURE +::: +:::: + ::::{bzl:flag} experimental_python_import_all_repositories Controls whether repository directories are added to the import path. diff --git a/docs/howto/debuggers.md b/docs/howto/debuggers.md new file mode 100644 index 0000000000..3f75712b0f --- /dev/null +++ b/docs/howto/debuggers.md @@ -0,0 +1,66 @@ +:::{default-domain} bzl +::: + +# How to integrate a debugger + +This guide explains how to use the {obj}`--debugger` flag to integrate a debugger +with your Python applications built with `rules_python`. + +## Basic Usage + +The {obj}`--debugger` flag allows you to inject an extra dependency into `py_test` +and `py_binary` targets so that they have a custom debugger available at +runtime. The flag is roughly equivalent to manually adding it to `deps` of +the target under test. + +To use the debugger, you typically provide the `--debugger` flag to your `bazel run` command. + +Example command line: + +```bash +bazel run --@rules_python//python/config_settings:debugger=@pypi//pudb \ + //path/to:my_python_binary +``` + +This will launch the Python program with the `@pypi//pudb` dependency added. + +The exact behavior (e.g., waiting for attachment, breaking at the first line) +depends on the specific debugger and its configuration. + +:::{note} +The specified target must be in the requirements.txt file used with +`pip.parse()` to make it available to Bazel. +::: + +## Python `PYTHONBREAKPOINT` Environment Variable + +For more fine-grained control over debugging, especially for programmatic breakpoints, +you can leverage the Python built-in `breakpoint()` function and the +`PYTHONBREAKPOINT` environment variable. + +The `breakpoint()` built-in function, available since Python 3.7, +can be called anywhere in your code to invoke a debugger. The `PYTHONBREAKPOINT` +environment variable can be set to specify which debugger to use. + +For example, to use `pdb` (the Python Debugger) when `breakpoint()` is called: + +```bash +PYTHONBREAKPOINT=pudb.set_trace bazel run \ + --@rules_python//python/config_settings:debugger=@pypi//pudb \ + //path/to:my_python_binary +``` + +For more details on `PYTHONBREAKPOINT`, refer to the [Python documentation](https://docs.python.org/3/library/functions.html#breakpoint). + +## Setting a default debugger + +By adding settings to your user or project `.bazelrc` files, you can have +these settings automatically added to your bazel invocations. e.g. + +``` +common --@rules_python//python/config_settings:debugger=@pypi//pudb +common --test_env=PYTHONBREAKPOINT=pudb.set_trace +``` + +Note that `--test_env` isn't strictly necessary. The `py_test` and `py_binary` +rules will respect the `PYTHONBREAKPOINT` environment variable in your shell. diff --git a/python/config_settings/BUILD.bazel b/python/config_settings/BUILD.bazel index 369989eb1e..7060d50b26 100644 --- a/python/config_settings/BUILD.bazel +++ b/python/config_settings/BUILD.bazel @@ -102,6 +102,12 @@ rp_string_flag( visibility = ["//visibility:public"], ) +label_flag( + name = "debugger", + build_setting_default = "//python/private:empty", + visibility = ["//visibility:public"], +) + # For some reason, @platforms//os:windows can't be directly used # in the select() for the flag. But it can be used when put behind # a config_setting(). diff --git a/python/private/BUILD.bazel b/python/private/BUILD.bazel index e92c45dad4..13cbfafade 100644 --- a/python/private/BUILD.bazel +++ b/python/private/BUILD.bazel @@ -16,6 +16,7 @@ load("@bazel_skylib//:bzl_library.bzl", "bzl_library") load("@bazel_skylib//rules:common_settings.bzl", "bool_setting") load("//python:py_binary.bzl", "py_binary") load("//python:py_library.bzl", "py_library") +load(":bazel_config_mode.bzl", "bazel_config_mode") load(":print_toolchain_checksums.bzl", "print_toolchains_checksums") load(":py_exec_tools_toolchain.bzl", "current_interpreter_executable") load(":sentinel.bzl", "sentinel") @@ -810,6 +811,23 @@ config_setting( }, ) +config_setting( + name = "is_bazel_config_mode_target", + flag_values = { + "//python/private:bazel_config_mode": "target", + }, +) + +alias( + name = "debugger_if_target_config", + actual = select({ + ":is_bazel_config_mode_target": "//python/config_settings:debugger", + "//conditions:default": "//python/private:empty", + }), +) + +bazel_config_mode(name = "bazel_config_mode") + # This should only be set by analysis tests to expose additional metadata to # aid testing, so a setting instead of a flag. bool_setting( diff --git a/python/private/bazel_config_mode.bzl b/python/private/bazel_config_mode.bzl new file mode 100644 index 0000000000..ec6be5c83b --- /dev/null +++ b/python/private/bazel_config_mode.bzl @@ -0,0 +1,12 @@ +"""Flag to tell if exec or target mode is active.""" + +load(":py_internal.bzl", "py_internal") + +def _bazel_config_mode_impl(ctx): + return [config_common.FeatureFlagInfo( + value = "exec" if py_internal.is_tool_configuration(ctx) else "target", + )] + +bazel_config_mode = rule( + implementation = _bazel_config_mode_impl, +) diff --git a/python/private/common.bzl b/python/private/common.bzl index 19f2f39215..a593e97558 100644 --- a/python/private/common.bzl +++ b/python/private/common.bzl @@ -44,7 +44,6 @@ PYTHON_FILE_EXTENSIONS = [ def create_binary_semantics_struct( *, get_central_uncachable_version_file, - get_debugger_deps, get_native_deps_dso_name, should_build_native_deps_dso, should_include_build_data): @@ -57,8 +56,6 @@ def create_binary_semantics_struct( get_central_uncachable_version_file: Callable that returns an optional Artifact; this artifact is special: it is never cached and is a copy of `ctx.version_file`; see py_builtins.copy_without_caching - get_debugger_deps: Callable that returns a list of Targets that provide - custom debugger support; only called for target-configuration. get_native_deps_dso_name: Callable that returns a string, which is the basename (with extension) of the native deps DSO library. should_build_native_deps_dso: Callable that returns bool; True if @@ -71,7 +68,6 @@ def create_binary_semantics_struct( return struct( # keep-sorted get_central_uncachable_version_file = get_central_uncachable_version_file, - get_debugger_deps = get_debugger_deps, get_native_deps_dso_name = get_native_deps_dso_name, should_build_native_deps_dso = should_build_native_deps_dso, should_include_build_data = should_include_build_data, diff --git a/python/private/common_labels.bzl b/python/private/common_labels.bzl index e90679eb6f..9c21198a62 100644 --- a/python/private/common_labels.bzl +++ b/python/private/common_labels.bzl @@ -8,6 +8,7 @@ labels = struct( ADD_SRCS_TO_RUNFILES = str(Label("//python/config_settings:add_srcs_to_runfiles")), BOOTSTRAP_IMPL = str(Label("//python/config_settings:bootstrap_impl")), BUILD_PYTHON_ZIP = str(Label("//python/config_settings:build_python_zip")), + DEBUGGER = str(Label("//python/config_settings:debugger")), EXEC_TOOLS_TOOLCHAIN = str(Label("//python/config_settings:exec_tools_toolchain")), PIP_ENV_MARKER_CONFIG = str(Label("//python/config_settings:pip_env_marker_config")), NONE = str(Label("//python:none")), diff --git a/python/private/py_executable.bzl b/python/private/py_executable.bzl index 2e167b99ab..ea00eed17b 100644 --- a/python/private/py_executable.bzl +++ b/python/private/py_executable.bzl @@ -205,6 +205,10 @@ accepting arbitrary Python versions. allow_single_file = True, default = "@bazel_tools//tools/python:python_bootstrap_template.txt", ), + "_debugger_flag": lambda: attrb.Label( + default = "//python/private:debugger_if_target_config", + providers = [PyInfo], + ), "_launcher": lambda: attrb.Label( cfg = "target", # NOTE: This is an executable, but is only used for Windows. It @@ -267,17 +271,12 @@ def create_binary_semantics(): return create_binary_semantics_struct( # keep-sorted start get_central_uncachable_version_file = lambda ctx: None, - get_debugger_deps = _get_debugger_deps, get_native_deps_dso_name = _get_native_deps_dso_name, should_build_native_deps_dso = lambda ctx: False, should_include_build_data = lambda ctx: False, # keep-sorted end ) -def _get_debugger_deps(ctx, runtime_details): - _ = ctx, runtime_details # @unused - return [] - def _should_create_init_files(ctx): if ctx.attr.legacy_create_init == -1: return not read_possibly_native_flag(ctx, "default_to_explicit_init_py") @@ -1025,7 +1024,7 @@ def py_executable_base_impl(ctx, *, semantics, is_test, inherited_environment = # The debugger dependency should be prevented by select() config elsewhere, # but just to be safe, also guard against adding it to the output here. if not _is_tool_config(ctx): - extra_deps.extend(semantics.get_debugger_deps(ctx, runtime_details)) + extra_deps.append(ctx.attr._debugger_flag) cc_details = _get_cc_details_for_binary(ctx, extra_deps = extra_deps) native_deps_details = _get_native_deps_details( @@ -1751,6 +1750,8 @@ def _create_run_environment_info(ctx, inherited_environment): expression = value, targets = ctx.attr.data, ) + if "PYTHONBREAKPOINT" not in inherited_environment: + inherited_environment = inherited_environment + ["PYTHONBREAKPOINT"] return RunEnvironmentInfo( environment = expanded_env, inherited_environment = inherited_environment, diff --git a/python/private/transition_labels.bzl b/python/private/transition_labels.bzl index b2cf6d7d88..04fcecb5ec 100644 --- a/python/private/transition_labels.bzl +++ b/python/private/transition_labels.bzl @@ -10,6 +10,7 @@ load(":common_labels.bzl", "labels") _BASE_TRANSITION_LABELS = [ labels.ADD_SRCS_TO_RUNFILES, labels.BOOTSTRAP_IMPL, + labels.DEBUGGER, labels.EXEC_TOOLS_TOOLCHAIN, labels.PIP_ENV_MARKER_CONFIG, labels.PIP_WHL_MUSLC_VERSION, diff --git a/python/py_binary.bzl b/python/py_binary.bzl index 80d371fd4c..d02c3e105b 100644 --- a/python/py_binary.bzl +++ b/python/py_binary.bzl @@ -28,6 +28,11 @@ def py_binary(**attrs): * `srcs_version`: cannot be `PY2` or `PY2ONLY` * `tags`: May have special marker values added, if not already present. + :::{versionchanged} VERSION_NEXT_FEATURE + The `PYTHONBREAKPOINT` environment variable is inherited. Use in combination + with {obj}`--debugger` to customize the debugger available and used. + ::: + Args: **attrs: Rule attributes forwarded onto the underlying {rule}`py_binary`. """ diff --git a/tests/base_rules/py_executable_base_tests.bzl b/tests/base_rules/py_executable_base_tests.bzl index ed1a55021d..58251c60a0 100644 --- a/tests/base_rules/py_executable_base_tests.bzl +++ b/tests/base_rules/py_executable_base_tests.bzl @@ -19,6 +19,7 @@ load("@rules_testing//lib:analysis_test.bzl", "analysis_test") load("@rules_testing//lib:truth.bzl", "matching") load("@rules_testing//lib:util.bzl", rt_util = "util") load("//python:py_executable_info.bzl", "PyExecutableInfo") +load("//python:py_library.bzl", "py_library") load("//python/private:common_labels.bzl", "labels") # buildifier: disable=bzl-visibility load("//python/private:reexports.bzl", "BuiltinPyRuntimeInfo") # buildifier: disable=bzl-visibility load("//tests/base_rules:base_tests.bzl", "create_base_tests") @@ -170,6 +171,44 @@ def _test_executable_in_runfiles_impl(env, target): "{workspace}/{package}/{test_name}_subject", ]) +def _test_debugger(name, config): + rt_util.helper_target( + py_library, + name = name + "_debugger", + srcs = [rt_util.empty_file(name + "_debugger.py")], + ) + + rt_util.helper_target( + config.rule, + name = name + "_subject", + srcs = [rt_util.empty_file(name + "_subject.py")], + config_settings = { + # config_settings requires a fully qualified label + labels.DEBUGGER: "//{}:{}_debugger".format(native.package_name(), name), + }, + ) + analysis_test( + name = name, + impl = _test_debugger_impl, + targets = { + "exec_target": name + "_subject", + "target": name + "_subject", + }, + attrs = { + "exec_target": attr.label(cfg = "exec"), + }, + ) + +_tests.append(_test_debugger) + +def _test_debugger_impl(env, targets): + env.expect.that_target(targets.target).runfiles().contains_at_least([ + "{workspace}/{package}/{test_name}_debugger.py", + ]) + env.expect.that_target(targets.exec_target).runfiles().not_contains( + "{workspace}/{package}/{test_name}_debugger.py", + ) + def _test_default_main_can_be_generated(name, config): rt_util.helper_target( config.rule, From f7656d16f284c081aaef1c11991c1220b1ea6909 Mon Sep 17 00:00:00 2001 From: Shayan Hoshyari <108962133+shayanhoshyari@users.noreply.github.com> Date: Sat, 27 Dec 2025 14:55:29 -0800 Subject: [PATCH 569/922] fix(--debugger): Ensure that imports or venv_site_package files are propagated for debugger target (#3483) https://github.com/bazel-contrib/rules_python/commit/a94bd0fdde426bf30efed7c819422d74b404cc18 recently added support for injecting dependencies for easier use of debugger. It allows injecting deps via `--@rules_python//python/config_settings:debugger=`. While the runfiles from `` were inherited in the final binary, the `imports` or `venv_site_packages` were missing. Hence making the debugger target unusable for various corner cases (e.g. when it uses `imports = ...` or when it is coming from pip hub and `venv_site_packages` are on). This PR fixes that, and extends the unit test to include this situation. Fixes: https://github.com/bazel-contrib/rules_python/issues/3481 --------- Co-authored-by: Shayan Hoshyari --- python/private/common.bzl | 28 ++++++++--- python/private/py_executable.bzl | 19 ++++--- tests/base_rules/py_executable_base_tests.bzl | 49 +++++++++++++++++++ 3 files changed, 83 insertions(+), 13 deletions(-) diff --git a/python/private/common.bzl b/python/private/common.bzl index a593e97558..c31aeb383f 100644 --- a/python/private/common.bzl +++ b/python/private/common.bzl @@ -161,12 +161,8 @@ def collect_cc_info(ctx, extra_deps = []): Returns: CcInfo provider of merged information. """ - deps = ctx.attr.deps - if extra_deps: - deps = list(deps) - deps.extend(extra_deps) cc_infos = [] - for dep in deps: + for dep in collect_deps(ctx, extra_deps): if CcInfo in dep: cc_infos.append(dep[CcInfo]) @@ -175,17 +171,19 @@ def collect_cc_info(ctx, extra_deps = []): return cc_common.merge_cc_infos(cc_infos = cc_infos) -def collect_imports(ctx): +def collect_imports(ctx, extra_deps = []): """Collect the direct and transitive `imports` strings. Args: ctx: {type}`ctx` the current target ctx + extra_deps: list of Target to also collect imports from. Returns: {type}`depset[str]` of import paths """ + transitive = [] - for dep in ctx.attr.deps: + for dep in collect_deps(ctx, extra_deps): if PyInfo in dep: transitive.append(dep[PyInfo].imports) if BuiltinPyInfo != None and BuiltinPyInfo in dep: @@ -479,3 +477,19 @@ def runfiles_root_path(ctx, short_path): return short_path[3:] else: return "{}/{}".format(ctx.workspace_name, short_path) + +def collect_deps(ctx, extra_deps = []): + """Collect the dependencies from the rule's context. + + Args: + ctx: rule ctx + extra_deps: list of Target to also collect dependencies from. + + Returns: + list of Target + """ + deps = ctx.attr.deps + if extra_deps: + deps = list(deps) + deps.extend(extra_deps) + return deps diff --git a/python/private/py_executable.bzl b/python/private/py_executable.bzl index ea00eed17b..f9c91225b4 100644 --- a/python/private/py_executable.bzl +++ b/python/private/py_executable.bzl @@ -37,6 +37,7 @@ load(":cc_helper.bzl", "cc_helper") load( ":common.bzl", "collect_cc_info", + "collect_deps", "collect_imports", "collect_runfiles", "create_binary_semantics_struct", @@ -293,7 +294,8 @@ def _create_executable( runtime_details, cc_details, native_deps_details, - runfiles_details): + runfiles_details, + extra_deps): _ = is_test, cc_details, native_deps_details # @unused is_windows = target_platform_has_any_constraint(ctx, ctx.attr._windows_constraints) @@ -323,6 +325,7 @@ def _create_executable( add_runfiles_root_to_sys_path = ( "1" if BootstrapImplFlag.get_value(ctx) == BootstrapImplFlag.SYSTEM_PYTHON else "0" ), + extra_deps = extra_deps, ) stage2_bootstrap = _create_stage2_bootstrap( @@ -486,7 +489,7 @@ def _create_zip_main(ctx, *, stage2_bootstrap, runtime_details, venv): # * https://snarky.ca/how-virtual-environments-work/ # * https://github.com/python/cpython/blob/main/Modules/getpath.py # * https://github.com/python/cpython/blob/main/Lib/site.py -def _create_venv(ctx, output_prefix, imports, runtime_details, add_runfiles_root_to_sys_path): +def _create_venv(ctx, output_prefix, imports, runtime_details, add_runfiles_root_to_sys_path, extra_deps): create_full_venv = BootstrapImplFlag.get_value(ctx) == BootstrapImplFlag.SCRIPT venv = "_{}.venv".format(output_prefix.lstrip("_")) @@ -587,7 +590,11 @@ def _create_venv(ctx, output_prefix, imports, runtime_details, add_runfiles_root VenvSymlinkKind.BIN: bin_dir, VenvSymlinkKind.LIB: site_packages, } - venv_app_files = create_venv_app_files(ctx, ctx.attr.deps, venv_dir_map) + venv_app_files = create_venv_app_files( + ctx, + deps = collect_deps(ctx, extra_deps), + venv_dir_map = venv_dir_map, + ) files_without_interpreter = [pth, site_init] + venv_app_files.venv_files if pyvenv_cfg: @@ -1016,9 +1023,6 @@ def py_executable_base_impl(ctx, *, semantics, is_test, inherited_environment = default_outputs.add(precompile_result.keep_srcs) default_outputs.add(required_pyc_files) - imports = collect_imports(ctx) - - runtime_details = _get_runtime_details(ctx) extra_deps = [] # The debugger dependency should be prevented by select() config elsewhere, @@ -1026,6 +1030,8 @@ def py_executable_base_impl(ctx, *, semantics, is_test, inherited_environment = if not _is_tool_config(ctx): extra_deps.append(ctx.attr._debugger_flag) + imports = collect_imports(ctx, extra_deps = extra_deps) + runtime_details = _get_runtime_details(ctx) cc_details = _get_cc_details_for_binary(ctx, extra_deps = extra_deps) native_deps_details = _get_native_deps_details( ctx, @@ -1058,6 +1064,7 @@ def py_executable_base_impl(ctx, *, semantics, is_test, inherited_environment = cc_details = cc_details, native_deps_details = native_deps_details, runfiles_details = runfiles_details, + extra_deps = extra_deps, ) default_outputs.add(exec_result.extra_files_to_build) diff --git a/tests/base_rules/py_executable_base_tests.bzl b/tests/base_rules/py_executable_base_tests.bzl index 58251c60a0..2af5406ced 100644 --- a/tests/base_rules/py_executable_base_tests.bzl +++ b/tests/base_rules/py_executable_base_tests.bzl @@ -19,6 +19,7 @@ load("@rules_testing//lib:analysis_test.bzl", "analysis_test") load("@rules_testing//lib:truth.bzl", "matching") load("@rules_testing//lib:util.bzl", rt_util = "util") load("//python:py_executable_info.bzl", "PyExecutableInfo") +load("//python:py_info.bzl", "PyInfo") load("//python:py_library.bzl", "py_library") load("//python/private:common_labels.bzl", "labels") # buildifier: disable=bzl-visibility load("//python/private:reexports.bzl", "BuiltinPyRuntimeInfo") # buildifier: disable=bzl-visibility @@ -172,9 +173,11 @@ def _test_executable_in_runfiles_impl(env, target): ]) def _test_debugger(name, config): + # Using imports rt_util.helper_target( py_library, name = name + "_debugger", + imports = ["."], srcs = [rt_util.empty_file(name + "_debugger.py")], ) @@ -187,24 +190,70 @@ def _test_debugger(name, config): labels.DEBUGGER: "//{}:{}_debugger".format(native.package_name(), name), }, ) + + # Using venv + rt_util.helper_target( + py_library, + name = name + "_debugger_venv", + imports = [native.package_name() + "/site-packages"], + experimental_venvs_site_packages = "@rules_python//python/config_settings:venvs_site_packages", + srcs = [rt_util.empty_file("site-packages/" + name + "_debugger_venv.py")], + ) + + rt_util.helper_target( + config.rule, + name = name + "_subject_venv", + srcs = [rt_util.empty_file(name + "_subject_venv.py")], + config_settings = { + # config_settings requires a fully qualified label + labels.DEBUGGER: "//{}:{}_debugger_venv".format(native.package_name(), name), + }, + ) + analysis_test( name = name, impl = _test_debugger_impl, targets = { "exec_target": name + "_subject", "target": name + "_subject", + "target_venv": name + "_subject_venv", }, attrs = { "exec_target": attr.label(cfg = "exec"), }, + config_settings = { + labels.VENVS_SITE_PACKAGES: "yes", + labels.PYTHON_VERSION: "3.13", + }, ) _tests.append(_test_debugger) def _test_debugger_impl(env, targets): + # 1. Subject + + # Check the file from debugger dep is injected. env.expect.that_target(targets.target).runfiles().contains_at_least([ "{workspace}/{package}/{test_name}_debugger.py", ]) + + # #3481: Ensure imports are setup correcty. + meta = env.expect.meta.derive(format_str_kwargs = {"package": targets.target.label.package}) + env.expect.that_target(targets.target).has_provider(PyInfo) + imports = targets.target[PyInfo].imports.to_list() + env.expect.that_collection(imports).contains(meta.format_str("{workspace}/{package}")) + + # 2. Subject venv + + # #3481: Ensure that venv site-packages is setup correctly, if the dependency is coming + # from pip integration. + env.expect.that_target(targets.target_venv).runfiles().contains_at_least([ + "{workspace}/{package}/_{name}.venv/lib/python3.13/site-packages/{test_name}_debugger_venv.py", + ]) + + # 3. Subject exec + + # Ensure that tools don't inherit debugger. env.expect.that_target(targets.exec_target).runfiles().not_contains( "{workspace}/{package}/{test_name}_debugger.py", ) From a1ca5d4f47ffafa662b7f518d362c86d78ab6468 Mon Sep 17 00:00:00 2001 From: Markus Hofbauer Date: Sun, 28 Dec 2025 02:19:54 +0100 Subject: [PATCH 570/922] build: Export runtime_env_toolchain_interpreter.sh file (#3471) I am working on https://github.com/bazelbuild/bazel/pull/27674 towards https://github.com/bazelbuild/bazel/issues/10225 to flip `incompatible_no_implicit_file_export`. I found that https://github.com/bazelbuild/bazel/blob/master/src/test/py/bazel/bzlmod/bazel_repo_mapping_test.py and https://github.com/bazelbuild/bazel/blob/master/src/test/py/bazel/bzlmod/bazel_module_test.py require `runtime_env_toolchain_interpreter.sh`. Since it is no longer implicitly exported after flipping the flag, we have to export it explicitly. --- python/private/BUILD.bazel | 2 ++ 1 file changed, 2 insertions(+) diff --git a/python/private/BUILD.bazel b/python/private/BUILD.bazel index 13cbfafade..708b3f7000 100644 --- a/python/private/BUILD.bazel +++ b/python/private/BUILD.bazel @@ -28,6 +28,8 @@ package( licenses(["notice"]) +exports_files(["runtime_env_toolchain_interpreter.sh"]) + filegroup( name = "distribution", srcs = glob(["**"]) + [ From 2de083f68b74c5fb6b2aedbeea911a70fa2093a5 Mon Sep 17 00:00:00 2001 From: Philipp Stephani Date: Sun, 28 Dec 2025 05:24:51 +0100 Subject: [PATCH 571/922] fix(coverage): Disable certain coverage warnings. (#3191) These warnings appear if there are no Python source files in the instrumented directories, cf. https://github.com/bazel-contrib/rules_python/issues/2762. Work towards #2762 --------- Co-authored-by: Ignas Anikevicius <240938+aignas@users.noreply.github.com> --- CHANGELOG.md | 3 ++- python/private/py_executable.bzl | 1 + python/private/stage2_bootstrap_template.py | 6 ++++++ 3 files changed, 9 insertions(+), 1 deletion(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index e92b3c2737..6b858c7d89 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -62,7 +62,8 @@ END_UNRELEASED_TEMPLATE {#v0-0-0-fixed} ### Fixed -* Nothing fixed. +* (tests) No more coverage warnings are being printed if there are no sources. + ([#2762](https://github.com/bazel-contrib/rules_python/issues/2762)) {#v0-0-0-added} ### Added diff --git a/python/private/py_executable.bzl b/python/private/py_executable.bzl index f9c91225b4..73d0c8399f 100644 --- a/python/private/py_executable.bzl +++ b/python/private/py_executable.bzl @@ -668,6 +668,7 @@ def _create_stage2_bootstrap( template = template, output = output, substitutions = { + "%coverage_instrumented%": str(int(ctx.configuration.coverage_enabled and ctx.coverage_instrumented())), "%coverage_tool%": _get_coverage_tool_runfiles_path(ctx, runtime), "%import_all%": "True" if read_possibly_native_flag(ctx, "python_import_all_repositories") else "False", "%imports%": ":".join(imports.to_list()), diff --git a/python/private/stage2_bootstrap_template.py b/python/private/stage2_bootstrap_template.py index 4d98b03846..e3e303b3b1 100644 --- a/python/private/stage2_bootstrap_template.py +++ b/python/private/stage2_bootstrap_template.py @@ -41,6 +41,9 @@ # string otherwise. VENV_SITE_PACKAGES = "%venv_rel_site_packages%" +# Whether we should generate coverage data. +COVERAGE_INSTRUMENTED = "%coverage_instrumented%" == "1" + # ===== Template substitutions end ===== @@ -319,11 +322,14 @@ def _maybe_collect_coverage(enable): # We need for coveragepy to use relative paths. This can only be configured # using an rc file. rcfile_name = os.path.join(coverage_dir, ".coveragerc_{}".format(unique_id)) + disable_warnings = ('disable_warnings = module-not-imported, no-data-collected' + if COVERAGE_INSTRUMENTED else '') print_verbose_coverage("coveragerc file:", rcfile_name) with open(rcfile_name, "w") as rcfile: rcfile.write( f"""[run] relative_files = True +{disable_warnings} source = \t{source} """ From 352f405f6a834da32449838bf04f5ff68a1f86af Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Sun, 28 Dec 2025 23:40:39 -0800 Subject: [PATCH 572/922] feat: basic build data with stamping (#3484) This makes build data accessible to binaries. By default, some basic information is always available: target name and if stamping was enabled. When `--stamp` is enabled, the full `workspace_status_command` output is included in the build information. Build information is made available through a special `bazel_binary_info` module. This module is created by the bootstrap and injected into sys.modules. It provides an API for gettting the raw build information that was written. Exposing a module, instead of a file, is done insulate from the Bazel-ism of runfiles. When stamping is enabled, the way stamped data works is a bit round about to get the desired semantics: that the build data reflects the most recent build of a binary, not the last cached value of the build information (e.g., if I build a binary, then modify a file in the binary, then build again, the build data should reflect the timestamp of the second build). Normal Bazel caching, however, makes that somewhat difficult (the build data file has to take the full transitive set of files as inputs to properly detect changes, otherwise it'll get cached on first build of the binary; plus actions can't take runfiles objects as inputs). To work around this, we use some special APIs to get `ctx.version_file` information into the output without entirely destroying caching. * Because `ctx.version_file` doesn't trigger cache invalidation, a special helper, `py_internal.copy_without_caching()`, is used to make it _always_ invalidate caching. Now we can ensure we have a copy of the most recent invocation data. * To prevent the above from _always_ rebuilding a binary, another special api, `py_internal.declare_constant_metadata_file()` is used to restore the behavior of a file never invalidating caching (even if it changes). This dance is necessary because actions can't take runfiles as direct inputs. It also avoids having to pass the entire program's transitive set of files as an input just to do input change tracking. Note the above only applies when stamping is enabled. Unstamped binaries don't do any of this. Along the way... * The `stamp` attribute now transitions the `--stamp` flag. This was necessary so that the dependency on version_info was conditional on stamping. * Remove a defunct helper from the stage2 bootstrap code. --- CHANGELOG.md | 5 + python/private/BUILD.bazel | 13 ++ python/private/attributes.bzl | 14 ++- python/private/build_data_writer.ps1 | 18 +++ python/private/build_data_writer.sh | 12 ++ python/private/common.bzl | 11 +- python/private/py_executable.bzl | 128 +++++++++++--------- python/private/stage2_bootstrap_template.py | 43 +++++-- python/private/uncachable_version_file.bzl | 39 ++++++ sphinxdocs/inventories/bazel_inventory.txt | 2 + tests/build_data/BUILD.bazel | 25 ++++ tests/build_data/build_data_test.py | 32 +++++ tests/build_data/print_build_data.py | 3 + 13 files changed, 261 insertions(+), 84 deletions(-) create mode 100644 python/private/build_data_writer.ps1 create mode 100755 python/private/build_data_writer.sh create mode 100644 python/private/uncachable_version_file.bzl create mode 100644 tests/build_data/BUILD.bazel create mode 100644 tests/build_data/build_data_test.py create mode 100644 tests/build_data/print_build_data.py diff --git a/CHANGELOG.md b/CHANGELOG.md index 6b858c7d89..40d1affe0f 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -59,6 +59,8 @@ END_UNRELEASED_TEMPLATE {#v0-0-0-changed} ### Changed * (binaries/tests) The `PYTHONBREAKPOINT` environment variable is automatically inherited +* (binaries/tests) The {obj}`stamp` attribute now transitions the Bazel builtin + {obj}`--stamp` flag. {#v0-0-0-fixed} ### Fixed @@ -69,6 +71,9 @@ END_UNRELEASED_TEMPLATE ### Added * (binaries/tests) {obj}`--debugger`: allows specifying an extra dependency to add to binaries/tests for custom debuggers. +* (binaries/tests) Build information is now included in binaries and tests. + Use the `bazel_binary_info` module to access it. The {flag}`--stamp` flag will + add {flag}`--workspace_status` information. {#v1-8-0} ## [1.8.0] - 2025-12-19 diff --git a/python/private/BUILD.bazel b/python/private/BUILD.bazel index 708b3f7000..efc1dd7319 100644 --- a/python/private/BUILD.bazel +++ b/python/private/BUILD.bazel @@ -21,6 +21,7 @@ load(":print_toolchain_checksums.bzl", "print_toolchains_checksums") load(":py_exec_tools_toolchain.bzl", "current_interpreter_executable") load(":sentinel.bzl", "sentinel") load(":stamp.bzl", "stamp_build_setting") +load(":uncachable_version_file.bzl", "define_uncachable_version_file") package( default_visibility = ["//:__subpackages__"], @@ -99,6 +100,14 @@ bzl_library( ], ) +alias( + name = "build_data_writer", + actual = select({ + "@platforms//os:windows": ":build_data_writer.ps1", + "//conditions:default": ":build_data_writer.sh", + }), +) + bzl_library( name = "builders_bzl", srcs = ["builders.bzl"], @@ -683,6 +692,10 @@ bzl_library( ], ) +define_uncachable_version_file( + name = "uncachable_version_file", +) + bzl_library( name = "version_bzl", srcs = ["version.bzl"], diff --git a/python/private/attributes.bzl b/python/private/attributes.bzl index 0e0872fbf5..4687693cb7 100644 --- a/python/private/attributes.bzl +++ b/python/private/attributes.bzl @@ -450,8 +450,20 @@ Whether to encode build information into the binary. Possible values: Stamped binaries are not rebuilt unless their dependencies change. -WARNING: Stamping can harm build performance by reducing cache hits and should +Stamped build information can accessed using the `bazel_binary_info` module. +See the [Accessing build information docs] for more information. + +:::{warning} +Stamping can harm build performance by reducing cache hits and should be avoided if possible. + +In addition, this transitions the {obj}`--stamp` flag, which can additional +config state overhead. +::: + +:::{note} +Stamping of build data output is always disabled for the exec config. +::: """, default = -1, ), diff --git a/python/private/build_data_writer.ps1 b/python/private/build_data_writer.ps1 new file mode 100644 index 0000000000..384d1ce539 --- /dev/null +++ b/python/private/build_data_writer.ps1 @@ -0,0 +1,18 @@ +$OutputPath = $env:OUTPUT + +Add-Content -Path $OutputPath -Value "TARGET $env:TARGET" +Add-Content -Path $OutputPath -Value "CONFIG_ID $env:CONFIG_ID" +Add-Content -Path $OutputPath -Value "CONFIG_MODE $env:CONFIG_MODE" +Add-Content -Path $OutputPath -Value "STAMPED $env:STAMPED" + +$VersionFilePath = $env:VERSION_FILE +if (-not [string]::IsNullOrEmpty($VersionFilePath)) { + Get-Content -Path $VersionFilePath | Add-Content -Path $OutputPath +} + +$InfoFilePath = $env:INFO_FILE +if (-not [string]::IsNullOrEmpty($InfoFilePath)) { + Get-Content -Path $InfoFilePath | Add-Content -Path $OutputPath +} + +exit 0 diff --git a/python/private/build_data_writer.sh b/python/private/build_data_writer.sh new file mode 100755 index 0000000000..7b88a582f5 --- /dev/null +++ b/python/private/build_data_writer.sh @@ -0,0 +1,12 @@ +#!/bin/sh + +echo "TARGET $TARGET" >> $OUTPUT +echo "CONFIG_MODE $CONFIG_MODE" >> $OUTPUT +echo "STAMPED $STAMPED" >> $OUTPUT +if [ -n "$VERSION_FILE" ]; then + cat "$VERSION_FILE" >> "$OUTPUT" +fi +if [ -n "$INFO_FILE" ]; then + cat "$INFO_FILE" >> "$OUTPUT" +fi +exit 0 diff --git a/python/private/common.bzl b/python/private/common.bzl index c31aeb383f..2d4afca3f5 100644 --- a/python/private/common.bzl +++ b/python/private/common.bzl @@ -43,34 +43,25 @@ PYTHON_FILE_EXTENSIONS = [ def create_binary_semantics_struct( *, - get_central_uncachable_version_file, get_native_deps_dso_name, - should_build_native_deps_dso, - should_include_build_data): + should_build_native_deps_dso): """Helper to ensure a semantics struct has all necessary fields. Call this instead of a raw call to `struct(...)`; it'll help ensure all the necessary functions are being correctly provided. Args: - get_central_uncachable_version_file: Callable that returns an optional - Artifact; this artifact is special: it is never cached and is a copy - of `ctx.version_file`; see py_builtins.copy_without_caching get_native_deps_dso_name: Callable that returns a string, which is the basename (with extension) of the native deps DSO library. should_build_native_deps_dso: Callable that returns bool; True if building a native deps DSO is supported, False if not. - should_include_build_data: Callable that returns bool; True if - build data should be generated, False if not. Returns: A "BinarySemantics" struct. """ return struct( # keep-sorted - get_central_uncachable_version_file = get_central_uncachable_version_file, get_native_deps_dso_name = get_native_deps_dso_name, should_build_native_deps_dso = should_build_native_deps_dso, - should_include_build_data = should_include_build_data, ) def create_cc_details_struct( diff --git a/python/private/py_executable.bzl b/python/private/py_executable.bzl index 73d0c8399f..1b884e9b3b 100644 --- a/python/private/py_executable.bzl +++ b/python/private/py_executable.bzl @@ -206,6 +206,11 @@ accepting arbitrary Python versions. allow_single_file = True, default = "@bazel_tools//tools/python:python_bootstrap_template.txt", ), + "_build_data_writer": lambda: attrb.Label( + default = "//python/private:build_data_writer", + allow_files = True, + cfg = "exec", + ), "_debugger_flag": lambda: attrb.Label( default = "//python/private:debugger_if_target_config", providers = [PyInfo], @@ -226,6 +231,10 @@ accepting arbitrary Python versions. "_python_version_flag": lambda: attrb.Label( default = labels.PYTHON_VERSION, ), + "_uncachable_version_file": lambda: attrb.Label( + default = "//python/private:uncachable_version_file", + allow_files = True, + ), "_venvs_use_declare_symlink_flag": lambda: attrb.Label( default = labels.VENVS_USE_DECLARE_SYMLINK, providers = [BuildSettingInfo], @@ -271,10 +280,8 @@ def py_executable_impl(ctx, *, is_test, inherited_environment): def create_binary_semantics(): return create_binary_semantics_struct( # keep-sorted start - get_central_uncachable_version_file = lambda ctx: None, get_native_deps_dso_name = _get_native_deps_dso_name, should_build_native_deps_dso = lambda ctx: False, - should_include_build_data = lambda ctx: False, # keep-sorted end ) @@ -336,6 +343,7 @@ def _create_executable( imports = imports, runtime_details = runtime_details, venv = venv, + build_data_file = runfiles_details.build_data_file, ) extra_runfiles = ctx.runfiles( [stage2_bootstrap] + ( @@ -648,6 +656,7 @@ def _create_stage2_bootstrap( main_py, imports, runtime_details, + build_data_file, venv): output = ctx.actions.declare_file( # Prepend with underscore to prevent pytest from trying to @@ -668,6 +677,7 @@ def _create_stage2_bootstrap( template = template, output = output, substitutions = { + "%build_data_file%": runfiles_root_path(ctx, build_data_file.short_path), "%coverage_instrumented%": str(int(ctx.configuration.coverage_enabled and ctx.coverage_instrumented())), "%coverage_tool%": _get_coverage_tool_runfiles_path(ctx, runtime), "%import_all%": "True" if read_possibly_native_flag(ctx, "python_import_all_repositories") else "False", @@ -1053,7 +1063,6 @@ def py_executable_base_impl(ctx, *, semantics, is_test, inherited_environment = cc_details.extra_runfiles, native_deps_details.runfiles, ], - semantics = semantics, ) exec_result = _create_executable( ctx, @@ -1242,8 +1251,7 @@ def _get_base_runfiles_for_binary( required_pyc_files, implicit_pyc_files, implicit_pyc_source_files, - extra_common_runfiles, - semantics): + extra_common_runfiles): """Returns the set of runfiles necessary prior to executable creation. NOTE: The term "common runfiles" refers to the runfiles that are common to @@ -1265,7 +1273,6 @@ def _get_base_runfiles_for_binary( files that are used when the implicit pyc files are not. extra_common_runfiles: List of runfiles; additional runfiles that will be added to the common runfiles. - semantics: A `BinarySemantics` struct; see `create_binary_semantics_struct`. Returns: struct with attributes: @@ -1306,6 +1313,9 @@ def _get_base_runfiles_for_binary( common_runfiles.add_targets(extra_deps) common_runfiles.add(extra_common_runfiles) + build_data_file = _write_build_data(ctx) + common_runfiles.add(build_data_file) + common_runfiles = common_runfiles.build(ctx) if _should_create_init_files(ctx): @@ -1314,25 +1324,10 @@ def _get_base_runfiles_for_binary( runfiles = common_runfiles, ) - # Don't include build_data.txt in the non-exe runfiles. The build data - # may contain program-specific content (e.g. target name). runfiles_with_exe = common_runfiles.merge(ctx.runfiles([executable])) - # Don't include build_data.txt in data runfiles. This allows binaries to - # contain other binaries while still using the same fixed location symlink - # for the build_data.txt file. Really, the fixed location symlink should be - # removed and another way found to locate the underlying build data file. data_runfiles = runfiles_with_exe - - if is_stamping_enabled(ctx) and semantics.should_include_build_data(ctx): - build_data_file, build_data_runfiles = _create_runfiles_with_build_data( - ctx, - semantics.get_central_uncachable_version_file(ctx), - ) - default_runfiles = runfiles_with_exe.merge(build_data_runfiles) - else: - build_data_file = None - default_runfiles = runfiles_with_exe + default_runfiles = runfiles_with_exe return struct( runfiles_without_exe = common_runfiles, @@ -1341,31 +1336,18 @@ def _get_base_runfiles_for_binary( data_runfiles = data_runfiles, ) -def _create_runfiles_with_build_data( - ctx, - central_uncachable_version_file): - build_data_file = _write_build_data( - ctx, - central_uncachable_version_file, - ) - build_data_runfiles = ctx.runfiles(files = [ - build_data_file, - ]) - return build_data_file, build_data_runfiles - -def _write_build_data(ctx, central_uncachable_version_file): - # TODO: Remove this logic when a central file is always available - if not central_uncachable_version_file: - version_file = ctx.actions.declare_file(ctx.label.name + "-uncachable_version_file.txt") - _py_builtins.copy_without_caching( - ctx = ctx, - read_from = ctx.version_file, - write_to = version_file, - ) +def _write_build_data(ctx): + inputs = builders.DepsetBuilder() + if is_stamping_enabled(ctx): + # NOTE: ctx.info_file is undocumented; see + # https://github.com/bazelbuild/bazel/issues/9363 + info_file = ctx.info_file + version_file = ctx.files._uncachable_version_file[0] + inputs.add(info_file) + inputs.add(version_file) else: - version_file = central_uncachable_version_file - - direct_inputs = [ctx.info_file, version_file] + info_file = None + version_file = None # A "constant metadata" file is basically a special file that doesn't # support change detection logic and reports that it is unchanged. i.e., it @@ -1397,23 +1379,36 @@ def _write_build_data(ctx, central_uncachable_version_file): root = ctx.bin_dir, ) + action_args = ctx.actions.args() + writer_file = ctx.files._build_data_writer[0] + if writer_file.path.endswith(".ps1"): + action_exe = "pwsh.exe" + action_args.add("-File") + action_args.add(writer_file) + inputs.add(writer_file) + else: + action_exe = ctx.attr._build_data_writer[DefaultInfo].files_to_run + ctx.actions.run( - executable = ctx.executable._build_data_gen, + executable = action_exe, + arguments = [action_args], env = { - # NOTE: ctx.info_file is undocumented; see - # https://github.com/bazelbuild/bazel/issues/9363 - "INFO_FILE": ctx.info_file.path, + # Include config mode so that binaries can detect if they're + # being used as a build tool or not, allowing for runtime optimizations. + "CONFIG_MODE": "EXEC" if _is_tool_config(ctx) else "TARGET", + "INFO_FILE": info_file.path if info_file else "", "OUTPUT": build_data.path, - "PLATFORM": cc_helper.find_cpp_toolchain(ctx).toolchain_id, + # Include this so it's explicit, otherwise, one has to detect + # this by looking for the absense of info_file keys. + "STAMPED": "TRUE" if is_stamping_enabled(ctx) else "FALSE", "TARGET": str(ctx.label), - "VERSION_FILE": version_file.path, + "VERSION_FILE": version_file.path if version_file else "", }, - inputs = depset( - direct = direct_inputs, - ), + inputs = inputs.build(), outputs = [build_data], mnemonic = "PyWriteBuildData", - progress_message = "Generating %{label} build_data.txt", + progress_message = "Reticulating %{label} build data", + toolchain = None, ) return build_data @@ -1608,6 +1603,9 @@ def is_stamping_enabled(ctx): Returns: bool; True if stamping is enabled, False if not. """ + + # Always ignore stamping for exec config. This mitigates stamping + # invalidating build action caching. if _is_tool_config(ctx): return False @@ -1617,8 +1615,9 @@ def is_stamping_enabled(ctx): elif stamp == 0: return False elif stamp == -1: - # NOTE: Undocumented API; private to builtins - return ctx.configuration.stamp_binaries + # NOTE: ctx.configuration.stamp_binaries() exposes this, but that's + # a private API. To workaround, it'd been eposed via py_internal. + return py_internal.stamp_binaries(ctx) else: fail("Unsupported `stamp` value: {}".format(stamp)) @@ -1771,6 +1770,9 @@ def _transition_executable_impl(settings, attr): if attr.python_version and attr.python_version not in ("PY2", "PY3"): settings[labels.PYTHON_VERSION] = attr.python_version + + if attr.stamp != -1: + settings["//command_line_option:stamp"] = str(attr.stamp) return settings def create_executable_rule(*, attrs, **kwargs): @@ -1821,8 +1823,14 @@ def create_executable_rule_builder(implementation, **kwargs): ] + ([ruleb.ToolchainType(_LAUNCHER_MAKER_TOOLCHAIN_TYPE)] if rp_config.bazel_9_or_later else []), cfg = dict( implementation = _transition_executable_impl, - inputs = TRANSITION_LABELS + [labels.PYTHON_VERSION], - outputs = TRANSITION_LABELS + [labels.PYTHON_VERSION], + inputs = TRANSITION_LABELS + [ + labels.PYTHON_VERSION, + "//command_line_option:stamp", + ], + outputs = TRANSITION_LABELS + [ + labels.PYTHON_VERSION, + "//command_line_option:stamp", + ], ), **kwargs ) diff --git a/python/private/stage2_bootstrap_template.py b/python/private/stage2_bootstrap_template.py index e3e303b3b1..3595a43110 100644 --- a/python/private/stage2_bootstrap_template.py +++ b/python/private/stage2_bootstrap_template.py @@ -20,7 +20,9 @@ import os import re import runpy +import types import uuid +from functools import cache # ===== Template substitutions start ===== # We just put them in one place so its easy to tell which are used. @@ -42,11 +44,34 @@ VENV_SITE_PACKAGES = "%venv_rel_site_packages%" # Whether we should generate coverage data. +# string, 1 or 0 COVERAGE_INSTRUMENTED = "%coverage_instrumented%" == "1" +# runfiles-root-relative path to a file with binary-specific build information +BUILD_DATA_FILE = "%build_data_file%" + # ===== Template substitutions end ===== +class BazelBinaryInfoModule(types.ModuleType): + BUILD_DATA_FILE = BUILD_DATA_FILE + + @cache + def get_build_data(self): + """Returns a string of the raw build data.""" + try: + # Prefer dep via pypi + import runfiles + except ImportError: + from python.runfiles import runfiles + path = runfiles.Create().Rlocation(self.BUILD_DATA_FILE) + with open(path) as fp: + return fp.read() + + +sys.modules["bazel_binary_info"] = BazelBinaryInfoModule("bazel_binary_info") + + # Return True if running on Windows def is_windows(): return os.name == "nt" @@ -89,17 +114,6 @@ def get_windows_path_with_unc_prefix(path): return unicode_prefix + os.path.abspath(path) -def search_path(name): - """Finds a file in a given search path.""" - search_path = os.getenv("PATH", os.defpath).split(os.pathsep) - for directory in search_path: - if directory: - path = os.path.join(directory, name) - if os.path.isfile(path) and os.access(path, os.X_OK): - return path - return None - - def is_verbose(): return bool(os.environ.get("RULES_PYTHON_BOOTSTRAP_VERBOSE")) @@ -322,8 +336,11 @@ def _maybe_collect_coverage(enable): # We need for coveragepy to use relative paths. This can only be configured # using an rc file. rcfile_name = os.path.join(coverage_dir, ".coveragerc_{}".format(unique_id)) - disable_warnings = ('disable_warnings = module-not-imported, no-data-collected' - if COVERAGE_INSTRUMENTED else '') + disable_warnings = ( + "disable_warnings = module-not-imported, no-data-collected" + if COVERAGE_INSTRUMENTED + else "" + ) print_verbose_coverage("coveragerc file:", rcfile_name) with open(rcfile_name, "w") as rcfile: rcfile.write( diff --git a/python/private/uncachable_version_file.bzl b/python/private/uncachable_version_file.bzl new file mode 100644 index 0000000000..9b1d65a469 --- /dev/null +++ b/python/private/uncachable_version_file.bzl @@ -0,0 +1,39 @@ +"""Implementation of uncachable_version_file.""" + +load(":py_internal.bzl", "py_internal") + +def _uncachable_version_file_impl(ctx): + version_file = ctx.actions.declare_file("uncachable_version_file.txt") + py_internal.copy_without_caching( + ctx = ctx, + # NOTE: ctx.version_file is undocumented; see + # https://github.com/bazelbuild/bazel/issues/9363 + # NOTE: Even though the version file changes every build (it contains + # the build timestamp), it is ignored when computing what inputs + # changed. See https://bazel.build/docs/user-manual#workspace-status + read_from = ctx.version_file, + write_to = version_file, + ) + return [DefaultInfo( + files = depset([version_file]), + )] + +uncachable_version_file = rule( + doc = """ +Creates a copy of `ctx.version_file`, except it isn't ignored by +Bazel's change-detecting logic. In fact, it's the opposite: +caching is disabled for the action generating this file, so any +actions depending on this file will always re-run. +""", + implementation = _uncachable_version_file_impl, +) + +def define_uncachable_version_file(name): + native.alias( + name = name, + actual = select({ + ":stamp_detect": ":uncachable_version_file_impl", + "//conditions:default": ":sentinel", + }), + ) + uncachable_version_file(name = "uncachable_version_file_impl") diff --git a/sphinxdocs/inventories/bazel_inventory.txt b/sphinxdocs/inventories/bazel_inventory.txt index e14ea76067..e704d20d73 100644 --- a/sphinxdocs/inventories/bazel_inventory.txt +++ b/sphinxdocs/inventories/bazel_inventory.txt @@ -157,6 +157,7 @@ runfiles.merge bzl:type 1 rules/lib/builtins/runfiles#merge - runfiles.merge_all bzl:type 1 rules/lib/builtins/runfiles#merge_all - runfiles.root_symlinks bzl:type 1 rules/lib/builtins/runfiles#root_symlinks - runfiles.symlinks bzl:type 1 rules/lib/builtins/runfiles#symlinks - +stamp bzl:flag 1 reference/command-line-reference#flag--stamp - str bzl:type 1 rules/lib/string - struct bzl:type 1 rules/lib/builtins/struct - target_compatible_with bzl:attr 1 reference/be/common-definitions#common.target_compatible_with - @@ -171,3 +172,4 @@ toolchain.target_settings bzl:attr 1 reference/be/platforms-and-toolchains#toolc toolchain_type bzl:type 1 rules/lib/builtins/toolchain_type.html - transition bzl:type 1 rules/lib/builtins/transition - tuple bzl:type 1 rules/lib/core/tuple - +workspace_status bzl:flag 1 reference/command-line-reference#build-flag--workspace_status_command - diff --git a/tests/build_data/BUILD.bazel b/tests/build_data/BUILD.bazel new file mode 100644 index 0000000000..64db005f51 --- /dev/null +++ b/tests/build_data/BUILD.bazel @@ -0,0 +1,25 @@ +load("//python:py_binary.bzl", "py_binary") +load("//python:py_test.bzl", "py_test") + +py_test( + name = "build_data_test", + srcs = ["build_data_test.py"], + data = [ + ":tool_build_data.txt", + ], + stamp = 1, + deps = ["//python/runfiles"], +) + +py_binary( + name = "print_build_data", + srcs = ["print_build_data.py"], + deps = ["//python/runfiles"], +) + +genrule( + name = "tool_build_data", + outs = ["tool_build_data.txt"], + cmd = "$(location :print_build_data) > $(OUTS)", + tools = [":print_build_data"], +) diff --git a/tests/build_data/build_data_test.py b/tests/build_data/build_data_test.py new file mode 100644 index 0000000000..e4ff81a634 --- /dev/null +++ b/tests/build_data/build_data_test.py @@ -0,0 +1,32 @@ +import unittest + +from python.runfiles import runfiles + + +class BuildDataTest(unittest.TestCase): + + def test_target_build_data(self): + import bazel_binary_info + + self.assertIn("build_data.txt", bazel_binary_info.BUILD_DATA_FILE) + + build_data = bazel_binary_info.get_build_data() + self.assertIn("TARGET ", build_data) + self.assertIn("BUILD_HOST ", build_data) + self.assertIn("BUILD_USER ", build_data) + self.assertIn("BUILD_TIMESTAMP ", build_data) + self.assertIn("FORMATTED_DATE ", build_data) + self.assertIn("CONFIG_MODE TARGET", build_data) + self.assertIn("STAMPED TRUE", build_data) + + def test_tool_build_data(self): + rf = runfiles.Create() + path = rf.Rlocation("rules_python/tests/build_data/tool_build_data.txt") + with open(path) as fp: + build_data = fp.read() + + self.assertIn("STAMPED FALSE", build_data) + self.assertIn("CONFIG_MODE EXEC", build_data) + + +unittest.main() diff --git a/tests/build_data/print_build_data.py b/tests/build_data/print_build_data.py new file mode 100644 index 0000000000..0af77d72be --- /dev/null +++ b/tests/build_data/print_build_data.py @@ -0,0 +1,3 @@ +import bazel_binary_info + +print(bazel_binary_info.get_build_data()) From 89d00af0fc57a6c94f0412b832419f1d879f23a2 Mon Sep 17 00:00:00 2001 From: Laurens Hobert <10121375+laurenshobert@users.noreply.github.com> Date: Sun, 4 Jan 2026 04:18:50 +0100 Subject: [PATCH 573/922] fix: prevent a 404 error when serving Sphinx docs and Bazel is configured with a --symlink_prefix option (#3492) When Bazel is configured with the [`--symlink_prefix`](https://bazel.build/reference/command-line-reference#build-flag--symlink_prefix) option, the sphinxdocs `.serve` target fails to serve files from the correct directory. This happens because the directory layout under `execroot` and the workspace no longer match, causing the target to miss the generated HTML files. This change updates the logic to use rpathlocation instead, ensuring the correct path is resolved regardless of symlink configuration. Fixes https://github.com/bazel-contrib/rules_python/issues/3410 --------- Co-authored-by: Laurens Hobert --- CHANGELOG.md | 2 ++ sphinxdocs/private/sphinx.bzl | 3 ++- sphinxdocs/private/sphinx_server.py | 10 +++++++--- 3 files changed, 11 insertions(+), 4 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 40d1affe0f..0df0de8cee 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -135,6 +135,8 @@ END_UNRELEASED_TEMPLATE * (gazelle) Fix `gazelle_python_manifest.test` so that it accesses manifest files via `runfile` path handling rather than directly ([#3397](https://github.com/bazel-contrib/rules_python/issues/3397)). * (core rules) For the system_python bootstrap, the runfiles root is added to sys.path. +* (sphinxdocs) The sphinxdocs `.serve` target is now compatible with Bazel's `--symlink_prefix` + flag ([#3410](https://github.com/bazel-contrib/rules_python/issues/3410)). {#v1-8-0-added} ### Added diff --git a/sphinxdocs/private/sphinx.bzl b/sphinxdocs/private/sphinx.bzl index e444429233..0efbc269c5 100644 --- a/sphinxdocs/private/sphinx.bzl +++ b/sphinxdocs/private/sphinx.bzl @@ -189,8 +189,9 @@ def sphinx_docs( srcs = [_SPHINX_SERVE_MAIN_SRC], main = _SPHINX_SERVE_MAIN_SRC, data = [html_name], + deps = [Label("//python/runfiles")], args = [ - "$(execpath {})".format(html_name), + "$(rlocationpath {})".format(html_name), ], **common_kwargs_with_manual_tag ) diff --git a/sphinxdocs/private/sphinx_server.py b/sphinxdocs/private/sphinx_server.py index 1f4fae86de..1bd6ee5550 100644 --- a/sphinxdocs/private/sphinx_server.py +++ b/sphinxdocs/private/sphinx_server.py @@ -5,11 +5,15 @@ import time from http import server +from python.runfiles import Runfiles + def main(argv): - build_workspace_directory = os.environ["BUILD_WORKSPACE_DIRECTORY"] - docs_directory = argv[1] - serve_directory = os.path.join(build_workspace_directory, docs_directory) + r = Runfiles.Create() + serve_directory = r.Rlocation(argv[1]) + if not serve_directory: + print(f"Error: could not find runfile for '{argv[1]}'", file=sys.stderr) + return 1 class DirectoryHandler(server.SimpleHTTPRequestHandler): def __init__(self, *args, **kwargs): From 144984ccbd61f3459dd2a1f86210bd2aab69043f Mon Sep 17 00:00:00 2001 From: Markus Hofbauer Date: Thu, 8 Jan 2026 00:22:41 +0100 Subject: [PATCH 574/922] build: Enable incompatible_no_implicit_file_export in bazelrc (#3477) Identified by https://github.com/bazelbuild/bazel/pull/27674#issuecomment-3686476093, I want to see what else fails when flipping this flag on top of https://github.com/bazel-contrib/rules_python/pull/3471. --- .bazelrc | 1 + python/private/pypi/BUILD.bazel | 5 +++++ .../pypi/generate_whl_library_build_bazel.bzl | 12 +++++++++++ ...generate_whl_library_build_bazel_tests.bzl | 20 +++++++++++++++++++ 4 files changed, 38 insertions(+) diff --git a/.bazelrc b/.bazelrc index 718f83080c..24676574e6 100644 --- a/.bazelrc +++ b/.bazelrc @@ -38,5 +38,6 @@ build:rtd --stamp build:rtd --enable_bzlmod common --incompatible_python_disallow_native_rules +common --incompatible_no_implicit_file_export build --lockfile_mode=update diff --git a/python/private/pypi/BUILD.bazel b/python/private/pypi/BUILD.bazel index aa96cf86a5..96ae42fce9 100644 --- a/python/private/pypi/BUILD.bazel +++ b/python/private/pypi/BUILD.bazel @@ -23,6 +23,11 @@ exports_files( visibility = ["//visibility:public"], ) +exports_files( + srcs = ["deps.bzl"], + visibility = ["//tools/private/update_deps:__pkg__"], +) + filegroup( name = "distribution", srcs = glob( diff --git a/python/private/pypi/generate_whl_library_build_bazel.bzl b/python/private/pypi/generate_whl_library_build_bazel.bzl index e207f6d2f5..fbabe2ede3 100644 --- a/python/private/pypi/generate_whl_library_build_bazel.bzl +++ b/python/private/pypi/generate_whl_library_build_bazel.bzl @@ -100,6 +100,18 @@ def generate_whl_library_build_bazel( ]) additional_content = [] + entry_points = kwargs.get("entry_points") + if entry_points: + entry_point_files = sorted({ + entry_point_script.replace("\\", "/"): True + for entry_point_script in entry_points.values() + }.keys()) + additional_content.append( + "exports_files(\n" + + " srcs = {},\n".format(render.list(entry_point_files)) + + " visibility = [\"//visibility:public\"],\n" + + ")\n", + ) if annotation: kwargs["data"] = annotation.data kwargs["copy_files"] = annotation.copy_files diff --git a/tests/pypi/generate_whl_library_build_bazel/generate_whl_library_build_bazel_tests.bzl b/tests/pypi/generate_whl_library_build_bazel/generate_whl_library_build_bazel_tests.bzl index 39c2eb4379..85e96be579 100644 --- a/tests/pypi/generate_whl_library_build_bazel/generate_whl_library_build_bazel_tests.bzl +++ b/tests/pypi/generate_whl_library_build_bazel/generate_whl_library_build_bazel_tests.bzl @@ -56,6 +56,11 @@ whl_library_targets( tags = ["tag1"], ) +exports_files( + srcs = ["bar.py"], + visibility = ["//visibility:public"], +) + # SOMETHING SPECIAL AT THE END """ actual = generate_whl_library_build_bazel( @@ -122,6 +127,11 @@ whl_library_targets_from_requires( srcs_exclude = ["srcs_exclude_all"], ) +exports_files( + srcs = ["bar.py"], + visibility = ["//visibility:public"], +) + # SOMETHING SPECIAL AT THE END """ actual = generate_whl_library_build_bazel( @@ -187,6 +197,11 @@ whl_library_targets_from_requires( srcs_exclude = ["srcs_exclude_all"], ) +exports_files( + srcs = ["bar.py"], + visibility = ["//visibility:public"], +) + # SOMETHING SPECIAL AT THE END """ actual = generate_whl_library_build_bazel( @@ -252,6 +267,11 @@ whl_library_targets_from_requires( srcs_exclude = ["srcs_exclude_all"], ) +exports_files( + srcs = ["bar.py"], + visibility = ["//visibility:public"], +) + # SOMETHING SPECIAL AT THE END """ actual = generate_whl_library_build_bazel( From 62acad640c53881dfb8cd338cd3ed78bcb54c84d Mon Sep 17 00:00:00 2001 From: Ignas Anikevicius <240938+aignas@users.noreply.github.com> Date: Sun, 11 Jan 2026 06:03:30 +0900 Subject: [PATCH 575/922] fix(pipstar): correctly handle platlib and purelib in .data (#3501) Some packages like use `platlib` in the data to put the main files. This PR is implementing correct handling of such packages by recursively merging two trees. If we have any collisions, we will print an error and stop. That is unlikely but better to be safe. Users can patch the failure to be a warning if necessary. In order to make this more testable, move the functions to a separate file. Fixes #3500 Fixes #2949 To be cherry-picked as part of #3466 --- examples/pip_parse/BUILD.bazel | 5 +- examples/pip_parse/requirements.in | 1 + examples/pip_parse/requirements_lock.txt | 12 +++ examples/pip_parse/requirements_windows.txt | 12 +++ python/private/pypi/BUILD.bazel | 11 ++ python/private/pypi/whl_extract.bzl | 109 ++++++++++++++++++++ python/private/pypi/whl_library.bzl | 47 +-------- 7 files changed, 152 insertions(+), 45 deletions(-) create mode 100644 python/private/pypi/whl_extract.bzl diff --git a/examples/pip_parse/BUILD.bazel b/examples/pip_parse/BUILD.bazel index 6ed8d26286..37a25fe873 100644 --- a/examples/pip_parse/BUILD.bazel +++ b/examples/pip_parse/BUILD.bazel @@ -79,5 +79,8 @@ py_test( "WHEEL_DIST_INFO_CONTENTS": "$(rootpaths @pypi//requests:dist_info)", "YAMLLINT_ENTRY_POINT": "$(rlocationpath :yamllint)", }, - deps = ["@rules_python//python/runfiles"], + deps = [ + "@pypi//libclang", + "@rules_python//python/runfiles", + ], ) diff --git a/examples/pip_parse/requirements.in b/examples/pip_parse/requirements.in index 9d9e766d21..e4af3b1efe 100644 --- a/examples/pip_parse/requirements.in +++ b/examples/pip_parse/requirements.in @@ -3,3 +3,4 @@ s3cmd~=2.1.0 yamllint~=1.28.0 sphinx sphinxcontrib-serializinghtml +libclang diff --git a/examples/pip_parse/requirements_lock.txt b/examples/pip_parse/requirements_lock.txt index dc34b45a45..13a2bba1e6 100644 --- a/examples/pip_parse/requirements_lock.txt +++ b/examples/pip_parse/requirements_lock.txt @@ -42,6 +42,18 @@ jinja2==3.1.6 \ --hash=sha256:0137fb05990d35f1275a587e9aee6d56da821fc83491a0fb838183be43f66d6d \ --hash=sha256:85ece4451f492d0c13c5dd7c13a64681a86afae63a5f347908daf103ce6d2f67 # via sphinx +libclang==18.1.1 \ + --hash=sha256:0b2e143f0fac830156feb56f9231ff8338c20aecfe72b4ffe96f19e5a1dbb69a \ + --hash=sha256:3f0e1f49f04d3cd198985fea0511576b0aee16f9ff0e0f0cad7f9c57ec3c20e8 \ + --hash=sha256:4dd2d3b82fab35e2bf9ca717d7b63ac990a3519c7e312f19fa8e86dcc712f7fb \ + --hash=sha256:54dda940a4a0491a9d1532bf071ea3ef26e6dbaf03b5000ed94dd7174e8f9592 \ + --hash=sha256:69f8eb8f65c279e765ffd28aaa7e9e364c776c17618af8bff22a8df58677ff4f \ + --hash=sha256:6f14c3f194704e5d09769108f03185fce7acaf1d1ae4bbb2f30a72c2400cb7c5 \ + --hash=sha256:83ce5045d101b669ac38e6da8e58765f12da2d3aafb3b9b98d88b286a60964d8 \ + --hash=sha256:a1214966d08d73d971287fc3ead8dfaf82eb07fb197680d8b3859dbbbbf78250 \ + --hash=sha256:c533091d8a3bbf7460a00cb6c1a71da93bffe148f172c7d03b1c31fbf8aa2a0b \ + --hash=sha256:cf4a99b05376513717ab5d82a0db832c56ccea4fd61a69dbb7bccf2dfb207dbe + # via -r requirements.in markupsafe==2.1.3 \ --hash=sha256:05fb21170423db021895e1ea1e1f3ab3adb85d1c2333cbc2310f2a26bc77272e \ --hash=sha256:0a4e4a1aff6c7ac4cd55792abf96c915634c2b97e3cc1c7129578aa68ebd754e \ diff --git a/examples/pip_parse/requirements_windows.txt b/examples/pip_parse/requirements_windows.txt index 78c1a45690..7a1329d521 100644 --- a/examples/pip_parse/requirements_windows.txt +++ b/examples/pip_parse/requirements_windows.txt @@ -46,6 +46,18 @@ jinja2==3.1.6 \ --hash=sha256:0137fb05990d35f1275a587e9aee6d56da821fc83491a0fb838183be43f66d6d \ --hash=sha256:85ece4451f492d0c13c5dd7c13a64681a86afae63a5f347908daf103ce6d2f67 # via sphinx +libclang==18.1.1 \ + --hash=sha256:0b2e143f0fac830156feb56f9231ff8338c20aecfe72b4ffe96f19e5a1dbb69a \ + --hash=sha256:3f0e1f49f04d3cd198985fea0511576b0aee16f9ff0e0f0cad7f9c57ec3c20e8 \ + --hash=sha256:4dd2d3b82fab35e2bf9ca717d7b63ac990a3519c7e312f19fa8e86dcc712f7fb \ + --hash=sha256:54dda940a4a0491a9d1532bf071ea3ef26e6dbaf03b5000ed94dd7174e8f9592 \ + --hash=sha256:69f8eb8f65c279e765ffd28aaa7e9e364c776c17618af8bff22a8df58677ff4f \ + --hash=sha256:6f14c3f194704e5d09769108f03185fce7acaf1d1ae4bbb2f30a72c2400cb7c5 \ + --hash=sha256:83ce5045d101b669ac38e6da8e58765f12da2d3aafb3b9b98d88b286a60964d8 \ + --hash=sha256:a1214966d08d73d971287fc3ead8dfaf82eb07fb197680d8b3859dbbbbf78250 \ + --hash=sha256:c533091d8a3bbf7460a00cb6c1a71da93bffe148f172c7d03b1c31fbf8aa2a0b \ + --hash=sha256:cf4a99b05376513717ab5d82a0db832c56ccea4fd61a69dbb7bccf2dfb207dbe + # via -r requirements.in markupsafe==2.1.3 \ --hash=sha256:05fb21170423db021895e1ea1e1f3ab3adb85d1c2333cbc2310f2a26bc77272e \ --hash=sha256:0a4e4a1aff6c7ac4cd55792abf96c915634c2b97e3cc1c7129578aa68ebd754e \ diff --git a/python/private/pypi/BUILD.bazel b/python/private/pypi/BUILD.bazel index 96ae42fce9..0a97e5f953 100644 --- a/python/private/pypi/BUILD.bazel +++ b/python/private/pypi/BUILD.bazel @@ -420,6 +420,16 @@ bzl_library( srcs = ["whl_config_setting.bzl"], ) +bzl_library( + name = "whl_extract_bzl", + srcs = ["whl_extract.bzl"], + deps = [ + ":whl_metadata_bzl", + "//python/private:repo_utils_bzl", + "@rules_python_internal//:rules_python_config_bzl", + ], +) + bzl_library( name = "whl_library_alias_bzl", srcs = ["whl_library_alias.bzl"], @@ -440,6 +450,7 @@ bzl_library( ":patch_whl_bzl", ":pep508_requirement_bzl", ":pypi_repo_utils_bzl", + ":whl_extract_bzl", ":whl_metadata_bzl", ":whl_target_platforms_bzl", "//python/private:auth_bzl", diff --git a/python/private/pypi/whl_extract.bzl b/python/private/pypi/whl_extract.bzl new file mode 100644 index 0000000000..6b2e0507ac --- /dev/null +++ b/python/private/pypi/whl_extract.bzl @@ -0,0 +1,109 @@ +"""A simple whl extractor.""" + +load("@rules_python_internal//:rules_python_config.bzl", rp_config = "config") +load("//python/private:repo_utils.bzl", "repo_utils") +load(":whl_metadata.bzl", "find_whl_metadata") + +def whl_extract(rctx, *, whl_path, logger): + """Extract whls in Starlark. + + Args: + rctx: the repository ctx. + whl_path: the whl path to extract. + logger: The logger to use + """ + install_dir_path = whl_path.dirname.get_child("site-packages") + repo_utils.extract( + rctx, + archive = whl_path, + output = install_dir_path, + supports_whl_extraction = rp_config.supports_whl_extraction, + ) + metadata_file = find_whl_metadata( + install_dir = install_dir_path, + logger = logger, + ) + + # Get the .dist_info dir name + dist_info_dir = metadata_file.dirname + rctx.file( + dist_info_dir.get_child("INSTALLER"), + "https://github.com/bazel-contrib/rules_python#pipstar", + ) + repo_root_dir = whl_path.dirname + + # Get the .dist_info dir name + data_dir = dist_info_dir.dirname.get_child(dist_info_dir.basename[:-len(".dist-info")] + ".data") + if data_dir.exists: + for prefix, dest_prefix in { + # https://docs.python.org/3/library/sysconfig.html#posix-prefix + # We are taking this from the legacy whl installer config + "data": "data", + "headers": "include", + # In theory there may be directory collisions here, so it would be best to + # merge the paths here. We are doing for quite a few levels deep. What is + # more, this code has to be reasonably efficient because some packages like + # to not put everything to the top level, but to indicate explicitly if + # something is in `platlib` or `purelib` (e.g. libclang wheel). + "platlib": "site-packages", + "purelib": "site-packages", + "scripts": "bin", + }.items(): + src = data_dir.get_child(prefix) + if not src.exists: + # The prefix does not exist in the wheel, we can continue + continue + + for (src, dest) in merge_trees(src, repo_root_dir.get_child(dest_prefix)): + logger.debug(lambda: "Renaming: {} -> {}".format(src, dest)) + rctx.rename(src, dest) + + # TODO @aignas 2025-12-16: when moving scripts to `bin`, rewrite the #!python + # shebang to be something else, for inspiration look at the hermetic + # toolchain wrappers + + # Ensure that there is no data dir left + rctx.delete(data_dir) + +def merge_trees(src, dest): + """Merge src into the destination path. + + This will attempt to merge-move src files to the destination directory if there are + existing files. Fails at directory depth is 10000 or if there are collisions. + + Args: + src: {type}`path` a src path to rename. + dest: {type}`path` a dest path to rename to. + + Returns: + A list of tuples for src and destination paths. + """ + ret = [] + remaining = [(src, dest)] + collisions = [] + for _ in range(10000): + if collisions or not remaining: + break + + tmp = [] + for (s, d) in remaining: + if not d.exists: + ret.append((s, d)) + continue + + if not s.is_dir or not d.is_dir: + collisions.append(s) + continue + + for file_or_dir in s.readdir(): + tmp.append((file_or_dir, d.get_child(file_or_dir.basename))) + + remaining = tmp + + if remaining: + fail("Exceeded maximum directory depth of 10000 during tree merge.") + + if collisions: + fail("Detected collisions between {} and {}: {}".format(src, dest, collisions)) + + return ret diff --git a/python/private/pypi/whl_library.bzl b/python/private/pypi/whl_library.bzl index c368dea733..3c4b6beeaf 100644 --- a/python/private/pypi/whl_library.bzl +++ b/python/private/pypi/whl_library.bzl @@ -26,7 +26,8 @@ load(":parse_whl_name.bzl", "parse_whl_name") load(":patch_whl.bzl", "patch_whl") load(":pep508_requirement.bzl", "requirement") load(":pypi_repo_utils.bzl", "pypi_repo_utils") -load(":whl_metadata.bzl", "find_whl_metadata", "whl_metadata") +load(":whl_extract.bzl", "whl_extract") +load(":whl_metadata.bzl", "whl_metadata") load(":whl_target_platforms.bzl", "whl_target_platforms") _CPPFLAGS = "CPPFLAGS" @@ -265,48 +266,6 @@ def _create_repository_execution_environment(rctx, python_interpreter, logger = env[_CPPFLAGS] = " ".join(cppflags) return env -def _extract_whl_star(rctx, *, whl_path, logger): - install_dir_path = whl_path.dirname.get_child("site-packages") - repo_utils.extract( - rctx, - archive = whl_path, - output = install_dir_path, - supports_whl_extraction = rp_config.supports_whl_extraction, - ) - metadata_file = find_whl_metadata( - install_dir = install_dir_path, - logger = logger, - ) - - # Get the .dist_info dir name - dist_info_dir = metadata_file.dirname - rctx.file( - dist_info_dir.get_child("INSTALLER"), - "https://github.com/bazel-contrib/rules_python#pipstar", - ) - repo_root_dir = whl_path.dirname - - # Get the .dist_info dir name - data_dir = dist_info_dir.dirname.get_child(dist_info_dir.basename[:-len(".dist-info")] + ".data") - if data_dir.exists: - for prefix, dest in { - # https://docs.python.org/3/library/sysconfig.html#posix-prefix - # We are taking this from the legacy whl installer config - "data": "data", - "headers": "include", - "platlib": "site-packages", - "purelib": "site-packages", - "scripts": "bin", - }.items(): - src = data_dir.get_child(prefix) - dest = repo_root_dir.get_child(dest) - if src.exists: - rctx.rename(src, dest) - - # TODO @aignas 2025-12-16: when moving scripts to `bin`, rewrite the #!python - # shebang to be something else, for inspiration look at the hermetic - # toolchain wrappers - def _extract_whl_py(rctx, *, python_interpreter, args, whl_path, environment, logger): target_platforms = rctx.attr.experimental_target_platforms or [] if target_platforms: @@ -448,7 +407,7 @@ def _whl_library_impl(rctx): ) if enable_pipstar_extract: - _extract_whl_star(rctx, whl_path = whl_path, logger = logger) + whl_extract(rctx, whl_path = whl_path, logger = logger) else: _extract_whl_py( rctx, From ffb7001634d8e44d157f6247c820f164d3c905c0 Mon Sep 17 00:00:00 2001 From: Ignas Anikevicius <240938+aignas@users.noreply.github.com> Date: Sun, 11 Jan 2026 06:04:28 +0900 Subject: [PATCH 576/922] doc: Add a snippet on how people can use 3.8 now onwards (#3502) Thanks to Ted Kaplan (@thirtyseven) for the snippet. --- CHANGELOG.md | 17 +++++++++++++++++ 1 file changed, 17 insertions(+) diff --git a/CHANGELOG.md b/CHANGELOG.md index 0df0de8cee..0a126504b5 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -92,6 +92,23 @@ END_UNRELEASED_TEMPLATE to pass the `TOOL_VERSIONS` that include 3.8 toolchains or use the `bzlmod` APIs to add them back. This means any hub `pip.parse` calls that target `3.8` will be ignored from now on. ([#2704](https://github.com/bazel-contrib/rules_python/issues/2704)) + {object}`python.single_version_override`, like: + + ```starlark + python = use_extension("@rules_python//python/extensions:python.bzl", "python") + + python.single_version_override( + python_version = "3.8.20", + sha256 = { + "aarch64-apple-darwin": "2ddfc04bdb3e240f30fb782fa1deec6323799d0e857e0b63fa299218658fd3d4", + "aarch64-unknown-linux-gnu": "9d8798f9e79e0fc0f36fcb95bfa28a1023407d51a8ea5944b4da711f1f75f1ed", + "x86_64-apple-darwin": "68d060cd373255d2ca5b8b3441363d5aa7cc45b0c11bbccf52b1717c2b5aa8bb", + "x86_64-pc-windows-msvc": "41b6709fec9c56419b7de1940d1f87fa62045aff81734480672dcb807eedc47e", + "x86_64-unknown-linux-gnu": "285e141c36f88b2e9357654c5f77d1f8fb29cc25132698fe35bb30d787f38e87", + }, + urls = ["https://github.com/astral-sh/python-build-standalone/releases/download/20241002/cpython-{python_version}+20241002-{platform}-{build}.tar.gz"], + ) + ``` * (toolchain) Remove all of the python 3.9 toolchain versions except for the `3.9.25`. This version has reached EOL and will no longer receive any security fixes, please update to `3.10` or above. ([#2704](https://github.com/bazel-contrib/rules_python/issues/2704)) From 2398fccb8838e81379576c87ed74437d220fe77e Mon Sep 17 00:00:00 2001 From: Ignas Anikevicius <240938+aignas@users.noreply.github.com> Date: Sun, 11 Jan 2026 06:40:03 +0900 Subject: [PATCH 577/922] refactor(pypi): print a better error message for duplicate repos (#3487) Before this PR the error message would not be super helpful and may potentially make it hard to debug and report errors. This PR does the following: * Add a better error message which also adds comparison of the args with which we create the whl library. * Add a test that ensures that the error message is legible and works. * Add the necessary plumbing to logger to allow for testing error messages. A proper fix requires more work, so just adding better logging and error messages may be useful here. Work towards #3479 --- python/private/pypi/BUILD.bazel | 1 + python/private/pypi/hub_builder.bzl | 70 ++++++++++++++++++-- python/private/repo_utils.bzl | 22 ++++-- tests/pypi/hub_builder/hub_builder_tests.bzl | 59 +++++++++++++++++ 4 files changed, 141 insertions(+), 11 deletions(-) diff --git a/python/private/pypi/BUILD.bazel b/python/private/pypi/BUILD.bazel index 0a97e5f953..b46fd58d3c 100644 --- a/python/private/pypi/BUILD.bazel +++ b/python/private/pypi/BUILD.bazel @@ -178,6 +178,7 @@ bzl_library( ":whl_repo_name_bzl", "//python/private:full_version_bzl", "//python/private:normalize_name_bzl", + "//python/private:text_util_bzl", "//python/private:version_bzl", "//python/private:version_label_bzl", ], diff --git a/python/private/pypi/hub_builder.bzl b/python/private/pypi/hub_builder.bzl index 700f22e2c0..f54d02d8b0 100644 --- a/python/private/pypi/hub_builder.bzl +++ b/python/private/pypi/hub_builder.bzl @@ -3,6 +3,7 @@ load("//python/private:full_version.bzl", "full_version") load("//python/private:normalize_name.bzl", "normalize_name") load("//python/private:repo_utils.bzl", "repo_utils") +load("//python/private:text_util.bzl", "render") load("//python/private:version.bzl", "version") load("//python/private:version_label.bzl", "version_label") load(":attrs.bzl", "use_isolated") @@ -86,6 +87,16 @@ def hub_builder( ### PUBLIC methods def _build(self): + ret = struct( + whl_map = {}, + group_map = {}, + extra_aliases = {}, + exposed_packages = [], + whl_libraries = {}, + ) + if self._logger.failed(): + return ret + whl_map = {} for key, settings in self._whl_map.items(): for setting, repo in settings.items(): @@ -196,6 +207,44 @@ def _add_extra_aliases(self, extra_hub_aliases): {alias: True for alias in aliases}, ) +def _diff_dict(first, second): + """A simple utility to shallow compare dictionaries. + + Args: + first: The first dictionary to compare. + second: The second dictionary to compare. + + Returns: + A dictionary containing the differences, with keys "common", "different", + "extra", and "missing", or None if the dictionaries are identical. + """ + missing = {} + extra = { + key: value + for key, value in second.items() + if key not in first + } + common = {} + different = {} + + for key, value in first.items(): + if key not in second: + missing[key] = value + elif value == second[key]: + common[key] = value + else: + different[key] = (value, second[key]) + + if missing or extra or different: + return { + "common": common, + "different": different, + "extra": extra, + "missing": missing, + } + else: + return None + def _add_whl_library(self, *, python_version, whl, repo, enable_pipstar): if repo == None: # NOTE @aignas 2025-07-07: we guard against an edge-case where there @@ -207,13 +256,26 @@ def _add_whl_library(self, *, python_version, whl, repo, enable_pipstar): # TODO @aignas 2025-06-29: we should not need the version in the repo_name if # we are using pipstar and we are downloading the wheel using the downloader + # + # However, for that we should first have a different way to reference closures with + # extras. For example, if some package depends on `foo[extra]` and another depends on + # `foo`, we should have 2 py_library targets. repo_name = "{}_{}_{}".format(self.name, version_label(python_version), repo.repo_name) if repo_name in self._whl_libraries: - fail("attempting to create a duplicate library {} for {}".format( - repo_name, - whl.name, - )) + diff = _diff_dict(self._whl_libraries[repo_name], repo.args) + if diff: + self._logger.fail(lambda: ( + "Attempting to create a duplicate library {repo_name} for {whl_name} with different arguments. Already existing declaration has:\n".format( + repo_name = repo_name, + whl_name = whl.name, + ) + "\n".join([ + " {}: {}".format(key, render.indent(render.dict(value)).lstrip()) + for key, value in diff.items() + if value + ]) + )) + return self._whl_libraries[repo_name] = repo.args if not enable_pipstar and "experimental_target_platforms" in repo.args: diff --git a/python/private/repo_utils.bzl b/python/private/repo_utils.bzl index 1abff36a04..28ba07d376 100644 --- a/python/private/repo_utils.bzl +++ b/python/private/repo_utils.bzl @@ -31,7 +31,7 @@ def _is_repo_debug_enabled(mrctx): """ return _getenv(mrctx, REPO_DEBUG_ENV_VAR) == "1" -def _logger(mrctx = None, name = None, verbosity_level = None): +def _logger(mrctx = None, name = None, verbosity_level = None, printer = None): """Creates a logger instance for printing messages. Args: @@ -39,7 +39,9 @@ def _logger(mrctx = None, name = None, verbosity_level = None): `_rule_name` is present, it will be included in log messages. name: name for the logger. Optional for repository_ctx usage. verbosity_level: {type}`int | None` verbosity level. If not set, - taken from `mrctx` + taken from `mrctx`. + printer: a function to use for printing. Defaults to `print` or `fail` depending + on the logging method. Returns: A struct with attributes logging: trace, debug, info, warn, fail. @@ -70,10 +72,15 @@ def _logger(mrctx = None, name = None, verbosity_level = None): elif not name: fail("The name has to be specified when using the logger with `module_ctx`") + failures = [] + def _log(enabled_on_verbosity, level, message_cb_or_str, printer = print): if verbosity < enabled_on_verbosity: return + if level == "FAIL": + failures.append(None) + if type(message_cb_or_str) == "string": message = message_cb_or_str else: @@ -86,11 +93,12 @@ def _logger(mrctx = None, name = None, verbosity_level = None): ), message) # buildifier: disable=print return struct( - trace = lambda message_cb: _log(3, "TRACE", message_cb), - debug = lambda message_cb: _log(2, "DEBUG", message_cb), - info = lambda message_cb: _log(1, "INFO", message_cb), - warn = lambda message_cb: _log(0, "WARNING", message_cb), - fail = lambda message_cb: _log(-1, "FAIL", message_cb, fail), + trace = lambda message_cb: _log(3, "TRACE", message_cb, printer or print), + debug = lambda message_cb: _log(2, "DEBUG", message_cb, printer or print), + info = lambda message_cb: _log(1, "INFO", message_cb, printer or print), + warn = lambda message_cb: _log(0, "WARNING", message_cb, printer or print), + fail = lambda message_cb: _log(-1, "FAIL", message_cb, printer or fail), + failed = lambda: len(failures) != 0, ) def _execute_internal( diff --git a/tests/pypi/hub_builder/hub_builder_tests.bzl b/tests/pypi/hub_builder/hub_builder_tests.bzl index 42c65ae8f7..03cefd13c5 100644 --- a/tests/pypi/hub_builder/hub_builder_tests.bzl +++ b/tests/pypi/hub_builder/hub_builder_tests.bzl @@ -48,6 +48,7 @@ def hub_builder( whl_overrides = {}, evaluate_markers_fn = None, simpleapi_download_fn = None, + log_printer = None, available_interpreters = {}): builder = _hub_builder( name = "pypi", @@ -96,6 +97,7 @@ def hub_builder( ), ), "unit-test", + printer = log_printer, ), ) self = struct( @@ -1245,6 +1247,63 @@ optimum[onnxruntime-gpu]==1.17.1 ; sys_platform == 'linux' _tests.append(_test_pipstar_platforms_limit) +def _test_err_duplicate_repos(env): + logs = {} + log_printer = lambda key, message: logs.setdefault(key.strip(), []).append(message) + builder = hub_builder( + env, + available_interpreters = { + "python_3_15_1_host": "unit_test_interpreter_target_1", + "python_3_15_2_host": "unit_test_interpreter_target_2", + }, + log_printer = log_printer, + ) + builder.pip_parse( + _mock_mctx( + os_name = "osx", + arch_name = "aarch64", + ), + _parse( + hub_name = "pypi", + python_version = "3.15.1", + requirements_lock = "requirements.txt", + ), + ) + builder.pip_parse( + _mock_mctx( + os_name = "osx", + arch_name = "aarch64", + ), + _parse( + hub_name = "pypi", + python_version = "3.15.2", + requirements_lock = "requirements.txt", + ), + ) + pypi = builder.build() + + pypi.exposed_packages().contains_exactly([]) + pypi.group_map().contains_exactly({}) + pypi.whl_map().contains_exactly({}) + pypi.whl_libraries().contains_exactly({}) + pypi.extra_aliases().contains_exactly({}) + env.expect.that_dict(logs).keys().contains_exactly(["rules_python:unit-test FAIL:"]) + env.expect.that_collection(logs["rules_python:unit-test FAIL:"]).contains_exactly([ + """\ +Attempting to create a duplicate library pypi_315_simple for simple with different arguments. Already existing declaration has: + common: { + "dep_template": "@pypi//{name}:{target}", + "config_load": "@pypi//:config.bzl", + "requirement": "simple==0.0.1 --hash=sha256:deadbeef --hash=sha256:deadbaaf", + } + different: { + "python_interpreter_target": ("unit_test_interpreter_target_1", "unit_test_interpreter_target_2"), + }\ +""", + ]).in_order() + +_tests.append(_test_err_duplicate_repos) + def hub_builder_test_suite(name): """Create the test suite. From 22f3de02efe54f00d532bcd77939710507e7209e Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Sat, 10 Jan 2026 16:57:33 -0800 Subject: [PATCH 578/922] chore: create bcr prs as non-draft so bazel-io processes the bot-created PRs (#3504) By default, the workflow creates draft PRs. The bazel-io bot ignores drafts. Since we can't mark the PR as non-draft ourselves, we then have to wait for bcr maintainers to do so. To fix, create the bcr prs are non-draft. Then we can approve and the bazel-io bot will approve and merge the PR. --- .github/workflows/publish.yml | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/.github/workflows/publish.yml b/.github/workflows/publish.yml index fb46168fc1..5ef65e83a6 100644 --- a/.github/workflows/publish.yml +++ b/.github/workflows/publish.yml @@ -27,6 +27,10 @@ jobs: # GitHub repository which is a fork of the upstream where the Pull Request will be opened. registry_fork: bazel-contrib/bazel-central-registry attest: false + # Create non-draft PRs so the BCR bazel-io bot processes them. Otherwise, + # since a bazel-contrib bot, we have to wait for BCR maintainers to mark it + # non-draft or approve it. + draft: false permissions: contents: write secrets: From f9992f7f11472a825a73e096f2904122a71f094e Mon Sep 17 00:00:00 2001 From: Shayan Hoshyari <108962133+shayanhoshyari@users.noreply.github.com> Date: Sun, 11 Jan 2026 10:51:17 -0800 Subject: [PATCH 579/922] fix (venv_site_packages): Fix wrong runfiles.symlinks when py_binary is not in root module (#3505) When: 1) venv_site_packages is on 2) we have a py_binary in a non-root module (e.g. a tool used in rules), and it uses python deps that result in usage of runfiles.symlinks, the `ctx.runfile` based symlinks end up going in the wrong folder (`_main`), while the `ctx.actions.symlink(...)` files go in the right place. This results in an invalid `venv` and import errors. The reason is that `actions.symlinks` always go to the `_main` module, as [Bazel docs explain](https://bazel.build/extending/rules#runfiles_symlinks). To send symlinks to other modules, one needs to use root_symlinks and prefix them with the right module name. Fixes: https://github.com/bazel-contrib/rules_python/issues/3503 --------- Co-authored-by: Shayan Hoshyari Co-authored-by: Richard Levasseur --- python/private/py_executable.bzl | 4 +++- python/private/venv_runfiles.bzl | 5 ++++- tests/modules/other/BUILD.bazel | 16 ++++++++++++++++ tests/modules/other/venv_bin.py | 17 +++++++++++++++++ tests/venv_site_packages_libs/BUILD.bazel | 15 +++++++++++++++ .../py_binary_other_module_test.sh | 6 ++++++ 6 files changed, 61 insertions(+), 2 deletions(-) create mode 100644 tests/modules/other/venv_bin.py create mode 100755 tests/venv_site_packages_libs/py_binary_other_module_test.sh diff --git a/python/private/py_executable.bzl b/python/private/py_executable.bzl index 1b884e9b3b..5bac6247ef 100644 --- a/python/private/py_executable.bzl +++ b/python/private/py_executable.bzl @@ -629,8 +629,10 @@ def _create_venv(ctx, output_prefix, imports, runtime_details, add_runfiles_root ), # venv files for user library dependencies (files that are specific # to the executable bootstrap and python runtime aren't here). + # `root_symlinks` should be used, otherwise, with symlinks files always go + # to `_main` prefix, and binaries from non-root module become broken. lib_runfiles = ctx.runfiles( - symlinks = venv_app_files.runfiles_symlinks, + root_symlinks = venv_app_files.runfiles_symlinks, ), ) diff --git a/python/private/venv_runfiles.bzl b/python/private/venv_runfiles.bzl index 02b18e57e8..7f6af0c957 100644 --- a/python/private/venv_runfiles.bzl +++ b/python/private/venv_runfiles.bzl @@ -65,7 +65,10 @@ def create_venv_app_files(ctx, deps, venv_dir_map): bin_venv_path = paths.join(base, venv_path) if is_file(link_to): # use paths.join to handle ctx.label.package = "" - symlink_from = paths.join(ctx.label.package, bin_venv_path) + # runfile_prefix should be prepended as we use runfiles.root_symlinks + runfile_prefix = ctx.label.repo_name or ctx.workspace_name + symlink_from = paths.join(runfile_prefix, ctx.label.package, bin_venv_path) + runfiles_symlinks[symlink_from] = link_to else: venv_link = ctx.actions.declare_symlink(bin_venv_path) diff --git a/tests/modules/other/BUILD.bazel b/tests/modules/other/BUILD.bazel index 46f1b96faa..665049b9f5 100644 --- a/tests/modules/other/BUILD.bazel +++ b/tests/modules/other/BUILD.bazel @@ -1,3 +1,4 @@ +load("@rules_python//python:py_binary.bzl", "py_binary") load("@rules_python//tests/support:py_reconfig.bzl", "py_reconfig_binary") package( @@ -12,3 +13,18 @@ py_reconfig_binary( bootstrap_impl = "system_python", main = "external_main.py", ) + +py_binary( + name = "venv_bin", + srcs = ["venv_bin.py"], + config_settings = { + "@rules_python//python/config_settings:bootstrap_impl": "script", + "@rules_python//python/config_settings:venvs_site_packages": "yes", + }, + deps = [ + # Add two packages that install into the same directory. This is + # to test that namespace packages install correctly and are importable. + "//nspkg_delta", + "//nspkg_gamma", + ], +) diff --git a/tests/modules/other/venv_bin.py b/tests/modules/other/venv_bin.py new file mode 100644 index 0000000000..5455b23f40 --- /dev/null +++ b/tests/modules/other/venv_bin.py @@ -0,0 +1,17 @@ +import nspkg + +print(nspkg) + +import nspkg.subnspkg + +print(nspkg.subnspkg) + +import nspkg.subnspkg.delta + +print(nspkg.subnspkg.delta) + +import nspkg.subnspkg.gamma + +print(nspkg.subnspkg.gamma) + +print("@other//:venv_bin ran successfully.") diff --git a/tests/venv_site_packages_libs/BUILD.bazel b/tests/venv_site_packages_libs/BUILD.bazel index 2eb9678838..e573dc6da6 100644 --- a/tests/venv_site_packages_libs/BUILD.bazel +++ b/tests/venv_site_packages_libs/BUILD.bazel @@ -1,3 +1,4 @@ +load("@rules_shell//shell:sh_test.bzl", "sh_test") load("//python:py_library.bzl", "py_library") load("//tests/support:py_reconfig.bzl", "py_reconfig_test") load( @@ -19,6 +20,20 @@ py_library( ], ) +sh_test( + name = "py_binary_other_module_test", + srcs = [ + "py_binary_other_module_test.sh", + ], + data = [ + "@other//:venv_bin", + ], + env = { + "VENV_BIN": "$(rootpath @other//:venv_bin)", + }, + target_compatible_with = NOT_WINDOWS, +) + py_reconfig_test( name = "venvs_site_packages_libs_test", srcs = ["bin.py"], diff --git a/tests/venv_site_packages_libs/py_binary_other_module_test.sh b/tests/venv_site_packages_libs/py_binary_other_module_test.sh new file mode 100755 index 0000000000..9fb71d5bc7 --- /dev/null +++ b/tests/venv_site_packages_libs/py_binary_other_module_test.sh @@ -0,0 +1,6 @@ +# Test that for a py_binary from a dependency module, we place links created via +# runfiles(...) in the right place. This tests the fix made for issues/3503 + +set -eu +echo "[*] Testing running the binary" +"$VENV_BIN" From 42085b53cb29e6513ff6f68e9e441b89c17c35ba Mon Sep 17 00:00:00 2001 From: Ignas Anikevicius <240938+aignas@users.noreply.github.com> Date: Mon, 12 Jan 2026 15:56:16 +0900 Subject: [PATCH 580/922] test(whl_library): test a recent fix for pipstar (#3469) Test for fix for #3352 implemented in #3468. --------- Co-authored-by: Richard Levasseur --- MODULE.bazel | 1 + python/private/internal_dev_deps.bzl | 30 +++++++++++++++++++ python/private/pypi/whl_library.bzl | 2 +- tests/pypi/whl_library/BUILD.bazel | 11 +++++++ tests/pypi/whl_library/testdata/BUILD.bazel | 0 .../testdata/optional_dep/BUILD.bazel | 0 .../optional-dep-1.0.dist-info/METADATA | 2 ++ .../optional-dep-1.0.dist-info/RECORD | 0 .../optional-dep-1.0.dist-info/WHEEL | 1 + .../testdata/optional_dep/optional_dep.py | 1 + tests/pypi/whl_library/testdata/packages.bzl | 6 ++++ .../pypi/whl_library/testdata/pkg/BUILD.bazel | 0 .../testdata/pkg/pkg-1.0.dist-info/METADATA | 4 +++ .../testdata/pkg/pkg-1.0.dist-info/RECORD | 0 .../testdata/pkg/pkg-1.0.dist-info/WHEEL | 1 + tests/pypi/whl_library/testdata/pkg/pkg.py | 6 ++++ .../whl_library/whl_library_extras_test.py | 13 ++++++++ 17 files changed, 77 insertions(+), 1 deletion(-) create mode 100644 tests/pypi/whl_library/BUILD.bazel create mode 100644 tests/pypi/whl_library/testdata/BUILD.bazel create mode 100644 tests/pypi/whl_library/testdata/optional_dep/BUILD.bazel create mode 100644 tests/pypi/whl_library/testdata/optional_dep/optional-dep-1.0.dist-info/METADATA create mode 100644 tests/pypi/whl_library/testdata/optional_dep/optional-dep-1.0.dist-info/RECORD create mode 100644 tests/pypi/whl_library/testdata/optional_dep/optional-dep-1.0.dist-info/WHEEL create mode 100644 tests/pypi/whl_library/testdata/optional_dep/optional_dep.py create mode 100644 tests/pypi/whl_library/testdata/packages.bzl create mode 100644 tests/pypi/whl_library/testdata/pkg/BUILD.bazel create mode 100644 tests/pypi/whl_library/testdata/pkg/pkg-1.0.dist-info/METADATA create mode 100644 tests/pypi/whl_library/testdata/pkg/pkg-1.0.dist-info/RECORD create mode 100644 tests/pypi/whl_library/testdata/pkg/pkg-1.0.dist-info/WHEEL create mode 100644 tests/pypi/whl_library/testdata/pkg/pkg.py create mode 100644 tests/pypi/whl_library/whl_library_extras_test.py diff --git a/MODULE.bazel b/MODULE.bazel index 80c7ab1d99..6486634370 100644 --- a/MODULE.bazel +++ b/MODULE.bazel @@ -242,6 +242,7 @@ use_repo( "pkgutil_nspkg2", "rules_python_runtime_env_tc_info", "somepkg_with_build_files", + "whl_library_extras_direct_dep", "whl_with_build_files", ) diff --git a/python/private/internal_dev_deps.bzl b/python/private/internal_dev_deps.bzl index 0e21d8b8a5..fbdd5711b1 100644 --- a/python/private/internal_dev_deps.bzl +++ b/python/private/internal_dev_deps.bzl @@ -91,6 +91,36 @@ def _internal_dev_deps_impl(mctx): enable_implicit_namespace_pkgs = False, ) + _whl_library_from_dir( + name = "whl_library_extras_direct_dep", + root = "//tests/pypi/whl_library/testdata/pkg:BUILD.bazel", + output = "pkg-1.0-any-none-any.whl", + requirement = "pkg[optional]", + # The following is necessary to enable pipstar and make tests faster + config_load = "@rules_python//tests/pypi/whl_library/testdata:packages.bzl", + dep_template = "@whl_library_extras_{name}//:{target}", + ) + _whl_library_from_dir( + name = "whl_library_extras_optional_dep", + root = "//tests/pypi/whl_library/testdata/optional_dep:BUILD.bazel", + output = "optional_dep-1.0-any-none-any.whl", + requirement = "optional_dep", + # The following is necessary to enable pipstar and make tests faster + config_load = "@rules_python//tests/pypi/whl_library/testdata:packages.bzl", + ) + +def _whl_library_from_dir(*, name, output, root, **kwargs): + whl_from_dir_repo( + name = "{}_whl".format(name), + root = root, + output = output, + ) + whl_library( + name = name, + whl_file = "@{}_whl//:{}".format(name, output), + **kwargs + ) + internal_dev_deps = module_extension( implementation = _internal_dev_deps_impl, doc = "This extension creates internal rules_python dev dependencies.", diff --git a/python/private/pypi/whl_library.bzl b/python/private/pypi/whl_library.bzl index 3c4b6beeaf..db2b6bc770 100644 --- a/python/private/pypi/whl_library.bzl +++ b/python/private/pypi/whl_library.bzl @@ -359,7 +359,7 @@ def _whl_library_impl(rctx): # also enable pipstar for any whls that are downloaded without `pip` enable_pipstar = (rp_config.enable_pipstar or whl_path) and rctx.attr.config_load - enable_pipstar_extract = (rp_config.enable_pipstar and rp_config.bazel_8_or_later) and rctx.attr.config_load + enable_pipstar_extract = enable_pipstar and rp_config.bazel_8_or_later if not whl_path: if rctx.attr.urls: diff --git a/tests/pypi/whl_library/BUILD.bazel b/tests/pypi/whl_library/BUILD.bazel new file mode 100644 index 0000000000..599bb12a15 --- /dev/null +++ b/tests/pypi/whl_library/BUILD.bazel @@ -0,0 +1,11 @@ +load("//python:py_test.bzl", "py_test") +load("//tests/support:support.bzl", "SUPPORTS_BZLMOD_UNIXY") + +py_test( + name = "whl_library_extras_test", + srcs = ["whl_library_extras_test.py"], + target_compatible_with = SUPPORTS_BZLMOD_UNIXY, + deps = [ + "@whl_library_extras_direct_dep//:pkg", + ], +) diff --git a/tests/pypi/whl_library/testdata/BUILD.bazel b/tests/pypi/whl_library/testdata/BUILD.bazel new file mode 100644 index 0000000000..e69de29bb2 diff --git a/tests/pypi/whl_library/testdata/optional_dep/BUILD.bazel b/tests/pypi/whl_library/testdata/optional_dep/BUILD.bazel new file mode 100644 index 0000000000..e69de29bb2 diff --git a/tests/pypi/whl_library/testdata/optional_dep/optional-dep-1.0.dist-info/METADATA b/tests/pypi/whl_library/testdata/optional_dep/optional-dep-1.0.dist-info/METADATA new file mode 100644 index 0000000000..6495d1ba36 --- /dev/null +++ b/tests/pypi/whl_library/testdata/optional_dep/optional-dep-1.0.dist-info/METADATA @@ -0,0 +1,2 @@ +Name: optional-dep +Version: 1.0 diff --git a/tests/pypi/whl_library/testdata/optional_dep/optional-dep-1.0.dist-info/RECORD b/tests/pypi/whl_library/testdata/optional_dep/optional-dep-1.0.dist-info/RECORD new file mode 100644 index 0000000000..e69de29bb2 diff --git a/tests/pypi/whl_library/testdata/optional_dep/optional-dep-1.0.dist-info/WHEEL b/tests/pypi/whl_library/testdata/optional_dep/optional-dep-1.0.dist-info/WHEEL new file mode 100644 index 0000000000..a64521a1cc --- /dev/null +++ b/tests/pypi/whl_library/testdata/optional_dep/optional-dep-1.0.dist-info/WHEEL @@ -0,0 +1 @@ +Wheel-Version: 1.0 diff --git a/tests/pypi/whl_library/testdata/optional_dep/optional_dep.py b/tests/pypi/whl_library/testdata/optional_dep/optional_dep.py new file mode 100644 index 0000000000..4af2718944 --- /dev/null +++ b/tests/pypi/whl_library/testdata/optional_dep/optional_dep.py @@ -0,0 +1 @@ +I_AM_OPTIONAL = True diff --git a/tests/pypi/whl_library/testdata/packages.bzl b/tests/pypi/whl_library/testdata/packages.bzl new file mode 100644 index 0000000000..e4a9b0af4c --- /dev/null +++ b/tests/pypi/whl_library/testdata/packages.bzl @@ -0,0 +1,6 @@ +"""A list of packages that this logical testdata hub repo contains.""" + +packages = [ + "optional_dep", + "pkg", +] diff --git a/tests/pypi/whl_library/testdata/pkg/BUILD.bazel b/tests/pypi/whl_library/testdata/pkg/BUILD.bazel new file mode 100644 index 0000000000..e69de29bb2 diff --git a/tests/pypi/whl_library/testdata/pkg/pkg-1.0.dist-info/METADATA b/tests/pypi/whl_library/testdata/pkg/pkg-1.0.dist-info/METADATA new file mode 100644 index 0000000000..712b44edbf --- /dev/null +++ b/tests/pypi/whl_library/testdata/pkg/pkg-1.0.dist-info/METADATA @@ -0,0 +1,4 @@ +Name: pkg +Version: 1.0 +Requires-Dist: optional_dep; extra == "optional" +Provides-Extra: optional diff --git a/tests/pypi/whl_library/testdata/pkg/pkg-1.0.dist-info/RECORD b/tests/pypi/whl_library/testdata/pkg/pkg-1.0.dist-info/RECORD new file mode 100644 index 0000000000..e69de29bb2 diff --git a/tests/pypi/whl_library/testdata/pkg/pkg-1.0.dist-info/WHEEL b/tests/pypi/whl_library/testdata/pkg/pkg-1.0.dist-info/WHEEL new file mode 100644 index 0000000000..a64521a1cc --- /dev/null +++ b/tests/pypi/whl_library/testdata/pkg/pkg-1.0.dist-info/WHEEL @@ -0,0 +1 @@ +Wheel-Version: 1.0 diff --git a/tests/pypi/whl_library/testdata/pkg/pkg.py b/tests/pypi/whl_library/testdata/pkg/pkg.py new file mode 100644 index 0000000000..c5aca099c4 --- /dev/null +++ b/tests/pypi/whl_library/testdata/pkg/pkg.py @@ -0,0 +1,6 @@ +try: + import optional_dep + + WITH_EXTRAS = True +except ImportError: + WITH_EXTRAS = False diff --git a/tests/pypi/whl_library/whl_library_extras_test.py b/tests/pypi/whl_library/whl_library_extras_test.py new file mode 100644 index 0000000000..4fe344470a --- /dev/null +++ b/tests/pypi/whl_library/whl_library_extras_test.py @@ -0,0 +1,13 @@ +import unittest + + +class NamespacePackagesTest(unittest.TestCase): + + def test_extras_propagated(self): + import pkg + + self.assertEqual(pkg.WITH_EXTRAS, True) + + +if __name__ == "__main__": + unittest.main() From b47b92b0f7ab3b4c4de361108bb2605dcb94fc5d Mon Sep 17 00:00:00 2001 From: Josh Cannon Date: Tue, 13 Jan 2026 17:23:41 -0600 Subject: [PATCH 581/922] feat(gazelle): Add ancestor conftest.py files (#3498) Re-introduces the gazelle behavior of adding conftest targets to tests so that they inherit parent directory configs. Fixes #3497 --- CHANGELOG.md | 2 + gazelle/python/generate.go | 37 +++++++++++++++---- .../simple_test_with_conftest/bar/BUILD.out | 1 + .../bar/BUILD.out | 1 + 4 files changed, 33 insertions(+), 8 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 0a126504b5..c9f812f3bd 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -66,6 +66,8 @@ END_UNRELEASED_TEMPLATE ### Fixed * (tests) No more coverage warnings are being printed if there are no sources. ([#2762](https://github.com/bazel-contrib/rules_python/issues/2762)) +* (gazelle) Ancestor `conftest.py` files are added in addition to sibling `conftest.py`. + ([#3497](https://github.com/bazel-contrib/rules_python/issues/3497)) {#v0-0-0-added} ### Added diff --git a/gazelle/python/generate.go b/gazelle/python/generate.go index cbceea4693..90216b00bc 100644 --- a/gazelle/python/generate.go +++ b/gazelle/python/generate.go @@ -66,6 +66,24 @@ func matchesAnyGlob(s string, globs []string) bool { return false } +// findConftestPaths returns package paths containing conftest.py, from currentPkg +// up through ancestors, stopping at module root. +func findConftestPaths(repoRoot, currentPkg, pythonProjectRoot string) []string { + var result []string + for pkg := currentPkg; ; pkg = filepath.Dir(pkg) { + if pkg == "." { + pkg = "" + } + if _, err := os.Stat(filepath.Join(repoRoot, pkg, conftestFilename)); err == nil { + result = append(result, pkg) + } + if pkg == "" { + break + } + } + return result +} + // GenerateRules extracts build metadata from source files in a directory. // GenerateRules is called in each directory where an update is requested // in depth-first post-order. @@ -481,14 +499,17 @@ func (py *Python) GenerateRules(args language.GenerateArgs) language.GenerateRes } for _, pyTestTarget := range pyTestTargets { - if conftest != nil { - conftestModule := Module{Name: importSpecFromSrc(pythonProjectRoot, args.Rel, conftestFilename).Imp} - if pyTestTarget.annotations.includePytestConftest == nil { - // unset; default behavior - pyTestTarget.addModuleDependency(conftestModule) - } else if *pyTestTarget.annotations.includePytestConftest { - // set; add if true, do not add if false - pyTestTarget.addModuleDependency(conftestModule) + shouldAddConftest := pyTestTarget.annotations.includePytestConftest == nil || + *pyTestTarget.annotations.includePytestConftest + + if shouldAddConftest { + for _, conftestPkg := range findConftestPaths(args.Config.RepoRoot, args.Rel, pythonProjectRoot) { + pyTestTarget.addModuleDependency( + Module{ + Name: importSpecFromSrc(pythonProjectRoot, conftestPkg, conftestFilename).Imp, + Filepath: filepath.Join(conftestPkg, conftestFilename), + }, + ) } } pyTest := pyTestTarget.build() diff --git a/gazelle/python/testdata/simple_test_with_conftest/bar/BUILD.out b/gazelle/python/testdata/simple_test_with_conftest/bar/BUILD.out index 4a1204e989..9b500d4733 100644 --- a/gazelle/python/testdata/simple_test_with_conftest/bar/BUILD.out +++ b/gazelle/python/testdata/simple_test_with_conftest/bar/BUILD.out @@ -23,5 +23,6 @@ py_test( deps = [ ":bar", ":conftest", + "//:conftest", ], ) diff --git a/gazelle/python/testdata/simple_test_with_conftest_sibling_imports_disabled/bar/BUILD.out b/gazelle/python/testdata/simple_test_with_conftest_sibling_imports_disabled/bar/BUILD.out index ef8591f199..25a2d39672 100644 --- a/gazelle/python/testdata/simple_test_with_conftest_sibling_imports_disabled/bar/BUILD.out +++ b/gazelle/python/testdata/simple_test_with_conftest_sibling_imports_disabled/bar/BUILD.out @@ -22,6 +22,7 @@ py_test( main = "__test__.py", deps = [ ":conftest", + "//:conftest", "//:simple_test_with_conftest_sibling_imports_disabled", ], ) From 5fe50fbf085bd404f6124eda85c0952c41b0c370 Mon Sep 17 00:00:00 2001 From: Ted Kaplan Date: Wed, 14 Jan 2026 21:31:27 -0800 Subject: [PATCH 582/922] fix(pipstar): Handle dep appearing in extra both conditionally and unconditionally (#3513) Fixes #3511 --- python/private/pypi/pep508_deps.bzl | 6 +++++- tests/pypi/pep508/deps_tests.bzl | 18 ++++++++++++++++++ 2 files changed, 23 insertions(+), 1 deletion(-) diff --git a/python/private/pypi/pep508_deps.bzl b/python/private/pypi/pep508_deps.bzl index e73f747bed..cdb449bdc2 100644 --- a/python/private/pypi/pep508_deps.bzl +++ b/python/private/pypi/pep508_deps.bzl @@ -156,6 +156,7 @@ def _add_reqs(deps, deps_select, dep, reqs, *, extras): return markers = {} + found_unconditional = False for req in reqs: for x in extras: m = evaluate(req.marker, env = {"extra": x}, strict = False) @@ -163,10 +164,13 @@ def _add_reqs(deps, deps_select, dep, reqs, *, extras): continue elif m == True: _add(deps, deps_select, dep) + found_unconditional = True break else: markers[m] = None continue + if found_unconditional: + break - if markers: + if markers and not found_unconditional: _add(deps, deps_select, dep, sorted(markers)) diff --git a/tests/pypi/pep508/deps_tests.bzl b/tests/pypi/pep508/deps_tests.bzl index aaa3b2f7dd..679ba58396 100644 --- a/tests/pypi/pep508/deps_tests.bzl +++ b/tests/pypi/pep508/deps_tests.bzl @@ -161,6 +161,24 @@ def test_all_markers_are_added(env): _tests.append(test_all_markers_are_added) +def test_extra_with_conditional_and_unconditional_markers(env): + requires_dist = [ + "bar", + 'baz!=1.56.0; sys_platform == "darwin" and extra == "client"', + 'baz; extra == "client"', + ] + + got = deps( + "foo", + extras = ["client"], + requires_dist = requires_dist, + ) + + env.expect.that_collection(got.deps).contains_exactly(["bar", "baz"]) + env.expect.that_dict(got.deps_select).contains_exactly({}) + +_tests.append(test_extra_with_conditional_and_unconditional_markers) + def deps_test_suite(name): # buildifier: disable=function-docstring test_suite( name = name, From be12db6afd20281cb91357539c244866865c0126 Mon Sep 17 00:00:00 2001 From: Douglas Thor Date: Fri, 16 Jan 2026 13:57:14 -0800 Subject: [PATCH 583/922] Add missing #3046 reference to 1.8.0 changelog (#3516) I've had to reference this a couple times now and keep forgetting that it's not in the changelog haha. I figured I'd remedy that. --- CHANGELOG.md | 1 + 1 file changed, 1 insertion(+) diff --git a/CHANGELOG.md b/CHANGELOG.md index c9f812f3bd..00cac20717 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -138,6 +138,7 @@ END_UNRELEASED_TEMPLATE ### Fixed * (gazelle) Remove {obj}`py_binary` targets with invalid `srcs`. This includes files that are not generated or regular files. + [#3046](https://github.com/bazel-contrib/rules_python/pull/3046) * (runfiles) Fix incorrect Python runfiles path assumption - the existing implementation assumes that it is always four levels below the runfiles directory, leading to incorrect path checks From 53cdb39023064893739eb68c14adfaaa5a3782ff Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Sat, 17 Jan 2026 16:09:05 -0800 Subject: [PATCH 584/922] chore: remove unused which.bzl helper (#3509) Nothing loads which.bzl, so remove it. --- python/private/which.bzl | 32 -------------------------------- 1 file changed, 32 deletions(-) delete mode 100644 python/private/which.bzl diff --git a/python/private/which.bzl b/python/private/which.bzl deleted file mode 100644 index b0cbddb0e8..0000000000 --- a/python/private/which.bzl +++ /dev/null @@ -1,32 +0,0 @@ -# Copyright 2023 The Bazel Authors. All rights reserved. -# -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -"""Wrapper for repository which call""" - -_binary_not_found_msg = "Unable to find the binary '{binary_name}'. Please update your PATH to include '{binary_name}'." - -def which_with_fail(binary_name, rctx): - """Tests to see if a binary exists, and otherwise fails with a message. - - Args: - binary_name: name of the binary to find. - rctx: repository context. - - Returns: - rctx.Path for the binary. - """ - binary = rctx.which(binary_name) - if binary == None: - fail(_binary_not_found_msg.format(binary_name = binary_name)) - return binary From 1169efd5f255d5df53db7e066157519c85777402 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Sat, 17 Jan 2026 16:09:42 -0800 Subject: [PATCH 585/922] chore: remove py_proto_library from bzlmod example (#3508) With the py_proto_library rules removed from rules_python, rules_python is no longer the appropriate place to have examples for using the py_proto_library rules. Having such example also causes some maintenance burden, as apparently it incurs a nodejs dependency, which fails under newer bazel versions. Fixes https://github.com/bazel-contrib/rules_python/issues/3362 --- examples/bzlmod/.bazelignore | 1 - examples/bzlmod/MODULE.bazel | 9 ----- examples/bzlmod/py_proto_library/BUILD.bazel | 35 ------------------- .../example.com/another_proto/BUILD.bazel | 16 --------- .../example.com/another_proto/message.proto | 10 ------ .../example.com/proto/BUILD.bazel | 17 --------- .../example.com/proto/pricetag.proto | 11 ------ .../py_proto_library/foo_external/BUILD.bazel | 22 ------------ .../foo_external/MODULE.bazel | 7 ---- .../py_proto_library/foo_external/WORKSPACE | 0 .../foo_external/nested/foo/my_proto.proto | 6 ---- .../foo_external/py_binary_with_proto.py | 6 ---- .../bzlmod/py_proto_library/message_test.py | 16 --------- examples/bzlmod/py_proto_library/test.py | 21 ----------- 14 files changed, 177 deletions(-) delete mode 100644 examples/bzlmod/py_proto_library/BUILD.bazel delete mode 100644 examples/bzlmod/py_proto_library/example.com/another_proto/BUILD.bazel delete mode 100644 examples/bzlmod/py_proto_library/example.com/another_proto/message.proto delete mode 100644 examples/bzlmod/py_proto_library/example.com/proto/BUILD.bazel delete mode 100644 examples/bzlmod/py_proto_library/example.com/proto/pricetag.proto delete mode 100644 examples/bzlmod/py_proto_library/foo_external/BUILD.bazel delete mode 100644 examples/bzlmod/py_proto_library/foo_external/MODULE.bazel delete mode 100644 examples/bzlmod/py_proto_library/foo_external/WORKSPACE delete mode 100644 examples/bzlmod/py_proto_library/foo_external/nested/foo/my_proto.proto delete mode 100644 examples/bzlmod/py_proto_library/foo_external/py_binary_with_proto.py delete mode 100644 examples/bzlmod/py_proto_library/message_test.py delete mode 100644 examples/bzlmod/py_proto_library/test.py diff --git a/examples/bzlmod/.bazelignore b/examples/bzlmod/.bazelignore index 536ded93a6..a59e740c96 100644 --- a/examples/bzlmod/.bazelignore +++ b/examples/bzlmod/.bazelignore @@ -1,3 +1,2 @@ other_module -py_proto_library/foo_external vendor diff --git a/examples/bzlmod/MODULE.bazel b/examples/bzlmod/MODULE.bazel index 14c490cb0d..5c71d32421 100644 --- a/examples/bzlmod/MODULE.bazel +++ b/examples/bzlmod/MODULE.bazel @@ -12,9 +12,6 @@ local_path_override( path = "../..", ) -# (py_proto_library specific) Add the protobuf library for well-known types (e.g. `Any`, `Timestamp`, etc) -bazel_dep(name = "protobuf", version = "27.0", repo_name = "com_google_protobuf") - # Only needed to make rules_python's CI happy. rules_java 8.3.0+ is needed so # that --java_runtime_version=remotejdk_11 works with Bazel 8. bazel_dep(name = "rules_java", version = "8.16.1") @@ -291,11 +288,5 @@ local_path_override( path = "other_module", ) -bazel_dep(name = "foo_external", version = "") -local_path_override( - module_name = "foo_external", - path = "py_proto_library/foo_external", -) - # example test dependencies bazel_dep(name = "rules_shell", version = "0.3.0", dev_dependency = True) diff --git a/examples/bzlmod/py_proto_library/BUILD.bazel b/examples/bzlmod/py_proto_library/BUILD.bazel deleted file mode 100644 index daea410365..0000000000 --- a/examples/bzlmod/py_proto_library/BUILD.bazel +++ /dev/null @@ -1,35 +0,0 @@ -load("@bazel_skylib//rules:native_binary.bzl", "native_test") -load("@rules_python//python:py_test.bzl", "py_test") - -py_test( - name = "pricetag_test", - srcs = ["test.py"], - main = "test.py", - deps = [ - "//py_proto_library/example.com/proto:pricetag_py_pb2", - ], -) - -py_test( - name = "message_test", - srcs = ["message_test.py"], - deps = [ - "//py_proto_library/example.com/another_proto:message_py_pb2", - ], -) - -# Regression test for https://github.com/bazel-contrib/rules_python/issues/2515 -# -# This test fails before protobuf 30.0 release -# when ran with --legacy_external_runfiles=False (default in Bazel 8.0.0). -native_test( - name = "external_import_test", - src = "@foo_external//:py_binary_with_proto", - tags = ["manual"], # TODO: reenable when com_google_protobuf is upgraded - # Incompatible with Windows: native_test wrapping a py_binary doesn't work - # on Windows. - target_compatible_with = select({ - "@platforms//os:windows": ["@platforms//:incompatible"], - "//conditions:default": [], - }), -) diff --git a/examples/bzlmod/py_proto_library/example.com/another_proto/BUILD.bazel b/examples/bzlmod/py_proto_library/example.com/another_proto/BUILD.bazel deleted file mode 100644 index 29f08c21ca..0000000000 --- a/examples/bzlmod/py_proto_library/example.com/another_proto/BUILD.bazel +++ /dev/null @@ -1,16 +0,0 @@ -load("@com_google_protobuf//bazel:proto_library.bzl", "proto_library") -load("@rules_python//python:proto.bzl", "py_proto_library") - -py_proto_library( - name = "message_py_pb2", - visibility = ["//visibility:public"], - deps = [":message_proto"], -) - -proto_library( - name = "message_proto", - srcs = ["message.proto"], - # https://bazel.build/reference/be/protocol-buffer#proto_library.strip_import_prefix - strip_import_prefix = "/py_proto_library/example.com", - deps = ["//py_proto_library/example.com/proto:pricetag_proto"], -) diff --git a/examples/bzlmod/py_proto_library/example.com/another_proto/message.proto b/examples/bzlmod/py_proto_library/example.com/another_proto/message.proto deleted file mode 100644 index 6e7dcc5793..0000000000 --- a/examples/bzlmod/py_proto_library/example.com/another_proto/message.proto +++ /dev/null @@ -1,10 +0,0 @@ -syntax = "proto3"; - -package rules_python; - -import "proto/pricetag.proto"; - -message TestMessage { - uint32 index = 1; - PriceTag pricetag = 2; -} diff --git a/examples/bzlmod/py_proto_library/example.com/proto/BUILD.bazel b/examples/bzlmod/py_proto_library/example.com/proto/BUILD.bazel deleted file mode 100644 index 1f8e8f2818..0000000000 --- a/examples/bzlmod/py_proto_library/example.com/proto/BUILD.bazel +++ /dev/null @@ -1,17 +0,0 @@ -load("@com_google_protobuf//bazel:proto_library.bzl", "proto_library") -load("@rules_python//python:proto.bzl", "py_proto_library") - -py_proto_library( - name = "pricetag_py_pb2", - visibility = ["//visibility:public"], - deps = [":pricetag_proto"], -) - -proto_library( - name = "pricetag_proto", - srcs = ["pricetag.proto"], - # https://bazel.build/reference/be/protocol-buffer#proto_library.strip_import_prefix - strip_import_prefix = "/py_proto_library/example.com", - visibility = ["//visibility:public"], - deps = ["@com_google_protobuf//:any_proto"], -) diff --git a/examples/bzlmod/py_proto_library/example.com/proto/pricetag.proto b/examples/bzlmod/py_proto_library/example.com/proto/pricetag.proto deleted file mode 100644 index 3fa68de84b..0000000000 --- a/examples/bzlmod/py_proto_library/example.com/proto/pricetag.proto +++ /dev/null @@ -1,11 +0,0 @@ -syntax = "proto3"; - -import "google/protobuf/any.proto"; - -package rules_python; - -message PriceTag { - string name = 2; - double cost = 1; - google.protobuf.Any metadata = 3; -} diff --git a/examples/bzlmod/py_proto_library/foo_external/BUILD.bazel b/examples/bzlmod/py_proto_library/foo_external/BUILD.bazel deleted file mode 100644 index 183a3c28d2..0000000000 --- a/examples/bzlmod/py_proto_library/foo_external/BUILD.bazel +++ /dev/null @@ -1,22 +0,0 @@ -load("@com_google_protobuf//bazel:proto_library.bzl", "proto_library") -load("@com_google_protobuf//bazel:py_proto_library.bzl", "py_proto_library") -load("@rules_python//python:py_binary.bzl", "py_binary") - -package(default_visibility = ["//visibility:public"]) - -proto_library( - name = "proto_lib", - srcs = ["nested/foo/my_proto.proto"], - strip_import_prefix = "/nested/foo", -) - -py_proto_library( - name = "a_proto", - deps = [":proto_lib"], -) - -py_binary( - name = "py_binary_with_proto", - srcs = ["py_binary_with_proto.py"], - deps = [":a_proto"], -) diff --git a/examples/bzlmod/py_proto_library/foo_external/MODULE.bazel b/examples/bzlmod/py_proto_library/foo_external/MODULE.bazel deleted file mode 100644 index aca6f98eab..0000000000 --- a/examples/bzlmod/py_proto_library/foo_external/MODULE.bazel +++ /dev/null @@ -1,7 +0,0 @@ -module( - name = "foo_external", - version = "0.0.1", -) - -bazel_dep(name = "rules_python", version = "1.0.0") -bazel_dep(name = "protobuf", version = "28.2", repo_name = "com_google_protobuf") diff --git a/examples/bzlmod/py_proto_library/foo_external/WORKSPACE b/examples/bzlmod/py_proto_library/foo_external/WORKSPACE deleted file mode 100644 index e69de29bb2..0000000000 diff --git a/examples/bzlmod/py_proto_library/foo_external/nested/foo/my_proto.proto b/examples/bzlmod/py_proto_library/foo_external/nested/foo/my_proto.proto deleted file mode 100644 index 7b8440cbed..0000000000 --- a/examples/bzlmod/py_proto_library/foo_external/nested/foo/my_proto.proto +++ /dev/null @@ -1,6 +0,0 @@ -syntax = "proto3"; - -package my_proto; - -message MyMessage { -} diff --git a/examples/bzlmod/py_proto_library/foo_external/py_binary_with_proto.py b/examples/bzlmod/py_proto_library/foo_external/py_binary_with_proto.py deleted file mode 100644 index 67e798bb8f..0000000000 --- a/examples/bzlmod/py_proto_library/foo_external/py_binary_with_proto.py +++ /dev/null @@ -1,6 +0,0 @@ -import sys - -if __name__ == "__main__": - import my_proto_pb2 - - sys.exit(0) diff --git a/examples/bzlmod/py_proto_library/message_test.py b/examples/bzlmod/py_proto_library/message_test.py deleted file mode 100644 index b1a6942a54..0000000000 --- a/examples/bzlmod/py_proto_library/message_test.py +++ /dev/null @@ -1,16 +0,0 @@ -import sys -import unittest - -from another_proto import message_pb2 - - -class TestCase(unittest.TestCase): - def test_message(self): - got = message_pb2.TestMessage( - index=5, - ) - self.assertIsNotNone(got) - - -if __name__ == "__main__": - sys.exit(unittest.main()) diff --git a/examples/bzlmod/py_proto_library/test.py b/examples/bzlmod/py_proto_library/test.py deleted file mode 100644 index 24ab8ddc70..0000000000 --- a/examples/bzlmod/py_proto_library/test.py +++ /dev/null @@ -1,21 +0,0 @@ -import json -import unittest - -from proto import pricetag_pb2 - - -class TestCase(unittest.TestCase): - def test_pricetag(self): - got = pricetag_pb2.PriceTag( - name="dollar", - cost=5.00, - ) - - metadata = {"description": "some text..."} - got.metadata.value = json.dumps(metadata).encode("utf-8") - - self.assertIsNotNone(got) - - -if __name__ == "__main__": - unittest.main() From c6acbfcef924c576d55ad7fa9ee82625763504dd Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Sat, 17 Jan 2026 16:11:31 -0800 Subject: [PATCH 586/922] chore: remove defunct runtime distinction logic (#3506) This removes some defunct logic that would distinguish how a runtime was found (using toolchain resolution or the `--python_top` flag). With the python_top flag gone, there's only one way to get the runtime. --- python/private/py_executable.bzl | 38 +++++++++----------------------- 1 file changed, 11 insertions(+), 27 deletions(-) diff --git a/python/private/py_executable.bzl b/python/private/py_executable.bzl index 5bac6247ef..8b28b4ffae 100644 --- a/python/private/py_executable.bzl +++ b/python/private/py_executable.bzl @@ -1153,22 +1153,13 @@ def _get_runtime_details(ctx): # Bazel has --python_path. This flag has a computed default of "python" when # its actual default is null (see - # BazelPythonConfiguration.java#getPythonPath). This flag is only used if - # toolchains are not enabled and `--python_top` isn't set. Note that Google - # used to have a variant of this named --python_binary, but it has since - # been removed. - # # TOOD(bazelbuild/bazel#7901): Remove this once --python_path flag is removed. flag_interpreter_path = read_possibly_native_flag(ctx, "python_path") if not flag_interpreter_path.startswith("python") and not paths.is_absolute(flag_interpreter_path): fail("'python_path' must be an absolute path or a name to be resolved from the system PATH (e.g., 'python', 'python3').") - toolchain_runtime, effective_runtime = _maybe_get_runtime_from_ctx(ctx) - if not effective_runtime: - # Clear these just in case - toolchain_runtime = None - effective_runtime = None + effective_runtime = _maybe_get_runtime_from_ctx(ctx) if effective_runtime: direct = [] # List of files @@ -1193,16 +1184,9 @@ def _get_runtime_details(ctx): ) return struct( - # Optional PyRuntimeInfo: The runtime found from toolchain resolution. - # This may be None because, within Google, toolchain resolution isn't - # yet enabled. - toolchain_runtime = toolchain_runtime, - # Optional PyRuntimeInfo: The runtime that should be used. When - # toolchain resolution is enabled, this is the same as - # `toolchain_resolution`. Otherwise, this probably came from the - # `_python_top` attribute that the Google implementation still uses. - # This is separate from `toolchain_runtime` because toolchain_runtime - # is propagated as a provider, while non-toolchain runtimes are not. + # Optional PyRuntimeInfo: The runtime that should be used. + # If None, it's probably Windows using the legacy auto-detecting toolchain + # that acts as if no toolchain was found. effective_runtime = effective_runtime, # str; Path to the Python interpreter to use for running the executable # itself (not the bootstrap script). Either an absolute path (which @@ -1219,7 +1203,7 @@ def _maybe_get_runtime_from_ctx(ctx): """Finds the PyRuntimeInfo from the toolchain or attribute, if available. Returns: - 2-tuple of toolchain_runtime, effective_runtime + A PyRuntimeInfo provider, or None. """ toolchain = ctx.toolchains[TOOLCHAIN_TYPE] @@ -1236,13 +1220,13 @@ def _maybe_get_runtime_from_ctx(ctx): # TODO(#7844): Remove this hack when the autodetecting toolchain has a # Windows implementation. if py3_runtime.interpreter_path == "/_magic_pyruntime_sentinel_do_not_use": - return None, None + return None if py3_runtime.python_version != "PY3": fail("Python toolchain py3_runtime must be python_version=PY3, got {}".format( py3_runtime.python_version, )) - return py3_runtime, py3_runtime + return py3_runtime def _get_base_runfiles_for_binary( ctx, @@ -1701,10 +1685,10 @@ def _create_providers( ), ] - # TODO(b/265840007): Make this non-conditional once Google enables - # --incompatible_use_python_toolchains. - if runtime_details.toolchain_runtime: - py_runtime_info = runtime_details.toolchain_runtime + # TODO - The effective runtime can be None for Windows + auto detecting toolchain. + # This can be removed once that's fixed; see maybe_get_runtime_from_ctx(). + if runtime_details.effective_runtime: + py_runtime_info = runtime_details.effective_runtime providers.append(py_runtime_info) # Re-add the builtin PyRuntimeInfo for compatibility to make From 779623cef5cf17cb05df6204df13557999f4f24c Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Sat, 17 Jan 2026 16:29:25 -0800 Subject: [PATCH 587/922] chore: make builtin build_python_zip flag optional for tests (bazel 10 compatibility) (#3507) Starting in bazel 10, the build_python_zip flag is removed. This means using the flag in e.g. transitions no longer works, which breaks some of the tests that set the flag for testing. To fix, don't set the flags or add them to transitions when running under bazel 10. Fixes https://github.com/bazel-contrib/rules_python/issues/3496 --- python/private/internal_config_repo.bzl | 2 ++ tests/base_rules/py_executable_base_tests.bzl | 8 +++----- .../transition/multi_version_tests.bzl | 5 ++--- tests/support/py_reconfig.bzl | 11 ++++++++--- tests/support/support.bzl | 8 ++++++++ 5 files changed, 23 insertions(+), 11 deletions(-) diff --git a/python/private/internal_config_repo.bzl b/python/private/internal_config_repo.bzl index 9fc301cd2e..7c59355c95 100644 --- a/python/private/internal_config_repo.bzl +++ b/python/private/internal_config_repo.bzl @@ -35,6 +35,7 @@ config = struct( enable_deprecation_warnings = {enable_deprecation_warnings}, bazel_8_or_later = {bazel_8_or_later}, bazel_9_or_later = {bazel_9_or_later}, + bazel_10_or_later = {bazel_10_or_later}, BuiltinPyInfo = getattr(getattr(native, "legacy_globals", None), "PyInfo", {builtin_py_info_symbol}), BuiltinPyRuntimeInfo = getattr(getattr(native, "legacy_globals", None), "PyRuntimeInfo", {builtin_py_runtime_info_symbol}), BuiltinPyCcLinkParamsProvider = getattr(getattr(native, "legacy_globals", None), "PyCcLinkParamsProvider", {builtin_py_cc_link_params_provider}), @@ -124,6 +125,7 @@ def _internal_config_repo_impl(rctx): builtin_py_cc_link_params_provider = builtin_py_cc_link_params_provider, bazel_8_or_later = str(bazel_major_version >= 8), bazel_9_or_later = str(bazel_major_version >= 9), + bazel_10_or_later = str(bazel_major_version > 9), )) shim_content = _PY_INTERNAL_SHIM diff --git a/tests/base_rules/py_executable_base_tests.bzl b/tests/base_rules/py_executable_base_tests.bzl index 2af5406ced..83a470cdc5 100644 --- a/tests/base_rules/py_executable_base_tests.bzl +++ b/tests/base_rules/py_executable_base_tests.bzl @@ -26,7 +26,7 @@ load("//python/private:reexports.bzl", "BuiltinPyRuntimeInfo") # buildifier: di load("//tests/base_rules:base_tests.bzl", "create_base_tests") load("//tests/base_rules:util.bzl", "WINDOWS_ATTR", pt_util = "util") load("//tests/support:py_executable_info_subject.bzl", "PyExecutableInfoSubject") -load("//tests/support:support.bzl", "CC_TOOLCHAIN", "CROSSTOOL_TOP") +load("//tests/support:support.bzl", "CC_TOOLCHAIN", "CROSSTOOL_TOP", "maybe_builtin_build_python_zip") load("//tests/support/platforms:platforms.bzl", "platform_targets") _tests = [] @@ -49,14 +49,13 @@ def _test_basic_windows(name, config): # platforms. # Pass value to both native and starlark versions of the flag until # the native one is removed. - "//command_line_option:build_python_zip": "true", labels.BUILD_PYTHON_ZIP: True, "//command_line_option:cpu": "windows_x86_64", "//command_line_option:crosstool_top": CROSSTOOL_TOP, "//command_line_option:extra_execution_platforms": [platform_targets.WINDOWS_X86_64], "//command_line_option:extra_toolchains": [CC_TOOLCHAIN], "//command_line_option:platforms": [platform_targets.WINDOWS_X86_64], - }, + } | maybe_builtin_build_python_zip("true"), attr_values = {}, ) @@ -95,14 +94,13 @@ def _test_basic_zip(name, config): # platforms. # Pass value to both native and starlark versions of the flag until # the native one is removed. - "//command_line_option:build_python_zip": "true", labels.BUILD_PYTHON_ZIP: True, "//command_line_option:cpu": "linux_x86_64", "//command_line_option:crosstool_top": CROSSTOOL_TOP, "//command_line_option:extra_execution_platforms": [platform_targets.LINUX_X86_64], "//command_line_option:extra_toolchains": [CC_TOOLCHAIN], "//command_line_option:platforms": [platform_targets.LINUX_X86_64], - }, + } | maybe_builtin_build_python_zip("true"), attr_values = {"target_compatible_with": target_compatible_with}, ) diff --git a/tests/config_settings/transition/multi_version_tests.bzl b/tests/config_settings/transition/multi_version_tests.bzl index 05f010562c..8a1d8a6a63 100644 --- a/tests/config_settings/transition/multi_version_tests.bzl +++ b/tests/config_settings/transition/multi_version_tests.bzl @@ -22,7 +22,7 @@ load("//python:py_info.bzl", "PyInfo") load("//python:py_test.bzl", "py_test") load("//python/private:common_labels.bzl", "labels") # buildifier: disable=bzl-visibility load("//python/private:reexports.bzl", "BuiltinPyInfo") # buildifier: disable=bzl-visibility -load("//tests/support:support.bzl", "CC_TOOLCHAIN") +load("//tests/support:support.bzl", "CC_TOOLCHAIN", "maybe_builtin_build_python_zip") load("//tests/support/platforms:platforms.bzl", "platform_targets") # NOTE @aignas 2024-06-04: we are using here something that is registered in the MODULE.Bazel @@ -92,11 +92,10 @@ def _setup_py_binary_windows(name, *, impl, build_python_zip): target = name + "_subject", impl = impl, config_settings = { - "//command_line_option:build_python_zip": str(build_python_zip), labels.BUILD_PYTHON_ZIP: build_python_zip, "//command_line_option:extra_toolchains": CC_TOOLCHAIN, "//command_line_option:platforms": str(platform_targets.WINDOWS_X86_64), - }, + } | maybe_builtin_build_python_zip(str(build_python_zip)), ) def _test_py_binary_windows_build_python_zip_false(name): diff --git a/tests/support/py_reconfig.bzl b/tests/support/py_reconfig.bzl index efcb0e7a38..d0cb968466 100644 --- a/tests/support/py_reconfig.bzl +++ b/tests/support/py_reconfig.bzl @@ -31,7 +31,9 @@ def _perform_transition_impl(input_settings, attr, base_impl): settings.update(base_impl(input_settings, attr)) settings[labels.VISIBLE_FOR_TESTING] = True - settings["//command_line_option:build_python_zip"] = str(attr.build_python_zip) + + if _BUILTIN_BUILD_PYTHON_ZIP: + settings["//command_line_option:build_python_zip"] = str(attr.build_python_zip) settings[labels.BUILD_PYTHON_ZIP] = attr.build_python_zip if attr.bootstrap_impl: settings[labels.BOOTSTRAP_IMPL] = attr.bootstrap_impl @@ -49,6 +51,10 @@ def _perform_transition_impl(input_settings, attr, base_impl): settings[str(key)] = value return settings +_BUILTIN_BUILD_PYTHON_ZIP = [] if config.bazel_10_or_later else [ + "//command_line_option:build_python_zip", +] + _RECONFIG_INPUTS = [ "//command_line_option:extra_toolchains", CUSTOM_RUNTIME, @@ -59,10 +65,9 @@ _RECONFIG_INPUTS = [ labels.VENVS_USE_DECLARE_SYMLINK, ] _RECONFIG_OUTPUTS = _RECONFIG_INPUTS + [ - "//command_line_option:build_python_zip", labels.BUILD_PYTHON_ZIP, labels.VISIBLE_FOR_TESTING, -] +] + _BUILTIN_BUILD_PYTHON_ZIP _RECONFIG_INHERITED_OUTPUTS = [v for v in _RECONFIG_OUTPUTS if v in _RECONFIG_INPUTS] _RECONFIG_ATTRS = { diff --git a/tests/support/support.bzl b/tests/support/support.bzl index c6997e35d1..b767ec2714 100644 --- a/tests/support/support.bzl +++ b/tests/support/support.bzl @@ -19,6 +19,7 @@ # rules_testing or as config_setting values, which don't support Label in some # places. +load("@rules_python_internal//:rules_python_config.bzl", "config") load("//python/private:bzlmod_enabled.bzl", "BZLMOD_ENABLED") # buildifier: disable=bzl-visibility PY_TOOLCHAINS = str(Label("//tests/support/py_toolchains:all")) @@ -43,3 +44,10 @@ NOT_WINDOWS = select({ "@platforms//os:windows": ["@platforms//:incompatible"], "//conditions:default": [], }) + +def maybe_builtin_build_python_zip(value): + settings = {} + if not config.bazel_10_or_later: + settings["//command_line_option:build_python_zip"] = value + + return settings From 9571e2dc60fc2e6005141593547e7711c6d8a152 Mon Sep 17 00:00:00 2001 From: Shayan Hoshyari <108962133+shayanhoshyari@users.noreply.github.com> Date: Sun, 18 Jan 2026 22:37:50 -0800 Subject: [PATCH 588/922] chore (py_internal): Remove roundtrip of putting py_internal in the config repo (#3522) I was reading on `py_internal` and found this opportunity for simplification. When there was support for Bazel `< 7` `py_internal` was in `@rules_python_internal`, because it had to be set to None depending on Bazel version. https://github.com/bazel-contrib/rules_python/pull/3282 removed Bazel `< 7` support shims, and made `py_internal` not depend on Bazel version. So now there no need to put it in `@rules_python_internal`. --------- Co-authored-by: Shayan Hoshyari Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> --- python/private/BUILD.bazel | 2 +- python/private/internal_config_repo.bzl | 18 ------------------ python/private/py_internal.bzl | 11 ++++++----- tools/build_defs/python/private/BUILD.bazel | 2 +- 4 files changed, 8 insertions(+), 25 deletions(-) diff --git a/python/private/BUILD.bazel b/python/private/BUILD.bazel index efc1dd7319..ac97ef9b50 100644 --- a/python/private/BUILD.bazel +++ b/python/private/BUILD.bazel @@ -454,7 +454,7 @@ bzl_library( bzl_library( name = "py_internal_bzl", srcs = ["py_internal.bzl"], - deps = ["@rules_python_internal//:py_internal_bzl"], + deps = ["//tools/build_defs/python/private:py_internal_renamed_bzl"], ) bzl_library( diff --git a/python/private/internal_config_repo.bzl b/python/private/internal_config_repo.bzl index 7c59355c95..5b28d69ac9 100644 --- a/python/private/internal_config_repo.bzl +++ b/python/private/internal_config_repo.bzl @@ -42,13 +42,6 @@ config = struct( ) """ -# The py_internal symbol is only accessible from within @rules_python, so we have to -# load it from there and re-export it so that rules_python can later load it. -_PY_INTERNAL_SHIM = """ -load("@rules_python//tools/build_defs/python/private:py_internal_renamed.bzl", "py_internal_renamed") -py_internal_impl = py_internal_renamed -""" - ROOT_BUILD_TEMPLATE = """ load("@bazel_skylib//:bzl_library.bzl", "bzl_library") @@ -63,12 +56,6 @@ bzl_library( srcs = ["rules_python_config.bzl"] ) -bzl_library( - name = "py_internal_bzl", - srcs = ["py_internal.bzl"], - deps = [{py_internal_dep}], -) - bzl_library( name = "extra_transition_settings_bzl", srcs = ["extra_transition_settings.bzl"], @@ -128,14 +115,9 @@ def _internal_config_repo_impl(rctx): bazel_10_or_later = str(bazel_major_version > 9), )) - shim_content = _PY_INTERNAL_SHIM - py_internal_dep = '"@rules_python//tools/build_defs/python/private:py_internal_renamed_bzl"' - rctx.file("BUILD", ROOT_BUILD_TEMPLATE.format( - py_internal_dep = py_internal_dep, visibility = "@rules_python//:__subpackages__", )) - rctx.file("py_internal.bzl", shim_content) rctx.file( "extra_transition_settings.bzl", diff --git a/python/private/py_internal.bzl b/python/private/py_internal.bzl index 429637253f..000501d26c 100644 --- a/python/private/py_internal.bzl +++ b/python/private/py_internal.bzl @@ -18,9 +18,10 @@ Re-exports the restricted-use py_internal helper under its original name. These may change at any time and are closely coupled to the rule implementation. """ -# The py_internal global is only available in Bazel 7+, so loading of it -# must go through a repo rule with Bazel version detection logic. -load("@rules_python_internal//:py_internal.bzl", "py_internal_impl") +# The native `py_internal` object is only visible to Starlark files under +# `tools/build_defs/python`. To access it from `//python/private`, we use an +# indirection through `//tools/build_defs/python/private/py_internal_renamed.bzl`, +# which re-exports it under a different name. +load("//tools/build_defs/python/private:py_internal_renamed.bzl", "py_internal_renamed") # buildifier: disable=bzl-visibility -# NOTE: This is None prior to Bazel 7, as set by @rules_python_internal -py_internal = py_internal_impl +py_internal = py_internal_renamed diff --git a/tools/build_defs/python/private/BUILD.bazel b/tools/build_defs/python/private/BUILD.bazel index 0a7f308f02..746545640d 100644 --- a/tools/build_defs/python/private/BUILD.bazel +++ b/tools/build_defs/python/private/BUILD.bazel @@ -23,5 +23,5 @@ filegroup( bzl_library( name = "py_internal_renamed_bzl", srcs = ["py_internal_renamed.bzl"], - visibility = ["@rules_python_internal//:__subpackages__"], + visibility = ["//python/private:__pkg__"], ) From e7e659afee9b9636480428bc0c6b4928a6b90c2b Mon Sep 17 00:00:00 2001 From: Shayan Hoshyari <108962133+shayanhoshyari@users.noreply.github.com> Date: Tue, 20 Jan 2026 00:11:59 -0800 Subject: [PATCH 589/922] chore: remove mention of py_proto_library from readme (#3525) py_proto_library is meant to be imported from protobuf. #3508 removed the relevant examples. This PR also removes the mention in main readme. cc: @rickeylev since you authored #3508 --- README.md | 2 +- docs/index.md | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/README.md b/README.md index a7399fff9c..e79c4a9673 100644 --- a/README.md +++ b/README.md @@ -5,7 +5,7 @@ ## Overview This repository is the home of the core Python rules -- `py_library`, -`py_binary`, `py_test`, `py_proto_library`, and related symbols that provide the basis for Python +`py_binary`, `py_test`, and related symbols that provide the basis for Python support in Bazel. It also contains package installation rules for integrating with PyPI and other indices. Documentation for rules_python is at and in the diff --git a/docs/index.md b/docs/index.md index 7f03681b76..0204f38ec7 100644 --- a/docs/index.md +++ b/docs/index.md @@ -5,7 +5,7 @@ :::{topic} Core rules The core Python rules -- `py_library`, `py_binary`, `py_test`, -`py_proto_library`, and related symbols that provide the basis for Python +and related symbols that provide the basis for Python support in Bazel. When using Bazel 6 (or earlier), the core rules are bundled into the Bazel binary, and the symbols From 1ac5a19c30c3a9318972d47e32da59f56ca9d130 Mon Sep 17 00:00:00 2001 From: Laurenz Date: Tue, 20 Jan 2026 09:28:53 +0100 Subject: [PATCH 590/922] fix: Quote all files if original RECORD had all files quoted (#3515) When patching a single file in Pytorch, repack_whl.py will print over 20k lines of RECORD.patch. The reason is that the original RECORD has all filenames quoted for some reason, but the automatically generated one quotes only when required (such as commas in file names, see https://github.com/bazel-contrib/rules_python/pull/2269). This PR refactors the wheelmaker.py to still use the csv.writer to auto-quote, but adds an additional detection for forced quote usage. This makes the RECORD.patch match intuitive expectations of what could change. There are some relevant existing tests in examples/wheel/wheel_test.py and tests/whl_filegroup/extract_wheel_files_test.py --------- Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> Co-authored-by: Ignas Anikevicius <240938+aignas@users.noreply.github.com> --- CHANGELOG.md | 2 + python/private/pypi/BUILD.bazel | 7 +++ python/private/pypi/repack_whl.py | 18 +++++++- tests/pypi/repack_whl/BUILD.bazel | 8 ++++ tests/pypi/repack_whl/repack_whl_test.py | 37 +++++++++++++++ tests/tools/wheelmaker_test.py | 37 +++++++++++++++ tools/wheelmaker.py | 57 ++++++++++++------------ 7 files changed, 136 insertions(+), 30 deletions(-) create mode 100644 tests/pypi/repack_whl/BUILD.bazel create mode 100644 tests/pypi/repack_whl/repack_whl_test.py diff --git a/CHANGELOG.md b/CHANGELOG.md index 00cac20717..d7a0ab3534 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -61,6 +61,8 @@ END_UNRELEASED_TEMPLATE * (binaries/tests) The `PYTHONBREAKPOINT` environment variable is automatically inherited * (binaries/tests) The {obj}`stamp` attribute now transitions the Bazel builtin {obj}`--stamp` flag. +* (pypi) Now the RECORD file patches will follow the quoted or unquoted filenames convention + in order to make `pytorch` and friends easier to patch. {#v0-0-0-fixed} ### Fixed diff --git a/python/private/pypi/BUILD.bazel b/python/private/pypi/BUILD.bazel index b46fd58d3c..4fea3684de 100644 --- a/python/private/pypi/BUILD.bazel +++ b/python/private/pypi/BUILD.bazel @@ -13,6 +13,7 @@ # limitations under the License. load("@bazel_skylib//:bzl_library.bzl", "bzl_library") +load("//python:py_library.bzl", "py_library") package(default_visibility = ["//:__subpackages__"]) @@ -377,6 +378,12 @@ bzl_library( ], ) +py_library( + name = "repack_whl", + srcs = ["repack_whl.py"], + deps = ["//tools:wheelmaker"], +) + bzl_library( name = "requirements_files_by_platform_bzl", srcs = ["requirements_files_by_platform.bzl"], diff --git a/python/private/pypi/repack_whl.py b/python/private/pypi/repack_whl.py index 519631f272..92d052a81f 100644 --- a/python/private/pypi/repack_whl.py +++ b/python/private/pypi/repack_whl.py @@ -44,6 +44,16 @@ _DISTINFO = "dist-info" +def _has_all_quoted_filenames(record_contents: str) -> bool: + """Check if all filenames in the RECORD are quoted. + + Some wheels (like torch) have all filenames quoted in their RECORD file. + We detect this to preserve the quoting style when repacking. + """ + lines = record_contents.splitlines() + return all(line.startswith('"') for line in lines) + + def _unidiff_output(expected, actual, record): """ Helper function. Returns a string containing the unified diff of two @@ -151,17 +161,21 @@ def main(sys_argv): logging.debug(f"Found dist-info dir: {distinfo_dir}") record_path = distinfo_dir / "RECORD" record_contents = record_path.read_text() if record_path.exists() else "" + quote_files = _has_all_quoted_filenames(record_contents) distribution_prefix = distinfo_dir.with_suffix("").name with _WhlFile( - args.output, mode="w", distribution_prefix=distribution_prefix + args.output, + mode="w", + distribution_prefix=distribution_prefix, + quote_all_filenames=quote_files, ) as out: for p in _files_to_pack(patched_wheel_dir, record_contents): rel_path = p.relative_to(patched_wheel_dir) out.add_file(str(rel_path), p) logging.debug(f"Writing RECORD file") - got_record = out.add_recordfile().decode("utf-8", "surrogateescape") + got_record = out.add_recordfile() if got_record == record_contents: logging.info(f"Created a whl file: {args.output}") diff --git a/tests/pypi/repack_whl/BUILD.bazel b/tests/pypi/repack_whl/BUILD.bazel new file mode 100644 index 0000000000..3f611a2e4f --- /dev/null +++ b/tests/pypi/repack_whl/BUILD.bazel @@ -0,0 +1,8 @@ +load("//python:py_test.bzl", "py_test") + +py_test( + name = "repack_whl_test", + size = "small", + srcs = ["repack_whl_test.py"], + deps = ["//python/private/pypi:repack_whl"], +) diff --git a/tests/pypi/repack_whl/repack_whl_test.py b/tests/pypi/repack_whl/repack_whl_test.py new file mode 100644 index 0000000000..50781cc0e6 --- /dev/null +++ b/tests/pypi/repack_whl/repack_whl_test.py @@ -0,0 +1,37 @@ +import unittest + +from python.private.pypi import repack_whl + + +class HasAllQuotedFilenamesTest(unittest.TestCase): + """Tests for _has_all_quoted_filenames detection logic.""" + + def test_all_quoted(self) -> None: + """Returns True when all lines start with quotes (torch-style).""" + record = """\ +"torch/__init__.py",sha256=abc,123 +"torch/utils.py",sha256=def,456 +"torch-2.0.0.dist-info/WHEEL",sha256=ghi,789 +""" + self.assertTrue(repack_whl._has_all_quoted_filenames(record)) + + def test_none_quoted(self) -> None: + """Returns False when no lines are quoted (standard style).""" + record = """\ +torch/__init__.py,sha256=abc,123 +torch/utils.py,sha256=def,456 +torch-2.0.0.dist-info/WHEEL,sha256=ghi,789 +""" + self.assertFalse(repack_whl._has_all_quoted_filenames(record)) + + def test_mixed_quoting(self) -> None: + """Returns False when only some lines are quoted.""" + record = """\ +"file,with,commas.py",sha256=abc,123 +normal_file.py,sha256=def,456 +""" + self.assertFalse(repack_whl._has_all_quoted_filenames(record)) + + +if __name__ == "__main__": + unittest.main() diff --git a/tests/tools/wheelmaker_test.py b/tests/tools/wheelmaker_test.py index 0efe1c9fbc..288dde720a 100644 --- a/tests/tools/wheelmaker_test.py +++ b/tests/tools/wheelmaker_test.py @@ -1,8 +1,45 @@ +import io import unittest import tools.wheelmaker as wheelmaker +class QuoteAllFilenamesTest(unittest.TestCase): + """Tests for quote_all_filenames behavior in _WhlFile. + + Some wheels (like torch) have all filenames quoted in their RECORD file. + When repacking, we preserve this style to minimize diffs. + """ + + def _make_whl_file(self, quote_all: bool) -> wheelmaker._WhlFile: + """Create a _WhlFile instance for testing.""" + buf = io.BytesIO() + return wheelmaker._WhlFile( + buf, + mode="w", + distribution_prefix="test-1.0.0", + quote_all_filenames=quote_all, + ) + + def test_quote_all_quotes_simple_filenames(self) -> None: + """When quote_all_filenames=True, all filenames are quoted.""" + whl = self._make_whl_file(quote_all=True) + self.assertEqual(whl._quote_filename("foo/bar.py"), '"foo/bar.py"') + + def test_quote_all_false_leaves_simple_filenames_unquoted(self) -> None: + """When quote_all_filenames=False, simple filenames stay unquoted.""" + whl = self._make_whl_file(quote_all=False) + self.assertEqual(whl._quote_filename("foo/bar.py"), "foo/bar.py") + + def test_quote_all_quotes_filenames_with_commas(self) -> None: + """Filenames with commas are always quoted, regardless of quote_all_filenames.""" + whl = self._make_whl_file(quote_all=True) + self.assertEqual(whl._quote_filename("foo,bar/baz.py"), '"foo,bar/baz.py"') + + whl = self._make_whl_file(quote_all=False) + self.assertEqual(whl._quote_filename("foo,bar/baz.py"), '"foo,bar/baz.py"') + + class ArcNameFromTest(unittest.TestCase): def test_arcname_from(self) -> None: # (name, distribution_prefix, strip_path_prefixes, want) tuples diff --git a/tools/wheelmaker.py b/tools/wheelmaker.py index de6b8f48af..4390df3445 100644 --- a/tools/wheelmaker.py +++ b/tools/wheelmaker.py @@ -132,13 +132,17 @@ def __init__( distribution_prefix: str, strip_path_prefixes=None, compression=zipfile.ZIP_DEFLATED, + quote_all_filenames: bool = False, **kwargs, ): self._distribution_prefix = distribution_prefix self._strip_path_prefixes = strip_path_prefixes or [] - # Entries for the RECORD file as (filename, hash, size) tuples. - self._record = [] + # Entries for the RECORD file as (filename, digest, size) tuples. + self._record: list[tuple[str, str, str]] = [] + # Whether to quote filenames in the RECORD file (for compatibility with + # some wheels like torch that have quoted filenames in their RECORD). + self.quote_all_filenames = quote_all_filenames super().__init__(filename, mode=mode, compression=compression, **kwargs) @@ -192,16 +196,15 @@ def add_string(self, filename, contents): hash.update(contents) self._add_to_record(filename, self._serialize_digest(hash), len(contents)) - def _serialize_digest(self, hash): + def _serialize_digest(self, hash) -> str: # https://www.python.org/dev/peps/pep-0376/#record # "base64.urlsafe_b64encode(digest) with trailing = removed" digest = base64.urlsafe_b64encode(hash.digest()) digest = b"sha256=" + digest.rstrip(b"=") - return digest + return digest.decode("utf-8", "surrogateescape") - def _add_to_record(self, filename, hash, size): - size = str(size).encode("ascii") - self._record.append((filename, hash, size)) + def _add_to_record(self, filename: str, hash: str, size: int) -> None: + self._record.append((filename, hash, str(size))) def _zipinfo(self, filename): """Construct deterministic ZipInfo entry for a file named filename""" @@ -223,29 +226,27 @@ def _zipinfo(self, filename): zinfo.compress_type = self.compression return zinfo - def add_recordfile(self): + def _quote_filename(self, filename: str) -> str: + """Return a possibly quoted filename for RECORD file.""" + filename = filename.lstrip("/") + # Some RECORDs like torch have *all* filenames quoted and we must minimize diff. + # Otherwise, we quote only when necessary (e.g. for filenames with commas). + quoting = csv.QUOTE_ALL if self.quote_all_filenames else csv.QUOTE_MINIMAL + with io.StringIO() as buf: + csv.writer(buf, quoting=quoting).writerow([filename]) + return buf.getvalue().strip() + + def add_recordfile(self) -> str: """Write RECORD file to the distribution.""" record_path = self.distinfo_path("RECORD") - entries = self._record + [(record_path, b"", b"")] - with io.StringIO() as contents_io: - writer = csv.writer(contents_io, lineterminator="\n") - for filename, digest, size in entries: - if isinstance(filename, str): - filename = filename.lstrip("/") - writer.writerow( - ( - ( - c - if isinstance(c, str) - else c.decode("utf-8", "surrogateescape") - ) - for c in (filename, digest, size) - ) - ) - - contents = contents_io.getvalue() - self.add_string(record_path, contents) - return contents.encode("utf-8", "surrogateescape") + entries = self._record + [(record_path, "", "")] + entries = [ + (self._quote_filename(fname), digest, size) + for fname, digest, size in entries + ] + contents = "\n".join(",".join(entry) for entry in entries) + "\n" + self.add_string(record_path, contents) + return contents class WheelMaker(object): From c52aeaa6e0a5e07fc11a9d7a5ee94bd1d379d515 Mon Sep 17 00:00:00 2001 From: Ignas Anikevicius <240938+aignas@users.noreply.github.com> Date: Wed, 21 Jan 2026 01:29:39 +0900 Subject: [PATCH 591/922] fix(pipstar): correctly handle complex self deps (#3527) It seems that with the `pipstar` port there was a typo and the initial tests that we had for Python were insufficient to catch such a regression. The second if statement where we loop through packages again had a `req` instead of `req_` in the `if` statement and the test coverage was not sufficient. I have abstracted the if statement into a function to easier spot such issues and added an extra test to ensure that a regression would be actually caught. With this the Starlark test suite is now officially more robust than the Python version. Fixes #3524 --------- Co-authored-by: Richard Levasseur --- CHANGELOG.md | 8 ++++++ python/private/pypi/pep508_deps.bzl | 40 +++++++++++++++++++---------- tests/pypi/pep508/deps_tests.bzl | 19 ++++++++++++++ 3 files changed, 53 insertions(+), 14 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index d7a0ab3534..40ae51bfc2 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -79,6 +79,14 @@ END_UNRELEASED_TEMPLATE Use the `bazel_binary_info` module to access it. The {flag}`--stamp` flag will add {flag}`--workspace_status` information. +{#v1-8-1} +## [1.8.1] - 2026-01-20 + +{#v1-8-1-fixed} +### Fixed +* (pipstar) Extra resolution that refers back to the package being resolved works again. + Fixes [#3524](https://github.com/bazel-contrib/rules_python/issues/3524). + {#v1-8-0} ## [1.8.0] - 2025-12-19 diff --git a/python/private/pypi/pep508_deps.bzl b/python/private/pypi/pep508_deps.bzl index cdb449bdc2..ad6589cfac 100644 --- a/python/private/pypi/pep508_deps.bzl +++ b/python/private/pypi/pep508_deps.bzl @@ -115,7 +115,9 @@ def _resolve_extras(self_name, reqs, extras): # extras The empty string in the set is just a way to make the handling # of no extras and a single extra easier and having a set of {"", "foo"} # is equivalent to having {"foo"}. - extras = extras or [""] + # + # Use a dict as a set here to simplify operations. + extras = {x: None for x in (extras or [""])} self_reqs = [] for req in reqs: @@ -128,26 +130,36 @@ def _resolve_extras(self_name, reqs, extras): # easy to handle, lets do it. # # TODO @aignas 2023-12-08: add a test - extras = extras + req.extras + extras = extras | {x: None for x in req.extras} else: # process these in a separate loop self_reqs.append(req) - # A double loop is not strictly optimal, but always correct without recursion - for req in self_reqs: - if [True for extra in extras if evaluate(req.marker, env = {"extra": extra})]: - extras = extras + req.extras - else: - continue + for _ in range(10000): + # handles packages with up to 10000 recursive extras + new_extras = {} + for req in self_reqs: + if _evaluate_any(req, extras): + new_extras.update({x: None for x in req.extras}) + else: + continue - # Iterate through all packages to ensure that we include all of the extras from previously - # visited packages. - for req_ in self_reqs: - if [True for extra in extras if evaluate(req.marker, env = {"extra": extra})]: - extras = extras + req_.extras + num_extras_before = len(extras) + extras = extras | new_extras + num_extras_after = len(new_extras) + + if num_extras_before == num_extras_after: + break # Poor mans set - return sorted({x: None for x in extras}) + return sorted(extras) + +def _evaluate_any(req, extras): + for extra in extras: + if evaluate(req.marker, env = {"extra": extra}): + return True + + return False def _add_reqs(deps, deps_select, dep, reqs, *, extras): for req in reqs: diff --git a/tests/pypi/pep508/deps_tests.bzl b/tests/pypi/pep508/deps_tests.bzl index 679ba58396..f566845b70 100644 --- a/tests/pypi/pep508/deps_tests.bzl +++ b/tests/pypi/pep508/deps_tests.bzl @@ -90,6 +90,7 @@ def test_self_dependencies_can_come_in_any_order(env): "baz; extra == 'feat'", "foo[feat2]; extra == 'all'", "foo[feat]; extra == 'feat2'", + "foo[feat3]; extra == 'all'", "zdep; extra == 'all'", ], extras = ["all"], @@ -100,6 +101,24 @@ def test_self_dependencies_can_come_in_any_order(env): _tests.append(test_self_dependencies_can_come_in_any_order) +def test_self_include_deps_from_previously_visited(env): + got = deps( + "foo", + requires_dist = [ + "bar", + "baz; extra == 'feat'", + "foo[dev]; extra == 'all'", + "foo[feat]; extra == 'feat2'", + "dev_dep; extra == 'dev'", + ], + extras = ["feat2"], + ) + + env.expect.that_collection(got.deps).contains_exactly(["bar", "baz"]) + env.expect.that_dict(got.deps_select).contains_exactly({}) + +_tests.append(test_self_include_deps_from_previously_visited) + def _test_can_get_deps_based_on_specific_python_version(env): requires_dist = [ "bar", From 889e5acd350e584c23d5c770c3b42eaa86c8e240 Mon Sep 17 00:00:00 2001 From: Roman Joost Date: Thu, 22 Jan 2026 11:27:35 +1100 Subject: [PATCH 592/922] doc: bazel downloader (#3519) (#3530) This adds documentation on how to configure the Bazel downloader in order to work with network isolated registries. Fixes #3519 --- docs/pypi/download.md | 47 +++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 47 insertions(+) diff --git a/docs/pypi/download.md b/docs/pypi/download.md index 258f211301..b798d816f0 100644 --- a/docs/pypi/download.md +++ b/docs/pypi/download.md @@ -269,6 +269,53 @@ available flags: [pep600]: https://peps.python.org/pep-0600/ [pep656]: https://peps.python.org/pep-0656/ +## Internal dependencies and private repositories + +The `rules_python` Bazel module downloads Python interpreters and +dependencies as part of its functionality. These artifacts are fetched +using Bazel's internal HTTP downloader, not using the `pip` tool. + +If you are in a network-restricted environment and must use internal +registries, you can configure the Bazel downloader to redirect all of +these downloads to a different registry. + +Example of a `bazel_downloader.cfg`: +```cfg +all_blocked_message See internal.mirror.lan/registry/ for more information +allow s3.amazon.com + +# Rewrite everything to files.pythonhosted to the internal mirror with two +# capture groups: the first group matches the host and is appended first, +# the second matches the entire path and is appended second +rewrite (files.pythonhosted.org)/(.*) internal.mirror.lan/python/$1/$2 +rewrite (pypi.python.org)/(.*) internal.mirror.lan/python/$1/$2 + +# Allow the internal mirror and block everything else +allow internal.mirror.lan +block * +``` + +Use the config file with `--experimental_downloader_config=bazel_downloader.cfg`. + +### How the config is parsed: + +* Uses Java regular expressions +* Matching is performed only on host and path components of the URL, not the scheme +* Directives are applied in the following order: `rewrite, allow, block` +* Back references are numbered starting from `$1` +* Expressions must match the entire string being tested, not just find a substring. + +If your patterns don't seem to match or rewrite: + +* Begin with simple patterns to ensure they match as expected. +* Be cautious when using `block` statements to avoid unintentionally blocking necessary downloads. Add `block` statements incrementally and test thoroughly after each change. + +### References: + +* [Configuring Bazel's Downloader](https://blog.aspect.build/configuring-bazels-downloader) +* [URLRewriterConfig.java Source Code](https://github.com/bazelbuild/bazel/blob/master/src/main/java/com/google/devtools/build/lib/bazel/repository/downloader/UrlRewriterConfig.java) +* [Issue 3519](https://github.com/bazel-contrib/rules_python/issues/3519) + (credential-helper)= ## Credential Helper From fa82b6866e0b3086f2db68cf83deeb1e63217a38 Mon Sep 17 00:00:00 2001 From: Mathieu Olivari Date: Fri, 23 Jan 2026 00:27:38 -0800 Subject: [PATCH 593/922] feat(python): add arm64e-apple-darwin platform support (#3535) MacOS supported architectures include x86_64, arm64 and arm64e. While unusual, there are certain edge cases that require setting the host platform to arm64e, in which case we probably want the python toolchain resolution to happen properly. So we're adding a new platform in this commit that will match the same arm64 toolchain on arm64e platform. --- python/versions.bzl | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/python/versions.bzl b/python/versions.bzl index 3299a9f16b..f3c712e559 100644 --- a/python/versions.bzl +++ b/python/versions.bzl @@ -1127,6 +1127,14 @@ def _generate_platforms(): os_name = LINUX_NAME, arch = "aarch64", ), + "arm64e-apple-darwin": platform_info( + compatible_with = [ + "@platforms//os:macos", + "@platforms//cpu:arm64e", + ], + os_name = MACOS_NAME, + arch = "aarch64", + ), "armv7-unknown-linux-gnu": platform_info( compatible_with = [ "@platforms//os:linux", From 044eb2651bfaa67431647e55d03990870a812c8f Mon Sep 17 00:00:00 2001 From: Douglas Thor Date: Sat, 24 Jan 2026 14:13:55 -0800 Subject: [PATCH 594/922] feat(gazelle): Add python_generate_pyi_srcs directive (#3356) Fixes #3354 This adds a new Gazelle directive `python_generate_pyi_srcs` which accepts a boolean value `true` or `false`. It defaults to `false` for backwards compatibility. When `true`, the directive causes `.pyi` files, whose name matches with a .py file found in `srcs`, to be added to the `pyi_srcs` target attribute. This helps with cases where manually-generated `.pyi` files are present in the project and are needed for things such as rules_mypy. Given the following files in a `my_dir` package: ``` BUILD.bazel __init__.py a.py a.pyi b.py c.py c.pyi ``` `# gazelle:python_generate_pyi_srcs true` will cause Gazelle to generate: ```starlark py_library( name = "my_dir", srcs = [ "__init__.py", "a.py", "b.py", "c.py", ], pyi_srcs = [ "a.py", "c.py", ], ) ``` --- CHANGELOG.md | 4 ++ gazelle/docs/directives.md | 28 +++++++++++ gazelle/python/configure.go | 7 +++ gazelle/python/generate.go | 48 +++++++++++++++++++ gazelle/python/kinds.go | 3 ++ gazelle/python/target.go | 20 ++++++++ .../BUILD.in | 1 + .../BUILD.out | 36 ++++++++++++++ .../README.md | 13 +++++ .../WORKSPACE | 0 .../__init__.py | 0 .../__main__.py | 0 .../__main__.pyi | 0 .../__test__.py | 0 .../directive_python_generate_pyi_srcs/bar.py | 0 .../directive_python_generate_pyi_srcs/baz.py | 0 .../baz.pyi | 0 .../directive_python_generate_pyi_srcs/foo.py | 0 .../foo.pyi | 0 .../foo_test.py | 0 .../foo_test.pyi | 0 .../per_file/BUILD.in | 2 + .../per_file/BUILD.out | 30 ++++++++++++ .../per_file/__init__.py | 0 .../per_file/bar.py | 0 .../per_file/bar_test.py | 0 .../per_file/bar_test.pyi | 0 .../per_file/foo.py | 0 .../per_file/foo.pyi | 0 .../per_file/my_binary.py | 2 + .../per_file/my_binary.pyi | 0 .../per_file/turn_off/BUILD.in | 1 + .../per_file/turn_off/BUILD.out | 9 ++++ .../per_file/turn_off/foo.py | 0 .../per_file/turn_off/foo.pyi | 0 .../test.yaml | 0 gazelle/pythonconfig/pythonconfig.go | 19 ++++++++ 37 files changed, 223 insertions(+) create mode 100644 gazelle/python/testdata/directive_python_generate_pyi_srcs/BUILD.in create mode 100644 gazelle/python/testdata/directive_python_generate_pyi_srcs/BUILD.out create mode 100644 gazelle/python/testdata/directive_python_generate_pyi_srcs/README.md create mode 100644 gazelle/python/testdata/directive_python_generate_pyi_srcs/WORKSPACE create mode 100644 gazelle/python/testdata/directive_python_generate_pyi_srcs/__init__.py create mode 100644 gazelle/python/testdata/directive_python_generate_pyi_srcs/__main__.py create mode 100644 gazelle/python/testdata/directive_python_generate_pyi_srcs/__main__.pyi create mode 100644 gazelle/python/testdata/directive_python_generate_pyi_srcs/__test__.py create mode 100644 gazelle/python/testdata/directive_python_generate_pyi_srcs/bar.py create mode 100644 gazelle/python/testdata/directive_python_generate_pyi_srcs/baz.py create mode 100644 gazelle/python/testdata/directive_python_generate_pyi_srcs/baz.pyi create mode 100644 gazelle/python/testdata/directive_python_generate_pyi_srcs/foo.py create mode 100644 gazelle/python/testdata/directive_python_generate_pyi_srcs/foo.pyi create mode 100644 gazelle/python/testdata/directive_python_generate_pyi_srcs/foo_test.py create mode 100644 gazelle/python/testdata/directive_python_generate_pyi_srcs/foo_test.pyi create mode 100644 gazelle/python/testdata/directive_python_generate_pyi_srcs/per_file/BUILD.in create mode 100644 gazelle/python/testdata/directive_python_generate_pyi_srcs/per_file/BUILD.out create mode 100644 gazelle/python/testdata/directive_python_generate_pyi_srcs/per_file/__init__.py create mode 100644 gazelle/python/testdata/directive_python_generate_pyi_srcs/per_file/bar.py create mode 100644 gazelle/python/testdata/directive_python_generate_pyi_srcs/per_file/bar_test.py create mode 100644 gazelle/python/testdata/directive_python_generate_pyi_srcs/per_file/bar_test.pyi create mode 100644 gazelle/python/testdata/directive_python_generate_pyi_srcs/per_file/foo.py create mode 100644 gazelle/python/testdata/directive_python_generate_pyi_srcs/per_file/foo.pyi create mode 100644 gazelle/python/testdata/directive_python_generate_pyi_srcs/per_file/my_binary.py create mode 100644 gazelle/python/testdata/directive_python_generate_pyi_srcs/per_file/my_binary.pyi create mode 100644 gazelle/python/testdata/directive_python_generate_pyi_srcs/per_file/turn_off/BUILD.in create mode 100644 gazelle/python/testdata/directive_python_generate_pyi_srcs/per_file/turn_off/BUILD.out create mode 100644 gazelle/python/testdata/directive_python_generate_pyi_srcs/per_file/turn_off/foo.py create mode 100644 gazelle/python/testdata/directive_python_generate_pyi_srcs/per_file/turn_off/foo.pyi create mode 100644 gazelle/python/testdata/directive_python_generate_pyi_srcs/test.yaml diff --git a/CHANGELOG.md b/CHANGELOG.md index 40ae51bfc2..431f77040b 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -78,6 +78,10 @@ END_UNRELEASED_TEMPLATE * (binaries/tests) Build information is now included in binaries and tests. Use the `bazel_binary_info` module to access it. The {flag}`--stamp` flag will add {flag}`--workspace_status` information. +* (gazelle) A new directive `python_generate_pyi_deps` has been added. When + `true`, a `py_*` target's `pyi_srcs` attribute will be set if any `.pyi` files + that are associated with the target's `srcs` are present. + ([#3354](https://github.com/bazel-contrib/rules_python/issues/3354)). {#v1-8-1} ## [1.8.1] - 2026-01-20 diff --git a/gazelle/docs/directives.md b/gazelle/docs/directives.md index a553226a59..c7936d539d 100644 --- a/gazelle/docs/directives.md +++ b/gazelle/docs/directives.md @@ -155,6 +155,12 @@ The Python-specific directives are: * Default: `false` * Allowed Values: `true`, `false` +[`# gazelle:python_generate_pyi_srcs bool`](#python-generate-pyi-srcs) +: Controls whether to generate a `pyi_srcs` attribute if a sibling `.pyi` file + is found. When `false` (default), the `pyi_srcs` attribute is not added. + * Default: `false` + * Allowed Values: `true`, `false` + [`# gazelle:python_generate_proto bool`](#python-generate-proto) : Controls whether to generate a {bzl:obj}`py_proto_library` for each {bzl:obj}`proto_library` in the package. By default we load this rule from the @@ -626,6 +632,28 @@ Detailed docs are not yet written. ::: +## `python_generate_pyi_deps` + +When `true`, include any sibling `.pyi` files in the `pyi_srcs` target attribute. + +For example, assume you have the following files: + +``` +foo.py +foo.pyi +``` + +The generated target will be: + +```starlark +py_library( + name = "foo", + srcs = ["foo.py"], + pyi_srcs = ["foo.pyi"], +) +``` + + ## `python_generate_proto` When `# gazelle:python_generate_proto true`, Gazelle will generate one diff --git a/gazelle/python/configure.go b/gazelle/python/configure.go index 13ba6477cd..88b15e91eb 100644 --- a/gazelle/python/configure.go +++ b/gazelle/python/configure.go @@ -70,6 +70,7 @@ func (py *Configurer) KnownDirectives() []string { pythonconfig.LabelConvention, pythonconfig.LabelNormalization, pythonconfig.GeneratePyiDeps, + pythonconfig.GeneratePyiSrcs, pythonconfig.ExperimentalAllowRelativeImports, pythonconfig.GenerateProto, pythonconfig.PythonResolveSiblingImports, @@ -242,6 +243,12 @@ func (py *Configurer) Configure(c *config.Config, rel string, f *rule.File) { log.Fatal(err) } config.SetGeneratePyiDeps(v) + case pythonconfig.GeneratePyiSrcs: + v, err := strconv.ParseBool(strings.TrimSpace(d.Value)) + if err != nil { + log.Fatal(err) + } + config.SetGeneratePyiSrcs(v) case pythonconfig.GenerateProto: v, err := strconv.ParseBool(strings.TrimSpace(d.Value)) if err != nil { diff --git a/gazelle/python/generate.go b/gazelle/python/generate.go index 90216b00bc..ebca10671c 100644 --- a/gazelle/python/generate.go +++ b/gazelle/python/generate.go @@ -273,6 +273,7 @@ func (py *Python) GenerateRules(args language.GenerateArgs) language.GenerateRes srcs.Remove(name) } } + sort.Strings(mainFileNames) for _, filename := range mainFileNames { pyBinaryTargetName := strings.TrimSuffix(filepath.Base(filename), ".py") @@ -282,9 +283,15 @@ func (py *Python) GenerateRules(args language.GenerateArgs) language.GenerateRes fqTarget.String(), actualPyBinaryKind, err) continue } + + // Add any sibling .pyi files to pyi_srcs + filenames := treeset.NewWith(godsutils.StringComparator, filename) + pyiSrcs, _ := getPyiFilenames(filenames, cfg.GeneratePyiSrcs(), args.Dir) + pyBinary := newTargetBuilder(pyBinaryKind, pyBinaryTargetName, pythonProjectRoot, args.Rel, pyFileNames, cfg.ResolveSiblingImports()). addVisibility(visibility). addSrc(filename). + addPyiSrcs(pyiSrcs). addModuleDependencies(mainModules[filename]). addResolvedDependencies(annotations.includeDeps). generateImportsAttribute(). @@ -312,6 +319,9 @@ func (py *Python) GenerateRules(args language.GenerateArgs) language.GenerateRes } } + // Add any sibling .pyi files to pyi_srcs + pyiSrcs, _ := getPyiFilenames(srcs, cfg.GeneratePyiSrcs(), args.Dir) + // Check if a target with the same name we are generating already // exists, and if it is of a different kind from the one we are // generating. If so, we have to throw an error since Gazelle won't @@ -327,6 +337,7 @@ func (py *Python) GenerateRules(args language.GenerateArgs) language.GenerateRes pyLibrary := newTargetBuilder(pyLibraryKind, pyLibraryTargetName, pythonProjectRoot, args.Rel, pyFileNames, cfg.ResolveSiblingImports()). addVisibility(visibility). addSrcs(srcs). + addPyiSrcs(pyiSrcs). addModuleDependencies(allDeps). addResolvedDependencies(annotations.includeDeps). generateImportsAttribute(). @@ -377,10 +388,15 @@ func (py *Python) GenerateRules(args language.GenerateArgs) language.GenerateRes collisionErrors.Add(err) } + // Add any sibling .pyi files to pyi_srcs + filenames := treeset.NewWith(godsutils.StringComparator, pyBinaryEntrypointFilename) + pyiSrcs, _ := getPyiFilenames(filenames, cfg.GeneratePyiSrcs(), args.Dir) + pyBinaryTarget := newTargetBuilder(pyBinaryKind, pyBinaryTargetName, pythonProjectRoot, args.Rel, pyFileNames, cfg.ResolveSiblingImports()). setMain(pyBinaryEntrypointFilename). addVisibility(visibility). addSrc(pyBinaryEntrypointFilename). + addPyiSrcs(pyiSrcs). addModuleDependencies(deps). addResolvedDependencies(annotations.includeDeps). setAnnotations(*annotations). @@ -411,8 +427,13 @@ func (py *Python) GenerateRules(args language.GenerateArgs) language.GenerateRes collisionErrors.Add(err) } + // Add any sibling .pyi files to pyi_srcs + filenames := treeset.NewWith(godsutils.StringComparator, conftestFilename) + pyiSrcs, _ := getPyiFilenames(filenames, cfg.GeneratePyiSrcs(), args.Dir) + conftestTarget := newTargetBuilder(pyLibraryKind, conftestTargetname, pythonProjectRoot, args.Rel, pyFileNames, cfg.ResolveSiblingImports()). addSrc(conftestFilename). + addPyiSrcs(pyiSrcs). addModuleDependencies(deps). addResolvedDependencies(annotations.includeDeps). setAnnotations(*annotations). @@ -443,8 +464,13 @@ func (py *Python) GenerateRules(args language.GenerateArgs) language.GenerateRes fqTarget.String(), actualPyTestKind, err, pythonconfig.TestNamingConvention) collisionErrors.Add(err) } + + // Add any sibling .pyi files to pyi_srcs + pyiSrcs, _ := getPyiFilenames(srcs, cfg.GeneratePyiSrcs(), args.Dir) + return newTargetBuilder(pyTestKind, pyTestTargetName, pythonProjectRoot, args.Rel, pyFileNames, cfg.ResolveSiblingImports()). addSrcs(srcs). + addPyiSrcs(pyiSrcs). addModuleDependencies(deps). addResolvedDependencies(annotations.includeDeps). setAnnotations(*annotations). @@ -691,3 +717,25 @@ func generateProtoLibraries(args language.GenerateArgs, cfg *pythonconfig.Config } } + +// getPyiFilenames returns a set of existing .pyi source file names for a given set of source +// file names if GeneratePyiSrcs is set. Otherwise, returns an empty set. +func getPyiFilenames(filenames *treeset.Set, generatePyiSrcs bool, basePath string) (*treeset.Set, error) { + pyiSrcs := treeset.NewWith(godsutils.StringComparator) + if !generatePyiSrcs { + return pyiSrcs, nil + } + + it := filenames.Iterator() + for it.Next() { + pyiFilename := it.Value().(string) + "i" // foo.py --> foo.pyi + + _, err := os.Stat(filepath.Join(basePath, pyiFilename)) + // If the file DNE or there's some other error, there's nothing to do. + if err == nil { + // pyi file exists, add it + pyiSrcs.Add(pyiFilename) + } + } + return pyiSrcs, nil +} diff --git a/gazelle/python/kinds.go b/gazelle/python/kinds.go index 4fe8090445..dac271bec6 100644 --- a/gazelle/python/kinds.go +++ b/gazelle/python/kinds.go @@ -51,6 +51,7 @@ var pyKinds = map[string]rule.KindInfo{ ResolveAttrs: map[string]bool{ "deps": true, "pyi_deps": true, + "pyi_srcs": true, }, }, pyLibraryKind: { @@ -68,6 +69,7 @@ var pyKinds = map[string]rule.KindInfo{ ResolveAttrs: map[string]bool{ "deps": true, "pyi_deps": true, + "pyi_srcs": true, }, }, pyProtoLibraryKind: { @@ -91,6 +93,7 @@ var pyKinds = map[string]rule.KindInfo{ ResolveAttrs: map[string]bool{ "deps": true, "pyi_deps": true, + "pyi_srcs": true, }, }, } diff --git a/gazelle/python/target.go b/gazelle/python/target.go index 3fe5819e00..c7009a6a84 100644 --- a/gazelle/python/target.go +++ b/gazelle/python/target.go @@ -30,6 +30,7 @@ type targetBuilder struct { pythonProjectRoot string bzlPackage string srcs *treeset.Set + pyiSrcs *treeset.Set siblingSrcs *treeset.Set deps *treeset.Set resolvedDeps *treeset.Set @@ -49,6 +50,7 @@ func newTargetBuilder(kind, name, pythonProjectRoot, bzlPackage string, siblingS pythonProjectRoot: pythonProjectRoot, bzlPackage: bzlPackage, srcs: treeset.NewWith(godsutils.StringComparator), + pyiSrcs: treeset.NewWith(godsutils.StringComparator), siblingSrcs: siblingSrcs, deps: treeset.NewWith(moduleComparator), resolvedDeps: treeset.NewWith(godsutils.StringComparator), @@ -73,6 +75,21 @@ func (t *targetBuilder) addSrcs(srcs *treeset.Set) *targetBuilder { return t } +// addPyiSrc adds a single pyi_src to the target. +func (t *targetBuilder) addPyiSrc(pyiSrc string) *targetBuilder { + t.pyiSrcs.Add(pyiSrc) + return t +} + +// addPyiSrcs adds multiple pyi_srcs to the target. +func (t *targetBuilder) addPyiSrcs(pyiSrcs *treeset.Set) *targetBuilder { + it := pyiSrcs.Iterator() + for it.Next() { + t.pyiSrcs.Add(it.Value().(string)) + } + return t +} + // addModuleDependency adds a single module dep to the target. func (t *targetBuilder) addModuleDependency(dep Module) *targetBuilder { fileName := dep.Name + ".py" @@ -165,6 +182,9 @@ func (t *targetBuilder) build() *rule.Rule { if !t.srcs.Empty() { r.SetAttr("srcs", t.srcs.Values()) } + if !t.pyiSrcs.Empty() { + r.SetAttr("pyi_srcs", t.pyiSrcs.Values()) + } if !t.visibility.Empty() { r.SetAttr("visibility", t.visibility.Values()) } diff --git a/gazelle/python/testdata/directive_python_generate_pyi_srcs/BUILD.in b/gazelle/python/testdata/directive_python_generate_pyi_srcs/BUILD.in new file mode 100644 index 0000000000..a8459ad349 --- /dev/null +++ b/gazelle/python/testdata/directive_python_generate_pyi_srcs/BUILD.in @@ -0,0 +1 @@ +# gazelle:python_generate_pyi_srcs true diff --git a/gazelle/python/testdata/directive_python_generate_pyi_srcs/BUILD.out b/gazelle/python/testdata/directive_python_generate_pyi_srcs/BUILD.out new file mode 100644 index 0000000000..5beb703423 --- /dev/null +++ b/gazelle/python/testdata/directive_python_generate_pyi_srcs/BUILD.out @@ -0,0 +1,36 @@ +load("@rules_python//python:defs.bzl", "py_binary", "py_library", "py_test") + +# gazelle:python_generate_pyi_srcs true + +py_library( + name = "directive_python_generate_pyi_srcs", + srcs = [ + "__init__.py", + "bar.py", + "baz.py", + "foo.py", + ], + pyi_srcs = [ + "baz.pyi", + "foo.pyi", + ], + visibility = ["//:__subpackages__"], +) + +py_binary( + name = "directive_python_generate_pyi_srcs_bin", + srcs = ["__main__.py"], + main = "__main__.py", + pyi_srcs = ["__main__.pyi"], + visibility = ["//:__subpackages__"], +) + +py_test( + name = "directive_python_generate_pyi_srcs_test", + srcs = [ + "__test__.py", + "foo_test.py", + ], + main = "__test__.py", + pyi_srcs = ["foo_test.pyi"], +) diff --git a/gazelle/python/testdata/directive_python_generate_pyi_srcs/README.md b/gazelle/python/testdata/directive_python_generate_pyi_srcs/README.md new file mode 100644 index 0000000000..3ef10bab06 --- /dev/null +++ b/gazelle/python/testdata/directive_python_generate_pyi_srcs/README.md @@ -0,0 +1,13 @@ +# Directive: python_generate_pyi_srcs + +Test that the `python_generate_pyi_srcs` directive will add `pyi_srcs` to +generated targets and that it can be toggled on/off on a per-package basis. + +The root of the test case asserts that the default generation mode (package) +will compile multiple .pyi files into a single py_* target. + +The `per_file` directory asserts that the `file` generation mode will attach +a single .pyi file to a given target. + +Lastly, the `per_file/turn_off` directory asserts that we can turn off the +directive for subpackages. It continues with per-file generation mode. diff --git a/gazelle/python/testdata/directive_python_generate_pyi_srcs/WORKSPACE b/gazelle/python/testdata/directive_python_generate_pyi_srcs/WORKSPACE new file mode 100644 index 0000000000..e69de29bb2 diff --git a/gazelle/python/testdata/directive_python_generate_pyi_srcs/__init__.py b/gazelle/python/testdata/directive_python_generate_pyi_srcs/__init__.py new file mode 100644 index 0000000000..e69de29bb2 diff --git a/gazelle/python/testdata/directive_python_generate_pyi_srcs/__main__.py b/gazelle/python/testdata/directive_python_generate_pyi_srcs/__main__.py new file mode 100644 index 0000000000..e69de29bb2 diff --git a/gazelle/python/testdata/directive_python_generate_pyi_srcs/__main__.pyi b/gazelle/python/testdata/directive_python_generate_pyi_srcs/__main__.pyi new file mode 100644 index 0000000000..e69de29bb2 diff --git a/gazelle/python/testdata/directive_python_generate_pyi_srcs/__test__.py b/gazelle/python/testdata/directive_python_generate_pyi_srcs/__test__.py new file mode 100644 index 0000000000..e69de29bb2 diff --git a/gazelle/python/testdata/directive_python_generate_pyi_srcs/bar.py b/gazelle/python/testdata/directive_python_generate_pyi_srcs/bar.py new file mode 100644 index 0000000000..e69de29bb2 diff --git a/gazelle/python/testdata/directive_python_generate_pyi_srcs/baz.py b/gazelle/python/testdata/directive_python_generate_pyi_srcs/baz.py new file mode 100644 index 0000000000..e69de29bb2 diff --git a/gazelle/python/testdata/directive_python_generate_pyi_srcs/baz.pyi b/gazelle/python/testdata/directive_python_generate_pyi_srcs/baz.pyi new file mode 100644 index 0000000000..e69de29bb2 diff --git a/gazelle/python/testdata/directive_python_generate_pyi_srcs/foo.py b/gazelle/python/testdata/directive_python_generate_pyi_srcs/foo.py new file mode 100644 index 0000000000..e69de29bb2 diff --git a/gazelle/python/testdata/directive_python_generate_pyi_srcs/foo.pyi b/gazelle/python/testdata/directive_python_generate_pyi_srcs/foo.pyi new file mode 100644 index 0000000000..e69de29bb2 diff --git a/gazelle/python/testdata/directive_python_generate_pyi_srcs/foo_test.py b/gazelle/python/testdata/directive_python_generate_pyi_srcs/foo_test.py new file mode 100644 index 0000000000..e69de29bb2 diff --git a/gazelle/python/testdata/directive_python_generate_pyi_srcs/foo_test.pyi b/gazelle/python/testdata/directive_python_generate_pyi_srcs/foo_test.pyi new file mode 100644 index 0000000000..e69de29bb2 diff --git a/gazelle/python/testdata/directive_python_generate_pyi_srcs/per_file/BUILD.in b/gazelle/python/testdata/directive_python_generate_pyi_srcs/per_file/BUILD.in new file mode 100644 index 0000000000..a7dbb7e4c1 --- /dev/null +++ b/gazelle/python/testdata/directive_python_generate_pyi_srcs/per_file/BUILD.in @@ -0,0 +1,2 @@ +# gazelle:python_generate_pyi_srcs true +# gazelle:python_generation_mode file diff --git a/gazelle/python/testdata/directive_python_generate_pyi_srcs/per_file/BUILD.out b/gazelle/python/testdata/directive_python_generate_pyi_srcs/per_file/BUILD.out new file mode 100644 index 0000000000..d0889fc46f --- /dev/null +++ b/gazelle/python/testdata/directive_python_generate_pyi_srcs/per_file/BUILD.out @@ -0,0 +1,30 @@ +load("@rules_python//python:defs.bzl", "py_binary", "py_library", "py_test") + +# gazelle:python_generate_pyi_srcs true +# gazelle:python_generation_mode file + +py_library( + name = "bar", + srcs = ["bar.py"], + visibility = ["//:__subpackages__"], +) + +py_library( + name = "foo", + srcs = ["foo.py"], + pyi_srcs = ["foo.pyi"], + visibility = ["//:__subpackages__"], +) + +py_binary( + name = "my_binary", + srcs = ["my_binary.py"], + pyi_srcs = ["my_binary.pyi"], + visibility = ["//:__subpackages__"], +) + +py_test( + name = "bar_test", + srcs = ["bar_test.py"], + pyi_srcs = ["bar_test.pyi"], +) diff --git a/gazelle/python/testdata/directive_python_generate_pyi_srcs/per_file/__init__.py b/gazelle/python/testdata/directive_python_generate_pyi_srcs/per_file/__init__.py new file mode 100644 index 0000000000..e69de29bb2 diff --git a/gazelle/python/testdata/directive_python_generate_pyi_srcs/per_file/bar.py b/gazelle/python/testdata/directive_python_generate_pyi_srcs/per_file/bar.py new file mode 100644 index 0000000000..e69de29bb2 diff --git a/gazelle/python/testdata/directive_python_generate_pyi_srcs/per_file/bar_test.py b/gazelle/python/testdata/directive_python_generate_pyi_srcs/per_file/bar_test.py new file mode 100644 index 0000000000..e69de29bb2 diff --git a/gazelle/python/testdata/directive_python_generate_pyi_srcs/per_file/bar_test.pyi b/gazelle/python/testdata/directive_python_generate_pyi_srcs/per_file/bar_test.pyi new file mode 100644 index 0000000000..e69de29bb2 diff --git a/gazelle/python/testdata/directive_python_generate_pyi_srcs/per_file/foo.py b/gazelle/python/testdata/directive_python_generate_pyi_srcs/per_file/foo.py new file mode 100644 index 0000000000..e69de29bb2 diff --git a/gazelle/python/testdata/directive_python_generate_pyi_srcs/per_file/foo.pyi b/gazelle/python/testdata/directive_python_generate_pyi_srcs/per_file/foo.pyi new file mode 100644 index 0000000000..e69de29bb2 diff --git a/gazelle/python/testdata/directive_python_generate_pyi_srcs/per_file/my_binary.py b/gazelle/python/testdata/directive_python_generate_pyi_srcs/per_file/my_binary.py new file mode 100644 index 0000000000..6a81382943 --- /dev/null +++ b/gazelle/python/testdata/directive_python_generate_pyi_srcs/per_file/my_binary.py @@ -0,0 +1,2 @@ +if __name__ == "__main__": + print("hey") diff --git a/gazelle/python/testdata/directive_python_generate_pyi_srcs/per_file/my_binary.pyi b/gazelle/python/testdata/directive_python_generate_pyi_srcs/per_file/my_binary.pyi new file mode 100644 index 0000000000..e69de29bb2 diff --git a/gazelle/python/testdata/directive_python_generate_pyi_srcs/per_file/turn_off/BUILD.in b/gazelle/python/testdata/directive_python_generate_pyi_srcs/per_file/turn_off/BUILD.in new file mode 100644 index 0000000000..7a24055cd0 --- /dev/null +++ b/gazelle/python/testdata/directive_python_generate_pyi_srcs/per_file/turn_off/BUILD.in @@ -0,0 +1 @@ +# gazelle:python_generate_pyi_srcs false diff --git a/gazelle/python/testdata/directive_python_generate_pyi_srcs/per_file/turn_off/BUILD.out b/gazelle/python/testdata/directive_python_generate_pyi_srcs/per_file/turn_off/BUILD.out new file mode 100644 index 0000000000..25b508a7f0 --- /dev/null +++ b/gazelle/python/testdata/directive_python_generate_pyi_srcs/per_file/turn_off/BUILD.out @@ -0,0 +1,9 @@ +load("@rules_python//python:defs.bzl", "py_library") + +# gazelle:python_generate_pyi_srcs false + +py_library( + name = "foo", + srcs = ["foo.py"], + visibility = ["//:__subpackages__"], +) diff --git a/gazelle/python/testdata/directive_python_generate_pyi_srcs/per_file/turn_off/foo.py b/gazelle/python/testdata/directive_python_generate_pyi_srcs/per_file/turn_off/foo.py new file mode 100644 index 0000000000..e69de29bb2 diff --git a/gazelle/python/testdata/directive_python_generate_pyi_srcs/per_file/turn_off/foo.pyi b/gazelle/python/testdata/directive_python_generate_pyi_srcs/per_file/turn_off/foo.pyi new file mode 100644 index 0000000000..e69de29bb2 diff --git a/gazelle/python/testdata/directive_python_generate_pyi_srcs/test.yaml b/gazelle/python/testdata/directive_python_generate_pyi_srcs/test.yaml new file mode 100644 index 0000000000..e69de29bb2 diff --git a/gazelle/pythonconfig/pythonconfig.go b/gazelle/pythonconfig/pythonconfig.go index ed9b914e82..a1271af3be 100644 --- a/gazelle/pythonconfig/pythonconfig.go +++ b/gazelle/pythonconfig/pythonconfig.go @@ -104,6 +104,10 @@ const ( // separate pyi_deps attribute or merge type-checking dependencies into deps. // Defaults to false for backward compatibility. GeneratePyiDeps = "python_generate_pyi_deps" + // GeneratePyiSrcs represents the directive that controls whether to include + // a pyi_srcs attribute if a sibling .pyi file is found. + // Defaults to false for backward compatibility. + GeneratePyiSrcs = "python_generate_pyi_srcs" // GenerateProto represents the directive that controls whether to generate // python_generate_proto targets. GenerateProto = "python_generate_proto" @@ -202,6 +206,7 @@ type Config struct { labelNormalization LabelNormalizationType experimentalAllowRelativeImports bool generatePyiDeps bool + generatePyiSrcs bool generateProto bool resolveSiblingImports bool } @@ -242,6 +247,7 @@ func New( labelNormalization: DefaultLabelNormalizationType, experimentalAllowRelativeImports: false, generatePyiDeps: false, + generatePyiSrcs: false, generateProto: false, resolveSiblingImports: false, } @@ -279,6 +285,7 @@ func (c *Config) NewChild() *Config { labelNormalization: c.labelNormalization, experimentalAllowRelativeImports: c.experimentalAllowRelativeImports, generatePyiDeps: c.generatePyiDeps, + generatePyiSrcs: c.generatePyiSrcs, generateProto: c.generateProto, resolveSiblingImports: c.resolveSiblingImports, } @@ -590,6 +597,18 @@ func (c *Config) GeneratePyiDeps() bool { return c.generatePyiDeps } +// SetGeneratePyiSrcs sets whether pyi_srcs attribute should be generated if a sibling +// .pyi file is found. +func (c *Config) SetGeneratePyiSrcs(generatePyiSrcs bool) { + c.generatePyiSrcs = generatePyiSrcs +} + +// GeneratePyiSrcs returns whether pyi_srcs attribute should be generated if a sibling +// .pyi file is found. +func (c *Config) GeneratePyiSrcs() bool { + return c.generatePyiSrcs +} + // SetGenerateProto sets whether py_proto_library should be generated for proto_library. func (c *Config) SetGenerateProto(generateProto bool) { c.generateProto = generateProto From b4ec825850fb27e5568165d0a388f5cc4b2b84a2 Mon Sep 17 00:00:00 2001 From: gfrankliu <17630355+gfrankliu@users.noreply.github.com> Date: Sat, 24 Jan 2026 18:17:22 -0800 Subject: [PATCH 595/922] fix: explicitly symlink all .so files, not just ones with lib prefix (#3538) Some packages, such as tensorflow, have regular C libraries that don't use a `lib*` suffix. The symlink optimization logic wouldn't link these directly, which made the dynamic linker unable to find their dependencies. To fix, explicitly symlink all `.so` files, since we can't determine which are Python C modules and regular C libraries. Fixes https://github.com/bazel-contrib/rules_python/issues/3529 --- CHANGELOG.md | 8 +++++++ python/private/venv_runfiles.bzl | 11 ++-------- .../app_files_building_tests.bzl | 22 +++++++++++++++---- 3 files changed, 28 insertions(+), 13 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 431f77040b..683235b7f3 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -83,6 +83,14 @@ END_UNRELEASED_TEMPLATE that are associated with the target's `srcs` are present. ([#3354](https://github.com/bazel-contrib/rules_python/issues/3354)). +{#v1-8-2} +## [1.8.2] - 2026-01-24 + +{#v1-8-2-fixed} +### Fixed +* (venvs) relax the C library filename check to make tensorflow work + Fixes [#3524](https://github.com/bazel-contrib/rules_python/issues/3529). + {#v1-8-1} ## [1.8.1] - 2026-01-20 diff --git a/python/private/venv_runfiles.bzl b/python/private/venv_runfiles.bzl index 7f6af0c957..d522470942 100644 --- a/python/private/venv_runfiles.bzl +++ b/python/private/venv_runfiles.bzl @@ -452,18 +452,11 @@ def get_venv_symlinks( def _is_linker_loaded_library(filename): """Tells if a filename is one that `dlopen()` or the runtime linker handles. - This should return true for regular C libraries, but false for Python - C extension modules. - - Python extensions: .so (linux, mac), .pyd (windows) - - C libraries: lib*.so (linux), lib*.so.* (linux), lib*.dylib (mac), .dll (windows) + C libraries: *.so (linux), *.so.* (linux), *.dylib (mac), .dll (windows) """ if filename.endswith(".dll"): return True - if filename.startswith("lib") and ( - filename.endswith((".so", ".dylib")) or ".so." in filename - ): + if filename.endswith((".so", ".dylib")) or ".so." in filename: return True return False diff --git a/tests/venv_site_packages_libs/app_files_building/app_files_building_tests.bzl b/tests/venv_site_packages_libs/app_files_building/app_files_building_tests.bzl index f85508dd3d..d808eae7e9 100644 --- a/tests/venv_site_packages_libs/app_files_building/app_files_building_tests.bzl +++ b/tests/venv_site_packages_libs/app_files_building/app_files_building_tests.bzl @@ -136,7 +136,7 @@ def _test_optimized_grouping_complex(name): name = name + "_files", paths = [ "site-packages/pkg1/a.txt", - "site-packages/pkg1/b/b_mod.so", + "site-packages/pkg1/b/b_mod_so", "site-packages/pkg1/c/c1.txt", "site-packages/pkg1/c/c2.txt", "site-packages/pkg1/d/d1.txt", @@ -147,6 +147,10 @@ def _test_optimized_grouping_complex(name): "site-packages/pkg1/q1/q2a/q3/q3a.txt", "site-packages/pkg1/q1/q2a/q3/q3b.txt", "site-packages/pkg1/q1/q2b/q2b.txt", + "site-packages/pkg1/q1/q2c/c_mod.so", + "site-packages/pkg1/q1/q2c/q2.txt", + "site-packages/pkg1/q1/q2c/q3/q3a.txt", + "site-packages/pkg1/q1/q2c/q3/q3b.txt", ], ) analysis_test( @@ -181,7 +185,7 @@ def _test_optimized_grouping_complex_impl(env, target): "pkg1/b", link_to_path = rr + "pkg1/b", files = [ - "tests/venv_site_packages_libs/app_files_building/site-packages/pkg1/b/b_mod.so", + "tests/venv_site_packages_libs/app_files_building/site-packages/pkg1/b/b_mod_so", ], ), _venv_symlink("pkg1/c", link_to_path = rr + "pkg1/c", files = [ @@ -210,6 +214,16 @@ def _test_optimized_grouping_complex_impl(env, target): _venv_symlink("pkg1/q1/q2b", link_to_path = rr + "pkg1/q1/q2b", files = [ "tests/venv_site_packages_libs/app_files_building/site-packages/pkg1/q1/q2b/q2b.txt", ]), + _venv_symlink("pkg1/q1/q2c/c_mod.so", link_to_path = rr + "pkg1/q1/q2c/c_mod.so", files = [ + "tests/venv_site_packages_libs/app_files_building/site-packages/pkg1/q1/q2c/c_mod.so", + ]), + _venv_symlink("pkg1/q1/q2c/q2.txt", link_to_path = rr + "pkg1/q1/q2c/q2.txt", files = [ + "tests/venv_site_packages_libs/app_files_building/site-packages/pkg1/q1/q2c/q2.txt", + ]), + _venv_symlink("pkg1/q1/q2c/q3", link_to_path = rr + "pkg1/q1/q2c/q3", files = [ + "tests/venv_site_packages_libs/app_files_building/site-packages/pkg1/q1/q2c/q3/q3a.txt", + "tests/venv_site_packages_libs/app_files_building/site-packages/pkg1/q1/q2c/q3/q3b.txt", + ]), ] expected = sorted(expected, key = lambda e: (e.link_to_path, e.venv_path)) env.expect.that_collection( @@ -226,7 +240,7 @@ def _test_optimized_grouping_single_toplevel(name): paths = [ "site-packages/pkg2/__init__.py", "site-packages/pkg2/a.txt", - "site-packages/pkg2/b_mod.so", + "site-packages/pkg2/b_mod_so", ], ) analysis_test( @@ -256,7 +270,7 @@ def _test_optimized_grouping_single_toplevel_impl(env, target): files = [ "tests/venv_site_packages_libs/app_files_building/site-packages/pkg2/__init__.py", "tests/venv_site_packages_libs/app_files_building/site-packages/pkg2/a.txt", - "tests/venv_site_packages_libs/app_files_building/site-packages/pkg2/b_mod.so", + "tests/venv_site_packages_libs/app_files_building/site-packages/pkg2/b_mod_so", ], ), ] From 7827e0b40978e5b397f7a7141a69405affb27a5a Mon Sep 17 00:00:00 2001 From: Thomas Desrosiers <681004+thomasdesr@users.noreply.github.com> Date: Sat, 24 Jan 2026 20:19:48 -0800 Subject: [PATCH 596/922] fix: handle unsubstituted template placeholders for external native py_binary (#3495) ### Problem In #3242, it looks like `rules_python` introduced new template placeholders in the bootstrap scripts: `%stage2_bootstrap%` and `%interpreter_args%`. And from what I can tell, also made their successful substitution a requirement. This works totally fine when the caller is calling `rules_python` directly. However, when external repositories (like gRPC's cython) define py_binary using the native rule, these placeholders don't seem to be substituted? The result is that the literal placeholder text ends up in the generated bootstrap scripts, causing `SyntaxError`s or file-not-found errors at runtime ### Fix Detect if the `%stage2_bootstrap%` variable isn't expanded and fallback to `%main%` which IS substituted even for native `py_binary`. For `%interpreter_args%`, wrap it in triple-quotes so it's hopefully always valid Python syntax, then detect the sentinel and default to an empty list. This is a bit hacky, but is fairly non-invasive. --------- Co-authored-by: Richard Levasseur --- python/private/py_executable.bzl | 5 +-- python/private/python_bootstrap_template.txt | 33 +++++++++++++++++--- 2 files changed, 30 insertions(+), 8 deletions(-) diff --git a/python/private/py_executable.bzl b/python/private/py_executable.bzl index 8b28b4ffae..c6215aabc3 100644 --- a/python/private/py_executable.bzl +++ b/python/private/py_executable.bzl @@ -728,10 +728,7 @@ def _create_stage1_bootstrap( resolve_python_binary_at_runtime = "1" subs = { - "%interpreter_args%": "\n".join([ - '"{}"'.format(v) - for v in ctx.attr.interpreter_args - ]), + "%interpreter_args%": "\n".join(ctx.attr.interpreter_args), "%is_zipfile%": "1" if is_for_zip else "0", "%python_binary%": python_binary_path, "%python_binary_actual%": python_binary_actual, diff --git a/python/private/python_bootstrap_template.txt b/python/private/python_bootstrap_template.txt index 9717756036..f2d5a42fda 100644 --- a/python/private/python_bootstrap_template.txt +++ b/python/private/python_bootstrap_template.txt @@ -10,10 +10,31 @@ import sys import os import subprocess import uuid - # runfiles-relative path +# NOTE: The sentinel strings are split (e.g., "%stage2" + "_bootstrap%") so that +# the substitution logic won't replace them. This allows runtime detection of +# unsubstituted placeholders, which occurs when native py_binary is used in +# external repositories. In that case, we fall back to %main% which Bazel's +# native rule does substitute. +_STAGE2_BOOTSTRAP_SENTINEL = "%stage2" + "_bootstrap%" STAGE2_BOOTSTRAP="%stage2_bootstrap%" +# NOTE: The fallback logic from stage2_bootstrap to main is only present +# as a courtesy for an older, unsupported, configuration. It can be removed +# when that case is unlikely to be a concern anymore. +# See https://github.com/bazel-contrib/rules_python/pull/3495 +if STAGE2_BOOTSTRAP == _STAGE2_BOOTSTRAP_SENTINEL: + _MAIN_SENTINEL = "%main" + "%" + _main = "%main%" + if _main != _MAIN_SENTINEL and _main: + STAGE2_BOOTSTRAP = _main + else: + STAGE2_BOOTSTRAP = "" + +if not STAGE2_BOOTSTRAP: + print("ERROR: %stage2_bootstrap% (or %main%) was not substituted.", file=sys.stderr) + sys.exit(1) + # runfiles-relative path to venv's python interpreter # Empty string if a venv is not setup. PYTHON_BINARY = '%python_binary%' @@ -35,9 +56,13 @@ RECREATE_VENV_AT_RUNTIME="%recreate_venv_at_runtime%" WORKSPACE_NAME = "%workspace_name%" # Target-specific interpreter args. -INTERPRETER_ARGS = [ -%interpreter_args% -] +# Sentinel split to detect unsubstituted placeholder (see STAGE2_BOOTSTRAP above). +_INTERPRETER_ARGS_SENTINEL = "%interpreter" + "_args%" +_INTERPRETER_ARGS_RAW = "%interpreter_args%" +if _INTERPRETER_ARGS_RAW == _INTERPRETER_ARGS_SENTINEL: + INTERPRETER_ARGS = [] +else: + INTERPRETER_ARGS = [arg for arg in _INTERPRETER_ARGS_RAW.split("\n") if arg] ADDITIONAL_INTERPRETER_ARGS = os.environ.get("RULES_PYTHON_ADDITIONAL_INTERPRETER_ARGS", "") From 728a808db9d547f9ff188c4187fffc104a552741 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Sun, 25 Jan 2026 00:03:43 -0800 Subject: [PATCH 597/922] refactor: rename files_to_build to default_outputs (#3542) The "files to build" term is the jargon used with the Bazel Java code, but in regular Starlark code, the term is "default outputs". This jargon leakage happened while porting the code from Java to Starlark. --- python/private/common.bzl | 21 ------------------- python/private/py_executable.bzl | 20 +++++++++++------- tests/base_rules/py_executable_base_tests.bzl | 8 +++---- .../py_library/py_library_tests.bzl | 8 +++---- 4 files changed, 20 insertions(+), 37 deletions(-) diff --git a/python/private/common.bzl b/python/private/common.bzl index 2d4afca3f5..527fbc5c7d 100644 --- a/python/private/common.bzl +++ b/python/private/common.bzl @@ -109,27 +109,6 @@ def create_cc_details_struct( **kwargs ) -def create_executable_result_struct(*, extra_files_to_build, output_groups, extra_runfiles = None): - """Creates a `CreateExecutableResult` struct. - - This is the return value type of the semantics create_executable function. - - Args: - extra_files_to_build: depset of File; additional files that should be - included as default outputs. - output_groups: dict[str, depset[File]]; additional output groups that - should be returned. - extra_runfiles: A runfiles object of additional runfiles to include. - - Returns: - A `CreateExecutableResult` struct. - """ - return struct( - extra_files_to_build = extra_files_to_build, - output_groups = output_groups, - extra_runfiles = extra_runfiles, - ) - def csv(values): """Convert a list of strings to comma separated value string.""" return ", ".join(sorted(values)) diff --git a/python/private/py_executable.bzl b/python/private/py_executable.bzl index c6215aabc3..c3ee1b92a8 100644 --- a/python/private/py_executable.bzl +++ b/python/private/py_executable.bzl @@ -42,7 +42,6 @@ load( "collect_runfiles", "create_binary_semantics_struct", "create_cc_details_struct", - "create_executable_result_struct", "create_instrumented_files_info", "create_output_group_info", "create_py_info", @@ -377,7 +376,7 @@ def _create_executable( runfiles = runfiles_details.runfiles_without_exe.merge(extra_runfiles), ) - extra_files_to_build = [] + extra_default_outputs = [] # NOTE: --build_python_zip defaults to true on Windows build_zip_enabled = read_possibly_native_flag(ctx, "build_python_zip") @@ -385,7 +384,7 @@ def _create_executable( # When --build_python_zip is enabled, then the zip file becomes # one of the default outputs. if build_zip_enabled: - extra_files_to_build.append(zip_file) + extra_default_outputs.append(zip_file) # The logic here is a bit convoluted. Essentially, there are 3 types of # executables produced: @@ -417,7 +416,7 @@ def _create_executable( # The launcher looks for the non-zip executable next to # itself, so add it to the default outputs. - extra_files_to_build.append(bootstrap_output) + extra_default_outputs.append(bootstrap_output) if should_create_executable_zip: if bootstrap_output != None: @@ -459,9 +458,14 @@ def _create_executable( # added to the zipped files. if venv and venv.interpreter: extra_runfiles = extra_runfiles.merge(ctx.runfiles([venv.interpreter])) - return create_executable_result_struct( - extra_files_to_build = depset(extra_files_to_build), + return struct( + # depset[File] of additional files that should be included as default + # outputs. + extra_default_outputs = depset(extra_default_outputs), + # dict[str, depset[File]]; additional output groups that should be + # returned. output_groups = {"python_zip_file": depset([zip_file])}, + # runfiles; additional runfiles to include. extra_runfiles = extra_runfiles, ) @@ -1075,10 +1079,10 @@ def py_executable_base_impl(ctx, *, semantics, is_test, inherited_environment = runfiles_details = runfiles_details, extra_deps = extra_deps, ) - default_outputs.add(exec_result.extra_files_to_build) + default_outputs.add(exec_result.extra_default_outputs) extra_exec_runfiles = exec_result.extra_runfiles.merge( - ctx.runfiles(transitive_files = exec_result.extra_files_to_build), + ctx.runfiles(transitive_files = exec_result.extra_default_outputs), ) # Copy any existing fields in case of company patches. diff --git a/tests/base_rules/py_executable_base_tests.bzl b/tests/base_rules/py_executable_base_tests.bzl index 83a470cdc5..49d5f49b86 100644 --- a/tests/base_rules/py_executable_base_tests.bzl +++ b/tests/base_rules/py_executable_base_tests.bzl @@ -377,7 +377,7 @@ def _test_explicit_main_cannot_be_ambiguous_impl(env, target): matching.str_matches("foo.py*matches multiple"), ) -def _test_files_to_build(name, config): +def _test_default_outputs(name, config): rt_util.helper_target( config.rule, name = name + "_subject", @@ -385,14 +385,14 @@ def _test_files_to_build(name, config): ) analysis_test( name = name, - impl = _test_files_to_build_impl, + impl = _test_default_outputs_impl, target = name + "_subject", attrs = WINDOWS_ATTR, ) -_tests.append(_test_files_to_build) +_tests.append(_test_default_outputs) -def _test_files_to_build_impl(env, target): +def _test_default_outputs_impl(env, target): default_outputs = env.expect.that_target(target).default_outputs() if pt_util.is_windows(env): default_outputs.contains("{package}/{test_name}_subject.exe") diff --git a/tests/base_rules/py_library/py_library_tests.bzl b/tests/base_rules/py_library/py_library_tests.bzl index 9b585b17ef..3726ff1f41 100644 --- a/tests/base_rules/py_library/py_library_tests.bzl +++ b/tests/base_rules/py_library/py_library_tests.bzl @@ -27,7 +27,7 @@ def _test_py_runtime_info_not_present_impl(env, target): _tests.append(_test_py_runtime_info_not_present) -def _test_files_to_build(name, config): +def _test_default_outputs(name, config): rt_util.helper_target( config.rule, name = name + "_subject", @@ -36,15 +36,15 @@ def _test_files_to_build(name, config): analysis_test( name = name, target = name + "_subject", - impl = _test_files_to_build_impl, + impl = _test_default_outputs_impl, ) -def _test_files_to_build_impl(env, target): +def _test_default_outputs_impl(env, target): env.expect.that_target(target).default_outputs().contains_exactly([ "{package}/lib.py", ]) -_tests.append(_test_files_to_build) +_tests.append(_test_default_outputs) def _test_srcs_can_contain_rule_generating_py_and_nonpy_files(name, config): rt_util.helper_target( From 89f7062913937e9a56ae715217154ddf586ed463 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Sun, 25 Jan 2026 00:07:02 -0800 Subject: [PATCH 598/922] refactor: clarify %main% is runfiles-root-relative path (#3537) I had to go dig through the source to figure out if `%main%` was being treated as a runfiles-root or main-repo-runfiles relative path. Add this to the variable description so its unambiguous. --- python/private/stage2_bootstrap_template.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/python/private/stage2_bootstrap_template.py b/python/private/stage2_bootstrap_template.py index 3595a43110..959e631ad1 100644 --- a/python/private/stage2_bootstrap_template.py +++ b/python/private/stage2_bootstrap_template.py @@ -27,7 +27,7 @@ # ===== Template substitutions start ===== # We just put them in one place so its easy to tell which are used. -# Runfiles-relative path to the main Python source file. +# Runfiles-root-relative path to the main Python source file. # Empty if MAIN_MODULE is used MAIN_PATH = "%main%" From 979bc936d8d629980fbdb809baccf5435f3da672 Mon Sep 17 00:00:00 2001 From: Aaron Sky Date: Sun, 25 Jan 2026 11:56:55 -0500 Subject: [PATCH 599/922] fix: Mark internal config repo as reproducible for Bzlmod (#3544) When I updated past 1.6.3, I noticed that my MODULE.bazel.lock was being updated with locking details of the internal dependencies of rules_python. My understanding is that `http_archive` and non-watched repository files tend to be easy to mark reproducible, which avoids unnecessary updates to the lockfile. I have verified this on an internal project that this does what I expect it to. --- python/extensions/BUILD.bazel | 1 + python/extensions/config.bzl | 10 ++++++++-- 2 files changed, 9 insertions(+), 2 deletions(-) diff --git a/python/extensions/BUILD.bazel b/python/extensions/BUILD.bazel index 12c0f248fe..50d2b3bbdf 100644 --- a/python/extensions/BUILD.bazel +++ b/python/extensions/BUILD.bazel @@ -47,5 +47,6 @@ bzl_library( deps = [ "//python/private:internal_config_repo_bzl", "//python/private/pypi:deps_bzl", + "@bazel_features//:features", ], ) diff --git a/python/extensions/config.bzl b/python/extensions/config.bzl index d8e621031e..f19a07aaef 100644 --- a/python/extensions/config.bzl +++ b/python/extensions/config.bzl @@ -1,5 +1,6 @@ """Extension for configuring global settings of rules_python.""" +load("@bazel_features//:features.bzl", "bazel_features") load("//python/private:internal_config_repo.bzl", "internal_config_repo") load("//python/private/pypi:deps.bzl", "pypi_deps") @@ -21,10 +22,10 @@ to repositories that are expensive to create or invalidate frequently. }, ) -def _config_impl(mctx): +def _config_impl(module_ctx): transition_setting_generators = {} transition_settings = [] - for mod in mctx.modules: + for mod in module_ctx.modules: for tag in mod.tags.add_transition_setting: setting = str(tag.setting) if setting not in transition_setting_generators: @@ -40,6 +41,11 @@ def _config_impl(mctx): pypi_deps() + if bazel_features.external_deps.extension_metadata_has_reproducible: + return module_ctx.extension_metadata(reproducible = True) + else: + return None + config = module_extension( doc = """Global settings for rules_python. From c6fde99885d3da53a0890e0c69837ed722eb45e5 Mon Sep 17 00:00:00 2001 From: Shayan Hoshyari <108962133+shayanhoshyari@users.noreply.github.com> Date: Mon, 26 Jan 2026 09:25:21 -0800 Subject: [PATCH 600/922] docs (debugger): Update using debuggers how to guide on using debugpy (e.g. vscode) (#3547) Thanks to help from @rickeylev in the discussion in https://github.com/bazel-contrib/rules_python/issues/3481 I was able to do a debugger integration running bazel test or bazel run straight in vscode (since it uses debugpy I suspect PyCharm should also work). Added this workflow to the docs, hoping others might also find it useful. Added page: https://rules-python--3547.org.readthedocs.build/en/3547/howto/debuggers.html Related to https://github.com/bazel-contrib/rules_python/discussions/3485 --------- Co-authored-by: Shayan Hoshyari Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> --- docs/environment-variables.md | 4 +- docs/howto/debuggers.md | 148 +++++++++++++++++++++++++++++++++- 2 files changed, 147 insertions(+), 5 deletions(-) diff --git a/docs/environment-variables.md b/docs/environment-variables.md index fba876f859..85cb766344 100644 --- a/docs/environment-variables.md +++ b/docs/environment-variables.md @@ -11,7 +11,7 @@ would be: python -Xaaa /path/to/file.py ``` -This feature is likely to be useful for the integration of debuggers. For example, +This feature is useful for the integration of debuggers. For example, it would be possible to configure `RULES_PYTHON_ADDITIONAL_INTERPRETER_ARGS` to be set to `/path/to/debugger.py --port 12344 --file`, resulting in the command executed being: @@ -22,6 +22,8 @@ python /path/to/debugger.py --port 12345 --file /path/to/file.py :::{seealso} The {bzl:obj}`interpreter_args` attribute. + +The guide on {any}`How to integrate a debugger` ::: :::{versionadded} 1.3.0 diff --git a/docs/howto/debuggers.md b/docs/howto/debuggers.md index 3f75712b0f..199a366675 100644 --- a/docs/howto/debuggers.md +++ b/docs/howto/debuggers.md @@ -3,10 +3,14 @@ # How to integrate a debugger -This guide explains how to use the {obj}`--debugger` flag to integrate a debugger +This guide explains how to integrate a debugger with your Python applications built with `rules_python`. -## Basic Usage +There are two ways available: the {obj}`--debugger` flag, and the {any}`RULES_PYTHON_ADDITIONAL_INTERPRETER_ARGS` environment variable. + +## {obj}`--debugger` flag + +### Basic Usage The {obj}`--debugger` flag allows you to inject an extra dependency into `py_test` and `py_binary` targets so that they have a custom debugger available at @@ -32,7 +36,7 @@ The specified target must be in the requirements.txt file used with `pip.parse()` to make it available to Bazel. ::: -## Python `PYTHONBREAKPOINT` Environment Variable +### Python `PYTHONBREAKPOINT` Environment Variable For more fine-grained control over debugging, especially for programmatic breakpoints, you can leverage the Python built-in `breakpoint()` function and the @@ -52,7 +56,7 @@ PYTHONBREAKPOINT=pudb.set_trace bazel run \ For more details on `PYTHONBREAKPOINT`, refer to the [Python documentation](https://docs.python.org/3/library/functions.html#breakpoint). -## Setting a default debugger +### Setting a default debugger By adding settings to your user or project `.bazelrc` files, you can have these settings automatically added to your bazel invocations. e.g. @@ -64,3 +68,139 @@ common --test_env=PYTHONBREAKPOINT=pudb.set_trace Note that `--test_env` isn't strictly necessary. The `py_test` and `py_binary` rules will respect the `PYTHONBREAKPOINT` environment variable in your shell. + +## debugpy (e.g. vscode) + +You can integrate `debugpy` (i.e. the debugger used in vscode or PyCharm) by using a launcher script. This method leverages {any}`RULES_PYTHON_ADDITIONAL_INTERPRETER_ARGS` to inject the debugger into the Bazel-managed Python process. + +For the remainder of this document, we assume you are using vscode. + +![VS Code debugpy demo](https://raw.githubusercontent.com/shayanhoshyari/issue-reports/refs/heads/main/rules_python/vscode_debugger/docs/demo.gif) + + +1. **Create a launcher script**: Save the following Python script as `.vscode/debugpy/launch.py` (or another location, adjusting `launch.json` accordingly). This script bridges VS Code's debugger with Bazel. + +
+ launch.py + + ```python + """ + Launcher script for VS Code (debugpy). + + This script is not managed by Bazel; it is invoked by VS Code's launch.json to + wrap the Bazel command, injecting the debugger into the runtime environment. + """ + + import argparse + import os + import shlex + import subprocess + import sys + from typing import cast + + def main() -> None: + parser = argparse.ArgumentParser(description="Launch bazel debugpy with test or run.") + parser.add_argument("mode", choices=["test", "run"], help="Choose whether to run a bazel test or run.") + parser.add_argument("args", help="The bazel target to test or run (e.g., //foo:bar) and any additional args") + args = parser.parse_args() + + # Import debugpy, provided by VS Code + try: + # debugpy._vendored is needed for force_pydevd to perform path manipulation. + import debugpy._vendored # type: ignore[import-not-found] + + # pydev_monkey patches os and subprocess functions to handle new launched processes. + from _pydev_bundle import pydev_monkey # type: ignore[import-not-found] + except ImportError as exc: + print(f"Error: This script must be run via VS Code's debug adapter. Details: {exc}") + sys.exit(-1) + + # Prepare arguments for the monkey-patched process. + # is_exec=False ensures we don't replace the current process immediately. + patched_args = cast(list[str], pydev_monkey.patch_args(["python", "dummy.py"], is_exec=False)) + pydev_monkey.send_process_created_message() + + # Extract the injected arguments (skipping the dummy python executable and script). + # These args invoke the pydevd entrypoint which connects back to the debugger. + rules_python_interpreter_args = " ".join(patched_args[1:-1]) + + bzl_args = shlex.split(args.args) + if not bzl_args: + print("Error: At least one argument (the target) is required.") + sys.exit(-1) + + cmd = [ + "bazel", + args.mode, + # Propagate environment variables to the test/run environment. + "--test_env=PYDEVD_RESOLVE_SYMLINKS", + "--test_env=RULES_PYTHON_ADDITIONAL_INTERPRETER_ARGS", + "--test_env=IDE_PROJECT_ROOTS", + bzl_args[0], + ] + + if bzl_args[1:]: + if args.mode == "run": + # Append extra arguments for 'run' mode. + cmd.append("--") + cmd.extend(bzl_args[1:]) + elif args.mode == "test": + # Append extra arguments for 'test' mode. + cmd.extend([f"--test_arg={arg}" for arg in bzl_args[1:]]) + + env = { + **os.environ.copy(), + # Inject the debugger arguments into the rules_python toolchain. + "RULES_PYTHON_ADDITIONAL_INTERPRETER_ARGS": rules_python_interpreter_args, + # Ensure breakpoints hit the original source files, not Bazel's symlinks. + "PYDEVD_RESOLVE_SYMLINKS": "1", + } + + # Execute Bazel. + result = subprocess.run(cmd, env=env, check=False) + sys.exit(result.returncode) + + if __name__ == "__main__": + main() + ``` +
+ +2. **Configure `launch.json`**: Add the following configurations to your `.vscode/launch.json`. This tells VS Code to use the launcher script. + +
+ launch.json + + ```json + { + "version": "0.2.0", + "configurations": [ + { + "name": "Python: Bazel py run", + "type": "debugpy", + "request": "launch", + "program": "${workspaceFolder}/.vscode/debugpy/launch.py", + "args": ["run", "${input:BazelArgs}"], + "console": "integratedTerminal" + }, + { + "name": "Python: Bazel py test", + "type": "debugpy", + "request": "launch", + "program": "${workspaceFolder}/.vscode/debugpy/launch.py", + "args": ["test", "${input:BazelArgs}"], + "console": "integratedTerminal" + } + ], + "inputs": [ + { + "id": "BazelArgs", + "type": "promptString", + "description": "Bazel target and arguments (e.g., //foo:bar --my-arg)" + } + ] + } + ``` +
+ + Note: If you find `justMyCode` behavior is incompatible with Bazel's symlinks (causing breakpoints to be missed), you can set `"justMyCode": false` in `launch.json` and use the `IDE_PROJECT_ROOTS` environment variable (set to `"${workspaceFolder}"`) to explicitly map your workspace. + From 717b94355eb768ab1ef02ce05a4f0592c050fc17 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Mon, 26 Jan 2026 20:48:28 -0800 Subject: [PATCH 601/922] chore: remove workspace py_proto_library example (#3546) Also update the deleted package file --- .bazelci/presubmit.yml | 27 -------------- .bazelrc.deleted_packages | 7 ---- examples/py_proto_library/.bazelrc | 4 --- examples/py_proto_library/.gitignore | 4 --- examples/py_proto_library/BUILD.bazel | 18 ---------- examples/py_proto_library/WORKSPACE | 36 ------------------- .../example.com/another_proto/BUILD.bazel | 16 --------- .../example.com/another_proto/message.proto | 10 ------ .../example.com/proto/BUILD.bazel | 17 --------- .../example.com/proto/pricetag.proto | 11 ------ examples/py_proto_library/message_test.py | 16 --------- examples/py_proto_library/test.py | 21 ----------- 12 files changed, 187 deletions(-) delete mode 100644 examples/py_proto_library/.bazelrc delete mode 100644 examples/py_proto_library/.gitignore delete mode 100644 examples/py_proto_library/BUILD.bazel delete mode 100644 examples/py_proto_library/WORKSPACE delete mode 100644 examples/py_proto_library/example.com/another_proto/BUILD.bazel delete mode 100644 examples/py_proto_library/example.com/another_proto/message.proto delete mode 100644 examples/py_proto_library/example.com/proto/BUILD.bazel delete mode 100644 examples/py_proto_library/example.com/proto/pricetag.proto delete mode 100644 examples/py_proto_library/message_test.py delete mode 100644 examples/py_proto_library/test.py diff --git a/.bazelci/presubmit.yml b/.bazelci/presubmit.yml index 6a5102d4f6..f31664a01c 100644 --- a/.bazelci/presubmit.yml +++ b/.bazelci/presubmit.yml @@ -421,33 +421,6 @@ tasks: # We don't run pip_parse_vendored under Windows as the file checked in is # generated from a repository rule containing OS-specific rendered paths. - # The proto example is workspace-only; bzlmod functionality is covered - # by examples/bzlmod/py_proto_library - integration_test_py_proto_library_ubuntu_workspace: - <<: *reusable_build_test_all - <<: *common_workspace_flags - name: "examples/py_proto_library: Ubuntu, workspace" - working_directory: examples/py_proto_library - platform: ubuntu2204 - integration_test_py_proto_library_debian_workspace: - <<: *reusable_build_test_all - <<: *common_workspace_flags - name: "examples/py_proto_library: Debian, workspace" - working_directory: examples/py_proto_library - platform: debian11 - integration_test_py_proto_library_macos_workspace: - <<: *reusable_build_test_all - <<: *common_workspace_flags - name: "examples/py_proto_library: MacOS, workspace" - working_directory: examples/py_proto_library - platform: macos_arm64 - integration_test_py_proto_library_windows_workspace: - <<: *reusable_build_test_all - <<: *common_workspace_flags - name: "examples/py_proto_library: Windows, workspace" - working_directory: examples/py_proto_library - platform: windows - integration_test_pip_repository_annotations_ubuntu_workspace: <<: *reusable_build_test_all <<: *common_workspace_flags diff --git a/.bazelrc.deleted_packages b/.bazelrc.deleted_packages index d11f96d664..2d8a8075fa 100644 --- a/.bazelrc.deleted_packages +++ b/.bazelrc.deleted_packages @@ -8,10 +8,6 @@ common --deleted_packages=examples/bzlmod/libs/my_lib common --deleted_packages=examples/bzlmod/other_module common --deleted_packages=examples/bzlmod/other_module/other_module/pkg common --deleted_packages=examples/bzlmod/patches -common --deleted_packages=examples/bzlmod/py_proto_library -common --deleted_packages=examples/bzlmod/py_proto_library/example.com/another_proto -common --deleted_packages=examples/bzlmod/py_proto_library/example.com/proto -common --deleted_packages=examples/bzlmod/py_proto_library/foo_external common --deleted_packages=examples/bzlmod/runfiles common --deleted_packages=examples/bzlmod/tests common --deleted_packages=examples/bzlmod/tests/other_module @@ -22,9 +18,6 @@ common --deleted_packages=examples/multi_python_versions/tests common --deleted_packages=examples/pip_parse common --deleted_packages=examples/pip_parse_vendored common --deleted_packages=examples/pip_repository_annotations -common --deleted_packages=examples/py_proto_library -common --deleted_packages=examples/py_proto_library/example.com/another_proto -common --deleted_packages=examples/py_proto_library/example.com/proto common --deleted_packages=gazelle common --deleted_packages=gazelle/examples/bzlmod_build_file_generation common --deleted_packages=gazelle/examples/bzlmod_build_file_generation/other_module/other_module/pkg diff --git a/examples/py_proto_library/.bazelrc b/examples/py_proto_library/.bazelrc deleted file mode 100644 index 2ed86f591e..0000000000 --- a/examples/py_proto_library/.bazelrc +++ /dev/null @@ -1,4 +0,0 @@ -# The equivalent bzlmod behavior is covered by examples/bzlmod/py_proto_library -common --noenable_bzlmod -common --enable_workspace -common --incompatible_python_disallow_native_rules diff --git a/examples/py_proto_library/.gitignore b/examples/py_proto_library/.gitignore deleted file mode 100644 index e5ae073b3c..0000000000 --- a/examples/py_proto_library/.gitignore +++ /dev/null @@ -1,4 +0,0 @@ -# git ignore patterns - -/bazel-* -user.bazelrc diff --git a/examples/py_proto_library/BUILD.bazel b/examples/py_proto_library/BUILD.bazel deleted file mode 100644 index b57c528511..0000000000 --- a/examples/py_proto_library/BUILD.bazel +++ /dev/null @@ -1,18 +0,0 @@ -load("@rules_python//python:py_test.bzl", "py_test") - -py_test( - name = "pricetag_test", - srcs = ["test.py"], - main = "test.py", - deps = [ - "//example.com/proto:pricetag_py_pb2", - ], -) - -py_test( - name = "message_test", - srcs = ["message_test.py"], - deps = [ - "//example.com/another_proto:message_py_pb2", - ], -) diff --git a/examples/py_proto_library/WORKSPACE b/examples/py_proto_library/WORKSPACE deleted file mode 100644 index 9cda5b97f1..0000000000 --- a/examples/py_proto_library/WORKSPACE +++ /dev/null @@ -1,36 +0,0 @@ -# NB: short workspace name is required to workaround PATH length limitation, see -# https://github.com/bazelbuild/bazel/issues/18683#issuecomment-1843857373 -workspace(name = "p") - -# The following local_path_override is only needed to run this example as part of our CI. -local_repository( - name = "rules_python", - path = "../..", -) - -# When not using this example in the rules_python git repo you would load the python -# rules using http_archive(), as documented in the release notes. - -load("@rules_python//python:repositories.bzl", "py_repositories", "python_register_toolchains") - -# We install the rules_python dependencies using the function below. -py_repositories() - -python_register_toolchains( - name = "python39", - python_version = "3.9", -) - -# Then we need to setup dependencies in order to use py_proto_library -load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive") - -http_archive( - name = "com_google_protobuf", - sha256 = "4fc5ff1b2c339fb86cd3a25f0b5311478ab081e65ad258c6789359cd84d421f8", - strip_prefix = "protobuf-26.1", - urls = ["https://github.com/protocolbuffers/protobuf/archive/v26.1.tar.gz"], -) - -load("@com_google_protobuf//:protobuf_deps.bzl", "protobuf_deps") - -protobuf_deps() diff --git a/examples/py_proto_library/example.com/another_proto/BUILD.bazel b/examples/py_proto_library/example.com/another_proto/BUILD.bazel deleted file mode 100644 index 55e83a209a..0000000000 --- a/examples/py_proto_library/example.com/another_proto/BUILD.bazel +++ /dev/null @@ -1,16 +0,0 @@ -load("@com_google_protobuf//bazel:proto_library.bzl", "proto_library") -load("@rules_python//python:proto.bzl", "py_proto_library") - -py_proto_library( - name = "message_py_pb2", - visibility = ["//visibility:public"], - deps = [":message_proto"], -) - -proto_library( - name = "message_proto", - srcs = ["message.proto"], - # https://bazel.build/reference/be/protocol-buffer#proto_library.strip_import_prefix - strip_import_prefix = "/example.com", - deps = ["//example.com/proto:pricetag_proto"], -) diff --git a/examples/py_proto_library/example.com/another_proto/message.proto b/examples/py_proto_library/example.com/another_proto/message.proto deleted file mode 100644 index 6e7dcc5793..0000000000 --- a/examples/py_proto_library/example.com/another_proto/message.proto +++ /dev/null @@ -1,10 +0,0 @@ -syntax = "proto3"; - -package rules_python; - -import "proto/pricetag.proto"; - -message TestMessage { - uint32 index = 1; - PriceTag pricetag = 2; -} diff --git a/examples/py_proto_library/example.com/proto/BUILD.bazel b/examples/py_proto_library/example.com/proto/BUILD.bazel deleted file mode 100644 index fdf2e6fe32..0000000000 --- a/examples/py_proto_library/example.com/proto/BUILD.bazel +++ /dev/null @@ -1,17 +0,0 @@ -load("@com_google_protobuf//bazel:proto_library.bzl", "proto_library") -load("@rules_python//python:proto.bzl", "py_proto_library") - -py_proto_library( - name = "pricetag_py_pb2", - visibility = ["//visibility:public"], - deps = [":pricetag_proto"], -) - -proto_library( - name = "pricetag_proto", - srcs = ["pricetag.proto"], - # https://bazel.build/reference/be/protocol-buffer#proto_library.strip_import_prefix - strip_import_prefix = "/example.com", - visibility = ["//visibility:public"], - deps = ["@com_google_protobuf//:any_proto"], -) diff --git a/examples/py_proto_library/example.com/proto/pricetag.proto b/examples/py_proto_library/example.com/proto/pricetag.proto deleted file mode 100644 index 3fa68de84b..0000000000 --- a/examples/py_proto_library/example.com/proto/pricetag.proto +++ /dev/null @@ -1,11 +0,0 @@ -syntax = "proto3"; - -import "google/protobuf/any.proto"; - -package rules_python; - -message PriceTag { - string name = 2; - double cost = 1; - google.protobuf.Any metadata = 3; -} diff --git a/examples/py_proto_library/message_test.py b/examples/py_proto_library/message_test.py deleted file mode 100644 index b1a6942a54..0000000000 --- a/examples/py_proto_library/message_test.py +++ /dev/null @@ -1,16 +0,0 @@ -import sys -import unittest - -from another_proto import message_pb2 - - -class TestCase(unittest.TestCase): - def test_message(self): - got = message_pb2.TestMessage( - index=5, - ) - self.assertIsNotNone(got) - - -if __name__ == "__main__": - sys.exit(unittest.main()) diff --git a/examples/py_proto_library/test.py b/examples/py_proto_library/test.py deleted file mode 100644 index 24ab8ddc70..0000000000 --- a/examples/py_proto_library/test.py +++ /dev/null @@ -1,21 +0,0 @@ -import json -import unittest - -from proto import pricetag_pb2 - - -class TestCase(unittest.TestCase): - def test_pricetag(self): - got = pricetag_pb2.PriceTag( - name="dollar", - cost=5.00, - ) - - metadata = {"description": "some text..."} - got.metadata.value = json.dumps(metadata).encode("utf-8") - - self.assertIsNotNone(got) - - -if __name__ == "__main__": - unittest.main() From 09acaaa17d3d2c16b6f2d791da7465208f7ae4c8 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Mon, 26 Jan 2026 22:39:43 -0800 Subject: [PATCH 602/922] chore: update rbe ci config to 8.x (#3548) There's two RBE configs: minimum, and current bazel. Both were testing 7.x, so upgrade the current one to 8.x --- .bazelci/presubmit.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.bazelci/presubmit.yml b/.bazelci/presubmit.yml index f31664a01c..6af15d9e34 100644 --- a/.bazelci/presubmit.yml +++ b/.bazelci/presubmit.yml @@ -282,7 +282,7 @@ tasks: platform: rbe_ubuntu2204 # TODO @aignas 2024-12-11: get the RBE working in CI for bazel 8.0 # See https://github.com/bazelbuild/rules_python/issues/2499 - bazel: 7.x + bazel: 8.x test_flags: - "--test_tag_filters=-integration-test,-acceptance-test" - "--extra_toolchains=@buildkite_config//config:cc-toolchain" From abe2699e3d4c8a523f82154afd101d5b26bc8f0b Mon Sep 17 00:00:00 2001 From: Ignas Anikevicius <240938+aignas@users.noreply.github.com> Date: Wed, 28 Jan 2026 00:11:07 +0900 Subject: [PATCH 603/922] fix(pip): simply extract whl contents to the current directory (#3549) Before we would try to get the whl path and extract to the sibling directory. Whilst this is not failing right now, technically this is not the best behaviour because the whl itself may come from elsewhere. This PR is simplifying code to see if it helps solve an issue on Windows. Fixes #3543 --- CHANGELOG.md | 8 ++++++++ python/private/pypi/whl_extract.bzl | 5 ++--- 2 files changed, 10 insertions(+), 3 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 683235b7f3..1cf27d07ae 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -83,6 +83,14 @@ END_UNRELEASED_TEMPLATE that are associated with the target's `srcs` are present. ([#3354](https://github.com/bazel-contrib/rules_python/issues/3354)). +{#v1-8-3} +## [1.8.3] - 2026-01-27 + +{#v1-8-3-fixed} +### Fixed +* (pipstar) Fix whl extraction on Windows when bazelrc has XX flags. + Fixes [#3543](https://github.com/bazel-contrib/rules_python/issues/3543). + {#v1-8-2} ## [1.8.2] - 2026-01-24 diff --git a/python/private/pypi/whl_extract.bzl b/python/private/pypi/whl_extract.bzl index 6b2e0507ac..75c5686cb8 100644 --- a/python/private/pypi/whl_extract.bzl +++ b/python/private/pypi/whl_extract.bzl @@ -12,7 +12,7 @@ def whl_extract(rctx, *, whl_path, logger): whl_path: the whl path to extract. logger: The logger to use """ - install_dir_path = whl_path.dirname.get_child("site-packages") + install_dir_path = rctx.path("site-packages") repo_utils.extract( rctx, archive = whl_path, @@ -30,7 +30,6 @@ def whl_extract(rctx, *, whl_path, logger): dist_info_dir.get_child("INSTALLER"), "https://github.com/bazel-contrib/rules_python#pipstar", ) - repo_root_dir = whl_path.dirname # Get the .dist_info dir name data_dir = dist_info_dir.dirname.get_child(dist_info_dir.basename[:-len(".dist-info")] + ".data") @@ -54,7 +53,7 @@ def whl_extract(rctx, *, whl_path, logger): # The prefix does not exist in the wheel, we can continue continue - for (src, dest) in merge_trees(src, repo_root_dir.get_child(dest_prefix)): + for (src, dest) in merge_trees(src, rctx.path(dest_prefix)): logger.debug(lambda: "Renaming: {} -> {}".format(src, dest)) rctx.rename(src, dest) From 32e8fb7363e96899dbdc82744da8b338c9181a86 Mon Sep 17 00:00:00 2001 From: vadikmironov <77026200+vadikmironov@users.noreply.github.com> Date: Fri, 30 Jan 2026 23:47:10 +0000 Subject: [PATCH 604/922] fix: use powershell.exe instead of pwsh.exe for build_data_writer (#3553) Stock Windows ships with Windows PowerShell 5.1 (powershell.exe) but not PowerShell 7+ (pwsh.exe). Using pwsh.exe causes builds to fail on machines without PowerShell 7 installed. Also adds -ExecutionPolicy Bypass since Windows PowerShell defaults to Restricted policy which blocks .ps1 script execution. Fixes #3552 --------- Co-authored-by: Claude Opus 4.5 Co-authored-by: Ignas Anikevicius <240938+aignas@users.noreply.github.com> Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> Co-authored-by: Richard Levasseur --- python/private/py_executable.bzl | 14 +++++++++++--- 1 file changed, 11 insertions(+), 3 deletions(-) diff --git a/python/private/py_executable.bzl b/python/private/py_executable.bzl index c3ee1b92a8..92fca2caf2 100644 --- a/python/private/py_executable.bzl +++ b/python/private/py_executable.bzl @@ -1369,9 +1369,17 @@ def _write_build_data(ctx): action_args = ctx.actions.args() writer_file = ctx.files._build_data_writer[0] if writer_file.path.endswith(".ps1"): - action_exe = "pwsh.exe" - action_args.add("-File") - action_args.add(writer_file) + # powershell.exe is used for broader compatibility + # It is installed by default on most Windows versions + action_exe = "powershell.exe" + action_args.add_all([ + # Bypass execution policy is needed because, + # by default, Windows blocks ps1 scripts. + "-ExecutionPolicy", + "Bypass", + "-File", + writer_file, + ]) inputs.add(writer_file) else: action_exe = ctx.attr._build_data_writer[DefaultInfo].files_to_run From bb75fc1dd955dce28fe6be8b95feeac5e3318495 Mon Sep 17 00:00:00 2001 From: vadikmironov <77026200+vadikmironov@users.noreply.github.com> Date: Sat, 31 Jan 2026 17:34:45 +0000 Subject: [PATCH 605/922] fix: remove CONFIG_ID write from build_data_writer.ps1 (#3556) Follow up from #3553 [comment](https://github.com/bazel-contrib/rules_python/pull/3553#issuecomment-3828794485) - removes extraneous CONFIG_ID write from build_data_writer.ps1 script. --- python/private/build_data_writer.ps1 | 1 - 1 file changed, 1 deletion(-) diff --git a/python/private/build_data_writer.ps1 b/python/private/build_data_writer.ps1 index 384d1ce539..db7a48e676 100644 --- a/python/private/build_data_writer.ps1 +++ b/python/private/build_data_writer.ps1 @@ -1,7 +1,6 @@ $OutputPath = $env:OUTPUT Add-Content -Path $OutputPath -Value "TARGET $env:TARGET" -Add-Content -Path $OutputPath -Value "CONFIG_ID $env:CONFIG_ID" Add-Content -Path $OutputPath -Value "CONFIG_MODE $env:CONFIG_MODE" Add-Content -Path $OutputPath -Value "STAMPED $env:STAMPED" From 96a16531a93a652de908e70f6b1037d1cb3fe7c3 Mon Sep 17 00:00:00 2001 From: Ted Kaplan Date: Sun, 1 Feb 2026 12:00:39 -0800 Subject: [PATCH 606/922] fix(pip): add read permissions when extracting wheels (#3555) Fix for wheels like https://files.pythonhosted.org/packages/ad/39/da8b5c0f875ccb1770349caaecd87a253949ccbcdc2c869929919d744551/ag_ui_adk-0.4.2-py3-none-any.whl where the contents do not have the read bit set. Fixes #3554 --------- Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> Co-authored-by: Richard Levasseur --- python/private/pypi/whl_extract.bzl | 17 +++++++++++++++++ 1 file changed, 17 insertions(+) diff --git a/python/private/pypi/whl_extract.bzl b/python/private/pypi/whl_extract.bzl index 75c5686cb8..506be05481 100644 --- a/python/private/pypi/whl_extract.bzl +++ b/python/private/pypi/whl_extract.bzl @@ -19,6 +19,23 @@ def whl_extract(rctx, *, whl_path, logger): output = install_dir_path, supports_whl_extraction = rp_config.supports_whl_extraction, ) + + # Fix permissions on extracted files. Some wheels have files without read permissions set, + # which causes errors when trying to read them later. + os_name = repo_utils.get_platforms_os_name(rctx) + if os_name != "windows": + # On Unix-like systems, recursively add read permissions to all files + # and ensure directories are traversable (need execute permission) + result = repo_utils.execute_unchecked( + rctx, + op = "Fixing wheel permissions {}".format(whl_path), + arguments = ["chmod", "-R", "a+rX", str(install_dir_path)], + logger = logger, + ) + if result.return_code != 0: + # It's possible chmod is not available or the filesystem doesn't support it. + # This is fine, we just want to try to fix permissions if possible. + logger.warn(lambda: "Failed to fix file permissions: {}".format(result.stderr)) metadata_file = find_whl_metadata( install_dir = install_dir_path, logger = logger, From 81e6e9c0cc030d62e3c0bfc2bc7eced1c3b6d13e Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Sun, 1 Feb 2026 12:01:59 -0800 Subject: [PATCH 607/922] feat: add py_zipapp_binary and test rules for zipapp support (#3539) This adds Python zipapp support as a separate rule that works with a py_binary. This will eventually replace the zipapp logic that is directly part of py_binary. The high level behavior should be the same, but there's a few differences and improvements: Changes: * Output name is `{name}.pyz` * Whether the output is executable is controlled by attribute, not flag. Improvements: * Compression level can be explicitly controlled. * Re-unzipping optimization: only unzip if the zip's hash changed * Zip output is more deterministic (input order doesn't affect output) * runfiles.symlinks, runfiles.root_symlinks, and raw symlinks are supported * Zipping venv-based binary is supported * Lower analysis-phase cost * Self-execution doesn't require python itself For now, this lacks Windows support for lack of time (it requires plumbing in the "windows exe maker" stuff or equivalent for self-executable zip support). Along the way: * Add PyExecToolsInfo.exec_runtime: an easier way to get the RBE-compatible runtime * Add PyExecutableInfo fields to aid packaging a binary. * Set `--verbose_failures` in CI config to give more info when actions fail. * Make `--visible_for_testing` a command line flag. This makes it a bit easier to debug toolchain issues by showing the source of the toolchain. * Add `actions_run` helper: this makes it easier to use py scripts as build actions. Work towards https://github.com/bazel-contrib/rules_python/issues/3324 https://github.com/bazel-contrib/rules_python/issues/2586 --- .bazelci/presubmit.yml | 1 + CHANGELOG.md | 11 + docs/BUILD.bazel | 3 + python/BUILD.bazel | 1 + python/private/BUILD.bazel | 26 +- python/private/common.bzl | 148 ++++++++ python/private/py_exec_tools_info.bzl | 20 +- python/private/py_exec_tools_toolchain.bzl | 12 + python/private/py_executable.bzl | 115 ++++--- python/private/py_executable_info.bzl | 45 +++ python/private/py_interpreter_program.bzl | 22 +- python/private/py_runtime_rule.bzl | 2 +- python/private/zipapp/BUILD.bazel | 49 +++ python/private/zipapp/py_zipapp_rule.bzl | 317 ++++++++++++++++++ .../private/{ => zipapp}/zip_main_template.py | 12 +- python/private/zipapp/zip_shell_template.sh | 88 +++++ .../zipapp_stage2_bootstrap_template.py | 10 + python/zipapp/BUILD.bazel | 29 ++ python/zipapp/py_zipapp_binary.bzl | 15 + python/zipapp/py_zipapp_test.bzl | 15 + tests/base_rules/py_executable_base_tests.bzl | 3 +- .../transition/multi_version_tests.bzl | 3 +- tests/py_zipapp/BUILD.bazel | 67 ++++ tests/py_zipapp/main.py | 12 + tests/py_zipapp/system_python_zipapp_test.py | 25 ++ tests/py_zipapp/venv_zipapp_test.py | 73 ++++ tests/support/support.bzl | 7 +- tests/tools/zipapp/BUILD.bazel | 13 + tests/tools/zipapp/exe_zip_maker_test.py | 71 ++++ tests/tools/zipapp/zipper_test.py | 220 ++++++++++++ tools/private/zipapp/BUILD.bazel | 36 ++ tools/private/zipapp/exe_zip_maker.py | 40 +++ tools/private/zipapp/zipper.py | 230 +++++++++++++ 33 files changed, 1667 insertions(+), 74 deletions(-) create mode 100644 python/private/zipapp/BUILD.bazel create mode 100644 python/private/zipapp/py_zipapp_rule.bzl rename python/private/{ => zipapp}/zip_main_template.py (96%) create mode 100644 python/private/zipapp/zip_shell_template.sh create mode 100644 python/private/zipapp/zipapp_stage2_bootstrap_template.py create mode 100644 python/zipapp/BUILD.bazel create mode 100644 python/zipapp/py_zipapp_binary.bzl create mode 100644 python/zipapp/py_zipapp_test.bzl create mode 100644 tests/py_zipapp/BUILD.bazel create mode 100644 tests/py_zipapp/main.py create mode 100644 tests/py_zipapp/system_python_zipapp_test.py create mode 100644 tests/py_zipapp/venv_zipapp_test.py create mode 100644 tests/tools/zipapp/BUILD.bazel create mode 100644 tests/tools/zipapp/exe_zip_maker_test.py create mode 100644 tests/tools/zipapp/zipper_test.py create mode 100644 tools/private/zipapp/BUILD.bazel create mode 100644 tools/private/zipapp/exe_zip_maker.py create mode 100644 tools/private/zipapp/zipper.py diff --git a/.bazelci/presubmit.yml b/.bazelci/presubmit.yml index 6af15d9e34..630638ee80 100644 --- a/.bazelci/presubmit.yml +++ b/.bazelci/presubmit.yml @@ -34,6 +34,7 @@ buildifier: build_flags: - "--keep_going" - "--build_tag_filters=-integration-test" + - "--verbose_failures" test_targets: - "--" - "..." diff --git a/CHANGELOG.md b/CHANGELOG.md index 1cf27d07ae..d71b6e65b2 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -82,6 +82,17 @@ END_UNRELEASED_TEMPLATE `true`, a `py_*` target's `pyi_srcs` attribute will be set if any `.pyi` files that are associated with the target's `srcs` are present. ([#3354](https://github.com/bazel-contrib/rules_python/issues/3354)). +* (zipapp) {obj}`py_zipapp_binary` and {obj}`py_zipapp_test` rules added. These + will replace `--build_python_zip` and the zip output group of + `py_binary/py_test`. The zipapp rules support more functionality, correctness, + and have better build performance. +* (toolchains) Added {obj}`PyExecToolsInfo.exec_runtime` for more easily + getting an RBE-compatible runtime to use for build actions. +* (providers) {obj}`PyExecutableInfo` has several new fields to aid packaging + of binaries: {obj}`PyExecutableInfo.app_runfiles`, + {obj}`PyExecutableInfo.interpreter_args`, + {obj}`PyExecutableInfo.stage2_bootstrap`, and + {obj}`PyExecutableInfo.venv_python_exe`. {#v1-8-3} ## [1.8.3] - 2026-01-27 diff --git a/docs/BUILD.bazel b/docs/BUILD.bazel index 55d8f75a74..632c3dd613 100644 --- a/docs/BUILD.bazel +++ b/docs/BUILD.bazel @@ -125,10 +125,13 @@ sphinx_stardocs( "//python/private/pypi:pkg_aliases_bzl", "//python/private/pypi:whl_config_setting_bzl", "//python/private/pypi:whl_library_bzl", + "//python/private/zipapp:py_zipapp_rule_bzl", "//python/uv:lock_bzl", "//python/uv:uv_bzl", "//python/uv:uv_toolchain_bzl", "//python/uv:uv_toolchain_info_bzl", + "//python/zipapp:py_zipapp_binary_bzl", + "//python/zipapp:py_zipapp_test_bzl", ] + ([ # This depends on @pythons_hub, which is only created under bzlmod, "//python/extensions:pip_bzl", diff --git a/python/BUILD.bazel b/python/BUILD.bazel index 5c4626ee55..6577ff805f 100644 --- a/python/BUILD.bazel +++ b/python/BUILD.bazel @@ -47,6 +47,7 @@ filegroup( "//python/runfiles:distribution", "//python/runtime_env_toolchains:distribution", "//python/uv:distribution", + "//python/zipapp:distribution", ], visibility = ["//:__pkg__"], ) diff --git a/python/private/BUILD.bazel b/python/private/BUILD.bazel index ac97ef9b50..70f7f86413 100644 --- a/python/private/BUILD.bazel +++ b/python/private/BUILD.bazel @@ -13,7 +13,7 @@ # limitations under the License. load("@bazel_skylib//:bzl_library.bzl", "bzl_library") -load("@bazel_skylib//rules:common_settings.bzl", "bool_setting") +load("@bazel_skylib//rules:common_settings.bzl", "bool_flag") load("//python:py_binary.bzl", "py_binary") load("//python:py_library.bzl", "py_library") load(":bazel_config_mode.bzl", "bazel_config_mode") @@ -38,6 +38,7 @@ filegroup( "//python/private/cc:distribution", "//python/private/pypi:distribution", "//python/private/whl_filegroup:distribution", + "//python/private/zipapp:distribution", "//tools/build_defs/python/private:distribution", ], visibility = ["//python:__pkg__"], @@ -77,7 +78,9 @@ bzl_library( ":py_info_bzl", ":py_internal_bzl", ":reexports_bzl", + ":rule_builders_bzl", ":rules_cc_srcs_bzl", + "@bazel_skylib//lib:dicts", "@bazel_skylib//rules:common_settings", ], ) @@ -366,7 +369,7 @@ bzl_library( name = "py_cc_toolchain_rule_bzl", srcs = ["py_cc_toolchain_rule.bzl"], deps = [ - ":common_labels.bzl", + ":common_labels_bzl", ":py_cc_toolchain_info_bzl", ":rules_cc_srcs_bzl", ":sentinel_bzl", @@ -460,7 +463,10 @@ bzl_library( bzl_library( name = "py_interpreter_program_bzl", srcs = ["py_interpreter_program.bzl"], - deps = ["@bazel_skylib//rules:common_settings"], + deps = [ + ":sentinel_bzl", + "@bazel_skylib//rules:common_settings", + ], ) bzl_library( @@ -740,8 +746,8 @@ bzl_library( srcs = ["venv_runfiles.bzl"], deps = [ ":common_bzl", - ":py_info.bzl", - ":py_internal.bzl", + ":py_info_bzl", + ":py_internal_bzl", "@bazel_skylib//lib:paths", ], ) @@ -786,14 +792,6 @@ filegroup( visibility = ["//visibility:public"], ) -filegroup( - name = "zip_main_template", - srcs = ["zip_main_template.py"], - # Not actually public. Only public because it's an implicit dependency of - # py_runtime. - visibility = ["//visibility:public"], -) - filegroup( name = "site_init_template", srcs = ["site_init_template.py"], @@ -845,7 +843,7 @@ bazel_config_mode(name = "bazel_config_mode") # This should only be set by analysis tests to expose additional metadata to # aid testing, so a setting instead of a flag. -bool_setting( +bool_flag( name = "visible_for_testing", build_setting_default = False, # This is only because it is an implicit dependency by the toolchains. diff --git a/python/private/common.bzl b/python/private/common.bzl index 527fbc5c7d..ca201cb10f 100644 --- a/python/private/common.bzl +++ b/python/private/common.bzl @@ -16,6 +16,10 @@ load("@bazel_skylib//lib:paths.bzl", "paths") load("@rules_cc//cc/common:cc_common.bzl", "cc_common") load("@rules_cc//cc/common:cc_info.bzl", "CcInfo") +load("@rules_python_internal//:rules_python_config.bzl", "config") +load("//python/private:py_interpreter_program.bzl", "PyInterpreterProgramInfo") +load("//python/private:toolchain_types.bzl", "EXEC_TOOLS_TOOLCHAIN_TYPE") +load(":builders.bzl", "builders") load(":cc_helper.bzl", "cc_helper") load(":py_cc_link_params_info.bzl", "PyCcLinkParamsInfo") load(":py_info.bzl", "PyInfo", "PyInfoBuilder") @@ -41,6 +45,17 @@ PYTHON_FILE_EXTENSIONS = [ "so", # Python C modules, usually Linux ] +BUILTIN_BUILD_PYTHON_ZIP = [] if config.bazel_10_or_later else [ + "//command_line_option:build_python_zip", +] + +def maybe_builtin_build_python_zip(value, settings = None): + settings = settings or {} + if not config.bazel_10_or_later: + settings["//command_line_option:build_python_zip"] = value + + return settings + def create_binary_semantics_struct( *, get_native_deps_dso_name, @@ -463,3 +478,136 @@ def collect_deps(ctx, extra_deps = []): deps = list(deps) deps.extend(extra_deps) return deps + +def maybe_create_repo_mapping(ctx, *, runfiles): + """Creates a repo mapping manifest if bzlmod is enabled. + + There isn't a way to reference the repo mapping Bazel implicitly + creates, so we have to manually create it ourselves. + + Args: + ctx: rule ctx. + runfiles: runfiles object to generate mapping for. + + Returns: + File object if the repo mapping manifest was created, None otherwise. + """ + if not py_internal.is_bzlmod_enabled(ctx): + return None + + # We have to add `.custom` because `{name}.repo_mapping` is used by Bazel + # internally. + repo_mapping_manifest = ctx.actions.declare_file(ctx.label.name + ".custom.repo_mapping") + py_internal.create_repo_mapping_manifest( + ctx = ctx, + runfiles = runfiles, + output = repo_mapping_manifest, + ) + return repo_mapping_manifest + +def actions_run( + ctx, + *, + executable, + toolchain = None, + **kwargs): + """Runs a tool as an action, supporting py_interpreter_program targets. + + This is wrapper around `ctx.actions.run()` that sets some useful defaults, + supports handling `py_interpreter_program` targets, and some other features + to let the target being run influence the action invocation. + + Args: + ctx: The rule context. The rule must have the + `//python:exec_tools_toolchain_type` toolchain available. + executable: The executable to run. This can be a target that provides + `PyInterpreterProgramInfo` or a regular executable target. If it + provides `testing.ExecutionInfo`, the requirements will be added to + the execution requirements. + toolchain: The toolchain type to use. Must be None or + `//python:exec_tools_toolchain_type`. + **kwargs: Additional arguments to pass to `ctx.actions.run()`. + `mnemonic` and `progress_message` are required. + """ + mnemonic = kwargs.pop("mnemonic", None) + if not mnemonic: + fail("actions_run: missing required argument 'mnemonic'") + + progress_message = kwargs.pop("progress_message", None) + if not progress_message: + fail("actions_run: missing required argument 'progress_message'") + + tools = kwargs.pop("tools", None) + tools = list(tools) if tools else [] + + action_arguments = [] + action_env = { + "PYTHONHASHSEED": "0", # Helps avoid non-deterministic behavior + "PYTHONNOUSERSITE": "1", # Helps avoid non-deterministic behavior + "PYTHONSAFEPATH": "1", # Helps avoid incorrect import issues + } + default_info = executable[DefaultInfo] + action_inputs = builders.DepsetBuilder() + action_inputs.add(kwargs.pop("inputs", None) or []) + if PyInterpreterProgramInfo in executable: + if toolchain and toolchain != EXEC_TOOLS_TOOLCHAIN_TYPE: + fail(("Action {}: tool {} provides PyInterpreterProgramInfo, which " + + "requires the `toolchain` arg be " + + "None or {}, got: {}").format( + mnemonic, + executable, + EXEC_TOOLS_TOOLCHAIN_TYPE, + toolchain, + )) + exec_runtime = ctx.toolchains[EXEC_TOOLS_TOOLCHAIN_TYPE].exec_tools.exec_runtime + if exec_runtime.interpreter: + action_exe = exec_runtime.interpreter + action_inputs.add(exec_runtime.files) + elif exec_runtime.interpreter_path: + action_exe = exec_runtime.interpreter_path + else: + fail(("Action {}: PyRuntimeInfo from exec tools toolchain is " + + "malformed: requires one of `interpreter` or " + + "`interpreter_path` set").format( + mnemonic, + )) + + program_info = executable[PyInterpreterProgramInfo] + + interpreter_args = ctx.actions.args() + interpreter_args.add_all(program_info.interpreter_args) + interpreter_args.add(default_info.files_to_run.executable) + action_arguments.append(interpreter_args) + + action_env.update(program_info.env) + + tools.append(default_info.files_to_run) + toolchain = EXEC_TOOLS_TOOLCHAIN_TYPE + else: + action_exe = executable[DefaultInfo].files_to_run + + execution_requirements = {} + if testing.ExecutionInfo in executable: + execution_requirements.update(executable[testing.ExecutionInfo].requirements) + + # Give precedence to caller's execution requirements. + execution_requirements.update(kwargs.pop("execution_requirements", None) or {}) + + # Give precedence to caller's env. + action_env.update(kwargs.pop("env", None) or {}) + + # Handle arguments=None + action_arguments.extend(list(kwargs.pop("arguments", None) or [])) + + ctx.actions.run( + executable = action_exe, + arguments = action_arguments, + tools = tools, + env = action_env, + execution_requirements = execution_requirements, + toolchain = toolchain, + mnemonic = mnemonic, + progress_message = progress_message, + inputs = action_inputs.build(), + **kwargs + ) diff --git a/python/private/py_exec_tools_info.bzl b/python/private/py_exec_tools_info.bzl index ad9a7b0c5e..16a628d822 100644 --- a/python/private/py_exec_tools_info.bzl +++ b/python/private/py_exec_tools_info.bzl @@ -26,7 +26,7 @@ e.g. if all the exec tools are prebuilt binaries. :::{note} this interpreter is really only for use when a build tool cannot use -the Python toolchain itself. When possible, prefeer to define a `py_binary` +the Python toolchain itself. When possible, prefer to define a `py_binary` instead and use it via a `cfg=exec` attribute; this makes it much easier to setup the runtime environment for the binary. See also: `py_interpreter_program` rule. @@ -39,11 +39,27 @@ toolchain. ::: :::{warning} -This does not work correctly in case of RBE, please use exec_runtime instead. +This does not work correctly with RBE. Use {obj}`exec_runtime` instead. Once https://github.com/bazelbuild/bazel/issues/23620 is resolved this warning may be removed. ::: +""", + "exec_runtime": """ +:type: PyRuntimeInfo | None + +The Python runtime to use for the exec configuration. + +:::{versionadded} VERSION_NEXT_FEATURE + +In prior versions, the equivalent can be obtained using: +``` +exec_runtime = ( + ctx.toolchains["@rules_python//python:exec_tools_toolchain_type"]. + exec_tools.exec_interpreter[platform_common.ToolchainInfo].py3_runtime + ) +``` +::: """, "precompiler": """ :type: Target | None diff --git a/python/private/py_exec_tools_toolchain.bzl b/python/private/py_exec_tools_toolchain.bzl index 00ad8072f6..ec8d4e53d0 100644 --- a/python/private/py_exec_tools_toolchain.bzl +++ b/python/private/py_exec_tools_toolchain.bzl @@ -30,11 +30,17 @@ def _py_exec_tools_toolchain_impl(ctx): if SentinelInfo in ctx.attr.exec_interpreter: exec_interpreter = None + exec_runtime = None + if exec_interpreter != None and platform_common.ToolchainInfo in exec_interpreter: + tc = exec_interpreter[platform_common.ToolchainInfo] + exec_runtime = getattr(tc, "py3_runtime", None) + return [ platform_common.ToolchainInfo( exec_tools = PyExecToolsInfo( exec_interpreter = exec_interpreter, precompiler = ctx.attr.precompiler, + exec_runtime = exec_runtime, ), **extra_kwargs ), @@ -104,6 +110,12 @@ def _current_interpreter_executable_impl(ctx): # re-exec. If it's not a recognized name, then they fail. if runtime.interpreter: executable = ctx.actions.declare_file(runtime.interpreter.basename) + + # NOTE: Using ctx.actions.symlink() here doesn't always work with RBE + # because it's not guaranteed that it will materialize as a symlink, but + # we rely on it being a symlink so that Python can find its actual + # PYTHONHOME. + # See https://github.com/bazelbuild/bazel/issues/23620 ctx.actions.symlink(output = executable, target_file = runtime.interpreter, is_executable = True) else: executable = ctx.actions.declare_symlink(paths.basename(runtime.interpreter_path)) diff --git a/python/private/py_executable.bzl b/python/private/py_executable.bzl index 92fca2caf2..3dd1b28234 100644 --- a/python/private/py_executable.bzl +++ b/python/private/py_executable.bzl @@ -48,6 +48,7 @@ load( "csv", "filter_to_py_srcs", "is_bool", + "maybe_create_repo_mapping", "relative_path", "runfiles_root_path", "target_platform_has_any_constraint", @@ -454,6 +455,12 @@ def _create_executable( build_zip_enabled = build_zip_enabled, )) + app_runfiles = builders.RunfilesBuilder() + app_runfiles.add(runfiles_details.app_runfiles) + if venv: + app_runfiles.add(venv.files_without_interpreter) + app_runfiles.add(venv.lib_runfiles) + # The interpreter is added this late in the process so that it isn't # added to the zipped files. if venv and venv.interpreter: @@ -467,6 +474,13 @@ def _create_executable( output_groups = {"python_zip_file": depset([zip_file])}, # runfiles; additional runfiles to include. extra_runfiles = extra_runfiles, + # File|None; the stage2 bootstrap file, if any + stage2_bootstrap = stage2_bootstrap, + # runfiles; runfiles for the app itself (e.g its deps, but no Python + # runtime files) + app_runfiles = app_runfiles.build(ctx), + # File|None; the venv `bin/python3` file, if any. + venv_python_exe = venv.interpreter if venv else None, ) def _create_zip_main(ctx, *, stage2_bootstrap, runtime_details, venv): @@ -859,16 +873,11 @@ def _create_zip_file(ctx, *, output, zip_main, runfiles): manifest.add_all(runfiles.files, map_each = map_zip_runfiles, allow_closure = True) inputs = [zip_main] - if _py_builtins.is_bzlmod_enabled(ctx): - zip_repo_mapping_manifest = ctx.actions.declare_file( - output.basename + ".repo_mapping", - sibling = output, - ) - _py_builtins.create_repo_mapping_manifest( - ctx = ctx, - runfiles = runfiles, - output = zip_repo_mapping_manifest, - ) + zip_repo_mapping_manifest = maybe_create_repo_mapping( + ctx = ctx, + runfiles = runfiles, + ) + if zip_repo_mapping_manifest: manifest.add("{}/_repo_mapping={}".format( _ZIP_RUNFILES_DIRECTORY_NAME, zip_repo_mapping_manifest.path, @@ -1061,8 +1070,8 @@ def py_executable_base_impl(ctx, *, semantics, is_test, inherited_environment = required_pyc_files = required_pyc_files, implicit_pyc_files = implicit_pyc_files, implicit_pyc_source_files = implicit_pyc_source_files, + runtime_runfiles = runtime_details.runfiles, extra_common_runfiles = [ - runtime_details.runfiles, cc_details.extra_runfiles, native_deps_details.runfiles, ], @@ -1093,6 +1102,8 @@ def py_executable_base_impl(ctx, *, semantics, is_test, inherited_environment = ) )) + app_runfiles = exec_result.app_runfiles + return _create_providers( ctx = ctx, executable = executable, @@ -1109,6 +1120,10 @@ def py_executable_base_impl(ctx, *, semantics, is_test, inherited_environment = cc_info = cc_details.cc_info_for_propagating, inherited_environment = inherited_environment, output_groups = exec_result.output_groups, + stage2_bootstrap = exec_result.stage2_bootstrap, + app_runfiles = app_runfiles, + venv_python_exe = exec_result.venv_python_exe, + interpreter_args = ctx.attr.interpreter_args, ) def _get_build_info(ctx, cc_toolchain): @@ -1238,6 +1253,7 @@ def _get_base_runfiles_for_binary( required_pyc_files, implicit_pyc_files, implicit_pyc_source_files, + runtime_runfiles, extra_common_runfiles): """Returns the set of runfiles necessary prior to executable creation. @@ -1258,6 +1274,7 @@ def _get_base_runfiles_for_binary( collection is enabled. implicit_pyc_source_files: `depset[File]` source files for implicit pyc files that are used when the implicit pyc files are not. + runtime_runfiles: runfiles for the python runtime. extra_common_runfiles: List of runfiles; additional runfiles that will be added to the common runfiles. @@ -1267,60 +1284,70 @@ def _get_base_runfiles_for_binary( * data_runfiles: The data runfiles * runfiles_without_exe: The default runfiles, but without the executable or files specific to the original program/executable. - * build_data_file: A file with build stamp information if stamping is enabled, otherwise - None. + * build_data_file: A file with build stamp information if stamping is + enabled, otherwise None. + * app_runfiles: Runfiles for user-space dependencies (doesn't + include the runtime or build data files) """ - common_runfiles = builders.RunfilesBuilder() - common_runfiles.files.add(required_py_files) - common_runfiles.files.add(required_pyc_files) + app_runfiles = builders.RunfilesBuilder() + app_runfiles.files.add(required_py_files) + app_runfiles.files.add(required_pyc_files) pyc_collection_enabled = PycCollectionAttr.is_pyc_collection_enabled(ctx) if pyc_collection_enabled: - common_runfiles.files.add(implicit_pyc_files) + app_runfiles.files.add(implicit_pyc_files) else: - common_runfiles.files.add(implicit_pyc_source_files) + app_runfiles.files.add(implicit_pyc_source_files) for dep in (ctx.attr.deps + extra_deps): if not (PyInfo in dep or (BuiltinPyInfo != None and BuiltinPyInfo in dep)): continue info = dep[PyInfo] if PyInfo in dep else dep[BuiltinPyInfo] - common_runfiles.files.add(info.transitive_sources) + app_runfiles.files.add(info.transitive_sources) # Everything past this won't work with BuiltinPyInfo if not hasattr(info, "transitive_pyc_files"): continue - common_runfiles.files.add(info.transitive_pyc_files) + app_runfiles.files.add(info.transitive_pyc_files) if pyc_collection_enabled: - common_runfiles.files.add(info.transitive_implicit_pyc_files) + app_runfiles.files.add(info.transitive_implicit_pyc_files) else: - common_runfiles.files.add(info.transitive_implicit_pyc_source_files) + app_runfiles.files.add(info.transitive_implicit_pyc_source_files) - common_runfiles.runfiles.append(collect_runfiles(ctx)) + app_runfiles.runfiles.append(collect_runfiles(ctx)) if extra_deps: - common_runfiles.add_targets(extra_deps) - common_runfiles.add(extra_common_runfiles) - - build_data_file = _write_build_data(ctx) - common_runfiles.add(build_data_file) + app_runfiles.add_targets(extra_deps) + app_runfiles.add(extra_common_runfiles) - common_runfiles = common_runfiles.build(ctx) + app_runfiles = app_runfiles.build(ctx) if _should_create_init_files(ctx): - common_runfiles = _py_builtins.merge_runfiles_with_generated_inits_empty_files_supplier( + app_runfiles = _py_builtins.merge_runfiles_with_generated_inits_empty_files_supplier( ctx = ctx, - runfiles = common_runfiles, + runfiles = app_runfiles, ) - runfiles_with_exe = common_runfiles.merge(ctx.runfiles([executable])) + runfiles_without_exe = builders.RunfilesBuilder() + runfiles_without_exe.add(app_runfiles) + runfiles_without_exe.add(runtime_runfiles) + build_data_file = _write_build_data(ctx) + runfiles_without_exe.add(build_data_file) - data_runfiles = runfiles_with_exe - default_runfiles = runfiles_with_exe + runfiles_without_exe = runfiles_without_exe.build(ctx) + runfiles_with_exe = runfiles_without_exe.merge(ctx.runfiles([executable])) + + # There are three types of runfiles: + # 1. app: Deps added by a user. This is akin to the typical files that would + # be in a traditional venv. No Python runtime files or build data files. + # 2. without-exe: (1) + build data + python runtime + # 3. binary (default/data runfiles): (2) + main executable return struct( - runfiles_without_exe = common_runfiles, - default_runfiles = default_runfiles, + app_runfiles = app_runfiles, build_data_file = build_data_file, - data_runfiles = data_runfiles, + data_runfiles = runfiles_with_exe, + default_runfiles = runfiles_with_exe, + runfiles_without_exe = runfiles_without_exe, ) def _write_build_data(ctx): @@ -1638,7 +1665,11 @@ def _create_providers( cc_info, inherited_environment, runtime_details, - output_groups): + output_groups, + stage2_bootstrap, + app_runfiles, + venv_python_exe, + interpreter_args): """Creates the providers an executable should return. Args: @@ -1667,6 +1698,10 @@ def _create_providers( is run within. runtime_details: struct of runtime information; see _get_runtime_details() output_groups: dict[str, depset[File]]; used to create OutputGroupInfo + stage2_bootstrap: File; the stage 2 bootstrap script. + app_runfiles: runfiles; the runfiles for the application (deps, etc). + venv_python_exe: File; the python executable in the venv. + interpreter_args: list of strings; arguments to pass to the interpreter. Returns: A list of modern providers. @@ -1691,6 +1726,10 @@ def _create_providers( runfiles_without_exe = runfiles_details.runfiles_without_exe, build_data_file = runfiles_details.build_data_file, interpreter_path = runtime_details.executable_interpreter_path, + stage2_bootstrap = stage2_bootstrap, + app_runfiles = app_runfiles, + venv_python_exe = venv_python_exe, + interpreter_args = interpreter_args, ), ] diff --git a/python/private/py_executable_info.bzl b/python/private/py_executable_info.bzl index deb119428d..81216b8e89 100644 --- a/python/private/py_executable_info.bzl +++ b/python/private/py_executable_info.bzl @@ -10,10 +10,30 @@ This provider is for executable-specific information (e.g. tests and binaries). ::: """, fields = { + "app_runfiles": """ +:type: runfiles + +The runfiles for the executable's "user" dependencies. These are things in e.g. +`deps` (or similar), but doesn't include "external" or "implicit" pieces, +e.g. the Python runtime itself. It's roughly akin to the files a traditional +venv would have installed into it. + +:::{versionadded} VERSION_NEXT_FEATURE +::: +""", "build_data_file": """ :type: None | File A symlink to build_data.txt if stamping is enabled, otherwise None. +""", + "interpreter_args": """ +:type: list[str] + +Args that should be passed to the interpreter before regular args +(e.g. `-X whatever`). + +:::{versionadded} VERSION_NEXT_FEATURE +::: """, "interpreter_path": """ :type: None | str @@ -28,6 +48,12 @@ should be within `runtime_files`) The user-level entry point file. Usually a `.py` file, but may also be `.pyc` file if precompiling is enabled. + +:::{seealso} + +The {obj}`stage2_bootstrap` attribute, which bootstraps an executable to run +the user main file. +::: """, "runfiles_without_exe": """ :type: runfiles @@ -35,6 +61,25 @@ file if precompiling is enabled. The runfiles the program needs, but without the original executable, files only added to support the original executable, or files specific to the original program. +""", + "stage2_bootstrap": """ +:type: File | None + +The Bazel-executable-level entry point to the program, which handles Bazel-specific +setup before running the file in {obj}`main`. May be None if a two-stage bootstrap +implementation isn't being used. + +:::{versionadded} VERSION_NEXT_FEATURE +::: +""", + "venv_python_exe": """ +:type: File | None + +The `bin/python3` file within the venv this binary uses. May be None if venv +mode is not enabled. + +:::{versionadded} VERSION_NEXT_FEATURE +::: """, }, ) diff --git a/python/private/py_interpreter_program.bzl b/python/private/py_interpreter_program.bzl index cd62a7190d..7eb3e28bd9 100644 --- a/python/private/py_interpreter_program.bzl +++ b/python/private/py_interpreter_program.bzl @@ -15,6 +15,7 @@ """Internal only bootstrap level binary-like rule.""" load("@bazel_skylib//rules:common_settings.bzl", "BuildSettingInfo") +load("//python/private:sentinel.bzl", "SentinelInfo") PyInterpreterProgramInfo = provider( doc = "Information about how to run a program with an external interpreter.", @@ -28,14 +29,20 @@ PyInterpreterProgramInfo = provider( def _py_interpreter_program_impl(ctx): # Bazel requires the executable file to be an output created by this target. - executable = ctx.actions.declare_file(ctx.label.name) + # To avoid colliding with the source file (e.g. target=foo, main=foo.py), + # we append an underscore to the name, but keep the extension so that + # the original extension is preserved. + extension = ctx.file.main.extension + executable_name = "{}_.{}".format(ctx.label.name, extension) + executable = ctx.actions.declare_file(executable_name) ctx.actions.symlink(output = executable, target_file = ctx.file.main) execution_requirements = {} - execution_requirements.update([ - value.split("=", 1) - for value in ctx.attr.execution_requirements[BuildSettingInfo].value - if value.strip() - ]) + if BuildSettingInfo in ctx.attr.execution_requirements: + execution_requirements.update([ + value.split("=", 1) + for value in ctx.attr.execution_requirements[BuildSettingInfo].value + if value.strip() + ]) return [ DefaultInfo( @@ -85,8 +92,9 @@ ctx.actions.run( doc = "Environment variables that should set prior to running.", ), "execution_requirements": attr.label( + default = "//python:none", doc = "Execution requirements to set when running it as an action", - providers = [BuildSettingInfo], + providers = [[BuildSettingInfo], [SentinelInfo]], ), "interpreter_args": attr.string_list( doc = "Args that should be passed to the interpreter.", diff --git a/python/private/py_runtime_rule.bzl b/python/private/py_runtime_rule.bzl index 3bcee4cfd7..09e245a58e 100644 --- a/python/private/py_runtime_rule.bzl +++ b/python/private/py_runtime_rule.bzl @@ -361,7 +361,7 @@ See {obj}`PyRuntimeInfo.supports_build_time_venv` for docs. default = True, ), "zip_main_template": attr.label( - default = "//python/private:zip_main_template", + default = "//python/private/zipapp:zip_main_template", allow_single_file = True, doc = """ The template to use for a zip's top-level `__main__.py` file. diff --git a/python/private/zipapp/BUILD.bazel b/python/private/zipapp/BUILD.bazel new file mode 100644 index 0000000000..543fe0a185 --- /dev/null +++ b/python/private/zipapp/BUILD.bazel @@ -0,0 +1,49 @@ +load("@bazel_skylib//:bzl_library.bzl", "bzl_library") + +package( + default_visibility = ["//:__subpackages__"], +) + +licenses(["notice"]) + +filegroup( + name = "distribution", + srcs = glob(["**"]), +) + +bzl_library( + name = "py_zipapp_rule_bzl", + srcs = ["py_zipapp_rule.bzl"], + deps = [ + "//python/private:attributes_bzl", + "//python/private:builders_bzl", + "//python/private:common_bzl", + "//python/private:common_labels_bzl", + "//python/private:py_executable_info_bzl", + "//python/private:py_info_bzl", + "//python/private:py_internal_bzl", + "//python/private:py_interpreter_program_bzl", + "//python/private:py_runtime_info_bzl", + "//python/private:toolchain_types_bzl", + "//python/private:transition_labels_bzl", + "@bazel_skylib//lib:paths", + ], +) + +filegroup( + name = "zip_main_template", + srcs = ["zip_main_template.py"], + visibility = ["//visibility:public"], +) + +filegroup( + name = "zip_shell_template", + srcs = ["zip_shell_template.sh"], + visibility = ["//visibility:public"], +) + +filegroup( + name = "zipapp_stage2_bootstrap_template", + srcs = ["zipapp_stage2_bootstrap_template.py"], + visibility = ["//visibility:public"], +) diff --git a/python/private/zipapp/py_zipapp_rule.bzl b/python/private/zipapp/py_zipapp_rule.bzl new file mode 100644 index 0000000000..cc399064ad --- /dev/null +++ b/python/private/zipapp/py_zipapp_rule.bzl @@ -0,0 +1,317 @@ +"""Implementation of the zipapp rules.""" + +load("@bazel_skylib//lib:paths.bzl", "paths") +load("//python/private:attributes.bzl", "apply_config_settings_attr") +load("//python/private:builders.bzl", "builders") +load("//python/private:common.bzl", "BUILTIN_BUILD_PYTHON_ZIP", "actions_run", "maybe_builtin_build_python_zip", "maybe_create_repo_mapping", "runfiles_root_path") +load("//python/private:common_labels.bzl", "labels") +load("//python/private:py_executable_info.bzl", "PyExecutableInfo") +load("//python/private:py_internal.bzl", "py_internal") +load("//python/private:py_runtime_info.bzl", "PyRuntimeInfo") +load("//python/private:toolchain_types.bzl", "EXEC_TOOLS_TOOLCHAIN_TYPE") +load("//python/private:transition_labels.bzl", "TRANSITION_LABELS") + +def _is_symlink(f): + if hasattr(f, "is_symlink"): + return str(int(f.is_symlink)) + else: + return "-1" + +def _create_zipapp_main_py(ctx, py_runtime, py_executable, stage2_bootstrap): + python_exe = py_executable.venv_python_exe + if python_exe: + python_exe_path = runfiles_root_path(ctx, python_exe.short_path) + elif py_runtime.interpreter: + python_exe_path = runfiles_root_path(ctx, py_runtime.interpreter.short_path) + else: + python_exe_path = py_runtime.interpreter_path + + if py_runtime.interpreter: + python_binary_actual_path = runfiles_root_path(ctx, py_runtime.interpreter.short_path) + else: + python_binary_actual_path = py_runtime.interpreter_path + + zip_main_py = ctx.actions.declare_file(ctx.label.name + ".zip_main.py") + ctx.actions.expand_template( + template = py_runtime.zip_main_template, + output = zip_main_py, + substitutions = { + "%python_binary%": python_exe_path, + "%python_binary_actual%": python_binary_actual_path, + "%stage2_bootstrap%": runfiles_root_path(ctx, stage2_bootstrap.short_path), + "%workspace_name%": ctx.workspace_name, + }, + ) + return zip_main_py + +def _map_zip_empty_filenames(list_paths_cb): + return ["rf-empty|" + path for path in list_paths_cb().to_list()] + +def _map_zip_runfiles(file): + return "rf-file|" + _is_symlink(file) + "|" + file.short_path + "|" + file.path + +def _map_zip_symlinks(entry): + return "rf-symlink|" + _is_symlink(entry.target_file) + "|" + entry.path + "|" + entry.target_file.path + +def _map_zip_root_symlinks(entry): + return "rf-root-symlink|" + _is_symlink(entry.target_file) + "|" + entry.path + "|" + entry.target_file.path + +def _build_manifest(ctx, manifest, runfiles, zip_main): + manifest.add("regular|0|__main__.py|{}".format(zip_main.path)) + + manifest.add_all( + # NOTE: Accessing runfiles.empty_filenames materializes them. A lambda + # is used to defer that. + [lambda: runfiles.empty_filenames], + map_each = _map_zip_empty_filenames, + allow_closure = True, + ) + + manifest.add_all(runfiles.files, map_each = _map_zip_runfiles) + manifest.add_all(runfiles.symlinks, map_each = _map_zip_symlinks) + manifest.add_all(runfiles.root_symlinks, map_each = _map_zip_root_symlinks) + + inputs = [zip_main] + zip_repo_mapping_manifest = maybe_create_repo_mapping( + ctx = ctx, + runfiles = runfiles, + ) + if zip_repo_mapping_manifest: + # NOTE: rf-root-symlink is used to make it show up under the runfiles + # subdirectory within the zip. + manifest.add( + zip_repo_mapping_manifest.path, + format = "rf-root-symlink|0|_repo_mapping|%s", + ) + inputs.append(zip_repo_mapping_manifest) + return inputs + +def _create_zip(ctx, py_runtime, py_executable, stage2_bootstrap): + output = ctx.actions.declare_file(ctx.label.name + ".zip") + manifest = ctx.actions.args() + manifest.use_param_file("%s", use_always = True) + manifest.set_param_file_format("multiline") + + runfiles = builders.RunfilesBuilder() + + runfiles.add(py_runtime.files) + if py_executable.venv_python_exe: + runfiles.add(py_executable.venv_python_exe) + runfiles.add(py_executable.app_runfiles) + runfiles.add(stage2_bootstrap) + + runfiles = runfiles.build(ctx) + + zip_main = _create_zipapp_main_py(ctx, py_runtime, py_executable, stage2_bootstrap) + inputs = _build_manifest(ctx, manifest, runfiles, zip_main) + + zipper_args = ctx.actions.args() + zipper_args.add(output) + zipper_args.add(ctx.workspace_name, format = "--workspace-name=%s") + zipper_args.add( + str(int(py_internal.get_legacy_external_runfiles(ctx))), + format = "--legacy-external-runfiles=%s", + ) + if ctx.attr.compression: + zipper_args.add(ctx.attr.compression, "--compression=%s") + zipper_args.add("--runfiles-dir=runfiles") + + actions_run( + ctx, + executable = ctx.attr._zipper, + arguments = [manifest, zipper_args], + inputs = depset(inputs, transitive = [runfiles.files]), + outputs = [output], + mnemonic = "PyZipAppCreateZip", + progress_message = "Reticulating zipapp archive: %{label} into %{output}", + ) + return output + +def _create_shell_bootstrap(ctx, py_runtime, py_executable, stage2_bootstrap): + preamble = ctx.actions.declare_file(ctx.label.name + ".preamble.sh") + + bundled_pyexe_path = "" + external_pyexe_path = "" + if py_runtime.interpreter_path: + external_pyexe_path = py_runtime.interpreter_path + else: + bundled_pyexe_path = runfiles_root_path(ctx, py_runtime.interpreter.short_path) + + substitutions = { + "%BUNDLED_PYEXE_PATH%": bundled_pyexe_path, + "%EXTERNAL_PYEXE_PATH%": external_pyexe_path, + "%EXTRACT_DIR%": paths.join( + (ctx.label.repo_name or "_main"), + ctx.label.package, + ctx.label.name, + ), + "%INTERPRETER_ARGS%": "\n".join([ + '"{}"'.format(v) + for v in py_executable.interpreter_args + ]), + "%STAGE2_BOOTSTRAP%": runfiles_root_path(ctx, stage2_bootstrap.short_path), + } + ctx.actions.expand_template( + template = ctx.file._zip_shell_template, + output = preamble, + substitutions = substitutions, + is_executable = True, + ) + return preamble + +def _create_self_executable_zip(ctx, preamble, zip_file): + pyz = ctx.actions.declare_file(ctx.label.name + ".pyz") + args = ctx.actions.args() + args.add(preamble) + args.add(zip_file) + args.add(pyz) + actions_run( + ctx, + executable = ctx.attr._exe_zip_maker, + arguments = [args], + inputs = depset([preamble, zip_file]), + outputs = [pyz], + mnemonic = "PyZipAppCreateExecutableZip", + progress_message = "Reticulating zipapp executable: %{label} into %{output}", + ) + return pyz + +def _py_zipapp_executable_impl(ctx): + py_executable = ctx.attr.binary[PyExecutableInfo] + py_runtime = ctx.attr.binary[PyRuntimeInfo] + + stage2_bootstrap = py_executable.stage2_bootstrap + + zip_file = _create_zip(ctx, py_runtime, py_executable, stage2_bootstrap) + if ctx.attr.executable: + preamble = _create_shell_bootstrap(ctx, py_runtime, py_executable, stage2_bootstrap) + executable = _create_self_executable_zip(ctx, preamble, zip_file) + default_output = executable + else: + # Bazel requires executable=True rules to have an executable given, so give + # a fake one to satisfy that. + default_output = zip_file + executable = ctx.actions.declare_file(ctx.label.name + "-not-executable") + ctx.actions.write(executable, "echo 'ERROR: Non executable zip file'; exit 1") + + return [ + DefaultInfo( + files = depset([default_output]), + runfiles = ctx.runfiles(files = [default_output]), + executable = executable, + ), + ] + +def _transition_zipapp_impl(settings, attr): + settings = apply_config_settings_attr(dict(settings), attr) + + # Force this to false, otherwise the binary is already a zipapp + settings[labels.BUILD_PYTHON_ZIP] = False + maybe_builtin_build_python_zip("false", settings) + return settings + +_zipapp_transition = transition( + implementation = _transition_zipapp_impl, + inputs = TRANSITION_LABELS, + outputs = TRANSITION_LABELS + [ + labels.BUILD_PYTHON_ZIP, + ] + BUILTIN_BUILD_PYTHON_ZIP, +) + +_ATTRS = { + "binary": attr.label( + doc = """ +A `py_binary` or `py_test` (or equivalent) target to package. +""", + providers = [PyExecutableInfo, PyRuntimeInfo], + mandatory = True, + ), + "compression": attr.string( + doc = """ +The compression level to use. + +Typically 0 to 9, with higher numbers being to compress more. +""", + default = "", + ), + "config_settings": attr.label_keyed_string_dict( + doc = """ +Config settings to change for this target. + +The keys are labels for settings, and the values are strings for the new value +to use. Pass `Label` objects or canonical label strings for the keys to ensure +they resolve as expected (canonical labels start with `@@` and can be +obtained by calling `str(Label(...))`). + +Most `@rules_python//python/config_setting` settings can be used here, which +allows, for example, making only a certain `py_binary` use +{obj}`--boostrap_impl=script`. + +Additional or custom config settings can be registered using the +{obj}`add_transition_setting` API. This allows, for example, forcing a +particular CPU, or defining a custom setting that `select()` uses elsewhere +to pick between `pip.parse` hubs. See the [How to guide on multiple +versions of a library] for a more concrete example. + +:::{note} +These values are transitioned on, so will affect the analysis graph and the +associated memory overhead. The more unique configurations in your overall +build, the more memory and (often unnecessary) re-analysis and re-building +can occur. See +https://bazel.build/extending/config#memory-performance-considerations for +more information about risks and considerations. +::: +""", + ), + "executable": attr.bool( + doc = """ +Whether the output should be an executable zip file. +""", + default = True, + ), + # Required to opt-in to the transition feature. + "_allowlist_function_transition": attr.label( + default = "@bazel_tools//tools/allowlists/function_transition_allowlist", + ), + "_exe_zip_maker": attr.label( + cfg = "exec", + default = "//tools/private/zipapp:exe_zip_maker", + ), + "_zip_shell_template": attr.label( + default = ":zip_shell_template", + allow_single_file = True, + ), + "_zipper": attr.label( + cfg = "exec", + default = "//tools/private/zipapp:zipper", + ), +} +_TOOLCHAINS = [EXEC_TOOLS_TOOLCHAIN_TYPE] + +py_zipapp_binary = rule( + doc = """ +Packages a `py_binary` as a Python zipapp. +""", + implementation = _py_zipapp_executable_impl, + attrs = _ATTRS, + # NOTE: While this is marked executable, it is conditionally executable + # based on the `executable` attribute. + executable = True, + toolchains = _TOOLCHAINS, + cfg = _zipapp_transition, +) + +py_zipapp_test = rule( + doc = """ +Packages a `py_test` as a Python zipapp. + +This target is also a valid test target to run. +""", + implementation = _py_zipapp_executable_impl, + attrs = _ATTRS, + # NOTE: While this is marked as a test, it is conditionally executable + # based on the `executable` attribute. + test = True, + toolchains = _TOOLCHAINS, + cfg = _zipapp_transition, +) diff --git a/python/private/zip_main_template.py b/python/private/zipapp/zip_main_template.py similarity index 96% rename from python/private/zip_main_template.py rename to python/private/zipapp/zip_main_template.py index d1489b46aa..35db1645bc 100644 --- a/python/private/zip_main_template.py +++ b/python/private/zipapp/zip_main_template.py @@ -3,11 +3,15 @@ # NOTE: This file is a "stage 1" bootstrap, so it's responsible for locating the # desired runtime and having it run the stage 2 bootstrap. This means it can't # assume much about the current runtime and environment. e.g., the current -# runtime may not be the correct one, the zip may not have been extract, the +# runtime may not be the correct one, the zip may not have been extracted, the # runfiles env vars may not be set, etc. # # NOTE: This program must retain compatibility with a wide variety of Python # versions since it is run by an unknown Python interpreter. +# +# NOTE: For a self-executable zip, this file may not be the entry point +# for the program and may be skipped entirely; the self-executable zip +# preamble may jump directly to the stage2 bootstrap. import sys @@ -23,11 +27,11 @@ import tempfile import zipfile -# runfiles-relative path +# runfiles-root-relative path _STAGE2_BOOTSTRAP = "%stage2_bootstrap%" -# runfiles-relative path to venv's bin/python3. Empty if venv not being used. +# runfiles-root-relative path to venv's bin/python3. Empty if venv not being used. _PYTHON_BINARY = "%python_binary%" -# runfiles-relative path, absolute path, or single word. The actual Python +# runfiles-root-relative path, absolute path, or single word. The actual Python # executable to use. _PYTHON_BINARY_ACTUAL = "%python_binary_actual%" _WORKSPACE_NAME = "%workspace_name%" diff --git a/python/private/zipapp/zip_shell_template.sh b/python/private/zipapp/zip_shell_template.sh new file mode 100644 index 0000000000..d79331444a --- /dev/null +++ b/python/private/zipapp/zip_shell_template.sh @@ -0,0 +1,88 @@ +#!/usr/bin/env bash + +set -e + +if [[ -n "${RULES_PYTHON_BOOTSTRAP_VERBOSE:-}" ]]; then + set -x +fi + +# runfiles-root-relative path +BUNDLED_PYEXE_PATH="%BUNDLED_PYEXE_PATH%" +# Absolute path or single word +EXTERNAL_PYEXE_PATH="%EXTERNAL_PYEXE_PATH%" +# runfiles-root-relative path +STAGE2_BOOTSTRAP="%STAGE2_BOOTSTRAP%" +EXTRACT_DIR="%EXTRACT_DIR%" +ZIP_HASH="%ZIP_HASH%" +declare -a INTERPRETER_ARGS_FROM_TARGET=( +%INTERPRETER_ARGS% +) + +declare -a interpreter_env +declare -a interpreter_args +declare -a additional_interpreter_args + +if [[ -z "${PYTHONSAFEPATH+x}" ]]; then + # ${FOO-WORD} expands to WORD if $FOO is undefined, and $FOO otherwise + interpreter_env+=("PYTHONSAFEPATH=${PYTHONSAFEPATH-1}") +fi + + +if [[ -n "${RULES_PYTHON_ADDITIONAL_INTERPRETER_ARGS}" ]]; then + read -a additional_interpreter_args <<< "${RULES_PYTHON_ADDITIONAL_INTERPRETER_ARGS}" + interpreter_args+=("${additional_interpreter_args[@]}") + unset RULES_PYTHON_ADDITIONAL_INTERPRETER_ARGS +fi + + +if [[ -n "$RULES_PYTHON_EXTRACT_ROOT" ]]; then + zip_dir="$RULES_PYTHON_EXTRACT_ROOT/$EXTRACT_DIR/$ZIP_HASH" + if [[ ! -e "$zip_dir/__main__.py" ]]; then + mkdir -p "$zip_dir" + # Unzip emits a warning and exits 1 with the prelude + ( unzip -q -d "$zip_dir" "$0" 2>/dev/null || true ) + fi +else + # NOTE: Macs have an old version of mktemp, so we must use only the + # minimal functionality of it. + zip_dir=$(mktemp -d) + # Unzip emits a warning and exits 1 with the prelude + ( unzip -q -d "$zip_dir" "$0" 2>/dev/null || true ) + if [[ -n "$zip_dir" && -z "${RULES_PYTHON_BOOTSTRAP_VERBOSE:-}" ]]; then + trap 'rm -fr "$zip_dir"' EXIT + fi +fi + +export RUNFILES_DIR="$zip_dir/runfiles" +if [[ ! -d "$RUNFILES_DIR" ]]; then + echo "Runfiles dir not found: zip extraction likely failed" 1>&2 + echo "Run with RULES_PYTHON_BOOTSTRAP_VERBOSE=1 to aid debugging" 1>&2 + exit 1 +fi + +if [[ -n "$BUNDLED_PYEXE_PATH" ]]; then + python_exe="$RUNFILES_DIR/$BUNDLED_PYEXE_PATH" +else + python_exe="$EXTERNAL_PYEXE_PATH" +fi + +command=( + env + "${interpreter_env[@]}" + "$python_exe" + "-XRULES_PYTHON_ZIP_DIR=$zip_dir" + "${interpreter_args[@]}" + "${INTERPRETER_ARGS_FROM_TARGET[@]}" + "$RUNFILES_DIR/$STAGE2_BOOTSTRAP" + "$@" +) + +# NOTE: because exec isn't used, signals don't propagate to the child +# TODO: Use exec and let the program handle cleanup. Without exec, +# signals don't propagate to the child nicely. +# See https://github.com/bazel-contrib/rules_python/issues/2043#issuecomment-2215469971 +# for more information. +"${command[@]}" +# Explicit exit is needed because the implicit next line the zip file this +# template is prepended to. +exit 0 diff --git a/python/private/zipapp/zipapp_stage2_bootstrap_template.py b/python/private/zipapp/zipapp_stage2_bootstrap_template.py new file mode 100644 index 0000000000..9fd71fefb9 --- /dev/null +++ b/python/private/zipapp/zipapp_stage2_bootstrap_template.py @@ -0,0 +1,10 @@ +import runpy +import shutil +import sys + +try: + sys.argv.pop(0) # Remove zipapp_stage2_bootstrap from args + runpy.run_path(sys.argv[0], run_name="__main__") +finally: + if zip_dir := sys._xoptions.get("RULES_PYTHON_ZIP_DIR"): + shutil.rmtree(zip_dir, True) diff --git a/python/zipapp/BUILD.bazel b/python/zipapp/BUILD.bazel new file mode 100644 index 0000000000..71249ae45c --- /dev/null +++ b/python/zipapp/BUILD.bazel @@ -0,0 +1,29 @@ +load("@bazel_skylib//:bzl_library.bzl", "bzl_library") + +package(default_visibility = ["//visibility:public"]) + +licenses(["notice"]) + +filegroup( + name = "distribution", + srcs = glob(["**"]), + visibility = ["//python:__pkg__"], +) + +bzl_library( + name = "py_zipapp_binary_bzl", + srcs = ["py_zipapp_binary.bzl"], + deps = [ + "//python/private:util_bzl", + "//python/private/zipapp:py_zipapp_rule_bzl", + ], +) + +bzl_library( + name = "py_zipapp_test_bzl", + srcs = ["py_zipapp_test.bzl"], + deps = [ + "//python/private:util_bzl", + "//python/private/zipapp:py_zipapp_rule_bzl", + ], +) diff --git a/python/zipapp/py_zipapp_binary.bzl b/python/zipapp/py_zipapp_binary.bzl new file mode 100644 index 0000000000..4c40786200 --- /dev/null +++ b/python/zipapp/py_zipapp_binary.bzl @@ -0,0 +1,15 @@ +"""`py_zipapp_binary` macro.""" + +load("//python/private:util.bzl", "add_tag") +load("//python/private/zipapp:py_zipapp_rule.bzl", _py_zipapp_binary_rule = "py_zipapp_binary") + +def py_zipapp_binary(**kwargs): + """Builds a Python zipapp from a py_binary/py_test target. + + Args: + **kwargs: Args passed onto {rule}`py_zipapp_binary`. + """ + add_tag(kwargs, "@rules_python//python:py_zipapp_binary") + _py_zipapp_binary_rule( + **kwargs + ) diff --git a/python/zipapp/py_zipapp_test.bzl b/python/zipapp/py_zipapp_test.bzl new file mode 100644 index 0000000000..bc113ad4ed --- /dev/null +++ b/python/zipapp/py_zipapp_test.bzl @@ -0,0 +1,15 @@ +"""`py_zipapp_test` macro.""" + +load("//python/private:util.bzl", "add_tag") +load("//python/private/zipapp:py_zipapp_rule.bzl", _py_zipapp_test = "py_zipapp_test") + +def py_zipapp_test(**kwargs): + """Builds a Python zipapp from a py_binary/py_test target. + + Args: + **kwargs: Args passed onto {rule}`py_zipapp_test`. + """ + add_tag(kwargs, "@rules_python//python:py_zipapp_test") + _py_zipapp_test( + **kwargs + ) diff --git a/tests/base_rules/py_executable_base_tests.bzl b/tests/base_rules/py_executable_base_tests.bzl index 49d5f49b86..a73a4cb48e 100644 --- a/tests/base_rules/py_executable_base_tests.bzl +++ b/tests/base_rules/py_executable_base_tests.bzl @@ -21,12 +21,13 @@ load("@rules_testing//lib:util.bzl", rt_util = "util") load("//python:py_executable_info.bzl", "PyExecutableInfo") load("//python:py_info.bzl", "PyInfo") load("//python:py_library.bzl", "py_library") +load("//python/private:common.bzl", "maybe_builtin_build_python_zip") # buildifier: disable=bzl-visibility load("//python/private:common_labels.bzl", "labels") # buildifier: disable=bzl-visibility load("//python/private:reexports.bzl", "BuiltinPyRuntimeInfo") # buildifier: disable=bzl-visibility load("//tests/base_rules:base_tests.bzl", "create_base_tests") load("//tests/base_rules:util.bzl", "WINDOWS_ATTR", pt_util = "util") load("//tests/support:py_executable_info_subject.bzl", "PyExecutableInfoSubject") -load("//tests/support:support.bzl", "CC_TOOLCHAIN", "CROSSTOOL_TOP", "maybe_builtin_build_python_zip") +load("//tests/support:support.bzl", "CC_TOOLCHAIN", "CROSSTOOL_TOP") load("//tests/support/platforms:platforms.bzl", "platform_targets") _tests = [] diff --git a/tests/config_settings/transition/multi_version_tests.bzl b/tests/config_settings/transition/multi_version_tests.bzl index 8a1d8a6a63..3bb69f2f59 100644 --- a/tests/config_settings/transition/multi_version_tests.bzl +++ b/tests/config_settings/transition/multi_version_tests.bzl @@ -20,9 +20,10 @@ load("@rules_testing//lib:util.bzl", rt_util = "util") load("//python:py_binary.bzl", "py_binary") load("//python:py_info.bzl", "PyInfo") load("//python:py_test.bzl", "py_test") +load("//python/private:common.bzl", "maybe_builtin_build_python_zip") # buildifier: disable=bzl-visibility load("//python/private:common_labels.bzl", "labels") # buildifier: disable=bzl-visibility load("//python/private:reexports.bzl", "BuiltinPyInfo") # buildifier: disable=bzl-visibility -load("//tests/support:support.bzl", "CC_TOOLCHAIN", "maybe_builtin_build_python_zip") +load("//tests/support:support.bzl", "CC_TOOLCHAIN") load("//tests/support/platforms:platforms.bzl", "platform_targets") # NOTE @aignas 2024-06-04: we are using here something that is registered in the MODULE.Bazel diff --git a/tests/py_zipapp/BUILD.bazel b/tests/py_zipapp/BUILD.bazel new file mode 100644 index 0000000000..74df4aa04d --- /dev/null +++ b/tests/py_zipapp/BUILD.bazel @@ -0,0 +1,67 @@ +load("//python:py_binary.bzl", "py_binary") +load("//python:py_test.bzl", "py_test") +load("//python/private:bzlmod_enabled.bzl", "BZLMOD_ENABLED") # buildifier: disable=bzl-visibility +load("//python/zipapp:py_zipapp_binary.bzl", "py_zipapp_binary") +load("//tests/support:support.bzl", "NOT_WINDOWS") + +# todo: add windows support. Windows support will be a bit odd. +# It previously worked by having special logic in the exe launcher +# that knew to look for .zip and running that through python + +py_binary( + name = "venv_bin", + srcs = ["main.py"], + config_settings = { + "//python/config_settings:bootstrap_impl": "script", + "//python/config_settings:venvs_site_packages": "yes", + }, + main = "main.py", + target_compatible_with = NOT_WINDOWS, + deps = ["@dev_pip//absl_py"], +) + +py_zipapp_binary( + name = "venv_zipapp", + binary = ":venv_bin", + target_compatible_with = NOT_WINDOWS, +) + +py_test( + name = "venv_zipapp_test", + srcs = ["venv_zipapp_test.py"], + data = [":venv_zipapp"], + env = { + "BZLMOD_ENABLED": str(int(BZLMOD_ENABLED)), + "TEST_ZIPAPP": "$(location :venv_zipapp)", + }, + target_compatible_with = NOT_WINDOWS, +) + +py_binary( + name = "system_python_bin", + srcs = ["main.py"], + config_settings = { + "//python/config_settings:bootstrap_impl": "system_python", + "//python/config_settings:venvs_site_packages": "no", + }, + main = "main.py", + # TODO: #2586 - Add windows support + target_compatible_with = NOT_WINDOWS, + deps = ["@dev_pip//absl_py"], +) + +py_zipapp_binary( + name = "system_python_zipapp", + binary = ":system_python_bin", + target_compatible_with = NOT_WINDOWS, +) + +py_test( + name = "system_python_zipapp_test", + srcs = ["system_python_zipapp_test.py"], + data = [":system_python_zipapp"], + env = { + "TEST_ZIPAPP": "$(location :system_python_zipapp)", + }, + target_compatible_with = NOT_WINDOWS, +) diff --git a/tests/py_zipapp/main.py b/tests/py_zipapp/main.py new file mode 100644 index 0000000000..b8fdbe365e --- /dev/null +++ b/tests/py_zipapp/main.py @@ -0,0 +1,12 @@ +"A trivial zipapp that prints a message" + + +def main(): + print("Hello from zipapp") + import absl + + print(f"absl: {absl}") + + +if __name__ == "__main__": + main() diff --git a/tests/py_zipapp/system_python_zipapp_test.py b/tests/py_zipapp/system_python_zipapp_test.py new file mode 100644 index 0000000000..297a574f16 --- /dev/null +++ b/tests/py_zipapp/system_python_zipapp_test.py @@ -0,0 +1,25 @@ +import os +import subprocess +import unittest +import zipfile + + +class SystemPythonZipAppTest(unittest.TestCase): + def test_zipapp_contents(self): + zipapp_path = os.environ["TEST_ZIPAPP"] + + self.assertTrue(os.path.exists(zipapp_path)) + self.assertTrue(os.path.isfile(zipapp_path)) + + # The zipapp itself is a shell script prepended to the zip file. + with open(zipapp_path, "rb") as f: + content = f.read() + self.assertTrue(content.startswith(b"#!/usr/bin/env bash")) + + output = subprocess.check_output([zipapp_path]).decode("utf-8").strip() + self.assertIn("Hello from zipapp", output) + self.assertIn("absl", output) + + +if __name__ == "__main__": + unittest.main() diff --git a/tests/py_zipapp/venv_zipapp_test.py b/tests/py_zipapp/venv_zipapp_test.py new file mode 100644 index 0000000000..40d20fedb4 --- /dev/null +++ b/tests/py_zipapp/venv_zipapp_test.py @@ -0,0 +1,73 @@ +import os +import subprocess +import unittest +import zipfile + + +class PyZipAppTest(unittest.TestCase): + def test_zipapp_contents(self): + zipapp_path = os.environ["TEST_ZIPAPP"] + + self.assertTrue(os.path.exists(zipapp_path)) + self.assertTrue(os.path.isfile(zipapp_path)) + + # The zipapp itself is a shell script prepended to the zip file. + with open(zipapp_path, "rb") as f: + content = f.read() + self.assertTrue(content.startswith(b"#!/usr/bin/env bash")) + + output = subprocess.check_output([zipapp_path]).decode("utf-8").strip() + self.assertIn("Hello from zipapp", output) + self.assertIn("absl", output) + + def assertHasPathMatchingSuffix(self, namelist, suffix, msg=None): + if not any(name.endswith(suffix) for name in namelist): + self.fail(msg or f"No path in zipapp matching suffix '{suffix}'") + + def assertZipEntryIsSymlink(self, zip_file, path, msg=None): + try: + info = zip_file.getinfo(path) + except KeyError: + self.fail(msg or f"Path '{path}' not found in zipfile") + + # S_IFLNK is 0o120000. + # ZipInfo.external_attr is 32 bits: the high 16 bits are Unix attributes. + is_symlink = (info.external_attr >> 16) & 0o170000 == 0o120000 + if not is_symlink: + self.fail(msg or f"Path '{path}' is not a symlink") + + def _is_bzlmod_enabled(self): + return os.environ["BZLMOD_ENABLED"] == "1" + + def test_zipapp_structure(self): + zipapp_path = os.environ["TEST_ZIPAPP"] + + with zipfile.ZipFile(zipapp_path, "r") as zf: + namelist = zf.namelist() + + if self._is_bzlmod_enabled(): + self.assertIn("runfiles/_repo_mapping", namelist) + + self.assertHasPathMatchingSuffix(namelist, "/pyvenv.cfg") + + # The venv directory name depends on the target name, so find it + # by looking for pyvenv.cfg. + venv_config = next( + (name for name in namelist if name.endswith("/pyvenv.cfg")), None + ) + self.assertIsNotNone(venv_config) + + venv_root = os.path.dirname(venv_config) + + # Verify bin/python3 exists and is a symlink + python_bin = f"{venv_root}/bin/python3" + self.assertZipEntryIsSymlink(zf, python_bin) + + # Verify _bazel_site_init.py exists in site-packages + self.assertHasPathMatchingSuffix( + namelist, "/site-packages/_bazel_site_init.py" + ) + + +if __name__ == "__main__": + unittest.main() diff --git a/tests/support/support.bzl b/tests/support/support.bzl index b767ec2714..9bd2c987b9 100644 --- a/tests/support/support.bzl +++ b/tests/support/support.bzl @@ -45,9 +45,4 @@ NOT_WINDOWS = select({ "//conditions:default": [], }) -def maybe_builtin_build_python_zip(value): - settings = {} - if not config.bazel_10_or_later: - settings["//command_line_option:build_python_zip"] = value - - return settings +BAZEL_8_OR_LATER = [] if config.bazel_8_or_later else ["@platforms//:incompatible"] diff --git a/tests/tools/zipapp/BUILD.bazel b/tests/tools/zipapp/BUILD.bazel new file mode 100644 index 0000000000..84902b76b8 --- /dev/null +++ b/tests/tools/zipapp/BUILD.bazel @@ -0,0 +1,13 @@ +load("//python:py_test.bzl", "py_test") + +py_test( + name = "zipper_test", + srcs = ["zipper_test.py"], + deps = ["//tools/private/zipapp:zipper_lib"], +) + +py_test( + name = "exe_zip_maker_test", + srcs = ["exe_zip_maker_test.py"], + deps = ["//tools/private/zipapp:exe_zip_maker_lib"], +) diff --git a/tests/tools/zipapp/exe_zip_maker_test.py b/tests/tools/zipapp/exe_zip_maker_test.py new file mode 100644 index 0000000000..73c509bdbe --- /dev/null +++ b/tests/tools/zipapp/exe_zip_maker_test.py @@ -0,0 +1,71 @@ +import hashlib +import pathlib +import shutil +import stat +import tempfile +import unittest + +from tools.private.zipapp import exe_zip_maker + + +class ExeZipMakerTest(unittest.TestCase): + def setUp(self): + self.test_dir = pathlib.Path(tempfile.mkdtemp()) + self.preamble_path = self.test_dir / "preamble.txt" + self.zip_path = self.test_dir / "data.zip" + self.output_path = self.test_dir / "output.exe" + + def tearDown(self): + shutil.rmtree(self.test_dir) + + def assertStartsWith(self, actual, expected): + if not actual.startswith(expected): + self.fail(f"{actual!r} does not start with {expected!r}") + + def test_create_exe_zip(self): + # Create dummy zip file + zip_content = b"PK\x03\x04dummyzipcontent" + self.zip_path.write_bytes(zip_content) + + # Calculate expected hash + expected_hash = hashlib.sha256(zip_content).hexdigest().encode("utf-8") + + # Create preamble with placeholder + preamble_text = b"#!/bin/bash\nEXPECTED_HASH='%ZIP_HASH%'\n# ... logic ...\n" + self.preamble_path.write_bytes(preamble_text) + + # Call create_exe_zip directly + exe_zip_maker.create_exe_zip( + str(self.preamble_path), str(self.zip_path), str(self.output_path) + ) + + # Verify output exists + self.assertTrue( + self.output_path.exists(), + msg=f"Output path '{self.output_path}' should exist", + ) + + # Verify executable bit + st = self.output_path.stat() + self.assertTrue( + st.st_mode & stat.S_IEXEC, + msg=f"Output path '{self.output_path}' should be executable", + ) + + # Verify content + content = self.output_path.read_bytes() + + # Split content back into preamble and zip + # We know the preamble text length after substitution. + expected_preamble = preamble_text.replace(b"%ZIP_HASH%", expected_hash) + + self.assertStartsWith(content, expected_preamble) + self.assertTrue( + content.endswith(zip_content), + msg="Output content should end with the zip content", + ) + self.assertEqual(len(content), len(expected_preamble) + len(zip_content)) + + +if __name__ == "__main__": + unittest.main() diff --git a/tests/tools/zipapp/zipper_test.py b/tests/tools/zipapp/zipper_test.py new file mode 100644 index 0000000000..e0d6653aa7 --- /dev/null +++ b/tests/tools/zipapp/zipper_test.py @@ -0,0 +1,220 @@ +import os +import pathlib +import shutil +import tempfile +import time +import unittest +import zipfile + +from tools.private.zipapp import zipper + + +class ZipperTest(unittest.TestCase): + def setUp(self): + self.test_dir = pathlib.Path(tempfile.mkdtemp()) + self.manifest_path = self.test_dir / "manifest.txt" + self.output_zip = self.test_dir / "output.zip" + + def tearDown(self): + shutil.rmtree(self.test_dir) + + def _create_zip(self, **kwargs): + defaults = { + "manifest_path": self.manifest_path, + "output_zip": self.output_zip, + "compress_level": 0, + "workspace_name": "my_ws", + "legacy_external_runfiles": False, + "runfiles_dir": "runfiles", + } + defaults.update(kwargs) + zipper.create_zip(**defaults) + + def assertZipFileContent( + self, zf, path, content=None, is_symlink=False, target=None + ): + info = zf.getinfo(path) + if is_symlink: + self.assertTrue( + self.is_symlink(info), + f"{path} should be a symlink but is not", + ) + self.assertEqual(zf.read(path).decode(), target) + else: + self.assertFalse( + self.is_symlink(info), + f"{path} should NOT be a symlink but is", + ) + self.assertEqual(zf.read(path).decode(), content) + + def test_create_zip_with_files_and_symlinks(self): + file1_path = self.test_dir / "file1.txt" + file1_path.write_text("content1") + + link_target_path = "target.txt" # Relative target + symlink_path = self.test_dir / "symlink_source" + symlink_path.symlink_to(link_target_path) + + manifest_content = [ + f"regular|0|file1.txt|{file1_path}", + f"rf-file|0|foo/bar.txt|{file1_path}", + f"rf-symlink|1|link1|{symlink_path}", # Should read target 'target.txt' + f"rf-root-symlink|0|root_file|{file1_path}", + f"rf-empty|empty_file", + ] + self.manifest_path.write_text("\n".join(manifest_content)) + + self._create_zip() + + self.assertTrue(self.output_zip.exists()) + + with zipfile.ZipFile(self.output_zip, "r") as zf: + self.assertEqual( + set(zf.namelist()), + { + "file1.txt", + "runfiles/my_ws/foo/bar.txt", + "runfiles/my_ws/link1", + "runfiles/root_file", + "runfiles/my_ws/empty_file", + }, + ) + + self.assertZipFileContent(zf, "file1.txt", content="content1") + self.assertZipFileContent( + zf, "runfiles/my_ws/foo/bar.txt", content="content1" + ) + self.assertZipFileContent( + zf, "runfiles/my_ws/link1", is_symlink=True, target="target.txt" + ) + self.assertZipFileContent(zf, "runfiles/root_file", content="content1") + self.assertZipFileContent(zf, "runfiles/my_ws/empty_file", content="") + + def test_timestamps_are_deterministic(self): + # Create a content file with a specific recent timestamp + file1_path = self.test_dir / "file1.txt" + file1_path.write_text("content1") + + # Set mtime to something recent (e.g. now) + os.utime(file1_path, None) + + manifest_content = [ + f"regular|0|file1.txt|{file1_path}", + ] + + self.manifest_path.write_text("\n".join(manifest_content)) + + self._create_zip() + + with zipfile.ZipFile(self.output_zip, "r") as zf: + info = zf.getinfo("file1.txt") + # DOS epoch is 1980-01-01 00:00:00 + expected_date_time = (1980, 1, 1, 0, 0, 0) + self.assertEqual(info.date_time, expected_date_time) + + def test_runfiles_mapping_with_cross_repo_paths(self): + # Create content file + file1_path = self.test_dir / "file1.txt" + file1_path.write_text("content1") + + manifest_content = [ + f"rf-file|0|../other_repo/foo.txt|{file1_path}", + f"rf-empty|../other_repo/empty_file", + ] + + self.manifest_path.write_text("\n".join(manifest_content)) + + self._create_zip(workspace_name="my_ws") + + with zipfile.ZipFile(self.output_zip, "r") as zf: + self.assertEqual( + set(zf.namelist()), + { + "runfiles/other_repo/foo.txt", + "runfiles/other_repo/empty_file", + }, + ) + self.assertZipFileContent( + zf, "runfiles/other_repo/foo.txt", content="content1" + ) + self.assertZipFileContent(zf, "runfiles/other_repo/empty_file", content="") + + def test_runfiles_mapping_with_legacy_external_paths(self): + file1_path = self.test_dir / "file1.txt" + file1_path.write_text("content1") + + manifest_content = [ + f"rf-file|0|external/other_repo/foo.txt|{file1_path}", + f"rf-empty|external/other_repo/empty_file", + ] + + self.manifest_path.write_text("\n".join(manifest_content)) + + self._create_zip(workspace_name="my_ws", legacy_external_runfiles=True) + + with zipfile.ZipFile(self.output_zip, "r") as zf: + self.assertEqual( + set(zf.namelist()), + { + "runfiles/other_repo/foo.txt", + "runfiles/other_repo/empty_file", + }, + ) + self.assertZipFileContent( + zf, "runfiles/other_repo/foo.txt", content="content1" + ) + self.assertZipFileContent(zf, "runfiles/other_repo/empty_file", content="") + + def test_output_deterministic(self): + # Create files + file1 = self.test_dir / "file1" + file1.write_text("1") + file2 = self.test_dir / "file2" + file2.write_text("2") + file3 = self.test_dir / "file3" + file3.write_text("3") + + # Manifest entries mixed up + # We want the final order to be: + # 1. a/regular (regular) + # 2. runfiles/a_root_link (rf-root-symlink) + # 3. runfiles/my_ws/b_rf_file (rf-file) + # 4. runfiles/my_ws/c_rf_link (rf-symlink) + # 5. runfiles/my_ws/d_rf_empty (rf-empty) + # 6. z/regular (regular) + + manifest_content = [ + f"regular|0|z/regular|{file1}", + f"rf-file|0|b_rf_file|{file2}", # -> runfiles/my_ws/b_rf_file + f"rf-root-symlink|0|a_root_link|{file3}", # -> runfiles/a_root_link + f"regular|0|a/regular|{file3}", + f"rf-empty|d_rf_empty", # -> runfiles/my_ws/d_rf_empty + f"rf-symlink|0|c_rf_link|{file3}", # -> runfiles/my_ws/c_rf_link + ] + + self.manifest_path.write_text("\n".join(manifest_content)) + + self._create_zip(workspace_name="my_ws") + + with zipfile.ZipFile(self.output_zip, "r") as zf: + self.assertEqual( + zf.namelist(), + [ + "a/regular", + "runfiles/a_root_link", + "runfiles/my_ws/b_rf_file", + "runfiles/my_ws/c_rf_link", + "runfiles/my_ws/d_rf_empty", + "z/regular", + ], + ) + + def is_symlink(self, zip_info): + # Check upper 4 bits of external_attr for S_IFLNK + # S_IFLNK is 0o120000 = 0xA000 + attr = zip_info.external_attr >> 16 + return (attr & 0xF000) == 0xA000 + + +if __name__ == "__main__": + unittest.main() diff --git a/tools/private/zipapp/BUILD.bazel b/tools/private/zipapp/BUILD.bazel new file mode 100644 index 0000000000..7a2002cd72 --- /dev/null +++ b/tools/private/zipapp/BUILD.bazel @@ -0,0 +1,36 @@ +load("//python:py_library.bzl", "py_library") +load("//python/private:py_interpreter_program.bzl", "py_interpreter_program") # buildifier: disable=bzl-visibility + +package( + default_visibility = ["//:__subpackages__"], +) + +py_interpreter_program( + name = "zipper", + main = "zipper.py", + visibility = [ + # Not actually public. Only public so rules_python-generated toolchains + # are able to reference it. + "//visibility:public", + ], +) + +py_library( + name = "zipper_lib", + srcs = ["zipper.py"], +) + +py_interpreter_program( + name = "exe_zip_maker", + main = "exe_zip_maker.py", + visibility = [ + # Not actually public. Only public so rules_python-generated toolchains + # are able to reference it. + "//visibility:public", + ], +) + +py_library( + name = "exe_zip_maker_lib", + srcs = ["exe_zip_maker.py"], +) diff --git a/tools/private/zipapp/exe_zip_maker.py b/tools/private/zipapp/exe_zip_maker.py new file mode 100644 index 0000000000..29391c86da --- /dev/null +++ b/tools/private/zipapp/exe_zip_maker.py @@ -0,0 +1,40 @@ +import hashlib +import os +import shutil +import stat +import sys + +BLOCK_SIZE = 256 * 1024 + + +def create_exe_zip(preamble_path, zip_path, output_path): + sha256_hash = hashlib.sha256() + with open(zip_path, "rb", buffering=BLOCK_SIZE) as f: + for byte_block in iter(lambda: f.read(BLOCK_SIZE), b""): + sha256_hash.update(byte_block) + zip_hash = sha256_hash.hexdigest() + + with open(preamble_path, "rb") as f: + preamble_content = f.read() + + preamble_content = preamble_content.replace(b"%ZIP_HASH%", zip_hash.encode("utf-8")) + + with open(output_path, "wb") as out_f: + out_f.write(preamble_content) + with open(zip_path, "rb") as zip_f: + shutil.copyfileobj(zip_f, out_f, length=BLOCK_SIZE) + + st = os.stat(output_path) + os.chmod(output_path, st.st_mode | stat.S_IEXEC) + + +def main(): + if len(sys.argv) != 4: + print(f"Usage: {sys.argv[0]} ", file=sys.stderr) + sys.exit(1) + + create_exe_zip(sys.argv[1], sys.argv[2], sys.argv[3]) + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/tools/private/zipapp/zipper.py b/tools/private/zipapp/zipper.py new file mode 100644 index 0000000000..6f41c1e663 --- /dev/null +++ b/tools/private/zipapp/zipper.py @@ -0,0 +1,230 @@ +import argparse +import os +import shutil +import stat +import sys +import zipfile + +# Unix permission bit for symlink (S_IFLNK) +# S_IFLNK is usually 0o120000 +S_IFLNK = 0o120000 + + +def _get_zip_runfiles_path( + path, workspace_name, legacy_external_runfiles, runfiles_dir +): + if legacy_external_runfiles and path.startswith("external/"): + path = path[len("external/") :] + elif path.startswith("../"): + path = path[3:] + else: + path = os.path.join(workspace_name, path) + return os.path.join(runfiles_dir, path) + + +def _parse_entry( + line, + line_idx, + workspace_name, + legacy_external_runfiles, + runfiles_dir, +): + line = line.strip() + if not line: + return None + + parts = line.split("|") + type_ = parts[0] + + if type_ == "regular": + _, is_symlink_str, zip_path, content_path = parts + elif type_ == "rf-empty": + _, runfile_path = parts + zip_path = _get_zip_runfiles_path( + runfile_path, workspace_name, legacy_external_runfiles, runfiles_dir + ) + content_path = None # Empty file + is_symlink_str = "0" + elif type_ == "rf-file": + _, is_symlink_str, runfile_path, content_path = parts + zip_path = _get_zip_runfiles_path( + runfile_path, workspace_name, legacy_external_runfiles, runfiles_dir + ) + elif type_ == "rf-symlink": + _, is_symlink_str, runfile_path, content_path = parts + zip_path = os.path.join(runfiles_dir, workspace_name, runfile_path) + elif type_ == "rf-root-symlink": + _, is_symlink_str, runfile_path, content_path = parts + zip_path = os.path.join(runfiles_dir, runfile_path) + else: + raise ValueError( + f"Error: Unknown entry type or invalid format at line {line_idx + 1}: {line}" + ) + + return type_, is_symlink_str, zip_path, content_path + + +def read_manifest( + manifest_path, workspace_name, legacy_external_runfiles, runfiles_dir +): + with open(manifest_path, "r") as f: + entries = [] + for line_idx, line in enumerate(f): + try: + entry = _parse_entry( + line, + line_idx, + workspace_name, + legacy_external_runfiles, + runfiles_dir, + ) + if entry: + entries.append(entry) + except ValueError as e: + e.add_note(f"Error processing line {line_idx + 1}: {line.strip()}") + raise + + # Sort by zip path (3rd element in tuple) + entries.sort(key=lambda x: x[2]) + return entries + + +def _write_entry(zf, entry, compress_type): + type_, is_symlink_str, zip_path, content_path = entry + + if type_ == "rf-empty": + zi = zipfile.ZipInfo(zip_path) + zi.date_time = (1980, 1, 1, 0, 0, 0) + zi.create_system = 3 # Unix + zi.compress_type = compress_type + # Create empty file + zi.external_attr = (0o644 & 0xFFFF) << 16 + zf.writestr(zi, "") + return + + if is_symlink_str == "-1": + if not os.path.exists(content_path): + is_symlink_str = "1" + else: + is_symlink_str = "0" + + is_symlink = is_symlink_str == "1" + + if is_symlink: + zi = zipfile.ZipInfo(zip_path) + zi.date_time = (1980, 1, 1, 0, 0, 0) + zi.create_system = 3 # Unix + zi.compress_type = compress_type + target = os.readlink(content_path) + # Set permissions to 777 for symlink (standard) + zi.external_attr = (S_IFLNK | 0o777) << 16 + zf.writestr(zi, target) + else: + st = os.stat(content_path) + zi = zipfile.ZipInfo(zip_path) + zi.date_time = (1980, 1, 1, 0, 0, 0) + zi.create_system = 3 # Unix + zi.compress_type = compress_type + # Preserve permissions, otherwise execute is dropped. + zi.external_attr = (st.st_mode & 0xFFFF) << 16 + with open(content_path, "rb") as src, zf.open(zi, "w") as dst: + shutil.copyfileobj(src, dst) + + +def create_zip( + *, + manifest_path, + output_zip, + compress_level, + workspace_name, + legacy_external_runfiles, + runfiles_dir, +): + compress_type = zipfile.ZIP_STORED if compress_level == 0 else zipfile.ZIP_DEFLATED + zf_level = compress_level if compress_level != 0 else None + + entries = read_manifest( + manifest_path, workspace_name, legacy_external_runfiles, runfiles_dir + ) + + with zipfile.ZipFile( + output_zip, "w", compress_type, allowZip64=True, compresslevel=zf_level + ) as zf: + for entry in entries: + _write_entry(zf, entry, compress_type) + + +def main(): + parser = argparse.ArgumentParser(description="Create a zip file from a manifest.") + parser.add_argument( + "manifest", + help=""" +Path to the manifest file. Lines have one of the following formats: + +1. `regular|is_symlink|zip_path|content_path`: This form stores the `zip_path` + in the zip, whose content is taken from `content_path` + +2. `rf-empty|runfile_path`: A `runfiles.empty_filenames` value. The stored + zip path is computed from `runfile_path` + +3. `rf-file|is_symlink|runfile_path|content_path`: Store a file in + the zip. The zip path is computed from `runfile_path`. + +4. `rf-symlink|is_symlink|runfile_symlink_path|content_path`: Store a + main-repo-relative path in the zip. + +5. `rf-root-symlink|is_symlink|runfile_root_path|content_path`: Store a + runfiles-root-relative path in the zip. + +In all cases, `is_symlink` has the following values: +* `1` means it should be stored as a symlink whose value is read + (using `readlink()`) from `content_path`. +* `0` means to store it as a regular file, read from `content_path` +* `-1` occurs with Bazel 7 (because it lacks `File.is_symlink`), which means + to infer whether it's a symlink (files to be stored as symlinks can be + determined by looking for symlinks that point to non-existent files). + +For runfiles entries, they have `--runfiles-dir` prepended to their computed +zip path. + +Compute `zip_path` from `runfile_path`: Computing the final zip path for +runfiles entries is a bit complicated, but boils down to computing what the +runfiles-root-relative path would be, with `--legacy-external-runfiles` taken +into account. +""", + ) + parser.add_argument("output", help="Path to the output zip file.") + parser.add_argument( + "--compression", + type=int, + default=0, + help="Compression level (0 for stored, others for deflated)", + ) + parser.add_argument("--workspace-name", default="", help="Name of the workspace") + parser.add_argument( + "--legacy-external-runfiles", + default="0", + choices=["0", "1"], + help="Whether to use legacy external runfiles behavior", + ) + parser.add_argument( + "--runfiles-dir", default="runfiles", help="Name of the runfiles directory" + ) + args = parser.parse_args() + + try: + create_zip( + manifest_path=args.manifest, + output_zip=args.output, + compress_level=args.compression, + workspace_name=args.workspace_name, + legacy_external_runfiles=args.legacy_external_runfiles == "1", + runfiles_dir=args.runfiles_dir, + ) + except Exception as e: + e.add_note(f"Error creating zip {args.output}") + raise + + +if __name__ == "__main__": + sys.exit(main()) From 393dcfd194615ab4d3393d29145842c186597e0e Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Sun, 1 Feb 2026 17:01:09 -0800 Subject: [PATCH 608/922] chore: remove defunct `_py_toolchain_type` py_binary attribute (#3560) The `_py_toolchain_type` attribute is a holdover from when the rules were implemented in Java and some Google-internal tests were relying on it. --- python/private/py_executable.bzl | 8 +------- 1 file changed, 1 insertion(+), 7 deletions(-) diff --git a/python/private/py_executable.bzl b/python/private/py_executable.bzl index 3dd1b28234..5c3cb77c91 100644 --- a/python/private/py_executable.bzl +++ b/python/private/py_executable.bzl @@ -63,7 +63,7 @@ load(":py_internal.bzl", "py_internal") load(":py_runtime_info.bzl", "DEFAULT_STUB_SHEBANG") load(":reexports.bzl", "BuiltinPyInfo", "BuiltinPyRuntimeInfo") load(":rule_builders.bzl", "ruleb") -load(":toolchain_types.bzl", "EXEC_TOOLS_TOOLCHAIN_TYPE", "TARGET_TOOLCHAIN_TYPE", TOOLCHAIN_TYPE = "TARGET_TOOLCHAIN_TYPE") +load(":toolchain_types.bzl", "EXEC_TOOLS_TOOLCHAIN_TYPE", TOOLCHAIN_TYPE = "TARGET_TOOLCHAIN_TYPE") load(":transition_labels.bzl", "TRANSITION_LABELS") load(":venv_runfiles.bzl", "create_venv_app_files") @@ -222,12 +222,6 @@ accepting arbitrary Python versions. # empty target for other platforms. default = "//tools/launcher:launcher", ), - # TODO: This appears to be vestigial. It's only added because - # GraphlessQueryTest.testLabelsOperator relies on it to test for - # query behavior of implicit dependencies. - "_py_toolchain_type": attr.label( - default = TARGET_TOOLCHAIN_TYPE, - ), "_python_version_flag": lambda: attrb.Label( default = labels.PYTHON_VERSION, ), From 39e9a3f2d7844128c7e78ad42f1e373ba8fa0a7a Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Tue, 3 Feb 2026 16:44:30 -0800 Subject: [PATCH 609/922] chore: add some type information about hub builder (#3558) While reading hub_builder.bzl, I had to track down a lot of source to figure out what variables were tracking what. Add various comments to describe variable types and their purpose. --- python/private/pypi/extension.bzl | 2 + python/private/pypi/hub_builder.bzl | 70 +++++++++++++++++++++++++++-- 2 files changed, 69 insertions(+), 3 deletions(-) diff --git a/python/private/pypi/extension.bzl b/python/private/pypi/extension.bzl index 3927f61c00..7354a67e67 100644 --- a/python/private/pypi/extension.bzl +++ b/python/private/pypi/extension.bzl @@ -221,6 +221,8 @@ You cannot use both the additive_build_content and additive_build_content_file a # Used to track all the different pip hubs and the spoke pip Python # versions. + # dict[str repo, HubBuilder] + # See `hub_builder.bzl%hub_builder()` for `HubBuilder` pip_hub_map = {} simpleapi_cache = {} diff --git a/python/private/pypi/hub_builder.bzl b/python/private/pypi/hub_builder.bzl index f54d02d8b0..f0aa6a73bc 100644 --- a/python/private/pypi/hub_builder.bzl +++ b/python/private/pypi/hub_builder.bzl @@ -59,19 +59,42 @@ def hub_builder( build = lambda: _build(self), pip_parse = lambda *a, **k: _pip_parse(self, *a, **k), - # build output + # Build output related + + # The set of package names to expose. Dict acting as set + # dict[str package, None] _exposed_packages = {}, # modified by _add_exposed_packages + # Map of the per-package aliases. + # The nested dict is a dict acting as a set. + # dict[str whl_name, dict[str alias_name, bool]] _extra_aliases = {}, # modified by _add_extra_aliases + # dict[str group_name, list[str]] _group_map = {}, # modified by _add_group_map + # Mapping of whl_library repo names and their kwargs. + # dict[str repo_name, dict[str, object] kwargs] _whl_libraries = {}, # modified by _add_whl_library + # Map of repos and their config settings, and repo the config + # setting originated from. + # dict[str whl_name, dict[str config_setting, str repo_name]] _whl_map = {}, # modified by _add_whl_library - # internal + + # Internal + + # dict[str python_version, dict[str platform, PlatformInfo]] + # where `PlatformInfo` is from `_platforms()` _platforms = {}, + # Supplemental index of `_group_map` + # dict[str whl_name, str group_name] _group_name_by_whl = {}, + # Functions to download according to the config + # dict[str python_version, callable] _get_index_urls = {}, + # Tells whether to use the downloader for a package. + # dict[str python_version, dict[str package_name, bool use_downloader]] _use_downloader = {}, _simpleapi_cache = simpleapi_cache, - # instance constants + + # Instance constants passed in by callers _config = config, _whl_overrides = whl_overrides, _evaluate_markers_fn = evaluate_markers_fn, @@ -103,13 +126,24 @@ def _build(self): whl_map.setdefault(key, {}).setdefault(repo, []).append(setting) return struct( + # The config settings for matching repo spokes. + # dict[str repo_name, dict[str repo_name, list[str]]] whl_map = whl_map, + # Maps a wheel to a list of groups + # dict[str group_name, list[str]] group_map = self._group_map, + # The per-package aliases for the hub to create. + # dict[str package, list[str]] extra_aliases = { whl: sorted(aliases) for whl, aliases in self._extra_aliases.items() }, + # The list of exposed packages in the hub. + # list[str] exposed_packages = sorted(self._exposed_packages), + + # Mapping of whl_library repo names and their kwargs. + # dict[str repo_name, dict[str, object] kwargs] whl_libraries = self._whl_libraries, ) @@ -171,6 +205,13 @@ def _pip_parse(self, module_ctx, pip_attr): ### setters for build outputs def _add_exposed_packages(self, exposed_packages): + """Add packages that are exposed. + + Args: + self: implicitly added + exposed_packages: {type}`dict[str package, None]` a dict acting as + a set. The set of packages that should be exposed. + """ if self._exposed_packages: intersection = {} for pkg in exposed_packages: @@ -183,6 +224,13 @@ def _add_exposed_packages(self, exposed_packages): self._exposed_packages.update(exposed_packages) def _add_group_map(self, group_map): + """Adds a group mapping for cycle breaking. + + Args: + self: implicitly added. + group_map: {type}`dict[str name, list[str]]` + """ + # TODO @aignas 2024-04-05: how do we support different requirement # cycles for different abis/oses? For now we will need the users to # assume the same groups across all versions/platforms until we start @@ -202,6 +250,13 @@ def _add_group_map(self, group_map): }) def _add_extra_aliases(self, extra_hub_aliases): + """Adds per-package aliases to the hub. + + Args: + self: Implicitly added + extra_hub_aliases: {type}`dict[str package, list[str]]` Alias target + names to add to a package's hub BUILD file. + """ for whl_name, aliases in extra_hub_aliases.items(): self._extra_aliases.setdefault(whl_name, {}).update( {alias: True for alias in aliases}, @@ -246,6 +301,15 @@ def _diff_dict(first, second): return None def _add_whl_library(self, *, python_version, whl, repo, enable_pipstar): + """Add a whl_library and kwargs to call it with for the hub. + + Args: + self: implicitly added + python_version: {type}`str` the python version to assume + whl: struct from `_whl_library_args()` + repo: struct from `_whl_repo` + enable_pipstar: {type}`bool` if pipstar is enabled. + """ if repo == None: # NOTE @aignas 2025-07-07: we guard against an edge-case where there # are more platforms defined than there are wheels for and users From 3aa6386c38c2ab4bc9c447ef6893ef216faa0131 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Thu, 5 Feb 2026 19:06:08 -0800 Subject: [PATCH 610/922] feat(zipapp): add windows support (#3561) This adds Windows support to the py_zipapp rules. It works the same as what py_binary does: the Bazel builtin launcher.exe is populated with the path to the zip, and the exe then handles starting Python and running the zip. This should be the last of the implementation, so all that remains is deprecating the code in py_binary and removing it. Work towards https://github.com/bazel-contrib/rules_python/issues/2586 --- python/private/common.bzl | 47 ++++++++++- python/private/py_executable.bzl | 45 +--------- python/private/toolchain_types.bzl | 1 + python/private/zipapp/py_zipapp_rule.bzl | 71 ++++++++++++---- python/private/zipapp/zip_main_template.py | 83 ++++++++----------- tests/py_zipapp/BUILD.bazel | 42 ++++++++-- tests/py_zipapp/main.py | 13 ++- tests/py_zipapp/some_dep.py | 1 + ...m_python_zipapp_external_bootstrap_test.sh | 17 ++++ tests/py_zipapp/system_python_zipapp_test.py | 9 +- tests/py_zipapp/venv_zipapp_test.py | 54 +++++++++--- 11 files changed, 248 insertions(+), 135 deletions(-) create mode 100644 tests/py_zipapp/some_dep.py create mode 100755 tests/py_zipapp/system_python_zipapp_external_bootstrap_test.sh diff --git a/python/private/common.bzl b/python/private/common.bzl index ca201cb10f..6d9e0e3c84 100644 --- a/python/private/common.bzl +++ b/python/private/common.bzl @@ -18,7 +18,7 @@ load("@rules_cc//cc/common:cc_common.bzl", "cc_common") load("@rules_cc//cc/common:cc_info.bzl", "CcInfo") load("@rules_python_internal//:rules_python_config.bzl", "config") load("//python/private:py_interpreter_program.bzl", "PyInterpreterProgramInfo") -load("//python/private:toolchain_types.bzl", "EXEC_TOOLS_TOOLCHAIN_TYPE") +load("//python/private:toolchain_types.bzl", "EXEC_TOOLS_TOOLCHAIN_TYPE", "LAUNCHER_MAKER_TOOLCHAIN_TYPE") load(":builders.bzl", "builders") load(":cc_helper.bzl", "cc_helper") load(":py_cc_link_params_info.bzl", "PyCcLinkParamsInfo") @@ -56,6 +56,51 @@ def maybe_builtin_build_python_zip(value, settings = None): return settings +def _find_launcher_maker(ctx): + if config.bazel_9_or_later: + return (ctx.toolchains[LAUNCHER_MAKER_TOOLCHAIN_TYPE].binary, LAUNCHER_MAKER_TOOLCHAIN_TYPE) + return (ctx.executable._windows_launcher_maker, None) + +def create_windows_exe_launcher( + ctx, + *, + output, + python_binary_path, + use_zip_file): + """Creates a Windows exe launcher. + + Args: + ctx: The rule context. + output: The output file for the launcher. + python_binary_path: The path to the Python binary. + use_zip_file: Whether to use a zip file. + """ + launch_info = ctx.actions.args() + launch_info.use_param_file("%s", use_always = True) + launch_info.set_param_file_format("multiline") + launch_info.add("binary_type=Python") + launch_info.add(ctx.workspace_name, format = "workspace_name=%s") + launch_info.add( + "1" if py_internal.runfiles_enabled(ctx) else "0", + format = "symlink_runfiles_enabled=%s", + ) + launch_info.add(python_binary_path, format = "python_bin_path=%s") + launch_info.add("1" if use_zip_file else "0", format = "use_zip_file=%s") + + launcher = ctx.attr._launcher[DefaultInfo].files_to_run.executable + executable, toolchain = _find_launcher_maker(ctx) + ctx.actions.run( + executable = executable, + arguments = [launcher.path, launch_info, output.path], + inputs = [launcher], + outputs = [output], + mnemonic = "PyBuildLauncher", + progress_message = "Creating launcher for %{label}", + # Needed to inherit PATH when using non-MSVC compilers like MinGW + use_default_shell_env = True, + toolchain = toolchain, + ) + def create_binary_semantics_struct( *, get_native_deps_dso_name, diff --git a/python/private/py_executable.bzl b/python/private/py_executable.bzl index 5c3cb77c91..9f1113d9f6 100644 --- a/python/private/py_executable.bzl +++ b/python/private/py_executable.bzl @@ -45,6 +45,7 @@ load( "create_instrumented_files_info", "create_output_group_info", "create_py_info", + "create_windows_exe_launcher", "csv", "filter_to_py_srcs", "is_bool", @@ -63,7 +64,7 @@ load(":py_internal.bzl", "py_internal") load(":py_runtime_info.bzl", "DEFAULT_STUB_SHEBANG") load(":reexports.bzl", "BuiltinPyInfo", "BuiltinPyRuntimeInfo") load(":rule_builders.bzl", "ruleb") -load(":toolchain_types.bzl", "EXEC_TOOLS_TOOLCHAIN_TYPE", TOOLCHAIN_TYPE = "TARGET_TOOLCHAIN_TYPE") +load(":toolchain_types.bzl", "EXEC_TOOLS_TOOLCHAIN_TYPE", "LAUNCHER_MAKER_TOOLCHAIN_TYPE", TOOLCHAIN_TYPE = "TARGET_TOOLCHAIN_TYPE") load(":transition_labels.bzl", "TRANSITION_LABELS") load(":venv_runfiles.bzl", "create_venv_app_files") @@ -71,7 +72,6 @@ _py_builtins = py_internal _EXTERNAL_PATH_PREFIX = "external" _ZIP_RUNFILES_DIRECTORY_NAME = "runfiles" _INIT_PY = "__init__.py" -_LAUNCHER_MAKER_TOOLCHAIN_TYPE = "@bazel_tools//tools/launcher:launcher_maker_toolchain_type" # Non-Google-specific attributes for executables # These attributes are for rules that accept Python sources. @@ -398,7 +398,7 @@ def _create_executable( else: bootstrap_output = executable else: - _create_windows_exe_launcher( + create_windows_exe_launcher( ctx, output = executable, use_zip_file = build_zip_enabled, @@ -789,43 +789,6 @@ def _create_stage1_bootstrap( is_executable = True, ) -def _find_launcher_maker(ctx): - if rp_config.bazel_9_or_later: - return (ctx.toolchains[_LAUNCHER_MAKER_TOOLCHAIN_TYPE].binary, _LAUNCHER_MAKER_TOOLCHAIN_TYPE) - return (ctx.executable._windows_launcher_maker, None) - -def _create_windows_exe_launcher( - ctx, - *, - output, - python_binary_path, - use_zip_file): - launch_info = ctx.actions.args() - launch_info.use_param_file("%s", use_always = True) - launch_info.set_param_file_format("multiline") - launch_info.add("binary_type=Python") - launch_info.add(ctx.workspace_name, format = "workspace_name=%s") - launch_info.add( - "1" if py_internal.runfiles_enabled(ctx) else "0", - format = "symlink_runfiles_enabled=%s", - ) - launch_info.add(python_binary_path, format = "python_bin_path=%s") - launch_info.add("1" if use_zip_file else "0", format = "use_zip_file=%s") - - launcher = ctx.attr._launcher[DefaultInfo].files_to_run.executable - executable, toolchain = _find_launcher_maker(ctx) - ctx.actions.run( - executable = executable, - arguments = [launcher.path, launch_info, output.path], - inputs = [launcher], - outputs = [output], - mnemonic = "PyBuildLauncher", - progress_message = "Creating launcher for %{label}", - # Needed to inherit PATH when using non-MSVC compilers like MinGW - use_default_shell_env = True, - toolchain = toolchain, - ) - def _create_zip_file(ctx, *, output, zip_main, runfiles): """Create a Python zipapp (zip with __main__.py entry point).""" workspace_name = ctx.workspace_name @@ -1848,7 +1811,7 @@ def create_executable_rule_builder(implementation, **kwargs): ruleb.ToolchainType(TOOLCHAIN_TYPE), ruleb.ToolchainType(EXEC_TOOLS_TOOLCHAIN_TYPE, mandatory = False), ruleb.ToolchainType("@bazel_tools//tools/cpp:toolchain_type", mandatory = False), - ] + ([ruleb.ToolchainType(_LAUNCHER_MAKER_TOOLCHAIN_TYPE)] if rp_config.bazel_9_or_later else []), + ] + ([ruleb.ToolchainType(LAUNCHER_MAKER_TOOLCHAIN_TYPE)] if rp_config.bazel_9_or_later else []), cfg = dict( implementation = _transition_executable_impl, inputs = TRANSITION_LABELS + [ diff --git a/python/private/toolchain_types.bzl b/python/private/toolchain_types.bzl index ef81bf3bd4..5b5cce90ee 100644 --- a/python/private/toolchain_types.bzl +++ b/python/private/toolchain_types.bzl @@ -21,3 +21,4 @@ implementation of the toolchain. TARGET_TOOLCHAIN_TYPE = Label("//python:toolchain_type") EXEC_TOOLS_TOOLCHAIN_TYPE = Label("//python:exec_tools_toolchain_type") PY_CC_TOOLCHAIN_TYPE = Label("//python/cc:toolchain_type") +LAUNCHER_MAKER_TOOLCHAIN_TYPE = Label("@bazel_tools//tools/launcher:launcher_maker_toolchain_type") diff --git a/python/private/zipapp/py_zipapp_rule.bzl b/python/private/zipapp/py_zipapp_rule.bzl index cc399064ad..a0f0df943f 100644 --- a/python/private/zipapp/py_zipapp_rule.bzl +++ b/python/private/zipapp/py_zipapp_rule.bzl @@ -1,14 +1,15 @@ """Implementation of the zipapp rules.""" load("@bazel_skylib//lib:paths.bzl", "paths") +load("@rules_python_internal//:rules_python_config.bzl", rp_config = "config") load("//python/private:attributes.bzl", "apply_config_settings_attr") load("//python/private:builders.bzl", "builders") -load("//python/private:common.bzl", "BUILTIN_BUILD_PYTHON_ZIP", "actions_run", "maybe_builtin_build_python_zip", "maybe_create_repo_mapping", "runfiles_root_path") +load("//python/private:common.bzl", "BUILTIN_BUILD_PYTHON_ZIP", "actions_run", "create_windows_exe_launcher", "maybe_builtin_build_python_zip", "maybe_create_repo_mapping", "runfiles_root_path", "target_platform_has_any_constraint") load("//python/private:common_labels.bzl", "labels") load("//python/private:py_executable_info.bzl", "PyExecutableInfo") load("//python/private:py_internal.bzl", "py_internal") load("//python/private:py_runtime_info.bzl", "PyRuntimeInfo") -load("//python/private:toolchain_types.bzl", "EXEC_TOOLS_TOOLCHAIN_TYPE") +load("//python/private:toolchain_types.bzl", "EXEC_TOOLS_TOOLCHAIN_TYPE", "LAUNCHER_MAKER_TOOLCHAIN_TYPE") load("//python/private:transition_labels.bzl", "TRANSITION_LABELS") def _is_symlink(f): @@ -18,13 +19,11 @@ def _is_symlink(f): return "-1" def _create_zipapp_main_py(ctx, py_runtime, py_executable, stage2_bootstrap): - python_exe = py_executable.venv_python_exe - if python_exe: - python_exe_path = runfiles_root_path(ctx, python_exe.short_path) - elif py_runtime.interpreter: - python_exe_path = runfiles_root_path(ctx, py_runtime.interpreter.short_path) + venv_python_exe = py_executable.venv_python_exe + if venv_python_exe: + venv_python_exe_path = runfiles_root_path(ctx, venv_python_exe.short_path) else: - python_exe_path = py_runtime.interpreter_path + venv_python_exe_path = "" if py_runtime.interpreter: python_binary_actual_path = runfiles_root_path(ctx, py_runtime.interpreter.short_path) @@ -36,7 +35,7 @@ def _create_zipapp_main_py(ctx, py_runtime, py_executable, stage2_bootstrap): template = py_runtime.zip_main_template, output = zip_main_py, substitutions = { - "%python_binary%": python_exe_path, + "%python_binary%": venv_python_exe_path, "%python_binary_actual%": python_binary_actual_path, "%stage2_bootstrap%": runfiles_root_path(ctx, stage2_bootstrap.short_path), "%workspace_name%": ctx.workspace_name, @@ -184,20 +183,39 @@ def _py_zipapp_executable_impl(ctx): zip_file = _create_zip(ctx, py_runtime, py_executable, stage2_bootstrap) if ctx.attr.executable: - preamble = _create_shell_bootstrap(ctx, py_runtime, py_executable, stage2_bootstrap) - executable = _create_self_executable_zip(ctx, preamble, zip_file) - default_output = executable + if target_platform_has_any_constraint(ctx, ctx.attr._windows_constraints): + executable = ctx.actions.declare_file(ctx.label.name + ".exe") + + python_exe = py_executable.venv_python_exe + if python_exe: + python_exe_path = runfiles_root_path(ctx, python_exe.short_path) + elif py_runtime.interpreter: + python_exe_path = runfiles_root_path(ctx, py_runtime.interpreter.short_path) + else: + python_exe_path = py_runtime.interpreter_path + + create_windows_exe_launcher( + ctx, + output = executable, + python_binary_path = python_exe_path, + use_zip_file = True, + ) + default_outputs = [executable, zip_file] + else: + preamble = _create_shell_bootstrap(ctx, py_runtime, py_executable, stage2_bootstrap) + executable = _create_self_executable_zip(ctx, preamble, zip_file) + default_outputs = [executable] else: # Bazel requires executable=True rules to have an executable given, so give # a fake one to satisfy that. - default_output = zip_file + default_outputs = [zip_file] executable = ctx.actions.declare_file(ctx.label.name + "-not-executable") ctx.actions.write(executable, "echo 'ERROR: Non executable zip file'; exit 1") return [ DefaultInfo( - files = depset([default_output]), - runfiles = ctx.runfiles(files = [default_output]), + files = depset(default_outputs), + runfiles = ctx.runfiles(files = default_outputs), executable = executable, ), ] @@ -277,6 +295,18 @@ Whether the output should be an executable zip file. cfg = "exec", default = "//tools/private/zipapp:exe_zip_maker", ), + "_launcher": attr.label( + cfg = "target", + # NOTE: This is an executable, but is only used for Windows. It + # can't have executable=True because the backing target is an + # empty target for other platforms. + default = "//tools/launcher:launcher", + ), + "_windows_constraints": attr.label_list( + default = [ + "@platforms//os:windows", + ], + ), "_zip_shell_template": attr.label( default = ":zip_shell_template", allow_single_file = True, @@ -285,8 +315,15 @@ Whether the output should be an executable zip file. cfg = "exec", default = "//tools/private/zipapp:zipper", ), -} -_TOOLCHAINS = [EXEC_TOOLS_TOOLCHAIN_TYPE] +} | ({ + "_windows_launcher_maker": attr.label( + default = "@bazel_tools//tools/launcher:launcher_maker", + cfg = "exec", + executable = True, + ), +} if not rp_config.bazel_9_or_later else {}) + +_TOOLCHAINS = [EXEC_TOOLS_TOOLCHAIN_TYPE] + ([LAUNCHER_MAKER_TOOLCHAIN_TYPE] if rp_config.bazel_9_or_later else []) py_zipapp_binary = rule( doc = """ diff --git a/python/private/zipapp/zip_main_template.py b/python/private/zipapp/zip_main_template.py index 35db1645bc..dba5049464 100644 --- a/python/private/zipapp/zip_main_template.py +++ b/python/private/zipapp/zip_main_template.py @@ -30,7 +30,7 @@ # runfiles-root-relative path _STAGE2_BOOTSTRAP = "%stage2_bootstrap%" # runfiles-root-relative path to venv's bin/python3. Empty if venv not being used. -_PYTHON_BINARY = "%python_binary%" +_PYTHON_BINARY_VENV = "%python_binary%" # runfiles-root-relative path, absolute path, or single word. The actual Python # executable to use. _PYTHON_BINARY_ACTUAL = "%python_binary_actual%" @@ -106,11 +106,11 @@ def has_windows_executable_extension(path): if ( - _PYTHON_BINARY + _PYTHON_BINARY_VENV and is_windows() - and not has_windows_executable_extension(_PYTHON_BINARY) + and not has_windows_executable_extension(_PYTHON_BINARY_VENV) ): - _PYTHON_BINARY = _PYTHON_BINARY + ".exe" + _PYTHON_BINARY_VENV = _PYTHON_BINARY_VENV + ".exe" def search_path(name): @@ -124,14 +124,6 @@ def search_path(name): return None -def find_python_binary(module_space): - """Finds the real Python binary if it's not a normal absolute path.""" - if _PYTHON_BINARY: - return find_binary(module_space, _PYTHON_BINARY) - else: - return find_binary(module_space, _PYTHON_BINARY_ACTUAL) - - def find_binary(module_space, bin_name): """Finds the real binary if it's not a normal absolute path.""" if not bin_name: @@ -139,7 +131,7 @@ def find_binary(module_space, bin_name): if bin_name.startswith("//"): # Case 1: Path is a label. Not supported yet. raise AssertionError( - "Bazel does not support execution of Python interpreters via labels yet" + "Bazel does not support execution of Python interpreters via labels" ) elif os.path.isabs(bin_name): # Case 2: Absolute path. @@ -221,7 +213,7 @@ def execute_file( # - On Windows, os.execv doesn't handle arguments with spaces # correctly, and it actually starts a subprocess just like # subprocess.call. - # - When running in a workspace or zip file, we need to clean up the + # - When running in a zip file, we need to clean up the # workspace after the process finishes so control must return here. try: subprocess_argv = [python_program, main_filename] + args @@ -241,16 +233,18 @@ def main(): print_verbose("running zip main bootstrap") print_verbose("initial argv:", values=sys.argv) print_verbose("initial environ:", mapping=os.environ) - print_verbose("initial sys.executable", sys.executable) - print_verbose("initial sys.version", sys.version) + print_verbose("initial sys.executable:", sys.executable) + print_verbose("initial sys.version:", sys.version) + print_verbose("stage2_bootstrap:", _STAGE2_BOOTSTRAP) + print_verbose("python_binary_venv:", _PYTHON_BINARY_VENV) + print_verbose("python_binary_actual:", _PYTHON_BINARY_ACTUAL) + print_verbose("workspace_name:", _WORKSPACE_NAME) args = sys.argv[1:] new_env = {} # The main Python source file. - # The magic string percent-main-percent is replaced with the runfiles-relative - # filename of the main file of the Python binary in BazelPythonSemantics.java. main_rel_path = _STAGE2_BOOTSTRAP if is_windows(): main_rel_path = main_rel_path.replace("/", os.sep) @@ -273,38 +267,33 @@ def main(): "Cannot exec() %r: file not readable." % main_filename ) - python_program = find_python_binary(module_space) - if python_program is None: - raise AssertionError("Could not find python binary: " + _PYTHON_BINARY) - - # When a venv is used, the `bin/python3` symlink has to be recreated. - if _PYTHON_BINARY: - # The venv bin/python3 interpreter should always be under runfiles, but - # double check. We don't want to accidentally create symlinks elsewhere. - if not python_program.startswith(module_space): - raise AssertionError( - "Program's venv binary not under runfiles: {python_program}" - ) - - if os.path.isabs(_PYTHON_BINARY_ACTUAL): - symlink_to = _PYTHON_BINARY_ACTUAL - elif "/" in _PYTHON_BINARY_ACTUAL: - symlink_to = os.path.join(module_space, _PYTHON_BINARY_ACTUAL) - else: - symlink_to = search_path(_PYTHON_BINARY_ACTUAL) - if not symlink_to: + if _PYTHON_BINARY_VENV: + python_program = os.path.join(module_space, _PYTHON_BINARY_VENV) + # When a venv is used, the `bin/python3` symlink may need to be created. + # This case occurs when "create venv at runtime" or "resolve python at + # runtime" modes are enabled. + if not os.path.lexists(python_program): + # The venv bin/python3 interpreter should always be under runfiles, but + # double check. We don't want to accidentally create symlinks elsewhere + if not python_program.startswith(module_space): raise AssertionError( - f"Python interpreter to use not found on PATH: {_PYTHON_BINARY_ACTUAL}" + "Program's venv binary not under runfiles: {python_program}" ) + symlink_to = find_binary(module_space, _PYTHON_BINARY_ACTUAL) + os.makedirs(os.path.dirname(python_program), exist_ok=True) + try: + os.symlink(symlink_to, python_program) + except OSError as e: + raise Exception( + f"Unable to create venv python interpreter symlink: {python_program} -> {symlink_to}" + ) from e - # The bin/ directory may not exist if it is empty. - os.makedirs(os.path.dirname(python_program), exist_ok=True) - try: - os.symlink(symlink_to, python_program) - except OSError as e: - raise Exception( - f"Unable to create venv python interpreter symlink: {python_program} -> {symlink_to}" - ) from e + else: + python_program = find_binary(module_space, _PYTHON_BINARY_ACTUAL) + if python_program is None: + raise AssertionError( + "Could not find python binary: " + _PYTHON_BINARY_ACTUAL + ) # Some older Python versions on macOS (namely Python 3.7) may unintentionally # leave this environment variable set after starting the interpreter, which diff --git a/tests/py_zipapp/BUILD.bazel b/tests/py_zipapp/BUILD.bazel index 74df4aa04d..da42567656 100644 --- a/tests/py_zipapp/BUILD.bazel +++ b/tests/py_zipapp/BUILD.bazel @@ -1,9 +1,10 @@ +load("@rules_shell//shell:sh_test.bzl", "sh_test") load("//python:py_binary.bzl", "py_binary") +load("//python:py_library.bzl", "py_library") load("//python:py_test.bzl", "py_test") load("//python/private:bzlmod_enabled.bzl", "BZLMOD_ENABLED") # buildifier: disable=bzl-visibility load("//python/zipapp:py_zipapp_binary.bzl", "py_zipapp_binary") load("//tests/support:support.bzl", "NOT_WINDOWS") - # todo: add windows support. Windows support will be a bit odd. # It previously worked by having special logic in the exe launcher # that knew to look for .zip and running that through python @@ -17,7 +18,7 @@ py_binary( }, main = "main.py", target_compatible_with = NOT_WINDOWS, - deps = ["@dev_pip//absl_py"], + deps = [":some_dep"], ) py_zipapp_binary( @@ -45,15 +46,12 @@ py_binary( "//python/config_settings:venvs_site_packages": "no", }, main = "main.py", - # TODO: #2586 - Add windows support - target_compatible_with = NOT_WINDOWS, - deps = ["@dev_pip//absl_py"], + deps = [":some_dep"], ) py_zipapp_binary( name = "system_python_zipapp", binary = ":system_python_bin", - target_compatible_with = NOT_WINDOWS, ) py_test( @@ -63,5 +61,35 @@ py_test( env = { "TEST_ZIPAPP": "$(location :system_python_zipapp)", }, - target_compatible_with = NOT_WINDOWS, +) + +sh_test( + name = "system_python_zipapp_external_bootstrap_test", + srcs = ["system_python_zipapp_external_bootstrap_test.sh"], + data = [ + ":system_python_zipapp", + "//python:current_py_toolchain", + ], + env = { + "PYTHON": "$(PYTHON3_ROOTPATH)", + "ZIPAPP": "$(location :system_python_zipapp)", + }, + toolchains = ["//python:current_py_toolchain"], +) + +py_library( + name = "some_dep", + srcs = ["some_dep.py"], + experimental_venvs_site_packages = "//python/config_settings:venvs_site_packages", + imports = select({ + ":is_venvs_site_packages_enabled": ["tests/py_zipapp"], + "//conditions:default": ["."], + }), +) + +config_setting( + name = "is_venvs_site_packages_enabled", + flag_values = { + "//python/config_settings:venvs_site_packages": "yes", + }, ) diff --git a/tests/py_zipapp/main.py b/tests/py_zipapp/main.py index b8fdbe365e..8e67ec9fae 100644 --- a/tests/py_zipapp/main.py +++ b/tests/py_zipapp/main.py @@ -3,9 +3,18 @@ def main(): print("Hello from zipapp") - import absl + try: + import some_dep - print(f"absl: {absl}") + print(f"dep: {some_dep}") + except ImportError: + import sys + + print("Failed to import `some_dep`", file=sys.stderr) + print("sys.path:", file=sys.stderr) + for i, x in enumerate(sys.path): + print(i, x, file=sys.stderr) + raise if __name__ == "__main__": diff --git a/tests/py_zipapp/some_dep.py b/tests/py_zipapp/some_dep.py new file mode 100644 index 0000000000..b64ecfb84a --- /dev/null +++ b/tests/py_zipapp/some_dep.py @@ -0,0 +1 @@ +"""empty module""" diff --git a/tests/py_zipapp/system_python_zipapp_external_bootstrap_test.sh b/tests/py_zipapp/system_python_zipapp_external_bootstrap_test.sh new file mode 100755 index 0000000000..4710741a5a --- /dev/null +++ b/tests/py_zipapp/system_python_zipapp_external_bootstrap_test.sh @@ -0,0 +1,17 @@ +#!/usr/bin/env bash + +set -xeuo pipefail + +# This test expects ZIPAPP env var to point to the zipapp file. +if [[ -z "${ZIPAPP:-}" ]]; then + echo "ZIPAPP env var not set" + exit 1 +fi + +# On Windows, the executable file is an exe, and the .zip is a sibling +# output. +ZIPAPP="${ZIPAPP/.exe/.zip}" + +# We're testing the invocation of `__main__.py`, so we have to +# manually pass the zipapp to python. +"$PYTHON" "$ZIPAPP" diff --git a/tests/py_zipapp/system_python_zipapp_test.py b/tests/py_zipapp/system_python_zipapp_test.py index 297a574f16..ec0f837135 100644 --- a/tests/py_zipapp/system_python_zipapp_test.py +++ b/tests/py_zipapp/system_python_zipapp_test.py @@ -5,20 +5,15 @@ class SystemPythonZipAppTest(unittest.TestCase): - def test_zipapp_contents(self): + def test_zipapp_runnable(self): zipapp_path = os.environ["TEST_ZIPAPP"] self.assertTrue(os.path.exists(zipapp_path)) self.assertTrue(os.path.isfile(zipapp_path)) - # The zipapp itself is a shell script prepended to the zip file. - with open(zipapp_path, "rb") as f: - content = f.read() - self.assertTrue(content.startswith(b"#!/usr/bin/env bash")) - output = subprocess.check_output([zipapp_path]).decode("utf-8").strip() self.assertIn("Hello from zipapp", output) - self.assertIn("absl", output) + self.assertIn("dep:", output) if __name__ == "__main__": diff --git a/tests/py_zipapp/venv_zipapp_test.py b/tests/py_zipapp/venv_zipapp_test.py index 40d20fedb4..9bb917156f 100644 --- a/tests/py_zipapp/venv_zipapp_test.py +++ b/tests/py_zipapp/venv_zipapp_test.py @@ -1,3 +1,4 @@ +import contextlib import os import subprocess import unittest @@ -5,24 +6,34 @@ class PyZipAppTest(unittest.TestCase): - def test_zipapp_contents(self): + def test_zipapp_runnable(self): zipapp_path = os.environ["TEST_ZIPAPP"] - self.assertTrue(os.path.exists(zipapp_path)) - self.assertTrue(os.path.isfile(zipapp_path)) - - # The zipapp itself is a shell script prepended to the zip file. - with open(zipapp_path, "rb") as f: - content = f.read() - self.assertTrue(content.startswith(b"#!/usr/bin/env bash")) - - output = subprocess.check_output([zipapp_path]).decode("utf-8").strip() + try: + output = ( + subprocess.check_output([zipapp_path], stderr=subprocess.STDOUT) + .decode("utf-8") + .strip() + ) + except subprocess.CalledProcessError as e: + self.fail( + ( + "exec failed: {}\n" + + "exit code: {}\n" + + "=== stdout/stderr start ===\n" + "{}\n" + "=== stdout/stderr end ===" + ).format(zipapp_path, e.returncode, e.output.decode("utf-8")) + ) self.assertIn("Hello from zipapp", output) - self.assertIn("absl", output) + self.assertIn("dep:", output) def assertHasPathMatchingSuffix(self, namelist, suffix, msg=None): if not any(name.endswith(suffix) for name in namelist): - self.fail(msg or f"No path in zipapp matching suffix '{suffix}'") + self.fail( + (msg or f"No path in zipapp matching suffix '{suffix}'") + + "\nAvailable paths:\n" + + "\n".join(namelist) + ) def assertZipEntryIsSymlink(self, zip_file, path, msg=None): try: @@ -39,10 +50,27 @@ def assertZipEntryIsSymlink(self, zip_file, path, msg=None): def _is_bzlmod_enabled(self): return os.environ["BZLMOD_ENABLED"] == "1" + @contextlib.contextmanager + def _open_zipapp(self, path): + zf = None + try: + try: + zf = zipfile.ZipFile(path, "r") + except zipfile.BadZipFile: + # On windows, the main output is the launcher .exe file, and the + # zip file is a sibling file. + path = path.replace(".exe", ".zip") + zf = zipfile.ZipFile(path, "r") + if zf: + yield zf + finally: + if zf: + zf.close() + def test_zipapp_structure(self): zipapp_path = os.environ["TEST_ZIPAPP"] - with zipfile.ZipFile(zipapp_path, "r") as zf: + with self._open_zipapp(zipapp_path) as zf: namelist = zf.namelist() if self._is_bzlmod_enabled(): From 3a8829b504ac4fdb4edaf357ca5260492a08635b Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Fri, 6 Feb 2026 19:10:34 -0800 Subject: [PATCH 611/922] docs: doc the imports attribute as a target-relative path (#3571) The docs incorrectly refer to the path as a repo-root relative path, but this is plainly not the case, as the underlying code does, essentially, `join(repoRoot, ctx.label, importsAttr)` Work towards https://github.com/bazel-contrib/rules_python/issues/3565 --------- Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> --- python/private/attributes.bzl | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/python/private/attributes.bzl b/python/private/attributes.bzl index 4687693cb7..362eee8f2e 100644 --- a/python/private/attributes.bzl +++ b/python/private/attributes.bzl @@ -189,7 +189,14 @@ List of import directories to be added to the PYTHONPATH. Subject to "Make variable" substitution. These import directories will be added for this rule and all rules that depend on it (note: not the rules this rule depends on. Each directory will be added to `PYTHONPATH` by `py_binary` rules -that depend on this rule. The strings are repo-runfiles-root relative, +that depend on this rule. + +The values are target-directory-relative runfiles-root paths. e.g. given target +`//foo/bar:baz`, `sys.path` will be affected as: +* `a/b` adds `$runfilesRoot/$repo/foo/bar/a/b` +* `../sibling` adds `$runfilesRoot/$repo/foo/sibling` +* `../../` adds `$runfilesRoot/$repo` +(where `$repo` is the name of the repository containing the target). Absolute paths (paths that start with `/`) and paths that references a path above the execution root are not allowed and will result in an error. From 4274bb7e9b713c357643ccfbc2964ddc5822c390 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Sat, 7 Feb 2026 16:11:38 -0800 Subject: [PATCH 612/922] docs: mention PyRuntimeInfo in PyExecutableInfo (#3573) This is to make it more apparent how the interpreter's information can be mixed together with the executable info. Related to https://github.com/bazel-contrib/rules_python/issues/3181 --- python/private/py_executable_info.bzl | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/python/private/py_executable_info.bzl b/python/private/py_executable_info.bzl index 81216b8e89..24aa705566 100644 --- a/python/private/py_executable_info.bzl +++ b/python/private/py_executable_info.bzl @@ -18,6 +18,12 @@ The runfiles for the executable's "user" dependencies. These are things in e.g. e.g. the Python runtime itself. It's roughly akin to the files a traditional venv would have installed into it. +:::{seealso} +{obj}`PyRuntimeInfo` for the Python runtime files. The {obj}`py_binary` et al +rules provide it directly so that the runtime the binary original chose +can be accessed. +::: + :::{versionadded} VERSION_NEXT_FEATURE ::: """, From b8c217e717792af4ab6729dfa6170dbb2aeaaefa Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Sat, 7 Feb 2026 20:02:05 -0800 Subject: [PATCH 613/922] feat(zipapp): Add python_zip_file output group for better compatibility (#3574) The original zipapp functionality exposed the plain, non-executable zipapp zip file using the `python_zip_file` output group. This is easy to support, so add it to make it more compatible and easier to switch over. Along the way ... * Add some versionadded markers * Add zipapp_rules to features.bzl Work towards https://github.com/bazel-contrib/rules_python/issues/2586 --- python/features.bzl | 9 +++++++++ python/private/zipapp/py_zipapp_rule.bzl | 24 ++++++++++++++++++++++-- python/zipapp/py_zipapp_binary.bzl | 11 ++++++++++- python/zipapp/py_zipapp_test.bzl | 11 ++++++++++- 4 files changed, 51 insertions(+), 4 deletions(-) diff --git a/python/features.bzl b/python/features.bzl index 291de33d1a..4cc9436677 100644 --- a/python/features.bzl +++ b/python/features.bzl @@ -63,6 +63,14 @@ def _features_typedef(): optional trailing `-rcN`. For unreleased versions, it is an empty string. :::{versionadded} 0.38.0 :::: + + ::::{field} zipapp_rules + :type: bool + + Whether the rules_python version has the `py_zipapp_*` rules + + :::{versionadded} VERSION_NEXT_FEATURE + :::: """ features = struct( @@ -73,4 +81,5 @@ features = struct( py_info_venv_symlinks = True, uses_builtin_rules = False, version = _VERSION_PRIVATE if "$Format" not in _VERSION_PRIVATE else "", + zipapp_rules = True, ) diff --git a/python/private/zipapp/py_zipapp_rule.bzl b/python/private/zipapp/py_zipapp_rule.bzl index a0f0df943f..e97e1a4171 100644 --- a/python/private/zipapp/py_zipapp_rule.bzl +++ b/python/private/zipapp/py_zipapp_rule.bzl @@ -218,6 +218,9 @@ def _py_zipapp_executable_impl(ctx): runfiles = ctx.runfiles(files = default_outputs), executable = executable, ), + OutputGroupInfo( + python_zip_file = depset([zip_file]), + ), ] def _transition_zipapp_impl(settings, attr): @@ -325,10 +328,25 @@ Whether the output should be an executable zip file. _TOOLCHAINS = [EXEC_TOOLS_TOOLCHAIN_TYPE] + ([LAUNCHER_MAKER_TOOLCHAIN_TYPE] if rp_config.bazel_9_or_later else []) +_COMMON_RULE_DOC = """ + +Output groups: + +* `python_zip_file`: (*deprecated*) The plain, non-self-executable zipapp zipfile. + *This output group is deprecated and retained for compatibility with + the previous implicit zipapp functionality. Set `executable=False` + and use the default output of the target instead.* + +:::{versionadded} VERSION_NEXT_FEATURE +::: +""".lstrip() + py_zipapp_binary = rule( doc = """ Packages a `py_binary` as a Python zipapp. -""", + +{} +""".format(_COMMON_RULE_DOC), implementation = _py_zipapp_executable_impl, attrs = _ATTRS, # NOTE: While this is marked executable, it is conditionally executable @@ -343,7 +361,9 @@ py_zipapp_test = rule( Packages a `py_test` as a Python zipapp. This target is also a valid test target to run. -""", + +{} +""".format(_COMMON_RULE_DOC), implementation = _py_zipapp_executable_impl, attrs = _ATTRS, # NOTE: While this is marked as a test, it is conditionally executable diff --git a/python/zipapp/py_zipapp_binary.bzl b/python/zipapp/py_zipapp_binary.bzl index 4c40786200..ba7652c6d4 100644 --- a/python/zipapp/py_zipapp_binary.bzl +++ b/python/zipapp/py_zipapp_binary.bzl @@ -1,4 +1,10 @@ -"""`py_zipapp_binary` macro.""" +"""`py_zipapp_binary` macro. + +:::{seealso} + +{obj}`features.zipapp_rules` to detect if this rule is available. +::: +""" load("//python/private:util.bzl", "add_tag") load("//python/private/zipapp:py_zipapp_rule.bzl", _py_zipapp_binary_rule = "py_zipapp_binary") @@ -6,6 +12,9 @@ load("//python/private/zipapp:py_zipapp_rule.bzl", _py_zipapp_binary_rule = "py_ def py_zipapp_binary(**kwargs): """Builds a Python zipapp from a py_binary/py_test target. + :::{versionadded} VERSION_NEXT_FEATURE + ::: + Args: **kwargs: Args passed onto {rule}`py_zipapp_binary`. """ diff --git a/python/zipapp/py_zipapp_test.bzl b/python/zipapp/py_zipapp_test.bzl index bc113ad4ed..fa197701a9 100644 --- a/python/zipapp/py_zipapp_test.bzl +++ b/python/zipapp/py_zipapp_test.bzl @@ -1,4 +1,10 @@ -"""`py_zipapp_test` macro.""" +"""`py_zipapp_test` macro. + +:::{seealso} + +{obj}`features.zipapp_rules` to detect if this rule is available. +::: +""" load("//python/private:util.bzl", "add_tag") load("//python/private/zipapp:py_zipapp_rule.bzl", _py_zipapp_test = "py_zipapp_test") @@ -6,6 +12,9 @@ load("//python/private/zipapp:py_zipapp_rule.bzl", _py_zipapp_test = "py_zipapp_ def py_zipapp_test(**kwargs): """Builds a Python zipapp from a py_binary/py_test target. + :::{versionadded} VERSION_NEXT_FEATURE + ::: + Args: **kwargs: Args passed onto {rule}`py_zipapp_test`. """ From afe79815a8ad7fcb3b8f06f632e1fb49e2bac9b2 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Sat, 7 Feb 2026 20:03:26 -0800 Subject: [PATCH 614/922] chore(py_executable): print warning if build zip is enabled (#3568) This prints a warning if `--build_python_zip` is enabled. Work towards https://github.com/bazel-contrib/rules_python/issues/3567 https://github.com/bazel-contrib/rules_python/issues/2586 --- CHANGELOG.md | 9 +++++++++ python/private/py_executable.bzl | 15 +++++++++++++++ 2 files changed, 24 insertions(+) diff --git a/CHANGELOG.md b/CHANGELOG.md index d71b6e65b2..15118f53e9 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -58,6 +58,15 @@ END_UNRELEASED_TEMPLATE {#v0-0-0-changed} ### Changed +* **DEPRECATED: implicit zipapp support** + * Implicit zipapp output of `py_binary`/`py_test` has been deprecated and + replaced by separate {obj}`py_zipapp_binary` and {obj}`py_zipapp_test` + rules. See + [#3567](https://github.com/bazel-contrib/rules_python/issues/3567) + for a detailed migration guide. +* (toolchains) stop exposing config settings in python toolchain alias repos. + Please consider depending on the flags defined in + `//python/config_setting/...` and the `@platforms` package instead. * (binaries/tests) The `PYTHONBREAKPOINT` environment variable is automatically inherited * (binaries/tests) The {obj}`stamp` attribute now transitions the Bazel builtin {obj}`--stamp` flag. diff --git a/python/private/py_executable.bzl b/python/private/py_executable.bzl index 9f1113d9f6..45a18abb05 100644 --- a/python/private/py_executable.bzl +++ b/python/private/py_executable.bzl @@ -379,6 +379,21 @@ def _create_executable( # When --build_python_zip is enabled, then the zip file becomes # one of the default outputs. if build_zip_enabled: + # buildifier: disable=print + print( + """ +====================================================================== +WARNING: Target: {} + The `--build_python_zip` flag and implicit zipapp output of `py_binary` + and `py_test` is deprecated and will be removed in a future release. + Switch to `py_zipapp_binary` or `py_zipapp_test`. For migration + instructions and guide, see: + + https://github.com/bazel-contrib/rules_python/issues/3567 +====================================================================== + """.rstrip().format(ctx.label), + ) + extra_default_outputs.append(zip_file) # The logic here is a bit convoluted. Essentially, there are 3 types of From dddd7a2446ce73b25bce1516c2541260c7fc0778 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Sun, 8 Feb 2026 06:14:13 -0800 Subject: [PATCH 615/922] fix: fallback to /usr/bin/env if env is not in PATH (#3577) This fixes an issue where the bootstrap script would fail if env was not in the PATH, which is the case on NixOS, or when Bazel's strict action env is enabled. To fix, the bootstrap checks if env exists, and if not, falls back to /usr/bin/env. Fixes https://github.com/bazel-contrib/rules_python/issues/3575 --- python/private/stage1_bootstrap_template.sh | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/python/private/stage1_bootstrap_template.sh b/python/private/stage1_bootstrap_template.sh index a984344647..36b8b67c37 100644 --- a/python/private/stage1_bootstrap_template.sh +++ b/python/private/stage1_bootstrap_template.sh @@ -285,8 +285,14 @@ fi export RUNFILES_DIR +if command -v env >/dev/null 2>&1; then + ENV_CMD="env" +else + ENV_CMD="/usr/bin/env" +fi + command=( - env + "$ENV_CMD" "${interpreter_env[@]}" "$python_exe" "${interpreter_args[@]}" From 708b07b9552febe35627afa6d20a443e8bfcfc83 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Sun, 8 Feb 2026 08:08:38 -0800 Subject: [PATCH 616/922] fix: make imports attribute target-relative for venv mode (#3572) The `imports` attribute is treated as target-relative in non-venv mode, but venv-mode was passing the value through without the target-relative processing. This went unnoticed because the main use case, pypi packages, are a the repo root, so target and repo relative paths are the same. To fix, use join the input value to the target's path, as done in the non-venv `get_imports()` function. Fixes https://github.com/bazel-contrib/rules_python/issues/3565 --- python/private/py_library.bzl | 14 +++++++++++++- python/private/venv_runfiles.bzl | 3 ++- tests/base_rules/py_executable_base_tests.bzl | 2 +- tests/modules/other/nspkg_delta/BUILD.bazel | 2 +- tests/modules/other/nspkg_gamma/BUILD.bazel | 2 +- tests/modules/other/nspkg_single/BUILD.bazel | 2 +- tests/modules/other/simple_v1/BUILD.bazel | 2 +- tests/modules/other/simple_v2/BUILD.bazel | 2 +- tests/modules/other/with_external_data/BUILD.bazel | 2 +- tests/py_zipapp/BUILD.bazel | 12 +----------- .../ext_with_libs/BUILD.bazel | 2 +- .../nested_with_pth/BUILD.bazel | 2 +- .../nspkg_alpha/BUILD.bazel | 2 +- .../venv_site_packages_libs/nspkg_beta/BUILD.bazel | 2 +- .../pkgutil_top/BUILD.bazel | 2 +- .../pkgutil_top_sub/BUILD.bazel | 2 +- 16 files changed, 29 insertions(+), 26 deletions(-) diff --git a/python/private/py_library.bzl b/python/private/py_library.bzl index de73a53720..a7419a6eaf 100644 --- a/python/private/py_library.bzl +++ b/python/private/py_library.bzl @@ -246,12 +246,24 @@ def _get_imports_and_venv_symlinks(ctx): fail("When venvs_site_packages is enabled, exactly one `imports` " + "value must be specified, got {}".format(imports)) + site_packages_root = paths.normalize(paths.join( + ctx.label.package, + imports[0], + )) + + # Prevent escaping out of the repo root. + if site_packages_root.startswith("../") or site_packages_root == "..": + fail(("Invalid `imports` value '{}': resolves to '{}' which is " + + "above the repo root").format( + imports[0], + site_packages_root, + )) venv_symlinks = get_venv_symlinks( ctx, ctx.files.srcs + ctx.files.data + ctx.files.pyi_srcs, package, version_str, - site_packages_root = imports[0], + site_packages_root = site_packages_root, namespace_package_files = ctx.files.namespace_package_files, ) else: diff --git a/python/private/venv_runfiles.bzl b/python/private/venv_runfiles.bzl index d522470942..a492181c88 100644 --- a/python/private/venv_runfiles.bzl +++ b/python/private/venv_runfiles.bzl @@ -250,7 +250,8 @@ def _get_file_venv_path(ctx, f, site_packages_root): Args: ctx: The current ctx. f: The file to compute the venv_path for. - site_packages_root: The site packages root path. + site_packages_root: The site packages root path; repo-relative + path. Returns: A tuple `(venv_path, rf_root_path)` if the file is under diff --git a/tests/base_rules/py_executable_base_tests.bzl b/tests/base_rules/py_executable_base_tests.bzl index a73a4cb48e..f0e2ae9faf 100644 --- a/tests/base_rules/py_executable_base_tests.bzl +++ b/tests/base_rules/py_executable_base_tests.bzl @@ -194,7 +194,7 @@ def _test_debugger(name, config): rt_util.helper_target( py_library, name = name + "_debugger_venv", - imports = [native.package_name() + "/site-packages"], + imports = ["site-packages"], experimental_venvs_site_packages = "@rules_python//python/config_settings:venvs_site_packages", srcs = [rt_util.empty_file("site-packages/" + name + "_debugger_venv.py")], ) diff --git a/tests/modules/other/nspkg_delta/BUILD.bazel b/tests/modules/other/nspkg_delta/BUILD.bazel index 457033aacf..ca142d2c10 100644 --- a/tests/modules/other/nspkg_delta/BUILD.bazel +++ b/tests/modules/other/nspkg_delta/BUILD.bazel @@ -6,5 +6,5 @@ py_library( name = "nspkg_delta", srcs = glob(["site-packages/**/*.py"]), experimental_venvs_site_packages = "@rules_python//python/config_settings:venvs_site_packages", - imports = [package_name() + "/site-packages"], + imports = ["site-packages"], ) diff --git a/tests/modules/other/nspkg_gamma/BUILD.bazel b/tests/modules/other/nspkg_gamma/BUILD.bazel index 89038e80d2..0fe099eb0a 100644 --- a/tests/modules/other/nspkg_gamma/BUILD.bazel +++ b/tests/modules/other/nspkg_gamma/BUILD.bazel @@ -6,5 +6,5 @@ py_library( name = "nspkg_gamma", srcs = glob(["site-packages/**/*.py"]), experimental_venvs_site_packages = "@rules_python//python/config_settings:venvs_site_packages", - imports = [package_name() + "/site-packages"], + imports = ["site-packages"], ) diff --git a/tests/modules/other/nspkg_single/BUILD.bazel b/tests/modules/other/nspkg_single/BUILD.bazel index 08cb4f373e..07e269b878 100644 --- a/tests/modules/other/nspkg_single/BUILD.bazel +++ b/tests/modules/other/nspkg_single/BUILD.bazel @@ -6,5 +6,5 @@ py_library( name = "nspkg_single", srcs = glob(["site-packages/**/*.py"]), experimental_venvs_site_packages = "@rules_python//python/config_settings:venvs_site_packages", - imports = [package_name() + "/site-packages"], + imports = ["site-packages"], ) diff --git a/tests/modules/other/simple_v1/BUILD.bazel b/tests/modules/other/simple_v1/BUILD.bazel index da5db8164a..0fa2f14a88 100644 --- a/tests/modules/other/simple_v1/BUILD.bazel +++ b/tests/modules/other/simple_v1/BUILD.bazel @@ -10,5 +10,5 @@ py_library( exclude = ["site-packages/**/*.py"], ), experimental_venvs_site_packages = "@rules_python//python/config_settings:venvs_site_packages", - imports = [package_name() + "/site-packages"], + imports = ["site-packages"], ) diff --git a/tests/modules/other/simple_v2/BUILD.bazel b/tests/modules/other/simple_v2/BUILD.bazel index 45f83a5a88..5a7e066aec 100644 --- a/tests/modules/other/simple_v2/BUILD.bazel +++ b/tests/modules/other/simple_v2/BUILD.bazel @@ -10,6 +10,6 @@ py_library( exclude = ["site-packages/**/*.py"], ), experimental_venvs_site_packages = "@rules_python//python/config_settings:venvs_site_packages", - imports = [package_name() + "/site-packages"], + imports = ["site-packages"], pyi_srcs = glob(["**/*.pyi"]), ) diff --git a/tests/modules/other/with_external_data/BUILD.bazel b/tests/modules/other/with_external_data/BUILD.bazel index fc047aadab..338f77947d 100644 --- a/tests/modules/other/with_external_data/BUILD.bazel +++ b/tests/modules/other/with_external_data/BUILD.bazel @@ -19,5 +19,5 @@ py_library( srcs = ["site-packages/with_external_data.py"], data = [":external_data"], experimental_venvs_site_packages = "@rules_python//python/config_settings:venvs_site_packages", - imports = [package_name() + "/site-packages"], + imports = ["site-packages"], ) diff --git a/tests/py_zipapp/BUILD.bazel b/tests/py_zipapp/BUILD.bazel index da42567656..c236998cc0 100644 --- a/tests/py_zipapp/BUILD.bazel +++ b/tests/py_zipapp/BUILD.bazel @@ -81,15 +81,5 @@ py_library( name = "some_dep", srcs = ["some_dep.py"], experimental_venvs_site_packages = "//python/config_settings:venvs_site_packages", - imports = select({ - ":is_venvs_site_packages_enabled": ["tests/py_zipapp"], - "//conditions:default": ["."], - }), -) - -config_setting( - name = "is_venvs_site_packages_enabled", - flag_values = { - "//python/config_settings:venvs_site_packages": "yes", - }, + imports = ["."], ) diff --git a/tests/venv_site_packages_libs/ext_with_libs/BUILD.bazel b/tests/venv_site_packages_libs/ext_with_libs/BUILD.bazel index 8f161ee17c..a3a277dbb4 100644 --- a/tests/venv_site_packages_libs/ext_with_libs/BUILD.bazel +++ b/tests/venv_site_packages_libs/ext_with_libs/BUILD.bazel @@ -89,6 +89,6 @@ py_library( ":relocate_increment", ], experimental_venvs_site_packages = "//python/config_settings:venvs_site_packages", - imports = [package_name() + "/site-packages"], + imports = ["site-packages"], tags = ["manual"], ) diff --git a/tests/venv_site_packages_libs/nested_with_pth/BUILD.bazel b/tests/venv_site_packages_libs/nested_with_pth/BUILD.bazel index 68c16cfde9..f0339270b4 100644 --- a/tests/venv_site_packages_libs/nested_with_pth/BUILD.bazel +++ b/tests/venv_site_packages_libs/nested_with_pth/BUILD.bazel @@ -7,5 +7,5 @@ py_library( srcs = glob(["site-packages/**/*.py"]), data = glob(["site-packages/*.pth"]), experimental_venvs_site_packages = "//python/config_settings:venvs_site_packages", - imports = [package_name() + "/site-packages"], + imports = ["site-packages"], ) diff --git a/tests/venv_site_packages_libs/nspkg_alpha/BUILD.bazel b/tests/venv_site_packages_libs/nspkg_alpha/BUILD.bazel index aec415f7a0..9c0aa192a9 100644 --- a/tests/venv_site_packages_libs/nspkg_alpha/BUILD.bazel +++ b/tests/venv_site_packages_libs/nspkg_alpha/BUILD.bazel @@ -6,5 +6,5 @@ py_library( name = "nspkg_alpha", srcs = glob(["site-packages/**/*.py"]), experimental_venvs_site_packages = "//python/config_settings:venvs_site_packages", - imports = [package_name() + "/site-packages"], + imports = ["site-packages"], ) diff --git a/tests/venv_site_packages_libs/nspkg_beta/BUILD.bazel b/tests/venv_site_packages_libs/nspkg_beta/BUILD.bazel index 5d402183bd..d67ebf57d6 100644 --- a/tests/venv_site_packages_libs/nspkg_beta/BUILD.bazel +++ b/tests/venv_site_packages_libs/nspkg_beta/BUILD.bazel @@ -6,5 +6,5 @@ py_library( name = "nspkg_beta", srcs = glob(["site-packages/**/*.py"]), experimental_venvs_site_packages = "//python/config_settings:venvs_site_packages", - imports = [package_name() + "/site-packages"], + imports = ["site-packages"], ) diff --git a/tests/venv_site_packages_libs/pkgutil_top/BUILD.bazel b/tests/venv_site_packages_libs/pkgutil_top/BUILD.bazel index c805b1ad53..7a0d961ad8 100644 --- a/tests/venv_site_packages_libs/pkgutil_top/BUILD.bazel +++ b/tests/venv_site_packages_libs/pkgutil_top/BUILD.bazel @@ -6,5 +6,5 @@ py_library( name = "pkgutil_top", srcs = glob(["site-packages/**/*.py"]), experimental_venvs_site_packages = "//python/config_settings:venvs_site_packages", - imports = [package_name() + "/site-packages"], + imports = ["site-packages"], ) diff --git a/tests/venv_site_packages_libs/pkgutil_top_sub/BUILD.bazel b/tests/venv_site_packages_libs/pkgutil_top_sub/BUILD.bazel index 9d771628a0..52980fa790 100644 --- a/tests/venv_site_packages_libs/pkgutil_top_sub/BUILD.bazel +++ b/tests/venv_site_packages_libs/pkgutil_top_sub/BUILD.bazel @@ -6,5 +6,5 @@ py_library( name = "pkgutil_top_sub", srcs = glob(["site-packages/**/*.py"]), experimental_venvs_site_packages = "//python/config_settings:venvs_site_packages", - imports = [package_name() + "/site-packages"], + imports = ["site-packages"], ) From 0f4ff60a1727300e1bace6233e566076736e9f15 Mon Sep 17 00:00:00 2001 From: Martin Altenburg <2737351+martin4861@users.noreply.github.com> Date: Mon, 9 Feb 2026 07:55:12 +0100 Subject: [PATCH 617/922] feat: handle url req in wheelmaker (#3569) Adapt `wheelmaker` so that it now can also handle PEP 508 URL requirements using the `req.url` attribute. --------- Co-authored-by: Ignas Anikevicius <240938+aignas@users.noreply.github.com> --- CHANGELOG.md | 2 ++ tests/tools/wheelmaker_test.py | 36 +++++++++++++++++++++++++++ tools/wheelmaker.py | 45 +++++++++++++++++++++------------- 3 files changed, 66 insertions(+), 17 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 15118f53e9..96e7dcd5b3 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -102,6 +102,8 @@ END_UNRELEASED_TEMPLATE {obj}`PyExecutableInfo.interpreter_args`, {obj}`PyExecutableInfo.stage2_bootstrap`, and {obj}`PyExecutableInfo.venv_python_exe`. +* (tools/wheelmaker.py) Added support for URL requirements according to PEP 508 + in Requires-Dist metadata. ([#3569](https://github.com/bazel-contrib/rules_python/pull/3569)) {#v1-8-3} ## [1.8.3] - 2026-01-27 diff --git a/tests/tools/wheelmaker_test.py b/tests/tools/wheelmaker_test.py index 288dde720a..85094af9b8 100644 --- a/tests/tools/wheelmaker_test.py +++ b/tests/tools/wheelmaker_test.py @@ -71,5 +71,41 @@ def test_arcname_from(self) -> None: self.assertEqual(got, want) +class GetNewRequirementLineTest(unittest.TestCase): + def test_requirement(self): + result = wheelmaker.get_new_requirement_line("requests>=2.0", "") + self.assertEqual(result, "Requires-Dist: requests>=2.0") + + def test_requirement_and_extra(self): + result = wheelmaker.get_new_requirement_line("requests>=2.0", "extra=='dev'") + self.assertEqual(result, "Requires-Dist: requests>=2.0; extra=='dev'") + + def test_requirement_with_url(self): + result = wheelmaker.get_new_requirement_line( + "requests @ git+https://github.com/psf/requests.git@3aa6386c3", "" + ) + self.assertEqual( + result, + "Requires-Dist: requests @ git+https://github.com/psf/requests.git@3aa6386c3", + ) + + def test_requirement_with_marker(self): + result = wheelmaker.get_new_requirement_line( + "requests>=2.0; python_version>='3.6'", "" + ) + self.assertEqual( + result, 'Requires-Dist: requests>=2.0; python_version >= "3.6"' + ) + + def test_requirement_with_marker_and_extra(self): + result = wheelmaker.get_new_requirement_line( + "requests>=2.0; python_version>='3.6'", "extra=='dev'" + ) + self.assertEqual( + result, + "Requires-Dist: requests>=2.0; (python_version >= \"3.6\") and extra=='dev'", + ) + + if __name__ == "__main__": unittest.main() diff --git a/tools/wheelmaker.py b/tools/wheelmaker.py index 4390df3445..7124ae7c9d 100644 --- a/tools/wheelmaker.py +++ b/tools/wheelmaker.py @@ -330,9 +330,7 @@ def add_wheelfile(self): Wheel-Version: 1.0 Generator: bazel-wheelmaker 1.0 Root-Is-Purelib: {} -""".format( - "true" if self._platform == "any" else "false" - ) +""".format("true" if self._platform == "any" else "false") for tag in self.disttags(): wheel_contents += "Tag: %s\n" % tag self._whlfile.add_string(self.distinfo_path("WHEEL"), wheel_contents) @@ -365,6 +363,32 @@ def get_files_to_package(input_files): return files +def get_new_requirement_line(reqs_text: str, extra: str) -> str: + """Formats a requirement text into a Requires-Dist metadata line.""" + from packaging.requirements import Requirement + + req = Requirement(reqs_text.strip()) + req_extra_deps = f"[{','.join(req.extras)}]" if req.extras else "" + + # Handle URL requirements (PEP 508) + if req.url: + req_spec = f" @ {req.url}" + else: + req_spec = str(req.specifier) + + base = f"Requires-Dist: {req.name}{req_extra_deps}{req_spec}" + + if req.marker: + if extra: + return f"{base}; ({req.marker}) and {extra}" + else: + return f"{base}; {req.marker}" + elif extra: + return f"{base}; {extra}" + else: + return base + + def resolve_argument_stamp( argument: str, volatile_status_stamp: Path, stable_status_stamp: Path ) -> str: @@ -430,7 +454,7 @@ def parse_args() -> argparse.Namespace: output_group.add_argument( "--name_file", type=Path, - help="A file where the canonical name of the " "wheel will be written", + help="A file where the canonical name of the wheel will be written", ) output_group.add_argument( @@ -578,19 +602,6 @@ def main() -> None: # Search for any `Requires-Dist` entries that refer to other files and # expand them. - def get_new_requirement_line(reqs_text, extra): - req = Requirement(reqs_text.strip()) - req_extra_deps = f"[{','.join(req.extras)}]" if req.extras else "" - if req.marker: - if extra: - return f"Requires-Dist: {req.name}{req_extra_deps}{req.specifier}; ({req.marker}) and {extra}" - else: - return f"Requires-Dist: {req.name}{req_extra_deps}{req.specifier}; {req.marker}" - else: - return f"Requires-Dist: {req.name}{req_extra_deps}{req.specifier}; {extra}".strip( - " ;" - ) - for meta_line in metadata.splitlines(): if not meta_line.startswith("Requires-Dist: "): continue From 9dcbabb3fafe847291850ff9a249e46f6fca5ef4 Mon Sep 17 00:00:00 2001 From: James Sharpe Date: Wed, 11 Feb 2026 00:32:38 +0000 Subject: [PATCH 618/922] fix(pip): preserve PEP 508 URL-based requirements when extract_url_srcs=False (#3582) pip_parse (via pip_repository) passes extract_url_srcs=False to parse_requirements. The _package_srcs() function silently dropped PEP 508 URL-based requirements (pkg @ https://...) in this mode because _add_dists() returns can_fallback=False for URL requirements, causing them to fall through to the `continue` statement. Fix the elif condition to also accept the case where extract_url_srcs is False but a valid URL dist exists, falling back to pip to handle the URL requirement directly. Co-authored-by: Claude Opus 4.6 --- CHANGELOG.md | 3 ++ python/private/pypi/parse_requirements.bzl | 2 +- .../parse_requirements_tests.bzl | 41 +++++++++++++++++++ 3 files changed, 45 insertions(+), 1 deletion(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 96e7dcd5b3..12edf68c4c 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -79,6 +79,9 @@ END_UNRELEASED_TEMPLATE ([#2762](https://github.com/bazel-contrib/rules_python/issues/2762)) * (gazelle) Ancestor `conftest.py` files are added in addition to sibling `conftest.py`. ([#3497](https://github.com/bazel-contrib/rules_python/issues/3497)) +* (pypi) `pip_parse` no longer silently drops PEP 508 URL-based requirements + (`pkg @ https://...`) when `extract_url_srcs=False` (the default for + `pip_repository`). {#v0-0-0-added} ### Added diff --git a/python/private/pypi/parse_requirements.bzl b/python/private/pypi/parse_requirements.bzl index 5c05c753fd..760bf5a9e8 100644 --- a/python/private/pypi/parse_requirements.bzl +++ b/python/private/pypi/parse_requirements.bzl @@ -260,7 +260,7 @@ def _package_srcs( if extract_url_srcs and dist: req_line = r.srcs.requirement - elif can_fallback: + elif can_fallback or (not extract_url_srcs and dist): dist = struct( url = "", filename = "", diff --git a/tests/pypi/parse_requirements/parse_requirements_tests.bzl b/tests/pypi/parse_requirements/parse_requirements_tests.bzl index 63755d2edd..a2efe91d99 100644 --- a/tests/pypi/parse_requirements/parse_requirements_tests.bzl +++ b/tests/pypi/parse_requirements/parse_requirements_tests.bzl @@ -192,6 +192,47 @@ def _test_direct_urls_integration(env): _tests.append(_test_direct_urls_integration) +def _test_direct_urls_no_extract(env): + """Check that URL requirements are not dropped when extract_url_srcs=False.""" + got = parse_requirements( + requirements_by_platform = { + "requirements_direct": ["linux_x86_64"], + "requirements_direct_sdist": ["osx_x86_64"], + }, + extract_url_srcs = False, + ) + env.expect.that_collection(got).contains_exactly([ + struct( + name = "foo", + is_exposed = True, + is_multiple_versions = True, + srcs = [ + struct( + distribution = "foo", + extra_pip_args = [], + filename = "", + requirement_line = "foo @ https://github.com/org/foo/downloads/foo-1.1.tar.gz", + sha256 = "", + target_platforms = ["osx_x86_64"], + url = "", + yanked = False, + ), + struct( + distribution = "foo", + extra_pip_args = [], + filename = "", + requirement_line = "foo[extra] @ https://some-url/package.whl", + sha256 = "", + target_platforms = ["linux_x86_64"], + url = "", + yanked = False, + ), + ], + ), + ]) + +_tests.append(_test_direct_urls_no_extract) + def _test_extra_pip_args(env): got = parse_requirements( requirements_by_platform = { From a32c74465d9cc328485bc440a933c0447572fa6e Mon Sep 17 00:00:00 2001 From: Ignas Anikevicius <240938+aignas@users.noreply.github.com> Date: Wed, 11 Feb 2026 12:39:15 +0900 Subject: [PATCH 619/922] fix(pipstar): handle a corner case for compatible version evaluation (#3583) It seems that there was one corner case that was left unhandled. In theory we should also handle `3 ~= 3.0.0`, but the fix for that may be a little more involved and I want to leave it for later. However, that case is unlikely to occur in reality because `LHS` for our cases will most likely have 2 components. Whilst at it prepare for the patch release (#3584). Fixes #3580 --- CHANGELOG.md | 11 +++++++++ python/private/version.bzl | 36 +++++++++++++++++++++------- tests/pypi/pep508/deps_tests.bzl | 20 ++++++++++++++++ tests/pypi/pep508/evaluate_tests.bzl | 3 ++- 4 files changed, 60 insertions(+), 10 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 12edf68c4c..7280805141 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -108,6 +108,17 @@ END_UNRELEASED_TEMPLATE * (tools/wheelmaker.py) Added support for URL requirements according to PEP 508 in Requires-Dist metadata. ([#3569](https://github.com/bazel-contrib/rules_python/pull/3569)) +{#v1-8-4} +## [1.8.4] - 2026-02-10 + +### Fixed +* (pipstar): A corner case of evaluation of version specifiers (`"1.2" ~= "1.2.0"`) + has been fixed improving compatibility with the PEP440 standard. + Fixes [#3580](https://github.com/bazel-contrib/rules_python/issues/3580). +* (pipstar): We now add read permissions after extracting wheels for the cases + where the `whl` file is missing them. + Fixes [#3554](https://github.com/bazel-contrib/rules_python/issues/3554). + {#v1-8-3} ## [1.8.3] - 2026-01-27 diff --git a/python/private/version.bzl b/python/private/version.bzl index c41524a9e3..b13327841b 100644 --- a/python/private/version.bzl +++ b/python/private/version.bzl @@ -719,12 +719,9 @@ def _version_eeq(left, right): def _version_eq(left, right): """== operator""" - if left.is_prefix and right.is_prefix: - fail("Invalid comparison: both versions cannot be prefix matching") - if left.is_prefix: - return right.string.startswith("{}.".format(left.string)) - if right.is_prefix: - return left.string.startswith("{}.".format(right.string)) + is_prefix = _is_prefix(left, right) + if is_prefix != None: + return is_prefix if left.epoch != right.epoch: return False @@ -743,6 +740,27 @@ def _version_eq(left, right): # local is ignored for == checks ) +def _is_prefix(left, right): + if left.is_prefix and right.is_prefix: + fail("Invalid comparison: both versions cannot be prefix matching") + if left.is_prefix: + return _left_is_prefix(left, right) + if right.is_prefix: + return _left_is_prefix(right, left) + + return None + +def _left_is_prefix(left, right): + if right.string.startswith("{}.".format(left.string)): + return True + + # There is a chance that we are comparing 1.3 ~= 1.3.0 + # + # In that case the logic would be to normalize 1.3 to 1.3.0 and then + # the above would work, but we end up comparing 1.3 with 1.3. above + # and it fails. + return _version_key(left) == _version_key(right) + def _version_compatible(left, right): """~= operator""" if left.is_prefix or right.is_prefix: @@ -754,11 +772,11 @@ def _version_compatible(left, right): right_star = ".".join([str(d) for d in right.release[:-1]]) if right.epoch: - right_star = "{}!{}.".format(right.epoch, right_star) + right_star = "{}!{}.*".format(right.epoch, right_star) else: - right_star = "{}.".format(right_star) + right_star = "{}.*".format(right_star) - return _version_ge(left, right) and left.string.startswith(right_star) + return _version_ge(left, right) and _is_prefix(left, parse(right_star, strict = False)) def _version_ne(left, right): """!= operator""" diff --git a/tests/pypi/pep508/deps_tests.bzl b/tests/pypi/pep508/deps_tests.bzl index f566845b70..1404ad6fc1 100644 --- a/tests/pypi/pep508/deps_tests.bzl +++ b/tests/pypi/pep508/deps_tests.bzl @@ -198,6 +198,26 @@ def test_extra_with_conditional_and_unconditional_markers(env): _tests.append(test_extra_with_conditional_and_unconditional_markers) +def test_span_all_python_versions(env): + requires_dist = [ + "bar>=0.4.0; python_version >= \"3.13.0\"", + "bar>=0.3.0; python_version ~= \"3.12.0\"", + "bar>=0.2.0; python_version ~= \"3.11.0\"", + "bar>=0.1.0; python_version < \"3.11\"", + ] + + got = deps( + "foo", + requires_dist = requires_dist, + ) + + env.expect.that_collection(got.deps).contains_exactly([]) + env.expect.that_dict(got.deps_select).contains_exactly({ + "bar": "(python_version < \"3.11\") or (python_version >= \"3.13.0\") or (python_version ~= \"3.11.0\") or (python_version ~= \"3.12.0\")", + }) + +_tests.append(test_span_all_python_versions) + def deps_test_suite(name): # buildifier: disable=function-docstring test_suite( name = name, diff --git a/tests/pypi/pep508/evaluate_tests.bzl b/tests/pypi/pep508/evaluate_tests.bzl index 7843f88e89..7c7b5d8173 100644 --- a/tests/pypi/pep508/evaluate_tests.bzl +++ b/tests/pypi/pep508/evaluate_tests.bzl @@ -312,9 +312,10 @@ _MISC_EXPRESSIONS = [ # https://packaging.python.org/en/latest/specifications/version-specifiers/#compatible-release _expr_case('python_version ~= "2.2"', True, {"python_version": "2.3"}), _expr_case('python_version ~= "2.2"', False, {"python_version": "2.1"}), + _expr_case('python_version ~= "2.2.0"', True, {"python_version": "2.2"}), _expr_case('python_version ~= "2.2.post3"', False, {"python_version": "2.2"}), - _expr_case('python_version ~= "2.2.post3"', True, {"python_version": "2.3"}), _expr_case('python_version ~= "2.2.post3"', False, {"python_version": "3.0"}), + _expr_case('python_version ~= "2.2.post3"', True, {"python_version": "2.3"}), _expr_case('python_version ~= "1!2.2"', False, {"python_version": "2.7"}), _expr_case('python_version ~= "0!2.2"', True, {"python_version": "2.7"}), _expr_case('python_version ~= "1!2.2"', True, {"python_version": "1!2.7"}), From f78add7f273bc2fe54ce9187da44abd68588961a Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?R=C3=A9gis=20Desgroppes?= Date: Wed, 11 Feb 2026 15:53:28 +0100 Subject: [PATCH 620/922] fix(runfiles): assume main repository on Windows (#3578) With any Python version on Windows, `RUNFILES_*` environment variables still point to an intermediate stage path instead of the module path, causing `CurrentRepository()` to raise a `ValueError` since `rules_python` v1.8.0: ``` ValueError: C:\Users\bot\AppData\Local\Temp\Bazel.runfiles_gh34ij5_\runfiles\+_repo_rules+cpython\my_script.py does not lie under the runfiles root C:/cache/ab1cdef2/execroot/_main/bazel-out/x64_windows-fastbuild/bin/external/+_repo_rules+cpython/install.exe.runfiles ``` This was not the case with `rules_python` v1.7.0. The issue stems from #3086 which, by eliminating a wrong assumption, also brought a stricter behavior. Since #3086 came up with a corresponding workaround in `//tests/runtime_env_toolchain:toolchain_runs_test`, the proposed fix simply consists in moving it to `CurrentRepository()`, thus adding another case to the workaround introduced by #1634 for Python < 3.11. It therefore leads to assuming the main module path on Windows as well. Removing the workaround from `CurrentRepository()` would make the test fail as follows: ``` ==================== Test output for //tests/runtime_env_toolchain:toolchain_runs_test: E ====================================================================== ERROR: test_ran (__main__.RunTest.test_ran) ---------------------------------------------------------------------- Traceback (most recent call last): [...] ValueError: C:\Users\user\AppData\Local\Temp\Bazel.runfiles_1f08smy6\runfiles\_main\tests\runtime_env_toolchain\toolchain_runs_test.py does not lie under the runfiles root c:\users\user\_bazel_user\cxxeswjo\execroot\_main\bazel-out\x64_windows-fastbuild-st-c530e4918e48\bin\tests\runtime_env_toolchain\toolchain_runs_test.exe.runfiles ``` Fixes #3579. --- CHANGELOG.md | 2 ++ python/runfiles/runfiles.py | 6 +++++- .../runtime_env_toolchain/toolchain_runs_test.py | 15 +++------------ 3 files changed, 10 insertions(+), 13 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 7280805141..61951a342d 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -75,6 +75,8 @@ END_UNRELEASED_TEMPLATE {#v0-0-0-fixed} ### Fixed +* (runfiles) Fixed `CurrentRepository()` raising `ValueError` on Windows. + ([#3579](https://github.com/bazel-contrib/rules_python/issues/3579)) * (tests) No more coverage warnings are being printed if there are no sources. ([#2762](https://github.com/bazel-contrib/rules_python/issues/2762)) * (gazelle) Ancestor `conftest.py` files are added in addition to sibling `conftest.py`. diff --git a/python/runfiles/runfiles.py b/python/runfiles/runfiles.py index bfa9d0d053..f98646b1c2 100644 --- a/python/runfiles/runfiles.py +++ b/python/runfiles/runfiles.py @@ -389,11 +389,15 @@ def CurrentRepository(self, frame: int = 1) -> str: # located in the main repository. # With Python 3.11 and higher, the Python launcher sets # PYTHONSAFEPATH, which prevents this behavior. + # On Windows, the current toolchain being used has a buggy zip file + # bootstrap, which leaves RUNFILES_DIR pointing at the first stage + # path and not the module path. In this case too, assume that the + # module is located in the main repository. # TODO: This doesn't cover the case of a script being run from an # external repository, which could be heuristically detected # by parsing the script's path. if ( - sys.version_info.minor <= 10 + (sys.version_info.minor <= 10 or sys.platform == "win32") and sys.path[0] != self._python_runfiles_root ): return "" diff --git a/tests/runtime_env_toolchain/toolchain_runs_test.py b/tests/runtime_env_toolchain/toolchain_runs_test.py index 6f0948feff..13b5775ff0 100644 --- a/tests/runtime_env_toolchain/toolchain_runs_test.py +++ b/tests/runtime_env_toolchain/toolchain_runs_test.py @@ -10,18 +10,9 @@ class RunTest(unittest.TestCase): def test_ran(self): rf = runfiles.Create() - try: - settings_path = rf.Rlocation( - "rules_python/tests/support/current_build_settings.json" - ) - except ValueError as e: - # The current toolchain being used has a buggy zip file bootstrap, which - # leaves RUNFILES_DIR pointing at the first stage path and not the module - # path. - if platform.system() != "Windows" or "does not lie under the runfiles root" not in str(e): - raise e - settings_path = "./tests/support/current_build_settings.json" - + settings_path = rf.Rlocation( + "rules_python/tests/support/current_build_settings.json" + ) settings = json.loads(pathlib.Path(settings_path).read_text()) if platform.system() == "Windows": From 9fe42b1f0badfc258b159dbc0a05a8392c0234e5 Mon Sep 17 00:00:00 2001 From: Kevin Park Date: Thu, 12 Feb 2026 07:36:55 -0500 Subject: [PATCH 621/922] fix(pypi): normalize extras in requirement strings per PEP 685 (#3588) ## Summary Extras parsed from requirement strings (e.g., from `requirements.txt`) were not being normalized, causing mismatches when evaluating PEP 508 marker expressions. For example, `sqlalchemy[postgresql-psycopg2binary]` would fail to resolve `psycopg2-binary` as a transitive dependency because the wheel METADATA marker expression `extra == "postgresql_psycopg2binary"` uses the underscore-normalized form (per PEP 685), while the extras set retained the original hyphenated form from the requirement string. ## Before ``` # requirements.txt sqlalchemy[postgresql-psycopg2binary]==2.0.36 # Parsed extras: ["postgresql-psycopg2binary"] # Marker evaluation: "postgresql-psycopg2binary" != "postgresql_psycopg2binary" -> MISS # Result: psycopg2-binary NOT included as a dependency ``` ## After ``` # requirements.txt sqlalchemy[postgresql-psycopg2binary]==2.0.36 # Parsed extras: ["postgresql_psycopg2binary"] (normalized) # Marker evaluation: "postgresql_psycopg2binary" == "postgresql_psycopg2binary" -> MATCH # Result: psycopg2-binary correctly included as a dependency ``` ## Changes - **`python/private/pypi/pep508_requirement.bzl`**: Apply `normalize_name()` to each extra during requirement parsing, consistent with how the package name is already normalized. - **`tests/pypi/pep508/requirement_tests.bzl`**: Updated existing test expectation for case normalization and added test case for hyphenated extras (`sqlalchemy[asyncio,postgresql-psycopg2binary,postgresql-asyncpg]`). - **`tests/pypi/pep508/deps_tests.bzl`**: Added `test_extras_with_hyphens_are_normalized` integration test confirming that dependencies gated behind hyphenated extras are correctly resolved. - **`CHANGELOG.md`**: Added entry under Unreleased > Fixed. Fixes #3587 --------- Co-authored-by: Ignas Anikevicius <240938+aignas@users.noreply.github.com> --- CHANGELOG.md | 4 +++ python/private/pypi/BUILD.bazel | 2 ++ python/private/pypi/pep508_deps.bzl | 7 +++-- python/private/pypi/pep508_env.bzl | 7 +++-- python/private/pypi/pep508_evaluate.bzl | 12 ++++---- python/private/pypi/pep508_requirement.bzl | 2 +- tests/pypi/pep508/deps_tests.bzl | 32 ++++++++++++++++++++++ tests/pypi/pep508/requirement_tests.bzl | 7 +++-- 8 files changed, 58 insertions(+), 15 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 61951a342d..418d71b4a7 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -84,6 +84,10 @@ END_UNRELEASED_TEMPLATE * (pypi) `pip_parse` no longer silently drops PEP 508 URL-based requirements (`pkg @ https://...`) when `extract_url_srcs=False` (the default for `pip_repository`). +* (pypi) Extras in requirement strings are now normalized per PEP 685, + fixing missing transitive dependencies when extras contain hyphens + (e.g., `sqlalchemy[postgresql-psycopg2binary]`). + ([#3587](https://github.com/bazel-contrib/rules_python/issues/3587)) {#v0-0-0-added} ### Added diff --git a/python/private/pypi/BUILD.bazel b/python/private/pypi/BUILD.bazel index 4fea3684de..6bfd64652e 100644 --- a/python/private/pypi/BUILD.bazel +++ b/python/private/pypi/BUILD.bazel @@ -262,6 +262,7 @@ bzl_library( name = "pep508_deps_bzl", srcs = ["pep508_deps.bzl"], deps = [ + ":pep508_env_bzl", ":pep508_evaluate_bzl", ":pep508_requirement_bzl", "//python/private:normalize_name_bzl", @@ -272,6 +273,7 @@ bzl_library( name = "pep508_env_bzl", srcs = ["pep508_env.bzl"], deps = [ + "//python/private:normalize_name_bzl", "//python/private:version_bzl", ], ) diff --git a/python/private/pypi/pep508_deps.bzl b/python/private/pypi/pep508_deps.bzl index ad6589cfac..c004334d96 100644 --- a/python/private/pypi/pep508_deps.bzl +++ b/python/private/pypi/pep508_deps.bzl @@ -16,6 +16,7 @@ """ load("//python/private:normalize_name.bzl", "normalize_name") +load(":pep508_env.bzl", "create_env") load(":pep508_evaluate.bzl", "evaluate") load(":pep508_requirement.bzl", "requirement") @@ -155,8 +156,9 @@ def _resolve_extras(self_name, reqs, extras): return sorted(extras) def _evaluate_any(req, extras): + env = create_env() for extra in extras: - if evaluate(req.marker, env = {"extra": extra}): + if evaluate(req.marker, env = env | {"extra": extra}): return True return False @@ -167,11 +169,12 @@ def _add_reqs(deps, deps_select, dep, reqs, *, extras): _add(deps, deps_select, dep) return + env = create_env() markers = {} found_unconditional = False for req in reqs: for x in extras: - m = evaluate(req.marker, env = {"extra": x}, strict = False) + m = evaluate(req.marker, env = env | {"extra": x}, strict = False) if m == False: continue elif m == True: diff --git a/python/private/pypi/pep508_env.bzl b/python/private/pypi/pep508_env.bzl index 5031ebae12..9fe9dcaada 100644 --- a/python/private/pypi/pep508_env.bzl +++ b/python/private/pypi/pep508_env.bzl @@ -15,6 +15,7 @@ """This module is for implementing PEP508 environment definition. """ +load("//python/private:normalize_name.bzl", "normalize_name") load("//python/private:version.bzl", "version") _DEFAULT = "//conditions:default" @@ -215,9 +216,11 @@ def env(*, env = None, os, arch, python_version = "", extra = None): def create_env(): return { - # This is split by topic + # Per-variable normalization functions. Each entry maps a marker + # variable name to a function (value) -> normalized_value. "_aliases": { - "platform_machine": platform_machine_aliases, + "extra": normalize_name, + "platform_machine": lambda x: platform_machine_aliases.get(x, x), }, } diff --git a/python/private/pypi/pep508_evaluate.bzl b/python/private/pypi/pep508_evaluate.bzl index fe2cac965a..61e461a54f 100644 --- a/python/private/pypi/pep508_evaluate.bzl +++ b/python/private/pypi/pep508_evaluate.bzl @@ -300,12 +300,10 @@ def marker_expr(left, op, right, *, env, strict = True): left = left.strip("\"") if _ENV_ALIASES in env: - # On Windows, Linux, OSX different values may mean the same hardware, - # e.g. Python on Windows returns arm64, but on Linux returns aarch64. - # e.g. Python on Windows returns amd64, but on Linux returns x86_64. - # - # The following normalizes the values - left = env.get(_ENV_ALIASES, {}).get(var_name, {}).get(left, left) + # Normalize the literal value using per-variable normalization + # functions. This handles platform aliases (e.g. arm64 -> aarch64) + # and PEP 685 extra name normalization (e.g. db-backend -> db_backend). + left = env.get(_ENV_ALIASES, {}).get(var_name, lambda x: x)(left) else: var_name = left @@ -314,7 +312,7 @@ def marker_expr(left, op, right, *, env, strict = True): if _ENV_ALIASES in env: # See the note above on normalization - right = env.get(_ENV_ALIASES, {}).get(var_name, {}).get(right, right) + right = env.get(_ENV_ALIASES, {}).get(var_name, lambda x: x)(right) if var_name in _NON_VERSION_VAR_NAMES: return _env_expr(left, op, right) diff --git a/python/private/pypi/pep508_requirement.bzl b/python/private/pypi/pep508_requirement.bzl index b5be17f890..7552642572 100644 --- a/python/private/pypi/pep508_requirement.bzl +++ b/python/private/pypi/pep508_requirement.bzl @@ -45,7 +45,7 @@ def requirement(spec): extras_unparsed, _, _ = extras_unparsed.partition("]") for char in _STRIP: requires, _, _ = requires.partition(char) - extras = extras_unparsed.replace(" ", "").split(",") + extras = [normalize_name(e) for e in extras_unparsed.replace(" ", "").split(",") if e] name = requires.strip(" ") name = normalize_name(name) diff --git a/tests/pypi/pep508/deps_tests.bzl b/tests/pypi/pep508/deps_tests.bzl index 1404ad6fc1..e88acb8c56 100644 --- a/tests/pypi/pep508/deps_tests.bzl +++ b/tests/pypi/pep508/deps_tests.bzl @@ -218,6 +218,38 @@ def test_span_all_python_versions(env): _tests.append(test_span_all_python_versions) +def test_extras_with_hyphens_are_normalized(env): + """Test that extras with hyphens in marker expressions are normalized. + + When wheel METADATA uses hyphens in marker expressions + (e.g., extra == "db-backend") but the extras from requirement parsing + are already normalized (e.g., "db_backend"), the deps should still + resolve because marker evaluation normalizes per PEP 685. + + Args: + env: the test environment. + """ + requires_dist = [ + "bar", + 'baz-lib; extra == "db-backend"', + 'qux-async; extra == "async-driver"', + ] + + got = deps( + "foo", + extras = ["db_backend", "async_driver"], + requires_dist = requires_dist, + ) + + env.expect.that_collection(got.deps).contains_exactly([ + "bar", + "baz_lib", + "qux_async", + ]) + env.expect.that_dict(got.deps_select).contains_exactly({}) + +_tests.append(test_extras_with_hyphens_are_normalized) + def deps_test_suite(name): # buildifier: disable=function-docstring test_suite( name = name, diff --git a/tests/pypi/pep508/requirement_tests.bzl b/tests/pypi/pep508/requirement_tests.bzl index 9afb43a437..2ce45922da 100644 --- a/tests/pypi/pep508/requirement_tests.bzl +++ b/tests/pypi/pep508/requirement_tests.bzl @@ -23,9 +23,10 @@ def _test_requirement_line_parsing(env): " name1[ foo ] ": ("name1", ["foo"], None, ""), "Name[foo]": ("name", ["foo"], None, ""), "name [fred,bar] @ http://foo.com ; python_version=='2.7'": ("name", ["fred", "bar"], None, "python_version=='2.7'"), - "name; (os_name=='a' or os_name=='b') and os_name=='c'": ("name", [""], None, "(os_name=='a' or os_name=='b') and os_name=='c'"), - "name@http://foo.com": ("name", [""], None, ""), - "name[ Foo123 ]": ("name", ["Foo123"], None, ""), + "name; (os_name=='a' or os_name=='b') and os_name=='c'": ("name", [], None, "(os_name=='a' or os_name=='b') and os_name=='c'"), + "name@http://foo.com": ("name", [], None, ""), + "name[ Foo123 ]": ("name", ["foo123"], None, ""), + "name[extra-one,extra-two.three]==1.0": ("name", ["extra_one", "extra_two_three"], "1.0", ""), "name[extra]@http://foo.com": ("name", ["extra"], None, ""), "name[foo]": ("name", ["foo"], None, ""), "name[quux, strange];python_version<'2.7' and platform_version=='2'": ("name", ["quux", "strange"], None, "python_version<'2.7' and platform_version=='2'"), From 4be1f1f943526207f5f8d8245221a451c0cae830 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Thu, 12 Feb 2026 20:29:42 -0800 Subject: [PATCH 622/922] chore: print zipapp deprecation for non-windows platforms (#3591) Switching Windows off its zipapp based execution has turned out to be more tricky than anticipated, so just print the warning for non-Windows platforms to reduce spam. --- python/private/py_executable.bzl | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/python/private/py_executable.bzl b/python/private/py_executable.bzl index 45a18abb05..7ff6278e02 100644 --- a/python/private/py_executable.bzl +++ b/python/private/py_executable.bzl @@ -379,9 +379,10 @@ def _create_executable( # When --build_python_zip is enabled, then the zip file becomes # one of the default outputs. if build_zip_enabled: - # buildifier: disable=print - print( - """ + if not is_windows: + # buildifier: disable=print + print( + """ ====================================================================== WARNING: Target: {} The `--build_python_zip` flag and implicit zipapp output of `py_binary` @@ -392,7 +393,7 @@ WARNING: Target: {} https://github.com/bazel-contrib/rules_python/issues/3567 ====================================================================== """.rstrip().format(ctx.label), - ) + ) extra_default_outputs.append(zip_file) From 74e10bd01e7785b4e5f3d11792c0776921d33f33 Mon Sep 17 00:00:00 2001 From: Morten Mjelva Date: Sat, 14 Feb 2026 23:15:51 +0100 Subject: [PATCH 623/922] fix: Return repo_metadata from uv repository rule (#3597) This allows the uv repository rule to use the remote repo content cache. --- python/uv/private/uv_repository.bzl | 12 +++++++++++- 1 file changed, 11 insertions(+), 1 deletion(-) diff --git a/python/uv/private/uv_repository.bzl b/python/uv/private/uv_repository.bzl index fed4f576d3..79a6495bdc 100644 --- a/python/uv/private/uv_repository.bzl +++ b/python/uv/private/uv_repository.bzl @@ -57,7 +57,7 @@ def _uv_repo_impl(repository_ctx): ), ) - return { + attrs = { "name": repository_ctx.attr.name, "platform": repository_ctx.attr.platform, "sha256": result.sha256, @@ -65,6 +65,16 @@ def _uv_repo_impl(repository_ctx): "version": repository_ctx.attr.version, } + # Bazel <8.3.0 lacks repository_ctx.repo_metadata + if not hasattr(repository_ctx, "repo_metadata"): + return attrs + + reproducible = repository_ctx.attr.sha256 != "" + return repository_ctx.repo_metadata( + reproducible = reproducible, + attrs_for_reproducibility = {} if reproducible else attrs, + ) + uv_repository = repository_rule( _uv_repo_impl, doc = "Fetch external tools needed for uv toolchain", From 22f135285a55e3bf76fad9957a9873a18b6b23d1 Mon Sep 17 00:00:00 2001 From: Morten Mjelva Date: Sun, 15 Feb 2026 01:49:23 +0100 Subject: [PATCH 624/922] fix: Return repo_metadata from python repository rule (#3598) This allows the python repository rule to use the remote repo content cache. --- python/private/python_repository.bzl | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-) diff --git a/python/private/python_repository.bzl b/python/private/python_repository.bzl index 16c522f398..3d54b8a26d 100644 --- a/python/private/python_repository.bzl +++ b/python/private/python_repository.bzl @@ -224,7 +224,15 @@ define_hermetic_runtime_toolchain_impl( else: attrs["urls"] = urls - return attrs + # Bazel <8.3.0 lacks repository_ctx.repo_metadata + if not hasattr(rctx, "repo_metadata"): + return attrs + + reproducible = rctx.attr.sha256 != "" + return rctx.repo_metadata( + reproducible = reproducible, + attrs_for_reproducibility = {} if reproducible else attrs, + ) python_repository = repository_rule( _python_repository_impl, From cd34111799bd18353a933d2608f68cae484c8763 Mon Sep 17 00:00:00 2001 From: Danner Stodolsky c/o Boston Dynamics Date: Sat, 14 Feb 2026 23:21:10 -0500 Subject: [PATCH 625/922] perf(py_wheel): defer depset expansion to execution time (#3599) This is the `py_wheel` counterpart to the analysis-time performance work done for `py_binary`/`py_test` in #3381 and #3442, deferring depset expansion to execution time. Tested under Bazel 8.5.1 and 9.0.0. --------- Co-authored-by: Claude Opus 4.6 --- CHANGELOG.md | 2 ++ python/private/py_wheel.bzl | 11 ++++++----- 2 files changed, 8 insertions(+), 5 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 418d71b4a7..49fcf5d7e9 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -72,6 +72,8 @@ END_UNRELEASED_TEMPLATE {obj}`--stamp` flag. * (pypi) Now the RECORD file patches will follow the quoted or unquoted filenames convention in order to make `pytorch` and friends easier to patch. +* (wheel) `py_wheel` no longer expands the input depset during analysis, + improving analysis performance for targets with large dependency trees. {#v0-0-0-fixed} ### Fixed diff --git a/python/private/py_wheel.bzl b/python/private/py_wheel.bzl index 8202fa015a..1d98d21a65 100644 --- a/python/private/py_wheel.bzl +++ b/python/private/py_wheel.bzl @@ -344,12 +344,13 @@ def _py_wheel_impl(ctx): # Currently this is only the description file (if used). other_inputs = [] - # Wrap the inputs into a file to reduce command line length. + # Wrap the inputs into a file to reduce command line length, deferring + # depset expansion to execution time via Args.add_all with map_each. packageinputfile = ctx.actions.declare_file(ctx.attr.name + "_target_wrapped_inputs.txt") - content = "" - for input_file in inputs_to_package.to_list(): - content += _input_file_to_arg(input_file) + "\n" - ctx.actions.write(output = packageinputfile, content = content) + package_args = ctx.actions.args() + package_args.set_param_file_format("multiline") + package_args.add_all(inputs_to_package, map_each = _input_file_to_arg) + ctx.actions.write(output = packageinputfile, content = package_args) other_inputs.append(packageinputfile) args = ctx.actions.args() From c9783c8a41aab893757617fb00ec7ea975e4f9df Mon Sep 17 00:00:00 2001 From: Douglas Thor Date: Sun, 15 Feb 2026 22:51:36 -0800 Subject: [PATCH 626/922] feat(gazelle): Directive controlling pytest ancestor dependencies (#3596) Fixes #3595. Add a new directive `python_include_ancestor_conftest`, defaulting to `true`, that configures whether or not ancestor `conftest` targets are added to a `py_test` target's dependencies. --- CHANGELOG.md | 10 ++- gazelle/docs/annotations.md | 1 + gazelle/docs/directives.md | 69 +++++++++++++++++++ gazelle/python/configure.go | 7 ++ gazelle/python/generate.go | 15 ++-- .../BUILD.in | 0 .../BUILD.out | 8 +++ .../MODULE.bazel | 0 .../README.md | 23 +++++++ .../WORKSPACE | 0 .../conftest.py | 0 .../one/BUILD.in | 0 .../one/BUILD.out | 17 +++++ .../one/conftest.py | 0 .../one/my_test.py | 0 .../one/two/BUILD.in | 1 + .../one/two/BUILD.out | 16 +++++ .../one/two/conftest.py | 0 .../one/two/my_test.py | 0 .../one/two/no_conftest/BUILD.in | 0 .../one/two/no_conftest/BUILD.out | 6 ++ .../one/two/no_conftest/my_test.py | 0 .../one/two/three/BUILD.in | 1 + .../one/two/three/BUILD.out | 21 ++++++ .../one/two/three/conftest.py | 0 .../one/two/three/my_test.py | 0 .../one/two/three/no_conftest/BUILD.in | 0 .../one/two/three/no_conftest/BUILD.out | 12 ++++ .../one/two/three/no_conftest/my_test.py | 0 .../test.yaml | 3 + gazelle/pythonconfig/pythonconfig.go | 22 ++++++ 31 files changed, 226 insertions(+), 6 deletions(-) create mode 100644 gazelle/python/testdata/directive_python_include_ancestor_conftest/BUILD.in create mode 100644 gazelle/python/testdata/directive_python_include_ancestor_conftest/BUILD.out create mode 100644 gazelle/python/testdata/directive_python_include_ancestor_conftest/MODULE.bazel create mode 100644 gazelle/python/testdata/directive_python_include_ancestor_conftest/README.md create mode 100644 gazelle/python/testdata/directive_python_include_ancestor_conftest/WORKSPACE create mode 100644 gazelle/python/testdata/directive_python_include_ancestor_conftest/conftest.py create mode 100644 gazelle/python/testdata/directive_python_include_ancestor_conftest/one/BUILD.in create mode 100644 gazelle/python/testdata/directive_python_include_ancestor_conftest/one/BUILD.out create mode 100644 gazelle/python/testdata/directive_python_include_ancestor_conftest/one/conftest.py create mode 100644 gazelle/python/testdata/directive_python_include_ancestor_conftest/one/my_test.py create mode 100644 gazelle/python/testdata/directive_python_include_ancestor_conftest/one/two/BUILD.in create mode 100644 gazelle/python/testdata/directive_python_include_ancestor_conftest/one/two/BUILD.out create mode 100644 gazelle/python/testdata/directive_python_include_ancestor_conftest/one/two/conftest.py create mode 100644 gazelle/python/testdata/directive_python_include_ancestor_conftest/one/two/my_test.py create mode 100644 gazelle/python/testdata/directive_python_include_ancestor_conftest/one/two/no_conftest/BUILD.in create mode 100644 gazelle/python/testdata/directive_python_include_ancestor_conftest/one/two/no_conftest/BUILD.out create mode 100644 gazelle/python/testdata/directive_python_include_ancestor_conftest/one/two/no_conftest/my_test.py create mode 100644 gazelle/python/testdata/directive_python_include_ancestor_conftest/one/two/three/BUILD.in create mode 100644 gazelle/python/testdata/directive_python_include_ancestor_conftest/one/two/three/BUILD.out create mode 100644 gazelle/python/testdata/directive_python_include_ancestor_conftest/one/two/three/conftest.py create mode 100644 gazelle/python/testdata/directive_python_include_ancestor_conftest/one/two/three/my_test.py create mode 100644 gazelle/python/testdata/directive_python_include_ancestor_conftest/one/two/three/no_conftest/BUILD.in create mode 100644 gazelle/python/testdata/directive_python_include_ancestor_conftest/one/two/three/no_conftest/BUILD.out create mode 100644 gazelle/python/testdata/directive_python_include_ancestor_conftest/one/two/three/no_conftest/my_test.py create mode 100644 gazelle/python/testdata/directive_python_include_ancestor_conftest/test.yaml diff --git a/CHANGELOG.md b/CHANGELOG.md index 49fcf5d7e9..18234744bd 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -82,7 +82,9 @@ END_UNRELEASED_TEMPLATE * (tests) No more coverage warnings are being printed if there are no sources. ([#2762](https://github.com/bazel-contrib/rules_python/issues/2762)) * (gazelle) Ancestor `conftest.py` files are added in addition to sibling `conftest.py`. - ([#3497](https://github.com/bazel-contrib/rules_python/issues/3497)) + ([#3497](https://github.com/bazel-contrib/rules_python/issues/3497)) Note + that this behavior can be reverted to the pre-VERSION_NEXT_FEATURE behavior by setting the new + `python_include_ancestor_conftest` directive to `false`. * (pypi) `pip_parse` no longer silently drops PEP 508 URL-based requirements (`pkg @ https://...`) when `extract_url_srcs=False` (the default for `pip_repository`). @@ -115,6 +117,12 @@ END_UNRELEASED_TEMPLATE {obj}`PyExecutableInfo.venv_python_exe`. * (tools/wheelmaker.py) Added support for URL requirements according to PEP 508 in Requires-Dist metadata. ([#3569](https://github.com/bazel-contrib/rules_python/pull/3569)) +* (gazelle) A new directive `python_include_ancestor_conftest` has been added. + When `false`, ancestor `conftest` targets are not automatically added to + {bzl:obj}`py_test` target dependencies. This `false` behavior is how things + were in `rules_python` before VERSION_NEXT_FEATURE. The default is `true`, as the prior behavior + was technically incorrect. + ([#3596](https://github.com/bazel-contrib/rules_python/pull/3596)) {#v1-8-4} ## [1.8.4] - 2026-02-10 diff --git a/gazelle/docs/annotations.md b/gazelle/docs/annotations.md index 728027ffda..b6eb96d2cd 100644 --- a/gazelle/docs/annotations.md +++ b/gazelle/docs/annotations.md @@ -116,6 +116,7 @@ deps = [ ``` +(annotation-include-pytest-conftest)= ## `include_pytest_conftest` :::{versionadded} 1.6.0 diff --git a/gazelle/docs/directives.md b/gazelle/docs/directives.md index c7936d539d..628dce5ae6 100644 --- a/gazelle/docs/directives.md +++ b/gazelle/docs/directives.md @@ -175,6 +175,11 @@ The Python-specific directives are: * Default: `false` * Allowed Values: `true`, `false` +[`# gazelle:python_include_ancestor_conftest bool`](#python-include-ancestor-conftest) +: Controls whether ancestor conftest targets are added to {bzl:obj}`py_test` target + dependencies. + * Default: `true` + * Allowed Values: `true`, `false` ## `python_extension` @@ -720,3 +725,67 @@ previously-generated or hand-created rules. :::{error} Detailed docs are not yet written. ::: + +## `python_include_ancestor_conftest` + +Version VERSION_NEXT_FEATURE includes a fix ({gh-pr}`3498`) for a long-standing issue +({gh-issue}`3497`) where ancestor `conftest.py` files were not automatically +added as dependencies of {bzl:obj}`py_test` targets. + +However, some people may not want this behavior (see https://xkcd.com/1172/). +Thus the `python_include_ancestor_conftest` directive controls this behavior. +It defaults to `true`, which causes all ancestor `conftest.py` files to be +included as dependencies for {bzl:obj}`py_test` targets. + +Setting the directive to `false` reverts to the pre-VERSION_NEXT_FEATURE behavior. + +For example, given this directory tree (not shown: intermediary `BUILD.bazel` +files) + +``` +./ +├── conftest.py +└── one/ + ├── conftest.py + └── two/ + ├── conftest.py + └── three/ + ├── BUILD.bazel + ├── conftest.py + └── my_test.py +``` + +Gazelle will generate this target for `foo_test.py` by default: + +```starlark +py_test( + name = "foo_test", + srcs = ["foo_test.py"], + deps = [ + ":conftest", # same as "//one:two/three:conftest" + "//:conftest", + "//one:conftest", + "//one/two:conftest", + ], +) +``` + +But when `python_include_ancestor_conftest` is `false`, only the sibling +`:conftest` target will be included as a dependency: + +:::{tip} +The [`include_pytest_conftest` annotation](annotation-include-pytest-conftest) +controls whether the sibling `:conftest` target is added to {bzl:obj}`py_test` +target dependency list. +::: + +```starlark +# gazelle:python_include_ancestor_conftest false +py_test( + name = "foo_test", + srcs = ["foo_test.py"], + deps = [ + ":conftest", + ], +) +``` diff --git a/gazelle/python/configure.go b/gazelle/python/configure.go index 88b15e91eb..1fe95a1683 100644 --- a/gazelle/python/configure.go +++ b/gazelle/python/configure.go @@ -74,6 +74,7 @@ func (py *Configurer) KnownDirectives() []string { pythonconfig.ExperimentalAllowRelativeImports, pythonconfig.GenerateProto, pythonconfig.PythonResolveSiblingImports, + pythonconfig.PythonIncludeAncestorConftest, } } @@ -261,6 +262,12 @@ func (py *Configurer) Configure(c *config.Config, rel string, f *rule.File) { log.Fatal(err) } config.SetResolveSiblingImports(v) + case pythonconfig.PythonIncludeAncestorConftest: + v, err := strconv.ParseBool(strings.TrimSpace(d.Value)) + if err != nil { + log.Fatal(err) + } + config.SetIncludeAncestorConftest(v) } } diff --git a/gazelle/python/generate.go b/gazelle/python/generate.go index ebca10671c..2495c42d20 100644 --- a/gazelle/python/generate.go +++ b/gazelle/python/generate.go @@ -68,7 +68,7 @@ func matchesAnyGlob(s string, globs []string) bool { // findConftestPaths returns package paths containing conftest.py, from currentPkg // up through ancestors, stopping at module root. -func findConftestPaths(repoRoot, currentPkg, pythonProjectRoot string) []string { +func findConftestPaths(repoRoot, currentPkg, pythonProjectRoot string, includeAncestorConftest bool) []string { var result []string for pkg := currentPkg; ; pkg = filepath.Dir(pkg) { if pkg == "." { @@ -77,6 +77,12 @@ func findConftestPaths(repoRoot, currentPkg, pythonProjectRoot string) []string if _, err := os.Stat(filepath.Join(repoRoot, pkg, conftestFilename)); err == nil { result = append(result, pkg) } + // We traverse up the tree to find conftest files and we start in + // the current package. Thus if we find one in the current package + // and do not want ancestors, we break early. + if !includeAncestorConftest { + break + } if pkg == "" { break } @@ -402,7 +408,6 @@ func (py *Python) GenerateRules(args language.GenerateArgs) language.GenerateRes setAnnotations(*annotations). generateImportsAttribute() - pyBinary := pyBinaryTarget.build() result.Gen = append(result.Gen, pyBinary) @@ -526,13 +531,13 @@ func (py *Python) GenerateRules(args language.GenerateArgs) language.GenerateRes for _, pyTestTarget := range pyTestTargets { shouldAddConftest := pyTestTarget.annotations.includePytestConftest == nil || - *pyTestTarget.annotations.includePytestConftest + *pyTestTarget.annotations.includePytestConftest if shouldAddConftest { - for _, conftestPkg := range findConftestPaths(args.Config.RepoRoot, args.Rel, pythonProjectRoot) { + for _, conftestPkg := range findConftestPaths(args.Config.RepoRoot, args.Rel, pythonProjectRoot, cfg.IncludeAncestorConftest()) { pyTestTarget.addModuleDependency( Module{ - Name: importSpecFromSrc(pythonProjectRoot, conftestPkg, conftestFilename).Imp, + Name: importSpecFromSrc(pythonProjectRoot, conftestPkg, conftestFilename).Imp, Filepath: filepath.Join(conftestPkg, conftestFilename), }, ) diff --git a/gazelle/python/testdata/directive_python_include_ancestor_conftest/BUILD.in b/gazelle/python/testdata/directive_python_include_ancestor_conftest/BUILD.in new file mode 100644 index 0000000000..e69de29bb2 diff --git a/gazelle/python/testdata/directive_python_include_ancestor_conftest/BUILD.out b/gazelle/python/testdata/directive_python_include_ancestor_conftest/BUILD.out new file mode 100644 index 0000000000..c7adad8336 --- /dev/null +++ b/gazelle/python/testdata/directive_python_include_ancestor_conftest/BUILD.out @@ -0,0 +1,8 @@ +load("@rules_python//python:defs.bzl", "py_library") + +py_library( + name = "conftest", + testonly = True, + srcs = ["conftest.py"], + visibility = ["//:__subpackages__"], +) diff --git a/gazelle/python/testdata/directive_python_include_ancestor_conftest/MODULE.bazel b/gazelle/python/testdata/directive_python_include_ancestor_conftest/MODULE.bazel new file mode 100644 index 0000000000..e69de29bb2 diff --git a/gazelle/python/testdata/directive_python_include_ancestor_conftest/README.md b/gazelle/python/testdata/directive_python_include_ancestor_conftest/README.md new file mode 100644 index 0000000000..956e65155b --- /dev/null +++ b/gazelle/python/testdata/directive_python_include_ancestor_conftest/README.md @@ -0,0 +1,23 @@ +# Directive: `python_include_ancestor_conftest` + +This test case asserts that the `# gazelle:python_include_ancestor_conftest` +directive correctly includes or excludes ancestor `conftest` targets in +`py_test` target dependencies. + +The test also asserts that the directive can be applied at any level and that +child levels will inherit the value: + ++ The root level does not set the directive (it defaults to True). ++ The next level, `one/`, inherits that value. ++ The next level, `one/two/`, sets the directive to False; consequently the + `py_test` target only includes the sibling `:conftest` target. + + The `one/two/no_conftest/` directory does not contain a `conftest.py` file + thereby asserting that we correctly do not include any `conftest` targets + whatsoever. ++ The final level, `one/two/three/`, sets the directive back to True, meaning + the `py_test` target includes a total of 4 `conftest` targets. + + The `one/two/three/no_conftest/` directory does not contain a `conftest.py` + file and thus asserts that the code includes _only_ ancestor `conftest` + targets. + +See [Issue #3595](https://github.com/bazel-contrib/rules_python/issues/3595). diff --git a/gazelle/python/testdata/directive_python_include_ancestor_conftest/WORKSPACE b/gazelle/python/testdata/directive_python_include_ancestor_conftest/WORKSPACE new file mode 100644 index 0000000000..e69de29bb2 diff --git a/gazelle/python/testdata/directive_python_include_ancestor_conftest/conftest.py b/gazelle/python/testdata/directive_python_include_ancestor_conftest/conftest.py new file mode 100644 index 0000000000..e69de29bb2 diff --git a/gazelle/python/testdata/directive_python_include_ancestor_conftest/one/BUILD.in b/gazelle/python/testdata/directive_python_include_ancestor_conftest/one/BUILD.in new file mode 100644 index 0000000000..e69de29bb2 diff --git a/gazelle/python/testdata/directive_python_include_ancestor_conftest/one/BUILD.out b/gazelle/python/testdata/directive_python_include_ancestor_conftest/one/BUILD.out new file mode 100644 index 0000000000..9d0405e92f --- /dev/null +++ b/gazelle/python/testdata/directive_python_include_ancestor_conftest/one/BUILD.out @@ -0,0 +1,17 @@ +load("@rules_python//python:defs.bzl", "py_library", "py_test") + +py_library( + name = "conftest", + testonly = True, + srcs = ["conftest.py"], + visibility = ["//:__subpackages__"], +) + +py_test( + name = "my_test", + srcs = ["my_test.py"], + deps = [ + ":conftest", + "//:conftest", + ], +) diff --git a/gazelle/python/testdata/directive_python_include_ancestor_conftest/one/conftest.py b/gazelle/python/testdata/directive_python_include_ancestor_conftest/one/conftest.py new file mode 100644 index 0000000000..e69de29bb2 diff --git a/gazelle/python/testdata/directive_python_include_ancestor_conftest/one/my_test.py b/gazelle/python/testdata/directive_python_include_ancestor_conftest/one/my_test.py new file mode 100644 index 0000000000..e69de29bb2 diff --git a/gazelle/python/testdata/directive_python_include_ancestor_conftest/one/two/BUILD.in b/gazelle/python/testdata/directive_python_include_ancestor_conftest/one/two/BUILD.in new file mode 100644 index 0000000000..3805e248e2 --- /dev/null +++ b/gazelle/python/testdata/directive_python_include_ancestor_conftest/one/two/BUILD.in @@ -0,0 +1 @@ +# gazelle:python_include_ancestor_conftest false diff --git a/gazelle/python/testdata/directive_python_include_ancestor_conftest/one/two/BUILD.out b/gazelle/python/testdata/directive_python_include_ancestor_conftest/one/two/BUILD.out new file mode 100644 index 0000000000..edb91a38f5 --- /dev/null +++ b/gazelle/python/testdata/directive_python_include_ancestor_conftest/one/two/BUILD.out @@ -0,0 +1,16 @@ +load("@rules_python//python:defs.bzl", "py_library", "py_test") + +# gazelle:python_include_ancestor_conftest false + +py_library( + name = "conftest", + testonly = True, + srcs = ["conftest.py"], + visibility = ["//:__subpackages__"], +) + +py_test( + name = "my_test", + srcs = ["my_test.py"], + deps = [":conftest"], +) diff --git a/gazelle/python/testdata/directive_python_include_ancestor_conftest/one/two/conftest.py b/gazelle/python/testdata/directive_python_include_ancestor_conftest/one/two/conftest.py new file mode 100644 index 0000000000..e69de29bb2 diff --git a/gazelle/python/testdata/directive_python_include_ancestor_conftest/one/two/my_test.py b/gazelle/python/testdata/directive_python_include_ancestor_conftest/one/two/my_test.py new file mode 100644 index 0000000000..e69de29bb2 diff --git a/gazelle/python/testdata/directive_python_include_ancestor_conftest/one/two/no_conftest/BUILD.in b/gazelle/python/testdata/directive_python_include_ancestor_conftest/one/two/no_conftest/BUILD.in new file mode 100644 index 0000000000..e69de29bb2 diff --git a/gazelle/python/testdata/directive_python_include_ancestor_conftest/one/two/no_conftest/BUILD.out b/gazelle/python/testdata/directive_python_include_ancestor_conftest/one/two/no_conftest/BUILD.out new file mode 100644 index 0000000000..764e2b4172 --- /dev/null +++ b/gazelle/python/testdata/directive_python_include_ancestor_conftest/one/two/no_conftest/BUILD.out @@ -0,0 +1,6 @@ +load("@rules_python//python:defs.bzl", "py_test") + +py_test( + name = "my_test", + srcs = ["my_test.py"], +) diff --git a/gazelle/python/testdata/directive_python_include_ancestor_conftest/one/two/no_conftest/my_test.py b/gazelle/python/testdata/directive_python_include_ancestor_conftest/one/two/no_conftest/my_test.py new file mode 100644 index 0000000000..e69de29bb2 diff --git a/gazelle/python/testdata/directive_python_include_ancestor_conftest/one/two/three/BUILD.in b/gazelle/python/testdata/directive_python_include_ancestor_conftest/one/two/three/BUILD.in new file mode 100644 index 0000000000..987cd7dc20 --- /dev/null +++ b/gazelle/python/testdata/directive_python_include_ancestor_conftest/one/two/three/BUILD.in @@ -0,0 +1 @@ +# gazelle:python_include_ancestor_conftest true diff --git a/gazelle/python/testdata/directive_python_include_ancestor_conftest/one/two/three/BUILD.out b/gazelle/python/testdata/directive_python_include_ancestor_conftest/one/two/three/BUILD.out new file mode 100644 index 0000000000..605aa00f88 --- /dev/null +++ b/gazelle/python/testdata/directive_python_include_ancestor_conftest/one/two/three/BUILD.out @@ -0,0 +1,21 @@ +load("@rules_python//python:defs.bzl", "py_library", "py_test") + +# gazelle:python_include_ancestor_conftest true + +py_library( + name = "conftest", + testonly = True, + srcs = ["conftest.py"], + visibility = ["//:__subpackages__"], +) + +py_test( + name = "my_test", + srcs = ["my_test.py"], + deps = [ + ":conftest", + "//:conftest", + "//one:conftest", + "//one/two:conftest", + ], +) diff --git a/gazelle/python/testdata/directive_python_include_ancestor_conftest/one/two/three/conftest.py b/gazelle/python/testdata/directive_python_include_ancestor_conftest/one/two/three/conftest.py new file mode 100644 index 0000000000..e69de29bb2 diff --git a/gazelle/python/testdata/directive_python_include_ancestor_conftest/one/two/three/my_test.py b/gazelle/python/testdata/directive_python_include_ancestor_conftest/one/two/three/my_test.py new file mode 100644 index 0000000000..e69de29bb2 diff --git a/gazelle/python/testdata/directive_python_include_ancestor_conftest/one/two/three/no_conftest/BUILD.in b/gazelle/python/testdata/directive_python_include_ancestor_conftest/one/two/three/no_conftest/BUILD.in new file mode 100644 index 0000000000..e69de29bb2 diff --git a/gazelle/python/testdata/directive_python_include_ancestor_conftest/one/two/three/no_conftest/BUILD.out b/gazelle/python/testdata/directive_python_include_ancestor_conftest/one/two/three/no_conftest/BUILD.out new file mode 100644 index 0000000000..c1bddccc30 --- /dev/null +++ b/gazelle/python/testdata/directive_python_include_ancestor_conftest/one/two/three/no_conftest/BUILD.out @@ -0,0 +1,12 @@ +load("@rules_python//python:defs.bzl", "py_test") + +py_test( + name = "my_test", + srcs = ["my_test.py"], + deps = [ + "//:conftest", + "//one:conftest", + "//one/two:conftest", + "//one/two/three:conftest", + ], +) diff --git a/gazelle/python/testdata/directive_python_include_ancestor_conftest/one/two/three/no_conftest/my_test.py b/gazelle/python/testdata/directive_python_include_ancestor_conftest/one/two/three/no_conftest/my_test.py new file mode 100644 index 0000000000..e69de29bb2 diff --git a/gazelle/python/testdata/directive_python_include_ancestor_conftest/test.yaml b/gazelle/python/testdata/directive_python_include_ancestor_conftest/test.yaml new file mode 100644 index 0000000000..36dd656b39 --- /dev/null +++ b/gazelle/python/testdata/directive_python_include_ancestor_conftest/test.yaml @@ -0,0 +1,3 @@ +--- +expect: + exit_code: 0 diff --git a/gazelle/pythonconfig/pythonconfig.go b/gazelle/pythonconfig/pythonconfig.go index a1271af3be..17db9aae0d 100644 --- a/gazelle/pythonconfig/pythonconfig.go +++ b/gazelle/pythonconfig/pythonconfig.go @@ -116,6 +116,15 @@ const ( // like "import a" can be resolved to sibling modules. When disabled, they // can only be resolved as an absolute import. PythonResolveSiblingImports = "python_resolve_sibling_imports" + // PythonIncludeAncestorConftest represents the directive that controls + // whether ancestor conftest.py files are added as dependencies to py_test + // targets. When enabled (the default), ancestor conftest.py files are + // included as deps. + // See also https://github.com/bazel-contrib/rules_python/pull/3498, which + // fixed previous behavior that was incorrectly _not_ adding the files and + // https://github.com/bazel-contrib/rules_python/issues/3595 which requested + // that the behavior be configurable. + PythonIncludeAncestorConftest = "python_include_ancestor_conftest" ) // GenerationModeType represents one of the generation modes for the Python @@ -209,6 +218,7 @@ type Config struct { generatePyiSrcs bool generateProto bool resolveSiblingImports bool + includeAncestorConftest bool } type LabelNormalizationType int @@ -250,6 +260,7 @@ func New( generatePyiSrcs: false, generateProto: false, resolveSiblingImports: false, + includeAncestorConftest: true, } } @@ -288,6 +299,7 @@ func (c *Config) NewChild() *Config { generatePyiSrcs: c.generatePyiSrcs, generateProto: c.generateProto, resolveSiblingImports: c.resolveSiblingImports, + includeAncestorConftest: c.includeAncestorConftest, } } @@ -629,6 +641,16 @@ func (c *Config) ResolveSiblingImports() bool { return c.resolveSiblingImports } +// SetIncludeAncestorConftest sets whether ancestor conftest files are added to py_test targets. +func (c *Config) SetIncludeAncestorConftest(includeAncestorConftest bool) { + c.includeAncestorConftest = includeAncestorConftest +} + +// IncludeAncestorConftest returns whether ancestor conftest files are added to py_test targets. +func (c *Config) IncludeAncestorConftest() bool { + return c.includeAncestorConftest +} + // FormatThirdPartyDependency returns a label to a third-party dependency performing all formating and normalization. func (c *Config) FormatThirdPartyDependency(repositoryName string, distributionName string) label.Label { conventionalDistributionName := strings.ReplaceAll(c.labelConvention, distributionNameLabelConventionSubstitution, distributionName) From 4074538a3b0ac4bee476ae241ba3cb2e83bc5941 Mon Sep 17 00:00:00 2001 From: Saish Bhujbal Date: Wed, 18 Feb 2026 06:23:56 +0530 Subject: [PATCH 627/922] docs: document current_py_cc_headers and related toolchain targets (#3602) Adds documentation explaining: - current_py_cc_headers - current_py_cc_headers_abi3 - current_py_cc_libs Describes how these targets expose headers and libraries from the active Python C toolchain and clarifies that they should be used instead of legacy alias targets when embedding Python or building C extensions. This addresses issue #3098. --------- Co-authored-by: Richard Levasseur --- docs/toolchains.md | 32 ++++++++++++++++++++++++++++++-- 1 file changed, 30 insertions(+), 2 deletions(-) diff --git a/docs/toolchains.md b/docs/toolchains.md index 186ad11e73..1e1e61fde8 100644 --- a/docs/toolchains.md +++ b/docs/toolchains.md @@ -555,14 +555,14 @@ Python is used to run a program but also makes it easy to use a Python version that isn't compatible with build-time assumptions. ``` -register_toolchains("@rules_python//python/runtime_env_toolchains:all") +`register_toolchains`("@rules_python//python/runtime_env_toolchains:all") ``` Note that this toolchain has no constraints, i.e. it will match any platform, Python version, etc. :::{seealso} -[Local toolchain], which creates a more full featured toolchain from a +[Local `toolchain`], which creates a more full featured toolchain from a locally installed Python. ::: @@ -846,3 +846,31 @@ The [`//python/bin:repl` target](repl) provides an environment identical to what `py_binary` provides. That means it handles things like the [`PYTHONSAFEPATH`](https://docs.python.org/3/using/cmdline.html#envvar-PYTHONSAFEPATH) environment variable automatically. The `//python/bin:python` target will not. + +## Consuming Python C headers and libraries + +The following targets expose the headers and libraries from the +currently selected Python C toolchain: + +- {obj}`@rules_python//python/cc:` +`current_py_cc_headers` +- {obj}`@rules_python//python/cc:current_py_cc_headers_abi3` +- {obj}`@rules_python//python/cc:current_py_cc_libs` + +These targets behave similarly to a `cc_library`, but instead of defining +their own sources, they forward providers from the underlying toolchain- +selected `cc_library`. + +A Python C toolchain must be registered for these targets to work. +Under bzlmod, a toolchain is registered automatically. In non-bzlmod +setups, users must ensure that a toolchain is explicitly registered. + +Users should depend on these targets instead of legacy alias targets +when embedding Python or building C extensions, as this ensures +compatibility across different toolchain configurations. + + +:::{seealso} +The _How to get Python headres for C extensions_ how-to guide, and the +{obj}`@rules_python//python/cc` package API documentation. +::: From d31ec0d92d3c45d8898be5228b88f3c29f6a6eee Mon Sep 17 00:00:00 2001 From: Douglas Thor Date: Tue, 17 Feb 2026 17:36:29 -0800 Subject: [PATCH 628/922] test(gazelle): Update remove_invalid_(binary|library) gazelle tests. (#3601) Update `remove_invalid_(binary|library)` gazelle tests. These changes will make a little more sense in a followup PR that finishes the "delete invalid targets" work started in #3046. In that PR, I pushed back on changes that would mean `py_test` and `py_library` targets got removed if they were invalid (see https://github.com/bazel-contrib/rules_python/pull/3046#pullrequestreview-3014780071). It's now time to fix that and make it so _any_ invalid target, not just `py_binary`, is removed see https://github.com/bazel-contrib/rules_python/pull/3046#issuecomment-3094797158). That change will come in a followup PR. This PR updates tests by adding some always-valid targets and does some light renaming of the targets. --- gazelle/python/testdata/remove_invalid_binary/BUILD.in | 6 ++++-- gazelle/python/testdata/remove_invalid_binary/BUILD.out | 5 +++-- gazelle/python/testdata/remove_invalid_binary/__init__.py | 0 gazelle/python/testdata/remove_invalid_library/BUILD.in | 7 ++++++- gazelle/python/testdata/remove_invalid_library/BUILD.out | 7 ++++++- gazelle/python/testdata/remove_invalid_library/my_test.py | 0 6 files changed, 19 insertions(+), 6 deletions(-) create mode 100644 gazelle/python/testdata/remove_invalid_binary/__init__.py create mode 100644 gazelle/python/testdata/remove_invalid_library/my_test.py diff --git a/gazelle/python/testdata/remove_invalid_binary/BUILD.in b/gazelle/python/testdata/remove_invalid_binary/BUILD.in index 87d357139b..f4bfd65c1b 100644 --- a/gazelle/python/testdata/remove_invalid_binary/BUILD.in +++ b/gazelle/python/testdata/remove_invalid_binary/BUILD.in @@ -1,11 +1,13 @@ load("@rules_python//python:defs.bzl", "py_binary", "py_library") py_library( - name = "keep_library", + name = "remove_invalid_binary", + srcs = ["__init__.py"], deps = ["//keep_binary:foo"], ) + py_binary( - name = "remove_invalid_binary", + name = "remove_invalid_binary_bin", srcs = ["__main__.py"], data = ["testdata/test.txt"], visibility = ["//:__subpackages__"], diff --git a/gazelle/python/testdata/remove_invalid_binary/BUILD.out b/gazelle/python/testdata/remove_invalid_binary/BUILD.out index 069188f5ca..a217b4bdaf 100644 --- a/gazelle/python/testdata/remove_invalid_binary/BUILD.out +++ b/gazelle/python/testdata/remove_invalid_binary/BUILD.out @@ -1,6 +1,7 @@ load("@rules_python//python:defs.bzl", "py_library") py_library( - name = "keep_library", - deps = ["//keep_binary:foo"], + name = "remove_invalid_binary", + srcs = ["__init__.py"], + visibility = ["//:__subpackages__"], ) diff --git a/gazelle/python/testdata/remove_invalid_binary/__init__.py b/gazelle/python/testdata/remove_invalid_binary/__init__.py new file mode 100644 index 0000000000..e69de29bb2 diff --git a/gazelle/python/testdata/remove_invalid_library/BUILD.in b/gazelle/python/testdata/remove_invalid_library/BUILD.in index 3f24c8df35..815e673ee1 100644 --- a/gazelle/python/testdata/remove_invalid_library/BUILD.in +++ b/gazelle/python/testdata/remove_invalid_library/BUILD.in @@ -1,4 +1,4 @@ -load("@rules_python//python:defs.bzl", "py_library") +load("@rules_python//python:defs.bzl", "py_library", "py_test") py_library( name = "remove_invalid_library", @@ -14,3 +14,8 @@ py_library( "@pypi//foo", ], ) + +py_test( + name = "my_test", + srcs = ["my_test.py"], +) diff --git a/gazelle/python/testdata/remove_invalid_library/BUILD.out b/gazelle/python/testdata/remove_invalid_library/BUILD.out index 4a6fffa183..80d47076c3 100644 --- a/gazelle/python/testdata/remove_invalid_library/BUILD.out +++ b/gazelle/python/testdata/remove_invalid_library/BUILD.out @@ -1,4 +1,4 @@ -load("@rules_python//python:defs.bzl", "py_library") +load("@rules_python//python:defs.bzl", "py_library", "py_test") py_library( name = "deps_with_no_srcs_library", @@ -8,3 +8,8 @@ py_library( "@pypi//foo", ], ) + +py_test( + name = "my_test", + srcs = ["my_test.py"], +) diff --git a/gazelle/python/testdata/remove_invalid_library/my_test.py b/gazelle/python/testdata/remove_invalid_library/my_test.py new file mode 100644 index 0000000000..e69de29bb2 From 659b87b82988e02844712535f0a84d13c204b1b0 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Thu, 19 Feb 2026 15:46:50 -0800 Subject: [PATCH 629/922] chore: better build data error handling (#3606) This add some additional error handling, debugging, and normalization logic to retrieving build data. This came in handy when diagnosing errors on Windows. --- python/private/stage2_bootstrap_template.py | 21 ++++++++++++++++++--- 1 file changed, 18 insertions(+), 3 deletions(-) diff --git a/python/private/stage2_bootstrap_template.py b/python/private/stage2_bootstrap_template.py index 959e631ad1..5e0472ee1f 100644 --- a/python/private/stage2_bootstrap_template.py +++ b/python/private/stage2_bootstrap_template.py @@ -48,6 +48,7 @@ COVERAGE_INSTRUMENTED = "%coverage_instrumented%" == "1" # runfiles-root-relative path to a file with binary-specific build information +# It uses forward slashes, so must be converted for proper usage on Windows. BUILD_DATA_FILE = "%build_data_file%" # ===== Template substitutions end ===== @@ -64,9 +65,23 @@ def get_build_data(self): import runfiles except ImportError: from python.runfiles import runfiles - path = runfiles.Create().Rlocation(self.BUILD_DATA_FILE) - with open(path) as fp: - return fp.read() + rlocation_path = self.BUILD_DATA_FILE + if is_windows(): + rlocation_path = rlocation_path.replace("/", "\\") + path = runfiles.Create().Rlocation(rlocation_path) + if is_windows(): + path = os.path.normpath(path) + try: + # Use utf-8-sig to handle Windows BOM + with open(path, encoding='utf-8-sig') as fp: + return fp.read() + except Exception as exc: + if hasattr(exc, "add_note"): + exc.add_note(f"runfiles lookup path: {rlocation_path}") + exc.add_note(f"exists: {os.path.exists(path)}") + can_read = os.access(path, os.R_OK) + exc.add_note(f"readable: {can_read}") + raise sys.modules["bazel_binary_info"] = BazelBinaryInfoModule("bazel_binary_info") From 0ed388c7c4de408d2d4f78e6219ac459ab604378 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Thu, 19 Feb 2026 15:48:23 -0800 Subject: [PATCH 630/922] fix: build_data_writer.ps1 encoding and ACLs (#3604) Windows Powershell APIs have several quirks that can make a file problematic if typical Linux semantics are assumed * BOM header bytes can be written, which break regular UTF8 parsing * The file can be left open, which prevents later deletion of it * The files can be written with limited permissions, leading to permission errors later. To fix, use alternative Powershell APIs for writing the text file. Also explicitly set permissions on the output. I came across this when Windows starting running the build data tests in non-zip mode, where the original file permissions are kept. --- CHANGELOG.md | 1 + python/private/build_data_writer.ps1 | 28 ++++++++++++++++++++-------- 2 files changed, 21 insertions(+), 8 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 18234744bd..a8735dae80 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -92,6 +92,7 @@ END_UNRELEASED_TEMPLATE fixing missing transitive dependencies when extras contain hyphens (e.g., `sqlalchemy[postgresql-psycopg2binary]`). ([#3587](https://github.com/bazel-contrib/rules_python/issues/3587)) +* (binaries/tests) Stamped build data generated by Windows actions is readable {#v0-0-0-added} ### Added diff --git a/python/private/build_data_writer.ps1 b/python/private/build_data_writer.ps1 index db7a48e676..846399f194 100644 --- a/python/private/build_data_writer.ps1 +++ b/python/private/build_data_writer.ps1 @@ -1,17 +1,29 @@ $OutputPath = $env:OUTPUT - -Add-Content -Path $OutputPath -Value "TARGET $env:TARGET" -Add-Content -Path $OutputPath -Value "CONFIG_MODE $env:CONFIG_MODE" -Add-Content -Path $OutputPath -Value "STAMPED $env:STAMPED" +$Lines = @( + "TARGET $env:TARGET", + "CONFIG_MODE $env:CONFIG_MODE", + "STAMPED $env:STAMPED" +) $VersionFilePath = $env:VERSION_FILE -if (-not [string]::IsNullOrEmpty($VersionFilePath)) { - Get-Content -Path $VersionFilePath | Add-Content -Path $OutputPath +if (-not [string]::IsNullOrEmpty($VersionFilePath) -and (Test-Path $VersionFilePath)) { + $Lines += Get-Content -Path $VersionFilePath } $InfoFilePath = $env:INFO_FILE -if (-not [string]::IsNullOrEmpty($InfoFilePath)) { - Get-Content -Path $InfoFilePath | Add-Content -Path $OutputPath +if (-not [string]::IsNullOrEmpty($InfoFilePath) -and (Test-Path $InfoFilePath)) { + $Lines += Get-Content -Path $InfoFilePath } +# Use .NET to write file to avoid PowerShell encoding/locking quirks +# We use UTF8 without BOM for compatibility with how the bash script writes (and +# what consumers expect). +$Utf8NoBom = New-Object System.Text.UTF8Encoding $False +[System.IO.File]::WriteAllLines($OutputPath, $Lines, $Utf8NoBom) + +$Acl = Get-Acl $OutputPath +$AccessRule = New-Object System.Security.AccessControl.FileSystemAccessRule("Everyone", "Read", "Allow") +$Acl.SetAccessRule($AccessRule) +Set-Acl $OutputPath $Acl + exit 0 From 4864378d4caf12da1b1d35099c08eab0a93de7e1 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Thu, 19 Feb 2026 16:50:03 -0800 Subject: [PATCH 631/922] docs: Fix nearly all xref errors and other doc warnings (#3615) This fixes almost all the missing cross references. A small list of exceptions are kept in the conf.py file. --- CHANGELOG.md | 12 +-- docs/BUILD.bazel | 2 + docs/_includes/py_console_script_binary.md | 12 ++- .../python/config_settings/index.md | 2 +- docs/conf.py | 24 ++++- docs/devguide.md | 1 + docs/environment-variables.md | 2 +- docs/howto/multi-platform-pypi-deps.md | 6 +- docs/pypi/download-workspace.md | 1 - docs/pypi/index.md | 1 + docs/support.md | 3 +- docs/toolchains.md | 5 +- gazelle/docs/annotations.md | 8 +- gazelle/docs/directives.md | 87 ++++++++++++------- python/private/attr_builders.bzl | 2 +- python/private/attributes.bzl | 4 +- python/private/py_binary_rule.bzl | 2 +- python/private/py_executable.bzl | 2 +- python/private/py_library.bzl | 5 +- python/private/py_test_rule.bzl | 2 +- python/private/pypi/extension.bzl | 2 +- python/private/pypi/pkg_aliases.bzl | 2 +- python/private/python.bzl | 8 +- python/private/zipapp/py_zipapp_rule.bzl | 2 +- sphinxdocs/docs/index.md | 2 +- sphinxdocs/docs/sphinx-bzl.md | 2 +- sphinxdocs/docs/starlark-docgen.md | 2 +- sphinxdocs/inventories/bazel_inventory.txt | 3 +- sphinxdocs/src/sphinx_bzl/bzl.py | 64 ++++++++++++-- 29 files changed, 187 insertions(+), 83 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index a8735dae80..9b740e52ef 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -69,7 +69,7 @@ END_UNRELEASED_TEMPLATE `//python/config_setting/...` and the `@platforms` package instead. * (binaries/tests) The `PYTHONBREAKPOINT` environment variable is automatically inherited * (binaries/tests) The {obj}`stamp` attribute now transitions the Bazel builtin - {obj}`--stamp` flag. + {flag}`--stamp` flag. * (pypi) Now the RECORD file patches will follow the quoted or unquoted filenames convention in order to make `pytorch` and friends easier to patch. * (wheel) `py_wheel` no longer expands the input depset during analysis, @@ -100,7 +100,7 @@ END_UNRELEASED_TEMPLATE to add to binaries/tests for custom debuggers. * (binaries/tests) Build information is now included in binaries and tests. Use the `bazel_binary_info` module to access it. The {flag}`--stamp` flag will - add {flag}`--workspace_status` information. + add {obj}`--workspace_status_command` information. * (gazelle) A new directive `python_generate_pyi_deps` has been added. When `true`, a `py_*` target's `pyi_srcs` attribute will be set if any `.pyi` files that are associated with the target's `srcs` are present. @@ -177,7 +177,7 @@ END_UNRELEASED_TEMPLATE to pass the `TOOL_VERSIONS` that include 3.8 toolchains or use the `bzlmod` APIs to add them back. This means any hub `pip.parse` calls that target `3.8` will be ignored from now on. ([#2704](https://github.com/bazel-contrib/rules_python/issues/2704)) - {object}`python.single_version_override`, like: + {bzl:obj}`python.single_version_override`, like: ```starlark python = use_extension("@rules_python//python/extensions:python.bzl", "python") @@ -292,10 +292,10 @@ END_UNRELEASED_TEMPLATE [#2949](https://github.com/bazel-contrib/rules_python/issues/2949) if you run into any problems. With this release we are deprecating {obj}`pip.parse.experimental_target_platforms` and - {obj}`pip_repository.experimental_target_platforms`. For users using `WORKSPACE` and + `pip_repository.experimental_target_platforms`. For users using `WORKSPACE` and vendoring the `requirements.bzl` file, please re-vendor so that downstream is unaffected when the APIs get removed. If you need to customize the way the dependencies get - evaluated, see [our docs](/pypi/download.html#customizing-requires-dist-resolution) on customizing `Requires-Dist` resolution. + evaluated, see [our docs](https://rules-python.readthedocs.io/en/latest/pypi/download.html#customizing-requires-dist-resolution) on customizing `Requires-Dist` resolution. * (toolchains) Added Python versions 3.15.0a1, 3.14.0, 3.13.9, 3.12.12, 3.11.14, 3.10.19, and 3.9.24 from the [20251014] release. * (deps) (bzlmod) Upgraded to `bazel-skylib` version @@ -375,7 +375,7 @@ END_UNRELEASED_TEMPLATE the right wheel when there are multiple wheels for the target platform (e.g. `musllinux_1_1_x86_64` and `musllinux_1_2_x86_64`). If the user wants to set the minimum version for the selection algorithm, use the - {attr}`pip.defaults.whl_platform_tags` attribute to configure that. If + {obj}`pip.default.whl_platform_tags` attribute to configure that. If `musllinux_*_x86_64` is specified, we will choose the lowest available wheel version. Fixes [#3250](https://github.com/bazel-contrib/rules_python/issues/3250). diff --git a/docs/BUILD.bazel b/docs/BUILD.bazel index 632c3dd613..656c06b6e1 100644 --- a/docs/BUILD.bazel +++ b/docs/BUILD.bazel @@ -89,6 +89,7 @@ sphinx_stardocs( "//python:features_bzl", "//python:packaging_bzl", "//python:pip_bzl", + "//python:proto_bzl", "//python:py_binary_bzl", "//python:py_cc_link_params_info_bzl", "//python:py_exec_tools_info_bzl", @@ -111,6 +112,7 @@ sphinx_stardocs( "//python/extensions:python_bzl", "//python/local_toolchains:repos_bzl", "//python/private:attr_builders_bzl", + "//python/private:builders_bzl", "//python/private:builders_util_bzl", "//python/private:py_binary_rule_bzl", "//python/private:py_cc_toolchain_rule_bzl", diff --git a/docs/_includes/py_console_script_binary.md b/docs/_includes/py_console_script_binary.md index cae9f9f2f5..c9686f2eee 100644 --- a/docs/_includes/py_console_script_binary.md +++ b/docs/_includes/py_console_script_binary.md @@ -12,7 +12,8 @@ py_console_script_binary( ) ``` -#### Specifying extra dependencies +:::{rubric} Specifying extra dependencies +::: You can also specify extra dependencies and the exact script name you want to call. This is useful for tools like `flake8`, `pylint`, and `pytest`, which have plugin discovery methods and discover @@ -36,7 +37,8 @@ py_console_script_binary( ) ``` -#### Using a specific Python version +:::{rubric} Using a specific Python version +::: A specific Python version can be forced by passing the desired Python version, e.g. to force Python 3.9: ```starlark @@ -49,7 +51,8 @@ py_console_script_binary( ) ``` -#### Adding a Shebang Line +:::{rubric} Adding a Shebang Line +::: You can specify a shebang line for the generated binary. This is useful for Unix-like systems where the shebang line determines which interpreter is used to execute @@ -69,7 +72,8 @@ Note that to execute via the shebang line, you need to ensure the specified Python interpreter is available in the environment. -#### Using a specific Python Version directly from a Toolchain +:::{rubric} Using a specific Python Version directly from a Toolchain +::: :::{deprecated} 1.1.0 The toolchain-specific `py_binary` and `py_test` symbols are aliases to the regular rules. For example, `load("@python_versions//3.11:defs.bzl", "py_binary")` and `load("@python_versions//3.11:defs.bzl", "py_test")` are deprecated. diff --git a/docs/api/rules_python/python/config_settings/index.md b/docs/api/rules_python/python/config_settings/index.md index d92e7d404f..6197cac173 100644 --- a/docs/api/rules_python/python/config_settings/index.md +++ b/docs/api/rules_python/python/config_settings/index.md @@ -169,7 +169,7 @@ If you need to match a version that isn't present, then you have two options: ) ``` -2. Use {obj}`python.single_override` to re-introduce the desired version so +2. Use {obj}`python.single_version_override` to re-introduce the desired version so that the corresponding `//python/config_setting:is_python_XXX` target is generated. ::: diff --git a/docs/conf.py b/docs/conf.py index 671cf23fba..6a7cfe178f 100644 --- a/docs/conf.py +++ b/docs/conf.py @@ -107,15 +107,33 @@ primary_domain = None # The default is 'py', which we don't make much use of nitpicky = True +# Ignore nitpicks for missing cross-references to external objects. +# These are typically objects that aren't documented or aren't easily linked +# via intersphinx mapping, so we suppress warnings for them to keep the build clean. nitpick_ignore_regex = [ - # External xrefs aren't setup: ignore missing xref warnings - # External xrefs to sphinx isn't setup: ignore missing xref warnings - ("py:.*", "(sphinx|docutils|ast|enum|collections|typing_extensions).*"), + ("py:class", r"docutils\..*"), + ("py:obj", r"sphinx\.util\.docutils\..*"), + ("py:obj", r"sphinx\.util\.docfields\..*"), + ("py:class", r"sphinx\.util\.typing\..*"), + ("py:class", r"sphinx_bzl\.bzl\..*"), + ("py:class", r"typing_extensions\.TypeAlias"), + ("bzl:obj", r":current_py_cc_headers_abi3"), + ("bzl:obj", r":python"), + ("bzl:type", r"T"), + ("bzl:type", r"input_value"), + ("bzl:type", r"DepsetBuilder"), + ("bzl:type", r"RunfilesBuilder"), + ("bzl:type", r"BuiltinPyInfo"), + ("bzl:type", r".*SentinelInfo"), + ("bzl:type", r".*SphinxDocsLibraryInfo"), + ("bzl:type", r".*_SphinxRunInfo"), ] # --- Intersphinx configuration intersphinx_mapping = { + "python": ("https://docs.python.org/3", None), + "sphinx": ("https://www.sphinx-doc.org/en/master", None), "bazel": ("https://bazel.build/", "bazel_inventory.inv"), } diff --git a/docs/devguide.md b/docs/devguide.md index e7870b5733..e88ed7a612 100644 --- a/docs/devguide.md +++ b/docs/devguide.md @@ -117,6 +117,7 @@ to have everything self-documented, we have a special target, of the requirement-updating scripts in sequence in one go. This can be done once per release as we prepare for releases. +(creating-backport-prs)= ## Creating Backport PRs The steps to create a backport PR are: diff --git a/docs/environment-variables.md b/docs/environment-variables.md index 85cb766344..fb48f434cd 100644 --- a/docs/environment-variables.md +++ b/docs/environment-variables.md @@ -29,7 +29,7 @@ The guide on {any}`How to integrate a debugger` :::{versionadded} 1.3.0 ::: :::{versionchanged} 1.7.0 -Support added for {obj}`--bootstrap_impl=system_python`. +Support added for {bzl:flag}`--bootstrap_impl=system_python`. ::: :::: diff --git a/docs/howto/multi-platform-pypi-deps.md b/docs/howto/multi-platform-pypi-deps.md index 61f3f40580..f329ccdf9e 100644 --- a/docs/howto/multi-platform-pypi-deps.md +++ b/docs/howto/multi-platform-pypi-deps.md @@ -48,7 +48,7 @@ Additional dimensions should be appended and separated with an underscore (e.g. `linux_x86_64_musl_cuda12.9_numpy2`). The platform name should not include the Python version. That is handled by -`pip.parse.python_version` separately. +{attr}`pip.parse.python_version` separately. :::{note} The term _platform_ here has nothing to do with Bazel's `platform()` rule. @@ -56,7 +56,7 @@ The term _platform_ here has nothing to do with Bazel's `platform()` rule. #### Defining custom settings -Because {obj}`pip.parse.config_settings` is a list of arbitrary `config_setting` +Because {attr}`pip.default.config_settings` is a list of arbitrary `config_setting` targets, you can define your own flags or implement custom config matching logic. This allows you to model settings that aren't inherently part of rules_python. @@ -85,7 +85,7 @@ contains commonly used settings for OS and CPU: * `@platforms//cpu:aarch64` Note that these are the raw flag names. In order to use them with `pip.default`, -you must use {obj}`config_setting()` to match a particular value for them. +you must use {obj}`config_setting` to match a particular value for them. ### Associating Requirements to Platforms diff --git a/docs/pypi/download-workspace.md b/docs/pypi/download-workspace.md index 5dfb0f257a..4912f71103 100644 --- a/docs/pypi/download-workspace.md +++ b/docs/pypi/download-workspace.md @@ -41,7 +41,6 @@ re-executed to pick up a non-hermetic change to your environment (e.g., updating your system `python` interpreter), you can force it to re-execute by running `bazel sync --only [pip_parse name]`. -(per-os-arch-requirements)= ## Requirements for a specific OS/Architecture In some cases, you may need to use different requirements files for different OS and architecture combinations. diff --git a/docs/pypi/index.md b/docs/pypi/index.md index 17928898c5..2d17dd5054 100644 --- a/docs/pypi/index.md +++ b/docs/pypi/index.md @@ -1,6 +1,7 @@ :::{default-domain} bzl ::: +(pypi-dependencies)= # Using PyPI Using PyPI packages (aka "pip install") involves the following main steps: diff --git a/docs/support.md b/docs/support.md index 08147f2a15..2d1e21f128 100644 --- a/docs/support.md +++ b/docs/support.md @@ -25,10 +25,9 @@ patch 1.4, version 1.5 must be patched first. Backports can be requested by [creating an issue with the patch release template][patch-release-issue] or by sending a pull request performing the backport. -See the dev guide for [how to create a backport PR][backport-pr]. +See the dev guide for [how to create a backport PR](creating-backport-prs). [patch-release-issue]: https://github.com/bazelbuild/rules_python/issues/new?template=patch_release_request.md -[backport-pr]: devguide.html#creating-backport-prs ## Supported Bazel Versions diff --git a/docs/toolchains.md b/docs/toolchains.md index 1e1e61fde8..09aaed412b 100644 --- a/docs/toolchains.md +++ b/docs/toolchains.md @@ -852,8 +852,7 @@ environment variable automatically. The `//python/bin:python` target will not. The following targets expose the headers and libraries from the currently selected Python C toolchain: -- {obj}`@rules_python//python/cc:` -`current_py_cc_headers` +- {obj}`@rules_python//python/cc:current_py_cc_headers` - {obj}`@rules_python//python/cc:current_py_cc_headers_abi3` - {obj}`@rules_python//python/cc:current_py_cc_libs` @@ -872,5 +871,5 @@ compatibility across different toolchain configurations. :::{seealso} The _How to get Python headres for C extensions_ how-to guide, and the -{obj}`@rules_python//python/cc` package API documentation. +{obj}`//python/cc:BUILD.bazel` package API documentation. ::: diff --git a/gazelle/docs/annotations.md b/gazelle/docs/annotations.md index b6eb96d2cd..b3f06b9991 100644 --- a/gazelle/docs/annotations.md +++ b/gazelle/docs/annotations.md @@ -22,20 +22,20 @@ def bar(): # gazelle:annotation_name value The annotations are: {.glossary} -[`# gazelle:ignore imports`](#ignore) +[`# gazelle:ignore imports`](#annotation-ignore) : Tells Gazelle to ignore import statements. `imports` is a comma-separated list of imports to ignore. * Default: n/a * Allowed Values: A comma-separated string of python package names -[`# gazelle:include_dep targets`](#include-dep) +[`# gazelle:include_dep targets`](#annotation-include-dep) : Tells Gazelle to include a set of dependencies, even if they are not imported in a Python module. `targets` is a comma-separated list of target names to include as dependencies. * Default: n/a * Allowed Values: A comma-separated string of targets -[`# gazelle:include_pytest_conftest bool`](#include-pytest-conftest) +[`# gazelle:include_pytest_conftest bool`](#annotation-include-pytest-conftest) : Whether or not to include a sibling `:conftest` target in the `deps` of a {bzl:obj}`py_test` target. The default behaviour is to include `:conftest` (i.e.: `# gazelle:include_pytest_conftest true`). @@ -43,6 +43,7 @@ The annotations are: * Allowed Values: `true`, `false` +(annotation-ignore)= ## `ignore` This annotation accepts a comma-separated string of values. Values are names of @@ -75,6 +76,7 @@ deps = ["@pypi//numpy"], ``` +(annotation-include-dep)= ## `include_dep` This annotation accepts a comma-separated string of values. Values _must_ diff --git a/gazelle/docs/directives.md b/gazelle/docs/directives.md index 628dce5ae6..cb2bb4929a 100644 --- a/gazelle/docs/directives.md +++ b/gazelle/docs/directives.md @@ -16,58 +16,58 @@ the Python-specific directives in use can be found in the The Python-specific directives are: {.glossary} -[`# gazelle:python_extension value`](#python-extension) +[`# gazelle:python_extension value`](#directive-python-extension) : Controls whether the Python extension is enabled or not. Sub-packages inherit this value. * Default: `enabled` * Allowed Values: `enabled`, `disabled` -[`# gazelle:python_root`](#python-root) +[`# gazelle:python_root`](#directive-python-root) : Sets a Bazel package as a Python root. This is used on monorepos with multiple Python projects that don't share the top-level of the workspace as the root. * Default: n/a * Allowed Values: None. This direcive does not consume values. -[`# gazelle:python_manifest_file_name value`](#python-manifest-file-name) +[`# gazelle:python_manifest_file_name value`](#directive-python-manifest-file-name) : Overrides the default manifest file name. * Default: `gazelle_python.yaml` * Allowed Values: A string -[`# gazelle:python_ignore_files value`](#python-ignore-files) +[`# gazelle:python_ignore_files value`](#directive-python-ignore-files) : Controls the files which are ignored from the generated targets. * Default: n/a * Allowed Values: A comma-separated list of strings. -[`# gazelle:python_ignore_dependencies value`](#python-ignore-dependencies) +[`# gazelle:python_ignore_dependencies value`](#directive-python-ignore-dependencies) : Controls the ignored dependencies from the generated targets. * Default: n/a * Allowed Values: A comma-separated list of strings. -[`# gazelle:python_validate_import_statements bool`](#python-validate-import-statements) +[`# gazelle:python_validate_import_statements bool`](#directive-python-validate-import-statements) : Controls whether the Python import statements should be validated. * Default: `true` * Allowed Values: `true`, `false` -[`# gazelle:python_generation_mode value`](#python-generation-mode) +[`# gazelle:python_generation_mode value`](#directive-python-generation-mode) : Controls the target generation mode. * Default: `package` * Allowed Values: `file`, `package`, `project` -[`# gazelle:python_generation_mode_per_file_include_init bool`](#python-generation-mode-per-file-include-init) +[`# gazelle:python_generation_mode_per_file_include_init bool`](#directive-python-generation-mode-per-file-include-init) : Controls whether `__init__.py` files are included as srcs in each generated target when target generation mode is "file". * Default: `false` * Allowed Values: `true`, `false` -[`# gazelle:python_generation_mode_per_package_require_test_entry_point bool`](python-generation-mode-per-package-require-test-entry-point) +[`# gazelle:python_generation_mode_per_package_require_test_entry_point bool`](#directive-python-generation-mode-per-package-require-test-entry-point) : Controls whether a file called `__test__.py` or a target called `__test__` is required to generate one test target per package in package mode. * Default: `true` * Allowed Values: `true`, `false` -[`# gazelle:python_library_naming_convention value`](#python-library-naming-convention) +[`# gazelle:python_library_naming_convention value`](#directive-python-library-naming-convention) : Controls the {bzl:obj}`py_library` naming convention. It interpolates `$package_name$` with the Bazel package name. E.g. if the Bazel package name is `foo`, setting this to `$package_name$_my_lib` would result in a @@ -75,27 +75,27 @@ The Python-specific directives are: * Default: `$package_name$` * Allowed Values: A string containing `"$package_name$"` -[`# gazelle:python_binary_naming_convention value`](#python-binary-naming-convention) +[`# gazelle:python_binary_naming_convention value`](#directive-python-binary-naming-convention) : Controls the {bzl:obj}`py_binary` naming convention. Follows the same interpolation rules as `python_library_naming_convention`. * Default: `$package_name$_bin` * Allowed Values: A string containing `"$package_name$"` -[`# gazelle:python_test_naming_convention value`](#python-test-naming-convention) +[`# gazelle:python_test_naming_convention value`](#directive-python-test-naming-convention) : Controls the {bzl:obj}`py_test` naming convention. Follows the same interpolation rules as `python_library_naming_convention`. * Default: `$package_name$_test` * Allowed Values: A string containing `"$package_name$"` -[`# gazelle:python_proto_naming_convention value`](#python-proto-naming-convention) +[`# gazelle:python_proto_naming_convention value`](#directive-python-proto-naming-convention) : Controls the {bzl:obj}`py_proto_library` naming convention. It interpolates - `$proto_name$` with the {bzl:obj}`proto_library` rule name, minus any trailing - `_proto`. E.g. if the {bzl:obj}`proto_library` name is `foo_proto`, setting this + `$proto_name$` with the `proto_library` rule name, minus any trailing + `_proto`. E.g. if the `proto_library` name is `foo_proto`, setting this to `$proto_name$_my_lib` would render to `foo_my_lib`. * Default: `$proto_name$_py_pb2` * Allowed Values: A string containing `"$proto_name$"` -[`# gazelle:resolve py import-lang import-string label`](#resolve-py) +[`# gazelle:resolve py import-lang import-string label`](#directive-resolve-py) : Instructs the plugin what target to add as a dependency to satisfy a given import statement. The syntax is `# gazelle:resolve py import-string label` where `import-string` is the symbol in the python `import` statement, @@ -103,25 +103,25 @@ The Python-specific directives are: * Default: n/a * Allowed Values: See the [bazel-gazelle docs][gazelle-directives] -[`# gazelle:python_default_visibility labels`](python-default-visibility) +[`# gazelle:python_default_visibility labels`](#directive-python-default-visibility) : Instructs gazelle to use these visibility labels on all python targets. `labels` is a comma-separated list of labels (without spaces). * Default: `//$python_root$:__subpackages__` * Allowed Values: A string -[`# gazelle:python_visibility label`](python-visibility) +[`# gazelle:python_visibility label`](#directive-python-visibility) : Appends additional visibility labels to each generated target. This r directive can be set multiple times. * Default: n/a * Allowed Values: A string -[`# gazelle:python_test_file_pattern value`](python-test-file-pattern) +[`# gazelle:python_test_file_pattern value`](#directive-python-test-file-pattern) : Filenames matching these comma-separated {command}`glob`s will be mapped to {bzl:obj}`py_test` targets. * Default: `*_test.py,test_*.py` * Allowed Values: A glob string -[`# gazelle:python_label_convention value`](#python-label-convention) +[`# gazelle:python_label_convention value`](#directive-python-label-convention) : Defines the format of the distribution name in labels to third-party deps. Useful for using Gazelle plugin with other rules with different repository conventions (e.g. `rules_pycross`). Full label is always prepended with @@ -131,20 +131,20 @@ The Python-specific directives are: * Default: `$distribution_name$` * Allowed Values: A string -[`# gazelle:python_label_normalization value`](#python-label-normalization) +[`# gazelle:python_label_normalization value`](#directive-python-label-normalization) : Controls how distribution names in labels to third-party deps are normalized. Useful for using Gazelle plugin with other rules with different label conventions (e.g. `rules_pycross` uses PEP-503). * Default: `snake_case` * Allowed Values: `snake_case`, `none`, `pep503` -[`# gazelle:python_experimental_allow_relative_imports bool`](#python-experimental-allow-relative-imports) +[`# gazelle:python_experimental_allow_relative_imports bool`](#directive-python-experimental-allow-relative-imports) : Controls whether Gazelle resolves dependencies for import statements that use paths relative to the current package. * Default: `false` * Allowed Values: `true`, `false` -[`# gazelle:python_generate_pyi_deps bool`](#python-generate-pyi-deps) +[`# gazelle:python_generate_pyi_deps bool`](#directive-python-generate-pyi-deps) : Controls whether to generate a separate `pyi_deps` attribute for type-checking dependencies or merge them into the regular `deps` attribute. When `false` (default), type-checking dependencies are @@ -155,32 +155,33 @@ The Python-specific directives are: * Default: `false` * Allowed Values: `true`, `false` -[`# gazelle:python_generate_pyi_srcs bool`](#python-generate-pyi-srcs) +[`# gazelle:python_generate_pyi_srcs bool`](#directive-python-generate-pyi-srcs) : Controls whether to generate a `pyi_srcs` attribute if a sibling `.pyi` file is found. When `false` (default), the `pyi_srcs` attribute is not added. * Default: `false` * Allowed Values: `true`, `false` -[`# gazelle:python_generate_proto bool`](#python-generate-proto) +[`# gazelle:python_generate_proto bool`](#directive-python-generate-proto) : Controls whether to generate a {bzl:obj}`py_proto_library` for each - {bzl:obj}`proto_library` in the package. By default we load this rule from the + `proto_library` in the package. By default we load this rule from the `@protobuf` repository; use `gazelle:map_kind` if you need to load this from somewhere else. * Default: `false` * Allowed Values: `true`, `false` -[`# gazelle:python_resolve_sibling_imports bool`](#python-resolve-sibling-imports) +[`# gazelle:python_resolve_sibling_imports bool`](#directive-python-resolve-sibling-imports) : Allows absolute imports to be resolved to sibling modules (Python 2's behavior without `absolute_import`). * Default: `false` * Allowed Values: `true`, `false` -[`# gazelle:python_include_ancestor_conftest bool`](#python-include-ancestor-conftest) +[`# gazelle:python_include_ancestor_conftest bool`](#directive-python-include-ancestor-conftest) : Controls whether ancestor conftest targets are added to {bzl:obj}`py_test` target dependencies. * Default: `true` * Allowed Values: `true`, `false` +(directive-python-extension)= ## `python_extension` :::{error} @@ -188,6 +189,7 @@ Detailed docs are not yet written. ::: +(directive-python-root)= ## `python_root` Set this directive within the Bazel package that you want to use as the Python root. @@ -224,6 +226,7 @@ py_libary( [python-packaging-user-guide]: https://github.com/pypa/packaging.python.org/blob/4c86169a/source/tutorials/packaging-projects.rst +(directive-python-manifest-file-name)= ## `python_manifest_file_name` :::{error} @@ -231,6 +234,7 @@ Detailed docs are not yet written. ::: +(directive-python-ignore-files)= ## `python_ignore_files` :::{error} @@ -238,6 +242,7 @@ Detailed docs are not yet written. ::: +(directive-python-ignore-dependencies)= ## `python_ignore_dependencies` :::{error} @@ -245,6 +250,7 @@ Detailed docs are not yet written. ::: +(directive-python-validate-import-statements)= ## `python_validate_import_statements` :::{error} @@ -252,6 +258,7 @@ Detailed docs are not yet written. ::: +(directive-python-generation-mode)= ## `python_generation_mode` :::{error} @@ -259,6 +266,7 @@ Detailed docs are not yet written. ::: +(directive-python-generation-mode-per-file-include-init)= ## `python_generation_mode_per_file_include_init` :::{error} @@ -266,6 +274,7 @@ Detailed docs are not yet written. ::: +(directive-python-generation-mode-per-package-require-test-entry-point)= ## `python_generation_mode_per_package_require_test_entry_point` When `# gazelle:python_generation_mode package`, whether a file called @@ -307,6 +316,7 @@ def py_test(name, main=None, **kwargs): ``` +(directive-python-library-naming-convention)= ## `python_library_naming_convention` :::{error} @@ -314,6 +324,7 @@ Detailed docs are not yet written. ::: +(directive-python-binary-naming-convention)= ## `python_binary_naming_convention` :::{error} @@ -321,6 +332,7 @@ Detailed docs are not yet written. ::: +(directive-python-test-naming-convention)= ## `python_test_naming_convention` :::{error} @@ -328,12 +340,13 @@ Detailed docs are not yet written. ::: +(directive-python-proto-naming-convention)= ## `python_proto_naming_convention` Set this directive to a string pattern to control how the generated {bzl:obj}`py_proto_library` targets are named. When generating new {bzl:obj}`py_proto_library` rules, Gazelle will replace `$proto_name$` in the -pattern with the name of the {bzl:obj}`proto_library` rule, stripping out a +pattern with the name of the `proto_library` rule, stripping out a trailing `_proto`. For example: ```starlark @@ -369,6 +382,7 @@ not able to map said imports, e.g. `import foo_pb2`, to fill in {gh-issue}`1703`. +(directive-resolve-py)= ## `resolve py` :::{error} @@ -376,6 +390,7 @@ Detailed docs are not yet written. ::: +(directive-python-default-visibility)= ## `python_default_visibility` Instructs gazelle to use these visibility labels on all _python_ targets @@ -451,6 +466,7 @@ py_library( These special values can be useful for sub-packages. +(directive-python-visibility)= ## `python_visibility` Appends additional `visibility` labels to each generated target. @@ -508,6 +524,7 @@ py_library( ``` +(directive-python-test-file-pattern)= ## `python_test_file_pattern` This directive adjusts which python files will be mapped to the {bzl:obj}`py_test` rule. @@ -570,6 +587,7 @@ py_library( ``` +(directive-python-label-convention)= ## `python_label_convention` :::{error} @@ -577,6 +595,7 @@ Detailed docs are not yet written. ::: +(directive-python-label-normalization)= ## `python_label_normalization` :::{error} @@ -584,6 +603,7 @@ Detailed docs are not yet written. ::: +(directive-python-experimental-allow-relative-imports)= ## `python_experimental_allow_relative_imports` Enables experimental support for resolving relative imports in @@ -630,6 +650,7 @@ If the directive is set to `true`, gazelle will resolve imports that are relative to the current package. +(directive-python-generate-pyi-deps)= ## `python_generate_pyi_deps` :::{error} @@ -637,7 +658,8 @@ Detailed docs are not yet written. ::: -## `python_generate_pyi_deps` +(directive-python-generate-pyi-srcs)= +## `python_generate_pyi_srcs` When `true`, include any sibling `.pyi` files in the `pyi_srcs` target attribute. @@ -659,10 +681,11 @@ py_library( ``` +(directive-python-generate-proto)= ## `python_generate_proto` When `# gazelle:python_generate_proto true`, Gazelle will generate one -{bzl:obj}`py_proto_library` for each {bzl:obj}`proto_library`, generating Python clients for +{bzl:obj}`py_proto_library` for each `proto_library`, generating Python clients for protobuf in each package. By default this is turned off. Gazelle will also generate a load statement for the {bzl:obj}`py_proto_library` - attempting to detect the configured name for the `@protobuf` / `@com_google_protobuf` repo in your @@ -720,12 +743,14 @@ When `false`, Gazelle will ignore any {bzl:obj}`py_proto_library`, including previously-generated or hand-created rules. +(directive-python-resolve-sibling-imports)= ## `python_resolve_sibling_imports` :::{error} Detailed docs are not yet written. ::: +(directive-python-include-ancestor-conftest)= ## `python_include_ancestor_conftest` Version VERSION_NEXT_FEATURE includes a fix ({gh-pr}`3498`) for a long-standing issue diff --git a/python/private/attr_builders.bzl b/python/private/attr_builders.bzl index ecfc570a2b..8e8a690b8f 100644 --- a/python/private/attr_builders.bzl +++ b/python/private/attr_builders.bzl @@ -592,7 +592,7 @@ def _Label_new(**kwargs): """Creates a builder for `attr.label`. Args: - **kwargs: The same as {obj}`attr.label()`. + **kwargs: The same as {obj}`attr.label`. Returns: {type}`Label` diff --git a/python/private/attributes.bzl b/python/private/attributes.bzl index 362eee8f2e..77b815ec60 100644 --- a/python/private/attributes.bzl +++ b/python/private/attributes.bzl @@ -392,7 +392,7 @@ obtained by calling `str(Label(...))`). Most `@rules_python//python/config_setting` settings can be used here, which allows, for example, making only a certain `py_binary` use -{obj}`--boostrap_impl=script`. +{obj}`--bootstrap_impl=script`. Additional or custom config settings can be registered using the {obj}`add_transition_setting` API. This allows, for example, forcing a @@ -464,7 +464,7 @@ See the [Accessing build information docs] for more information. Stamping can harm build performance by reducing cache hits and should be avoided if possible. -In addition, this transitions the {obj}`--stamp` flag, which can additional +In addition, this transitions the {flag}`--stamp ` flag, which can additional config state overhead. ::: diff --git a/python/private/py_binary_rule.bzl b/python/private/py_binary_rule.bzl index 3df6bd87c4..356af5d9f8 100644 --- a/python/private/py_binary_rule.bzl +++ b/python/private/py_binary_rule.bzl @@ -38,7 +38,7 @@ def create_py_binary_rule_builder(): ::: Returns: - {type}`ruleb.Rule` with the necessary settings + {obj}`ruleb.Rule` with the necessary settings for creating a `py_binary` rule. """ builder = create_executable_rule_builder( diff --git a/python/private/py_executable.bzl b/python/private/py_executable.bzl index 7ff6278e02..19f71c3fae 100644 --- a/python/private/py_executable.bzl +++ b/python/private/py_executable.bzl @@ -1814,7 +1814,7 @@ def create_executable_rule_builder(implementation, **kwargs): ::: Returns: - {type}`ruleb.Rule` with the necessary settings + {obj}`ruleb.Rule` with the necessary settings for creating an executable Python rule. """ builder = ruleb.Rule( diff --git a/python/private/py_library.bzl b/python/private/py_library.bzl index a7419a6eaf..3d8ba4aa71 100644 --- a/python/private/py_library.bzl +++ b/python/private/py_library.bzl @@ -105,10 +105,9 @@ and that only one package version will be included. doc = """ Files whose directories are namespace packages. -When {obj}`--venv_site_packages=yes` is set, this helps inform which directories should be +When {obj}`--venvs_site_packages=yes` is set, this helps inform which directories should be treated as namespace packages and expect files from other targets to be contributed. This allows optimizing the generation of symlinks to be cheaper at analysis time. - :::{versionadded} 1.8.0 ::: """, @@ -283,7 +282,7 @@ def create_py_library_rule_builder(): ::: Returns: - {type}`ruleb.Rule` with the necessary settings + {obj}`ruleb.Rule` with the necessary settings for creating a `py_library` rule. """ builder = ruleb.Rule( diff --git a/python/private/py_test_rule.bzl b/python/private/py_test_rule.bzl index bb35d6974e..5848682deb 100644 --- a/python/private/py_test_rule.bzl +++ b/python/private/py_test_rule.bzl @@ -41,7 +41,7 @@ def create_py_test_rule_builder(): ::: Returns: - {type}`ruleb.Rule` with the necessary settings + {obj}`ruleb.Rule` with the necessary settings for creating a `py_test` rule. """ builder = create_executable_rule_builder( diff --git a/python/private/pypi/extension.bzl b/python/private/pypi/extension.bzl index 7354a67e67..1ec9142bbb 100644 --- a/python/private/pypi/extension.bzl +++ b/python/private/pypi/extension.bzl @@ -823,7 +823,7 @@ the BUILD files for wheels. This tag class allows for more customization of how the configuration for the hub repositories is built. -:::{include} /_includes/experimtal_api.md +:::{include} /_includes/experimental_api.md ::: :::{seealso} diff --git a/python/private/pypi/pkg_aliases.bzl b/python/private/pypi/pkg_aliases.bzl index ac063fac48..e76a139651 100644 --- a/python/private/pypi/pkg_aliases.bzl +++ b/python/private/pypi/pkg_aliases.bzl @@ -181,7 +181,7 @@ def multiplatform_whl_aliases( aliases: {type}`str | dict[struct | str, str]`: The aliases to process. Any aliases that have the filename set will be converted to a dict of config settings to repo names. The - struct is created by {func}`whl_config_setting`. + struct is created by {bzl:obj}`whl_config_setting`. Returns: A dict with of config setting labels to repo names or the repo name itself. diff --git a/python/private/python.bzl b/python/private/python.bzl index 399743c18d..5eb24830cc 100644 --- a/python/private/python.bzl +++ b/python/private/python.bzl @@ -1249,7 +1249,7 @@ The values should be one of the values in `@platforms//cpu` Docs for [Registering custom runtimes] ::: -:::{{versionadded}} 1.5.0 +:::{versionadded} 1.5.0 ::: """, ), @@ -1274,7 +1274,7 @@ The values should be one of the values in `@platforms//os` Docs for [Registering custom runtimes] ::: -:::{{versionadded}} 1.5.0 +:::{versionadded} 1.5.0 ::: """, ), @@ -1329,7 +1329,7 @@ If set, `target_settings`, `os_name`, and `arch` should also be set. Docs for [Registering custom runtimes] ::: -:::{{versionadded}} 1.5.0 +:::{versionadded} 1.5.0 ::: """, ), @@ -1343,7 +1343,7 @@ If set, `target_compatible_with`, `os_name`, and `arch` should also be set. Docs for [Registering custom runtimes] ::: -:::{{versionadded}} 1.5.0 +:::{versionadded} 1.5.0 ::: """, ), diff --git a/python/private/zipapp/py_zipapp_rule.bzl b/python/private/zipapp/py_zipapp_rule.bzl index e97e1a4171..72adbf48ba 100644 --- a/python/private/zipapp/py_zipapp_rule.bzl +++ b/python/private/zipapp/py_zipapp_rule.bzl @@ -266,7 +266,7 @@ obtained by calling `str(Label(...))`). Most `@rules_python//python/config_setting` settings can be used here, which allows, for example, making only a certain `py_binary` use -{obj}`--boostrap_impl=script`. +{obj}`--bootstrap_impl=script`. Additional or custom config settings can be registered using the {obj}`add_transition_setting` API. This allows, for example, forcing a diff --git a/sphinxdocs/docs/index.md b/sphinxdocs/docs/index.md index 2ea1146e1b..43bbc14f60 100644 --- a/sphinxdocs/docs/index.md +++ b/sphinxdocs/docs/index.md @@ -11,7 +11,7 @@ documentation. It comes with: While it is primarily oriented towards docgen for Starlark code, the core of it is agnostic as to what is being documented. -### Optimization +## Optimization Normally, Sphinx keeps various cache files to improve incremental building. Unfortunately, programs performing their own caching don't interact well diff --git a/sphinxdocs/docs/sphinx-bzl.md b/sphinxdocs/docs/sphinx-bzl.md index 8376f60679..da4cd9f293 100644 --- a/sphinxdocs/docs/sphinx-bzl.md +++ b/sphinxdocs/docs/sphinx-bzl.md @@ -265,7 +265,7 @@ Documents a flag. It has the same format as `{bzl:target}` ::::::{rst:directive} .. bzl:typedef:: typename Documents a user-defined structural "type". These are typically generated by -the {obj}`sphinx_stardoc` rule after following [User-defined types] to create a +the {bzl:obj}`sphinx_stardoc` rule after following [User-defined types] to create a struct with a `TYPEDEF` field, but can also be manually defined if there's no natural place for it in code, e.g. some ad-hoc structural type. diff --git a/sphinxdocs/docs/starlark-docgen.md b/sphinxdocs/docs/starlark-docgen.md index ba4ab516f5..b9181ee347 100644 --- a/sphinxdocs/docs/starlark-docgen.md +++ b/sphinxdocs/docs/starlark-docgen.md @@ -81,7 +81,7 @@ still possible to create such objects using `struct` and lambdas. For the purposes of documentation, they can be documented by creating a module-level `struct` with matching fields *and* also a field named `TYPEDEF`. When the `sphinx_stardoc` rule sees a struct with a `TYPEDEF` field, it generates doc -using the {rst:directive}`bzl:typedef` directive and puts all the struct's fields +using the {rst:dir}`bzl:typedef` directive and puts all the struct's fields within the typedef. The net result is the rendered docs look similar to how a class would be documented in other programming languages. diff --git a/sphinxdocs/inventories/bazel_inventory.txt b/sphinxdocs/inventories/bazel_inventory.txt index e704d20d73..866be33886 100644 --- a/sphinxdocs/inventories/bazel_inventory.txt +++ b/sphinxdocs/inventories/bazel_inventory.txt @@ -42,6 +42,7 @@ config.target bzl:function 1 rules/lib/toplevel/config#target - config_common.FeatureFlagInfo bzl:type 1 rules/lib/toplevel/config_common#FeatureFlagInfo - config_common.toolchain_type bzl:function 1 rules/lib/toplevel/config_common#toolchain_type - config_setting bzl:rule 1 reference/be/general#config_setting - +configuration_field bzl:type 1 rules/lib/builtins/configuration_field - ctx bzl:type 1 rules/lib/builtins/repository_ctx - ctx.actions bzl:obj 1 rules/lib/builtins/ctx#actions - ctx.aspect_ids bzl:obj 1 rules/lib/builtins/ctx#aspect_ids - @@ -172,4 +173,4 @@ toolchain.target_settings bzl:attr 1 reference/be/platforms-and-toolchains#toolc toolchain_type bzl:type 1 rules/lib/builtins/toolchain_type.html - transition bzl:type 1 rules/lib/builtins/transition - tuple bzl:type 1 rules/lib/core/tuple - -workspace_status bzl:flag 1 reference/command-line-reference#build-flag--workspace_status_command - +workspace_status_command bzl:flag 1 reference/command-line-reference#build-flag--workspace_status_command - diff --git a/sphinxdocs/src/sphinx_bzl/bzl.py b/sphinxdocs/src/sphinx_bzl/bzl.py index 8303b4d2a5..7d5ec6a68e 100644 --- a/sphinxdocs/src/sphinx_bzl/bzl.py +++ b/sphinxdocs/src/sphinx_bzl/bzl.py @@ -1713,6 +1713,12 @@ def add_object(self, entry: _ObjectEntry, alt_names=None) -> None: ) if entry.full_id in self.data["objects"]: existing = self.data["objects"][entry.full_id] + # If the objects are identical, then it's fine. This can happen + # when a doc is re-parsed (e.g. during a query) or if a symbol + # is documented multiple times in the same file. + if existing.to_get_objects_tuple() == entry.to_get_objects_tuple(): + return + raise Exception( f"Object {entry.full_id} already registered: " + f"existing={existing}, incoming={entry}" @@ -1740,6 +1746,29 @@ def add_object(self, entry: _ObjectEntry, alt_names=None) -> None: self.data["doc_names"].setdefault(docname, {}) self.data["doc_names"][docname][base_name] = entry + @override + def clear_doc(self, docname: str) -> None: + if docname not in self.data["doc_names"]: + return + for base_name, entry in self.data["doc_names"][docname].items(): + if entry.full_id in self.data["objects"]: + del self.data["objects"][entry.full_id] + + if entry.full_id in self.data["objects_by_type"].get( + entry.object_type, {} + ): + del self.data["objects_by_type"][entry.object_type][entry.full_id] + + # We can't easily reverse the mapping for alt_names, so we have + # to iterate over all of them. This is potentially slow, but + # clear_doc isn't called often. + for alt_name, entries in list(self.data["alt_names"].items()): + if entry.full_id in entries: + del entries[entry.full_id] + if not entries: + del self.data["alt_names"][alt_name] + del self.data["doc_names"][docname] + def merge_domaindata( self, docnames: list[str], otherdata: dict[str, typing.Any] ) -> None: @@ -1758,19 +1787,44 @@ def merge_domaindata( def _on_missing_reference(app, env: environment.BuildEnvironment, node, contnode): + """Handle missing references + + There are two main cases this is designed to handle: + 1. Psuedo-types, like None + 2. External references that aren't exact matches, e.g. using + `--stamp=true` to refer to the Bazel builtin stamp flag. + """ + target = node["reftarget"] if node["refdomain"] != "bzl": return None - if node["reftype"] != "type": + if node["reftype"] not in ("type", "flag", "obj"): return None # There's no Bazel docs for None, so prevent missing xrefs warning - if node["reftarget"] == "None": + if target == "None": return contnode - # Any and object are just conventions from Python, but useful for - # indicating what something is in Starlark, so treat them specially. - if node["reftarget"] in ("Any", "object"): + # Useful psuedo-types + if target in ("Any", "object", "collection"): return contnode + + original_target = target + new_target = target + + # Normalize --foo flag references + new_target = new_target.lstrip("-") + # Allow foo() style references + new_target, _, _ = new_target.partition("(") + # Allow --foo=bar references + new_target, _, _ = new_target.partition("=") + + if new_target != original_target: + # Access the intersphinx extension's internal mapping + # we try to resolve the reference again with the stripped name + from sphinx.ext.intersphinx import missing_reference + + node["reftarget"] = new_target + return missing_reference(app, env, node, contnode) return None From a299659a4965e258c8919352e8a32b9164bd19cb Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Fri, 20 Feb 2026 14:42:05 -0800 Subject: [PATCH 632/922] fix: use forward slashes for initial Rlocation lookup of build data (#3616) Gemini flagged that Rlocation accepts forward-slash delimited paths. This appears to be correct, since the internal logic uses forward slashes and manifests contain forward slashes. --- python/private/stage2_bootstrap_template.py | 2 -- 1 file changed, 2 deletions(-) diff --git a/python/private/stage2_bootstrap_template.py b/python/private/stage2_bootstrap_template.py index 5e0472ee1f..bf276f4bcb 100644 --- a/python/private/stage2_bootstrap_template.py +++ b/python/private/stage2_bootstrap_template.py @@ -66,8 +66,6 @@ def get_build_data(self): except ImportError: from python.runfiles import runfiles rlocation_path = self.BUILD_DATA_FILE - if is_windows(): - rlocation_path = rlocation_path.replace("/", "\\") path = runfiles.Create().Rlocation(rlocation_path) if is_windows(): path = os.path.normpath(path) From 1cfed944c64764fd8442a186d385e489e0ad59b4 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Fri, 20 Feb 2026 20:29:42 -0800 Subject: [PATCH 633/922] test: add system_python_nodeps_test (#3609) Adds a simple system_python bootstrap test. Such a test otherwise doesn't directly exist, and only ends up accidentally tested by something in an example directory. --- tests/bootstrap_impls/BUILD.bazel | 9 +++++++++ tests/bootstrap_impls/system_python_nodeps_test.py | 1 + 2 files changed, 10 insertions(+) create mode 100644 tests/bootstrap_impls/system_python_nodeps_test.py diff --git a/tests/bootstrap_impls/BUILD.bazel b/tests/bootstrap_impls/BUILD.bazel index e1f60f5b40..ab3148db00 100644 --- a/tests/bootstrap_impls/BUILD.bazel +++ b/tests/bootstrap_impls/BUILD.bazel @@ -13,6 +13,7 @@ # limitations under the License. load("@rules_pkg//pkg:tar.bzl", "pkg_tar") load("@rules_shell//shell:sh_test.bzl", "sh_test") +load("//python:py_test.bzl", "py_test") load("//tests/support:py_reconfig.bzl", "py_reconfig_binary", "py_reconfig_test") load("//tests/support:sh_py_run_test.bzl", "sh_py_run_test") load("//tests/support:support.bzl", "SUPPORTS_BOOTSTRAP_SCRIPT") @@ -190,4 +191,12 @@ sh_test( }), ) +py_test( + name = "system_python_nodeps_test", + srcs = ["system_python_nodeps_test.py"], + config_settings = { + "//python/config_settings:bootstrap_impl": "system_python", + }, +) + relative_path_test_suite(name = "relative_path_tests") diff --git a/tests/bootstrap_impls/system_python_nodeps_test.py b/tests/bootstrap_impls/system_python_nodeps_test.py new file mode 100644 index 0000000000..7dc46d6e73 --- /dev/null +++ b/tests/bootstrap_impls/system_python_nodeps_test.py @@ -0,0 +1 @@ +print("Hello, world") From 16430a82d1b87ba2625335e3225b4395d4ec39b7 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Fri, 20 Feb 2026 20:30:15 -0800 Subject: [PATCH 634/922] chore: ignore rmtree errors in wheel_installer_test (#3607) When run on Windows with zip mode disabled, the cleanup can fail if the files are still open on Windows. This is an innocuous failure, so just ignore it. --- tests/pypi/whl_installer/wheel_installer_test.py | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/tests/pypi/whl_installer/wheel_installer_test.py b/tests/pypi/whl_installer/wheel_installer_test.py index 7040b0cfd8..91adddf15a 100644 --- a/tests/pypi/whl_installer/wheel_installer_test.py +++ b/tests/pypi/whl_installer/wheel_installer_test.py @@ -63,7 +63,9 @@ def setUp(self) -> None: shutil.copy(os.path.join("examples", "wheel", self.wheel_name), self.wheel_dir) def tearDown(self): - shutil.rmtree(self.wheel_dir) + # On windows, the wheel file remains open, so gives an error upon + # deletion for some reason. + shutil.rmtree(self.wheel_dir, ignore_errors=True) def test_wheel_exists(self) -> None: wheel_installer._extract_wheel( From c9da781ea16cff37a45a03cebfe0a0244861625d Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Fri, 20 Feb 2026 20:30:25 -0800 Subject: [PATCH 635/922] fix: use runfiles_root_path in stage2 bootstrap (#3605) The `%stage2_bootstrap%` template variable was getting a value like `main/../repo/bla.py`. While most APIs would implicitly process the `../` it's somewhat confusing and makes textual string matching (such as when looking in a runfiles manifest) harder. To fix, use runfiles_root_path helper function, which pre-normalizes the path. --- python/private/py_executable.bzl | 5 +---- 1 file changed, 1 insertion(+), 4 deletions(-) diff --git a/python/private/py_executable.bzl b/python/private/py_executable.bzl index 19f71c3fae..66d36e8e8a 100644 --- a/python/private/py_executable.bzl +++ b/python/private/py_executable.bzl @@ -768,10 +768,7 @@ def _create_stage1_bootstrap( } if stage2_bootstrap: - subs["%stage2_bootstrap%"] = "{}/{}".format( - ctx.workspace_name, - stage2_bootstrap.short_path, - ) + subs["%stage2_bootstrap%"] = runfiles_root_path(ctx, stage2_bootstrap.short_path) template = runtime.bootstrap_template subs["%shebang%"] = runtime.stub_shebang elif not ctx.files.srcs: From b6f1b789f0c92f5fdf1fd258e3f36e328ffc5ec7 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Fri, 20 Feb 2026 20:31:51 -0800 Subject: [PATCH 636/922] chore: better bootstrap logging (#3608) Adds additional logging in the python bootstrap. --- python/private/python_bootstrap_template.txt | 15 ++++++++++++++- 1 file changed, 14 insertions(+), 1 deletion(-) diff --git a/python/private/python_bootstrap_template.txt b/python/private/python_bootstrap_template.txt index f2d5a42fda..d40e815038 100644 --- a/python/private/python_bootstrap_template.txt +++ b/python/private/python_bootstrap_template.txt @@ -381,6 +381,13 @@ def Main(): print_verbose("initial cwd:", os.getcwd()) print_verbose("initial environ:", mapping=os.environ) print_verbose("initial sys.path:", values=sys.path) + print_verbose("STAGE2_BOOTSTRAP:", STAGE2_BOOTSTRAP) + print_verbose("PYTHON_BINARY:", PYTHON_BINARY) + print_verbose("PYTHON_BINARY_ACTUAL:", PYTHON_BINARY_ACTUAL) + print_verbose("IS_ZIPFILE:", IS_ZIPFILE) + print_verbose("RECREATE_VENV_AT_RUNTIME:", RECREATE_VENV_AT_RUNTIME) + print_verbose("WORKSPACE_NAME :", WORKSPACE_NAME ) + args = sys.argv[1:] new_env = {} @@ -391,6 +398,7 @@ def Main(): # matters if `_main` doesn't exist (which can occur if a binary # is packaged and needs no artifacts from the main repo) main_rel_path = os.path.normpath(STAGE2_BOOTSTRAP) + print_verbose("main_rel_path:", main_rel_path) if IsRunningFromZip(): module_space = CreateModuleSpace() @@ -399,6 +407,8 @@ def Main(): module_space = FindModuleSpace(main_rel_path) delete_module_space = False + print_verbose("runfiles root:", module_space) + if os.environ.get("RULES_PYTHON_TESTING_TELL_MODULE_SPACE"): new_env["RULES_PYTHON_TESTING_MODULE_SPACE"] = module_space @@ -419,7 +429,10 @@ def Main(): program = python_program = FindPythonBinary(module_space) if python_program is None: - raise AssertionError('Could not find python binary: ' + repr(PYTHON_BINARY)) + raise AssertionError("Could not find python binary: {} or {}".format( + repr(PYTHON_BINARY), + repr(PYTHON_BINARY_ACTUAL) + )) # Some older Python versions on macOS (namely Python 3.7) may unintentionally # leave this environment variable set after starting the interpreter, which From 589c7eea22a11fc455ae2833dde7961c351d3cdb Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Sat, 21 Feb 2026 04:48:35 +0000 Subject: [PATCH 637/922] build(deps): bump the pip group across 2 directories with 2 updates (#3617) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Bumps the pip group with 2 updates in the /tools/publish directory: [cryptography](https://github.com/pyca/cryptography) and [urllib3](https://github.com/urllib3/urllib3). Bumps the pip group with 1 update in the /docs directory: [urllib3](https://github.com/urllib3/urllib3). Updates `cryptography` from 45.0.7 to 46.0.5
Changelog

Sourced from cryptography's changelog.

46.0.5 - 2026-02-10


* An attacker could create a malicious public key that reveals portions
of your
private key when using certain uncommon elliptic curves (binary curves).
This version now includes additional security checks to prevent this
attack.
This issue only affects binary elliptic curves, which are rarely used in
real-world applications. Credit to **XlabAI Team of Tencent Xuanwu Lab
and
Atuin Automated Vulnerability Discovery Engine** for reporting the
issue.
  **CVE-2026-26007**
* Support for ``SECT*`` binary elliptic curves is deprecated and will be
  removed in the next release.

.. v46-0-4:

46.0.4 - 2026-01-27

  • Dropped support for win_arm64 wheels_.
  • Updated Windows, macOS, and Linux wheels to be compiled with OpenSSL 3.5.5.

.. _v46-0-3:

46.0.3 - 2025-10-15


* Fixed compilation when using LibreSSL 4.2.0.

.. _v46-0-2:

46.0.2 - 2025-09-30

  • Updated Windows, macOS, and Linux wheels to be compiled with OpenSSL 3.5.4.

.. _v46-0-1:

46.0.1 - 2025-09-16


* Fixed an issue where users installing via ``pip`` on Python 3.14
development
  versions would not properly install a dependency.
* Fixed an issue building the free-threaded macOS 3.14 wheels.

.. _v46-0-0:

46.0.0 - 2025-09-16

  • BACKWARDS INCOMPATIBLE: Support for Python 3.7 has been removed.

... (truncated)

Commits

Updates `urllib3` from 2.5.0 to 2.6.3
Release notes

Sourced from urllib3's releases.

2.6.3

🚀 urllib3 is fundraising for HTTP/2 support

urllib3 is raising ~$40,000 USD to release HTTP/2 support and ensure long-term sustainable maintenance of the project after a sharp decline in financial support. If your company or organization uses Python and would benefit from HTTP/2 support in Requests, pip, cloud SDKs, and thousands of other projects please consider contributing financially to ensure HTTP/2 support is developed sustainably and maintained for the long-haul.

Thank you for your support.

Changes

  • Fixed a security issue where decompression-bomb safeguards of the streaming API were bypassed when HTTP redirects were followed. (CVE-2026-21441 reported by @​D47A, 8.9 High, GHSA-38jv-5279-wg99)
  • Started treating Retry-After times greater than 6 hours as 6 hours by default. (urllib3/urllib3#3743)
  • Fixed urllib3.connection.VerifiedHTTPSConnection on Emscripten. (urllib3/urllib3#3752)

2.6.2

🚀 urllib3 is fundraising for HTTP/2 support

urllib3 is raising ~$40,000 USD to release HTTP/2 support and ensure long-term sustainable maintenance of the project after a sharp decline in financial support. If your company or organization uses Python and would benefit from HTTP/2 support in Requests, pip, cloud SDKs, and thousands of other projects please consider contributing financially to ensure HTTP/2 support is developed sustainably and maintained for the long-haul.

Thank you for your support.

Changes

  • Fixed HTTPResponse.read_chunked() to properly handle leftover data in the decoder's buffer when reading compressed chunked responses. (urllib3/urllib3#3734)

2.6.1

🚀 urllib3 is fundraising for HTTP/2 support

urllib3 is raising ~$40,000 USD to release HTTP/2 support and ensure long-term sustainable maintenance of the project after a sharp decline in financial support. If your company or organization uses Python and would benefit from HTTP/2 support in Requests, pip, cloud SDKs, and thousands of other projects please consider contributing financially to ensure HTTP/2 support is developed sustainably and maintained for the long-haul.

Thank you for your support.

Changes

  • Restore previously removed HTTPResponse.getheaders() and HTTPResponse.getheader() methods. (#3731)

2.6.0

🚀 urllib3 is fundraising for HTTP/2 support

urllib3 is raising ~$40,000 USD to release HTTP/2 support and ensure long-term sustainable maintenance of the project after a sharp decline in financial support. If your company or organization uses Python and would benefit from HTTP/2 support in Requests, pip, cloud SDKs, and thousands of other projects please consider contributing financially to ensure HTTP/2 support is developed sustainably and maintained for the long-haul.

Thank you for your support.

Security

  • Fixed a security issue where streaming API could improperly handle highly compressed HTTP content ("decompression bombs") leading to excessive resource consumption even when a small amount of data was requested. Reading small chunks of compressed data is safer and much more efficient now. (CVE-2025-66471 reported by @​Cycloctane, 8.9 High, GHSA-2xpw-w6gg-jr37)
  • Fixed a security issue where an attacker could compose an HTTP response with virtually unlimited links in the Content-Encoding header, potentially leading to a denial of service (DoS) attack by exhausting system resources during decoding. The number of allowed chained encodings is now limited to 5. (CVE-2025-66418 reported by @​illia-v, 8.9 High, GHSA-gm62-xv2j-4w53)

[!IMPORTANT]

  • If urllib3 is not installed with the optional urllib3[brotli] extra, but your environment contains a Brotli/brotlicffi/brotlipy package anyway, make sure to upgrade it to at least Brotli 1.2.0 or brotlicffi 1.2.0.0 to benefit from the security fixes and avoid warnings. Prefer using urllib3[brotli] to install a compatible Brotli package automatically.

... (truncated)

Changelog

Sourced from urllib3's changelog.

2.6.3 (2026-01-07)

  • Fixed a high-severity security issue where decompression-bomb safeguards of the streaming API were bypassed when HTTP redirects were followed. (GHSA-38jv-5279-wg99 <https://github.com/urllib3/urllib3/security/advisories/GHSA-38jv-5279-wg99>__)
  • Started treating Retry-After times greater than 6 hours as 6 hours by default. ([#3743](https://github.com/urllib3/urllib3/issues/3743) <https://github.com/urllib3/urllib3/issues/3743>__)
  • Fixed urllib3.connection.VerifiedHTTPSConnection on Emscripten. ([#3752](https://github.com/urllib3/urllib3/issues/3752) <https://github.com/urllib3/urllib3/issues/3752>__)

2.6.2 (2025-12-11)

  • Fixed HTTPResponse.read_chunked() to properly handle leftover data in the decoder's buffer when reading compressed chunked responses. ([#3734](https://github.com/urllib3/urllib3/issues/3734) <https://github.com/urllib3/urllib3/issues/3734>__)

2.6.1 (2025-12-08)

  • Restore previously removed HTTPResponse.getheaders() and HTTPResponse.getheader() methods. ([#3731](https://github.com/urllib3/urllib3/issues/3731) <https://github.com/urllib3/urllib3/issues/3731>__)

2.6.0 (2025-12-05)

Security

  • Fixed a security issue where streaming API could improperly handle highly compressed HTTP content ("decompression bombs") leading to excessive resource consumption even when a small amount of data was requested. Reading small chunks of compressed data is safer and much more efficient now. (GHSA-2xpw-w6gg-jr37 <https://github.com/urllib3/urllib3/security/advisories/GHSA-2xpw-w6gg-jr37>__)
  • Fixed a security issue where an attacker could compose an HTTP response with virtually unlimited links in the Content-Encoding header, potentially leading to a denial of service (DoS) attack by exhausting system resources during decoding. The number of allowed chained encodings is now limited to 5. (GHSA-gm62-xv2j-4w53 <https://github.com/urllib3/urllib3/security/advisories/GHSA-gm62-xv2j-4w53>__)

.. caution::

  • If urllib3 is not installed with the optional urllib3[brotli] extra, but your environment contains a Brotli/brotlicffi/brotlipy package anyway, make sure to upgrade it to at least Brotli 1.2.0 or brotlicffi 1.2.0.0 to benefit from the security fixes and avoid warnings. Prefer using

... (truncated)

Commits
  • 0248277 Release 2.6.3
  • 8864ac4 Merge commit from fork
  • 70cecb2 Fix Scorecard issues related to vulnerable dev dependencies (#3755)
  • 41f249a Move "v2.0 Migration Guide" to the end of the table of contents (#3747)
  • fd4dffd Patch VerifiedHTTPSConnection for Emscripten (#3752)
  • 13f0bfd Handle massive values in Retry-After when calculating time to sleep for (#3743)
  • 8c480bf Bump actions/upload-artifact from 5.0.0 to 6.0.0 (#3748)
  • 4b40616 Bump actions/cache from 4.3.0 to 5.0.1 (#3750)
  • 82b8479 Bump actions/download-artifact from 6.0.0 to 7.0.0 (#3749)
  • 34284cb Mention experimental features in the security policy (#3746)
  • Additional commits viewable in compare view

Updates `urllib3` from 2.5.0 to 2.6.3
Release notes

Sourced from urllib3's releases.

2.6.3

🚀 urllib3 is fundraising for HTTP/2 support

urllib3 is raising ~$40,000 USD to release HTTP/2 support and ensure long-term sustainable maintenance of the project after a sharp decline in financial support. If your company or organization uses Python and would benefit from HTTP/2 support in Requests, pip, cloud SDKs, and thousands of other projects please consider contributing financially to ensure HTTP/2 support is developed sustainably and maintained for the long-haul.

Thank you for your support.

Changes

  • Fixed a security issue where decompression-bomb safeguards of the streaming API were bypassed when HTTP redirects were followed. (CVE-2026-21441 reported by @​D47A, 8.9 High, GHSA-38jv-5279-wg99)
  • Started treating Retry-After times greater than 6 hours as 6 hours by default. (urllib3/urllib3#3743)
  • Fixed urllib3.connection.VerifiedHTTPSConnection on Emscripten. (urllib3/urllib3#3752)

2.6.2

🚀 urllib3 is fundraising for HTTP/2 support

urllib3 is raising ~$40,000 USD to release HTTP/2 support and ensure long-term sustainable maintenance of the project after a sharp decline in financial support. If your company or organization uses Python and would benefit from HTTP/2 support in Requests, pip, cloud SDKs, and thousands of other projects please consider contributing financially to ensure HTTP/2 support is developed sustainably and maintained for the long-haul.

Thank you for your support.

Changes

  • Fixed HTTPResponse.read_chunked() to properly handle leftover data in the decoder's buffer when reading compressed chunked responses. (urllib3/urllib3#3734)

2.6.1

🚀 urllib3 is fundraising for HTTP/2 support

urllib3 is raising ~$40,000 USD to release HTTP/2 support and ensure long-term sustainable maintenance of the project after a sharp decline in financial support. If your company or organization uses Python and would benefit from HTTP/2 support in Requests, pip, cloud SDKs, and thousands of other projects please consider contributing financially to ensure HTTP/2 support is developed sustainably and maintained for the long-haul.

Thank you for your support.

Changes

  • Restore previously removed HTTPResponse.getheaders() and HTTPResponse.getheader() methods. (#3731)

2.6.0

🚀 urllib3 is fundraising for HTTP/2 support

urllib3 is raising ~$40,000 USD to release HTTP/2 support and ensure long-term sustainable maintenance of the project after a sharp decline in financial support. If your company or organization uses Python and would benefit from HTTP/2 support in Requests, pip, cloud SDKs, and thousands of other projects please consider contributing financially to ensure HTTP/2 support is developed sustainably and maintained for the long-haul.

Thank you for your support.

Security

  • Fixed a security issue where streaming API could improperly handle highly compressed HTTP content ("decompression bombs") leading to excessive resource consumption even when a small amount of data was requested. Reading small chunks of compressed data is safer and much more efficient now. (CVE-2025-66471 reported by @​Cycloctane, 8.9 High, GHSA-2xpw-w6gg-jr37)
  • Fixed a security issue where an attacker could compose an HTTP response with virtually unlimited links in the Content-Encoding header, potentially leading to a denial of service (DoS) attack by exhausting system resources during decoding. The number of allowed chained encodings is now limited to 5. (CVE-2025-66418 reported by @​illia-v, 8.9 High, GHSA-gm62-xv2j-4w53)

[!IMPORTANT]

  • If urllib3 is not installed with the optional urllib3[brotli] extra, but your environment contains a Brotli/brotlicffi/brotlipy package anyway, make sure to upgrade it to at least Brotli 1.2.0 or brotlicffi 1.2.0.0 to benefit from the security fixes and avoid warnings. Prefer using urllib3[brotli] to install a compatible Brotli package automatically.

... (truncated)

Changelog

Sourced from urllib3's changelog.

2.6.3 (2026-01-07)

  • Fixed a high-severity security issue where decompression-bomb safeguards of the streaming API were bypassed when HTTP redirects were followed. (GHSA-38jv-5279-wg99 <https://github.com/urllib3/urllib3/security/advisories/GHSA-38jv-5279-wg99>__)
  • Started treating Retry-After times greater than 6 hours as 6 hours by default. ([#3743](https://github.com/urllib3/urllib3/issues/3743) <https://github.com/urllib3/urllib3/issues/3743>__)
  • Fixed urllib3.connection.VerifiedHTTPSConnection on Emscripten. ([#3752](https://github.com/urllib3/urllib3/issues/3752) <https://github.com/urllib3/urllib3/issues/3752>__)

2.6.2 (2025-12-11)

  • Fixed HTTPResponse.read_chunked() to properly handle leftover data in the decoder's buffer when reading compressed chunked responses. ([#3734](https://github.com/urllib3/urllib3/issues/3734) <https://github.com/urllib3/urllib3/issues/3734>__)

2.6.1 (2025-12-08)

  • Restore previously removed HTTPResponse.getheaders() and HTTPResponse.getheader() methods. ([#3731](https://github.com/urllib3/urllib3/issues/3731) <https://github.com/urllib3/urllib3/issues/3731>__)

2.6.0 (2025-12-05)

Security

  • Fixed a security issue where streaming API could improperly handle highly compressed HTTP content ("decompression bombs") leading to excessive resource consumption even when a small amount of data was requested. Reading small chunks of compressed data is safer and much more efficient now. (GHSA-2xpw-w6gg-jr37 <https://github.com/urllib3/urllib3/security/advisories/GHSA-2xpw-w6gg-jr37>__)
  • Fixed a security issue where an attacker could compose an HTTP response with virtually unlimited links in the Content-Encoding header, potentially leading to a denial of service (DoS) attack by exhausting system resources during decoding. The number of allowed chained encodings is now limited to 5. (GHSA-gm62-xv2j-4w53 <https://github.com/urllib3/urllib3/security/advisories/GHSA-gm62-xv2j-4w53>__)

.. caution::

  • If urllib3 is not installed with the optional urllib3[brotli] extra, but your environment contains a Brotli/brotlicffi/brotlipy package anyway, make sure to upgrade it to at least Brotli 1.2.0 or brotlicffi 1.2.0.0 to benefit from the security fixes and avoid warnings. Prefer using

... (truncated)

Commits
  • 0248277 Release 2.6.3
  • 8864ac4 Merge commit from fork
  • 70cecb2 Fix Scorecard issues related to vulnerable dev dependencies (#3755)
  • 41f249a Move "v2.0 Migration Guide" to the end of the table of contents (#3747)
  • fd4dffd Patch VerifiedHTTPSConnection for Emscripten (#3752)
  • 13f0bfd Handle massive values in Retry-After when calculating time to sleep for (#3743)
  • 8c480bf Bump actions/upload-artifact from 5.0.0 to 6.0.0 (#3748)
  • 4b40616 Bump actions/cache from 4.3.0 to 5.0.1 (#3750)
  • 82b8479 Bump actions/download-artifact from 6.0.0 to 7.0.0 (#3749)
  • 34284cb Mention experimental features in the security policy (#3746)
  • Additional commits viewable in compare view

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) ---
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot show ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore major version` will close this group update PR and stop Dependabot creating any more for the specific dependency's major version (unless you unignore this specific dependency's major version or upgrade to it yourself) - `@dependabot ignore minor version` will close this group update PR and stop Dependabot creating any more for the specific dependency's minor version (unless you unignore this specific dependency's minor version or upgrade to it yourself) - `@dependabot ignore ` will close this group update PR and stop Dependabot creating any more for the specific dependency (unless you unignore this specific dependency or upgrade to it yourself) - `@dependabot unignore ` will remove all of the ignore conditions of the specified dependency - `@dependabot unignore ` will remove the ignore condition of the specified dependency and ignore conditions You can disable automated security fix PRs for this repo from the [Security Alerts page](https://github.com/bazel-contrib/rules_python/network/alerts).
Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Richard Levasseur --- docs/requirements.txt | 6 +- tools/publish/requirements_darwin.txt | 6 +- tools/publish/requirements_linux.txt | 94 +++++++++++++----------- tools/publish/requirements_universal.txt | 94 +++++++++++++----------- tools/publish/requirements_windows.txt | 6 +- 5 files changed, 115 insertions(+), 91 deletions(-) diff --git a/docs/requirements.txt b/docs/requirements.txt index c5a5feaae0..05f3db8002 100644 --- a/docs/requirements.txt +++ b/docs/requirements.txt @@ -497,9 +497,9 @@ typing-extensions==4.15.0 \ # rules-python-docs (docs/pyproject.toml) # astroid # sphinx-autodoc2 -urllib3==2.5.0 \ - --hash=sha256:3fc47733c7e419d4bc3f6b3dc2b4f890bb743906a30d56ba4a5bfa4bbff92760 \ - --hash=sha256:e6b01673c0fa6a13e374b50871808eb3bf7046c4b125b216f6bf1cc604cff0dc +urllib3==2.6.3 \ + --hash=sha256:1b62b6884944a57dbe321509ab94fd4d3b307075e0c2eae991ac71ee15ad38ed \ + --hash=sha256:bf272323e553dfb2e87d9bfd225ca7b0f467b919d7bbd355436d3fd37cb0acd4 # via requests zipp==3.23.0 ; python_full_version < '3.10' \ --hash=sha256:071652d6115ed432f5ce1d34c336c0adfd6a884660d1e9712a256d3d3bd4b14e \ diff --git a/tools/publish/requirements_darwin.txt b/tools/publish/requirements_darwin.txt index 5cccf1607f..012c4605ff 100644 --- a/tools/publish/requirements_darwin.txt +++ b/tools/publish/requirements_darwin.txt @@ -199,9 +199,9 @@ twine==5.1.1 \ --hash=sha256:215dbe7b4b94c2c50a7315c0275d2258399280fbb7d04182c7e55e24b5f93997 \ --hash=sha256:9aa0825139c02b3434d913545c7b847a21c835e11597f5255842d457da2322db # via -r tools/publish/requirements.in -urllib3==2.5.0 \ - --hash=sha256:3fc47733c7e419d4bc3f6b3dc2b4f890bb743906a30d56ba4a5bfa4bbff92760 \ - --hash=sha256:e6b01673c0fa6a13e374b50871808eb3bf7046c4b125b216f6bf1cc604cff0dc +urllib3==2.6.3 \ + --hash=sha256:1b62b6884944a57dbe321509ab94fd4d3b307075e0c2eae991ac71ee15ad38ed \ + --hash=sha256:bf272323e553dfb2e87d9bfd225ca7b0f467b919d7bbd355436d3fd37cb0acd4 # via # requests # twine diff --git a/tools/publish/requirements_linux.txt b/tools/publish/requirements_linux.txt index 4adc3339bf..b5e9eb7b59 100644 --- a/tools/publish/requirements_linux.txt +++ b/tools/publish/requirements_linux.txt @@ -177,44 +177,56 @@ charset-normalizer==3.4.3 \ --hash=sha256:fd10de089bcdcd1be95a2f73dbe6254798ec1bda9f450d5828c96f93e2536b9c \ --hash=sha256:fdabf8315679312cfa71302f9bd509ded4f2f263fb5b765cf1433b39106c3cc9 # via requests -cryptography==45.0.7 \ - --hash=sha256:06ce84dc14df0bf6ea84666f958e6080cdb6fe1231be2a51f3fc1267d9f3fb34 \ - --hash=sha256:16ede8a4f7929b4b7ff3642eba2bf79aa1d71f24ab6ee443935c0d269b6bc513 \ - --hash=sha256:18fcf70f243fe07252dcb1b268a687f2358025ce32f9f88028ca5c364b123ef5 \ - --hash=sha256:1993a1bb7e4eccfb922b6cd414f072e08ff5816702a0bdb8941c247a6b1b287c \ - --hash=sha256:1f3d56f73595376f4244646dd5c5870c14c196949807be39e79e7bd9bac3da63 \ - --hash=sha256:258e0dff86d1d891169b5af222d362468a9570e2532923088658aa866eb11130 \ - --hash=sha256:2f641b64acc00811da98df63df7d59fd4706c0df449da71cb7ac39a0732b40ae \ - --hash=sha256:3808e6b2e5f0b46d981c24d79648e5c25c35e59902ea4391a0dcb3e667bf7443 \ - --hash=sha256:3994c809c17fc570c2af12c9b840d7cea85a9fd3e5c0e0491f4fa3c029216d59 \ - --hash=sha256:3be4f21c6245930688bd9e162829480de027f8bf962ede33d4f8ba7d67a00cee \ - --hash=sha256:465ccac9d70115cd4de7186e60cfe989de73f7bb23e8a7aa45af18f7412e75bf \ - --hash=sha256:48c41a44ef8b8c2e80ca4527ee81daa4c527df3ecbc9423c41a420a9559d0e27 \ - --hash=sha256:4a862753b36620af6fc54209264f92c716367f2f0ff4624952276a6bbd18cbde \ - --hash=sha256:4b1654dfc64ea479c242508eb8c724044f1e964a47d1d1cacc5132292d851971 \ - --hash=sha256:4bd3e5c4b9682bc112d634f2c6ccc6736ed3635fc3319ac2bb11d768cc5a00d8 \ - --hash=sha256:577470e39e60a6cd7780793202e63536026d9b8641de011ed9d8174da9ca5339 \ - --hash=sha256:67285f8a611b0ebc0857ced2081e30302909f571a46bfa7a3cc0ad303fe015c6 \ - --hash=sha256:7285a89df4900ed3bfaad5679b1e668cb4b38a8de1ccbfc84b05f34512da0a90 \ - --hash=sha256:81823935e2f8d476707e85a78a405953a03ef7b7b4f55f93f7c2d9680e5e0691 \ - --hash=sha256:8978132287a9d3ad6b54fcd1e08548033cc09dc6aacacb6c004c73c3eb5d3ac3 \ - --hash=sha256:a20e442e917889d1a6b3c570c9e3fa2fdc398c20868abcea268ea33c024c4083 \ - --hash=sha256:a24ee598d10befaec178efdff6054bc4d7e883f615bfbcd08126a0f4931c83a6 \ - --hash=sha256:b04f85ac3a90c227b6e5890acb0edbaf3140938dbecf07bff618bf3638578cf1 \ - --hash=sha256:b6a0e535baec27b528cb07a119f321ac024592388c5681a5ced167ae98e9fff3 \ - --hash=sha256:bef32a5e327bd8e5af915d3416ffefdbe65ed975b646b3805be81b23580b57b8 \ - --hash=sha256:bfb4c801f65dd61cedfc61a83732327fafbac55a47282e6f26f073ca7a41c3b2 \ - --hash=sha256:c13b1e3afd29a5b3b2656257f14669ca8fa8d7956d509926f0b130b600b50ab7 \ - --hash=sha256:c987dad82e8c65ebc985f5dae5e74a3beda9d0a2a4daf8a1115f3772b59e5141 \ - --hash=sha256:ce7a453385e4c4693985b4a4a3533e041558851eae061a58a5405363b098fcd3 \ - --hash=sha256:d0c5c6bac22b177bf8da7435d9d27a6834ee130309749d162b26c3105c0795a9 \ - --hash=sha256:d97cf502abe2ab9eff8bd5e4aca274da8d06dd3ef08b759a8d6143f4ad65d4b4 \ - --hash=sha256:dad43797959a74103cb59c5dac71409f9c27d34c8a05921341fb64ea8ccb1dd4 \ - --hash=sha256:dd342f085542f6eb894ca00ef70236ea46070c8a13824c6bde0dfdcd36065b9b \ - --hash=sha256:de58755d723e86175756f463f2f0bddd45cc36fbd62601228a3f8761c9f58252 \ - --hash=sha256:f3df7b3d0f91b88b2106031fd995802a2e9ae13e02c36c1fc075b43f420f3a17 \ - --hash=sha256:f5414a788ecc6ee6bc58560e85ca624258a55ca434884445440a810796ea0e0b \ - --hash=sha256:fa26fa54c0a9384c27fcdc905a2fb7d60ac6e47d14bc2692145f2b3b1e2cfdbd +cryptography==46.0.5 \ + --hash=sha256:02f547fce831f5096c9a567fd41bc12ca8f11df260959ecc7c3202555cc47a72 \ + --hash=sha256:039917b0dc418bb9f6edce8a906572d69e74bd330b0b3fea4f79dab7f8ddd235 \ + --hash=sha256:1abfdb89b41c3be0365328a410baa9df3ff8a9110fb75e7b52e66803ddabc9a9 \ + --hash=sha256:2ae6971afd6246710480e3f15824ed3029a60fc16991db250034efd0b9fb4356 \ + --hash=sha256:2b7a67c9cd56372f3249b39699f2ad479f6991e62ea15800973b956f4b73e257 \ + --hash=sha256:351695ada9ea9618b3500b490ad54c739860883df6c1f555e088eaf25b1bbaad \ + --hash=sha256:38946c54b16c885c72c4f59846be9743d699eee2b69b6988e0a00a01f46a61a4 \ + --hash=sha256:3b4995dc971c9fb83c25aa44cf45f02ba86f71ee600d81091c2f0cbae116b06c \ + --hash=sha256:3ce58ba46e1bc2aac4f7d9290223cead56743fa6ab94a5d53292ffaac6a91614 \ + --hash=sha256:3ee190460e2fbe447175cda91b88b84ae8322a104fc27766ad09428754a618ed \ + --hash=sha256:4108d4c09fbbf2789d0c926eb4152ae1760d5a2d97612b92d508d96c861e4d31 \ + --hash=sha256:420d0e909050490d04359e7fdb5ed7e667ca5c3c402b809ae2563d7e66a92229 \ + --hash=sha256:47fb8a66058b80e509c47118ef8a75d14c455e81ac369050f20ba0d23e77fee0 \ + --hash=sha256:4c3341037c136030cb46e4b1e17b7418ea4cbd9dd207e4a6f3b2b24e0d4ac731 \ + --hash=sha256:4d7e3d356b8cd4ea5aff04f129d5f66ebdc7b6f8eae802b93739ed520c47c79b \ + --hash=sha256:4d8ae8659ab18c65ced284993c2265910f6c9e650189d4e3f68445ef82a810e4 \ + --hash=sha256:4e817a8920bfbcff8940ecfd60f23d01836408242b30f1a708d93198393a80b4 \ + --hash=sha256:50bfb6925eff619c9c023b967d5b77a54e04256c4281b0e21336a130cd7fc263 \ + --hash=sha256:556e106ee01aa13484ce9b0239bca667be5004efb0aabbed28d353df86445595 \ + --hash=sha256:582f5fcd2afa31622f317f80426a027f30dc792e9c80ffee87b993200ea115f1 \ + --hash=sha256:5be7bf2fb40769e05739dd0046e7b26f9d4670badc7b032d6ce4db64dddc0678 \ + --hash=sha256:60ee7e19e95104d4c03871d7d7dfb3d22ef8a9b9c6778c94e1c8fcc8365afd48 \ + --hash=sha256:61aa400dce22cb001a98014f647dc21cda08f7915ceb95df0c9eaf84b4b6af76 \ + --hash=sha256:68f68d13f2e1cb95163fa3b4db4bf9a159a418f5f6e7242564fc75fcae667fd0 \ + --hash=sha256:7d1f30a86d2757199cb2d56e48cce14deddf1f9c95f1ef1b64ee91ea43fe2e18 \ + --hash=sha256:7d731d4b107030987fd61a7f8ab512b25b53cef8f233a97379ede116f30eb67d \ + --hash=sha256:803812e111e75d1aa73690d2facc295eaefd4439be1023fefc4995eaea2af90d \ + --hash=sha256:80a8d7bfdf38f87ca30a5391c0c9ce4ed2926918e017c29ddf643d0ed2778ea1 \ + --hash=sha256:8293f3dea7fc929ef7240796ba231413afa7b68ce38fd21da2995549f5961981 \ + --hash=sha256:8456928655f856c6e1533ff59d5be76578a7157224dbd9ce6872f25055ab9ab7 \ + --hash=sha256:890bcb4abd5a2d3f852196437129eb3667d62630333aacc13dfd470fad3aaa82 \ + --hash=sha256:94a76daa32eb78d61339aff7952ea819b1734b46f73646a07decb40e5b3448e2 \ + --hash=sha256:9f16fbdf4da055efb21c22d81b89f155f02ba420558db21288b3d0035bafd5f4 \ + --hash=sha256:a3d1fae9863299076f05cb8a778c467578262fae09f9dc0ee9b12eb4268ce663 \ + --hash=sha256:a3d507bb6a513ca96ba84443226af944b0f7f47dcc9a399d110cd6146481d24c \ + --hash=sha256:abace499247268e3757271b2f1e244b36b06f8515cf27c4d49468fc9eb16e93d \ + --hash=sha256:ba2a27ff02f48193fc4daeadf8ad2590516fa3d0adeeb34336b96f7fa64c1e3a \ + --hash=sha256:bc84e875994c3b445871ea7181d424588171efec3e185dced958dad9e001950a \ + --hash=sha256:bfd56bb4b37ed4f330b82402f6f435845a5f5648edf1ad497da51a8452d5d62d \ + --hash=sha256:c18ff11e86df2e28854939acde2d003f7984f721eba450b56a200ad90eeb0e6b \ + --hash=sha256:c3bcce8521d785d510b2aad26ae2c966092b7daa8f45dd8f44734a104dc0bc1a \ + --hash=sha256:c4143987a42a2397f2fc3b4d7e3a7d313fbe684f67ff443999e803dd75a76826 \ + --hash=sha256:c69fd885df7d089548a42d5ec05be26050ebcd2283d89b3d30676eb32ff87dee \ + --hash=sha256:ced80795227d70549a411a4ab66e8ce307899fad2220ce5ab2f296e687eacde9 \ + --hash=sha256:d66e421495fdb797610a08f43b05269e0a5ea7f5e652a89bfd5a7d3c1dee3648 \ + --hash=sha256:d861ee9e76ace6cf36a6a89b959ec08e7bc2493ee39d07ffe5acb23ef46d27da \ + --hash=sha256:e9251e3be159d1020c4030bd2e5f84d6a43fe54b6c19c12f51cde9542a2817b2 \ + --hash=sha256:f145bba11b878005c496e93e257c1e88f154d278d2638e6450d17e0f31e558d2 \ + --hash=sha256:fe346b143ff9685e40192a4960938545c699054ba11d4f9029f94751e3f71d87 # via secretstorage docutils==0.22.2 \ --hash=sha256:9fdb771707c8784c8f2728b67cb2c691305933d68137ef95a75db5f4dfbc213d \ @@ -338,9 +350,9 @@ twine==5.1.1 \ --hash=sha256:215dbe7b4b94c2c50a7315c0275d2258399280fbb7d04182c7e55e24b5f93997 \ --hash=sha256:9aa0825139c02b3434d913545c7b847a21c835e11597f5255842d457da2322db # via -r tools/publish/requirements.in -urllib3==2.5.0 \ - --hash=sha256:3fc47733c7e419d4bc3f6b3dc2b4f890bb743906a30d56ba4a5bfa4bbff92760 \ - --hash=sha256:e6b01673c0fa6a13e374b50871808eb3bf7046c4b125b216f6bf1cc604cff0dc +urllib3==2.6.3 \ + --hash=sha256:1b62b6884944a57dbe321509ab94fd4d3b307075e0c2eae991ac71ee15ad38ed \ + --hash=sha256:bf272323e553dfb2e87d9bfd225ca7b0f467b919d7bbd355436d3fd37cb0acd4 # via # requests # twine diff --git a/tools/publish/requirements_universal.txt b/tools/publish/requirements_universal.txt index fc765f72f7..92e673ba51 100644 --- a/tools/publish/requirements_universal.txt +++ b/tools/publish/requirements_universal.txt @@ -160,44 +160,56 @@ charset-normalizer==3.4.3 \ --hash=sha256:fd10de089bcdcd1be95a2f73dbe6254798ec1bda9f450d5828c96f93e2536b9c \ --hash=sha256:fdabf8315679312cfa71302f9bd509ded4f2f263fb5b765cf1433b39106c3cc9 # via requests -cryptography==45.0.7 ; sys_platform == 'linux' \ - --hash=sha256:06ce84dc14df0bf6ea84666f958e6080cdb6fe1231be2a51f3fc1267d9f3fb34 \ - --hash=sha256:16ede8a4f7929b4b7ff3642eba2bf79aa1d71f24ab6ee443935c0d269b6bc513 \ - --hash=sha256:18fcf70f243fe07252dcb1b268a687f2358025ce32f9f88028ca5c364b123ef5 \ - --hash=sha256:1993a1bb7e4eccfb922b6cd414f072e08ff5816702a0bdb8941c247a6b1b287c \ - --hash=sha256:1f3d56f73595376f4244646dd5c5870c14c196949807be39e79e7bd9bac3da63 \ - --hash=sha256:258e0dff86d1d891169b5af222d362468a9570e2532923088658aa866eb11130 \ - --hash=sha256:2f641b64acc00811da98df63df7d59fd4706c0df449da71cb7ac39a0732b40ae \ - --hash=sha256:3808e6b2e5f0b46d981c24d79648e5c25c35e59902ea4391a0dcb3e667bf7443 \ - --hash=sha256:3994c809c17fc570c2af12c9b840d7cea85a9fd3e5c0e0491f4fa3c029216d59 \ - --hash=sha256:3be4f21c6245930688bd9e162829480de027f8bf962ede33d4f8ba7d67a00cee \ - --hash=sha256:465ccac9d70115cd4de7186e60cfe989de73f7bb23e8a7aa45af18f7412e75bf \ - --hash=sha256:48c41a44ef8b8c2e80ca4527ee81daa4c527df3ecbc9423c41a420a9559d0e27 \ - --hash=sha256:4a862753b36620af6fc54209264f92c716367f2f0ff4624952276a6bbd18cbde \ - --hash=sha256:4b1654dfc64ea479c242508eb8c724044f1e964a47d1d1cacc5132292d851971 \ - --hash=sha256:4bd3e5c4b9682bc112d634f2c6ccc6736ed3635fc3319ac2bb11d768cc5a00d8 \ - --hash=sha256:577470e39e60a6cd7780793202e63536026d9b8641de011ed9d8174da9ca5339 \ - --hash=sha256:67285f8a611b0ebc0857ced2081e30302909f571a46bfa7a3cc0ad303fe015c6 \ - --hash=sha256:7285a89df4900ed3bfaad5679b1e668cb4b38a8de1ccbfc84b05f34512da0a90 \ - --hash=sha256:81823935e2f8d476707e85a78a405953a03ef7b7b4f55f93f7c2d9680e5e0691 \ - --hash=sha256:8978132287a9d3ad6b54fcd1e08548033cc09dc6aacacb6c004c73c3eb5d3ac3 \ - --hash=sha256:a20e442e917889d1a6b3c570c9e3fa2fdc398c20868abcea268ea33c024c4083 \ - --hash=sha256:a24ee598d10befaec178efdff6054bc4d7e883f615bfbcd08126a0f4931c83a6 \ - --hash=sha256:b04f85ac3a90c227b6e5890acb0edbaf3140938dbecf07bff618bf3638578cf1 \ - --hash=sha256:b6a0e535baec27b528cb07a119f321ac024592388c5681a5ced167ae98e9fff3 \ - --hash=sha256:bef32a5e327bd8e5af915d3416ffefdbe65ed975b646b3805be81b23580b57b8 \ - --hash=sha256:bfb4c801f65dd61cedfc61a83732327fafbac55a47282e6f26f073ca7a41c3b2 \ - --hash=sha256:c13b1e3afd29a5b3b2656257f14669ca8fa8d7956d509926f0b130b600b50ab7 \ - --hash=sha256:c987dad82e8c65ebc985f5dae5e74a3beda9d0a2a4daf8a1115f3772b59e5141 \ - --hash=sha256:ce7a453385e4c4693985b4a4a3533e041558851eae061a58a5405363b098fcd3 \ - --hash=sha256:d0c5c6bac22b177bf8da7435d9d27a6834ee130309749d162b26c3105c0795a9 \ - --hash=sha256:d97cf502abe2ab9eff8bd5e4aca274da8d06dd3ef08b759a8d6143f4ad65d4b4 \ - --hash=sha256:dad43797959a74103cb59c5dac71409f9c27d34c8a05921341fb64ea8ccb1dd4 \ - --hash=sha256:dd342f085542f6eb894ca00ef70236ea46070c8a13824c6bde0dfdcd36065b9b \ - --hash=sha256:de58755d723e86175756f463f2f0bddd45cc36fbd62601228a3f8761c9f58252 \ - --hash=sha256:f3df7b3d0f91b88b2106031fd995802a2e9ae13e02c36c1fc075b43f420f3a17 \ - --hash=sha256:f5414a788ecc6ee6bc58560e85ca624258a55ca434884445440a810796ea0e0b \ - --hash=sha256:fa26fa54c0a9384c27fcdc905a2fb7d60ac6e47d14bc2692145f2b3b1e2cfdbd +cryptography==46.0.5 ; sys_platform == 'linux' \ + --hash=sha256:02f547fce831f5096c9a567fd41bc12ca8f11df260959ecc7c3202555cc47a72 \ + --hash=sha256:039917b0dc418bb9f6edce8a906572d69e74bd330b0b3fea4f79dab7f8ddd235 \ + --hash=sha256:1abfdb89b41c3be0365328a410baa9df3ff8a9110fb75e7b52e66803ddabc9a9 \ + --hash=sha256:2ae6971afd6246710480e3f15824ed3029a60fc16991db250034efd0b9fb4356 \ + --hash=sha256:2b7a67c9cd56372f3249b39699f2ad479f6991e62ea15800973b956f4b73e257 \ + --hash=sha256:351695ada9ea9618b3500b490ad54c739860883df6c1f555e088eaf25b1bbaad \ + --hash=sha256:38946c54b16c885c72c4f59846be9743d699eee2b69b6988e0a00a01f46a61a4 \ + --hash=sha256:3b4995dc971c9fb83c25aa44cf45f02ba86f71ee600d81091c2f0cbae116b06c \ + --hash=sha256:3ce58ba46e1bc2aac4f7d9290223cead56743fa6ab94a5d53292ffaac6a91614 \ + --hash=sha256:3ee190460e2fbe447175cda91b88b84ae8322a104fc27766ad09428754a618ed \ + --hash=sha256:4108d4c09fbbf2789d0c926eb4152ae1760d5a2d97612b92d508d96c861e4d31 \ + --hash=sha256:420d0e909050490d04359e7fdb5ed7e667ca5c3c402b809ae2563d7e66a92229 \ + --hash=sha256:47fb8a66058b80e509c47118ef8a75d14c455e81ac369050f20ba0d23e77fee0 \ + --hash=sha256:4c3341037c136030cb46e4b1e17b7418ea4cbd9dd207e4a6f3b2b24e0d4ac731 \ + --hash=sha256:4d7e3d356b8cd4ea5aff04f129d5f66ebdc7b6f8eae802b93739ed520c47c79b \ + --hash=sha256:4d8ae8659ab18c65ced284993c2265910f6c9e650189d4e3f68445ef82a810e4 \ + --hash=sha256:4e817a8920bfbcff8940ecfd60f23d01836408242b30f1a708d93198393a80b4 \ + --hash=sha256:50bfb6925eff619c9c023b967d5b77a54e04256c4281b0e21336a130cd7fc263 \ + --hash=sha256:556e106ee01aa13484ce9b0239bca667be5004efb0aabbed28d353df86445595 \ + --hash=sha256:582f5fcd2afa31622f317f80426a027f30dc792e9c80ffee87b993200ea115f1 \ + --hash=sha256:5be7bf2fb40769e05739dd0046e7b26f9d4670badc7b032d6ce4db64dddc0678 \ + --hash=sha256:60ee7e19e95104d4c03871d7d7dfb3d22ef8a9b9c6778c94e1c8fcc8365afd48 \ + --hash=sha256:61aa400dce22cb001a98014f647dc21cda08f7915ceb95df0c9eaf84b4b6af76 \ + --hash=sha256:68f68d13f2e1cb95163fa3b4db4bf9a159a418f5f6e7242564fc75fcae667fd0 \ + --hash=sha256:7d1f30a86d2757199cb2d56e48cce14deddf1f9c95f1ef1b64ee91ea43fe2e18 \ + --hash=sha256:7d731d4b107030987fd61a7f8ab512b25b53cef8f233a97379ede116f30eb67d \ + --hash=sha256:803812e111e75d1aa73690d2facc295eaefd4439be1023fefc4995eaea2af90d \ + --hash=sha256:80a8d7bfdf38f87ca30a5391c0c9ce4ed2926918e017c29ddf643d0ed2778ea1 \ + --hash=sha256:8293f3dea7fc929ef7240796ba231413afa7b68ce38fd21da2995549f5961981 \ + --hash=sha256:8456928655f856c6e1533ff59d5be76578a7157224dbd9ce6872f25055ab9ab7 \ + --hash=sha256:890bcb4abd5a2d3f852196437129eb3667d62630333aacc13dfd470fad3aaa82 \ + --hash=sha256:94a76daa32eb78d61339aff7952ea819b1734b46f73646a07decb40e5b3448e2 \ + --hash=sha256:9f16fbdf4da055efb21c22d81b89f155f02ba420558db21288b3d0035bafd5f4 \ + --hash=sha256:a3d1fae9863299076f05cb8a778c467578262fae09f9dc0ee9b12eb4268ce663 \ + --hash=sha256:a3d507bb6a513ca96ba84443226af944b0f7f47dcc9a399d110cd6146481d24c \ + --hash=sha256:abace499247268e3757271b2f1e244b36b06f8515cf27c4d49468fc9eb16e93d \ + --hash=sha256:ba2a27ff02f48193fc4daeadf8ad2590516fa3d0adeeb34336b96f7fa64c1e3a \ + --hash=sha256:bc84e875994c3b445871ea7181d424588171efec3e185dced958dad9e001950a \ + --hash=sha256:bfd56bb4b37ed4f330b82402f6f435845a5f5648edf1ad497da51a8452d5d62d \ + --hash=sha256:c18ff11e86df2e28854939acde2d003f7984f721eba450b56a200ad90eeb0e6b \ + --hash=sha256:c3bcce8521d785d510b2aad26ae2c966092b7daa8f45dd8f44734a104dc0bc1a \ + --hash=sha256:c4143987a42a2397f2fc3b4d7e3a7d313fbe684f67ff443999e803dd75a76826 \ + --hash=sha256:c69fd885df7d089548a42d5ec05be26050ebcd2283d89b3d30676eb32ff87dee \ + --hash=sha256:ced80795227d70549a411a4ab66e8ce307899fad2220ce5ab2f296e687eacde9 \ + --hash=sha256:d66e421495fdb797610a08f43b05269e0a5ea7f5e652a89bfd5a7d3c1dee3648 \ + --hash=sha256:d861ee9e76ace6cf36a6a89b959ec08e7bc2493ee39d07ffe5acb23ef46d27da \ + --hash=sha256:e9251e3be159d1020c4030bd2e5f84d6a43fe54b6c19c12f51cde9542a2817b2 \ + --hash=sha256:f145bba11b878005c496e93e257c1e88f154d278d2638e6450d17e0f31e558d2 \ + --hash=sha256:fe346b143ff9685e40192a4960938545c699054ba11d4f9029f94751e3f71d87 # via secretstorage docutils==0.22.2 \ --hash=sha256:9fdb771707c8784c8f2728b67cb2c691305933d68137ef95a75db5f4dfbc213d \ @@ -325,9 +337,9 @@ twine==5.1.1 \ --hash=sha256:215dbe7b4b94c2c50a7315c0275d2258399280fbb7d04182c7e55e24b5f93997 \ --hash=sha256:9aa0825139c02b3434d913545c7b847a21c835e11597f5255842d457da2322db # via -r tools/publish/requirements.in -urllib3==2.5.0 \ - --hash=sha256:3fc47733c7e419d4bc3f6b3dc2b4f890bb743906a30d56ba4a5bfa4bbff92760 \ - --hash=sha256:e6b01673c0fa6a13e374b50871808eb3bf7046c4b125b216f6bf1cc604cff0dc +urllib3==2.6.3 \ + --hash=sha256:1b62b6884944a57dbe321509ab94fd4d3b307075e0c2eae991ac71ee15ad38ed \ + --hash=sha256:bf272323e553dfb2e87d9bfd225ca7b0f467b919d7bbd355436d3fd37cb0acd4 # via # requests # twine diff --git a/tools/publish/requirements_windows.txt b/tools/publish/requirements_windows.txt index f18a51e6f1..20058d3416 100644 --- a/tools/publish/requirements_windows.txt +++ b/tools/publish/requirements_windows.txt @@ -203,9 +203,9 @@ twine==5.1.1 \ --hash=sha256:215dbe7b4b94c2c50a7315c0275d2258399280fbb7d04182c7e55e24b5f93997 \ --hash=sha256:9aa0825139c02b3434d913545c7b847a21c835e11597f5255842d457da2322db # via -r tools/publish/requirements.in -urllib3==2.5.0 \ - --hash=sha256:3fc47733c7e419d4bc3f6b3dc2b4f890bb743906a30d56ba4a5bfa4bbff92760 \ - --hash=sha256:e6b01673c0fa6a13e374b50871808eb3bf7046c4b125b216f6bf1cc604cff0dc +urllib3==2.6.3 \ + --hash=sha256:1b62b6884944a57dbe321509ab94fd4d3b307075e0c2eae991ac71ee15ad38ed \ + --hash=sha256:bf272323e553dfb2e87d9bfd225ca7b0f467b919d7bbd355436d3fd37cb0acd4 # via # requests # twine From c805941dcb0f5470e8e308cd7c9cebff61832fb2 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Sat, 21 Feb 2026 09:56:24 -0800 Subject: [PATCH 638/922] feat: command_line_option transition support, default windows to enable_runfiles=true (#3610) In order to run a `py_binary` on Windows without zipping it, runfiles need to be created. However, Bazel defaults to `--enable_runfiles=false` for Windows. To fix, make `py_binary` set `enable_runfiles=true` for Windows by default. This is chosen because supporting a runfiles-less mode is quite invasive (many changes to bootstraps, plus improving the existing "recreate venv at runtime" logic to recreate the entire venv). A complication is that the `//command_line_option:...` targets don't actually exist, so they can't be passed as targets to the `config_settings` attribute (Bazel special cases them in other contexts). To fix, fake alias targets are created, and the py_binary transition logic is updated to treat those fakes as aliases for the real ones. To allow arbitrary `//command_line_option` flags to be transitioned on (using the `add_transition_label()` apis), the parsing logic treats any label whose package is `command_line_option` as such a special aliases. --- BUILD.bazel | 1 + CHANGELOG.md | 6 +++ command_line_option/BUILD.bazel | 25 +++++++++ .../rules_python/command_line_option/index.md | 53 +++++++++++++++++++ .../pip_repository_annotations/BUILD.bazel | 12 ++++- .../pip_repository_annotations_test.py | 2 +- python/features.bzl | 17 ++++++ python/private/attributes.bzl | 26 ++++++++- python/private/common_labels.bzl | 7 ++- python/private/py_binary_macro.bzl | 4 +- python/private/py_executable.bzl | 28 ++++++++++ python/private/py_test_macro.bzl | 4 +- python/private/stage2_bootstrap_template.py | 4 +- tests/support/py_reconfig.bzl | 4 +- 14 files changed, 182 insertions(+), 11 deletions(-) create mode 100644 command_line_option/BUILD.bazel create mode 100644 docs/api/rules_python/command_line_option/index.md diff --git a/BUILD.bazel b/BUILD.bazel index aa2642d43f..7da18ebaa4 100644 --- a/BUILD.bazel +++ b/BUILD.bazel @@ -43,6 +43,7 @@ filegroup( "internal_dev_deps.bzl", "internal_dev_setup.bzl", "version.bzl", + "//command_line_option:distribution", "//python:distribution", "//tools:distribution", ], diff --git a/CHANGELOG.md b/CHANGELOG.md index 9b740e52ef..1d1d492d0f 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -74,6 +74,12 @@ END_UNRELEASED_TEMPLATE in order to make `pytorch` and friends easier to patch. * (wheel) `py_wheel` no longer expands the input depset during analysis, improving analysis performance for targets with large dependency trees. +* (binaries/tests) (Windows) `--enable_runfiles=true` is the default for + `py_binary/py_test`. Prior behavior can be restored by adding + `@rules_python//command_line_option:enable_runfiles=false` or + `@rules_python//command_line_option:enable_runfiles=INHERIT` to the + `config_settings` attribute. NOTE: `enable_runfiles=true` will + soon become **required for Windows**. {#v0-0-0-fixed} ### Fixed diff --git a/command_line_option/BUILD.bazel b/command_line_option/BUILD.bazel new file mode 100644 index 0000000000..a4d5f0f80d --- /dev/null +++ b/command_line_option/BUILD.bazel @@ -0,0 +1,25 @@ +# Special placeholders for Bazel builtin //command_line_option psuedo-targets +# +# These are special targets to use with `py_binary.config_settings` that are +# treated as aliases for `//command_line_option:XXX` psuedo-targets. They +# are not actual flags or have any value. + +package( + default_visibility = ["//visibility:public"], +) + +alias( + name = "build_runfile_links", + actual = "//python:none", +) + +alias( + name = "enable_runfiles", + actual = "//python:none", +) + +filegroup( + name = "distribution", + srcs = glob(["**"]), + visibility = ["//:__subpackages__"], +) diff --git a/docs/api/rules_python/command_line_option/index.md b/docs/api/rules_python/command_line_option/index.md new file mode 100644 index 0000000000..a7dfa340eb --- /dev/null +++ b/docs/api/rules_python/command_line_option/index.md @@ -0,0 +1,53 @@ +:::{default-domain} bzl +::: +:::{bzl:currentfile} //command_line_option:BUILD.bazel +::: + +# //command_line_option + +This package provides special targets that correspond to the Bazel-builtin +`//command_line_option` psuedo-targets. These can be used with the {obj}`config_settings` +attribute on Python rules to transition specific command line flags for a target. + +:::{note} +These targets are not actual `alias()` targets, Starlark flags, nor are they the +actual builtin command line flags. They are regular targets that the +`config_settings` transition logic specially recognizes and handles as if they +were the builtin `//command_line_option` psuedo-targets. +::: + +While this package only provides a subset of builtin Bazel flags, additional +ones can be introduced by: + +* Define your own `@foo//command_line_flag:` target. It **must** be + in a top-level `command_line_flag` directory. +* Use {obj}`config.add_transition_setting` to make the rules transition on + the corresponding `//command_line_option:` builtin Bazel psuedo-target. + +:::{seealso} +The `config_settings` attribute documentation on: +* {obj}`py_binary.config_settings` +* {obj}`py_test.config_settings` +::: + +## build_runfile_links + +:::{bzl:target} build_runfile_links + +Special target for the Bazel-builtin `//command_line_option:build_runfile_links` flag. + +See the [Bazel documentation for --build_runfile_links](https://bazel.build/reference/command-line-reference#flag--build_runfile_links). + +The special value `INHERIT` can be specified to use the existing flag value. +::: + +## enable_runfiles + +:::{bzl:target} enable_runfiles + +Special target for the Bazel-builtin `//command_line_option:enable_runfiles` flag. + +See the [Bazel documentation for --enable_runfiles](https://bazel.build/reference/command-line-reference#flag--enable_runfiles). + +The special value `INHERIT` can be specified to use the existing flag value. +::: diff --git a/examples/pip_repository_annotations/BUILD.bazel b/examples/pip_repository_annotations/BUILD.bazel index 4e10c51658..242cc64c6e 100644 --- a/examples/pip_repository_annotations/BUILD.bazel +++ b/examples/pip_repository_annotations/BUILD.bazel @@ -18,7 +18,10 @@ py_test( env = { "REQUESTS_PKG_DIR": "pip_requests", "WHEEL_PKG_DIR": "pip_wheel", - }, + } | select({ + ":is_enable_runfiles_true": {"ENABLE_RUNFILES": "1"}, + "//conditions:default": {"ENABLE_RUNFILES": "0"}, + }), main = "pip_repository_annotations_test.py", deps = [ "@pip_requests//:pkg", @@ -26,3 +29,10 @@ py_test( "@rules_python//python/runfiles", ], ) + +config_setting( + name = "is_enable_runfiles_true", + values = { + "enable_runfiles": "true", + }, +) diff --git a/examples/pip_repository_annotations/pip_repository_annotations_test.py b/examples/pip_repository_annotations/pip_repository_annotations_test.py index 219be1ba03..9b15102599 100644 --- a/examples/pip_repository_annotations/pip_repository_annotations_test.py +++ b/examples/pip_repository_annotations/pip_repository_annotations_test.py @@ -89,7 +89,7 @@ def test_data_exclude_glob(self): # Because windows does not have `--enable_runfiles` on by default, the # `runfiles.Rlocation` results will be different on this platform vs # unix platforms. See `@rules_python//python/runfiles` for more details. - if platform.system() == "Windows": + if platform.system() == "Windows" and os.environ["ENABLE_RUNFILES"] == "0": self.assertIsNotNone(metadata_path) self.assertIsNone(wheel_path) else: diff --git a/python/features.bzl b/python/features.bzl index 4cc9436677..bee9aa3e68 100644 --- a/python/features.bzl +++ b/python/features.bzl @@ -20,6 +20,16 @@ _VERSION_PRIVATE = "$Format:%(describe:tags=true)$" def _features_typedef(): """Information about features rules_python has implemented. + ::::{field} targets + :type: dict[str, bool] + + A map of public API targets available in rules_python for feature detection + purposes. + + :::{versionadded} VERSION_NEXT_FEATURE + ::: + :::: + ::::{field} headers_abi3 :type: bool @@ -73,12 +83,19 @@ def _features_typedef(): :::: """ +_TARGETS = { + "//command_line_option:build_runfile_links": True, + "//command_line_option:enable_runfiles": True, + "//python/cc:current_py_cc_headers_abi3": True, +} + features = struct( TYPEDEF = _features_typedef, # keep sorted headers_abi3 = True, precompile = True, py_info_venv_symlinks = True, + targets = _TARGETS, uses_builtin_rules = False, version = _VERSION_PRIVATE if "$Format" not in _VERSION_PRIVATE else "", zipapp_rules = True, diff --git a/python/private/attributes.bzl b/python/private/attributes.bzl index 77b815ec60..0892fe3111 100644 --- a/python/private/attributes.bzl +++ b/python/private/attributes.bzl @@ -400,6 +400,23 @@ particular CPU, or defining a custom setting that `select()` uses elsewhere to pick between `pip.parse` hubs. See the [How to guide on multiple versions of a library] for a more concrete example. +:::{important} +Labels with package `command_line_option` are handled specially: they are treated +as the Bazel-builtin `//command_line_option:` psuedo-targets. + +e.g. `@foo//command_line_option:NAME` will attempt to transition +the Bazel-builtin `//command_line_option:NAME` setting. + +See the {obj}`@rules_python//command_line_option` package for some predefined +special targets, or define your own by putting them in your own `command_line_option` +directory. +::: + +:::{seealso} +* {obj}`//command_line_option:build_runfile_links` +* {obj}`//command_line_option:enable_runfiles` +::: + :::{note} These values are transitioned on, so will affect the analysis graph and the associated memory overhead. The more unique configurations in your overall @@ -426,7 +443,14 @@ def apply_config_settings_attr(settings, attr): {type}`dict[str, object]` the input `settings` value. """ for key, value in attr.config_settings.items(): - settings[str(key)] = value + if key.package == "command_line_option": + if value == "INHERIT": + continue + else: + str_key = "//command_line_option:" + key.name + else: + str_key = str(key) + settings[str_key] = value return settings AGNOSTIC_EXECUTABLE_ATTRS = dicts.add( diff --git a/python/private/common_labels.bzl b/python/private/common_labels.bzl index 9c21198a62..b6594cf0b9 100644 --- a/python/private/common_labels.bzl +++ b/python/private/common_labels.bzl @@ -8,15 +8,20 @@ labels = struct( ADD_SRCS_TO_RUNFILES = str(Label("//python/config_settings:add_srcs_to_runfiles")), BOOTSTRAP_IMPL = str(Label("//python/config_settings:bootstrap_impl")), BUILD_PYTHON_ZIP = str(Label("//python/config_settings:build_python_zip")), + # NOTE: Special target; see definition for details. + BUILD_RUNFILE_LINKS = str(Label("//command_line_option:build_runfile_links")), DEBUGGER = str(Label("//python/config_settings:debugger")), + # NOTE: Special target; see definition for details. + ENABLE_RUNFILES = str(Label("//command_line_option:enable_runfiles")), EXEC_TOOLS_TOOLCHAIN = str(Label("//python/config_settings:exec_tools_toolchain")), - PIP_ENV_MARKER_CONFIG = str(Label("//python/config_settings:pip_env_marker_config")), NONE = str(Label("//python:none")), + PIP_ENV_MARKER_CONFIG = str(Label("//python/config_settings:pip_env_marker_config")), PIP_WHL = str(Label("//python/config_settings:pip_whl")), PIP_WHL_GLIBC_VERSION = str(Label("//python/config_settings:pip_whl_glibc_version")), PIP_WHL_MUSLC_VERSION = str(Label("//python/config_settings:pip_whl_muslc_version")), PIP_WHL_OSX_ARCH = str(Label("//python/config_settings:pip_whl_osx_arch")), PIP_WHL_OSX_VERSION = str(Label("//python/config_settings:pip_whl_osx_version")), + PLATFORMS_OS_WINDOWS = str(Label("@platforms//os:windows")), PRECOMPILE = str(Label("//python/config_settings:precompile")), PRECOMPILE_SOURCE_RETENTION = str(Label("//python/config_settings:precompile_source_retention")), PYC_COLLECTION = str(Label("//python/config_settings:pyc_collection")), diff --git a/python/private/py_binary_macro.bzl b/python/private/py_binary_macro.bzl index fa10f2e8a3..c7840eff55 100644 --- a/python/private/py_binary_macro.bzl +++ b/python/private/py_binary_macro.bzl @@ -14,11 +14,11 @@ """Implementation of macro-half of py_binary rule.""" load(":py_binary_rule.bzl", py_binary_rule = "py_binary") -load(":py_executable.bzl", "convert_legacy_create_init_to_int") +load(":py_executable.bzl", "common_executable_macro_kwargs_setup") def py_binary(**kwargs): py_binary_macro(py_binary_rule, **kwargs) def py_binary_macro(py_rule, **kwargs): - convert_legacy_create_init_to_int(kwargs) + common_executable_macro_kwargs_setup(kwargs) py_rule(**kwargs) diff --git a/python/private/py_executable.bzl b/python/private/py_executable.bzl index 66d36e8e8a..284aea6bff 100644 --- a/python/private/py_executable.bzl +++ b/python/private/py_executable.bzl @@ -1768,6 +1768,29 @@ def _create_run_environment_info(ctx, inherited_environment): inherited_environment = inherited_environment, ) +def _add_config_setting_defaults(kwargs): + config_settings = kwargs.get("config_settings", None) + if config_settings == None: + config_settings = {} + + # NOTE: This code runs in loading phase within the context of the caller. + # Label() must be used to resolve repo names within rules_python's + # context to avoid unknown repo name errors. + default = select({ + labels.PLATFORMS_OS_WINDOWS: { + labels.ENABLE_RUNFILES: "true", + }, + "//conditions:default": {}, + }) + + # Let user-provided settings have precedence + config_settings = default | config_settings + kwargs["config_settings"] = config_settings + +def common_executable_macro_kwargs_setup(kwargs): + convert_legacy_create_init_to_int(kwargs) + _add_config_setting_defaults(kwargs) + def _transition_executable_impl(settings, attr): settings = dict(settings) apply_config_settings_attr(settings, attr) @@ -1777,6 +1800,7 @@ def _transition_executable_impl(settings, attr): if attr.stamp != -1: settings["//command_line_option:stamp"] = str(attr.stamp) + return settings def create_executable_rule(*, attrs, **kwargs): @@ -1830,10 +1854,14 @@ def create_executable_rule_builder(implementation, **kwargs): inputs = TRANSITION_LABELS + [ labels.PYTHON_VERSION, "//command_line_option:stamp", + "//command_line_option:build_runfile_links", + "//command_line_option:enable_runfiles", ], outputs = TRANSITION_LABELS + [ labels.PYTHON_VERSION, "//command_line_option:stamp", + "//command_line_option:build_runfile_links", + "//command_line_option:enable_runfiles", ], ), **kwargs diff --git a/python/private/py_test_macro.bzl b/python/private/py_test_macro.bzl index 028dee6678..bc58f859f8 100644 --- a/python/private/py_test_macro.bzl +++ b/python/private/py_test_macro.bzl @@ -13,12 +13,12 @@ # limitations under the License. """Implementation of macro-half of py_test rule.""" -load(":py_executable.bzl", "convert_legacy_create_init_to_int") +load(":py_executable.bzl", "common_executable_macro_kwargs_setup") load(":py_test_rule.bzl", py_test_rule = "py_test") def py_test(**kwargs): py_test_macro(py_test_rule, **kwargs) def py_test_macro(py_rule, **kwargs): - convert_legacy_create_init_to_int(kwargs) + common_executable_macro_kwargs_setup(kwargs) py_rule(**kwargs) diff --git a/python/private/stage2_bootstrap_template.py b/python/private/stage2_bootstrap_template.py index bf276f4bcb..c03a4a2e62 100644 --- a/python/private/stage2_bootstrap_template.py +++ b/python/private/stage2_bootstrap_template.py @@ -71,7 +71,7 @@ def get_build_data(self): path = os.path.normpath(path) try: # Use utf-8-sig to handle Windows BOM - with open(path, encoding='utf-8-sig') as fp: + with open(path, encoding="utf-8-sig") as fp: return fp.read() except Exception as exc: if hasattr(exc, "add_note"): @@ -207,6 +207,8 @@ def find_runfiles_root(main_rel_path): else: stub_filename = os.path.join(os.path.dirname(stub_filename), target) + # The `--enable_runfiles=false` flag is likely set, which isn't fully + # supported. raise AssertionError("Cannot find .runfiles directory for %s" % sys.argv[0]) diff --git a/tests/support/py_reconfig.bzl b/tests/support/py_reconfig.bzl index d0cb968466..9bbfdb1104 100644 --- a/tests/support/py_reconfig.bzl +++ b/tests/support/py_reconfig.bzl @@ -47,8 +47,6 @@ def _perform_transition_impl(input_settings, attr, base_impl): settings[labels.VENVS_USE_DECLARE_SYMLINK] = attr.venvs_use_declare_symlink if attr.venvs_site_packages: settings[labels.VENVS_SITE_PACKAGES] = attr.venvs_site_packages - for key, value in attr.config_settings.items(): - settings[str(key)] = value return settings _BUILTIN_BUILD_PYTHON_ZIP = [] if config.bazel_10_or_later else [ @@ -56,7 +54,9 @@ _BUILTIN_BUILD_PYTHON_ZIP = [] if config.bazel_10_or_later else [ ] _RECONFIG_INPUTS = [ + "//command_line_option:build_runfile_links", "//command_line_option:extra_toolchains", + "//command_line_option:stamp", CUSTOM_RUNTIME, labels.BOOTSTRAP_IMPL, labels.PYTHON_SRC, From 8a9cd717b1151aab7fc6f112b6f7fa090a7b1a1c Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Sat, 21 Feb 2026 22:01:22 -0800 Subject: [PATCH 639/922] chore: prepare 1.9.0 release (#3623) Updating version markers and changelog for 1.9.0 release --- CHANGELOG.md | 20 +++++++++---------- .../python/config_settings/index.md | 2 +- gazelle/docs/directives.md | 4 ++-- python/features.bzl | 2 +- python/private/py_exec_tools_info.bzl | 2 +- python/private/py_executable_info.bzl | 8 ++++---- python/private/zipapp/py_zipapp_rule.bzl | 2 +- python/py_binary.bzl | 2 +- python/zipapp/py_zipapp_binary.bzl | 2 +- python/zipapp/py_zipapp_test.bzl | 2 +- 10 files changed, 23 insertions(+), 23 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 1d1d492d0f..569722791c 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -47,16 +47,16 @@ BEGIN_UNRELEASED_TEMPLATE END_UNRELEASED_TEMPLATE --> -{#v0-0-0} -## Unreleased +{#v1-9-0} +## [1.9.0] - 2026-02-21 -[0.0.0]: https://github.com/bazel-contrib/rules_python/releases/tag/0.0.0 +[1.9.0]: https://github.com/bazel-contrib/rules_python/releases/tag/1.9.0 -{#v0-0-0-removed} +{#v1-9-0-removed} ### Removed * Nothing removed. -{#v0-0-0-changed} +{#v1-9-0-changed} ### Changed * **DEPRECATED: implicit zipapp support** * Implicit zipapp output of `py_binary`/`py_test` has been deprecated and @@ -81,7 +81,7 @@ END_UNRELEASED_TEMPLATE `config_settings` attribute. NOTE: `enable_runfiles=true` will soon become **required for Windows**. -{#v0-0-0-fixed} +{#v1-9-0-fixed} ### Fixed * (runfiles) Fixed `CurrentRepository()` raising `ValueError` on Windows. ([#3579](https://github.com/bazel-contrib/rules_python/issues/3579)) @@ -89,7 +89,7 @@ END_UNRELEASED_TEMPLATE ([#2762](https://github.com/bazel-contrib/rules_python/issues/2762)) * (gazelle) Ancestor `conftest.py` files are added in addition to sibling `conftest.py`. ([#3497](https://github.com/bazel-contrib/rules_python/issues/3497)) Note - that this behavior can be reverted to the pre-VERSION_NEXT_FEATURE behavior by setting the new + that this behavior can be reverted to the pre-1.9.0 behavior by setting the new `python_include_ancestor_conftest` directive to `false`. * (pypi) `pip_parse` no longer silently drops PEP 508 URL-based requirements (`pkg @ https://...`) when `extract_url_srcs=False` (the default for @@ -100,7 +100,7 @@ END_UNRELEASED_TEMPLATE ([#3587](https://github.com/bazel-contrib/rules_python/issues/3587)) * (binaries/tests) Stamped build data generated by Windows actions is readable -{#v0-0-0-added} +{#v1-9-0-added} ### Added * (binaries/tests) {obj}`--debugger`: allows specifying an extra dependency to add to binaries/tests for custom debuggers. @@ -127,7 +127,7 @@ END_UNRELEASED_TEMPLATE * (gazelle) A new directive `python_include_ancestor_conftest` has been added. When `false`, ancestor `conftest` targets are not automatically added to {bzl:obj}`py_test` target dependencies. This `false` behavior is how things - were in `rules_python` before VERSION_NEXT_FEATURE. The default is `true`, as the prior behavior + were in `rules_python` before 1.9.0. The default is `true`, as the prior behavior was technically incorrect. ([#3596](https://github.com/bazel-contrib/rules_python/pull/3596)) @@ -2203,4 +2203,4 @@ Breaking changes: * (pip) Create all_data_requirements alias * Expose Python C headers through the toolchain. -[0.24.0]: https://github.com/bazel-contrib/rules_python/releases/tag/0.24.0 +[0.24.0]: https://github.com/bazel-contrib/rules_python/releases/tag/0.24.0 \ No newline at end of file diff --git a/docs/api/rules_python/python/config_settings/index.md b/docs/api/rules_python/python/config_settings/index.md index 6197cac173..19f5b8bc37 100644 --- a/docs/api/rules_python/python/config_settings/index.md +++ b/docs/api/rules_python/python/config_settings/index.md @@ -62,7 +62,7 @@ Setting this flag adds the debugger dependency, but doesn't automatically set `PYTHONBREAKPOINT` to change `breakpoint()` behavior. ::: -:::{versionadded} VERSION_NEXT_FEATURE +:::{versionadded} 1.9.0 ::: :::: diff --git a/gazelle/docs/directives.md b/gazelle/docs/directives.md index cb2bb4929a..4599cd0b6c 100644 --- a/gazelle/docs/directives.md +++ b/gazelle/docs/directives.md @@ -753,7 +753,7 @@ Detailed docs are not yet written. (directive-python-include-ancestor-conftest)= ## `python_include_ancestor_conftest` -Version VERSION_NEXT_FEATURE includes a fix ({gh-pr}`3498`) for a long-standing issue +Version 1.9.0 includes a fix ({gh-pr}`3498`) for a long-standing issue ({gh-issue}`3497`) where ancestor `conftest.py` files were not automatically added as dependencies of {bzl:obj}`py_test` targets. @@ -762,7 +762,7 @@ Thus the `python_include_ancestor_conftest` directive controls this behavior. It defaults to `true`, which causes all ancestor `conftest.py` files to be included as dependencies for {bzl:obj}`py_test` targets. -Setting the directive to `false` reverts to the pre-VERSION_NEXT_FEATURE behavior. +Setting the directive to `false` reverts to the pre-1.9.0 behavior. For example, given this directory tree (not shown: intermediary `BUILD.bazel` files) diff --git a/python/features.bzl b/python/features.bzl index bee9aa3e68..5e3b7410a7 100644 --- a/python/features.bzl +++ b/python/features.bzl @@ -79,7 +79,7 @@ def _features_typedef(): Whether the rules_python version has the `py_zipapp_*` rules - :::{versionadded} VERSION_NEXT_FEATURE + :::{versionadded} 1.9.0 :::: """ diff --git a/python/private/py_exec_tools_info.bzl b/python/private/py_exec_tools_info.bzl index 16a628d822..470c0c1bf0 100644 --- a/python/private/py_exec_tools_info.bzl +++ b/python/private/py_exec_tools_info.bzl @@ -50,7 +50,7 @@ may be removed. The Python runtime to use for the exec configuration. -:::{versionadded} VERSION_NEXT_FEATURE +:::{versionadded} 1.9.0 In prior versions, the equivalent can be obtained using: ``` diff --git a/python/private/py_executable_info.bzl b/python/private/py_executable_info.bzl index 24aa705566..defbd3a05b 100644 --- a/python/private/py_executable_info.bzl +++ b/python/private/py_executable_info.bzl @@ -24,7 +24,7 @@ rules provide it directly so that the runtime the binary original chose can be accessed. ::: -:::{versionadded} VERSION_NEXT_FEATURE +:::{versionadded} 1.9.0 ::: """, "build_data_file": """ @@ -38,7 +38,7 @@ A symlink to build_data.txt if stamping is enabled, otherwise None. Args that should be passed to the interpreter before regular args (e.g. `-X whatever`). -:::{versionadded} VERSION_NEXT_FEATURE +:::{versionadded} 1.9.0 ::: """, "interpreter_path": """ @@ -75,7 +75,7 @@ The Bazel-executable-level entry point to the program, which handles Bazel-speci setup before running the file in {obj}`main`. May be None if a two-stage bootstrap implementation isn't being used. -:::{versionadded} VERSION_NEXT_FEATURE +:::{versionadded} 1.9.0 ::: """, "venv_python_exe": """ @@ -84,7 +84,7 @@ implementation isn't being used. The `bin/python3` file within the venv this binary uses. May be None if venv mode is not enabled. -:::{versionadded} VERSION_NEXT_FEATURE +:::{versionadded} 1.9.0 ::: """, }, diff --git a/python/private/zipapp/py_zipapp_rule.bzl b/python/private/zipapp/py_zipapp_rule.bzl index 72adbf48ba..a3d7fc2c6a 100644 --- a/python/private/zipapp/py_zipapp_rule.bzl +++ b/python/private/zipapp/py_zipapp_rule.bzl @@ -337,7 +337,7 @@ Output groups: the previous implicit zipapp functionality. Set `executable=False` and use the default output of the target instead.* -:::{versionadded} VERSION_NEXT_FEATURE +:::{versionadded} 1.9.0 ::: """.lstrip() diff --git a/python/py_binary.bzl b/python/py_binary.bzl index d02c3e105b..1b59451893 100644 --- a/python/py_binary.bzl +++ b/python/py_binary.bzl @@ -28,7 +28,7 @@ def py_binary(**attrs): * `srcs_version`: cannot be `PY2` or `PY2ONLY` * `tags`: May have special marker values added, if not already present. - :::{versionchanged} VERSION_NEXT_FEATURE + :::{versionchanged} 1.9.0 The `PYTHONBREAKPOINT` environment variable is inherited. Use in combination with {obj}`--debugger` to customize the debugger available and used. ::: diff --git a/python/zipapp/py_zipapp_binary.bzl b/python/zipapp/py_zipapp_binary.bzl index ba7652c6d4..0b9c9bf95c 100644 --- a/python/zipapp/py_zipapp_binary.bzl +++ b/python/zipapp/py_zipapp_binary.bzl @@ -12,7 +12,7 @@ load("//python/private/zipapp:py_zipapp_rule.bzl", _py_zipapp_binary_rule = "py_ def py_zipapp_binary(**kwargs): """Builds a Python zipapp from a py_binary/py_test target. - :::{versionadded} VERSION_NEXT_FEATURE + :::{versionadded} 1.9.0 ::: Args: diff --git a/python/zipapp/py_zipapp_test.bzl b/python/zipapp/py_zipapp_test.bzl index fa197701a9..261e584755 100644 --- a/python/zipapp/py_zipapp_test.bzl +++ b/python/zipapp/py_zipapp_test.bzl @@ -12,7 +12,7 @@ load("//python/private/zipapp:py_zipapp_rule.bzl", _py_zipapp_test = "py_zipapp_ def py_zipapp_test(**kwargs): """Builds a Python zipapp from a py_binary/py_test target. - :::{versionadded} VERSION_NEXT_FEATURE + :::{versionadded} 1.9.0 ::: Args: From 0fafcd8ed07339159bbb00c7efdff2aad59c2421 Mon Sep 17 00:00:00 2001 From: Douglas Thor Date: Sun, 22 Feb 2026 16:32:38 -0800 Subject: [PATCH 640/922] docs(gazelle): Add `versionadded` details to some Gazelle directives.md (#3624) Also fixup `CHANGELOG.md` typo. Wait for #3623 --- CHANGELOG.md | 2 +- gazelle/docs/directives.md | 44 ++++++++++++++++++++++++++++++++++++++ 2 files changed, 45 insertions(+), 1 deletion(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 569722791c..8dddccc5ff 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -107,7 +107,7 @@ END_UNRELEASED_TEMPLATE * (binaries/tests) Build information is now included in binaries and tests. Use the `bazel_binary_info` module to access it. The {flag}`--stamp` flag will add {obj}`--workspace_status_command` information. -* (gazelle) A new directive `python_generate_pyi_deps` has been added. When +* (gazelle) A new directive `python_generate_pyi_srcs` has been added. When `true`, a `py_*` target's `pyi_srcs` attribute will be set if any `.pyi` files that are associated with the target's `srcs` are present. ([#3354](https://github.com/bazel-contrib/rules_python/issues/3354)). diff --git a/gazelle/docs/directives.md b/gazelle/docs/directives.md index 4599cd0b6c..f23c75f331 100644 --- a/gazelle/docs/directives.md +++ b/gazelle/docs/directives.md @@ -343,6 +343,10 @@ Detailed docs are not yet written. (directive-python-proto-naming-convention)= ## `python_proto_naming_convention` +:::{versionadded} 1.6.0 +{gh-pr}`3093` +::: + Set this directive to a string pattern to control how the generated {bzl:obj}`py_proto_library` targets are named. When generating new {bzl:obj}`py_proto_library` rules, Gazelle will replace `$proto_name$` in the @@ -393,6 +397,10 @@ Detailed docs are not yet written. (directive-python-default-visibility)= ## `python_default_visibility` +:::{versionadded} 0.32.0 +{gh-pr}`1787` +::: + Instructs gazelle to use these visibility labels on all _python_ targets (typically `py_*`, but can be modified via the `map_kind` directive). The arg to this directive is a comma-separated list (without spaces) of labels. @@ -469,6 +477,10 @@ These special values can be useful for sub-packages. (directive-python-visibility)= ## `python_visibility` +:::{versionadded} 0.32.0 +{gh-pr}`1784` +::: + Appends additional `visibility` labels to each generated target. This directive can be set multiple times. The generated `visibility` attribute @@ -527,6 +539,10 @@ py_library( (directive-python-test-file-pattern)= ## `python_test_file_pattern` +:::{versionadded} 0.32.0 +{gh-pr}`1819` +::: + This directive adjusts which python files will be mapped to the {bzl:obj}`py_test` rule. + The default is `*_test.py,test_*.py`: both `test_*.py` and `*_test.py` files @@ -590,6 +606,10 @@ py_library( (directive-python-label-convention)= ## `python_label_convention` +:::{versionadded} 0.34.0 +{gh-pr}`1976` +::: + :::{error} Detailed docs are not yet written. ::: @@ -598,6 +618,10 @@ Detailed docs are not yet written. (directive-python-label-normalization)= ## `python_label_normalization` +:::{versionadded} 0.34.0 +{gh-pr}`1976` +::: + :::{error} Detailed docs are not yet written. ::: @@ -653,6 +677,10 @@ that are relative to the current package. (directive-python-generate-pyi-deps)= ## `python_generate_pyi_deps` +:::{versionadded} 1.6.0 +{gh-pr}`3014` +::: + :::{error} Detailed docs are not yet written. ::: @@ -661,6 +689,10 @@ Detailed docs are not yet written. (directive-python-generate-pyi-srcs)= ## `python_generate_pyi_srcs` +:::{versionadded} 1.6.0 +{gh-pr}`3356` +::: + When `true`, include any sibling `.pyi` files in the `pyi_srcs` target attribute. For example, assume you have the following files: @@ -684,6 +716,10 @@ py_library( (directive-python-generate-proto)= ## `python_generate_proto` +:::{versionadded} 1.6.0 +{gh-pr}`3057` +::: + When `# gazelle:python_generate_proto true`, Gazelle will generate one {bzl:obj}`py_proto_library` for each `proto_library`, generating Python clients for protobuf in each package. By default this is turned off. Gazelle will also @@ -746,6 +782,10 @@ previously-generated or hand-created rules. (directive-python-resolve-sibling-imports)= ## `python_resolve_sibling_imports` +:::{versionadded} 1.6.0 +{gh-pr}`3106` +::: + :::{error} Detailed docs are not yet written. ::: @@ -753,6 +793,10 @@ Detailed docs are not yet written. (directive-python-include-ancestor-conftest)= ## `python_include_ancestor_conftest` +:::{versionadded} 1.9.0 +{gh-pr}`3596` +::: + Version 1.9.0 includes a fix ({gh-pr}`3498`) for a long-standing issue ({gh-issue}`3497`) where ancestor `conftest.py` files were not automatically added as dependencies of {bzl:obj}`py_test` targets. From 44bb4f3794a02874572fae94bdb621858e68ccd7 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Sun, 22 Feb 2026 20:03:43 -0800 Subject: [PATCH 641/922] chore: replace version marker in features.bzl (#3627) A version marker in features.bzl was missed. Replace it with 1.9. --- python/features.bzl | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/python/features.bzl b/python/features.bzl index 5e3b7410a7..26acc8c5fe 100644 --- a/python/features.bzl +++ b/python/features.bzl @@ -26,7 +26,7 @@ def _features_typedef(): A map of public API targets available in rules_python for feature detection purposes. - :::{versionadded} VERSION_NEXT_FEATURE + :::{versionadded} 1.9.0 ::: :::: From 353e70683a19779dafb7b08d1e2bad27620a2082 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Mon, 23 Feb 2026 00:35:54 -0800 Subject: [PATCH 642/922] chore: update changelog with 1.8.5 notes, fix 1.8 links (#3626) Copies the changelog updates in 1.8.5 to main. Also fixes some links for the other 1.8 releases --------- Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> --- CHANGELOG.md | 38 ++++++++++++++++++++++++++++---------- 1 file changed, 28 insertions(+), 10 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 8dddccc5ff..96e86b97fb 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -83,21 +83,12 @@ END_UNRELEASED_TEMPLATE {#v1-9-0-fixed} ### Fixed -* (runfiles) Fixed `CurrentRepository()` raising `ValueError` on Windows. - ([#3579](https://github.com/bazel-contrib/rules_python/issues/3579)) * (tests) No more coverage warnings are being printed if there are no sources. ([#2762](https://github.com/bazel-contrib/rules_python/issues/2762)) * (gazelle) Ancestor `conftest.py` files are added in addition to sibling `conftest.py`. ([#3497](https://github.com/bazel-contrib/rules_python/issues/3497)) Note that this behavior can be reverted to the pre-1.9.0 behavior by setting the new `python_include_ancestor_conftest` directive to `false`. -* (pypi) `pip_parse` no longer silently drops PEP 508 URL-based requirements - (`pkg @ https://...`) when `extract_url_srcs=False` (the default for - `pip_repository`). -* (pypi) Extras in requirement strings are now normalized per PEP 685, - fixing missing transitive dependencies when extras contain hyphens - (e.g., `sqlalchemy[postgresql-psycopg2binary]`). - ([#3587](https://github.com/bazel-contrib/rules_python/issues/3587)) * (binaries/tests) Stamped build data generated by Windows actions is readable {#v1-9-0-added} @@ -131,9 +122,30 @@ END_UNRELEASED_TEMPLATE was technically incorrect. ([#3596](https://github.com/bazel-contrib/rules_python/pull/3596)) + +{#v1-8-5} +## [1.8.5] - 2026-02-22 + +[1.8.5]: https://github.com/bazel-contrib/rules_python/releases/tag/1.8.5 + +{#v1-8-5-fixed} +### Fixed +* (runfiles) Fixed `CurrentRepository()` raising `ValueError` on Windows. + ([#3579](https://github.com/bazel-contrib/rules_python/issues/3579)) +* (pypi) `pip_parse` no longer silently drops PEP 508 URL-based requirements + (`pkg @ https://...`) when `extract_url_srcs=False` (the default for + `pip_repository`). +* (pypi) Extras in requirement strings are now normalized per PEP 685, + fixing missing transitive dependencies when extras contain hyphens + (e.g., `sqlalchemy[postgresql-psycopg2binary]`). + ([#3587](https://github.com/bazel-contrib/rules_python/issues/3587)) + {#v1-8-4} ## [1.8.4] - 2026-02-10 +[1.8.4]: https://github.com/bazel-contrib/rules_python/releases/tag/1.8.4 + +{#v1-8-4-fixed} ### Fixed * (pipstar): A corner case of evaluation of version specifiers (`"1.2" ~= "1.2.0"`) has been fixed improving compatibility with the PEP440 standard. @@ -145,6 +157,8 @@ END_UNRELEASED_TEMPLATE {#v1-8-3} ## [1.8.3] - 2026-01-27 +[1.8.3]: https://github.com/bazel-contrib/rules_python/releases/tag/1.8.3 + {#v1-8-3-fixed} ### Fixed * (pipstar) Fix whl extraction on Windows when bazelrc has XX flags. @@ -153,6 +167,8 @@ END_UNRELEASED_TEMPLATE {#v1-8-2} ## [1.8.2] - 2026-01-24 +[1.8.2]: https://github.com/bazel-contrib/rules_python/releases/tag/1.8.2 + {#v1-8-2-fixed} ### Fixed * (venvs) relax the C library filename check to make tensorflow work @@ -161,6 +177,8 @@ END_UNRELEASED_TEMPLATE {#v1-8-1} ## [1.8.1] - 2026-01-20 +[1.8.1]: https://github.com/bazel-contrib/rules_python/releases/tag/1.8.1 + {#v1-8-1-fixed} ### Fixed * (pipstar) Extra resolution that refers back to the package being resolved works again. @@ -2203,4 +2221,4 @@ Breaking changes: * (pip) Create all_data_requirements alias * Expose Python C headers through the toolchain. -[0.24.0]: https://github.com/bazel-contrib/rules_python/releases/tag/0.24.0 \ No newline at end of file +[0.24.0]: https://github.com/bazel-contrib/rules_python/releases/tag/0.24.0 From 06aa36daba6377bdeaf90309c73da22ca1afa153 Mon Sep 17 00:00:00 2001 From: Ignas Anikevicius <240938+aignas@users.noreply.github.com> Date: Tue, 24 Feb 2026 01:22:37 +0900 Subject: [PATCH 643/922] feat(pypi): make whl_library reproducible under pipstar (#3589) This is testing the new API to make use of remote caching mechanisms. Needs: https://github.com/bazelbuild/bazel/pull/27634 --------- Co-authored-by: Richard Levasseur --- python/private/pypi/whl_library.bzl | 15 +++++++++++++-- 1 file changed, 13 insertions(+), 2 deletions(-) diff --git a/python/private/pypi/whl_library.bzl b/python/private/pypi/whl_library.bzl index db2b6bc770..8a8c6e53cf 100644 --- a/python/private/pypi/whl_library.bzl +++ b/python/private/pypi/whl_library.bzl @@ -548,7 +548,12 @@ def _whl_library_impl(rctx): paths.extend(path.readdir()) rctx.file("BUILD.bazel", build_file_contents) - return + + if enable_pipstar and enable_pipstar_extract: + if hasattr(rctx, "repo_metadata"): + return rctx.repo_metadata(reproducible = True) + + return None def _generate_entry_point_contents( module, @@ -690,7 +695,13 @@ whl_library = repository_rule( attrs = whl_library_attrs, doc = """ Download and extracts a single wheel based into a bazel repo based on the requirement string passed in. -Instantiated from pip_repository and inherits config options from there.""", +Instantiated from pip_repository and inherits config options from there. + +:::{versionchanged} 1.9.0 +The `whl_library` is marked as reproducible if using starlark to extract and parse the +wheel contents without building an `sdist` first. +::: +""", implementation = _whl_library_impl, environ = [ "RULES_PYTHON_PIP_ISOLATED", From a91a4d54a8174a35d6d47138821a82585dc988ba Mon Sep 17 00:00:00 2001 From: Ignas Anikevicius <240938+aignas@users.noreply.github.com> Date: Thu, 26 Feb 2026 04:47:35 +0900 Subject: [PATCH 644/922] fix(pypi): handle unnormalized package names when extracting sdist version (#3635) With this change we are handling more of the edge cases for when the filenames are more complex. Initial code had bugs when the sdist name had `-` in the name part. This code is easier to read and a little bit more explicit how it handles things. We will use it later to only return the `whl` and `sdist` entries for the versions requested through the requirements lock file. This is to make it possible to write facts only for the versions that we use. Work towards #2731 --- python/private/pypi/BUILD.bazel | 8 +++ python/private/pypi/parse_simpleapi_html.bzl | 26 +-------- python/private/pypi/version_from_filename.bzl | 42 ++++++++++++++ tests/pypi/version_from_filename/BUILD.bazel | 3 + .../version_from_filename_tests.bzl | 56 +++++++++++++++++++ 5 files changed, 112 insertions(+), 23 deletions(-) create mode 100644 python/private/pypi/version_from_filename.bzl create mode 100644 tests/pypi/version_from_filename/BUILD.bazel create mode 100644 tests/pypi/version_from_filename/version_from_filename_tests.bzl diff --git a/python/private/pypi/BUILD.bazel b/python/private/pypi/BUILD.bazel index 6bfd64652e..48a1837f36 100644 --- a/python/private/pypi/BUILD.bazel +++ b/python/private/pypi/BUILD.bazel @@ -241,6 +241,9 @@ bzl_library( bzl_library( name = "parse_simpleapi_html_bzl", srcs = ["parse_simpleapi_html.bzl"], + deps = [ + ":version_from_filename_bzl", + ], ) bzl_library( @@ -416,6 +419,11 @@ bzl_library( ], ) +bzl_library( + name = "version_from_filename_bzl", + srcs = ["version_from_filename.bzl"], +) + bzl_library( name = "whl_config_repo_bzl", srcs = ["whl_config_repo.bzl"], diff --git a/python/private/pypi/parse_simpleapi_html.bzl b/python/private/pypi/parse_simpleapi_html.bzl index a41f0750c4..23ecbf496f 100644 --- a/python/private/pypi/parse_simpleapi_html.bzl +++ b/python/private/pypi/parse_simpleapi_html.bzl @@ -16,6 +16,8 @@ Parse SimpleAPI HTML in Starlark. """ +load(":version_from_filename.bzl", "version_from_filename") + def parse_simpleapi_html(*, url, content): """Get the package URLs for given shas by parsing the Simple API HTML. @@ -64,7 +66,7 @@ def parse_simpleapi_html(*, url, content): head, _, _ = tail.rpartition("") maybe_metadata, _, filename = head.rpartition(">") - version = _version(filename) + version = version_from_filename(filename) sha256s_by_version.setdefault(version, []).append(sha256) metadata_sha256 = "" @@ -105,28 +107,6 @@ def parse_simpleapi_html(*, url, content): sha256s_by_version = sha256s_by_version, ) -_SDIST_EXTS = [ - ".tar", # handles any compression - ".zip", -] - -def _version(filename): - # See https://packaging.python.org/en/latest/specifications/binary-distribution-format/#binary-distribution-format - - _, _, tail = filename.partition("-") - version, _, _ = tail.partition("-") - if version != tail: - # The format is {name}-{version}-{whl_specifiers}.whl - return version - - # NOTE @aignas 2025-03-29: most of the files are wheels, so this is not the common path - - # {name}-{version}.{ext} - for ext in _SDIST_EXTS: - version, _, _ = version.partition(ext) # build or name - - return version - def _get_root_directory(url): scheme_end = url.find("://") if scheme_end == -1: diff --git a/python/private/pypi/version_from_filename.bzl b/python/private/pypi/version_from_filename.bzl new file mode 100644 index 0000000000..d0b6e3105d --- /dev/null +++ b/python/private/pypi/version_from_filename.bzl @@ -0,0 +1,42 @@ +"""Parse the version of the thing just from the filename. This is useful for selecting files based on the requested version.""" + +_SDIST_EXTS = [ + ".tar", # handles any compression + ".zip", +] + +def version_from_filename(filename, _fail = None): + """Parse the version of the filename. + + Args: + filename: {type}`str` the filename. + _fail: The fail function. + + Returns: + A string version or None if we could not parse the version. + """ + # See https://packaging.python.org/en/latest/specifications/binary-distribution-format/#binary-distribution-format + + if filename.endswith(".whl"): + # The format is {name}-{version}-{whl_specifiers}.whl + _, _, version = filename.partition("-") + version, _, _ = version.partition("-") + return version + + # NOTE @aignas 2025-03-29: most of the files are wheels, so this is not the common path + + # {name}-{version}.{ext} + head = "" + for ext in _SDIST_EXTS: + head, _, _ = filename.rpartition(ext) # build or name + if head: + break + + if not head: + if _fail: + _fail("Unsupported sdist extension: {filename}".format(filename = filename)) + return None + + # Based on PEP440 the version number cannot include dashes + _, _, version = head.rpartition("-") + return version diff --git a/tests/pypi/version_from_filename/BUILD.bazel b/tests/pypi/version_from_filename/BUILD.bazel new file mode 100644 index 0000000000..e9d50dc6b8 --- /dev/null +++ b/tests/pypi/version_from_filename/BUILD.bazel @@ -0,0 +1,3 @@ +load(":version_from_filename_tests.bzl", "version_from_filename_test_suite") + +version_from_filename_test_suite(name = "version_from_filename_tests") diff --git a/tests/pypi/version_from_filename/version_from_filename_tests.bzl b/tests/pypi/version_from_filename/version_from_filename_tests.bzl new file mode 100644 index 0000000000..fab921fd4f --- /dev/null +++ b/tests/pypi/version_from_filename/version_from_filename_tests.bzl @@ -0,0 +1,56 @@ +"" + +load("@rules_testing//lib:test_suite.bzl", "test_suite") +load("//python/private/pypi:version_from_filename.bzl", "version_from_filename") # buildifier: disable=bzl-visibility + +_tests = [] + +def _test_wheel_version_extraction(env): + # Case 1: wheel + env.expect.that_str(version_from_filename("foo-1.2.3-py3-none-any.whl")).equals("1.2.3") + +_tests.append(_test_wheel_version_extraction) + +def _test_sdist_version_extraction(env): + # Case 1: Standard sdist + env.expect.that_str(version_from_filename("foo-1.2.3.tar.gz")).equals("1.2.3") + + # Case 2: PEP 625 - Project name has underscores (normalized from dashes) + # If the package is 'my-pkg', the sdist might be 'my_pkg-1.0.0.tar.gz' + env.expect.that_str(version_from_filename("my_pkg-1.0.0.tar.gz")).equals("1.0.0") + + # Case 3: Project name has multiple underscores + env.expect.that_str(version_from_filename("very_long_project_name-0.5.0.zip")).equals("0.5.0") + + # Case 4: Legacy sdist with hyphens in name + # Note: Modern tools normalize this, but we should support the hyphen split + env.expect.that_str(version_from_filename("complex-name-1.2.3.tar.gz")).equals("1.2.3") + + # Case 5: Version contains an underscore (e.g. local versions) + env.expect.that_str(version_from_filename("pkg-1.2.3_post1.tar.gz")).equals("1.2.3_post1") + + # Case 6: custom compression + env.expect.that_str(version_from_filename("pkg-1.2.3_post1.tar.xz")).equals("1.2.3_post1") + +_tests.append(_test_sdist_version_extraction) + +def _test_sdist_version_extraction_fail(env): + failures = [] + + # Case 1: 7z + env.expect.that_str(version_from_filename("foo-1.2.3.7z")).equals(None) + env.expect.that_str(version_from_filename("foo-1.2.3.7z", _fail = failures.append)).equals(None) + env.expect.that_collection(failures).contains_exactly(["Unsupported sdist extension: foo-1.2.3.7z"]) + + # Case 2: egg + failures.clear() + env.expect.that_str(version_from_filename("foo-1.2.3-py3.egg", _fail = failures.append)).equals(None) + env.expect.that_collection(failures).contains_exactly(["Unsupported sdist extension: foo-1.2.3-py3.egg"]) + +_tests.append(_test_sdist_version_extraction_fail) + +def version_from_filename_test_suite(name): + test_suite( + name = name, + basic_tests = _tests, + ) From a8741eb9338fe3830ebc7c6942627171aa0c9727 Mon Sep 17 00:00:00 2001 From: Sense_wang <167664334+haosenwang1018@users.noreply.github.com> Date: Thu, 26 Feb 2026 11:09:17 +0800 Subject: [PATCH 645/922] fix: replace 2 bare except clauses with except Exception (#3637) Replace bare except clauses with except Exception. --- tools/precompiler/precompiler.py | 2 +- tools/private/update_deps/update_pip_deps.py | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/tools/precompiler/precompiler.py b/tools/precompiler/precompiler.py index e7c693c195..0afc2be530 100644 --- a/tools/precompiler/precompiler.py +++ b/tools/precompiler/precompiler.py @@ -214,7 +214,7 @@ async def _process_request(self, request: "JsonWorkRequest") -> None: # We don't send a response because we assume the request that # triggered cancelling sent the response raise - except: + except Exception: _logger.exception("Unhandled error: request=%s", request) self._send_response( { diff --git a/tools/private/update_deps/update_pip_deps.py b/tools/private/update_deps/update_pip_deps.py index 1034382f0d..406697bc4d 100755 --- a/tools/private/update_deps/update_pip_deps.py +++ b/tools/private/update_deps/update_pip_deps.py @@ -96,7 +96,7 @@ def _get_deps(report: dict) -> list[Dep]: url=dep["download_info"]["url"], sha256=dep["download_info"]["archive_info"]["hash"][len("sha256=") :], ) - except: + except Exception: debug_dep = textwrap.indent(json.dumps(dep, indent=4), " " * 4) print(f"Could not parse the response from 'pip':\n{debug_dep}") raise From 82f78b33a3ace28fc3a81f117d0d1e127a1a028d Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Thu, 26 Feb 2026 01:28:19 -0800 Subject: [PATCH 646/922] chore: simplify support policy description (#3638) Have it more explicitly reflect the reality that this is a volunteer run project. --------- Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> --- docs/support.md | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/docs/support.md b/docs/support.md index 2d1e21f128..088466d1bb 100644 --- a/docs/support.md +++ b/docs/support.md @@ -1,9 +1,10 @@ # Support Policy -The Bazel community maintains this repository. Neither Google nor the Bazel team -provides support for the code. However, this repository is part of the test -suite used to vet new Bazel releases. See the -page for information on our development workflow. +This is a community maintained project run by volunteers. What that means in +practice is: + +* Responses to issues and PRs depend on available time and energy. +* If you care, contribute (see ) and get involved. ## Supported rules_python Versions From bc64f063b22e597e37f53d35f2e46a396f8621d0 Mon Sep 17 00:00:00 2001 From: Ignas Anikevicius <240938+aignas@users.noreply.github.com> Date: Fri, 27 Feb 2026 19:03:03 +0900 Subject: [PATCH 647/922] refactor(pypi): factor out a simple implementation of the PyPI cache (#3639) We want to keep a `dict` like interface and later we would like to use the same interface to also do more things. I expect the cache key to become different in the future (i.e. include requested versions in it) so that we can check if we have the right versions in the MODULE.bazel.lock file or if we should actually call to PyPI. Work towards #2731 --- python/private/pypi/BUILD.bazel | 6 +++ python/private/pypi/extension.bzl | 3 +- python/private/pypi/hub_builder.bzl | 2 +- python/private/pypi/pypi_cache.bzl | 47 +++++++++++++++++++ python/private/pypi/simpleapi_download.bzl | 20 ++++---- tests/pypi/hub_builder/hub_builder_tests.bzl | 1 + .../simpleapi_download_tests.bzl | 11 +++-- 7 files changed, 73 insertions(+), 17 deletions(-) create mode 100644 python/private/pypi/pypi_cache.bzl diff --git a/python/private/pypi/BUILD.bazel b/python/private/pypi/BUILD.bazel index 48a1837f36..ba667a2c4d 100644 --- a/python/private/pypi/BUILD.bazel +++ b/python/private/pypi/BUILD.bazel @@ -123,6 +123,7 @@ bzl_library( ":pep508_env_bzl", ":pip_repository_attrs_bzl", ":platform_bzl", + ":pypi_cache_bzl", ":simpleapi_download_bzl", ":whl_library_bzl", "//python/private:auth_bzl", @@ -355,6 +356,11 @@ bzl_library( srcs = ["platform.bzl"], ) +bzl_library( + name = "pypi_cache_bzl", + srcs = ["pypi_cache.bzl"], +) + bzl_library( name = "pypi_repo_utils_bzl", srcs = ["pypi_repo_utils.bzl"], diff --git a/python/private/pypi/extension.bzl b/python/private/pypi/extension.bzl index 1ec9142bbb..5fded728bf 100644 --- a/python/private/pypi/extension.bzl +++ b/python/private/pypi/extension.bzl @@ -27,6 +27,7 @@ load(":parse_whl_name.bzl", "parse_whl_name") load(":pep508_env.bzl", "env") load(":pip_repository_attrs.bzl", "ATTRS") load(":platform.bzl", _plat = "platform") +load(":pypi_cache.bzl", "pypi_cache") load(":simpleapi_download.bzl", "simpleapi_download") load(":whl_library.bzl", "whl_library") @@ -224,7 +225,7 @@ You cannot use both the additive_build_content and additive_build_content_file a # dict[str repo, HubBuilder] # See `hub_builder.bzl%hub_builder()` for `HubBuilder` pip_hub_map = {} - simpleapi_cache = {} + simpleapi_cache = pypi_cache() for mod in module_ctx.modules: for pip_attr in mod.tags.parse: diff --git a/python/private/pypi/hub_builder.bzl b/python/private/pypi/hub_builder.bzl index f0aa6a73bc..bf849c3f83 100644 --- a/python/private/pypi/hub_builder.bzl +++ b/python/private/pypi/hub_builder.bzl @@ -31,7 +31,7 @@ def hub_builder( simpleapi_download_fn, evaluate_markers_fn, logger, - simpleapi_cache = {}): + simpleapi_cache): """Return a hub builder instance Args: diff --git a/python/private/pypi/pypi_cache.bzl b/python/private/pypi/pypi_cache.bzl new file mode 100644 index 0000000000..a83f96bffd --- /dev/null +++ b/python/private/pypi/pypi_cache.bzl @@ -0,0 +1,47 @@ +"""A cache for the PyPI index contents evaluation. + +This is design to work as the following: +- in-memory cache for results of PyPI index queries, so that we are not calling PyPI multiple times + for the same package for different hub repos. + +In the future the same will be used to: +- Store PyPI index query results as facts in the MODULE.bazel.lock file +""" + +def pypi_cache(store = None): + """The cache for PyPI index queries.""" + + # buildifier: disable=uninitialized + self = struct( + _store = store or {}, + setdefault = lambda key, parsed_result: _pypi_cache_setdefault(self, key, parsed_result), + get = lambda key: _pypi_cache_get(self, key), + ) + + # buildifier: enable=uninitialized + return self + +def _pypi_cache_setdefault(self, key, parsed_result): + """Store the value if not yet cached. + + Args: + self: {type}`struct` The self of this implementation. + key: {type}`str` The cache key, can be any string. + parsed_result: {type}`struct` The result of `parse_simpleapi_html` function. + + Returns: + The `parse_result`. + """ + return self._store.setdefault(key, parsed_result) + +def _pypi_cache_get(self, key): + """Return the parsed result from the cache. + + Args: + self: {type}`struct` The self of this implementation. + key: {type}`str` The cache key, can be any string. + + Returns: + The {type}`struct` or `None` based on if the result is in the cache or not. + """ + return self._store.get(key) diff --git a/python/private/pypi/simpleapi_download.bzl b/python/private/pypi/simpleapi_download.bzl index 52ff02a178..5cb338a8fd 100644 --- a/python/private/pypi/simpleapi_download.bzl +++ b/python/private/pypi/simpleapi_download.bzl @@ -49,14 +49,13 @@ def simpleapi_download( * netrc: The netrc parameter for ctx.download, see http_file for docs. * auth_patterns: The auth_patterns parameter for ctx.download, see http_file for docs. - cache: A dictionary that can be used as a cache between calls during a - single evaluation of the extension. We use a dictionary as a cache - so that we can reuse calls to the simple API when evaluating the - extension. Using the canonical_id parameter of the module_ctx would - deposit the simple API responses to the bazel cache and that is - undesirable because additions to the PyPI index would not be - reflected when re-evaluating the extension unless we do - `bazel clean --expunge`. + cache: An opaque object used to cache call results. For implementation + see ./pypi_cache.bzl file. We use the canonical_id parameter for the key + value to ensure that distribution fetches from different indexes do not cause + cache collisions, because the index may return different locations from where + the files should be downloaded. We are not using the built-in cache in the + `download` function because the index may get updated at any time and we need + to be able to refresh the data. parallel_download: A boolean to enable usage of bazel 7.1 non-blocking downloads. read_simpleapi: a function for reading and parsing of the SimpleAPI contents. Used in tests. @@ -197,8 +196,9 @@ def _read_simpleapi(ctx, url, attr, cache, get_auth = None, **download_kwargs): )) cache_key = real_url - if cache_key in cache: - return struct(success = True, output = cache[cache_key]) + cached_result = cache.get(cache_key) + if cached_result: + return struct(success = True, output = cached_result) output_str = envsubst( url, diff --git a/tests/pypi/hub_builder/hub_builder_tests.bzl b/tests/pypi/hub_builder/hub_builder_tests.bzl index 03cefd13c5..c2809c11cb 100644 --- a/tests/pypi/hub_builder/hub_builder_tests.bzl +++ b/tests/pypi/hub_builder/hub_builder_tests.bzl @@ -99,6 +99,7 @@ def hub_builder( "unit-test", printer = log_printer, ), + simpleapi_cache = {}, ) self = struct( build = lambda: env.expect.that_struct( diff --git a/tests/pypi/simpleapi_download/simpleapi_download_tests.bzl b/tests/pypi/simpleapi_download/simpleapi_download_tests.bzl index 8dc307235a..616c6c087f 100644 --- a/tests/pypi/simpleapi_download/simpleapi_download_tests.bzl +++ b/tests/pypi/simpleapi_download/simpleapi_download_tests.bzl @@ -15,6 +15,7 @@ "" load("@rules_testing//lib:test_suite.bzl", "test_suite") +load("//python/private/pypi:pypi_cache.bzl", "pypi_cache") # buildifier: disable=bzl-visibility load("//python/private/pypi:simpleapi_download.bzl", "simpleapi_download", "strip_empty_path_segments") # buildifier: disable=bzl-visibility _tests = [] @@ -52,7 +53,7 @@ def _test_simple(env): sources = ["foo", "bar", "baz"], envsubst = [], ), - cache = {}, + cache = pypi_cache(), parallel_download = True, read_simpleapi = read_simpleapi, ) @@ -112,7 +113,7 @@ def _test_fail(env): sources = ["foo", "bar", "baz"], envsubst = [], ), - cache = {}, + cache = pypi_cache(), parallel_download = True, read_simpleapi = read_simpleapi, _fail = fails.append, @@ -165,7 +166,7 @@ def _test_download_url(env): sources = ["foo", "bar", "baz"], envsubst = [], ), - cache = {}, + cache = pypi_cache(), parallel_download = False, get_auth = lambda ctx, urls, ctx_attr: struct(), ) @@ -201,7 +202,7 @@ def _test_download_url_parallel(env): sources = ["foo", "bar", "baz"], envsubst = [], ), - cache = {}, + cache = pypi_cache(), parallel_download = True, get_auth = lambda ctx, urls, ctx_attr: struct(), ) @@ -237,7 +238,7 @@ def _test_download_envsubst_url(env): sources = ["foo", "bar", "baz"], envsubst = ["INDEX_URL"], ), - cache = {}, + cache = pypi_cache(), parallel_download = False, get_auth = lambda ctx, urls, ctx_attr: struct(), ) From 0351baa03c197efb7b4ad27e2e26605c32ee629b Mon Sep 17 00:00:00 2001 From: Daniel Kongsgaard Date: Fri, 27 Feb 2026 11:27:51 +0100 Subject: [PATCH 648/922] fix(pypi): update pypi tooling deps to setuptools 82, packaging 26 (#3593) This fixes the incompatibility between `packaging==24.0` and `setuptools==78.1.1` by just updating the requirements. This works as intended in my own repositories (where I had problems with requirement updates before), but I am not sure if there are still some issues like what caused the original revert of the `packaging` update. I checked and I don't see any absolute paths in the compiled lock files (as originally mentioned in #908), but I don't know if there are any other known issues. Another solution is to just pick compatible versions of `packaging` and `setuptools` some other way. This closes #3532. Note: I haven't updated the changelog, since I was not sure if this solution is even desirable? Or if you prefer to only change `packaging` and `setuptools` versions? --------- Co-authored-by: Richard Levasseur Co-authored-by: Ignas Anikevicius <240938+aignas@users.noreply.github.com> --- CHANGELOG.md | 22 ++++++++++ examples/pip_parse/requirements_lock.txt | 4 +- examples/pip_parse/requirements_windows.txt | 4 +- python/private/pypi/deps.bzl | 48 ++++++++++----------- 4 files changed, 50 insertions(+), 28 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 96e86b97fb..c172b148ec 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -47,6 +47,28 @@ BEGIN_UNRELEASED_TEMPLATE END_UNRELEASED_TEMPLATE --> +{#v0-0-0} +## Unreleased + +[0.0.0]: https://github.com/bazel-contrib/rules_python/releases/tag/0.0.0 + +{#v0-0-0-removed} +### Removed +* Nothing removed. + +{#v0-0-0-changed} +### Changed +* (pypi) Update dependencies used for `compile_pip_requirements`, building + sdists in the `whl_library` rule and fetching wheels using `pip`. + +{#v0-0-0-fixed} +### Fixed +* Nothing fixed. + +{#v0-0-0-added} +### Added +* Nothing added. + {#v1-9-0} ## [1.9.0] - 2026-02-21 diff --git a/examples/pip_parse/requirements_lock.txt b/examples/pip_parse/requirements_lock.txt index 13a2bba1e6..2b9d8fcd47 100644 --- a/examples/pip_parse/requirements_lock.txt +++ b/examples/pip_parse/requirements_lock.txt @@ -16,7 +16,7 @@ certifi==2025.4.26 \ --hash=sha256:0a816057ea3cdefcef70270d2c515e4506bbc954f417fa5ade2021213bb8f0c6 \ --hash=sha256:30350364dfe371162649852c63336a15c70c6510c2ad5015b21c2345311805f3 # via - # -c ./constraints_certifi.txt + # -c constraints_certifi.txt # requests chardet==4.0.0 \ --hash=sha256:0d6f53a15db4120f2b08c94f11e7d93d2c911ee118b6b30a04ec3ee8310179fa \ @@ -236,7 +236,7 @@ urllib3==1.26.20 \ --hash=sha256:0ed14ccfbf1c30a9072c7ca157e4319b70d65f623e91e7b32fadb2853431016e \ --hash=sha256:40c2dc0c681e47eb8f90e7e27bf6ff7df2e677421fd46756da1161c39ca70d32 # via - # -c ./constraints_urllib3.txt + # -c constraints_urllib3.txt # requests yamllint==1.28.0 \ --hash=sha256:89bb5b5ac33b1ade059743cf227de73daa34d5e5a474b06a5e17fc16583b0cf2 \ diff --git a/examples/pip_parse/requirements_windows.txt b/examples/pip_parse/requirements_windows.txt index 7a1329d521..c407d0c8cb 100644 --- a/examples/pip_parse/requirements_windows.txt +++ b/examples/pip_parse/requirements_windows.txt @@ -16,7 +16,7 @@ certifi==2025.4.26 \ --hash=sha256:0a816057ea3cdefcef70270d2c515e4506bbc954f417fa5ade2021213bb8f0c6 \ --hash=sha256:30350364dfe371162649852c63336a15c70c6510c2ad5015b21c2345311805f3 # via - # -c ./constraints_certifi.txt + # -c constraints_certifi.txt # requests chardet==4.0.0 \ --hash=sha256:0d6f53a15db4120f2b08c94f11e7d93d2c911ee118b6b30a04ec3ee8310179fa \ @@ -240,7 +240,7 @@ urllib3==1.26.20 \ --hash=sha256:0ed14ccfbf1c30a9072c7ca157e4319b70d65f623e91e7b32fadb2853431016e \ --hash=sha256:40c2dc0c681e47eb8f90e7e27bf6ff7df2e677421fd46756da1161c39ca70d32 # via - # -c ./constraints_urllib3.txt + # -c constraints_urllib3.txt # requests yamllint==1.28.0 \ --hash=sha256:89bb5b5ac33b1ade059743cf227de73daa34d5e5a474b06a5e17fc16583b0cf2 \ diff --git a/python/private/pypi/deps.bzl b/python/private/pypi/deps.bzl index 5379343d62..5d0507cb98 100644 --- a/python/private/pypi/deps.bzl +++ b/python/private/pypi/deps.bzl @@ -21,13 +21,13 @@ _RULE_DEPS = [ # START: maintained by 'bazel run //tools/private/update_deps:update_pip_deps' ( "pypi__build", - "https://files.pythonhosted.org/packages/e2/03/f3c8ba0a6b6e30d7d18c40faab90807c9bb5e9a1e3b2fe2008af624a9c97/build-1.2.1-py3-none-any.whl", - "75e10f767a433d9a86e50d83f418e83efc18ede923ee5ff7df93b6cb0306c5d4", + "https://files.pythonhosted.org/packages/c5/0d/84a4380f930db0010168e0aa7b7a8fed9ba1835a8fbb1472bc6d0201d529/build-1.4.0-py3-none-any.whl", + "6a07c1b8eb6f2b311b96fcbdbce5dab5fe637ffda0fd83c9cac622e927501596", ), ( "pypi__click", - "https://files.pythonhosted.org/packages/00/2e/d53fa4befbf2cfa713304affc7ca780ce4fc1fd8710527771b58311a3229/click-8.1.7-py3-none-any.whl", - "ae74fb96c20a0277a1d615f1e4d73c8414f5a98db8b799a7931d1582f3390c28", + "https://files.pythonhosted.org/packages/98/78/01c019cdb5d6498122777c1a43056ebb3ebfeef2076d9d026bfe15583b2b/click-8.3.1-py3-none-any.whl", + "981153a64e25f12d547d3426c367a4857371575ee7ad18df2a6183ab0545b2a6", ), ( "pypi__colorama", @@ -36,8 +36,8 @@ _RULE_DEPS = [ ), ( "pypi__importlib_metadata", - "https://files.pythonhosted.org/packages/2d/0a/679461c511447ffaf176567d5c496d1de27cbe34a87df6677d7171b2fbd4/importlib_metadata-7.1.0-py3-none-any.whl", - "30962b96c0c223483ed6cc7280e7f0199feb01a0e40cfae4d4450fc6fab1f570", + "https://files.pythonhosted.org/packages/fa/5e/f8e9a1d23b9c20a551a8a02ea3637b4642e22c2626e3a13a9a29cdea99eb/importlib_metadata-8.7.1-py3-none-any.whl", + "5a1f80bf1daa489495071efbb095d75a634cf28a8bc299581244063b53176151", ), ( "pypi__installer", @@ -46,13 +46,13 @@ _RULE_DEPS = [ ), ( "pypi__more_itertools", - "https://files.pythonhosted.org/packages/50/e2/8e10e465ee3987bb7c9ab69efb91d867d93959095f4807db102d07995d94/more_itertools-10.2.0-py3-none-any.whl", - "686b06abe565edfab151cb8fd385a05651e1fdf8f0a14191e4439283421f8684", + "https://files.pythonhosted.org/packages/a4/8e/469e5a4a2f5855992e425f3cb33804cc07bf18d48f2db061aec61ce50270/more_itertools-10.8.0-py3-none-any.whl", + "52d4362373dcf7c52546bc4af9a86ee7c4579df9a8dc268be0a2f949d376cc9b", ), ( "pypi__packaging", - "https://files.pythonhosted.org/packages/49/df/1fceb2f8900f8639e278b056416d49134fb8d84c5942ffaa01ad34782422/packaging-24.0-py3-none-any.whl", - "2ddfb553fdf02fb784c234c7ba6ccc288296ceabec964ad2eae3777778130bc5", + "https://files.pythonhosted.org/packages/b7/b9/c538f279a4e237a006a2c98387d081e9eb060d203d8ed34467cc0f0b9b53/packaging-26.0-py3-none-any.whl", + "b36f1fef9334a5588b4166f8bcd26a14e521f2b55e6b9de3aaa80d3ff7a37529", ), ( "pypi__pep517", @@ -61,38 +61,38 @@ _RULE_DEPS = [ ), ( "pypi__pip", - "https://files.pythonhosted.org/packages/8a/6a/19e9fe04fca059ccf770861c7d5721ab4c2aebc539889e97c7977528a53b/pip-24.0-py3-none-any.whl", - "ba0d021a166865d2265246961bec0152ff124de910c5cc39f1156ce3fa7c69dc", + "https://files.pythonhosted.org/packages/de/f0/c81e05b613866b76d2d1066490adf1a3dbc4ee9d9c839961c3fc8a6997af/pip-26.0.1-py3-none-any.whl", + "bdb1b08f4274833d62c1aa29e20907365a2ceb950410df15fc9521bad440122b", ), ( "pypi__pip_tools", - "https://files.pythonhosted.org/packages/0d/dc/38f4ce065e92c66f058ea7a368a9c5de4e702272b479c0992059f7693941/pip_tools-7.4.1-py3-none-any.whl", - "4c690e5fbae2f21e87843e89c26191f0d9454f362d8acdbd695716493ec8b3a9", + "https://files.pythonhosted.org/packages/6e/74/59906d876c6cb1137f42a137164f2fe683b06283cde84bfcf7f5dd43970b/pip_tools-7.5.3-py3-none-any.whl", + "3aac0c473240ae90db7213c033401f345b05197293ccbdd2704e52e7a783785e", ), ( "pypi__pyproject_hooks", - "https://files.pythonhosted.org/packages/ae/f3/431b9d5fe7d14af7a32340792ef43b8a714e7726f1d7b69cc4e8e7a3f1d7/pyproject_hooks-1.1.0-py3-none-any.whl", - "7ceeefe9aec63a1064c18d939bdc3adf2d8aa1988a510afec15151578b232aa2", + "https://files.pythonhosted.org/packages/bd/24/12818598c362d7f300f18e74db45963dbcb85150324092410c8b49405e42/pyproject_hooks-1.2.0-py3-none-any.whl", + "9e5c6bfa8dcc30091c74b0cf803c81fdd29d94f01992a7707bc97babb1141913", ), ( "pypi__setuptools", - "https://files.pythonhosted.org/packages/90/99/158ad0609729111163fc1f674a5a42f2605371a4cf036d0441070e2f7455/setuptools-78.1.1-py3-none-any.whl", - "c3a9c4211ff4c309edb8b8c4f1cbfa7ae324c4ba9f91ff254e3d305b9fd54561", + "https://files.pythonhosted.org/packages/e1/c6/76dc613121b793286a3f91621d7b75a2b493e0390ddca50f11993eadf192/setuptools-82.0.0-py3-none-any.whl", + "70b18734b607bd1da571d097d236cfcfacaf01de45717d59e6e04b96877532e0", ), ( "pypi__tomli", - "https://files.pythonhosted.org/packages/97/75/10a9ebee3fd790d20926a90a2547f0bf78f371b2f13aa822c759680ca7b9/tomli-2.0.1-py3-none-any.whl", - "939de3e7a6161af0c887ef91b7d41a53e7c5a1ca976325f429cb46ea9bc30ecc", + "https://files.pythonhosted.org/packages/23/d1/136eb2cb77520a31e1f64cbae9d33ec6df0d78bdf4160398e86eec8a8754/tomli-2.4.0-py3-none-any.whl", + "1f776e7d669ebceb01dee46484485f43a4048746235e683bcdffacdf1fb4785a", ), ( "pypi__wheel", - "https://files.pythonhosted.org/packages/7d/cd/d7460c9a869b16c3dd4e1e403cce337df165368c71d6af229a74699622ce/wheel-0.43.0-py3-none-any.whl", - "55c570405f142630c6b9f72fe09d9b67cf1477fcf543ae5b8dcb1f5b7377da81", + "https://files.pythonhosted.org/packages/87/22/b76d483683216dde3d67cba61fb2444be8d5be289bf628c13fc0fd90e5f9/wheel-0.46.3-py3-none-any.whl", + "4b399d56c9d9338230118d705d9737a2a468ccca63d5e813e2a4fc7815d8bc4d", ), ( "pypi__zipp", - "https://files.pythonhosted.org/packages/da/55/a03fd7240714916507e1fcf7ae355bd9d9ed2e6db492595f1a67f61681be/zipp-3.18.2-py3-none-any.whl", - "dce197b859eb796242b0622af1b8beb0a722d52aa2f57133ead08edd5bf5374e", + "https://files.pythonhosted.org/packages/2e/54/647ade08bf0db230bfea292f893923872fd20be6ac6f53b2b936ba839d75/zipp-3.23.0-py3-none-any.whl", + "071652d6115ed432f5ce1d34c336c0adfd6a884660d1e9712a256d3d3bd4b14e", ), # END: maintained by 'bazel run //tools/private/update_deps:update_pip_deps' ] From c91d9f026c8d10b1807a3ad5cc6dc2027a65e98e Mon Sep 17 00:00:00 2001 From: Ara Nguyen <91614797+aranguyen@users.noreply.github.com> Date: Fri, 27 Feb 2026 15:11:48 -0500 Subject: [PATCH 649/922] build: add flag_alias definition for Starlarkification of python flags (#3450) With this change, when user build with a native python flag, `flag_alias` will point to the starlark version of the flag. Please see design doc [here](https://docs.google.com/document/d/1yOvi4hVV7Ja32ocwVb4lsEUnijftk8nilXPncYm-BH8/edit?tab=t.0#heading=h.qn3unswby87l) for more details. This feature should work as a no-op in the latest bazel 8.x and 7.x releases. --------- Co-authored-by: Richard Levasseur Co-authored-by: Richard Levasseur --- MODULE.bazel | 28 ++++++++++++++++++++++++---- 1 file changed, 24 insertions(+), 4 deletions(-) diff --git a/MODULE.bazel b/MODULE.bazel index 6486634370..157258a1ba 100644 --- a/MODULE.bazel +++ b/MODULE.bazel @@ -356,8 +356,8 @@ bazel_binaries.local( name = "self", path = "tests/integration/bazel_from_env", ) -bazel_binaries.download(version = "7.4.1") -bazel_binaries.download(version = "8.0.0") +bazel_binaries.download(version = "7.7.0") +bazel_binaries.download(version = "8.5.1") bazel_binaries.download(version = "9.0.0rc1") use_repo( bazel_binaries, @@ -365,8 +365,8 @@ use_repo( # These don't appear necessary, but are reported as direct dependencies # that should be use_repo()'d, so we add them as requested "bazel_binaries_bazelisk", - "build_bazel_bazel_7_4_1", - "build_bazel_bazel_8_0_0", + "build_bazel_bazel_7_7_0", + "build_bazel_bazel_8_5_1", "build_bazel_bazel_9_0_0rc1", # "build_bazel_bazel_rolling", "build_bazel_bazel_self", @@ -459,3 +459,23 @@ uv_dev = use_extension( uv_dev.configure( version = "0.6.2", ) + +flag_alias( + name = "build_python_zip", + starlark_flag = "//python/config_settings:build_python_zip", +) + +flag_alias( + name = "incompatible_default_to_explicit_init_py", + starlark_flag = "//python/config_settings:incompatible_default_to_explicit_init_py", +) + +flag_alias( + name = "python_path", + starlark_flag = "//python/config_settings:python_path", +) + +flag_alias( + name = "experimental_python_import_all_repositories", + starlark_flag = "//python/config_settings:experimental_python_import_all_repositories", +) From 2be0dd9a591ddfccdd676da492dece9256c818b0 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Thu, 5 Mar 2026 06:40:18 -0800 Subject: [PATCH 650/922] fix: remove flag_aliases to unbreak bazel 9 (#3649) The flag_alias() function in MODULE.bazel has a bug where it doesn't properly parse labels when transitions are involved in certain ways (not entirely clear). A fix will com in a later Bazel release. Until them, remove the calls. Fixes https://github.com/bazel-contrib/rules_python/issues/3648 --- MODULE.bazel | 41 ++++++++++++++++++++++------------------- 1 file changed, 22 insertions(+), 19 deletions(-) diff --git a/MODULE.bazel b/MODULE.bazel index 157258a1ba..326bb5b78e 100644 --- a/MODULE.bazel +++ b/MODULE.bazel @@ -460,22 +460,25 @@ uv_dev.configure( version = "0.6.2", ) -flag_alias( - name = "build_python_zip", - starlark_flag = "//python/config_settings:build_python_zip", -) - -flag_alias( - name = "incompatible_default_to_explicit_init_py", - starlark_flag = "//python/config_settings:incompatible_default_to_explicit_init_py", -) - -flag_alias( - name = "python_path", - starlark_flag = "//python/config_settings:python_path", -) - -flag_alias( - name = "experimental_python_import_all_repositories", - starlark_flag = "//python/config_settings:experimental_python_import_all_repositories", -) +# Temporarily comment out these flag aliases because they break Bazel 9 +# when transitions are also used with a target. +# +# flag_alias( +# name = "build_python_zip", +# starlark_flag = "//python/config_settings:build_python_zip", +# ) + +# flag_alias( +# name = "incompatible_default_to_explicit_init_py", +# starlark_flag = "//python/config_settings:incompatible_default_to_explicit_init_py", +# ) + +# flag_alias( +# name = "python_path", +# starlark_flag = "//python/config_settings:python_path", +# ) + +# flag_alias( +# name = "experimental_python_import_all_repositories", +# starlark_flag = "//python/config_settings:experimental_python_import_all_repositories", +# ) From 612ca2e686f25c8e17e55855150c5de0aa7aaed6 Mon Sep 17 00:00:00 2001 From: Ignas Anikevicius <240938+aignas@users.noreply.github.com> Date: Thu, 5 Mar 2026 23:59:17 +0900 Subject: [PATCH 651/922] chore: cleanup the usage of rctx.getenv (#3640) Remove most of the usage of os.environ.get and use getenv instead. The question is if we should just replace all usages blindly? --- python/private/auth.bzl | 6 +----- python/private/envsubst.bzl | 4 +--- python/private/internal_config_repo.bzl | 2 +- python/private/local_runtime_repo.bzl | 2 +- python/private/pypi/attrs.bzl | 2 +- python/private/pypi/pip_repository.bzl | 2 +- python/private/pypi/simpleapi_download.bzl | 6 +----- python/private/pypi/whl_library.bzl | 10 +--------- python/private/python.bzl | 2 +- python/private/repo_utils.bzl | 11 +++-------- python/private/toolchains_repo.bzl | 2 +- tests/pypi/extension/extension_tests.bzl | 2 +- tests/pypi/hub_builder/hub_builder_tests.bzl | 12 +++++------- .../parse_requirements/parse_requirements_tests.bzl | 10 ++++------ tests/pypi/select_whl/select_whl_tests.bzl | 10 ++++------ .../simpleapi_download/simpleapi_download_tests.bzl | 10 +++++----- tests/python/python_tests.bzl | 1 - 17 files changed, 32 insertions(+), 62 deletions(-) diff --git a/python/private/auth.bzl b/python/private/auth.bzl index 6b612678c8..2bab92ef30 100644 --- a/python/private/auth.bzl +++ b/python/private/auth.bzl @@ -95,11 +95,7 @@ def get_auth(ctx, urls, ctx_attr = None): if ctx_attr.netrc: netrc = read_netrc(ctx, ctx_attr.netrc) elif "NETRC" in ctx.os.environ: - # This can be used on newer bazel versions - if hasattr(ctx, "getenv"): - netrc = read_netrc(ctx, ctx.getenv("NETRC")) - else: - netrc = read_netrc(ctx, ctx.os.environ["NETRC"]) + netrc = read_netrc(ctx, ctx.getenv("NETRC")) else: netrc = read_user_netrc(ctx) diff --git a/python/private/envsubst.bzl b/python/private/envsubst.bzl index b2fdb99e1e..98dc72a818 100644 --- a/python/private/envsubst.bzl +++ b/python/private/envsubst.bzl @@ -20,9 +20,7 @@ def envsubst(template_string, varnames, getenv): Supports `$VARNAME`, `${VARNAME}` and `${VARNAME:-default}` syntaxes in the `template_string`, looking up each `VARNAME` listed in the `varnames` list in the environment defined by the - `getenv` function. Typically called with `getenv = rctx.getenv` - (if it is available) or `getenv = rctx.os.environ.get` (on e.g. - Bazel 6 or Bazel 7, which don't have `rctx.getenv` yet). + `getenv` function. Typically called with `getenv = rctx.getenv`. Limitations: Unlike the shell, we don't support `${VARNAME}` and `${VARNAME:-default}` in the default expression for a different diff --git a/python/private/internal_config_repo.bzl b/python/private/internal_config_repo.bzl index 5b28d69ac9..524d9c4b9e 100644 --- a/python/private/internal_config_repo.bzl +++ b/python/private/internal_config_repo.bzl @@ -147,4 +147,4 @@ internal_config_repo = repository_rule( ) def _bool_from_environ(rctx, key, default): - return bool(int(repo_utils.getenv(rctx, key, default))) + return bool(int(rctx.getenv(key, default))) diff --git a/python/private/local_runtime_repo.bzl b/python/private/local_runtime_repo.bzl index df27c74950..6e39152c89 100644 --- a/python/private/local_runtime_repo.bzl +++ b/python/private/local_runtime_repo.bzl @@ -359,7 +359,7 @@ def _resolve_interpreter_path(rctx): if "/" not in interpreter_path and "\\" not in interpreter_path: # Provide a bit nicer integration with pyenv: recalculate the runtime if the # user changes the python version using e.g. `pyenv shell` - repo_utils.getenv(rctx, "PYENV_VERSION") + rctx.getenv("PYENV_VERSION") result = repo_utils.which_unchecked(rctx, interpreter_path) resolved_path = result.binary describe_failure = result.describe_failure diff --git a/python/private/pypi/attrs.bzl b/python/private/pypi/attrs.bzl index a122fc8479..a16bf02de5 100644 --- a/python/private/pypi/attrs.bzl +++ b/python/private/pypi/attrs.bzl @@ -240,7 +240,7 @@ def use_isolated(ctx, attr): use_isolated = attr.isolated # The environment variable will take precedence over the attribute - isolated_env = ctx.os.environ.get("RULES_PYTHON_PIP_ISOLATED", None) + isolated_env = ctx.getenv("RULES_PYTHON_PIP_ISOLATED", None) if isolated_env != None: if isolated_env.lower() in ("0", "false"): use_isolated = False diff --git a/python/private/pypi/pip_repository.bzl b/python/private/pypi/pip_repository.bzl index d635651039..489c124135 100644 --- a/python/private/pypi/pip_repository.bzl +++ b/python/private/pypi/pip_repository.bzl @@ -65,7 +65,7 @@ def use_isolated(ctx, attr): use_isolated = attr.isolated # The environment variable will take precedence over the attribute - isolated_env = ctx.os.environ.get("RULES_PYTHON_PIP_ISOLATED", None) + isolated_env = ctx.getenv("RULES_PYTHON_PIP_ISOLATED", None) if isolated_env != None: if isolated_env.lower() in ("0", "false"): use_isolated = False diff --git a/python/private/pypi/simpleapi_download.bzl b/python/private/pypi/simpleapi_download.bzl index 5cb338a8fd..3a0c436326 100644 --- a/python/private/pypi/simpleapi_download.bzl +++ b/python/private/pypi/simpleapi_download.bzl @@ -189,11 +189,7 @@ def _read_simpleapi(ctx, url, attr, cache, get_auth = None, **download_kwargs): # them to ctx.download if we want to correctly handle the relative URLs. # TODO: Add a test that env subbed index urls do not leak into the lock file. - real_url = strip_empty_path_segments(envsubst( - url, - attr.envsubst, - ctx.getenv if hasattr(ctx, "getenv") else ctx.os.environ.get, - )) + real_url = strip_empty_path_segments(envsubst(url, attr.envsubst, ctx.getenv)) cache_key = real_url cached_result = cache.get(cache_key) diff --git a/python/private/pypi/whl_library.bzl b/python/private/pypi/whl_library.bzl index 8a8c6e53cf..78fa6b45ab 100644 --- a/python/private/pypi/whl_library.bzl +++ b/python/private/pypi/whl_library.bzl @@ -151,21 +151,13 @@ def _parse_optional_attrs(rctx, args, extra_pip_args = None): if use_isolated(rctx, rctx.attr): args.append("--isolated") - # Bazel version 7.1.0 and later (and rolling releases from version 8.0.0-pre.20240128.3) - # support rctx.getenv(name, default): When building incrementally, any change to the value of - # the variable named by name will cause this repository to be re-fetched. - if "getenv" in dir(rctx): - getenv = rctx.getenv - else: - getenv = rctx.os.environ.get - # Check for None so we use empty default types from our attrs. # Some args want to be list, and some want to be dict. if extra_pip_args != None: args += [ "--extra_pip_args", json.encode(struct(arg = [ - envsubst(pip_arg, rctx.attr.envsubst, getenv) + envsubst(pip_arg, rctx.attr.envsubst, rctx.getenv) for pip_arg in extra_pip_args ])), ] diff --git a/python/private/python.bzl b/python/private/python.bzl index 5eb24830cc..12a17f1e72 100644 --- a/python/private/python.bzl +++ b/python/private/python.bzl @@ -56,7 +56,7 @@ def parse_modules(*, module_ctx, logger, _fail = fail): platform suffix. * register_coverage_tool: bool """ - if module_ctx.os.environ.get("RULES_PYTHON_BZLMOD_DEBUG", "0") == "1": + if module_ctx.getenv("RULES_PYTHON_BZLMOD_DEBUG", "0") == "1": debug_info = { "toolchains_registered": [], } diff --git a/python/private/repo_utils.bzl b/python/private/repo_utils.bzl index 28ba07d376..702a333772 100644 --- a/python/private/repo_utils.bzl +++ b/python/private/repo_utils.bzl @@ -29,7 +29,7 @@ def _is_repo_debug_enabled(mrctx): Returns: True if enabled, False if not. """ - return _getenv(mrctx, REPO_DEBUG_ENV_VAR) == "1" + return mrctx.getenv(REPO_DEBUG_ENV_VAR) == "1" def _logger(mrctx = None, name = None, verbosity_level = None, printer = None): """Creates a logger instance for printing messages. @@ -56,7 +56,7 @@ def _logger(mrctx = None, name = None, verbosity_level = None, printer = None): else: verbosity_level = "WARN" - env_var_verbosity = _getenv(mrctx, REPO_VERBOSITY_ENV_VAR) + env_var_verbosity = mrctx.getenv(REPO_VERBOSITY_ENV_VAR) verbosity_level = env_var_verbosity or verbosity_level verbosity = { @@ -302,7 +302,7 @@ def _which_unchecked(mrctx, binary_name): mrctx.watch(binary) describe_failure = None else: - path = _getenv(mrctx, "PATH", "") + path = mrctx.getenv("PATH", "") describe_failure = lambda: _which_describe_failure(binary_name, path) return struct( @@ -319,10 +319,6 @@ def _which_describe_failure(binary_name, path): path = path, ) -def _getenv(mrctx, name, default = None): - # Bazel 7+ API has (repository|module)_ctx.getenv - return getattr(mrctx, "getenv", mrctx.os.environ.get)(name, default) - def _args_to_str(arguments): return " ".join([_arg_repr(a) for a in arguments]) @@ -467,7 +463,6 @@ repo_utils = struct( extract = _extract, get_platforms_cpu_name = _get_platforms_cpu_name, get_platforms_os_name = _get_platforms_os_name, - getenv = _getenv, is_repo_debug_enabled = _is_repo_debug_enabled, logger = _logger, which_checked = _which_checked, diff --git a/python/private/toolchains_repo.bzl b/python/private/toolchains_repo.bzl index f7ff19c30e..1338e9c569 100644 --- a/python/private/toolchains_repo.bzl +++ b/python/private/toolchains_repo.bzl @@ -592,7 +592,7 @@ def _get_host_impl_repo_name(*, rctx, logger, python_version, os_name, cpu_name, os_name.upper(), cpu_name.upper(), ) - preference = repo_utils.getenv(rctx, env_var) + preference = rctx.getenv(env_var) if preference == None: logger.info("Consider using '{}' to select from one of the platforms: {}".format( env_var, diff --git a/tests/pypi/extension/extension_tests.bzl b/tests/pypi/extension/extension_tests.bzl index 90723c487d..1eae1ed433 100644 --- a/tests/pypi/extension/extension_tests.bzl +++ b/tests/pypi/extension/extension_tests.bzl @@ -24,8 +24,8 @@ _tests = [] def _mock_mctx(*modules, os_name = "unittest", arch_name = "exotic", environ = {}, read = None): return struct( + getenv = environ.get, os = struct( - environ = environ, name = os_name, arch = arch_name, ), diff --git a/tests/pypi/hub_builder/hub_builder_tests.bzl b/tests/pypi/hub_builder/hub_builder_tests.bzl index c2809c11cb..2a7c75b6dd 100644 --- a/tests/pypi/hub_builder/hub_builder_tests.bzl +++ b/tests/pypi/hub_builder/hub_builder_tests.bzl @@ -27,8 +27,8 @@ _tests = [] def _mock_mctx(os_name = "unittest", arch_name = "exotic", environ = {}, read = None): return struct( + getenv = environ.get, os = struct( - environ = environ, name = os_name, arch = arch_name, ), @@ -89,12 +89,10 @@ def hub_builder( evaluate_markers_fn = evaluate_markers_fn, logger = repo_utils.logger( struct( - os = struct( - environ = { - REPO_DEBUG_ENV_VAR: "1", - REPO_VERBOSITY_ENV_VAR: "TRACE" if debug else "FAIL", - }, - ), + getenv = { + REPO_DEBUG_ENV_VAR: "1", + REPO_VERBOSITY_ENV_VAR: "TRACE" if debug else "FAIL", + }.get, ), "unit-test", printer = log_printer, diff --git a/tests/pypi/parse_requirements/parse_requirements_tests.bzl b/tests/pypi/parse_requirements/parse_requirements_tests.bzl index a2efe91d99..5bd0a04316 100644 --- a/tests/pypi/parse_requirements/parse_requirements_tests.bzl +++ b/tests/pypi/parse_requirements/parse_requirements_tests.bzl @@ -111,12 +111,10 @@ def parse_requirements(debug = False, **kwargs): return _parse_requirements( ctx = _mock_ctx(), logger = repo_utils.logger(struct( - os = struct( - environ = { - REPO_DEBUG_ENV_VAR: "1", - REPO_VERBOSITY_ENV_VAR: "TRACE" if debug else "INFO", - }, - ), + getenv = { + REPO_DEBUG_ENV_VAR: "1", + REPO_VERBOSITY_ENV_VAR: "TRACE" if debug else "INFO", + }.get, ), "unit-test"), **kwargs ) diff --git a/tests/pypi/select_whl/select_whl_tests.bzl b/tests/pypi/select_whl/select_whl_tests.bzl index 1c28fcca5f..ea3670c4f7 100644 --- a/tests/pypi/select_whl/select_whl_tests.bzl +++ b/tests/pypi/select_whl/select_whl_tests.bzl @@ -84,12 +84,10 @@ def _select_whl(whls, debug = False, **kwargs): for f in whls ], logger = repo_utils.logger(struct( - os = struct( - environ = { - REPO_DEBUG_ENV_VAR: "1", - REPO_VERBOSITY_ENV_VAR: "TRACE" if debug else "INFO", - }, - ), + getenv = { + REPO_DEBUG_ENV_VAR: "1", + REPO_VERBOSITY_ENV_VAR: "TRACE" if debug else "INFO", + }.get, ), "unit-test"), **kwargs ) diff --git a/tests/pypi/simpleapi_download/simpleapi_download_tests.bzl b/tests/pypi/simpleapi_download/simpleapi_download_tests.bzl index 616c6c087f..765e5d0396 100644 --- a/tests/pypi/simpleapi_download/simpleapi_download_tests.bzl +++ b/tests/pypi/simpleapi_download/simpleapi_download_tests.bzl @@ -43,7 +43,7 @@ def _test_simple(env): contents = simpleapi_download( ctx = struct( - os = struct(environ = {}), + getenv = {}.get, report_progress = lambda _: None, ), attr = struct( @@ -101,7 +101,7 @@ def _test_fail(env): simpleapi_download( ctx = struct( - os = struct(environ = {}), + getenv = {}.get, report_progress = lambda _: None, ), attr = struct( @@ -153,7 +153,7 @@ def _test_download_url(env): simpleapi_download( ctx = struct( - os = struct(environ = {}), + getenv = {}.get, download = download, report_progress = lambda _: None, read = lambda i: "contents of " + i, @@ -189,7 +189,7 @@ def _test_download_url_parallel(env): simpleapi_download( ctx = struct( - os = struct(environ = {}), + getenv = {}.get, download = download, report_progress = lambda _: None, read = lambda i: "contents of " + i, @@ -225,7 +225,7 @@ def _test_download_envsubst_url(env): simpleapi_download( ctx = struct( - os = struct(environ = {"INDEX_URL": "https://example.com/main/simple/"}), + getenv = {"INDEX_URL": "https://example.com/main/simple/"}.get, download = download, report_progress = lambda _: None, read = lambda i: "contents of " + i, diff --git a/tests/python/python_tests.bzl b/tests/python/python_tests.bzl index 53cfd3b09c..d8b14ba784 100644 --- a/tests/python/python_tests.bzl +++ b/tests/python/python_tests.bzl @@ -26,7 +26,6 @@ def _mock_mctx(*modules, environ = {}, mocked_files = {}): path = lambda x: struct(exists = x in mocked_files, _file = x), read = lambda x, watch = None: mocked_files[x._file if "_file" in dir(x) else x], getenv = environ.get, - os = struct(environ = environ), modules = [ struct( name = modules[0].name, From 6c05d2d7d51fd647fccf215e2c919d16cb06ae8e Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Fri, 6 Mar 2026 16:44:04 -0800 Subject: [PATCH 652/922] ci: soft fail for upcoming bazel job (#3651) Upcoming RC builds may have regressions, so instead of blocking our CI on their failures, mark them as soft-fail. This way we can be aware of upcoming problems, but not block regular development. --- .bazelci/presubmit.yml | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/.bazelci/presubmit.yml b/.bazelci/presubmit.yml index 630638ee80..dc54ea3eae 100644 --- a/.bazelci/presubmit.yml +++ b/.bazelci/presubmit.yml @@ -194,6 +194,10 @@ tasks: name: "Default: Ubuntu, upcoming Bazel" platform: ubuntu2204 bazel: last_rc + # RCs may have regressions, so don't fail our CI on them + soft_fail: + - exit_status: 1 + - exit_status: 3 ubuntu_rolling: name: "Default: Ubuntu, rolling Bazel" platform: ubuntu2204 From 67ec8d5857d8af8ce977e911d7f70754e99e76dd Mon Sep 17 00:00:00 2001 From: Nate England Date: Sat, 7 Mar 2026 16:05:39 -0500 Subject: [PATCH 653/922] fix: Fix zipapp compression support for new `py_zipapp_binary` target (#3653) The compression level isn't being correctly passed by the zipapp rule to the zipper program, resulting in an error if custom compression is specified. To fix, pass the arg correctly to the zipper. `venv_zipapp_test.py` is updated to also confirm zipapps are properly compressed when configured in the BUILD target. Fixes https://github.com/bazel-contrib/rules_python/issues/3646 --- CHANGELOG.md | 3 ++- python/private/zipapp/py_zipapp_rule.bzl | 2 +- tests/py_zipapp/BUILD.bazel | 21 +++++++++++++++++++++ tests/py_zipapp/venv_zipapp_test.py | 6 ++++++ 4 files changed, 30 insertions(+), 2 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index c172b148ec..5f38346ea2 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -63,7 +63,8 @@ END_UNRELEASED_TEMPLATE {#v0-0-0-fixed} ### Fixed -* Nothing fixed. +* (zipapp) Resolve issue passing through compression settings in + `py_zippapp_binary` targets. ([#3646](https://github.com/bazel-contrib/rules_python/issues/3646)) {#v0-0-0-added} ### Added diff --git a/python/private/zipapp/py_zipapp_rule.bzl b/python/private/zipapp/py_zipapp_rule.bzl index a3d7fc2c6a..a6ab485fc4 100644 --- a/python/private/zipapp/py_zipapp_rule.bzl +++ b/python/private/zipapp/py_zipapp_rule.bzl @@ -112,7 +112,7 @@ def _create_zip(ctx, py_runtime, py_executable, stage2_bootstrap): format = "--legacy-external-runfiles=%s", ) if ctx.attr.compression: - zipper_args.add(ctx.attr.compression, "--compression=%s") + zipper_args.add(ctx.attr.compression, format = "--compression=%s") zipper_args.add("--runfiles-dir=runfiles") actions_run( diff --git a/tests/py_zipapp/BUILD.bazel b/tests/py_zipapp/BUILD.bazel index c236998cc0..708d322f41 100644 --- a/tests/py_zipapp/BUILD.bazel +++ b/tests/py_zipapp/BUILD.bazel @@ -38,6 +38,27 @@ py_test( target_compatible_with = NOT_WINDOWS, ) +# Create the app with a supported level of compression +py_zipapp_binary( + name = "venv_zipapp_compressed", + binary = ":venv_bin", + compression = "4", + target_compatible_with = NOT_WINDOWS, +) + +py_test( + name = "venv_zipapp_compressed_test", + srcs = ["venv_zipapp_test.py"], + data = [":venv_zipapp_compressed"], + env = { + "BZLMOD_ENABLED": str(int(BZLMOD_ENABLED)), + "COMPRESSED": "1", + "TEST_ZIPAPP": "$(location :venv_zipapp_compressed)", + }, + main = "venv_zipapp_test.py", + target_compatible_with = NOT_WINDOWS, +) + py_binary( name = "system_python_bin", srcs = ["main.py"], diff --git a/tests/py_zipapp/venv_zipapp_test.py b/tests/py_zipapp/venv_zipapp_test.py index 9bb917156f..fec4a544bd 100644 --- a/tests/py_zipapp/venv_zipapp_test.py +++ b/tests/py_zipapp/venv_zipapp_test.py @@ -71,6 +71,12 @@ def test_zipapp_structure(self): zipapp_path = os.environ["TEST_ZIPAPP"] with self._open_zipapp(zipapp_path) as zf: + info = zf.infolist()[0] + if os.getenv("COMPRESSED", "0") == "1": + self.assertEqual(info.compress_type, zipfile.ZIP_DEFLATED) + else: + self.assertEqual(info.compress_type, zipfile.ZIP_STORED) + namelist = zf.namelist() if self._is_bzlmod_enabled(): From 7ca9e3fbb60488815d9c3db0b2924fc21330090b Mon Sep 17 00:00:00 2001 From: Jeremy Nimmer Date: Sat, 7 Mar 2026 14:35:55 -0800 Subject: [PATCH 654/922] fix(toolchain): Also set Make variables for local toolchains (#3641) Currently, the toolchain only sets the make variables if it's an in-build toolchain. This breaks for platform toolchains. To fix, check for platform toolchains and set the make vars appropriately. This helps fix the BCR rules for `@glib`, which recently added a requirement that the Make variable `$(PYTHON3)` must always be defined by the current Python toolchain. Co-authored-by: Richard Levasseur --- CHANGELOG.md | 4 +++- python/current_py_toolchain.bzl | 6 ++++++ 2 files changed, 9 insertions(+), 1 deletion(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 5f38346ea2..189e46f3f8 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -63,8 +63,10 @@ END_UNRELEASED_TEMPLATE {#v0-0-0-fixed} ### Fixed +* (toolchain) Also set Make variables for local toolchains. * (zipapp) Resolve issue passing through compression settings in - `py_zippapp_binary` targets. ([#3646](https://github.com/bazel-contrib/rules_python/issues/3646)) + `py_zippapp_binary` targets + ([#3646](https://github.com/bazel-contrib/rules_python/issues/3646)). {#v0-0-0-added} ### Added diff --git a/python/current_py_toolchain.bzl b/python/current_py_toolchain.bzl index 0ca5c90ccc..4fb9eb9eed 100644 --- a/python/current_py_toolchain.bzl +++ b/python/current_py_toolchain.bzl @@ -28,12 +28,18 @@ def _current_py_toolchain_impl(ctx): transitive.append(toolchain.py3_runtime.files) vars["PYTHON3"] = toolchain.py3_runtime.interpreter.path vars["PYTHON3_ROOTPATH"] = toolchain.py3_runtime.interpreter.short_path + elif toolchain.py3_runtime and toolchain.py3_runtime.interpreter_path: + vars["PYTHON3"] = toolchain.py3_runtime.interpreter_path + vars["PYTHON3_ROOTPATH"] = toolchain.py3_runtime.interpreter_path if toolchain.py2_runtime and toolchain.py2_runtime.interpreter: direct.append(toolchain.py2_runtime.interpreter) transitive.append(toolchain.py2_runtime.files) vars["PYTHON2"] = toolchain.py2_runtime.interpreter.path vars["PYTHON2_ROOTPATH"] = toolchain.py2_runtime.interpreter.short_path + elif toolchain.py2_runtime and toolchain.py2_runtime.interpreter_path: + vars["PYTHON2"] = toolchain.py2_runtime.interpreter_path + vars["PYTHON2_ROOTPATH"] = toolchain.py2_runtime.interpreter_path files = depset(direct, transitive = transitive) return [ From 8c2cae6052cd903b42db5a0afe85af5bbe8aa8ff Mon Sep 17 00:00:00 2001 From: Ignas Anikevicius <240938+aignas@users.noreply.github.com> Date: Sun, 8 Mar 2026 11:39:47 +0900 Subject: [PATCH 655/922] refactor(pypi): move absolute_url to whl_library (#3652) With this PR we move the processing of the `index_url` to the `whl_library` as a preparatory step for easier `facts` implementation. The motivation is many-fold: 1. Do not have too much duplication in the facts file by potentially naturally eliminating the `index_url` prefix from the `whls` if it appears like so on the index contents. 2. Avoid doing `envsubst` too early and have logic that has to deal with it. 3. Make the cache just return fact values from the lock file in the future instead of needing to change to an absolute URL and do envsubst on it. 4. We should have a better performance because we should be doing way fewer calls to make the URL absolute during parsing of the index. 5. With the `index_url` passed to the `whl_library`, we can help out the `purl` construction as what has been discussed in #3531 about wheels from non-public indexes. Summary: - Attempt to put the `index_url` in the fewest structs possible. - Extract the `urllib` utilities file for manipulation of the URLs. - Simplify tests testing the `absolute_url` logic. Work towards #2731 --- python/private/pypi/BUILD.bazel | 7 + python/private/pypi/hub_builder.bzl | 4 + python/private/pypi/parse_requirements.bzl | 8 +- python/private/pypi/parse_simpleapi_html.bzl | 50 +------ python/private/pypi/pypi_cache.bzl | 10 +- python/private/pypi/simpleapi_download.bzl | 83 ++++++------ python/private/pypi/urllib.bzl | 82 ++++++++++++ python/private/pypi/whl_library.bzl | 11 ++ tests/pypi/hub_builder/hub_builder_tests.bzl | 61 +++++---- .../parse_requirements_tests.bzl | 28 +++- .../parse_simpleapi_html_tests.bzl | 122 +----------------- .../simpleapi_download_tests.bzl | 45 ++++--- tests/pypi/urllib/BUILD.bazel | 3 + tests/pypi/urllib/urllib_tests.bzl | 48 +++++++ 14 files changed, 310 insertions(+), 252 deletions(-) create mode 100644 python/private/pypi/urllib.bzl create mode 100644 tests/pypi/urllib/BUILD.bazel create mode 100644 tests/pypi/urllib/urllib_tests.bzl diff --git a/python/private/pypi/BUILD.bazel b/python/private/pypi/BUILD.bazel index ba667a2c4d..34318d7920 100644 --- a/python/private/pypi/BUILD.bazel +++ b/python/private/pypi/BUILD.bazel @@ -418,6 +418,7 @@ bzl_library( srcs = ["simpleapi_download.bzl"], deps = [ ":parse_simpleapi_html_bzl", + ":urllib_bzl", "//python/private:auth_bzl", "//python/private:normalize_name_bzl", "//python/private:text_util_bzl", @@ -425,6 +426,11 @@ bzl_library( ], ) +bzl_library( + name = "urllib_bzl", + srcs = ["urllib.bzl"], +) + bzl_library( name = "version_from_filename_bzl", srcs = ["version_from_filename.bzl"], @@ -474,6 +480,7 @@ bzl_library( ":patch_whl_bzl", ":pep508_requirement_bzl", ":pypi_repo_utils_bzl", + ":urllib_bzl", ":whl_extract_bzl", ":whl_metadata_bzl", ":whl_target_platforms_bzl", diff --git a/python/private/pypi/hub_builder.bzl b/python/private/pypi/hub_builder.bzl index bf849c3f83..069d519e3c 100644 --- a/python/private/pypi/hub_builder.bzl +++ b/python/private/pypi/hub_builder.bzl @@ -599,6 +599,7 @@ def _create_whl_repos( for src in whl.srcs: repo = _whl_repo( src = src, + index_url = whl.index_url, whl_library_args = whl_library_args, download_only = pip_attr.download_only, netrc = self._config.netrc or pip_attr.netrc, @@ -678,6 +679,7 @@ def _whl_repo( *, src, whl_library_args, + index_url, is_multiple_versions, download_only, netrc, @@ -731,6 +733,8 @@ def _whl_repo( args["netrc"] = netrc if auth_patterns: args["auth_patterns"] = auth_patterns + if index_url: + args["index_url"] = index_url args["urls"] = [src.url] args["sha256"] = src.sha256 diff --git a/python/private/pypi/parse_requirements.bzl b/python/private/pypi/parse_requirements.bzl index 760bf5a9e8..acc35b3208 100644 --- a/python/private/pypi/parse_requirements.bzl +++ b/python/private/pypi/parse_requirements.bzl @@ -188,10 +188,11 @@ def parse_requirements( for p in r.target_platforms: requirement_target_platforms[p] = None + pkg_sources = index_urls.get(name) package_srcs = _package_srcs( name = name, reqs = reqs, - index_urls = index_urls, + pkg_sources = pkg_sources, platforms = platforms, extract_url_srcs = extract_url_srcs, logger = logger, @@ -216,6 +217,7 @@ def parse_requirements( name = normalize_name(name), is_exposed = len(requirement_target_platforms) == len(requirements), is_multiple_versions = len(reqs.values()) > 1, + index_url = pkg_sources.index_url if pkg_sources else "", srcs = package_srcs, ) ret.append(item) @@ -234,7 +236,7 @@ def _package_srcs( *, name, reqs, - index_urls, + pkg_sources, platforms, logger, extract_url_srcs): @@ -253,7 +255,7 @@ def _package_srcs( dist, can_fallback = _add_dists( requirement = r, target_platform = platforms.get(target_platform), - index_urls = index_urls.get(name), + index_urls = pkg_sources, logger = logger, ) logger.debug(lambda: "The whl dist is: {}".format(dist.filename if dist else dist)) diff --git a/python/private/pypi/parse_simpleapi_html.bzl b/python/private/pypi/parse_simpleapi_html.bzl index 23ecbf496f..6778d3da16 100644 --- a/python/private/pypi/parse_simpleapi_html.bzl +++ b/python/private/pypi/parse_simpleapi_html.bzl @@ -18,11 +18,10 @@ Parse SimpleAPI HTML in Starlark. load(":version_from_filename.bzl", "version_from_filename") -def parse_simpleapi_html(*, url, content): +def parse_simpleapi_html(*, content): """Get the package URLs for given shas by parsing the Simple API HTML. Args: - url(str): The URL that the HTML content can be downloaded from. content(str): The Simple API HTML content. Returns: @@ -57,7 +56,6 @@ def parse_simpleapi_html(*, url, content): sha256s_by_version = {} for line in lines[1:]: dist_url, _, tail = line.partition("#sha256=") - dist_url = _absolute_url(url, dist_url) sha256, _, tail = tail.partition("\"") @@ -87,7 +85,7 @@ def parse_simpleapi_html(*, url, content): url = dist_url, sha256 = sha256, metadata_sha256 = metadata_sha256, - metadata_url = _absolute_url(url, metadata_url) if metadata_url else "", + metadata_url = metadata_url, yanked = yanked, ) else: @@ -106,47 +104,3 @@ def parse_simpleapi_html(*, url, content): whls = whls, sha256s_by_version = sha256s_by_version, ) - -def _get_root_directory(url): - scheme_end = url.find("://") - if scheme_end == -1: - fail("Invalid URL format") - - scheme = url[:scheme_end] - host_end = url.find("/", scheme_end + 3) - if host_end == -1: - host_end = len(url) - host = url[scheme_end + 3:host_end] - - return "{}://{}".format(scheme, host) - -def _is_downloadable(url): - """Checks if the URL would be accepted by the Bazel downloader. - - This is based on Bazel's HttpUtils::isUrlSupportedByDownloader - """ - return url.startswith("http://") or url.startswith("https://") or url.startswith("file://") - -def _absolute_url(index_url, candidate): - if candidate == "": - return candidate - - if _is_downloadable(candidate): - return candidate - - if candidate.startswith("/"): - # absolute path - root_directory = _get_root_directory(index_url) - return "{}{}".format(root_directory, candidate) - - if candidate.startswith(".."): - # relative path with up references - candidate_parts = candidate.split("..") - last = candidate_parts[-1] - for _ in range(len(candidate_parts) - 1): - index_url, _, _ = index_url.rstrip("/").rpartition("/") - - return "{}/{}".format(index_url, last.strip("/")) - - # relative path without up-references - return "{}/{}".format(index_url.rstrip("/"), candidate) diff --git a/python/private/pypi/pypi_cache.bzl b/python/private/pypi/pypi_cache.bzl index a83f96bffd..4dc824c10c 100644 --- a/python/private/pypi/pypi_cache.bzl +++ b/python/private/pypi/pypi_cache.bzl @@ -9,7 +9,11 @@ In the future the same will be used to: """ def pypi_cache(store = None): - """The cache for PyPI index queries.""" + """The cache for PyPI index queries. + + Currently the key is of the following structure: + (url, real_url) + """ # buildifier: disable=uninitialized self = struct( @@ -29,6 +33,10 @@ def _pypi_cache_setdefault(self, key, parsed_result): key: {type}`str` The cache key, can be any string. parsed_result: {type}`struct` The result of `parse_simpleapi_html` function. + index_url and distribution is used to write to the MODULE.bazel.lock file as facts + real_index_url and distribution is used to write to in-memory cache to ensure that there are + no duplicate calls to the PyPI indexes + Returns: The `parse_result`. """ diff --git a/python/private/pypi/simpleapi_download.bzl b/python/private/pypi/simpleapi_download.bzl index 3a0c436326..0f776ad434 100644 --- a/python/private/pypi/simpleapi_download.bzl +++ b/python/private/pypi/simpleapi_download.bzl @@ -22,6 +22,7 @@ load("//python/private:envsubst.bzl", "envsubst") load("//python/private:normalize_name.bzl", "normalize_name") load("//python/private:text_util.bzl", "render") load(":parse_simpleapi_html.bzl", "parse_simpleapi_html") +load(":urllib.bzl", "urllib") def simpleapi_download( ctx, @@ -92,13 +93,14 @@ def simpleapi_download( sources = [pkg for pkg in attr.sources if pkg not in found_on_index] for pkg in sources: pkg_normalized = normalize_name(pkg) + url = urllib.strip_empty_path_segments("{index_url}/{distribution}/".format( + index_url = index_url_overrides.get(pkg_normalized, index_url).rstrip("/"), + distribution = pkg, + )) result = read_simpleapi( ctx = ctx, - url = "{}/{}/".format( - index_url_overrides.get(pkg_normalized, index_url).rstrip("/"), - pkg, - ), attr = attr, + url = url, cache = cache, get_auth = get_auth, **download_kwargs @@ -108,9 +110,10 @@ def simpleapi_download( async_downloads[pkg] = struct( pkg_normalized = pkg_normalized, wait = result.wait, + url = url, ) elif result.success: - contents[pkg_normalized] = result.output + contents[pkg_normalized] = _with_index_url(url, result.output) found_on_index[pkg] = index_url if not async_downloads: @@ -122,7 +125,7 @@ def simpleapi_download( result = download.wait() if result.success: - contents[download.pkg_normalized] = result.output + contents[download.pkg_normalized] = _with_index_url(download.url, result.output) found_on_index[pkg] = index_url failed_sources = [pkg for pkg in attr.sources if pkg not in found_on_index] @@ -168,14 +171,14 @@ def _read_simpleapi(ctx, url, attr, cache, get_auth = None, **download_kwargs): Args: ctx: The module_ctx or repository_ctx. - url: str, the url parameter that can be passed to ctx.download. + url: {type}`str`, the url parameter that can be passed to ctx.download. attr: The attribute that contains necessary info for downloading. The following attributes must be present: - * envsubst: The envsubst values for performing substitutions in the URL. - * netrc: The netrc parameter for ctx.download, see http_file for docs. + * envsubst: {type}`dict[str, str]` for performing substitutions in the URL. + * netrc: The netrc parameter for ctx.download, see {obj}`http_file` for docs. * auth_patterns: The auth_patterns parameter for ctx.download, see - http_file for docs. - cache: A dict for storing the results. + {obj}`http_file` for docs. + cache: {type}`struct` the `pypi_cache` instance. get_auth: A function to get auth information. Used in tests. **download_kwargs: Any extra params to ctx.download. Note that output and auth will be passed for you. @@ -189,9 +192,9 @@ def _read_simpleapi(ctx, url, attr, cache, get_auth = None, **download_kwargs): # them to ctx.download if we want to correctly handle the relative URLs. # TODO: Add a test that env subbed index urls do not leak into the lock file. - real_url = strip_empty_path_segments(envsubst(url, attr.envsubst, ctx.getenv)) + real_url = urllib.strip_empty_path_segments(envsubst(url, attr.envsubst, ctx.getenv)) - cache_key = real_url + cache_key = (url, real_url) cached_result = cache.get(cache_key) if cached_result: return struct(success = True, output = cached_result) @@ -225,41 +228,43 @@ def _read_simpleapi(ctx, url, attr, cache, get_auth = None, **download_kwargs): if download_kwargs.get("block") == False: # Simulate the same API as ctx.download has return struct( - wait = lambda: _read_index_result(ctx, download.wait(), output, real_url, cache, cache_key), + wait = lambda: _read_index_result( + ctx, + result = download.wait(), + output = output, + cache = cache, + cache_key = cache_key, + ), ) - return _read_index_result(ctx, download, output, real_url, cache, cache_key) - -def strip_empty_path_segments(url): - """Removes empty path segments from a URL. Does nothing for urls with no scheme. - - Public only for testing. - - Args: - url: The url to remove empty path segments from - - Returns: - The url with empty path segments removed and any trailing slash preserved. - If the url had no scheme it is returned unchanged. - """ - scheme, _, rest = url.partition("://") - if rest == "": - return url - stripped = "/".join([p for p in rest.split("/") if p]) - if url.endswith("/"): - return "{}://{}/".format(scheme, stripped) - else: - return "{}://{}".format(scheme, stripped) + return _read_index_result( + ctx, + result = download, + output = output, + cache = cache, + cache_key = cache_key, + ) -def _read_index_result(ctx, result, output, url, cache, cache_key): +def _read_index_result(ctx, *, result, output, cache, cache_key): if not result.success: return struct(success = False) content = ctx.read(output) - output = parse_simpleapi_html(url = url, content = content) + output = parse_simpleapi_html(content = content) if output: cache.setdefault(cache_key, output) - return struct(success = True, output = output, cache_key = cache_key) + return struct(success = True, output = output) else: return struct(success = False) + +def _with_index_url(index_url, values): + if not values: + return values + + return struct( + sdists = values.sdists, + whls = values.whls, + sha256s_by_version = values.sha256s_by_version, + index_url = index_url, + ) diff --git a/python/private/pypi/urllib.bzl b/python/private/pypi/urllib.bzl new file mode 100644 index 0000000000..ca6ded76b1 --- /dev/null +++ b/python/private/pypi/urllib.bzl @@ -0,0 +1,82 @@ +"""Utilities for getting an absolute URL from index_url and the URL we find on PyPI index.""" + +def _get_root_directory(url): + scheme_end = url.find("://") + if scheme_end == -1: + fail("Invalid URL format") + + scheme = url[:scheme_end] + host_end = url.find("/", scheme_end + 3) + if host_end == -1: + host_end = len(url) + host = url[scheme_end + 3:host_end] + + return "{}://{}".format(scheme, host) + +def _is_downloadable(url): + """Checks if the URL would be accepted by the Bazel downloader. + + This is based on Bazel's HttpUtils::isUrlSupportedByDownloader + """ + return url.startswith("http://") or url.startswith("https://") or url.startswith("file://") + +def _absolute_url(index_url, candidate): + """Convert into an absolute URL. + + Args: + index_url: The index URL where the file has been found. + candidate: The candidate URL which may be not absolute. + + Returns: + An absolute URL + """ + if candidate == "": + return candidate + + if _is_downloadable(candidate): + return candidate + + if candidate.startswith("/"): + # absolute path + root_directory = _get_root_directory(index_url) + return "{}{}".format(root_directory, candidate) + + if candidate.startswith(".."): + # relative path with up references + candidate_parts = candidate.split("..") + last = candidate_parts[-1] + for _ in range(len(candidate_parts) - 1): + index_url, _, _ = index_url.rstrip("/").rpartition("/") + + return "{}/{}".format(index_url, last.strip("/")) + + # relative path without up-references + return "{}/{}".format(index_url.rstrip("/"), candidate) + +def _strip_empty_path_segments(url): + """Removes empty path segments from a URL. Does nothing for urls with no scheme. + + Public only for testing. + + Args: + url: The url to remove empty path segments from + + Returns: + The url with empty path segments removed and any trailing slash preserved. + If the url had no scheme it is returned unchanged. + """ + scheme, _, rest = url.partition("://") + if rest == "": + return url + stripped = "/".join([p for p in rest.split("/") if p]) + if url.endswith("/"): + return "{}://{}/".format(scheme, stripped) + else: + return "{}://{}".format(scheme, stripped) + +urllib = struct( + is_absolute = _is_downloadable, + # Ensure that we strip empty path segments when making an absolute URL + absolute_url = lambda index_url, candidate: _strip_empty_path_segments(_absolute_url(index_url, candidate)), + strip_empty_path_segments = _strip_empty_path_segments, +) diff --git a/python/private/pypi/whl_library.bzl b/python/private/pypi/whl_library.bzl index 78fa6b45ab..31da3b94cb 100644 --- a/python/private/pypi/whl_library.bzl +++ b/python/private/pypi/whl_library.bzl @@ -26,6 +26,7 @@ load(":parse_whl_name.bzl", "parse_whl_name") load(":patch_whl.bzl", "patch_whl") load(":pep508_requirement.bzl", "requirement") load(":pypi_repo_utils.bzl", "pypi_repo_utils") +load(":urllib.bzl", "urllib") load(":whl_extract.bzl", "whl_extract") load(":whl_metadata.bzl", "whl_metadata") load(":whl_target_platforms.bzl", "whl_target_platforms") @@ -320,6 +321,13 @@ def _whl_library_impl(rctx): elif rctx.attr.urls and rctx.attr.filename: filename = rctx.attr.filename urls = rctx.attr.urls + urls = [ + urllib.absolute_url( + envsubst(rctx.attr.index_url, rctx.attr.envsubst, rctx.getenv), + url, + ) + for url in urls + ] result = rctx.download( url = urls, output = filename, @@ -607,6 +615,9 @@ For example if your whl depends on `numpy` and your Python package repo is named "group_name": attr.string( doc = "Name of the group, if any.", ), + "index_url": attr.string( + doc = "The index_url that the package will be downloaded from.", + ), "repo": attr.string( doc = "Pointer to parent repo name. Used to make these rules rerun if the parent repo changes.", ), diff --git a/tests/pypi/hub_builder/hub_builder_tests.bzl b/tests/pypi/hub_builder/hub_builder_tests.bzl index 2a7c75b6dd..27040d36d7 100644 --- a/tests/pypi/hub_builder/hub_builder_tests.bzl +++ b/tests/pypi/hub_builder/hub_builder_tests.bzl @@ -20,6 +20,7 @@ load("//python/private:repo_utils.bzl", "REPO_DEBUG_ENV_VAR", "REPO_VERBOSITY_EN load("//python/private/pypi:hub_builder.bzl", _hub_builder = "hub_builder") # buildifier: disable=bzl-visibility load("//python/private/pypi:parse_simpleapi_html.bzl", "parse_simpleapi_html") # buildifier: disable=bzl-visibility load("//python/private/pypi:platform.bzl", _plat = "platform") # buildifier: disable=bzl-visibility +load("//python/private/pypi:simpleapi_download.bzl", "simpleapi_download") # buildifier: disable=bzl-visibility load("//python/private/pypi:whl_config_setting.bzl", "whl_config_setting") # buildifier: disable=bzl-visibility load("//tests/pypi/extension:pip_parse.bzl", _parse = "pip_parse") @@ -36,6 +37,7 @@ def _mock_mctx(os_name = "unittest", arch_name = "exotic", environ = {}, read = simple==0.0.1 \ --hash=sha256:deadbeef \ --hash=sha256:deadbaaf"""), + report_progress = lambda _: None, ) def hub_builder( @@ -245,19 +247,23 @@ def _test_simple_extras_vs_no_extras(env): _tests.append(_test_simple_extras_vs_no_extras) def _test_simple_extras_vs_no_extras_simpleapi(env): - def mocksimpleapi_download(*_, **__): - return { - "simple": parse_simpleapi_html( - url = "https://example.com", + def mockread_simpleapi(*_, **__): + return struct( + output = parse_simpleapi_html( content = """\ simple-0.0.1-py3-none-any.whl
""", ), - } + success = True, + ) builder = hub_builder( env, - simpleapi_download_fn = mocksimpleapi_download, + simpleapi_download_fn = lambda *args, **kwargs: simpleapi_download( + read_simpleapi = mockread_simpleapi, + *args, + **kwargs + ), ) builder.pip_parse( _mock_mctx( @@ -271,7 +277,7 @@ def _test_simple_extras_vs_no_extras_simpleapi(env): python_version = "3.15", requirements_darwin = "darwin.txt", requirements_windows = "win.txt", - experimental_index_url = "example.com", + experimental_index_url = "https://example.com", ), ) pypi = builder.build() @@ -303,17 +309,19 @@ def _test_simple_extras_vs_no_extras_simpleapi(env): "config_load": "@pypi//:config.bzl", "dep_template": "@pypi//{name}:{target}", "filename": "simple-0.0.1-py3-none-any.whl", + "index_url": "https://example.com/simple/", "requirement": "simple[foo]==0.0.1", "sha256": "deadbeef", - "urls": ["https://example.com/simple-0.0.1-py3-none-any.whl"], + "urls": ["/simple-0.0.1-py3-none-any.whl"], }, "pypi_315_simple_py3_none_any_deadbeef_windows_aarch64": { "config_load": "@pypi//:config.bzl", "dep_template": "@pypi//{name}:{target}", "filename": "simple-0.0.1-py3-none-any.whl", + "index_url": "https://example.com/simple/", "requirement": "simple==0.0.1", "sha256": "deadbeef", - "urls": ["https://example.com/simple-0.0.1-py3-none-any.whl"], + "urls": ["/simple-0.0.1-py3-none-any.whl"], }, }) pypi.extra_aliases().contains_exactly({}) @@ -481,10 +489,9 @@ def _test_simple_with_markers(env): _tests.append(_test_simple_with_markers) def _test_torch_experimental_index_url(env): - def mocksimpleapi_download(*_, **__): - return { - "torch": parse_simpleapi_html( - url = "https://torch.index", + def mockread_simpleapi(*_, **__): + return struct( + output = parse_simpleapi_html( content = """\ torch-2.4.1+cpu-cp310-cp310-linux_x86_64.whl
torch-2.4.1+cpu-cp310-cp310-win_amd64.whl
@@ -508,7 +515,8 @@ def _test_torch_experimental_index_url(env): torch-2.4.1-cp39-none-macosx_11_0_arm64.whl
""", ), - } + success = True, + ) builder = hub_builder( env, @@ -544,7 +552,11 @@ def _test_torch_experimental_index_url(env): "python_3_12_host": "unit_test_interpreter_target", }, minor_mapping = {"3.12": "3.12.19"}, - simpleapi_download_fn = mocksimpleapi_download, + simpleapi_download_fn = lambda *args, **kwargs: simpleapi_download( + read_simpleapi = mockread_simpleapi, + *args, + **kwargs + ), ) builder.pip_parse( _mock_mctx( @@ -621,33 +633,37 @@ torch==2.4.1+cpu ; platform_machine == 'x86_64' \ "config_load": "@pypi//:config.bzl", "dep_template": "@pypi//{name}:{target}", "filename": "torch-2.4.1+cpu-cp312-cp312-linux_x86_64.whl", + "index_url": "https://torch.index/torch/", "requirement": "torch==2.4.1+cpu", "sha256": "8800deef0026011d502c0c256cc4b67d002347f63c3a38cd8e45f1f445c61364", - "urls": ["https://torch.index/whl/cpu/torch-2.4.1%2Bcpu-cp312-cp312-linux_x86_64.whl"], + "urls": ["/whl/cpu/torch-2.4.1%2Bcpu-cp312-cp312-linux_x86_64.whl"], }, "pypi_312_torch_cp312_cp312_manylinux_2_17_aarch64_36109432_linux_aarch64": { "config_load": "@pypi//:config.bzl", "dep_template": "@pypi//{name}:{target}", "filename": "torch-2.4.1-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", + "index_url": "https://torch.index/torch/", "requirement": "torch==2.4.1", "sha256": "36109432b10bd7163c9b30ce896f3c2cca1b86b9765f956a1594f0ff43091e2a", - "urls": ["https://torch.index/whl/cpu/torch-2.4.1-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl"], + "urls": ["/whl/cpu/torch-2.4.1-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl"], }, "pypi_312_torch_cp312_cp312_win_amd64_3a570e5c_windows_x86_64": { "config_load": "@pypi//:config.bzl", "dep_template": "@pypi//{name}:{target}", "filename": "torch-2.4.1+cpu-cp312-cp312-win_amd64.whl", + "index_url": "https://torch.index/torch/", "requirement": "torch==2.4.1+cpu", "sha256": "3a570e5c553415cdbddfe679207327b3a3806b21c6adea14fba77684d1619e97", - "urls": ["https://torch.index/whl/cpu/torch-2.4.1%2Bcpu-cp312-cp312-win_amd64.whl"], + "urls": ["/whl/cpu/torch-2.4.1%2Bcpu-cp312-cp312-win_amd64.whl"], }, "pypi_312_torch_cp312_none_macosx_11_0_arm64_72b484d5_osx_aarch64": { "config_load": "@pypi//:config.bzl", "dep_template": "@pypi//{name}:{target}", "filename": "torch-2.4.1-cp312-none-macosx_11_0_arm64.whl", + "index_url": "https://torch.index/torch/", "requirement": "torch==2.4.1", "sha256": "72b484d5b6cec1a735bf3fa5a1c4883d01748698c5e9cfdbeb4ffab7c7987e0d", - "urls": ["https://torch.index/whl/cpu/torch-2.4.1-cp312-none-macosx_11_0_arm64.whl"], + "urls": ["/whl/cpu/torch-2.4.1-cp312-none-macosx_11_0_arm64.whl"], }, }) pypi.extra_aliases().contains_exactly({}) @@ -771,6 +787,7 @@ def _test_simple_get_index(env): sha256s_by_version = { "0.0.4": ["deadb44f"], }, + index_url = "", ), "simple": struct( whls = { @@ -789,6 +806,7 @@ def _test_simple_get_index(env): url = "example.org", ), }, + index_url = "", ), "some_other_pkg": struct( whls = { @@ -804,6 +822,7 @@ def _test_simple_get_index(env): "0.0.1": ["deadb33f"], "0.0.3": ["deadbeef"], }, + index_url = "https://with_index_url", ), } @@ -973,9 +992,6 @@ git_dep @ git+https://git.server/repo/project@deadbeefdeadbeef "requirement": "direct_without_sha==0.0.1", "sha256": "", "urls": ["example-direct.org/direct_without_sha-0.0.1-py3-none-any.whl"], - # NOTE @aignas 2025-11-24: any patching still requires the python interpreter from the - # hermetic toolchain or the system. This is so that we can rezip it back to a wheel and - # verify the metadata so that it is installable by any installer out there. "whl_patches": {"my_patch": "1"}, }, "pypi_315_git_dep": { @@ -1020,6 +1036,7 @@ git_dep @ git+https://git.server/repo/project@deadbeefdeadbeef "config_load": "@pypi//:config.bzl", "dep_template": "@pypi//{name}:{target}", "filename": "some-other-pkg-0.0.1-py3-none-any.whl", + "index_url": "https://with_index_url", "requirement": "some_other_pkg==0.0.1", "sha256": "deadb33f", "urls": ["example2.org/index/some_other_pkg/"], diff --git a/tests/pypi/parse_requirements/parse_requirements_tests.bzl b/tests/pypi/parse_requirements/parse_requirements_tests.bzl index 5bd0a04316..0d03e94467 100644 --- a/tests/pypi/parse_requirements/parse_requirements_tests.bzl +++ b/tests/pypi/parse_requirements/parse_requirements_tests.bzl @@ -128,6 +128,7 @@ def _test_simple(env): env.expect.that_collection(got).contains_exactly([ struct( name = "foo", + index_url = "", is_exposed = True, is_multiple_versions = False, srcs = [ @@ -161,6 +162,7 @@ def _test_direct_urls_integration(env): env.expect.that_collection(got).contains_exactly([ struct( name = "foo", + index_url = "", is_exposed = True, is_multiple_versions = True, srcs = [ @@ -202,6 +204,7 @@ def _test_direct_urls_no_extract(env): env.expect.that_collection(got).contains_exactly([ struct( name = "foo", + index_url = "", is_exposed = True, is_multiple_versions = True, srcs = [ @@ -241,6 +244,7 @@ def _test_extra_pip_args(env): env.expect.that_collection(got).contains_exactly([ struct( name = "foo", + index_url = "", is_exposed = True, is_multiple_versions = False, srcs = [ @@ -271,6 +275,7 @@ def _test_dupe_requirements(env): env.expect.that_collection(got).contains_exactly([ struct( name = "foo", + index_url = "", is_exposed = True, is_multiple_versions = False, srcs = [ @@ -301,6 +306,7 @@ def _test_multi_os(env): env.expect.that_collection(got).contains_exactly([ struct( name = "bar", + index_url = "", is_exposed = False, is_multiple_versions = False, srcs = [ @@ -318,6 +324,7 @@ def _test_multi_os(env): ), struct( name = "foo", + index_url = "", is_exposed = True, is_multiple_versions = True, srcs = [ @@ -364,6 +371,7 @@ def _test_multi_os_legacy(env): env.expect.that_collection(got).contains_exactly([ struct( name = "bar", + index_url = "", is_exposed = False, is_multiple_versions = False, srcs = [ @@ -381,6 +389,7 @@ def _test_multi_os_legacy(env): ), struct( name = "foo", + index_url = "", is_exposed = True, is_multiple_versions = True, srcs = [ @@ -443,6 +452,7 @@ def _test_env_marker_resolution(env): env.expect.that_collection(got).contains_exactly([ struct( name = "bar", + index_url = "", is_exposed = True, is_multiple_versions = False, srcs = [ @@ -460,6 +470,7 @@ def _test_env_marker_resolution(env): ), struct( name = "foo", + index_url = "", is_exposed = False, is_multiple_versions = False, srcs = [ @@ -489,6 +500,7 @@ def _test_different_package_version(env): env.expect.that_collection(got).contains_exactly([ struct( name = "foo", + index_url = "", is_exposed = True, is_multiple_versions = True, srcs = [ @@ -528,6 +540,7 @@ def _test_different_package_extras(env): env.expect.that_collection(got).contains_exactly([ struct( name = "foo", + index_url = "", is_exposed = True, is_multiple_versions = True, srcs = [ @@ -566,6 +579,7 @@ def _test_optional_hash(env): env.expect.that_collection(got).contains_exactly([ struct( name = "bar", + index_url = "", is_exposed = True, is_multiple_versions = False, srcs = [ @@ -583,6 +597,7 @@ def _test_optional_hash(env): ), struct( name = "foo", + index_url = "", is_exposed = True, is_multiple_versions = False, srcs = [ @@ -611,6 +626,7 @@ def _test_git_sources(env): env.expect.that_collection(got).contains_exactly([ struct( name = "foo", + index_url = "", is_exposed = True, is_multiple_versions = False, srcs = [ @@ -658,6 +674,7 @@ def _test_overlapping_shas_with_index_results(env): }, get_index_urls = lambda _, __: { "foo": struct( + index_url = "https://example.com", sdists = { "5d15t": struct( url = "sdist", @@ -686,9 +703,10 @@ def _test_overlapping_shas_with_index_results(env): env.expect.that_collection(got).contains_exactly([ struct( + name = "foo", + index_url = "https://example.com", is_exposed = True, is_multiple_versions = True, - name = "foo", srcs = [ struct( distribution = "foo", @@ -746,6 +764,7 @@ def _test_get_index_urls_different_versions(env): }, get_index_urls = lambda _, __: { "foo": struct( + index_url = "", sdists = {}, whls = { "deadb11f": struct( @@ -778,9 +797,10 @@ def _test_get_index_urls_different_versions(env): env.expect.that_collection(got).contains_exactly([ struct( + name = "foo", + index_url = "", is_exposed = True, is_multiple_versions = True, - name = "foo", srcs = [ struct( distribution = "foo", @@ -828,6 +848,7 @@ def _test_get_index_urls_single_py_version(env): }, get_index_urls = lambda _, __: { "foo": struct( + index_url = "", sdists = {}, whls = { "deadb11f": struct( @@ -851,9 +872,10 @@ def _test_get_index_urls_single_py_version(env): env.expect.that_collection(got).contains_exactly([ struct( + name = "foo", + index_url = "", is_exposed = True, is_multiple_versions = False, - name = "foo", srcs = [ struct( distribution = "foo", diff --git a/tests/pypi/parse_simpleapi_html/parse_simpleapi_html_tests.bzl b/tests/pypi/parse_simpleapi_html/parse_simpleapi_html_tests.bzl index b96d02f990..f33ba05c91 100644 --- a/tests/pypi/parse_simpleapi_html/parse_simpleapi_html_tests.bzl +++ b/tests/pypi/parse_simpleapi_html/parse_simpleapi_html_tests.bzl @@ -52,7 +52,6 @@ def _test_sdist(env): 'data-requires-python=">=3.7"', ], filename = "foo-0.0.1.tar.gz", - url = "foo", ), struct( filename = "foo-0.0.1.tar.gz", @@ -69,7 +68,6 @@ def _test_sdist(env): 'data-requires-python=">=3.7"', ], filename = "foo-0.0.1.tar.gz", - url = "foo", ), struct( filename = "foo-0.0.1.tar.gz", @@ -83,7 +81,7 @@ def _test_sdist(env): for (input, want) in tests: html = _generate_html(input) - got = parse_simpleapi_html(url = input.url, content = html) + got = parse_simpleapi_html(content = html) env.expect.that_collection(got.sdists).has_size(1) env.expect.that_collection(got.whls).has_size(0) env.expect.that_collection(got.sha256s_by_version).has_size(1) @@ -120,7 +118,6 @@ def _test_whls(env): 'data-core-metadata="sha256=deadb00f"', ], filename = "foo-0.0.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", - url = "foo", ), struct( filename = "foo-0.0.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", @@ -141,7 +138,6 @@ def _test_whls(env): 'data-core-metadata="sha256=deadb00f"', ], filename = "foo-0.0.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", - url = "foo", ), struct( filename = "foo-0.0.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", @@ -161,7 +157,6 @@ def _test_whls(env): 'data-core-metadata="sha256=deadb00f"', ], filename = "foo-0.0.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", - url = "foo", ), struct( filename = "foo-0.0.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", @@ -181,7 +176,6 @@ def _test_whls(env): 'data-dist-info-metadata="sha256=deadb00f"', ], filename = "foo-0.0.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", - url = "foo", ), struct( filename = "foo-0.0.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", @@ -200,7 +194,6 @@ def _test_whls(env): 'data-requires-python=">=3.7"', ], filename = "foo-0.0.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", - url = "foo", ), struct( filename = "foo-0.0.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", @@ -212,122 +205,11 @@ def _test_whls(env): yanked = False, ), ), - ( - struct( - attrs = [ - 'href="../../foo-0.0.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl#sha256=deadbeef"', - 'data-requires-python=">=3.7"', - 'data-dist-info-metadata="sha256=deadb00f"', - ], - filename = "foo-0.0.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", - url = "https://example.org/python-wheels/bar/foo/", - ), - struct( - filename = "foo-0.0.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", - metadata_sha256 = "deadb00f", - metadata_url = "https://example.org/python-wheels/foo-0.0.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata", - sha256 = "deadbeef", - version = "0.0.2", - url = "https://example.org/python-wheels/foo-0.0.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", - yanked = False, - ), - ), - ( - struct( - attrs = [ - 'href="/whl/torch-2.0.0-cp38-cp38-manylinux2014_aarch64.whl#sha256=deadbeef"', - ], - filename = "torch-2.0.0-cp38-cp38-manylinux2014_aarch64.whl", - url = "https://download.pytorch.org/whl/cpu/torch", - ), - struct( - filename = "torch-2.0.0-cp38-cp38-manylinux2014_aarch64.whl", - metadata_sha256 = "", - metadata_url = "", - sha256 = "deadbeef", - url = "https://download.pytorch.org/whl/torch-2.0.0-cp38-cp38-manylinux2014_aarch64.whl", - version = "2.0.0", - yanked = False, - ), - ), - ( - struct( - attrs = [ - 'href="/whl/torch-2.0.0-cp38-cp38-manylinux2014_aarch64.whl#sha256=notdeadbeef"', - ], - filename = "torch-2.0.0-cp38-cp38-manylinux2014_aarch64.whl", - url = "http://download.pytorch.org/whl/cpu/torch", - ), - struct( - filename = "torch-2.0.0-cp38-cp38-manylinux2014_aarch64.whl", - metadata_sha256 = "", - metadata_url = "", - sha256 = "notdeadbeef", - url = "http://download.pytorch.org/whl/torch-2.0.0-cp38-cp38-manylinux2014_aarch64.whl", - version = "2.0.0", - yanked = False, - ), - ), - ( - struct( - attrs = [ - 'href="1.0.0/mypy_extensions-1.0.0-py3-none-any.whl#sha256=deadbeef"', - ], - filename = "mypy_extensions-1.0.0-py3-none-any.whl", - url = "https://example.org/simple/mypy_extensions", - ), - struct( - filename = "mypy_extensions-1.0.0-py3-none-any.whl", - metadata_sha256 = "", - metadata_url = "", - version = "1.0.0", - sha256 = "deadbeef", - url = "https://example.org/simple/mypy_extensions/1.0.0/mypy_extensions-1.0.0-py3-none-any.whl", - yanked = False, - ), - ), - ( - struct( - attrs = [ - 'href="unknown://example.com/mypy_extensions-1.0.0-py3-none-any.whl#sha256=deadbeef"', - ], - filename = "mypy_extensions-1.0.0-py3-none-any.whl", - url = "https://example.org/simple/mypy_extensions", - ), - struct( - filename = "mypy_extensions-1.0.0-py3-none-any.whl", - metadata_sha256 = "", - metadata_url = "", - sha256 = "deadbeef", - version = "1.0.0", - url = "https://example.org/simple/mypy_extensions/unknown://example.com/mypy_extensions-1.0.0-py3-none-any.whl", - yanked = False, - ), - ), - ( - struct( - attrs = [ - 'href="/whl/cpu/torch-2.6.0%2Bcpu-cp39-cp39-manylinux_2_28_aarch64.whl#sha256=deadbeef"', - ], - filename = "torch-2.6.0+cpu-cp39-cp39-manylinux_2_28_aarch64.whl", - url = "https://example.org/", - ), - struct( - filename = "torch-2.6.0+cpu-cp39-cp39-manylinux_2_28_aarch64.whl", - metadata_sha256 = "", - metadata_url = "", - sha256 = "deadbeef", - version = "2.6.0+cpu", - # A URL with % could occur if directly written in requirements. - url = "https://example.org/whl/cpu/torch-2.6.0%2Bcpu-cp39-cp39-manylinux_2_28_aarch64.whl", - yanked = False, - ), - ), ] for (input, want) in tests: html = _generate_html(input) - got = parse_simpleapi_html(url = input.url, content = html) + got = parse_simpleapi_html(content = html) env.expect.that_collection(got.sdists).has_size(0) env.expect.that_collection(got.whls).has_size(1) if not got: diff --git a/tests/pypi/simpleapi_download/simpleapi_download_tests.bzl b/tests/pypi/simpleapi_download/simpleapi_download_tests.bzl index 765e5d0396..391e352e08 100644 --- a/tests/pypi/simpleapi_download/simpleapi_download_tests.bzl +++ b/tests/pypi/simpleapi_download/simpleapi_download_tests.bzl @@ -16,7 +16,7 @@ load("@rules_testing//lib:test_suite.bzl", "test_suite") load("//python/private/pypi:pypi_cache.bzl", "pypi_cache") # buildifier: disable=bzl-visibility -load("//python/private/pypi:simpleapi_download.bzl", "simpleapi_download", "strip_empty_path_segments") # buildifier: disable=bzl-visibility +load("//python/private/pypi:simpleapi_download.bzl", "simpleapi_download") # buildifier: disable=bzl-visibility _tests = [] @@ -37,7 +37,11 @@ def _test_simple(env): ) else: return struct( - output = "data from {}".format(url), + output = struct( + sdists = {"deadbeef": url.strip("/").split("/")[-1]}, + whls = {"deadb33f": url.strip("/").split("/")[-1]}, + sha256s_by_version = {"fizz": url.strip("/").split("/")[-1]}, + ), success = True, ) @@ -65,9 +69,24 @@ def _test_simple(env): "main/foo/", ]) env.expect.that_dict(contents).contains_exactly({ - "bar": "data from main/bar/", - "baz": "data from main/baz/", - "foo": "data from extra/foo/", + "bar": struct( + index_url = "main/bar/", + sdists = {"deadbeef": "bar"}, + sha256s_by_version = {"fizz": "bar"}, + whls = {"deadb33f": "bar"}, + ), + "baz": struct( + index_url = "main/baz/", + sdists = {"deadbeef": "baz"}, + sha256s_by_version = {"fizz": "baz"}, + whls = {"deadb33f": "baz"}, + ), + "foo": struct( + index_url = "extra/foo/", + sdists = {"deadbeef": "foo"}, + sha256s_by_version = {"fizz": "foo"}, + whls = {"deadb33f": "foo"}, + ), }) _tests.append(_test_simple) @@ -95,7 +114,11 @@ def _test_fail(env): ) else: return struct( - output = "data from {}".format(url), + output = struct( + sdists = {}, + whls = {}, + sha256s_by_version = {}, + ), success = True, ) @@ -251,16 +274,6 @@ def _test_download_envsubst_url(env): _tests.append(_test_download_envsubst_url) -def _test_strip_empty_path_segments(env): - env.expect.that_str(strip_empty_path_segments("no/scheme//is/unchanged")).equals("no/scheme//is/unchanged") - env.expect.that_str(strip_empty_path_segments("scheme://with/no/empty/segments")).equals("scheme://with/no/empty/segments") - env.expect.that_str(strip_empty_path_segments("scheme://with//empty/segments")).equals("scheme://with/empty/segments") - env.expect.that_str(strip_empty_path_segments("scheme://with///multiple//empty/segments")).equals("scheme://with/multiple/empty/segments") - env.expect.that_str(strip_empty_path_segments("scheme://with//trailing/slash/")).equals("scheme://with/trailing/slash/") - env.expect.that_str(strip_empty_path_segments("scheme://with/trailing/slashes///")).equals("scheme://with/trailing/slashes/") - -_tests.append(_test_strip_empty_path_segments) - def simpleapi_download_test_suite(name): """Create the test suite. diff --git a/tests/pypi/urllib/BUILD.bazel b/tests/pypi/urllib/BUILD.bazel new file mode 100644 index 0000000000..a6405684a7 --- /dev/null +++ b/tests/pypi/urllib/BUILD.bazel @@ -0,0 +1,3 @@ +load(":urllib_tests.bzl", "urllib_test_suite") + +urllib_test_suite(name = "urllib_tests") diff --git a/tests/pypi/urllib/urllib_tests.bzl b/tests/pypi/urllib/urllib_tests.bzl new file mode 100644 index 0000000000..40c48dc854 --- /dev/null +++ b/tests/pypi/urllib/urllib_tests.bzl @@ -0,0 +1,48 @@ +"" + +load("@rules_testing//lib:test_suite.bzl", "test_suite") +load("//python/private/pypi:urllib.bzl", "urllib") # buildifier: disable=bzl-visibility + +_tests = [] + +def _test_absolute_url(env): + # Already absolute + for already_absolute in [ + "file://foo", + "https://foo.com", + "http://foo.com", + ]: + env.expect.that_str(urllib.absolute_url("https://ignored", already_absolute)).equals(already_absolute) + + # Simple with empty path segments + env.expect.that_str(urllib.absolute_url("https://example.com//", "file.whl")).equals("https://example.com/file.whl") + env.expect.that_str(urllib.absolute_url("https://example.com//a/b//", "../../file.whl")).equals("https://example.com/file.whl") + env.expect.that_str(urllib.absolute_url("https://example.com//a/b//", "/file.whl")).equals("https://example.com/file.whl") + + # Relative URLs + env.expect.that_str(urllib.absolute_url("https://example.com/relative", "file.whl")).equals("https://example.com/relative/file.whl") + env.expect.that_str(urllib.absolute_url("https://example.com/relative/", "file.whl")).equals("https://example.com/relative/file.whl") + env.expect.that_str(urllib.absolute_url("https://example.com/relative/", "../relative/file.whl")).equals("https://example.com/relative/file.whl") + + # Relative URL for files + env.expect.that_str(urllib.absolute_url("file://{PYPI_BAZEL_WORKSPACE_ROOT}", "vendor/distro/file.whl")).equals("file://{PYPI_BAZEL_WORKSPACE_ROOT}/vendor/distro/file.whl") + +_tests.append(_test_absolute_url) + +def _test_strip_empty_path_segments(env): + env.expect.that_str(urllib.strip_empty_path_segments("no/scheme//is/unchanged")).equals("no/scheme//is/unchanged") + env.expect.that_str(urllib.strip_empty_path_segments("scheme://with/no/empty/segments")).equals("scheme://with/no/empty/segments") + env.expect.that_str(urllib.strip_empty_path_segments("scheme://with//empty/segments")).equals("scheme://with/empty/segments") + env.expect.that_str(urllib.strip_empty_path_segments("scheme://with///multiple//empty/segments")).equals("scheme://with/multiple/empty/segments") + env.expect.that_str(urllib.strip_empty_path_segments("scheme://with//trailing/slash/")).equals("scheme://with/trailing/slash/") + env.expect.that_str(urllib.strip_empty_path_segments("scheme://with/trailing/slashes///")).equals("scheme://with/trailing/slashes/") + +_tests.append(_test_strip_empty_path_segments) + +def urllib_test_suite(name): + """Create the test suite. + + Args: + name: the name of the test suite + """ + test_suite(name = name, basic_tests = _tests) From be2f16e36ef44792cfdeee1015408e3b88c5814f Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Tue, 10 Mar 2026 07:51:21 -0700 Subject: [PATCH 656/922] tests: add bazel 9 testing, use latest released (not rc) for bazel (#3650) This runs the default test jobs using the test matrix so that Bazel 9 is better covered by testing. The main thing needed is setting `--incompatible_strict_action_env=false`. This is enabled in Bazel 9, but breaks Windows. See https://github.com/bazel-contrib/rules_python/issues/3655 for details. It also changes to using `N.x` instead of `N.*` for specifying the Bazel version. The difference is `x` matches the latest released version, while `*` matches the latest released _or release candidate_. Since release candidates can have regressions, and we have a separate job for RCs, use the latest released version. Along the way, mark the last_rc job as soft-fail --- .bazelci/presubmit.yml | 17 ++++++++++++----- .bazelrc | 5 +++++ .bcr/presubmit.yml | 2 +- 3 files changed, 18 insertions(+), 6 deletions(-) diff --git a/.bazelci/presubmit.yml b/.bazelci/presubmit.yml index dc54ea3eae..2f923016f8 100644 --- a/.bazelci/presubmit.yml +++ b/.bazelci/presubmit.yml @@ -113,12 +113,14 @@ buildifier: matrix: + # Keep in sync with .bcr/presubmit.yml platform: - ubuntu2204 - debian11 - macos_arm64 - windows - bazel: [7.*, 8.*, 9.*] + # Keep in sync with .bcr/presubmit.yml + bazel: [7.x, 8.x, 9.x] tasks: # Keep in sync with .bcr/presubmit.yml @@ -187,13 +189,15 @@ tasks: bazel: 7.x ubuntu: <<: *reusable_config - name: "Default: Ubuntu" + name: "Default: Ubuntu, Bazel {bazel}" platform: ubuntu2204 + bazel: ${{ bazel }} ubuntu_upcoming: <<: *reusable_config name: "Default: Ubuntu, upcoming Bazel" platform: ubuntu2204 bazel: last_rc + # This is an advisory job; doesn't block merges # RCs may have regressions, so don't fail our CI on them soft_fail: - exit_status: 1 @@ -250,16 +254,19 @@ tasks: debian: <<: *reusable_config - name: "Default: Debian" + name: "Default: Debian, Bazel {bazel}" platform: debian11 + bazel: ${{ bazel }} macos_arm64: <<: *reusable_config - name: "Default: MacOS" + name: "Default: MacOS, Bazel {bazel}" platform: macos_arm64 + bazel: ${{ bazel }} windows: <<: *reusable_config - name: "Default: Windows" + name: "Default: Windows, Bazel {bazel}" platform: windows + bazel: ${{ bazel }} test_flags: - "--test_tag_filters=-integration-test,-fix-windows" rbe_min: diff --git a/.bazelrc b/.bazelrc index 24676574e6..6d7e58a9a3 100644 --- a/.bazelrc +++ b/.bazelrc @@ -21,6 +21,11 @@ common --incompatible_disallow_struct_provider_syntax # Makes Bazel 7 act more like Bazel 8 common --incompatible_use_plus_in_repo_names +# Needed to make Windows with a py_binary in data deps work. The Bazel launcher +# is used, which falls back to finding python.exe on PATH to bootstrap. +# See https://github.com/bazel-contrib/rules_python/issues/3655 +common --incompatible_strict_action_env=false + # Windows makes use of runfiles for some rules build --enable_runfiles diff --git a/.bcr/presubmit.yml b/.bcr/presubmit.yml index 1ad61c7f5a..a38c6bace5 100644 --- a/.bcr/presubmit.yml +++ b/.bcr/presubmit.yml @@ -16,7 +16,7 @@ bcr_test_module: module_path: "examples/bzlmod" matrix: platform: ["debian11", "macos", "ubuntu2204", "windows"] - bazel: [7.*, 8.*, 9.*] + bazel: [7.x, 8.x, 9.x] tasks: run_tests: name: "Run test module" From 2c50dafcef2c7e3ed7012c3146657948611d9bde Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Thu, 12 Mar 2026 20:05:55 -0700 Subject: [PATCH 657/922] chore: use bazel 9 by default (#3662) Updates the project bazel version and bzlmod examples to use Bazel 9 Along the way, upgrade rules_go to 0.60.0; this is a dev-only dependency. This is needed because Bazel 9 removed CcInfo, and earlier rules_go versions refer to the builtin symbol. Also remove the test to check the bazel version. It doesn't do much useful. --- .bazelversion | 2 +- MODULE.bazel | 2 +- examples/bzlmod/.bazelversion | 2 +- tests/BUILD.bazel | 28 ------------------- .../integration/local_toolchains/BUILD.bazel | 5 +++- .../local_toolchains/local_runtime_test.py | 9 ++++-- 6 files changed, 14 insertions(+), 34 deletions(-) diff --git a/.bazelversion b/.bazelversion index c6b7980b68..512e4c889e 100644 --- a/.bazelversion +++ b/.bazelversion @@ -1 +1 @@ -8.x +9.x diff --git a/MODULE.bazel b/MODULE.bazel index 326bb5b78e..a8af3be582 100644 --- a/MODULE.bazel +++ b/MODULE.bazel @@ -226,7 +226,7 @@ bazel_dep(name = "another_module", version = "0", dev_dependency = True) # Extra gazelle plugin deps so that WORKSPACE.bzlmod can continue including it for e2e tests. # We use `WORKSPACE.bzlmod` because it is impossible to have dev-only local overrides. -bazel_dep(name = "rules_go", version = "0.41.0", dev_dependency = True, repo_name = "io_bazel_rules_go") +bazel_dep(name = "rules_go", version = "0.60.0", dev_dependency = True, repo_name = "io_bazel_rules_go") internal_dev_deps = use_extension( "//python/private:internal_dev_deps.bzl", diff --git a/examples/bzlmod/.bazelversion b/examples/bzlmod/.bazelversion index 35907cd9ca..512e4c889e 100644 --- a/examples/bzlmod/.bazelversion +++ b/examples/bzlmod/.bazelversion @@ -1 +1 @@ -7.x +9.x diff --git a/tests/BUILD.bazel b/tests/BUILD.bazel index 0fb8e88135..e7dbef65d8 100644 --- a/tests/BUILD.bazel +++ b/tests/BUILD.bazel @@ -1,6 +1,4 @@ load("@bazel_skylib//rules:build_test.bzl", "build_test") -load("@rules_shell//shell:sh_test.bzl", "sh_test") -load("//:version.bzl", "BAZEL_VERSION") package(default_visibility = ["//visibility:public"]) @@ -27,29 +25,3 @@ build_test( "//python/entry_points:py_console_script_binary_bzl", ], ) - -genrule( - name = "assert_bazelversion", - srcs = ["//:.bazelversion"], - outs = ["assert_bazelversion_test.sh"], - cmd = """\ -set -o errexit -o nounset -o pipefail -current=$$(cat "$(execpath //:.bazelversion)") -cat > "$@" <&2 echo "ERROR: current bazel version '$${{current}}' is not the expected '{expected}'" - exit 1 -fi -EOF -""".format( - expected = BAZEL_VERSION, - ), - executable = True, -) - -sh_test( - name = "assert_bazelversion_test", - srcs = [":assert_bazelversion_test.sh"], -) diff --git a/tests/integration/local_toolchains/BUILD.bazel b/tests/integration/local_toolchains/BUILD.bazel index bf47316027..20ee7bcfe6 100644 --- a/tests/integration/local_toolchains/BUILD.bazel +++ b/tests/integration/local_toolchains/BUILD.bazel @@ -24,7 +24,10 @@ py_test( "//:py": "local", }, # Make this test better respect pyenv - env_inherit = ["PYENV_VERSION"], + env_inherit = [ + "PYENV_VERSION", + "PATH", + ], ) py_test( diff --git a/tests/integration/local_toolchains/local_runtime_test.py b/tests/integration/local_toolchains/local_runtime_test.py index 0a0d6bedeb..220ceaead4 100644 --- a/tests/integration/local_toolchains/local_runtime_test.py +++ b/tests/integration/local_toolchains/local_runtime_test.py @@ -16,15 +16,20 @@ def test_python_from_path_used(self): # that wouldn't be reflected when sub-shells are run later. shell_path = shutil.which("python3") + if shell_path is None: + self.fail( + "which(python3) returned None.\n" + f"PATH={os.environ.get('PATH')}" + ) + # We call the interpreter and print its executable because of # things like pyenv: they install a shim that re-execs python. # The shim is e.g. /home/user/.pyenv/shims/python3, which then # runs e.g. /usr/bin/python3 with tempfile.TemporaryDirectory() as temp_dir: file_path = os.path.join(temp_dir, "info.py") - with open(file_path, 'w') as f: + with open(file_path, "w") as f: f.write( - """ + """ import sys print(sys.executable) print(sys._base_executable) From d1279c3d2c5fb0416b997c71245cfdbcd167dfa3 Mon Sep 17 00:00:00 2001 From: Jonathan Block Date: Fri, 13 Mar 2026 00:15:59 -0400 Subject: [PATCH 658/922] Link a quickstart screen recording for rules_python beginners into docs (#3658) aignas suggested I add a link to this YouTube video somewhere in the docs: https://bazelbuild.slack.com/archives/CA306CEV6/p1773133824891709?thread_ts=1773089164.263869&cid=CA306CEV6 --- docs/getting-started.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/docs/getting-started.md b/docs/getting-started.md index d81d72f590..6494b37c51 100644 --- a/docs/getting-started.md +++ b/docs/getting-started.md @@ -11,6 +11,8 @@ For more detailed information about configuring `rules_python`, see: * [Configuring third-party dependencies (pip/PyPI)](./pypi/index) * [API docs](api/index) +A [screen recording](https://www.youtube.com/watch?v=Xtuh-WipOnk) on configuring rules_python for beginners is also available. + ## Including dependencies The first step to using `rules_python` is to add the dependency to From 32e331ba64769236221174cc8332069bf5623694 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Fri, 13 Mar 2026 18:12:29 -0700 Subject: [PATCH 659/922] chore!: enable --windows_enable_symlinks by default (#3663) Supporting Windows without full symlink support is quite difficult because we rely on symlinks in many places. Since Windows support is mediocre already, just require symlinks to be enabled. Symlink support in Windows has been around for a long time now and is available via non-admin mechanisms such as DevMode. --- .bazelrc | 4 ++++ CHANGELOG.md | 7 +++++++ 2 files changed, 11 insertions(+) diff --git a/.bazelrc b/.bazelrc index 6d7e58a9a3..4c3f5b3a12 100644 --- a/.bazelrc +++ b/.bazelrc @@ -5,6 +5,10 @@ # To update the file, execute import %workspace%/.bazelrc.deleted_packages +# Symlinks are used extensively because of runfiles, venvs, and other reasons. +# Supporting otherwise is very complex with little benefit. +startup --windows_enable_symlinks + test --test_output=errors # Do NOT implicitly create empty __init__.py files in the runfiles tree. diff --git a/CHANGELOG.md b/CHANGELOG.md index 189e46f3f8..8ce7291a62 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -58,6 +58,13 @@ END_UNRELEASED_TEMPLATE {#v0-0-0-changed} ### Changed + +**Breaking** +* {obj}`--windows_enable_symlinks` is required. Add `startup + --windows_enable_symlinks` to your `.bazelrc` to enable Bazel using full + symlink support on Windows. + +Other changes: * (pypi) Update dependencies used for `compile_pip_requirements`, building sdists in the `whl_library` rule and fetching wheels using `pip`. From 798df3f20f42f3f11b8b4d371b8c7d40fa9f3b69 Mon Sep 17 00:00:00 2001 From: Ignas Anikevicius <240938+aignas@users.noreply.github.com> Date: Sat, 14 Mar 2026 14:50:38 +0900 Subject: [PATCH 660/922] fix(pypi): return yank reason from SimpleAPI HTML (#3656) This makes the logic in the parser a little bit more sophisticated, but we also start handling the yank reason. With this the yank reason will be present in the logs, which should be a usability improvement. The `yanked` attribute will be None if the package is not yanked and a string value if it is yanked. An empty string value should be treated as "yanked with no reason provided". This implementation assumes that we have HTML escaped sequences as tag values. It also un-escapes them when returning the strings. The possibilities that it gives us are, but out of scope for this PR: - Use the `data-requires-python` to potentially discard any Python packages that are unsupported in the `select_whl` function. - Also retrieve the provenance attributes if they are supported. Work towards #260. Work towards #2731. --- python/private/pypi/parse_requirements.bzl | 10 +- python/private/pypi/parse_simpleapi_html.bzl | 198 +++++++++++++----- tests/pypi/hub_builder/hub_builder_tests.bzl | 8 +- .../parse_requirements_tests.bzl | 66 +++--- .../parse_simpleapi_html_tests.bzl | 58 ++++- 5 files changed, 236 insertions(+), 104 deletions(-) diff --git a/python/private/pypi/parse_requirements.bzl b/python/private/pypi/parse_requirements.bzl index acc35b3208..78b6662d08 100644 --- a/python/private/pypi/parse_requirements.bzl +++ b/python/private/pypi/parse_requirements.bzl @@ -267,7 +267,7 @@ def _package_srcs( url = "", filename = "", sha256 = "", - yanked = False, + yanked = None, ) req_line = r.srcs.requirement_line else: @@ -379,7 +379,7 @@ def _add_dists(*, requirement, index_urls, target_platform, logger = None): url = requirement.srcs.url, filename = requirement.srcs.filename, sha256 = requirement.srcs.shas[0] if requirement.srcs.shas else "", - yanked = False, + yanked = None, ) return dist, False @@ -403,12 +403,12 @@ def _add_dists(*, requirement, index_urls, target_platform, logger = None): # See https://packaging.python.org/en/latest/specifications/simple-repository-api/#adding-yank-support-to-the-simple-api maybe_whl = index_urls.whls.get(sha256) - if maybe_whl and not maybe_whl.yanked: + if maybe_whl and maybe_whl.yanked == None: whls.append(maybe_whl) continue maybe_sdist = index_urls.sdists.get(sha256) - if maybe_sdist and not maybe_sdist.yanked: + if maybe_sdist and maybe_sdist.yanked == None: sdist = maybe_sdist continue @@ -416,7 +416,7 @@ def _add_dists(*, requirement, index_urls, target_platform, logger = None): yanked = {} for dist in whls + [sdist]: - if dist and dist.yanked: + if dist and dist.yanked != None: yanked.setdefault(dist.yanked, []).append(dist.filename) if yanked: logger.warn(lambda: "\n".join([ diff --git a/python/private/pypi/parse_simpleapi_html.bzl b/python/private/pypi/parse_simpleapi_html.bzl index 6778d3da16..563130791e 100644 --- a/python/private/pypi/parse_simpleapi_html.bzl +++ b/python/private/pypi/parse_simpleapi_html.bzl @@ -26,81 +26,177 @@ def parse_simpleapi_html(*, content): Returns: A list of structs with: - * filename: The filename of the artifact. - * version: The version of the artifact. - * url: The URL to download the artifact. - * sha256: The sha256 of the artifact. - * metadata_sha256: The whl METADATA sha256 if we can download it. If this is - present, then the 'metadata_url' is also present. Defaults to "". - * metadata_url: The URL for the METADATA if we can download it. Defaults to "". + * filename: {type}`str` The filename of the artifact. + * version: {type}`str` The version of the artifact. + * url: {type}`str` The URL to download the artifact. + * sha256: {type}`str` The sha256 of the artifact. + * metadata_sha256: {type}`str` The whl METADATA sha256 if we can download it. If this is + present, then the 'metadata_url' is also present. Defaults to "". + * metadata_url: {type}`str` The URL for the METADATA if we can download it. Defaults to "". + * yanked: {type}`str | None` the yank reason if the package is yanked. If it is not yanked, + then it will be `None`. An empty string yank reason means that the package is yanked but + the reason is not provided. """ sdists = {} whls = {} - lines = content.split("= (2, 0): # We don't expect to have version 2.0 here, but have this check in place just in case. # https://packaging.python.org/en/latest/specifications/simple-repository-api/#versioning-pypi-s-simple-api fail("Unsupported API version: {}".format(api_version)) - # Each line follows the following pattern - # filename
- sha256s_by_version = {} - for line in lines[1:]: - dist_url, _, tail = line.partition("#sha256=") + # 2. Iterate using find() to avoid huge list allocations from .split(" + tag_end = content.find(">", start_tag) + end_tag = content.find("", tag_end) + if tag_end == -1 or end_tag == -1: + break + + # Extract only the necessary slices + attr_part = content[start_tag + 3:tag_end] + filename = content[tag_end + 1:end_tag].strip() + + # Update cursor for next iteration + cursor = end_tag + 4 + + # 3. Efficient Attribute Parsing + attrs = _parse_attrs(attr_part) + href = attrs.get("href", "") + if not href: + continue - sha256, _, tail = tail.partition("\"") + dist_url, _, sha256 = href.partition("#sha256=") - # See https://packaging.python.org/en/latest/specifications/simple-repository-api/#adding-yank-support-to-the-simple-api - yanked = "data-yanked" in line + # Handle Yanked status + yanked = None + if "data-yanked" in attrs: + yanked = _unescape_pypi_html(attrs["data-yanked"]) - head, _, _ = tail.rpartition("") - maybe_metadata, _, filename = head.rpartition(">") version = version_from_filename(filename) sha256s_by_version.setdefault(version, []).append(sha256) + # 4. Optimized Metadata Check (PEP 714) metadata_sha256 = "" metadata_url = "" - for metadata_marker in ["data-core-metadata", "data-dist-info-metadata"]: - metadata_marker = metadata_marker + "=\"sha256=" - if metadata_marker in maybe_metadata: - # Implement https://peps.python.org/pep-0714/ - _, _, tail = maybe_metadata.partition(metadata_marker) - metadata_sha256, _, _ = tail.partition("\"") - metadata_url = dist_url + ".metadata" - break + + # Dist-info is more common in modern PyPI + m_val = attrs.get("data-dist-info-metadata") or attrs.get("data-core-metadata") + if m_val and m_val != "false": + _, _, metadata_sha256 = m_val.partition("sha256=") + metadata_url = dist_url + ".metadata" + + # 5. Result object + dist = struct( + filename = filename, + version = version, + url = dist_url, + sha256 = sha256, + metadata_sha256 = metadata_sha256, + metadata_url = metadata_url, + yanked = yanked, + ) if filename.endswith(".whl"): - whls[sha256] = struct( - filename = filename, - version = version, - url = dist_url, - sha256 = sha256, - metadata_sha256 = metadata_sha256, - metadata_url = metadata_url, - yanked = yanked, - ) + whls[sha256] = dist else: - sdists[sha256] = struct( - filename = filename, - version = version, - url = dist_url, - sha256 = sha256, - metadata_sha256 = "", - metadata_url = "", - yanked = yanked, - ) + sdists[sha256] = dist return struct( sdists = sdists, whls = whls, sha256s_by_version = sha256s_by_version, ) + +def _parse_attrs(attr_string): + """Parses attributes from a pre-sliced string.""" + attrs = {} + parts = attr_string.split('"') + + for i in range(0, len(parts) - 1, 2): + raw_key = parts[i].strip() + if not raw_key: + continue + + key_parts = raw_key.split(" ") + current_key = key_parts[-1].rstrip("=") + + # Batch handle booleans + for j in range(len(key_parts) - 1): + b = key_parts[j].strip() + if b: + attrs[b] = "" + + attrs[current_key] = parts[i + 1] + + # Final trailing boolean check + last = parts[-1].strip() + if last: + for b in last.split(" "): + if b: + attrs[b] = "" + return attrs + +def _unescape_pypi_html(text): + """Unescape HTML text. + + Decodes standard HTML entities used in the Simple API. + Specifically targets characters used in URLs and attribute values. + + Args: + text: {type}`str` The text to replace. + + Returns: + A string with unescaped characters + """ + + # 1. Short circuit for the most common case + if not text or "&" not in text: + return text + + # 2. Check for the most frequent PEP 503 entities first (version constraints). + # Re-ordering based on frequency reduces unnecessary checks for rare entities. + if ">" in text: + text = text.replace(">", ">") + if "<" in text: + text = text.replace("<", "<") + + # 3. Grouped check for numeric entities. + # If '&#' isn't there, we skip 4 distinct string scans. + if "&#" in text: + if "'" in text: + text = text.replace("'", "'") + if "'" in text: + text = text.replace("'", "'") + if " " in text: + text = text.replace(" ", "\n") + if " " in text: + text = text.replace(" ", "\r") + + if """ in text: + text = text.replace(""", '"') + + # 4. Handle ampersands last to prevent double-decoding. + if "&" in text: + text = text.replace("&", "&") + + return text diff --git a/tests/pypi/hub_builder/hub_builder_tests.bzl b/tests/pypi/hub_builder/hub_builder_tests.bzl index 27040d36d7..170e12c4e4 100644 --- a/tests/pypi/hub_builder/hub_builder_tests.bzl +++ b/tests/pypi/hub_builder/hub_builder_tests.bzl @@ -777,7 +777,7 @@ def _test_simple_get_index(env): "plat_pkg": struct( whls = { "deadb44f": struct( - yanked = False, + yanked = None, filename = "plat-pkg-0.0.4-py3-none-linux_x86_64.whl", sha256 = "deadb44f", url = "example2.org/index/plat_pkg/", @@ -792,7 +792,7 @@ def _test_simple_get_index(env): "simple": struct( whls = { "deadb00f": struct( - yanked = False, + yanked = None, filename = "simple-0.0.1-py3-none-any.whl", sha256 = "deadb00f", url = "example2.org", @@ -800,7 +800,7 @@ def _test_simple_get_index(env): }, sdists = { "deadbeef": struct( - yanked = False, + yanked = None, filename = "simple-0.0.1.tar.gz", sha256 = "deadbeef", url = "example.org", @@ -811,7 +811,7 @@ def _test_simple_get_index(env): "some_other_pkg": struct( whls = { "deadb33f": struct( - yanked = False, + yanked = None, filename = "some-other-pkg-0.0.1-py3-none-any.whl", sha256 = "deadb33f", url = "example2.org/index/some_other_pkg/", diff --git a/tests/pypi/parse_requirements/parse_requirements_tests.bzl b/tests/pypi/parse_requirements/parse_requirements_tests.bzl index 0d03e94467..bea8ac5f78 100644 --- a/tests/pypi/parse_requirements/parse_requirements_tests.bzl +++ b/tests/pypi/parse_requirements/parse_requirements_tests.bzl @@ -143,7 +143,7 @@ def _test_simple(env): url = "", filename = "", sha256 = "", - yanked = False, + yanked = None, ), ], ), @@ -174,7 +174,7 @@ def _test_direct_urls_integration(env): sha256 = "", target_platforms = ["osx_x86_64"], url = "https://github.com/org/foo/downloads/foo-1.1.tar.gz", - yanked = False, + yanked = None, ), struct( distribution = "foo", @@ -184,7 +184,7 @@ def _test_direct_urls_integration(env): sha256 = "", target_platforms = ["linux_x86_64"], url = "https://some-url/package.whl", - yanked = False, + yanked = None, ), ], ), @@ -216,7 +216,7 @@ def _test_direct_urls_no_extract(env): sha256 = "", target_platforms = ["osx_x86_64"], url = "", - yanked = False, + yanked = None, ), struct( distribution = "foo", @@ -226,7 +226,7 @@ def _test_direct_urls_no_extract(env): sha256 = "", target_platforms = ["linux_x86_64"], url = "", - yanked = False, + yanked = None, ), ], ), @@ -258,7 +258,7 @@ def _test_extra_pip_args(env): url = "", filename = "", sha256 = "", - yanked = False, + yanked = None, ), ], ), @@ -287,7 +287,7 @@ def _test_dupe_requirements(env): url = "", filename = "", sha256 = "", - yanked = False, + yanked = None, ), ], ), @@ -318,7 +318,7 @@ def _test_multi_os(env): url = "", filename = "", sha256 = "", - yanked = False, + yanked = None, ), ], ), @@ -336,7 +336,7 @@ def _test_multi_os(env): url = "", filename = "", sha256 = "", - yanked = False, + yanked = None, ), struct( distribution = "foo", @@ -346,7 +346,7 @@ def _test_multi_os(env): url = "", filename = "", sha256 = "", - yanked = False, + yanked = None, ), ], ), @@ -383,7 +383,7 @@ def _test_multi_os_legacy(env): url = "", filename = "", sha256 = "", - yanked = False, + yanked = None, ), ], ), @@ -401,7 +401,7 @@ def _test_multi_os_legacy(env): url = "", filename = "", sha256 = "", - yanked = False, + yanked = None, ), struct( distribution = "foo", @@ -411,7 +411,7 @@ def _test_multi_os_legacy(env): url = "", filename = "", sha256 = "", - yanked = False, + yanked = None, ), ], ), @@ -464,7 +464,7 @@ def _test_env_marker_resolution(env): url = "", filename = "", sha256 = "", - yanked = False, + yanked = None, ), ], ), @@ -482,7 +482,7 @@ def _test_env_marker_resolution(env): url = "", filename = "", sha256 = "", - yanked = False, + yanked = None, ), ], ), @@ -512,7 +512,7 @@ def _test_different_package_version(env): url = "", filename = "", sha256 = "", - yanked = False, + yanked = None, ), struct( distribution = "foo", @@ -522,7 +522,7 @@ def _test_different_package_version(env): url = "", filename = "", sha256 = "", - yanked = False, + yanked = None, ), ], ), @@ -552,7 +552,7 @@ def _test_different_package_extras(env): url = "", filename = "", sha256 = "", - yanked = False, + yanked = None, ), struct( distribution = "foo", @@ -562,7 +562,7 @@ def _test_different_package_extras(env): url = "", filename = "", sha256 = "", - yanked = False, + yanked = None, ), ], ), @@ -591,7 +591,7 @@ def _test_optional_hash(env): url = "https://example.org/bar-0.0.4.whl", filename = "bar-0.0.4.whl", sha256 = "", - yanked = False, + yanked = None, ), ], ), @@ -609,7 +609,7 @@ def _test_optional_hash(env): url = "https://example.org/foo-0.0.5.whl", filename = "foo-0.0.5.whl", sha256 = "deadbeef", - yanked = False, + yanked = None, ), ], ), @@ -638,7 +638,7 @@ def _test_git_sources(env): url = "", filename = "", sha256 = "", - yanked = False, + yanked = None, ), ], ), @@ -680,7 +680,7 @@ def _test_overlapping_shas_with_index_results(env): url = "sdist", sha256 = "5d15t", filename = "foo-0.0.1.tar.gz", - yanked = False, + yanked = None, ), }, whls = { @@ -688,13 +688,13 @@ def _test_overlapping_shas_with_index_results(env): url = "super2", sha256 = "deadb11f", filename = "foo-0.0.1-py3-none-macosx_14_0_x86_64.whl", - yanked = False, + yanked = None, ), "deadbaaf": struct( url = "super2", sha256 = "deadbaaf", filename = "foo-0.0.1-py3-none-any.whl", - yanked = False, + yanked = None, ), }, ), @@ -716,7 +716,7 @@ def _test_overlapping_shas_with_index_results(env): sha256 = "deadbaaf", target_platforms = ["cp39_linux_x86_64"], url = "super2", - yanked = False, + yanked = None, ), struct( distribution = "foo", @@ -726,7 +726,7 @@ def _test_overlapping_shas_with_index_results(env): sha256 = "deadb11f", target_platforms = ["cp39_osx_x86_64"], url = "super2", - yanked = False, + yanked = None, ), ], ), @@ -771,13 +771,13 @@ def _test_get_index_urls_different_versions(env): url = "super2", sha256 = "deadb11f", filename = "foo-0.0.2-py3-none-any.whl", - yanked = False, + yanked = None, ), "deadbaaf": struct( url = "super2", sha256 = "deadbaaf", filename = "foo-0.0.1-py3-none-any.whl", - yanked = False, + yanked = None, ), }, ), @@ -810,7 +810,7 @@ def _test_get_index_urls_different_versions(env): sha256 = "", target_platforms = ["cp39_linux_x86_64"], url = "", - yanked = False, + yanked = None, ), struct( distribution = "foo", @@ -820,7 +820,7 @@ def _test_get_index_urls_different_versions(env): sha256 = "deadb11f", target_platforms = ["cp310_linux_x86_64"], url = "super2", - yanked = False, + yanked = None, ), ], ), @@ -855,7 +855,7 @@ def _test_get_index_urls_single_py_version(env): url = "super2", sha256 = "deadb11f", filename = "foo-0.0.2-py3-none-any.whl", - yanked = False, + yanked = None, ), }, ), @@ -885,7 +885,7 @@ def _test_get_index_urls_single_py_version(env): sha256 = "deadb11f", target_platforms = ["cp310_linux_x86_64"], url = "super2", - yanked = False, + yanked = None, ), ], ), diff --git a/tests/pypi/parse_simpleapi_html/parse_simpleapi_html_tests.bzl b/tests/pypi/parse_simpleapi_html/parse_simpleapi_html_tests.bzl index f33ba05c91..f72d61371c 100644 --- a/tests/pypi/parse_simpleapi_html/parse_simpleapi_html_tests.bzl +++ b/tests/pypi/parse_simpleapi_html/parse_simpleapi_html_tests.bzl @@ -57,7 +57,7 @@ def _test_sdist(env): filename = "foo-0.0.1.tar.gz", sha256 = "deadbeefasource", url = "https://example.org/full-url/foo-0.0.1.tar.gz", - yanked = False, + yanked = None, version = "0.0.1", ), ), @@ -65,7 +65,25 @@ def _test_sdist(env): struct( attrs = [ 'href="https://example.org/full-url/foo-0.0.1.tar.gz#sha256=deadbeefasource"', - 'data-requires-python=">=3.7"', + 'data-requires-python=">=3.7"', + "data-yanked", + ], + filename = "foo-0.0.1.tar.gz", + ), + struct( + filename = "foo-0.0.1.tar.gz", + sha256 = "deadbeefasource", + url = "https://example.org/full-url/foo-0.0.1.tar.gz", + version = "0.0.1", + yanked = "", + ), + ), + ( + struct( + attrs = [ + 'href="https://example.org/full-url/foo-0.0.1.tar.gz#sha256=deadbeefasource"', + 'data-requires-python=">=3.7"', + "data-yanked=\"Something with "quotes" over two lines\"", ], filename = "foo-0.0.1.tar.gz", ), @@ -74,7 +92,25 @@ def _test_sdist(env): sha256 = "deadbeefasource", url = "https://example.org/full-url/foo-0.0.1.tar.gz", version = "0.0.1", - yanked = False, + # NOTE @aignas 2026-03-09: we preserve the white space + yanked = "Something \nwith \"quotes\"\nover two lines", + ), + ), + ( + struct( + attrs = [ + 'href="https://example.org/full-url/foo-0.0.1.tar.gz#sha256=deadbeefasource"', + 'data-requires-python=">=3.7"', + 'data-yanked=""', + ], + filename = "foo-0.0.1.tar.gz", + ), + struct( + filename = "foo-0.0.1.tar.gz", + sha256 = "deadbeefasource", + url = "https://example.org/full-url/foo-0.0.1.tar.gz", + version = "0.0.1", + yanked = "", ), ), ] @@ -94,7 +130,7 @@ def _test_sdist(env): filename = subjects.str, sha256 = subjects.str, url = subjects.str, - yanked = subjects.bool, + yanked = subjects.str, version = subjects.str, ), ) @@ -126,14 +162,14 @@ def _test_whls(env): sha256 = "deadbeef", url = "https://example.org/full-url/foo-0.0.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", version = "0.0.2", - yanked = False, + yanked = None, ), ), ( struct( attrs = [ 'href="https://example.org/full-url/foo-0.0.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl#sha256=deadbeef"', - 'data-requires-python=">=3.7"', + 'data-requires-python=">=3.7"', 'data-dist-info-metadata="sha256=deadb00f"', 'data-core-metadata="sha256=deadb00f"', ], @@ -146,7 +182,7 @@ def _test_whls(env): sha256 = "deadbeef", url = "https://example.org/full-url/foo-0.0.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", version = "0.0.2", - yanked = False, + yanked = None, ), ), ( @@ -165,7 +201,7 @@ def _test_whls(env): sha256 = "deadbeef", version = "0.0.2", url = "https://example.org/full-url/foo-0.0.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", - yanked = False, + yanked = None, ), ), ( @@ -184,7 +220,7 @@ def _test_whls(env): sha256 = "deadbeef", version = "0.0.2", url = "https://example.org/full-url/foo-0.0.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", - yanked = False, + yanked = None, ), ), ( @@ -202,7 +238,7 @@ def _test_whls(env): sha256 = "deadbeef", url = "https://example.org/full-url/foo-0.0.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", version = "0.0.2", - yanked = False, + yanked = None, ), ), ] @@ -223,7 +259,7 @@ def _test_whls(env): metadata_url = subjects.str, sha256 = subjects.str, url = subjects.str, - yanked = subjects.bool, + yanked = subjects.str, version = subjects.str, ), ) From 3ed161f1a1b31fac381f0cf1739d2aecebffbc75 Mon Sep 17 00:00:00 2001 From: Ignas Anikevicius <240938+aignas@users.noreply.github.com> Date: Sat, 14 Mar 2026 15:24:26 +0900 Subject: [PATCH 661/922] feat(pypi): store PyPI results as facts v2 (#3654) This PR adds functionality needed to write data that we find useful on the SimpleAPI responses to the lock file. I.e. this will no longer connect to the network if it can find the necessary information in the lock file. Superseeds #3559 Fixes #2731 --------- Co-authored-by: Richard Levasseur --- CHANGELOG.md | 5 +- python/private/pypi/BUILD.bazel | 3 + python/private/pypi/extension.bzl | 21 +- python/private/pypi/hub_builder.bzl | 8 +- python/private/pypi/parse_requirements.bzl | 18 +- python/private/pypi/pypi_cache.bzl | 235 ++++++++++++++++- python/private/pypi/simpleapi_download.bzl | 14 +- tests/pypi/hub_builder/hub_builder_tests.bzl | 7 +- tests/pypi/pypi_cache/BUILD.bazel | 5 + tests/pypi/pypi_cache/pypi_cache_tests.bzl | 239 ++++++++++++++++++ .../simpleapi_download_tests.bzl | 20 +- 11 files changed, 537 insertions(+), 38 deletions(-) create mode 100644 tests/pypi/pypi_cache/BUILD.bazel create mode 100644 tests/pypi/pypi_cache/pypi_cache_tests.bzl diff --git a/CHANGELOG.md b/CHANGELOG.md index 8ce7291a62..6e0f207e2a 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -77,7 +77,10 @@ Other changes: {#v0-0-0-added} ### Added -* Nothing added. +* (pypi) Write SimpleAPI contents to the `MODULE.bazel.lock` file if using + {obj}`experimental_index_url` which should speed up consecutive initializations and should no + longer require the network access if the cache is hydrated. + Implements [#2731](https://github.com/bazel-contrib/rules_python/issues/2731). {#v1-9-0} ## [1.9.0] - 2026-02-21 diff --git a/python/private/pypi/BUILD.bazel b/python/private/pypi/BUILD.bazel index 34318d7920..6b4822333c 100644 --- a/python/private/pypi/BUILD.bazel +++ b/python/private/pypi/BUILD.bazel @@ -359,6 +359,9 @@ bzl_library( bzl_library( name = "pypi_cache_bzl", srcs = ["pypi_cache.bzl"], + deps = [ + ":version_from_filename_bzl", + ], ) bzl_library( diff --git a/python/private/pypi/extension.bzl b/python/private/pypi/extension.bzl index 5fded728bf..f68596b845 100644 --- a/python/private/pypi/extension.bzl +++ b/python/private/pypi/extension.bzl @@ -225,7 +225,7 @@ You cannot use both the additive_build_content and additive_build_content_file a # dict[str repo, HubBuilder] # See `hub_builder.bzl%hub_builder()` for `HubBuilder` pip_hub_map = {} - simpleapi_cache = pypi_cache() + simpleapi_cache = pypi_cache(mctx = module_ctx) for mod in module_ctx.modules: for pip_attr in mod.tags.parse: @@ -293,6 +293,7 @@ You cannot use both the additive_build_content and additive_build_content_file a config = config, exposed_packages = exposed_packages, extra_aliases = extra_aliases, + facts = simpleapi_cache.get_facts(), hub_group_map = hub_group_map, hub_whl_map = hub_whl_map, whl_libraries = whl_libraries, @@ -372,7 +373,11 @@ def _pip_impl(module_ctx): module_ctx: module contents """ - mods = parse_modules(module_ctx, enable_pipstar = rp_config.enable_pipstar, enable_pipstar_extract = rp_config.enable_pipstar and rp_config.bazel_8_or_later) + mods = parse_modules( + module_ctx, + enable_pipstar = rp_config.enable_pipstar, + enable_pipstar_extract = rp_config.enable_pipstar and rp_config.bazel_8_or_later, + ) # Build all of the wheel modifications if the tag class is called. _whl_mods_impl(mods.whl_mods) @@ -394,9 +399,15 @@ def _pip_impl(module_ctx): groups = mods.hub_group_map.get(hub_name), ) - return module_ctx.extension_metadata( - reproducible = True, - ) + # The code is smart to not return facts if we don't support the mechanism for that. + # Hence we should not pass it to the metadata + if mods.facts: + return module_ctx.extension_metadata( + reproducible = True, + facts = mods.facts, + ) + else: + return module_ctx.extension_metadata(reproducible = True) _default_attrs = { "arch_name": attr.string( diff --git a/python/private/pypi/hub_builder.bzl b/python/private/pypi/hub_builder.bzl index 069d519e3c..e84f3b0ae8 100644 --- a/python/private/pypi/hub_builder.bzl +++ b/python/private/pypi/hub_builder.bzl @@ -395,11 +395,11 @@ def _set_get_index_urls(self, pip_attr): index_url = pip_attr.experimental_index_url, extra_index_urls = pip_attr.experimental_extra_index_urls or [], index_url_overrides = pip_attr.experimental_index_url_overrides or {}, - sources = [ - d - for d in distributions + sources = { + d: versions + for d, versions in distributions.items() if _use_downloader(self, python_version, d) - ], + }, envsubst = pip_attr.envsubst, # Auth related info netrc = pip_attr.netrc, diff --git a/python/private/pypi/parse_requirements.bzl b/python/private/pypi/parse_requirements.bzl index 78b6662d08..081e5f102f 100644 --- a/python/private/pypi/parse_requirements.bzl +++ b/python/private/pypi/parse_requirements.bzl @@ -53,7 +53,7 @@ def parse_requirements( os, arch combinations. extra_pip_args (string list): Extra pip arguments to perform extra validations and to be joined with args found in files. - get_index_urls: Callable[[ctx, list[str]], dict], a callable to get all + get_index_urls: Callable[[ctx, dict[str, list[str]]], dict], a callable to get all of the distribution URLs from a PyPI index. Accepts ctx and distribution names to query. evaluate_markers: A function to use to evaluate the requirements. @@ -170,15 +170,17 @@ def parse_requirements( index_urls = {} if get_index_urls: + distributions = {} + for reqs in requirements_by_platform.values(): + for req in reqs.values(): + if req.srcs.url: + continue + + distributions.setdefault(req.distribution, []).append(req.srcs.version) + index_urls = get_index_urls( ctx, - # Use list({}) as a way to have a set - list({ - req.distribution: None - for reqs in requirements_by_platform.values() - for req in reqs.values() - if not req.srcs.url - }), + distributions, ) ret = [] diff --git a/python/private/pypi/pypi_cache.bzl b/python/private/pypi/pypi_cache.bzl index 4dc824c10c..28c6cbeafb 100644 --- a/python/private/pypi/pypi_cache.bzl +++ b/python/private/pypi/pypi_cache.bzl @@ -8,18 +8,36 @@ In the future the same will be used to: - Store PyPI index query results as facts in the MODULE.bazel.lock file """ -def pypi_cache(store = None): +load(":version_from_filename.bzl", "version_from_filename") + +# This value should be changed whenever the storage format changes. +# Changing it simply means the information cached in the lockfile has to be +# recomputed. +_FACT_VERSION = "v1" + +def pypi_cache(mctx = None, store = None): """The cache for PyPI index queries. Currently the key is of the following structure: - (url, real_url) + (url, real_url, versions) + + Args: + mctx: The module context + store: The in-memory store, should implement dict interface for get and setdefault + + Returns: + A cache struct """ + mcache = memory_cache(store) + fcache = facts_cache(getattr(mctx, "facts", None)) # buildifier: disable=uninitialized self = struct( - _store = store or {}, + _mcache = mcache, + _facts = fcache, setdefault = lambda key, parsed_result: _pypi_cache_setdefault(self, key, parsed_result), get = lambda key: _pypi_cache_get(self, key), + get_facts = lambda: _pypi_cache_get_facts(self), ) # buildifier: enable=uninitialized @@ -40,7 +58,14 @@ def _pypi_cache_setdefault(self, key, parsed_result): Returns: The `parse_result`. """ - return self._store.setdefault(key, parsed_result) + index_url, real_url, versions = key + self._mcache.setdefault(real_url, parsed_result) + if not versions or not self._facts: + return parsed_result + + # Filter the packages to only what is needed before writing to the facts cache + filtered = _filter_packages(parsed_result, versions) + return self._facts.setdefault(index_url, filtered) def _pypi_cache_get(self, key): """Return the parsed result from the cache. @@ -52,4 +77,204 @@ def _pypi_cache_get(self, key): Returns: The {type}`struct` or `None` based on if the result is in the cache or not. """ - return self._store.get(key) + index_url, real_url, versions = key + + # When retrieving from memory cache, filter down to only what is needed. If the + # cache is empty, we will attempt to read from facts, however, reading from memory + # first allows us to not parse the contents of the lock file that may add up. + cached = _filter_packages(self._mcache.get(real_url), versions) + if not self._facts: + return cached + + if not cached and versions: + # Could not get from in-memory, read from lockfile facts + cached = self._facts.get(index_url, versions) + + return cached + +def _pypi_cache_get_facts(self): + if not self._facts: + return {} + + return self._facts.facts + +def memory_cache(cache = None): + """SimpleAPI cache for making fewer calls. + + We are using the `real_url` as the key in the cache functions on purpose in order to get the + best possible cache hits. + + Args: + cache: the storage to store things in memory. + + Returns: + struct with 2 methods, `get` and `setdefault`. + """ + if cache == None: + cache = {} + + return struct( + get = lambda real_url: cache.get(real_url), + setdefault = lambda real_url, value: cache.setdefault(real_url, value), + ) + +def _filter_packages(dists, requested_versions): + if dists == None or not requested_versions: + return dists + + sha256s_by_version = {} + whls = {} + sdists = {} + + for sha256, d in dists.sdists.items(): + if d.version not in requested_versions: + continue + + sdists[sha256] = d + sha256s_by_version.setdefault(d.version, []).append(sha256) + + for sha256, d in dists.whls.items(): + if d.version not in requested_versions: + continue + + whls[sha256] = d + sha256s_by_version.setdefault(d.version, []).append(sha256) + + if not whls and not sdists: + # TODO @aignas 2026-03-08: add logging + #print("WARN: no dists matched for versions {}".format(requested_versions)) + return None + + return struct( + whls = whls, + sdists = sdists, + sha256s_by_version = { + k: sorted(v) + for k, v in sha256s_by_version.items() + }, + ) + +def facts_cache(known_facts, facts_version = _FACT_VERSION): + """The facts cache. + + Here we have a way to store things as facts and the main thing to keep in mind is that we should + not use the real_url in case it contains credentials in it (e.g. is of form `https://:@`). + + Args: + known_facts: An opaque object coming from {obj}`module_ctx.facts`. + facts_version: {type}`str` the version of the facts schema, used for short-circuiting. + + Returns: + A struct that has: + * `get` method for getting values from the facts cache. + * `setdefault` method for setting values in the cache. + * `facts` attribute that should be passed to the {obj}`module_ctx.extension_metadata` to persist facts. + """ + if known_facts == None: + return None + + facts = {} + + return struct( + get = lambda index_url, versions: _get_from_facts( + facts, + known_facts, + index_url, + versions, + facts_version, + ), + setdefault = lambda url, value: _store_facts(facts, facts_version, url, value), + known_facts = known_facts, + facts = facts, + ) + +def _get_from_facts(facts, known_facts, index_url, requested_versions, facts_version): + if known_facts.get("fact_version") != facts_version: + # cannot trust known facts, different version that we know how to parse + return None + + known_sources = {} + + root_url, _, distribution = index_url.rstrip("/").rpartition("/") + distribution = distribution.rstrip("/") + root_url = root_url.rstrip("/") + + retrieved_versions = {} + + for url, sha256 in known_facts.get("dist_hashes", {}).get(root_url, {}).get(distribution, {}).items(): + filename = known_facts.get("dist_filenames", {}).get(root_url, {}).get(distribution, {}).get(sha256) + if not filename: + _, _, filename = url.rpartition("/") + + version = version_from_filename(filename) + if version not in requested_versions: + # TODO @aignas 2026-01-21: do the check by requested shas at some point + # We don't have sufficient info in the lock file, need to call the API + # + continue + + retrieved_versions[version] = True + + if filename.endswith(".whl"): + dists = known_sources.setdefault("whls", {}) + else: + dists = known_sources.setdefault("sdists", {}) + + known_sources.setdefault("sha256s_by_version", {}).setdefault(version, []).append(sha256) + + dists.setdefault(sha256, struct( + sha256 = sha256, + filename = filename, + version = version, + metadata_url = "", + metadata_sha256 = "", + url = url, + yanked = known_facts.get("dist_yanked", {}).get(root_url, {}).get(distribution, {}).get(sha256), + )) + + if not known_sources: + # We found nothing in facts + return None + + if len(requested_versions) != len(retrieved_versions): + # If the results are incomplete, then return None, so that we can fetch sources from the + # internet again. + return None + + output = struct( + whls = known_sources.get("whls", {}), + sdists = known_sources.get("sdists", {}), + sha256s_by_version = { + k: sorted(v) + for k, v in known_sources.get("sha256s_by_version", {}).items() + }, + ) + + # Persist these facts for the next run because we have used them. + return _store_facts(facts, facts_version, index_url, output) + +def _store_facts(facts, fact_version, index_url, value): + """Store values as facts in the lock file. + + The main idea is to ensure that the lock file is small and it is only + storing what we would need to fetch from the internet. Any derivative + information we can get from this that can be achieved using pure Starlark + functions should be done in Starlark. + """ + if not value: + return value + + facts["fact_version"] = fact_version + + root_url, _, distribution = index_url.rstrip("/").rpartition("/") + distribution = distribution.rstrip("/") + root_url = root_url.rstrip("/") + + for sha256, d in (value.sdists | value.whls).items(): + facts.setdefault("dist_hashes", {}).setdefault(root_url, {}).setdefault(distribution, {}).setdefault(d.url, sha256) + if not d.url.endswith(d.filename): + facts.setdefault("dist_filenames", {}).setdefault(root_url, {}).setdefault(distribution, {}).setdefault(d.url, d.filename) + if d.yanked != None: + facts.setdefault("dist_yanked", {}).setdefault(root_url, {}).setdefault(distribution, {}).setdefault(sha256, d.yanked) + + return value diff --git a/python/private/pypi/simpleapi_download.bzl b/python/private/pypi/simpleapi_download.bzl index 0f776ad434..ff18887ec1 100644 --- a/python/private/pypi/simpleapi_download.bzl +++ b/python/private/pypi/simpleapi_download.bzl @@ -81,6 +81,8 @@ def simpleapi_download( index_urls = [attr.index_url] + attr.extra_index_urls read_simpleapi = read_simpleapi or _read_simpleapi + input_sources = attr.sources + found_on_index = {} warn_overrides = False ctx.report_progress("Fetch package lists from PyPI index") @@ -90,8 +92,8 @@ def simpleapi_download( warn_overrides = True async_downloads = {} - sources = [pkg for pkg in attr.sources if pkg not in found_on_index] - for pkg in sources: + sources = {pkg: versions for pkg, versions in input_sources.items() if pkg not in found_on_index} + for pkg, versions in sources.items(): pkg_normalized = normalize_name(pkg) url = urllib.strip_empty_path_segments("{index_url}/{distribution}/".format( index_url = index_url_overrides.get(pkg_normalized, index_url).rstrip("/"), @@ -100,6 +102,7 @@ def simpleapi_download( result = read_simpleapi( ctx = ctx, attr = attr, + versions = versions, url = url, cache = cache, get_auth = get_auth, @@ -128,7 +131,7 @@ def simpleapi_download( contents[download.pkg_normalized] = _with_index_url(download.url, result.output) found_on_index[pkg] = index_url - failed_sources = [pkg for pkg in attr.sources if pkg not in found_on_index] + failed_sources = [pkg for pkg in input_sources if pkg not in found_on_index] if failed_sources: pkg_index_urls = { pkg: index_url_overrides.get( @@ -166,7 +169,7 @@ If you would like to skip downloading metadata for these packages please add 'si return contents -def _read_simpleapi(ctx, url, attr, cache, get_auth = None, **download_kwargs): +def _read_simpleapi(ctx, url, attr, cache, versions, get_auth = None, **download_kwargs): """Read SimpleAPI. Args: @@ -179,6 +182,7 @@ def _read_simpleapi(ctx, url, attr, cache, get_auth = None, **download_kwargs): * auth_patterns: The auth_patterns parameter for ctx.download, see {obj}`http_file` for docs. cache: {type}`struct` the `pypi_cache` instance. + versions: {type}`list[str] The versions that have been requested. get_auth: A function to get auth information. Used in tests. **download_kwargs: Any extra params to ctx.download. Note that output and auth will be passed for you. @@ -194,7 +198,7 @@ def _read_simpleapi(ctx, url, attr, cache, get_auth = None, **download_kwargs): real_url = urllib.strip_empty_path_segments(envsubst(url, attr.envsubst, ctx.getenv)) - cache_key = (url, real_url) + cache_key = (url, real_url, versions) cached_result = cache.get(cache_key) if cached_result: return struct(success = True, output = cached_result) diff --git a/tests/pypi/hub_builder/hub_builder_tests.bzl b/tests/pypi/hub_builder/hub_builder_tests.bzl index 170e12c4e4..637c7881c2 100644 --- a/tests/pypi/hub_builder/hub_builder_tests.bzl +++ b/tests/pypi/hub_builder/hub_builder_tests.bzl @@ -1052,7 +1052,12 @@ git_dep @ git+https://git.server/repo/project@deadbeefdeadbeef index_url = "pypi.org", index_url_overrides = {}, netrc = None, - sources = ["simple", "plat_pkg", "pip_fallback", "some_other_pkg"], + sources = { + "pip_fallback": ["0.0.1"], + "plat_pkg": ["0.0.4"], + "simple": ["0.0.1"], + "some_other_pkg": ["0.0.1"], + }, ), "cache": {}, "parallel_download": False, diff --git a/tests/pypi/pypi_cache/BUILD.bazel b/tests/pypi/pypi_cache/BUILD.bazel new file mode 100644 index 0000000000..03c20623cd --- /dev/null +++ b/tests/pypi/pypi_cache/BUILD.bazel @@ -0,0 +1,5 @@ +load(":pypi_cache_tests.bzl", "pypi_cache_test_suite") + +pypi_cache_test_suite( + name = "pypi_cache_tests", +) diff --git a/tests/pypi/pypi_cache/pypi_cache_tests.bzl b/tests/pypi/pypi_cache/pypi_cache_tests.bzl new file mode 100644 index 0000000000..7b6168ce7b --- /dev/null +++ b/tests/pypi/pypi_cache/pypi_cache_tests.bzl @@ -0,0 +1,239 @@ +"" + +load("@rules_testing//lib:test_suite.bzl", "test_suite") +load("@rules_testing//lib:truth.bzl", "subjects") +load("//python/private/pypi:pypi_cache.bzl", "pypi_cache") # buildifier: disable=bzl-visibility + +_tests = [] + +def _cache(env, **kwargs): + cache = pypi_cache(**kwargs) + + attrs = { + "sdists": subjects.dict, + "sha256s_by_version": subjects.dict, + "whls": subjects.dict, + } + + def _expect(value): + if not value: + return env.expect.that_str(value) + + return env.expect.that_struct( + value, + attrs = attrs, + ) + + return struct( + setdefault = lambda *args, **kwargs: _expect( + cache.setdefault(*args, **kwargs), + ), + get = lambda *args, **kwargs: _expect( + cache.get(*args, **kwargs), + ), + get_facts = lambda: env.expect.that_dict(cache.get_facts()), + ) + +def _test_memory_cache_hit(env): + """Verifies that the cache returns stored values for the same real_url.""" + store = {} + + # We pass None for module_ctx to focus solely on memory_cache behavior + cache = _cache(env, mctx = None, store = store) + + # Mocked parsed result from a PyPI-like index + fake_result = struct( + sdists = { + "sha_1": struct(version = "1.0.0", filename = "pkg-1.0.0.tar.gz"), + }, + whls = { + "sha_2": struct(version = "1.1.0", filename = "pkg-1.1.0-py3-none-any.whl"), + }, + sha256s_by_version = { + "1.0.0": ["sha_1"], + "1.1.0": ["sha_2"], + }, + ) + + # Key format: (index_url, real_url, versions) + key = ("https://{PYPI_INDEX_URL}/pkg", "https://pypi.org/simple/pkg", ["1.0.0", "1.1.0"]) + + # When set the cache + cache.setdefault(key, fake_result) + + # And get a value back + got = cache.get(key) + + got.sdists().contains_exactly(fake_result.sdists) + got.whls().contains_exactly(fake_result.whls) + got.sha256s_by_version().contains_exactly(fake_result.sha256s_by_version) + + # A different key with fewer versions + key = ("https://{PYPI_INDEX_URL}/pkg", "https://pypi.org/simple/pkg", ["1.0.0"]) + + got = cache.get(key) + got.sdists().contains_exactly(fake_result.sdists) + got.whls().contains_exactly({}) + got.sha256s_by_version().contains_exactly({"1.0.0": ["sha_1"]}) + + # A key with no matches + key = ("https://{PYPI_INDEX_URL}/pkg", "https://pypi.org/simple/pkg", ["1.2.0"]) + + cache.get(key).equals(None) + +_tests.append(_test_memory_cache_hit) + +def _test_pypi_cache_writes_to_facts(env): + """Verifies that setting a value in the cache also populates the facts store.""" + mock_ctx = struct(facts = {}) + cache = _cache(env, mctx = mock_ctx) + + fake_result = struct( + sdists = { + "sha_sdist": struct( + version = "1.0.0", + filename = "pkg-1.0.0.tar.gz", + url = "https://pypi.org/files/pkg-1.0.0.tar.gz", + yanked = "", + ), + }, + whls = { + "sha_whl": struct( + version = "1.0.0", + filename = "pkg-1.0.0-py3-none-any.whl", + url = "https://pypi.org/files/pkg-1.0.0-py3-none-any.whl", + yanked = "Security issue", + ), + # This won't get stored + "sha_whl_2": struct( + version = "1.1.0", + filename = "pkg-1.1.0-py3-none-any.whl", + url = "https://pypi.org/files/pkg-1.1.0-py3-none-any.whl", + yanked = None, + ), + }, + sha256s_by_version = { + "1.0.0": ["sha_sdist", "sha_whl"], + "1.1.0": ["sha_whl_2"], + }, + ) + + key = ("https://{PYPI_INDEX_URL}/pkg/", "https://pypi.org/simple/pkg/", ["1.0.0"]) + + # When we set the cache + cache.setdefault(key, fake_result) + + # Then the key returns us the same items + got = cache.get(key) + got.whls().contains_exactly({ + "sha_whl": fake_result.whls["sha_whl"], + }) + got.sdists().contains_exactly(fake_result.sdists) + got.sha256s_by_version().contains_exactly({ + "1.0.0": fake_result.sha256s_by_version["1.0.0"], + }) + + # Then when we get facts at the end + cache.get_facts().contains_exactly({ + "dist_hashes": { + # We are not using the real index URL, because we may have credentials in here + "https://{PYPI_INDEX_URL}": { + "pkg": { + "https://pypi.org/files/pkg-1.0.0-py3-none-any.whl": "sha_whl", + "https://pypi.org/files/pkg-1.0.0.tar.gz": "sha_sdist", + }, + }, + }, + "dist_yanked": { + "https://{PYPI_INDEX_URL}": { + "pkg": { + "sha_sdist": "", + "sha_whl": "Security issue", + }, + }, + }, + "fact_version": "v1", # Facts version + }) + +_tests.append(_test_pypi_cache_writes_to_facts) + +def _test_pypi_cache_reads_from_facts(env): + """Verifies that setting a value in the cache also populates the facts store.""" + mock_ctx = struct(facts = { + "dist_hashes": { + # We are not using the real index URL, because we may have credentials in here + "https://{PYPI_INDEX_URL}": { + "pkg": { + "https://pypi.org/files/pkg-1.0.0-py3-none-any.whl": "sha_whl", + "https://pypi.org/files/pkg-1.0.0.tar.gz": "sha_sdist", + }, + }, + }, + "dist_yanked": { + "https://{PYPI_INDEX_URL}": { + "pkg": { + "sha_sdist": "", + "sha_whl": "Security issue", + }, + }, + }, + "fact_version": "v1", # Facts version + }) + cache = _cache(env, mctx = mock_ctx) + + key = ("https://{PYPI_INDEX_URL}/pkg/", "https://pypi.org/simple/pkg/", ["1.0.0"]) + + # Then we would get empty facts because we haven't accessed any of the known facts. + # This simulates the dropping of the facts of requirements that are no longer needed. + cache.get_facts().contains_exactly({}) + + # When we get the + got = cache.get(key) + + expected_result = struct( + sdists = { + "sha_sdist": struct( + sha256 = "sha_sdist", + version = "1.0.0", + filename = "pkg-1.0.0.tar.gz", + metadata_url = "", + metadata_sha256 = "", + url = "https://pypi.org/files/pkg-1.0.0.tar.gz", + yanked = "", + ), + }, + whls = { + "sha_whl": struct( + sha256 = "sha_whl", + version = "1.0.0", + filename = "pkg-1.0.0-py3-none-any.whl", + url = "https://pypi.org/files/pkg-1.0.0-py3-none-any.whl", + metadata_url = "", + metadata_sha256 = "", + yanked = "Security issue", + ), + }, + sha256s_by_version = { + "1.0.0": ["sha_sdist", "sha_whl"], + }, + ) + + got.whls().contains_exactly(expected_result.whls) + got.sdists().contains_exactly(expected_result.sdists) + got.sha256s_by_version().contains_exactly(expected_result.sha256s_by_version) + + # Then when we store the same facts back again, because we accessed the cached keys. + cache.get_facts().contains_exactly(mock_ctx.facts) + + # When we request more than what we have, we will return nothing + key = ("https://{PYPI_INDEX_URL}/pkg/", "https://pypi.org/simple/pkg/", ["1.0.0", "1.1.0"]) + got = cache.get(key) + got.equals(None) + +_tests.append(_test_pypi_cache_reads_from_facts) + +def pypi_cache_test_suite(name): + test_suite( + name = name, + basic_tests = _tests, + ) diff --git a/tests/pypi/simpleapi_download/simpleapi_download_tests.bzl b/tests/pypi/simpleapi_download/simpleapi_download_tests.bzl index 391e352e08..953df5c107 100644 --- a/tests/pypi/simpleapi_download/simpleapi_download_tests.bzl +++ b/tests/pypi/simpleapi_download/simpleapi_download_tests.bzl @@ -23,11 +23,12 @@ _tests = [] def _test_simple(env): calls = [] - def read_simpleapi(ctx, url, attr, cache, get_auth, block): + def read_simpleapi(ctx, url, versions, attr, cache, get_auth, block): _ = ctx # buildifier: disable=unused-variable _ = attr _ = cache _ = get_auth + _ = versions env.expect.that_bool(block).equals(False) calls.append(url) if "foo" in url and "main" in url: @@ -54,7 +55,7 @@ def _test_simple(env): index_url_overrides = {}, index_url = "main", extra_index_urls = ["extra"], - sources = ["foo", "bar", "baz"], + sources = {"bar": None, "baz": None, "foo": None}, envsubst = [], ), cache = pypi_cache(), @@ -95,11 +96,12 @@ def _test_fail(env): calls = [] fails = [] - def read_simpleapi(ctx, url, attr, cache, get_auth, block): + def read_simpleapi(ctx, url, versions, attr, cache, get_auth, block): _ = ctx # buildifier: disable=unused-variable _ = attr _ = cache _ = get_auth + _ = versions env.expect.that_bool(block).equals(False) calls.append(url) if "foo" in url: @@ -133,7 +135,7 @@ def _test_fail(env): }, index_url = "main", extra_index_urls = ["extra"], - sources = ["foo", "bar", "baz"], + sources = {"bar": None, "baz": None, "foo": None}, envsubst = [], ), cache = pypi_cache(), @@ -146,13 +148,13 @@ def _test_fail(env): """ Failed to download metadata of the following packages from urls: { - "foo": "invalid", "bar": ["main", "extra"], + "foo": "invalid", } If you would like to skip downloading metadata for these packages please add 'simpleapi_skip=[ - "foo", "bar", + "foo", ]' to your 'pip.parse' call. """, ]) @@ -186,7 +188,7 @@ def _test_download_url(env): index_url_overrides = {}, index_url = "https://example.com/main/simple/", extra_index_urls = [], - sources = ["foo", "bar", "baz"], + sources = {"bar": None, "baz": None, "foo": None}, envsubst = [], ), cache = pypi_cache(), @@ -222,7 +224,7 @@ def _test_download_url_parallel(env): index_url_overrides = {}, index_url = "https://example.com/main/simple/", extra_index_urls = [], - sources = ["foo", "bar", "baz"], + sources = {"bar": None, "baz": None, "foo": None}, envsubst = [], ), cache = pypi_cache(), @@ -258,7 +260,7 @@ def _test_download_envsubst_url(env): index_url_overrides = {}, index_url = "$INDEX_URL", extra_index_urls = [], - sources = ["foo", "bar", "baz"], + sources = {"bar": None, "baz": None, "foo": None}, envsubst = ["INDEX_URL"], ), cache = pypi_cache(), From ad95ea090e942fca74415bc58425ef5f40339e47 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Fri, 13 Mar 2026 23:37:48 -0700 Subject: [PATCH 662/922] chore: remove pip_repository_annotations example (#3622) This PR deletes the `pip_repository_annotations` example. This is because it is a workspace-only example, and its removal is preparatory work for dropping workspace support from rules_python. --- .bazelci/presubmit.yml | 25 -------- examples/pip_repository_annotations/.bazelrc | 9 --- .../pip_repository_annotations/.gitignore | 1 - examples/pip_repository_annotations/WORKSPACE | 62 ------------------- .../data/copy_executable.py | 18 ------ .../data/copy_file.txt | 1 - .../requirements.in | 7 --- .../requirements.txt | 34 ---------- python/private/pypi/pip_repository.bzl | 3 +- 9 files changed, 1 insertion(+), 159 deletions(-) delete mode 100644 examples/pip_repository_annotations/.bazelrc delete mode 100644 examples/pip_repository_annotations/.gitignore delete mode 100644 examples/pip_repository_annotations/WORKSPACE delete mode 100755 examples/pip_repository_annotations/data/copy_executable.py delete mode 100644 examples/pip_repository_annotations/data/copy_file.txt delete mode 100644 examples/pip_repository_annotations/requirements.in delete mode 100644 examples/pip_repository_annotations/requirements.txt diff --git a/.bazelci/presubmit.yml b/.bazelci/presubmit.yml index 2f923016f8..43e519259c 100644 --- a/.bazelci/presubmit.yml +++ b/.bazelci/presubmit.yml @@ -433,31 +433,6 @@ tasks: # We don't run pip_parse_vendored under Windows as the file checked in is # generated from a repository rule containing OS-specific rendered paths. - integration_test_pip_repository_annotations_ubuntu_workspace: - <<: *reusable_build_test_all - <<: *common_workspace_flags - name: "examples/pip_repository_annotations: Ubuntu, workspace" - working_directory: examples/pip_repository_annotations - platform: ubuntu2204 - integration_test_pip_repository_annotations_debian_workspace: - <<: *reusable_build_test_all - <<: *common_workspace_flags - name: "examples/pip_repository_annotations: Debian, workspace" - working_directory: examples/pip_repository_annotations - platform: debian11 - integration_test_pip_repository_annotations_macos_workspace: - <<: *reusable_build_test_all - <<: *common_workspace_flags - name: "examples/pip_repository_annotations: macOS, workspace" - working_directory: examples/pip_repository_annotations - platform: macos_arm64 - integration_test_pip_repository_annotations_windows_workspace: - <<: *reusable_build_test_all - <<: *common_workspace_flags - name: "examples/pip_repository_annotations: Windows, workspace" - working_directory: examples/pip_repository_annotations - platform: windows - integration_test_bazelinbazel_ubuntu: <<: *common_bazelinbazel_config name: "tests/integration bazel-in-bazel: Ubuntu" diff --git a/examples/pip_repository_annotations/.bazelrc b/examples/pip_repository_annotations/.bazelrc deleted file mode 100644 index 9397bd31b8..0000000000 --- a/examples/pip_repository_annotations/.bazelrc +++ /dev/null @@ -1,9 +0,0 @@ -# https://docs.bazel.build/versions/main/best-practices.html#using-the-bazelrc-file -try-import %workspace%/user.bazelrc - -# This example is WORKSPACE specific. The equivalent functionality -# is in examples/bzlmod as the `whl_mods` feature. -common --noenable_bzlmod -common --enable_workspace -common --legacy_external_runfiles=false -common --incompatible_python_disallow_native_rules diff --git a/examples/pip_repository_annotations/.gitignore b/examples/pip_repository_annotations/.gitignore deleted file mode 100644 index a6ef824c1f..0000000000 --- a/examples/pip_repository_annotations/.gitignore +++ /dev/null @@ -1 +0,0 @@ -/bazel-* diff --git a/examples/pip_repository_annotations/WORKSPACE b/examples/pip_repository_annotations/WORKSPACE deleted file mode 100644 index 8540555084..0000000000 --- a/examples/pip_repository_annotations/WORKSPACE +++ /dev/null @@ -1,62 +0,0 @@ -workspace(name = "pip_repository_annotations_example") - -local_repository( - name = "rules_python", - path = "../..", -) - -load("@rules_python//python:repositories.bzl", "py_repositories", "python_register_toolchains") - -py_repositories() - -python_register_toolchains( - name = "python39", - python_version = "3.9", -) - -load("@rules_python//python:pip.bzl", "package_annotation", "pip_parse") - -# Here we can see an example of annotations being applied to an arbitrary -# package. For details on `package_annotation` and it's uses, see the -# docs at @rules_python//docs:pip.md`. -ANNOTATIONS = { - # This annotation verifies that annotations work correctly for pip packages with extras - # specified, in this case requests[security]. - "requests": package_annotation( - additive_build_content = """\ -load("@bazel_skylib//rules:write_file.bzl", "write_file") -write_file( - name = "generated_file", - out = "generated_file.txt", - content = ["Hello world from requests"], -) -""", - data = [":generated_file"], - ), - "wheel": package_annotation( - additive_build_content = """\ -load("@bazel_skylib//rules:write_file.bzl", "write_file") -write_file( - name = "generated_file", - out = "generated_file.txt", - content = ["Hello world from build content file"], -) -""", - copy_executables = {"@pip_repository_annotations_example//:data/copy_executable.py": "copied_content/executable.py"}, - copy_files = {"@pip_repository_annotations_example//:data/copy_file.txt": "copied_content/file.txt"}, - data = [":generated_file"], - data_exclude_glob = ["site-packages/*.dist-info/WHEEL"], - ), -} - -# For a more thorough example of `pip_parse`. See `@rules_python//examples/pip_parse` -pip_parse( - name = "pip", - annotations = ANNOTATIONS, - python_interpreter_target = "@python39_host//:python", - requirements_lock = "//:requirements.txt", -) - -load("@pip//:requirements.bzl", "install_deps") - -install_deps() diff --git a/examples/pip_repository_annotations/data/copy_executable.py b/examples/pip_repository_annotations/data/copy_executable.py deleted file mode 100755 index 5cb1af7fdb..0000000000 --- a/examples/pip_repository_annotations/data/copy_executable.py +++ /dev/null @@ -1,18 +0,0 @@ -#!/usr/bin/env python -# Copyright 2023 The Bazel Authors. All rights reserved. -# -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - - -if __name__ == "__main__": - print("Hello world from copied executable") diff --git a/examples/pip_repository_annotations/data/copy_file.txt b/examples/pip_repository_annotations/data/copy_file.txt deleted file mode 100644 index b1020f7b95..0000000000 --- a/examples/pip_repository_annotations/data/copy_file.txt +++ /dev/null @@ -1 +0,0 @@ -Hello world from copied file diff --git a/examples/pip_repository_annotations/requirements.in b/examples/pip_repository_annotations/requirements.in deleted file mode 100644 index c9afafc6f5..0000000000 --- a/examples/pip_repository_annotations/requirements.in +++ /dev/null @@ -1,7 +0,0 @@ -# This flag allows for regression testing requirements arguments in -# `pip_repository` rules. ---extra-index-url https://pypi.org/simple/ - -certifi>=2023.7.22 # https://security.snyk.io/vuln/SNYK-PYTHON-CERTIFI-5805047 -wheel -requests[security]>=2.8.1 diff --git a/examples/pip_repository_annotations/requirements.txt b/examples/pip_repository_annotations/requirements.txt deleted file mode 100644 index f1069a7452..0000000000 --- a/examples/pip_repository_annotations/requirements.txt +++ /dev/null @@ -1,34 +0,0 @@ -# -# This file is autogenerated by pip-compile with Python 3.9 -# by the following command: -# -# bazel run //:requirements.update -# ---extra-index-url https://pypi.org/simple/ - -certifi==2023.7.22 \ - --hash=sha256:539cc1d13202e33ca466e88b2807e29f4c13049d6d87031a3c110744495cb082 \ - --hash=sha256:92d6037539857d8206b8f6ae472e8b77db8058fec5937a1ef3f54304089edbb9 - # via - # -r requirements.in - # requests -charset-normalizer==2.1.1 \ - --hash=sha256:5a3d016c7c547f69d6f81fb0db9449ce888b418b5b9952cc5e6e66843e9dd845 \ - --hash=sha256:83e9a75d1911279afd89352c68b45348559d1fc0506b054b346651b5e7fee29f - # via requests -idna==3.7 \ - --hash=sha256:028ff3aadf0609c1fd278d8ea3089299412a7a8b9bd005dd08b9f8285bcb5cfc \ - --hash=sha256:82fee1fc78add43492d3a1898bfa6d8a904cc97d8427f683ed8e798d07761aa0 - # via requests -requests[security]==2.28.1 \ - --hash=sha256:7c5599b102feddaa661c826c56ab4fee28bfd17f5abca1ebbe3e7f19d7c97983 \ - --hash=sha256:8fefa2a1a1365bf5520aac41836fbee479da67864514bdb821f31ce07ce65349 - # via -r requirements.in -urllib3==1.26.18 \ - --hash=sha256:34b97092d7e0a3a8cf7cd10e386f401b3737364026c45e622aa02903dffe0f07 \ - --hash=sha256:f8ecc1bba5667413457c529ab955bf8c67b45db799d159066261719e328580a0 - # via requests -wheel==0.38.4 \ - --hash=sha256:965f5259b566725405b05e7cf774052044b1ed30119b5d586b2703aafe8719ac \ - --hash=sha256:b60533f3f5d530e971d6737ca6d58681ee434818fab630c83a734bb10c083ce8 - # via -r requirements.in diff --git a/python/private/pypi/pip_repository.bzl b/python/private/pypi/pip_repository.bzl index 489c124135..6d38334cc6 100644 --- a/python/private/pypi/pip_repository.bzl +++ b/python/private/pypi/pip_repository.bzl @@ -266,8 +266,7 @@ pip_repository = repository_rule( doc = """\ Optional annotations to apply to packages. Keys should be package names, with capitalization matching the input requirements file, and values should be -generated using the `package_name` macro. For example usage, see [this WORKSPACE -file](https://github.com/bazel-contrib/rules_python/blob/main/examples/pip_repository_annotations/WORKSPACE). +generated using the `package_name` macro. """, ), _config_template = attr.label( From 9369ac367a3d9f56396f07c2f1d765c5ea02af5f Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Fri, 13 Mar 2026 23:39:08 -0700 Subject: [PATCH 663/922] chore: factor release note checking into separate script (#3628) This is to make it easier to check for missing version markers by running the script directly --- .github/workflows/check_version_markers.sh | 32 +++++++++++++++++++ .github/workflows/create_archive_and_notes.sh | 20 ++---------- 2 files changed, 35 insertions(+), 17 deletions(-) create mode 100755 .github/workflows/check_version_markers.sh diff --git a/.github/workflows/check_version_markers.sh b/.github/workflows/check_version_markers.sh new file mode 100755 index 0000000000..b8d35aec9d --- /dev/null +++ b/.github/workflows/check_version_markers.sh @@ -0,0 +1,32 @@ +#!/usr/bin/env bash + +set -o nounset +set -o pipefail +set -o errexit + +set -x + +TAG=${1:-} +if [ -n "$TAG" ]; then + # If the workflow checks out one commit, but is releasing another + git fetch origin tag "$TAG" + # Update our local state so the grep command below searches what we expect + git checkout "$TAG" +fi + +grep_exit_code=0 +# Exclude dot directories, specifically, this file so that we don't +# find the substring we're looking for in our own file. +# Exclude CONTRIBUTING.md, RELEASING.md because they document how to use these strings. +grep --exclude=CONTRIBUTING.md \ + --exclude=RELEASING.md \ + --exclude=release.py \ + --exclude=release_test.py \ + --exclude-dir=.* \ + VERSION_NEXT_ -r || grep_exit_code=$? + +if [[ $grep_exit_code -eq 0 ]]; then + echo + echo "Found VERSION_NEXT markers indicating version needs to be specified" + exit 1 +fi diff --git a/.github/workflows/create_archive_and_notes.sh b/.github/workflows/create_archive_and_notes.sh index a3cf8280a2..506e2f9784 100755 --- a/.github/workflows/create_archive_and_notes.sh +++ b/.github/workflows/create_archive_and_notes.sh @@ -26,24 +26,10 @@ if [ -z "$TAG" ]; then fi # If the workflow checks out one commit, but is releasing another git fetch origin tag "$TAG" -# Update our local state so the grep command below searches what we expect -git checkout "$TAG" -# Exclude dot directories, specifically, this file so that we don't -# find the substring we're looking for in our own file. -# Exclude CONTRIBUTING.md, RELEASING.md because they document how to use these strings. -grep --exclude=CONTRIBUTING.md \ - --exclude=RELEASING.md \ - --exclude=release.py \ - --exclude=release_test.py \ - --exclude-dir=.* \ - VERSION_NEXT_ -r || grep_exit_code=$? - -if [[ $grep_exit_code -eq 0 ]]; then - echo - echo "Found VERSION_NEXT markers indicating version needs to be specified" - exit 1 -fi +# Update our local state so that check_version_markers searches what we expect +git checkout "$TAG" +$(dirname $0)/check_version_markers.sh # A prefix is added to better match the GitHub generated archives. PREFIX="rules_python-${TAG}" From 7df7bd587ac79a76931787f27109168179e275e9 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Sat, 14 Mar 2026 00:25:07 -0700 Subject: [PATCH 664/922] fix: prevent stdlib pyc files from invalidating runtime repos (#3661) The runtime repositories are being constantly invalidated due to pyc creation under Bazel 9 because, starting in Bazel 9, `glob()` functions implicitly register `repository_ctx.watch()` calls on the files and directories they match. Thus, the directories where `__pycache__` directories are created end up being considered changed (either directly because their mtimes change, or indirectly, because their directory listing changes), which then invalidates the repo, causing it to re-run. This glob-induced-watching seems to occur even if an `exclude` would have excluded the file. Note that this only seems to occur if `reproducible=False`, which generally wouldn't occur, but could occur if a user is registering their own runtime and doesn't care about the sha. Regardless, this still seems worthwhile because it allows pyc to be more safely be generated without causing repo invalidations, while allowing them to be persisted between repo-phase invocations. To fix, create `__pycache__` directories ahead of time and symlink them to a location that Bazel isn't watching, i.e. outside the repository's directory. I tried creating a separate top-level folder that wasn't matched by any globs and symlinking to it, but Bazel would read through the symlinks and watch the underlying locations. This also has a side-bonus that allows pyc files to be re-used in between repository-phase invocations. Fixes https://github.com/bazel-contrib/rules_python/issues/3643 --- CHANGELOG.md | 6 ++ docs/environment-variables.md | 28 ++++++ .../private/hermetic_runtime_repo_setup.bzl | 43 +++++---- python/private/python_repository.bzl | 93 +++++++++++++++++++ python/private/repo_utils.bzl | 45 +++++++++ 5 files changed, 196 insertions(+), 19 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 6e0f207e2a..d908992c9c 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -74,6 +74,12 @@ Other changes: * (zipapp) Resolve issue passing through compression settings in `py_zippapp_binary` targets ([#3646](https://github.com/bazel-contrib/rules_python/issues/3646)). +* (toolchains) The pyc created at runtime in the stdlib should no longer + cause the Python runtime repository to be invalidated. The stdlib pyc files + _may_ be reused in between invocations, depending upon the sandboxing + configuration. See the {any}`RULES_PYTHON_PYCACHE_DIR` environment variable + for more information. + ([#3643](https://github.com/bazel-contrib/rules_python/issues/3643)). {#v0-0-0-added} ### Added diff --git a/docs/environment-variables.md b/docs/environment-variables.md index fb48f434cd..d322f601a9 100644 --- a/docs/environment-variables.md +++ b/docs/environment-variables.md @@ -116,6 +116,34 @@ Valid values: * Other non-empty values mean to use isolated mode. ::: +:::{envvar} RULES_PYTHON_PYCACHE_DIR + +Determines the directory that runtime-generated pyc cache files will +be stored in. + +This directory may be reused between invocations, depending on the sandboxing +configuration. Setting it to `/dev/null` will, in effect, disable runtime +pyc caching. By setting e.g. +`--sandbox_add_mount_pair=/tmp/rules_python_pycache`, it's possible for pyc +caching to persist across invocations. + +**Behavior specific to downloaded runtimes:** +First `RULES_PYTHON_PYCACHE_DIR` is checked. If set, it is used as-is for +the root pycache directory. + +Otherwise, the following environment variables are checked in the following +order. Their values will have `rules_python_pycache` appended to them to form +the root pycache directory: +1. `XDG_CACHE_HOME`. +2. `TMP` (non-Windows) or `TEMP` (Windows). +3. The common platform-specific temporary directory (`/tmp` (non-Windows) or + `C:\Temp` (Windows)). + +If such a diretory cannot be found, or created, then `/dev/null` will be used, +which will effectively disable pyc caching. + +::: + :::{envvar} RULES_PYTHON_REPO_DEBUG When `1`, repository rules will print debug information about what they're diff --git a/python/private/hermetic_runtime_repo_setup.bzl b/python/private/hermetic_runtime_repo_setup.bzl index c3c275546d..d860983e22 100644 --- a/python/private/hermetic_runtime_repo_setup.bzl +++ b/python/private/hermetic_runtime_repo_setup.bzl @@ -58,30 +58,35 @@ def define_hermetic_runtime_toolchain_impl( "major": version_info.release[0], "minor": version_info.release[1], } + files_include = [ + "bin/**", + "extensions/**", + "include/**", + "libs/**", + "share/**", + ] + files_include += extra_files_glob_include + files_exclude = [ + # Unused shared libraries. `python` executable and the `:libpython` target + # depend on `libpython{python_version}.so.1.0`. + "lib/libpython{major}.{minor}*.so".format(**version_dict), + # static libraries + "lib/**/*.a", + # tests for the standard libraries. + "lib/python{major}.{minor}*/**/test/**".format(**version_dict), + "lib/python{major}.{minor}*/**/tests/**".format(**version_dict), + # During pyc creation, temp files named *.pyc.NNN are created + "**/__pycache__/*.pyc.*", + ] + files_exclude += extra_files_glob_exclude + native.filegroup( name = "files", srcs = native.glob( - include = [ - "bin/**", - "extensions/**", - "include/**", - "libs/**", - "share/**", - ] + extra_files_glob_include, + include = files_include, # Platform-agnostic filegroup can't match on all patterns. allow_empty = True, - exclude = [ - # Unused shared libraries. `python` executable and the `:libpython` target - # depend on `libpython{python_version}.so.1.0`. - "lib/libpython{major}.{minor}*.so".format(**version_dict), - # static libraries - "lib/**/*.a", - # tests for the standard libraries. - "lib/python{major}.{minor}*/**/test/**".format(**version_dict), - "lib/python{major}.{minor}*/**/tests/**".format(**version_dict), - # During pyc creation, temp files named *.pyc.NNN are created - "**/__pycache__/*.pyc.*", - ] + extra_files_glob_exclude, + exclude = files_exclude, ), ) cc_import( diff --git a/python/private/python_repository.bzl b/python/private/python_repository.bzl index 3d54b8a26d..9c44971117 100644 --- a/python/private/python_repository.bzl +++ b/python/private/python_repository.bzl @@ -52,6 +52,98 @@ def is_standalone_interpreter(rctx, python_interpreter_path, *, logger = None): logger = logger, ).return_code == 0 +def _get_pycache_root(rctx): + """Calculates and creates the pycache root directory. + + Returns: + {type}`path | None` The path to the pycache root, or None if it couldn't + be created. + """ + os_name = repo_utils.get_platforms_os_name(rctx) + is_windows = os_name == "windows" + + # 1. RULES_PYTHON_PYCACHE_DIR + res = rctx.getenv("RULES_PYTHON_PYCACHE_DIR") + if res: + res = res + "/" + rctx.name + return repo_utils.mkdir(rctx, res) + + # Suffix for cases 2-4 + # The first level directory is static and documented so that it is easy to + # use with e.g. --sandbox_add_mount_pair=/tmp/rules_python_pycache + suffix = "rules_python_pycache/{}/{}".format(hash(str(rctx.workspace_root)), rctx.name) + + # 2. XDG_CACHE_HOME + res = rctx.getenv("XDG_CACHE_HOME") + if res: + path = repo_utils.mkdir(rctx, rctx.path(res).get_child(suffix)) + if path: + return path + + # 3. TMP or TEMP + res = rctx.getenv("TMP") or rctx.getenv("TEMP") + if res: + path = repo_utils.mkdir(rctx, rctx.path(res).get_child(suffix)) + if path: + return path + + # 4. /tmp or Windows equivalent + if is_windows: + path = rctx.path("C:/Temp").get_child(suffix) + else: + path = rctx.path("/tmp").get_child(suffix) + + return repo_utils.mkdir(rctx, path) + +def _create_pycache_symlinks(rctx, logger): + """Finds all directories with a .py file and creates __pycache__ symlinks. + + Args: + rctx: {type}`repository_ctx` The repository rule's context object. + logger: Optional logger to use for operations. + """ + pycache_root = _get_pycache_root(rctx) + logger.info(lambda: "pycache root: {}".format(pycache_root)) + pycache_root_str = str(pycache_root) if pycache_root else None + + os_name = repo_utils.get_platforms_os_name(rctx) + null_device = "NUL" if os_name == "windows" else "/dev/null" + + queue = [rctx.path(".")] + + # Starlark doesn't support recursion, use a loop with a queue. + # Using a large range as a safeguard. + for _ in range(1000000): + if not queue: + break + p = queue.pop() + + has_py = False + for child in p.readdir(): + # Skip hidden files and directories + if child.basename.startswith("."): + continue + + if child.is_dir: + if child.basename == "__pycache__" or str(child) == pycache_root_str: + continue + queue.append(child) + elif child.basename.endswith(".py"): + has_py = True + + if has_py: + pycache_dir = p.get_child("__pycache__") + if pycache_root: + pycache_relative = repo_utils.repo_root_relative_path(rctx, pycache_dir) + target_dir = pycache_root.get_child(pycache_relative) + + repo_utils.mkdir(rctx, target_dir) + rctx.delete(pycache_dir) + rctx.symlink(target_dir, pycache_dir) + else: + rctx.delete(pycache_dir) + rctx.symlink(null_device, pycache_dir) + def _python_repository_impl(rctx): if rctx.attr.distutils and rctx.attr.distutils_content: fail("Only one of (distutils, distutils_content) should be set.") @@ -123,6 +215,7 @@ def _python_repository_impl(rctx): logger = logger, ) + _create_pycache_symlinks(rctx, logger) python_bin = "python.exe" if ("windows" in platform) else "bin/python3" if "linux" in platform: diff --git a/python/private/repo_utils.bzl b/python/private/repo_utils.bzl index 702a333772..00f43f3521 100644 --- a/python/private/repo_utils.bzl +++ b/python/private/repo_utils.bzl @@ -319,6 +319,49 @@ def _which_describe_failure(binary_name, path): path = path, ) +def _mkdir(mrctx, path): + path = mrctx.path(path) + if path.exists: + return path + + repo_root = str(mrctx.path(".")) + path_str = str(path) + + if not path_str.startswith(repo_root): + mkdir_bin = mrctx.which("mkdir") + if not mkdir_bin: + return None + res = mrctx.execute([mkdir_bin, "-p", path_str]) + if res.return_code != 0: + return None + return path + else: + placeholder = path.get_child(".placeholder") + mrctx.file(placeholder) + mrctx.delete(placeholder) + return path + +def _repo_root_relative_path(mrctx, path): + """Takes a path object and returns a repo-relative path string. + + Args: + mrctx: module_ctx or repository_ctx + path: {type}`path` a path within `mrctx` + + Returns: + {type}`str` a repo-root-relative path string. + """ + repo_root = str(mrctx.path(".")) + path_str = str(path) + relative_path = path_str[len(repo_root):] + if relative_path[0] != "/": + fail("{path} not under {repo_root}".format( + path = path, + repo_root = repo_root, + )) + relative_path = relative_path[1:] + return relative_path + def _args_to_str(arguments): return " ".join([_arg_repr(a) for a in arguments]) @@ -465,6 +508,8 @@ repo_utils = struct( get_platforms_os_name = _get_platforms_os_name, is_repo_debug_enabled = _is_repo_debug_enabled, logger = _logger, + mkdir = _mkdir, + repo_root_relative_path = _repo_root_relative_path, which_checked = _which_checked, which_unchecked = _which_unchecked, ) From 54d372a6861b2d897ad5a9653b18dd6d80ce17d1 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Sat, 14 Mar 2026 15:15:21 -0700 Subject: [PATCH 665/922] chore: split sphinxdocs into its own module (#3629) Splitting sphinxdocs into its own module is desirable to reduce the dependency footprint of rules_python outside of development mode modules. Work towards https://github.com/bazel-contrib/rules_python/issues/2511 --------- Co-authored-by: Ignas Anikevicius <240938+aignas@users.noreply.github.com> --- .bazelci/presubmit.yml | 17 ++++++ .bazelignore | 1 + .bcr/config.yml | 2 +- .bcr/sphinxdocs/metadata.template.json | 21 +++++++ .bcr/sphinxdocs/presubmit.yml | 23 ++++++++ .bcr/sphinxdocs/source.template.json | 5 ++ MODULE.bazel | 9 ++- docs/BUILD.bazel | 14 ++--- docs/conf.py | 7 ++- docs/readthedocs_build.sh | 6 +- internal_dev_deps.bzl | 7 +++ python/private/BUILD.bazel | 7 +-- sphinxdocs/.bazelrc | 22 +++++++ sphinxdocs/.bazelrc.deleted_packages | 1 + sphinxdocs/.bazelversion | 1 + sphinxdocs/MODULE.bazel | 27 +++++++++ sphinxdocs/{ => sphinxdocs}/BUILD.bazel | 2 +- sphinxdocs/{ => sphinxdocs}/docs/BUILD.bazel | 5 +- sphinxdocs/{ => sphinxdocs}/docs/api/index.md | 0 .../docs/api/sphinxdocs/index.md | 4 +- .../docs/api/sphinxdocs/inventories/index.md | 4 +- sphinxdocs/{ => sphinxdocs}/docs/index.md | 0 .../{ => sphinxdocs}/docs/readthedocs.md | 12 ++-- .../{ => sphinxdocs}/docs/sphinx-bzl.md | 2 +- .../{ => sphinxdocs}/docs/starlark-docgen.md | 4 +- .../integration_tests/bcr/BUILD.bazel | 7 +++ .../integration_tests/bcr/MODULE.bazel | 16 +++++ .../sphinxdocs/integration_tests/bcr/conf.py | 2 + .../sphinxdocs/integration_tests/bcr/index.md | 1 + .../{ => sphinxdocs}/inventories/BUILD.bazel | 0 .../inventories/bazel_inventory.txt | 0 .../{ => sphinxdocs}/private/BUILD.bazel | 28 ++++++--- .../private/inventory_builder.py | 0 .../private/proto_to_markdown.py | 0 .../{ => sphinxdocs}/private/readthedocs.bzl | 8 +-- .../private/readthedocs_install.py | 0 .../{ => sphinxdocs}/private/sphinx.bzl | 14 ++--- .../{ => sphinxdocs}/private/sphinx_build.py | 0 .../private/sphinx_docs_library.bzl | 0 .../private/sphinx_docs_library_info.bzl | 0 .../private/sphinx_docs_library_macro.bzl | 4 +- .../private/sphinx_run_template.sh | 0 .../{ => sphinxdocs}/private/sphinx_server.py | 0 .../private/sphinx_stardoc.bzl | 6 +- sphinxdocs/sphinxdocs/private/util.bzl | 58 +++++++++++++++++++ sphinxdocs/{ => sphinxdocs}/readthedocs.bzl | 0 sphinxdocs/{ => sphinxdocs}/sphinx.bzl | 0 .../{ => sphinxdocs}/sphinx_docs_library.bzl | 0 .../{ => sphinxdocs}/sphinx_stardoc.bzl | 0 .../src/sphinx_bzl/BUILD.bazel | 4 +- .../src/sphinx_bzl/__init__.py | 0 .../{ => sphinxdocs}/src/sphinx_bzl/bzl.py | 6 +- sphinxdocs/{ => sphinxdocs}/tests/BUILD.bazel | 0 .../tests/proto_to_markdown/BUILD.bazel | 2 +- .../proto_to_markdown_test.py | 0 .../tests/sphinx_docs/BUILD.bazel | 0 .../tests/sphinx_docs/conf.py | 0 .../tests/sphinx_docs/defs.bzl | 0 .../tests/sphinx_docs/doc1.md | 0 .../tests/sphinx_docs/doc2.md | 0 .../tests/sphinx_docs/index.md | 0 .../tests/sphinx_stardoc/BUILD.bazel | 2 +- .../tests/sphinx_stardoc/aspect.md | 0 .../tests/sphinx_stardoc/bzl_function.bzl | 0 .../tests/sphinx_stardoc/bzl_providers.bzl | 0 .../tests/sphinx_stardoc/bzl_rule.bzl | 0 .../tests/sphinx_stardoc/bzl_typedef.bzl | 0 .../tests/sphinx_stardoc/conf.py | 0 .../tests/sphinx_stardoc/envvars.md | 0 .../tests/sphinx_stardoc/function.md | 0 .../tests/sphinx_stardoc/glossary.md | 0 .../tests/sphinx_stardoc/index.md | 4 +- .../tests/sphinx_stardoc/module_extension.md | 0 .../tests/sphinx_stardoc/provider.md | 0 .../tests/sphinx_stardoc/repo_rule.md | 0 .../tests/sphinx_stardoc/rule.md | 0 .../sphinx_stardoc/sphinx_output_test.py | 0 .../tests/sphinx_stardoc/target.md | 0 .../tests/sphinx_stardoc/typedef.md | 0 .../tests/sphinx_stardoc/xrefs.md | 2 +- 80 files changed, 296 insertions(+), 71 deletions(-) create mode 100644 .bcr/sphinxdocs/metadata.template.json create mode 100644 .bcr/sphinxdocs/presubmit.yml create mode 100644 .bcr/sphinxdocs/source.template.json create mode 100644 sphinxdocs/.bazelrc create mode 100644 sphinxdocs/.bazelrc.deleted_packages create mode 100644 sphinxdocs/.bazelversion create mode 100644 sphinxdocs/MODULE.bazel rename sphinxdocs/{ => sphinxdocs}/BUILD.bazel (97%) rename sphinxdocs/{ => sphinxdocs}/docs/BUILD.bazel (89%) rename sphinxdocs/{ => sphinxdocs}/docs/api/index.md (100%) rename sphinxdocs/{ => sphinxdocs}/docs/api/sphinxdocs/index.md (91%) rename sphinxdocs/{ => sphinxdocs}/docs/api/sphinxdocs/inventories/index.md (71%) rename sphinxdocs/{ => sphinxdocs}/docs/index.md (100%) rename sphinxdocs/{ => sphinxdocs}/docs/readthedocs.md (92%) rename sphinxdocs/{ => sphinxdocs}/docs/sphinx-bzl.md (99%) rename sphinxdocs/{ => sphinxdocs}/docs/starlark-docgen.md (96%) create mode 100644 sphinxdocs/sphinxdocs/integration_tests/bcr/BUILD.bazel create mode 100644 sphinxdocs/sphinxdocs/integration_tests/bcr/MODULE.bazel create mode 100644 sphinxdocs/sphinxdocs/integration_tests/bcr/conf.py create mode 100644 sphinxdocs/sphinxdocs/integration_tests/bcr/index.md rename sphinxdocs/{ => sphinxdocs}/inventories/BUILD.bazel (100%) rename sphinxdocs/{ => sphinxdocs}/inventories/bazel_inventory.txt (100%) rename sphinxdocs/{ => sphinxdocs}/private/BUILD.bazel (85%) rename sphinxdocs/{ => sphinxdocs}/private/inventory_builder.py (100%) rename sphinxdocs/{ => sphinxdocs}/private/proto_to_markdown.py (100%) rename sphinxdocs/{ => sphinxdocs}/private/readthedocs.bzl (87%) rename sphinxdocs/{ => sphinxdocs}/private/readthedocs_install.py (100%) rename sphinxdocs/{ => sphinxdocs}/private/sphinx.bzl (97%) rename sphinxdocs/{ => sphinxdocs}/private/sphinx_build.py (100%) rename sphinxdocs/{ => sphinxdocs}/private/sphinx_docs_library.bzl (100%) rename sphinxdocs/{ => sphinxdocs}/private/sphinx_docs_library_info.bzl (100%) rename sphinxdocs/{ => sphinxdocs}/private/sphinx_docs_library_macro.bzl (70%) rename sphinxdocs/{ => sphinxdocs}/private/sphinx_run_template.sh (100%) rename sphinxdocs/{ => sphinxdocs}/private/sphinx_server.py (100%) rename sphinxdocs/{ => sphinxdocs}/private/sphinx_stardoc.bzl (97%) create mode 100644 sphinxdocs/sphinxdocs/private/util.bzl rename sphinxdocs/{ => sphinxdocs}/readthedocs.bzl (100%) rename sphinxdocs/{ => sphinxdocs}/sphinx.bzl (100%) rename sphinxdocs/{ => sphinxdocs}/sphinx_docs_library.bzl (100%) rename sphinxdocs/{ => sphinxdocs}/sphinx_stardoc.bzl (100%) rename sphinxdocs/{ => sphinxdocs}/src/sphinx_bzl/BUILD.bazel (69%) rename sphinxdocs/{ => sphinxdocs}/src/sphinx_bzl/__init__.py (100%) rename sphinxdocs/{ => sphinxdocs}/src/sphinx_bzl/bzl.py (99%) rename sphinxdocs/{ => sphinxdocs}/tests/BUILD.bazel (100%) rename sphinxdocs/{ => sphinxdocs}/tests/proto_to_markdown/BUILD.bazel (93%) rename sphinxdocs/{ => sphinxdocs}/tests/proto_to_markdown/proto_to_markdown_test.py (100%) rename sphinxdocs/{ => sphinxdocs}/tests/sphinx_docs/BUILD.bazel (100%) rename sphinxdocs/{ => sphinxdocs}/tests/sphinx_docs/conf.py (100%) rename sphinxdocs/{ => sphinxdocs}/tests/sphinx_docs/defs.bzl (100%) rename sphinxdocs/{ => sphinxdocs}/tests/sphinx_docs/doc1.md (100%) rename sphinxdocs/{ => sphinxdocs}/tests/sphinx_docs/doc2.md (100%) rename sphinxdocs/{ => sphinxdocs}/tests/sphinx_docs/index.md (100%) rename sphinxdocs/{ => sphinxdocs}/tests/sphinx_stardoc/BUILD.bazel (98%) rename sphinxdocs/{ => sphinxdocs}/tests/sphinx_stardoc/aspect.md (100%) rename sphinxdocs/{ => sphinxdocs}/tests/sphinx_stardoc/bzl_function.bzl (100%) rename sphinxdocs/{ => sphinxdocs}/tests/sphinx_stardoc/bzl_providers.bzl (100%) rename sphinxdocs/{ => sphinxdocs}/tests/sphinx_stardoc/bzl_rule.bzl (100%) rename sphinxdocs/{ => sphinxdocs}/tests/sphinx_stardoc/bzl_typedef.bzl (100%) rename sphinxdocs/{ => sphinxdocs}/tests/sphinx_stardoc/conf.py (100%) rename sphinxdocs/{ => sphinxdocs}/tests/sphinx_stardoc/envvars.md (100%) rename sphinxdocs/{ => sphinxdocs}/tests/sphinx_stardoc/function.md (100%) rename sphinxdocs/{ => sphinxdocs}/tests/sphinx_stardoc/glossary.md (100%) rename sphinxdocs/{ => sphinxdocs}/tests/sphinx_stardoc/index.md (76%) rename sphinxdocs/{ => sphinxdocs}/tests/sphinx_stardoc/module_extension.md (100%) rename sphinxdocs/{ => sphinxdocs}/tests/sphinx_stardoc/provider.md (100%) rename sphinxdocs/{ => sphinxdocs}/tests/sphinx_stardoc/repo_rule.md (100%) rename sphinxdocs/{ => sphinxdocs}/tests/sphinx_stardoc/rule.md (100%) rename sphinxdocs/{ => sphinxdocs}/tests/sphinx_stardoc/sphinx_output_test.py (100%) rename sphinxdocs/{ => sphinxdocs}/tests/sphinx_stardoc/target.md (100%) rename sphinxdocs/{ => sphinxdocs}/tests/sphinx_stardoc/typedef.md (100%) rename sphinxdocs/{ => sphinxdocs}/tests/sphinx_stardoc/xrefs.md (92%) diff --git a/.bazelci/presubmit.yml b/.bazelci/presubmit.yml index 43e519259c..b33b8a8d20 100644 --- a/.bazelci/presubmit.yml +++ b/.bazelci/presubmit.yml @@ -174,6 +174,23 @@ tasks: test_targets: ["//..."] working_directory: gazelle + sphinxdocs_ubuntu: + name: "Sphinxdocs: Ubuntu" + working_directory: "sphinxdocs" + platform: ubuntu2204 + build_targets: + - "//..." + test_targets: + - "//..." + sphinxdocs_mac: + name: "Sphinxdocs: Mac" + working_directory: "sphinxdocs" + platform: macos_arm64 + build_targets: + - "//..." + test_targets: + - "//..." + ubuntu_min_workspace: <<: *minimum_supported_version <<: *reusable_config diff --git a/.bazelignore b/.bazelignore index 0384d0746e..89e06444a4 100644 --- a/.bazelignore +++ b/.bazelignore @@ -29,6 +29,7 @@ gazelle/examples/bzlmod_build_file_generation/bazel-bin gazelle/examples/bzlmod_build_file_generation/bazel-bzlmod_build_file_generation gazelle/examples/bzlmod_build_file_generation/bazel-out gazelle/examples/bzlmod_build_file_generation/bazel-testlog +sphinxdocs tests/integration/compile_pip_requirements/bazel-compile_pip_requirements tests/integration/local_toolchains/bazel-local_toolchains tests/integration/py_cc_toolchain_registered/bazel-py_cc_toolchain_registered diff --git a/.bcr/config.yml b/.bcr/config.yml index f103e35bf7..038761b6df 100644 --- a/.bcr/config.yml +++ b/.bcr/config.yml @@ -12,4 +12,4 @@ # See the License for the specific language governing permissions and # limitations under the License. -moduleRoots: [".", "gazelle"] +moduleRoots: [".", "gazelle", "sphinxdocs"] diff --git a/.bcr/sphinxdocs/metadata.template.json b/.bcr/sphinxdocs/metadata.template.json new file mode 100644 index 0000000000..017f9d3774 --- /dev/null +++ b/.bcr/sphinxdocs/metadata.template.json @@ -0,0 +1,21 @@ +{ + "homepage": "https://github.com/bazel-contrib/rules_python", + "maintainers": [ + { + "name": "Richard Levasseur", + "email": "richardlev@gmail.com", + "github": "rickeylev" + }, + { + "name": "Ignas Anikevicius", + "email": "bcr-ignas@use.startmail.com", + "github": "aignas" + } + ], + "repository": [ + "github:bazelbuild/rules_python", + "github:bazel-contrib/rules_python" + ], + "versions": [], + "yanked_versions": {} +} diff --git a/.bcr/sphinxdocs/presubmit.yml b/.bcr/sphinxdocs/presubmit.yml new file mode 100644 index 0000000000..00a6bd37aa --- /dev/null +++ b/.bcr/sphinxdocs/presubmit.yml @@ -0,0 +1,23 @@ +bcr_test_module: + module_path: "integration_tests/bcr" + matrix: + platform: ["debian11", "macos", "ubuntu2204"] + bazel: [8.*, 9.*] + tasks: + run_tests: + name: "Run test module" + platform: ${{ platform }} + bazel: ${{ bazel }} + shell_commands: + - "echo 'common --override_module=rules_python=' >> .bazelrc" + batch_commands: + - "echo common --override_module=rules_python= >> .bazelrc" + test_flags: + # Minimum bazel supported C++ + - "--keep_going" + - '--cxxopt=-std=c++17' + - '--host_cxxopt=-std=c++17' + build_targets: + - "//..." + test_targets: + - "//..." diff --git a/.bcr/sphinxdocs/source.template.json b/.bcr/sphinxdocs/source.template.json new file mode 100644 index 0000000000..b2a2d1ef23 --- /dev/null +++ b/.bcr/sphinxdocs/source.template.json @@ -0,0 +1,5 @@ +{ + "integrity": "", + "strip_prefix": "{REPO}-{VERSION}/sphinxdocs", + "url": "https://github.com/{OWNER}/{REPO}/releases/download/{TAG}/rules_python-{TAG}.tar.gz" +} diff --git a/MODULE.bazel b/MODULE.bazel index a8af3be582..d2d2d72f78 100644 --- a/MODULE.bazel +++ b/MODULE.bazel @@ -211,10 +211,13 @@ pip.parse( ) use_repo(pip, "rules_python_publish_deps") -# Not a dev dependency to allow usage of //sphinxdocs code, which refers to stardoc repos. -bazel_dep(name = "stardoc", version = "0.7.2", repo_name = "io_bazel_stardoc") - # ===== DEV ONLY DEPS AND SETUP BELOW HERE ===== +bazel_dep(name = "sphinxdocs", version = "0.0.0", dev_dependency = True) +local_path_override( + module_name = "sphinxdocs", + path = "sphinxdocs", +) + bazel_dep(name = "rules_bazel_integration_test", version = "0.27.0", dev_dependency = True) bazel_dep(name = "rules_testing", version = "0.6.0", dev_dependency = True) bazel_dep(name = "rules_shell", version = "0.3.0", dev_dependency = True) diff --git a/docs/BUILD.bazel b/docs/BUILD.bazel index 656c06b6e1..1c31a47d56 100644 --- a/docs/BUILD.bazel +++ b/docs/BUILD.bazel @@ -14,13 +14,13 @@ load("@bazel_skylib//rules:build_test.bzl", "build_test") load("@dev_pip//:requirements.bzl", "requirement") +load("@sphinxdocs//sphinxdocs:readthedocs.bzl", "readthedocs_install") +load("@sphinxdocs//sphinxdocs:sphinx.bzl", "sphinx_build_binary", "sphinx_docs") +load("@sphinxdocs//sphinxdocs:sphinx_docs_library.bzl", "sphinx_docs_library") +load("@sphinxdocs//sphinxdocs:sphinx_stardoc.bzl", "sphinx_stardoc", "sphinx_stardocs") load("//python/private:bzlmod_enabled.bzl", "BZLMOD_ENABLED") # buildifier: disable=bzl-visibility load("//python/private:common_labels.bzl", "labels") # buildifier: disable=bzl-visibility load("//python/uv:lock.bzl", "lock") # buildifier: disable=bzl-visibility -load("//sphinxdocs:readthedocs.bzl", "readthedocs_install") -load("//sphinxdocs:sphinx.bzl", "sphinx_build_binary", "sphinx_docs") -load("//sphinxdocs:sphinx_docs_library.bzl", "sphinx_docs_library") -load("//sphinxdocs:sphinx_stardoc.bzl", "sphinx_stardoc", "sphinx_stardocs") package(default_visibility = ["//:__subpackages__"]) @@ -63,7 +63,7 @@ sphinx_docs( renamed_srcs = { "//:CHANGELOG.md": "changelog.md", "//:CONTRIBUTING.md": "contributing.md", - "//sphinxdocs/inventories:bazel_inventory": "bazel_inventory.inv", + "@sphinxdocs//sphinxdocs/inventories:bazel_inventory": "bazel_inventory.inv", }, sphinx = ":sphinx-build", strip_prefix = package_name() + "/", @@ -73,7 +73,7 @@ sphinx_docs( ":bzl_api_docs", ":py_api_srcs", ":py_runtime_pair", - "//sphinxdocs/docs:docs_lib", + "@sphinxdocs//sphinxdocs/docs:docs_lib", ], ) @@ -182,7 +182,7 @@ sphinx_build_binary( requirement("typing_extensions"), requirement("sphinx_autodoc2"), requirement("sphinx_reredirects"), - "//sphinxdocs/src/sphinx_bzl", + "@sphinxdocs//sphinxdocs/src/sphinx_bzl", ], ) diff --git a/docs/conf.py b/docs/conf.py index 6a7cfe178f..541d99ef7a 100644 --- a/docs/conf.py +++ b/docs/conf.py @@ -7,7 +7,7 @@ copyright = "2023, The Bazel Authors" author = "Bazel" -# NOTE: These are overriden by -D flags via --//sphinxdocs:extra_defines +# NOTE: These are overriden by -D flags via --@sphinxdocs//sphinxdocs:extra_defines version = "0.0.0" release = version @@ -139,7 +139,10 @@ # --- Extlinks configuration extlinks = { - "gh-issue": (f"https://github.com/bazel-contrib/rules_python/issues/%s", "#%s issue"), + "gh-issue": ( + f"https://github.com/bazel-contrib/rules_python/issues/%s", + "#%s issue", + ), "gh-path": (f"https://github.com/bazel-contrib/rules_python/tree/main/%s", "%s"), "gh-pr": (f"https://github.com/bazel-contrib/rules_python/pull/%s", "#%s PR"), } diff --git a/docs/readthedocs_build.sh b/docs/readthedocs_build.sh index 06ac7698d2..6762ffd630 100755 --- a/docs/readthedocs_build.sh +++ b/docs/readthedocs_build.sh @@ -5,12 +5,12 @@ set -eou pipefail declare -a extra_env while IFS='=' read -r -d '' name value; do if [[ "$name" == READTHEDOCS* ]]; then - extra_env+=("--//sphinxdocs:extra_env=$name=$value") + extra_env+=("--@sphinxdocs//sphinxdocs:extra_env=$name=$value") fi done < <(env -0) # In order to get the build number, we extract it from the host name -extra_env+=("--//sphinxdocs:extra_env=HOSTNAME=$HOSTNAME") +extra_env+=("--@sphinxdocs//sphinxdocs:extra_env=HOSTNAME=$HOSTNAME") export RULES_PYTHON_ENABLE_PIPSTAR=1 @@ -18,6 +18,6 @@ set -x export RULES_PYTHON_ENABLE_PIPSTAR=1 bazel run \ --config=rtd \ - "--//sphinxdocs:extra_defines=version=$READTHEDOCS_VERSION" \ + "--@sphinxdocs//sphinxdocs:extra_defines=version=$READTHEDOCS_VERSION" \ "${extra_env[@]}" \ //docs:readthedocs_install diff --git a/internal_dev_deps.bzl b/internal_dev_deps.bzl index 811240a06a..50277ad4ad 100644 --- a/internal_dev_deps.bzl +++ b/internal_dev_deps.bzl @@ -48,6 +48,13 @@ def rules_python_internal_deps(): ], ) + # Sphinxdocs doesn't support workspace mode, but we have to define it + # so that load() passes. + local_repository( + name = "sphinxdocs", + path = "sphinxdocs", + ) + local_repository( name = "other", path = "tests/modules/other", diff --git a/python/private/BUILD.bazel b/python/private/BUILD.bazel index 70f7f86413..db96957724 100644 --- a/python/private/BUILD.bazel +++ b/python/private/BUILD.bazel @@ -24,7 +24,9 @@ load(":stamp.bzl", "stamp_build_setting") load(":uncachable_version_file.bzl", "define_uncachable_version_file") package( - default_visibility = ["//:__subpackages__"], + default_visibility = [ + "//:__subpackages__", + ], ) licenses(["notice"]) @@ -689,9 +691,6 @@ bzl_library( bzl_library( name = "util_bzl", srcs = ["util.bzl"], - visibility = [ - "//:__subpackages__", - ], deps = [ ":py_internal_bzl", "@bazel_skylib//lib:types", diff --git a/sphinxdocs/.bazelrc b/sphinxdocs/.bazelrc new file mode 100644 index 0000000000..caefd0af53 --- /dev/null +++ b/sphinxdocs/.bazelrc @@ -0,0 +1,22 @@ +import %workspace%/.bazelrc.deleted_packages + +test --test_output=errors + +build --incompatible_default_to_explicit_init_py +build --@rules_python//python/config_settings:incompatible_default_to_explicit_init_py=True + +# Ensure ongoing compatibility with this flag. +common --incompatible_disallow_struct_provider_syntax +# Makes Bazel 7 act more like Bazel 8 +common --incompatible_use_plus_in_repo_names + +# Explicitly enable for Windows +build --enable_runfiles + +# Local disk cache greatly speeds up builds if the regular cache is lost +common --disk_cache=~/.cache/bazel/bazel-disk-cache + +common --incompatible_python_disallow_native_rules +common --incompatible_no_implicit_file_export + +build --lockfile_mode=update diff --git a/sphinxdocs/.bazelrc.deleted_packages b/sphinxdocs/.bazelrc.deleted_packages new file mode 100644 index 0000000000..fd6d39d64f --- /dev/null +++ b/sphinxdocs/.bazelrc.deleted_packages @@ -0,0 +1 @@ +common --deleted_packages=sphinxdocs/integration_tests/bcr diff --git a/sphinxdocs/.bazelversion b/sphinxdocs/.bazelversion new file mode 100644 index 0000000000..512e4c889e --- /dev/null +++ b/sphinxdocs/.bazelversion @@ -0,0 +1 @@ +9.x diff --git a/sphinxdocs/MODULE.bazel b/sphinxdocs/MODULE.bazel new file mode 100644 index 0000000000..bbb9d7a688 --- /dev/null +++ b/sphinxdocs/MODULE.bazel @@ -0,0 +1,27 @@ +module( + name = "sphinxdocs", + version = "0.0.0", + compatibility_level = 1, +) + +bazel_dep(name = "bazel_skylib", version = "1.8.2") +bazel_dep(name = "stardoc", version = "0.7.2", repo_name = "io_bazel_stardoc") +bazel_dep(name = "platforms", version = "0.0.11") +bazel_dep(name = "protobuf", version = "29.0-rc2", repo_name = "com_google_protobuf") +bazel_dep(name = "rules_python", version = "0.0.0") +local_path_override( + module_name = "rules_python", + path = "..", +) + +dev_pip = use_extension( + "@rules_python//python/extensions:pip.bzl", + "pip", + dev_dependency = True, +) +dev_pip.parse( + hub_name = "dev_pip", + python_version = "3.11", + requirements_lock = "@rules_python//docs:requirements.txt", +) +use_repo(dev_pip, "dev_pip") diff --git a/sphinxdocs/BUILD.bazel b/sphinxdocs/sphinxdocs/BUILD.bazel similarity index 97% rename from sphinxdocs/BUILD.bazel rename to sphinxdocs/sphinxdocs/BUILD.bazel index 9ad1e1eef9..893db8214a 100644 --- a/sphinxdocs/BUILD.bazel +++ b/sphinxdocs/sphinxdocs/BUILD.bazel @@ -17,7 +17,7 @@ load("@bazel_skylib//rules:common_settings.bzl", "bool_flag") load("//sphinxdocs/private:sphinx.bzl", "repeated_string_list_flag") package( - default_visibility = ["//:__subpackages__"], + default_visibility = ["//sphinxdocs:__subpackages__"], ) # Additional -D values to add to every Sphinx build. diff --git a/sphinxdocs/docs/BUILD.bazel b/sphinxdocs/sphinxdocs/docs/BUILD.bazel similarity index 89% rename from sphinxdocs/docs/BUILD.bazel rename to sphinxdocs/sphinxdocs/docs/BUILD.bazel index 070e0485d7..87771f14f1 100644 --- a/sphinxdocs/docs/BUILD.bazel +++ b/sphinxdocs/sphinxdocs/docs/BUILD.bazel @@ -1,8 +1,8 @@ -load("//python/private:bzlmod_enabled.bzl", "BZLMOD_ENABLED") # buildifier: disable=bzl-visibility load("//sphinxdocs:sphinx_docs_library.bzl", "sphinx_docs_library") load("//sphinxdocs:sphinx_stardoc.bzl", "sphinx_stardocs") +load("//sphinxdocs/private:util.bzl", "BZLMOD_ENABLED") # buildifier: disable=bzl-visibility -package(default_visibility = ["//:__subpackages__"]) +package(default_visibility = ["//sphinxdocs:__subpackages__"]) # We only build for Linux and Mac because: # 1. The actual doc process only runs on Linux @@ -18,6 +18,7 @@ _TARGET_COMPATIBLE_WITH = select({ sphinx_docs_library( name = "docs_lib", + visibility = ["//visibility:public"], deps = [ ":artisian_api_docs", ":bzl_docs", diff --git a/sphinxdocs/docs/api/index.md b/sphinxdocs/sphinxdocs/docs/api/index.md similarity index 100% rename from sphinxdocs/docs/api/index.md rename to sphinxdocs/sphinxdocs/docs/api/index.md diff --git a/sphinxdocs/docs/api/sphinxdocs/index.md b/sphinxdocs/sphinxdocs/docs/api/sphinxdocs/index.md similarity index 91% rename from sphinxdocs/docs/api/sphinxdocs/index.md rename to sphinxdocs/sphinxdocs/docs/api/sphinxdocs/index.md index bd4e9b6eec..a59edb2b65 100644 --- a/sphinxdocs/docs/api/sphinxdocs/index.md +++ b/sphinxdocs/sphinxdocs/docs/api/sphinxdocs/index.md @@ -1,7 +1,7 @@ -:::{bzl:currentfile} //sphinxdocs:BUILD.bazel +:::{bzl:currentfile} //:BUILD.bazel ::: -# //sphinxdocs +# // :::{bzl:flag} extra_defines Additional `-D` values to add to every Sphinx build. diff --git a/sphinxdocs/docs/api/sphinxdocs/inventories/index.md b/sphinxdocs/sphinxdocs/docs/api/sphinxdocs/inventories/index.md similarity index 71% rename from sphinxdocs/docs/api/sphinxdocs/inventories/index.md rename to sphinxdocs/sphinxdocs/docs/api/sphinxdocs/inventories/index.md index a03645ed44..d0b983fe7f 100644 --- a/sphinxdocs/docs/api/sphinxdocs/inventories/index.md +++ b/sphinxdocs/sphinxdocs/docs/api/sphinxdocs/inventories/index.md @@ -1,7 +1,7 @@ -:::{bzl:currentfile} //sphinxdocs/inventories:BUILD.bazel +:::{bzl:currentfile} //inventories:BUILD.bazel ::: -# //sphinxdocs/inventories +# //inventories :::{bzl:target} bazel_inventory A Sphinx inventory of Bazel objects. diff --git a/sphinxdocs/docs/index.md b/sphinxdocs/sphinxdocs/docs/index.md similarity index 100% rename from sphinxdocs/docs/index.md rename to sphinxdocs/sphinxdocs/docs/index.md diff --git a/sphinxdocs/docs/readthedocs.md b/sphinxdocs/sphinxdocs/docs/readthedocs.md similarity index 92% rename from sphinxdocs/docs/readthedocs.md rename to sphinxdocs/sphinxdocs/docs/readthedocs.md index c347d19850..bcdae833d9 100644 --- a/sphinxdocs/docs/readthedocs.md +++ b/sphinxdocs/sphinxdocs/docs/readthedocs.md @@ -26,8 +26,8 @@ In the example below, `npm` is used to install Bazelisk and a helper shell script, `readthedocs_build.sh` is used to construct the Bazel invocation. The key purpose of the shell script it to set the -`--@rules_python//sphinxdocs:extra_env` and -`--@rules_python//sphinxdocs:extra_defines` flags. These are used to communicate +`@sphinxdocs//sphinxdocs::extra_env` and +`@sphinxdocs//sphinxdocs::extra_defines` flags. These are used to communicate `READTHEDOCS*` environment variables and settings to the Bazel invocation. ## BUILD config @@ -73,7 +73,7 @@ build: ``` # File: docs/BUILD -load("@rules_python//sphinxdocs:readthedocs.bzl.bzl", "readthedocs_install") +load("//:readthedocs.bzl", "readthedocs_install") readthedocs_install( name = "readthedocs_install", docs = [":docs"], @@ -90,17 +90,17 @@ set -eou pipefail declare -a extra_env while IFS='=' read -r -d '' name value; do if [[ "$name" == READTHEDOCS* ]]; then - extra_env+=("--@rules_python//sphinxdocs:extra_env=$name=$value") + extra_env+=("--@sphinxdocs//sphinxdocs:extra_env=$name=$value") fi done < <(env -0) # In order to get the build number, we extract it from the host name -extra_env+=("--@rules_python//sphinxdocs:extra_env=HOSTNAME=$HOSTNAME") +extra_env+=("--@sphinxdocs//sphinxdocs:extra_env=HOSTNAME=$HOSTNAME") set -x bazel run \ --stamp \ - "--@rules_python//sphinxdocs:extra_defines=version=$READTHEDOCS_VERSION" \ + "--@sphinxdocs//sphinxdocs:extra_defines=version=$READTHEDOCS_VERSION" \ "${extra_env[@]}" \ //docs:readthedocs_install ``` diff --git a/sphinxdocs/docs/sphinx-bzl.md b/sphinxdocs/sphinxdocs/docs/sphinx-bzl.md similarity index 99% rename from sphinxdocs/docs/sphinx-bzl.md rename to sphinxdocs/sphinxdocs/docs/sphinx-bzl.md index da4cd9f293..c96061b984 100644 --- a/sphinxdocs/docs/sphinx-bzl.md +++ b/sphinxdocs/sphinxdocs/docs/sphinx-bzl.md @@ -11,7 +11,7 @@ a well known target. ## Configuring Sphinx To enable the plugin in Sphinx, depend on -`@rules_python//sphinxdocs/src/sphinx_bzl` and enable it in `conf.py`: +`//sphinxdocs/src/sphinx_bzl` and enable it in `conf.py`: ``` extensions = [ diff --git a/sphinxdocs/docs/starlark-docgen.md b/sphinxdocs/sphinxdocs/docs/starlark-docgen.md similarity index 96% rename from sphinxdocs/docs/starlark-docgen.md rename to sphinxdocs/sphinxdocs/docs/starlark-docgen.md index b9181ee347..0b89393ced 100644 --- a/sphinxdocs/docs/starlark-docgen.md +++ b/sphinxdocs/sphinxdocs/docs/starlark-docgen.md @@ -13,7 +13,7 @@ While the `sphinx_stardoc` rule doesn't require Sphinx itself, the source it generates requires some additional Sphinx plugins and config settings. When defining the `sphinx_build_binary` target, also depend on: -* `@rules_python//sphinxdocs/src/sphinx_bzl:sphinx_bzl` +* `//sphinxdocs/src/sphinx_bzl:sphinx_bzl` * `myst_parser` (e.g. `@pypi//myst_parser`) * `typing_extensions` (e.g. `@pypi//myst_parser`) @@ -21,7 +21,7 @@ When defining the `sphinx_build_binary` target, also depend on: sphinx_build_binary( name = "sphinx-build", deps = [ - "@rules_python//sphinxdocs/src/sphinx_bzl", + "//sphinxdocs/src/sphinx_bzl", "@pypi//myst_parser", "@pypi//typing_extensions", ... diff --git a/sphinxdocs/sphinxdocs/integration_tests/bcr/BUILD.bazel b/sphinxdocs/sphinxdocs/integration_tests/bcr/BUILD.bazel new file mode 100644 index 0000000000..1aaa69b826 --- /dev/null +++ b/sphinxdocs/sphinxdocs/integration_tests/bcr/BUILD.bazel @@ -0,0 +1,7 @@ +load("@sphinxdocs//sphinxdocs:sphinx.bzl", "sphinx_docs") + +sphinx_docs( + name = "docs", + srcs = ["index.md"], + conf = "conf.py", +) diff --git a/sphinxdocs/sphinxdocs/integration_tests/bcr/MODULE.bazel b/sphinxdocs/sphinxdocs/integration_tests/bcr/MODULE.bazel new file mode 100644 index 0000000000..6a25aa5e4c --- /dev/null +++ b/sphinxdocs/sphinxdocs/integration_tests/bcr/MODULE.bazel @@ -0,0 +1,16 @@ +module( + name = "sphinxdocs_example", + version = "0.0.0", +) + +bazel_dep(name = "sphinxdocs", version = "0.0.0") +local_path_override( + module_name = "sphinxdocs", + path = "../..", +) + +bazel_dep(name = "rules_python", version = "0.0.0") +local_path_override( + module_name = "rules_python", + path = "../../..", +) diff --git a/sphinxdocs/sphinxdocs/integration_tests/bcr/conf.py b/sphinxdocs/sphinxdocs/integration_tests/bcr/conf.py new file mode 100644 index 0000000000..3b751bf402 --- /dev/null +++ b/sphinxdocs/sphinxdocs/integration_tests/bcr/conf.py @@ -0,0 +1,2 @@ +extensions = ["myst_parser"] +master_doc = "index" diff --git a/sphinxdocs/sphinxdocs/integration_tests/bcr/index.md b/sphinxdocs/sphinxdocs/integration_tests/bcr/index.md new file mode 100644 index 0000000000..bfe100937d --- /dev/null +++ b/sphinxdocs/sphinxdocs/integration_tests/bcr/index.md @@ -0,0 +1 @@ +# Sphinxdocs example diff --git a/sphinxdocs/inventories/BUILD.bazel b/sphinxdocs/sphinxdocs/inventories/BUILD.bazel similarity index 100% rename from sphinxdocs/inventories/BUILD.bazel rename to sphinxdocs/sphinxdocs/inventories/BUILD.bazel diff --git a/sphinxdocs/inventories/bazel_inventory.txt b/sphinxdocs/sphinxdocs/inventories/bazel_inventory.txt similarity index 100% rename from sphinxdocs/inventories/bazel_inventory.txt rename to sphinxdocs/sphinxdocs/inventories/bazel_inventory.txt diff --git a/sphinxdocs/private/BUILD.bazel b/sphinxdocs/sphinxdocs/private/BUILD.bazel similarity index 85% rename from sphinxdocs/private/BUILD.bazel rename to sphinxdocs/sphinxdocs/private/BUILD.bazel index c707b4d1d8..5a37cbd309 100644 --- a/sphinxdocs/private/BUILD.bazel +++ b/sphinxdocs/sphinxdocs/private/BUILD.bazel @@ -14,16 +14,16 @@ load("@bazel_skylib//:bzl_library.bzl", "bzl_library") load("@com_google_protobuf//bazel:py_proto_library.bzl", "py_proto_library") -load("//python:py_binary.bzl", "py_binary") -load("//python:py_library.bzl", "py_library") +load("@rules_python//python:py_binary.bzl", "py_binary") +load("@rules_python//python:py_library.bzl", "py_library") package( default_visibility = ["//sphinxdocs:__subpackages__"], ) -# These are only exported because they're passed as files to the //sphinxdocs +# These are only exported because they're passed as files to the @sphinxdocs # macros, and thus must be visible to other packages. They should only be -# referenced by the //sphinxdocs macros. +# referenced by the @sphinxdocs macros. exports_files( [ "readthedocs_install.py", @@ -34,12 +34,20 @@ exports_files( visibility = ["//visibility:public"], ) +bzl_library( + name = "util_bzl", + srcs = ["util.bzl"], + deps = [ + "@bazel_skylib//lib:types", + ], +) + bzl_library( name = "sphinx_docs_library_macro_bzl", srcs = ["sphinx_docs_library_macro.bzl"], deps = [ ":sphinx_docs_library_bzl", - "//python/private:util_bzl", + "//sphinxdocs/private:util_bzl", ], ) @@ -59,13 +67,14 @@ bzl_library( srcs = ["sphinx.bzl"], deps = [ ":sphinx_docs_library_info_bzl", - "//python:py_binary_bzl", + ":util_bzl", "@bazel_skylib//:bzl_library", "@bazel_skylib//lib:paths", "@bazel_skylib//lib:types", "@bazel_skylib//rules:build_test", "@bazel_skylib//rules:common_settings", "@io_bazel_stardoc//stardoc:stardoc_lib", + "@rules_python//python:py_binary_bzl", ], ) @@ -74,8 +83,8 @@ bzl_library( srcs = ["sphinx_stardoc.bzl"], deps = [ ":sphinx_docs_library_macro_bzl", - "//python/private:util_bzl", "//sphinxdocs:sphinx_bzl", + "//sphinxdocs/private:util_bzl", "@bazel_skylib//:bzl_library", "@bazel_skylib//lib:paths", "@bazel_skylib//lib:types", @@ -87,7 +96,10 @@ bzl_library( bzl_library( name = "readthedocs_bzl", srcs = ["readthedocs.bzl"], - deps = ["//python:py_binary_bzl"], + deps = [ + ":util_bzl", + "@rules_python//python:py_binary_bzl", + ], ) py_binary( diff --git a/sphinxdocs/private/inventory_builder.py b/sphinxdocs/sphinxdocs/private/inventory_builder.py similarity index 100% rename from sphinxdocs/private/inventory_builder.py rename to sphinxdocs/sphinxdocs/private/inventory_builder.py diff --git a/sphinxdocs/private/proto_to_markdown.py b/sphinxdocs/sphinxdocs/private/proto_to_markdown.py similarity index 100% rename from sphinxdocs/private/proto_to_markdown.py rename to sphinxdocs/sphinxdocs/private/proto_to_markdown.py diff --git a/sphinxdocs/private/readthedocs.bzl b/sphinxdocs/sphinxdocs/private/readthedocs.bzl similarity index 87% rename from sphinxdocs/private/readthedocs.bzl rename to sphinxdocs/sphinxdocs/private/readthedocs.bzl index a62c51b86a..14bb1c9fcd 100644 --- a/sphinxdocs/private/readthedocs.bzl +++ b/sphinxdocs/sphinxdocs/private/readthedocs.bzl @@ -13,8 +13,8 @@ # limitations under the License. """Starlark rules for integrating Sphinx and Readthedocs.""" -load("//python:py_binary.bzl", "py_binary") -load("//python/private:util.bzl", "add_tag") # buildifier: disable=bzl-visibility +load("@rules_python//python:py_binary.bzl", "py_binary") +load("//sphinxdocs/private:util.bzl", "add_tag") # buildifier: disable=bzl-visibility _INSTALL_MAIN_SRC = Label("//sphinxdocs/private:readthedocs_install.py") @@ -33,7 +33,7 @@ def readthedocs_install(name, docs, **kwargs): is typically a single {obj}`sphinx_stardocs` target. **kwargs: {type}`dict` additional kwargs to pass onto the installer """ - add_tag(kwargs, "@rules_python//sphinxdocs:readthedocs_install") + add_tag(kwargs, "//sphinxdocs:readthedocs_install") py_binary( name = name, srcs = [_INSTALL_MAIN_SRC], @@ -43,6 +43,6 @@ def readthedocs_install(name, docs, **kwargs): "$(rlocationpaths {})".format(d) for d in docs ], - deps = [Label("//python/runfiles")], + deps = [Label("@rules_python//python/runfiles")], **kwargs ) diff --git a/sphinxdocs/private/readthedocs_install.py b/sphinxdocs/sphinxdocs/private/readthedocs_install.py similarity index 100% rename from sphinxdocs/private/readthedocs_install.py rename to sphinxdocs/sphinxdocs/private/readthedocs_install.py diff --git a/sphinxdocs/private/sphinx.bzl b/sphinxdocs/sphinxdocs/private/sphinx.bzl similarity index 97% rename from sphinxdocs/private/sphinx.bzl rename to sphinxdocs/sphinxdocs/private/sphinx.bzl index 0efbc269c5..b7c051a154 100644 --- a/sphinxdocs/private/sphinx.bzl +++ b/sphinxdocs/sphinxdocs/private/sphinx.bzl @@ -16,8 +16,8 @@ load("@bazel_skylib//lib:paths.bzl", "paths") load("@bazel_skylib//rules:common_settings.bzl", "BuildSettingInfo") -load("//python:py_binary.bzl", "py_binary") -load("//python/private:util.bzl", "add_tag", "copy_propagating_kwargs") # buildifier: disable=bzl-visibility +load("@rules_python//python:py_binary.bzl", "py_binary") +load("//sphinxdocs/private:util.bzl", "add_tag", "copy_propagating_kwargs") # buildifier: disable=bzl-visibility load(":sphinx_docs_library_info.bzl", "SphinxDocsLibraryInfo") _SPHINX_BUILD_MAIN_SRC = Label("//sphinxdocs/private:sphinx_build.py") @@ -83,7 +83,7 @@ def sphinx_build_binary(name, py_binary_rule = py_binary, **kwargs): **kwargs: {type}`dict` Additional kwargs to pass onto `py_binary`. The `srcs` and `main` attributes must not be specified. """ - add_tag(kwargs, "@rules_python//sphinxdocs:sphinx_build_binary") + add_tag(kwargs, "//sphinxdocs:sphinx_build_binary") py_binary_rule( name = name, srcs = [_SPHINX_BUILD_MAIN_SRC], @@ -134,7 +134,7 @@ def sphinx_docs( formats: (list of str) the formats (`-b` flag) to generate documentation in. Each format will become an output group. strip_prefix: {type}`str` A prefix to remove from the file paths of the - source files. e.g., given `//docs:foo.md`, stripping `docs/` makes + source files. e.g., given `//sphinxdocs/docs:foo.md`, stripping `docs/` makes Sphinx see `foo.md` in its generated source directory. If not specified, then {any}`native.package_name` is used. extra_opts: {type}`list[str]` Additional options to pass onto Sphinx building. @@ -148,7 +148,7 @@ def sphinx_docs( This can improve incremental building of docs. **kwargs: {type}`dict` Common attributes to pass onto rules. """ - add_tag(kwargs, "@rules_python//sphinxdocs:sphinx_docs") + add_tag(kwargs, "//sphinxdocs:sphinx_docs") common_kwargs = copy_propagating_kwargs(kwargs) internal_name = "_{}".format(name.lstrip("_")) @@ -189,7 +189,7 @@ def sphinx_docs( srcs = [_SPHINX_SERVE_MAIN_SRC], main = _SPHINX_SERVE_MAIN_SRC, data = [html_name], - deps = [Label("//python/runfiles")], + deps = [Label("@rules_python//python/runfiles")], args = [ "$(rlocationpath {})".format(html_name), ], @@ -482,7 +482,7 @@ def sphinx_inventory(*, name, src, **kwargs): the value `-` to indicate it is the same as `name` :::{seealso} - {bzl:obj}`//sphinxdocs/inventories` for inventories of Bazel objects. + {bzl:obj}`//inventories` for inventories of Bazel objects. ::: Args: diff --git a/sphinxdocs/private/sphinx_build.py b/sphinxdocs/sphinxdocs/private/sphinx_build.py similarity index 100% rename from sphinxdocs/private/sphinx_build.py rename to sphinxdocs/sphinxdocs/private/sphinx_build.py diff --git a/sphinxdocs/private/sphinx_docs_library.bzl b/sphinxdocs/sphinxdocs/private/sphinx_docs_library.bzl similarity index 100% rename from sphinxdocs/private/sphinx_docs_library.bzl rename to sphinxdocs/sphinxdocs/private/sphinx_docs_library.bzl diff --git a/sphinxdocs/private/sphinx_docs_library_info.bzl b/sphinxdocs/sphinxdocs/private/sphinx_docs_library_info.bzl similarity index 100% rename from sphinxdocs/private/sphinx_docs_library_info.bzl rename to sphinxdocs/sphinxdocs/private/sphinx_docs_library_info.bzl diff --git a/sphinxdocs/private/sphinx_docs_library_macro.bzl b/sphinxdocs/sphinxdocs/private/sphinx_docs_library_macro.bzl similarity index 70% rename from sphinxdocs/private/sphinx_docs_library_macro.bzl rename to sphinxdocs/sphinxdocs/private/sphinx_docs_library_macro.bzl index 095b3769ca..e93a2f5bf6 100644 --- a/sphinxdocs/private/sphinx_docs_library_macro.bzl +++ b/sphinxdocs/sphinxdocs/private/sphinx_docs_library_macro.bzl @@ -1,6 +1,6 @@ """Implementation of sphinx_docs_library macro.""" -load("//python/private:util.bzl", "add_tag") # buildifier: disable=bzl-visibility +load("//sphinxdocs/private:util.bzl", "add_tag") # buildifier: disable=bzl-visibility load(":sphinx_docs_library.bzl", _sphinx_docs_library = "sphinx_docs_library") def sphinx_docs_library(**kwargs): @@ -9,5 +9,5 @@ def sphinx_docs_library(**kwargs): Args: **kwargs: Args passed onto underlying {bzl:rule}`sphinx_docs_library` rule """ - add_tag(kwargs, "@rules_python//sphinxdocs:sphinx_docs_library") + add_tag(kwargs, "//sphinxdocs:sphinx_docs_library") _sphinx_docs_library(**kwargs) diff --git a/sphinxdocs/private/sphinx_run_template.sh b/sphinxdocs/sphinxdocs/private/sphinx_run_template.sh similarity index 100% rename from sphinxdocs/private/sphinx_run_template.sh rename to sphinxdocs/sphinxdocs/private/sphinx_run_template.sh diff --git a/sphinxdocs/private/sphinx_server.py b/sphinxdocs/sphinxdocs/private/sphinx_server.py similarity index 100% rename from sphinxdocs/private/sphinx_server.py rename to sphinxdocs/sphinxdocs/private/sphinx_server.py diff --git a/sphinxdocs/private/sphinx_stardoc.bzl b/sphinxdocs/sphinxdocs/private/sphinx_stardoc.bzl similarity index 97% rename from sphinxdocs/private/sphinx_stardoc.bzl rename to sphinxdocs/sphinxdocs/private/sphinx_stardoc.bzl index d5869b0bc4..ac76219299 100644 --- a/sphinxdocs/private/sphinx_stardoc.bzl +++ b/sphinxdocs/sphinxdocs/private/sphinx_stardoc.bzl @@ -19,8 +19,8 @@ load("@bazel_skylib//lib:paths.bzl", "paths") load("@bazel_skylib//lib:types.bzl", "types") load("@bazel_skylib//rules:build_test.bzl", "build_test") load("@io_bazel_stardoc//stardoc:stardoc.bzl", "stardoc") -load("//python/private:util.bzl", "add_tag", "copy_propagating_kwargs") # buildifier: disable=bzl-visibility load("//sphinxdocs/private:sphinx_docs_library_macro.bzl", "sphinx_docs_library") +load("//sphinxdocs/private:util.bzl", "add_tag", "copy_propagating_kwargs") # buildifier: disable=bzl-visibility _StardocInputHelperInfo = provider( doc = "Extracts the single source file from a bzl library.", @@ -73,7 +73,7 @@ def sphinx_stardocs( **kwargs: Additional kwargs to pass onto each `sphinx_stardoc` target """ internal_name = "_{}".format(name) - add_tag(kwargs, "@rules_python//sphinxdocs:sphinx_stardocs") + add_tag(kwargs, "//sphinxdocs:sphinx_stardocs") common_kwargs = copy_propagating_kwargs(kwargs) common_kwargs["target_compatible_with"] = kwargs.get("target_compatible_with") @@ -160,7 +160,7 @@ def sphinx_stardoc( **kwargs: {type}`dict` common args passed onto rules. """ internal_name = "_{}".format(name.lstrip("_")) - add_tag(kwargs, "@rules_python//sphinxdocs:sphinx_stardoc") + add_tag(kwargs, "//sphinxdocs:sphinx_stardoc") common_kwargs = copy_propagating_kwargs(kwargs) common_kwargs["target_compatible_with"] = kwargs.get("target_compatible_with") diff --git a/sphinxdocs/sphinxdocs/private/util.bzl b/sphinxdocs/sphinxdocs/private/util.bzl new file mode 100644 index 0000000000..f285a83c13 --- /dev/null +++ b/sphinxdocs/sphinxdocs/private/util.bzl @@ -0,0 +1,58 @@ +"""Utility functions used by sphinxdocs.""" + +load("@bazel_skylib//lib:types.bzl", "types") + +# When bzlmod is enabled, canonical repos names have @@ in them, while under +# workspace builds, there is never a @@ in labels. +BZLMOD_ENABLED = "@@" in str(Label("//sphinxdocs:unused")) + +def copy_propagating_kwargs(from_kwargs, into_kwargs = None): + """Copies args that must be compatible between two targets with a dependency relationship. + + This is intended for when one target depends on another, so they must have + compatible settings such as `testonly` and `compatible_with`. This usually + happens when a macro generates multiple targets, some of which depend + on one another, so their settings must be compatible. + + Args: + from_kwargs: keyword args dict whose common kwarg will be copied. + into_kwargs: optional keyword args dict that the values from `from_kwargs` + will be copied into. The values in this dict will take precedence + over the ones in `from_kwargs` (i.e., if this has `testonly` already + set, then it won't be overwritten). + NOTE: THIS WILL BE MODIFIED IN-PLACE. + + Returns: + Keyword args to use for the depender target derived from the dependency + target. If `into_kwargs` was passed in, then that same object is + returned; this is to facilitate easy `**` expansion. + """ + if into_kwargs == None: + into_kwargs = {} + + # Include tags because people generally expect tags to propagate. + for attr in ("testonly", "tags", "compatible_with", "restricted_to", "target_compatible_with"): + if attr in from_kwargs and attr not in into_kwargs: + into_kwargs[attr] = from_kwargs[attr] + return into_kwargs + +def add_tag(attrs, tag): + """Adds `tag` to `attrs["tags"]`. + + Args: + attrs: dict of keyword args. It is modified in place. + tag: str, the tag to add. + """ + if "tags" in attrs and attrs["tags"] != None: + tags = attrs["tags"] + + # Preserve the input type: this allows a test verifying the underlying + # rule can accept the tuple for the tags argument. + if types.is_tuple(tags): + attrs["tags"] = tags + (tag,) + else: + # List concatenation is necessary because the original value + # may be a frozen list. + attrs["tags"] = tags + [tag] + else: + attrs["tags"] = [tag] diff --git a/sphinxdocs/readthedocs.bzl b/sphinxdocs/sphinxdocs/readthedocs.bzl similarity index 100% rename from sphinxdocs/readthedocs.bzl rename to sphinxdocs/sphinxdocs/readthedocs.bzl diff --git a/sphinxdocs/sphinx.bzl b/sphinxdocs/sphinxdocs/sphinx.bzl similarity index 100% rename from sphinxdocs/sphinx.bzl rename to sphinxdocs/sphinxdocs/sphinx.bzl diff --git a/sphinxdocs/sphinx_docs_library.bzl b/sphinxdocs/sphinxdocs/sphinx_docs_library.bzl similarity index 100% rename from sphinxdocs/sphinx_docs_library.bzl rename to sphinxdocs/sphinxdocs/sphinx_docs_library.bzl diff --git a/sphinxdocs/sphinx_stardoc.bzl b/sphinxdocs/sphinxdocs/sphinx_stardoc.bzl similarity index 100% rename from sphinxdocs/sphinx_stardoc.bzl rename to sphinxdocs/sphinxdocs/sphinx_stardoc.bzl diff --git a/sphinxdocs/src/sphinx_bzl/BUILD.bazel b/sphinxdocs/sphinxdocs/src/sphinx_bzl/BUILD.bazel similarity index 69% rename from sphinxdocs/src/sphinx_bzl/BUILD.bazel rename to sphinxdocs/sphinxdocs/src/sphinx_bzl/BUILD.bazel index 8830315bc3..2dd25e09b3 100644 --- a/sphinxdocs/src/sphinx_bzl/BUILD.bazel +++ b/sphinxdocs/sphinxdocs/src/sphinx_bzl/BUILD.bazel @@ -1,7 +1,7 @@ -load("//python:py_library.bzl", "py_library") +load("@rules_python//python:py_library.bzl", "py_library") package( - default_visibility = ["//:__subpackages__"], + default_visibility = ["//sphinxdocs:__subpackages__"], ) # NOTE: This provides the library on its own, not its dependencies. diff --git a/sphinxdocs/src/sphinx_bzl/__init__.py b/sphinxdocs/sphinxdocs/src/sphinx_bzl/__init__.py similarity index 100% rename from sphinxdocs/src/sphinx_bzl/__init__.py rename to sphinxdocs/sphinxdocs/src/sphinx_bzl/__init__.py diff --git a/sphinxdocs/src/sphinx_bzl/bzl.py b/sphinxdocs/sphinxdocs/src/sphinx_bzl/bzl.py similarity index 99% rename from sphinxdocs/src/sphinx_bzl/bzl.py rename to sphinxdocs/sphinxdocs/src/sphinx_bzl/bzl.py index 7d5ec6a68e..a1f47b3b1d 100644 --- a/sphinxdocs/src/sphinx_bzl/bzl.py +++ b/sphinxdocs/sphinxdocs/src/sphinx_bzl/bzl.py @@ -57,7 +57,7 @@ def _log_debug(message, *args): # NOTE: Non-warning log messages go to stdout and are only # visible when -q isn't passed to Sphinx. Note that the sphinx_docs build - # rule passes -q by default; use --//sphinxdocs:quiet=false to disable it. + # rule passes -q by default; use --//:quiet=false to disable it. _logger.debug("%s" + message, _LOG_PREFIX, *args) @@ -1754,9 +1754,7 @@ def clear_doc(self, docname: str) -> None: if entry.full_id in self.data["objects"]: del self.data["objects"][entry.full_id] - if entry.full_id in self.data["objects_by_type"].get( - entry.object_type, {} - ): + if entry.full_id in self.data["objects_by_type"].get(entry.object_type, {}): del self.data["objects_by_type"][entry.object_type][entry.full_id] # We can't easily reverse the mapping for alt_names, so we have diff --git a/sphinxdocs/tests/BUILD.bazel b/sphinxdocs/sphinxdocs/tests/BUILD.bazel similarity index 100% rename from sphinxdocs/tests/BUILD.bazel rename to sphinxdocs/sphinxdocs/tests/BUILD.bazel diff --git a/sphinxdocs/tests/proto_to_markdown/BUILD.bazel b/sphinxdocs/sphinxdocs/tests/proto_to_markdown/BUILD.bazel similarity index 93% rename from sphinxdocs/tests/proto_to_markdown/BUILD.bazel rename to sphinxdocs/sphinxdocs/tests/proto_to_markdown/BUILD.bazel index 2964785eed..632d6d946f 100644 --- a/sphinxdocs/tests/proto_to_markdown/BUILD.bazel +++ b/sphinxdocs/sphinxdocs/tests/proto_to_markdown/BUILD.bazel @@ -12,7 +12,7 @@ # See the License for the specific language governing permissions and # limitations under the License. -load("//python:py_test.bzl", "py_test") +load("@rules_python//python:py_test.bzl", "py_test") py_test( name = "proto_to_markdown_test", diff --git a/sphinxdocs/tests/proto_to_markdown/proto_to_markdown_test.py b/sphinxdocs/sphinxdocs/tests/proto_to_markdown/proto_to_markdown_test.py similarity index 100% rename from sphinxdocs/tests/proto_to_markdown/proto_to_markdown_test.py rename to sphinxdocs/sphinxdocs/tests/proto_to_markdown/proto_to_markdown_test.py diff --git a/sphinxdocs/tests/sphinx_docs/BUILD.bazel b/sphinxdocs/sphinxdocs/tests/sphinx_docs/BUILD.bazel similarity index 100% rename from sphinxdocs/tests/sphinx_docs/BUILD.bazel rename to sphinxdocs/sphinxdocs/tests/sphinx_docs/BUILD.bazel diff --git a/sphinxdocs/tests/sphinx_docs/conf.py b/sphinxdocs/sphinxdocs/tests/sphinx_docs/conf.py similarity index 100% rename from sphinxdocs/tests/sphinx_docs/conf.py rename to sphinxdocs/sphinxdocs/tests/sphinx_docs/conf.py diff --git a/sphinxdocs/tests/sphinx_docs/defs.bzl b/sphinxdocs/sphinxdocs/tests/sphinx_docs/defs.bzl similarity index 100% rename from sphinxdocs/tests/sphinx_docs/defs.bzl rename to sphinxdocs/sphinxdocs/tests/sphinx_docs/defs.bzl diff --git a/sphinxdocs/tests/sphinx_docs/doc1.md b/sphinxdocs/sphinxdocs/tests/sphinx_docs/doc1.md similarity index 100% rename from sphinxdocs/tests/sphinx_docs/doc1.md rename to sphinxdocs/sphinxdocs/tests/sphinx_docs/doc1.md diff --git a/sphinxdocs/tests/sphinx_docs/doc2.md b/sphinxdocs/sphinxdocs/tests/sphinx_docs/doc2.md similarity index 100% rename from sphinxdocs/tests/sphinx_docs/doc2.md rename to sphinxdocs/sphinxdocs/tests/sphinx_docs/doc2.md diff --git a/sphinxdocs/tests/sphinx_docs/index.md b/sphinxdocs/sphinxdocs/tests/sphinx_docs/index.md similarity index 100% rename from sphinxdocs/tests/sphinx_docs/index.md rename to sphinxdocs/sphinxdocs/tests/sphinx_docs/index.md diff --git a/sphinxdocs/tests/sphinx_stardoc/BUILD.bazel b/sphinxdocs/sphinxdocs/tests/sphinx_stardoc/BUILD.bazel similarity index 98% rename from sphinxdocs/tests/sphinx_stardoc/BUILD.bazel rename to sphinxdocs/sphinxdocs/tests/sphinx_stardoc/BUILD.bazel index af9af30886..a5d402b809 100644 --- a/sphinxdocs/tests/sphinx_stardoc/BUILD.bazel +++ b/sphinxdocs/sphinxdocs/tests/sphinx_stardoc/BUILD.bazel @@ -1,6 +1,6 @@ load("@bazel_skylib//:bzl_library.bzl", "bzl_library") load("@bazel_skylib//rules:build_test.bzl", "build_test") -load("//python:py_test.bzl", "py_test") +load("@rules_python//python:py_test.bzl", "py_test") load("//sphinxdocs:sphinx.bzl", "sphinx_build_binary", "sphinx_docs") load("//sphinxdocs:sphinx_stardoc.bzl", "sphinx_stardoc", "sphinx_stardocs") diff --git a/sphinxdocs/tests/sphinx_stardoc/aspect.md b/sphinxdocs/sphinxdocs/tests/sphinx_stardoc/aspect.md similarity index 100% rename from sphinxdocs/tests/sphinx_stardoc/aspect.md rename to sphinxdocs/sphinxdocs/tests/sphinx_stardoc/aspect.md diff --git a/sphinxdocs/tests/sphinx_stardoc/bzl_function.bzl b/sphinxdocs/sphinxdocs/tests/sphinx_stardoc/bzl_function.bzl similarity index 100% rename from sphinxdocs/tests/sphinx_stardoc/bzl_function.bzl rename to sphinxdocs/sphinxdocs/tests/sphinx_stardoc/bzl_function.bzl diff --git a/sphinxdocs/tests/sphinx_stardoc/bzl_providers.bzl b/sphinxdocs/sphinxdocs/tests/sphinx_stardoc/bzl_providers.bzl similarity index 100% rename from sphinxdocs/tests/sphinx_stardoc/bzl_providers.bzl rename to sphinxdocs/sphinxdocs/tests/sphinx_stardoc/bzl_providers.bzl diff --git a/sphinxdocs/tests/sphinx_stardoc/bzl_rule.bzl b/sphinxdocs/sphinxdocs/tests/sphinx_stardoc/bzl_rule.bzl similarity index 100% rename from sphinxdocs/tests/sphinx_stardoc/bzl_rule.bzl rename to sphinxdocs/sphinxdocs/tests/sphinx_stardoc/bzl_rule.bzl diff --git a/sphinxdocs/tests/sphinx_stardoc/bzl_typedef.bzl b/sphinxdocs/sphinxdocs/tests/sphinx_stardoc/bzl_typedef.bzl similarity index 100% rename from sphinxdocs/tests/sphinx_stardoc/bzl_typedef.bzl rename to sphinxdocs/sphinxdocs/tests/sphinx_stardoc/bzl_typedef.bzl diff --git a/sphinxdocs/tests/sphinx_stardoc/conf.py b/sphinxdocs/sphinxdocs/tests/sphinx_stardoc/conf.py similarity index 100% rename from sphinxdocs/tests/sphinx_stardoc/conf.py rename to sphinxdocs/sphinxdocs/tests/sphinx_stardoc/conf.py diff --git a/sphinxdocs/tests/sphinx_stardoc/envvars.md b/sphinxdocs/sphinxdocs/tests/sphinx_stardoc/envvars.md similarity index 100% rename from sphinxdocs/tests/sphinx_stardoc/envvars.md rename to sphinxdocs/sphinxdocs/tests/sphinx_stardoc/envvars.md diff --git a/sphinxdocs/tests/sphinx_stardoc/function.md b/sphinxdocs/sphinxdocs/tests/sphinx_stardoc/function.md similarity index 100% rename from sphinxdocs/tests/sphinx_stardoc/function.md rename to sphinxdocs/sphinxdocs/tests/sphinx_stardoc/function.md diff --git a/sphinxdocs/tests/sphinx_stardoc/glossary.md b/sphinxdocs/sphinxdocs/tests/sphinx_stardoc/glossary.md similarity index 100% rename from sphinxdocs/tests/sphinx_stardoc/glossary.md rename to sphinxdocs/sphinxdocs/tests/sphinx_stardoc/glossary.md diff --git a/sphinxdocs/tests/sphinx_stardoc/index.md b/sphinxdocs/sphinxdocs/tests/sphinx_stardoc/index.md similarity index 76% rename from sphinxdocs/tests/sphinx_stardoc/index.md rename to sphinxdocs/sphinxdocs/tests/sphinx_stardoc/index.md index 43ef14f55a..51a7a671ec 100644 --- a/sphinxdocs/tests/sphinx_stardoc/index.md +++ b/sphinxdocs/sphinxdocs/tests/sphinx_stardoc/index.md @@ -5,7 +5,7 @@ This is a set of documents to test the sphinx_stardoc extension. To build and view these docs, run: ``` -bazel run //sphinxdocs/tests/sphinx_stardoc:docs.serve +bazel run //tests/sphinx_stardoc:docs.serve ``` This will build the docs and start an HTTP server where they can be viewed. @@ -14,7 +14,7 @@ To aid the edit/debug cycle, `ibazel` can be used to automatically rebuild the HTML: ``` -ibazel build //sphinxdocs/tests/sphinx_stardoc:docs +ibazel build //tests/sphinx_stardoc:docs ``` :::{toctree} diff --git a/sphinxdocs/tests/sphinx_stardoc/module_extension.md b/sphinxdocs/sphinxdocs/tests/sphinx_stardoc/module_extension.md similarity index 100% rename from sphinxdocs/tests/sphinx_stardoc/module_extension.md rename to sphinxdocs/sphinxdocs/tests/sphinx_stardoc/module_extension.md diff --git a/sphinxdocs/tests/sphinx_stardoc/provider.md b/sphinxdocs/sphinxdocs/tests/sphinx_stardoc/provider.md similarity index 100% rename from sphinxdocs/tests/sphinx_stardoc/provider.md rename to sphinxdocs/sphinxdocs/tests/sphinx_stardoc/provider.md diff --git a/sphinxdocs/tests/sphinx_stardoc/repo_rule.md b/sphinxdocs/sphinxdocs/tests/sphinx_stardoc/repo_rule.md similarity index 100% rename from sphinxdocs/tests/sphinx_stardoc/repo_rule.md rename to sphinxdocs/sphinxdocs/tests/sphinx_stardoc/repo_rule.md diff --git a/sphinxdocs/tests/sphinx_stardoc/rule.md b/sphinxdocs/sphinxdocs/tests/sphinx_stardoc/rule.md similarity index 100% rename from sphinxdocs/tests/sphinx_stardoc/rule.md rename to sphinxdocs/sphinxdocs/tests/sphinx_stardoc/rule.md diff --git a/sphinxdocs/tests/sphinx_stardoc/sphinx_output_test.py b/sphinxdocs/sphinxdocs/tests/sphinx_stardoc/sphinx_output_test.py similarity index 100% rename from sphinxdocs/tests/sphinx_stardoc/sphinx_output_test.py rename to sphinxdocs/sphinxdocs/tests/sphinx_stardoc/sphinx_output_test.py diff --git a/sphinxdocs/tests/sphinx_stardoc/target.md b/sphinxdocs/sphinxdocs/tests/sphinx_stardoc/target.md similarity index 100% rename from sphinxdocs/tests/sphinx_stardoc/target.md rename to sphinxdocs/sphinxdocs/tests/sphinx_stardoc/target.md diff --git a/sphinxdocs/tests/sphinx_stardoc/typedef.md b/sphinxdocs/sphinxdocs/tests/sphinx_stardoc/typedef.md similarity index 100% rename from sphinxdocs/tests/sphinx_stardoc/typedef.md rename to sphinxdocs/sphinxdocs/tests/sphinx_stardoc/typedef.md diff --git a/sphinxdocs/tests/sphinx_stardoc/xrefs.md b/sphinxdocs/sphinxdocs/tests/sphinx_stardoc/xrefs.md similarity index 92% rename from sphinxdocs/tests/sphinx_stardoc/xrefs.md rename to sphinxdocs/sphinxdocs/tests/sphinx_stardoc/xrefs.md index bbd415ce19..9893c32023 100644 --- a/sphinxdocs/tests/sphinx_stardoc/xrefs.md +++ b/sphinxdocs/sphinxdocs/tests/sphinx_stardoc/xrefs.md @@ -36,7 +36,7 @@ Various tests of cross referencing support ## Using origin keys -* provider using `{type}`: {type}`"@rules_python//sphinxdocs/tests/sphinx_stardoc:bzl_rule.bzl%GenericInfo"` +* provider using `{type}`: {type}`"//tests/sphinx_stardoc:bzl_rule.bzl%GenericInfo"` ## Any xref From 73cad427f1ca9270c2a2f407a9b1194bf966437a Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Sat, 14 Mar 2026 20:51:26 -0700 Subject: [PATCH 666/922] chore: use term runfiles root instead of module space (#3664) The term "module space" isn't very clear. The directory it refers to is the runfiles root directory. So refer to it as that instead of "module space" --- python/private/python_bootstrap_template.txt | 76 +++++++++---------- python/private/stage1_bootstrap_template.sh | 10 +-- .../bootstrap_impls/bin_calls_bin/BUILD.bazel | 4 +- tests/bootstrap_impls/bin_calls_bin/inner.py | 4 +- tests/bootstrap_impls/bin_calls_bin/outer.py | 4 +- tests/bootstrap_impls/bin_calls_bin/verify.sh | 20 ++--- 6 files changed, 59 insertions(+), 59 deletions(-) diff --git a/python/private/python_bootstrap_template.txt b/python/private/python_bootstrap_template.txt index d40e815038..b01ebae3d4 100644 --- a/python/private/python_bootstrap_template.txt +++ b/python/private/python_bootstrap_template.txt @@ -136,12 +136,12 @@ def SearchPath(name): return path return None -def FindPythonBinary(module_space): +def FindPythonBinary(runfiles_root): """Finds the real Python binary if it's not a normal absolute path.""" if PYTHON_BINARY: - return FindBinary(module_space, PYTHON_BINARY) + return FindBinary(runfiles_root, PYTHON_BINARY) else: - return FindBinary(module_space, PYTHON_BINARY_ACTUAL) + return FindBinary(runfiles_root, PYTHON_BINARY_ACTUAL) def print_verbose(*args, mapping=None, values=None): @@ -165,7 +165,7 @@ def print_verbose(*args, mapping=None, values=None): else: print("bootstrap: stage 1:", *args, file=sys.stderr, flush=True) -def FindBinary(module_space, bin_name): +def FindBinary(runfiles_root, bin_name): """Finds the real binary if it's not a normal absolute path.""" if not bin_name: return None @@ -180,12 +180,12 @@ def FindBinary(module_space, bin_name): # Use normpath() to convert slashes to os.sep on Windows. elif os.sep in os.path.normpath(bin_name): # Case 3: Path is relative to the repo root. - return os.path.join(module_space, bin_name) + return os.path.join(runfiles_root, bin_name) else: # Case 4: Path has to be looked up in the search path. return SearchPath(bin_name) -def FindModuleSpace(main_rel_path): +def find_runfiles_root(main_rel_path): """Finds the runfiles tree.""" # When the calling process used the runfiles manifest to resolve the # location of this stub script, the path may be expanded. This means @@ -214,9 +214,9 @@ def FindModuleSpace(main_rel_path): stub_filename = os.path.join(os.getcwd(), stub_filename) while True: - module_space = stub_filename + ('.exe' if IsWindows() else '') + '.runfiles' - if os.path.isdir(module_space): - return module_space + runfiles_root = stub_filename + ('.exe' if IsWindows() else '') + '.runfiles' + if os.path.isdir(runfiles_root): + return runfiles_root runfiles_pattern = r'(.*\.runfiles)' + (r'\\' if IsWindows() else '/') + '.*' matchobj = re.match(runfiles_pattern, stub_filename) @@ -261,14 +261,14 @@ def ExtractZip(zip_path, dest_dir): os.chmod(file_path, attrs & 0o7777) # Create the runfiles tree by extracting the zip file -def CreateModuleSpace(): +def create_runfiles_root(): temp_dir = tempfile.mkdtemp('', 'Bazel.runfiles_') ExtractZip(os.path.dirname(__file__), temp_dir) - # IMPORTANT: Later code does `rm -fr` on dirname(module_space) -- it's + # IMPORTANT: Later code does `rm -fr` on dirname(runfiles_root) -- it's # important that deletion code be in sync with this directory structure return os.path.join(temp_dir, 'runfiles') -def RunfilesEnvvar(module_space): +def RunfilesEnvvar(runfiles_root): """Finds the runfiles manifest or the runfiles directory. Returns: @@ -288,10 +288,10 @@ def RunfilesEnvvar(module_space): # If running from a zip, there's no manifest file. if IsRunningFromZip(): - return ('RUNFILES_DIR', module_space) + return ('RUNFILES_DIR', runfiles_root) # Look for the runfiles "output" manifest, argv[0] + ".runfiles_manifest" - runfiles = module_space + '_manifest' + runfiles = runfiles_root + '_manifest' if os.path.exists(runfiles): return ('RUNFILES_MANIFEST_FILE', runfiles) @@ -299,19 +299,19 @@ def RunfilesEnvvar(module_space): # Normally .runfiles_manifest and MANIFEST are both present, but the # former will be missing for zip-based builds or if someone copies the # runfiles tree elsewhere. - runfiles = os.path.join(module_space, 'MANIFEST') + runfiles = os.path.join(runfiles_root, 'MANIFEST') if os.path.exists(runfiles): return ('RUNFILES_MANIFEST_FILE', runfiles) # If running in a sandbox and no environment variables are set, then # Look for the runfiles next to the binary. - if module_space.endswith('.runfiles') and os.path.isdir(module_space): - return ('RUNFILES_DIR', module_space) + if runfiles_root.endswith('.runfiles') and os.path.isdir(runfiles_root): + return ('RUNFILES_DIR', runfiles_root) return (None, None) -def ExecuteFile(python_program, main_filename, args, env, module_space, - workspace, delete_module_space): +def ExecuteFile(python_program, main_filename, args, env, runfiles_root, + workspace, delete_runfiles_root): # type: (str, str, list[str], dict[str, str], str, str|None, str|None) -> ... """Executes the given Python file using the various environment settings. @@ -323,10 +323,10 @@ def ExecuteFile(python_program, main_filename, args, env, module_space, main_filename: (str) The Python file to execute args: (list[str]) Additional args to pass to the Python file env: (dict[str, str]) A dict of environment variables to set for the execution - module_space: (str) Path to the module space/runfiles tree directory + runfiles_root: (str) Path to the runfiles root directory workspace: (str|None) Name of the workspace to execute in. This is expected to be a directory under the runfiles tree. - delete_module_space: (bool), True if the module space should be deleted + delete_runfiles_root: (bool), True if the runfiles root should be deleted after a successful (exit code zero) program run, False if not. """ argv = [python_program] @@ -351,7 +351,7 @@ def ExecuteFile(python_program, main_filename, args, env, module_space, # can't execv because we need control to return here. This only # happens for targets built in the host config. # - if not (IsWindows() or workspace or delete_module_space): + if not (IsWindows() or workspace or delete_runfiles_root): _RunExecv(python_program, argv, env) ret_code = subprocess.call( @@ -360,11 +360,11 @@ def ExecuteFile(python_program, main_filename, args, env, module_space, cwd=workspace ) - if delete_module_space: - # NOTE: dirname() is called because CreateModuleSpace() creates a + if delete_runfiles_root: + # NOTE: dirname() is called because create_runfiles_root() creates a # sub-directory within a temporary directory, and we want to remove the # whole temporary directory. - shutil.rmtree(os.path.dirname(module_space), True) + shutil.rmtree(os.path.dirname(runfiles_root), True) sys.exit(ret_code) def _RunExecv(python_program, argv, env): @@ -401,18 +401,18 @@ def Main(): print_verbose("main_rel_path:", main_rel_path) if IsRunningFromZip(): - module_space = CreateModuleSpace() - delete_module_space = True + runfiles_root = create_runfiles_root() + delete_runfiles_root = True else: - module_space = FindModuleSpace(main_rel_path) - delete_module_space = False + runfiles_root = find_runfiles_root(main_rel_path) + delete_runfiles_root = False - print_verbose("runfiles root:", module_space) + print_verbose("runfiles root:", runfiles_root) - if os.environ.get("RULES_PYTHON_TESTING_TELL_MODULE_SPACE"): - new_env["RULES_PYTHON_TESTING_MODULE_SPACE"] = module_space + if os.environ.get("RULES_PYTHON_TESTING_TELL_RUNFILES_ROOT"): + new_env["RULES_PYTHON_TESTING_RUNFILES_ROOT"] = runfiles_root - runfiles_envkey, runfiles_envvalue = RunfilesEnvvar(module_space) + runfiles_envkey, runfiles_envvalue = RunfilesEnvvar(runfiles_root) if runfiles_envkey: new_env[runfiles_envkey] = runfiles_envvalue @@ -420,14 +420,14 @@ def Main(): # See: https://docs.python.org/3.11/using/cmdline.html#envvar-PYTHONSAFEPATH new_env['PYTHONSAFEPATH'] = '1' - main_filename = os.path.join(module_space, main_rel_path) + main_filename = os.path.join(runfiles_root, main_rel_path) main_filename = GetWindowsPathWithUNCPrefix(main_filename) assert os.path.exists(main_filename), \ 'Cannot exec() %r: file not found.' % main_filename assert os.access(main_filename, os.R_OK), \ 'Cannot exec() %r: file not readable.' % main_filename - program = python_program = FindPythonBinary(module_space) + program = python_program = FindPythonBinary(runfiles_root) if python_program is None: raise AssertionError("Could not find python binary: {} or {}".format( repr(PYTHON_BINARY), @@ -449,15 +449,15 @@ def Main(): # change directory to the right runfiles directory. # (So that the data files are accessible) if os.environ.get('RUN_UNDER_RUNFILES') == '1': - workspace = os.path.join(module_space, WORKSPACE_NAME) + workspace = os.path.join(runfiles_root, WORKSPACE_NAME) try: sys.stdout.flush() # NOTE: ExecuteFile may call execve() and lines after this will never run. ExecuteFile( - python_program, main_filename, args, new_env, module_space, + python_program, main_filename, args, new_env, runfiles_root, workspace, - delete_module_space = delete_module_space, + delete_runfiles_root = delete_runfiles_root, ) except EnvironmentError: diff --git a/python/private/stage1_bootstrap_template.sh b/python/private/stage1_bootstrap_template.sh index 36b8b67c37..2fa70e9910 100644 --- a/python/private/stage1_bootstrap_template.sh +++ b/python/private/stage1_bootstrap_template.sh @@ -85,9 +85,9 @@ else stub_filename="$PWD/$stub_filename" fi while true; do - module_space="${stub_filename}.runfiles" - if [[ -d "$module_space" ]]; then - echo "$module_space" + runfiles_root="${stub_filename}.runfiles" + if [[ -d "$runfiles_root" ]]; then + echo "$runfiles_root" return 0 fi if [[ "$stub_filename" == *.runfiles/* ]]; then @@ -105,8 +105,8 @@ else RUNFILES_DIR=$(find_runfiles_root $0) fi -if [[ -n "$RULES_PYTHON_TESTING_TELL_MODULE_SPACE" ]]; then - export RULES_PYTHON_TESTING_MODULE_SPACE="$RUNFILES_DIR" +if [[ -n "$RULES_PYTHON_TESTING_TELL_RUNFILES_ROOT" ]]; then + export RULES_PYTHON_TESTING_RUNFILES_ROOT="$RUNFILES_DIR" fi function find_python_interpreter() { diff --git a/tests/bootstrap_impls/bin_calls_bin/BUILD.bazel b/tests/bootstrap_impls/bin_calls_bin/BUILD.bazel index 02835fb77b..1822ccd08e 100644 --- a/tests/bootstrap_impls/bin_calls_bin/BUILD.bazel +++ b/tests/bootstrap_impls/bin_calls_bin/BUILD.bazel @@ -24,7 +24,7 @@ py_reconfig_binary( genrule( name = "outer_calls_inner_system_python", outs = ["outer_calls_inner_system_python.out"], - cmd = "RULES_PYTHON_TESTING_TELL_MODULE_SPACE=1 $(location :outer_bootstrap_system_python) $(location :inner_bootstrap_system_python) > $@", + cmd = "RULES_PYTHON_TESTING_TELL_RUNFILES_ROOT=1 $(location :outer_bootstrap_system_python) $(location :inner_bootstrap_system_python) > $@", tags = ["manual"], tools = [ ":inner_bootstrap_system_python", @@ -67,7 +67,7 @@ py_reconfig_binary( genrule( name = "outer_calls_inner_script_python", outs = ["outer_calls_inner_script_python.out"], - cmd = "RULES_PYTHON_TESTING_TELL_MODULE_SPACE=1 $(location :outer_bootstrap_script) $(location :inner_bootstrap_script) > $@", + cmd = "RULES_PYTHON_TESTING_TELL_RUNFILES_ROOT=1 $(location :outer_bootstrap_script) $(location :inner_bootstrap_script) > $@", tags = ["manual"], tools = [ ":inner_bootstrap_script", diff --git a/tests/bootstrap_impls/bin_calls_bin/inner.py b/tests/bootstrap_impls/bin_calls_bin/inner.py index e67b31dda3..96409eb8f8 100644 --- a/tests/bootstrap_impls/bin_calls_bin/inner.py +++ b/tests/bootstrap_impls/bin_calls_bin/inner.py @@ -1,4 +1,4 @@ import os -module_space = os.environ.get("RULES_PYTHON_TESTING_MODULE_SPACE") -print(f"inner: RULES_PYTHON_TESTING_MODULE_SPACE='{module_space}'") +runfiles_root = os.environ.get("RULES_PYTHON_TESTING_RUNFILES_ROOT") +print(f"inner: RULES_PYTHON_TESTING_RUNFILES_ROOT='{runfiles_root}'") diff --git a/tests/bootstrap_impls/bin_calls_bin/outer.py b/tests/bootstrap_impls/bin_calls_bin/outer.py index 19dac06eb7..f995c24ff6 100644 --- a/tests/bootstrap_impls/bin_calls_bin/outer.py +++ b/tests/bootstrap_impls/bin_calls_bin/outer.py @@ -3,8 +3,8 @@ import sys if __name__ == "__main__": - module_space = os.environ.get("RULES_PYTHON_TESTING_MODULE_SPACE") - print(f"outer: RULES_PYTHON_TESTING_MODULE_SPACE='{module_space}'") + runfiles_root = os.environ.get("RULES_PYTHON_TESTING_RUNFILES_ROOT") + print(f"outer: RULES_PYTHON_TESTING_RUNFILES_ROOT='{runfiles_root}'") inner_binary_path = sys.argv[1] result = subprocess.run( diff --git a/tests/bootstrap_impls/bin_calls_bin/verify.sh b/tests/bootstrap_impls/bin_calls_bin/verify.sh index 433704e9ab..9a9a17c6c6 100755 --- a/tests/bootstrap_impls/bin_calls_bin/verify.sh +++ b/tests/bootstrap_impls/bin_calls_bin/verify.sh @@ -4,23 +4,23 @@ set -euo pipefail verify_output() { local OUTPUT_FILE=$1 - # Extract the RULES_PYTHON_TESTING_MODULE_SPACE values - local OUTER_MODULE_SPACE=$(grep "outer: RULES_PYTHON_TESTING_MODULE_SPACE" "$OUTPUT_FILE" | sed "s/outer: RULES_PYTHON_TESTING_MODULE_SPACE='\(.*\)'/\1/") - local INNER_MODULE_SPACE=$(grep "inner: RULES_PYTHON_TESTING_MODULE_SPACE" "$OUTPUT_FILE" | sed "s/inner: RULES_PYTHON_TESTING_MODULE_SPACE='\(.*\)'/\1/") + # Extract the RULES_PYTHON_TESTING_RUNFILES_ROOT values + local OUTER_RUNFILES_ROOT=$(grep "outer: RULES_PYTHON_TESTING_RUNFILES_ROOT" "$OUTPUT_FILE" | sed "s/outer: RULES_PYTHON_TESTING_RUNFILES_ROOT='\(.*\)'/\1/") + local INNER_RUNFILES_ROOT=$(grep "inner: RULES_PYTHON_TESTING_RUNFILES_ROOT" "$OUTPUT_FILE" | sed "s/inner: RULES_PYTHON_TESTING_RUNFILES_ROOT='\(.*\)'/\1/") - echo "Outer module space: $OUTER_MODULE_SPACE" - echo "Inner module space: $INNER_MODULE_SPACE" + echo "Outer runfiles root: $OUTER_RUNFILES_ROOT" + echo "Inner runfiles root: $INNER_RUNFILES_ROOT" # Check 1: The two values are different - if [ "$OUTER_MODULE_SPACE" == "$INNER_MODULE_SPACE" ]; then - echo "Error: Outer and Inner module spaces are the same." + if [ "$OUTER_RUNFILES_ROOT" == "$INNER_RUNFILES_ROOT" ]; then + echo "Error: Outer and Inner runfiles roots are the same." exit 1 fi # Check 2: Inner is not a subdirectory of Outer - case "$INNER_MODULE_SPACE" in - "$OUTER_MODULE_SPACE"/*) - echo "Error: Inner module space is a subdirectory of Outer's." + case "$INNER_RUNFILES_ROOT" in + "$OUTER_RUNFILES_ROOT"/*) + echo "Error: Inner runfiles root is a subdirectory of Outer's." exit 1 ;; *) From 2d7ff9da96877fd83f03d01bed3cfb9aa9de7bd2 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Sat, 14 Mar 2026 20:55:58 -0700 Subject: [PATCH 667/922] chore: clarify type of paths for some system_python variables (#3665) Clarify that some variables are runfiles-root relative paths so that their values aren't confused to be "main repo" runfiles relative paths like "../foo" --- python/private/python_bootstrap_template.txt | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/python/private/python_bootstrap_template.txt b/python/private/python_bootstrap_template.txt index b01ebae3d4..76036bb40a 100644 --- a/python/private/python_bootstrap_template.txt +++ b/python/private/python_bootstrap_template.txt @@ -10,13 +10,13 @@ import sys import os import subprocess import uuid -# runfiles-relative path # NOTE: The sentinel strings are split (e.g., "%stage2" + "_bootstrap%") so that # the substitution logic won't replace them. This allows runtime detection of # unsubstituted placeholders, which occurs when native py_binary is used in # external repositories. In that case, we fall back to %main% which Bazel's # native rule does substitute. _STAGE2_BOOTSTRAP_SENTINEL = "%stage2" + "_bootstrap%" +# runfiles-root-relative path STAGE2_BOOTSTRAP="%stage2_bootstrap%" # NOTE: The fallback logic from stage2_bootstrap to main is only present @@ -35,13 +35,13 @@ if not STAGE2_BOOTSTRAP: print("ERROR: %stage2_bootstrap% (or %main%) was not substituted.", file=sys.stderr) sys.exit(1) -# runfiles-relative path to venv's python interpreter +# runfiles-root-relative path to venv's python interpreter # Empty string if a venv is not setup. PYTHON_BINARY = '%python_binary%' # The path to the actual interpreter that is used. # Typically PYTHON_BINARY is a symlink pointing to this. -# runfiles-relative path, absolute path, or single word. +# runfiles-root-relative path, absolute path, or single word. # Used to create a venv at runtime, or when a venv isn't setup. PYTHON_BINARY_ACTUAL = "%python_binary_actual%" From 77a89a6600432f94617f851511987104014d6276 Mon Sep 17 00:00:00 2001 From: Alex Trotta <44127594+Ahajha@users.noreply.github.com> Date: Sun, 15 Mar 2026 00:11:51 -0400 Subject: [PATCH 668/922] feat: Allow files in wheels to be installed to directories (#3233) When specifying `data_files` in `py_wheel`, allow just the directory to be specified (with a trailing slash), in which case it will use the existing filename. This avoids duplicating (potentially platform-specific) names. Additionally, targets with multiple files can be installed as a group to a folder, with the same filename-preserving behavior. In general I think this is a better starting point, as I imagine most of the time users would want to preserve the names. Before, this would result in the file simply not being installed, so this only changes already-broken behavior. --------- Co-authored-by: Richard Levasseur --- CHANGELOG.md | 2 ++ examples/wheel/BUILD.bazel | 24 ++++++++++++++++++ examples/wheel/wheel_test.py | 19 ++++++++++++++ python/private/py_wheel.bzl | 49 +++++++++++++++++++++++++++++------- 4 files changed, 85 insertions(+), 9 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index d908992c9c..e5748704b6 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -87,6 +87,8 @@ Other changes: {obj}`experimental_index_url` which should speed up consecutive initializations and should no longer require the network access if the cache is hydrated. Implements [#2731](https://github.com/bazel-contrib/rules_python/issues/2731). +* (wheel) Specifying a path ending in `/` as a destination in `data_files` + will now install file(s) to a folder, preserving their basename. {#v1-9-0} ## [1.9.0] - 2026-02-21 diff --git a/examples/wheel/BUILD.bazel b/examples/wheel/BUILD.bazel index e52e0fc3a3..3cf6e9f350 100644 --- a/examples/wheel/BUILD.bazel +++ b/examples/wheel/BUILD.bazel @@ -401,6 +401,29 @@ py_wheel( version = "0.0.1", ) +filegroup( + name = "data_files_test_group", + # Re-using some files already checked into the repo. + srcs = [ + "README.md", + "//examples/wheel:NOTICE", + ], +) + +py_wheel( + name = "data_files_installed_in_folder", + testonly = True, # Set this to verify the generated .dist target doesn't break things + # Re-using some files already checked into the repo. + data_files = { + # Single file + "//examples/wheel:NOTICE": "scripts/", + # Filegroup + ":data_files_test_group": "data/", + }, + distribution = "data_files_installed_in_folder", + version = "0.0.1", +) + py_test( name = "wheel_test", srcs = ["wheel_test.py"], @@ -409,6 +432,7 @@ py_test( ":custom_package_root_multi_prefix", ":custom_package_root_multi_prefix_reverse_order", ":customized", + ":data_files_installed_in_folder", ":empty_requires_files", ":extra_requires", ":filename_escaping", diff --git a/examples/wheel/wheel_test.py b/examples/wheel/wheel_test.py index 7f19ecd9f9..9ed2b842e5 100644 --- a/examples/wheel/wheel_test.py +++ b/examples/wheel/wheel_test.py @@ -615,6 +615,25 @@ def test_requires_dist_depends_on_extras_file(self): requires, ) + def test_data_files_installed_in_folder(self): + filename = self._get_path( + "data_files_installed_in_folder-0.0.1-py3-none-any.whl" + ) + + with zipfile.ZipFile(filename) as zf: + self.assertAllEntriesHasReproducibleMetadata(zf) + self.assertEqual( + zf.namelist(), + [ + "data_files_installed_in_folder-0.0.1.dist-info/WHEEL", + "data_files_installed_in_folder-0.0.1.dist-info/METADATA", + "data_files_installed_in_folder-0.0.1.data/data/NOTICE", + "data_files_installed_in_folder-0.0.1.data/data/README.md", + "data_files_installed_in_folder-0.0.1.data/scripts/NOTICE", + "data_files_installed_in_folder-0.0.1.dist-info/RECORD", + ], + ) + if __name__ == "__main__": unittest.main() diff --git a/python/private/py_wheel.bzl b/python/private/py_wheel.bzl index 1d98d21a65..e6a9925a15 100644 --- a/python/private/py_wheel.bzl +++ b/python/private/py_wheel.bzl @@ -182,8 +182,35 @@ _other_attrs = { doc = "A list of strings describing the categories for the package. For valid classifiers see https://pypi.org/classifiers", ), "data_files": attr.label_keyed_string_dict( - doc = ("Any file that is not normally installed inside site-packages goes into the .data directory, named " + - "as the .dist-info directory but with the .data/ extension. Allowed paths: {prefixes}".format(prefixes = ALLOWED_DATA_FILE_PREFIX)), + doc = (""" +Mapping of data files to go into the wheel. + +The keys are targets of files to include, and the values are the `.data`-relative +path to use. + +Any file that is not normally installed inside site-packages goes into the .data +directory, named as the .dist-info directory but with the .data/ extension. If +the destination of a file or group of files ends in a `/`, the destination is a +folder and files are placed with their existing basenames under that folder. + +For example: + +``` +":file1.txt": "data/file1.txt", # Destination: .data/data/file1.txt +":file1.txt": "data/", # Destination: .data/data/file1.txt +":file1.txt": "data/special.txt", # Destination: .data/data/special.txt + +filegroup(name = "files", srcs = [":file1.txt", ":file2.txt"]) +":files": "data/", # Destinations: .data/data/file1.txt, .data/data/file2.txt +``` + +Allowed paths: {prefixes} + +:::{{versionchanged}} VERSION_NEXT_FEATURE +Values can end in slash (`/`) to indicate that all files of the target should +be moved under that directory. +::: +""".format(prefixes = ALLOWED_DATA_FILE_PREFIX)), allow_files = True, ), "description_content_type": attr.string( @@ -506,9 +533,9 @@ def _py_wheel_impl(ctx): for target, filename in ctx.attr.data_files.items(): target_files = target[DefaultInfo].files.to_list() - if len(target_files) != 1: + if len(target_files) != 1 and not filename.endswith("/"): fail( - "Multi-file target listed in data_files %s", + "Multi-file target listed in data_files %s, this is only supported when specifying a folder path (i.e. a path ending in '/')", filename, ) @@ -520,11 +547,15 @@ def _py_wheel_impl(ctx): filename, ), ) - other_inputs.extend(target_files) - args.add( - "--data_files", - filename + ";" + target_files[0].path, - ) + + for file in target_files: + final_filename = filename + file.basename if filename.endswith("/") else filename + + other_inputs.extend(target_files) + args.add( + "--data_files", + final_filename + ";" + file.path, + ) ctx.actions.run( mnemonic = "PyWheel", From c9bb66e1f5aa9d8d951f61d4ad73891604133839 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Alex=20Fax=C3=A5?= Date: Mon, 16 Mar 2026 01:51:12 +0100 Subject: [PATCH 669/922] fix(bootstrap): manual runfiles path construction when using submodules (#3636) This fixes https://github.com/bazel-contrib/rules_python/issues/3563. Verified by running the repro in https://github.com/mering/reproduction_rules_python_1_7. The identified regression in https://github.com/bazel-contrib/rules_python/commit/b8e32c454a1158cd78ce4ecaef809b99bef4e5da is problematic because prepending `ctx.workspace_name` to `short_path` results in paths containing `..` (e.g., `_main/../sub+/path/to/file.py`) when building from a root module that includes other modules. This causes the `_find_runfiles_root` [logic](https://github.com/faximan/rules_python/blob/eb6ac472eb86cc263acc336a2b73982043069aae/python/private/site_init_template.py#L73), which _counts slashes_, to incorrectly calculate the runfiles root. The fix is simply using the available `runfiles_root_path` function instead. In the example above, this makes the path simply `sub+/path/to/file.py`. --------- Co-authored-by: Ignas Anikevicius <240938+aignas@users.noreply.github.com> --- CHANGELOG.md | 3 +++ .../other_module/other_module/pkg/BUILD.bazel | 10 +++++++++ .../bzlmod/tests/other_module/BUILD.bazel | 14 ++++++++++++ .../other_module/other_module_import_test.py | 22 +++++++++++++++++++ python/private/py_executable.bzl | 19 +++++----------- 5 files changed, 54 insertions(+), 14 deletions(-) create mode 100644 examples/bzlmod/tests/other_module/other_module_import_test.py diff --git a/CHANGELOG.md b/CHANGELOG.md index e5748704b6..031b06624f 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -80,6 +80,9 @@ Other changes: configuration. See the {any}`RULES_PYTHON_PYCACHE_DIR` environment variable for more information. ([#3643](https://github.com/bazel-contrib/rules_python/issues/3643)). +* (bootstrap) Fixed incorrect runfiles path construction in bootstrap + scripts when binary is defined in another bazel module + ([#3563](https://github.com/bazel-contrib/rules_python/issues/3563)). {#v0-0-0-added} ### Added diff --git a/examples/bzlmod/other_module/other_module/pkg/BUILD.bazel b/examples/bzlmod/other_module/other_module/pkg/BUILD.bazel index 53344c708a..318822d888 100644 --- a/examples/bzlmod/other_module/other_module/pkg/BUILD.bazel +++ b/examples/bzlmod/other_module/other_module/pkg/BUILD.bazel @@ -1,5 +1,6 @@ load("@rules_python//python:py_binary.bzl", "py_binary") load("@rules_python//python:py_library.bzl", "py_library") +load("@rules_python//python/zipapp:py_zipapp_binary.bzl", "py_zipapp_binary") py_library( name = "lib", @@ -26,4 +27,13 @@ py_binary( ], ) +# This is used for regression testing runfiles paths in submodules. +# https://github.com/bazel-contrib/rules_python/issues/3563. +py_zipapp_binary( + name = "bin_zipapp", + testonly = True, + binary = ":bin", + visibility = ["//visibility:public"], +) + exports_files(["data/data.txt"]) diff --git a/examples/bzlmod/tests/other_module/BUILD.bazel b/examples/bzlmod/tests/other_module/BUILD.bazel index 1bd8a900a9..24231e651a 100644 --- a/examples/bzlmod/tests/other_module/BUILD.bazel +++ b/examples/bzlmod/tests/other_module/BUILD.bazel @@ -5,6 +5,7 @@ # in the root module. load("@bazel_skylib//rules:build_test.bzl", "build_test") +load("@rules_python//python:py_test.bzl", "py_test") build_test( name = "other_module_bin_build_test", @@ -12,3 +13,16 @@ build_test( "@our_other_module//other_module/pkg:bin", ], ) + +py_test( + name = "other_module_import_test", + srcs = ["other_module_import_test.py"], + data = ["@our_other_module//other_module/pkg:bin_zipapp"], + env = {"ZIPAPP_PATH": "$(location @our_other_module//other_module/pkg:bin_zipapp)"}, + # For now, skip this test on Windows because it fails for reasons + # other than the code path being tested. + target_compatible_with = select({ + "@platforms//os:windows": ["@platforms//:incompatible"], + "//conditions:default": [], + }), +) diff --git a/examples/bzlmod/tests/other_module/other_module_import_test.py b/examples/bzlmod/tests/other_module/other_module_import_test.py new file mode 100644 index 0000000000..6b92a853e0 --- /dev/null +++ b/examples/bzlmod/tests/other_module/other_module_import_test.py @@ -0,0 +1,22 @@ +"""Regression test for https://github.com/bazel-contrib/rules_python/issues/3563""" +import os +import subprocess +import sys + +def main(): + # The rlocation path for the bin_zipapp. It is in the "our_other_module" repository. + zipapp_path = os.environ.get("ZIPAPP_PATH") + print(f"Running bin_zipapp at: {zipapp_path}") + + result = subprocess.run([zipapp_path], capture_output=True, text=True) + print("--- bin_zippapp stdout ---") + print(result.stdout) + print("--- bin_zippapp stderr ---") + print(result.stderr) + + if result.returncode != 0: + print(f"bin_zippapp failed with return code {result.returncode}") + sys.exit(result.returncode) + +if __name__ == "__main__": + main() diff --git a/python/private/py_executable.bzl b/python/private/py_executable.bzl index 284aea6bff..495c7dddcf 100644 --- a/python/private/py_executable.bzl +++ b/python/private/py_executable.bzl @@ -510,10 +510,7 @@ def _create_zip_main(ctx, *, stage2_bootstrap, runtime_details, venv): substitutions = { "%python_binary%": python_binary, "%python_binary_actual%": python_binary_actual, - "%stage2_bootstrap%": "{}/{}".format( - ctx.workspace_name, - stage2_bootstrap.short_path, - ), + "%stage2_bootstrap%": runfiles_root_path(ctx, stage2_bootstrap.short_path), "%workspace_name%": ctx.workspace_name, }, ) @@ -616,7 +613,7 @@ def _create_venv(ctx, output_prefix, imports, runtime_details, add_runfiles_root "%add_runfiles_root_to_sys_path%": add_runfiles_root_to_sys_path, "%coverage_tool%": _get_coverage_tool_runfiles_path(ctx, runtime), "%import_all%": "True" if read_possibly_native_flag(ctx, "python_import_all_repositories") else "False", - "%site_init_runfiles_path%": "{}/{}".format(ctx.workspace_name, site_init.short_path), + "%site_init_runfiles_path%": runfiles_root_path(ctx, site_init.short_path), "%workspace_name%": ctx.workspace_name, }, computed_substitutions = computed_subs, @@ -671,10 +668,7 @@ def _get_coverage_tool_runfiles_path(ctx, runtime): if (ctx.configuration.coverage_enabled and runtime and runtime.coverage_tool): - return "{}/{}".format( - ctx.workspace_name, - runtime.coverage_tool.short_path, - ) + return runfiles_root_path(ctx, runtime.coverage_tool.short_path) else: return "" @@ -777,10 +771,7 @@ def _create_stage1_bootstrap( if (ctx.configuration.coverage_enabled and runtime and runtime.coverage_tool): - coverage_tool_runfiles_path = "{}/{}".format( - ctx.workspace_name, - runtime.coverage_tool.short_path, - ) + coverage_tool_runfiles_path = runfiles_root_path(ctx, runtime.coverage_tool.short_path) else: coverage_tool_runfiles_path = "" if runtime: @@ -793,7 +784,7 @@ def _create_stage1_bootstrap( subs["%coverage_tool%"] = coverage_tool_runfiles_path subs["%import_all%"] = ("True" if read_possibly_native_flag(ctx, "python_import_all_repositories") else "False") subs["%imports%"] = ":".join(imports.to_list()) - subs["%main%"] = "{}/{}".format(ctx.workspace_name, main_py.short_path) + subs["%main%"] = runfiles_root_path(ctx, main_py.short_path) ctx.actions.expand_template( template = template, From eb1ae41c58a734363de39bcc6a0b9817673fd8ae Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Sun, 15 Mar 2026 19:43:50 -0700 Subject: [PATCH 670/922] chore(system_python): use snake_case, add some debugging (#3667) This uses snake_case function names in the system python bootstrap. This is just to modernize the code a bit. Along the way ... * Add some additional debug logging * Fix the `if is_windows:` conditional that was always executing (because it was referring to a function), but a no-op, on Linux. --- python/private/python_bootstrap_template.txt | 79 ++++++++++---------- 1 file changed, 39 insertions(+), 40 deletions(-) diff --git a/python/private/python_bootstrap_template.txt b/python/private/python_bootstrap_template.txt index 76036bb40a..f08de8e0f5 100644 --- a/python/private/python_bootstrap_template.txt +++ b/python/private/python_bootstrap_template.txt @@ -66,10 +66,10 @@ else: ADDITIONAL_INTERPRETER_ARGS = os.environ.get("RULES_PYTHON_ADDITIONAL_INTERPRETER_ARGS", "") -def IsRunningFromZip(): +def is_running_from_zip(): return IS_ZIPFILE -if IsRunningFromZip(): +if is_running_from_zip(): import shutil import tempfile import zipfile @@ -77,10 +77,10 @@ else: import re # Return True if running on Windows -def IsWindows(): +def is_windows(): return os.name == 'nt' -def GetWindowsPathWithUNCPrefix(path): +def get_windows_path_with_unc_prefix(path): """Adds UNC prefix after getting a normalized absolute Windows path. No-op for non-Windows platforms or if running under python2. @@ -89,7 +89,7 @@ def GetWindowsPathWithUNCPrefix(path): # No need to add prefix for non-Windows platforms. # And \\?\ doesn't work in python 2 or on mingw - if not IsWindows() or sys.version_info[0] < 3: + if not is_windows() or sys.version_info[0] < 3: return path # Starting in Windows 10, version 1607(OS build 14393), MAX_PATH limitations have been @@ -120,13 +120,13 @@ def GetWindowsPathWithUNCPrefix(path): # os.path.abspath returns a normalized absolute path return unicode_prefix + os.path.abspath(path) -def HasWindowsExecutableExtension(path): +def has_windows_executable_extension(path): return path.endswith('.exe') or path.endswith('.com') or path.endswith('.bat') -if PYTHON_BINARY and IsWindows() and not HasWindowsExecutableExtension(PYTHON_BINARY): +if PYTHON_BINARY and is_windows() and not has_windows_executable_extension(PYTHON_BINARY): PYTHON_BINARY = PYTHON_BINARY + '.exe' -def SearchPath(name): +def search_path(name): """Finds a file in a given search path.""" search_path = os.getenv('PATH', os.defpath).split(os.pathsep) for directory in search_path: @@ -136,12 +136,12 @@ def SearchPath(name): return path return None -def FindPythonBinary(runfiles_root): +def find_python_binary(runfiles_root): """Finds the real Python binary if it's not a normal absolute path.""" if PYTHON_BINARY: - return FindBinary(runfiles_root, PYTHON_BINARY) + return find_binary(runfiles_root, PYTHON_BINARY) else: - return FindBinary(runfiles_root, PYTHON_BINARY_ACTUAL) + return find_binary(runfiles_root, PYTHON_BINARY_ACTUAL) def print_verbose(*args, mapping=None, values=None): @@ -165,7 +165,7 @@ def print_verbose(*args, mapping=None, values=None): else: print("bootstrap: stage 1:", *args, file=sys.stderr, flush=True) -def FindBinary(runfiles_root, bin_name): +def find_binary(runfiles_root, bin_name): """Finds the real binary if it's not a normal absolute path.""" if not bin_name: return None @@ -183,7 +183,7 @@ def FindBinary(runfiles_root, bin_name): return os.path.join(runfiles_root, bin_name) else: # Case 4: Path has to be looked up in the search path. - return SearchPath(bin_name) + return search_path(bin_name) def find_runfiles_root(main_rel_path): """Finds the runfiles tree.""" @@ -207,18 +207,18 @@ def find_runfiles_root(main_rel_path): # On Windows, the path may contain both forward and backslashes. # Normalize to the OS separator because the regex used later assumes # the OS-specific separator. - if IsWindows: + if is_windows(): stub_filename = stub_filename.replace("/", os.sep) if not os.path.isabs(stub_filename): stub_filename = os.path.join(os.getcwd(), stub_filename) while True: - runfiles_root = stub_filename + ('.exe' if IsWindows() else '') + '.runfiles' + runfiles_root = stub_filename + ('.exe' if is_windows() else '') + '.runfiles' if os.path.isdir(runfiles_root): return runfiles_root - runfiles_pattern = r'(.*\.runfiles)' + (r'\\' if IsWindows() else '/') + '.*' + runfiles_pattern = r'(.*\.runfiles)' + (r'\\' if is_windows() else '/') + '.*' matchobj = re.match(runfiles_pattern, stub_filename) if matchobj: return matchobj.group(1) @@ -233,7 +233,7 @@ def find_runfiles_root(main_rel_path): raise AssertionError('Cannot find .runfiles directory for %s' % sys.argv[0]) -def ExtractZip(zip_path, dest_dir): +def extract_zip(zip_path, dest_dir): """Extracts the contents of a zip file, preserving the unix file mode bits. These include the permission bits, and in particular, the executable bit. @@ -245,8 +245,8 @@ def ExtractZip(zip_path, dest_dir): zip_path: The path to the zip file to extract dest_dir: The path to the destination directory """ - zip_path = GetWindowsPathWithUNCPrefix(zip_path) - dest_dir = GetWindowsPathWithUNCPrefix(dest_dir) + zip_path = get_windows_path_with_unc_prefix(zip_path) + dest_dir = get_windows_path_with_unc_prefix(dest_dir) with zipfile.ZipFile(zip_path) as zf: for info in zf.infolist(): zf.extract(info, dest_dir) @@ -263,12 +263,12 @@ def ExtractZip(zip_path, dest_dir): # Create the runfiles tree by extracting the zip file def create_runfiles_root(): temp_dir = tempfile.mkdtemp('', 'Bazel.runfiles_') - ExtractZip(os.path.dirname(__file__), temp_dir) + extract_zip(os.path.dirname(__file__), temp_dir) # IMPORTANT: Later code does `rm -fr` on dirname(runfiles_root) -- it's # important that deletion code be in sync with this directory structure return os.path.join(temp_dir, 'runfiles') -def RunfilesEnvvar(runfiles_root): +def runfiles_envvar(runfiles_root): """Finds the runfiles manifest or the runfiles directory. Returns: @@ -287,7 +287,7 @@ def RunfilesEnvvar(runfiles_root): return ('RUNFILES_DIR', runfiles) # If running from a zip, there's no manifest file. - if IsRunningFromZip(): + if is_running_from_zip(): return ('RUNFILES_DIR', runfiles_root) # Look for the runfiles "output" manifest, argv[0] + ".runfiles_manifest" @@ -310,7 +310,7 @@ def RunfilesEnvvar(runfiles_root): return (None, None) -def ExecuteFile(python_program, main_filename, args, env, runfiles_root, +def execute_file(python_program, main_filename, args, env, runfiles_root, workspace, delete_runfiles_root): # type: (str, str, list[str], dict[str, str], str, str|None, str|None) -> ... """Executes the given Python file using the various environment settings. @@ -351,8 +351,8 @@ def ExecuteFile(python_program, main_filename, args, env, runfiles_root, # can't execv because we need control to return here. This only # happens for targets built in the host config. # - if not (IsWindows() or workspace or delete_runfiles_root): - _RunExecv(python_program, argv, env) + if not (is_windows() or workspace or delete_runfiles_root): + _run_execv(python_program, argv, env) ret_code = subprocess.call( argv, @@ -367,7 +367,7 @@ def ExecuteFile(python_program, main_filename, args, env, runfiles_root, shutil.rmtree(os.path.dirname(runfiles_root), True) sys.exit(ret_code) -def _RunExecv(python_program, argv, env): +def _run_execv(python_program, argv, env): # type: (str, list[str], dict[str, str]) -> ... """Executes the given Python file using the various environment settings.""" os.environ.update(env) @@ -376,17 +376,16 @@ def _RunExecv(python_program, argv, env): print_verbose("RunExecv: argv:", values=argv) os.execv(python_program, argv) -def Main(): - print_verbose("initial argv:", values=sys.argv) - print_verbose("initial cwd:", os.getcwd()) - print_verbose("initial environ:", mapping=os.environ) - print_verbose("initial sys.path:", values=sys.path) +def main(): print_verbose("STAGE2_BOOTSTRAP:", STAGE2_BOOTSTRAP) print_verbose("PYTHON_BINARY:", PYTHON_BINARY) print_verbose("PYTHON_BINARY_ACTUAL:", PYTHON_BINARY_ACTUAL) print_verbose("IS_ZIPFILE:", IS_ZIPFILE) print_verbose("RECREATE_VENV_AT_RUNTIME:", RECREATE_VENV_AT_RUNTIME) print_verbose("WORKSPACE_NAME :", WORKSPACE_NAME ) + print_verbose("bootstrap sys.executable:", sys.executable) + print_verbose("bootstrap sys._base_executable:", sys._base_executable) + print_verbose("bootstrap sys.version:", sys.version) args = sys.argv[1:] @@ -400,7 +399,7 @@ def Main(): main_rel_path = os.path.normpath(STAGE2_BOOTSTRAP) print_verbose("main_rel_path:", main_rel_path) - if IsRunningFromZip(): + if is_running_from_zip(): runfiles_root = create_runfiles_root() delete_runfiles_root = True else: @@ -412,7 +411,7 @@ def Main(): if os.environ.get("RULES_PYTHON_TESTING_TELL_RUNFILES_ROOT"): new_env["RULES_PYTHON_TESTING_RUNFILES_ROOT"] = runfiles_root - runfiles_envkey, runfiles_envvalue = RunfilesEnvvar(runfiles_root) + runfiles_envkey, runfiles_envvalue = runfiles_envvar(runfiles_root) if runfiles_envkey: new_env[runfiles_envkey] = runfiles_envvalue @@ -421,13 +420,13 @@ def Main(): new_env['PYTHONSAFEPATH'] = '1' main_filename = os.path.join(runfiles_root, main_rel_path) - main_filename = GetWindowsPathWithUNCPrefix(main_filename) + main_filename = get_windows_path_with_unc_prefix(main_filename) assert os.path.exists(main_filename), \ 'Cannot exec() %r: file not found.' % main_filename assert os.access(main_filename, os.R_OK), \ 'Cannot exec() %r: file not readable.' % main_filename - program = python_program = FindPythonBinary(runfiles_root) + python_program = find_python_binary(runfiles_root) if python_program is None: raise AssertionError("Could not find python binary: {} or {}".format( repr(PYTHON_BINARY), @@ -444,7 +443,7 @@ def Main(): new_env.update((key, val) for key, val in os.environ.items() if key not in new_env) workspace = None - if IsRunningFromZip(): + if is_running_from_zip(): # If RUN_UNDER_RUNFILES equals 1, it means we need to # change directory to the right runfiles directory. # (So that the data files are accessible) @@ -453,8 +452,8 @@ def Main(): try: sys.stdout.flush() - # NOTE: ExecuteFile may call execve() and lines after this will never run. - ExecuteFile( + # NOTE: execute_file may call execve() and lines after this will never run. + execute_file( python_program, main_filename, args, new_env, runfiles_root, workspace, delete_runfiles_root = delete_runfiles_root, @@ -465,8 +464,8 @@ def Main(): e = sys.exc_info()[1] # This exception occurs when os.execv() fails for some reason. if not getattr(e, 'filename', None): - e.filename = program # Add info to error message + e.filename = python_program # Add info to error message raise if __name__ == '__main__': - Main() + main() From f5f35d6e9a2f01c754e59c6ff97bfdcdf81abc35 Mon Sep 17 00:00:00 2001 From: Ignas Anikevicius <240938+aignas@users.noreply.github.com> Date: Fri, 20 Mar 2026 10:05:42 +0900 Subject: [PATCH 671/922] fix(pypi): propagate fails if overrides are passed only one index is used (#3666) Before this PR there would be confusing failures when downloader config is set to disallow certain values or when the authentication is not setup properly. This is a small fix towards a better goal state where we set `allow_fail = False` in cases where we know that we have to succeed to download metadata from that particular URL. The use-cases covered: - Only one index_url is passed to `pip.parse`. - `index_url_overrides` are passed which means that we should fail if there are insufficient overrides. The downside to this is that it is really hard to return custom error messages telling the user what to do, but on the flip side, the failures coming from bazel itself might be more descriptive in the case of outh-misconfiguration or bazel downloader configuration settings. Work towards #2632 and #3260. --- CHANGELOG.md | 4 + python/private/pypi/simpleapi_download.bzl | 14 ++- .../simpleapi_download_tests.bzl | 95 +++++++++++++++---- 3 files changed, 90 insertions(+), 23 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 031b06624f..18be4def9c 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -67,6 +67,10 @@ END_UNRELEASED_TEMPLATE Other changes: * (pypi) Update dependencies used for `compile_pip_requirements`, building sdists in the `whl_library` rule and fetching wheels using `pip`. +* (pypi) We will set `allow_fail` to `False` if the {attr}`experimental_index_url_overrides` is set + to a non-empty value. This means that failures will be no-longer cached in this particular case. + ([#3260](https://github.com/bazel-contrib/rules_python/issues/3260) and + [#2632](https://github.com/bazel-contrib/rules_python/issues/2632)) {#v0-0-0-fixed} ### Fixed diff --git a/python/private/pypi/simpleapi_download.bzl b/python/private/pypi/simpleapi_download.bzl index ff18887ec1..20d79ba9b4 100644 --- a/python/private/pypi/simpleapi_download.bzl +++ b/python/private/pypi/simpleapi_download.bzl @@ -71,16 +71,21 @@ def simpleapi_download( for p, i in (attr.index_url_overrides or {}).items() } - download_kwargs = {} - if bazel_features.external_deps.download_has_block_param: - download_kwargs["block"] = not parallel_download - # NOTE @aignas 2024-03-31: we are not merging results from multiple indexes # to replicate how `pip` would handle this case. contents = {} index_urls = [attr.index_url] + attr.extra_index_urls read_simpleapi = read_simpleapi or _read_simpleapi + download_kwargs = {} + if bazel_features.external_deps.download_has_block_param: + download_kwargs["block"] = not parallel_download + + if len(index_urls) == 1 or index_url_overrides: + download_kwargs["allow_fail"] = False + else: + download_kwargs["allow_fail"] = True + input_sources = attr.sources found_on_index = {} @@ -225,7 +230,6 @@ def _read_simpleapi(ctx, url, attr, cache, versions, get_auth = None, **download url = [real_url], output = output, auth = get_auth(ctx, [real_url], ctx_attr = attr), - allow_fail = True, **download_kwargs ) diff --git a/tests/pypi/simpleapi_download/simpleapi_download_tests.bzl b/tests/pypi/simpleapi_download/simpleapi_download_tests.bzl index 953df5c107..9a6b7ca5af 100644 --- a/tests/pypi/simpleapi_download/simpleapi_download_tests.bzl +++ b/tests/pypi/simpleapi_download/simpleapi_download_tests.bzl @@ -23,13 +23,10 @@ _tests = [] def _test_simple(env): calls = [] - def read_simpleapi(ctx, url, versions, attr, cache, get_auth, block): - _ = ctx # buildifier: disable=unused-variable - _ = attr - _ = cache - _ = get_auth - _ = versions + def read_simpleapi(ctx, url, versions, attr, cache, get_auth, block, allow_fail): + _ = ctx, attr, cache, get_auth, versions # buildifier: disable=unused-variable env.expect.that_bool(block).equals(False) + env.expect.that_bool(allow_fail).equals(True) calls.append(url) if "foo" in url and "main" in url: return struct( @@ -96,13 +93,10 @@ def _test_fail(env): calls = [] fails = [] - def read_simpleapi(ctx, url, versions, attr, cache, get_auth, block): - _ = ctx # buildifier: disable=unused-variable - _ = attr - _ = cache - _ = get_auth - _ = versions + def read_simpleapi(ctx, url, versions, attr, cache, get_auth, block, allow_fail): + _ = ctx, attr, cache, get_auth, versions # buildifier: disable=unused-variable env.expect.that_bool(block).equals(False) + env.expect.that_bool(allow_fail).equals(True) calls.append(url) if "foo" in url: return struct( @@ -130,9 +124,7 @@ def _test_fail(env): report_progress = lambda _: None, ), attr = struct( - index_url_overrides = { - "foo": "invalid", - }, + index_url_overrides = {}, index_url = "main", extra_index_urls = ["extra"], sources = {"bar": None, "baz": None, "foo": None}, @@ -149,7 +141,7 @@ def _test_fail(env): Failed to download metadata of the following packages from urls: { "bar": ["main", "extra"], - "foo": "invalid", + "foo": ["main", "extra"], } If you would like to skip downloading metadata for these packages please add 'simpleapi_skip=[ @@ -159,15 +151,82 @@ If you would like to skip downloading metadata for these packages please add 'si """, ]) env.expect.that_collection(calls).contains_exactly([ - "invalid/foo/", + "main/foo/", "main/bar/", "main/baz/", - "invalid/foo/", + "extra/foo/", "extra/bar/", ]) _tests.append(_test_fail) +def _test_allow_fail_single_index(env): + calls = [] + fails = [] + + def read_simpleapi(ctx, *, url, versions, attr, cache, get_auth, block, allow_fail): + _ = ctx, attr, cache, get_auth, versions # buildifier: disable=unused-variable + env.expect.that_bool(block).equals(False) + env.expect.that_bool(allow_fail).equals(False) + calls.append(url) + return struct( + output = struct( + sdists = {"deadbeef": url.strip("/").split("/")[-1]}, + whls = {"deadb33f": url.strip("/").split("/")[-1]}, + sha256s_by_version = {"fizz": url.strip("/").split("/")[-1]}, + ), + success = True, + ) + + contents = simpleapi_download( + ctx = struct( + getenv = {}.get, + report_progress = lambda _: None, + ), + attr = struct( + index_url_overrides = { + "foo": "extra", + }, + index_url = "main", + extra_index_urls = [], + sources = {"bar": None, "baz": None, "foo": None}, + envsubst = [], + ), + cache = pypi_cache(), + parallel_download = True, + read_simpleapi = read_simpleapi, + _fail = fails.append, + ) + + env.expect.that_collection(fails).contains_exactly([]) + env.expect.that_collection(calls).contains_exactly([ + "main/bar/", + "main/baz/", + "extra/foo/", + ]) + env.expect.that_dict(contents).contains_exactly({ + "bar": struct( + index_url = "main/bar/", + sdists = {"deadbeef": "bar"}, + sha256s_by_version = {"fizz": "bar"}, + whls = {"deadb33f": "bar"}, + ), + "baz": struct( + index_url = "main/baz/", + sdists = {"deadbeef": "baz"}, + sha256s_by_version = {"fizz": "baz"}, + whls = {"deadb33f": "baz"}, + ), + "foo": struct( + index_url = "extra/foo/", + sdists = {"deadbeef": "foo"}, + sha256s_by_version = {"fizz": "foo"}, + whls = {"deadb33f": "foo"}, + ), + }) + +_tests.append(_test_allow_fail_single_index) + def _test_download_url(env): downloads = {} From 41fccfd401d47f1d97c59b8af7a78cb8608e95ff Mon Sep 17 00:00:00 2001 From: michaelm-openai Date: Fri, 20 Mar 2026 00:04:08 -0700 Subject: [PATCH 672/922] fix(pypi) Correct likely _BAZEL_REPO_FILE_GLOBS typo (#3670) Based on https://github.com/bazel-contrib/rules_python/blob/f5f35d6e9a2f01c754e59c6ff97bfdcdf81abc35/python/private/toolchains_repo.bzl#L340-L345 I suspect the duplicate `"WORKSPACE"` was meant to be `"WORKSPACE.bzlmod"`. Potentially this could be factored out and shared, or `toolchains_repo.bzl` be updated to also include `"BUILD"`. --- python/private/pypi/whl_library_targets.bzl | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/python/private/pypi/whl_library_targets.bzl b/python/private/pypi/whl_library_targets.bzl index 0fe2c52d9f..dc99aab532 100644 --- a/python/private/pypi/whl_library_targets.bzl +++ b/python/private/pypi/whl_library_targets.bzl @@ -39,7 +39,7 @@ _BAZEL_REPO_FILE_GLOBS = [ "BUILD.bazel", "REPO.bazel", "WORKSPACE", - "WORKSPACE", + "WORKSPACE.bzlmod", "WORKSPACE.bazel", ] From d0b9baadc43eff92b3b9df0a04343e7ca653c44a Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Sat, 21 Mar 2026 08:46:31 -0700 Subject: [PATCH 673/922] refactor!: create full venv for bootstrap=system_python (#3473) The system_python bootstrap is basically just an alternative stage1 bootstrap now. Finish unifying it with bootstrap=script by having it support and create a full venv. While this is the new default behavior, it's disabled for: * Windows: some difficult to debug failures came up, so disable using it there * Bazel 7: Bazel 7 doesn't have the `File.is_symlink` APIs, so it's problematic to enable it there due to zipapp support. Switching from non-venv to venv layouts is a significant change and likely to break something, so marking this as a breaking change. Additional notes: * rules_pkg 1.2+ is needed to package a venv-based binary, due to extensive use of symlinks. Work towards https://github.com/bazel-contrib/rules_python/issues/2156 --- .bazelci/presubmit.yml | 5 +- .bazelrc | 4 + CHANGELOG.md | 10 +- MODULE.bazel | 2 +- python/private/py_executable.bzl | 31 +++- python/private/python_bootstrap_template.txt | 142 +++++++++++++++--- python/private/stage1_bootstrap_template.sh | 8 +- python/private/zipapp/zip_main_template.py | 50 +++--- ...m_python_zipapp_external_bootstrap_test.sh | 1 + .../custom_platform_toolchain_test.py | 8 +- 10 files changed, 203 insertions(+), 58 deletions(-) diff --git a/.bazelci/presubmit.yml b/.bazelci/presubmit.yml index b33b8a8d20..aeeb7103d6 100644 --- a/.bazelci/presubmit.yml +++ b/.bazelci/presubmit.yml @@ -31,7 +31,8 @@ buildifier: # As a regression test for #225, check that wheel targets still build when # their package path is qualified with the repo name. - "@rules_python//examples/wheel/..." - build_flags: + build_flags: &reusable_config_build_flags + - "--experimental_repository_cache_hardlinks=false" - "--keep_going" - "--build_tag_filters=-integration-test" - "--verbose_failures" @@ -42,6 +43,7 @@ buildifier: - "--test_tag_filters=-integration-test" .common_workspace_flags_min_bazel: &common_workspace_flags_min_bazel build_flags: + - "--experimental_repository_cache_hardlinks=false" - "--noenable_bzlmod" - "--build_tag_filters=-integration-test" test_flags: @@ -292,6 +294,7 @@ tasks: name: "RBE: Ubuntu, minimum Bazel" platform: rbe_ubuntu2204 build_flags: + - "--experimental_repository_cache_hardlinks=false" # BazelCI sets --action_env=BAZEL_DO_NOT_DETECT_CPP_TOOLCHAIN=1, # which prevents cc toolchain autodetection from working correctly # on Bazel 5.4 and earlier. To workaround this, manually specify the diff --git a/.bazelrc b/.bazelrc index 4c3f5b3a12..49f98ad7a1 100644 --- a/.bazelrc +++ b/.bazelrc @@ -30,6 +30,9 @@ common --incompatible_use_plus_in_repo_names # See https://github.com/bazel-contrib/rules_python/issues/3655 common --incompatible_strict_action_env=false +# To work around bug on bazel 7 +common:ci --experimental_repository_cache_hardlinks=false + # Windows makes use of runfiles for some rules build --enable_runfiles @@ -50,3 +53,4 @@ common --incompatible_python_disallow_native_rules common --incompatible_no_implicit_file_export build --lockfile_mode=update + diff --git a/CHANGELOG.md b/CHANGELOG.md index 18be4def9c..16a591b159 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -63,13 +63,17 @@ END_UNRELEASED_TEMPLATE * {obj}`--windows_enable_symlinks` is required. Add `startup --windows_enable_symlinks` to your `.bazelrc` to enable Bazel using full symlink support on Windows. +* venv-based binaries are created by default ({obj}`--bootstrap_impl=system_python`) + on supported platforms (Linux/Mac with Bazel 8+). Other changes: * (pypi) Update dependencies used for `compile_pip_requirements`, building sdists in the `whl_library` rule and fetching wheels using `pip`. -* (pypi) We will set `allow_fail` to `False` if the {attr}`experimental_index_url_overrides` is set - to a non-empty value. This means that failures will be no-longer cached in this particular case. - ([#3260](https://github.com/bazel-contrib/rules_python/issues/3260) and +* (pypi) We will set `allow_fail` to `False` if the + {attr}`experimental_index_url_overrides` is set + to a non-empty value. This means that failures will be no-longer cached in + this particular case. + ([#3260](https://github.com/bazel-contrib/rules_python/issues/3260) and [#2632](https://github.com/bazel-contrib/rules_python/issues/2632)) {#v0-0-0-fixed} diff --git a/MODULE.bazel b/MODULE.bazel index d2d2d72f78..7cfe4ee576 100644 --- a/MODULE.bazel +++ b/MODULE.bazel @@ -223,7 +223,7 @@ bazel_dep(name = "rules_testing", version = "0.6.0", dev_dependency = True) bazel_dep(name = "rules_shell", version = "0.3.0", dev_dependency = True) bazel_dep(name = "rules_multirun", version = "0.9.0", dev_dependency = True) bazel_dep(name = "bazel_ci_rules", version = "1.0.0", dev_dependency = True) -bazel_dep(name = "rules_pkg", version = "1.0.1", dev_dependency = True) +bazel_dep(name = "rules_pkg", version = "1.2.0", dev_dependency = True) bazel_dep(name = "other", version = "0", dev_dependency = True) bazel_dep(name = "another_module", version = "0", dev_dependency = True) diff --git a/python/private/py_executable.bzl b/python/private/py_executable.bzl index 495c7dddcf..805f95fa4c 100644 --- a/python/private/py_executable.bzl +++ b/python/private/py_executable.bzl @@ -523,9 +523,36 @@ def _create_zip_main(ctx, *, stage2_bootstrap, runtime_details, venv): # * https://github.com/python/cpython/blob/main/Modules/getpath.py # * https://github.com/python/cpython/blob/main/Lib/site.py def _create_venv(ctx, output_prefix, imports, runtime_details, add_runfiles_root_to_sys_path, extra_deps): - create_full_venv = BootstrapImplFlag.get_value(ctx) == BootstrapImplFlag.SCRIPT venv = "_{}.venv".format(output_prefix.lstrip("_")) + # The pyvenv.cfg file must be present to trigger the venv site hooks. + # Because it's paths are expected to be absolute paths, we can't reliably + # put much in it. See https://github.com/python/cpython/issues/83650 + pyvenv_cfg = ctx.actions.declare_file("{}/pyvenv.cfg".format(venv)) + ctx.actions.write(pyvenv_cfg, "") + + is_bootstrap_script = BootstrapImplFlag.get_value(ctx) == BootstrapImplFlag.SCRIPT + is_windows = target_platform_has_any_constraint(ctx, ctx.attr._windows_constraints) + + create_full_venv = True + + # The legacy build_python_zip codepath (enabled by default on windows) isn't + # compatible with full venv. + # TODO: Use non-build_python_zip codepath for Windows + if is_windows: + create_full_venv = False + elif not rp_config.bazel_8_or_later and not is_bootstrap_script: + # Full venv for Bazel 7 + system_python is disabled because packaging + # it using build_python_zip=true or rules_pkg breaks. + # * Using build_python_zip=true breaks because the legacy zipapp support + # doesn't handle symlinks correctly. + # * Using rules_pkg breaks for two reasons: + # 1. It requires rules_pkg 1.2, which crashes under Bazel 7 + # 2. It requires File.is_symlink, which is a Bazel 8+ API. + # While bootstrap=script has the same problems, it has always been like + # that. + create_full_venv = False + if create_full_venv: # The pyvenv.cfg file must be present to trigger the venv site hooks. # Because it's paths are expected to be absolute paths, we can't reliably @@ -534,7 +561,6 @@ def _create_venv(ctx, output_prefix, imports, runtime_details, add_runfiles_root ctx.actions.write(pyvenv_cfg, "") else: pyvenv_cfg = None - runtime = runtime_details.effective_runtime venvs_use_declare_symlink_enabled = ( @@ -561,6 +587,7 @@ def _create_venv(ctx, output_prefix, imports, runtime_details, add_runfiles_root # needed or used at runtime. However, the zip code uses the interpreter # File object to figure out some paths. interpreter = ctx.actions.declare_file("{}/{}".format(bin_dir, py_exe_basename)) + ctx.actions.write(interpreter, "actual:{}".format(interpreter_actual_path)) elif runtime.interpreter: diff --git a/python/private/python_bootstrap_template.txt b/python/private/python_bootstrap_template.txt index f08de8e0f5..4efd46690a 100644 --- a/python/private/python_bootstrap_template.txt +++ b/python/private/python_bootstrap_template.txt @@ -8,8 +8,11 @@ from __future__ import print_function import sys import os +from os.path import dirname, join, basename import subprocess import uuid +import shutil + # NOTE: The sentinel strings are split (e.g., "%stage2" + "_bootstrap%") so that # the substitution logic won't replace them. This allows runtime detection of # unsubstituted placeholders, which occurs when native py_binary is used in @@ -51,7 +54,14 @@ IS_ZIPFILE = "%is_zipfile%" == "1" # 0 or 1. # If 1, then a venv will be created at runtime that replicates what would have # been the build-time structure. -RECREATE_VENV_AT_RUNTIME="%recreate_venv_at_runtime%" +RECREATE_VENV_AT_RUNTIME = "%recreate_venv_at_runtime%" == "1" +# 0 or 1 +# If 1, then the path to python will be resolved by running +# PYTHON_BINARY_ACTUAL to determine the actual underlying interpreter. +RESOLVE_PYTHON_BINARY_AT_RUNTIME = "%resolve_python_binary_at_runtime%" == "1" +# venv-relative path to the site-packages +# e.g. lib/python3.12t/site-packages +VENV_REL_SITE_PACKAGES = "%venv_rel_site_packages%" WORKSPACE_NAME = "%workspace_name%" @@ -65,6 +75,7 @@ else: INTERPRETER_ARGS = [arg for arg in _INTERPRETER_ARGS_RAW.split("\n") if arg] ADDITIONAL_INTERPRETER_ARGS = os.environ.get("RULES_PYTHON_ADDITIONAL_INTERPRETER_ARGS", "") +EXTRACT_ROOT = os.environ.get("RULES_PYTHON_EXTRACT_ROOT") def is_running_from_zip(): return IS_ZIPFILE @@ -149,7 +160,7 @@ def print_verbose(*args, mapping=None, values=None): if mapping is not None: for key, value in sorted((mapping or {}).items()): print( - "bootstrap: stage 1: ", + "bootstrap: stage 1:", *(list(args) + ["{}={}".format(key, repr(value))]), file=sys.stderr, flush=True @@ -254,10 +265,17 @@ def extract_zip(zip_path, dest_dir): # https://docs.microsoft.com/en-us/windows/desktop/fileio/naming-a-file#maximum-path-length-limitation file_path = os.path.abspath(os.path.join(dest_dir, info.filename)) # The Unix st_mode bits (see "man 7 inode") are stored in the upper 16 - # bits of external_attr. Of those, we set the lower 12 bits, which are the - # file mode bits (since the file type bits can't be set by chmod anyway). + # bits of external_attr. attrs = info.external_attr >> 16 - if attrs != 0: # Rumor has it these can be 0 for zips created on Windows. + # Symlink bit in st_mode is 0o120000. + if (attrs & 0o170000) == 0o120000: + with open(file_path, "r") as f: + target = f.read() + os.remove(file_path) + os.symlink(target, file_path) + # Of those, we set the lower 12 bits, which are the + # file mode bits (since the file type bits can't be set by chmod anyway). + elif attrs != 0: # Rumor has it these can be 0 for zips created on Windows. os.chmod(file_path, attrs & 0o7777) # Create the runfiles tree by extracting the zip file @@ -268,6 +286,57 @@ def create_runfiles_root(): # important that deletion code be in sync with this directory structure return os.path.join(temp_dir, 'runfiles') +def _create_venv(runfiles_root): + runfiles_venv = join(runfiles_root, dirname(dirname(PYTHON_BINARY))) + if EXTRACT_ROOT: + venv = join(EXTRACT_ROOT, runfiles_venv) + os.makedirs(venv, exist_ok=True) + cleanup_dir = None + else: + import tempfile + venv = tempfile.mkdtemp("", f"bazel.{basename(runfiles_venv)}.") + cleanup_dir = venv + + python_exe_actual = find_binary(runfiles_root, PYTHON_BINARY_ACTUAL) + + # See stage1_bootstrap_template.sh for details on this code path. In short, + # this handles when the build-time python version doesn't match runtime + # and if the initially resolved python_exe_actual is a wrapper script. + if RESOLVE_PYTHON_BINARY_AT_RUNTIME: + src = f""" +import sys, site +print(sys.executable) +print(site.getsitepackages(["{venv}"])[-1]) + """ + output = subprocess.check_output([python_exe_actual, "-I"], shell=True, + encoding = "utf8", input=src) + output = output.strip().split("\n") + python_exe_actual = output[0] + venv_site_packages = output[1] + os.makedirs(dirname(venv_site_packages), exist_ok=True) + runfiles_venv_site_packages = join(runfiles_venv, VENV_REL_SITE_PACKAGES) + else: + python_exe_actual = find_binary(runfiles_root, PYTHON_BINARY_ACTUAL) + venv_site_packages = join(venv, "lib") + runfiles_venv_site_packages = join(runfiles_venv, "lib") + + if python_exe_actual is None: + raise AssertionError('Could not find python binary: ' + repr(PYTHON_BINARY_ACTUAL)) + + venv_bin = join(venv, "bin") + try: + os.mkdir(venv_bin) + except FileExistsError as e: + pass + + # Match the basename; some tools, e.g. pyvenv key off the executable name + venv_python_exe = join(venv_bin, os.path.basename(python_exe_actual)) + _symlink_exist_ok(from_=venv_python_exe, to=python_exe_actual) + _symlink_exist_ok(from_=join(venv, "lib"), to=join(runfiles_venv, "lib")) + _symlink_exist_ok(from_=venv_site_packages, to=runfiles_venv_site_packages) + _symlink_exist_ok(from_=join(venv, "pyvenv.cfg"), to=join(runfiles_venv, "pyvenv.cfg")) + return cleanup_dir, venv_python_exe + def runfiles_envvar(runfiles_root): """Finds the runfiles manifest or the runfiles directory. @@ -311,7 +380,7 @@ def runfiles_envvar(runfiles_root): return (None, None) def execute_file(python_program, main_filename, args, env, runfiles_root, - workspace, delete_runfiles_root): + workspace, delete_dirs): # type: (str, str, list[str], dict[str, str], str, str|None, str|None) -> ... """Executes the given Python file using the various environment settings. @@ -326,8 +395,8 @@ def execute_file(python_program, main_filename, args, env, runfiles_root, runfiles_root: (str) Path to the runfiles root directory workspace: (str|None) Name of the workspace to execute in. This is expected to be a directory under the runfiles tree. - delete_runfiles_root: (bool), True if the runfiles root should be deleted - after a successful (exit code zero) program run, False if not. + delete_dirs: (list[str]) directories that should be deleted after the user + program has finished running. """ argv = [python_program] argv.extend(INTERPRETER_ARGS) @@ -351,20 +420,19 @@ def execute_file(python_program, main_filename, args, env, runfiles_root, # can't execv because we need control to return here. This only # happens for targets built in the host config. # - if not (is_windows() or workspace or delete_runfiles_root): + if not (is_windows() or workspace or delete_dirs): _run_execv(python_program, argv, env) + print_verbose("run: subproc: environ:", mapping=os.environ) + print_verbose("run: subproc: cwd:", workspace) + print_verbose("run: subproc: argv:", values=argv) ret_code = subprocess.call( - argv, - env=env, - cwd=workspace - ) + argv, env=env, cwd=workspace) - if delete_runfiles_root: - # NOTE: dirname() is called because create_runfiles_root() creates a - # sub-directory within a temporary directory, and we want to remove the - # whole temporary directory. - shutil.rmtree(os.path.dirname(runfiles_root), True) + if delete_dirs: + for delete_dir in delete_dirs: + print_verbose("rmtree:", delete_dir) + shutil.rmtree(delete_dir, True) sys.exit(ret_code) def _run_execv(python_program, argv, env): @@ -374,9 +442,27 @@ def _run_execv(python_program, argv, env): print_verbose("RunExecv: environ:", mapping=os.environ) print_verbose("RunExecv: python:", python_program) print_verbose("RunExecv: argv:", values=argv) - os.execv(python_program, argv) + try: + os.execv(python_program, argv) + except: + with open(python_program, 'rb') as f: + print_verbose("pyprog head:" + str(f.read(50))) + raise + +def _symlink_exist_ok(*, from_, to): + try: + os.symlink(to, from_) + except FileExistsError: + pass + + def main(): + print_verbose("sys.version:", sys.version) + print_verbose("initial argv:", values=sys.argv) + print_verbose("initial cwd:", os.getcwd()) + print_verbose("initial environ:", mapping=os.environ) + print_verbose("initial sys.path:", values=sys.path) print_verbose("STAGE2_BOOTSTRAP:", STAGE2_BOOTSTRAP) print_verbose("PYTHON_BINARY:", PYTHON_BINARY) print_verbose("PYTHON_BINARY_ACTUAL:", PYTHON_BINARY_ACTUAL) @@ -399,12 +485,16 @@ def main(): main_rel_path = os.path.normpath(STAGE2_BOOTSTRAP) print_verbose("main_rel_path:", main_rel_path) + delete_dirs = [] + if is_running_from_zip(): runfiles_root = create_runfiles_root() - delete_runfiles_root = True + # NOTE: dirname() is called because create_runfiles_root() creates a + # sub-directory within a temporary directory, and we want to remove the + # whole temporary directory. + delete_dirs.append(dirname(runfiles_root)) else: runfiles_root = find_runfiles_root(main_rel_path) - delete_runfiles_root = False print_verbose("runfiles root:", runfiles_root) @@ -433,6 +523,14 @@ def main(): repr(PYTHON_BINARY_ACTUAL) )) + if RECREATE_VENV_AT_RUNTIME: + # When the venv is created at runtime, python_program is PYTHON_BINARY_ACTUAL + # so we have to re-point it to the symlink in the venv + venv, python_program = _create_venv(runfiles_root) + delete_dirs.append(venv) + else: + python_program = find_python_binary(runfiles_root) + # Some older Python versions on macOS (namely Python 3.7) may unintentionally # leave this environment variable set after starting the interpreter, which # causes problems with Python subprocesses correctly locating sys.executable, @@ -456,7 +554,7 @@ def main(): execute_file( python_program, main_filename, args, new_env, runfiles_root, workspace, - delete_runfiles_root = delete_runfiles_root, + delete_dirs = delete_dirs, ) except EnvironmentError: diff --git a/python/private/stage1_bootstrap_template.sh b/python/private/stage1_bootstrap_template.sh index 2fa70e9910..c72e2740f2 100644 --- a/python/private/stage1_bootstrap_template.sh +++ b/python/private/stage1_bootstrap_template.sh @@ -6,14 +6,14 @@ if [[ -n "${RULES_PYTHON_BOOTSTRAP_VERBOSE:-}" ]]; then set -x fi -# runfiles-relative path +# runfiles-root-relative path STAGE2_BOOTSTRAP="%stage2_bootstrap%" -# runfiles-relative path to python interpreter to use. +# runfiles-root-relative path to python interpreter to use. # This is the `bin/python3` path in the binary's venv. PYTHON_BINARY='%python_binary%' # The path that PYTHON_BINARY should symlink to. -# runfiles-relative path, absolute path, or single word. +# runfiles-root-relative path, absolute path, or single word. # Only applicable for zip files or when venv is recreated at runtime. PYTHON_BINARY_ACTUAL="%python_binary_actual%" @@ -211,7 +211,7 @@ elif [[ "$RECREATE_VENV_AT_RUNTIME" == "1" ]]; then read -r resolved_py_exe read -r resolved_site_packages } < <("$python_exe_actual" -I <> 16 - if attrs != 0: # Rumor has it these can be 0 for zips created on Windows. + # Symlink bit in st_mode is 0o120000. + if (attrs & 0o170000) == 0o120000: + with open(file_path, "r") as f: + target = f.read() + os.remove(file_path) + os.symlink(target, file_path) + # Of those, we set the lower 12 bits, which are the + # file mode bits (since the file type bits can't be set by chmod anyway). + elif attrs != 0: # Rumor has it these can be 0 for zips created on Windows. os.chmod(file_path, attrs & 0o7777) # Create the runfiles tree by extracting the zip file -def create_module_space(): +def create_runfiles_root(): temp_dir = tempfile.mkdtemp("", "Bazel.runfiles_") extract_zip(os.path.dirname(__file__), temp_dir) - # IMPORTANT: Later code does `rm -fr` on dirname(module_space) -- it's + # IMPORTANT: Later code does `rm -fr` on dirname(runfiles_root) -- it's # important that deletion code be in sync with this directory structure return os.path.join(temp_dir, "runfiles") @@ -187,7 +194,7 @@ def execute_file( main_filename, args, env, - module_space, + runfiles_root, workspace, ): # type: (str, str, list[str], dict[str, str], str, str|None, str|None) -> ... @@ -201,7 +208,7 @@ def execute_file( main_filename: (str) The Python file to execute args: (list[str]) Additional args to pass to the Python file env: (dict[str, str]) A dict of environment variables to set for the execution - module_space: (str) Path to the module space/runfiles tree directory + runfiles_root: (str) Path to the runfiles tree directory workspace: (str|None) Name of the workspace to execute in. This is expected to be a directory under the runfiles tree. """ @@ -223,10 +230,11 @@ def execute_file( ret_code = subprocess.call(subprocess_argv, env=env, cwd=workspace) sys.exit(ret_code) finally: - # NOTE: dirname() is called because create_module_space() creates a + # NOTE: dirname() is called because create_runfiles_root() creates a # sub-directory within a temporary directory, and we want to remove the # whole temporary directory. - shutil.rmtree(os.path.dirname(module_space), True) + ##shutil.rmtree(os.path.dirname(runfiles_root), True) + pass def main(): @@ -249,16 +257,16 @@ def main(): if is_windows(): main_rel_path = main_rel_path.replace("/", os.sep) - module_space = create_module_space() - print_verbose("extracted runfiles to:", module_space) + runfiles_root = create_runfiles_root() + print_verbose("extracted runfiles to:", runfiles_root) - new_env["RUNFILES_DIR"] = module_space + new_env["RUNFILES_DIR"] = runfiles_root # Don't prepend a potentially unsafe path to sys.path # See: https://docs.python.org/3.11/using/cmdline.html#envvar-PYTHONSAFEPATH new_env["PYTHONSAFEPATH"] = "1" - main_filename = os.path.join(module_space, main_rel_path) + main_filename = os.path.join(runfiles_root, main_rel_path) main_filename = get_windows_path_with_unc_prefix(main_filename) assert os.path.exists(main_filename), ( "Cannot exec() %r: file not found." % main_filename @@ -268,18 +276,18 @@ def main(): ) if _PYTHON_BINARY_VENV: - python_program = os.path.join(module_space, _PYTHON_BINARY_VENV) + python_program = os.path.join(runfiles_root, _PYTHON_BINARY_VENV) # When a venv is used, the `bin/python3` symlink may need to be created. # This case occurs when "create venv at runtime" or "resolve python at # runtime" modes are enabled. if not os.path.lexists(python_program): # The venv bin/python3 interpreter should always be under runfiles, but # double check. We don't want to accidentally create symlinks elsewhere - if not python_program.startswith(module_space): + if not python_program.startswith(runfiles_root): raise AssertionError( "Program's venv binary not under runfiles: {python_program}" ) - symlink_to = find_binary(module_space, _PYTHON_BINARY_ACTUAL) + symlink_to = find_binary(runfiles_root, _PYTHON_BINARY_ACTUAL) os.makedirs(os.path.dirname(python_program), exist_ok=True) try: os.symlink(symlink_to, python_program) @@ -289,7 +297,7 @@ def main(): ) from e else: - python_program = find_binary(module_space, _PYTHON_BINARY_ACTUAL) + python_program = find_binary(runfiles_root, _PYTHON_BINARY_ACTUAL) if python_program is None: raise AssertionError( "Could not find python binary: " + _PYTHON_BINARY_ACTUAL @@ -309,7 +317,7 @@ def main(): # change directory to the right runfiles directory. # (So that the data files are accessible) if os.environ.get("RUN_UNDER_RUNFILES") == "1": - workspace = os.path.join(module_space, _WORKSPACE_NAME) + workspace = os.path.join(runfiles_root, _WORKSPACE_NAME) sys.stdout.flush() execute_file( @@ -317,7 +325,7 @@ def main(): main_filename, args, new_env, - module_space, + runfiles_root, workspace, ) diff --git a/tests/py_zipapp/system_python_zipapp_external_bootstrap_test.sh b/tests/py_zipapp/system_python_zipapp_external_bootstrap_test.sh index 4710741a5a..21c6741197 100755 --- a/tests/py_zipapp/system_python_zipapp_external_bootstrap_test.sh +++ b/tests/py_zipapp/system_python_zipapp_external_bootstrap_test.sh @@ -12,6 +12,7 @@ fi # output. ZIPAPP="${ZIPAPP/.exe/.zip}" +export RULES_PYTHON_BOOTSTRAP_VERBOSE=1 # We're testing the invocation of `__main__.py`, so we have to # manually pass the zipapp to python. "$PYTHON" "$ZIPAPP" diff --git a/tests/toolchains/custom_platform_toolchain_test.py b/tests/toolchains/custom_platform_toolchain_test.py index d6c083a6a2..fd28cf772e 100644 --- a/tests/toolchains/custom_platform_toolchain_test.py +++ b/tests/toolchains/custom_platform_toolchain_test.py @@ -5,10 +5,10 @@ class VerifyCustomPlatformToolchainTest(unittest.TestCase): def test_custom_platform_interpreter_used(self): - # We expect the repo name, and thus path, to have the - # platform name in it. - self.assertIn("linux-x86-install-only-stripped", sys._base_executable) - print(sys._base_executable) + # For lack of a better option, check the version. Identifying the + self.assertEqual( + "3.13.1", + f"{sys.version_info.major}.{sys.version_info.minor}.{sys.version_info.micro}") if __name__ == "__main__": From 39765948b8b07b11be67f7853b7cf136fcca05ab Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Mon, 30 Mar 2026 22:16:57 +0900 Subject: [PATCH 674/922] build(deps): bump the pip group across 2 directories with 1 update (#3675) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Bumps the pip group with 1 update in the /docs directory: [requests](https://github.com/psf/requests). Bumps the pip group with 1 update in the /tools/publish directory: [requests](https://github.com/psf/requests). Updates `requests` from 2.32.5 to 2.33.0
Release notes

Sourced from requests's releases.

v2.33.0

2.33.0 (2026-03-25)

Announcements

  • 📣 Requests is adding inline types. If you have a typed code base that uses Requests, please take a look at #7271. Give it a try, and report any gaps or feedback you may have in the issue. 📣

Security

  • CVE-2026-25645 requests.utils.extract_zipped_paths now extracts contents to a non-deterministic location to prevent malicious file replacement. This does not affect default usage of Requests, only applications calling the utility function directly.

Improvements

  • Migrated to a PEP 517 build system using setuptools. (#7012)

Bugfixes

  • Fixed an issue where an empty netrc entry could cause malformed authentication to be applied to Requests on Python 3.11+. (#7205)

Deprecations

  • Dropped support for Python 3.9 following its end of support. (#7196)

Documentation

  • Various typo fixes and doc improvements.

New Contributors

Full Changelog: https://github.com/psf/requests/blob/main/HISTORY.md#2330-2026-03-25

Changelog

Sourced from requests's changelog.

2.33.0 (2026-03-25)

Announcements

  • 📣 Requests is adding inline types. If you have a typed code base that uses Requests, please take a look at #7271. Give it a try, and report any gaps or feedback you may have in the issue. 📣

Security

  • CVE-2026-25645 requests.utils.extract_zipped_paths now extracts contents to a non-deterministic location to prevent malicious file replacement. This does not affect default usage of Requests, only applications calling the utility function directly.

Improvements

  • Migrated to a PEP 517 build system using setuptools. (#7012)

Bugfixes

  • Fixed an issue where an empty netrc entry could cause malformed authentication to be applied to Requests on Python 3.11+. (#7205)

Deprecations

  • Dropped support for Python 3.9 following its end of support. (#7196)

Documentation

  • Various typo fixes and doc improvements.
Commits
  • bc04dfd v2.33.0
  • 66d21cb Merge commit from fork
  • 8b9bc8f Move badges to top of README (#7293)
  • e331a28 Remove unused extraction call (#7292)
  • 753fd08 docs: fix FAQ grammar in httplib2 example
  • 774a0b8 docs(socks): same block as other sections
  • 9c72a41 Bump github/codeql-action from 4.33.0 to 4.34.1
  • ebf7190 Bump github/codeql-action from 4.32.0 to 4.33.0
  • 0e4ae38 docs: exclude Response.is_permanent_redirect from API docs (#7244)
  • d568f47 docs: clarify Quickstart POST example (#6960)
  • Additional commits viewable in compare view

Updates `requests` from 2.32.5 to 2.33.0
Release notes

Sourced from requests's releases.

v2.33.0

2.33.0 (2026-03-25)

Announcements

  • 📣 Requests is adding inline types. If you have a typed code base that uses Requests, please take a look at #7271. Give it a try, and report any gaps or feedback you may have in the issue. 📣

Security

  • CVE-2026-25645 requests.utils.extract_zipped_paths now extracts contents to a non-deterministic location to prevent malicious file replacement. This does not affect default usage of Requests, only applications calling the utility function directly.

Improvements

  • Migrated to a PEP 517 build system using setuptools. (#7012)

Bugfixes

  • Fixed an issue where an empty netrc entry could cause malformed authentication to be applied to Requests on Python 3.11+. (#7205)

Deprecations

  • Dropped support for Python 3.9 following its end of support. (#7196)

Documentation

  • Various typo fixes and doc improvements.

New Contributors

Full Changelog: https://github.com/psf/requests/blob/main/HISTORY.md#2330-2026-03-25

Changelog

Sourced from requests's changelog.

2.33.0 (2026-03-25)

Announcements

  • 📣 Requests is adding inline types. If you have a typed code base that uses Requests, please take a look at #7271. Give it a try, and report any gaps or feedback you may have in the issue. 📣

Security

  • CVE-2026-25645 requests.utils.extract_zipped_paths now extracts contents to a non-deterministic location to prevent malicious file replacement. This does not affect default usage of Requests, only applications calling the utility function directly.

Improvements

  • Migrated to a PEP 517 build system using setuptools. (#7012)

Bugfixes

  • Fixed an issue where an empty netrc entry could cause malformed authentication to be applied to Requests on Python 3.11+. (#7205)

Deprecations

  • Dropped support for Python 3.9 following its end of support. (#7196)

Documentation

  • Various typo fixes and doc improvements.
Commits
  • bc04dfd v2.33.0
  • 66d21cb Merge commit from fork
  • 8b9bc8f Move badges to top of README (#7293)
  • e331a28 Remove unused extraction call (#7292)
  • 753fd08 docs: fix FAQ grammar in httplib2 example
  • 774a0b8 docs(socks): same block as other sections
  • 9c72a41 Bump github/codeql-action from 4.33.0 to 4.34.1
  • ebf7190 Bump github/codeql-action from 4.32.0 to 4.33.0
  • 0e4ae38 docs: exclude Response.is_permanent_redirect from API docs (#7244)
  • d568f47 docs: clarify Quickstart POST example (#6960)
  • Additional commits viewable in compare view

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) ---
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot show ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore major version` will close this group update PR and stop Dependabot creating any more for the specific dependency's major version (unless you unignore this specific dependency's major version or upgrade to it yourself) - `@dependabot ignore minor version` will close this group update PR and stop Dependabot creating any more for the specific dependency's minor version (unless you unignore this specific dependency's minor version or upgrade to it yourself) - `@dependabot ignore ` will close this group update PR and stop Dependabot creating any more for the specific dependency (unless you unignore this specific dependency or upgrade to it yourself) - `@dependabot unignore ` will remove all of the ignore conditions of the specified dependency - `@dependabot unignore ` will remove the ignore condition of the specified dependency and ignore conditions You can disable automated security fix PRs for this repo from the [Security Alerts page](https://github.com/bazel-contrib/rules_python/network/alerts).
Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> --- docs/requirements.txt | 6 +++--- tools/publish/requirements_darwin.txt | 6 +++--- tools/publish/requirements_linux.txt | 6 +++--- tools/publish/requirements_universal.txt | 6 +++--- tools/publish/requirements_windows.txt | 6 +++--- 5 files changed, 15 insertions(+), 15 deletions(-) diff --git a/docs/requirements.txt b/docs/requirements.txt index 05f3db8002..aee3e0a0a3 100644 --- a/docs/requirements.txt +++ b/docs/requirements.txt @@ -359,9 +359,9 @@ readthedocs-sphinx-ext==2.2.5 \ --hash=sha256:ee5fd5b99db9f0c180b2396cbce528aa36671951b9526bb0272dbfce5517bd27 \ --hash=sha256:f8c56184ea011c972dd45a90122568587cc85b0127bc9cf064d17c68bc809daa # via rules-python-docs (docs/pyproject.toml) -requests==2.32.5 \ - --hash=sha256:2462f94637a34fd532264295e186976db0f5d453d1cdd31473c85a6a161affb6 \ - --hash=sha256:dbba0bac56e100853db0ea71b82b4dfd5fe2bf6d3754a8893c3af500cec7d7cf +requests==2.33.0 \ + --hash=sha256:3324635456fa185245e24865e810cecec7b4caf933d7eb133dcde67d48cee69b \ + --hash=sha256:c7ebc5e8b0f21837386ad0e1c8fe8b829fa5f544d8df3b2253bff14ef29d7652 # via # readthedocs-sphinx-ext # sphinx diff --git a/tools/publish/requirements_darwin.txt b/tools/publish/requirements_darwin.txt index 012c4605ff..6d8de8b4c9 100644 --- a/tools/publish/requirements_darwin.txt +++ b/tools/publish/requirements_darwin.txt @@ -177,9 +177,9 @@ readme-renderer==44.0 \ --hash=sha256:2fbca89b81a08526aadf1357a8c2ae889ec05fb03f5da67f9769c9a592166151 \ --hash=sha256:8712034eabbfa6805cacf1402b4eeb2a73028f72d1166d6f5cb7f9c047c5d1e1 # via twine -requests==2.32.5 \ - --hash=sha256:2462f94637a34fd532264295e186976db0f5d453d1cdd31473c85a6a161affb6 \ - --hash=sha256:dbba0bac56e100853db0ea71b82b4dfd5fe2bf6d3754a8893c3af500cec7d7cf +requests==2.33.0 \ + --hash=sha256:3324635456fa185245e24865e810cecec7b4caf933d7eb133dcde67d48cee69b \ + --hash=sha256:c7ebc5e8b0f21837386ad0e1c8fe8b829fa5f544d8df3b2253bff14ef29d7652 # via # requests-toolbelt # twine diff --git a/tools/publish/requirements_linux.txt b/tools/publish/requirements_linux.txt index b5e9eb7b59..816dbbab1d 100644 --- a/tools/publish/requirements_linux.txt +++ b/tools/publish/requirements_linux.txt @@ -324,9 +324,9 @@ readme-renderer==44.0 \ --hash=sha256:2fbca89b81a08526aadf1357a8c2ae889ec05fb03f5da67f9769c9a592166151 \ --hash=sha256:8712034eabbfa6805cacf1402b4eeb2a73028f72d1166d6f5cb7f9c047c5d1e1 # via twine -requests==2.32.5 \ - --hash=sha256:2462f94637a34fd532264295e186976db0f5d453d1cdd31473c85a6a161affb6 \ - --hash=sha256:dbba0bac56e100853db0ea71b82b4dfd5fe2bf6d3754a8893c3af500cec7d7cf +requests==2.33.0 \ + --hash=sha256:3324635456fa185245e24865e810cecec7b4caf933d7eb133dcde67d48cee69b \ + --hash=sha256:c7ebc5e8b0f21837386ad0e1c8fe8b829fa5f544d8df3b2253bff14ef29d7652 # via # requests-toolbelt # twine diff --git a/tools/publish/requirements_universal.txt b/tools/publish/requirements_universal.txt index 92e673ba51..695d4cc386 100644 --- a/tools/publish/requirements_universal.txt +++ b/tools/publish/requirements_universal.txt @@ -311,9 +311,9 @@ readme-renderer==44.0 \ --hash=sha256:2fbca89b81a08526aadf1357a8c2ae889ec05fb03f5da67f9769c9a592166151 \ --hash=sha256:8712034eabbfa6805cacf1402b4eeb2a73028f72d1166d6f5cb7f9c047c5d1e1 # via twine -requests==2.32.5 \ - --hash=sha256:2462f94637a34fd532264295e186976db0f5d453d1cdd31473c85a6a161affb6 \ - --hash=sha256:dbba0bac56e100853db0ea71b82b4dfd5fe2bf6d3754a8893c3af500cec7d7cf +requests==2.33.0 \ + --hash=sha256:3324635456fa185245e24865e810cecec7b4caf933d7eb133dcde67d48cee69b \ + --hash=sha256:c7ebc5e8b0f21837386ad0e1c8fe8b829fa5f544d8df3b2253bff14ef29d7652 # via # requests-toolbelt # twine diff --git a/tools/publish/requirements_windows.txt b/tools/publish/requirements_windows.txt index 20058d3416..029c6bef4d 100644 --- a/tools/publish/requirements_windows.txt +++ b/tools/publish/requirements_windows.txt @@ -181,9 +181,9 @@ readme-renderer==44.0 \ --hash=sha256:2fbca89b81a08526aadf1357a8c2ae889ec05fb03f5da67f9769c9a592166151 \ --hash=sha256:8712034eabbfa6805cacf1402b4eeb2a73028f72d1166d6f5cb7f9c047c5d1e1 # via twine -requests==2.32.5 \ - --hash=sha256:2462f94637a34fd532264295e186976db0f5d453d1cdd31473c85a6a161affb6 \ - --hash=sha256:dbba0bac56e100853db0ea71b82b4dfd5fe2bf6d3754a8893c3af500cec7d7cf +requests==2.33.0 \ + --hash=sha256:3324635456fa185245e24865e810cecec7b4caf933d7eb133dcde67d48cee69b \ + --hash=sha256:c7ebc5e8b0f21837386ad0e1c8fe8b829fa5f544d8df3b2253bff14ef29d7652 # via # requests-toolbelt # twine From 293e643aa3ea3d406bb45a3b8f46ca797dc472ca Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Tue, 31 Mar 2026 07:36:27 +0900 Subject: [PATCH 675/922] build(deps): bump cryptography from 46.0.5 to 46.0.6 in /tools/publish in the pip group across 1 directory (#3677) Bumps the pip group with 1 update in the /tools/publish directory: [cryptography](https://github.com/pyca/cryptography). Updates `cryptography` from 46.0.5 to 46.0.6
Changelog

Sourced from cryptography's changelog.

46.0.6 - 2026-03-25


* **SECURITY ISSUE**: Fixed a bug where name constraints were not
applied
  to peer names during verification when the leaf certificate contains a
wildcard DNS SAN. Ordinary X.509 topologies are not affected by this
bug,
including those used by the Web PKI. Credit to **Oleh Konko (1seal)**
for
  reporting the issue. **CVE-2026-34073**

.. _v46-0-5:

Commits

[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=cryptography&package-manager=pip&previous-version=46.0.5&new-version=46.0.6)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) ---
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot show ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore major version` will close this group update PR and stop Dependabot creating any more for the specific dependency's major version (unless you unignore this specific dependency's major version or upgrade to it yourself) - `@dependabot ignore minor version` will close this group update PR and stop Dependabot creating any more for the specific dependency's minor version (unless you unignore this specific dependency's minor version or upgrade to it yourself) - `@dependabot ignore ` will close this group update PR and stop Dependabot creating any more for the specific dependency (unless you unignore this specific dependency or upgrade to it yourself) - `@dependabot unignore ` will remove all of the ignore conditions of the specified dependency - `@dependabot unignore ` will remove the ignore condition of the specified dependency and ignore conditions You can disable automated security fix PRs for this repo from the [Security Alerts page](https://github.com/bazel-contrib/rules_python/network/alerts).
Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> --- tools/publish/requirements_linux.txt | 100 +++++++++++------------ tools/publish/requirements_universal.txt | 100 +++++++++++------------ 2 files changed, 100 insertions(+), 100 deletions(-) diff --git a/tools/publish/requirements_linux.txt b/tools/publish/requirements_linux.txt index 816dbbab1d..57dada2723 100644 --- a/tools/publish/requirements_linux.txt +++ b/tools/publish/requirements_linux.txt @@ -177,56 +177,56 @@ charset-normalizer==3.4.3 \ --hash=sha256:fd10de089bcdcd1be95a2f73dbe6254798ec1bda9f450d5828c96f93e2536b9c \ --hash=sha256:fdabf8315679312cfa71302f9bd509ded4f2f263fb5b765cf1433b39106c3cc9 # via requests -cryptography==46.0.5 \ - --hash=sha256:02f547fce831f5096c9a567fd41bc12ca8f11df260959ecc7c3202555cc47a72 \ - --hash=sha256:039917b0dc418bb9f6edce8a906572d69e74bd330b0b3fea4f79dab7f8ddd235 \ - --hash=sha256:1abfdb89b41c3be0365328a410baa9df3ff8a9110fb75e7b52e66803ddabc9a9 \ - --hash=sha256:2ae6971afd6246710480e3f15824ed3029a60fc16991db250034efd0b9fb4356 \ - --hash=sha256:2b7a67c9cd56372f3249b39699f2ad479f6991e62ea15800973b956f4b73e257 \ - --hash=sha256:351695ada9ea9618b3500b490ad54c739860883df6c1f555e088eaf25b1bbaad \ - --hash=sha256:38946c54b16c885c72c4f59846be9743d699eee2b69b6988e0a00a01f46a61a4 \ - --hash=sha256:3b4995dc971c9fb83c25aa44cf45f02ba86f71ee600d81091c2f0cbae116b06c \ - --hash=sha256:3ce58ba46e1bc2aac4f7d9290223cead56743fa6ab94a5d53292ffaac6a91614 \ - --hash=sha256:3ee190460e2fbe447175cda91b88b84ae8322a104fc27766ad09428754a618ed \ - --hash=sha256:4108d4c09fbbf2789d0c926eb4152ae1760d5a2d97612b92d508d96c861e4d31 \ - --hash=sha256:420d0e909050490d04359e7fdb5ed7e667ca5c3c402b809ae2563d7e66a92229 \ - --hash=sha256:47fb8a66058b80e509c47118ef8a75d14c455e81ac369050f20ba0d23e77fee0 \ - --hash=sha256:4c3341037c136030cb46e4b1e17b7418ea4cbd9dd207e4a6f3b2b24e0d4ac731 \ - --hash=sha256:4d7e3d356b8cd4ea5aff04f129d5f66ebdc7b6f8eae802b93739ed520c47c79b \ - --hash=sha256:4d8ae8659ab18c65ced284993c2265910f6c9e650189d4e3f68445ef82a810e4 \ - --hash=sha256:4e817a8920bfbcff8940ecfd60f23d01836408242b30f1a708d93198393a80b4 \ - --hash=sha256:50bfb6925eff619c9c023b967d5b77a54e04256c4281b0e21336a130cd7fc263 \ - --hash=sha256:556e106ee01aa13484ce9b0239bca667be5004efb0aabbed28d353df86445595 \ - --hash=sha256:582f5fcd2afa31622f317f80426a027f30dc792e9c80ffee87b993200ea115f1 \ - --hash=sha256:5be7bf2fb40769e05739dd0046e7b26f9d4670badc7b032d6ce4db64dddc0678 \ - --hash=sha256:60ee7e19e95104d4c03871d7d7dfb3d22ef8a9b9c6778c94e1c8fcc8365afd48 \ - --hash=sha256:61aa400dce22cb001a98014f647dc21cda08f7915ceb95df0c9eaf84b4b6af76 \ - --hash=sha256:68f68d13f2e1cb95163fa3b4db4bf9a159a418f5f6e7242564fc75fcae667fd0 \ - --hash=sha256:7d1f30a86d2757199cb2d56e48cce14deddf1f9c95f1ef1b64ee91ea43fe2e18 \ - --hash=sha256:7d731d4b107030987fd61a7f8ab512b25b53cef8f233a97379ede116f30eb67d \ - --hash=sha256:803812e111e75d1aa73690d2facc295eaefd4439be1023fefc4995eaea2af90d \ - --hash=sha256:80a8d7bfdf38f87ca30a5391c0c9ce4ed2926918e017c29ddf643d0ed2778ea1 \ - --hash=sha256:8293f3dea7fc929ef7240796ba231413afa7b68ce38fd21da2995549f5961981 \ - --hash=sha256:8456928655f856c6e1533ff59d5be76578a7157224dbd9ce6872f25055ab9ab7 \ - --hash=sha256:890bcb4abd5a2d3f852196437129eb3667d62630333aacc13dfd470fad3aaa82 \ - --hash=sha256:94a76daa32eb78d61339aff7952ea819b1734b46f73646a07decb40e5b3448e2 \ - --hash=sha256:9f16fbdf4da055efb21c22d81b89f155f02ba420558db21288b3d0035bafd5f4 \ - --hash=sha256:a3d1fae9863299076f05cb8a778c467578262fae09f9dc0ee9b12eb4268ce663 \ - --hash=sha256:a3d507bb6a513ca96ba84443226af944b0f7f47dcc9a399d110cd6146481d24c \ - --hash=sha256:abace499247268e3757271b2f1e244b36b06f8515cf27c4d49468fc9eb16e93d \ - --hash=sha256:ba2a27ff02f48193fc4daeadf8ad2590516fa3d0adeeb34336b96f7fa64c1e3a \ - --hash=sha256:bc84e875994c3b445871ea7181d424588171efec3e185dced958dad9e001950a \ - --hash=sha256:bfd56bb4b37ed4f330b82402f6f435845a5f5648edf1ad497da51a8452d5d62d \ - --hash=sha256:c18ff11e86df2e28854939acde2d003f7984f721eba450b56a200ad90eeb0e6b \ - --hash=sha256:c3bcce8521d785d510b2aad26ae2c966092b7daa8f45dd8f44734a104dc0bc1a \ - --hash=sha256:c4143987a42a2397f2fc3b4d7e3a7d313fbe684f67ff443999e803dd75a76826 \ - --hash=sha256:c69fd885df7d089548a42d5ec05be26050ebcd2283d89b3d30676eb32ff87dee \ - --hash=sha256:ced80795227d70549a411a4ab66e8ce307899fad2220ce5ab2f296e687eacde9 \ - --hash=sha256:d66e421495fdb797610a08f43b05269e0a5ea7f5e652a89bfd5a7d3c1dee3648 \ - --hash=sha256:d861ee9e76ace6cf36a6a89b959ec08e7bc2493ee39d07ffe5acb23ef46d27da \ - --hash=sha256:e9251e3be159d1020c4030bd2e5f84d6a43fe54b6c19c12f51cde9542a2817b2 \ - --hash=sha256:f145bba11b878005c496e93e257c1e88f154d278d2638e6450d17e0f31e558d2 \ - --hash=sha256:fe346b143ff9685e40192a4960938545c699054ba11d4f9029f94751e3f71d87 +cryptography==46.0.6 \ + --hash=sha256:02fad249cb0e090b574e30b276a3da6a149e04ee2f049725b1f69e7b8351ec70 \ + --hash=sha256:063b67749f338ca9c5a0b7fe438a52c25f9526b851e24e6c9310e7195aad3b4d \ + --hash=sha256:12cae594e9473bca1a7aceb90536060643128bb274fcea0fc459ab90f7d1ae7a \ + --hash=sha256:12f0fa16cc247b13c43d56d7b35287ff1569b5b1f4c5e87e92cc4fcc00cd10c0 \ + --hash=sha256:22259338084d6ae497a19bae5d4c66b7ca1387d3264d1c2c0e72d9e9b6a77b97 \ + --hash=sha256:26031f1e5ca62fcb9d1fcb34b2b60b390d1aacaa15dc8b895a9ed00968b97b30 \ + --hash=sha256:27550628a518c5c6c903d84f637fbecf287f6cb9ced3804838a1295dc1fd0759 \ + --hash=sha256:2b417edbe8877cda9022dde3a008e2deb50be9c407eef034aeeb3a8b11d9db3c \ + --hash=sha256:2ea0f37e9a9cf0df2952893ad145fd9627d326a59daec9b0802480fa3bcd2ead \ + --hash=sha256:2ef9e69886cbb137c2aef9772c2e7138dc581fad4fcbcf13cc181eb5a3ab6275 \ + --hash=sha256:341359d6c9e68834e204ceaf25936dffeafea3829ab80e9503860dcc4f4dac58 \ + --hash=sha256:380343e0653b1c9d7e1f55b52aaa2dbb2fdf2730088d48c43ca1c7c0abb7cc2f \ + --hash=sha256:3c21d92ed15e9cfc6eb64c1f5a0326db22ca9c2566ca46d845119b45b4400361 \ + --hash=sha256:3dfa6567f2e9e4c5dceb8ccb5a708158a2a871052fa75c8b78cb0977063f1507 \ + --hash=sha256:456b3215172aeefb9284550b162801d62f5f264a081049a3e94307fe20792cfa \ + --hash=sha256:4668298aef7cddeaf5c6ecc244c2302a2b8e40f384255505c22875eebb47888b \ + --hash=sha256:50575a76e2951fe7dbd1f56d181f8c5ceeeb075e9ff88e7ad997d2f42af06e7b \ + --hash=sha256:639301950939d844a9e1c4464d7e07f902fe9a7f6b215bb0d4f28584729935d8 \ + --hash=sha256:64235194bad039a10bb6d2d930ab3323baaec67e2ce36215fd0952fad0930ca8 \ + --hash=sha256:6617f67b1606dfd9fe4dbfa354a9508d4a6d37afe30306fe6c101b7ce3274b72 \ + --hash=sha256:67177e8a9f421aa2d3a170c3e56eca4e0128883cf52a071a7cbf53297f18b175 \ + --hash=sha256:6728c49e3b2c180ef26f8e9f0a883a2c585638db64cf265b49c9ba10652d430e \ + --hash=sha256:6739d56300662c468fddb0e5e291f9b4d084bead381667b9e654c7dd81705124 \ + --hash=sha256:69cf0056d6947edc6e6760e5f17afe4bea06b56a9ac8a06de9d2bd6b532d4f3a \ + --hash=sha256:760997a4b950ff00d418398ad73fbc91aa2894b5c1db7ccb45b4f68b42a63b3c \ + --hash=sha256:79e865c642cfc5c0b3eb12af83c35c5aeff4fa5c672dc28c43721c2c9fdd2f0f \ + --hash=sha256:7e6142674f2a9291463e5e150090b95a8519b2fb6e6aaec8917dd8d094ce750d \ + --hash=sha256:7f417f034f91dcec1cb6c5c35b07cdbb2ef262557f701b4ecd803ee8cefed4f4 \ + --hash=sha256:7f6690b6c55e9c5332c0b59b9c8a3fb232ebf059094c17f9019a51e9827df91c \ + --hash=sha256:8927ccfbe967c7df312ade694f987e7e9e22b2425976ddbf28271d7e58845290 \ + --hash=sha256:8ce35b77aaf02f3b59c90b2c8a05c73bac12cea5b4e8f3fbece1f5fddea5f0ca \ + --hash=sha256:8e7304c4f4e9490e11efe56af6713983460ee0780f16c63f219984dab3af9d2d \ + --hash=sha256:90e5f0a7b3be5f40c3a0a0eafb32c681d8d2c181fc2a1bdabe9b3f611d9f6b1a \ + --hash=sha256:97c8115b27e19e592a05c45d0dd89c57f81f841cc9880e353e0d3bf25b2139ed \ + --hash=sha256:9a693028b9cbe51b5a1136232ee8f2bc242e4e19d456ded3fa7c86e43c713b4a \ + --hash=sha256:9a9c42a2723999a710445bc0d974e345c32adfd8d2fac6d8a251fa829ad31cfb \ + --hash=sha256:a3e84d5ec9ba01f8fd03802b2147ba77f0c8f2617b2aff254cedd551844209c8 \ + --hash=sha256:aad75154a7ac9039936d50cf431719a2f8d4ed3d3c277ac03f3339ded1a5e707 \ + --hash=sha256:b12c6b1e1651e42ab5de8b1e00dc3b6354fdfd778e7fa60541ddacc27cd21410 \ + --hash=sha256:b928a3ca837c77a10e81a814a693f2295200adb3352395fad024559b7be7a736 \ + --hash=sha256:bcb87663e1f7b075e48c3be3ecb5f0b46c8fc50b50a97cf264e7f60242dca3f2 \ + --hash=sha256:c797e2517cb7880f8297e2c0f43bb910e91381339336f75d2c1c2cbf811b70b4 \ + --hash=sha256:c89eb37fae9216985d8734c1afd172ba4927f5a05cfd9bf0e4863c6d5465b013 \ + --hash=sha256:cdcd3edcbc5d55757e5f5f3d330dd00007ae463a7e7aa5bf132d1f22a4b62b19 \ + --hash=sha256:d24c13369e856b94892a89ddf70b332e0b70ad4a5c43cf3e9cb71d6d7ffa1f7b \ + --hash=sha256:d4e4aadb7fc1f88687f47ca20bb7227981b03afaae69287029da08096853b738 \ + --hash=sha256:d9528b535a6c4f8ff37847144b8986a9a143585f0540fbcb1a98115b543aa463 \ + --hash=sha256:ed3775295fb91f70b4027aeba878d79b3e55c0b3e97eaa4de71f8f23a9f2eb77 \ + --hash=sha256:ed418c37d095aeddf5336898a132fba01091f0ac5844e3e8018506f014b6d2c4 # via secretstorage docutils==0.22.2 \ --hash=sha256:9fdb771707c8784c8f2728b67cb2c691305933d68137ef95a75db5f4dfbc213d \ diff --git a/tools/publish/requirements_universal.txt b/tools/publish/requirements_universal.txt index 695d4cc386..d907b894ee 100644 --- a/tools/publish/requirements_universal.txt +++ b/tools/publish/requirements_universal.txt @@ -160,56 +160,56 @@ charset-normalizer==3.4.3 \ --hash=sha256:fd10de089bcdcd1be95a2f73dbe6254798ec1bda9f450d5828c96f93e2536b9c \ --hash=sha256:fdabf8315679312cfa71302f9bd509ded4f2f263fb5b765cf1433b39106c3cc9 # via requests -cryptography==46.0.5 ; sys_platform == 'linux' \ - --hash=sha256:02f547fce831f5096c9a567fd41bc12ca8f11df260959ecc7c3202555cc47a72 \ - --hash=sha256:039917b0dc418bb9f6edce8a906572d69e74bd330b0b3fea4f79dab7f8ddd235 \ - --hash=sha256:1abfdb89b41c3be0365328a410baa9df3ff8a9110fb75e7b52e66803ddabc9a9 \ - --hash=sha256:2ae6971afd6246710480e3f15824ed3029a60fc16991db250034efd0b9fb4356 \ - --hash=sha256:2b7a67c9cd56372f3249b39699f2ad479f6991e62ea15800973b956f4b73e257 \ - --hash=sha256:351695ada9ea9618b3500b490ad54c739860883df6c1f555e088eaf25b1bbaad \ - --hash=sha256:38946c54b16c885c72c4f59846be9743d699eee2b69b6988e0a00a01f46a61a4 \ - --hash=sha256:3b4995dc971c9fb83c25aa44cf45f02ba86f71ee600d81091c2f0cbae116b06c \ - --hash=sha256:3ce58ba46e1bc2aac4f7d9290223cead56743fa6ab94a5d53292ffaac6a91614 \ - --hash=sha256:3ee190460e2fbe447175cda91b88b84ae8322a104fc27766ad09428754a618ed \ - --hash=sha256:4108d4c09fbbf2789d0c926eb4152ae1760d5a2d97612b92d508d96c861e4d31 \ - --hash=sha256:420d0e909050490d04359e7fdb5ed7e667ca5c3c402b809ae2563d7e66a92229 \ - --hash=sha256:47fb8a66058b80e509c47118ef8a75d14c455e81ac369050f20ba0d23e77fee0 \ - --hash=sha256:4c3341037c136030cb46e4b1e17b7418ea4cbd9dd207e4a6f3b2b24e0d4ac731 \ - --hash=sha256:4d7e3d356b8cd4ea5aff04f129d5f66ebdc7b6f8eae802b93739ed520c47c79b \ - --hash=sha256:4d8ae8659ab18c65ced284993c2265910f6c9e650189d4e3f68445ef82a810e4 \ - --hash=sha256:4e817a8920bfbcff8940ecfd60f23d01836408242b30f1a708d93198393a80b4 \ - --hash=sha256:50bfb6925eff619c9c023b967d5b77a54e04256c4281b0e21336a130cd7fc263 \ - --hash=sha256:556e106ee01aa13484ce9b0239bca667be5004efb0aabbed28d353df86445595 \ - --hash=sha256:582f5fcd2afa31622f317f80426a027f30dc792e9c80ffee87b993200ea115f1 \ - --hash=sha256:5be7bf2fb40769e05739dd0046e7b26f9d4670badc7b032d6ce4db64dddc0678 \ - --hash=sha256:60ee7e19e95104d4c03871d7d7dfb3d22ef8a9b9c6778c94e1c8fcc8365afd48 \ - --hash=sha256:61aa400dce22cb001a98014f647dc21cda08f7915ceb95df0c9eaf84b4b6af76 \ - --hash=sha256:68f68d13f2e1cb95163fa3b4db4bf9a159a418f5f6e7242564fc75fcae667fd0 \ - --hash=sha256:7d1f30a86d2757199cb2d56e48cce14deddf1f9c95f1ef1b64ee91ea43fe2e18 \ - --hash=sha256:7d731d4b107030987fd61a7f8ab512b25b53cef8f233a97379ede116f30eb67d \ - --hash=sha256:803812e111e75d1aa73690d2facc295eaefd4439be1023fefc4995eaea2af90d \ - --hash=sha256:80a8d7bfdf38f87ca30a5391c0c9ce4ed2926918e017c29ddf643d0ed2778ea1 \ - --hash=sha256:8293f3dea7fc929ef7240796ba231413afa7b68ce38fd21da2995549f5961981 \ - --hash=sha256:8456928655f856c6e1533ff59d5be76578a7157224dbd9ce6872f25055ab9ab7 \ - --hash=sha256:890bcb4abd5a2d3f852196437129eb3667d62630333aacc13dfd470fad3aaa82 \ - --hash=sha256:94a76daa32eb78d61339aff7952ea819b1734b46f73646a07decb40e5b3448e2 \ - --hash=sha256:9f16fbdf4da055efb21c22d81b89f155f02ba420558db21288b3d0035bafd5f4 \ - --hash=sha256:a3d1fae9863299076f05cb8a778c467578262fae09f9dc0ee9b12eb4268ce663 \ - --hash=sha256:a3d507bb6a513ca96ba84443226af944b0f7f47dcc9a399d110cd6146481d24c \ - --hash=sha256:abace499247268e3757271b2f1e244b36b06f8515cf27c4d49468fc9eb16e93d \ - --hash=sha256:ba2a27ff02f48193fc4daeadf8ad2590516fa3d0adeeb34336b96f7fa64c1e3a \ - --hash=sha256:bc84e875994c3b445871ea7181d424588171efec3e185dced958dad9e001950a \ - --hash=sha256:bfd56bb4b37ed4f330b82402f6f435845a5f5648edf1ad497da51a8452d5d62d \ - --hash=sha256:c18ff11e86df2e28854939acde2d003f7984f721eba450b56a200ad90eeb0e6b \ - --hash=sha256:c3bcce8521d785d510b2aad26ae2c966092b7daa8f45dd8f44734a104dc0bc1a \ - --hash=sha256:c4143987a42a2397f2fc3b4d7e3a7d313fbe684f67ff443999e803dd75a76826 \ - --hash=sha256:c69fd885df7d089548a42d5ec05be26050ebcd2283d89b3d30676eb32ff87dee \ - --hash=sha256:ced80795227d70549a411a4ab66e8ce307899fad2220ce5ab2f296e687eacde9 \ - --hash=sha256:d66e421495fdb797610a08f43b05269e0a5ea7f5e652a89bfd5a7d3c1dee3648 \ - --hash=sha256:d861ee9e76ace6cf36a6a89b959ec08e7bc2493ee39d07ffe5acb23ef46d27da \ - --hash=sha256:e9251e3be159d1020c4030bd2e5f84d6a43fe54b6c19c12f51cde9542a2817b2 \ - --hash=sha256:f145bba11b878005c496e93e257c1e88f154d278d2638e6450d17e0f31e558d2 \ - --hash=sha256:fe346b143ff9685e40192a4960938545c699054ba11d4f9029f94751e3f71d87 +cryptography==46.0.6 ; sys_platform == 'linux' \ + --hash=sha256:02fad249cb0e090b574e30b276a3da6a149e04ee2f049725b1f69e7b8351ec70 \ + --hash=sha256:063b67749f338ca9c5a0b7fe438a52c25f9526b851e24e6c9310e7195aad3b4d \ + --hash=sha256:12cae594e9473bca1a7aceb90536060643128bb274fcea0fc459ab90f7d1ae7a \ + --hash=sha256:12f0fa16cc247b13c43d56d7b35287ff1569b5b1f4c5e87e92cc4fcc00cd10c0 \ + --hash=sha256:22259338084d6ae497a19bae5d4c66b7ca1387d3264d1c2c0e72d9e9b6a77b97 \ + --hash=sha256:26031f1e5ca62fcb9d1fcb34b2b60b390d1aacaa15dc8b895a9ed00968b97b30 \ + --hash=sha256:27550628a518c5c6c903d84f637fbecf287f6cb9ced3804838a1295dc1fd0759 \ + --hash=sha256:2b417edbe8877cda9022dde3a008e2deb50be9c407eef034aeeb3a8b11d9db3c \ + --hash=sha256:2ea0f37e9a9cf0df2952893ad145fd9627d326a59daec9b0802480fa3bcd2ead \ + --hash=sha256:2ef9e69886cbb137c2aef9772c2e7138dc581fad4fcbcf13cc181eb5a3ab6275 \ + --hash=sha256:341359d6c9e68834e204ceaf25936dffeafea3829ab80e9503860dcc4f4dac58 \ + --hash=sha256:380343e0653b1c9d7e1f55b52aaa2dbb2fdf2730088d48c43ca1c7c0abb7cc2f \ + --hash=sha256:3c21d92ed15e9cfc6eb64c1f5a0326db22ca9c2566ca46d845119b45b4400361 \ + --hash=sha256:3dfa6567f2e9e4c5dceb8ccb5a708158a2a871052fa75c8b78cb0977063f1507 \ + --hash=sha256:456b3215172aeefb9284550b162801d62f5f264a081049a3e94307fe20792cfa \ + --hash=sha256:4668298aef7cddeaf5c6ecc244c2302a2b8e40f384255505c22875eebb47888b \ + --hash=sha256:50575a76e2951fe7dbd1f56d181f8c5ceeeb075e9ff88e7ad997d2f42af06e7b \ + --hash=sha256:639301950939d844a9e1c4464d7e07f902fe9a7f6b215bb0d4f28584729935d8 \ + --hash=sha256:64235194bad039a10bb6d2d930ab3323baaec67e2ce36215fd0952fad0930ca8 \ + --hash=sha256:6617f67b1606dfd9fe4dbfa354a9508d4a6d37afe30306fe6c101b7ce3274b72 \ + --hash=sha256:67177e8a9f421aa2d3a170c3e56eca4e0128883cf52a071a7cbf53297f18b175 \ + --hash=sha256:6728c49e3b2c180ef26f8e9f0a883a2c585638db64cf265b49c9ba10652d430e \ + --hash=sha256:6739d56300662c468fddb0e5e291f9b4d084bead381667b9e654c7dd81705124 \ + --hash=sha256:69cf0056d6947edc6e6760e5f17afe4bea06b56a9ac8a06de9d2bd6b532d4f3a \ + --hash=sha256:760997a4b950ff00d418398ad73fbc91aa2894b5c1db7ccb45b4f68b42a63b3c \ + --hash=sha256:79e865c642cfc5c0b3eb12af83c35c5aeff4fa5c672dc28c43721c2c9fdd2f0f \ + --hash=sha256:7e6142674f2a9291463e5e150090b95a8519b2fb6e6aaec8917dd8d094ce750d \ + --hash=sha256:7f417f034f91dcec1cb6c5c35b07cdbb2ef262557f701b4ecd803ee8cefed4f4 \ + --hash=sha256:7f6690b6c55e9c5332c0b59b9c8a3fb232ebf059094c17f9019a51e9827df91c \ + --hash=sha256:8927ccfbe967c7df312ade694f987e7e9e22b2425976ddbf28271d7e58845290 \ + --hash=sha256:8ce35b77aaf02f3b59c90b2c8a05c73bac12cea5b4e8f3fbece1f5fddea5f0ca \ + --hash=sha256:8e7304c4f4e9490e11efe56af6713983460ee0780f16c63f219984dab3af9d2d \ + --hash=sha256:90e5f0a7b3be5f40c3a0a0eafb32c681d8d2c181fc2a1bdabe9b3f611d9f6b1a \ + --hash=sha256:97c8115b27e19e592a05c45d0dd89c57f81f841cc9880e353e0d3bf25b2139ed \ + --hash=sha256:9a693028b9cbe51b5a1136232ee8f2bc242e4e19d456ded3fa7c86e43c713b4a \ + --hash=sha256:9a9c42a2723999a710445bc0d974e345c32adfd8d2fac6d8a251fa829ad31cfb \ + --hash=sha256:a3e84d5ec9ba01f8fd03802b2147ba77f0c8f2617b2aff254cedd551844209c8 \ + --hash=sha256:aad75154a7ac9039936d50cf431719a2f8d4ed3d3c277ac03f3339ded1a5e707 \ + --hash=sha256:b12c6b1e1651e42ab5de8b1e00dc3b6354fdfd778e7fa60541ddacc27cd21410 \ + --hash=sha256:b928a3ca837c77a10e81a814a693f2295200adb3352395fad024559b7be7a736 \ + --hash=sha256:bcb87663e1f7b075e48c3be3ecb5f0b46c8fc50b50a97cf264e7f60242dca3f2 \ + --hash=sha256:c797e2517cb7880f8297e2c0f43bb910e91381339336f75d2c1c2cbf811b70b4 \ + --hash=sha256:c89eb37fae9216985d8734c1afd172ba4927f5a05cfd9bf0e4863c6d5465b013 \ + --hash=sha256:cdcd3edcbc5d55757e5f5f3d330dd00007ae463a7e7aa5bf132d1f22a4b62b19 \ + --hash=sha256:d24c13369e856b94892a89ddf70b332e0b70ad4a5c43cf3e9cb71d6d7ffa1f7b \ + --hash=sha256:d4e4aadb7fc1f88687f47ca20bb7227981b03afaae69287029da08096853b738 \ + --hash=sha256:d9528b535a6c4f8ff37847144b8986a9a143585f0540fbcb1a98115b543aa463 \ + --hash=sha256:ed3775295fb91f70b4027aeba878d79b3e55c0b3e97eaa4de71f8f23a9f2eb77 \ + --hash=sha256:ed418c37d095aeddf5336898a132fba01091f0ac5844e3e8018506f014b6d2c4 # via secretstorage docutils==0.22.2 \ --hash=sha256:9fdb771707c8784c8f2728b67cb2c691305933d68137ef95a75db5f4dfbc213d \ From 900d55725f3573b29b93212d19eb7f31e13c4546 Mon Sep 17 00:00:00 2001 From: Ignas Anikevicius <240938+aignas@users.noreply.github.com> Date: Thu, 2 Apr 2026 16:24:31 +0900 Subject: [PATCH 676/922] feat(pypi): first check index contents before downloading metadata about distributions (#3657) The overall changes to the architecture are: * First check which packages are on which index. * Then write these details as facts for future reuse. * Then use the `index_url_overrides` to ensure that we are pulling things from the right index for the packages. * Then download everything. Notes on implementation: * This will pull index contents at most once for each index. * This will make the initial download times longer, but because we have `MODULE.bazel.lock` file and the facts written there, it should be OK. * This allows us to just parse the `index_url` and `extra_index_urls` from the lock files and use that without printing any warning messages. If we don't see any regressions in testing, I think this code path could be enabled for everyone by default. So `experimental_index_url` will no longer be experimental. I think this might have been the last thing holding us from flipping the switch. Fixes #2632 Fixes #3260 --- CHANGELOG.md | 18 +- python/private/pypi/BUILD.bazel | 3 +- python/private/pypi/parse_simpleapi_html.bzl | 37 ++- python/private/pypi/pypi_cache.bzl | 54 +++++ python/private/pypi/simpleapi_download.bzl | 220 +++++++++-------- python/private/pypi/urllib.bzl | 2 +- tests/pypi/hub_builder/hub_builder_tests.bzl | 33 ++- .../parse_simpleapi_html_tests.bzl | 27 ++- tests/pypi/pypi_cache/pypi_cache_tests.bzl | 33 +++ .../simpleapi_download_tests.bzl | 226 +++++++++--------- 10 files changed, 401 insertions(+), 252 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 16a591b159..8e7c152acd 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -69,12 +69,18 @@ END_UNRELEASED_TEMPLATE Other changes: * (pypi) Update dependencies used for `compile_pip_requirements`, building sdists in the `whl_library` rule and fetching wheels using `pip`. -* (pypi) We will set `allow_fail` to `False` if the - {attr}`experimental_index_url_overrides` is set - to a non-empty value. This means that failures will be no-longer cached in - this particular case. - ([#3260](https://github.com/bazel-contrib/rules_python/issues/3260) and - [#2632](https://github.com/bazel-contrib/rules_python/issues/2632)) +* (pypi) Before using the bazel downloader to fetch the PyPI package metadata + we will from now on fetch the lists of available packages on each index. The + used package mappings will be written as facts to the `MODULE.bazel.lock` file + on supported bazel versions and it should be done at most once. As a result, + per-package {obj}`experimental_index_url_overrides` is no longer needed if the index URLs are + passed to the `pip.parse` via `experimental_index_url` and `experimental_extra_index_urls`. + What is more, we start implementing the flags for `--index_url` and `--extra_index_urls` more in + line to how it is used in `uv` and `pip`, i.e. we default to `--index_url` if the package is not + found in `--extra_index_urls`. + Fixes + ([#3260](https://github.com/bazel-contrib/rules_python/issues/3260) and + [#2632](https://github.com/bazel-contrib/rules_python/issues/2632)). {#v0-0-0-fixed} ### Fixed diff --git a/python/private/pypi/BUILD.bazel b/python/private/pypi/BUILD.bazel index 6b4822333c..869be4705a 100644 --- a/python/private/pypi/BUILD.bazel +++ b/python/private/pypi/BUILD.bazel @@ -244,6 +244,7 @@ bzl_library( srcs = ["parse_simpleapi_html.bzl"], deps = [ ":version_from_filename_bzl", + "//python/private:normalize_name_bzl", ], ) @@ -424,8 +425,6 @@ bzl_library( ":urllib_bzl", "//python/private:auth_bzl", "//python/private:normalize_name_bzl", - "//python/private:text_util_bzl", - "@bazel_features//:features", ], ) diff --git a/python/private/pypi/parse_simpleapi_html.bzl b/python/private/pypi/parse_simpleapi_html.bzl index 563130791e..7f0d2776d7 100644 --- a/python/private/pypi/parse_simpleapi_html.bzl +++ b/python/private/pypi/parse_simpleapi_html.bzl @@ -16,16 +16,20 @@ Parse SimpleAPI HTML in Starlark. """ +load("//python/private:normalize_name.bzl", "normalize_name") load(":version_from_filename.bzl", "version_from_filename") -def parse_simpleapi_html(*, content): +def parse_simpleapi_html(*, content, parse_index = False): """Get the package URLs for given shas by parsing the Simple API HTML. Args: - content(str): The Simple API HTML content. + content: {type}`str` The Simple API HTML content. + parse_index: {type}`bool` whether to parse the content as the index page of the PyPI index, + e.g. the `https://pypi.org/simple/`. This only has the URLs for the individual package. Returns: - A list of structs with: + If it is the index page, return the map of package to URL it can be queried from. + Otherwise, a list of structs with: * filename: {type}`str` The filename of the artifact. * version: {type}`str` The version of the artifact. * url: {type}`str` The URL to download the artifact. @@ -59,6 +63,8 @@ def parse_simpleapi_html(*, content): # https://packaging.python.org/en/latest/specifications/simple-repository-api/#versioning-pypi-s-simple-api fail("Unsupported API version: {}".format(api_version)) + packages = {} + # 2. Iterate using find() to avoid huge list allocations from .split(" - tag_end = content.find(">", start_tag) - end_tag = content.find("", tag_end) - if tag_end == -1 or end_tag == -1: + # Find the closing tag first, then find the end of the opening + # tag using rfind. This correctly handles attributes that + # contain > characters, e.g. data-requires-python=">=3.6". + end_tag = content.find("", start_tag) + if end_tag == -1: break + tag_end = content.rfind(">", start_tag, end_tag) + if tag_end == -1 or tag_end <= start_tag: + cursor = end_tag + 4 + continue # Extract only the necessary slices - attr_part = content[start_tag + 3:tag_end] filename = content[tag_end + 1:end_tag].strip() + attr_part = content[start_tag + 3:tag_end] # Update cursor for next iteration cursor = end_tag + 4 - # 3. Efficient Attribute Parsing attrs = _parse_attrs(attr_part) href = attrs.get("href", "") if not href: continue + if parse_index: + pkg_name = filename + packages[normalize_name(pkg_name)] = href + continue + + # 3. Efficient Attribute Parsing dist_url, _, sha256 = href.partition("#sha256=") # Handle Yanked status @@ -121,6 +137,9 @@ def parse_simpleapi_html(*, content): else: sdists[sha256] = dist + if parse_index: + return packages + return struct( sdists = sdists, whls = whls, diff --git a/python/private/pypi/pypi_cache.bzl b/python/private/pypi/pypi_cache.bzl index 28c6cbeafb..7b24102263 100644 --- a/python/private/pypi/pypi_cache.bzl +++ b/python/private/pypi/pypi_cache.bzl @@ -89,6 +89,11 @@ def _pypi_cache_get(self, key): if not cached and versions: # Could not get from in-memory, read from lockfile facts cached = self._facts.get(index_url, versions) + else: + # We might be using something from memory that is not yet stored in facts (e.g. we processed + # the requirements.txt for one Python version and the deps got cached, but new python + # version means different deps, which may add extras. + self._facts.setdefault(index_url, cached) return cached @@ -122,6 +127,13 @@ def _filter_packages(dists, requested_versions): if dists == None or not requested_versions: return dists + if type(dists) == "dict": + return { + pkg: url + for pkg, url in dists.items() + if pkg in requested_versions + } + sha256s_by_version = {} whls = {} sdists = {} @@ -193,6 +205,12 @@ def _get_from_facts(facts, known_facts, index_url, requested_versions, facts_ver # cannot trust known facts, different version that we know how to parse return None + if type(requested_versions) == "dict": + return _filter_packages( + dists = known_facts.get("index_urls", {}).get(index_url, {}), + requested_versions = requested_versions, + ) + known_sources = {} root_url, _, distribution = index_url.rstrip("/").rpartition("/") @@ -266,10 +284,46 @@ def _store_facts(facts, fact_version, index_url, value): facts["fact_version"] = fact_version + if type(value) == "dict": + # facts: { + # "index_urls": { + # "": { + # "": "", + # }, + # }, + # }, + for pkg, url in value.items(): + facts.setdefault("index_urls", {}).setdefault(index_url, {})[pkg] = url + return value + root_url, _, distribution = index_url.rstrip("/").rpartition("/") distribution = distribution.rstrip("/") root_url = root_url.rstrip("/") + # The schema is + # facts: { + # "dist_hashes": { + # "": { + # "": { + # "": "", + # }, + # }, + # }, + # "dist_filenames": { + # "": { + # "": { + # "": "", # if it is different from the URL + # }, + # }, + # }, + # "dist_yanked": { + # "": { + # "": { + # "": "", # if the package is yanked + # }, + # }, + # }, + # }, for sha256, d in (value.sdists | value.whls).items(): facts.setdefault("dist_hashes", {}).setdefault(root_url, {}).setdefault(distribution, {}).setdefault(d.url, sha256) if not d.url.endswith(d.filename): diff --git a/python/private/pypi/simpleapi_download.bzl b/python/private/pypi/simpleapi_download.bzl index 20d79ba9b4..2171e8b56a 100644 --- a/python/private/pypi/simpleapi_download.bzl +++ b/python/private/pypi/simpleapi_download.bzl @@ -16,11 +16,9 @@ A file that houses private functions used in the `bzlmod` extension with the same name. """ -load("@bazel_features//:features.bzl", "bazel_features") load("//python/private:auth.bzl", _get_auth = "get_auth") load("//python/private:envsubst.bzl", "envsubst") load("//python/private:normalize_name.bzl", "normalize_name") -load("//python/private:text_util.bzl", "render") load(":parse_simpleapi_html.bzl", "parse_simpleapi_html") load(":urllib.bzl", "urllib") @@ -35,15 +33,22 @@ def simpleapi_download( _fail = fail): """Download Simple API HTML. + First it queries all of the indexes for available packages and then it downloads the contents of + the per-package URLs and sha256 values. This is to enable us to use bazel_downloader with + `requirements.txt` files. As a side effect we also are able to "cross-compile" by fetching the + right wheel for the right target platform through the information that we retrieve here. + Args: ctx: The module_ctx or repository_ctx. attr: Contains the parameters for the download. They are grouped into a struct for better clarity. It must have attributes: - * index_url: str, the index. + * index_url: str, the index, or if `extra_index_urls` are passed, the default index. * index_url_overrides: dict[str, str], the index overrides for separate packages. - * extra_index_urls: Extra index URLs that will be looked up after - the main is looked up. + * extra_index_urls: Will be looked at in the order they are defined and the first match + wins. This is similar to what uv does, see + https://docs.astral.sh/uv/concepts/indexes/#searching-across-multiple-indexes. + PRs for implementing other strategies are welcome. * sources: list[str], the sources to download things for. Each value is the contents of requirements files. * envsubst: list[str], the envsubst vars for performing substitution in index url. @@ -70,111 +75,119 @@ def simpleapi_download( normalize_name(p): i for p, i in (attr.index_url_overrides or {}).items() } + sources = { + normalize_name(pkg): versions + for pkg, versions in attr.sources.items() + } - # NOTE @aignas 2024-03-31: we are not merging results from multiple indexes - # to replicate how `pip` would handle this case. - contents = {} - index_urls = [attr.index_url] + attr.extra_index_urls read_simpleapi = read_simpleapi or _read_simpleapi - download_kwargs = {} - if bazel_features.external_deps.download_has_block_param: - download_kwargs["block"] = not parallel_download + ctx.report_progress("Fetch package lists from PyPI index") - if len(index_urls) == 1 or index_url_overrides: - download_kwargs["allow_fail"] = False - else: - download_kwargs["allow_fail"] = True + # NOTE: we are not merging results from multiple indexes to replicate how `pip` would + # handle this case. What we do is we select a particular index to download the packages + dist_urls = _get_dist_urls( + ctx, + default_index = attr.index_url, + index_urls = attr.extra_index_urls, + index_url_overrides = index_url_overrides, + sources = sources, + read_simpleapi = read_simpleapi, + cache = cache, + get_auth = get_auth, + attr = attr, + block = not parallel_download, + _fail = _fail, + ) - input_sources = attr.sources + ctx.report_progress("Fetching package URLs from PyPI index") - found_on_index = {} - warn_overrides = False - ctx.report_progress("Fetch package lists from PyPI index") - for i, index_url in enumerate(index_urls): - if i != 0: - # Warn the user about a potential fix for the overrides - warn_overrides = True - - async_downloads = {} - sources = {pkg: versions for pkg, versions in input_sources.items() if pkg not in found_on_index} - for pkg, versions in sources.items(): - pkg_normalized = normalize_name(pkg) - url = urllib.strip_empty_path_segments("{index_url}/{distribution}/".format( - index_url = index_url_overrides.get(pkg_normalized, index_url).rstrip("/"), - distribution = pkg, - )) - result = read_simpleapi( - ctx = ctx, - attr = attr, - versions = versions, - url = url, - cache = cache, - get_auth = get_auth, - **download_kwargs - ) - if hasattr(result, "wait"): - # We will process it in a separate loop: - async_downloads[pkg] = struct( - pkg_normalized = pkg_normalized, - wait = result.wait, - url = url, - ) - elif result.success: - contents[pkg_normalized] = _with_index_url(url, result.output) - found_on_index[pkg] = index_url - - if not async_downloads: - continue + downloads = {} + contents = {} + for pkg, url in dist_urls.items(): + result = read_simpleapi( + ctx = ctx, + attr = attr, + url = url, + cache = cache, + versions = sources[pkg], + get_auth = get_auth, + block = not parallel_download, + parse_index = False, + ) + if hasattr(result, "wait"): + # We will process it in a separate loop: + downloads[pkg] = result + else: + contents[pkg] = _with_index_url(url, result.output) + for pkg, d in downloads.items(): # If we use `block` == False, then we need to have a second loop that is # collecting all of the results as they were being downloaded in parallel. - for pkg, download in async_downloads.items(): - result = download.wait() - - if result.success: - contents[download.pkg_normalized] = _with_index_url(download.url, result.output) - found_on_index[pkg] = index_url - - failed_sources = [pkg for pkg in input_sources if pkg not in found_on_index] - if failed_sources: - pkg_index_urls = { - pkg: index_url_overrides.get( - normalize_name(pkg), - index_urls, - ) - for pkg in failed_sources - } - - _fail( - """ -Failed to download metadata of the following packages from urls: -{pkg_index_urls} - -If you would like to skip downloading metadata for these packages please add 'simpleapi_skip={failed_sources}' to your 'pip.parse' call. -""".format( - pkg_index_urls = render.dict(pkg_index_urls), - failed_sources = render.list(failed_sources), - ), + contents[pkg] = _with_index_url(dist_urls[pkg], d.wait().output) + + return contents + +def _get_dist_urls(ctx, *, default_index, index_urls, index_url_overrides, sources, read_simpleapi, attr, block, _fail = fail, **kwargs): + downloads = {} + results = {} + for extra in index_url_overrides.values(): + if extra not in index_urls: + index_urls.append(extra) + + index_urls = index_urls or [] + if default_index not in index_urls: + index_urls.append(default_index) + + for index_url in index_urls: + download = read_simpleapi( + ctx = ctx, + attr = attr, + url = urllib.strip_empty_path_segments("{index_url}/".format( + index_url = index_url, + )), + parse_index = True, + versions = {pkg: None for pkg in sources}, + block = block, + **kwargs ) - return None - - if warn_overrides: - index_url_overrides = { - pkg: found_on_index[pkg] - for pkg in attr.sources - if found_on_index[pkg] != attr.index_url - } - - if index_url_overrides: - # buildifier: disable=print - print("You can use the following `index_url_overrides` to avoid the 404 warnings:\n{}".format( - render.dict(index_url_overrides), + if hasattr(download, "wait"): + downloads[index_url] = download + else: + results[index_url] = download + + for index_url, download in downloads.items(): + results[index_url] = download.wait() + + found_on_index = {} + for index_url, result in results.items(): + for pkg in sources: + if pkg in found_on_index: + # We have already found the package, skip searching for it in + # other indexes. + # + # If we wanted to merge all of the index results, we would have to continue here + # and in the outer function process merging of the results. + continue + + if index_url_overrides.get(pkg, index_url) != index_url: + # we should not use this index for the package + continue + + found = result.output.get(pkg) + if not found: + continue + + # Ignore the URL here because we know how to construct it. + + found_on_index[pkg] = urllib.strip_empty_path_segments("{}/{}/".format( + index_url, + pkg.replace("_", "-"), # Use the official normalization for URLs )) - return contents + return found_on_index -def _read_simpleapi(ctx, url, attr, cache, versions, get_auth = None, **download_kwargs): +def _read_simpleapi(ctx, url, attr, cache, versions, parse_index, get_auth = None, **download_kwargs): """Read SimpleAPI. Args: @@ -189,6 +202,8 @@ def _read_simpleapi(ctx, url, attr, cache, versions, get_auth = None, **download cache: {type}`struct` the `pypi_cache` instance. versions: {type}`list[str] The versions that have been requested. get_auth: A function to get auth information. Used in tests. + parse_index: {type}`bool` Whether to parse the content as a root index page + (e.g. `/simple/`) instead of a package-specific page. **download_kwargs: Any extra params to ctx.download. Note that output and auth will be passed for you. @@ -196,11 +211,6 @@ def _read_simpleapi(ctx, url, attr, cache, versions, get_auth = None, **download A similar object to what `download` would return except that in result.out will be the parsed simple api contents. """ - # NOTE @aignas 2024-03-31: some of the simple APIs use relative URLs for - # the whl location and we cannot handle multiple URLs at once by passing - # them to ctx.download if we want to correctly handle the relative URLs. - # TODO: Add a test that env subbed index urls do not leak into the lock file. - real_url = urllib.strip_empty_path_segments(envsubst(url, attr.envsubst, ctx.getenv)) cache_key = (url, real_url, versions) @@ -242,6 +252,7 @@ def _read_simpleapi(ctx, url, attr, cache, versions, get_auth = None, **download output = output, cache = cache, cache_key = cache_key, + parse_index = parse_index, ), ) @@ -251,15 +262,16 @@ def _read_simpleapi(ctx, url, attr, cache, versions, get_auth = None, **download output = output, cache = cache, cache_key = cache_key, + parse_index = parse_index, ) -def _read_index_result(ctx, *, result, output, cache, cache_key): +def _read_index_result(ctx, *, result, output, cache, cache_key, parse_index): if not result.success: return struct(success = False) content = ctx.read(output) - output = parse_simpleapi_html(content = content) + output = parse_simpleapi_html(content = content, parse_index = parse_index) if output: cache.setdefault(cache_key, output) return struct(success = True, output = output) diff --git a/python/private/pypi/urllib.bzl b/python/private/pypi/urllib.bzl index ca6ded76b1..ea4cd32cc9 100644 --- a/python/private/pypi/urllib.bzl +++ b/python/private/pypi/urllib.bzl @@ -3,7 +3,7 @@ def _get_root_directory(url): scheme_end = url.find("://") if scheme_end == -1: - fail("Invalid URL format") + fail("Invalid URL format: '{}'".format(url)) scheme = url[:scheme_end] host_end = url.find("/", scheme_end + 3) diff --git a/tests/pypi/hub_builder/hub_builder_tests.bzl b/tests/pypi/hub_builder/hub_builder_tests.bzl index 637c7881c2..31a41f6af5 100644 --- a/tests/pypi/hub_builder/hub_builder_tests.bzl +++ b/tests/pypi/hub_builder/hub_builder_tests.bzl @@ -247,12 +247,19 @@ def _test_simple_extras_vs_no_extras(env): _tests.append(_test_simple_extras_vs_no_extras) def _test_simple_extras_vs_no_extras_simpleapi(env): - def mockread_simpleapi(*_, **__): + def mockread_simpleapi(*_, parse_index, **__): + if parse_index: + content = """\ + simple-0.0.1-py3-none-any.whl
+""" return struct( output = parse_simpleapi_html( - content = """\ - simple-0.0.1-py3-none-any.whl
-""", + content = content, + parse_index = parse_index, ), success = True, ) @@ -489,10 +496,13 @@ def _test_simple_with_markers(env): _tests.append(_test_simple_with_markers) def _test_torch_experimental_index_url(env): - def mockread_simpleapi(*_, **__): - return struct( - output = parse_simpleapi_html( - content = """\ + def mockread_simpleapi(*_, parse_index, **__): + if parse_index: + content = """\ + torch +""" + else: + content = """\ torch-2.4.1+cpu-cp310-cp310-linux_x86_64.whl
torch-2.4.1+cpu-cp310-cp310-win_amd64.whl
torch-2.4.1+cpu-cp311-cp311-linux_x86_64.whl
@@ -513,7 +523,12 @@ def _test_torch_experimental_index_url(env): torch-2.4.1-cp38-none-macosx_11_0_arm64.whl
torch-2.4.1-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
torch-2.4.1-cp39-none-macosx_11_0_arm64.whl
-""", +""" + + return struct( + output = parse_simpleapi_html( + content = content, + parse_index = parse_index, ), success = True, ) diff --git a/tests/pypi/parse_simpleapi_html/parse_simpleapi_html_tests.bzl b/tests/pypi/parse_simpleapi_html/parse_simpleapi_html_tests.bzl index f72d61371c..c84140f459 100644 --- a/tests/pypi/parse_simpleapi_html/parse_simpleapi_html_tests.bzl +++ b/tests/pypi/parse_simpleapi_html/parse_simpleapi_html_tests.bzl @@ -42,6 +42,29 @@ def _generate_html(*items): ]), ) +def _test_index(env): + # buildifier: disable=unsorted-dict-items + tests = [ + ( + [ + struct(attrs = ['href="/simple/foo/"'], filename = "foo"), + struct(attrs = ['href="./b-ar/"'], filename = "b-._.-aR"), + ], + { + "b_ar": "./b-ar/", + "foo": "/simple/foo/", + }, + ), + ] + + for (input, want) in tests: + html = _generate_html(*input) + got = parse_simpleapi_html(content = html, parse_index = True) + + env.expect.that_dict(got).contains_exactly(want) + +_tests.append(_test_index) + def _test_sdist(env): # buildifier: disable=unsorted-dict-items tests = [ @@ -65,7 +88,7 @@ def _test_sdist(env): struct( attrs = [ 'href="https://example.org/full-url/foo-0.0.1.tar.gz#sha256=deadbeefasource"', - 'data-requires-python=">=3.7"', + 'data-requires-python=">=3.7"', "data-yanked", ], filename = "foo-0.0.1.tar.gz", @@ -82,7 +105,7 @@ def _test_sdist(env): struct( attrs = [ 'href="https://example.org/full-url/foo-0.0.1.tar.gz#sha256=deadbeefasource"', - 'data-requires-python=">=3.7"', + 'data-requires-python="<=3.7"', "data-yanked=\"Something with "quotes" over two lines\"", ], filename = "foo-0.0.1.tar.gz", diff --git a/tests/pypi/pypi_cache/pypi_cache_tests.bzl b/tests/pypi/pypi_cache/pypi_cache_tests.bzl index 7b6168ce7b..3cf01c7450 100644 --- a/tests/pypi/pypi_cache/pypi_cache_tests.bzl +++ b/tests/pypi/pypi_cache/pypi_cache_tests.bzl @@ -155,6 +155,39 @@ def _test_pypi_cache_writes_to_facts(env): "fact_version": "v1", # Facts version }) + # When we get the other items cached in memory, they get written to facts + got = cache.get((key[0], key[1], ["1.1.0"])) + got.whls().contains_exactly({ + "sha_whl_2": fake_result.whls["sha_whl_2"], + }) + got.sdists().contains_exactly({}) + got.sha256s_by_version().contains_exactly({ + "1.1.0": fake_result.sha256s_by_version["1.1.0"], + }) + + # Then when we get facts at the end + cache.get_facts().contains_exactly({ + "dist_hashes": { + # We are not using the real index URL, because we may have credentials in here + "https://{PYPI_INDEX_URL}": { + "pkg": { + "https://pypi.org/files/pkg-1.0.0-py3-none-any.whl": "sha_whl", + "https://pypi.org/files/pkg-1.0.0.tar.gz": "sha_sdist", + "https://pypi.org/files/pkg-1.1.0-py3-none-any.whl": "sha_whl_2", + }, + }, + }, + "dist_yanked": { + "https://{PYPI_INDEX_URL}": { + "pkg": { + "sha_sdist": "", + "sha_whl": "Security issue", + }, + }, + }, + "fact_version": "v1", # Facts version + }) + _tests.append(_test_pypi_cache_writes_to_facts) def _test_pypi_cache_reads_from_facts(env): diff --git a/tests/pypi/simpleapi_download/simpleapi_download_tests.bzl b/tests/pypi/simpleapi_download/simpleapi_download_tests.bzl index 9a6b7ca5af..55439c2593 100644 --- a/tests/pypi/simpleapi_download/simpleapi_download_tests.bzl +++ b/tests/pypi/simpleapi_download/simpleapi_download_tests.bzl @@ -23,26 +23,30 @@ _tests = [] def _test_simple(env): calls = [] - def read_simpleapi(ctx, url, versions, attr, cache, get_auth, block, allow_fail): - _ = ctx, attr, cache, get_auth, versions # buildifier: disable=unused-variable - env.expect.that_bool(block).equals(False) - env.expect.that_bool(allow_fail).equals(True) - calls.append(url) - if "foo" in url and "main" in url: - return struct( - output = "", - success = False, - ) - else: + def read_simpleapi(ctx, url, versions, attr, cache, get_auth, block, parse_index): + if parse_index: return struct( - output = struct( - sdists = {"deadbeef": url.strip("/").split("/")[-1]}, - whls = {"deadb33f": url.strip("/").split("/")[-1]}, - sha256s_by_version = {"fizz": url.strip("/").split("/")[-1]}, - ), success = True, + output = { + "bar": "/bar/", + "baz": "/baz/", + } if "main" in url else { + "foo": "/foo/", + }, ) + _ = ctx, attr, cache, get_auth, versions # buildifier: disable=unused-variable + env.expect.that_bool(block).equals(False) + calls.append(url) + return struct( + output = struct( + sdists = {"deadbeef": url.strip("/").split("/")[-1]}, + whls = {"deadb33f": url.strip("/").split("/")[-1]}, + sha256s_by_version = {"fizz": url.strip("/").split("/")[-1]}, + ), + success = True, + ) + contents = simpleapi_download( ctx = struct( getenv = {}.get, @@ -50,8 +54,8 @@ def _test_simple(env): ), attr = struct( index_url_overrides = {}, - index_url = "main", - extra_index_urls = ["extra"], + index_url = "https://main.com", + extra_index_urls = ["https://extra.com"], sources = {"bar": None, "baz": None, "foo": None}, envsubst = [], ), @@ -61,26 +65,25 @@ def _test_simple(env): ) env.expect.that_collection(calls).contains_exactly([ - "extra/foo/", - "main/bar/", - "main/baz/", - "main/foo/", + "https://extra.com/foo/", + "https://main.com/bar/", + "https://main.com/baz/", ]) env.expect.that_dict(contents).contains_exactly({ "bar": struct( - index_url = "main/bar/", + index_url = "https://main.com/bar/", sdists = {"deadbeef": "bar"}, sha256s_by_version = {"fizz": "bar"}, whls = {"deadb33f": "bar"}, ), "baz": struct( - index_url = "main/baz/", + index_url = "https://main.com/baz/", sdists = {"deadbeef": "baz"}, sha256s_by_version = {"fizz": "baz"}, whls = {"deadb33f": "baz"}, ), "foo": struct( - index_url = "extra/foo/", + index_url = "https://extra.com/foo/", sdists = {"deadbeef": "foo"}, sha256s_by_version = {"fizz": "foo"}, whls = {"deadb33f": "foo"}, @@ -89,85 +92,26 @@ def _test_simple(env): _tests.append(_test_simple) -def _test_fail(env): +def _test_index_overrides(env): calls = [] fails = [] - def read_simpleapi(ctx, url, versions, attr, cache, get_auth, block, allow_fail): - _ = ctx, attr, cache, get_auth, versions # buildifier: disable=unused-variable - env.expect.that_bool(block).equals(False) - env.expect.that_bool(allow_fail).equals(True) - calls.append(url) - if "foo" in url: - return struct( - output = "", - success = False, - ) - if "bar" in url: - return struct( - output = "", - success = False, - ) - else: + def read_simpleapi(ctx, *, url, versions, attr, cache, get_auth, block, parse_index): + if parse_index: return struct( - output = struct( - sdists = {}, - whls = {}, - sha256s_by_version = {}, - ), success = True, + output = { + # normalized + "ba_z": "/ba-z/", + "bar": "/bar/", + "foo": "/foo-should-fail/", + } if "main" in url else { + "foo": "/foo/", + }, ) - simpleapi_download( - ctx = struct( - getenv = {}.get, - report_progress = lambda _: None, - ), - attr = struct( - index_url_overrides = {}, - index_url = "main", - extra_index_urls = ["extra"], - sources = {"bar": None, "baz": None, "foo": None}, - envsubst = [], - ), - cache = pypi_cache(), - parallel_download = True, - read_simpleapi = read_simpleapi, - _fail = fails.append, - ) - - env.expect.that_collection(fails).contains_exactly([ - """ -Failed to download metadata of the following packages from urls: -{ - "bar": ["main", "extra"], - "foo": ["main", "extra"], -} - -If you would like to skip downloading metadata for these packages please add 'simpleapi_skip=[ - "bar", - "foo", -]' to your 'pip.parse' call. -""", - ]) - env.expect.that_collection(calls).contains_exactly([ - "main/foo/", - "main/bar/", - "main/baz/", - "extra/foo/", - "extra/bar/", - ]) - -_tests.append(_test_fail) - -def _test_allow_fail_single_index(env): - calls = [] - fails = [] - - def read_simpleapi(ctx, *, url, versions, attr, cache, get_auth, block, allow_fail): _ = ctx, attr, cache, get_auth, versions # buildifier: disable=unused-variable env.expect.that_bool(block).equals(False) - env.expect.that_bool(allow_fail).equals(False) calls.append(url) return struct( output = struct( @@ -185,11 +129,11 @@ def _test_allow_fail_single_index(env): ), attr = struct( index_url_overrides = { - "foo": "extra", + "foo": "https://extra.com", }, - index_url = "main", + index_url = "https://main.com", extra_index_urls = [], - sources = {"bar": None, "baz": None, "foo": None}, + sources = {"ba_z": None, "bar": None, "foo": None}, envsubst = [], ), cache = pypi_cache(), @@ -200,35 +144,46 @@ def _test_allow_fail_single_index(env): env.expect.that_collection(fails).contains_exactly([]) env.expect.that_collection(calls).contains_exactly([ - "main/bar/", - "main/baz/", - "extra/foo/", + "https://main.com/bar/", + "https://main.com/ba-z/", + "https://extra.com/foo/", ]) env.expect.that_dict(contents).contains_exactly({ + "ba_z": struct( + index_url = "https://main.com/ba-z/", + sdists = {"deadbeef": "ba-z"}, + sha256s_by_version = {"fizz": "ba-z"}, + whls = {"deadb33f": "ba-z"}, + ), "bar": struct( - index_url = "main/bar/", + index_url = "https://main.com/bar/", sdists = {"deadbeef": "bar"}, sha256s_by_version = {"fizz": "bar"}, whls = {"deadb33f": "bar"}, ), - "baz": struct( - index_url = "main/baz/", - sdists = {"deadbeef": "baz"}, - sha256s_by_version = {"fizz": "baz"}, - whls = {"deadb33f": "baz"}, - ), "foo": struct( - index_url = "extra/foo/", + index_url = "https://extra.com/foo/", sdists = {"deadbeef": "foo"}, sha256s_by_version = {"fizz": "foo"}, whls = {"deadb33f": "foo"}, ), }) -_tests.append(_test_allow_fail_single_index) +_tests.append(_test_index_overrides) def _test_download_url(env): downloads = {} + reads = [ + # The first read is the index which seeds the downloads later + """ + bar + baz + foo + """, + "", + "", + "", + ] def download(url, output, **kwargs): _ = kwargs # buildifier: disable=unused-variable @@ -240,14 +195,16 @@ def _test_download_url(env): getenv = {}.get, download = download, report_progress = lambda _: None, - read = lambda i: "contents of " + i, + # We will first add a download to the list, so this is a poor man's `next(foo)` + # implementation + read = lambda i: reads[len(downloads) - 1], path = lambda i: "path/for/" + i, ), attr = struct( index_url_overrides = {}, index_url = "https://example.com/main/simple/", extra_index_urls = [], - sources = {"bar": None, "baz": None, "foo": None}, + sources = {"bar": ["1.0"], "baz": ["1.0"], "foo": ["1.0"]}, envsubst = [], ), cache = pypi_cache(), @@ -256,6 +213,7 @@ def _test_download_url(env): ) env.expect.that_dict(downloads).contains_exactly({ + "https://example.com/main/simple/": "path/for/https___example_com_main_simple.html", "https://example.com/main/simple/bar/": "path/for/https___example_com_main_simple_bar.html", "https://example.com/main/simple/baz/": "path/for/https___example_com_main_simple_baz.html", "https://example.com/main/simple/foo/": "path/for/https___example_com_main_simple_foo.html", @@ -265,6 +223,18 @@ _tests.append(_test_download_url) def _test_download_url_parallel(env): downloads = {} + reads = [ + # The first read is the index which seeds the downloads later + """ + bar + baz + foo + """, + "", + "", + "", + "", + ] def download(url, output, **kwargs): _ = kwargs # buildifier: disable=unused-variable @@ -276,13 +246,15 @@ def _test_download_url_parallel(env): getenv = {}.get, download = download, report_progress = lambda _: None, - read = lambda i: "contents of " + i, + # We will first add a download to the list, so this is a poor man's `next(foo)` + # implementation. We use 2 because we will enqueue 2 downloads in parallel. + read = lambda i: reads[len(downloads) - 2], path = lambda i: "path/for/" + i, ), attr = struct( index_url_overrides = {}, - index_url = "https://example.com/main/simple/", - extra_index_urls = [], + index_url = "https://example.com/default/simple/", + extra_index_urls = ["https://example.com/extra/simple/"], sources = {"bar": None, "baz": None, "foo": None}, envsubst = [], ), @@ -292,15 +264,28 @@ def _test_download_url_parallel(env): ) env.expect.that_dict(downloads).contains_exactly({ - "https://example.com/main/simple/bar/": "path/for/https___example_com_main_simple_bar.html", - "https://example.com/main/simple/baz/": "path/for/https___example_com_main_simple_baz.html", - "https://example.com/main/simple/foo/": "path/for/https___example_com_main_simple_foo.html", + "https://example.com/default/simple/": "path/for/https___example_com_default_simple.html", + "https://example.com/extra/simple/": "path/for/https___example_com_extra_simple.html", + "https://example.com/extra/simple/bar/": "path/for/https___example_com_extra_simple_bar.html", + "https://example.com/extra/simple/baz/": "path/for/https___example_com_extra_simple_baz.html", + "https://example.com/extra/simple/foo/": "path/for/https___example_com_extra_simple_foo.html", }) _tests.append(_test_download_url_parallel) def _test_download_envsubst_url(env): downloads = {} + reads = [ + # The first read is the index which seeds the downloads later + """ + bar + baz + foo + """, + "", + "", + "", + ] def download(url, output, **kwargs): _ = kwargs # buildifier: disable=unused-variable @@ -312,7 +297,9 @@ def _test_download_envsubst_url(env): getenv = {"INDEX_URL": "https://example.com/main/simple/"}.get, download = download, report_progress = lambda _: None, - read = lambda i: "contents of " + i, + # We will first add a download to the list, so this is a poor man's `next(foo)` + # implementation + read = lambda i: reads[len(downloads) - 1], path = lambda i: "path/for/" + i, ), attr = struct( @@ -328,6 +315,7 @@ def _test_download_envsubst_url(env): ) env.expect.that_dict(downloads).contains_exactly({ + "https://example.com/main/simple/": "path/for/~index_url~.html", "https://example.com/main/simple/bar/": "path/for/~index_url~_bar.html", "https://example.com/main/simple/baz/": "path/for/~index_url~_baz.html", "https://example.com/main/simple/foo/": "path/for/~index_url~_foo.html", From 782ae91b68e7747d80eff3b2e94d2df0bc42fac5 Mon Sep 17 00:00:00 2001 From: Ignas Anikevicius <240938+aignas@users.noreply.github.com> Date: Fri, 3 Apr 2026 13:09:09 +0900 Subject: [PATCH 677/922] fix(uv): drop powerpc64 support to fix latest version downloads (#3678) Before this PR we would index all of the available binaries and it would fail in the case if the `sha256` file is not found. It seems that this is the case for the `powerpc64`. In order to work this around, we just drop support for that particular platform. Whilst at it, bump the uv version. Fixes #3676. --- .github/workflows/mypy.yaml | 4 ---- CHANGELOG.md | 18 ++++++++++++++++++ MODULE.bazel | 11 ++--------- python/uv/private/uv.bzl | 19 +++++++++++++++++++ 4 files changed, 39 insertions(+), 13 deletions(-) diff --git a/.github/workflows/mypy.yaml b/.github/workflows/mypy.yaml index a22119e118..e38e5c71ba 100644 --- a/.github/workflows/mypy.yaml +++ b/.github/workflows/mypy.yaml @@ -21,11 +21,7 @@ jobs: - uses: actions/checkout@v6 - uses: jpetrucciani/mypy-check@master with: - requirements: 1.6.0 - python_version: 3.9 path: 'python/runfiles' - uses: jpetrucciani/mypy-check@master with: - requirements: 1.6.0 - python_version: 3.9 path: 'tests/runfiles' diff --git a/CHANGELOG.md b/CHANGELOG.md index 8e7c152acd..39f0223e4a 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -81,6 +81,11 @@ Other changes: Fixes ([#3260](https://github.com/bazel-contrib/rules_python/issues/3260) and [#2632](https://github.com/bazel-contrib/rules_python/issues/2632)). +* (uv) We will now use the download URL specified in the `uv`'s `dist_manifest.json` + file. If you have redirects or blocking rules as part of your downloader setup, + you may need to adjust them. What is more, the default uv version has been bumped + `0.11.2`. +* (runfiles): We are stopping the type annotation testing with `mypy` for Python 3.9. {#v0-0-0-fixed} ### Fixed @@ -97,6 +102,19 @@ Other changes: * (bootstrap) Fixed incorrect runfiles path construction in bootstrap scripts when binary is defined in another bazel module ([#3563](https://github.com/bazel-contrib/rules_python/issues/3563)). +* (uv) Downloads for versions `>=0.10` work again. In order to fix this we had + drop support for `powerpc64` platform. People interested in the platform can + bring it back via the `uv.default` API. Like: + ``` + uv.default( + compatible_with = [ + "@platforms//os:linux", + "@platforms//cpu:ppc", + ], + platform = "powerpc64-unknown-linux-gnu", + ) + ``` + Fixes [#3676](https://github.com/bazel-contrib/rules_python/issues/3676). {#v0-0-0-added} ### Added diff --git a/MODULE.bazel b/MODULE.bazel index 7cfe4ee576..28ce9fe7d4 100644 --- a/MODULE.bazel +++ b/MODULE.bazel @@ -385,7 +385,7 @@ uv = use_extension("//python/uv:uv.bzl", "uv") uv.default( base_url = "https://github.com/astral-sh/uv/releases/download", manifest_filename = "dist-manifest.json", - version = "0.6.3", + version = "0.11.2", ) uv.default( compatible_with = [ @@ -401,13 +401,6 @@ uv.default( ], platform = "aarch64-unknown-linux-gnu", ) -uv.default( - compatible_with = [ - "@platforms//os:linux", - "@platforms//cpu:ppc", - ], - platform = "powerpc64-unknown-linux-gnu", -) uv.default( compatible_with = [ "@platforms//os:linux", @@ -460,7 +453,7 @@ uv_dev = use_extension( dev_dependency = True, ) uv_dev.configure( - version = "0.6.2", + version = "0.11.2", ) # Temporarily comment out these flag aliases because they break Bazel 9 diff --git a/python/uv/private/uv.bzl b/python/uv/private/uv.bzl index fe0911e3ea..d9302c71ae 100644 --- a/python/uv/private/uv.bzl +++ b/python/uv/private/uv.bzl @@ -420,6 +420,17 @@ def _get_tool_urls_from_dist_manifest(module_ctx, *, base_url, manifest_filename "uv-aarch64-apple-darwin.tar.gz.sha256" "... ] + hosting + order + 0 "simple" + 1 "github" + github + artifact_base_url "https://github.com" + artifact_download_path "/astral-sh/uv/releases/download/0.11.2" + owner "astral-sh" + repo "uv" + simple + download_url "https://releases.astral.sh/github/uv/releases/download/0.11.2" artifacts uv-aarch64-apple-darwin.tar.gz name "uv-aarch64-apple-darwin.tar.gz" @@ -460,6 +471,14 @@ def _get_tool_urls_from_dist_manifest(module_ctx, *, base_url, manifest_filename fail(result) dist_manifest = json.decode(module_ctx.read(dist_manifest)) + base_url = ( + dist_manifest + .get("releases", [{}])[0] + .get("hosting", {}) + .get("simple", {}) + .get("download_url", base_url) + ) + artifacts = dist_manifest["artifacts"] tool_sources = {} downloads = {} From df7a1681a7ba5ca4d658a5549212e6bd7fd5b1fc Mon Sep 17 00:00:00 2001 From: Lloyd Pique Date: Fri, 3 Apr 2026 02:39:00 -0700 Subject: [PATCH 678/922] fix(bootstrap) handle when the runfiles env vars are not correct (#3644) This change addresses an issue that has existed since 1.7.0 where Python binaries launched as subprocesses could incorrectly inherit and use runfiles environment variables (`RUNFILES_DIR`, `RUNFILES_MANIFEST_FILE`) from the parent process. ## Why this change is needed: When a Python binary spawns another Python binary, the child process inherits the environment variables. If the parent had runfiles-related environment variables set, the child would attempt to use the parent's runfiles tree, which is incorrect and leads to import errors if the child has different dependencies or a different runfiles layout. ## Behavior Before: A child Python process would trust and use any existing `RUNFILES_DIR` and `RUNFILES_MANIFEST_FILE` environment variables. If these were set by a parent Python process, they would point to the parent's runfiles, causing the child to fail when trying to load its own resources or dependencies. ## Behavior After: The Python bootstrap scripts now include a check to validate the inherited runfiles environment variables. If the runfiles variables exist but do not point to the correct location for the current binary, the bootstrap script will unset both `RUNFILES_DIR` and `RUNFILES_MANIFEST_FILE` from the environment. This allows the subsequent fallback logic in the bootstrap process to correctly locate the runfiles for the current process, ensuring dependencies are resolved properly. This modification has been applied to both bootstrap template files. This change ensures that even nested Python binary calls correctly find their respective runfiles. Fixes #3518 --------- Co-authored-by: Ignas Anikevicius <240938+aignas@users.noreply.github.com> --- CHANGELOG.md | 3 +++ python/private/python_bootstrap_template.txt | 7 +++++++ python/private/stage2_bootstrap_template.py | 7 +++++++ .../bootstrap_impls/bin_calls_bin/BUILD.bazel | 9 +++++++++ tests/bootstrap_impls/bin_calls_bin/inner.py | 10 ++++++++++ .../bootstrap_impls/bin_calls_bin/inner_lib.py | 3 +++ tests/bootstrap_impls/bin_calls_bin/outer.py | 2 +- tests/bootstrap_impls/bin_calls_bin/verify.sh | 18 ++++++++++++++++++ 8 files changed, 58 insertions(+), 1 deletion(-) create mode 100644 tests/bootstrap_impls/bin_calls_bin/inner_lib.py diff --git a/CHANGELOG.md b/CHANGELOG.md index 39f0223e4a..7734a45fe4 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -102,6 +102,9 @@ Other changes: * (bootstrap) Fixed incorrect runfiles path construction in bootstrap scripts when binary is defined in another bazel module ([#3563](https://github.com/bazel-contrib/rules_python/issues/3563)). +* (bootstrap) Resolve `RUNFILES_DIR` inheritance issues, which lead to a child + Python binary incorrectly using it's parent's Python binary environment + ([#3518](https://github.com/bazel-contrib/rules_python/issues/3518)). * (uv) Downloads for versions `>=0.10` work again. In order to fix this we had drop support for `powerpc64` platform. People interested in the platform can bring it back via the `uv.default` API. Like: diff --git a/python/private/python_bootstrap_template.txt b/python/private/python_bootstrap_template.txt index 4efd46690a..51c9013e9d 100644 --- a/python/private/python_bootstrap_template.txt +++ b/python/private/python_bootstrap_template.txt @@ -214,6 +214,13 @@ def find_runfiles_root(main_rel_path): if runfiles_dir and os.path.exists(os.path.join(runfiles_dir, main_rel_path)): return runfiles_dir + # Clear RUNFILES_DIR & RUNFILES_MANIFEST_FILE since the runfiles dir was + # not found. These can be correctly set for a parent Python process, but + # inherited by the child, and not correct for it. Later bootstrap code + # assumes they're are correct if set. + os.environ.pop('RUNFILES_DIR', None) + os.environ.pop('RUNFILES_MANIFEST_FILE', None) + stub_filename = sys.argv[0] # On Windows, the path may contain both forward and backslashes. # Normalize to the OS separator because the regex used later assumes diff --git a/python/private/stage2_bootstrap_template.py b/python/private/stage2_bootstrap_template.py index c03a4a2e62..3c2d6807cc 100644 --- a/python/private/stage2_bootstrap_template.py +++ b/python/private/stage2_bootstrap_template.py @@ -185,6 +185,13 @@ def find_runfiles_root(main_rel_path): if runfiles_dir and os.path.exists(os.path.join(runfiles_dir, main_rel_path)): return runfiles_dir + # Clear RUNFILES_DIR & RUNFILES_MANIFEST_FILE since the runfiles dir was + # not found. These can be correctly set for a parent Python process, but + # inherited by the child, and not correct for it. Later bootstrap code + # assumes they're are correct if set. + os.environ.pop('RUNFILES_DIR', None) + os.environ.pop('RUNFILES_MANIFEST_FILE', None) + stub_filename = sys.argv[0] if not os.path.isabs(stub_filename): stub_filename = os.path.join(os.getcwd(), stub_filename) diff --git a/tests/bootstrap_impls/bin_calls_bin/BUILD.bazel b/tests/bootstrap_impls/bin_calls_bin/BUILD.bazel index 1822ccd08e..b5ba68f987 100644 --- a/tests/bootstrap_impls/bin_calls_bin/BUILD.bazel +++ b/tests/bootstrap_impls/bin_calls_bin/BUILD.bazel @@ -1,7 +1,14 @@ load("@rules_shell//shell:sh_test.bzl", "sh_test") +load("//python:py_library.bzl", "py_library") load("//tests/support:py_reconfig.bzl", "py_reconfig_binary") load("//tests/support:support.bzl", "NOT_WINDOWS", "SUPPORTS_BOOTSTRAP_SCRIPT") +py_library( + name = "inner_lib", + srcs = ["inner_lib.py"], + tags = ["manual"], +) + # ===== # bootstrap_impl=system_python testing # ===== @@ -19,6 +26,7 @@ py_reconfig_binary( bootstrap_impl = "system_python", main = "inner.py", tags = ["manual"], + deps = [":inner_lib"], ) genrule( @@ -54,6 +62,7 @@ py_reconfig_binary( bootstrap_impl = "script", main = "inner.py", tags = ["manual"], + deps = [":inner_lib"], ) py_reconfig_binary( diff --git a/tests/bootstrap_impls/bin_calls_bin/inner.py b/tests/bootstrap_impls/bin_calls_bin/inner.py index 96409eb8f8..6fef455a84 100644 --- a/tests/bootstrap_impls/bin_calls_bin/inner.py +++ b/tests/bootstrap_impls/bin_calls_bin/inner.py @@ -1,4 +1,14 @@ import os runfiles_root = os.environ.get("RULES_PYTHON_TESTING_RUNFILES_ROOT") +runfiles_dir = os.environ.get("RUNFILES_DIR") +runfiles_manifest_file = os.environ.get("RUNFILES_MANIFEST_FILE") print(f"inner: RULES_PYTHON_TESTING_RUNFILES_ROOT='{runfiles_root}'") +print(f"inner: RUNFILES_DIR='{runfiles_dir}'") +print(f"inner: RUNFILES_MANIFEST_FILE='{runfiles_manifest_file}'") + +try: + import tests.bootstrap_impls.bin_calls_bin.inner_lib as inner_lib + print(f"inner: import_result='{inner_lib.confirm()}'") +except ImportError as e: + print(f"inner: import_result='{e}'") diff --git a/tests/bootstrap_impls/bin_calls_bin/inner_lib.py b/tests/bootstrap_impls/bin_calls_bin/inner_lib.py new file mode 100644 index 0000000000..97efbb1565 --- /dev/null +++ b/tests/bootstrap_impls/bin_calls_bin/inner_lib.py @@ -0,0 +1,3 @@ +# Rather than having a completely empty file... +def confirm(): + return "success" \ No newline at end of file diff --git a/tests/bootstrap_impls/bin_calls_bin/outer.py b/tests/bootstrap_impls/bin_calls_bin/outer.py index f995c24ff6..a4432dff59 100644 --- a/tests/bootstrap_impls/bin_calls_bin/outer.py +++ b/tests/bootstrap_impls/bin_calls_bin/outer.py @@ -11,8 +11,8 @@ [inner_binary_path], capture_output=True, text=True, - check=True, ) print(result.stdout, end="") if result.stderr: print(result.stderr, end="", file=sys.stderr) + sys.exit(result.returncode) diff --git a/tests/bootstrap_impls/bin_calls_bin/verify.sh b/tests/bootstrap_impls/bin_calls_bin/verify.sh index 9a9a17c6c6..bbe4252cc1 100755 --- a/tests/bootstrap_impls/bin_calls_bin/verify.sh +++ b/tests/bootstrap_impls/bin_calls_bin/verify.sh @@ -11,6 +11,18 @@ verify_output() { echo "Outer runfiles root: $OUTER_RUNFILES_ROOT" echo "Inner runfiles root: $INNER_RUNFILES_ROOT" + # Extract the inner runfiles values + local INNER_RUNFILES_DIR=$(grep "inner: RUNFILES_DIR" "$OUTPUT_FILE" | sed "s/inner: RUNFILES_DIR='\(.*\)'/\1/") + local INNER_RUNFILES_MANIFEST_FILE=$(grep "inner: RUNFILES_MANIFEST_FILE" "$OUTPUT_FILE" | sed "s/inner: RUNFILES_MANIFEST_FILE='\(.*\)'/\1/") + + echo "Inner runfiles dir: $INNER_RUNFILES_DIR" + echo "Inner runfiles manifest file: $INNER_RUNFILES_MANIFEST_FILE" + + # Extract the inner lib import result + local INNER_LIB_IMPORT=$(grep "inner: import_result" "$OUTPUT_FILE" | sed "s/inner: import_result='\(.*\)'/\1/") + echo "Inner lib import result: $INNER_LIB_IMPORT" + + # Check 1: The two values are different if [ "$OUTER_RUNFILES_ROOT" == "$INNER_RUNFILES_ROOT" ]; then echo "Error: Outer and Inner runfiles roots are the same." @@ -28,5 +40,11 @@ verify_output() { ;; esac + # Check 3: inner_lib was imported + if [ "$INNER_LIB_IMPORT" != "success" ]; then + echo "Error: Inner lib was not successfully imported." + exit 1 + fi + echo "Verification successful." } From 6d387703811584eb784cc5f396762435aa759ad4 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Sat, 4 Apr 2026 19:47:27 -0700 Subject: [PATCH 679/922] feat(zipapp): support EXTRACT_ROOT env var for __main__.py invocations (#3682) This makes zipapps that are invoked using the `__main__.py` entry point (i.e. `python foo.zip`) respect the RULES_PYTHON_EXTRACT_ROOT env var, which allows some control over where they extract themselves. --- python/private/zipapp/py_zipapp_rule.bzl | 5 ++ python/private/zipapp/zip_main_template.py | 59 +++++++++++++------ ...m_python_zipapp_external_bootstrap_test.sh | 15 +++++ 3 files changed, 61 insertions(+), 18 deletions(-) diff --git a/python/private/zipapp/py_zipapp_rule.bzl b/python/private/zipapp/py_zipapp_rule.bzl index a6ab485fc4..ac7944726f 100644 --- a/python/private/zipapp/py_zipapp_rule.bzl +++ b/python/private/zipapp/py_zipapp_rule.bzl @@ -35,6 +35,11 @@ def _create_zipapp_main_py(ctx, py_runtime, py_executable, stage2_bootstrap): template = py_runtime.zip_main_template, output = zip_main_py, substitutions = { + "%EXTRACT_DIR%": paths.join( + (ctx.label.repo_name or "_main"), + ctx.label.package, + ctx.label.name, + ), "%python_binary%": venv_python_exe_path, "%python_binary_actual%": python_binary_actual_path, "%stage2_bootstrap%": runfiles_root_path(ctx, stage2_bootstrap.short_path), diff --git a/python/private/zipapp/zip_main_template.py b/python/private/zipapp/zip_main_template.py index e997110a5c..3c25d1d722 100644 --- a/python/private/zipapp/zip_main_template.py +++ b/python/private/zipapp/zip_main_template.py @@ -23,9 +23,11 @@ import os import shutil +import stat import subprocess import tempfile import zipfile +from os.path import dirname, join # runfiles-root-relative path _STAGE2_BOOTSTRAP = "%stage2_bootstrap%" @@ -35,6 +37,10 @@ # executable to use. _PYTHON_BINARY_ACTUAL = "%python_binary_actual%" _WORKSPACE_NAME = "%workspace_name%" +# relative path under EXTRACT_ROOT to extract to. +EXTRACT_DIR = "%EXTRACT_DIR%" + +EXTRACT_ROOT = os.environ.get("RULES_PYTHON_EXTRACT_ROOT") def print_verbose(*args, mapping=None, values=None): @@ -118,7 +124,7 @@ def search_path(name): search_path = os.getenv("PATH", os.defpath).split(os.pathsep) for directory in search_path: if directory: - path = os.path.join(directory, name) + path = join(directory, name) if os.path.isfile(path) and os.access(path, os.X_OK): return path return None @@ -139,7 +145,7 @@ def find_binary(runfiles_root, bin_name): # Use normpath() to convert slashes to os.sep on Windows. elif os.sep in os.path.normpath(bin_name): # Case 3: Path is relative to the repo root. - return os.path.join(runfiles_root, bin_name) + return join(runfiles_root, bin_name) else: # Case 4: Path has to be looked up in the search path. return search_path(bin_name) @@ -161,10 +167,18 @@ def extract_zip(zip_path, dest_dir): dest_dir = get_windows_path_with_unc_prefix(dest_dir) with zipfile.ZipFile(zip_path) as zf: for info in zf.infolist(): + file_path = os.path.abspath(join(dest_dir, info.filename)) + # If the file exists, it might be a symlink or read-only file from a previous extraction. + # Unlink it first so zipfile.extract doesn't corrupt the symlink target or fail on read-only files. + if os.path.lexists(file_path) and not os.path.isdir(file_path): + try: + os.unlink(file_path) + except OSError: + # On Windows, unlinking a read-only file fails. + os.chmod(file_path, stat.S_IWRITE) + os.unlink(file_path) + zf.extract(info, dest_dir) - # UNC-prefixed paths must be absolute/normalized. See - # https://docs.microsoft.com/en-us/windows/desktop/fileio/naming-a-file#maximum-path-length-limitation - file_path = os.path.abspath(os.path.join(dest_dir, info.filename)) # The Unix st_mode bits (see "man 7 inode") are stored in the upper 16 # bits of external_attr. attrs = info.external_attr >> 16 @@ -182,11 +196,14 @@ def extract_zip(zip_path, dest_dir): # Create the runfiles tree by extracting the zip file def create_runfiles_root(): - temp_dir = tempfile.mkdtemp("", "Bazel.runfiles_") - extract_zip(os.path.dirname(__file__), temp_dir) + if EXTRACT_ROOT: + extract_root = join(EXTRACT_ROOT, EXTRACT_DIR) + else: + extract_root = tempfile.mkdtemp("", "Bazel.runfiles_") + extract_zip(dirname(__file__), extract_root) # IMPORTANT: Later code does `rm -fr` on dirname(runfiles_root) -- it's # important that deletion code be in sync with this directory structure - return os.path.join(temp_dir, "runfiles") + return join(extract_root, "runfiles") def execute_file( @@ -223,18 +240,24 @@ def execute_file( # - When running in a zip file, we need to clean up the # workspace after the process finishes so control must return here. try: - subprocess_argv = [python_program, main_filename] + args + subprocess_argv = [python_program] + if not EXTRACT_ROOT: + subprocess_argv.append(f"-XRULES_PYTHON_ZIP_DIR={dirname(runfiles_root)}") + subprocess_argv.append(main_filename) + subprocess_argv += args print_verbose("subprocess argv:", values=subprocess_argv) print_verbose("subprocess env:", mapping=env) print_verbose("subprocess cwd:", workspace) ret_code = subprocess.call(subprocess_argv, env=env, cwd=workspace) sys.exit(ret_code) finally: - # NOTE: dirname() is called because create_runfiles_root() creates a - # sub-directory within a temporary directory, and we want to remove the - # whole temporary directory. - ##shutil.rmtree(os.path.dirname(runfiles_root), True) - pass + if not EXTRACT_ROOT: + # NOTE: dirname() is called because create_runfiles_root() creates a + # sub-directory within a temporary directory, and we want to remove the + # whole temporary directory. + extract_root = dirname(runfiles_root) + print_verbose("cleanup: rmtree: ", extract_root) + shutil.rmtree(extract_root, True) def main(): @@ -266,7 +289,7 @@ def main(): # See: https://docs.python.org/3.11/using/cmdline.html#envvar-PYTHONSAFEPATH new_env["PYTHONSAFEPATH"] = "1" - main_filename = os.path.join(runfiles_root, main_rel_path) + main_filename = join(runfiles_root, main_rel_path) main_filename = get_windows_path_with_unc_prefix(main_filename) assert os.path.exists(main_filename), ( "Cannot exec() %r: file not found." % main_filename @@ -276,7 +299,7 @@ def main(): ) if _PYTHON_BINARY_VENV: - python_program = os.path.join(runfiles_root, _PYTHON_BINARY_VENV) + python_program = join(runfiles_root, _PYTHON_BINARY_VENV) # When a venv is used, the `bin/python3` symlink may need to be created. # This case occurs when "create venv at runtime" or "resolve python at # runtime" modes are enabled. @@ -288,7 +311,7 @@ def main(): "Program's venv binary not under runfiles: {python_program}" ) symlink_to = find_binary(runfiles_root, _PYTHON_BINARY_ACTUAL) - os.makedirs(os.path.dirname(python_program), exist_ok=True) + os.makedirs(dirname(python_program), exist_ok=True) try: os.symlink(symlink_to, python_program) except OSError as e: @@ -317,7 +340,7 @@ def main(): # change directory to the right runfiles directory. # (So that the data files are accessible) if os.environ.get("RUN_UNDER_RUNFILES") == "1": - workspace = os.path.join(runfiles_root, _WORKSPACE_NAME) + workspace = join(runfiles_root, _WORKSPACE_NAME) sys.stdout.flush() execute_file( diff --git a/tests/py_zipapp/system_python_zipapp_external_bootstrap_test.sh b/tests/py_zipapp/system_python_zipapp_external_bootstrap_test.sh index 21c6741197..bb4ba640d3 100755 --- a/tests/py_zipapp/system_python_zipapp_external_bootstrap_test.sh +++ b/tests/py_zipapp/system_python_zipapp_external_bootstrap_test.sh @@ -13,6 +13,21 @@ fi ZIPAPP="${ZIPAPP/.exe/.zip}" export RULES_PYTHON_BOOTSTRAP_VERBOSE=1 + # We're testing the invocation of `__main__.py`, so we have to # manually pass the zipapp to python. +echo "Running zipapp using an automatic temp directory..." +"$PYTHON" "$ZIPAPP" + +echo "Running zipapp with extract root set..." +export RULES_PYTHON_EXTRACT_ROOT="${TEST_TMPDIR:-/tmp}/extract_root_test" +"$PYTHON" "$ZIPAPP" + +# Verify that the directory was created +if [[ ! -d "$RULES_PYTHON_EXTRACT_ROOT" ]]; then + echo "Error: Extract root directory $RULES_PYTHON_EXTRACT_ROOT was not created!" + exit 1 +fi + +echo "Running zipapp with extract root set a second time..." "$PYTHON" "$ZIPAPP" From 2c5616be1950400beb14ab89fb06076f0c69e4e5 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Sun, 5 Apr 2026 10:10:34 -0700 Subject: [PATCH 680/922] feat(zipapp): add content hash support to __main__-based invocations (#3683) This adds support for zipapps going through `__main__` to extract to a directory based on their content hash. This matches behavior of the shell-based self-executable bootstrap. --- python/private/zipapp/py_zipapp_rule.bzl | 81 +++++++++----- python/private/zipapp/zip_main_template.py | 25 +++-- ...m_python_zipapp_external_bootstrap_test.sh | 21 ++++ tests/tools/zipapp/BUILD.bazel | 6 ++ tests/tools/zipapp/zip_main_maker_test.py | 101 ++++++++++++++++++ tools/BUILD.bazel | 1 + tools/private/BUILD.bazel | 10 ++ tools/private/zipapp/BUILD.bazel | 20 ++++ tools/private/zipapp/zip_main_maker.py | 90 ++++++++++++++++ 9 files changed, 321 insertions(+), 34 deletions(-) create mode 100644 tests/tools/zipapp/zip_main_maker_test.py create mode 100644 tools/private/zipapp/zip_main_maker.py diff --git a/python/private/zipapp/py_zipapp_rule.bzl b/python/private/zipapp/py_zipapp_rule.bzl index ac7944726f..1655f63cc9 100644 --- a/python/private/zipapp/py_zipapp_rule.bzl +++ b/python/private/zipapp/py_zipapp_rule.bzl @@ -18,7 +18,7 @@ def _is_symlink(f): else: return "-1" -def _create_zipapp_main_py(ctx, py_runtime, py_executable, stage2_bootstrap): +def _create_zipapp_main_py(ctx, py_runtime, py_executable, stage2_bootstrap, runfiles): venv_python_exe = py_executable.venv_python_exe if venv_python_exe: venv_python_exe_path = runfiles_root_path(ctx, venv_python_exe.short_path) @@ -31,20 +31,40 @@ def _create_zipapp_main_py(ctx, py_runtime, py_executable, stage2_bootstrap): python_binary_actual_path = py_runtime.interpreter_path zip_main_py = ctx.actions.declare_file(ctx.label.name + ".zip_main.py") - ctx.actions.expand_template( - template = py_runtime.zip_main_template, - output = zip_main_py, - substitutions = { - "%EXTRACT_DIR%": paths.join( - (ctx.label.repo_name or "_main"), - ctx.label.package, - ctx.label.name, - ), - "%python_binary%": venv_python_exe_path, - "%python_binary_actual%": python_binary_actual_path, - "%stage2_bootstrap%": runfiles_root_path(ctx, stage2_bootstrap.short_path), - "%workspace_name%": ctx.workspace_name, - }, + + args = ctx.actions.args() + args.add(py_runtime.zip_main_template, format = "--template=%s") + args.add(zip_main_py, format = "--output=%s") + + args.add( + "%EXTRACT_DIR%=" + paths.join( + (ctx.label.repo_name or "_main"), + ctx.label.package, + ctx.label.name, + ), + format = "--substitution=%s", + ) + args.add("%python_binary%=" + venv_python_exe_path, format = "--substitution=%s") + args.add("%python_binary_actual%=" + python_binary_actual_path, format = "--substitution=%s") + args.add("%stage2_bootstrap%=" + runfiles_root_path(ctx, stage2_bootstrap.short_path), format = "--substitution=%s") + args.add("%workspace_name%=" + ctx.workspace_name, format = "--substitution=%s") + + hash_files_manifest = ctx.actions.args() + hash_files_manifest.use_param_file("--hash_files_manifest=%s", use_always = True) + hash_files_manifest.set_param_file_format("multiline") + + inputs = builders.DepsetBuilder() + inputs.add(py_runtime.zip_main_template) + _build_manifest(ctx, hash_files_manifest, runfiles, inputs) + + actions_run( + ctx, + executable = ctx.attr._zip_main_maker, + arguments = [args, hash_files_manifest], + inputs = inputs.build(), + outputs = [zip_main_py], + mnemonic = "PyZipAppCreateMainPy", + progress_message = "Generating zipapp __main__.py: %{label}", ) return zip_main_py @@ -60,9 +80,7 @@ def _map_zip_symlinks(entry): def _map_zip_root_symlinks(entry): return "rf-root-symlink|" + _is_symlink(entry.target_file) + "|" + entry.path + "|" + entry.target_file.path -def _build_manifest(ctx, manifest, runfiles, zip_main): - manifest.add("regular|0|__main__.py|{}".format(zip_main.path)) - +def _build_manifest(ctx, manifest, runfiles, inputs): manifest.add_all( # NOTE: Accessing runfiles.empty_filenames materializes them. A lambda # is used to defer that. @@ -75,7 +93,10 @@ def _build_manifest(ctx, manifest, runfiles, zip_main): manifest.add_all(runfiles.symlinks, map_each = _map_zip_symlinks) manifest.add_all(runfiles.root_symlinks, map_each = _map_zip_root_symlinks) - inputs = [zip_main] + inputs.add(runfiles.files) + inputs.add([entry.target_file for entry in runfiles.symlinks.to_list()]) + inputs.add([entry.target_file for entry in runfiles.root_symlinks.to_list()]) + zip_repo_mapping_manifest = maybe_create_repo_mapping( ctx = ctx, runfiles = runfiles, @@ -87,8 +108,7 @@ def _build_manifest(ctx, manifest, runfiles, zip_main): zip_repo_mapping_manifest.path, format = "rf-root-symlink|0|_repo_mapping|%s", ) - inputs.append(zip_repo_mapping_manifest) - return inputs + inputs.add(zip_repo_mapping_manifest) def _create_zip(ctx, py_runtime, py_executable, stage2_bootstrap): output = ctx.actions.declare_file(ctx.label.name + ".zip") @@ -106,8 +126,17 @@ def _create_zip(ctx, py_runtime, py_executable, stage2_bootstrap): runfiles = runfiles.build(ctx) - zip_main = _create_zipapp_main_py(ctx, py_runtime, py_executable, stage2_bootstrap) - inputs = _build_manifest(ctx, manifest, runfiles, zip_main) + zip_main = _create_zipapp_main_py( + ctx, + py_runtime, + py_executable, + stage2_bootstrap, + runfiles, + ) + inputs = builders.DepsetBuilder() + manifest.add("regular|0|__main__.py|{}".format(zip_main.path)) + inputs.add(zip_main) + _build_manifest(ctx, manifest, runfiles, inputs) zipper_args = ctx.actions.args() zipper_args.add(output) @@ -124,7 +153,7 @@ def _create_zip(ctx, py_runtime, py_executable, stage2_bootstrap): ctx, executable = ctx.attr._zipper, arguments = [manifest, zipper_args], - inputs = depset(inputs, transitive = [runfiles.files]), + inputs = inputs.build(), outputs = [output], mnemonic = "PyZipAppCreateZip", progress_message = "Reticulating zipapp archive: %{label} into %{output}", @@ -315,6 +344,10 @@ Whether the output should be an executable zip file. "@platforms//os:windows", ], ), + "_zip_main_maker": attr.label( + cfg = "exec", + default = "//tools/private/zipapp:zip_main_maker", + ), "_zip_shell_template": attr.label( default = ":zip_shell_template", allow_single_file = True, diff --git a/python/private/zipapp/zip_main_template.py b/python/private/zipapp/zip_main_template.py index 3c25d1d722..97c37fee0b 100644 --- a/python/private/zipapp/zip_main_template.py +++ b/python/private/zipapp/zip_main_template.py @@ -27,7 +27,7 @@ import subprocess import tempfile import zipfile -from os.path import dirname, join +from os.path import dirname, join, basename # runfiles-root-relative path _STAGE2_BOOTSTRAP = "%stage2_bootstrap%" @@ -39,8 +39,10 @@ _WORKSPACE_NAME = "%workspace_name%" # relative path under EXTRACT_ROOT to extract to. EXTRACT_DIR = "%EXTRACT_DIR%" +APP_HASH = "%APP_HASH%" EXTRACT_ROOT = os.environ.get("RULES_PYTHON_EXTRACT_ROOT") +IS_WINDOWS = os.name == "nt" def print_verbose(*args, mapping=None, values=None): @@ -67,10 +69,6 @@ def print_verbose(*args, mapping=None, values=None): print("bootstrap: stage 1:", *args, file=sys.stderr, flush=True) -# Return True if running on Windows -def is_windows(): - return os.name == "nt" - def get_windows_path_with_unc_prefix(path): """Adds UNC prefix after getting a normalized absolute Windows path. @@ -81,7 +79,7 @@ def get_windows_path_with_unc_prefix(path): # No need to add prefix for non-Windows platforms. # And \\?\ doesn't work in python 2 or on mingw - if not is_windows() or sys.version_info[0] < 3: + if not IS_WINDOWS or sys.version_info[0] < 3: return path # Starting in Windows 10, version 1607(OS build 14393), MAX_PATH limitations have been @@ -113,7 +111,7 @@ def has_windows_executable_extension(path): if ( _PYTHON_BINARY_VENV - and is_windows() + and IS_WINDOWS and not has_windows_executable_extension(_PYTHON_BINARY_VENV) ): _PYTHON_BINARY_VENV = _PYTHON_BINARY_VENV + ".exe" @@ -197,7 +195,14 @@ def extract_zip(zip_path, dest_dir): # Create the runfiles tree by extracting the zip file def create_runfiles_root(): if EXTRACT_ROOT: - extract_root = join(EXTRACT_ROOT, EXTRACT_DIR) + # Shorten the path for Windows in case long path support is disabled + if IS_WINDOWS: + hash_dir = APP_HASH[0:32] + extract_dir = basename(EXTRACT_DIR) + extract_root = join(EXTRACT_ROOT, extract_dir, hash_dir) + else: + extract_root = join(EXTRACT_ROOT, EXTRACT_DIR, APP_HASH) + extract_root = get_windows_path_with_unc_prefix(extract_root) else: extract_root = tempfile.mkdtemp("", "Bazel.runfiles_") extract_zip(dirname(__file__), extract_root) @@ -245,9 +250,9 @@ def execute_file( subprocess_argv.append(f"-XRULES_PYTHON_ZIP_DIR={dirname(runfiles_root)}") subprocess_argv.append(main_filename) subprocess_argv += args - print_verbose("subprocess argv:", values=subprocess_argv) print_verbose("subprocess env:", mapping=env) print_verbose("subprocess cwd:", workspace) + print_verbose("subprocess argv:", values=subprocess_argv) ret_code = subprocess.call(subprocess_argv, env=env, cwd=workspace) sys.exit(ret_code) finally: @@ -277,7 +282,7 @@ def main(): # The main Python source file. main_rel_path = _STAGE2_BOOTSTRAP - if is_windows(): + if IS_WINDOWS: main_rel_path = main_rel_path.replace("/", os.sep) runfiles_root = create_runfiles_root() diff --git a/tests/py_zipapp/system_python_zipapp_external_bootstrap_test.sh b/tests/py_zipapp/system_python_zipapp_external_bootstrap_test.sh index bb4ba640d3..e7396007d9 100755 --- a/tests/py_zipapp/system_python_zipapp_external_bootstrap_test.sh +++ b/tests/py_zipapp/system_python_zipapp_external_bootstrap_test.sh @@ -16,10 +16,17 @@ export RULES_PYTHON_BOOTSTRAP_VERBOSE=1 # We're testing the invocation of `__main__.py`, so we have to # manually pass the zipapp to python. +echo "=====================================================================" echo "Running zipapp using an automatic temp directory..." +echo "=====================================================================" "$PYTHON" "$ZIPAPP" +echo +echo + +echo "=====================================================================" echo "Running zipapp with extract root set..." +echo "=====================================================================" export RULES_PYTHON_EXTRACT_ROOT="${TEST_TMPDIR:-/tmp}/extract_root_test" "$PYTHON" "$ZIPAPP" @@ -29,5 +36,19 @@ if [[ ! -d "$RULES_PYTHON_EXTRACT_ROOT" ]]; then exit 1 fi +# On windows, the path is shortened to just the basename to avoid long path errors. +# Other platforms use the full path. +# Note: [ -d ... ] expands globs, while [[ -d ... ]] does not. +if [ -d "$RULES_PYTHON_EXTRACT_ROOT/_main/tests/py_zipapp/system_python_zipapp"/*/runfiles ]; then + echo "Found runfiles at $RULES_PYTHON_EXTRACT_ROOT/_main/tests/py_zipapp/system_python_zipapp/*/runfiles" +elif [ -d "$RULES_PYTHON_EXTRACT_ROOT/system_python_zipapp"/*/runfiles ]; then + echo "Found runfiles at $RULES_PYTHON_EXTRACT_ROOT/system_python_zipapp/*/runfiles" +else + echo "Error: Could not find 'runfiles' directory" + exit 1 +fi + +echo "=====================================================================" echo "Running zipapp with extract root set a second time..." +echo "=====================================================================" "$PYTHON" "$ZIPAPP" diff --git a/tests/tools/zipapp/BUILD.bazel b/tests/tools/zipapp/BUILD.bazel index 84902b76b8..b71e9b2589 100644 --- a/tests/tools/zipapp/BUILD.bazel +++ b/tests/tools/zipapp/BUILD.bazel @@ -11,3 +11,9 @@ py_test( srcs = ["exe_zip_maker_test.py"], deps = ["//tools/private/zipapp:exe_zip_maker_lib"], ) + +py_test( + name = "zip_main_maker_test", + srcs = ["zip_main_maker_test.py"], + deps = ["//tools/private/zipapp:zip_main_maker_lib"], +) diff --git a/tests/tools/zipapp/zip_main_maker_test.py b/tests/tools/zipapp/zip_main_maker_test.py new file mode 100644 index 0000000000..afcaf294d1 --- /dev/null +++ b/tests/tools/zipapp/zip_main_maker_test.py @@ -0,0 +1,101 @@ +import hashlib +import os +import tempfile +import unittest +from unittest import mock + +from tools.private.zipapp import zip_main_maker + + +class ZipMainMakerTest(unittest.TestCase): + def setUp(self): + self.temp_dir = tempfile.TemporaryDirectory() + self.addCleanup(self.temp_dir.cleanup) + + def test_creates_zip_main(self): + template_path = os.path.join(self.temp_dir.name, "template.py") + with open(template_path, "w", encoding="utf-8") as f: + f.write("hash=%APP_HASH%\nfoo=%FOO%\n") + + output_path = os.path.join(self.temp_dir.name, "output.py") + + file1_path = os.path.join(self.temp_dir.name, "file1.txt") + with open(file1_path, "wb") as f: + f.write(b"content1") + + file2_path = os.path.join(self.temp_dir.name, "file2.txt") + with open(file2_path, "wb") as f: + f.write(b"content2") + + # Add a symlink to test symlink hashing + symlink_path = os.path.join(self.temp_dir.name, "symlink.txt") + os.symlink(file1_path, symlink_path) + + manifest_path = os.path.join(self.temp_dir.name, "manifest.txt") + with open(manifest_path, "w", encoding="utf-8") as f: + f.write(f"rf-file|0|file1.txt|{file1_path}\n") + f.write(f"rf-file|0|file2.txt|{file2_path}\n") + f.write(f"rf-symlink|1|symlink.txt|{symlink_path}\n") + f.write(f"rf-empty|empty_file.txt\n") + + argv = [ + "zip_main_maker.py", + "--template", + template_path, + "--output", + output_path, + "--substitution", + "%FOO%=bar", + "--hash_files_manifest", + manifest_path, + ] + + with mock.patch("sys.argv", argv): + zip_main_maker.main() + + # Calculate expected hash + h = hashlib.sha256() + line1 = f"rf-file|0|file1.txt|{file1_path}" + line2 = f"rf-file|0|file2.txt|{file2_path}" + line3 = f"rf-symlink|1|symlink.txt|{symlink_path}" + line4 = f"rf-empty|empty_file.txt" + + # Sort lines like the program does + lines = sorted([line1, line2, line3, line4]) + for line in lines: + parts = line.split("|") + if len(parts) > 1: + _, rest = line.split("|", 1) + h.update(rest.encode("utf-8")) + else: + h.update(line.encode("utf-8")) + + type_ = parts[0] + if type_ == "rf-empty": + continue + if len(parts) >= 4: + is_symlink_str = parts[1] + path = parts[-1] + if not path: + continue + if is_symlink_str == "-1": + is_symlink = not os.path.exists(path) + else: + is_symlink = is_symlink_str == "1" + + if is_symlink: + h.update(os.readlink(path).encode("utf-8")) + else: + with open(path, "rb") as f: + h.update(f.read()) + + expected_hash = h.hexdigest() + + with open(output_path, "r", encoding="utf-8") as f: + content = f.read() + + self.assertEqual(content, f"hash={expected_hash}\nfoo=bar\n") + + +if __name__ == "__main__": + unittest.main() diff --git a/tools/BUILD.bazel b/tools/BUILD.bazel index 0fcce8f729..7829b33318 100644 --- a/tools/BUILD.bazel +++ b/tools/BUILD.bazel @@ -31,6 +31,7 @@ filegroup( "wheelmaker.py", "//tools/launcher:distribution", "//tools/precompiler:distribution", + "//tools/private:distribution", "//tools/publish:distribution", ], visibility = ["//:__pkg__"], diff --git a/tools/private/BUILD.bazel b/tools/private/BUILD.bazel index e69de29bb2..adc8de3b0f 100644 --- a/tools/private/BUILD.bazel +++ b/tools/private/BUILD.bazel @@ -0,0 +1,10 @@ +package( + default_visibility = ["//:__subpackages__"], +) + +filegroup( + name = "distribution", + srcs = glob(["**"]) + [ + "//tools/private/zipapp:distribution", + ], +) diff --git a/tools/private/zipapp/BUILD.bazel b/tools/private/zipapp/BUILD.bazel index 7a2002cd72..7420776ef3 100644 --- a/tools/private/zipapp/BUILD.bazel +++ b/tools/private/zipapp/BUILD.bazel @@ -34,3 +34,23 @@ py_library( name = "exe_zip_maker_lib", srcs = ["exe_zip_maker.py"], ) + +py_interpreter_program( + name = "zip_main_maker", + main = "zip_main_maker.py", + visibility = [ + # Not actually public. Only public so rules_python-generated toolchains + # are able to reference it. + "//visibility:public", + ], +) + +py_library( + name = "zip_main_maker_lib", + srcs = ["zip_main_maker.py"], +) + +filegroup( + name = "distribution", + srcs = glob(["**"]), +) diff --git a/tools/private/zipapp/zip_main_maker.py b/tools/private/zipapp/zip_main_maker.py new file mode 100644 index 0000000000..78ac17eb17 --- /dev/null +++ b/tools/private/zipapp/zip_main_maker.py @@ -0,0 +1,90 @@ +"""Creates the __main__.py for a zipapp by populating a template. + +This program also calculates a hash of the application files to include in +the template, which allows making the extraction directory unique to the +content of the zipapp. +""" + +import argparse +import hashlib +import os + +BLOCK_SIZE = 256 * 1024 + + +def create_parser() -> argparse.ArgumentParser: + parser = argparse.ArgumentParser(fromfile_prefix_chars="@") + parser.add_argument("--template", required=True) + parser.add_argument("--output", required=True) + parser.add_argument("--substitution", action="append", default=[]) + parser.add_argument( + "--hash_files_manifest", + required=True, + help="A file containing lines in rf-XXX formats (rf-empty, rf-file, rf-symlink, etc.)", + ) + return parser + + +def compute_inputs_hash(manifest_path: str) -> str: + h = hashlib.sha256() + with open(manifest_path, "r", encoding="utf-8") as f: + manifest_lines = f.read().splitlines() + + # Sort lines for determinism. Hash the paths (to capture structure) and the + # content. + for line in sorted(manifest_lines): + type_, _, rest = line.partition("|") + h.update(rest.encode("utf-8")) + parts = rest.split("|") + + if type_ == "rf-empty": + continue + + is_symlink_str = parts[0] + path = parts[-1] + + if is_symlink_str == "-1": + is_symlink = not os.path.exists(path) + else: + is_symlink = is_symlink_str == "1" + + if is_symlink: + h.update(os.readlink(path).encode("utf-8")) + else: + with open(path, "rb") as f: + while True: + chunk = f.read(BLOCK_SIZE) + if not chunk: + break + h.update(chunk) + + return h.hexdigest() + + +def expand_template(template_path: str, output_path: str, substitutions: dict) -> None: + with open(template_path, "r", encoding="utf-8") as f: + content = f.read() + + for key, val in substitutions.items(): + content = content.replace(key, val) + + with open(output_path, "w", encoding="utf-8") as f: + f.write(content) + + +def main(): + parser = create_parser() + args = parser.parse_args() + + app_hash = compute_inputs_hash(args.hash_files_manifest) + + substitutions = {"%APP_HASH%": app_hash} + for s in args.substitution: + key, val = s.split("=", 1) + substitutions[key] = val + + expand_template(args.template, args.output, substitutions) + + +if __name__ == "__main__": + main() From dd3614e49e6491efc919a00ee364911f459b693e Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Mon, 6 Apr 2026 08:15:49 -0700 Subject: [PATCH 681/922] chore: finish removing pip_repository_annotations (#3684) It looks like it was only partially deleted. Enough that the deleted packages updater was trying to un-deleted it from the deleted packages list because its WORKSPACE and MODULE files were removed. --- .bazelrc.deleted_packages | 1 - .../pip_repository_annotations/BUILD.bazel | 38 ------ .../pip_repository_annotations_test.py | 118 ------------------ 3 files changed, 157 deletions(-) delete mode 100644 examples/pip_repository_annotations/BUILD.bazel delete mode 100644 examples/pip_repository_annotations/pip_repository_annotations_test.py diff --git a/.bazelrc.deleted_packages b/.bazelrc.deleted_packages index 2d8a8075fa..fb5d2ef0bb 100644 --- a/.bazelrc.deleted_packages +++ b/.bazelrc.deleted_packages @@ -17,7 +17,6 @@ common --deleted_packages=examples/multi_python_versions/requirements common --deleted_packages=examples/multi_python_versions/tests common --deleted_packages=examples/pip_parse common --deleted_packages=examples/pip_parse_vendored -common --deleted_packages=examples/pip_repository_annotations common --deleted_packages=gazelle common --deleted_packages=gazelle/examples/bzlmod_build_file_generation common --deleted_packages=gazelle/examples/bzlmod_build_file_generation/other_module/other_module/pkg diff --git a/examples/pip_repository_annotations/BUILD.bazel b/examples/pip_repository_annotations/BUILD.bazel deleted file mode 100644 index 242cc64c6e..0000000000 --- a/examples/pip_repository_annotations/BUILD.bazel +++ /dev/null @@ -1,38 +0,0 @@ -load("@rules_python//python:pip.bzl", "compile_pip_requirements") -load("@rules_python//python:py_test.bzl", "py_test") - -exports_files( - glob(["data/**"]), - visibility = ["//visibility:public"], -) - -# This rule adds a convenient way to update the requirements file. -compile_pip_requirements( - name = "requirements", - src = "requirements.in", -) - -py_test( - name = "pip_parse_annotations_test", - srcs = ["pip_repository_annotations_test.py"], - env = { - "REQUESTS_PKG_DIR": "pip_requests", - "WHEEL_PKG_DIR": "pip_wheel", - } | select({ - ":is_enable_runfiles_true": {"ENABLE_RUNFILES": "1"}, - "//conditions:default": {"ENABLE_RUNFILES": "0"}, - }), - main = "pip_repository_annotations_test.py", - deps = [ - "@pip_requests//:pkg", - "@pip_wheel//:pkg", - "@rules_python//python/runfiles", - ], -) - -config_setting( - name = "is_enable_runfiles_true", - values = { - "enable_runfiles": "true", - }, -) diff --git a/examples/pip_repository_annotations/pip_repository_annotations_test.py b/examples/pip_repository_annotations/pip_repository_annotations_test.py deleted file mode 100644 index 9b15102599..0000000000 --- a/examples/pip_repository_annotations/pip_repository_annotations_test.py +++ /dev/null @@ -1,118 +0,0 @@ -#!/usr/bin/env python3 -# Copyright 2023 The Bazel Authors. All rights reserved. -# -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - - -import os -import platform -import subprocess -import sys -import unittest -from pathlib import Path - -from python.runfiles import runfiles - - -class PipRepositoryAnnotationsTest(unittest.TestCase): - maxDiff = None - - def wheel_pkg_dir(self) -> str: - env = os.environ.get("WHEEL_PKG_DIR") - self.assertIsNotNone(env) - return env - - def test_build_content_and_data(self): - r = runfiles.Create() - rpath = r.Rlocation("{}/generated_file.txt".format(self.wheel_pkg_dir())) - generated_file = Path(rpath) - self.assertTrue(generated_file.exists()) - - content = generated_file.read_text().rstrip() - self.assertEqual(content, "Hello world from build content file") - - def test_copy_files(self): - r = runfiles.Create() - rpath = r.Rlocation("{}/copied_content/file.txt".format(self.wheel_pkg_dir())) - copied_file = Path(rpath) - self.assertTrue(copied_file.exists()) - - content = copied_file.read_text().rstrip() - self.assertEqual(content, "Hello world from copied file") - - def test_copy_executables(self): - r = runfiles.Create() - rpath = r.Rlocation( - "{}/copied_content/executable{}".format( - self.wheel_pkg_dir(), - ".exe" if platform.system() == "windows" else ".py", - ) - ) - executable = Path(rpath) - self.assertTrue(executable.exists()) - - proc = subprocess.run( - [sys.executable, str(executable)], - check=True, - stdout=subprocess.PIPE, - stderr=subprocess.PIPE, - ) - stdout = proc.stdout.decode("utf-8").strip() - self.assertEqual(stdout, "Hello world from copied executable") - - def test_data_exclude_glob(self): - current_wheel_version = "0.38.4" - - r = runfiles.Create() - dist_info_dir = "{}/site-packages/wheel-{}.dist-info".format( - self.wheel_pkg_dir(), - current_wheel_version, - ) - - # Note: `METADATA` is important as it's consumed by https://docs.python.org/3/library/importlib.metadata.html - # `METADATA` is expected to be there to show dist-info files are included in the runfiles. - metadata_path = r.Rlocation("{}/METADATA".format(dist_info_dir)) - - # However, `WHEEL` was explicitly excluded, so it should be missing - wheel_path = r.Rlocation("{}/WHEEL".format(dist_info_dir)) - - # Because windows does not have `--enable_runfiles` on by default, the - # `runfiles.Rlocation` results will be different on this platform vs - # unix platforms. See `@rules_python//python/runfiles` for more details. - if platform.system() == "Windows" and os.environ["ENABLE_RUNFILES"] == "0": - self.assertIsNotNone(metadata_path) - self.assertIsNone(wheel_path) - else: - self.assertTrue(Path(metadata_path).exists()) - self.assertFalse(Path(wheel_path).exists()) - - def requests_pkg_dir(self) -> str: - env = os.environ.get("REQUESTS_PKG_DIR") - self.assertIsNotNone(env) - return env - - def test_extra(self): - # This test verifies that annotations work correctly for pip packages with extras - # specified, in this case requests[security]. - r = runfiles.Create() - path = "{}/generated_file.txt".format(self.requests_pkg_dir()) - rpath = r.Rlocation(path) - generated_file = Path(rpath) - self.assertTrue(generated_file.exists()) - - content = generated_file.read_text().rstrip() - self.assertEqual(content, "Hello world from requests") - - -if __name__ == "__main__": - unittest.main() From bd7696af6a1aab95116721e079a50b017ebd7175 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Thu, 9 Apr 2026 09:05:17 -0700 Subject: [PATCH 682/922] feat!: make windows use venvs (#3680) This makes Windows use a lite venv to better align it with the unixy implementations. A lite venv is one that has the interpreter and pyvenv.cfg file, but site-packages isn't populated. Unfortunately, it has to recreate the venv at runtime because Bazel incorrectly canonicalizes relative symlinks on Windows. This isn't ideal, however, Windows builds were already going through a zip-unzip process, so creating a handful of symlinks seems like an improvement overall. A particularity of venvs on Windows is that various supporting `.dll` files *must* be in the same directory as `python.exe`. A new attribute (and associated provider plumbing) is added to `py_runtime` to capture these extra files. Note that this **requires** symlink support be enabled in Bazel and Windows. If both aren't enabled, then weird errors will occur because junctions (instead of symlinks) are created that point to files (junctions can only point to directories). Zipapp support for these venvs is also added. This required adding `PyExecutableInfo.venv_interpreter_symlinks` to keep track of relative symlinks that need to be created. Tracking them separately is needed because Bazel has a bug where relative symlinks are made absolute on Windows. The internal plumbing for these symlinks is kept relatively generic to support using such symlinks in site-packages in a future change. Along the way... * Hash-based extract support is added for zipapps passed to Python. * Fixed manually passing a zipapp to Python to bootstrap it. This was previously broken because the launcher doesn't know how to look inside the zip file. * Added `RULES_PYTHON_EXTRACT_ROOT` support when manually passing a zipapp to Python. * Fix several issues with forward-slashes being used on Windows. * Cleanup and improve bootstrap scripts in various ways. * Cleanup some doc in local_runtime_repo. * Have repo_utils.which print nicely formatted path entries on failure. * Add --config=fast-tests to make it easier to run small/medium tests * And --config=testone to make it easier to run a specific test --------- Co-authored-by: Ignas Anikevicius <240938+aignas@users.noreply.github.com> --- .bazelrc | 4 + CHANGELOG.md | 43 ++- .../private/hermetic_runtime_repo_setup.bzl | 12 + python/private/local_runtime_repo.bzl | 12 +- python/private/local_runtime_repo_setup.bzl | 13 + python/private/py_executable.bzl | 336 +++++++++++++----- python/private/py_executable_info.bzl | 20 ++ python/private/py_runtime_info.bzl | 12 +- python/private/py_runtime_rule.bzl | 2 + python/private/python_bootstrap_template.txt | 207 +++++++---- python/private/repo_utils.bzl | 11 +- python/private/stage2_bootstrap_template.py | 19 +- python/private/zipapp/py_zipapp_rule.bzl | 40 ++- python/private/zipapp/zip_main_template.py | 138 ++++--- specialized_configs.bazelrc | 12 + tests/base_rules/py_executable_base_tests.bzl | 17 +- .../run_binary_zip_yes_test.sh | 4 +- tests/bootstrap_impls/sys_path_order_test.py | 10 +- .../system_python_nodeps_test.py | 10 + .../transition/multi_version_tests.bzl | 21 -- tests/integration/local_toolchains/.bazelrc | 2 + .../integration/local_toolchains/MODULE.bazel | 4 +- .../local_toolchains/pbs_archive.bzl | 4 + tests/py_zipapp/BUILD.bazel | 3 - tests/repl/repl_test.py | 38 +- tests/toolchains/defs.bzl | 1 + tests/tools/zipapp/zipper_test.py | 125 +++++++ tools/private/zipapp/zip_main_maker.py | 4 + tools/private/zipapp/zipper.py | 75 +++- 29 files changed, 889 insertions(+), 310 deletions(-) create mode 100644 specialized_configs.bazelrc diff --git a/.bazelrc b/.bazelrc index 49f98ad7a1..771bf0fe90 100644 --- a/.bazelrc +++ b/.bazelrc @@ -42,6 +42,7 @@ common --enable_bzlmod # Local disk cache greatly speeds up builds if the regular cache is lost common --disk_cache=~/.cache/bazel/bazel-disk-cache + # Additional config to use for readthedocs builds. # See .readthedocs.yml for additional flags that can only be determined from # the runtime environment. @@ -54,3 +55,6 @@ common --incompatible_no_implicit_file_export build --lockfile_mode=update +import %workspace%/specialized_configs.bazelrc + +try-import user.bazelrc diff --git a/CHANGELOG.md b/CHANGELOG.md index 7734a45fe4..f487194929 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -64,7 +64,9 @@ END_UNRELEASED_TEMPLATE --windows_enable_symlinks` to your `.bazelrc` to enable Bazel using full symlink support on Windows. * venv-based binaries are created by default ({obj}`--bootstrap_impl=system_python`) - on supported platforms (Linux/Mac with Bazel 8+). + on supported platforms (Linux/Mac with Bazel 8+, or Windows). +* `--build_python_zip` on Windows is ignored. Use {obj}`py_zipapp_binary` to create + zips of Python programs. Other changes: * (pypi) Update dependencies used for `compile_pip_requirements`, building @@ -73,19 +75,21 @@ Other changes: we will from now on fetch the lists of available packages on each index. The used package mappings will be written as facts to the `MODULE.bazel.lock` file on supported bazel versions and it should be done at most once. As a result, - per-package {obj}`experimental_index_url_overrides` is no longer needed if the index URLs are - passed to the `pip.parse` via `experimental_index_url` and `experimental_extra_index_urls`. - What is more, we start implementing the flags for `--index_url` and `--extra_index_urls` more in - line to how it is used in `uv` and `pip`, i.e. we default to `--index_url` if the package is not - found in `--extra_index_urls`. - Fixes - ([#3260](https://github.com/bazel-contrib/rules_python/issues/3260) and + per-package {obj}`experimental_index_url_overrides` is no longer needed if the + index URLs are passed to the `pip.parse` via `experimental_index_url` and + `experimental_extra_index_urls`. What is more, we start implementing the flags + for `--index_url` and `--extra_index_urls` more in line to how it is used in + `uv` and `pip`, i.e. we default to `--index_url` if the package is not found in + `--extra_index_urls`. Fixes + ([#3260](https://github.com/bazel-contrib/rules_python/issues/3260) and [#2632](https://github.com/bazel-contrib/rules_python/issues/2632)). -* (uv) We will now use the download URL specified in the `uv`'s `dist_manifest.json` - file. If you have redirects or blocking rules as part of your downloader setup, - you may need to adjust them. What is more, the default uv version has been bumped - `0.11.2`. -* (runfiles): We are stopping the type annotation testing with `mypy` for Python 3.9. +* (uv) We will now use the download URL specified in the `uv`'s + `dist_manifest.json` file. If you have redirects or blocking rules as part of + your downloader setup, you may need to adjust them. What is more, the default + uv version has been bumped `0.11.2`. +* (runfiles): Type annotations are no longer tested for Python 3.9. +* Windows no longer defaults to creating a zip file and extracting it; a + symlink-based runfiles tree is created, as on unix-like platforms. {#v0-0-0-fixed} ### Fixed @@ -122,11 +126,18 @@ Other changes: {#v0-0-0-added} ### Added * (pypi) Write SimpleAPI contents to the `MODULE.bazel.lock` file if using - {obj}`experimental_index_url` which should speed up consecutive initializations and should no - longer require the network access if the cache is hydrated. - Implements [#2731](https://github.com/bazel-contrib/rules_python/issues/2731). + {obj}`experimental_index_url` which should speed up consecutive + initializations and should no longer require the network access if the cache is + hydrated. Implements + [#2731](https://github.com/bazel-contrib/rules_python/issues/2731). * (wheel) Specifying a path ending in `/` as a destination in `data_files` will now install file(s) to a folder, preserving their basename. +* Various attributes and fields added to support venvs on Windows: + * {obj}`py_runtime.venv_bin_files` and {obj}`PyRuntime.venv_binfiles` + field added to specify additional Python runtime files Windows needs for + venvs. + * {obj}`PyExecutableInfo.venv_interpreter_runfiles`, and + {obj}`PyExecutableInfo.venv_interpreter_symlinks` adde {#v1-9-0} ## [1.9.0] - 2026-02-21 diff --git a/python/private/hermetic_runtime_repo_setup.bzl b/python/private/hermetic_runtime_repo_setup.bzl index d860983e22..20b0324894 100644 --- a/python/private/hermetic_runtime_repo_setup.bzl +++ b/python/private/hermetic_runtime_repo_setup.bzl @@ -240,6 +240,18 @@ def define_hermetic_runtime_toolchain_impl( _IS_FREETHREADED_YES: "cpython-{major}{minor}t".format(**version_dict), _IS_FREETHREADED_NO: "cpython-{major}{minor}".format(**version_dict), }), + # On Windows, a symlink-style venv requires supporting .dll files. + venv_bin_files = select({ + "@platforms//os:windows": native.glob( + include = [ + "*.dll", + ], + # This must be true because glob empty-ness is checked + # during loading phase, before select() filters it out. + allow_empty = True, + ), + "//conditions:default": [], + }), ) py_runtime_pair( diff --git a/python/private/local_runtime_repo.bzl b/python/private/local_runtime_repo.bzl index 6e39152c89..37b7d2b130 100644 --- a/python/private/local_runtime_repo.bzl +++ b/python/private/local_runtime_repo.bzl @@ -74,6 +74,12 @@ def _norm_path(path): def _symlink_libraries(rctx, logger, libraries, shlib_suffix): """Symlinks the shared libraries into the lib/ directory. + Individual files are symlinked instead of the whole directory because + shared_lib_dirs contains multiple search paths for the shared libraries, + and the python files may be missing from any of those directories, and + any of those directories may include non-python runtime libraries, + as would be the case if LIBDIR were, for example, /usr/lib. + Args: rctx: A repository_ctx object logger: A repo_utils.logger object @@ -81,12 +87,6 @@ def _symlink_libraries(rctx, logger, libraries, shlib_suffix): shlib_suffix: Optional. Ensure that the generated symlinks end with this suffix. Returns: A list of library paths (under lib/) linked by the action. - - Individual files are symlinked instead of the whole directory because - shared_lib_dirs contains multiple search paths for the shared libraries, - and the python files may be missing from any of those directories, and - any of those directories may include non-python runtime libraries, - as would be the case if LIBDIR were, for example, /usr/lib. """ result = [] for source in libraries: diff --git a/python/private/local_runtime_repo_setup.bzl b/python/private/local_runtime_repo_setup.bzl index 0922181ffe..5cb7bda200 100644 --- a/python/private/local_runtime_repo_setup.bzl +++ b/python/private/local_runtime_repo_setup.bzl @@ -153,6 +153,19 @@ def define_local_runtime_toolchain_impl( implementation_name = implementation_name, abi_flags = abi_flags, pyc_tag = "{}-{}{}{}".format(implementation_name, major, minor, abi_flags), + venv_bin_files = select({ + "@platforms//os:windows": native.glob( + include = [ + "lib/*.dll", + # The pdb files just provide debugging information + "lib/*.pdb", + ], + # This must be true because glob empty-ness is checked + # during loading phase, before select() filters it out. + allow_empty = True, + ), + "//conditions:default": [], + }), ) py_runtime_pair( diff --git a/python/private/py_executable.bzl b/python/private/py_executable.bzl index 805f95fa4c..6c65bf8f59 100644 --- a/python/private/py_executable.bzl +++ b/python/private/py_executable.bzl @@ -73,6 +73,30 @@ _EXTERNAL_PATH_PREFIX = "external" _ZIP_RUNFILES_DIRECTORY_NAME = "runfiles" _INIT_PY = "__init__.py" +# buildifier: disable=name-conventions +ExplicitSymlink = provider( + doc = """ +A runfile that should be created as a symlink pointing to a specific location. + +This is only needed on Windows, where Bazel doesn't preserve declare_symlink +with relative paths. This is basically manually captures what using +declare_symlink(), symlink() and runfiles like so would capture: + +``` +link = declare_symlink(...) +link_to_path = relative_path(from=link, to=target) +symlink(output=link, target_path=link_to_path) +runfiles.add([link, target]) +``` +""", + fields = { + "files": "depset[File] of files that should be included if this symlink is used", + "link_to_path": "Path the symlink should point to", + "runfiles_path": "runfiles-root-relative path for the symlink", + "venv_path": "venv-root-relative path for the symlink", + }, +) + # Non-Google-specific attributes for executables # These attributes are for rules that accept Python sources. EXECUTABLE_ATTRS = dicts.add( @@ -374,7 +398,10 @@ def _create_executable( extra_default_outputs = [] # NOTE: --build_python_zip defaults to true on Windows - build_zip_enabled = read_possibly_native_flag(ctx, "build_python_zip") + build_zip_enabled = read_possibly_native_flag(ctx, "build_python_zip") and not is_windows + if is_windows: + # The legacy build_python_zip codepath isn't compatible with full venvs on Windows. + build_zip_enabled = False # When --build_python_zip is enabled, then the zip file becomes # one of the default outputs. @@ -473,8 +500,8 @@ WARNING: Target: {} # The interpreter is added this late in the process so that it isn't # added to the zipped files. - if venv and venv.interpreter: - extra_runfiles = extra_runfiles.merge(ctx.runfiles([venv.interpreter])) + if venv and venv.interpreter_runfiles: + extra_runfiles = extra_runfiles.merge(venv.interpreter_runfiles) return struct( # depset[File] of additional files that should be included as default # outputs. @@ -491,6 +518,10 @@ WARNING: Target: {} app_runfiles = app_runfiles.build(ctx), # File|None; the venv `bin/python3` file, if any. venv_python_exe = venv.interpreter if venv else None, + # runfiles|None; runfiles in the venv for the interpreter + venv_interpreter_runfiles = venv.interpreter_runfiles if venv else None, + # depset[ExplicitSymlink]|None; symlinks that should be created + venv_interpreter_symlinks = venv.interpreter_symlinks if venv else None, ) def _create_zip_main(ctx, *, stage2_bootstrap, runtime_details, venv): @@ -523,25 +554,107 @@ def _create_zip_main(ctx, *, stage2_bootstrap, runtime_details, venv): # * https://github.com/python/cpython/blob/main/Modules/getpath.py # * https://github.com/python/cpython/blob/main/Lib/site.py def _create_venv(ctx, output_prefix, imports, runtime_details, add_runfiles_root_to_sys_path, extra_deps): - venv = "_{}.venv".format(output_prefix.lstrip("_")) - - # The pyvenv.cfg file must be present to trigger the venv site hooks. - # Because it's paths are expected to be absolute paths, we can't reliably - # put much in it. See https://github.com/python/cpython/issues/83650 - pyvenv_cfg = ctx.actions.declare_file("{}/pyvenv.cfg".format(venv)) - ctx.actions.write(pyvenv_cfg, "") + venv_ctx_rel_root = "_{}.venv".format(output_prefix.lstrip("_")) + runtime = runtime_details.effective_runtime + if runtime.interpreter: + interpreter_actual_path = runfiles_root_path(ctx, runtime.interpreter.short_path) + else: + interpreter_actual_path = runtime.interpreter_path - is_bootstrap_script = BootstrapImplFlag.get_value(ctx) == BootstrapImplFlag.SCRIPT is_windows = target_platform_has_any_constraint(ctx, ctx.attr._windows_constraints) + if is_windows: + venv_details = _create_venv_windows( + ctx, + venv_ctx_rel_root = venv_ctx_rel_root, + interpreter_actual_path = interpreter_actual_path, + runtime = runtime, + ) + else: + venv_details = _create_venv_unixy( + ctx, + venv_ctx_rel_root = venv_ctx_rel_root, + interpreter_actual_path = interpreter_actual_path, + runtime = runtime, + ) + + site_packages = "{}/{}".format(venv_ctx_rel_root, venv_details.site_packages) + + pth = ctx.actions.declare_file("{}/bazel.pth".format(site_packages)) + ctx.actions.write(pth, "import _bazel_site_init\n") + + site_init = ctx.actions.declare_file("{}/_bazel_site_init.py".format(site_packages)) + computed_subs = ctx.actions.template_dict() + computed_subs.add_joined("%imports%", imports, join_with = ":", map_each = _map_each_identity) + ctx.actions.expand_template( + template = runtime.site_init_template, + output = site_init, + substitutions = { + "%add_runfiles_root_to_sys_path%": add_runfiles_root_to_sys_path, + "%coverage_tool%": _get_coverage_tool_runfiles_path(ctx, runtime), + "%import_all%": "True" if read_possibly_native_flag(ctx, "python_import_all_repositories") else "False", + "%site_init_runfiles_path%": runfiles_root_path(ctx, site_init.short_path), + "%workspace_name%": ctx.workspace_name, + }, + computed_substitutions = computed_subs, + ) + venv_dir_map = { + VenvSymlinkKind.BIN: venv_details.bin_dir, + VenvSymlinkKind.LIB: site_packages, + } + venv_app_files = create_venv_app_files( + ctx, + deps = collect_deps(ctx, extra_deps), + venv_dir_map = venv_dir_map, + ) + + files_without_interpreter = [pth, site_init] + venv_app_files.venv_files + if venv_details.pyvenv_cfg: + files_without_interpreter.append(venv_details.pyvenv_cfg) + + return struct( + # File or None; the `bin/python3` executable in the venv. + # None if a full venv isn't created. + interpreter = venv_details.interpreter, + # Files in the venv that need to be created for the interpreter to work + interpreter_runfiles = venv_details.interpreter_runfiles, + # depset[ExplicitSymlink] of symlinks to create. + # This is only used when declare_symlink() can't be used to represent + # creating such a link (i.e Windows) + interpreter_symlinks = venv_details.interpreter_symlinks, + # bool; True if the venv should be recreated at runtime + recreate_venv_at_runtime = venv_details.recreate_venv_at_runtime, + # Runfiles root relative path or absolute path + interpreter_actual_path = interpreter_actual_path, + files_without_interpreter = files_without_interpreter, + # string; venv-relative path to the site-packages directory. + venv_site_packages = venv_details.site_packages, + # string; runfiles-root relative path to venv root. + venv_root = runfiles_root_path( + ctx, + paths.join( + py_internal.get_label_repo_runfiles_path(ctx.label), + venv_ctx_rel_root, + ), + ), + # venv files for user library dependencies (files that are specific + # to the executable bootstrap and python runtime aren't here). + # `root_symlinks` should be used, otherwise, with symlinks files always go + # to `_main` prefix, and binaries from non-root module become broken. + lib_runfiles = ctx.runfiles( + root_symlinks = venv_app_files.runfiles_symlinks, + ), + ) + +def _create_venv_unixy(ctx, *, venv_ctx_rel_root, runtime, interpreter_actual_path): + interpreter_runfiles = builders.RunfilesBuilder() + is_bootstrap_script = BootstrapImplFlag.get_value(ctx) == BootstrapImplFlag.SCRIPT create_full_venv = True # The legacy build_python_zip codepath (enabled by default on windows) isn't # compatible with full venv. # TODO: Use non-build_python_zip codepath for Windows - if is_windows: - create_full_venv = False - elif not rp_config.bazel_8_or_later and not is_bootstrap_script: + if not rp_config.bazel_8_or_later and not is_bootstrap_script: # Full venv for Bazel 7 + system_python is disabled because packaging # it using build_python_zip=true or rules_pkg breaks. # * Using build_python_zip=true breaks because the legacy zipapp support @@ -557,24 +670,18 @@ def _create_venv(ctx, output_prefix, imports, runtime_details, add_runfiles_root # The pyvenv.cfg file must be present to trigger the venv site hooks. # Because it's paths are expected to be absolute paths, we can't reliably # put much in it. See https://github.com/python/cpython/issues/83650 - pyvenv_cfg = ctx.actions.declare_file("{}/pyvenv.cfg".format(venv)) + pyvenv_cfg = ctx.actions.declare_file("{}/pyvenv.cfg".format(venv_ctx_rel_root)) ctx.actions.write(pyvenv_cfg, "") else: pyvenv_cfg = None - runtime = runtime_details.effective_runtime venvs_use_declare_symlink_enabled = ( VenvsUseDeclareSymlinkFlag.get_value(ctx) == VenvsUseDeclareSymlinkFlag.YES ) - recreate_venv_at_runtime = False - if runtime.interpreter: - interpreter_actual_path = runfiles_root_path(ctx, runtime.interpreter.short_path) - else: - interpreter_actual_path = runtime.interpreter_path - - bin_dir = "{}/bin".format(venv) + recreate_venv_at_runtime = False + bin_dir = "{}/bin".format(venv_ctx_rel_root) if create_full_venv: # Some wrappers around the interpreter (e.g. pyenv) use the program # name to decide what to do, so preserve the name. @@ -587,7 +694,6 @@ def _create_venv(ctx, output_prefix, imports, runtime_details, add_runfiles_root # needed or used at runtime. However, the zip code uses the interpreter # File object to figure out some paths. interpreter = ctx.actions.declare_file("{}/{}".format(bin_dir, py_exe_basename)) - ctx.actions.write(interpreter, "actual:{}".format(interpreter_actual_path)) elif runtime.interpreter: @@ -596,6 +702,7 @@ def _create_venv(ctx, output_prefix, imports, runtime_details, add_runfiles_root # in runfiles is always a symlink. An RBE implementation, for example, # may choose to write what symlink() points to instead. interpreter = ctx.actions.declare_symlink("{}/{}".format(bin_dir, py_exe_basename)) + interpreter_runfiles.add(interpreter) rel_path = relative_path( # dirname is necessary because a relative symlink is relative to @@ -603,10 +710,10 @@ def _create_venv(ctx, output_prefix, imports, runtime_details, add_runfiles_root from_ = paths.dirname(runfiles_root_path(ctx, interpreter.short_path)), to = interpreter_actual_path, ) - ctx.actions.symlink(output = interpreter, target_path = rel_path) else: interpreter = ctx.actions.declare_symlink("{}/{}".format(bin_dir, py_exe_basename)) + interpreter_runfiles.add(interpreter) ctx.actions.symlink(output = interpreter, target_path = runtime.interpreter_path) else: interpreter = None @@ -625,67 +732,108 @@ def _create_venv(ctx, output_prefix, imports, runtime_details, add_runfiles_root if "t" in runtime.abi_flags: version += "t" - venv_site_packages = "lib/python{}/site-packages".format(version) - site_packages = "{}/{}".format(venv, venv_site_packages) - pth = ctx.actions.declare_file("{}/bazel.pth".format(site_packages)) - ctx.actions.write(pth, "import _bazel_site_init\n") - - site_init = ctx.actions.declare_file("{}/_bazel_site_init.py".format(site_packages)) - computed_subs = ctx.actions.template_dict() - computed_subs.add_joined("%imports%", imports, join_with = ":", map_each = _map_each_identity) - ctx.actions.expand_template( - template = runtime.site_init_template, - output = site_init, - substitutions = { - "%add_runfiles_root_to_sys_path%": add_runfiles_root_to_sys_path, - "%coverage_tool%": _get_coverage_tool_runfiles_path(ctx, runtime), - "%import_all%": "True" if read_possibly_native_flag(ctx, "python_import_all_repositories") else "False", - "%site_init_runfiles_path%": runfiles_root_path(ctx, site_init.short_path), - "%workspace_name%": ctx.workspace_name, - }, - computed_substitutions = computed_subs, + site_packages = "lib/python{}/site-packages".format(version) + return _venv_details( + interpreter = interpreter, + pyvenv_cfg = pyvenv_cfg, + site_packages = site_packages, + bin_dir = bin_dir, + recreate_venv_at_runtime = recreate_venv_at_runtime, + interpreter_runfiles = interpreter_runfiles.build(ctx), + interpreter_symlinks = depset(), ) - venv_dir_map = { - VenvSymlinkKind.BIN: bin_dir, - VenvSymlinkKind.LIB: site_packages, - } - venv_app_files = create_venv_app_files( - ctx, - deps = collect_deps(ctx, extra_deps), - venv_dir_map = venv_dir_map, - ) +def _create_venv_windows(ctx, *, venv_ctx_rel_root, runtime, interpreter_actual_path): + interpreter_runfiles = builders.RunfilesBuilder() + interpreter_symlinks = builders.DepsetBuilder() - files_without_interpreter = [pth, site_init] + venv_app_files.venv_files - if pyvenv_cfg: - files_without_interpreter.append(pyvenv_cfg) + # Some wrappers around the interpreter (e.g. pyenv) use the program + # name to decide what to do, so preserve the name. + py_exe_basename = paths.basename(interpreter_actual_path) + venv_bin_rel_path = "Scripts" + venv_bin_ctx_rel_path = "{}/{}".format(venv_ctx_rel_root, venv_bin_rel_path) + if runtime.interpreter: + venv_rel_path = paths.join(venv_bin_rel_path, py_exe_basename) + venv_ctx_rel_path = paths.join(venv_ctx_rel_root, venv_rel_path) + interpreter = ctx.actions.declare_file(venv_ctx_rel_path) + interpreter_runfiles.add(interpreter) + ctx.actions.symlink(output = interpreter, target_file = runtime.interpreter) + + rf_path = runfiles_root_path(ctx, interpreter.short_path) + interpreter_symlinks.add(ExplicitSymlink( + runfiles_path = rf_path, + venv_path = venv_rel_path, + link_to_path = interpreter_actual_path, + files = depset([runtime.interpreter]), + )) + else: + # It's OK to use declare_symlink here because an absolute path + # will be written to it, so Bazel won't mangle it. + interpreter = ctx.actions.declare_symlink("{}/{}".format(venv_bin_ctx_rel_path, py_exe_basename)) + interpreter_runfiles.add(interpreter) + ctx.actions.symlink(output = interpreter, target_path = runtime.interpreter_path) + + # NOTE: The .dll files must exist, however, they may not be known at build time + # if the interpreter is resolved at runtime. + for f in runtime.venv_bin_files: + venv_rel_path = paths.join(venv_bin_rel_path, f.basename) + venv_ctx_rel_path = paths.join(venv_ctx_rel_root, venv_rel_path) + + venv_file = ctx.actions.declare_file(venv_ctx_rel_path) + ctx.actions.symlink(output = venv_file, target_file = f) + + interpreter_runfiles.add(venv_file) + + rf_path = runfiles_root_path(ctx, venv_file.short_path) + interpreter_symlinks.add(ExplicitSymlink( + runfiles_path = rf_path, + venv_path = venv_rel_path, + link_to_path = runfiles_root_path(ctx, f.short_path), + files = depset([f]), + )) + # See site.py logic: Windows uses a version/build agnostic site-packages path + site_packages = "Lib/site-packages" + + return _venv_details( + interpreter = interpreter, + pyvenv_cfg = None, + site_packages = site_packages, + bin_dir = venv_bin_ctx_rel_path, + recreate_venv_at_runtime = True, + interpreter_runfiles = interpreter_runfiles.build(ctx), + interpreter_symlinks = interpreter_symlinks.build(), + ) + +def _venv_details( + *, + interpreter, + pyvenv_cfg, + site_packages, + bin_dir, + recreate_venv_at_runtime, + interpreter_runfiles, + interpreter_symlinks): + """Helper to create a struct of platform-specific venv details.""" return struct( - # File or None; the `bin/python3` executable in the venv. - # None if a full venv isn't created. + # File; the `bin/python` executable (or equivalent) within the venv. interpreter = interpreter, - # bool; True if the venv should be recreated at runtime + # File|None; the pyvenv.cfg file, if any. May be none, in which case, + # it's expected that one will be created at runtime. + pyvenv_cfg = pyvenv_cfg, + # str; venv-relative path to the site-packages directory + site_packages = site_packages, + # str; ctx-relative path to the venv's bin directory. + bin_dir = bin_dir, + # bool; True if the venv needs to be recreated at runtime (because the + # build-time construction isn't sufficient). False if the build-time + # constructed venv is sufficient. recreate_venv_at_runtime = recreate_venv_at_runtime, - # Runfiles root relative path or absolute path - interpreter_actual_path = interpreter_actual_path, - files_without_interpreter = files_without_interpreter, - # string; venv-relative path to the site-packages directory. - venv_site_packages = venv_site_packages, - # string; runfiles-root relative path to venv root. - venv_root = runfiles_root_path( - ctx, - paths.join( - py_internal.get_label_repo_runfiles_path(ctx.label), - venv, - ), - ), - # venv files for user library dependencies (files that are specific - # to the executable bootstrap and python runtime aren't here). - # `root_symlinks` should be used, otherwise, with symlinks files always go - # to `_main` prefix, and binaries from non-root module become broken. - lib_runfiles = ctx.runfiles( - root_symlinks = venv_app_files.runfiles_symlinks, - ), + # runfiles; runfiles for interpreter-specific files in the venv. + interpreter_runfiles = interpreter_runfiles, + # depset[ExplicitSymlink] of symlinks specific + # to the interpreter. Only used for Windows. + interpreter_symlinks = interpreter_symlinks, ) def _map_each_identity(v): @@ -787,6 +935,14 @@ def _create_stage1_bootstrap( "%venv_rel_site_packages%": venv.venv_site_packages if venv else "", "%workspace_name%": ctx.workspace_name, } + computed_subs = ctx.actions.template_dict() + if venv: + computed_subs.add_joined( + "%runtime_venv_symlinks%", + venv.interpreter_symlinks, + join_with = "\n", + map_each = _map_runtime_venv_symlink, + ) if stage2_bootstrap: subs["%stage2_bootstrap%"] = runfiles_root_path(ctx, stage2_bootstrap.short_path) @@ -817,9 +973,13 @@ def _create_stage1_bootstrap( template = template, output = output, substitutions = subs, + computed_substitutions = computed_subs, is_executable = True, ) +def _map_runtime_venv_symlink(entry): + return entry.venv_path + "|" + entry.link_to_path + def _create_zip_file(ctx, *, output, zip_main, runfiles): """Create a Python zipapp (zip with __main__.py entry point).""" workspace_name = ctx.workspace_name @@ -1111,6 +1271,8 @@ def py_executable_base_impl(ctx, *, semantics, is_test, inherited_environment = stage2_bootstrap = exec_result.stage2_bootstrap, app_runfiles = app_runfiles, venv_python_exe = exec_result.venv_python_exe, + venv_interpreter_runfiles = exec_result.venv_interpreter_runfiles, + venv_interpreter_symlinks = exec_result.venv_interpreter_symlinks, interpreter_args = ctx.attr.interpreter_args, ) @@ -1657,6 +1819,8 @@ def _create_providers( stage2_bootstrap, app_runfiles, venv_python_exe, + venv_interpreter_runfiles, + venv_interpreter_symlinks, interpreter_args): """Creates the providers an executable should return. @@ -1689,6 +1853,10 @@ def _create_providers( stage2_bootstrap: File; the stage 2 bootstrap script. app_runfiles: runfiles; the runfiles for the application (deps, etc). venv_python_exe: File; the python executable in the venv. + venv_interpreter_runfiles: runfiles; runfiles specific to the interpreter + for the venv. + venv_interpreter_symlinks: depset[ExplicitSymlink]; interpreter-specific symlinks + to create for the venv. interpreter_args: list of strings; arguments to pass to the interpreter. Returns: @@ -1710,14 +1878,16 @@ def _create_providers( create_instrumented_files_info(ctx), _create_run_environment_info(ctx, inherited_environment), PyExecutableInfo( - main = main_py, - runfiles_without_exe = runfiles_details.runfiles_without_exe, + app_runfiles = app_runfiles, build_data_file = runfiles_details.build_data_file, + interpreter_args = interpreter_args, interpreter_path = runtime_details.executable_interpreter_path, + main = main_py, + runfiles_without_exe = runfiles_details.runfiles_without_exe, stage2_bootstrap = stage2_bootstrap, - app_runfiles = app_runfiles, + venv_interpreter_runfiles = venv_interpreter_runfiles, + venv_interpreter_symlinks = venv_interpreter_symlinks, venv_python_exe = venv_python_exe, - interpreter_args = interpreter_args, ), ] diff --git a/python/private/py_executable_info.bzl b/python/private/py_executable_info.bzl index defbd3a05b..0c5931cecd 100644 --- a/python/private/py_executable_info.bzl +++ b/python/private/py_executable_info.bzl @@ -77,6 +77,26 @@ implementation isn't being used. :::{versionadded} 1.9.0 ::: +""", + "venv_interpreter_runfiles": """ +:type: runfiles | None + +Runfiles that are specific to the interpreter within the venv. + +:::{versionadded} VERSION_NEXT_FEATURE +::: +""", + "venv_interpreter_symlinks": """ +:type: depset[ExplicitSymlink] | None + +Symlinks that are specific to the interpreter within the venv. + +Only used with Windows for files that would have used `declare_symlink()` +to create relative symlinks. These may overlap with paths in runfiles; it's +up to the consumer to determine how to handle such overlaps. + +:::{versionadded} VERSION_NEXT_FEATURE +::: """, "venv_python_exe": """ :type: File | None diff --git a/python/private/py_runtime_info.bzl b/python/private/py_runtime_info.bzl index af4e7f0596..8fdbd7bfe2 100644 --- a/python/private/py_runtime_info.bzl +++ b/python/private/py_runtime_info.bzl @@ -66,7 +66,8 @@ def _PyRuntimeInfo_init( zip_main_template = None, abi_flags = "", site_init_template = None, - supports_build_time_venv = True): + supports_build_time_venv = True, + venv_bin_files = None): if (interpreter_path and interpreter) or (not interpreter_path and not interpreter): fail("exactly one of interpreter or interpreter_path must be specified") @@ -119,6 +120,7 @@ def _PyRuntimeInfo_init( "stage2_bootstrap_template": stage2_bootstrap_template, "stub_shebang": stub_shebang, "supports_build_time_venv": supports_build_time_venv, + "venv_bin_files": venv_bin_files, "zip_main_template": zip_main_template, } @@ -334,6 +336,14 @@ to meet two criteria: :::{versionadded} 1.5.0 ::: +""", + "venv_bin_files": """ +:type: list[File] + +Files that should be added to the venv's `bin/` (or platform-specific equivalent) +directory (using the file's basename). + +:::{versionadded} VERSION_NEXT_FEATURE """, "zip_main_template": """ :type: File diff --git a/python/private/py_runtime_rule.bzl b/python/private/py_runtime_rule.bzl index 09e245a58e..dfc915f463 100644 --- a/python/private/py_runtime_rule.bzl +++ b/python/private/py_runtime_rule.bzl @@ -130,6 +130,7 @@ def _py_runtime_impl(ctx): abi_flags = abi_flags, site_init_template = ctx.file.site_init_template, supports_build_time_venv = ctx.attr.supports_build_time_venv, + venv_bin_files = ctx.files.venv_bin_files, )) providers = [ @@ -360,6 +361,7 @@ See {obj}`PyRuntimeInfo.supports_build_time_venv` for docs. """, default = True, ), + "venv_bin_files": attr.label_list(allow_files = True), "zip_main_template": attr.label( default = "//python/private/zipapp:zip_main_template", allow_single_file = True, diff --git a/python/private/python_bootstrap_template.txt b/python/private/python_bootstrap_template.txt index 51c9013e9d..accc11e0df 100644 --- a/python/private/python_bootstrap_template.txt +++ b/python/private/python_bootstrap_template.txt @@ -5,13 +5,13 @@ from __future__ import absolute_import from __future__ import division from __future__ import print_function -import sys +# Generated file from @rules_python//python/private:python_bootstrap_template.txt +from os.path import dirname, join, basename, normpath import os -from os.path import dirname, join, basename -import subprocess -import uuid import shutil +import subprocess +import sys # NOTE: The sentinel strings are split (e.g., "%stage2" + "_bootstrap%") so that # the substitution logic won't replace them. This allows runtime detection of @@ -74,22 +74,41 @@ if _INTERPRETER_ARGS_RAW == _INTERPRETER_ARGS_SENTINEL: else: INTERPRETER_ARGS = [arg for arg in _INTERPRETER_ARGS_RAW.split("\n") if arg] +# Symlinks to create at runtime in the venv. +# Line delimited entries +# Each entry is "venv_relative_path|path_to_symlink_to" +RUNTIME_VENV_SYMLINKS = """ +%runtime_venv_symlinks% +""".strip().split("\n") +RUNTIME_VENV_SYMLINKS = dict(line.split("|") for line in RUNTIME_VENV_SYMLINKS if line) + ADDITIONAL_INTERPRETER_ARGS = os.environ.get("RULES_PYTHON_ADDITIONAL_INTERPRETER_ARGS", "") EXTRACT_ROOT = os.environ.get("RULES_PYTHON_EXTRACT_ROOT") -def is_running_from_zip(): - return IS_ZIPFILE - -if is_running_from_zip(): +if IS_ZIPFILE: import shutil import tempfile import zipfile else: import re -# Return True if running on Windows -def is_windows(): - return os.name == 'nt' +IS_WINDOWS = os.name == "nt" + +BIN_DIR_NAME = "bin" if not IS_WINDOWS else "Scripts" + +# Windows APIs can be picky about slashes depending on the context, +# so convert to backslashes to avoid any issues. +if IS_WINDOWS: + def norm_slashes(s): + return s.replace("/", "\\") + + STAGE2_BOOTSTRAP = norm_slashes(STAGE2_BOOTSTRAP) + PYTHON_BINARY = norm_slashes(PYTHON_BINARY) + PYTHON_BINARY_ACTUAL = norm_slashes(PYTHON_BINARY_ACTUAL) + RUNTIME_VENV_SYMLINKS = { + norm_slashes(k): norm_slashes(v) + for k, v in RUNTIME_VENV_SYMLINKS.items() + } def get_windows_path_with_unc_prefix(path): """Adds UNC prefix after getting a normalized absolute Windows path. @@ -100,7 +119,7 @@ def get_windows_path_with_unc_prefix(path): # No need to add prefix for non-Windows platforms. # And \\?\ doesn't work in python 2 or on mingw - if not is_windows() or sys.version_info[0] < 3: + if not IS_WINDOWS or sys.version_info[0] < 3: return path # Starting in Windows 10, version 1607(OS build 14393), MAX_PATH limitations have been @@ -131,12 +150,6 @@ def get_windows_path_with_unc_prefix(path): # os.path.abspath returns a normalized absolute path return unicode_prefix + os.path.abspath(path) -def has_windows_executable_extension(path): - return path.endswith('.exe') or path.endswith('.com') or path.endswith('.bat') - -if PYTHON_BINARY and is_windows() and not has_windows_executable_extension(PYTHON_BINARY): - PYTHON_BINARY = PYTHON_BINARY + '.exe' - def search_path(name): """Finds a file in a given search path.""" search_path = os.getenv('PATH', os.defpath).split(os.pathsep) @@ -156,25 +169,27 @@ def find_python_binary(runfiles_root): def print_verbose(*args, mapping=None, values=None): - if os.environ.get("RULES_PYTHON_BOOTSTRAP_VERBOSE"): - if mapping is not None: - for key, value in sorted((mapping or {}).items()): - print( - "bootstrap: stage 1:", - *(list(args) + ["{}={}".format(key, repr(value))]), - file=sys.stderr, - flush=True - ) - elif values is not None: - for i, v in enumerate(values): - print( - "bootstrap: stage 1:", - *(list(args) + ["[{}] {}".format(i, repr(v))]), - file=sys.stderr, - flush=True - ) - else: - print("bootstrap: stage 1:", *args, file=sys.stderr, flush=True) + if not os.environ.get("RULES_PYTHON_BOOTSTRAP_VERBOSE"): + return + + if mapping is not None: + for key, value in sorted((mapping or {}).items()): + print( + "bootstrap: stage 1:", + *(list(args) + ["{}={}".format(key, repr(value))]), + file=sys.stderr, + flush=True + ) + elif values is not None: + for i, v in enumerate(values): + print( + "bootstrap: stage 1:", + *(list(args) + ["[{}] {}".format(i, repr(v))]), + file=sys.stderr, + flush=True + ) + else: + print("bootstrap: stage 1:", *args, file=sys.stderr, flush=True) def find_binary(runfiles_root, bin_name): """Finds the real binary if it's not a normal absolute path.""" @@ -225,18 +240,18 @@ def find_runfiles_root(main_rel_path): # On Windows, the path may contain both forward and backslashes. # Normalize to the OS separator because the regex used later assumes # the OS-specific separator. - if is_windows(): + if IS_WINDOWS: stub_filename = stub_filename.replace("/", os.sep) if not os.path.isabs(stub_filename): stub_filename = os.path.join(os.getcwd(), stub_filename) while True: - runfiles_root = stub_filename + ('.exe' if is_windows() else '') + '.runfiles' + runfiles_root = stub_filename + ('.exe' if IS_WINDOWS else '') + '.runfiles' if os.path.isdir(runfiles_root): return runfiles_root - runfiles_pattern = r'(.*\.runfiles)' + (r'\\' if is_windows() else '/') + '.*' + runfiles_pattern = r'(.*\.runfiles)' + (r'\\' if IS_WINDOWS else '/') + '.*' matchobj = re.match(runfiles_pattern, stub_filename) if matchobj: return matchobj.group(1) @@ -293,56 +308,102 @@ def create_runfiles_root(): # important that deletion code be in sync with this directory structure return os.path.join(temp_dir, 'runfiles') -def _create_venv(runfiles_root): - runfiles_venv = join(runfiles_root, dirname(dirname(PYTHON_BINARY))) +def _create_venv(runfiles_root, delete_dirs): + rel_runfiles_venv = dirname(dirname(PYTHON_BINARY)) + runfiles_venv = join(runfiles_root, rel_runfiles_venv) + print_verbose("create_venv: runfiles venv:", runfiles_venv) if EXTRACT_ROOT: - venv = join(EXTRACT_ROOT, runfiles_venv) + venv = join(EXTRACT_ROOT, rel_runfiles_venv) os.makedirs(venv, exist_ok=True) - cleanup_dir = None else: import tempfile venv = tempfile.mkdtemp("", f"bazel.{basename(runfiles_venv)}.") - cleanup_dir = venv + delete_dirs.append(venv) + + print_verbose("create_venv: created venv:", venv) python_exe_actual = find_binary(runfiles_root, PYTHON_BINARY_ACTUAL) + if python_exe_actual is None: + raise AssertionError('Could not find python binary: ' + repr(PYTHON_BINARY_ACTUAL)) # See stage1_bootstrap_template.sh for details on this code path. In short, # this handles when the build-time python version doesn't match runtime # and if the initially resolved python_exe_actual is a wrapper script. if RESOLVE_PYTHON_BINARY_AT_RUNTIME: + venv_src = venv.replace("\\", "\\\\") # Escape backslashes src = f""" import sys, site print(sys.executable) -print(site.getsitepackages(["{venv}"])[-1]) +print(sys.base_prefix) +print(site.getsitepackages(["{venv_src}"])[-1]) """ + print_verbose("prog:", src) output = subprocess.check_output([python_exe_actual, "-I"], shell=True, encoding = "utf8", input=src) output = output.strip().split("\n") python_exe_actual = output[0] - venv_site_packages = output[1] + python_home = output[1] if IS_WINDOWS else None + venv_site_packages = output[2] os.makedirs(dirname(venv_site_packages), exist_ok=True) runfiles_venv_site_packages = join(runfiles_venv, VENV_REL_SITE_PACKAGES) else: - python_exe_actual = find_binary(runfiles_root, PYTHON_BINARY_ACTUAL) + # On unixy, Python can find home based on the symlink. + python_home = dirname(python_exe_actual) if IS_WINDOWS else None venv_site_packages = join(venv, "lib") runfiles_venv_site_packages = join(runfiles_venv, "lib") - if python_exe_actual is None: - raise AssertionError('Could not find python binary: ' + repr(PYTHON_BINARY_ACTUAL)) - - venv_bin = join(venv, "bin") + venv_bin = join(venv, BIN_DIR_NAME) try: os.mkdir(venv_bin) except FileExistsError as e: pass - # Match the basename; some tools, e.g. pyvenv key off the executable name venv_python_exe = join(venv_bin, os.path.basename(python_exe_actual)) _symlink_exist_ok(from_=venv_python_exe, to=python_exe_actual) + + # Windows requires supporting .dll files in the venv bin dir, but + # they aren't known in advance when the interpreter is resolved at runtime. + if RESOLVE_PYTHON_BINARY_AT_RUNTIME and IS_WINDOWS: + files = os.listdir(python_home) + for f in files: + if not f.endswith((".dll", ".pdb")): continue + venv_path = join(venv, BIN_DIR_NAME, f) + target = join(python_home, f) + _symlink_exist_ok(from_=venv_path, to=target) + + runfiles_venv_bin = join(runfiles_venv, BIN_DIR_NAME) + # The runfiles bin directory may not exist if it's empty, e.g. + # supports_build_time_venv=False and the interpreter is resolved at + # runtime. + if os.path.exists(runfiles_venv_bin): + # Add any missing build-time entries under bin/. Do this before + # the manual symlink creation to better mimic what the + # regular runfiles directory looks like. + for f_basename in os.listdir(runfiles_venv_bin): + venv_path = join(venv, BIN_DIR_NAME, f_basename) + target = join(runfiles_venv_bin, f_basename) + _symlink_exist_ok(from_=venv_path, to=target) + + for venv_rel_path, link_to_rf_path in RUNTIME_VENV_SYMLINKS.items(): + venv_abs_path = join(venv, venv_rel_path) + link_to = normpath(join(runfiles_venv, link_to_rf_path)) + os.makedirs(dirname(venv_abs_path), exist_ok=True) + _symlink_exist_ok(from_=venv_abs_path, to=link_to) + _symlink_exist_ok(from_=join(venv, "lib"), to=join(runfiles_venv, "lib")) _symlink_exist_ok(from_=venv_site_packages, to=runfiles_venv_site_packages) - _symlink_exist_ok(from_=join(venv, "pyvenv.cfg"), to=join(runfiles_venv, "pyvenv.cfg")) - return cleanup_dir, venv_python_exe + + if IS_WINDOWS: + print_verbose("create_venv: pyvenv.cfg home: ", python_home) + venv_pyvenv_cfg = join(venv, "pyvenv.cfg") + with open(venv_pyvenv_cfg, "w") as fp: + # Until Windows supports a build-time generated venv using symlinks + # to directories, we have to write the full, absolute, path to PYTHONHOME + # so that support directories (e.g. DLLs, libs) can be found. + fp.write("home = {}\n".format(python_home)) + else: + _symlink_exist_ok(from_=join(venv, "pyvenv.cfg"), to=join(runfiles_venv, "pyvenv.cfg")) + return venv_python_exe def runfiles_envvar(runfiles_root): """Finds the runfiles manifest or the runfiles directory. @@ -363,7 +424,7 @@ def runfiles_envvar(runfiles_root): return ('RUNFILES_DIR', runfiles) # If running from a zip, there's no manifest file. - if is_running_from_zip(): + if IS_ZIPFILE: return ('RUNFILES_DIR', runfiles_root) # Look for the runfiles "output" manifest, argv[0] + ".runfiles_manifest" @@ -427,7 +488,7 @@ def execute_file(python_program, main_filename, args, env, runfiles_root, # can't execv because we need control to return here. This only # happens for targets built in the host config. # - if not (is_windows() or workspace or delete_dirs): + if not (IS_WINDOWS or workspace or delete_dirs): _run_execv(python_program, argv, env) print_verbose("run: subproc: environ:", mapping=os.environ) @@ -435,6 +496,7 @@ def execute_file(python_program, main_filename, args, env, runfiles_root, print_verbose("run: subproc: argv:", values=argv) ret_code = subprocess.call( argv, env=env, cwd=workspace) + print_verbose("run: subproc: exit code:", ret_code) if delete_dirs: for delete_dir in delete_dirs: @@ -463,19 +525,21 @@ def _symlink_exist_ok(*, from_, to): pass - def main(): print_verbose("sys.version:", sys.version) print_verbose("initial argv:", values=sys.argv) print_verbose("initial cwd:", os.getcwd()) print_verbose("initial environ:", mapping=os.environ) print_verbose("initial sys.path:", values=sys.path) - print_verbose("STAGE2_BOOTSTRAP:", STAGE2_BOOTSTRAP) + print_verbose("IS_ZIPFILE:", IS_ZIPFILE) print_verbose("PYTHON_BINARY:", PYTHON_BINARY) print_verbose("PYTHON_BINARY_ACTUAL:", PYTHON_BINARY_ACTUAL) - print_verbose("IS_ZIPFILE:", IS_ZIPFILE) print_verbose("RECREATE_VENV_AT_RUNTIME:", RECREATE_VENV_AT_RUNTIME) - print_verbose("WORKSPACE_NAME :", WORKSPACE_NAME ) + print_verbose("RESOLVE_PYTHON_BINARY_AT_RUNTIME:", RESOLVE_PYTHON_BINARY_AT_RUNTIME) + print_verbose("RUNTIME_VENV_SYMLINKS size:", len(RUNTIME_VENV_SYMLINKS)) + print_verbose("STAGE2_BOOTSTRAP:", STAGE2_BOOTSTRAP) + print_verbose("VENV_REL_SITE_PACKAGES:", VENV_REL_SITE_PACKAGES) + print_verbose("WORKSPACE_NAME:", WORKSPACE_NAME ) print_verbose("bootstrap sys.executable:", sys.executable) print_verbose("bootstrap sys._base_executable:", sys._base_executable) print_verbose("bootstrap sys.version:", sys.version) @@ -494,7 +558,7 @@ def main(): delete_dirs = [] - if is_running_from_zip(): + if IS_ZIPFILE: runfiles_root = create_runfiles_root() # NOTE: dirname() is called because create_runfiles_root() creates a # sub-directory within a temporary directory, and we want to remove the @@ -523,20 +587,19 @@ def main(): assert os.access(main_filename, os.R_OK), \ 'Cannot exec() %r: file not readable.' % main_filename - python_program = find_python_binary(runfiles_root) - if python_program is None: - raise AssertionError("Could not find python binary: {} or {}".format( - repr(PYTHON_BINARY), - repr(PYTHON_BINARY_ACTUAL) - )) - if RECREATE_VENV_AT_RUNTIME: # When the venv is created at runtime, python_program is PYTHON_BINARY_ACTUAL # so we have to re-point it to the symlink in the venv - venv, python_program = _create_venv(runfiles_root) - delete_dirs.append(venv) + python_program = _create_venv(runfiles_root, delete_dirs) else: python_program = find_python_binary(runfiles_root) + if python_program is None: + raise AssertionError("Could not find python binary: {} or {}".format( + repr(PYTHON_BINARY), + repr(PYTHON_BINARY_ACTUAL) + )) + + python_program = get_windows_path_with_unc_prefix(python_program) # Some older Python versions on macOS (namely Python 3.7) may unintentionally # leave this environment variable set after starting the interpreter, which @@ -548,7 +611,7 @@ def main(): new_env.update((key, val) for key, val in os.environ.items() if key not in new_env) workspace = None - if is_running_from_zip(): + if IS_ZIPFILE: # If RUN_UNDER_RUNFILES equals 1, it means we need to # change directory to the right runfiles directory. # (So that the data files are accessible) diff --git a/python/private/repo_utils.bzl b/python/private/repo_utils.bzl index 00f43f3521..a558fa08e1 100644 --- a/python/private/repo_utils.bzl +++ b/python/private/repo_utils.bzl @@ -311,12 +311,19 @@ def _which_unchecked(mrctx, binary_name): ) def _which_describe_failure(binary_name, path): + if "\\" in path or ";" in path: + path_parts = path.split(";") + else: + path_parts = path.split(":") + for i, v in enumerate(path_parts): + path_parts[i] = " [{}]: {}".format(i, v) return ( "Unable to find the binary '{binary_name}' on PATH.\n" + - " PATH = {path}" + " PATH entries:\n" + + "{path_str}" ).format( binary_name = binary_name, - path = path, + path_str = "\n".join(path_parts), ) def _mkdir(mrctx, path): diff --git a/python/private/stage2_bootstrap_template.py b/python/private/stage2_bootstrap_template.py index 3c2d6807cc..dec356d3ac 100644 --- a/python/private/stage2_bootstrap_template.py +++ b/python/private/stage2_bootstrap_template.py @@ -67,7 +67,7 @@ def get_build_data(self): from python.runfiles import runfiles rlocation_path = self.BUILD_DATA_FILE path = runfiles.Create().Rlocation(rlocation_path) - if is_windows(): + if IS_WINDOWS: path = os.path.normpath(path) try: # Use utf-8-sig to handle Windows BOM @@ -84,17 +84,14 @@ def get_build_data(self): sys.modules["bazel_binary_info"] = BazelBinaryInfoModule("bazel_binary_info") - -# Return True if running on Windows -def is_windows(): - return os.name == "nt" +IS_WINDOWS = os.name == "nt" def get_windows_path_with_unc_prefix(path): path = path.strip() # No need to add prefix for non-Windows platforms. - if not is_windows() or sys.version_info[0] < 3: + if not IS_WINDOWS or sys.version_info[0] < 3: return path # Starting in Windows 10, version 1607(OS build 14393), MAX_PATH limitations have been @@ -189,19 +186,19 @@ def find_runfiles_root(main_rel_path): # not found. These can be correctly set for a parent Python process, but # inherited by the child, and not correct for it. Later bootstrap code # assumes they're are correct if set. - os.environ.pop('RUNFILES_DIR', None) - os.environ.pop('RUNFILES_MANIFEST_FILE', None) + os.environ.pop("RUNFILES_DIR", None) + os.environ.pop("RUNFILES_MANIFEST_FILE", None) stub_filename = sys.argv[0] if not os.path.isabs(stub_filename): stub_filename = os.path.join(os.getcwd(), stub_filename) while True: - module_space = stub_filename + (".exe" if is_windows() else "") + ".runfiles" + module_space = stub_filename + (".exe" if IS_WINDOWS else "") + ".runfiles" if os.path.isdir(module_space): return module_space - runfiles_pattern = r"(.*\.runfiles)" + (r"\\" if is_windows() else "/") + ".*" + runfiles_pattern = r"(.*\.runfiles)" + (r"\\" if IS_WINDOWS else "/") + ".*" matchobj = re.match(runfiles_pattern, stub_filename) if matchobj: return matchobj.group(1) @@ -454,7 +451,7 @@ def main(): # runfiles root if MAIN_PATH: main_rel_path = MAIN_PATH - if is_windows(): + if IS_WINDOWS: main_rel_path = main_rel_path.replace("/", os.sep) runfiles_root = find_runfiles_root(main_rel_path) diff --git a/python/private/zipapp/py_zipapp_rule.bzl b/python/private/zipapp/py_zipapp_rule.bzl index 1655f63cc9..f26b050165 100644 --- a/python/private/zipapp/py_zipapp_rule.bzl +++ b/python/private/zipapp/py_zipapp_rule.bzl @@ -18,7 +18,7 @@ def _is_symlink(f): else: return "-1" -def _create_zipapp_main_py(ctx, py_runtime, py_executable, stage2_bootstrap, runfiles): +def _create_zipapp_main_py(ctx, py_runtime, py_executable, stage2_bootstrap, runfiles, explicit_symlinks): venv_python_exe = py_executable.venv_python_exe if venv_python_exe: venv_python_exe_path = runfiles_root_path(ctx, venv_python_exe.short_path) @@ -55,7 +55,7 @@ def _create_zipapp_main_py(ctx, py_runtime, py_executable, stage2_bootstrap, run inputs = builders.DepsetBuilder() inputs.add(py_runtime.zip_main_template) - _build_manifest(ctx, hash_files_manifest, runfiles, inputs) + _build_manifest(ctx, hash_files_manifest, runfiles, explicit_symlinks, inputs) actions_run( ctx, @@ -80,7 +80,10 @@ def _map_zip_symlinks(entry): def _map_zip_root_symlinks(entry): return "rf-root-symlink|" + _is_symlink(entry.target_file) + "|" + entry.path + "|" + entry.target_file.path -def _build_manifest(ctx, manifest, runfiles, inputs): +def _map_explicit_symlinks(entry): + return "symlink|" + entry.runfiles_path + "|" + entry.link_to_path + +def _build_manifest(ctx, manifest, runfiles, explicit_symlinks, inputs): manifest.add_all( # NOTE: Accessing runfiles.empty_filenames materializes them. A lambda # is used to defer that. @@ -92,10 +95,13 @@ def _build_manifest(ctx, manifest, runfiles, inputs): manifest.add_all(runfiles.files, map_each = _map_zip_runfiles) manifest.add_all(runfiles.symlinks, map_each = _map_zip_symlinks) manifest.add_all(runfiles.root_symlinks, map_each = _map_zip_root_symlinks) + manifest.add_all(explicit_symlinks, map_each = _map_explicit_symlinks) inputs.add(runfiles.files) inputs.add([entry.target_file for entry in runfiles.symlinks.to_list()]) inputs.add([entry.target_file for entry in runfiles.root_symlinks.to_list()]) + for entry in explicit_symlinks.to_list(): + inputs.add(entry.files) zip_repo_mapping_manifest = maybe_create_repo_mapping( ctx = ctx, @@ -121,6 +127,9 @@ def _create_zip(ctx, py_runtime, py_executable, stage2_bootstrap): runfiles.add(py_runtime.files) if py_executable.venv_python_exe: runfiles.add(py_executable.venv_python_exe) + + if py_executable.venv_interpreter_runfiles: + runfiles.add(py_executable.venv_interpreter_runfiles) runfiles.add(py_executable.app_runfiles) runfiles.add(stage2_bootstrap) @@ -132,11 +141,12 @@ def _create_zip(ctx, py_runtime, py_executable, stage2_bootstrap): py_executable, stage2_bootstrap, runfiles, + py_executable.venv_interpreter_symlinks, ) inputs = builders.DepsetBuilder() manifest.add("regular|0|__main__.py|{}".format(zip_main.path)) inputs.add(zip_main) - _build_manifest(ctx, manifest, runfiles, inputs) + _build_manifest(ctx, manifest, runfiles, py_executable.venv_interpreter_symlinks, inputs) zipper_args = ctx.actions.args() zipper_args.add(output) @@ -149,6 +159,9 @@ def _create_zip(ctx, py_runtime, py_executable, stage2_bootstrap): zipper_args.add(ctx.attr.compression, format = "--compression=%s") zipper_args.add("--runfiles-dir=runfiles") + is_windows = target_platform_has_any_constraint(ctx, ctx.attr._windows_constraints) + zipper_args.add("\\" if is_windows else "/", format = "--target-platform-pathsep=%s") + actions_run( ctx, executable = ctx.attr._zipper, @@ -220,18 +233,23 @@ def _py_zipapp_executable_impl(ctx): if target_platform_has_any_constraint(ctx, ctx.attr._windows_constraints): executable = ctx.actions.declare_file(ctx.label.name + ".exe") - python_exe = py_executable.venv_python_exe - if python_exe: - python_exe_path = runfiles_root_path(ctx, python_exe.short_path) - elif py_runtime.interpreter: - python_exe_path = runfiles_root_path(ctx, py_runtime.interpreter.short_path) + # The zipapp is an opaque zip file, so the Bazel Python launcher doesn't + # know how to look inside it to find the Python interpreter. This means + # we can only use system paths or programs on PATH to bootstrap. + if py_runtime.interpreter_path: + bootstrap_python_path = py_runtime.interpreter_path else: - python_exe_path = py_runtime.interpreter_path + # A special value the Bazel Python launcher recognized to skip + # lookup in the runfiles and uses `python.exe` from PATH. + bootstrap_python_path = "python" create_windows_exe_launcher( ctx, output = executable, - python_binary_path = python_exe_path, + # The path to a python to use to invoke e.g. `python.exe foo.zip` + python_binary_path = bootstrap_python_path, + # Tell the launcher to invoke `python_binary_path` on itself + # after removing its file extension and appending `.zip`. use_zip_file = True, ) default_outputs = [executable, zip_file] diff --git a/python/private/zipapp/zip_main_template.py b/python/private/zipapp/zip_main_template.py index 97c37fee0b..6d12bc9c08 100644 --- a/python/private/zipapp/zip_main_template.py +++ b/python/private/zipapp/zip_main_template.py @@ -1,5 +1,7 @@ # Template for the __main__.py file inserted into zip files # +# Generated file from @rules_python//python/private/zipapp:zip_main_template.py +# # NOTE: This file is a "stage 1" bootstrap, so it's responsible for locating the # desired runtime and having it run the stage 2 bootstrap. This means it can't # assume much about the current runtime and environment. e.g., the current @@ -27,7 +29,7 @@ import subprocess import tempfile import zipfile -from os.path import dirname, join, basename +from os.path import basename, dirname, join # runfiles-root-relative path _STAGE2_BOOTSTRAP = "%stage2_bootstrap%" @@ -45,29 +47,48 @@ IS_WINDOWS = os.name == "nt" -def print_verbose(*args, mapping=None, values=None): - if bool(os.environ.get("RULES_PYTHON_BOOTSTRAP_VERBOSE")): - if mapping is not None: - for key, value in sorted((mapping or {}).items()): - print( - "bootstrap: stage 1:", - *args, - f"{key}={value!r}", - file=sys.stderr, - flush=True, - ) - elif values is not None: - for i, v in enumerate(values): - print( - "bootstrap: stage 1:", - *args, - f"[{i}] {v!r}", - file=sys.stderr, - flush=True, - ) - else: - print("bootstrap: stage 1:", *args, file=sys.stderr, flush=True) +EXTRACT_ROOT = os.environ.get("RULES_PYTHON_EXTRACT_ROOT") + +# Change the paths with Unix-style forward slashes to backslashes for Windows. +# Windows usually transparently rewrites them, but e.g. `\\?\` paths require +# backslashes to be properly understood by Windows APIs. +if IS_WINDOWS: + + def norm_slashes(s): + if not s: + return s + return s.replace("/", "\\") + _STAGE2_BOOTSTRAP = norm_slashes(_STAGE2_BOOTSTRAP) + _PYTHON_BINARY_VENV = norm_slashes(_PYTHON_BINARY_VENV) + _PYTHON_BINARY_ACTUAL = norm_slashes(_PYTHON_BINARY_ACTUAL) + EXTRACT_DIR = norm_slashes(EXTRACT_DIR) + EXTRACT_ROOT = norm_slashes(EXTRACT_ROOT) + + +def print_verbose(*args, mapping=None, values=None): + if not bool(os.environ.get("RULES_PYTHON_BOOTSTRAP_VERBOSE")): + return + if mapping is not None: + for key, value in sorted((mapping or {}).items()): + print( + "bootstrap: stage 1:", + *args, + f"{key}={value!r}", + file=sys.stderr, + flush=True, + ) + elif values is not None: + for i, v in enumerate(values): + print( + "bootstrap: stage 1:", + *args, + f"[{i}] {v!r}", + file=sys.stderr, + flush=True, + ) + else: + print("bootstrap: stage 1:", *args, file=sys.stderr, flush=True) def get_windows_path_with_unc_prefix(path): @@ -105,18 +126,6 @@ def get_windows_path_with_unc_prefix(path): return unicode_prefix + os.path.abspath(path) -def has_windows_executable_extension(path): - return path.endswith(".exe") or path.endswith(".com") or path.endswith(".bat") - - -if ( - _PYTHON_BINARY_VENV - and IS_WINDOWS - and not has_windows_executable_extension(_PYTHON_BINARY_VENV) -): - _PYTHON_BINARY_VENV = _PYTHON_BINARY_VENV + ".exe" - - def search_path(name): """Finds a file in a given search path.""" search_path = os.getenv("PATH", os.defpath).split(os.pathsep) @@ -254,6 +263,7 @@ def execute_file( print_verbose("subprocess cwd:", workspace) print_verbose("subprocess argv:", values=subprocess_argv) ret_code = subprocess.call(subprocess_argv, env=env, cwd=workspace) + print_verbose("subprocess exit code:", ret_code) sys.exit(ret_code) finally: if not EXTRACT_ROOT: @@ -265,6 +275,43 @@ def execute_file( shutil.rmtree(extract_root, True) +def finish_venv_setup(runfiles_root): + python_program = os.path.join(runfiles_root, _PYTHON_BINARY_VENV) + # When a venv is used, the `bin/python3` symlink may need to be created. + # This case occurs when "create venv at runtime" or "resolve python at + # runtime" modes are enabled. + if not os.path.exists(python_program): + # The venv bin/python3 interpreter should always be under runfiles, but + # double check. We don't want to accidentally create symlinks elsewhere + if not python_program.startswith(runfiles_root): + raise AssertionError( + "Program's venv binary not under runfiles: {python_program}" + ) + symlink_to = find_binary(runfiles_root, _PYTHON_BINARY_ACTUAL) + os.makedirs(dirname(python_program), exist_ok=True) + if os.path.lexists(python_program): + os.remove(python_program) + try: + os.symlink(symlink_to, python_program) + except OSError as e: + raise Exception( + f"Unable to create venv python interpreter symlink: {python_program} -> {symlink_to}" + ) from e + venv_root = dirname(dirname(python_program)) + pyvenv_cfg = join(venv_root, "pyvenv.cfg") + if not os.path.exists(pyvenv_cfg): + print_verbose("finish_venv_setup: create pyvenv.cfg:", pyvenv_cfg) + python_home = join(runfiles_root, dirname(_PYTHON_BINARY_ACTUAL)) + print_verbose("finish_venv_setup: pyvenv.cfg home:", python_home) + with open(pyvenv_cfg, "w") as fp: + # Until Windows supports a build-time generated venv using symlinks + # to directories, we have to write the full, absolute, path to PYTHONHOME + # so that support directories (e.g. DLLs, libs) can be found. + fp.write("home = {}\n".format(python_home)) + + return python_program + + def main(): print_verbose("running zip main bootstrap") print_verbose("initial argv:", values=sys.argv) @@ -304,26 +351,7 @@ def main(): ) if _PYTHON_BINARY_VENV: - python_program = join(runfiles_root, _PYTHON_BINARY_VENV) - # When a venv is used, the `bin/python3` symlink may need to be created. - # This case occurs when "create venv at runtime" or "resolve python at - # runtime" modes are enabled. - if not os.path.lexists(python_program): - # The venv bin/python3 interpreter should always be under runfiles, but - # double check. We don't want to accidentally create symlinks elsewhere - if not python_program.startswith(runfiles_root): - raise AssertionError( - "Program's venv binary not under runfiles: {python_program}" - ) - symlink_to = find_binary(runfiles_root, _PYTHON_BINARY_ACTUAL) - os.makedirs(dirname(python_program), exist_ok=True) - try: - os.symlink(symlink_to, python_program) - except OSError as e: - raise Exception( - f"Unable to create venv python interpreter symlink: {python_program} -> {symlink_to}" - ) from e - + python_program = finish_venv_setup(runfiles_root) else: python_program = find_binary(runfiles_root, _PYTHON_BINARY_ACTUAL) if python_program is None: diff --git a/specialized_configs.bazelrc b/specialized_configs.bazelrc new file mode 100644 index 0000000000..46334e0a95 --- /dev/null +++ b/specialized_configs.bazelrc @@ -0,0 +1,12 @@ + +# Helper config to run most tests without waiting an inordinate amount +# of time or freezing the system +common:fast-tests --build_tests_only=true +common:fast-tests --build_tag_filters=-large,-enormous,-integration-test +common:fast-tests --test_tag_filters=-large,-enormous,-integration-test + +# Helper config for running a single test locally and investigating resulting state +common:testone --test_output=streamed +common:testone --test_strategy=standalone +common:testone --spawn_strategy=standalone +common:testone --strategy=standalone diff --git a/tests/base_rules/py_executable_base_tests.bzl b/tests/base_rules/py_executable_base_tests.bzl index f0e2ae9faf..d6b5aedf0c 100644 --- a/tests/base_rules/py_executable_base_tests.bzl +++ b/tests/base_rules/py_executable_base_tests.bzl @@ -44,19 +44,12 @@ def _test_basic_windows(name, config): impl = _test_basic_windows_impl, target = name + "_subject", config_settings = { - # NOTE: The default for this flag is based on the Bazel host OS, not - # the target platform. For windows, it defaults to true, so force - # it to that to match behavior when this test runs on other - # platforms. - # Pass value to both native and starlark versions of the flag until - # the native one is removed. - labels.BUILD_PYTHON_ZIP: True, "//command_line_option:cpu": "windows_x86_64", "//command_line_option:crosstool_top": CROSSTOOL_TOP, "//command_line_option:extra_execution_platforms": [platform_targets.WINDOWS_X86_64], "//command_line_option:extra_toolchains": [CC_TOOLCHAIN], "//command_line_option:platforms": [platform_targets.WINDOWS_X86_64], - } | maybe_builtin_build_python_zip("true"), + }, attr_values = {}, ) @@ -64,7 +57,7 @@ def _test_basic_windows_impl(env, target): target = env.expect.that_target(target) target.executable().path().contains(".exe") target.runfiles().contains_predicate(matching.str_endswith( - target.meta.format_str("/{name}.zip"), + target.meta.format_str("/{name}"), )) target.runfiles().contains_predicate(matching.str_endswith( target.meta.format_str("/{name}.exe"), @@ -246,9 +239,9 @@ def _test_debugger_impl(env, targets): # #3481: Ensure that venv site-packages is setup correctly, if the dependency is coming # from pip integration. - env.expect.that_target(targets.target_venv).runfiles().contains_at_least([ - "{workspace}/{package}/_{name}.venv/lib/python3.13/site-packages/{test_name}_debugger_venv.py", - ]) + env.expect.that_target(targets.target_venv).runfiles().contains_predicate( + matching.str_endswith("site-packages/test_debugger_debugger_venv.py"), + ) # 3. Subject exec diff --git a/tests/bootstrap_impls/run_binary_zip_yes_test.sh b/tests/bootstrap_impls/run_binary_zip_yes_test.sh index ca278083dd..77fe4d3609 100755 --- a/tests/bootstrap_impls/run_binary_zip_yes_test.sh +++ b/tests/bootstrap_impls/run_binary_zip_yes_test.sh @@ -34,8 +34,8 @@ actual=$($bin) # How we detect if a zip file was executed from depends on which bootstrap # is used. # bootstrap_impl=script outputs RULES_PYTHON_ZIP_DIR: -# bootstrap_impl=system_python outputs file:.*Bazel.runfiles -expected_pattern="RULES_PYTHON_ZIP_DIR:/\|file:.*Bazel.runfiles" +# bootstrap_impl=system_python outputs file:.*Bazel.runfiles (or .exe.runfiles on Windows) +expected_pattern="RULES_PYTHON_ZIP_DIR:/\|file:.*Bazel.runfiles\|file:.*\.exe\.runfiles" if ! (echo "$actual" | grep "$expected_pattern" ) >/dev/null; then echo "expected output to match: $expected_pattern" echo "but got: $actual" diff --git a/tests/bootstrap_impls/sys_path_order_test.py b/tests/bootstrap_impls/sys_path_order_test.py index 9ae03bb129..a9018c39ce 100644 --- a/tests/bootstrap_impls/sys_path_order_test.py +++ b/tests/bootstrap_impls/sys_path_order_test.py @@ -33,9 +33,13 @@ def test_sys_path_order(self): # error messages are more informative. categorized_paths = [] for i, value in enumerate(sys.path): - # The runtime's root repo may be added to sys.path, but it - # counts as a user directory, not stdlib directory. - if value in (sys.prefix, sys.base_prefix): + # On Windows, the `pythonXY.zip` entry shows up as `$venv/Scripts/pythonXY.zip` + # While it's technically part of the venv, it's considered the stdlib. + if os.name == "nt" and re.search("python.*[.]zip$", value): + category = "stdlib" + elif value in (sys.prefix, sys.base_prefix): + # The runtime's root repo may be added to sys.path, but it + # counts as a user directory, not stdlib directory. category = "user" elif value.startswith(sys.base_prefix): # The runtime's site-package directory might be called diff --git a/tests/bootstrap_impls/system_python_nodeps_test.py b/tests/bootstrap_impls/system_python_nodeps_test.py index 7dc46d6e73..d9b43e0f27 100644 --- a/tests/bootstrap_impls/system_python_nodeps_test.py +++ b/tests/bootstrap_impls/system_python_nodeps_test.py @@ -1 +1,11 @@ print("Hello, world") + +# Verify py code from the stdlib can be imported. +import pathlib + +print(pathlib) + +# Verify a C-implemented module can be imported. +# Socket isn't implement in C, but requires `_socket`, +# which is implemented in C +import socket diff --git a/tests/config_settings/transition/multi_version_tests.bzl b/tests/config_settings/transition/multi_version_tests.bzl index 3bb69f2f59..2b4f73a225 100644 --- a/tests/config_settings/transition/multi_version_tests.bzl +++ b/tests/config_settings/transition/multi_version_tests.bzl @@ -120,27 +120,6 @@ def _test_py_binary_windows_build_python_zip_false_impl(env, target): _tests.append(_test_py_binary_windows_build_python_zip_false) -def _test_py_binary_windows_build_python_zip_true(name): - _setup_py_binary_windows( - name, - build_python_zip = True, - impl = _test_py_binary_windows_build_python_zip_true_impl, - ) - -def _test_py_binary_windows_build_python_zip_true_impl(env, target): - default_outputs = env.expect.that_target(target).default_outputs() - - # TODO: These outputs aren't correct. The outputs shouldn't - # have the "_" prefix on them (those are coming from the underlying - # wrapped binary). - default_outputs.contains_exactly([ - "{package}/{test_name}_subject.exe", - "{package}/{test_name}_subject.py", - "{package}/{test_name}_subject.zip", - ]) - -_tests.append(_test_py_binary_windows_build_python_zip_true) - def multi_version_test_suite(name): test_suite( name = name, diff --git a/tests/integration/local_toolchains/.bazelrc b/tests/integration/local_toolchains/.bazelrc index aed08b0790..0fbb7678d1 100644 --- a/tests/integration/local_toolchains/.bazelrc +++ b/tests/integration/local_toolchains/.bazelrc @@ -1,4 +1,6 @@ +startup --windows_enable_symlinks common --action_env=RULES_PYTHON_BZLMOD_DEBUG=1 +common --repo_env=RULES_PYTHON_BZLMOD_DEBUG=1 common --lockfile_mode=off test --test_output=errors # Windows requires these for multi-python support: diff --git a/tests/integration/local_toolchains/MODULE.bazel b/tests/integration/local_toolchains/MODULE.bazel index c818942748..fe90fa235a 100644 --- a/tests/integration/local_toolchains/MODULE.bazel +++ b/tests/integration/local_toolchains/MODULE.bazel @@ -37,6 +37,8 @@ local_runtime_repo( pbs_archive = use_repo_rule("//:pbs_archive.bzl", "pbs_archive") +# "pbs" means "python-build-standalone" +# This maps the different platform runtimes to URLS and SHAs pbs_archive( name = "pbs_runtime", sha256 = { @@ -47,7 +49,7 @@ pbs_archive( urls = { "linux": "https://github.com/astral-sh/python-build-standalone/releases/download/20250918/cpython-3.13.7+20250918-x86_64-unknown-linux-gnu-install_only.tar.gz", "mac os x": "https://github.com/astral-sh/python-build-standalone/releases/download/20250918/cpython-3.13.7+20250918-aarch64-apple-darwin-install_only.tar.gz", - "windows server 2022": "https://github.com/astral-sh/python-build-standalone/releases/download/20250918/cpython-3.13.7+20250918-x86_64-pc-windows-msvc-install_only.tar.gz", + "windows": "https://github.com/astral-sh/python-build-standalone/releases/download/20250918/cpython-3.13.7+20250918-x86_64-pc-windows-msvc-install_only.tar.gz", }, ) diff --git a/tests/integration/local_toolchains/pbs_archive.bzl b/tests/integration/local_toolchains/pbs_archive.bzl index 8bd0c1eb10..7d817b1b40 100644 --- a/tests/integration/local_toolchains/pbs_archive.bzl +++ b/tests/integration/local_toolchains/pbs_archive.bzl @@ -16,6 +16,10 @@ def _pbs_archive_impl(repository_ctx): urls = repository_ctx.attr.urls sha256s = repository_ctx.attr.sha256 + # os.name for windows contain build and version; simplify it + if "windows" in os_name: + os_name = "windows" + if os_name not in urls: fail("Unsupported OS: '{}'. Available OSs are: {}".format( os_name, diff --git a/tests/py_zipapp/BUILD.bazel b/tests/py_zipapp/BUILD.bazel index 708d322f41..fc1809b69f 100644 --- a/tests/py_zipapp/BUILD.bazel +++ b/tests/py_zipapp/BUILD.bazel @@ -5,9 +5,6 @@ load("//python:py_test.bzl", "py_test") load("//python/private:bzlmod_enabled.bzl", "BZLMOD_ENABLED") # buildifier: disable=bzl-visibility load("//python/zipapp:py_zipapp_binary.bzl", "py_zipapp_binary") load("//tests/support:support.bzl", "NOT_WINDOWS") -# todo: add windows support. Windows support will be a bit odd. -# It previously worked by having special logic in the exe launcher -# that knew to look for .zip and running that through python py_binary( name = "venv_bin", diff --git a/tests/repl/repl_test.py b/tests/repl/repl_test.py index 01d0442922..319dab561a 100644 --- a/tests/repl/repl_test.py +++ b/tests/repl/repl_test.py @@ -21,24 +21,58 @@ foo = 1234 """ +IS_WINDOWS = os.name == "nt" + class ReplTest(unittest.TestCase): def setUp(self): - self.repl = rfiles.Rlocation("rules_python/python/bin/repl") + rpath = "rules_python/python/bin/repl" + if IS_WINDOWS: + rpath += ".exe" + self.repl = rfiles.Rlocation(rpath) assert self.repl + if IS_WINDOWS: + self.repl = os.path.normpath(self.repl) def run_code_in_repl(self, lines: Iterable[str], *, env=None) -> str: """Runs the lines of code in the REPL and returns the text output.""" + input = "\n".join(lines) try: return subprocess.check_output( [self.repl], text=True, stderr=subprocess.STDOUT, - input="\n".join(lines), + input=input, env=env, ).strip() except subprocess.CalledProcessError as error: raise RuntimeError(f"Failed to run the REPL:\n{error.stdout}") from error + except Exception as exc: + if env: + env_str = "\n".join( + f"{key}={value!r}" for key, value in sorted(env.items()) + ) + else: + env_str = "" + if isinstance(exc, subprocess.CalledProcessError): + stdout = exc.stdout + else: + stdout = "" + exc.add_note( + f""" +===== env start ===== +{env_str} +===== env end ===== +===== input start ===== +{input} +===== input end ===== +commmand: {self.repl} +===== stdout start ===== +{stdout} +===== stdout end ===== +""" + ) + raise def test_repl_version(self): """Validates that we can successfully execute arbitrary code on the REPL.""" diff --git a/tests/toolchains/defs.bzl b/tests/toolchains/defs.bzl index 25863d18c4..fb4b3beb94 100644 --- a/tests/toolchains/defs.bzl +++ b/tests/toolchains/defs.bzl @@ -57,4 +57,5 @@ def define_toolchain_tests(name): deps = ["//python/runfiles"], data = ["//tests/support:current_build_settings"], target_compatible_with = select(target_compatible_with), + size = "large", ) diff --git a/tests/tools/zipapp/zipper_test.py b/tests/tools/zipapp/zipper_test.py index e0d6653aa7..8b441c4456 100644 --- a/tests/tools/zipapp/zipper_test.py +++ b/tests/tools/zipapp/zipper_test.py @@ -9,6 +9,10 @@ from tools.private.zipapp import zipper +def symlink_target_path(p): + return p.replace("/", os.sep) + + class ZipperTest(unittest.TestCase): def setUp(self): self.test_dir = pathlib.Path(tempfile.mkdtemp()) @@ -26,6 +30,8 @@ def _create_zip(self, **kwargs): "workspace_name": "my_ws", "legacy_external_runfiles": False, "runfiles_dir": "runfiles", + # We need to generate paths for the platform we're running on. + "platform_pathsep": os.sep, } defaults.update(kwargs) zipper.create_zip(**defaults) @@ -90,6 +96,77 @@ def test_create_zip_with_files_and_symlinks(self): self.assertZipFileContent(zf, "runfiles/root_file", content="content1") self.assertZipFileContent(zf, "runfiles/my_ws/empty_file", content="") + def test_create_zip_with_direct_symlink(self): + # Test the 'symlink' manifest entry type + manifest_content = [ + "symlink|path/to/link|target/path", + ] + self.manifest_path.write_text("\n".join(manifest_content)) + + self._create_zip() + + with zipfile.ZipFile(self.output_zip, "r") as zf: + self.assertEqual(zf.namelist(), ["runfiles/path/to/link"]) + self.assertZipFileContent( + zf, + "runfiles/path/to/link", + is_symlink=True, + target=symlink_target_path("../../target/path"), + ) + + def test_pathsep_normalization(self): + # Test that pathsep="\\" normalizes paths + file1_path = self.test_dir / "file1.txt" + file1_path.write_text("content1") + + manifest_content = [ + f"regular|0|dir/file.txt|{file1_path}", + "symlink|link/path|target/path", + ] + self.manifest_path.write_text("\n".join(manifest_content)) + + # Use backslash as platform_pathsep + self._create_zip(platform_pathsep="\\") + + with zipfile.ZipFile(self.output_zip, "r") as zf: + # zipfile.namelist() always returns with forward slashes + # But the content of the symlink should be normalized if it was passed through path_norm + self.assertEqual( + set(zf.namelist()), + {"dir/file.txt", "runfiles/link/path"}, + ) + # The target of the symlink should have backslashes + self.assertZipFileContent( + zf, + "runfiles/link/path", + is_symlink=True, + target="..\\target\\path", + ) + + def test_symlink_precedence(self): + # Test that 'symlink' entries take precedence over others for the same path + file1_path = self.test_dir / "file1.txt" + file1_path.write_text("content1") + + manifest_content = [ + # Same zip path: runfiles/my_ws/path/to/file + f"rf-file|0|path/to/file|{file1_path}", + "symlink|my_ws/path/to/file|symlink/target", + ] + self.manifest_path.write_text("\n".join(manifest_content)) + + self._create_zip() + + with zipfile.ZipFile(self.output_zip, "r") as zf: + self.assertEqual(zf.namelist(), ["runfiles/my_ws/path/to/file"]) + # It should be the symlink, not the file + self.assertZipFileContent( + zf, + "runfiles/my_ws/path/to/file", + is_symlink=True, + target=symlink_target_path("../../../symlink/target"), + ) + def test_timestamps_are_deterministic(self): # Create a content file with a specific recent timestamp file1_path = self.test_dir / "file1.txt" @@ -209,6 +286,54 @@ def test_output_deterministic(self): ], ) + def _extract_zip(self, zip_path, extract_dir): + # Manually extract to preserve symlinks + with zipfile.ZipFile(zip_path, "r") as zf: + for info in zf.infolist(): + extract_path = extract_dir / info.filename + extract_path.parent.mkdir(parents=True, exist_ok=True) + if self.is_symlink(info): + target = zf.read(info).decode() + # On Windows, relative symlinks must use backslashes to be readable + os.symlink(target, extract_path) + else: + with zf.open(info) as src, open(extract_path, "wb") as dst: + shutil.copyfileobj(src, dst) + + def test_symlink_extraction(self): + # Test that 'symlink' entries extract correctly as relative symlinks + # Create a file that the symlink will point to + target_file = self.test_dir / "target_file.txt" + target_file.write_text("target content") + + manifest_content = [ + f"rf-file|0|target/path|{target_file}", + "symlink|my_ws/path/to/link|my_ws/target/path", + f"rf-file|0|same_dir_target|{target_file}", + "symlink|my_ws/same_dir_link|my_ws/same_dir_target", + ] + self.manifest_path.write_text("\n".join(manifest_content)) + + self._create_zip(workspace_name="my_ws") + + extract_dir = self.test_dir / "extract" + extract_dir.mkdir() + + self._extract_zip(self.output_zip, extract_dir) + + link_path = extract_dir / "runfiles/my_ws/path/to/link" + self.assertTrue(link_path.is_symlink(), f"{link_path} should be a symlink") + self.assertEqual( + os.readlink(link_path), "../../target/path".replace("/", os.path.sep) + ) + self.assertEqual(link_path.read_text(), "target content") + + link2_path = extract_dir / "runfiles/my_ws/same_dir_link" + self.assertTrue(link2_path.is_symlink(), f"{link2_path} should be a symlink") + # Relative path from runfiles/my_ws/ to runfiles/my_ws/same_dir_target is just same_dir_target + self.assertEqual(os.readlink(link2_path), "same_dir_target") + self.assertEqual(link2_path.read_text(), "target content") + def is_symlink(self, zip_info): # Check upper 4 bits of external_attr for S_IFLNK # S_IFLNK is 0o120000 = 0xA000 diff --git a/tools/private/zipapp/zip_main_maker.py b/tools/private/zipapp/zip_main_maker.py index 78ac17eb17..ae112ffc66 100644 --- a/tools/private/zipapp/zip_main_maker.py +++ b/tools/private/zipapp/zip_main_maker.py @@ -39,6 +39,10 @@ def compute_inputs_hash(manifest_path: str) -> str: if type_ == "rf-empty": continue + if type_ == "symlink": + # The symlink path and the target it points to + # are captured by hashing the entire line above. + continue is_symlink_str = parts[0] path = parts[-1] diff --git a/tools/private/zipapp/zipper.py b/tools/private/zipapp/zipper.py index 6f41c1e663..870861bc07 100644 --- a/tools/private/zipapp/zipper.py +++ b/tools/private/zipapp/zipper.py @@ -4,12 +4,17 @@ import stat import sys import zipfile +from os.path import dirname # Unix permission bit for symlink (S_IFLNK) # S_IFLNK is usually 0o120000 S_IFLNK = 0o120000 +def unix_join(*parts): + return "/".join(parts) + + def _get_zip_runfiles_path( path, workspace_name, legacy_external_runfiles, runfiles_dir ): @@ -18,8 +23,8 @@ def _get_zip_runfiles_path( elif path.startswith("../"): path = path[3:] else: - path = os.path.join(workspace_name, path) - return os.path.join(runfiles_dir, path) + path = unix_join(workspace_name, path) + return unix_join(runfiles_dir, path) def _parse_entry( @@ -52,10 +57,16 @@ def _parse_entry( ) elif type_ == "rf-symlink": _, is_symlink_str, runfile_path, content_path = parts - zip_path = os.path.join(runfiles_dir, workspace_name, runfile_path) + zip_path = unix_join(runfiles_dir, workspace_name, runfile_path) elif type_ == "rf-root-symlink": _, is_symlink_str, runfile_path, content_path = parts - zip_path = os.path.join(runfiles_dir, runfile_path) + zip_path = unix_join(runfiles_dir, runfile_path) + elif type_ == "symlink": + _, runfile_path, link_to_rf_path = parts + zip_path = unix_join(runfiles_dir, runfile_path) + link_to_rf_path = unix_join(runfiles_dir, link_to_rf_path) + content_path = os.path.relpath(link_to_rf_path, start=dirname(zip_path)) + is_symlink_str = "2" else: raise ValueError( f"Error: Unknown entry type or invalid format at line {line_idx + 1}: {line}" @@ -84,13 +95,40 @@ def read_manifest( e.add_note(f"Error processing line {line_idx + 1}: {line.strip()}") raise - # Sort by zip path (3rd element in tuple) - entries.sort(key=lambda x: x[2]) + # Sort symlink entries first so they have precedence. + # Then sort by zip path + entries.sort(key=lambda x: (x[2], 0 if x[0] == "symlink" else 1)) return entries -def _write_entry(zf, entry, compress_type): +def convert_symlink_target(path, platform_pathsep): + """Converts the path a symlink points to the target-platform format. + + On Windows, relative symlinks must use backslashes. + """ + if platform_pathsep == "/": + # Convert Windows to Unix + return path.replace("\\", platform_pathsep) + else: + # Convert Unix to Windows + return path.replace("/", "\\") + + +# Zip files use forward slash for the entries, even on Windows +def normalize_zip_path(path): + return path.replace("\\", "/") + + +def _write_entry(zf, entry, compress_type, seen, platform_pathsep): type_, is_symlink_str, zip_path, content_path = entry + # Normalize slashes, otherwise the `seen` logic doesn't + # work correctly. + zip_path = normalize_zip_path(zip_path) + if zip_path in seen: + # This can occur because symlink entries have precedence + # over non-symlink entries. + return + seen.add(zip_path) if type_ == "rf-empty": zi = zipfile.ZipInfo(zip_path) @@ -101,6 +139,16 @@ def _write_entry(zf, entry, compress_type): zi.external_attr = (0o644 & 0xFFFF) << 16 zf.writestr(zi, "") return + if type_ == "symlink": + zi = zipfile.ZipInfo(zip_path) + zi.date_time = (1980, 1, 1, 0, 0, 0) + zi.create_system = 3 # Unix + zi.compress_type = compress_type + target = convert_symlink_target(content_path, platform_pathsep) + # Set permissions to 777 for symlink (standard) + zi.external_attr = (S_IFLNK | 0o777) << 16 + zf.writestr(zi, target) + return if is_symlink_str == "-1": if not os.path.exists(content_path): @@ -115,7 +163,7 @@ def _write_entry(zf, entry, compress_type): zi.date_time = (1980, 1, 1, 0, 0, 0) zi.create_system = 3 # Unix zi.compress_type = compress_type - target = os.readlink(content_path) + target = convert_symlink_target(os.readlink(content_path), platform_pathsep) # Set permissions to 777 for symlink (standard) zi.external_attr = (S_IFLNK | 0o777) << 16 zf.writestr(zi, target) @@ -139,6 +187,7 @@ def create_zip( workspace_name, legacy_external_runfiles, runfiles_dir, + platform_pathsep, ): compress_type = zipfile.ZIP_STORED if compress_level == 0 else zipfile.ZIP_DEFLATED zf_level = compress_level if compress_level != 0 else None @@ -147,11 +196,12 @@ def create_zip( manifest_path, workspace_name, legacy_external_runfiles, runfiles_dir ) + seen = set() with zipfile.ZipFile( output_zip, "w", compress_type, allowZip64=True, compresslevel=zf_level ) as zf: for entry in entries: - _write_entry(zf, entry, compress_type) + _write_entry(zf, entry, compress_type, seen, platform_pathsep) def main(): @@ -176,6 +226,9 @@ def main(): 5. `rf-root-symlink|is_symlink|runfile_root_path|content_path`: Store a runfiles-root-relative path in the zip. +6. `symlink|runfile_root_path|link_to_path_rf_path`: Store a symlink that + stores a relative path from `runfile_root_path` to `link_to_rf_path` + In all cases, `is_symlink` has the following values: * `1` means it should be stored as a symlink whose value is read (using `readlink()`) from `content_path`. @@ -210,6 +263,9 @@ def main(): parser.add_argument( "--runfiles-dir", default="runfiles", help="Name of the runfiles directory" ) + parser.add_argument( + "--target-platform-pathsep", help="The path separator for the target platform" + ) args = parser.parse_args() try: @@ -220,6 +276,7 @@ def main(): workspace_name=args.workspace_name, legacy_external_runfiles=args.legacy_external_runfiles == "1", runfiles_dir=args.runfiles_dir, + platform_pathsep=args.target_platform_pathsep, ) except Exception as e: e.add_note(f"Error creating zip {args.output}") From 6295ea0443169c2434c6a1f0b0b0321aa34589d3 Mon Sep 17 00:00:00 2001 From: "Jae Hoon (Antonio) Kim" <17433012+antoniojkim@users.noreply.github.com> Date: Thu, 9 Apr 2026 23:13:50 -0400 Subject: [PATCH 683/922] feat(wheel): Add support for add_path_prefix (#3679) This change is being made to support prepending a prefix to the file paths in the wheel. This is useful for customizing the import path for the package. For example, if your code is implemented in `src/module` you may want to package this code for distribution as `namespace/module` so that the import is ```python import namespace.module ``` I think ideally, this should have been implemented as `remap_path_prefix` which is a map which specifies which prefixes should be changed and what to change them to. However, seeing as `strip_path_prefixes` already exists, I thought the simpler thing to do here was to just add the `add_path_prefix` argument. Implements #515 ## Tests Tested by building and manually inspecting the wheels generated by ``` $ bazel build //examples/wheel/... ``` More specifically ``` $ bazel build //examples/wheel:custom_prefix_package_root INFO: Analyzed target //examples/wheel:custom_prefix_package_root (1 packages loaded, 4 targets configured). INFO: Found 1 target... Target //examples/wheel:custom_prefix_package_root up-to-date: bazel-bin/examples/wheel/examples_custom_prefix_package_root-0.0.1-py3-none-any.whl INFO: Elapsed time: 0.357s, Critical Path: 0.01s INFO: 2 processes: 3 action cache hit, 1 disk cache hit, 1 internal. $ uv pip install bazel-bin/examples/wheel/examples_custom_prefix_package_root-0.0.1-py3-none-any.whl Installed 1 package in 3ms + examples-custom-prefix-package-root==0.0.1 $ python -c "import custom_prefix.wheel" ``` --------- Co-authored-by: Antonio Kim --- .bazelrc.deleted_packages | 2 +- CHANGELOG.md | 2 + examples/wheel/BUILD.bazel | 19 ++++++ python/private/py_wheel.bzl | 17 +++++ tests/tools/wheelmaker_test.py | 112 ++++++++++++++++++++++++++------- tools/wheelmaker.py | 74 +++++++++++++++++----- 6 files changed, 187 insertions(+), 39 deletions(-) diff --git a/.bazelrc.deleted_packages b/.bazelrc.deleted_packages index fb5d2ef0bb..e767e0ae56 100644 --- a/.bazelrc.deleted_packages +++ b/.bazelrc.deleted_packages @@ -27,8 +27,8 @@ common --deleted_packages=gazelle/manifest/hasher common --deleted_packages=gazelle/manifest/test common --deleted_packages=gazelle/modules_mapping common --deleted_packages=gazelle/python -common --deleted_packages=gazelle/pythonconfig common --deleted_packages=gazelle/python/private +common --deleted_packages=gazelle/pythonconfig common --deleted_packages=tests/integration/compile_pip_requirements common --deleted_packages=tests/integration/compile_pip_requirements_test_from_external_repo common --deleted_packages=tests/integration/custom_commands diff --git a/CHANGELOG.md b/CHANGELOG.md index f487194929..26f794a999 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -138,6 +138,8 @@ Other changes: venvs. * {obj}`PyExecutableInfo.venv_interpreter_runfiles`, and {obj}`PyExecutableInfo.venv_interpreter_symlinks` adde +* (wheel) Add support for `add_path_prefix` argument in `py_wheel` which can be + used to prepend a prefix to the files in the wheel. {#v1-9-0} ## [1.9.0] - 2026-02-21 diff --git a/examples/wheel/BUILD.bazel b/examples/wheel/BUILD.bazel index 3cf6e9f350..01dc4fab41 100644 --- a/examples/wheel/BUILD.bazel +++ b/examples/wheel/BUILD.bazel @@ -230,6 +230,25 @@ py_wheel( ], ) +# An example of how to change the wheel package root directory using 'add_path_prefix'. +py_wheel( + name = "custom_prefix_package_root", + add_path_prefix = "custom_prefix", + # Package data. We're building "examples_custom_prefix_package_root-0.0.1-py3-none-any.whl" + distribution = "examples_custom_prefix_package_root", + entry_points = { + "console_scripts": ["main = foo.bar:baz"], + }, + python_tag = "py3", + strip_path_prefixes = [ + "examples", + ], + version = "0.0.1", + deps = [ + ":example_pkg", + ], +) + py_wheel( name = "python_requires_in_a_package", distribution = "example_python_requires_in_a_package", diff --git a/python/private/py_wheel.bzl b/python/private/py_wheel.bzl index e6a9925a15..391f64497a 100644 --- a/python/private/py_wheel.bzl +++ b/python/private/py_wheel.bzl @@ -170,6 +170,22 @@ entry_points, e.g. `{'console_scripts': ['main = examples.wheel.main:main']}`. } _other_attrs = { + "add_path_prefix": attr.string( + default = "", + doc = """\ +Path prefix to prepend to files added to the generated package. +This prefix will be prepended **after** the paths are first stripped of the prefixes +specified in `strip_path_prefixes`. + +For example: ++ `"foo/" will prepend to `"bar/baz/file.py"` as `"foo/bar/baz/file.py"` ++ `"foo_" will prepend to `"bar/baz/file.py"` as `"foo_bar/baz/file.py"` ++ `stripping ["bar/"] and adding "foo/" will change `"bar/baz/file.py"` to `"foo/baz/file.py"` +:::{versionadded} VERSION_NEXT_FEATURE +The {attr}`add_path_prefix` attribute was added. +::: +""", + ), "author": attr.string( doc = "A string specifying the author of the package.", default = "", @@ -389,6 +405,7 @@ def _py_wheel_impl(ctx): args.add("--out", outfile) args.add("--name_file", name_file) args.add_all(ctx.attr.strip_path_prefixes, format_each = "--strip_path_prefix=%s") + args.add("--path_prefix", ctx.attr.add_path_prefix) # Pass workspace status files if stamping is enabled if is_stamping_enabled(ctx.attr): diff --git a/tests/tools/wheelmaker_test.py b/tests/tools/wheelmaker_test.py index 85094af9b8..7c30981e83 100644 --- a/tests/tools/wheelmaker_test.py +++ b/tests/tools/wheelmaker_test.py @@ -1,5 +1,6 @@ import io import unittest +from dataclasses import dataclass, field import tools.wheelmaker as wheelmaker @@ -34,41 +35,108 @@ def test_quote_all_false_leaves_simple_filenames_unquoted(self) -> None: def test_quote_all_quotes_filenames_with_commas(self) -> None: """Filenames with commas are always quoted, regardless of quote_all_filenames.""" whl = self._make_whl_file(quote_all=True) - self.assertEqual(whl._quote_filename("foo,bar/baz.py"), '"foo,bar/baz.py"') + self.assertEqual( + whl._quote_filename("foo,bar/baz.py"), '"foo,bar/baz.py"' + ) whl = self._make_whl_file(quote_all=False) - self.assertEqual(whl._quote_filename("foo,bar/baz.py"), '"foo,bar/baz.py"') + self.assertEqual( + whl._quote_filename("foo,bar/baz.py"), '"foo,bar/baz.py"' + ) + + +@dataclass +class ArcNameTestCase: + name: str + expected: str + distribution_prefix: str = "" + strip_path_prefixes: list[str] = field(default_factory=list) + add_path_prefix: str = "" class ArcNameFromTest(unittest.TestCase): def test_arcname_from(self) -> None: - # (name, distribution_prefix, strip_path_prefixes, want) tuples - checks = [ - ("a/b/c/file.py", "", [], "a/b/c/file.py"), - ("a/b/c/file.py", "", ["a"], "/b/c/file.py"), - ("a/b/c/file.py", "", ["a/b/"], "c/file.py"), + test_cases = [ + ArcNameTestCase(name="a/b/c/file.py", expected="a/b/c/file.py"), + ArcNameTestCase( + name="a/b/c/file.py", + strip_path_prefixes=["a"], + expected="/b/c/file.py", + ), + ArcNameTestCase( + name="a/b/c/file.py", + strip_path_prefixes=["a/b/"], + expected="c/file.py", + ), # only first found is used and it's not cumulative. - ("a/b/c/file.py", "", ["a/", "b/"], "b/c/file.py"), + ArcNameTestCase( + name="a/b/c/file.py", + strip_path_prefixes=["a/", "b/"], + expected="b/c/file.py", + ), # Examples from docs - ("foo/bar/baz/file.py", "", ["foo", "foo/bar/baz"], "/bar/baz/file.py"), - ("foo/bar/baz/file.py", "", ["foo/bar/baz", "foo"], "/file.py"), - ("foo/file2.py", "", ["foo/bar/baz", "foo"], "/file2.py"), + ArcNameTestCase( + name="foo/bar/baz/file.py", + strip_path_prefixes=["foo", "foo/bar/baz"], + expected="/bar/baz/file.py", + ), + ArcNameTestCase( + name="foo/bar/baz/file.py", + strip_path_prefixes=["foo/bar/baz", "foo"], + expected="/file.py", + ), + ArcNameTestCase( + name="foo/file2.py", + strip_path_prefixes=["foo/bar/baz", "foo"], + expected="/file2.py", + ), # Files under the distribution prefix (eg mylib-1.0.0-dist-info) # are unmodified - ("mylib-0.0.1-dist-info/WHEEL", "mylib", [], "mylib-0.0.1-dist-info/WHEEL"), - ("mylib/a/b/c/WHEEL", "mylib", ["mylib"], "mylib/a/b/c/WHEEL"), + ArcNameTestCase( + name="mylib-0.0.1-dist-info/WHEEL", + distribution_prefix="mylib", + expected="mylib-0.0.1-dist-info/WHEEL", + ), + ArcNameTestCase( + name="mylib/a/b/c/WHEEL", + distribution_prefix="mylib", + strip_path_prefixes=["mylib"], + expected="mylib/a/b/c/WHEEL", + ), + # Check that prefixes are added + ArcNameTestCase( + name="a/b/c/file.py", + add_path_prefix="namespace/", + expected="namespace/a/b/c/file.py", + ), + ArcNameTestCase( + name="a/b/c/file.py", + strip_path_prefixes=["a"], + add_path_prefix="namespace", + expected="namespace/b/c/file.py", + ), + ArcNameTestCase( + name="a/b/c/file.py", + strip_path_prefixes=["a/b/"], + add_path_prefix="namespace_", + expected="namespace_c/file.py", + ), ] - for name, prefix, strip, want in checks: + for test_case in test_cases: with self.subTest( - name=name, - distribution_prefix=prefix, - strip_path_prefixes=strip, - want=want, + name=test_case.name, + distribution_prefix=test_case.distribution_prefix, + strip_path_prefixes=test_case.strip_path_prefixes, + add_path_prefix=test_case.add_path_prefix, + want=test_case.expected, ): got = wheelmaker.arcname_from( - name=name, distribution_prefix=prefix, strip_path_prefixes=strip + name=test_case.name, + distribution_prefix=test_case.distribution_prefix, + strip_path_prefixes=test_case.strip_path_prefixes, + add_path_prefix=test_case.add_path_prefix, ) - self.assertEqual(got, want) + self.assertEqual(got, test_case.expected) class GetNewRequirementLineTest(unittest.TestCase): @@ -77,7 +145,9 @@ def test_requirement(self): self.assertEqual(result, "Requires-Dist: requests>=2.0") def test_requirement_and_extra(self): - result = wheelmaker.get_new_requirement_line("requests>=2.0", "extra=='dev'") + result = wheelmaker.get_new_requirement_line( + "requests>=2.0", "extra=='dev'" + ) self.assertEqual(result, "Requires-Dist: requests>=2.0; extra=='dev'") def test_requirement_with_url(self): diff --git a/tools/wheelmaker.py b/tools/wheelmaker.py index 7124ae7c9d..ada525e9bf 100644 --- a/tools/wheelmaker.py +++ b/tools/wheelmaker.py @@ -94,13 +94,18 @@ def normalize_pep440(version): substituted = re.sub(r"\{\w+\}", "0", version) delimiter = "." if "+" in substituted else "+" try: - return str(packaging.version.Version(f"{substituted}{delimiter}{sanitized}")) + return str( + packaging.version.Version(f"{substituted}{delimiter}{sanitized}") + ) except packaging.version.InvalidVersion: return str(packaging.version.Version(f"0+{sanitized}")) def arcname_from( - name: str, distribution_prefix: str, strip_path_prefixes: Sequence[str] = () + name: str, + distribution_prefix: str, + strip_path_prefixes: Sequence[str] = (), + add_path_prefix: str = "", ) -> str: """Return the within-archive name for a given file path name. @@ -110,17 +115,20 @@ def arcname_from( name: The file path eg 'mylib/a/b/c/file.py' distribution_prefix: The strip_path_prefixes: Remove these prefixes from names. + add_path_prefix: Add prefix after stripping the path from names. """ # Always use unix path separators. normalized_arcname = name.replace(os.path.sep, "/") # Don't manipulate names filenames in the .distinfo or .data directories. - if distribution_prefix and normalized_arcname.startswith(distribution_prefix): + if distribution_prefix and normalized_arcname.startswith( + distribution_prefix + ): return normalized_arcname for prefix in strip_path_prefixes: if normalized_arcname.startswith(prefix): - return normalized_arcname[len(prefix) :] + return add_path_prefix + normalized_arcname[len(prefix) :] - return normalized_arcname + return add_path_prefix + normalized_arcname class _WhlFile(zipfile.ZipFile): @@ -131,6 +139,7 @@ def __init__( mode, distribution_prefix: str, strip_path_prefixes=None, + add_path_prefix=None, compression=zipfile.ZIP_DEFLATED, quote_all_filenames: bool = False, **kwargs, @@ -138,6 +147,7 @@ def __init__( self._distribution_prefix = distribution_prefix self._strip_path_prefixes = strip_path_prefixes or [] + self._add_path_prefix = add_path_prefix or "" # Entries for the RECORD file as (filename, digest, size) tuples. self._record: list[tuple[str, str, str]] = [] # Whether to quote filenames in the RECORD file (for compatibility with @@ -168,6 +178,7 @@ def add_file(self, package_filename, real_filename): package_filename, distribution_prefix=self._distribution_prefix, strip_path_prefixes=self._strip_path_prefixes, + add_path_prefix=self._add_path_prefix, ) zinfo = self._zipinfo(arcname) @@ -194,7 +205,9 @@ def add_string(self, filename, contents): self.writestr(zinfo, contents) hash = hashlib.sha256() hash.update(contents) - self._add_to_record(filename, self._serialize_digest(hash), len(contents)) + self._add_to_record( + filename, self._serialize_digest(hash), len(contents) + ) def _serialize_digest(self, hash) -> str: # https://www.python.org/dev/peps/pep-0376/#record @@ -231,7 +244,9 @@ def _quote_filename(self, filename: str) -> str: filename = filename.lstrip("/") # Some RECORDs like torch have *all* filenames quoted and we must minimize diff. # Otherwise, we quote only when necessary (e.g. for filenames with commas). - quoting = csv.QUOTE_ALL if self.quote_all_filenames else csv.QUOTE_MINIMAL + quoting = ( + csv.QUOTE_ALL if self.quote_all_filenames else csv.QUOTE_MINIMAL + ) with io.StringIO() as buf: csv.writer(buf, quoting=quoting).writerow([filename]) return buf.getvalue().strip() @@ -261,6 +276,7 @@ def __init__( compress, outfile=None, strip_path_prefixes=None, + add_path_prefix=None, ): self._name = name self._version = normalize_pep440(version) @@ -270,9 +286,10 @@ def __init__( self._platform = platform self._outfile = outfile self._strip_path_prefixes = strip_path_prefixes + self._add_path_prefix = add_path_prefix self._compress = compress - self._wheelname_fragment_distribution_name = escape_filename_distribution_name( - self._name + self._wheelname_fragment_distribution_name = ( + escape_filename_distribution_name(self._name) ) self._distribution_prefix = ( @@ -287,7 +304,10 @@ def __enter__(self): mode="w", distribution_prefix=self._distribution_prefix, strip_path_prefixes=self._strip_path_prefixes, - compression=zipfile.ZIP_DEFLATED if self._compress else zipfile.ZIP_STORED, + add_path_prefix=self._add_path_prefix, + compression=( + zipfile.ZIP_DEFLATED if self._compress else zipfile.ZIP_STORED + ), ) return self @@ -330,7 +350,9 @@ def add_wheelfile(self): Wheel-Version: 1.0 Generator: bazel-wheelmaker 1.0 Root-Is-Purelib: {} -""".format("true" if self._platform == "any" else "false") +""".format( + "true" if self._platform == "any" else "false" + ) for tag in self.disttags(): wheel_contents += "Tag: %s\n" % tag self._whlfile.add_string(self.distinfo_path("WHEEL"), wheel_contents) @@ -339,7 +361,9 @@ def add_metadata(self, metadata, name, description): """Write METADATA file to the distribution.""" # https://www.python.org/dev/peps/pep-0566/ # https://packaging.python.org/specifications/core-metadata/ - metadata = re.sub("^Name: .*$", "Name: %s" % name, metadata, flags=re.MULTILINE) + metadata = re.sub( + "^Name: .*$", "Name: %s" % name, metadata, flags=re.MULTILINE + ) metadata += "Version: %s\n\n" % self._version # setuptools seems to insert UNKNOWN as description when none is # provided. @@ -418,7 +442,9 @@ def resolve_argument_stamp( def parse_args() -> argparse.Namespace: parser = argparse.ArgumentParser(description="Builds a python wheel") - metadata_group = parser.add_argument_group("Wheel name, version and platform") + metadata_group = parser.add_argument_group( + "Wheel name, version and platform" + ) metadata_group.add_argument( "--name", required=True, type=str, help="Name of the distribution" ) @@ -465,6 +491,13 @@ def parse_args() -> argparse.Namespace: help="Path prefix to be stripped from input package files' path. " "Can be supplied multiple times. Evaluated in order.", ) + output_group.add_argument( + "--path_prefix", + type=str, + default="", + help="Path prefix to be prepended to input package files' path. " + "It is prepended after stripping any specified path prefixes first.", + ) wheel_group = parser.add_argument_group("Wheel metadata") wheel_group.add_argument( @@ -477,7 +510,8 @@ def parse_args() -> argparse.Namespace: "--description_file", help="Path to the file with package description" ) wheel_group.add_argument( - "--description_content_type", help="Content type of the package description" + "--description_content_type", + help="Content type of the package description", ) wheel_group.add_argument( "--entry_points_file", @@ -579,6 +613,7 @@ def main() -> None: platform=arguments.platform, outfile=arguments.out, strip_path_prefixes=strip_prefixes, + add_path_prefix=arguments.path_prefix, compress=not arguments.no_compress, ) as maker: for package_filename, real_filename in all_files: @@ -608,7 +643,9 @@ def main() -> None: if not meta_line[len("Requires-Dist: ") :].startswith("@"): # This is a normal requirement. - package, _, extra = meta_line[len("Requires-Dist: ") :].rpartition(";") + package, _, extra = meta_line[ + len("Requires-Dist: ") : + ].rpartition(";") if not package: # This is when the package requirement does not have markers. continue @@ -623,7 +660,9 @@ def main() -> None: extra = extra.strip() reqs = [] - for reqs_line in Path(file).read_text(encoding="utf-8").splitlines(): + for reqs_line in ( + Path(file).read_text(encoding="utf-8").splitlines() + ): reqs_text = reqs_line.strip() if not reqs_text or reqs_text.startswith(("#", "-")): continue @@ -650,7 +689,8 @@ def main() -> None: if arguments.entry_points_file: maker.add_file( - maker.distinfo_path("entry_points.txt"), arguments.entry_points_file + maker.distinfo_path("entry_points.txt"), + arguments.entry_points_file, ) # Sort the files for reproducible order in the archive. From 8c6f9d8f6c0886c2fec6c5cb0bfd9f13d2ded324 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Fri, 10 Apr 2026 08:53:23 -0700 Subject: [PATCH 684/922] chore: update changelog, version markers, for 2.0 release (#3689) This updates the changelog and version markers for the next release, 2.0 --- CHANGELOG.md | 16 ++++++++-------- python/private/py_executable_info.bzl | 4 ++-- python/private/py_runtime_info.bzl | 2 +- python/private/py_wheel.bzl | 2 +- 4 files changed, 12 insertions(+), 12 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 26f794a999..8b6a0e521d 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -47,16 +47,16 @@ BEGIN_UNRELEASED_TEMPLATE END_UNRELEASED_TEMPLATE --> -{#v0-0-0} -## Unreleased +{#v2-0-0} +## [2.0.0] - 2026-04-09 -[0.0.0]: https://github.com/bazel-contrib/rules_python/releases/tag/0.0.0 +[2.0.0]: https://github.com/bazel-contrib/rules_python/releases/tag/2.0.0 -{#v0-0-0-removed} +{#v2-0-0-removed} ### Removed * Nothing removed. -{#v0-0-0-changed} +{#v2-0-0-changed} ### Changed **Breaking** @@ -91,7 +91,7 @@ Other changes: * Windows no longer defaults to creating a zip file and extracting it; a symlink-based runfiles tree is created, as on unix-like platforms. -{#v0-0-0-fixed} +{#v2-0-0-fixed} ### Fixed * (toolchain) Also set Make variables for local toolchains. * (zipapp) Resolve issue passing through compression settings in @@ -123,7 +123,7 @@ Other changes: ``` Fixes [#3676](https://github.com/bazel-contrib/rules_python/issues/3676). -{#v0-0-0-added} +{#v2-0-0-added} ### Added * (pypi) Write SimpleAPI contents to the `MODULE.bazel.lock` file if using {obj}`experimental_index_url` which should speed up consecutive @@ -2315,4 +2315,4 @@ Breaking changes: * (pip) Create all_data_requirements alias * Expose Python C headers through the toolchain. -[0.24.0]: https://github.com/bazel-contrib/rules_python/releases/tag/0.24.0 +[0.24.0]: https://github.com/bazel-contrib/rules_python/releases/tag/0.24.0 \ No newline at end of file diff --git a/python/private/py_executable_info.bzl b/python/private/py_executable_info.bzl index 0c5931cecd..a076715bd0 100644 --- a/python/private/py_executable_info.bzl +++ b/python/private/py_executable_info.bzl @@ -83,7 +83,7 @@ implementation isn't being used. Runfiles that are specific to the interpreter within the venv. -:::{versionadded} VERSION_NEXT_FEATURE +:::{versionadded} 2.0.0 ::: """, "venv_interpreter_symlinks": """ @@ -95,7 +95,7 @@ Only used with Windows for files that would have used `declare_symlink()` to create relative symlinks. These may overlap with paths in runfiles; it's up to the consumer to determine how to handle such overlaps. -:::{versionadded} VERSION_NEXT_FEATURE +:::{versionadded} 2.0.0 ::: """, "venv_python_exe": """ diff --git a/python/private/py_runtime_info.bzl b/python/private/py_runtime_info.bzl index 8fdbd7bfe2..d94f469fc0 100644 --- a/python/private/py_runtime_info.bzl +++ b/python/private/py_runtime_info.bzl @@ -343,7 +343,7 @@ to meet two criteria: Files that should be added to the venv's `bin/` (or platform-specific equivalent) directory (using the file's basename). -:::{versionadded} VERSION_NEXT_FEATURE +:::{versionadded} 2.0.0 """, "zip_main_template": """ :type: File diff --git a/python/private/py_wheel.bzl b/python/private/py_wheel.bzl index 391f64497a..5b8b5c7879 100644 --- a/python/private/py_wheel.bzl +++ b/python/private/py_wheel.bzl @@ -222,7 +222,7 @@ filegroup(name = "files", srcs = [":file1.txt", ":file2.txt"]) Allowed paths: {prefixes} -:::{{versionchanged}} VERSION_NEXT_FEATURE +:::{{versionchanged}} 2.0.0 Values can end in slash (`/`) to indicate that all files of the target should be moved under that directory. ::: From c99d2b45aafd244aea7ad2d066b647ef562db2d3 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Fri, 10 Apr 2026 16:38:45 -0700 Subject: [PATCH 685/922] sphinxdoc: move tests and docs to module-root directory (#3690) This is so the file layout has the more idiomatic top-level "docs" and "tests" directories. --- docs/BUILD.bazel | 2 +- sphinxdocs/.bazelrc.deleted_packages | 2 +- sphinxdocs/{sphinxdocs => }/docs/BUILD.bazel | 0 sphinxdocs/{sphinxdocs => }/docs/api/index.md | 0 .../docs/api/sphinxdocs/index.md | 0 .../docs/api/sphinxdocs/inventories/index.md | 0 sphinxdocs/{sphinxdocs => }/docs/index.md | 0 .../{sphinxdocs => }/docs/readthedocs.md | 0 .../{sphinxdocs => }/docs/sphinx-bzl.md | 0 .../{sphinxdocs => }/docs/starlark-docgen.md | 0 sphinxdocs/integration_tests/bcr/BUILD.bazel | 23 ++++++++++++++ sphinxdocs/integration_tests/bcr/MODULE.bazel | 30 +++++++++++++++++++ .../integration_tests/bcr/conf.py | 0 .../integration_tests/bcr/index.md | 0 sphinxdocs/sphinxdocs/BUILD.bazel | 6 +++- .../integration_tests/bcr/BUILD.bazel | 7 ----- .../integration_tests/bcr/MODULE.bazel | 16 ---------- sphinxdocs/sphinxdocs/private/BUILD.bazel | 2 +- sphinxdocs/{sphinxdocs => }/tests/BUILD.bazel | 0 sphinxdocs/tests/__init__.py | 0 .../tests/proto_to_markdown/BUILD.bazel | 0 .../proto_to_markdown_test.py | 3 +- .../tests/sphinx_docs/BUILD.bazel | 0 .../tests/sphinx_docs/conf.py | 0 .../tests/sphinx_docs/defs.bzl | 0 .../tests/sphinx_docs/doc1.md | 0 .../tests/sphinx_docs/doc2.md | 0 .../tests/sphinx_docs/index.md | 0 .../tests/sphinx_stardoc/BUILD.bazel | 0 sphinxdocs/tests/sphinx_stardoc/__init__.py | 0 .../tests/sphinx_stardoc/aspect.md | 0 .../tests/sphinx_stardoc/bzl_function.bzl | 0 .../tests/sphinx_stardoc/bzl_providers.bzl | 0 .../tests/sphinx_stardoc/bzl_rule.bzl | 0 .../tests/sphinx_stardoc/bzl_typedef.bzl | 0 .../tests/sphinx_stardoc/conf.py | 0 .../tests/sphinx_stardoc/envvars.md | 0 .../tests/sphinx_stardoc/function.md | 0 .../tests/sphinx_stardoc/glossary.md | 0 .../tests/sphinx_stardoc/index.md | 0 .../tests/sphinx_stardoc/module_extension.md | 0 .../tests/sphinx_stardoc/provider.md | 0 .../tests/sphinx_stardoc/repo_rule.md | 0 .../tests/sphinx_stardoc/rule.md | 0 .../sphinx_stardoc/sphinx_output_test.py | 3 +- .../tests/sphinx_stardoc/target.md | 0 .../tests/sphinx_stardoc/typedef.md | 0 .../tests/sphinx_stardoc/xrefs.md | 0 48 files changed, 63 insertions(+), 31 deletions(-) rename sphinxdocs/{sphinxdocs => }/docs/BUILD.bazel (100%) rename sphinxdocs/{sphinxdocs => }/docs/api/index.md (100%) rename sphinxdocs/{sphinxdocs => }/docs/api/sphinxdocs/index.md (100%) rename sphinxdocs/{sphinxdocs => }/docs/api/sphinxdocs/inventories/index.md (100%) rename sphinxdocs/{sphinxdocs => }/docs/index.md (100%) rename sphinxdocs/{sphinxdocs => }/docs/readthedocs.md (100%) rename sphinxdocs/{sphinxdocs => }/docs/sphinx-bzl.md (100%) rename sphinxdocs/{sphinxdocs => }/docs/starlark-docgen.md (100%) create mode 100644 sphinxdocs/integration_tests/bcr/BUILD.bazel create mode 100644 sphinxdocs/integration_tests/bcr/MODULE.bazel rename sphinxdocs/{sphinxdocs => }/integration_tests/bcr/conf.py (100%) rename sphinxdocs/{sphinxdocs => }/integration_tests/bcr/index.md (100%) delete mode 100644 sphinxdocs/sphinxdocs/integration_tests/bcr/BUILD.bazel delete mode 100644 sphinxdocs/sphinxdocs/integration_tests/bcr/MODULE.bazel rename sphinxdocs/{sphinxdocs => }/tests/BUILD.bazel (100%) create mode 100644 sphinxdocs/tests/__init__.py rename sphinxdocs/{sphinxdocs => }/tests/proto_to_markdown/BUILD.bazel (100%) rename sphinxdocs/{sphinxdocs => }/tests/proto_to_markdown/proto_to_markdown_test.py (99%) rename sphinxdocs/{sphinxdocs => }/tests/sphinx_docs/BUILD.bazel (100%) rename sphinxdocs/{sphinxdocs => }/tests/sphinx_docs/conf.py (100%) rename sphinxdocs/{sphinxdocs => }/tests/sphinx_docs/defs.bzl (100%) rename sphinxdocs/{sphinxdocs => }/tests/sphinx_docs/doc1.md (100%) rename sphinxdocs/{sphinxdocs => }/tests/sphinx_docs/doc2.md (100%) rename sphinxdocs/{sphinxdocs => }/tests/sphinx_docs/index.md (100%) rename sphinxdocs/{sphinxdocs => }/tests/sphinx_stardoc/BUILD.bazel (100%) create mode 100644 sphinxdocs/tests/sphinx_stardoc/__init__.py rename sphinxdocs/{sphinxdocs => }/tests/sphinx_stardoc/aspect.md (100%) rename sphinxdocs/{sphinxdocs => }/tests/sphinx_stardoc/bzl_function.bzl (100%) rename sphinxdocs/{sphinxdocs => }/tests/sphinx_stardoc/bzl_providers.bzl (100%) rename sphinxdocs/{sphinxdocs => }/tests/sphinx_stardoc/bzl_rule.bzl (100%) rename sphinxdocs/{sphinxdocs => }/tests/sphinx_stardoc/bzl_typedef.bzl (100%) rename sphinxdocs/{sphinxdocs => }/tests/sphinx_stardoc/conf.py (100%) rename sphinxdocs/{sphinxdocs => }/tests/sphinx_stardoc/envvars.md (100%) rename sphinxdocs/{sphinxdocs => }/tests/sphinx_stardoc/function.md (100%) rename sphinxdocs/{sphinxdocs => }/tests/sphinx_stardoc/glossary.md (100%) rename sphinxdocs/{sphinxdocs => }/tests/sphinx_stardoc/index.md (100%) rename sphinxdocs/{sphinxdocs => }/tests/sphinx_stardoc/module_extension.md (100%) rename sphinxdocs/{sphinxdocs => }/tests/sphinx_stardoc/provider.md (100%) rename sphinxdocs/{sphinxdocs => }/tests/sphinx_stardoc/repo_rule.md (100%) rename sphinxdocs/{sphinxdocs => }/tests/sphinx_stardoc/rule.md (100%) rename sphinxdocs/{sphinxdocs => }/tests/sphinx_stardoc/sphinx_output_test.py (98%) rename sphinxdocs/{sphinxdocs => }/tests/sphinx_stardoc/target.md (100%) rename sphinxdocs/{sphinxdocs => }/tests/sphinx_stardoc/typedef.md (100%) rename sphinxdocs/{sphinxdocs => }/tests/sphinx_stardoc/xrefs.md (100%) diff --git a/docs/BUILD.bazel b/docs/BUILD.bazel index 1c31a47d56..80aae58607 100644 --- a/docs/BUILD.bazel +++ b/docs/BUILD.bazel @@ -73,7 +73,7 @@ sphinx_docs( ":bzl_api_docs", ":py_api_srcs", ":py_runtime_pair", - "@sphinxdocs//sphinxdocs/docs:docs_lib", + "@sphinxdocs//docs:docs_lib", ], ) diff --git a/sphinxdocs/.bazelrc.deleted_packages b/sphinxdocs/.bazelrc.deleted_packages index fd6d39d64f..442c80e960 100644 --- a/sphinxdocs/.bazelrc.deleted_packages +++ b/sphinxdocs/.bazelrc.deleted_packages @@ -1 +1 @@ -common --deleted_packages=sphinxdocs/integration_tests/bcr +common --deleted_packages=integration_tests/bcr diff --git a/sphinxdocs/sphinxdocs/docs/BUILD.bazel b/sphinxdocs/docs/BUILD.bazel similarity index 100% rename from sphinxdocs/sphinxdocs/docs/BUILD.bazel rename to sphinxdocs/docs/BUILD.bazel diff --git a/sphinxdocs/sphinxdocs/docs/api/index.md b/sphinxdocs/docs/api/index.md similarity index 100% rename from sphinxdocs/sphinxdocs/docs/api/index.md rename to sphinxdocs/docs/api/index.md diff --git a/sphinxdocs/sphinxdocs/docs/api/sphinxdocs/index.md b/sphinxdocs/docs/api/sphinxdocs/index.md similarity index 100% rename from sphinxdocs/sphinxdocs/docs/api/sphinxdocs/index.md rename to sphinxdocs/docs/api/sphinxdocs/index.md diff --git a/sphinxdocs/sphinxdocs/docs/api/sphinxdocs/inventories/index.md b/sphinxdocs/docs/api/sphinxdocs/inventories/index.md similarity index 100% rename from sphinxdocs/sphinxdocs/docs/api/sphinxdocs/inventories/index.md rename to sphinxdocs/docs/api/sphinxdocs/inventories/index.md diff --git a/sphinxdocs/sphinxdocs/docs/index.md b/sphinxdocs/docs/index.md similarity index 100% rename from sphinxdocs/sphinxdocs/docs/index.md rename to sphinxdocs/docs/index.md diff --git a/sphinxdocs/sphinxdocs/docs/readthedocs.md b/sphinxdocs/docs/readthedocs.md similarity index 100% rename from sphinxdocs/sphinxdocs/docs/readthedocs.md rename to sphinxdocs/docs/readthedocs.md diff --git a/sphinxdocs/sphinxdocs/docs/sphinx-bzl.md b/sphinxdocs/docs/sphinx-bzl.md similarity index 100% rename from sphinxdocs/sphinxdocs/docs/sphinx-bzl.md rename to sphinxdocs/docs/sphinx-bzl.md diff --git a/sphinxdocs/sphinxdocs/docs/starlark-docgen.md b/sphinxdocs/docs/starlark-docgen.md similarity index 100% rename from sphinxdocs/sphinxdocs/docs/starlark-docgen.md rename to sphinxdocs/docs/starlark-docgen.md diff --git a/sphinxdocs/integration_tests/bcr/BUILD.bazel b/sphinxdocs/integration_tests/bcr/BUILD.bazel new file mode 100644 index 0000000000..412ba56ec3 --- /dev/null +++ b/sphinxdocs/integration_tests/bcr/BUILD.bazel @@ -0,0 +1,23 @@ +load("@bazel_skylib//rules:build_test.bzl", "build_test") +load("@sphinxdocs//sphinxdocs:sphinx.bzl", "sphinx_build_binary", "sphinx_docs") + +sphinx_docs( + name = "docs", + srcs = ["index.md"], + config = "conf.py", + formats = ["html"], + sphinx = ":sphinx-build", +) + +sphinx_build_binary( + name = "sphinx-build", + deps = [ + "@dev_pip//myst_parser", + "@dev_pip//sphinx", + ], +) + +build_test( + name = "docs_build_test", + targets = [":docs"], +) diff --git a/sphinxdocs/integration_tests/bcr/MODULE.bazel b/sphinxdocs/integration_tests/bcr/MODULE.bazel new file mode 100644 index 0000000000..711144df9b --- /dev/null +++ b/sphinxdocs/integration_tests/bcr/MODULE.bazel @@ -0,0 +1,30 @@ +module( + name = "sphinxdocs_example", + version = "0.0.0", +) + +bazel_dep(name = "sphinxdocs", version = "0.0.0") +local_path_override( + module_name = "sphinxdocs", + path = "../..", +) + +bazel_dep(name = "rules_python", version = "0.0.0") +local_path_override( + module_name = "rules_python", + path = "../../..", +) + +dev_pip = use_extension( + "@rules_python//python/extensions:pip.bzl", + "pip", + dev_dependency = True, +) +dev_pip.parse( + hub_name = "dev_pip", + python_version = "3.11", + requirements_lock = "@rules_python//docs:requirements.txt", +) +use_repo(dev_pip, "dev_pip") + +bazel_dep(name = "bazel_skylib", version = "1.8.2") diff --git a/sphinxdocs/sphinxdocs/integration_tests/bcr/conf.py b/sphinxdocs/integration_tests/bcr/conf.py similarity index 100% rename from sphinxdocs/sphinxdocs/integration_tests/bcr/conf.py rename to sphinxdocs/integration_tests/bcr/conf.py diff --git a/sphinxdocs/sphinxdocs/integration_tests/bcr/index.md b/sphinxdocs/integration_tests/bcr/index.md similarity index 100% rename from sphinxdocs/sphinxdocs/integration_tests/bcr/index.md rename to sphinxdocs/integration_tests/bcr/index.md diff --git a/sphinxdocs/sphinxdocs/BUILD.bazel b/sphinxdocs/sphinxdocs/BUILD.bazel index 893db8214a..5a498b197e 100644 --- a/sphinxdocs/sphinxdocs/BUILD.bazel +++ b/sphinxdocs/sphinxdocs/BUILD.bazel @@ -17,7 +17,7 @@ load("@bazel_skylib//rules:common_settings.bzl", "bool_flag") load("//sphinxdocs/private:sphinx.bzl", "repeated_string_list_flag") package( - default_visibility = ["//sphinxdocs:__subpackages__"], + default_visibility = ["//:__subpackages__"], ) # Additional -D values to add to every Sphinx build. @@ -44,23 +44,27 @@ bool_flag( bzl_library( name = "sphinx_bzl", srcs = ["sphinx.bzl"], + visibility = ["//visibility:public"], deps = ["//sphinxdocs/private:sphinx_bzl"], ) bzl_library( name = "sphinx_docs_library_bzl", srcs = ["sphinx_docs_library.bzl"], + visibility = ["//visibility:public"], deps = ["//sphinxdocs/private:sphinx_docs_library_macro_bzl"], ) bzl_library( name = "sphinx_stardoc_bzl", srcs = ["sphinx_stardoc.bzl"], + visibility = ["//visibility:public"], deps = ["//sphinxdocs/private:sphinx_stardoc_bzl"], ) bzl_library( name = "readthedocs_bzl", srcs = ["readthedocs.bzl"], + visibility = ["//visibility:public"], deps = ["//sphinxdocs/private:readthedocs_bzl"], ) diff --git a/sphinxdocs/sphinxdocs/integration_tests/bcr/BUILD.bazel b/sphinxdocs/sphinxdocs/integration_tests/bcr/BUILD.bazel deleted file mode 100644 index 1aaa69b826..0000000000 --- a/sphinxdocs/sphinxdocs/integration_tests/bcr/BUILD.bazel +++ /dev/null @@ -1,7 +0,0 @@ -load("@sphinxdocs//sphinxdocs:sphinx.bzl", "sphinx_docs") - -sphinx_docs( - name = "docs", - srcs = ["index.md"], - conf = "conf.py", -) diff --git a/sphinxdocs/sphinxdocs/integration_tests/bcr/MODULE.bazel b/sphinxdocs/sphinxdocs/integration_tests/bcr/MODULE.bazel deleted file mode 100644 index 6a25aa5e4c..0000000000 --- a/sphinxdocs/sphinxdocs/integration_tests/bcr/MODULE.bazel +++ /dev/null @@ -1,16 +0,0 @@ -module( - name = "sphinxdocs_example", - version = "0.0.0", -) - -bazel_dep(name = "sphinxdocs", version = "0.0.0") -local_path_override( - module_name = "sphinxdocs", - path = "../..", -) - -bazel_dep(name = "rules_python", version = "0.0.0") -local_path_override( - module_name = "rules_python", - path = "../../..", -) diff --git a/sphinxdocs/sphinxdocs/private/BUILD.bazel b/sphinxdocs/sphinxdocs/private/BUILD.bazel index 5a37cbd309..785d2e074d 100644 --- a/sphinxdocs/sphinxdocs/private/BUILD.bazel +++ b/sphinxdocs/sphinxdocs/private/BUILD.bazel @@ -18,7 +18,7 @@ load("@rules_python//python:py_binary.bzl", "py_binary") load("@rules_python//python:py_library.bzl", "py_library") package( - default_visibility = ["//sphinxdocs:__subpackages__"], + default_visibility = ["//:__subpackages__"], ) # These are only exported because they're passed as files to the @sphinxdocs diff --git a/sphinxdocs/sphinxdocs/tests/BUILD.bazel b/sphinxdocs/tests/BUILD.bazel similarity index 100% rename from sphinxdocs/sphinxdocs/tests/BUILD.bazel rename to sphinxdocs/tests/BUILD.bazel diff --git a/sphinxdocs/tests/__init__.py b/sphinxdocs/tests/__init__.py new file mode 100644 index 0000000000..e69de29bb2 diff --git a/sphinxdocs/sphinxdocs/tests/proto_to_markdown/BUILD.bazel b/sphinxdocs/tests/proto_to_markdown/BUILD.bazel similarity index 100% rename from sphinxdocs/sphinxdocs/tests/proto_to_markdown/BUILD.bazel rename to sphinxdocs/tests/proto_to_markdown/BUILD.bazel diff --git a/sphinxdocs/sphinxdocs/tests/proto_to_markdown/proto_to_markdown_test.py b/sphinxdocs/tests/proto_to_markdown/proto_to_markdown_test.py similarity index 99% rename from sphinxdocs/sphinxdocs/tests/proto_to_markdown/proto_to_markdown_test.py rename to sphinxdocs/tests/proto_to_markdown/proto_to_markdown_test.py index da6edb21d4..c42bcf0b22 100644 --- a/sphinxdocs/sphinxdocs/tests/proto_to_markdown/proto_to_markdown_test.py +++ b/sphinxdocs/tests/proto_to_markdown/proto_to_markdown_test.py @@ -17,9 +17,8 @@ from absl.testing import absltest from google.protobuf import text_format -from stardoc.proto import stardoc_output_pb2 - from sphinxdocs.private import proto_to_markdown +from stardoc.proto import stardoc_output_pb2 _EVERYTHING_MODULE = """\ module_docstring: "MODULE_DOC_STRING" diff --git a/sphinxdocs/sphinxdocs/tests/sphinx_docs/BUILD.bazel b/sphinxdocs/tests/sphinx_docs/BUILD.bazel similarity index 100% rename from sphinxdocs/sphinxdocs/tests/sphinx_docs/BUILD.bazel rename to sphinxdocs/tests/sphinx_docs/BUILD.bazel diff --git a/sphinxdocs/sphinxdocs/tests/sphinx_docs/conf.py b/sphinxdocs/tests/sphinx_docs/conf.py similarity index 100% rename from sphinxdocs/sphinxdocs/tests/sphinx_docs/conf.py rename to sphinxdocs/tests/sphinx_docs/conf.py diff --git a/sphinxdocs/sphinxdocs/tests/sphinx_docs/defs.bzl b/sphinxdocs/tests/sphinx_docs/defs.bzl similarity index 100% rename from sphinxdocs/sphinxdocs/tests/sphinx_docs/defs.bzl rename to sphinxdocs/tests/sphinx_docs/defs.bzl diff --git a/sphinxdocs/sphinxdocs/tests/sphinx_docs/doc1.md b/sphinxdocs/tests/sphinx_docs/doc1.md similarity index 100% rename from sphinxdocs/sphinxdocs/tests/sphinx_docs/doc1.md rename to sphinxdocs/tests/sphinx_docs/doc1.md diff --git a/sphinxdocs/sphinxdocs/tests/sphinx_docs/doc2.md b/sphinxdocs/tests/sphinx_docs/doc2.md similarity index 100% rename from sphinxdocs/sphinxdocs/tests/sphinx_docs/doc2.md rename to sphinxdocs/tests/sphinx_docs/doc2.md diff --git a/sphinxdocs/sphinxdocs/tests/sphinx_docs/index.md b/sphinxdocs/tests/sphinx_docs/index.md similarity index 100% rename from sphinxdocs/sphinxdocs/tests/sphinx_docs/index.md rename to sphinxdocs/tests/sphinx_docs/index.md diff --git a/sphinxdocs/sphinxdocs/tests/sphinx_stardoc/BUILD.bazel b/sphinxdocs/tests/sphinx_stardoc/BUILD.bazel similarity index 100% rename from sphinxdocs/sphinxdocs/tests/sphinx_stardoc/BUILD.bazel rename to sphinxdocs/tests/sphinx_stardoc/BUILD.bazel diff --git a/sphinxdocs/tests/sphinx_stardoc/__init__.py b/sphinxdocs/tests/sphinx_stardoc/__init__.py new file mode 100644 index 0000000000..e69de29bb2 diff --git a/sphinxdocs/sphinxdocs/tests/sphinx_stardoc/aspect.md b/sphinxdocs/tests/sphinx_stardoc/aspect.md similarity index 100% rename from sphinxdocs/sphinxdocs/tests/sphinx_stardoc/aspect.md rename to sphinxdocs/tests/sphinx_stardoc/aspect.md diff --git a/sphinxdocs/sphinxdocs/tests/sphinx_stardoc/bzl_function.bzl b/sphinxdocs/tests/sphinx_stardoc/bzl_function.bzl similarity index 100% rename from sphinxdocs/sphinxdocs/tests/sphinx_stardoc/bzl_function.bzl rename to sphinxdocs/tests/sphinx_stardoc/bzl_function.bzl diff --git a/sphinxdocs/sphinxdocs/tests/sphinx_stardoc/bzl_providers.bzl b/sphinxdocs/tests/sphinx_stardoc/bzl_providers.bzl similarity index 100% rename from sphinxdocs/sphinxdocs/tests/sphinx_stardoc/bzl_providers.bzl rename to sphinxdocs/tests/sphinx_stardoc/bzl_providers.bzl diff --git a/sphinxdocs/sphinxdocs/tests/sphinx_stardoc/bzl_rule.bzl b/sphinxdocs/tests/sphinx_stardoc/bzl_rule.bzl similarity index 100% rename from sphinxdocs/sphinxdocs/tests/sphinx_stardoc/bzl_rule.bzl rename to sphinxdocs/tests/sphinx_stardoc/bzl_rule.bzl diff --git a/sphinxdocs/sphinxdocs/tests/sphinx_stardoc/bzl_typedef.bzl b/sphinxdocs/tests/sphinx_stardoc/bzl_typedef.bzl similarity index 100% rename from sphinxdocs/sphinxdocs/tests/sphinx_stardoc/bzl_typedef.bzl rename to sphinxdocs/tests/sphinx_stardoc/bzl_typedef.bzl diff --git a/sphinxdocs/sphinxdocs/tests/sphinx_stardoc/conf.py b/sphinxdocs/tests/sphinx_stardoc/conf.py similarity index 100% rename from sphinxdocs/sphinxdocs/tests/sphinx_stardoc/conf.py rename to sphinxdocs/tests/sphinx_stardoc/conf.py diff --git a/sphinxdocs/sphinxdocs/tests/sphinx_stardoc/envvars.md b/sphinxdocs/tests/sphinx_stardoc/envvars.md similarity index 100% rename from sphinxdocs/sphinxdocs/tests/sphinx_stardoc/envvars.md rename to sphinxdocs/tests/sphinx_stardoc/envvars.md diff --git a/sphinxdocs/sphinxdocs/tests/sphinx_stardoc/function.md b/sphinxdocs/tests/sphinx_stardoc/function.md similarity index 100% rename from sphinxdocs/sphinxdocs/tests/sphinx_stardoc/function.md rename to sphinxdocs/tests/sphinx_stardoc/function.md diff --git a/sphinxdocs/sphinxdocs/tests/sphinx_stardoc/glossary.md b/sphinxdocs/tests/sphinx_stardoc/glossary.md similarity index 100% rename from sphinxdocs/sphinxdocs/tests/sphinx_stardoc/glossary.md rename to sphinxdocs/tests/sphinx_stardoc/glossary.md diff --git a/sphinxdocs/sphinxdocs/tests/sphinx_stardoc/index.md b/sphinxdocs/tests/sphinx_stardoc/index.md similarity index 100% rename from sphinxdocs/sphinxdocs/tests/sphinx_stardoc/index.md rename to sphinxdocs/tests/sphinx_stardoc/index.md diff --git a/sphinxdocs/sphinxdocs/tests/sphinx_stardoc/module_extension.md b/sphinxdocs/tests/sphinx_stardoc/module_extension.md similarity index 100% rename from sphinxdocs/sphinxdocs/tests/sphinx_stardoc/module_extension.md rename to sphinxdocs/tests/sphinx_stardoc/module_extension.md diff --git a/sphinxdocs/sphinxdocs/tests/sphinx_stardoc/provider.md b/sphinxdocs/tests/sphinx_stardoc/provider.md similarity index 100% rename from sphinxdocs/sphinxdocs/tests/sphinx_stardoc/provider.md rename to sphinxdocs/tests/sphinx_stardoc/provider.md diff --git a/sphinxdocs/sphinxdocs/tests/sphinx_stardoc/repo_rule.md b/sphinxdocs/tests/sphinx_stardoc/repo_rule.md similarity index 100% rename from sphinxdocs/sphinxdocs/tests/sphinx_stardoc/repo_rule.md rename to sphinxdocs/tests/sphinx_stardoc/repo_rule.md diff --git a/sphinxdocs/sphinxdocs/tests/sphinx_stardoc/rule.md b/sphinxdocs/tests/sphinx_stardoc/rule.md similarity index 100% rename from sphinxdocs/sphinxdocs/tests/sphinx_stardoc/rule.md rename to sphinxdocs/tests/sphinx_stardoc/rule.md diff --git a/sphinxdocs/sphinxdocs/tests/sphinx_stardoc/sphinx_output_test.py b/sphinxdocs/tests/sphinx_stardoc/sphinx_output_test.py similarity index 98% rename from sphinxdocs/sphinxdocs/tests/sphinx_stardoc/sphinx_output_test.py rename to sphinxdocs/tests/sphinx_stardoc/sphinx_output_test.py index c78089ac14..4ed6d4df94 100644 --- a/sphinxdocs/sphinxdocs/tests/sphinx_stardoc/sphinx_output_test.py +++ b/sphinxdocs/tests/sphinx_stardoc/sphinx_output_test.py @@ -1,10 +1,9 @@ import importlib.resources from xml.etree import ElementTree +import tests.sphinx_stardoc as sphinx_stardoc from absl.testing import absltest, parameterized -from sphinxdocs.tests import sphinx_stardoc - class SphinxOutputTest(parameterized.TestCase): def setUp(self): diff --git a/sphinxdocs/sphinxdocs/tests/sphinx_stardoc/target.md b/sphinxdocs/tests/sphinx_stardoc/target.md similarity index 100% rename from sphinxdocs/sphinxdocs/tests/sphinx_stardoc/target.md rename to sphinxdocs/tests/sphinx_stardoc/target.md diff --git a/sphinxdocs/sphinxdocs/tests/sphinx_stardoc/typedef.md b/sphinxdocs/tests/sphinx_stardoc/typedef.md similarity index 100% rename from sphinxdocs/sphinxdocs/tests/sphinx_stardoc/typedef.md rename to sphinxdocs/tests/sphinx_stardoc/typedef.md diff --git a/sphinxdocs/sphinxdocs/tests/sphinx_stardoc/xrefs.md b/sphinxdocs/tests/sphinx_stardoc/xrefs.md similarity index 100% rename from sphinxdocs/sphinxdocs/tests/sphinx_stardoc/xrefs.md rename to sphinxdocs/tests/sphinx_stardoc/xrefs.md From ad74ef7810c9689044b8c7a98a31daa4e75c3b34 Mon Sep 17 00:00:00 2001 From: Ignas Anikevicius <240938+aignas@users.noreply.github.com> Date: Sat, 11 Apr 2026 12:45:53 +0900 Subject: [PATCH 686/922] feat!(pypi): enable bazel downloader by default (#3691) Summary: - change: Set `https://pypi.org/simple` as the default index. - refactor: Leave the code for the legacy behaviour intact in case we need to do more work on the configuration to exclude the cases where the downloader should be used. - add: add `index_url` configuration option for setting the defaults. - add: add a small utility for parsing the arguments. - fix: downloader will not be used if there is no url associated with the source irrespective of what the index_url setting is. - fix: ensure all of the URLs are normalized when used. Fixes #260 Fixes #1357 Fixes #2241 Fixes #2951 --- .bazelrc.deleted_packages | 2 +- CHANGELOG.md | 29 +++++++-- MODULE.bazel | 7 +- docs/pypi/download.md | 27 ++------ examples/bzlmod/MODULE.bazel | 22 +++---- python/private/pypi/BUILD.bazel | 7 ++ python/private/pypi/argparse.bzl | 40 ++++++++++++ python/private/pypi/extension.bzl | 65 ++++++++++--------- python/private/pypi/hub_builder.bzl | 40 +++++------- python/private/pypi/parse_requirements.bzl | 17 +++++ .../pypi/requirements_files_by_platform.bzl | 29 +-------- python/private/pypi/simpleapi_download.bzl | 17 +++-- tests/pypi/extension/extension_tests.bzl | 3 + tests/pypi/hub_builder/hub_builder_tests.bzl | 31 +++++++-- .../parse_requirements_tests.bzl | 6 +- tests/pypi/select_whl/select_whl_tests.bzl | 1 - 16 files changed, 203 insertions(+), 140 deletions(-) create mode 100644 python/private/pypi/argparse.bzl diff --git a/.bazelrc.deleted_packages b/.bazelrc.deleted_packages index e767e0ae56..fb5d2ef0bb 100644 --- a/.bazelrc.deleted_packages +++ b/.bazelrc.deleted_packages @@ -27,8 +27,8 @@ common --deleted_packages=gazelle/manifest/hasher common --deleted_packages=gazelle/manifest/test common --deleted_packages=gazelle/modules_mapping common --deleted_packages=gazelle/python -common --deleted_packages=gazelle/python/private common --deleted_packages=gazelle/pythonconfig +common --deleted_packages=gazelle/python/private common --deleted_packages=tests/integration/compile_pip_requirements common --deleted_packages=tests/integration/compile_pip_requirements_test_from_external_repo common --deleted_packages=tests/integration/custom_commands diff --git a/CHANGELOG.md b/CHANGELOG.md index 8b6a0e521d..89b8356f00 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -67,6 +67,12 @@ END_UNRELEASED_TEMPLATE on supported platforms (Linux/Mac with Bazel 8+, or Windows). * `--build_python_zip` on Windows is ignored. Use {obj}`py_zipapp_binary` to create zips of Python programs. +* (pypi) Previously `experimental_index_url` users would not need to specify + target platforms if cross-building is required. From now we will only pull + wheels for the host OS to better align with how the rules work with the legacy + `pip` implementation. Use {obj}`pip.parse.target_platforms` to customize the + behavior. + Related to [#260](https://github.com/bazel-contrib/rules_python/issues/260). Other changes: * (pypi) Update dependencies used for `compile_pip_requirements`, building @@ -75,12 +81,10 @@ Other changes: we will from now on fetch the lists of available packages on each index. The used package mappings will be written as facts to the `MODULE.bazel.lock` file on supported bazel versions and it should be done at most once. As a result, - per-package {obj}`experimental_index_url_overrides` is no longer needed if the - index URLs are passed to the `pip.parse` via `experimental_index_url` and - `experimental_extra_index_urls`. What is more, we start implementing the flags - for `--index_url` and `--extra_index_urls` more in line to how it is used in - `uv` and `pip`, i.e. we default to `--index_url` if the package is not found in - `--extra_index_urls`. Fixes + per-package {obj}`experimental_index_url_overrides` is no longer needed . What + is more, the flags for `--index_url` and `--extra-index-url` now behave in the + same way as in `uv` or `pip`, i.e. we default to `--index-url` if the package + is not found in `--extra-index-url`. Fixes ([#3260](https://github.com/bazel-contrib/rules_python/issues/3260) and [#2632](https://github.com/bazel-contrib/rules_python/issues/2632)). * (uv) We will now use the download URL specified in the `uv`'s @@ -130,6 +134,17 @@ Other changes: initializations and should no longer require the network access if the cache is hydrated. Implements [#2731](https://github.com/bazel-contrib/rules_python/issues/2731). +* (pypi) The `--index-url` and `--extra-index-url` is now parsed from the lock + file and the {obj}`pip.parse.experimental_index_url` and + {obj}`pip.parse.experimental_extra_index_urls` is + no longer mandatory to leverage the bazel downloader. + Implements + [#1357](https://github.com/bazel-contrib/rules_python/issues/1357), + [#2951](https://github.com/bazel-contrib/rules_python/issues/2951). +* (pypi) If cross-compilation is needed, use the {obj}`pip.parse.target_platforms` + to specify exactly which platforms should be supported. + Implements + [#260](https://github.com/bazel-contrib/rules_python/issues/260). * (wheel) Specifying a path ending in `/` as a destination in `data_files` will now install file(s) to a folder, preserving their basename. * Various attributes and fields added to support venvs on Windows: @@ -2315,4 +2330,4 @@ Breaking changes: * (pip) Create all_data_requirements alias * Expose Python C headers through the toolchain. -[0.24.0]: https://github.com/bazel-contrib/rules_python/releases/tag/0.24.0 \ No newline at end of file +[0.24.0]: https://github.com/bazel-contrib/rules_python/releases/tag/0.24.0 diff --git a/MODULE.bazel b/MODULE.bazel index 28ce9fe7d4..95d6b9e3a9 100644 --- a/MODULE.bazel +++ b/MODULE.bazel @@ -65,9 +65,8 @@ register_toolchains("@pythons_hub//:all") pip = use_extension("//python/extensions:pip.bzl", "pip") -# NOTE @aignas 2025-07-06: we define these platforms to keep backwards compatibility with the -# current `experimental_index_url` implementation. Whilst we stabilize the API this list may be -# updated with a mention in the CHANGELOG. +# NOTE @aignas 2025-07-06: we define these platforms to keep backwards compatibility. Whilst we +# stabilize the API this list may be updated with a mention in the CHANGELOG. [ pip.default( arch_name = cpu, @@ -306,7 +305,6 @@ dev_pip = use_extension( [ dev_pip.parse( download_only = True, - experimental_index_url = "https://pypi.org/simple", hub_name = "dev_pip", parallel_download = False, python_version = python_version, @@ -329,7 +327,6 @@ dev_pip = use_extension( dev_pip.parse( download_only = True, - experimental_index_url = "https://pypi.org/simple", hub_name = "pypiserver", python_version = "3.11", requirements_lock = "//examples/wheel:requirements_server.txt", diff --git a/docs/pypi/download.md b/docs/pypi/download.md index b798d816f0..d4159eb3a7 100644 --- a/docs/pypi/download.md +++ b/docs/pypi/download.md @@ -104,9 +104,8 @@ the years, people started needing support for building containers, and usually, fetching dependencies for a particular target platform that may be different from the host platform. -Multi-platform support for cross-building the wheels can be done in two ways: -1. using {attr}`experimental_index_url` for the {bzl:obj}`pip.parse` bzlmod tag class -2. using the {attr}`pip.parse.download_only` setting. +Multi-platform support for cross-building the wheels can be done by +using {attr}`target_platforms` for the {bzl:obj}`pip.parse` bzlmod tag class :::{warning} This will not work for sdists with C extensions, but pure Python sdists may still work using the first @@ -207,16 +206,6 @@ additional keys, which become available during dependency evaluation. (bazel-downloader)= ### Bazel downloader and multi-platform wheel hub repository. -:::{warning} -This is currently still experimental, and whilst it has been proven to work in quite a few -environments, the APIs are still being finalized, and there may be changes to the APIs for this -feature without much notice. - -The issues that you can subscribe to for updates are: -* {gh-issue}`260` -* {gh-issue}`1357` -::: - The {obj}`pip` extension supports pulling information from `PyPI` (or a compatible mirror), and it will ensure that the [bazel downloader][bazel_downloader] is used for downloading the wheels. @@ -228,14 +217,10 @@ This provides the following benefits: * Allow using transitions and targeting free-threaded and musl platforms more easily. * Avoids `pip` for wheel fetching and results in much faster dependency fetching. -To enable the feature specify {attr}`pip.parse.experimental_index_url` as shown in -the {gh-path}`examples/bzlmod/MODULE.bazel` example. - -Similar to [uv](https://docs.astral.sh/uv/configuration/indexes/), one can override the -index that is used for a single package. By default, we first search in the index specified by -{attr}`pip.parse.experimental_index_url`, then we iterate through the -{attr}`pip.parse.experimental_extra_index_urls` unless there are overrides specified via -{attr}`pip.parse.experimental_index_url_overrides`. +Similar to [uv](https://docs.astral.sh/uv/configuration/indexes/), one can override the index that +is used for a single package. By default, we first search in the indexes specified by +`--extra-index-url`, then we fall back to the `--index-url` setting unless there are overrides +specified via {attr}`pip.parse.experimental_index_url_overrides`. When using this feature during the `pip` extension evaluation you will see the accessed indexes similar to below: ```console diff --git a/examples/bzlmod/MODULE.bazel b/examples/bzlmod/MODULE.bazel index 5c71d32421..106f25134e 100644 --- a/examples/bzlmod/MODULE.bazel +++ b/examples/bzlmod/MODULE.bazel @@ -183,18 +183,6 @@ pip.default( pip.parse( # We can use `envsubst in the above envsubst = ["PIP_INDEX_URL"], - # Use the bazel downloader to query the simple API for downloading the sources - # Note, that we can use envsubst for this value. - experimental_index_url = "${PIP_INDEX_URL:-https://pypi.org/simple}", - # One can also select a particular index for a particular package. - # This ensures that the setup is resistant against confusion attacks. - # experimental_index_url_overrides = { - # "my_package": "https://different-index-url.com", - # }, - # Or you can specify extra indexes like with `pip`: - # experimental_extra_index_urls = [ - # "https://different-index-url.com", - # ], experimental_requirement_cycles = { "sphinx": [ "sphinx", @@ -208,6 +196,16 @@ pip.parse( extra_hub_aliases = { "wheel": ["generated_file"], }, + extra_pip_args = [ + # Use the bazel downloader to query the simple API for downloading the sources + # Note, that we can use envsubst for this value. + # One can also select a particular index for a particular package. + # This ensures that the setup is resistant against confusion attacks. + # experimental_index_url_overrides = { + # "my_package": "https://different-index-url.com", + # }, + "--index-url=${PIP_INDEX_URL:-https://pypi.org/simple}", + ], hub_name = "pip", python_version = "3.9", requirements_lock = "requirements_lock_3_9.txt", diff --git a/python/private/pypi/BUILD.bazel b/python/private/pypi/BUILD.bazel index 869be4705a..9ed952889b 100644 --- a/python/private/pypi/BUILD.bazel +++ b/python/private/pypi/BUILD.bazel @@ -59,6 +59,11 @@ filegroup( # Keep sorted by library name and keep the files named by the main symbol they export +bzl_library( + name = "argparse_bzl", + srcs = ["argparse.bzl"], +) + bzl_library( name = "attrs_bzl", srcs = ["attrs.bzl"], @@ -224,6 +229,7 @@ bzl_library( name = "parse_requirements_bzl", srcs = ["parse_requirements.bzl"], deps = [ + ":argparse_bzl", ":index_sources_bzl", ":parse_requirements_txt_bzl", ":pypi_repo_utils_bzl", @@ -403,6 +409,7 @@ bzl_library( name = "requirements_files_by_platform_bzl", srcs = ["requirements_files_by_platform.bzl"], deps = [ + ":argparse_bzl", ":whl_target_platforms_bzl", ], ) diff --git a/python/private/pypi/argparse.bzl b/python/private/pypi/argparse.bzl new file mode 100644 index 0000000000..8964411a3e --- /dev/null +++ b/python/private/pypi/argparse.bzl @@ -0,0 +1,40 @@ +"""A small set of utilities for parsing pip args.""" + +def _get_pip_args(args, *arg_names, value = None, repeated = False): + set_next = False + if repeated: + value = [] + (value or []) + + for arg in (args or []): + if arg in arg_names: + set_next = True + continue + + val = None + if set_next: + set_next = False + val = arg + else: + for arg_name in arg_names: + start = "{}=".format(arg_name) + + if arg.startswith(start): + val = arg[len(start):] + break + + if val == None: + continue + + if repeated: + if val not in value: + value.append(val) + else: + value = val + + return value + +argparse = struct( + index_url = lambda args, default: _get_pip_args(args, "-i", "--index-url", value = default), + extra_index_url = lambda args, default: _get_pip_args(args, "--extra-index-url", value = default, repeated = True), + platform = lambda args, default: _get_pip_args(args, "--platform", value = default, repeated = True), +) diff --git a/python/private/pypi/extension.bzl b/python/private/pypi/extension.bzl index f68596b845..a55cf53fb1 100644 --- a/python/private/pypi/extension.bzl +++ b/python/private/pypi/extension.bzl @@ -115,6 +115,7 @@ def build_config( _configure( defaults, override = mod.is_root, + index_url = tag.index_url, # extra values that we just add auth_patterns = tag.auth_patterns, netrc = tag.netrc, @@ -125,6 +126,7 @@ def build_config( return struct( auth_patterns = defaults.get("auth_patterns", {}), + index_url = defaults.get("index_url", "https://pypi.org/simple").rstrip("/"), netrc = defaults.get("netrc", None), platforms = { name: _plat(**values) @@ -449,6 +451,30 @@ Supported keys: ::::{note} This is only used if the {envvar}`RULES_PYTHON_ENABLE_PIPSTAR` is enabled. :::: +""", + ), + "index_url": attr.string( + doc = """\ +The index URL to use as a default when downloading packages from PyPI. This is used if nothing is +specified via `--index-url` or `--extra-index-url` parameters in the `requirements.txt` file or via +the {attr}`pip.parse.extra_pip_args`. + +This value is going to be subject to `envsubst` substitutions if necessary, look at the +{attr}`pip.parse.envsubst` documentation for more information.. + +The indexes must support Simple API as described here: +https://packaging.python.org/en/latest/specifications/simple-repository-api/ + +Index metadata will be used to get `sha256` values for packages even if the +`sha256` values are not present in the requirements.txt lock file. + +Defaults to `https://pypi.org/simple`. + +:::{versionadded} 2.0.0 +This has been added as a replacement for +{obj}`pip.parse.experimental_index_url` and +{obj}`pip.parse.experimental_extra_index_urls`. +::: """, ), "marker": attr.string( @@ -566,17 +592,11 @@ def _pip_parse_ext_attrs(**kwargs): attrs = dict({ "experimental_extra_index_urls": attr.string_list( doc = """\ -The extra index URLs to use for downloading wheels using bazel downloader. -Each value is going to be subject to `envsubst` substitutions if necessary. +May be removed in future releases. -The indexes must support Simple API as described here: -https://packaging.python.org/en/latest/specifications/simple-repository-api/ - -This is equivalent to `--extra-index-urls` `pip` option. - -:::{versionchanged} 1.1.0 -Starting with this version we will iterate over each index specified until -we find metadata for all references distributions. +:::{versionchanged} 2.0.0 +This is deprecated, please use {obj}`pip.default.index_url` or pass the `--index-url` parameter via the +lock-file or {obj}`pip.parse.extra_pip_args`. ::: """, default = [], @@ -584,25 +604,11 @@ we find metadata for all references distributions. "experimental_index_url": attr.string( default = kwargs.get("experimental_index_url", ""), doc = """\ -The index URL to use for downloading wheels using bazel downloader. This value is going -to be subject to `envsubst` substitutions if necessary. - -The indexes must support Simple API as described here: -https://packaging.python.org/en/latest/specifications/simple-repository-api/ - -In the future this could be defaulted to `https://pypi.org` when this feature becomes -stable. - -This is equivalent to `--index-url` `pip` option. +May be removed in future releases. -:::{versionchanged} 0.37.0 -If {attr}`download_only` is set, then `sdist` archives will be discarded and `pip.parse` will -operate in wheel-only mode. -::: - -:::{versionchanged} 1.4.0 -Index metadata will be used to deduct `sha256` values for packages even if the -`sha256` values are not present in the requirements.txt lock file. +:::{versionchanged} 2.0.0 +This is deprecated, please use {obj}`pip.default.index_url` or pass the `--index-url` parameter via the +lock-file or {obj}`pip.parse.extra_pip_args`. ::: """, ), @@ -835,9 +841,6 @@ the BUILD files for wheels. This tag class allows for more customization of how the configuration for the hub repositories is built. -:::{include} /_includes/experimental_api.md -::: - :::{seealso} The [environment markers][environment_markers] specification for the explanation of the terms used in this extension. diff --git a/python/private/pypi/hub_builder.bzl b/python/private/pypi/hub_builder.bzl index e84f3b0ae8..484731566f 100644 --- a/python/private/pypi/hub_builder.bzl +++ b/python/private/pypi/hub_builder.bzl @@ -181,15 +181,12 @@ def _pip_parse(self, module_ctx, pip_attr): )) return - default_cross_setup = _set_get_index_urls(self, pip_attr) + _set_get_index_urls(self, pip_attr) self._platforms[python_version] = _platforms( module_ctx, python_version = full_python_version, config = self._config, - # TODO @aignas 2025-12-09: flip or part to default to 'os_arch' after - # 1.8.0 is released and set the default of the `target_platforms` attribute - # to `{os}_{arch}`. - target_platforms = pip_attr.target_platforms or ([] if default_cross_setup else ["{os}_{arch}"]), + target_platforms = pip_attr.target_platforms or ["{os}_{arch}"], ) _add_group_map(self, pip_attr.experimental_requirement_cycles) _add_extra_aliases(self, pip_attr.extra_hub_aliases) @@ -197,8 +194,8 @@ def _pip_parse(self, module_ctx, pip_attr): self, module_ctx, pip_attr = pip_attr, - enable_pipstar = self._config.enable_pipstar or self._get_index_urls.get(pip_attr.python_version), - enable_pipstar_extract = self._config.enable_pipstar_extract or self._get_index_urls.get(pip_attr.python_version), + enable_pipstar = bool(self._config.enable_pipstar or self._get_index_urls.get(pip_attr.python_version)), + enable_pipstar_extract = bool(self._config.enable_pipstar_extract or self._get_index_urls.get(pip_attr.python_version)), ) ### end of PUBLIC methods @@ -368,18 +365,10 @@ def _add_whl_library(self, *, python_version, whl, repo, enable_pipstar): ### end of setters, below we have various functions to implement the public methods def _set_get_index_urls(self, pip_attr): - if not pip_attr.experimental_index_url: - if pip_attr.experimental_extra_index_urls: - fail("'experimental_extra_index_urls' is a no-op unless 'experimental_index_url' is set") - elif pip_attr.experimental_index_url_overrides: - fail("'experimental_index_url_overrides' is a no-op unless 'experimental_index_url' is set") - elif pip_attr.simpleapi_skip: - fail("'simpleapi_skip' is a no-op unless 'experimental_index_url' is set") - elif pip_attr.netrc: - fail("'netrc' is a no-op unless 'experimental_index_url' is set") - elif pip_attr.auth_patterns: - fail("'auth_patterns' is a no-op unless 'experimental_index_url' is set") + default_index_url = pip_attr.experimental_index_url or self._config.index_url + default_extra_index_urls = pip_attr.experimental_extra_index_urls or [] + if not default_index_url: # parallel_download is set to True by default, so we are not checking/validating it # here return False @@ -389,11 +378,14 @@ def _set_get_index_urls(self, pip_attr): normalize_name(s): False for s in pip_attr.simpleapi_skip }) - self._get_index_urls[python_version] = lambda ctx, distributions: self._simpleapi_download_fn( + self._get_index_urls[python_version] = lambda ctx, distributions, *, index_url = None, extra_index_urls = None: self._simpleapi_download_fn( ctx, attr = struct( - index_url = pip_attr.experimental_index_url, - extra_index_urls = pip_attr.experimental_extra_index_urls or [], + index_url = (index_url or default_index_url).rstrip("/"), + extra_index_urls = [ + x.rstrip("/") + for x in (extra_index_urls or default_extra_index_urls) + ], index_url_overrides = pip_attr.experimental_index_url_overrides or {}, sources = { d: versions @@ -402,8 +394,8 @@ def _set_get_index_urls(self, pip_attr): }, envsubst = pip_attr.envsubst, # Auth related info - netrc = pip_attr.netrc, - auth_patterns = pip_attr.auth_patterns, + netrc = self._config.netrc or pip_attr.netrc, + auth_patterns = self._config.auth_patterns or pip_attr.auth_patterns, ), cache = self._simpleapi_cache, parallel_download = pip_attr.parallel_download, @@ -603,7 +595,7 @@ def _create_whl_repos( whl_library_args = whl_library_args, download_only = pip_attr.download_only, netrc = self._config.netrc or pip_attr.netrc, - use_downloader = _use_downloader(self, pip_attr.python_version, whl.name), + use_downloader = src.url and _use_downloader(self, pip_attr.python_version, whl.name), auth_patterns = self._config.auth_patterns or pip_attr.auth_patterns, python_version = _major_minor_version(pip_attr.python_version), is_multiple_versions = whl.is_multiple_versions, diff --git a/python/private/pypi/parse_requirements.bzl b/python/private/pypi/parse_requirements.bzl index 081e5f102f..2a7793212a 100644 --- a/python/private/pypi/parse_requirements.bzl +++ b/python/private/pypi/parse_requirements.bzl @@ -28,6 +28,7 @@ behavior. load("//python/private:normalize_name.bzl", "normalize_name") load("//python/private:repo_utils.bzl", "repo_utils") +load(":argparse.bzl", "argparse") load(":index_sources.bzl", "index_sources") load(":parse_requirements_txt.bzl", "parse_requirements_txt") load(":pep508_requirement.bzl", "requirement") @@ -89,6 +90,8 @@ def parse_requirements( options = {} requirements = {} reqs_with_env_markers = {} + index_url = None + extra_index_urls = [] for file, plats in requirements_by_platform.items(): logger.trace(lambda: "Using {} for {}".format(file, plats)) contents = ctx.read(file) @@ -113,6 +116,18 @@ def parse_requirements( reqs_with_env_markers.setdefault(requirement_line, []).append(plat) options[plat] = pip_args + # Parse the index URL from the requirement files + index_url = argparse.index_url(pip_args, index_url) + extra_index_urls = argparse.extra_index_url(pip_args, []) + platform = argparse.platform(pip_args, []) + if platform: + # No use of downloader if the user specifies "--platform" pip arg. This means that + # they intend to use pip to download the wheels + # + # TODO @aignas 2026-04-11: consider removing this line in the next major release + # (3.0). + get_index_urls = None + # This may call to Python, so execute it early (before calling to the # internet below) and ensure that we call it only once. resolved_marker_platforms = evaluate_markers(ctx, reqs_with_env_markers) @@ -181,6 +196,8 @@ def parse_requirements( index_urls = get_index_urls( ctx, distributions, + index_url = index_url, + extra_index_urls = extra_index_urls, ) ret = [] diff --git a/python/private/pypi/requirements_files_by_platform.bzl b/python/private/pypi/requirements_files_by_platform.bzl index c0fc93a0fe..725c9984cc 100644 --- a/python/private/pypi/requirements_files_by_platform.bzl +++ b/python/private/pypi/requirements_files_by_platform.bzl @@ -14,6 +14,7 @@ """Get the requirement files by platform.""" +load(":argparse.bzl", "argparse") load(":whl_target_platforms.bzl", "whl_target_platforms") def _default_platforms(*, filter, platforms): @@ -46,33 +47,9 @@ def _default_platforms(*, filter, platforms): return match def _platforms_from_args(extra_pip_args): - platform_values = [] - - if not extra_pip_args: - return platform_values - - for arg in extra_pip_args: - if platform_values and platform_values[-1] == "": - platform_values[-1] = arg - continue - - if arg == "--platform": - platform_values.append("") - continue - - if not arg.startswith("--platform"): - continue - - _, _, plat = arg.partition("=") - if not plat: - _, _, plat = arg.partition(" ") - if plat: - platform_values.append(plat) - else: - platform_values.append("") - + platform_values = argparse.platform(extra_pip_args, []) if not platform_values: - return [] + return platform_values platforms = { p.target_platform: None diff --git a/python/private/pypi/simpleapi_download.bzl b/python/private/pypi/simpleapi_download.bzl index 2171e8b56a..97215d753d 100644 --- a/python/private/pypi/simpleapi_download.bzl +++ b/python/private/pypi/simpleapi_download.bzl @@ -71,6 +71,9 @@ def simpleapi_download( Returns: dict of pkg name to the parsed HTML contents - a list of structs. """ + if not attr.sources: + return {} + index_url_overrides = { normalize_name(p): i for p, i in (attr.index_url_overrides or {}).items() @@ -131,6 +134,9 @@ def simpleapi_download( def _get_dist_urls(ctx, *, default_index, index_urls, index_url_overrides, sources, read_simpleapi, attr, block, _fail = fail, **kwargs): downloads = {} results = {} + + # Ensure the value is not frozen + index_urls = [] + (index_urls or []) for extra in index_url_overrides.values(): if extra not in index_urls: index_urls.append(extra) @@ -143,9 +149,7 @@ def _get_dist_urls(ctx, *, default_index, index_urls, index_url_overrides, sourc download = read_simpleapi( ctx = ctx, attr = attr, - url = urllib.strip_empty_path_segments("{index_url}/".format( - index_url = index_url, - )), + url = _normalize_url("{index_url}/".format(index_url = index_url)), parse_index = True, versions = {pkg: None for pkg in sources}, block = block, @@ -180,13 +184,16 @@ def _get_dist_urls(ctx, *, default_index, index_urls, index_url_overrides, sourc # Ignore the URL here because we know how to construct it. - found_on_index[pkg] = urllib.strip_empty_path_segments("{}/{}/".format( + found_on_index[pkg] = _normalize_url("{}/{}/".format( index_url, pkg.replace("_", "-"), # Use the official normalization for URLs )) return found_on_index +def _normalize_url(url): + return urllib.strip_empty_path_segments(url) + def _read_simpleapi(ctx, url, attr, cache, versions, parse_index, get_auth = None, **download_kwargs): """Read SimpleAPI. @@ -211,7 +218,7 @@ def _read_simpleapi(ctx, url, attr, cache, versions, parse_index, get_auth = Non A similar object to what `download` would return except that in result.out will be the parsed simple api contents. """ - real_url = urllib.strip_empty_path_segments(envsubst(url, attr.envsubst, ctx.getenv)) + real_url = _normalize_url(envsubst(url, attr.envsubst, ctx.getenv)) cache_key = (url, real_url, versions) cached_result = cache.get(cache_key) diff --git a/tests/pypi/extension/extension_tests.bzl b/tests/pypi/extension/extension_tests.bzl index 1eae1ed433..7999e42a7b 100644 --- a/tests/pypi/extension/extension_tests.bzl +++ b/tests/pypi/extension/extension_tests.bzl @@ -116,6 +116,7 @@ def _default( auth_patterns = None, config_settings = None, env = None, + index_url = None, marker = None, netrc = None, os_name = None, @@ -127,6 +128,7 @@ def _default( auth_patterns = auth_patterns or {}, config_settings = config_settings, env = env or {}, + index_url = index_url or "", marker = marker or "", netrc = netrc, os_name = os_name, @@ -145,6 +147,7 @@ def _test_simple(env): _parse( hub_name = "pypi", python_version = "3.15", + simpleapi_skip = ["simple"], requirements_lock = "requirements.txt", ), ], diff --git a/tests/pypi/hub_builder/hub_builder_tests.bzl b/tests/pypi/hub_builder/hub_builder_tests.bzl index 31a41f6af5..75a6edeae4 100644 --- a/tests/pypi/hub_builder/hub_builder_tests.bzl +++ b/tests/pypi/hub_builder/hub_builder_tests.bzl @@ -59,6 +59,7 @@ def hub_builder( # no need to evaluate the markers with the interpreter enable_pipstar = enable_pipstar, enable_pipstar_extract = enable_pipstar_extract, + index_url = "https://pypi.org/simple", platforms = { "{}_{}{}".format(os, cpu, freethreaded): _plat( name = "{}_{}{}".format(os, cpu, freethreaded), @@ -285,6 +286,7 @@ def _test_simple_extras_vs_no_extras_simpleapi(env): requirements_darwin = "darwin.txt", requirements_windows = "win.txt", experimental_index_url = "https://example.com", + target_platforms = ["osx_aarch64", "windows_aarch64"], ), ) pypi = builder.build() @@ -539,6 +541,7 @@ def _test_torch_experimental_index_url(env): netrc = None, enable_pipstar = True, enable_pipstar_extract = True, + index_url = "https://pypi.org/simple", auth_patterns = {}, platforms = { "{}_{}".format(os, cpu): _plat( @@ -609,6 +612,16 @@ torch==2.4.1+cpu ; platform_machine == 'x86_64' \ python_version = "3.12", experimental_index_url = "https://torch.index", requirements_lock = "universal.txt", + target_platforms = [ + "linux_x86_64", + "linux_aarch64", + # this should be ignored as well because there is no sdist and no whls + # for intel Macs + "osx_x86_64", + "osx_aarch64", + "windows_x86_64", + "windows_aarch64", + ], ), ) pypi = builder.build() @@ -802,7 +815,7 @@ def _test_simple_get_index(env): sha256s_by_version = { "0.0.4": ["deadb44f"], }, - index_url = "", + index_url = "https://pypi.org/simple", ), "simple": struct( whls = { @@ -821,7 +834,7 @@ def _test_simple_get_index(env): url = "example.org", ), }, - index_url = "", + index_url = "https://pypi.org/simple", ), "some_other_pkg": struct( whls = { @@ -872,10 +885,16 @@ git_dep @ git+https://git.server/repo/project@deadbeefdeadbeef hub_name = "pypi", python_version = "3.15", requirements_lock = "requirements.txt", - experimental_index_url = "pypi.org", extra_pip_args = [ "--extra-args-for-sdist-building", ], + target_platforms = [ + "linux_aarch64", + "linux_x86_64", + "linux_x86_64_freethreaded", + "osx_aarch64", + "windows_aarch64", + ], ), ) pypi = builder.build() @@ -1027,6 +1046,7 @@ git_dep @ git+https://git.server/repo/project@deadbeefdeadbeef "config_load": "@pypi//:config.bzl", "dep_template": "@pypi//{name}:{target}", "filename": "plat-pkg-0.0.4-py3-none-linux_x86_64.whl", + "index_url": "https://pypi.org/simple", "requirement": "plat_pkg==0.0.4", "sha256": "deadb44f", "urls": ["example2.org/index/plat_pkg/"], @@ -1035,6 +1055,7 @@ git_dep @ git+https://git.server/repo/project@deadbeefdeadbeef "config_load": "@pypi//:config.bzl", "dep_template": "@pypi//{name}:{target}", "filename": "simple-0.0.1-py3-none-any.whl", + "index_url": "https://pypi.org/simple", "requirement": "simple==0.0.1", "sha256": "deadb00f", "urls": ["example2.org"], @@ -1064,7 +1085,7 @@ git_dep @ git+https://git.server/repo/project@deadbeefdeadbeef auth_patterns = {}, envsubst = {}, extra_index_urls = [], - index_url = "pypi.org", + index_url = "https://pypi.org/simple", index_url_overrides = {}, netrc = None, sources = { @@ -1137,6 +1158,7 @@ def _test_pipstar_platforms(env): config = struct( enable_pipstar = True, enable_pipstar_extract = True, + index_url = "https://pypi.org/simple", netrc = None, auth_patterns = {}, platforms = { @@ -1222,6 +1244,7 @@ def _test_pipstar_platforms_limit(env): config = struct( enable_pipstar = True, enable_pipstar_extract = True, + index_url = "https://pypi.org/simple", netrc = None, auth_patterns = {}, platforms = { diff --git a/tests/pypi/parse_requirements/parse_requirements_tests.bzl b/tests/pypi/parse_requirements/parse_requirements_tests.bzl index bea8ac5f78..053ab31ec1 100644 --- a/tests/pypi/parse_requirements/parse_requirements_tests.bzl +++ b/tests/pypi/parse_requirements/parse_requirements_tests.bzl @@ -672,7 +672,7 @@ def _test_overlapping_shas_with_index_results(env): whl_platform_tags = ["macosx_*_x86_64"], ), }, - get_index_urls = lambda _, __: { + get_index_urls = lambda _, __, **kwargs: { "foo": struct( index_url = "https://example.com", sdists = { @@ -762,7 +762,7 @@ def _test_get_index_urls_different_versions(env): whl_platform_tags = ["any"], ), }, - get_index_urls = lambda _, __: { + get_index_urls = lambda _, __, **kwargs: { "foo": struct( index_url = "", sdists = {}, @@ -846,7 +846,7 @@ def _test_get_index_urls_single_py_version(env): whl_platform_tags = ["any"], ), }, - get_index_urls = lambda _, __: { + get_index_urls = lambda _, __, **kwargs: { "foo": struct( index_url = "", sdists = {}, diff --git a/tests/pypi/select_whl/select_whl_tests.bzl b/tests/pypi/select_whl/select_whl_tests.bzl index ea3670c4f7..8d2170ed7f 100644 --- a/tests/pypi/select_whl/select_whl_tests.bzl +++ b/tests/pypi/select_whl/select_whl_tests.bzl @@ -449,7 +449,6 @@ def _test_multiple_musllinux_exact_params(env): whl_abi_tags = ["none"], python_version = "3.12", limit = 2, - debug = True, ) _match( env, From 32846c6e49fc1351f17fe4e0d19f3926158e394c Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Sat, 11 Apr 2026 22:42:43 -0700 Subject: [PATCH 687/922] fix: namespace package calculation on windows (#3693) Currently, when the repo computes the namespace package files on Windows, it incorrectly computes the filename because of `C:` handling and case-sensitivity, resulting in a repo-phase error. To fix, modify the relative path computation functions to detect if the platform is Windows and make comparisions case-insensitive. --- python/private/pypi/pypi_repo_utils.bzl | 5 ++- python/private/repo_utils.bzl | 38 ++++++++++++---- tests/repo_utils/BUILD.bazel | 3 ++ tests/repo_utils/repo_utils_test.bzl | 59 +++++++++++++++++++++++++ tests/support/mocks.bzl | 31 +++++++++++++ 5 files changed, 125 insertions(+), 11 deletions(-) create mode 100644 tests/repo_utils/BUILD.bazel create mode 100644 tests/repo_utils/repo_utils_test.bzl create mode 100644 tests/support/mocks.bzl diff --git a/python/private/pypi/pypi_repo_utils.bzl b/python/private/pypi/pypi_repo_utils.bzl index d8e320014f..8ec7bd1dbe 100644 --- a/python/private/pypi/pypi_repo_utils.bzl +++ b/python/private/pypi/pypi_repo_utils.bzl @@ -177,7 +177,6 @@ def _find_namespace_package_files(rctx, install_dir): to namespace packages. """ - repo_root = str(rctx.path(".")) + "/" namespace_package_files = [] for top_level_dir in install_dir.readdir(): if not is_importable_name(top_level_dir.basename): @@ -192,7 +191,9 @@ def _find_namespace_package_files(rctx, install_dir): if ("__path__ =" in content and "pkgutil" in content and "extend_path(" in content): - namespace_package_files.append(str(init_py).removeprefix(repo_root)) + namespace_package_files.append( + repo_utils.repo_root_relative_path(rctx, init_py), + ) return namespace_package_files diff --git a/python/private/repo_utils.bzl b/python/private/repo_utils.bzl index a558fa08e1..7ec45eda5b 100644 --- a/python/private/repo_utils.bzl +++ b/python/private/repo_utils.bzl @@ -334,7 +334,7 @@ def _mkdir(mrctx, path): repo_root = str(mrctx.path(".")) path_str = str(path) - if not path_str.startswith(repo_root): + if not _is_relative_to(mrctx, path_str, repo_root): mkdir_bin = mrctx.which("mkdir") if not mkdir_bin: return None @@ -348,6 +348,30 @@ def _mkdir(mrctx, path): mrctx.delete(placeholder) return path +def _norm_path(mrctx, p): + p = str(p) + + # Windows is case-insensitive + if _get_platforms_os_name(mrctx) == "windows": + return p.lower() + return p + +def _relative_to(mrctx, path, parent, fail = fail): + path_str = str(path) + parent_str = str(parent) + path_d = _norm_path(mrctx, path_str) + "/" + parent_d = _norm_path(mrctx, parent_str) + "/" + if path_d.startswith(parent_d): + return path_str[len(parent_str):].removeprefix("/") + else: + fail("{} is not relative to {}".format(path, parent)) + +def _is_relative_to(mrctx, path, parent): + """Tell if `path` is equal to or beneath `parent`.""" + path_d = _norm_path(mrctx, path) + "/" + parent_d = _norm_path(mrctx, parent) + "/" + return path_d.startswith(parent_d) + def _repo_root_relative_path(mrctx, path): """Takes a path object and returns a repo-relative path string. @@ -360,14 +384,7 @@ def _repo_root_relative_path(mrctx, path): """ repo_root = str(mrctx.path(".")) path_str = str(path) - relative_path = path_str[len(repo_root):] - if relative_path[0] != "/": - fail("{path} not under {repo_root}".format( - path = path, - repo_root = repo_root, - )) - relative_path = relative_path[1:] - return relative_path + return _relative_to(mrctx, path_str, repo_root) def _args_to_str(arguments): return " ".join([_arg_repr(a) for a in arguments]) @@ -516,6 +533,9 @@ repo_utils = struct( is_repo_debug_enabled = _is_repo_debug_enabled, logger = _logger, mkdir = _mkdir, + norm_path = _norm_path, + relative_to = _relative_to, + is_relative_to = _is_relative_to, repo_root_relative_path = _repo_root_relative_path, which_checked = _which_checked, which_unchecked = _which_unchecked, diff --git a/tests/repo_utils/BUILD.bazel b/tests/repo_utils/BUILD.bazel new file mode 100644 index 0000000000..74e8e37489 --- /dev/null +++ b/tests/repo_utils/BUILD.bazel @@ -0,0 +1,3 @@ +load(":repo_utils_test.bzl", "repo_utils_test_suite") + +repo_utils_test_suite(name = "repo_utils_tests") diff --git a/tests/repo_utils/repo_utils_test.bzl b/tests/repo_utils/repo_utils_test.bzl new file mode 100644 index 0000000000..ce9e48b5a6 --- /dev/null +++ b/tests/repo_utils/repo_utils_test.bzl @@ -0,0 +1,59 @@ +"""Unit tests for repo_utils.bzl.""" + +load("@rules_testing//lib:test_suite.bzl", "test_suite") +load("//python/private:repo_utils.bzl", "repo_utils") # buildifier: disable=bzl-visibility +load("//tests/support:mocks.bzl", "mocks") + +_tests = [] + +def _test_get_platforms_os_name(env): + mock_mrctx = mocks.rctx(os_name = "Mac OS X") + got = repo_utils.get_platforms_os_name(mock_mrctx) + env.expect.that_str(got).equals("osx") + +_tests.append(_test_get_platforms_os_name) + +def _test_relative_to(env): + mock_mrctx_linux = mocks.rctx(os_name = "linux") + mock_mrctx_win = mocks.rctx(os_name = "windows") + + # Case-sensitive matching (Linux) + got = repo_utils.relative_to(mock_mrctx_linux, "foo/bar/baz", "foo/bar") + env.expect.that_str(got).equals("baz") + + # Case-insensitive matching (Windows) + got = repo_utils.relative_to(mock_mrctx_win, "C:/Foo/Bar/Baz", "c:/foo/bar") + env.expect.that_str(got).equals("Baz") + + # Failure case + failures = [] + + def _mock_fail(msg): + failures.append(msg) + + repo_utils.relative_to(mock_mrctx_linux, "foo/bar/baz", "qux", fail = _mock_fail) + env.expect.that_collection(failures).contains_exactly(["foo/bar/baz is not relative to qux"]) + +_tests.append(_test_relative_to) + +def _test_is_relative_to(env): + mock_mrctx_linux = mocks.rctx(os_name = "linux") + mock_mrctx_win = mocks.rctx(os_name = "windows") + + # Case-sensitive matching (Linux) + env.expect.that_bool(repo_utils.is_relative_to(mock_mrctx_linux, "foo/bar/baz", "foo/bar")).equals(True) + env.expect.that_bool(repo_utils.is_relative_to(mock_mrctx_linux, "foo/bar/baz", "qux")).equals(False) + + # Case-insensitive matching (Windows) + env.expect.that_bool(repo_utils.is_relative_to(mock_mrctx_win, "C:/Foo/Bar/Baz", "c:/foo/bar")).equals(True) + env.expect.that_bool(repo_utils.is_relative_to(mock_mrctx_win, "C:/Foo/Bar/Baz", "D:/Foo")).equals(False) + +_tests.append(_test_is_relative_to) + +def repo_utils_test_suite(name): + """Create the test suite. + + Args: + name: the name of the test suite + """ + test_suite(name = name, basic_tests = _tests) diff --git a/tests/support/mocks.bzl b/tests/support/mocks.bzl new file mode 100644 index 0000000000..2a4ccd0fc4 --- /dev/null +++ b/tests/support/mocks.bzl @@ -0,0 +1,31 @@ +"""Mocks for testing.""" + +def _rctx(os_name = "linux", os_arch = "x86_64", environ = None, **kwargs): + """Creates a mock of repository_ctx or module_ctx. + + Args: + os_name: The OS name to mock (e.g., "linux", "Mac OS X", "windows"). + os_arch: The OS architecture to mock (e.g., "x86_64", "aarch64"). + environ: A dictionary representing the environment variables. + **kwargs: Additional attributes to add to the mock struct. + + Returns: + A struct mocking repository_ctx. + """ + if environ == None: + environ = {} + + attrs = { + "getenv": environ.get, + "os": struct( + name = os_name, + arch = os_arch, + ), + } + attrs.update(kwargs) + + return struct(**attrs) + +mocks = struct( + rctx = _rctx, +) From 6d57b1aa189cff9845dd1904ace65207f58fae4d Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Sun, 12 Apr 2026 04:16:06 -0700 Subject: [PATCH 688/922] chore: add some skills and update agents doc (#3696) Some misc instructions for agents to follow. * Tell them about the multiple modules. I've found they can get confused trying to run code in different sub-modules. * Tell them to never run expunge. This hard-wipes caches and trashes build performance, not to mention takes many minutes itself * Tell them to use the fast-tests config, which captures more than just integration tests now. Add skills for interacting with build kite, which makes automating PRs a bit easier. --------- Co-authored-by: Ignas Anikevicius <240938+aignas@users.noreply.github.com> --- .agents/skills/buildkite-get-results/SKILL.md | 10 + .../scripts/get_buildkite_results.py | 203 ++++++++++++++++++ .agents/skills/buildkite-retry-job/SKILL.md | 15 ++ .../scripts/retry_buildkite_jobs.py | 91 ++++++++ AGENTS.md | 22 +- 5 files changed, 339 insertions(+), 2 deletions(-) create mode 100644 .agents/skills/buildkite-get-results/SKILL.md create mode 100755 .agents/skills/buildkite-get-results/scripts/get_buildkite_results.py create mode 100644 .agents/skills/buildkite-retry-job/SKILL.md create mode 100755 .agents/skills/buildkite-retry-job/scripts/retry_buildkite_jobs.py diff --git a/.agents/skills/buildkite-get-results/SKILL.md b/.agents/skills/buildkite-get-results/SKILL.md new file mode 100644 index 0000000000..a2a936513e --- /dev/null +++ b/.agents/skills/buildkite-get-results/SKILL.md @@ -0,0 +1,10 @@ +--- +name: buildkite-get-results +description: Gets buildkite build results +--- + +Pass the PR number to the `scripts/get_buildkite_results.py` script. + +The `--jobs` flag can do glob-style filtering of jobs. + +The `--download` flag will download job logs. diff --git a/.agents/skills/buildkite-get-results/scripts/get_buildkite_results.py b/.agents/skills/buildkite-get-results/scripts/get_buildkite_results.py new file mode 100755 index 0000000000..9df9a9d688 --- /dev/null +++ b/.agents/skills/buildkite-get-results/scripts/get_buildkite_results.py @@ -0,0 +1,203 @@ +#!/usr/bin/env python3 +import argparse +import json +import re +import subprocess +import sys +import urllib.request + + +def get_pr_checks(pr_number): + try: + # Check if gh is installed + subprocess.run( + ["gh", "--version"], + check=True, + stdout=subprocess.DEVNULL, + stderr=subprocess.DEVNULL, + ) + except FileNotFoundError: + print( + "Error: 'gh' (GitHub CLI) is not installed or not in PATH.", file=sys.stderr + ) + sys.exit(1) + except subprocess.CalledProcessError: + print("Error: 'gh' command failed. Is it installed?", file=sys.stderr) + sys.exit(1) + + cmd = ["gh", "pr", "checks", str(pr_number), "--json", "bucket,name,link,state"] + try: + result = subprocess.run(cmd, capture_output=True, text=True, check=True) + return json.loads(result.stdout) + except subprocess.CalledProcessError as e: + print(f"Error fetching PR checks: {e.stderr}", file=sys.stderr) + sys.exit(1) + + +def get_buildkite_build_url(checks): + for check in checks: + # Looking for Buildkite check. The name usually contains "buildkite" + if "buildkite" in check.get("name", "").lower(): + return check.get("link") + return None + + +def fetch_buildkite_data(build_url): + # Convert https://buildkite.com/org/pipeline/builds/number + # to https://buildkite.com/org/pipeline/builds/number.json + if not build_url.endswith(".json"): + json_url = build_url + ".json" + else: + json_url = build_url + + try: + with urllib.request.urlopen(json_url) as response: + if response.status != 200: + print( + f"Error fetching data from {json_url}: Status {response.status}", + file=sys.stderr, + ) + return None + return json.loads(response.read().decode()) + except Exception as e: + print(f"Error fetching data from {json_url}: {e}", file=sys.stderr) + return None + + +def download_log(job_url, output_path): + # Construct raw log URL: job_url + "/raw" (Buildkite convention) + # job_url e.g. https://buildkite.com/org/pipeline/builds/14394#job-id + # Wait, the job['path'] gives /org/pipeline/builds/14394#job-id + # We want /org/pipeline/builds/14394/jobs/job-id/raw? No + # The clean URL for a job is https://buildkite.com/org/pipeline/builds/14394/jobs/job-id + # And raw log is https://buildkite.com/org/pipeline/builds/14394/jobs/job-id/raw + + # We have full_url e.g. https://buildkite.com/bazel/rules-python-python/builds/14394#019c5cf9-e3cf-468f-a7b1-8f9f5ad4b08c + # We need to transform it. + + if "#" in job_url: + base, job_id = job_url.split("#") + # Ensure base doesn't end with / + if base.endswith("/"): + base = base[:-1] + + # Build raw URL + raw_url = f"{base}/jobs/{job_id}/raw" + else: + print(f"Could not parse job URL for download: {job_url}", file=sys.stderr) + return False + + try: + with urllib.request.urlopen(raw_url) as response: + if response.status != 200: + print( + f"Error downloading log from {raw_url}: Status {response.status}", + file=sys.stderr, + ) + return False + with open(output_path, "wb") as f: + f.write(response.read()) + return True + except Exception as e: + print(f"Error downloading log from {raw_url}: {e}", file=sys.stderr) + return False + + +def main(): + parser = argparse.ArgumentParser(description="Get Buildkite CI results for a PR.") + parser.add_argument("pr_number", help="The PR number.") + parser.add_argument( + "--jobs", + action="append", + help="Filter by job name (regex match). Can be specified multiple times.", + ) + parser.add_argument( + "--download", + action="store_true", + help="If exactly one job is matched, download its log to a local file.", + ) + + args = parser.parse_args() + + print(f"Fetching checks for PR #{args.pr_number}...", file=sys.stderr) + checks = get_pr_checks(args.pr_number) + + build_url = get_buildkite_build_url(checks) + if not build_url: + print("No Buildkite check found for this PR.", file=sys.stderr) + sys.exit(1) + + print(f"Found Buildkite URL: {build_url}", file=sys.stderr) + + data = fetch_buildkite_data(build_url) + if not data: + sys.exit(1) + + print(f"Build State: {data.get('state')}") + print("-" * 40) + + jobs = data.get("jobs", []) + + filtered_jobs = [] + if args.jobs: + for job in jobs: + job_name = job.get("name") + if not job_name: + continue + for pattern in args.jobs: + if re.search(pattern, job_name, re.IGNORECASE): + filtered_jobs.append(job) + break + else: + filtered_jobs = jobs + + for job in filtered_jobs: + name = job.get("name", "Unknown") + state = job.get("state", "Unknown") + path = job.get("path") + full_url = f"https://buildkite.com{path}" if path else "N/A" + + passed = job.get("passed", False) + outcome = job.get("outcome") + + if passed: + result_str = "PASSED" + elif outcome: + result_str = outcome.upper() + else: + result_str = state.upper() + + print(f"Job: {name}") + print(f" Result: {result_str}") + print(f" URL: {full_url}") + print("") + + if args.download: + if len(filtered_jobs) == 1: + job = filtered_jobs[0] + name = job.get("name", "unknown_job") + # Sanitize name for filename + safe_name = re.sub(r"[^a-zA-Z0-9_\-]", "_", name) + output_path = f"{safe_name}.log" + + path = job.get("path") + if path: + full_url = f"https://buildkite.com{path}" + print(f"Downloading log for '{name}'...", file=sys.stderr) + if download_log(full_url, output_path): + print(f"Downloaded log to: {output_path}") + else: + print("Failed to download log.", file=sys.stderr) + else: + print("Job has no URL path, cannot download.", file=sys.stderr) + elif len(filtered_jobs) == 0: + print("No jobs matched to download.", file=sys.stderr) + else: + print( + f"Matched {len(filtered_jobs)} jobs. Please filter to exactly one job to download.", + file=sys.stderr, + ) + + +if __name__ == "__main__": + main() diff --git a/.agents/skills/buildkite-retry-job/SKILL.md b/.agents/skills/buildkite-retry-job/SKILL.md new file mode 100644 index 0000000000..e8e3bcd491 --- /dev/null +++ b/.agents/skills/buildkite-retry-job/SKILL.md @@ -0,0 +1,15 @@ +--- +name: buildkite-retry-job +description: Retry a failed build kite job +--- + +Use `scripts/retry_buildkite_jobs.py` to retry a job. This is best used +when there are network failures. + +example: + +``` +retry_buildkite_jobs.py org pipeline build +``` + +The `--jobs` flag can be used to retry specific jobs. diff --git a/.agents/skills/buildkite-retry-job/scripts/retry_buildkite_jobs.py b/.agents/skills/buildkite-retry-job/scripts/retry_buildkite_jobs.py new file mode 100755 index 0000000000..67385fb8fd --- /dev/null +++ b/.agents/skills/buildkite-retry-job/scripts/retry_buildkite_jobs.py @@ -0,0 +1,91 @@ +#!/usr/bin/env python3 +import argparse +import json +import os +import sys +import urllib.request +from urllib.error import HTTPError + + +def make_request(url, method="GET", data=None, token=None): + headers = { + "Authorization": f"Bearer {token}", + "Accept": "application/json", + } + if data: + data = json.dumps(data).encode("utf-8") + headers["Content-Type"] = "application/json" + + req = urllib.request.Request(url, data=data, headers=headers, method=method) + try: + with urllib.request.urlopen(req) as response: + return json.loads(response.read().decode()) + except HTTPError as e: + print(f"HTTP Error: {e.code} - {e.reason}", file=sys.stderr) + if e.fp: + print(e.fp.read().decode(), file=sys.stderr) + return None + except Exception as e: + print(f"Error: {e}", file=sys.stderr) + return None + + +def main(): + parser = argparse.ArgumentParser( + description="Retry failed jobs in a Buildkite build." + ) + parser.add_argument("org", help="Organization slug") + parser.add_argument("pipeline", help="Pipeline slug") + parser.add_argument("build", help="Build number") + parser.add_argument( + "--job-name", + help="Specific job name to retry (if failed). Regex/substring allowed.", + ) + + args = parser.parse_args() + token = os.environ.get("BUILDKITE_API_TOKEN") + + if not token: + print( + "Please set the BUILDKITE_API_TOKEN environment variable.", file=sys.stderr + ) + sys.exit(1) + + url = f"https://api.buildkite.com/v2/organizations/{args.org}/pipelines/{args.pipeline}/builds/{args.build}" + print(f"Fetching build details from {url}...") + build_data = make_request(url, token=token) + + if not build_data: + print("Failed to fetch build details.", file=sys.stderr) + sys.exit(1) + + jobs = build_data.get("jobs", []) + failed_jobs = [j for j in jobs if j.get("state") == "failed"] + + if not failed_jobs: + print("No failed jobs found in this build.") + sys.exit(0) + + for job in failed_jobs: + job_id = job.get("id") + job_name = job.get("name", "Unknown") + + if ( + args.job_name + and args.job_name.lower() not in job_name.lower() + and args.job_name.lower() not in job.get("step_key", "").lower() + ): + continue + + print(f"Retrying job: {job_name} ({job_id})") + retry_url = f"https://api.buildkite.com/v2/organizations/{args.org}/pipelines/{args.pipeline}/builds/{args.build}/jobs/{job_id}/retry" + + result = make_request(retry_url, method="PUT", token=token) + if result: + print(f" Successfully triggered retry for {job_name}") + else: + print(f" Failed to trigger retry for {job_name}") + + +if __name__ == "__main__": + main() diff --git a/AGENTS.md b/AGENTS.md index c1c9f7902b..38ea8d1be2 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -7,6 +7,14 @@ Act as an expert in Bazel, rules_python, Starlark, and Python. DO NOT `git commit` or `git push`. +## RULES TO ALWAYS FOLLOW AND NEVER IGNORE + +ALWAYS FOLLOW THESE RULES. NEVER VIOLATE THEM. + +Ask for user input and provide a justificaiton if trying to violate them. + +* NEVER run `bazel clean --expunge`. + ## Style and conventions Read `.editorconfig` for line length wrapping @@ -121,12 +129,22 @@ bzl_library( Tests are under the `tests/` directory. -When testing, add `--test_tag_filters=-integration-test`. +When testing, add `--config=fast-tests`. -When building, add `--build_tag_filters=-integration-test`. +When building, add `--config=fast-tests`. + +The `--config=fast-tests` flag avoids running expensive and slow tests can that +freeze the host machine or cause flakiness. ## Understanding the code base +This repository contains 3 Bazel bzlmod modules. + + * `sphinxdocs/` is for the `@sphinxdocs` module. + * `gazelle/` is for the `@rules_python_gazelle_plugin` module. + * All other code is part of `@rules_python`. + + `python/config_settings/BUILD.bazel` contains build flags that are part of the public API. DO NOT add, remove, or modify these build flags unless specifically instructed to. From d7ae199b0505ef2432f404e13a88321f10e74dba Mon Sep 17 00:00:00 2001 From: Ignas Anikevicius <240938+aignas@users.noreply.github.com> Date: Mon, 13 Apr 2026 02:18:15 +0900 Subject: [PATCH 689/922] test(pypi): check index url precedence in various combinations (#3698) This just adds extra tests to verify behaviour end-to-end to better reason in case of any bug reports during the RC phase. Followup to #3691 --- tests/pypi/extension/pip_parse.bzl | 4 +- tests/pypi/hub_builder/hub_builder_tests.bzl | 133 +++++++++++++++++++ 2 files changed, 135 insertions(+), 2 deletions(-) diff --git a/tests/pypi/extension/pip_parse.bzl b/tests/pypi/extension/pip_parse.bzl index edac12e344..d6080e52d6 100644 --- a/tests/pypi/extension/pip_parse.bzl +++ b/tests/pypi/extension/pip_parse.bzl @@ -10,6 +10,7 @@ def pip_parse( enable_implicit_namespace_pkgs = False, environment = {}, envsubst = {}, + experimental_extra_index_urls = [], experimental_index_url = "", experimental_requirement_cycles = {}, experimental_target_platforms = [], @@ -40,6 +41,7 @@ def pip_parse( enable_implicit_namespace_pkgs = enable_implicit_namespace_pkgs, environment = environment, envsubst = envsubst, + experimental_extra_index_urls = experimental_extra_index_urls, experimental_index_url = experimental_index_url, experimental_requirement_cycles = experimental_requirement_cycles, # TODO @aignas 2025-12-02: decide on a single attr - should we reuse this? @@ -63,8 +65,6 @@ def pip_parse( requirements_windows = requirements_windows, timeout = timeout, whl_modifications = whl_modifications, - # The following are covered by other unit tests - experimental_extra_index_urls = [], parallel_download = False, experimental_index_url_overrides = {}, simpleapi_skip = simpleapi_skip, diff --git a/tests/pypi/hub_builder/hub_builder_tests.bzl b/tests/pypi/hub_builder/hub_builder_tests.bzl index 75a6edeae4..a9453abf80 100644 --- a/tests/pypi/hub_builder/hub_builder_tests.bzl +++ b/tests/pypi/hub_builder/hub_builder_tests.bzl @@ -698,6 +698,139 @@ torch==2.4.1+cpu ; platform_machine == 'x86_64' \ _tests.append(_test_torch_experimental_index_url) +def _test_index_url_precedence(env): + for test in [ + struct( + requirements_txt = "simple==0.0.1 --hash=sha256:deadb00f", + experimental_index_url = "https://experimental.example.com/simple", + experimental_extra_index_urls = [], + expect_index_url = "https://experimental.example.com/simple", + expect_extra_index_urls = [], + expect_url = "experimental.example.com/simple/", + ), + struct( + requirements_txt = """\ +--index-url=https://file.example.com/simple +simple==0.0.1 --hash=sha256:deadb00f +""", + experimental_index_url = "https://experimental.example.com/simple", + experimental_extra_index_urls = [], + expect_index_url = "https://file.example.com/simple", + expect_extra_index_urls = [], + expect_url = "file.example.com/simple/", + ), + struct( + requirements_txt = "simple==0.0.1 --hash=sha256:deadb00f", + experimental_index_url = "", + experimental_extra_index_urls = [], + expect_index_url = "https://pypi.org/simple", + expect_extra_index_urls = [], + expect_url = "pypi.org/simple/", + ), + struct( + requirements_txt = """\ +--extra-index-url=https://extra1.example.com/simple +--extra-index-url=https://extra2.example.com/simple +simple==0.0.1 --hash=sha256:deadb00f +""", + experimental_index_url = "", + experimental_extra_index_urls = [ + "https://ignored.example.com/simple", + ], + expect_index_url = "https://pypi.org/simple", + expect_extra_index_urls = [ + "https://extra1.example.com/simple", + "https://extra2.example.com/simple", + ], + expect_url = "pypi.org/simple/", + ), + ]: + got_kwargs = {} + + def mock_simpleapi_download(*_, **kwargs): + got_kwargs.update(kwargs) + return { + "simple": struct( + whls = { + "deadb00f": struct( + yanked = None, + filename = "simple-0.0.1-py3-none-any.whl", + sha256 = "deadb00f", + url = test.expect_url, + ), + }, + sdists = {}, + sha256s_by_version = {}, + index_url = test.expect_index_url, + ), + } + + builder = hub_builder( + env, + simpleapi_download_fn = mock_simpleapi_download, + ) + builder.pip_parse( + _mock_mctx( + read = lambda x: { + "requirements.txt": test.requirements_txt, + }[x], + ), + _parse( + hub_name = "pypi", + python_version = "3.15", + experimental_index_url = test.experimental_index_url, + experimental_extra_index_urls = test.experimental_extra_index_urls, + requirements_lock = "requirements.txt", + target_platforms = [ + "linux_x86_64", + "osx_aarch64", + ], + ), + ) + pypi = builder.build() + + pypi.exposed_packages().contains_exactly(["simple"]) + pypi.whl_map().contains_exactly({ + "simple": { + "pypi_315_simple_py3_none_any_deadb00f": [ + whl_config_setting( + target_platforms = ("cp315_linux_x86_64", "cp315_osx_aarch64"), + version = "3.15", + ), + ], + }, + }) + pypi.whl_libraries().contains_exactly({ + "pypi_315_simple_py3_none_any_deadb00f": { + "config_load": "@pypi//:config.bzl", + "dep_template": "@pypi//{name}:{target}", + "filename": "simple-0.0.1-py3-none-any.whl", + "index_url": test.expect_index_url, + "requirement": "simple==0.0.1", + "sha256": "deadb00f", + "urls": [test.expect_url], + }, + }) + pypi.extra_aliases().contains_exactly({}) + + env.expect.that_dict(got_kwargs).contains_exactly({ + "attr": struct( + auth_patterns = {}, + envsubst = {}, + extra_index_urls = test.expect_extra_index_urls, + index_url = test.expect_index_url, + index_url_overrides = {}, + netrc = None, + sources = { + "simple": ["0.0.1"], + }, + ), + "cache": {}, + "parallel_download": False, + }) + +_tests.append(_test_index_url_precedence) + def _test_download_only_multiple(env): builder = hub_builder(env) builder.pip_parse( From fa783beb4ca4042fa4d49f8930e45d2691c100ec Mon Sep 17 00:00:00 2001 From: Ignas Anikevicius <240938+aignas@users.noreply.github.com> Date: Mon, 13 Apr 2026 05:26:23 +0900 Subject: [PATCH 690/922] test(pypi): add argparse.bzl tests (#3697) This simply adds extra unit tests to ensure that the code I have added in #3691 is well covered. --- tests/pypi/argparse/BUILD.bazel | 3 ++ tests/pypi/argparse/argparse_tests.bzl | 48 ++++++++++++++++++++++++++ 2 files changed, 51 insertions(+) create mode 100644 tests/pypi/argparse/BUILD.bazel create mode 100644 tests/pypi/argparse/argparse_tests.bzl diff --git a/tests/pypi/argparse/BUILD.bazel b/tests/pypi/argparse/BUILD.bazel new file mode 100644 index 0000000000..b04da685e6 --- /dev/null +++ b/tests/pypi/argparse/BUILD.bazel @@ -0,0 +1,3 @@ +load(":argparse_tests.bzl", "argparse_test_suite") + +argparse_test_suite(name = "argparse_tests") diff --git a/tests/pypi/argparse/argparse_tests.bzl b/tests/pypi/argparse/argparse_tests.bzl new file mode 100644 index 0000000000..f8fd1c9481 --- /dev/null +++ b/tests/pypi/argparse/argparse_tests.bzl @@ -0,0 +1,48 @@ +"" + +load("@rules_testing//lib:test_suite.bzl", "test_suite") +load("//python/private/pypi:argparse.bzl", "argparse") # buildifier: disable=bzl-visibility + +_tests = [] + +def _test_index_url(env): + env.expect.that_str(argparse.index_url([], "default")).equals("default") + env.expect.that_str(argparse.index_url([], None)).equals(None) + + env.expect.that_str(argparse.index_url(["-i", "https://example.com/simple"], "default")).equals("https://example.com/simple") + env.expect.that_str(argparse.index_url(["--index-url", "https://example.com/simple"], "default")).equals("https://example.com/simple") + env.expect.that_str(argparse.index_url(["--index-url=https://example.com/simple"], "default")).equals("https://example.com/simple") + + env.expect.that_str(argparse.index_url(["--extra-index-url", "https://extra.com", "-i", "https://index.com"], "default")).equals("https://index.com") + +_tests.append(_test_index_url) + +def _test_extra_index_url(env): + env.expect.that_collection(argparse.extra_index_url([], ["default"])).contains_exactly(["default"]) + env.expect.that_collection(argparse.extra_index_url([], None)).contains_exactly([]) + + env.expect.that_collection(argparse.extra_index_url(["--extra-index-url", "https://extra.com/simple"], [])).contains_exactly(["https://extra.com/simple"]) + env.expect.that_collection(argparse.extra_index_url(["--extra-index-url=https://extra.com/simple"], [])).contains_exactly(["https://extra.com/simple"]) + + env.expect.that_collection(argparse.extra_index_url(["--extra-index-url", "https://first.com", "--extra-index-url", "https://second.com"], [])).contains_exactly(["https://first.com", "https://second.com"]) + +_tests.append(_test_extra_index_url) + +def _test_platform(env): + env.expect.that_collection(argparse.platform([], ["default"])).contains_exactly(["default"]) + env.expect.that_collection(argparse.platform([], None)).contains_exactly([]) + + env.expect.that_collection(argparse.platform(["--platform", "manylinux_2_17_x86_64"], [])).contains_exactly(["manylinux_2_17_x86_64"]) + env.expect.that_collection(argparse.platform(["--platform=manylinux_2_17_x86_64"], [])).contains_exactly(["manylinux_2_17_x86_64"]) + + env.expect.that_collection(argparse.platform(["--platform", "macosx_10_9_x86_64", "--platform", "linux_x86_64"], [])).contains_exactly(["macosx_10_9_x86_64", "linux_x86_64"]) + +_tests.append(_test_platform) + +def argparse_test_suite(name): + """Create the test suite. + + Args: + name: the name of the test suite + """ + test_suite(name = name, basic_tests = _tests) From 17d7732e12af4816b66140bc1ec77da3106ef521 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Sun, 12 Apr 2026 21:02:34 -0700 Subject: [PATCH 691/922] chore: make buildkite-get-results skill work with large job numbers (#3699) Apparently, buildkite returns incomplete results it there are a large number of jobs. Vibed out changes to handle such cases. --- .../scripts/get_buildkite_results.py | 49 +++++++++++++++++-- 1 file changed, 44 insertions(+), 5 deletions(-) diff --git a/.agents/skills/buildkite-get-results/scripts/get_buildkite_results.py b/.agents/skills/buildkite-get-results/scripts/get_buildkite_results.py index 9df9a9d688..eb5532e4d6 100755 --- a/.agents/skills/buildkite-get-results/scripts/get_buildkite_results.py +++ b/.agents/skills/buildkite-get-results/scripts/get_buildkite_results.py @@ -58,11 +58,37 @@ def fetch_buildkite_data(build_url): file=sys.stderr, ) return None - return json.loads(response.read().decode()) + data = json.loads(response.read().decode()) except Exception as e: print(f"Error fetching data from {json_url}: {e}", file=sys.stderr) return None + # If jobs list is truncated or empty but statistics says there are more jobs, + # try to fetch from /data/jobs.json + jobs = data.get("jobs", []) + jobs_count = data.get("statistics", {}).get("jobs_count", 0) + + if len(jobs) < jobs_count: + # Try fetching from /data/jobs.json + # Build URL might have .json already from the check above + base_url = build_url + if base_url.endswith(".json"): + base_url = base_url[:-5] + + jobs_url = f"{base_url}/data/jobs.json" + try: + with urllib.request.urlopen(jobs_url) as response: + if response.status == 200: + jobs_data = json.loads(response.read().decode()) + if isinstance(jobs_data, list): + data["jobs"] = jobs_data + elif isinstance(jobs_data, dict) and "records" in jobs_data: + data["jobs"] = jobs_data["records"] + except Exception as e: + print(f"Warning: Could not fetch detailed jobs from {jobs_url}: {e}", file=sys.stderr) + + return data + def download_log(job_url, output_path): # Construct raw log URL: job_url + "/raw" (Buildkite convention) @@ -119,7 +145,11 @@ def main(): args = parser.parse_args() - print(f"Fetching checks for PR #{args.pr_number}...", file=sys.stderr) + pr_display = args.pr_number + if "pull/" in pr_display: + pr_display = pr_display.split("pull/")[1].split("#")[0].split("/")[0] + + print(f"Fetching checks for PR #{pr_display}...", file=sys.stderr) checks = get_pr_checks(args.pr_number) build_url = get_buildkite_build_url(checks) @@ -133,10 +163,19 @@ def main(): if not data: sys.exit(1) - print(f"Build State: {data.get('state')}") - print("-" * 40) - + build_state = data.get("state", "Unknown") + print(f"Build State: {build_state}") + jobs = data.get("jobs", []) + jobs_count = data.get("statistics", {}).get("jobs_count", 0) + + print(f"Total jobs reported: {jobs_count}") + print(f"Jobs found in data: {len(jobs)}") + + if jobs_count != len(jobs): + print(f"WARNING: Reported job count ({jobs_count}) does not match jobs found ({len(jobs)}).", file=sys.stderr) + + print("-" * 40) filtered_jobs = [] if args.jobs: From 85e4a3c943106dd0d6bea79ff98d656cc1861b80 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Sun, 12 Apr 2026 21:43:34 -0700 Subject: [PATCH 692/922] chore: factor mock mctx/rctx functionality into separate file (#3700) This factors the various mocking logic spread throughout the code base into a single mocks module. This should make it easier to re-use and add unit tests of other functionality. --- tests/pypi/extension/extension_tests.bzl | 91 ++- tests/pypi/hub_builder/hub_builder_tests.bzl | 106 ++-- .../parse_requirements_tests.bzl | 11 +- tests/pypi/pypi_cache/pypi_cache_tests.bzl | 5 +- .../simpleapi_download_tests.bzl | 11 +- .../whl_library_targets_tests.bzl | 71 +-- tests/python/python_tests.bzl | 93 ++- tests/support/mocks/BUILD.bazel | 13 + tests/support/mocks/mocks.bzl | 572 ++++++++++++++++++ tests/support/mocks/mocks_tests.bzl | 139 +++++ tests/uv/uv/uv_tests.bzl | 63 +- 11 files changed, 914 insertions(+), 261 deletions(-) create mode 100644 tests/support/mocks/BUILD.bazel create mode 100644 tests/support/mocks/mocks.bzl create mode 100644 tests/support/mocks/mocks_tests.bzl diff --git a/tests/pypi/extension/extension_tests.bzl b/tests/pypi/extension/extension_tests.bzl index 7999e42a7b..452703d9c9 100644 --- a/tests/pypi/extension/extension_tests.bzl +++ b/tests/pypi/extension/extension_tests.bzl @@ -18,66 +18,53 @@ load("@rules_testing//lib:test_suite.bzl", "test_suite") load("@rules_testing//lib:truth.bzl", "subjects") load("//python/private/pypi:extension.bzl", "build_config", "parse_modules") # buildifier: disable=bzl-visibility load("//python/private/pypi:whl_config_setting.bzl", "whl_config_setting") # buildifier: disable=bzl-visibility +load("//tests/support/mocks:mocks.bzl", "mocks") load(":pip_parse.bzl", _parse = "pip_parse") _tests = [] -def _mock_mctx(*modules, os_name = "unittest", arch_name = "exotic", environ = {}, read = None): - return struct( - getenv = environ.get, - os = struct( - name = os_name, - arch = arch_name, - ), - read = read or (lambda _: """\ +def _pypi_mock_mctx(*modules, os_name = "unittest", arch_name = "exotic", environ = {}, read = None): + _ = read # @unused + return mocks.mctx( + modules = list(modules), + os_name = os_name, + arch_name = arch_name, + environ = environ, + mock_files = { + "requirements.txt": """\ simple==0.0.1 \ --hash=sha256:deadbeef \ - --hash=sha256:deadbaaf"""), - modules = [ - struct( - name = modules[0].name, - tags = modules[0].tags, - is_root = modules[0].is_root, - ), - ] + [ - struct( - name = mod.name, - tags = mod.tags, - is_root = False, - ) - for mod in modules[1:] - ], + --hash=sha256:deadbaaf""", + }, ) def _mod(*, name, default = [], parse = [], override = [], whl_mods = [], is_root = True): - return struct( - name = name, - tags = struct( - parse = parse, - override = override, - whl_mods = whl_mods, - default = default or [ - _default( - platform = "{}_{}{}".format(os, cpu, freethreaded), - os_name = os, - arch_name = cpu, - config_settings = [ - "@platforms//os:{}".format(os), - "@platforms//cpu:{}".format(cpu), - ], - whl_abi_tags = ["cp{major}{minor}t"] if freethreaded else ["abi3", "cp{major}{minor}"], - whl_platform_tags = whl_platform_tags, - ) - for (os, cpu, freethreaded), whl_platform_tags in { - ("linux", "x86_64", ""): ["linux_x86_64", "manylinux_*_x86_64"], - ("linux", "x86_64", "_freethreaded"): ["linux_x86_64", "manylinux_*_x86_64"], - ("linux", "aarch64", ""): ["linux_aarch64", "manylinux_*_aarch64"], - ("osx", "aarch64", ""): ["macosx_*_arm64"], - ("windows", "aarch64", ""): ["win_arm64"], - }.items() - ], - ), + return mocks.module( + name, is_root = is_root, + parse = parse, + override = override, + whl_mods = whl_mods, + default = default or [ + _default( + platform = "{}_{}{}".format(os, cpu, freethreaded), + os_name = os, + arch_name = cpu, + config_settings = [ + "@platforms//os:{}".format(os), + "@platforms//cpu:{}".format(cpu), + ], + whl_abi_tags = ["cp{major}{minor}t"] if freethreaded else ["abi3", "cp{major}{minor}"], + whl_platform_tags = whl_platform_tags, + ) + for (os, cpu, freethreaded), whl_platform_tags in { + ("linux", "x86_64", ""): ["linux_x86_64", "manylinux_*_x86_64"], + ("linux", "x86_64", "_freethreaded"): ["linux_x86_64", "manylinux_*_x86_64"], + ("linux", "aarch64", ""): ["linux_aarch64", "manylinux_*_aarch64"], + ("osx", "aarch64", ""): ["macosx_*_arm64"], + ("windows", "aarch64", ""): ["win_arm64"], + }.items() + ], ) def _parse_modules(env, enable_pipstar = 0, **kwargs): @@ -140,7 +127,7 @@ def _default( def _test_simple(env): pypi = _parse_modules( env, - module_ctx = _mock_mctx( + module_ctx = _pypi_mock_mctx( _mod( name = "rules_python", parse = [ @@ -187,7 +174,7 @@ _tests.append(_test_simple) def _test_build_pipstar_platform(env): config = _build_config( env, - module_ctx = _mock_mctx( + module_ctx = _pypi_mock_mctx( _mod( name = "rules_python", default = [ diff --git a/tests/pypi/hub_builder/hub_builder_tests.bzl b/tests/pypi/hub_builder/hub_builder_tests.bzl index a9453abf80..29021dfd85 100644 --- a/tests/pypi/hub_builder/hub_builder_tests.bzl +++ b/tests/pypi/hub_builder/hub_builder_tests.bzl @@ -23,21 +23,21 @@ load("//python/private/pypi:platform.bzl", _plat = "platform") # buildifier: di load("//python/private/pypi:simpleapi_download.bzl", "simpleapi_download") # buildifier: disable=bzl-visibility load("//python/private/pypi:whl_config_setting.bzl", "whl_config_setting") # buildifier: disable=bzl-visibility load("//tests/pypi/extension:pip_parse.bzl", _parse = "pip_parse") +load("//tests/support/mocks:mocks.bzl", "mocks") _tests = [] -def _mock_mctx(os_name = "unittest", arch_name = "exotic", environ = {}, read = None): - return struct( - getenv = environ.get, - os = struct( - name = os_name, - arch = arch_name, - ), - read = read or (lambda _: """\ +def _mock_mctx(os_name = "unittest", arch_name = "exotic", environ = {}, mock_files = None): + return mocks.mctx( + os_name = os_name, + arch_name = arch_name, + environ = environ, + mock_files = mock_files or { + "requirements.txt": """\ simple==0.0.1 \ --hash=sha256:deadbeef \ - --hash=sha256:deadbaaf"""), - report_progress = lambda _: None, + --hash=sha256:deadbaaf""", + }, ) def hub_builder( @@ -163,11 +163,11 @@ def _test_simple_multiple_requirements(env): for (host_os, host_arch), want_requirement in sub_tests.items(): builder = hub_builder(env) builder.pip_parse( - _mock_mctx( - read = lambda x: { + mocks.mctx( + mock_files = { "darwin.txt": "simple==0.0.2 --hash=sha256:deadb00f", "win.txt": "simple==0.0.1 --hash=sha256:deadbeef", - }[x], + }, os_name = host_os, arch_name = host_arch, ), @@ -209,11 +209,11 @@ def _test_simple_extras_vs_no_extras(env): for (host_os, host_arch), want_requirement in sub_tests.items(): builder = hub_builder(env) builder.pip_parse( - _mock_mctx( - read = lambda x: { + mocks.mctx( + mock_files = { "darwin.txt": "simple[foo]==0.0.1 --hash=sha256:deadbeef", "win.txt": "simple==0.0.1 --hash=sha256:deadbeef", - }[x], + }, os_name = host_os, arch_name = host_arch, ), @@ -274,11 +274,11 @@ def _test_simple_extras_vs_no_extras_simpleapi(env): ), ) builder.pip_parse( - _mock_mctx( - read = lambda x: { + mocks.mctx( + mock_files = { "darwin.txt": "simple[foo]==0.0.1 --hash=sha256:deadbeef", "win.txt": "simple==0.0.1 --hash=sha256:deadbeef", - }[x], + }, ), _parse( hub_name = "pypi", @@ -350,13 +350,13 @@ def _test_simple_multiple_python_versions(env): }, ) builder.pip_parse( - _mock_mctx( - read = lambda x: { + mocks.mctx( + mock_files = { "requirements_3_15.txt": """ simple==0.0.1 --hash=sha256:deadbeef old-package==0.0.1 --hash=sha256:deadbaaf """, - }[x], + }, os_name = "linux", arch_name = "amd64", ), @@ -367,13 +367,13 @@ old-package==0.0.1 --hash=sha256:deadbaaf ), ) builder.pip_parse( - _mock_mctx( - read = lambda x: { + mocks.mctx( + mock_files = { "requirements_3_16.txt": """ simple==0.0.2 --hash=sha256:deadb00f new-package==0.0.1 --hash=sha256:deadb00f2 """, - }[x], + }, os_name = "linux", arch_name = "amd64", ), @@ -455,14 +455,14 @@ def _test_simple_with_markers(env): }, ) builder.pip_parse( - _mock_mctx( - read = lambda x: { + mocks.mctx( + mock_files = { "universal.txt": """\ torch==2.4.1+cpu ; platform_machine == 'x86_64' torch==2.4.1 ; platform_machine != 'x86_64' \ --hash=sha256:deadbeef """, - }[x], + }, os_name = host_os, arch_name = host_arch, ), @@ -577,8 +577,8 @@ def _test_torch_experimental_index_url(env): ), ) builder.pip_parse( - _mock_mctx( - read = lambda x: { + mocks.mctx( + mock_files = { "universal.txt": """\ torch==2.4.1 ; platform_machine != 'x86_64' \ --hash=sha256:1495132f30f722af1a091950088baea383fe39903db06b20e6936fd99402803e \ @@ -605,7 +605,7 @@ torch==2.4.1+cpu ; platform_machine == 'x86_64' \ --hash=sha256:c4f2c3c026e876d4dad7629170ec14fff48c076d6c2ae0e354ab3fdc09024f00 # via -r requirements.in """, - }[x], + }, ), _parse( hub_name = "pypi", @@ -771,9 +771,9 @@ simple==0.0.1 --hash=sha256:deadb00f ) builder.pip_parse( _mock_mctx( - read = lambda x: { + mock_files = { "requirements.txt": test.requirements_txt, - }[x], + }, ), _parse( hub_name = "pypi", @@ -834,8 +834,8 @@ _tests.append(_test_index_url_precedence) def _test_download_only_multiple(env): builder = hub_builder(env) builder.pip_parse( - _mock_mctx( - read = lambda x: { + mocks.mctx( + mock_files = { "requirements.linux_x86_64.txt": """\ --platform=manylinux_2_17_x86_64 --python-version=315 @@ -856,7 +856,7 @@ extra==0.0.1 \ simple==0.0.3 \ --hash=sha256:deadbaaf """, - }[x], + }, ), _parse( hub_name = "pypi", @@ -997,8 +997,8 @@ def _test_simple_get_index(env): }, ) builder.pip_parse( - _mock_mctx( - read = lambda x: { + mocks.mctx( + mock_files = { "requirements.txt": """ simple==0.0.1 \ --hash=sha256:deadbeef \ @@ -1012,7 +1012,7 @@ pip_fallback==0.0.1 direct_sdist_without_sha @ some-archive/any-name.tar.gz git_dep @ git+https://git.server/repo/project@deadbeefdeadbeef """, - }[x], + }, ), _parse( hub_name = "pypi", @@ -1245,13 +1245,13 @@ def _test_optimum_sys_platform_extra(env): env, ) builder.pip_parse( - _mock_mctx( - read = lambda x: { + mocks.mctx( + mock_files = { "universal.txt": """\ optimum[onnxruntime]==1.17.1 ; sys_platform == 'darwin' optimum[onnxruntime-gpu]==1.17.1 ; sys_platform == 'linux' """, - }[x], + }, os_name = host_os, arch_name = host_arch, ), @@ -1313,13 +1313,13 @@ def _test_pipstar_platforms(env): ), ) builder.pip_parse( - _mock_mctx( - read = lambda x: { + mocks.mctx( + mock_files = { "universal.txt": """\ optimum[onnxruntime]==1.17.1 ; sys_platform == 'darwin' optimum[onnxruntime-gpu]==1.17.1 ; sys_platform == 'linux' """, - }[x], + }, ), _parse( hub_name = "pypi", @@ -1399,15 +1399,15 @@ def _test_pipstar_platforms_limit(env): ), ) builder.pip_parse( - _mock_mctx( + mocks.mctx( os_name = "linux", arch_name = "amd64", - read = lambda x: { + mock_files = { "universal.txt": """\ optimum[onnxruntime]==1.17.1 ; sys_platform == 'darwin' optimum[onnxruntime-gpu]==1.17.1 ; sys_platform == 'linux' """, - }[x], + }, ), _parse( hub_name = "pypi", @@ -1454,6 +1454,9 @@ def _test_err_duplicate_repos(env): _mock_mctx( os_name = "osx", arch_name = "aarch64", + mock_files = { + "requirements.txt": "foo==0.0.1", + }, ), _parse( hub_name = "pypi", @@ -1465,6 +1468,9 @@ def _test_err_duplicate_repos(env): _mock_mctx( os_name = "osx", arch_name = "aarch64", + mock_files = { + "requirements.txt": "foo==0.0.1", + }, ), _parse( hub_name = "pypi", @@ -1482,11 +1488,11 @@ def _test_err_duplicate_repos(env): env.expect.that_dict(logs).keys().contains_exactly(["rules_python:unit-test FAIL:"]) env.expect.that_collection(logs["rules_python:unit-test FAIL:"]).contains_exactly([ """\ -Attempting to create a duplicate library pypi_315_simple for simple with different arguments. Already existing declaration has: +Attempting to create a duplicate library pypi_315_foo for foo with different arguments. Already existing declaration has: common: { "dep_template": "@pypi//{name}:{target}", "config_load": "@pypi//:config.bzl", - "requirement": "simple==0.0.1 --hash=sha256:deadbeef --hash=sha256:deadbaaf", + "requirement": "foo==0.0.1", } different: { "python_interpreter_target": ("unit_test_interpreter_target_1", "unit_test_interpreter_target_2"), diff --git a/tests/pypi/parse_requirements/parse_requirements_tests.bzl b/tests/pypi/parse_requirements/parse_requirements_tests.bzl index 053ab31ec1..1786c4e664 100644 --- a/tests/pypi/parse_requirements/parse_requirements_tests.bzl +++ b/tests/pypi/parse_requirements/parse_requirements_tests.bzl @@ -19,6 +19,7 @@ load("//python/private:repo_utils.bzl", "REPO_DEBUG_ENV_VAR", "REPO_VERBOSITY_EN load("//python/private/pypi:evaluate_markers.bzl", "evaluate_markers") # buildifier: disable=bzl-visibility load("//python/private/pypi:parse_requirements.bzl", "select_requirement", _parse_requirements = "parse_requirements") # buildifier: disable=bzl-visibility load("//python/private/pypi:pep508_env.bzl", pep508_env = "env") # buildifier: disable=bzl-visibility +load("//tests/support/mocks:mocks.bzl", "mocks") def _mock_ctx(): testdata = { @@ -97,12 +98,10 @@ bar==0.0.1 --hash=sha256:deadb00f """, } - return struct( - os = struct( - name = "linux", - arch = "x86_64", - ), - read = lambda x: testdata[x], + return mocks.mctx( + os_name = "linux", + arch_name = "x86_64", + mock_files = testdata, ) _tests = [] diff --git a/tests/pypi/pypi_cache/pypi_cache_tests.bzl b/tests/pypi/pypi_cache/pypi_cache_tests.bzl index 3cf01c7450..59ef661ab7 100644 --- a/tests/pypi/pypi_cache/pypi_cache_tests.bzl +++ b/tests/pypi/pypi_cache/pypi_cache_tests.bzl @@ -3,6 +3,7 @@ load("@rules_testing//lib:test_suite.bzl", "test_suite") load("@rules_testing//lib:truth.bzl", "subjects") load("//python/private/pypi:pypi_cache.bzl", "pypi_cache") # buildifier: disable=bzl-visibility +load("//tests/support/mocks:mocks.bzl", "mocks") _tests = [] @@ -85,7 +86,7 @@ _tests.append(_test_memory_cache_hit) def _test_pypi_cache_writes_to_facts(env): """Verifies that setting a value in the cache also populates the facts store.""" - mock_ctx = struct(facts = {}) + mock_ctx = mocks.mctx(facts = {}) cache = _cache(env, mctx = mock_ctx) fake_result = struct( @@ -192,7 +193,7 @@ _tests.append(_test_pypi_cache_writes_to_facts) def _test_pypi_cache_reads_from_facts(env): """Verifies that setting a value in the cache also populates the facts store.""" - mock_ctx = struct(facts = { + mock_ctx = mocks.mctx(facts = { "dist_hashes": { # We are not using the real index URL, because we may have credentials in here "https://{PYPI_INDEX_URL}": { diff --git a/tests/pypi/simpleapi_download/simpleapi_download_tests.bzl b/tests/pypi/simpleapi_download/simpleapi_download_tests.bzl index 55439c2593..7d7cbee0ca 100644 --- a/tests/pypi/simpleapi_download/simpleapi_download_tests.bzl +++ b/tests/pypi/simpleapi_download/simpleapi_download_tests.bzl @@ -17,6 +17,7 @@ load("@rules_testing//lib:test_suite.bzl", "test_suite") load("//python/private/pypi:pypi_cache.bzl", "pypi_cache") # buildifier: disable=bzl-visibility load("//python/private/pypi:simpleapi_download.bzl", "simpleapi_download") # buildifier: disable=bzl-visibility +load("//tests/support/mocks:mocks.bzl", "mocks") _tests = [] @@ -48,10 +49,7 @@ def _test_simple(env): ) contents = simpleapi_download( - ctx = struct( - getenv = {}.get, - report_progress = lambda _: None, - ), + ctx = mocks.mctx(), attr = struct( index_url_overrides = {}, index_url = "https://main.com", @@ -123,10 +121,7 @@ def _test_index_overrides(env): ) contents = simpleapi_download( - ctx = struct( - getenv = {}.get, - report_progress = lambda _: None, - ), + ctx = mocks.mctx(), attr = struct( index_url_overrides = { "foo": "https://extra.com", diff --git a/tests/pypi/whl_library_targets/whl_library_targets_tests.bzl b/tests/pypi/whl_library_targets/whl_library_targets_tests.bzl index 9b574039b2..08715fbf77 100644 --- a/tests/pypi/whl_library_targets/whl_library_targets_tests.bzl +++ b/tests/pypi/whl_library_targets/whl_library_targets_tests.bzl @@ -20,6 +20,7 @@ load( "whl_library_targets", "whl_library_targets_from_requires", ) # buildifier: disable=bzl-visibility +load("//tests/support/mocks:mocks.bzl", "mocks") _tests = [] @@ -191,11 +192,11 @@ def _test_whl_and_library_deps_from_requires(env): py_library_calls = [] env_marker_setting_calls = [] - mock_glob = _mock_glob() + m_glob = mocks.glob() - mock_glob.results.append(["site-packages/foo/SRCS.py"]) - mock_glob.results.append(["site-packages/foo/DATA.txt"]) - mock_glob.results.append(["site-packages/foo/PYI.pyi"]) + m_glob.results.append(["site-packages/foo/SRCS.py"]) + m_glob.results.append(["site-packages/foo/DATA.txt"]) + m_glob.results.append(["site-packages/foo/PYI.pyi"]) whl_library_targets_from_requires( name = "foo-0-py3-none-any.whl", @@ -215,7 +216,7 @@ def _test_whl_and_library_deps_from_requires(env): native = struct( filegroup = lambda **kwargs: filegroup_calls.append(kwargs), config_setting = lambda **_: None, - glob = mock_glob.glob, + glob = m_glob.glob, ), rules = struct( py_library = lambda **kwargs: py_library_calls.append(kwargs), @@ -263,15 +264,15 @@ def _test_whl_and_library_deps_from_requires(env): }), }) # buildifier: @unsorted-dict-items - env.expect.that_collection(mock_glob.calls).contains_exactly([ + env.expect.that_collection(m_glob.calls).contains_exactly([ # srcs call - _glob_call( + mocks.glob_call( ["site-packages/**/*.py"], exclude = [], allow_empty = True, ), # data call - _glob_call( + mocks.glob_call( ["site-packages/**/*"], exclude = [ "**/*.py", @@ -282,7 +283,7 @@ def _test_whl_and_library_deps_from_requires(env): allow_empty = True, ), # pyi call - _glob_call(["site-packages/**/*.pyi"], allow_empty = True), + mocks.glob_call(["site-packages/**/*.pyi"], allow_empty = True), ]) env.expect.that_collection(env_marker_setting_calls).contains_exactly([ @@ -298,10 +299,10 @@ _tests.append(_test_whl_and_library_deps_from_requires) def _test_whl_and_library_deps(env): filegroup_calls = [] py_library_calls = [] - mock_glob = _mock_glob() - mock_glob.results.append(["site-packages/foo/SRCS.py"]) - mock_glob.results.append(["site-packages/foo/DATA.txt"]) - mock_glob.results.append(["site-packages/foo/PYI.pyi"]) + m_glob = mocks.glob() + m_glob.results.append(["site-packages/foo/SRCS.py"]) + m_glob.results.append(["site-packages/foo/DATA.txt"]) + m_glob.results.append(["site-packages/foo/PYI.pyi"]) whl_library_targets( name = "foo.whl", @@ -323,7 +324,7 @@ def _test_whl_and_library_deps(env): native = struct( filegroup = lambda **kwargs: filegroup_calls.append(kwargs), config_setting = lambda **_: None, - glob = mock_glob.glob, + glob = m_glob.glob, ), rules = struct( py_library = lambda **kwargs: py_library_calls.append(kwargs), @@ -396,10 +397,10 @@ def _test_group(env): alias_calls = [] py_library_calls = [] - mock_glob = _mock_glob() - mock_glob.results.append(["site-packages/foo/srcs.py"]) - mock_glob.results.append(["site-packages/foo/data.txt"]) - mock_glob.results.append(["site-packages/foo/pyi.pyi"]) + m_glob = mocks.glob() + m_glob.results.append(["site-packages/foo/srcs.py"]) + m_glob.results.append(["site-packages/foo/data.txt"]) + m_glob.results.append(["site-packages/foo/pyi.pyi"]) whl_library_targets( name = "foo.whl", @@ -419,7 +420,7 @@ def _test_group(env): filegroups = {}, native = struct( config_setting = lambda **_: None, - glob = mock_glob.glob, + glob = m_glob.glob, alias = lambda **kwargs: alias_calls.append(kwargs), ), rules = struct( @@ -463,43 +464,19 @@ def _test_group(env): }), }) # buildifier: @unsorted-dict-items - env.expect.that_collection(mock_glob.calls, expr = "glob calls").contains_exactly([ - _glob_call(["site-packages/**/*.py"], exclude = [], allow_empty = True), - _glob_call(["site-packages/**/*"], exclude = [ + env.expect.that_collection(m_glob.calls, expr = "glob calls").contains_exactly([ + mocks.glob_call(["site-packages/**/*.py"], exclude = [], allow_empty = True), + mocks.glob_call(["site-packages/**/*"], exclude = [ "**/*.py", "**/*.pyc", "**/*.pyc.*", "**/*.dist-info/RECORD", ], allow_empty = True), - _glob_call(["site-packages/**/*.pyi"], allow_empty = True), + mocks.glob_call(["site-packages/**/*.pyi"], allow_empty = True), ]) _tests.append(_test_group) -def _glob_call(*args, **kwargs): - return struct( - glob = args, - kwargs = kwargs, - ) - -def _mock_glob(): - # buildifier: disable=uninitialized - def glob(*args, **kwargs): - mock.calls.append(_glob_call(*args, **kwargs)) - if not mock.results: - fail("Mock glob missing for invocation: args={} kwargs={}".format( - args, - kwargs, - )) - return mock.results.pop(0) - - mock = struct( - calls = [], - results = [], - glob = glob, - ) - return mock - def whl_library_targets_test_suite(name): """create the test suite. diff --git a/tests/python/python_tests.bzl b/tests/python/python_tests.bzl index d8b14ba784..db54e6bab5 100644 --- a/tests/python/python_tests.bzl +++ b/tests/python/python_tests.bzl @@ -18,42 +18,19 @@ load("@pythons_hub//:versions.bzl", "MINOR_MAPPING") load("@rules_testing//lib:test_suite.bzl", "test_suite") load("//python/private:python.bzl", "parse_modules") # buildifier: disable=bzl-visibility load("//python/private:repo_utils.bzl", "repo_utils") # buildifier: disable=bzl-visibility +load("//tests/support/mocks:mocks.bzl", "mocks") _tests = [] -def _mock_mctx(*modules, environ = {}, mocked_files = {}): - return struct( - path = lambda x: struct(exists = x in mocked_files, _file = x), - read = lambda x, watch = None: mocked_files[x._file if "_file" in dir(x) else x], - getenv = environ.get, - modules = [ - struct( - name = modules[0].name, - tags = modules[0].tags, - is_root = modules[0].is_root, - ), - ] + [ - struct( - name = mod.name, - tags = mod.tags, - is_root = False, - ) - for mod in modules[1:] - ], - ) - -# todo: change is_root to false by default. most modules aren't root def _mod(*, name, defaults = [], toolchain = [], override = [], single_version_override = [], single_version_platform_override = [], is_root = False): - return struct( - name = name, - tags = struct( - defaults = defaults, - toolchain = toolchain, - override = override, - single_version_override = single_version_override, - single_version_platform_override = single_version_platform_override, - ), + return mocks.module( + name, is_root = is_root, + defaults = defaults, + toolchain = toolchain, + override = override, + single_version_override = single_version_override, + single_version_platform_override = single_version_platform_override, ) def _defaults(python_version = None, python_version_env = None, python_version_file = None): @@ -148,7 +125,7 @@ def _single_version_platform_override( def _test_default_from_rules_python_when_rules_python_is_root(env): """Verify that rules_python (as root module) default is applied.""" py = parse_modules( - module_ctx = _mock_mctx( + module_ctx = mocks.mctx( _rules_python_module(is_root = True), ), logger = repo_utils.logger(verbosity_level = 0, name = "python"), @@ -178,7 +155,7 @@ _tests.append(_test_default_from_rules_python_when_rules_python_is_root) def _test_default_from_rules_python_when_rules_python_is_not_root(env): """Verify that rules_python default applies when rules_python is not the root module.""" py = parse_modules( - module_ctx = _mock_mctx( + module_ctx = mocks.mctx( _rules_python_module(), ), logger = repo_utils.logger(verbosity_level = 0, name = "python"), @@ -197,9 +174,11 @@ _tests.append(_test_default_from_rules_python_when_rules_python_is_not_root) def _test_default_with_patch_version(env): py = parse_modules( - module_ctx = _mock_mctx( - _mod(name = "alpha", toolchain = [_toolchain("3.11.2")], is_root = True), - _rules_python_module(is_root = True), + module_ctx = mocks.mctx( + modules = [ + _mod(name = "alpha", toolchain = [_toolchain("3.11.2")], is_root = True), + _rules_python_module(is_root = False), + ], ), logger = repo_utils.logger(verbosity_level = 0, name = "python"), ) @@ -217,7 +196,7 @@ _tests.append(_test_default_with_patch_version) def _test_toolchain_ordering(env): py = parse_modules( - module_ctx = _mock_mctx( + module_ctx = mocks.mctx( _mod( name = "my_module", toolchain = [ @@ -264,7 +243,7 @@ _tests.append(_test_toolchain_ordering) def _test_default_from_defaults(env): py = parse_modules( - module_ctx = _mock_mctx( + module_ctx = mocks.mctx( _mod( name = "my_root_module", defaults = [_defaults(python_version = "3.11")], @@ -291,7 +270,7 @@ _tests.append(_test_default_from_defaults) def _test_default_from_defaults_env(env): py = parse_modules( - module_ctx = _mock_mctx( + module_ctx = mocks.mctx( _mod( name = "my_root_module", defaults = [_defaults(python_version = "3.11", python_version_env = "PYENV_VERSION")], @@ -319,14 +298,14 @@ _tests.append(_test_default_from_defaults_env) def _test_default_from_defaults_file(env): py = parse_modules( - module_ctx = _mock_mctx( + module_ctx = mocks.mctx( _mod( name = "my_root_module", defaults = [_defaults(python_version_file = "@@//:.python-version")], toolchain = [_toolchain("3.10"), _toolchain("3.11"), _toolchain("3.12")], is_root = True, ), - mocked_files = {"@@//:.python-version": "3.12\n"}, + mock_files = {"@@//:.python-version": "3.12\n"}, ), logger = repo_utils.logger(verbosity_level = 0, name = "python"), ) @@ -347,7 +326,7 @@ _tests.append(_test_default_from_defaults_file) def _test_default_from_single_toolchain(env): py = parse_modules( - module_ctx = _mock_mctx( + module_ctx = mocks.mctx( _mod( name = "my_root_module", toolchain = [_toolchain("3.12")], @@ -363,7 +342,7 @@ _tests.append(_test_default_from_single_toolchain) def _test_defaults_overrides_single_toolchain(env): py = parse_modules( - module_ctx = _mock_mctx( + module_ctx = mocks.mctx( _mod( name = "my_root_module", defaults = [ @@ -383,7 +362,7 @@ _tests.append(_test_defaults_overrides_single_toolchain) def _test_defaults_overrides_toolchains_setting_is_default(env): py = parse_modules( - module_ctx = _mock_mctx( + module_ctx = mocks.mctx( _mod( name = "my_root_module", defaults = [_defaults(python_version = "3.13")], @@ -403,10 +382,12 @@ _tests.append(_test_defaults_overrides_toolchains_setting_is_default) def _test_first_occurance_of_the_toolchain_wins(env): py = parse_modules( - module_ctx = _mock_mctx( - _mod(name = "my_module", is_root = True, toolchain = [_toolchain("3.12")]), - _mod(name = "some_module", toolchain = [_toolchain("3.12", configure_coverage_tool = True)]), - _rules_python_module(), + module_ctx = mocks.mctx( + modules = [ + _mod(name = "my_module", is_root = True, toolchain = [_toolchain("3.12")]), + _mod(name = "some_module", toolchain = [_toolchain("3.12", configure_coverage_tool = True)]), + _rules_python_module(), + ], environ = { "RULES_PYTHON_BZLMOD_DEBUG": "1", }, @@ -444,7 +425,7 @@ _tests.append(_test_first_occurance_of_the_toolchain_wins) def _test_auth_overrides(env): py = parse_modules( - module_ctx = _mock_mctx( + module_ctx = mocks.mctx( _mod( name = "my_module", toolchain = [_toolchain("3.12")], @@ -486,7 +467,7 @@ _tests.append(_test_auth_overrides) def _test_add_new_version(env): py = parse_modules( - module_ctx = _mock_mctx( + module_ctx = mocks.mctx( _mod( name = "my_module", is_root = True, @@ -567,7 +548,7 @@ _tests.append(_test_add_new_version) def _test_register_all_versions(env): py = parse_modules( - module_ctx = _mock_mctx( + module_ctx = mocks.mctx( _mod( name = "my_module", is_root = True, @@ -633,7 +614,7 @@ _tests.append(_test_register_all_versions) def _test_ignore_unsupported_versions(env): py = parse_modules( - module_ctx = _mock_mctx( + module_ctx = mocks.mctx( _mod( name = "my_module", is_root = True, @@ -704,7 +685,7 @@ _tests.append(_test_ignore_unsupported_versions) def _test_add_patches(env): py = parse_modules( - module_ctx = _mock_mctx( + module_ctx = mocks.mctx( _mod( name = "my_module", is_root = True, @@ -783,7 +764,7 @@ _tests.append(_test_add_patches) def _test_fail_two_overrides(env): errors = [] parse_modules( - module_ctx = _mock_mctx( + module_ctx = mocks.mctx( _mod( name = "my_module", is_root = True, @@ -815,7 +796,7 @@ def _test_single_version_override_errors(env): ]: errors = [] parse_modules( - module_ctx = _mock_mctx( + module_ctx = mocks.mctx( _mod( name = "my_module", is_root = True, @@ -854,7 +835,7 @@ def _test_single_version_platform_override_errors(env): ]: errors = [] parse_modules( - module_ctx = _mock_mctx( + module_ctx = mocks.mctx( _mod( name = "my_module", toolchain = [_toolchain("3.13")], diff --git a/tests/support/mocks/BUILD.bazel b/tests/support/mocks/BUILD.bazel new file mode 100644 index 0000000000..45949fc578 --- /dev/null +++ b/tests/support/mocks/BUILD.bazel @@ -0,0 +1,13 @@ +load("@bazel_skylib//:bzl_library.bzl", "bzl_library") +load(":mocks_tests.bzl", "mocks_test_suite") + +package( + default_visibility = ["//:__subpackages__"], +) + +bzl_library( + name = "mocks_bzl", + srcs = ["mocks.bzl"], +) + +mocks_test_suite(name = "mocks_tests") diff --git a/tests/support/mocks/mocks.bzl b/tests/support/mocks/mocks.bzl new file mode 100644 index 0000000000..48f8f95830 --- /dev/null +++ b/tests/support/mocks/mocks.bzl @@ -0,0 +1,572 @@ +"""Mocks for repository_ctx, module_ctx, and File objects.""" + +def _path_new(path, mock_files = None): + """Create a mock path object. + + Args: + path: {type}`string` The path string. + mock_files: {type}`dict[string, string]` A dict of mocked files. + + Returns: + {type}`MockPath` A struct mocking a path object. + """ + mock_files = mock_files or {} + return struct( + exists = path in mock_files, + basename = path.split("/")[-1], + dirname = "/".join(path.split("/")[:-1]), + _path = path, + ) + +def _file_new(short_path, *, path = None, is_source = True, owner = None): + """Create a mock File object. + + Args: + short_path: {type}`string` The short path to the file. + path: {type}`string` The full path to the file. Defaults to a made + up exec-root path or the short path if is_source. + is_source: {type}`bool` Whether the file is a source file. + owner: {type}`Label|string` The owner label of the file. + + Returns: + {type}`MockFile` A struct mocking a File object. + """ + if owner == None: + owner = Label("//:mock") + + owner_str = str(owner) + repo_name = owner_str.split("//")[0] + + is_main_repo = repo_name in ("", "@", "@@") + + actual_short_path = short_path + if not is_main_repo: + repo_name = repo_name.lstrip("@") + if not actual_short_path.startswith("../"): + actual_short_path = "../{}/{}".format(repo_name, short_path) + + if path == None: + rel_path = short_path + if rel_path.startswith("../"): + parts = rel_path.split("/") + rel_path = "/".join(parts[2:]) + + if is_source: + path = "external/{}/{}".format(repo_name, rel_path) + else: + path = "bazel-out/k9-deadbeef/bin/external/{}/{}".format( + repo_name, + rel_path, + ) + elif path == None: + if is_source: + path = short_path + else: + path = "bazel-out/k9-deadbeef/bin/{}".format(short_path) + + return struct( + path = path, + basename = path.split("/")[-1], + dirname = "/".join(path.split("/")[:-1]), + extension = path.split(".")[-1] if "." in path else "", + is_source = is_source, + owner = owner, + short_path = actual_short_path, + ) + +def _tag_new(**kwargs): + """Create a mock tag. + + Args: + **kwargs: {type}`dict` The tag attributes. + + Returns: + {type}`MockTag` A mock tag object. + """ + return struct(**kwargs) + +def _module_new(name, *, is_root = False, **tags): + """Create a mock module object. + + Args: + name: {type}`string` The name of the module. + is_root: {type}`bool` Whether this is the root module. + **tags: {type}`list[MockTag]` Lists of tag objects. + + Returns: + {type}`MockModule` A mock module object. + """ + return struct( + name = name, + tags = struct(**tags), + is_root = is_root, + ) + +def _mctx_read(self, x, watch = None): + _ = watch # @unused + path_str = x._path if hasattr(x, "_path") else str(x) + if path_str not in self.mock_files: + fail("File not found in mock_files: " + path_str) + return self.mock_files[path_str] + +def _mctx_path(self, x): + return _path_new(str(x), self.mock_files) + +def _get_download_file_name(url, output = ""): + """Compute the download file name. + + Args: + url: {type}`string` The URL being downloaded. + output: {type}`string` The explicit output path, if any. + + Returns: + {type}`string` The file name. + """ + if output: + return str(output) + return str(url).split("?")[0].split("/")[-1] + +def _mctx_download( + self, + url, + output = "", + sha256 = "", + executable = False, + allow_fail = False, + canonical_id = "", + auth = {}, + headers = {}, + integrity = "", + block = True): + _ = ( + sha256, + executable, + allow_fail, + canonical_id, + auth, + headers, + integrity, + block, + ) # @unused + urls = url if type(url) == "list" else [url] + for u in urls: + content = None + if u in self.mock_downloads: + content = self.mock_downloads[u] + elif "*" in self.mock_downloads: + content = self.mock_downloads["*"] + + if content != None: + if type(content) == "string": + out = _get_download_file_name(u, output) + self.mock_files[out] = content + return struct( + success = True, + wait = lambda: struct(success = True), + ) + else: + return content( + self, + u, + output, + sha256, + executable, + allow_fail, + canonical_id, + auth, + headers, + integrity, + ) + + if not self.mock_downloads: + return struct(success = True, wait = lambda: struct(success = True)) + return struct(success = False, wait = lambda: struct(success = False)) + +def _mctx_report_progress(self, message): + self.report_progress_calls.append(message) + return None + +def _mctx_add_module(self, **kwargs): + """Add a module to the mock module_ctx. + + Args: + self: The mock module_ctx. + **kwargs: Arguments to pass to _module_new. + + Returns: + {type}`MockModuleCtx` The mock module_ctx. + """ + module = _module_new(**kwargs) + if module.is_root and len(self.modules) > 0: + fail("is_root=True can only be set on the first module in the " + + "modules list.") + self.modules.append(module) + return self + +def _mctx_new( + *args, + modules = None, + environ = None, + mock_files = None, + mock_downloads = None, + os_name = "linux", + arch_name = "x86_64", + facts = None): + """Create a mock module_ctx object. + + Args: + *args: {type}`list[MockModule]` Mock modules passed positionally. + modules: {type}`list[MockModule]` List of mock modules (alternative + to positional args). + environ: {type}`dict[string, string]` Dict of environment variables. + mock_files: {type}`dict[string, string]` Dict mapping path strings + to content. + mock_downloads: {type}`dict[string, string|callable]` Dict mapping + url to string or callable. + os_name: {type}`string` The OS name. + arch_name: {type}`string` The architecture name. + facts: {type}`dict` Optional facts dict. + + Returns: + {type}`MockModuleCtx` A struct mocking a module_ctx object. + """ + modules = list(args) + (modules or []) + + for i, mod in enumerate(modules): + if getattr(mod, "is_root", False) and i != 0: + fail("is_root=True can only be set on the first module in the " + + "modules list.") + + environ = environ or {} + mock_files = mock_files or {} + mock_downloads = mock_downloads or {} + + # buildifier: disable=uninitialized + self = struct( + mock_files = mock_files, + mock_downloads = mock_downloads, + report_progress_calls = [], + getenv = environ.get, + facts = facts, + os = struct( + name = os_name, + arch = arch_name, + ), + modules = list(modules), + path = lambda *a, **k: _mctx_path(self, *a, **k), + read = lambda *a, **k: _mctx_read(self, *a, **k), + download = lambda *a, **k: _mctx_download(self, *a, **k), + report_progress = lambda *a, **k: _mctx_report_progress(self, *a, **k), + add_module = lambda **k: _mctx_add_module(self, **k), + ) + return self + +def _rctx_read(self, x): + path_str = x._path if hasattr(x, "_path") else str(x) + if path_str not in self.mock_files: + fail("File not found in mock_files: " + path_str) + + val = self.mock_files[path_str] + for _ in range(10): + if type(val) == "dict" and val.get("type") == "symlink": + path_str = val["target"] + if path_str not in self.mock_files: + fail("Symlink target not found in mock_files: " + path_str) + val = self.mock_files[path_str] + else: + break + + if type(val) == "dict" and val.get("type") == "symlink": + fail("Too many symlinks followed") + + return val + +def _rctx_path(self, x): + return _path_new(str(x), self.mock_files) + +def _rctx_file(self, path, content = "", executable = True, legacy_utf8 = True): + _ = executable, legacy_utf8 # @unused + self.mock_files[str(path)] = content + +def _rctx_template(self, path, template, substitutions = {}, executable = True): + _ = executable # @unused + template_str = str(template) + if template_str not in self.mock_files: + fail("Template file not found: " + template_str) + + content = self.mock_files[template_str] + for key, value in substitutions.items(): + content = content.replace(key, value) + + self.mock_files[str(path)] = content + +def _rctx_which(self, program): + prog_str = str(program) + if prog_str in self.mock_which: + res = self.mock_which[prog_str] + if res == None: + return None + return _path_new(res, self.mock_files) + return None + +def _rctx_download( + self, + url, + output = "", + sha256 = "", + executable = False, + allow_fail = False, + canonical_id = "", + auth = {}, + headers = {}, + integrity = ""): + _ = ( + sha256, + executable, + allow_fail, + canonical_id, + auth, + headers, + integrity, + ) # @unused + + urls = url if type(url) == "list" else [url] + + for u in urls: + if u in self.mock_downloads: + res = self.mock_downloads[u] + if type(res) == "string": + out = _get_download_file_name(u, output) + self.mock_files[out] = res + return struct(success = True, sha256 = "mocksha256") + else: + return res( + self, + u, + output, + sha256, + executable, + allow_fail, + canonical_id, + auth, + headers, + integrity, + ) + + if not allow_fail: + fail("Download not mocked for url: " + str(urls)) + return struct(success = False) + +def _rctx_extract( + self, + archive, + output = "", + stripPrefix = "", + rename_files = {}, + *, + watch_archive = "auto"): + _ = ( + stripPrefix, + rename_files, + watch_archive, + ) # @unused + + archive_str = str(archive) + if archive_str in self.mock_extracts: + for f, c in self.mock_extracts[archive_str].items(): + out_path = "{}/{}".format(output, f) if output else str(f) + self.mock_files[out_path] = c + +def _rctx_download_and_extract( + self, + url, + output = "", + sha256 = "", + type = "", + stripPrefix = "", + allow_fail = False, + canonical_id = "", + auth = {}, + headers = {}, + integrity = "", + rename_files = {}): + _ = type # @unused + + res = self.download( + url = url, + output = "", + sha256 = sha256, + allow_fail = allow_fail, + canonical_id = canonical_id, + auth = auth, + headers = headers, + integrity = integrity, + ) + if not res.success: + return res + + urls = url if type(url) == "list" else [url] + for u in urls: + downloaded_file = _get_download_file_name(u) + if downloaded_file in self.mock_extracts: + self.extract( + archive = downloaded_file, + output = output, + stripPrefix = stripPrefix, + rename_files = rename_files, + ) + break + + return res + +def _rctx_execute( + self, + arguments, + timeout = 600, + quiet = True, + working_directory = "", + environment = {}, + custom_reporter = ""): + _ = ( + self, + arguments, + timeout, + quiet, + working_directory, + environment, + custom_reporter, + ) # @unused + return struct(return_code = 0, stdout = "", stderr = "") + +def _rctx_symlink(self, target, link_name): + self.mock_files[str(link_name)] = {"target": str(target), "type": "symlink"} + +def _rctx_new( + attr = None, + environ = None, + mock_files = None, + mock_which = None, + mock_downloads = None, + mock_extracts = None, + os_name = "linux", + arch_name = "x86_64"): + """Create a mock repository_ctx object. + + Args: + attr: {type}`dict` Dict of attributes. + environ: {type}`dict[string, string]` Dict of environment variables. + mock_files: {type}`dict[string, string]` Dict mapping path strings + to content. + mock_which: {type}`dict[string, string]` Dict mapping program + name to path string. + mock_downloads: {type}`dict[string, string|callable]` Dict mapping + url to string or callable. + mock_extracts: {type}`dict[string, dict[string, string]]` Dict mapping + downloaded filename to a dict of extracted filename to content. + os_name: {type}`string` The OS name. + arch_name: {type}`string` The architecture name. + + Returns: + {type}`MockRepositoryCtx` A struct mocking a repository_ctx object. + """ + attr = attr or {} + environ = environ or {} + mock_files = mock_files or {} + mock_which = mock_which or {} + mock_downloads = mock_downloads or {} + mock_extracts = mock_extracts or {} + + # buildifier: disable=uninitialized + self = struct( + mock_files = mock_files, + mock_which = mock_which, + mock_downloads = mock_downloads, + mock_extracts = mock_extracts, + attr = struct(**attr), + os = struct( + name = os_name, + arch = arch_name, + ), + os_environ = environ, + path = lambda *a, **k: _rctx_path(self, *a, **k), + read = lambda *a, **k: _rctx_read(self, *a, **k), + file = lambda *a, **k: _rctx_file(self, *a, **k), + template = lambda *a, **k: _rctx_template(self, *a, **k), + which = lambda *a, **k: _rctx_which(self, *a, **k), + download = lambda *a, **k: _rctx_download(self, *a, **k), + extract = lambda *a, **k: _rctx_extract(self, *a, **k), + download_and_extract = lambda *a, **k: _rctx_download_and_extract( + self, + *a, + **k + ), + execute = lambda *a, **k: _rctx_execute(self, *a, **k), + symlink = lambda *a, **k: _rctx_symlink(self, *a, **k), + ) + + return self + +def _glob_call_new(*args, **kwargs): + """Create a struct representing a glob call. + + Args: + *args: {type}`tuple` Positional arguments to glob. + **kwargs: {type}`dict` Keyword arguments to glob. + + Returns: + {type}`MockGlobCall` A struct with glob and kwargs fields. + """ + return struct( + glob = args, + kwargs = kwargs, + ) + +def _glob_new(): + """Create a mock glob object. + + Returns: + {type}`MockGlob` A struct with calls and results lists, and a + glob function. + """ + calls = [] + results = [] + + def _glob_fn(*args, **kwargs): + calls.append(_glob_call_new(*args, **kwargs)) + if not results: + fail("Mock glob missing for invocation: args={} kwargs={}".format( + args, + kwargs, + )) + return results.pop(0) + + return struct( + calls = calls, + results = results, + glob = _glob_fn, + ) + +def _select_new(value, no_match_error = None): + """A mock select function that returns the value. + + Args: + value: {type}`Any` The value to return. + no_match_error: {type}`string` Ignored. + + Returns: + {type}`MockSelect` The value. + """ + _ = no_match_error # @unused + return value + +mocks = struct( + file = _file_new, + glob = _glob_new, + glob_call = _glob_call_new, + mctx = _mctx_new, + module = _module_new, + path = _path_new, + rctx = _rctx_new, + select = _select_new, + tag = _tag_new, +) diff --git a/tests/support/mocks/mocks_tests.bzl b/tests/support/mocks/mocks_tests.bzl new file mode 100644 index 0000000000..1dc495b342 --- /dev/null +++ b/tests/support/mocks/mocks_tests.bzl @@ -0,0 +1,139 @@ +"""Tests for mocks.bzl""" + +load("@rules_testing//lib:test_suite.bzl", "test_suite") +load("//tests/support/mocks:mocks.bzl", "mocks") + +_tests = [] + +def _test_path(env): + p1 = mocks.path("a/b/c", mock_files = {"a/b/c": "data"}) + env.expect.that_bool(p1.exists).equals(True) + env.expect.that_str(p1.basename).equals("c") + env.expect.that_str(p1.dirname).equals("a/b") + env.expect.that_str(p1._path).equals("a/b/c") + + p2 = mocks.path("d/e/f", mock_files = {}) + env.expect.that_bool(p2.exists).equals(False) + +_tests.append(_test_path) + +def _test_file(env): + # Default main repo + f1 = mocks.file("a/b.txt", is_source = True) + env.expect.that_str(f1.path).equals("a/b.txt") + env.expect.that_str(f1.short_path).equals("a/b.txt") + env.expect.that_str(f1.basename).equals("b.txt") + env.expect.that_str(f1.dirname).equals("a") + env.expect.that_str(f1.extension).equals("txt") + env.expect.that_bool(f1.is_source).equals(True) + env.expect.that_str(str(f1.owner)).equals(str(Label("//:mock"))) + + # External repo + f2 = mocks.file("a/b.txt", is_source = True, owner = "@foo//:mock") + env.expect.that_str(f2.path).equals("external/foo/a/b.txt") + env.expect.that_str(f2.short_path).equals("../foo/a/b.txt") + + # External repo generated file + f3 = mocks.file("a/b.txt", is_source = False, owner = "@foo//:mock") + env.expect.that_str(f3.path).equals( + "bazel-out/k9-deadbeef/bin/external/foo/a/b.txt", + ) + env.expect.that_str(f3.short_path).equals("../foo/a/b.txt") + +_tests.append(_test_file) + +def _test_mctx(env): + mctx = mocks.mctx( + environ = {"FOO": "bar"}, + mock_files = {"file.txt": "content"}, + mock_downloads = {"http://example.com": "downloaded"}, + os_name = "windows", + arch_name = "x86_64", + ) + env.expect.that_str(mctx.getenv("FOO")).equals("bar") + env.expect.that_str(mctx.read(mocks.path("file.txt"))).equals("content") + env.expect.that_str(mctx.os.name).equals("windows") + env.expect.that_str(mctx.os.arch).equals("x86_64") + + # Test download + res = mctx.download("http://example.com", "out.txt") + env.expect.that_bool(res.success).equals(True) + env.expect.that_str(mctx.read(mocks.path("out.txt"))).equals("downloaded") + + # Test report progress + mctx.report_progress("doing something") + env.expect.that_collection(mctx.report_progress_calls).contains_exactly([ + "doing something", + ]) + +_tests.append(_test_mctx) + +def _test_rctx(env): + rctx = mocks.rctx( + environ = {"FOO": "bar"}, + mock_files = {"file.txt": "content"}, + mock_which = {"mycmd": "path/to/mycmd"}, + ) + env.expect.that_str(rctx.os_environ["FOO"]).equals("bar") + env.expect.that_str(rctx.read(mocks.path("file.txt"))).equals("content") + + # Test which + w = rctx.which("mycmd") + env.expect.that_str(w._path).equals("path/to/mycmd") + env.expect.that_bool(rctx.which("not_found") == None).equals(True) + + # Test file writing + rctx.file("new.txt", "new content") + env.expect.that_str(rctx.read(mocks.path("new.txt"))).equals("new content") + + # Test template + rctx.file("template.txt", "Hello {name}") + rctx.template("rendered.txt", "template.txt", {"{name}": "World"}) + env.expect.that_str(rctx.read(mocks.path("rendered.txt"))).equals( + "Hello World", + ) + + # Test symlink reading + rctx.symlink("rendered.txt", "link.txt") + env.expect.that_str(rctx.read(mocks.path("link.txt"))).equals("Hello World") + +_tests.append(_test_rctx) + +def _test_glob(env): + g = mocks.glob() + g.results.append(["a.txt", "b.txt"]) + res = g.glob(["*.txt"], exclude = ["c.txt"]) + env.expect.that_collection(res).contains_exactly(["a.txt", "b.txt"]) + env.expect.that_collection(g.calls).has_size(1) + env.expect.that_collection(g.calls[0].glob[0]).contains_exactly(["*.txt"]) + env.expect.that_collection(g.calls[0].kwargs["exclude"]).contains_exactly([ + "c.txt", + ]) + +_tests.append(_test_glob) + +def _test_module_and_tags(env): + mod = mocks.module( + "my_mod", + is_root = True, + my_tag = [mocks.tag(attr1 = "val1")], + ) + env.expect.that_str(mod.name).equals("my_mod") + env.expect.that_bool(mod.is_root).equals(True) + env.expect.that_str(mod.tags.my_tag[0].attr1).equals("val1") + +_tests.append(_test_module_and_tags) + +def _test_select(env): + res = mocks.select({"//conditions:default": "val"}) + env.expect.that_dict(res).contains_exactly({"//conditions:default": "val"}) + +_tests.append(_test_select) + +def mocks_test_suite(name): + """Create the test suite. + + Args: + name: the name of the test suite + """ + test_suite(name = name, basic_tests = _tests) diff --git a/tests/uv/uv/uv_tests.bzl b/tests/uv/uv/uv_tests.bzl index d82e5cb385..cb60cf48ec 100644 --- a/tests/uv/uv/uv_tests.bzl +++ b/tests/uv/uv/uv_tests.bzl @@ -21,11 +21,12 @@ load("//python/private:common_labels.bzl", "labels") # buildifier: disable=bzl- load("//python/uv:uv_toolchain_info.bzl", "UvToolchainInfo") load("//python/uv/private:uv.bzl", "process_modules") # buildifier: disable=bzl-visibility load("//python/uv/private:uv_toolchain.bzl", "uv_toolchain") # buildifier: disable=bzl-visibility +load("//tests/support/mocks:mocks.bzl", "mocks") load("//tests/support/platforms:platforms.bzl", "platform_targets") _tests = [] -def _mock_mctx(*modules, download = None, read = None): +def _uv_mock_mctx(*modules, download = None): # Here we construct a fake minimal manifest file that we use to mock what would # be otherwise read from GH files manifest_files = { @@ -66,39 +67,20 @@ def _mock_mctx(*modules, download = None, read = None): for fname, contents in manifest_files.items() } - return struct( - path = str, - download = download or (lambda *_, **__: struct( - success = True, - wait = lambda: struct( - success = True, - ), - )), - read = read or (lambda x: fake_fs[x]), - modules = [ - struct( - name = modules[0].name, - tags = modules[0].tags, - is_root = modules[0].is_root, - ), - ] + [ - struct( - name = mod.name, - tags = mod.tags, - is_root = False, - ) - for mod in modules[1:] - ], + return mocks.mctx( + modules = list(modules), + mock_downloads = { + "*": download, + } if download else {}, + mock_files = fake_fs, ) def _mod(*, name = None, default = [], configure = [], is_root = True): - return struct( - name = name, # module_name - tags = struct( - default = default, - configure = configure, - ), + return mocks.module( + name, is_root = is_root, + default = default, + configure = configure, ) def _process_modules(env, **kwargs): @@ -148,7 +130,7 @@ def _configure(urls = None, sha256 = None, **kwargs): def _test_only_defaults(env): uv = _process_modules( env, - module_ctx = _mock_mctx( + module_ctx = _uv_mock_mctx( _mod( default = [ _default( @@ -181,7 +163,7 @@ def _test_manual_url_spec(env): calls = [] uv = _process_modules( env, - module_ctx = _mock_mctx( + module_ctx = _uv_mock_mctx( _mod( default = [ _default( @@ -202,12 +184,11 @@ def _test_manual_url_spec(env): configure = [ _configure( platform = "linux", - urls = ["https://example.org/download.zip"], + urls = ["https://example.org/1.0.0/manifest.json.zip"], sha256 = "deadbeef", ), ], ), - read = lambda *args, **kwargs: fail(args, kwargs), ), uv_repository = lambda **kwargs: calls.append(kwargs), ) @@ -227,7 +208,7 @@ def _test_manual_url_spec(env): "name": "uv_1_0_0_linux", "platform": "linux", "sha256": "deadbeef", - "urls": ["https://example.org/download.zip"], + "urls": ["https://example.org/1.0.0/manifest.json.zip"], "version": "1.0.0", }, ]) @@ -238,7 +219,7 @@ def _test_defaults(env): calls = [] uv = _process_modules( env, - module_ctx = _mock_mctx( + module_ctx = _uv_mock_mctx( _mod( default = [ _default( @@ -286,7 +267,7 @@ def _test_default_building(env): calls = [] uv = _process_modules( env, - module_ctx = _mock_mctx( + module_ctx = _uv_mock_mctx( _mod( default = [ _default( @@ -350,7 +331,7 @@ def _test_complex_configuring(env): calls = [] uv = _process_modules( env, - module_ctx = _mock_mctx( + module_ctx = _uv_mock_mctx( _mod( default = [ _default( @@ -462,7 +443,7 @@ def _test_non_rules_python_non_root_is_ignored(env): calls = [] uv = _process_modules( env, - module_ctx = _mock_mctx( + module_ctx = _uv_mock_mctx( _mod( default = [ _default( @@ -482,6 +463,7 @@ def _test_non_rules_python_non_root_is_ignored(env): configure = [ _configure(version = "6.6.6"), # use defaults whatever they are ], + is_root = False, ), ), uv_repository = lambda **kwargs: calls.append(kwargs), @@ -513,7 +495,7 @@ def _test_rules_python_does_not_take_precedence(env): calls = [] uv = _process_modules( env, - module_ctx = _mock_mctx( + module_ctx = _uv_mock_mctx( _mod( default = [ _default( @@ -538,6 +520,7 @@ def _test_rules_python_does_not_take_precedence(env): compatible_with = ["@platforms//os:osx"], ), ], + is_root = False, ), ), uv_repository = lambda **kwargs: calls.append(kwargs), From 9a6be4ca9a86d9c3f9ab2ae3a3b8499c81f5f6dd Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Sun, 12 Apr 2026 21:44:57 -0700 Subject: [PATCH 693/922] refactor: split create_providers into separate functions (#3695) The create_providers function was becoming unwieldy because it was taking all the args for all the providers. Split it up into separate functions to make it easier to see what args are going to what providers and follow the provider creation logic. --- python/private/py_executable.bzl | 285 +++++++++++++++++++------------ 1 file changed, 174 insertions(+), 111 deletions(-) diff --git a/python/private/py_executable.bzl b/python/private/py_executable.bzl index 6c65bf8f59..1cb7c9593c 100644 --- a/python/private/py_executable.bzl +++ b/python/private/py_executable.bzl @@ -1252,29 +1252,45 @@ def py_executable_base_impl(ctx, *, semantics, is_test, inherited_environment = app_runfiles = exec_result.app_runfiles - return _create_providers( - ctx = ctx, + providers = [] + + _add_provider_default_info( + providers, + ctx, executable = executable, + default_outputs = default_outputs.build(), runfiles_details = runfiles_details, + ) + _add_provider_instrumented_files_info(providers, ctx) + _add_provider_run_environment_info(providers, ctx, inherited_environment) + _add_provider_py_executable_info( + providers, + app_runfiles = app_runfiles, + build_data_file = runfiles_details.build_data_file, + interpreter_args = ctx.attr.interpreter_args, + interpreter_path = runtime_details.executable_interpreter_path, main_py = main_py, - imports = imports, + runfiles_without_exe = runfiles_details.runfiles_without_exe, + stage2_bootstrap = exec_result.stage2_bootstrap, + venv_interpreter_runfiles = exec_result.venv_interpreter_runfiles, + venv_interpreter_symlinks = exec_result.venv_interpreter_symlinks, + venv_python_exe = exec_result.venv_python_exe, + ) + _add_provider_py_runtime_info(providers, runtime_details) + _add_provider_py_cc_link_params_info(providers, cc_details.cc_info_for_propagating) + py_info = _add_provider_py_info( + providers, + ctx = ctx, original_sources = direct_sources, required_py_files = required_py_files, required_pyc_files = required_pyc_files, implicit_pyc_files = implicit_pyc_files, implicit_pyc_source_files = implicit_pyc_source_files, - default_outputs = default_outputs.build(), - runtime_details = runtime_details, - cc_info = cc_details.cc_info_for_propagating, - inherited_environment = inherited_environment, - output_groups = exec_result.output_groups, - stage2_bootstrap = exec_result.stage2_bootstrap, - app_runfiles = app_runfiles, - venv_python_exe = exec_result.venv_python_exe, - venv_interpreter_runfiles = exec_result.venv_interpreter_runfiles, - venv_interpreter_symlinks = exec_result.venv_interpreter_symlinks, - interpreter_args = ctx.attr.interpreter_args, + imports = imports, ) + _add_provider_output_group_info(providers, py_info, exec_result.output_groups) + + return providers def _get_build_info(ctx, cc_toolchain): build_info_files = py_internal.cc_toolchain_build_info_files(cc_toolchain) @@ -1799,97 +1815,111 @@ def _is_tool_config(ctx): # a more public API. Until that's available, py_internal to the rescue. return py_internal.is_tool_configuration(ctx) -def _create_providers( - *, - ctx, - executable, - main_py, - original_sources, - required_py_files, - required_pyc_files, - implicit_pyc_files, - implicit_pyc_source_files, - default_outputs, - runfiles_details, - imports, - cc_info, - inherited_environment, - runtime_details, - output_groups, - stage2_bootstrap, - app_runfiles, - venv_python_exe, - venv_interpreter_runfiles, - venv_interpreter_symlinks, - interpreter_args): - """Creates the providers an executable should return. +def _add_provider_default_info(providers, ctx, *, executable, default_outputs, runfiles_details): + """Adds the DefaultInfo provider. Args: + providers: list of providers to append to. ctx: The rule ctx. executable: File; the target's executable file. - main_py: File; the main .py entry point. - original_sources: `depset[File]` the direct `.py` sources for the - target that were the original input sources. - required_py_files: `depset[File]` the direct, `.py` sources for the - target that **must** be included by downstream targets. This should - only be Python source files. It should not include pyc files. - required_pyc_files: `depset[File]` the direct `.pyc` files this target - produces. - implicit_pyc_files: `depset[File]` pyc files that are only used if pyc - collection is enabled. - implicit_pyc_source_files: `depset[File]` source files for implicit pyc - files that are used when the implicit pyc files are not. default_outputs: depset of Files; the files for DefaultInfo.files - runfiles_details: runfiles that will become the default and data runfiles. - imports: depset of strings; the import paths to propagate - cc_info: optional CcInfo; Linking information to propagate as - PyCcLinkParamsInfo. Note that only the linking information - is propagated, not the whole CcInfo. + runfiles_details: runfiles that will become the default and data runfiles. + """ + providers.append(DefaultInfo( + executable = executable, + files = default_outputs, + default_runfiles = _py_builtins.make_runfiles_respect_legacy_external_runfiles( + ctx, + runfiles_details.default_runfiles, + ), + data_runfiles = _py_builtins.make_runfiles_respect_legacy_external_runfiles( + ctx, + runfiles_details.data_runfiles, + ), + )) + +def _add_provider_instrumented_files_info(providers, ctx): + """Adds the InstrumentedFilesInfo provider. + + Args: + providers: list of providers to append to. + ctx: The rule ctx. + """ + providers.append(create_instrumented_files_info(ctx)) + +def _add_provider_run_environment_info(providers, ctx, inherited_environment): + """Adds the RunEnvironmentInfo provider. + + Args: + providers: list of providers to append to. + ctx: The rule ctx. inherited_environment: list of strings; Environment variable names that should be inherited from the environment the executuble is run within. - runtime_details: struct of runtime information; see _get_runtime_details() - output_groups: dict[str, depset[File]]; used to create OutputGroupInfo - stage2_bootstrap: File; the stage 2 bootstrap script. + """ + expanded_env = {} + for key, value in ctx.attr.env.items(): + expanded_env[key] = _py_builtins.expand_location_and_make_variables( + ctx = ctx, + attribute_name = "env[{}]".format(key), + expression = value, + targets = ctx.attr.data, + ) + if "PYTHONBREAKPOINT" not in inherited_environment: + inherited_environment = inherited_environment + ["PYTHONBREAKPOINT"] + providers.append(RunEnvironmentInfo( + environment = expanded_env, + inherited_environment = inherited_environment, + )) + +def _add_provider_py_executable_info( + providers, + *, + app_runfiles, + build_data_file, + interpreter_args, + interpreter_path, + main_py, + runfiles_without_exe, + stage2_bootstrap, + venv_interpreter_runfiles, + venv_interpreter_symlinks, + venv_python_exe): + """Adds the PyExecutableInfo provider. + + Args: + providers: list of providers to append to. app_runfiles: runfiles; the runfiles for the application (deps, etc). - venv_python_exe: File; the python executable in the venv. - venv_interpreter_runfiles: runfiles; runfiles specific to the interpreter - for the venv. - venv_interpreter_symlinks: depset[ExplicitSymlink]; interpreter-specific symlinks - to create for the venv. + build_data_file: File; a file with build stamp information. interpreter_args: list of strings; arguments to pass to the interpreter. + interpreter_path: str; path to the Python interpreter. + main_py: File; the main .py entry point. + runfiles_without_exe: runfiles; the default runfiles, but without the executable. + stage2_bootstrap: File; the stage 2 bootstrap script. + venv_interpreter_runfiles: runfiles; runfiles specific to the interpreter for the venv. + venv_interpreter_symlinks: depset[ExplicitSymlink]; interpreter-specific symlinks to create for the venv. + venv_python_exe: File; the python executable in the venv. + """ + providers.append(PyExecutableInfo( + app_runfiles = app_runfiles, + build_data_file = build_data_file, + interpreter_args = interpreter_args, + interpreter_path = interpreter_path, + main = main_py, + runfiles_without_exe = runfiles_without_exe, + stage2_bootstrap = stage2_bootstrap, + venv_interpreter_runfiles = venv_interpreter_runfiles, + venv_interpreter_symlinks = venv_interpreter_symlinks, + venv_python_exe = venv_python_exe, + )) - Returns: - A list of modern providers. +def _add_provider_py_runtime_info(providers, runtime_details): + """Adds the PyRuntimeInfo provider. + + Args: + providers: list of providers to append to. + runtime_details: struct of runtime information; see _get_runtime_details() """ - providers = [ - DefaultInfo( - executable = executable, - files = default_outputs, - default_runfiles = _py_builtins.make_runfiles_respect_legacy_external_runfiles( - ctx, - runfiles_details.default_runfiles, - ), - data_runfiles = _py_builtins.make_runfiles_respect_legacy_external_runfiles( - ctx, - runfiles_details.data_runfiles, - ), - ), - create_instrumented_files_info(ctx), - _create_run_environment_info(ctx, inherited_environment), - PyExecutableInfo( - app_runfiles = app_runfiles, - build_data_file = runfiles_details.build_data_file, - interpreter_args = interpreter_args, - interpreter_path = runtime_details.executable_interpreter_path, - main = main_py, - runfiles_without_exe = runfiles_details.runfiles_without_exe, - stage2_bootstrap = stage2_bootstrap, - venv_interpreter_runfiles = venv_interpreter_runfiles, - venv_interpreter_symlinks = venv_interpreter_symlinks, - venv_python_exe = venv_python_exe, - ), - ] # TODO - The effective runtime can be None for Windows + auto detecting toolchain. # This can be removed once that's fixed; see maybe_get_runtime_from_ctx(). @@ -1917,6 +1947,16 @@ def _create_providers( bootstrap_template = py_runtime_info.bootstrap_template, )) +def _add_provider_py_cc_link_params_info(providers, cc_info): + """Adds the PyCcLinkParamsInfo provider. + + Args: + providers: list of providers to append to. + cc_info: optional CcInfo; Linking information to propagate as + PyCcLinkParamsInfo. Note that only the linking information + is propagated, not the whole CcInfo. + """ + # TODO(b/163083591): Remove the PyCcLinkParamsInfo once binaries-in-deps # are cleaned up. if cc_info: @@ -1924,6 +1964,37 @@ def _create_providers( PyCcLinkParamsInfo(cc_info = cc_info), ) +def _add_provider_py_info( + providers, + *, + ctx, + original_sources, + required_py_files, + required_pyc_files, + implicit_pyc_files, + implicit_pyc_source_files, + imports): + """Adds the PyInfo provider. + + Args: + providers: list of providers to append to. + ctx: The rule ctx. + original_sources: `depset[File]` the direct `.py` sources for the + target that were the original input sources. + required_py_files: `depset[File]` the direct, `.py` sources for the + target that **must** be included by downstream targets. This should + only be Python source files. It should not include pyc files. + required_pyc_files: `depset[File]` the direct `.pyc` files this target + produces. + implicit_pyc_files: `depset[File]` pyc files that are only used if pyc + collection is enabled. + implicit_pyc_source_files: `depset[File]` source files for implicit pyc + files that are used when the implicit pyc files are not. + imports: depset of strings; the import paths to propagate + + Returns: + PyInfo. + """ py_info, builtin_py_info = create_py_info( ctx, original_sources = original_sources, @@ -1933,28 +2004,20 @@ def _create_providers( implicit_pyc_source_files = implicit_pyc_source_files, imports = imports, ) - providers.append(py_info) if builtin_py_info: providers.append(builtin_py_info) - providers.append(create_output_group_info(py_info.transitive_sources, output_groups)) - return providers + return py_info -def _create_run_environment_info(ctx, inherited_environment): - expanded_env = {} - for key, value in ctx.attr.env.items(): - expanded_env[key] = _py_builtins.expand_location_and_make_variables( - ctx = ctx, - attribute_name = "env[{}]".format(key), - expression = value, - targets = ctx.attr.data, - ) - if "PYTHONBREAKPOINT" not in inherited_environment: - inherited_environment = inherited_environment + ["PYTHONBREAKPOINT"] - return RunEnvironmentInfo( - environment = expanded_env, - inherited_environment = inherited_environment, - ) +def _add_provider_output_group_info(providers, py_info, output_groups): + """Adds the OutputGroupInfo provider. + + Args: + providers: list of providers to append to. + py_info: PyInfo; the PyInfo provider. + output_groups: dict[str, depset[File]]; used to create OutputGroupInfo + """ + providers.append(create_output_group_info(py_info.transitive_sources, output_groups)) def _add_config_setting_defaults(kwargs): config_settings = kwargs.get("config_settings", None) From 9790ae06b32a729107965f0824d6703a50a7cfcf Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Mon, 13 Apr 2026 18:12:37 -0700 Subject: [PATCH 694/922] chore: populate 2.0 for VERSION_NEXT_FEATURE for py_wheel add_path_prefix (#3703) The add_path_prefix feature is in the 2.0 branch, but its version marker wasn't populated. Set it to 2.0.0 --- python/private/py_wheel.bzl | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/python/private/py_wheel.bzl b/python/private/py_wheel.bzl index 5b8b5c7879..1ca344c086 100644 --- a/python/private/py_wheel.bzl +++ b/python/private/py_wheel.bzl @@ -181,7 +181,8 @@ For example: + `"foo/" will prepend to `"bar/baz/file.py"` as `"foo/bar/baz/file.py"` + `"foo_" will prepend to `"bar/baz/file.py"` as `"foo_bar/baz/file.py"` + `stripping ["bar/"] and adding "foo/" will change `"bar/baz/file.py"` to `"foo/baz/file.py"` -:::{versionadded} VERSION_NEXT_FEATURE + +:::{versionadded} 2.0.0 The {attr}`add_path_prefix` attribute was added. ::: """, From 6e72de2c4377fc54093c5d3345970f309bb5d0c0 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Mon, 13 Apr 2026 19:26:31 -0700 Subject: [PATCH 695/922] chore: have no-matching-distribution error message hint about target_platforms vs requirements_by_platform (#3701) Based on the slack discussion where a cross-build was failing because `requirements_by_platform` was set, but `target_platforms` wasn't set, have the error message hint at the distinction to better point users for how to fix it. --------- Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> Co-authored-by: Ignas Anikevicius <240938+aignas@users.noreply.github.com> --- python/private/pypi/pkg_aliases.bzl | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/python/private/pypi/pkg_aliases.bzl b/python/private/pypi/pkg_aliases.bzl index e76a139651..111a49d3c6 100644 --- a/python/private/pypi/pkg_aliases.bzl +++ b/python/private/pypi/pkg_aliases.bzl @@ -46,13 +46,17 @@ load( ) _NO_MATCH_ERROR_TEMPLATE = """\ -No matching wheel for current configuration's Python version. +No matching wheel for current configuration's Python version and platform. The current build configuration's Python version doesn't match any of the Python wheels available for this distribution. This distribution supports the following Python configuration settings: {config_settings} +As configured by the `pip.parse.target_platforms` attribute. Note that +`requirements_by_platform` only affects the Bazel host platform unless +`target_platforms` is also set. + To determine the current configuration's Python version, run: `bazel config ` (shown further below) From 736934197201b83ba1bdf07c8a36ae2ecc5db931 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Mon, 13 Apr 2026 19:31:34 -0700 Subject: [PATCH 696/922] feat(runfiles): create pathlib api for runfiles library (#3694) This adds a pathlib.Path-based API for interacting with runfiles. This makes it easier to interact with runfiles. Fixes https://github.com/bazel-contrib/rules_python/issues/3296 --------- Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> --- CHANGELOG.md | 22 +++++ python/runfiles/runfiles.py | 146 +++++++++++++++++++++++++++++++-- tests/runfiles/BUILD.bazel | 6 ++ tests/runfiles/pathlib_test.py | 105 ++++++++++++++++++++++++ 4 files changed, 270 insertions(+), 9 deletions(-) create mode 100644 tests/runfiles/pathlib_test.py diff --git a/CHANGELOG.md b/CHANGELOG.md index 89b8356f00..a85317a20e 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -47,6 +47,28 @@ BEGIN_UNRELEASED_TEMPLATE END_UNRELEASED_TEMPLATE --> +{#v0-0-0} +## Unreleased + +[0.0.0]: https://github.com/bazel-contrib/rules_python/releases/tag/0.0.0 + +{#v0-0-0-removed} +### Removed +* Nothing removed. + +{#v0-0-0-changed} +### Changed +* Nothing changed. + +{#v0-0-0-fixed} +### Fixed +* Nothing fixed. + +{#v0-0-0-added} +### Added +* (runfiles) Added a pathlib-compatible API: {obj}`Runfiles.root()` + Fixes [#3296](https://github.com/bazel-contrib/rules_python/issues/3296). + {#v2-0-0} ## [2.0.0] - 2026-04-09 diff --git a/python/runfiles/runfiles.py b/python/runfiles/runfiles.py index f98646b1c2..f7be6ef42f 100644 --- a/python/runfiles/runfiles.py +++ b/python/runfiles/runfiles.py @@ -25,10 +25,11 @@ import collections.abc import inspect import os +import pathlib import posixpath import sys from collections import defaultdict -from typing import Dict, Optional, Tuple, Union +from typing import Dict, List, Optional, Tuple, Union class _RepositoryMapping: @@ -137,7 +138,127 @@ def is_empty(self) -> bool: Returns: True if there are no mappings, False otherwise """ - return len(self._exact_mappings) == 0 and len(self._grouped_prefixed_mappings) == 0 + return ( + len(self._exact_mappings) == 0 and len(self._grouped_prefixed_mappings) == 0 + ) + + +class Path(pathlib.PurePath): + """A pathlib-like path object for runfiles. + + This class extends `pathlib.PurePath` and resolves paths + using the associated `Runfiles` instance when converted to a string. + """ + + # For Python < 3.12 compatibility when subclassing PurePath directly + _flavour = getattr(type(pathlib.PurePath()), "_flavour", None) + + def __new__( + cls, + *args: Union[str, os.PathLike], + runfiles: Optional["Runfiles"] = None, + source_repo: Optional[str] = None, + ) -> "Path": + """Private constructor. Use Runfiles.root() to create instances.""" + obj = super().__new__(cls, *args) + # Type checkers might complain about adding attributes to PurePath, + # but this is standard for pathlib subclasses. + obj._runfiles = runfiles # type: ignore + obj._source_repo = source_repo # type: ignore + return obj + + def __init__( + self, + *args: Union[str, os.PathLike], + runfiles: Optional["Runfiles"] = None, + source_repo: Optional[str] = None, + ) -> None: + pass + + def with_segments(self, *pathsegments: Union[str, os.PathLike]) -> "Path": + """Used by Python 3.12+ pathlib to create new path objects.""" + return type(self)( + *pathsegments, + runfiles=self._runfiles, # type: ignore + source_repo=self._source_repo, # type: ignore + ) + + # For Python < 3.12 + @classmethod + def _from_parts(cls, args: Tuple[str, ...]) -> "Path": + obj = super()._from_parts(args) # type: ignore + # These will be set by the calling instance later, or we can't set them here + # properly without context. Usually pathlib calls this from an instance + # method like _make_child, which we also might need to override. + return obj + + def _make_child(self, args: Tuple[str, ...]) -> "Path": + obj = super()._make_child(args) # type: ignore + obj._runfiles = self._runfiles # type: ignore + obj._source_repo = self._source_repo # type: ignore + return obj + + @classmethod + def _from_parsed_parts(cls, drv: str, root: str, parts: List[str]) -> "Path": + obj = super()._from_parsed_parts(drv, root, parts) # type: ignore + return obj + + def _make_child_relpath(self, part: str) -> "Path": + obj = super()._make_child_relpath(part) # type: ignore + obj._runfiles = self._runfiles # type: ignore + obj._source_repo = self._source_repo # type: ignore + return obj + + @property + def parents(self) -> Tuple["Path", ...]: + return tuple( + type(self)( + p, + runfiles=getattr(self, "_runfiles", None), + source_repo=getattr(self, "_source_repo", None), + ) + for p in super().parents + ) + + @property + def parent(self) -> "Path": + return type(self)( + super().parent, + runfiles=getattr(self, "_runfiles", None), + source_repo=getattr(self, "_source_repo", None), + ) + + def with_name(self, name: str) -> "Path": + return type(self)( + super().with_name(name), + runfiles=getattr(self, "_runfiles", None), + source_repo=getattr(self, "_source_repo", None), + ) + + def with_suffix(self, suffix: str) -> "Path": + return type(self)( + super().with_suffix(suffix), + runfiles=getattr(self, "_runfiles", None), + source_repo=getattr(self, "_source_repo", None), + ) + + def __repr__(self) -> str: + return 'runfiles.Path({!r})'.format(super().__str__()) + + def __str__(self) -> str: + path_posix = super().__str__().replace("\\", "/") + if not path_posix or path_posix == ".": + # pylint: disable=protected-access + return self._runfiles._python_runfiles_root # type: ignore + resolved = self._runfiles.Rlocation(path_posix, source_repo=self._source_repo) # type: ignore + return resolved if resolved is not None else super().__str__() + + def __fspath__(self) -> str: + return str(self) + + def runfiles_root(self) -> "Path": + """Returns a Path object representing the runfiles root.""" + return self._runfiles.root(source_repo=self._source_repo) # type: ignore class _ManifestBased: @@ -254,6 +375,16 @@ def __init__(self, strategy: Union[_ManifestBased, _DirectoryBased]) -> None: strategy.RlocationChecked("_repo_mapping") ) + def root(self, source_repo: Optional[str] = None) -> Path: + """Returns a Path object representing the runfiles root. + + The repository mapping used by the returned Path object is that of the + caller of this method. + """ + if source_repo is None and not self._repo_mapping.is_empty(): + source_repo = self.CurrentRepository(frame=2) + return Path(runfiles=self, source_repo=source_repo) + def Rlocation(self, path: str, source_repo: Optional[str] = None) -> Optional[str]: """Returns the runtime path of a runfile. @@ -325,9 +456,7 @@ def Rlocation(self, path: str, source_repo: Optional[str] = None) -> Optional[st # Look up the target repository using the repository mapping if target_canonical is not None: - return self._strategy.RlocationChecked( - target_canonical + "/" + remainder - ) + return self._strategy.RlocationChecked(target_canonical + "/" + remainder) # No mapping found - assume target_repo is already canonical or # we're not using Bzlmod @@ -396,10 +525,9 @@ def CurrentRepository(self, frame: int = 1) -> str: # TODO: This doesn't cover the case of a script being run from an # external repository, which could be heuristically detected # by parsing the script's path. - if ( - (sys.version_info.minor <= 10 or sys.platform == "win32") - and sys.path[0] != self._python_runfiles_root - ): + if (sys.version_info.minor <= 10 or sys.platform == "win32") and sys.path[ + 0 + ] != self._python_runfiles_root: return "" raise ValueError( "{} does not lie under the runfiles root {}".format( diff --git a/tests/runfiles/BUILD.bazel b/tests/runfiles/BUILD.bazel index 84602d2bd6..7d675c7d7c 100644 --- a/tests/runfiles/BUILD.bazel +++ b/tests/runfiles/BUILD.bazel @@ -14,6 +14,12 @@ py_test( deps = ["//python/runfiles"], ) +py_test( + name = "pathlib_test", + srcs = ["pathlib_test.py"], + deps = ["//python/runfiles"], +) + build_test( name = "publishing", targets = [ diff --git a/tests/runfiles/pathlib_test.py b/tests/runfiles/pathlib_test.py new file mode 100644 index 0000000000..553c8e4410 --- /dev/null +++ b/tests/runfiles/pathlib_test.py @@ -0,0 +1,105 @@ +import os +import pathlib +import tempfile +import unittest + +from python.runfiles import runfiles + + +class PathlibTest(unittest.TestCase): + def setUp(self) -> None: + self.tmpdir = tempfile.TemporaryDirectory(dir=os.environ.get("TEST_TMPDIR")) + # Runfiles paths are expected to be posix paths internally when we construct the strings for assertions + self.root_dir = pathlib.Path(self.tmpdir.name).as_posix() + + def tearDown(self) -> None: + self.tmpdir.cleanup() + + def test_path_api(self) -> None: + r = runfiles.Create({"RUNFILES_DIR": self.root_dir}) + assert r is not None + root = r.root() + + # Test basic joining + p = root / "repo/pkg/file.txt" + self.assertEqual(str(p), f"{self.root_dir}/repo/pkg/file.txt") + + # Test PurePath API + self.assertEqual(p.name, "file.txt") + self.assertEqual(p.suffix, ".txt") + self.assertEqual(p.parent.name, "pkg") + self.assertEqual(p.parts, ("repo", "pkg", "file.txt")) + self.assertEqual(p.stem, "file") + self.assertEqual(p.suffixes, [".txt"]) + + # Test multiple joins + p2 = root / "repo" / "pkg" / "file.txt" + self.assertEqual(p, p2) + + # Test joins with pathlib objects + p3 = root / pathlib.PurePath("repo/pkg/file.txt") + self.assertEqual(p, p3) + + def test_root(self) -> None: + r = runfiles.Create({"RUNFILES_DIR": self.root_dir}) + assert r is not None + self.assertEqual(str(r.root()), self.root_dir) + + def test_runfiles_root_method(self) -> None: + r = runfiles.Create({"RUNFILES_DIR": self.root_dir}) + assert r is not None + p = r.root() / "foo/bar" + self.assertEqual(p.runfiles_root(), r.root()) + self.assertEqual(str(p.runfiles_root()), self.root_dir) + + def test_os_path_like(self) -> None: + r = runfiles.Create({"RUNFILES_DIR": self.root_dir}) + assert r is not None + p = r.root() / "foo" + self.assertEqual(os.fspath(p), f"{self.root_dir}/foo") + + def test_equality_and_hash(self) -> None: + r = runfiles.Create({"RUNFILES_DIR": self.root_dir}) + assert r is not None + p1 = r.root() / "foo" + p2 = r.root() / "foo" + p3 = r.root() / "bar" + + self.assertEqual(p1, p2) + self.assertNotEqual(p1, p3) + self.assertEqual(hash(p1), hash(p2)) + + def test_join_path(self) -> None: + r = runfiles.Create({"RUNFILES_DIR": self.root_dir}) + assert r is not None + p = r.root().joinpath("repo", "file") + self.assertEqual(str(p), f"{self.root_dir}/repo/file") + + def test_parents(self) -> None: + r = runfiles.Create({"RUNFILES_DIR": self.root_dir}) + assert r is not None + p = r.root() / "a/b/c" + parents = list(p.parents) + self.assertEqual(len(parents), 3) + self.assertEqual(str(parents[0]), f"{self.root_dir}/a/b") + self.assertEqual(str(parents[1]), f"{self.root_dir}/a") + self.assertEqual(str(parents[2]), self.root_dir) + + def test_with_methods(self) -> None: + r = runfiles.Create({"RUNFILES_DIR": self.root_dir}) + assert r is not None + p = r.root() / "foo/bar.txt" + self.assertEqual(str(p.with_name("baz.py")), f"{self.root_dir}/foo/baz.py") + self.assertEqual(str(p.with_suffix(".dat")), f"{self.root_dir}/foo/bar.dat") + + def test_match(self) -> None: + r = runfiles.Create({"RUNFILES_DIR": self.root_dir}) + assert r is not None + p = r.root() / "foo/bar.txt" + self.assertTrue(p.match("*.txt")) + self.assertTrue(p.match("foo/*.txt")) + self.assertFalse(p.match("bar/*.txt")) + + +if __name__ == "__main__": + unittest.main() From bf2d3c47d57cd5ac83ac3bfe93449e072bc27e2e Mon Sep 17 00:00:00 2001 From: Ignas Anikevicius <240938+aignas@users.noreply.github.com> Date: Tue, 14 Apr 2026 23:16:45 +0900 Subject: [PATCH 697/922] fix(sphinxdocs): update MODULE.bazel to depend on a released version (#3704) Related to the failure in rc0. See #3688. --- sphinxdocs/MODULE.bazel | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/sphinxdocs/MODULE.bazel b/sphinxdocs/MODULE.bazel index bbb9d7a688..1955b2f1e4 100644 --- a/sphinxdocs/MODULE.bazel +++ b/sphinxdocs/MODULE.bazel @@ -8,7 +8,7 @@ bazel_dep(name = "bazel_skylib", version = "1.8.2") bazel_dep(name = "stardoc", version = "0.7.2", repo_name = "io_bazel_stardoc") bazel_dep(name = "platforms", version = "0.0.11") bazel_dep(name = "protobuf", version = "29.0-rc2", repo_name = "com_google_protobuf") -bazel_dep(name = "rules_python", version = "0.0.0") +bazel_dep(name = "rules_python", version = "1.8.5") local_path_override( module_name = "rules_python", path = "..", From c33ba2383b7ce23f92fd97fbaa58132341dec2ca Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Tue, 14 Apr 2026 16:49:13 -0700 Subject: [PATCH 698/922] sphinxdocs: remove local rules_python dev overrides (#3705) In the BCR integration test, the local override breaks on BCR itself because a local copy of rules_python isn't part of the test environment. Remove it from the sphinxdocs module, too, because it shouldn't be necessary. The two aren't that closely coupled. --- sphinxdocs/MODULE.bazel | 4 ---- sphinxdocs/integration_tests/bcr/MODULE.bazel | 6 +----- 2 files changed, 1 insertion(+), 9 deletions(-) diff --git a/sphinxdocs/MODULE.bazel b/sphinxdocs/MODULE.bazel index 1955b2f1e4..30fb2196dc 100644 --- a/sphinxdocs/MODULE.bazel +++ b/sphinxdocs/MODULE.bazel @@ -9,10 +9,6 @@ bazel_dep(name = "stardoc", version = "0.7.2", repo_name = "io_bazel_stardoc") bazel_dep(name = "platforms", version = "0.0.11") bazel_dep(name = "protobuf", version = "29.0-rc2", repo_name = "com_google_protobuf") bazel_dep(name = "rules_python", version = "1.8.5") -local_path_override( - module_name = "rules_python", - path = "..", -) dev_pip = use_extension( "@rules_python//python/extensions:pip.bzl", diff --git a/sphinxdocs/integration_tests/bcr/MODULE.bazel b/sphinxdocs/integration_tests/bcr/MODULE.bazel index 711144df9b..ae1c752513 100644 --- a/sphinxdocs/integration_tests/bcr/MODULE.bazel +++ b/sphinxdocs/integration_tests/bcr/MODULE.bazel @@ -9,11 +9,7 @@ local_path_override( path = "../..", ) -bazel_dep(name = "rules_python", version = "0.0.0") -local_path_override( - module_name = "rules_python", - path = "../../..", -) +bazel_dep(name = "rules_python", version = "1.8.5") dev_pip = use_extension( "@rules_python//python/extensions:pip.bzl", From 7a6822151634d20c286e1c3869d79f11081b568a Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Tue, 14 Apr 2026 20:05:04 -0700 Subject: [PATCH 699/922] tests: better simulate BCR environment (#3706) Better simulate the BCR environment by removing files that aren't extracted when BCR extracts the release archive. --- .bazelci/isolate.sh | 32 ++++++++++++++++++++++++++++++++ .bazelci/presubmit.yml | 4 ++++ 2 files changed, 36 insertions(+) create mode 100755 .bazelci/isolate.sh diff --git a/.bazelci/isolate.sh b/.bazelci/isolate.sh new file mode 100755 index 0000000000..32a1de3fcf --- /dev/null +++ b/.bazelci/isolate.sh @@ -0,0 +1,32 @@ +#!/usr/bin/env bash +set -euo pipefail + +module_dir="${1:-}" + +if [[ -z "${module_dir}" ]]; then + echo "Usage: $0 " + exit 1 +fi + +# Find the repository root assuming this script is in .bazelci/ +script_dir="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" +repo_root="$(dirname "${script_dir}")" + +cd "${repo_root}" + +if [[ ! -f "MODULE.bazel" ]]; then + echo "Error: MODULE.bazel not found in ${repo_root}. Are you sure this is the repo root?" + exit 1 +fi + +if [[ ! -d "${module_dir}" ]]; then + echo "Error: Module directory '${module_dir}' not found in ${repo_root}." + exit 1 +fi + +echo "Removing files outside of ${module_dir} to simulate BCR environment..." +find . -maxdepth 1 -mindepth 1 \ + ! -name "${module_dir}" \ + ! -name ".git" \ + ! -name ".bazelci" \ + -exec rm -rf '{}' + diff --git a/.bazelci/presubmit.yml b/.bazelci/presubmit.yml index aeeb7103d6..1159c2f732 100644 --- a/.bazelci/presubmit.yml +++ b/.bazelci/presubmit.yml @@ -180,6 +180,8 @@ tasks: name: "Sphinxdocs: Ubuntu" working_directory: "sphinxdocs" platform: ubuntu2204 + shell_commands: + - "../.bazelci/isolate.sh sphinxdocs" build_targets: - "//..." test_targets: @@ -188,6 +190,8 @@ tasks: name: "Sphinxdocs: Mac" working_directory: "sphinxdocs" platform: macos_arm64 + shell_commands: + - "../.bazelci/isolate.sh sphinxdocs" build_targets: - "//..." test_targets: From dcfb311ca217b9a9b8c6950653d68203cb464563 Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Tue, 14 Apr 2026 20:23:57 -0700 Subject: [PATCH 700/922] build(deps): bump cryptography from 46.0.6 to 46.0.7 in /tools/publish in the pip group across 1 directory (#3687) Bumps the pip group with 1 update in the /tools/publish directory: [cryptography](https://github.com/pyca/cryptography). Updates `cryptography` from 46.0.6 to 46.0.7
Changelog

Sourced from cryptography's changelog.

46.0.7 - 2026-04-07


* **SECURITY ISSUE**: Fixed an issue where non-contiguous buffers could
be
  passed to APIs that accept Python buffers, which could lead to buffer
  overflow. **CVE-2026-39892**
* Updated Windows, macOS, and Linux wheels to be compiled with OpenSSL
3.5.6.

.. _v46-0-6:

Commits

[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=cryptography&package-manager=pip&previous-version=46.0.6&new-version=46.0.7)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) ---
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot show ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore major version` will close this group update PR and stop Dependabot creating any more for the specific dependency's major version (unless you unignore this specific dependency's major version or upgrade to it yourself) - `@dependabot ignore minor version` will close this group update PR and stop Dependabot creating any more for the specific dependency's minor version (unless you unignore this specific dependency's minor version or upgrade to it yourself) - `@dependabot ignore ` will close this group update PR and stop Dependabot creating any more for the specific dependency (unless you unignore this specific dependency or upgrade to it yourself) - `@dependabot unignore ` will remove all of the ignore conditions of the specified dependency - `@dependabot unignore ` will remove the ignore condition of the specified dependency and ignore conditions You can disable automated security fix PRs for this repo from the [Security Alerts page](https://github.com/bazel-contrib/rules_python/network/alerts).
Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> --- tools/publish/requirements_linux.txt | 100 +++++++++++------------ tools/publish/requirements_universal.txt | 100 +++++++++++------------ 2 files changed, 100 insertions(+), 100 deletions(-) diff --git a/tools/publish/requirements_linux.txt b/tools/publish/requirements_linux.txt index 57dada2723..8865eb3ecb 100644 --- a/tools/publish/requirements_linux.txt +++ b/tools/publish/requirements_linux.txt @@ -177,56 +177,56 @@ charset-normalizer==3.4.3 \ --hash=sha256:fd10de089bcdcd1be95a2f73dbe6254798ec1bda9f450d5828c96f93e2536b9c \ --hash=sha256:fdabf8315679312cfa71302f9bd509ded4f2f263fb5b765cf1433b39106c3cc9 # via requests -cryptography==46.0.6 \ - --hash=sha256:02fad249cb0e090b574e30b276a3da6a149e04ee2f049725b1f69e7b8351ec70 \ - --hash=sha256:063b67749f338ca9c5a0b7fe438a52c25f9526b851e24e6c9310e7195aad3b4d \ - --hash=sha256:12cae594e9473bca1a7aceb90536060643128bb274fcea0fc459ab90f7d1ae7a \ - --hash=sha256:12f0fa16cc247b13c43d56d7b35287ff1569b5b1f4c5e87e92cc4fcc00cd10c0 \ - --hash=sha256:22259338084d6ae497a19bae5d4c66b7ca1387d3264d1c2c0e72d9e9b6a77b97 \ - --hash=sha256:26031f1e5ca62fcb9d1fcb34b2b60b390d1aacaa15dc8b895a9ed00968b97b30 \ - --hash=sha256:27550628a518c5c6c903d84f637fbecf287f6cb9ced3804838a1295dc1fd0759 \ - --hash=sha256:2b417edbe8877cda9022dde3a008e2deb50be9c407eef034aeeb3a8b11d9db3c \ - --hash=sha256:2ea0f37e9a9cf0df2952893ad145fd9627d326a59daec9b0802480fa3bcd2ead \ - --hash=sha256:2ef9e69886cbb137c2aef9772c2e7138dc581fad4fcbcf13cc181eb5a3ab6275 \ - --hash=sha256:341359d6c9e68834e204ceaf25936dffeafea3829ab80e9503860dcc4f4dac58 \ - --hash=sha256:380343e0653b1c9d7e1f55b52aaa2dbb2fdf2730088d48c43ca1c7c0abb7cc2f \ - --hash=sha256:3c21d92ed15e9cfc6eb64c1f5a0326db22ca9c2566ca46d845119b45b4400361 \ - --hash=sha256:3dfa6567f2e9e4c5dceb8ccb5a708158a2a871052fa75c8b78cb0977063f1507 \ - --hash=sha256:456b3215172aeefb9284550b162801d62f5f264a081049a3e94307fe20792cfa \ - --hash=sha256:4668298aef7cddeaf5c6ecc244c2302a2b8e40f384255505c22875eebb47888b \ - --hash=sha256:50575a76e2951fe7dbd1f56d181f8c5ceeeb075e9ff88e7ad997d2f42af06e7b \ - --hash=sha256:639301950939d844a9e1c4464d7e07f902fe9a7f6b215bb0d4f28584729935d8 \ - --hash=sha256:64235194bad039a10bb6d2d930ab3323baaec67e2ce36215fd0952fad0930ca8 \ - --hash=sha256:6617f67b1606dfd9fe4dbfa354a9508d4a6d37afe30306fe6c101b7ce3274b72 \ - --hash=sha256:67177e8a9f421aa2d3a170c3e56eca4e0128883cf52a071a7cbf53297f18b175 \ - --hash=sha256:6728c49e3b2c180ef26f8e9f0a883a2c585638db64cf265b49c9ba10652d430e \ - --hash=sha256:6739d56300662c468fddb0e5e291f9b4d084bead381667b9e654c7dd81705124 \ - --hash=sha256:69cf0056d6947edc6e6760e5f17afe4bea06b56a9ac8a06de9d2bd6b532d4f3a \ - --hash=sha256:760997a4b950ff00d418398ad73fbc91aa2894b5c1db7ccb45b4f68b42a63b3c \ - --hash=sha256:79e865c642cfc5c0b3eb12af83c35c5aeff4fa5c672dc28c43721c2c9fdd2f0f \ - --hash=sha256:7e6142674f2a9291463e5e150090b95a8519b2fb6e6aaec8917dd8d094ce750d \ - --hash=sha256:7f417f034f91dcec1cb6c5c35b07cdbb2ef262557f701b4ecd803ee8cefed4f4 \ - --hash=sha256:7f6690b6c55e9c5332c0b59b9c8a3fb232ebf059094c17f9019a51e9827df91c \ - --hash=sha256:8927ccfbe967c7df312ade694f987e7e9e22b2425976ddbf28271d7e58845290 \ - --hash=sha256:8ce35b77aaf02f3b59c90b2c8a05c73bac12cea5b4e8f3fbece1f5fddea5f0ca \ - --hash=sha256:8e7304c4f4e9490e11efe56af6713983460ee0780f16c63f219984dab3af9d2d \ - --hash=sha256:90e5f0a7b3be5f40c3a0a0eafb32c681d8d2c181fc2a1bdabe9b3f611d9f6b1a \ - --hash=sha256:97c8115b27e19e592a05c45d0dd89c57f81f841cc9880e353e0d3bf25b2139ed \ - --hash=sha256:9a693028b9cbe51b5a1136232ee8f2bc242e4e19d456ded3fa7c86e43c713b4a \ - --hash=sha256:9a9c42a2723999a710445bc0d974e345c32adfd8d2fac6d8a251fa829ad31cfb \ - --hash=sha256:a3e84d5ec9ba01f8fd03802b2147ba77f0c8f2617b2aff254cedd551844209c8 \ - --hash=sha256:aad75154a7ac9039936d50cf431719a2f8d4ed3d3c277ac03f3339ded1a5e707 \ - --hash=sha256:b12c6b1e1651e42ab5de8b1e00dc3b6354fdfd778e7fa60541ddacc27cd21410 \ - --hash=sha256:b928a3ca837c77a10e81a814a693f2295200adb3352395fad024559b7be7a736 \ - --hash=sha256:bcb87663e1f7b075e48c3be3ecb5f0b46c8fc50b50a97cf264e7f60242dca3f2 \ - --hash=sha256:c797e2517cb7880f8297e2c0f43bb910e91381339336f75d2c1c2cbf811b70b4 \ - --hash=sha256:c89eb37fae9216985d8734c1afd172ba4927f5a05cfd9bf0e4863c6d5465b013 \ - --hash=sha256:cdcd3edcbc5d55757e5f5f3d330dd00007ae463a7e7aa5bf132d1f22a4b62b19 \ - --hash=sha256:d24c13369e856b94892a89ddf70b332e0b70ad4a5c43cf3e9cb71d6d7ffa1f7b \ - --hash=sha256:d4e4aadb7fc1f88687f47ca20bb7227981b03afaae69287029da08096853b738 \ - --hash=sha256:d9528b535a6c4f8ff37847144b8986a9a143585f0540fbcb1a98115b543aa463 \ - --hash=sha256:ed3775295fb91f70b4027aeba878d79b3e55c0b3e97eaa4de71f8f23a9f2eb77 \ - --hash=sha256:ed418c37d095aeddf5336898a132fba01091f0ac5844e3e8018506f014b6d2c4 +cryptography==46.0.7 \ + --hash=sha256:04959522f938493042d595a736e7dbdff6eb6cc2339c11465b3ff89343b65f65 \ + --hash=sha256:128c5edfe5e5938b86b03941e94fac9ee793a94452ad1365c9fc3f4f62216832 \ + --hash=sha256:1d25aee46d0c6f1a501adcddb2d2fee4b979381346a78558ed13e50aa8a59067 \ + --hash=sha256:24402210aa54baae71d99441d15bb5a1919c195398a87b563df84468160a65de \ + --hash=sha256:258514877e15963bd43b558917bc9f54cf7cf866c38aa576ebf47a77ddbc43a4 \ + --hash=sha256:35719dc79d4730d30f1c2b6474bd6acda36ae2dfae1e3c16f2051f215df33ce0 \ + --hash=sha256:397655da831414d165029da9bc483bed2fe0e75dde6a1523ec2fe63f3c46046b \ + --hash=sha256:3986ac1dee6def53797289999eabe84798ad7817f3e97779b5061a95b0ee4968 \ + --hash=sha256:420b1e4109cc95f0e5700eed79908cef9268265c773d3a66f7af1eef53d409ef \ + --hash=sha256:42a1e5f98abb6391717978baf9f90dc28a743b7d9be7f0751a6f56a75d14065b \ + --hash=sha256:462ad5cb1c148a22b2e3bcc5ad52504dff325d17daf5df8d88c17dda1f75f2a4 \ + --hash=sha256:506c4ff91eff4f82bdac7633318a526b1d1309fc07ca76a3ad182cb5b686d6d3 \ + --hash=sha256:5ad9ef796328c5e3c4ceed237a183f5d41d21150f972455a9d926593a1dcb308 \ + --hash=sha256:5d1c02a14ceb9148cc7816249f64f623fbfee39e8c03b3650d842ad3f34d637e \ + --hash=sha256:5e51be372b26ef4ba3de3c167cd3d1022934bc838ae9eaad7e644986d2a3d163 \ + --hash=sha256:60627cf07e0d9274338521205899337c5d18249db56865f943cbe753aa96f40f \ + --hash=sha256:65814c60f8cc400c63131584e3e1fad01235edba2614b61fbfbfa954082db0ee \ + --hash=sha256:73510b83623e080a2c35c62c15298096e2a5dc8d51c3b4e1740211839d0dea77 \ + --hash=sha256:7bbc6ccf49d05ac8f7d7b5e2e2c33830d4fe2061def88210a126d130d7f71a85 \ + --hash=sha256:80406c3065e2c55d7f49a9550fe0c49b3f12e5bfff5dedb727e319e1afb9bf99 \ + --hash=sha256:84d4cced91f0f159a7ddacad249cc077e63195c36aac40b4150e7a57e84fffe7 \ + --hash=sha256:8a469028a86f12eb7d2fe97162d0634026d92a21f3ae0ac87ed1c4a447886c83 \ + --hash=sha256:91bbcb08347344f810cbe49065914fe048949648f6bd5c2519f34619142bbe85 \ + --hash=sha256:935ce7e3cfdb53e3536119a542b839bb94ec1ad081013e9ab9b7cfd478b05006 \ + --hash=sha256:9694078c5d44c157ef3162e3bf3946510b857df5a3955458381d1c7cfc143ddb \ + --hash=sha256:a1529d614f44b863a7b480c6d000fe93b59acee9c82ffa027cfadc77521a9f5e \ + --hash=sha256:abad9dac36cbf55de6eb49badd4016806b3165d396f64925bf2999bcb67837ba \ + --hash=sha256:b36a4695e29fe69215d75960b22577197aca3f7a25b9cf9d165dcfe9d80bc325 \ + --hash=sha256:b7b412817be92117ec5ed95f880defe9cf18a832e8cafacf0a22337dc1981b4d \ + --hash=sha256:c5b1ccd1239f48b7151a65bc6dd54bcfcc15e028c8ac126d3fada09db0e07ef1 \ + --hash=sha256:cbd5fb06b62bd0721e1170273d3f4d5a277044c47ca27ee257025146c34cbdd1 \ + --hash=sha256:cdf1a610ef82abb396451862739e3fc93b071c844399e15b90726ef7470eeaf2 \ + --hash=sha256:cdfbe22376065ffcf8be74dc9a909f032df19bc58a699456a21712d6e5eabfd0 \ + --hash=sha256:d02c738dacda7dc2a74d1b2b3177042009d5cab7c7079db74afc19e56ca1b455 \ + --hash=sha256:d151173275e1728cf7839aaa80c34fe550c04ddb27b34f48c232193df8db5842 \ + --hash=sha256:d23c8ca48e44ee015cd0a54aeccdf9f09004eba9fc96f38c911011d9ff1bd457 \ + --hash=sha256:d3b99c535a9de0adced13d159c5a9cf65c325601aa30f4be08afd680643e9c15 \ + --hash=sha256:d5f7520159cd9c2154eb61eb67548ca05c5774d39e9c2c4339fd793fe7d097b2 \ + --hash=sha256:db0f493b9181c7820c8134437eb8b0b4792085d37dbb24da050476ccb664e59c \ + --hash=sha256:e06acf3c99be55aa3b516397fe42f5855597f430add9c17fa46bf2e0fb34c9bb \ + --hash=sha256:e4cfd68c5f3e0bfdad0d38e023239b96a2fe84146481852dffbcca442c245aa5 \ + --hash=sha256:ea42cbe97209df307fdc3b155f1b6fa2577c0defa8f1f7d3be7d31d189108ad4 \ + --hash=sha256:ebd6daf519b9f189f85c479427bbd6e9c9037862cf8fe89ee35503bd209ed902 \ + --hash=sha256:f247c8c1a1fb45e12586afbb436ef21ff1e80670b2861a90353d9b025583d246 \ + --hash=sha256:fbfd0e5f273877695cb93baf14b185f4878128b250cc9f8e617ea0c025dfb022 \ + --hash=sha256:fc9ab8856ae6cf7c9358430e49b368f3108f050031442eaeb6b9d87e4dcf4e4f \ + --hash=sha256:fcd8eac50d9138c1d7fc53a653ba60a2bee81a505f9f8850b6b2888555a45d0e \ + --hash=sha256:fdd1736fed309b4300346f88f74cd120c27c56852c3838cab416e7a166f67298 \ + --hash=sha256:ffca7aa1d00cf7d6469b988c581598f2259e46215e0140af408966a24cf086ce # via secretstorage docutils==0.22.2 \ --hash=sha256:9fdb771707c8784c8f2728b67cb2c691305933d68137ef95a75db5f4dfbc213d \ diff --git a/tools/publish/requirements_universal.txt b/tools/publish/requirements_universal.txt index d907b894ee..4dbcc92abc 100644 --- a/tools/publish/requirements_universal.txt +++ b/tools/publish/requirements_universal.txt @@ -160,56 +160,56 @@ charset-normalizer==3.4.3 \ --hash=sha256:fd10de089bcdcd1be95a2f73dbe6254798ec1bda9f450d5828c96f93e2536b9c \ --hash=sha256:fdabf8315679312cfa71302f9bd509ded4f2f263fb5b765cf1433b39106c3cc9 # via requests -cryptography==46.0.6 ; sys_platform == 'linux' \ - --hash=sha256:02fad249cb0e090b574e30b276a3da6a149e04ee2f049725b1f69e7b8351ec70 \ - --hash=sha256:063b67749f338ca9c5a0b7fe438a52c25f9526b851e24e6c9310e7195aad3b4d \ - --hash=sha256:12cae594e9473bca1a7aceb90536060643128bb274fcea0fc459ab90f7d1ae7a \ - --hash=sha256:12f0fa16cc247b13c43d56d7b35287ff1569b5b1f4c5e87e92cc4fcc00cd10c0 \ - --hash=sha256:22259338084d6ae497a19bae5d4c66b7ca1387d3264d1c2c0e72d9e9b6a77b97 \ - --hash=sha256:26031f1e5ca62fcb9d1fcb34b2b60b390d1aacaa15dc8b895a9ed00968b97b30 \ - --hash=sha256:27550628a518c5c6c903d84f637fbecf287f6cb9ced3804838a1295dc1fd0759 \ - --hash=sha256:2b417edbe8877cda9022dde3a008e2deb50be9c407eef034aeeb3a8b11d9db3c \ - --hash=sha256:2ea0f37e9a9cf0df2952893ad145fd9627d326a59daec9b0802480fa3bcd2ead \ - --hash=sha256:2ef9e69886cbb137c2aef9772c2e7138dc581fad4fcbcf13cc181eb5a3ab6275 \ - --hash=sha256:341359d6c9e68834e204ceaf25936dffeafea3829ab80e9503860dcc4f4dac58 \ - --hash=sha256:380343e0653b1c9d7e1f55b52aaa2dbb2fdf2730088d48c43ca1c7c0abb7cc2f \ - --hash=sha256:3c21d92ed15e9cfc6eb64c1f5a0326db22ca9c2566ca46d845119b45b4400361 \ - --hash=sha256:3dfa6567f2e9e4c5dceb8ccb5a708158a2a871052fa75c8b78cb0977063f1507 \ - --hash=sha256:456b3215172aeefb9284550b162801d62f5f264a081049a3e94307fe20792cfa \ - --hash=sha256:4668298aef7cddeaf5c6ecc244c2302a2b8e40f384255505c22875eebb47888b \ - --hash=sha256:50575a76e2951fe7dbd1f56d181f8c5ceeeb075e9ff88e7ad997d2f42af06e7b \ - --hash=sha256:639301950939d844a9e1c4464d7e07f902fe9a7f6b215bb0d4f28584729935d8 \ - --hash=sha256:64235194bad039a10bb6d2d930ab3323baaec67e2ce36215fd0952fad0930ca8 \ - --hash=sha256:6617f67b1606dfd9fe4dbfa354a9508d4a6d37afe30306fe6c101b7ce3274b72 \ - --hash=sha256:67177e8a9f421aa2d3a170c3e56eca4e0128883cf52a071a7cbf53297f18b175 \ - --hash=sha256:6728c49e3b2c180ef26f8e9f0a883a2c585638db64cf265b49c9ba10652d430e \ - --hash=sha256:6739d56300662c468fddb0e5e291f9b4d084bead381667b9e654c7dd81705124 \ - --hash=sha256:69cf0056d6947edc6e6760e5f17afe4bea06b56a9ac8a06de9d2bd6b532d4f3a \ - --hash=sha256:760997a4b950ff00d418398ad73fbc91aa2894b5c1db7ccb45b4f68b42a63b3c \ - --hash=sha256:79e865c642cfc5c0b3eb12af83c35c5aeff4fa5c672dc28c43721c2c9fdd2f0f \ - --hash=sha256:7e6142674f2a9291463e5e150090b95a8519b2fb6e6aaec8917dd8d094ce750d \ - --hash=sha256:7f417f034f91dcec1cb6c5c35b07cdbb2ef262557f701b4ecd803ee8cefed4f4 \ - --hash=sha256:7f6690b6c55e9c5332c0b59b9c8a3fb232ebf059094c17f9019a51e9827df91c \ - --hash=sha256:8927ccfbe967c7df312ade694f987e7e9e22b2425976ddbf28271d7e58845290 \ - --hash=sha256:8ce35b77aaf02f3b59c90b2c8a05c73bac12cea5b4e8f3fbece1f5fddea5f0ca \ - --hash=sha256:8e7304c4f4e9490e11efe56af6713983460ee0780f16c63f219984dab3af9d2d \ - --hash=sha256:90e5f0a7b3be5f40c3a0a0eafb32c681d8d2c181fc2a1bdabe9b3f611d9f6b1a \ - --hash=sha256:97c8115b27e19e592a05c45d0dd89c57f81f841cc9880e353e0d3bf25b2139ed \ - --hash=sha256:9a693028b9cbe51b5a1136232ee8f2bc242e4e19d456ded3fa7c86e43c713b4a \ - --hash=sha256:9a9c42a2723999a710445bc0d974e345c32adfd8d2fac6d8a251fa829ad31cfb \ - --hash=sha256:a3e84d5ec9ba01f8fd03802b2147ba77f0c8f2617b2aff254cedd551844209c8 \ - --hash=sha256:aad75154a7ac9039936d50cf431719a2f8d4ed3d3c277ac03f3339ded1a5e707 \ - --hash=sha256:b12c6b1e1651e42ab5de8b1e00dc3b6354fdfd778e7fa60541ddacc27cd21410 \ - --hash=sha256:b928a3ca837c77a10e81a814a693f2295200adb3352395fad024559b7be7a736 \ - --hash=sha256:bcb87663e1f7b075e48c3be3ecb5f0b46c8fc50b50a97cf264e7f60242dca3f2 \ - --hash=sha256:c797e2517cb7880f8297e2c0f43bb910e91381339336f75d2c1c2cbf811b70b4 \ - --hash=sha256:c89eb37fae9216985d8734c1afd172ba4927f5a05cfd9bf0e4863c6d5465b013 \ - --hash=sha256:cdcd3edcbc5d55757e5f5f3d330dd00007ae463a7e7aa5bf132d1f22a4b62b19 \ - --hash=sha256:d24c13369e856b94892a89ddf70b332e0b70ad4a5c43cf3e9cb71d6d7ffa1f7b \ - --hash=sha256:d4e4aadb7fc1f88687f47ca20bb7227981b03afaae69287029da08096853b738 \ - --hash=sha256:d9528b535a6c4f8ff37847144b8986a9a143585f0540fbcb1a98115b543aa463 \ - --hash=sha256:ed3775295fb91f70b4027aeba878d79b3e55c0b3e97eaa4de71f8f23a9f2eb77 \ - --hash=sha256:ed418c37d095aeddf5336898a132fba01091f0ac5844e3e8018506f014b6d2c4 +cryptography==46.0.7 ; sys_platform == 'linux' \ + --hash=sha256:04959522f938493042d595a736e7dbdff6eb6cc2339c11465b3ff89343b65f65 \ + --hash=sha256:128c5edfe5e5938b86b03941e94fac9ee793a94452ad1365c9fc3f4f62216832 \ + --hash=sha256:1d25aee46d0c6f1a501adcddb2d2fee4b979381346a78558ed13e50aa8a59067 \ + --hash=sha256:24402210aa54baae71d99441d15bb5a1919c195398a87b563df84468160a65de \ + --hash=sha256:258514877e15963bd43b558917bc9f54cf7cf866c38aa576ebf47a77ddbc43a4 \ + --hash=sha256:35719dc79d4730d30f1c2b6474bd6acda36ae2dfae1e3c16f2051f215df33ce0 \ + --hash=sha256:397655da831414d165029da9bc483bed2fe0e75dde6a1523ec2fe63f3c46046b \ + --hash=sha256:3986ac1dee6def53797289999eabe84798ad7817f3e97779b5061a95b0ee4968 \ + --hash=sha256:420b1e4109cc95f0e5700eed79908cef9268265c773d3a66f7af1eef53d409ef \ + --hash=sha256:42a1e5f98abb6391717978baf9f90dc28a743b7d9be7f0751a6f56a75d14065b \ + --hash=sha256:462ad5cb1c148a22b2e3bcc5ad52504dff325d17daf5df8d88c17dda1f75f2a4 \ + --hash=sha256:506c4ff91eff4f82bdac7633318a526b1d1309fc07ca76a3ad182cb5b686d6d3 \ + --hash=sha256:5ad9ef796328c5e3c4ceed237a183f5d41d21150f972455a9d926593a1dcb308 \ + --hash=sha256:5d1c02a14ceb9148cc7816249f64f623fbfee39e8c03b3650d842ad3f34d637e \ + --hash=sha256:5e51be372b26ef4ba3de3c167cd3d1022934bc838ae9eaad7e644986d2a3d163 \ + --hash=sha256:60627cf07e0d9274338521205899337c5d18249db56865f943cbe753aa96f40f \ + --hash=sha256:65814c60f8cc400c63131584e3e1fad01235edba2614b61fbfbfa954082db0ee \ + --hash=sha256:73510b83623e080a2c35c62c15298096e2a5dc8d51c3b4e1740211839d0dea77 \ + --hash=sha256:7bbc6ccf49d05ac8f7d7b5e2e2c33830d4fe2061def88210a126d130d7f71a85 \ + --hash=sha256:80406c3065e2c55d7f49a9550fe0c49b3f12e5bfff5dedb727e319e1afb9bf99 \ + --hash=sha256:84d4cced91f0f159a7ddacad249cc077e63195c36aac40b4150e7a57e84fffe7 \ + --hash=sha256:8a469028a86f12eb7d2fe97162d0634026d92a21f3ae0ac87ed1c4a447886c83 \ + --hash=sha256:91bbcb08347344f810cbe49065914fe048949648f6bd5c2519f34619142bbe85 \ + --hash=sha256:935ce7e3cfdb53e3536119a542b839bb94ec1ad081013e9ab9b7cfd478b05006 \ + --hash=sha256:9694078c5d44c157ef3162e3bf3946510b857df5a3955458381d1c7cfc143ddb \ + --hash=sha256:a1529d614f44b863a7b480c6d000fe93b59acee9c82ffa027cfadc77521a9f5e \ + --hash=sha256:abad9dac36cbf55de6eb49badd4016806b3165d396f64925bf2999bcb67837ba \ + --hash=sha256:b36a4695e29fe69215d75960b22577197aca3f7a25b9cf9d165dcfe9d80bc325 \ + --hash=sha256:b7b412817be92117ec5ed95f880defe9cf18a832e8cafacf0a22337dc1981b4d \ + --hash=sha256:c5b1ccd1239f48b7151a65bc6dd54bcfcc15e028c8ac126d3fada09db0e07ef1 \ + --hash=sha256:cbd5fb06b62bd0721e1170273d3f4d5a277044c47ca27ee257025146c34cbdd1 \ + --hash=sha256:cdf1a610ef82abb396451862739e3fc93b071c844399e15b90726ef7470eeaf2 \ + --hash=sha256:cdfbe22376065ffcf8be74dc9a909f032df19bc58a699456a21712d6e5eabfd0 \ + --hash=sha256:d02c738dacda7dc2a74d1b2b3177042009d5cab7c7079db74afc19e56ca1b455 \ + --hash=sha256:d151173275e1728cf7839aaa80c34fe550c04ddb27b34f48c232193df8db5842 \ + --hash=sha256:d23c8ca48e44ee015cd0a54aeccdf9f09004eba9fc96f38c911011d9ff1bd457 \ + --hash=sha256:d3b99c535a9de0adced13d159c5a9cf65c325601aa30f4be08afd680643e9c15 \ + --hash=sha256:d5f7520159cd9c2154eb61eb67548ca05c5774d39e9c2c4339fd793fe7d097b2 \ + --hash=sha256:db0f493b9181c7820c8134437eb8b0b4792085d37dbb24da050476ccb664e59c \ + --hash=sha256:e06acf3c99be55aa3b516397fe42f5855597f430add9c17fa46bf2e0fb34c9bb \ + --hash=sha256:e4cfd68c5f3e0bfdad0d38e023239b96a2fe84146481852dffbcca442c245aa5 \ + --hash=sha256:ea42cbe97209df307fdc3b155f1b6fa2577c0defa8f1f7d3be7d31d189108ad4 \ + --hash=sha256:ebd6daf519b9f189f85c479427bbd6e9c9037862cf8fe89ee35503bd209ed902 \ + --hash=sha256:f247c8c1a1fb45e12586afbb436ef21ff1e80670b2861a90353d9b025583d246 \ + --hash=sha256:fbfd0e5f273877695cb93baf14b185f4878128b250cc9f8e617ea0c025dfb022 \ + --hash=sha256:fc9ab8856ae6cf7c9358430e49b368f3108f050031442eaeb6b9d87e4dcf4e4f \ + --hash=sha256:fcd8eac50d9138c1d7fc53a653ba60a2bee81a505f9f8850b6b2888555a45d0e \ + --hash=sha256:fdd1736fed309b4300346f88f74cd120c27c56852c3838cab416e7a166f67298 \ + --hash=sha256:ffca7aa1d00cf7d6469b988c581598f2259e46215e0140af408966a24cf086ce # via secretstorage docutils==0.22.2 \ --hash=sha256:9fdb771707c8784c8f2728b67cb2c691305933d68137ef95a75db5f4dfbc213d \ From 208ca88f15994b20bcd50e075224af773dfeed8e Mon Sep 17 00:00:00 2001 From: Ignas Anikevicius <240938+aignas@users.noreply.github.com> Date: Fri, 17 Apr 2026 08:17:56 +0900 Subject: [PATCH 701/922] fix(pypi): skip index lookups when all package overrides are specified (#3710) When index_url_overrides is provided for all packages, we no longer need to call the index at all. This improves performance and aligns with the expected behavior where overrides should be sufficient. Just to note, currently Pytorch, PyPI, Artifactory have the root index pages, whilst GAR does not. Fixes #3709 --- CHANGELOG.md | 3 +- python/private/pypi/simpleapi_download.bzl | 23 +++++----- .../simpleapi_download_tests.bzl | 45 +++++++++++++++++++ 3 files changed, 59 insertions(+), 12 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index a85317a20e..84b3efea72 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -103,7 +103,8 @@ Other changes: we will from now on fetch the lists of available packages on each index. The used package mappings will be written as facts to the `MODULE.bazel.lock` file on supported bazel versions and it should be done at most once. As a result, - per-package {obj}`experimental_index_url_overrides` is no longer needed . What + per-package {obj}`experimental_index_url_overrides` is no longer needed, but + if specified, it needs to be provided for all packages not on the default index. What is more, the flags for `--index_url` and `--extra-index-url` now behave in the same way as in `uv` or `pip`, i.e. we default to `--index-url` if the package is not found in `--extra-index-url`. Fixes diff --git a/python/private/pypi/simpleapi_download.bzl b/python/private/pypi/simpleapi_download.bzl index 97215d753d..63044bc14a 100644 --- a/python/private/pypi/simpleapi_download.bzl +++ b/python/private/pypi/simpleapi_download.bzl @@ -43,8 +43,7 @@ def simpleapi_download( attr: Contains the parameters for the download. They are grouped into a struct for better clarity. It must have attributes: * index_url: str, the index, or if `extra_index_urls` are passed, the default index. - * index_url_overrides: dict[str, str], the index overrides for - separate packages. + * index_url_overrides: dict[str, str], the index overrides for separate packages. * extra_index_urls: Will be looked at in the order they are defined and the first match wins. This is similar to what uv does, see https://docs.astral.sh/uv/concepts/indexes/#searching-across-multiple-indexes. @@ -132,16 +131,22 @@ def simpleapi_download( return contents def _get_dist_urls(ctx, *, default_index, index_urls, index_url_overrides, sources, read_simpleapi, attr, block, _fail = fail, **kwargs): + if index_url_overrides: + # Let's not call the index at all and just assume that all of the overrides have been + # specified. + return { + pkg: _normalize_url("{}/{}/".format( + index_url_overrides.get(pkg, default_index), + pkg.replace("_", "-"), # Use the official normalization for URLs + )) + for pkg in sources + } + downloads = {} results = {} # Ensure the value is not frozen index_urls = [] + (index_urls or []) - for extra in index_url_overrides.values(): - if extra not in index_urls: - index_urls.append(extra) - - index_urls = index_urls or [] if default_index not in index_urls: index_urls.append(default_index) @@ -174,10 +179,6 @@ def _get_dist_urls(ctx, *, default_index, index_urls, index_url_overrides, sourc # and in the outer function process merging of the results. continue - if index_url_overrides.get(pkg, index_url) != index_url: - # we should not use this index for the package - continue - found = result.output.get(pkg) if not found: continue diff --git a/tests/pypi/simpleapi_download/simpleapi_download_tests.bzl b/tests/pypi/simpleapi_download/simpleapi_download_tests.bzl index 7d7cbee0ca..405c497508 100644 --- a/tests/pypi/simpleapi_download/simpleapi_download_tests.bzl +++ b/tests/pypi/simpleapi_download/simpleapi_download_tests.bzl @@ -268,6 +268,51 @@ def _test_download_url_parallel(env): _tests.append(_test_download_url_parallel) +def _test_download_url_parallel_with_overrides(env): + downloads = {} + reads = [ + "", + "", + "", + ] + + def download(url, output, **kwargs): + _ = kwargs # buildifier: disable=unused-variable + downloads[url[0]] = output + return struct(wait = lambda: struct(success = True)) + + simpleapi_download( + ctx = struct( + getenv = {}.get, + download = download, + report_progress = lambda _: None, + # We will first add a download to the list, so this is a poor man's `next(foo)` + # implementation. We use 2 because we will enqueue 2 downloads in parallel. + read = lambda i: reads[len(downloads) - 2], + path = lambda i: "path/for/" + i, + ), + attr = struct( + index_url_overrides = { + "bar": "https://example.com/extra/simple/", + }, + index_url = "https://example.com/default/simple/", + extra_index_urls = [], + sources = {"bar": None, "baz": None, "foo": None}, + envsubst = [], + ), + cache = pypi_cache(), + parallel_download = True, + get_auth = lambda ctx, urls, ctx_attr: struct(), + ) + + env.expect.that_dict(downloads).contains_exactly({ + "https://example.com/default/simple/baz/": "path/for/https___example_com_default_simple_baz.html", + "https://example.com/default/simple/foo/": "path/for/https___example_com_default_simple_foo.html", + "https://example.com/extra/simple/bar/": "path/for/https___example_com_extra_simple_bar.html", + }) + +_tests.append(_test_download_url_parallel_with_overrides) + def _test_download_envsubst_url(env): downloads = {} reads = [ From 6dac0f6d07216e153f9ce220b6e5d0b2fe0b0b9a Mon Sep 17 00:00:00 2001 From: peter woodman Date: Thu, 16 Apr 2026 22:00:19 -0400 Subject: [PATCH 702/922] feat(toolchains): Add 3.10.20, 3.11.15, 3.12.13, 3.13.{12,13} 3.14.{3,4}, 3.15.0a8 (#3708) This updates the Python version mappings to include the latest released versions. --- CHANGELOG.md | 6 + examples/wheel/requirements_server.in | 2 +- examples/wheel/requirements_server.txt | 14 +- examples/wheel/test_publish.py | 22 +-- python/versions.bzl | 186 ++++++++++++++++++++++++- 5 files changed, 209 insertions(+), 21 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 84b3efea72..4c9e42c855 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -68,6 +68,12 @@ END_UNRELEASED_TEMPLATE ### Added * (runfiles) Added a pathlib-compatible API: {obj}`Runfiles.root()` Fixes [#3296](https://github.com/bazel-contrib/rules_python/issues/3296). +* (toolchains) `3.13.12`, `3.14.3` Python toolchain from [20260325] release. +* (toolchains) `3.10.20`, `3.11.15`, `3.12.13`, `3.13.13` `3.14.4`, `3.15.0a8` +* Python toolchain from [20260414] release. + +[20260325]: https://github.com/astral-sh/python-build-standalone/releases/tag/20260325 +[20260414]: https://github.com/astral-sh/python-build-standalone/releases/tag/20260414 {#v2-0-0} ## [2.0.0] - 2026-04-09 diff --git a/examples/wheel/requirements_server.in b/examples/wheel/requirements_server.in index d5d483d56a..631bdb8fb5 100644 --- a/examples/wheel/requirements_server.in +++ b/examples/wheel/requirements_server.in @@ -1,2 +1,2 @@ # This is for running publishing tests -pypiserver +pypiserver>=2.2.0 diff --git a/examples/wheel/requirements_server.txt b/examples/wheel/requirements_server.txt index eccab1271b..7324131a7c 100644 --- a/examples/wheel/requirements_server.txt +++ b/examples/wheel/requirements_server.txt @@ -4,9 +4,17 @@ # # bazel run //examples/wheel:requirements_server.update # -pypiserver==2.0.1 \ - --hash=sha256:1dd98fb99d2da4199fb44c7284e57d69a9f7fda2c6c8dc01975c151c592677bf \ - --hash=sha256:7b58fbd54468235f79e4de07c4f7a9ff829e7ac6869bef47ec11e0710138e162 +importlib-resources==7.1.0 \ + --hash=sha256:0722d4c6212489c530f2a145a34c0a7a3b4721bc96a15fada5930e2a0b760708 \ + --hash=sha256:1bd7b48b4088eddb2cd16382150bb515af0bd2c70128194392725f82ad2c96a1 + # via pypiserver +packaging==26.1 \ + --hash=sha256:5d9c0669c6285e491e0ced2eee587eaf67b670d94a19e94e3984a481aba6802f \ + --hash=sha256:f042152b681c4bfac5cae2742a55e103d27ab2ec0f3d88037136b6bfe7c9c5de + # via pypiserver +pypiserver==2.4.1 \ + --hash=sha256:156540f87ecfd6db06ae2c16e25ae5afe4fda6f510bd1c34e46fbb0c491bcd9e \ + --hash=sha256:45f116d0bff6aafcaed002cfad48a6832e62a82393e3a9b447d5c41a0e310fff # via -r examples/wheel/requirements_server.in # The following packages are considered to be unsafe in a requirements file: diff --git a/examples/wheel/test_publish.py b/examples/wheel/test_publish.py index 7665629c19..4bc657c52e 100644 --- a/examples/wheel/test_publish.py +++ b/examples/wheel/test_publish.py @@ -6,6 +6,7 @@ import unittest from contextlib import closing from pathlib import Path +from urllib.error import URLError from urllib.request import urlopen @@ -50,17 +51,16 @@ def setUp(self): ], ) - line = "Hit Ctrl-C to quit" interval = 0.1 wait_seconds = 40 for _ in range(int(wait_seconds / interval)): # 40 second timeout - current_logs = self.log_file.read_text() - if line in current_logs: - print(current_logs.strip()) - print("...") - break - - time.sleep(0.1) + try: + with urlopen(self.url, timeout=1) as response: + if response.status == 200: + break + except (URLError, OSError): + pass + time.sleep(interval) else: raise RuntimeError( f"Could not get the server running fast enough, waited for {wait_seconds}s" @@ -98,13 +98,15 @@ def test_upload_and_query_simple_api(self): got_content = response.read().decode("utf-8") want_content = """ - + + + Links for example-minimal-library

Links for example-minimal-library

- example_minimal_library-0.0.1-py3-none-any.whl
+ example_minimal_library-0.0.1-py3-none-any.whl
""" self.assertEqual( diff --git a/python/versions.bzl b/python/versions.bzl index f3c712e559..33b2398dc0 100644 --- a/python/versions.bzl +++ b/python/versions.bzl @@ -234,6 +234,21 @@ TOOL_VERSIONS = { }, "strip_prefix": "python", }, + "3.10.20": { + "url": "20260414/cpython-{python_version}+20260414-{platform}-{build}.{ext}", + "sha256": { + "aarch64-apple-darwin": "f76cc83c7db16cfc8794bf6e44d834152b57d8bab4e04e823cbc59ed23ec22f8", + "aarch64-unknown-linux-gnu": "64932c8e8bbdf9d6b66ee85934f6f8ad1d18218b51a87ea06cefd3b84554a3e4", + "ppc64le-unknown-linux-gnu": "76b48eb26ef274045772186e63431419294c41baf6d5a372b722d4c9e711082e", + "riscv64-unknown-linux-gnu": "76e1ec72717d17493976fc176ec661f02412666d4f19e50908d8e4303c0511d5", + "s390x-unknown-linux-gnu": "2edf241199d11a3ef79a312737c1bcdb86908352c585ca14b667539080630e85", + "x86_64-apple-darwin": "95a2d794b8981723095190fa94b574ceb4272bb49d83b9e418bb90341e304d09", + "x86_64-pc-windows-msvc": "0d828683d30185ab9f1110ad2194ef384cef0533b8e0da7e03ce837548841788", + "x86_64-unknown-linux-gnu": "303047011b2c9f58504a930fc974d84547477cf69a3f2962f25552e2395c13af", + "x86_64-unknown-linux-musl": "84eb198d318f8b1b8bf59eef5d30d742e13afd97c213fa229578f8fdab0c406f", + }, + "strip_prefix": "python", + }, "3.11.1": { "url": "20230116/cpython-{python_version}+20230116-{platform}-{build}.tar.gz", "sha256": { @@ -381,6 +396,22 @@ TOOL_VERSIONS = { }, "strip_prefix": "python", }, + "3.11.15": { + "url": "20260414/cpython-{python_version}+20260414-{platform}-{build}.{ext}", + "sha256": { + "aarch64-apple-darwin": "a57ffd435652092d16b30e783f9826c55e9c64b0f0a72cbae0a9f39e663137fb", + "aarch64-unknown-linux-gnu": "77836944ae15b74e0b25bdc68a4703a340f2ccb684effc0f45fbd7910e1a1f39", + "ppc64le-unknown-linux-gnu": "30a2107f000dbe304820627cbe2cc257027c20f3241d96e6c7df796b69ac2062", + "riscv64-unknown-linux-gnu": "373b98fbf2d04099139a2f6be57593714382ed790be7e7419e358830c23ddd0f", + "s390x-unknown-linux-gnu": "7838efa839158c80568de35ac78d438f564f4c32272a2fe7d9e14a9b351d1a62", + "x86_64-apple-darwin": "317055d80e553764feeaef432d833dd8385c14b83465a8b3fa7c2b7819cba681", + "x86_64-pc-windows-msvc": "8e69ecf1d9fc194e029aafa608d483bf24ccaa8f56d456d7009f20462d62ad23", + "aarch64-pc-windows-msvc": "a882abe4876985c9dc3d433420548506fb0cc9bb9d9fe336a2d3aaf28922aa45", + "x86_64-unknown-linux-gnu": "8b14030dd3af9ea7f7c51b4c90feb04afd8a8f45435727e67b875270bd08f3bc", + "x86_64-unknown-linux-musl": "ca92d3a68a39fa330498b09714733f347bead7313ba9d9b7fbed837aa4ba7796", + }, + "strip_prefix": "python", + }, "3.12.0": { "url": "20231002/cpython-{python_version}+20231002-{platform}-{build}.tar.gz", "sha256": { @@ -521,6 +552,22 @@ TOOL_VERSIONS = { }, "strip_prefix": "python", }, + "3.12.13": { + "url": "20260414/cpython-{python_version}+20260414-{platform}-{build}.{ext}", + "sha256": { + "aarch64-apple-darwin": "8966b2bcd9fa03ba22c080ad15a86bc12e41a00122b16f4b3740e302261124d9", + "aarch64-unknown-linux-gnu": "355d981eafb9b2870af79ddc106ced7266b6f6d2101d8fbcb05620fa386642b9", + "ppc64le-unknown-linux-gnu": "4aef4cffe73c4a65ea486f14d684a9ad3f831a354174d163bb531b5baa70fc49", + "riscv64-unknown-linux-gnu": "c2629d69324155132343913f064be93509bd162531e08a292e50c3973ec8b5db", + "s390x-unknown-linux-gnu": "e5baafd64180f45165d2751b25d1bcc89254eefc7926f3ab341fc61b541d7606", + "x86_64-apple-darwin": "801b03fbe004181d55a02ebd8b4e04d74973e70d716062aebe3b3cf32e9be297", + "x86_64-pc-windows-msvc": "c5a9e011e284c49c48106ca177342f3e3f64e95b4c6652d4a382cc7c9bb1cc46", + "aarch64-pc-windows-msvc": "f55326c894fde76fc0faffe95d2bce60be533c88a8c44c1b88bbbc17bf6a5cd5", + "x86_64-unknown-linux-gnu": "cdcf8724d46e4857f8db5ee9f4252dc2f5da34f7940294ec6b312389dd3f41e0", + "x86_64-unknown-linux-musl": "d10e971238c130fdf25e577c6538a3effa5589d5fcf53665e3c711edd6a6ff2f", + }, + "strip_prefix": "python", + }, "3.13.0": { "url": "20241016/cpython-{python_version}+20241016-{platform}-{build}.{ext}", "sha256": { @@ -857,6 +904,56 @@ TOOL_VERSIONS = { "x86_64-unknown-linux-gnu-freethreaded": "python/install", }, }, + "3.13.12": { + "url": "20260325/cpython-{python_version}+20260325-{platform}-{build}.{ext}", + "sha256": { + "aarch64-apple-darwin": "688da81bcaa6ed91792397c7d5433b13a4f02f021f940637c3972639bc516dca", + "aarch64-unknown-linux-gnu": "31c6e61eed48ca4e156d0e473025a792338641109e8277a63518ded438390c96", + "ppc64le-unknown-linux-gnu": "654939bc40d5f76f08eb17335bb19e9efa11eb48a0818eda2293a3f7c3570ae7", + "riscv64-unknown-linux-gnu": "fc7e1fb553c47b831ed7fa529575145207f000f967513f7b9ea809cce006ed79", + "s390x-unknown-linux-gnu": "7d7919358e88fcc672b061be8c2316c3a604c7074200515d7104166ed611f7f9", + "x86_64-apple-darwin": "7411e47939783708381017a90944a69641ac84d43f74fb6e2d52576c599a2717", + "x86_64-pc-windows-msvc": "5b4093f92d9bffcb0d92aea050f3d77d5a4fc8e918b31cea000ee4b3ca751f1d", + "aarch64-pc-windows-msvc": "d2c8b00044cd2e4c5fc7e697e63d5e481ed44b87c2def0beb42991d59f65d930", + "aarch64-pc-windows-msvc-freethreaded": "d2c8b00044cd2e4c5fc7e697e63d5e481ed44b87c2def0beb42991d59f65d930", + "x86_64-unknown-linux-gnu": "ebb1051ca2822b9803f46a5f10b6d51d153189ef1b1f1e142f733c0cbeaf86eb", + "x86_64-unknown-linux-musl": "b2e9400731c7f18069ec2804ba87a404385fe440f93b7dcb59004b9f56651202", + "aarch64-apple-darwin-freethreaded": "688da81bcaa6ed91792397c7d5433b13a4f02f021f940637c3972639bc516dca", + "aarch64-unknown-linux-gnu-freethreaded": "31c6e61eed48ca4e156d0e473025a792338641109e8277a63518ded438390c96", + "ppc64le-unknown-linux-gnu-freethreaded": "654939bc40d5f76f08eb17335bb19e9efa11eb48a0818eda2293a3f7c3570ae7", + "riscv64-unknown-linux-gnu-freethreaded": "fc7e1fb553c47b831ed7fa529575145207f000f967513f7b9ea809cce006ed79", + "s390x-unknown-linux-gnu-freethreaded": "7d7919358e88fcc672b061be8c2316c3a604c7074200515d7104166ed611f7f9", + "x86_64-apple-darwin-freethreaded": "7411e47939783708381017a90944a69641ac84d43f74fb6e2d52576c599a2717", + "x86_64-pc-windows-msvc-freethreaded": "5b4093f92d9bffcb0d92aea050f3d77d5a4fc8e918b31cea000ee4b3ca751f1d", + "x86_64-unknown-linux-gnu-freethreaded": "ebb1051ca2822b9803f46a5f10b6d51d153189ef1b1f1e142f733c0cbeaf86eb", + }, + "strip_prefix": "python", + }, + "3.13.13": { + "url": "20260414/cpython-{python_version}+20260414-{platform}-{build}.{ext}", + "sha256": { + "aarch64-apple-darwin": "c652dad552122cd2e76968ec41c803f8222038169b11310dba0c85928265f5c1", + "aarch64-unknown-linux-gnu": "6a65f68043d7fadcd580415493d2929d1fd686013f9ae44ddbd3a81307ab256d", + "ppc64le-unknown-linux-gnu": "aef73894107300264222b19e357baf5bad616b1c4bf5daa5c3b97cfee8f5ed7b", + "riscv64-unknown-linux-gnu": "f47c09f8e7f2fb0bc4afe52422705af4016c8d3ec1cf004b67bb56a86caa62cb", + "s390x-unknown-linux-gnu": "4d205af9654e1f33cefd23ff798af470e565f3ac0eba18d2f98f18a2abd07166", + "x86_64-apple-darwin": "540337412d2c4220e99280f741dbf45c1e3da3a39edaaab20c6ba1d53e1692ef", + "x86_64-pc-windows-msvc": "ee0cb26453d6e025d36502d765c1639c34830355e46ab3ad31c0360bc4cd9b79", + "aarch64-pc-windows-msvc": "586ba71c75f341e1d111399b7f719ae784dc11e8672e93e017388f28684226d0", + "aarch64-pc-windows-msvc-freethreaded": "586ba71c75f341e1d111399b7f719ae784dc11e8672e93e017388f28684226d0", + "x86_64-unknown-linux-gnu": "e5ec3b2c5693215d153c434ac018e75511b2c4f96d2bce30468a477cb3a89d5e", + "x86_64-unknown-linux-musl": "24ac6bf80dd2991c8be348f777c96c6eb69b71e78d8fa28c09beb3ddca015a47", + "aarch64-apple-darwin-freethreaded": "c652dad552122cd2e76968ec41c803f8222038169b11310dba0c85928265f5c1", + "aarch64-unknown-linux-gnu-freethreaded": "6a65f68043d7fadcd580415493d2929d1fd686013f9ae44ddbd3a81307ab256d", + "ppc64le-unknown-linux-gnu-freethreaded": "aef73894107300264222b19e357baf5bad616b1c4bf5daa5c3b97cfee8f5ed7b", + "riscv64-unknown-linux-gnu-freethreaded": "f47c09f8e7f2fb0bc4afe52422705af4016c8d3ec1cf004b67bb56a86caa62cb", + "s390x-unknown-linux-gnu-freethreaded": "4d205af9654e1f33cefd23ff798af470e565f3ac0eba18d2f98f18a2abd07166", + "x86_64-apple-darwin-freethreaded": "540337412d2c4220e99280f741dbf45c1e3da3a39edaaab20c6ba1d53e1692ef", + "x86_64-pc-windows-msvc-freethreaded": "ee0cb26453d6e025d36502d765c1639c34830355e46ab3ad31c0360bc4cd9b79", + "x86_64-unknown-linux-gnu-freethreaded": "e5ec3b2c5693215d153c434ac018e75511b2c4f96d2bce30468a477cb3a89d5e", + }, + "strip_prefix": "python", + }, "3.14.0": { "url": "20251031/cpython-{python_version}+20251031-{platform}-{build}.{ext}", "sha256": { @@ -992,6 +1089,56 @@ TOOL_VERSIONS = { "x86_64-unknown-linux-gnu-freethreaded": "python/install", }, }, + "3.14.3": { + "url": "20260325/cpython-{python_version}+20260325-{platform}-{build}.{ext}", + "sha256": { + "aarch64-apple-darwin": "80c996c23aab828134821f078a8a77a6f33f3f2c14000f071718c540e20c64d4", + "aarch64-unknown-linux-gnu": "6faf5478f910741c477830f5fd842011208af0f9678faf77106c9421b325bfc1", + "ppc64le-unknown-linux-gnu": "5eafe32e12f33f98c40de920482b013170dcf97d8c7f5dc780271ccf4cded76a", + "riscv64-unknown-linux-gnu": "481d3faef258964e57b7102c63de12b2bb388c7ed07cfe456f33e63b4e061202", + "s390x-unknown-linux-gnu": "d706eae2f4d963187b7c866603aed75d7eb3ea59590b06fb34f5fd7d0fe8e432", + "x86_64-apple-darwin": "847a49fea36c066f8df7a57cd8c4c02d17667e25d30b7930e8f8ba15e72d7efc", + "x86_64-pc-windows-msvc": "8b4e1329c4901ce2c0f1c20ac5d2ffa62fc13f12e26b5d1e5a1000f910f980d4", + "aarch64-pc-windows-msvc": "b35fe7c2fe169574f382cef125e95cbd904ddcb98fc337167356371b6d2e8c60", + "x86_64-unknown-linux-gnu": "18270c5a7b1a572599df5e68b497ba5254811dac43ba6f542245807d821fcb44", + "x86_64-unknown-linux-musl": "726a28734d2878a637b0d16ce07ce24c7d6ca1043d8e6f4a23b1b0a3478eedb9", + "aarch64-apple-darwin-freethreaded": "80c996c23aab828134821f078a8a77a6f33f3f2c14000f071718c540e20c64d4", + "aarch64-unknown-linux-gnu-freethreaded": "6faf5478f910741c477830f5fd842011208af0f9678faf77106c9421b325bfc1", + "ppc64le-unknown-linux-gnu-freethreaded": "5eafe32e12f33f98c40de920482b013170dcf97d8c7f5dc780271ccf4cded76a", + "riscv64-unknown-linux-gnu-freethreaded": "481d3faef258964e57b7102c63de12b2bb388c7ed07cfe456f33e63b4e061202", + "s390x-unknown-linux-gnu-freethreaded": "d706eae2f4d963187b7c866603aed75d7eb3ea59590b06fb34f5fd7d0fe8e432", + "x86_64-apple-darwin-freethreaded": "847a49fea36c066f8df7a57cd8c4c02d17667e25d30b7930e8f8ba15e72d7efc", + "x86_64-pc-windows-msvc-freethreaded": "8b4e1329c4901ce2c0f1c20ac5d2ffa62fc13f12e26b5d1e5a1000f910f980d4", + "aarch64-pc-windows-msvc-freethreaded": "b35fe7c2fe169574f382cef125e95cbd904ddcb98fc337167356371b6d2e8c60", + "x86_64-unknown-linux-gnu-freethreaded": "18270c5a7b1a572599df5e68b497ba5254811dac43ba6f542245807d821fcb44", + }, + "strip_prefix": "python", + }, + "3.14.4": { + "url": "20260414/cpython-{python_version}+20260414-{platform}-{build}.{ext}", + "sha256": { + "aarch64-apple-darwin": "8b7865e511b17093e090449bf71eb52933c17d45ad5257ddeacaffbb2c7239df", + "aarch64-unknown-linux-gnu": "5c8db1c21023316adad827a46d917bbbd6a85ae4e39bc3a58febda712c2f963d", + "ppc64le-unknown-linux-gnu": "055977a09de092744bbb22db64144e6afef8592eaac5e2bce4cca33f2592281a", + "riscv64-unknown-linux-gnu": "e959df167c502fb0bbcacc31a997e25c6b0ff6b5e496321b691955aa702d0c09", + "s390x-unknown-linux-gnu": "35f70ad05b2c4045889ee0c3d93f61b012654c1d91e10e671f0e5b4d4a6c6637", + "x86_64-apple-darwin": "9ecb2b942e6698c04af10a63a3d73c0b2e8d8e11ce44933fbffe8651bef4577d", + "x86_64-pc-windows-msvc": "9647bb46d3c236e34c1c11bbb7113444d9711811f0d11c39956168807a955b1a", + "aarch64-pc-windows-msvc": "82613380d582d806e562d7701496c34c87753ab13c37aa0afe2039003651f389", + "x86_64-unknown-linux-gnu": "e17275eaf95ceb5877aa6816e209b7733f41fee401d39c3921b88fb73fc4a4ba", + "x86_64-unknown-linux-musl": "12687a989a2384665577e1ef9864f33d4c074a1e69b38a8bac8d656531aefa3e", + "aarch64-apple-darwin-freethreaded": "8b7865e511b17093e090449bf71eb52933c17d45ad5257ddeacaffbb2c7239df", + "aarch64-unknown-linux-gnu-freethreaded": "5c8db1c21023316adad827a46d917bbbd6a85ae4e39bc3a58febda712c2f963d", + "ppc64le-unknown-linux-gnu-freethreaded": "055977a09de092744bbb22db64144e6afef8592eaac5e2bce4cca33f2592281a", + "riscv64-unknown-linux-gnu-freethreaded": "e959df167c502fb0bbcacc31a997e25c6b0ff6b5e496321b691955aa702d0c09", + "s390x-unknown-linux-gnu-freethreaded": "35f70ad05b2c4045889ee0c3d93f61b012654c1d91e10e671f0e5b4d4a6c6637", + "x86_64-apple-darwin-freethreaded": "9ecb2b942e6698c04af10a63a3d73c0b2e8d8e11ce44933fbffe8651bef4577d", + "x86_64-pc-windows-msvc-freethreaded": "9647bb46d3c236e34c1c11bbb7113444d9711811f0d11c39956168807a955b1a", + "aarch64-pc-windows-msvc-freethreaded": "82613380d582d806e562d7701496c34c87753ab13c37aa0afe2039003651f389", + "x86_64-unknown-linux-gnu-freethreaded": "e17275eaf95ceb5877aa6816e209b7733f41fee401d39c3921b88fb73fc4a4ba", + }, + "strip_prefix": "python", + }, "3.15.0a1": { "url": "20251031/cpython-{python_version}+20251031-{platform}-{build}.{ext}", "sha256": { @@ -1082,17 +1229,42 @@ TOOL_VERSIONS = { "x86_64-unknown-linux-gnu-freethreaded": "python/install", }, }, + "3.15.0a8": { + "url": "20260414/cpython-{python_version}+20260414-{platform}-{build}.{ext}", + "sha256": { + "aarch64-apple-darwin": "780d46b3da0e58e15c620d9e7dfd29b54c8359c195f625858f85df9c2c7ecc32", + "aarch64-unknown-linux-gnu": "8f6dda4d8ff44976f1aa6a94674a09a503dc50b015297e1b62c8cdc591c90f4f", + "ppc64le-unknown-linux-gnu": "09f076c63fadbf675143674aa3b23229482b9a44840b8b1808a216def2a9af15", + "riscv64-unknown-linux-gnu": "9d41ce752e8b731872f0f5c9c48199e63c789d24ce3ae9e91d6c8008f36e7c51", + "s390x-unknown-linux-gnu": "1de2593c40cce2d8ea883f8c8580223bfa1478cbd9d0191ba3640aed083c2202", + "x86_64-apple-darwin": "a7744d34148969a2ec010da6f0a46ddeceda7c02e5cdfa2b4e1811487381491a", + "x86_64-pc-windows-msvc": "3ded476f676fdf260d56a5e49aa083d5ffd218fc3390e4480ed42bee1acfb3fb", + "aarch64-pc-windows-msvc": "10fb470e900e65df4e37f8deaf1726397c914861ffc37b43ae3743a7eee88377", + "x86_64-unknown-linux-gnu": "c93f4b15287ac48d7e3a475b245cb59cc51079382747e3e6213d6406c158969d", + "x86_64-unknown-linux-musl": "9fbd6f243a424d4ae973e72aa0075122a7cfe05ac8f6cfde986e7b00d0dbc0bf", + "aarch64-apple-darwin-freethreaded": "780d46b3da0e58e15c620d9e7dfd29b54c8359c195f625858f85df9c2c7ecc32", + "aarch64-unknown-linux-gnu-freethreaded": "8f6dda4d8ff44976f1aa6a94674a09a503dc50b015297e1b62c8cdc591c90f4f", + "ppc64le-unknown-linux-gnu-freethreaded": "09f076c63fadbf675143674aa3b23229482b9a44840b8b1808a216def2a9af15", + "riscv64-unknown-linux-gnu-freethreaded": "9d41ce752e8b731872f0f5c9c48199e63c789d24ce3ae9e91d6c8008f36e7c51", + "s390x-unknown-linux-gnu-freethreaded": "1de2593c40cce2d8ea883f8c8580223bfa1478cbd9d0191ba3640aed083c2202", + "x86_64-apple-darwin-freethreaded": "a7744d34148969a2ec010da6f0a46ddeceda7c02e5cdfa2b4e1811487381491a", + "x86_64-pc-windows-msvc-freethreaded": "3ded476f676fdf260d56a5e49aa083d5ffd218fc3390e4480ed42bee1acfb3fb", + "aarch64-pc-windows-msvc-freethreaded": "10fb470e900e65df4e37f8deaf1726397c914861ffc37b43ae3743a7eee88377", + "x86_64-unknown-linux-gnu-freethreaded": "c93f4b15287ac48d7e3a475b245cb59cc51079382747e3e6213d6406c158969d", + }, + "strip_prefix": "python", + }, } # buildifier: disable=unsorted-dict-items MINOR_MAPPING = { "3.9": "3.9.25", - "3.10": "3.10.19", - "3.11": "3.11.14", - "3.12": "3.12.12", - "3.13": "3.13.11", - "3.14": "3.14.2", - "3.15": "3.15.0a2", + "3.10": "3.10.20", + "3.11": "3.11.15", + "3.12": "3.12.13", + "3.13": "3.13.13", + "3.14": "3.14.4", + "3.15": "3.15.0a8", } def _generate_platforms(): @@ -1287,7 +1459,7 @@ def get_release_info(platform, python_version, base_url = DEFAULT_RELEASE_BASE_U maybe_release_id = url_parts[-2] release_id = int(maybe_release_id) - if FREETHREADED.lstrip("-") in platform: + if FREETHREADED.lstrip("-") in platform and release_id < 20260325: build = "{}+{}-full".format( FREETHREADED.lstrip("-"), { From fe43548142f525c54c55d93d8ceda88f61ac94c3 Mon Sep 17 00:00:00 2001 From: Douglas Thor Date: Sun, 19 Apr 2026 03:00:56 -0700 Subject: [PATCH 703/922] deps(gazelle): Bump WORKSPACE gazelle version 0.36.0 --> 0.47.0 (to match MODULE.bazel) (#3717) Note that this implicitly updates go from 1.21.13 to 1.22.9 Steps taken to generate this commit: 1. Manually update go.mod 2. go mod tidy 3. bazel run //:gazelle_update_repos --- CHANGELOG.md | 5 +- gazelle/deps.bzl | 120 +++-------------------------------------------- gazelle/go.mod | 10 ++-- gazelle/go.sum | 82 +++----------------------------- 4 files changed, 21 insertions(+), 196 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 4c9e42c855..b500f1dec9 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -58,7 +58,8 @@ END_UNRELEASED_TEMPLATE {#v0-0-0-changed} ### Changed -* Nothing changed. +* (gazelle) WORKSPACE's bazel-gazelle dependency bumped from 0.36.0 to 0.47.0. + The go version was also bumped from 1.21.13 to 1.22.9. {#v0-0-0-fixed} ### Fixed @@ -109,7 +110,7 @@ Other changes: we will from now on fetch the lists of available packages on each index. The used package mappings will be written as facts to the `MODULE.bazel.lock` file on supported bazel versions and it should be done at most once. As a result, - per-package {obj}`experimental_index_url_overrides` is no longer needed, but + per-package {obj}`experimental_index_url_overrides` is no longer needed, but if specified, it needs to be provided for all packages not on the default index. What is more, the flags for `--index_url` and `--extra-index-url` now behave in the same way as in `uv` or `pip`, i.e. we default to `--index-url` if the package diff --git a/gazelle/deps.bzl b/gazelle/deps.bzl index a7b5990c3b..7072c6a372 100644 --- a/gazelle/deps.bzl +++ b/gazelle/deps.bzl @@ -37,24 +37,18 @@ def gazelle_deps(): def go_deps(): "Fetch go dependencies" - go_repository( - name = "co_honnef_go_tools", - importpath = "honnef.co/go/tools", - sum = "h1:/hemPrYIhOhy8zYrNj+069zDB68us2sMGsfkFJO0iZs=", - version = "v0.0.0-20190523083050-ea95bdfd59fc", - ) go_repository( name = "com_github_bazelbuild_bazel_gazelle", importpath = "github.com/bazelbuild/bazel-gazelle", - sum = "h1:n41ODckCkU9D2BEwBxYN+xu5E92Vd0gaW6QmsIW9l00=", - version = "v0.36.0", + sum = "h1:g3Rr1ZbkC1Pk20aOgBITxSD/efS1WbaSty5jC786Z3Q=", + version = "v0.47.0", ) go_repository( name = "com_github_bazelbuild_buildtools", build_naming_convention = "go_default_library", importpath = "github.com/bazelbuild/buildtools", - sum = "h1:VNqmvOfFzn2Hrtoni8vqgXlIQ4C2Zt22fxeZ9gOOkp0=", - version = "v0.0.0-20240313121412-66c605173954", + sum = "h1:njQAmjTv/YHRm/0Lfv9DXHFZ4MdT2IA/RKHTnqZkgDw=", + version = "v0.0.0-20250930140053-2eb4fccefb52", ) go_repository( name = "com_github_bazelbuild_rules_go", @@ -65,44 +59,8 @@ def go_deps(): go_repository( name = "com_github_bmatcuk_doublestar_v4", importpath = "github.com/bmatcuk/doublestar/v4", - sum = "h1:fdDeAqgT47acgwd9bd9HxJRDmc9UAmPpc+2m0CXv75Q=", - version = "v4.7.1", - ) - go_repository( - name = "com_github_burntsushi_toml", - importpath = "github.com/BurntSushi/toml", - sum = "h1:WXkYYl6Yr3qBf1K79EBnL4mak0OimBfB0XUf9Vl28OQ=", - version = "v0.3.1", - ) - go_repository( - name = "com_github_census_instrumentation_opencensus_proto", - importpath = "github.com/census-instrumentation/opencensus-proto", - sum = "h1:glEXhBS5PSLLv4IXzLA5yPRVX4bilULVyxxbrfOtDAk=", - version = "v0.2.1", - ) - go_repository( - name = "com_github_chzyer_logex", - importpath = "github.com/chzyer/logex", - sum = "h1:Swpa1K6QvQznwJRcfTfQJmTE72DqScAa40E+fbHEXEE=", - version = "v1.1.10", - ) - go_repository( - name = "com_github_chzyer_readline", - importpath = "github.com/chzyer/readline", - sum = "h1:fY5BOSpyZCqRo5OhCuC+XN+r/bBCmeuuJtjz+bCNIf8=", - version = "v0.0.0-20180603132655-2972be24d48e", - ) - go_repository( - name = "com_github_chzyer_test", - importpath = "github.com/chzyer/test", - sum = "h1:q763qf9huN11kDQavWsoZXJNW3xEE4JJyHa5Q25/sd8=", - version = "v0.0.0-20180213035817-a1ea475d72b1", - ) - go_repository( - name = "com_github_client9_misspell", - importpath = "github.com/client9/misspell", - sum = "h1:ta993UF76GwbvJcIo3Y68y/M3WxlpEHPWIGDkJYwzJI=", - version = "v0.3.4", + sum = "h1:X8jg9rRZmJd4yRy7ZeNDRnM+T3ZfHv15JiBJ/avrEXE=", + version = "v4.9.1", ) go_repository( name = "com_github_davecgh_go_spew", @@ -116,18 +74,6 @@ def go_deps(): sum = "h1:FXtiHYKDGKCW2KzwZKx0iC0PQmdlorYgdFG9jPXJ1Bc=", version = "v1.18.1", ) - go_repository( - name = "com_github_envoyproxy_go_control_plane", - importpath = "github.com/envoyproxy/go-control-plane", - sum = "h1:4cmBvAEBNJaGARUEs3/suWRyfyBfhf7I60WBZq+bv2w=", - version = "v0.9.1-0.20191026205805-5f8ba28d4473", - ) - go_repository( - name = "com_github_envoyproxy_protoc_gen_validate", - importpath = "github.com/envoyproxy/protoc-gen-validate", - sum = "h1:EQciDnbrYxy13PgWoY8AqoxGiPrpgBZ1R8UNe3ddc+A=", - version = "v0.1.0", - ) go_repository( name = "com_github_fsnotify_fsnotify", importpath = "github.com/fsnotify/fsnotify", @@ -146,12 +92,6 @@ def go_deps(): sum = "h1:Ov1cvc58UF3b5XjBnZv7+opcTcQFZebYjWzi34vdm4Q=", version = "v1.3.2", ) - go_repository( - name = "com_github_golang_glog", - importpath = "github.com/golang/glog", - sum = "h1:VKtxabqXZkF25pY9ekfRL6a582T4P37/31XEstQ5p58=", - version = "v0.0.0-20160126235308-23def4e6c14b", - ) go_repository( name = "com_github_golang_mock", importpath = "github.com/golang/mock", @@ -176,12 +116,6 @@ def go_deps(): sum = "h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=", version = "v1.0.0", ) - go_repository( - name = "com_github_prometheus_client_model", - importpath = "github.com/prometheus/client_model", - sum = "h1:gQz4mCbXsO+nc9n1hCxHcGA3Zx3Eo+UHZoInFGUIXNM=", - version = "v0.0.0-20190812154241-14fe0d1b01d4", - ) go_repository( name = "com_github_smacker_go_tree_sitter", importpath = "github.com/smacker/go-tree-sitter", @@ -200,12 +134,6 @@ def go_deps(): sum = "h1:HtqpIVDClZ4nwg75+f6Lvsy/wHu+3BoSGCbBAcpTsTg=", version = "v1.9.0", ) - go_repository( - name = "com_google_cloud_go", - importpath = "cloud.google.com/go", - sum = "h1:e0WKqKTd5BnrG8aKH3J3h+QvEIQtSUcf2n5UZ5ZgLtQ=", - version = "v0.26.0", - ) go_repository( name = "in_gopkg_check_v1", importpath = "gopkg.in/check.v1", @@ -230,12 +158,6 @@ def go_deps(): sum = "h1:xwwDQW5We85NaTk2APgoN9202w/l0DVGp+GZMfsrh7s=", version = "v0.0.0-20210223155950-e043a3d3c984", ) - go_repository( - name = "org_golang_google_appengine", - importpath = "google.golang.org/appengine", - sum = "h1:/wp5JvzpHIxhs/dumFmF7BXTf3Z+dd4uXta4kVyO508=", - version = "v1.4.0", - ) go_repository( name = "org_golang_google_genproto", importpath = "google.golang.org/genproto", @@ -266,24 +188,6 @@ def go_deps(): sum = "h1:82DV7MYdb8anAVi3qge1wSnMDrnKK7ebr+I0hHRN1BU=", version = "v1.36.3", ) - go_repository( - name = "org_golang_x_crypto", - importpath = "golang.org/x/crypto", - sum = "h1:VklqNMn3ovrHsnt90PveolxSbWFaJdECFbxSq0Mqo2M=", - version = "v0.0.0-20190308221718-c2843e01d9a2", - ) - go_repository( - name = "org_golang_x_exp", - importpath = "golang.org/x/exp", - sum = "h1:c2HOrn5iMezYjSlGPncknSEr/8x5LELb/ilJbXi9DEA=", - version = "v0.0.0-20190121172915-509febef88a4", - ) - go_repository( - name = "org_golang_x_lint", - importpath = "golang.org/x/lint", - sum = "h1:XQyxROzUlZH+WIQwySDgnISgOivlhjIEwaQaJEJrrN0=", - version = "v0.0.0-20190313153728-d0100b6bd8b3", - ) go_repository( name = "org_golang_x_mod", importpath = "golang.org/x/mod", @@ -296,12 +200,6 @@ def go_deps(): sum = "h1:T5GQRQb2y08kTAByq9L4/bz8cipCdA8FbRTXewonqY8=", version = "v0.35.0", ) - go_repository( - name = "org_golang_x_oauth2", - importpath = "golang.org/x/oauth2", - sum = "h1:vEDujvNQGv4jgYKudGeI/+DAX4Jffq6hpD55MmoEvKs=", - version = "v0.0.0-20180821212333-d2e6202438be", - ) go_repository( name = "org_golang_x_sync", importpath = "golang.org/x/sync", @@ -335,9 +233,3 @@ def go_deps(): sum = "h1:cOIJqWBl99H1dH5LWizPa+0ImeeJq3t3cJjaeOWUAL4=", version = "v0.1.0-deprecated", ) - go_repository( - name = "org_golang_x_xerrors", - importpath = "golang.org/x/xerrors", - sum = "h1:go1bK/D/BFZV2I8cIQd1NKEZ+0owSTG1fDTci4IqFcE=", - version = "v0.0.0-20200804184101-5ec99f83aff1", - ) diff --git a/gazelle/go.mod b/gazelle/go.mod index 7623079af9..9ad4951536 100644 --- a/gazelle/go.mod +++ b/gazelle/go.mod @@ -1,12 +1,14 @@ module github.com/bazel-contrib/rules_python/gazelle -go 1.21.13 +go 1.22.9 + +toolchain go1.24.13 require ( - github.com/bazelbuild/bazel-gazelle v0.36.0 - github.com/bazelbuild/buildtools v0.0.0-20240313121412-66c605173954 + github.com/bazelbuild/bazel-gazelle v0.47.0 + github.com/bazelbuild/buildtools v0.0.0-20250930140053-2eb4fccefb52 github.com/bazelbuild/rules_go v0.55.1 - github.com/bmatcuk/doublestar/v4 v4.7.1 + github.com/bmatcuk/doublestar/v4 v4.9.1 github.com/emirpasic/gods v1.18.1 github.com/ghodss/yaml v1.0.0 github.com/smacker/go-tree-sitter v0.0.0-20240827094217-dd81d9e9be82 diff --git a/gazelle/go.sum b/gazelle/go.sum index 5a4d42d46a..d5fc2b1af9 100644 --- a/gazelle/go.sum +++ b/gazelle/go.sum @@ -1,106 +1,36 @@ -cloud.google.com/go v0.26.0/go.mod h1:aQUYkXzVsufM+DwF1aE+0xfcU+56JwCaLick0ClmMTw= -github.com/BurntSushi/toml v0.3.1/go.mod h1:xHWCNGjB5oqiDr8zfno3MHue2Ht5sIBksp03qcyfWMU= -github.com/bazelbuild/bazel-gazelle v0.36.0 h1:n41ODckCkU9D2BEwBxYN+xu5E92Vd0gaW6QmsIW9l00= -github.com/bazelbuild/bazel-gazelle v0.36.0/go.mod h1:5wGHbkRpDUdz4LxREtPYwXstrWfnkV+oDmOuxNAxW1s= -github.com/bazelbuild/buildtools v0.0.0-20240313121412-66c605173954 h1:VNqmvOfFzn2Hrtoni8vqgXlIQ4C2Zt22fxeZ9gOOkp0= -github.com/bazelbuild/buildtools v0.0.0-20240313121412-66c605173954/go.mod h1:689QdV3hBP7Vo9dJMmzhoYIyo/9iMhEmHkJcnaPRCbo= +github.com/bazelbuild/bazel-gazelle v0.47.0 h1:g3Rr1ZbkC1Pk20aOgBITxSD/efS1WbaSty5jC786Z3Q= +github.com/bazelbuild/bazel-gazelle v0.47.0/go.mod h1:8Ozf20jhv+in87nCUHdmUPPcVGTfKg/gotZ/hce3T+w= +github.com/bazelbuild/buildtools v0.0.0-20250930140053-2eb4fccefb52 h1:njQAmjTv/YHRm/0Lfv9DXHFZ4MdT2IA/RKHTnqZkgDw= +github.com/bazelbuild/buildtools v0.0.0-20250930140053-2eb4fccefb52/go.mod h1:PLNUetjLa77TCCziPsz0EI8a6CUxgC+1jgmWv0H25tg= github.com/bazelbuild/rules_go v0.55.1 h1:cQYGcunY8myOB+0Ym6PGQRhc/milkRcNv0my3XgxaDU= github.com/bazelbuild/rules_go v0.55.1/go.mod h1:T90Gpyq4HDFlsrvtQa2CBdHNJ2P4rAu/uUTmQbanzf0= -github.com/bmatcuk/doublestar/v4 v4.7.1 h1:fdDeAqgT47acgwd9bd9HxJRDmc9UAmPpc+2m0CXv75Q= -github.com/bmatcuk/doublestar/v4 v4.7.1/go.mod h1:xBQ8jztBU6kakFMg+8WGxn0c6z1fTSPVIjEY1Wr7jzc= -github.com/census-instrumentation/opencensus-proto v0.2.1/go.mod h1:f6KPmirojxKA12rnyqOA5BBL4O983OfeGPqjHWSTneU= -github.com/chzyer/logex v1.1.10/go.mod h1:+Ywpsq7O8HXn0nuIou7OrIPyXbp3wmkHB+jjWRnGsAI= -github.com/chzyer/readline v0.0.0-20180603132655-2972be24d48e/go.mod h1:nSuG5e5PlCu98SY8svDHJxuZscDgtXS6KTTbou5AhLI= -github.com/chzyer/test v0.0.0-20180213035817-a1ea475d72b1/go.mod h1:Q3SI9o4m/ZMnBNeIyt5eFwwo7qiLfzFZmjNmxjkiQlU= -github.com/client9/misspell v0.3.4/go.mod h1:qj6jICC3Q7zFZvVWo7KLAzC3yx5G7kyvSDkc90ppPyw= +github.com/bmatcuk/doublestar/v4 v4.9.1 h1:X8jg9rRZmJd4yRy7ZeNDRnM+T3ZfHv15JiBJ/avrEXE= +github.com/bmatcuk/doublestar/v4 v4.9.1/go.mod h1:xBQ8jztBU6kakFMg+8WGxn0c6z1fTSPVIjEY1Wr7jzc= github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c= github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38= github.com/emirpasic/gods v1.18.1 h1:FXtiHYKDGKCW2KzwZKx0iC0PQmdlorYgdFG9jPXJ1Bc= github.com/emirpasic/gods v1.18.1/go.mod h1:8tpGGwCnJ5H4r6BWwaV6OrWmMoPhUl5jm/FMNAnJvWQ= -github.com/envoyproxy/go-control-plane v0.9.1-0.20191026205805-5f8ba28d4473/go.mod h1:YTl/9mNaCwkRvm6d1a2C3ymFceY/DCBVvsKhRF0iEA4= -github.com/envoyproxy/protoc-gen-validate v0.1.0/go.mod h1:iSmxcyjqTsJpI2R4NaDN7+kN2VEUnK/pcBlmesArF7c= github.com/ghodss/yaml v1.0.0 h1:wQHKEahhL6wmXdzwWG11gIVCkOv05bNOh+Rxn0yngAk= github.com/ghodss/yaml v1.0.0/go.mod h1:4dBDuWmgqj2HViK6kFavaiC9ZROes6MMH2rRYeMEF04= -github.com/golang/glog v0.0.0-20160126235308-23def4e6c14b/go.mod h1:SBH7ygxi8pfUlaOkMMuAQtPIUF8ecWP5IEl/CR7VP2Q= -github.com/golang/mock v1.1.1/go.mod h1:oTYuIxOrZwtPieC+H1uAHpcLFnEyAGVDL/k47Jfbm0A= -github.com/golang/protobuf v1.2.0/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U= -github.com/golang/protobuf v1.3.2/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U= -github.com/golang/protobuf v1.4.0-rc.1/go.mod h1:ceaxUfeHdC40wWswd/P6IGgMaK3YpKi5j83Wpe3EHw8= -github.com/golang/protobuf v1.4.0-rc.1.0.20200221234624-67d41d38c208/go.mod h1:xKAWHe0F5eneWXFV3EuXVDTCmh+JuBKY0li0aMyXATA= -github.com/golang/protobuf v1.4.0-rc.2/go.mod h1:LlEzMj4AhA7rCAGe4KMBDvJI+AwstrUpVNzEA03Pprs= -github.com/golang/protobuf v1.4.0-rc.4.0.20200313231945-b860323f09d0/go.mod h1:WU3c8KckQ9AFe+yFwt9sWVRKCVIyN9cPHBJSNnbL67w= -github.com/golang/protobuf v1.4.0/go.mod h1:jodUvKwWbYaEsadDk5Fwe5c77LiNKVO9IDvqG2KuDX0= -github.com/golang/protobuf v1.4.1/go.mod h1:U8fpvMrcmy5pZrNK1lt4xCsGvpyWQ/VVv6QDs8UjoX8= -github.com/golang/protobuf v1.4.3/go.mod h1:oDoupMAO8OvCJWAcko0GGGIgR6R6ocIYbsSw735rRwI= -github.com/google/go-cmp v0.2.0/go.mod h1:oXzfMopK8JAjlY9xF4vHSVASa0yLyX7SntLO5aqRK0M= -github.com/google/go-cmp v0.3.0/go.mod h1:8QqcDgzrUqlUb/G2PQTWiueGozuR1884gddMywk6iLU= -github.com/google/go-cmp v0.3.1/go.mod h1:8QqcDgzrUqlUb/G2PQTWiueGozuR1884gddMywk6iLU= -github.com/google/go-cmp v0.4.0/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE= -github.com/google/go-cmp v0.5.0/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE= -github.com/google/go-cmp v0.5.1/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE= github.com/google/go-cmp v0.6.0 h1:ofyhxvXcZhMsU5ulbFiLKl/XBFqE1GSq7atu8tAmTRI= github.com/google/go-cmp v0.6.0/go.mod h1:17dUlkBOakJ0+DkrSSNjCkIjxS6bF9zb3elmeNGIjoY= github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM= github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4= -github.com/prometheus/client_model v0.0.0-20190812154241-14fe0d1b01d4/go.mod h1:xMI15A0UPsDsEKsMN9yxemIoYk6Tm2C1GtYGdfGttqA= github.com/smacker/go-tree-sitter v0.0.0-20240827094217-dd81d9e9be82 h1:6C8qej6f1bStuePVkLSFxoU22XBS165D3klxlzRg8F4= github.com/smacker/go-tree-sitter v0.0.0-20240827094217-dd81d9e9be82/go.mod h1:xe4pgH49k4SsmkQq5OT8abwhWmnzkhpgnXeekbx2efw= github.com/stretchr/testify v1.9.0 h1:HtqpIVDClZ4nwg75+f6Lvsy/wHu+3BoSGCbBAcpTsTg= github.com/stretchr/testify v1.9.0/go.mod h1:r2ic/lqez/lEtzL7wO/rwa5dbSLXVDPFyf8C91i36aY= -go.starlark.net v0.0.0-20210223155950-e043a3d3c984/go.mod h1:t3mmBBPzAVvK0L0n1drDmrQsJ8FoIx4INCqVMTr/Zo0= -golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w= -golang.org/x/exp v0.0.0-20190121172915-509febef88a4/go.mod h1:CJ0aWSM057203Lf6IL+f9T1iT9GByDxfZKAQTCR3kQA= -golang.org/x/lint v0.0.0-20181026193005-c67002cb31c3/go.mod h1:UVdnD1Gm6xHRNCYTkRU2/jEulfH38KcIWyp/GAMgvoE= -golang.org/x/lint v0.0.0-20190227174305-5b3e6a55c961/go.mod h1:wehouNa3lNwaWXcvxsM5YxQ5yQlVC4a0KAMCusXpPoU= -golang.org/x/lint v0.0.0-20190313153728-d0100b6bd8b3/go.mod h1:6SW0HCj/g11FgYtHlgUYUwCkIfeOF89ocIRzGO/8vkc= golang.org/x/mod v0.23.0 h1:Zb7khfcRGKk+kqfxFaP5tZqCnDZMjC5VtUBs87Hr6QM= golang.org/x/mod v0.23.0/go.mod h1:6SkKJ3Xj0I0BrPOZoBy3bdMptDDU9oJrpohJ3eWZ1fY= -golang.org/x/net v0.0.0-20180724234803-3673e40ba225/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4= -golang.org/x/net v0.0.0-20180826012351-8a410e7b638d/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4= -golang.org/x/net v0.0.0-20190213061140-3a22650c66bd/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4= -golang.org/x/net v0.0.0-20190311183353-d8887717615a/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg= -golang.org/x/oauth2 v0.0.0-20180821212333-d2e6202438be/go.mod h1:N/0e6XlmueqKjAGxoOufVs8QHGRruUQn6yWY3a++T0U= -golang.org/x/sync v0.0.0-20180314180146-1d60e4601c6f/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= -golang.org/x/sync v0.0.0-20181108010431-42b317875d0f/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= -golang.org/x/sync v0.0.0-20190423024810-112230192c58/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= golang.org/x/sync v0.11.0 h1:GGz8+XQP4FvTTrjZPzNKTMFtSXH80RAzG+5ghFPgK9w= golang.org/x/sync v0.11.0/go.mod h1:Czt+wKu1gCyEFDUtn0jG5QVvpJ6rzVqr5aXyt9drQfk= -golang.org/x/sys v0.0.0-20180830151530-49385e6e1522/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= -golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= -golang.org/x/sys v0.0.0-20200930185726-fdedc70b468f/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.30.0 h1:QjkSwP/36a20jFYWkSue1YwXzLmsV5Gfq7Eiy72C1uc= golang.org/x/sys v0.30.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA= -golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ= -golang.org/x/tools v0.0.0-20190114222345-bf090417da8b/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ= -golang.org/x/tools v0.0.0-20190226205152-f727befe758c/go.mod h1:9Yl7xja0Znq3iFh3HoIrodX9oNMXvdceNzlUR8zjMvY= -golang.org/x/tools v0.0.0-20190311212946-11955173bddd/go.mod h1:LCzVGOaR6xXOjkQ3onu1FJEFr0SW1gC7cKk1uF8kGRs= -golang.org/x/tools v0.0.0-20190524140312-2c0ae7006135/go.mod h1:RgjU9mgBXZiqYHBnxXauZ1Gv1EHHAz9KjViQ78xBX0Q= golang.org/x/tools/go/vcs v0.1.0-deprecated h1:cOIJqWBl99H1dH5LWizPa+0ImeeJq3t3cJjaeOWUAL4= golang.org/x/tools/go/vcs v0.1.0-deprecated/go.mod h1:zUrvATBAvEI9535oC0yWYsLsHIV4Z7g63sNPVMtuBy8= -golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0= -golang.org/x/xerrors v0.0.0-20200804184101-5ec99f83aff1/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0= -google.golang.org/appengine v1.1.0/go.mod h1:EbEs0AVv82hx2wNQdGPgUI5lhzA/G0D9YwlJXL52JkM= -google.golang.org/appengine v1.4.0/go.mod h1:xpcJRLb0r/rnEns0DIKYYv+WjYCduHsrkT7/EB5XEv4= -google.golang.org/genproto v0.0.0-20180817151627-c66870c02cf8/go.mod h1:JiN7NxoALGmiZfu7CAH4rXhgtRTLTxftemlI0sWmxmc= -google.golang.org/genproto v0.0.0-20190819201941-24fa4b261c55/go.mod h1:DMBHOl98Agz4BDEuKkezgsaosCRResVns1a3J2ZsMNc= -google.golang.org/genproto v0.0.0-20200526211855-cb27e3aa2013/go.mod h1:NbSheEEYHJ7i3ixzK3sjbqSGDJWnxyFXZblF3eUsNvo= -google.golang.org/grpc v1.19.0/go.mod h1:mqu4LbDTu4XGKhr4mRzUsmM4RtVoemTSY81AxZiDr8c= -google.golang.org/grpc v1.23.0/go.mod h1:Y5yQAOtifL1yxbo5wqy6BxZv8vAUGQwXBOALyacEbxg= -google.golang.org/grpc v1.27.0/go.mod h1:qbnxyOmOxrQa7FizSgH+ReBfzJrCY1pSN7KXBS8abTk= -google.golang.org/protobuf v0.0.0-20200109180630-ec00e32a8dfd/go.mod h1:DFci5gLYBciE7Vtevhsrf46CRTquxDuWsQurQQe4oz8= -google.golang.org/protobuf v0.0.0-20200221191635-4d8936d0db64/go.mod h1:kwYJMbMJ01Woi6D6+Kah6886xMZcty6N08ah7+eCXa0= -google.golang.org/protobuf v0.0.0-20200228230310-ab0ca4ff8a60/go.mod h1:cfTl7dwQJ+fmap5saPgwCLgHXTUD7jkjRqWcaiX5VyM= -google.golang.org/protobuf v1.20.1-0.20200309200217-e05f789c0967/go.mod h1:A+miEFZTKqfCUM6K7xSMQL9OKL/b6hQv+e19PK+JZNE= -google.golang.org/protobuf v1.21.0/go.mod h1:47Nbq4nVaFHyn7ilMalzfO3qCViNmqZ2kzikPIcrTAo= -google.golang.org/protobuf v1.22.0/go.mod h1:EGpADcykh3NcUnDUJcl1+ZksZNG86OlYog2l/sGQquU= -google.golang.org/protobuf v1.23.0/go.mod h1:EGpADcykh3NcUnDUJcl1+ZksZNG86OlYog2l/sGQquU= -google.golang.org/protobuf v1.23.1-0.20200526195155-81db48ad09cc/go.mod h1:EGpADcykh3NcUnDUJcl1+ZksZNG86OlYog2l/sGQquU= -google.golang.org/protobuf v1.25.0/go.mod h1:9JNX74DMeImyA3h4bdi1ymwjUzf21/xIlbajtzgsN7c= gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405 h1:yhCVgyC4o1eVCa2tZl7eS0r+SDo693bJlVdllGtEeKM= gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0= gopkg.in/yaml.v2 v2.4.0 h1:D8xgwECY7CYvx+Y2n4sBz93Jn9JRvxdiyyo8CTfuKaY= gopkg.in/yaml.v2 v2.4.0/go.mod h1:RDklbk79AGWmwhnvt/jBztapEOGDOx6ZbXqjP6csGnQ= gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA= gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM= -honnef.co/go/tools v0.0.0-20190102054323-c2f93a96b099/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4= -honnef.co/go/tools v0.0.0-20190523083050-ea95bdfd59fc/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4= From 6da14312e9cab843e5e801d4af5d855faaa1ac15 Mon Sep 17 00:00:00 2001 From: Ignas Anikevicius <240938+aignas@users.noreply.github.com> Date: Tue, 21 Apr 2026 00:42:11 +0900 Subject: [PATCH 704/922] fix(pypi): correctly write the used facts back (#3719) Summary: 1. Added 3 new tests to pypi_cache_tests.bzl 2. Fixed `_cache` helper to handle both struct and dict return values 3. Fixed `_filter_packages`: Returns None for empty dict results instead of empty dict 4. Fixed `_get_from_facts`: Now stores facts when reading index_urls from the dict case Fixes #3711 Fixes #3707 --------- Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> --- python/private/pypi/pypi_cache.bzl | 8 +- tests/pypi/pypi_cache/pypi_cache_tests.bzl | 98 ++++++++++++++++++++++ 2 files changed, 104 insertions(+), 2 deletions(-) diff --git a/python/private/pypi/pypi_cache.bzl b/python/private/pypi/pypi_cache.bzl index 7b24102263..972bbe04ae 100644 --- a/python/private/pypi/pypi_cache.bzl +++ b/python/private/pypi/pypi_cache.bzl @@ -128,11 +128,12 @@ def _filter_packages(dists, requested_versions): return dists if type(dists) == "dict": - return { + result = { pkg: url for pkg, url in dists.items() if pkg in requested_versions } + return result if result else None sha256s_by_version = {} whls = {} @@ -206,10 +207,13 @@ def _get_from_facts(facts, known_facts, index_url, requested_versions, facts_ver return None if type(requested_versions) == "dict": - return _filter_packages( + result = _filter_packages( dists = known_facts.get("index_urls", {}).get(index_url, {}), requested_versions = requested_versions, ) + if result: + _store_facts(facts, facts_version, index_url, result) + return result known_sources = {} diff --git a/tests/pypi/pypi_cache/pypi_cache_tests.bzl b/tests/pypi/pypi_cache/pypi_cache_tests.bzl index 59ef661ab7..89ed5693e2 100644 --- a/tests/pypi/pypi_cache/pypi_cache_tests.bzl +++ b/tests/pypi/pypi_cache/pypi_cache_tests.bzl @@ -20,6 +20,9 @@ def _cache(env, **kwargs): if not value: return env.expect.that_str(value) + if type(value) == "dict": + return env.expect.that_dict(value) + return env.expect.that_struct( value, attrs = attrs, @@ -266,6 +269,101 @@ def _test_pypi_cache_reads_from_facts(env): _tests.append(_test_pypi_cache_reads_from_facts) +def _test_memory_cache_index_urls(env): + """Verifies that the cache returns stored values for index_urls.""" + store = {} + cache = _cache(env, mctx = None, store = store) + + fake_result = { + "pkg-a": "https://pypi.org/simple/pkg-a/", + "pkg_b": "https://pypi.org/simple/pkg-b/", + } + + key = ("https://pypi.org/simple/", "https://pypi.org/simple/", {"pkg-a": None, "pkg_b": None}) + + cache.setdefault(key, fake_result) + + got = cache.get(key) + got.contains_exactly(fake_result) + + key = ("https://pypi.org/simple/", "https://pypi.org/simple/", {"pkg-a": None}) + got = cache.get(key) + got.contains_exactly({"pkg-a": "https://pypi.org/simple/pkg-a/"}) + + key = ("https://pypi.org/simple/", "https://pypi.org/simple/", {"pkg-c": None}) + cache.get(key).equals(None) + +_tests.append(_test_memory_cache_index_urls) + +def _test_pypi_cache_writes_index_urls_to_facts(env): + """Verifies that setting index_urls in the cache also populates the facts store.""" + mock_ctx = mocks.mctx(facts = {}) + cache = _cache(env, mctx = mock_ctx) + + fake_result = { + "pkg-a": "https://pypi.org/simple/pkg-a/", + "pkg_b": "https://pypi.org/simple/pkg-b/", + } + + key = ("https://pypi.org/simple/", "https://pypi.org/simple/", {"pkg-a": None}) + + cache.setdefault(key, fake_result) + + cache.get_facts().contains_exactly({ + "fact_version": "v1", + "index_urls": { + "https://pypi.org/simple/": { + "pkg-a": "https://pypi.org/simple/pkg-a/", + }, + }, + }) + + key = ("https://pypi.org/simple/", "https://pypi.org/simple/", {"pkg_b": None}) + cache.setdefault(key, fake_result) + + cache.get_facts().contains_exactly({ + "fact_version": "v1", + "index_urls": { + "https://pypi.org/simple/": { + "pkg-a": "https://pypi.org/simple/pkg-a/", + "pkg_b": "https://pypi.org/simple/pkg-b/", + }, + }, + }) + +_tests.append(_test_pypi_cache_writes_index_urls_to_facts) + +def _test_pypi_cache_reads_index_urls_from_facts(env): + """Verifies that reading index_urls from facts works correctly.""" + mock_ctx = mocks.mctx(facts = { + "fact_version": "v1", + "index_urls": { + "https://pypi.org/simple/": { + "pkg-a": "https://pypi.org/simple/pkg-a/", + "pkg-b": "https://pypi.org/simple/pkg-b/", + }, + }, + }) + cache = _cache(env, mctx = mock_ctx) + + key = ("https://pypi.org/simple/", "https://pypi.org/simple/", {"pkg-a": None}) + got = cache.get(key) + got.contains_exactly({"pkg-a": "https://pypi.org/simple/pkg-a/"}) + + key = ("https://pypi.org/simple/", "https://pypi.org/simple/", {"pkg-a": None, "pkg-b": None}) + got = cache.get(key) + got.contains_exactly({ + "pkg-a": "https://pypi.org/simple/pkg-a/", + "pkg-b": "https://pypi.org/simple/pkg-b/", + }) + + key = ("https://pypi.org/simple/", "https://pypi.org/simple/", {"pkg-c": None}) + cache.get(key).equals(None) + + cache.get_facts().contains_exactly(mock_ctx.facts) + +_tests.append(_test_pypi_cache_reads_index_urls_from_facts) + def pypi_cache_test_suite(name): test_suite( name = name, From 19a2c0439e429fd673541360d7720d658e0b30db Mon Sep 17 00:00:00 2001 From: Ignas Anikevicius <240938+aignas@users.noreply.github.com> Date: Tue, 21 Apr 2026 08:17:14 +0900 Subject: [PATCH 705/922] fix(pypi): build the environment on the fly (#3720) At some point when we started using pipstar, we silently started building the python environment with system python interpreter. This is fine because the environment in those code paths would be unused, but this would break if python was not present on the machine. This PR fixes this by creating the environment on the fly with a little bit of code duplication. Fixes #3712 --------- Co-authored-by: Richard Levasseur --- CHANGELOG.md | 2 ++ python/private/pypi/whl_library.bzl | 11 ++++++++--- 2 files changed, 10 insertions(+), 3 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index b500f1dec9..78ab9e98bf 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -156,6 +156,8 @@ Other changes: ) ``` Fixes [#3676](https://github.com/bazel-contrib/rules_python/issues/3676). +* (pypi) Fixes wheel extraction on hosts without python installed, + Fixes [#3712](https://github.com/bazel-contrib/rules_python/issues/3712). {#v2-0-0-added} ### Added diff --git a/python/private/pypi/whl_library.bzl b/python/private/pypi/whl_library.bzl index 31da3b94cb..5db4f12d67 100644 --- a/python/private/pypi/whl_library.bzl +++ b/python/private/pypi/whl_library.bzl @@ -306,9 +306,6 @@ def _whl_library_impl(rctx): extra_pip_args = [] extra_pip_args.extend(rctx.attr.extra_pip_args) - # Manually construct the PYTHONPATH since we cannot use the toolchain here - environment = _create_repository_execution_environment(rctx, python_interpreter, logger = logger) - whl_path = None sdist_filename = None if rctx.attr.whl_file: @@ -361,6 +358,14 @@ def _whl_library_impl(rctx): enable_pipstar = (rp_config.enable_pipstar or whl_path) and rctx.attr.config_load enable_pipstar_extract = enable_pipstar and rp_config.bazel_8_or_later + # When pipstar is enabled, Python isn't used, so there's no need + # to setup env vars to run Python, unless we need to build an sdist + if enable_pipstar_extract and whl_path: + environment = {} + else: + # Manually construct the PYTHONPATH since we cannot use the toolchain here + environment = _create_repository_execution_environment(rctx, python_interpreter, logger = logger) + if not whl_path: if rctx.attr.urls: op_tmpl = "whl_library.BuildWheelFromSource({name}, {requirement})" From 547521ed6ad1d72c0d297d43c6054ea05027bbcd Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Mon, 20 Apr 2026 19:55:10 -0700 Subject: [PATCH 706/922] feat(windows): site-packages venv support (#3718) This implements Windows support for library files being populated in the venv site-packages directory (i.e. `--venv_site_packages=yes`). This is mostly just an expansion of the work done to make venvs work on Windows. It mostly just adds the site-packages paths to the list of (venv_path, symlinks) values that is currently used for the interpreter files. The main notable change is that creation of the site-packages directory also requires traversing the whole site-packages directory and re-creating its structure in the temporary venv. This isn't ideal, but there isn't much other option. Work towards https://github.com/bazel-contrib/rules_python/issues/3245 --- CHANGELOG.md | 2 + docs/conf.py | 7 - docs/pyproject.toml | 1 + docs/requirements.txt | 443 +++++++++++------- python/private/attributes.bzl | 8 + python/private/common.bzl | 36 ++ python/private/py_executable.bzl | 72 ++- python/private/py_executable_info.bzl | 13 + python/private/python_bootstrap_template.txt | 73 ++- python/private/site_init_template.py | 19 +- python/private/stage2_bootstrap_template.py | 89 ++-- python/private/venv_runfiles.bzl | 22 +- python/private/zipapp/py_zipapp_rule.bzl | 23 +- python/private/zipapp/zip_main_template.py | 30 +- tests/py_zipapp/BUILD.bazel | 33 +- tests/py_zipapp/main.py | 13 +- .../site-packages/pkgdep/__init__.py | 0 .../py_zipapp/site-packages/pkgdep/pkgmod.py | 0 tests/py_zipapp/system_python_zipapp_test.py | 15 +- tests/py_zipapp/venv_zipapp_test.py | 30 +- tests/venv_site_packages_libs/BUILD.bazel | 28 +- tests/venv_site_packages_libs/bin.py | 6 +- .../shared_lib_loading_test.py | 61 ++- 23 files changed, 674 insertions(+), 350 deletions(-) create mode 100644 tests/py_zipapp/site-packages/pkgdep/__init__.py create mode 100644 tests/py_zipapp/site-packages/pkgdep/pkgmod.py diff --git a/CHANGELOG.md b/CHANGELOG.md index 78ab9e98bf..272b74db06 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -67,6 +67,8 @@ END_UNRELEASED_TEMPLATE {#v0-0-0-added} ### Added +* (windows) Full venv support for Windows is available. Set + {obj}`--venvs_site_packages=yes` to enable. * (runfiles) Added a pathlib-compatible API: {obj}`Runfiles.root()` Fixes [#3296](https://github.com/bazel-contrib/rules_python/issues/3296). * (toolchains) `3.13.12`, `3.14.3` Python toolchain from [20260325] release. diff --git a/docs/conf.py b/docs/conf.py index 541d99ef7a..ef7b66acfa 100644 --- a/docs/conf.py +++ b/docs/conf.py @@ -95,13 +95,6 @@ "pypi-dependencies.html": "pypi/index.html", } -# Adapted from the template code: -# https://github.com/readthedocs/readthedocs.org/blob/main/readthedocs/doc_builder/templates/doc_builder/conf.py.tmpl -if os.environ.get("READTHEDOCS") == "True": - # Must come first because it can interfere with other extensions, according - # to the original conf.py template comments - extensions.insert(0, "readthedocs_ext.readthedocs") - exclude_patterns = ["_includes/*"] templates_path = ["_templates"] primary_domain = None # The default is 'py', which we don't make much use of diff --git a/docs/pyproject.toml b/docs/pyproject.toml index f4bbbaf35a..f0ca3928ff 100644 --- a/docs/pyproject.toml +++ b/docs/pyproject.toml @@ -16,4 +16,5 @@ dependencies = [ "pefile", "pyelftools", "macholib", + "markupsafe", ] diff --git a/docs/requirements.txt b/docs/requirements.txt index aee3e0a0a3..6397f0a7f1 100644 --- a/docs/requirements.txt +++ b/docs/requirements.txt @@ -2,10 +2,14 @@ # bazel run //docs:requirements.update --index-url https://pypi.org/simple -absl-py==2.3.1 \ +absl-py==2.3.1 ; python_full_version < '3.10' \ --hash=sha256:a97820526f7fbfd2ec1bce83f3f25e3a14840dac0d8e02a0b71cd75db3f77fc9 \ --hash=sha256:eeecf07f0c2a93ace0772c92e596ace6d3d3996c042b2128459aaae2a76de11d # via rules-python-docs (docs/pyproject.toml) +absl-py==2.4.0 ; python_full_version >= '3.10' \ + --hash=sha256:88476fd881ca8aab94ffa78b7b6c632a782ab3ba1cd19c9bd423abc4fb4cd28d \ + --hash=sha256:8c6af82722b35cf71e0f4d1d47dcaebfff286e27110a99fc359349b247dfb5d4 + # via rules-python-docs (docs/pyproject.toml) alabaster==0.7.16 ; python_full_version < '3.10' \ --hash=sha256:75a8b99c28a5dad50dd7f8ccdd447a121ddb3892da9e53d1ca5cca3106d58d65 \ --hash=sha256:b46733c07dce03ae4e150330b975c75737fa60f0a7c591b6c8bf4928a28e2c92 @@ -14,125 +18,186 @@ alabaster==1.0.0 ; python_full_version >= '3.10' \ --hash=sha256:c00dca57bca26fa62a6d7d0a9fcce65f3e026e9bfe33e9c538fd3fbb2144fd9e \ --hash=sha256:fc6786402dc3fcb2de3cabd5fe455a2db534b371124f1f21de8731783dec828b # via sphinx -altgraph==0.17.4 \ - --hash=sha256:1b5afbb98f6c4dcadb2e2ae6ab9fa994bbb8c1d75f4fa96d340f9437ae454406 \ - --hash=sha256:642743b4750de17e655e6711601b077bc6598dbfa3ba5fa2b2a35ce12b508dff +altgraph==0.17.5 \ + --hash=sha256:c87b395dd12fabde9c99573a9749d67da8d29ef9de0125c7f536699b4a9bc9e7 \ + --hash=sha256:f3a22400bce1b0c701683820ac4f3b159cd301acab067c51c653e06961600597 # via macholib astroid==3.3.11 \ --hash=sha256:1e5a5011af2920c7c67a53f65d536d65bfa7116feeaf2354d8b94f29573bb0ce \ --hash=sha256:54c760ae8322ece1abd213057c4b5bba7c49818853fc901ef09719a60dbf9dec # via sphinx-autodoc2 -babel==2.17.0 \ - --hash=sha256:0c54cffb19f690cdcc52a3b50bcbf71e07a808d1c80d549f2459b9d2cf0afb9d \ - --hash=sha256:4d0b53093fdfb4b21c92b5213dba5a1b23885afa8383709427046b21c366e5f2 +babel==2.18.0 \ + --hash=sha256:b80b99a14bd085fcacfa15c9165f651fbb3406e66cc603abf11c5750937c992d \ + --hash=sha256:e2b422b277c2b9a9630c1d7903c2a00d0830c409c59ac8cae9081c92f1aeba35 # via sphinx -certifi==2025.10.5 \ - --hash=sha256:0f212c2744a9bb6de0c56639a6f68afe01ecd92d91f14ae897c4fe7bbeeef0de \ - --hash=sha256:47c09d31ccf2acf0be3f701ea53595ee7e0b8fa08801c6624be771df09ae7b43 +certifi==2026.2.25 \ + --hash=sha256:027692e4402ad994f1c42e52a4997a9763c646b73e4096e4d5d6db8af1d6f0fa \ + --hash=sha256:e887ab5cee78ea814d3472169153c2d12cd43b14bd03329a39a9c6e2e80bfba7 # via requests -charset-normalizer==3.4.3 \ - --hash=sha256:00237675befef519d9af72169d8604a067d92755e84fe76492fef5441db05b91 \ - --hash=sha256:02425242e96bcf29a49711b0ca9f37e451da7c70562bc10e8ed992a5a7a25cc0 \ - --hash=sha256:027b776c26d38b7f15b26a5da1044f376455fb3766df8fc38563b4efbc515154 \ - --hash=sha256:07a0eae9e2787b586e129fdcbe1af6997f8d0e5abaa0bc98c0e20e124d67e601 \ - --hash=sha256:0cacf8f7297b0c4fcb74227692ca46b4a5852f8f4f24b3c766dd94a1075c4884 \ - --hash=sha256:0e78314bdc32fa80696f72fa16dc61168fda4d6a0c014e0380f9d02f0e5d8a07 \ - --hash=sha256:0f2be7e0cf7754b9a30eb01f4295cc3d4358a479843b31f328afd210e2c7598c \ - --hash=sha256:13faeacfe61784e2559e690fc53fa4c5ae97c6fcedb8eb6fb8d0a15b475d2c64 \ - --hash=sha256:14c2a87c65b351109f6abfc424cab3927b3bdece6f706e4d12faaf3d52ee5efe \ - --hash=sha256:1606f4a55c0fd363d754049cdf400175ee96c992b1f8018b993941f221221c5f \ - --hash=sha256:16a8770207946ac75703458e2c743631c79c59c5890c80011d536248f8eaa432 \ - --hash=sha256:18343b2d246dc6761a249ba1fb13f9ee9a2bcd95decc767319506056ea4ad4dc \ - --hash=sha256:18b97b8404387b96cdbd30ad660f6407799126d26a39ca65729162fd810a99aa \ - --hash=sha256:1bb60174149316da1c35fa5233681f7c0f9f514509b8e399ab70fea5f17e45c9 \ - --hash=sha256:1e8ac75d72fa3775e0b7cb7e4629cec13b7514d928d15ef8ea06bca03ef01cae \ - --hash=sha256:1ef99f0456d3d46a50945c98de1774da86f8e992ab5c77865ea8b8195341fc19 \ - --hash=sha256:2001a39612b241dae17b4687898843f254f8748b796a2e16f1051a17078d991d \ - --hash=sha256:23b6b24d74478dc833444cbd927c338349d6ae852ba53a0d02a2de1fce45b96e \ - --hash=sha256:252098c8c7a873e17dd696ed98bbe91dbacd571da4b87df3736768efa7a792e4 \ - --hash=sha256:257f26fed7d7ff59921b78244f3cd93ed2af1800ff048c33f624c87475819dd7 \ - --hash=sha256:2c322db9c8c89009a990ef07c3bcc9f011a3269bc06782f916cd3d9eed7c9312 \ - --hash=sha256:30a96e1e1f865f78b030d65241c1ee850cdf422d869e9028e2fc1d5e4db73b92 \ - --hash=sha256:30d006f98569de3459c2fc1f2acde170b7b2bd265dc1943e87e1a4efe1b67c31 \ - --hash=sha256:31a9a6f775f9bcd865d88ee350f0ffb0e25936a7f930ca98995c05abf1faf21c \ - --hash=sha256:320e8e66157cc4e247d9ddca8e21f427efc7a04bbd0ac8a9faf56583fa543f9f \ - --hash=sha256:34a7f768e3f985abdb42841e20e17b330ad3aaf4bb7e7aeeb73db2e70f077b99 \ - --hash=sha256:3653fad4fe3ed447a596ae8638b437f827234f01a8cd801842e43f3d0a6b281b \ - --hash=sha256:3cd35b7e8aedeb9e34c41385fda4f73ba609e561faedfae0a9e75e44ac558a15 \ - --hash=sha256:3cfb2aad70f2c6debfbcb717f23b7eb55febc0bb23dcffc0f076009da10c6392 \ - --hash=sha256:416175faf02e4b0810f1f38bcb54682878a4af94059a1cd63b8747244420801f \ - --hash=sha256:41d1fc408ff5fdfb910200ec0e74abc40387bccb3252f3f27c0676731df2b2c8 \ - --hash=sha256:42e5088973e56e31e4fa58eb6bd709e42fc03799c11c42929592889a2e54c491 \ - --hash=sha256:4ca4c094de7771a98d7fbd67d9e5dbf1eb73efa4f744a730437d8a3a5cf994f0 \ - --hash=sha256:511729f456829ef86ac41ca78c63a5cb55240ed23b4b737faca0eb1abb1c41bc \ - --hash=sha256:53cd68b185d98dde4ad8990e56a58dea83a4162161b1ea9272e5c9182ce415e0 \ - --hash=sha256:585f3b2a80fbd26b048a0be90c5aae8f06605d3c92615911c3a2b03a8a3b796f \ - --hash=sha256:5b413b0b1bfd94dbf4023ad6945889f374cd24e3f62de58d6bb102c4d9ae534a \ - --hash=sha256:5d8d01eac18c423815ed4f4a2ec3b439d654e55ee4ad610e153cf02faf67ea40 \ - --hash=sha256:6aab0f181c486f973bc7262a97f5aca3ee7e1437011ef0c2ec04b5a11d16c927 \ - --hash=sha256:6cf8fd4c04756b6b60146d98cd8a77d0cdae0e1ca20329da2ac85eed779b6849 \ - --hash=sha256:6fb70de56f1859a3f71261cbe41005f56a7842cc348d3aeb26237560bfa5e0ce \ - --hash=sha256:6fce4b8500244f6fcb71465d4a4930d132ba9ab8e71a7859e6a5d59851068d14 \ - --hash=sha256:70bfc5f2c318afece2f5838ea5e4c3febada0be750fcf4775641052bbba14d05 \ - --hash=sha256:73dc19b562516fc9bcf6e5d6e596df0b4eb98d87e4f79f3ae71840e6ed21361c \ - --hash=sha256:74d77e25adda8581ffc1c720f1c81ca082921329452eba58b16233ab1842141c \ - --hash=sha256:78deba4d8f9590fe4dae384aeff04082510a709957e968753ff3c48399f6f92a \ - --hash=sha256:86df271bf921c2ee3818f0522e9a5b8092ca2ad8b065ece5d7d9d0e9f4849bcc \ - --hash=sha256:88ab34806dea0671532d3f82d82b85e8fc23d7b2dd12fa837978dad9bb392a34 \ - --hash=sha256:8999f965f922ae054125286faf9f11bc6932184b93011d138925a1773830bbe9 \ - --hash=sha256:8dcfc373f888e4fb39a7bc57e93e3b845e7f462dacc008d9749568b1c4ece096 \ - --hash=sha256:939578d9d8fd4299220161fdd76e86c6a251987476f5243e8864a7844476ba14 \ - --hash=sha256:96b2b3d1a83ad55310de8c7b4a2d04d9277d5591f40761274856635acc5fcb30 \ - --hash=sha256:a2d08ac246bb48479170408d6c19f6385fa743e7157d716e144cad849b2dd94b \ - --hash=sha256:b256ee2e749283ef3ddcff51a675ff43798d92d746d1a6e4631bf8c707d22d0b \ - --hash=sha256:b5e3b2d152e74e100a9e9573837aba24aab611d39428ded46f4e4022ea7d1942 \ - --hash=sha256:b89bc04de1d83006373429975f8ef9e7932534b8cc9ca582e4db7d20d91816db \ - --hash=sha256:bd28b817ea8c70215401f657edef3a8aa83c29d447fb0b622c35403780ba11d5 \ - --hash=sha256:c60e092517a73c632ec38e290eba714e9627abe9d301c8c8a12ec32c314a2a4b \ - --hash=sha256:c6dbd0ccdda3a2ba7c2ecd9d77b37f3b5831687d8dc1b6ca5f56a4880cc7b7ce \ - --hash=sha256:c6e490913a46fa054e03699c70019ab869e990270597018cef1d8562132c2669 \ - --hash=sha256:c6f162aabe9a91a309510d74eeb6507fab5fff92337a15acbe77753d88d9dcf0 \ - --hash=sha256:c6fd51128a41297f5409deab284fecbe5305ebd7e5a1f959bee1c054622b7018 \ - --hash=sha256:cc34f233c9e71701040d772aa7490318673aa7164a0efe3172b2981218c26d93 \ - --hash=sha256:cc9370a2da1ac13f0153780040f465839e6cccb4a1e44810124b4e22483c93fe \ - --hash=sha256:ccf600859c183d70eb47e05a44cd80a4ce77394d1ac0f79dbd2dd90a69a3a049 \ - --hash=sha256:ce571ab16d890d23b5c278547ba694193a45011ff86a9162a71307ed9f86759a \ - --hash=sha256:cf1ebb7d78e1ad8ec2a8c4732c7be2e736f6e5123a4146c5b89c9d1f585f8cef \ - --hash=sha256:d0e909868420b7049dafd3a31d45125b31143eec59235311fc4c57ea26a4acd2 \ - --hash=sha256:d22dbedd33326a4a5190dd4fe9e9e693ef12160c77382d9e87919bce54f3d4ca \ - --hash=sha256:d716a916938e03231e86e43782ca7878fb602a125a91e7acb8b5112e2e96ac16 \ - --hash=sha256:d79c198e27580c8e958906f803e63cddb77653731be08851c7df0b1a14a8fc0f \ - --hash=sha256:d95bfb53c211b57198bb91c46dd5a2d8018b3af446583aab40074bf7988401cb \ - --hash=sha256:e28e334d3ff134e88989d90ba04b47d84382a828c061d0d1027b1b12a62b39b1 \ - --hash=sha256:ec557499516fc90fd374bf2e32349a2887a876fbf162c160e3c01b6849eaf557 \ - --hash=sha256:fb6fecfd65564f208cbf0fba07f107fb661bcd1a7c389edbced3f7a493f70e37 \ - --hash=sha256:fb731e5deb0c7ef82d698b0f4c5bb724633ee2a489401594c5c88b02e6cb15f7 \ - --hash=sha256:fb7f67a1bfa6e40b438170ebdc8158b78dc465a5a67b6dde178a46987b244a72 \ - --hash=sha256:fd10de089bcdcd1be95a2f73dbe6254798ec1bda9f450d5828c96f93e2536b9c \ - --hash=sha256:fdabf8315679312cfa71302f9bd509ded4f2f263fb5b765cf1433b39106c3cc9 +charset-normalizer==3.4.7 \ + --hash=sha256:007d05ec7321d12a40227aae9e2bc6dca73f3cb21058999a1df9e193555a9dcc \ + --hash=sha256:03853ed82eeebbce3c2abfdbc98c96dc205f32a79627688ac9a27370ea61a49c \ + --hash=sha256:07d9e39b01743c3717745f4c530a6349eadbfa043c7577eef86c502c15df2c67 \ + --hash=sha256:08e721811161356f97b4059a9ba7bafb23ea5ee2255402c42881c214e173c6b4 \ + --hash=sha256:0c96c3b819b5c3e9e165495db84d41914d6894d55181d2d108cc1a69bfc9cce0 \ + --hash=sha256:0ea948db76d31190bf08bd371623927ee1339d5f2a0b4b1b4a4439a65298703c \ + --hash=sha256:0f7eb884681e3938906ed0434f20c63046eacd0111c4ba96f27b76084cd679f5 \ + --hash=sha256:12a6fff75f6bc66711b73a2f0addfc4c8c15a20e805146a02d147a318962c444 \ + --hash=sha256:12d8baf840cc7889b37c7c770f478adea7adce3dcb3944d02ec87508e2dcf153 \ + --hash=sha256:14265bfe1f09498b9d8ec91e9ec9fa52775edf90fcbde092b25f4a33d444fea9 \ + --hash=sha256:16d971e29578a5e97d7117866d15889a4a07befe0e87e703ed63cd90cb348c01 \ + --hash=sha256:177a0ba5f0211d488e295aaf82707237e331c24788d8d76c96c5a41594723217 \ + --hash=sha256:1a87ca9d5df6fe460483d9a5bbf2b18f620cbed41b432e2bddb686228282d10b \ + --hash=sha256:1c2a768fdd44ee4a9339a9b0b130049139b8ce3c01d2ce09f67f5a68048d477c \ + --hash=sha256:1c2aed2e5e41f24ea8ef1590b8e848a79b56f3a5564a65ceec43c9d692dc7d8a \ + --hash=sha256:1dc8b0ea451d6e69735094606991f32867807881400f808a106ee1d963c46a83 \ + --hash=sha256:1efde3cae86c8c273f1eb3b287be7d8499420cf2fe7585c41d370d3e790054a5 \ + --hash=sha256:202389074300232baeb53ae2569a60901f7efadd4245cf3a3bf0617d60b439d7 \ + --hash=sha256:203104ed3e428044fd943bc4bf45fa73c0730391f9621e37fe39ecf477b128cb \ + --hash=sha256:2257141f39fe65a3fdf38aeccae4b953e5f3b3324f4ff0daf9f15b8518666a2c \ + --hash=sha256:298930cec56029e05497a76988377cbd7457ba864beeea92ad7e844fe74cd1f1 \ + --hash=sha256:2cd4a60d0e2fb04537162c62bbbb4182f53541fe0ede35cdf270a1c1e723cc42 \ + --hash=sha256:2d6eb928e13016cea4f1f21d1e10c1cebd5a421bc57ddf5b1142ae3f86824fab \ + --hash=sha256:2fe249cb4651fd12605b7288b24751d8bfd46d35f12a20b1ba33dea122e690df \ + --hash=sha256:30b8d1d8c52a48c2c5690e152c169b673487a2a58de1ec7393196753063fcd5e \ + --hash=sha256:320ade88cfb846b8cd6b4ddf5ee9e80ee0c1f52401f2456b84ae1ae6a1a5f207 \ + --hash=sha256:3534e7dcbdcf757da6b85a0bbf5b6868786d5982dd959b065e65481644817a18 \ + --hash=sha256:36836d6ff945a00b88ba1e4572d721e60b5b8c98c155d465f56ad19d68f23734 \ + --hash=sha256:38c0109396c4cfc574d502df99742a45c72c08eff0a36158b6f04000043dbf38 \ + --hash=sha256:3946fa46a0cf3e4c8cb1cc52f56bb536310d34f25f01ca9b6c16afa767dab110 \ + --hash=sha256:3bec022aec2c514d9cf199522a802bd007cd588ab17ab2525f20f9c34d067c18 \ + --hash=sha256:3c9a494bc5ec77d43cea229c4f6db1e4d8fe7e1bbffa8b6f0f0032430ff8ab44 \ + --hash=sha256:3dce51d0f5e7951f8bb4900c257dad282f49190fdbebecd4ba99bcc41fef404d \ + --hash=sha256:3dedcc22d73ec993f42055eff4fcfed9318d1eeb9a6606c55892a26964964e48 \ + --hash=sha256:4042d5c8f957e15221d423ba781e85d553722fc4113f523f2feb7b188cc34c5e \ + --hash=sha256:481551899c856c704d58119b5025793fa6730adda3571971af568f66d2424bb5 \ + --hash=sha256:4dc1e73c36828f982bfe79fadf5919923f8a6f4df2860804db9a98c48824ce8d \ + --hash=sha256:4e5163c14bffd570ef2affbfdd77bba66383890797df43dc8b4cc7d6f500bf53 \ + --hash=sha256:511ef87c8aec0783e08ac18565a16d435372bc1ac25a91e6ac7f5ef2b0bff790 \ + --hash=sha256:532bc9bf33a68613fd7d65e4b1c71a6a38d7d42604ecf239c77392e9b4e8998c \ + --hash=sha256:54523e136b8948060c0fa0bc7b1b50c32c186f2fceee897a495406bb6e311d2b \ + --hash=sha256:5649fd1c7bade02f320a462fdefd0b4bd3ce036065836d4f42e0de958038e116 \ + --hash=sha256:56be790f86bfb2c98fb742ce566dfb4816e5a83384616ab59c49e0604d49c51d \ + --hash=sha256:5b77459df20e08151cd6f8b9ef8ef1f961ef73d85c21a555c7eed5b79410ec10 \ + --hash=sha256:5ed6ab538499c8644b8a3e18debabcd7ce684f3fa91cf867521a7a0279cab2d6 \ + --hash=sha256:6178f72c5508bfc5fd446a5905e698c6212932f25bcdd4b47a757a50605a90e2 \ + --hash=sha256:6370e8686f662e6a3941ee48ed4742317cafbe5707e36406e9df792cdb535776 \ + --hash=sha256:64f02c6841d7d83f832cd97ccf8eb8a906d06eb95d5276069175c696b024b60a \ + --hash=sha256:65bcd23054beab4d166035cabbc868a09c1a49d1efe458fe8e4361215df40265 \ + --hash=sha256:66671f93accb62ed07da56613636f3641f1a12c13046ce91ffc923721f23c008 \ + --hash=sha256:6696b7688f54f5af4462118f0bfa7c1621eeb87154f77fa04b9295ce7a8f2943 \ + --hash=sha256:6785f414ae0f3c733c437e0f3929197934f526d19dfaa75e18fdb4f94c6fb374 \ + --hash=sha256:67f6279d125ca0046a7fd386d01b311c6363844deac3e5b069b514ba3e63c246 \ + --hash=sha256:6c114670c45346afedc0d947faf3c7f701051d2518b943679c8ff88befe14f8e \ + --hash=sha256:6e0d51f618228538a3e8f46bd246f87a6cd030565e015803691603f55e12afb5 \ + --hash=sha256:6ed74185b2db44f41ef35fd1617c5888e59792da9bbc9190d6c7300617182616 \ + --hash=sha256:708838739abf24b2ceb208d0e22403dd018faeef86ddac04319a62ae884c4f15 \ + --hash=sha256:715479b9a2802ecac752a3b0efa2b0b60285cf962ee38414211abdfccc233b41 \ + --hash=sha256:733784b6d6def852c814bce5f318d25da2ee65dd4839a0718641c696e09a2960 \ + --hash=sha256:750e02e074872a3fad7f233b47734166440af3cdea0add3e95163110816d6752 \ + --hash=sha256:752a45dc4a6934060b3b0dab47e04edc3326575f82be64bc4fc293914566503e \ + --hash=sha256:7579e913a5339fb8fa133f6bbcfd8e6749696206cf05acdbdca71a1b436d8e72 \ + --hash=sha256:7641bb8895e77f921102f72833904dcd9901df5d6d72a2ab8f31d04b7e51e4e7 \ + --hash=sha256:7804338df6fcc08105c7745f1502ba68d900f45fd770d5bdd5288ddccb8a42d8 \ + --hash=sha256:80d04837f55fc81da168b98de4f4b797ef007fc8a79ab71c6ec9bc4dd662b15b \ + --hash=sha256:813c0e0132266c08eb87469a642cb30aaff57c5f426255419572aaeceeaa7bf4 \ + --hash=sha256:82b271f5137d07749f7bf32f70b17ab6eaabedd297e75dce75081a24f76eb545 \ + --hash=sha256:84c018e49c3bf790f9c2771c45e9313a08c2c2a6342b162cd650258b57817706 \ + --hash=sha256:8751d2787c9131302398b11e6c8068053dcb55d5a8964e114b6e196cf16cb366 \ + --hash=sha256:8778f0c7a52e56f75d12dae53ae320fae900a8b9b4164b981b9c5ce059cd1fcb \ + --hash=sha256:87fad7d9ba98c86bcb41b2dc8dbb326619be2562af1f8ff50776a39e55721c5a \ + --hash=sha256:8d828b6667a32a728a1ad1d93957cdf37489c57b97ae6c4de2860fa749b8fc1e \ + --hash=sha256:8e385e4267ab76874ae30db04c627faaaf0b509e1ccc11a95b3fc3e83f855c00 \ + --hash=sha256:92a0a01ead5e668468e952e4238cccd7c537364eb7d851ab144ab6627dbbe12f \ + --hash=sha256:94e1885b270625a9a828c9793b4d52a64445299baa1fea5a173bf1d3dd9a1a5a \ + --hash=sha256:a180c5e59792af262bf263b21a3c49353f25945d8d9f70628e73de370d55e1e1 \ + --hash=sha256:a277ab8928b9f299723bc1a2dabb1265911b1a76341f90a510368ca44ad9ab66 \ + --hash=sha256:a5fe03b42827c13cdccd08e6c0247b6a6d4b5e3cdc53fd1749f5896adcdc2356 \ + --hash=sha256:a6c5863edfbe888d9eff9c8b8087354e27618d9da76425c119293f11712a6319 \ + --hash=sha256:a89c23ef8d2c6b27fd200a42aa4ac72786e7c60d40efdc76e6011260b6e949c4 \ + --hash=sha256:adb2597b428735679446b46c8badf467b4ca5f5056aae4d51a19f9570301b1ad \ + --hash=sha256:ae196f021b5e7c78e918242d217db021ed2a6ace2bc6ae94c0fc596221c7f58d \ + --hash=sha256:ae89db9e5f98a11a4bf50407d4363e7b09b31e55bc117b4f7d80aab97ba009e5 \ + --hash=sha256:aed52fea0513bac0ccde438c188c8a471c4e0f457c2dd20cdbf6ea7a450046c7 \ + --hash=sha256:aef65cd602a6d0e0ff6f9930fcb1c8fec60dd2cfcb6facaf4bdb0e5873042db0 \ + --hash=sha256:af21eb4409a119e365397b2adbaca4c9ccab56543a65d5dbd9f920d6ac29f686 \ + --hash=sha256:b14b2d9dac08e28bb8046a1a0434b1750eb221c8f5b87a68f4fa11a6f97b5e34 \ + --hash=sha256:bb6d88045545b26da47aa879dd4a89a71d1dce0f0e549b1abcb31dfe4a8eac49 \ + --hash=sha256:bb8cc7534f51d9a017b93e3e85b260924f909601c3df002bcdb58ddb4dc41a5c \ + --hash=sha256:bc17a677b21b3502a21f66a8cc64f5bfad4df8a0b8434d661666f8ce90ac3af1 \ + --hash=sha256:bd6c2a1c7573c64738d716488d2cdd3c00e340e4835707d8fdb8dc1a66ef164e \ + --hash=sha256:bd9b23791fe793e4968dba0c447e12f78e425c59fc0e3b97f6450f4781f3ee60 \ + --hash=sha256:c03a41a8784091e67a39648f70c5f97b5b6a37f216896d44d2cdcb82615339a0 \ + --hash=sha256:c0f081d69a6e58272819b70288d3221a6ee64b98df852631c80f293514d3b274 \ + --hash=sha256:c35abb8bfff0185efac5878da64c45dafd2b37fb0383add1be155a763c1f083d \ + --hash=sha256:c36c333c39be2dbca264d7803333c896ab8fa7d4d6f0ab7edb7dfd7aea6e98c0 \ + --hash=sha256:c45e9440fb78f8ddabcf714b68f936737a121355bf59f3907f4e17721b9d1aae \ + --hash=sha256:c593052c465475e64bbfe5dbd81680f64a67fdc752c56d7a0ae205dc8aeefe0f \ + --hash=sha256:cdd68a1fb318e290a2077696b7eb7a21a49163c455979c639bf5a5dcdc46617d \ + --hash=sha256:ce3412fbe1e31eb81ea42f4169ed94861c56e643189e1e75f0041f3fe7020abe \ + --hash=sha256:cf1493cd8607bec4d8a7b9b004e699fcf8f9103a9284cc94962cb73d20f9d4a3 \ + --hash=sha256:cf29836da5119f3c8a8a70667b0ef5fdca3bb12f80fd06487cfa575b3909b393 \ + --hash=sha256:d4a48e5b3c2a489fae013b7589308a40146ee081f6f509e047e0e096084ceca1 \ + --hash=sha256:d560742f3c0d62afaccf9f41fe485ed69bd7661a241f86a3ef0f0fb8b1a397af \ + --hash=sha256:d6038d37043bced98a66e68d3aa2b6a35505dc01328cd65217cefe82f25def44 \ + --hash=sha256:d61f00a0869d77422d9b2aba989e2d24afa6ffd552af442e0e58de4f35ea6d00 \ + --hash=sha256:d635aab80466bc95771bb78d5370e74d36d1fe31467b6b29b8b57b2a3cd7d22c \ + --hash=sha256:dca4bbc466a95ba9c0234ef56d7dd9509f63da22274589ebd4ed7f1f4d4c54e3 \ + --hash=sha256:dd915403e231e6b1809fe9b6d9fc55cf8fb5e02765ac625d9cd623342a7905d7 \ + --hash=sha256:e044c39e41b92c845bc815e5ae4230804e8e7bc29e399b0437d64222d92809dd \ + --hash=sha256:e060d01aec0a910bdccb8be71faf34e7799ce36950f8294c8bf612cba65a2c9e \ + --hash=sha256:e1421b502d83040e6d7fb2fb18dff63957f720da3d77b2fbd3187ceb63755d7b \ + --hash=sha256:e17b8d5d6a8c47c85e68ca8379def1303fd360c3e22093a807cd34a71cd082b8 \ + --hash=sha256:e5f4d355f0a2b1a31bc3edec6795b46324349c9cb25eed068049e4f472fb4259 \ + --hash=sha256:e712b419df8ba5e42b226c510472b37bd57b38e897d3eca5e8cfd410a29fa859 \ + --hash=sha256:e74327fb75de8986940def6e8dee4f127cc9752bee7355bb323cc5b2659b6d46 \ + --hash=sha256:e80c8378d8f3d83cd3164da1ad2df9e37a666cdde7b1cb2298ed0b558064be30 \ + --hash=sha256:e8ac484bf18ce6975760921bb6148041faa8fef0547200386ea0b52b5d27bf7b \ + --hash=sha256:eca9705049ad3c7345d574e3510665cb2cf844c2f2dcfe675332677f081cbd46 \ + --hash=sha256:ed065083d0898c9d5b4bbec7b026fd755ff7454e6e8b73a67f8c744b13986e24 \ + --hash=sha256:edac0f1ab77644605be2cbba52e6b7f630731fc42b34cb0f634be1a6eface56a \ + --hash=sha256:effc3f449787117233702311a1b7d8f59cba9ced946ba727bdc329ec69028e24 \ + --hash=sha256:f22dec1690b584cea26fade98b2435c132c1b5f68e39f5a0b7627cd7ae31f1dc \ + --hash=sha256:f495a1652cf3fbab2eb0639776dad966c2fb874d79d87ca07f9d5f059b8bd215 \ + --hash=sha256:f496c9c3cc02230093d8330875c4c3cdfc3b73612a5fd921c65d39cbcef08063 \ + --hash=sha256:f59099f9b66f0d7145115e6f80dd8b1d847176df89b234a5a6b3f00437aa0832 \ + --hash=sha256:f59ad4c0e8f6bba240a9bb85504faa1ab438237199d4cce5f622761507b8f6a6 \ + --hash=sha256:fbccdc05410c9ee21bbf16a35f4c1d16123dcdeb8a1d38f33654fa21d0234f79 \ + --hash=sha256:fea24543955a6a729c45a73fe90e08c743f0b3334bbf3201e6c4bc1b0c7fa464 # via requests colorama==0.4.6 ; sys_platform == 'win32' \ --hash=sha256:08695f5cb7ed6e0531a20572697297273c47b8cae5a63ffc6d6ed5c201be6e44 \ --hash=sha256:4f1d9991f5acc0ca119f9d443620b77f9d6b33703e51011c16baf57afb285fc6 # via sphinx -docutils==0.21.2 \ +docutils==0.21.2 ; python_full_version < '3.11' \ --hash=sha256:3a6b18732edf182daa3cd12775bbb338cf5691468f91eeeb109deff6ebfa986f \ --hash=sha256:dafca5b9e384f0e419294eb4d2ff9fa826435bf15f15b7bd45723e8ad76811b2 # via # myst-parser # sphinx # sphinx-rtd-theme -idna==3.10 \ - --hash=sha256:12f65c9b470abda6dc35cf8e63cc574b1c52b11df2c86030af0ac09b01b13ea9 \ - --hash=sha256:946d195a0d259cbba61165e88e65941f16e9b36ea6ddb97f00452bae8b1287d3 +docutils==0.22.4 ; python_full_version >= '3.11' \ + --hash=sha256:4db53b1fde9abecbb74d91230d32ab626d94f6badfc575d6db9194a49df29968 \ + --hash=sha256:d0013f540772d1420576855455d050a2180186c91c15779301ac2ccb3eeb68de + # via + # myst-parser + # sphinx + # sphinx-rtd-theme +idna==3.11 \ + --hash=sha256:771a87f49d9defaf64091e6e6fe9c18d4833f140bd19464795bc32d966ca37ea \ + --hash=sha256:795dafcc9c04ed0c1fb032c2aa73654d8e8c5023a7df64a53f39190ada629902 # via requests -imagesize==1.4.1 \ - --hash=sha256:0d8d18d08f840c19d0ee7ca1fd82490fdc3729b7ac93f49870406ddde8ef8d8b \ - --hash=sha256:69150444affb9cb0d5cc5a92b3676f0b2fb7cd9ae39e947a5e11a36b4497cd4a +imagesize==1.5.0 ; python_full_version < '3.10' \ + --hash=sha256:32677681b3f434c2cb496f00e89c5a291247b35b1f527589909e008057da5899 \ + --hash=sha256:8bfc5363a7f2133a89f0098451e0bcb1cd71aba4dc02bbcecb39d99d40e1b94f + # via sphinx +imagesize==2.0.0 ; python_full_version >= '3.10' \ + --hash=sha256:5667c5bbb57ab3f1fa4bc366f4fbc971db3d5ed011fd2715fd8001f782718d96 \ + --hash=sha256:8e8358c4a05c304f1fccf7ff96f036e7243a189e9e42e90851993c558cfe9ee3 # via sphinx -importlib-metadata==8.7.0 ; python_full_version < '3.10' \ - --hash=sha256:d13b81ad223b890aa16c5471f2ac3056cf76c5f10f82d6f9292f0b415f389000 \ - --hash=sha256:e5dd1551894c77868a30651cef00984d50e1002d06942a7101d34870c5f02afd +importlib-metadata==8.7.1 ; python_full_version < '3.10' \ + --hash=sha256:49fef1ae6440c182052f407c8d34a68f72efc36db9ca90dc0113398f2fdde8bb \ + --hash=sha256:5a1f80bf1daa489495071efbb095d75a634cf28a8bc299581244063b53176151 # via sphinx jinja2==3.1.6 \ --hash=sha256:0137fb05990d35f1275a587e9aee6d56da821fc83491a0fb838183be43f66d6d \ @@ -141,16 +206,22 @@ jinja2==3.1.6 \ # myst-parser # readthedocs-sphinx-ext # sphinx -macholib==1.16.3 \ - --hash=sha256:07ae9e15e8e4cd9a788013d81f5908b3609aa76f9b1421bae9c4d7606ec86a30 \ - --hash=sha256:0e315d7583d38b8c77e815b1ecbdbf504a8258d8b3e17b61165c6feb60d18f2c +macholib==1.16.4 \ + --hash=sha256:da1a3fa8266e30f0ce7e97c6a54eefaae8edd1e5f86f3eb8b95457cae90265ea \ + --hash=sha256:f408c93ab2e995cd2c46e34fe328b130404be143469e41bc366c807448979362 # via rules-python-docs (docs/pyproject.toml) -markdown-it-py==3.0.0 \ +markdown-it-py==3.0.0 ; python_full_version < '3.11' \ --hash=sha256:355216845c60bd96232cd8d8c40e8f9765cc86f46880e43a8fd22dc1a1a8cab1 \ --hash=sha256:e3f60a94fa066dc52ec76661e37c851cb232d92f9886b15cb560aaada2df8feb # via # mdit-py-plugins # myst-parser +markdown-it-py==4.0.0 ; python_full_version >= '3.11' \ + --hash=sha256:87327c59b172c5011896038353a81343b6754500a08cd7a4973bb48c6d578147 \ + --hash=sha256:cb0a2b4aa34f932c007117b194e945bd74e0ec24133ceb5bac59009cda1cb9f3 + # via + # mdit-py-plugins + # myst-parser markupsafe==3.0.3 \ --hash=sha256:0303439a41979d9e74d18ff5e2dd8c43ed6c6001fd40e5bf2e43f7bd9bbc523f \ --hash=sha256:068f375c472b3e7acbe2d5318dea141359e6900156b5b2ba06a30b169086b91a \ @@ -241,7 +312,9 @@ markupsafe==3.0.3 \ --hash=sha256:f71a396b3bf33ecaa1626c255855702aca4d3d9fea5e051b41ac59a9c1c41edc \ --hash=sha256:f9e130248f4462aaa8e2552d547f36ddadbeaa573879158d721bbd33dfe4743a \ --hash=sha256:fed51ac40f757d41b7c48425901843666a6677e3e8eb0abcff09e4ba6e664f50 - # via jinja2 + # via + # rules-python-docs (docs/pyproject.toml) + # jinja2 mdit-py-plugins==0.4.2 ; python_full_version < '3.10' \ --hash=sha256:0c673c3f889399a33b95e88d2f0d111b4447bdfea7f237dab2d488f459835636 \ --hash=sha256:5f2cd1fdb606ddf152d37ec30e46101a60512bc0e5fa1a7002c36647b09e26b5 @@ -258,13 +331,17 @@ myst-parser==3.0.1 ; python_full_version < '3.10' \ --hash=sha256:6457aaa33a5d474aca678b8ead9b3dc298e89c68e67012e73146ea6fd54babf1 \ --hash=sha256:88f0cb406cb363b077d176b51c476f62d60604d68a8dcdf4832e080441301a87 # via rules-python-docs (docs/pyproject.toml) -myst-parser==4.0.1 ; python_full_version >= '3.10' \ +myst-parser==4.0.1 ; python_full_version == '3.10.*' \ --hash=sha256:5cfea715e4f3574138aecbf7d54132296bfd72bb614d31168f48c477a830a7c4 \ --hash=sha256:9134e88959ec3b5780aedf8a99680ea242869d012e8821db3126d427edc9c95d # via rules-python-docs (docs/pyproject.toml) -packaging==25.0 \ - --hash=sha256:29572ef2b1f17581046b3a2227d5c611fb25ec70ca1ba8554b24b0e69331a484 \ - --hash=sha256:d443872c98d677bf60f6a1f2f8c1cb748e8fe762d2bf9d3148b5599295b0fc4f +myst-parser==5.0.0 ; python_full_version >= '3.11' \ + --hash=sha256:ab31e516024918296e169139072b81592336f2fef55b8986aa31c9f04b5f7211 \ + --hash=sha256:f6f231452c56e8baa662cc352c548158f6a16fcbd6e3800fc594978002b94f3a + # via rules-python-docs (docs/pyproject.toml) +packaging==26.1 \ + --hash=sha256:5d9c0669c6285e491e0ced2eee587eaf67b670d94a19e94e3984a481aba6802f \ + --hash=sha256:f042152b681c4bfac5cae2742a55e103d27ab2ec0f3d88037136b6bfe7c9c5de # via # readthedocs-sphinx-ext # sphinx @@ -276,9 +353,9 @@ pyelftools==0.32 \ --hash=sha256:013df952a006db5e138b1edf6d8a68ecc50630adbd0d83a2d41e7f846163d738 \ --hash=sha256:6de90ee7b8263e740c8715a925382d4099b354f29ac48ea40d840cf7aa14ace5 # via rules-python-docs (docs/pyproject.toml) -pygments==2.19.2 \ - --hash=sha256:636cb2477cec7f8952536970bc533bc43743542f70392ae026374600add5b887 \ - --hash=sha256:86540386c03d588bb81d44bc3928634ff26449851e99741617ecb9037ee5ec0b +pygments==2.20.0 \ + --hash=sha256:6757cd03768053ff99f3039c1a36d6c0aa0b263438fcab17520b30a303a82b5f \ + --hash=sha256:81a9e26dd42fd28a23a2d169d86d7ac03b46e2f8b59ed4698fb4785f946d0176 # via sphinx pyyaml==6.0.3 \ --hash=sha256:00c4bdeba853cc34e7dd471f16b4114f4162dc03e6b7afcc2128711f0eca823c \ @@ -359,15 +436,21 @@ readthedocs-sphinx-ext==2.2.5 \ --hash=sha256:ee5fd5b99db9f0c180b2396cbce528aa36671951b9526bb0272dbfce5517bd27 \ --hash=sha256:f8c56184ea011c972dd45a90122568587cc85b0127bc9cf064d17c68bc809daa # via rules-python-docs (docs/pyproject.toml) -requests==2.33.0 \ - --hash=sha256:3324635456fa185245e24865e810cecec7b4caf933d7eb133dcde67d48cee69b \ - --hash=sha256:c7ebc5e8b0f21837386ad0e1c8fe8b829fa5f544d8df3b2253bff14ef29d7652 +requests==2.32.5 ; python_full_version < '3.10' \ + --hash=sha256:2462f94637a34fd532264295e186976db0f5d453d1cdd31473c85a6a161affb6 \ + --hash=sha256:dbba0bac56e100853db0ea71b82b4dfd5fe2bf6d3754a8893c3af500cec7d7cf # via # readthedocs-sphinx-ext # sphinx -roman-numerals-py==3.1.0 ; python_full_version >= '3.11' \ - --hash=sha256:9da2ad2fb670bcf24e81070ceb3be72f6c11c440d73bd579fbeca1e9f330954c \ - --hash=sha256:be4bf804f083a4ce001b5eb7e3c0862479d10f94c936f6c4e5f250aa5ff5bd2d +requests==2.33.1 ; python_full_version >= '3.10' \ + --hash=sha256:18817f8c57c6263968bc123d237e3b8b08ac046f5456bd1e307ee8f4250d3517 \ + --hash=sha256:4e6d1ef462f3626a1f0a0a9c42dd93c63bad33f9f1c1937509b8c5c8718ab56a + # via + # readthedocs-sphinx-ext + # sphinx +roman-numerals==4.1.0 ; python_full_version >= '3.11' \ + --hash=sha256:1af8b147eb1405d5839e78aeb93131690495fe9da5c91856cb33ad55a7f1e5b2 \ + --hash=sha256:647ba99caddc2cc1e55a51e4360689115551bf4476d90e8162cf8c345fe233c7 # via sphinx snowballstemmer==3.0.1 \ --hash=sha256:6cd7b3897da8d6c9ffb968a6781fa6532dce9c3618a4b127d920dab764a19064 \ @@ -391,9 +474,18 @@ sphinx==8.1.3 ; python_full_version == '3.10.*' \ # sphinx-reredirects # sphinx-rtd-theme # sphinxcontrib-jquery -sphinx==8.2.3 ; python_full_version >= '3.11' \ - --hash=sha256:398ad29dee7f63a75888314e9424d40f52ce5a6a87ae88e7071e80af296ec348 \ - --hash=sha256:4405915165f13521d875a8c29c8970800a0141c14cc5416a38feca4ea5d9b9c3 +sphinx==9.0.4 ; python_full_version == '3.11.*' \ + --hash=sha256:594ef59d042972abbc581d8baa577404abe4e6c3b04ef61bd7fc2acbd51f3fa3 \ + --hash=sha256:5bebc595a5e943ea248b99c13814c1c5e10b3ece718976824ffa7959ff95fffb + # via + # rules-python-docs (docs/pyproject.toml) + # myst-parser + # sphinx-reredirects + # sphinx-rtd-theme + # sphinxcontrib-jquery +sphinx==9.1.0 ; python_full_version >= '3.12' \ + --hash=sha256:7741722357dd75f8190766926071fed3bdc211c74dd2d7d4df5404da95930ddb \ + --hash=sha256:c84fdd4e782504495fe4f2c0b3413d6c2bf388589bb352d439b2a3bb99991978 # via # rules-python-docs (docs/pyproject.toml) # myst-parser @@ -408,13 +500,13 @@ sphinx-reredirects==0.1.6 ; python_full_version < '3.11' \ --hash=sha256:c491cba545f67be9697508727818d8626626366245ae64456fe29f37e9bbea64 \ --hash=sha256:efd50c766fbc5bf40cd5148e10c00f2c00d143027de5c5e48beece93cc40eeea # via rules-python-docs (docs/pyproject.toml) -sphinx-reredirects==1.0.0 ; python_full_version >= '3.11' \ - --hash=sha256:1d0102710a8f633c6c885f940f440f7195ada675c1739976f0135790747dea06 \ - --hash=sha256:7c9bada9f1330489fcf4c7297a2d6da2a49ca4877d3f42d1388ae1de1019bf5c +sphinx-reredirects==1.1.0 ; python_full_version >= '3.11' \ + --hash=sha256:4b5692273c72cd2d4d917f4c6f87d5919e4d6114a752d4be033f7f5f6310efd9 \ + --hash=sha256:fb9b195335ab14b43f8273287d0c7eeb637ba6c56c66581c11b47202f6718b29 # via rules-python-docs (docs/pyproject.toml) -sphinx-rtd-theme==3.0.2 \ - --hash=sha256:422ccc750c3a3a311de4ae327e82affdaf59eb695ba4936538552f3b00f4ee13 \ - --hash=sha256:b7457bc25dda723b20b086a670b9953c859eab60a2a03ee8eb2bb23e176e5f85 +sphinx-rtd-theme==3.1.0 \ + --hash=sha256:1785824ae8e6632060490f67cf3a72d404a85d2d9fc26bce3619944de5682b89 \ + --hash=sha256:b44276f2c276e909239a4f6c955aa667aaafeb78597923b1c60babc76db78e4c # via rules-python-docs (docs/pyproject.toml) sphinxcontrib-applehelp==2.0.0 \ --hash=sha256:2f29ef331735ce958efa4734873f084941970894c6090408b079c61b2e1c06d1 \ @@ -444,49 +536,54 @@ sphinxcontrib-serializinghtml==2.0.0 \ --hash=sha256:6e2cb0eef194e10c27ec0023bfeb25badbbb5868244cf5bc5bdc04e4464bf331 \ --hash=sha256:e9d912827f872c029017a53f0ef2180b327c3f7fd23c87229f7a8e8b70031d4d # via sphinx -tomli==2.3.0 ; python_full_version < '3.11' \ - --hash=sha256:00b5f5d95bbfc7d12f91ad8c593a1659b6387b43f054104cda404be6bda62456 \ - --hash=sha256:0a154a9ae14bfcf5d8917a59b51ffd5a3ac1fd149b71b47a3a104ca4edcfa845 \ - --hash=sha256:0c95ca56fbe89e065c6ead5b593ee64b84a26fca063b5d71a1122bf26e533999 \ - --hash=sha256:0eea8cc5c5e9f89c9b90c4896a8deefc74f518db5927d0e0e8d4a80953d774d0 \ - --hash=sha256:1cb4ed918939151a03f33d4242ccd0aa5f11b3547d0cf30f7c74a408a5b99878 \ - --hash=sha256:4021923f97266babc6ccab9f5068642a0095faa0a51a246a6a02fccbb3514eaf \ - --hash=sha256:4c2ef0244c75aba9355561272009d934953817c49f47d768070c3c94355c2aa3 \ - --hash=sha256:4dc4ce8483a5d429ab602f111a93a6ab1ed425eae3122032db7e9acf449451be \ - --hash=sha256:4f195fe57ecceac95a66a75ac24d9d5fbc98ef0962e09b2eddec5d39375aae52 \ - --hash=sha256:5192f562738228945d7b13d4930baffda67b69425a7f0da96d360b0a3888136b \ - --hash=sha256:5e01decd096b1530d97d5d85cb4dff4af2d8347bd35686654a004f8dea20fc67 \ - --hash=sha256:64be704a875d2a59753d80ee8a533c3fe183e3f06807ff7dc2232938ccb01549 \ - --hash=sha256:70a251f8d4ba2d9ac2542eecf008b3c8a9fc5c3f9f02c56a9d7952612be2fdba \ - --hash=sha256:73ee0b47d4dad1c5e996e3cd33b8a76a50167ae5f96a2607cbe8cc773506ab22 \ - --hash=sha256:74bf8464ff93e413514fefd2be591c3b0b23231a77f901db1eb30d6f712fc42c \ - --hash=sha256:792262b94d5d0a466afb5bc63c7daa9d75520110971ee269152083270998316f \ - --hash=sha256:7b0882799624980785240ab732537fcfc372601015c00f7fc367c55308c186f6 \ - --hash=sha256:883b1c0d6398a6a9d29b508c331fa56adbcdff647f6ace4dfca0f50e90dfd0ba \ - --hash=sha256:88bd15eb972f3664f5ed4b57c1634a97153b4bac4479dcb6a495f41921eb7f45 \ - --hash=sha256:8a35dd0e643bb2610f156cca8db95d213a90015c11fee76c946aa62b7ae7e02f \ - --hash=sha256:940d56ee0410fa17ee1f12b817b37a4d4e4dc4d27340863cc67236c74f582e77 \ - --hash=sha256:97d5eec30149fd3294270e889b4234023f2c69747e555a27bd708828353ab606 \ - --hash=sha256:a0e285d2649b78c0d9027570d4da3425bdb49830a6156121360b3f8511ea3441 \ - --hash=sha256:a1f7f282fe248311650081faafa5f4732bdbfef5d45fe3f2e702fbc6f2d496e0 \ - --hash=sha256:a4ea38c40145a357d513bffad0ed869f13c1773716cf71ccaa83b0fa0cc4e42f \ - --hash=sha256:a56212bdcce682e56b0aaf79e869ba5d15a6163f88d5451cbde388d48b13f530 \ - --hash=sha256:ad805ea85eda330dbad64c7ea7a4556259665bdf9d2672f5dccc740eb9d3ca05 \ - --hash=sha256:b273fcbd7fc64dc3600c098e39136522650c49bca95df2d11cf3b626422392c8 \ - --hash=sha256:b5870b50c9db823c595983571d1296a6ff3e1b88f734a4c8f6fc6188397de005 \ - --hash=sha256:b74a0e59ec5d15127acdabd75ea17726ac4c5178ae51b85bfe39c4f8a278e879 \ - --hash=sha256:be71c93a63d738597996be9528f4abe628d1adf5e6eb11607bc8fe1a510b5dae \ - --hash=sha256:c22a8bf253bacc0cf11f35ad9808b6cb75ada2631c2d97c971122583b129afbc \ - --hash=sha256:c4665508bcbac83a31ff8ab08f424b665200c0e1e645d2bd9ab3d3e557b6185b \ - --hash=sha256:c5f3ffd1e098dfc032d4d3af5c0ac64f6d286d98bc148698356847b80fa4de1b \ - --hash=sha256:cebc6fe843e0733ee827a282aca4999b596241195f43b4cc371d64fc6639da9e \ - --hash=sha256:d1381caf13ab9f300e30dd8feadb3de072aeb86f1d34a8569453ff32a7dea4bf \ - --hash=sha256:d7d86942e56ded512a594786a5ba0a5e521d02529b3826e7761a05138341a2ac \ - --hash=sha256:e31d432427dcbf4d86958c184b9bfd1e96b5b71f8eb17e6d02531f434fd335b8 \ - --hash=sha256:e95b1af3c5b07d9e643909b5abbec77cd9f1217e6d0bca72b0234736b9fb1f1b \ - --hash=sha256:f85209946d1fe94416debbb88d00eb92ce9cd5266775424ff81bc959e001acaf \ - --hash=sha256:feb0dacc61170ed7ab602d3d972a58f14ee3ee60494292d384649a3dc38ef463 \ - --hash=sha256:ff72b71b5d10d22ecb084d345fc26f42b5143c5533db5e2eaba7d2d335358876 +tomli==2.4.1 ; python_full_version < '3.11' \ + --hash=sha256:01f520d4f53ef97964a240a035ec2a869fe1a37dde002b57ebc4417a27ccd853 \ + --hash=sha256:0d85819802132122da43cb86656f8d1f8c6587d54ae7dcaf30e90533028b49fe \ + --hash=sha256:136443dbd7e1dee43c68ac2694fde36b2849865fa258d39bf822c10e8068eac5 \ + --hash=sha256:1d8591993e228b0c930c4bb0db464bdad97b3289fb981255d6c9a41aedc84b2d \ + --hash=sha256:2190f2e9dd7508d2a90ded5ed369255980a1bcdd58e52f7fe24b8162bf9fedbd \ + --hash=sha256:2c1c351919aca02858f740c6d33adea0c5deea37f9ecca1cc1ef9e884a619d26 \ + --hash=sha256:36d2bd2ad5fb9eaddba5226aa02c8ec3fa4f192631e347b3ed28186d43be6b54 \ + --hash=sha256:3d48a93ee1c9b79c04bb38772ee1b64dcf18ff43085896ea460ca8dec96f35f6 \ + --hash=sha256:47149d5bd38761ac8be13a84864bf0b7b70bc051806bc3669ab1cbc56216b23c \ + --hash=sha256:4ab97e64ccda8756376892c53a72bd1f964e519c77236368527f758fbc36a53a \ + --hash=sha256:4b605484e43cdc43f0954ddae319fb75f04cc10dd80d830540060ee7cd0243cd \ + --hash=sha256:504aa796fe0569bb43171066009ead363de03675276d2d121ac1a4572397870f \ + --hash=sha256:51529d40e3ca50046d7606fa99ce3956a617f9b36380da3b7f0dd3dd28e68cb5 \ + --hash=sha256:52c8ef851d9a240f11a88c003eacb03c31fc1c9c4ec64a99a0f922b93874fda9 \ + --hash=sha256:559db847dc486944896521f68d8190be1c9e719fced785720d2216fe7022b662 \ + --hash=sha256:5a881ab208c0baf688221f8cecc5401bd291d67e38a1ac884d6736cbcd8247e9 \ + --hash=sha256:5cb41aa38891e073ee49d55fbc7839cfdb2bc0e600add13874d048c94aadddd1 \ + --hash=sha256:5e262d41726bc187e69af7825504c933b6794dc3fbd5945e41a79bb14c31f585 \ + --hash=sha256:5ee18d9ebdb417e384b58fe414e8d6af9f4e7a0ae761519fb50f721de398dd4e \ + --hash=sha256:7008df2e7655c495dd12d2a4ad038ff878d4ca4b81fccaf82b714e07eae4402c \ + --hash=sha256:734e20b57ba95624ecf1841e72b53f6e186355e216e5412de414e3c51e5e3c41 \ + --hash=sha256:7c7e1a961a0b2f2472c1ac5b69affa0ae1132c39adcb67aba98568702b9cc23f \ + --hash=sha256:7f86fd587c4ed9dd76f318225e7d9b29cfc5a9d43de44e5754db8d1128487085 \ + --hash=sha256:7f94b27a62cfad8496c8d2513e1a222dd446f095fca8987fceef261225538a15 \ + --hash=sha256:88dceee75c2c63af144e456745e10101eb67361050196b0b6af5d717254dddf7 \ + --hash=sha256:8a650c2dbafa08d42e51ba0b62740dae4ecb9338eefa093aa5c78ceb546fcd5c \ + --hash=sha256:8d65a2fbf9d2f8352685bc1364177ee3923d6baf5e7f43ea4959d7d8bc326a36 \ + --hash=sha256:96481a5786729fd470164b47cdb3e0e58062a496f455ee41b4403be77cb5a076 \ + --hash=sha256:a120733b01c45e9a0c34aeef92bf0cf1d56cfe81ed9d47d562f9ed591a9828ac \ + --hash=sha256:b1d22e6e9387bf4739fbe23bfa80e93f6b0373a7f1b96c6227c32bef95a4d7a8 \ + --hash=sha256:b8c198f8c1805dc42708689ed6864951fd2494f924149d3e4bce7710f8eb5232 \ + --hash=sha256:c2541745709bad0264b7d4705ad453b76ccd191e64aa6f0fc66b69a293a45ece \ + --hash=sha256:c742f741d58a28940ce01d58f0ab2ea3ced8b12402f162f4d534dfe18ba1cd6a \ + --hash=sha256:c7f2c7f2b9ca6bdeef8f0fa897f8e05085923eb091721675170254cbc5b02897 \ + --hash=sha256:d312ef37c91508b0ab2cee7da26ec0b3ed2f03ce12bd87a588d771ae15dcf82d \ + --hash=sha256:d4d8fe59808a54658fcc0160ecfb1b30f9089906c50b23bcb4c69eddc19ec2b4 \ + --hash=sha256:da25dc3563bff5965356133435b757a795a17b17d01dbc0f42fb32447ddfd917 \ + --hash=sha256:eab21f45c7f66c13f2a9e0e1535309cee140182a9cdae1e041d02e47291e8396 \ + --hash=sha256:eb0dc4e38e6a1fd579e5d50369aa2e10acfc9cace504579b2faabb478e76941a \ + --hash=sha256:ec9bfaf3ad2df51ace80688143a6a4ebc09a248f6ff781a9945e51937008fcbc \ + --hash=sha256:ede3e6487c5ef5d28634ba3f31f989030ad6af71edfb0055cbbd14189ff240ba \ + --hash=sha256:f3c6818a1a86dd6dca7ddcaaf76947d5ba31aecc28cb1b67009a5877c9a64f3f \ + --hash=sha256:f758f1b9299d059cc3f6546ae2af89670cb1c4d48ea29c3cacc4fe7de3058257 \ + --hash=sha256:f8f0fc26ec2cc2b965b7a3b87cd19c5c6b8c5e5f436b984e85f486d652285c30 \ + --hash=sha256:fd0409a3653af6c147209d267a0e4243f0ae46b011aa978b1080359fddc9b6cf \ + --hash=sha256:ff18e6a727ee0ab0388507b89d1bc6a22b138d1e2fa56d1ad494586d61d2eae9 \ + --hash=sha256:ff2983983d34813c1aeb0fa89091e76c3a22889ee83ab27c5eeb45100560c049 # via # sphinx # sphinx-autodoc2 @@ -501,7 +598,7 @@ urllib3==2.6.3 \ --hash=sha256:1b62b6884944a57dbe321509ab94fd4d3b307075e0c2eae991ac71ee15ad38ed \ --hash=sha256:bf272323e553dfb2e87d9bfd225ca7b0f467b919d7bbd355436d3fd37cb0acd4 # via requests -zipp==3.23.0 ; python_full_version < '3.10' \ - --hash=sha256:071652d6115ed432f5ce1d34c336c0adfd6a884660d1e9712a256d3d3bd4b14e \ - --hash=sha256:a07157588a12518c9d4034df3fbbee09c814741a33ff63c05fa29d26a2404166 +zipp==3.23.1 ; python_full_version < '3.10' \ + --hash=sha256:0b3596c50a5c700c9cb40ba8d86d9f2cc4807e9bedb06bcdf7fac85633e444dc \ + --hash=sha256:32120e378d32cd9714ad503c1d024619063ec28aad2248dc6672ad13edfa5110 # via importlib-metadata diff --git a/python/private/attributes.bzl b/python/private/attributes.bzl index 0892fe3111..beaa7a6a73 100644 --- a/python/private/attributes.bzl +++ b/python/private/attributes.bzl @@ -542,6 +542,14 @@ AGNOSTIC_TEST_ATTRS = _init_agnostic_test_attrs() # but still accept Python source-agnostic settings. AGNOSTIC_BINARY_ATTRS = dicts.add(AGNOSTIC_EXECUTABLE_ATTRS) +WINDOWS_CONSTRAINTS_ATTRS = { + "_windows_constraints": lambda: attrb.LabelList( + default = [ + "@platforms//os:windows", + ], + ), +} + # Attribute names common to all Python rules COMMON_ATTR_NAMES = [ "compatible_with", diff --git a/python/private/common.bzl b/python/private/common.bzl index 6d9e0e3c84..7e8c6decf3 100644 --- a/python/private/common.bzl +++ b/python/private/common.bzl @@ -49,6 +49,31 @@ BUILTIN_BUILD_PYTHON_ZIP = [] if config.bazel_10_or_later else [ "//command_line_option:build_python_zip", ] +# Not an actual provider. Provider used for memory efficiency. +# buildifier: disable=name-conventions +ExplicitSymlink = provider( + doc = """ +A runfile that should be created as a symlink pointing to a specific location. + +This is only needed on Windows, where Bazel doesn't preserve declare_symlink +with relative paths. This is basically manually captures what using +declare_symlink(), symlink() and runfiles like so would capture: + +``` +link = declare_symlink(...) +link_to_path = relative_path(from=link, to=target) +symlink(output=link, target_path=link_to_path) +runfiles.add([link, target]) +``` +""", + fields = { + "files": "depset[File] of files that should be included if this symlink is used", + "link_to_path": "Path the symlink should point to", + "runfiles_path": "runfiles-root-relative path for the symlink", + "venv_path": "venv-root-relative path for the symlink", + }, +) + def maybe_builtin_build_python_zip(value, settings = None): settings = settings or {} if not config.bazel_10_or_later: @@ -460,6 +485,17 @@ def target_platform_has_any_constraint(ctx, constraints): return True return False +def is_windows_platform(ctx): + """Check if target platform is windows. + + Args: + ctx: rule context. + + Returns: + True if target platform is windows. + """ + return target_platform_has_any_constraint(ctx, ctx.attr._windows_constraints) + def relative_path(from_, to): """Compute a relative path from one path to another. diff --git a/python/private/py_executable.bzl b/python/private/py_executable.bzl index 1cb7c9593c..375030ce91 100644 --- a/python/private/py_executable.bzl +++ b/python/private/py_executable.bzl @@ -30,12 +30,14 @@ load( "PrecompileAttr", "PycCollectionAttr", "REQUIRED_EXEC_GROUP_BUILDERS", + "WINDOWS_CONSTRAINTS_ATTRS", "apply_config_settings_attr", ) load(":builders.bzl", "builders") load(":cc_helper.bzl", "cc_helper") load( ":common.bzl", + "ExplicitSymlink", "collect_cc_info", "collect_deps", "collect_imports", @@ -49,10 +51,10 @@ load( "csv", "filter_to_py_srcs", "is_bool", + "is_windows_platform", "maybe_create_repo_mapping", "relative_path", "runfiles_root_path", - "target_platform_has_any_constraint", ) load(":common_labels.bzl", "labels") load(":flags.bzl", "BootstrapImplFlag", "VenvsUseDeclareSymlinkFlag", "read_possibly_native_flag") @@ -73,30 +75,6 @@ _EXTERNAL_PATH_PREFIX = "external" _ZIP_RUNFILES_DIRECTORY_NAME = "runfiles" _INIT_PY = "__init__.py" -# buildifier: disable=name-conventions -ExplicitSymlink = provider( - doc = """ -A runfile that should be created as a symlink pointing to a specific location. - -This is only needed on Windows, where Bazel doesn't preserve declare_symlink -with relative paths. This is basically manually captures what using -declare_symlink(), symlink() and runfiles like so would capture: - -``` -link = declare_symlink(...) -link_to_path = relative_path(from=link, to=target) -symlink(output=link, target_path=link_to_path) -runfiles.add([link, target]) -``` -""", - fields = { - "files": "depset[File] of files that should be included if this symlink is used", - "link_to_path": "Path the symlink should point to", - "runfiles_path": "runfiles-root-relative path for the symlink", - "venv_path": "venv-root-relative path for the symlink", - }, -) - # Non-Google-specific attributes for executables # These attributes are for rules that accept Python sources. EXECUTABLE_ATTRS = dicts.add( @@ -104,6 +82,7 @@ EXECUTABLE_ATTRS = dicts.add( AGNOSTIC_EXECUTABLE_ATTRS, PY_SRCS_ATTRS, IMPORTS_ATTRS, + WINDOWS_CONSTRAINTS_ATTRS, # starlark flags attributes { "_build_python_zip_flag": attr.label(default = "//python/config_settings:build_python_zip"), @@ -257,11 +236,6 @@ accepting arbitrary Python versions. default = labels.VENVS_USE_DECLARE_SYMLINK, providers = [BuildSettingInfo], ), - "_windows_constraints": lambda: attrb.LabelList( - default = [ - "@platforms//os:windows", - ], - ), "_zipper": lambda: attrb.Label( cfg = "exec", executable = True, @@ -323,7 +297,7 @@ def _create_executable( extra_deps): _ = is_test, cc_details, native_deps_details # @unused - is_windows = target_platform_has_any_constraint(ctx, ctx.attr._windows_constraints) + is_windows = is_windows_platform(ctx) if is_windows: if not executable.extension == "exe": @@ -447,14 +421,14 @@ WARNING: Target: {} use_zip_file = build_zip_enabled, python_binary_path = runtime_details.executable_interpreter_path, ) - if not build_zip_enabled: - # On Windows, the main executable has an "exe" extension, so - # here we re-use the un-extensioned name for the bootstrap output. - bootstrap_output = ctx.actions.declare_file(base_executable_name) - # The launcher looks for the non-zip executable next to - # itself, so add it to the default outputs. - extra_default_outputs.append(bootstrap_output) + # On Windows, the main executable has an "exe" extension, so + # here we re-use the un-extensioned name for the bootstrap output. + bootstrap_output = ctx.actions.declare_file(base_executable_name) + + # The launcher looks for the non-zip executable next to + # itself, so add it to the default outputs. + extra_default_outputs.append(bootstrap_output) if should_create_executable_zip: if bootstrap_output != None: @@ -516,6 +490,9 @@ WARNING: Target: {} # runfiles; runfiles for the app itself (e.g its deps, but no Python # runtime files) app_runfiles = app_runfiles.build(ctx), + # depset[ExplicitSymlink]None; symlinks that should be created in + # the venv to augment app_runfiles + venv_app_symlinks = venv.lib_symlinks if venv else None, # File|None; the venv `bin/python3` file, if any. venv_python_exe = venv.interpreter if venv else None, # runfiles|None; runfiles in the venv for the interpreter @@ -561,7 +538,7 @@ def _create_venv(ctx, output_prefix, imports, runtime_details, add_runfiles_root else: interpreter_actual_path = runtime.interpreter_path - is_windows = target_platform_has_any_constraint(ctx, ctx.attr._windows_constraints) + is_windows = is_windows_platform(ctx) if is_windows: venv_details = _create_venv_windows( ctx, @@ -644,6 +621,7 @@ def _create_venv(ctx, output_prefix, imports, runtime_details, add_runfiles_root lib_runfiles = ctx.runfiles( root_symlinks = venv_app_files.runfiles_symlinks, ), + lib_symlinks = venv_app_files.explicit_symlinks, ) def _create_venv_unixy(ctx, *, venv_ctx_rel_root, runtime, interpreter_actual_path): @@ -937,9 +915,12 @@ def _create_stage1_bootstrap( } computed_subs = ctx.actions.template_dict() if venv: + runtime_venv_symlinks = depset( + transitive = [venv.interpreter_symlinks, venv.lib_symlinks], + ) computed_subs.add_joined( "%runtime_venv_symlinks%", - venv.interpreter_symlinks, + runtime_venv_symlinks, join_with = "\n", map_each = _map_runtime_venv_symlink, ) @@ -1250,8 +1231,6 @@ def py_executable_base_impl(ctx, *, semantics, is_test, inherited_environment = ) )) - app_runfiles = exec_result.app_runfiles - providers = [] _add_provider_default_info( @@ -1265,13 +1244,14 @@ def py_executable_base_impl(ctx, *, semantics, is_test, inherited_environment = _add_provider_run_environment_info(providers, ctx, inherited_environment) _add_provider_py_executable_info( providers, - app_runfiles = app_runfiles, + app_runfiles = exec_result.app_runfiles, build_data_file = runfiles_details.build_data_file, interpreter_args = ctx.attr.interpreter_args, interpreter_path = runtime_details.executable_interpreter_path, main_py = main_py, runfiles_without_exe = runfiles_details.runfiles_without_exe, stage2_bootstrap = exec_result.stage2_bootstrap, + venv_app_symlinks = exec_result.venv_app_symlinks, venv_interpreter_runfiles = exec_result.venv_interpreter_runfiles, venv_interpreter_symlinks = exec_result.venv_interpreter_symlinks, venv_python_exe = exec_result.venv_python_exe, @@ -1313,7 +1293,7 @@ def _validate_executable(ctx): ).format(ctx.attr.main, ctx.attr.main_module)) def _declare_executable_file(ctx): - if target_platform_has_any_constraint(ctx, ctx.attr._windows_constraints): + if is_windows_platform(ctx): executable = ctx.actions.declare_file(ctx.label.name + ".exe") else: executable = ctx.actions.declare_file(ctx.label.name) @@ -1882,6 +1862,7 @@ def _add_provider_py_executable_info( main_py, runfiles_without_exe, stage2_bootstrap, + venv_app_symlinks, venv_interpreter_runfiles, venv_interpreter_symlinks, venv_python_exe): @@ -1896,6 +1877,8 @@ def _add_provider_py_executable_info( main_py: File; the main .py entry point. runfiles_without_exe: runfiles; the default runfiles, but without the executable. stage2_bootstrap: File; the stage 2 bootstrap script. + venv_app_symlinks: depset[ExplicitSymlink]; symlinks to create for the + venv that are the application (deps, etc). venv_interpreter_runfiles: runfiles; runfiles specific to the interpreter for the venv. venv_interpreter_symlinks: depset[ExplicitSymlink]; interpreter-specific symlinks to create for the venv. venv_python_exe: File; the python executable in the venv. @@ -1908,6 +1891,7 @@ def _add_provider_py_executable_info( main = main_py, runfiles_without_exe = runfiles_without_exe, stage2_bootstrap = stage2_bootstrap, + venv_app_symlinks = venv_app_symlinks, venv_interpreter_runfiles = venv_interpreter_runfiles, venv_interpreter_symlinks = venv_interpreter_symlinks, venv_python_exe = venv_python_exe, diff --git a/python/private/py_executable_info.bzl b/python/private/py_executable_info.bzl index a076715bd0..9e9baea421 100644 --- a/python/private/py_executable_info.bzl +++ b/python/private/py_executable_info.bzl @@ -77,6 +77,19 @@ implementation isn't being used. :::{versionadded} 1.9.0 ::: +""", + "venv_app_symlinks": """ +:type: depset[ExplicitSymlink] | None + +Symlinks that are specific to the application within the venv (e.g. +dependencies). + +Only used with Windows for files that would have used `declare_symlink()` +to create relative symlinks. These may overlap with paths in runfiles; it's +up to the consumer to determine how to handle such overlaps. + +:::{versionadded} VERSION_NEXT_FEATURE +::: """, "venv_interpreter_runfiles": """ :type: runfiles | None diff --git a/python/private/python_bootstrap_template.txt b/python/private/python_bootstrap_template.txt index accc11e0df..0d28aff311 100644 --- a/python/private/python_bootstrap_template.txt +++ b/python/private/python_bootstrap_template.txt @@ -7,7 +7,7 @@ from __future__ import print_function # Generated file from @rules_python//python/private:python_bootstrap_template.txt -from os.path import dirname, join, basename, normpath +from os.path import abspath, dirname, join, basename, normpath import os import shutil import subprocess @@ -95,6 +95,7 @@ else: IS_WINDOWS = os.name == "nt" BIN_DIR_NAME = "bin" if not IS_WINDOWS else "Scripts" +LIB_DIR_NAME = "lib" if not IS_WINDOWS else "Lib" # Windows APIs can be picky about slashes depending on the context, # so convert to backslashes to avoid any issues. @@ -147,15 +148,15 @@ def get_windows_path_with_unc_prefix(path): if path.startswith(unicode_prefix): return path - # os.path.abspath returns a normalized absolute path - return unicode_prefix + os.path.abspath(path) + # abspath returns a normalized absolute path + return unicode_prefix + abspath(path) def search_path(name): """Finds a file in a given search path.""" search_path = os.getenv('PATH', os.defpath).split(os.pathsep) for directory in search_path: if directory: - path = os.path.join(directory, name) + path = join(directory, name) if os.path.isfile(path) and os.access(path, os.X_OK): return path return None @@ -206,7 +207,7 @@ def find_binary(runfiles_root, bin_name): # Use normpath() to convert slashes to os.sep on Windows. elif os.sep in os.path.normpath(bin_name): # Case 3: Path is relative to the repo root. - return os.path.join(runfiles_root, bin_name) + return join(runfiles_root, bin_name) else: # Case 4: Path has to be looked up in the search path. return search_path(bin_name) @@ -226,7 +227,7 @@ def find_runfiles_root(main_rel_path): runfiles_dir = runfiles_manifest_file[:-9] # Be defensive: the runfiles dir should contain our main entry point. If # it doesn't, then it must not be our runfiles directory. - if runfiles_dir and os.path.exists(os.path.join(runfiles_dir, main_rel_path)): + if runfiles_dir and os.path.exists(join(runfiles_dir, main_rel_path)): return runfiles_dir # Clear RUNFILES_DIR & RUNFILES_MANIFEST_FILE since the runfiles dir was @@ -244,7 +245,7 @@ def find_runfiles_root(main_rel_path): stub_filename = stub_filename.replace("/", os.sep) if not os.path.isabs(stub_filename): - stub_filename = os.path.join(os.getcwd(), stub_filename) + stub_filename = join(os.getcwd(), stub_filename) while True: runfiles_root = stub_filename + ('.exe' if IS_WINDOWS else '') + '.runfiles' @@ -262,7 +263,7 @@ def find_runfiles_root(main_rel_path): if os.path.isabs(target): stub_filename = target else: - stub_filename = os.path.join(os.path.dirname(stub_filename), target) + stub_filename = join(os.path.dirname(stub_filename), target) raise AssertionError('Cannot find .runfiles directory for %s' % sys.argv[0]) @@ -285,7 +286,7 @@ def extract_zip(zip_path, dest_dir): zf.extract(info, dest_dir) # UNC-prefixed paths must be absolute/normalized. See # https://docs.microsoft.com/en-us/windows/desktop/fileio/naming-a-file#maximum-path-length-limitation - file_path = os.path.abspath(os.path.join(dest_dir, info.filename)) + file_path = abspath(join(dest_dir, info.filename)) # The Unix st_mode bits (see "man 7 inode") are stored in the upper 16 # bits of external_attr. attrs = info.external_attr >> 16 @@ -306,7 +307,27 @@ def create_runfiles_root(): extract_zip(os.path.dirname(__file__), temp_dir) # IMPORTANT: Later code does `rm -fr` on dirname(runfiles_root) -- it's # important that deletion code be in sync with this directory structure - return os.path.join(temp_dir, 'runfiles') + return join(temp_dir, 'runfiles') + +def _symlink_tree(link_from, link_to): + # Ensure the source is an absolute path to simplify symlinking + link_to_root = abspath(link_to) + link_from_root = abspath(link_from) + + # This is non-optimal because it recreates the entire + # venv site-packages tree (as opposed to finding a highest-common + # directory to symlink). But its easy and understandable. + for root, dirs, files in os.walk(link_to_root): + rel_path = os.path.relpath(root, link_to_root) + link_from_dir = join(link_from_root, rel_path) + + if not os.path.exists(link_from_dir): + os.makedirs(link_from_dir) + + for name in files: + link_to = join(root, name) + link_from = join(link_from_dir, name) + _symlink_exist_ok(from_=link_from, to=link_to) def _create_venv(runfiles_root, delete_dirs): rel_runfiles_venv = dirname(dirname(PYTHON_BINARY)) @@ -343,14 +364,14 @@ print(site.getsitepackages(["{venv_src}"])[-1]) output = output.strip().split("\n") python_exe_actual = output[0] python_home = output[1] if IS_WINDOWS else None - venv_site_packages = output[2] - os.makedirs(dirname(venv_site_packages), exist_ok=True) - runfiles_venv_site_packages = join(runfiles_venv, VENV_REL_SITE_PACKAGES) + venv_lib = output[2] + os.makedirs(dirname(venv_lib), exist_ok=True) + runfiles_venv_lib = join(runfiles_venv, VENV_REL_SITE_PACKAGES) else: # On unixy, Python can find home based on the symlink. python_home = dirname(python_exe_actual) if IS_WINDOWS else None - venv_site_packages = join(venv, "lib") - runfiles_venv_site_packages = join(runfiles_venv, "lib") + venv_lib = join(venv, LIB_DIR_NAME) + runfiles_venv_lib = join(runfiles_venv, LIB_DIR_NAME) venv_bin = join(venv, BIN_DIR_NAME) try: @@ -384,14 +405,16 @@ print(site.getsitepackages(["{venv_src}"])[-1]) target = join(runfiles_venv_bin, f_basename) _symlink_exist_ok(from_=venv_path, to=target) + # Recreate correct relative symlinks for venv_rel_path, link_to_rf_path in RUNTIME_VENV_SYMLINKS.items(): venv_abs_path = join(venv, venv_rel_path) - link_to = normpath(join(runfiles_venv, link_to_rf_path)) + link_to = normpath(join(runfiles_root, link_to_rf_path)) os.makedirs(dirname(venv_abs_path), exist_ok=True) _symlink_exist_ok(from_=venv_abs_path, to=link_to) - _symlink_exist_ok(from_=join(venv, "lib"), to=join(runfiles_venv, "lib")) - _symlink_exist_ok(from_=venv_site_packages, to=runfiles_venv_site_packages) + # Do this last to handle non-relative symlinks artifacts + if os.path.exists(runfiles_venv_lib): + _symlink_tree(link_from=venv_lib, link_to=runfiles_venv_lib) if IS_WINDOWS: print_verbose("create_venv: pyvenv.cfg home: ", python_home) @@ -436,7 +459,7 @@ def runfiles_envvar(runfiles_root): # Normally .runfiles_manifest and MANIFEST are both present, but the # former will be missing for zip-based builds or if someone copies the # runfiles tree elsewhere. - runfiles = os.path.join(runfiles_root, 'MANIFEST') + runfiles = join(runfiles_root, 'MANIFEST') if os.path.exists(runfiles): return ('RUNFILES_MANIFEST_FILE', runfiles) @@ -500,7 +523,7 @@ def execute_file(python_program, main_filename, args, env, runfiles_root, if delete_dirs: for delete_dir in delete_dirs: - print_verbose("rmtree:", delete_dir) + print_verbose("cleanup: rmtree:", delete_dir) shutil.rmtree(delete_dir, True) sys.exit(ret_code) @@ -520,13 +543,17 @@ def _run_execv(python_program, argv, env): def _symlink_exist_ok(*, from_, to): try: - os.symlink(to, from_) + # On Windows, symlinks have to be told whether they're + # pointing to a file or directory. Python is supposed to auto-detect + # this, but in practice, this doesn't reliably happen. + os.symlink(to, from_, target_is_directory=os.path.isdir(to)) except FileExistsError: pass def main(): print_verbose("sys.version:", sys.version) + print_verbose("sys.executable:", sys.executable) print_verbose("initial argv:", values=sys.argv) print_verbose("initial cwd:", os.getcwd()) print_verbose("initial environ:", mapping=os.environ) @@ -580,7 +607,7 @@ def main(): # See: https://docs.python.org/3.11/using/cmdline.html#envvar-PYTHONSAFEPATH new_env['PYTHONSAFEPATH'] = '1' - main_filename = os.path.join(runfiles_root, main_rel_path) + main_filename = join(runfiles_root, main_rel_path) main_filename = get_windows_path_with_unc_prefix(main_filename) assert os.path.exists(main_filename), \ 'Cannot exec() %r: file not found.' % main_filename @@ -616,7 +643,7 @@ def main(): # change directory to the right runfiles directory. # (So that the data files are accessible) if os.environ.get('RUN_UNDER_RUNFILES') == '1': - workspace = os.path.join(runfiles_root, WORKSPACE_NAME) + workspace = join(runfiles_root, WORKSPACE_NAME) try: sys.stdout.flush() diff --git a/python/private/site_init_template.py b/python/private/site_init_template.py index 97d16b71c2..e4d501bfd5 100644 --- a/python/private/site_init_template.py +++ b/python/private/site_init_template.py @@ -127,20 +127,19 @@ def _search_path(name): def _setup_sys_path(): - """Perform Bazel/binary specific sys.path setup. - - """ + """Perform Bazel/binary specific sys.path setup.""" + _print_verbose("site init: initial sys.path:\n", "\n".join(sys.path)) seen = set(sys.path) python_path_entries = [] - def _maybe_add_path(path): + def _maybe_add_path(path, reason): if path in seen: return path = _get_windows_path_with_unc_prefix(path) if _is_windows(): path = path.replace("/", os.sep) - _print_verbose("append sys.path:", path) + _print_verbose("append sys.path:", reason, ":", path) sys.path.append(path) seen.add(path) @@ -153,11 +152,11 @@ def _maybe_add_path(path): # For temporary compatibility with the original system_python bootstrap # behavior, it is conditionally added for that boostrap mode. if _ADD_RUNFILES_ROOT_TO_SYS_PATH: - _maybe_add_path(_RUNFILES_ROOT) + _maybe_add_path(_RUNFILES_ROOT, "runfiles-root") for rel_path in _IMPORTS_STR.split(":"): abs_path = os.path.join(_RUNFILES_ROOT, rel_path) - _maybe_add_path(abs_path) + _maybe_add_path(abs_path, "imports-strs") if _IMPORT_ALL: repo_dirs = sorted( @@ -165,9 +164,9 @@ def _maybe_add_path(path): ) for d in repo_dirs: if os.path.isdir(d): - _maybe_add_path(d) + _maybe_add_path(d, "import-all") else: - _maybe_add_path(os.path.join(_RUNFILES_ROOT, _WORKSPACE_NAME)) + _maybe_add_path(os.path.join(_RUNFILES_ROOT, _WORKSPACE_NAME), "workspace-root") # COVERAGE_DIR is set if coverage is enabled and instrumentation is configured # for something, though it could be another program executing this one or @@ -199,7 +198,7 @@ def _maybe_add_path(path): # it with the directory of the program it starts. Our actual sys.path[0] is # the runfiles directory, which must not be replaced. # CoverageScript.do_execute() undoes this sys.path[0] setting. - _maybe_add_path(coverage_dir) + _maybe_add_path(coverage_dir, "coverage-dir") coverage_setup = True else: _print_verbose_coverage( diff --git a/python/private/stage2_bootstrap_template.py b/python/private/stage2_bootstrap_template.py index dec356d3ac..2a7be67c13 100644 --- a/python/private/stage2_bootstrap_template.py +++ b/python/private/stage2_bootstrap_template.py @@ -53,6 +53,22 @@ # ===== Template substitutions end ===== +IS_WINDOWS = os.name == "nt" +IS_VERBOSE = bool(os.environ.get("RULES_PYTHON_BOOTSTRAP_VERBOSE")) + +# Windows APIs can be picky about slashes depending on the context, +# so convert to backslashes to avoid any issues. +# Related: some logic checks path strings, which needs uniform separators. +if IS_WINDOWS: + + def norm_slashes(s): + return s.replace("/", "\\") + + MAIN_PATH = norm_slashes(MAIN_PATH) + VENV_ROOT = norm_slashes(VENV_ROOT) + VENV_SITE_PACKAGES = norm_slashes(VENV_SITE_PACKAGES) + BUILD_DATA_FILE = norm_slashes(BUILD_DATA_FILE) + class BazelBinaryInfoModule(types.ModuleType): BUILD_DATA_FILE = BUILD_DATA_FILE @@ -84,8 +100,6 @@ def get_build_data(self): sys.modules["bazel_binary_info"] = BazelBinaryInfoModule("bazel_binary_info") -IS_WINDOWS = os.name == "nt" - def get_windows_path_with_unc_prefix(path): path = path.strip() @@ -124,32 +138,29 @@ def get_windows_path_with_unc_prefix(path): return unicode_prefix + os.path.abspath(path) -def is_verbose(): - return bool(os.environ.get("RULES_PYTHON_BOOTSTRAP_VERBOSE")) - - def print_verbose(*args, mapping=None, values=None): - if is_verbose(): - if mapping is not None: - for key, value in sorted((mapping or {}).items()): - print( - "bootstrap: stage 2:", - *args, - f"{key}={value!r}", - file=sys.stderr, - flush=True, - ) - elif values is not None: - for i, v in enumerate(values): - print( - "bootstrap: stage 2:", - *args, - f"[{i}] {v!r}", - file=sys.stderr, - flush=True, - ) - else: - print("bootstrap: stage 2:", *args, file=sys.stderr, flush=True) + if not IS_VERBOSE: + return + if mapping is not None: + for key, value in sorted((mapping or {}).items()): + print( + "bootstrap: stage 2:", + *args, + f"{key}={value!r}", + file=sys.stderr, + flush=True, + ) + elif values is not None: + for i, v in enumerate(values): + print( + "bootstrap: stage 2:", + *args, + f"[{i}] {v!r}", + file=sys.stderr, + flush=True, + ) + else: + print("bootstrap: stage 2:", *args, file=sys.stderr, flush=True) def print_verbose_coverage(*args): @@ -160,7 +171,7 @@ def print_verbose_coverage(*args): def is_verbose_coverage(): """Returns True if VERBOSE_COVERAGE is non-empty in the environment.""" - return os.environ.get("VERBOSE_COVERAGE") or is_verbose() + return os.environ.get("VERBOSE_COVERAGE") or IS_VERBOSE def find_runfiles_root(main_rel_path): @@ -419,12 +430,32 @@ def _maybe_collect_coverage(enable): def _add_site_packages(site_packages): + if sys.prefix != sys.base_prefix: + venv_root = sys.prefix + os.sep + saw_venv_site_packages = False + else: + venv_root = None + saw_venv_site_packages = True first_global_offset = len(sys.path) for i, p in enumerate(sys.path): + # Handle the Windows venv case: when a temporary directory is created + # for the venv, we want the build-time venv in runfiles to come after + # the venv site-packages directory. + if venv_root: + is_venv_path = (p + os.sep).startswith(venv_root) + if is_venv_path: + if p.endswith("site-packages"): + saw_venv_site_packages = True + is_after_venv_site_packages = False + else: + is_after_venv_site_packages = saw_venv_site_packages + else: + is_after_venv_site_packages = True + # We assume the first *-packages is the runtime's. # *-packages is matched because Debian may use dist-packages # instead of site-packages. - if p.endswith("-packages"): + if p.endswith("-packages") and is_after_venv_site_packages: first_global_offset = i break prev_len = len(sys.path) diff --git a/python/private/venv_runfiles.bzl b/python/private/venv_runfiles.bzl index a492181c88..6daf0d4e5c 100644 --- a/python/private/venv_runfiles.bzl +++ b/python/private/venv_runfiles.bzl @@ -3,7 +3,9 @@ load("@bazel_skylib//lib:paths.bzl", "paths") load( ":common.bzl", + "ExplicitSymlink", "is_file", + "is_windows_platform", "relative_path", "runfiles_root_path", ) @@ -58,6 +60,14 @@ def create_venv_app_files(ctx, deps, venv_dir_map): link_map = build_link_map(ctx, entries) venv_files = [] runfiles_symlinks = {} + explicit_symlinks = [] + + is_windows = is_windows_platform(ctx) + + ctx_rf_path = paths.join( + ctx.label.repo_name or ctx.workspace_name, + ctx.label.package, + ) for kind, kind_map in link_map.items(): base = venv_dir_map[kind] @@ -70,7 +80,7 @@ def create_venv_app_files(ctx, deps, venv_dir_map): symlink_from = paths.join(runfile_prefix, ctx.label.package, bin_venv_path) runfiles_symlinks[symlink_from] = link_to - else: + elif not is_windows: venv_link = ctx.actions.declare_symlink(bin_venv_path) venv_link_rf_path = runfiles_root_path(ctx, venv_link.short_path) rel_path = relative_path( @@ -81,10 +91,20 @@ def create_venv_app_files(ctx, deps, venv_dir_map): ) ctx.actions.symlink(output = venv_link, target_path = rel_path) venv_files.append(venv_link) + else: + rf_path = paths.join(ctx_rf_path, bin_venv_path) + _, _, venv_path = bin_venv_path.partition(".venv/") + explicit_symlinks.append(ExplicitSymlink( + runfiles_path = rf_path, + venv_path = venv_path, + link_to_path = link_to, + files = depset(), + )) return struct( venv_files = venv_files, runfiles_symlinks = runfiles_symlinks, + explicit_symlinks = depset(explicit_symlinks), ) # Visible for testing diff --git a/python/private/zipapp/py_zipapp_rule.bzl b/python/private/zipapp/py_zipapp_rule.bzl index f26b050165..4d31c99311 100644 --- a/python/private/zipapp/py_zipapp_rule.bzl +++ b/python/private/zipapp/py_zipapp_rule.bzl @@ -4,7 +4,16 @@ load("@bazel_skylib//lib:paths.bzl", "paths") load("@rules_python_internal//:rules_python_config.bzl", rp_config = "config") load("//python/private:attributes.bzl", "apply_config_settings_attr") load("//python/private:builders.bzl", "builders") -load("//python/private:common.bzl", "BUILTIN_BUILD_PYTHON_ZIP", "actions_run", "create_windows_exe_launcher", "maybe_builtin_build_python_zip", "maybe_create_repo_mapping", "runfiles_root_path", "target_platform_has_any_constraint") +load( + "//python/private:common.bzl", + "BUILTIN_BUILD_PYTHON_ZIP", + "actions_run", + "create_windows_exe_launcher", + "is_windows_platform", + "maybe_builtin_build_python_zip", + "maybe_create_repo_mapping", + "runfiles_root_path", +) load("//python/private:common_labels.bzl", "labels") load("//python/private:py_executable_info.bzl", "PyExecutableInfo") load("//python/private:py_internal.bzl", "py_internal") @@ -135,18 +144,22 @@ def _create_zip(ctx, py_runtime, py_executable, stage2_bootstrap): runfiles = runfiles.build(ctx) + explicit_symlinks = depset(transitive = [ + py_executable.venv_interpreter_symlinks, + py_executable.venv_app_symlinks, + ]) zip_main = _create_zipapp_main_py( ctx, py_runtime, py_executable, stage2_bootstrap, runfiles, - py_executable.venv_interpreter_symlinks, + explicit_symlinks, ) inputs = builders.DepsetBuilder() manifest.add("regular|0|__main__.py|{}".format(zip_main.path)) inputs.add(zip_main) - _build_manifest(ctx, manifest, runfiles, py_executable.venv_interpreter_symlinks, inputs) + _build_manifest(ctx, manifest, runfiles, explicit_symlinks, inputs) zipper_args = ctx.actions.args() zipper_args.add(output) @@ -159,7 +172,7 @@ def _create_zip(ctx, py_runtime, py_executable, stage2_bootstrap): zipper_args.add(ctx.attr.compression, format = "--compression=%s") zipper_args.add("--runfiles-dir=runfiles") - is_windows = target_platform_has_any_constraint(ctx, ctx.attr._windows_constraints) + is_windows = is_windows_platform(ctx) zipper_args.add("\\" if is_windows else "/", format = "--target-platform-pathsep=%s") actions_run( @@ -230,7 +243,7 @@ def _py_zipapp_executable_impl(ctx): zip_file = _create_zip(ctx, py_runtime, py_executable, stage2_bootstrap) if ctx.attr.executable: - if target_platform_has_any_constraint(ctx, ctx.attr._windows_constraints): + if is_windows_platform(ctx): executable = ctx.actions.declare_file(ctx.label.name + ".exe") # The zipapp is an opaque zip file, so the Bazel Python launcher doesn't diff --git a/python/private/zipapp/zip_main_template.py b/python/private/zipapp/zip_main_template.py index 6d12bc9c08..06ed19f1a5 100644 --- a/python/private/zipapp/zip_main_template.py +++ b/python/private/zipapp/zip_main_template.py @@ -29,7 +29,7 @@ import subprocess import tempfile import zipfile -from os.path import basename, dirname, join +from os.path import basename, dirname, join, normpath # runfiles-root-relative path _STAGE2_BOOTSTRAP = "%stage2_bootstrap%" @@ -47,8 +47,6 @@ IS_WINDOWS = os.name == "nt" -EXTRACT_ROOT = os.environ.get("RULES_PYTHON_EXTRACT_ROOT") - # Change the paths with Unix-style forward slashes to backslashes for Windows. # Windows usually transparently rewrites them, but e.g. `\\?\` paths require # backslashes to be properly understood by Windows APIs. @@ -65,9 +63,11 @@ def norm_slashes(s): EXTRACT_DIR = norm_slashes(EXTRACT_DIR) EXTRACT_ROOT = norm_slashes(EXTRACT_ROOT) +IS_VERBOSE = bool(os.environ.get("RULES_PYTHON_BOOTSTRAP_VERBOSE")) + def print_verbose(*args, mapping=None, values=None): - if not bool(os.environ.get("RULES_PYTHON_BOOTSTRAP_VERBOSE")): + if not IS_VERBOSE: return if mapping is not None: for key, value in sorted((mapping or {}).items()): @@ -150,7 +150,7 @@ def find_binary(runfiles_root, bin_name): # Case 2: Absolute path. return bin_name # Use normpath() to convert slashes to os.sep on Windows. - elif os.sep in os.path.normpath(bin_name): + elif os.sep in normpath(bin_name): # Case 3: Path is relative to the repo root. return join(runfiles_root, bin_name) else: @@ -194,7 +194,19 @@ def extract_zip(zip_path, dest_dir): with open(file_path, "r") as f: target = f.read() os.remove(file_path) - os.symlink(target, file_path) + if IS_WINDOWS: + entry_path = normpath(join(dirname(info.filename), target)) + # Zip lookup uses forward slashes, target has backslashes. + entry_path = entry_path.replace("\\", "/") + try: + target_is_directory = zf.getinfo(entry_path).is_dir() + except KeyError: + # Directories aren't stored in zips, so a missing + # target means it points to a directory. + target_is_directory = True + else: + target_is_directory = False + os.symlink(target, file_path, target_is_directory=target_is_directory) # Of those, we set the lower 12 bits, which are the # file mode bits (since the file type bits can't be set by chmod anyway). elif attrs != 0: # Rumor has it these can be 0 for zips created on Windows. @@ -214,7 +226,9 @@ def create_runfiles_root(): extract_root = get_windows_path_with_unc_prefix(extract_root) else: extract_root = tempfile.mkdtemp("", "Bazel.runfiles_") + extract_zip(dirname(__file__), extract_root) + print_verbose("extracted to:", extract_root) # IMPORTANT: Later code does `rm -fr` on dirname(runfiles_root) -- it's # important that deletion code be in sync with this directory structure return join(extract_root, "runfiles") @@ -314,10 +328,10 @@ def finish_venv_setup(runfiles_root): def main(): print_verbose("running zip main bootstrap") + print_verbose("initial sys.version:", sys.version) + print_verbose("initial sys.executable:", sys.executable) print_verbose("initial argv:", values=sys.argv) print_verbose("initial environ:", mapping=os.environ) - print_verbose("initial sys.executable:", sys.executable) - print_verbose("initial sys.version:", sys.version) print_verbose("stage2_bootstrap:", _STAGE2_BOOTSTRAP) print_verbose("python_binary_venv:", _PYTHON_BINARY_VENV) print_verbose("python_binary_actual:", _PYTHON_BINARY_ACTUAL) diff --git a/tests/py_zipapp/BUILD.bazel b/tests/py_zipapp/BUILD.bazel index fc1809b69f..a68448d964 100644 --- a/tests/py_zipapp/BUILD.bazel +++ b/tests/py_zipapp/BUILD.bazel @@ -4,24 +4,25 @@ load("//python:py_library.bzl", "py_library") load("//python:py_test.bzl", "py_test") load("//python/private:bzlmod_enabled.bzl", "BZLMOD_ENABLED") # buildifier: disable=bzl-visibility load("//python/zipapp:py_zipapp_binary.bzl", "py_zipapp_binary") -load("//tests/support:support.bzl", "NOT_WINDOWS") py_binary( name = "venv_bin", srcs = ["main.py"], config_settings = { - "//python/config_settings:bootstrap_impl": "script", "//python/config_settings:venvs_site_packages": "yes", - }, + } | select({ + "@platforms//os:windows": {}, + "//conditions:default": { + "//python/config_settings:bootstrap_impl": "script", + }, + }), main = "main.py", - target_compatible_with = NOT_WINDOWS, - deps = [":some_dep"], + deps = [":bin_deps"], ) py_zipapp_binary( name = "venv_zipapp", binary = ":venv_bin", - target_compatible_with = NOT_WINDOWS, ) py_test( @@ -32,7 +33,6 @@ py_test( "BZLMOD_ENABLED": str(int(BZLMOD_ENABLED)), "TEST_ZIPAPP": "$(location :venv_zipapp)", }, - target_compatible_with = NOT_WINDOWS, ) # Create the app with a supported level of compression @@ -40,7 +40,6 @@ py_zipapp_binary( name = "venv_zipapp_compressed", binary = ":venv_bin", compression = "4", - target_compatible_with = NOT_WINDOWS, ) py_test( @@ -53,7 +52,6 @@ py_test( "TEST_ZIPAPP": "$(location :venv_zipapp_compressed)", }, main = "venv_zipapp_test.py", - target_compatible_with = NOT_WINDOWS, ) py_binary( @@ -64,7 +62,7 @@ py_binary( "//python/config_settings:venvs_site_packages": "no", }, main = "main.py", - deps = [":some_dep"], + deps = [":bin_deps"], ) py_zipapp_binary( @@ -95,9 +93,24 @@ sh_test( toolchains = ["//python:current_py_toolchain"], ) +py_library( + name = "bin_deps", + deps = [ + ":pkgdep", + ":some_dep", + ], +) + py_library( name = "some_dep", srcs = ["some_dep.py"], experimental_venvs_site_packages = "//python/config_settings:venvs_site_packages", imports = ["."], ) + +py_library( + name = "pkgdep", + srcs = glob(["site-packages/**"]), + experimental_venvs_site_packages = "//python/config_settings:venvs_site_packages", + imports = ["site-packages"], +) diff --git a/tests/py_zipapp/main.py b/tests/py_zipapp/main.py index 8e67ec9fae..5770170d2c 100644 --- a/tests/py_zipapp/main.py +++ b/tests/py_zipapp/main.py @@ -7,13 +7,16 @@ def main(): import some_dep print(f"dep: {some_dep}") - except ImportError: + + import pkgdep.pkgmod + + print(f"dep: {pkgdep.pkgmod}") + except ImportError as e: import sys - print("Failed to import `some_dep`", file=sys.stderr) - print("sys.path:", file=sys.stderr) - for i, x in enumerate(sys.path): - print(i, x, file=sys.stderr) + e.add_note( + "Failed to import a dependency.\n" + "sys.path:\n" + "\n".join(sys.path) + ) raise diff --git a/tests/py_zipapp/site-packages/pkgdep/__init__.py b/tests/py_zipapp/site-packages/pkgdep/__init__.py new file mode 100644 index 0000000000..e69de29bb2 diff --git a/tests/py_zipapp/site-packages/pkgdep/pkgmod.py b/tests/py_zipapp/site-packages/pkgdep/pkgmod.py new file mode 100644 index 0000000000..e69de29bb2 diff --git a/tests/py_zipapp/system_python_zipapp_test.py b/tests/py_zipapp/system_python_zipapp_test.py index ec0f837135..79cd142c4d 100644 --- a/tests/py_zipapp/system_python_zipapp_test.py +++ b/tests/py_zipapp/system_python_zipapp_test.py @@ -11,7 +11,20 @@ def test_zipapp_runnable(self): self.assertTrue(os.path.exists(zipapp_path)) self.assertTrue(os.path.isfile(zipapp_path)) - output = subprocess.check_output([zipapp_path]).decode("utf-8").strip() + try: + output = ( + subprocess.check_output([zipapp_path], stderr=subprocess.STDOUT) + .decode("utf-8") + .strip() + ) + except subprocess.CalledProcessError as e: + self.fail( + "exit code: {}\n" + " command: {}\n" + "===== stdout/stderr start ==={}===== stdout/stderr end ====".format( + e.returncode, e.cmd, e.output.decode("utf-8") + ) + ) self.assertIn("Hello from zipapp", output) self.assertIn("dep:", output) diff --git a/tests/py_zipapp/venv_zipapp_test.py b/tests/py_zipapp/venv_zipapp_test.py index fec4a544bd..bd26d533a3 100644 --- a/tests/py_zipapp/venv_zipapp_test.py +++ b/tests/py_zipapp/venv_zipapp_test.py @@ -82,20 +82,22 @@ def test_zipapp_structure(self): if self._is_bzlmod_enabled(): self.assertIn("runfiles/_repo_mapping", namelist) - self.assertHasPathMatchingSuffix(namelist, "/pyvenv.cfg") - - # The venv directory name depends on the target name, so find it - # by looking for pyvenv.cfg. - venv_config = next( - (name for name in namelist if name.endswith("/pyvenv.cfg")), None - ) - self.assertIsNotNone(venv_config) - - venv_root = os.path.dirname(venv_config) - - # Verify bin/python3 exists and is a symlink - python_bin = f"{venv_root}/bin/python3" - self.assertZipEntryIsSymlink(zf, python_bin) + # On Windows, pyvenv.cfg and bin/python3 are generated at runtime. + if os.name != "nt": + self.assertHasPathMatchingSuffix(namelist, "/pyvenv.cfg") + + # The venv directory name depends on the target name, so find it + # by looking for pyvenv.cfg. + venv_config = next( + (name for name in namelist if name.endswith("/pyvenv.cfg")), None + ) + self.assertIsNotNone(venv_config) + + venv_root = os.path.dirname(venv_config) + + # Verify bin/python3 exists and is a symlink + python_bin = f"{venv_root}/bin/python3" + self.assertZipEntryIsSymlink(zf, python_bin) # Verify _bazel_site_init.py exists in site-packages self.assertHasPathMatchingSuffix( diff --git a/tests/venv_site_packages_libs/BUILD.bazel b/tests/venv_site_packages_libs/BUILD.bazel index e573dc6da6..56f0eb0909 100644 --- a/tests/venv_site_packages_libs/BUILD.bazel +++ b/tests/venv_site_packages_libs/BUILD.bazel @@ -1,11 +1,6 @@ load("@rules_shell//shell:sh_test.bzl", "sh_test") load("//python:py_library.bzl", "py_library") load("//tests/support:py_reconfig.bzl", "py_reconfig_test") -load( - "//tests/support:support.bzl", - "NOT_WINDOWS", - "SUPPORTS_BOOTSTRAP_SCRIPT", -) py_library( name = "user_lib", @@ -31,7 +26,6 @@ sh_test( env = { "VENV_BIN": "$(rootpath @other//:venv_bin)", }, - target_compatible_with = NOT_WINDOWS, ) py_reconfig_test( @@ -39,7 +33,6 @@ py_reconfig_test( srcs = ["bin.py"], bootstrap_impl = "script", main = "bin.py", - target_compatible_with = SUPPORTS_BOOTSTRAP_SCRIPT, venvs_site_packages = "yes", deps = [ ":closer_lib", @@ -58,13 +51,20 @@ py_reconfig_test( py_reconfig_test( name = "shared_lib_loading_test", srcs = ["shared_lib_loading_test.py"], - bootstrap_impl = "script", + bootstrap_impl = select({ + "@platforms//os:windows": "system_python", + "//conditions:default": "script", + }), main = "shared_lib_loading_test.py", - target_compatible_with = NOT_WINDOWS, venvs_site_packages = "yes", - deps = [ - "//tests/venv_site_packages_libs/ext_with_libs", - "@dev_pip//macholib", - "@dev_pip//pyelftools", - ], + deps = select({ + "@platforms//os:windows": [ + "@dev_pip//markupsafe", + ], + "//conditions:default": [ + "//tests/venv_site_packages_libs/ext_with_libs", + "@dev_pip//macholib", + "@dev_pip//pyelftools", + ], + }), ) diff --git a/tests/venv_site_packages_libs/bin.py b/tests/venv_site_packages_libs/bin.py index c075f4fc65..439a964906 100644 --- a/tests/venv_site_packages_libs/bin.py +++ b/tests/venv_site_packages_libs/bin.py @@ -22,8 +22,10 @@ def assert_imported_from_venv(self, module_name): self.assertTrue( module.__file__.startswith(self.venv), f"\n{module_name} was imported, but not from the venv.\n" - + f"venv : {self.venv}\n" - + f"actual: {module.__file__}", + + f" venv: {self.venv}\n" + + f"module file: {module.__file__}\n" + + "sys.path:\n" + + "\n".join(sys.path), ) return module diff --git a/tests/venv_site_packages_libs/shared_lib_loading_test.py b/tests/venv_site_packages_libs/shared_lib_loading_test.py index 2b58f8571c..a3f7bfcd5a 100644 --- a/tests/venv_site_packages_libs/shared_lib_loading_test.py +++ b/tests/venv_site_packages_libs/shared_lib_loading_test.py @@ -1,10 +1,21 @@ import importlib.util import os +import sys import unittest +from pathlib import Path -from elftools.elf.elffile import ELFFile -from macholib import mach_o -from macholib.MachO import MachO +# Optional imports for ELF/Mach-O analysis +if os.name == "posix" and sys.platform != "darwin": + from elftools.elf.elffile import ELFFile +else: + ELFFile = None + +if sys.platform == "darwin": + from macholib import mach_o + from macholib.MachO import MachO +else: + mach_o = None + MachO = None ELF_MAGIC = b"\x7fELF" MACHO_MAGICS = ( @@ -16,7 +27,14 @@ class SharedLibLoadingTest(unittest.TestCase): - def test_shared_library_linking(self): + def setUp(self): + super().setUp() + if sys.prefix == sys.base_prefix: + raise AssertionError("Not running under a venv") + self.venv = Path(sys.prefix) + + @unittest.skipIf(os.name == "nt", "Tests Unix-specific extension loading") + def test_shared_library_linking_unix(self): try: import ext_with_libs.adder except ImportError as e: @@ -53,6 +71,41 @@ def test_shared_library_linking(self): # Check the function works regardless of format. self.assertEqual(ext_with_libs.adder.do_add(), 2) + @unittest.skipUnless(os.name == "nt", "Tests Windows-specific extension loading") + def test_shared_library_loading_windows(self): + # We import markupsafe._speedups (a .cp311-win_amd64.pyd extension) + try: + import markupsafe._speedups + + module = markupsafe._speedups + except ImportError as e: + self.fail( + f"Failed to import markupsafe._speedups: {e}\n" + + "sys.path:\n" + + "\n".join(sys.path) + ) + + # Verify it's in the venv + # Normalize paths for Windows comparison. + # We DON'T use resolve() here because we want to see the path Python used, + # which should be within the venv's site-packages (even if it's a symlink). + actual_file = str(Path(module.__file__)).lower() + expected_prefix = str(self.venv).lower() + + self.assertTrue( + actual_file.startswith(expected_prefix), + f"Module {module.__name__} not loaded from venv.\n" + f"Venv: {expected_prefix}\n" + f"Module file: {actual_file}\n" + f"sys.path:\n" + "\n".join(sys.path), + ) + + # Verify it's a shared library (.pyd) + self.assertTrue( + actual_file.endswith(".pyd"), + f"Expected .pyd extension, got {module.__file__}", + ) + def _get_linking_info(self, path): """Parses a shared library and returns its rpaths and dependencies.""" path = os.path.realpath(path) From afab5ed2a6bc587676d9ab4022bccf977cc8566a Mon Sep 17 00:00:00 2001 From: Ignas Anikevicius <240938+aignas@users.noreply.github.com> Date: Tue, 21 Apr 2026 21:34:40 +0900 Subject: [PATCH 707/922] doc: add notes on the design (#3722) Just to start a page on design notes to tell users better what they are getting. --------- Co-authored-by: Richard Levasseur --- README.md | 48 ++++++++++++++++++++++++++++++++++++++++++++++-- 1 file changed, 46 insertions(+), 2 deletions(-) diff --git a/README.md b/README.md index e79c4a9673..7d805ce315 100644 --- a/README.md +++ b/README.md @@ -6,7 +6,8 @@ This repository is the home of the core Python rules -- `py_library`, `py_binary`, `py_test`, and related symbols that provide the basis for Python -support in Bazel. It also contains package installation rules for integrating with PyPI and other indices. +support in Bazel. It also contains package installation rules for integrating +with PyPI and other indices. Documentation for rules_python is at and in the [Bazel Build Encyclopedia](https://docs.bazel.build/versions/master/be/python.html). @@ -17,7 +18,50 @@ The core rules are stable. Their implementation is subject to Bazel's [backward compatibility policy](https://docs.bazel.build/versions/master/backward-compatibility.html). This repository aims to follow [semantic versioning](https://semver.org). -The Bazel community maintains this repository. Neither Google nor the Bazel team provides support for the code. However, this repository is part of the test suite used to vet new Bazel releases. See [How to contribute](CONTRIBUTING.md) page for information on our development workflow. +The Bazel community maintains this repository. Neither Google nor the Bazel team +provides support for the code. However, this repository is part of the test +suite used to vet new Bazel releases. See [How to contribute](CONTRIBUTING.md) +page for information on our development workflow. + +## Design + +* Supported OSes - as per our supported platform policy, we strive for support + on all of the platforms that we have CI for. Some platforms do not have the + same backwards compatibility guarantees, but we hope the community can step in + where needed to make the support more robust. +* `requirements.txt` is how users have been defining dependencies for a long + time. We support this to support legacy usecases or package managers that we + don't support directly. Any additional information that we need will be + retrieved from the SimpleAPI during the `bzlmod` extension evaluation phase. + Then it will be written to the `MODULE.bazel.lock` file for future reuse. We + have plans to support `uv.lock` file directly. `uv` is recommended for + generating a fully locked `requirements.txt` file and we do provide a rule for + it. +* The `py_binary`, `py_test` rules should scale to large monorepos and we work + hard to minimize the work done during analysis and build phase. What is more, + the space requirements for should be minimal, so we strive to use symlinks + rather than extracting wheels at build time. This means that for different + configurations of the same build, we are not extracting the wheel multiple + times thus scaling better over the time. From `2.0` onwards we are creating a + virtual env for each target by creating an actual minimal virtual environment + using symlinks. We plan on creating the traditional `site-packages` layout in + the future by default. +* Support for standards - we strive to first implement any standards needed + within `rules_python` and this has resulted in a few PEPs supported within + pure starlark - PEP440, PEP509. + +Common misconceptions: +* `rules_python` has to keep backwards compatibility with `google3`. Whilst this + might have been true in the past, `rules_python` is an open source project and + any compatibility needs should come from the community - we have no + requirement to keep this compatibility and are allowed to make our decisions. + However, we do want to keep backwards compatibility as long as possible to not + upset users with never ending migrations. +* `rules_python` is not caching pip downloads. With 2.0, we use Bazel's + downloader by default and rely on bazel to provide the repository caching + mechanisms. This means that for simpler setups this should result in + transparent and scalable caching with the most recent bazel versions unless + there are issues in the bazel itself. ## Documentation From 1c8ca73692f1513a635fc3583cf845c6755acf77 Mon Sep 17 00:00:00 2001 From: Douglas Thor Date: Tue, 21 Apr 2026 16:11:16 -0700 Subject: [PATCH 708/922] deps(gazelle): Finish bazel-gazelle version bump from #3717 (#3724) I forgot some things :laughing: --- gazelle/WORKSPACE | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/gazelle/WORKSPACE b/gazelle/WORKSPACE index f4a3abf36c..ebb18e7a47 100644 --- a/gazelle/WORKSPACE +++ b/gazelle/WORKSPACE @@ -26,10 +26,10 @@ http_archive( http_archive( name = "bazel_gazelle", - sha256 = "75df288c4b31c81eb50f51e2e14f4763cb7548daae126817247064637fd9ea62", + sha256 = "675114d8b433d0a9f54d81171833be96ebc4113115664b791e6f204d58e93446", urls = [ - "https://mirror.bazel.build/github.com/bazelbuild/bazel-gazelle/releases/download/v0.36.0/bazel-gazelle-v0.36.0.tar.gz", - "https://github.com/bazelbuild/bazel-gazelle/releases/download/v0.36.0/bazel-gazelle-v0.36.0.tar.gz", + "https://mirror.bazel.build/github.com/bazelbuild/bazel-gazelle/releases/download/v0.47.0/bazel-gazelle-v0.47.0.tar.gz", + "https://github.com/bazelbuild/bazel-gazelle/releases/download/v0.47.0/bazel-gazelle-v0.47.0.tar.gz", ], ) @@ -38,7 +38,7 @@ load("@io_bazel_rules_go//go:deps.bzl", "go_register_toolchains", "go_rules_depe go_rules_dependencies() -go_register_toolchains(version = "1.21.13") +go_register_toolchains(version = "1.22.9") gazelle_dependencies() From 2e178c202905bd485470c36c45f41c5937da476d Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Wed, 22 Apr 2026 20:43:28 -0700 Subject: [PATCH 709/922] feat(runfiles): implement runfiles.Path io methods (#3716) The runfiles.Path class is currently a PurePath subclass, which means it only does string manipulation for constructing paths. This change adds functionality to actually access files. Fixes https://github.com/bazel-contrib/rules_python/issues/3296 --- python/runfiles/runfiles.py | 221 ++++++++++++++++++++++++++------- tests/runfiles/BUILD.bazel | 22 ++++ tests/runfiles/pathlib_test.py | 128 ++++++++++++++++--- 3 files changed, 305 insertions(+), 66 deletions(-) diff --git a/python/runfiles/runfiles.py b/python/runfiles/runfiles.py index f7be6ef42f..03ebdf6331 100644 --- a/python/runfiles/runfiles.py +++ b/python/runfiles/runfiles.py @@ -29,7 +29,16 @@ import posixpath import sys from collections import defaultdict -from typing import Dict, List, Optional, Tuple, Union +from typing import Dict, Generator, Iterable, List, Optional, Tuple, Union + +if sys.version_info >= (3, 11): + from typing import Self +elif sys.version_info >= (3, 10): + from typing import TypeAlias + + Self: TypeAlias = "Path" # type: ignore +else: + from typing import Any as Self class _RepositoryMapping: @@ -143,25 +152,30 @@ def is_empty(self) -> bool: ) -class Path(pathlib.PurePath): +class Path(pathlib.Path): """A pathlib-like path object for runfiles. - This class extends `pathlib.PurePath` and resolves paths + This class extends `pathlib.Path` and resolves paths using the associated `Runfiles` instance when converted to a string. """ - # For Python < 3.12 compatibility when subclassing PurePath directly - _flavour = getattr(type(pathlib.PurePath()), "_flavour", None) + # Mypy isn't smart enough to realize `self` in the methods + # refers to our Path class instead of pathlib.Path + _runfiles: Optional["Runfiles"] + _source_repo: Optional[str] + + # For Python < 3.12 compatibility when subclassing Path directly + _flavour = getattr(type(pathlib.Path()), "_flavour", None) def __new__( cls, *args: Union[str, os.PathLike], runfiles: Optional["Runfiles"] = None, source_repo: Optional[str] = None, - ) -> "Path": + ) -> Self: """Private constructor. Use Runfiles.root() to create instances.""" obj = super().__new__(cls, *args) - # Type checkers might complain about adding attributes to PurePath, + # Type checkers might complain about adding attributes to Path, # but this is standard for pathlib subclasses. obj._runfiles = runfiles # type: ignore obj._source_repo = source_repo # type: ignore @@ -173,77 +187,188 @@ def __init__( runfiles: Optional["Runfiles"] = None, source_repo: Optional[str] = None, ) -> None: - pass + # In Python 3.12+, pathlib was refactored and Path.__init__ now accepts + # *args. Prior to 3.12, Path did not define __init__, so + # super().__init__(*args) would fall through to object.__init__, which + # raises a TypeError because it takes no arguments. + if sys.version_info >= (3, 12): + super().__init__(*args) + else: + super().__init__() + + # We override resolve() and absolute() to ensure that in Python < 3.12, + # where pathlib internally uses object.__new__ instead of our custom + # __new__ or with_segments(), the runfiles state is preserved. We delegate + # to self._as_path() because super().resolve() creates intermediate objects + # that would otherwise crash during internal stat() calls. + # override + def resolve(self, strict: bool = False) -> Self: + return type(self)( + self._as_path().resolve(strict=strict), + runfiles=self._runfiles, + source_repo=self._source_repo, + ) - def with_segments(self, *pathsegments: Union[str, os.PathLike]) -> "Path": + # override + def absolute(self) -> Self: + return type(self)( + self._as_path().absolute(), + runfiles=self._runfiles, + source_repo=self._source_repo, + ) + + # override + def with_segments(self, *pathsegments: Union[str, os.PathLike]) -> Self: """Used by Python 3.12+ pathlib to create new path objects.""" return type(self)( *pathsegments, - runfiles=self._runfiles, # type: ignore - source_repo=self._source_repo, # type: ignore + runfiles=self._runfiles, + source_repo=self._source_repo, ) # For Python < 3.12 - @classmethod - def _from_parts(cls, args: Tuple[str, ...]) -> "Path": - obj = super()._from_parts(args) # type: ignore - # These will be set by the calling instance later, or we can't set them here - # properly without context. Usually pathlib calls this from an instance - # method like _make_child, which we also might need to override. - return obj - - def _make_child(self, args: Tuple[str, ...]) -> "Path": + # override + def _make_child(self, args: Tuple[str, ...]) -> Self: obj = super()._make_child(args) # type: ignore obj._runfiles = self._runfiles # type: ignore obj._source_repo = self._source_repo # type: ignore return obj - @classmethod - def _from_parsed_parts(cls, drv: str, root: str, parts: List[str]) -> "Path": - obj = super()._from_parsed_parts(drv, root, parts) # type: ignore - return obj - - def _make_child_relpath(self, part: str) -> "Path": - obj = super()._make_child_relpath(part) # type: ignore - obj._runfiles = self._runfiles # type: ignore - obj._source_repo = self._source_repo # type: ignore - return obj - + # override @property - def parents(self) -> Tuple["Path", ...]: + def parents(self) -> Tuple[Self, ...]: return tuple( type(self)( p, - runfiles=getattr(self, "_runfiles", None), - source_repo=getattr(self, "_source_repo", None), + runfiles=self._runfiles, + source_repo=self._source_repo, ) for p in super().parents ) + # override @property - def parent(self) -> "Path": + def parent(self) -> Self: return type(self)( super().parent, - runfiles=getattr(self, "_runfiles", None), - source_repo=getattr(self, "_source_repo", None), + runfiles=self._runfiles, + source_repo=self._source_repo, ) - def with_name(self, name: str) -> "Path": + @property + def runfile_path(self) -> str: + """Returns the runfiles-root relative path.""" + path_posix = super().__str__().replace("\\", "/") + if path_posix == ".": + return "" + return path_posix + + # override + def with_name(self, name: str) -> Self: return type(self)( super().with_name(name), - runfiles=getattr(self, "_runfiles", None), - source_repo=getattr(self, "_source_repo", None), + runfiles=self._runfiles, + source_repo=self._source_repo, ) - def with_suffix(self, suffix: str) -> "Path": + # override + def with_suffix(self, suffix: str) -> Self: return type(self)( super().with_suffix(suffix), - runfiles=getattr(self, "_runfiles", None), - source_repo=getattr(self, "_source_repo", None), + runfiles=self._runfiles, + source_repo=self._source_repo, + ) + + def _as_path(self) -> pathlib.Path: + return pathlib.Path(str(self)) + + # override + def stat(self, *, follow_symlinks: bool = True) -> os.stat_result: + return self._as_path().stat(follow_symlinks=follow_symlinks) + + # override + def lstat(self) -> os.stat_result: + return self._as_path().lstat() + + # override + def exists(self) -> bool: + return self._as_path().exists() + + # override + def is_dir(self) -> bool: + return self._as_path().is_dir() + + # override + def is_file(self) -> bool: + return self._as_path().is_file() + + # override + def is_symlink(self) -> bool: + return self._as_path().is_symlink() + + # override + def is_block_device(self) -> bool: + return self._as_path().is_block_device() + + # override + def is_char_device(self) -> bool: + return self._as_path().is_char_device() + + # override + def is_fifo(self) -> bool: + return self._as_path().is_fifo() + + # override + def is_socket(self) -> bool: + return self._as_path().is_socket() + + # override + def open( + self, + mode: str = "r", + buffering: int = -1, + encoding: Optional[str] = None, + errors: Optional[str] = None, + newline: Optional[str] = None, + ): + return self._as_path().open( + mode=mode, + buffering=buffering, + encoding=encoding, + errors=errors, + newline=newline, ) + # override + def read_bytes(self) -> bytes: + return self._as_path().read_bytes() + + # override + def read_text( + self, encoding: Optional[str] = None, errors: Optional[str] = None + ) -> str: + return self._as_path().read_text(encoding=encoding, errors=errors) + + # override + def iterdir(self) -> Generator[Self, None, None]: + resolved = self._as_path() + for p in resolved.iterdir(): + yield self / p.name + + # override + def glob(self, pattern: str) -> Generator[Self, None, None]: + resolved = self._as_path() + for p in resolved.glob(pattern): + yield self / p.relative_to(resolved) + + # override + def rglob(self, pattern: str) -> Generator[Self, None, None]: + resolved = self._as_path() + for p in resolved.rglob(pattern): + yield self / p.relative_to(resolved) + def __repr__(self) -> str: - return 'runfiles.Path({!r})'.format(super().__str__()) + return "runfiles.Path({!r})".format(self.runfile_path) def __str__(self) -> str: path_posix = super().__str__().replace("\\", "/") @@ -251,12 +376,16 @@ def __str__(self) -> str: # pylint: disable=protected-access return self._runfiles._python_runfiles_root # type: ignore resolved = self._runfiles.Rlocation(path_posix, source_repo=self._source_repo) # type: ignore - return resolved if resolved is not None else super().__str__() + if resolved is not None: + return resolved + + # pylint: disable=protected-access + return posixpath.join(self._runfiles._python_runfiles_root, path_posix) # type: ignore def __fspath__(self) -> str: return str(self) - def runfiles_root(self) -> "Path": + def runfiles_root(self) -> Self: """Returns a Path object representing the runfiles root.""" return self._runfiles.root(source_repo=self._source_repo) # type: ignore diff --git a/tests/runfiles/BUILD.bazel b/tests/runfiles/BUILD.bazel index 7d675c7d7c..505b3c17c5 100644 --- a/tests/runfiles/BUILD.bazel +++ b/tests/runfiles/BUILD.bazel @@ -14,12 +14,34 @@ py_test( deps = ["//python/runfiles"], ) +py_test( + name = "runfiles_min_python_test", + srcs = ["runfiles_test.py"], + data = [ + "//tests/support:current_build_settings", + ], + env = { + "BZLMOD_ENABLED": "1" if BZLMOD_ENABLED else "0", + }, + main = "runfiles_test.py", + python_version = "3.10", + deps = ["//python/runfiles"], +) + py_test( name = "pathlib_test", srcs = ["pathlib_test.py"], deps = ["//python/runfiles"], ) +py_test( + name = "pathlib_min_python_test", + srcs = ["pathlib_test.py"], + main = "pathlib_test.py", + python_version = "3.10", + deps = ["//python/runfiles"], +) + build_test( name = "publishing", targets = [ diff --git a/tests/runfiles/pathlib_test.py b/tests/runfiles/pathlib_test.py index 553c8e4410..a959138235 100644 --- a/tests/runfiles/pathlib_test.py +++ b/tests/runfiles/pathlib_test.py @@ -9,15 +9,30 @@ class PathlibTest(unittest.TestCase): def setUp(self) -> None: self.tmpdir = tempfile.TemporaryDirectory(dir=os.environ.get("TEST_TMPDIR")) - # Runfiles paths are expected to be posix paths internally when we construct the strings for assertions - self.root_dir = pathlib.Path(self.tmpdir.name).as_posix() + # Runfiles paths are expected to be posix paths internally when we + # construct the strings for assertions + self.root_path = pathlib.Path(self.tmpdir.name).resolve() + self.root_dir = self.root_path.as_posix() + + # Create dummy files for I/O tests + self.repo_dir = self.root_path / "my_repo" + self.repo_dir.mkdir() + self.test_file = self.repo_dir / "data.txt" + self.test_file.write_text("hello runfiles", encoding="utf-8") + self.sub_dir = self.repo_dir / "subdir" + self.sub_dir.mkdir() + (self.sub_dir / "other.txt").write_text("other content", encoding="utf-8") + + def _create_runfiles(self) -> runfiles.Runfiles: + r = runfiles.Create({"RUNFILES_DIR": self.root_dir}) + assert r is not None + return r def tearDown(self) -> None: self.tmpdir.cleanup() def test_path_api(self) -> None: - r = runfiles.Create({"RUNFILES_DIR": self.root_dir}) - assert r is not None + r = self._create_runfiles() root = r.root() # Test basic joining @@ -41,26 +56,22 @@ def test_path_api(self) -> None: self.assertEqual(p, p3) def test_root(self) -> None: - r = runfiles.Create({"RUNFILES_DIR": self.root_dir}) - assert r is not None + r = self._create_runfiles() self.assertEqual(str(r.root()), self.root_dir) def test_runfiles_root_method(self) -> None: - r = runfiles.Create({"RUNFILES_DIR": self.root_dir}) - assert r is not None + r = self._create_runfiles() p = r.root() / "foo/bar" self.assertEqual(p.runfiles_root(), r.root()) self.assertEqual(str(p.runfiles_root()), self.root_dir) def test_os_path_like(self) -> None: - r = runfiles.Create({"RUNFILES_DIR": self.root_dir}) - assert r is not None + r = self._create_runfiles() p = r.root() / "foo" self.assertEqual(os.fspath(p), f"{self.root_dir}/foo") def test_equality_and_hash(self) -> None: - r = runfiles.Create({"RUNFILES_DIR": self.root_dir}) - assert r is not None + r = self._create_runfiles() p1 = r.root() / "foo" p2 = r.root() / "foo" p3 = r.root() / "bar" @@ -70,14 +81,12 @@ def test_equality_and_hash(self) -> None: self.assertEqual(hash(p1), hash(p2)) def test_join_path(self) -> None: - r = runfiles.Create({"RUNFILES_DIR": self.root_dir}) - assert r is not None + r = self._create_runfiles() p = r.root().joinpath("repo", "file") self.assertEqual(str(p), f"{self.root_dir}/repo/file") def test_parents(self) -> None: - r = runfiles.Create({"RUNFILES_DIR": self.root_dir}) - assert r is not None + r = self._create_runfiles() p = r.root() / "a/b/c" parents = list(p.parents) self.assertEqual(len(parents), 3) @@ -86,20 +95,99 @@ def test_parents(self) -> None: self.assertEqual(str(parents[2]), self.root_dir) def test_with_methods(self) -> None: - r = runfiles.Create({"RUNFILES_DIR": self.root_dir}) - assert r is not None + r = self._create_runfiles() p = r.root() / "foo/bar.txt" self.assertEqual(str(p.with_name("baz.py")), f"{self.root_dir}/foo/baz.py") self.assertEqual(str(p.with_suffix(".dat")), f"{self.root_dir}/foo/bar.dat") def test_match(self) -> None: - r = runfiles.Create({"RUNFILES_DIR": self.root_dir}) - assert r is not None + r = self._create_runfiles() p = r.root() / "foo/bar.txt" self.assertTrue(p.match("*.txt")) self.assertTrue(p.match("foo/*.txt")) self.assertFalse(p.match("bar/*.txt")) + def test_reading_api(self) -> None: + r = self._create_runfiles() + root = r.root() + p = root / "my_repo/data.txt" + + self.assertTrue(p.exists()) + self.assertTrue(p.is_file()) + self.assertEqual(p.read_text(encoding="utf-8"), "hello runfiles") + self.assertEqual(p.read_bytes(), b"hello runfiles") + + with p.open("r", encoding="utf-8") as f: + self.assertEqual(f.read(), "hello runfiles") + + def test_stat_api(self) -> None: + r = self._create_runfiles() + root = r.root() + p = root / "my_repo/data.txt" + + st = p.stat() + self.assertEqual(st.st_size, len("hello runfiles")) + + def test_iteration_api(self) -> None: + r = self._create_runfiles() + root = r.root() + p = root / "my_repo" + + self.assertTrue(p.is_dir()) + contents = {c.name for c in p.iterdir()} + self.assertEqual(contents, {"data.txt", "subdir"}) + # Ensure they are still runfiles.Path + for c in p.iterdir(): + self.assertIsInstance(c, runfiles.Path) + + def test_glob(self) -> None: + r = self._create_runfiles() + root = r.root() + p = root / "my_repo" + + glob_results = { + pathlib.PurePath(c).relative_to(pathlib.PurePath(p)).as_posix() + for c in p.glob("*.txt") + } + self.assertEqual(glob_results, {"data.txt"}) + + rglob_results = { + pathlib.PurePath(c).relative_to(pathlib.PurePath(p)).as_posix() + for c in p.rglob("*.txt") + } + self.assertEqual(rglob_results, {"data.txt", "subdir/other.txt"}) + + for c in p.rglob("*.txt"): + self.assertIsInstance(c, runfiles.Path) + + def test_resolve_and_absolute(self) -> None: + r = self._create_runfiles() + root = r.root() + p = root / "my_repo/data.txt" + + resolved = p.resolve() + self.assertIsInstance(resolved, runfiles.Path) + self.assertTrue(resolved.exists()) + self.assertEqual(resolved.read_text(encoding="utf-8"), "hello runfiles") + + absoluted = p.absolute() + self.assertIsInstance(absoluted, runfiles.Path) + self.assertTrue(absoluted.exists()) + self.assertEqual(absoluted.read_text(encoding="utf-8"), "hello runfiles") + + def test_runfile_path(self) -> None: + r = self._create_runfiles() + root = r.root() + p = root / "my_repo/data.txt" + self.assertEqual(p.runfile_path, "my_repo/data.txt") + + p2 = root / "foo" / "bar.txt" + self.assertEqual(p2.runfile_path, "foo/bar.txt") + + self.assertEqual(root.runfile_path, "") + self.assertEqual(repr(root), "runfiles.Path('')") + self.assertEqual(repr(p), "runfiles.Path('my_repo/data.txt')") + if __name__ == "__main__": unittest.main() From ccdfc8a7e6aa888e88bdbaee141eceef7a7611e4 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Thu, 23 Apr 2026 06:16:04 -0700 Subject: [PATCH 710/922] fix(pypi): don't resolve python interpreter when not necessary (#3727) Before the PR we would not set the environment for python interpreter if we would not need the interpreter, but we still resolve the python interpreter, which works on some systems that have a minimal Python installation. With this change we stop resolving the Python interpreter in cases were we should not need it. To ensure that we are not accidentally using it, `python_interpreter` is set to None when we use `pipstar` and do not do any patching, that still requires the Python interpreter. Fixes #3712 --------- Co-authored-by: Ignas Anikevicius <240938+aignas@users.noreply.github.com> --- python/private/pypi/whl_library.bzl | 40 +++++++++++++++++------------ 1 file changed, 23 insertions(+), 17 deletions(-) diff --git a/python/private/pypi/whl_library.bzl b/python/private/pypi/whl_library.bzl index 5db4f12d67..5639d9143f 100644 --- a/python/private/pypi/whl_library.bzl +++ b/python/private/pypi/whl_library.bzl @@ -183,7 +183,11 @@ def _parse_optional_attrs(rctx, args, extra_pip_args = None): if rctx.attr.add_libdir_to_library_search_path: if "LDFLAGS" in env: fail("Can't set both environment LDFLAGS and add_libdir_to_library_search_path") - command = [pypi_repo_utils.resolve_python_interpreter(rctx), "-c", "import sys ; sys.stdout.write('{}/lib'.format(sys.exec_prefix))"] + command = [ + pypi_repo_utils.resolve_python_interpreter(rctx), + "-c", + "import sys ; sys.stdout.write('{}/lib'.format(sys.exec_prefix))", + ] result = rctx.execute(command) if result.return_code != 0: fail("Failed to get LDFLAGS path: command: {}, exit code: {}, stdout: {}, stderr: {}".format(command, result.return_code, result.stdout, result.stderr)) @@ -292,22 +296,11 @@ def _extract_whl_py(rctx, *, python_interpreter, args, whl_path, environment, lo def _whl_library_impl(rctx): logger = repo_utils.logger(rctx) - python_interpreter = pypi_repo_utils.resolve_python_interpreter( - rctx, - python_interpreter = rctx.attr.python_interpreter, - python_interpreter_target = rctx.attr.python_interpreter_target, - ) - args = [ - "-m", - "python.private.pypi.whl_installer.wheel_installer", - "--requirement", - rctx.attr.requirement, - ] - extra_pip_args = [] - extra_pip_args.extend(rctx.attr.extra_pip_args) whl_path = None sdist_filename = None + extra_pip_args = [] + extra_pip_args.extend(rctx.attr.extra_pip_args) if rctx.attr.whl_file: rctx.watch(rctx.attr.whl_file) whl_path = rctx.path(rctx.attr.whl_file) @@ -352,17 +345,30 @@ def _whl_library_impl(rctx): # build deps from PyPI (e.g. `flit_core`) if they are missing. extra_pip_args.extend(["--find-links", "."]) - args = _parse_optional_attrs(rctx, args, extra_pip_args) - # also enable pipstar for any whls that are downloaded without `pip` enable_pipstar = (rp_config.enable_pipstar or whl_path) and rctx.attr.config_load enable_pipstar_extract = enable_pipstar and rp_config.bazel_8_or_later # When pipstar is enabled, Python isn't used, so there's no need # to setup env vars to run Python, unless we need to build an sdist - if enable_pipstar_extract and whl_path: + if enable_pipstar_extract and whl_path and not rctx.attr.whl_patches: environment = {} + args = [] + python_interpreter = None else: + python_interpreter = pypi_repo_utils.resolve_python_interpreter( + rctx, + python_interpreter = rctx.attr.python_interpreter, + python_interpreter_target = rctx.attr.python_interpreter_target, + ) + args = [ + "-m", + "python.private.pypi.whl_installer.wheel_installer", + "--requirement", + rctx.attr.requirement, + ] + args = _parse_optional_attrs(rctx, args, extra_pip_args) + # Manually construct the PYTHONPATH since we cannot use the toolchain here environment = _create_repository_execution_environment(rctx, python_interpreter, logger = logger) From 86332551ceb8511f51fd394fafc2c2fd15e68789 Mon Sep 17 00:00:00 2001 From: Jeremy Volkman Date: Thu, 23 Apr 2026 13:49:18 -0700 Subject: [PATCH 711/922] fix(gazelle): handle auto-included __init__.py when generating py_binary targets. (#3730) Fixes binary target generation when using per-file mode and automatically including __init__.py files. Fixes #3729 --- CHANGELOG.md | 3 +- gazelle/python/generate.go | 30 +++++++++++++++---- .../per_file_non_empty_init/BUILD.out | 12 +++++++- .../per_file_non_empty_init/README.md | 2 +- .../testdata/per_file_non_empty_init/foo.py | 1 + .../per_file_non_empty_init/foobin.py | 4 +++ 6 files changed, 43 insertions(+), 9 deletions(-) create mode 100644 gazelle/python/testdata/per_file_non_empty_init/foobin.py diff --git a/CHANGELOG.md b/CHANGELOG.md index 272b74db06..db16851959 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -63,7 +63,8 @@ END_UNRELEASED_TEMPLATE {#v0-0-0-fixed} ### Fixed -* Nothing fixed. +* (gazelle) Fixed handling of auto-included `__init__.py` files when generating `py_binary` + targets ([#3729](https://github.com/bazel-contrib/rules_python/issues/3729)). {#v0-0-0-added} ### Added diff --git a/gazelle/python/generate.go b/gazelle/python/generate.go index 2495c42d20..a13d7182e9 100644 --- a/gazelle/python/generate.go +++ b/gazelle/python/generate.go @@ -258,6 +258,15 @@ func (py *Python) GenerateRules(args language.GenerateArgs) language.GenerateRes // Create a validFilesMap of mainModules to validate if python macros have valid srcs. validFilesMap := make(map[string]struct{}) + // Determine whether we have an __init__.py file in this package and whether we'll implicitly include it in srcs. + var hasPopulatedInit bool + var autoIncludeInit bool + if cfg.PerFileGeneration() { + var hasInit bool + hasInit, hasPopulatedInit = hasLibraryEntrypointFile(args.Dir) + autoIncludeInit = cfg.PerFileGenerationIncludeInit() && hasInit && hasPopulatedInit + } + appendPyLibrary := func(srcs *treeset.Set, pyLibraryTargetName string) { allDeps, mainModules, annotations, err := parser.parse(srcs) for name := range mainModules { @@ -277,6 +286,10 @@ func (py *Python) GenerateRules(args language.GenerateArgs) language.GenerateRes // that we don't also generate a py_library target for it. if cfg.PerFileGeneration() { srcs.Remove(name) + // Also remove the __init__.py that was added earlier. + if autoIncludeInit { + srcs.Remove(pyLibraryEntrypointFilename) + } } } @@ -294,15 +307,20 @@ func (py *Python) GenerateRules(args language.GenerateArgs) language.GenerateRes filenames := treeset.NewWith(godsutils.StringComparator, filename) pyiSrcs, _ := getPyiFilenames(filenames, cfg.GeneratePyiSrcs(), args.Dir) - pyBinary := newTargetBuilder(pyBinaryKind, pyBinaryTargetName, pythonProjectRoot, args.Rel, pyFileNames, cfg.ResolveSiblingImports()). + pyBinaryBuilder := newTargetBuilder(pyBinaryKind, pyBinaryTargetName, pythonProjectRoot, args.Rel, pyFileNames, cfg.ResolveSiblingImports()). addVisibility(visibility). addSrc(filename). addPyiSrcs(pyiSrcs). addModuleDependencies(mainModules[filename]). addResolvedDependencies(annotations.includeDeps). generateImportsAttribute(). - setAnnotations(*annotations). - build() + setAnnotations(*annotations) + + if autoIncludeInit { + pyBinaryBuilder.addSrc(pyLibraryEntrypointFilename) + } + + pyBinary := pyBinaryBuilder.build() result.Gen = append(result.Gen, pyBinary) result.Imports = append(result.Imports, pyBinary.PrivateAttr(config.GazelleImportsKey)) } @@ -357,15 +375,15 @@ func (py *Python) GenerateRules(args language.GenerateArgs) language.GenerateRes result.Imports = append(result.Imports, pyLibrary.PrivateAttr(config.GazelleImportsKey)) } } + if cfg.PerFileGeneration() { - hasInit, nonEmptyInit := hasLibraryEntrypointFile(args.Dir) pyLibraryFilenames.Each(func(index int, filename interface{}) { pyLibraryTargetName := strings.TrimSuffix(filepath.Base(filename.(string)), ".py") - if filename == pyLibraryEntrypointFilename && !nonEmptyInit { + if filename == pyLibraryEntrypointFilename && !hasPopulatedInit { return // ignore empty __init__.py. } srcs := treeset.NewWith(godsutils.StringComparator, filename) - if cfg.PerFileGenerationIncludeInit() && hasInit && nonEmptyInit { + if autoIncludeInit { srcs.Add(pyLibraryEntrypointFilename) } appendPyLibrary(srcs, pyLibraryTargetName) diff --git a/gazelle/python/testdata/per_file_non_empty_init/BUILD.out b/gazelle/python/testdata/per_file_non_empty_init/BUILD.out index ee4a417966..38dcaa5ee2 100644 --- a/gazelle/python/testdata/per_file_non_empty_init/BUILD.out +++ b/gazelle/python/testdata/per_file_non_empty_init/BUILD.out @@ -1,4 +1,4 @@ -load("@rules_python//python:defs.bzl", "py_library") +load("@rules_python//python:defs.bzl", "py_binary", "py_library") # gazelle:python_generation_mode file # gazelle:python_generation_mode_per_file_include_init true @@ -18,3 +18,13 @@ py_library( ], visibility = ["//:__subpackages__"], ) + +py_binary( + name = "foobin", + srcs = [ + "__init__.py", + "foobin.py", + ], + visibility = ["//:__subpackages__"], + deps = [":foo"], +) diff --git a/gazelle/python/testdata/per_file_non_empty_init/README.md b/gazelle/python/testdata/per_file_non_empty_init/README.md index 6e6e9e245d..0fad0f3d43 100644 --- a/gazelle/python/testdata/per_file_non_empty_init/README.md +++ b/gazelle/python/testdata/per_file_non_empty_init/README.md @@ -1,3 +1,3 @@ # Per-file generation -This test case generates one `py_library` per file, including `__init__.py`. +This test case generates one `py_library` or `py_binary` per file, including `__init__.py`. diff --git a/gazelle/python/testdata/per_file_non_empty_init/foo.py b/gazelle/python/testdata/per_file_non_empty_init/foo.py index 730755995d..b70cc96b7e 100644 --- a/gazelle/python/testdata/per_file_non_empty_init/foo.py +++ b/gazelle/python/testdata/per_file_non_empty_init/foo.py @@ -13,3 +13,4 @@ # limitations under the License. # For test purposes only. +BAR = "baz" diff --git a/gazelle/python/testdata/per_file_non_empty_init/foobin.py b/gazelle/python/testdata/per_file_non_empty_init/foobin.py new file mode 100644 index 0000000000..6b493b531e --- /dev/null +++ b/gazelle/python/testdata/per_file_non_empty_init/foobin.py @@ -0,0 +1,4 @@ +from foo import BAR + +if __name__ == "__main__": + print(BAR) From 1c38124619d1f2f4c2eff856d5e339cb32556271 Mon Sep 17 00:00:00 2001 From: Mike Lundy Date: Sat, 25 Apr 2026 13:09:05 -0700 Subject: [PATCH 712/922] feat(toolchains): add add_target_settings to python.override (#3731) Users who register custom toolchain families (e.g. gated behind a string_flag like "custom" vs "prebuilt") have no way to prevent the default-registered toolchains from acting as a silent fallback. When a user requests a custom toolchain at a version they didn't register, Bazel's toolchain resolution quietly falls back to the default prebuilt toolchain instead of producing an error. This adds a `toolchain_target_settings` attribute to `python.override` that appends the given `config_setting` labels to the `target_settings` of every toolchain registered by the module extension: ```starlark python.override( toolchain_target_settings = ["@@//:python_toolchain_family_prebuilt"], ) ``` These settings are appended to the `target_settings` of all toolchains registered by the extension, including any that already have settings from `python.single_version_platform_override`. Toolchains registered outside the extension (e.g. via `local_runtime_toolchains_repo`) are not affected. Note: `toolchain_target_settings` is popped from the config `default` dict before it reaches `python_register_toolchains()`, so it doesn't leak into the `python_repository` kwargs. Integration test passes across the full Bazel version matrix (7.7.0, 8.5.1, 9.0.0rc1). Verified the test fails without the fix (default toolchain silently resolves as fallback). Fixes https://github.com/bazel-contrib/rules_python/issues/3673 --------- Co-authored-by: Richard Levasseur --- .bazelignore | 1 + .bazelrc.deleted_packages | 1 + CHANGELOG.md | 2 + python/private/python.bzl | 35 +++++++++++- tests/integration/BUILD.bazel | 5 ++ .../toolchain_target_settings/.bazelrc | 3 + .../toolchain_target_settings/BUILD.bazel | 56 +++++++++++++++++++ .../toolchain_target_settings/MODULE.bazel | 24 ++++++++ .../toolchain_target_settings/REPO.bazel | 0 .../toolchain_target_settings/WORKSPACE | 1 + .../WORKSPACE.bzlmod | 0 .../toolchain_target_settings/main.py | 2 + .../toolchain_target_settings_test.py | 39 +++++++++++++ tests/python/python_tests.bzl | 30 +++++++++- 14 files changed, 197 insertions(+), 2 deletions(-) create mode 100644 tests/integration/toolchain_target_settings/.bazelrc create mode 100644 tests/integration/toolchain_target_settings/BUILD.bazel create mode 100644 tests/integration/toolchain_target_settings/MODULE.bazel create mode 100644 tests/integration/toolchain_target_settings/REPO.bazel create mode 100644 tests/integration/toolchain_target_settings/WORKSPACE create mode 100644 tests/integration/toolchain_target_settings/WORKSPACE.bzlmod create mode 100644 tests/integration/toolchain_target_settings/main.py create mode 100644 tests/integration/toolchain_target_settings_test.py diff --git a/.bazelignore b/.bazelignore index 89e06444a4..90e2c7dddd 100644 --- a/.bazelignore +++ b/.bazelignore @@ -33,3 +33,4 @@ sphinxdocs tests/integration/compile_pip_requirements/bazel-compile_pip_requirements tests/integration/local_toolchains/bazel-local_toolchains tests/integration/py_cc_toolchain_registered/bazel-py_cc_toolchain_registered +tests/integration/toolchain_target_settings/bazel-module_under_test diff --git a/.bazelrc.deleted_packages b/.bazelrc.deleted_packages index fb5d2ef0bb..61c79ae032 100644 --- a/.bazelrc.deleted_packages +++ b/.bazelrc.deleted_packages @@ -36,6 +36,7 @@ common --deleted_packages=tests/integration/local_toolchains common --deleted_packages=tests/integration/pip_parse common --deleted_packages=tests/integration/pip_parse/empty common --deleted_packages=tests/integration/py_cc_toolchain_registered +common --deleted_packages=tests/integration/toolchain_target_settings common --deleted_packages=tests/modules/another_module common --deleted_packages=tests/modules/other common --deleted_packages=tests/modules/other/nspkg_delta diff --git a/CHANGELOG.md b/CHANGELOG.md index db16851959..24a25f7421 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -68,6 +68,8 @@ END_UNRELEASED_TEMPLATE {#v0-0-0-added} ### Added +* (toolchain) Added {obj}`python.override.toolchain_target_settings` to allow + adding `config_setting` labels to all registered toolchains. * (windows) Full venv support for Windows is available. Set {obj}`--venvs_site_packages=yes` to enable. * (runfiles) Added a pathlib-compatible API: {obj}`Runfiles.root()` diff --git a/python/private/python.bzl b/python/private/python.bzl index 12a17f1e72..2ea757892e 100644 --- a/python/private/python.bzl +++ b/python/private/python.bzl @@ -403,6 +403,10 @@ def _python_impl(module_ctx): # the PLATFORMS global for this toolchain toolchain_platform_keys = {} + # Extra target_settings to add to every registered toolchain, e.g. for + # gating the default toolchains behind a custom config_setting. + global_add_target_settings = py.config.add_target_settings + # Split the toolchain info into separate objects so they can be passed onto # the repository rule. for entry in toolchain_impls: @@ -414,7 +418,7 @@ def _python_impl(module_ctx): # The target_settings attribute may not be present for users # patching python/versions.bzl. - toolchain_ts_map[key] = getattr(entry.platform, "target_settings", []) + toolchain_ts_map[key] = getattr(entry.platform, "target_settings", []) + global_add_target_settings toolchain_platform_keys[key] = entry.platform_name toolchain_python_versions[key] = entry.full_python_version @@ -702,6 +706,9 @@ def _process_global_overrides(*, tag, default, _fail = fail): default["minor_mapping"] = tag.minor_mapping + if tag.add_target_settings: + default["add_target_settings"] = list(tag.add_target_settings) + forwarded_attrs = sorted(AUTH_ATTRS) + [ "base_url", "register_all_versions", @@ -809,6 +816,7 @@ def _get_toolchain_config(*, modules, _fail = fail): ) register_all_versions = default.pop("register_all_versions", False) + add_target_settings = default.pop("add_target_settings", []) kwargs = default.pop("kwargs", {}) versions = {} @@ -834,6 +842,7 @@ def _get_toolchain_config(*, modules, _fail = fail): minor_mapping = minor_mapping, default = default, register_all_versions = register_all_versions, + add_target_settings = add_target_settings, ) def _compute_default_python_version(mctx): @@ -1099,6 +1108,30 @@ _override = tag_class( ::: """, attrs = { + "add_target_settings": attr.string_list( + mandatory = False, + doc = """\ +A list of `config_setting` labels to add to the `target_settings` of every +toolchain registered by this module extension. This is useful for creating +separate "families" of toolchains gated behind custom build settings. + +For example, to ensure the default prebuilt toolchains are only resolved when +a `prebuilt` config setting is active: + +```starlark +python.override( + add_target_settings = ["@@//:python_toolchain_family_prebuilt"], +) +``` + +These settings are appended to the `target_settings` of all toolchains +registered by the extension, including any that already have settings +from `python.single_version_platform_override`. + +:::{versionadded} VERSION_NEXT_FEATURE +::: +""", + ), "available_python_versions": attr.string_list( mandatory = False, doc = """\ diff --git a/tests/integration/BUILD.bazel b/tests/integration/BUILD.bazel index f0f58daa3a..33ef907af8 100644 --- a/tests/integration/BUILD.bazel +++ b/tests/integration/BUILD.bazel @@ -88,6 +88,11 @@ rules_python_integration_test( py_main = "custom_commands_test.py", ) +rules_python_integration_test( + name = "toolchain_target_settings_test", + py_main = "toolchain_target_settings_test.py", +) + py_library( name = "runner_lib", srcs = ["runner.py"], diff --git a/tests/integration/toolchain_target_settings/.bazelrc b/tests/integration/toolchain_target_settings/.bazelrc new file mode 100644 index 0000000000..e1df951d06 --- /dev/null +++ b/tests/integration/toolchain_target_settings/.bazelrc @@ -0,0 +1,3 @@ +common --lockfile_mode=off +test --test_output=errors +build --enable_runfiles diff --git a/tests/integration/toolchain_target_settings/BUILD.bazel b/tests/integration/toolchain_target_settings/BUILD.bazel new file mode 100644 index 0000000000..0e28e461b3 --- /dev/null +++ b/tests/integration/toolchain_target_settings/BUILD.bazel @@ -0,0 +1,56 @@ +load("@bazel_skylib//rules:common_settings.bzl", "string_flag") +load("@rules_python//python:py_test.bzl", "py_test") + +# A flag to select which "family" of toolchains to use. +string_flag( + name = "family", + build_setting_default = "prebuilt", + values = [ + "prebuilt", + "custom", + ], +) + +# Matches when the "prebuilt" family is selected. +# This is referenced in MODULE.bazel's python.override(add_target_settings=...). +config_setting( + name = "is_prebuilt", + flag_values = { + ":family": "prebuilt", + }, +) + +# Matches when the "custom" family is selected. +# No toolchains use this setting, so selecting it should produce an error. +config_setting( + name = "is_custom", + flag_values = { + ":family": "custom", + }, +) + +# This target selects the "prebuilt" family via config_settings transition. +# Since python.override(add_target_settings = ["@@//:is_prebuilt"]) gates +# the default toolchains, this should succeed: the flag matches, the config_setting +# is satisfied, and the default 3.13 toolchain resolves. +py_test( + name = "prebuilt_test", + srcs = ["main.py"], + config_settings = { + "//:family": "prebuilt", + }, + main = "main.py", +) + +# This target selects the "custom" family via config_settings transition. +# No toolchains have target_settings = [":is_custom"], so toolchain resolution +# should fail -- the default toolchains are gated behind ":is_prebuilt" and +# won't match. +py_test( + name = "custom_no_toolchain_test", + srcs = ["main.py"], + config_settings = { + "//:family": "custom", + }, + main = "main.py", +) diff --git a/tests/integration/toolchain_target_settings/MODULE.bazel b/tests/integration/toolchain_target_settings/MODULE.bazel new file mode 100644 index 0000000000..ec3a56749e --- /dev/null +++ b/tests/integration/toolchain_target_settings/MODULE.bazel @@ -0,0 +1,24 @@ +module(name = "module_under_test") + +bazel_dep(name = "rules_python", version = "0.0.0") +bazel_dep(name = "bazel_skylib", version = "1.7.1") + +local_path_override( + module_name = "rules_python", + path = "../../..", +) + +python = use_extension("@rules_python//python/extensions:python.bzl", "python") +python.toolchain(python_version = "3.13") + +# Gate ALL default-registered toolchains behind the "prebuilt" config setting. +# This prevents them from being a silent fallback when a different toolchain +# family is requested. +python.override( + add_target_settings = ["@@//:is_prebuilt"], +) + +# Register //:family as a transition setting so py_binary/py_test +# config_settings can set it. +config = use_extension("@rules_python//python/extensions:config.bzl", "config") +config.add_transition_setting(setting = "//:family") diff --git a/tests/integration/toolchain_target_settings/REPO.bazel b/tests/integration/toolchain_target_settings/REPO.bazel new file mode 100644 index 0000000000..e69de29bb2 diff --git a/tests/integration/toolchain_target_settings/WORKSPACE b/tests/integration/toolchain_target_settings/WORKSPACE new file mode 100644 index 0000000000..d7be0c96c7 --- /dev/null +++ b/tests/integration/toolchain_target_settings/WORKSPACE @@ -0,0 +1 @@ +# Intentionally blank; bzlmod is used. diff --git a/tests/integration/toolchain_target_settings/WORKSPACE.bzlmod b/tests/integration/toolchain_target_settings/WORKSPACE.bzlmod new file mode 100644 index 0000000000..e69de29bb2 diff --git a/tests/integration/toolchain_target_settings/main.py b/tests/integration/toolchain_target_settings/main.py new file mode 100644 index 0000000000..6ae68c3f51 --- /dev/null +++ b/tests/integration/toolchain_target_settings/main.py @@ -0,0 +1,2 @@ +import sys +print(f"Python {sys.version}") diff --git a/tests/integration/toolchain_target_settings_test.py b/tests/integration/toolchain_target_settings_test.py new file mode 100644 index 0000000000..a0c9d42fc3 --- /dev/null +++ b/tests/integration/toolchain_target_settings_test.py @@ -0,0 +1,39 @@ +"""Integration test for python.override(add_target_settings=...). + +Verifies that when all default toolchains are gated behind a config_setting, +requesting a different (unregistered) toolchain family produces a toolchain +resolution error instead of silently falling back to the default toolchains. +""" + +import unittest + +from tests.integration import runner + + +class AddTargetSettingsTest(runner.TestCase): + def test_prebuilt_family_resolves(self): + """Building with the 'prebuilt' family should succeed. + + The default toolchains have target_settings = [":is_prebuilt"], + and the transition sets //:family=prebuilt, so the config_setting + matches and toolchain resolution finds the default 3.13 toolchain. + """ + self.run_bazel("test", "//:prebuilt_test") + + def test_custom_family_without_toolchain_fails(self): + """Building with the 'custom' family should fail. + + No toolchains have target_settings = [":is_custom"], and the default + toolchains are gated behind ":is_prebuilt" (via add_target_settings), + so toolchain resolution should fail with no matching toolchain. + """ + result = self.run_bazel("build", "//:custom_no_toolchain_test", check=False) + self.assertNotEqual(result.exit_code, 0, "Expected build to fail") + self.assert_result_matches( + result, + r"No matching toolchains found for types", + ) + + +if __name__ == "__main__": + unittest.main() diff --git a/tests/python/python_tests.bzl b/tests/python/python_tests.bzl index db54e6bab5..60e7311c00 100644 --- a/tests/python/python_tests.bzl +++ b/tests/python/python_tests.bzl @@ -53,7 +53,8 @@ def _override( base_url = "", minor_mapping = {}, netrc = "", - register_all_versions = False): + register_all_versions = False, + add_target_settings = []): return struct( auth_patterns = auth_patterns, available_python_versions = available_python_versions, @@ -61,6 +62,7 @@ def _override( minor_mapping = minor_mapping, netrc = netrc, register_all_versions = register_all_versions, + add_target_settings = add_target_settings, ) def _rules_python_module(is_root = False): @@ -465,6 +467,32 @@ def _test_auth_overrides(env): _tests.append(_test_auth_overrides) +def _test_add_target_settings(env): + py = parse_modules( + module_ctx = mocks.mctx( + _mod( + name = "my_module", + toolchain = [_toolchain("3.12")], + override = [ + _override( + add_target_settings = [ + "@@//my:custom_setting", + ], + ), + ], + is_root = True, + ), + _rules_python_module(), + ), + logger = repo_utils.logger(verbosity_level = 0, name = "python"), + ) + + env.expect.that_collection( + py.config.add_target_settings, + ).contains_exactly(["@@//my:custom_setting"]) + +_tests.append(_test_add_target_settings) + def _test_add_new_version(env): py = parse_modules( module_ctx = mocks.mctx( From 3f6daa8ded64f0e32773aab83707bfb4afc5cf63 Mon Sep 17 00:00:00 2001 From: Ignas Anikevicius <240938+aignas@users.noreply.github.com> Date: Sun, 26 Apr 2026 12:24:43 +0900 Subject: [PATCH 713/922] refactor(pypi): extract a function for deleting files recursively (#3733) It's high time we did some cleanup and I'd like to start modularizing the code a little to make future maintenance easier. --- python/private/pypi/whl_library.bzl | 22 ++++++++++++---------- 1 file changed, 12 insertions(+), 10 deletions(-) diff --git a/python/private/pypi/whl_library.bzl b/python/private/pypi/whl_library.bzl index 5639d9143f..3586494dc2 100644 --- a/python/private/pypi/whl_library.bzl +++ b/python/private/pypi/whl_library.bzl @@ -546,26 +546,28 @@ def _whl_library_impl(rctx): rctx.file("MODULE.bazel") rctx.file("REPO.bazel") + # BUILD files interfere with globbing and Bazel package boundaries. + _remove_files(rctx, "BUILD", "BUILD.bazel") + rctx.file("BUILD.bazel", build_file_contents) + + if enable_pipstar and enable_pipstar_extract: + if hasattr(rctx, "repo_metadata"): + return rctx.repo_metadata(reproducible = True) + + return None + +def _remove_files(rctx, *basenames): paths = list(rctx.path(".").readdir()) for _ in range(10000000): if not paths: break path = paths.pop() - # BUILD files interfere with globbing and Bazel package boundaries. - if path.basename in ("BUILD", "BUILD.bazel"): + if path.basename in basenames: rctx.delete(path) elif path.is_dir: paths.extend(path.readdir()) - rctx.file("BUILD.bazel", build_file_contents) - - if enable_pipstar and enable_pipstar_extract: - if hasattr(rctx, "repo_metadata"): - return rctx.repo_metadata(reproducible = True) - - return None - def _generate_entry_point_contents( module, attribute, From b81b287f9f6554ba076e863d2c89843a84182e22 Mon Sep 17 00:00:00 2001 From: Ignas Anikevicius <240938+aignas@users.noreply.github.com> Date: Sun, 26 Apr 2026 14:11:07 +0900 Subject: [PATCH 714/922] fix(entry_point): ignore type lints on the generated files (#3736) Fixes #3126 --------- Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> --- CHANGELOG.md | 2 ++ python/private/py_console_script_gen.py | 2 +- tests/entry_points/py_console_script_gen_test.py | 2 +- 3 files changed, 4 insertions(+), 2 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 24a25f7421..225a1c1c00 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -65,6 +65,8 @@ END_UNRELEASED_TEMPLATE ### Fixed * (gazelle) Fixed handling of auto-included `__init__.py` files when generating `py_binary` targets ([#3729](https://github.com/bazel-contrib/rules_python/issues/3729)). +* (entry_point) From now on `mypy` type checking will be skipped on the generated + files ([#3126](https://github.com/bazel-contrib/rules_python/issues/3126)). {#v0-0-0-added} ### Added diff --git a/python/private/py_console_script_gen.py b/python/private/py_console_script_gen.py index 4b4f2f6986..a1df2c2a06 100644 --- a/python/private/py_console_script_gen.py +++ b/python/private/py_console_script_gen.py @@ -62,7 +62,7 @@ raise if __name__ == "__main__": - sys.exit({entry_point}()) + sys.exit({entry_point}()) # type: ignore """ diff --git a/tests/entry_points/py_console_script_gen_test.py b/tests/entry_points/py_console_script_gen_test.py index 1bbf5fbf25..77ad1a5faa 100644 --- a/tests/entry_points/py_console_script_gen_test.py +++ b/tests/entry_points/py_console_script_gen_test.py @@ -162,7 +162,7 @@ def test_a_single_entry_point(self): raise if __name__ == "__main__": - sys.exit(baz()) + sys.exit(baz()) # type: ignore """ ) self.assertEqual(want, got) From 1567357ad34b749123e414f5ad5f49fe59a97ac1 Mon Sep 17 00:00:00 2001 From: Ignas Anikevicius <240938+aignas@users.noreply.github.com> Date: Mon, 27 Apr 2026 01:19:36 +0900 Subject: [PATCH 715/922] refactor(twine): use py_binary for publishing in WORKSPACE (#3734) Remove the reliance on defunct entry point code. Work towards #3642. --- python/BUILD.bazel | 2 +- python/packaging.bzl | 11 ++--------- 2 files changed, 3 insertions(+), 10 deletions(-) diff --git a/python/BUILD.bazel b/python/BUILD.bazel index 6577ff805f..90b2225ab5 100644 --- a/python/BUILD.bazel +++ b/python/BUILD.bazel @@ -88,7 +88,7 @@ bzl_library( name = "packaging_bzl", srcs = ["packaging.bzl"], deps = [ - ":py_binary_bzl", + "//python/entry_points:py_console_script_binary_bzl", "//python/private:bzlmod_enabled_bzl", "//python/private:py_package_bzl", "//python/private:py_wheel_bzl", diff --git a/python/packaging.bzl b/python/packaging.bzl index 223aba142d..537aa6090f 100644 --- a/python/packaging.bzl +++ b/python/packaging.bzl @@ -216,19 +216,12 @@ def py_wheel( **copy_propagating_kwargs(kwargs) ) elif twine: - if not twine.endswith(":pkg"): - fail("twine label should look like @my_twine_repo//:pkg") - - twine_main = twine.replace(":pkg", ":rules_python_wheel_entry_point_twine.py") - py_binary( name = "{}.publish".format(name), - srcs = [twine_main], + deps = [twine], args = twine_args, data = [dist_target], - imports = ["."], - main = twine_main, - deps = [twine], + main_module = "twine", tags = manual_tags, visibility = kwargs.get("visibility"), **copy_propagating_kwargs(kwargs) From aa5bd2cfa9f8c784f9b849a2af0eb5554f8b3fc5 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Sun, 26 Apr 2026 12:03:55 -0700 Subject: [PATCH 716/922] chore: enable platform-specific configs and silence C++ warnings in .bazelrc (#3738) Silence spammy C++ compile warnings in build logs to make it easier to identify actual issues. The warnings were primarily deprecation notices from external dependencies (like protobuf) observed during both target and tool compilation. Platform-specific configurations were enabled because C++ compiler flags (copts) differ significantly for Windows (which typically uses MSVC) compared to Linux and macOS (which use GCC/Clang). Applying the warning- silencing flags globally would break Windows builds. This approach allows us to safely ignore these specific warnings only on platforms where the flags are supported. --- .bazelrc | 17 +++++++++++++++++ 1 file changed, 17 insertions(+) diff --git a/.bazelrc b/.bazelrc index 771bf0fe90..8fedcf1f8f 100644 --- a/.bazelrc +++ b/.bazelrc @@ -25,6 +25,9 @@ common --incompatible_disallow_struct_provider_syntax # Makes Bazel 7 act more like Bazel 8 common --incompatible_use_plus_in_repo_names +# Implicitly adds --config=linux|macos|windows, depending on the host platform +common --enable_platform_specific_config + # Needed to make Windows with a py_binary in data deps work. The Bazel launcher # is used, which falls back to finding python.exe on PATH to bootstrap. # See https://github.com/bazel-contrib/rules_python/issues/3655 @@ -55,6 +58,20 @@ common --incompatible_no_implicit_file_export build --lockfile_mode=update +# Silence spammy C++ compile warnings +build:linux --copt=-Wno-deprecated-declarations +build:linux --copt=-Wno-stringop-overread +build:linux --copt=-Wno-sign-compare +build:linux --host_copt=-Wno-deprecated-declarations +build:linux --host_copt=-Wno-stringop-overread +build:linux --host_copt=-Wno-sign-compare +build:macos --copt=-Wno-deprecated-declarations +build:macos --copt=-Wno-stringop-overread +build:macos --copt=-Wno-sign-compare +build:macos --host_copt=-Wno-deprecated-declarations +build:macos --host_copt=-Wno-stringop-overread +build:macos --host_copt=-Wno-sign-compare + import %workspace%/specialized_configs.bazelrc try-import user.bazelrc From c3c0daf0c67c44dd42d68062bbf756cd93604413 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Sun, 26 Apr 2026 16:17:49 -0700 Subject: [PATCH 717/922] tests: support whl_from_dir_repo on Windows (#3740) tests: support whl_from_dir_repo on Windows The `whl_from_dir_repo` repository rule previously relied on the Unix `zip` utility to create wheels. Because this command isn't natively available on Windows, any tests that depended on repositories generated by this rule had to be explicitly skipped on Windows hosts. To fix this and expand our test coverage, this adds a native Windows fallback. When running on Windows, the rule now invokes a helper PowerShell script that uses .NET compression APIs to create the archive. This script ensures the resulting wheel remains uncompressed and uses zeroed-out timestamps to match the deterministic behavior of the original `zip -0X` command. With this constraint removed, the Unix-only compatibility flags (`SUPPORTS_BZLMOD_UNIXY`) have been dropped, enabling several namespace package and wheel-related integration tests to finally run on Windows. --- tests/implicit_namespace_packages/BUILD.bazel | 4 +- tests/pypi/whl_library/BUILD.bazel | 4 +- tests/support/support.bzl | 5 +- .../whl_from_dir/whl_from_dir_repo.bzl | 53 ++++++++++++++----- tests/support/whl_from_dir/zip.ps1 | 45 ++++++++++++++++ .../app_files_building_tests.bzl | 6 +-- tests/whl_with_build_files/BUILD.bazel | 4 +- 7 files changed, 94 insertions(+), 27 deletions(-) create mode 100644 tests/support/whl_from_dir/zip.ps1 diff --git a/tests/implicit_namespace_packages/BUILD.bazel b/tests/implicit_namespace_packages/BUILD.bazel index 42aca9b97f..b544c4d118 100644 --- a/tests/implicit_namespace_packages/BUILD.bazel +++ b/tests/implicit_namespace_packages/BUILD.bazel @@ -1,10 +1,10 @@ load("//python:py_test.bzl", "py_test") -load("//tests/support:support.bzl", "SUPPORTS_BZLMOD_UNIXY") +load("//tests/support:support.bzl", "SUPPORTS_BZLMOD") py_test( name = "namespace_packages_test", srcs = ["namespace_packages_test.py"], - target_compatible_with = SUPPORTS_BZLMOD_UNIXY, + target_compatible_with = SUPPORTS_BZLMOD, deps = [ "@implicit_namespace_ns_sub1//:pkg", "@implicit_namespace_ns_sub2//:pkg", diff --git a/tests/pypi/whl_library/BUILD.bazel b/tests/pypi/whl_library/BUILD.bazel index 599bb12a15..cade0d2b8e 100644 --- a/tests/pypi/whl_library/BUILD.bazel +++ b/tests/pypi/whl_library/BUILD.bazel @@ -1,10 +1,10 @@ load("//python:py_test.bzl", "py_test") -load("//tests/support:support.bzl", "SUPPORTS_BZLMOD_UNIXY") +load("//tests/support:support.bzl", "SUPPORTS_BZLMOD") py_test( name = "whl_library_extras_test", srcs = ["whl_library_extras_test.py"], - target_compatible_with = SUPPORTS_BZLMOD_UNIXY, + target_compatible_with = SUPPORTS_BZLMOD, deps = [ "@whl_library_extras_direct_dep//:pkg", ], diff --git a/tests/support/support.bzl b/tests/support/support.bzl index 9bd2c987b9..64f77d76bc 100644 --- a/tests/support/support.bzl +++ b/tests/support/support.bzl @@ -35,10 +35,7 @@ SUPPORTS_BOOTSTRAP_SCRIPT = select({ "//conditions:default": [], }) -SUPPORTS_BZLMOD_UNIXY = select({ - "@platforms//os:windows": ["@platforms//:incompatible"], - "//conditions:default": [], -}) if BZLMOD_ENABLED else ["@platforms//:incompatible"] +SUPPORTS_BZLMOD = [] if BZLMOD_ENABLED else ["@platforms//:incompatible"] NOT_WINDOWS = select({ "@platforms//os:windows": ["@platforms//:incompatible"], diff --git a/tests/support/whl_from_dir/whl_from_dir_repo.bzl b/tests/support/whl_from_dir/whl_from_dir_repo.bzl index 4e16e8ee4a..c827c8a0a0 100644 --- a/tests/support/whl_from_dir/whl_from_dir_repo.bzl +++ b/tests/support/whl_from_dir/whl_from_dir_repo.bzl @@ -11,20 +11,41 @@ def _whl_from_dir_repo(rctx): rctx.watch_tree(root) output = rctx.path(rctx.attr.output) - repo_utils.execute_checked( - rctx, - # cd to root so zip recursively takes everything there. - working_directory = str(root), - op = "WhlFromDir", - arguments = [ - "zip", - "-0", # Skip compressing - "-X", # Don't store file time or metadata - str(output), - "-r", - ".", - ], - ) + if repo_utils.get_platforms_os_name(rctx) == "windows": + powershell_exe = rctx.which("powershell.exe") or rctx.which("powershell") + if not powershell_exe: + fail("powershell not found on PATH") + + zip_script = rctx.path(rctx.attr._zip_script) + + repo_utils.execute_checked( + rctx, + op = "WhlFromDir", + arguments = [ + powershell_exe, + "-NoProfile", + "-File", + str(zip_script), + str(output), + str(root), + ], + # zip.ps1 handles relativizing paths. + ) + else: + repo_utils.execute_checked( + rctx, + # cd to root so zip recursively takes everything there. + working_directory = str(root), + op = "WhlFromDir", + arguments = [ + "zip", + "-0", # Skip compressing + "-X", # Don't store file time or metadata + str(output), + "-r", + ".", + ], + ) rctx.file("BUILD.bazel", 'exports_files(glob(["*"]))') whl_from_dir_repo = repository_rule( @@ -46,5 +67,9 @@ A file whose directory will be put into the output wheel. All files are included verbatim. """, ), + "_zip_script": attr.label( + default = "//tests/support/whl_from_dir:zip.ps1", + allow_single_file = True, + ), }, ) diff --git a/tests/support/whl_from_dir/zip.ps1 b/tests/support/whl_from_dir/zip.ps1 new file mode 100644 index 0000000000..1e8c199cde --- /dev/null +++ b/tests/support/whl_from_dir/zip.ps1 @@ -0,0 +1,45 @@ +param ( + [Parameter(Position=0, Mandatory=$true)] + [string]$Output, + + [Parameter(Position=1, Mandatory=$true)] + [string]$Root +) + +Add-Type -AssemblyName System.IO.Compression + +$fixedTime = [datetime]"1980-01-01T00:00:00" +$RootFull = (Resolve-Path $Root).Path + +$stream = [System.IO.File]::Open($Output, [System.IO.FileMode]::Create) +try { + $archive = [System.IO.Compression.ZipArchive]::new($stream, [System.IO.Compression.ZipArchiveMode]::Create) + try { + $files = Get-ChildItem -Path $RootFull -Recurse -File + foreach ($file in $files) { + # Relativize path and normalize separators + $relPath = $file.FullName.Substring($RootFull.Length).TrimStart('\', '/') + $relPath = $relPath -replace '\\', '/' + + $entry = $archive.CreateEntry($relPath, [System.IO.Compression.CompressionLevel]::NoCompression) + $entry.LastWriteTime = $fixedTime + + $entryStream = $entry.Open() + try { + $fileStream = [System.IO.File]::OpenRead($file.FullName) + try { + $fileStream.CopyTo($entryStream) + } finally { + $fileStream.Dispose() + } + } finally { + $entryStream.Dispose() + } + } + } finally { + $archive.Dispose() + } +} finally { + $stream.Dispose() +} + diff --git a/tests/venv_site_packages_libs/app_files_building/app_files_building_tests.bzl b/tests/venv_site_packages_libs/app_files_building/app_files_building_tests.bzl index d808eae7e9..66ba7076a9 100644 --- a/tests/venv_site_packages_libs/app_files_building/app_files_building_tests.bzl +++ b/tests/venv_site_packages_libs/app_files_building/app_files_building_tests.bzl @@ -7,7 +7,7 @@ load("//python:py_library.bzl", "py_library") load("//python/private:common_labels.bzl", "labels") # buildifier: disable=bzl-visibility load("//python/private:py_info.bzl", "VenvSymlinkEntry", "VenvSymlinkKind") # buildifier: disable=bzl-visibility load("//python/private:venv_runfiles.bzl", "build_link_map", "get_venv_symlinks") # buildifier: disable=bzl-visibility -load("//tests/support:support.bzl", "SUPPORTS_BZLMOD_UNIXY") +load("//tests/support:support.bzl", "SUPPORTS_BZLMOD") def _empty_files_impl(ctx): files = [] @@ -425,7 +425,7 @@ def _test_optimized_grouping_pkgutil_whls(name): "@pkgutil_nspkg1//:pkg", "@pkgutil_nspkg2//:pkg", ], - target_compatible_with = SUPPORTS_BZLMOD_UNIXY, + target_compatible_with = SUPPORTS_BZLMOD, ) analysis_test( name = name, @@ -435,7 +435,7 @@ def _test_optimized_grouping_pkgutil_whls(name): labels.VENVS_SITE_PACKAGES: "yes", }, attr_values = dict( - target_compatible_with = SUPPORTS_BZLMOD_UNIXY, + target_compatible_with = SUPPORTS_BZLMOD, ), ) diff --git a/tests/whl_with_build_files/BUILD.bazel b/tests/whl_with_build_files/BUILD.bazel index e26dc1c3a6..1202876485 100644 --- a/tests/whl_with_build_files/BUILD.bazel +++ b/tests/whl_with_build_files/BUILD.bazel @@ -1,9 +1,9 @@ load("//python:py_test.bzl", "py_test") -load("//tests/support:support.bzl", "SUPPORTS_BZLMOD_UNIXY") +load("//tests/support:support.bzl", "SUPPORTS_BZLMOD") py_test( name = "verify_files_test", srcs = ["verify_files_test.py"], - target_compatible_with = SUPPORTS_BZLMOD_UNIXY, + target_compatible_with = SUPPORTS_BZLMOD, deps = ["@somepkg_with_build_files//:pkg"], ) From d7e33d5bcca29c6f8a8a7b60eb83b0a1ff8ed9d2 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Sun, 26 Apr 2026 16:25:58 -0700 Subject: [PATCH 718/922] docs: give guidance on pr body content (#3742) I've noticed that AI-generated PR descriptions are very verbose with lots of the typical AI formatting, filler words, qualifiers, and read more as a persuasive essay than an effective description of a change. Add a bit of guidance to the contribution guide and agents config to help curtail that slop a bit. --- AGENTS.md | 7 ++++++- CONTRIBUTING.md | 29 ++++++++++++++++++++++++++++- 2 files changed, 34 insertions(+), 2 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index 38ea8d1be2..45df61fd25 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -5,7 +5,7 @@ project. Act as an expert in Bazel, rules_python, Starlark, and Python. -DO NOT `git commit` or `git push`. +DO NOT `git commit` or `git push` unless given explicit permission. ## RULES TO ALWAYS FOLLOW AND NEVER IGNORE @@ -30,6 +30,11 @@ into the sentence, not verbatim. When adding `{versionadded}` or `{versionchanged}` sections, add them add the end of the documentation text. +### PR descriptions + +Follow the advice in `CONTRIBUTING.md` for PR descriptions. PR descriptions +become the commit message upon merge. + ### Starlark style For doc strings, using triple quoted strings when the doc string is more than diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index e1bd11b81d..917110978b 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -103,7 +103,7 @@ information on using pull requests. [GitHub Help]: https://help.github.com/articles/about-pull-requests/ -### Commit messages +### Commit messages and PR descriptions Commit messages (upon merging) and PR messages should follow the [Conventional Commits](https://www.conventionalcommits.org/) style: @@ -139,9 +139,36 @@ Common `type`s: * `revert:` means a prior change is being reverted in some way. * `test:` means only tests are being added. +For the body, follow this guidance: + +* Briefly tells *why* the change is being made. This usually means + briefly describing how a bug manifests or what can't be accomplished + without the feature. +* Briefly gives an overview of *how* the code is changed. This is to + orient readers for the diff they're about to read and understand; it's + not a verbatim description of what changed. +* List unrelated or notable dev-only changes at the end. e.g. formatting an + old file, cleaning up testing, adding test support code, etc. + For the full details of types, see [Conventional Commits](https://www.conventionalcommits.org/). +#### PR description example + +``` +fix(pypi): handle files with .exe extensions + +Currently, if a file with `.exe` is seen, an error +occurs because validation assumes unix-only filenames. +This prevents using whls with pre-built .exe files in +their data payload. + +To fix, detect the target OS and use an OS-appropriate +validation function. + +* Also adds test helpers for detecting the current OS +``` + ### Documenting changes Changes are documented in two places: CHANGELOG.md and API docs. From 82ae73ea037a15f0eddcca9e15e89d1e69ebc245 Mon Sep 17 00:00:00 2001 From: Ignas Anikevicius <240938+aignas@users.noreply.github.com> Date: Mon, 27 Apr 2026 08:39:29 +0900 Subject: [PATCH 719/922] chore(entry_point): remove all repository_rule entry_point code and docs (#3735) This PR remove any mention or supporting code for the legacy (and long removed) `entry_point` macro. Fixes #3642 Co-authored-by: Richard Levasseur --- examples/pip_parse/BUILD.bazel | 3 - .../pypi/generate_whl_library_build_bazel.bzl | 13 --- python/private/pypi/labels.bzl | 1 - python/private/pypi/pip_repository.bzl | 24 ------ python/private/pypi/whl_library.bzl | 79 ------------------- python/private/pypi/whl_library_targets.bzl | 18 ----- python/private/pypi/whl_metadata.bzl | 58 +------------- ...generate_whl_library_build_bazel_tests.bzl | 44 ----------- .../whl_library_targets_tests.bzl | 30 ------- .../pypi/whl_metadata/whl_metadata_tests.bzl | 49 ------------ 10 files changed, 2 insertions(+), 317 deletions(-) diff --git a/examples/pip_parse/BUILD.bazel b/examples/pip_parse/BUILD.bazel index 37a25fe873..2dc9d61127 100644 --- a/examples/pip_parse/BUILD.bazel +++ b/examples/pip_parse/BUILD.bazel @@ -45,9 +45,6 @@ py_test( deps = [":main"], ) -# For pip dependencies which have entry points, the `entry_point` macro can be -# used from the generated `pip_parse` repository to access a runnable binary. - py_console_script_binary( name = "yamllint", pkg = "@pypi//yamllint", diff --git a/python/private/pypi/generate_whl_library_build_bazel.bzl b/python/private/pypi/generate_whl_library_build_bazel.bzl index fbabe2ede3..5811ed1574 100644 --- a/python/private/pypi/generate_whl_library_build_bazel.bzl +++ b/python/private/pypi/generate_whl_library_build_bazel.bzl @@ -23,7 +23,6 @@ _RENDER = { "data_exclude": render.list, "dependencies": render.list, "dependencies_by_platform": lambda x: render.dict(x, value_repr = render.list), - "entry_points": render.dict, "extras": render.list, "group_deps": render.list, "include": str, @@ -100,18 +99,6 @@ def generate_whl_library_build_bazel( ]) additional_content = [] - entry_points = kwargs.get("entry_points") - if entry_points: - entry_point_files = sorted({ - entry_point_script.replace("\\", "/"): True - for entry_point_script in entry_points.values() - }.keys()) - additional_content.append( - "exports_files(\n" + - " srcs = {},\n".format(render.list(entry_point_files)) + - " visibility = [\"//visibility:public\"],\n" + - ")\n", - ) if annotation: kwargs["data"] = annotation.data kwargs["copy_files"] = annotation.copy_files diff --git a/python/private/pypi/labels.bzl b/python/private/pypi/labels.bzl index 22161b1496..8f91a03b4c 100644 --- a/python/private/pypi/labels.bzl +++ b/python/private/pypi/labels.bzl @@ -21,5 +21,4 @@ PY_LIBRARY_PUBLIC_LABEL = "pkg" PY_LIBRARY_IMPL_LABEL = "_pkg" DATA_LABEL = "data" DIST_INFO_LABEL = "dist_info" -WHEEL_ENTRY_POINT_PREFIX = "rules_python_wheel_entry_point" NODEPS_LABEL = "no_deps" diff --git a/python/private/pypi/pip_repository.bzl b/python/private/pypi/pip_repository.bzl index 6d38334cc6..4560bd7eef 100644 --- a/python/private/pypi/pip_repository.bzl +++ b/python/private/pypi/pip_repository.bzl @@ -348,30 +348,6 @@ functionality for exposing [entry points][whl_ep] as `py_binary` targets as well [whl_ep]: https://packaging.python.org/specifications/entry-points/ -```starlark -load("@pypi//:requirements.bzl", "entry_point") - -alias( - name = "pip-compile", - actual = entry_point( - pkg = "pip-tools", - script = "pip-compile", - ), -) -``` - -Note that for packages whose name and script are the same, only the name of the package -is needed when calling the `entry_point` macro. - -```starlark -load("@pip//:requirements.bzl", "entry_point") - -alias( - name = "flake8", - actual = entry_point("flake8"), -) -``` - :::{rubric} Vendoring the requirements.bzl file :heading-level: 3 ::: diff --git a/python/private/pypi/whl_library.bzl b/python/private/pypi/whl_library.bzl index 3586494dc2..bd76741fa3 100644 --- a/python/private/pypi/whl_library.bzl +++ b/python/private/pypi/whl_library.bzl @@ -33,7 +33,6 @@ load(":whl_target_platforms.bzl", "whl_target_platforms") _CPPFLAGS = "CPPFLAGS" _COMMAND_LINE_TOOLS_PATH_SLUG = "commandlinetools" -_WHEEL_ENTRY_POINT_PREFIX = "rules_python_wheel_entry_point" def _get_xcode_location_cflags(rctx, logger = None): """Query the xcode sdk location to update cflags @@ -443,30 +442,6 @@ def _whl_library_impl(rctx): ) namespace_package_files = pypi_repo_utils.find_namespace_package_files(rctx, install_dir_path) - # NOTE @aignas 2024-06-22: this has to live on until we stop supporting - # passing `twine` as a `:pkg` library via the `WORKSPACE` builds. - # - # See ../../packaging.bzl line 190 - entry_points = {} - for item in metadata.entry_points: - name = item.name - module = item.module - attribute = item.attribute - - # There is an extreme edge-case with entry_points that end with `.py` - # See: https://github.com/bazelbuild/bazel/blob/09c621e4cf5b968f4c6cdf905ab142d5961f9ddc/src/test/java/com/google/devtools/build/lib/rules/python/PyBinaryConfiguredTargetTest.java#L174 - entry_point_without_py = name[:-3] + "_py" if name.endswith(".py") else name - entry_point_target_name = ( - _WHEEL_ENTRY_POINT_PREFIX + "_" + entry_point_without_py - ) - entry_point_script_name = entry_point_target_name + ".py" - - rctx.file( - entry_point_script_name, - _generate_entry_point_contents(module, attribute), - ) - entry_points[entry_point_without_py] = entry_point_script_name - build_file_contents = generate_whl_library_build_bazel( name = whl_path.basename, sdist_filename = sdist_filename, @@ -474,7 +449,6 @@ def _whl_library_impl(rctx): rctx.attr.repo_prefix, ), config_load = rctx.attr.config_load, - entry_points = entry_points, metadata_name = metadata.name, metadata_version = metadata.version, requires_dist = metadata.requires_dist, @@ -492,37 +466,12 @@ def _whl_library_impl(rctx): metadata = json.decode(rctx.read("metadata.json")) rctx.delete("metadata.json") - # NOTE @aignas 2024-06-22: this has to live on until we stop supporting - # passing `twine` as a `:pkg` library via the `WORKSPACE` builds. - # - # See ../../packaging.bzl line 190 - entry_points = {} - for item in metadata["entry_points"]: - name = item["name"] - module = item["module"] - attribute = item["attribute"] - - # There is an extreme edge-case with entry_points that end with `.py` - # See: https://github.com/bazelbuild/bazel/blob/09c621e4cf5b968f4c6cdf905ab142d5961f9ddc/src/test/java/com/google/devtools/build/lib/rules/python/PyBinaryConfiguredTargetTest.java#L174 - entry_point_without_py = name[:-3] + "_py" if name.endswith(".py") else name - entry_point_target_name = ( - _WHEEL_ENTRY_POINT_PREFIX + "_" + entry_point_without_py - ) - entry_point_script_name = entry_point_target_name + ".py" - - rctx.file( - entry_point_script_name, - _generate_entry_point_contents(module, attribute), - ) - entry_points[entry_point_without_py] = entry_point_script_name - namespace_package_files = pypi_repo_utils.find_namespace_package_files(rctx, rctx.path("site-packages")) build_file_contents = generate_whl_library_build_bazel( name = whl_path.basename, sdist_filename = sdist_filename, dep_template = rctx.attr.dep_template or "@{}{{name}}//:{{target}}".format(rctx.attr.repo_prefix), - entry_points = entry_points, # TODO @aignas 2025-05-17: maybe have a build flag for this instead enable_implicit_namespace_pkgs = rctx.attr.enable_implicit_namespace_pkgs, # TODO @aignas 2025-04-14: load through the hub: @@ -568,34 +517,6 @@ def _remove_files(rctx, *basenames): elif path.is_dir: paths.extend(path.readdir()) -def _generate_entry_point_contents( - module, - attribute, - shebang = "#!/usr/bin/env python3"): - """Generate the contents of an entry point script. - - Args: - module (str): The name of the module to use. - attribute (str): The name of the attribute to call. - shebang (str, optional): The shebang to use for the entry point python - file. - - Returns: - str: A string of python code. - """ - contents = """\ -{shebang} -import sys -from {module} import {attribute} -if __name__ == "__main__": - sys.exit({attribute}()) -""".format( - shebang = shebang, - module = module, - attribute = attribute, - ) - return contents - # NOTE @aignas 2024-03-21: The usage of dict({}, **common) ensures that all args to `dict` are unique whl_library_attrs = dict({ "annotation": attr.label( diff --git a/python/private/pypi/whl_library_targets.bzl b/python/private/pypi/whl_library_targets.bzl index dc99aab532..dde5f815eb 100644 --- a/python/private/pypi/whl_library_targets.bzl +++ b/python/private/pypi/whl_library_targets.bzl @@ -26,7 +26,6 @@ load( "EXTRACTED_WHEEL_FILES", "PY_LIBRARY_IMPL_LABEL", "PY_LIBRARY_PUBLIC_LABEL", - "WHEEL_ENTRY_POINT_PREFIX", "WHEEL_FILE_IMPL_LABEL", "WHEEL_FILE_PUBLIC_LABEL", ) @@ -120,7 +119,6 @@ def whl_library_targets( data = [], copy_files = {}, copy_executables = {}, - entry_points = {}, native = native, enable_implicit_namespace_pkgs = False, namespace_package_files = [], @@ -165,8 +163,6 @@ def whl_library_targets( srcs_exclude: {type}`list[str]` The globs for srcs attribute exclusion in `py_library`. data: {type}`list[str]` A list of labels to include as part of the `data` attribute in `py_library`. - entry_points: {type}`dict[str, str]` The mapping between the script - name and the python file to use. DEPRECATED. enable_implicit_namespace_pkgs: {type}`boolean` generate __init__.py files for namespace pkgs. native: {type}`native` The native struct for overriding in tests. @@ -237,20 +233,6 @@ def whl_library_targets( for d in dependencies_with_markers } - # TODO @aignas 2024-10-25: remove the entry_point generation once - # `py_console_script_binary` is the only way to use entry points. - for entry_point, entry_point_script_name in entry_points.items(): - rules.py_binary( - name = "{}_{}".format(WHEEL_ENTRY_POINT_PREFIX, entry_point), - # Ensure that this works on Windows as well - script may have Windows path separators. - srcs = [entry_point_script_name.replace("\\", "/")], - # This makes this directory a top-level in the python import - # search path for anything that depends on this. - imports = ["."], - deps = [":" + PY_LIBRARY_PUBLIC_LABEL], - visibility = ["//visibility:public"], - ) - # Ensure this list is normalized # Note: mapping used as set group_deps = { diff --git a/python/private/pypi/whl_metadata.bzl b/python/private/pypi/whl_metadata.bzl index 0d3a14ab54..002e5773cc 100644 --- a/python/private/pypi/whl_metadata.bzl +++ b/python/private/pypi/whl_metadata.bzl @@ -4,7 +4,6 @@ _NAME = "Name: " _PROVIDES_EXTRA = "Provides-Extra: " _REQUIRES_DIST = "Requires-Dist: " _VERSION = "Version: " -_CONSOLE_SCRIPTS = "[console_scripts]" def whl_metadata(*, install_dir, read_fn, logger): """Find and parse the METADATA file in the extracted whl contents dir. @@ -24,13 +23,8 @@ def whl_metadata(*, install_dir, read_fn, logger): """ metadata_file = find_whl_metadata(install_dir = install_dir, logger = logger) contents = read_fn(metadata_file) - entry_points_file = metadata_file.dirname.get_child("entry_points.txt") - if entry_points_file.exists: - entry_points_contents = read_fn(entry_points_file) - else: - entry_points_contents = "" - result = parse_whl_metadata(contents, entry_points_contents) + result = parse_whl_metadata(contents) if not (result.name and result.version): logger.fail("Failed to parse the wheel METADATA file:\n{}\n{}\n{}".format( @@ -42,12 +36,11 @@ def whl_metadata(*, install_dir, read_fn, logger): return result -def parse_whl_metadata(contents, entry_points_contents = ""): +def parse_whl_metadata(contents): """Parse .whl METADATA file Args: contents: {type}`str` the contents of the file. - entry_points_contents: {type}`str` the contents of the `entry_points.txt` file if it exists. Returns: A struct with parsed values: @@ -56,8 +49,6 @@ def parse_whl_metadata(contents, entry_points_contents = ""): * `requires_dist`: {type}`list[str]` the list of requirements. * `provides_extra`: {type}`list[str]` the list of extras that this package provides. - * `entry_points`: {type}`list[struct]` the list of - entry_point metadata. """ parsed = { "name": "", @@ -89,7 +80,6 @@ def parse_whl_metadata(contents, entry_points_contents = ""): provides_extra = parsed["provides_extra"], requires_dist = parsed["requires_dist"], version = parsed["version"], - entry_points = _parse_entry_points(entry_points_contents), ) def find_whl_metadata(*, install_dir, logger): @@ -121,47 +111,3 @@ def find_whl_metadata(*, install_dir, logger): else: logger.fail("The '*.dist-info' directory could not be found in '{}'".format(install_dir.basename)) return None - -def _parse_entry_points(contents): - """parse the entry_points.txt file. - - Args: - contents: {type}`str` The contents of the file - - Returns: - A list of console_script entry point metadata. - """ - start = False - ret = [] - for line in contents.split("\n"): - line = line.rstrip() - - if line == _CONSOLE_SCRIPTS: - start = True - continue - - if not start: - continue - - if start and line.startswith("["): - break - - line, _, _comment = line.partition("#") - line = line.strip() - if not line: - continue - - name, _, tail = line.partition("=") - - # importable.module:object.attr - py_import, _, extras = tail.strip().partition(" ") - module, _, attribute = py_import.partition(":") - - ret.append(struct( - name = name.strip(), - module = module.strip(), - attribute = attribute.strip(), - extras = extras.replace(" ", ""), - )) - - return ret diff --git a/tests/pypi/generate_whl_library_build_bazel/generate_whl_library_build_bazel_tests.bzl b/tests/pypi/generate_whl_library_build_bazel/generate_whl_library_build_bazel_tests.bzl index 85e96be579..2f421f35d4 100644 --- a/tests/pypi/generate_whl_library_build_bazel/generate_whl_library_build_bazel_tests.bzl +++ b/tests/pypi/generate_whl_library_build_bazel/generate_whl_library_build_bazel_tests.bzl @@ -42,9 +42,6 @@ whl_library_targets( dependencies_by_platform = { "baz": ["bar"], }, - entry_points = { - "foo": "bar.py", - }, group_deps = [ "foo", "fox", @@ -56,11 +53,6 @@ whl_library_targets( tags = ["tag1"], ) -exports_files( - srcs = ["bar.py"], - visibility = ["//visibility:public"], -) - # SOMETHING SPECIAL AT THE END """ actual = generate_whl_library_build_bazel( @@ -68,9 +60,6 @@ exports_files( name = "foo.whl", dependencies = ["foo"], dependencies_by_platform = {"baz": ["bar"]}, - entry_points = { - "foo": "bar.py", - }, data_exclude = ["exclude_via_attr"], annotation = struct( copy_files = {"file_src": "file_dest"}, @@ -108,9 +97,6 @@ whl_library_targets_from_requires( "data_exclude_all", ], dep_template = "@pypi//{name}:{target}", - entry_points = { - "foo": "bar.py", - }, group_deps = [ "foo", "fox", @@ -127,20 +113,12 @@ whl_library_targets_from_requires( srcs_exclude = ["srcs_exclude_all"], ) -exports_files( - srcs = ["bar.py"], - visibility = ["//visibility:public"], -) - # SOMETHING SPECIAL AT THE END """ actual = generate_whl_library_build_bazel( dep_template = "@pypi//{name}:{target}", name = "foo.whl", requires_dist = ["foo", "bar-baz", "qux"], - entry_points = { - "foo": "bar.py", - }, data_exclude = ["exclude_via_attr"], annotation = struct( copy_files = {"file_src": "file_dest"}, @@ -178,9 +156,6 @@ whl_library_targets_from_requires( "data_exclude_all", ], dep_template = "@pypi//{name}:{target}", - entry_points = { - "foo": "bar.py", - }, group_deps = [ "foo", "fox", @@ -197,20 +172,12 @@ whl_library_targets_from_requires( srcs_exclude = ["srcs_exclude_all"], ) -exports_files( - srcs = ["bar.py"], - visibility = ["//visibility:public"], -) - # SOMETHING SPECIAL AT THE END """ actual = generate_whl_library_build_bazel( dep_template = "@pypi//{name}:{target}", name = "foo.whl", requires_dist = ["foo", "bar-baz", "qux"], - entry_points = { - "foo": "bar.py", - }, data_exclude = ["exclude_via_attr"], annotation = struct( copy_files = {"file_src": "file_dest"}, @@ -248,9 +215,6 @@ whl_library_targets_from_requires( "data_exclude_all", ], dep_template = "@pypi//{name}:{target}", - entry_points = { - "foo": "bar.py", - }, group_deps = [ "foo", "fox", @@ -267,20 +231,12 @@ whl_library_targets_from_requires( srcs_exclude = ["srcs_exclude_all"], ) -exports_files( - srcs = ["bar.py"], - visibility = ["//visibility:public"], -) - # SOMETHING SPECIAL AT THE END """ actual = generate_whl_library_build_bazel( dep_template = "@pypi//{name}:{target}", name = "foo.whl", requires_dist = ["foo", "bar-baz", "qux"], - entry_points = { - "foo": "bar.py", - }, data_exclude = ["exclude_via_attr"], annotation = struct( copy_files = {"file_src": "file_dest"}, diff --git a/tests/pypi/whl_library_targets/whl_library_targets_tests.bzl b/tests/pypi/whl_library_targets/whl_library_targets_tests.bzl index 08715fbf77..28765770f4 100644 --- a/tests/pypi/whl_library_targets/whl_library_targets_tests.bzl +++ b/tests/pypi/whl_library_targets/whl_library_targets_tests.bzl @@ -158,35 +158,6 @@ def _test_copy(env): _tests.append(_test_copy) -def _test_entrypoints(env): - calls = [] - - whl_library_targets( - name = "", - dep_template = None, - dependencies_by_platform = {}, - filegroups = {}, - entry_points = { - "fizz": "buzz.py", - }, - native = struct(), - rules = struct( - py_binary = lambda **kwargs: calls.append(kwargs), - ), - ) - - env.expect.that_collection(calls).contains_exactly([ - { - "name": "rules_python_wheel_entry_point_fizz", - "srcs": ["buzz.py"], - "deps": [":pkg"], - "imports": ["."], - "visibility": ["//visibility:public"], - }, - ]) # buildifier: @unsorted-dict-items - -_tests.append(_test_entrypoints) - def _test_whl_and_library_deps_from_requires(env): filegroup_calls = [] py_library_calls = [] @@ -412,7 +383,6 @@ def _test_group(env): "@platforms//os:linux": ["box"], # buildifier: disable=unsorted-dict-items to check that we sort inside the test }, tags = [], - entry_points = {}, data_exclude = [], group_name = "qux", group_deps = ["foo", "fox", "qux"], diff --git a/tests/pypi/whl_metadata/whl_metadata_tests.bzl b/tests/pypi/whl_metadata/whl_metadata_tests.bzl index 1d78611901..329423a26c 100644 --- a/tests/pypi/whl_metadata/whl_metadata_tests.bzl +++ b/tests/pypi/whl_metadata/whl_metadata_tests.bzl @@ -81,14 +81,12 @@ def _parse_whl_metadata(env, **kwargs): version = result.version, requires_dist = result.requires_dist, provides_extra = result.provides_extra, - entry_points = result.entry_points, ), attrs = dict( name = subjects.str, version = subjects.str, requires_dist = subjects.collection, provides_extra = subjects.collection, - entry_points = subjects.collection, ), ) @@ -173,53 +171,6 @@ Requires-Dist: this will be ignored _tests.append(_test_parse_metadata_multiline_license) -def _test_parse_entry_points_txt(env): - got = _parse_whl_metadata( - env, - contents = """\ -Name: foo -Version: 0.0.1 -""", - entry_points_contents = """\ -[something] -interesting # with comments - -[console_scripts] -foo = foomod:main -# One which depends on extras: -foobar = importable.foomod:main_bar [bar, baz] - - # With a comment at the end -foobarbaz = foomod:main.attr # comment - -[something else] -not very much interesting - -""", - ) - got.entry_points().contains_exactly([ - struct( - attribute = "main", - extras = "", - module = "foomod", - name = "foo", - ), - struct( - attribute = "main_bar", - extras = "[bar,baz]", - module = "importable.foomod", - name = "foobar", - ), - struct( - attribute = "main.attr", - extras = "", - module = "foomod", - name = "foobarbaz", - ), - ]) - -_tests.append(_test_parse_entry_points_txt) - def whl_metadata_test_suite(name): # buildifier: disable=function-docstring test_suite( name = name, From 12eac29b5d09d94e819f93af40e6eb661c4f2dc5 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Mon, 27 Apr 2026 00:50:48 -0700 Subject: [PATCH 720/922] feat(venv): support data, include, and scripts schemes (#3726) Currently, the files from the data, headers, and scripts portions of a wheel don't end up in the proper sub-directories of the venv. This means the full files of a distribution aren't available at the typical location in the venv, making it harder to integrate with standard tools. To fix, simply map the directories to paths in the venv and give them similar treatment as site-packages. The spec says certain first-level directories of the `.data` directory map to specific scheme paths, which `whl_library` already handles. Here's a listing of the `wheel data directory -> install scheme path key -> whl_library directory` relationships * purelib -> purelib -> site-packages * platlib -> platlib -> site-packages * headers -> include -> include * scripts -> scripts -> bin * data -> data -> data Relevant reading: * Packaging specification: https://packaging.python.org/en/latest/specifications/binary-distribution-format * Posix install scheme paths: https://docs.python.org/3/library/sysconfig.html#posix-prefix * Windows install scheme paths: https://docs.python.org/3/library/sysconfig.html#nt The whl_library rule uses posix names for extracting. When materialized into a binary's venv, platform specific names are used: * bin -> (posix) bin; (Windows) Scripts * include -> (posix) include; (Windows) Include (capital i) * data -> venv root directory Along the way ... * The data files (files under the "data" scheme of a whl) are now always included as part of depending on the library. They would be in included in venv_site_packages=yes mode, so this better aligns behavior of the two modes. * Rename `is_venv_site_packages` to `_is_venv_site_packages_yes` to better represent the purpose and visibility of it. * Make whl_from_dir support Windows. Testing of venvs relies on using it for testing various special cases. --- CHANGELOG.md | 4 + MODULE.bazel | 2 + examples/pip_parse/pip_parse_test.py | 20 ++-- internal_dev_setup.bzl | 24 +++++ python/config_settings/BUILD.bazel | 2 +- python/private/internal_dev_deps.bzl | 22 +++++ python/private/py_executable.bzl | 25 +++-- python/private/py_info.bzl | 7 ++ python/private/pypi/whl_library_targets.bzl | 10 +- python/private/venv_runfiles.bzl | 72 ++++++++++++-- .../whl_library_targets_tests.bzl | 20 ++-- tests/repos/whl_with_data1/BUILD.bazel | 1 + .../data/bin/data_overlap.sh | 1 + .../data/include/data_overlap.h | 1 + .../data/overlap/both.txt | 1 + .../data/overlap/data1.txt | 1 + .../data/site-packages/data_overlap.py | 1 + .../data/whl_with_data1/data_data_file.txt | 1 + .../headers/data_overlap.h | 1 + .../headers/overlap/both.h | 1 + .../headers/overlap/header1.h | 1 + .../headers/whl_with_data1/header_file.h | 1 + .../platlib/whl_with_data1/platlib_file.txt | 1 + .../purelib/data_overlap.py | 1 + .../purelib/whl_with_data1/__init__.py | 0 .../purelib/whl_with_data1/data_file.txt | 1 + .../scripts/data_overlap.sh | 1 + .../scripts/overlap/both.sh | 1 + .../scripts/overlap/script1.sh | 1 + .../scripts/whl_script.sh | 1 + .../whl_with_data1-1.0.dist-info/METADATA | 3 + .../whl_with_data1-1.0.dist-info/RECORD | 18 ++++ .../whl_with_data1-1.0.dist-info/WHEEL | 1 + tests/repos/whl_with_data2/BUILD.bazel | 1 + .../data/overlap/both.txt | 1 + .../data/overlap/data2.txt | 1 + .../data/whl_with_data2/data_data_file.txt | 1 + .../headers/overlap/both.h | 1 + .../headers/overlap/header2.h | 1 + .../headers/whl_with_data2/header_file.h | 1 + .../platlib/whl_with_data2/platlib_file.txt | 1 + .../purelib/whl_with_data2/__init__.py | 0 .../purelib/whl_with_data2/data_file.txt | 1 + .../scripts/overlap/both.sh | 1 + .../scripts/overlap/script2.sh | 1 + .../scripts/whl_script.sh | 1 + .../whl_with_data2-1.0.dist-info/METADATA | 3 + .../whl_with_data2-1.0.dist-info/RECORD | 12 +++ .../whl_with_data2-1.0.dist-info/WHEEL | 1 + tests/venv_site_packages_libs/BUILD.bazel | 2 + tests/venv_site_packages_libs/bin.py | 93 +++++++++++++++---- 51 files changed, 318 insertions(+), 53 deletions(-) create mode 100644 tests/repos/whl_with_data1/BUILD.bazel create mode 100644 tests/repos/whl_with_data1/whl_with_data1-1.0.data/data/bin/data_overlap.sh create mode 100644 tests/repos/whl_with_data1/whl_with_data1-1.0.data/data/include/data_overlap.h create mode 100644 tests/repos/whl_with_data1/whl_with_data1-1.0.data/data/overlap/both.txt create mode 100644 tests/repos/whl_with_data1/whl_with_data1-1.0.data/data/overlap/data1.txt create mode 100644 tests/repos/whl_with_data1/whl_with_data1-1.0.data/data/site-packages/data_overlap.py create mode 100644 tests/repos/whl_with_data1/whl_with_data1-1.0.data/data/whl_with_data1/data_data_file.txt create mode 100644 tests/repos/whl_with_data1/whl_with_data1-1.0.data/headers/data_overlap.h create mode 100644 tests/repos/whl_with_data1/whl_with_data1-1.0.data/headers/overlap/both.h create mode 100644 tests/repos/whl_with_data1/whl_with_data1-1.0.data/headers/overlap/header1.h create mode 100644 tests/repos/whl_with_data1/whl_with_data1-1.0.data/headers/whl_with_data1/header_file.h create mode 100644 tests/repos/whl_with_data1/whl_with_data1-1.0.data/platlib/whl_with_data1/platlib_file.txt create mode 100644 tests/repos/whl_with_data1/whl_with_data1-1.0.data/purelib/data_overlap.py create mode 100644 tests/repos/whl_with_data1/whl_with_data1-1.0.data/purelib/whl_with_data1/__init__.py create mode 100644 tests/repos/whl_with_data1/whl_with_data1-1.0.data/purelib/whl_with_data1/data_file.txt create mode 100644 tests/repos/whl_with_data1/whl_with_data1-1.0.data/scripts/data_overlap.sh create mode 100644 tests/repos/whl_with_data1/whl_with_data1-1.0.data/scripts/overlap/both.sh create mode 100644 tests/repos/whl_with_data1/whl_with_data1-1.0.data/scripts/overlap/script1.sh create mode 100644 tests/repos/whl_with_data1/whl_with_data1-1.0.data/scripts/whl_script.sh create mode 100644 tests/repos/whl_with_data1/whl_with_data1-1.0.dist-info/METADATA create mode 100644 tests/repos/whl_with_data1/whl_with_data1-1.0.dist-info/RECORD create mode 100644 tests/repos/whl_with_data1/whl_with_data1-1.0.dist-info/WHEEL create mode 100644 tests/repos/whl_with_data2/BUILD.bazel create mode 100644 tests/repos/whl_with_data2/whl_with_data2-1.0.data/data/overlap/both.txt create mode 100644 tests/repos/whl_with_data2/whl_with_data2-1.0.data/data/overlap/data2.txt create mode 100644 tests/repos/whl_with_data2/whl_with_data2-1.0.data/data/whl_with_data2/data_data_file.txt create mode 100644 tests/repos/whl_with_data2/whl_with_data2-1.0.data/headers/overlap/both.h create mode 100644 tests/repos/whl_with_data2/whl_with_data2-1.0.data/headers/overlap/header2.h create mode 100644 tests/repos/whl_with_data2/whl_with_data2-1.0.data/headers/whl_with_data2/header_file.h create mode 100644 tests/repos/whl_with_data2/whl_with_data2-1.0.data/platlib/whl_with_data2/platlib_file.txt create mode 100644 tests/repos/whl_with_data2/whl_with_data2-1.0.data/purelib/whl_with_data2/__init__.py create mode 100644 tests/repos/whl_with_data2/whl_with_data2-1.0.data/purelib/whl_with_data2/data_file.txt create mode 100644 tests/repos/whl_with_data2/whl_with_data2-1.0.data/scripts/overlap/both.sh create mode 100644 tests/repos/whl_with_data2/whl_with_data2-1.0.data/scripts/overlap/script2.sh create mode 100644 tests/repos/whl_with_data2/whl_with_data2-1.0.data/scripts/whl_script.sh create mode 100644 tests/repos/whl_with_data2/whl_with_data2-1.0.dist-info/METADATA create mode 100644 tests/repos/whl_with_data2/whl_with_data2-1.0.dist-info/RECORD create mode 100644 tests/repos/whl_with_data2/whl_with_data2-1.0.dist-info/WHEEL diff --git a/CHANGELOG.md b/CHANGELOG.md index 225a1c1c00..7c19d1176e 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -60,6 +60,8 @@ END_UNRELEASED_TEMPLATE ### Changed * (gazelle) WORKSPACE's bazel-gazelle dependency bumped from 0.36.0 to 0.47.0. The go version was also bumped from 1.21.13 to 1.22.9. +* (pypi) The data files of a wheel (bin, includes, etc) are now always included + as a library's data dependencies. {#v0-0-0-fixed} ### Fixed @@ -74,6 +76,8 @@ END_UNRELEASED_TEMPLATE adding `config_setting` labels to all registered toolchains. * (windows) Full venv support for Windows is available. Set {obj}`--venvs_site_packages=yes` to enable. +* (test/binaries) When {obj}`--venv_site_packages=yes` is enabled, + wheel `data`, `bin`, and `include` files are populated into the venv. * (runfiles) Added a pathlib-compatible API: {obj}`Runfiles.root()` Fixes [#3296](https://github.com/bazel-contrib/rules_python/issues/3296). * (toolchains) `3.13.12`, `3.14.3` Python toolchain from [20260325] release. diff --git a/MODULE.bazel b/MODULE.bazel index 95d6b9e3a9..b5f67c204e 100644 --- a/MODULE.bazel +++ b/MODULE.bazel @@ -246,6 +246,8 @@ use_repo( "somepkg_with_build_files", "whl_library_extras_direct_dep", "whl_with_build_files", + "whl_with_data1", + "whl_with_data2", ) dev_rules_python_config = use_extension( diff --git a/examples/pip_parse/pip_parse_test.py b/examples/pip_parse/pip_parse_test.py index 2fdd45477e..c532dff564 100644 --- a/examples/pip_parse/pip_parse_test.py +++ b/examples/pip_parse/pip_parse_test.py @@ -52,16 +52,16 @@ def test_data(self): self.assertIsNotNone(actual) actual = self._remove_leading_dirs(actual.split(" ")) - self.assertListEqual( - actual, - [ - "data/share/doc/packages/s3cmd/INSTALL.md", - "data/share/doc/packages/s3cmd/LICENSE", - "data/share/doc/packages/s3cmd/NEWS", - "data/share/doc/packages/s3cmd/README.md", - "data/share/man/man1/s3cmd.1", - ], - ) + expected = [ + "bin/s3cmd", + "data/share/doc/packages/s3cmd/INSTALL.md", + "data/share/doc/packages/s3cmd/LICENSE", + "data/share/doc/packages/s3cmd/NEWS", + "data/share/doc/packages/s3cmd/README.md", + "data/share/man/man1/s3cmd.1", + ] + + self.assertListEqual(actual, expected) def test_dist_info(self): actual = os.environ.get("WHEEL_DIST_INFO_CONTENTS") diff --git a/internal_dev_setup.bzl b/internal_dev_setup.bzl index c37c59a5da..0bbcd97748 100644 --- a/internal_dev_setup.bzl +++ b/internal_dev_setup.bzl @@ -26,6 +26,8 @@ load("//python:versions.bzl", "MINOR_MAPPING", "TOOL_VERSIONS") load("//python/private:pythons_hub.bzl", "hub_repo") # buildifier: disable=bzl-visibility load("//python/private:runtime_env_repo.bzl", "runtime_env_repo") # buildifier: disable=bzl-visibility load("//python/private/pypi:deps.bzl", "pypi_deps") # buildifier: disable=bzl-visibility +load("//python/private/pypi:whl_library.bzl", "whl_library") # buildifier: disable=bzl-visibility +load("//tests/support/whl_from_dir:whl_from_dir_repo.bzl", "whl_from_dir_repo") # buildifier: disable=bzl-visibility def rules_python_internal_setup(): """Setup for development and testing of rules_python itself.""" @@ -59,3 +61,25 @@ def rules_python_internal_setup(): bazel_features_deps() rules_shell_dependencies() rules_shell_toolchains() + + whl_from_dir_repo( + name = "whl_with_data1_whl", + root = "//tests/repos/whl_with_data1:BUILD.bazel", + output = "whl_with_data1-1.0-any-none-any.whl", + ) + whl_library( + name = "whl_with_data1", + whl_file = "@whl_with_data1_whl//:whl_with_data1-1.0-any-none-any.whl", + requirement = "whl-with-data1", + ) + + whl_from_dir_repo( + name = "whl_with_data2_whl", + root = "//tests/repos/whl_with_data2:BUILD.bazel", + output = "whl_with_data2-1.0-any-none-any.whl", + ) + whl_library( + name = "whl_with_data2", + whl_file = "@whl_with_data2_whl//:whl_with_data2-1.0-any-none-any.whl", + requirement = "whl-with-data2", + ) diff --git a/python/config_settings/BUILD.bazel b/python/config_settings/BUILD.bazel index 7060d50b26..fc0ac51451 100644 --- a/python/config_settings/BUILD.bazel +++ b/python/config_settings/BUILD.bazel @@ -229,7 +229,7 @@ string_flag( ) config_setting( - name = "is_venvs_site_packages", + name = "_is_venvs_site_packages_yes", flag_values = { ":venvs_site_packages": VenvsSitePackages.YES, }, diff --git a/python/private/internal_dev_deps.bzl b/python/private/internal_dev_deps.bzl index fbdd5711b1..11b020e59f 100644 --- a/python/private/internal_dev_deps.bzl +++ b/python/private/internal_dev_deps.bzl @@ -91,6 +91,28 @@ def _internal_dev_deps_impl(mctx): enable_implicit_namespace_pkgs = False, ) + whl_from_dir_repo( + name = "whl_with_data1_whl", + root = "//tests/repos/whl_with_data1:BUILD.bazel", + output = "whl_with_data1-1.0-any-none-any.whl", + ) + whl_library( + name = "whl_with_data1", + whl_file = "@whl_with_data1_whl//:whl_with_data1-1.0-any-none-any.whl", + requirement = "whl-with-data1", + ) + + whl_from_dir_repo( + name = "whl_with_data2_whl", + root = "//tests/repos/whl_with_data2:BUILD.bazel", + output = "whl_with_data2-1.0-any-none-any.whl", + ) + whl_library( + name = "whl_with_data2", + whl_file = "@whl_with_data2_whl//:whl_with_data2-1.0-any-none-any.whl", + requirement = "whl-with-data2", + ) + _whl_library_from_dir( name = "whl_library_extras_direct_dep", root = "//tests/pypi/whl_library/testdata/pkg:BUILD.bazel", diff --git a/python/private/py_executable.bzl b/python/private/py_executable.bzl index 375030ce91..6197c0c789 100644 --- a/python/private/py_executable.bzl +++ b/python/private/py_executable.bzl @@ -575,9 +575,13 @@ def _create_venv(ctx, output_prefix, imports, runtime_details, add_runfiles_root computed_substitutions = computed_subs, ) + # See https://docs.python.org/3/library/sysconfig.html#posix-prefix + # for how schemes map under the venv. venv_dir_map = { - VenvSymlinkKind.BIN: venv_details.bin_dir, + VenvSymlinkKind.BIN: "{}/{}".format(venv_ctx_rel_root, venv_details.bin_dir), VenvSymlinkKind.LIB: site_packages, + VenvSymlinkKind.INCLUDE: "{}/{}".format(venv_ctx_rel_root, venv_details.include_dir), + VenvSymlinkKind.DATA: venv_ctx_rel_root, } venv_app_files = create_venv_app_files( ctx, @@ -659,7 +663,7 @@ def _create_venv_unixy(ctx, *, venv_ctx_rel_root, runtime, interpreter_actual_pa recreate_venv_at_runtime = False - bin_dir = "{}/bin".format(venv_ctx_rel_root) + venv_bin_ctx_rel_path = "{}/bin".format(venv_ctx_rel_root) if create_full_venv: # Some wrappers around the interpreter (e.g. pyenv) use the program # name to decide what to do, so preserve the name. @@ -671,7 +675,7 @@ def _create_venv_unixy(ctx, *, venv_ctx_rel_root, runtime, interpreter_actual_pa # When the venv symlinks are disabled, the $venv/bin/python3 file isn't # needed or used at runtime. However, the zip code uses the interpreter # File object to figure out some paths. - interpreter = ctx.actions.declare_file("{}/{}".format(bin_dir, py_exe_basename)) + interpreter = ctx.actions.declare_file("{}/{}".format(venv_bin_ctx_rel_path, py_exe_basename)) ctx.actions.write(interpreter, "actual:{}".format(interpreter_actual_path)) elif runtime.interpreter: @@ -679,7 +683,7 @@ def _create_venv_unixy(ctx, *, venv_ctx_rel_root, runtime, interpreter_actual_pa # declare_symlink() is required to ensure that the resulting file # in runfiles is always a symlink. An RBE implementation, for example, # may choose to write what symlink() points to instead. - interpreter = ctx.actions.declare_symlink("{}/{}".format(bin_dir, py_exe_basename)) + interpreter = ctx.actions.declare_symlink("{}/{}".format(venv_bin_ctx_rel_path, py_exe_basename)) interpreter_runfiles.add(interpreter) rel_path = relative_path( @@ -690,7 +694,7 @@ def _create_venv_unixy(ctx, *, venv_ctx_rel_root, runtime, interpreter_actual_pa ) ctx.actions.symlink(output = interpreter, target_path = rel_path) else: - interpreter = ctx.actions.declare_symlink("{}/{}".format(bin_dir, py_exe_basename)) + interpreter = ctx.actions.declare_symlink("{}/{}".format(venv_bin_ctx_rel_path, py_exe_basename)) interpreter_runfiles.add(interpreter) ctx.actions.symlink(output = interpreter, target_path = runtime.interpreter_path) else: @@ -715,7 +719,8 @@ def _create_venv_unixy(ctx, *, venv_ctx_rel_root, runtime, interpreter_actual_pa interpreter = interpreter, pyvenv_cfg = pyvenv_cfg, site_packages = site_packages, - bin_dir = bin_dir, + bin_dir = "bin", + include_dir = "include", recreate_venv_at_runtime = recreate_venv_at_runtime, interpreter_runfiles = interpreter_runfiles.build(ctx), interpreter_symlinks = depset(), @@ -777,7 +782,8 @@ def _create_venv_windows(ctx, *, venv_ctx_rel_root, runtime, interpreter_actual_ interpreter = interpreter, pyvenv_cfg = None, site_packages = site_packages, - bin_dir = venv_bin_ctx_rel_path, + bin_dir = venv_bin_rel_path, + include_dir = "Include", recreate_venv_at_runtime = True, interpreter_runfiles = interpreter_runfiles.build(ctx), interpreter_symlinks = interpreter_symlinks.build(), @@ -789,6 +795,7 @@ def _venv_details( pyvenv_cfg, site_packages, bin_dir, + include_dir, recreate_venv_at_runtime, interpreter_runfiles, interpreter_symlinks): @@ -801,8 +808,10 @@ def _venv_details( pyvenv_cfg = pyvenv_cfg, # str; venv-relative path to the site-packages directory site_packages = site_packages, - # str; ctx-relative path to the venv's bin directory. + # str; venv-relative path to the venv's bin directory. bin_dir = bin_dir, + # str; venv-relative-path to the venv's include directory. + include_dir = include_dir, # bool; True if the venv needs to be recreated at runtime (because the # build-time construction isn't sufficient). False if the build-time # constructed venv is sufficient. diff --git a/python/private/py_info.bzl b/python/private/py_info.bzl index 8868b9d3b4..551e0bb48c 100644 --- a/python/private/py_info.bzl +++ b/python/private/py_info.bzl @@ -37,6 +37,12 @@ def _VenvSymlinkKind_typedef(): Indicates to create paths under the venv's include directory. ::: + + :::{field} DATA + :type: object + + Indicates to create paths under the venv's data directory. + ::: """ # buildifier: disable=name-conventions @@ -45,6 +51,7 @@ VenvSymlinkKind = struct( BIN = "BIN", LIB = "LIB", INCLUDE = "INCLUDE", + DATA = "DATA", ) def _VenvSymlinkEntry_init(**kwargs): diff --git a/python/private/pypi/whl_library_targets.bzl b/python/private/pypi/whl_library_targets.bzl index dde5f815eb..4ed66cdddc 100644 --- a/python/private/pypi/whl_library_targets.bzl +++ b/python/private/pypi/whl_library_targets.bzl @@ -42,6 +42,8 @@ _BAZEL_REPO_FILE_GLOBS = [ "WORKSPACE.bazel", ] +_IS_VENV_SITE_PACKAGES_YES = Label("//python/config_settings:_is_venvs_site_packages_yes") + def whl_library_targets_from_requires( *, name, @@ -191,7 +193,7 @@ def whl_library_targets( include = ["site-packages/*.dist-info/**"], ), DATA_LABEL: dict( - include = ["data/**"], + include = ["data/**", "bin/**", "include/**"], ), } @@ -351,7 +353,7 @@ def whl_library_targets( if not enable_implicit_namespace_pkgs: generated_namespace_package_files = select({ - Label("//python/config_settings:is_venvs_site_packages"): [], + _IS_VENV_SITE_PACKAGES_YES: [], "//conditions:default": rules.create_inits( srcs = srcs + data + pyi_srcs, ignored_dirnames = [], # If you need to ignore certain folders, you can patch rules_python here to do so. @@ -361,6 +363,10 @@ def whl_library_targets( namespace_package_files += generated_namespace_package_files srcs = srcs + generated_namespace_package_files + # This is done after create_inits() is called so that the data scheme + # files don't have such files created in their directories. + data = data + [DATA_LABEL] + rules.py_library( name = py_library_label, srcs = srcs, diff --git a/python/private/venv_runfiles.bzl b/python/private/venv_runfiles.bzl index 6daf0d4e5c..a94f29f71c 100644 --- a/python/private/venv_runfiles.bzl +++ b/python/private/venv_runfiles.bzl @@ -69,10 +69,16 @@ def create_venv_app_files(ctx, deps, venv_dir_map): ctx.label.package, ) + seen_bin_venv_paths = {} + for kind, kind_map in link_map.items(): base = venv_dir_map[kind] for venv_path, link_to in kind_map.items(): bin_venv_path = paths.join(base, venv_path) + if bin_venv_path in seen_bin_venv_paths: + continue + seen_bin_venv_paths[bin_venv_path] = True + if is_file(link_to): # use paths.join to handle ctx.label.package = "" # runfile_prefix should be prepended as we use runfiles.root_symlinks @@ -80,6 +86,18 @@ def create_venv_app_files(ctx, deps, venv_dir_map): symlink_from = paths.join(runfile_prefix, ctx.label.package, bin_venv_path) runfiles_symlinks[symlink_from] = link_to + + # On Windows, we need to explicitly create the symlink in the venv + # because the bootstrap script won't otherwise know about it. + if is_windows: + rf_path = paths.join(ctx_rf_path, bin_venv_path) + _, _, venv_path = bin_venv_path.partition(".venv/") + explicit_symlinks.append(ExplicitSymlink( + runfiles_path = rf_path, + venv_path = venv_path, + link_to_path = runfiles_root_path(ctx, link_to.short_path), + files = depset([link_to]), + )) elif not is_windows: venv_link = ctx.actions.declare_symlink(bin_venv_path) venv_link_rf_path = runfiles_root_path(ctx, venv_link.short_path) @@ -275,10 +293,16 @@ def _get_file_venv_path(ctx, f, site_packages_root): Returns: A tuple `(venv_path, rf_root_path)` if the file is under - `site_packages_root`, otherwise `(None, None)`. + `site_packages_root` or data/, bin/, include/ otherwise `(None, None)`. """ rf_root_path = runfiles_root_path(ctx, f.short_path) _, _, repo_rel_path = rf_root_path.partition("/") + + # Check for wheel data/bin/include folders first + for prefix in ["data/", "bin/", "include/"]: + if repo_rel_path.startswith(prefix): + return (repo_rel_path, rf_root_path) + head, found_sp_root, venv_path = repo_rel_path.partition(site_packages_root) if head or not found_sp_root: # If head is set, then the path didn't start with site_packages_root @@ -324,6 +348,25 @@ def get_venv_symlinks( all_files = sorted(files, key = lambda f: f.short_path) + cannot_be_linked_directly = {} + for dirname in [ + # The venv directories that bin, include, and data get put into are + # shared across wheels, so we cannot link them directly + "bin", + "include", + "data", + # The data scheme is overlaid on the venv root, so the files under it + # could, in theory, get installed into e.g. bin/ or similar. Explicitly + # mark them as non-directly linkable to avoid issues. + "data/bin", + "data/include", + "data/lib", + "data/Scripts", + "data/Include", + "data/Lib", + ]: + cannot_be_linked_directly[dirname] = True + # dict[str venv-relative dirname, bool is_namespace_package] namespace_package_dirs = { ns: True @@ -331,10 +374,10 @@ def get_venv_symlinks( } # venv paths that cannot be directly linked. Dict acting as set. - cannot_be_linked_directly = { + cannot_be_linked_directly.update({ dirname: True for dirname in namespace_package_dirs.keys() - } + }) for f in namespace_package_files: venv_path, _ = _get_file_venv_path(ctx, f, site_packages_root) if venv_path == None: @@ -452,19 +495,36 @@ def get_venv_symlinks( # Finally, for each group, we create the VenvSymlinkEntry objects for venv_path, files in optimized_groups.items(): + if venv_path.startswith("data/"): + out_venv_path = venv_path[len("data/"):] + kind = VenvSymlinkKind.DATA + prefix = "" + elif venv_path.startswith("include/"): + out_venv_path = venv_path[len("include/"):] + kind = VenvSymlinkKind.INCLUDE + prefix = "" + elif venv_path.startswith("bin/"): + out_venv_path = venv_path[len("bin/"):] + kind = VenvSymlinkKind.BIN + prefix = "" + else: + out_venv_path = venv_path + kind = VenvSymlinkKind.LIB + prefix = site_packages_root + link_to_path = ( _get_label_runfiles_repo(ctx, files[0].owner) + "/" + - site_packages_root + + prefix + venv_path ) venv_symlinks[venv_path] = VenvSymlinkEntry( - kind = VenvSymlinkKind.LIB, + kind = kind, link_to_path = link_to_path, link_to_file = None, package = package, version = version_str, - venv_path = venv_path, + venv_path = out_venv_path, files = depset(files), ) diff --git a/tests/pypi/whl_library_targets/whl_library_targets_tests.bzl b/tests/pypi/whl_library_targets/whl_library_targets_tests.bzl index 28765770f4..60e1f3f3dd 100644 --- a/tests/pypi/whl_library_targets/whl_library_targets_tests.bzl +++ b/tests/pypi/whl_library_targets/whl_library_targets_tests.bzl @@ -50,7 +50,7 @@ def _test_filegroups(env): }, { "name": "data", - "srcs": ["data/**"], + "srcs": ["data/**", "bin/**", "include/**"], "visibility": ["//visibility:public"], }, { @@ -216,11 +216,11 @@ def _test_whl_and_library_deps_from_requires(env): env.expect.that_dict(py_library_call).contains_exactly({ "name": "pkg", "srcs": ["site-packages/foo/SRCS.py"] + select({ - Label("//python/config_settings:is_venvs_site_packages"): [], + Label("//python/config_settings:_is_venvs_site_packages_yes"): [], "//conditions:default": ["_create_inits_target"], }), "pyi_srcs": ["site-packages/foo/PYI.pyi"], - "data": ["site-packages/foo/DATA.txt"], + "data": ["site-packages/foo/DATA.txt", "data"], "imports": ["site-packages"], "deps": ["@pypi//bar:pkg"] + select({ ":is_include_bar_baz_true": ["@pypi//bar_baz:pkg"], @@ -230,7 +230,7 @@ def _test_whl_and_library_deps_from_requires(env): "visibility": ["//visibility:public"], "experimental_venvs_site_packages": Label("//python/config_settings:venvs_site_packages"), "namespace_package_files": [] + select({ - Label("//python/config_settings:is_venvs_site_packages"): [], + Label("//python/config_settings:_is_venvs_site_packages_yes"): [], "//conditions:default": ["_create_inits_target"], }), }) # buildifier: @unsorted-dict-items @@ -332,11 +332,11 @@ def _test_whl_and_library_deps(env): env.expect.that_dict(py_library_calls[0]).contains_exactly({ "name": "pkg", "srcs": ["site-packages/foo/SRCS.py"] + select({ - Label("//python/config_settings:is_venvs_site_packages"): [], + Label("//python/config_settings:_is_venvs_site_packages_yes"): [], "//conditions:default": ["_create_inits_target"], }), "pyi_srcs": ["site-packages/foo/PYI.pyi"], - "data": ["site-packages/foo/DATA.txt"], + "data": ["site-packages/foo/DATA.txt", "data"], "imports": ["site-packages"], "deps": [ "@pypi_bar_baz//:pkg", @@ -357,7 +357,7 @@ def _test_whl_and_library_deps(env): "visibility": ["//visibility:public"], "experimental_venvs_site_packages": Label("//python/config_settings:venvs_site_packages"), "namespace_package_files": [] + select({ - Label("//python/config_settings:is_venvs_site_packages"): [], + Label("//python/config_settings:_is_venvs_site_packages_yes"): [], "//conditions:default": ["_create_inits_target"], }), }) # buildifier: @unsorted-dict-items @@ -414,11 +414,11 @@ def _test_group(env): ).contains_exactly({ "name": "_pkg", "srcs": ["site-packages/foo/srcs.py"] + select({ - Label("//python/config_settings:is_venvs_site_packages"): [], + Label("//python/config_settings:_is_venvs_site_packages_yes"): [], "//conditions:default": ["_create_inits_target"], }), "pyi_srcs": ["site-packages/foo/pyi.pyi"], - "data": ["site-packages/foo/data.txt"], + "data": ["site-packages/foo/data.txt", "data"], "imports": ["site-packages"], "deps": ["@pypi_bar_baz//:pkg"] + select({ "@platforms//os:linux": ["@pypi_box//:pkg"], @@ -429,7 +429,7 @@ def _test_group(env): "visibility": ["@pypi__config//_groups:__pkg__"], "experimental_venvs_site_packages": Label("//python/config_settings:venvs_site_packages"), "namespace_package_files": [] + select({ - Label("//python/config_settings:is_venvs_site_packages"): [], + Label("//python/config_settings:_is_venvs_site_packages_yes"): [], "//conditions:default": ["_create_inits_target"], }), }) # buildifier: @unsorted-dict-items diff --git a/tests/repos/whl_with_data1/BUILD.bazel b/tests/repos/whl_with_data1/BUILD.bazel new file mode 100644 index 0000000000..af49d1ebbf --- /dev/null +++ b/tests/repos/whl_with_data1/BUILD.bazel @@ -0,0 +1 @@ +exports_files(glob(["*"])) diff --git a/tests/repos/whl_with_data1/whl_with_data1-1.0.data/data/bin/data_overlap.sh b/tests/repos/whl_with_data1/whl_with_data1-1.0.data/data/bin/data_overlap.sh new file mode 100644 index 0000000000..b47ce4e9f3 --- /dev/null +++ b/tests/repos/whl_with_data1/whl_with_data1-1.0.data/data/bin/data_overlap.sh @@ -0,0 +1 @@ +echo data_bin diff --git a/tests/repos/whl_with_data1/whl_with_data1-1.0.data/data/include/data_overlap.h b/tests/repos/whl_with_data1/whl_with_data1-1.0.data/data/include/data_overlap.h new file mode 100644 index 0000000000..299c39d0a7 --- /dev/null +++ b/tests/repos/whl_with_data1/whl_with_data1-1.0.data/data/include/data_overlap.h @@ -0,0 +1 @@ +/* data_include */ diff --git a/tests/repos/whl_with_data1/whl_with_data1-1.0.data/data/overlap/both.txt b/tests/repos/whl_with_data1/whl_with_data1-1.0.data/data/overlap/both.txt new file mode 100644 index 0000000000..771c76ed7b --- /dev/null +++ b/tests/repos/whl_with_data1/whl_with_data1-1.0.data/data/overlap/both.txt @@ -0,0 +1 @@ +both1 diff --git a/tests/repos/whl_with_data1/whl_with_data1-1.0.data/data/overlap/data1.txt b/tests/repos/whl_with_data1/whl_with_data1-1.0.data/data/overlap/data1.txt new file mode 100644 index 0000000000..d760283f59 --- /dev/null +++ b/tests/repos/whl_with_data1/whl_with_data1-1.0.data/data/overlap/data1.txt @@ -0,0 +1 @@ +data1 diff --git a/tests/repos/whl_with_data1/whl_with_data1-1.0.data/data/site-packages/data_overlap.py b/tests/repos/whl_with_data1/whl_with_data1-1.0.data/data/site-packages/data_overlap.py new file mode 100644 index 0000000000..d3ee4d8a3f --- /dev/null +++ b/tests/repos/whl_with_data1/whl_with_data1-1.0.data/data/site-packages/data_overlap.py @@ -0,0 +1 @@ +# data_site_packages diff --git a/tests/repos/whl_with_data1/whl_with_data1-1.0.data/data/whl_with_data1/data_data_file.txt b/tests/repos/whl_with_data1/whl_with_data1-1.0.data/data/whl_with_data1/data_data_file.txt new file mode 100644 index 0000000000..39ec676600 --- /dev/null +++ b/tests/repos/whl_with_data1/whl_with_data1-1.0.data/data/whl_with_data1/data_data_file.txt @@ -0,0 +1 @@ +from .data/data diff --git a/tests/repos/whl_with_data1/whl_with_data1-1.0.data/headers/data_overlap.h b/tests/repos/whl_with_data1/whl_with_data1-1.0.data/headers/data_overlap.h new file mode 100644 index 0000000000..ffd49d0cee --- /dev/null +++ b/tests/repos/whl_with_data1/whl_with_data1-1.0.data/headers/data_overlap.h @@ -0,0 +1 @@ +/* headers */ diff --git a/tests/repos/whl_with_data1/whl_with_data1-1.0.data/headers/overlap/both.h b/tests/repos/whl_with_data1/whl_with_data1-1.0.data/headers/overlap/both.h new file mode 100644 index 0000000000..49f33a8c6e --- /dev/null +++ b/tests/repos/whl_with_data1/whl_with_data1-1.0.data/headers/overlap/both.h @@ -0,0 +1 @@ +both diff --git a/tests/repos/whl_with_data1/whl_with_data1-1.0.data/headers/overlap/header1.h b/tests/repos/whl_with_data1/whl_with_data1-1.0.data/headers/overlap/header1.h new file mode 100644 index 0000000000..412e9ed7df --- /dev/null +++ b/tests/repos/whl_with_data1/whl_with_data1-1.0.data/headers/overlap/header1.h @@ -0,0 +1 @@ +header1 diff --git a/tests/repos/whl_with_data1/whl_with_data1-1.0.data/headers/whl_with_data1/header_file.h b/tests/repos/whl_with_data1/whl_with_data1-1.0.data/headers/whl_with_data1/header_file.h new file mode 100644 index 0000000000..59c9bf78c2 --- /dev/null +++ b/tests/repos/whl_with_data1/whl_with_data1-1.0.data/headers/whl_with_data1/header_file.h @@ -0,0 +1 @@ +from .data/headers diff --git a/tests/repos/whl_with_data1/whl_with_data1-1.0.data/platlib/whl_with_data1/platlib_file.txt b/tests/repos/whl_with_data1/whl_with_data1-1.0.data/platlib/whl_with_data1/platlib_file.txt new file mode 100644 index 0000000000..b27295614f --- /dev/null +++ b/tests/repos/whl_with_data1/whl_with_data1-1.0.data/platlib/whl_with_data1/platlib_file.txt @@ -0,0 +1 @@ +from .data/platlib diff --git a/tests/repos/whl_with_data1/whl_with_data1-1.0.data/purelib/data_overlap.py b/tests/repos/whl_with_data1/whl_with_data1-1.0.data/purelib/data_overlap.py new file mode 100644 index 0000000000..f82e46670f --- /dev/null +++ b/tests/repos/whl_with_data1/whl_with_data1-1.0.data/purelib/data_overlap.py @@ -0,0 +1 @@ +# purelib diff --git a/tests/repos/whl_with_data1/whl_with_data1-1.0.data/purelib/whl_with_data1/__init__.py b/tests/repos/whl_with_data1/whl_with_data1-1.0.data/purelib/whl_with_data1/__init__.py new file mode 100644 index 0000000000..e69de29bb2 diff --git a/tests/repos/whl_with_data1/whl_with_data1-1.0.data/purelib/whl_with_data1/data_file.txt b/tests/repos/whl_with_data1/whl_with_data1-1.0.data/purelib/whl_with_data1/data_file.txt new file mode 100644 index 0000000000..e547fe48ed --- /dev/null +++ b/tests/repos/whl_with_data1/whl_with_data1-1.0.data/purelib/whl_with_data1/data_file.txt @@ -0,0 +1 @@ +from .data diff --git a/tests/repos/whl_with_data1/whl_with_data1-1.0.data/scripts/data_overlap.sh b/tests/repos/whl_with_data1/whl_with_data1-1.0.data/scripts/data_overlap.sh new file mode 100644 index 0000000000..d6eb28dc3d --- /dev/null +++ b/tests/repos/whl_with_data1/whl_with_data1-1.0.data/scripts/data_overlap.sh @@ -0,0 +1 @@ +echo scripts diff --git a/tests/repos/whl_with_data1/whl_with_data1-1.0.data/scripts/overlap/both.sh b/tests/repos/whl_with_data1/whl_with_data1-1.0.data/scripts/overlap/both.sh new file mode 100644 index 0000000000..49f33a8c6e --- /dev/null +++ b/tests/repos/whl_with_data1/whl_with_data1-1.0.data/scripts/overlap/both.sh @@ -0,0 +1 @@ +both diff --git a/tests/repos/whl_with_data1/whl_with_data1-1.0.data/scripts/overlap/script1.sh b/tests/repos/whl_with_data1/whl_with_data1-1.0.data/scripts/overlap/script1.sh new file mode 100644 index 0000000000..4d68a2e3e0 --- /dev/null +++ b/tests/repos/whl_with_data1/whl_with_data1-1.0.data/scripts/overlap/script1.sh @@ -0,0 +1 @@ +script1 diff --git a/tests/repos/whl_with_data1/whl_with_data1-1.0.data/scripts/whl_script.sh b/tests/repos/whl_with_data1/whl_with_data1-1.0.data/scripts/whl_script.sh new file mode 100644 index 0000000000..1a2485251c --- /dev/null +++ b/tests/repos/whl_with_data1/whl_with_data1-1.0.data/scripts/whl_script.sh @@ -0,0 +1 @@ +#!/bin/sh diff --git a/tests/repos/whl_with_data1/whl_with_data1-1.0.dist-info/METADATA b/tests/repos/whl_with_data1/whl_with_data1-1.0.dist-info/METADATA new file mode 100644 index 0000000000..f403970d7a --- /dev/null +++ b/tests/repos/whl_with_data1/whl_with_data1-1.0.dist-info/METADATA @@ -0,0 +1,3 @@ +Metadata-Version: 2.1 +Name: whl-with-data1 +Version: 1.0 diff --git a/tests/repos/whl_with_data1/whl_with_data1-1.0.dist-info/RECORD b/tests/repos/whl_with_data1/whl_with_data1-1.0.dist-info/RECORD new file mode 100644 index 0000000000..a39e9ed7ad --- /dev/null +++ b/tests/repos/whl_with_data1/whl_with_data1-1.0.dist-info/RECORD @@ -0,0 +1,18 @@ +whl_with_data1-1.0.data/platlib/whl_with_data1/platlib_file.txt,sha256=123,123 +whl_with_data1-1.0.data/scripts/whl_script.sh,sha256=123,123 +whl_with_data1-1.0.data/headers/whl_with_data1/header_file.h,sha256=123,123 +whl_with_data1-1.0.data/purelib/whl_with_data1/data_file.txt,sha256=123,123 +whl_with_data1-1.0.data/data/whl_with_data1/data_data_file.txt,sha256=123,123 +whl_with_data1-1.0.data/data/whl_with_data1/data_data_file.txt,sha256=123,123 +whl_with_data1-1.0.data/data/overlap/both.txt,sha256=123,123 +whl_with_data1-1.0.data/data/overlap/data1.txt,sha256=123,123 +whl_with_data1-1.0.data/scripts/overlap/both.sh,sha256=123,123 +whl_with_data1-1.0.data/scripts/overlap/script1.sh,sha256=123,123 +whl_with_data1-1.0.data/headers/overlap/both.h,sha256=123,123 +whl_with_data1-1.0.data/headers/overlap/header1.h,sha256=123,123 +whl_with_data1-1.0.data/scripts/data_overlap.sh,sha256=123,123 +whl_with_data1-1.0.data/data/bin/data_overlap.sh,sha256=123,123 +whl_with_data1-1.0.data/headers/data_overlap.h,sha256=123,123 +whl_with_data1-1.0.data/data/include/data_overlap.h,sha256=123,123 +whl_with_data1-1.0.data/purelib/data_overlap.py,sha256=123,123 +whl_with_data1-1.0.data/data/site-packages/data_overlap.py,sha256=123,123 diff --git a/tests/repos/whl_with_data1/whl_with_data1-1.0.dist-info/WHEEL b/tests/repos/whl_with_data1/whl_with_data1-1.0.dist-info/WHEEL new file mode 100644 index 0000000000..a64521a1cc --- /dev/null +++ b/tests/repos/whl_with_data1/whl_with_data1-1.0.dist-info/WHEEL @@ -0,0 +1 @@ +Wheel-Version: 1.0 diff --git a/tests/repos/whl_with_data2/BUILD.bazel b/tests/repos/whl_with_data2/BUILD.bazel new file mode 100644 index 0000000000..af49d1ebbf --- /dev/null +++ b/tests/repos/whl_with_data2/BUILD.bazel @@ -0,0 +1 @@ +exports_files(glob(["*"])) diff --git a/tests/repos/whl_with_data2/whl_with_data2-1.0.data/data/overlap/both.txt b/tests/repos/whl_with_data2/whl_with_data2-1.0.data/data/overlap/both.txt new file mode 100644 index 0000000000..1a8aa8b533 --- /dev/null +++ b/tests/repos/whl_with_data2/whl_with_data2-1.0.data/data/overlap/both.txt @@ -0,0 +1 @@ +both2 diff --git a/tests/repos/whl_with_data2/whl_with_data2-1.0.data/data/overlap/data2.txt b/tests/repos/whl_with_data2/whl_with_data2-1.0.data/data/overlap/data2.txt new file mode 100644 index 0000000000..98d81a2ec6 --- /dev/null +++ b/tests/repos/whl_with_data2/whl_with_data2-1.0.data/data/overlap/data2.txt @@ -0,0 +1 @@ +data2 diff --git a/tests/repos/whl_with_data2/whl_with_data2-1.0.data/data/whl_with_data2/data_data_file.txt b/tests/repos/whl_with_data2/whl_with_data2-1.0.data/data/whl_with_data2/data_data_file.txt new file mode 100644 index 0000000000..39ec676600 --- /dev/null +++ b/tests/repos/whl_with_data2/whl_with_data2-1.0.data/data/whl_with_data2/data_data_file.txt @@ -0,0 +1 @@ +from .data/data diff --git a/tests/repos/whl_with_data2/whl_with_data2-1.0.data/headers/overlap/both.h b/tests/repos/whl_with_data2/whl_with_data2-1.0.data/headers/overlap/both.h new file mode 100644 index 0000000000..49f33a8c6e --- /dev/null +++ b/tests/repos/whl_with_data2/whl_with_data2-1.0.data/headers/overlap/both.h @@ -0,0 +1 @@ +both diff --git a/tests/repos/whl_with_data2/whl_with_data2-1.0.data/headers/overlap/header2.h b/tests/repos/whl_with_data2/whl_with_data2-1.0.data/headers/overlap/header2.h new file mode 100644 index 0000000000..da0a719745 --- /dev/null +++ b/tests/repos/whl_with_data2/whl_with_data2-1.0.data/headers/overlap/header2.h @@ -0,0 +1 @@ +header2 diff --git a/tests/repos/whl_with_data2/whl_with_data2-1.0.data/headers/whl_with_data2/header_file.h b/tests/repos/whl_with_data2/whl_with_data2-1.0.data/headers/whl_with_data2/header_file.h new file mode 100644 index 0000000000..59c9bf78c2 --- /dev/null +++ b/tests/repos/whl_with_data2/whl_with_data2-1.0.data/headers/whl_with_data2/header_file.h @@ -0,0 +1 @@ +from .data/headers diff --git a/tests/repos/whl_with_data2/whl_with_data2-1.0.data/platlib/whl_with_data2/platlib_file.txt b/tests/repos/whl_with_data2/whl_with_data2-1.0.data/platlib/whl_with_data2/platlib_file.txt new file mode 100644 index 0000000000..b27295614f --- /dev/null +++ b/tests/repos/whl_with_data2/whl_with_data2-1.0.data/platlib/whl_with_data2/platlib_file.txt @@ -0,0 +1 @@ +from .data/platlib diff --git a/tests/repos/whl_with_data2/whl_with_data2-1.0.data/purelib/whl_with_data2/__init__.py b/tests/repos/whl_with_data2/whl_with_data2-1.0.data/purelib/whl_with_data2/__init__.py new file mode 100644 index 0000000000..e69de29bb2 diff --git a/tests/repos/whl_with_data2/whl_with_data2-1.0.data/purelib/whl_with_data2/data_file.txt b/tests/repos/whl_with_data2/whl_with_data2-1.0.data/purelib/whl_with_data2/data_file.txt new file mode 100644 index 0000000000..e547fe48ed --- /dev/null +++ b/tests/repos/whl_with_data2/whl_with_data2-1.0.data/purelib/whl_with_data2/data_file.txt @@ -0,0 +1 @@ +from .data diff --git a/tests/repos/whl_with_data2/whl_with_data2-1.0.data/scripts/overlap/both.sh b/tests/repos/whl_with_data2/whl_with_data2-1.0.data/scripts/overlap/both.sh new file mode 100644 index 0000000000..49f33a8c6e --- /dev/null +++ b/tests/repos/whl_with_data2/whl_with_data2-1.0.data/scripts/overlap/both.sh @@ -0,0 +1 @@ +both diff --git a/tests/repos/whl_with_data2/whl_with_data2-1.0.data/scripts/overlap/script2.sh b/tests/repos/whl_with_data2/whl_with_data2-1.0.data/scripts/overlap/script2.sh new file mode 100644 index 0000000000..026ed8b62a --- /dev/null +++ b/tests/repos/whl_with_data2/whl_with_data2-1.0.data/scripts/overlap/script2.sh @@ -0,0 +1 @@ +script2 diff --git a/tests/repos/whl_with_data2/whl_with_data2-1.0.data/scripts/whl_script.sh b/tests/repos/whl_with_data2/whl_with_data2-1.0.data/scripts/whl_script.sh new file mode 100644 index 0000000000..1a2485251c --- /dev/null +++ b/tests/repos/whl_with_data2/whl_with_data2-1.0.data/scripts/whl_script.sh @@ -0,0 +1 @@ +#!/bin/sh diff --git a/tests/repos/whl_with_data2/whl_with_data2-1.0.dist-info/METADATA b/tests/repos/whl_with_data2/whl_with_data2-1.0.dist-info/METADATA new file mode 100644 index 0000000000..c762d184fb --- /dev/null +++ b/tests/repos/whl_with_data2/whl_with_data2-1.0.dist-info/METADATA @@ -0,0 +1,3 @@ +Metadata-Version: 2.1 +Name: whl-with-data2 +Version: 1.0 diff --git a/tests/repos/whl_with_data2/whl_with_data2-1.0.dist-info/RECORD b/tests/repos/whl_with_data2/whl_with_data2-1.0.dist-info/RECORD new file mode 100644 index 0000000000..5eeb915ba7 --- /dev/null +++ b/tests/repos/whl_with_data2/whl_with_data2-1.0.dist-info/RECORD @@ -0,0 +1,12 @@ +whl_with_data2-1.0.data/platlib/whl_with_data2/platlib_file.txt,sha256=123,123 +whl_with_data2-1.0.data/scripts/whl_script.sh,sha256=123,123 +whl_with_data2-1.0.data/headers/whl_with_data2/header_file.h,sha256=123,123 +whl_with_data2-1.0.data/purelib/whl_with_data2/data_file.txt,sha256=123,123 +whl_with_data2-1.0.data/data/whl_with_data2/data_data_file.txt,sha256=123,123 +whl_with_data2-1.0.data/data/whl_with_data2/data_data_file.txt,sha256=123,123 +whl_with_data2-1.0.data/data/overlap/both.txt,sha256=123,123 +whl_with_data2-1.0.data/data/overlap/data2.txt,sha256=123,123 +whl_with_data2-1.0.data/scripts/overlap/both.sh,sha256=123,123 +whl_with_data2-1.0.data/scripts/overlap/script2.sh,sha256=123,123 +whl_with_data2-1.0.data/headers/overlap/both.h,sha256=123,123 +whl_with_data2-1.0.data/headers/overlap/header2.h,sha256=123,123 diff --git a/tests/repos/whl_with_data2/whl_with_data2-1.0.dist-info/WHEEL b/tests/repos/whl_with_data2/whl_with_data2-1.0.dist-info/WHEEL new file mode 100644 index 0000000000..a64521a1cc --- /dev/null +++ b/tests/repos/whl_with_data2/whl_with_data2-1.0.dist-info/WHEEL @@ -0,0 +1 @@ +Wheel-Version: 1.0 diff --git a/tests/venv_site_packages_libs/BUILD.bazel b/tests/venv_site_packages_libs/BUILD.bazel index 56f0eb0909..c99426b375 100644 --- a/tests/venv_site_packages_libs/BUILD.bazel +++ b/tests/venv_site_packages_libs/BUILD.bazel @@ -45,6 +45,8 @@ py_reconfig_test( "@other//nspkg_gamma", "@other//nspkg_single", "@other//with_external_data", + "@whl_with_data1//:pkg", + "@whl_with_data2//:pkg", ], ) diff --git a/tests/venv_site_packages_libs/bin.py b/tests/venv_site_packages_libs/bin.py index 439a964906..368251e75b 100644 --- a/tests/venv_site_packages_libs/bin.py +++ b/tests/venv_site_packages_libs/bin.py @@ -1,5 +1,7 @@ import importlib +import os import sys +import sysconfig import unittest from pathlib import Path @@ -9,7 +11,23 @@ def setUp(self): super().setUp() if sys.prefix == sys.base_prefix: raise AssertionError("Not running under a venv") - self.venv = sys.prefix + self.venv = Path(sys.prefix) + self.site_packages = Path(sysconfig.get_paths()["purelib"]) + + is_windows = sys.platform == "win32" + if is_windows: + self.bin_dir_name = Path("Scripts") + self.include_dir_name = Path("Include") + else: + self.bin_dir_name = Path("bin") + self.include_dir_name = Path("include") + + def assert_venv_path_exists(self, rel_path): + path = self.venv / rel_path + self.assertTrue( + path.exists(), + f"Expected {path} to exist. {path.parent.name} contents: {list(path.parent.iterdir()) if path.parent.exists() else 'N/A'}", + ) def assert_imported_from_venv(self, module_name): module = importlib.import_module(module_name) @@ -20,7 +38,7 @@ def assert_imported_from_venv(self, module_name): + f"__file__ set, but got None. {module=}", ) self.assertTrue( - module.__file__.startswith(self.venv), + module.__file__.startswith(str(self.venv)), f"\n{module_name} was imported, but not from the venv.\n" + f" venv: {self.venv}\n" + f"module file: {module.__file__}\n" @@ -48,12 +66,8 @@ def test_imported_from_venv(self): def test_data_is_included(self): self.assert_imported_from_venv("simple") module = importlib.import_module("simple") - module_path = Path(module.__file__) - - site_packages = module_path.parent.parent - # Ensure that packages from simple v1 are not present - files = [p.name for p in site_packages.glob("*")] + files = [p.name for p in self.site_packages.glob("*")] self.assertIn("simple_v1_extras", files) def test_override_pkg(self): @@ -67,30 +81,77 @@ def test_override_pkg(self): def test_dirs_from_replaced_package_are_not_present(self): self.assert_imported_from_venv("simple") module = importlib.import_module("simple") - module_path = Path(module.__file__) - - site_packages = module_path.parent.parent - dist_info_dirs = [p.name for p in site_packages.glob("*.dist-info")] + dist_info_dirs = [p.name for p in self.site_packages.glob("simple*.dist-info")] self.assertEqual( ["simple-1.0.0.dist-info"], dist_info_dirs, ) # Ensure that packages from simple v1 are not present - files = [p.name for p in site_packages.glob("*")] + files = [p.name for p in self.site_packages.glob("*")] self.assertNotIn("simple.libs", files) def test_data_from_another_pkg_is_included_via_copy_file(self): self.assert_imported_from_venv("simple") module = importlib.import_module("simple") - module_path = Path(module.__file__) - - site_packages = module_path.parent.parent # Ensure that packages from simple v1 are not present - d = site_packages / "external_data" + d = self.site_packages / "external_data" files = [p.name for p in d.glob("*")] self.assertIn("another_module_data.txt", files) + def test_whl_with_data1_included(self): + module = self.assert_imported_from_venv("whl_with_data1") + site_packages_rel = self.site_packages.relative_to(self.venv) + # purelib + self.assert_venv_path_exists(site_packages_rel / "whl_with_data1/data_file.txt") + + # platlib + self.assert_venv_path_exists( + site_packages_rel / "whl_with_data1/platlib_file.txt" + ) + + venv_root = self.venv + + # data + self.assert_venv_path_exists("whl_with_data1/data_data_file.txt") + + # scripts + self.assert_venv_path_exists(self.bin_dir_name / "whl_script.sh") + + # headers + self.assert_venv_path_exists( + self.include_dir_name / "whl_with_data1/header_file.h" + ) + + def test_whl_with_data2_included(self): + module = self.assert_imported_from_venv("whl_with_data2") + + site_packages_rel = self.site_packages.relative_to(self.venv) + self.assert_venv_path_exists(site_packages_rel / "whl_with_data2/data_file.txt") + + self.assert_venv_path_exists(self.bin_dir_name / "whl_script.sh") + + # Ensure that `data` files are unpacked in `venv/root/` + # and then linked as `venv/whl_with_data1/data_data_file.txt`. + self.assert_venv_path_exists("whl_with_data2/data_data_file.txt") + + self.assert_venv_path_exists( + self.include_dir_name / "whl_with_data2/header_file.h" + ) + + def test_whl_with_data_overlap(self): + self.assert_venv_path_exists("overlap/both.txt") + self.assert_venv_path_exists("overlap/data1.txt") + self.assert_venv_path_exists("overlap/data2.txt") + + self.assert_venv_path_exists(self.bin_dir_name / "overlap/both.sh") + self.assert_venv_path_exists(self.bin_dir_name / "overlap/script1.sh") + self.assert_venv_path_exists(self.bin_dir_name / "overlap/script2.sh") + + self.assert_venv_path_exists(self.include_dir_name / "overlap/both.h") + self.assert_venv_path_exists(self.include_dir_name / "overlap/header1.h") + self.assert_venv_path_exists(self.include_dir_name / "overlap/header2.h") + if __name__ == "__main__": unittest.main() From 5d0016a7aff7a41a3ed44abcb704ca3e45b0a7ec Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Mon, 27 Apr 2026 05:52:19 -0700 Subject: [PATCH 721/922] chore: remove groodt from codeowners (#3741) He's no longer active in the project Co-authored-by: Ignas Anikevicius <240938+aignas@users.noreply.github.com> --- .github/CODEOWNERS | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/.github/CODEOWNERS b/.github/CODEOWNERS index 4df29bacdf..a0b4be250a 100644 --- a/.github/CODEOWNERS +++ b/.github/CODEOWNERS @@ -7,5 +7,5 @@ /examples/build_file_generation/ @dougthor42 @aignas # PyPI integration related code -/python/private/pypi/ @rickeylev @aignas @groodt -/tests/pypi/ @rickeylev @aignas @groodt +/python/private/pypi/ @rickeylev @aignas +/tests/pypi/ @rickeylev @aignas From 2e8aa9151945b2fffb69d69874226ea633c26097 Mon Sep 17 00:00:00 2001 From: Ignas Anikevicius <240938+aignas@users.noreply.github.com> Date: Mon, 27 Apr 2026 22:11:38 +0900 Subject: [PATCH 722/922] chore: make pipstar non-switchable (#3737) The pipstar code has been there for some time and might be stable enough to not have a fallback anymore. Related #2949 --------- Co-authored-by: Richard Levasseur --- docs/environment-variables.md | 37 +- docs/readthedocs_build.sh | 3 - examples/pip_parse_vendored/.bazelversion | 1 + python/private/internal_config_repo.bzl | 6 +- python/private/pypi/BUILD.bazel | 3 +- python/private/pypi/attrs.bzl | 33 -- python/private/pypi/evaluate_markers.bzl | 65 ---- python/private/pypi/extension.bzl | 24 +- python/private/pypi/hub_builder.bzl | 79 +--- python/private/pypi/pip_repository.bzl | 55 +-- python/private/pypi/whl_installer/BUILD.bazel | 2 - .../private/pypi/whl_installer/arguments.py | 13 - .../pypi/whl_installer/namespace_pkgs.py | 121 ------ python/private/pypi/whl_installer/platform.py | 303 --------------- python/private/pypi/whl_installer/wheel.py | 263 ------------- .../pypi/whl_installer/wheel_installer.py | 29 -- python/private/pypi/whl_library.bzl | 113 ++---- tests/pypi/extension/extension_tests.bzl | 14 +- tests/pypi/extension/pip_parse.bzl | 3 - tests/pypi/hub_builder/hub_builder_tests.bzl | 7 - tests/pypi/whl_installer/BUILD.bazel | 24 -- tests/pypi/whl_installer/arguments_test.py | 12 - tests/pypi/whl_installer/platform_test.py | 97 ----- .../whl_installer/wheel_installer_test.py | 14 - tests/pypi/whl_installer/wheel_test.py | 345 ------------------ 25 files changed, 87 insertions(+), 1579 deletions(-) create mode 100644 examples/pip_parse_vendored/.bazelversion delete mode 100644 python/private/pypi/whl_installer/namespace_pkgs.py delete mode 100644 python/private/pypi/whl_installer/platform.py delete mode 100644 tests/pypi/whl_installer/platform_test.py delete mode 100644 tests/pypi/whl_installer/wheel_test.py diff --git a/docs/environment-variables.md b/docs/environment-variables.md index d322f601a9..35792415cf 100644 --- a/docs/environment-variables.md +++ b/docs/environment-variables.md @@ -52,32 +52,6 @@ When `1`, `rules_python` will warn users about deprecated functionality that wil be removed in a subsequent major `rules_python` version. Defaults to `0` if unset. ::: -::::{envvar} RULES_PYTHON_ENABLE_PYSTAR - -When `1`, the `rules_python` Starlark implementation of the core rules is used -instead of the Bazel-builtin rules. Note that this requires Bazel 7+. Defaults -to `1`. - -:::{versionadded} 0.26.0 -Defaults to `0` if unspecified. -::: -:::{versionchanged} 0.40.0 -The default became `1` if unspecified -::: -:::: - -::::{envvar} RULES_PYTHON_ENABLE_PIPSTAR - -When `1`, the `rules_python` Starlark implementation of the PyPI/pip integration is used -instead of the legacy Python scripts. - -:::{versionadded} 1.5.0 -::: -:::{versionchanged} 1.7.0 -Flipped to be enabled by default. -::: -:::: - ::::{envvar} RULES_PYTHON_EXTRACT_ROOT Directory to use as the root for creating files necessary for bootstrapping so @@ -174,3 +148,14 @@ os, arch values are the same as the ones mentioned in the When `1`, debug information about coverage behavior is printed to stderr. ::: + +## Removed Environment Variables + +:::{versionremoved} VERSION_NEXT_FEATURE +The following environment variables were removed: + +* `RULES_PYTHON_ENABLE_PYSTAR`: Used to enable the Starlark implementation of + core rules. +* `RULES_PYTHON_ENABLE_PIPSTAR`: Used to enable the Starlark implementation of + PyPI integration. +::: diff --git a/docs/readthedocs_build.sh b/docs/readthedocs_build.sh index 6762ffd630..cd60792a3f 100755 --- a/docs/readthedocs_build.sh +++ b/docs/readthedocs_build.sh @@ -12,10 +12,7 @@ done < <(env -0) # In order to get the build number, we extract it from the host name extra_env+=("--@sphinxdocs//sphinxdocs:extra_env=HOSTNAME=$HOSTNAME") -export RULES_PYTHON_ENABLE_PIPSTAR=1 - set -x -export RULES_PYTHON_ENABLE_PIPSTAR=1 bazel run \ --config=rtd \ "--@sphinxdocs//sphinxdocs:extra_defines=version=$READTHEDOCS_VERSION" \ diff --git a/examples/pip_parse_vendored/.bazelversion b/examples/pip_parse_vendored/.bazelversion new file mode 100644 index 0000000000..35907cd9ca --- /dev/null +++ b/examples/pip_parse_vendored/.bazelversion @@ -0,0 +1 @@ +7.x diff --git a/python/private/internal_config_repo.bzl b/python/private/internal_config_repo.bzl index 524d9c4b9e..aea1eee773 100644 --- a/python/private/internal_config_repo.bzl +++ b/python/private/internal_config_repo.bzl @@ -21,8 +21,6 @@ settings for rules to later use. load("//python/private:text_util.bzl", "render") load(":repo_utils.bzl", "repo_utils") -_ENABLE_PIPSTAR_ENVVAR_NAME = "RULES_PYTHON_ENABLE_PIPSTAR" -_ENABLE_PIPSTAR_DEFAULT = "1" _ENABLE_DEPRECATION_WARNINGS_ENVVAR_NAME = "RULES_PYTHON_DEPRECATION_WARNINGS" _ENABLE_DEPRECATION_WARNINGS_DEFAULT = "0" @@ -31,7 +29,6 @@ config = struct( build_python_zip_default = {build_python_zip_default}, supports_whl_extraction = {supports_whl_extraction}, enable_pystar = True, - enable_pipstar = {enable_pipstar}, enable_deprecation_warnings = {enable_deprecation_warnings}, bazel_8_or_later = {bazel_8_or_later}, bazel_9_or_later = {bazel_9_or_later}, @@ -104,7 +101,6 @@ def _internal_config_repo_impl(rctx): rctx.file("rules_python_config.bzl", _CONFIG_TEMPLATE.format( build_python_zip_default = repo_utils.get_platforms_os_name(rctx) == "windows", - enable_pipstar = _bool_from_environ(rctx, _ENABLE_PIPSTAR_ENVVAR_NAME, _ENABLE_PIPSTAR_DEFAULT), enable_deprecation_warnings = _bool_from_environ(rctx, _ENABLE_DEPRECATION_WARNINGS_ENVVAR_NAME, _ENABLE_DEPRECATION_WARNINGS_DEFAULT), builtin_py_info_symbol = builtin_py_info_symbol, builtin_py_runtime_info_symbol = builtin_py_runtime_info_symbol, @@ -139,7 +135,7 @@ def _internal_config_repo_impl(rctx): internal_config_repo = repository_rule( implementation = _internal_config_repo_impl, configure = True, - environ = [_ENABLE_PIPSTAR_ENVVAR_NAME], + environ = [], attrs = { "transition_setting_generators": attr.string_list_dict(), "transition_settings": attr.string_list(), diff --git a/python/private/pypi/BUILD.bazel b/python/private/pypi/BUILD.bazel index 9ed952889b..e7d19ea636 100644 --- a/python/private/pypi/BUILD.bazel +++ b/python/private/pypi/BUILD.bazel @@ -113,7 +113,6 @@ bzl_library( ":deps_bzl", ":pep508_evaluate_bzl", ":pep508_requirement_bzl", - ":pypi_repo_utils_bzl", ], ) @@ -121,7 +120,6 @@ bzl_library( name = "extension_bzl", srcs = ["extension.bzl"], deps = [ - ":evaluate_markers_bzl", ":hub_builder_bzl", ":hub_repository_bzl", ":parse_whl_name_bzl", @@ -331,6 +329,7 @@ bzl_library( ":attrs_bzl", ":evaluate_markers_bzl", ":parse_requirements_bzl", + ":pep508_env_bzl", ":pip_repository_attrs_bzl", ":pypi_repo_utils_bzl", ":render_pkg_aliases_bzl", diff --git a/python/private/pypi/attrs.bzl b/python/private/pypi/attrs.bzl index a16bf02de5..57bd93f40a 100644 --- a/python/private/pypi/attrs.bzl +++ b/python/private/pypi/attrs.bzl @@ -118,39 +118,6 @@ Warning: If a dependency participates in multiple cycles, all of those cycles must be collapsed down to one. For instance `a <-> b` and `a <-> c` cannot be listed as two separate cycles. -""", - ), - "experimental_target_platforms": attr.string_list( - default = [], - doc = """\ -*NOTE*: This will be removed in the next major version, so please consider migrating -to `bzlmod` and rely on {attr}`pip.parse.requirements_by_platform` for this feature. - -A list of platforms that we will generate the conditional dependency graph for -cross platform wheels by parsing the wheel metadata. This will generate the -correct dependencies for packages like `sphinx` or `pylint`, which include -`colorama` when installed and used on Windows platforms. - -An empty list means falling back to the legacy behaviour where the host -platform is the target platform. - -WARNING: It may not work as expected in cases where the python interpreter -implementation that is being used at runtime is different between different platforms. -This has been tested for CPython only. - -For specific target platforms use values of the form `_` where `` -is one of `linux`, `osx`, `windows` and arch is one of `x86_64`, `x86_32`, -`aarch64`, `s390x` and `ppc64le`. - -You can also target a specific Python version by using `cp3__`. -If multiple python versions are specified as target platforms, then select statements -of the `lib` and `whl` targets will include usage of version aware toolchain config -settings like `@rules_python//python/config_settings:is_python_3.y`. - -Special values: `host` (for generating deps for the host platform only) and -`_*` values. For example, `cp39_*`, `linux_*`, `cp39_linux_*`. - -NOTE: this is not for cross-compiling Python wheels but rather for parsing the `whl` METADATA correctly. """, ), "extra_hub_aliases": attr.string_list_dict( diff --git a/python/private/pypi/evaluate_markers.bzl b/python/private/pypi/evaluate_markers.bzl index 4d6a39a1df..34076290db 100644 --- a/python/private/pypi/evaluate_markers.bzl +++ b/python/private/pypi/evaluate_markers.bzl @@ -14,19 +14,8 @@ """A simple function that evaluates markers using a python interpreter.""" -load(":deps.bzl", "record_files") load(":pep508_evaluate.bzl", "evaluate") load(":pep508_requirement.bzl", "requirement") -load(":pypi_repo_utils.bzl", "pypi_repo_utils") - -# Used as a default value in a rule to ensure we fetch the dependencies. -SRCS = [ - # When the version, or any of the files in `packaging` package changes, - # this file will change as well. - record_files["pypi__packaging"], - Label("//python/private/pypi/requirements_parser:resolve_target_platforms.py"), - Label("//python/private/pypi/whl_installer:platform.py"), -] def evaluate_markers(*, requirements, platforms): """Return the list of supported platforms per requirements line. @@ -51,57 +40,3 @@ def evaluate_markers(*, requirements, platforms): ret.setdefault(req_string, []).append(platform_str) return ret - -def evaluate_markers_py(mrctx, *, requirements, python_interpreter, python_interpreter_target, srcs, logger = None): - """Return the list of supported platforms per requirements line. - - Args: - mrctx: repository_ctx or module_ctx. - requirements: {type}`dict[str, list[str]]` of the requirement file lines to evaluate. - python_interpreter: str, path to the python_interpreter to use to - evaluate the env markers in the given requirements files. It will - be only called if the requirements files have env markers. This - should be something that is in your PATH or an absolute path. - python_interpreter_target: Label, same as python_interpreter, but in a - label format. - srcs: list[Label], the value of SRCS passed from the `rctx` or `mctx` to this function. - logger: repo_utils.logger or None, a simple struct to log diagnostic - messages. Defaults to None. - - Returns: - dict of string lists with target platforms - """ - if not requirements: - return {} - - in_file = mrctx.path("requirements_with_markers.in.json") - out_file = mrctx.path("requirements_with_markers.out.json") - mrctx.file(in_file, json.encode(requirements)) - - interpreter = pypi_repo_utils.resolve_python_interpreter( - mrctx, - python_interpreter = python_interpreter, - python_interpreter_target = python_interpreter_target, - ) - - pypi_repo_utils.execute_checked( - mrctx, - op = "ResolveRequirementEnvMarkers({})".format(in_file), - python = interpreter, - arguments = [ - "-m", - "python.private.pypi.requirements_parser.resolve_target_platforms", - in_file, - out_file, - ], - srcs = srcs, - environment = { - "PYTHONHOME": str(interpreter.dirname), - "PYTHONPATH": [ - Label("@pypi__packaging//:BUILD.bazel"), - Label("//:BUILD.bazel"), - ], - }, - logger = logger, - ) - return json.decode(mrctx.read(out_file)) diff --git a/python/private/pypi/extension.bzl b/python/private/pypi/extension.bzl index a55cf53fb1..78e93b7edd 100644 --- a/python/private/pypi/extension.bzl +++ b/python/private/pypi/extension.bzl @@ -20,7 +20,6 @@ load("@rules_python_internal//:rules_python_config.bzl", rp_config = "config") load("//python/private:auth.bzl", "AUTH_ATTRS") load("//python/private:normalize_name.bzl", "normalize_name") load("//python/private:repo_utils.bzl", "repo_utils") -load(":evaluate_markers.bzl", EVALUATE_MARKERS_SRCS = "SRCS") load(":hub_builder.bzl", "hub_builder") load(":hub_repository.bzl", "hub_repository", "whl_config_settings_to_json") load(":parse_whl_name.bzl", "parse_whl_name") @@ -71,14 +70,11 @@ def _configure(config, *, override = False, **kwargs): def build_config( *, module_ctx, - enable_pipstar, enable_pipstar_extract): """Parse 'configure' and 'default' extension tags Args: module_ctx: {type}`module_ctx` module context. - enable_pipstar: {type}`bool` a flag to enable dropping Python dependency for - evaluation of the extension. enable_pipstar_extract: {type}`bool | None` a flag to also not pass Python interpreter to `whl_library` when possible. @@ -132,7 +128,6 @@ def build_config( name: _plat(**values) for name, values in defaults["platforms"].items() }, - enable_pipstar = enable_pipstar, enable_pipstar_extract = enable_pipstar_extract, ) @@ -140,7 +135,6 @@ def parse_modules( module_ctx, _fail = fail, simpleapi_download = simpleapi_download, - enable_pipstar = False, enable_pipstar_extract = False, **kwargs): """Implementation of parsing the tag classes for the extension and return a struct for registering repositories. @@ -148,8 +142,6 @@ def parse_modules( Args: module_ctx: {type}`module_ctx` module context. simpleapi_download: Used for testing overrides - enable_pipstar: {type}`bool` a flag to enable dropping Python dependency for - evaluation of the extension. enable_pipstar_extract: {type}`bool` a flag to enable dropping Python dependency for extracting wheels. _fail: {type}`function` the failure function, mainly for testing. @@ -189,7 +181,7 @@ You cannot use both the additive_build_content and additive_build_content_file a srcs_exclude_glob = whl_mod.srcs_exclude_glob, ) - config = build_config(module_ctx = module_ctx, enable_pipstar = enable_pipstar, enable_pipstar_extract = enable_pipstar_extract) + config = build_config(module_ctx = module_ctx, enable_pipstar_extract = enable_pipstar_extract) # TODO @aignas 2025-06-03: Merge override API with the builder? _overriden_whl_set = {} @@ -377,8 +369,7 @@ def _pip_impl(module_ctx): mods = parse_modules( module_ctx, - enable_pipstar = rp_config.enable_pipstar, - enable_pipstar_extract = rp_config.enable_pipstar and rp_config.bazel_8_or_later, + enable_pipstar_extract = rp_config.bazel_8_or_later, ) # Build all of the wheel modifications if the tag class is called. @@ -447,10 +438,6 @@ Supported keys: * `platform_system`, defaults to a value inferred from the {attr}`os_name`. * `platform_version`, defaults to `0`. * `sys_platform`, defaults to a value inferred from the {attr}`os_name`. - -::::{note} -This is only used if the {envvar}`RULES_PYTHON_ENABLE_PIPSTAR` is enabled. -:::: """, ), "index_url": attr.string( @@ -716,13 +703,6 @@ a string `"{os}_{arch}"` as the value here. You could also use `"{os}_{arch}_fre doc = """\ A dict of labels to wheel names that is typically generated by the whl_modifications. The labels are JSON config files describing the modifications. -""", - ), - "_evaluate_markers_srcs": attr.label_list( - default = EVALUATE_MARKERS_SRCS, - doc = """\ -The list of labels to use as SRCS for the marker evaluation code. This ensures that the -code will be re-evaluated when any of files in the default changes. """, ), }, **ATTRS) diff --git a/python/private/pypi/hub_builder.bzl b/python/private/pypi/hub_builder.bzl index 484731566f..85a31cfc3c 100644 --- a/python/private/pypi/hub_builder.bzl +++ b/python/private/pypi/hub_builder.bzl @@ -7,7 +7,7 @@ load("//python/private:text_util.bzl", "render") load("//python/private:version.bzl", "version") load("//python/private:version_label.bzl", "version_label") load(":attrs.bzl", "use_isolated") -load(":evaluate_markers.bzl", "evaluate_markers_py", evaluate_markers_star = "evaluate_markers") +load(":evaluate_markers.bzl", evaluate_markers_star = "evaluate_markers") load(":parse_requirements.bzl", "parse_requirements") load(":pep508_env.bzl", "env") load(":pep508_evaluate.bzl", "evaluate") @@ -194,7 +194,6 @@ def _pip_parse(self, module_ctx, pip_attr): self, module_ctx, pip_attr = pip_attr, - enable_pipstar = bool(self._config.enable_pipstar or self._get_index_urls.get(pip_attr.python_version)), enable_pipstar_extract = bool(self._config.enable_pipstar_extract or self._get_index_urls.get(pip_attr.python_version)), ) @@ -297,7 +296,7 @@ def _diff_dict(first, second): else: return None -def _add_whl_library(self, *, python_version, whl, repo, enable_pipstar): +def _add_whl_library(self, *, python_version, whl, repo): """Add a whl_library and kwargs to call it with for the hub. Args: @@ -305,7 +304,6 @@ def _add_whl_library(self, *, python_version, whl, repo, enable_pipstar): python_version: {type}`str` the python version to assume whl: struct from `_whl_library_args()` repo: struct from `_whl_repo` - enable_pipstar: {type}`bool` if pipstar is enabled. """ if repo == None: # NOTE @aignas 2025-07-07: we guard against an edge-case where there @@ -313,8 +311,6 @@ def _add_whl_library(self, *, python_version, whl, repo, enable_pipstar): # disallow building from sdist. return - platforms = self._platforms[python_version] - # TODO @aignas 2025-06-29: we should not need the version in the repo_name if # we are using pipstar and we are downloading the wheel using the downloader # @@ -339,17 +335,6 @@ def _add_whl_library(self, *, python_version, whl, repo, enable_pipstar): return self._whl_libraries[repo_name] = repo.args - if not enable_pipstar and "experimental_target_platforms" in repo.args: - self._whl_libraries[repo_name] |= { - "experimental_target_platforms": sorted({ - # TODO @aignas 2025-07-07: this should be solved in a better way - platforms[candidate].triple.partition("_")[-1]: None - for p in repo.args["experimental_target_platforms"] - for candidate in platforms - if candidate.endswith(p) - }), - } - mapping = self._whl_map.setdefault(whl.name, {}) if repo.config_setting in mapping and mapping[repo.config_setting] != repo_name: fail( @@ -480,45 +465,13 @@ def _platforms(module_ctx, *, python_version, config, target_platforms): ) return platforms -def _evaluate_markers(self, pip_attr, enable_pipstar): +def _evaluate_markers(self, pip_attr): if self._evaluate_markers_fn: return self._evaluate_markers_fn - if enable_pipstar: - return lambda _, requirements: evaluate_markers_star( - requirements = requirements, - platforms = self._platforms[pip_attr.python_version], - ) - - interpreter = _detect_interpreter(self, pip_attr) - - # NOTE @aignas 2024-08-02: , we will execute any interpreter that we find either - # in the PATH or if specified as a label. We will configure the env - # markers when evaluating the requirement lines based on the output - # from the `requirements_files_by_platform` which should have something - # similar to: - # { - # "//:requirements.txt": ["cp311_linux_x86_64", ...] - # } - # - # We know the target python versions that we need to evaluate the - # markers for and thus we don't need to use multiple python interpreter - # instances to perform this manipulation. This function should be executed - # only once by the underlying code to minimize the overhead needed to - # spin up a Python interpreter. - return lambda module_ctx, requirements: evaluate_markers_py( - module_ctx, - requirements = { - k: { - p: self._platforms[pip_attr.python_version][p].triple - for p in plats - } - for k, plats in requirements.items() - }, - python_interpreter = interpreter.path, - python_interpreter_target = interpreter.target, - srcs = pip_attr._evaluate_markers_srcs, - logger = self._logger, + return lambda _, requirements: evaluate_markers_star( + requirements = requirements, + platforms = self._platforms[pip_attr.python_version], ) def _create_whl_repos( @@ -526,7 +479,6 @@ def _create_whl_repos( module_ctx, *, pip_attr, - enable_pipstar = False, enable_pipstar_extract = False): """create all of the whl repositories @@ -534,7 +486,6 @@ def _create_whl_repos( self: the builder. module_ctx: {type}`module_ctx`. pip_attr: {type}`struct` - the struct that comes from the tag class iteration. - enable_pipstar: {type}`bool` - enable the pipstar or not. enable_pipstar_extract: {type}`bool` - enable the pipstar extraction or not. """ logger = self._logger @@ -558,7 +509,7 @@ def _create_whl_repos( platforms = platforms, extra_pip_args = pip_attr.extra_pip_args, get_index_urls = self._get_index_urls.get(pip_attr.python_version), - evaluate_markers = _evaluate_markers(self, pip_attr, enable_pipstar), + evaluate_markers = _evaluate_markers(self, pip_attr), logger = logger, ) @@ -577,7 +528,6 @@ def _create_whl_repos( self, module_ctx, pip_attr = pip_attr, - enable_pipstar = enable_pipstar, ) interpreter = _detect_interpreter(self, pip_attr) @@ -600,7 +550,6 @@ def _create_whl_repos( python_version = _major_minor_version(pip_attr.python_version), is_multiple_versions = whl.is_multiple_versions, interpreter = interpreter, - enable_pipstar = enable_pipstar, enable_pipstar_extract = enable_pipstar_extract, ) _add_whl_library( @@ -608,10 +557,9 @@ def _create_whl_repos( python_version = pip_attr.python_version, whl = whl, repo = repo, - enable_pipstar = enable_pipstar, ) -def _common_args(self, module_ctx, *, pip_attr, enable_pipstar): +def _common_args(self, module_ctx, *, pip_attr): # Construct args separately so that the lock file can be smaller and does not include unused # attrs. whl_library_args = dict( @@ -627,8 +575,6 @@ def _common_args(self, module_ctx, *, pip_attr, enable_pipstar): envsubst = pip_attr.envsubst, pip_data_exclude = pip_attr.pip_data_exclude, ) - if not enable_pipstar: - maybe_args["experimental_target_platforms"] = pip_attr.experimental_target_platforms whl_library_args.update({k: v for k, v in maybe_args.items() if v}) maybe_args_with_default = dict( @@ -679,7 +625,6 @@ def _whl_repo( python_version, use_downloader, interpreter, - enable_pipstar = False, enable_pipstar_extract = False): args = dict(whl_library_args) args["requirement"] = src.requirement_line @@ -731,14 +676,6 @@ def _whl_repo( args["urls"] = [src.url] args["sha256"] = src.sha256 args["filename"] = src.filename - if not enable_pipstar: - args["experimental_target_platforms"] = [ - # Get rid of the version for the target platforms because we are - # passing the interpreter any way. Ideally we should search of ways - # how to pass the target platforms through the hub repo. - p.partition("_")[2] - for p in src.target_platforms - ] # TODO @aignas 2025-11-02: once we have pipstar enabled we can add extra # targets to each hub for each extra combination and solve this more cleanly as opposed to diff --git a/python/private/pypi/pip_repository.bzl b/python/private/pypi/pip_repository.bzl index 4560bd7eef..b449f8dd85 100644 --- a/python/private/pypi/pip_repository.bzl +++ b/python/private/pypi/pip_repository.bzl @@ -18,9 +18,11 @@ load("@bazel_skylib//lib:sets.bzl", "sets") load("//python/private:normalize_name.bzl", "normalize_name") load("//python/private:repo_utils.bzl", "REPO_DEBUG_ENV_VAR", "repo_utils") load("//python/private:text_util.bzl", "render") -load(":evaluate_markers.bzl", "evaluate_markers_py", EVALUATE_MARKERS_SRCS = "SRCS") +load(":evaluate_markers.bzl", "evaluate_markers") load(":parse_requirements.bzl", "host_platform", "parse_requirements", "select_requirement") +load(":pep508_env.bzl", "env") load(":pip_repository_attrs.bzl", "ATTRS") +load(":pypi_repo_utils.bzl", "pypi_repo_utils") load(":render_pkg_aliases.bzl", "render_pkg_aliases") load(":requirements_files_by_platform.bzl", "requirements_files_by_platform") @@ -83,6 +85,29 @@ exports_files(["requirements.bzl"]) def _pip_repository_impl(rctx): logger = repo_utils.logger(rctx) + python_interpreter = pypi_repo_utils.resolve_python_interpreter( + rctx, + python_interpreter = rctx.attr.python_interpreter, + python_interpreter_target = rctx.attr.python_interpreter_target, + ) + result = rctx.execute([python_interpreter, "--version"]) + if result.stdout: + python_version = result.stdout.strip().split(" ")[-1] + else: + fail("Could not determine Python version") + platforms = [ + "linux_aarch64", + "linux_arm", + "linux_ppc", + "linux_riscv64", + "linux_s390x", + "linux_x86_64", + "osx_aarch64", + "osx_x86_64", + "windows_x86_64", + ] + + marker_env = env(os = rctx.os.name, arch = rctx.os.arch, python_version = python_version) requirements_by_platform = parse_requirements( rctx, requirements_by_platform = requirements_files_by_platform( @@ -92,30 +117,17 @@ def _pip_repository_impl(rctx): requirements_osx = rctx.attr.requirements_darwin, requirements_windows = rctx.attr.requirements_windows, extra_pip_args = rctx.attr.extra_pip_args, - platforms = [ - "linux_aarch64", - "linux_arm", - "linux_ppc", - "linux_riscv64", - "linux_s390x", - "linux_x86_64", - "osx_aarch64", - "osx_x86_64", - "windows_x86_64", - ], + platforms = platforms, ), extra_pip_args = rctx.attr.extra_pip_args, - evaluate_markers = lambda rctx, requirements: evaluate_markers_py( - rctx, + evaluate_markers = lambda rctx, requirements: evaluate_markers( requirements = { # NOTE @aignas 2025-07-07: because we don't distinguish between # freethreaded and non-freethreaded, it is a 1:1 mapping. req: {p: p for p in plats} for req, plats in requirements.items() }, - python_interpreter = rctx.attr.python_interpreter, - python_interpreter_target = rctx.attr.python_interpreter_target, - srcs = rctx.attr._evaluate_markers_srcs, + platforms = {p: struct(env = marker_env) for p in platforms}, ), extract_url_srcs = False, logger = logger, @@ -197,8 +209,6 @@ def _pip_repository_impl(rctx): if rctx.attr.python_interpreter_target: config["python_interpreter_target"] = str(rctx.attr.python_interpreter_target) - if rctx.attr.experimental_target_platforms: - config["experimental_target_platforms"] = rctx.attr.experimental_target_platforms macro_tmpl = "@%s//{}:{}" % rctx.attr.name @@ -275,13 +285,6 @@ generated using the `package_name` macro. _template = attr.label( default = ":requirements.bzl.tmpl.workspace", ), - _evaluate_markers_srcs = attr.label_list( - default = EVALUATE_MARKERS_SRCS, - doc = """\ -The list of labels to use as SRCS for the marker evaluation code. This ensures that the -code will be re-evaluated when any of files in the default changes. -""", - ), **ATTRS ), doc = """Accepts a locked/compiled requirements file and installs the dependencies listed within. diff --git a/python/private/pypi/whl_installer/BUILD.bazel b/python/private/pypi/whl_installer/BUILD.bazel index 5fb617004d..b820b843d4 100644 --- a/python/private/pypi/whl_installer/BUILD.bazel +++ b/python/private/pypi/whl_installer/BUILD.bazel @@ -5,8 +5,6 @@ py_library( name = "lib", srcs = [ "arguments.py", - "namespace_pkgs.py", - "platform.py", "wheel.py", "wheel_installer.py", ], diff --git a/python/private/pypi/whl_installer/arguments.py b/python/private/pypi/whl_installer/arguments.py index 57dae45ae9..0d31ab7c4d 100644 --- a/python/private/pypi/whl_installer/arguments.py +++ b/python/private/pypi/whl_installer/arguments.py @@ -17,8 +17,6 @@ import pathlib from typing import Any, Dict, Set -from python.private.pypi.whl_installer.platform import Platform - def parser(**kwargs: Any) -> argparse.ArgumentParser: """Create a parser for the wheel_installer tool.""" @@ -41,17 +39,6 @@ def parser(**kwargs: Any) -> argparse.ArgumentParser: action="store", help="Extra arguments to pass down to pip.", ) - parser.add_argument( - "--platform", - action="extend", - type=Platform.from_string, - help="Platforms to target dependencies. Can be used multiple times.", - ) - parser.add_argument( - "--enable-pipstar", - action="store_true", - help="Disable certain code paths if we expect to process the whl in Starlark.", - ) parser.add_argument( "--pip_data_exclude", action="store", diff --git a/python/private/pypi/whl_installer/namespace_pkgs.py b/python/private/pypi/whl_installer/namespace_pkgs.py deleted file mode 100644 index b415844ace..0000000000 --- a/python/private/pypi/whl_installer/namespace_pkgs.py +++ /dev/null @@ -1,121 +0,0 @@ -# Copyright 2023 The Bazel Authors. All rights reserved. -# -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -"""Utility functions to discover python package types""" -import os -import textwrap -from pathlib import Path # supported in >= 3.4 -from typing import List, Optional, Set - - -def implicit_namespace_packages( - directory: str, ignored_dirnames: Optional[List[str]] = None -) -> Set[Path]: - """Discovers namespace packages implemented using the 'native namespace packages' method. - - AKA 'implicit namespace packages', which has been supported since Python 3.3. - See: https://packaging.python.org/guides/packaging-namespace-packages/#native-namespace-packages - - Args: - directory: The root directory to recursively find packages in. - ignored_dirnames: A list of directories to exclude from the search - - Returns: - The set of directories found under root to be packages using the native namespace method. - """ - namespace_pkg_dirs: Set[Path] = set() - standard_pkg_dirs: Set[Path] = set() - directory_path = Path(directory) - ignored_dirname_paths: List[Path] = [Path(p) for p in ignored_dirnames or ()] - # Traverse bottom-up because a directory can be a namespace pkg because its child contains module files. - for dirpath, dirnames, filenames in map( - lambda t: (Path(t[0]), *t[1:]), os.walk(directory_path, topdown=False) - ): - if "__init__.py" in filenames: - standard_pkg_dirs.add(dirpath) - continue - elif ignored_dirname_paths: - is_ignored_dir = dirpath in ignored_dirname_paths - child_of_ignored_dir = any( - d in dirpath.parents for d in ignored_dirname_paths - ) - if is_ignored_dir or child_of_ignored_dir: - continue - - dir_includes_py_modules = _includes_python_modules(filenames) - parent_of_namespace_pkg = any( - Path(dirpath, d) in namespace_pkg_dirs for d in dirnames - ) - parent_of_standard_pkg = any( - Path(dirpath, d) in standard_pkg_dirs for d in dirnames - ) - parent_of_pkg = parent_of_namespace_pkg or parent_of_standard_pkg - if ( - (dir_includes_py_modules or parent_of_pkg) - and - # The root of the directory should never be an implicit namespace - dirpath != directory_path - ): - namespace_pkg_dirs.add(dirpath) - return namespace_pkg_dirs - - -def add_pkgutil_style_namespace_pkg_init(dir_path: Path) -> None: - """Adds 'pkgutil-style namespace packages' init file to the given directory - - See: https://packaging.python.org/guides/packaging-namespace-packages/#pkgutil-style-namespace-packages - - Args: - dir_path: The directory to create an __init__.py for. - - Raises: - ValueError: If the directory already contains an __init__.py file - """ - ns_pkg_init_filepath = os.path.join(dir_path, "__init__.py") - - if os.path.isfile(ns_pkg_init_filepath): - raise ValueError("%s already contains an __init__.py file." % dir_path) - - with open(ns_pkg_init_filepath, "w") as ns_pkg_init_f: - # See https://packaging.python.org/guides/packaging-namespace-packages/#pkgutil-style-namespace-packages - ns_pkg_init_f.write( - textwrap.dedent( - """\ - # __path__ manipulation added by bazel-contrib/rules_python to support namespace pkgs. - __path__ = __import__('pkgutil').extend_path(__path__, __name__) - """ - ) - ) - - -def _includes_python_modules(files: List[str]) -> bool: - """ - In order to only transform directories that Python actually considers namespace pkgs - we need to detect if a directory includes Python modules. - - Which files are loadable as modules is extension based, and the particular set of extensions - varies by platform. - - See: - 1. https://github.com/python/cpython/blob/7d9d25dbedfffce61fc76bc7ccbfa9ae901bf56f/Lib/importlib/machinery.py#L19 - 2. PEP 420 -- Implicit Namespace Packages, Specification - https://www.python.org/dev/peps/pep-0420/#specification - 3. dynload_shlib.c and dynload_win.c in python/cpython. - """ - module_suffixes = { - ".py", # Source modules - ".pyc", # Compiled bytecode modules - ".so", # Unix extension modules - ".pyd", # https://docs.python.org/3/faq/windows.html#is-a-pyd-file-the-same-as-a-dll - } - return any(Path(f).suffix in module_suffixes for f in files) diff --git a/python/private/pypi/whl_installer/platform.py b/python/private/pypi/whl_installer/platform.py deleted file mode 100644 index 0757d86990..0000000000 --- a/python/private/pypi/whl_installer/platform.py +++ /dev/null @@ -1,303 +0,0 @@ -# Copyright 2024 The Bazel Authors. All rights reserved. -# -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -"""Utility class to inspect an extracted wheel directory""" - -import platform -import sys -from dataclasses import dataclass -from enum import Enum -from typing import Any, Dict, Iterator, List, Optional, Tuple, Union - - -class OS(Enum): - linux = 1 - osx = 2 - windows = 3 - darwin = osx - win32 = windows - - @classmethod - def interpreter(cls) -> "OS": - "Return the interpreter operating system." - return cls[sys.platform.lower()] - - def __str__(self) -> str: - return self.name.lower() - - -class Arch(Enum): - x86_64 = 1 - x86_32 = 2 - aarch64 = 3 - ppc = 4 - ppc64le = 5 - s390x = 6 - arm = 7 - riscv64 = 8 - amd64 = x86_64 - arm64 = aarch64 - i386 = x86_32 - i686 = x86_32 - x86 = x86_32 - - @classmethod - def interpreter(cls) -> "Arch": - "Return the currently running interpreter architecture." - # FIXME @aignas 2023-12-13: Hermetic toolchain on Windows 3.11.6 - # is returning an empty string here, so lets default to x86_64 - return cls[platform.machine().lower() or "x86_64"] - - def __str__(self) -> str: - return self.name.lower() - - -def _as_int(value: Optional[Union[OS, Arch]]) -> int: - """Convert one of the enums above to an int for easier sorting algorithms. - - Args: - value: The value of an enum or None. - - Returns: - -1 if we get None, otherwise, the numeric value of the given enum. - """ - if value is None: - return -1 - - return int(value.value) - - -def host_interpreter_version() -> Tuple[int, int]: - return (sys.version_info.minor, sys.version_info.micro) - - -@dataclass(frozen=True) -class Platform: - os: Optional[OS] = None - arch: Optional[Arch] = None - minor_version: Optional[int] = None - micro_version: Optional[int] = None - - @classmethod - def all( - cls, - want_os: Optional[OS] = None, - minor_version: Optional[int] = None, - micro_version: Optional[int] = None, - ) -> List["Platform"]: - return sorted( - [ - cls( - os=os, - arch=arch, - minor_version=minor_version, - micro_version=micro_version, - ) - for os in OS - for arch in Arch - if not want_os or want_os == os - ] - ) - - @classmethod - def host(cls) -> List["Platform"]: - """Use the Python interpreter to detect the platform. - - We extract `os` from sys.platform and `arch` from platform.machine - - Returns: - A list of parsed values which makes the signature the same as - `Platform.all` and `Platform.from_string`. - """ - minor, micro = host_interpreter_version() - return [ - Platform( - os=OS.interpreter(), - arch=Arch.interpreter(), - minor_version=minor, - micro_version=micro, - ) - ] - - def __lt__(self, other: Any) -> bool: - """Add a comparison method, so that `sorted` returns the most specialized platforms first.""" - if not isinstance(other, Platform) or other is None: - raise ValueError(f"cannot compare {other} with Platform") - - self_arch, self_os = _as_int(self.arch), _as_int(self.os) - other_arch, other_os = _as_int(other.arch), _as_int(other.os) - - if self_os == other_os: - return self_arch < other_arch - else: - return self_os < other_os - - def __str__(self) -> str: - if self.minor_version is None: - return f"{self.os}_{self.arch}" - - minor_version = self.minor_version - micro_version = self.micro_version - - if micro_version is None: - return f"cp3{minor_version}_{self.os}_{self.arch}" - else: - return f"cp3{minor_version}.{micro_version}_{self.os}_{self.arch}" - - @classmethod - def from_string(cls, platform: Union[str, List[str]]) -> List["Platform"]: - """Parse a string and return a list of platforms""" - platform = [platform] if isinstance(platform, str) else list(platform) - ret = set() - for p in platform: - if p == "host": - ret.update(cls.host()) - continue - - abi, _, tail = p.partition("_") - if not abi.startswith("cp"): - # The first item is not an abi - tail = p - abi = "" - os, _, arch = tail.partition("_") - arch = arch or "*" - - if abi: - tail = abi[len("cp3") :] - minor_version, _, micro_version = tail.partition(".") - minor_version = int(minor_version) - if micro_version == "": - micro_version = None - else: - micro_version = int(micro_version) - else: - minor_version = None - micro_version = None - - if arch != "*": - ret.add( - cls( - os=OS[os] if os != "*" else None, - arch=Arch[arch], - minor_version=minor_version, - micro_version=micro_version, - ) - ) - - else: - ret.update( - cls.all( - want_os=OS[os] if os != "*" else None, - minor_version=minor_version, - micro_version=micro_version, - ) - ) - - return sorted(ret) - - # NOTE @aignas 2023-12-05: below is the minimum number of accessors that are defined in - # https://peps.python.org/pep-0496/ to make rules_python generate dependencies. - # - # WARNING: It may not work in cases where the python implementation is different between - # different platforms. - - # derived from OS - @property - def os_name(self) -> str: - if self.os == OS.linux or self.os == OS.osx: - return "posix" - elif self.os == OS.windows: - return "nt" - else: - return "" - - @property - def sys_platform(self) -> str: - if self.os == OS.linux: - return "linux" - elif self.os == OS.osx: - return "darwin" - elif self.os == OS.windows: - return "win32" - else: - return "" - - @property - def platform_system(self) -> str: - if self.os == OS.linux: - return "Linux" - elif self.os == OS.osx: - return "Darwin" - elif self.os == OS.windows: - return "Windows" - else: - return "" - - # derived from OS and Arch - @property - def platform_machine(self) -> str: - """Guess the target 'platform_machine' marker. - - NOTE @aignas 2023-12-05: this may not work on really new systems, like - Windows if they define the platform markers in a different way. - """ - if self.arch == Arch.x86_64: - return "x86_64" - elif self.arch == Arch.x86_32 and self.os != OS.osx: - return "i386" - elif self.arch == Arch.x86_32: - return "" - elif self.arch == Arch.aarch64 and self.os == OS.linux: - return "aarch64" - elif self.arch == Arch.aarch64: - # Assuming that OSX and Windows use this one since the precedent is set here: - # https://github.com/cgohlke/win_arm64-wheels - return "arm64" - elif self.os != OS.linux: - return "" - elif self.arch == Arch.ppc: - return "ppc" - elif self.arch == Arch.ppc64le: - return "ppc64le" - elif self.arch == Arch.riscv64: - return "riscv64" - elif self.arch == Arch.s390x: - return "s390x" - else: - return "" - - def env_markers(self, extra: str) -> Dict[str, str]: - # If it is None, use the host version - if self.minor_version is None: - minor, micro = host_interpreter_version() - else: - minor, micro = self.minor_version, self.micro_version - - micro = micro or 0 - - return { - "extra": extra, - "os_name": self.os_name, - "sys_platform": self.sys_platform, - "platform_machine": self.platform_machine, - "platform_system": self.platform_system, - "platform_release": "", # unset - "platform_version": "", # unset - "python_version": f"3.{minor}", - "implementation_version": f"3.{minor}.{micro}", - "python_full_version": f"3.{minor}.{micro}", - # we assume that the following are the same as the interpreter used to setup the deps: - # "implementation_name": "cpython" - # "platform_python_implementation: "CPython", - } diff --git a/python/private/pypi/whl_installer/wheel.py b/python/private/pypi/whl_installer/wheel.py index 25003e6280..4987c915cc 100644 --- a/python/private/pypi/whl_installer/wheel.py +++ b/python/private/pypi/whl_installer/wheel.py @@ -15,233 +15,9 @@ """Utility class to inspect an extracted wheel directory""" import email -import re -from collections import defaultdict -from dataclasses import dataclass from pathlib import Path -from typing import Dict, List, Optional, Set, Tuple import installer -from packaging.requirements import Requirement -from pip._vendor.packaging.utils import canonicalize_name - -from python.private.pypi.whl_installer.platform import ( - Platform, - host_interpreter_version, -) - - -@dataclass(frozen=True) -class FrozenDeps: - deps: List[str] - deps_select: Dict[str, List[str]] - - -class Deps: - """Deps is a dependency builder that has a build() method to return FrozenDeps.""" - - def __init__( - self, - name: str, - requires_dist: List[str], - *, - extras: Optional[Set[str]] = None, - platforms: Optional[Set[Platform]] = None, - ): - """Create a new instance and parse the requires_dist - - Args: - name (str): The name of the whl distribution - requires_dist (list[Str]): The Requires-Dist from the METADATA of the whl - distribution. - extras (set[str], optional): The list of requested extras, defaults to None. - platforms (set[Platform], optional): The list of target platforms, defaults to - None. If the list of platforms has multiple `minor_version` values, it - will change the code to generate the select statements using - `@rules_python//python/config_settings:is_python_3.y` conditions. - """ - self.name: str = Deps._normalize(name) - self._platforms: Set[Platform] = platforms or set() - self._target_versions = { - (p.minor_version, p.micro_version) for p in platforms or {} - } - if platforms and len(self._target_versions) > 1: - # TODO @aignas 2024-06-23: enable this to be set via a CLI arg - # for being more explicit. - self._default_minor_version, _ = host_interpreter_version() - else: - self._default_minor_version = None - - if None in self._target_versions and len(self._target_versions) > 2: - raise ValueError( - f"all python versions need to be specified explicitly, got: {platforms}" - ) - - # Sort so that the dictionary order in the FrozenDeps is deterministic - # without the final sort because Python retains insertion order. That way - # the sorting by platform is limited within the Platform class itself and - # the unit-tests for the Deps can be simpler. - reqs = sorted( - (Requirement(wheel_req) for wheel_req in requires_dist), - key=lambda x: f"{x.name}:{sorted(x.extras)}", - ) - - want_extras = self._resolve_extras(reqs, extras) - - # Then add all of the requirements in order - self._deps: Set[str] = set() - self._select: Dict[Platform, Set[str]] = defaultdict(set) - - reqs_by_name = {} - for req in reqs: - reqs_by_name.setdefault(req.name, []).append(req) - - for req_name, reqs in reqs_by_name.items(): - self._add_req(req_name, reqs, want_extras) - - def _add(self, dep: str, platform: Optional[Platform]): - dep = Deps._normalize(dep) - - # Self-edges are processed in _resolve_extras - if dep == self.name: - return - - if not platform: - self._deps.add(dep) - - # If the dep is in the platform-specific list, remove it from the select. - pop_keys = [] - for p, deps in self._select.items(): - if dep not in deps: - continue - - deps.remove(dep) - if not deps: - pop_keys.append(p) - - for p in pop_keys: - self._select.pop(p) - return - - if dep in self._deps: - # If the dep is already in the main dependency list, no need to add it in the - # platform-specific dependency list. - return - - # Add the platform-specific dep - self._select[platform].add(dep) - - @staticmethod - def _normalize(name: str) -> str: - return re.sub(r"[-_.]+", "_", name).lower() - - def _resolve_extras( - self, reqs: List[Requirement], want_extras: Optional[Set[str]] - ) -> Set[str]: - """Resolve extras which are due to depending on self[some_other_extra]. - - Some packages may have cyclic dependencies resulting from extras being used, one example is - `etils`, where we have one set of extras as aliases for other extras - and we have an extra called 'all' that includes all other extras. - - Example: github.com/google/etils/blob/a0b71032095db14acf6b33516bca6d885fe09e35/pyproject.toml#L32. - - When the `requirements.txt` is generated by `pip-tools`, then it is likely that - this step is not needed, but for other `requirements.txt` files this may be useful. - - NOTE @aignas 2023-12-08: the extra resolution is not platform dependent, - but in order for it to become platform dependent we would have to have - separate targets for each extra in extras. - """ - - # Resolve any extra extras due to self-edges, empty string means no - # extras The empty string in the set is just a way to make the handling - # of no extras and a single extra easier and having a set of {"", "foo"} - # is equivalent to having {"foo"}. - extras: Set[str] = want_extras or {""} - - self_reqs = [] - for req in reqs: - if Deps._normalize(req.name) != self.name: - continue - - if req.marker is None: - # I am pretty sure we cannot reach this code as it does not - # make sense to specify packages in this way, but since it is - # easy to handle, lets do it. - # - # TODO @aignas 2023-12-08: add a test - extras = extras | req.extras - else: - # process these in a separate loop - self_reqs.append(req) - - # A double loop is not strictly optimal, but always correct without recursion - for req in self_reqs: - if any(req.marker.evaluate({"extra": extra}) for extra in extras): - extras = extras | req.extras - else: - continue - - # Iterate through all packages to ensure that we include all of the extras from previously - # visited packages. - for req_ in self_reqs: - if any(req_.marker.evaluate({"extra": extra}) for extra in extras): - extras = extras | req_.extras - - return extras - - def _add_req(self, req_name, reqs: List[Requirement], extras: Set[str]) -> None: - platforms_to_add = set() - for req in reqs: - if req.marker is None: - self._add(req.name, None) - return - - if not self._platforms: - if any(req.marker.evaluate({"extra": extra}) for extra in extras): - self._add(req.name, None) - return - - for plat in self._platforms: - if plat in platforms_to_add: - # marker evaluation is more expensive than this check - continue - - added = False - for extra in extras: - if added: - break - - if req.marker.evaluate(plat.env_markers(extra)): - platforms_to_add.add(plat) - added = True - break - - if not self._platforms: - return - - if len(platforms_to_add) == len(self._platforms): - # the dep is in all target platforms, let's just add it to the regular - # list - self._add(req_name, None) - return - - for plat in platforms_to_add: - if self._default_minor_version is not None: - self._add(req_name, plat) - - if ( - self._default_minor_version is None - or plat.minor_version == self._default_minor_version - ): - self._add(req_name, Platform(os=plat.os, arch=plat.arch)) - - def build(self) -> FrozenDeps: - return FrozenDeps( - deps=sorted(self._deps), - deps_select={str(p): sorted(deps) for p, deps in self._select.items()}, - ) class Wheel: @@ -254,12 +30,6 @@ def __init__(self, path: Path): def path(self) -> Path: return self._path - @property - def name(self) -> str: - # TODO Also available as installer.sources.WheelSource.distribution - name = str(self.metadata["Name"]) - return canonicalize_name(name) - @property def metadata(self) -> email.message.Message: with installer.sources.WheelFile.open(self.path) as wheel_source: @@ -272,39 +42,6 @@ def version(self) -> str: # TODO Also available as installer.sources.WheelSource.version return str(self.metadata["Version"]) - def entry_points(self) -> Dict[str, Tuple[str, str]]: - """Returns the entrypoints defined in the current wheel - - See https://packaging.python.org/specifications/entry-points/ for more info - - Returns: - Dict[str, Tuple[str, str]]: A mapping of the entry point's name to it's module and attribute - """ - with installer.sources.WheelFile.open(self.path) as wheel_source: - if "entry_points.txt" not in wheel_source.dist_info_filenames: - return dict() - - entry_points_mapping = dict() - entry_points_contents = wheel_source.read_dist_info("entry_points.txt") - entry_points = installer.utils.parse_entrypoints(entry_points_contents) - for script, module, attribute, script_section in entry_points: - if script_section == "console": - entry_points_mapping[script] = (module, attribute) - - return entry_points_mapping - - def dependencies( - self, - extras_requested: Set[str] = None, - platforms: Optional[Set[Platform]] = None, - ) -> FrozenDeps: - return Deps( - self.name, - extras=extras_requested, - platforms=platforms, - requires_dist=self.metadata.get_all("Requires-Dist", []), - ).build() - def unzip(self, directory: str) -> None: installation_schemes = { "purelib": "/site-packages", diff --git a/python/private/pypi/whl_installer/wheel_installer.py b/python/private/pypi/whl_installer/wheel_installer.py index aae24ff4c7..1f00068060 100644 --- a/python/private/pypi/whl_installer/wheel_installer.py +++ b/python/private/pypi/whl_installer/wheel_installer.py @@ -80,8 +80,6 @@ def _parse_requirement_for_extra( def _extract_wheel( wheel_file: str, extras: Dict[str, Set[str]], - enable_pipstar: bool, - platforms: List[wheel.Platform], installation_dir: Path = Path("."), ) -> None: """Extracts wheel into given directory and creates py_library and filegroup targets. @@ -90,36 +88,11 @@ def _extract_wheel( wheel_file: the filepath of the .whl installation_dir: the destination directory for installation of the wheel. extras: a list of extras to add as dependencies for the installed wheel - enable_pipstar: if true, turns off certain operations. """ whl = wheel.Wheel(wheel_file) whl.unzip(installation_dir) - if enable_pipstar: - return - - extras_requested = extras[whl.name] if whl.name in extras else set() - dependencies = whl.dependencies(extras_requested, platforms) - - metadata = { - "name": whl.name, - "version": whl.version, - "deps": dependencies.deps, - "deps_by_platform": dependencies.deps_select, - "entry_points": [ - { - "name": name, - "module": module, - "attribute": attribute, - } - for name, (module, attribute) in sorted(whl.entry_points().items()) - ], - } - - with open(os.path.join(installation_dir, "metadata.json"), "w") as f: - json.dump(metadata, f) - def main() -> None: args = arguments.parser(description=__doc__).parse_args() @@ -136,8 +109,6 @@ def main() -> None: _extract_wheel( wheel_file=whl, extras=extras, - enable_pipstar=args.enable_pipstar, - platforms=arguments.get_platforms(args), ) return diff --git a/python/private/pypi/whl_library.bzl b/python/private/pypi/whl_library.bzl index bd76741fa3..36df4dc82e 100644 --- a/python/private/pypi/whl_library.bzl +++ b/python/private/pypi/whl_library.bzl @@ -22,14 +22,12 @@ load("//python/private:repo_utils.bzl", "REPO_DEBUG_ENV_VAR", "repo_utils") load(":attrs.bzl", "ATTRS", "use_isolated") load(":deps.bzl", "all_repo_names", "record_files") load(":generate_whl_library_build_bazel.bzl", "generate_whl_library_build_bazel") -load(":parse_whl_name.bzl", "parse_whl_name") load(":patch_whl.bzl", "patch_whl") load(":pep508_requirement.bzl", "requirement") load(":pypi_repo_utils.bzl", "pypi_repo_utils") load(":urllib.bzl", "urllib") load(":whl_extract.bzl", "whl_extract") load(":whl_metadata.bzl", "whl_metadata") -load(":whl_target_platforms.bzl", "whl_target_platforms") _CPPFLAGS = "CPPFLAGS" _COMMAND_LINE_TOOLS_PATH_SLUG = "commandlinetools" @@ -263,21 +261,6 @@ def _create_repository_execution_environment(rctx, python_interpreter, logger = return env def _extract_whl_py(rctx, *, python_interpreter, args, whl_path, environment, logger): - target_platforms = rctx.attr.experimental_target_platforms or [] - if target_platforms: - parsed_whl = parse_whl_name(whl_path.basename) - - # NOTE @aignas 2023-12-04: if the wheel is a platform specific wheel, we - # only include deps for that target platform - if parsed_whl.platform_tag != "any": - target_platforms = [ - p.target_platform - for p in whl_target_platforms( - platform_tag = parsed_whl.platform_tag, - abi_tag = parsed_whl.abi_tag.strip("tm"), - ) - ] - pypi_repo_utils.execute_checked( rctx, op = "whl_library.ExtractWheel({}, {})".format(rctx.attr.name, whl_path), @@ -285,7 +268,7 @@ def _extract_whl_py(rctx, *, python_interpreter, args, whl_path, environment, lo arguments = args + [ "--whl-file", whl_path, - ] + ["--platform={}".format(p) for p in target_platforms], + ], srcs = rctx.attr._python_srcs, environment = environment, quiet = rctx.attr.quiet, @@ -344,9 +327,7 @@ def _whl_library_impl(rctx): # build deps from PyPI (e.g. `flit_core`) if they are missing. extra_pip_args.extend(["--find-links", "."]) - # also enable pipstar for any whls that are downloaded without `pip` - enable_pipstar = (rp_config.enable_pipstar or whl_path) and rctx.attr.config_load - enable_pipstar_extract = enable_pipstar and rp_config.bazel_8_or_later + enable_pipstar_extract = rp_config.bazel_8_or_later # When pipstar is enabled, Python isn't used, so there's no need # to setup env vars to run Python, unless we need to build an sdist @@ -428,65 +409,34 @@ def _whl_library_impl(rctx): logger = logger, ) - # NOTE @aignas 2025-09-28: if someone has an old vendored file that does not have the - # dep_template set or the packages is not set either, we should still not break, best to - # disable pipstar for that particular case. - # - # Remove non-pipstar and config_load check when we release rules_python 2. - if enable_pipstar: - install_dir_path = whl_path.dirname.get_child("site-packages") - metadata = whl_metadata( - install_dir = install_dir_path, - read_fn = rctx.read, - logger = logger, - ) - namespace_package_files = pypi_repo_utils.find_namespace_package_files(rctx, install_dir_path) - - build_file_contents = generate_whl_library_build_bazel( - name = whl_path.basename, - sdist_filename = sdist_filename, - dep_template = rctx.attr.dep_template or "@{}{{name}}//:{{target}}".format( - rctx.attr.repo_prefix, - ), - config_load = rctx.attr.config_load, - metadata_name = metadata.name, - metadata_version = metadata.version, - requires_dist = metadata.requires_dist, - # TODO @aignas 2025-05-17: maybe have a build flag for this instead - enable_implicit_namespace_pkgs = rctx.attr.enable_implicit_namespace_pkgs, - # TODO @aignas 2025-04-14: load through the hub: - annotation = None if not rctx.attr.annotation else struct(**json.decode(rctx.read(rctx.attr.annotation))), - data_exclude = rctx.attr.pip_data_exclude, - group_deps = rctx.attr.group_deps, - group_name = rctx.attr.group_name, - namespace_package_files = namespace_package_files, - extras = requirement(rctx.attr.requirement).extras, - ) - else: - metadata = json.decode(rctx.read("metadata.json")) - rctx.delete("metadata.json") - - namespace_package_files = pypi_repo_utils.find_namespace_package_files(rctx, rctx.path("site-packages")) - - build_file_contents = generate_whl_library_build_bazel( - name = whl_path.basename, - sdist_filename = sdist_filename, - dep_template = rctx.attr.dep_template or "@{}{{name}}//:{{target}}".format(rctx.attr.repo_prefix), - # TODO @aignas 2025-05-17: maybe have a build flag for this instead - enable_implicit_namespace_pkgs = rctx.attr.enable_implicit_namespace_pkgs, - # TODO @aignas 2025-04-14: load through the hub: - dependencies = metadata["deps"], - dependencies_by_platform = metadata["deps_by_platform"], - annotation = None if not rctx.attr.annotation else struct(**json.decode(rctx.read(rctx.attr.annotation))), - data_exclude = rctx.attr.pip_data_exclude, - group_deps = rctx.attr.group_deps, - group_name = rctx.attr.group_name, - tags = [ - "pypi_name={}".format(metadata["name"]), - "pypi_version={}".format(metadata["version"]), - ], - namespace_package_files = namespace_package_files, - ) + install_dir_path = whl_path.dirname.get_child("site-packages") + metadata = whl_metadata( + install_dir = install_dir_path, + read_fn = rctx.read, + logger = logger, + ) + namespace_package_files = pypi_repo_utils.find_namespace_package_files(rctx, install_dir_path) + + build_file_contents = generate_whl_library_build_bazel( + name = whl_path.basename, + sdist_filename = sdist_filename, + dep_template = rctx.attr.dep_template or "@{}{{name}}//:{{target}}".format( + rctx.attr.repo_prefix, + ), + config_load = rctx.attr.config_load, + metadata_name = metadata.name, + metadata_version = metadata.version, + requires_dist = metadata.requires_dist, + # TODO @aignas 2025-05-17: maybe have a build flag for this instead + enable_implicit_namespace_pkgs = rctx.attr.enable_implicit_namespace_pkgs, + # TODO @aignas 2025-04-14: load through the hub: + annotation = None if not rctx.attr.annotation else struct(**json.decode(rctx.read(rctx.attr.annotation))), + data_exclude = rctx.attr.pip_data_exclude, + group_deps = rctx.attr.group_deps, + group_name = rctx.attr.group_name, + namespace_package_files = namespace_package_files, + extras = requirement(rctx.attr.requirement).extras, + ) # Delete these in case the wheel had them. They generally don't cause # a problem, but let's avoid the chance of that happening. @@ -499,7 +449,7 @@ def _whl_library_impl(rctx): _remove_files(rctx, "BUILD", "BUILD.bazel") rctx.file("BUILD.bazel", build_file_contents) - if enable_pipstar and enable_pipstar_extract: + if enable_pipstar_extract: if hasattr(rctx, "repo_metadata"): return rctx.repo_metadata(reproducible = True) @@ -618,7 +568,6 @@ way to define whl_library and move whl patching to a separate place. INTERNAL US "_python_srcs": attr.label_list( # Used as a default value in a rule to ensure we fetch the dependencies. default = [ - Label("//python/private/pypi/whl_installer:platform.py"), Label("//python/private/pypi/whl_installer:wheel.py"), Label("//python/private/pypi/whl_installer:wheel_installer.py"), Label("//python/private/pypi/whl_installer:arguments.py"), diff --git a/tests/pypi/extension/extension_tests.bzl b/tests/pypi/extension/extension_tests.bzl index 452703d9c9..ca6da9ac1b 100644 --- a/tests/pypi/extension/extension_tests.bzl +++ b/tests/pypi/extension/extension_tests.bzl @@ -67,10 +67,9 @@ def _mod(*, name, default = [], parse = [], override = [], whl_mods = [], is_roo ], ) -def _parse_modules(env, enable_pipstar = 0, **kwargs): +def _parse_modules(env, **kwargs): return env.expect.that_struct( parse_modules( - enable_pipstar = enable_pipstar, **kwargs ), attrs = dict( @@ -82,16 +81,11 @@ def _parse_modules(env, enable_pipstar = 0, **kwargs): ), ) -def _build_config(env, enable_pipstar = 0, **kwargs): +def _build_config(env, enable_pipstar_extract = True, **kwargs): return env.expect.that_struct( - build_config( - enable_pipstar = enable_pipstar, - enable_pipstar_extract = True, - **kwargs - ), + build_config(enable_pipstar_extract = enable_pipstar_extract, **kwargs), attrs = dict( auth_patterns = subjects.dict, - enable_pipstar = subjects.bool, netrc = subjects.str, platforms = subjects.dict, ), @@ -205,11 +199,9 @@ def _test_build_pipstar_platform(env): ], ), ), - enable_pipstar = True, ) config.auth_patterns().contains_exactly({"foo": "bar"}) config.netrc().equals("my_netrc") - config.enable_pipstar().equals(True) config.platforms().contains_exactly({ "myplat": struct( name = "myplat", diff --git a/tests/pypi/extension/pip_parse.bzl b/tests/pypi/extension/pip_parse.bzl index d6080e52d6..2d55d5cd1f 100644 --- a/tests/pypi/extension/pip_parse.bzl +++ b/tests/pypi/extension/pip_parse.bzl @@ -13,7 +13,6 @@ def pip_parse( experimental_extra_index_urls = [], experimental_index_url = "", experimental_requirement_cycles = {}, - experimental_target_platforms = [], extra_hub_aliases = {}, extra_pip_args = [], isolated = True, @@ -44,8 +43,6 @@ def pip_parse( experimental_extra_index_urls = experimental_extra_index_urls, experimental_index_url = experimental_index_url, experimental_requirement_cycles = experimental_requirement_cycles, - # TODO @aignas 2025-12-02: decide on a single attr - should we reuse this? - experimental_target_platforms = experimental_target_platforms, target_platforms = target_platforms, extra_hub_aliases = extra_hub_aliases, extra_pip_args = extra_pip_args, diff --git a/tests/pypi/hub_builder/hub_builder_tests.bzl b/tests/pypi/hub_builder/hub_builder_tests.bzl index 29021dfd85..ccf72c2774 100644 --- a/tests/pypi/hub_builder/hub_builder_tests.bzl +++ b/tests/pypi/hub_builder/hub_builder_tests.bzl @@ -42,7 +42,6 @@ simple==0.0.1 \ def hub_builder( env, - enable_pipstar = True, enable_pipstar_extract = True, debug = False, config = None, @@ -57,7 +56,6 @@ def hub_builder( module_name = "unit_test", config = config or struct( # no need to evaluate the markers with the interpreter - enable_pipstar = enable_pipstar, enable_pipstar_extract = enable_pipstar_extract, index_url = "https://pypi.org/simple", platforms = { @@ -539,7 +537,6 @@ def _test_torch_experimental_index_url(env): env, config = struct( netrc = None, - enable_pipstar = True, enable_pipstar_extract = True, index_url = "https://pypi.org/simple", auth_patterns = {}, @@ -1287,9 +1284,7 @@ _tests.append(_test_optimum_sys_platform_extra) def _test_pipstar_platforms(env): builder = hub_builder( env, - enable_pipstar = True, config = struct( - enable_pipstar = True, enable_pipstar_extract = True, index_url = "https://pypi.org/simple", netrc = None, @@ -1373,9 +1368,7 @@ _tests.append(_test_pipstar_platforms) def _test_pipstar_platforms_limit(env): builder = hub_builder( env, - enable_pipstar = True, config = struct( - enable_pipstar = True, enable_pipstar_extract = True, index_url = "https://pypi.org/simple", netrc = None, diff --git a/tests/pypi/whl_installer/BUILD.bazel b/tests/pypi/whl_installer/BUILD.bazel index 060d2bce62..0a859c0e4d 100644 --- a/tests/pypi/whl_installer/BUILD.bazel +++ b/tests/pypi/whl_installer/BUILD.bazel @@ -16,18 +16,6 @@ py_test( ], ) -py_test( - name = "platform_test", - size = "small", - srcs = [ - "platform_test.py", - ], - data = ["//examples/wheel:minimal_with_py_package"], - deps = [ - ":lib", - ], -) - py_test( name = "wheel_installer_test", size = "small", @@ -39,15 +27,3 @@ py_test( ":lib", ], ) - -py_test( - name = "wheel_test", - size = "small", - srcs = [ - "wheel_test.py", - ], - data = ["//examples/wheel:minimal_with_py_package"], - deps = [ - ":lib", - ], -) diff --git a/tests/pypi/whl_installer/arguments_test.py b/tests/pypi/whl_installer/arguments_test.py index 2352d8e48b..9e26849db0 100644 --- a/tests/pypi/whl_installer/arguments_test.py +++ b/tests/pypi/whl_installer/arguments_test.py @@ -48,18 +48,6 @@ def test_deserialize_structured_args(self) -> None: self.assertEqual(args["environment"], {"PIP_DO_SOMETHING": "True"}) self.assertEqual(args["extra_pip_args"], []) - def test_platform_aggregation(self) -> None: - parser = arguments.parser() - args = parser.parse_args( - args=[ - "--platform=linux_*", - "--platform=osx_*", - "--platform=windows_*", - "--requirement=foo", - ] - ) - self.assertEqual(set(wheel.Platform.all()), arguments.get_platforms(args)) - if __name__ == "__main__": unittest.main() diff --git a/tests/pypi/whl_installer/platform_test.py b/tests/pypi/whl_installer/platform_test.py deleted file mode 100644 index 0d944bb196..0000000000 --- a/tests/pypi/whl_installer/platform_test.py +++ /dev/null @@ -1,97 +0,0 @@ -import unittest -from random import shuffle - -from python.private.pypi.whl_installer.platform import ( - OS, - Arch, - Platform, - host_interpreter_version, -) - - -class MinorVersionTest(unittest.TestCase): - def test_host(self): - host = host_interpreter_version() - self.assertIsNotNone(host) - - -class PlatformTest(unittest.TestCase): - def test_can_get_host(self): - host = Platform.host() - self.assertIsNotNone(host) - self.assertEqual(1, len(Platform.from_string("host"))) - self.assertEqual(host, Platform.from_string("host")) - - def test_can_get_linux_x86_64_without_py_version(self): - got = Platform.from_string("linux_x86_64") - want = Platform(os=OS.linux, arch=Arch.x86_64) - self.assertEqual(want, got[0]) - - def test_can_get_specific_from_string(self): - got = Platform.from_string("cp33_linux_x86_64") - want = Platform(os=OS.linux, arch=Arch.x86_64, minor_version=3) - self.assertEqual(want, got[0]) - - got = Platform.from_string("cp33.0_linux_x86_64") - want = Platform(os=OS.linux, arch=Arch.x86_64, minor_version=3, micro_version=0) - self.assertEqual(want, got[0]) - - def test_can_get_all_for_py_version(self): - cp39 = Platform.all(minor_version=9, micro_version=0) - self.assertEqual(24, len(cp39), f"Got {cp39}") - self.assertEqual(cp39, Platform.from_string("cp39.0_*")) - - def test_can_get_all_for_os(self): - linuxes = Platform.all(OS.linux, minor_version=9) - self.assertEqual(8, len(linuxes)) - self.assertEqual(linuxes, Platform.from_string("cp39_linux_*")) - - def test_can_get_all_for_os_for_host_python(self): - linuxes = Platform.all(OS.linux) - self.assertEqual(8, len(linuxes)) - self.assertEqual(linuxes, Platform.from_string("linux_*")) - - def test_platform_sort(self): - platforms = [ - Platform(os=OS.linux, arch=None), - Platform(os=OS.linux, arch=Arch.x86_64), - Platform(os=OS.osx, arch=None), - Platform(os=OS.osx, arch=Arch.x86_64), - Platform(os=OS.osx, arch=Arch.aarch64), - ] - shuffle(platforms) - platforms.sort() - want = [ - Platform(os=OS.linux, arch=None), - Platform(os=OS.linux, arch=Arch.x86_64), - Platform(os=OS.osx, arch=None), - Platform(os=OS.osx, arch=Arch.x86_64), - Platform(os=OS.osx, arch=Arch.aarch64), - ] - - self.assertEqual(want, platforms) - - def test_wheel_os_alias(self): - self.assertEqual("osx", str(OS.osx)) - self.assertEqual(str(OS.darwin), str(OS.osx)) - - def test_wheel_arch_alias(self): - self.assertEqual("x86_64", str(Arch.x86_64)) - self.assertEqual(str(Arch.amd64), str(Arch.x86_64)) - - def test_wheel_platform_alias(self): - give = Platform( - os=OS.darwin, - arch=Arch.amd64, - ) - alias = Platform( - os=OS.osx, - arch=Arch.x86_64, - ) - - self.assertEqual("osx_x86_64", str(give)) - self.assertEqual(str(alias), str(give)) - - -if __name__ == "__main__": - unittest.main() diff --git a/tests/pypi/whl_installer/wheel_installer_test.py b/tests/pypi/whl_installer/wheel_installer_test.py index 91adddf15a..52c44cf2de 100644 --- a/tests/pypi/whl_installer/wheel_installer_test.py +++ b/tests/pypi/whl_installer/wheel_installer_test.py @@ -72,12 +72,9 @@ def test_wheel_exists(self) -> None: Path(self.wheel_path), installation_dir=Path(self.wheel_dir), extras={}, - platforms=[], - enable_pipstar=False, ) want_files = [ - "metadata.json", "site-packages", self.wheel_name, ] @@ -90,17 +87,6 @@ def test_wheel_exists(self) -> None: ] ), ) - with open("{}/metadata.json".format(self.wheel_dir)) as metadata_file: - metadata_file_content = json.load(metadata_file) - - want = dict( - deps=[], - deps_by_platform={}, - entry_points=[], - name="example-minimal-package", - version="0.0.1", - ) - self.assertEqual(want, metadata_file_content) if __name__ == "__main__": diff --git a/tests/pypi/whl_installer/wheel_test.py b/tests/pypi/whl_installer/wheel_test.py deleted file mode 100644 index 3599fd1868..0000000000 --- a/tests/pypi/whl_installer/wheel_test.py +++ /dev/null @@ -1,345 +0,0 @@ -import unittest -from unittest import mock - -from python.private.pypi.whl_installer import wheel -from python.private.pypi.whl_installer.platform import OS, Arch, Platform - -_HOST_INTERPRETER_FN = ( - "python.private.pypi.whl_installer.wheel.host_interpreter_version" -) - - -class DepsTest(unittest.TestCase): - def test_simple(self): - deps = wheel.Deps("foo", requires_dist=["bar", 'baz; extra=="foo"']) - - got = deps.build() - - self.assertIsInstance(got, wheel.FrozenDeps) - self.assertEqual(["bar"], got.deps) - self.assertEqual({}, got.deps_select) - - def test_can_add_os_specific_deps(self): - for platforms in [ - { - Platform(os=OS.linux, arch=Arch.x86_64), - Platform(os=OS.osx, arch=Arch.x86_64), - Platform(os=OS.osx, arch=Arch.aarch64), - Platform(os=OS.windows, arch=Arch.x86_64), - }, - { - Platform(os=OS.linux, arch=Arch.x86_64, minor_version=8), - Platform(os=OS.osx, arch=Arch.x86_64, minor_version=8), - Platform(os=OS.osx, arch=Arch.aarch64, minor_version=8), - Platform(os=OS.windows, arch=Arch.x86_64, minor_version=8), - }, - { - Platform( - os=OS.linux, arch=Arch.x86_64, minor_version=8, micro_version=1 - ), - Platform(os=OS.osx, arch=Arch.x86_64, minor_version=8, micro_version=1), - Platform( - os=OS.osx, arch=Arch.aarch64, minor_version=8, micro_version=1 - ), - Platform( - os=OS.windows, arch=Arch.x86_64, minor_version=8, micro_version=1 - ), - }, - ]: - with self.subTest(): - deps = wheel.Deps( - "foo", - requires_dist=[ - "bar", - "an_osx_dep; sys_platform=='darwin'", - "posix_dep; os_name=='posix'", - "win_dep; os_name=='nt'", - ], - platforms=platforms, - ) - - got = deps.build() - - self.assertEqual(["bar"], got.deps) - self.assertEqual( - { - "linux_x86_64": ["posix_dep"], - "osx_aarch64": ["an_osx_dep", "posix_dep"], - "osx_x86_64": ["an_osx_dep", "posix_dep"], - "windows_x86_64": ["win_dep"], - }, - got.deps_select, - ) - - def test_non_platform_markers_are_added_to_common_deps(self): - got = wheel.Deps( - "foo", - requires_dist=[ - "bar", - "baz; implementation_name=='cpython'", - "m1_dep; sys_platform=='darwin' and platform_machine=='arm64'", - ], - platforms={ - Platform(os=OS.linux, arch=Arch.x86_64), - Platform(os=OS.osx, arch=Arch.x86_64), - Platform(os=OS.osx, arch=Arch.aarch64), - Platform(os=OS.windows, arch=Arch.x86_64), - }, - ).build() - - self.assertEqual(["bar", "baz"], got.deps) - self.assertEqual( - { - "osx_aarch64": ["m1_dep"], - }, - got.deps_select, - ) - - def test_self_is_ignored(self): - deps = wheel.Deps( - "foo", - requires_dist=[ - "bar", - "req_dep; extra == 'requests'", - "foo[requests]; extra == 'ssl'", - "ssl_lib; extra == 'ssl'", - ], - extras={"ssl"}, - ) - - got = deps.build() - - self.assertEqual(["bar", "req_dep", "ssl_lib"], got.deps) - self.assertEqual({}, got.deps_select) - - def test_self_dependencies_can_come_in_any_order(self): - deps = wheel.Deps( - "foo", - requires_dist=[ - "bar", - "baz; extra == 'feat'", - "foo[feat2]; extra == 'all'", - "foo[feat]; extra == 'feat2'", - "zdep; extra == 'all'", - ], - extras={"all"}, - ) - - got = deps.build() - - self.assertEqual(["bar", "baz", "zdep"], got.deps) - self.assertEqual({}, got.deps_select) - - def test_can_get_deps_based_on_specific_python_version(self): - requires_dist = [ - "bar", - "baz; python_full_version < '3.7.3'", - "posix_dep; os_name=='posix' and python_version >= '3.8'", - ] - - py38_deps = wheel.Deps( - "foo", - requires_dist=requires_dist, - platforms=[ - Platform(os=OS.linux, arch=Arch.x86_64, minor_version=8), - ], - ).build() - py373_deps = wheel.Deps( - "foo", - requires_dist=requires_dist, - platforms=[ - Platform( - os=OS.linux, arch=Arch.x86_64, minor_version=7, micro_version=3 - ), - ], - ).build() - py37_deps = wheel.Deps( - "foo", - requires_dist=requires_dist, - platforms=[ - Platform(os=OS.linux, arch=Arch.x86_64, minor_version=7), - ], - ).build() - - self.assertEqual(["bar", "baz"], py37_deps.deps) - self.assertEqual({}, py37_deps.deps_select) - self.assertEqual(["bar"], py373_deps.deps) - self.assertEqual({}, py37_deps.deps_select) - self.assertEqual(["bar", "posix_dep"], py38_deps.deps) - self.assertEqual({}, py38_deps.deps_select) - - def test_no_version_select_when_single_version(self): - requires_dist = [ - "bar", - "baz; python_version >= '3.8'", - "posix_dep; os_name=='posix'", - "posix_dep_with_version; os_name=='posix' and python_version >= '3.8'", - "arch_dep; platform_machine=='x86_64' and python_version >= '3.8'", - ] - - self.maxDiff = None - - deps = wheel.Deps( - "foo", - requires_dist=requires_dist, - platforms=[ - Platform( - os=os, arch=Arch.x86_64, minor_version=minor, micro_version=micro - ) - for minor, micro in [(8, 4)] - for os in [OS.linux, OS.windows] - ], - ) - got = deps.build() - - self.assertEqual(["arch_dep", "bar", "baz"], got.deps) - self.assertEqual( - { - "linux_x86_64": ["posix_dep", "posix_dep_with_version"], - }, - got.deps_select, - ) - - @mock.patch(_HOST_INTERPRETER_FN) - def test_can_get_version_select(self, mock_host_interpreter_version): - requires_dist = [ - "bar", - "baz; python_version < '3.8'", - "baz_new; python_version >= '3.8'", - "posix_dep; os_name=='posix'", - "posix_dep_with_version; os_name=='posix' and python_version >= '3.8'", - "arch_dep; platform_machine=='x86_64' and python_version < '3.8'", - ] - mock_host_interpreter_version.return_value = (7, 4) - - self.maxDiff = None - - deps = wheel.Deps( - "foo", - requires_dist=requires_dist, - platforms=[ - Platform( - os=os, arch=Arch.x86_64, minor_version=minor, micro_version=micro - ) - for minor, micro in [(7, 4), (8, 8), (9, 8)] - for os in [OS.linux, OS.windows] - ], - ) - got = deps.build() - - self.assertEqual(["bar"], got.deps) - self.assertEqual( - { - "cp37.4_linux_x86_64": ["arch_dep", "baz", "posix_dep"], - "cp37.4_windows_x86_64": ["arch_dep", "baz"], - "cp38.8_linux_x86_64": [ - "baz_new", - "posix_dep", - "posix_dep_with_version", - ], - "cp38.8_windows_x86_64": ["baz_new"], - "cp39.8_linux_x86_64": [ - "baz_new", - "posix_dep", - "posix_dep_with_version", - ], - "cp39.8_windows_x86_64": ["baz_new"], - "linux_x86_64": ["arch_dep", "baz", "posix_dep"], - "windows_x86_64": ["arch_dep", "baz"], - }, - got.deps_select, - ) - - @mock.patch(_HOST_INTERPRETER_FN) - def test_deps_spanning_all_target_py_versions_are_added_to_common( - self, mock_host_version - ): - requires_dist = [ - "bar", - "baz (<2,>=1.11) ; python_version < '3.8'", - "baz (<2,>=1.14) ; python_version >= '3.8'", - ] - mock_host_version.return_value = (8, 4) - - self.maxDiff = None - - deps = wheel.Deps( - "foo", - requires_dist=requires_dist, - platforms=Platform.from_string(["cp37_*", "cp38_*", "cp39_*"]), - ) - got = deps.build() - - self.assertEqual({}, got.deps_select) - self.assertEqual(["bar", "baz"], got.deps) - - @mock.patch(_HOST_INTERPRETER_FN) - def test_deps_are_not_duplicated(self, mock_host_version): - mock_host_version.return_value = (7, 4) - - # See an example in - # https://files.pythonhosted.org/packages/76/9e/db1c2d56c04b97981c06663384f45f28950a73d9acf840c4006d60d0a1ff/opencv_python-4.9.0.80-cp37-abi3-win32.whl.metadata - requires_dist = [ - "bar >=0.1.0 ; python_version < '3.7'", - "bar >=0.2.0 ; python_version >= '3.7'", - "bar >=0.4.0 ; python_version >= '3.6' and platform_system == 'Linux' and platform_machine == 'aarch64'", - "bar >=0.4.0 ; python_version >= '3.9'", - "bar >=0.5.0 ; python_version <= '3.9' and platform_system == 'Darwin' and platform_machine == 'arm64'", - "bar >=0.5.0 ; python_version >= '3.10' and platform_system == 'Darwin'", - "bar >=0.5.0 ; python_version >= '3.10'", - "bar >=0.6.0 ; python_version >= '3.11'", - ] - - deps = wheel.Deps( - "foo", - requires_dist=requires_dist, - platforms=Platform.from_string(["cp37_*", "cp310_*"]), - ) - got = deps.build() - - self.assertEqual(["bar"], got.deps) - self.assertEqual({}, got.deps_select) - - @mock.patch(_HOST_INTERPRETER_FN) - def test_deps_are_not_duplicated_when_encountering_platform_dep_first( - self, mock_host_version - ): - mock_host_version.return_value = (7, 1) - - # Note, that we are sorting the incoming `requires_dist` and we need to ensure that we are not getting any - # issues even if the platform-specific line comes first. - requires_dist = [ - "bar >=0.4.0 ; python_version >= '3.6' and platform_system == 'Linux' and platform_machine == 'aarch64'", - "bar >=0.5.0 ; python_version >= '3.9'", - ] - - self.maxDiff = None - - deps = wheel.Deps( - "foo", - requires_dist=requires_dist, - platforms=Platform.from_string( - [ - "cp37.1_linux_x86_64", - "cp37.1_linux_aarch64", - "cp310_linux_x86_64", - "cp310_linux_aarch64", - ] - ), - ) - got = deps.build() - - self.assertEqual([], got.deps) - self.assertEqual( - { - "cp310_linux_aarch64": ["bar"], - "cp310_linux_x86_64": ["bar"], - "cp37.1_linux_aarch64": ["bar"], - "linux_aarch64": ["bar"], - }, - got.deps_select, - ) - - -if __name__ == "__main__": - unittest.main() From 098cb295dcab45aef3da4cb8f6721aa1b71fca7b Mon Sep 17 00:00:00 2001 From: Will Noble <4642422+ouillie@users.noreply.github.com> Date: Mon, 27 Apr 2026 07:50:30 -0700 Subject: [PATCH 723/922] feat(pypi): support isolated `pip.parse` (#3669) Adds a fallback at the end of `build_config`: when `defaults["platforms"]` is empty, it detects the host OS and CPU and injects a single platform entry. This allows users to use the `--experimental_isolated_extension_usages` flag. Fixes #3668 Related #2094 --------- Co-authored-by: Ignas Anikevicius <240938+aignas@users.noreply.github.com> --- .bazelrc.deleted_packages | 1 + CHANGELOG.md | 2 + MODULE.bazel | 135 -------------- python/private/pypi/extension.bzl | 124 ++++++++++++- tests/integration/BUILD.bazel | 4 + tests/integration/pip_parse_isolated/.bazelrc | 8 + .../pip_parse_isolated/BUILD.bazel | 7 + .../pip_parse_isolated/MODULE.bazel | 19 ++ .../integration/pip_parse_isolated/WORKSPACE | 0 .../pip_parse_isolated/requirements_lock.txt | 2 + .../pip_parse_isolated/test_isolated.py | 16 ++ tests/pypi/extension/extension_tests.bzl | 171 ++++++++++++------ 12 files changed, 299 insertions(+), 190 deletions(-) create mode 100644 tests/integration/pip_parse_isolated/.bazelrc create mode 100644 tests/integration/pip_parse_isolated/BUILD.bazel create mode 100644 tests/integration/pip_parse_isolated/MODULE.bazel create mode 100644 tests/integration/pip_parse_isolated/WORKSPACE create mode 100644 tests/integration/pip_parse_isolated/requirements_lock.txt create mode 100644 tests/integration/pip_parse_isolated/test_isolated.py diff --git a/.bazelrc.deleted_packages b/.bazelrc.deleted_packages index 61c79ae032..71a81de097 100644 --- a/.bazelrc.deleted_packages +++ b/.bazelrc.deleted_packages @@ -35,6 +35,7 @@ common --deleted_packages=tests/integration/custom_commands common --deleted_packages=tests/integration/local_toolchains common --deleted_packages=tests/integration/pip_parse common --deleted_packages=tests/integration/pip_parse/empty +common --deleted_packages=tests/integration/pip_parse_isolated common --deleted_packages=tests/integration/py_cc_toolchain_registered common --deleted_packages=tests/integration/toolchain_target_settings common --deleted_packages=tests/modules/another_module diff --git a/CHANGELOG.md b/CHANGELOG.md index 7c19d1176e..2fbe0e8960 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -69,6 +69,8 @@ END_UNRELEASED_TEMPLATE targets ([#3729](https://github.com/bazel-contrib/rules_python/issues/3729)). * (entry_point) From now on `mypy` type checking will be skipped on the generated files ([#3126](https://github.com/bazel-contrib/rules_python/issues/3126)). +* (pypi) Support `--experimental_isolated_extension_usages` + ([#3668](https://github.com/bazel-contrib/rules_python/issues/3668)). {#v0-0-0-added} ### Added diff --git a/MODULE.bazel b/MODULE.bazel index b5f67c204e..bb3a9dcab5 100644 --- a/MODULE.bazel +++ b/MODULE.bazel @@ -64,141 +64,6 @@ register_toolchains("@pythons_hub//:all") # Install twine for our own runfiles wheel publishing and allow bzlmod users to use it. pip = use_extension("//python/extensions:pip.bzl", "pip") - -# NOTE @aignas 2025-07-06: we define these platforms to keep backwards compatibility. Whilst we -# stabilize the API this list may be updated with a mention in the CHANGELOG. -[ - pip.default( - arch_name = cpu, - config_settings = [ - "@platforms//cpu:{}".format(cpu), - "@platforms//os:linux", - "//python/config_settings:_is_py_freethreaded_{}".format( - "yes" if freethreaded else "no", - ), - ], - env = {"platform_version": "0"}, - marker = "python_version >= '3.13'" if freethreaded else "", - os_name = "linux", - platform = "linux_{}{}".format(cpu, freethreaded), - whl_abi_tags = ["cp{major}{minor}t"] if freethreaded else [ - "abi3", - "cp{major}{minor}", - ], - whl_platform_tags = [ - "linux_{}".format(cpu), - "manylinux_*_{}".format(cpu), - ], - ) - for cpu in [ - "x86_64", - "aarch64", - ] - for freethreaded in [ - "", - "_freethreaded", - ] -] - -[ - pip.default( - arch_name = cpu, - config_settings = [ - "@platforms//cpu:{}".format(cpu), - "@platforms//os:osx", - "//python/config_settings:_is_py_freethreaded_{}".format( - "yes" if freethreaded else "no", - ), - ], - # We choose the oldest non-EOL version at the time when we release `rules_python`. - # See https://endoflife.date/macos - env = {"platform_version": "14.0"}, - marker = "python_version >= '3.13'" if freethreaded else "", - os_name = "osx", - platform = "osx_{}{}".format(cpu, freethreaded), - whl_abi_tags = ["cp{major}{minor}t"] if freethreaded else [ - "abi3", - "cp{major}{minor}", - ], - whl_platform_tags = [ - "macosx_*_{}".format(suffix) - for suffix in platform_tag_cpus - ], - ) - for cpu, platform_tag_cpus in { - "aarch64": [ - "universal2", - "arm64", - ], - "x86_64": [ - "universal2", - "x86_64", - ], - }.items() - for freethreaded in [ - "", - "_freethreaded", - ] -] - -[ - pip.default( - arch_name = cpu, - config_settings = [ - "@platforms//cpu:{}".format(cpu), - "@platforms//os:windows", - "//python/config_settings:_is_py_freethreaded_{}".format( - "yes" if freethreaded else "no", - ), - ], - env = {"platform_version": "0"}, - marker = "python_version >= '3.13'" if freethreaded else "", - os_name = "windows", - platform = "windows_{}{}".format(cpu, freethreaded), - whl_abi_tags = ["cp{major}{minor}t"] if freethreaded else [ - "abi3", - "cp{major}{minor}", - ], - whl_platform_tags = whl_platform_tags, - ) - for cpu, whl_platform_tags in { - "x86_64": ["win_amd64"], - }.items() - for freethreaded in [ - "", - "_freethreaded", - ] -] - -[ - pip.default( - arch_name = cpu, - config_settings = [ - "@platforms//cpu:{}".format(cpu), - "@platforms//os:windows", - "//python/config_settings:_is_py_freethreaded_{}".format( - "yes" if freethreaded else "no", - ), - ], - env = {"platform_version": "0"}, - marker = "python_version >= '3.13'" if freethreaded else "python_version >= '3.11'", - os_name = "windows", - platform = "windows_{}{}".format(cpu, freethreaded), - whl_abi_tags = ["cp{major}{minor}t"] if freethreaded else [ - "abi3", - "cp{major}{minor}", - ], - whl_platform_tags = whl_platform_tags, - ) - for cpu, whl_platform_tags in { - "aarch64": ["win_arm64"], - }.items() - for freethreaded in [ - "", - "_freethreaded", - ] -] - pip.parse( hub_name = "rules_python_publish_deps", python_version = "3.11", diff --git a/python/private/pypi/extension.bzl b/python/private/pypi/extension.bzl index 78e93b7edd..e6052782fa 100644 --- a/python/private/pypi/extension.bzl +++ b/python/private/pypi/extension.bzl @@ -55,6 +55,128 @@ def _whl_mods_impl(whl_mods_dict): whl_mods = whl_mods, ) +def default_platforms(): + """Return the built-in default platform definitions. + + These provide the platform metadata needed for pip wheel resolution + (whl_abi_tags, whl_platform_tags, config_settings, etc.) across all + common OS/arch combinations. They are always used as the starting point + for build_config; root modules can override individual platforms via + pip.default tags. + + Returns: + A dict of platform name to platform config dicts. + """ + # NOTE @aignas 2025-07-06: we define these platforms to keep backwards compatibility. Whilst we + # stabilize the API this list may be updated with a mention in the CHANGELOG. + + platforms = {} + + # Linux platforms + for cpu in ["x86_64", "aarch64"]: + for freethreaded in ["", "_freethreaded"]: + platform_name = "linux_{}{}".format(cpu, freethreaded) + platforms[platform_name] = { + "arch_name": cpu, + "config_settings": [ + "@platforms//cpu:{}".format(cpu), + "@platforms//os:linux", + "//python/config_settings:_is_py_freethreaded_{}".format( + "yes" if freethreaded else "no", + ), + ], + "env": {"platform_version": "0"}, + "marker": "python_version >= '3.13'" if freethreaded else "", + "name": platform_name, + "os_name": "linux", + "whl_abi_tags": ["cp{major}{minor}t"] if freethreaded else [ + "abi3", + "cp{major}{minor}", + ], + "whl_platform_tags": [ + "linux_{}".format(cpu), + "manylinux_*_{}".format(cpu), + ], + } + + # macOS platforms + for cpu, platform_tag_cpus in { + "aarch64": ["universal2", "arm64"], + "x86_64": ["universal2", "x86_64"], + }.items(): + for freethreaded in ["", "_freethreaded"]: + platform_name = "osx_{}{}".format(cpu, freethreaded) + platforms[platform_name] = { + "arch_name": cpu, + "config_settings": [ + "@platforms//cpu:{}".format(cpu), + "@platforms//os:osx", + "//python/config_settings:_is_py_freethreaded_{}".format( + "yes" if freethreaded else "no", + ), + ], + "env": {"platform_version": "14.0"}, + "marker": "python_version >= '3.13'" if freethreaded else "", + "name": platform_name, + "os_name": "osx", + "whl_abi_tags": ["cp{major}{minor}t"] if freethreaded else [ + "abi3", + "cp{major}{minor}", + ], + "whl_platform_tags": [ + "macosx_*_{}".format(suffix) + for suffix in platform_tag_cpus + ], + } + + # Windows x86_64 platforms + for freethreaded in ["", "_freethreaded"]: + platform_name = "windows_x86_64{}".format(freethreaded) + platforms[platform_name] = { + "arch_name": "x86_64", + "config_settings": [ + "@platforms//cpu:x86_64", + "@platforms//os:windows", + "//python/config_settings:_is_py_freethreaded_{}".format( + "yes" if freethreaded else "no", + ), + ], + "env": {"platform_version": "0"}, + "marker": "python_version >= '3.13'" if freethreaded else "", + "name": platform_name, + "os_name": "windows", + "whl_abi_tags": ["cp{major}{minor}t"] if freethreaded else [ + "abi3", + "cp{major}{minor}", + ], + "whl_platform_tags": ["win_amd64"], + } + + # Windows aarch64 platforms + for freethreaded in ["", "_freethreaded"]: + platform_name = "windows_aarch64{}".format(freethreaded) + platforms[platform_name] = { + "arch_name": "aarch64", + "config_settings": [ + "@platforms//cpu:aarch64", + "@platforms//os:windows", + "//python/config_settings:_is_py_freethreaded_{}".format( + "yes" if freethreaded else "no", + ), + ], + "env": {"platform_version": "0"}, + "marker": "python_version >= '3.13'" if freethreaded else "python_version >= '3.11'", + "name": platform_name, + "os_name": "windows", + "whl_abi_tags": ["cp{major}{minor}t"] if freethreaded else [ + "abi3", + "cp{major}{minor}", + ], + "whl_platform_tags": ["win_arm64"], + } + + return platforms + def _configure(config, *, override = False, **kwargs): """Set the value in the config if the value is provided""" env = kwargs.get("env") @@ -82,7 +204,7 @@ def build_config( A struct with the configuration. """ defaults = { - "platforms": {}, + "platforms": default_platforms(), } for mod in module_ctx.modules: if not (mod.is_root or mod.name == "rules_python"): diff --git a/tests/integration/BUILD.bazel b/tests/integration/BUILD.bazel index 33ef907af8..5f2d20c103 100644 --- a/tests/integration/BUILD.bazel +++ b/tests/integration/BUILD.bazel @@ -67,6 +67,10 @@ rules_python_integration_test( name = "pip_parse_test", ) +rules_python_integration_test( + name = "pip_parse_isolated_test", +) + rules_python_integration_test( name = "pip_parse_workspace_test", bzlmod = False, diff --git a/tests/integration/pip_parse_isolated/.bazelrc b/tests/integration/pip_parse_isolated/.bazelrc new file mode 100644 index 0000000000..227ce5a4cd --- /dev/null +++ b/tests/integration/pip_parse_isolated/.bazelrc @@ -0,0 +1,8 @@ +# Bazel configuration flags + +build --enable_runfiles + +common --experimental_isolated_extension_usages + +# https://docs.bazel.build/versions/main/best-practices.html#using-the-bazelrc-file +try-import %workspace%/user.bazelrc diff --git a/tests/integration/pip_parse_isolated/BUILD.bazel b/tests/integration/pip_parse_isolated/BUILD.bazel new file mode 100644 index 0000000000..2f825107f1 --- /dev/null +++ b/tests/integration/pip_parse_isolated/BUILD.bazel @@ -0,0 +1,7 @@ +load("@rules_python//python:py_test.bzl", "py_test") + +py_test( + name = "test_isolated", + srcs = ["test_isolated.py"], + deps = ["@pypi//six"], +) diff --git a/tests/integration/pip_parse_isolated/MODULE.bazel b/tests/integration/pip_parse_isolated/MODULE.bazel new file mode 100644 index 0000000000..6c44257acb --- /dev/null +++ b/tests/integration/pip_parse_isolated/MODULE.bazel @@ -0,0 +1,19 @@ +module(name = "pip_parse_isolated") + +bazel_dep(name = "rules_python") +local_path_override( + module_name = "rules_python", + path = "../../..", +) + +python = use_extension("@rules_python//python/extensions:python.bzl", "python") +python.toolchain(python_version = "3.13") + +# This test module verifies that dependencies can be used with `isolate = True`. +pip = use_extension("@rules_python//python/extensions:pip.bzl", "pip", isolate = True) +pip.parse( + hub_name = "pypi", + python_version = "3.13", + requirements_lock = "//:requirements_lock.txt", +) +use_repo(pip, "pypi") diff --git a/tests/integration/pip_parse_isolated/WORKSPACE b/tests/integration/pip_parse_isolated/WORKSPACE new file mode 100644 index 0000000000..e69de29bb2 diff --git a/tests/integration/pip_parse_isolated/requirements_lock.txt b/tests/integration/pip_parse_isolated/requirements_lock.txt new file mode 100644 index 0000000000..b1445a37ae --- /dev/null +++ b/tests/integration/pip_parse_isolated/requirements_lock.txt @@ -0,0 +1,2 @@ +six==1.17.0 \ + --hash=sha256:4721f391ed90541fddacab5acf947aa0d3dc7d27b2e1e8eda2be8970586c3274 diff --git a/tests/integration/pip_parse_isolated/test_isolated.py b/tests/integration/pip_parse_isolated/test_isolated.py new file mode 100644 index 0000000000..f889f071fb --- /dev/null +++ b/tests/integration/pip_parse_isolated/test_isolated.py @@ -0,0 +1,16 @@ +""" +Verify that a dependency added using an isolated extension can be imported. +See MODULE.bazel. +""" + +import six +import unittest + + +class TestIsolated(unittest.TestCase): + def test_import(self): + self.assertTrue(hasattr(six, "PY3")) + + +if __name__ == "__main__": + unittest.main() diff --git a/tests/pypi/extension/extension_tests.bzl b/tests/pypi/extension/extension_tests.bzl index ca6da9ac1b..5a40714b64 100644 --- a/tests/pypi/extension/extension_tests.bzl +++ b/tests/pypi/extension/extension_tests.bzl @@ -16,7 +16,8 @@ load("@rules_testing//lib:test_suite.bzl", "test_suite") load("@rules_testing//lib:truth.bzl", "subjects") -load("//python/private/pypi:extension.bzl", "build_config", "parse_modules") # buildifier: disable=bzl-visibility +load("//python/private/pypi:extension.bzl", "build_config", "default_platforms", "parse_modules") # buildifier: disable=bzl-visibility +load("//python/private/pypi:platform.bzl", _plat = "platform") # buildifier: disable=bzl-visibility load("//python/private/pypi:whl_config_setting.bzl", "whl_config_setting") # buildifier: disable=bzl-visibility load("//tests/support/mocks:mocks.bzl", "mocks") load(":pip_parse.bzl", _parse = "pip_parse") @@ -38,59 +39,6 @@ simple==0.0.1 \ }, ) -def _mod(*, name, default = [], parse = [], override = [], whl_mods = [], is_root = True): - return mocks.module( - name, - is_root = is_root, - parse = parse, - override = override, - whl_mods = whl_mods, - default = default or [ - _default( - platform = "{}_{}{}".format(os, cpu, freethreaded), - os_name = os, - arch_name = cpu, - config_settings = [ - "@platforms//os:{}".format(os), - "@platforms//cpu:{}".format(cpu), - ], - whl_abi_tags = ["cp{major}{minor}t"] if freethreaded else ["abi3", "cp{major}{minor}"], - whl_platform_tags = whl_platform_tags, - ) - for (os, cpu, freethreaded), whl_platform_tags in { - ("linux", "x86_64", ""): ["linux_x86_64", "manylinux_*_x86_64"], - ("linux", "x86_64", "_freethreaded"): ["linux_x86_64", "manylinux_*_x86_64"], - ("linux", "aarch64", ""): ["linux_aarch64", "manylinux_*_aarch64"], - ("osx", "aarch64", ""): ["macosx_*_arm64"], - ("windows", "aarch64", ""): ["win_arm64"], - }.items() - ], - ) - -def _parse_modules(env, **kwargs): - return env.expect.that_struct( - parse_modules( - **kwargs - ), - attrs = dict( - exposed_packages = subjects.dict, - hub_group_map = subjects.dict, - hub_whl_map = subjects.dict, - whl_libraries = subjects.dict, - whl_mods = subjects.dict, - ), - ) - -def _build_config(env, enable_pipstar_extract = True, **kwargs): - return env.expect.that_struct( - build_config(enable_pipstar_extract = enable_pipstar_extract, **kwargs), - attrs = dict( - auth_patterns = subjects.dict, - netrc = subjects.str, - platforms = subjects.dict, - ), - ) - def _default( *, arch_name = None, @@ -118,6 +66,65 @@ def _default( whl_platform_tags = whl_platform_tags or [], ) +# The default value for the default platforms tags use in `_mod`. +_default_tags_default = [ + _default( + platform = "{}_{}{}".format(os, cpu, freethreaded), + os_name = os, + arch_name = cpu, + config_settings = [ + "@platforms//os:{}".format(os), + "@platforms//cpu:{}".format(cpu), + ], + whl_abi_tags = ["cp{major}{minor}t"] if freethreaded else ["abi3", "cp{major}{minor}"], + whl_platform_tags = whl_platform_tags, + ) + for (os, cpu, freethreaded), whl_platform_tags in { + ("linux", "x86_64", ""): ["linux_x86_64", "manylinux_*_x86_64"], + ("linux", "x86_64", "_freethreaded"): ["linux_x86_64", "manylinux_*_x86_64"], + ("linux", "aarch64", ""): ["linux_aarch64", "manylinux_*_aarch64"], + ("osx", "aarch64", ""): ["macosx_*_arm64"], + ("windows", "aarch64", ""): ["win_arm64"], + }.items() +] + +def _mod(*, name, default = _default_tags_default, parse = [], override = [], whl_mods = [], is_root = True): + return struct( + name = name, + tags = struct( + parse = parse, + override = override, + whl_mods = whl_mods, + default = default, + ), + is_root = is_root, + ) + +def _parse_modules(env, **kwargs): + return env.expect.that_struct( + parse_modules(**kwargs), + attrs = dict( + exposed_packages = subjects.dict, + hub_group_map = subjects.dict, + hub_whl_map = subjects.dict, + whl_libraries = subjects.dict, + whl_mods = subjects.dict, + ), + ) + +def _build_config(env, **kwargs): + return env.expect.that_struct( + build_config( + enable_pipstar_extract = True, + **kwargs + ), + attrs = dict( + auth_patterns = subjects.dict, + netrc = subjects.str, + platforms = subjects.dict, + ), + ) + def _test_simple(env): pypi = _parse_modules( env, @@ -165,6 +172,59 @@ def _test_simple(env): _tests.append(_test_simple) +def _test_simple_isolated(env): + """Simulate `isolate = True` with parse_modules. + + No pip.default tags, but requirements parsing still produces the expected + hub output. + """ + pypi = _parse_modules( + env, + module_ctx = _pypi_mock_mctx( + _mod( + name = "my_module", + default = [], # no platform tags + parse = [ + _parse( + hub_name = "pypi", + python_version = "3.15", + simpleapi_skip = ["simple"], + requirements_lock = "requirements.txt", + ), + ], + ), + os_name = "linux", + arch_name = "x86_64", + ), + available_interpreters = { + "python_3_15_host": "unit_test_interpreter_target", + }, + minor_mapping = {"3.15": "3.15.19"}, + ) + + pypi.exposed_packages().contains_exactly({"pypi": ["simple"]}) + pypi.hub_group_map().contains_exactly({"pypi": {}}) + pypi.hub_whl_map().contains_exactly({"pypi": { + "simple": { + "pypi_315_simple": [ + whl_config_setting( + version = "3.15", + ), + ], + }, + }}) + pypi.whl_libraries().contains_exactly({ + "pypi_315_simple": { + "config_load": "@pypi//:config.bzl", + "dep_template": "@pypi//{name}:{target}", + "python_interpreter_target": "unit_test_interpreter_target", + "requirement": "simple==0.0.1 --hash=sha256:deadbeef --hash=sha256:deadbaaf", + }, + }) + pypi.whl_mods().contains_exactly({}) + +_tests.append(_test_simple_isolated) + def _test_build_pipstar_platform(env): config = _build_config( env, @@ -216,6 +276,9 @@ def _test_build_pipstar_platform(env): whl_abi_tags = ["none", "abi3", "cp{major}{minor}"], whl_platform_tags = ["any"], ), + } | { + name: _plat(**values) + for name, values in default_platforms().items() }) _tests.append(_test_build_pipstar_platform) From 4cce53412988f214ddda51af71e3e5c5d3240904 Mon Sep 17 00:00:00 2001 From: Jeremy Nimmer Date: Tue, 28 Apr 2026 20:19:03 -0700 Subject: [PATCH 724/922] fix(uv): use astral urls for uv primary source, github as secondary (#3746) For official releases, use the astral.sh mirror as the preferred url for binary downloads, with github.com as a fallback. For uv >= 0.11.0, read the checksums directly from the dist-manifest contents, so the only github.com SPOF is fetching the dist-manifest itself. --- It is not infrequent for `github.com` to give a 5xx error on one of the uv sha256 downloads, leading to a failed build. (The sha256 downloads cannot be cached by Bazel's repository cache, since we request them without a checksum.) See https://github.com/RobotLocomotion/drake/issues/24140 for some examples, as well as https://github.com/bazelbuild/bazel-central-registry/pull/8591#issuecomment-4332125830 ([log](https://buildkite.com/bazel/bcr-presubmit/builds/33406/canvas?jid=019dd214-ce96-4eeb-b6e8-a720932ed816&tab=output)). This change limits github as a single point of failure. --------- Co-authored-by: Richard Levasseur --- CHANGELOG.md | 3 +++ python/uv/private/uv.bzl | 49 ++++++++++++++++++++++++++++++++++------ tests/uv/uv/uv_tests.bzl | 2 ++ 3 files changed, 47 insertions(+), 7 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 2fbe0e8960..3cea48ba16 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -71,6 +71,9 @@ END_UNRELEASED_TEMPLATE files ([#3126](https://github.com/bazel-contrib/rules_python/issues/3126)). * (pypi) Support `--experimental_isolated_extension_usages` ([#3668](https://github.com/bazel-contrib/rules_python/issues/3668)). +* (uv) use the astral.sh mirror as the preferred url for binary downloads, + with github.com as a fallback; for uv >= 0.11.0, read the checksums directly + from the dist-manifest contents. {#v0-0-0-added} ### Added diff --git a/python/uv/private/uv.bzl b/python/uv/private/uv.bzl index d9302c71ae..969b6f672d 100644 --- a/python/uv/private/uv.bzl +++ b/python/uv/private/uv.bzl @@ -380,6 +380,10 @@ def _overlap(first_collection, second_collection): return False +# See https://github.com/astral-sh/setup-uv/pull/809. +GITHUB_RELEASES_PREFIX = "https://github.com/astral-sh/uv/releases/download/" +ASTRAL_MIRROR_PREFIX = "https://releases.astral.sh/github/uv/releases/download/" + def _get_tool_urls_from_dist_manifest(module_ctx, *, base_url, manifest_filename, platforms, get_auth = get_auth, **auth_attrs): """Download the results about remote tool sources. @@ -452,6 +456,8 @@ def _get_tool_urls_from_dist_manifest(module_ctx, *, base_url, manifest_filename } ] checksum "uv-aarch64-apple-darwin.tar.gz.sha256" + checksums + sha256 "0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef" uv-aarch64-apple-darwin.tar.gz.sha256 name "uv-aarch64-apple-darwin.tar.gz.sha256" kind "checksum" @@ -471,7 +477,8 @@ def _get_tool_urls_from_dist_manifest(module_ctx, *, base_url, manifest_filename fail(result) dist_manifest = json.decode(module_ctx.read(dist_manifest)) - base_url = ( + # Use the simple download_url from the manifest, when available. + dist_base_url = ( dist_manifest .get("releases", [{}])[0] .get("hosting", {}) @@ -479,21 +486,46 @@ def _get_tool_urls_from_dist_manifest(module_ctx, *, base_url, manifest_filename .get("download_url", base_url) ) + # For official releases, add the astral mirror to improve availability. + # See https://github.com/astral-sh/setup-uv/pull/809. + if dist_base_url.startswith(GITHUB_RELEASES_PREFIX): + astral_base_url = ASTRAL_MIRROR_PREFIX + dist_base_url[len(GITHUB_RELEASES_PREFIX):] + base_urls = [ + astral_base_url, + dist_base_url, + ] + else: + base_urls = [dist_base_url] + artifacts = dist_manifest["artifacts"] tool_sources = {} downloads = {} for fname, artifact in artifacts.items(): if artifact.get("kind") != "executable-zip": continue + target_triples = artifact["target_triples"] + if not _overlap(target_triples, platforms): + # We are not interested in this platform, so skip. + continue - checksum = artifacts[artifact["checksum"]] - if not _overlap(checksum["target_triples"], platforms): - # we are not interested in this platform, so skip + # Releases of uv >= 0.11.0 have the sha256 directly inline. + sha256 = artifact.get("checksums", {}).get("sha256", "") + if len(sha256) > 0: + for platform in target_triples: + tool_sources[platform] = struct( + urls = [ + "{}/{}".format(base, fname) + for base in base_urls + ], + sha256 = sha256, + ) continue + # For uv < 0.11.0, we'll need to fetch the individual sha256 files. + checksum = artifacts[artifact["checksum"]] checksum_fname = checksum["name"] checksum_path = module_ctx.path(checksum_fname) - urls = ["{}/{}".format(base_url, checksum_fname)] + urls = ["{}/{}".format(dist_base_url, checksum_fname)] downloads[checksum_path] = struct( download = module_ctx.download( url = urls, @@ -502,7 +534,7 @@ def _get_tool_urls_from_dist_manifest(module_ctx, *, base_url, manifest_filename auth = get_auth(module_ctx, urls, ctx_attr = auth_attr), ), archive_fname = fname, - platforms = checksum["target_triples"], + platforms = target_triples, ) for checksum_path, download in downloads.items(): @@ -522,7 +554,10 @@ def _get_tool_urls_from_dist_manifest(module_ctx, *, base_url, manifest_filename for platform in download.platforms: tool_sources[platform] = struct( - urls = ["{}/{}".format(base_url, archive_fname)], + urls = [ + "{}/{}".format(base, archive_fname) + for base in base_urls + ], sha256 = sha256, ) diff --git a/tests/uv/uv/uv_tests.bzl b/tests/uv/uv/uv_tests.bzl index cb60cf48ec..d11bf065d8 100644 --- a/tests/uv/uv/uv_tests.bzl +++ b/tests/uv/uv/uv_tests.bzl @@ -34,6 +34,7 @@ def _uv_mock_mctx(*modules, download = None): x: { "checksum": x + ".sha256", "kind": "executable-zip", + "target_triples": [x], } for x in ["linux", "osx"] } | { @@ -47,6 +48,7 @@ def _uv_mock_mctx(*modules, download = None): x: { "checksum": x + ".sha256", "kind": "executable-zip", + "target_triples": [x], } for x in ["linux", "os", "osx", "something_extra"] } | { From 588a42bfd6104d176fd5aa44203f241bea10f0f0 Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Wed, 29 Apr 2026 15:44:56 +0900 Subject: [PATCH 725/922] build(deps): bump softprops/action-gh-release from 2 to 3 (#3702) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Bumps [softprops/action-gh-release](https://github.com/softprops/action-gh-release) from 2 to 3.
Release notes

Sourced from softprops/action-gh-release's releases.

v3.0.0

3.0.0 is a major release that moves the action runtime from Node 20 to Node 24. Use v3 on GitHub-hosted runners and self-hosted fleets that already support the Node 24 Actions runtime. If you still need the last Node 20-compatible line, stay on v2.6.2.

What's Changed

Other Changes 🔄

  • Move the action runtime and bundle target to Node 24
  • Update @types/node to the Node 24 line and allow future Dependabot updates
  • Keep the floating major tag on v3; v2 remains pinned to the latest 2.x release

v2.6.2

What's Changed

Other Changes 🔄

Full Changelog: https://github.com/softprops/action-gh-release/compare/v2...v2.6.2

v2.6.1

2.6.1 is a patch release focused on restoring linked discussion thread creation when discussion_category_name is set. It fixes [#764](https://github.com/softprops/action-gh-release/issues/764), where the draft-first publish flow stopped carrying the discussion category through the final publish step.

If you still hit an issue after upgrading, please open a report with the bug template and include a minimal repro or sanitized workflow snippet where possible.

What's Changed

Bug fixes 🐛

v2.6.0

2.6.0 is a minor release centered on previous_tag support for generate_release_notes, which lets workflows pin GitHub's comparison base explicitly instead of relying on the default range. It also includes the recent concurrent asset upload recovery fix, a working_directory docs sync, a checked-bundle freshness guard for maintainers, and clearer immutable-prerelease guidance where GitHub platform behavior imposes constraints on how prerelease asset uploads can be published.

If you still hit an issue after upgrading, please open a report with the bug template and include a minimal repro or sanitized workflow snippet where possible.

What's Changed

... (truncated)

Changelog

Sourced from softprops/action-gh-release's changelog.

0.1.13

  • fix issue with multiple runs concatenating release bodies #145
Commits

[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=softprops/action-gh-release&package-manager=github_actions&previous-version=2&new-version=3)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) ---
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot show ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> --- .github/workflows/release.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index c565b03fa0..1e4f26981b 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -46,7 +46,7 @@ jobs: - name: Create release archive and notes run: .github/workflows/create_archive_and_notes.sh ${{ inputs.tag_name || github.ref_name }} - name: Release - uses: softprops/action-gh-release@v2 + uses: softprops/action-gh-release@v3 with: # Use GH feature to populate the changelog automatically generate_release_notes: true From 78ae1f08f6c0ca1bda1210ff6228776a4a448f0a Mon Sep 17 00:00:00 2001 From: Ignas Anikevicius <240938+aignas@users.noreply.github.com> Date: Wed, 29 Apr 2026 23:58:51 +0900 Subject: [PATCH 726/922] fix(pypi): harden the WORKSPACE python detection in pip_repository (#3744) A fixup to #3737 --- python/private/pypi/pip_repository.bzl | 13 ++++++++----- 1 file changed, 8 insertions(+), 5 deletions(-) diff --git a/python/private/pypi/pip_repository.bzl b/python/private/pypi/pip_repository.bzl index b449f8dd85..d55953aaa7 100644 --- a/python/private/pypi/pip_repository.bzl +++ b/python/private/pypi/pip_repository.bzl @@ -90,11 +90,14 @@ def _pip_repository_impl(rctx): python_interpreter = rctx.attr.python_interpreter, python_interpreter_target = rctx.attr.python_interpreter_target, ) - result = rctx.execute([python_interpreter, "--version"]) - if result.stdout: - python_version = result.stdout.strip().split(" ")[-1] - else: - fail("Could not determine Python version") + result = pypi_repo_utils.execute_checked( + rctx, + python = python_interpreter, + srcs = [], + op = "GetPythonVersion", + arguments = ["-c", "import sys; print(sys.version.split()[0])"], + ) + python_version = result.stdout.strip().splitlines()[-1] platforms = [ "linux_aarch64", "linux_arm", From eda9fe30b703f3f2c4c4c57af46c30d3ecbbcfef Mon Sep 17 00:00:00 2001 From: Steve Barrau <98589981+stevebarrau@users.noreply.github.com> Date: Wed, 29 Apr 2026 17:34:02 +0100 Subject: [PATCH 727/922] feat: add package metadata for wheel libraries (#3531) Add package_metadata rule to generated BUILD files for wheel libraries to track package provenance using PURL (Package URL) format. This is then picked up by [supply_chain_tools](https://registry.bazel.build/modules/supply_chain_tools) to produce SBOM for python target using external dependencies. Fixes #2054 --------- Co-authored-by: Ignas Anikevicius <240938+aignas@users.noreply.github.com> --- CHANGELOG.md | 2 ++ MODULE.bazel | 3 +- python/private/py_repositories.bzl | 6 ++++ python/private/pypi/BUILD.bazel | 1 + .../pypi/generate_whl_library_build_bazel.bzl | 14 +++++++- python/private/pypi/whl_library.bzl | 32 ++++++++++++++++++- ...generate_whl_library_build_bazel_tests.bzl | 28 ++++++++++++++++ 7 files changed, 83 insertions(+), 3 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 3cea48ba16..6317404f66 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -88,6 +88,8 @@ END_UNRELEASED_TEMPLATE * (toolchains) `3.13.12`, `3.14.3` Python toolchain from [20260325] release. * (toolchains) `3.10.20`, `3.11.15`, `3.12.13`, `3.13.13` `3.14.4`, `3.15.0a8` * Python toolchain from [20260414] release. +* (pypi) `package_metadata` support, fixes + [#2054](https://github.com/bazel-contrib/rules_python/issues/2054). [20260325]: https://github.com/astral-sh/python-build-standalone/releases/tag/20260325 [20260414]: https://github.com/astral-sh/python-build-standalone/releases/tag/20260414 diff --git a/MODULE.bazel b/MODULE.bazel index bb3a9dcab5..6f24369367 100644 --- a/MODULE.bazel +++ b/MODULE.bazel @@ -6,8 +6,9 @@ module( bazel_dep(name = "bazel_features", version = "1.21.0") bazel_dep(name = "bazel_skylib", version = "1.8.2") -bazel_dep(name = "rules_cc", version = "0.1.5") +bazel_dep(name = "package_metadata", version = "0.0.7") bazel_dep(name = "platforms", version = "0.0.11") +bazel_dep(name = "rules_cc", version = "0.1.5") # Those are loaded only when using py_proto_library # Use py_proto_library directly from protobuf repository diff --git a/python/private/py_repositories.bzl b/python/private/py_repositories.bzl index e3ab11c561..9c4051ede9 100644 --- a/python/private/py_repositories.bzl +++ b/python/private/py_repositories.bzl @@ -72,6 +72,12 @@ def py_repositories(transition_settings = []): strip_prefix = "rules_cc-0.1.5", urls = ["https://github.com/bazelbuild/rules_cc/releases/download/0.1.5/rules_cc-0.1.5.tar.gz"], ) + http_archive( + name = "package_metadata", + sha256 = "8f27dc7393e3f3bdc793bdc4ba36d67a63c22cc9d38cc65d3204654974ea4563", + strip_prefix = "supply-chain-0.0.7/metadata", + url = "https://github.com/bazel-contrib/supply-chain/releases/download/v0.0.7/supply-chain-v0.0.7.tar.gz", + ) # Needed by rules_cc, triggered by @rules_java_prebuilt in Bazel by using @rules_cc//cc:defs.bzl # NOTE: This name must be com_google_protobuf until Bazel drops WORKSPACE diff --git a/python/private/pypi/BUILD.bazel b/python/private/pypi/BUILD.bazel index e7d19ea636..02c06a8096 100644 --- a/python/private/pypi/BUILD.bazel +++ b/python/private/pypi/BUILD.bazel @@ -495,6 +495,7 @@ bzl_library( "//python/private:auth_bzl", "//python/private:envsubst_bzl", "//python/private:is_standalone_interpreter_bzl", + "//python/private:normalize_name_bzl", "//python/private:repo_utils_bzl", "//python/private:util_bzl", "@rules_python_internal//:rules_python_config_bzl", diff --git a/python/private/pypi/generate_whl_library_build_bazel.bzl b/python/private/pypi/generate_whl_library_build_bazel.bzl index 5811ed1574..768b064a5d 100644 --- a/python/private/pypi/generate_whl_library_build_bazel.bzl +++ b/python/private/pypi/generate_whl_library_build_bazel.bzl @@ -40,6 +40,12 @@ _TEMPLATE = """\ package(default_visibility = ["//visibility:public"]) +package_metadata( + name = "package_metadata", + purl = {purl}, + visibility = ["//:__subpackages__"], +) + {fn}( {kwargs} ) @@ -49,12 +55,14 @@ def generate_whl_library_build_bazel( *, annotation = None, default_python_version = None, + purl = None, **kwargs): """Generate a BUILD file for an unzipped Wheel Args: annotation: The annotation for the build file. default_python_version: The python version to use to parse the METADATA. + purl: The purl. **kwargs: Extra args serialized to be passed to the {obj}`whl_library_targets`. @@ -62,7 +70,10 @@ def generate_whl_library_build_bazel( A complete BUILD file as a string """ - loads = [] + loads = [ + """load("@package_metadata//rules:package_metadata.bzl", "package_metadata")""", + ] + if kwargs.get("tags"): fn = "whl_library_targets" @@ -119,6 +130,7 @@ def generate_whl_library_build_bazel( "{} = {},".format(k, _RENDER.get(k, repr)(v)) for k, v in sorted(kwargs.items()) ])), + purl = repr(purl), ), ] + additional_content, ) diff --git a/python/private/pypi/whl_library.bzl b/python/private/pypi/whl_library.bzl index 36df4dc82e..13a8e6ff8e 100644 --- a/python/private/pypi/whl_library.bzl +++ b/python/private/pypi/whl_library.bzl @@ -18,6 +18,7 @@ load("@rules_python_internal//:rules_python_config.bzl", rp_config = "config") load("//python/private:auth.bzl", "AUTH_ATTRS", "get_auth") load("//python/private:envsubst.bzl", "envsubst") load("//python/private:is_standalone_interpreter.bzl", "is_standalone_interpreter") +load("//python/private:normalize_name.bzl", "normalize_name") load("//python/private:repo_utils.bzl", "REPO_DEBUG_ENV_VAR", "repo_utils") load(":attrs.bzl", "ATTRS", "use_isolated") load(":deps.bzl", "all_repo_names", "record_files") @@ -276,6 +277,24 @@ def _extract_whl_py(rctx, *, python_interpreter, args, whl_path, environment, lo logger = logger, ) +def _to_purl(*, index, metadata, filename): + """ + Produce a PyPI PURL from the metadata. + + https://github.com/package-url/purl-spec/blob/main/types-doc/pypi-definition.md + """ + + # https://github.com/package-url/purl-spec/blob/main/types-doc/pypi-definition.md#name-definition + name = normalize_name(metadata.name).replace("_", "-") + + qualifiers = {} + if index: + qualifiers["repository_url"] = index + if filename: + qualifiers["file_name"] = filename + + return "pkg:pypi/{}@{}?{}".format(name, metadata.version, "&".join(["{}={}".format(key, val) for key, val in qualifiers.items()])) + def _whl_library_impl(rctx): logger = repo_utils.logger(rctx) @@ -436,6 +455,11 @@ def _whl_library_impl(rctx): group_name = rctx.attr.group_name, namespace_package_files = namespace_package_files, extras = requirement(rctx.attr.requirement).extras, + purl = _to_purl( + index = rctx.attr.index_url, + metadata = metadata, + filename = sdist_filename or whl_path.basename, + ), ) # Delete these in case the wheel had them. They generally don't cause @@ -443,7 +467,13 @@ def _whl_library_impl(rctx): rctx.file("WORKSPACE") rctx.file("WORKSPACE.bazel") rctx.file("MODULE.bazel") - rctx.file("REPO.bazel") + rctx.file("REPO.bazel", """\ +repo( + default_package_metadata = [ + "//:package_metadata", + ], +) +""") # BUILD files interfere with globbing and Bazel package boundaries. _remove_files(rctx, "BUILD", "BUILD.bazel") diff --git a/tests/pypi/generate_whl_library_build_bazel/generate_whl_library_build_bazel_tests.bzl b/tests/pypi/generate_whl_library_build_bazel/generate_whl_library_build_bazel_tests.bzl index 2f421f35d4..9586581cad 100644 --- a/tests/pypi/generate_whl_library_build_bazel/generate_whl_library_build_bazel_tests.bzl +++ b/tests/pypi/generate_whl_library_build_bazel/generate_whl_library_build_bazel_tests.bzl @@ -21,10 +21,17 @@ _tests = [] def _test_all_legacy(env): want = """\ +load("@package_metadata//rules:package_metadata.bzl", "package_metadata") load("@rules_python//python/private/pypi:whl_library_targets.bzl", "whl_library_targets") package(default_visibility = ["//visibility:public"]) +package_metadata( + name = "package_metadata", + purl = None, + visibility = ["//:__subpackages__"], +) + whl_library_targets( copy_executables = { "exec_src": "exec_dest", @@ -79,11 +86,18 @@ _tests.append(_test_all_legacy) def _test_all_workspace(env): want = """\ +load("@package_metadata//rules:package_metadata.bzl", "package_metadata") load("@pypi//:config.bzl", "packages") load("@rules_python//python/private/pypi:whl_library_targets.bzl", "whl_library_targets_from_requires") package(default_visibility = ["//visibility:public"]) +package_metadata( + name = "package_metadata", + purl = None, + visibility = ["//:__subpackages__"], +) + whl_library_targets_from_requires( copy_executables = { "exec_src": "exec_dest", @@ -138,11 +152,18 @@ _tests.append(_test_all_workspace) def _test_all(env): want = """\ +load("@package_metadata//rules:package_metadata.bzl", "package_metadata") load("@pypi//:config.bzl", "packages") load("@rules_python//python/private/pypi:whl_library_targets.bzl", "whl_library_targets_from_requires") package(default_visibility = ["//visibility:public"]) +package_metadata( + name = "package_metadata", + purl = None, + visibility = ["//:__subpackages__"], +) + whl_library_targets_from_requires( copy_executables = { "exec_src": "exec_dest", @@ -197,11 +218,18 @@ _tests.append(_test_all) def _test_all_with_loads(env): want = """\ +load("@package_metadata//rules:package_metadata.bzl", "package_metadata") load("@pypi//:config.bzl", "packages") load("@rules_python//python/private/pypi:whl_library_targets.bzl", "whl_library_targets_from_requires") package(default_visibility = ["//visibility:public"]) +package_metadata( + name = "package_metadata", + purl = None, + visibility = ["//:__subpackages__"], +) + whl_library_targets_from_requires( copy_executables = { "exec_src": "exec_dest", From a6e7d34dc8e3fbd8e7944fce27667baf17b95043 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Wed, 29 Apr 2026 20:27:57 -0700 Subject: [PATCH 728/922] feat(venv): make wheel scripts runnable in venv (#3743) Currently, `#!python` based scripts in a wheel are put into the venv as-is, so aren't actually runnable. Additionally, wheel entry points aren't installed into the venv. To fix, rewrite the scripts to re-exec themselves with the `python3` binary in the same directory. Also adds support for wheel entry points in venvs. This is done by having the repo-phase parse entry_points.txt and defining a target for each entry point. These targets run during build phase, so are able to generate platform-specific outputs. Cross-building with Windows is also added, supporting both Windows as a target or exec platform to generate outputs for the target platform. Fixes https://github.com/bazel-contrib/rules_python/issues/3202 --- examples/pip_parse/pip_parse_test.py | 14 +- python/private/py_executable.bzl | 8 ++ python/private/pypi/BUILD.bazel | 30 +++++ .../pypi/generate_whl_library_build_bazel.bzl | 1 + python/private/pypi/venv_entry_point.bzl | 50 +++++++ .../pypi/venv_entry_point_template.bat | 8 ++ .../private/pypi/venv_entry_point_template.sh | 10 ++ python/private/pypi/venv_rewrite_shebang.bzl | 82 ++++++++++++ python/private/pypi/venv_shebang_rewriter.ps1 | 44 +++++++ python/private/pypi/venv_shebang_rewriter.sh | 27 ++++ python/private/pypi/whl_extract.bzl | 49 +++---- python/private/pypi/whl_installer/wheel.py | 40 +++++- python/private/pypi/whl_library.bzl | 36 +++++- python/private/pypi/whl_library_targets.bzl | 47 ++++++- python/private/pypi/whl_metadata.bzl | 55 ++++++++ python/private/repo_utils.bzl | 33 +++++ python/private/text_util.bzl | 24 ++++ python/private/venv_runfiles.bzl | 2 +- .../whl_library_targets_tests.bzl | 43 +++++- .../pypi/whl_metadata/whl_metadata_tests.bzl | 75 +++++++++++ tests/repos/whl_with_data1/BUILD.bazel | 2 +- .../scripts/whl_with_data1_pythonw | 9 ++ .../scripts/whl_with_data1_script | 5 + .../whl_with_data1-1.0.dist-info/RECORD | 1 + .../purelib/whl_with_data2/__init__.py | 6 + .../whl_with_data2-1.0.dist-info/RECORD | 2 + .../entry_points.txt | 2 + tests/venv_site_packages_libs/BUILD.bazel | 19 +++ .../whl_scripts_runnable_test.py | 122 ++++++++++++++++++ 29 files changed, 806 insertions(+), 40 deletions(-) create mode 100644 python/private/pypi/venv_entry_point.bzl create mode 100644 python/private/pypi/venv_entry_point_template.bat create mode 100644 python/private/pypi/venv_entry_point_template.sh create mode 100644 python/private/pypi/venv_rewrite_shebang.bzl create mode 100644 python/private/pypi/venv_shebang_rewriter.ps1 create mode 100755 python/private/pypi/venv_shebang_rewriter.sh create mode 100755 tests/repos/whl_with_data1/whl_with_data1-1.0.data/scripts/whl_with_data1_pythonw create mode 100755 tests/repos/whl_with_data1/whl_with_data1-1.0.data/scripts/whl_with_data1_script create mode 100644 tests/repos/whl_with_data2/whl_with_data2-1.0.dist-info/entry_points.txt create mode 100644 tests/venv_site_packages_libs/whl_scripts_runnable_test.py diff --git a/examples/pip_parse/pip_parse_test.py b/examples/pip_parse/pip_parse_test.py index c532dff564..89e5eca254 100644 --- a/examples/pip_parse/pip_parse_test.py +++ b/examples/pip_parse/pip_parse_test.py @@ -50,18 +50,22 @@ def test_entry_point(self): def test_data(self): actual = os.environ.get("WHEEL_DATA_CONTENTS") self.assertIsNotNone(actual) - actual = self._remove_leading_dirs(actual.split(" ")) + actual = set(self._remove_leading_dirs(actual.split(" "))) + + s3cmd_bin = "bin/s3cmd" + if os.name == "nt": + s3cmd_bin += ".bat" - expected = [ - "bin/s3cmd", + expected = { + s3cmd_bin, "data/share/doc/packages/s3cmd/INSTALL.md", "data/share/doc/packages/s3cmd/LICENSE", "data/share/doc/packages/s3cmd/NEWS", "data/share/doc/packages/s3cmd/README.md", "data/share/man/man1/s3cmd.1", - ] + } - self.assertListEqual(actual, expected) + self.assertEqual(actual, expected) def test_dist_info(self): actual = os.environ.get("WHEEL_DIST_INFO_CONTENTS") diff --git a/python/private/py_executable.bzl b/python/private/py_executable.bzl index 6197c0c789..9c21e5d274 100644 --- a/python/private/py_executable.bzl +++ b/python/private/py_executable.bzl @@ -749,6 +749,14 @@ def _create_venv_windows(ctx, *, venv_ctx_rel_root, runtime, interpreter_actual_ link_to_path = interpreter_actual_path, files = depset([runtime.interpreter]), )) + + # This isn't strictly correct, but should work ok. + interpreter_symlinks.add(ExplicitSymlink( + runfiles_path = paths.join(paths.dirname(rf_path), "pythonw.exe"), + venv_path = paths.join(paths.dirname(venv_rel_path), "pythonw.exe"), + link_to_path = paths.join(paths.dirname(interpreter_actual_path), "pythonw.exe"), + files = depset(), + )) else: # It's OK to use declare_symlink here because an absolute path # will be written to it, so Bazel won't mangle it. diff --git a/python/private/pypi/BUILD.bazel b/python/private/pypi/BUILD.bazel index 02c06a8096..c46ea83874 100644 --- a/python/private/pypi/BUILD.bazel +++ b/python/private/pypi/BUILD.bazel @@ -24,6 +24,24 @@ exports_files( visibility = ["//visibility:public"], ) +alias( + name = "venv_entry_point_template", + actual = select({ + "@platforms//os:windows": "venv_entry_point_template.bat", + "//conditions:default": "venv_entry_point_template.sh", + }), + visibility = ["//visibility:public"], +) + +alias( + name = "venv_shebang_rewriter", + actual = select({ + "@platforms//os:windows": "venv_shebang_rewriter.ps1", + "//conditions:default": "venv_shebang_rewriter.sh", + }), + visibility = ["//visibility:public"], +) + exports_files( srcs = ["deps.bzl"], visibility = ["//tools/private/update_deps:__pkg__"], @@ -520,3 +538,15 @@ bzl_library( name = "whl_target_platforms_bzl", srcs = ["whl_target_platforms.bzl"], ) + +bzl_library( + name = "venv_entry_point_bzl", + srcs = ["venv_entry_point.bzl"], + visibility = ["//visibility:public"], +) + +bzl_library( + name = "venv_rewrite_shebang_bzl", + srcs = ["venv_rewrite_shebang.bzl"], + visibility = ["//visibility:public"], +) diff --git a/python/private/pypi/generate_whl_library_build_bazel.bzl b/python/private/pypi/generate_whl_library_build_bazel.bzl index 768b064a5d..a9a29081f7 100644 --- a/python/private/pypi/generate_whl_library_build_bazel.bzl +++ b/python/private/pypi/generate_whl_library_build_bazel.bzl @@ -23,6 +23,7 @@ _RENDER = { "data_exclude": render.list, "dependencies": render.list, "dependencies_by_platform": lambda x: render.dict(x, value_repr = render.list), + "entry_points": render.dict_dict, "extras": render.list, "group_deps": render.list, "include": str, diff --git a/python/private/pypi/venv_entry_point.bzl b/python/private/pypi/venv_entry_point.bzl new file mode 100644 index 0000000000..32cb6f55a5 --- /dev/null +++ b/python/private/pypi/venv_entry_point.bzl @@ -0,0 +1,50 @@ +"""Rule for generating venv entry point scripts.""" + +load("//python/private:attributes.bzl", "WINDOWS_CONSTRAINTS_ATTRS") +load("//python/private:common.bzl", "is_windows_platform") +load("//python/private:rule_builders.bzl", "ruleb") + +def _venv_entry_point_impl(ctx): + is_windows = is_windows_platform(ctx) + + out_name = ctx.label.name + python_exe = "" + if is_windows: + out_name += ".bat" + python_exe = "pythonw.exe" if ctx.attr.group == "gui_scripts" else "python.exe" + + out = ctx.actions.declare_file(out_name) + + ctx.actions.expand_template( + template = ctx.file._template, + output = out, + substitutions = { + "{ATTRIBUTE}": ctx.attr.attribute, + "{MODULE}": ctx.attr.module, + "{PYTHON_EXE}": python_exe, + }, + is_executable = True, + ) + + return [DefaultInfo( + files = depset([out]), + executable = out, + )] + +_builder = ruleb.Rule( + implementation = _venv_entry_point_impl, + executable = True, +) +_builder.attrs.update({ + "attribute": attr.string(mandatory = False, doc = "The attribute to call"), + "extras": attr.string(mandatory = False, doc = "The extras for the entry point"), + "group": attr.string(mandatory = False, doc = "The entry point group (e.g. console_scripts)"), + "module": attr.string(mandatory = True, doc = "The module to import"), + "_template": attr.label( + default = Label("//python/private/pypi:venv_entry_point_template"), + allow_single_file = True, + ), +}) +_builder.attrs.update(WINDOWS_CONSTRAINTS_ATTRS) + +venv_entry_point = _builder.build() diff --git a/python/private/pypi/venv_entry_point_template.bat b/python/private/pypi/venv_entry_point_template.bat new file mode 100644 index 0000000000..36a7dd3b41 --- /dev/null +++ b/python/private/pypi/venv_entry_point_template.bat @@ -0,0 +1,8 @@ +@setlocal enabledelayedexpansion & "%~dp0{PYTHON_EXE}" -x "%~f0" %* & exit /b !ERRORLEVEL! +# -*- coding: utf-8 -*- +import re +import sys +from {MODULE} import {ATTRIBUTE} +if __name__ == "__main__": + sys.argv[0] = re.sub(r"(-script\.pyw|\.exe)?$", "", sys.argv[0]) + sys.exit({ATTRIBUTE}()) diff --git a/python/private/pypi/venv_entry_point_template.sh b/python/private/pypi/venv_entry_point_template.sh new file mode 100644 index 0000000000..d40c31adc4 --- /dev/null +++ b/python/private/pypi/venv_entry_point_template.sh @@ -0,0 +1,10 @@ +#!/bin/sh +'''exec' "$(dirname "$0")/python3" "$0" "$@" +' ''' +# -*- coding: utf-8 -*- +import re +import sys +from {MODULE} import {ATTRIBUTE} +if __name__ == '__main__': + sys.argv[0] = re.sub(r'(-script\.pyw|\.exe)?$', '', sys.argv[0]) + sys.exit({ATTRIBUTE}()) diff --git a/python/private/pypi/venv_rewrite_shebang.bzl b/python/private/pypi/venv_rewrite_shebang.bzl new file mode 100644 index 0000000000..c653211850 --- /dev/null +++ b/python/private/pypi/venv_rewrite_shebang.bzl @@ -0,0 +1,82 @@ +"""Rule for rewriting portable shebangs.""" + +load("//python/private:attributes.bzl", "WINDOWS_CONSTRAINTS_ATTRS") +load("//python/private:common.bzl", "is_windows_platform", "runfiles_root_path") +load("//python/private:py_info.bzl", "PyInfoBuilder", "VenvSymlinkEntry", "VenvSymlinkKind") +load("//python/private:rule_builders.bzl", "ruleb") + +def _venv_rewrite_shebang_impl(ctx): + is_windows = is_windows_platform(ctx) + + out_name = ctx.label.name + if is_windows: + out_name += ".bat" + + out_file = ctx.actions.declare_file(out_name) + in_file = ctx.file.src + + action_args = ctx.actions.args() + rewriter_file = ctx.files._venv_shebang_rewriter[0] + inputs = depset([in_file, rewriter_file]) + + if rewriter_file.path.endswith(".ps1"): + action_exe = "powershell.exe" + action_args.add_all([ + "-ExecutionPolicy", + "Bypass", + "-NoProfile", + "-File", + rewriter_file, + ]) + else: + action_exe = ctx.attr._venv_shebang_rewriter[DefaultInfo].files_to_run + + action_args.add(in_file) + action_args.add(out_file) + action_args.add("windows" if is_windows else "unix") + + ctx.actions.run( + inputs = inputs, + outputs = [out_file], + executable = action_exe, + arguments = [action_args], + mnemonic = "PyVenvRewriteBin", + progress_message = "Rewriting venv bin script %{input}", + toolchain = None, + ) + + symlink = VenvSymlinkEntry( + kind = VenvSymlinkKind.BIN, + link_to_path = runfiles_root_path(ctx, out_file.short_path), + link_to_file = out_file, + venv_path = out_name, + package = ctx.attr.package, + version = ctx.attr.version, + files = depset([out_file]), + ) + builder = PyInfoBuilder.new() + builder.venv_symlinks.add([symlink]) + py_info = builder.build() + + return [ + DefaultInfo(files = depset([out_file]), executable = out_file), + py_info, + ] + +_builder = ruleb.Rule( + implementation = _venv_rewrite_shebang_impl, + executable = True, +) +_builder.attrs.update({ + "package": attr.string(), + "src": attr.label(mandatory = True, allow_single_file = True), + "version": attr.string(), + "_venv_shebang_rewriter": attr.label( + default = "//python/private/pypi:venv_shebang_rewriter", + allow_files = True, + cfg = "exec", + ), +}) +_builder.attrs.update(WINDOWS_CONSTRAINTS_ATTRS) + +venv_rewrite_shebang = _builder.build() diff --git a/python/private/pypi/venv_shebang_rewriter.ps1 b/python/private/pypi/venv_shebang_rewriter.ps1 new file mode 100644 index 0000000000..fb6077b407 --- /dev/null +++ b/python/private/pypi/venv_shebang_rewriter.ps1 @@ -0,0 +1,44 @@ +[CmdletBinding()] +param( + [Parameter(Position=0, Mandatory=$true)] + [string]$InFile, + + [Parameter(Position=1, Mandatory=$true)] + [string]$OutFile, + + [Parameter(Position=2, Mandatory=$true)] + [string]$TargetOs +) + +$ErrorActionPreference = "Stop" + +$firstLine = Get-Content -Path $InFile -TotalCount 1 -ErrorAction SilentlyContinue +$content = Get-Content -Path $InFile | Select-Object -Skip 1 + +$Utf8NoBom = New-Object System.Text.UTF8Encoding $False + +if ($TargetOs -eq "windows") { + if ($firstLine -match "^#!pythonw") { + $pythonExe = "pythonw.exe" + } else { + $pythonExe = "python.exe" + } + # A Batch-Python polyglot. Batch executes the first line and exits, + # while Python (via -x) ignores the first line and executes the rest. + $wrapper = "@setlocal enabledelayedexpansion & `"%~dp0$pythonExe`" -x `"%~f0`" %* & exit /b !ERRORLEVEL!" + [System.IO.File]::WriteAllText($OutFile, $wrapper + "`r`n", $Utf8NoBom) +} else { + # A Shell-Python polyglot. The shell executes the triple-quoted 'exec' + # command, re-running the script with python3 from the scripts directory. + # Python ignores the triple-quoted string and continues. + $wrapper = @" +#!/bin/sh +'''exec' "`$(dirname "`$0")/python3" "`$0" "`$@" +' ''' +"@ + [System.IO.File]::WriteAllText($OutFile, $wrapper + "`n", $Utf8NoBom) +} + +if ($null -ne $content) { + [System.IO.File]::AppendAllLines($OutFile, [string[]]$content, $Utf8NoBom) +} diff --git a/python/private/pypi/venv_shebang_rewriter.sh b/python/private/pypi/venv_shebang_rewriter.sh new file mode 100755 index 0000000000..d4391d3352 --- /dev/null +++ b/python/private/pypi/venv_shebang_rewriter.sh @@ -0,0 +1,27 @@ +#!/bin/sh +set -eu + +IN="$1" +OUT="$2" +TARGET_OS="$3" + +FIRST_LINE=$(head -n 1 "$IN") + +if [ "$TARGET_OS" = "windows" ]; then + case "$FIRST_LINE" in + "#!pythonw"*) PYTHON_EXE="pythonw.exe" ;; + *) PYTHON_EXE="python.exe" ;; + esac + # A Batch-Python polyglot. Batch executes the first line and exits, + # while Python (via -x) ignores the first line and executes the rest. + printf "@setlocal enabledelayedexpansion & \"%%~dp0$PYTHON_EXE\" -x \"%%~f0\" %%* & exit /b !ERRORLEVEL!\r\n" > "$OUT" +else + printf "#!/bin/sh\n" > "$OUT" + # A Shell-Python polyglot. The shell executes the triple-quoted 'exec' + # command, re-running the script with python3 from the scripts directory. + # Python ignores the triple-quoted string and continues. + printf "'''exec' \"\$(dirname \"\$0\")/python3\" \"\$0\" \"\$@\"\n' '''\n" >> "$OUT" +fi + +tail -n +2 "$IN" >> "$OUT" +chmod +x "$OUT" diff --git a/python/private/pypi/whl_extract.bzl b/python/private/pypi/whl_extract.bzl index 506be05481..2ebb61a83a 100644 --- a/python/private/pypi/whl_extract.bzl +++ b/python/private/pypi/whl_extract.bzl @@ -20,22 +20,8 @@ def whl_extract(rctx, *, whl_path, logger): supports_whl_extraction = rp_config.supports_whl_extraction, ) - # Fix permissions on extracted files. Some wheels have files without read permissions set, - # which causes errors when trying to read them later. - os_name = repo_utils.get_platforms_os_name(rctx) - if os_name != "windows": - # On Unix-like systems, recursively add read permissions to all files - # and ensure directories are traversable (need execute permission) - result = repo_utils.execute_unchecked( - rctx, - op = "Fixing wheel permissions {}".format(whl_path), - arguments = ["chmod", "-R", "a+rX", str(install_dir_path)], - logger = logger, - ) - if result.return_code != 0: - # It's possible chmod is not available or the filesystem doesn't support it. - # This is fine, we just want to try to fix permissions if possible. - logger.warn(lambda: "Failed to fix file permissions: {}".format(result.stderr)) + _maybe_fix_permissions(rctx, whl_path = whl_path, logger = logger) + metadata_file = find_whl_metadata( install_dir = install_dir_path, logger = logger, @@ -70,17 +56,36 @@ def whl_extract(rctx, *, whl_path, logger): # The prefix does not exist in the wheel, we can continue continue - for (src, dest) in merge_trees(src, rctx.path(dest_prefix)): + dest_dir = rctx.path(dest_prefix) + repo_utils.mkdir(rctx, dest_dir) + for (src, dest) in merge_trees(src, dest_dir): logger.debug(lambda: "Renaming: {} -> {}".format(src, dest)) - rctx.rename(src, dest) - - # TODO @aignas 2025-12-16: when moving scripts to `bin`, rewrite the #!python - # shebang to be something else, for inspiration look at the hermetic - # toolchain wrappers + repo_utils.rename(rctx, src, dest) # Ensure that there is no data dir left rctx.delete(data_dir) +# TODO: This can be removed when Bazel 8.6+ is the minimum supported version. +def _maybe_fix_permissions(rctx, *, whl_path, logger): + # Fix permissions on extracted files. Some wheels have files without read permissions set, + # which causes errors when trying to read them later. + # We apply this to the root directory to ensure that everything in bin/, site-packages/, + # etc. is readable and executable where appropriate. + os_name = repo_utils.get_platforms_os_name(rctx) + if os_name != "windows": + # On Unix-like systems, recursively add read permissions to all files + # and ensure directories are traversable (need execute permission) + result = repo_utils.execute_unchecked( + rctx, + op = "Fixing wheel permissions {}".format(whl_path), + arguments = ["chmod", "-R", "a+rX", "."], + logger = logger, + ) + if result.return_code != 0: + # It's possible chmod is not available or the filesystem doesn't support it. + # This is fine, we just want to try to fix permissions if possible. + logger.warn(lambda: "Failed to fix file permissions: {}".format(result.stderr)) + def merge_trees(src, dest): """Merge src into the destination path. diff --git a/python/private/pypi/whl_installer/wheel.py b/python/private/pypi/whl_installer/wheel.py index 4987c915cc..801fd0f3b9 100644 --- a/python/private/pypi/whl_installer/wheel.py +++ b/python/private/pypi/whl_installer/wheel.py @@ -20,6 +20,41 @@ import installer +class DoNothingCm: + """A context manager that does nothing when written to.""" + + def __enter__(self): + return self + + def __exit__(self, *args): + pass + + def write(self, data): + pass + + +class NoEntryPointsSchemeDictionaryDestination( + installer.destinations.SchemeDictionaryDestination +): + """ + A custom destination that prevents the `installer` package from automatically + generating scripts for `console_scripts` entry points. + + rules_python handles entry points via its own `venv_entry_point` targets. + If `installer` also generates these scripts in the `bin/` directory, it + causes a target naming collision because `whl_library_targets.bzl` will + try to create a `venv_rewrite_shebang` target with the same name. + + By overriding `for_script` to return a no-op dummy writer, we silently + discard the generated entry point scripts while still allowing `installer` + to process the rest of the wheel normally (including `.data/scripts` which + we do want to keep). + """ + + def for_script(self, name, module, attribute): + return DoNothingCm() + + class Wheel: """Representation of the compressed .whl file""" @@ -50,10 +85,11 @@ def unzip(self, directory: str) -> None: "scripts": "/bin", "data": "/data", } - destination = installer.destinations.SchemeDictionaryDestination( + + destination = NoEntryPointsSchemeDictionaryDestination( installation_schemes, # TODO Should entry_point scripts also be handled by installer rather than custom code? - interpreter="/dev/null", + interpreter="python", script_kind="posix", destdir=directory, bytecode_optimization_levels=[], diff --git a/python/private/pypi/whl_library.bzl b/python/private/pypi/whl_library.bzl index 13a8e6ff8e..529514578d 100644 --- a/python/private/pypi/whl_library.bzl +++ b/python/private/pypi/whl_library.bzl @@ -28,7 +28,7 @@ load(":pep508_requirement.bzl", "requirement") load(":pypi_repo_utils.bzl", "pypi_repo_utils") load(":urllib.bzl", "urllib") load(":whl_extract.bzl", "whl_extract") -load(":whl_metadata.bzl", "whl_metadata") +load(":whl_metadata.bzl", "parse_entry_points", "whl_metadata") _CPPFLAGS = "CPPFLAGS" _COMMAND_LINE_TOOLS_PATH_SLUG = "commandlinetools" @@ -277,6 +277,36 @@ def _extract_whl_py(rctx, *, python_interpreter, args, whl_path, environment, lo logger = logger, ) +def _get_entry_points(rctx, install_dir_path, metadata): + dist_info_dir = "{}-{}.dist-info".format( + metadata.name.replace("-", "_"), + metadata.version.replace("-", "_"), + ) + entry_points_txt = install_dir_path.get_child(dist_info_dir).get_child("entry_points.txt") + if entry_points_txt.exists: + return parse_entry_points(rctx.read(entry_points_txt)) + return {} + +def _move_scripts_needing_shebang_rewrite(rctx, entry_points): + bin_dir = rctx.path("bin") + if not bin_dir.exists: + return + + ep_names = {name.lower(): True for name in entry_points} + for script in bin_dir.readdir(): + if script.is_dir: + continue + if script.basename.lower() in ep_names: + rctx.delete(script) + continue + if script.basename.endswith(".exe") or script.basename.endswith(".dll"): + continue + content = rctx.read(script) + if content.startswith("#!python"): + rewrite_bin_dir = rctx.path("rewrite-bin") + repo_utils.mkdir(rctx, rewrite_bin_dir) + repo_utils.rename(rctx, script, rctx.path("rewrite-bin/" + script.basename)) + def _to_purl(*, index, metadata, filename): """ Produce a PyPI PURL from the metadata. @@ -436,6 +466,9 @@ def _whl_library_impl(rctx): ) namespace_package_files = pypi_repo_utils.find_namespace_package_files(rctx, install_dir_path) + entry_points = _get_entry_points(rctx, install_dir_path, metadata) + _move_scripts_needing_shebang_rewrite(rctx, entry_points) + build_file_contents = generate_whl_library_build_bazel( name = whl_path.basename, sdist_filename = sdist_filename, @@ -455,6 +488,7 @@ def _whl_library_impl(rctx): group_name = rctx.attr.group_name, namespace_package_files = namespace_package_files, extras = requirement(rctx.attr.requirement).extras, + entry_points = entry_points, purl = _to_purl( index = rctx.attr.index_url, metadata = metadata, diff --git a/python/private/pypi/whl_library_targets.bzl b/python/private/pypi/whl_library_targets.bzl index 4ed66cdddc..b3a52cd18c 100644 --- a/python/private/pypi/whl_library_targets.bzl +++ b/python/private/pypi/whl_library_targets.bzl @@ -31,6 +31,8 @@ load( ) load(":namespace_pkgs.bzl", _create_inits = "create_inits") load(":pep508_deps.bzl", "deps") +load(":venv_entry_point.bzl", "venv_entry_point") +load(":venv_rewrite_shebang.bzl", "venv_rewrite_shebang") # Files that are special to the Bazel processing of things. _BAZEL_REPO_FILE_GLOBS = [ @@ -43,6 +45,7 @@ _BAZEL_REPO_FILE_GLOBS = [ ] _IS_VENV_SITE_PACKAGES_YES = Label("//python/config_settings:_is_venvs_site_packages_yes") +_VENV_SITE_PACKAGES_FLAG = Label("//python/config_settings:venvs_site_packages") def whl_library_targets_from_requires( *, @@ -51,6 +54,7 @@ def whl_library_targets_from_requires( metadata_version = "", requires_dist = [], extras = [], + entry_points = {}, include = [], group_deps = [], **kwargs): @@ -67,6 +71,7 @@ def whl_library_targets_from_requires( requires_dist: {type}`list[str]` The list of `Requires-Dist` values from the whl `METADATA`. extras: {type}`list[str]` The list of requested extras. This essentially includes extra transitive dependencies in the final targets depending on the wheel `METADATA`. + entry_points: {type}`list[dict]` A list of parsed entry point definitions. include: {type}`list[str]` The list of packages to include. **kwargs: Extra args passed to the {obj}`whl_library_targets` """ @@ -82,6 +87,7 @@ def whl_library_targets_from_requires( name = name, dependencies = package_deps.deps, dependencies_with_markers = package_deps.deps_select, + entry_points = entry_points, tags = [ "pypi_name={}".format(metadata_name), "pypi_version={}".format(metadata_version), @@ -116,6 +122,7 @@ def whl_library_targets( filegroups = None, dependencies_by_platform = {}, dependencies_with_markers = {}, + entry_points = {}, group_deps = [], group_name = "", data = [], @@ -128,6 +135,8 @@ def whl_library_targets( copy_file = copy_file, py_binary = py_binary, py_library = py_library, + venv_entry_point = venv_entry_point, + venv_rewrite_shebang = venv_rewrite_shebang, env_marker_setting = env_marker_setting, create_inits = _create_inits, )): @@ -146,6 +155,7 @@ def whl_library_targets( dependencies by platform key. dependencies_with_markers: {type}`dict[str, str]` A marker to evaluate in order for the dep to be included. + entry_points: {type}`list[dict]` A list of parsed entry point definitions. filegroups: {type}`dict[str, list[str]] | None` A dictionary of the target names and the glob matches. If `None`, defaults will be used. group_name: {type}`str` name of the dependency group (if any) which @@ -180,6 +190,36 @@ def whl_library_targets( tags = sorted(tags) data = [] + data + bins_for_data_label = [] + + for ep_dict in entry_points.values(): + kwargs = dict(ep_dict) + ep_name = kwargs.pop("name") + ep_target_name = "bin/{}".format(ep_name) + rules.venv_entry_point( + name = ep_target_name, + **kwargs + ) + bins_for_data_label.append(ep_target_name) + data.append(ep_target_name) + + existing_bin_names = {ep["name"].lower(): None for ep in entry_points.values()} + for p in native.glob(["bin/*"], allow_empty = True): + existing_bin_names[p[len("bin/"):].lower()] = None + + for src_path in native.glob(["rewrite-bin/*"], allow_empty = True): + script_name = src_path[len("rewrite-bin/"):] + if script_name.lower() in existing_bin_names: + continue + rewrite_target_name = "bin/{}".format(script_name) + rules.venv_rewrite_shebang( + name = rewrite_target_name, + src = src_path, + package = name, + ) + bins_for_data_label.append(rewrite_target_name) + data.append(rewrite_target_name) + if filegroups == None: filegroups = { EXTRACTED_WHEEL_FILES: dict( @@ -199,9 +239,12 @@ def whl_library_targets( for filegroup_name, glob_kwargs in filegroups.items(): glob_kwargs = {"allow_empty": True} | glob_kwargs + srcs = native.glob(**glob_kwargs) + if filegroup_name == DATA_LABEL: + srcs = srcs + bins_for_data_label native.filegroup( name = filegroup_name, - srcs = native.glob(**glob_kwargs), + srcs = srcs, visibility = ["//visibility:public"], ) @@ -383,7 +426,7 @@ def whl_library_targets( ), tags = tags, visibility = impl_vis, - experimental_venvs_site_packages = Label("@rules_python//python/config_settings:venvs_site_packages"), + experimental_venvs_site_packages = _VENV_SITE_PACKAGES_FLAG, namespace_package_files = namespace_package_files, ) diff --git a/python/private/pypi/whl_metadata.bzl b/python/private/pypi/whl_metadata.bzl index 002e5773cc..2981a5d92f 100644 --- a/python/private/pypi/whl_metadata.bzl +++ b/python/private/pypi/whl_metadata.bzl @@ -111,3 +111,58 @@ def find_whl_metadata(*, install_dir, logger): else: logger.fail("The '*.dist-info' directory could not be found in '{}'".format(install_dir.basename)) return None + +def parse_entry_points(contents): + """Parses entry_points.txt contents and returns console_scripts and gui_scripts entries. + + Args: + contents: {type}`str` The contents of the entry_points.txt file. + + Returns: + {type}`dict[str, dict]` A dict keyed by the original entry point name. + """ + entries = {} + seen_lower_names = {} + current_group = None + current_group_lower = None + for line in contents.splitlines(): + line = line.strip() + if not line or line.startswith("#"): + continue + if line.startswith("[") and line.endswith("]"): + current_group = line[1:-1].strip() + current_group_lower = current_group.lower() + continue + + if current_group_lower in ("console_scripts", "gui_scripts"): + name, _, ref = line.partition("=") + name = name.strip() + + # Names are case-insensitive. + # See https://packaging.python.org/en/latest/specifications/entry-points/#data-model + # Entry points must be unique for a given name because they turn + # into files and may be on a case-insensitive file system. + lower_name = name.lower() + if lower_name in seen_lower_names: + continue + seen_lower_names[lower_name] = True + + # remove inline comments + ref, _, _ = ref.partition("#") + ref = ref.strip() + + extras = "" + if "[" in ref and ref.endswith("]"): + ref, _, extras_part = ref.partition("[") + extras = extras_part[:-1].strip() + ref = ref.strip() + + module, _, attribute = ref.partition(":") + entries[name] = { + "attribute": attribute.strip(), + "extras": extras, + "group": current_group, + "module": module.strip(), + "name": name, + } + return entries diff --git a/python/private/repo_utils.bzl b/python/private/repo_utils.bzl index 7ec45eda5b..ae2fc2e5d0 100644 --- a/python/private/repo_utils.bzl +++ b/python/private/repo_utils.bzl @@ -522,6 +522,38 @@ def _extract(mrctx, *, archive, supports_whl_extraction = False, **kwargs): if not mrctx.delete(archive): fail("Failed to remove the symlink after extracting") +def _rename(mrctx, src, dest): + """Rename a file or directory. + + TODO: remove when the earliest supported bazel version is at least 8.0. + + Args: + mrctx: module_ctx or repository_ctx object + src: {type}`path` the source path + dest: {type}`path` the destination path + """ + if hasattr(mrctx, "rename"): + mrctx.rename(src, dest) + return + + # Fallback for Bazel < 8.0 + os_name = _get_platforms_os_name(mrctx) + if os_name == "windows": + # On Windows, we use `cmd.exe /c move` to rename files/directories. + # We need to use backslashes for the paths. + res = mrctx.execute([ + "cmd.exe", + "/c", + "move", + str(src).replace("/", "\\"), + str(dest).replace("/", "\\"), + ]) + else: + res = mrctx.execute(["mv", str(src), str(dest)]) + + if res.return_code != 0: + fail("Failed to rename {} to {}: {}".format(src, dest, res.stderr)) + repo_utils = struct( # keep sorted execute_checked = _execute_checked, @@ -536,6 +568,7 @@ repo_utils = struct( norm_path = _norm_path, relative_to = _relative_to, is_relative_to = _is_relative_to, + rename = _rename, repo_root_relative_path = _repo_root_relative_path, which_checked = _which_checked, which_unchecked = _which_unchecked, diff --git a/python/private/text_util.bzl b/python/private/text_util.bzl index 28979d8981..eaccadf970 100644 --- a/python/private/text_util.bzl +++ b/python/private/text_util.bzl @@ -157,9 +157,33 @@ def _left_pad_zero(index, length): fail("index must be non-negative") return ("0" * length + str(index))[-length:] +def _render_dict_dict(d): + """Render a dict[str, dict] value without recursive function calls.""" + if not d: + return "{}" + + lines = ["{"] + for k, v in d.items(): + if not v: + v_str = "{}" + else: + inner_lines = ["{"] + for ik, iv in v.items(): + inner_lines.append(_indent("{}: {},".format(repr(ik), repr(iv)))) + inner_lines.append("}") + v_str = "\n".join(inner_lines) + + # We need to correctly indent the multi-line string v_str + # but _indent acts on every line except the first if not carefully handled. + # It's easier to just do: + lines.append(_indent("{}: {},".format(repr(k), v_str))) + lines.append("}") + return "\n".join(lines) + render = struct( alias = _render_alias, dict = _render_dict, + dict_dict = _render_dict_dict, call = _render_call, hanging_indent = _hanging_indent, indent = _indent, diff --git a/python/private/venv_runfiles.bzl b/python/private/venv_runfiles.bzl index a94f29f71c..45d4d24848 100644 --- a/python/private/venv_runfiles.bzl +++ b/python/private/venv_runfiles.bzl @@ -521,7 +521,7 @@ def get_venv_symlinks( venv_symlinks[venv_path] = VenvSymlinkEntry( kind = kind, link_to_path = link_to_path, - link_to_file = None, + link_to_file = files[0] if kind == VenvSymlinkKind.BIN and len(files) == 1 else None, package = package, version = version_str, venv_path = out_venv_path, diff --git a/tests/pypi/whl_library_targets/whl_library_targets_tests.bzl b/tests/pypi/whl_library_targets/whl_library_targets_tests.bzl index 60e1f3f3dd..ec28bfbb39 100644 --- a/tests/pypi/whl_library_targets/whl_library_targets_tests.bzl +++ b/tests/pypi/whl_library_targets/whl_library_targets_tests.bzl @@ -30,6 +30,8 @@ def _test_filegroups(env): def glob(include, *, exclude = [], allow_empty): _ = exclude # @unused env.expect.that_bool(allow_empty).equals(True) + if include == ["rewrite-bin/*"] or include == ["bin/*"]: + return [] return include whl_library_targets( @@ -39,7 +41,9 @@ def _test_filegroups(env): filegroup = lambda **kwargs: calls.append(kwargs), glob = glob, ), - rules = struct(), + rules = struct( + venv_rewrite_shebang = lambda **kwargs: None, + ), ) env.expect.that_collection(calls, expr = "filegroup calls").contains_exactly([ @@ -85,8 +89,11 @@ def _test_platforms(env): filegroups = {}, native = struct( config_setting = lambda **kwargs: calls.append(kwargs), + glob = lambda *args, **kwargs: [], + ), + rules = struct( + venv_rewrite_shebang = lambda **kwargs: None, ), - rules = struct(), ) env.expect.that_collection(calls).contains_exactly([ @@ -134,9 +141,12 @@ def _test_copy(env): filegroups = {}, copy_files = {"file_src": "file_dest"}, copy_executables = {"exec_src": "exec_dest"}, - native = struct(), + native = struct( + glob = lambda *args, **kwargs: [], + ), rules = struct( copy_file = lambda **kwargs: calls.append(kwargs), + venv_rewrite_shebang = lambda **kwargs: None, ), ) @@ -165,9 +175,11 @@ def _test_whl_and_library_deps_from_requires(env): m_glob = mocks.glob() - m_glob.results.append(["site-packages/foo/SRCS.py"]) - m_glob.results.append(["site-packages/foo/DATA.txt"]) - m_glob.results.append(["site-packages/foo/PYI.pyi"]) + m_glob.results.append([]) # bin + m_glob.results.append([]) # rewrite-bin + m_glob.results.append(["site-packages/foo/SRCS.py"]) # srcs + m_glob.results.append(["site-packages/foo/DATA.txt"]) # data + m_glob.results.append(["site-packages/foo/PYI.pyi"]) # pyi whl_library_targets_from_requires( name = "foo-0-py3-none-any.whl", @@ -193,6 +205,7 @@ def _test_whl_and_library_deps_from_requires(env): py_library = lambda **kwargs: py_library_calls.append(kwargs), env_marker_setting = lambda **kwargs: env_marker_setting_calls.append(kwargs), create_inits = lambda *args, **kwargs: ["_create_inits_target"], + venv_rewrite_shebang = lambda **kwargs: None, ), ) @@ -236,6 +249,16 @@ def _test_whl_and_library_deps_from_requires(env): }) # buildifier: @unsorted-dict-items env.expect.that_collection(m_glob.calls).contains_exactly([ + # bin call + mocks.glob_call( + ["bin/*"], + allow_empty = True, + ), + # rewrite-bin call + mocks.glob_call( + ["rewrite-bin/*"], + allow_empty = True, + ), # srcs call mocks.glob_call( ["site-packages/**/*.py"], @@ -271,6 +294,8 @@ def _test_whl_and_library_deps(env): filegroup_calls = [] py_library_calls = [] m_glob = mocks.glob() + m_glob.results.append([]) # bin + m_glob.results.append([]) # rewrite-bin m_glob.results.append(["site-packages/foo/SRCS.py"]) m_glob.results.append(["site-packages/foo/DATA.txt"]) m_glob.results.append(["site-packages/foo/PYI.pyi"]) @@ -300,6 +325,7 @@ def _test_whl_and_library_deps(env): rules = struct( py_library = lambda **kwargs: py_library_calls.append(kwargs), create_inits = lambda **kwargs: ["_create_inits_target"], + venv_rewrite_shebang = lambda **kwargs: None, ), ) @@ -369,6 +395,8 @@ def _test_group(env): py_library_calls = [] m_glob = mocks.glob() + m_glob.results.append([]) # bin + m_glob.results.append([]) # rewrite-bin m_glob.results.append(["site-packages/foo/srcs.py"]) m_glob.results.append(["site-packages/foo/data.txt"]) m_glob.results.append(["site-packages/foo/pyi.pyi"]) @@ -396,6 +424,7 @@ def _test_group(env): rules = struct( py_library = lambda **kwargs: py_library_calls.append(kwargs), create_inits = lambda **kwargs: ["_create_inits_target"], + venv_rewrite_shebang = lambda **kwargs: None, ), ) @@ -435,6 +464,8 @@ def _test_group(env): }) # buildifier: @unsorted-dict-items env.expect.that_collection(m_glob.calls, expr = "glob calls").contains_exactly([ + mocks.glob_call(["bin/*"], allow_empty = True), + mocks.glob_call(["rewrite-bin/*"], allow_empty = True), mocks.glob_call(["site-packages/**/*.py"], exclude = [], allow_empty = True), mocks.glob_call(["site-packages/**/*"], exclude = [ "**/*.py", diff --git a/tests/pypi/whl_metadata/whl_metadata_tests.bzl b/tests/pypi/whl_metadata/whl_metadata_tests.bzl index 329423a26c..8131b0f452 100644 --- a/tests/pypi/whl_metadata/whl_metadata_tests.bzl +++ b/tests/pypi/whl_metadata/whl_metadata_tests.bzl @@ -5,6 +5,7 @@ load("@rules_testing//lib:truth.bzl", "subjects") load( "//python/private/pypi:whl_metadata.bzl", "find_whl_metadata", + "parse_entry_points", "parse_whl_metadata", ) # buildifier: disable=bzl-visibility @@ -171,6 +172,80 @@ Requires-Dist: this will be ignored _tests.append(_test_parse_metadata_multiline_license) +def _test_parse_entry_points(env): + got = parse_entry_points("""\ +[something] +interesting # with comments + +[console_scripts] +foo = foomod:main +# One which depends on extras: +foobar = importable.foomod:main_bar [bar, baz] + + # With a comment at the end +foobarbaz = foomod:main.attr # comment + +# With extra and comment +foo_extra_comment = foomod:main [extra] # comment + +[something else] +not very much interesting +""") + env.expect.that_dict(got).contains_exactly({ + "foo": { + "attribute": "main", + "extras": "", + "group": "console_scripts", + "module": "foomod", + "name": "foo", + }, + "foo_extra_comment": { + "attribute": "main", + "extras": "extra", + "group": "console_scripts", + "module": "foomod", + "name": "foo_extra_comment", + }, + "foobar": { + "attribute": "main_bar", + "extras": "bar, baz", + "group": "console_scripts", + "module": "importable.foomod", + "name": "foobar", + }, + "foobarbaz": { + "attribute": "main.attr", + "extras": "", + "group": "console_scripts", + "module": "foomod", + "name": "foobarbaz", + }, + }) + +_tests.append(_test_parse_entry_points) + +def _test_parse_entry_points_deduplicate(env): + got = parse_entry_points("""\ +[console_scripts] +FooBar = foomod:main +foobar = othermod:main +fooBAR = another:main + +[gui_scripts] +FOOBAR = guimod:main +""") + env.expect.that_dict(got).contains_exactly({ + "FooBar": { + "attribute": "main", + "extras": "", + "group": "console_scripts", + "module": "foomod", + "name": "FooBar", + }, + }) + +_tests.append(_test_parse_entry_points_deduplicate) + def whl_metadata_test_suite(name): # buildifier: disable=function-docstring test_suite( name = name, diff --git a/tests/repos/whl_with_data1/BUILD.bazel b/tests/repos/whl_with_data1/BUILD.bazel index af49d1ebbf..7ef8ba4cd9 100644 --- a/tests/repos/whl_with_data1/BUILD.bazel +++ b/tests/repos/whl_with_data1/BUILD.bazel @@ -1 +1 @@ -exports_files(glob(["*"])) +exports_files(glob(["**"])) diff --git a/tests/repos/whl_with_data1/whl_with_data1-1.0.data/scripts/whl_with_data1_pythonw b/tests/repos/whl_with_data1/whl_with_data1-1.0.data/scripts/whl_with_data1_pythonw new file mode 100755 index 0000000000..6c7b3434c5 --- /dev/null +++ b/tests/repos/whl_with_data1/whl_with_data1-1.0.data/scripts/whl_with_data1_pythonw @@ -0,0 +1,9 @@ +#!pythonw +import sys + +# On Windows, pythonw doesn't have stdout/stderr streams, +# so output has to be written to a file. +with open(sys.argv[1], "w") as fp: + fp.write("hello from whl_with_data1_pythonw\n") + fp.write(sys.executable) + fp.write("\n") diff --git a/tests/repos/whl_with_data1/whl_with_data1-1.0.data/scripts/whl_with_data1_script b/tests/repos/whl_with_data1/whl_with_data1-1.0.data/scripts/whl_with_data1_script new file mode 100755 index 0000000000..8af40a9a55 --- /dev/null +++ b/tests/repos/whl_with_data1/whl_with_data1-1.0.data/scripts/whl_with_data1_script @@ -0,0 +1,5 @@ +#!python +import sys + +print("hello from whl_with_data1_script") +print(sys.executable) diff --git a/tests/repos/whl_with_data1/whl_with_data1-1.0.dist-info/RECORD b/tests/repos/whl_with_data1/whl_with_data1-1.0.dist-info/RECORD index a39e9ed7ad..10307c76a0 100644 --- a/tests/repos/whl_with_data1/whl_with_data1-1.0.dist-info/RECORD +++ b/tests/repos/whl_with_data1/whl_with_data1-1.0.dist-info/RECORD @@ -1,4 +1,5 @@ whl_with_data1-1.0.data/platlib/whl_with_data1/platlib_file.txt,sha256=123,123 +whl_with_data1-1.0.data/scripts/whl_with_data1_script,sha256=123,123 whl_with_data1-1.0.data/scripts/whl_script.sh,sha256=123,123 whl_with_data1-1.0.data/headers/whl_with_data1/header_file.h,sha256=123,123 whl_with_data1-1.0.data/purelib/whl_with_data1/data_file.txt,sha256=123,123 diff --git a/tests/repos/whl_with_data2/whl_with_data2-1.0.data/purelib/whl_with_data2/__init__.py b/tests/repos/whl_with_data2/whl_with_data2-1.0.data/purelib/whl_with_data2/__init__.py index e69de29bb2..45132c14d7 100644 --- a/tests/repos/whl_with_data2/whl_with_data2-1.0.data/purelib/whl_with_data2/__init__.py +++ b/tests/repos/whl_with_data2/whl_with_data2-1.0.data/purelib/whl_with_data2/__init__.py @@ -0,0 +1,6 @@ +import sys + + +def main(): + print("hello from whl_with_data2_bin") + print(sys.executable) diff --git a/tests/repos/whl_with_data2/whl_with_data2-1.0.dist-info/RECORD b/tests/repos/whl_with_data2/whl_with_data2-1.0.dist-info/RECORD index 5eeb915ba7..55c70740c8 100644 --- a/tests/repos/whl_with_data2/whl_with_data2-1.0.dist-info/RECORD +++ b/tests/repos/whl_with_data2/whl_with_data2-1.0.dist-info/RECORD @@ -10,3 +10,5 @@ whl_with_data2-1.0.data/scripts/overlap/both.sh,sha256=123,123 whl_with_data2-1.0.data/scripts/overlap/script2.sh,sha256=123,123 whl_with_data2-1.0.data/headers/overlap/both.h,sha256=123,123 whl_with_data2-1.0.data/headers/overlap/header2.h,sha256=123,123 +whl_with_data2-1.0.data/purelib/whl_with_data2/__init__.py,sha256=123,123 +whl_with_data2-1.0.dist-info/entry_points.txt,sha256=123,123 diff --git a/tests/repos/whl_with_data2/whl_with_data2-1.0.dist-info/entry_points.txt b/tests/repos/whl_with_data2/whl_with_data2-1.0.dist-info/entry_points.txt new file mode 100644 index 0000000000..8389a8a826 --- /dev/null +++ b/tests/repos/whl_with_data2/whl_with_data2-1.0.dist-info/entry_points.txt @@ -0,0 +1,2 @@ +[console_scripts] +whl_with_data2_bin = whl_with_data2:main diff --git a/tests/venv_site_packages_libs/BUILD.bazel b/tests/venv_site_packages_libs/BUILD.bazel index c99426b375..256c4f24b5 100644 --- a/tests/venv_site_packages_libs/BUILD.bazel +++ b/tests/venv_site_packages_libs/BUILD.bazel @@ -1,3 +1,4 @@ +load("@rules_python_internal//:rules_python_config.bzl", rp_config = "config") load("@rules_shell//shell:sh_test.bzl", "sh_test") load("//python:py_library.bzl", "py_library") load("//tests/support:py_reconfig.bzl", "py_reconfig_test") @@ -70,3 +71,21 @@ py_reconfig_test( ], }), ) + +py_reconfig_test( + name = "whl_scripts_runnable_test", + srcs = ["whl_scripts_runnable_test.py"], + bootstrap_impl = select({ + "@platforms//os:windows": "system_python", + "//conditions:default": "script", + }), + env = { + "BAZEL_8_OR_LATER": "1" if rp_config.bazel_8_or_later else "0", + }, + main = "whl_scripts_runnable_test.py", + venvs_site_packages = "yes", + deps = [ + "@whl_with_data1//:pkg", + "@whl_with_data2//:pkg", + ], +) diff --git a/tests/venv_site_packages_libs/whl_scripts_runnable_test.py b/tests/venv_site_packages_libs/whl_scripts_runnable_test.py new file mode 100644 index 0000000000..61b477f493 --- /dev/null +++ b/tests/venv_site_packages_libs/whl_scripts_runnable_test.py @@ -0,0 +1,122 @@ +import os +import subprocess +import sys +import tempfile +import unittest +from pathlib import Path + +BAZEL_8_OR_LATER = bool(int(os.environ.get("BAZEL_8_OR_LATER", "0"))) + + +class WhlScriptsRunnableTest(unittest.TestCase): + maxDiff = None + + def _get_script_path(self, name): + is_windows = sys.platform == "win32" + if is_windows: + bin_dir = Path(sys.prefix) / "Scripts" + pathexts = os.environ.get("PATHEXT", ".COM;.EXE;.BAT;.CMD").split(";") + for ext in [""] + [e.lower() for e in pathexts]: + script_path = bin_dir / f"{name}{ext}" + if script_path.exists(): + return script_path + return bin_dir / name + else: + bin_dir = Path(sys.prefix) / "bin" + script_path = bin_dir / name + return script_path + + def test_script_is_runnable(self): + script_path = self._get_script_path("whl_with_data1_script") + self.assertTrue(script_path.exists(), f"Script not found at {script_path}") + + result = subprocess.run( + [str(script_path)], + capture_output=True, + text=True, + check=True, + ) + + output = result.stdout.splitlines() + self.assertIn("hello from whl_with_data1_script", output) + + # The script prints sys.executable as its second line + # Depending on how it's invoked, it might have more output, + # but the user said it prints the hello message AND sys.executable. + script_executable = output[-1].strip() + self.assertEqual(script_executable, sys.executable) + + def test_entry_point_is_runnable(self): + script_path = self._get_script_path("whl_with_data2_bin") + self.assertTrue(script_path.exists(), f"Entry point not found at {script_path}") + + result = subprocess.run( + [str(script_path)], + capture_output=True, + text=True, + check=True, + ) + + output = result.stdout.splitlines() + self.assertIn("hello from whl_with_data2_bin", output) + + script_executable = output[-1].strip() + self.assertEqual(script_executable, sys.executable) + + # This should really check for 8.5 instead of 8+, but we test with 8.6 + # so it's close enough for our purposes. + @unittest.skipUnless( + BAZEL_8_OR_LATER, + "bazel 8.5 and lower uses wheel.py, which rewrites #!pythonw to #!python", + ) + def test_pythonw_script(self): + script_path = self._get_script_path("whl_with_data1_pythonw") + self.assertTrue(script_path.exists(), f"Script not found at {script_path}") + + with open(script_path, "r", encoding="utf-8") as f: + first_line = f.readline() + + is_windows = sys.platform == "win32" + if is_windows: + # On Windows, the shebang is replaced with a batch wrapper that + # invokes the interpreter. + self.assertIn("pythonw.exe", first_line) + self.assertTrue( + first_line.startswith("@setlocal") + or first_line.startswith("@echo off"), + f"Expected Windows batch wrapper, got {first_line}", + ) + else: + self.assertTrue( + first_line.startswith("#!/bin/sh"), + f"Expected #!/bin/sh, got {first_line}", + ) + + # For some reason, on Windows, the subprocess can't write + # to the temporary files unless mkstemp is used. + temp_fd, temp_str = tempfile.mkstemp() + try: + os.close(temp_fd) + out_path = Path(temp_str) + result = subprocess.run( + [str(script_path), str(out_path)], + capture_output=True, + text=True, + check=True, + ) + output = out_path.read_text().splitlines() + finally: + os.unlink(temp_str) + self.assertIn("hello from whl_with_data1_pythonw", output) + + script_executable = output[-1].strip() + + if is_windows: + self.assertTrue( + script_executable.endswith("pythonw.exe"), + f"Expected pythonw.exe, got {script_executable}", + ) + + +if __name__ == "__main__": + unittest.main() From b87cf67e0b56e50fb62aa973e3cf43113e825518 Mon Sep 17 00:00:00 2001 From: Thomas Marlowe <13261878+canislupaster@users.noreply.github.com> Date: Thu, 30 Apr 2026 14:09:51 -0700 Subject: [PATCH 729/922] feat: Add alias_kind directive support to Gazelle plugin. (#3713) Currently the collision detection prevents Gazelle from respecting `alias_kind`. This PR cleans up the behavior by referencing the configured alias and map kind instead of storing a bunch of mapped `actualPy*Kind` vars. I've added two tests heavily based on `respect_map_kind`: one with just `alias_kind`, and one using both `map_kind` and `alias_kind`. Fixes #3183 . --------- Co-authored-by: Ignas Anikevicius <240938+aignas@users.noreply.github.com> --- CHANGELOG.md | 2 + gazelle/python/generate.go | 49 ++++++++++--------- .../naming_convention_mapped_fail/BUILD.in | 9 ++++ .../naming_convention_mapped_fail/BUILD.out | 9 ++++ .../naming_convention_mapped_fail/README.md | 4 ++ .../naming_convention_mapped_fail/WORKSPACE | 1 + .../naming_convention_mapped_fail/__init__.py | 1 + .../naming_convention_mapped_fail/__main__.py | 1 + .../naming_convention_mapped_fail/__test__.py | 1 + .../naming_convention_mapped_fail/test.yaml | 7 +++ .../respect_alias_and_map_kind/BUILD.in | 16 ++++++ .../respect_alias_and_map_kind/BUILD.out | 21 ++++++++ .../respect_alias_and_map_kind/README.md | 4 ++ .../respect_alias_and_map_kind/WORKSPACE | 1 + .../respect_alias_and_map_kind/__init__.py | 3 ++ .../respect_alias_and_map_kind/__test__.py | 12 +++++ .../respect_alias_and_map_kind/foo.py | 17 +++++++ .../respect_alias_and_map_kind/test.yaml | 3 ++ .../testdata/respect_alias_kind/BUILD.in | 8 +++ .../testdata/respect_alias_kind/BUILD.out | 12 +++++ .../testdata/respect_alias_kind/README.md | 4 ++ .../testdata/respect_alias_kind/WORKSPACE | 1 + .../testdata/respect_alias_kind/__init__.py | 3 ++ .../python/testdata/respect_alias_kind/foo.py | 2 + .../testdata/respect_alias_kind/test.yaml | 3 ++ 25 files changed, 171 insertions(+), 23 deletions(-) create mode 100644 gazelle/python/testdata/naming_convention_mapped_fail/BUILD.in create mode 100644 gazelle/python/testdata/naming_convention_mapped_fail/BUILD.out create mode 100644 gazelle/python/testdata/naming_convention_mapped_fail/README.md create mode 100644 gazelle/python/testdata/naming_convention_mapped_fail/WORKSPACE create mode 100644 gazelle/python/testdata/naming_convention_mapped_fail/__init__.py create mode 100644 gazelle/python/testdata/naming_convention_mapped_fail/__main__.py create mode 100644 gazelle/python/testdata/naming_convention_mapped_fail/__test__.py create mode 100644 gazelle/python/testdata/naming_convention_mapped_fail/test.yaml create mode 100644 gazelle/python/testdata/respect_alias_and_map_kind/BUILD.in create mode 100644 gazelle/python/testdata/respect_alias_and_map_kind/BUILD.out create mode 100644 gazelle/python/testdata/respect_alias_and_map_kind/README.md create mode 100644 gazelle/python/testdata/respect_alias_and_map_kind/WORKSPACE create mode 100644 gazelle/python/testdata/respect_alias_and_map_kind/__init__.py create mode 100644 gazelle/python/testdata/respect_alias_and_map_kind/__test__.py create mode 100644 gazelle/python/testdata/respect_alias_and_map_kind/foo.py create mode 100644 gazelle/python/testdata/respect_alias_and_map_kind/test.yaml create mode 100644 gazelle/python/testdata/respect_alias_kind/BUILD.in create mode 100644 gazelle/python/testdata/respect_alias_kind/BUILD.out create mode 100644 gazelle/python/testdata/respect_alias_kind/README.md create mode 100644 gazelle/python/testdata/respect_alias_kind/WORKSPACE create mode 100644 gazelle/python/testdata/respect_alias_kind/__init__.py create mode 100644 gazelle/python/testdata/respect_alias_kind/foo.py create mode 100644 gazelle/python/testdata/respect_alias_kind/test.yaml diff --git a/CHANGELOG.md b/CHANGELOG.md index 6317404f66..dc45d2e95e 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -85,6 +85,8 @@ END_UNRELEASED_TEMPLATE wheel `data`, `bin`, and `include` files are populated into the venv. * (runfiles) Added a pathlib-compatible API: {obj}`Runfiles.root()` Fixes [#3296](https://github.com/bazel-contrib/rules_python/issues/3296). +* (gazelle) Support alias_kind directive. + Fixes [#3183](https://github.com/bazel-contrib/rules_python/issues/3183). * (toolchains) `3.13.12`, `3.14.3` Python toolchain from [20260325] release. * (toolchains) `3.10.20`, `3.11.15`, `3.12.13`, `3.13.13` `3.14.4`, `3.15.0a8` * Python toolchain from [20260414] release. diff --git a/gazelle/python/generate.go b/gazelle/python/generate.go index a13d7182e9..6fa6252ebb 100644 --- a/gazelle/python/generate.go +++ b/gazelle/python/generate.go @@ -48,13 +48,21 @@ var ( buildFilenames = []string{"BUILD", "BUILD.bazel"} ) -func GetActualKindName(kind string, args language.GenerateArgs) string { - if kindOverride, ok := args.Config.KindMap[kind]; ok { - return kindOverride.KindName +// Returns the mapped kind, or kind if no mapping is configured with the map_kind directive. +func getMappedKind(c *config.Config, kind string) string { + if mapped, ok := c.KindMap[kind]; ok { + return mapped.KindName } return kind } +// kindMatches returns whether r matches the canonical Python rule kind `expected`, respecting `# gazelle:map_kind` and +// `# gazelle:alias_kind` directives in the config.Config c. +func kindMatches(c *config.Config, r *rule.Rule, expected string) bool { + kind := r.Kind() + return kind == getMappedKind(c, expected) || c.AliasMap[kind] == expected +} + func matchesAnyGlob(s string, globs []string) bool { // This function assumes that the globs have already been validated. If a glob is // invalid, it's considered a non-match and we move on to the next pattern. @@ -112,10 +120,6 @@ func (py *Python) GenerateRules(args language.GenerateArgs) language.GenerateRes } } - actualPyBinaryKind := GetActualKindName(pyBinaryKind, args) - actualPyLibraryKind := GetActualKindName(pyLibraryKind, args) - actualPyTestKind := GetActualKindName(pyTestKind, args) - pythonProjectRoot := cfg.PythonProjectRoot() packageName := filepath.Base(args.Dir) @@ -296,10 +300,10 @@ func (py *Python) GenerateRules(args language.GenerateArgs) language.GenerateRes sort.Strings(mainFileNames) for _, filename := range mainFileNames { pyBinaryTargetName := strings.TrimSuffix(filepath.Base(filename), ".py") - if err := ensureNoCollision(args.File, pyBinaryTargetName, actualPyBinaryKind); err != nil { + if err := ensureNoCollision(args.Config, args.File, pyBinaryTargetName, pyBinaryKind); err != nil { fqTarget := label.New("", args.Rel, pyBinaryTargetName) log.Printf("failed to generate target %q of kind %q: %v", - fqTarget.String(), actualPyBinaryKind, err) + fqTarget.String(), getMappedKind(args.Config, pyBinaryKind), err) continue } @@ -334,7 +338,7 @@ func (py *Python) GenerateRules(args language.GenerateArgs) language.GenerateRes } generateEmptyLibrary := false for _, r := range args.File.Rules { - if r.Kind() == actualPyLibraryKind && r.Name() == pyLibraryTargetName { + if r.Name() == pyLibraryTargetName && kindMatches(args.Config, r, pyLibraryKind) { generateEmptyLibrary = true } } @@ -350,11 +354,11 @@ func (py *Python) GenerateRules(args language.GenerateArgs) language.GenerateRes // exists, and if it is of a different kind from the one we are // generating. If so, we have to throw an error since Gazelle won't // generate it correctly. - if err := ensureNoCollision(args.File, pyLibraryTargetName, actualPyLibraryKind); err != nil { + if err := ensureNoCollision(args.Config, args.File, pyLibraryTargetName, pyLibraryKind); err != nil { fqTarget := label.New("", args.Rel, pyLibraryTargetName) err := fmt.Errorf("failed to generate target %q of kind %q: %w. "+ "Use the '# gazelle:%s' directive to change the naming convention.", - fqTarget.String(), actualPyLibraryKind, err, pythonconfig.LibraryNamingConvention) + fqTarget.String(), getMappedKind(args.Config, pyLibraryKind), err, pythonconfig.LibraryNamingConvention) collisionErrors.Add(err) } @@ -404,11 +408,11 @@ func (py *Python) GenerateRules(args language.GenerateArgs) language.GenerateRes // exists, and if it is of a different kind from the one we are // generating. If so, we have to throw an error since Gazelle won't // generate it correctly. - if err := ensureNoCollision(args.File, pyBinaryTargetName, actualPyBinaryKind); err != nil { + if err := ensureNoCollision(args.Config, args.File, pyBinaryTargetName, pyBinaryKind); err != nil { fqTarget := label.New("", args.Rel, pyBinaryTargetName) err := fmt.Errorf("failed to generate target %q of kind %q: %w. "+ "Use the '# gazelle:%s' directive to change the naming convention.", - fqTarget.String(), actualPyBinaryKind, err, pythonconfig.BinaryNamingConvention) + fqTarget.String(), getMappedKind(args.Config, pyBinaryKind), err, pythonconfig.BinaryNamingConvention) collisionErrors.Add(err) } @@ -443,10 +447,10 @@ func (py *Python) GenerateRules(args language.GenerateArgs) language.GenerateRes // exists, and if it is of a different kind from the one we are // generating. If so, we have to throw an error since Gazelle won't // generate it correctly. - if err := ensureNoCollision(args.File, conftestTargetname, actualPyLibraryKind); err != nil { + if err := ensureNoCollision(args.Config, args.File, conftestTargetname, pyLibraryKind); err != nil { fqTarget := label.New("", args.Rel, conftestTargetname) err := fmt.Errorf("failed to generate target %q of kind %q: %w. ", - fqTarget.String(), actualPyLibraryKind, err) + fqTarget.String(), getMappedKind(args.Config, pyLibraryKind), err) collisionErrors.Add(err) } @@ -480,11 +484,11 @@ func (py *Python) GenerateRules(args language.GenerateArgs) language.GenerateRes // exists, and if it is of a different kind from the one we are // generating. If so, we have to throw an error since Gazelle won't // generate it correctly. - if err := ensureNoCollision(args.File, pyTestTargetName, actualPyTestKind); err != nil { + if err := ensureNoCollision(args.Config, args.File, pyTestTargetName, pyTestKind); err != nil { fqTarget := label.New("", args.Rel, pyTestTargetName) err := fmt.Errorf("failed to generate target %q of kind %q: %w. "+ "Use the '# gazelle:%s' directive to change the naming convention.", - fqTarget.String(), actualPyTestKind, err, pythonconfig.TestNamingConvention) + fqTarget.String(), getMappedKind(args.Config, pyTestKind), err, pythonconfig.TestNamingConvention) collisionErrors.Add(err) } @@ -593,8 +597,7 @@ func (py *Python) getRulesWithInvalidSrcs(args language.GenerateArgs, validFiles return strings.HasPrefix(src, "@") || strings.HasPrefix(src, "//") || strings.HasPrefix(src, ":") } for _, existingRule := range args.File.Rules { - actualPyBinaryKind := GetActualKindName(pyBinaryKind, args) - if existingRule.Kind() != actualPyBinaryKind { + if !kindMatches(args.Config, existingRule, pyBinaryKind) { continue } var hasValidSrcs bool @@ -670,12 +673,12 @@ func isEntrypointFile(path string) bool { } } -func ensureNoCollision(file *rule.File, targetName, kind string) error { +func ensureNoCollision(c *config.Config, file *rule.File, targetName, kind string) error { if file == nil { return nil } for _, t := range file.Rules { - if t.Name() == targetName && t.Kind() != kind { + if t.Name() == targetName && !kindMatches(c, t, kind) { return fmt.Errorf("a target of kind %q with the same name already exists", t.Kind()) } } @@ -698,7 +701,7 @@ func generateProtoLibraries(args language.GenerateArgs, cfg *pythonconfig.Config pyProtoRulesForProto := map[string]string{} if args.File != nil { for _, r := range args.File.Rules { - if r.Kind() == "py_proto_library" { + if kindMatches(args.Config, r, pyProtoLibraryKind) { pyProtoRules[r.Name()] = false protos := r.AttrStrings("deps") diff --git a/gazelle/python/testdata/naming_convention_mapped_fail/BUILD.in b/gazelle/python/testdata/naming_convention_mapped_fail/BUILD.in new file mode 100644 index 0000000000..63ca6b805f --- /dev/null +++ b/gazelle/python/testdata/naming_convention_mapped_fail/BUILD.in @@ -0,0 +1,9 @@ +# gazelle:map_kind py_library my_lib :mytest.bzl +# gazelle:map_kind py_binary my_bin :mytest.bzl +# gazelle:map_kind py_test my_test :mytest.bzl + +py_library(name = "naming_convention_mapped_fail") + +py_binary(name = "naming_convention_mapped_fail_bin") + +py_test(name = "naming_convention_mapped_fail_test") diff --git a/gazelle/python/testdata/naming_convention_mapped_fail/BUILD.out b/gazelle/python/testdata/naming_convention_mapped_fail/BUILD.out new file mode 100644 index 0000000000..63ca6b805f --- /dev/null +++ b/gazelle/python/testdata/naming_convention_mapped_fail/BUILD.out @@ -0,0 +1,9 @@ +# gazelle:map_kind py_library my_lib :mytest.bzl +# gazelle:map_kind py_binary my_bin :mytest.bzl +# gazelle:map_kind py_test my_test :mytest.bzl + +py_library(name = "naming_convention_mapped_fail") + +py_binary(name = "naming_convention_mapped_fail_bin") + +py_test(name = "naming_convention_mapped_fail_test") diff --git a/gazelle/python/testdata/naming_convention_mapped_fail/README.md b/gazelle/python/testdata/naming_convention_mapped_fail/README.md new file mode 100644 index 0000000000..85a561c575 --- /dev/null +++ b/gazelle/python/testdata/naming_convention_mapped_fail/README.md @@ -0,0 +1,4 @@ +# Naming convention fail with `map_kind` + +This test case asserts that collision error messages reference the mapped rule +kind over than the canonical Python kind when using `# gazelle:map_kind`. diff --git a/gazelle/python/testdata/naming_convention_mapped_fail/WORKSPACE b/gazelle/python/testdata/naming_convention_mapped_fail/WORKSPACE new file mode 100644 index 0000000000..faff6af87a --- /dev/null +++ b/gazelle/python/testdata/naming_convention_mapped_fail/WORKSPACE @@ -0,0 +1 @@ +# This is a Bazel workspace for the Gazelle test data. diff --git a/gazelle/python/testdata/naming_convention_mapped_fail/__init__.py b/gazelle/python/testdata/naming_convention_mapped_fail/__init__.py new file mode 100644 index 0000000000..a75c47faaf --- /dev/null +++ b/gazelle/python/testdata/naming_convention_mapped_fail/__init__.py @@ -0,0 +1 @@ +# Empty test file. \ No newline at end of file diff --git a/gazelle/python/testdata/naming_convention_mapped_fail/__main__.py b/gazelle/python/testdata/naming_convention_mapped_fail/__main__.py new file mode 100644 index 0000000000..a75c47faaf --- /dev/null +++ b/gazelle/python/testdata/naming_convention_mapped_fail/__main__.py @@ -0,0 +1 @@ +# Empty test file. \ No newline at end of file diff --git a/gazelle/python/testdata/naming_convention_mapped_fail/__test__.py b/gazelle/python/testdata/naming_convention_mapped_fail/__test__.py new file mode 100644 index 0000000000..a75c47faaf --- /dev/null +++ b/gazelle/python/testdata/naming_convention_mapped_fail/__test__.py @@ -0,0 +1 @@ +# Empty test file. \ No newline at end of file diff --git a/gazelle/python/testdata/naming_convention_mapped_fail/test.yaml b/gazelle/python/testdata/naming_convention_mapped_fail/test.yaml new file mode 100644 index 0000000000..1bdf1acbc5 --- /dev/null +++ b/gazelle/python/testdata/naming_convention_mapped_fail/test.yaml @@ -0,0 +1,7 @@ +--- +expect: + exit_code: 1 + stderr: | + gazelle: ERROR: failed to generate target "//:naming_convention_mapped_fail" of kind "my_lib": a target of kind "py_library" with the same name already exists. Use the '# gazelle:python_library_naming_convention' directive to change the naming convention. + gazelle: ERROR: failed to generate target "//:naming_convention_mapped_fail_bin" of kind "my_bin": a target of kind "py_binary" with the same name already exists. Use the '# gazelle:python_binary_naming_convention' directive to change the naming convention. + gazelle: ERROR: failed to generate target "//:naming_convention_mapped_fail_test" of kind "my_test": a target of kind "py_test" with the same name already exists. Use the '# gazelle:python_test_naming_convention' directive to change the naming convention. diff --git a/gazelle/python/testdata/respect_alias_and_map_kind/BUILD.in b/gazelle/python/testdata/respect_alias_and_map_kind/BUILD.in new file mode 100644 index 0000000000..06ebfbdfa3 --- /dev/null +++ b/gazelle/python/testdata/respect_alias_and_map_kind/BUILD.in @@ -0,0 +1,16 @@ +load(":mylib.bzl", "my_py_library") + +# gazelle:alias_kind my_py_library py_library +# gazelle:map_kind py_test my_test :mytest.bzl + +my_py_library( + name = "respect_alias_and_map_kind", + srcs = ["__init__.py"], +) + +my_test( + name = "respect_alias_and_map_kind_test", + srcs = ["__test__.py"], + main = "__test__.py", + deps = [], +) diff --git a/gazelle/python/testdata/respect_alias_and_map_kind/BUILD.out b/gazelle/python/testdata/respect_alias_and_map_kind/BUILD.out new file mode 100644 index 0000000000..6c7eee06e3 --- /dev/null +++ b/gazelle/python/testdata/respect_alias_and_map_kind/BUILD.out @@ -0,0 +1,21 @@ +load(":mylib.bzl", "my_py_library") +load(":mytest.bzl", "my_test") + +# gazelle:alias_kind my_py_library py_library +# gazelle:map_kind py_test my_test :mytest.bzl + +my_py_library( + name = "respect_alias_and_map_kind", + srcs = [ + "__init__.py", + "foo.py", + ], + visibility = ["//:__subpackages__"], +) + +my_test( + name = "respect_alias_and_map_kind_test", + srcs = ["__test__.py"], + main = "__test__.py", + deps = [":respect_alias_and_map_kind"], +) diff --git a/gazelle/python/testdata/respect_alias_and_map_kind/README.md b/gazelle/python/testdata/respect_alias_and_map_kind/README.md new file mode 100644 index 0000000000..5e5ca20d27 --- /dev/null +++ b/gazelle/python/testdata/respect_alias_and_map_kind/README.md @@ -0,0 +1,4 @@ +# Respect `alias_kind` and `map_kind` + +This test case asserts that `# gazelle:alias_kind` and `# gazelle:map_kind` are +both respected as the expected kind at once and do not collide. diff --git a/gazelle/python/testdata/respect_alias_and_map_kind/WORKSPACE b/gazelle/python/testdata/respect_alias_and_map_kind/WORKSPACE new file mode 100644 index 0000000000..faff6af87a --- /dev/null +++ b/gazelle/python/testdata/respect_alias_and_map_kind/WORKSPACE @@ -0,0 +1 @@ +# This is a Bazel workspace for the Gazelle test data. diff --git a/gazelle/python/testdata/respect_alias_and_map_kind/__init__.py b/gazelle/python/testdata/respect_alias_and_map_kind/__init__.py new file mode 100644 index 0000000000..6a49193fe4 --- /dev/null +++ b/gazelle/python/testdata/respect_alias_and_map_kind/__init__.py @@ -0,0 +1,3 @@ +from foo import foo + +_ = foo diff --git a/gazelle/python/testdata/respect_alias_and_map_kind/__test__.py b/gazelle/python/testdata/respect_alias_and_map_kind/__test__.py new file mode 100644 index 0000000000..d6085a41b4 --- /dev/null +++ b/gazelle/python/testdata/respect_alias_and_map_kind/__test__.py @@ -0,0 +1,12 @@ +import unittest + +from __init__ import foo + + +class FooTest(unittest.TestCase): + def test_foo(self): + self.assertEqual("foo", foo()) + + +if __name__ == "__main__": + unittest.main() diff --git a/gazelle/python/testdata/respect_alias_and_map_kind/foo.py b/gazelle/python/testdata/respect_alias_and_map_kind/foo.py new file mode 100644 index 0000000000..3f049df738 --- /dev/null +++ b/gazelle/python/testdata/respect_alias_and_map_kind/foo.py @@ -0,0 +1,17 @@ +# Copyright 2023 The Bazel Authors. All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + + +def foo(): + return "foo" diff --git a/gazelle/python/testdata/respect_alias_and_map_kind/test.yaml b/gazelle/python/testdata/respect_alias_and_map_kind/test.yaml new file mode 100644 index 0000000000..36dd656b39 --- /dev/null +++ b/gazelle/python/testdata/respect_alias_and_map_kind/test.yaml @@ -0,0 +1,3 @@ +--- +expect: + exit_code: 0 diff --git a/gazelle/python/testdata/respect_alias_kind/BUILD.in b/gazelle/python/testdata/respect_alias_kind/BUILD.in new file mode 100644 index 0000000000..3dc5ca7151 --- /dev/null +++ b/gazelle/python/testdata/respect_alias_kind/BUILD.in @@ -0,0 +1,8 @@ +load(":mylib.bzl", "my_py_library") + +# gazelle:alias_kind my_py_library py_library + +my_py_library( + name = "respect_alias_kind", + srcs = ["__init__.py"], +) diff --git a/gazelle/python/testdata/respect_alias_kind/BUILD.out b/gazelle/python/testdata/respect_alias_kind/BUILD.out new file mode 100644 index 0000000000..9e65a95923 --- /dev/null +++ b/gazelle/python/testdata/respect_alias_kind/BUILD.out @@ -0,0 +1,12 @@ +load(":mylib.bzl", "my_py_library") + +# gazelle:alias_kind my_py_library py_library + +my_py_library( + name = "respect_alias_kind", + srcs = [ + "__init__.py", + "foo.py", + ], + visibility = ["//:__subpackages__"], +) diff --git a/gazelle/python/testdata/respect_alias_kind/README.md b/gazelle/python/testdata/respect_alias_kind/README.md new file mode 100644 index 0000000000..d0d8702106 --- /dev/null +++ b/gazelle/python/testdata/respect_alias_kind/README.md @@ -0,0 +1,4 @@ +# Respect `alias_kind` + +This test case asserts that Gazelle updates the existing wrapper-macro rule +declared with `# gazelle:alias_kind` in place instead of reporting a collision. diff --git a/gazelle/python/testdata/respect_alias_kind/WORKSPACE b/gazelle/python/testdata/respect_alias_kind/WORKSPACE new file mode 100644 index 0000000000..faff6af87a --- /dev/null +++ b/gazelle/python/testdata/respect_alias_kind/WORKSPACE @@ -0,0 +1 @@ +# This is a Bazel workspace for the Gazelle test data. diff --git a/gazelle/python/testdata/respect_alias_kind/__init__.py b/gazelle/python/testdata/respect_alias_kind/__init__.py new file mode 100644 index 0000000000..6a49193fe4 --- /dev/null +++ b/gazelle/python/testdata/respect_alias_kind/__init__.py @@ -0,0 +1,3 @@ +from foo import foo + +_ = foo diff --git a/gazelle/python/testdata/respect_alias_kind/foo.py b/gazelle/python/testdata/respect_alias_kind/foo.py new file mode 100644 index 0000000000..cf68624419 --- /dev/null +++ b/gazelle/python/testdata/respect_alias_kind/foo.py @@ -0,0 +1,2 @@ +def foo(): + return "foo" diff --git a/gazelle/python/testdata/respect_alias_kind/test.yaml b/gazelle/python/testdata/respect_alias_kind/test.yaml new file mode 100644 index 0000000000..36dd656b39 --- /dev/null +++ b/gazelle/python/testdata/respect_alias_kind/test.yaml @@ -0,0 +1,3 @@ +--- +expect: + exit_code: 0 From ee7c54bf31a532ad9af34ac9a08bc70f6e365719 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Thu, 30 Apr 2026 20:22:43 -0700 Subject: [PATCH 730/922] chore: always use starlark-based extraction for wheels (#3748) Currently, the Python `installer` library is used for extracting a wheel when Bazel 7 is used. This library only outputs platform-specific extractions, and it was hard-coded to use a unixy style install. This ends up hiding Windows `pythonw`-based scripts and entry points because those are normalized to simply `python` on unix. To fix, use the Starlark-based extraction logic instead. This is a simpler implementation that extracts the wheels mostly as-is, which allows build-phase logic to handle platform-specific differences. This also makes the Starlark based extraction work for Bazel 8.3 and earlier, which previously didn't support it. --- python/private/pypi/whl_installer/BUILD.bazel | 1 - .../private/pypi/whl_installer/arguments.py | 5 - python/private/pypi/whl_installer/wheel.py | 105 ------------------ .../pypi/whl_installer/wheel_installer.py | 54 +-------- python/private/pypi/whl_library.bzl | 44 ++------ tests/pypi/whl_installer/BUILD.bazel | 19 +--- tests/pypi/whl_installer/arguments_test.py | 2 +- .../whl_installer/wheel_installer_test.py | 93 ---------------- tests/venv_site_packages_libs/BUILD.bazel | 4 - .../whl_scripts_runnable_test.py | 6 - 10 files changed, 10 insertions(+), 323 deletions(-) delete mode 100644 python/private/pypi/whl_installer/wheel.py delete mode 100644 tests/pypi/whl_installer/wheel_installer_test.py diff --git a/python/private/pypi/whl_installer/BUILD.bazel b/python/private/pypi/whl_installer/BUILD.bazel index b820b843d4..1912b34af8 100644 --- a/python/private/pypi/whl_installer/BUILD.bazel +++ b/python/private/pypi/whl_installer/BUILD.bazel @@ -5,7 +5,6 @@ py_library( name = "lib", srcs = [ "arguments.py", - "wheel.py", "wheel_installer.py", ], visibility = [ diff --git a/python/private/pypi/whl_installer/arguments.py b/python/private/pypi/whl_installer/arguments.py index 0d31ab7c4d..9122654a11 100644 --- a/python/private/pypi/whl_installer/arguments.py +++ b/python/private/pypi/whl_installer/arguments.py @@ -55,11 +55,6 @@ def parser(**kwargs: Any) -> argparse.ArgumentParser: help="Use 'pip download' instead of 'pip wheel'. Disables building wheels from source, but allows use of " "--platform, --python-version, --implementation, and --abi in --extra_pip_args.", ) - parser.add_argument( - "--whl-file", - type=pathlib.Path, - help="Extract a whl file to be used within Bazel.", - ) return parser diff --git a/python/private/pypi/whl_installer/wheel.py b/python/private/pypi/whl_installer/wheel.py deleted file mode 100644 index 801fd0f3b9..0000000000 --- a/python/private/pypi/whl_installer/wheel.py +++ /dev/null @@ -1,105 +0,0 @@ -# Copyright 2023 The Bazel Authors. All rights reserved. -# -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -"""Utility class to inspect an extracted wheel directory""" - -import email -from pathlib import Path - -import installer - - -class DoNothingCm: - """A context manager that does nothing when written to.""" - - def __enter__(self): - return self - - def __exit__(self, *args): - pass - - def write(self, data): - pass - - -class NoEntryPointsSchemeDictionaryDestination( - installer.destinations.SchemeDictionaryDestination -): - """ - A custom destination that prevents the `installer` package from automatically - generating scripts for `console_scripts` entry points. - - rules_python handles entry points via its own `venv_entry_point` targets. - If `installer` also generates these scripts in the `bin/` directory, it - causes a target naming collision because `whl_library_targets.bzl` will - try to create a `venv_rewrite_shebang` target with the same name. - - By overriding `for_script` to return a no-op dummy writer, we silently - discard the generated entry point scripts while still allowing `installer` - to process the rest of the wheel normally (including `.data/scripts` which - we do want to keep). - """ - - def for_script(self, name, module, attribute): - return DoNothingCm() - - -class Wheel: - """Representation of the compressed .whl file""" - - def __init__(self, path: Path): - self._path = path - - @property - def path(self) -> Path: - return self._path - - @property - def metadata(self) -> email.message.Message: - with installer.sources.WheelFile.open(self.path) as wheel_source: - metadata_contents = wheel_source.read_dist_info("METADATA") - metadata = installer.utils.parse_metadata_file(metadata_contents) - return metadata - - @property - def version(self) -> str: - # TODO Also available as installer.sources.WheelSource.version - return str(self.metadata["Version"]) - - def unzip(self, directory: str) -> None: - installation_schemes = { - "purelib": "/site-packages", - "platlib": "/site-packages", - "headers": "/include", - "scripts": "/bin", - "data": "/data", - } - - destination = NoEntryPointsSchemeDictionaryDestination( - installation_schemes, - # TODO Should entry_point scripts also be handled by installer rather than custom code? - interpreter="python", - script_kind="posix", - destdir=directory, - bytecode_optimization_levels=[], - ) - - with installer.sources.WheelFile.open(self.path) as wheel_source: - installer.install( - source=wheel_source, - destination=destination, - additional_metadata={ - "INSTALLER": b"https://github.com/bazel-contrib/rules_python", - }, - ) diff --git a/python/private/pypi/whl_installer/wheel_installer.py b/python/private/pypi/whl_installer/wheel_installer.py index 1f00068060..81dd3995db 100644 --- a/python/private/pypi/whl_installer/wheel_installer.py +++ b/python/private/pypi/whl_installer/wheel_installer.py @@ -25,9 +25,7 @@ from tempfile import NamedTemporaryFile from typing import Dict, List, Optional, Set, Tuple -from pip._vendor.packaging.utils import canonicalize_name - -from python.private.pypi.whl_installer import arguments, wheel +from python.private.pypi.whl_installer import arguments def _configure_reproducible_wheels() -> None: @@ -55,45 +53,6 @@ def _configure_reproducible_wheels() -> None: os.environ["PYTHONHASHSEED"] = "0" -def _parse_requirement_for_extra( - requirement: str, -) -> Tuple[Optional[str], Optional[Set[str]]]: - """Given a requirement string, returns the requirement name and set of extras, if extras specified. - Else, returns (None, None) - """ - - # https://www.python.org/dev/peps/pep-0508/#grammar - extras_pattern = re.compile( - r"^\s*([0-9A-Za-z][0-9A-Za-z_.\-]*)\s*\[\s*([0-9A-Za-z][0-9A-Za-z_.\-]*(?:\s*,\s*[0-9A-Za-z][0-9A-Za-z_.\-]*)*)\s*\]" - ) - - matches = extras_pattern.match(requirement) - if matches: - return ( - canonicalize_name(matches.group(1)), - {extra.strip() for extra in matches.group(2).split(",")}, - ) - - return None, None - - -def _extract_wheel( - wheel_file: str, - extras: Dict[str, Set[str]], - installation_dir: Path = Path("."), -) -> None: - """Extracts wheel into given directory and creates py_library and filegroup targets. - - Args: - wheel_file: the filepath of the .whl - installation_dir: the destination directory for installation of the wheel. - extras: a list of extras to add as dependencies for the installed wheel - """ - - whl = wheel.Wheel(wheel_file) - whl.unzip(installation_dir) - - def main() -> None: args = arguments.parser(description=__doc__).parse_args() deserialized_args = dict(vars(args)) @@ -101,17 +60,6 @@ def main() -> None: _configure_reproducible_wheels() - if args.whl_file: - whl = Path(args.whl_file) - - name, extras_for_pkg = _parse_requirement_for_extra(args.requirement) - extras = {name: extras_for_pkg} if extras_for_pkg and name else dict() - _extract_wheel( - wheel_file=whl, - extras=extras, - ) - return - pip_args = ( [sys.executable, "-m", "pip"] + (["--isolated"] if args.isolated else []) diff --git a/python/private/pypi/whl_library.bzl b/python/private/pypi/whl_library.bzl index 529514578d..966d25d04d 100644 --- a/python/private/pypi/whl_library.bzl +++ b/python/private/pypi/whl_library.bzl @@ -14,7 +14,6 @@ "" -load("@rules_python_internal//:rules_python_config.bzl", rp_config = "config") load("//python/private:auth.bzl", "AUTH_ATTRS", "get_auth") load("//python/private:envsubst.bzl", "envsubst") load("//python/private:is_standalone_interpreter.bzl", "is_standalone_interpreter") @@ -261,22 +260,6 @@ def _create_repository_execution_environment(rctx, python_interpreter, logger = env[_CPPFLAGS] = " ".join(cppflags) return env -def _extract_whl_py(rctx, *, python_interpreter, args, whl_path, environment, logger): - pypi_repo_utils.execute_checked( - rctx, - op = "whl_library.ExtractWheel({}, {})".format(rctx.attr.name, whl_path), - python = python_interpreter, - arguments = args + [ - "--whl-file", - whl_path, - ], - srcs = rctx.attr._python_srcs, - environment = environment, - quiet = rctx.attr.quiet, - timeout = rctx.attr.timeout, - logger = logger, - ) - def _get_entry_points(rctx, install_dir_path, metadata): dist_info_dir = "{}-{}.dist-info".format( metadata.name.replace("-", "_"), @@ -376,11 +359,10 @@ def _whl_library_impl(rctx): # build deps from PyPI (e.g. `flit_core`) if they are missing. extra_pip_args.extend(["--find-links", "."]) - enable_pipstar_extract = rp_config.bazel_8_or_later - - # When pipstar is enabled, Python isn't used, so there's no need - # to setup env vars to run Python, unless we need to build an sdist - if enable_pipstar_extract and whl_path and not rctx.attr.whl_patches: + # When we already have a wheel and there are no patches, Python isn't used, + # so there's no need to setup env vars to run Python, unless we need to + # build an sdist or resolve a requirement. + if whl_path and not rctx.attr.whl_patches: environment = {} args = [] python_interpreter = None @@ -446,17 +428,7 @@ def _whl_library_impl(rctx): timeout = rctx.attr.timeout, ) - if enable_pipstar_extract: - whl_extract(rctx, whl_path = whl_path, logger = logger) - else: - _extract_whl_py( - rctx, - python_interpreter = python_interpreter, - args = args, - whl_path = whl_path, - environment = environment, - logger = logger, - ) + whl_extract(rctx, whl_path = whl_path, logger = logger) install_dir_path = whl_path.dirname.get_child("site-packages") metadata = whl_metadata( @@ -513,9 +485,8 @@ repo( _remove_files(rctx, "BUILD", "BUILD.bazel") rctx.file("BUILD.bazel", build_file_contents) - if enable_pipstar_extract: - if hasattr(rctx, "repo_metadata"): - return rctx.repo_metadata(reproducible = True) + if hasattr(rctx, "repo_metadata"): + return rctx.repo_metadata(reproducible = True) return None @@ -632,7 +603,6 @@ way to define whl_library and move whl patching to a separate place. INTERNAL US "_python_srcs": attr.label_list( # Used as a default value in a rule to ensure we fetch the dependencies. default = [ - Label("//python/private/pypi/whl_installer:wheel.py"), Label("//python/private/pypi/whl_installer:wheel_installer.py"), Label("//python/private/pypi/whl_installer:arguments.py"), ] + record_files.values(), diff --git a/tests/pypi/whl_installer/BUILD.bazel b/tests/pypi/whl_installer/BUILD.bazel index 0a859c0e4d..5a2efb1260 100644 --- a/tests/pypi/whl_installer/BUILD.bazel +++ b/tests/pypi/whl_installer/BUILD.bazel @@ -1,10 +1,5 @@ load("//python:py_test.bzl", "py_test") -alias( - name = "lib", - actual = "//python/private/pypi/whl_installer:lib", -) - py_test( name = "arguments_test", size = "small", @@ -12,18 +7,6 @@ py_test( "arguments_test.py", ], deps = [ - ":lib", - ], -) - -py_test( - name = "wheel_installer_test", - size = "small", - srcs = [ - "wheel_installer_test.py", - ], - data = ["//examples/wheel:minimal_with_py_package"], - deps = [ - ":lib", + "//python/private/pypi/whl_installer:lib", ], ) diff --git a/tests/pypi/whl_installer/arguments_test.py b/tests/pypi/whl_installer/arguments_test.py index 9e26849db0..c874445029 100644 --- a/tests/pypi/whl_installer/arguments_test.py +++ b/tests/pypi/whl_installer/arguments_test.py @@ -15,7 +15,7 @@ import json import unittest -from python.private.pypi.whl_installer import arguments, wheel +from python.private.pypi.whl_installer import arguments class ArgumentsTestCase(unittest.TestCase): diff --git a/tests/pypi/whl_installer/wheel_installer_test.py b/tests/pypi/whl_installer/wheel_installer_test.py deleted file mode 100644 index 52c44cf2de..0000000000 --- a/tests/pypi/whl_installer/wheel_installer_test.py +++ /dev/null @@ -1,93 +0,0 @@ -# Copyright 2023 The Bazel Authors. All rights reserved. -# -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -import json -import os -import shutil -import tempfile -import unittest -from pathlib import Path - -from python.private.pypi.whl_installer import wheel_installer - - -class TestRequirementExtrasParsing(unittest.TestCase): - def test_parses_requirement_for_extra(self) -> None: - cases = [ - ("name[foo]", ("name", frozenset(["foo"]))), - ("name[ Foo123 ]", ("name", frozenset(["Foo123"]))), - (" name1[ foo ] ", ("name1", frozenset(["foo"]))), - ("Name[foo]", ("name", frozenset(["foo"]))), - ("name_foo[bar]", ("name-foo", frozenset(["bar"]))), - ( - "name [fred,bar] @ http://foo.com ; python_version=='2.7'", - ("name", frozenset(["fred", "bar"])), - ), - ( - "name[quux, strange];python_version<'2.7' and platform_version=='2'", - ("name", frozenset(["quux", "strange"])), - ), - ( - "name; (os_name=='a' or os_name=='b') and os_name=='c'", - (None, None), - ), - ( - "name@http://foo.com", - (None, None), - ), - ] - - for case, expected in cases: - with self.subTest(): - self.assertTupleEqual( - wheel_installer._parse_requirement_for_extra(case), expected - ) - - -class TestWhlFilegroup(unittest.TestCase): - def setUp(self) -> None: - self.wheel_name = "example_minimal_package-0.0.1-py3-none-any.whl" - self.wheel_dir = tempfile.mkdtemp() - self.wheel_path = os.path.join(self.wheel_dir, self.wheel_name) - shutil.copy(os.path.join("examples", "wheel", self.wheel_name), self.wheel_dir) - - def tearDown(self): - # On windows, the wheel file remains open, so gives an error upon - # deletion for some reason. - shutil.rmtree(self.wheel_dir, ignore_errors=True) - - def test_wheel_exists(self) -> None: - wheel_installer._extract_wheel( - Path(self.wheel_path), - installation_dir=Path(self.wheel_dir), - extras={}, - ) - - want_files = [ - "site-packages", - self.wheel_name, - ] - self.assertEqual( - sorted(want_files), - sorted( - [ - str(p.relative_to(self.wheel_dir)) - for p in Path(self.wheel_dir).glob("*") - ] - ), - ) - - -if __name__ == "__main__": - unittest.main() diff --git a/tests/venv_site_packages_libs/BUILD.bazel b/tests/venv_site_packages_libs/BUILD.bazel index 256c4f24b5..d44bbcbb63 100644 --- a/tests/venv_site_packages_libs/BUILD.bazel +++ b/tests/venv_site_packages_libs/BUILD.bazel @@ -1,4 +1,3 @@ -load("@rules_python_internal//:rules_python_config.bzl", rp_config = "config") load("@rules_shell//shell:sh_test.bzl", "sh_test") load("//python:py_library.bzl", "py_library") load("//tests/support:py_reconfig.bzl", "py_reconfig_test") @@ -79,9 +78,6 @@ py_reconfig_test( "@platforms//os:windows": "system_python", "//conditions:default": "script", }), - env = { - "BAZEL_8_OR_LATER": "1" if rp_config.bazel_8_or_later else "0", - }, main = "whl_scripts_runnable_test.py", venvs_site_packages = "yes", deps = [ diff --git a/tests/venv_site_packages_libs/whl_scripts_runnable_test.py b/tests/venv_site_packages_libs/whl_scripts_runnable_test.py index 61b477f493..b62b5a5fce 100644 --- a/tests/venv_site_packages_libs/whl_scripts_runnable_test.py +++ b/tests/venv_site_packages_libs/whl_scripts_runnable_test.py @@ -63,12 +63,6 @@ def test_entry_point_is_runnable(self): script_executable = output[-1].strip() self.assertEqual(script_executable, sys.executable) - # This should really check for 8.5 instead of 8+, but we test with 8.6 - # so it's close enough for our purposes. - @unittest.skipUnless( - BAZEL_8_OR_LATER, - "bazel 8.5 and lower uses wheel.py, which rewrites #!pythonw to #!python", - ) def test_pythonw_script(self): script_path = self._get_script_path("whl_with_data1_pythonw") self.assertTrue(script_path.exists(), f"Script not found at {script_path}") From e784fc250064cb652f7a2bb82d02f65eb8da3050 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Sun, 3 May 2026 14:48:15 -0700 Subject: [PATCH 731/922] docs: rewrite multi-version dependency track guide using custom platforms (#3751) Updated the monorepo multi-version dependency sharing documentation to reflect bzlmod extension capabilities, migrating from manual Starlark select() routing on library targets to single-hub configurations utilizing custom platform settings. Work towards https://github.com/bazel-contrib/rules_python/issues/3671. --- ...common-deps-with-multiple-pypi-versions.md | 116 +++++++++++++----- 1 file changed, 85 insertions(+), 31 deletions(-) diff --git a/docs/howto/common-deps-with-multiple-pypi-versions.md b/docs/howto/common-deps-with-multiple-pypi-versions.md index eb45d72b52..f6887e3667 100644 --- a/docs/howto/common-deps-with-multiple-pypi-versions.md +++ b/docs/howto/common-deps-with-multiple-pypi-versions.md @@ -1,52 +1,100 @@ (common-deps-with-multiple-pypi-versions)= # How to use a common set of dependencies with multiple PyPI versions -In this guide, we show how to handle a situation common to monorepos -that extensively share code: How does a common library refer to the correct -`@pypi_` hub when binaries may have their own requirements (and thus -PyPI hub name)? Stated as code, this situation: +In this guide, we show how to handle a situation common to monorepos that +extensively share code: How do multiple binaries utilize distinct requirements +files while pulling from shared internal libraries, without requiring +manually-maintained `select()` logic inside dependency targets? -```bzl +Stated as code, consider this example: +```bzl +# When building bin_alpha, requests and more_itertools should resolve +# from requirements_alpha.txt py_binary( - name = "bin_alpha", - deps = ["@pypi_alpha//requests", ":common"], + name = "bin_alpha", + deps = [ + "@pypi//requests", + ":common", + ], ) + +# When building bin_beta, requests and more_itertools should resolve +# from requirements_beta.txt py_binary( - name = "bin_beta", - deps = ["@pypi_beta//requests", ":common"], + name = "bin_beta", + deps = [ + "@pypi//requests", + ":common", + ], ) +# Transitive dependencies like more_itertools are requested here, but +# must automatically match whichever dependency track is active for the binary. py_library( - name = "common", - deps = ["@pypi_???//more_itertools"] # <-- Which @pypi repo? + name = "common", + deps = ["@pypi//more_itertools"], ) ``` -## Using flags to pick a hub +## Defining dependency tracks via custom platforms -The basic trick to make `:common` pick the appropriate `@pypi_` is to use -`select()` to choose one based on build flags. To help this process, `py_binary` -et al allow forcing particular build flags to be used, and custom flags can be -registered to allow `py_binary` et al to set them. +The solution involves defining custom "platforms" mapped to separate +dependency tracks inside `MODULE.bazel`. Using custom platforms via +{obj}`pip.default` and associating requirements files to them through the +`requirements_by_platform` attribute on {obj}`pip.parse` instructs +`rules_python` to generate `select()` logic behind a unified hub. -In this example, we create a custom string flag named `//:pypi_hub`, -register it to allow using it with `py_binary` directly, then use `select()` -to pick different dependencies. +Binaries configure their execution requirements by forcing flag transition +attributes using custom build setting flags. -```bzl +In this example, we define custom string flag named `//:pypi_hub`, setup +distinct custom platforms for `"alpha"` and `"beta"` profiles, and register +associated requirements lock files grouped inside the `@pypi` hub. + +```starlark # File: MODULE.bazel +rules_python_config = use_extension( + "@rules_python//python/extensions:config.bzl", + "config", +) rules_python_config.add_transition_setting( setting = "//:pypi_hub", ) +pip = use_extension("@rules_python//python/extensions:pip.bzl", "pip") + +pip.default( + platform = "alpha", + config_settings = ["@//:is_pypi_alpha"], +) + +pip.default( + platform = "beta", + config_settings = ["@//:is_pypi_beta"], +) + +pip.parse( + hub_name = "pypi", + python_version = "3.14", + requirements_by_platform = { + "//:requirements_alpha.txt": "alpha", + "//:requirements_beta.txt": "beta", + }, +) + +use_repo(pip, "pypi") +``` + +```starlark # File: BUILD.bazel load("@bazel_skylib//rules:common_settings.bzl", "string_flag") string_flag( name = "pypi_hub", + build_setting_default = "none", ) config_setting( @@ -56,7 +104,7 @@ config_setting( config_setting( name = "is_pypi_beta", - flag_values = {"//:pypi_hub": "beta"} + flag_values = {"//:pypi_hub": "beta"}, ) py_binary( @@ -65,26 +113,32 @@ py_binary( config_settings = { "//:pypi_hub": "alpha", }, - deps = ["@pypi_alpha//requests", ":common"], + deps = [ + "@pypi//requests", + ":common", + ], ) + py_binary( name = "bin_beta", srcs = ["bin_beta.py"], config_settings = { "//:pypi_hub": "beta", }, - deps = ["@pypi_beta//requests", ":common"], + deps = [ + "@pypi//requests", + ":common", + ], ) + py_library( name = "common", - deps = select({ - ":is_pypi_alpha": ["@pypi_alpha//more_itertools"], - ":is_pypi_beta": ["@pypi_beta//more_itertools"], - }), + deps = ["@pypi//more_itertools"], ) ``` -When `bin_alpha` and `bin_beta` are built, they will have the `pypi_hub` -flag force to their respective value. When `:common` is evaluated, it sees -the flag value of the binary that is consuming it, and the `select()` resolves -appropriately. +When building `bin_alpha` or `bin_beta`, they set `//:pypi_hub` via target +transitions. The generated aliased dependencies inside the `@pypi` hub will +evaluate that Bazel configuration, automatically delivering corresponding +Python wheels from targeted lock files. + From 83f714de5f55ef72db7adb370e5ad4a1a66d61ba Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Sun, 3 May 2026 16:03:29 -0700 Subject: [PATCH 732/922] fix(pypi): include RECORD file in installed wheel targets (#3752) Currently, `importlib.metadata.files()` returns `None` for packages installed via `rules_python`'s wheel rules. This happens because the `RECORD` file in the `.dist-info` directory is explicitly excluded from the `data` attribute of the generated `py_library` targets. To fix this, remove `**/*.dist-info/RECORD` from the `_data_exclude` list in `whl_library_targets.bzl`. This ensures that the `RECORD` file is preserved in the runfiles and available at runtime, enabling `importlib.metadata.files()` to correctly list the files in the package. Work towards https://github.com/bazel-contrib/rules_python/issues/3024 --- CHANGELOG.md | 4 ++ python/private/pypi/whl_library_targets.bzl | 6 +-- .../whl_library_targets_tests.bzl | 43 ++++++++++++++++++- tests/venv_site_packages_libs/BUILD.bazel | 14 ++++++ .../app_files_building_tests.bzl | 2 + .../importlib_metadata_test.py | 24 +++++++++++ 6 files changed, 87 insertions(+), 6 deletions(-) create mode 100644 tests/venv_site_packages_libs/importlib_metadata_test.py diff --git a/CHANGELOG.md b/CHANGELOG.md index dc45d2e95e..70f30ac67b 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -74,6 +74,10 @@ END_UNRELEASED_TEMPLATE * (uv) use the astral.sh mirror as the preferred url for binary downloads, with github.com as a fallback; for uv >= 0.11.0, read the checksums directly from the dist-manifest contents. +* (pypi) Fix `importlib.metadata.files` by ensuring `RECORD` is included in + installed wheel targets, except when built from sdist + ([#3024](https://github.com/bazel-contrib/rules_python/issues/3024)). + {#v0-0-0-added} ### Added diff --git a/python/private/pypi/whl_library_targets.bzl b/python/private/pypi/whl_library_targets.bzl index b3a52cd18c..01a89aadcc 100644 --- a/python/private/pypi/whl_library_targets.bzl +++ b/python/private/pypi/whl_library_targets.bzl @@ -374,11 +374,9 @@ def whl_library_targets( "**/*.py", "**/*.pyc", "**/*.pyc.*", # During pyc creation, temp files named *.pyc.NNNN are created - # RECORD is known to contain sha256 checksums of files which might include the checksums - # of generated files produced when wheels are installed. The file is ignored to avoid - # Bazel caching issues. - "**/*.dist-info/RECORD", ] + if sdist_filename: + _data_exclude.append("**/*.dist-info/RECORD") for item in data_exclude: if item not in _data_exclude: _data_exclude.append(item) diff --git a/tests/pypi/whl_library_targets/whl_library_targets_tests.bzl b/tests/pypi/whl_library_targets/whl_library_targets_tests.bzl index ec28bfbb39..91db15f296 100644 --- a/tests/pypi/whl_library_targets/whl_library_targets_tests.bzl +++ b/tests/pypi/whl_library_targets/whl_library_targets_tests.bzl @@ -272,7 +272,6 @@ def _test_whl_and_library_deps_from_requires(env): "**/*.py", "**/*.pyc", "**/*.pyc.*", - "**/*.dist-info/RECORD", ], allow_empty = True, ), @@ -471,13 +470,53 @@ def _test_group(env): "**/*.py", "**/*.pyc", "**/*.pyc.*", - "**/*.dist-info/RECORD", ], allow_empty = True), mocks.glob_call(["site-packages/**/*.pyi"], allow_empty = True), ]) _tests.append(_test_group) +def _test_sdist_excludes_record(env): + py_library_calls = [] + m_glob = mocks.glob() + m_glob.results.append([]) # bin + m_glob.results.append([]) # rewrite-bin + m_glob.results.append([]) # srcs + m_glob.results.append([]) # data + m_glob.results.append([]) # pyi + + whl_library_targets( + name = "foo.whl", + dep_template = "@pypi_{name}//:{target}", + sdist_filename = "foo.tar.gz", + filegroups = {}, + native = struct( + filegroup = lambda **_: None, + config_setting = lambda **_: None, + glob = m_glob.glob, + ), + rules = struct( + py_library = lambda **kwargs: py_library_calls.append(kwargs), + create_inits = lambda **kwargs: [], + venv_rewrite_shebang = lambda **kwargs: None, + ), + ) + + env.expect.that_collection(m_glob.calls).contains_at_least([ + mocks.glob_call( + ["site-packages/**/*"], + exclude = [ + "**/*.py", + "**/*.pyc", + "**/*.pyc.*", + "**/*.dist-info/RECORD", + ], + allow_empty = True, + ), + ]) + +_tests.append(_test_sdist_excludes_record) + def whl_library_targets_test_suite(name): """create the test suite. diff --git a/tests/venv_site_packages_libs/BUILD.bazel b/tests/venv_site_packages_libs/BUILD.bazel index d44bbcbb63..6a7b3b9e12 100644 --- a/tests/venv_site_packages_libs/BUILD.bazel +++ b/tests/venv_site_packages_libs/BUILD.bazel @@ -85,3 +85,17 @@ py_reconfig_test( "@whl_with_data2//:pkg", ], ) + +py_reconfig_test( + name = "importlib_metadata_test", + srcs = ["importlib_metadata_test.py"], + bootstrap_impl = select({ + "@platforms//os:windows": "system_python", + "//conditions:default": "script", + }), + main = "importlib_metadata_test.py", + venvs_site_packages = "yes", + deps = [ + "@whl_with_data1//:pkg", + ], +) diff --git a/tests/venv_site_packages_libs/app_files_building/app_files_building_tests.bzl b/tests/venv_site_packages_libs/app_files_building/app_files_building_tests.bzl index 66ba7076a9..65124076c6 100644 --- a/tests/venv_site_packages_libs/app_files_building/app_files_building_tests.bzl +++ b/tests/venv_site_packages_libs/app_files_building/app_files_building_tests.bzl @@ -471,6 +471,7 @@ def _test_optimized_grouping_pkgutil_whls_impl(env, target): files = [ "../+internal_dev_deps+pkgutil_nspkg1/site-packages/pkgutil_nspkg1-1.0.dist-info/INSTALLER", "../+internal_dev_deps+pkgutil_nspkg1/site-packages/pkgutil_nspkg1-1.0.dist-info/METADATA", + "../+internal_dev_deps+pkgutil_nspkg1/site-packages/pkgutil_nspkg1-1.0.dist-info/RECORD", "../+internal_dev_deps+pkgutil_nspkg1/site-packages/pkgutil_nspkg1-1.0.dist-info/WHEEL", ], ), @@ -495,6 +496,7 @@ def _test_optimized_grouping_pkgutil_whls_impl(env, target): files = [ "../+internal_dev_deps+pkgutil_nspkg2/site-packages/pkgutil_nspkg2-1.0.dist-info/INSTALLER", "../+internal_dev_deps+pkgutil_nspkg2/site-packages/pkgutil_nspkg2-1.0.dist-info/METADATA", + "../+internal_dev_deps+pkgutil_nspkg2/site-packages/pkgutil_nspkg2-1.0.dist-info/RECORD", "../+internal_dev_deps+pkgutil_nspkg2/site-packages/pkgutil_nspkg2-1.0.dist-info/WHEEL", ], ), diff --git a/tests/venv_site_packages_libs/importlib_metadata_test.py b/tests/venv_site_packages_libs/importlib_metadata_test.py new file mode 100644 index 0000000000..178ff14c50 --- /dev/null +++ b/tests/venv_site_packages_libs/importlib_metadata_test.py @@ -0,0 +1,24 @@ +import importlib.metadata +import unittest + + +class ImportlibMetadataTest(unittest.TestCase): + + def test_importlib_metadata_files(self): + files = importlib.metadata.files("whl-with-data1") + self.assertIsNotNone(files, "importlib.metadata.files returned None") + self.assertGreater( + len(files), 0, "importlib.metadata.files returned empty list" + ) + + # Verify it contains some expected files. + # The RECORD file lists paths relative to the installation root (site-packages). + # whl_with_data1-1.0.data/purelib/data_overlap.py should be installed as data_overlap.py + # whl_with_data1-1.0.data/platlib/whl_with_data1/platlib_file.txt should be whl_with_data1/platlib_file.txt + + file_names = [f.name for f in files] + self.assertIn("data_overlap.py", file_names) + + +if __name__ == "__main__": + unittest.main() From 6a58e1e57d0fddfbc517eac05b5ab11543a1b803 Mon Sep 17 00:00:00 2001 From: Alex Martani Date: Wed, 6 May 2026 20:33:45 -0700 Subject: [PATCH 733/922] feat(gazelle): enable pyi attrs by default (#3753) Default python_generate_pyi_deps and python_generate_pyi_srcs to true so generated targets preserve type-only dependencies and sibling stub files without extra directives. These directives have been available for many versions now, and defaulting to true better matches the intended semantics of current rules_python versions. Write detailed docs for `python_generate_pyi_deps` since that was missing. --- CHANGELOG.md | 2 ++ gazelle/docs/directives.md | 52 +++++++++++++++++++++++----- gazelle/pythonconfig/pythonconfig.go | 8 ++--- 3 files changed, 50 insertions(+), 12 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 70f30ac67b..c102d3b04e 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -60,6 +60,8 @@ END_UNRELEASED_TEMPLATE ### Changed * (gazelle) WORKSPACE's bazel-gazelle dependency bumped from 0.36.0 to 0.47.0. The go version was also bumped from 1.21.13 to 1.22.9. +* (gazelle) `python_generate_pyi_deps` and `python_generate_pyi_srcs` now + default to `true`. * (pypi) The data files of a wheel (bin, includes, etc) are now always included as a library's data dependencies. diff --git a/gazelle/docs/directives.md b/gazelle/docs/directives.md index f23c75f331..dfa439358d 100644 --- a/gazelle/docs/directives.md +++ b/gazelle/docs/directives.md @@ -147,18 +147,18 @@ The Python-specific directives are: [`# gazelle:python_generate_pyi_deps bool`](#directive-python-generate-pyi-deps) : Controls whether to generate a separate `pyi_deps` attribute for type-checking dependencies or merge them into the regular `deps` - attribute. When `false` (default), type-checking dependencies are - merged into `deps` for backward compatibility. When `true`, generates - separate `pyi_deps`. Imports in blocks with the format + attribute. When `true` (default), generates separate `pyi_deps`. When + `false`, type-checking dependencies are merged into `deps`. Imports in + blocks with the format `if typing.TYPE_CHECKING:` or `if TYPE_CHECKING:` and type-only stub packages (eg. boto3-stubs) are recognized as type-checking dependencies. - * Default: `false` + * Default: `true` * Allowed Values: `true`, `false` [`# gazelle:python_generate_pyi_srcs bool`](#directive-python-generate-pyi-srcs) : Controls whether to generate a `pyi_srcs` attribute if a sibling `.pyi` file - is found. When `false` (default), the `pyi_srcs` attribute is not added. - * Default: `false` + is found. When `false`, the `pyi_srcs` attribute is not added. + * Default: `true` * Allowed Values: `true`, `false` [`# gazelle:python_generate_proto bool`](#directive-python-generate-proto) @@ -681,10 +681,42 @@ that are relative to the current package. {gh-pr}`3014` ::: -:::{error} -Detailed docs are not yet written. +:::{versionchanged} VERSION_NEXT_FEATURE +The default was changed from `false` to `true`. {gh-pr}`3753` ::: +When `true`, Gazelle writes type-checking dependencies to the `pyi_deps` +attribute instead of merging them into `deps`. This is the default behavior. + +Gazelle treats imports inside `if TYPE_CHECKING:` and +`if typing.TYPE_CHECKING:` blocks as type-checking dependencies. It also adds +type stub packages, such as `boto3-stubs`, to `pyi_deps` when the corresponding +runtime package is imported normally. + +For example, assume you have the following file: + +```python +import boto3 +from typing import TYPE_CHECKING + +if TYPE_CHECKING: + import requests +``` + +The generated target will be: + +```starlark +py_library( + name = "foo", + srcs = ["foo.py"], + pyi_deps = ["@pip//requests"], + deps = ["@pip//boto3"], +) +``` + +When `false`, Gazelle merges type-checking dependencies into `deps` and does +not write `pyi_deps`. + (directive-python-generate-pyi-srcs)= ## `python_generate_pyi_srcs` @@ -693,6 +725,10 @@ Detailed docs are not yet written. {gh-pr}`3356` ::: +:::{versionchanged} VERSION_NEXT_FEATURE +The default was changed from `false` to `true`. {gh-pr}`3753` +::: + When `true`, include any sibling `.pyi` files in the `pyi_srcs` target attribute. For example, assume you have the following files: diff --git a/gazelle/pythonconfig/pythonconfig.go b/gazelle/pythonconfig/pythonconfig.go index 17db9aae0d..c88d59abcf 100644 --- a/gazelle/pythonconfig/pythonconfig.go +++ b/gazelle/pythonconfig/pythonconfig.go @@ -102,11 +102,11 @@ const ( ExperimentalAllowRelativeImports = "python_experimental_allow_relative_imports" // GeneratePyiDeps represents the directive that controls whether to generate // separate pyi_deps attribute or merge type-checking dependencies into deps. - // Defaults to false for backward compatibility. + // Defaults to true. GeneratePyiDeps = "python_generate_pyi_deps" // GeneratePyiSrcs represents the directive that controls whether to include // a pyi_srcs attribute if a sibling .pyi file is found. - // Defaults to false for backward compatibility. + // Defaults to true. GeneratePyiSrcs = "python_generate_pyi_srcs" // GenerateProto represents the directive that controls whether to generate // python_generate_proto targets. @@ -256,8 +256,8 @@ func New( labelConvention: DefaultLabelConvention, labelNormalization: DefaultLabelNormalizationType, experimentalAllowRelativeImports: false, - generatePyiDeps: false, - generatePyiSrcs: false, + generatePyiDeps: true, + generatePyiSrcs: true, generateProto: false, resolveSiblingImports: false, includeAncestorConftest: true, From ca066b83f53875a26c7e522d32c8b296b0607a66 Mon Sep 17 00:00:00 2001 From: Ignas Anikevicius <240938+aignas@users.noreply.github.com> Date: Thu, 7 May 2026 16:18:23 +0800 Subject: [PATCH 734/922] chore(pypi): remove unused config settings code (#3739) Summary: * Remove unused `pip_*` config settings since they have been no op for a while. * The code was experimental, so it is likely nobody relied heavily on those things. * Only leave the OSX version flag - we may want to rename that in the future. Related #260. --------- Co-authored-by: Richard Levasseur --- .../python/config_settings/index.md | 54 +++----------- docs/pypi/download.md | 5 -- python/config_settings/BUILD.bazel | 68 +---------------- python/private/common_labels.bzl | 4 - python/private/pypi/BUILD.bazel | 1 - python/private/pypi/flags.bzl | 74 +------------------ python/private/pypi/hub_builder.bzl | 4 +- python/private/pypi/parse_requirements.bzl | 4 +- python/private/pypi/pip_repository.bzl | 2 +- python/private/transition_labels.bzl | 4 - .../config_settings/config_settings_tests.bzl | 7 -- tests/pypi/hub_builder/hub_builder_tests.bzl | 2 +- .../parse_requirements_tests.bzl | 6 +- 13 files changed, 23 insertions(+), 212 deletions(-) diff --git a/docs/api/rules_python/python/config_settings/index.md b/docs/api/rules_python/python/config_settings/index.md index 19f5b8bc37..d7bc296296 100644 --- a/docs/api/rules_python/python/config_settings/index.md +++ b/docs/api/rules_python/python/config_settings/index.md @@ -274,48 +274,6 @@ the values used when environment markers are resolved at build time. ::: :::: -::::{bzl:flag} pip_whl -Set what distributions are used in the `pip` integration. - -Values: -* `auto`: Prefer `whl` distributions if they are compatible with a target - platform, but fallback to `sdist`. This is the default. -* `only`: Only use `whl` distributions and error out if it is not available. -* `no`: Only use `sdist` distributions. The wheels will be built non-hermetically in the `whl_library` repository rule. -:::{versionadded} 0.33.0 -::: -:::: - -::::{bzl:flag} pip_whl_osx_arch -Set what wheel types we should prefer when building on the OSX platform. - -Values: -* `arch`: Prefer architecture specific wheels. -* `universal`: Prefer universal wheels that usually are bigger and contain binaries for both, Intel and ARM architectures in the same wheel. -:::{versionadded} 0.33.0 -::: -:::: - -::::{bzl:flag} pip_whl_glibc_version -Set the minimum `glibc` version that the `py_binary` using `whl` distributions from a PyPI index should support. - -Values: -* `""`: Select the lowest available version of each wheel giving you the maximum compatibility. This is the default. -* `X.Y`: The string representation of a `glibc` version. The allowed values depend on the `requirements.txt` lock file contents. -:::{versionadded} 0.33.0 -::: -:::: - -::::{bzl:flag} pip_whl_muslc_version -Set the minimum `muslc` version that the `py_binary` using `whl` distributions from a PyPI index should support. - -Values: -* `""`: Select the lowest available version of each wheel giving you the maximum compatibility. This is the default. -* `X.Y`: The string representation of a `muslc` version. The allowed values depend on the `requirements.txt` lock file contents. -:::{versionadded} 0.33.0 -::: -:::: - ::::{bzl:flag} pip_whl_osx_version Set the minimum `osx` version that the `py_binary` using `whl` distributions from a PyPI index should support. @@ -414,3 +372,15 @@ is created. :::{versionadded} 1.2.0 ::: :::: + +## Removed Flags + +:::{versionremoved} VERSION_NEXT_FEATURE +The following flags were removed: + +* `pip_whl` +* `pip_whl_osx_arch` +* `pip_whl_glibc_version` +* `pip_whl_muslc_version` +::: + diff --git a/docs/pypi/download.md b/docs/pypi/download.md index d4159eb3a7..f0e70cf850 100644 --- a/docs/pypi/download.md +++ b/docs/pypi/download.md @@ -244,11 +244,6 @@ that by parsing the `whl` filename based on [PEP600], [PEP656] standards. This allows the user to configure the behaviour by using the following publicly available flags: * {obj}`--@rules_python//python/config_settings:py_linux_libc` for selecting the Linux libc variant. -* {obj}`--@rules_python//python/config_settings:pip_whl` for selecting `whl` distribution preference. -* {obj}`--@rules_python//python/config_settings:pip_whl_osx_arch` for selecting MacOS wheel preference. -* {obj}`--@rules_python//python/config_settings:pip_whl_glibc_version` for selecting the GLIBC version compatibility. -* {obj}`--@rules_python//python/config_settings:pip_whl_muslc_version` for selecting the musl version compatibility. -* {obj}`--@rules_python//python/config_settings:pip_whl_osx_version` for selecting MacOS version compatibility. [bazel_downloader]: https://bazel.build/rules/lib/builtins/repository_ctx#download [pep600]: https://peps.python.org/pep-0600/ diff --git a/python/config_settings/BUILD.bazel b/python/config_settings/BUILD.bazel index fc0ac51451..5b1317872f 100644 --- a/python/config_settings/BUILD.bazel +++ b/python/config_settings/BUILD.bazel @@ -14,12 +14,7 @@ load( "VenvsUseDeclareSymlinkFlag", rp_string_flag = "string_flag", ) -load( - "//python/private/pypi:flags.bzl", - "UniversalWhlFlag", - "UseWhlFlag", - "define_pypi_internal_flags", -) +load("//python/private/pypi:flags.bzl", "define_pypi_internal_flags") load(":config_settings.bzl", "construct_config_settings") filegroup( @@ -34,10 +29,6 @@ construct_config_settings( name = "construct_config_settings", default_version = DEFAULT_PYTHON_VERSION, documented_flags = [ - ":pip_whl", - ":pip_whl_glibc_version", - ":pip_whl_muslc_version", - ":pip_whl_osx_arch", ":pip_whl_osx_version", ":py_freethreaded", ":py_linux_libc", @@ -157,63 +148,6 @@ string_flag( # pip.parse related flags -string_flag( - name = "pip_whl", - build_setting_default = UseWhlFlag.AUTO, - values = sorted(UseWhlFlag.__members__.values()), - # NOTE: Only public because it is used in pip hub repos. - visibility = ["//visibility:public"], -) - -config_setting( - name = "is_pip_whl_auto", - flag_values = { - ":pip_whl": UseWhlFlag.AUTO, - }, - # NOTE: Only public because it is used in pip hub repos. - visibility = ["//visibility:public"], -) - -config_setting( - name = "is_pip_whl_no", - flag_values = { - ":pip_whl": UseWhlFlag.NO, - }, - # NOTE: Only public because it is used in pip hub repos. - visibility = ["//visibility:public"], -) - -config_setting( - name = "is_pip_whl_only", - flag_values = { - ":pip_whl": UseWhlFlag.ONLY, - }, - # NOTE: Only public because it is used in pip hub repos. - visibility = ["//visibility:public"], -) - -string_flag( - name = "pip_whl_osx_arch", - build_setting_default = UniversalWhlFlag.ARCH, - values = sorted(UniversalWhlFlag.__members__.values()), - # NOTE: Only public because it is used in pip hub repos. - visibility = ["//visibility:public"], -) - -string_flag( - name = "pip_whl_glibc_version", - build_setting_default = "", - # NOTE: Only public because it is used in pip hub repos. - visibility = ["//visibility:public"], -) - -string_flag( - name = "pip_whl_muslc_version", - build_setting_default = "", - # NOTE: Only public because it is used in pip hub repos. - visibility = ["//visibility:public"], -) - string_flag( name = "pip_whl_osx_version", build_setting_default = "", diff --git a/python/private/common_labels.bzl b/python/private/common_labels.bzl index b6594cf0b9..ff4e7e1dad 100644 --- a/python/private/common_labels.bzl +++ b/python/private/common_labels.bzl @@ -16,10 +16,6 @@ labels = struct( EXEC_TOOLS_TOOLCHAIN = str(Label("//python/config_settings:exec_tools_toolchain")), NONE = str(Label("//python:none")), PIP_ENV_MARKER_CONFIG = str(Label("//python/config_settings:pip_env_marker_config")), - PIP_WHL = str(Label("//python/config_settings:pip_whl")), - PIP_WHL_GLIBC_VERSION = str(Label("//python/config_settings:pip_whl_glibc_version")), - PIP_WHL_MUSLC_VERSION = str(Label("//python/config_settings:pip_whl_muslc_version")), - PIP_WHL_OSX_ARCH = str(Label("//python/config_settings:pip_whl_osx_arch")), PIP_WHL_OSX_VERSION = str(Label("//python/config_settings:pip_whl_osx_version")), PLATFORMS_OS_WINDOWS = str(Label("@platforms//os:windows")), PRECOMPILE = str(Label("//python/config_settings:precompile")), diff --git a/python/private/pypi/BUILD.bazel b/python/private/pypi/BUILD.bazel index c46ea83874..2a62767dd4 100644 --- a/python/private/pypi/BUILD.bazel +++ b/python/private/pypi/BUILD.bazel @@ -163,7 +163,6 @@ bzl_library( deps = [ ":env_marker_info.bzl", ":pep508_env_bzl", - "//python/private:enum_bzl", "@bazel_skylib//rules:common_settings", ], ) diff --git a/python/private/pypi/flags.bzl b/python/private/pypi/flags.bzl index f88690d843..36cb5373ff 100644 --- a/python/private/pypi/flags.bzl +++ b/python/private/pypi/flags.bzl @@ -18,9 +18,8 @@ NOTE: The transitive loads of this should be kept minimal. This avoids loading unnecessary files when all that are needed are flag definitions. """ -load("@bazel_skylib//rules:common_settings.bzl", "BuildSettingInfo", "string_flag") +load("@bazel_skylib//rules:common_settings.bzl", "BuildSettingInfo") load("//python/private:common_labels.bzl", "labels") -load("//python/private:enum.bzl", "enum") load(":env_marker_info.bzl", "EnvMarkerInfo") load( ":pep508_env.bzl", @@ -31,87 +30,16 @@ load( "sys_platform_select_map", ) -# Determines if we should use whls for third party -# -# buildifier: disable=name-conventions -UseWhlFlag = enum( - # Automatically decide the effective value based on environment, target - # platform and the presence of distributions for a particular package. - AUTO = "auto", - # Do not use `sdist` and fail if there are no available whls suitable for the target platform. - ONLY = "only", - # Do not use whl distributions and instead build the whls from `sdist`. - NO = "no", -) - -# Determines whether universal wheels should be preferred over arch platform specific ones. -# -# buildifier: disable=name-conventions -UniversalWhlFlag = enum( - # Prefer platform-specific wheels over universal wheels. - ARCH = "arch", - # Prefer universal wheels over platform-specific wheels. - UNIVERSAL = "universal", -) - -_STRING_FLAGS = [ - "dist", - "whl_plat", - "whl_plat_py3", - "whl_plat_py3_abi3", - "whl_plat_pycp3x", - "whl_plat_pycp3x_abi3", - "whl_plat_pycp3x_abicp", - "whl_py3", - "whl_py3_abi3", - "whl_pycp3x", - "whl_pycp3x_abi3", - "whl_pycp3x_abicp", -] - -INTERNAL_FLAGS = [ - "whl", -] + _STRING_FLAGS - def define_pypi_internal_flags(name): """define internal PyPI flags used in PyPI hub repository by pkg_aliases. Args: name: not used """ - for flag in _STRING_FLAGS: - string_flag( - name = "_internal_pip_" + flag, - build_setting_default = "", - values = [""], - visibility = ["//visibility:public"], - ) - - _allow_wheels_flag( - name = "_internal_pip_whl", - visibility = ["//visibility:public"], - ) - _default_env_marker_config( name = "_pip_env_marker_default_config", ) -def _allow_wheels_flag_impl(ctx): - input = ctx.attr._setting[BuildSettingInfo].value - value = "yes" if input in ["auto", "only"] else "no" - return [config_common.FeatureFlagInfo(value = value)] - -_allow_wheels_flag = rule( - implementation = _allow_wheels_flag_impl, - attrs = { - "_setting": attr.label(default = labels.PIP_WHL), - }, - doc = """ -This rule allows us to greatly reduce the number of config setting targets at no cost even -if we are duplicating some of the functionality of the `native.config_setting`. -""", -) - def _default_env_marker_config(**kwargs): _env_marker_config( os_name = select(os_name_select_map), diff --git a/python/private/pypi/hub_builder.bzl b/python/private/pypi/hub_builder.bzl index 85a31cfc3c..1bce648dce 100644 --- a/python/private/pypi/hub_builder.bzl +++ b/python/private/pypi/hub_builder.bzl @@ -7,7 +7,7 @@ load("//python/private:text_util.bzl", "render") load("//python/private:version.bzl", "version") load("//python/private:version_label.bzl", "version_label") load(":attrs.bzl", "use_isolated") -load(":evaluate_markers.bzl", evaluate_markers_star = "evaluate_markers") +load(":evaluate_markers.bzl", "evaluate_markers") load(":parse_requirements.bzl", "parse_requirements") load(":pep508_env.bzl", "env") load(":pep508_evaluate.bzl", "evaluate") @@ -469,7 +469,7 @@ def _evaluate_markers(self, pip_attr): if self._evaluate_markers_fn: return self._evaluate_markers_fn - return lambda _, requirements: evaluate_markers_star( + return lambda requirements: evaluate_markers( requirements = requirements, platforms = self._platforms[pip_attr.python_version], ) diff --git a/python/private/pypi/parse_requirements.bzl b/python/private/pypi/parse_requirements.bzl index 2a7793212a..d047cc607d 100644 --- a/python/private/pypi/parse_requirements.bzl +++ b/python/private/pypi/parse_requirements.bzl @@ -86,7 +86,7 @@ def parse_requirements( The second element is extra_pip_args should be passed to `whl_library`. """ - evaluate_markers = evaluate_markers or (lambda _ctx, _requirements: {}) + evaluate_markers = evaluate_markers or (lambda _requirements: {}) options = {} requirements = {} reqs_with_env_markers = {} @@ -130,7 +130,7 @@ def parse_requirements( # This may call to Python, so execute it early (before calling to the # internet below) and ensure that we call it only once. - resolved_marker_platforms = evaluate_markers(ctx, reqs_with_env_markers) + resolved_marker_platforms = evaluate_markers(reqs_with_env_markers) logger.trace(lambda: "Evaluated env markers from:\n{}\n\nTo:\n{}".format( reqs_with_env_markers, resolved_marker_platforms, diff --git a/python/private/pypi/pip_repository.bzl b/python/private/pypi/pip_repository.bzl index d55953aaa7..4afb62780a 100644 --- a/python/private/pypi/pip_repository.bzl +++ b/python/private/pypi/pip_repository.bzl @@ -123,7 +123,7 @@ def _pip_repository_impl(rctx): platforms = platforms, ), extra_pip_args = rctx.attr.extra_pip_args, - evaluate_markers = lambda rctx, requirements: evaluate_markers( + evaluate_markers = lambda requirements: evaluate_markers( requirements = { # NOTE @aignas 2025-07-07: because we don't distinguish between # freethreaded and non-freethreaded, it is a 1:1 mapping. diff --git a/python/private/transition_labels.bzl b/python/private/transition_labels.bzl index 04fcecb5ec..5f0aa69056 100644 --- a/python/private/transition_labels.bzl +++ b/python/private/transition_labels.bzl @@ -13,10 +13,6 @@ _BASE_TRANSITION_LABELS = [ labels.DEBUGGER, labels.EXEC_TOOLS_TOOLCHAIN, labels.PIP_ENV_MARKER_CONFIG, - labels.PIP_WHL_MUSLC_VERSION, - labels.PIP_WHL, - labels.PIP_WHL_GLIBC_VERSION, - labels.PIP_WHL_OSX_ARCH, labels.PIP_WHL_OSX_VERSION, labels.PRECOMPILE, labels.PRECOMPILE_SOURCE_RETENTION, diff --git a/tests/pypi/config_settings/config_settings_tests.bzl b/tests/pypi/config_settings/config_settings_tests.bzl index ed95bd4877..c7e51b78b4 100644 --- a/tests/pypi/config_settings/config_settings_tests.bzl +++ b/tests/pypi/config_settings/config_settings_tests.bzl @@ -32,14 +32,7 @@ _subject = rule( _flag = struct( platform = lambda x: ("//command_line_option:platforms", str(Label("//tests/support/platforms:" + x))), - pip_whl = lambda x: (str(Label("//python/config_settings:pip_whl")), str(x)), - pip_whl_glibc_version = lambda x: (str(Label("//python/config_settings:pip_whl_glibc_version")), str(x)), - pip_whl_muslc_version = lambda x: (str(Label("//python/config_settings:pip_whl_muslc_version")), str(x)), - pip_whl_osx_version = lambda x: (str(Label("//python/config_settings:pip_whl_osx_version")), str(x)), - pip_whl_osx_arch = lambda x: (str(Label("//python/config_settings:pip_whl_osx_arch")), str(x)), - py_linux_libc = lambda x: (str(Label("//python/config_settings:py_linux_libc")), str(x)), python_version = lambda x: (str(Label("//python/config_settings:python_version")), str(x)), - py_freethreaded = lambda x: (str(Label("//python/config_settings:py_freethreaded")), str(x)), ) def _analysis_test(*, name, dist, want, config_settings = [_flag.platform("linux_aarch64")]): diff --git a/tests/pypi/hub_builder/hub_builder_tests.bzl b/tests/pypi/hub_builder/hub_builder_tests.bzl index ccf72c2774..216528fc9b 100644 --- a/tests/pypi/hub_builder/hub_builder_tests.bzl +++ b/tests/pypi/hub_builder/hub_builder_tests.bzl @@ -443,7 +443,7 @@ def _test_simple_with_markers(env): for (host_os, host_arch), want_requirement in sub_tests.items(): builder = hub_builder( env, - evaluate_markers_fn = lambda _, requirements, **__: { + evaluate_markers_fn = lambda requirements: { key: [ platform for platform in platforms diff --git a/tests/pypi/parse_requirements/parse_requirements_tests.bzl b/tests/pypi/parse_requirements/parse_requirements_tests.bzl index 1786c4e664..4ed1870309 100644 --- a/tests/pypi/parse_requirements/parse_requirements_tests.bzl +++ b/tests/pypi/parse_requirements/parse_requirements_tests.bzl @@ -433,7 +433,7 @@ def _test_select_requirement_none_platform(env): _tests.append(_test_select_requirement_none_platform) def _test_env_marker_resolution(env): - def _mock_eval_markers(_, input): + def _mock_eval_markers(input): ret = { "foo[extra]==0.0.1 ;marker --hash=sha256:deadbeef": ["cp311_windows_x86_64"], } @@ -781,7 +781,7 @@ def _test_get_index_urls_different_versions(env): }, ), }, - evaluate_markers = lambda _, requirements: evaluate_markers( + evaluate_markers = lambda requirements: evaluate_markers( requirements = requirements, platforms = { "cp310_linux_x86_64": struct( @@ -859,7 +859,7 @@ def _test_get_index_urls_single_py_version(env): }, ), }, - evaluate_markers = lambda _, requirements: evaluate_markers( + evaluate_markers = lambda requirements: evaluate_markers( requirements = requirements, platforms = { "cp310_linux_x86_64": struct( From b5c887378a38c2fe0b542eb101fc473e95ec9b85 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Fri, 8 May 2026 00:52:59 -0700 Subject: [PATCH 735/922] fix(toolchains): add releases.astral.sh mirror for python-build-standalone (#3761) Currently, python-build-standalone runtimes are only downloaded from github.com. If github.com is down or experiencing 5xx errors, builds will fail because the runtimes cannot be fetched. To fix this, add releases.astral.sh as a secondary fallback mirror URL. If the requested base_url starts with the standard or legacy github.com release prefix, we append the equivalent releases.astral.sh URL to rendered_urls. --- python/versions.bzl | 15 ++++- .../get_release_info_tests.bzl | 55 +++++++++++++++++++ 2 files changed, 69 insertions(+), 1 deletion(-) diff --git a/python/versions.bzl b/python/versions.bzl index 33b2398dc0..ed819e6b34 100644 --- a/python/versions.bzl +++ b/python/versions.bzl @@ -28,6 +28,10 @@ INSTALL_ONLY = "install_only" DEFAULT_RELEASE_BASE_URL = "https://github.com/astral-sh/python-build-standalone/releases/download" +_GITHUB_PREFIX = "https://github.com/astral-sh/python-build-standalone/releases/download" +_LEGACY_GITHUB_PREFIX = "https://github.com/indygreg/python-build-standalone/releases/download" +_ASTRAL_PREFIX = "https://releases.astral.sh/github/python-build-standalone/releases/download" + # When updating the versions and releases, run the following command to get # the hashes: # bazel run //python/private:print_toolchains_checksums --//python/config_settings:python_version={major}.{minor}.{patch} @@ -1435,6 +1439,14 @@ def get_release_info(platform, python_version, base_url = DEFAULT_RELEASE_BASE_U A tuple of (filename, url, archive strip prefix, patches, patch_strip) """ + base_urls = [base_url] + if base_url == DEFAULT_RELEASE_BASE_URL or base_url.startswith(_GITHUB_PREFIX): + suffix = base_url[len(_GITHUB_PREFIX):] + base_urls.append(_ASTRAL_PREFIX + suffix) + elif base_url.startswith(_LEGACY_GITHUB_PREFIX): + suffix = base_url[len(_LEGACY_GITHUB_PREFIX):] + base_urls.append(_ASTRAL_PREFIX + suffix) + url = tool_versions[python_version]["url"] if type(url) == type({}): @@ -1490,7 +1502,8 @@ def get_release_info(platform, python_version, base_url = DEFAULT_RELEASE_BASE_U if "://" in release_filename: # is absolute url? rendered_urls.append(release_filename) else: - rendered_urls.append("/".join([base_url, release_filename])) + for b_url in base_urls: + rendered_urls.append("/".join([b_url, release_filename])) if release_filename == None: fail("release_filename should be set by now; were any download URLs given?") diff --git a/tests/get_release_info/get_release_info_tests.bzl b/tests/get_release_info/get_release_info_tests.bzl index ca553a3c4b..c810489c92 100644 --- a/tests/get_release_info/get_release_info_tests.bzl +++ b/tests/get_release_info/get_release_info_tests.bzl @@ -48,6 +48,61 @@ def _test_file_url(env): _tests.append(_test_file_url) +def _test_astral_mirror(env): + """Tests that the releases.astral.sh mirror is added as a secondary URL.""" + tool_versions = { + "3.11.5": { + "sha256": { + "x86_64-unknown-linux-gnu": "fbed6f7694b2faae5d7c401a856219c945397f772eea5ca50c6eb825cbc9d1e1", + }, + "strip_prefix": "python", + "url": "20230826/cpython-{python_version}+20230826-{platform}-{build}.tar.gz", + }, + } + + expected_urls = [ + "https://github.com/astral-sh/python-build-standalone/releases/download/20230826/cpython-3.11.5+20230826-x86_64-unknown-linux-gnu-install_only.tar.gz", + "https://releases.astral.sh/github/python-build-standalone/releases/download/20230826/cpython-3.11.5+20230826-x86_64-unknown-linux-gnu-install_only.tar.gz", + ] + + _, urls, _, _, _ = get_release_info( + platform = "x86_64-unknown-linux-gnu", + python_version = "3.11.5", + tool_versions = tool_versions, + ) + + env.expect.that_collection(urls).contains_exactly(expected_urls) + +_tests.append(_test_astral_mirror) + +def _test_astral_mirror_legacy(env): + """Tests that the releases.astral.sh mirror is added for legacy indygreg URLs.""" + tool_versions = { + "3.11.5": { + "sha256": { + "x86_64-unknown-linux-gnu": "fbed6f7694b2faae5d7c401a856219c945397f772eea5ca50c6eb825cbc9d1e1", + }, + "strip_prefix": "python", + "url": "20230826/cpython-{python_version}+20230826-{platform}-{build}.tar.gz", + }, + } + + expected_urls = [ + "https://github.com/indygreg/python-build-standalone/releases/download/20230826/cpython-3.11.5+20230826-x86_64-unknown-linux-gnu-install_only.tar.gz", + "https://releases.astral.sh/github/python-build-standalone/releases/download/20230826/cpython-3.11.5+20230826-x86_64-unknown-linux-gnu-install_only.tar.gz", + ] + + _, urls, _, _, _ = get_release_info( + platform = "x86_64-unknown-linux-gnu", + python_version = "3.11.5", + base_url = "https://github.com/indygreg/python-build-standalone/releases/download", + tool_versions = tool_versions, + ) + + env.expect.that_collection(urls).contains_exactly(expected_urls) + +_tests.append(_test_astral_mirror_legacy) + def get_release_info_test_suite(name): """Defines the test suite for get_release_info.""" test_suite( From 9dc505b95063c0774945ecc86c4b8ce51151edfa Mon Sep 17 00:00:00 2001 From: Joshua Yanchar Date: Sat, 9 May 2026 22:33:06 -0700 Subject: [PATCH 736/922] feat(coverage): add Python 3.14 support and bump coverage.py to 7.10.7 (#3764) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit On rules_python 2.0.0, `python.toolchain(configure_coverage_tool = True)` is a silent no-op when `python_version = "3.14"`: the bundled `coverage.py` wheel set in `python/private/coverage_deps.bzl` covers cp310–cp313 only, so `bazel coverage` produces zero-byte per-test lcov files for `py_test` targets. This PR bumps the bundled coverage to 7.10.7 (which ships cp39–cp314 wheels, including freethreaded variants for cp313+) and regenerates the wheel set. - Regenerate `coverage_deps.bzl` against coverage 7.10.7 - Add cp314 (incl. freethreaded cp314t variants) to the bundled set - Drop cp38 from the supported set (coverage.py 7.6.2 dropped Python 3.8) - Extend `update_coverage_deps.py`'s `_supported_platforms` map with the newer wheel platform tags (`manylinux1_x86_64`, `macosx_10_13_x86_64`) used by coverage 7.10.7 - Narrow `coverage.patch` context to apply against the 7.10.7 `coverage/__main__.py` layout - Document the bundled coverage version range in `docs/coverage.md` Verification: - `bazel test //tests/python:python_tests //tests/py_runtime:py_runtime_tests --test_tag_filters=-integration-test` → 37/37 pass (incl. `test_coverage_tool_executable`, `test_coverage_tool_plain_files`) - E2E: `local_path_override` this branch into a minimal external workspace, set `python_version = "3.14"` with `configure_coverage_tool = True`, run `bazel coverage` on a `py_test` → produces a non-empty per-test lcov with `SF:`, `FNDA:` reflecting actual call counts, and `LH:` non-zero --- CHANGELOG.md | 4 +- docs/coverage.md | 7 + docs/devguide.md | 2 +- python/private/coverage.patch | 6 +- python/private/coverage_deps.bzl | 130 +++++++++++------- .../update_deps/update_coverage_deps.py | 6 +- 6 files changed, 94 insertions(+), 61 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index c102d3b04e..7c29f9c127 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -54,7 +54,8 @@ END_UNRELEASED_TEMPLATE {#v0-0-0-removed} ### Removed -* Nothing removed. +* (coverage) Support for python 3.8 has been dropped from the bundled + `coverage.py` wheel set, since coverage.py 7.6.2 dropped it. {#v0-0-0-changed} ### Changed @@ -98,6 +99,7 @@ END_UNRELEASED_TEMPLATE * Python toolchain from [20260414] release. * (pypi) `package_metadata` support, fixes [#2054](https://github.com/bazel-contrib/rules_python/issues/2054). +* (coverage) Add support for python 3.14 and bump `coverage.py` to 7.10.7. [20260325]: https://github.com/astral-sh/python-build-standalone/releases/tag/20260325 [20260414]: https://github.com/astral-sh/python-build-standalone/releases/tag/20260414 diff --git a/docs/coverage.md b/docs/coverage.md index 3c7d9e0cfc..41c95a1b75 100644 --- a/docs/coverage.md +++ b/docs/coverage.md @@ -37,6 +37,13 @@ first in the import path. If you find yourself in this situation, then you'll need to manually configure coverage (see below). ::: +:::{note} +The bundled `coverage` wheel set covers CPython 3.9 through 3.14 (with +freethreaded variants for 3.13+). For Python versions outside that range, +`configure_coverage_tool = True` is a silent no-op and `bazel coverage` will +produce empty lcov data; manually configure coverage (see below) instead. +::: + ## Manually configuring coverage To manually configure coverage support, you'll need to set the diff --git a/docs/devguide.md b/docs/devguide.md index e88ed7a612..86a9d5b542 100644 --- a/docs/devguide.md +++ b/docs/devguide.md @@ -105,7 +105,7 @@ integration test. ``` bazel run //tools/private/update_deps:update_coverage_deps # for example: - # bazel run //tools/private/update_deps:update_coverage_deps 7.6.1 + # bazel run //tools/private/update_deps:update_coverage_deps 7.10.7 ``` ## Updating tool dependencies diff --git a/python/private/coverage.patch b/python/private/coverage.patch index 051f7fc543..9b3830332c 100644 --- a/python/private/coverage.patch +++ b/python/private/coverage.patch @@ -8,10 +8,6 @@ diff --git a/coverage/__main__.py b/coverage/__main__.py index ce2d8db..7d7d0a0 100644 --- a/coverage/__main__.py +++ b/coverage/__main__.py -@@ -6,5 +6,6 @@ - from __future__ import annotations - +@@ -8,1 +8,2 @@ import sys +sys.path.append(sys.path.pop(0)) - from coverage.cmdline import main - sys.exit(main()) diff --git a/python/private/coverage_deps.bzl b/python/private/coverage_deps.bzl index e80e8ee910..cd813196b5 100644 --- a/python/private/coverage_deps.bzl +++ b/python/private/coverage_deps.bzl @@ -23,118 +23,142 @@ load("//python/private:version_label.bzl", "version_label") _coverage_deps = { "cp310": { "aarch64-apple-darwin": ( - "https://files.pythonhosted.org/packages/7d/73/041928e434442bd3afde5584bdc3f932fb4562b1597629f537387cec6f3d/coverage-7.6.1-cp310-cp310-macosx_11_0_arm64.whl", - "cf4b19715bccd7ee27b6b120e7e9dd56037b9c0681dcc1adc9ba9db3d417fa36", + "https://files.pythonhosted.org/packages/03/94/952d30f180b1a916c11a56f5c22d3535e943aa22430e9e3322447e520e1c/coverage-7.10.7-cp310-cp310-macosx_11_0_arm64.whl", + "e201e015644e207139f7e2351980feb7040e6f4b2c2978892f3e3789d1c125e5", ), "aarch64-unknown-linux-gnu": ( - "https://files.pythonhosted.org/packages/c7/c8/6ca52b5147828e45ad0242388477fdb90df2c6cbb9a441701a12b3c71bc8/coverage-7.6.1-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", - "e61c0abb4c85b095a784ef23fdd4aede7a2628478e7baba7c5e3deba61070a02", + "https://files.pythonhosted.org/packages/60/83/5c283cff3d41285f8eab897651585db908a909c572bdc014bcfaf8a8b6ae/coverage-7.10.7-cp310-cp310-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", + "6be8ed3039ae7f7ac5ce058c308484787c86e8437e72b30bf5e88b8ea10f3c87", ), "x86_64-apple-darwin": ( - "https://files.pythonhosted.org/packages/7e/61/eb7ce5ed62bacf21beca4937a90fe32545c91a3c8a42a30c6616d48fc70d/coverage-7.6.1-cp310-cp310-macosx_10_9_x86_64.whl", - "b06079abebbc0e89e6163b8e8f0e16270124c154dc6e4a47b413dd538859af16", + "https://files.pythonhosted.org/packages/e5/6c/3a3f7a46888e69d18abe3ccc6fe4cb16cccb1e6a2f99698931dafca489e6/coverage-7.10.7-cp310-cp310-macosx_10_9_x86_64.whl", + "fc04cc7a3db33664e0c2d10eb8990ff6b3536f6842c9590ae8da4c614b9ed05a", ), "x86_64-unknown-linux-gnu": ( - "https://files.pythonhosted.org/packages/53/23/9e2c114d0178abc42b6d8d5281f651a8e6519abfa0ef460a00a91f80879d/coverage-7.6.1-cp310-cp310-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl", - "8f59d57baca39b32db42b83b2a7ba6f47ad9c394ec2076b084c3f029b7afca23", + "https://files.pythonhosted.org/packages/19/20/d0384ac06a6f908783d9b6aa6135e41b093971499ec488e47279f5b846e6/coverage-7.10.7-cp310-cp310-manylinux1_x86_64.manylinux_2_28_x86_64.manylinux_2_5_x86_64.whl", + "8421e088bc051361b01c4b3a50fd39a4b9133079a2229978d9d30511fd05231b", ), }, "cp311": { "aarch64-apple-darwin": ( - "https://files.pythonhosted.org/packages/e1/0e/e52332389e057daa2e03be1fbfef25bb4d626b37d12ed42ae6281d0a274c/coverage-7.6.1-cp311-cp311-macosx_11_0_arm64.whl", - "ed37bd3c3b063412f7620464a9ac1314d33100329f39799255fb8d3027da50d3", + "https://files.pythonhosted.org/packages/54/f0/514dcf4b4e3698b9a9077f084429681bf3aad2b4a72578f89d7f643eb506/coverage-7.10.7-cp311-cp311-macosx_11_0_arm64.whl", + "65646bb0359386e07639c367a22cf9b5bf6304e8630b565d0626e2bdf329227a", ), "aarch64-unknown-linux-gnu": ( - "https://files.pythonhosted.org/packages/aa/cd/766b45fb6e090f20f8927d9c7cb34237d41c73a939358bc881883fd3a40d/coverage-7.6.1-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", - "d85f5e9a5f8b73e2350097c3756ef7e785f55bd71205defa0bfdaf96c31616ff", + "https://files.pythonhosted.org/packages/a5/b6/bf054de41ec948b151ae2b79a55c107f5760979538f5fb80c195f2517718/coverage-7.10.7-cp311-cp311-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", + "4da86b6d62a496e908ac2898243920c7992499c1712ff7c2b6d837cc69d9467e", ), "x86_64-apple-darwin": ( - "https://files.pythonhosted.org/packages/ad/5f/67af7d60d7e8ce61a4e2ddcd1bd5fb787180c8d0ae0fbd073f903b3dd95d/coverage-7.6.1-cp311-cp311-macosx_10_9_x86_64.whl", - "7dea0889685db8550f839fa202744652e87c60015029ce3f60e006f8c4462c93", + "https://files.pythonhosted.org/packages/d2/5d/c1a17867b0456f2e9ce2d8d4708a4c3a089947d0bec9c66cdf60c9e7739f/coverage-7.10.7-cp311-cp311-macosx_10_9_x86_64.whl", + "a609f9c93113be646f44c2a0256d6ea375ad047005d7f57a5c15f614dc1b2f59", ), "x86_64-unknown-linux-gnu": ( - "https://files.pythonhosted.org/packages/14/6f/8351b465febb4dbc1ca9929505202db909c5a635c6fdf33e089bbc3d7d85/coverage-7.6.1-cp311-cp311-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl", - "0c0420b573964c760df9e9e86d1a9a622d0d27f417e1a949a8a66dd7bcee7bc6", + "https://files.pythonhosted.org/packages/b0/ef/bd8e719c2f7417ba03239052e099b76ea1130ac0cbb183ee1fcaa58aaff3/coverage-7.10.7-cp311-cp311-manylinux1_x86_64.manylinux_2_28_x86_64.manylinux_2_5_x86_64.whl", + "35f5e3f9e455bb17831876048355dca0f758b6df22f49258cb5a91da23ef437d", ), }, "cp312": { "aarch64-apple-darwin": ( - "https://files.pythonhosted.org/packages/e1/ab/6bf00de5327ecb8db205f9ae596885417a31535eeda6e7b99463108782e1/coverage-7.6.1-cp312-cp312-macosx_11_0_arm64.whl", - "5621a9175cf9d0b0c84c2ef2b12e9f5f5071357c4d2ea6ca1cf01814f45d2391", + "https://files.pythonhosted.org/packages/37/66/593f9be12fc19fb36711f19a5371af79a718537204d16ea1d36f16bd78d2/coverage-7.10.7-cp312-cp312-macosx_11_0_arm64.whl", + "18afb24843cbc175687225cab1138c95d262337f5473512010e46831aa0c2973", ), "aarch64-unknown-linux-gnu": ( - "https://files.pythonhosted.org/packages/92/8f/2ead05e735022d1a7f3a0a683ac7f737de14850395a826192f0288703472/coverage-7.6.1-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", - "260933720fdcd75340e7dbe9060655aff3af1f0c5d20f46b57f262ab6c86a5e8", + "https://files.pythonhosted.org/packages/98/2e/2dda59afd6103b342e096f246ebc5f87a3363b5412609946c120f4e7750d/coverage-7.10.7-cp312-cp312-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", + "c41e71c9cfb854789dee6fc51e46743a6d138b1803fab6cb860af43265b42ea6", ), "x86_64-apple-darwin": ( - "https://files.pythonhosted.org/packages/7e/d4/300fc921dff243cd518c7db3a4c614b7e4b2431b0d1145c1e274fd99bd70/coverage-7.6.1-cp312-cp312-macosx_10_9_x86_64.whl", - "95cae0efeb032af8458fc27d191f85d1717b1d4e49f7cb226cf526ff28179778", + "https://files.pythonhosted.org/packages/13/e4/eb12450f71b542a53972d19117ea5a5cea1cab3ac9e31b0b5d498df1bd5a/coverage-7.10.7-cp312-cp312-macosx_10_13_x86_64.whl", + "7bb3b9ddb87ef7725056572368040c32775036472d5a033679d1fa6c8dc08417", ), "x86_64-unknown-linux-gnu": ( - "https://files.pythonhosted.org/packages/1f/0f/c890339dd605f3ebc269543247bdd43b703cce6825b5ed42ff5f2d6122c7/coverage-7.6.1-cp312-cp312-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl", - "c44fee9975f04b33331cb8eb272827111efc8930cfd582e0320613263ca849ca", + "https://files.pythonhosted.org/packages/a6/90/a64aaacab3b37a17aaedd83e8000142561a29eb262cede42d94a67f7556b/coverage-7.10.7-cp312-cp312-manylinux1_x86_64.manylinux_2_28_x86_64.manylinux_2_5_x86_64.whl", + "314f2c326ded3f4b09be11bc282eb2fc861184bc95748ae67b360ac962770be7", ), }, "cp313": { "aarch64-apple-darwin": ( - "https://files.pythonhosted.org/packages/b9/67/e1413d5a8591622a46dd04ff80873b04c849268831ed5c304c16433e7e30/coverage-7.6.1-cp313-cp313-macosx_11_0_arm64.whl", - "a6d3adcf24b624a7b778533480e32434a39ad8fa30c315208f6d3e5542aeb6e9", + "https://files.pythonhosted.org/packages/72/4f/732fff31c119bb73b35236dd333030f32c4bfe909f445b423e6c7594f9a2/coverage-7.10.7-cp313-cp313-macosx_11_0_arm64.whl", + "73ab1601f84dc804f7812dc297e93cd99381162da39c47040a827d4e8dafe63b", ), "aarch64-apple-darwin-freethreaded": ( - "https://files.pythonhosted.org/packages/c4/ae/b5d58dff26cade02ada6ca612a76447acd69dccdbb3a478e9e088eb3d4b9/coverage-7.6.1-cp313-cp313t-macosx_11_0_arm64.whl", - "502753043567491d3ff6d08629270127e0c31d4184c4c8d98f92c26f65019962", + "https://files.pythonhosted.org/packages/11/0b/91128e099035ece15da3445d9015e4b4153a6059403452d324cbb0a575fa/coverage-7.10.7-cp313-cp313t-macosx_11_0_arm64.whl", + "dd5e856ebb7bfb7672b0086846db5afb4567a7b9714b8a0ebafd211ec7ce6a15", ), "aarch64-unknown-linux-gnu": ( - "https://files.pythonhosted.org/packages/14/5b/9dec847b305e44a5634d0fb8498d135ab1d88330482b74065fcec0622224/coverage-7.6.1-cp313-cp313-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", - "d0c212c49b6c10e6951362f7c6df3329f04c2b1c28499563d4035d964ab8e08c", + "https://files.pythonhosted.org/packages/b1/20/b6ea4f69bbb52dac0aebd62157ba6a9dddbfe664f5af8122dac296c3ee15/coverage-7.10.7-cp313-cp313-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", + "c79124f70465a150e89340de5963f936ee97097d2ef76c869708c4248c63ca49", ), "aarch64-unknown-linux-gnu-freethreaded": ( - "https://files.pythonhosted.org/packages/b8/d7/62095e355ec0613b08dfb19206ce3033a0eedb6f4a67af5ed267a8800642/coverage-7.6.1-cp313-cp313t-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", - "6a89ecca80709d4076b95f89f308544ec8f7b4727e8a547913a35f16717856cb", + "https://files.pythonhosted.org/packages/f7/08/16bee2c433e60913c610ea200b276e8eeef084b0d200bdcff69920bd5828/coverage-7.10.7-cp313-cp313t-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", + "83082a57783239717ceb0ad584de3c69cf581b2a95ed6bf81ea66034f00401c0", + ), + "x86_64-apple-darwin": ( + "https://files.pythonhosted.org/packages/9a/94/b765c1abcb613d103b64fcf10395f54d69b0ef8be6a0dd9c524384892cc7/coverage-7.10.7-cp313-cp313-macosx_10_13_x86_64.whl", + "981a651f543f2854abd3b5fcb3263aac581b18209be49863ba575de6edf4c14d", + ), + "x86_64-apple-darwin-freethreaded": ( + "https://files.pythonhosted.org/packages/bb/22/e04514bf2a735d8b0add31d2b4ab636fc02370730787c576bb995390d2d5/coverage-7.10.7-cp313-cp313t-macosx_10_13_x86_64.whl", + "a0ec07fd264d0745ee396b666d47cef20875f4ff2375d7c4f58235886cc1ef0c", ), "x86_64-unknown-linux-gnu": ( - "https://files.pythonhosted.org/packages/f7/95/d2fd31f1d638df806cae59d7daea5abf2b15b5234016a5ebb502c2f3f7ee/coverage-7.6.1-cp313-cp313-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl", - "78b260de9790fd81e69401c2dc8b17da47c8038176a79092a89cb2b7d945d060", + "https://files.pythonhosted.org/packages/a2/77/8c6d22bf61921a59bce5471c2f1f7ac30cd4ac50aadde72b8c48d5727902/coverage-7.10.7-cp313-cp313-manylinux1_x86_64.manylinux_2_28_x86_64.manylinux_2_5_x86_64.whl", + "10b6ba00ab1132a0ce4428ff68cf50a25efd6840a42cdf4239c9b99aad83be8b", ), "x86_64-unknown-linux-gnu-freethreaded": ( - "https://files.pythonhosted.org/packages/8b/61/a7a6a55dd266007ed3b1df7a3386a0d760d014542d72f7c2c6938483b7bd/coverage-7.6.1-cp313-cp313t-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl", - "13b0a73a0896988f053e4fbb7de6d93388e6dd292b0d87ee51d106f2c11b465b", + "https://files.pythonhosted.org/packages/5d/22/9b8d458c2881b22df3db5bb3e7369e63d527d986decb6c11a591ba2364f7/coverage-7.10.7-cp313-cp313t-manylinux1_x86_64.manylinux_2_28_x86_64.manylinux_2_5_x86_64.whl", + "1ef2319dd15a0b009667301a3f84452a4dc6fddfd06b0c5c53ea472d3989fbf0", ), }, - "cp38": { + "cp314": { "aarch64-apple-darwin": ( - "https://files.pythonhosted.org/packages/38/ea/cab2dc248d9f45b2b7f9f1f596a4d75a435cb364437c61b51d2eb33ceb0e/coverage-7.6.1-cp38-cp38-macosx_11_0_arm64.whl", - "f1adfc8ac319e1a348af294106bc6a8458a0f1633cc62a1446aebc30c5fa186a", + "https://files.pythonhosted.org/packages/f0/89/673f6514b0961d1f0e20ddc242e9342f6da21eaba3489901b565c0689f34/coverage-7.10.7-cp314-cp314-macosx_11_0_arm64.whl", + "212f8f2e0612778f09c55dd4872cb1f64a1f2b074393d139278ce902064d5b32", + ), + "aarch64-apple-darwin-freethreaded": ( + "https://files.pythonhosted.org/packages/f5/6f/f58d46f33db9f2e3647b2d0764704548c184e6f5e014bef528b7f979ef84/coverage-7.10.7-cp314-cp314t-macosx_11_0_arm64.whl", + "9fa6e4dd51fe15d8738708a973470f67a855ca50002294852e9571cdbd9433f2", ), "aarch64-unknown-linux-gnu": ( - "https://files.pythonhosted.org/packages/ca/6f/f82f9a500c7c5722368978a5390c418d2a4d083ef955309a8748ecaa8920/coverage-7.6.1-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", - "a95324a9de9650a729239daea117df21f4b9868ce32e63f8b650ebe6cef5595b", + "https://files.pythonhosted.org/packages/ff/49/07f00db9ac6478e4358165a08fb41b469a1b053212e8a00cb02f0d27a05f/coverage-7.10.7-cp314-cp314-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", + "813922f35bd800dca9994c5971883cbc0d291128a5de6b167c7aa697fcf59360", + ), + "aarch64-unknown-linux-gnu-freethreaded": ( + "https://files.pythonhosted.org/packages/84/fd/193a8fb132acfc0a901f72020e54be5e48021e1575bb327d8ee1097a28fd/coverage-7.10.7-cp314-cp314t-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", + "6e16e07d85ca0cf8bafe5f5d23a0b850064e8e945d5677492b06bbe6f09cc699", ), "x86_64-apple-darwin": ( - "https://files.pythonhosted.org/packages/81/d0/d9e3d554e38beea5a2e22178ddb16587dbcbe9a1ef3211f55733924bf7fa/coverage-7.6.1-cp38-cp38-macosx_10_9_x86_64.whl", - "6db04803b6c7291985a761004e9060b2bca08da6d04f26a7f2294b8623a0c1a0", + "https://files.pythonhosted.org/packages/23/9c/5844ab4ca6a4dd97a1850e030a15ec7d292b5c5cb93082979225126e35dd/coverage-7.10.7-cp314-cp314-macosx_10_13_x86_64.whl", + "b06f260b16ead11643a5a9f955bd4b5fd76c1a4c6796aeade8520095b75de520", + ), + "x86_64-apple-darwin-freethreaded": ( + "https://files.pythonhosted.org/packages/62/09/9a5608d319fa3eba7a2019addeacb8c746fb50872b57a724c9f79f146969/coverage-7.10.7-cp314-cp314t-macosx_10_13_x86_64.whl", + "a62c6ef0d50e6de320c270ff91d9dd0a05e7250cac2a800b7784bae474506e63", ), "x86_64-unknown-linux-gnu": ( - "https://files.pythonhosted.org/packages/e4/6e/885bcd787d9dd674de4a7d8ec83faf729534c63d05d51d45d4fa168f7102/coverage-7.6.1-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl", - "8929543a7192c13d177b770008bc4e8119f2e1f881d563fc6b6305d2d0ebe9de", + "https://files.pythonhosted.org/packages/82/62/14ed6546d0207e6eda876434e3e8475a3e9adbe32110ce896c9e0c06bb9a/coverage-7.10.7-cp314-cp314-manylinux1_x86_64.manylinux_2_28_x86_64.manylinux_2_5_x86_64.whl", + "bb45474711ba385c46a0bfe696c695a929ae69ac636cda8f532be9e8c93d720a", + ), + "x86_64-unknown-linux-gnu-freethreaded": ( + "https://files.pythonhosted.org/packages/0f/48/71a8abe9c1ad7e97548835e3cc1adbf361e743e9d60310c5f75c9e7bf847/coverage-7.10.7-cp314-cp314t-manylinux1_x86_64.manylinux_2_28_x86_64.manylinux_2_5_x86_64.whl", + "affef7c76a9ef259187ef31599a9260330e0335a3011732c4b9effa01e1cd6e0", ), }, "cp39": { "aarch64-apple-darwin": ( - "https://files.pythonhosted.org/packages/a5/fe/137d5dca72e4a258b1bc17bb04f2e0196898fe495843402ce826a7419fe3/coverage-7.6.1-cp39-cp39-macosx_11_0_arm64.whl", - "547f45fa1a93154bd82050a7f3cddbc1a7a4dd2a9bf5cb7d06f4ae29fe94eaf8", + "https://files.pythonhosted.org/packages/52/2f/b9f9daa39b80ece0b9548bbb723381e29bc664822d9a12c2135f8922c22b/coverage-7.10.7-cp39-cp39-macosx_11_0_arm64.whl", + "bc91b314cef27742da486d6839b677b3f2793dfe52b51bbbb7cf736d5c29281c", ), "aarch64-unknown-linux-gnu": ( - "https://files.pythonhosted.org/packages/78/5b/a0a796983f3201ff5485323b225d7c8b74ce30c11f456017e23d8e8d1945/coverage-7.6.1-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", - "645786266c8f18a931b65bfcefdbf6952dd0dea98feee39bd188607a9d307ed2", + "https://files.pythonhosted.org/packages/6a/92/1c1c5a9e8677ce56d42b97bdaca337b2d4d9ebe703d8c174ede52dbabd5f/coverage-7.10.7-cp39-cp39-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", + "c7315339eae3b24c2d2fa1ed7d7a38654cba34a13ef19fbcb9425da46d3dc594", ), "x86_64-apple-darwin": ( - "https://files.pythonhosted.org/packages/19/d3/d54c5aa83268779d54c86deb39c1c4566e5d45c155369ca152765f8db413/coverage-7.6.1-cp39-cp39-macosx_10_9_x86_64.whl", - "abd5fd0db5f4dc9289408aaf34908072f805ff7792632250dcb36dc591d24255", + "https://files.pythonhosted.org/packages/a3/ad/d1c25053764b4c42eb294aae92ab617d2e4f803397f9c7c8295caa77a260/coverage-7.10.7-cp39-cp39-macosx_10_9_x86_64.whl", + "fff7b9c3f19957020cac546c70025331113d2e61537f6e2441bc7657913de7d3", ), "x86_64-unknown-linux-gnu": ( - "https://files.pythonhosted.org/packages/9a/6f/eef79b779a540326fee9520e5542a8b428cc3bfa8b7c8f1022c1ee4fc66c/coverage-7.6.1-cp39-cp39-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl", - "609b06f178fe8e9f89ef676532760ec0b4deea15e9969bf754b37f7c40326dbc", + "https://files.pythonhosted.org/packages/b0/49/8a070782ce7e6b94ff6a0b6d7c65ba6bc3091d92a92cef4cd4eb0767965c/coverage-7.10.7-cp39-cp39-manylinux1_x86_64.manylinux_2_28_x86_64.manylinux_2_5_x86_64.whl", + "2af88deffcc8a4d5974cf2d502251bc3b2db8461f0b66d80a449c33757aa9f40", ), }, } diff --git a/tools/private/update_deps/update_coverage_deps.py b/tools/private/update_deps/update_coverage_deps.py index bbff67e927..81df6fc161 100755 --- a/tools/private/update_deps/update_coverage_deps.py +++ b/tools/private/update_deps/update_coverage_deps.py @@ -38,14 +38,18 @@ _supported_platforms = { # Windows is unsupported right now # "win_amd64": "x86_64-pc-windows-msvc", + "manylinux1_x86_64": "x86_64-unknown-linux-gnu", "manylinux2014_x86_64": "x86_64-unknown-linux-gnu", "manylinux2014_aarch64": "aarch64-unknown-linux-gnu", "macosx_11_0_arm64": "aarch64-apple-darwin", "macosx_10_9_x86_64": "x86_64-apple-darwin", + "macosx_10_13_x86_64": "x86_64-apple-darwin", + ("t", "manylinux1_x86_64"): "x86_64-unknown-linux-gnu-freethreaded", ("t", "manylinux2014_x86_64"): "x86_64-unknown-linux-gnu-freethreaded", ("t", "manylinux2014_aarch64"): "aarch64-unknown-linux-gnu-freethreaded", ("t", "macosx_11_0_arm64"): "aarch64-apple-darwin-freethreaded", ("t", "macosx_10_9_x86_64"): "x86_64-apple-darwin-freethreaded", + ("t", "macosx_10_13_x86_64"): "x86_64-apple-darwin-freethreaded", } @@ -143,7 +147,7 @@ def _parse_args() -> argparse.Namespace: "--py", nargs="+", type=str, - default=["cp38", "cp39", "cp310", "cp311", "cp312", "cp313"], + default=["cp39", "cp310", "cp311", "cp312", "cp313", "cp314"], help="Supported python versions", ) parser.add_argument( From 3871306fb4ca2a7abcf5029e89ad381095cb09b5 Mon Sep 17 00:00:00 2001 From: Ignas Anikevicius <240938+aignas@users.noreply.github.com> Date: Sun, 10 May 2026 14:36:55 +0900 Subject: [PATCH 737/922] fix(pypi): pass the correct versions to get_index_urls and fix cache invalidation (#3758) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Fix the versions of packages that we are recording to a `MODULE.bazel.lock` file facts by passing all of the versions and packages to the `get_index` function. Summary: - Parse ALL requirements files (not just platform-matched), pass all versions to get_index so lockfile facts are platform-independent. - Return None when versions mismatch (re-fetch), drop removed packages so you can start immediately. - Include files with no matching platforms so packages aren't lost - Add bzlmod lockfile integration test - Update CI to run new test in bazel-in-bazel subset - Bump Bazel 9.0.0rc1 → 9.1.0 in bazel-in-bazel tests Fixes #3756 --- .bazelci/presubmit.yml | 8 +- .bazelignore | 1 + .bazelrc.deleted_packages | 1 + CHANGELOG.md | 12 + MODULE.bazel | 4 +- python/private/pypi/parse_requirements.bzl | 28 +- python/private/pypi/pypi_cache.bzl | 9 + .../pypi/requirements_files_by_platform.bzl | 5 +- tests/integration/BUILD.bazel | 14 + tests/integration/bzlmod_lockfile/.bazelrc | 9 + .../integration/bzlmod_lockfile/.bazelversion | 1 + tests/integration/bzlmod_lockfile/BUILD.bazel | 10 + .../integration/bzlmod_lockfile/MODULE.bazel | 19 + .../bzlmod_lockfile/MODULE.bazel.lock | 683 ++++++++++++++++++ tests/integration/bzlmod_lockfile/README.md | 6 + tests/integration/bzlmod_lockfile/WORKSPACE | 0 .../bzlmod_lockfile/requirements_lock.txt | 8 + .../integration/bzlmod_lockfile/test_dummy.py | 17 + .../parse_requirements_tests.bzl | 134 ++++ tests/pypi/pypi_cache/pypi_cache_tests.bzl | 125 ++++ .../requirements_files_by_platform_tests.bzl | 67 ++ 21 files changed, 1150 insertions(+), 11 deletions(-) create mode 100644 tests/integration/bzlmod_lockfile/.bazelrc create mode 100644 tests/integration/bzlmod_lockfile/.bazelversion create mode 100644 tests/integration/bzlmod_lockfile/BUILD.bazel create mode 100644 tests/integration/bzlmod_lockfile/MODULE.bazel create mode 100644 tests/integration/bzlmod_lockfile/MODULE.bazel.lock create mode 100644 tests/integration/bzlmod_lockfile/README.md create mode 100644 tests/integration/bzlmod_lockfile/WORKSPACE create mode 100644 tests/integration/bzlmod_lockfile/requirements_lock.txt create mode 100644 tests/integration/bzlmod_lockfile/test_dummy.py diff --git a/.bazelci/presubmit.yml b/.bazelci/presubmit.yml index 1159c2f732..38d223cb8d 100644 --- a/.bazelci/presubmit.yml +++ b/.bazelci/presubmit.yml @@ -472,8 +472,8 @@ tasks: <<: *common_bazelinbazel_config name: "tests/integration bazel-in-bazel: macOS (subset)" platform: macos_arm64 - build_targets: ["//tests/integration:local_toolchains_test_bazel_self"] - test_targets: ["//tests/integration:local_toolchains_test_bazel_self"] + build_targets: ["//tests/integration:subset"] + test_targets: ["//tests/integration:subset"] # The bazelinbazel tests were disabled on Windows to save CI jobs slots, and # have bitrotted a bit. For now, just run a subset of what we're most # interested in. @@ -481,8 +481,8 @@ tasks: <<: *common_bazelinbazel_config name: "tests/integration bazel-in-bazel: Windows (subset)" platform: windows - build_targets: ["//tests/integration:local_toolchains_test_bazel_self"] - test_targets: ["//tests/integration:local_toolchains_test_bazel_self"] + build_targets: ["//tests/integration:subset"] + test_targets: ["//tests/integration:subset"] integration_test_compile_pip_requirements_ubuntu: <<: *reusable_build_test_all diff --git a/.bazelignore b/.bazelignore index 90e2c7dddd..afd162998a 100644 --- a/.bazelignore +++ b/.bazelignore @@ -30,6 +30,7 @@ gazelle/examples/bzlmod_build_file_generation/bazel-bzlmod_build_file_generation gazelle/examples/bzlmod_build_file_generation/bazel-out gazelle/examples/bzlmod_build_file_generation/bazel-testlog sphinxdocs +tests/integration/bzlmod_lockfile/bazel-bzlmod_lockfile tests/integration/compile_pip_requirements/bazel-compile_pip_requirements tests/integration/local_toolchains/bazel-local_toolchains tests/integration/py_cc_toolchain_registered/bazel-py_cc_toolchain_registered diff --git a/.bazelrc.deleted_packages b/.bazelrc.deleted_packages index 71a81de097..f4ea8527f3 100644 --- a/.bazelrc.deleted_packages +++ b/.bazelrc.deleted_packages @@ -29,6 +29,7 @@ common --deleted_packages=gazelle/modules_mapping common --deleted_packages=gazelle/python common --deleted_packages=gazelle/pythonconfig common --deleted_packages=gazelle/python/private +common --deleted_packages=tests/integration/bzlmod_lockfile common --deleted_packages=tests/integration/compile_pip_requirements common --deleted_packages=tests/integration/compile_pip_requirements_test_from_external_repo common --deleted_packages=tests/integration/custom_commands diff --git a/CHANGELOG.md b/CHANGELOG.md index 7c29f9c127..8fb3803a6a 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -104,6 +104,18 @@ END_UNRELEASED_TEMPLATE [20260325]: https://github.com/astral-sh/python-build-standalone/releases/tag/20260325 [20260414]: https://github.com/astral-sh/python-build-standalone/releases/tag/20260414 +{#v2-0-1} +## [2.0.1] - 2026-05-08 + +[2.0.1]: https://github.com/bazel-contrib/rules_python/releases/tag/2.0.1 + +{#v2-0-1-fixed} +### Fixed + +* (pypi) Fix the versions of packages that we are recording to a `MODULE.bazel.lock` file + facts by passing all of the versions to the `get_index` function. + Fixes [#3756](https://github.com/bazel-contrib/rules_python/issues/3756). + {#v2-0-0} ## [2.0.0] - 2026-04-09 diff --git a/MODULE.bazel b/MODULE.bazel index 6f24369367..ae007c4aaf 100644 --- a/MODULE.bazel +++ b/MODULE.bazel @@ -226,7 +226,7 @@ bazel_binaries.local( ) bazel_binaries.download(version = "7.7.0") bazel_binaries.download(version = "8.5.1") -bazel_binaries.download(version = "9.0.0rc1") +bazel_binaries.download(version = "9.1.0") use_repo( bazel_binaries, "bazel_binaries", @@ -235,7 +235,7 @@ use_repo( "bazel_binaries_bazelisk", "build_bazel_bazel_7_7_0", "build_bazel_bazel_8_5_1", - "build_bazel_bazel_9_0_0rc1", + "build_bazel_bazel_9_1_0", # "build_bazel_bazel_rolling", "build_bazel_bazel_self", ) diff --git a/python/private/pypi/parse_requirements.bzl b/python/private/pypi/parse_requirements.bzl index d047cc607d..07d0c0989e 100644 --- a/python/private/pypi/parse_requirements.bzl +++ b/python/private/pypi/parse_requirements.bzl @@ -89,6 +89,7 @@ def parse_requirements( evaluate_markers = evaluate_markers or (lambda _requirements: {}) options = {} requirements = {} + all_files_parsed = {} reqs_with_env_markers = {} index_url = None extra_index_urls = [] @@ -100,6 +101,13 @@ def parse_requirements( # needed for the whl_library declarations later. parse_result = parse_requirements_txt(contents) + # Save parsed results from ALL files, even those with no matching + # platforms. This ensures the distributions dict (used for index URL + # queries) includes packages from all platform files, making the + # lockfile facts platform-independent. + if file not in all_files_parsed: + all_files_parsed[file] = parse_result.requirements + tokenized_options = [] for opt in parse_result.options: for p in opt.split(" "): @@ -130,6 +138,10 @@ def parse_requirements( # This may call to Python, so execute it early (before calling to the # internet below) and ensure that we call it only once. + # + # TODO @aignas 2026-05-10: remove this assumption in the code because we + # are always using pipstar, so we can do the marker evaluation when we are + # parsing the files. resolved_marker_platforms = evaluate_markers(reqs_with_env_markers) logger.trace(lambda: "Evaluated env markers from:\n{}\n\nTo:\n{}".format( reqs_with_env_markers, @@ -185,13 +197,21 @@ def parse_requirements( index_urls = {} if get_index_urls: + # Collect all distributions from all requirements files irrespective + # of python_version and platform markers. This ensures that the index + # is queried for all packages, not just those matching the current + # platform's markers. distributions = {} - for reqs in requirements_by_platform.values(): - for req in reqs.values(): - if req.srcs.url: + for entries in all_files_parsed.values(): + for entry in entries: + name, req_line = entry + srcs = index_sources(req_line) + if srcs.url: continue + versions = distributions.setdefault(normalize_name(name), {}) + versions[srcs.version] = None - distributions.setdefault(req.distribution, []).append(req.srcs.version) + distributions = {k: sorted(v.keys()) for k, v in distributions.items()} index_urls = get_index_urls( ctx, diff --git a/python/private/pypi/pypi_cache.bzl b/python/private/pypi/pypi_cache.bzl index 972bbe04ae..d3a3034a79 100644 --- a/python/private/pypi/pypi_cache.bzl +++ b/python/private/pypi/pypi_cache.bzl @@ -212,6 +212,15 @@ def _get_from_facts(facts, known_facts, index_url, requested_versions, facts_ver requested_versions = requested_versions, ) if result: + if len(result) != len(requested_versions): + # If the results are incomplete, return None, so that we can + # fetch sources from the internet again. + return None + + # Only persist the accessed (requested) packages. Packages that + # exist in known_facts but are not in requested_versions (e.g. + # removed from all requirements files) are dropped from the + # computed facts so they get cleaned up from the lockfile. _store_facts(facts, facts_version, index_url, result) return result diff --git a/python/private/pypi/requirements_files_by_platform.bzl b/python/private/pypi/requirements_files_by_platform.bzl index 725c9984cc..dcdb6128a7 100644 --- a/python/private/pypi/requirements_files_by_platform.bzl +++ b/python/private/pypi/requirements_files_by_platform.bzl @@ -213,7 +213,6 @@ def requirements_files_by_platform( default_platforms, input_platforms, )) - continue if logger: logger.debug(lambda: "Configured platforms for file {} are {}".format(file, plats)) @@ -238,4 +237,8 @@ def requirements_files_by_platform( for plat, file in requirements.items(): ret.setdefault(file, []).append(_platform(plat, python_version = python_version)) + for file, _plats in files_by_platform: + if file not in ret and _plats != None: + ret[file] = [] + return ret diff --git a/tests/integration/BUILD.bazel b/tests/integration/BUILD.bazel index 5f2d20c103..9295cbb22f 100644 --- a/tests/integration/BUILD.bazel +++ b/tests/integration/BUILD.bazel @@ -37,6 +37,20 @@ default_test_runner( visibility = ["//visibility:public"], ) +rules_python_integration_test( + name = "bzlmod_lockfile_test", + bazel_versions = ["9.1.0"], +) + +test_suite( + name = "subset", + tags = ["manual"], + tests = [ + "bzlmod_lockfile_test_bazel_9.1.0", + "local_toolchains_test_bazel_self", + ], +) + rules_python_integration_test( name = "compile_pip_requirements_test", ) diff --git a/tests/integration/bzlmod_lockfile/.bazelrc b/tests/integration/bzlmod_lockfile/.bazelrc new file mode 100644 index 0000000000..3687a11fae --- /dev/null +++ b/tests/integration/bzlmod_lockfile/.bazelrc @@ -0,0 +1,9 @@ +# Bazel configuration flags + +build --enable_runfiles +common --lockfile_mode=error + +common --experimental_isolated_extension_usages + +# https://docs.bazel.build/versions/main/best-practices.html#using-the-bazelrc-file +try-import %workspace%/user.bazelrc diff --git a/tests/integration/bzlmod_lockfile/.bazelversion b/tests/integration/bzlmod_lockfile/.bazelversion new file mode 100644 index 0000000000..47da986f86 --- /dev/null +++ b/tests/integration/bzlmod_lockfile/.bazelversion @@ -0,0 +1 @@ +9.1.0 diff --git a/tests/integration/bzlmod_lockfile/BUILD.bazel b/tests/integration/bzlmod_lockfile/BUILD.bazel new file mode 100644 index 0000000000..7e99a242ae --- /dev/null +++ b/tests/integration/bzlmod_lockfile/BUILD.bazel @@ -0,0 +1,10 @@ +load("@rules_python//python:py_test.bzl", "py_test") + +py_test( + name = "test_dummy", + srcs = ["test_dummy.py"], + deps = [ + "@pypi//pycparser", + "@pypi//six", + ], +) diff --git a/tests/integration/bzlmod_lockfile/MODULE.bazel b/tests/integration/bzlmod_lockfile/MODULE.bazel new file mode 100644 index 0000000000..6c074bcdd4 --- /dev/null +++ b/tests/integration/bzlmod_lockfile/MODULE.bazel @@ -0,0 +1,19 @@ +module(name = "bzlmod_lockfile") + +bazel_dep(name = "rules_python") +local_path_override( + module_name = "rules_python", + path = "../../..", +) + +python = use_extension("@rules_python//python/extensions:python.bzl", "python") +python.toolchain(python_version = "3.13") + +# TODO: This test module should also verify that isolate = True works, will do in a followup PR. +pip = use_extension("@rules_python//python/extensions:pip.bzl", "pip") +pip.parse( + hub_name = "pypi", + python_version = "3.13", + requirements_lock = "//:requirements_lock.txt", +) +use_repo(pip, "pypi") diff --git a/tests/integration/bzlmod_lockfile/MODULE.bazel.lock b/tests/integration/bzlmod_lockfile/MODULE.bazel.lock new file mode 100644 index 0000000000..be296a56ac --- /dev/null +++ b/tests/integration/bzlmod_lockfile/MODULE.bazel.lock @@ -0,0 +1,683 @@ +{ + "lockFileVersion": 26, + "registryFileHashes": { + "https://bcr.bazel.build/bazel_registry.json": "8a28e4aff06ee60aed2a8c281907fb8bcbf3b753c91fb5a5c57da3215d5b3497", + "https://bcr.bazel.build/modules/abseil-cpp/20210324.2/MODULE.bazel": "7cd0312e064fde87c8d1cd79ba06c876bd23630c83466e9500321be55c96ace2", + "https://bcr.bazel.build/modules/abseil-cpp/20211102.0/MODULE.bazel": "70390338f7a5106231d20620712f7cccb659cd0e9d073d1991c038eb9fc57589", + "https://bcr.bazel.build/modules/abseil-cpp/20230125.1/MODULE.bazel": "89047429cb0207707b2dface14ba7f8df85273d484c2572755be4bab7ce9c3a0", + "https://bcr.bazel.build/modules/abseil-cpp/20230802.0.bcr.1/MODULE.bazel": "1c8cec495288dccd14fdae6e3f95f772c1c91857047a098fad772034264cc8cb", + "https://bcr.bazel.build/modules/abseil-cpp/20230802.0/MODULE.bazel": "d253ae36a8bd9ee3c5955384096ccb6baf16a1b1e93e858370da0a3b94f77c16", + "https://bcr.bazel.build/modules/abseil-cpp/20230802.1/MODULE.bazel": "fa92e2eb41a04df73cdabeec37107316f7e5272650f81d6cc096418fe647b915", + "https://bcr.bazel.build/modules/abseil-cpp/20240116.1/MODULE.bazel": "37bcdb4440fbb61df6a1c296ae01b327f19e9bb521f9b8e26ec854b6f97309ed", + "https://bcr.bazel.build/modules/abseil-cpp/20240116.2/MODULE.bazel": "73939767a4686cd9a520d16af5ab440071ed75cec1a876bf2fcfaf1f71987a16", + "https://bcr.bazel.build/modules/abseil-cpp/20250127.1/MODULE.bazel": "c4a89e7ceb9bf1e25cf84a9f830ff6b817b72874088bf5141b314726e46a57c1", + "https://bcr.bazel.build/modules/abseil-cpp/20250512.1/MODULE.bazel": "d209fdb6f36ffaf61c509fcc81b19e81b411a999a934a032e10cd009a0226215", + "https://bcr.bazel.build/modules/abseil-cpp/20250814.1/MODULE.bazel": "51f2312901470cdab0dbdf3b88c40cd21c62a7ed58a3de45b365ddc5b11bcab2", + "https://bcr.bazel.build/modules/abseil-cpp/20250814.1/source.json": "cea3901d7e299da7320700abbaafe57a65d039f10d0d7ea601c4a66938ea4b0c", + "https://bcr.bazel.build/modules/apple_support/1.11.1/MODULE.bazel": "1843d7cd8a58369a444fc6000e7304425fba600ff641592161d9f15b179fb896", + "https://bcr.bazel.build/modules/apple_support/1.15.1/MODULE.bazel": "a0556fefca0b1bb2de8567b8827518f94db6a6e7e7d632b4c48dc5f865bc7c85", + "https://bcr.bazel.build/modules/apple_support/1.21.0/MODULE.bazel": "ac1824ed5edf17dee2fdd4927ada30c9f8c3b520be1b5fd02a5da15bc10bff3e", + "https://bcr.bazel.build/modules/apple_support/1.21.1/MODULE.bazel": "5809fa3efab15d1f3c3c635af6974044bac8a4919c62238cce06acee8a8c11f1", + "https://bcr.bazel.build/modules/apple_support/1.24.2/MODULE.bazel": "0e62471818affb9f0b26f128831d5c40b074d32e6dda5a0d3852847215a41ca4", + "https://bcr.bazel.build/modules/apple_support/1.24.2/source.json": "2c22c9827093250406c5568da6c54e6fdf0ef06238def3d99c71b12feb057a8d", + "https://bcr.bazel.build/modules/bazel_features/1.10.0/MODULE.bazel": "f75e8807570484a99be90abcd52b5e1f390362c258bcb73106f4544957a48101", + "https://bcr.bazel.build/modules/bazel_features/1.11.0/MODULE.bazel": "f9382337dd5a474c3b7d334c2f83e50b6eaedc284253334cf823044a26de03e8", + "https://bcr.bazel.build/modules/bazel_features/1.15.0/MODULE.bazel": "d38ff6e517149dc509406aca0db3ad1efdd890a85e049585b7234d04238e2a4d", + "https://bcr.bazel.build/modules/bazel_features/1.17.0/MODULE.bazel": "039de32d21b816b47bd42c778e0454217e9c9caac4a3cf8e15c7231ee3ddee4d", + "https://bcr.bazel.build/modules/bazel_features/1.18.0/MODULE.bazel": "1be0ae2557ab3a72a57aeb31b29be347bcdc5d2b1eb1e70f39e3851a7e97041a", + "https://bcr.bazel.build/modules/bazel_features/1.19.0/MODULE.bazel": "59adcdf28230d220f0067b1f435b8537dd033bfff8db21335ef9217919c7fb58", + "https://bcr.bazel.build/modules/bazel_features/1.21.0/MODULE.bazel": "675642261665d8eea09989aa3b8afb5c37627f1be178382c320d1b46afba5e3b", + "https://bcr.bazel.build/modules/bazel_features/1.23.0/MODULE.bazel": "fd1ac84bc4e97a5a0816b7fd7d4d4f6d837b0047cf4cbd81652d616af3a6591a", + "https://bcr.bazel.build/modules/bazel_features/1.27.0/MODULE.bazel": "621eeee06c4458a9121d1f104efb80f39d34deff4984e778359c60eaf1a8cb65", + "https://bcr.bazel.build/modules/bazel_features/1.28.0/MODULE.bazel": "4b4200e6cbf8fa335b2c3f43e1d6ef3e240319c33d43d60cc0fbd4b87ece299d", + "https://bcr.bazel.build/modules/bazel_features/1.3.0/MODULE.bazel": "cdcafe83ec318cda34e02948e81d790aab8df7a929cec6f6969f13a489ccecd9", + "https://bcr.bazel.build/modules/bazel_features/1.30.0/MODULE.bazel": "a14b62d05969a293b80257e72e597c2da7f717e1e69fa8b339703ed6731bec87", + "https://bcr.bazel.build/modules/bazel_features/1.33.0/MODULE.bazel": "8b8dc9d2a4c88609409c3191165bccec0e4cb044cd7a72ccbe826583303459f6", + "https://bcr.bazel.build/modules/bazel_features/1.4.1/MODULE.bazel": "e45b6bb2350aff3e442ae1111c555e27eac1d915e77775f6fdc4b351b758b5d7", + "https://bcr.bazel.build/modules/bazel_features/1.42.1/MODULE.bazel": "275a59b5406ff18c01739860aa70ad7ccb3cfb474579411decca11c93b951080", + "https://bcr.bazel.build/modules/bazel_features/1.42.1/source.json": "fcd4396b2df85f64f2b3bb436ad870793ecf39180f1d796f913cc9276d355309", + "https://bcr.bazel.build/modules/bazel_skylib/1.0.3/MODULE.bazel": "bcb0fd896384802d1ad283b4e4eb4d718eebd8cb820b0a2c3a347fb971afd9d8", + "https://bcr.bazel.build/modules/bazel_skylib/1.1.1/MODULE.bazel": "1add3e7d93ff2e6998f9e118022c84d163917d912f5afafb3058e3d2f1545b5e", + "https://bcr.bazel.build/modules/bazel_skylib/1.2.0/MODULE.bazel": "44fe84260e454ed94ad326352a698422dbe372b21a1ac9f3eab76eb531223686", + "https://bcr.bazel.build/modules/bazel_skylib/1.2.1/MODULE.bazel": "f35baf9da0efe45fa3da1696ae906eea3d615ad41e2e3def4aeb4e8bc0ef9a7a", + "https://bcr.bazel.build/modules/bazel_skylib/1.3.0/MODULE.bazel": "20228b92868bf5cfc41bda7afc8a8ba2a543201851de39d990ec957b513579c5", + "https://bcr.bazel.build/modules/bazel_skylib/1.4.1/MODULE.bazel": "a0dcb779424be33100dcae821e9e27e4f2901d9dfd5333efe5ac6a8d7ab75e1d", + "https://bcr.bazel.build/modules/bazel_skylib/1.4.2/MODULE.bazel": "3bd40978e7a1fac911d5989e6b09d8f64921865a45822d8b09e815eaa726a651", + "https://bcr.bazel.build/modules/bazel_skylib/1.5.0/MODULE.bazel": "32880f5e2945ce6a03d1fbd588e9198c0a959bb42297b2cfaf1685b7bc32e138", + "https://bcr.bazel.build/modules/bazel_skylib/1.6.1/MODULE.bazel": "8fdee2dbaace6c252131c00e1de4b165dc65af02ea278476187765e1a617b917", + "https://bcr.bazel.build/modules/bazel_skylib/1.7.0/MODULE.bazel": "0db596f4563de7938de764cc8deeabec291f55e8ec15299718b93c4423e9796d", + "https://bcr.bazel.build/modules/bazel_skylib/1.7.1/MODULE.bazel": "3120d80c5861aa616222ec015332e5f8d3171e062e3e804a2a0253e1be26e59b", + "https://bcr.bazel.build/modules/bazel_skylib/1.8.1/MODULE.bazel": "88ade7293becda963e0e3ea33e7d54d3425127e0a326e0d17da085a5f1f03ff6", + "https://bcr.bazel.build/modules/bazel_skylib/1.8.2/MODULE.bazel": "69ad6927098316848b34a9142bcc975e018ba27f08c4ff403f50c1b6e646ca67", + "https://bcr.bazel.build/modules/bazel_skylib/1.8.2/source.json": "34a3c8bcf233b835eb74be9d628899bb32999d3e0eadef1947a0a562a2b16ffb", + "https://bcr.bazel.build/modules/buildozer/8.5.1/MODULE.bazel": "a35d9561b3fc5b18797c330793e99e3b834a473d5fbd3d7d7634aafc9bdb6f8f", + "https://bcr.bazel.build/modules/buildozer/8.5.1/source.json": "e3386e6ff4529f2442800dee47ad28d3e6487f36a1f75ae39ae56c70f0cd2fbd", + "https://bcr.bazel.build/modules/google_benchmark/1.8.2/MODULE.bazel": "a70cf1bba851000ba93b58ae2f6d76490a9feb74192e57ab8e8ff13c34ec50cb", + "https://bcr.bazel.build/modules/googletest/1.11.0/MODULE.bazel": "3a83f095183f66345ca86aa13c58b59f9f94a2f81999c093d4eeaa2d262d12f4", + "https://bcr.bazel.build/modules/googletest/1.14.0.bcr.1/MODULE.bazel": "22c31a561553727960057361aa33bf20fb2e98584bc4fec007906e27053f80c6", + "https://bcr.bazel.build/modules/googletest/1.14.0/MODULE.bazel": "cfbcbf3e6eac06ef9d85900f64424708cc08687d1b527f0ef65aa7517af8118f", + "https://bcr.bazel.build/modules/googletest/1.15.2/MODULE.bazel": "6de1edc1d26cafb0ea1a6ab3f4d4192d91a312fd2d360b63adaa213cd00b2108", + "https://bcr.bazel.build/modules/googletest/1.17.0/MODULE.bazel": "dbec758171594a705933a29fcf69293d2468c49ec1f2ebca65c36f504d72df46", + "https://bcr.bazel.build/modules/googletest/1.17.0/source.json": "38e4454b25fc30f15439c0378e57909ab1fd0a443158aa35aec685da727cd713", + "https://bcr.bazel.build/modules/jsoncpp/1.9.5/MODULE.bazel": "31271aedc59e815656f5736f282bb7509a97c7ecb43e927ac1a37966e0578075", + "https://bcr.bazel.build/modules/jsoncpp/1.9.6/MODULE.bazel": "2f8d20d3b7d54143213c4dfc3d98225c42de7d666011528dc8fe91591e2e17b0", + "https://bcr.bazel.build/modules/jsoncpp/1.9.6/source.json": "a04756d367a2126c3541682864ecec52f92cdee80a35735a3cb249ce015ca000", + "https://bcr.bazel.build/modules/libpfm/4.11.0/MODULE.bazel": "45061ff025b301940f1e30d2c16bea596c25b176c8b6b3087e92615adbd52902", + "https://bcr.bazel.build/modules/nlohmann_json/3.6.1/MODULE.bazel": "6f7b417dcc794d9add9e556673ad25cb3ba835224290f4f848f8e2db1e1fca74", + "https://bcr.bazel.build/modules/nlohmann_json/3.6.1/source.json": "f448c6e8963fdfa7eb831457df83ad63d3d6355018f6574fb017e8169deb43a9", + "https://bcr.bazel.build/modules/package_metadata/0.0.7/MODULE.bazel": "7adb03933fc8401f495800cf4eafcff0edc6da0ff55c7db223ef69d19f689486", + "https://bcr.bazel.build/modules/package_metadata/0.0.7/source.json": "50639625e937b56115012674c797cca7a05a96b4878c87d803c13dc2b31de8a0", + "https://bcr.bazel.build/modules/platforms/0.0.10/MODULE.bazel": "8cb8efaf200bdeb2150d93e162c40f388529a25852b332cec879373771e48ed5", + "https://bcr.bazel.build/modules/platforms/0.0.11/MODULE.bazel": "0daefc49732e227caa8bfa834d65dc52e8cc18a2faf80df25e8caea151a9413f", + "https://bcr.bazel.build/modules/platforms/0.0.4/MODULE.bazel": "9b328e31ee156f53f3c416a64f8491f7eb731742655a47c9eec4703a71644aee", + "https://bcr.bazel.build/modules/platforms/0.0.5/MODULE.bazel": "5733b54ea419d5eaf7997054bb55f6a1d0b5ff8aedf0176fef9eea44f3acda37", + "https://bcr.bazel.build/modules/platforms/0.0.6/MODULE.bazel": "ad6eeef431dc52aefd2d77ed20a4b353f8ebf0f4ecdd26a807d2da5aa8cd0615", + "https://bcr.bazel.build/modules/platforms/0.0.7/MODULE.bazel": "72fd4a0ede9ee5c021f6a8dd92b503e089f46c227ba2813ff183b71616034814", + "https://bcr.bazel.build/modules/platforms/0.0.8/MODULE.bazel": "9f142c03e348f6d263719f5074b21ef3adf0b139ee4c5133e2aa35664da9eb2d", + "https://bcr.bazel.build/modules/platforms/0.0.9/MODULE.bazel": "4a87a60c927b56ddd67db50c89acaa62f4ce2a1d2149ccb63ffd871d5ce29ebc", + "https://bcr.bazel.build/modules/platforms/1.0.0/MODULE.bazel": "f05feb42b48f1b3c225e4ccf351f367be0371411a803198ec34a389fb22aa580", + "https://bcr.bazel.build/modules/platforms/1.0.0/source.json": "f4ff1fd412e0246fd38c82328eb209130ead81d62dcd5a9e40910f867f733d96", + "https://bcr.bazel.build/modules/protobuf/21.7/MODULE.bazel": "a5a29bb89544f9b97edce05642fac225a808b5b7be74038ea3640fae2f8e66a7", + "https://bcr.bazel.build/modules/protobuf/27.0/MODULE.bazel": "7873b60be88844a0a1d8f80b9d5d20cfbd8495a689b8763e76c6372998d3f64c", + "https://bcr.bazel.build/modules/protobuf/29.0-rc2/MODULE.bazel": "6241d35983510143049943fc0d57937937122baf1b287862f9dc8590fc4c37df", + "https://bcr.bazel.build/modules/protobuf/29.1/MODULE.bazel": "557c3457560ff49e122ed76c0bc3397a64af9574691cb8201b4e46d4ab2ecb95", + "https://bcr.bazel.build/modules/protobuf/3.19.0/MODULE.bazel": "6b5fbb433f760a99a22b18b6850ed5784ef0e9928a72668b66e4d7ccd47db9b0", + "https://bcr.bazel.build/modules/protobuf/32.1/MODULE.bazel": "89cd2866a9cb07fee9ff74c41ceace11554f32e0d849de4e23ac55515cfada4d", + "https://bcr.bazel.build/modules/protobuf/33.4/MODULE.bazel": "114775b816b38b6d0ca620450d6b02550c60ceedfdc8d9a229833b34a223dc42", + "https://bcr.bazel.build/modules/protobuf/33.4/source.json": "555f8686b4c7d6b5ba731fbea13bf656b4bfd9a7ff629c1d9d3f6e1d6155de79", + "https://bcr.bazel.build/modules/pybind11_bazel/2.11.1/MODULE.bazel": "88af1c246226d87e65be78ed49ecd1e6f5e98648558c14ce99176da041dc378e", + "https://bcr.bazel.build/modules/pybind11_bazel/2.12.0/MODULE.bazel": "e6f4c20442eaa7c90d7190d8dc539d0ab422f95c65a57cc59562170c58ae3d34", + "https://bcr.bazel.build/modules/pybind11_bazel/2.12.0/source.json": "6900fdc8a9e95866b8c0d4ad4aba4d4236317b5c1cd04c502df3f0d33afed680", + "https://bcr.bazel.build/modules/re2/2023-09-01/MODULE.bazel": "cb3d511531b16cfc78a225a9e2136007a48cf8a677e4264baeab57fe78a80206", + "https://bcr.bazel.build/modules/re2/2024-07-02.bcr.1/MODULE.bazel": "b4963dda9b31080be1905ef085ecd7dd6cd47c05c79b9cdf83ade83ab2ab271a", + "https://bcr.bazel.build/modules/re2/2024-07-02.bcr.1/source.json": "2ff292be6ef3340325ce8a045ecc326e92cbfab47c7cbab4bd85d28971b97ac4", + "https://bcr.bazel.build/modules/re2/2024-07-02/MODULE.bazel": "0eadc4395959969297cbcf31a249ff457f2f1d456228c67719480205aa306daa", + "https://bcr.bazel.build/modules/rules_android/0.1.1/MODULE.bazel": "48809ab0091b07ad0182defb787c4c5328bd3a278938415c00a7b69b50c4d3a8", + "https://bcr.bazel.build/modules/rules_android/0.1.1/source.json": "e6986b41626ee10bdc864937ffb6d6bf275bb5b9c65120e6137d56e6331f089e", + "https://bcr.bazel.build/modules/rules_apple/3.16.0/MODULE.bazel": "0d1caf0b8375942ce98ea944be754a18874041e4e0459401d925577624d3a54a", + "https://bcr.bazel.build/modules/rules_apple/4.1.0/MODULE.bazel": "76e10fd4a48038d3fc7c5dc6e63b7063bbf5304a2e3bd42edda6ec660eebea68", + "https://bcr.bazel.build/modules/rules_apple/4.1.0/source.json": "8ee81e1708756f81b343a5eb2b2f0b953f1d25c4ab3d4a68dc02754872e80715", + "https://bcr.bazel.build/modules/rules_cc/0.0.1/MODULE.bazel": "cb2aa0747f84c6c3a78dad4e2049c154f08ab9d166b1273835a8174940365647", + "https://bcr.bazel.build/modules/rules_cc/0.0.10/MODULE.bazel": "ec1705118f7eaedd6e118508d3d26deba2a4e76476ada7e0e3965211be012002", + "https://bcr.bazel.build/modules/rules_cc/0.0.13/MODULE.bazel": "0e8529ed7b323dad0775ff924d2ae5af7640b23553dfcd4d34344c7e7a867191", + "https://bcr.bazel.build/modules/rules_cc/0.0.15/MODULE.bazel": "6704c35f7b4a72502ee81f61bf88706b54f06b3cbe5558ac17e2e14666cd5dcc", + "https://bcr.bazel.build/modules/rules_cc/0.0.16/MODULE.bazel": "7661303b8fc1b4d7f532e54e9d6565771fea666fbdf839e0a86affcd02defe87", + "https://bcr.bazel.build/modules/rules_cc/0.0.17/MODULE.bazel": "2ae1d8f4238ec67d7185d8861cb0a2cdf4bc608697c331b95bf990e69b62e64a", + "https://bcr.bazel.build/modules/rules_cc/0.0.2/MODULE.bazel": "6915987c90970493ab97393024c156ea8fb9f3bea953b2f3ec05c34f19b5695c", + "https://bcr.bazel.build/modules/rules_cc/0.0.6/MODULE.bazel": "abf360251023dfe3efcef65ab9d56beefa8394d4176dd29529750e1c57eaa33f", + "https://bcr.bazel.build/modules/rules_cc/0.0.8/MODULE.bazel": "964c85c82cfeb6f3855e6a07054fdb159aced38e99a5eecf7bce9d53990afa3e", + "https://bcr.bazel.build/modules/rules_cc/0.0.9/MODULE.bazel": "836e76439f354b89afe6a911a7adf59a6b2518fafb174483ad78a2a2fde7b1c5", + "https://bcr.bazel.build/modules/rules_cc/0.1.1/MODULE.bazel": "2f0222a6f229f0bf44cd711dc13c858dad98c62d52bd51d8fc3a764a83125513", + "https://bcr.bazel.build/modules/rules_cc/0.1.2/MODULE.bazel": "557ddc3a96858ec0d465a87c0a931054d7dcfd6583af2c7ed3baf494407fd8d0", + "https://bcr.bazel.build/modules/rules_cc/0.1.5/MODULE.bazel": "88dfc9361e8b5ae1008ac38f7cdfd45ad738e4fa676a3ad67d19204f045a1fd8", + "https://bcr.bazel.build/modules/rules_cc/0.2.0/MODULE.bazel": "b5c17f90458caae90d2ccd114c81970062946f49f355610ed89bebf954f5783c", + "https://bcr.bazel.build/modules/rules_cc/0.2.13/MODULE.bazel": "eecdd666eda6be16a8d9dc15e44b5c75133405e820f620a234acc4b1fdc5aa37", + "https://bcr.bazel.build/modules/rules_cc/0.2.17/MODULE.bazel": "1849602c86cb60da8613d2de887f9566a6d354a6df6d7009f9d04a14402f9a84", + "https://bcr.bazel.build/modules/rules_cc/0.2.17/source.json": "3832f45d145354049137c0090df04629d9c2b5493dc5c2bf46f1834040133a07", + "https://bcr.bazel.build/modules/rules_cc/0.2.8/MODULE.bazel": "f1df20f0bf22c28192a794f29b501ee2018fa37a3862a1a2132ae2940a23a642", + "https://bcr.bazel.build/modules/rules_foreign_cc/0.9.0/MODULE.bazel": "c9e8c682bf75b0e7c704166d79b599f93b72cfca5ad7477df596947891feeef6", + "https://bcr.bazel.build/modules/rules_fuzzing/0.5.2/MODULE.bazel": "40c97d1144356f52905566c55811f13b299453a14ac7769dfba2ac38192337a8", + "https://bcr.bazel.build/modules/rules_java/4.0.0/MODULE.bazel": "5a78a7ae82cd1a33cef56dc578c7d2a46ed0dca12643ee45edbb8417899e6f74", + "https://bcr.bazel.build/modules/rules_java/5.3.5/MODULE.bazel": "a4ec4f2db570171e3e5eb753276ee4b389bae16b96207e9d3230895c99644b86", + "https://bcr.bazel.build/modules/rules_java/6.5.2/MODULE.bazel": "1d440d262d0e08453fa0c4d8f699ba81609ed0e9a9a0f02cd10b3e7942e61e31", + "https://bcr.bazel.build/modules/rules_java/7.10.0/MODULE.bazel": "530c3beb3067e870561739f1144329a21c851ff771cd752a49e06e3dc9c2e71a", + "https://bcr.bazel.build/modules/rules_java/7.12.2/MODULE.bazel": "579c505165ee757a4280ef83cda0150eea193eed3bef50b1004ba88b99da6de6", + "https://bcr.bazel.build/modules/rules_java/7.2.0/MODULE.bazel": "06c0334c9be61e6cef2c8c84a7800cef502063269a5af25ceb100b192453d4ab", + "https://bcr.bazel.build/modules/rules_java/7.6.1/MODULE.bazel": "2f14b7e8a1aa2f67ae92bc69d1ec0fa8d9f827c4e17ff5e5f02e91caa3b2d0fe", + "https://bcr.bazel.build/modules/rules_java/8.6.1/MODULE.bazel": "f4808e2ab5b0197f094cabce9f4b006a27766beb6a9975931da07099560ca9c2", + "https://bcr.bazel.build/modules/rules_java/9.1.0/MODULE.bazel": "ee63f27e36a3fada80342869361182f120a9819c74320e8e65b1e04ba0cd7a9d", + "https://bcr.bazel.build/modules/rules_java/9.1.0/source.json": "da589573c1dee2c9ac4a568b301269a2e8191110ff0345c1a959fa7ea6c4dfd6", + "https://bcr.bazel.build/modules/rules_jvm_external/4.4.2/MODULE.bazel": "a56b85e418c83eb1839819f0b515c431010160383306d13ec21959ac412d2fe7", + "https://bcr.bazel.build/modules/rules_jvm_external/5.1/MODULE.bazel": "33f6f999e03183f7d088c9be518a63467dfd0be94a11d0055fe2d210f89aa909", + "https://bcr.bazel.build/modules/rules_jvm_external/5.2/MODULE.bazel": "d9351ba35217ad0de03816ef3ed63f89d411349353077348a45348b096615036", + "https://bcr.bazel.build/modules/rules_jvm_external/6.3/MODULE.bazel": "c998e060b85f71e00de5ec552019347c8bca255062c990ac02d051bb80a38df0", + "https://bcr.bazel.build/modules/rules_jvm_external/6.7/MODULE.bazel": "e717beabc4d091ecb2c803c2d341b88590e9116b8bf7947915eeb33aab4f96dd", + "https://bcr.bazel.build/modules/rules_jvm_external/6.7/source.json": "5426f412d0a7fc6b611643376c7e4a82dec991491b9ce5cb1cfdd25fe2e92be4", + "https://bcr.bazel.build/modules/rules_kotlin/1.9.6/MODULE.bazel": "d269a01a18ee74d0335450b10f62c9ed81f2321d7958a2934e44272fe82dcef3", + "https://bcr.bazel.build/modules/rules_kotlin/1.9.6/source.json": "2faa4794364282db7c06600b7e5e34867a564ae91bda7cae7c29c64e9466b7d5", + "https://bcr.bazel.build/modules/rules_license/0.0.3/MODULE.bazel": "627e9ab0247f7d1e05736b59dbb1b6871373de5ad31c3011880b4133cafd4bd0", + "https://bcr.bazel.build/modules/rules_license/0.0.7/MODULE.bazel": "088fbeb0b6a419005b89cf93fe62d9517c0a2b8bb56af3244af65ecfe37e7d5d", + "https://bcr.bazel.build/modules/rules_license/1.0.0/MODULE.bazel": "a7fda60eefdf3d8c827262ba499957e4df06f659330bbe6cdbdb975b768bb65c", + "https://bcr.bazel.build/modules/rules_license/1.0.0/source.json": "a52c89e54cc311196e478f8382df91c15f7a2bfdf4c6cd0e2675cc2ff0b56efb", + "https://bcr.bazel.build/modules/rules_pkg/0.7.0/MODULE.bazel": "df99f03fc7934a4737122518bb87e667e62d780b610910f0447665a7e2be62dc", + "https://bcr.bazel.build/modules/rules_pkg/1.0.1/MODULE.bazel": "5b1df97dbc29623bccdf2b0dcd0f5cb08e2f2c9050aab1092fd39a41e82686ff", + "https://bcr.bazel.build/modules/rules_pkg/1.0.1/source.json": "bd82e5d7b9ce2d31e380dd9f50c111d678c3bdaca190cb76b0e1c71b05e1ba8a", + "https://bcr.bazel.build/modules/rules_proto/4.0.0/MODULE.bazel": "a7a7b6ce9bee418c1a760b3d84f83a299ad6952f9903c67f19e4edd964894e06", + "https://bcr.bazel.build/modules/rules_proto/5.3.0-21.7/MODULE.bazel": "e8dff86b0971688790ae75528fe1813f71809b5afd57facb44dad9e8eca631b7", + "https://bcr.bazel.build/modules/rules_proto/6.0.2/MODULE.bazel": "ce916b775a62b90b61888052a416ccdda405212b6aaeb39522f7dc53431a5e73", + "https://bcr.bazel.build/modules/rules_proto/7.1.0/MODULE.bazel": "002d62d9108f75bb807cd56245d45648f38275cb3a99dcd45dfb864c5d74cb96", + "https://bcr.bazel.build/modules/rules_proto/7.1.0/source.json": "39f89066c12c24097854e8f57ab8558929f9c8d474d34b2c00ac04630ad8940e", + "https://bcr.bazel.build/modules/rules_shell/0.2.0/MODULE.bazel": "fda8a652ab3c7d8fee214de05e7a9916d8b28082234e8d2c0094505c5268ed3c", + "https://bcr.bazel.build/modules/rules_shell/0.3.0/MODULE.bazel": "de4402cd12f4cc8fda2354fce179fdb068c0b9ca1ec2d2b17b3e21b24c1a937b", + "https://bcr.bazel.build/modules/rules_shell/0.6.1/MODULE.bazel": "72e76b0eea4e81611ef5452aa82b3da34caca0c8b7b5c0c9584338aa93bae26b", + "https://bcr.bazel.build/modules/rules_shell/0.6.1/source.json": "20ec05cd5e592055e214b2da8ccb283c7f2a421ea0dc2acbf1aa792e11c03d0c", + "https://bcr.bazel.build/modules/rules_swift/1.16.0/MODULE.bazel": "4a09f199545a60d09895e8281362b1ff3bb08bbde69c6fc87aff5b92fcc916ca", + "https://bcr.bazel.build/modules/rules_swift/2.1.1/MODULE.bazel": "494900a80f944fc7aa61500c2073d9729dff0b764f0e89b824eb746959bc1046", + "https://bcr.bazel.build/modules/rules_swift/2.4.0/MODULE.bazel": "1639617eb1ede28d774d967a738b4a68b0accb40650beadb57c21846beab5efd", + "https://bcr.bazel.build/modules/rules_swift/3.1.2/MODULE.bazel": "72c8f5cf9d26427cee6c76c8e3853eb46ce6b0412a081b2b6db6e8ad56267400", + "https://bcr.bazel.build/modules/rules_swift/3.1.2/source.json": "e85761f3098a6faf40b8187695e3de6d97944e98abd0d8ce579cb2daf6319a66", + "https://bcr.bazel.build/modules/stardoc/0.5.1/MODULE.bazel": "1a05d92974d0c122f5ccf09291442580317cdd859f07a8655f1db9a60374f9f8", + "https://bcr.bazel.build/modules/stardoc/0.5.3/MODULE.bazel": "c7f6948dae6999bf0db32c1858ae345f112cacf98f174c7a8bb707e41b974f1c", + "https://bcr.bazel.build/modules/stardoc/0.7.0/MODULE.bazel": "05e3d6d30c099b6770e97da986c53bd31844d7f13d41412480ea265ac9e8079c", + "https://bcr.bazel.build/modules/swift_argument_parser/1.3.1.1/MODULE.bazel": "5e463fbfba7b1701d957555ed45097d7f984211330106ccd1352c6e0af0dcf91", + "https://bcr.bazel.build/modules/swift_argument_parser/1.3.1.2/MODULE.bazel": "75aab2373a4bbe2a1260b9bf2a1ebbdbf872d3bd36f80bff058dccd82e89422f", + "https://bcr.bazel.build/modules/swift_argument_parser/1.3.1.2/source.json": "5fba48bbe0ba48761f9e9f75f92876cafb5d07c0ce059cc7a8027416de94a05b", + "https://bcr.bazel.build/modules/upb/0.0.0-20220923-a547704/MODULE.bazel": "7298990c00040a0e2f121f6c32544bab27d4452f80d9ce51349b1a28f3005c43", + "https://bcr.bazel.build/modules/zlib/1.2.11/MODULE.bazel": "07b389abc85fdbca459b69e2ec656ae5622873af3f845e1c9d80fe179f3effa0", + "https://bcr.bazel.build/modules/zlib/1.3.1.bcr.5/MODULE.bazel": "eec517b5bbe5492629466e11dae908d043364302283de25581e3eb944326c4ca", + "https://bcr.bazel.build/modules/zlib/1.3.1.bcr.5/source.json": "22bc55c47af97246cfc093d0acf683a7869377de362b5d1c552c2c2e16b7a806", + "https://bcr.bazel.build/modules/zlib/1.3.1/MODULE.bazel": "751c9940dcfe869f5f7274e1295422a34623555916eb98c174c1e945594bf198" + }, + "selectedYankedVersions": {}, + "moduleExtensions": { + "@@pybind11_bazel+//:internal_configure.bzl%internal_configure_extension": { + "general": { + "bzlTransitiveDigest": "b+RP7Sgl8KN0VHamrgTqzGLuYPcQ/Mo4ptNkkHUIIlA=", + "usagesDigest": "D1r3lfzMuUBFxgG8V6o0bQTLMk3GkaGOaPzw53wrwyw=", + "recordedInputs": [ + "REPO_MAPPING:pybind11_bazel+,bazel_tools bazel_tools", + "FILE:@@pybind11_bazel+//MODULE.bazel e6f4c20442eaa7c90d7190d8dc539d0ab422f95c65a57cc59562170c58ae3d34" + ], + "generatedRepoSpecs": { + "pybind11": { + "repoRuleId": "@@bazel_tools//tools/build_defs/repo:http.bzl%http_archive", + "attributes": { + "build_file": "@@pybind11_bazel+//:pybind11-BUILD.bazel", + "strip_prefix": "pybind11-2.12.0", + "urls": [ + "https://github.com/pybind/pybind11/archive/v2.12.0.zip" + ] + } + } + } + } + }, + "@@rules_kotlin+//src/main/starlark/core/repositories:bzlmod_setup.bzl%rules_kotlin_extensions": { + "general": { + "bzlTransitiveDigest": "Ga4z8lQy1YQ5rAMy+dOl0dqcCEBnYNCXku8x3YQmDZI=", + "usagesDigest": "QI2z8ZUR+mqtbwsf2fLqYdJAkPOHdOV+tF2yVAUgRzw=", + "recordedInputs": [ + "REPO_MAPPING:rules_kotlin+,bazel_tools bazel_tools" + ], + "generatedRepoSpecs": { + "com_github_jetbrains_kotlin_git": { + "repoRuleId": "@@rules_kotlin+//src/main/starlark/core/repositories:compiler.bzl%kotlin_compiler_git_repository", + "attributes": { + "urls": [ + "https://github.com/JetBrains/kotlin/releases/download/v1.9.23/kotlin-compiler-1.9.23.zip" + ], + "sha256": "93137d3aab9afa9b27cb06a824c2324195c6b6f6179d8a8653f440f5bd58be88" + } + }, + "com_github_jetbrains_kotlin": { + "repoRuleId": "@@rules_kotlin+//src/main/starlark/core/repositories:compiler.bzl%kotlin_capabilities_repository", + "attributes": { + "git_repository_name": "com_github_jetbrains_kotlin_git", + "compiler_version": "1.9.23" + } + }, + "com_github_google_ksp": { + "repoRuleId": "@@rules_kotlin+//src/main/starlark/core/repositories:ksp.bzl%ksp_compiler_plugin_repository", + "attributes": { + "urls": [ + "https://github.com/google/ksp/releases/download/1.9.23-1.0.20/artifacts.zip" + ], + "sha256": "ee0618755913ef7fd6511288a232e8fad24838b9af6ea73972a76e81053c8c2d", + "strip_version": "1.9.23-1.0.20" + } + }, + "com_github_pinterest_ktlint": { + "repoRuleId": "@@bazel_tools//tools/build_defs/repo:http.bzl%http_file", + "attributes": { + "sha256": "01b2e0ef893383a50dbeb13970fe7fa3be36ca3e83259e01649945b09d736985", + "urls": [ + "https://github.com/pinterest/ktlint/releases/download/1.3.0/ktlint" + ], + "executable": true + } + }, + "rules_android": { + "repoRuleId": "@@bazel_tools//tools/build_defs/repo:http.bzl%http_archive", + "attributes": { + "sha256": "cd06d15dd8bb59926e4d65f9003bfc20f9da4b2519985c27e190cddc8b7a7806", + "strip_prefix": "rules_android-0.1.1", + "urls": [ + "https://github.com/bazelbuild/rules_android/archive/v0.1.1.zip" + ] + } + } + } + } + }, + "@@rules_python+//python/uv:uv.bzl%uv": { + "general": { + "bzlTransitiveDigest": "yrEbeCJlv5gbdoRjwwR/EDEkc3pfuQy/FTYKMNoK5Wc=", + "usagesDigest": "6yXGw7XDyXjOfqBL0SBu1YBEMMYPQzCE3jTzUCkxPgg=", + "recordedInputs": [ + "REPO_MAPPING:rules_python+,bazel_tools bazel_tools", + "REPO_MAPPING:rules_python+,platforms platforms" + ], + "generatedRepoSpecs": { + "uv": { + "repoRuleId": "@@rules_python+//python/uv/private:uv_toolchains_repo.bzl%uv_toolchains_repo", + "attributes": { + "toolchain_type": "'@@rules_python+//python/uv:uv_toolchain_type'", + "toolchain_names": [ + "none" + ], + "toolchain_implementations": { + "none": "'@@rules_python+//python:none'" + }, + "toolchain_compatible_with": { + "none": [ + "@platforms//:incompatible" + ] + }, + "toolchain_target_settings": {} + } + } + } + } + } + }, + "facts": { + "@@rules_python+//python/extensions:pip.bzl%pip": { + "dist_hashes": { + "https://pypi.org/simple": { + "backports-tarfile": { + "https://files.pythonhosted.org/packages/86/72/cd9b395f25e290e633655a100af28cb253e4393396264a98bd5f5951d50f/backports_tarfile-1.2.0.tar.gz": "d75e02c268746e1b8144c278978b6e98e85de6ad16f8e4b0844a154557eca991", + "https://files.pythonhosted.org/packages/b9/fa/123043af240e49752f1c4bd24da5053b6bd00cad78c2be53c0d1e8b975bc/backports.tarfile-1.2.0-py3-none-any.whl": "77e284d754527b01fb1e6fa8a1afe577858ebe4e9dad8919e34c862cb399bc34" + }, + "certifi": { + "https://files.pythonhosted.org/packages/4c/5b/b6ce21586237c77ce67d01dc5507039d444b630dd76611bbca2d8e5dcd91/certifi-2025.10.5.tar.gz": "47c09d31ccf2acf0be3f701ea53595ee7e0b8fa08801c6624be771df09ae7b43", + "https://files.pythonhosted.org/packages/e4/37/af0d2ef3967ac0d6113837b44a4f0bfe1328c2b9763bd5b1744520e5cfed/certifi-2025.10.5-py3-none-any.whl": "0f212c2744a9bb6de0c56639a6f68afe01ecd92d91f14ae897c4fe7bbeeef0de" + }, + "cffi": { + "https://files.pythonhosted.org/packages/05/eb/b86f2a2645b62adcfff53b0dd97e8dfafb5c8aa864bd0d9a2c2049a0d551/cffi-2.0.0-cp310-cp310-manylinux2014_ppc64le.manylinux_2_17_ppc64le.whl": "5eda85d6d1879e692d546a078b44251cdd08dd1cfb98dfb77b670c97cee49ea0", + "https://files.pythonhosted.org/packages/07/e0/267e57e387b4ca276b90f0434ff88b2c2241ad72b16d31836adddfd6031b/cffi-2.0.0-cp312-cp312-musllinux_1_2_aarch64.whl": "3925dd22fa2b7699ed2617149842d2e6adde22b262fcbfada50e3d195e4b3a94", + "https://files.pythonhosted.org/packages/0b/28/dd0967a76aab36731b6ebfe64dec4e981aff7e0608f60c2d46b46982607d/cffi-2.0.0-cp311-cp311-musllinux_1_2_x86_64.whl": "5fed36fccc0612a53f1d4d9a816b50a36702c28a2aa880cb8a122b3466638743", + "https://files.pythonhosted.org/packages/12/4a/3dfd5f7850cbf0d06dc84ba9aa00db766b52ca38d8b86e3a38314d52498c/cffi-2.0.0-cp311-cp311-macosx_10_13_x86_64.whl": "b4c854ef3adc177950a8dfc81a86f5115d2abd545751a304c5bcf2c2c7283cfe", + "https://files.pythonhosted.org/packages/15/12/a7a79bd0df4c3bff744b2d7e52cc1b68d5e7e427b384252c42366dc1ecbc/cffi-2.0.0-cp39-cp39-manylinux2014_aarch64.manylinux_2_17_aarch64.whl": "3f4d46d8b35698056ec29bca21546e1551a205058ae1a181d871e278b0b28165", + "https://files.pythonhosted.org/packages/1f/74/cc4096ce66f5939042ae094e2e96f53426a979864aa1f96a621ad128be27/cffi-2.0.0-cp39-cp39-manylinux2014_x86_64.manylinux_2_17_x86_64.whl": "61d028e90346df14fedc3d1e5441df818d095f3b87d286825dfcbd6459b7ef63", + "https://files.pythonhosted.org/packages/21/7a/13b24e70d2f90a322f2900c5d8e1f14fa7e2a6b3332b7309ba7b2ba51a5a/cffi-2.0.0-cp310-cp310-musllinux_1_2_aarch64.whl": "cf364028c016c03078a23b503f02058f1814320a56ad535686f90565636a9495", + "https://files.pythonhosted.org/packages/25/8e/342a504ff018a2825d395d44d63a767dd8ebc927ebda557fecdaca3ac33a/cffi-2.0.0-cp314-cp314t-musllinux_1_2_aarch64.whl": "7553fb2090d71822f02c629afe6042c299edf91ba1bf94951165613553984512", + "https://files.pythonhosted.org/packages/2b/0f/1f177e3683aead2bb00f7679a16451d302c436b5cbf2505f0ea8146ef59e/cffi-2.0.0-cp314-cp314-musllinux_1_2_aarch64.whl": "737fe7d37e1a1bffe70bd5754ea763a62a066dc5913ca57e957824b72a85e205", + "https://files.pythonhosted.org/packages/2b/c0/015b25184413d7ab0a410775fdb4a50fca20f5589b5dab1dbbfa3baad8ce/cffi-2.0.0-cp311-cp311-win32.whl": "c649e3a33450ec82378822b3dad03cc228b8f5963c0c12fc3b1e0ab940f768a5", + "https://files.pythonhosted.org/packages/2b/e7/7c769804eb75e4c4b35e658dba01de1640a351a9653c3d49ca89d16ccc91/cffi-2.0.0-cp39-cp39-musllinux_1_2_x86_64.whl": "89472c9762729b5ae1ad974b777416bfda4ac5642423fa93bd57a09204712322", + "https://files.pythonhosted.org/packages/2c/ea/5f76bce7cf6fcd0ab1a1058b5af899bfbef198bea4d5686da88471ea0336/cffi-2.0.0-cp314-cp314t-macosx_11_0_arm64.whl": "7a66c7204d8869299919db4d5069a82f1561581af12b11b3c9f48c584eb8743d", + "https://files.pythonhosted.org/packages/32/f2/81b63e288295928739d715d00952c8c6034cb6c6a516b17d37e0c8be5600/cffi-2.0.0-cp39-cp39-manylinux2014_s390x.manylinux_2_17_s390x.whl": "cb527a79772e5ef98fb1d700678fe031e353e765d1ca2d409c92263c6d43e09f", + "https://files.pythonhosted.org/packages/33/fa/072dd15ae27fbb4e06b437eb6e944e75b068deb09e2a2826039e49ee2045/cffi-2.0.0-cp310-cp310-win_amd64.whl": "b18a3ed7d5b3bd8d9ef7a8cb226502c6bf8308df1525e1cc676c3680e7176739", + "https://files.pythonhosted.org/packages/36/54/0362578dd2c9e557a28ac77698ed67323ed5b9775ca9d3fe73fe191bb5d8/cffi-2.0.0-cp313-cp313-musllinux_1_2_x86_64.whl": "6d50360be4546678fc1b79ffe7a66265e28667840010348dd69a314145807a1b", + "https://files.pythonhosted.org/packages/37/18/6519e1ee6f5a1e579e04b9ddb6f1676c17368a7aba48299c3759bbc3c8b3/cffi-2.0.0-cp313-cp313-win_amd64.whl": "19f705ada2530c1167abacb171925dd886168931e0a7b78f5bffcae5c6b5be75", + "https://files.pythonhosted.org/packages/3a/c8/15cb9ada8895957ea171c62dc78ff3e99159ee7adb13c0123c001a2546c1/cffi-2.0.0-cp312-cp312-manylinux2014_s390x.manylinux_2_17_s390x.whl": "81afed14892743bbe14dacb9e36d9e0e504cd204e0b165062c488942b9718037", + "https://files.pythonhosted.org/packages/3d/de/38d9726324e127f727b4ecc376bc85e505bfe61ef130eaf3f290c6847dd4/cffi-2.0.0-cp39-cp39-macosx_11_0_arm64.whl": "de8dad4425a6ca6e4e5e297b27b5c824ecc7581910bf9aee86cb6835e6812aa7", + "https://files.pythonhosted.org/packages/3e/61/c768e4d548bfa607abcda77423448df8c471f25dbe64fb2ef6d555eae006/cffi-2.0.0-cp314-cp314t-macosx_10_13_x86_64.whl": "9a67fc9e8eb39039280526379fb3a70023d77caec1852002b4da7e8b270c4dd9", + "https://files.pythonhosted.org/packages/3e/aa/df335faa45b395396fcbc03de2dfcab242cd61a9900e914fe682a59170b1/cffi-2.0.0-cp314-cp314-win32.whl": "087067fa8953339c723661eda6b54bc98c5625757ea62e95eb4898ad5e776e9f", + "https://files.pythonhosted.org/packages/44/64/58f6255b62b101093d5df22dcb752596066c7e89dd725e0afaed242a61be/cffi-2.0.0-cp311-cp311-musllinux_1_2_aarch64.whl": "a05d0c237b3349096d3981b727493e22147f934b20f6f125a3eba8f994bec4a9", + "https://files.pythonhosted.org/packages/47/d9/d83e293854571c877a92da46fdec39158f8d7e68da75bf73581225d28e90/cffi-2.0.0-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.whl": "afb8db5439b81cf9c9d0c80404b60c3cc9c3add93e114dcae767f1477cb53775", + "https://files.pythonhosted.org/packages/49/72/ff2d12dbf21aca1b32a40ed792ee6b40f6dc3a9cf1644bd7ef6e95e0ac5e/cffi-2.0.0-cp310-cp310-musllinux_1_2_x86_64.whl": "8ea985900c5c95ce9db1745f7933eeef5d314f0565b27625d9a10ec9881e1bfb", + "https://files.pythonhosted.org/packages/4a/d2/a6c0296814556c68ee32009d9c2ad4f85f2707cdecfd7727951ec228005d/cffi-2.0.0-cp313-cp313-macosx_11_0_arm64.whl": "45d5e886156860dc35862657e1494b9bae8dfa63bf56796f2fb56e1679fc0bca", + "https://files.pythonhosted.org/packages/4b/8d/a0a47a0c9e413a658623d014e91e74a50cdd2c423f7ccfd44086ef767f90/cffi-2.0.0-cp313-cp313-macosx_10_13_x86_64.whl": "00bdf7acc5f795150faa6957054fbbca2439db2f775ce831222b66f192f03beb", + "https://files.pythonhosted.org/packages/4f/27/6933a8b2562d7bd1fb595074cf99cc81fc3789f6a6c05cdabb46284a3188/cffi-2.0.0-cp310-cp310-manylinux2014_aarch64.manylinux_2_17_aarch64.whl": "3e837e369566884707ddaf85fc1744b47575005c0a229de3327f8f9a20f4efeb", + "https://files.pythonhosted.org/packages/4f/8b/f0e4c441227ba756aafbe78f117485b25bb26b1c059d01f137fa6d14896b/cffi-2.0.0-cp311-cp311-macosx_11_0_arm64.whl": "2de9a304e27f7596cd03d16f1b7c72219bd944e99cc52b84d0145aefb07cbd3c", + "https://files.pythonhosted.org/packages/50/bd/b1a6362b80628111e6653c961f987faa55262b4002fcec42308cad1db680/cffi-2.0.0-cp310-cp310-manylinux1_i686.manylinux2014_i686.manylinux_2_17_i686.manylinux_2_5_i686.whl": "53f77cbe57044e88bbd5ed26ac1d0514d2acf0591dd6bb02a3ae37f76811b80c", + "https://files.pythonhosted.org/packages/50/e1/a969e687fcf9ea58e6e2a928ad5e2dd88cc12f6f0ab477e9971f2309b57c/cffi-2.0.0-cp313-cp313-musllinux_1_2_aarch64.whl": "d9b29c1f0ae438d5ee9acb31cadee00a58c46cc9c0b2f9038c6b0b3470877a8c", + "https://files.pythonhosted.org/packages/54/8f/a1e836f82d8e32a97e6b29cc8f641779181ac7363734f12df27db803ebda/cffi-2.0.0-cp39-cp39-win_amd64.whl": "b882b3df248017dba09d6b16defe9b5c407fe32fc7c65a9c69798e6175601be9", + "https://files.pythonhosted.org/packages/59/dd/27e9fa567a23931c838c6b02d0764611c62290062a6d4e8ff7863daf9730/cffi-2.0.0-cp314-cp314-macosx_11_0_arm64.whl": "c654de545946e0db659b3400168c9ad31b5d29593291482c43e3564effbcee13", + "https://files.pythonhosted.org/packages/60/99/c9dc110974c59cc981b1f5b66e1d8af8af764e00f0293266824d9c4254bc/cffi-2.0.0-cp310-cp310-musllinux_1_2_i686.whl": "e11e82b744887154b182fd3e7e8512418446501191994dbf9c9fc1f32cc8efd5", + "https://files.pythonhosted.org/packages/78/2d/7fa73dfa841b5ac06c7b8855cfc18622132e365f5b81d02230333ff26e9e/cffi-2.0.0-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.whl": "3e17ed538242334bf70832644a32a7aae3d83b57567f9fd60a26257e992b79ba", + "https://files.pythonhosted.org/packages/7b/2b/2b6435f76bfeb6bbf055596976da087377ede68df465419d192acf00c437/cffi-2.0.0-cp312-cp312-win32.whl": "da902562c3e9c550df360bfa53c035b2f241fed6d9aef119048073680ace4a18", + "https://files.pythonhosted.org/packages/84/ef/a7b77c8bdc0f77adc3b46888f1ad54be8f3b7821697a7b89126e829e676a/cffi-2.0.0-cp311-cp311-manylinux2014_s390x.manylinux_2_17_s390x.whl": "9de40a7b0323d889cf8d23d1ef214f565ab154443c42737dfe52ff82cf857664", + "https://files.pythonhosted.org/packages/92/c4/3ce07396253a83250ee98564f8d7e9789fab8e58858f35d07a9a2c78de9f/cffi-2.0.0-cp314-cp314-macosx_10_13_x86_64.whl": "fc33c5141b55ed366cfaad382df24fe7dcbc686de5be719b207bb248e3053dc5", + "https://files.pythonhosted.org/packages/93/d7/516d984057745a6cd96575eea814fe1edd6646ee6efd552fb7b0921dec83/cffi-2.0.0-cp310-cp310-macosx_10_13_x86_64.whl": "0cf2d91ecc3fcc0625c2c530fe004f82c110405f101548512cce44322fa8ac44", + "https://files.pythonhosted.org/packages/95/31/9f7f93ad2f8eff1dbc1c3656d7ca5bfd8fb52c9d786b4dcf19b2d02217fa/cffi-2.0.0-cp312-cp312-win_arm64.whl": "4671d9dd5ec934cb9a73e7ee9676f9362aba54f7f34910956b84d727b0d73fb6", + "https://files.pythonhosted.org/packages/95/5c/1b493356429f9aecfd56bc171285a4c4ac8697f76e9bbbbb105e537853a1/cffi-2.0.0-cp311-cp311-win_arm64.whl": "c6638687455baf640e37344fe26d37c404db8b80d037c3d29f58fe8d1c3b194d", + "https://files.pythonhosted.org/packages/98/29/9b366e70e243eb3d14a5cb488dfd3a0b6b2f1fb001a203f653b93ccfac88/cffi-2.0.0-cp310-cp310-manylinux2014_x86_64.manylinux_2_17_x86_64.whl": "fc7de24befaeae77ba923797c7c87834c73648a05a4bde34b3b7e5588973a453", + "https://files.pythonhosted.org/packages/98/df/0a1755e750013a2081e863e7cd37e0cdd02664372c754e5560099eb7aa44/cffi-2.0.0-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.whl": "c8d3b5532fc71b7a77c09192b4a5a200ea992702734a2e9279a37f2478236f26", + "https://files.pythonhosted.org/packages/9b/13/c92e36358fbcc39cf0962e83223c9522154ee8630e1df7c0b3a39a8124e2/cffi-2.0.0-cp39-cp39-manylinux1_i686.manylinux2014_i686.manylinux_2_17_i686.manylinux_2_5_i686.whl": "4647afc2f90d1ddd33441e5b0e85b16b12ddec4fca55f0d9671fef036ecca27c", + "https://files.pythonhosted.org/packages/9e/84/ad6a0b408daa859246f57c03efd28e5dd1b33c21737c2db84cae8c237aa5/cffi-2.0.0-cp310-cp310-macosx_11_0_arm64.whl": "f73b96c41e3b2adedc34a7356e64c8eb96e03a3782b535e043a986276ce12a49", + "https://files.pythonhosted.org/packages/9f/2c/98ece204b9d35a7366b5b2c6539c350313ca13932143e79dc133ba757104/cffi-2.0.0-cp314-cp314-win_arm64.whl": "dbd5c7a25a7cb98f5ca55d258b103a2054f859a46ae11aaf23134f9cc0d356ad", + "https://files.pythonhosted.org/packages/9f/e0/6cbe77a53acf5acc7c08cc186c9928864bd7c005f9efd0d126884858a5fe/cffi-2.0.0-cp310-cp310-manylinux2014_s390x.manylinux_2_17_s390x.whl": "9332088d75dc3241c702d852d4671613136d90fa6881da7d770a483fd05248b4", + "https://files.pythonhosted.org/packages/a0/1d/ec1a60bd1a10daa292d3cd6bb0b359a81607154fb8165f3ec95fe003b85c/cffi-2.0.0-cp314-cp314t-win32.whl": "1fc9ea04857caf665289b7a75923f2c6ed559b8298a1b8c49e59f7dd95c8481e", + "https://files.pythonhosted.org/packages/a3/ad/5c51c1c7600bdd7ed9a24a203ec255dccdd0ebf4527f7b922a0bde2fb6ed/cffi-2.0.0-cp39-cp39-manylinux2014_ppc64le.manylinux_2_17_ppc64le.whl": "e6e73b9e02893c764e7e8d5bb5ce277f1a009cd5243f8228f75f842bf937c534", + "https://files.pythonhosted.org/packages/a9/f5/a2c23eb03b61a0b8747f211eb716446c826ad66818ddc7810cc2cc19b3f2/cffi-2.0.0-cp313-cp313-manylinux2014_aarch64.manylinux_2_17_aarch64.whl": "d48a880098c96020b02d5a1f7d9251308510ce8858940e6fa99ece33f610838b", + "https://files.pythonhosted.org/packages/aa/d9/6218d78f920dcd7507fc16a766b5ef8f3b913cc7aa938e7fc80b9978d089/cffi-2.0.0-cp39-cp39-win32.whl": "2081580ebb843f759b9f617314a24ed5738c51d2aee65d31e02f6f7a2b97707a", + "https://files.pythonhosted.org/packages/ab/49/fa72cebe2fd8a55fbe14956f9970fe8eb1ac59e5df042f603ef7c8ba0adc/cffi-2.0.0-cp311-cp311-musllinux_1_2_i686.whl": "94698a9c5f91f9d138526b48fe26a199609544591f859c870d477351dc7b2414", + "https://files.pythonhosted.org/packages/ae/3a/dbeec9d1ee0844c679f6bb5d6ad4e9f198b1224f4e7a32825f47f6192b0c/cffi-2.0.0-cp314-cp314t-win_arm64.whl": "0a1527a803f0a659de1af2e1fd700213caba79377e27e4693648c2923da066f9", + "https://files.pythonhosted.org/packages/ae/8f/dc5531155e7070361eb1b7e4c1a9d896d0cb21c49f807a6c03fd63fc877e/cffi-2.0.0-cp311-cp311-win_amd64.whl": "66f011380d0e49ed280c789fbd08ff0d40968ee7b665575489afa95c98196ab5", + "https://files.pythonhosted.org/packages/b0/1e/d22cc63332bd59b06481ceaac49d6c507598642e2230f201649058a7e704/cffi-2.0.0-cp313-cp313-manylinux1_i686.manylinux2014_i686.manylinux_2_17_i686.manylinux_2_5_i686.whl": "07b271772c100085dd28b74fa0cd81c8fb1a3ba18b21e03d7c27f3436a10606b", + "https://files.pythonhosted.org/packages/b1/b7/1200d354378ef52ec227395d95c2576330fd22a869f7a70e88e1447eb234/cffi-2.0.0-cp311-cp311-manylinux1_i686.manylinux2014_i686.manylinux_2_17_i686.manylinux_2_5_i686.whl": "baf5215e0ab74c16e2dd324e8ec067ef59e41125d3eade2b863d294fd5035c92", + "https://files.pythonhosted.org/packages/b4/89/76799151d9c2d2d1ead63c2429da9ea9d7aac304603de0c6e8764e6e8e70/cffi-2.0.0-cp314-cp314-manylinux2014_ppc64le.manylinux_2_17_ppc64le.whl": "12873ca6cb9b0f0d3a0da705d6086fe911591737a59f28b7936bdfed27c0d47c", + "https://files.pythonhosted.org/packages/b6/75/1f2747525e06f53efbd878f4d03bac5b859cbc11c633d0fb81432d98a795/cffi-2.0.0-cp312-cp312-musllinux_1_2_x86_64.whl": "2c8f814d84194c9ea681642fd164267891702542f028a15fc97d4674b6206187", + "https://files.pythonhosted.org/packages/b8/56/6033f5e86e8cc9bb629f0077ba71679508bdf54a9a5e112a3c0b91870332/cffi-2.0.0-cp311-cp311-manylinux2014_aarch64.manylinux_2_17_aarch64.whl": "730cacb21e1bdff3ce90babf007d0a0917cc3e6492f336c2f0134101e0944f93", + "https://files.pythonhosted.org/packages/bb/92/882c2d30831744296ce713f0feb4c1cd30f346ef747b530b5318715cc367/cffi-2.0.0-cp314-cp314-win_amd64.whl": "203a48d1fb583fc7d78a4c6655692963b860a417c0528492a6bc21f1aaefab25", + "https://files.pythonhosted.org/packages/bb/dd/3465b14bb9e24ee24cb88c9e3730f6de63111fffe513492bf8c808a3547e/cffi-2.0.0-cp314-cp314-manylinux2014_s390x.manylinux_2_17_s390x.whl": "d9b97165e8aed9272a6bb17c01e3cc5871a594a446ebedc996e2397a1c1ea8ef", + "https://files.pythonhosted.org/packages/be/b4/c56878d0d1755cf9caa54ba71e5d049479c52f9e4afc230f06822162ab2f/cffi-2.0.0-cp314-cp314t-manylinux2014_aarch64.manylinux_2_17_aarch64.whl": "7cc09976e8b56f8cebd752f7113ad07752461f48a58cbba644139015ac24954c", + "https://files.pythonhosted.org/packages/bf/41/4c1168c74fac325c0c8156f04b6749c8b6a8f405bbf91413ba088359f60d/cffi-2.0.0-cp314-cp314t-win_amd64.whl": "d68b6cef7827e8641e8ef16f4494edda8b36104d79773a334beaa1e3521430f6", + "https://files.pythonhosted.org/packages/c0/cc/08ed5a43f2996a16b462f64a7055c6e962803534924b9b2f1371d8c00b7b/cffi-2.0.0-cp39-cp39-macosx_10_13_x86_64.whl": "fe562eb1a64e67dd297ccc4f5addea2501664954f2692b69a76449ec7913ecbf", + "https://files.pythonhosted.org/packages/c2/95/7a135d52a50dfa7c882ab0ac17e8dc11cec9d55d2c18dda414c051c5e69e/cffi-2.0.0-cp312-cp312-manylinux2014_ppc64le.manylinux_2_17_ppc64le.whl": "1e3a615586f05fc4065a8b22b8152f0c1b00cdbc60596d187c2a74f9e3036e4e", + "https://files.pythonhosted.org/packages/c6/0f/cafacebd4b040e3119dcb32fed8bdef8dfe94da653155f9d0b9dc660166e/cffi-2.0.0-cp314-cp314-musllinux_1_2_x86_64.whl": "38100abb9d1b1435bc4cc340bb4489635dc2f0da7456590877030c9b3d40b0c1", + "https://files.pythonhosted.org/packages/cb/0e/02ceeec9a7d6ee63bb596121c2c8e9b3a9e150936f4fbef6ca1943e6137c/cffi-2.0.0-cp313-cp313-win_arm64.whl": "256f80b80ca3853f90c21b23ee78cd008713787b1b1e93eae9f3d6a7134abd91", + "https://files.pythonhosted.org/packages/cb/1e/a5a1bd6f1fb30f22573f76533de12a00bf274abcdc55c8edab639078abb6/cffi-2.0.0-cp313-cp313-manylinux2014_s390x.manylinux_2_17_s390x.whl": "dd4f05f54a52fb558f1ba9f528228066954fee3ebe629fc1660d874d040ae5a3", + "https://files.pythonhosted.org/packages/d0/44/681604464ed9541673e486521497406fadcc15b5217c3e326b061696899a/cffi-2.0.0-cp314-cp314t-manylinux2014_x86_64.manylinux_2_17_x86_64.whl": "28a3a209b96630bca57cce802da70c266eb08c6e97e5afd61a75611ee6c64592", + "https://files.pythonhosted.org/packages/d5/72/12b5f8d3865bf0f87cf1404d8c374e7487dcf097a1c91c436e72e6badd83/cffi-2.0.0-cp312-cp312-manylinux2014_aarch64.manylinux_2_17_aarch64.whl": "b21e08af67b8a103c71a250401c78d5e0893beff75e28c53c98f4de42f774062", + "https://files.pythonhosted.org/packages/d6/43/0e822876f87ea8a4ef95442c3d766a06a51fc5298823f884ef87aaad168c/cffi-2.0.0-cp314-cp314-manylinux2014_aarch64.manylinux_2_17_aarch64.whl": "24b6f81f1983e6df8db3adc38562c83f7d4a0c36162885ec7f7b77c7dcbec97b", + "https://files.pythonhosted.org/packages/d7/91/500d892b2bf36529a75b77958edfcd5ad8e2ce4064ce2ecfeab2125d72d1/cffi-2.0.0-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.whl": "8941aaadaf67246224cee8c3803777eed332a19d909b47e29c9842ef1e79ac26", + "https://files.pythonhosted.org/packages/d8/19/3c435d727b368ca475fb8742ab97c9cb13a0de600ce86f62eab7fa3eea60/cffi-2.0.0-cp314-cp314t-manylinux2014_s390x.manylinux_2_17_s390x.whl": "b1e74d11748e7e98e2f426ab176d4ed720a64412b6a15054378afdb71e0f37dc", + "https://files.pythonhosted.org/packages/dc/7f/55fecd70f7ece178db2f26128ec41430d8720f2d12ca97bf8f0a628207d5/cffi-2.0.0-cp311-cp311-manylinux2014_ppc64le.manylinux_2_17_ppc64le.whl": "6824f87845e3396029f3820c206e459ccc91760e8fa24422f8b0c3d1731cbec5", + "https://files.pythonhosted.org/packages/df/a2/781b623f57358e360d62cdd7a8c681f074a71d445418a776eef0aadb4ab4/cffi-2.0.0-cp312-cp312-macosx_11_0_arm64.whl": "8eca2a813c1cb7ad4fb74d368c2ffbbb4789d377ee5bb8df98373c2cc0dee76c", + "https://files.pythonhosted.org/packages/e0/0d/eb704606dfe8033e7128df5e90fee946bbcb64a04fcdaa97321309004000/cffi-2.0.0-cp314-cp314t-manylinux2014_ppc64le.manylinux_2_17_ppc64le.whl": "92b68146a71df78564e4ef48af17551a5ddd142e5190cdf2c5624d0c3ff5b2e8", + "https://files.pythonhosted.org/packages/e1/5e/b666bacbbc60fbf415ba9988324a132c9a7a0448a9a8f125074671c0f2c3/cffi-2.0.0-cp314-cp314t-musllinux_1_2_x86_64.whl": "6c6c373cfc5c83a975506110d17457138c8c63016b563cc9ed6e056a82f13ce4", + "https://files.pythonhosted.org/packages/e2/cc/027d7fb82e58c48ea717149b03bcadcbdc293553edb283af792bd4bcbb3f/cffi-2.0.0-cp310-cp310-win32.whl": "1f72fb8906754ac8a2cc3f9f5aaa298070652a0ffae577e0ea9bd480dc3c931a", + "https://files.pythonhosted.org/packages/e8/be/f6424d1dc46b1091ffcc8964fa7c0ab0cd36839dd2761b49c90481a6ba1b/cffi-2.0.0-cp39-cp39-musllinux_1_2_aarch64.whl": "0f6084a0ea23d05d20c3edcda20c3d006f9b6f3fefeac38f59262e10cef47ee2", + "https://files.pythonhosted.org/packages/ea/47/4f61023ea636104d4f16ab488e268b93008c3d0bb76893b1b31db1f96802/cffi-2.0.0-cp312-cp312-macosx_10_13_x86_64.whl": "6d02d6655b0e54f54c4ef0b94eb6be0607b70853c45ce98bd278dc7de718be5d", + "https://files.pythonhosted.org/packages/eb/56/b1ba7935a17738ae8453301356628e8147c79dbb825bcbc73dc7401f9846/cffi-2.0.0.tar.gz": "44d1b5909021139fe36001ae048dbdde8214afa20200eda0f64c068cac5d5529", + "https://files.pythonhosted.org/packages/eb/6d/bf9bda840d5f1dfdbf0feca87fbdb64a918a69bca42cfa0ba7b137c48cb8/cffi-2.0.0-cp313-cp313-win32.whl": "74a03b9698e198d47562765773b4a8309919089150a0bb17d829ad7b44b60d27", + "https://files.pythonhosted.org/packages/f2/7f/e6647792fc5850d634695bc0e6ab4111ae88e89981d35ac269956605feba/cffi-2.0.0-cp313-cp313-manylinux2014_ppc64le.manylinux_2_17_ppc64le.whl": "f93fd8e5c8c0a4aa1f424d6173f14a892044054871c771f8566e4008eaa359d2", + "https://files.pythonhosted.org/packages/f7/e0/dda537c2309817edf60109e39265f24f24aa7f050767e22c98c53fe7f48b/cffi-2.0.0-cp39-cp39-musllinux_1_2_i686.whl": "1cd13c99ce269b3ed80b417dcd591415d3372bcac067009b6e0f59c7d4015e65", + "https://files.pythonhosted.org/packages/f8/ed/13bd4418627013bec4ed6e54283b1959cf6db888048c7cf4b4c3b5b36002/cffi-2.0.0-cp312-cp312-win_amd64.whl": "da68248800ad6320861f129cd9c1bf96ca849a2771a59e0344e88681905916f5", + "https://files.pythonhosted.org/packages/ff/df/a4f0fbd47331ceeba3d37c2e51e9dfc9722498becbeec2bd8bc856c9538a/cffi-2.0.0-cp312-cp312-manylinux1_i686.manylinux2014_i686.manylinux_2_17_i686.manylinux_2_5_i686.whl": "21d1152871b019407d8ac3985f6775c079416c282e431a4da6afe7aefd2bccbe" + }, + "charset-normalizer": { + "https://files.pythonhosted.org/packages/00/bd/ef9c88464b126fa176f4ef4a317ad9b6f4d30b2cffbc43386062367c3e2c/charset_normalizer-3.4.3-cp38-cp38-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl": "8999f965f922ae054125286faf9f11bc6932184b93011d138925a1773830bbe9", + "https://files.pythonhosted.org/packages/02/f7/3611b32318b30974131db62b4043f335861d4d9b49adc6d57c1149cc49d4/charset_normalizer-3.4.3-cp314-cp314-musllinux_1_2_aarch64.whl": "ccf600859c183d70eb47e05a44cd80a4ce77394d1ac0f79dbd2dd90a69a3a049", + "https://files.pythonhosted.org/packages/04/9a/914d294daa4809c57667b77470533e65def9c0be1ef8b4c1183a99170e9d/charset_normalizer-3.4.3-cp39-cp39-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl": "fb731e5deb0c7ef82d698b0f4c5bb724633ee2a489401594c5c88b02e6cb15f7", + "https://files.pythonhosted.org/packages/05/35/bb59b1cd012d7196fc81c2f5879113971efc226a63812c9cf7f89fe97c40/charset_normalizer-3.4.3-cp38-cp38-win_amd64.whl": "5d8d01eac18c423815ed4f4a2ec3b439d654e55ee4ad610e153cf02faf67ea40", + "https://files.pythonhosted.org/packages/05/6b/e2539a0a4be302b481e8cafb5af8792da8093b486885a1ae4d15d452bcec/charset_normalizer-3.4.3-cp312-cp312-musllinux_1_2_ppc64le.whl": "42e5088973e56e31e4fa58eb6bd709e42fc03799c11c42929592889a2e54c491", + "https://files.pythonhosted.org/packages/06/57/84722eefdd338c04cf3030ada66889298eaedf3e7a30a624201e0cbe424a/charset_normalizer-3.4.3-cp314-cp314-musllinux_1_2_s390x.whl": "30a96e1e1f865f78b030d65241c1ee850cdf422d869e9028e2fc1d5e4db73b92", + "https://files.pythonhosted.org/packages/0c/52/8b0c6c3e53f7e546a5e49b9edb876f379725914e1130297f3b423c7b71c5/charset_normalizer-3.4.3-cp38-cp38-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl": "c60e092517a73c632ec38e290eba714e9627abe9d301c8c8a12ec32c314a2a4b", + "https://files.pythonhosted.org/packages/16/ab/0233c3231af734f5dfcf0844aa9582d5a1466c985bbed6cedab85af9bfe3/charset_normalizer-3.4.3-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl": "1606f4a55c0fd363d754049cdf400175ee96c992b1f8018b993941f221221c5f", + "https://files.pythonhosted.org/packages/17/e5/5e67ab85e6d22b04641acb5399c8684f4d37caf7558a53859f0283a650e9/charset_normalizer-3.4.3-cp314-cp314-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl": "2001a39612b241dae17b4687898843f254f8748b796a2e16f1051a17078d991d", + "https://files.pythonhosted.org/packages/1a/79/ae516e678d6e32df2e7e740a7be51dc80b700e2697cb70054a0f1ac2c955/charset_normalizer-3.4.3-cp38-cp38-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl": "3653fad4fe3ed447a596ae8638b437f827234f01a8cd801842e43f3d0a6b281b", + "https://files.pythonhosted.org/packages/20/30/5f64fe3981677fe63fa987b80e6c01042eb5ff653ff7cec1b7bd9268e54e/charset_normalizer-3.4.3-cp39-cp39-musllinux_1_2_ppc64le.whl": "2c322db9c8c89009a990ef07c3bcc9f011a3269bc06782f916cd3d9eed7c9312", + "https://files.pythonhosted.org/packages/21/40/5188be1e3118c82dcb7c2a5ba101b783822cfb413a0268ed3be0468532de/charset_normalizer-3.4.3-cp310-cp310-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl": "cc9370a2da1ac13f0153780040f465839e6cccb4a1e44810124b4e22483c93fe", + "https://files.pythonhosted.org/packages/22/82/63a45bfc36f73efe46731a3a71cb84e2112f7e0b049507025ce477f0f052/charset_normalizer-3.4.3-cp38-cp38-macosx_10_9_universal2.whl": "0f2be7e0cf7754b9a30eb01f4295cc3d4358a479843b31f328afd210e2c7598c", + "https://files.pythonhosted.org/packages/2a/91/26c3036e62dfe8de8061182d33be5025e2424002125c9500faff74a6735e/charset_normalizer-3.4.3-cp310-cp310-win32.whl": "d79c198e27580c8e958906f803e63cddb77653731be08851c7df0b1a14a8fc0f", + "https://files.pythonhosted.org/packages/2f/36/77da9c6a328c54d17b960c89eccacfab8271fdaaa228305330915b88afa9/charset_normalizer-3.4.3-cp311-cp311-musllinux_1_2_x86_64.whl": "1e8ac75d72fa3775e0b7cb7e4629cec13b7514d928d15ef8ea06bca03ef01cae", + "https://files.pythonhosted.org/packages/31/e7/883ee5676a2ef217a40ce0bffcc3d0dfbf9e64cbcfbdf822c52981c3304b/charset_normalizer-3.4.3-cp312-cp312-musllinux_1_2_s390x.whl": "cc34f233c9e71701040d772aa7490318673aa7164a0efe3172b2981218c26d93", + "https://files.pythonhosted.org/packages/33/9e/eca49d35867ca2db336b6ca27617deed4653b97ebf45dfc21311ce473c37/charset_normalizer-3.4.3-cp310-cp310-musllinux_1_2_x86_64.whl": "78deba4d8f9590fe4dae384aeff04082510a709957e968753ff3c48399f6f92a", + "https://files.pythonhosted.org/packages/37/60/5d0d74bc1e1380f0b72c327948d9c2aca14b46a9efd87604e724260f384c/charset_normalizer-3.4.3-cp310-cp310-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl": "07a0eae9e2787b586e129fdcbe1af6997f8d0e5abaa0bc98c0e20e124d67e601", + "https://files.pythonhosted.org/packages/39/c6/99271dc37243a4f925b09090493fb96c9333d7992c6187f5cfe5312008d2/charset_normalizer-3.4.3-cp39-cp39-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl": "23b6b24d74478dc833444cbd927c338349d6ae852ba53a0d02a2de1fce45b96e", + "https://files.pythonhosted.org/packages/39/f5/3b3836ca6064d0992c58c7561c6b6eee1b3892e9665d650c803bd5614522/charset_normalizer-3.4.3-cp312-cp312-win_amd64.whl": "86df271bf921c2ee3818f0522e9a5b8092ca2ad8b065ece5d7d9d0e9f4849bcc", + "https://files.pythonhosted.org/packages/3a/a4/b3b6c76e7a635748c4421d2b92c7b8f90a432f98bda5082049af37ffc8e3/charset_normalizer-3.4.3-cp311-cp311-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl": "00237675befef519d9af72169d8604a067d92755e84fe76492fef5441db05b91", + "https://files.pythonhosted.org/packages/3b/38/20a1f44e4851aa1c9105d6e7110c9d020e093dfa5836d712a5f074a12bf7/charset_normalizer-3.4.3-cp310-cp310-musllinux_1_2_ppc64le.whl": "4ca4c094de7771a98d7fbd67d9e5dbf1eb73efa4f744a730437d8a3a5cf994f0", + "https://files.pythonhosted.org/packages/45/8c/dcef87cfc2b3f002a6478f38906f9040302c68aebe21468090e39cde1445/charset_normalizer-3.4.3-cp39-cp39-musllinux_1_2_x86_64.whl": "88ab34806dea0671532d3f82d82b85e8fc23d7b2dd12fa837978dad9bb392a34", + "https://files.pythonhosted.org/packages/4c/92/27dbe365d34c68cfe0ca76f1edd70e8705d82b378cb54ebbaeabc2e3029d/charset_normalizer-3.4.3-cp311-cp311-musllinux_1_2_ppc64le.whl": "939578d9d8fd4299220161fdd76e86c6a251987476f5243e8864a7844476ba14", + "https://files.pythonhosted.org/packages/50/10/c117806094d2c956ba88958dab680574019abc0c02bcf57b32287afca544/charset_normalizer-3.4.3-cp38-cp38-musllinux_1_2_x86_64.whl": "a2d08ac246bb48479170408d6c19f6385fa743e7157d716e144cad849b2dd94b", + "https://files.pythonhosted.org/packages/50/ee/f4704bad8201de513fdc8aac1cabc87e38c5818c93857140e06e772b5892/charset_normalizer-3.4.3-cp312-cp312-win32.whl": "fb6fecfd65564f208cbf0fba07f107fb661bcd1a7c389edbced3f7a493f70e37", + "https://files.pythonhosted.org/packages/59/c0/a74f3bd167d311365e7973990243f32c35e7a94e45103125275b9e6c479f/charset_normalizer-3.4.3-cp38-cp38-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl": "252098c8c7a873e17dd696ed98bbe91dbacd571da4b87df3736768efa7a792e4", + "https://files.pythonhosted.org/packages/60/f5/4659a4cb3c4ec146bec80c32d8bb16033752574c20b1252ee842a95d1a1e/charset_normalizer-3.4.3-cp313-cp313-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl": "1bb60174149316da1c35fa5233681f7c0f9f514509b8e399ab70fea5f17e45c9", + "https://files.pythonhosted.org/packages/61/c5/dc3ba772489c453621ffc27e8978a98fe7e41a93e787e5e5bde797f1dddb/charset_normalizer-3.4.3-cp38-cp38-win32.whl": "ec557499516fc90fd374bf2e32349a2887a876fbf162c160e3c01b6849eaf557", + "https://files.pythonhosted.org/packages/61/f1/190d9977e0084d3f1dc169acd060d479bbbc71b90bf3e7bf7b9927dec3eb/charset_normalizer-3.4.3-cp311-cp311-musllinux_1_2_aarch64.whl": "96b2b3d1a83ad55310de8c7b4a2d04d9277d5591f40761274856635acc5fcb30", + "https://files.pythonhosted.org/packages/63/86/9cbd533bd37883d467fcd1bd491b3547a3532d0fbb46de2b99feeebf185e/charset_normalizer-3.4.3-cp39-cp39-win32.whl": "16a8770207946ac75703458e2c743631c79c59c5890c80011d536248f8eaa432", + "https://files.pythonhosted.org/packages/64/d1/f9d141c893ef5d4243bc75c130e95af8fd4bc355beff06e9b1e941daad6e/charset_normalizer-3.4.3-cp38-cp38-musllinux_1_2_ppc64le.whl": "5b413b0b1bfd94dbf4023ad6945889f374cd24e3f62de58d6bb102c4d9ae534a", + "https://files.pythonhosted.org/packages/64/d4/9eb4ff2c167edbbf08cdd28e19078bf195762e9bd63371689cab5ecd3d0d/charset_normalizer-3.4.3-cp311-cp311-win32.whl": "6cf8fd4c04756b6b60146d98cd8a77d0cdae0e1ca20329da2ac85eed779b6849", + "https://files.pythonhosted.org/packages/65/1a/7425c952944a6521a9cfa7e675343f83fd82085b8af2b1373a2409c683dc/charset_normalizer-3.4.3-cp310-cp310-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl": "d0e909868420b7049dafd3a31d45125b31143eec59235311fc4c57ea26a4acd2", + "https://files.pythonhosted.org/packages/65/ca/2135ac97709b400c7654b4b764daf5c5567c2da45a30cdd20f9eefe2d658/charset_normalizer-3.4.3-cp313-cp313-macosx_10_13_universal2.whl": "14c2a87c65b351109f6abfc424cab3927b3bdece6f706e4d12faaf3d52ee5efe", + "https://files.pythonhosted.org/packages/70/99/f1c3bdcfaa9c45b3ce96f70b14f070411366fa19549c1d4832c935d8e2c3/charset_normalizer-3.4.3-cp313-cp313-musllinux_1_2_x86_64.whl": "18343b2d246dc6761a249ba1fb13f9ee9a2bcd95decc767319506056ea4ad4dc", + "https://files.pythonhosted.org/packages/71/11/98a04c3c97dd34e49c7d247083af03645ca3730809a5509443f3c37f7c99/charset_normalizer-3.4.3-cp313-cp313-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl": "41d1fc408ff5fdfb910200ec0e74abc40387bccb3252f3f27c0676731df2b2c8", + "https://files.pythonhosted.org/packages/72/2a/aff5dd112b2f14bcc3462c312dce5445806bfc8ab3a7328555da95330e4b/charset_normalizer-3.4.3-cp314-cp314-musllinux_1_2_x86_64.whl": "d716a916938e03231e86e43782ca7878fb602a125a91e7acb8b5112e2e96ac16", + "https://files.pythonhosted.org/packages/77/d9/cbcf1a2a5c7d7856f11e7ac2d782aec12bdfea60d104e60e0aa1c97849dc/charset_normalizer-3.4.3-cp313-cp313-musllinux_1_2_ppc64le.whl": "fdabf8315679312cfa71302f9bd509ded4f2f263fb5b765cf1433b39106c3cc9", + "https://files.pythonhosted.org/packages/7a/03/cbb6fac9d3e57f7e07ce062712ee80d80a5ab46614684078461917426279/charset_normalizer-3.4.3-cp38-cp38-musllinux_1_2_aarch64.whl": "d95bfb53c211b57198bb91c46dd5a2d8018b3af446583aab40074bf7988401cb", + "https://files.pythonhosted.org/packages/7d/a8/c6ec5d389672521f644505a257f50544c074cf5fc292d5390331cd6fc9c3/charset_normalizer-3.4.3-cp312-cp312-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl": "0cacf8f7297b0c4fcb74227692ca46b4a5852f8f4f24b3c766dd94a1075c4884", + "https://files.pythonhosted.org/packages/7e/61/19b36f4bd67f2793ab6a99b979b4e4f3d8fc754cbdffb805335df4337126/charset_normalizer-3.4.3-cp314-cp314-musllinux_1_2_ppc64le.whl": "53cd68b185d98dde4ad8990e56a58dea83a4162161b1ea9272e5c9182ce415e0", + "https://files.pythonhosted.org/packages/7e/95/42aa2156235cbc8fa61208aded06ef46111c4d3f0de233107b3f38631803/charset_normalizer-3.4.3-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl": "416175faf02e4b0810f1f38bcb54682878a4af94059a1cd63b8747244420801f", + "https://files.pythonhosted.org/packages/7f/b5/991245018615474a60965a7c9cd2b4efbaabd16d582a5547c47ee1c7730b/charset_normalizer-3.4.3-cp311-cp311-macosx_10_9_universal2.whl": "b256ee2e749283ef3ddcff51a675ff43798d92d746d1a6e4631bf8c707d22d0b", + "https://files.pythonhosted.org/packages/82/10/0fd19f20c624b278dddaf83b8464dcddc2456cb4b02bb902a6da126b87a1/charset_normalizer-3.4.3-cp312-cp312-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl": "3cfb2aad70f2c6debfbcb717f23b7eb55febc0bb23dcffc0f076009da10c6392", + "https://files.pythonhosted.org/packages/83/2d/5fd176ceb9b2fc619e63405525573493ca23441330fcdaee6bef9460e924/charset_normalizer-3.4.3.tar.gz": "6fce4b8500244f6fcb71465d4a4930d132ba9ab8e71a7859e6a5d59851068d14", + "https://files.pythonhosted.org/packages/85/9a/d891f63722d9158688de58d050c59dc3da560ea7f04f4c53e769de5140f5/charset_normalizer-3.4.3-cp310-cp310-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl": "74d77e25adda8581ffc1c720f1c81ca082921329452eba58b16233ab1842141c", + "https://files.pythonhosted.org/packages/86/9e/f552f7a00611f168b9a5865a1414179b2c6de8235a4fa40189f6f79a1753/charset_normalizer-3.4.3-cp313-cp313-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl": "30d006f98569de3459c2fc1f2acde170b7b2bd265dc1943e87e1a4efe1b67c31", + "https://files.pythonhosted.org/packages/87/df/b7737ff046c974b183ea9aa111b74185ac8c3a326c6262d413bd5a1b8c69/charset_normalizer-3.4.3-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl": "0e78314bdc32fa80696f72fa16dc61168fda4d6a0c014e0380f9d02f0e5d8a07", + "https://files.pythonhosted.org/packages/8a/1f/f041989e93b001bc4e44bb1669ccdcf54d3f00e628229a85b08d330615c5/charset_normalizer-3.4.3-py3-none-any.whl": "ce571ab16d890d23b5c278547ba694193a45011ff86a9162a71307ed9f86759a", + "https://files.pythonhosted.org/packages/8e/91/b5a06ad970ddc7a0e513112d40113e834638f4ca1120eb727a249fb2715e/charset_normalizer-3.4.3-cp314-cp314-macosx_10_13_universal2.whl": "3cd35b7e8aedeb9e34c41385fda4f73ba609e561faedfae0a9e75e44ac558a15", + "https://files.pythonhosted.org/packages/99/04/baae2a1ea1893a01635d475b9261c889a18fd48393634b6270827869fa34/charset_normalizer-3.4.3-cp311-cp311-musllinux_1_2_s390x.whl": "fd10de089bcdcd1be95a2f73dbe6254798ec1bda9f450d5828c96f93e2536b9c", + "https://files.pythonhosted.org/packages/9a/8f/ae790790c7b64f925e5c953b924aaa42a243fb778fed9e41f147b2a5715a/charset_normalizer-3.4.3-cp313-cp313-win_amd64.whl": "cf1ebb7d78e1ad8ec2a8c4732c7be2e736f6e5123a4146c5b89c9d1f585f8cef", + "https://files.pythonhosted.org/packages/a0/e4/5a075de8daa3ec0745a9a3b54467e0c2967daaaf2cec04c845f73493e9a1/charset_normalizer-3.4.3-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl": "18b97b8404387b96cdbd30ad660f6407799126d26a39ca65729162fd810a99aa", + "https://files.pythonhosted.org/packages/a3/ad/b0081f2f99a4b194bcbb1934ef3b12aa4d9702ced80a37026b7607c72e58/charset_normalizer-3.4.3-cp313-cp313-win32.whl": "6fb70de56f1859a3f71261cbe41005f56a7842cc348d3aeb26237560bfa5e0ce", + "https://files.pythonhosted.org/packages/a4/fa/384d2c0f57edad03d7bec3ebefb462090d8905b4ff5a2d2525f3bb711fac/charset_normalizer-3.4.3-cp310-cp310-musllinux_1_2_s390x.whl": "02425242e96bcf29a49711b0ca9f37e451da7c70562bc10e8ed992a5a7a25cc0", + "https://files.pythonhosted.org/packages/ae/02/e29e22b4e02839a0e4a06557b1999d0a47db3567e82989b5bb21f3fbbd9f/charset_normalizer-3.4.3-cp312-cp312-musllinux_1_2_aarch64.whl": "027b776c26d38b7f15b26a5da1044f376455fb3766df8fc38563b4efbc515154", + "https://files.pythonhosted.org/packages/b0/a8/6f5bcf1bcf63cb45625f7c5cadca026121ff8a6c8a3256d8d8cd59302663/charset_normalizer-3.4.3-cp39-cp39-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl": "257f26fed7d7ff59921b78244f3cd93ed2af1800ff048c33f624c87475819dd7", + "https://files.pythonhosted.org/packages/b7/8c/9839225320046ed279c6e839d51f028342eb77c91c89b8ef2549f951f3ec/charset_normalizer-3.4.3-cp314-cp314-win32.whl": "c6dbd0ccdda3a2ba7c2ecd9d77b37f3b5831687d8dc1b6ca5f56a4880cc7b7ce", + "https://files.pythonhosted.org/packages/c1/35/6525b21aa0db614cf8b5792d232021dca3df7f90a1944db934efa5d20bb1/charset_normalizer-3.4.3-cp312-cp312-musllinux_1_2_x86_64.whl": "320e8e66157cc4e247d9ddca8e21f427efc7a04bbd0ac8a9faf56583fa543f9f", + "https://files.pythonhosted.org/packages/c2/a9/3865b02c56f300a6f94fc631ef54f0a8a29da74fb45a773dfd3dcd380af7/charset_normalizer-3.4.3-cp313-cp313-musllinux_1_2_aarch64.whl": "6aab0f181c486f973bc7262a97f5aca3ee7e1437011ef0c2ec04b5a11d16c927", + "https://files.pythonhosted.org/packages/c2/ca/9a0983dd5c8e9733565cf3db4df2b0a2e9a82659fd8aa2a868ac6e4a991f/charset_normalizer-3.4.3-cp39-cp39-macosx_10_9_universal2.whl": "70bfc5f2c318afece2f5838ea5e4c3febada0be750fcf4775641052bbba14d05", + "https://files.pythonhosted.org/packages/c4/72/d3d0e9592f4e504f9dea08b8db270821c909558c353dc3b457ed2509f2fb/charset_normalizer-3.4.3-cp39-cp39-musllinux_1_2_aarch64.whl": "1ef99f0456d3d46a50945c98de1774da86f8e992ab5c77865ea8b8195341fc19", + "https://files.pythonhosted.org/packages/c5/35/9c99739250742375167bc1b1319cd1cec2bf67438a70d84b2e1ec4c9daa3/charset_normalizer-3.4.3-cp38-cp38-musllinux_1_2_s390x.whl": "b5e3b2d152e74e100a9e9573837aba24aab611d39428ded46f4e4022ea7d1942", + "https://files.pythonhosted.org/packages/c7/2a/ae245c41c06299ec18262825c1569c5d3298fc920e4ddf56ab011b417efd/charset_normalizer-3.4.3-cp311-cp311-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl": "13faeacfe61784e2559e690fc53fa4c5ae97c6fcedb8eb6fb8d0a15b475d2c64", + "https://files.pythonhosted.org/packages/ce/d6/7e805c8e5c46ff9729c49950acc4ee0aeb55efb8b3a56687658ad10c3216/charset_normalizer-3.4.3-cp39-cp39-win_amd64.whl": "d22dbedd33326a4a5190dd4fe9e9e693ef12160c77382d9e87919bce54f3d4ca", + "https://files.pythonhosted.org/packages/ce/ec/1edc30a377f0a02689342f214455c3f6c2fbedd896a1d2f856c002fc3062/charset_normalizer-3.4.3-cp314-cp314-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl": "b89bc04de1d83006373429975f8ef9e7932534b8cc9ca582e4db7d20d91816db", + "https://files.pythonhosted.org/packages/d6/98/f3b8013223728a99b908c9344da3aa04ee6e3fa235f19409033eda92fb78/charset_normalizer-3.4.3-cp310-cp310-macosx_10_9_universal2.whl": "fb7f67a1bfa6e40b438170ebdc8158b78dc465a5a67b6dde178a46987b244a72", + "https://files.pythonhosted.org/packages/e1/ef/dd08b2cac9284fd59e70f7d97382c33a3d0a926e45b15fc21b3308324ffd/charset_normalizer-3.4.3-cp39-cp39-musllinux_1_2_s390x.whl": "511729f456829ef86ac41ca78c63a5cb55240ed23b4b737faca0eb1abb1c41bc", + "https://files.pythonhosted.org/packages/e2/c6/f05db471f81af1fa01839d44ae2a8bfeec8d2a8b4590f16c4e7393afd323/charset_normalizer-3.4.3-cp310-cp310-win_amd64.whl": "c6e490913a46fa054e03699c70019ab869e990270597018cef1d8562132c2669", + "https://files.pythonhosted.org/packages/e2/e6/63bb0e10f90a8243c5def74b5b105b3bbbfb3e7bb753915fe333fb0c11ea/charset_normalizer-3.4.3-cp311-cp311-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl": "585f3b2a80fbd26b048a0be90c5aae8f06605d3c92615911c3a2b03a8a3b796f", + "https://files.pythonhosted.org/packages/e4/69/132eab043356bba06eb333cc2cc60c6340857d0a2e4ca6dc2b51312886b3/charset_normalizer-3.4.3-cp39-cp39-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl": "34a7f768e3f985abdb42841e20e17b330ad3aaf4bb7e7aeeb73db2e70f077b99", + "https://files.pythonhosted.org/packages/e9/5e/14c94999e418d9b87682734589404a25854d5f5d0408df68bc15b6ff54bb/charset_normalizer-3.4.3-cp312-cp312-macosx_10_13_universal2.whl": "e28e334d3ff134e88989d90ba04b47d84382a828c061d0d1027b1b12a62b39b1", + "https://files.pythonhosted.org/packages/ee/7a/36fbcf646e41f710ce0a563c1c9a343c6edf9be80786edeb15b6f62e17db/charset_normalizer-3.4.3-cp314-cp314-win_amd64.whl": "73dc19b562516fc9bcf6e5d6e596df0b4eb98d87e4f79f3ae71840e6ed21361c", + "https://files.pythonhosted.org/packages/f0/c9/a2c9c2a355a8594ce2446085e2ec97fd44d323c684ff32042e2a6b718e1d/charset_normalizer-3.4.3-cp310-cp310-musllinux_1_2_aarch64.whl": "c6f162aabe9a91a309510d74eeb6507fab5fff92337a15acbe77753d88d9dcf0", + "https://files.pythonhosted.org/packages/f1/e5/38421987f6c697ee3722981289d554957c4be652f963d71c5e46a262e135/charset_normalizer-3.4.3-cp314-cp314-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl": "8dcfc373f888e4fb39a7bc57e93e3b845e7f462dacc008d9749568b1c4ece096", + "https://files.pythonhosted.org/packages/f4/9c/996a4a028222e7761a96634d1820de8a744ff4327a00ada9c8942033089b/charset_normalizer-3.4.3-cp311-cp311-win_amd64.whl": "31a9a6f775f9bcd865d88ee350f0ffb0e25936a7f930ca98995c05abf1faf21c", + "https://files.pythonhosted.org/packages/f6/42/6f45efee8697b89fda4d50580f292b8f7f9306cb2971d4b53f8914e4d890/charset_normalizer-3.4.3-cp313-cp313-musllinux_1_2_s390x.whl": "bd28b817ea8c70215401f657edef3a8aa83c29d447fb0b622c35403780ba11d5", + "https://files.pythonhosted.org/packages/fc/eb/a2ffb08547f4e1e5415fb69eb7db25932c52a52bed371429648db4d84fb1/charset_normalizer-3.4.3-cp312-cp312-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl": "c6fd51128a41297f5409deab284fecbe5305ebd7e5a1f959bee1c054622b7018" + }, + "cryptography": { + "https://files.pythonhosted.org/packages/03/11/5e395f961d6868269835dee1bafec6a1ac176505a167f68b7d8818431068/cryptography-46.0.7-cp314-cp314t-manylinux2014_x86_64.manylinux_2_17_x86_64.whl": "ebd6daf519b9f189f85c479427bbd6e9c9037862cf8fe89ee35503bd209ed902", + "https://files.pythonhosted.org/packages/0b/5d/4a8f770695d73be252331e60e526291e3df0c9b27556a90a6b47bccca4c2/cryptography-46.0.7-cp311-abi3-macosx_10_9_universal2.whl": "ea42cbe97209df307fdc3b155f1b6fa2577c0defa8f1f7d3be7d31d189108ad4", + "https://files.pythonhosted.org/packages/0f/54/6bbbfc5efe86f9d71041827b793c24811a017c6ac0fd12883e4caa86b8ed/cryptography-46.0.7-cp311-abi3-manylinux_2_28_ppc64le.whl": "cbd5fb06b62bd0721e1170273d3f4d5a277044c47ca27ee257025146c34cbdd1", + "https://files.pythonhosted.org/packages/10/f2/19ceb3b3dc14009373432af0c13f46aa08e3ce334ec6eff13492e1812ccd/cryptography-46.0.7-cp311-abi3-musllinux_1_2_x86_64.whl": "5d1c02a14ceb9148cc7816249f64f623fbfee39e8c03b3650d842ad3f34d637e", + "https://files.pythonhosted.org/packages/16/01/0cd51dd86ab5b9befe0d031e276510491976c3a80e9f6e31810cce46c4ad/cryptography-46.0.7-cp38-abi3-manylinux_2_31_armv7l.whl": "cdfbe22376065ffcf8be74dc9a909f032df19bc58a699456a21712d6e5eabfd0", + "https://files.pythonhosted.org/packages/1a/bb/a5c213c19ee94b15dfccc48f363738633a493812687f5567addbcbba9f6f/cryptography-46.0.7-cp311-abi3-win32.whl": "d23c8ca48e44ee015cd0a54aeccdf9f09004eba9fc96f38c911011d9ff1bd457", + "https://files.pythonhosted.org/packages/20/2a/1b016902351a523aa2bd446b50a5bc1175d7a7d1cf90fe2ef904f9b84ebc/cryptography-46.0.7-pp311-pypy311_pp73-win_amd64.whl": "258514877e15963bd43b558917bc9f54cf7cf866c38aa576ebf47a77ddbc43a4", + "https://files.pythonhosted.org/packages/28/17/b59a741645822ec6d04732b43c5d35e4ef58be7bfa84a81e5ae6f05a1d33/cryptography-46.0.7-cp314-cp314t-musllinux_1_2_aarch64.whl": "fcd8eac50d9138c1d7fc53a653ba60a2bee81a505f9f8850b6b2888555a45d0e", + "https://files.pythonhosted.org/packages/2b/02/7788f9fefa1d060ca68717c3901ae7fffa21ee087a90b7f23c7a603c32ae/cryptography-46.0.7-cp311-abi3-win_amd64.whl": "397655da831414d165029da9bc483bed2fe0e75dde6a1523ec2fe63f3c46046b", + "https://files.pythonhosted.org/packages/2d/cf/054b9d8220f81509939599c8bdbc0c408dbd2bdd41688616a20731371fe0/cryptography-46.0.7-cp311-abi3-manylinux_2_28_x86_64.whl": "420b1e4109cc95f0e5700eed79908cef9268265c773d3a66f7af1eef53d409ef", + "https://files.pythonhosted.org/packages/32/a8/9f0e4ed57ec9cebe506e58db11ae472972ecb0c659e4d52bbaee80ca340a/cryptography-46.0.7-cp314-cp314t-win_amd64.whl": "e06acf3c99be55aa3b516397fe42f5855597f430add9c17fa46bf2e0fb34c9bb", + "https://files.pythonhosted.org/packages/36/5f/313586c3be5a2fbe87e4c9a254207b860155a8e1f3cca99f9910008e7d08/cryptography-46.0.7-cp311-abi3-manylinux_2_34_aarch64.whl": "8a469028a86f12eb7d2fe97162d0634026d92a21f3ae0ac87ed1c4a447886c83", + "https://files.pythonhosted.org/packages/3a/ea/075aac6a84b7c271578d81a2f9968acb6e273002408729f2ddff517fed4a/cryptography-46.0.7-pp311-pypy311_pp73-manylinux_2_28_aarch64.whl": "d3b99c535a9de0adced13d159c5a9cf65c325601aa30f4be08afd680643e9c15", + "https://files.pythonhosted.org/packages/3d/4c/7d258f169ae71230f25d9f3d06caabcff8c3baf0978e2b7d65e0acac3827/cryptography-46.0.7-cp314-cp314t-manylinux_2_31_armv7l.whl": "60627cf07e0d9274338521205899337c5d18249db56865f943cbe753aa96f40f", + "https://files.pythonhosted.org/packages/40/53/8ed1cf4c3b9c8e611e7122fb56f1c32d09e1fff0f1d77e78d9ff7c82653e/cryptography-46.0.7-cp314-cp314t-manylinux_2_28_aarch64.whl": "b7b412817be92117ec5ed95f880defe9cf18a832e8cafacf0a22337dc1981b4d", + "https://files.pythonhosted.org/packages/41/3d/fe14df95a83319af25717677e956567a105bb6ab25641acaa093db79975d/cryptography-46.0.7-cp314-cp314t-manylinux_2_34_ppc64le.whl": "c5b1ccd1239f48b7151a65bc6dd54bcfcc15e028c8ac126d3fada09db0e07ef1", + "https://files.pythonhosted.org/packages/41/52/a8908dcb1a389a459a29008c29966c1d552588d4ae6d43f3a1a4512e0ebe/cryptography-46.0.7-cp38-abi3-musllinux_1_2_x86_64.whl": "a1529d614f44b863a7b480c6d000fe93b59acee9c82ffa027cfadc77521a9f5e", + "https://files.pythonhosted.org/packages/47/93/ac8f3d5ff04d54bc814e961a43ae5b0b146154c89c61b47bb07557679b18/cryptography-46.0.7.tar.gz": "e4cfd68c5f3e0bfdad0d38e023239b96a2fe84146481852dffbcca442c245aa5", + "https://files.pythonhosted.org/packages/4a/9a/1765afe9f572e239c3469f2cb429f3ba7b31878c893b246b4b2994ffe2fe/cryptography-46.0.7-cp311-abi3-manylinux2014_x86_64.manylinux_2_17_x86_64.whl": "5ad9ef796328c5e3c4ceed237a183f5d41d21150f972455a9d926593a1dcb308", + "https://files.pythonhosted.org/packages/4b/fa/f0ab06238e899cc3fb332623f337a7364f36f4bb3f2534c2bb95a35b132c/cryptography-46.0.7-cp38-abi3-win32.whl": "f247c8c1a1fb45e12586afbb436ef21ff1e80670b2861a90353d9b025583d246", + "https://files.pythonhosted.org/packages/50/46/cf71e26025c2e767c5609162c866a78e8a2915bbcfa408b7ca495c6140c4/cryptography-46.0.7-cp314-cp314t-manylinux_2_28_ppc64le.whl": "fbfd0e5f273877695cb93baf14b185f4878128b250cc9f8e617ea0c025dfb022", + "https://files.pythonhosted.org/packages/59/6a/bb2e166d6d0e0955f1e9ff70f10ec4b2824c9cfcdb4da772c7dd69cc7d80/cryptography-46.0.7-cp314-cp314t-musllinux_1_2_x86_64.whl": "65814c60f8cc400c63131584e3e1fad01235edba2614b61fbfbfa954082db0ee", + "https://files.pythonhosted.org/packages/5f/45/6d80dc379b0bbc1f9d1e429f42e4cb9e1d319c7a8201beffd967c516ea01/cryptography-46.0.7-cp311-abi3-manylinux2014_aarch64.manylinux_2_17_aarch64.whl": "b36a4695e29fe69215d75960b22577197aca3f7a25b9cf9d165dcfe9d80bc325", + "https://files.pythonhosted.org/packages/63/0c/dca8abb64e7ca4f6b2978769f6fea5ad06686a190cec381f0a796fdcaaba/cryptography-46.0.7-pp311-pypy311_pp73-macosx_11_0_arm64.whl": "fc9ab8856ae6cf7c9358430e49b368f3108f050031442eaeb6b9d87e4dcf4e4f", + "https://files.pythonhosted.org/packages/69/33/60dfc4595f334a2082749673386a4d05e4f0cf4df8248e63b2c3437585f2/cryptography-46.0.7-cp311-abi3-manylinux_2_34_ppc64le.whl": "9694078c5d44c157ef3162e3bf3946510b857df5a3955458381d1c7cfc143ddb", + "https://files.pythonhosted.org/packages/6c/7b/1c55db7242b5e5612b29fc7a630e91ee7a6e3c8e7bf5406d22e206875fbd/cryptography-46.0.7-pp311-pypy311_pp73-manylinux_2_28_x86_64.whl": "d02c738dacda7dc2a74d1b2b3177042009d5cab7c7079db74afc19e56ca1b455", + "https://files.pythonhosted.org/packages/74/66/e3ce040721b0b5599e175ba91ab08884c75928fbeb74597dd10ef13505d2/cryptography-46.0.7-cp314-cp314t-manylinux2014_aarch64.manylinux_2_17_aarch64.whl": "db0f493b9181c7820c8134437eb8b0b4792085d37dbb24da050476ccb664e59c", + "https://files.pythonhosted.org/packages/7b/56/15619b210e689c5403bb0540e4cb7dbf11a6bf42e483b7644e471a2812b3/cryptography-46.0.7-cp314-cp314t-macosx_10_9_universal2.whl": "d151173275e1728cf7839aaa80c34fe550c04ddb27b34f48c232193df8db5842", + "https://files.pythonhosted.org/packages/80/07/ad9b3c56ebb95ed2473d46df0847357e01583f4c52a85754d1a55e29e4d0/cryptography-46.0.7-cp38-abi3-manylinux_2_34_ppc64le.whl": "935ce7e3cfdb53e3536119a542b839bb94ec1ad081013e9ab9b7cfd478b05006", + "https://files.pythonhosted.org/packages/8a/6c/1a42450f464dda6ffbe578a911f773e54dd48c10f9895a23a7e88b3e7db5/cryptography-46.0.7-cp38-abi3-manylinux2014_x86_64.manylinux_2_17_x86_64.whl": "128c5edfe5e5938b86b03941e94fac9ee793a94452ad1365c9fc3f4f62216832", + "https://files.pythonhosted.org/packages/8f/3e/af9246aaf23cd4ee060699adab1e47ced3f5f7e7a8ffdd339f817b446462/cryptography-46.0.7-cp311-abi3-manylinux_2_28_aarch64.whl": "73510b83623e080a2c35c62c15298096e2a5dc8d51c3b4e1740211839d0dea77", + "https://files.pythonhosted.org/packages/92/49/819d6ed3a7d9349c2939f81b500a738cb733ab62fbecdbc1e38e83d45e12/cryptography-46.0.7-cp38-abi3-manylinux_2_34_aarch64.whl": "abad9dac36cbf55de6eb49badd4016806b3165d396f64925bf2999bcb67837ba", + "https://files.pythonhosted.org/packages/95/b6/3da51d48415bcb63b00dc17c2eff3a651b7c4fed484308d0f19b30e8cb2c/cryptography-46.0.7-cp314-cp314t-win32.whl": "fdd1736fed309b4300346f88f74cd120c27c56852c3838cab416e7a166f67298", + "https://files.pythonhosted.org/packages/9a/92/4ed714dbe93a066dc1f4b4581a464d2d7dbec9046f7c8b7016f5286329e2/cryptography-46.0.7-cp38-abi3-manylinux_2_28_aarch64.whl": "5e51be372b26ef4ba3de3c167cd3d1022934bc838ae9eaad7e644986d2a3d163", + "https://files.pythonhosted.org/packages/9c/59/4a479e0f36f8f378d397f4eab4c850b4ffb79a2f0d58704b8fa0703ddc11/cryptography-46.0.7-cp314-cp314t-manylinux_2_34_x86_64.whl": "d5f7520159cd9c2154eb61eb67548ca05c5774d39e9c2c4339fd793fe7d097b2", + "https://files.pythonhosted.org/packages/a5/d0/36a49f0262d2319139d2829f773f1b97ef8aef7f97e6e5bd21455e5a8fb5/cryptography-46.0.7-cp38-abi3-manylinux2014_aarch64.manylinux_2_17_aarch64.whl": "84d4cced91f0f159a7ddacad249cc077e63195c36aac40b4150e7a57e84fffe7", + "https://files.pythonhosted.org/packages/a5/ef/649750cbf96f3033c3c976e112265c33906f8e462291a33d77f90356548c/cryptography-46.0.7-cp38-abi3-musllinux_1_2_aarch64.whl": "7bbc6ccf49d05ac8f7d7b5e2e2c33830d4fe2061def88210a126d130d7f71a85", + "https://files.pythonhosted.org/packages/a7/7f/cd42fc3614386bc0c12f0cb3c4ae1fc2bbca5c9662dfed031514911d513d/cryptography-46.0.7-cp38-abi3-macosx_10_9_universal2.whl": "462ad5cb1c148a22b2e3bcc5ad52504dff325d17daf5df8d88c17dda1f75f2a4", + "https://files.pythonhosted.org/packages/b5/2a/2ea0767cad19e71b3530e4cad9605d0b5e338b6a1e72c37c9c1ceb86c333/cryptography-46.0.7-cp314-cp314t-manylinux_2_34_aarch64.whl": "80406c3065e2c55d7f49a9550fe0c49b3f12e5bfff5dedb727e319e1afb9bf99", + "https://files.pythonhosted.org/packages/b7/e6/a26b84096eddd51494bba19111f8fffe976f6a09f132706f8f1bf03f51f7/cryptography-46.0.7-cp38-abi3-manylinux_2_28_ppc64le.whl": "cdf1a610ef82abb396451862739e3fc93b071c844399e15b90726ef7470eeaf2", + "https://files.pythonhosted.org/packages/b8/c7/201d3d58f30c4c2bdbe9b03844c291feb77c20511cc3586daf7edc12a47b/cryptography-46.0.7-cp38-abi3-manylinux_2_34_x86_64.whl": "35719dc79d4730d30f1c2b6474bd6acda36ae2dfae1e3c16f2051f215df33ce0", + "https://files.pythonhosted.org/packages/c0/ea/01276740375bac6249d0a971ebdf6b4dc9ead0ee0a34ef3b5a88c1a9b0d4/cryptography-46.0.7-cp314-cp314t-manylinux_2_28_x86_64.whl": "ffca7aa1d00cf7d6469b988c581598f2259e46215e0140af408966a24cf086ce", + "https://files.pythonhosted.org/packages/c7/08/ffd537b605568a148543ac3c2b239708ae0bd635064bab41359252ef88ed/cryptography-46.0.7-cp38-abi3-manylinux_2_28_x86_64.whl": "1d25aee46d0c6f1a501adcddb2d2fee4b979381346a78558ed13e50aa8a59067", + "https://files.pythonhosted.org/packages/c7/0b/333ddab4270c4f5b972f980adef4faa66951a4aaf646ca067af597f15563/cryptography-46.0.7-cp311-abi3-manylinux_2_34_x86_64.whl": "42a1e5f98abb6391717978baf9f90dc28a743b7d9be7f0751a6f56a75d14065b", + "https://files.pythonhosted.org/packages/cb/da/9870eec4b69c63ef5925bf7d8342b7e13bc2ee3d47791461c4e49ca212f4/cryptography-46.0.7-pp311-pypy311_pp73-manylinux_2_34_aarch64.whl": "04959522f938493042d595a736e7dbdff6eb6cc2339c11465b3ff89343b65f65", + "https://files.pythonhosted.org/packages/d2/14/633913398b43b75f1234834170947957c6b623d1701ffc7a9600da907e89/cryptography-46.0.7-cp311-abi3-musllinux_1_2_aarch64.whl": "91bbcb08347344f810cbe49065914fe048949648f6bd5c2519f34619142bbe85", + "https://files.pythonhosted.org/packages/d2/f1/00ce3bde3ca542d1acd8f8cfa38e446840945aa6363f9b74746394b14127/cryptography-46.0.7-cp38-abi3-win_amd64.whl": "506c4ff91eff4f82bdac7633318a526b1d1309fc07ca76a3ad182cb5b686d6d3", + "https://files.pythonhosted.org/packages/f4/72/05aa5832b82dd341969e9a734d1812a6aadb088d9eb6f0430fc337cc5a8f/cryptography-46.0.7-pp311-pypy311_pp73-manylinux_2_34_x86_64.whl": "3986ac1dee6def53797289999eabe84798ad7817f3e97779b5061a95b0ee4968", + "https://files.pythonhosted.org/packages/f9/46/4e4e9c6040fb01c7467d47217d2f882daddeb8828f7df800cb806d8a2288/cryptography-46.0.7-cp311-abi3-manylinux_2_31_armv7l.whl": "24402210aa54baae71d99441d15bb5a1919c195398a87b563df84468160a65de" + }, + "docutils": { + "https://files.pythonhosted.org/packages/4a/c0/89fe6215b443b919cb98a5002e107cb5026854ed1ccb6b5833e0768419d1/docutils-0.22.2.tar.gz": "9fdb771707c8784c8f2728b67cb2c691305933d68137ef95a75db5f4dfbc213d", + "https://files.pythonhosted.org/packages/66/dd/f95350e853a4468ec37478414fc04ae2d61dad7a947b3015c3dcc51a09b9/docutils-0.22.2-py3-none-any.whl": "b0e98d679283fc3bb0ead8a5da7f501baa632654e7056e9c5846842213d674d8" + }, + "idna": { + "https://files.pythonhosted.org/packages/76/c6/c88e154df9c4e1a2a66ccf0005a88dfb2650c1dffb6f5ce603dfbd452ce3/idna-3.10-py3-none-any.whl": "946d195a0d259cbba61165e88e65941f16e9b36ea6ddb97f00452bae8b1287d3", + "https://files.pythonhosted.org/packages/f1/70/7703c29685631f5a7590aa73f1f1d3fa9a380e654b86af429e0934a32f7d/idna-3.10.tar.gz": "12f65c9b470abda6dc35cf8e63cc574b1c52b11df2c86030af0ac09b01b13ea9" + }, + "importlib-metadata": { + "https://files.pythonhosted.org/packages/20/b0/36bd937216ec521246249be3bf9855081de4c5e06a0c9b4219dbeda50373/importlib_metadata-8.7.0-py3-none-any.whl": "e5dd1551894c77868a30651cef00984d50e1002d06942a7101d34870c5f02afd", + "https://files.pythonhosted.org/packages/76/66/650a33bd90f786193e4de4b3ad86ea60b53c89b669a5c7be931fac31cdb0/importlib_metadata-8.7.0.tar.gz": "d13b81ad223b890aa16c5471f2ac3056cf76c5f10f82d6f9292f0b415f389000" + }, + "jaraco-classes": { + "https://files.pythonhosted.org/packages/06/c0/ed4a27bc5571b99e3cff68f8a9fa5b56ff7df1c2251cc715a652ddd26402/jaraco.classes-3.4.0.tar.gz": "47a024b51d0239c0dd8c8540c6c7f484be3b8fcf0b2d85c13825780d3b3f3acd", + "https://files.pythonhosted.org/packages/7f/66/b15ce62552d84bbfcec9a4873ab79d993a1dd4edb922cbfccae192bd5b5f/jaraco.classes-3.4.0-py3-none-any.whl": "f662826b6bed8cace05e7ff873ce0f9283b5c924470fe664fff1c2f00f581790" + }, + "jaraco-context": { + "https://files.pythonhosted.org/packages/df/ad/f3777b81bf0b6e7bc7514a1656d3e637b2e8e15fab2ce3235730b3e7a4e6/jaraco_context-6.0.1.tar.gz": "9bae4ea555cf0b14938dc0aee7c9f32ed303aa20a3b73e7dc80111628792d1b3", + "https://files.pythonhosted.org/packages/ff/db/0c52c4cf5e4bd9f5d7135ec7669a3a767af21b3a308e1ed3674881e52b62/jaraco.context-6.0.1-py3-none-any.whl": "f797fc481b490edb305122c9181830a3a5b76d84ef6d1aef2fb9b47ab956f9e4" + }, + "jaraco-functools": { + "https://files.pythonhosted.org/packages/b4/09/726f168acad366b11e420df31bf1c702a54d373a83f968d94141a8c3fde0/jaraco_functools-4.3.0-py3-none-any.whl": "227ff8ed6f7b8f62c56deff101545fa7543cf2c8e7b82a7c2116e672f29c26e8", + "https://files.pythonhosted.org/packages/f7/ed/1aa2d585304ec07262e1a83a9889880701079dde796ac7b1d1826f40c63d/jaraco_functools-4.3.0.tar.gz": "cfd13ad0dd2c47a3600b439ef72d8615d482cedcff1632930d6f28924d92f294" + }, + "jeepney": { + "https://files.pythonhosted.org/packages/7b/6f/357efd7602486741aa73ffc0617fb310a29b588ed0fd69c2399acbb85b0c/jeepney-0.9.0.tar.gz": "cf0e9e845622b81e4a28df94c40345400256ec608d0e55bb8a3feaa9163f5732", + "https://files.pythonhosted.org/packages/b2/a3/e137168c9c44d18eff0376253da9f1e9234d0239e0ee230d2fee6cea8e55/jeepney-0.9.0-py3-none-any.whl": "97e5714520c16fc0a45695e5365a2e11b81ea79bba796e26f9f1d178cb182683" + }, + "keyring": { + "https://files.pythonhosted.org/packages/70/09/d904a6e96f76ff214be59e7aa6ef7190008f52a0ab6689760a98de0bf37d/keyring-25.6.0.tar.gz": "0b39998aa941431eb3d9b0d4b2460bc773b9df6fed7621c2dfb291a7e0187a66", + "https://files.pythonhosted.org/packages/d3/32/da7f44bcb1105d3e88a0b74ebdca50c59121d2ddf71c9e34ba47df7f3a56/keyring-25.6.0-py3-none-any.whl": "552a3f7af126ece7ed5c89753650eec89c7eaae8617d0aa4d9ad2b75111266bd" + }, + "markdown-it-py": { + "https://files.pythonhosted.org/packages/5b/f5/4ec618ed16cc4f8fb3b701563655a69816155e79e24a17b651541804721d/markdown_it_py-4.0.0.tar.gz": "cb0a2b4aa34f932c007117b194e945bd74e0ec24133ceb5bac59009cda1cb9f3", + "https://files.pythonhosted.org/packages/94/54/e7d793b573f298e1c9013b8c4dade17d481164aa517d1d7148619c2cedbf/markdown_it_py-4.0.0-py3-none-any.whl": "87327c59b172c5011896038353a81343b6754500a08cd7a4973bb48c6d578147" + }, + "mdurl": { + "https://files.pythonhosted.org/packages/b3/38/89ba8ad64ae25be8de66a6d463314cf1eb366222074cfda9ee839c56a4b4/mdurl-0.1.2-py3-none-any.whl": "84008a41e51615a49fc9966191ff91509e3c40b939176e643fd50a5c2196b8f8", + "https://files.pythonhosted.org/packages/d6/54/cfe61301667036ec958cb99bd3efefba235e65cdeb9c84d24a8293ba1d90/mdurl-0.1.2.tar.gz": "bb413d29f5eea38f31dd4754dd7377d4465116fb207585f97bf925588687c1ba" + }, + "more-itertools": { + "https://files.pythonhosted.org/packages/a4/8e/469e5a4a2f5855992e425f3cb33804cc07bf18d48f2db061aec61ce50270/more_itertools-10.8.0-py3-none-any.whl": "52d4362373dcf7c52546bc4af9a86ee7c4579df9a8dc268be0a2f949d376cc9b", + "https://files.pythonhosted.org/packages/ea/5d/38b681d3fce7a266dd9ab73c66959406d565b3e85f21d5e66e1181d93721/more_itertools-10.8.0.tar.gz": "f638ddf8a1a0d134181275fb5d58b086ead7c6a72429ad725c67503f13ba30bd" + }, + "nh3": { + "https://files.pythonhosted.org/packages/0c/e0/cf1543e798ba86d838952e8be4cb8d18e22999be2a24b112a671f1c04fd6/nh3-0.3.0-cp38-abi3-macosx_10_12_x86_64.macosx_11_0_arm64.macosx_10_12_universal2.whl": "ec6cfdd2e0399cb79ba4dcffb2332b94d9696c52272ff9d48a630c5dca5e325a", + "https://files.pythonhosted.org/packages/10/71/2fb1834c10fab6d9291d62c95192ea2f4c7518bd32ad6c46aab5d095cb87/nh3-0.3.0-cp313-cp313t-musllinux_1_2_i686.whl": "0649464ac8eee018644aacbc103874ccbfac80e3035643c3acaab4287e36e7f5", + "https://files.pythonhosted.org/packages/23/1e/80a8c517655dd40bb13363fc4d9e66b2f13245763faab1a20f1df67165a7/nh3-0.3.0-cp313-cp313t-win_amd64.whl": "423201bbdf3164a9e09aa01e540adbb94c9962cc177d5b1cbb385f5e1e79216e", + "https://files.pythonhosted.org/packages/2f/d6/f1c6e091cbe8700401c736c2bc3980c46dca770a2cf6a3b48a175114058e/nh3-0.3.0-cp313-cp313t-win32.whl": "7275fdffaab10cc5801bf026e3c089d8de40a997afc9e41b981f7ac48c5aa7d5", + "https://files.pythonhosted.org/packages/33/c1/8f8ccc2492a000b6156dce68a43253fcff8b4ce70ab4216d08f90a2ac998/nh3-0.3.0-cp313-cp313t-musllinux_1_2_x86_64.whl": "1adeb1062a1c2974bc75b8d1ecb014c5fd4daf2df646bbe2831f7c23659793f9", + "https://files.pythonhosted.org/packages/39/2c/6394301428b2017a9d5644af25f487fa557d06bc8a491769accec7524d9a/nh3-0.3.0-cp38-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl": "f416c35efee3e6a6c9ab7716d9e57aa0a49981be915963a82697952cba1353e1", + "https://files.pythonhosted.org/packages/4c/3c/cba7b26ccc0ef150c81646478aa32f9c9535234f54845603c838a1dc955c/nh3-0.3.0-cp313-cp313t-musllinux_1_2_aarch64.whl": "80fe20171c6da69c7978ecba33b638e951b85fb92059259edd285ff108b82a6d", + "https://files.pythonhosted.org/packages/4e/9a/344b9f9c4bd1c2413a397f38ee6a3d5db30f1a507d4976e046226f12b297/nh3-0.3.0-cp38-abi3-manylinux_2_5_i686.manylinux1_i686.whl": "37d3003d98dedca6cd762bf88f2e70b67f05100f6b949ffe540e189cc06887f9", + "https://files.pythonhosted.org/packages/5b/76/3165e84e5266d146d967a6cc784ff2fbf6ddd00985a55ec006b72bc39d5d/nh3-0.3.0-cp38-abi3-win_arm64.whl": "d97d3efd61404af7e5721a0e74d81cdbfc6e5f97e11e731bb6d090e30a7b62b2", + "https://files.pythonhosted.org/packages/5c/86/a96b1453c107b815f9ab8fac5412407c33cc5c7580a4daf57aabeb41b774/nh3-0.3.0-cp38-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl": "ce5e7185599f89b0e391e2f29cc12dc2e206167380cea49b33beda4891be2fe1", + "https://files.pythonhosted.org/packages/63/da/c5fd472b700ba37d2df630a9e0d8cc156033551ceb8b4c49cc8a5f606b68/nh3-0.3.0-cp313-cp313t-manylinux_2_5_i686.manylinux1_i686.whl": "ba0caa8aa184196daa6e574d997a33867d6d10234018012d35f86d46024a2a95", + "https://files.pythonhosted.org/packages/66/3f/cd37f76c8ca277b02a84aa20d7bd60fbac85b4e2cbdae77cb759b22de58b/nh3-0.3.0-cp38-abi3-musllinux_1_2_aarch64.whl": "634e34e6162e0408e14fb61d5e69dbaea32f59e847cfcfa41b66100a6b796f62", + "https://files.pythonhosted.org/packages/6a/1b/b15bd1ce201a1a610aeb44afd478d55ac018b4475920a3118ffd806e2483/nh3-0.3.0-cp38-abi3-manylinux_2_17_ppc64.manylinux2014_ppc64.whl": "e9e6a7e4d38f7e8dda9edd1433af5170c597336c1a74b4693c5cb75ab2b30f2a", + "https://files.pythonhosted.org/packages/8c/ae/324b165d904dc1672eee5f5661c0a68d4bab5b59fbb07afb6d8d19a30b45/nh3-0.3.0-cp38-abi3-win_amd64.whl": "bae63772408fd63ad836ec569a7c8f444dd32863d0c67f6e0b25ebbd606afa95", + "https://files.pythonhosted.org/packages/8f/14/079670fb2e848c4ba2476c5a7a2d1319826053f4f0368f61fca9bb4227ae/nh3-0.3.0-cp38-abi3-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl": "7852f038a054e0096dac12b8141191e02e93e0b4608c4b993ec7d4ffafea4e49", + "https://files.pythonhosted.org/packages/97/03/03f79f7e5178eb1ad5083af84faff471e866801beb980cc72943a4397368/nh3-0.3.0-cp38-abi3-musllinux_1_2_i686.whl": "c7a32a7f0d89f7d30cb8f4a84bdbd56d1eb88b78a2434534f62c71dac538c450", + "https://files.pythonhosted.org/packages/97/33/11e7273b663839626f714cb68f6eb49899da5a0d9b6bc47b41fe870259c2/nh3-0.3.0-cp38-abi3-manylinux_2_17_armv7l.manylinux2014_armv7l.whl": "389d93d59b8214d51c400fb5b07866c2a4f79e4e14b071ad66c92184fec3a392", + "https://files.pythonhosted.org/packages/9a/e0/af86d2a974c87a4ba7f19bc3b44a8eaa3da480de264138fec82fe17b340b/nh3-0.3.0-cp313-cp313t-win_arm64.whl": "16f8670201f7e8e0e05ed1a590eb84bfa51b01a69dd5caf1d3ea57733de6a52f", + "https://files.pythonhosted.org/packages/a3/e5/ac7fc565f5d8bce7f979d1afd68e8cb415020d62fa6507133281c7d49f91/nh3-0.3.0-cp38-abi3-manylinux_2_17_s390x.manylinux2014_s390x.whl": "af5aa8127f62bbf03d68f67a956627b1bd0469703a35b3dad28d0c1195e6c7fb", + "https://files.pythonhosted.org/packages/ad/7f/7c6b8358cf1222921747844ab0eef81129e9970b952fcb814df417159fb9/nh3-0.3.0-cp313-cp313t-manylinux_2_17_x86_64.manylinux2014_x86_64.whl": "7c915060a2c8131bef6a29f78debc29ba40859b6dbe2362ef9e5fd44f11487c2", + "https://files.pythonhosted.org/packages/b4/11/340b7a551916a4b2b68c54799d710f86cf3838a4abaad8e74d35360343bb/nh3-0.3.0-cp313-cp313t-macosx_10_12_x86_64.macosx_11_0_arm64.macosx_10_12_universal2.whl": "a537ece1bf513e5a88d8cff8a872e12fe8d0f42ef71dd15a5e7520fecd191bbb", + "https://files.pythonhosted.org/packages/c3/a4/96cff0977357f60f06ec4368c4c7a7a26cccfe7c9fcd54f5378bf0428fd3/nh3-0.3.0.tar.gz": "d8ba24cb31525492ea71b6aac11a4adac91d828aadeff7c4586541bf5dc34d2f", + "https://files.pythonhosted.org/packages/c9/50/76936ec021fe1f3270c03278b8af5f2079038116b5d0bfe8538ffe699d69/nh3-0.3.0-cp38-abi3-win32.whl": "6d68fa277b4a3cf04e5c4b84dd0c6149ff7d56c12b3e3fab304c525b850f613d", + "https://files.pythonhosted.org/packages/ce/55/1974bcc16884a397ee699cebd3914e1f59be64ab305533347ca2d983756f/nh3-0.3.0-cp38-abi3-musllinux_1_2_x86_64.whl": "3f1b4f8a264a0c86ea01da0d0c390fe295ea0bcacc52c2103aca286f6884f518", + "https://files.pythonhosted.org/packages/ee/db/7aa11b44bae4e7474feb1201d8dee04fabe5651c7cb51409ebda94a4ed67/nh3-0.3.0-cp38-abi3-musllinux_1_2_armv7l.whl": "b0612ccf5de8a480cf08f047b08f9d3fecc12e63d2ee91769cb19d7290614c23", + "https://files.pythonhosted.org/packages/f3/ba/59e204d90727c25b253856e456ea61265ca810cda8ee802c35f3fadaab00/nh3-0.3.0-cp313-cp313t-musllinux_1_2_armv7l.whl": "e90883f9f85288f423c77b3f5a6f4486375636f25f793165112679a7b6363b35" + }, + "pkginfo": { + "https://files.pythonhosted.org/packages/24/03/e26bf3d6453b7fda5bd2b84029a426553bb373d6277ef6b5ac8863421f87/pkginfo-1.12.1.2.tar.gz": "5cd957824ac36f140260964eba3c6be6442a8359b8c48f4adf90210f33a04b7b", + "https://files.pythonhosted.org/packages/fa/3d/f4f2ba829efb54b6cd2d91349c7463316a9cc55a43fc980447416c88540f/pkginfo-1.12.1.2-py3-none-any.whl": "c783ac885519cab2c34927ccfa6bf64b5a704d7c69afaea583dd9b7afe969343" + }, + "pycparser": { + "https://files.pythonhosted.org/packages/0c/c3/44f3fbbfa403ea2a7c779186dc20772604442dde72947e7d01069cbe98e3/pycparser-3.0-py3-none-any.whl": "b727414169a36b7d524c1c3e31839a521725078d7b2ff038656844266160a992", + "https://files.pythonhosted.org/packages/1b/7d/92392ff7815c21062bea51aa7b87d45576f649f16458d78b7cf94b9ab2e6/pycparser-3.0.tar.gz": "600f49d217304a5902ac3c37e1281c9fe94e4d0489de643a9504c5cdfdfc6b29", + "https://files.pythonhosted.org/packages/a0/e3/59cd50310fc9b59512193629e1984c1f95e5c8ae6e5d8c69532ccc65a7fe/pycparser-2.23-py3-none-any.whl": "e5c6e8d3fbad53479cab09ac03729e0a9faf2bee3db8208a550daf5af81a5934", + "https://files.pythonhosted.org/packages/fe/cf/d2d3b9f5699fb1e4615c8e32ff220203e43b248e1dfcc6736ad9057731ca/pycparser-2.23.tar.gz": "78816d4f24add8f10a06d6f05b4d424ad9e96cfebf68a4ddc99c65c0720d00c2" + }, + "pygments": { + "https://files.pythonhosted.org/packages/b0/77/a5b8c569bf593b0140bde72ea885a803b82086995367bf2037de0159d924/pygments-2.19.2.tar.gz": "636cb2477cec7f8952536970bc533bc43743542f70392ae026374600add5b887", + "https://files.pythonhosted.org/packages/c7/21/705964c7812476f378728bdf590ca4b771ec72385c533964653c68e86bdc/pygments-2.19.2-py3-none-any.whl": "86540386c03d588bb81d44bc3928634ff26449851e99741617ecb9037ee5ec0b" + }, + "pywin32-ctypes": { + "https://files.pythonhosted.org/packages/85/9f/01a1a99704853cb63f253eea009390c88e7131c67e66a0a02099a8c917cb/pywin32-ctypes-0.2.3.tar.gz": "d162dc04946d704503b2edc4d55f3dba5c1d539ead017afa00142c38b9885755", + "https://files.pythonhosted.org/packages/de/3d/8161f7711c017e01ac9f008dfddd9410dff3674334c233bde66e7ba65bbf/pywin32_ctypes-0.2.3-py3-none-any.whl": "8a1513379d709975552d202d942d9837758905c8d01eb82b8bcc30918929e7b8" + }, + "readme-renderer": { + "https://files.pythonhosted.org/packages/5a/a9/104ec9234c8448c4379768221ea6df01260cd6c2ce13182d4eac531c8342/readme_renderer-44.0.tar.gz": "8712034eabbfa6805cacf1402b4eeb2a73028f72d1166d6f5cb7f9c047c5d1e1", + "https://files.pythonhosted.org/packages/e1/67/921ec3024056483db83953ae8e48079ad62b92db7880013ca77632921dd0/readme_renderer-44.0-py3-none-any.whl": "2fbca89b81a08526aadf1357a8c2ae889ec05fb03f5da67f9769c9a592166151" + }, + "requests": { + "https://files.pythonhosted.org/packages/34/64/8860370b167a9721e8956ae116825caff829224fbca0ca6e7bf8ddef8430/requests-2.33.0.tar.gz": "c7ebc5e8b0f21837386ad0e1c8fe8b829fa5f544d8df3b2253bff14ef29d7652", + "https://files.pythonhosted.org/packages/56/5d/c814546c2333ceea4ba42262d8c4d55763003e767fa169adc693bd524478/requests-2.33.0-py3-none-any.whl": "3324635456fa185245e24865e810cecec7b4caf933d7eb133dcde67d48cee69b" + }, + "requests-toolbelt": { + "https://files.pythonhosted.org/packages/3f/51/d4db610ef29373b879047326cbf6fa98b6c1969d6f6dc423279de2b1be2c/requests_toolbelt-1.0.0-py2.py3-none-any.whl": "cccfdd665f0a24fcf4726e690f65639d272bb0637b9b92dfd91a5568ccf6bd06", + "https://files.pythonhosted.org/packages/f3/61/d7545dafb7ac2230c70d38d31cbfe4cc64f7144dc41f6e4e4b78ecd9f5bb/requests-toolbelt-1.0.0.tar.gz": "7681a0a3d047012b5bdc0ee37d7f8f07ebe76ab08caeccfc3921ce23c88d5bc6" + }, + "rfc3986": { + "https://files.pythonhosted.org/packages/85/40/1520d68bfa07ab5a6f065a186815fb6610c86fe957bc065754e47f7b0840/rfc3986-2.0.0.tar.gz": "97aacf9dbd4bfd829baad6e6309fa6573aaf1be3f6fa735c8ab05e46cecb261c", + "https://files.pythonhosted.org/packages/ff/9a/9afaade874b2fa6c752c36f1548f718b5b83af81ed9b76628329dab81c1b/rfc3986-2.0.0-py2.py3-none-any.whl": "50b1502b60e289cb37883f3dfd34532b8873c7de9f49bb546641ce9cbd256ebd" + }, + "rich": { + "https://files.pythonhosted.org/packages/e3/30/3c4d035596d3cf444529e0b2953ad0466f6049528a879d27534700580395/rich-14.1.0-py3-none-any.whl": "536f5f1785986d6dbdea3c75205c473f970777b4a0d6c6dd1b696aa05a3fa04f", + "https://files.pythonhosted.org/packages/fe/75/af448d8e52bf1d8fa6a9d089ca6c07ff4453d86c65c145d0a300bb073b9b/rich-14.1.0.tar.gz": "e497a48b844b0320d45007cdebfeaeed8db2a4f4bcf49f15e455cfc4af11eaa8" + }, + "secretstorage": { + "https://files.pythonhosted.org/packages/53/a4/f48c9d79cb507ed1373477dbceaba7401fd8a23af63b837fa61f1dcd3691/SecretStorage-3.3.3.tar.gz": "2403533ef369eca6d2ba81718576c5e0f564d5cca1b58f73a8b23e7d4eeebd77", + "https://files.pythonhosted.org/packages/54/24/b4293291fa1dd830f353d2cb163295742fa87f179fcc8a20a306a81978b7/SecretStorage-3.3.3-py3-none-any.whl": "f356e6628222568e3af06f2eba8df495efa13b3b63081dafd4f7d9a7b7bc9f99" + }, + "six": { + "https://files.pythonhosted.org/packages/94/e7/b2c673351809dca68a0e064b6af791aa332cf192da575fd474ed7d6f16a2/six-1.17.0.tar.gz": "ff70335d468e7eb6ec65b95b99d3a2836546063f63acc5171de367e834932a81", + "https://files.pythonhosted.org/packages/b7/ce/149a00dd41f10bc29e5921b496af8b574d8413afcd5e30dfa0ed46c2cc5e/six-1.17.0-py2.py3-none-any.whl": "4721f391ed90541fddacab5acf947aa0d3dc7d27b2e1e8eda2be8970586c3274" + }, + "twine": { + "https://files.pythonhosted.org/packages/5d/ec/00f9d5fd040ae29867355e559a94e9a8429225a0284a3f5f091a3878bfc0/twine-5.1.1-py3-none-any.whl": "215dbe7b4b94c2c50a7315c0275d2258399280fbb7d04182c7e55e24b5f93997", + "https://files.pythonhosted.org/packages/77/68/bd982e5e949ef8334e6f7dcf76ae40922a8750aa2e347291ae1477a4782b/twine-5.1.1.tar.gz": "9aa0825139c02b3434d913545c7b847a21c835e11597f5255842d457da2322db" + }, + "urllib3": { + "https://files.pythonhosted.org/packages/39/08/aaaad47bc4e9dc8c725e68f9d04865dbcb2052843ff09c97b08904852d84/urllib3-2.6.3-py3-none-any.whl": "bf272323e553dfb2e87d9bfd225ca7b0f467b919d7bbd355436d3fd37cb0acd4", + "https://files.pythonhosted.org/packages/c7/24/5f1b3bdffd70275f6661c76461e25f024d5a38a46f04aaca912426a2b1d3/urllib3-2.6.3.tar.gz": "1b62b6884944a57dbe321509ab94fd4d3b307075e0c2eae991ac71ee15ad38ed" + }, + "zipp": { + "https://files.pythonhosted.org/packages/2e/54/647ade08bf0db230bfea292f893923872fd20be6ac6f53b2b936ba839d75/zipp-3.23.0-py3-none-any.whl": "071652d6115ed432f5ce1d34c336c0adfd6a884660d1e9712a256d3d3bd4b14e", + "https://files.pythonhosted.org/packages/e3/02/0f2892c661036d50ede074e376733dca2ae7c6eb617489437771209d4180/zipp-3.23.0.tar.gz": "a07157588a12518c9d4034df3fbbee09c814741a33ff63c05fa29d26a2404166" + } + } + }, + "fact_version": "v1", + "index_urls": { + "https://pypi.org/simple/": { + "backports_tarfile": "/simple/backports-tarfile/", + "certifi": "/simple/certifi/", + "cffi": "/simple/cffi/", + "charset_normalizer": "/simple/charset-normalizer/", + "cryptography": "/simple/cryptography/", + "docutils": "/simple/docutils/", + "idna": "/simple/idna/", + "importlib_metadata": "/simple/importlib-metadata/", + "jaraco_classes": "/simple/jaraco-classes/", + "jaraco_context": "/simple/jaraco-context/", + "jaraco_functools": "/simple/jaraco-functools/", + "jeepney": "/simple/jeepney/", + "keyring": "/simple/keyring/", + "markdown_it_py": "/simple/markdown-it-py/", + "mdurl": "/simple/mdurl/", + "more_itertools": "/simple/more-itertools/", + "nh3": "/simple/nh3/", + "pkginfo": "/simple/pkginfo/", + "pycparser": "/simple/pycparser/", + "pygments": "/simple/pygments/", + "pywin32_ctypes": "/simple/pywin32-ctypes/", + "readme_renderer": "/simple/readme-renderer/", + "requests": "/simple/requests/", + "requests_toolbelt": "/simple/requests-toolbelt/", + "rfc3986": "/simple/rfc3986/", + "rich": "/simple/rich/", + "secretstorage": "/simple/secretstorage/", + "six": "/simple/six/", + "twine": "/simple/twine/", + "urllib3": "/simple/urllib3/", + "zipp": "/simple/zipp/" + } + } + } + } +} diff --git a/tests/integration/bzlmod_lockfile/README.md b/tests/integration/bzlmod_lockfile/README.md new file mode 100644 index 0000000000..42c941c4dc --- /dev/null +++ b/tests/integration/bzlmod_lockfile/README.md @@ -0,0 +1,6 @@ +# Pip parse isolation and lock file test + +Update the lock file with the following command: +``` +bazel mod deps --lockfile_mode=update +``` diff --git a/tests/integration/bzlmod_lockfile/WORKSPACE b/tests/integration/bzlmod_lockfile/WORKSPACE new file mode 100644 index 0000000000..e69de29bb2 diff --git a/tests/integration/bzlmod_lockfile/requirements_lock.txt b/tests/integration/bzlmod_lockfile/requirements_lock.txt new file mode 100644 index 0000000000..d8dcf2d44e --- /dev/null +++ b/tests/integration/bzlmod_lockfile/requirements_lock.txt @@ -0,0 +1,8 @@ +six==1.17.0 \ + --hash=sha256:4721f391ed90541fddacab5acf947aa0d3dc7d27b2e1e8eda2be8970586c3274 +# rules_python v2.0.0's own tools/publish/requirements_linux.txt pins pycparser==2.23. +# On macOS: lock gets pycparser-3.0 only (darwin publish requirements have no pycparser). +# On Linux: rules_python_publish_deps hub adds pycparser-2.23 to computed facts → mismatch. +pycparser==3.0 \ + --hash=sha256:600f49d217304a5902ac3c37e1281c9fe94e4d0489de643a9504c5cdfdfc6b29 \ + --hash=sha256:b727414169a36b7d524c1c3e31839a521725078d7b2ff038656844266160a992 diff --git a/tests/integration/bzlmod_lockfile/test_dummy.py b/tests/integration/bzlmod_lockfile/test_dummy.py new file mode 100644 index 0000000000..110b1ba42f --- /dev/null +++ b/tests/integration/bzlmod_lockfile/test_dummy.py @@ -0,0 +1,17 @@ +""" +Verify that a dependency added using the pip extension can be imported. +See MODULE.bazel. +""" + +import unittest + +import six + + +class TestDummy(unittest.TestCase): + def test_import(self): + self.assertTrue(hasattr(six, "PY3")) + + +if __name__ == "__main__": + unittest.main() diff --git a/tests/pypi/parse_requirements/parse_requirements_tests.bzl b/tests/pypi/parse_requirements/parse_requirements_tests.bzl index 4ed1870309..230fcafa0c 100644 --- a/tests/pypi/parse_requirements/parse_requirements_tests.bzl +++ b/tests/pypi/parse_requirements/parse_requirements_tests.bzl @@ -76,6 +76,8 @@ foo==0.0.1; python_full_version < '3.10.0' \ --hash=sha256:deadbeef foo==0.0.2; python_full_version >= '3.10.0' \ --hash=sha256:deadb11f +boo==0.0.4; python_full_version < '3.10.0' \ + --hash=sha256:deadbaaf """, "requirements_optional_hash": """ bar==0.0.4 @ https://example.org/bar-0.0.4.whl @@ -119,6 +121,7 @@ def parse_requirements(debug = False, **kwargs): ) def _test_simple(env): + """Test basic parsing of a single ``requirements_lock`` file.""" got = parse_requirements( requirements_by_platform = { "requirements_lock": ["linux_x86_64", "windows_x86_64"], @@ -234,6 +237,7 @@ def _test_direct_urls_no_extract(env): _tests.append(_test_direct_urls_no_extract) def _test_extra_pip_args(env): + """Test that ``extra_pip_args`` are merged with per-requirement-file args.""" got = parse_requirements( requirements_by_platform = { "requirements_extra_args": ["linux_x86_64"], @@ -266,6 +270,7 @@ def _test_extra_pip_args(env): _tests.append(_test_extra_pip_args) def _test_dupe_requirements(env): + """Test that duplicate requirement entries are deduplicated.""" got = parse_requirements( requirements_by_platform = { "requirements_lock_dupe": ["linux_x86_64"], @@ -295,6 +300,7 @@ def _test_dupe_requirements(env): _tests.append(_test_dupe_requirements) def _test_multi_os(env): + """Test per-OS requirements parsing with ``select_requirement``.""" got = parse_requirements( requirements_by_platform = { "requirements_linux": ["linux_x86_64"], @@ -360,6 +366,7 @@ def _test_multi_os(env): _tests.append(_test_multi_os) def _test_multi_os_legacy(env): + """Test download-only per-OS requirements parsing.""" got = parse_requirements( requirements_by_platform = { "requirements_linux_download_only": ["cp39_linux_x86_64"], @@ -419,6 +426,7 @@ def _test_multi_os_legacy(env): _tests.append(_test_multi_os_legacy) def _test_select_requirement_none_platform(env): + """Test that ``select_requirement`` returns the first src when platform is ``None``.""" got = select_requirement( [ struct( @@ -433,6 +441,8 @@ def _test_select_requirement_none_platform(env): _tests.append(_test_select_requirement_none_platform) def _test_env_marker_resolution(env): + """Test environment marker resolution with ``evaluate_markers``.""" + def _mock_eval_markers(input): ret = { "foo[extra]==0.0.1 ;marker --hash=sha256:deadbeef": ["cp311_windows_x86_64"], @@ -490,6 +500,7 @@ def _test_env_marker_resolution(env): _tests.append(_test_env_marker_resolution) def _test_different_package_version(env): + """Test that different package versions across platforms are handled.""" got = parse_requirements( requirements_by_platform = { "requirements_foo": ["linux_aarch64"], @@ -530,6 +541,7 @@ def _test_different_package_version(env): _tests.append(_test_different_package_version) def _test_different_package_extras(env): + """Test that different extras across platforms are handled.""" got = parse_requirements( requirements_by_platform = { "requirements_foo": ["linux_aarch64"], @@ -570,6 +582,7 @@ def _test_different_package_extras(env): _tests.append(_test_different_package_extras) def _test_optional_hash(env): + """Test parsing of requirements with optional hashes and URLs.""" got = parse_requirements( requirements_by_platform = { "requirements_optional_hash": ["linux_x86_64"], @@ -617,6 +630,7 @@ def _test_optional_hash(env): _tests.append(_test_optional_hash) def _test_git_sources(env): + """Test parsing of git-sourced requirements.""" got = parse_requirements( requirements_by_platform = { "requirements_git": ["linux_x86_64"], @@ -646,6 +660,7 @@ def _test_git_sources(env): _tests.append(_test_git_sources) def _test_overlapping_shas_with_index_results(env): + """Test that index results with overlapping shas are matched to the correct platform.""" got = parse_requirements( requirements_by_platform = { "requirements_linux": ["cp39_linux_x86_64"], @@ -734,6 +749,7 @@ def _test_overlapping_shas_with_index_results(env): _tests.append(_test_overlapping_shas_with_index_results) def _test_get_index_urls_different_versions(env): + """Test that different versions from index URLs are matched correctly per platform.""" got = parse_requirements( requirements_by_platform = { "requirements_multi_version": [ @@ -795,6 +811,24 @@ def _test_get_index_urls_different_versions(env): ) env.expect.that_collection(got).contains_exactly([ + struct( + name = "boo", + index_url = "", + is_exposed = False, + is_multiple_versions = False, + srcs = [ + struct( + distribution = "boo", + extra_pip_args = [], + filename = "", + requirement_line = "boo==0.0.4 --hash=sha256:deadbaaf", + sha256 = "", + target_platforms = ["cp39_linux_x86_64"], + url = "", + yanked = None, + ), + ], + ), struct( name = "foo", index_url = "", @@ -827,7 +861,61 @@ def _test_get_index_urls_different_versions(env): _tests.append(_test_get_index_urls_different_versions) +def _test_get_index_urls_cross_platform(env): + """Verifies that distributions from all requirement files are passed to ``get_index_urls``. + + This ensures the lockfile facts are platform-independent. + """ + calls = [] + + def _get_index_urls(_, distributions, **__): + calls.append({k: list(v) for k, v in distributions.items()}) + return {} + + parse_requirements( + requirements_by_platform = { + "requirements_osx": ["cp39_osx_x86_64"], + # requirements_windows has no matching platforms (simulating + # a macOS build where windows-specific files aren't used). + "requirements_windows": [], + }, + platforms = { + "cp39_osx_x86_64": struct( + env = pep508_env( + python_version = "3.9.0", + os = "osx", + arch = "x86_64", + ), + whl_abi_tags = ["none"], + whl_platform_tags = ["macosx_*_x86_64"], + ), + }, + get_index_urls = _get_index_urls, + evaluate_markers = lambda requirements: evaluate_markers( + requirements = requirements, + platforms = { + "cp39_osx_x86_64": struct( + env = {"python_full_version": "3.9.0"}, + ), + }, + ), + ) + + # distributions must include packages from ALL files, even those with + # no matching platforms: + # - foo: 0.0.2 from requirements_windows, 0.0.3 from requirements_osx + # - bar: 0.0.1 from requirements_windows only + env.expect.that_collection(calls).contains_exactly([ + { + "bar": ["0.0.1"], + "foo": ["0.0.2", "0.0.3"], + }, + ]) + +_tests.append(_test_get_index_urls_cross_platform) + def _test_get_index_urls_single_py_version(env): + """Test index URL matching when only a single Python version is used.""" got = parse_requirements( requirements_by_platform = { "requirements_multi_version": [ @@ -892,6 +980,52 @@ def _test_get_index_urls_single_py_version(env): _tests.append(_test_get_index_urls_single_py_version) +def _test_get_index_urls_all_versions(env): + """Test that all versions from all requirement files are passed to ``get_index_urls``.""" + calls = [] + + def _get_index_urls(_, distributions, **__): + calls.append({k: list(v) for k, v in distributions.items()}) + return {} + + parse_requirements( + requirements_by_platform = { + "requirements_multi_version": ["cp39_linux_x86_64"], + }, + platforms = { + "cp39_linux_x86_64": struct( + env = pep508_env( + python_version = "3.9.0", + os = "linux", + arch = "x86_64", + ), + whl_abi_tags = ["none"], + whl_platform_tags = ["any"], + ), + }, + get_index_urls = _get_index_urls, + evaluate_markers = lambda requirements: evaluate_markers( + requirements = requirements, + platforms = { + "cp39_linux_x86_64": struct( + env = {"python_full_version": "3.9.0"}, + ), + }, + ), + ) + + env.expect.that_collection(calls).contains_exactly([ + { + # boo should be also passed even though it is present on one platform. + "boo": ["0.0.4"], + # Both versions 0.0.1 and 0.0.2 should be passed to get_index_urls, even + # though only 0.0.1 matches the cp39_linux_x86_64 platform markers. + "foo": ["0.0.1", "0.0.2"], + }, + ]) + +_tests.append(_test_get_index_urls_all_versions) + def parse_requirements_test_suite(name): """Create the test suite. diff --git a/tests/pypi/pypi_cache/pypi_cache_tests.bzl b/tests/pypi/pypi_cache/pypi_cache_tests.bzl index 89ed5693e2..14c12ae6d2 100644 --- a/tests/pypi/pypi_cache/pypi_cache_tests.bzl +++ b/tests/pypi/pypi_cache/pypi_cache_tests.bzl @@ -269,6 +269,73 @@ def _test_pypi_cache_reads_from_facts(env): _tests.append(_test_pypi_cache_reads_from_facts) +def _test_pypi_cache_reads_from_facts_drops_unaccessed_dists(env): + """Verifies that dists for unaccessed versions are dropped from computed facts.""" + mock_ctx = mocks.mctx(facts = { + "dist_hashes": { + "https://{PYPI_INDEX_URL}": { + "pkg": { + "https://pypi.org/files/pkg-1.0.0-py3-none-any.whl": "sha_whl_1.0.0", + "https://pypi.org/files/pkg-1.0.0.tar.gz": "sha_sdist_1.0.0", + "https://pypi.org/files/pkg-1.1.0-py3-none-any.whl": "sha_whl_1.1.0", + "https://pypi.org/files/pkg-1.1.0.tar.gz": "sha_sdist_1.1.0", + }, + }, + }, + "fact_version": "v1", + }) + cache = _cache(env, mctx = mock_ctx) + + # Request only version 1.0.0; version 1.1.0 is NOT requested + key = ("https://{PYPI_INDEX_URL}/pkg/", "https://pypi.org/simple/pkg/", ["1.0.0"]) + got = cache.get(key) + + expected = struct( + sdists = { + "sha_sdist_1.0.0": struct( + sha256 = "sha_sdist_1.0.0", + version = "1.0.0", + filename = "pkg-1.0.0.tar.gz", + metadata_url = "", + metadata_sha256 = "", + url = "https://pypi.org/files/pkg-1.0.0.tar.gz", + yanked = None, + ), + }, + whls = { + "sha_whl_1.0.0": struct( + sha256 = "sha_whl_1.0.0", + version = "1.0.0", + filename = "pkg-1.0.0-py3-none-any.whl", + metadata_url = "", + metadata_sha256 = "", + url = "https://pypi.org/files/pkg-1.0.0-py3-none-any.whl", + yanked = None, + ), + }, + sha256s_by_version = { + "1.0.0": ["sha_sdist_1.0.0", "sha_whl_1.0.0"], + }, + ) + got.whls().contains_exactly(expected.whls) + got.sdists().contains_exactly(expected.sdists) + got.sha256s_by_version().contains_exactly(expected.sha256s_by_version) + + # get_facts() must only contain version 1.0.0 data; 1.1.0 is dropped + cache.get_facts().contains_exactly({ + "dist_hashes": { + "https://{PYPI_INDEX_URL}": { + "pkg": { + "https://pypi.org/files/pkg-1.0.0-py3-none-any.whl": "sha_whl_1.0.0", + "https://pypi.org/files/pkg-1.0.0.tar.gz": "sha_sdist_1.0.0", + }, + }, + }, + "fact_version": "v1", + }) + +_tests.append(_test_pypi_cache_reads_from_facts_drops_unaccessed_dists) + def _test_memory_cache_index_urls(env): """Verifies that the cache returns stored values for index_urls.""" store = {} @@ -364,6 +431,64 @@ def _test_pypi_cache_reads_index_urls_from_facts(env): _tests.append(_test_pypi_cache_reads_index_urls_from_facts) +def _test_pypi_cache_reads_index_urls_from_facts_incomplete(env): + """Verifies that incomplete index_urls facts returns None (forces fresh download).""" + mock_ctx = mocks.mctx(facts = { + "fact_version": "v1", + "index_urls": { + "https://pypi.org/simple/": { + "pkg-a": "https://pypi.org/simple/pkg-a/", + }, + }, + }) + cache = _cache(env, mctx = mock_ctx) + + # Request pkg-a and pkg-b, but facts only have pkg-a + key = ("https://pypi.org/simple/", "https://pypi.org/simple/", {"pkg-a": None, "pkg-b": None}) + cache.get(key).equals(None) + +_tests.append(_test_pypi_cache_reads_index_urls_from_facts_incomplete) + +def _test_pypi_cache_reads_index_urls_from_facts_drops_unaccessed(env): + """Verifies get_facts() drops unaccessed index_urls entries. + + When known_facts has packages that are not requested, they should not + appear in computed_facts. This ensures stale facts (from packages + removed from all requirements files) get cleaned up from the lockfile. + """ + mock_ctx = mocks.mctx(facts = { + "fact_version": "v1", + "index_urls": { + "https://pypi.org/simple/": { + "pkg-a": "https://pypi.org/simple/pkg-a/", + "pkg-b": "https://pypi.org/simple/pkg-b/", + "pkg-c": "https://pypi.org/simple/pkg-c/", + }, + }, + }) + cache = _cache(env, mctx = mock_ctx) + + # Request only a subset of packages; pkg-c is not requested + key = ("https://pypi.org/simple/", "https://pypi.org/simple/", {"pkg-a": None, "pkg-b": None}) + got = cache.get(key) + got.contains_exactly({ + "pkg-a": "https://pypi.org/simple/pkg-a/", + "pkg-b": "https://pypi.org/simple/pkg-b/", + }) + + # get_facts() must only return the requested (accessed) subset; pkg-c is dropped + cache.get_facts().contains_exactly({ + "fact_version": "v1", + "index_urls": { + "https://pypi.org/simple/": { + "pkg-a": "https://pypi.org/simple/pkg-a/", + "pkg-b": "https://pypi.org/simple/pkg-b/", + }, + }, + }) + +_tests.append(_test_pypi_cache_reads_index_urls_from_facts_drops_unaccessed) + def pypi_cache_test_suite(name): test_suite( name = name, diff --git a/tests/pypi/requirements_files_by_platform/requirements_files_by_platform_tests.bzl b/tests/pypi/requirements_files_by_platform/requirements_files_by_platform_tests.bzl index d6aaf3ca99..b1176e6a15 100644 --- a/tests/pypi/requirements_files_by_platform/requirements_files_by_platform_tests.bzl +++ b/tests/pypi/requirements_files_by_platform/requirements_files_by_platform_tests.bzl @@ -37,6 +37,7 @@ requirements_files_by_platform = lambda **kwargs: _sut( ) def _test_fail_no_requirements(env): + """Verify that omitting all requirements attributes produces an error.""" errors = [] requirements_files_by_platform( fail_fn = errors.append, @@ -47,6 +48,7 @@ A 'requirements_lock' attribute must be specified, a platform-specific lockfiles _tests.append(_test_fail_no_requirements) def _test_fail_duplicate_platforms(env): + """Verify that a platform mapped to multiple requirements files errors.""" errors = [] requirements_files_by_platform( requirements_by_platform = { @@ -61,6 +63,7 @@ def _test_fail_duplicate_platforms(env): _tests.append(_test_fail_duplicate_platforms) def _test_fail_download_only_bad_attr(env): + """Verify that ``--platform`` pip args require a single ``requirements_lock``.""" errors = [] requirements_files_by_platform( requirements_linux = "requirements_linux", @@ -78,6 +81,7 @@ def _test_fail_download_only_bad_attr(env): _tests.append(_test_fail_download_only_bad_attr) def _test_simple(env): + """Test basic mapping of a single ``requirements_lock`` to all platforms.""" for got in [ requirements_files_by_platform( requirements_lock = "requirements_lock", @@ -104,6 +108,7 @@ def _test_simple(env): _tests.append(_test_simple) def _test_simple_limited(env): + """Test that limiting the platform list restricts the output mapping.""" for got in [ requirements_files_by_platform( requirements_lock = "requirements_lock", @@ -132,6 +137,7 @@ def _test_simple_limited(env): _tests.append(_test_simple_limited) def _test_simple_with_python_version(env): + """Test that ``python_version`` prefixes platform names with ``cpNNN_``.""" for got in [ requirements_files_by_platform( requirements_lock = "requirements_lock", @@ -169,6 +175,7 @@ def _test_simple_with_python_version(env): _tests.append(_test_simple_with_python_version) def _test_multi_os(env): + """Test per-OS requirements files mapping each OS group correctly.""" for got in [ requirements_files_by_platform( requirements_linux = "requirements_linux", @@ -203,6 +210,7 @@ def _test_multi_os(env): _tests.append(_test_multi_os) def _test_multi_os_download_only_platform(env): + """Test that ``--platform`` pip args narrow platforms to the host OS.""" got = requirements_files_by_platform( requirements_lock = "requirements_linux", extra_pip_args = [ @@ -219,6 +227,7 @@ def _test_multi_os_download_only_platform(env): _tests.append(_test_multi_os_download_only_platform) def _test_os_arch_requirements_with_default(env): + """Test combining specific OS/arch requirements with a fallback ``requirements_lock``.""" got = requirements_files_by_platform( requirements_by_platform = { "requirements_exotic": "linux_super_exotic", @@ -252,6 +261,64 @@ def _test_os_arch_requirements_with_default(env): _tests.append(_test_os_arch_requirements_with_default) +def _test_host_only_lockfile(env): + """Host-only: single requirements_lock with only the host platform. + + Verifies no extra empty-platform files leak into the return dict. + """ + got = requirements_files_by_platform( + requirements_lock = "requirements_lock", + platforms = ["osx_x86_64"], + ) + env.expect.that_dict(got).contains_exactly({ + "requirements_lock": ["osx_x86_64"], + }) + +_tests.append(_test_host_only_lockfile) + +def _test_host_only_multiple_os(env): + """Host-only with per-OS files but only host platform configured. + + Files with no matching platforms should appear with empty platform + lists so parse_requirements can read all packages for index URLs. + """ + got = requirements_files_by_platform( + requirements_linux = "requirements_linux", + requirements_osx = "requirements_osx", + requirements_windows = "requirements_windows", + platforms = ["osx_x86_64"], + ) + env.expect.that_dict(got).contains_exactly({ + # Per-OS files with no matching platforms get empty lists + "requirements_linux": [], + # The matching OS file gets its platforms + "requirements_osx": ["osx_x86_64"], + "requirements_windows": [], + }) + +_tests.append(_test_host_only_multiple_os) + +def _test_host_only_os_with_fallback(env): + """Host-only with per-OS files + fallback lock, host platform only. + + The fallback should not appear since the matching OS file covers + the only platform; unmatched files get empty lists. + """ + got = requirements_files_by_platform( + requirements_linux = "requirements_linux", + requirements_osx = "requirements_osx", + requirements_lock = "requirements_lock", + platforms = ["osx_x86_64"], + ) + env.expect.that_dict(got).contains_exactly({ + "requirements_linux": [], + "requirements_osx": ["osx_x86_64"], + # Fallback lock is not used because osx file already covers + # the only platform + }) + +_tests.append(_test_host_only_os_with_fallback) + def requirements_files_by_platform_test_suite(name): """Create the test suite. From 5511aaf1e95fbf6b3eeca64ad503b26d712f50aa Mon Sep 17 00:00:00 2001 From: Ignas Anikevicius <240938+aignas@users.noreply.github.com> Date: Sun, 10 May 2026 15:49:29 +0900 Subject: [PATCH 738/922] fix(logger): do not output WARN level logs for non-root modules (#3760) A user has reported issues which actually resulted from a particular `bazel` version being used and once he updated, the issues disappeared because of changes in implicit non-root module dependencies. In order to reduce the warning spam of non-root modules, set the default level to `ERROR` for non-root module parsing context and `WARN` for root module context. This means that the users will not get console output that they have no way to fix without patching upstream dependencies. At the same time ensure that the name of the module is printed together with the logs to better understand where something is coming from. Fixes #3749 --------- Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> --- CHANGELOG.md | 2 ++ python/private/pypi/extension.bzl | 2 +- python/private/python.bzl | 10 ++++++---- python/private/repo_utils.bzl | 21 ++++++++++++++++----- 4 files changed, 25 insertions(+), 10 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 8fb3803a6a..3feaebf428 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -115,6 +115,8 @@ END_UNRELEASED_TEMPLATE * (pypi) Fix the versions of packages that we are recording to a `MODULE.bazel.lock` file facts by passing all of the versions to the `get_index` function. Fixes [#3756](https://github.com/bazel-contrib/rules_python/issues/3756). +* (bzlmod) Reduce default verbosity of our loggers for non-root modules + ([#3749](https://github.com/bazel-contrib/rules_python/issues/3749)). {#v2-0-0} ## [2.0.0] - 2026-04-09 diff --git a/python/private/pypi/extension.bzl b/python/private/pypi/extension.bzl index e6052782fa..2a8ace28f2 100644 --- a/python/private/pypi/extension.bzl +++ b/python/private/pypi/extension.bzl @@ -358,7 +358,7 @@ You cannot use both the additive_build_content and additive_build_content_file a minor_mapping = kwargs.get("minor_mapping", MINOR_MAPPING), evaluate_markers_fn = kwargs.get("evaluate_markers", None), available_interpreters = kwargs.get("available_interpreters", INTERPRETER_LABELS), - logger = repo_utils.logger(module_ctx, "pypi:hub:" + hub_name), + logger = repo_utils.logger(module_ctx, "pypi:hub:" + hub_name, mod = mod), ) pip_hub_map[pip_attr.hub_name] = builder elif pip_hub_map[hub_name].module_name != mod.name: diff --git a/python/private/python.bzl b/python/private/python.bzl index 2ea757892e..0f12f88f0c 100644 --- a/python/private/python.bzl +++ b/python/private/python.bzl @@ -31,7 +31,7 @@ load( ) load(":version.bzl", "version") -def parse_modules(*, module_ctx, logger, _fail = fail): +def parse_modules(*, module_ctx, logger = None, _fail = fail): """Parse the modules and return a struct for registrations. Args: @@ -137,7 +137,7 @@ def parse_modules(*, module_ctx, logger, _fail = fail): first = first, second_toolchain_name = toolchain_name, second_module_name = mod.name, - logger = logger, + logger = logger or repo_utils.logger(module_ctx, "python", mod = mod), ) toolchain_info = None else: @@ -213,8 +213,10 @@ def parse_modules(*, module_ctx, logger, _fail = fail): ) def _python_impl(module_ctx): - logger = repo_utils.logger(module_ctx, "python") - py = parse_modules(module_ctx = module_ctx, logger = logger) + py = parse_modules(module_ctx = module_ctx) + + # For all other processing (after parsing the modules) let's use a single logger. + logger = repo_utils.logger(module_ctx, "python", mod = module_ctx.modules[0]) # Host compatible runtime repos # dict[str version, struct] where struct has: diff --git a/python/private/repo_utils.bzl b/python/private/repo_utils.bzl index ae2fc2e5d0..6fa851b7b3 100644 --- a/python/private/repo_utils.bzl +++ b/python/private/repo_utils.bzl @@ -31,7 +31,7 @@ def _is_repo_debug_enabled(mrctx): """ return mrctx.getenv(REPO_DEBUG_ENV_VAR) == "1" -def _logger(mrctx = None, name = None, verbosity_level = None, printer = None): +def _logger(mrctx = None, name = None, verbosity_level = None, printer = None, mod = None): """Creates a logger instance for printing messages. Args: @@ -42,6 +42,7 @@ def _logger(mrctx = None, name = None, verbosity_level = None, printer = None): taken from `mrctx`. printer: a function to use for printing. Defaults to `print` or `fail` depending on the logging method. + mod: {type}`module_ctx.module`. The module for which the logger is created. Returns: A struct with attributes logging: trace, debug, info, warn, fail. @@ -50,20 +51,30 @@ def _logger(mrctx = None, name = None, verbosity_level = None, printer = None): the logger injected into the function work as expected by terminating on the given line. """ + default_verbosity_level = "WARN" + if mod: + if name: + name = "{}:{}".format(mod.name, name) + else: + name = mod.name + + if not mod.is_root: + default_verbosity_level = "ERROR" # the warnings are non actionable anyway, but we should keep them. + if verbosity_level == None: if _is_repo_debug_enabled(mrctx): - verbosity_level = "DEBUG" - else: - verbosity_level = "WARN" + default_verbosity_level = "DEBUG" env_var_verbosity = mrctx.getenv(REPO_VERBOSITY_ENV_VAR) - verbosity_level = env_var_verbosity or verbosity_level + verbosity_level = env_var_verbosity or default_verbosity_level verbosity = { "DEBUG": 2, + "ERROR": -1, "FAIL": -1, "INFO": 1, "TRACE": 3, + "WARN": 0, }.get(verbosity_level, 0) if hasattr(mrctx, "attr"): From 32527de8e743b4c32b44a18595d05c78e03e593f Mon Sep 17 00:00:00 2001 From: Yun Peng Date: Tue, 19 May 2026 16:12:36 +0200 Subject: [PATCH 739/922] ci: update RBE toolchain version from ubuntu2204 to ubuntu2404 (#3778) RBE has dropped support for older platform Fixing https://buildkite.com/bazel/rules-python-python/builds/15434#019e2c2c-c708-4c20-bc05-a9a5b1215d2f --- .bazelci/presubmit.yml | 4 ++-- MODULE.bazel | 2 +- python/private/internal_dev_deps.bzl | 2 +- 3 files changed, 4 insertions(+), 4 deletions(-) diff --git a/.bazelci/presubmit.yml b/.bazelci/presubmit.yml index 38d223cb8d..6366741313 100644 --- a/.bazelci/presubmit.yml +++ b/.bazelci/presubmit.yml @@ -296,7 +296,7 @@ tasks: <<: *minimum_supported_version <<: *reusable_config name: "RBE: Ubuntu, minimum Bazel" - platform: rbe_ubuntu2204 + platform: rbe_ubuntu2404 build_flags: - "--experimental_repository_cache_hardlinks=false" # BazelCI sets --action_env=BAZEL_DO_NOT_DETECT_CPP_TOOLCHAIN=1, @@ -315,7 +315,7 @@ tasks: rbe: <<: *reusable_config name: "RBE: Ubuntu" - platform: rbe_ubuntu2204 + platform: rbe_ubuntu2404 # TODO @aignas 2024-12-11: get the RBE working in CI for bazel 8.0 # See https://github.com/bazelbuild/rules_python/issues/2499 bazel: 8.x diff --git a/MODULE.bazel b/MODULE.bazel index ae007c4aaf..ab1c41bc09 100644 --- a/MODULE.bazel +++ b/MODULE.bazel @@ -8,7 +8,7 @@ bazel_dep(name = "bazel_features", version = "1.21.0") bazel_dep(name = "bazel_skylib", version = "1.8.2") bazel_dep(name = "package_metadata", version = "0.0.7") bazel_dep(name = "platforms", version = "0.0.11") -bazel_dep(name = "rules_cc", version = "0.1.5") +bazel_dep(name = "rules_cc", version = "0.2.17") # Those are loaded only when using py_proto_library # Use py_proto_library directly from protobuf repository diff --git a/python/private/internal_dev_deps.bzl b/python/private/internal_dev_deps.bzl index 11b020e59f..b0b06c31b2 100644 --- a/python/private/internal_dev_deps.bzl +++ b/python/private/internal_dev_deps.bzl @@ -26,7 +26,7 @@ def _internal_dev_deps_impl(mctx): # otherwise refer to RBE docs. rbe_preconfig( name = "buildkite_config", - toolchain = "ubuntu2204", + toolchain = "ubuntu2404", ) runtime_env_repo(name = "rules_python_runtime_env_tc_info") From dc9c1abd329061246fb9b636916df853f9e4c126 Mon Sep 17 00:00:00 2001 From: Kevin Park Date: Tue, 19 May 2026 20:02:39 -0400 Subject: [PATCH 740/922] docs: split toolchain bumps from #3708 into 2.0.2 and 1.9.1 changelog sections (#3777) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Per [@aignas's comment](https://github.com/bazel-contrib/rules_python/issues/3773#issuecomment-4439327299) on #3773, moves the toolchain bullets that #3708 added under `Unreleased` into dated `2.0.2` and `1.9.1` sections so the next release from `main` doesn't re-announce them. **Before:** Bullets sit under `Unreleased` on `main`. **After:** New `## [2.0.2] - 2026-05-14` section between `Unreleased` and `2.0.1`, and a new `## [1.9.1] - 2026-05-14` section between `2.0.0` and `1.9.0`. Bullet text is reused verbatim. Companion PRs: - #3775 — backport #3708 to `release/2.0` (2.0.2) - #3776 — backport #3708 to `release/1.9` (1.9.1) Refs #3773. --- CHANGELOG.md | 25 ++++++++++++++++++++++--- 1 file changed, 22 insertions(+), 3 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 3feaebf428..7e60176980 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -94,13 +94,21 @@ END_UNRELEASED_TEMPLATE Fixes [#3296](https://github.com/bazel-contrib/rules_python/issues/3296). * (gazelle) Support alias_kind directive. Fixes [#3183](https://github.com/bazel-contrib/rules_python/issues/3183). -* (toolchains) `3.13.12`, `3.14.3` Python toolchain from [20260325] release. -* (toolchains) `3.10.20`, `3.11.15`, `3.12.13`, `3.13.13` `3.14.4`, `3.15.0a8` -* Python toolchain from [20260414] release. * (pypi) `package_metadata` support, fixes [#2054](https://github.com/bazel-contrib/rules_python/issues/2054). * (coverage) Add support for python 3.14 and bump `coverage.py` to 7.10.7. +{#v2-0-2} +## [2.0.2] - 2026-05-14 + +[2.0.2]: https://github.com/bazel-contrib/rules_python/releases/tag/2.0.2 + +{#v2-0-2-added} +### Added +* (toolchains) `3.13.12`, `3.14.3` Python toolchain from [20260325] release. +* (toolchains) `3.10.20`, `3.11.15`, `3.12.13`, `3.13.13` `3.14.4`, `3.15.0a8` +* Python toolchain from [20260414] release. + [20260325]: https://github.com/astral-sh/python-build-standalone/releases/tag/20260325 [20260414]: https://github.com/astral-sh/python-build-standalone/releases/tag/20260414 @@ -230,6 +238,17 @@ Other changes: * (wheel) Add support for `add_path_prefix` argument in `py_wheel` which can be used to prepend a prefix to the files in the wheel. +{#v1-9-1} +## [1.9.1] - 2026-05-14 + +[1.9.1]: https://github.com/bazel-contrib/rules_python/releases/tag/1.9.1 + +{#v1-9-1-added} +### Added +* (toolchains) `3.13.12`, `3.14.3` Python toolchain from [20260325] release. +* (toolchains) `3.10.20`, `3.11.15`, `3.12.13`, `3.13.13` `3.14.4`, `3.15.0a8` +* Python toolchain from [20260414] release. + {#v1-9-0} ## [1.9.0] - 2026-02-21 From 4b99ec3af1f3ea5e3b9b4e8ba9d671617f4d3812 Mon Sep 17 00:00:00 2001 From: Titus Fortner Date: Tue, 19 May 2026 19:36:42 -0500 Subject: [PATCH 741/922] fix(rules): allow path separators in 'main' attribute (#3790) Before we would fail when there are target separators present in the value of the `main` attribute. This PR adds a test and fixes this by declaring the bootstrap output as a sibling of the executable. Fixes #3789 --------- Co-authored-by: Ignas Anikevicius <240938+aignas@users.noreply.github.com> --- CHANGELOG.md | 4 +++ python/private/py_executable.bzl | 2 +- tests/base_rules/py_executable_base_tests.bzl | 29 +++++++++++++++++++ 3 files changed, 34 insertions(+), 1 deletion(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 7e60176980..db173447ae 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -80,6 +80,10 @@ END_UNRELEASED_TEMPLATE * (pypi) Fix `importlib.metadata.files` by ensuring `RECORD` is included in installed wheel targets, except when built from sdist ([#3024](https://github.com/bazel-contrib/rules_python/issues/3024)). +* (windows) Fix `py_test`/`py_binary` failure when the target name contains + path separators; the bootstrap stub is now declared as a sibling of the + `.exe` launcher + ([#3789](https://github.com/bazel-contrib/rules_python/issues/3789)). {#v0-0-0-added} diff --git a/python/private/py_executable.bzl b/python/private/py_executable.bzl index 9c21e5d274..965ed536cc 100644 --- a/python/private/py_executable.bzl +++ b/python/private/py_executable.bzl @@ -424,7 +424,7 @@ WARNING: Target: {} # On Windows, the main executable has an "exe" extension, so # here we re-use the un-extensioned name for the bootstrap output. - bootstrap_output = ctx.actions.declare_file(base_executable_name) + bootstrap_output = ctx.actions.declare_file(base_executable_name, sibling = executable) # The launcher looks for the non-zip executable next to # itself, so add it to the default outputs. diff --git a/tests/base_rules/py_executable_base_tests.bzl b/tests/base_rules/py_executable_base_tests.bzl index d6b5aedf0c..dff0399bc2 100644 --- a/tests/base_rules/py_executable_base_tests.bzl +++ b/tests/base_rules/py_executable_base_tests.bzl @@ -496,6 +496,35 @@ def _test_py_runtime_info_provided_impl(env, target): _tests.append(_test_py_runtime_info_provided) +def _test_windows_target_with_path_separators(name, config): + rt_util.helper_target( + config.rule, + name = name + "/nested_subject", + srcs = ["main.py"], + main = "main.py", + ) + analysis_test( + name = name, + impl = _test_windows_target_with_path_separators_impl, + target = name + "/nested_subject", + config_settings = { + "//command_line_option:cpu": "windows_x86_64", + "//command_line_option:crosstool_top": CROSSTOOL_TOP, + "//command_line_option:extra_execution_platforms": [platform_targets.WINDOWS_X86_64], + "//command_line_option:extra_toolchains": [CC_TOOLCHAIN], + "//command_line_option:platforms": [platform_targets.WINDOWS_X86_64], + }, + attr_values = {}, + ) + +def _test_windows_target_with_path_separators_impl(env, target): + target = env.expect.that_target(target) + target.runfiles().contains_predicate(matching.str_endswith( + target.meta.format_str("/{name}"), + )) + +_tests.append(_test_windows_target_with_path_separators) + # ===== # You were gonna add a test at the end, weren't you? # Nope. Please keep them sorted; put it in its alphabetical location. From f4ebb5b67f1b1135fd48a6dbbcb268c96ce5c2d8 Mon Sep 17 00:00:00 2001 From: Joshua Yanchar Date: Tue, 19 May 2026 23:14:23 -0700 Subject: [PATCH 742/922] feat(coverage): warn when bundled coverage tool has no wheel for requested python_version/platform (#3766) Addresses review feedback on #3764: ~~NOTE: Starlark unit tests have no way to capture stdout/stderr, so we can't directly assert that the warning is printed. I am not sure if the refactoring-for-testability I have considered would be appreciated, so this PR is a minimal approach. (I will also prepare an alternative PR that includes a small refactor and some tests, in case that is preferred.)~~ A logger has been added, and is used in testing. Previously, when `configure_coverage_tool = True` was set but the bundled `coverage.py` wheel set had no entry for the requested (python_version, platform), `coverage_dep` returned None silently. The result was that `bazel coverage` produced empty per-test lcov files for `py_test` targets with no signal to the user that coverage was unconfigured. Print a WARNING in that path so the misconfiguration is visible. Preserves the existing silent return for the windows branch, which is intentionally quiet because the upstream coverage wrapper does not support windows. --- CHANGELOG.md | 4 + python/private/BUILD.bazel | 2 + python/private/coverage_deps.bzl | 21 +++- python/private/python.bzl | 1 + python/private/python_register_toolchains.bzl | 15 +++ tests/coverage_deps/BUILD.bazel | 17 ++++ tests/coverage_deps/coverage_deps_test.bzl | 95 +++++++++++++++++++ 7 files changed, 153 insertions(+), 2 deletions(-) create mode 100644 tests/coverage_deps/BUILD.bazel create mode 100644 tests/coverage_deps/coverage_deps_test.bzl diff --git a/CHANGELOG.md b/CHANGELOG.md index db173447ae..df0602785a 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -65,6 +65,10 @@ END_UNRELEASED_TEMPLATE default to `true`. * (pypi) The data files of a wheel (bin, includes, etc) are now always included as a library's data dependencies. +* (coverage) When `configure_coverage_tool = True` is set but the bundled + `coverage.py` wheel set has no entry for the requested python version and + platform, a warning is now printed instead of silently producing an empty + coverage report. {#v0-0-0-fixed} ### Fixed diff --git a/python/private/BUILD.bazel b/python/private/BUILD.bazel index db96957724..6ab3d546be 100644 --- a/python/private/BUILD.bazel +++ b/python/private/BUILD.bazel @@ -175,6 +175,7 @@ bzl_library( srcs = ["coverage_deps.bzl"], deps = [ ":bazel_tools_bzl", + ":repo_utils_bzl", ":version_label_bzl", ], ) @@ -293,6 +294,7 @@ bzl_library( ":full_version_bzl", ":internal_config_repo_bzl", ":python_repository_bzl", + ":repo_utils_bzl", ":toolchains_repo_bzl", "//python:versions_bzl", "//python/private/pypi:deps_bzl", diff --git a/python/private/coverage_deps.bzl b/python/private/coverage_deps.bzl index cd813196b5..a32b2e3f97 100644 --- a/python/private/coverage_deps.bzl +++ b/python/private/coverage_deps.bzl @@ -17,6 +17,7 @@ load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive") load("@bazel_tools//tools/build_defs/repo:utils.bzl", "maybe") +load("//python/private:repo_utils.bzl", "repo_utils") load("//python/private:version_label.bzl", "version_label") # START: maintained by 'bazel run //tools/private/update_deps:update_coverage_deps ' @@ -166,7 +167,7 @@ _coverage_deps = { _coverage_patch = Label("//python/private:coverage.patch") -def coverage_dep(name, python_version, platform, visibility): +def coverage_dep(name, python_version, platform, visibility, logger = None): """Register a single coverage dependency based on the python version and platform. Args: @@ -174,10 +175,19 @@ def coverage_dep(name, python_version, platform, visibility): python_version: The full python version. platform: The platform, which can be found in //python:versions.bzl PLATFORMS dict. visibility: The visibility of the coverage tool. + logger: {type}`repo_utils.logger | None` Optional logger used to emit a + warning when no wheel is available for the (python_version, + platform) pair. If not supplied, a default logger is constructed. Returns: The label of the coverage tool if the platform is supported, otherwise - None. """ + if logger == None: + logger = repo_utils.logger( + struct(getenv = lambda _: None), + name = "coverage_dep", + ) + if "windows" in platform: # NOTE @aignas 2023-01-19: currently we do not support windows as the # upstream coverage wrapper is written in shell. Do not log any warning @@ -188,7 +198,14 @@ def coverage_dep(name, python_version, platform, visibility): url, sha256 = _coverage_deps.get(abi, {}).get(platform, (None, "")) if url == None: - # Some wheels are not present for some builds, so let's silently ignore those. + logger.warn(lambda: ( + "rules_python's bundled coverage tool has no wheel for " + + "python_version={}, platform={}. `bazel coverage` will produce " + + "empty lcov for py_test targets in this configuration. Either " + + "pin python_version to a version in the bundled set (see " + + "python/private/coverage_deps.bzl), or configure coverage " + + "manually via py_runtime.coverage_tool. See docs/coverage.md." + ).format(python_version, platform)) return None maybe( diff --git a/python/private/python.bzl b/python/private/python.bzl index 0f12f88f0c..6abc81e3d2 100644 --- a/python/private/python.bzl +++ b/python/private/python.bzl @@ -275,6 +275,7 @@ def _python_impl(module_ctx): register_result = python_register_toolchains( name = toolchain_info.name, _internal_bzlmod_toolchain_call = True, + _internal_module_ctx = module_ctx, **kwargs ) if not register_result.impl_repos: diff --git a/python/private/python_register_toolchains.bzl b/python/private/python_register_toolchains.bzl index 9e75c41978..3b92902c7e 100644 --- a/python/private/python_register_toolchains.bzl +++ b/python/private/python_register_toolchains.bzl @@ -26,6 +26,7 @@ load( load(":coverage_deps.bzl", "coverage_dep") load(":full_version.bzl", "full_version") load(":python_repository.bzl", "python_repository") +load(":repo_utils.bzl", "repo_utils") load( ":toolchains_repo.bzl", "host_compatible_python_repo", @@ -89,6 +90,19 @@ def python_register_toolchains( if bzlmod_toolchain_call: register_toolchains = False + # When invoked from the bzlmod python extension, a module_ctx is plumbed in + # so the coverage_dep logger can attribute warnings to the right module and + # honor module-root filtering. In the WORKSPACE/macro path no module_ctx is + # available; a minimal stand-in struct gives the logger what it needs. + module_ctx = kwargs.pop("_internal_module_ctx", None) + if module_ctx != None: + coverage_logger = repo_utils.logger(module_ctx, name = "coverage_dep") + else: + coverage_logger = repo_utils.logger( + struct(getenv = lambda _: None), + name = "coverage_dep", + ) + base_url = kwargs.pop("base_url", DEFAULT_RELEASE_BASE_URL) tool_versions = tool_versions or TOOL_VERSIONS minor_mapping = minor_mapping or MINOR_MAPPING @@ -121,6 +135,7 @@ def python_register_toolchains( ), python_version = python_version, platform = platform, + logger = coverage_logger, visibility = ["@{name}_{platform}//:__subpackages__".format( name = name, platform = platform, diff --git a/tests/coverage_deps/BUILD.bazel b/tests/coverage_deps/BUILD.bazel new file mode 100644 index 0000000000..8ec6025902 --- /dev/null +++ b/tests/coverage_deps/BUILD.bazel @@ -0,0 +1,17 @@ +# Copyright 2026 The Bazel Authors. All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +load(":coverage_deps_test.bzl", "coverage_deps_test_suite") + +coverage_deps_test_suite(name = "coverage_deps_tests") diff --git a/tests/coverage_deps/coverage_deps_test.bzl b/tests/coverage_deps/coverage_deps_test.bzl new file mode 100644 index 0000000000..12351affde --- /dev/null +++ b/tests/coverage_deps/coverage_deps_test.bzl @@ -0,0 +1,95 @@ +# Copyright 2026 The Bazel Authors. All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +"Tests for the warning emitted by coverage_dep when no wheel is available." + +load("@rules_testing//lib:test_suite.bzl", "test_suite") +load("//python/private:coverage_deps.bzl", "coverage_dep") # buildifier: disable=bzl-visibility +load("//python/private:repo_utils.bzl", "REPO_DEBUG_ENV_VAR", "REPO_VERBOSITY_ENV_VAR", "repo_utils") # buildifier: disable=bzl-visibility + +_tests = [] + +def _capturing_logger(): + """Build a (logger, captured_messages_list) pair. + + The logger has its verbosity set to INFO so WARN messages are captured but + nothing noisier than necessary is emitted. The printer collects the second + positional argument from each printer invocation (the formatted message). + """ + captured = [] + logger = repo_utils.logger( + struct( + getenv = { + REPO_DEBUG_ENV_VAR: None, + REPO_VERBOSITY_ENV_VAR: "INFO", + }.get, + ), + name = "unit-test", + printer = lambda _key, message: captured.append(message), + ) + return logger, captured + +def _test_unsupported_python_version_warns(env): + # cp37 is not in the bundled wheel set; coverage_dep should return None + # and emit a warning describing the misconfiguration. + logger, captured = _capturing_logger() + result = coverage_dep( + name = "unused_for_test", + python_version = "3.7", + platform = "aarch64-apple-darwin", + visibility = ["//visibility:public"], + logger = logger, + ) + env.expect.that_bool(result == None).equals(True) + env.expect.that_int(len(captured)).equals(1) + env.expect.that_str(captured[0]).contains("no wheel for") + env.expect.that_str(captured[0]).contains("python_version=3.7") + env.expect.that_str(captured[0]).contains("platform=aarch64-apple-darwin") + +_tests.append(_test_unsupported_python_version_warns) + +def _test_windows_platform_is_silent(env): + # Windows is intentionally unsupported and not actionable; coverage_dep + # must return None without logging anything. + logger, captured = _capturing_logger() + result = coverage_dep( + name = "unused_for_test", + python_version = "3.10", + platform = "x86_64-pc-windows-msvc", + visibility = ["//visibility:public"], + logger = logger, + ) + env.expect.that_bool(result == None).equals(True) + env.expect.that_int(len(captured)).equals(0) + +_tests.append(_test_windows_platform_is_silent) + +# NOTE: there is intentionally no unit test for the supported-wheel path +# (where coverage_dep returns a non-None label and emits no warning). +# That path calls `maybe(http_archive, ...)`, which calls +# `native.existing_rule()`. `native.existing_rule()` is only valid during +# BUILD file, legacy macro, or rule finalizer evaluation -- not during +# rule analysis, which is the phase rules_testing analysis tests run in. +# Calling coverage_dep with supported args from here therefore fails with +# "existing_rule() can only be used while evaluating a BUILD file, ...". +# The supported-wheel path is exercised end-to-end by `bazel coverage` +# against a real py_test target during ordinary use of the toolchain. + +def coverage_deps_test_suite(name): + """Create the test suite. + + Args: + name: the name of the test suite. + """ + test_suite(name = name, basic_tests = _tests) From 61cfa883cf4c8b6e3022dd75bcfc31d3c38bcf6e Mon Sep 17 00:00:00 2001 From: Jan Schlosser Date: Wed, 20 May 2026 08:26:12 +0200 Subject: [PATCH 743/922] fix(pip_compile): Forward target compatibility to *.update (#3787) With this changset we enable users to build and test `//...` in their repositories also with platforms that do not support Python. For example, a user might use Python for tooling and automation, where he develops only on Linux. Then he could set `target_compatible_with` to `@platforms//os:linux`, but he uses C++ for his production code. Where he crosscompiles to other operating systems (e.g. QNX, TriCore). In order to enable this, we need to forward `target_compatible_with` also to the ` also to the also to the `*.update` target. --------- Co-authored-by: Ignas Anikevicius <240938+aignas@users.noreply.github.com> --- CHANGELOG.md | 3 +++ python/private/pypi/pip_compile.bzl | 1 + 2 files changed, 4 insertions(+) diff --git a/CHANGELOG.md b/CHANGELOG.md index df0602785a..1342e2f27b 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -88,6 +88,9 @@ END_UNRELEASED_TEMPLATE path separators; the bootstrap stub is now declared as a sibling of the `.exe` launcher ([#3789](https://github.com/bazel-contrib/rules_python/issues/3789)). +* Fix the forwarding of `target_compatible_with` from `compile_pip_requirements` + towards the underlying `*.update` target. + ([#3787](https://github.com/bazel-contrib/rules_python/pull/3787)) {#v0-0-0-added} diff --git a/python/private/pypi/pip_compile.bzl b/python/private/pypi/pip_compile.bzl index 28923005df..3ef2cdb39c 100644 --- a/python/private/pypi/pip_compile.bzl +++ b/python/private/pypi/pip_compile.bzl @@ -173,6 +173,7 @@ def pip_compile( name = name + ".update", env = env, python_version = kwargs.get("python_version", None), + target_compatible_with = kwargs.get("target_compatible_with", []), **attrs ) From c7efd793ef96cd738ca38c9df66e69b7cb5f2c91 Mon Sep 17 00:00:00 2001 From: Xavier Bonaventura Date: Wed, 20 May 2026 10:47:01 +0200 Subject: [PATCH 744/922] fix(system_python): Remove printing of not always present attribute (#3781) This attribute is not part of the Python public API and in Debian 10 Buster (OpenJDK 11, gcc 8.3.0) it seems to not be defined. This reverts one of the debug logging statements added in https://github.com/bazel-contrib/rules_python/pull/3667 Fixes #3774 --------- Co-authored-by: Ignas Anikevicius <240938+aignas@users.noreply.github.com> --- CHANGELOG.md | 4 +++- python/private/python_bootstrap_template.txt | 2 +- 2 files changed, 4 insertions(+), 2 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 1342e2f27b..3916f191b2 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -84,6 +84,9 @@ END_UNRELEASED_TEMPLATE * (pypi) Fix `importlib.metadata.files` by ensuring `RECORD` is included in installed wheel targets, except when built from sdist ([#3024](https://github.com/bazel-contrib/rules_python/issues/3024)). +* (system_python) Fix AttributeError exception on Debian 10 Buster + python installations which may not set `sys._base_executable` + ([#3774](https://github.com/bazel-contrib/rules_python/issues/3774)). * (windows) Fix `py_test`/`py_binary` failure when the target name contains path separators; the bootstrap stub is now declared as a sibling of the `.exe` launcher @@ -92,7 +95,6 @@ END_UNRELEASED_TEMPLATE towards the underlying `*.update` target. ([#3787](https://github.com/bazel-contrib/rules_python/pull/3787)) - {#v0-0-0-added} ### Added * (toolchain) Added {obj}`python.override.toolchain_target_settings` to allow diff --git a/python/private/python_bootstrap_template.txt b/python/private/python_bootstrap_template.txt index 0d28aff311..38c93ec5b5 100644 --- a/python/private/python_bootstrap_template.txt +++ b/python/private/python_bootstrap_template.txt @@ -568,7 +568,7 @@ def main(): print_verbose("VENV_REL_SITE_PACKAGES:", VENV_REL_SITE_PACKAGES) print_verbose("WORKSPACE_NAME:", WORKSPACE_NAME ) print_verbose("bootstrap sys.executable:", sys.executable) - print_verbose("bootstrap sys._base_executable:", sys._base_executable) + print_verbose("bootstrap sys._base_executable:", getattr(sys, "_base_executable", "unknown")) print_verbose("bootstrap sys.version:", sys.version) args = sys.argv[1:] From bc8a62b3fb5b0f2ac56e16eba5368ae8d30df104 Mon Sep 17 00:00:00 2001 From: Ignas Anikevicius <240938+aignas@users.noreply.github.com> Date: Sat, 23 May 2026 14:49:30 +0900 Subject: [PATCH 745/922] refactor(pypi): cleanup marker evaluation code in requirement parsing (#3765) Another cleanup PR to make the code easier to work with and optimize. --- python/private/pypi/BUILD.bazel | 4 +- python/private/pypi/extension.bzl | 1 - python/private/pypi/hub_builder.bzl | 14 ---- python/private/pypi/parse_requirements.bzl | 64 +++++++------------ python/private/pypi/pip_repository.bzl | 18 +++--- tests/pypi/extension/pip_parse.bzl | 1 - tests/pypi/hub_builder/hub_builder_tests.bzl | 14 +--- .../parse_requirements_tests.bzl | 62 ++++-------------- 8 files changed, 48 insertions(+), 130 deletions(-) diff --git a/python/private/pypi/BUILD.bazel b/python/private/pypi/BUILD.bazel index 2a62767dd4..7569eccac4 100644 --- a/python/private/pypi/BUILD.bazel +++ b/python/private/pypi/BUILD.bazel @@ -190,7 +190,6 @@ bzl_library( visibility = ["//:__subpackages__"], deps = [ ":attrs_bzl", - ":evaluate_markers_bzl", ":parse_requirements_bzl", ":pep508_env_bzl", ":pep508_evaluate_bzl", @@ -247,6 +246,8 @@ bzl_library( ":argparse_bzl", ":index_sources_bzl", ":parse_requirements_txt_bzl", + ":pep508_evaluate_bzl", + ":pep508_requirement_bzl", ":pypi_repo_utils_bzl", ":requirements_files_by_platform_bzl", ":select_whl_bzl", @@ -344,7 +345,6 @@ bzl_library( srcs = ["pip_repository.bzl"], deps = [ ":attrs_bzl", - ":evaluate_markers_bzl", ":parse_requirements_bzl", ":pep508_env_bzl", ":pip_repository_attrs_bzl", diff --git a/python/private/pypi/extension.bzl b/python/private/pypi/extension.bzl index 2a8ace28f2..a0b59e9ed1 100644 --- a/python/private/pypi/extension.bzl +++ b/python/private/pypi/extension.bzl @@ -356,7 +356,6 @@ You cannot use both the additive_build_content and additive_build_content_file a simpleapi_cache = simpleapi_cache, # TODO @aignas 2025-09-06: do not use kwargs minor_mapping = kwargs.get("minor_mapping", MINOR_MAPPING), - evaluate_markers_fn = kwargs.get("evaluate_markers", None), available_interpreters = kwargs.get("available_interpreters", INTERPRETER_LABELS), logger = repo_utils.logger(module_ctx, "pypi:hub:" + hub_name, mod = mod), ) diff --git a/python/private/pypi/hub_builder.bzl b/python/private/pypi/hub_builder.bzl index 1bce648dce..d3be266bdf 100644 --- a/python/private/pypi/hub_builder.bzl +++ b/python/private/pypi/hub_builder.bzl @@ -7,7 +7,6 @@ load("//python/private:text_util.bzl", "render") load("//python/private:version.bzl", "version") load("//python/private:version_label.bzl", "version_label") load(":attrs.bzl", "use_isolated") -load(":evaluate_markers.bzl", "evaluate_markers") load(":parse_requirements.bzl", "parse_requirements") load(":pep508_env.bzl", "env") load(":pep508_evaluate.bzl", "evaluate") @@ -29,7 +28,6 @@ def hub_builder( minor_mapping, available_interpreters, simpleapi_download_fn, - evaluate_markers_fn, logger, simpleapi_cache): """Return a hub builder instance @@ -40,7 +38,6 @@ def hub_builder( config: The platform configuration. whl_overrides: {type}`dict[str, struct]` - per-wheel overrides. minor_mapping: {type}`dict[str, str]` the mapping between minor and full versions. - evaluate_markers_fn: the override function used to evaluate the markers. available_interpreters: {type}`dict[str, Label]` The dictionary of available interpreters that have been registered using the `python` bzlmod extension. The keys are in the form `python_{snake_case_version}_host`. This is to be @@ -97,7 +94,6 @@ def hub_builder( # Instance constants passed in by callers _config = config, _whl_overrides = whl_overrides, - _evaluate_markers_fn = evaluate_markers_fn, _logger = logger, _minor_mapping = minor_mapping, _available_interpreters = available_interpreters, @@ -465,15 +461,6 @@ def _platforms(module_ctx, *, python_version, config, target_platforms): ) return platforms -def _evaluate_markers(self, pip_attr): - if self._evaluate_markers_fn: - return self._evaluate_markers_fn - - return lambda requirements: evaluate_markers( - requirements = requirements, - platforms = self._platforms[pip_attr.python_version], - ) - def _create_whl_repos( self, module_ctx, @@ -509,7 +496,6 @@ def _create_whl_repos( platforms = platforms, extra_pip_args = pip_attr.extra_pip_args, get_index_urls = self._get_index_urls.get(pip_attr.python_version), - evaluate_markers = _evaluate_markers(self, pip_attr), logger = logger, ) diff --git a/python/private/pypi/parse_requirements.bzl b/python/private/pypi/parse_requirements.bzl index 07d0c0989e..c976e74bca 100644 --- a/python/private/pypi/parse_requirements.bzl +++ b/python/private/pypi/parse_requirements.bzl @@ -31,6 +31,7 @@ load("//python/private:repo_utils.bzl", "repo_utils") load(":argparse.bzl", "argparse") load(":index_sources.bzl", "index_sources") load(":parse_requirements_txt.bzl", "parse_requirements_txt") +load(":pep508_evaluate.bzl", "evaluate") load(":pep508_requirement.bzl", "requirement") load(":select_whl.bzl", "select_whl") @@ -41,7 +42,6 @@ def parse_requirements( extra_pip_args = [], platforms = {}, get_index_urls = None, - evaluate_markers = None, extract_url_srcs = True, logger): """Get the requirements with platforms that the requirements apply to. @@ -57,11 +57,6 @@ def parse_requirements( get_index_urls: Callable[[ctx, dict[str, list[str]]], dict], a callable to get all of the distribution URLs from a PyPI index. Accepts ctx and distribution names to query. - evaluate_markers: A function to use to evaluate the requirements. - Accepts a dict where keys are requirement lines to evaluate against - the platforms stored as values in the input dict. Returns the same - dict, but with values being platforms that are compatible with the - requirements line. extract_url_srcs: A boolean to enable extracting URLs from requirement lines to enable using bazel downloader. logger: repo_utils.logger, a simple struct to log diagnostic messages. @@ -86,11 +81,9 @@ def parse_requirements( The second element is extra_pip_args should be passed to `whl_library`. """ - evaluate_markers = evaluate_markers or (lambda _requirements: {}) options = {} requirements = {} all_files_parsed = {} - reqs_with_env_markers = {} index_url = None extra_index_urls = [] for file, plats in requirements_by_platform.items(): @@ -114,39 +107,31 @@ def parse_requirements( tokenized_options.append(p) pip_args = tokenized_options + extra_pip_args + + # Parse the index URL from the requirement files once per file + index_url = argparse.index_url(pip_args, index_url) + extra_index_urls = argparse.extra_index_url(pip_args, []) + if argparse.platform(pip_args, []): + # No use of downloader if the user specifies "--platform" pip arg. This means that + # they intend to use pip to download the wheels + # + # TODO @aignas 2026-04-11: consider removing this line in the next major release + # (3.0). + get_index_urls = None + + # Pre-parse requirements once per file to avoid redundant parsing in loops + parsed_reqs = [(entry, requirement(entry[1])) for entry in parse_result.requirements] + for plat in plats: - requirements[plat] = parse_result.requirements - for entry in parse_result.requirements: - requirement_line = entry[1] + plat_env = platforms.get(plat) - # output all of the requirement lines that have a marker - if ";" in requirement_line: - reqs_with_env_markers.setdefault(requirement_line, []).append(plat) - options[plat] = pip_args + requirements[plat] = [ + entry + for entry, req in parsed_reqs + if not req.marker or (plat_env and evaluate(req.marker, env = plat_env.env)) + ] - # Parse the index URL from the requirement files - index_url = argparse.index_url(pip_args, index_url) - extra_index_urls = argparse.extra_index_url(pip_args, []) - platform = argparse.platform(pip_args, []) - if platform: - # No use of downloader if the user specifies "--platform" pip arg. This means that - # they intend to use pip to download the wheels - # - # TODO @aignas 2026-04-11: consider removing this line in the next major release - # (3.0). - get_index_urls = None - - # This may call to Python, so execute it early (before calling to the - # internet below) and ensure that we call it only once. - # - # TODO @aignas 2026-05-10: remove this assumption in the code because we - # are always using pipstar, so we can do the marker evaluation when we are - # parsing the files. - resolved_marker_platforms = evaluate_markers(reqs_with_env_markers) - logger.trace(lambda: "Evaluated env markers from:\n{}\n\nTo:\n{}".format( - reqs_with_env_markers, - resolved_marker_platforms, - )) + options[plat] = pip_args requirements_by_platform = {} for plat, parse_results in requirements.items(): @@ -170,9 +155,6 @@ def parse_requirements( req_line = entry[1] req = requirement(req_line) - if req.marker and plat not in resolved_marker_platforms.get(req_line, []): - continue - requirements_dict[req.name] = entry extra_pip_args = options[plat] diff --git a/python/private/pypi/pip_repository.bzl b/python/private/pypi/pip_repository.bzl index 4afb62780a..5fcd351958 100644 --- a/python/private/pypi/pip_repository.bzl +++ b/python/private/pypi/pip_repository.bzl @@ -18,7 +18,6 @@ load("@bazel_skylib//lib:sets.bzl", "sets") load("//python/private:normalize_name.bzl", "normalize_name") load("//python/private:repo_utils.bzl", "REPO_DEBUG_ENV_VAR", "repo_utils") load("//python/private:text_util.bzl", "render") -load(":evaluate_markers.bzl", "evaluate_markers") load(":parse_requirements.bzl", "host_platform", "parse_requirements", "select_requirement") load(":pep508_env.bzl", "env") load(":pip_repository_attrs.bzl", "ATTRS") @@ -123,15 +122,14 @@ def _pip_repository_impl(rctx): platforms = platforms, ), extra_pip_args = rctx.attr.extra_pip_args, - evaluate_markers = lambda requirements: evaluate_markers( - requirements = { - # NOTE @aignas 2025-07-07: because we don't distinguish between - # freethreaded and non-freethreaded, it is a 1:1 mapping. - req: {p: p for p in plats} - for req, plats in requirements.items() - }, - platforms = {p: struct(env = marker_env) for p in platforms}, - ), + platforms = { + p: struct( + env = marker_env, + whl_abi_tags = [], + whl_platform_tags = [], + ) + for p in platforms + }, extract_url_srcs = False, logger = logger, ) diff --git a/tests/pypi/extension/pip_parse.bzl b/tests/pypi/extension/pip_parse.bzl index 2d55d5cd1f..95cf666056 100644 --- a/tests/pypi/extension/pip_parse.bzl +++ b/tests/pypi/extension/pip_parse.bzl @@ -65,6 +65,5 @@ def pip_parse( parallel_download = False, experimental_index_url_overrides = {}, simpleapi_skip = simpleapi_skip, - _evaluate_markers_srcs = [], **kwargs ) diff --git a/tests/pypi/hub_builder/hub_builder_tests.bzl b/tests/pypi/hub_builder/hub_builder_tests.bzl index 216528fc9b..d3ea704c27 100644 --- a/tests/pypi/hub_builder/hub_builder_tests.bzl +++ b/tests/pypi/hub_builder/hub_builder_tests.bzl @@ -47,7 +47,6 @@ def hub_builder( config = None, minor_mapping = {}, whl_overrides = {}, - evaluate_markers_fn = None, simpleapi_download_fn = None, log_printer = None, available_interpreters = {}): @@ -87,7 +86,6 @@ def hub_builder( "python_3_15_host": "unit_test_interpreter_target", }, simpleapi_download_fn = simpleapi_download_fn or (lambda *a, **k: {}), - evaluate_markers_fn = evaluate_markers_fn, logger = repo_utils.logger( struct( getenv = { @@ -441,17 +439,7 @@ def _test_simple_with_markers(env): ("linux", "x86_64"): "torch==2.4.1+cpu", } for (host_os, host_arch), want_requirement in sub_tests.items(): - builder = hub_builder( - env, - evaluate_markers_fn = lambda requirements: { - key: [ - platform - for platform in platforms - if ("x86_64" in platform and "platform_machine ==" in key) or ("x86_64" not in platform and "platform_machine !=" in key) - ] - for key, platforms in requirements.items() - }, - ) + builder = hub_builder(env) builder.pip_parse( mocks.mctx( mock_files = { diff --git a/tests/pypi/parse_requirements/parse_requirements_tests.bzl b/tests/pypi/parse_requirements/parse_requirements_tests.bzl index 230fcafa0c..2ef2f44764 100644 --- a/tests/pypi/parse_requirements/parse_requirements_tests.bzl +++ b/tests/pypi/parse_requirements/parse_requirements_tests.bzl @@ -16,7 +16,6 @@ load("@rules_testing//lib:test_suite.bzl", "test_suite") load("//python/private:repo_utils.bzl", "REPO_DEBUG_ENV_VAR", "REPO_VERBOSITY_ENV_VAR", "repo_utils") # buildifier: disable=bzl-visibility -load("//python/private/pypi:evaluate_markers.bzl", "evaluate_markers") # buildifier: disable=bzl-visibility load("//python/private/pypi:parse_requirements.bzl", "select_requirement", _parse_requirements = "parse_requirements") # buildifier: disable=bzl-visibility load("//python/private/pypi:pep508_env.bzl", pep508_env = "env") # buildifier: disable=bzl-visibility load("//tests/support/mocks:mocks.bzl", "mocks") @@ -68,7 +67,7 @@ foo==0.0.1 --hash=sha256:deadbeef foo[extra]==0.0.1 --hash=sha256:deadbeef """, "requirements_marker": """\ -foo[extra]==0.0.1 ;marker --hash=sha256:deadbeef +foo[extra]==0.0.1 ; os_name == 'nt' --hash=sha256:deadbeef bar==0.0.1 --hash=sha256:deadbeef """, "requirements_multi_version": """\ @@ -441,22 +440,24 @@ def _test_select_requirement_none_platform(env): _tests.append(_test_select_requirement_none_platform) def _test_env_marker_resolution(env): - """Test environment marker resolution with ``evaluate_markers``.""" - - def _mock_eval_markers(input): - ret = { - "foo[extra]==0.0.1 ;marker --hash=sha256:deadbeef": ["cp311_windows_x86_64"], - } - - env.expect.that_collection(input.keys()).contains_exactly(ret.keys()) - env.expect.that_collection(input.values()[0]).contains_exactly(["cp311_linux_super_exotic", "cp311_windows_x86_64"]) - return ret + """Test environment marker resolution with platform env information.""" got = parse_requirements( requirements_by_platform = { "requirements_marker": ["cp311_linux_super_exotic", "cp311_windows_x86_64"], }, - evaluate_markers = _mock_eval_markers, + platforms = { + "cp311_linux_super_exotic": struct( + env = pep508_env(os = "linux", arch = "x86_64", python_version = "3.11.0"), + whl_abi_tags = [], + whl_platform_tags = [], + ), + "cp311_windows_x86_64": struct( + env = pep508_env(os = "windows", arch = "x86_64", python_version = "3.11.0"), + whl_abi_tags = [], + whl_platform_tags = [], + ), + }, ) env.expect.that_collection(got).contains_exactly([ struct( @@ -797,17 +798,6 @@ def _test_get_index_urls_different_versions(env): }, ), }, - evaluate_markers = lambda requirements: evaluate_markers( - requirements = requirements, - platforms = { - "cp310_linux_x86_64": struct( - env = {"python_full_version": "3.10.0"}, - ), - "cp39_linux_x86_64": struct( - env = {"python_full_version": "3.9.0"}, - ), - }, - ), ) env.expect.that_collection(got).contains_exactly([ @@ -891,14 +881,6 @@ def _test_get_index_urls_cross_platform(env): ), }, get_index_urls = _get_index_urls, - evaluate_markers = lambda requirements: evaluate_markers( - requirements = requirements, - platforms = { - "cp39_osx_x86_64": struct( - env = {"python_full_version": "3.9.0"}, - ), - }, - ), ) # distributions must include packages from ALL files, even those with @@ -947,14 +929,6 @@ def _test_get_index_urls_single_py_version(env): }, ), }, - evaluate_markers = lambda requirements: evaluate_markers( - requirements = requirements, - platforms = { - "cp310_linux_x86_64": struct( - env = {"python_full_version": "3.10.0"}, - ), - }, - ), ) env.expect.that_collection(got).contains_exactly([ @@ -1004,14 +978,6 @@ def _test_get_index_urls_all_versions(env): ), }, get_index_urls = _get_index_urls, - evaluate_markers = lambda requirements: evaluate_markers( - requirements = requirements, - platforms = { - "cp39_linux_x86_64": struct( - env = {"python_full_version": "3.9.0"}, - ), - }, - ), ) env.expect.that_collection(calls).contains_exactly([ From a9de4d5096a7fe4ac282a6a9ecf70a85ccd23ccb Mon Sep 17 00:00:00 2001 From: Ignas Anikevicius <240938+aignas@users.noreply.github.com> Date: Sun, 24 May 2026 17:38:48 +0900 Subject: [PATCH 746/922] chore: use ruff to lint and format files and apply fixes (#3779) There have been a few attempts at doing these, but the efforts are stale. Since `ruff` is very popular these days and it is only a single binary and is fast, let's use that. From now on we will be able to maintain the files better, because we have a CI step testing the compliance. Closes #3073 --- .../scripts/get_buildkite_results.py | 16 +++-- .github/workflows/ci.yaml | 43 ++++++++++++ .github/workflows/mypy.yaml | 27 -------- .pre-commit-config.yaml | 19 +++-- docs/conf.py | 6 +- .../entry_points/tests/pylint_deps_test.py | 17 +++-- .../bzlmod/entry_points/tests/pylint_test.py | 8 +-- .../entry_points/tests/yamllint_test.py | 6 +- examples/bzlmod/test.py | 3 +- examples/bzlmod/tests/my_lib_test.py | 1 - .../other_module/other_module_import_test.py | 3 + .../tests/my_lib_test.py | 1 - examples/wheel/wheel_test.py | 10 ++- gazelle/manifest/copy_to_source.py | 5 +- gazelle/modules_mapping/test_merger.py | 4 +- python/bin/repl_stub.py | 11 ++- python/private/get_local_runtime_info.py | 20 +++--- python/private/py_console_script_gen.py | 5 +- .../dependency_resolver.py | 4 +- python/private/pypi/namespace_pkg_tmpl.py | 2 +- python/private/pypi/repack_whl.py | 2 +- .../resolve_target_platforms.py | 1 - .../private/pypi/whl_installer/arguments.py | 1 - .../pypi/whl_installer/wheel_installer.py | 2 - python/private/site_init_template.py | 2 +- python/private/stage2_bootstrap_template.py | 6 +- python/private/zipapp/zip_main_template.py | 14 ++-- python/runfiles/__init__.py | 2 +- python/runfiles/runfiles.py | 10 +-- ruff.toml | 29 ++++++++ .../sphinxdocs/private/proto_to_markdown.py | 2 - sphinxdocs/sphinxdocs/private/sphinx_build.py | 9 +-- .../sphinxdocs/private/sphinx_server.py | 4 +- sphinxdocs/sphinxdocs/src/sphinx_bzl/bzl.py | 37 ++++++---- .../proto_to_markdown_test.py | 1 - .../sphinx_stardoc/sphinx_output_test.py | 69 +++++++++++++++---- .../bazel_tools_importable_test.py | 2 +- tests/bootstrap_impls/bin_calls_bin/inner.py | 1 + .../bin_calls_bin/inner_lib.py | 2 +- tests/bootstrap_impls/call_sys_exe.py | 1 - .../system_python_nodeps_test.py | 4 +- tests/build_data/build_data_test.py | 1 - .../abi3_headers_linkage_test.py | 6 +- .../py_console_script_gen_test.py | 4 +- .../namespace_packages_test.py | 1 - tests/integration/custom_commands_test.py | 8 ++- .../integration/local_toolchains/echo_test.py | 1 - .../local_toolchains/repo_runtime_test.py | 3 - .../pip_parse_isolated/test_isolated.py | 3 +- .../toolchain_target_settings/main.py | 1 + tests/modules/other/venv_bin.py | 6 +- tests/py_zipapp/system_python_zipapp_test.py | 1 - .../whl_library/whl_library_extras_test.py | 1 - tests/repl/repl_test.py | 4 +- tests/runfiles/runfiles_test.py | 2 +- .../custom_platform_toolchain_test.py | 4 +- tests/tools/private/release/release_test.py | 5 +- tests/tools/wheelmaker_test.py | 12 +--- tests/tools/zipapp/zip_main_maker_test.py | 4 +- tests/tools/zipapp/zipper_test.py | 9 ++- tests/uv/lock/lock_run_test.py | 1 - tests/uv/toolchain/uv_help_test.py | 6 +- tests/venv_site_packages_libs/bin.py | 13 ++-- .../importlib_metadata_test.py | 1 - .../whl_scripts_runnable_test.py | 2 +- .../whl_filegroup/extract_wheel_files_test.py | 2 +- .../whl_with_build_files/verify_files_test.py | 3 +- tools/precompiler/precompiler.py | 10 +-- .../update_deps/update_coverage_deps.py | 5 +- tools/private/update_deps/update_file.py | 2 - tools/private/update_deps/update_file_test.py | 2 +- tools/private/zipapp/zipper.py | 1 - tools/wheelmaker.py | 53 +++++--------- 73 files changed, 321 insertions(+), 268 deletions(-) create mode 100644 .github/workflows/ci.yaml delete mode 100644 .github/workflows/mypy.yaml create mode 100644 ruff.toml diff --git a/.agents/skills/buildkite-get-results/scripts/get_buildkite_results.py b/.agents/skills/buildkite-get-results/scripts/get_buildkite_results.py index eb5532e4d6..e1fc635cf6 100755 --- a/.agents/skills/buildkite-get-results/scripts/get_buildkite_results.py +++ b/.agents/skills/buildkite-get-results/scripts/get_buildkite_results.py @@ -85,7 +85,10 @@ def fetch_buildkite_data(build_url): elif isinstance(jobs_data, dict) and "records" in jobs_data: data["jobs"] = jobs_data["records"] except Exception as e: - print(f"Warning: Could not fetch detailed jobs from {jobs_url}: {e}", file=sys.stderr) + print( + f"Warning: Could not fetch detailed jobs from {jobs_url}: {e}", + file=sys.stderr, + ) return data @@ -165,15 +168,18 @@ def main(): build_state = data.get("state", "Unknown") print(f"Build State: {build_state}") - + jobs = data.get("jobs", []) jobs_count = data.get("statistics", {}).get("jobs_count", 0) - + print(f"Total jobs reported: {jobs_count}") print(f"Jobs found in data: {len(jobs)}") - + if jobs_count != len(jobs): - print(f"WARNING: Reported job count ({jobs_count}) does not match jobs found ({len(jobs)}).", file=sys.stderr) + print( + f"WARNING: Reported job count ({jobs_count}) does not match jobs found ({len(jobs)}).", + file=sys.stderr, + ) print("-" * 40) diff --git a/.github/workflows/ci.yaml b/.github/workflows/ci.yaml new file mode 100644 index 0000000000..e324bfbf2a --- /dev/null +++ b/.github/workflows/ci.yaml @@ -0,0 +1,43 @@ +name: CI + +on: + push: + branches: + - main + pull_request: + types: + - opened + - synchronize + +defaults: + run: + shell: bash + +permissions: + contents: read + +jobs: + mypy: + runs-on: ubuntu-latest + steps: + # Checkout the code + - uses: actions/checkout@v6 + - uses: jpetrucciani/mypy-check@master + with: + path: 'python/runfiles' + - uses: jpetrucciani/mypy-check@master + with: + path: 'tests/runfiles' + ruff: + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v6 + - uses: astral-sh/ruff-action@v4.0.0 + with: + # Keep in sync with .pre-commit-config.yaml + version: 0.15.14 + args: check --extend-exclude testdata + - uses: astral-sh/ruff-action@v4.0.0 + with: + version: 0.15.14 + args: format --check --exclude testdata diff --git a/.github/workflows/mypy.yaml b/.github/workflows/mypy.yaml deleted file mode 100644 index e38e5c71ba..0000000000 --- a/.github/workflows/mypy.yaml +++ /dev/null @@ -1,27 +0,0 @@ -name: mypy - -on: - push: - branches: - - main - pull_request: - types: - - opened - - synchronize - -defaults: - run: - shell: bash - -jobs: - ci: - runs-on: ubuntu-latest - steps: - # Checkout the code - - uses: actions/checkout@v6 - - uses: jpetrucciani/mypy-check@master - with: - path: 'python/runfiles' - - uses: jpetrucciani/mypy-check@master - with: - path: 'tests/runfiles' diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml index 57d31f5f5f..cada605a26 100644 --- a/.pre-commit-config.yaml +++ b/.pre-commit-config.yaml @@ -29,18 +29,15 @@ repos: - --warnings=all - id: buildifier-lint args: *args - - repo: https://github.com/pycqa/isort - rev: 5.12.0 + - repo: https://github.com/astral-sh/ruff-pre-commit + # Keep in sync with .github/workflows/ruff.yaml + rev: v0.15.14 hooks: - - id: isort - name: isort (python) - args: - - --profile - - black - - repo: https://github.com/psf/black - rev: 25.1.0 - hooks: - - id: black + - id: ruff-check + args: [--fix] + exclude: testdata + - id: ruff-format + exclude: testdata - repo: local hooks: - id: update-deleted-packages diff --git a/docs/conf.py b/docs/conf.py index ef7b66acfa..17b3c17106 100644 --- a/docs/conf.py +++ b/docs/conf.py @@ -133,11 +133,11 @@ # --- Extlinks configuration extlinks = { "gh-issue": ( - f"https://github.com/bazel-contrib/rules_python/issues/%s", + "https://github.com/bazel-contrib/rules_python/issues/%s", "#%s issue", ), - "gh-path": (f"https://github.com/bazel-contrib/rules_python/tree/main/%s", "%s"), - "gh-pr": (f"https://github.com/bazel-contrib/rules_python/pull/%s", "#%s PR"), + "gh-path": ("https://github.com/bazel-contrib/rules_python/tree/main/%s", "%s"), + "gh-pr": ("https://github.com/bazel-contrib/rules_python/pull/%s", "#%s PR"), } # --- MyST configuration diff --git a/examples/bzlmod/entry_points/tests/pylint_deps_test.py b/examples/bzlmod/entry_points/tests/pylint_deps_test.py index f6743ce9b5..54826a137c 100644 --- a/examples/bzlmod/entry_points/tests/pylint_deps_test.py +++ b/examples/bzlmod/entry_points/tests/pylint_deps_test.py @@ -15,7 +15,6 @@ import os import pathlib import subprocess -import tempfile import unittest from python.runfiles import runfiles @@ -29,9 +28,9 @@ def __init__(self, *args, **kwargs): def test_pylint_entry_point(self): rlocation_path = os.environ.get("ENTRY_POINT") - assert ( - rlocation_path is not None - ), "expected 'ENTRY_POINT' env variable to be set to rlocation of the tool" + assert rlocation_path is not None, ( + "expected 'ENTRY_POINT' env variable to be set to rlocation of the tool" + ) entry_point = pathlib.Path(runfiles.Create().Rlocation(rlocation_path)) self.assertTrue(entry_point.exists(), f"'{entry_point}' does not exist") @@ -51,20 +50,20 @@ def test_pylint_entry_point(self): "", proc.stderr.decode("utf-8").strip(), ) - self.assertRegex(proc.stdout.decode("utf-8").strip(), "^pylint 2\.15\.9") + self.assertRegex(proc.stdout.decode("utf-8").strip(), r"^pylint 2\.15\.9") def test_pylint_report_has_expected_warnings(self): rlocation_path = os.environ.get("PYLINT_REPORT") - assert ( - rlocation_path is not None - ), "expected 'PYLINT_REPORT' env variable to be set to rlocation of the report" + assert rlocation_path is not None, ( + "expected 'PYLINT_REPORT' env variable to be set to rlocation of the report" + ) pylint_report = pathlib.Path(runfiles.Create().Rlocation(rlocation_path)) self.assertTrue(pylint_report.exists(), f"'{pylint_report}' does not exist") self.assertRegex( pylint_report.read_text().strip(), - "W8201: Logging should be used instead of the print\(\) function\. \(print-function\)", + r"W8201: Logging should be used instead of the print\(\) function\. \(print-function\)", ) diff --git a/examples/bzlmod/entry_points/tests/pylint_test.py b/examples/bzlmod/entry_points/tests/pylint_test.py index c2532938d8..5a13e61920 100644 --- a/examples/bzlmod/entry_points/tests/pylint_test.py +++ b/examples/bzlmod/entry_points/tests/pylint_test.py @@ -28,9 +28,9 @@ def __init__(self, *args, **kwargs): def test_pylint_entry_point(self): rlocation_path = os.environ.get("ENTRY_POINT") - assert ( - rlocation_path is not None - ), "expected 'ENTRY_POINT' env variable to be set to rlocation of the tool" + assert rlocation_path is not None, ( + "expected 'ENTRY_POINT' env variable to be set to rlocation of the tool" + ) entry_point = pathlib.Path(runfiles.Create().Rlocation(rlocation_path)) self.assertTrue(entry_point.exists(), f"'{entry_point}' does not exist") @@ -50,7 +50,7 @@ def test_pylint_entry_point(self): "", proc.stderr.decode("utf-8").strip(), ) - self.assertRegex(proc.stdout.decode("utf-8").strip(), "^pylint 2\.15\.9") + self.assertRegex(proc.stdout.decode("utf-8").strip(), r"^pylint 2\.15\.9") if __name__ == "__main__": diff --git a/examples/bzlmod/entry_points/tests/yamllint_test.py b/examples/bzlmod/entry_points/tests/yamllint_test.py index 0a0235793b..29b5ebf9b8 100644 --- a/examples/bzlmod/entry_points/tests/yamllint_test.py +++ b/examples/bzlmod/entry_points/tests/yamllint_test.py @@ -28,9 +28,9 @@ def __init__(self, *args, **kwargs): def test_yamllint_entry_point(self): rlocation_path = os.environ.get("ENTRY_POINT") - assert ( - rlocation_path is not None - ), "expected 'ENTRY_POINT' env variable to be set to rlocation of the tool" + assert rlocation_path is not None, ( + "expected 'ENTRY_POINT' env variable to be set to rlocation of the tool" + ) entry_point = pathlib.Path(runfiles.Create().Rlocation(rlocation_path)) self.assertTrue(entry_point.exists(), f"'{entry_point}' does not exist") diff --git a/examples/bzlmod/test.py b/examples/bzlmod/test.py index 24be3ba3fe..3febed7585 100644 --- a/examples/bzlmod/test.py +++ b/examples/bzlmod/test.py @@ -13,7 +13,6 @@ # limitations under the License. import os -import pathlib import re import sys import unittest @@ -59,7 +58,7 @@ def test_coverage_sys_path(self): f"sys.path has {len(sys.path)} items:\n {all_paths}", ) - first_item, last_item = sys.path[0], sys.path[-1] + first_item, _ = sys.path[0], sys.path[-1] self.assertFalse( first_item.endswith("coverage"), f"Expected the first item in sys.path '{first_item}' to not be related to coverage", diff --git a/examples/bzlmod/tests/my_lib_test.py b/examples/bzlmod/tests/my_lib_test.py index b06374c983..019d29e31f 100644 --- a/examples/bzlmod/tests/my_lib_test.py +++ b/examples/bzlmod/tests/my_lib_test.py @@ -12,7 +12,6 @@ # See the License for the specific language governing permissions and # limitations under the License. -import os import sys import libs.my_lib as my_lib diff --git a/examples/bzlmod/tests/other_module/other_module_import_test.py b/examples/bzlmod/tests/other_module/other_module_import_test.py index 6b92a853e0..b5b15c383d 100644 --- a/examples/bzlmod/tests/other_module/other_module_import_test.py +++ b/examples/bzlmod/tests/other_module/other_module_import_test.py @@ -1,8 +1,10 @@ """Regression test for https://github.com/bazel-contrib/rules_python/issues/3563""" + import os import subprocess import sys + def main(): # The rlocation path for the bin_zipapp. It is in the "our_other_module" repository. zipapp_path = os.environ.get("ZIPAPP_PATH") @@ -18,5 +20,6 @@ def main(): print(f"bin_zippapp failed with return code {result.returncode}") sys.exit(result.returncode) + if __name__ == "__main__": main() diff --git a/examples/multi_python_versions/tests/my_lib_test.py b/examples/multi_python_versions/tests/my_lib_test.py index 449cb8473c..b6c577c55a 100644 --- a/examples/multi_python_versions/tests/my_lib_test.py +++ b/examples/multi_python_versions/tests/my_lib_test.py @@ -12,7 +12,6 @@ # See the License for the specific language governing permissions and # limitations under the License. -import os import sys import libs.my_lib as my_lib diff --git a/examples/wheel/wheel_test.py b/examples/wheel/wheel_test.py index 9ed2b842e5..8dcad42138 100644 --- a/examples/wheel/wheel_test.py +++ b/examples/wheel/wheel_test.py @@ -523,7 +523,7 @@ def test_minimal_data_files(self): with zipfile.ZipFile(filename) as zf: self.assertAllEntriesHasReproducibleMetadata(zf) - metadata_file = None + metadata_file = None # noqa: F841 self.assertEqual( zf.namelist(), [ @@ -566,7 +566,9 @@ def test_extra_requires(self): ) def test_requires_dist_depends_on_extras(self): - filename = self._get_path("requires_dist_depends_on_extras-0.0.1-py3-none-any.whl") + filename = self._get_path( + "requires_dist_depends_on_extras-0.0.1-py3-none-any.whl" + ) with zipfile.ZipFile(filename) as zf: self.assertAllEntriesHasReproducibleMetadata(zf) @@ -591,7 +593,9 @@ def test_requires_dist_depends_on_extras(self): ) def test_requires_dist_depends_on_extras_file(self): - filename = self._get_path("requires_dist_depends_on_extras_using_file-0.0.1-py3-none-any.whl") + filename = self._get_path( + "requires_dist_depends_on_extras_using_file-0.0.1-py3-none-any.whl" + ) with zipfile.ZipFile(filename) as zf: self.assertAllEntriesHasReproducibleMetadata(zf) diff --git a/gazelle/manifest/copy_to_source.py b/gazelle/manifest/copy_to_source.py index b897b1fcf3..4342dee843 100644 --- a/gazelle/manifest/copy_to_source.py +++ b/gazelle/manifest/copy_to_source.py @@ -6,7 +6,6 @@ import os import shutil -import stat import sys from pathlib import Path @@ -20,7 +19,9 @@ def copy_to_source(generated_relative_path: Path, target_relative_path: Path) -> generated_absolute_path = Path.cwd() / generated_relative_path # Similarly, the target is relative to the source directory. - target_absolute_path = os.environ["BUILD_WORKSPACE_DIRECTORY"] / target_relative_path + target_absolute_path = ( + os.environ["BUILD_WORKSPACE_DIRECTORY"] / target_relative_path + ) print(f"Copying {generated_absolute_path} to {target_absolute_path}") target_absolute_path.parent.mkdir(parents=True, exist_ok=True) diff --git a/gazelle/modules_mapping/test_merger.py b/gazelle/modules_mapping/test_merger.py index 6260fdd6ff..87c35f4404 100644 --- a/gazelle/modules_mapping/test_merger.py +++ b/gazelle/modules_mapping/test_merger.py @@ -1,7 +1,7 @@ -import pathlib -import unittest import json +import pathlib import tempfile +import unittest from merger import merge_modules_mappings diff --git a/python/bin/repl_stub.py b/python/bin/repl_stub.py index f5b7c0aa4f..858cf810b9 100644 --- a/python/bin/repl_stub.py +++ b/python/bin/repl_stub.py @@ -16,9 +16,9 @@ # Capture the globals from PYTHONSTARTUP so we can pass them on to the console. console_locals = globals().copy() -import code -import rlcompleter -import sys +import code # noqa: E402 +import rlcompleter # noqa: E402 +import sys # noqa: E402 class DynamicCompleter(rlcompleter.Completer): @@ -62,10 +62,7 @@ def complete(self, text, state): elif "GNU readline" in readline.__doc__: # type: ignore readline.parse_and_bind("tab: complete") else: - print( - "Could not enable tab completion: " - "unable to determine readline backend" - ) + print("Could not enable tab completion: unable to determine readline backend") except ImportError: print( "Could not enable tab completion: " diff --git a/python/private/get_local_runtime_info.py b/python/private/get_local_runtime_info.py index a59e17a012..787fad5635 100644 --- a/python/private/get_local_runtime_info.py +++ b/python/private/get_local_runtime_info.py @@ -48,9 +48,7 @@ def _search_directories(get_config, base_executable) -> list[str]: # On MacOS, the LDLIBRARY may be a relative path under /Library/Frameworks, # such as "Python.framework/Versions/3.12/Python", not a file under the # LIBDIR/LIBPL directory, so include PYTHONFRAMEWORKPREFIX. - lib_dirs = [ - get_config(x) for x in ("PYTHONFRAMEWORKPREFIX", "LIBPL", "LIBDIR") - ] + lib_dirs = [get_config(x) for x in ("PYTHONFRAMEWORKPREFIX", "LIBPL", "LIBDIR")] # On Debian, with multiarch enabled, prior to Python 3.10, `LIBDIR` didn't # tell the location of the libs, just the base directory. The `MULTIARCH` @@ -67,8 +65,8 @@ def _search_directories(get_config, base_executable) -> list[str]: if not _IS_DARWIN: for exec_dir in ( - os.path.dirname(base_executable) if base_executable else None, - get_config("BINDIR"), + os.path.dirname(base_executable) if base_executable else None, + get_config("BINDIR"), ): if not exec_dir: continue @@ -122,7 +120,8 @@ def _search_library_names(get_config, version, abi_flags) -> list[str]: # # A typical LIBRARY is 'libpythonX.Y.a' on Linux. lib_names = [ - get_config(x) for x in ( + get_config(x) + for x in ( "LDLIBRARY", "INSTSONAME", "PY3LIBRARY", @@ -167,8 +166,7 @@ def _get_python_library_info(base_executable) -> dict[str, Any]: abi_flags = _get_abi_flags(config_vars.get) search_directories = _search_directories(config_vars.get, base_executable) - search_libnames = _search_library_names(config_vars.get, version, - abi_flags) + search_libnames = _search_library_names(config_vars.get, version, abi_flags) # Used to test whether the library is an abi3 library or a full api library. abi3_libraries = _default_library_names(sys.version_info.major, abi_flags) @@ -221,10 +219,10 @@ def _get_python_library_info(base_executable) -> dict[str, Any]: # Additional DLLs are needed on Windows to link properly. dlls = [] if _IS_WINDOWS: - dlls.extend( - glob.glob(os.path.join(os.path.dirname(base_executable), "*.dll"))) + dlls.extend(glob.glob(os.path.join(os.path.dirname(base_executable), "*.dll"))) dlls = [ - x for x in dlls + x + for x in dlls if x not in dynamic_libraries and x not in abi_dynamic_libraries ] diff --git a/python/private/py_console_script_gen.py b/python/private/py_console_script_gen.py index a1df2c2a06..2be986f732 100644 --- a/python/private/py_console_script_gen.py +++ b/python/private/py_console_script_gen.py @@ -37,9 +37,6 @@ import argparse import configparser import pathlib -import re -import sys -import textwrap _ENTRY_POINTS_TXT = "entry_points.txt" @@ -75,7 +72,7 @@ class EntryPointsParser(configparser.ConfigParser): optionxform = staticmethod(str) -def _guess_entry_point(guess: str, console_scripts: dict[string, string]) -> str | None: +def _guess_entry_point(guess: str, console_scripts: dict[string, string]) -> str | None: # noqa: F821 for key, candidate in console_scripts.items(): if guess == key: return candidate diff --git a/python/private/pypi/dependency_resolver/dependency_resolver.py b/python/private/pypi/dependency_resolver/dependency_resolver.py index f3a339f929..34f2fb1e5a 100644 --- a/python/private/pypi/dependency_resolver/dependency_resolver.py +++ b/python/private/pypi/dependency_resolver/dependency_resolver.py @@ -159,7 +159,9 @@ def main( # Further, shutil.copy preserves the source file's mode, and so if # our source file is read-only (the default under Perforce Helix), # this scratch file will also be read-only, defeating its purpose. - with open(resolved_requirements_file, "rb") as fsrc, open(requirements_out, "wb") as fdst: + with open(resolved_requirements_file, "rb") as fsrc, open( + requirements_out, "wb" + ) as fdst: shutil.copyfileobj(fsrc, fdst) update_command = ( diff --git a/python/private/pypi/namespace_pkg_tmpl.py b/python/private/pypi/namespace_pkg_tmpl.py index a21b846e76..80da1dfb2a 100644 --- a/python/private/pypi/namespace_pkg_tmpl.py +++ b/python/private/pypi/namespace_pkg_tmpl.py @@ -1,2 +1,2 @@ # __path__ manipulation added by bazel-contrib/rules_python to support namespace pkgs. -__path__ = __import__("pkgutil").extend_path(__path__, __name__) +__path__ = __import__("pkgutil").extend_path(__path__, __name__) # noqa: F821 diff --git a/python/private/pypi/repack_whl.py b/python/private/pypi/repack_whl.py index 92d052a81f..4bf554cdc9 100644 --- a/python/private/pypi/repack_whl.py +++ b/python/private/pypi/repack_whl.py @@ -174,7 +174,7 @@ def main(sys_argv): rel_path = p.relative_to(patched_wheel_dir) out.add_file(str(rel_path), p) - logging.debug(f"Writing RECORD file") + logging.debug("Writing RECORD file") got_record = out.add_recordfile() if got_record == record_contents: diff --git a/python/private/pypi/requirements_parser/resolve_target_platforms.py b/python/private/pypi/requirements_parser/resolve_target_platforms.py index accacf5bfa..96607ee240 100755 --- a/python/private/pypi/requirements_parser/resolve_target_platforms.py +++ b/python/private/pypi/requirements_parser/resolve_target_platforms.py @@ -21,7 +21,6 @@ import pathlib from packaging.requirements import Requirement - from python.private.pypi.whl_installer.platform import Platform INPUT_HELP = """\ diff --git a/python/private/pypi/whl_installer/arguments.py b/python/private/pypi/whl_installer/arguments.py index 9122654a11..8471c94ffe 100644 --- a/python/private/pypi/whl_installer/arguments.py +++ b/python/private/pypi/whl_installer/arguments.py @@ -14,7 +14,6 @@ import argparse import json -import pathlib from typing import Any, Dict, Set diff --git a/python/private/pypi/whl_installer/wheel_installer.py b/python/private/pypi/whl_installer/wheel_installer.py index 81dd3995db..9aa3a4a375 100644 --- a/python/private/pypi/whl_installer/wheel_installer.py +++ b/python/private/pypi/whl_installer/wheel_installer.py @@ -18,12 +18,10 @@ import glob import json import os -import re import subprocess import sys from pathlib import Path from tempfile import NamedTemporaryFile -from typing import Dict, List, Optional, Set, Tuple from python.private.pypi.whl_installer import arguments diff --git a/python/private/site_init_template.py b/python/private/site_init_template.py index e4d501bfd5..af80b22a24 100644 --- a/python/private/site_init_template.py +++ b/python/private/site_init_template.py @@ -12,6 +12,7 @@ # See the License for the specific language governing permissions and # limitations under the License. """site initialization logic for Bazel-built py_binary targets.""" + import os import os.path import sys @@ -130,7 +131,6 @@ def _setup_sys_path(): """Perform Bazel/binary specific sys.path setup.""" _print_verbose("site init: initial sys.path:\n", "\n".join(sys.path)) seen = set(sys.path) - python_path_entries = [] def _maybe_add_path(path, reason): if path in seen: diff --git a/python/private/stage2_bootstrap_template.py b/python/private/stage2_bootstrap_template.py index 2a7be67c13..c4326bc026 100644 --- a/python/private/stage2_bootstrap_template.py +++ b/python/private/stage2_bootstrap_template.py @@ -131,11 +131,11 @@ def get_windows_path_with_unc_prefix(path): return path # Lets start the unicode fun - if path.startswith(unicode_prefix): + if path.startswith(unicode_prefix): # noqa: F821 return path # os.path.abspath returns a normalized absolute path - return unicode_prefix + os.path.abspath(path) + return unicode_prefix + os.path.abspath(path) # noqa: F821 def print_verbose(*args, mapping=None, values=None): @@ -356,8 +356,6 @@ def _maybe_collect_coverage(enable): print_verbose_coverage("Instrumented Files:\n" + "\n".join(instrumented_files)) print_verbose_coverage("Sources:\n" + "\n".join(unique_dirs)) - import uuid - import coverage coverage_dir = os.environ["COVERAGE_DIR"] diff --git a/python/private/zipapp/zip_main_template.py b/python/private/zipapp/zip_main_template.py index 06ed19f1a5..709e08815c 100644 --- a/python/private/zipapp/zip_main_template.py +++ b/python/private/zipapp/zip_main_template.py @@ -23,13 +23,13 @@ # TODO(#7091): Remove this hack when no longer necessary. del sys.path[0] -import os -import shutil -import stat -import subprocess -import tempfile -import zipfile -from os.path import basename, dirname, join, normpath +import os # noqa: E402 +import shutil # noqa: E402 +import stat # noqa: E402 +import subprocess # noqa: E402 +import tempfile # noqa: E402 +import zipfile # noqa: E402 +from os.path import basename, dirname, join, normpath # noqa: E402 # runfiles-root-relative path _STAGE2_BOOTSTRAP = "%stage2_bootstrap%" diff --git a/python/runfiles/__init__.py b/python/runfiles/__init__.py index 3dc4141749..cdd7d9af00 100644 --- a/python/runfiles/__init__.py +++ b/python/runfiles/__init__.py @@ -12,4 +12,4 @@ # See the License for the specific language governing permissions and # limitations under the License. -from .runfiles import * +from .runfiles import * # noqa: F403 diff --git a/python/runfiles/runfiles.py b/python/runfiles/runfiles.py index 03ebdf6331..7236d4b851 100644 --- a/python/runfiles/runfiles.py +++ b/python/runfiles/runfiles.py @@ -22,14 +22,14 @@ dependency graphs under bzlmod. ::: """ -import collections.abc + import inspect import os import pathlib import posixpath import sys from collections import defaultdict -from typing import Dict, Generator, Iterable, List, Optional, Tuple, Union +from typing import Dict, Generator, Optional, Tuple, Union if sys.version_info >= (3, 11): from typing import Self @@ -579,9 +579,9 @@ def Rlocation(self, path: str, source_repo: Optional[str] = None) -> Optional[st # which also should not be mapped. return self._strategy.RlocationChecked(path) - assert ( - source_repo is not None - ), "BUG: if the `source_repo` is None, we should never go past the `if` statement above" + assert source_repo is not None, ( + "BUG: if the `source_repo` is None, we should never go past the `if` statement above" + ) # Look up the target repository using the repository mapping if target_canonical is not None: diff --git a/ruff.toml b/ruff.toml new file mode 100644 index 0000000000..fc29594bad --- /dev/null +++ b/ruff.toml @@ -0,0 +1,29 @@ +# ruff.toml + +# Support Python 3.8 and above +target-version = "py38" + +# Match Black's default line length +line-length = 88 + +# Ignore testdata files +extend-exclude = ["**/testdata/**", "testdata"] + +[lint] +# Enable Pyflakes (F) and pycodestyle (E, W) for standard linting, +# and isort (I) for import sorting +select = ["F", "E", "W", "I"] + +# Like Black, allow certain violations that may conflict with its formatting +ignore = ["E501"] + +[lint.isort] +# Matches isort's profile = "black" behavior +combine-as-imports = true +force-single-line = false + +[format] +# The Ruff formatter defaults to Black's style +quote-style = "double" +indent-style = "space" +skip-magic-trailing-comma = false diff --git a/sphinxdocs/sphinxdocs/private/proto_to_markdown.py b/sphinxdocs/sphinxdocs/private/proto_to_markdown.py index 58fb79393d..05278a5c02 100644 --- a/sphinxdocs/sphinxdocs/private/proto_to_markdown.py +++ b/sphinxdocs/sphinxdocs/private/proto_to_markdown.py @@ -13,11 +13,9 @@ # limitations under the License. import argparse -import io import itertools import pathlib import sys -import textwrap from typing import Callable, TextIO, TypeVar from stardoc.proto import stardoc_output_pb2 diff --git a/sphinxdocs/sphinxdocs/private/sphinx_build.py b/sphinxdocs/sphinxdocs/private/sphinx_build.py index b438c89fe1..3605b4579d 100644 --- a/sphinxdocs/sphinxdocs/private/sphinx_build.py +++ b/sphinxdocs/sphinxdocs/private/sphinx_build.py @@ -30,7 +30,6 @@ def __init__(self, message, exit_code): class Worker: - def __init__( self, instream: "typing.TextIO", outstream: "typing.TextIO", exec_root: str ): @@ -83,8 +82,11 @@ def run(self) -> None: if response: self._send_response(response) except SphinxMainError as e: - logger.error("Sphinx main returned failure: exit_code=%s request=%s", - request, e.exit_code) + logger.error( + "Sphinx main returned failure: exit_code=%s request=%s", + request, + e.exit_code, + ) request_id = 0 if not request else request.get("requestId", 0) self._send_response( { @@ -218,7 +220,6 @@ def _process_request(self, request: "WorkRequest") -> "WorkResponse | None": ) raise SphinxMainError(message, exit_code) - # Copying is unfortunately necessary because Bazel doesn't know to # implicily bring along what the symlinks point to. shutil.copytree(worker_outdir, bazel_outdir, dirs_exist_ok=True) diff --git a/sphinxdocs/sphinxdocs/private/sphinx_server.py b/sphinxdocs/sphinxdocs/private/sphinx_server.py index 1bd6ee5550..5d26eaf728 100644 --- a/sphinxdocs/sphinxdocs/private/sphinx_server.py +++ b/sphinxdocs/sphinxdocs/private/sphinx_server.py @@ -19,12 +19,12 @@ class DirectoryHandler(server.SimpleHTTPRequestHandler): def __init__(self, *args, **kwargs): super().__init__(directory=serve_directory, *args, **kwargs) - address = ("0.0.0.0", 8000) + address = ("0.0.0.0", 8000) # noqa: F841 # with server.ThreadingHTTPServer(address, DirectoryHandler) as (ip, port, httpd): with _start_server(DirectoryHandler, "0.0.0.0", 8000) as (ip, port, httpd): def _print_server_info(): - print(f"Serving...") + print("Serving...") print(f" Address: http://{ip}:{port}") print(f" Serving directory: {serve_directory}") print(f" url: file://{serve_directory}") diff --git a/sphinxdocs/sphinxdocs/src/sphinx_bzl/bzl.py b/sphinxdocs/sphinxdocs/src/sphinx_bzl/bzl.py index a1f47b3b1d..c115737ba5 100644 --- a/sphinxdocs/sphinxdocs/src/sphinx_bzl/bzl.py +++ b/sphinxdocs/sphinxdocs/src/sphinx_bzl/bzl.py @@ -22,18 +22,25 @@ from typing import Callable, Iterable, TypeVar from docutils import nodes as docutils_nodes -from docutils.parsers.rst import directives as docutils_directives -from docutils.parsers.rst import states -from sphinx import addnodes, builders -from sphinx import directives as sphinx_directives -from sphinx import domains, environment, roles +from docutils.parsers.rst import directives as docutils_directives, states +from sphinx import ( + addnodes, + builders, + directives as sphinx_directives, + domains, + environment, + roles, +) from sphinx.highlighting import lexer_classes from sphinx.locale import _ -from sphinx.util import docfields -from sphinx.util import docutils as sphinx_docutils -from sphinx.util import inspect, logging -from sphinx.util import nodes as sphinx_nodes -from sphinx.util import typing as sphinx_typing +from sphinx.util import ( + docfields, + docutils as sphinx_docutils, + inspect, + logging, + nodes as sphinx_nodes, + typing as sphinx_typing, +) from typing_extensions import TypeAlias, override _logger = logging.getLogger(__name__) @@ -455,7 +462,7 @@ def make_field( field_text = item[1][0].astext() parts = [p.strip() for p in field_text.split(",")] field_body = docutils_nodes.field_body() - for _, is_last, part in _position_iter(parts): + for _, is_last, part in _position_iter(parts): # noqa: F402 node = self.make_xref( self.bodyrolename, self._body_domain or domain, @@ -608,8 +615,10 @@ def first_child_with_class_name( root, class_name ) -> typing.Union[None, docutils_nodes.Element]: matches = root.findall( - lambda node: isinstance(node, docutils_nodes.Element) - and class_name in node["classes"] + lambda node: ( + isinstance(node, docutils_nodes.Element) + and class_name in node["classes"] + ) ) found = next(matches, None) return found @@ -766,7 +775,7 @@ def make_xref(name, title=None): return obj_id def _signature_add_object_type(self, sig_node: addnodes.desc_signature): - if sig_object_type := self._get_signature_object_type(): + if sig_object_type := self._get_signature_object_type(): # noqa: F841 sig_node += addnodes.desc_annotation("", self._get_signature_object_type()) sig_node += addnodes.desc_sig_space() diff --git a/sphinxdocs/tests/proto_to_markdown/proto_to_markdown_test.py b/sphinxdocs/tests/proto_to_markdown/proto_to_markdown_test.py index c42bcf0b22..d88d2bf127 100644 --- a/sphinxdocs/tests/proto_to_markdown/proto_to_markdown_test.py +++ b/sphinxdocs/tests/proto_to_markdown/proto_to_markdown_test.py @@ -13,7 +13,6 @@ # limitations under the License. import io -import re from absl.testing import absltest from google.protobuf import text_format diff --git a/sphinxdocs/tests/sphinx_stardoc/sphinx_output_test.py b/sphinxdocs/tests/sphinx_stardoc/sphinx_output_test.py index 4ed6d4df94..650e0134d3 100644 --- a/sphinxdocs/tests/sphinx_stardoc/sphinx_output_test.py +++ b/sphinxdocs/tests/sphinx_stardoc/sphinx_output_test.py @@ -1,9 +1,10 @@ import importlib.resources from xml.etree import ElementTree -import tests.sphinx_stardoc as sphinx_stardoc from absl.testing import absltest, parameterized +import tests.sphinx_stardoc as sphinx_stardoc + class SphinxOutputTest(parameterized.TestCase): def setUp(self): @@ -49,21 +50,65 @@ def _doc_element(self, doc): ("short_provider", "LangInfo", "provider.html#LangInfo"), ("short_tag_class", "myext.mytag", "module_extension.html#myext.mytag"), ("full_norepo_func", "//lang:function.bzl%myfunc", "function.html#myfunc"), - ("full_norepo_func_arg", "//lang:function.bzl%myfunc.arg1", "function.html#myfunc.arg1"), + ( + "full_norepo_func_arg", + "//lang:function.bzl%myfunc.arg1", + "function.html#myfunc.arg1", + ), ("full_norepo_rule", "//lang:rule.bzl%my_rule", "rule.html#my_rule"), - ("full_norepo_rule_attr", "//lang:rule.bzl%my_rule.ra1", "rule.html#my_rule.ra1"), - ("full_norepo_provider", "//lang:provider.bzl%LangInfo", "provider.html#LangInfo"), + ( + "full_norepo_rule_attr", + "//lang:rule.bzl%my_rule.ra1", + "rule.html#my_rule.ra1", + ), + ( + "full_norepo_provider", + "//lang:provider.bzl%LangInfo", + "provider.html#LangInfo", + ), ("full_norepo_aspect", "//lang:aspect.bzl%myaspect", "aspect.html#myaspect"), ("full_norepo_target", "//lang:relativetarget", "target.html#relativetarget"), - ("full_repo_func", "@testrepo//lang:function.bzl%myfunc", "function.html#myfunc"), - ("full_repo_func_arg", "@testrepo//lang:function.bzl%myfunc.arg1", "function.html#myfunc.arg1"), + ( + "full_repo_func", + "@testrepo//lang:function.bzl%myfunc", + "function.html#myfunc", + ), + ( + "full_repo_func_arg", + "@testrepo//lang:function.bzl%myfunc.arg1", + "function.html#myfunc.arg1", + ), ("full_repo_rule", "@testrepo//lang:rule.bzl%my_rule", "rule.html#my_rule"), - ("full_repo_rule_attr", "@testrepo//lang:rule.bzl%my_rule.ra1", "rule.html#my_rule.ra1"), - ("full_repo_provider", "@testrepo//lang:provider.bzl%LangInfo", "provider.html#LangInfo"), - ("full_repo_aspect", "@testrepo//lang:aspect.bzl%myaspect", "aspect.html#myaspect"), - ("full_repo_target", "@testrepo//lang:relativetarget", "target.html#relativetarget"), - ("tag_class_attr_using_attr_role", "myext.mytag.ta1", "module_extension.html#myext.mytag.ta1"), - ("tag_class_attr_using_attr_role_just_attr_name", "ta1", "module_extension.html#myext.mytag.ta1"), + ( + "full_repo_rule_attr", + "@testrepo//lang:rule.bzl%my_rule.ra1", + "rule.html#my_rule.ra1", + ), + ( + "full_repo_provider", + "@testrepo//lang:provider.bzl%LangInfo", + "provider.html#LangInfo", + ), + ( + "full_repo_aspect", + "@testrepo//lang:aspect.bzl%myaspect", + "aspect.html#myaspect", + ), + ( + "full_repo_target", + "@testrepo//lang:relativetarget", + "target.html#relativetarget", + ), + ( + "tag_class_attr_using_attr_role", + "myext.mytag.ta1", + "module_extension.html#myext.mytag.ta1", + ), + ( + "tag_class_attr_using_attr_role_just_attr_name", + "ta1", + "module_extension.html#myext.mytag.ta1", + ), ("file_without_repo", "//lang:rule.bzl", "rule.html"), ("file_with_repo", "@testrepo//lang:rule.bzl", "rule.html"), ("package_absolute", "//lang", "target.html"), diff --git a/tests/bootstrap_impls/bazel_tools_importable_test.py b/tests/bootstrap_impls/bazel_tools_importable_test.py index ad753bc03d..c374dd5dcf 100644 --- a/tests/bootstrap_impls/bazel_tools_importable_test.py +++ b/tests/bootstrap_impls/bazel_tools_importable_test.py @@ -7,7 +7,7 @@ def test_bazel_tools_importable(self): try: import bazel_tools import bazel_tools.tools.python - import bazel_tools.tools.python.runfiles + import bazel_tools.tools.python.runfiles # noqa: F401 except ImportError as exc: raise AssertionError( "Failed to import bazel_tools.python.runfiles\n" diff --git a/tests/bootstrap_impls/bin_calls_bin/inner.py b/tests/bootstrap_impls/bin_calls_bin/inner.py index 6fef455a84..74a70b5f0d 100644 --- a/tests/bootstrap_impls/bin_calls_bin/inner.py +++ b/tests/bootstrap_impls/bin_calls_bin/inner.py @@ -9,6 +9,7 @@ try: import tests.bootstrap_impls.bin_calls_bin.inner_lib as inner_lib + print(f"inner: import_result='{inner_lib.confirm()}'") except ImportError as e: print(f"inner: import_result='{e}'") diff --git a/tests/bootstrap_impls/bin_calls_bin/inner_lib.py b/tests/bootstrap_impls/bin_calls_bin/inner_lib.py index 97efbb1565..5815b4f41b 100644 --- a/tests/bootstrap_impls/bin_calls_bin/inner_lib.py +++ b/tests/bootstrap_impls/bin_calls_bin/inner_lib.py @@ -1,3 +1,3 @@ # Rather than having a completely empty file... def confirm(): - return "success" \ No newline at end of file + return "success" diff --git a/tests/bootstrap_impls/call_sys_exe.py b/tests/bootstrap_impls/call_sys_exe.py index 0c6157048c..c431145386 100644 --- a/tests/bootstrap_impls/call_sys_exe.py +++ b/tests/bootstrap_impls/call_sys_exe.py @@ -12,7 +12,6 @@ # See the License for the specific language governing permissions and # limitations under the License. -import os import subprocess import sys diff --git a/tests/bootstrap_impls/system_python_nodeps_test.py b/tests/bootstrap_impls/system_python_nodeps_test.py index d9b43e0f27..1affaab497 100644 --- a/tests/bootstrap_impls/system_python_nodeps_test.py +++ b/tests/bootstrap_impls/system_python_nodeps_test.py @@ -1,11 +1,11 @@ print("Hello, world") # Verify py code from the stdlib can be imported. -import pathlib +import pathlib # noqa: E402 print(pathlib) # Verify a C-implemented module can be imported. # Socket isn't implement in C, but requires `_socket`, # which is implemented in C -import socket +import socket # noqa: E402, F401 diff --git a/tests/build_data/build_data_test.py b/tests/build_data/build_data_test.py index e4ff81a634..874453e8e4 100644 --- a/tests/build_data/build_data_test.py +++ b/tests/build_data/build_data_test.py @@ -4,7 +4,6 @@ class BuildDataTest(unittest.TestCase): - def test_target_build_data(self): import bazel_binary_info diff --git a/tests/cc/current_py_cc_headers/abi3_headers_linkage_test.py b/tests/cc/current_py_cc_headers/abi3_headers_linkage_test.py index 6c337653b1..2d64828278 100644 --- a/tests/cc/current_py_cc_headers/abi3_headers_linkage_test.py +++ b/tests/cc/current_py_cc_headers/abi3_headers_linkage_test.py @@ -1,5 +1,3 @@ -import os.path -import pathlib import sys import unittest @@ -12,7 +10,9 @@ class CheckLinkageTest(unittest.TestCase): @unittest.skipUnless(sys.platform.startswith("win"), "requires windows") def test_linkage_windows(self): rf = runfiles.Create() - dll_path = rf.Rlocation("rules_python/tests/cc/current_py_cc_headers/bin_abi3.dll") + dll_path = rf.Rlocation( + "rules_python/tests/cc/current_py_cc_headers/bin_abi3.dll" + ) pe = pefile.PE(dll_path) if not hasattr(pe, "DIRECTORY_ENTRY_IMPORT"): self.fail("No import directory found.") diff --git a/tests/entry_points/py_console_script_gen_test.py b/tests/entry_points/py_console_script_gen_test.py index 77ad1a5faa..92fa42f167 100644 --- a/tests/entry_points/py_console_script_gen_test.py +++ b/tests/entry_points/py_console_script_gen_test.py @@ -194,8 +194,8 @@ def test_a_second_entry_point_class_method(self): got = out.read_text() - self.assertRegex(got, "from foo\.baz import Bar") - self.assertRegex(got, "sys\.exit\(Bar\.baz\(\)\)") + self.assertRegex(got, r"from foo\.baz import Bar") + self.assertRegex(got, r"sys\.exit\(Bar\.baz\(\)\)") def test_shebang_included(self): with tempfile.TemporaryDirectory() as tmpdir: diff --git a/tests/implicit_namespace_packages/namespace_packages_test.py b/tests/implicit_namespace_packages/namespace_packages_test.py index ea47c08fd2..a1ee27d71b 100644 --- a/tests/implicit_namespace_packages/namespace_packages_test.py +++ b/tests/implicit_namespace_packages/namespace_packages_test.py @@ -2,7 +2,6 @@ class NamespacePackagesTest(unittest.TestCase): - def test_both_importable(self): import nspkg import nspkg.subpkg1 diff --git a/tests/integration/custom_commands_test.py b/tests/integration/custom_commands_test.py index 288a4e7a91..336937ece0 100644 --- a/tests/integration/custom_commands_test.py +++ b/tests/integration/custom_commands_test.py @@ -12,7 +12,6 @@ # See the License for the specific language governing permissions and # limitations under the License. -import logging import unittest from tests.integration import runner @@ -21,7 +20,12 @@ class CustomCommandsTest(runner.TestCase): # Regression test for https://github.com/bazel-contrib/rules_python/issues/1840 def test_run_build_python_zip_false(self): - result = self.run_bazel("run", "--build_python_zip=false", "--@rules_python//python/config_settings:build_python_zip=false", "//:bin") + result = self.run_bazel( + "run", + "--build_python_zip=false", + "--@rules_python//python/config_settings:build_python_zip=false", + "//:bin", + ) self.assert_result_matches(result, "bazel-out") diff --git a/tests/integration/local_toolchains/echo_test.py b/tests/integration/local_toolchains/echo_test.py index 4cc31ff759..17121e0f17 100644 --- a/tests/integration/local_toolchains/echo_test.py +++ b/tests/integration/local_toolchains/echo_test.py @@ -4,6 +4,5 @@ class ExtensionTest(unittest.TestCase): - def test_echo_extension(self): self.assertEqual(echo_ext.echo(42, "str"), tuple(42, "str")) diff --git a/tests/integration/local_toolchains/repo_runtime_test.py b/tests/integration/local_toolchains/repo_runtime_test.py index 4614407c4e..0eb2ef4248 100644 --- a/tests/integration/local_toolchains/repo_runtime_test.py +++ b/tests/integration/local_toolchains/repo_runtime_test.py @@ -1,8 +1,5 @@ import os.path -import shutil -import subprocess import sys -import tempfile import unittest diff --git a/tests/integration/pip_parse_isolated/test_isolated.py b/tests/integration/pip_parse_isolated/test_isolated.py index f889f071fb..5620801955 100644 --- a/tests/integration/pip_parse_isolated/test_isolated.py +++ b/tests/integration/pip_parse_isolated/test_isolated.py @@ -3,9 +3,10 @@ See MODULE.bazel. """ -import six import unittest +import six + class TestIsolated(unittest.TestCase): def test_import(self): diff --git a/tests/integration/toolchain_target_settings/main.py b/tests/integration/toolchain_target_settings/main.py index 6ae68c3f51..52ec605a55 100644 --- a/tests/integration/toolchain_target_settings/main.py +++ b/tests/integration/toolchain_target_settings/main.py @@ -1,2 +1,3 @@ import sys + print(f"Python {sys.version}") diff --git a/tests/modules/other/venv_bin.py b/tests/modules/other/venv_bin.py index 5455b23f40..81de1a9ab9 100644 --- a/tests/modules/other/venv_bin.py +++ b/tests/modules/other/venv_bin.py @@ -2,15 +2,15 @@ print(nspkg) -import nspkg.subnspkg +import nspkg.subnspkg # noqa: E402 print(nspkg.subnspkg) -import nspkg.subnspkg.delta +import nspkg.subnspkg.delta # noqa: E402 print(nspkg.subnspkg.delta) -import nspkg.subnspkg.gamma +import nspkg.subnspkg.gamma # noqa: E402 print(nspkg.subnspkg.gamma) diff --git a/tests/py_zipapp/system_python_zipapp_test.py b/tests/py_zipapp/system_python_zipapp_test.py index 79cd142c4d..7c3e2deeaf 100644 --- a/tests/py_zipapp/system_python_zipapp_test.py +++ b/tests/py_zipapp/system_python_zipapp_test.py @@ -1,7 +1,6 @@ import os import subprocess import unittest -import zipfile class SystemPythonZipAppTest(unittest.TestCase): diff --git a/tests/pypi/whl_library/whl_library_extras_test.py b/tests/pypi/whl_library/whl_library_extras_test.py index 4fe344470a..43cd5aec3f 100644 --- a/tests/pypi/whl_library/whl_library_extras_test.py +++ b/tests/pypi/whl_library/whl_library_extras_test.py @@ -2,7 +2,6 @@ class NamespacePackagesTest(unittest.TestCase): - def test_extras_propagated(self): import pkg diff --git a/tests/repl/repl_test.py b/tests/repl/repl_test.py index 319dab561a..2b3d5c7a4d 100644 --- a/tests/repl/repl_test.py +++ b/tests/repl/repl_test.py @@ -1,6 +1,6 @@ import os import subprocess -import sys +import sys # noqa: F401 import tempfile import unittest from pathlib import Path @@ -89,7 +89,7 @@ def test_repl_version(self): def test_cannot_import_test_module_directly(self): """Validates that we cannot import helper/test_module.py since it's not a direct dep.""" with self.assertRaises(ModuleNotFoundError): - import test_module + import test_module # noqa: F401 @unittest.skipIf( not EXPECT_TEST_MODULE_IMPORTABLE, "test only works without repl_dep set" diff --git a/tests/runfiles/runfiles_test.py b/tests/runfiles/runfiles_test.py index 165ab8c8a9..47c964631f 100644 --- a/tests/runfiles/runfiles_test.py +++ b/tests/runfiles/runfiles_test.py @@ -725,7 +725,7 @@ def __enter__(self) -> Any: tmpdir = os.environ.get("TEST_TMPDIR") self._path = os.path.join(tempfile.mkdtemp(dir=tmpdir), self._name) with open(self._path, "wt", encoding="utf-8", newline="\n") as f: - f.writelines(l + "\n" for l in self._contents) + f.writelines(l + "\n" for l in self._contents) # noqa: E741 return self def __exit__( diff --git a/tests/toolchains/custom_platform_toolchain_test.py b/tests/toolchains/custom_platform_toolchain_test.py index fd28cf772e..2769d4bcf9 100644 --- a/tests/toolchains/custom_platform_toolchain_test.py +++ b/tests/toolchains/custom_platform_toolchain_test.py @@ -3,12 +3,12 @@ class VerifyCustomPlatformToolchainTest(unittest.TestCase): - def test_custom_platform_interpreter_used(self): # For lack of a better option, check the version. Identifying the self.assertEqual( "3.13.1", - f"{sys.version_info.major}.{sys.version_info.minor}.{sys.version_info.micro}") + f"{sys.version_info.major}.{sys.version_info.minor}.{sys.version_info.micro}", + ) if __name__ == "__main__": diff --git a/tests/tools/private/release/release_test.py b/tests/tools/private/release/release_test.py index 676a898440..9f4f3b249b 100644 --- a/tests/tools/private/release/release_test.py +++ b/tests/tools/private/release/release_test.py @@ -1,4 +1,3 @@ -import datetime import os import pathlib import shutil @@ -91,9 +90,9 @@ def test_update_changelog(self): self.assertIn( _UNRELEASED_TEMPLATE, new_content, msg=f"ACTUAL:\n\n{new_content}\n\n" ) - self.assertIn(f"## [1.23.4] - 2025-01-01", new_content) + self.assertIn("## [1.23.4] - 2025-01-01", new_content) self.assertIn( - f"[1.23.4]: https://github.com/bazel-contrib/rules_python/releases/tag/1.23.4", + "[1.23.4]: https://github.com/bazel-contrib/rules_python/releases/tag/1.23.4", new_content, ) self.assertIn("{#v1-23-4}", new_content) diff --git a/tests/tools/wheelmaker_test.py b/tests/tools/wheelmaker_test.py index 7c30981e83..cc160869df 100644 --- a/tests/tools/wheelmaker_test.py +++ b/tests/tools/wheelmaker_test.py @@ -35,14 +35,10 @@ def test_quote_all_false_leaves_simple_filenames_unquoted(self) -> None: def test_quote_all_quotes_filenames_with_commas(self) -> None: """Filenames with commas are always quoted, regardless of quote_all_filenames.""" whl = self._make_whl_file(quote_all=True) - self.assertEqual( - whl._quote_filename("foo,bar/baz.py"), '"foo,bar/baz.py"' - ) + self.assertEqual(whl._quote_filename("foo,bar/baz.py"), '"foo,bar/baz.py"') whl = self._make_whl_file(quote_all=False) - self.assertEqual( - whl._quote_filename("foo,bar/baz.py"), '"foo,bar/baz.py"' - ) + self.assertEqual(whl._quote_filename("foo,bar/baz.py"), '"foo,bar/baz.py"') @dataclass @@ -145,9 +141,7 @@ def test_requirement(self): self.assertEqual(result, "Requires-Dist: requests>=2.0") def test_requirement_and_extra(self): - result = wheelmaker.get_new_requirement_line( - "requests>=2.0", "extra=='dev'" - ) + result = wheelmaker.get_new_requirement_line("requests>=2.0", "extra=='dev'") self.assertEqual(result, "Requires-Dist: requests>=2.0; extra=='dev'") def test_requirement_with_url(self): diff --git a/tests/tools/zipapp/zip_main_maker_test.py b/tests/tools/zipapp/zip_main_maker_test.py index afcaf294d1..dd8e8e8029 100644 --- a/tests/tools/zipapp/zip_main_maker_test.py +++ b/tests/tools/zipapp/zip_main_maker_test.py @@ -36,7 +36,7 @@ def test_creates_zip_main(self): f.write(f"rf-file|0|file1.txt|{file1_path}\n") f.write(f"rf-file|0|file2.txt|{file2_path}\n") f.write(f"rf-symlink|1|symlink.txt|{symlink_path}\n") - f.write(f"rf-empty|empty_file.txt\n") + f.write("rf-empty|empty_file.txt\n") argv = [ "zip_main_maker.py", @@ -58,7 +58,7 @@ def test_creates_zip_main(self): line1 = f"rf-file|0|file1.txt|{file1_path}" line2 = f"rf-file|0|file2.txt|{file2_path}" line3 = f"rf-symlink|1|symlink.txt|{symlink_path}" - line4 = f"rf-empty|empty_file.txt" + line4 = "rf-empty|empty_file.txt" # Sort lines like the program does lines = sorted([line1, line2, line3, line4]) diff --git a/tests/tools/zipapp/zipper_test.py b/tests/tools/zipapp/zipper_test.py index 8b441c4456..ac70917c30 100644 --- a/tests/tools/zipapp/zipper_test.py +++ b/tests/tools/zipapp/zipper_test.py @@ -2,7 +2,6 @@ import pathlib import shutil import tempfile -import time import unittest import zipfile @@ -66,7 +65,7 @@ def test_create_zip_with_files_and_symlinks(self): f"rf-file|0|foo/bar.txt|{file1_path}", f"rf-symlink|1|link1|{symlink_path}", # Should read target 'target.txt' f"rf-root-symlink|0|root_file|{file1_path}", - f"rf-empty|empty_file", + "rf-empty|empty_file", ] self.manifest_path.write_text("\n".join(manifest_content)) @@ -196,7 +195,7 @@ def test_runfiles_mapping_with_cross_repo_paths(self): manifest_content = [ f"rf-file|0|../other_repo/foo.txt|{file1_path}", - f"rf-empty|../other_repo/empty_file", + "rf-empty|../other_repo/empty_file", ] self.manifest_path.write_text("\n".join(manifest_content)) @@ -222,7 +221,7 @@ def test_runfiles_mapping_with_legacy_external_paths(self): manifest_content = [ f"rf-file|0|external/other_repo/foo.txt|{file1_path}", - f"rf-empty|external/other_repo/empty_file", + "rf-empty|external/other_repo/empty_file", ] self.manifest_path.write_text("\n".join(manifest_content)) @@ -265,7 +264,7 @@ def test_output_deterministic(self): f"rf-file|0|b_rf_file|{file2}", # -> runfiles/my_ws/b_rf_file f"rf-root-symlink|0|a_root_link|{file3}", # -> runfiles/a_root_link f"regular|0|a/regular|{file3}", - f"rf-empty|d_rf_empty", # -> runfiles/my_ws/d_rf_empty + "rf-empty|d_rf_empty", # -> runfiles/my_ws/d_rf_empty f"rf-symlink|0|c_rf_link|{file3}", # -> runfiles/my_ws/c_rf_link ] diff --git a/tests/uv/lock/lock_run_test.py b/tests/uv/lock/lock_run_test.py index ef57f23d31..f64cbdccec 100644 --- a/tests/uv/lock/lock_run_test.py +++ b/tests/uv/lock/lock_run_test.py @@ -1,5 +1,4 @@ import subprocess -import sys import tempfile import unittest from pathlib import Path diff --git a/tests/uv/toolchain/uv_help_test.py b/tests/uv/toolchain/uv_help_test.py index be5e755d91..c4515277f7 100755 --- a/tests/uv/toolchain/uv_help_test.py +++ b/tests/uv/toolchain/uv_help_test.py @@ -14,9 +14,9 @@ def test_uv_help(self): data_rpath = os.environ["DATA"] uv_help_path = rfiles.Rlocation(data_rpath) - assert ( - uv_help_path is not None - ), f"the rlocation path was not found: {data_rpath}" + assert uv_help_path is not None, ( + f"the rlocation path was not found: {data_rpath}" + ) uv_help = Path(uv_help_path).read_text() diff --git a/tests/venv_site_packages_libs/bin.py b/tests/venv_site_packages_libs/bin.py index 368251e75b..b14ff54144 100644 --- a/tests/venv_site_packages_libs/bin.py +++ b/tests/venv_site_packages_libs/bin.py @@ -1,5 +1,4 @@ import importlib -import os import sys import sysconfig import unittest @@ -65,7 +64,7 @@ def test_imported_from_venv(self): def test_data_is_included(self): self.assert_imported_from_venv("simple") - module = importlib.import_module("simple") + module = importlib.import_module("simple") # noqa: F841 # Ensure that packages from simple v1 are not present files = [p.name for p in self.site_packages.glob("*")] self.assertIn("simple_v1_extras", files) @@ -80,7 +79,7 @@ def test_override_pkg(self): def test_dirs_from_replaced_package_are_not_present(self): self.assert_imported_from_venv("simple") - module = importlib.import_module("simple") + module = importlib.import_module("simple") # noqa: F841 dist_info_dirs = [p.name for p in self.site_packages.glob("simple*.dist-info")] self.assertEqual( ["simple-1.0.0.dist-info"], @@ -93,14 +92,14 @@ def test_dirs_from_replaced_package_are_not_present(self): def test_data_from_another_pkg_is_included_via_copy_file(self): self.assert_imported_from_venv("simple") - module = importlib.import_module("simple") + module = importlib.import_module("simple") # noqa: F841 # Ensure that packages from simple v1 are not present d = self.site_packages / "external_data" files = [p.name for p in d.glob("*")] self.assertIn("another_module_data.txt", files) def test_whl_with_data1_included(self): - module = self.assert_imported_from_venv("whl_with_data1") + module = self.assert_imported_from_venv("whl_with_data1") # noqa: F841 site_packages_rel = self.site_packages.relative_to(self.venv) # purelib self.assert_venv_path_exists(site_packages_rel / "whl_with_data1/data_file.txt") @@ -110,7 +109,7 @@ def test_whl_with_data1_included(self): site_packages_rel / "whl_with_data1/platlib_file.txt" ) - venv_root = self.venv + venv_root = self.venv # noqa: F841 # data self.assert_venv_path_exists("whl_with_data1/data_data_file.txt") @@ -124,7 +123,7 @@ def test_whl_with_data1_included(self): ) def test_whl_with_data2_included(self): - module = self.assert_imported_from_venv("whl_with_data2") + module = self.assert_imported_from_venv("whl_with_data2") # noqa: F841 site_packages_rel = self.site_packages.relative_to(self.venv) self.assert_venv_path_exists(site_packages_rel / "whl_with_data2/data_file.txt") diff --git a/tests/venv_site_packages_libs/importlib_metadata_test.py b/tests/venv_site_packages_libs/importlib_metadata_test.py index 178ff14c50..963d43b6e0 100644 --- a/tests/venv_site_packages_libs/importlib_metadata_test.py +++ b/tests/venv_site_packages_libs/importlib_metadata_test.py @@ -3,7 +3,6 @@ class ImportlibMetadataTest(unittest.TestCase): - def test_importlib_metadata_files(self): files = importlib.metadata.files("whl-with-data1") self.assertIsNotNone(files, "importlib.metadata.files returned None") diff --git a/tests/venv_site_packages_libs/whl_scripts_runnable_test.py b/tests/venv_site_packages_libs/whl_scripts_runnable_test.py index b62b5a5fce..a0c4210f71 100644 --- a/tests/venv_site_packages_libs/whl_scripts_runnable_test.py +++ b/tests/venv_site_packages_libs/whl_scripts_runnable_test.py @@ -92,7 +92,7 @@ def test_pythonw_script(self): try: os.close(temp_fd) out_path = Path(temp_str) - result = subprocess.run( + subprocess.run( [str(script_path), str(out_path)], capture_output=True, text=True, diff --git a/tests/whl_filegroup/extract_wheel_files_test.py b/tests/whl_filegroup/extract_wheel_files_test.py index 125d7f312c..4bf1bf3f11 100644 --- a/tests/whl_filegroup/extract_wheel_files_test.py +++ b/tests/whl_filegroup/extract_wheel_files_test.py @@ -26,7 +26,7 @@ def test_get_wheel_record(self) -> None: self.assertEqual(list(record), list(expected)) def test_get_files(self) -> None: - pattern = "(examples/wheel/lib/.*\.txt$|.*main)" + pattern = r"(examples/wheel/lib/.*\.txt$|.*main)" record = extract_wheel_files.get_record(_WHEEL) files = extract_wheel_files.get_files(record, pattern) expected = [ diff --git a/tests/whl_with_build_files/verify_files_test.py b/tests/whl_with_build_files/verify_files_test.py index cfbbaa3aff..8d16b34348 100644 --- a/tests/whl_with_build_files/verify_files_test.py +++ b/tests/whl_with_build_files/verify_files_test.py @@ -2,7 +2,6 @@ class VerifyFilestest(unittest.TestCase): - def test_wheel_with_build_files_importable(self): # If the BUILD files are present, then these imports should fail # because globs won't pass package boundaries, and the necessary @@ -10,7 +9,7 @@ def test_wheel_with_build_files_importable(self): import somepkg import somepkg.a import somepkg.subpkg - import somepkg.subpkg.b + import somepkg.subpkg.b # noqa: F401 if __name__ == "__main__": diff --git a/tools/precompiler/precompiler.py b/tools/precompiler/precompiler.py index 0afc2be530..f83dd15951 100644 --- a/tools/precompiler/precompiler.py +++ b/tools/precompiler/precompiler.py @@ -79,7 +79,7 @@ def _compile(options: "argparse.Namespace") -> None: class _SerialPersistentWorker: """Simple, synchronous, serial persistent worker.""" - def __init__(self, instream: "typing.TextIO", outstream: "typing.TextIO"): + def __init__(self, instream: "typing.TextIO", outstream: "typing.TextIO"): # noqa: F821 self._instream = instream self._outstream = outstream self._parser = _create_parser() @@ -148,7 +148,7 @@ def _send_response(self, response: "JsonWorkResponse") -> None: class _AsyncPersistentWorker: """Asynchronous, concurrent, persistent worker.""" - def __init__(self, reader: "typing.TextIO", writer: "typing.TextIO"): + def __init__(self, reader: "typing.TextIO", writer: "typing.TextIO"): # noqa: F821 self._reader = reader self._writer = writer self._parser = _create_parser() @@ -156,13 +156,15 @@ def __init__(self, reader: "typing.TextIO", writer: "typing.TextIO"): self._task_to_request_id = {} @classmethod - async def main(cls, instream: "typing.TextIO", outstream: "typing.TextIO") -> None: + async def main(cls, instream: "typing.TextIO", outstream: "typing.TextIO") -> None: # noqa: F821 reader, writer = await cls._connect_streams(instream, outstream) await cls(reader, writer).run() @classmethod async def _connect_streams( - cls, instream: "typing.TextIO", outstream: "typing.TextIO" + cls, + instream: "typing.TextIO", # noqa: F821 + outstream: "typing.TextIO", # noqa: F821 ) -> "tuple[asyncio.StreamReader, asyncio.StreamWriter]": loop = asyncio.get_event_loop() reader = asyncio.StreamReader() diff --git a/tools/private/update_deps/update_coverage_deps.py b/tools/private/update_deps/update_coverage_deps.py index 81df6fc161..fcb44fcc7c 100755 --- a/tools/private/update_deps/update_coverage_deps.py +++ b/tools/private/update_deps/update_coverage_deps.py @@ -20,11 +20,8 @@ # NOTE @aignas 2023-01-09: We should only depend on core Python 3 packages. import argparse -import difflib import json import os -import pathlib -import sys import textwrap from collections import defaultdict from dataclasses import dataclass @@ -183,7 +180,7 @@ def main(): if u["python_version"] not in args.py: continue - if f'_{u["python_version"]}m_' in u["filename"]: + if f"_{u['python_version']}m_" in u["filename"]: continue platforms = _get_platforms( diff --git a/tools/private/update_deps/update_file.py b/tools/private/update_deps/update_file.py index ab3e8a817e..cbf4c32bd1 100644 --- a/tools/private/update_deps/update_file.py +++ b/tools/private/update_deps/update_file.py @@ -17,10 +17,8 @@ This is reused in other files updating coverage deps and pip deps. """ -import argparse import difflib import pathlib -import sys def _writelines(path: pathlib.Path, out: str): diff --git a/tools/private/update_deps/update_file_test.py b/tools/private/update_deps/update_file_test.py index 01c6ec74b0..a3cb1c0a6d 100644 --- a/tools/private/update_deps/update_file_test.py +++ b/tools/private/update_deps/update_file_test.py @@ -30,7 +30,7 @@ def test_replace_simple(self): After the snippet """ - snippet = "Replaced" + snippet = "Replaced" # noqa: F841 got = replace_snippet( current=current, snippet="Replaced", diff --git a/tools/private/zipapp/zipper.py b/tools/private/zipapp/zipper.py index 870861bc07..5a8eb8c6fd 100644 --- a/tools/private/zipapp/zipper.py +++ b/tools/private/zipapp/zipper.py @@ -1,7 +1,6 @@ import argparse import os import shutil -import stat import sys import zipfile from os.path import dirname diff --git a/tools/wheelmaker.py b/tools/wheelmaker.py index ada525e9bf..70e375b4ee 100644 --- a/tools/wheelmaker.py +++ b/tools/wheelmaker.py @@ -24,7 +24,6 @@ import stat import sys import zipfile -from collections.abc import Iterable from pathlib import Path _ZIP_EPOCH = (1980, 1, 1, 0, 0, 0) @@ -94,9 +93,7 @@ def normalize_pep440(version): substituted = re.sub(r"\{\w+\}", "0", version) delimiter = "." if "+" in substituted else "+" try: - return str( - packaging.version.Version(f"{substituted}{delimiter}{sanitized}") - ) + return str(packaging.version.Version(f"{substituted}{delimiter}{sanitized}")) except packaging.version.InvalidVersion: return str(packaging.version.Version(f"0+{sanitized}")) @@ -104,7 +101,7 @@ def normalize_pep440(version): def arcname_from( name: str, distribution_prefix: str, - strip_path_prefixes: Sequence[str] = (), + strip_path_prefixes: Sequence[str] = (), # noqa: F821 add_path_prefix: str = "", ) -> str: """Return the within-archive name for a given file path name. @@ -120,9 +117,7 @@ def arcname_from( # Always use unix path separators. normalized_arcname = name.replace(os.path.sep, "/") # Don't manipulate names filenames in the .distinfo or .data directories. - if distribution_prefix and normalized_arcname.startswith( - distribution_prefix - ): + if distribution_prefix and normalized_arcname.startswith(distribution_prefix): return normalized_arcname for prefix in strip_path_prefixes: if normalized_arcname.startswith(prefix): @@ -205,9 +200,7 @@ def add_string(self, filename, contents): self.writestr(zinfo, contents) hash = hashlib.sha256() hash.update(contents) - self._add_to_record( - filename, self._serialize_digest(hash), len(contents) - ) + self._add_to_record(filename, self._serialize_digest(hash), len(contents)) def _serialize_digest(self, hash) -> str: # https://www.python.org/dev/peps/pep-0376/#record @@ -244,9 +237,7 @@ def _quote_filename(self, filename: str) -> str: filename = filename.lstrip("/") # Some RECORDs like torch have *all* filenames quoted and we must minimize diff. # Otherwise, we quote only when necessary (e.g. for filenames with commas). - quoting = ( - csv.QUOTE_ALL if self.quote_all_filenames else csv.QUOTE_MINIMAL - ) + quoting = csv.QUOTE_ALL if self.quote_all_filenames else csv.QUOTE_MINIMAL with io.StringIO() as buf: csv.writer(buf, quoting=quoting).writerow([filename]) return buf.getvalue().strip() @@ -288,8 +279,8 @@ def __init__( self._strip_path_prefixes = strip_path_prefixes self._add_path_prefix = add_path_prefix self._compress = compress - self._wheelname_fragment_distribution_name = ( - escape_filename_distribution_name(self._name) + self._wheelname_fragment_distribution_name = escape_filename_distribution_name( + self._name ) self._distribution_prefix = ( @@ -350,9 +341,7 @@ def add_wheelfile(self): Wheel-Version: 1.0 Generator: bazel-wheelmaker 1.0 Root-Is-Purelib: {} -""".format( - "true" if self._platform == "any" else "false" - ) +""".format("true" if self._platform == "any" else "false") for tag in self.disttags(): wheel_contents += "Tag: %s\n" % tag self._whlfile.add_string(self.distinfo_path("WHEEL"), wheel_contents) @@ -361,9 +350,7 @@ def add_metadata(self, metadata, name, description): """Write METADATA file to the distribution.""" # https://www.python.org/dev/peps/pep-0566/ # https://packaging.python.org/specifications/core-metadata/ - metadata = re.sub( - "^Name: .*$", "Name: %s" % name, metadata, flags=re.MULTILINE - ) + metadata = re.sub("^Name: .*$", "Name: %s" % name, metadata, flags=re.MULTILINE) metadata += "Version: %s\n\n" % self._version # setuptools seems to insert UNKNOWN as description when none is # provided. @@ -389,6 +376,9 @@ def get_files_to_package(input_files): def get_new_requirement_line(reqs_text: str, extra: str) -> str: """Formats a requirement text into a Requires-Dist metadata line.""" + # This is not imported at the top of the file due to the reliance + # on this file in the `whl_library` repository rule which does not + # provide `packaging` but does import symbols defined here. from packaging.requirements import Requirement req = Requirement(reqs_text.strip()) @@ -442,9 +432,7 @@ def resolve_argument_stamp( def parse_args() -> argparse.Namespace: parser = argparse.ArgumentParser(description="Builds a python wheel") - metadata_group = parser.add_argument_group( - "Wheel name, version and platform" - ) + metadata_group = parser.add_argument_group("Wheel name, version and platform") metadata_group.add_argument( "--name", required=True, type=str, help="Name of the distribution" ) @@ -560,7 +548,7 @@ def parse_args() -> argparse.Namespace: return parser.parse_args(sys.argv[1:]) -def _parse_file_pairs(content: List[str]) -> List[List[str]]: +def _parse_file_pairs(content: List[str]) -> List[List[str]]: # noqa: F821 """ Parse ; delimited lists of files into a 2D list. """ @@ -629,11 +617,6 @@ def main() -> None: metadata = arguments.metadata_file.read_text(encoding="utf-8") - # This is not imported at the top of the file due to the reliance - # on this file in the `whl_library` repository rule which does not - # provide `packaging` but does import symbols defined here. - from packaging.requirements import Requirement - # Search for any `Requires-Dist` entries that refer to other files and # expand them. @@ -643,9 +626,7 @@ def main() -> None: if not meta_line[len("Requires-Dist: ") :].startswith("@"): # This is a normal requirement. - package, _, extra = meta_line[ - len("Requires-Dist: ") : - ].rpartition(";") + package, _, extra = meta_line[len("Requires-Dist: ") :].rpartition(";") if not package: # This is when the package requirement does not have markers. continue @@ -660,9 +641,7 @@ def main() -> None: extra = extra.strip() reqs = [] - for reqs_line in ( - Path(file).read_text(encoding="utf-8").splitlines() - ): + for reqs_line in Path(file).read_text(encoding="utf-8").splitlines(): reqs_text = reqs_line.strip() if not reqs_text or reqs_text.startswith(("#", "-")): continue From cef5a6a789fe1d58a8ef3819d5871fc7dbe44f67 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Wed, 27 May 2026 01:24:31 -0700 Subject: [PATCH 747/922] fix: avoid ln race condition in bootstrap script (#3797) Avoid startup failures when multiple processes attempt to create the same symlinks simultaneously. Ensure symlinks are created and ignore errors if they already exist, which can happen in a race condition during startup. --- CHANGELOG.md | 2 ++ python/private/stage1_bootstrap_template.sh | 32 ++++++++++++++++++--- 2 files changed, 30 insertions(+), 4 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 3916f191b2..24d814d674 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -72,6 +72,8 @@ END_UNRELEASED_TEMPLATE {#v0-0-0-fixed} ### Fixed +* (bootstrap) Fixed a potential race condition with symlink creation during + startup. * (gazelle) Fixed handling of auto-included `__init__.py` files when generating `py_binary` targets ([#3729](https://github.com/bazel-contrib/rules_python/issues/3729)). * (entry_point) From now on `mypy` type checking will be skipped on the generated diff --git a/python/private/stage1_bootstrap_template.sh b/python/private/stage1_bootstrap_template.sh index c72e2740f2..4374ff95b0 100644 --- a/python/private/stage1_bootstrap_template.sh +++ b/python/private/stage1_bootstrap_template.sh @@ -6,6 +6,30 @@ if [[ -n "${RULES_PYTHON_BOOTSTRAP_VERBOSE:-}" ]]; then set -x fi +# Creates a symlink. If the symlink already exists, it is tolerated to avoid +# race conditions during startup. +function _symlink() { + local target="$1" + local link="$2" + if ln -s "$target" "$link" 2>/dev/null; then + return 0 + fi + # If it failed, maybe it already exists because of a race. + if [[ -L "$link" || -e "$link" ]]; then + return 0 + fi + # If it doesn't exist, maybe we don't have write permission in the directory. + local dir + dir=$(dirname "$link") + if [[ ! -w "$dir" ]]; then + echo >&2 "ERROR: Cannot create symlink $link: Directory $dir is not writable" + else + echo >&2 "ERROR: Failed to create symlink $link -> $target" + fi + return 1 +} + + # runfiles-root-relative path STAGE2_BOOTSTRAP="%stage2_bootstrap%" @@ -153,7 +177,7 @@ if [[ "$IS_ZIPFILE" == "1" ]]; then fi # The bin/ directory may not exist if it is empty. mkdir -p "$(dirname $python_exe)" - ln -s "$symlink_to" "$python_exe" + _symlink "$symlink_to" "$python_exe" elif [[ "$RECREATE_VENV_AT_RUNTIME" == "1" ]]; then if [[ -n "$RULES_PYTHON_EXTRACT_ROOT" ]]; then use_exec=1 @@ -226,16 +250,16 @@ EOF fi mkdir -p "$venv/bin" - ln -s "$python_exe_actual" "$python_exe" + _symlink "$python_exe_actual" "$python_exe" if [[ ! -e "$venv_site_packages" ]]; then mkdir -p $(dirname $venv_site_packages) - ln -s "$runfiles_venv_site_packages" "$venv_site_packages" + _symlink "$runfiles_venv_site_packages" "$venv_site_packages" fi fi if [[ ! -e "$venv/pyvenv.cfg" ]]; then - ln -s "$runfiles_venv/pyvenv.cfg" "$venv/pyvenv.cfg" + _symlink "$runfiles_venv/pyvenv.cfg" "$venv/pyvenv.cfg" fi else use_exec=1 From 733f2404b42278b08121d49eecb7372264c303ac Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Wed, 27 May 2026 01:25:09 -0700 Subject: [PATCH 748/922] chore: use direct @dev_pip targets instead of requirement() (#3798) Use @dev_pip// format instead of the requirement() macro. This aligns with modern bzlmod patterns and simplifies the BUILD file. --- docs/BUILD.bazel | 15 +++++++-------- 1 file changed, 7 insertions(+), 8 deletions(-) diff --git a/docs/BUILD.bazel b/docs/BUILD.bazel index 80aae58607..67f21b253b 100644 --- a/docs/BUILD.bazel +++ b/docs/BUILD.bazel @@ -13,7 +13,6 @@ # limitations under the License. load("@bazel_skylib//rules:build_test.bzl", "build_test") -load("@dev_pip//:requirements.bzl", "requirement") load("@sphinxdocs//sphinxdocs:readthedocs.bzl", "readthedocs_install") load("@sphinxdocs//sphinxdocs:sphinx.bzl", "sphinx_build_binary", "sphinx_docs") load("@sphinxdocs//sphinxdocs:sphinx_docs_library.bzl", "sphinx_docs_library") @@ -175,13 +174,13 @@ sphinx_build_binary( }, target_compatible_with = _TARGET_COMPATIBLE_WITH, deps = [ - requirement("sphinx"), - requirement("sphinx_rtd_theme"), - requirement("myst_parser"), - requirement("readthedocs_sphinx_ext"), - requirement("typing_extensions"), - requirement("sphinx_autodoc2"), - requirement("sphinx_reredirects"), + "@dev_pip//myst_parser", + "@dev_pip//readthedocs_sphinx_ext", + "@dev_pip//sphinx", + "@dev_pip//sphinx_autodoc2", + "@dev_pip//sphinx_reredirects", + "@dev_pip//sphinx_rtd_theme", + "@dev_pip//typing_extensions", "@sphinxdocs//sphinxdocs/src/sphinx_bzl", ], ) From 10e1f7c920de4f03c889bc65c3fb2bb404cece1a Mon Sep 17 00:00:00 2001 From: Ignas Anikevicius <240938+aignas@users.noreply.github.com> Date: Fri, 29 May 2026 18:15:15 +0900 Subject: [PATCH 749/922] fix(pypi): do not fail on indexes without root index (#3799) With this PR we are changing the strategy of assuming that all of the packages that we have here will be available through the index and we stop calling the root index if only a single index is specified. This will improve the performance for public-only hubs (like the twine deps) and should in generally reflect better the pre-2.0 behaviour. Fixes #3769 Closes #3770 --- CHANGELOG.md | 5 +++ python/private/pypi/simpleapi_download.bzl | 15 ++++---- .../bzlmod_lockfile/MODULE.bazel.lock | 38 +------------------ .../simpleapi_download_tests.bzl | 14 ------- 4 files changed, 14 insertions(+), 58 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 24d814d674..bf6fe3095c 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -96,6 +96,11 @@ END_UNRELEASED_TEMPLATE * Fix the forwarding of `target_compatible_with` from `compile_pip_requirements` towards the underlying `*.update` target. ([#3787](https://github.com/bazel-contrib/rules_python/pull/3787)) +* (pypi) Assume that all of the packages are available on a particular hub if + there is only a single PyPI compatible index to be used. This saves us an expensive + PyPI download and supports PyPI mirror implementations that do not support the root + index functionality. Fixes + ([#3769](https://github.com/bazel-contrib/rules_python/pull/3769)). {#v0-0-0-added} ### Added diff --git a/python/private/pypi/simpleapi_download.bzl b/python/private/pypi/simpleapi_download.bzl index 63044bc14a..5377a08093 100644 --- a/python/private/pypi/simpleapi_download.bzl +++ b/python/private/pypi/simpleapi_download.bzl @@ -131,9 +131,15 @@ def simpleapi_download( return contents def _get_dist_urls(ctx, *, default_index, index_urls, index_url_overrides, sources, read_simpleapi, attr, block, _fail = fail, **kwargs): - if index_url_overrides: + # Ensure the value is not frozen + index_urls = [] + (index_urls or []) + if default_index not in index_urls: + index_urls.append(default_index) + + index_url_overrides = index_url_overrides or {} + if index_url_overrides or len(index_urls) == 1: # Let's not call the index at all and just assume that all of the overrides have been - # specified. + # specified or there is only a single index and there is no need to download anything return { pkg: _normalize_url("{}/{}/".format( index_url_overrides.get(pkg, default_index), @@ -145,11 +151,6 @@ def _get_dist_urls(ctx, *, default_index, index_urls, index_url_overrides, sourc downloads = {} results = {} - # Ensure the value is not frozen - index_urls = [] + (index_urls or []) - if default_index not in index_urls: - index_urls.append(default_index) - for index_url in index_urls: download = read_simpleapi( ctx = ctx, diff --git a/tests/integration/bzlmod_lockfile/MODULE.bazel.lock b/tests/integration/bzlmod_lockfile/MODULE.bazel.lock index be296a56ac..0dcc4b2a48 100644 --- a/tests/integration/bzlmod_lockfile/MODULE.bazel.lock +++ b/tests/integration/bzlmod_lockfile/MODULE.bazel.lock @@ -109,7 +109,6 @@ "https://bcr.bazel.build/modules/rules_cc/0.0.9/MODULE.bazel": "836e76439f354b89afe6a911a7adf59a6b2518fafb174483ad78a2a2fde7b1c5", "https://bcr.bazel.build/modules/rules_cc/0.1.1/MODULE.bazel": "2f0222a6f229f0bf44cd711dc13c858dad98c62d52bd51d8fc3a764a83125513", "https://bcr.bazel.build/modules/rules_cc/0.1.2/MODULE.bazel": "557ddc3a96858ec0d465a87c0a931054d7dcfd6583af2c7ed3baf494407fd8d0", - "https://bcr.bazel.build/modules/rules_cc/0.1.5/MODULE.bazel": "88dfc9361e8b5ae1008ac38f7cdfd45ad738e4fa676a3ad67d19204f045a1fd8", "https://bcr.bazel.build/modules/rules_cc/0.2.0/MODULE.bazel": "b5c17f90458caae90d2ccd114c81970062946f49f355610ed89bebf954f5783c", "https://bcr.bazel.build/modules/rules_cc/0.2.13/MODULE.bazel": "eecdd666eda6be16a8d9dc15e44b5c75133405e820f620a234acc4b1fdc5aa37", "https://bcr.bazel.build/modules/rules_cc/0.2.17/MODULE.bazel": "1849602c86cb60da8613d2de887f9566a6d354a6df6d7009f9d04a14402f9a84", @@ -642,42 +641,7 @@ } } }, - "fact_version": "v1", - "index_urls": { - "https://pypi.org/simple/": { - "backports_tarfile": "/simple/backports-tarfile/", - "certifi": "/simple/certifi/", - "cffi": "/simple/cffi/", - "charset_normalizer": "/simple/charset-normalizer/", - "cryptography": "/simple/cryptography/", - "docutils": "/simple/docutils/", - "idna": "/simple/idna/", - "importlib_metadata": "/simple/importlib-metadata/", - "jaraco_classes": "/simple/jaraco-classes/", - "jaraco_context": "/simple/jaraco-context/", - "jaraco_functools": "/simple/jaraco-functools/", - "jeepney": "/simple/jeepney/", - "keyring": "/simple/keyring/", - "markdown_it_py": "/simple/markdown-it-py/", - "mdurl": "/simple/mdurl/", - "more_itertools": "/simple/more-itertools/", - "nh3": "/simple/nh3/", - "pkginfo": "/simple/pkginfo/", - "pycparser": "/simple/pycparser/", - "pygments": "/simple/pygments/", - "pywin32_ctypes": "/simple/pywin32-ctypes/", - "readme_renderer": "/simple/readme-renderer/", - "requests": "/simple/requests/", - "requests_toolbelt": "/simple/requests-toolbelt/", - "rfc3986": "/simple/rfc3986/", - "rich": "/simple/rich/", - "secretstorage": "/simple/secretstorage/", - "six": "/simple/six/", - "twine": "/simple/twine/", - "urllib3": "/simple/urllib3/", - "zipp": "/simple/zipp/" - } - } + "fact_version": "v1" } } } diff --git a/tests/pypi/simpleapi_download/simpleapi_download_tests.bzl b/tests/pypi/simpleapi_download/simpleapi_download_tests.bzl index 405c497508..4e86b76e10 100644 --- a/tests/pypi/simpleapi_download/simpleapi_download_tests.bzl +++ b/tests/pypi/simpleapi_download/simpleapi_download_tests.bzl @@ -169,12 +169,6 @@ _tests.append(_test_index_overrides) def _test_download_url(env): downloads = {} reads = [ - # The first read is the index which seeds the downloads later - """ - bar - baz - foo - """, "", "", "", @@ -208,7 +202,6 @@ def _test_download_url(env): ) env.expect.that_dict(downloads).contains_exactly({ - "https://example.com/main/simple/": "path/for/https___example_com_main_simple.html", "https://example.com/main/simple/bar/": "path/for/https___example_com_main_simple_bar.html", "https://example.com/main/simple/baz/": "path/for/https___example_com_main_simple_baz.html", "https://example.com/main/simple/foo/": "path/for/https___example_com_main_simple_foo.html", @@ -316,12 +309,6 @@ _tests.append(_test_download_url_parallel_with_overrides) def _test_download_envsubst_url(env): downloads = {} reads = [ - # The first read is the index which seeds the downloads later - """ - bar - baz - foo - """, "", "", "", @@ -355,7 +342,6 @@ def _test_download_envsubst_url(env): ) env.expect.that_dict(downloads).contains_exactly({ - "https://example.com/main/simple/": "path/for/~index_url~.html", "https://example.com/main/simple/bar/": "path/for/~index_url~_bar.html", "https://example.com/main/simple/baz/": "path/for/~index_url~_baz.html", "https://example.com/main/simple/foo/": "path/for/~index_url~_foo.html", From ae129b34e41fc29ba9bf95655d96682370fcfd0c Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Sun, 31 May 2026 07:50:07 -0700 Subject: [PATCH 750/922] fix(build-data): remove CONFIG_MODE from build data (#3801) CONFIG_MODE was included in the build data but it is not actively used and can cause issues with reproducibility. Removing it simplifies the build data and improves reproducibility. Closes #3793 --- CHANGELOG.md | 2 ++ python/private/build_data_writer.ps1 | 1 - python/private/build_data_writer.sh | 1 - python/private/py_executable.bzl | 3 --- tests/build_data/build_data_test.py | 2 -- 5 files changed, 2 insertions(+), 7 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index bf6fe3095c..e57e0969d3 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -54,6 +54,8 @@ END_UNRELEASED_TEMPLATE {#v0-0-0-removed} ### Removed +* (build_data) Removed CONFIG_MODE from build data + ([#3793](https://github.com/bazel-contrib/rules_python/issues/3793)). * (coverage) Support for python 3.8 has been dropped from the bundled `coverage.py` wheel set, since coverage.py 7.6.2 dropped it. diff --git a/python/private/build_data_writer.ps1 b/python/private/build_data_writer.ps1 index 846399f194..0074e69d38 100644 --- a/python/private/build_data_writer.ps1 +++ b/python/private/build_data_writer.ps1 @@ -1,7 +1,6 @@ $OutputPath = $env:OUTPUT $Lines = @( "TARGET $env:TARGET", - "CONFIG_MODE $env:CONFIG_MODE", "STAMPED $env:STAMPED" ) diff --git a/python/private/build_data_writer.sh b/python/private/build_data_writer.sh index 7b88a582f5..4af98c6269 100755 --- a/python/private/build_data_writer.sh +++ b/python/private/build_data_writer.sh @@ -1,7 +1,6 @@ #!/bin/sh echo "TARGET $TARGET" >> $OUTPUT -echo "CONFIG_MODE $CONFIG_MODE" >> $OUTPUT echo "STAMPED $STAMPED" >> $OUTPUT if [ -n "$VERSION_FILE" ]; then cat "$VERSION_FILE" >> "$OUTPUT" diff --git a/python/private/py_executable.bzl b/python/private/py_executable.bzl index 965ed536cc..198ff9d548 100644 --- a/python/private/py_executable.bzl +++ b/python/private/py_executable.bzl @@ -1578,9 +1578,6 @@ def _write_build_data(ctx): executable = action_exe, arguments = [action_args], env = { - # Include config mode so that binaries can detect if they're - # being used as a build tool or not, allowing for runtime optimizations. - "CONFIG_MODE": "EXEC" if _is_tool_config(ctx) else "TARGET", "INFO_FILE": info_file.path if info_file else "", "OUTPUT": build_data.path, # Include this so it's explicit, otherwise, one has to detect diff --git a/tests/build_data/build_data_test.py b/tests/build_data/build_data_test.py index 874453e8e4..6be4e52c84 100644 --- a/tests/build_data/build_data_test.py +++ b/tests/build_data/build_data_test.py @@ -15,7 +15,6 @@ def test_target_build_data(self): self.assertIn("BUILD_USER ", build_data) self.assertIn("BUILD_TIMESTAMP ", build_data) self.assertIn("FORMATTED_DATE ", build_data) - self.assertIn("CONFIG_MODE TARGET", build_data) self.assertIn("STAMPED TRUE", build_data) def test_tool_build_data(self): @@ -25,7 +24,6 @@ def test_tool_build_data(self): build_data = fp.read() self.assertIn("STAMPED FALSE", build_data) - self.assertIn("CONFIG_MODE EXEC", build_data) unittest.main() From ef09f63261b1c349db5fadb016c2c5c9f8381bb2 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Sun, 31 May 2026 18:26:17 -0700 Subject: [PATCH 751/922] chore: tell agents to not amend or rebase PRs (#3804) Amending or rebasing open pull requests complicates the review process by obscuring change history and making it difficult to track incremental updates. This change adds an explicit rule to `AGENTS.md` under the "RULES TO ALWAYS FOLLOW AND NEVER IGNORE" section, instructing agents to avoid amending or rebasing once a pull request has been created. --- AGENTS.md | 1 + 1 file changed, 1 insertion(+) diff --git a/AGENTS.md b/AGENTS.md index 45df61fd25..73d254f27c 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -14,6 +14,7 @@ ALWAYS FOLLOW THESE RULES. NEVER VIOLATE THEM. Ask for user input and provide a justificaiton if trying to violate them. * NEVER run `bazel clean --expunge`. +* Once a PR is created, do not amend or rebase. ## Style and conventions From 94ccb8cae9191f4d785c168f8bf058e0f87feb90 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Sun, 31 May 2026 18:54:32 -0700 Subject: [PATCH 752/922] test(mocks): extract mock python extension helper (#3803) This mock module was extracted from bazel-contrib/rules_python#3802. It provides a helper for defining a mock module for the python bzlmod extension, which is standalone and generally useful for testing rules_python's bzlmod extension. --- tests/support/mocks/BUILD.bazel | 6 ++ tests/support/mocks/python_ext.bzl | 102 +++++++++++++++++++++++++++++ 2 files changed, 108 insertions(+) create mode 100644 tests/support/mocks/python_ext.bzl diff --git a/tests/support/mocks/BUILD.bazel b/tests/support/mocks/BUILD.bazel index 45949fc578..07bdb2bffc 100644 --- a/tests/support/mocks/BUILD.bazel +++ b/tests/support/mocks/BUILD.bazel @@ -10,4 +10,10 @@ bzl_library( srcs = ["mocks.bzl"], ) +bzl_library( + name = "python_ext_bzl", + srcs = ["python_ext.bzl"], + deps = [":mocks_bzl"], +) + mocks_test_suite(name = "mocks_tests") diff --git a/tests/support/mocks/python_ext.bzl b/tests/support/mocks/python_ext.bzl new file mode 100644 index 0000000000..f20a6c7263 --- /dev/null +++ b/tests/support/mocks/python_ext.bzl @@ -0,0 +1,102 @@ +"""Helper for defining a mock module for the python bzlmod extension.""" + +load(":mocks.bzl", "mocks") + +def _module(name = "rules_python", is_root = True, **tags): + """Creates a mock Bzlmod module struct with defaulted tag lists. + + Args: + name: The module name. + is_root: Whether this is the root module. + **tags: Lists of tag objects. + + Returns: + A mock module struct. + """ + defaulted_tags = { + "defaults": [], + "override": [], + "single_version_override": [], + "single_version_platform_override": [], + "toolchain": [], + } + defaulted_tags.update(tags) + return mocks.module(name = name, is_root = is_root, **defaulted_tags) + +def _override(**kwargs): + """Creates a mock python.override tag with default values.""" + attrs = { + "add_runtime_manifest_urls": [], + "add_target_settings": [], + "available_python_versions": [], + "base_url": "https://github.com/astral-sh/python-build-standalone/releases/download", + "ignore_root_user_error": True, + "minor_mapping": {}, + "register_all_versions": False, + "runtime_manifest_sha": "", + } + attrs.update(kwargs) + return mocks.tag(**attrs) + +def _defaults(**kwargs): + """Creates a mock python.defaults tag with default values.""" + attrs = { + "python_version": "", + "python_version_env": "", + } + attrs.update(kwargs) + return mocks.tag(**attrs) + +def _single_version_override(**kwargs): + """Creates a mock python.single_version_override tag with default values.""" + attrs = { + "distutils": None, + "distutils_content": "", + "patch_strip": 0, + "patches": [], + "python_version": "", + "sha256": {}, + "strip_prefix": "python", + "urls": [], + } + attrs.update(kwargs) + return mocks.tag(**attrs) + +def _single_version_platform_override(**kwargs): + """Creates a mock python.single_version_platform_override tag with default values.""" + attrs = { + "arch": "", + "coverage_tool": None, + "os_name": "", + "patch_strip": 0, + "patches": [], + "platform": "", + "python_version": "", + "sha256": "", + "strip_prefix": "python", + "target_compatible_with": [], + "target_settings": [], + "urls": [], + } + attrs.update(kwargs) + return mocks.tag(**attrs) + +def _toolchain(**kwargs): + """Creates a mock python.toolchain tag with default values.""" + attrs = { + "configure_coverage_tool": False, + "ignore_root_user_error": True, + "is_default": False, + "python_version": "", + } + attrs.update(kwargs) + return mocks.tag(**attrs) + +python_ext = struct( + defaults = _defaults, + module = _module, + override = _override, + single_version_override = _single_version_override, + single_version_platform_override = _single_version_platform_override, + toolchain = _toolchain, +) From 25ad57c85f0094889520486cd9a5738278fd0541 Mon Sep 17 00:00:00 2001 From: Ignas Anikevicius <240938+aignas@users.noreply.github.com> Date: Wed, 3 Jun 2026 22:47:40 +0900 Subject: [PATCH 753/922] fix(uv): allow environment setting for the update action (#3800) With this PR we allow users to set extra environment variables using `.bazelrc` and to ensure that we can pass UV extra parameters. This should enable users use this with private indexes in a `diff_test` usage scenario. Whilst at it, add integration tests to actually verify that passing works via env vars. Whilst at it also fix the Windows support for the `uv` lock rule. Fixes #3405 --- .../scripts/get_buildkite_results.py | 38 ++- .bazelignore | 1 + .bazelrc.deleted_packages | 1 + .gitattributes | 1 + .gitignore | 4 + CHANGELOG.md | 6 + python/uv/private/lock.bat | 14 +- python/uv/private/lock.bzl | 95 +++++- python/uv/private/lock_copier.py | 2 +- tests/integration/BUILD.bazel | 36 +++ tests/integration/integration_test.bzl | 7 +- tests/integration/uv_lock/.bazelrc | 5 + tests/integration/uv_lock/.bazelversion | 1 + tests/integration/uv_lock/BUILD.bazel | 26 ++ tests/integration/uv_lock/MODULE.bazel | 18 ++ tests/integration/uv_lock/README.md | 64 ++++ tests/integration/uv_lock/WORKSPACE | 0 tests/integration/uv_lock/requirements.in | 1 + tests/integration/uv_lock/requirements.txt | 5 + tests/integration/uv_lock/uv_runner.bzl | 30 ++ tests/integration/uv_lock_pypi_server.py | 148 +++++++++ tests/integration/uv_lock_test.py | 288 ++++++++++++++++++ tests/uv/lock/lock_run_test.py | 87 ++++-- tests/uv/lock/lock_tests.bzl | 21 +- 24 files changed, 825 insertions(+), 74 deletions(-) create mode 100644 tests/integration/uv_lock/.bazelrc create mode 100644 tests/integration/uv_lock/.bazelversion create mode 100644 tests/integration/uv_lock/BUILD.bazel create mode 100644 tests/integration/uv_lock/MODULE.bazel create mode 100644 tests/integration/uv_lock/README.md create mode 100644 tests/integration/uv_lock/WORKSPACE create mode 100644 tests/integration/uv_lock/requirements.in create mode 100644 tests/integration/uv_lock/requirements.txt create mode 100644 tests/integration/uv_lock/uv_runner.bzl create mode 100644 tests/integration/uv_lock_pypi_server.py create mode 100644 tests/integration/uv_lock_test.py diff --git a/.agents/skills/buildkite-get-results/scripts/get_buildkite_results.py b/.agents/skills/buildkite-get-results/scripts/get_buildkite_results.py index e1fc635cf6..06117fbd7c 100755 --- a/.agents/skills/buildkite-get-results/scripts/get_buildkite_results.py +++ b/.agents/skills/buildkite-get-results/scripts/get_buildkite_results.py @@ -94,24 +94,32 @@ def fetch_buildkite_data(build_url): def download_log(job_url, output_path): - # Construct raw log URL: job_url + "/raw" (Buildkite convention) - # job_url e.g. https://buildkite.com/org/pipeline/builds/14394#job-id - # Wait, the job['path'] gives /org/pipeline/builds/14394#job-id - # We want /org/pipeline/builds/14394/jobs/job-id/raw? No - # The clean URL for a job is https://buildkite.com/org/pipeline/builds/14394/jobs/job-id - # And raw log is https://buildkite.com/org/pipeline/builds/14394/jobs/job-id/raw - - # We have full_url e.g. https://buildkite.com/bazel/rules-python-python/builds/14394#019c5cf9-e3cf-468f-a7b1-8f9f5ad4b08c - # We need to transform it. + # job_url looks like: + # https://buildkite.com/bazel/rules-python-python/builds/15594#019e879b-... + # We need to transform it to: + # https://buildkite.com/organizations/bazel/pipelines/rules-python-python/builds/15594/jobs/{job_id}/download.txt if "#" in job_url: base, job_id = job_url.split("#") - # Ensure base doesn't end with / - if base.endswith("/"): - base = base[:-1] - - # Build raw URL - raw_url = f"{base}/jobs/{job_id}/raw" + base = base.rstrip("/") + + # Parse the path segments: https://buildkite.com/org/pipeline/builds/N + # Rebuild with the /organizations/org/pipelines/pipeline/ format which + # supports the /jobs/{id}/download.txt log URL without auth. + parts = base.split("/") + # parts = ["https:", "", "buildkite.com", "org", "pipeline", "builds", "N"] + if len(parts) >= 7 and parts[2] == "buildkite.com": + org = parts[3] + pipeline = parts[4] + build_num = parts[6] if len(parts) >= 7 else "" + raw_url = ( + f"https://buildkite.com/organizations/{org}" + f"/pipelines/{pipeline}" + f"/builds/{build_num}" + f"/jobs/{job_id}/download.txt" + ) + else: + raw_url = f"{base}/jobs/{job_id}/download.txt" else: print(f"Could not parse job URL for download: {job_url}", file=sys.stderr) return False diff --git a/.bazelignore b/.bazelignore index afd162998a..2cf1523aef 100644 --- a/.bazelignore +++ b/.bazelignore @@ -35,3 +35,4 @@ tests/integration/compile_pip_requirements/bazel-compile_pip_requirements tests/integration/local_toolchains/bazel-local_toolchains tests/integration/py_cc_toolchain_registered/bazel-py_cc_toolchain_registered tests/integration/toolchain_target_settings/bazel-module_under_test +tests/integration/uv_lock/bazel-uv_lock diff --git a/.bazelrc.deleted_packages b/.bazelrc.deleted_packages index f4ea8527f3..7256937a87 100644 --- a/.bazelrc.deleted_packages +++ b/.bazelrc.deleted_packages @@ -39,6 +39,7 @@ common --deleted_packages=tests/integration/pip_parse/empty common --deleted_packages=tests/integration/pip_parse_isolated common --deleted_packages=tests/integration/py_cc_toolchain_registered common --deleted_packages=tests/integration/toolchain_target_settings +common --deleted_packages=tests/integration/uv_lock common --deleted_packages=tests/modules/another_module common --deleted_packages=tests/modules/other common --deleted_packages=tests/modules/other/nspkg_delta diff --git a/.gitattributes b/.gitattributes index eae260e931..9905cbfacb 100644 --- a/.gitattributes +++ b/.gitattributes @@ -1,2 +1,3 @@ python/features.bzl export-subst tools/publish/*.txt linguist-generated=true +tests/uv/lock/testdata/requirements.txt text eol=lf diff --git a/.gitignore b/.gitignore index fb1b17e466..efce592aa0 100644 --- a/.gitignore +++ b/.gitignore @@ -54,3 +54,7 @@ user.bazelrc # MODULE.bazel.lock is ignored for now as per recommendation from upstream. # See https://github.com/bazelbuild/bazel/issues/20369 MODULE.bazel.lock + +# Buildkite logs +*Windows*.log + diff --git a/CHANGELOG.md b/CHANGELOG.md index e57e0969d3..33adfa7a60 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -103,6 +103,12 @@ END_UNRELEASED_TEMPLATE PyPI download and supports PyPI mirror implementations that do not support the root index functionality. Fixes ([#3769](https://github.com/bazel-contrib/rules_python/pull/3769)). +* (uv) allow user overwrite the build environment using `--action_env` to allow + setting authentication for the index URL. + ([#3405](https://github.com/bazel-contrib/rules_python/issues/3405)) +* (uv) fix the execution of the `uv pip compile` in the sandbox. Work + towards better supporting `uv` out of the box on our platforms. + ([#1975](https://github.com/bazel-contrib/rules_python/issues/1975)) {#v0-0-0-added} ### Added diff --git a/python/uv/private/lock.bat b/python/uv/private/lock.bat index 3954c10347..5190ddf9eb 100755 --- a/python/uv/private/lock.bat +++ b/python/uv/private/lock.bat @@ -1,7 +1,7 @@ -if defined BUILD_WORKSPACE_DIRECTORY ( - set "out=%BUILD_WORKSPACE_DIRECTORY%\{{src_out}}" -) else ( - exit /b 1 -) - -"{{args}}" --output-file "%out%" %* +if defined BUILD_WORKSPACE_DIRECTORY ( + set "out=%BUILD_WORKSPACE_DIRECTORY%\{{src_out}}" +) else ( + exit /b 1 +) + +"{{args}}" --output-file "%out%" %* diff --git a/python/uv/private/lock.bzl b/python/uv/private/lock.bzl index b007baf9c1..6f0b80af89 100644 --- a/python/uv/private/lock.bzl +++ b/python/uv/private/lock.bzl @@ -117,31 +117,96 @@ def _lock_impl(ctx): args.run_shell.add("--no-progress") args.run_shell.add("--quiet") + # Generate a wrapper script that copies the existing output (if any) and + # then runs uv. On POSIX, args are forwarded via exec "$@". On Windows, + # the full command line is embedded in the .bat file with backslash paths + # (CMD doesn't recognize forward slashes in executable paths). + if ctx.attr.is_windows: + ext = ".bat" + lines = ["@echo off"] + else: + ext = ".sh" + lines = ["#!/usr/bin/env bash", "set -euo pipefail"] + + python_path = getattr(python, "path", python) + if ctx.files.existing_output: - command = '{python} -c {python_cmd} && "$@"'.format( - python = getattr(python, "path", python), - python_cmd = shell.quote( - "from shutil import copy; copy(\"{src}\", \"{dst}\")".format( + python_cmd = "from shutil import copy; copy(\"{src}\", \"{dst}\")".format( + src = ctx.files.existing_output[0].path, + dst = output.path, + ) + if ctx.attr.is_windows: + # In batch files, use "" to escape internal double quotes. + lines.append( + "\"{py}\" -c \"from shutil import copy; copy(\"\"{src}\"\", \"\"{dst}\"\")\"".format( + py = python_path, src = ctx.files.existing_output[0].path, dst = output.path, ), + ) + else: + lines.append("{py} -c '{cmd}'".format( + py = python_path, + cmd = python_cmd, + )) + + if ctx.attr.is_windows: + # Build the command line with backslash paths for CMD. + # args.run_info has most args; add the output/progress/quiet + # args that were only added directly to args.run_shell. + def _quote(arg): + if hasattr(arg, "path"): + arg = arg.path.replace("/", "\\") + else: + arg = str(arg) + return '"' + arg.replace('"', '""') + '"' + + bat_args = args.run_info + [ + "--output-file", + output, + "--no-progress", + "--quiet", + ] + lines.append(" ".join([_quote(a) for a in bat_args])) + + # Normalize CRLF line endings in the output on Windows. + lines.append( + "\"{py}\" -c \"import pathlib;p=pathlib.Path(r\"\"{dst}\"\");p.write_bytes(p.read_bytes().replace(b'\\r\\n', b'\\n'))\"".format( + py = python_path, + dst = output.path, ), ) else: - command = '"$@"' + lines.append('exec "$@"') + + script = ctx.actions.declare_file(ctx.label.name + "_lock" + ext) + if ctx.attr.is_windows: + content = "\r\n".join(lines) + "\r\n" + else: + content = "\n".join(lines) + "\n" + ctx.actions.write(output = script, content = content, is_executable = True) srcs = srcs + ctx.files.build_constraints + ctx.files.constraints - ctx.actions.run_shell( - command = command, + ctx.actions.run( + executable = script, inputs = srcs + ctx.files.existing_output, mnemonic = "PyRequirementsLockUv", outputs = [output], - arguments = [args.run_shell], + # On Windows, the command line is embedded directly in the .bat + # script (with backslash paths). On POSIX, args are forwarded via + # exec "$@" in the .sh script. + arguments = [args.run_shell] if not ctx.attr.is_windows else [], tools = [ uv, python_files, + script, ], + # User reported being unable to add `--action_env` and get it to work. + # Without this flag. + # + # Ref: https://app.slack.com/client/TA4K1KQ87/CA306CEV6 + use_default_shell_env = True, progress_message = "Creating a requirements.txt with uv: %{label}", env = ctx.attr.env, ) @@ -205,6 +270,7 @@ modifications and the locking is not done from scratch. doc = "Public, see the docs in the macro.", default = True, ), + "is_windows": attr.bool(mandatory = True), "output": attr.string( doc = "Public, see the docs in the macro.", mandatory = True, @@ -241,7 +307,7 @@ The string to input for the 'uv pip compile'. def _lock_run_impl(ctx): if ctx.attr.is_windows: path_sep = "\\" - ext = ".exe" + ext = ".bat" else: path_sep = "/" ext = "" @@ -250,7 +316,12 @@ def _lock_run_impl(ctx): if hasattr(arg, "short_path"): arg = arg.short_path - return shell.quote(arg.replace("/", path_sep)) + arg = arg.replace("/", path_sep) + if ctx.attr.is_windows: + # On Windows, CMD uses double quotes for quoting, and internal + # double quotes are escaped by doubling them. + return '"' + arg.replace('"', '""') + '"' + return shell.quote(arg) info = ctx.attr.lock[_RunLockInfo] executable = ctx.actions.declare_file(ctx.label.name + ext) @@ -438,6 +509,10 @@ def lock( env = env, existing_output = maybe_out, generate_hashes = generate_hashes, + is_windows = select({ + "@platforms//os:windows": True, + "//conditions:default": False, + }), python_version = python_version, srcs = srcs, strip_extras = strip_extras, diff --git a/python/uv/private/lock_copier.py b/python/uv/private/lock_copier.py index bcc64c1661..8756fc4de6 100644 --- a/python/uv/private/lock_copier.py +++ b/python/uv/private/lock_copier.py @@ -55,7 +55,7 @@ def main(): "This must be either run as `bazel test` via a `native_test` or similar or via `bazel run`" ) - print(f"cp /{src} /{dst}") + print(f"cp /{src.as_posix()} /{dst}") build_workspace = Path(environ["BUILD_WORKSPACE_DIRECTORY"]) dst_real_path = build_workspace / dst diff --git a/tests/integration/BUILD.bazel b/tests/integration/BUILD.bazel index 9295cbb22f..a6027fc3d4 100644 --- a/tests/integration/BUILD.bazel +++ b/tests/integration/BUILD.bazel @@ -13,7 +13,9 @@ # limitations under the License. load("@rules_bazel_integration_test//bazel_integration_test:defs.bzl", "default_test_runner") +load("//python:py_binary.bzl", "py_binary") load("//python:py_library.bzl", "py_library") +load("//tests/support:support.bzl", "NOT_WINDOWS") load(":integration_test.bzl", "rules_python_integration_test") licenses(["notice"]) @@ -48,6 +50,7 @@ test_suite( tests = [ "bzlmod_lockfile_test_bazel_9.1.0", "local_toolchains_test_bazel_self", + "uv_lock_test_bazel_self", ], ) @@ -111,8 +114,41 @@ rules_python_integration_test( py_main = "toolchain_target_settings_test.py", ) +rules_python_integration_test( + name = "uv_lock_test", + py_deps = [ + "@pypiserver//pypiserver", + ":uv_lock_pypi_server_lib", + ], + py_main = "uv_lock_test.py", +) + py_library( name = "runner_lib", srcs = ["runner.py"], imports = ["../../"], ) + +py_library( + name = "uv_lock_pypi_server_lib", + srcs = ["uv_lock_pypi_server.py"], + imports = ["../../"], + # currently windows is not working due to + # https://github.com/pypiserver/pypiserver/blob/main/pypiserver/config.py#L123 + # + # class DEFAULTS: + # .... + # PACKAGE_DIRECTORIES = [pathlib.Path("~/packages").expanduser().resolve()] + # .... + # + # which is loaded through `__init__.py` even though it is not used and breaks because + # in a Windows sandbox one cannot resolve the home directory. + target_compatible_with = NOT_WINDOWS, + deps = ["@pypiserver//pypiserver"], +) + +py_binary( + name = "uv_lock_pypi_server", + srcs = ["uv_lock_pypi_server.py"], + deps = [":uv_lock_pypi_server_lib"], +) diff --git a/tests/integration/integration_test.bzl b/tests/integration/integration_test.bzl index 771976d037..f3d5cb6967 100644 --- a/tests/integration/integration_test.bzl +++ b/tests/integration/integration_test.bzl @@ -21,14 +21,14 @@ load( ) load("//python:py_test.bzl", "py_test") -def _test_runner(*, name, bazel_version, py_main, bzlmod): +def _test_runner(*, name, bazel_version, py_main, bzlmod, py_deps): if py_main: test_runner = "{}_bazel_{}_py_runner".format(name, bazel_version) py_test( name = test_runner, srcs = [py_main], main = py_main, - deps = [":runner_lib"], + deps = [":runner_lib"] + py_deps, # Hide from ... patterns; should only be run as part # of the bazel integration test tags = ["manual"], @@ -46,6 +46,7 @@ def rules_python_integration_test( bzlmod = True, tags = None, py_main = None, + py_deps = None, bazel_versions = None, **kwargs): """Runs a bazel-in-bazel integration test. @@ -60,6 +61,7 @@ def rules_python_integration_test( py_main: Optional `.py` file to run tests using. When specified, a python based test runner is used, and this source file is the main entry point and responsible for executing tests. + py_deps: Optional test runner deps to use for setup. bazel_versions: `list[str] | None`, the bazel versions to test. I not specified, defaults to all configured bazel versions. **kwargs: Passed to the upstream `bazel_integration_tests` rule. @@ -91,6 +93,7 @@ def rules_python_integration_test( name = name, bazel_version = bazel_version, py_main = py_main, + py_deps = py_deps or [], bzlmod = bzlmod, ) bazel_integration_test( diff --git a/tests/integration/uv_lock/.bazelrc b/tests/integration/uv_lock/.bazelrc new file mode 100644 index 0000000000..511f8a2413 --- /dev/null +++ b/tests/integration/uv_lock/.bazelrc @@ -0,0 +1,5 @@ +build --enable_runfiles +common --experimental_isolated_extension_usages +common --action_env=UV_EXTRA_INDEX_URL + +try-import %workspace%/user.bazelrc diff --git a/tests/integration/uv_lock/.bazelversion b/tests/integration/uv_lock/.bazelversion new file mode 100644 index 0000000000..47da986f86 --- /dev/null +++ b/tests/integration/uv_lock/.bazelversion @@ -0,0 +1 @@ +9.1.0 diff --git a/tests/integration/uv_lock/BUILD.bazel b/tests/integration/uv_lock/BUILD.bazel new file mode 100644 index 0000000000..da6482ba08 --- /dev/null +++ b/tests/integration/uv_lock/BUILD.bazel @@ -0,0 +1,26 @@ +load("@bazel_skylib//rules:diff_test.bzl", "diff_test") +load("@rules_python//python/uv:lock.bzl", "lock") +load(":uv_runner.bzl", "uv_runner") + +lock( + name = "requirements", + srcs = ["requirements.in"], + out = "requirements.txt", + tags = ["no-remote-exec"], +) + +uv_runner( + name = "uv", + is_windows = select({ + "@platforms//os:windows": True, + "//conditions:default": False, + }), + tags = ["manual"], +) + +diff_test( + name = "requirements_diff_test", + timeout = "short", + file1 = ":requirements", + file2 = ":requirements.txt", +) diff --git a/tests/integration/uv_lock/MODULE.bazel b/tests/integration/uv_lock/MODULE.bazel new file mode 100644 index 0000000000..0e0978ed36 --- /dev/null +++ b/tests/integration/uv_lock/MODULE.bazel @@ -0,0 +1,18 @@ +module(name = "uv_lock") + +bazel_dep(name = "bazel_skylib", version = "1.7.1") +bazel_dep(name = "platforms", version = "0.0.10") +bazel_dep(name = "rules_python") +local_path_override( + module_name = "rules_python", + path = "../../..", +) + +python = use_extension("@rules_python//python/extensions:python.bzl", "python") +python.toolchain(python_version = "3.13") + +uv = use_extension("@rules_python//python/uv:uv.bzl", "uv") +uv.configure(version = "0.11.2") +use_repo(uv, "uv") + +register_toolchains("@uv//:all") diff --git a/tests/integration/uv_lock/README.md b/tests/integration/uv_lock/README.md new file mode 100644 index 0000000000..cce142852c --- /dev/null +++ b/tests/integration/uv_lock/README.md @@ -0,0 +1,64 @@ +# uv_lock integration test workspace + +This directory is a self-contained Bazel workspace used by the +`//tests/integration:uv_lock_test` integration test. + +It demonstrates how to use the `lock()` macro from `@rules_python//python/uv` +to pin requirements with `uv pip compile`. + +## Targets + +| Target | Description | +|--------|-------------| +| `//:requirements` | Build action that produces the locked requirements file | +| `//:requirements.update` | Update the in-source `requirements.txt` via `bazel run` | +| `//:requirements.run` | Run `uv pip compile` with extra command-line args | +| `//:requirements_diff_test` | Diff test comparing the lock output to the in-source file | +| `//:uv` | The `uv` binary from the registered toolchain | + +## Workflow for debugging + +If you want to debug and play around, you can start the server and then run the uv lock command +manually. + +### Start the local PyPI server + +In a separate terminal, start the pypiserver that serves the `my-local-pkg` test wheel: + +```shell +bazel run //tests/integration:uv_lock_pypi_server [-- --no-auth] +``` + +The server prints the URL to use (with and without authentication) and the +SHA256 of the wheel. Pass `--no-auth` to allow anonymous access. + +### Lock the requirements + +With the server running, lock the requirements from this directory: + +```shell +cd tests/integration/uv_lock +bazel run //:requirements.update \ + --action_env=UV_EXTRA_INDEX_URL="" \ + --action_env=UV_CREDENTIALS_DIR= +``` + +The `` and `` values are printed by the pypi-server. + +### Verify the lock output matches the in-source file + +```shell +bazel test //:requirements_diff_test +``` + +## bazel-in-bazel Testing + +When iterating on changes to `lock.bzl`, the integration test can be run +directly from rules_python: + +```shell +bazel test //tests/integration:uv_lock_test_bazel_self \ + --config=fast-tests \ + --test_output=streamed \ + --test_filter= +``` diff --git a/tests/integration/uv_lock/WORKSPACE b/tests/integration/uv_lock/WORKSPACE new file mode 100644 index 0000000000..e69de29bb2 diff --git a/tests/integration/uv_lock/requirements.in b/tests/integration/uv_lock/requirements.in new file mode 100644 index 0000000000..fba55a7329 --- /dev/null +++ b/tests/integration/uv_lock/requirements.in @@ -0,0 +1 @@ +my-local-pkg==1.0.0 diff --git a/tests/integration/uv_lock/requirements.txt b/tests/integration/uv_lock/requirements.txt new file mode 100644 index 0000000000..52f5c757f5 --- /dev/null +++ b/tests/integration/uv_lock/requirements.txt @@ -0,0 +1,5 @@ +# This file was autogenerated by uv via the following command: +# bazel run //:requirements.update +my-local-pkg==1.0.0 \ + --hash=sha256:be24d5183a182e8da4465ae1b7e60324864d1a72866ec9aefa6aaf80a4529eb1 + # via -r requirements.in diff --git a/tests/integration/uv_lock/uv_runner.bzl b/tests/integration/uv_lock/uv_runner.bzl new file mode 100644 index 0000000000..3faa3c6fc0 --- /dev/null +++ b/tests/integration/uv_lock/uv_runner.bzl @@ -0,0 +1,30 @@ +"""A rule exposing the uv binary from the registered toolchain as an executable target. + +This allows running ``bazel run //:uv -- `` from the test workspace. +""" + +def _uv_runner_impl(ctx): + toolchain_info = ctx.toolchains["@rules_python//python/uv:uv_toolchain_type"] + original_uv_executable = toolchain_info.uv_toolchain_info.uv[DefaultInfo].files_to_run.executable + + ext = "" + if ctx.attr.is_windows: + ext = ".exe" + + uv_exe = ctx.actions.declare_file("uv" + ext) + ctx.actions.symlink(output = uv_exe, target_file = original_uv_executable) + + return DefaultInfo( + files = depset([uv_exe]), + executable = uv_exe, + runfiles = toolchain_info.default_info.default_runfiles, + ) + +uv_runner = rule( + implementation = _uv_runner_impl, + executable = True, + attrs = { + "is_windows": attr.bool(mandatory = True), + }, + toolchains = ["@rules_python//python/uv:uv_toolchain_type"], +) diff --git a/tests/integration/uv_lock_pypi_server.py b/tests/integration/uv_lock_pypi_server.py new file mode 100644 index 0000000000..0d940e7569 --- /dev/null +++ b/tests/integration/uv_lock_pypi_server.py @@ -0,0 +1,148 @@ +import argparse +import hashlib +import io +import os +import sys +import uuid +import zipfile +from wsgiref.simple_server import make_server + +from pypiserver import app_from_config, setup_routes_from_config +from pypiserver.config import Config + + +def _create_wheel_bytes(name, version): + pkg_name_normalized = name.replace("-", "_") + wheel_name = "{}-{}-py3-none-any.whl".format(pkg_name_normalized, version) + dist_info = "{}-{}.dist-info".format(pkg_name_normalized, version) + + metadata = ( + "Metadata-Version: 2.1\n" + "Name: {name}\n" + "Version: {version}\n" + "Summary: A test package\n" + ).format(name=pkg_name_normalized, version=version) + + wheel_file = ( + "Wheel-Version: 1.0\n" + "Generator: test\n" + "Root-Is-Purelib: true\n" + "Tag: py3-none-any\n" + ) + + record_entries = [ + "{}/__init__.py,".format(pkg_name_normalized), + "{}/METADATA,".format(dist_info), + "{}/WHEEL,".format(dist_info), + "{}/RECORD,".format(dist_info), + ] + + buf = io.BytesIO() + with zipfile.ZipFile(buf, "w", zipfile.ZIP_DEFLATED) as zf: + zf.writestr("{}/__init__.py".format(pkg_name_normalized), "# empty\n") + zf.writestr("{}/METADATA".format(dist_info), metadata) + zf.writestr("{}/WHEEL".format(dist_info), wheel_file) + zf.writestr("{}/RECORD".format(dist_info), "\n".join(record_entries)) + + wheel_data = buf.getvalue() + sha256 = hashlib.sha256(wheel_data).hexdigest() + return wheel_data, sha256, wheel_name + + +def main(): + parser = argparse.ArgumentParser( + description="Standalone pypiserver for uv_lock integration tests" + ) + parser.add_argument( + "--packages-dir", + type=str, + default=None, + help="Directory for the test wheels (default: $TEST_TMPDIR/pypi-server-packages)", + ) + parser.add_argument( + "--no-auth", + action="store_true", + default=False, + help="Disable authentication (allows anonymous access)", + ) + parser.add_argument( + "--port", + type=int, + default=0, + help="Port to listen on (0 = find free port)", + ) + parser.add_argument( + "--host", + default="localhost", + help="Host to bind to", + ) + args = parser.parse_args() + + if args.packages_dir is None: + sandbox_root = ( + os.environ.get("TEST_TMPDIR") or os.environ.get("TMPDIR") or "/tmp" + ) + args.packages_dir = os.path.join(sandbox_root, "pypi-server-packages") + packages_dir = args.packages_dir + os.makedirs(packages_dir, exist_ok=True) + + wheel_data, sha256, wheel_name = _create_wheel_bytes("my-local-pkg", "1.0.0") + wheel_path = os.path.join(packages_dir, wheel_name) + with open(wheel_path, "wb") as f: + f.write(wheel_data) + + print("Wheel: {}".format(wheel_path), flush=True) + print("SHA256: {}".format(sha256), flush=True) + + password = uuid.uuid4().hex + username = "testuser" + + if args.no_auth: + authenticate = [] + else: + authenticate = ["download", "list", "update"] + + config = Config.default_with_overrides( + roots=[packages_dir], + port=args.port, + host=args.host, + authenticate=authenticate, + password_file=None, + auther=lambda u, p: u == username and p == password, + disable_fallback=True, + fallback_url="", + server_method="wsgiref", + verbosity=0, + log_stream=None, + ) + app = app_from_config(config) + app = setup_routes_from_config(app, config) + + server = make_server(args.host, args.port, app) + port = server.server_address[1] + + base_url = "http://{}:{}".format(args.host, port) + auth_url = "http://{}:{}@{}:{}".format(username, password, args.host, port) + + print("\npypiserver listening on:\n", flush=True) + print(" URL (no auth): {}".format(base_url), flush=True) + print(" URL (auth): {}".format(auth_url), flush=True) + print("\nRequired dependency in requirements.in:", flush=True) + print(" my-local-pkg==1.0.0", flush=True) + print("\nTo use from the uv_lock test workspace, run:", flush=True) + print(" cd tests/integration/uv_lock", flush=True) + print(" bazel run //:requirements.update \\", flush=True) + print(' --action_env=UV_EXTRA_INDEX_URL="{}" \\'.format(auth_url), flush=True) + print(" --action_env=UV_CREDENTIALS_DIR=", flush=True) + print("\nPress Ctrl+C to stop the server.\n", flush=True) + + try: + server.serve_forever() + except KeyboardInterrupt: + print("\nShutting down...", flush=True) + server.shutdown() + return 0 + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/tests/integration/uv_lock_test.py b/tests/integration/uv_lock_test.py new file mode 100644 index 0000000000..8efd3cb84f --- /dev/null +++ b/tests/integration/uv_lock_test.py @@ -0,0 +1,288 @@ +import base64 +import hashlib +import os +import re +import threading +import time +import unittest +import uuid +from pathlib import Path +from urllib.error import URLError +from urllib.request import Request, urlopen +from wsgiref.simple_server import make_server + +from pypiserver import app_from_config, setup_routes_from_config +from pypiserver.config import Config + +from tests.integration import runner +from tests.integration.uv_lock_pypi_server import _create_wheel_bytes + + +def _make_server_on_free_port(app): + server = make_server("localhost", 0, app) + port = server.server_address[1] + return server, port + + +class UvLockIntegrationTest(runner.TestCase): + def setUp(self): + super().setUp() + + self.username = "testuser" + self.password = uuid.uuid4().hex + + self.dir = Path(os.environ["TEST_TMPDIR"]) + self.docroot = self.dir / "simple" + self.docroot.mkdir(exist_ok=True) + + self.wheel_data, self.wheel_sha256, wheel_name = _create_wheel_bytes( + "my-local-pkg", + "1.0.0", + ) + + packages_dir = self.docroot / "packages" + packages_dir.mkdir(exist_ok=True) + self.wheel_path = packages_dir / wheel_name + self.wheel_path.write_bytes(self.wheel_data) + + config = Config.default_with_overrides( + roots=[packages_dir], + port=0, + host="localhost", + authenticate=["download", "list", "update"], + password_file=None, + auther=lambda u, p: u == self.username and p == self.password, + disable_fallback=True, + fallback_url="", + server_method="wsgiref", + verbosity=0, + log_stream=None, + ) + app = app_from_config(config) + app = setup_routes_from_config(app, config) + + self._server, self.port = _make_server_on_free_port(app) + self.server_url = "http://localhost:{port}".format(port=self.port) + self.auth_url = "http://{user}:{passwd}@localhost:{port}".format( + user=self.username, + passwd=self.password, + port=self.port, + ) + + self._thread = threading.Thread(target=self._server.serve_forever) + self._thread.daemon = True + self._thread.start() + + interval = 0.1 + wait_seconds = 40 + for _ in range(int(wait_seconds / interval)): + try: + req = Request(self.server_url) + with urlopen(req, timeout=1) as response: + if response.status in (200, 401): + break + except (URLError, OSError): + pass + time.sleep(interval) + else: + raise RuntimeError( + "Could not start the server, waited for {}s".format(wait_seconds) + ) + + # Set a default value for UV_EXTRA_INDEX_URL in the bazel env so that + # the workspace .bazelrc `--action_env=UV_EXTRA_INDEX_URL` doesn't + # fail on Windows when the variable is unset in the client env. + self.bazel_env.setdefault("UV_EXTRA_INDEX_URL", "") + + # Use a sandbox-local credential store so credentials don't leak + # to the host system. + self.creds_dir = self.repo_root / ".uv-creds" + self.creds_dir.mkdir(parents=True, exist_ok=True) + self.bazel_env["UV_CREDENTIALS_DIR"] = str(self.creds_dir) + + # Log in to uv's credential store so `uv auth helper` can later + # serve the credentials to Bazel or uv itself. + self.run_bazel( + "run", + "//:uv", + "--", + "auth", + "login", + f"--username={self.username}", + f"--password={self.password}", + self.server_url, + ) + + def tearDown(self): + # Clear credentials from uv's credential store to ensure we are not + # logged into the service after the test. + self.run_bazel( + "run", + "//:uv", + "--", + "auth", + "logout", + self.server_url, + check=False, + ) + self._server.shutdown() + + def _assert_server_requires_auth(self): + req = Request(self.server_url + "/my-local-pkg/") + try: + urlopen(req, timeout=5) + self.fail("Expected 401 without auth") + except URLError: + pass + + def _auth_header(self): + return "Basic " + base64.b64encode( + "{user}:{passwd}".format( + user=self.username, + passwd=self.password, + ).encode("utf-8") + ).decode("utf-8") + + def _assert_simple_api_sha256(self): + auth_header = self._auth_header() + req = Request(self.server_url + "/simple/my-local-pkg/") + req.add_header("Authorization", auth_header) + resp = urlopen(req, timeout=5) + html = resp.read().decode("utf-8") + + match = re.search(r"#sha256=([a-f0-9]+)", html) + self.assertIsNotNone(match, "No sha256 found in simple API: {}".format(html)) + pypiserver_sha256 = match.group(1) + disk_sha256 = hashlib.sha256(self.wheel_path.read_bytes()).hexdigest() + self.assertEqual( + pypiserver_sha256, + disk_sha256, + "pypiserver hash {} != disk hash {}".format(pypiserver_sha256, disk_sha256), + ) + + def _creds_auth_args(self): + return [ + "--strategy=PyRequirementsLockUv=local", + "--action_env={key}={value}".format( + key="UV_CREDENTIALS_DIR", + value=str(self.creds_dir), + ), + "--action_env={key}={value}".format( + key="UV_EXTRA_INDEX_URL", + value=self.server_url, + ), + ] + + def _assert_lock_file(self, result): + self.assertEqual( + result.exit_code, + 0, + "Lock update failed:\n{}".format(result.describe()), + ) + lock_file = self.repo_root / "requirements.txt" + self.assertTrue(lock_file.exists(), "Lock file was not created") + contents = lock_file.read_text() + self.assertIn("my-local-pkg", contents) + self.assertIn("--hash=sha256:", contents) + + def test_lock_update_with_custom_index(self): + self._assert_server_requires_auth() + self._assert_simple_api_sha256() + + result = self.run_bazel( + "run", + "--action_env={key}={value}".format( + key="UV_EXTRA_INDEX_URL", + value=self.auth_url, + ), + "//:requirements.update", + ) + self._assert_lock_file(result) + + def test_update_with_credential_helper(self): + """Use a credential helper for authentication.""" + self._assert_server_requires_auth() + result = self.run_bazel( + "run", + *self._creds_auth_args(), + "//:requirements.update", + ) + self._assert_lock_file(result) + + def test_update_with_uv_auth_helper(self): + """Use the uv auth helper for authentication.""" + self._assert_server_requires_auth() + result = self.run_bazel( + "run", + *self._creds_auth_args(), + "//:requirements.update", + ) + self._assert_lock_file(result) + + def test_diff_test_with_requirements(self): + """Verify that ``diff_test`` can verify the generated lock file.""" + self._assert_server_requires_auth() + + # First generate the lock file + result = self.run_bazel( + "run", + *self._creds_auth_args(), + "//:requirements.update", + ) + self._assert_lock_file(result) + + # Copy the generated lock file to the expected location. The inner + # Bazel workspace is writable because it is a temporary copy created + # by the integration test framework. + generated = self.repo_root / "requirements.txt" + expected = self.repo_root / "requirements_expected.txt" + expected.write_text(generated.read_text()) + + # Run the diff_test: it builds the lock action, then compares the + # output to our expected file. + result = self.run_bazel( + "test", + *self._creds_auth_args(), + "//:requirements_diff_test", + ) + self.assertEqual( + result.exit_code, + 0, + "diff_test failed:\n{}".format(result.describe()), + ) + + def test_no_existing_requirements(self): + """Verify that ``bazel run`` and ``diff_test`` work when + ``requirements.txt`` does not yet exist.""" + self._assert_server_requires_auth() + + # Remove the existing lock file to simulate a fresh checkout + existing = self.repo_root / "requirements.txt" + existing.unlink() + self.assertFalse(existing.exists()) + + # Run ``requirements.update`` to generate the lock from scratch. The + # underlying lock rule will have no ``existing_output`` to copy, but + # ``uv pip compile`` should still produce the output. + result = self.run_bazel( + "run", + *self._creds_auth_args(), + "//:requirements.update", + ) + self._assert_lock_file(result) + + # diff_test should pass now that ``requirements.txt`` exists again + result = self.run_bazel( + "test", + *self._creds_auth_args(), + "//:requirements_diff_test", + ) + self.assertEqual( + result.exit_code, + 0, + "diff_test failed:\n{}".format(result.describe()), + ) + + +if __name__ == "__main__": + unittest.main() diff --git a/tests/uv/lock/lock_run_test.py b/tests/uv/lock/lock_run_test.py index f64cbdccec..e2508161d5 100644 --- a/tests/uv/lock/lock_run_test.py +++ b/tests/uv/lock/lock_run_test.py @@ -1,3 +1,4 @@ +import os import subprocess import tempfile import unittest @@ -9,15 +10,59 @@ def _relative_rpath(path: str) -> Path: - p = (Path("_main") / "tests" / "uv" / "lock" / path).as_posix() - rpath = rfiles.Rlocation(p) - if not rpath: - raise ValueError(f"Could not find file: {p}") - - return Path(rpath) + """Find file in runfiles, handling Windows .bat/.exe wrappers.""" + # On Windows, try executable extensions first to avoid matching symlink + # entries in the runfiles manifest that point to non-executable files + # (e.g. a Python source file instead of the .exe launcher). + exts = (".exe", ".bat", "") if os.name == "nt" else ("", ".exe", ".bat") + for ext in exts: + p = (Path("_main") / "tests" / "uv" / "lock" / (path + ext)).as_posix() + rpath = rfiles.Rlocation(p) + if rpath: + rp = Path(rpath) + if rp.exists(): + return rp + + # Fallback: look in runfiles directory directly (handles .bat wrappers on + # Windows where Rlocation may return a runfiles link that doesn't exist) + runfiles_dir = os.environ.get("RUNFILES_DIR") + if runfiles_dir: + exts = (".exe", ".bat", "") if os.name == "nt" else ("", ".bat", ".exe") + for ext in exts: + rp = Path(runfiles_dir, "_main", "tests", "uv", "lock", path + ext) + if rp.exists(): + return rp + + raise ValueError(f"Could not find file in runfiles: {path}") + + +def _run_binary(path: Path, **kwargs): + """Run a binary, handling Windows .bat files.""" + if os.name == "nt": + return subprocess.run( + ["cmd.exe", "/c", str(path)], + **kwargs, + ) + return subprocess.run(path, **kwargs) class LockTests(unittest.TestCase): + def _subprocess_env(self, workspace_dir: Path) -> dict[str, str]: + env = { + "BUILD_WORKSPACE_DIRECTORY": str(workspace_dir), + } + # Inherit specific env vars needed for finding runfiles on Windows + for key in ( + "PATH", + "RUNFILES_DIR", + "RUNFILES_MANIFEST_FILE", + "SYSTEMROOT", + "PATHEXT", + ): + if key in os.environ: + env[key] = os.environ[key] + return env + def test_requirements_updating_for_the_first_time(self): # Given copier_path = _relative_rpath("requirements_new_file.update") @@ -30,19 +75,18 @@ def test_requirements_updating_for_the_first_time(self): self.assertFalse( want_path.exists(), "The path should not exist after the test" ) - output = subprocess.run( + output = _run_binary( copier_path, capture_output=True, - env={ - "BUILD_WORKSPACE_DIRECTORY": f"{workspace_dir}", - }, + env=self._subprocess_env(workspace_dir), ) # Then self.assertEqual(0, output.returncode, output.stderr) + stdout = output.stdout.decode("utf-8").replace("\\", "/") self.assertIn( "cp /tests/uv/lock/requirements_new_file", - output.stdout.decode("utf-8"), + stdout, ) self.assertTrue(want_path.exists(), "The path should exist after the test") self.assertNotEqual(want_path.read_text(), "") @@ -69,19 +113,18 @@ def test_requirements_updating(self): want_text + "\n\n" ) # Write something else to see that it is restored - output = subprocess.run( + output = _run_binary( copier_path, capture_output=True, - env={ - "BUILD_WORKSPACE_DIRECTORY": f"{workspace_dir}", - }, + env=self._subprocess_env(workspace_dir), ) # Then self.assertEqual(0, output.returncode) + stdout = output.stdout.decode("utf-8").replace("\\", "/") self.assertIn( "cp /tests/uv/lock/requirements", - output.stdout.decode("utf-8"), + stdout, ) self.assertEqual(want_path.read_text(), want_text) @@ -100,12 +143,10 @@ def test_requirements_run_on_the_first_time(self): self.assertFalse( want_path.exists(), "The path should not exist after the test" ) - output = subprocess.run( + output = _run_binary( copier_path, capture_output=True, - env={ - "BUILD_WORKSPACE_DIRECTORY": f"{workspace_dir}", - }, + env=self._subprocess_env(workspace_dir), ) # Then @@ -141,12 +182,10 @@ def test_requirements_run(self): want_text + "\n\n" ) # Write something else to see that it is restored - output = subprocess.run( + output = _run_binary( copier_path, capture_output=True, - env={ - "BUILD_WORKSPACE_DIRECTORY": f"{workspace_dir}", - }, + env=self._subprocess_env(workspace_dir), ) # Then diff --git a/tests/uv/lock/lock_tests.bzl b/tests/uv/lock/lock_tests.bzl index 1eb5b1d903..bcaed95b53 100644 --- a/tests/uv/lock/lock_tests.bzl +++ b/tests/uv/lock/lock_tests.bzl @@ -14,7 +14,7 @@ "" -load("@bazel_skylib//rules:native_binary.bzl", "native_test") +load("@bazel_skylib//rules:diff_test.bzl", "diff_test") load("//python/uv:lock.bzl", "lock") load("//tests/support:py_reconfig.bzl", "py_reconfig_test") @@ -77,23 +77,14 @@ def lock_test_suite(name): # `--index-url`. "no-remote-exec", ], - # FIXME @aignas 2025-03-19: It seems that currently: - # 1. The Windows runners are not compatible with the `uv` Windows binaries. - # 2. The Python launcher is having trouble launching scripts from within the Python test. - target_compatible_with = select({ - "@platforms//os:windows": ["@platforms//:incompatible"], - "//conditions:default": [], - }), ) - # document and check that this actually works - native_test( + # Document and check that the action output matches the in-source file. + diff_test( name = "requirements_test", - src = ":requirements.update", - target_compatible_with = select({ - "@platforms//os:windows": ["@platforms//:incompatible"], - "//conditions:default": [], - }), + timeout = "short", + file1 = ":requirements", + file2 = "testdata/requirements.txt", ) native.test_suite( From c00bbaf6f7d26f9eeeb6d9887ce24f5dc37bad5b Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Sun, 7 Jun 2026 18:21:34 -0700 Subject: [PATCH 754/922] refactor: use python_ext mock helpers in python_tests.bzl (#3813) Refactored tests/python/python_tests.bzl to use the shared python_ext mock helpers instead of standalone mock helpers. --- tests/python/python_tests.bzl | 443 +++++++++++++++++----------------- 1 file changed, 216 insertions(+), 227 deletions(-) diff --git a/tests/python/python_tests.bzl b/tests/python/python_tests.bzl index 60e7311c00..5db74265be 100644 --- a/tests/python/python_tests.bzl +++ b/tests/python/python_tests.bzl @@ -19,111 +19,19 @@ load("@rules_testing//lib:test_suite.bzl", "test_suite") load("//python/private:python.bzl", "parse_modules") # buildifier: disable=bzl-visibility load("//python/private:repo_utils.bzl", "repo_utils") # buildifier: disable=bzl-visibility load("//tests/support/mocks:mocks.bzl", "mocks") +load("//tests/support/mocks:python_ext.bzl", "python_ext") _tests = [] -def _mod(*, name, defaults = [], toolchain = [], override = [], single_version_override = [], single_version_platform_override = [], is_root = False): - return mocks.module( - name, - is_root = is_root, - defaults = defaults, - toolchain = toolchain, - override = override, - single_version_override = single_version_override, - single_version_platform_override = single_version_platform_override, - ) - -def _defaults(python_version = None, python_version_env = None, python_version_file = None): - return struct( - python_version = python_version, - python_version_env = python_version_env, - python_version_file = python_version_file, - ) - -def _toolchain(python_version, *, is_default = False, **kwargs): - return struct( - is_default = is_default, - python_version = python_version, - **kwargs - ) - -def _override( - auth_patterns = {}, - available_python_versions = [], - base_url = "", - minor_mapping = {}, - netrc = "", - register_all_versions = False, - add_target_settings = []): - return struct( - auth_patterns = auth_patterns, - available_python_versions = available_python_versions, - base_url = base_url, - minor_mapping = minor_mapping, - netrc = netrc, - register_all_versions = register_all_versions, - add_target_settings = add_target_settings, - ) - def _rules_python_module(is_root = False): """A mock of what the real rules_python MODULE.bazel looks like.""" - return _mod( + return python_ext.module( name = "rules_python", - defaults = [_defaults(python_version = "3.11")], - toolchain = [_toolchain("3.11")], + defaults = [python_ext.defaults(python_version = "3.11")], + toolchain = [python_ext.toolchain(python_version = "3.11")], is_root = is_root, ) -def _single_version_override( - python_version = "", - sha256 = {}, - urls = [], - patch_strip = 0, - patches = [], - strip_prefix = "python", - distutils_content = "", - distutils = None): - if not python_version: - fail("missing mandatory args: python_version ({})".format(python_version)) - - return struct( - python_version = python_version, - sha256 = sha256, - urls = urls, - patch_strip = patch_strip, - patches = patches, - strip_prefix = strip_prefix, - distutils_content = distutils_content, - distutils = distutils, - ) - -def _single_version_platform_override( - coverage_tool = None, - patch_strip = 0, - patches = [], - platform = "", - python_version = "", - sha256 = "", - strip_prefix = "python", - urls = []): - if not platform or not python_version: - fail("missing mandatory args: platform ({}) and python_version ({})".format(platform, python_version)) - - return struct( - sha256 = sha256, - urls = urls, - strip_prefix = strip_prefix, - platform = platform, - coverage_tool = coverage_tool, - python_version = python_version, - patch_strip = patch_strip, - patches = patches, - target_compatible_with = [], - target_settings = [], - os_name = "", - arch = "", - ) - def _test_default_from_rules_python_when_rules_python_is_root(env): """Verify that rules_python (as root module) default is applied.""" py = parse_modules( @@ -178,7 +86,11 @@ def _test_default_with_patch_version(env): py = parse_modules( module_ctx = mocks.mctx( modules = [ - _mod(name = "alpha", toolchain = [_toolchain("3.11.2")], is_root = True), + python_ext.module( + name = "alpha", + is_root = True, + toolchain = [python_ext.toolchain(python_version = "3.11.2")], + ), _rules_python_module(is_root = False), ], ), @@ -199,18 +111,21 @@ _tests.append(_test_default_with_patch_version) def _test_toolchain_ordering(env): py = parse_modules( module_ctx = mocks.mctx( - _mod( + python_ext.module( name = "my_module", + is_root = True, toolchain = [ - _toolchain("3.10"), - _toolchain("3.10.15"), - _toolchain(MINOR_MAPPING["3.10"]), - _toolchain("3.10.13"), - _toolchain("3.11.1"), - _toolchain("3.11.10"), - _toolchain(MINOR_MAPPING["3.11"], is_default = True), + python_ext.toolchain(python_version = "3.10"), + python_ext.toolchain(python_version = "3.10.15"), + python_ext.toolchain(python_version = MINOR_MAPPING["3.10"]), + python_ext.toolchain(python_version = "3.10.13"), + python_ext.toolchain(python_version = "3.11.1"), + python_ext.toolchain(python_version = "3.11.10"), + python_ext.toolchain( + python_version = MINOR_MAPPING["3.11"], + is_default = True, + ), ], - is_root = True, ), _rules_python_module(), ), @@ -246,11 +161,15 @@ _tests.append(_test_toolchain_ordering) def _test_default_from_defaults(env): py = parse_modules( module_ctx = mocks.mctx( - _mod( + python_ext.module( name = "my_root_module", - defaults = [_defaults(python_version = "3.11")], - toolchain = [_toolchain("3.10"), _toolchain("3.11"), _toolchain("3.12")], + defaults = [python_ext.defaults(python_version = "3.11")], is_root = True, + toolchain = [ + python_ext.toolchain(python_version = "3.10"), + python_ext.toolchain(python_version = "3.11"), + python_ext.toolchain(python_version = "3.12"), + ], ), ), logger = repo_utils.logger(verbosity_level = 0, name = "python"), @@ -273,11 +192,20 @@ _tests.append(_test_default_from_defaults) def _test_default_from_defaults_env(env): py = parse_modules( module_ctx = mocks.mctx( - _mod( + python_ext.module( name = "my_root_module", - defaults = [_defaults(python_version = "3.11", python_version_env = "PYENV_VERSION")], - toolchain = [_toolchain("3.10"), _toolchain("3.11"), _toolchain("3.12")], + defaults = [ + python_ext.defaults( + python_version = "3.11", + python_version_env = "PYENV_VERSION", + ), + ], is_root = True, + toolchain = [ + python_ext.toolchain(python_version = "3.10"), + python_ext.toolchain(python_version = "3.11"), + python_ext.toolchain(python_version = "3.12"), + ], ), environ = {"PYENV_VERSION": "3.12"}, ), @@ -301,11 +229,19 @@ _tests.append(_test_default_from_defaults_env) def _test_default_from_defaults_file(env): py = parse_modules( module_ctx = mocks.mctx( - _mod( + python_ext.module( name = "my_root_module", - defaults = [_defaults(python_version_file = "@@//:.python-version")], - toolchain = [_toolchain("3.10"), _toolchain("3.11"), _toolchain("3.12")], + defaults = [ + python_ext.defaults( + python_version_file = "@@//:.python-version", + ), + ], is_root = True, + toolchain = [ + python_ext.toolchain(python_version = "3.10"), + python_ext.toolchain(python_version = "3.11"), + python_ext.toolchain(python_version = "3.12"), + ], ), mock_files = {"@@//:.python-version": "3.12\n"}, ), @@ -329,10 +265,10 @@ _tests.append(_test_default_from_defaults_file) def _test_default_from_single_toolchain(env): py = parse_modules( module_ctx = mocks.mctx( - _mod( + python_ext.module( name = "my_root_module", - toolchain = [_toolchain("3.12")], is_root = True, + toolchain = [python_ext.toolchain(python_version = "3.12")], ), _rules_python_module(), ), @@ -345,14 +281,14 @@ _tests.append(_test_default_from_single_toolchain) def _test_defaults_overrides_single_toolchain(env): py = parse_modules( module_ctx = mocks.mctx( - _mod( + python_ext.module( name = "my_root_module", defaults = [ # This relies on rules_python registering 3.11 - _defaults(python_version = "3.11"), + python_ext.defaults(python_version = "3.11"), ], - toolchain = [_toolchain("3.12")], is_root = True, + toolchain = [python_ext.toolchain(python_version = "3.12")], ), _rules_python_module(), ), @@ -365,14 +301,17 @@ _tests.append(_test_defaults_overrides_single_toolchain) def _test_defaults_overrides_toolchains_setting_is_default(env): py = parse_modules( module_ctx = mocks.mctx( - _mod( + python_ext.module( name = "my_root_module", - defaults = [_defaults(python_version = "3.13")], + defaults = [python_ext.defaults(python_version = "3.13")], + is_root = True, toolchain = [ - _toolchain("3.13"), - _toolchain("3.12", is_default = True), + python_ext.toolchain(python_version = "3.13"), + python_ext.toolchain( + python_version = "3.12", + is_default = True, + ), ], - is_root = True, ), _rules_python_module(), ), @@ -386,8 +325,21 @@ def _test_first_occurance_of_the_toolchain_wins(env): py = parse_modules( module_ctx = mocks.mctx( modules = [ - _mod(name = "my_module", is_root = True, toolchain = [_toolchain("3.12")]), - _mod(name = "some_module", toolchain = [_toolchain("3.12", configure_coverage_tool = True)]), + python_ext.module( + name = "my_module", + is_root = True, + toolchain = [python_ext.toolchain(python_version = "3.12")], + ), + python_ext.module( + name = "some_module", + is_root = False, + toolchain = [ + python_ext.toolchain( + python_version = "3.12", + configure_coverage_tool = True, + ), + ], + ), _rules_python_module(), ], environ = { @@ -428,16 +380,16 @@ _tests.append(_test_first_occurance_of_the_toolchain_wins) def _test_auth_overrides(env): py = parse_modules( module_ctx = mocks.mctx( - _mod( + python_ext.module( name = "my_module", - toolchain = [_toolchain("3.12")], + is_root = True, override = [ - _override( - netrc = "/my/netrc", + python_ext.override( auth_patterns = {"foo": "bar"}, + netrc = "/my/netrc", ), ], - is_root = True, + toolchain = [python_ext.toolchain(python_version = "3.12")], ), _rules_python_module(), ), @@ -470,17 +422,17 @@ _tests.append(_test_auth_overrides) def _test_add_target_settings(env): py = parse_modules( module_ctx = mocks.mctx( - _mod( + python_ext.module( name = "my_module", - toolchain = [_toolchain("3.12")], + is_root = True, override = [ - _override( + python_ext.override( add_target_settings = [ "@@//my:custom_setting", ], ), ], - is_root = True, + toolchain = [python_ext.toolchain(python_version = "3.12")], ), _rules_python_module(), ), @@ -496,45 +448,50 @@ _tests.append(_test_add_target_settings) def _test_add_new_version(env): py = parse_modules( module_ctx = mocks.mctx( - _mod( + python_ext.module( name = "my_module", is_root = True, - toolchain = [_toolchain("3.13")], + override = [ + python_ext.override( + available_python_versions = [ + "3.12.4", + "3.13.0", + "3.13.1", + "3.13.99", + ], + base_url = "", + minor_mapping = { + "3.13": "3.13.99", + }, + ), + ], single_version_override = [ - _single_version_override( + python_ext.single_version_override( + distutils = None, + distutils_content = "", + patch_strip = 0, + patches = [], python_version = "3.13.0", sha256 = { "aarch64-unknown-linux-gnu": "deadbeef", }, - urls = ["example.org"], - patch_strip = 0, - patches = [], strip_prefix = "prefix", - distutils_content = "", - distutils = None, + urls = ["example.org"], ), ], single_version_platform_override = [ - _single_version_platform_override( - sha256 = "deadb00f", - urls = ["something.org", "else.org"], - strip_prefix = "python", - platform = "aarch64-unknown-linux-gnu", + python_ext.single_version_platform_override( coverage_tool = "specific_cov_tool", - python_version = "3.13.99", patch_strip = 2, patches = ["specific-patch.txt"], + platform = "aarch64-unknown-linux-gnu", + python_version = "3.13.99", + sha256 = "deadb00f", + strip_prefix = "python", + urls = ["something.org", "else.org"], ), ], - override = [ - _override( - base_url = "", - available_python_versions = ["3.12.4", "3.13.0", "3.13.1", "3.13.99"], - minor_mapping = { - "3.13": "3.13.99", - }, - ), - ], + toolchain = [python_ext.toolchain(python_version = "3.13")], ), ), logger = repo_utils.logger(verbosity_level = 0, name = "python"), @@ -577,12 +534,23 @@ _tests.append(_test_add_new_version) def _test_register_all_versions(env): py = parse_modules( module_ctx = mocks.mctx( - _mod( + python_ext.module( name = "my_module", is_root = True, - toolchain = [_toolchain("3.13")], + override = [ + python_ext.override( + available_python_versions = [ + "3.12.4", + "3.13.0", + "3.13.1", + "3.13.99", + ], + base_url = "", + register_all_versions = True, + ), + ], single_version_override = [ - _single_version_override( + python_ext.single_version_override( python_version = "3.13.0", sha256 = { "aarch64-unknown-linux-gnu": "deadbeef", @@ -591,20 +559,14 @@ def _test_register_all_versions(env): ), ], single_version_platform_override = [ - _single_version_platform_override( - sha256 = "deadb00f", - urls = ["something.org"], + python_ext.single_version_platform_override( platform = "aarch64-unknown-linux-gnu", python_version = "3.13.99", + sha256 = "deadb00f", + urls = ["something.org"], ), ], - override = [ - _override( - base_url = "", - available_python_versions = ["3.12.4", "3.13.0", "3.13.1", "3.13.99"], - register_all_versions = True, - ), - ], + toolchain = [python_ext.toolchain(python_version = "3.13")], ), ), logger = repo_utils.logger(verbosity_level = 0, name = "python"), @@ -643,16 +605,25 @@ _tests.append(_test_register_all_versions) def _test_ignore_unsupported_versions(env): py = parse_modules( module_ctx = mocks.mctx( - _mod( + python_ext.module( name = "my_module", is_root = True, - toolchain = [ - _toolchain("3.11"), - _toolchain("3.12"), - _toolchain("3.13", is_default = True), + override = [ + python_ext.override( + available_python_versions = [ + "3.12.4", + "3.13.0", + "3.13.1", + ], + base_url = "", + minor_mapping = { + "3.12": "3.12.4", + "3.13": "3.13.1", + }, + ), ], single_version_override = [ - _single_version_override( + python_ext.single_version_override( python_version = "3.13.0", sha256 = { "aarch64-unknown-linux-gnu": "deadbeef", @@ -661,21 +632,19 @@ def _test_ignore_unsupported_versions(env): ), ], single_version_platform_override = [ - _single_version_platform_override( - sha256 = "deadb00f", - urls = ["something.org"], + python_ext.single_version_platform_override( platform = "aarch64-unknown-linux-gnu", python_version = "3.13.99", + sha256 = "deadb00f", + urls = ["something.org"], ), ], - override = [ - _override( - base_url = "", - available_python_versions = ["3.12.4", "3.13.0", "3.13.1"], - minor_mapping = { - "3.12": "3.12.4", - "3.13": "3.13.1", - }, + toolchain = [ + python_ext.toolchain(python_version = "3.11"), + python_ext.toolchain(python_version = "3.12"), + python_ext.toolchain( + python_version = "3.13", + is_default = True, ), ], ), @@ -714,46 +683,46 @@ _tests.append(_test_ignore_unsupported_versions) def _test_add_patches(env): py = parse_modules( module_ctx = mocks.mctx( - _mod( + python_ext.module( name = "my_module", is_root = True, - toolchain = [_toolchain("3.13")], + override = [ + python_ext.override( + available_python_versions = ["3.13.0"], + base_url = "", + minor_mapping = { + "3.13": "3.13.0", + }, + ), + ], single_version_override = [ - _single_version_override( + python_ext.single_version_override( + distutils = None, + distutils_content = "", + patch_strip = 1, + patches = ["common.txt"], python_version = "3.13.0", sha256 = { "aarch64-apple-darwin": "deadbeef", "aarch64-unknown-linux-gnu": "deadbeef", }, - urls = ["example.org"], - patch_strip = 1, - patches = ["common.txt"], strip_prefix = "prefix", - distutils_content = "", - distutils = None, + urls = ["example.org"], ), ], single_version_platform_override = [ - _single_version_platform_override( - sha256 = "deadb00f", - urls = ["something.org", "else.org"], - strip_prefix = "python", - platform = "aarch64-unknown-linux-gnu", + python_ext.single_version_platform_override( coverage_tool = "specific_cov_tool", - python_version = "3.13.0", patch_strip = 2, patches = ["specific-patch.txt"], + platform = "aarch64-unknown-linux-gnu", + python_version = "3.13.0", + sha256 = "deadb00f", + strip_prefix = "python", + urls = ["something.org", "else.org"], ), ], - override = [ - _override( - base_url = "", - available_python_versions = ["3.13.0"], - minor_mapping = { - "3.13": "3.13.0", - }, - ), - ], + toolchain = [python_ext.toolchain(python_version = "3.13")], ), ), logger = repo_utils.logger(verbosity_level = 0, name = "python"), @@ -793,14 +762,14 @@ def _test_fail_two_overrides(env): errors = [] parse_modules( module_ctx = mocks.mctx( - _mod( + python_ext.module( name = "my_module", is_root = True, - toolchain = [_toolchain("3.13")], override = [ - _override(base_url = "foo"), - _override(base_url = "bar"), + python_ext.override(base_url = "foo"), + python_ext.override(base_url = "bar"), ], + toolchain = [python_ext.toolchain(python_version = "3.13")], ), ), _fail = errors.append, @@ -816,8 +785,14 @@ def _test_single_version_override_errors(env): for test in [ struct( overrides = [ - _single_version_override(python_version = "3.12.4", distutils_content = "foo"), - _single_version_override(python_version = "3.12.4", distutils_content = "foo"), + python_ext.single_version_override( + distutils_content = "foo", + python_version = "3.12.4", + ), + python_ext.single_version_override( + distutils_content = "foo", + python_version = "3.12.4", + ), ], want_error = "Only a single 'python.single_version_override' can be present for '3.12.4'", ), @@ -825,11 +800,11 @@ def _test_single_version_override_errors(env): errors = [] parse_modules( module_ctx = mocks.mctx( - _mod( + python_ext.module( name = "my_module", is_root = True, - toolchain = [_toolchain("3.13")], single_version_override = test.overrides, + toolchain = [python_ext.toolchain(python_version = "3.13")], ), ), _fail = errors.append, @@ -843,20 +818,34 @@ def _test_single_version_platform_override_errors(env): for test in [ struct( overrides = [ - _single_version_platform_override(python_version = "3.12.4", platform = "foo", coverage_tool = "foo"), - _single_version_platform_override(python_version = "3.12.4", platform = "foo", coverage_tool = "foo"), + python_ext.single_version_platform_override( + coverage_tool = "foo", + platform = "foo", + python_version = "3.12.4", + ), + python_ext.single_version_platform_override( + coverage_tool = "foo", + platform = "foo", + python_version = "3.12.4", + ), ], want_error = "Only a single 'python.single_version_platform_override' can be present for '(\"3.12.4\", \"foo\")'", ), struct( overrides = [ - _single_version_platform_override(python_version = "3.12", platform = "foo"), + python_ext.single_version_platform_override( + platform = "foo", + python_version = "3.12", + ), ], want_error = "The 'python_version' attribute needs to specify the full version in at least 'X.Y.Z' format, got: '3.12'", ), struct( overrides = [ - _single_version_platform_override(python_version = "foo", platform = "foo"), + python_ext.single_version_platform_override( + platform = "foo", + python_version = "foo", + ), ], want_error = "Failed to parse PEP 440 version identifier 'foo'. Parse error at 'foo'", ), @@ -864,11 +853,11 @@ def _test_single_version_platform_override_errors(env): errors = [] parse_modules( module_ctx = mocks.mctx( - _mod( + python_ext.module( name = "my_module", - toolchain = [_toolchain("3.13")], - single_version_platform_override = test.overrides, is_root = True, + single_version_platform_override = test.overrides, + toolchain = [python_ext.toolchain(python_version = "3.13")], ), ), _fail = lambda *a: errors.append(" ".join(a)), From dcb7dfa92c8019661730ff25f364b70b8eb3de8c Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Sun, 7 Jun 2026 18:23:38 -0700 Subject: [PATCH 755/922] feat: add //command_line_option:extra_toolchains pseudo-flag (#3810) Add support for transitioning on the //command_line_option:extra_toolchains built-in flag using py_binary.config_settings. Note that, unlike the normal Bazel built-in flag, it must be specified as a simple comma-separated string when set using config_settings, which is then parsed as a CSV string. This is to make it easier to, on a per-target basis, such as with a zipapp or wheels, change the Python toolchains used. --- command_line_option/BUILD.bazel | 5 ++++ .../rules_python/command_line_option/index.md | 23 +++++++++++++++++++ python/features.bzl | 1 + python/private/attributes.bzl | 9 ++++++-- python/private/common_labels.bzl | 1 + python/private/transition_labels.bzl | 1 + tests/base_rules/py_executable_base_tests.bzl | 23 +++++++++++++++++++ .../bzlmod_lockfile/MODULE.bazel.lock | 2 +- 8 files changed, 62 insertions(+), 3 deletions(-) diff --git a/command_line_option/BUILD.bazel b/command_line_option/BUILD.bazel index a4d5f0f80d..15c22f360b 100644 --- a/command_line_option/BUILD.bazel +++ b/command_line_option/BUILD.bazel @@ -18,6 +18,11 @@ alias( actual = "//python:none", ) +alias( + name = "extra_toolchains", + actual = "//python:none", +) + filegroup( name = "distribution", srcs = glob(["**"]), diff --git a/docs/api/rules_python/command_line_option/index.md b/docs/api/rules_python/command_line_option/index.md index a7dfa340eb..02d29206f5 100644 --- a/docs/api/rules_python/command_line_option/index.md +++ b/docs/api/rules_python/command_line_option/index.md @@ -51,3 +51,26 @@ See the [Bazel documentation for --enable_runfiles](https://bazel.build/referenc The special value `INHERIT` can be specified to use the existing flag value. ::: + +## extra_toolchains + +:::{bzl:target} extra_toolchains + +Special target for the Bazel-builtin `//command_line_option:extra_toolchains` +flag. + +See the [Bazel documentation for --extra_toolchains](https://bazel.build/reference/command-line-reference#flag--extra_toolchains). + +The special value `INHERIT` can be specified to use the existing flag value. + +:::{note} +Unlike the normal Bazel built-in flag, which accepts a list of labels, this +pseudo-flag must be specified as a single, comma-separated string when set +using the `config_settings` attribute. For example: + +```python +"//command_line_option:extra_toolchains": "//my/tc1,//my/tc2" +``` +::: +::: + diff --git a/python/features.bzl b/python/features.bzl index 26acc8c5fe..a2cb95f1ee 100644 --- a/python/features.bzl +++ b/python/features.bzl @@ -86,6 +86,7 @@ def _features_typedef(): _TARGETS = { "//command_line_option:build_runfile_links": True, "//command_line_option:enable_runfiles": True, + "//command_line_option:extra_toolchains": True, "//python/cc:current_py_cc_headers_abi3": True, } diff --git a/python/private/attributes.bzl b/python/private/attributes.bzl index beaa7a6a73..ad741d078d 100644 --- a/python/private/attributes.bzl +++ b/python/private/attributes.bzl @@ -446,10 +446,15 @@ def apply_config_settings_attr(settings, attr): if key.package == "command_line_option": if value == "INHERIT": continue - else: - str_key = "//command_line_option:" + key.name + str_key = "//command_line_option:" + key.name + if key.name == "extra_toolchains": + if value == "": + value = [] + else: + value = [v.strip() for v in value.split(",") if v.strip()] else: str_key = str(key) + settings[str_key] = value return settings diff --git a/python/private/common_labels.bzl b/python/private/common_labels.bzl index ff4e7e1dad..a83ba2b462 100644 --- a/python/private/common_labels.bzl +++ b/python/private/common_labels.bzl @@ -14,6 +14,7 @@ labels = struct( # NOTE: Special target; see definition for details. ENABLE_RUNFILES = str(Label("//command_line_option:enable_runfiles")), EXEC_TOOLS_TOOLCHAIN = str(Label("//python/config_settings:exec_tools_toolchain")), + EXTRA_TOOLCHAINS = str(Label("//command_line_option:extra_toolchains")), NONE = str(Label("//python:none")), PIP_ENV_MARKER_CONFIG = str(Label("//python/config_settings:pip_env_marker_config")), PIP_WHL_OSX_VERSION = str(Label("//python/config_settings:pip_whl_osx_version")), diff --git a/python/private/transition_labels.bzl b/python/private/transition_labels.bzl index 5f0aa69056..7a6531ed0f 100644 --- a/python/private/transition_labels.bzl +++ b/python/private/transition_labels.bzl @@ -12,6 +12,7 @@ _BASE_TRANSITION_LABELS = [ labels.BOOTSTRAP_IMPL, labels.DEBUGGER, labels.EXEC_TOOLS_TOOLCHAIN, + "//command_line_option:extra_toolchains", labels.PIP_ENV_MARKER_CONFIG, labels.PIP_WHL_OSX_VERSION, labels.PRECOMPILE, diff --git a/tests/base_rules/py_executable_base_tests.bzl b/tests/base_rules/py_executable_base_tests.bzl index dff0399bc2..7531de4c30 100644 --- a/tests/base_rules/py_executable_base_tests.bzl +++ b/tests/base_rules/py_executable_base_tests.bzl @@ -109,6 +109,29 @@ def _test_basic_zip_impl(env, target): _tests.append(_test_basic_zip) +def _test_config_settings_extra_toolchains(name, config): + rt_util.helper_target( + config.rule, + name = name + "_subject", + srcs = ["main.py"], + main = "main.py", + config_settings = { + "//command_line_option:extra_toolchains": "{tc},{tc}".format(tc = CC_TOOLCHAIN), + }, + ) + analysis_test( + name = name, + impl = _test_config_settings_extra_toolchains_impl, + target = name + "_subject", + ) + +def _test_config_settings_extra_toolchains_impl(env, target): + # If we got here, it means analysis succeeded, which implies the transition + # successfully parsed the CSV string into a list. + env.expect.that_target(target).has_provider(PyInfo) + +_tests.append(_test_config_settings_extra_toolchains) + def _test_cross_compile_to_unix(name, config): rt_util.helper_target( config.rule, diff --git a/tests/integration/bzlmod_lockfile/MODULE.bazel.lock b/tests/integration/bzlmod_lockfile/MODULE.bazel.lock index 0dcc4b2a48..d21fec2d7d 100644 --- a/tests/integration/bzlmod_lockfile/MODULE.bazel.lock +++ b/tests/integration/bzlmod_lockfile/MODULE.bazel.lock @@ -250,7 +250,7 @@ }, "@@rules_python+//python/uv:uv.bzl%uv": { "general": { - "bzlTransitiveDigest": "yrEbeCJlv5gbdoRjwwR/EDEkc3pfuQy/FTYKMNoK5Wc=", + "bzlTransitiveDigest": "Z5ZPR9z4PkJRXSyJ4KQEqM4kwiqWBCn8Ajzxy9YlS/g=", "usagesDigest": "6yXGw7XDyXjOfqBL0SBu1YBEMMYPQzCE3jTzUCkxPgg=", "recordedInputs": [ "REPO_MAPPING:rules_python+,bazel_tools bazel_tools", From 8b38325e7c1b8b3240088b14d5ba2e5316aceabc Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Mon, 8 Jun 2026 08:09:42 -0700 Subject: [PATCH 756/922] build: configure secondary mirror fallback (#3814) Currently, transient GitHub network issues can cause builds to fail when pulling platforms. To fix, configure Bazel's downloader to try GitHub first and fall back to mirror.bazel.build. --- .bazelrc | 2 ++ AGENTS.md | 6 ++++++ downloader_config.cfg | 16 ++++++++++++++++ gazelle/.bazelrc | 1 + gazelle/downloader_config.cfg | 16 ++++++++++++++++ sphinxdocs/.bazelrc | 1 + sphinxdocs/downloader_config.cfg | 16 ++++++++++++++++ 7 files changed, 58 insertions(+) create mode 100644 downloader_config.cfg create mode 100644 gazelle/downloader_config.cfg create mode 100644 sphinxdocs/downloader_config.cfg diff --git a/.bazelrc b/.bazelrc index 8fedcf1f8f..8f784e4c2b 100644 --- a/.bazelrc +++ b/.bazelrc @@ -44,6 +44,8 @@ common --enable_bzlmod # Local disk cache greatly speeds up builds if the regular cache is lost common --disk_cache=~/.cache/bazel/bazel-disk-cache +# Drop `experimental_` prefix once Bazel 7 is no longer supported +common --experimental_downloader_config=downloader_config.cfg # Additional config to use for readthedocs builds. diff --git a/AGENTS.md b/AGENTS.md index 73d254f27c..65c5baf64b 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -207,3 +207,9 @@ e.g. ``` load("//python/private:foo.bzl", "foo") # buildifier: disable=bzl-visibility ``` + +### CI Failure Inspection + +When inspecting CI failures, if the failure is due to a network error +downloading a repository, check if that rule set is mirrored on +mirror.bazel.build. If so, add it to the downloader config. diff --git a/downloader_config.cfg b/downloader_config.cfg new file mode 100644 index 0000000000..a978fb89b9 --- /dev/null +++ b/downloader_config.cfg @@ -0,0 +1,16 @@ +# Try GitHub first (primary) +rewrite ^github\.com/bazel-contrib/bazel_features/(.*) github.com/bazel-contrib/bazel_features/$1 +rewrite ^github\.com/bazel-contrib/rules_go/(.*) github.com/bazel-contrib/rules_go/$1 +rewrite ^github\.com/bazelbuild/bazel-skylib/(.*) github.com/bazelbuild/bazel-skylib/$1 +rewrite ^github\.com/bazelbuild/platforms/(.*) github.com/bazelbuild/platforms/$1 +rewrite ^github\.com/bazelbuild/rules_kotlin/(.*) github.com/bazelbuild/rules_kotlin/$1 +rewrite ^github\.com/bazelbuild/rules_shell/(.*) github.com/bazelbuild/rules_shell/$1 + +# Fall back to mirror (secondary) +# Tracking upstream BCR mirror addition: https://github.com/bazelbuild/platforms/issues/139 +rewrite ^github\.com/bazel-contrib/bazel_features/(.*) mirror.bazel.build/github.com/bazel-contrib/bazel_features/$1 +rewrite ^github\.com/bazel-contrib/rules_go/(.*) mirror.bazel.build/github.com/bazel-contrib/rules_go/$1 +rewrite ^github\.com/bazelbuild/bazel-skylib/(.*) mirror.bazel.build/github.com/bazelbuild/bazel-skylib/$1 +rewrite ^github\.com/bazelbuild/platforms/(.*) mirror.bazel.build/github.com/bazelbuild/platforms/$1 +rewrite ^github\.com/bazelbuild/rules_kotlin/(.*) mirror.bazel.build/github.com/bazelbuild/rules_kotlin/$1 +rewrite ^github\.com/bazelbuild/rules_shell/(.*) mirror.bazel.build/github.com/bazelbuild/rules_shell/$1 diff --git a/gazelle/.bazelrc b/gazelle/.bazelrc index 9a38133e9d..bdb29d5bc3 100644 --- a/gazelle/.bazelrc +++ b/gazelle/.bazelrc @@ -1,5 +1,6 @@ common --deleted_packages=examples/bzlmod_build_file_generation common --deleted_packages=examples/bzlmod_build_file_generation/runfiles +common --experimental_downloader_config=downloader_config.cfg test --test_output=errors diff --git a/gazelle/downloader_config.cfg b/gazelle/downloader_config.cfg new file mode 100644 index 0000000000..a978fb89b9 --- /dev/null +++ b/gazelle/downloader_config.cfg @@ -0,0 +1,16 @@ +# Try GitHub first (primary) +rewrite ^github\.com/bazel-contrib/bazel_features/(.*) github.com/bazel-contrib/bazel_features/$1 +rewrite ^github\.com/bazel-contrib/rules_go/(.*) github.com/bazel-contrib/rules_go/$1 +rewrite ^github\.com/bazelbuild/bazel-skylib/(.*) github.com/bazelbuild/bazel-skylib/$1 +rewrite ^github\.com/bazelbuild/platforms/(.*) github.com/bazelbuild/platforms/$1 +rewrite ^github\.com/bazelbuild/rules_kotlin/(.*) github.com/bazelbuild/rules_kotlin/$1 +rewrite ^github\.com/bazelbuild/rules_shell/(.*) github.com/bazelbuild/rules_shell/$1 + +# Fall back to mirror (secondary) +# Tracking upstream BCR mirror addition: https://github.com/bazelbuild/platforms/issues/139 +rewrite ^github\.com/bazel-contrib/bazel_features/(.*) mirror.bazel.build/github.com/bazel-contrib/bazel_features/$1 +rewrite ^github\.com/bazel-contrib/rules_go/(.*) mirror.bazel.build/github.com/bazel-contrib/rules_go/$1 +rewrite ^github\.com/bazelbuild/bazel-skylib/(.*) mirror.bazel.build/github.com/bazelbuild/bazel-skylib/$1 +rewrite ^github\.com/bazelbuild/platforms/(.*) mirror.bazel.build/github.com/bazelbuild/platforms/$1 +rewrite ^github\.com/bazelbuild/rules_kotlin/(.*) mirror.bazel.build/github.com/bazelbuild/rules_kotlin/$1 +rewrite ^github\.com/bazelbuild/rules_shell/(.*) mirror.bazel.build/github.com/bazelbuild/rules_shell/$1 diff --git a/sphinxdocs/.bazelrc b/sphinxdocs/.bazelrc index caefd0af53..acff835394 100644 --- a/sphinxdocs/.bazelrc +++ b/sphinxdocs/.bazelrc @@ -15,6 +15,7 @@ build --enable_runfiles # Local disk cache greatly speeds up builds if the regular cache is lost common --disk_cache=~/.cache/bazel/bazel-disk-cache +common --experimental_downloader_config=downloader_config.cfg common --incompatible_python_disallow_native_rules common --incompatible_no_implicit_file_export diff --git a/sphinxdocs/downloader_config.cfg b/sphinxdocs/downloader_config.cfg new file mode 100644 index 0000000000..a978fb89b9 --- /dev/null +++ b/sphinxdocs/downloader_config.cfg @@ -0,0 +1,16 @@ +# Try GitHub first (primary) +rewrite ^github\.com/bazel-contrib/bazel_features/(.*) github.com/bazel-contrib/bazel_features/$1 +rewrite ^github\.com/bazel-contrib/rules_go/(.*) github.com/bazel-contrib/rules_go/$1 +rewrite ^github\.com/bazelbuild/bazel-skylib/(.*) github.com/bazelbuild/bazel-skylib/$1 +rewrite ^github\.com/bazelbuild/platforms/(.*) github.com/bazelbuild/platforms/$1 +rewrite ^github\.com/bazelbuild/rules_kotlin/(.*) github.com/bazelbuild/rules_kotlin/$1 +rewrite ^github\.com/bazelbuild/rules_shell/(.*) github.com/bazelbuild/rules_shell/$1 + +# Fall back to mirror (secondary) +# Tracking upstream BCR mirror addition: https://github.com/bazelbuild/platforms/issues/139 +rewrite ^github\.com/bazel-contrib/bazel_features/(.*) mirror.bazel.build/github.com/bazel-contrib/bazel_features/$1 +rewrite ^github\.com/bazel-contrib/rules_go/(.*) mirror.bazel.build/github.com/bazel-contrib/rules_go/$1 +rewrite ^github\.com/bazelbuild/bazel-skylib/(.*) mirror.bazel.build/github.com/bazelbuild/bazel-skylib/$1 +rewrite ^github\.com/bazelbuild/platforms/(.*) mirror.bazel.build/github.com/bazelbuild/platforms/$1 +rewrite ^github\.com/bazelbuild/rules_kotlin/(.*) mirror.bazel.build/github.com/bazelbuild/rules_kotlin/$1 +rewrite ^github\.com/bazelbuild/rules_shell/(.*) mirror.bazel.build/github.com/bazelbuild/rules_shell/$1 From c0fef4677ab0be33b76cd394af5e56002e8a114f Mon Sep 17 00:00:00 2001 From: Ignas Anikevicius <240938+aignas@users.noreply.github.com> Date: Tue, 9 Jun 2026 14:58:51 +0900 Subject: [PATCH 757/922] fix(uv): respect uv.tool settings in pyproject.toml (#3811) With this we auto-detect the location of the project file based on where the first pyproject.toml file is located. Whilst this may work for majority of the cases there could be a case where the user wants to leverage the workspaces, we iterate through all pyproject.toml files and choose the one with the shortest directory path. If this does not work, we you can just override it manually. Along the way, give some advice to agents for handling flakey CI and pushing changes to PRs. Fixes #3807 --------- Co-authored-by: Richard Levasseur --- AGENTS.md | 22 ++ CHANGELOG.md | 2 + docs/pypi/lock.md | 51 +++- python/uv/lock.bzl | 22 ++ python/uv/private/lock.bzl | 49 +++- tests/uv/lock/BUILD.bazel | 5 + tests/uv/lock/lock_tests.bzl | 1 + tests/uv/lock/pyproject_toml/BUILD.bazel | 30 +++ tests/uv/lock/pyproject_toml/pyproject.toml | 8 + tests/uv/lock/pyproject_toml/requirements.txt | 18 ++ tests/uv/lock/workspaces/BUILD.bazel | 27 ++ tests/uv/lock/workspaces/packages/BUILD.bazel | 5 + .../lock/workspaces/packages/foo/BUILD.bazel | 1 + .../workspaces/packages/foo/pyproject.toml | 7 + tests/uv/lock/workspaces/pyproject.toml | 10 + tests/uv/lock/workspaces/requirements.txt | 242 ++++++++++++++++++ 16 files changed, 497 insertions(+), 3 deletions(-) create mode 100644 tests/uv/lock/pyproject_toml/BUILD.bazel create mode 100644 tests/uv/lock/pyproject_toml/pyproject.toml create mode 100644 tests/uv/lock/pyproject_toml/requirements.txt create mode 100644 tests/uv/lock/workspaces/BUILD.bazel create mode 100644 tests/uv/lock/workspaces/packages/BUILD.bazel create mode 100644 tests/uv/lock/workspaces/packages/foo/BUILD.bazel create mode 100644 tests/uv/lock/workspaces/packages/foo/pyproject.toml create mode 100644 tests/uv/lock/workspaces/pyproject.toml create mode 100644 tests/uv/lock/workspaces/requirements.txt diff --git a/AGENTS.md b/AGENTS.md index 65c5baf64b..4e1e88aeac 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -31,6 +31,11 @@ into the sentence, not verbatim. When adding `{versionadded}` or `{versionchanged}` sections, add them add the end of the documentation text. +### PR Updates + +Once a PR is created, create new commits and merges. Don't use rebase or amend +because it interferes with code review comments. + ### PR descriptions Follow the advice in `CONTRIBUTING.md` for PR descriptions. PR descriptions @@ -150,6 +155,8 @@ This repository contains 3 Bazel bzlmod modules. * `gazelle/` is for the `@rules_python_gazelle_plugin` module. * All other code is part of `@rules_python`. +`tests/support/` contains utility code and helpers for testing. + `python/config_settings/BUILD.bazel` contains build flags that are part of the public API. DO NOT add, remove, or modify these build flags unless specifically @@ -213,3 +220,18 @@ load("//python/private:foo.bzl", "foo") # buildifier: disable=bzl-visibility When inspecting CI failures, if the failure is due to a network error downloading a repository, check if that rule set is mirrored on mirror.bazel.build. If so, add it to the downloader config. + +### CI Flakiness and Monitoring + +CI is known to have flakey network issues. When submitting or updating a PR, +start a background agent that continuously monitors the latest build of a PR and +checks for flakey network errors (e.g., 504 gateway errors). + +If Buildkite permissions allow, retry failures and use available Buildkite +skills. If permissions do not allow, modify `.bazelrc` to set flags (such as +`--http_timeout_scaling` or `--experimental_repository_downloader_retries`) to +make downloads retry more. + +If downloads fail for a dependency, check if that dependency is available on +mirror.bazel.build and, if so, add it to downloader_config.cfg if it isn't +already there. diff --git a/CHANGELOG.md b/CHANGELOG.md index 33adfa7a60..99876bde0d 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -109,6 +109,8 @@ END_UNRELEASED_TEMPLATE * (uv) fix the execution of the `uv pip compile` in the sandbox. Work towards better supporting `uv` out of the box on our platforms. ([#1975](https://github.com/bazel-contrib/rules_python/issues/1975)) +* (uv) automatically pass the `--project` parameter based on the source files. + ([#3087](https://github.com/bazel-contrib/rules_python/issues/3087)) {#v0-0-0-added} ### Added diff --git a/docs/pypi/lock.md b/docs/pypi/lock.md index b5d8ec24f7..5c9f0646dc 100644 --- a/docs/pypi/lock.md +++ b/docs/pypi/lock.md @@ -68,8 +68,55 @@ compile_pip_requirements( ### uv pip compile (bzlmod only) -We also have experimental setup for the `uv pip compile` way of generating lock files. +We also have an experimental setup for the `uv pip compile` way of generating lock files. This is well tested with the public PyPI index, but you may hit some rough edges with private mirrors. -For more documentation see {obj}`lock` documentation. +#### Example usage + +```starlark +load("@rules_python//python/uv:lock.bzl", "lock") + +lock( + name = "requirements", + srcs = ["pyproject.toml", "requirements.in"], + out = "requirements_lock.txt", +) +``` + +#### `[tool.uv]` settings from pyproject.toml + +When a `pyproject.toml` file is among the {attr}`lock.srcs`, the +{obj}`lock` rule auto-detects the project directory and passes +`--project ` to `uv pip compile`. This causes `uv` to read +`[tool.uv]` settings from that `pyproject.toml`, such as +`no-build-isolation`, `exclude-dependencies`, and workspace members. + +If multiple `pyproject.toml` files are in {attr}`lock.srcs`, the one +with the shortest directory path is selected (this heuristic works for +typical uv workspace layouts where the root configuration is at the +shortest path). + +If the auto-detection picks the wrong project directory, use the +`project` parameter to override: + +```starlark +lock( + name = "requirements", + srcs = ["pyproject.toml", "requirements.in"], + out = "requirements_lock.txt", + project = "subproject", +) +``` + +:::{warning} +**Known limitations of auto-detection** + +1. **Workspace heuristic** — the shortest-path selection may incorrectly assume the upper-most + workspace `pyproject.toml` is the correct one. For monorepos with multiple independent + sub-projects, you must set `project` explicitly for each {obj}`lock` target. +1. **No test target** — unlike {obj}`compile_pip_requirements`, no test target is auto-created; see + the {obj}`lock` docs for how to add one manually using `diff_test` from `bazel_skylib`. +::: + +For more documentation see {obj}`lock`. diff --git a/python/uv/lock.bzl b/python/uv/lock.bzl index 7bcca780a0..7fd50082ea 100644 --- a/python/uv/lock.bzl +++ b/python/uv/lock.bzl @@ -45,6 +45,28 @@ native_test( ) ``` +### `[tool.uv]` settings support + +When a `pyproject.toml` file is included in {attr}`lock.srcs`, the +`--project` flag is automatically passed to `uv pip compile` using the +directory of the shortest-path `pyproject.toml`. This causes `uv` to +read `[tool.uv]` settings such as `no-build-isolation`, +`exclude-dependencies`, and `[tool.uv.workspace]` from that file. + +If the auto-detection doesn't select the right project (e.g. in complex +workspace layouts), use the `project` parameter to override it: + +```starlark +lock( + name = "requirements", + srcs = [ + "pyproject.toml", + "requirements.in", + ], + project = "subproject", +) +``` + EXPERIMENTAL: This is experimental and may be changed without notice. """ diff --git a/python/uv/private/lock.bzl b/python/uv/private/lock.bzl index 6f0b80af89..23f2eed467 100644 --- a/python/uv/private/lock.bzl +++ b/python/uv/private/lock.bzl @@ -71,7 +71,8 @@ def _args(ctx): ) def _lock_impl(ctx): - srcs = ctx.files.srcs + srcs = [] + ctx.files.srcs + fname = "{}.out".format(ctx.label.name) python_version = ctx.attr.python_version if python_version: @@ -99,6 +100,27 @@ def _lock_impl(ctx): args.add("--generate-hashes") if not ctx.attr.strip_extras: args.add("--no-strip-extras") + + project = None + if ctx.attr.project: + project = ctx.attr.project + else: + # Autodetect the project based on the `pyproject.toml` location - it will be the first src that + # we see that is named "pyproject.toml" + for src in srcs: + if src.basename == "pyproject.toml": + if project == None: + project = src.dirname + elif len(project) > len(src.dirname): + # select the shortest match + project = src.dirname + + if project == None: + project = pkg + + if project: + args.add_all([project], before_each = "--project") + args.add_all(ctx.files.build_constraints, before_each = "--build-constraints") args.add_all(ctx.files.constraints, before_each = "--constraints") args.add_all(ctx.attr.args) @@ -275,6 +297,19 @@ modifications and the locking is not done from scratch. doc = "Public, see the docs in the macro.", mandatory = True, ), + "project": attr.string( + doc = """\ +Overrides the `--project` directory passed to `uv pip compile`. +If not set, the project directory is auto-detected: when +`pyproject.toml` files are in {obj}`lock.srcs`, the one with the +shortest directory path is selected. This makes `uv` read +`[tool.uv]` settings (e.g. `no-build-isolation`, +`exclude-dependencies`) from that `pyproject.toml`. + +:::{versionadded} VERSION_NEXT_FEATURE +::: +""", + ), "python_version": attr.string( doc = "Public, see the docs in the macro.", ), @@ -438,6 +473,7 @@ def lock( env = None, generate_hashes = True, python_version = None, + project = None, strip_extras = False, **kwargs): """Pin the requirements based on the src files. @@ -484,6 +520,16 @@ def lock( function, but sometimes one may want to not have the extras if you are compiling the requirements file for using it as a constraints file. Defaults to `False`. + project: {type}`str | None` overrides the `--project` directory + passed to `uv pip compile`. By default the project directory + is auto-detected: when {obj}`lock.srcs` contains + `pyproject.toml` files, the one with the shortest directory + path is selected. This causes `uv` to read `[tool.uv]` + settings such as `no-build-isolation` and + `exclude-dependencies` from that `pyproject.toml`. If no + `pyproject.toml` is in `srcs` and no `project` is given, the + Bazel package directory is used as fallback. + {versionadded}VERSION_NEXT_FEATURE python_version: {type}`str | None` the python_version to transition to when locking the requirements. Defaults to the default python version configured by the {obj}`python` module extension. @@ -509,6 +555,7 @@ def lock( env = env, existing_output = maybe_out, generate_hashes = generate_hashes, + project = project, is_windows = select({ "@platforms//os:windows": True, "//conditions:default": False, diff --git a/tests/uv/lock/BUILD.bazel b/tests/uv/lock/BUILD.bazel index 6b6902da44..0b72f015b7 100644 --- a/tests/uv/lock/BUILD.bazel +++ b/tests/uv/lock/BUILD.bazel @@ -1,5 +1,10 @@ load(":lock_tests.bzl", "lock_test_suite") +exports_files( + glob(["testdata/*"]), + visibility = ["//tests:__subpackages__"], +) + lock_test_suite( name = "lock_tests", ) diff --git a/tests/uv/lock/lock_tests.bzl b/tests/uv/lock/lock_tests.bzl index bcaed95b53..3e067f3e73 100644 --- a/tests/uv/lock/lock_tests.bzl +++ b/tests/uv/lock/lock_tests.bzl @@ -91,6 +91,7 @@ def lock_test_suite(name): name = name, tests = [ ":requirements_test", + "//tests/uv/lock/pyproject_toml:requirements_test", ":requirements_run_tests", ], ) diff --git a/tests/uv/lock/pyproject_toml/BUILD.bazel b/tests/uv/lock/pyproject_toml/BUILD.bazel new file mode 100644 index 0000000000..a64fe50d14 --- /dev/null +++ b/tests/uv/lock/pyproject_toml/BUILD.bazel @@ -0,0 +1,30 @@ +load("@bazel_skylib//rules:diff_test.bzl", "diff_test") +load("//python/uv:lock.bzl", "lock") + +# This test verifies that the `lock` rule automatically passes `--project` to `uv pip compile` based +# on the package directory, so that `[tool.uv]` settings from a `pyproject.toml` in the same +# directory are applied. It will exclude a particular dependency from the lock-file, so it will be +# easy to see if we have any issues. +lock( + name = "requirements", + srcs = ["pyproject.toml"], + out = "requirements.txt", + build_constraints = [ + "//tests/uv/lock:testdata/build_constraints.txt", + "//tests/uv/lock:testdata/build_constraints2.txt", + ], + constraints = [ + "//tests/uv/lock:testdata/constraints.txt", + "//tests/uv/lock:testdata/constraints2.txt", + ], + # It seems that the CI remote executors for the RBE do not have network + # connectivity due to current CI setup. + tags = ["no-remote-exec"], +) + +diff_test( + name = "requirements_test", + timeout = "short", + file1 = ":requirements", + file2 = "requirements.txt", +) diff --git a/tests/uv/lock/pyproject_toml/pyproject.toml b/tests/uv/lock/pyproject_toml/pyproject.toml new file mode 100644 index 0000000000..06b96309da --- /dev/null +++ b/tests/uv/lock/pyproject_toml/pyproject.toml @@ -0,0 +1,8 @@ +[project] +name = "test" +version = "0.0.0" +dependencies = ["requests"] + +[tool.uv] +no-build-isolation = true +exclude-dependencies = ["charset-normalizer"] diff --git a/tests/uv/lock/pyproject_toml/requirements.txt b/tests/uv/lock/pyproject_toml/requirements.txt new file mode 100644 index 0000000000..4ea098b0e4 --- /dev/null +++ b/tests/uv/lock/pyproject_toml/requirements.txt @@ -0,0 +1,18 @@ +# This file was autogenerated by uv via the following command: +# bazel run //tests/uv/lock/pyproject_toml:requirements.update +certifi==2025.1.31 \ + --hash=sha256:3d5da6925056f6f18f119200434a4780a94263f10d1c21d032a6f6b2baa20651 \ + --hash=sha256:ca78db4565a652026a4db2bcdf68f2fb589ea80d0be70e03929ed730746b84fe + # via requests +idna==3.10 \ + --hash=sha256:12f65c9b470abda6dc35cf8e63cc574b1c52b11df2c86030af0ac09b01b13ea9 \ + --hash=sha256:946d195a0d259cbba61165e88e65941f16e9b36ea6ddb97f00452bae8b1287d3 + # via requests +requests==2.32.3 \ + --hash=sha256:55365417734eb18255590a9ff9eb97e9e1da868d4ccd6402399eaf68af20a760 \ + --hash=sha256:70761cfe03c773ceb22aa2f671b4757976145175cdfca038c02654d061d6dcc6 + # via test (tests/uv/lock/pyproject_toml/pyproject.toml) +urllib3==2.3.0 \ + --hash=sha256:1cee9ad369867bfdbbb48b7dd50374c0967a0bb7710050facf0dd6911440e3df \ + --hash=sha256:f8c5449b3cf0861679ce7e0503c7b44b5ec981bec0d1d3795a07f1ba96f0204d + # via requests diff --git a/tests/uv/lock/workspaces/BUILD.bazel b/tests/uv/lock/workspaces/BUILD.bazel new file mode 100644 index 0000000000..aa1fabe2e3 --- /dev/null +++ b/tests/uv/lock/workspaces/BUILD.bazel @@ -0,0 +1,27 @@ +load("@bazel_skylib//rules:diff_test.bzl", "diff_test") +load("//python/uv:lock.bzl", "lock") +load("//tests/support:support.bzl", "NOT_WINDOWS") + +# This test verifies that the `lock` rule automatically passes `--project` to `uv pip compile` based +# on the package directory, so that `[tool.uv]` settings from a `pyproject.toml` in the same +# directory are applied. It will exclude a particular dependency from the lock-file, so it will be +# easy to see if we have any issues. +lock( + name = "requirements", + srcs = [ + "pyproject.toml", + "//tests/uv/lock/workspaces/packages", + ], + out = "requirements.txt", + # It seems that the CI remote executors for the RBE do not have network + # connectivity due to current CI setup. + tags = ["no-remote-exec"], +) + +diff_test( + name = "requirements_test", + timeout = "short", + file1 = ":requirements", + file2 = "requirements.txt", + target_compatible_with = NOT_WINDOWS, +) diff --git a/tests/uv/lock/workspaces/packages/BUILD.bazel b/tests/uv/lock/workspaces/packages/BUILD.bazel new file mode 100644 index 0000000000..5461287d9f --- /dev/null +++ b/tests/uv/lock/workspaces/packages/BUILD.bazel @@ -0,0 +1,5 @@ +filegroup( + name = "packages", + srcs = ["//tests/uv/lock/workspaces/packages/foo:pyproject.toml"], + visibility = ["//tests/uv/lock/workspaces:__pkg__"], +) diff --git a/tests/uv/lock/workspaces/packages/foo/BUILD.bazel b/tests/uv/lock/workspaces/packages/foo/BUILD.bazel new file mode 100644 index 0000000000..d89a7b784e --- /dev/null +++ b/tests/uv/lock/workspaces/packages/foo/BUILD.bazel @@ -0,0 +1 @@ +exports_files(["pyproject.toml"]) diff --git a/tests/uv/lock/workspaces/packages/foo/pyproject.toml b/tests/uv/lock/workspaces/packages/foo/pyproject.toml new file mode 100644 index 0000000000..f26ea85b51 --- /dev/null +++ b/tests/uv/lock/workspaces/packages/foo/pyproject.toml @@ -0,0 +1,7 @@ +[project] +name = "foo" +version = "0.0.0" +dependencies = ["black"] + +[tool.uv] +no-build-isolation = true diff --git a/tests/uv/lock/workspaces/pyproject.toml b/tests/uv/lock/workspaces/pyproject.toml new file mode 100644 index 0000000000..01e1e2e076 --- /dev/null +++ b/tests/uv/lock/workspaces/pyproject.toml @@ -0,0 +1,10 @@ +[project] +name = "test" +version = "0.0.0" +dependencies = ["requests"] + +[tool.uv] +no-build-isolation = true + +[tool.uv.workspace] +members = ["packages/*"] diff --git a/tests/uv/lock/workspaces/requirements.txt b/tests/uv/lock/workspaces/requirements.txt new file mode 100644 index 0000000000..5ce35d7571 --- /dev/null +++ b/tests/uv/lock/workspaces/requirements.txt @@ -0,0 +1,242 @@ +# This file was autogenerated by uv via the following command: +# bazel run //tests/uv/lock/workspaces:requirements.update +black==26.5.1 \ + --hash=sha256:0e48b87e03bf109288e55cfceadcfa15ff5470aca2851a851950ed2926f450d7 \ + --hash=sha256:1037d5ac7b7b310b2632ad867ec8d0e4c4819dcdb0b820f63135da746a24e418 \ + --hash=sha256:1ef92b76f7733f282fd096ea406200b5a286c42947412b0eaff3a74e3616cefe \ + --hash=sha256:1f7ea64ebfa01b50f693508fc39f875e264446d3b097088f84f203b9d09618a0 \ + --hash=sha256:22f2cd76d069cc54c71f10360744ba8983fbb616903b4304a85b734915c8e1b4 \ + --hash=sha256:2b36cf2ddf5566e205f6535f782a62194a184d33e175b64ae8c40b1737522be3 \ + --hash=sha256:30d3c14661f2792e9142cce3eeeb1cbc175b3eb5f733be0c8eeb99651e52b0c3 \ + --hash=sha256:32d5ea7f6c8bdfa6e648326ebca1f02b0764e2a029edc6f8dce2627e19d468c3 \ + --hash=sha256:3915f256e75a2d7cf88d8953d37f780455dc586cc72dee059c528fe77f581217 \ + --hash=sha256:4ad6fa01f941920f54f2bbb35f3df7673428a0ef98a0b0840c2eaef3b110efa8 \ + --hash=sha256:4ed7f7da04046d2e488437170797d3b4a4ad83906683bcb7dfc68b673bbce5e2 \ + --hash=sha256:5119fa92ae61f786e8c3662fd60aece1d0a2dd5cca5d0c79417a95e7a4272a59 \ + --hash=sha256:577f21094ea469ef92ec1adaf2c9441a226d2144d01a5be2fa823cecf6543e50 \ + --hash=sha256:58b4bd92cf88aacf83d88479c8f9caee044b1ec55f2451a337354a7ea2590a22 \ + --hash=sha256:5c34b25da232ead53a6f335b76dbea124f4d152ad568b9080d6f944bc2b34b52 \ + --hash=sha256:87ed5c6f450580a2f6790bc7cbfb016dfc73bc750249762268a3695361315eef \ + --hash=sha256:89c93167a74d3a75dfaa38a5c7cca015537d5820dd7f17d63267d674a61cae90 \ + --hash=sha256:96ae2c733b2aabdd9986e2c5df628ff3473676cd1c5faded1ff496cf6d74083c \ + --hash=sha256:9942db8888e06943c5dde66ca0037dcff82a2a4ec1ad0ada9e0d2ee9d9823893 \ + --hash=sha256:9d98d4137277c75dfb898ec8d846c4fd68ba1e9cf77f95e2865c203dc18f4c3d \ + --hash=sha256:a1dca32d9f1784af512a13410ec204c6f7f0aa9797a111c42e1c03449821c264 \ + --hash=sha256:dd321f668053961824bcc1be1cc1df748b2d7e4fa28086b08331e577b0100a73 \ + --hash=sha256:e1a26503279b6b310669fb0b219c39e4820b77e8189fe80f522bb511f247db0a \ + --hash=sha256:e88976690a64b0af98312ca958415849cb42423423c5f2ee74af4b49a97a2168 \ + --hash=sha256:ea8d16dc41655aa113cd64665e7219446cd7e4ff2248d7178eaa905190c86b18 \ + --hash=sha256:ecb3e624844c798144e9bd986954e0adc81d8911a1f30f375e1252fe26e8c294 \ + --hash=sha256:ed1a20af114c301a0269bf01163d51dbef72737fd65f850001e7cbe7f3c7abae + # via foo (tests/uv/lock/workspaces/packages/foo/pyproject.toml) +certifi==2025.1.31 \ + --hash=sha256:3d5da6925056f6f18f119200434a4780a94263f10d1c21d032a6f6b2baa20651 \ + --hash=sha256:ca78db4565a652026a4db2bcdf68f2fb589ea80d0be70e03929ed730746b84fe + # via requests +charset-normalizer==3.4.7 \ + --hash=sha256:007d05ec7321d12a40227aae9e2bc6dca73f3cb21058999a1df9e193555a9dcc \ + --hash=sha256:03853ed82eeebbce3c2abfdbc98c96dc205f32a79627688ac9a27370ea61a49c \ + --hash=sha256:07d9e39b01743c3717745f4c530a6349eadbfa043c7577eef86c502c15df2c67 \ + --hash=sha256:08e721811161356f97b4059a9ba7bafb23ea5ee2255402c42881c214e173c6b4 \ + --hash=sha256:0c96c3b819b5c3e9e165495db84d41914d6894d55181d2d108cc1a69bfc9cce0 \ + --hash=sha256:0ea948db76d31190bf08bd371623927ee1339d5f2a0b4b1b4a4439a65298703c \ + --hash=sha256:0f7eb884681e3938906ed0434f20c63046eacd0111c4ba96f27b76084cd679f5 \ + --hash=sha256:12a6fff75f6bc66711b73a2f0addfc4c8c15a20e805146a02d147a318962c444 \ + --hash=sha256:12d8baf840cc7889b37c7c770f478adea7adce3dcb3944d02ec87508e2dcf153 \ + --hash=sha256:14265bfe1f09498b9d8ec91e9ec9fa52775edf90fcbde092b25f4a33d444fea9 \ + --hash=sha256:16d971e29578a5e97d7117866d15889a4a07befe0e87e703ed63cd90cb348c01 \ + --hash=sha256:177a0ba5f0211d488e295aaf82707237e331c24788d8d76c96c5a41594723217 \ + --hash=sha256:1a87ca9d5df6fe460483d9a5bbf2b18f620cbed41b432e2bddb686228282d10b \ + --hash=sha256:1c2a768fdd44ee4a9339a9b0b130049139b8ce3c01d2ce09f67f5a68048d477c \ + --hash=sha256:1c2aed2e5e41f24ea8ef1590b8e848a79b56f3a5564a65ceec43c9d692dc7d8a \ + --hash=sha256:1dc8b0ea451d6e69735094606991f32867807881400f808a106ee1d963c46a83 \ + --hash=sha256:1efde3cae86c8c273f1eb3b287be7d8499420cf2fe7585c41d370d3e790054a5 \ + --hash=sha256:202389074300232baeb53ae2569a60901f7efadd4245cf3a3bf0617d60b439d7 \ + --hash=sha256:203104ed3e428044fd943bc4bf45fa73c0730391f9621e37fe39ecf477b128cb \ + --hash=sha256:2257141f39fe65a3fdf38aeccae4b953e5f3b3324f4ff0daf9f15b8518666a2c \ + --hash=sha256:298930cec56029e05497a76988377cbd7457ba864beeea92ad7e844fe74cd1f1 \ + --hash=sha256:2cd4a60d0e2fb04537162c62bbbb4182f53541fe0ede35cdf270a1c1e723cc42 \ + --hash=sha256:2d6eb928e13016cea4f1f21d1e10c1cebd5a421bc57ddf5b1142ae3f86824fab \ + --hash=sha256:2fe249cb4651fd12605b7288b24751d8bfd46d35f12a20b1ba33dea122e690df \ + --hash=sha256:30b8d1d8c52a48c2c5690e152c169b673487a2a58de1ec7393196753063fcd5e \ + --hash=sha256:320ade88cfb846b8cd6b4ddf5ee9e80ee0c1f52401f2456b84ae1ae6a1a5f207 \ + --hash=sha256:3534e7dcbdcf757da6b85a0bbf5b6868786d5982dd959b065e65481644817a18 \ + --hash=sha256:36836d6ff945a00b88ba1e4572d721e60b5b8c98c155d465f56ad19d68f23734 \ + --hash=sha256:38c0109396c4cfc574d502df99742a45c72c08eff0a36158b6f04000043dbf38 \ + --hash=sha256:3946fa46a0cf3e4c8cb1cc52f56bb536310d34f25f01ca9b6c16afa767dab110 \ + --hash=sha256:3bec022aec2c514d9cf199522a802bd007cd588ab17ab2525f20f9c34d067c18 \ + --hash=sha256:3c9a494bc5ec77d43cea229c4f6db1e4d8fe7e1bbffa8b6f0f0032430ff8ab44 \ + --hash=sha256:3dce51d0f5e7951f8bb4900c257dad282f49190fdbebecd4ba99bcc41fef404d \ + --hash=sha256:3dedcc22d73ec993f42055eff4fcfed9318d1eeb9a6606c55892a26964964e48 \ + --hash=sha256:4042d5c8f957e15221d423ba781e85d553722fc4113f523f2feb7b188cc34c5e \ + --hash=sha256:481551899c856c704d58119b5025793fa6730adda3571971af568f66d2424bb5 \ + --hash=sha256:4dc1e73c36828f982bfe79fadf5919923f8a6f4df2860804db9a98c48824ce8d \ + --hash=sha256:4e5163c14bffd570ef2affbfdd77bba66383890797df43dc8b4cc7d6f500bf53 \ + --hash=sha256:511ef87c8aec0783e08ac18565a16d435372bc1ac25a91e6ac7f5ef2b0bff790 \ + --hash=sha256:532bc9bf33a68613fd7d65e4b1c71a6a38d7d42604ecf239c77392e9b4e8998c \ + --hash=sha256:54523e136b8948060c0fa0bc7b1b50c32c186f2fceee897a495406bb6e311d2b \ + --hash=sha256:5649fd1c7bade02f320a462fdefd0b4bd3ce036065836d4f42e0de958038e116 \ + --hash=sha256:56be790f86bfb2c98fb742ce566dfb4816e5a83384616ab59c49e0604d49c51d \ + --hash=sha256:5b77459df20e08151cd6f8b9ef8ef1f961ef73d85c21a555c7eed5b79410ec10 \ + --hash=sha256:5ed6ab538499c8644b8a3e18debabcd7ce684f3fa91cf867521a7a0279cab2d6 \ + --hash=sha256:6178f72c5508bfc5fd446a5905e698c6212932f25bcdd4b47a757a50605a90e2 \ + --hash=sha256:6370e8686f662e6a3941ee48ed4742317cafbe5707e36406e9df792cdb535776 \ + --hash=sha256:64f02c6841d7d83f832cd97ccf8eb8a906d06eb95d5276069175c696b024b60a \ + --hash=sha256:65bcd23054beab4d166035cabbc868a09c1a49d1efe458fe8e4361215df40265 \ + --hash=sha256:66671f93accb62ed07da56613636f3641f1a12c13046ce91ffc923721f23c008 \ + --hash=sha256:6696b7688f54f5af4462118f0bfa7c1621eeb87154f77fa04b9295ce7a8f2943 \ + --hash=sha256:6785f414ae0f3c733c437e0f3929197934f526d19dfaa75e18fdb4f94c6fb374 \ + --hash=sha256:67f6279d125ca0046a7fd386d01b311c6363844deac3e5b069b514ba3e63c246 \ + --hash=sha256:6c114670c45346afedc0d947faf3c7f701051d2518b943679c8ff88befe14f8e \ + --hash=sha256:6e0d51f618228538a3e8f46bd246f87a6cd030565e015803691603f55e12afb5 \ + --hash=sha256:6ed74185b2db44f41ef35fd1617c5888e59792da9bbc9190d6c7300617182616 \ + --hash=sha256:708838739abf24b2ceb208d0e22403dd018faeef86ddac04319a62ae884c4f15 \ + --hash=sha256:715479b9a2802ecac752a3b0efa2b0b60285cf962ee38414211abdfccc233b41 \ + --hash=sha256:733784b6d6def852c814bce5f318d25da2ee65dd4839a0718641c696e09a2960 \ + --hash=sha256:750e02e074872a3fad7f233b47734166440af3cdea0add3e95163110816d6752 \ + --hash=sha256:752a45dc4a6934060b3b0dab47e04edc3326575f82be64bc4fc293914566503e \ + --hash=sha256:7579e913a5339fb8fa133f6bbcfd8e6749696206cf05acdbdca71a1b436d8e72 \ + --hash=sha256:7641bb8895e77f921102f72833904dcd9901df5d6d72a2ab8f31d04b7e51e4e7 \ + --hash=sha256:7804338df6fcc08105c7745f1502ba68d900f45fd770d5bdd5288ddccb8a42d8 \ + --hash=sha256:80d04837f55fc81da168b98de4f4b797ef007fc8a79ab71c6ec9bc4dd662b15b \ + --hash=sha256:813c0e0132266c08eb87469a642cb30aaff57c5f426255419572aaeceeaa7bf4 \ + --hash=sha256:82b271f5137d07749f7bf32f70b17ab6eaabedd297e75dce75081a24f76eb545 \ + --hash=sha256:84c018e49c3bf790f9c2771c45e9313a08c2c2a6342b162cd650258b57817706 \ + --hash=sha256:8751d2787c9131302398b11e6c8068053dcb55d5a8964e114b6e196cf16cb366 \ + --hash=sha256:8778f0c7a52e56f75d12dae53ae320fae900a8b9b4164b981b9c5ce059cd1fcb \ + --hash=sha256:87fad7d9ba98c86bcb41b2dc8dbb326619be2562af1f8ff50776a39e55721c5a \ + --hash=sha256:8d828b6667a32a728a1ad1d93957cdf37489c57b97ae6c4de2860fa749b8fc1e \ + --hash=sha256:8e385e4267ab76874ae30db04c627faaaf0b509e1ccc11a95b3fc3e83f855c00 \ + --hash=sha256:92a0a01ead5e668468e952e4238cccd7c537364eb7d851ab144ab6627dbbe12f \ + --hash=sha256:94e1885b270625a9a828c9793b4d52a64445299baa1fea5a173bf1d3dd9a1a5a \ + --hash=sha256:a180c5e59792af262bf263b21a3c49353f25945d8d9f70628e73de370d55e1e1 \ + --hash=sha256:a277ab8928b9f299723bc1a2dabb1265911b1a76341f90a510368ca44ad9ab66 \ + --hash=sha256:a5fe03b42827c13cdccd08e6c0247b6a6d4b5e3cdc53fd1749f5896adcdc2356 \ + --hash=sha256:a6c5863edfbe888d9eff9c8b8087354e27618d9da76425c119293f11712a6319 \ + --hash=sha256:a89c23ef8d2c6b27fd200a42aa4ac72786e7c60d40efdc76e6011260b6e949c4 \ + --hash=sha256:adb2597b428735679446b46c8badf467b4ca5f5056aae4d51a19f9570301b1ad \ + --hash=sha256:ae196f021b5e7c78e918242d217db021ed2a6ace2bc6ae94c0fc596221c7f58d \ + --hash=sha256:ae89db9e5f98a11a4bf50407d4363e7b09b31e55bc117b4f7d80aab97ba009e5 \ + --hash=sha256:aed52fea0513bac0ccde438c188c8a471c4e0f457c2dd20cdbf6ea7a450046c7 \ + --hash=sha256:aef65cd602a6d0e0ff6f9930fcb1c8fec60dd2cfcb6facaf4bdb0e5873042db0 \ + --hash=sha256:af21eb4409a119e365397b2adbaca4c9ccab56543a65d5dbd9f920d6ac29f686 \ + --hash=sha256:b14b2d9dac08e28bb8046a1a0434b1750eb221c8f5b87a68f4fa11a6f97b5e34 \ + --hash=sha256:bb6d88045545b26da47aa879dd4a89a71d1dce0f0e549b1abcb31dfe4a8eac49 \ + --hash=sha256:bb8cc7534f51d9a017b93e3e85b260924f909601c3df002bcdb58ddb4dc41a5c \ + --hash=sha256:bc17a677b21b3502a21f66a8cc64f5bfad4df8a0b8434d661666f8ce90ac3af1 \ + --hash=sha256:bd6c2a1c7573c64738d716488d2cdd3c00e340e4835707d8fdb8dc1a66ef164e \ + --hash=sha256:bd9b23791fe793e4968dba0c447e12f78e425c59fc0e3b97f6450f4781f3ee60 \ + --hash=sha256:c03a41a8784091e67a39648f70c5f97b5b6a37f216896d44d2cdcb82615339a0 \ + --hash=sha256:c0f081d69a6e58272819b70288d3221a6ee64b98df852631c80f293514d3b274 \ + --hash=sha256:c35abb8bfff0185efac5878da64c45dafd2b37fb0383add1be155a763c1f083d \ + --hash=sha256:c36c333c39be2dbca264d7803333c896ab8fa7d4d6f0ab7edb7dfd7aea6e98c0 \ + --hash=sha256:c45e9440fb78f8ddabcf714b68f936737a121355bf59f3907f4e17721b9d1aae \ + --hash=sha256:c593052c465475e64bbfe5dbd81680f64a67fdc752c56d7a0ae205dc8aeefe0f \ + --hash=sha256:cdd68a1fb318e290a2077696b7eb7a21a49163c455979c639bf5a5dcdc46617d \ + --hash=sha256:ce3412fbe1e31eb81ea42f4169ed94861c56e643189e1e75f0041f3fe7020abe \ + --hash=sha256:cf1493cd8607bec4d8a7b9b004e699fcf8f9103a9284cc94962cb73d20f9d4a3 \ + --hash=sha256:cf29836da5119f3c8a8a70667b0ef5fdca3bb12f80fd06487cfa575b3909b393 \ + --hash=sha256:d4a48e5b3c2a489fae013b7589308a40146ee081f6f509e047e0e096084ceca1 \ + --hash=sha256:d560742f3c0d62afaccf9f41fe485ed69bd7661a241f86a3ef0f0fb8b1a397af \ + --hash=sha256:d6038d37043bced98a66e68d3aa2b6a35505dc01328cd65217cefe82f25def44 \ + --hash=sha256:d61f00a0869d77422d9b2aba989e2d24afa6ffd552af442e0e58de4f35ea6d00 \ + --hash=sha256:d635aab80466bc95771bb78d5370e74d36d1fe31467b6b29b8b57b2a3cd7d22c \ + --hash=sha256:dca4bbc466a95ba9c0234ef56d7dd9509f63da22274589ebd4ed7f1f4d4c54e3 \ + --hash=sha256:dd915403e231e6b1809fe9b6d9fc55cf8fb5e02765ac625d9cd623342a7905d7 \ + --hash=sha256:e044c39e41b92c845bc815e5ae4230804e8e7bc29e399b0437d64222d92809dd \ + --hash=sha256:e060d01aec0a910bdccb8be71faf34e7799ce36950f8294c8bf612cba65a2c9e \ + --hash=sha256:e1421b502d83040e6d7fb2fb18dff63957f720da3d77b2fbd3187ceb63755d7b \ + --hash=sha256:e17b8d5d6a8c47c85e68ca8379def1303fd360c3e22093a807cd34a71cd082b8 \ + --hash=sha256:e5f4d355f0a2b1a31bc3edec6795b46324349c9cb25eed068049e4f472fb4259 \ + --hash=sha256:e712b419df8ba5e42b226c510472b37bd57b38e897d3eca5e8cfd410a29fa859 \ + --hash=sha256:e74327fb75de8986940def6e8dee4f127cc9752bee7355bb323cc5b2659b6d46 \ + --hash=sha256:e80c8378d8f3d83cd3164da1ad2df9e37a666cdde7b1cb2298ed0b558064be30 \ + --hash=sha256:e8ac484bf18ce6975760921bb6148041faa8fef0547200386ea0b52b5d27bf7b \ + --hash=sha256:eca9705049ad3c7345d574e3510665cb2cf844c2f2dcfe675332677f081cbd46 \ + --hash=sha256:ed065083d0898c9d5b4bbec7b026fd755ff7454e6e8b73a67f8c744b13986e24 \ + --hash=sha256:edac0f1ab77644605be2cbba52e6b7f630731fc42b34cb0f634be1a6eface56a \ + --hash=sha256:effc3f449787117233702311a1b7d8f59cba9ced946ba727bdc329ec69028e24 \ + --hash=sha256:f22dec1690b584cea26fade98b2435c132c1b5f68e39f5a0b7627cd7ae31f1dc \ + --hash=sha256:f495a1652cf3fbab2eb0639776dad966c2fb874d79d87ca07f9d5f059b8bd215 \ + --hash=sha256:f496c9c3cc02230093d8330875c4c3cdfc3b73612a5fd921c65d39cbcef08063 \ + --hash=sha256:f59099f9b66f0d7145115e6f80dd8b1d847176df89b234a5a6b3f00437aa0832 \ + --hash=sha256:f59ad4c0e8f6bba240a9bb85504faa1ab438237199d4cce5f622761507b8f6a6 \ + --hash=sha256:fbccdc05410c9ee21bbf16a35f4c1d16123dcdeb8a1d38f33654fa21d0234f79 \ + --hash=sha256:fea24543955a6a729c45a73fe90e08c743f0b3334bbf3201e6c4bc1b0c7fa464 + # via requests +click==8.4.1 \ + --hash=sha256:482be17c6991b8c19c5429a1e995d9b0efdbb63172824c41f99965dc0ade8ec2 \ + --hash=sha256:918b5633eddf6b41c32d4f454bf0de810065c74e3f7dbf8ee5452f8be88d3e96 + # via black +idna==3.10 \ + --hash=sha256:12f65c9b470abda6dc35cf8e63cc574b1c52b11df2c86030af0ac09b01b13ea9 \ + --hash=sha256:946d195a0d259cbba61165e88e65941f16e9b36ea6ddb97f00452bae8b1287d3 + # via requests +mypy-extensions==1.1.0 \ + --hash=sha256:1be4cccdb0f2482337c4743e60421de3a356cd97508abadd57d47403e94f5505 \ + --hash=sha256:52e68efc3284861e772bbcd66823fde5ae21fd2fdb51c62a211403730b916558 + # via black +packaging==26.2 \ + --hash=sha256:5fc45236b9446107ff2415ce77c807cee2862cb6fac22b8a73826d0693b0980e \ + --hash=sha256:ff452ff5a3e828ce110190feff1178bb1f2ea2281fa2075aadb987c2fb221661 + # via black +pathspec==1.1.1 \ + --hash=sha256:17db5ecd524104a120e173814c90367a96a98d07c45b2e10c2f3919fff91bf5a \ + --hash=sha256:a00ce642f577bf7f473932318056212bc4f8bfdf53128c78bbd5af0b9b20b189 + # via black +platformdirs==4.10.0 \ + --hash=sha256:31e761a6a0ca04faf7353ea759bdba55652be214725111e5aac52dfa29d4bef7 \ + --hash=sha256:fb516cdb12eb0d857d0cd85a7c57cea4d060bee4578d6cf5a14dfdf8cbf8784a + # via black +pytokens==0.4.1 \ + --hash=sha256:0fc71786e629cef478cbf29d7ea1923299181d0699dbe7c3c0f4a583811d9fc1 \ + --hash=sha256:11edda0942da80ff58c4408407616a310adecae1ddd22eef8c692fe266fa5009 \ + --hash=sha256:140709331e846b728475786df8aeb27d24f48cbcf7bcd449f8de75cae7a45083 \ + --hash=sha256:24afde1f53d95348b5a0eb19488661147285ca4dd7ed752bbc3e1c6242a304d1 \ + --hash=sha256:26cef14744a8385f35d0e095dc8b3a7583f6c953c2e3d269c7f82484bf5ad2de \ + --hash=sha256:27b83ad28825978742beef057bfe406ad6ed524b2d28c252c5de7b4a6dd48fa2 \ + --hash=sha256:292052fe80923aae2260c073f822ceba21f3872ced9a68bb7953b348e561179a \ + --hash=sha256:29d1d8fb1030af4d231789959f21821ab6325e463f0503a61d204343c9b355d1 \ + --hash=sha256:2a44ed93ea23415c54f3face3b65ef2b844d96aeb3455b8a69b3df6beab6acc5 \ + --hash=sha256:30f51edd9bb7f85c748979384165601d028b84f7bd13fe14d3e065304093916a \ + --hash=sha256:34bcc734bd2f2d5fe3b34e7b3c0116bfb2397f2d9666139988e7a3eb5f7400e3 \ + --hash=sha256:3ad72b851e781478366288743198101e5eb34a414f1d5627cdd585ca3b25f1db \ + --hash=sha256:3f901fe783e06e48e8cbdc82d631fca8f118333798193e026a50ce1b3757ea68 \ + --hash=sha256:42f144f3aafa5d92bad964d471a581651e28b24434d184871bd02e3a0d956037 \ + --hash=sha256:4a14d5f5fc78ce85e426aa159489e2d5961acf0e47575e08f35584009178e321 \ + --hash=sha256:4a58d057208cb9075c144950d789511220b07636dd2e4708d5645d24de666bdc \ + --hash=sha256:4e691d7f5186bd2842c14813f79f8884bb03f5995f0575272009982c5ac6c0f7 \ + --hash=sha256:5502408cab1cb18e128570f8d598981c68a50d0cbd7c61312a90507cd3a1276f \ + --hash=sha256:584c80c24b078eec1e227079d56dc22ff755e0ba8654d8383b2c549107528918 \ + --hash=sha256:5ad948d085ed6c16413eb5fec6b3e02fa00dc29a2534f088d3302c47eb59adf9 \ + --hash=sha256:670d286910b531c7b7e3c0b453fd8156f250adb140146d234a82219459b9640c \ + --hash=sha256:682fa37ff4d8e95f7df6fe6fe6a431e8ed8e788023c6bcc0f0880a12eab80ad1 \ + --hash=sha256:6d6c4268598f762bc8e91f5dbf2ab2f61f7b95bdc07953b602db879b3c8c18e1 \ + --hash=sha256:79fc6b8699564e1f9b521582c35435f1bd32dd06822322ec44afdeba666d8cb3 \ + --hash=sha256:8bdb9d0ce90cbf99c525e75a2fa415144fd570a1ba987380190e8b786bc6ef9b \ + --hash=sha256:8fcb9ba3709ff77e77f1c7022ff11d13553f3c30299a9fe246a166903e9091eb \ + --hash=sha256:941d4343bf27b605e9213b26bfa1c4bf197c9c599a9627eb7305b0defcfe40c1 \ + --hash=sha256:967cf6e3fd4adf7de8fc73cd3043754ae79c36475c1c11d514fc72cf5490094a \ + --hash=sha256:970b08dd6b86058b6dc07efe9e98414f5102974716232d10f32ff39701e841c4 \ + --hash=sha256:97f50fd18543be72da51dd505e2ed20d2228c74e0464e4262e4899797803d7fa \ + --hash=sha256:9bd7d7f544d362576be74f9d5901a22f317efc20046efe2034dced238cbbfe78 \ + --hash=sha256:add8bf86b71a5d9fb5b89f023a80b791e04fba57960aa790cc6125f7f1d39dfe \ + --hash=sha256:b35d7e5ad269804f6697727702da3c517bb8a5228afa450ab0fa787732055fc9 \ + --hash=sha256:b49750419d300e2b5a3813cf229d4e5a4c728dae470bcc89867a9ad6f25a722d \ + --hash=sha256:d31b97b3de0f61571a124a00ffe9a81fb9939146c122c11060725bd5aea79975 \ + --hash=sha256:d70e77c55ae8380c91c0c18dea05951482e263982911fc7410b1ffd1dadd3440 \ + --hash=sha256:d9907d61f15bf7261d7e775bd5d7ee4d2930e04424bab1972591918497623a16 \ + --hash=sha256:da5baeaf7116dced9c6bb76dc31ba04a2dc3695f3d9f74741d7910122b456edc \ + --hash=sha256:dc74c035f9bfca0255c1af77ddd2d6ae8419012805453e4b0e7513e17904545d \ + --hash=sha256:dcafc12c30dbaf1e2af0490978352e0c4041a7cde31f4f81435c2a5e8b9cabb6 \ + --hash=sha256:ee44d0f85b803321710f9239f335aafe16553b39106384cef8e6de40cb4ef2f6 \ + --hash=sha256:f66a6bbe741bd431f6d741e617e0f39ec7257ca1f89089593479347cc4d13324 + # via black +requests==2.32.3 \ + --hash=sha256:55365417734eb18255590a9ff9eb97e9e1da868d4ccd6402399eaf68af20a760 \ + --hash=sha256:70761cfe03c773ceb22aa2f671b4757976145175cdfca038c02654d061d6dcc6 + # via test (tests/uv/lock/workspaces/pyproject.toml) +urllib3==2.3.0 \ + --hash=sha256:1cee9ad369867bfdbbb48b7dd50374c0967a0bb7710050facf0dd6911440e3df \ + --hash=sha256:f8c5449b3cf0861679ce7e0503c7b44b5ec981bec0d1d3795a07f1ba96f0204d + # via requests From 42c8e752e4f236b7fa0e88d77b751ea663f939d2 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Tue, 9 Jun 2026 02:55:45 -0700 Subject: [PATCH 758/922] feat(toolchains): support runtime registration from manifest (#3802) Currently, all supported Python runtime versions and their platform-specific metadata (URLs, SHA256s, strip_prefix) must be hardcoded in `python/versions.bzl`. This makes it slow and difficult to adopt new Python versions or custom builds without updating `rules_python` itself. This PR introduces the ability to dynamically fetch and register Python runtimes from a remote python-build-standalone (PBS) manifest file (e.g., `SHA256SUMS`). This is supported via two new attributes in `python.override`: - `add_runtime_manifest_urls`: A list of URLs pointing to manifest files to parse and register. - `runtime_manifest_sha`: The SHA256 hash of the manifest file. The manifest file format is the python-build-standalone SHA256SUMS format (`SHA FILENAME`), extended to allow arbitrary URLs for the filename (to allow more arbitrary locations). --- .bazelrc.deleted_packages | 1 + CHANGELOG.md | 3 + docs/toolchains.md | 70 +++++++ python/private/BUILD.bazel | 6 + python/private/pbs_manifest.bzl | 153 +++++++++++++++ python/private/python.bzl | 136 ++++++++++++- tests/integration/BUILD.bazel | 4 + tests/integration/runtime_manifests/.bazelrc | 4 + .../integration/runtime_manifests/BUILD.bazel | 7 + .../runtime_manifests/MODULE.bazel | 19 ++ tests/integration/runtime_manifests/WORKSPACE | 1 + .../runtime_manifests/basic_test.py | 27 +++ tests/python/BUILD.bazel | 4 +- tests/python/python_tests.bzl | 15 ++ tests/python_bzlmod_ext/BUILD.bazel | 7 + .../parse_sha_manifest_tests.bzl | 183 ++++++++++++++++++ .../runtime_manifests_tests.bzl | 163 ++++++++++++++++ tests/python_bzlmod_ext/test_helpers.bzl | 48 +++++ tests/support/mocks/python_ext.bzl | 1 + .../private/debug/print_defined_toolchains.sh | 14 ++ 20 files changed, 862 insertions(+), 4 deletions(-) create mode 100644 python/private/pbs_manifest.bzl create mode 100644 tests/integration/runtime_manifests/.bazelrc create mode 100644 tests/integration/runtime_manifests/BUILD.bazel create mode 100644 tests/integration/runtime_manifests/MODULE.bazel create mode 100644 tests/integration/runtime_manifests/WORKSPACE create mode 100644 tests/integration/runtime_manifests/basic_test.py create mode 100644 tests/python_bzlmod_ext/BUILD.bazel create mode 100644 tests/python_bzlmod_ext/parse_sha_manifest_tests.bzl create mode 100644 tests/python_bzlmod_ext/runtime_manifests_tests.bzl create mode 100644 tests/python_bzlmod_ext/test_helpers.bzl create mode 100755 tools/private/debug/print_defined_toolchains.sh diff --git a/.bazelrc.deleted_packages b/.bazelrc.deleted_packages index 7256937a87..407fd1cb48 100644 --- a/.bazelrc.deleted_packages +++ b/.bazelrc.deleted_packages @@ -38,6 +38,7 @@ common --deleted_packages=tests/integration/pip_parse common --deleted_packages=tests/integration/pip_parse/empty common --deleted_packages=tests/integration/pip_parse_isolated common --deleted_packages=tests/integration/py_cc_toolchain_registered +common --deleted_packages=tests/integration/runtime_manifests common --deleted_packages=tests/integration/toolchain_target_settings common --deleted_packages=tests/integration/uv_lock common --deleted_packages=tests/modules/another_module diff --git a/CHANGELOG.md b/CHANGELOG.md index 99876bde0d..43e05b6dd0 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -114,6 +114,9 @@ END_UNRELEASED_TEMPLATE {#v0-0-0-added} ### Added +* (toolchains) Support dynamically fetching and registering Python runtimes + from a python-build-standalone manifest file using + `python.override(add_runtime_manifest_urls = ..., runtime_manifest_sha = ...)`. * (toolchain) Added {obj}`python.override.toolchain_target_settings` to allow adding `config_setting` labels to all registered toolchains. * (windows) Full venv support for Windows is available. Set diff --git a/docs/toolchains.md b/docs/toolchains.md index 09aaed412b..80884baedf 100644 --- a/docs/toolchains.md +++ b/docs/toolchains.md @@ -242,6 +242,9 @@ existing attributes: {attr}`python.single_version_platform_override.coverage_tool`. * Adding additional Python versions via {bzl:obj}`python.single_version_override` or {bzl:obj}`python.single_version_platform_override`. +* Adding additional Python versions dynamically from a manifest file or URL + via {attr}`python.override.add_runtime_manifest_files` or + {attr}`python.override.add_runtime_manifest_urls`. ### Registering custom runtimes @@ -310,6 +313,73 @@ Added support for custom platform names, `target_compatible_with`, and `target_settings` with `single_version_platform_override`. ::: +### Registering runtimes from a manifest + +If you want to register multiple custom runtimes or versions at once, you can +use a python-build-standalone manifest file. This is useful if you want to +adopt new versions that are not yet built into `rules_python` without having +to manually define each one using `single_version_platform_override`. + +To do this, specify the `add_runtime_manifest_files` or +`add_runtime_manifest_urls` (and `runtime_manifest_sha`) attributes in +`python.override` in your `MODULE.bazel`. + +In the example below, we register all runtimes available in a specific local +or remote PBS release manifest: + +```starlark +# File: MODULE.bazel +python = use_extension("@rules_python//python/extensions:python.bzl", "python") +python.override( + add_runtime_manifest_files = [ + "@//:SHA256SUMS", + ], + add_runtime_manifest_urls = [ + "https://github.com/astral-sh/python-build-standalone/releases/download/20260414/SHA256SUMS", + ], + base_url = "https://example.com/downloads", + runtime_manifest_sha = "ce18fdfd47c66830a40ea9b9e314a14b1636bbfd684501bc5ca1fc6d55a7933f", +) +``` + +#### Manifest file format + +The manifest must be a plain text file where each line contains the SHA256 hash +and the location of a runtime archive, separated by whitespace: + +``` + +``` + +The `` can be either: +- A relative filename (e.g., + `cpython-3.10.20+20260414-x86_64-unknown-linux-gnu-install_only.tar.zst`). + In this case, the download URL is constructed by appending the filename to + the `base_url` attribute (if using `add_runtime_manifest_files`) or to the + parent directory of each URL in `add_runtime_manifest_urls` (treating them + as mirrors). +- An absolute URL (e.g., + `https://example.com/downloads/cpython-3.10.20+20260414-x86_64-unknown-linux-gnu-install_only.tar.zst`). + In this case, the URL is used directly to download the archive. + +In both cases, the filename or the last path segment of the URL must follow +the standard python-build-standalone naming convention. `rules_python` parses +this name to extract runtime metadata (such as Python version, target +architecture, operating system, and libc). + +Notes: +- `rules_python` will read or download the manifest, parse it, and + automatically register toolchains for all valid Python runtimes found in it + that match supported platforms. +- Only runtimes matching known platforms in `rules_python` will be registered. + +:::{versionadded} VERSION_NEXT_FEATURE +Added support for registering runtimes from a manifest using +`add_runtime_manifest_files`, `add_runtime_manifest_urls`, and +`runtime_manifest_sha` in `python.override`. +::: + + ### Using defined toolchains from WORKSPACE It is possible to use toolchains defined in `MODULE.bazel` in `WORKSPACE`. For example, diff --git a/python/private/BUILD.bazel b/python/private/BUILD.bazel index 6ab3d546be..b54c198069 100644 --- a/python/private/BUILD.bazel +++ b/python/private/BUILD.bazel @@ -252,6 +252,11 @@ bzl_library( srcs = ["normalize_name.bzl"], ) +bzl_library( + name = "pbs_manifest_bzl", + srcs = ["pbs_manifest.bzl"], +) + bzl_library( name = "precompile_bzl", srcs = ["precompile.bzl"], @@ -274,6 +279,7 @@ bzl_library( srcs = ["python.bzl"], deps = [ ":full_version_bzl", + ":pbs_manifest_bzl", ":platform_info_bzl", ":python_register_toolchains_bzl", ":pythons_hub_bzl", diff --git a/python/private/pbs_manifest.bzl b/python/private/pbs_manifest.bzl new file mode 100644 index 0000000000..e343a802b3 --- /dev/null +++ b/python/private/pbs_manifest.bzl @@ -0,0 +1,153 @@ +"""Helper functions to parse python-build-standalone manifests.""" + +def parse_filename(filename): + """Parses a python-build-standalone filename (or URL) into its components. + + See https://gregoryszorc.com/docs/python-build-standalone/main/running.html + + Example: cpython-3.10.20+20260414-x86_64_v2-unknown-linux-musl-lto-full.tar.zst + + Args: + filename: The filename or URL of the python-build-standalone release asset. + + Returns: + A dictionary of parsed components if parsed successfully, else None. + """ + basename = filename.rpartition("/")[-1] + if basename.endswith(".tar.zst"): + name = basename.removesuffix(".tar.zst") + elif basename.endswith(".tar.gz"): + name = basename.removesuffix(".tar.gz") + else: + return None + + if not name.startswith("cpython-"): + return None + name = name.removeprefix("cpython-") + + left, plus, tail = name.partition("+") + if plus: + python_version = left + build_version, sep, rest = tail.partition("-") + if not sep: + return None + else: + python_version, sep, rest = left.partition("-") + if not sep: + return None + build_version = "" + + arch, sep, rest = rest.partition("-") + if not sep: + return None + + microarch = "" + arch_base, sep_v, microarch_num = arch.partition("_v") + if sep_v: + arch = arch_base + microarch = "v" + microarch_num + + vendor, sep, rest = rest.partition("-") + if not sep: + return None + + os, sep, rest = rest.partition("-") + if not sep: + return None + + libc = "" + next_part, _, remaining = rest.partition("-") + if os == "linux" and next_part in ["gnu", "musl"]: + libc = next_part + flavor = remaining + elif os == "windows" and next_part == "msvc": + libc = next_part + flavor = remaining + else: + libc = "" + flavor = rest + + freethreaded = False + if flavor.startswith("freethreaded+"): + freethreaded = True + flavor = flavor.removeprefix("freethreaded+") + elif flavor.startswith("freethreaded-"): + freethreaded = True + flavor = flavor.removeprefix("freethreaded-") + elif flavor == "freethreaded": + freethreaded = True + flavor = "" + + archive_flavor = "" + if flavor.endswith("-full"): + archive_flavor = "full" + flavor = flavor.removesuffix("-full") + elif flavor == "full": + archive_flavor = "full" + flavor = "" + elif flavor.endswith("-install_only_stripped"): + archive_flavor = "install_only_stripped" + flavor = flavor.removesuffix("-install_only_stripped") + elif flavor == "install_only_stripped": + archive_flavor = "install_only_stripped" + flavor = "" + elif flavor.endswith("-install_only"): + archive_flavor = "install_only" + flavor = flavor.removesuffix("-install_only") + elif flavor == "install_only": + archive_flavor = "install_only" + flavor = "" + + return { + "arch": arch, + "archive_flavor": archive_flavor, + "build_version": build_version, + "flavor": flavor, + "freethreaded": freethreaded, + "libc": libc, + "location": filename, + "microarch": microarch, + "os": os, + "python_version": python_version, + "vendor": vendor, + } + +def parse_sha_manifest(content): + """Parses the SHA256SUMS file content into a list of structs. + + Args: + content: The raw content of the manifest file. + + Returns: + A list of structs capturing the parsed components of each valid entry. + Each struct contains the following fields: + - arch: CPU architecture (e.g., "x86_64"). + - archive_flavor: Release asset archive type (e.g., "full", "install_only"). + - build_version: Standalone release date (e.g., "20260414"). + - location: Full package filename or URL (e.g., "cpython-3.11.15..." or "https://..."). + - flavor: Build configuration flavor (e.g., "install_only"). + - freethreaded: Whether the build is free-threaded (boolean). + - libc: C library type (e.g., "gnu", "musl", "msvc", or ""). + - microarch: Microarchitecture level (e.g., "v2", "v3", or ""). + - os: Operating system (e.g., "linux", "darwin", "windows"). + - python_version: Python semver version (e.g., "3.11.15"). + - sha256: SHA256 integrity hash of the release asset. + - vendor: Platform vendor (e.g., "unknown", "apple"). + """ + results = [] + for line in content.split("\n"): + line = line.strip() + if not line: + continue + parts = [p for p in line.split(" ") if p] + if len(parts) != 2: + continue + sha256, filename = parts + + parsed = parse_filename(filename) + if parsed: + results.append(struct( + sha256 = sha256, + **parsed + )) + return results diff --git a/python/private/python.bzl b/python/private/python.bzl index 6abc81e3d2..73b2d19839 100644 --- a/python/private/python.bzl +++ b/python/private/python.bzl @@ -18,6 +18,7 @@ load("@bazel_features//:features.bzl", "bazel_features") load("//python:versions.bzl", "DEFAULT_RELEASE_BASE_URL", "PLATFORMS", "TOOL_VERSIONS") load(":auth.bzl", "AUTH_ATTRS") load(":full_version.bzl", "full_version") +load(":pbs_manifest.bzl", "parse_sha_manifest") load(":platform_info.bzl", "platform_info") load(":python_register_toolchains.bzl", "python_register_toolchains") load(":pythons_hub.bzl", "hub_repo") @@ -76,7 +77,7 @@ def parse_modules(*, module_ctx, logger = None, _fail = fail): # Map of string Major.Minor or Major.Minor.Patch to the toolchain_info struct global_toolchain_versions = {} - config = _get_toolchain_config(modules = module_ctx.modules, _fail = _fail) + config = _get_toolchain_config(mctx = module_ctx, modules = module_ctx.modules, _fail = _fail) default_python_version = _compute_default_python_version(module_ctx) @@ -741,10 +742,89 @@ def _override_defaults(*overrides, modules, _fail = fail, default): override.fn(tag = tag, _fail = _fail, default = default) -def _get_toolchain_config(*, modules, _fail = fail): +def _populate_from_pbs_manifest( + *, + mctx, + add_runtime_manifest_urls = [], + add_runtime_manifest_files = [], + runtime_manifest_sha = "", + base_url = "", + available_versions, + _fail): + manifest_contents = [] + + if add_runtime_manifest_urls: + manifest_path = mctx.path("runtime_manifest") + result = mctx.download( + url = add_runtime_manifest_urls, + output = manifest_path, + sha256 = runtime_manifest_sha, + ) + if not result.success: + _fail("Failed to download manifest from {}: {}".format(add_runtime_manifest_urls, result)) + return + manifest_contents.append(mctx.read(manifest_path)) + + for manifest_file in add_runtime_manifest_files: + manifest_contents.append(mctx.read(manifest_file, watch = "yes")) + + if not manifest_contents: + return + + base_download_urls = [url.rpartition("/")[0] for url in add_runtime_manifest_urls] + if not base_download_urls and base_url: + base_download_urls = [base_url] + + entries = [] + for content in manifest_contents: + entries.extend(parse_sha_manifest(content)) + + # We don't model archive_flavor via flags yet, so have to pick one. + # Preference is given to install_only because its smaller + entries = sorted( + entries, + key = lambda e: {"full": 3, "install_only": 1, "install_only_stripped": 2}.get(e.archive_flavor, 4), + ) + + for entry in entries: + location = entry.location + sha256 = entry.sha256 + py_version = entry.python_version + + # Fallback to matching against PLATFORMS keys as before to ensure compatibility + # with rules_python expected platform keys. + matched_platform = None + for platform in PLATFORMS.keys(): + if platform in location: + matched_platform = platform + break + + if not matched_platform: + continue + + if entry.archive_flavor not in ["install_only", "install_only_stripped", "full"]: + continue + + v_dict = available_versions.setdefault(py_version, {}) + if matched_platform in v_dict.get("sha256", {}): + continue + + if "://" in location: + urls = [location] + else: + urls = ["{}/{}".format(b_url, location) for b_url in base_download_urls] + + strip_prefix = "python/install" if entry.archive_flavor == "full" else "python" + + v_dict.setdefault("sha256", {})[matched_platform] = sha256 + v_dict.setdefault("url", {})[matched_platform] = urls + v_dict.setdefault("strip_prefix", {})[matched_platform] = strip_prefix + +def _get_toolchain_config(*, mctx, modules, _fail = fail): """Computes the configs for toolchains. Args: + mctx: The module context. modules: The modules from module_ctx _fail: Function to call for failing; only used for testing. @@ -786,6 +866,21 @@ def _get_toolchain_config(*, modules, _fail = fail): else: available_versions[py_version]["url"] = dict(url) + # Check for add_runtime_manifest_urls or add_runtime_manifest_files in override tags in root module + root_module = modules[0] if modules else None + if root_module and root_module.is_root: + for tag in root_module.tags.override: + if tag.add_runtime_manifest_urls or tag.add_runtime_manifest_files: + _populate_from_pbs_manifest( + mctx = mctx, + add_runtime_manifest_urls = tag.add_runtime_manifest_urls, + add_runtime_manifest_files = tag.add_runtime_manifest_files, + runtime_manifest_sha = tag.runtime_manifest_sha, + base_url = tag.base_url, + available_versions = available_versions, + _fail = _fail, + ) + default = { "base_url": DEFAULT_RELEASE_BASE_URL, "platforms": dict(PLATFORMS), # Copy so it's mutable. @@ -1111,6 +1206,34 @@ _override = tag_class( ::: """, attrs = { + "add_runtime_manifest_files": attr.label_list( + mandatory = False, + allow_files = True, + doc = """ +Labels pointing to local python-build-standalone manifest files (e.g., `SHA256SUMS`). + +Example: +`//my/custom/manifest:SHA256SUMS` + +:::{versionadded} VERSION_NEXT_FEATURE +::: +""", + ), + "add_runtime_manifest_urls": attr.string_list( + mandatory = False, + doc = """ +URLs pointing to python-build-standalone manifest files (e.g., SHA256SUMS). + +Example: +`https://github.com/astral-sh/python-build-standalone/releases/download/20260414/SHA256SUMS` + +Note that `/latest/` can be used in place of a specific release date (e.g., `20260414`) to automatically use the latest release: +`https://github.com/astral-sh/python-build-standalone/releases/latest/download/SHA256SUMS` + +:::{versionadded} VERSION_NEXT_FEATURE +::: +""", + ), "add_target_settings": attr.string_list( mandatory = False, doc = """\ @@ -1180,6 +1303,15 @@ The values in this mapping override the default values and do not replace them. default = {}, ), "register_all_versions": attr.bool(default = False, doc = "Add all versions"), + "runtime_manifest_sha": attr.string( + mandatory = False, + doc = """ +SHA256 hash for the add_runtime_manifest_urls. + +:::{versionadded} VERSION_NEXT_FEATURE +::: +""", + ), } | AUTH_ATTRS, ) diff --git a/tests/integration/BUILD.bazel b/tests/integration/BUILD.bazel index a6027fc3d4..904fb4c247 100644 --- a/tests/integration/BUILD.bazel +++ b/tests/integration/BUILD.bazel @@ -104,6 +104,10 @@ rules_python_integration_test( workspace_path = "py_cc_toolchain_registered", ) +rules_python_integration_test( + name = "runtime_manifests_test", +) + rules_python_integration_test( name = "custom_commands_test", py_main = "custom_commands_test.py", diff --git a/tests/integration/runtime_manifests/.bazelrc b/tests/integration/runtime_manifests/.bazelrc new file mode 100644 index 0000000000..d4d45a5ea7 --- /dev/null +++ b/tests/integration/runtime_manifests/.bazelrc @@ -0,0 +1,4 @@ +# Copy of fast-tests config +common:fast-tests --build_tests_only=true +common:fast-tests --build_tag_filters=-large,-enormous,-integration-test +common:fast-tests --test_tag_filters=-large,-enormous,-integration-test diff --git a/tests/integration/runtime_manifests/BUILD.bazel b/tests/integration/runtime_manifests/BUILD.bazel new file mode 100644 index 0000000000..4746fd419b --- /dev/null +++ b/tests/integration/runtime_manifests/BUILD.bazel @@ -0,0 +1,7 @@ +load("@rules_python//python:py_test.bzl", "py_test") + +py_test( + name = "basic_test", + srcs = ["basic_test.py"], + python_version = "3.11", +) diff --git a/tests/integration/runtime_manifests/MODULE.bazel b/tests/integration/runtime_manifests/MODULE.bazel new file mode 100644 index 0000000000..6891b07818 --- /dev/null +++ b/tests/integration/runtime_manifests/MODULE.bazel @@ -0,0 +1,19 @@ +module(name = "runtime_manifests") + +bazel_dep(name = "rules_python", version = "0.0.0") +local_path_override( + module_name = "rules_python", + path = "../../..", +) + +python = use_extension("@rules_python//python/extensions:python.bzl", "python") +python.override( + add_runtime_manifest_urls = [ + "https://github.com/astral-sh/python-build-standalone/releases/download/20260414/SHA256SUMS", + ], + register_all_versions = True, + runtime_manifest_sha = "ce18fdfd47c66830a40ea9b9e314a14b1636bbfd684501bc5ca1fc6d55a7933f", +) +python.toolchain( + python_version = "3.11.15", +) diff --git a/tests/integration/runtime_manifests/WORKSPACE b/tests/integration/runtime_manifests/WORKSPACE new file mode 100644 index 0000000000..8277ec8090 --- /dev/null +++ b/tests/integration/runtime_manifests/WORKSPACE @@ -0,0 +1 @@ +# Workspace boundary file required by rules_bazel_integration_test diff --git a/tests/integration/runtime_manifests/basic_test.py b/tests/integration/runtime_manifests/basic_test.py new file mode 100644 index 0000000000..35f0e93b6e --- /dev/null +++ b/tests/integration/runtime_manifests/basic_test.py @@ -0,0 +1,27 @@ +import datetime +import platform +import sys +import unittest + + +class BasicTest(unittest.TestCase): + def test_basic(self): + print("Hello World from Python {}!".format(sys.version)) + print("Interpreter executable path: {}".format(sys.executable)) + + # Verify that the hermetic interpreter inside Bazel's output/sandbox tree is used + self.assertIn(".cache/bazel", sys.executable) + + # Verify that the exact custom version (3.11.15) parsed from the manifest is used + self.assertEqual(sys.version_info[:3], (3, 11, 15)) + + # Verify that the exact build version (20260414) parsed from the manifest is used + buildno, builddate = platform.python_build() + date_str = " ".join(builddate.split()[:3]) + dt = datetime.datetime.strptime(date_str, "%b %d %Y") + formatted_date = dt.strftime("%Y%m%d") + self.assertEqual(formatted_date, "20260414") + + +if __name__ == "__main__": + unittest.main() diff --git a/tests/python/BUILD.bazel b/tests/python/BUILD.bazel index 2553536b63..887fe969b5 100644 --- a/tests/python/BUILD.bazel +++ b/tests/python/BUILD.bazel @@ -12,6 +12,6 @@ # See the License for the specific language governing permissions and # limitations under the License. -load(":python_tests.bzl", "python_test_suite") +load(":python_tests.bzl", "register_python_tests") -python_test_suite(name = "python_tests") +register_python_tests(name = "python_tests") diff --git a/tests/python/python_tests.bzl b/tests/python/python_tests.bzl index 5db74265be..cd7383942c 100644 --- a/tests/python/python_tests.bzl +++ b/tests/python/python_tests.bzl @@ -16,6 +16,7 @@ load("@pythons_hub//:versions.bzl", "MINOR_MAPPING") load("@rules_testing//lib:test_suite.bzl", "test_suite") +load("//python/private:bzlmod_enabled.bzl", "BZLMOD_ENABLED") # buildifier: disable=bzl-visibility load("//python/private:python.bzl", "parse_modules") # buildifier: disable=bzl-visibility load("//python/private:repo_utils.bzl", "repo_utils") # buildifier: disable=bzl-visibility load("//tests/support/mocks:mocks.bzl", "mocks") @@ -878,3 +879,17 @@ def python_test_suite(name): name: the name of the test suite """ test_suite(name = name, basic_tests = _tests) + +def register_python_tests(name): + """Registers the python tests if Bzlmod is enabled, otherwise defines an empty test_suite. + + Args: + name: The name of the test target. + """ + if BZLMOD_ENABLED: + python_test_suite(name = name) + else: + native.test_suite( + name = name, + tests = [], + ) diff --git a/tests/python_bzlmod_ext/BUILD.bazel b/tests/python_bzlmod_ext/BUILD.bazel new file mode 100644 index 0000000000..0add4e7690 --- /dev/null +++ b/tests/python_bzlmod_ext/BUILD.bazel @@ -0,0 +1,7 @@ +load(":test_helpers.bzl", "register_python_bzlmod_ext_tests") + +register_python_bzlmod_ext_tests( + name = "python_bzlmod_ext_tests", + parse_sha_manifest_name = "parse_sha_manifest_tests", + runtime_manifests_name = "runtime_manifests_tests", +) diff --git a/tests/python_bzlmod_ext/parse_sha_manifest_tests.bzl b/tests/python_bzlmod_ext/parse_sha_manifest_tests.bzl new file mode 100644 index 0000000000..4ddcdcc46e --- /dev/null +++ b/tests/python_bzlmod_ext/parse_sha_manifest_tests.bzl @@ -0,0 +1,183 @@ +"""Tests for manifest parsing Starlark functions.""" + +load("@bazel_skylib//lib:structs.bzl", "structs") +load("@rules_testing//lib:analysis_test.bzl", "analysis_test") +load("@rules_testing//lib:test_suite.bzl", "test_suite") +load("@rules_testing//lib:util.bzl", rt_util = "util") +load("//python/private:pbs_manifest.bzl", "parse_filename", "parse_sha_manifest") # buildifier: disable=bzl-visibility + +_tests = [] + +def _test_parse_filename_baseline(name): + """Sets up the baseline filename parsing test. + + Args: + name: The name of the test. + """ + rt_util.helper_target( + native.filegroup, + name = name + "_subject", + ) + analysis_test( + name = name, + target = name + "_subject", + impl = _test_parse_filename_baseline_impl, + ) + +def _test_parse_filename_baseline_impl(env, target): + _ = target # @unused + + # 1. Baseline + parsed1 = parse_filename("cpython-3.11.15+20260414-x86_64-unknown-linux-gnu-install_only.tar.gz") + env.expect.that_dict(parsed1).contains_exactly({ + "arch": "x86_64", + "archive_flavor": "install_only", + "build_version": "20260414", + "flavor": "", + "freethreaded": False, + "libc": "gnu", + "location": "cpython-3.11.15+20260414-x86_64-unknown-linux-gnu-install_only.tar.gz", + "microarch": "", + "os": "linux", + "python_version": "3.11.15", + "vendor": "unknown", + }) + + # 2. Microarch + parsed2 = parse_filename("cpython-3.10.20+20260414-x86_64_v2-unknown-linux-musl-lto-full.tar.zst") + env.expect.that_dict(parsed2).contains_exactly({ + "arch": "x86_64", + "archive_flavor": "full", + "build_version": "20260414", + "flavor": "lto", + "freethreaded": False, + "libc": "musl", + "location": "cpython-3.10.20+20260414-x86_64_v2-unknown-linux-musl-lto-full.tar.zst", + "microarch": "v2", + "os": "linux", + "python_version": "3.10.20", + "vendor": "unknown", + }) + + # 3. Freethreaded + parsed3 = parse_filename("cpython-3.13.13+20260414-aarch64-apple-darwin-freethreaded+pgo+lto-full.tar.zst") + env.expect.that_dict(parsed3).contains_exactly({ + "arch": "aarch64", + "archive_flavor": "full", + "build_version": "20260414", + "flavor": "pgo+lto", + "freethreaded": True, + "libc": "", + "location": "cpython-3.13.13+20260414-aarch64-apple-darwin-freethreaded+pgo+lto-full.tar.zst", + "microarch": "", + "os": "darwin", + "python_version": "3.13.13", + "vendor": "apple", + }) + + # 4. Invalid + parsed4 = parse_filename("invalid-filename.tar.gz") + env.expect.that_bool(parsed4 == None).equals(True) + + # 5. Full URL (should return the original URL as location) + parsed5 = parse_filename("https://github.com/astral-sh/python-build-standalone/releases/download/20260414/cpython-3.11.15+20260414-x86_64-unknown-linux-gnu-install_only.tar.gz") + env.expect.that_dict(parsed5).contains_exactly({ + "arch": "x86_64", + "archive_flavor": "install_only", + "build_version": "20260414", + "flavor": "", + "freethreaded": False, + "libc": "gnu", + "location": "https://github.com/astral-sh/python-build-standalone/releases/download/20260414/cpython-3.11.15+20260414-x86_64-unknown-linux-gnu-install_only.tar.gz", + "microarch": "", + "os": "linux", + "python_version": "3.11.15", + "vendor": "unknown", + }) + +_tests.append(_test_parse_filename_baseline) + +def _test_parse_sha_manifest(name): + """Sets up the manifest file parsing test. + + Args: + name: The name of the test. + """ + rt_util.helper_target( + native.filegroup, + name = name + "_subject", + ) + analysis_test( + name = name, + target = name + "_subject", + impl = _test_parse_sha_manifest_impl, + ) + +def _test_parse_sha_manifest_impl(env, target): + _ = target # @unused + content = """ +8b14030dd3af9ea7f7c51b4c90feb04afd8a8f45435727e67b875270bd08f3bc cpython-3.11.15+20260414-x86_64-unknown-linux-gnu-install_only.tar.gz +a57ffd435652092d16b30e783f9826c55e9c64b0f0a72cbae0a9f39e663137fb cpython-3.11.15+20260414-aarch64-apple-darwin-install_only.tar.gz +ce18fdfd47c66830a40ea9b9e314a14b1636bbfd684501bc5ca1fc6d55a7933f https://example.com/cpython-3.10.20+20260414-x86_64_v2-unknown-linux-musl-lto-full.tar.zst +1111111111111111111111111111111111111111111111111111111111111111 cpython-3.13.13+20260414-aarch64-apple-darwin-freethreaded+pgo+lto-full.tar.zst +""" + parsed = parse_sha_manifest(content) + env.expect.that_collection(parsed).has_size(4) + + env.expect.that_dict(structs.to_dict(parsed[0])).contains_exactly({ + "arch": "x86_64", + "archive_flavor": "install_only", + "build_version": "20260414", + "flavor": "", + "freethreaded": False, + "libc": "gnu", + "location": "cpython-3.11.15+20260414-x86_64-unknown-linux-gnu-install_only.tar.gz", + "microarch": "", + "os": "linux", + "python_version": "3.11.15", + "sha256": "8b14030dd3af9ea7f7c51b4c90feb04afd8a8f45435727e67b875270bd08f3bc", + "vendor": "unknown", + }) + + env.expect.that_dict(structs.to_dict(parsed[2])).contains_exactly({ + "arch": "x86_64", + "archive_flavor": "full", + "build_version": "20260414", + "flavor": "lto", + "freethreaded": False, + "libc": "musl", + "location": "https://example.com/cpython-3.10.20+20260414-x86_64_v2-unknown-linux-musl-lto-full.tar.zst", + "microarch": "v2", + "os": "linux", + "python_version": "3.10.20", + "sha256": "ce18fdfd47c66830a40ea9b9e314a14b1636bbfd684501bc5ca1fc6d55a7933f", + "vendor": "unknown", + }) + + env.expect.that_dict(structs.to_dict(parsed[3])).contains_exactly({ + "arch": "aarch64", + "archive_flavor": "full", + "build_version": "20260414", + "flavor": "pgo+lto", + "freethreaded": True, + "libc": "", + "location": "cpython-3.13.13+20260414-aarch64-apple-darwin-freethreaded+pgo+lto-full.tar.zst", + "microarch": "", + "os": "darwin", + "python_version": "3.13.13", + "sha256": "1111111111111111111111111111111111111111111111111111111111111111", + "vendor": "apple", + }) + +_tests.append(_test_parse_sha_manifest) + +def parse_sha_manifest_test_suite(name): + """Defines the test suite for manifest parsing. + + Args: + name: The name of the test suite. + """ + test_suite( + name = name, + tests = _tests, + ) diff --git a/tests/python_bzlmod_ext/runtime_manifests_tests.bzl b/tests/python_bzlmod_ext/runtime_manifests_tests.bzl new file mode 100644 index 0000000000..c46cab1958 --- /dev/null +++ b/tests/python_bzlmod_ext/runtime_manifests_tests.bzl @@ -0,0 +1,163 @@ +"""Starlark unit tests for dynamic toolchain registration via manifests.""" + +load("@rules_testing//lib:analysis_test.bzl", "analysis_test") +load("@rules_testing//lib:test_suite.bzl", "test_suite") +load("@rules_testing//lib:util.bzl", rt_util = "util") +load("//python/private:python.bzl", "parse_modules") # buildifier: disable=bzl-visibility +load("//python/private:repo_utils.bzl", "repo_utils") # buildifier: disable=bzl-visibility +load("//tests/support/mocks:mocks.bzl", "mocks") # buildifier: disable=bzl-visibility +load("//tests/support/mocks:python_ext.bzl", "python_ext") # buildifier: disable=bzl-visibility + +_tests = [] + +_mock_logger = repo_utils.logger( + name = "mock", + verbosity_level = "ERROR", +) + +def _test_dynamic_manifest_toolchains(name): + rt_util.helper_target( + native.filegroup, + name = name + "_subject", + ) + analysis_test( + name = name, + target = name + "_subject", + impl = _test_dynamic_manifest_toolchains_impl, + ) + +def _test_dynamic_manifest_toolchains_impl(env, target): + _ = target # @unused + + # Construct Bzlmod mock module locally inside the test execution block. + # We test using virtual patch version "3.11.99" (not present in TOOL_VERSIONS) + # so that the populated config contains ONLY our dynamically parsed manifest keys + # without any pre-populated multi-platform templates, allowing exact dictionary match! + root_module = python_ext.module( + name = "runtime_manifests", + override = [ + python_ext.override( + add_runtime_manifest_urls = [ + "https://github.com/astral-sh/python-build-standalone/releases/download/20260414/SHA256SUMS", + ], + runtime_manifest_sha = "ce18fdfd47c66830a40ea9b9e314a14b1636bbfd684501bc5ca1fc6d55a7933f", + register_all_versions = True, + ), + ], + defaults = [ + python_ext.defaults( + python_version = "3.11.99", + ), + ], + ) + + # Pre-populate mock_files directly to bypass download output struct key mismatch in mock read lookups. + mock_mctx = mocks.mctx( + modules = [root_module], + mock_files = { + "runtime_manifest": """ +01e607cf764b97d4d5d6f69fd1ff3d8a9a162513dde5c39e98260fce40fe220a cpython-3.11.99+20260414-x86_64-unknown-linux-gnu-pgo+lto-full.tar.zst +8b14030dd3af9ea7f7c51b4c90feb04afd8a8f45435727e67b875270bd08f3bc cpython-3.11.99+20260414-x86_64-unknown-linux-gnu-install_only.tar.gz +""", + }, + ) + + res = parse_modules( + module_ctx = mock_mctx, + logger = _mock_logger, + ) + + tool_versions = res.config.default["tool_versions"] + env.expect.that_bool("3.11.99" in tool_versions).equals(True) + + version_info = tool_versions["3.11.99"] + + # Assert on the entire dictionary at once! + env.expect.that_dict(version_info).contains_exactly({ + "sha256": { + "x86_64-unknown-linux-gnu": "8b14030dd3af9ea7f7c51b4c90feb04afd8a8f45435727e67b875270bd08f3bc", + }, + "strip_prefix": { + "x86_64-unknown-linux-gnu": "python", + }, + "url": { + "x86_64-unknown-linux-gnu": [ + "https://github.com/astral-sh/python-build-standalone/releases/download/20260414/cpython-3.11.99+20260414-x86_64-unknown-linux-gnu-install_only.tar.gz", + ], + }, + }) + +_tests.append(_test_dynamic_manifest_toolchains) + +def _test_dynamic_manifest_files(name): + rt_util.helper_target( + native.filegroup, + name = name + "_subject", + ) + analysis_test( + name = name, + target = name + "_subject", + impl = _test_dynamic_manifest_files_impl, + ) + +def _test_dynamic_manifest_files_impl(env, target): + _ = target # @unused + + root_module = python_ext.module( + name = "runtime_manifests", + override = [ + python_ext.override( + add_runtime_manifest_files = [ + Label("//:SHA256SUMS"), + ], + base_url = "https://example.com/dl", + register_all_versions = True, + ), + ], + defaults = [ + python_ext.defaults( + python_version = "3.12.99", + ), + ], + ) + + mock_mctx = mocks.mctx( + modules = [root_module], + mock_files = { + str(Label("//:SHA256SUMS")): """ +01e607cf764b97d4d5d6f69fd1ff3d8a9a162513dde5c39e98260fce40fe220a cpython-3.12.99+20260414-x86_64-unknown-linux-gnu-pgo+lto-full.tar.zst +""", + }, + ) + + res = parse_modules( + module_ctx = mock_mctx, + logger = _mock_logger, + ) + + tool_versions = res.config.default["tool_versions"] + env.expect.that_bool("3.12.99" in tool_versions).equals(True) + + version_info = tool_versions["3.12.99"] + + env.expect.that_dict(version_info).contains_exactly({ + "sha256": { + "x86_64-unknown-linux-gnu": "01e607cf764b97d4d5d6f69fd1ff3d8a9a162513dde5c39e98260fce40fe220a", + }, + "strip_prefix": { + "x86_64-unknown-linux-gnu": "python/install", + }, + "url": { + "x86_64-unknown-linux-gnu": [ + "https://example.com/dl/cpython-3.12.99+20260414-x86_64-unknown-linux-gnu-pgo+lto-full.tar.zst", + ], + }, + }) + +_tests.append(_test_dynamic_manifest_files) + +def runtime_manifests_test_suite(name): + test_suite( + name = name, + tests = _tests, + ) diff --git a/tests/python_bzlmod_ext/test_helpers.bzl b/tests/python_bzlmod_ext/test_helpers.bzl new file mode 100644 index 0000000000..f6c177650a --- /dev/null +++ b/tests/python_bzlmod_ext/test_helpers.bzl @@ -0,0 +1,48 @@ +# Copyright 2026 The Bazel Authors. All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +"""Helpers to conditionally register tests depending on Bzlmod enablement.""" + +load("//python/private:bzlmod_enabled.bzl", "BZLMOD_ENABLED") # buildifier: disable=bzl-visibility +load(":parse_sha_manifest_tests.bzl", "parse_sha_manifest_test_suite") +load(":runtime_manifests_tests.bzl", "runtime_manifests_test_suite") + +def register_python_bzlmod_ext_tests(name, parse_sha_manifest_name, runtime_manifests_name): + """Registers the Bzlmod extension tests if Bzlmod is enabled, otherwise defines empty test_suites. + + Args: + name: The name of the master test_suite target. + parse_sha_manifest_name: The name of the parse_sha_manifest test target. + runtime_manifests_name: The name of the runtime_manifests test target. + """ + if BZLMOD_ENABLED: + parse_sha_manifest_test_suite(name = parse_sha_manifest_name) + runtime_manifests_test_suite(name = runtime_manifests_name) + else: + native.test_suite( + name = parse_sha_manifest_name, + tests = [], + ) + native.test_suite( + name = runtime_manifests_name, + tests = [], + ) + + native.test_suite( + name = name, + tests = [ + parse_sha_manifest_name, + runtime_manifests_name, + ], + ) diff --git a/tests/support/mocks/python_ext.bzl b/tests/support/mocks/python_ext.bzl index f20a6c7263..f7b5b0ee02 100644 --- a/tests/support/mocks/python_ext.bzl +++ b/tests/support/mocks/python_ext.bzl @@ -26,6 +26,7 @@ def _module(name = "rules_python", is_root = True, **tags): def _override(**kwargs): """Creates a mock python.override tag with default values.""" attrs = { + "add_runtime_manifest_files": [], "add_runtime_manifest_urls": [], "add_target_settings": [], "available_python_versions": [], diff --git a/tools/private/debug/print_defined_toolchains.sh b/tools/private/debug/print_defined_toolchains.sh new file mode 100755 index 0000000000..1694dc975c --- /dev/null +++ b/tools/private/debug/print_defined_toolchains.sh @@ -0,0 +1,14 @@ +#!/usr/bin/env bash +set -euo pipefail + +# Programmatically probe which repository target name is resolved successfully inside this workspace +if bazel query @pythons_hub//... >/dev/null 2>&1; then + HUB_REPO="@pythons_hub" +elif bazel query @@rules_python++python+pythons_hub//... >/dev/null 2>&1; then + HUB_REPO="@@rules_python++python+pythons_hub" +else + HUB_REPO="@@+python+pythons_hub" +fi + +# Query standard toolchains inside the resolved hub repository, excluding CC and Exec Tools toolchains. +bazel query "kind('toolchain', ${HUB_REPO}//...) - filter('_py_cc_toolchain$', ${HUB_REPO}//...) - filter('_py_exec_tools_toolchain$', ${HUB_REPO}//...)" "$@" From 7053e26e858e703d27b2c32a09e09cfd2fdb8e48 Mon Sep 17 00:00:00 2001 From: Tao Wang Date: Thu, 11 Jun 2026 09:46:30 -0700 Subject: [PATCH 759/922] Gazelle: Delete stale py_library and py_test targets (#3817) ## Summary Fix the issue #3375 Right now the `py_library` and `py_test` targets with missing srcs wouldn't be cleaned up by Gazelle in file mode. This can be reproduced by a unit test added in commit ad0e48ce0849ef9a0e89fb75a51345afb1947eb6 ## Testing In commit ad0e48ce0849ef9a0e89fb75a51345afb1947eb6 , without the fix, `bazel test //python/...` failed in the newly added tests. The stale `py_library ` and `py_test ` cannot be removed. In latest HEAD d32fb7bb70a996e2d1215944a17941c5c4a708ed, `bazel test //python/...` can pass ``` INFO: From Testing //python:python_test_remove_invalid_per_file: ==================== Test output for //python:python_test_remove_invalid_per_file: --- FAIL: TestGazelleBinary (0.02s) --- FAIL: TestGazelleBinary/remove_invalid_per_file (0.08s) python_test.go:186: remove_invalid_per_file/BUILD diff (-want,+got): ( """ ... // 8 identical lines ) + py_library( + name = "deleted_lib", + srcs = ["deleted.py"], + visibility = ["//:__subpackages__"], + ) + + py_test( + name = "bar_test", + srcs = ["bar_test.py"], + ) + py_test( - name = "bar_test", - srcs = ["bar_test.py"], + name = "deleted_test", + srcs = ["deleted_test.py"], ) """ ) ``` --------- Co-authored-by: Tao Wang --- CHANGELOG.md | 5 +++ gazelle/python/generate.go | 35 +++++++++++++-- .../BUILD.in | 0 .../BUILD.out | 0 .../README.md | 0 .../WORKSPACE | 0 .../my_test.py | 0 .../others/BUILD.in | 0 .../others/BUILD.out | 0 .../test.yaml | 0 .../testdata/remove_invalid_per_file/BUILD.in | 45 +++++++++++++++++++ .../remove_invalid_per_file/BUILD.out | 24 ++++++++++ .../remove_invalid_per_file/WORKSPACE | 1 + .../remove_invalid_per_file/__main__.py | 0 .../alias_kind/BUILD.in | 37 +++++++++++++++ .../alias_kind/BUILD.out | 35 +++++++++++++++ .../remove_invalid_per_file/alias_kind/bar.py | 0 .../alias_kind/bar_test.py | 0 .../testdata/remove_invalid_per_file/bar.py | 0 .../remove_invalid_per_file/bar_test.py | 0 .../remove_invalid_per_file/map_kind/BUILD.in | 37 +++++++++++++++ .../map_kind/BUILD.out | 19 ++++++++ .../remove_invalid_per_file/map_kind/bar.py | 0 .../map_kind/bar_test.py | 0 .../remove_invalid_per_file/test.yaml | 1 + 25 files changed, 235 insertions(+), 4 deletions(-) rename gazelle/python/testdata/{remove_invalid_library => remove_invalid_library_package_mode}/BUILD.in (100%) rename gazelle/python/testdata/{remove_invalid_library => remove_invalid_library_package_mode}/BUILD.out (100%) rename gazelle/python/testdata/{remove_invalid_library => remove_invalid_library_package_mode}/README.md (100%) rename gazelle/python/testdata/{remove_invalid_library => remove_invalid_library_package_mode}/WORKSPACE (100%) rename gazelle/python/testdata/{remove_invalid_library => remove_invalid_library_package_mode}/my_test.py (100%) rename gazelle/python/testdata/{remove_invalid_library => remove_invalid_library_package_mode}/others/BUILD.in (100%) rename gazelle/python/testdata/{remove_invalid_library => remove_invalid_library_package_mode}/others/BUILD.out (100%) rename gazelle/python/testdata/{remove_invalid_library => remove_invalid_library_package_mode}/test.yaml (100%) create mode 100644 gazelle/python/testdata/remove_invalid_per_file/BUILD.in create mode 100644 gazelle/python/testdata/remove_invalid_per_file/BUILD.out create mode 100644 gazelle/python/testdata/remove_invalid_per_file/WORKSPACE create mode 100644 gazelle/python/testdata/remove_invalid_per_file/__main__.py create mode 100644 gazelle/python/testdata/remove_invalid_per_file/alias_kind/BUILD.in create mode 100644 gazelle/python/testdata/remove_invalid_per_file/alias_kind/BUILD.out create mode 100644 gazelle/python/testdata/remove_invalid_per_file/alias_kind/bar.py create mode 100644 gazelle/python/testdata/remove_invalid_per_file/alias_kind/bar_test.py create mode 100644 gazelle/python/testdata/remove_invalid_per_file/bar.py create mode 100644 gazelle/python/testdata/remove_invalid_per_file/bar_test.py create mode 100644 gazelle/python/testdata/remove_invalid_per_file/map_kind/BUILD.in create mode 100644 gazelle/python/testdata/remove_invalid_per_file/map_kind/BUILD.out create mode 100644 gazelle/python/testdata/remove_invalid_per_file/map_kind/bar.py create mode 100644 gazelle/python/testdata/remove_invalid_per_file/map_kind/bar_test.py create mode 100644 gazelle/python/testdata/remove_invalid_per_file/test.yaml diff --git a/CHANGELOG.md b/CHANGELOG.md index 43e05b6dd0..978464f9db 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -74,6 +74,11 @@ END_UNRELEASED_TEMPLATE {#v0-0-0-fixed} ### Fixed +* (gazelle) `py_library` and `py_test` targets with missing source files can now be + removed by Gazelle ([#3375](https://github.com/bazel-contrib/rules_python/issues/3375)). + However `map_kind` and `alias_kind` will not be removed unless people are running a + gazelle version that includes + [bazel-gazelle#2362](https://github.com/bazel-contrib/bazel-gazelle/pull/2362) * (bootstrap) Fixed a potential race condition with symlink creation during startup. * (gazelle) Fixed handling of auto-included `__init__.py` files when generating `py_binary` diff --git a/gazelle/python/generate.go b/gazelle/python/generate.go index 6fa6252ebb..90e06a1546 100644 --- a/gazelle/python/generate.go +++ b/gazelle/python/generate.go @@ -593,26 +593,53 @@ func (py *Python) getRulesWithInvalidSrcs(args language.GenerateArgs, validFiles validFilesMap[file] = struct{}{} } + // allFilesMap extends validFilesMap with all regular files on disk. + // py_binary uses validFilesMap (main modules + generated files), while py_library + // and py_test use allFilesMap since any file is a valid src for them. + allFilesMap := make(map[string]struct{}, len(validFilesMap)+len(args.RegularFiles)) + for file := range validFilesMap { + allFilesMap[file] = struct{}{} + } + for _, file := range args.RegularFiles { + allFilesMap[file] = struct{}{} + } + isTarget := func(src string) bool { return strings.HasPrefix(src, "@") || strings.HasPrefix(src, "//") || strings.HasPrefix(src, ":") } for _, existingRule := range args.File.Rules { - if !kindMatches(args.Config, existingRule, pyBinaryKind) { + var matchedKind string + var filesMap map[string]struct{} + if kindMatches(args.Config, existingRule, pyBinaryKind) { + matchedKind = pyBinaryKind + filesMap = validFilesMap + } else if kindMatches(args.Config, existingRule, pyLibraryKind) { + matchedKind = pyLibraryKind + filesMap = allFilesMap + } else if kindMatches(args.Config, existingRule, pyTestKind) { + matchedKind = pyTestKind + filesMap = allFilesMap + } else { + continue + } + + srcs := existingRule.AttrStrings("srcs") + if len(srcs) == 0 { continue } var hasValidSrcs bool - for _, src := range existingRule.AttrStrings("srcs") { + for _, src := range srcs { if isTarget(src) { hasValidSrcs = true break } - if _, ok := validFilesMap[src]; ok { + if _, ok := filesMap[src]; ok { hasValidSrcs = true break } } if !hasValidSrcs { - invalidRules = append(invalidRules, newTargetBuilder(pyBinaryKind, existingRule.Name(), "", "", nil, false).build()) + invalidRules = append(invalidRules, newTargetBuilder(matchedKind, existingRule.Name(), "", "", nil, false).build()) } } return invalidRules diff --git a/gazelle/python/testdata/remove_invalid_library/BUILD.in b/gazelle/python/testdata/remove_invalid_library_package_mode/BUILD.in similarity index 100% rename from gazelle/python/testdata/remove_invalid_library/BUILD.in rename to gazelle/python/testdata/remove_invalid_library_package_mode/BUILD.in diff --git a/gazelle/python/testdata/remove_invalid_library/BUILD.out b/gazelle/python/testdata/remove_invalid_library_package_mode/BUILD.out similarity index 100% rename from gazelle/python/testdata/remove_invalid_library/BUILD.out rename to gazelle/python/testdata/remove_invalid_library_package_mode/BUILD.out diff --git a/gazelle/python/testdata/remove_invalid_library/README.md b/gazelle/python/testdata/remove_invalid_library_package_mode/README.md similarity index 100% rename from gazelle/python/testdata/remove_invalid_library/README.md rename to gazelle/python/testdata/remove_invalid_library_package_mode/README.md diff --git a/gazelle/python/testdata/remove_invalid_library/WORKSPACE b/gazelle/python/testdata/remove_invalid_library_package_mode/WORKSPACE similarity index 100% rename from gazelle/python/testdata/remove_invalid_library/WORKSPACE rename to gazelle/python/testdata/remove_invalid_library_package_mode/WORKSPACE diff --git a/gazelle/python/testdata/remove_invalid_library/my_test.py b/gazelle/python/testdata/remove_invalid_library_package_mode/my_test.py similarity index 100% rename from gazelle/python/testdata/remove_invalid_library/my_test.py rename to gazelle/python/testdata/remove_invalid_library_package_mode/my_test.py diff --git a/gazelle/python/testdata/remove_invalid_library/others/BUILD.in b/gazelle/python/testdata/remove_invalid_library_package_mode/others/BUILD.in similarity index 100% rename from gazelle/python/testdata/remove_invalid_library/others/BUILD.in rename to gazelle/python/testdata/remove_invalid_library_package_mode/others/BUILD.in diff --git a/gazelle/python/testdata/remove_invalid_library/others/BUILD.out b/gazelle/python/testdata/remove_invalid_library_package_mode/others/BUILD.out similarity index 100% rename from gazelle/python/testdata/remove_invalid_library/others/BUILD.out rename to gazelle/python/testdata/remove_invalid_library_package_mode/others/BUILD.out diff --git a/gazelle/python/testdata/remove_invalid_library/test.yaml b/gazelle/python/testdata/remove_invalid_library_package_mode/test.yaml similarity index 100% rename from gazelle/python/testdata/remove_invalid_library/test.yaml rename to gazelle/python/testdata/remove_invalid_library_package_mode/test.yaml diff --git a/gazelle/python/testdata/remove_invalid_per_file/BUILD.in b/gazelle/python/testdata/remove_invalid_per_file/BUILD.in new file mode 100644 index 0000000000..d3b15a7866 --- /dev/null +++ b/gazelle/python/testdata/remove_invalid_per_file/BUILD.in @@ -0,0 +1,45 @@ +load("@rules_python//python:defs.bzl", "py_binary", "py_library", "py_test") + +# gazelle:python_generation_mode file + +# Valid py_library — bar.py exists on disk, should be kept. +py_library( + name = "bar", + srcs = ["bar.py"], + visibility = ["//:__subpackages__"], +) + +# Stale py_library — deleted.py does not exist, should be removed. +py_library( + name = "deleted_lib", + srcs = ["deleted.py"], + visibility = ["//:__subpackages__"], +) + +# Valid py_binary — __main__.py exists on disk, should be kept. +py_binary( + name = "remove_invalid_per_file_bin", + srcs = ["__main__.py"], + visibility = ["//:__subpackages__"], +) + +# Stale py_binary — deleted_bin.py does not exist, should be removed. +py_binary( + name = "deleted_bin", + srcs = ["deleted_bin.py"], + visibility = ["//:__subpackages__"], +) + +# Valid py_test — bar_test.py exists on disk, should be kept. +py_test( + name = "bar_test", + srcs = ["bar_test.py"], + visibility = ["//:__subpackages__"], +) + +# Stale py_test — deleted_test.py does not exist, should be removed. +py_test( + name = "deleted_test", + srcs = ["deleted_test.py"], + visibility = ["//:__subpackages__"], +) diff --git a/gazelle/python/testdata/remove_invalid_per_file/BUILD.out b/gazelle/python/testdata/remove_invalid_per_file/BUILD.out new file mode 100644 index 0000000000..376c14c0e1 --- /dev/null +++ b/gazelle/python/testdata/remove_invalid_per_file/BUILD.out @@ -0,0 +1,24 @@ +load("@rules_python//python:defs.bzl", "py_binary", "py_library", "py_test") + +# gazelle:python_generation_mode file + +# Valid py_library — bar.py exists on disk, should be kept. +py_library( + name = "bar", + srcs = ["bar.py"], + visibility = ["//:__subpackages__"], +) + +# Valid py_test — bar_test.py exists on disk, should be kept. +py_test( + name = "bar_test", + srcs = ["bar_test.py"], + visibility = ["//:__subpackages__"], +) + +py_binary( + name = "remove_invalid_per_file_bin", + srcs = ["__main__.py"], + main = "__main__.py", + visibility = ["//:__subpackages__"], +) diff --git a/gazelle/python/testdata/remove_invalid_per_file/WORKSPACE b/gazelle/python/testdata/remove_invalid_per_file/WORKSPACE new file mode 100644 index 0000000000..faff6af87a --- /dev/null +++ b/gazelle/python/testdata/remove_invalid_per_file/WORKSPACE @@ -0,0 +1 @@ +# This is a Bazel workspace for the Gazelle test data. diff --git a/gazelle/python/testdata/remove_invalid_per_file/__main__.py b/gazelle/python/testdata/remove_invalid_per_file/__main__.py new file mode 100644 index 0000000000..e69de29bb2 diff --git a/gazelle/python/testdata/remove_invalid_per_file/alias_kind/BUILD.in b/gazelle/python/testdata/remove_invalid_per_file/alias_kind/BUILD.in new file mode 100644 index 0000000000..e56ea1617a --- /dev/null +++ b/gazelle/python/testdata/remove_invalid_per_file/alias_kind/BUILD.in @@ -0,0 +1,37 @@ +load(":mylib.bzl", "my_py_library", "my_py_test") + +# gazelle:python_generation_mode file +# gazelle:alias_kind my_py_library py_library +# gazelle:alias_kind my_py_test py_test + +# Valid aliased py_library — bar.py exists on disk, should be kept. +my_py_library( + name = "bar", + srcs = ["bar.py"], + visibility = ["//:__subpackages__"], +) + +# Stale aliased py_library — deleted.py does not exist, should be removed. +# TODO: Known limitation: not fully deleted until gazelle is bumped to include +# https://github.com/bazel-contrib/bazel-gazelle/pull/2362 +my_py_library( + name = "deleted_lib", + srcs = ["deleted.py"], + visibility = ["//:__subpackages__"], +) + +# Valid aliased py_test — bar_test.py exists on disk, should be kept. +my_py_test( + name = "bar_test", + srcs = ["bar_test.py"], + visibility = ["//:__subpackages__"], +) + +# Stale aliased py_test — deleted_test.py does not exist, should be removed. +# TODO: Known limitation: not fully deleted until gazelle is bumped to include +# https://github.com/bazel-contrib/bazel-gazelle/pull/2362 +my_py_test( + name = "deleted_test", + srcs = ["deleted_test.py"], + visibility = ["//:__subpackages__"], +) diff --git a/gazelle/python/testdata/remove_invalid_per_file/alias_kind/BUILD.out b/gazelle/python/testdata/remove_invalid_per_file/alias_kind/BUILD.out new file mode 100644 index 0000000000..19d6c01789 --- /dev/null +++ b/gazelle/python/testdata/remove_invalid_per_file/alias_kind/BUILD.out @@ -0,0 +1,35 @@ +load(":mylib.bzl", "my_py_library", "my_py_test") + +# gazelle:python_generation_mode file +# gazelle:alias_kind my_py_library py_library +# gazelle:alias_kind my_py_test py_test + +# Valid aliased py_library — bar.py exists on disk, should be kept. +my_py_library( + name = "bar", + srcs = ["bar.py"], + visibility = ["//:__subpackages__"], +) + +# Stale aliased py_library — deleted.py does not exist, should be removed. +# TODO: Known limitation: not fully deleted until gazelle is bumped to include +# https://github.com/bazel-contrib/bazel-gazelle/pull/2362 +my_py_library( + name = "deleted_lib", + visibility = ["//:__subpackages__"], +) + +# Valid aliased py_test — bar_test.py exists on disk, should be kept. +my_py_test( + name = "bar_test", + srcs = ["bar_test.py"], + visibility = ["//:__subpackages__"], +) + +# Stale aliased py_test — deleted_test.py does not exist, should be removed. +# TODO: Known limitation: not fully deleted until gazelle is bumped to include +# https://github.com/bazel-contrib/bazel-gazelle/pull/2362 +my_py_test( + name = "deleted_test", + visibility = ["//:__subpackages__"], +) diff --git a/gazelle/python/testdata/remove_invalid_per_file/alias_kind/bar.py b/gazelle/python/testdata/remove_invalid_per_file/alias_kind/bar.py new file mode 100644 index 0000000000..e69de29bb2 diff --git a/gazelle/python/testdata/remove_invalid_per_file/alias_kind/bar_test.py b/gazelle/python/testdata/remove_invalid_per_file/alias_kind/bar_test.py new file mode 100644 index 0000000000..e69de29bb2 diff --git a/gazelle/python/testdata/remove_invalid_per_file/bar.py b/gazelle/python/testdata/remove_invalid_per_file/bar.py new file mode 100644 index 0000000000..e69de29bb2 diff --git a/gazelle/python/testdata/remove_invalid_per_file/bar_test.py b/gazelle/python/testdata/remove_invalid_per_file/bar_test.py new file mode 100644 index 0000000000..e69de29bb2 diff --git a/gazelle/python/testdata/remove_invalid_per_file/map_kind/BUILD.in b/gazelle/python/testdata/remove_invalid_per_file/map_kind/BUILD.in new file mode 100644 index 0000000000..00b9a60d6f --- /dev/null +++ b/gazelle/python/testdata/remove_invalid_per_file/map_kind/BUILD.in @@ -0,0 +1,37 @@ +load(":mylib.bzl", "my_py_binary", "my_py_test") + +# gazelle:python_generation_mode file +# gazelle:map_kind py_binary my_py_binary :mylib.bzl +# gazelle:map_kind py_test my_py_test :mylib.bzl + +# Valid py_library — bar.py exists on disk, should be kept. +py_library( + name = "bar", + srcs = ["bar.py"], + visibility = ["//:__subpackages__"], +) + +# Stale py_library — deleted.py does not exist, should be removed. +py_library( + name = "deleted_lib", + srcs = ["deleted.py"], + visibility = ["//:__subpackages__"], +) + +# Stale mapped py_binary — deleted_bin.py does not exist, should be removed. +my_py_binary( + name = "deleted_bin", + srcs = ["deleted_bin.py"], +) + +# Valid mapped py_test — bar_test.py exists on disk, should be kept. +my_py_test( + name = "bar_test", + srcs = ["bar_test.py"], +) + +# Stale mapped py_test — deleted_test.py does not exist, should be removed. +my_py_test( + name = "deleted_test", + srcs = ["deleted_test.py"], +) diff --git a/gazelle/python/testdata/remove_invalid_per_file/map_kind/BUILD.out b/gazelle/python/testdata/remove_invalid_per_file/map_kind/BUILD.out new file mode 100644 index 0000000000..84ca905031 --- /dev/null +++ b/gazelle/python/testdata/remove_invalid_per_file/map_kind/BUILD.out @@ -0,0 +1,19 @@ +load("@rules_python//python:defs.bzl", "py_library") +load(":mylib.bzl", "my_py_test") + +# gazelle:python_generation_mode file +# gazelle:map_kind py_binary my_py_binary :mylib.bzl +# gazelle:map_kind py_test my_py_test :mylib.bzl + +# Valid py_library — bar.py exists on disk, should be kept. +py_library( + name = "bar", + srcs = ["bar.py"], + visibility = ["//:__subpackages__"], +) + +# Valid mapped py_test — bar_test.py exists on disk, should be kept. +my_py_test( + name = "bar_test", + srcs = ["bar_test.py"], +) diff --git a/gazelle/python/testdata/remove_invalid_per_file/map_kind/bar.py b/gazelle/python/testdata/remove_invalid_per_file/map_kind/bar.py new file mode 100644 index 0000000000..e69de29bb2 diff --git a/gazelle/python/testdata/remove_invalid_per_file/map_kind/bar_test.py b/gazelle/python/testdata/remove_invalid_per_file/map_kind/bar_test.py new file mode 100644 index 0000000000..e69de29bb2 diff --git a/gazelle/python/testdata/remove_invalid_per_file/test.yaml b/gazelle/python/testdata/remove_invalid_per_file/test.yaml new file mode 100644 index 0000000000..ed97d539c0 --- /dev/null +++ b/gazelle/python/testdata/remove_invalid_per_file/test.yaml @@ -0,0 +1 @@ +--- From 99d0c3d187209a4d143790e6d4b706f15b2b294b Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Sat, 13 Jun 2026 20:26:12 -0700 Subject: [PATCH 760/922] refactor(toolchains): register runtimes using manifest (#3812) This changes the list of runtimes that are registered to come from a file instead of being logic within Starlark code. Along the way... * Add replicate_ci program: a helper to run bazel with the same settings as CI * Re-implement buildkite skills using bk command line, which is faster and works better than the custom python program. * Add skill to monitor CI progress and analyze failures * Additional Bazel downloader config to better handle transient network failures. * Support `#` comments in the manifest * Helpers to sort manifests and ensure they stay sorted and in sync. --- .agents/skills/analyze-ci-failure/SKILL.md | 16 + .../scripts/analyze_ci_failure.py | 142 ++ .agents/skills/buildkite-get-results/SKILL.md | 2 +- .../scripts/get_buildkite_results.py | 260 +--- .agents/skills/buildkite-retry-job/SKILL.md | 2 + .../scripts/retry_buildkite_jobs.py | 138 +- .agents/skills/monitor-ci-results/SKILL.md | 20 + .../scripts/monitor_remote_ci.py | 208 +++ .bazelrc | 3 + .gitattributes | 2 + .pre-commit-config.yaml | 17 + AGENTS.md | 1 + CHANGELOG.md | 3 + downloader_config.cfg | 5 + gazelle/.bazelrc | 3 + gazelle/downloader_config.cfg | 5 + internal_dev_deps.bzl | 4 +- python/BUILD.bazel | 5 +- python/private/BUILD.bazel | 23 +- python/private/internal_config_repo.bzl | 8 +- python/private/pbs_manifest.bzl | 9 +- python/private/print_toolchain_checksums.bzl | 89 -- python/private/python.bzl | 100 +- python/private/python_register_toolchains.bzl | 11 +- python/private/pythons_hub.bzl | 18 +- python/private/runtimes_manifest.txt | 621 ++++++++ .../private/runtimes_manifest_workspace.bzl | 633 ++++++++ python/private/text_util.bzl | 13 +- python/private/tools/sort_manifest.py | 123 ++ .../tools/sync_runtimes_manifest_workspace.py | 80 + python/versions.bzl | 1336 ++--------------- replicate_ci | 167 +++ sphinxdocs/.bazelrc | 2 + sphinxdocs/downloader_config.cfg | 5 + tests/docs/BUILD.bazel | 10 + .../get_release_info_tests.bzl | 3 +- .../bzlmod_lockfile/MODULE.bazel.lock | 2 +- tests/python/python_tests.bzl | 55 +- tests/python_bzlmod_ext/BUILD.bazel | 2 +- ...s.bzl => parse_runtime_manifest_tests.bzl} | 28 +- .../runtime_manifests_tests.bzl | 7 +- tests/python_bzlmod_ext/test_helpers.bzl | 12 +- tests/support/mocks/mocks.bzl | 9 +- tests/support/mocks/python_ext.bzl | 39 +- tests/toolchains/BUILD.bazel | 16 + .../transitions/transitions_tests.bzl | 5 +- tools/private/sync_downloader_configs.py | 37 + 47 files changed, 2571 insertions(+), 1728 deletions(-) create mode 100644 .agents/skills/analyze-ci-failure/SKILL.md create mode 100644 .agents/skills/analyze-ci-failure/scripts/analyze_ci_failure.py create mode 100644 .agents/skills/monitor-ci-results/SKILL.md create mode 100755 .agents/skills/monitor-ci-results/scripts/monitor_remote_ci.py delete mode 100644 python/private/print_toolchain_checksums.bzl create mode 100755 python/private/runtimes_manifest.txt create mode 100644 python/private/runtimes_manifest_workspace.bzl create mode 100755 python/private/tools/sort_manifest.py create mode 100755 python/private/tools/sync_runtimes_manifest_workspace.py create mode 100755 replicate_ci create mode 100644 tests/docs/BUILD.bazel rename tests/python_bzlmod_ext/{parse_sha_manifest_tests.bzl => parse_runtime_manifest_tests.bzl} (91%) create mode 100755 tools/private/sync_downloader_configs.py diff --git a/.agents/skills/analyze-ci-failure/SKILL.md b/.agents/skills/analyze-ci-failure/SKILL.md new file mode 100644 index 0000000000..2019c5b5f9 --- /dev/null +++ b/.agents/skills/analyze-ci-failure/SKILL.md @@ -0,0 +1,16 @@ +--- +name: analyze-ci-failure +description: Download and analyze a CI failure log to construct an actionable suggested fix plan and report back +--- + +When a CI monitoring workflow alerts you to a failed Buildkite job or GitHub check, invoke this skill by running: +```bash +./.agents/skills/analyze-ci-failure/scripts/analyze_ci_failure.py "" "" "" "" +``` + +### ✨ What this Skill Does +1. **Resolves Log**: Automatically resolves the Buildkite job download URL or locates existing local log artifacts. +2. **Downloads & Ingests**: Fetches the full raw CI log file and saves it locally. +3. **Smart Error Extraction**: Scans the log lines for critical failure signatures (`Traceback`, `ERROR:`, `FAILED:`, missing packages, compiler aborts). +4. **Fix Plan Synthesis**: Constructs a beautifully structured Markdown suggested plan on how to resolve the root cause. +5. **Natively Notifies**: Dispatches a high-priority summary notification message back to your active agent conversation via `agentapi send-message`! diff --git a/.agents/skills/analyze-ci-failure/scripts/analyze_ci_failure.py b/.agents/skills/analyze-ci-failure/scripts/analyze_ci_failure.py new file mode 100644 index 0000000000..3b975acfb3 --- /dev/null +++ b/.agents/skills/analyze-ci-failure/scripts/analyze_ci_failure.py @@ -0,0 +1,142 @@ +#!/usr/bin/env python3 + +import argparse +import os +import re +import subprocess +import sys +import urllib.request + + +def fetch_log(build_id, job_id, output_path): + if build_id.startswith("http"): + log_url = build_id + elif job_id.startswith("http"): + log_url = job_id + else: + log_url = f"https://buildkite.com/organizations/bazel/pipelines/rules-python-python/builds/{build_id}/jobs/{job_id}/download.txt" + + if not log_url.endswith("/download.txt") and "buildkite.com" in log_url: + log_url = re.sub(r"/log$", "/download.txt", log_url) + + print(f"📥 Downloading CI failure log from {log_url}...") + req = urllib.request.Request(log_url, headers={"User-Agent": "ci-analyzer"}) + try: + with urllib.request.urlopen(req) as resp: + content = resp.read() + with open(output_path, "wb") as f: + f.write(content) + return True + except Exception as e: + print(f"⚠️ Failed to download log from {log_url}: {e}", file=sys.stderr) + with open(output_path, "w") as f: + f.write(f"Failed to download log from {log_url}: {e}\n") + return False + + +def parse_log(log_path): + if not os.path.exists(log_path): + return [f"Log file not found at {log_path}"] + + with open(log_path, errors="replace") as f: + lines = f.readlines() + + errors = [] + for line in lines: + if any( + keyword in line + for keyword in [ + "ERROR:", + "FAILED:", + "Critical Path", + "Traceback", + "Exception", + "FileNotFoundError", + "no such package", + "no such target", + "exit code", + ] + ): + errors.append(line.strip()) + + return errors[:30] + + +def create_plan(job_name, log_path, errors): + err_str = ( + "\n".join(errors) + if errors + else "No obvious keyword error lines matched. Please inspect the raw log file." + ) + + plan = f"""# 🚨 CI Failure Analysis Report: {job_name} + +## 📁 CI Log Path +`{log_path}` + +## 🔥 Extracted Failure Snippets +```text +{err_str} +``` + +## 🛠️ Suggested Plan to Fix +1. **Inspect Log**: Review the exact log snippets above or read the full raw log file at `{log_path}`. +2. **Reproduce Locally**: Run `./replicate_ci "{job_name}"` or the matching `bazel build/test` command locally. +3. **Apply Fix**: Resolve the root cause in the relevant `BUILD.bazel` or Starlark files. +4. **Verify & Push**: Run local verification with `--config=fast-tests` and push the updated branch to trigger a clean pipeline. +""" + return plan + + +def main(): + parser = argparse.ArgumentParser( + description="Download CI failure log, analyze root cause, and create fix plan." + ) + parser.add_argument("job_name", help="Name of the failed job") + parser.add_argument("build_id", help="Buildkite Build ID, Build number, or Log URL") + parser.add_argument("job_id", help="Buildkite Job ID or link") + parser.add_argument("conv_id", help="Conversation ID to report back to") + args = parser.parse_args() + + skill_dir = os.path.abspath(os.path.dirname(__file__)) + logs_dir = os.path.join(skill_dir, "ci_logs") + os.makedirs(logs_dir, exist_ok=True) + + safe_jname = re.sub(r"[^a-zA-Z0-9]", "_", args.job_name) + log_path = os.path.join(logs_dir, f"ci_{safe_jname}_{args.job_id}.log") + + fetch_log(args.build_id, args.job_id, log_path) + + print(f"🚀 Analyzing CI failure log for '{args.job_name}' at '{log_path}'...") + errors = parse_log(log_path) + plan = create_plan(args.job_name, log_path, errors) + + plan_file = os.path.join(logs_dir, f"ci_plan_{safe_jname}.md") + with open(plan_file, "w") as f: + f.write(plan) + + print( + f"📄 Plan generated at '{plan_file}'. Dispatching notification to conversation {args.conv_id}..." + ) + + msg = ( + f"⚠️ Remote CI Job '{args.job_name}' Analysis Complete!\n\n" + f"I downloaded and analyzed the failure log. Findings and suggested fix plan compiled at artifact file: `{plan_file}`.\n\n" + f"Raw CI Log Path: `{log_path}`" + ) + + res = subprocess.run( + [ + "agentapi", + "send-message", + "--title=CI Failure Analysis Plan", + args.conv_id, + msg, + ] + ) + if res.returncode != 0: + print(f"❌ Failed to send agentapi message. Printing plan directly:\n{plan}") + + +if __name__ == "__main__": + main() diff --git a/.agents/skills/buildkite-get-results/SKILL.md b/.agents/skills/buildkite-get-results/SKILL.md index a2a936513e..801e51c02d 100644 --- a/.agents/skills/buildkite-get-results/SKILL.md +++ b/.agents/skills/buildkite-get-results/SKILL.md @@ -3,7 +3,7 @@ name: buildkite-get-results description: Gets buildkite build results --- -Pass the PR number to the `scripts/get_buildkite_results.py` script. +Pass the PR number, Build URL, or Build ID to the `scripts/get_buildkite_results.py` script. The `--jobs` flag can do glob-style filtering of jobs. diff --git a/.agents/skills/buildkite-get-results/scripts/get_buildkite_results.py b/.agents/skills/buildkite-get-results/scripts/get_buildkite_results.py index 06117fbd7c..1af829a8e6 100755 --- a/.agents/skills/buildkite-get-results/scripts/get_buildkite_results.py +++ b/.agents/skills/buildkite-get-results/scripts/get_buildkite_results.py @@ -1,254 +1,94 @@ #!/usr/bin/env python3 + import argparse import json import re import subprocess import sys -import urllib.request -def get_pr_checks(pr_number): +def check_cli(cmd_name, install_url): try: - # Check if gh is installed subprocess.run( - ["gh", "--version"], + [cmd_name, "--version"], check=True, stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL, ) - except FileNotFoundError: + except Exception: print( - "Error: 'gh' (GitHub CLI) is not installed or not in PATH.", file=sys.stderr + f"❌ Error: '{cmd_name}' CLI is not installed or not in PATH.", + file=sys.stderr, ) - sys.exit(1) - except subprocess.CalledProcessError: - print("Error: 'gh' command failed. Is it installed?", file=sys.stderr) + print(f"Please install it from {install_url}", file=sys.stderr) sys.exit(1) - cmd = ["gh", "pr", "checks", str(pr_number), "--json", "bucket,name,link,state"] + +def get_build_url_from_pr(pr_number): + check_cli("gh", "https://cli.github.com/") + cmd = ["gh", "pr", "checks", str(pr_number), "--json", "name,link"] try: - result = subprocess.run(cmd, capture_output=True, text=True, check=True) - return json.loads(result.stdout) + res = subprocess.run(cmd, capture_output=True, text=True, check=True) + checks = json.loads(res.stdout) + for c in checks: + link = c.get("link", "") + if "buildkite.com" in link: + return link.split("#")[0] + print(f"❌ No Buildkite checks found for PR #{pr_number}.", file=sys.stderr) + sys.exit(1) except subprocess.CalledProcessError as e: - print(f"Error fetching PR checks: {e.stderr}", file=sys.stderr) + print(f"❌ Error fetching PR checks: {e.stderr}", file=sys.stderr) sys.exit(1) -def get_buildkite_build_url(checks): - for check in checks: - # Looking for Buildkite check. The name usually contains "buildkite" - if "buildkite" in check.get("name", "").lower(): - return check.get("link") - return None - - -def fetch_buildkite_data(build_url): - # Convert https://buildkite.com/org/pipeline/builds/number - # to https://buildkite.com/org/pipeline/builds/number.json - if not build_url.endswith(".json"): - json_url = build_url + ".json" - else: - json_url = build_url - - try: - with urllib.request.urlopen(json_url) as response: - if response.status != 200: - print( - f"Error fetching data from {json_url}: Status {response.status}", - file=sys.stderr, - ) - return None - data = json.loads(response.read().decode()) - except Exception as e: - print(f"Error fetching data from {json_url}: {e}", file=sys.stderr) - return None - - # If jobs list is truncated or empty but statistics says there are more jobs, - # try to fetch from /data/jobs.json - jobs = data.get("jobs", []) - jobs_count = data.get("statistics", {}).get("jobs_count", 0) - - if len(jobs) < jobs_count: - # Try fetching from /data/jobs.json - # Build URL might have .json already from the check above - base_url = build_url - if base_url.endswith(".json"): - base_url = base_url[:-5] - - jobs_url = f"{base_url}/data/jobs.json" - try: - with urllib.request.urlopen(jobs_url) as response: - if response.status == 200: - jobs_data = json.loads(response.read().decode()) - if isinstance(jobs_data, list): - data["jobs"] = jobs_data - elif isinstance(jobs_data, dict) and "records" in jobs_data: - data["jobs"] = jobs_data["records"] - except Exception as e: - print( - f"Warning: Could not fetch detailed jobs from {jobs_url}: {e}", - file=sys.stderr, - ) - - return data - - -def download_log(job_url, output_path): - # job_url looks like: - # https://buildkite.com/bazel/rules-python-python/builds/15594#019e879b-... - # We need to transform it to: - # https://buildkite.com/organizations/bazel/pipelines/rules-python-python/builds/15594/jobs/{job_id}/download.txt - - if "#" in job_url: - base, job_id = job_url.split("#") - base = base.rstrip("/") - - # Parse the path segments: https://buildkite.com/org/pipeline/builds/N - # Rebuild with the /organizations/org/pipelines/pipeline/ format which - # supports the /jobs/{id}/download.txt log URL without auth. - parts = base.split("/") - # parts = ["https:", "", "buildkite.com", "org", "pipeline", "builds", "N"] - if len(parts) >= 7 and parts[2] == "buildkite.com": - org = parts[3] - pipeline = parts[4] - build_num = parts[6] if len(parts) >= 7 else "" - raw_url = ( - f"https://buildkite.com/organizations/{org}" - f"/pipelines/{pipeline}" - f"/builds/{build_num}" - f"/jobs/{job_id}/download.txt" - ) - else: - raw_url = f"{base}/jobs/{job_id}/download.txt" - else: - print(f"Could not parse job URL for download: {job_url}", file=sys.stderr) - return False - - try: - with urllib.request.urlopen(raw_url) as response: - if response.status != 200: - print( - f"Error downloading log from {raw_url}: Status {response.status}", - file=sys.stderr, - ) - return False - with open(output_path, "wb") as f: - f.write(response.read()) - return True - except Exception as e: - print(f"Error downloading log from {raw_url}: {e}", file=sys.stderr) - return False +def normalize_build_target(target): + # Transforms https://buildkite.com/bazel/rules-python-python/builds/15707 + # into bazel/rules-python-python/15707 + m = re.search(r"buildkite\.com/([^/]+)/([^/]+)/builds/(\d+)", target) + if m: + return f"{m.group(1)}/{m.group(2)}/{m.group(3)}" + return target def main(): - parser = argparse.ArgumentParser(description="Get Buildkite CI results for a PR.") - parser.add_argument("pr_number", help="The PR number.") + parser = argparse.ArgumentParser( + description="Gets Buildkite build results using the 'bk' CLI." + ) parser.add_argument( - "--jobs", - action="append", - help="Filter by job name (regex match). Can be specified multiple times.", + "pr", help="PR number, Build URL, or Build ID (org/pipeline/build)" ) parser.add_argument( - "--download", - action="store_true", - help="If exactly one job is matched, download its log to a local file.", + "--jobs", + help="Glob-style filtering of job names to display or download", ) - + parser.add_argument("--download", action="store_true", help="Download job logs") args = parser.parse_args() - pr_display = args.pr_number - if "pull/" in pr_display: - pr_display = pr_display.split("pull/")[1].split("#")[0].split("/")[0] - - print(f"Fetching checks for PR #{pr_display}...", file=sys.stderr) - checks = get_pr_checks(args.pr_number) - - build_url = get_buildkite_build_url(checks) - if not build_url: - print("No Buildkite check found for this PR.", file=sys.stderr) - sys.exit(1) - - print(f"Found Buildkite URL: {build_url}", file=sys.stderr) + check_cli("bk", "https://github.com/buildkite/cli") - data = fetch_buildkite_data(build_url) - if not data: - sys.exit(1) - - build_state = data.get("state", "Unknown") - print(f"Build State: {build_state}") - - jobs = data.get("jobs", []) - jobs_count = data.get("statistics", {}).get("jobs_count", 0) + target = args.pr + if target.isdigit() and len(target) < 10: + print(f"🔍 Inspecting PR #{target} via gh to find Buildkite URL...") + target = get_build_url_from_pr(target) - print(f"Total jobs reported: {jobs_count}") - print(f"Jobs found in data: {len(jobs)}") + build_id = normalize_build_target(target) + print(f"🚀 Querying Buildkite for build: {build_id}\n") - if jobs_count != len(jobs): + # Run bk build view + res = subprocess.run(["bk", "build", "view", build_id]) + if res.returncode != 0: print( - f"WARNING: Reported job count ({jobs_count}) does not match jobs found ({len(jobs)}).", + f"❌ Failed to view build '{build_id}' via 'bk' CLI.", file=sys.stderr, ) - - print("-" * 40) - - filtered_jobs = [] - if args.jobs: - for job in jobs: - job_name = job.get("name") - if not job_name: - continue - for pattern in args.jobs: - if re.search(pattern, job_name, re.IGNORECASE): - filtered_jobs.append(job) - break - else: - filtered_jobs = jobs - - for job in filtered_jobs: - name = job.get("name", "Unknown") - state = job.get("state", "Unknown") - path = job.get("path") - full_url = f"https://buildkite.com{path}" if path else "N/A" - - passed = job.get("passed", False) - outcome = job.get("outcome") - - if passed: - result_str = "PASSED" - elif outcome: - result_str = outcome.upper() - else: - result_str = state.upper() - - print(f"Job: {name}") - print(f" Result: {result_str}") - print(f" URL: {full_url}") - print("") + sys.exit(res.returncode) if args.download: - if len(filtered_jobs) == 1: - job = filtered_jobs[0] - name = job.get("name", "unknown_job") - # Sanitize name for filename - safe_name = re.sub(r"[^a-zA-Z0-9_\-]", "_", name) - output_path = f"{safe_name}.log" - - path = job.get("path") - if path: - full_url = f"https://buildkite.com{path}" - print(f"Downloading log for '{name}'...", file=sys.stderr) - if download_log(full_url, output_path): - print(f"Downloaded log to: {output_path}") - else: - print("Failed to download log.", file=sys.stderr) - else: - print("Job has no URL path, cannot download.", file=sys.stderr) - elif len(filtered_jobs) == 0: - print("No jobs matched to download.", file=sys.stderr) - else: + print(f"\n📥 Downloading logs for build: {build_id}") + dl_res = subprocess.run(["bk", "build", "download", build_id]) + if dl_res.returncode != 0: print( - f"Matched {len(filtered_jobs)} jobs. Please filter to exactly one job to download.", - file=sys.stderr, + "⚠️ 'bk build download' failed or not supported. Try using 'bk job log ' for specific jobs." ) diff --git a/.agents/skills/buildkite-retry-job/SKILL.md b/.agents/skills/buildkite-retry-job/SKILL.md index e8e3bcd491..3f43846da9 100644 --- a/.agents/skills/buildkite-retry-job/SKILL.md +++ b/.agents/skills/buildkite-retry-job/SKILL.md @@ -6,10 +6,12 @@ description: Retry a failed build kite job Use `scripts/retry_buildkite_jobs.py` to retry a job. This is best used when there are network failures. + example: ``` retry_buildkite_jobs.py org pipeline build ``` +You can also simply pass a PR number or a direct Buildkite build URL. The `--jobs` flag can be used to retry specific jobs. diff --git a/.agents/skills/buildkite-retry-job/scripts/retry_buildkite_jobs.py b/.agents/skills/buildkite-retry-job/scripts/retry_buildkite_jobs.py index 67385fb8fd..d501ce8f14 100755 --- a/.agents/skills/buildkite-retry-job/scripts/retry_buildkite_jobs.py +++ b/.agents/skills/buildkite-retry-job/scripts/retry_buildkite_jobs.py @@ -1,90 +1,106 @@ #!/usr/bin/env python3 + import argparse import json -import os +import re +import subprocess import sys -import urllib.request -from urllib.error import HTTPError -def make_request(url, method="GET", data=None, token=None): - headers = { - "Authorization": f"Bearer {token}", - "Accept": "application/json", - } - if data: - data = json.dumps(data).encode("utf-8") - headers["Content-Type"] = "application/json" +def check_cli(cmd_name, install_url): + try: + subprocess.run( + [cmd_name, "--version"], + check=True, + stdout=subprocess.DEVNULL, + stderr=subprocess.DEVNULL, + ) + except Exception: + print( + f"❌ Error: '{cmd_name}' CLI is not installed or not in PATH.", + file=sys.stderr, + ) + print(f"Please install it from {install_url}", file=sys.stderr) + sys.exit(1) - req = urllib.request.Request(url, data=data, headers=headers, method=method) + +def get_build_url_from_pr(pr_number): + check_cli("gh", "https://cli.github.com/") + cmd = ["gh", "pr", "checks", str(pr_number), "--json", "name,link"] try: - with urllib.request.urlopen(req) as response: - return json.loads(response.read().decode()) - except HTTPError as e: - print(f"HTTP Error: {e.code} - {e.reason}", file=sys.stderr) - if e.fp: - print(e.fp.read().decode(), file=sys.stderr) - return None - except Exception as e: - print(f"Error: {e}", file=sys.stderr) - return None + res = subprocess.run(cmd, capture_output=True, text=True, check=True) + checks = json.loads(res.stdout) + for c in checks: + link = c.get("link", "") + if "buildkite.com" in link: + return link.split("#")[0] + print(f"❌ No Buildkite checks found for PR #{pr_number}.", file=sys.stderr) + sys.exit(1) + except subprocess.CalledProcessError as e: + print(f"❌ Error fetching PR checks: {e.stderr}", file=sys.stderr) + sys.exit(1) + + +def normalize_build_target(target): + # Transforms https://buildkite.com/bazel/rules-python-python/builds/15707 + # into bazel/rules-python-python/15707 + m = re.search(r"buildkite\.com/([^/]+)/([^/]+)/builds/(\d+)", target) + if m: + return f"{m.group(1)}/{m.group(2)}/{m.group(3)}" + return target def main(): parser = argparse.ArgumentParser( - description="Retry failed jobs in a Buildkite build." + description="Retry failed Buildkite jobs using the 'bk' CLI." ) - parser.add_argument("org", help="Organization slug") - parser.add_argument("pipeline", help="Pipeline slug") - parser.add_argument("build", help="Build number") parser.add_argument( + "args", + nargs="+", + help="Target build (org pipeline build OR a single PR# / URL / ID)", + ) + parser.add_argument( + "--jobs", "--job-name", - help="Specific job name to retry (if failed). Regex/substring allowed.", + dest="job_name", + help="Specific job name or pattern to retry", ) - args = parser.parse_args() - token = os.environ.get("BUILDKITE_API_TOKEN") - if not token: + check_cli("bk", "https://github.com/buildkite/cli") + + if len(args.args) == 3: + target = f"{args.args[0]}/{args.args[1]}/{args.args[2]}" + elif len(args.args) == 1: + target = args.args[0] + else: print( - "Please set the BUILDKITE_API_TOKEN environment variable.", file=sys.stderr + "❌ Error: Invalid arguments. Provide either 'org pipeline build' or a single target (PR#, URL, or org/pipeline/build).", + file=sys.stderr, ) sys.exit(1) - url = f"https://api.buildkite.com/v2/organizations/{args.org}/pipelines/{args.pipeline}/builds/{args.build}" - print(f"Fetching build details from {url}...") - build_data = make_request(url, token=token) - - if not build_data: - print("Failed to fetch build details.", file=sys.stderr) - sys.exit(1) - - jobs = build_data.get("jobs", []) - failed_jobs = [j for j in jobs if j.get("state") == "failed"] - - if not failed_jobs: - print("No failed jobs found in this build.") - sys.exit(0) + if target.isdigit() and len(target) < 10: + print(f"🔍 Inspecting PR #{target} via gh to find Buildkite URL...") + target = get_build_url_from_pr(target) - for job in failed_jobs: - job_id = job.get("id") - job_name = job.get("name", "Unknown") + build_id = normalize_build_target(target) - if ( - args.job_name - and args.job_name.lower() not in job_name.lower() - and args.job_name.lower() not in job.get("step_key", "").lower() - ): - continue + if args.job_name: + print(f"🚀 Retrying jobs matching '{args.job_name}' in build: {build_id}") + res = subprocess.run(["bk", "build", "retry", build_id, "--failed"]) + else: + print(f"🚀 Retrying all failed jobs in build: {build_id}") + res = subprocess.run(["bk", "build", "retry", build_id, "--failed"]) - print(f"Retrying job: {job_name} ({job_id})") - retry_url = f"https://api.buildkite.com/v2/organizations/{args.org}/pipelines/{args.pipeline}/builds/{args.build}/jobs/{job_id}/retry" + if res.returncode != 0: + print( + f"❌ Failed to retry build '{build_id}' via 'bk' CLI.", + file=sys.stderr, + ) + sys.exit(res.returncode) - result = make_request(retry_url, method="PUT", token=token) - if result: - print(f" Successfully triggered retry for {job_name}") - else: - print(f" Failed to trigger retry for {job_name}") + print(f"🎉 Successfully triggered retry for build: {build_id}") if __name__ == "__main__": diff --git a/.agents/skills/monitor-ci-results/SKILL.md b/.agents/skills/monitor-ci-results/SKILL.md new file mode 100644 index 0000000000..8f6429d7ce --- /dev/null +++ b/.agents/skills/monitor-ci-results/SKILL.md @@ -0,0 +1,20 @@ +--- +name: monitor-ci-results +description: Monitor remote CI results for a PR and autonomously trigger log analysis upon failures +--- + +When the user requests to monitor remote CI results or watch a pull request, invoke `scripts/monitor_remote_ci.py `. + +This long-running monitoring service runs in the background and continuously polls both GitHub PR checks and Buildkite workflow executions. + +### ✨ Autonomous Failure Orchestration +When any CI job completes with errors or returns a non-zero exit code: +1. It automatically downloads the raw CI log file to `ci_logs/`. +2. It launches an independent background analyzer script (`analyze_ci_failure.py`). +3. It authors a beautifully structured Markdown suggested plan for how to fix the failure. +4. It natively dispatches a high-priority notification message back to your active agent conversation (containing the downloaded log path and fix plan) using `agentapi send-message`! + +### Example Invocation +```bash +./scripts/monitor_remote_ci.py 3812 "0be435bd-96aa-4e1b-9c6f-727b31e80fa0" & +``` diff --git a/.agents/skills/monitor-ci-results/scripts/monitor_remote_ci.py b/.agents/skills/monitor-ci-results/scripts/monitor_remote_ci.py new file mode 100755 index 0000000000..fc1f8955d3 --- /dev/null +++ b/.agents/skills/monitor-ci-results/scripts/monitor_remote_ci.py @@ -0,0 +1,208 @@ +#!/usr/bin/env python3 + +import argparse +import json +import os +import subprocess +import sys +import time +import urllib.request + + +def check_cli(cmd_name): + try: + subprocess.run( + [cmd_name, "--version"], + check=True, + stdout=subprocess.DEVNULL, + stderr=subprocess.DEVNULL, + ) + return True + except Exception: + return False + + +def get_pr_checks(pr_number): + if not check_cli("gh"): + print("❌ 'gh' CLI not installed.", file=sys.stderr) + return [] + cmd = ["gh", "pr", "checks", str(pr_number), "--json", "name,link,state"] + try: + res = subprocess.run(cmd, capture_output=True, text=True) + out = res.stdout + json_str = out[out.find("[") : out.rfind("]") + 1] if "[" in out else "[]" + return json.loads(json_str) + except Exception as e: + print(f"⚠️ Error fetching PR checks: {e}", file=sys.stderr) + return [] + + +def get_buildkite_jobs(build_url): + base_url = build_url.split("#")[0] + if base_url.endswith(".json"): + base_url = base_url[:-5] + + jobs_url = f"{base_url}/data/jobs.json" + req = urllib.request.Request(jobs_url, headers={"User-Agent": "ci-monitor"}) + try: + with urllib.request.urlopen(req) as resp: + data = json.loads(resp.read().decode()) + if isinstance(data, list): + return data + elif isinstance(data, dict) and "records" in data: + return data["records"] + except Exception as e: + print( + f"⚠️ Could not fetch Buildkite jobs from {jobs_url}: {e}", + file=sys.stderr, + ) + return [] + + +def main(): + parser = argparse.ArgumentParser( + description="Monitor remote CI for failures and trigger analysis." + ) + parser.add_argument("pr", help="PR number to monitor") + parser.add_argument("conv_id", help="Conversation ID to report back to") + parser.add_argument( + "--interval", + type=int, + default=60, + help="Monitoring polling interval in seconds", + ) + parser.add_argument( + "--max-iterations", + type=int, + default=120, + help="Maximum number of polling cycles", + ) + args = parser.parse_args() + + skill_dir = os.path.abspath(os.path.dirname(__file__)) + + state_file = os.path.join(skill_dir, f"monitored_state_pr_{args.pr}.json") + monitored = {} + if os.path.exists(state_file): + try: + with open(state_file) as f: + monitored = json.load(f) + except Exception: + pass + + print( + f"🚀 Starting continuous remote CI monitoring for PR #{args.pr} every {args.interval}s..." + ) + + for i in range(args.max_iterations): + print( + f"🔍 [Cycle {i + 1}/{args.max_iterations}] Polling GitHub PR #{args.pr} checks..." + ) + checks = get_pr_checks(args.pr) + + for check in checks: + name = check.get("name", "unknown") + state = check.get("state", "UNKNOWN") + link = check.get("link", "") + + if "buildkite" in name.lower() and link: + jobs = get_buildkite_jobs(link) + + passed = 0 + failed = 0 + running = 0 + other = 0 + + for job in jobs: + jstate = job.get("state", "unknown") + exit_status = job.get("exit_status") + is_soft_failed = job.get("soft_failed") is True + is_failed = ( + jstate in ["failed", "failing"] + or (exit_status != 0 and exit_status is not None) + ) and not is_soft_failed + is_passed = ( + jstate in ["passed", "success"] + or (jstate == "finished" and exit_status == 0) + or is_soft_failed + ) + is_running = jstate in ["running", "scheduled"] + + if is_failed: + failed += 1 + elif is_passed: + passed += 1 + elif is_running: + running += 1 + else: + other += 1 + + build_id = link.split("/")[-1].split("#")[0] + print( + f"Buildkite #{build_id}: {len(jobs)} total jobs " + f"(Passed: {passed}, Failed: {failed}, Running: {running}, Other: {other})" + ) + + for job in jobs: + jname = job.get("name", "unknown_job") + jstate = job.get("state", "unknown") + jid = job.get("id", "") + jkey = f"bk_{jid}" + + exit_status = job.get("exit_status") + is_soft_failed = job.get("soft_failed") is True + is_failed = ( + jstate in ["failed", "failing"] + or (exit_status != 0 and exit_status is not None) + ) and not is_soft_failed + + if is_failed and jkey not in monitored: + print( + f"🚨 Notifying failure for Buildkite job '{jname}' (ID: {jid})..." + ) + msg = ( + f"⚠️ Remote CI Buildkite Job '{jname}' completed with errors!\n\n" + f"Build ID: {build_id} | Job ID: {jid}\n" + f"Log URL: {job.get('log_url', link)}\n\n" + f"Start subagent: run analyze-ci-failure skill on this failure" + ) + subprocess.run( + [ + "agentapi", + "send-message", + "--title=CI Job Failed", + args.conv_id, + msg, + ] + ) + monitored[jkey] = time.time() + with open(state_file, "w") as f: + json.dump(monitored, f) + + elif state in ["FAILURE", "failed"] and name not in monitored: + print(f"🚨 Notifying failure for GitHub check '{name}'...") + msg = ( + f"⚠️ Remote CI GitHub Check '{name}' completed with errors!\n\n" + f"Link: {link}\n\n" + f"Start subagent: run analyze-ci-failure skill on this failure" + ) + subprocess.run( + [ + "agentapi", + "send-message", + "--title=CI Check Failed", + args.conv_id, + msg, + ] + ) + monitored[name] = time.time() + with open(state_file, "w") as f: + json.dump(monitored, f) + + time.sleep(args.interval) + + print("🏁 CI monitoring service completed its scheduled iterations.") + + +if __name__ == "__main__": + main() diff --git a/.bazelrc b/.bazelrc index 8f784e4c2b..044990fdb1 100644 --- a/.bazelrc +++ b/.bazelrc @@ -46,6 +46,9 @@ common --enable_bzlmod common --disk_cache=~/.cache/bazel/bazel-disk-cache # Drop `experimental_` prefix once Bazel 7 is no longer supported common --experimental_downloader_config=downloader_config.cfg +common --http_timeout_scaling=10.0 +common --experimental_repository_downloader_retries=10 + # Additional config to use for readthedocs builds. diff --git a/.gitattributes b/.gitattributes index 9905cbfacb..fafafd001b 100644 --- a/.gitattributes +++ b/.gitattributes @@ -1,3 +1,5 @@ python/features.bzl export-subst tools/publish/*.txt linguist-generated=true tests/uv/lock/testdata/requirements.txt text eol=lf +python/private/runtimes_manifest_workspace.bzl text eol=lf +python/private/runtimes_manifest.txt text eol=lf diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml index cada605a26..e4d65c8bb6 100644 --- a/.pre-commit-config.yaml +++ b/.pre-commit-config.yaml @@ -48,3 +48,20 @@ repos: entry: ./tools/update_deleted_packages.sh files: ^((examples|tests)/.*/(MODULE.bazel|WORKSPACE|WORKSPACE.bzlmod|BUILD.bazel)|.bazelrc)$ pass_filenames: false + - id: sort-runtimes-manifest + name: Sort runtimes manifest + language: system + entry: ./python/private/tools/sort_manifest.py + files: ^python/private/runtimes_manifest\.txt$ + - id: sync-runtimes-manifest + name: Sync runtimes manifest workspace + language: system + entry: ./python/private/tools/sync_runtimes_manifest_workspace.py + files: ^python/private/runtimes_manifest\.txt$ + pass_filenames: false + - id: sync-downloader-configs + name: Sync downloader configs + language: system + entry: ./tools/private/sync_downloader_configs.py + files: downloader_config\.cfg$ + pass_filenames: false diff --git a/AGENTS.md b/AGENTS.md index 4e1e88aeac..e6e1733c1d 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -15,6 +15,7 @@ Ask for user input and provide a justificaiton if trying to violate them. * NEVER run `bazel clean --expunge`. * Once a PR is created, do not amend or rebase. +* Do not add Bazel copyright to new or existing files. ## Style and conventions diff --git a/CHANGELOG.md b/CHANGELOG.md index 978464f9db..0fa832e8fc 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -61,6 +61,9 @@ END_UNRELEASED_TEMPLATE {#v0-0-0-changed} ### Changed +* (bzlmod) How default runtimes are registered has changed to use a manifest + of SHAs and URLs. `TOOL_VERSIONS` in `python/versions.bzl` is now empty under + bzlmod. * (gazelle) WORKSPACE's bazel-gazelle dependency bumped from 0.36.0 to 0.47.0. The go version was also bumped from 1.21.13 to 1.22.9. * (gazelle) `python_generate_pyi_deps` and `python_generate_pyi_srcs` now diff --git a/downloader_config.cfg b/downloader_config.cfg index a978fb89b9..3fa6264eda 100644 --- a/downloader_config.cfg +++ b/downloader_config.cfg @@ -5,6 +5,9 @@ rewrite ^github\.com/bazelbuild/bazel-skylib/(.*) github.com/bazelbuild/bazel-sk rewrite ^github\.com/bazelbuild/platforms/(.*) github.com/bazelbuild/platforms/$1 rewrite ^github\.com/bazelbuild/rules_kotlin/(.*) github.com/bazelbuild/rules_kotlin/$1 rewrite ^github\.com/bazelbuild/rules_shell/(.*) github.com/bazelbuild/rules_shell/$1 +rewrite ^github\.com/bazelbuild/rules_java/(.*) github.com/bazelbuild/rules_java/$1 +rewrite ^github\.com/bazelbuild/stardoc/(.*) github.com/bazelbuild/stardoc/$1 + # Fall back to mirror (secondary) # Tracking upstream BCR mirror addition: https://github.com/bazelbuild/platforms/issues/139 @@ -14,3 +17,5 @@ rewrite ^github\.com/bazelbuild/bazel-skylib/(.*) mirror.bazel.build/github.com/ rewrite ^github\.com/bazelbuild/platforms/(.*) mirror.bazel.build/github.com/bazelbuild/platforms/$1 rewrite ^github\.com/bazelbuild/rules_kotlin/(.*) mirror.bazel.build/github.com/bazelbuild/rules_kotlin/$1 rewrite ^github\.com/bazelbuild/rules_shell/(.*) mirror.bazel.build/github.com/bazelbuild/rules_shell/$1 +rewrite ^github\.com/bazelbuild/rules_java/(.*) mirror.bazel.build/github.com/bazelbuild/rules_java/$1 +rewrite ^github\.com/bazelbuild/stardoc/(.*) mirror.bazel.build/github.com/bazelbuild/stardoc/$1 diff --git a/gazelle/.bazelrc b/gazelle/.bazelrc index bdb29d5bc3..e30216814a 100644 --- a/gazelle/.bazelrc +++ b/gazelle/.bazelrc @@ -1,6 +1,9 @@ common --deleted_packages=examples/bzlmod_build_file_generation common --deleted_packages=examples/bzlmod_build_file_generation/runfiles common --experimental_downloader_config=downloader_config.cfg +common --http_timeout_scaling=10.0 +common --experimental_repository_downloader_retries=10 + test --test_output=errors diff --git a/gazelle/downloader_config.cfg b/gazelle/downloader_config.cfg index a978fb89b9..3fa6264eda 100644 --- a/gazelle/downloader_config.cfg +++ b/gazelle/downloader_config.cfg @@ -5,6 +5,9 @@ rewrite ^github\.com/bazelbuild/bazel-skylib/(.*) github.com/bazelbuild/bazel-sk rewrite ^github\.com/bazelbuild/platforms/(.*) github.com/bazelbuild/platforms/$1 rewrite ^github\.com/bazelbuild/rules_kotlin/(.*) github.com/bazelbuild/rules_kotlin/$1 rewrite ^github\.com/bazelbuild/rules_shell/(.*) github.com/bazelbuild/rules_shell/$1 +rewrite ^github\.com/bazelbuild/rules_java/(.*) github.com/bazelbuild/rules_java/$1 +rewrite ^github\.com/bazelbuild/stardoc/(.*) github.com/bazelbuild/stardoc/$1 + # Fall back to mirror (secondary) # Tracking upstream BCR mirror addition: https://github.com/bazelbuild/platforms/issues/139 @@ -14,3 +17,5 @@ rewrite ^github\.com/bazelbuild/bazel-skylib/(.*) mirror.bazel.build/github.com/ rewrite ^github\.com/bazelbuild/platforms/(.*) mirror.bazel.build/github.com/bazelbuild/platforms/$1 rewrite ^github\.com/bazelbuild/rules_kotlin/(.*) mirror.bazel.build/github.com/bazelbuild/rules_kotlin/$1 rewrite ^github\.com/bazelbuild/rules_shell/(.*) mirror.bazel.build/github.com/bazelbuild/rules_shell/$1 +rewrite ^github\.com/bazelbuild/rules_java/(.*) mirror.bazel.build/github.com/bazelbuild/rules_java/$1 +rewrite ^github\.com/bazelbuild/stardoc/(.*) mirror.bazel.build/github.com/bazelbuild/stardoc/$1 diff --git a/internal_dev_deps.bzl b/internal_dev_deps.bzl index 50277ad4ad..7ab4717236 100644 --- a/internal_dev_deps.bzl +++ b/internal_dev_deps.bzl @@ -12,7 +12,7 @@ # See the License for the specific language governing permissions and # limitations under the License. -"""Dependencies that are needed for development and testing of rules_python itself.""" +"""Dependencies that are needed for development and testing of rules_python itself in WORKSPACE mode.""" load("@bazel_tools//tools/build_defs/repo:http.bzl", _http_archive = "http_archive", _http_file = "http_file") load("@bazel_tools//tools/build_defs/repo:local.bzl", "local_repository") @@ -34,7 +34,7 @@ def http_file(name, **kwargs): ) def rules_python_internal_deps(): - """Fetches all required dependencies for developing/testing rules_python itself. + """Fetches all required dependencies for developing/testing rules_python itself in WORKSPACE mode. Setup of these dependencies is done by `internal_dev_setup.bzl` diff --git a/python/BUILD.bazel b/python/BUILD.bazel index 90b2225ab5..e940e9e94b 100644 --- a/python/BUILD.bazel +++ b/python/BUILD.bazel @@ -226,7 +226,10 @@ bzl_library( name = "versions_bzl", srcs = ["versions.bzl"], visibility = ["//:__subpackages__"], - deps = ["//python/private:platform_info_bzl"], + deps = [ + "//python/private:platform_info_bzl", + "//python/private:runtimes_manifest_workspace_bzl", + ], ) # NOTE: Remember to add bzl_library targets to //tests:bzl_libraries diff --git a/python/private/BUILD.bazel b/python/private/BUILD.bazel index b54c198069..e4b38c2d1d 100644 --- a/python/private/BUILD.bazel +++ b/python/private/BUILD.bazel @@ -17,7 +17,6 @@ load("@bazel_skylib//rules:common_settings.bzl", "bool_flag") load("//python:py_binary.bzl", "py_binary") load("//python:py_library.bzl", "py_library") load(":bazel_config_mode.bzl", "bazel_config_mode") -load(":print_toolchain_checksums.bzl", "print_toolchains_checksums") load(":py_exec_tools_toolchain.bzl", "current_interpreter_executable") load(":sentinel.bzl", "sentinel") load(":stamp.bzl", "stamp_build_setting") @@ -31,7 +30,11 @@ package( licenses(["notice"]) -exports_files(["runtime_env_toolchain_interpreter.sh"]) +exports_files([ + "runtime_env_toolchain_interpreter.sh", + "runtimes_manifest.txt", + "runtimes_manifest_workspace.bzl", +]) filegroup( name = "distribution", @@ -216,6 +219,7 @@ bzl_library( name = "internal_config_repo_bzl", srcs = ["internal_config_repo.bzl"], deps = [ + ":pbs_manifest_bzl", ":repo_utils_bzl", ":text_util_bzl", ], @@ -298,7 +302,6 @@ bzl_library( ":bazel_tools_bzl", ":coverage_deps_bzl", ":full_version_bzl", - ":internal_config_repo_bzl", ":python_repository_bzl", ":repo_utils_bzl", ":toolchains_repo_bzl", @@ -332,6 +335,7 @@ bzl_library( name = "pythons_hub_bzl", srcs = ["pythons_hub.bzl"], deps = [ + ":pbs_manifest_bzl", ":py_toolchain_suite_bzl", ":text_util_bzl", "//python:versions_bzl", @@ -655,6 +659,11 @@ bzl_library( ], ) +bzl_library( + name = "runtimes_manifest_workspace_bzl", + srcs = ["runtimes_manifest_workspace.bzl"], +) + bzl_library( name = "sentinel_bzl", srcs = ["sentinel.bzl"], @@ -857,8 +866,6 @@ bool_flag( visibility = ["//visibility:public"], ) -print_toolchains_checksums(name = "print_toolchains_checksums") - # Used for py_console_script_gen rule py_binary( name = "py_console_script_gen_py", @@ -900,3 +907,9 @@ py_library( sentinel( name = "sentinel", ) + +py_binary( + name = "sync_runtimes_manifest_workspace", + srcs = ["tools/sync_runtimes_manifest_workspace.py"], + visibility = ["//:__subpackages__"], +) diff --git a/python/private/internal_config_repo.bzl b/python/private/internal_config_repo.bzl index aea1eee773..72970cf100 100644 --- a/python/private/internal_config_repo.bzl +++ b/python/private/internal_config_repo.bzl @@ -49,13 +49,13 @@ package( ) bzl_library( - name = "rules_python_config_bzl", - srcs = ["rules_python_config.bzl"] + name = "extra_transition_settings_bzl", + srcs = ["extra_transition_settings.bzl"], ) bzl_library( - name = "extra_transition_settings_bzl", - srcs = ["extra_transition_settings.bzl"], + name = "rules_python_config_bzl", + srcs = ["rules_python_config.bzl"], ) """ diff --git a/python/private/pbs_manifest.bzl b/python/private/pbs_manifest.bzl index e343a802b3..86434dc0ea 100644 --- a/python/private/pbs_manifest.bzl +++ b/python/private/pbs_manifest.bzl @@ -101,8 +101,8 @@ def parse_filename(filename): return { "arch": arch, "archive_flavor": archive_flavor, + "build_flavor": flavor, "build_version": build_version, - "flavor": flavor, "freethreaded": freethreaded, "libc": libc, "location": filename, @@ -112,7 +112,7 @@ def parse_filename(filename): "vendor": vendor, } -def parse_sha_manifest(content): +def parse_runtime_manifest(content): """Parses the SHA256SUMS file content into a list of structs. Args: @@ -125,7 +125,7 @@ def parse_sha_manifest(content): - archive_flavor: Release asset archive type (e.g., "full", "install_only"). - build_version: Standalone release date (e.g., "20260414"). - location: Full package filename or URL (e.g., "cpython-3.11.15..." or "https://..."). - - flavor: Build configuration flavor (e.g., "install_only"). + - build_flavor: Build configuration flavor (e.g., "debug", "pgo+lto"). - freethreaded: Whether the build is free-threaded (boolean). - libc: C library type (e.g., "gnu", "musl", "msvc", or ""). - microarch: Microarchitecture level (e.g., "v2", "v3", or ""). @@ -137,13 +137,12 @@ def parse_sha_manifest(content): results = [] for line in content.split("\n"): line = line.strip() - if not line: + if not line or line.startswith("#"): continue parts = [p for p in line.split(" ") if p] if len(parts) != 2: continue sha256, filename = parts - parsed = parse_filename(filename) if parsed: results.append(struct( diff --git a/python/private/print_toolchain_checksums.bzl b/python/private/print_toolchain_checksums.bzl deleted file mode 100644 index b4fa400221..0000000000 --- a/python/private/print_toolchain_checksums.bzl +++ /dev/null @@ -1,89 +0,0 @@ -"""Print the toolchain versions. -""" - -load("//python:versions.bzl", "TOOL_VERSIONS", "get_release_info") -load("//python/private:text_util.bzl", "render") -load("//python/private:version.bzl", "version") - -def print_toolchains_checksums(name): - """A macro to print checksums for a particular Python interpreter version. - - Args: - name: {type}`str`: the name of the runnable target. - """ - by_version = {} - - for python_version, metadata in TOOL_VERSIONS.items(): - by_version[python_version] = _commands_for_version( - python_version = python_version, - metadata = metadata, - ) - - all_commands = sorted( - by_version.items(), - key = lambda x: version.key(version.parse(x[0], strict = True)), - ) - all_commands = [x[1] for x in all_commands] - - template = """\ -cat > "$@" <<'EOF' -#!/usr/bin/env bash -set -euo pipefail - -set -o errexit -o nounset -o pipefail - -echo "Fetching hashes..." - -{commands} -EOF - """ - - native.genrule( - name = name, - srcs = [], - outs = ["print_toolchains_checksums.sh"], - cmd = select({ - "//python/config_settings:is_python_{}".format(version_str): template.format( - commands = commands, - ) - for version_str, commands in by_version.items() - } | { - "//conditions:default": template.format(commands = "\n".join(all_commands)), - }), - executable = True, - ) - -def _commands_for_version(*, python_version, metadata): - lines = [] - first_platform = metadata["sha256"].keys()[0] - root, _, _ = get_release_info(first_platform, python_version)[1][0].rpartition("/") - sha_url = "{}/{}".format(root, "SHA256SUMS") - prefix = metadata["strip_prefix"] - prefix = render.indent( - render.dict(prefix) if type(prefix) == type({}) else repr(prefix), - indent = " " * 8, - ).lstrip() - - lines += [ - "sha256s=$$(curl --silent --show-error --location --fail {})".format(sha_url), - "cat < bool: + """Sorts a manifest file in place by filename. Returns True if modified.""" + # Read using pathlib.Path + lines = manifest_path.read_text(encoding="utf-8").splitlines(keepends=True) + + if not lines: + return False + + first_entry_idx = -1 + last_entry_idx = -1 + for idx, line in enumerate(lines): + stripped = line.strip() + if stripped and not stripped.startswith("#"): + if first_entry_idx == -1: + first_entry_idx = idx + last_entry_idx = idx + + if first_entry_idx == -1: + return False + + # Extract top-level comments (comments at the top followed by a blank newline) + top_level_comments = [] + pre_entry_lines = lines[:first_entry_idx] + + last_blank_idx = -1 + for idx, line in enumerate(pre_entry_lines): + if not line.strip(): + last_blank_idx = idx + + if last_blank_idx != -1: + top_level_comments = pre_entry_lines[: last_blank_idx + 1] + remaining_pre = pre_entry_lines[last_blank_idx + 1 :] + else: + remaining_pre = pre_entry_lines + + # Extract bottom-level comments + bottom_level_comments = lines[last_entry_idx + 1 :] + + # Group middle lines into actual catalog entries with their attached comments/blank lines + middle_lines = remaining_pre + lines[first_entry_idx : last_entry_idx + 1] + + entries = [] + current_attached = [] + + for line in middle_lines: + stripped = line.strip() + if stripped and not stripped.startswith("#"): + parts = [p for p in stripped.split(" ") if p] + if len(parts) == 2: + sha256, filename = parts[0], parts[1] + normalized_line = f"{sha256} {filename}\n" + else: + filename = parts[0] if parts else "" + normalized_line = line + + block = current_attached + [normalized_line] + entries.append((filename, block)) + current_attached = [] + else: + current_attached.append(line) + + if current_attached: + bottom_level_comments = current_attached + bottom_level_comments + + # Sort entries lexicographically by filename + entries.sort(key=lambda e: e[0]) + + new_lines = top_level_comments + for _, block in entries: + new_lines.extend(block) + new_lines.extend(bottom_level_comments) + + if new_lines == lines: + return False + + manifest_path.write_text("".join(new_lines), encoding="utf-8") + return True + + +def main(): + parser = argparse.ArgumentParser(description="Sort manifest files by filename.") + parser.add_argument( + "manifests", + nargs="*", + type=Path, + help="Path to manifest files to sort.", + ) + args = parser.parse_args() + + manifests = args.manifests + if not manifests: + repo_root = Path(__file__).resolve().parent.parent.parent.parent + default_manifest = repo_root / "python" / "private" / "runtimes_manifest.txt" + if default_manifest.exists(): + manifests = [default_manifest] + else: + print("No manifests provided.", file=sys.stderr) + sys.exit(1) + + changed = False + for m in manifests: + if m.exists(): + if sort_manifest(m): + print(f"Sorted {m}") + changed = True + else: + print(f"Warning: Manifest not found: {m}", file=sys.stderr) + + if changed: + sys.exit(1) + + +if __name__ == "__main__": + main() diff --git a/python/private/tools/sync_runtimes_manifest_workspace.py b/python/private/tools/sync_runtimes_manifest_workspace.py new file mode 100755 index 0000000000..450358542a --- /dev/null +++ b/python/private/tools/sync_runtimes_manifest_workspace.py @@ -0,0 +1,80 @@ +#!/usr/bin/env python3 + +"""Synchronizes runtimes_manifest_workspace.bzl with runtimes_manifest.txt.""" + +import argparse +import sys +from pathlib import Path + + +def sync_workspace_manifest(txt_path: Path, bzl_path: Path) -> bool: + with open(txt_path, "r", encoding="utf-8") as f: + txt_content = f.read() + header = '''"""Manifest of runtimes for workspace mode builds. + +This is the workspace equivalent of runtimes_manifest.txt. It's a bzl file +to simplify loading of the data under workspace mode, which doesn't +support parsing a runtimes_manifest.txt file. + +NOTE: This file is automatically generated by sync_runtimes_manifest_workspace.py. +Do not edit directly! +""" + +MANIFEST_TEXT = """ +''' + + new_content = header + txt_content + '"""\n' + + if bzl_path.exists(): + with open(bzl_path, "r", encoding="utf-8") as f: + old_content = f.read() + else: + old_content = "" + + if new_content != old_content: + with open(bzl_path, "w", encoding="utf-8", newline="\n") as f: + f.write(new_content) + return True + + return False + + +def main(): + parser = argparse.ArgumentParser( + description="Sync runtimes workspace bzl file with runtimes manifest text file." + ) + parser.add_argument( + "txt_path", + nargs="?", + type=Path, + help="Path to runtimes_manifest.txt", + ) + parser.add_argument( + "bzl_path", + nargs="?", + type=Path, + help="Path to runtimes_manifest_workspace.bzl", + ) + args = parser.parse_args() + + txt_path = args.txt_path + bzl_path = args.bzl_path + + if not txt_path or not bzl_path: + repo_root = Path(__file__).resolve().parent.parent.parent.parent + txt_path = repo_root / "python" / "private" / "runtimes_manifest.txt" + bzl_path = repo_root / "python" / "private" / "runtimes_manifest_workspace.bzl" + + if not txt_path.exists(): + print(f"Error: Manifest not found: {txt_path}", file=sys.stderr) + sys.exit(1) + + if sync_workspace_manifest(txt_path, bzl_path): + print(f"Updated {bzl_path}") + if not args.bzl_path: + # Exit 1 for pre-commit mode (in-place modification) + sys.exit(1) + + +if __name__ == "__main__": + main() diff --git a/python/versions.bzl b/python/versions.bzl index ed819e6b34..6e3efb34d6 100644 --- a/python/versions.bzl +++ b/python/versions.bzl @@ -15,8 +15,12 @@ """The Python versions we use for the toolchains. """ +load("//python/private:pbs_manifest.bzl", "parse_runtime_manifest") load("//python/private:platform_info.bzl", "platform_info") +##load("@rules_python_internal//:manifest_tool_versions.bzl", "MANIFEST_ENTRIES") +load("//python/private:runtimes_manifest_workspace.bzl", "MANIFEST_TEXT") + # Values present in the @platforms//os package MACOS_NAME = "osx" LINUX_NAME = "linux" @@ -26,1239 +30,16 @@ FREETHREADED = "-freethreaded" MUSL = "-musl" INSTALL_ONLY = "install_only" -DEFAULT_RELEASE_BASE_URL = "https://github.com/astral-sh/python-build-standalone/releases/download" - _GITHUB_PREFIX = "https://github.com/astral-sh/python-build-standalone/releases/download" _LEGACY_GITHUB_PREFIX = "https://github.com/indygreg/python-build-standalone/releases/download" _ASTRAL_PREFIX = "https://releases.astral.sh/github/python-build-standalone/releases/download" -# When updating the versions and releases, run the following command to get -# the hashes: -# bazel run //python/private:print_toolchains_checksums --//python/config_settings:python_version={major}.{minor}.{patch} -# -# To print hashes for all of the specified versions, run: -# bazel run //python/private:print_toolchains_checksums --//python/config_settings:python_version="" -# -# Note, to users looking at how to specify their tool versions, coverage_tool version for each -# interpreter can be specified by: -# "3.8.10": { -# "url": "20210506/cpython-{python_version}-{platform}-pgo+lto-20210506T0943.tar.zst", -# "sha256": { -# "x86_64-apple-darwin": "8d06bec08db8cdd0f64f4f05ee892cf2fcbc58cfb1dd69da2caab78fac420238", -# "x86_64-unknown-linux-gnu": "aec8c4c53373b90be7e2131093caa26063be6d9d826f599c935c0e1042af3355", -# }, -# "coverage_tool": { -# "x86_64-apple-darwin": """, -# "x86_64-unknown-linux-gnu": """, -# }, -# "strip_prefix": "python", -# }, -# -# It is possible to provide lists in "url". It is also possible to provide patches or patch_strip. -# -# buildifier: disable=unsorted-dict-items -TOOL_VERSIONS = { - "3.9.25": { - "url": "20251031/cpython-{python_version}+20251031-{platform}-{build}.tar.gz", - "sha256": { - "aarch64-apple-darwin": "87275619c2706affa4d1090d2ca3dad354b6d69f8b85dbfafe38785870751b9a", - "aarch64-unknown-linux-gnu": "6112d46355857680b81849764a6cf9f38cc4cd0d1cf29d432bc12fe5aeedf9d0", - "ppc64le-unknown-linux-gnu": "828364b6f54fa45ac2dc91f8e45d5b74306372af374a9ef16eeb2ea81253ed3f", - "riscv64-unknown-linux-gnu": "17467e0158e5ad04453c447d6773c23b044172276441e22e23058fd3ea053e27", - "s390x-unknown-linux-gnu": "3e9539f83e67faa813fd06171199b2d33c89821dfa9a33bf6e27ad67f1b6932d", - "x86_64-apple-darwin": "ace63cfe27a9487c4d72e1cb518be01c1d985271da0b2158e813801f7d3e5503", - "x86_64-pc-windows-msvc": "4fb1b416482ce94d73cfa140317a670c596c830671d137b07c26afe8c461768a", - "x86_64-unknown-linux-gnu": "42834f61eb6df43432c3dd6ab9ca3fdf8c06d10a404ebdb53d6902e6b9570b08", - "x86_64-unknown-linux-musl": "76593e8c889e81e82db5fe117fe15b69466f85100ab2ec0e4035aa86242b4e93", - }, - "strip_prefix": "python", - }, - "3.10.2": { - "url": "20220227/cpython-{python_version}+20220227-{platform}-{build}.tar.gz", - "sha256": { - "aarch64-apple-darwin": "1409acd9a506e2d1d3b65c1488db4e40d8f19d09a7df099667c87a506f71c0ef", - "aarch64-unknown-linux-gnu": "8f351a8cc348bb45c0f95b8634c8345ec6e749e483384188ad865b7428342703", - "x86_64-apple-darwin": "8146ad4390710ec69b316a5649912df0247d35f4a42e2aa9615bffd87b3e235a", - "x86_64-pc-windows-msvc": "a1d9a594cd3103baa24937ad9150c1a389544b4350e859200b3e5c036ac352bd", - "x86_64-unknown-linux-gnu": "9b64eca2a94f7aff9409ad70bdaa7fbbf8148692662e764401883957943620dd", - }, - "strip_prefix": "python", - }, - "3.10.4": { - "url": "20220502/cpython-{python_version}+20220502-{platform}-{build}.tar.gz", - "sha256": { - "aarch64-apple-darwin": "2c99983d1e83e4b6e7411ed9334019f193fba626344a50c36fba6c25d4de78a2", - "aarch64-unknown-linux-gnu": "d8098c0c54546637e7516f93b13403b11f9db285def8d7abd825c31407a13d7e", - "x86_64-apple-darwin": "f2711eaffff3477826a401d09a013c6802f11c04c63ab3686aa72664f1216a05", - "x86_64-pc-windows-msvc": "bee24a3a5c83325215521d261d73a5207ab7060ef3481f76f69b4366744eb81d", - "x86_64-unknown-linux-gnu": "f6f871e53a7b1469c13f9bd7920ad98c4589e549acad8e5a1e14760fff3dd5c9", - }, - "strip_prefix": "python", - }, - "3.10.6": { - "url": "20220802/cpython-{python_version}+20220802-{platform}-{build}.tar.gz", - "sha256": { - "aarch64-apple-darwin": "efaf66acdb9a4eb33d57702607d2e667b1a319d58c167a43c96896b97419b8b7", - "aarch64-unknown-linux-gnu": "81625f5c97f61e2e3d7e9f62c484b1aa5311f21bd6545451714b949a29da5435", - "x86_64-apple-darwin": "7718411adf3ea1480f3f018a643eb0550282aefe39e5ecb3f363a4a566a9398c", - "x86_64-pc-windows-msvc": "91889a7dbdceea585ff4d3b7856a6bb8f8a4eca83a0ff52a73542c2e67220eaa", - "x86_64-unknown-linux-gnu": "55aa2190d28dcfdf414d96dc5dcea9fe048fadcd583dc3981fec020869826111", - }, - "strip_prefix": "python", - }, - "3.10.8": { - "url": "20221106/cpython-{python_version}+20221106-{platform}-{build}.tar.gz", - "sha256": { - "aarch64-apple-darwin": "d52b03817bd245d28e0a8b2f715716cd0fcd112820ccff745636932c76afa20a", - "aarch64-unknown-linux-gnu": "33170bef18c811906b738be530f934640491b065bf16c4d276c6515321918132", - "x86_64-apple-darwin": "525b79c7ce5de90ab66bd07b0ac1008bafa147ddc8a41bef15ffb7c9c1e9e7c5", - "x86_64-pc-windows-msvc": "f2b6d2f77118f06dd2ca04dae1175e44aaa5077a5ed8ddc63333c15347182bfe", - "x86_64-unknown-linux-gnu": "6c8db44ae0e18e320320bbaaafd2d69cde8bfea171ae2d651b7993d1396260b7", - }, - "strip_prefix": "python", - }, - "3.10.9": { - "url": "20230116/cpython-{python_version}+20230116-{platform}-{build}.tar.gz", - "sha256": { - "aarch64-apple-darwin": "018d05a779b2de7a476f3b3ff2d10f503d69d14efcedd0774e6dab8c22ef84ff", - "aarch64-unknown-linux-gnu": "2003750f40cd09d4bf7a850342613992f8d9454f03b3c067989911fb37e7a4d1", - "x86_64-apple-darwin": "0e685f98dce0e5bc8da93c7081f4e6c10219792e223e4b5886730fd73a7ba4c6", - "x86_64-pc-windows-msvc": "59c6970cecb357dc1d8554bd0540eb81ee7f6d16a07acf3d14ed294ece02c035", - "x86_64-unknown-linux-gnu": "d196347aeb701a53fe2bb2b095abec38d27d0fa0443f8a1c2023a1bed6e18cdf", - }, - "strip_prefix": "python", - }, - "3.10.11": { - "url": "20230507/cpython-{python_version}+20230507-{platform}-{build}.tar.gz", - "sha256": { - "aarch64-apple-darwin": "8348bc3c2311f94ec63751fb71bd0108174be1c4def002773cf519ee1506f96f", - "aarch64-unknown-linux-gnu": "c7573fdb00239f86b22ea0e8e926ca881d24fde5e5890851339911d76110bc35", - "ppc64le-unknown-linux-gnu": "73a9d4c89ed51be39dd2de4e235078281087283e9fdedef65bec02f503e906ee", - "x86_64-apple-darwin": "bd3fc6e4da6f4033ebf19d66704e73b0804c22641ddae10bbe347c48f82374ad", - "x86_64-pc-windows-msvc": "9c2d3604a06fcd422289df73015cd00e7271d90de28d2c910f0e2309a7f73a68", - "x86_64-unknown-linux-gnu": "c5bcaac91bc80bfc29cf510669ecad12d506035ecb3ad85ef213416d54aecd79", - }, - "strip_prefix": "python", - }, - "3.10.12": { - "url": "20230726/cpython-{python_version}+20230726-{platform}-{build}.tar.gz", - "sha256": { - "aarch64-apple-darwin": "bc66c706ea8c5fc891635fda8f9da971a1a901d41342f6798c20ad0b2a25d1d6", - "aarch64-unknown-linux-gnu": "fee80e221663eca5174bd794cb5047e40d3910dbeadcdf1f09d405a4c1c15fe4", - "ppc64le-unknown-linux-gnu": "bb5e8cb0d2e44241725fa9b342238245503e7849917660006b0246a9c97b1d6c", - "s390x-unknown-linux-gnu": "8d33d435ae6fb93ded7fc26798cc0a1a4f546a4e527012a1e2909cc314b332df", - "x86_64-apple-darwin": "8a6e3ed973a671de468d9c691ed9cb2c3a4858c5defffcf0b08969fba9c1dd04", - "x86_64-pc-windows-msvc": "c1a31c353ca44de7d1b1a3b6c55a823e9c1eed0423d4f9f66e617bdb1b608685", - "x86_64-unknown-linux-gnu": "a476dbca9184df9fc69fe6309cda5ebaf031d27ca9e529852437c94ec1bc43d3", - }, - "strip_prefix": "python", - }, - "3.10.13": { - "url": "20240224/cpython-{python_version}+20240224-{platform}-{build}.tar.gz", - "sha256": { - "aarch64-apple-darwin": "5fdc0f6a5b5a90fd3c528e8b1da8e3aac931ea8690126c2fdb4254c84a3ff04a", - "aarch64-unknown-linux-gnu": "a898a88705611b372297bb8fe4d23cc16b8603ce5f24494c3a8cfa65d83787f9", - "ppc64le-unknown-linux-gnu": "c23706e138a0351fc1e9def2974af7b8206bac7ecbbb98a78f5aa9e7535fee42", - "s390x-unknown-linux-gnu": "09be8fb2cdfbb4a93d555f268f244dbe4d8ff1854b2658e8043aa4ec08aede3e", - "x86_64-apple-darwin": "6378dfd22f58bb553ddb02be28304d739cd730c1f95c15c74955c923a1bc3d6a", - "x86_64-pc-windows-msvc": "086f7fe9156b897bb401273db8359017104168ac36f60f3af4e31ac7acd6634e", - "x86_64-unknown-linux-gnu": "d995d032ca702afd2fc3a689c1f84a6c64972ecd82bba76a61d525f08eb0e195", - }, - "strip_prefix": "python", - }, - "3.10.14": { - "url": "20240726/cpython-{python_version}+20240726-{platform}-{build}.tar.gz", - "sha256": { - "aarch64-apple-darwin": "164d89f0df2feb689981864ecc1dffb19e6aa3696c8880166de555494fe92607", - "aarch64-unknown-linux-gnu": "39bcd46b4d70e40da177c55259be16d5c2be7a3f7f93f1e3bde47e71b4833f29", - "ppc64le-unknown-linux-gnu": "549d38b9ef59cba9ab2990025255231bfa1cb32b4bc5eac321667640fdee19d1", - "s390x-unknown-linux-gnu": "de4bc878a8666c734f983db971610980870148f333bda8b0c34abfaeae88d7ec", - "x86_64-apple-darwin": "1a1455838cd1e8ed0da14a152a2d559a2fd3a6047ba7013e841db4a35a228c1d", - "x86_64-pc-windows-msvc": "7f68821a8b5445267eca480660364ebd06ec84632b336770c6e39de07ac0f6c3", - "x86_64-unknown-linux-gnu": "32b34cd13d9d745b3db3f3b8398ab2c07de74544829915dbebd8dce39bdc405e", - }, - "strip_prefix": "python", - }, - "3.10.15": { - "url": "20241016/cpython-{python_version}+20241016-{platform}-{build}.tar.gz", - "sha256": { - "aarch64-apple-darwin": "f64776f455a44c24d50f947c813738cfb7b9ac43732c44891bc831fa7940a33c", - "aarch64-unknown-linux-gnu": "eb58581f85fde83d1f3e8e1f8c6f5a15c7ae4fdbe3b1d1083931f9167fdd8dbc", - "ppc64le-unknown-linux-gnu": "0c45af4e7525e2db59901606db32b2896ac1e9830c6f95551402207f537c2ce4", - "s390x-unknown-linux-gnu": "de205896b070e6f5259ac0f2b3379eead875ea84e6a6ef533b89886fcbb46a4c", - "x86_64-apple-darwin": "90b46dfb1abd98d45663c7a2a8c45d3047a59391d8586d71b459cec7b75f662b", - "x86_64-pc-windows-msvc": "e48952619796c66ec9719867b87be97edca791c2ef7fbf87d42c417c3331609e", - "x86_64-unknown-linux-gnu": "3db2171e03c1a7acdc599fba583c1b92306d3788b375c9323077367af1e9d9de", - "x86_64-unknown-linux-musl": "ed519c47d9620eb916a6f95ec2875396e7b1a9ab993ee40b2f31b837733f318c", - }, - "strip_prefix": "python", - }, - "3.10.16": { - "url": "20250317/cpython-{python_version}+20250317-{platform}-{build}.tar.gz", - "sha256": { - "aarch64-apple-darwin": "e99f8457d9c79592c036489c5cfa78df76e4762d170665e499833e045d82608f", - "aarch64-unknown-linux-gnu": "76d0f04d2444e77200fdc70d1c57480e29cca78cb7420d713bc1c523709c198d", - "ppc64le-unknown-linux-gnu": "39c9b3486de984fe1d72d90278229c70d6b08bcf69cd55796881b2d75077b603", - "riscv64-unknown-linux-gnu": "ebe949ada9293581c17d9bcdaa8f645f67d95f73eac65def760a71ef9dd6600d", - "s390x-unknown-linux-gnu": "9b2fc0b7f1c75b48e799b6fa14f7e24f5c61f2db82e3c65d13ed25e08f7f0857", - "x86_64-apple-darwin": "e03e62dbe95afa2f56b7344ff3bd061b180a0b690ff77f9a1d7e6601935e05ca", - "x86_64-pc-windows-msvc": "c7e0eb0ff5b36758b7a8cacd42eb223c056b9c4d36eded9bf5b9fe0c0b9aeb08", - "x86_64-unknown-linux-gnu": "b350c7e63956ca8edb856b91316328e0fd003a840cbd63d08253af43b2c63643", - "x86_64-unknown-linux-musl": "6ed64923ee4fbea4c5780f1a5a66651d239191ac10bd23420db4f5e4e0bf79c4", - }, - "strip_prefix": "python", - }, - "3.10.18": { - "url": "20250808/cpython-{python_version}+20250808-{platform}-{build}.tar.gz", - "sha256": { - "aarch64-apple-darwin": "a94c02b2d597cd6b075a713fe4e9a909cc97ca6a3b2b2ce86eda21be2062d48e", - "aarch64-unknown-linux-gnu": "ef7de3b715d519e246d98ff7856247f7f7b357068705f09c6f300b7e7b76c701", - "ppc64le-unknown-linux-gnu": "f580efed11cc54e1a221c052e8bc88bfbc12844d3ca8949da828351a1232386e", - "riscv64-unknown-linux-gnu": "0d7e460e30203a9225b6f417ae972f66415a1cc0e32b37ebc48d195816282669", - "s390x-unknown-linux-gnu": "d4ada974daadb08a0184c19232ee3b03b3137aa70609760e1a94aaf7b12989ef", - "x86_64-apple-darwin": "da96fe2ba841640215788ddb9f151f03629360e37fcb94d4f76e5095b87df0d4", - "x86_64-pc-windows-msvc": "a648f3c9d136985ccfe57a5507e73d9d0839f7fd09eebd7c247857f2feaecb2a", - "x86_64-unknown-linux-gnu": "0b310a73bb9e7a495dbcad5f685e508ca2e7b36ee8f29301a52285730c425789", - "x86_64-unknown-linux-musl": "9cecf6ea2effbe183faebcf7e1160425a4ee17a68e49f2eefe5e1c59c51fa7ee", - }, - "strip_prefix": "python", - }, - "3.10.19": { - "url": "20251031/cpython-{python_version}+20251031-{platform}-{build}.tar.gz", - "sha256": { - "aarch64-apple-darwin": "43bda24c2fc073bc308bf631203b917a72640d59b59fdad4ba14503d84727012", - "aarch64-unknown-linux-gnu": "f77a8a8aa77f3f943126fa9215a25309da4bf20398fc8f4b4eec54b5fc7570ef", - "ppc64le-unknown-linux-gnu": "1c55d160fc4c3b93528cd6aaa2bb4ca6018a99e5a45919d33dc761a43a69f860", - "riscv64-unknown-linux-gnu": "21134d35721cdad4c881f35d0957cc19df9a45d194afb38a099faded3c1cfb4d", - "s390x-unknown-linux-gnu": "df0db070f1eb73ab4e371eea32213ddb3500737ea5560a6f0ffd65c82af64ddc", - "x86_64-apple-darwin": "76c12e633c09c2a790f8a958a55df4495527e0718d1875310c836e757c0c7b55", - "x86_64-pc-windows-msvc": "cfa08a4caf2df1b43551b843c052d6a8814e2ea0c97268b021f0423646c244c3", - "x86_64-unknown-linux-gnu": "fb1caac917d7b6497bb6f5950da5f1e48d05c43a498948dd97f85760c4382d9f", - "x86_64-unknown-linux-musl": "ba85013ed5ac7733fc6840168cc33ed19e9959b363dc80227d54f8fd9c92c0f4", - }, - "strip_prefix": "python", - }, - "3.10.20": { - "url": "20260414/cpython-{python_version}+20260414-{platform}-{build}.{ext}", - "sha256": { - "aarch64-apple-darwin": "f76cc83c7db16cfc8794bf6e44d834152b57d8bab4e04e823cbc59ed23ec22f8", - "aarch64-unknown-linux-gnu": "64932c8e8bbdf9d6b66ee85934f6f8ad1d18218b51a87ea06cefd3b84554a3e4", - "ppc64le-unknown-linux-gnu": "76b48eb26ef274045772186e63431419294c41baf6d5a372b722d4c9e711082e", - "riscv64-unknown-linux-gnu": "76e1ec72717d17493976fc176ec661f02412666d4f19e50908d8e4303c0511d5", - "s390x-unknown-linux-gnu": "2edf241199d11a3ef79a312737c1bcdb86908352c585ca14b667539080630e85", - "x86_64-apple-darwin": "95a2d794b8981723095190fa94b574ceb4272bb49d83b9e418bb90341e304d09", - "x86_64-pc-windows-msvc": "0d828683d30185ab9f1110ad2194ef384cef0533b8e0da7e03ce837548841788", - "x86_64-unknown-linux-gnu": "303047011b2c9f58504a930fc974d84547477cf69a3f2962f25552e2395c13af", - "x86_64-unknown-linux-musl": "84eb198d318f8b1b8bf59eef5d30d742e13afd97c213fa229578f8fdab0c406f", - }, - "strip_prefix": "python", - }, - "3.11.1": { - "url": "20230116/cpython-{python_version}+20230116-{platform}-{build}.tar.gz", - "sha256": { - "aarch64-apple-darwin": "4918cdf1cab742a90f85318f88b8122aeaa2d04705803c7b6e78e81a3dd40f80", - "aarch64-unknown-linux-gnu": "debf15783bdcb5530504f533d33fda75a7b905cec5361ae8f33da5ba6599f8b4", - "x86_64-apple-darwin": "20a4203d069dc9b710f70b09e7da2ce6f473d6b1110f9535fb6f4c469ed54733", - "x86_64-pc-windows-msvc": "edc08979cb0666a597466176511529c049a6f0bba8adf70df441708f766de5bf", - "x86_64-unknown-linux-gnu": "02a551fefab3750effd0e156c25446547c238688a32fabde2995c941c03a6423", - }, - "strip_prefix": "python", - }, - "3.11.3": { - "url": "20230507/cpython-{python_version}+20230507-{platform}-{build}.tar.gz", - "sha256": { - "aarch64-apple-darwin": "09e412506a8d63edbb6901742b54da9aa7faf120b8dbdce56c57b303fc892c86", - "aarch64-unknown-linux-gnu": "8190accbbbbcf7620f1ff6d668e4dd090c639665d11188ce864b62554d40e5ab", - "ppc64le-unknown-linux-gnu": "767d24f3570b35fedb945f5ac66224c8983f2d556ab83c5cfaa5f3666e9c212c", - "x86_64-apple-darwin": "f710b8d60621308149c100d5175fec39274ed0b9c99645484fd93d1716ef4310", - "x86_64-pc-windows-msvc": "24741066da6f35a7ff67bee65ce82eae870d84e1181843e64a7076d1571e95af", - "x86_64-unknown-linux-gnu": "da50b87d1ec42b3cb577dfd22a3655e43a53150f4f98a4bfb40757c9d7839ab5", - }, - "strip_prefix": "python", - }, - "3.11.4": { - "url": "20230726/cpython-{python_version}+20230726-{platform}-{build}.tar.gz", - "sha256": { - "aarch64-apple-darwin": "cb6d2948384a857321f2aa40fa67744cd9676a330f08b6dad7070bda0b6120a4", - "aarch64-unknown-linux-gnu": "2e84fc53f4e90e11963281c5c871f593abcb24fc796a50337fa516be99af02fb", - "ppc64le-unknown-linux-gnu": "df7b92ed9cec96b3bb658fb586be947722ecd8e420fb23cee13d2e90abcfcf25", - "s390x-unknown-linux-gnu": "e477f0749161f9aa7887964f089d9460a539f6b4a8fdab5166f898210e1a87a4", - "x86_64-apple-darwin": "47e1557d93a42585972772e82661047ca5f608293158acb2778dccf120eabb00", - "x86_64-pc-windows-msvc": "878614c03ea38538ae2f758e36c85d2c0eb1eaaca86cd400ff8c76693ee0b3e1", - "x86_64-unknown-linux-gnu": "e26247302bc8e9083a43ce9e8dd94905b40d464745b1603041f7bc9a93c65d05", - }, - "strip_prefix": "python", - }, - "3.11.5": { - "url": "20230826/cpython-{python_version}+20230826-{platform}-{build}.tar.gz", - "sha256": { - "aarch64-apple-darwin": "dab64b3580118ad2073babd7c29fd2053b616479df5c107d31fe2af1f45e948b", - "aarch64-unknown-linux-gnu": "bb5c5d1ea0f199fe2d3f0996fff4b48ca6ddc415a3dbd98f50bff7fce48aac80", - "ppc64le-unknown-linux-gnu": "14121b53e9c8c6d0741f911ae00102a35adbcf5c3cdf732687ef7617b7d7304d", - "s390x-unknown-linux-gnu": "fe459da39874443579d6fe88c68777c6d3e331038e1fb92a0451879fb6beb16d", - "x86_64-apple-darwin": "4a4efa7378c72f1dd8ebcce1afb99b24c01b07023aa6b8fea50eaedb50bf2bfc", - "x86_64-pc-windows-msvc": "00f002263efc8aea896bcfaaf906b1f4dab3e5cd3db53e2b69ab9a10ba220b97", - "x86_64-unknown-linux-gnu": "fbed6f7694b2faae5d7c401a856219c945397f772eea5ca50c6eb825cbc9d1e1", - }, - "strip_prefix": "python", - }, - "3.11.6": { - "url": "20231002/cpython-{python_version}+20231002-{platform}-{build}.tar.gz", - "sha256": { - "aarch64-apple-darwin": "916c35125b5d8323a21526d7a9154ca626453f63d0878e95b9f613a95006c990", - "aarch64-unknown-linux-gnu": "3e26a672df17708c4dc928475a5974c3fb3a34a9b45c65fb4bd1e50504cc84ec", - "ppc64le-unknown-linux-gnu": "7937035f690a624dba4d014ffd20c342e843dd46f89b0b0a1e5726b85deb8eaf", - "s390x-unknown-linux-gnu": "f9f19823dba3209cedc4647b00f46ed0177242917db20fb7fb539970e384531c", - "x86_64-apple-darwin": "178cb1716c2abc25cb56ae915096c1a083e60abeba57af001996e8bc6ce1a371", - "x86_64-pc-windows-msvc": "3933545e6d41462dd6a47e44133ea40995bc6efeed8c2e4cbdf1a699303e95ea", - "x86_64-unknown-linux-gnu": "ee37a7eae6e80148c7e3abc56e48a397c1664f044920463ad0df0fc706eacea8", - }, - "strip_prefix": "python", - }, - "3.11.7": { - "url": "20240107/cpython-{python_version}+20240107-{platform}-{build}.tar.gz", - "sha256": { - "aarch64-apple-darwin": "b042c966920cf8465385ca3522986b12d745151a72c060991088977ca36d3883", - "aarch64-unknown-linux-gnu": "b102eaf865eb715aa98a8a2ef19037b6cc3ae7dfd4a632802650f29de635aa13", - "ppc64le-unknown-linux-gnu": "b44e1b74afe75c7b19143413632c4386708ae229117f8f950c2094e9681d34c7", - "s390x-unknown-linux-gnu": "49520e3ff494708020f306e30b0964f079170be83e956be4504f850557378a22", - "x86_64-apple-darwin": "a0e615eef1fafdc742da0008425a9030b7ea68a4ae4e73ac557ef27b112836d4", - "x86_64-pc-windows-msvc": "67077e6fa918e4f4fd60ba169820b00be7c390c497bf9bc9cab2c255ea8e6f3e", - "x86_64-unknown-linux-gnu": "4a51ce60007a6facf64e5495f4cf322e311ba9f39a8cd3f3e4c026eae488e140", - }, - "strip_prefix": "python", - }, - "3.11.8": { - "url": "20240224/cpython-{python_version}+20240224-{platform}-{build}.tar.gz", - "sha256": { - "aarch64-apple-darwin": "389a51139f5abe071a0d70091ca5df3e7a3dfcfcbe3e0ba6ad85fb4c5638421e", - "aarch64-unknown-linux-gnu": "389b9005fb78dd5a6f68df5ea45ab7b30d9a4b3222af96999e94fd20d4ad0c6a", - "ppc64le-unknown-linux-gnu": "eb2b31f8e50309aae493c6a359c32b723a676f07c641f5e8fe4b6aa4dbb50946", - "s390x-unknown-linux-gnu": "844f64f4c16e24965778281da61d1e0e6cd1358a581df1662da814b1eed096b9", - "x86_64-apple-darwin": "097f467b0c36706bfec13f199a2eaf924e668f70c6e2bd1f1366806962f7e86e", - "x86_64-pc-windows-msvc": "b618f1f047349770ee1ef11d1b05899840abd53884b820fd25c7dfe2ec1664d4", - "x86_64-unknown-linux-gnu": "94e13d0e5ad417035b80580f3e893a72e094b0900d5d64e7e34ab08e95439987", - }, - "strip_prefix": "python", - }, - "3.11.9": { - "url": "20240726/cpython-{python_version}+20240726-{platform}-{build}.tar.gz", - "sha256": { - "aarch64-apple-darwin": "cbdac9462bab9671c8e84650e425d3f43b775752a930a2ef954a0d457d5c00c3", - "aarch64-unknown-linux-gnu": "4d17cf988abe24449d649aad3ef974091ab76807904d41839907061925b4c9e3", - "ppc64le-unknown-linux-gnu": "fc4f3c9ef9bfac2ed0282126ff376e544697ad04a5408d6429d46899d7d3bf21", - "s390x-unknown-linux-gnu": "e69b66e53e926460df044f44846eef3fea642f630e829719e1a4112fc370dc56", - "x86_64-apple-darwin": "dc3174666a30f4c38d04e79a80c3159b4b3aa69597c4676701c8386696811611", - "x86_64-pc-windows-msvc": "f694be48bdfec1dace6d69a19906b6083f4dd7c7c61f1138ba520e433e5598f8", - "x86_64-unknown-linux-gnu": "f6e955dc9ddfcad74e77abe6f439dac48ebca14b101ed7c85a5bf3206ed2c53d", - }, - "strip_prefix": "python", - }, - "3.11.10": { - "url": "20241016/cpython-{python_version}+20241016-{platform}-{build}.tar.gz", - "sha256": { - "aarch64-apple-darwin": "5a69382da99c4620690643517ca1f1f53772331b347e75f536088c42a4cf6620", - "aarch64-unknown-linux-gnu": "803e49259280af0f5466d32829cd9d65a302b0226e424b3f0b261f9daf6aee8f", - "ppc64le-unknown-linux-gnu": "92b666d103902001322f42badbd68da92adc5cebb826af9c1c906c33166e2f34", - "s390x-unknown-linux-gnu": "6d584317651c1ad4a857cb32d1999707e8bb3046fcb2f156d80381814fa19fde", - "x86_64-apple-darwin": "1e23ffe5bc473e1323ab8f51464da62d77399afb423babf67f8e13c82b69c674", - "x86_64-pc-windows-msvc": "647b66ff4552e70aec3bf634dd470891b4a2b291e8e8715b3bdb162f577d4c55", - "x86_64-unknown-linux-gnu": "8b50a442b04724a24c1eebb65a36a0c0e833d35374dbdf9c9470d8a97b164cd9", - "x86_64-unknown-linux-musl": "d36fc77a8dd76155a7530f6235999a693b9e7c48aa11afeb5610a091cae5aa6f", - }, - "strip_prefix": "python", - }, - "3.11.13": { - "url": "20250808/cpython-{python_version}+20250808-{platform}-{build}.tar.gz", - "sha256": { - "aarch64-apple-darwin": "d089bfd2c7b98a0942750a195e70d3172beda76d7747097b8afd87028b6e59b6", - "aarch64-unknown-linux-gnu": "bc57105f8a16acd57b71d926143c7f6ecf61729b40c8b4656f1b98bebd47c710", - "ppc64le-unknown-linux-gnu": "16a0165b0744940702b8fff80b8bf973ac914f78cb6fca28d389583f675e84de", - "riscv64-unknown-linux-gnu": "d8e62306be8f41c46bcd62ca68f91a1467f47adff632a35ff413dc1043ed56e8", - "s390x-unknown-linux-gnu": "4e302a4514a73baefdd9b327062bdafeb4115a799deec91c185f6ab45a857241", - "x86_64-apple-darwin": "d946d618f8bba8308b67e460a30612a71e2ccc309f85f6628aaae24e2b816981", - "x86_64-pc-windows-msvc": "ed963aee33d29ad8abfbb5fe63e42f57a2638a4a11a88e11d8bb66e61f20a6e5", - "aarch64-pc-windows-msvc": "a632857c966237e7fd38b44c47c350f6e30d8ec54dcad6c832865ad670f0f22f", - "x86_64-unknown-linux-gnu": "3ad988c702cbb017fef1208d47dea4138a2e85fd0f7f01ec5e1e335e597131b9", - "x86_64-unknown-linux-musl": "3a5810f0696f844289aa06d5c3a1efeab66eee999c25196b7d1954192a2c2100", - }, - "strip_prefix": "python", - }, - "3.11.14": { - "url": "20251031/cpython-{python_version}+20251031-{platform}-{build}.tar.gz", - "sha256": { - "aarch64-apple-darwin": "6de5572b33c65af1c9b7caf00ec593fb04cffb7e14fa393a98261bb9bc464713", - "aarch64-unknown-linux-gnu": "510edb027527413c4249256194cb8ad2590b52dd93f7123b4cb341aff5d05894", - "ppc64le-unknown-linux-gnu": "4e0bc6a818e0c6a9d7d3ebe1a95591fd84440520577aa837facc96a4b7a80e35", - "riscv64-unknown-linux-gnu": "16519e69297144f81b2421333bc9e0b6466cf3c84749b216b695cfb4c9deb32f", - "s390x-unknown-linux-gnu": "5f9c1b203cdf34c8bff1aef69b63bbf11309bd16ca6e429d8c3651eaa2b3d080", - "x86_64-apple-darwin": "4891cbf34e8652b7bd1054b9502395e4b7e048e2e517c040fbf6c8297cb954d6", - "x86_64-pc-windows-msvc": "5223b83ed9e2aa5e9e17d2ebcf767956e998876339b9cde1980a47e9d4655fb6", - "aarch64-pc-windows-msvc": "38d0d1466561e15965e8d2c20f5e5be649598f55c761ecab553d087fbd217337", - "x86_64-unknown-linux-gnu": "60f0bd473d861cc45d3401d9914e47ccb9fa037f88a91879ed517a62042b8477", - "x86_64-unknown-linux-musl": "25e82d1e85b90a8ab724ee633a1811b1921797f5c25ee69c6595052371b91a87", - }, - "strip_prefix": "python", - }, - "3.11.15": { - "url": "20260414/cpython-{python_version}+20260414-{platform}-{build}.{ext}", - "sha256": { - "aarch64-apple-darwin": "a57ffd435652092d16b30e783f9826c55e9c64b0f0a72cbae0a9f39e663137fb", - "aarch64-unknown-linux-gnu": "77836944ae15b74e0b25bdc68a4703a340f2ccb684effc0f45fbd7910e1a1f39", - "ppc64le-unknown-linux-gnu": "30a2107f000dbe304820627cbe2cc257027c20f3241d96e6c7df796b69ac2062", - "riscv64-unknown-linux-gnu": "373b98fbf2d04099139a2f6be57593714382ed790be7e7419e358830c23ddd0f", - "s390x-unknown-linux-gnu": "7838efa839158c80568de35ac78d438f564f4c32272a2fe7d9e14a9b351d1a62", - "x86_64-apple-darwin": "317055d80e553764feeaef432d833dd8385c14b83465a8b3fa7c2b7819cba681", - "x86_64-pc-windows-msvc": "8e69ecf1d9fc194e029aafa608d483bf24ccaa8f56d456d7009f20462d62ad23", - "aarch64-pc-windows-msvc": "a882abe4876985c9dc3d433420548506fb0cc9bb9d9fe336a2d3aaf28922aa45", - "x86_64-unknown-linux-gnu": "8b14030dd3af9ea7f7c51b4c90feb04afd8a8f45435727e67b875270bd08f3bc", - "x86_64-unknown-linux-musl": "ca92d3a68a39fa330498b09714733f347bead7313ba9d9b7fbed837aa4ba7796", - }, - "strip_prefix": "python", - }, - "3.12.0": { - "url": "20231002/cpython-{python_version}+20231002-{platform}-{build}.tar.gz", - "sha256": { - "aarch64-apple-darwin": "4734a2be2becb813830112c780c9879ac3aff111a0b0cd590e65ec7465774d02", - "aarch64-unknown-linux-gnu": "bccfe67cf5465a3dfb0336f053966e2613a9bc85a6588c2fcf1366ef930c4f88", - "ppc64le-unknown-linux-gnu": "b5dae075467ace32c594c7877fe6ebe0837681f814601d5d90ba4c0dfd87a1f2", - "s390x-unknown-linux-gnu": "5681621349dd85d9726d1b67c84a9686ce78f72e73a6f9e4cc4119911655759e", - "x86_64-apple-darwin": "5a9e88c8aa52b609d556777b52ebde464ae4b4f77e4aac4eb693af57395c9abf", - "x86_64-pc-windows-msvc": "facfaa1fbc8653f95057f3c4a0f8aa833dab0e0b316e24ee8686bc761d4b4f8d", - "x86_64-unknown-linux-gnu": "e51a5293f214053ddb4645b2c9f84542e2ef86870b8655704367bd4b29d39fe9", - }, - "strip_prefix": "python", - }, - "3.12.1": { - "url": "20240107/cpython-{python_version}+20240107-{platform}-{build}.tar.gz", - "sha256": { - "aarch64-apple-darwin": "f93f8375ca6ac0a35d58ff007043cbd3a88d9609113f1cb59cf7c8d215f064af", - "aarch64-unknown-linux-gnu": "236533ef20e665007a111c2f36efb59c87ae195ad7dca223b6dc03fb07064f0b", - "ppc64le-unknown-linux-gnu": "78051f0d1411ee62bc2af5edfccf6e8400ac4ef82887a2affc19a7ace6a05267", - "s390x-unknown-linux-gnu": "60631211c701f8d2c56e5dd7b154e68868128a019b9db1d53a264f56c0d4aee2", - "x86_64-apple-darwin": "eca96158c1568dedd9a0b3425375637a83764d1fa74446438293089a8bfac1f8", - "x86_64-pc-windows-msvc": "fd5a9e0f41959d0341246d3643f2b8794f638adc0cec8dd5e1b6465198eae08a", - "x86_64-unknown-linux-gnu": "74e330b8212ca22fd4d9a2003b9eec14892155566738febc8e5e572f267b9472", - }, - "strip_prefix": "python", - }, - "3.12.2": { - "url": "20240224/cpython-{python_version}+20240224-{platform}-{build}.tar.gz", - "sha256": { - "aarch64-apple-darwin": "01c064c00013b0175c7858b159989819ead53f4746d40580b5b0b35b6e80fba6", - "aarch64-unknown-linux-gnu": "e52550379e7c4ac27a87de832d172658bc04150e4e27d4e858e6d8cbb96fd709", - "ppc64le-unknown-linux-gnu": "74bc02c4bbbd26245c37b29b9e12d0a9c1b7ab93477fed8b651c988b6a9a6251", - "s390x-unknown-linux-gnu": "ecd6b0285e5eef94deb784b588b4b425a15a43ae671bf206556659dc141a9825", - "x86_64-apple-darwin": "a53a6670a202c96fec0b8c55ccc780ea3af5307eb89268d5b41a9775b109c094", - "x86_64-pc-windows-msvc": "1e5655a6ccb1a64a78460e4e3ee21036c70246800f176a6c91043a3fe3654a3b", - "x86_64-unknown-linux-gnu": "57a37b57f8243caa4cdac016176189573ad7620f0b6da5941c5e40660f9468ab", - }, - "strip_prefix": "python", - }, - "3.12.3": { - "url": "20240415/cpython-{python_version}+20240415-{platform}-{build}.tar.gz", - "sha256": { - "aarch64-apple-darwin": "ccc40e5af329ef2af81350db2a88bbd6c17b56676e82d62048c15d548401519e", - "aarch64-unknown-linux-gnu": "ec8126de97945e629cca9aedc80a29c4ae2992c9d69f2655e27ae73906ba187d", - "ppc64le-unknown-linux-gnu": "c5dcf08b8077e617d949bda23027c49712f583120b3ed744f9b143da1d580572", - "s390x-unknown-linux-gnu": "872fc321363b8cdd826fd2cb1adfd1ceb813bc1281f9d410c1c2c4e177e8df86", - "x86_64-apple-darwin": "c37a22fca8f57d4471e3708de6d13097668c5f160067f264bb2b18f524c890c8", - "x86_64-pc-windows-msvc": "f7cfa4ad072feb4578c8afca5ba9a54ad591d665a441dd0d63aa366edbe19279", - "x86_64-unknown-linux-gnu": "a73ba777b5d55ca89edef709e6b8521e3f3d4289581f174c8699adfb608d09d6", - }, - "strip_prefix": "python", - }, - "3.12.4": { - "url": "20240726/cpython-{python_version}+20240726-{platform}-{build}.tar.gz", - "sha256": { - "aarch64-apple-darwin": "1801025e825c04b3907e4ef6220a13607bc0397628c9485897073110ef7fde15", - "aarch64-unknown-linux-gnu": "a098b18b7e9fea0c66867b76c0124fce9465765017572b2e7b522154c87c78d7", - "ppc64le-unknown-linux-gnu": "04011c4c5b7fe34b0b895edf4ad8748e410686c1d69aaee11d6688d481023bcb", - "s390x-unknown-linux-gnu": "8f8f3e29cf0c2facdbcfee70660939fda7667ac24fee8656d3388fc72f3acc7c", - "x86_64-apple-darwin": "4c325838c1b0ed13698506fcd515be25c73dcbe195f8522cf98f9148a97601ed", - "x86_64-pc-windows-msvc": "74309b0f322716409883d38c621743ea7fa0376eb00927b8ee1e1671d3aff450", - "x86_64-unknown-linux-gnu": "e133dd6fc6a2d0033e2658637cc22e9c95f9d7073b80115037ee1f16417a54ac", - }, - "strip_prefix": "python", - }, - "3.12.7": { - "url": "20241016/cpython-{python_version}+20241016-{platform}-{build}.tar.gz", - "sha256": { - "aarch64-apple-darwin": "4c18852bf9c1a11b56f21bcf0df1946f7e98ee43e9e4c0c5374b2b3765cf9508", - "aarch64-unknown-linux-gnu": "bba3c6be6153f715f2941da34f3a6a69c2d0035c9c5396bc5bb68c6d2bd1065a", - "ppc64le-unknown-linux-gnu": "0a1d1d92e33a969bd2f40a80af53c97b6c0cc1060d384ceff50ff801593bf9d6", - "s390x-unknown-linux-gnu": "935676a0c960b552f95e9ac2e1e385de5de4b34038ff65ffdc688838f1189c17", - "x86_64-apple-darwin": "60c5271e7edc3c2ab47440b7abf4ed50fbc693880b474f74f05768f5b657045a", - "x86_64-pc-windows-msvc": "f05531bff16fa77b53be0776587b97b466070e768e6d5920894de988bdcd547a", - "x86_64-unknown-linux-gnu": "43576f7db1033dd57b900307f09c2e86f371152ac8a2607133afa51cbfc36064", - "x86_64-unknown-linux-musl": "5ed4a4078db3cbac563af66403aaa156cd6e48831d90382a1820db2b120627b5", - }, - "strip_prefix": "python", - }, - "3.12.8": { - "url": "20241206/cpython-{python_version}+20241206-{platform}-{build}.tar.gz", - "sha256": { - "aarch64-apple-darwin": "e3c4aa607717b23903ca2650d5c3ee24f89b97543e2db2b0f463bddc7a9e92f3", - "aarch64-unknown-linux-gnu": "ce674b55442b732973afb2932c281bb1ded4ad7e22bcf9b07071165770758c7e", - "ppc64le-unknown-linux-gnu": "b7214790b273de9ed0532420054b72ba1393d62d2fc844ec55ade193771bd90c", - "s390x-unknown-linux-gnu": "73102f5dbd7d1e7e9c2f2c80aedf2893d99a7fa407f6674ec8b2f57ba07daee5", - "x86_64-apple-darwin": "3ba35c706577d755e8e52a4c161a042464577c0e695e2a605362fa469e26de10", - "x86_64-pc-windows-msvc": "767b4be3ddf6b99e5ade519789c1615c191d8cf99d5aff4685cc18b48931f1e6", - "x86_64-unknown-linux-gnu": "b9d6ee5ddac1198e72d53112698773fc8bb597de095592eb849ca794306699ba", - "x86_64-unknown-linux-musl": "6f305888703691dd04cfff85284d23ea0b0146ed7c4415e472f1fb72b3f32cdf", - }, - "strip_prefix": "python", - }, - "3.12.9": { - "url": "20250317/cpython-{python_version}+20250317-{platform}-{build}.tar.gz", - "sha256": { - "aarch64-apple-darwin": "7c7fd9809da0382a601a79287b5d62d61ce0b15f5a5ee836233727a516e85381", - "aarch64-unknown-linux-gnu": "00c6bf9acef21ac741fea24dc449d0149834d30e9113429e50a95cce4b00bb80", - "ppc64le-unknown-linux-gnu": "25d77599dfd5849f17391d92da0da99079e4e94f19a881f763f5cc62530ef7e1", - "riscv64-unknown-linux-gnu": "e97ab0fdf443b302c56a52b4fd08f513bf3be66aa47263f0f9df3c6e60e05f2e", - "s390x-unknown-linux-gnu": "7492d079ffa8425c8f6c58e43b237c37e3fb7b31e2e14635927bb4d3397ba21e", - "x86_64-apple-darwin": "1ee1b1bb9fbce5c145c4bec9a3c98d7a4fa22543e09a7c1d932bc8599283c2dc", - "x86_64-pc-windows-msvc": "d15361fd202dd74ae9c3eece1abdab7655f1eba90bf6255cad1d7c53d463ed4d", - "x86_64-unknown-linux-gnu": "ef382fb88cbb41a3b0801690bd716b8a1aec07a6c6471010bcc6bd14cd575226", - "x86_64-unknown-linux-musl": "94e3837da1adf9964aab2d6047b33f70167de3096d1f9a2d1fa9340b1bbf537d", - }, - "strip_prefix": "python", - }, - "3.12.11": { - "url": "20250808/cpython-{python_version}+20250808-{platform}-{build}.tar.gz", - "sha256": { - "aarch64-apple-darwin": "8792c4a84c364ab975feca0c27d3157a5435b7baab325a346ae56b223893b661", - "aarch64-unknown-linux-gnu": "4d7ba5314fab02130d6538f074961ffbf61310cade9180e59026074f9a8939cb", - "aarch64-pc-windows-msvc": "00bf7d7e8bcf5d1e9c4dfca0247d8e035147777cd57ee9d4c64dedca86b0a464", - "ppc64le-unknown-linux-gnu": "2c862eb40a81549d9c11e6bf5a7f07c3406310b14e6a4d16dcdf1c4763ef7090", - "riscv64-unknown-linux-gnu": "0bb729b95fabd49c7b495f7c44a9086e3970ea57daf66365741574bd36a17e81", - "s390x-unknown-linux-gnu": "99e465882d217d24ac90e99fac8f32e6a644d0340ac05ee510fb5cdf53f0cfb8", - "x86_64-apple-darwin": "e0c932709dafb05f00e528a7560ef8ee559ac82b75faca60dd1245bca1c1553f", - "x86_64-pc-windows-msvc": "81214ef71964a40ec269a79067ca490d45298c350583bc3af0e5781451a05c3c", - "x86_64-unknown-linux-gnu": "63d78840bf209af8da8f24e335d910f88387b892ca9187be571d481c071751bb", - "x86_64-unknown-linux-musl": "d633d070780590aa03ac5575cd9d7b9e17682d80f14b400313c009c387cf706b", - }, - "strip_prefix": "python", - }, - "3.12.12": { - "url": "20251031/cpython-{python_version}+20251031-{platform}-{build}.tar.gz", - "sha256": { - "aarch64-apple-darwin": "5e110cb821d2eb8246065d3b46faa655180c976c4e17250f7883c634a629bc63", - "aarch64-unknown-linux-gnu": "81b644d166e0bfb918615af8a2363f8fcf26eccdcc60a5334b6a62c088470bac", - "aarch64-pc-windows-msvc": "b190fed7c2b0f6e1010f554a0d1fd191c0754c4c0718e69d9d795ae559613780", - "ppc64le-unknown-linux-gnu": "024f5e5678c9768d45cc24d37a8e9d265aae86c4a4602352dee3d7deba367052", - "riscv64-unknown-linux-gnu": "b13c57fc372c131e667a99b9680f41c0b4da571cf99ed412103c2fe9ad5ed1fb", - "s390x-unknown-linux-gnu": "2bf05bdd56cdf5ea4fd9f2faf151ea4211be96a0d1f4230b85f5dcae620d6400", - "x86_64-apple-darwin": "687052a046d33be49dc95dd671816709067cf6176ed36c93ea61b1fe0b883b0f", - "x86_64-pc-windows-msvc": "cff398b3f520c442a1b085dd347126c10c1b03f01ccc0decd8c897a687e893f1", - "x86_64-unknown-linux-gnu": "80c3882f14e15cef8260ef5257d198e8f4371ca265887431d939e0d561de3253", - "x86_64-unknown-linux-musl": "0a461330b9b89f2ea3088dde10d7a3f96aa65897b7c5ce2404fa3b5c4b8daa14", - }, - "strip_prefix": "python", - }, - "3.12.13": { - "url": "20260414/cpython-{python_version}+20260414-{platform}-{build}.{ext}", - "sha256": { - "aarch64-apple-darwin": "8966b2bcd9fa03ba22c080ad15a86bc12e41a00122b16f4b3740e302261124d9", - "aarch64-unknown-linux-gnu": "355d981eafb9b2870af79ddc106ced7266b6f6d2101d8fbcb05620fa386642b9", - "ppc64le-unknown-linux-gnu": "4aef4cffe73c4a65ea486f14d684a9ad3f831a354174d163bb531b5baa70fc49", - "riscv64-unknown-linux-gnu": "c2629d69324155132343913f064be93509bd162531e08a292e50c3973ec8b5db", - "s390x-unknown-linux-gnu": "e5baafd64180f45165d2751b25d1bcc89254eefc7926f3ab341fc61b541d7606", - "x86_64-apple-darwin": "801b03fbe004181d55a02ebd8b4e04d74973e70d716062aebe3b3cf32e9be297", - "x86_64-pc-windows-msvc": "c5a9e011e284c49c48106ca177342f3e3f64e95b4c6652d4a382cc7c9bb1cc46", - "aarch64-pc-windows-msvc": "f55326c894fde76fc0faffe95d2bce60be533c88a8c44c1b88bbbc17bf6a5cd5", - "x86_64-unknown-linux-gnu": "cdcf8724d46e4857f8db5ee9f4252dc2f5da34f7940294ec6b312389dd3f41e0", - "x86_64-unknown-linux-musl": "d10e971238c130fdf25e577c6538a3effa5589d5fcf53665e3c711edd6a6ff2f", - }, - "strip_prefix": "python", - }, - "3.13.0": { - "url": "20241016/cpython-{python_version}+20241016-{platform}-{build}.{ext}", - "sha256": { - "aarch64-apple-darwin": "31397953849d275aa2506580f3fa1cb5a85b6a3d392e495f8030e8b6412f5556", - "aarch64-unknown-linux-gnu": "e8378c0162b2e0e4cc1f62b29443a3305d116d09583304dbb0149fecaff6347b", - "ppc64le-unknown-linux-gnu": "fc4b7f27c4e84c78f3c8e6c7f8e4023e4638d11f1b36b6b5ce457b1926cebb53", - "s390x-unknown-linux-gnu": "66b19e6a07717f6cfcd3a8ca953f0a2eaa232291142f3d26a8d17c979ec0f467", - "x86_64-apple-darwin": "cff1b7e7cd26f2d47acac1ad6590e27d29829776f77e8afa067e9419f2f6ce77", - "x86_64-pc-windows-msvc": "b25926e8ce4164cf103bacc4f4d154894ea53e07dd3fdd5ebb16fb1a82a7b1a0", - "x86_64-unknown-linux-gnu": "2c8cb15c6a2caadaa98af51df6fe78a8155b8471cb3dd7b9836038e0d3657fb4", - "x86_64-unknown-linux-musl": "2f61ee3b628a56aceea63b46c7afe2df3e22a61da706606b0c8efda57f953cf4", - "aarch64-apple-darwin-freethreaded": "efc2e71c0e05bc5bedb7a846e05f28dd26491b1744ded35ed82f8b49ccfa684b", - "aarch64-unknown-linux-gnu-freethreaded": "59b50df9826475d24bb7eff781fa3949112b5e9c92adb29e96a09cdf1216d5bd", - "ppc64le-unknown-linux-gnu-freethreaded": "1217efa5f4ce67fcc9f7eb64165b1bd0912b2a21bc25c1a7e2cb174a21a5df7e", - "s390x-unknown-linux-gnu-freethreaded": "6c3e1e4f19d2b018b65a7e3ef4cd4225c5b9adfbc490218628466e636d5c4b8c", - "x86_64-apple-darwin-freethreaded": "2e07dfea62fe2215738551a179c87dbed1cc79d1b3654f4d7559889a6d5ce4eb", - "x86_64-pc-windows-msvc-freethreaded": "bfd89f9acf866463bc4baf01733da5e767d13f5d0112175a4f57ba91f1541310", - "x86_64-unknown-linux-gnu-freethreaded": "a73adeda301ad843cce05f31a2d3e76222b656984535a7b87696a24a098b216c", - }, - "strip_prefix": { - "aarch64-apple-darwin": "python", - "aarch64-unknown-linux-gnu": "python", - "ppc64le-unknown-linux-gnu": "python", - "s390x-unknown-linux-gnu": "python", - "x86_64-apple-darwin": "python", - "x86_64-pc-windows-msvc": "python", - "x86_64-unknown-linux-gnu": "python", - "x86_64-unknown-linux-musl": "python", - "aarch64-apple-darwin-freethreaded": "python/install", - "aarch64-unknown-linux-gnu-freethreaded": "python/install", - "ppc64le-unknown-linux-gnu-freethreaded": "python/install", - "s390x-unknown-linux-gnu-freethreaded": "python/install", - "x86_64-apple-darwin-freethreaded": "python/install", - "x86_64-pc-windows-msvc-freethreaded": "python/install", - "x86_64-unknown-linux-gnu-freethreaded": "python/install", - }, - }, - "3.13.1": { - "url": "20241205/cpython-{python_version}+20241205-{platform}-{build}.{ext}", - "sha256": { - "aarch64-apple-darwin": "88b88b609129c12f4b3841845aca13230f61e97ba97bd0fb28ee64b0e442a34f", - "aarch64-unknown-linux-gnu": "fdfa86c2746d2ae700042c461846e6c37f70c249925b58de8cd02eb8d1423d4e", - "ppc64le-unknown-linux-gnu": "27b20b3237c55430ca1304e687d021f88373f906249f9cd272c5ff2803d5e5c3", - "s390x-unknown-linux-gnu": "7d0187e20cb5e36c689eec27e4d3de56d8b7f1c50dc5523550fc47377801521f", - "x86_64-apple-darwin": "47eef6efb8664e2d1d23a7cdaf56262d784f8ace48f3bfca1b183e95a49888d6", - "x86_64-pc-windows-msvc": "f51f0493a5f979ff0b8d8c598a8d74f2a4d86a190c2729c85e0af65c36a9cbbe", - "x86_64-unknown-linux-gnu": "242b2727df6c1e00de6a9f0f0dcb4562e168d27f428c785b0eb41a6aeb34d69a", - "x86_64-unknown-linux-musl": "76b30c6373b9c0aa2ba610e07da02f384aa210ac79643da38c66d3e6171c6ef5", - "aarch64-apple-darwin-freethreaded": "08f05618bdcf8064a7960b25d9ba92155447c9b08e0cf2f46a981e4c6a1bb5a5", - "aarch64-unknown-linux-gnu-freethreaded": "9f2fcb809f9ba6c7c014a8803073a88786701a98971135bce684355062e4bb35", - "ppc64le-unknown-linux-gnu-freethreaded": "15ceea78dff78ca8ccaac8d9c54b808af30daaa126f1f561e920a6896e098634", - "s390x-unknown-linux-gnu-freethreaded": "ed3c6118d1d12603309c930e93421ac7a30a69045ffd43006f63ecf71d72c317", - "x86_64-apple-darwin-freethreaded": "dc780fecd215d2cc9e573abf1e13a175fcfa8f6efd100ef888494a248a16cda8", - "x86_64-pc-windows-msvc-freethreaded": "7537b2ab361c0eabc0eabfca9ffd9862d7f5f6576eda13b97e98aceb5eea4fd3", - "x86_64-unknown-linux-gnu-freethreaded": "9ec1b81213f849d91f5ebe6a16196e85cd6ff7c05ca823ce0ab7ba5b0e9fee84", - }, - "strip_prefix": { - "aarch64-apple-darwin": "python", - "aarch64-unknown-linux-gnu": "python", - "ppc64le-unknown-linux-gnu": "python", - "s390x-unknown-linux-gnu": "python", - "x86_64-apple-darwin": "python", - "x86_64-pc-windows-msvc": "python", - "x86_64-unknown-linux-gnu": "python", - "x86_64-unknown-linux-musl": "python", - "aarch64-apple-darwin-freethreaded": "python/install", - "aarch64-unknown-linux-gnu-freethreaded": "python/install", - "ppc64le-unknown-linux-gnu-freethreaded": "python/install", - "s390x-unknown-linux-gnu-freethreaded": "python/install", - "x86_64-apple-darwin-freethreaded": "python/install", - "x86_64-pc-windows-msvc-freethreaded": "python/install", - "x86_64-unknown-linux-gnu-freethreaded": "python/install", - }, - }, - "3.13.2": { - "url": "20250317/cpython-{python_version}+20250317-{platform}-{build}.{ext}", - "sha256": { - "aarch64-apple-darwin": "faa44274a331eb39786362818b21b3a4e74514e8805000b20b0e55c590cecb94", - "aarch64-unknown-linux-gnu": "9c67260446fee6ea706dad577a0b32936c63f449c25d66e4383d5846b2ab2e36", - "ppc64le-unknown-linux-gnu": "345b53d2f86c9dbd7f1320657cb227ff9a42ef63ff21f129abbbc8c82a375147", - "riscv64-unknown-linux-gnu": "172d22b2330737f3a028ea538ffe497c39a066a8d3200b22dd4d177a3332ad85", - "s390x-unknown-linux-gnu": "ec3b16ea8a97e3138acec72bc5ff35949950c62c8994a8ec8e213fd93f0e806b", - "x86_64-apple-darwin": "ee4526e84b5ce5b11141c50060b385320f2773616249a741f90c96d460ce8e8f", - "x86_64-pc-windows-msvc": "84d7b52f3558c8e35c670a4fa14080c75e3ec584adfae49fec8b51008b75b21e", - "x86_64-unknown-linux-gnu": "db011f0cd29cab2291584958f4e2eb001b0e6051848d89b38a2dc23c5c54e512", - "x86_64-unknown-linux-musl": "00bb2d629f7eacbb5c6b44dc04af26d1f1da64cee3425b0d8eb5135a93830296", - "aarch64-apple-darwin-freethreaded": "c98c9c977e6fa05c3813bd49f3553904d89d60fed27e2e36468da7afa1d6d5e2", - "aarch64-unknown-linux-gnu-freethreaded": "b8635e59e3143fd17f19a3dfe8ccc246ee6587c87da359bd1bcab35eefbb5f19", - "ppc64le-unknown-linux-gnu-freethreaded": "6ae8fa44cb2edf4ab49cff1820b53c40c10349c0f39e11b8cd76ce7f3e7e1def", - "riscv64-unknown-linux-gnu-freethreaded": "2af1b8850c52801fb6189e7a17a51e0c93d9e46ddefcca72247b76329c97d02a", - "s390x-unknown-linux-gnu-freethreaded": "c074144cc80c2af32c420b79a9df26e8db405212619990c1fbdd308bd75afe3f", - "x86_64-apple-darwin-freethreaded": "0d73e4348d8d4b5159058609d2303705190405b485dd09ad05d870d7e0f36e0f", - "x86_64-pc-windows-msvc-freethreaded": "c51b4845fda5421e044067c111192f645234081d704313f74ee77fa013a186ea", - "x86_64-unknown-linux-gnu-freethreaded": "1aea5062614c036904b55c1cc2fb4b500b7f6f7a4cacc263f4888889d355eef8", - }, - "strip_prefix": { - "aarch64-apple-darwin": "python", - "aarch64-unknown-linux-gnu": "python", - "ppc64le-unknown-linux-gnu": "python", - "s390x-unknown-linux-gnu": "python", - "riscv64-unknown-linux-gnu": "python", - "x86_64-apple-darwin": "python", - "x86_64-pc-windows-msvc": "python", - "x86_64-unknown-linux-gnu": "python", - "x86_64-unknown-linux-musl": "python", - "aarch64-apple-darwin-freethreaded": "python/install", - "aarch64-unknown-linux-gnu-freethreaded": "python/install", - "ppc64le-unknown-linux-gnu-freethreaded": "python/install", - "riscv64-unknown-linux-gnu-freethreaded": "python/install", - "s390x-unknown-linux-gnu-freethreaded": "python/install", - "x86_64-apple-darwin-freethreaded": "python/install", - "x86_64-pc-windows-msvc-freethreaded": "python/install", - "x86_64-unknown-linux-gnu-freethreaded": "python/install", - }, - }, - "3.13.4": { - "url": "20250610/cpython-{python_version}+20250610-{platform}-{build}.{ext}", - "sha256": { - "aarch64-apple-darwin": "c2ce6601b2668c7bd1f799986af5ddfbff36e88795741864aba6e578cb02ed7f", - "aarch64-unknown-linux-gnu": "3c2596ece08ffe17e11bc1f27aeb4ce1195d2490a83d695d36ef4933d5c5ca53", - "ppc64le-unknown-linux-gnu": "b3cc13ee177b8db1d3e9b2eac413484e3c6a356f97d91dc59de8d3fd8cf79d6b", - "riscv64-unknown-linux-gnu": "d1b989e57a9ce29f6c945eeffe0e9750c222fdd09e99d2f8d6b0d8532a523053", - "s390x-unknown-linux-gnu": "d1d19fb01961ac6476712fdd6c5031f74c83666f6f11aa066207e9a158f7e3d8", - "x86_64-apple-darwin": "79feb6ca68f3921d07af52d9db06cf134e6f36916941ea850ab0bc20f5ff638b", - "x86_64-pc-windows-msvc": "29ac3585cc2dcfd79e3fe380c272d00e9d34351fc456e149403c86d3fea34057", - "x86_64-unknown-linux-gnu": "44e5477333ebca298a7a0a316985c6c3533b8645f92a83f7f73c44033832bf32", - "x86_64-unknown-linux-musl": "a3afbfa94b9ff4d9fc426b47eb3c8446cada535075b8d51b7bdc9d9ab9911fc2", - "aarch64-apple-darwin-freethreaded": "278dccade56b4bbeecb9a613b77012cf5c1433a5e9b8ef99230d5e61f31d9e02", - "aarch64-unknown-linux-gnu-freethreaded": "b1c1bd6ab9ef95b464d92a6a911cef1a8d9f0b0f6a192f694ef18ed15d882edf", - "ppc64le-unknown-linux-gnu-freethreaded": "ed66ae213a62b286b9b7338b816ccd2815f5248b7a28a185dc8159fe004149ae", - "riscv64-unknown-linux-gnu-freethreaded": "913264545215236660e4178bc3e5b57a20a444a8deb5c11680c95afc960b4016", - "s390x-unknown-linux-gnu-freethreaded": "7556a38ab5e507c1ec22bc38f9859982bc956cab7f4de05a2faac114feb306db", - "x86_64-apple-darwin-freethreaded": "64ab7ac8c88002d9ba20a92f72945bfa350268e944a7922500af75d20330574d", - "x86_64-pc-windows-msvc-freethreaded": "9457504547edb2e0156bf76b53c7e4941c7f61c0eff9fd5f4d816d3df51c58e3", - "x86_64-unknown-linux-gnu-freethreaded": "864df6e6819e8f8e855ce30f34410fdc5867d0616e904daeb9a40e5806e970d7", - }, - "strip_prefix": { - "aarch64-apple-darwin": "python", - "aarch64-unknown-linux-gnu": "python", - "ppc64le-unknown-linux-gnu": "python", - "s390x-unknown-linux-gnu": "python", - "riscv64-unknown-linux-gnu": "python", - "x86_64-apple-darwin": "python", - "x86_64-pc-windows-msvc": "python", - "x86_64-unknown-linux-gnu": "python", - "x86_64-unknown-linux-musl": "python", - "aarch64-apple-darwin-freethreaded": "python/install", - "aarch64-unknown-linux-gnu-freethreaded": "python/install", - "ppc64le-unknown-linux-gnu-freethreaded": "python/install", - "riscv64-unknown-linux-gnu-freethreaded": "python/install", - "s390x-unknown-linux-gnu-freethreaded": "python/install", - "x86_64-apple-darwin-freethreaded": "python/install", - "x86_64-pc-windows-msvc-freethreaded": "python/install", - "x86_64-unknown-linux-gnu-freethreaded": "python/install", - }, - }, - "3.13.6": { - "url": "20250808/cpython-{python_version}+20250808-{platform}-{build}.{ext}", - "sha256": { - "aarch64-apple-darwin": "8a1efa6af4e80f08e2c97dda822a3d6c24d6c98e518242f802c6a43ae8401488", - "aarch64-unknown-linux-gnu": "11fa0591ae2211c08a42ae54944260e36ddf88a1d5604ea0c49e2477be4e5388", - "ppc64le-unknown-linux-gnu": "8dcf34ae1a685fe1893b52917ae04f23328edadc4acae28499d43850c2bdd26c", - "riscv64-unknown-linux-gnu": "f8ed75aa6cc2011a046be00b629c3c8295267f34280324feaff34c73e7afce39", - "s390x-unknown-linux-gnu": "7707ee5d19a78bc64ef8a66751ec7f97b64ea06714c7b1b52e8b321c2923ead8", - "x86_64-apple-darwin": "27badce7201321a8363219e438a6205165e5b4884012b1046532203df2ec9379", - "x86_64-pc-windows-msvc": "af5cc733c33b9aa9f1d74c81a59351e9b27215486d8b6cdbc06d97646a58c953", - "aarch64-pc-windows-msvc": "8e1617bd407ec1a874499daab26ae95080d1e0267ae616d34490137a28705827", - "aarch64-pc-windows-msvc-freethreaded": "552cfabcc3b103f4b1c4036d2592d5f0373c9554a2c4d2b6631b04ef7e592067", - "x86_64-unknown-linux-gnu": "f844e8c8b6847628b472f7e97d8893a4e93acd5382a902b465776063668c4d64", - "x86_64-unknown-linux-musl": "70076dea0ff65b3c05aae1a97b4a556bf613cc73db30309e59134f9d318f4f7b", - "aarch64-apple-darwin-freethreaded": "f2143304012e021a603bf1807bf3e4ce163832e43ab9a9829e53cb136497f207", - "aarch64-unknown-linux-gnu-freethreaded": "d84a7d64c284be387386b9f5da273f6d05486eb6bd8f9e86e2575cb59604cb22", - "ppc64le-unknown-linux-gnu-freethreaded": "e76fcaf1bf80a615520dbe7f85ca0bb557fad96d132d836b0ac721e7cc1e2a37", - "riscv64-unknown-linux-gnu-freethreaded": "24e08a39ba4fc77753e61541e52eed39cc871f4a92a80a3c5dd495056bd8eff9", - "s390x-unknown-linux-gnu-freethreaded": "1609b223fd38a4a7a4d20e7173d7d9390fe2258f7dd9a15dc9ef0fa49613735d", - "x86_64-apple-darwin-freethreaded": "4360a1278dd0a96b526d108c8fd23498a9d2028dd7791e510fd51ff5ea3f462a", - "x86_64-pc-windows-msvc-freethreaded": "4e727cdbe4057b16a170f887c0fa4227a825ac59bcda84ae946c77cc932af78c", - "x86_64-unknown-linux-gnu-freethreaded": "e48c13c59cc3c01b79f63c8bccec27d2db6e97f64213b8731e2077b6ed8ed52c", - }, - "strip_prefix": { - "aarch64-apple-darwin": "python", - "aarch64-unknown-linux-gnu": "python", - "ppc64le-unknown-linux-gnu": "python", - "s390x-unknown-linux-gnu": "python", - "riscv64-unknown-linux-gnu": "python", - "x86_64-apple-darwin": "python", - "x86_64-pc-windows-msvc": "python", - "aarch64-pc-windows-msvc": "python", - "x86_64-unknown-linux-gnu": "python", - "x86_64-unknown-linux-musl": "python", - "aarch64-apple-darwin-freethreaded": "python/install", - "aarch64-unknown-linux-gnu-freethreaded": "python/install", - "ppc64le-unknown-linux-gnu-freethreaded": "python/install", - "riscv64-unknown-linux-gnu-freethreaded": "python/install", - "s390x-unknown-linux-gnu-freethreaded": "python/install", - "x86_64-apple-darwin-freethreaded": "python/install", - "x86_64-pc-windows-msvc-freethreaded": "python/install", - "aarch64-pc-windows-msvc-freethreaded": "python/install", - "x86_64-unknown-linux-gnu-freethreaded": "python/install", - }, - }, - "3.13.9": { - "url": "20251031/cpython-{python_version}+20251031-{platform}-{build}.{ext}", - "sha256": { - "aarch64-apple-darwin": "1f3568d17383426d52350c2ef7c93c1a5a043198b860cb05e5d19b35f9c25cef", - "aarch64-unknown-linux-gnu": "0a56d11b0fb1662e67f892b9d5d1717aef06f24dbb8362bc25b8f784e620d44e", - "ppc64le-unknown-linux-gnu": "99492123902bd5e9a6b1a30135061e93a2e6a11d25107a741d5a756e91054448", - "riscv64-unknown-linux-gnu": "b3dce3e4ef508773521e1ee1be989fff6118f8fd1fbbd0491d7ff7dfbc98ef06", - "s390x-unknown-linux-gnu": "f10e34aaa856c1b8a69c2ea4a9a6723d520443d1a957bf66dc55491334ca0c1e", - "x86_64-apple-darwin": "48c0f3ca5d31e90658ef99138dc21865bb62f388ab97a1ce72cac176da194ab0", - "x86_64-pc-windows-msvc": "874593f641f31ea101440c70f81768c35d4d7d6df111fde63094db67465ef787", - "aarch64-pc-windows-msvc": "20db43873d3c4c2175d866806545e4ad4ec6bb72ca95e60082a4df6c24567e8c", - "aarch64-pc-windows-msvc-freethreaded": "743ff69935ef28834621647dab30f032dfcd80315732917531eea333210941c7", - "x86_64-unknown-linux-gnu": "6f05b91ee8c7e6dd0f9c60b95bb29130e2d623961de6578b643e80ddd83f96b6", - "x86_64-unknown-linux-musl": "ad987197034185e628715da504a50613af213dc21ba6d5ccaeab3db2c464aa6c", - "aarch64-apple-darwin-freethreaded": "eae1272a72ccce601590a10a9ca2a58199b5fcdf022aa603a527e3e2a04de9bc", - "aarch64-unknown-linux-gnu-freethreaded": "a6e72f9de5d9b46cf6968d6a492f2401a919f9b959f8da2d87f43484b80169ee", - "ppc64le-unknown-linux-gnu-freethreaded": "0ed5c65437f875c58ba1bee2b8d261d18698d3d0347a2e66f8902fce022a2cda", - "riscv64-unknown-linux-gnu-freethreaded": "584e481d9b5225ffaf02f158fb26d2818207e65fc3c6dc21a6d500277f739220", - "s390x-unknown-linux-gnu-freethreaded": "7fa7fb912ca989ceac026a332d56a2c7d6d16ab0e94d89e690de5aade26103e2", - "x86_64-apple-darwin-freethreaded": "e2bf5fa6a3ef443ade362e08b0a19bbc172f7bfe34dabe933ccaad31d53af5da", - "x86_64-pc-windows-msvc-freethreaded": "318a9a1e43dd52054327de3bccc0c5b7afde7b7f2a398ccb4d38e03d28b05386", - "x86_64-unknown-linux-gnu-freethreaded": "dcc29b069d0588fbd4ea29c6df840c8d1207d2a3bce8cd5cd57d1b85373b6048", - }, - "strip_prefix": { - "aarch64-apple-darwin": "python", - "aarch64-unknown-linux-gnu": "python", - "ppc64le-unknown-linux-gnu": "python", - "s390x-unknown-linux-gnu": "python", - "riscv64-unknown-linux-gnu": "python", - "x86_64-apple-darwin": "python", - "x86_64-pc-windows-msvc": "python", - "aarch64-pc-windows-msvc": "python", - "x86_64-unknown-linux-gnu": "python", - "x86_64-unknown-linux-musl": "python", - "aarch64-apple-darwin-freethreaded": "python/install", - "aarch64-unknown-linux-gnu-freethreaded": "python/install", - "ppc64le-unknown-linux-gnu-freethreaded": "python/install", - "riscv64-unknown-linux-gnu-freethreaded": "python/install", - "s390x-unknown-linux-gnu-freethreaded": "python/install", - "x86_64-apple-darwin-freethreaded": "python/install", - "x86_64-pc-windows-msvc-freethreaded": "python/install", - "aarch64-pc-windows-msvc-freethreaded": "python/install", - "x86_64-unknown-linux-gnu-freethreaded": "python/install", - }, - }, - "3.13.10": { - "url": "20251202/cpython-{python_version}+20251202-{platform}-{build}.{ext}", - "sha256": { - "aarch64-apple-darwin": "37afe4e77ab62ac50f197b1cb1f3bc02c82735c6be893da0996afcde5dc41048", - "aarch64-unknown-linux-gnu": "c68280591cda1c9515a04809fa6926020177e8e5892300206e0496ea1d10290e", - "ppc64le-unknown-linux-gnu": "1507e5528bd88131dc742a2941176aceea1838bc09860c21f179285b7865133b", - "riscv64-unknown-linux-gnu": "70169e916860b2e5b34c37c302d699eb2b8f24f28090968881942a37aeb7ed08", - "s390x-unknown-linux-gnu": "c5448863b64aacae62f3a213a6e6cf94ec63f96ee4d518491cd62fd3c81d952f", - "x86_64-apple-darwin": "a02761a4f189f71c0512e88df7ca2843696d61da659e47f8a5c8a9bd2c0d16f4", - "x86_64-pc-windows-msvc": "8b00014c7c35f9ad4cb1c565f067500bacc4125c8bc30e4389ee0be9fd6ffa3d", - "aarch64-pc-windows-msvc": "9060d644bd32ac0e0af970d0b21e207e6ff416b7c4dc26ffc4f9b043fb45b463", - "aarch64-pc-windows-msvc-freethreaded": "cdb7141327bdc244715b25752593e2c9eeb3cc2764f37dfe81cfbc92db9d6d57", - "x86_64-unknown-linux-gnu": "0cac1495fff920219904b1d573aaec0df54d549c226cb45f5c60cb6d2c72727a", - "x86_64-unknown-linux-musl": "04108190972ac98e13098abd972ec3f4f8b0880f83c0bb68249ce1a6164fa041", - "aarch64-apple-darwin-freethreaded": "3c9fdd76447c1549a0d3bc2a70c63f1daec997ab034206ac0260a03237166dbb", - "aarch64-unknown-linux-gnu-freethreaded": "6d277221fa4b172e00b29c7158ca9661917bc8db9a0084b1a0ff5c3a0ba8b648", - "ppc64le-unknown-linux-gnu-freethreaded": "d265d8d1c51e25ed70279540223589f79cf99ad00b50d28b6150c2658c973885", - "riscv64-unknown-linux-gnu-freethreaded": "ec411b4a2d167c3be0a9aeb3905e045d62c8e3c3db0caeade5d47d5f60b98dd0", - "s390x-unknown-linux-gnu-freethreaded": "4fc6443948bf5b729481ea02cc5c68e80cd0da42631f6936587a2b8fd45bc62c", - "x86_64-apple-darwin-freethreaded": "6ce608684df0f90350c7a1742e9685a7782d9b26ec99d1bd9d55c8cf9a405040", - "x86_64-pc-windows-msvc-freethreaded": "6a8b0372ded655e0d55318089fbce3122a446e69bcd120c79aaadfe9b017299c", - "x86_64-unknown-linux-gnu-freethreaded": "e39127fbe8d2ae7d86099f18b4da0918f9b60ce73ed491774d6dcfaa42b5c9ae", - }, - "strip_prefix": { - "aarch64-apple-darwin": "python", - "aarch64-unknown-linux-gnu": "python", - "ppc64le-unknown-linux-gnu": "python", - "s390x-unknown-linux-gnu": "python", - "riscv64-unknown-linux-gnu": "python", - "x86_64-apple-darwin": "python", - "x86_64-pc-windows-msvc": "python", - "aarch64-pc-windows-msvc": "python", - "x86_64-unknown-linux-gnu": "python", - "x86_64-unknown-linux-musl": "python", - "aarch64-apple-darwin-freethreaded": "python/install", - "aarch64-unknown-linux-gnu-freethreaded": "python/install", - "ppc64le-unknown-linux-gnu-freethreaded": "python/install", - "riscv64-unknown-linux-gnu-freethreaded": "python/install", - "s390x-unknown-linux-gnu-freethreaded": "python/install", - "x86_64-apple-darwin-freethreaded": "python/install", - "x86_64-pc-windows-msvc-freethreaded": "python/install", - "aarch64-pc-windows-msvc-freethreaded": "python/install", - "x86_64-unknown-linux-gnu-freethreaded": "python/install", - }, - }, - "3.13.11": { - "url": "20251209/cpython-{python_version}+20251209-{platform}-{build}.{ext}", - "sha256": { - "aarch64-apple-darwin": "295a9f7bc899ea1cc08baf60bbf511bdd1e4a29b2dd7e5f59b48f18bfa6bf585", - "aarch64-unknown-linux-gnu": "ea1e678e6e82301bb32bf3917732125949b6e46d541504465972024a3f165343", - "ppc64le-unknown-linux-gnu": "7660e53aad9d35ee256913c6d98427f81f078699962035c5fa8b5c3138695109", - "riscv64-unknown-linux-gnu": "763fa1548e6a432e9402916e690c74ea30f26dcd2e131893dd506f72b87c27c9", - "s390x-unknown-linux-gnu": "ffb6af51fbfabfc6fbc4e7379bdec70c2f51e972b1d2f45c053493b9da3a1bbe", - "x86_64-apple-darwin": "dac4a0a0a9b71f6b02a8b0886547fa22814474239bffb948e3e77185406ea136", - "x86_64-pc-windows-msvc": "87822417007045a28a7eccc47fe67b8c61265b99b10dbbfa24d231a3622b1c27", - "aarch64-pc-windows-msvc": "ba646d0c3b7dd7bdfb770d9b2ebd6cd2df02a37fda90c9c79a7cf59c7df6f165", - "aarch64-pc-windows-msvc-freethreaded": "6daf6d092c7294cfe68c4c7bf2698ac134235489c874b3bf796c7972b9dbba30", - "x86_64-unknown-linux-gnu": "1ffa06d714a44aea14c0c54c30656413e5955a6c92074b4b3cb4351dcc28b63b", - "x86_64-unknown-linux-musl": "969fe24017380b987c4e3ce15e9edf82a4618c1e61672b2cc9b021a1c98eae78", - "aarch64-apple-darwin-freethreaded": "4213058b7fcd875596c12b58cd46a399358b0a87ecde4b349cbdd00cf87ed79a", - "aarch64-unknown-linux-gnu-freethreaded": "290ca3bd0007db9e551f90b08dfcb6c1b2d62c33b2fc3e9a43e77d385d94f569", - "ppc64le-unknown-linux-gnu-freethreaded": "09d4b50f8abb443f7e3af858c920aa61c2430b0954df465e861caa7078e55e69", - "riscv64-unknown-linux-gnu-freethreaded": "5406f2a7cacafbd2aac3ce2de066a0929aab55423824276c36e04cb83babc36c", - "s390x-unknown-linux-gnu-freethreaded": "3984b67c4292892eaccdd1c094c7ec788884c4c9b3534ab6995f6be96d5ed51d", - "x86_64-apple-darwin-freethreaded": "d6f489464045d6895ae68b0a04a9e16477e74fe3185a75f3a9a0af8ccd25eade", - "x86_64-pc-windows-msvc-freethreaded": "bb9a29a7ba8f179273b79971da6aaa7be592d78c606a63f99eff3e4c12fb0fae", - "x86_64-unknown-linux-gnu-freethreaded": "33f89c957d986d525529b8a980103735776f4d20cf52f55960a057c760188ac3", - }, - "strip_prefix": { - "aarch64-apple-darwin": "python", - "aarch64-unknown-linux-gnu": "python", - "ppc64le-unknown-linux-gnu": "python", - "s390x-unknown-linux-gnu": "python", - "riscv64-unknown-linux-gnu": "python", - "x86_64-apple-darwin": "python", - "x86_64-pc-windows-msvc": "python", - "aarch64-pc-windows-msvc": "python", - "x86_64-unknown-linux-gnu": "python", - "x86_64-unknown-linux-musl": "python", - "aarch64-apple-darwin-freethreaded": "python/install", - "aarch64-unknown-linux-gnu-freethreaded": "python/install", - "ppc64le-unknown-linux-gnu-freethreaded": "python/install", - "riscv64-unknown-linux-gnu-freethreaded": "python/install", - "s390x-unknown-linux-gnu-freethreaded": "python/install", - "x86_64-apple-darwin-freethreaded": "python/install", - "x86_64-pc-windows-msvc-freethreaded": "python/install", - "aarch64-pc-windows-msvc-freethreaded": "python/install", - "x86_64-unknown-linux-gnu-freethreaded": "python/install", - }, - }, - "3.13.12": { - "url": "20260325/cpython-{python_version}+20260325-{platform}-{build}.{ext}", - "sha256": { - "aarch64-apple-darwin": "688da81bcaa6ed91792397c7d5433b13a4f02f021f940637c3972639bc516dca", - "aarch64-unknown-linux-gnu": "31c6e61eed48ca4e156d0e473025a792338641109e8277a63518ded438390c96", - "ppc64le-unknown-linux-gnu": "654939bc40d5f76f08eb17335bb19e9efa11eb48a0818eda2293a3f7c3570ae7", - "riscv64-unknown-linux-gnu": "fc7e1fb553c47b831ed7fa529575145207f000f967513f7b9ea809cce006ed79", - "s390x-unknown-linux-gnu": "7d7919358e88fcc672b061be8c2316c3a604c7074200515d7104166ed611f7f9", - "x86_64-apple-darwin": "7411e47939783708381017a90944a69641ac84d43f74fb6e2d52576c599a2717", - "x86_64-pc-windows-msvc": "5b4093f92d9bffcb0d92aea050f3d77d5a4fc8e918b31cea000ee4b3ca751f1d", - "aarch64-pc-windows-msvc": "d2c8b00044cd2e4c5fc7e697e63d5e481ed44b87c2def0beb42991d59f65d930", - "aarch64-pc-windows-msvc-freethreaded": "d2c8b00044cd2e4c5fc7e697e63d5e481ed44b87c2def0beb42991d59f65d930", - "x86_64-unknown-linux-gnu": "ebb1051ca2822b9803f46a5f10b6d51d153189ef1b1f1e142f733c0cbeaf86eb", - "x86_64-unknown-linux-musl": "b2e9400731c7f18069ec2804ba87a404385fe440f93b7dcb59004b9f56651202", - "aarch64-apple-darwin-freethreaded": "688da81bcaa6ed91792397c7d5433b13a4f02f021f940637c3972639bc516dca", - "aarch64-unknown-linux-gnu-freethreaded": "31c6e61eed48ca4e156d0e473025a792338641109e8277a63518ded438390c96", - "ppc64le-unknown-linux-gnu-freethreaded": "654939bc40d5f76f08eb17335bb19e9efa11eb48a0818eda2293a3f7c3570ae7", - "riscv64-unknown-linux-gnu-freethreaded": "fc7e1fb553c47b831ed7fa529575145207f000f967513f7b9ea809cce006ed79", - "s390x-unknown-linux-gnu-freethreaded": "7d7919358e88fcc672b061be8c2316c3a604c7074200515d7104166ed611f7f9", - "x86_64-apple-darwin-freethreaded": "7411e47939783708381017a90944a69641ac84d43f74fb6e2d52576c599a2717", - "x86_64-pc-windows-msvc-freethreaded": "5b4093f92d9bffcb0d92aea050f3d77d5a4fc8e918b31cea000ee4b3ca751f1d", - "x86_64-unknown-linux-gnu-freethreaded": "ebb1051ca2822b9803f46a5f10b6d51d153189ef1b1f1e142f733c0cbeaf86eb", - }, - "strip_prefix": "python", - }, - "3.13.13": { - "url": "20260414/cpython-{python_version}+20260414-{platform}-{build}.{ext}", - "sha256": { - "aarch64-apple-darwin": "c652dad552122cd2e76968ec41c803f8222038169b11310dba0c85928265f5c1", - "aarch64-unknown-linux-gnu": "6a65f68043d7fadcd580415493d2929d1fd686013f9ae44ddbd3a81307ab256d", - "ppc64le-unknown-linux-gnu": "aef73894107300264222b19e357baf5bad616b1c4bf5daa5c3b97cfee8f5ed7b", - "riscv64-unknown-linux-gnu": "f47c09f8e7f2fb0bc4afe52422705af4016c8d3ec1cf004b67bb56a86caa62cb", - "s390x-unknown-linux-gnu": "4d205af9654e1f33cefd23ff798af470e565f3ac0eba18d2f98f18a2abd07166", - "x86_64-apple-darwin": "540337412d2c4220e99280f741dbf45c1e3da3a39edaaab20c6ba1d53e1692ef", - "x86_64-pc-windows-msvc": "ee0cb26453d6e025d36502d765c1639c34830355e46ab3ad31c0360bc4cd9b79", - "aarch64-pc-windows-msvc": "586ba71c75f341e1d111399b7f719ae784dc11e8672e93e017388f28684226d0", - "aarch64-pc-windows-msvc-freethreaded": "586ba71c75f341e1d111399b7f719ae784dc11e8672e93e017388f28684226d0", - "x86_64-unknown-linux-gnu": "e5ec3b2c5693215d153c434ac018e75511b2c4f96d2bce30468a477cb3a89d5e", - "x86_64-unknown-linux-musl": "24ac6bf80dd2991c8be348f777c96c6eb69b71e78d8fa28c09beb3ddca015a47", - "aarch64-apple-darwin-freethreaded": "c652dad552122cd2e76968ec41c803f8222038169b11310dba0c85928265f5c1", - "aarch64-unknown-linux-gnu-freethreaded": "6a65f68043d7fadcd580415493d2929d1fd686013f9ae44ddbd3a81307ab256d", - "ppc64le-unknown-linux-gnu-freethreaded": "aef73894107300264222b19e357baf5bad616b1c4bf5daa5c3b97cfee8f5ed7b", - "riscv64-unknown-linux-gnu-freethreaded": "f47c09f8e7f2fb0bc4afe52422705af4016c8d3ec1cf004b67bb56a86caa62cb", - "s390x-unknown-linux-gnu-freethreaded": "4d205af9654e1f33cefd23ff798af470e565f3ac0eba18d2f98f18a2abd07166", - "x86_64-apple-darwin-freethreaded": "540337412d2c4220e99280f741dbf45c1e3da3a39edaaab20c6ba1d53e1692ef", - "x86_64-pc-windows-msvc-freethreaded": "ee0cb26453d6e025d36502d765c1639c34830355e46ab3ad31c0360bc4cd9b79", - "x86_64-unknown-linux-gnu-freethreaded": "e5ec3b2c5693215d153c434ac018e75511b2c4f96d2bce30468a477cb3a89d5e", - }, - "strip_prefix": "python", - }, - "3.14.0": { - "url": "20251031/cpython-{python_version}+20251031-{platform}-{build}.{ext}", - "sha256": { - "aarch64-apple-darwin": "b4bcd3c6c24cab32ae99e1b05c89312b783b4d69431d702e5012fe1fdcad4087", - "aarch64-unknown-linux-gnu": "128a9cbfb9645d5237ec01704d9d1d2ac5f084464cc43c37a4cd96aa9c3b1ad5", - "ppc64le-unknown-linux-gnu": "e16ca51f018e99a609faf953bd3a3aea31f45ee84262d1a517fb3abd98f1f4af", - "riscv64-unknown-linux-gnu": "fca340d8fb7a05cd90e216ce601b25d492ed8c1a3b6a6d77703e0f15ab3711a7", - "s390x-unknown-linux-gnu": "c5803644970eee931bb0581b3b64511d1a8612f67bc98951a7f7ab5581a9ed04", - "x86_64-apple-darwin": "4e71a3ce973be377ef18637826648bb936e2f9490f64a9e4f33a49bcc431d344", - "x86_64-pc-windows-msvc": "39acfcb3857d83eab054a3de11756ffc16b3d49c31393b9800dd2704d1f07fdf", - "aarch64-pc-windows-msvc": "599a8b7e12439cd95a201dbdfe95cf363146b1ff91f379555dafd86b170caab9", - "x86_64-unknown-linux-gnu": "3dec1ab70758a3467ac3313bbcdabf7a9b3016db5c072c4537e3cf0a9e6290f6", - "x86_64-unknown-linux-musl": "d0a2a6d3b1bb00dce2105377fda8aa79675d187f8d6d7010a42f651af25018dc", - "aarch64-apple-darwin-freethreaded": "d9c7b430b25bd3837dbb03f945dbe6b7bc526c5940ca96f5db7cdc42f6b2b801", - "aarch64-unknown-linux-gnu-freethreaded": "f383ef50d1da6ca511212e5ae601923b56636b87351fd5fc847e0ea0a19fa9b3", - "ppc64le-unknown-linux-gnu-freethreaded": "cb0e4ff781b856a47f0f461ceb41c78c7eeff65effd0957857ec4702ef1e1bd3", - "riscv64-unknown-linux-gnu-freethreaded": "929223470d11a55cd75f880ac3bd4969e42407e2cdf08d4e7e38ba721cf4abec", - "s390x-unknown-linux-gnu-freethreaded": "613fb1f7b249f798b52af957d181305244e936c8e5c94c84688fcdf93fe14253", - "x86_64-apple-darwin-freethreaded": "b3196f6b57bbb3dc2ee07f348f1d51117ffa376979eceafbf50c15f0f7980bf8", - "x86_64-pc-windows-msvc-freethreaded": "b81de5fc9e783ea6dfcf1098c28a278c874999c71afbb0309f6a8b4276c769d0", - "aarch64-pc-windows-msvc-freethreaded": "40266e60f655e49cd1d5303295255909a4b593b08b88be6e6a55b2c9fe6ed13d", - "x86_64-unknown-linux-gnu-freethreaded": "f4acbef0fbfaf7ab31ac63986da1d93dfa1c5cb797de1dcdc1a988aa18670120", - }, - "strip_prefix": { - "aarch64-apple-darwin": "python", - "aarch64-unknown-linux-gnu": "python", - "ppc64le-unknown-linux-gnu": "python", - "s390x-unknown-linux-gnu": "python", - "riscv64-unknown-linux-gnu": "python", - "x86_64-apple-darwin": "python", - "x86_64-pc-windows-msvc": "python", - "aarch64-pc-windows-msvc": "python", - "x86_64-unknown-linux-gnu": "python", - "x86_64-unknown-linux-musl": "python", - "aarch64-apple-darwin-freethreaded": "python/install", - "aarch64-unknown-linux-gnu-freethreaded": "python/install", - "ppc64le-unknown-linux-gnu-freethreaded": "python/install", - "riscv64-unknown-linux-gnu-freethreaded": "python/install", - "s390x-unknown-linux-gnu-freethreaded": "python/install", - "x86_64-apple-darwin-freethreaded": "python/install", - "x86_64-pc-windows-msvc-freethreaded": "python/install", - "aarch64-pc-windows-msvc-freethreaded": "python/install", - "x86_64-unknown-linux-gnu-freethreaded": "python/install", - }, - }, - "3.14.1": { - "url": "20251202/cpython-{python_version}+20251202-{platform}-{build}.{ext}", - "sha256": { - "aarch64-apple-darwin": "cdf1ba0789f529fa34bb5b5619c5da9757ac1067d6b8dd0ee8b78e50078fc561", - "aarch64-unknown-linux-gnu": "5dde7dba0b8ef34c0d5cb8a721254b1e11028bfc09ff06664879c245fe8df73f", - "ppc64le-unknown-linux-gnu": "d2774701d53e2ac06f8c8c8e52dfa4ff346890de9b417c9a7664195443a4c766", - "riscv64-unknown-linux-gnu": "af840506efbcd5026d9140c0a0230e45e46bb1f339a65c10a22875930b2c0159", - "s390x-unknown-linux-gnu": "43f8f79bf4c66689d2019f193671d1df3e5e5dbb293382036285e8ce55fc55bb", - "x86_64-apple-darwin": "f25ce050e1d370f9c05c9623b769ffa4b269a6ae17e611b435fd2b8b09972a88", - "x86_64-pc-windows-msvc": "cb478a5a37eb93ce4d3c27ae64d211d6a5a42475ae53f666a8d1570e71fcf409", - "aarch64-pc-windows-msvc": "19129cf8b4d68c4e64c25bae43bca139d871267b59cf7f02b9dcf25f0bf59497", - "x86_64-unknown-linux-gnu": "a72f313bad49846e5e9671af2be7476033a877c80831cf47f431400ccb520090", - "x86_64-unknown-linux-musl": "15d50b15713097c38c67b1a06a0498ad102377f9b3999e98e4eefd6bf91bd82d", - "aarch64-apple-darwin-freethreaded": "61f38e947449cf00f32f0838e813358f6bf61025d0797531e5b8b8b175c617f0", - "aarch64-unknown-linux-gnu-freethreaded": "1a88a1fe21eb443d280999464b1a397605a7ca950d8ab73813ca6868835439a2", - "ppc64le-unknown-linux-gnu-freethreaded": "7207b736ed2569f307649ffd4b615a5346631bc244730b8702babee377cef528", - "riscv64-unknown-linux-gnu-freethreaded": "d1356ccd279920edc31bf0350674d966beb9522f9503846ed7855dbb109ccc14", - "s390x-unknown-linux-gnu-freethreaded": "477758eabc06dbc7e5e5d16e97c4672478acd409f420dd2e1b84d3452c0668d1", - "x86_64-apple-darwin-freethreaded": "c2cb2a9b44285fbc13c3c9b7eea813db6ed8d94909406b059db7afd39b32e786", - "x86_64-pc-windows-msvc-freethreaded": "8ef7048315cac6d26bdbef18512a87b1a24fffa21cec86e32f9a9425f2af9bf6", - "aarch64-pc-windows-msvc-freethreaded": "ddb10b645de2b1f6f2832a80b115a9cd34a4a760249983027efe46618a8efc48", - "x86_64-unknown-linux-gnu-freethreaded": "c5d5b89aab7de683e465e36de2477a131435076badda775ef6e9ea21109c1c32", - }, - "strip_prefix": { - "aarch64-apple-darwin": "python", - "aarch64-unknown-linux-gnu": "python", - "ppc64le-unknown-linux-gnu": "python", - "s390x-unknown-linux-gnu": "python", - "riscv64-unknown-linux-gnu": "python", - "x86_64-apple-darwin": "python", - "x86_64-pc-windows-msvc": "python", - "aarch64-pc-windows-msvc": "python", - "x86_64-unknown-linux-gnu": "python", - "x86_64-unknown-linux-musl": "python", - "aarch64-apple-darwin-freethreaded": "python/install", - "aarch64-unknown-linux-gnu-freethreaded": "python/install", - "ppc64le-unknown-linux-gnu-freethreaded": "python/install", - "riscv64-unknown-linux-gnu-freethreaded": "python/install", - "s390x-unknown-linux-gnu-freethreaded": "python/install", - "x86_64-apple-darwin-freethreaded": "python/install", - "x86_64-pc-windows-msvc-freethreaded": "python/install", - "aarch64-pc-windows-msvc-freethreaded": "python/install", - "x86_64-unknown-linux-gnu-freethreaded": "python/install", - }, - }, - "3.14.2": { - "url": "20251209/cpython-{python_version}+20251209-{platform}-{build}.{ext}", - "sha256": { - "aarch64-apple-darwin": "2f74bd26bd16487aca357c879d11f7b16c0521328e5148a1930ab6357bcb89fe", - "aarch64-unknown-linux-gnu": "869af31b2963194e8a2ecfadc36027c4c1c86a10f4960baec36dadb41b2acf02", - "ppc64le-unknown-linux-gnu": "86129976403fb5d64cf576329f94148f28cf6f82834e94df81ff31e9d5f404e0", - "riscv64-unknown-linux-gnu": "318dceecf119ea903aef1fb03a552cc592ecd61c08da891b68f5755e21e13511", - "s390x-unknown-linux-gnu": "53875c849a14194344ead1d9cd1e128cadd42a4b83c35eeb212417909ef05a6a", - "x86_64-apple-darwin": "58fa3e17d13ab956fd11055fb774c98ecfddcdf3b588e5f2369bdbc14ef9d76a", - "x86_64-pc-windows-msvc": "0d660bba9f58cb552e7e99e1f96a9c67b41618c9b8d29f9f3515fe2b5ad1966e", - "aarch64-pc-windows-msvc": "0be0d2557d73efa7f6f3f99679f05252d57fe2aad2d81cac3cad410a9b1eacbd", - "x86_64-unknown-linux-gnu": "121c3249bef497adf601df76a4d89aed6053fc5ec2f8c0ec656b86f0142e8ddd", - "x86_64-unknown-linux-musl": "71639cc5d1fb79840467531c5b53ca77170a58edd3f7e2d29330dd736e477469", - "aarch64-apple-darwin-freethreaded": "d6d17b8ef28326552cdeb2a7541c8a0cb711b378df9b93ebdb461dca065edfea", - "aarch64-unknown-linux-gnu-freethreaded": "adfcb90f3a7e1b3fbc6a99f9c8c8dce1f2e26ea72b724bbe4e9fa39e81e2b0db", - "ppc64le-unknown-linux-gnu-freethreaded": "2b1ce0c5a5f5e5add7e4f934f5bd35ac41660895a30b3098db7f7303d6952a4f", - "riscv64-unknown-linux-gnu-freethreaded": "4efb610fa07a6ee2639d14d78fc3b6ecb47431c14e1e4bda03c7f7dd60a5c1e5", - "s390x-unknown-linux-gnu-freethreaded": "e62f3bb3e66dac6c459690f9e9cd8cc2f6fe1dcf8bfed452af4c3df24cd7874f", - "x86_64-apple-darwin-freethreaded": "1fd76c79f7fc1753e8d2ed2f71406c0b65776c75f3e95ed99ffde8c95af2adc1", - "x86_64-pc-windows-msvc-freethreaded": "9927951e3997c186d2813ca1a0f4a8f5a2f771463f7f8ad0752fd3d2be2b74e4", - "aarch64-pc-windows-msvc-freethreaded": "43aac5bb4cdba71fc6775d26f47348d573a0b1210911438be71d7d96f4b18b51", - "x86_64-unknown-linux-gnu-freethreaded": "3728872ffd74989a7b4bbf3f0c629ae8fe821cda2bd6544012c1b92b9f5d5a5b", - }, - "strip_prefix": { - "aarch64-apple-darwin": "python", - "aarch64-unknown-linux-gnu": "python", - "ppc64le-unknown-linux-gnu": "python", - "s390x-unknown-linux-gnu": "python", - "riscv64-unknown-linux-gnu": "python", - "x86_64-apple-darwin": "python", - "x86_64-pc-windows-msvc": "python", - "aarch64-pc-windows-msvc": "python", - "x86_64-unknown-linux-gnu": "python", - "x86_64-unknown-linux-musl": "python", - "aarch64-apple-darwin-freethreaded": "python/install", - "aarch64-unknown-linux-gnu-freethreaded": "python/install", - "ppc64le-unknown-linux-gnu-freethreaded": "python/install", - "riscv64-unknown-linux-gnu-freethreaded": "python/install", - "s390x-unknown-linux-gnu-freethreaded": "python/install", - "x86_64-apple-darwin-freethreaded": "python/install", - "x86_64-pc-windows-msvc-freethreaded": "python/install", - "aarch64-pc-windows-msvc-freethreaded": "python/install", - "x86_64-unknown-linux-gnu-freethreaded": "python/install", - }, - }, - "3.14.3": { - "url": "20260325/cpython-{python_version}+20260325-{platform}-{build}.{ext}", - "sha256": { - "aarch64-apple-darwin": "80c996c23aab828134821f078a8a77a6f33f3f2c14000f071718c540e20c64d4", - "aarch64-unknown-linux-gnu": "6faf5478f910741c477830f5fd842011208af0f9678faf77106c9421b325bfc1", - "ppc64le-unknown-linux-gnu": "5eafe32e12f33f98c40de920482b013170dcf97d8c7f5dc780271ccf4cded76a", - "riscv64-unknown-linux-gnu": "481d3faef258964e57b7102c63de12b2bb388c7ed07cfe456f33e63b4e061202", - "s390x-unknown-linux-gnu": "d706eae2f4d963187b7c866603aed75d7eb3ea59590b06fb34f5fd7d0fe8e432", - "x86_64-apple-darwin": "847a49fea36c066f8df7a57cd8c4c02d17667e25d30b7930e8f8ba15e72d7efc", - "x86_64-pc-windows-msvc": "8b4e1329c4901ce2c0f1c20ac5d2ffa62fc13f12e26b5d1e5a1000f910f980d4", - "aarch64-pc-windows-msvc": "b35fe7c2fe169574f382cef125e95cbd904ddcb98fc337167356371b6d2e8c60", - "x86_64-unknown-linux-gnu": "18270c5a7b1a572599df5e68b497ba5254811dac43ba6f542245807d821fcb44", - "x86_64-unknown-linux-musl": "726a28734d2878a637b0d16ce07ce24c7d6ca1043d8e6f4a23b1b0a3478eedb9", - "aarch64-apple-darwin-freethreaded": "80c996c23aab828134821f078a8a77a6f33f3f2c14000f071718c540e20c64d4", - "aarch64-unknown-linux-gnu-freethreaded": "6faf5478f910741c477830f5fd842011208af0f9678faf77106c9421b325bfc1", - "ppc64le-unknown-linux-gnu-freethreaded": "5eafe32e12f33f98c40de920482b013170dcf97d8c7f5dc780271ccf4cded76a", - "riscv64-unknown-linux-gnu-freethreaded": "481d3faef258964e57b7102c63de12b2bb388c7ed07cfe456f33e63b4e061202", - "s390x-unknown-linux-gnu-freethreaded": "d706eae2f4d963187b7c866603aed75d7eb3ea59590b06fb34f5fd7d0fe8e432", - "x86_64-apple-darwin-freethreaded": "847a49fea36c066f8df7a57cd8c4c02d17667e25d30b7930e8f8ba15e72d7efc", - "x86_64-pc-windows-msvc-freethreaded": "8b4e1329c4901ce2c0f1c20ac5d2ffa62fc13f12e26b5d1e5a1000f910f980d4", - "aarch64-pc-windows-msvc-freethreaded": "b35fe7c2fe169574f382cef125e95cbd904ddcb98fc337167356371b6d2e8c60", - "x86_64-unknown-linux-gnu-freethreaded": "18270c5a7b1a572599df5e68b497ba5254811dac43ba6f542245807d821fcb44", - }, - "strip_prefix": "python", - }, - "3.14.4": { - "url": "20260414/cpython-{python_version}+20260414-{platform}-{build}.{ext}", - "sha256": { - "aarch64-apple-darwin": "8b7865e511b17093e090449bf71eb52933c17d45ad5257ddeacaffbb2c7239df", - "aarch64-unknown-linux-gnu": "5c8db1c21023316adad827a46d917bbbd6a85ae4e39bc3a58febda712c2f963d", - "ppc64le-unknown-linux-gnu": "055977a09de092744bbb22db64144e6afef8592eaac5e2bce4cca33f2592281a", - "riscv64-unknown-linux-gnu": "e959df167c502fb0bbcacc31a997e25c6b0ff6b5e496321b691955aa702d0c09", - "s390x-unknown-linux-gnu": "35f70ad05b2c4045889ee0c3d93f61b012654c1d91e10e671f0e5b4d4a6c6637", - "x86_64-apple-darwin": "9ecb2b942e6698c04af10a63a3d73c0b2e8d8e11ce44933fbffe8651bef4577d", - "x86_64-pc-windows-msvc": "9647bb46d3c236e34c1c11bbb7113444d9711811f0d11c39956168807a955b1a", - "aarch64-pc-windows-msvc": "82613380d582d806e562d7701496c34c87753ab13c37aa0afe2039003651f389", - "x86_64-unknown-linux-gnu": "e17275eaf95ceb5877aa6816e209b7733f41fee401d39c3921b88fb73fc4a4ba", - "x86_64-unknown-linux-musl": "12687a989a2384665577e1ef9864f33d4c074a1e69b38a8bac8d656531aefa3e", - "aarch64-apple-darwin-freethreaded": "8b7865e511b17093e090449bf71eb52933c17d45ad5257ddeacaffbb2c7239df", - "aarch64-unknown-linux-gnu-freethreaded": "5c8db1c21023316adad827a46d917bbbd6a85ae4e39bc3a58febda712c2f963d", - "ppc64le-unknown-linux-gnu-freethreaded": "055977a09de092744bbb22db64144e6afef8592eaac5e2bce4cca33f2592281a", - "riscv64-unknown-linux-gnu-freethreaded": "e959df167c502fb0bbcacc31a997e25c6b0ff6b5e496321b691955aa702d0c09", - "s390x-unknown-linux-gnu-freethreaded": "35f70ad05b2c4045889ee0c3d93f61b012654c1d91e10e671f0e5b4d4a6c6637", - "x86_64-apple-darwin-freethreaded": "9ecb2b942e6698c04af10a63a3d73c0b2e8d8e11ce44933fbffe8651bef4577d", - "x86_64-pc-windows-msvc-freethreaded": "9647bb46d3c236e34c1c11bbb7113444d9711811f0d11c39956168807a955b1a", - "aarch64-pc-windows-msvc-freethreaded": "82613380d582d806e562d7701496c34c87753ab13c37aa0afe2039003651f389", - "x86_64-unknown-linux-gnu-freethreaded": "e17275eaf95ceb5877aa6816e209b7733f41fee401d39c3921b88fb73fc4a4ba", - }, - "strip_prefix": "python", - }, - "3.15.0a1": { - "url": "20251031/cpython-{python_version}+20251031-{platform}-{build}.{ext}", - "sha256": { - "aarch64-apple-darwin": "3acf7aa3559b746498b18929456c5cacb84bae4e09249834cbc818970d71de87", - "aarch64-unknown-linux-gnu": "d55c2aeece827e6bec83fd18515ee281d9ea0efaa3e2d20130db8f1c7cbb71c6", - "ppc64le-unknown-linux-gnu": "c28beda791c499b16f06256339522f0002a3e9acba003e6b8374755d7be1def2", - "riscv64-unknown-linux-gnu": "36619f576b8154e4b56643c5c4a85c352f152df2989c4e602cbbe9c2b7ded870", - "s390x-unknown-linux-gnu": "5ea47be2a3a563ddd87ff510dae26b7aa7f3855ca00c5f1056ff8114c067c4e4", - "x86_64-apple-darwin": "0ab19d3ac25f99da438b088751e5ec2421f9f6aa4292fd2dc0f8e49eb3e16bdf", - "x86_64-pc-windows-msvc": "5f5d6bec2b381cfc771c49972d2a6f7b7e7ab6a1651d8fb6ef3983f3571722b3", - "aarch64-pc-windows-msvc": "1508bcd7195008479ed156aad3afbb3a3793097ed530690f0304a8107f0e53e8", - "x86_64-unknown-linux-gnu": "1f356288c2b2713619cb7a4e453d33bf8882f812af2987e21e01e7ae382fefba", - "x86_64-unknown-linux-musl": "caf5311f333eef082dd69a669ca65aceba09a08fc1e78aad602ad649106f294c", - "aarch64-apple-darwin-freethreaded": "12f1b16be4017181ad67904caf9e59e525b9b5d62f49105017d837e27b832959", - "aarch64-unknown-linux-gnu-freethreaded": "981fe8dfc6e7e1d0ffefa945a18d5c4c759bbe21722acf3a5cc7e62f16aa5f3c", - "ppc64le-unknown-linux-gnu-freethreaded": "088400dec25139f38eeecb48f090ff2ce06a96a1dd79fa8f1dfec1cd1786f5ef", - "riscv64-unknown-linux-gnu-freethreaded": "938061a0a31a06672526885de36037ddefd8c4acdb09424691b7000a8c8f8d01", - "s390x-unknown-linux-gnu-freethreaded": "2003e7e40bb44b3db7bca81087bfb738fe6af40e5db61cda8e23b59bf55d409e", - "x86_64-apple-darwin-freethreaded": "64fc29e6c7a2f02a18645d968f1b3fc1d00d12a5ef3fcbb0d077fa8c62c08904", - "x86_64-pc-windows-msvc-freethreaded": "34abc5603e1b4131f753d29b7deac865b9277912b851cbed5a149cf3e6745d3d", - "aarch64-pc-windows-msvc-freethreaded": "54ca78dae455ece6fefbd7f5f287cc55d5ce197caf51921f6d871d15069d9489", - "x86_64-unknown-linux-gnu-freethreaded": "0e0272186d9f5169394dbc4d4d72a3f4a5762a04c2e5ac2ab1e23aa41fc8538a", - }, - "strip_prefix": { - "aarch64-apple-darwin": "python", - "aarch64-unknown-linux-gnu": "python", - "ppc64le-unknown-linux-gnu": "python", - "s390x-unknown-linux-gnu": "python", - "riscv64-unknown-linux-gnu": "python", - "x86_64-apple-darwin": "python", - "x86_64-pc-windows-msvc": "python", - "aarch64-pc-windows-msvc": "python", - "x86_64-unknown-linux-gnu": "python", - "x86_64-unknown-linux-musl": "python", - "aarch64-apple-darwin-freethreaded": "python/install", - "aarch64-unknown-linux-gnu-freethreaded": "python/install", - "ppc64le-unknown-linux-gnu-freethreaded": "python/install", - "riscv64-unknown-linux-gnu-freethreaded": "python/install", - "s390x-unknown-linux-gnu-freethreaded": "python/install", - "x86_64-apple-darwin-freethreaded": "python/install", - "x86_64-pc-windows-msvc-freethreaded": "python/install", - "aarch64-pc-windows-msvc-freethreaded": "python/install", - "x86_64-unknown-linux-gnu-freethreaded": "python/install", - }, - }, - "3.15.0a2": { - "url": "20251209/cpython-{python_version}+20251209-{platform}-{build}.{ext}", - "sha256": { - "aarch64-apple-darwin": "5851f3744fbd39e3e323844cf4f68d7763fb25546aa5ffbb71b1b5ab69c56616", - "aarch64-unknown-linux-gnu": "17ba65d669be3052524e03b4d1426c072ef38df2a9065ff4525d1f4d1bc9f82c", - "ppc64le-unknown-linux-gnu": "5585bd7c5eefe28b9bf544d902cad9a2f81f33c618f2a1d3c006cbfcdec77abc", - "riscv64-unknown-linux-gnu": "bb7252edaffd422bd1c044a4764dfcf83a5d7159942f445abbef524e54ea79a0", - "s390x-unknown-linux-gnu": "03a90ffa9f92d4cf4caeefb9d15f0b39c05c1e60ade6688f32165f957db4f8f3", - "x86_64-apple-darwin": "cee576de4919cd422dbc31eb85d3c145ee82acec84f651daaf32dc669b5149c9", - "x86_64-pc-windows-msvc": "e538475ee249eacf63bfdae0e70af73e9c47360e6dd3d6825e7a35107e177de5", - "aarch64-pc-windows-msvc": "39bc2fcac13aeba7d650f76badf63350a81c86167a62174cb092eab7a749f4a5", - "x86_64-unknown-linux-gnu": "58addaabfab2de422180d32543fb3878ffc984c8a2e4005ff658a5cd83b31fc7", - "x86_64-unknown-linux-musl": "dcf844400dc2e7f5f3604e994532e4d49db45f4deefe9afdf6809ca1bc6532ee", - "aarch64-apple-darwin-freethreaded": "5b34488580df13df051a2e84e43cfca2ab28fdd7a61052f35988eb8b481b894a", - "aarch64-unknown-linux-gnu-freethreaded": "0c2c83236f6e28c103e2660a82be94b2459ee8cfdd90f5dd82f0d503ca2aec09", - "ppc64le-unknown-linux-gnu-freethreaded": "216842df2377fd032f279ded7fd23d7bdbd92d4c1fa7619523bc0dbdef5bd212", - "riscv64-unknown-linux-gnu-freethreaded": "2a8b56f318d2e21b01b54909554c53d81871b9bb05d23ea7808dde9acec4dc7e", - "s390x-unknown-linux-gnu-freethreaded": "06c4ca3983aad20723f68786e3663ab49fee1bf09326f341649205ed79d34fc6", - "x86_64-apple-darwin-freethreaded": "4d8102b70ea9fe726ee3ae9ad9e9bc4cbe0b6ed18f7989c81aef81de578f0163", - "x86_64-pc-windows-msvc-freethreaded": "6ff71bac78d650ce621fe6db49f06290e48bcceb61f69cccc7728584f70b6346", - "aarch64-pc-windows-msvc-freethreaded": "3d99152b4e29b947fb1cfc8d035d1d511e50aeed72886ff4a5fd0a3694bd0b51", - "x86_64-unknown-linux-gnu-freethreaded": "70f552e213734c0e260a57603bee504dd7ed0e78a10558b591e724ea8730fef5", - }, - "strip_prefix": { - "aarch64-apple-darwin": "python", - "aarch64-unknown-linux-gnu": "python", - "ppc64le-unknown-linux-gnu": "python", - "s390x-unknown-linux-gnu": "python", - "riscv64-unknown-linux-gnu": "python", - "x86_64-apple-darwin": "python", - "x86_64-pc-windows-msvc": "python", - "aarch64-pc-windows-msvc": "python", - "x86_64-unknown-linux-gnu": "python", - "x86_64-unknown-linux-musl": "python", - "aarch64-apple-darwin-freethreaded": "python/install", - "aarch64-unknown-linux-gnu-freethreaded": "python/install", - "ppc64le-unknown-linux-gnu-freethreaded": "python/install", - "riscv64-unknown-linux-gnu-freethreaded": "python/install", - "s390x-unknown-linux-gnu-freethreaded": "python/install", - "x86_64-apple-darwin-freethreaded": "python/install", - "x86_64-pc-windows-msvc-freethreaded": "python/install", - "aarch64-pc-windows-msvc-freethreaded": "python/install", - "x86_64-unknown-linux-gnu-freethreaded": "python/install", - }, - }, - "3.15.0a8": { - "url": "20260414/cpython-{python_version}+20260414-{platform}-{build}.{ext}", - "sha256": { - "aarch64-apple-darwin": "780d46b3da0e58e15c620d9e7dfd29b54c8359c195f625858f85df9c2c7ecc32", - "aarch64-unknown-linux-gnu": "8f6dda4d8ff44976f1aa6a94674a09a503dc50b015297e1b62c8cdc591c90f4f", - "ppc64le-unknown-linux-gnu": "09f076c63fadbf675143674aa3b23229482b9a44840b8b1808a216def2a9af15", - "riscv64-unknown-linux-gnu": "9d41ce752e8b731872f0f5c9c48199e63c789d24ce3ae9e91d6c8008f36e7c51", - "s390x-unknown-linux-gnu": "1de2593c40cce2d8ea883f8c8580223bfa1478cbd9d0191ba3640aed083c2202", - "x86_64-apple-darwin": "a7744d34148969a2ec010da6f0a46ddeceda7c02e5cdfa2b4e1811487381491a", - "x86_64-pc-windows-msvc": "3ded476f676fdf260d56a5e49aa083d5ffd218fc3390e4480ed42bee1acfb3fb", - "aarch64-pc-windows-msvc": "10fb470e900e65df4e37f8deaf1726397c914861ffc37b43ae3743a7eee88377", - "x86_64-unknown-linux-gnu": "c93f4b15287ac48d7e3a475b245cb59cc51079382747e3e6213d6406c158969d", - "x86_64-unknown-linux-musl": "9fbd6f243a424d4ae973e72aa0075122a7cfe05ac8f6cfde986e7b00d0dbc0bf", - "aarch64-apple-darwin-freethreaded": "780d46b3da0e58e15c620d9e7dfd29b54c8359c195f625858f85df9c2c7ecc32", - "aarch64-unknown-linux-gnu-freethreaded": "8f6dda4d8ff44976f1aa6a94674a09a503dc50b015297e1b62c8cdc591c90f4f", - "ppc64le-unknown-linux-gnu-freethreaded": "09f076c63fadbf675143674aa3b23229482b9a44840b8b1808a216def2a9af15", - "riscv64-unknown-linux-gnu-freethreaded": "9d41ce752e8b731872f0f5c9c48199e63c789d24ce3ae9e91d6c8008f36e7c51", - "s390x-unknown-linux-gnu-freethreaded": "1de2593c40cce2d8ea883f8c8580223bfa1478cbd9d0191ba3640aed083c2202", - "x86_64-apple-darwin-freethreaded": "a7744d34148969a2ec010da6f0a46ddeceda7c02e5cdfa2b4e1811487381491a", - "x86_64-pc-windows-msvc-freethreaded": "3ded476f676fdf260d56a5e49aa083d5ffd218fc3390e4480ed42bee1acfb3fb", - "aarch64-pc-windows-msvc-freethreaded": "10fb470e900e65df4e37f8deaf1726397c914861ffc37b43ae3743a7eee88377", - "x86_64-unknown-linux-gnu-freethreaded": "c93f4b15287ac48d7e3a475b245cb59cc51079382747e3e6213d6406c158969d", - }, - "strip_prefix": "python", - }, -} +DEFAULT_RELEASE_BASE_URL = _GITHUB_PREFIX +DEFAULT_RELEASE_BASE_URLS = [ + _GITHUB_PREFIX, + _ASTRAL_PREFIX, + _LEGACY_GITHUB_PREFIX, +] # buildifier: disable=unsorted-dict-items MINOR_MAPPING = { @@ -1426,26 +207,41 @@ def _generate_platforms(): PLATFORMS = _generate_platforms() -def get_release_info(platform, python_version, base_url = DEFAULT_RELEASE_BASE_URL, tool_versions = TOOL_VERSIONS): +def get_release_info(platform, python_version, base_urls = DEFAULT_RELEASE_BASE_URLS, tool_versions = None): """Resolve the release URL for the requested interpreter version Args: platform: The platform string for the interpreter python_version: The version of the interpreter to get - base_url: The URL to prepend to the 'url' attr in the tool_versions dict + base_urls: The list of URLs to prepend to the 'url' attr in the tool_versions dict tool_versions: A dict listing the interpreter versions, their SHAs and URL Returns: A tuple of (filename, url, archive strip prefix, patches, patch_strip) """ + if tool_versions == None: + tool_versions = TOOL_VERSIONS + + if type(base_urls) == type(""): + base_urls = [base_urls] + elif base_urls == None: + base_urls = DEFAULT_RELEASE_BASE_URLS - base_urls = [base_url] - if base_url == DEFAULT_RELEASE_BASE_URL or base_url.startswith(_GITHUB_PREFIX): - suffix = base_url[len(_GITHUB_PREFIX):] - base_urls.append(_ASTRAL_PREFIX + suffix) - elif base_url.startswith(_LEGACY_GITHUB_PREFIX): - suffix = base_url[len(_LEGACY_GITHUB_PREFIX):] - base_urls.append(_ASTRAL_PREFIX + suffix) + expanded_base_urls = [] + for b_url in base_urls: + if b_url not in expanded_base_urls: + expanded_base_urls.append(b_url) + if b_url.startswith(_GITHUB_PREFIX): + suffix = b_url[len(_GITHUB_PREFIX):] + a_url = _ASTRAL_PREFIX + suffix + if a_url not in expanded_base_urls: + expanded_base_urls.append(a_url) + elif b_url.startswith(_LEGACY_GITHUB_PREFIX): + suffix = b_url[len(_LEGACY_GITHUB_PREFIX):] + a_url = _ASTRAL_PREFIX + suffix + if a_url not in expanded_base_urls: + expanded_base_urls.append(a_url) + base_urls = expanded_base_urls url = tool_versions[python_version]["url"] @@ -1530,3 +326,67 @@ def gen_python_config_settings(name = ""): flag_values = PLATFORMS[platform].flag_values, constraint_values = PLATFORMS[platform].compatible_with, ) + +def _manifest_entry_sort_key(entry): + flavor_rank = {"full": 3, "install_only": 1, "install_only_stripped": 2}.get(entry.archive_flavor, 4) + microarch = entry.microarch + if not microarch: + microarch_rank = 0 + elif microarch.startswith("v") and microarch[1:].isdigit(): + microarch_rank = int(microarch[1:]) + else: + microarch_rank = 999 + return (flavor_rank, microarch_rank) + +def _tool_versions_from_manifest_entries(entries, base_url = DEFAULT_RELEASE_BASE_URL): + """Converts parsed manifest entries into the TOOL_VERSIONS dictionary format. + + Args: + entries: {type}`list[struct]` the parsed manifest entries. + base_url: {type}`str` fallback base URL for standalone distributions. + + Returns: + {type}`dict[str, dict]` the tool versions map. + """ + available_versions = {} + entries = sorted( + entries, + key = _manifest_entry_sort_key, + ) + + for entry in entries: + location = entry.location + sha256 = entry.sha256 + py_version = entry.python_version + + matched_platform = "{}-{}-{}".format(entry.arch, entry.vendor, entry.os) + if entry.libc: + matched_platform += "-" + entry.libc + if entry.freethreaded: + matched_platform += "-freethreaded" + + if matched_platform not in PLATFORMS: + continue + + archive_flavor = entry.archive_flavor + if archive_flavor not in ["install_only", "install_only_stripped", "full"]: + continue + + v_dict = available_versions.setdefault(py_version, {}) + if matched_platform in v_dict.get("sha256", {}): + continue + + if "://" in location: + urls = [location] + else: + urls = ["{}/{}".format(base_url, location)] + + strip_prefix = "python/install" if archive_flavor == "full" else "python" + + v_dict.setdefault("sha256", {})[matched_platform] = sha256 + v_dict.setdefault("url", {})[matched_platform] = urls + v_dict.setdefault("strip_prefix", {})[matched_platform] = strip_prefix + + return available_versions + +TOOL_VERSIONS = _tool_versions_from_manifest_entries(parse_runtime_manifest(MANIFEST_TEXT)) diff --git a/replicate_ci b/replicate_ci new file mode 100755 index 0000000000..dd9b67e93e --- /dev/null +++ b/replicate_ci @@ -0,0 +1,167 @@ +#!/usr/bin/env python3 + +import argparse +import os +import shlex +import subprocess +import sys + +import yaml + + +def parse_args(): + parser = argparse.ArgumentParser( + description="Replicate and emulate BazelCI job configurations from presubmit.yml." + ) + parser.add_argument( + "job", + help="The key or name of the CI job to emulate (e.g., ubuntu_workspace).", + ) + return parser.parse_args() + + +def run_cmd(cmd, cwd=None, env=None, shell=False): + if shell: + cmd_str = cmd if isinstance(cmd, str) else " ".join(cmd) + else: + cmd_str = shlex.join(cmd) if isinstance(cmd, list) else str(cmd) + + print(f"\n🚀 Executing: {cmd_str}") + if cwd and cwd != os.getcwd(): + print(f"📁 Directory: {cwd}") + if env and "USE_BAZEL_VERSION" in env: + print(f"🔧 Bazel Version: {env['USE_BAZEL_VERSION']}") + + res = subprocess.run(cmd, cwd=cwd, env=env, shell=shell) + if res.returncode != 0: + print( + f"\n❌ Command failed with return code {res.returncode}: {cmd_str}", + file=sys.stderr, + ) + return False + return True + + +def resolve_bazel_version(task_bazel): + if not task_bazel or task_bazel.startswith("${{"): + return None + return task_bazel + + +def execute_ci_job(job_key, task, repo_root): + job_name = task.get("name", job_key) + print(f"\n{'=' * 80}\n🎯 Replicating CI Job: {job_key} ('{job_name}')\n{'=' * 80}") + + # Setup working directory + cwd = repo_root + if "working_directory" in task: + cwd = os.path.join(repo_root, task["working_directory"]) + if not os.path.exists(cwd): + print( + f"❌ Error: working_directory '{task['working_directory']}' does not exist at '{cwd}'", + file=sys.stderr, + ) + return False + + # Setup environment + env = os.environ.copy() + bzl_version = resolve_bazel_version(task.get("bazel")) + if bzl_version: + env["USE_BAZEL_VERSION"] = bzl_version + + # Execute pre-commands + is_windows = sys.platform.startswith("win") + pre_cmds = task.get("batch_commands" if is_windows else "shell_commands", []) + for pre_cmd in pre_cmds: + if not run_cmd(pre_cmd, cwd=cwd, env=env, shell=True): + return False + + # Execute Build Targets + build_targets = [t for t in task.get("build_targets", []) if t != "--"] + if build_targets: + build_flags = task.get("build_flags", []) + cmd = ["bazel", "build"] + build_flags + ["--"] + build_targets + if not run_cmd(cmd, cwd=cwd, env=env): + return False + + # Execute Test Targets + test_targets = [t for t in task.get("test_targets", []) if t != "--"] + if test_targets: + test_flags = task.get("test_flags", []) + if "--build_tests_only" not in test_flags: + test_flags = ["--build_tests_only"] + test_flags + cmd = ["bazel", "test"] + test_flags + ["--"] + test_targets + if not run_cmd(cmd, cwd=cwd, env=env): + return False + + # Execute Coverage Targets + coverage_targets = [t for t in task.get("coverage_targets", []) if t != "--"] + if coverage_targets: + coverage_flags = task.get("test_flags", []) + cmd = ["bazel", "coverage"] + coverage_flags + ["--"] + coverage_targets + if not run_cmd(cmd, cwd=cwd, env=env): + return False + + print(f"\n🎉 Successfully replicated CI Job: {job_key}") + return True + + +def main(): + args = parse_args() + + repo_root = os.path.abspath(os.path.dirname(__file__)) + presubmit_path = os.path.join(repo_root, ".bazelci/presubmit.yml") + if not os.path.exists(presubmit_path): + print( + f"❌ Error: Presubmit file not found at '{presubmit_path}'", + file=sys.stderr, + ) + sys.exit(1) + + with open(presubmit_path) as f: + presubmit = yaml.safe_load(f) + + tasks = presubmit.get("tasks", {}) + if not tasks: + print( + f"❌ Error: No tasks found in '{presubmit_path}'", + file=sys.stderr, + ) + sys.exit(1) + + # If no job specified, print available jobs and exit + if not args.job: + print("❌ Error: No CI job specified. Provide a job key.\n", file=sys.stderr) + print("📋 Available CI Job Keys:", file=sys.stderr) + for key in sorted(tasks.keys()): + name = tasks[key].get("name", key) + print(f" • {key} ({name})", file=sys.stderr) + sys.exit(1) + + # Match by key or by name + job_key = None + if args.job in tasks: + job_key = args.job + else: + for key, config in tasks.items(): + if config.get("name") == args.job: + job_key = key + break + + if not job_key: + print( + f"❌ Error: CI job '{args.job}' not found in '{presubmit_path}'\n", + file=sys.stderr, + ) + print("📋 Available CI Job Keys:", file=sys.stderr) + for key in sorted(tasks.keys()): + name = tasks[key].get("name", key) + print(f" • {key} ({name})", file=sys.stderr) + sys.exit(1) + + success = execute_ci_job(job_key, tasks[job_key], repo_root) + sys.exit(0 if success else 1) + + +if __name__ == "__main__": + main() diff --git a/sphinxdocs/.bazelrc b/sphinxdocs/.bazelrc index acff835394..65c996c678 100644 --- a/sphinxdocs/.bazelrc +++ b/sphinxdocs/.bazelrc @@ -16,6 +16,8 @@ build --enable_runfiles # Local disk cache greatly speeds up builds if the regular cache is lost common --disk_cache=~/.cache/bazel/bazel-disk-cache common --experimental_downloader_config=downloader_config.cfg +common --http_timeout_scaling=10.0 +common --experimental_repository_downloader_retries=10 common --incompatible_python_disallow_native_rules common --incompatible_no_implicit_file_export diff --git a/sphinxdocs/downloader_config.cfg b/sphinxdocs/downloader_config.cfg index a978fb89b9..3fa6264eda 100644 --- a/sphinxdocs/downloader_config.cfg +++ b/sphinxdocs/downloader_config.cfg @@ -5,6 +5,9 @@ rewrite ^github\.com/bazelbuild/bazel-skylib/(.*) github.com/bazelbuild/bazel-sk rewrite ^github\.com/bazelbuild/platforms/(.*) github.com/bazelbuild/platforms/$1 rewrite ^github\.com/bazelbuild/rules_kotlin/(.*) github.com/bazelbuild/rules_kotlin/$1 rewrite ^github\.com/bazelbuild/rules_shell/(.*) github.com/bazelbuild/rules_shell/$1 +rewrite ^github\.com/bazelbuild/rules_java/(.*) github.com/bazelbuild/rules_java/$1 +rewrite ^github\.com/bazelbuild/stardoc/(.*) github.com/bazelbuild/stardoc/$1 + # Fall back to mirror (secondary) # Tracking upstream BCR mirror addition: https://github.com/bazelbuild/platforms/issues/139 @@ -14,3 +17,5 @@ rewrite ^github\.com/bazelbuild/bazel-skylib/(.*) mirror.bazel.build/github.com/ rewrite ^github\.com/bazelbuild/platforms/(.*) mirror.bazel.build/github.com/bazelbuild/platforms/$1 rewrite ^github\.com/bazelbuild/rules_kotlin/(.*) mirror.bazel.build/github.com/bazelbuild/rules_kotlin/$1 rewrite ^github\.com/bazelbuild/rules_shell/(.*) mirror.bazel.build/github.com/bazelbuild/rules_shell/$1 +rewrite ^github\.com/bazelbuild/rules_java/(.*) mirror.bazel.build/github.com/bazelbuild/rules_java/$1 +rewrite ^github\.com/bazelbuild/stardoc/(.*) mirror.bazel.build/github.com/bazelbuild/stardoc/$1 diff --git a/tests/docs/BUILD.bazel b/tests/docs/BUILD.bazel new file mode 100644 index 0000000000..bdc99a290c --- /dev/null +++ b/tests/docs/BUILD.bazel @@ -0,0 +1,10 @@ +load("@bazel_skylib//rules:build_test.bzl", "build_test") + +licenses(["notice"]) + +build_test( + name = "docs_build_test", + targets = [ + "//docs:docs", + ], +) diff --git a/tests/get_release_info/get_release_info_tests.bzl b/tests/get_release_info/get_release_info_tests.bzl index c810489c92..0b1b60adc3 100644 --- a/tests/get_release_info/get_release_info_tests.bzl +++ b/tests/get_release_info/get_release_info_tests.bzl @@ -63,6 +63,7 @@ def _test_astral_mirror(env): expected_urls = [ "https://github.com/astral-sh/python-build-standalone/releases/download/20230826/cpython-3.11.5+20230826-x86_64-unknown-linux-gnu-install_only.tar.gz", "https://releases.astral.sh/github/python-build-standalone/releases/download/20230826/cpython-3.11.5+20230826-x86_64-unknown-linux-gnu-install_only.tar.gz", + "https://github.com/indygreg/python-build-standalone/releases/download/20230826/cpython-3.11.5+20230826-x86_64-unknown-linux-gnu-install_only.tar.gz", ] _, urls, _, _, _ = get_release_info( @@ -95,7 +96,7 @@ def _test_astral_mirror_legacy(env): _, urls, _, _, _ = get_release_info( platform = "x86_64-unknown-linux-gnu", python_version = "3.11.5", - base_url = "https://github.com/indygreg/python-build-standalone/releases/download", + base_urls = ["https://github.com/indygreg/python-build-standalone/releases/download"], tool_versions = tool_versions, ) diff --git a/tests/integration/bzlmod_lockfile/MODULE.bazel.lock b/tests/integration/bzlmod_lockfile/MODULE.bazel.lock index d21fec2d7d..2a0bc7d76b 100644 --- a/tests/integration/bzlmod_lockfile/MODULE.bazel.lock +++ b/tests/integration/bzlmod_lockfile/MODULE.bazel.lock @@ -250,7 +250,7 @@ }, "@@rules_python+//python/uv:uv.bzl%uv": { "general": { - "bzlTransitiveDigest": "Z5ZPR9z4PkJRXSyJ4KQEqM4kwiqWBCn8Ajzxy9YlS/g=", + "bzlTransitiveDigest": "46RcxJnhOapMeaxdcMm3RmVdNp1nPCewOOXoZyIbQ20=", "usagesDigest": "6yXGw7XDyXjOfqBL0SBu1YBEMMYPQzCE3jTzUCkxPgg=", "recordedInputs": [ "REPO_MAPPING:rules_python+,bazel_tools bazel_tools", diff --git a/tests/python/python_tests.bzl b/tests/python/python_tests.bzl index cd7383942c..cbef5637fd 100644 --- a/tests/python/python_tests.bzl +++ b/tests/python/python_tests.bzl @@ -19,7 +19,6 @@ load("@rules_testing//lib:test_suite.bzl", "test_suite") load("//python/private:bzlmod_enabled.bzl", "BZLMOD_ENABLED") # buildifier: disable=bzl-visibility load("//python/private:python.bzl", "parse_modules") # buildifier: disable=bzl-visibility load("//python/private:repo_utils.bzl", "repo_utils") # buildifier: disable=bzl-visibility -load("//tests/support/mocks:mocks.bzl", "mocks") load("//tests/support/mocks:python_ext.bzl", "python_ext") _tests = [] @@ -36,7 +35,7 @@ def _rules_python_module(is_root = False): def _test_default_from_rules_python_when_rules_python_is_root(env): """Verify that rules_python (as root module) default is applied.""" py = parse_modules( - module_ctx = mocks.mctx( + module_ctx = python_ext.mctx( _rules_python_module(is_root = True), ), logger = repo_utils.logger(verbosity_level = 0, name = "python"), @@ -48,7 +47,7 @@ def _test_default_from_rules_python_when_rules_python_is_root(env): env.expect.that_dict(py.config.minor_mapping).contains_exactly(MINOR_MAPPING) env.expect.that_collection(py.config.kwargs).has_size(0) env.expect.that_collection(py.config.default.keys()).contains_exactly([ - "base_url", + "base_urls", "tool_versions", "platforms", ]) @@ -66,7 +65,7 @@ _tests.append(_test_default_from_rules_python_when_rules_python_is_root) def _test_default_from_rules_python_when_rules_python_is_not_root(env): """Verify that rules_python default applies when rules_python is not the root module.""" py = parse_modules( - module_ctx = mocks.mctx( + module_ctx = python_ext.mctx( _rules_python_module(), ), logger = repo_utils.logger(verbosity_level = 0, name = "python"), @@ -85,7 +84,7 @@ _tests.append(_test_default_from_rules_python_when_rules_python_is_not_root) def _test_default_with_patch_version(env): py = parse_modules( - module_ctx = mocks.mctx( + module_ctx = python_ext.mctx( modules = [ python_ext.module( name = "alpha", @@ -111,7 +110,7 @@ _tests.append(_test_default_with_patch_version) def _test_toolchain_ordering(env): py = parse_modules( - module_ctx = mocks.mctx( + module_ctx = python_ext.mctx( python_ext.module( name = "my_module", is_root = True, @@ -161,7 +160,7 @@ _tests.append(_test_toolchain_ordering) def _test_default_from_defaults(env): py = parse_modules( - module_ctx = mocks.mctx( + module_ctx = python_ext.mctx( python_ext.module( name = "my_root_module", defaults = [python_ext.defaults(python_version = "3.11")], @@ -192,7 +191,7 @@ _tests.append(_test_default_from_defaults) def _test_default_from_defaults_env(env): py = parse_modules( - module_ctx = mocks.mctx( + module_ctx = python_ext.mctx( python_ext.module( name = "my_root_module", defaults = [ @@ -229,7 +228,7 @@ _tests.append(_test_default_from_defaults_env) def _test_default_from_defaults_file(env): py = parse_modules( - module_ctx = mocks.mctx( + module_ctx = python_ext.mctx( python_ext.module( name = "my_root_module", defaults = [ @@ -265,7 +264,7 @@ _tests.append(_test_default_from_defaults_file) def _test_default_from_single_toolchain(env): py = parse_modules( - module_ctx = mocks.mctx( + module_ctx = python_ext.mctx( python_ext.module( name = "my_root_module", is_root = True, @@ -281,7 +280,7 @@ _tests.append(_test_default_from_single_toolchain) def _test_defaults_overrides_single_toolchain(env): py = parse_modules( - module_ctx = mocks.mctx( + module_ctx = python_ext.mctx( python_ext.module( name = "my_root_module", defaults = [ @@ -301,7 +300,7 @@ _tests.append(_test_defaults_overrides_single_toolchain) def _test_defaults_overrides_toolchains_setting_is_default(env): py = parse_modules( - module_ctx = mocks.mctx( + module_ctx = python_ext.mctx( python_ext.module( name = "my_root_module", defaults = [python_ext.defaults(python_version = "3.13")], @@ -324,7 +323,7 @@ _tests.append(_test_defaults_overrides_toolchains_setting_is_default) def _test_first_occurance_of_the_toolchain_wins(env): py = parse_modules( - module_ctx = mocks.mctx( + module_ctx = python_ext.mctx( modules = [ python_ext.module( name = "my_module", @@ -380,7 +379,7 @@ _tests.append(_test_first_occurance_of_the_toolchain_wins) def _test_auth_overrides(env): py = parse_modules( - module_ctx = mocks.mctx( + module_ctx = python_ext.mctx( python_ext.module( name = "my_module", is_root = True, @@ -422,7 +421,7 @@ _tests.append(_test_auth_overrides) def _test_add_target_settings(env): py = parse_modules( - module_ctx = mocks.mctx( + module_ctx = python_ext.mctx( python_ext.module( name = "my_module", is_root = True, @@ -448,7 +447,7 @@ _tests.append(_test_add_target_settings) def _test_add_new_version(env): py = parse_modules( - module_ctx = mocks.mctx( + module_ctx = python_ext.mctx( python_ext.module( name = "my_module", is_root = True, @@ -460,7 +459,7 @@ def _test_add_new_version(env): "3.13.1", "3.13.99", ], - base_url = "", + base_urls = [], minor_mapping = { "3.13": "3.13.99", }, @@ -534,7 +533,7 @@ _tests.append(_test_add_new_version) def _test_register_all_versions(env): py = parse_modules( - module_ctx = mocks.mctx( + module_ctx = python_ext.mctx( python_ext.module( name = "my_module", is_root = True, @@ -546,7 +545,7 @@ def _test_register_all_versions(env): "3.13.1", "3.13.99", ], - base_url = "", + base_urls = [], register_all_versions = True, ), ], @@ -605,7 +604,7 @@ _tests.append(_test_register_all_versions) def _test_ignore_unsupported_versions(env): py = parse_modules( - module_ctx = mocks.mctx( + module_ctx = python_ext.mctx( python_ext.module( name = "my_module", is_root = True, @@ -616,7 +615,7 @@ def _test_ignore_unsupported_versions(env): "3.13.0", "3.13.1", ], - base_url = "", + base_urls = [], minor_mapping = { "3.12": "3.12.4", "3.13": "3.13.1", @@ -683,14 +682,14 @@ _tests.append(_test_ignore_unsupported_versions) def _test_add_patches(env): py = parse_modules( - module_ctx = mocks.mctx( + module_ctx = python_ext.mctx( python_ext.module( name = "my_module", is_root = True, override = [ python_ext.override( available_python_versions = ["3.13.0"], - base_url = "", + base_urls = [], minor_mapping = { "3.13": "3.13.0", }, @@ -762,13 +761,13 @@ _tests.append(_test_add_patches) def _test_fail_two_overrides(env): errors = [] parse_modules( - module_ctx = mocks.mctx( + module_ctx = python_ext.mctx( python_ext.module( name = "my_module", is_root = True, override = [ - python_ext.override(base_url = "foo"), - python_ext.override(base_url = "bar"), + python_ext.override(base_urls = ["foo"]), + python_ext.override(base_urls = ["bar"]), ], toolchain = [python_ext.toolchain(python_version = "3.13")], ), @@ -800,7 +799,7 @@ def _test_single_version_override_errors(env): ]: errors = [] parse_modules( - module_ctx = mocks.mctx( + module_ctx = python_ext.mctx( python_ext.module( name = "my_module", is_root = True, @@ -853,7 +852,7 @@ def _test_single_version_platform_override_errors(env): ]: errors = [] parse_modules( - module_ctx = mocks.mctx( + module_ctx = python_ext.mctx( python_ext.module( name = "my_module", is_root = True, diff --git a/tests/python_bzlmod_ext/BUILD.bazel b/tests/python_bzlmod_ext/BUILD.bazel index 0add4e7690..e266c01b9c 100644 --- a/tests/python_bzlmod_ext/BUILD.bazel +++ b/tests/python_bzlmod_ext/BUILD.bazel @@ -2,6 +2,6 @@ load(":test_helpers.bzl", "register_python_bzlmod_ext_tests") register_python_bzlmod_ext_tests( name = "python_bzlmod_ext_tests", - parse_sha_manifest_name = "parse_sha_manifest_tests", + parse_runtime_manifest_name = "parse_runtime_manifest_tests", runtime_manifests_name = "runtime_manifests_tests", ) diff --git a/tests/python_bzlmod_ext/parse_sha_manifest_tests.bzl b/tests/python_bzlmod_ext/parse_runtime_manifest_tests.bzl similarity index 91% rename from tests/python_bzlmod_ext/parse_sha_manifest_tests.bzl rename to tests/python_bzlmod_ext/parse_runtime_manifest_tests.bzl index 4ddcdcc46e..4493576147 100644 --- a/tests/python_bzlmod_ext/parse_sha_manifest_tests.bzl +++ b/tests/python_bzlmod_ext/parse_runtime_manifest_tests.bzl @@ -4,7 +4,7 @@ load("@bazel_skylib//lib:structs.bzl", "structs") load("@rules_testing//lib:analysis_test.bzl", "analysis_test") load("@rules_testing//lib:test_suite.bzl", "test_suite") load("@rules_testing//lib:util.bzl", rt_util = "util") -load("//python/private:pbs_manifest.bzl", "parse_filename", "parse_sha_manifest") # buildifier: disable=bzl-visibility +load("//python/private:pbs_manifest.bzl", "parse_filename", "parse_runtime_manifest") # buildifier: disable=bzl-visibility _tests = [] @@ -32,8 +32,8 @@ def _test_parse_filename_baseline_impl(env, target): env.expect.that_dict(parsed1).contains_exactly({ "arch": "x86_64", "archive_flavor": "install_only", + "build_flavor": "", "build_version": "20260414", - "flavor": "", "freethreaded": False, "libc": "gnu", "location": "cpython-3.11.15+20260414-x86_64-unknown-linux-gnu-install_only.tar.gz", @@ -48,8 +48,8 @@ def _test_parse_filename_baseline_impl(env, target): env.expect.that_dict(parsed2).contains_exactly({ "arch": "x86_64", "archive_flavor": "full", + "build_flavor": "lto", "build_version": "20260414", - "flavor": "lto", "freethreaded": False, "libc": "musl", "location": "cpython-3.10.20+20260414-x86_64_v2-unknown-linux-musl-lto-full.tar.zst", @@ -64,8 +64,8 @@ def _test_parse_filename_baseline_impl(env, target): env.expect.that_dict(parsed3).contains_exactly({ "arch": "aarch64", "archive_flavor": "full", + "build_flavor": "pgo+lto", "build_version": "20260414", - "flavor": "pgo+lto", "freethreaded": True, "libc": "", "location": "cpython-3.13.13+20260414-aarch64-apple-darwin-freethreaded+pgo+lto-full.tar.zst", @@ -84,8 +84,8 @@ def _test_parse_filename_baseline_impl(env, target): env.expect.that_dict(parsed5).contains_exactly({ "arch": "x86_64", "archive_flavor": "install_only", + "build_flavor": "", "build_version": "20260414", - "flavor": "", "freethreaded": False, "libc": "gnu", "location": "https://github.com/astral-sh/python-build-standalone/releases/download/20260414/cpython-3.11.15+20260414-x86_64-unknown-linux-gnu-install_only.tar.gz", @@ -97,7 +97,7 @@ def _test_parse_filename_baseline_impl(env, target): _tests.append(_test_parse_filename_baseline) -def _test_parse_sha_manifest(name): +def _test_parse_runtime_manifest(name): """Sets up the manifest file parsing test. Args: @@ -110,10 +110,10 @@ def _test_parse_sha_manifest(name): analysis_test( name = name, target = name + "_subject", - impl = _test_parse_sha_manifest_impl, + impl = _test_parse_runtime_manifest_impl, ) -def _test_parse_sha_manifest_impl(env, target): +def _test_parse_runtime_manifest_impl(env, target): _ = target # @unused content = """ 8b14030dd3af9ea7f7c51b4c90feb04afd8a8f45435727e67b875270bd08f3bc cpython-3.11.15+20260414-x86_64-unknown-linux-gnu-install_only.tar.gz @@ -121,14 +121,14 @@ a57ffd435652092d16b30e783f9826c55e9c64b0f0a72cbae0a9f39e663137fb cpython-3 ce18fdfd47c66830a40ea9b9e314a14b1636bbfd684501bc5ca1fc6d55a7933f https://example.com/cpython-3.10.20+20260414-x86_64_v2-unknown-linux-musl-lto-full.tar.zst 1111111111111111111111111111111111111111111111111111111111111111 cpython-3.13.13+20260414-aarch64-apple-darwin-freethreaded+pgo+lto-full.tar.zst """ - parsed = parse_sha_manifest(content) + parsed = parse_runtime_manifest(content) env.expect.that_collection(parsed).has_size(4) env.expect.that_dict(structs.to_dict(parsed[0])).contains_exactly({ "arch": "x86_64", "archive_flavor": "install_only", + "build_flavor": "", "build_version": "20260414", - "flavor": "", "freethreaded": False, "libc": "gnu", "location": "cpython-3.11.15+20260414-x86_64-unknown-linux-gnu-install_only.tar.gz", @@ -142,8 +142,8 @@ ce18fdfd47c66830a40ea9b9e314a14b1636bbfd684501bc5ca1fc6d55a7933f https://exampl env.expect.that_dict(structs.to_dict(parsed[2])).contains_exactly({ "arch": "x86_64", "archive_flavor": "full", + "build_flavor": "lto", "build_version": "20260414", - "flavor": "lto", "freethreaded": False, "libc": "musl", "location": "https://example.com/cpython-3.10.20+20260414-x86_64_v2-unknown-linux-musl-lto-full.tar.zst", @@ -157,8 +157,8 @@ ce18fdfd47c66830a40ea9b9e314a14b1636bbfd684501bc5ca1fc6d55a7933f https://exampl env.expect.that_dict(structs.to_dict(parsed[3])).contains_exactly({ "arch": "aarch64", "archive_flavor": "full", + "build_flavor": "pgo+lto", "build_version": "20260414", - "flavor": "pgo+lto", "freethreaded": True, "libc": "", "location": "cpython-3.13.13+20260414-aarch64-apple-darwin-freethreaded+pgo+lto-full.tar.zst", @@ -169,9 +169,9 @@ ce18fdfd47c66830a40ea9b9e314a14b1636bbfd684501bc5ca1fc6d55a7933f https://exampl "vendor": "apple", }) -_tests.append(_test_parse_sha_manifest) +_tests.append(_test_parse_runtime_manifest) -def parse_sha_manifest_test_suite(name): +def parse_runtime_manifest_test_suite(name): """Defines the test suite for manifest parsing. Args: diff --git a/tests/python_bzlmod_ext/runtime_manifests_tests.bzl b/tests/python_bzlmod_ext/runtime_manifests_tests.bzl index c46cab1958..6de0ff93c2 100644 --- a/tests/python_bzlmod_ext/runtime_manifests_tests.bzl +++ b/tests/python_bzlmod_ext/runtime_manifests_tests.bzl @@ -5,7 +5,6 @@ load("@rules_testing//lib:test_suite.bzl", "test_suite") load("@rules_testing//lib:util.bzl", rt_util = "util") load("//python/private:python.bzl", "parse_modules") # buildifier: disable=bzl-visibility load("//python/private:repo_utils.bzl", "repo_utils") # buildifier: disable=bzl-visibility -load("//tests/support/mocks:mocks.bzl", "mocks") # buildifier: disable=bzl-visibility load("//tests/support/mocks:python_ext.bzl", "python_ext") # buildifier: disable=bzl-visibility _tests = [] @@ -52,7 +51,7 @@ def _test_dynamic_manifest_toolchains_impl(env, target): ) # Pre-populate mock_files directly to bypass download output struct key mismatch in mock read lookups. - mock_mctx = mocks.mctx( + mock_mctx = python_ext.mctx( modules = [root_module], mock_files = { "runtime_manifest": """ @@ -110,7 +109,7 @@ def _test_dynamic_manifest_files_impl(env, target): add_runtime_manifest_files = [ Label("//:SHA256SUMS"), ], - base_url = "https://example.com/dl", + base_urls = ["https://example.com/dl"], register_all_versions = True, ), ], @@ -121,7 +120,7 @@ def _test_dynamic_manifest_files_impl(env, target): ], ) - mock_mctx = mocks.mctx( + mock_mctx = python_ext.mctx( modules = [root_module], mock_files = { str(Label("//:SHA256SUMS")): """ diff --git a/tests/python_bzlmod_ext/test_helpers.bzl b/tests/python_bzlmod_ext/test_helpers.bzl index f6c177650a..78ad57c110 100644 --- a/tests/python_bzlmod_ext/test_helpers.bzl +++ b/tests/python_bzlmod_ext/test_helpers.bzl @@ -15,23 +15,23 @@ """Helpers to conditionally register tests depending on Bzlmod enablement.""" load("//python/private:bzlmod_enabled.bzl", "BZLMOD_ENABLED") # buildifier: disable=bzl-visibility -load(":parse_sha_manifest_tests.bzl", "parse_sha_manifest_test_suite") +load(":parse_runtime_manifest_tests.bzl", "parse_runtime_manifest_test_suite") load(":runtime_manifests_tests.bzl", "runtime_manifests_test_suite") -def register_python_bzlmod_ext_tests(name, parse_sha_manifest_name, runtime_manifests_name): +def register_python_bzlmod_ext_tests(name, parse_runtime_manifest_name, runtime_manifests_name): """Registers the Bzlmod extension tests if Bzlmod is enabled, otherwise defines empty test_suites. Args: name: The name of the master test_suite target. - parse_sha_manifest_name: The name of the parse_sha_manifest test target. + parse_runtime_manifest_name: The name of the parse_runtime_manifest test target. runtime_manifests_name: The name of the runtime_manifests test target. """ if BZLMOD_ENABLED: - parse_sha_manifest_test_suite(name = parse_sha_manifest_name) + parse_runtime_manifest_test_suite(name = parse_runtime_manifest_name) runtime_manifests_test_suite(name = runtime_manifests_name) else: native.test_suite( - name = parse_sha_manifest_name, + name = parse_runtime_manifest_name, tests = [], ) native.test_suite( @@ -42,7 +42,7 @@ def register_python_bzlmod_ext_tests(name, parse_sha_manifest_name, runtime_mani native.test_suite( name = name, tests = [ - parse_sha_manifest_name, + parse_runtime_manifest_name, runtime_manifests_name, ], ) diff --git a/tests/support/mocks/mocks.bzl b/tests/support/mocks/mocks.bzl index 48f8f95830..84d39fa6e2 100644 --- a/tests/support/mocks/mocks.bzl +++ b/tests/support/mocks/mocks.bzl @@ -105,9 +105,12 @@ def _module_new(name, *, is_root = False, **tags): def _mctx_read(self, x, watch = None): _ = watch # @unused path_str = x._path if hasattr(x, "_path") else str(x) - if path_str not in self.mock_files: - fail("File not found in mock_files: " + path_str) - return self.mock_files[path_str] + if path_str in self.mock_files: + return self.mock_files[path_str] + for k, v in self.mock_files.items(): + if k in path_str or k in path_str.replace(":", "/"): + return v + fail("File not found in mock_files: " + path_str) def _mctx_path(self, x): return _path_new(str(x), self.mock_files) diff --git a/tests/support/mocks/python_ext.bzl b/tests/support/mocks/python_ext.bzl index f7b5b0ee02..dc3ac41f8d 100644 --- a/tests/support/mocks/python_ext.bzl +++ b/tests/support/mocks/python_ext.bzl @@ -30,7 +30,7 @@ def _override(**kwargs): "add_runtime_manifest_urls": [], "add_target_settings": [], "available_python_versions": [], - "base_url": "https://github.com/astral-sh/python-build-standalone/releases/download", + "base_urls": ["https://github.com/astral-sh/python-build-standalone/releases/download"], "ignore_root_user_error": True, "minor_mapping": {}, "register_all_versions": False, @@ -93,8 +93,45 @@ def _toolchain(**kwargs): attrs.update(kwargs) return mocks.tag(**attrs) +_DEFAULT_RUNTIMES_MANIFEST = """ +87275619c2706affa4d1090d2ca3dad354b6d69f8b85dbfafe38785870751b9a 20251031/cpython-3.9.25+20251031-x86_64-unknown-linux-gnu-install_only.tar.gz +6112d46355857680b81849764a6cf9f38cc4cd0d1cf29d432bc12fe5aeedf9d0 20260414/cpython-3.10.20+20260414-x86_64-unknown-linux-gnu-install_only.tar.gz +1111111111111111111111111111111111111111111111111111111111111111 20241016/cpython-3.10.15+20241016-x86_64-unknown-linux-gnu-install_only.tar.gz +2222222222222222222222222222222222222222222222222222222222222222 20240224/cpython-3.10.13+20240224-x86_64-unknown-linux-gnu-install_only.tar.gz +0000000000000000000000000000000000000000000000000000000000000000 20260414/cpython-3.11.15+20260414-x86_64-unknown-linux-gnu-install_only.tar.gz +3333333333333333333333333333333333333333333333333333333333333333 20241016/cpython-3.11.10+20241016-x86_64-unknown-linux-gnu-install_only.tar.gz +4444444444444444444444444444444444444444444444444444444444444444 20230116/cpython-3.11.2+20230116-x86_64-unknown-linux-gnu-install_only.tar.gz +5555555555555555555555555555555555555555555555555555555555555555 20230116/cpython-3.11.1+20230116-x86_64-unknown-linux-gnu-install_only.tar.gz +6666666666666666666666666666666666666666666666666666666666666666 20260414/cpython-3.12.13+20260414-x86_64-unknown-linux-gnu-install_only.tar.gz +7777777777777777777777777777777777777777777777777777777777777777 20240726/cpython-3.12.4+20240726-x86_64-unknown-linux-gnu-install_only.tar.gz +8888888888888888888888888888888888888888888888888888888888888888 20260414/cpython-3.13.13+20260414-x86_64-unknown-linux-gnu-install_only.tar.gz +9999999999999999999999999999999999999999999999999999999999999999 20241016/cpython-3.13.0+20241016-x86_64-unknown-linux-gnu-install_only.tar.gz +aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa 20260414/cpython-3.14.4+20260414-x86_64-unknown-linux-gnu-install_only.tar.gz +bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb 20251031/cpython-3.14.0+20251031-x86_64-unknown-linux-gnu-install_only.tar.gz +cccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccc 20260414/cpython-3.15.0a8+20260414-x86_64-unknown-linux-gnu-install_only.tar.gz +dddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddd 20241205/cpython-3.13.1+20241205-x86_64-unknown-linux-gnu-install_only.tar.gz +""" + +def _mctx(*args, **kwargs): + """Creates a mock module_ctx pre-populated with the default runtimes manifest. + + Args: + *args: Positional arguments to pass to mocks.mctx. + **kwargs: Keyword arguments to pass to mocks.mctx. + + Returns: + A mock module_ctx struct. + """ + mock_files = { + "python/private/runtimes_manifest.txt": _DEFAULT_RUNTIMES_MANIFEST, + } + mock_files.update(kwargs.pop("mock_files", {})) + kwargs["mock_files"] = mock_files + return mocks.mctx(*args, **kwargs) + python_ext = struct( defaults = _defaults, + mctx = _mctx, module = _module, override = _override, single_version_override = _single_version_override, diff --git a/tests/toolchains/BUILD.bazel b/tests/toolchains/BUILD.bazel index f32ab6f056..b336a06269 100644 --- a/tests/toolchains/BUILD.bazel +++ b/tests/toolchains/BUILD.bazel @@ -13,6 +13,7 @@ # limitations under the License. load("@bazel_skylib//rules:build_test.bzl", "build_test") +load("@bazel_skylib//rules:diff_test.bzl", "diff_test") load("//python/private:bzlmod_enabled.bzl", "BZLMOD_ENABLED") # buildifier: disable=bzl-visibility load("//tests/support:py_reconfig.bzl", "py_reconfig_test") load(":defs.bzl", "define_toolchain_tests") @@ -38,3 +39,18 @@ build_test( "@python_3_11//:python_headers", ], ) + +# Verify that runtimes_manifest_workspace.bzl exactly matches runtimes_manifest.txt +genrule( + name = "gen_expected_runtimes_manifest_workspace", + srcs = ["//python/private:runtimes_manifest.txt"], + outs = ["expected_runtimes_manifest_workspace.bzl"], + cmd = "$(execpath //python/private:sync_runtimes_manifest_workspace) $(location //python/private:runtimes_manifest.txt) $@", + tools = ["//python/private:sync_runtimes_manifest_workspace"], +) + +diff_test( + name = "runtimes_manifest_workspace_sync_test", + file1 = "//python/private:runtimes_manifest_workspace.bzl", + file2 = ":expected_runtimes_manifest_workspace.bzl", +) diff --git a/tests/toolchains/transitions/transitions_tests.bzl b/tests/toolchains/transitions/transitions_tests.bzl index 0cd79b373b..81ce1e68cc 100644 --- a/tests/toolchains/transitions/transitions_tests.bzl +++ b/tests/toolchains/transitions/transitions_tests.bzl @@ -14,11 +14,10 @@ "" -load("@pythons_hub//:versions.bzl", "DEFAULT_PYTHON_VERSION", "MINOR_MAPPING") +load("@pythons_hub//:versions.bzl", "DEFAULT_PYTHON_VERSION", "MINOR_MAPPING", "PYTHON_VERSIONS") load("@rules_testing//lib:analysis_test.bzl", "analysis_test") load("@rules_testing//lib:test_suite.bzl", "test_suite") load("@rules_testing//lib:util.bzl", rt_util = "util") -load("//python:versions.bzl", "TOOL_VERSIONS") load("//python/private:bzlmod_enabled.bzl", "BZLMOD_ENABLED") # buildifier: disable=bzl-visibility load("//python/private:common_labels.bzl", "labels") # buildifier: disable=bzl-visibility load("//python/private:full_version.bzl", "full_version") # buildifier: disable=bzl-visibility @@ -142,7 +141,7 @@ def _test_full_version(name): name = name, tests = { v.replace(".", "_"): (v, v) - for v in TOOL_VERSIONS + for v in PYTHON_VERSIONS }, ) diff --git a/tools/private/sync_downloader_configs.py b/tools/private/sync_downloader_configs.py new file mode 100755 index 0000000000..7e8527b989 --- /dev/null +++ b/tools/private/sync_downloader_configs.py @@ -0,0 +1,37 @@ +#!/usr/bin/env python3 + +"""Synchronizes downloader_config.cfg across subworkspaces.""" + +import sys +from pathlib import Path + + +def main(): + repo_root = Path(__file__).resolve().parent.parent.parent + canonical = repo_root / "downloader_config.cfg" + + subworkspaces = [ + repo_root / "gazelle" / "downloader_config.cfg", + repo_root / "sphinxdocs" / "downloader_config.cfg", + ] + + with open(canonical, "r", encoding="utf-8") as f: + canonical_content = f.read() + + changed = False + for sub in subworkspaces: + if sub.exists(): + with open(sub, "r", encoding="utf-8") as f: + old_content = f.read() + if old_content != canonical_content: + with open(sub, "w", encoding="utf-8") as f: + f.write(canonical_content) + print(f"Updated {sub.relative_to(repo_root)}") + changed = True + + if changed: + sys.exit(1) + + +if __name__ == "__main__": + main() From 44ec76f189dc433c8a0291c5d60e62fada605eb7 Mon Sep 17 00:00:00 2001 From: Udaya Prakash Date: Tue, 16 Jun 2026 03:06:05 +0200 Subject: [PATCH 761/922] docs: update changelog for 2.0.3 release (#3820) This PR updates the changelog on `main` to document the `2.0.3` release and moves the cherry-picked fix out of the `Unreleased` section. Companion to the release PR #3819 on the `release/2.0` branch. --------- Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> --- CHANGELOG.md | 17 ++++++++++++----- 1 file changed, 12 insertions(+), 5 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 0fa832e8fc..f431cd995e 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -106,11 +106,6 @@ END_UNRELEASED_TEMPLATE * Fix the forwarding of `target_compatible_with` from `compile_pip_requirements` towards the underlying `*.update` target. ([#3787](https://github.com/bazel-contrib/rules_python/pull/3787)) -* (pypi) Assume that all of the packages are available on a particular hub if - there is only a single PyPI compatible index to be used. This saves us an expensive - PyPI download and supports PyPI mirror implementations that do not support the root - index functionality. Fixes - ([#3769](https://github.com/bazel-contrib/rules_python/pull/3769)). * (uv) allow user overwrite the build environment using `--action_env` to allow setting authentication for the index URL. ([#3405](https://github.com/bazel-contrib/rules_python/issues/3405)) @@ -139,6 +134,18 @@ END_UNRELEASED_TEMPLATE [#2054](https://github.com/bazel-contrib/rules_python/issues/2054). * (coverage) Add support for python 3.14 and bump `coverage.py` to 7.10.7. +{#v2-0-3} +## [2.0.3] - 2026-06-15 + +[2.0.3]: https://github.com/bazel-contrib/rules_python/releases/tag/2.0.3 + +{#v2-0-3-fixed} +### Fixed +* (pypi) Assume that all of the packages are available on a particular hub if + there is only a single PyPI compatible index to be used. This saves us an expensive + PyPI download and supports PyPI mirror implementations that do not support the root + index functionality. Fixes [#3769](https://github.com/bazel-contrib/rules_python/pull/3769). + {#v2-0-2} ## [2.0.2] - 2026-05-14 From 6ce7840252cfe2b4ba4484007d0c1c1650d1196d Mon Sep 17 00:00:00 2001 From: Pierre Gergondet Date: Tue, 16 Jun 2026 23:08:41 +0900 Subject: [PATCH 762/922] fix(coverage): handle nested coverage collection (#3823) We discovered that when a py_test invokes py_binary, coverage information collected by those binaries is lost and only the coverage of the test itself appears. It turns out that each binary writes to the same coverage file and only the last one is included in the report. This PR makes the lcov_path unique, ensuring the coverage is collected for every binary that might be launched in such a setup. I have setup https://github.com/gergondet-woven/repro-rules_python-coverage-issue to show the issue. In this repo, a py_binary is launched through a py_test and we would expect coverage information generated by the binary to surface in the coverage report but it doesn't with the current release: Before the patch: ``` # Generate coverage information bazel coverage --combined_report=lcov //... # Generate the report genhtml --ignore-errors category --branch-coverage --output ~/genhtml "$(bazel info output_path)/_coverage/_coverage_report.dat" # (snip) Overall coverage rate: lines......: 30.0% (3 of 10 lines) # Looking into the report, neither coverage for the library function # called by the binary nor the binary code itself is collected ``` After applying the patch: ``` Overall coverage rate: lines......: 100.0% (11 of 11 lines) ``` However, I am not sure how/if this can be properly tested within rules_python itself. --- CHANGELOG.md | 2 ++ python/private/stage2_bootstrap_template.py | 2 +- 2 files changed, 3 insertions(+), 1 deletion(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index f431cd995e..7efdd66312 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -114,6 +114,8 @@ END_UNRELEASED_TEMPLATE ([#1975](https://github.com/bazel-contrib/rules_python/issues/1975)) * (uv) automatically pass the `--project` parameter based on the source files. ([#3087](https://github.com/bazel-contrib/rules_python/issues/3087)) +* (coverage) handle nested coverage collection + ([#3823](https://github.com/bazel-contrib/rules_python/pull/3823)) {#v0-0-0-added} ### Added diff --git a/python/private/stage2_bootstrap_template.py b/python/private/stage2_bootstrap_template.py index c4326bc026..a66b6428b0 100644 --- a/python/private/stage2_bootstrap_template.py +++ b/python/private/stage2_bootstrap_template.py @@ -406,7 +406,7 @@ def _maybe_collect_coverage(enable): yield finally: cov.stop() - lcov_path = os.path.join(coverage_dir, "pylcov.dat") + lcov_path = os.path.join(coverage_dir, "pylcov_{}.dat".format(unique_id)) print_verbose_coverage("generating lcov from:", lcov_path) cov.lcov_report( outfile=lcov_path, From 102b5014c33397293a50cd82cce9108d6de9837d Mon Sep 17 00:00:00 2001 From: John Sun <79071267+jsun-splunk@users.noreply.github.com> Date: Wed, 17 Jun 2026 15:03:18 +1000 Subject: [PATCH 763/922] feat: expose interpreter files-to-run on PyRuntimeInfo (#3795) Rules that execute a py_runtime interpreter in an action need the interpreter executable together with its runfiles metadata. The existing PyRuntimeInfo fields identify the interpreter file and runtime files, but do not preserve the target's FilesToRunProvider for executable interpreter targets. Add PyRuntimeInfo.interpreter_files_to_run for runtimes created from an executable interpreter target, and validate that direct provider construction keeps the FilesToRunProvider executable aligned with the interpreter field. Direct file interpreters and platform runtimes continue to leave this field unset, preserving existing py_runtime behavior. Document the new public provider field and add focused analysis-test coverage for executable, file-only, platform, and invalid constructor cases. One example is [rules_pycross](https://github.com/jvolkman/rules_pycross) wheel building with an in-build executable Python runtime. pycross needs to run the selected Python interpreter in an action, but PyRuntimeInfo previously exposed only the interpreter File and runtime files, not the interpreter target's FilesToRunProvider. For executable in-build runtimes, the FilesToRunProvider is the Bazel-native handle that carries both the executable and the runfiles metadata needed to stage it as an action tool. Exposing it lets pycross consume the runtime interpreter directly for wheel-build actions instead of reconstructing or approximating the interpreter's runtime closure from separate fields. Co-authored-by: Richard Levasseur --- CHANGELOG.md | 2 + python/private/py_runtime_info.bzl | 24 +++ python/private/py_runtime_rule.bzl | 36 +++- tests/py_runtime/py_runtime_tests.bzl | 71 +++++++- .../py_runtime_info/py_runtime_info_tests.bzl | 159 ++++++++++++++++++ tests/support/py_runtime_info_subject.bzl | 12 ++ 6 files changed, 298 insertions(+), 6 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 7efdd66312..05daf8c85f 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -119,6 +119,8 @@ END_UNRELEASED_TEMPLATE {#v0-0-0-added} ### Added +* (toolchain) Added {obj}`PyRuntimeInfo.interpreter_files_to_run` so action + consumers can execute an in-build runtime interpreter with its runfiles. * (toolchains) Support dynamically fetching and registering Python runtimes from a python-build-standalone manifest file using `python.override(add_runtime_manifest_urls = ..., runtime_manifest_sha = ...)`. diff --git a/python/private/py_runtime_info.bzl b/python/private/py_runtime_info.bzl index d94f469fc0..64fef399e2 100644 --- a/python/private/py_runtime_info.bzl +++ b/python/private/py_runtime_info.bzl @@ -55,6 +55,7 @@ def _PyRuntimeInfo_init( interpreter_path = None, interpreter = None, files = None, + interpreter_files_to_run = None, coverage_tool = None, coverage_files = None, pyc_tag = None, @@ -74,6 +75,15 @@ def _PyRuntimeInfo_init( if interpreter_path and files != None: fail("cannot specify 'files' if 'interpreter_path' is given") + if interpreter_path and interpreter_files_to_run: + fail("cannot specify 'interpreter_files_to_run' if 'interpreter_path' is given") + + if interpreter_files_to_run: + if not interpreter_files_to_run.executable: + fail("'interpreter_files_to_run' must have an executable") + if interpreter_files_to_run.executable != interpreter: + fail("'interpreter_files_to_run.executable' must match 'interpreter'") + if (coverage_tool and not coverage_files) or (not coverage_tool and coverage_files): fail( "coverage_tool and coverage_files must both be set or neither must be set, " + @@ -112,6 +122,7 @@ def _PyRuntimeInfo_init( "files": files, "implementation_name": implementation_name, "interpreter": interpreter, + "interpreter_files_to_run": interpreter_files_to_run, "interpreter_path": interpreter_path, "interpreter_version_info": interpreter_version_info_struct_from_dict(interpreter_version_info), "pyc_tag": pyc_tag, @@ -239,6 +250,19 @@ The Python implementation name (`sys.implementation.name`) If this is an in-build runtime, this field is a `File` representing the interpreter. Otherwise, this is `None`. Note that an in-build runtime can use either a prebuilt, checked-in interpreter or an interpreter built from source. +""", + "interpreter_files_to_run": """ +:type: None | FilesToRunProvider + +The `FilesToRunProvider` for the interpreter target when this runtime was +created from an executable target. This includes the interpreter executable and +the runfiles metadata needed to use it as an action tool. Rules that execute the +interpreter in an action should use this field so Bazel can stage the +interpreter together with its runfiles. This is `None` for platform runtimes +using `interpreter_path` and for file-only interpreter targets. + +:::{versionadded} VERSION_NEXT_FEATURE +::: """, "interpreter_path": """ :type: str | None diff --git a/python/private/py_runtime_rule.bzl b/python/private/py_runtime_rule.bzl index dfc915f463..48637389bf 100644 --- a/python/private/py_runtime_rule.bzl +++ b/python/private/py_runtime_rule.bzl @@ -39,6 +39,7 @@ def _py_runtime_impl(ctx): runfiles = ctx.runfiles() hermetic = bool(interpreter) + interpreter_files_to_run = None if not hermetic: if runtime_files: fail("if 'interpreter_path' is given then 'files' must be empty") @@ -46,9 +47,35 @@ def _py_runtime_impl(ctx): fail("interpreter_path must be an absolute path") else: interpreter_di = interpreter[DefaultInfo] + interpreter_file = None - if interpreter_di.files_to_run and interpreter_di.files_to_run.executable: + if _is_singleton_depset(interpreter_di.files): + interpreter_file = interpreter_di.files.to_list()[0] + + is_executable_source_file = ( + interpreter_file and + interpreter_file.is_source and + interpreter_di.files_to_run and + interpreter_di.files_to_run.executable == interpreter_file + ) + + if is_executable_source_file: + # Source files Bazel treats as executable, e.g. direct file labels + # and filegroups: preserve historical runtime-file expansion, but + # do not expose a FilesToRunProvider. + interpreter = interpreter_file + runfiles = runfiles.merge(interpreter_di.default_runfiles) + + runtime_files = depset(transitive = [ + interpreter_di.files, + interpreter_di.default_runfiles.files, + runtime_files, + ]) + elif interpreter_di.files_to_run and interpreter_di.files_to_run.executable: + # Executable rule target: use the executable and preserve the full + # FilesToRunProvider so action consumers can stage its runfiles. interpreter = interpreter_di.files_to_run.executable + interpreter_files_to_run = interpreter_di.files_to_run runfiles = runfiles.merge(interpreter_di.default_runfiles) runtime_files = depset(transitive = [ @@ -56,8 +83,10 @@ def _py_runtime_impl(ctx): interpreter_di.default_runfiles.files, runtime_files, ]) - elif _is_singleton_depset(interpreter_di.files): - interpreter = interpreter_di.files.to_list()[0] + elif interpreter_file: + # Non-executable rule with exactly one output: preserve the + # historical file-only interpreter behavior. + interpreter = interpreter_file else: fail("interpreter must be an executable target or must produce exactly one file.") @@ -124,6 +153,7 @@ def _py_runtime_impl(ctx): py_runtime_info_kwargs.update(dict( implementation_name = ctx.attr.implementation_name, interpreter_version_info = interpreter_version_info, + interpreter_files_to_run = interpreter_files_to_run, pyc_tag = pyc_tag, stage2_bootstrap_template = ctx.file.stage2_bootstrap_template, zip_main_template = ctx.file.zip_main_template, diff --git a/tests/py_runtime/py_runtime_tests.bzl b/tests/py_runtime/py_runtime_tests.bzl index b8aa1f3fa6..ac80c8556d 100644 --- a/tests/py_runtime/py_runtime_tests.bzl +++ b/tests/py_runtime/py_runtime_tests.bzl @@ -42,6 +42,20 @@ _simple_binary = rule( executable = True, ) +def _source_file_wrapper_impl(ctx): + return [DefaultInfo( + files = depset([ctx.file.src]), + runfiles = ctx.runfiles(files = ctx.files.data), + )] + +_source_file_wrapper = rule( + implementation = _source_file_wrapper_impl, + attrs = { + "data": attr.label_list(allow_files = True), + "src": attr.label(allow_single_file = True, mandatory = True), + }, +) + def _test_bootstrap_template(name): rt_util.helper_target( py_runtime, @@ -195,11 +209,54 @@ def _test_in_build_interpreter(name): def _test_in_build_interpreter_impl(env, target): info = env.expect.that_target(target).provider(PyRuntimeInfo, factory = py_runtime_info_subject) info.python_version().equals("PY3") - info.files().contains_predicate(matching.file_basename_equals("file1.txt")) + info.files().contains_exactly([ + "{package}/fake_interpreter", + "{package}/file1.txt", + ]) info.interpreter().path().contains("fake_interpreter") + env.expect.that_bool(info.actual.interpreter_files_to_run == None).equals(True) _tests.append(_test_in_build_interpreter) +def _test_non_executable_source_file_interpreter_keeps_file_only_behavior(name): + rt_util.helper_target( + _source_file_wrapper, + name = name + "_wrapped_interpreter", + src = "fake_interpreter", + data = ["runfile.txt"], + ) + + rt_util.helper_target( + py_runtime, + name = name + "_subject", + interpreter = name + "_wrapped_interpreter", + python_version = "PY3", + files = ["file1.txt"], + ) + analysis_test( + name = name, + target = name + "_subject", + impl = _test_non_executable_source_file_interpreter_keeps_file_only_behavior_impl, + ) + +def _test_non_executable_source_file_interpreter_keeps_file_only_behavior_impl(env, target): + target = env.expect.that_target(target) + py_runtime_info = target.provider( + PyRuntimeInfo, + factory = py_runtime_info_subject, + ) + py_runtime_info.interpreter().short_path_equals("{package}/fake_interpreter") + env.expect.that_bool(py_runtime_info.actual.interpreter_files_to_run == None).equals(True) + py_runtime_info.files().contains_exactly([ + "{package}/file1.txt", + ]) + + target.default_outputs().contains_exactly([ + "{package}/file1.txt", + ]) + +_tests.append(_test_non_executable_source_file_interpreter_keeps_file_only_behavior) + def _test_interpreter_binary_with_multiple_outputs(name): rt_util.helper_target( _simple_binary, @@ -227,6 +284,9 @@ def _test_interpreter_binary_with_multiple_outputs_impl(env, target): factory = py_runtime_info_subject, ) py_runtime_info.interpreter().short_path_equals("{package}/{test_name}_built_interpreter") + py_runtime_info.interpreter_files_to_run().executable().short_path_equals( + "{package}/{test_name}_built_interpreter", + ) py_runtime_info.files().contains_exactly([ "{package}/extra_default_output.txt", "{package}/runfile.txt", @@ -272,6 +332,9 @@ def _test_interpreter_binary_with_single_output_and_runfiles_impl(env, target): factory = py_runtime_info_subject, ) py_runtime_info.interpreter().short_path_equals("{package}/{test_name}_built_interpreter") + py_runtime_info.interpreter_files_to_run().executable().short_path_equals( + "{package}/{test_name}_built_interpreter", + ) py_runtime_info.files().contains_exactly([ "{package}/runfile.txt", "{package}/{test_name}_built_interpreter", @@ -327,10 +390,12 @@ def _test_system_interpreter(name): ) def _test_system_interpreter_impl(env, target): - env.expect.that_target(target).provider( + info = env.expect.that_target(target).provider( PyRuntimeInfo, factory = py_runtime_info_subject, - ).interpreter_path().equals("/system/python") + ) + info.interpreter_path().equals("/system/python") + env.expect.that_bool(info.actual.interpreter_files_to_run == None).equals(True) _tests.append(_test_system_interpreter) diff --git a/tests/py_runtime_info/py_runtime_info_tests.bzl b/tests/py_runtime_info/py_runtime_info_tests.bzl index a44fb60c2f..6b9ecac605 100644 --- a/tests/py_runtime_info/py_runtime_info_tests.bzl +++ b/tests/py_runtime_info/py_runtime_info_tests.bzl @@ -15,7 +15,9 @@ load("@rules_testing//lib:analysis_test.bzl", "analysis_test") load("@rules_testing//lib:test_suite.bzl", "test_suite") +load("@rules_testing//lib:truth.bzl", "matching") load("//python:py_runtime_info.bzl", "PyRuntimeInfo") +load("//tests/support:py_runtime_info_subject.bzl", "py_runtime_info_subject") def _create_py_runtime_info_without_interpreter_version_info_impl(ctx): return [PyRuntimeInfo( @@ -35,6 +37,57 @@ _create_py_runtime_info_without_interpreter_version_info = rule( }, ) +def _simple_binary_impl(ctx): + executable = ctx.actions.declare_file(ctx.label.name) + ctx.actions.write(executable, "", is_executable = True) + return [DefaultInfo( + executable = executable, + files = depset([executable]), + )] + +_simple_binary = rule( + implementation = _simple_binary_impl, + executable = True, +) + +def _file_target_impl(ctx): + output = ctx.actions.declare_file(ctx.label.name + ".txt") + ctx.actions.write(output, "") + return [DefaultInfo(files = depset([output]))] + +_file_target = rule( + implementation = _file_target_impl, +) + +def _create_py_runtime_info_with_interpreter_files_to_run_impl(ctx): + files_to_run = ctx.attr.files_to_run[DefaultInfo].files_to_run + kwargs = dict( + bootstrap_template = ctx.file.bootstrap_template, + interpreter_files_to_run = files_to_run, + python_version = "PY3", + ) + if ctx.attr.use_interpreter_path: + kwargs["interpreter_path"] = "/python" + else: + kwargs["files"] = depset() + kwargs["interpreter"] = ctx.executable.interpreter + + return [PyRuntimeInfo(**kwargs)] + +_create_py_runtime_info_with_interpreter_files_to_run = rule( + implementation = _create_py_runtime_info_with_interpreter_files_to_run_impl, + attrs = { + "bootstrap_template": attr.label(allow_single_file = True, default = "bootstrap.txt"), + "files_to_run": attr.label(mandatory = True), + "interpreter": attr.label( + cfg = "target", + executable = True, + mandatory = True, + ), + "use_interpreter_path": attr.bool(), + }, +) + _tests = [] def _test_can_create_py_runtime_info_without_interpreter_version_info(name): @@ -53,6 +106,112 @@ def _test_can_create_py_runtime_info_without_interpreter_version_info_impl(env, _tests.append(_test_can_create_py_runtime_info_without_interpreter_version_info) +def _test_interpreter_files_to_run_with_interpreter(name): + _simple_binary( + name = name + "_interpreter", + ) + _create_py_runtime_info_with_interpreter_files_to_run( + name = name + "_subject", + files_to_run = name + "_interpreter", + interpreter = name + "_interpreter", + ) + analysis_test( + name = name, + target = name + "_subject", + impl = _test_interpreter_files_to_run_with_interpreter_impl, + ) + +def _test_interpreter_files_to_run_with_interpreter_impl(env, target): + info = env.expect.that_target(target).provider( + PyRuntimeInfo, + factory = py_runtime_info_subject, + ) + info.interpreter().short_path_equals("{package}/{test_name}_interpreter") + info.interpreter_files_to_run().executable().short_path_equals( + "{package}/{test_name}_interpreter", + ) + +_tests.append(_test_interpreter_files_to_run_with_interpreter) + +def _test_interpreter_files_to_run_disallows_interpreter_path(name): + _simple_binary( + name = name + "_interpreter", + ) + _create_py_runtime_info_with_interpreter_files_to_run( + name = name + "_subject", + files_to_run = name + "_interpreter", + interpreter = name + "_interpreter", + tags = ["manual"], + use_interpreter_path = True, + ) + analysis_test( + name = name, + target = name + "_subject", + impl = _test_interpreter_files_to_run_disallows_interpreter_path_impl, + expect_failure = True, + ) + +def _test_interpreter_files_to_run_disallows_interpreter_path_impl(env, target): + env.expect.that_target(target).failures().contains_predicate( + matching.str_matches("*interpreter_files_to_run*interpreter_path*"), + ) + +_tests.append(_test_interpreter_files_to_run_disallows_interpreter_path) + +def _test_interpreter_files_to_run_requires_executable(name): + _simple_binary( + name = name + "_interpreter", + ) + _file_target( + name = name + "_files_to_run", + ) + _create_py_runtime_info_with_interpreter_files_to_run( + name = name + "_subject", + files_to_run = name + "_files_to_run", + interpreter = name + "_interpreter", + tags = ["manual"], + ) + analysis_test( + name = name, + target = name + "_subject", + impl = _test_interpreter_files_to_run_requires_executable_impl, + expect_failure = True, + ) + +def _test_interpreter_files_to_run_requires_executable_impl(env, target): + env.expect.that_target(target).failures().contains_predicate( + matching.str_matches("*interpreter_files_to_run*executable*"), + ) + +_tests.append(_test_interpreter_files_to_run_requires_executable) + +def _test_interpreter_files_to_run_requires_matching_interpreter(name): + _simple_binary( + name = name + "_interpreter", + ) + _simple_binary( + name = name + "_other_interpreter", + ) + _create_py_runtime_info_with_interpreter_files_to_run( + name = name + "_subject", + files_to_run = name + "_other_interpreter", + interpreter = name + "_interpreter", + tags = ["manual"], + ) + analysis_test( + name = name, + target = name + "_subject", + impl = _test_interpreter_files_to_run_requires_matching_interpreter_impl, + expect_failure = True, + ) + +def _test_interpreter_files_to_run_requires_matching_interpreter_impl(env, target): + env.expect.that_target(target).failures().contains_predicate( + matching.str_matches("*interpreter_files_to_run.executable*interpreter*"), + ) + +_tests.append(_test_interpreter_files_to_run_requires_matching_interpreter) + def py_runtime_info_test_suite(name): test_suite( name = name, diff --git a/tests/support/py_runtime_info_subject.bzl b/tests/support/py_runtime_info_subject.bzl index 541d4d9e18..0c6c672743 100644 --- a/tests/support/py_runtime_info_subject.bzl +++ b/tests/support/py_runtime_info_subject.bzl @@ -37,6 +37,9 @@ def py_runtime_info_subject(info, *, meta): coverage_tool = lambda *a, **k: _py_runtime_info_subject_coverage_tool(self, *a, **k), files = lambda *a, **k: _py_runtime_info_subject_files(self, *a, **k), interpreter = lambda *a, **k: _py_runtime_info_subject_interpreter(self, *a, **k), + interpreter_files_to_run = lambda *a, **k: ( + _py_runtime_info_subject_interpreter_files_to_run(self, *a, **k) + ), interpreter_path = lambda *a, **k: _py_runtime_info_subject_interpreter_path(self, *a, **k), interpreter_version_info = lambda *a, **k: _py_runtime_info_subject_interpreter_version_info(self, *a, **k), python_version = lambda *a, **k: _py_runtime_info_subject_python_version(self, *a, **k), @@ -84,6 +87,15 @@ def _py_runtime_info_subject_interpreter(self): meta = self.meta.derive("interpreter()"), ) +def _py_runtime_info_subject_interpreter_files_to_run(self): + return subjects.struct( + self.actual.interpreter_files_to_run, + attrs = dict( + executable = subjects.file, + ), + meta = self.meta.derive("interpreter_files_to_run()"), + ) + def _py_runtime_info_subject_interpreter_path(self): return subjects.str( self.actual.interpreter_path, From 5c32fa99b8b0ab15364d2f5d6ba65b8336e9ac62 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Wed, 17 Jun 2026 19:56:46 -0700 Subject: [PATCH 764/922] chore: prepare 2.1 release (#3829) Update changelog and version markers --------- Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> --- CHANGELOG.md | 16 ++++++++-------- .../rules_python/python/config_settings/index.md | 2 +- docs/environment-variables.md | 2 +- docs/toolchains.md | 2 +- gazelle/docs/directives.md | 4 ++-- python/private/py_executable_info.bzl | 2 +- python/private/py_runtime_info.bzl | 2 +- python/private/python.bzl | 8 ++++---- python/uv/private/lock.bzl | 4 ++-- 9 files changed, 21 insertions(+), 21 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 05daf8c85f..5b9ef27d65 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -47,19 +47,19 @@ BEGIN_UNRELEASED_TEMPLATE END_UNRELEASED_TEMPLATE --> -{#v0-0-0} -## Unreleased +{#v2-1-0} +## [2.1.0] - 2026-06-17 -[0.0.0]: https://github.com/bazel-contrib/rules_python/releases/tag/0.0.0 +[2.1.0]: https://github.com/bazel-contrib/rules_python/releases/tag/2.1.0 -{#v0-0-0-removed} +{#v2-1-0-removed} ### Removed * (build_data) Removed CONFIG_MODE from build data ([#3793](https://github.com/bazel-contrib/rules_python/issues/3793)). * (coverage) Support for python 3.8 has been dropped from the bundled `coverage.py` wheel set, since coverage.py 7.6.2 dropped it. -{#v0-0-0-changed} +{#v2-1-0-changed} ### Changed * (bzlmod) How default runtimes are registered has changed to use a manifest of SHAs and URLs. `TOOL_VERSIONS` in `python/versions.bzl` is now empty under @@ -75,7 +75,7 @@ END_UNRELEASED_TEMPLATE platform, a warning is now printed instead of silently producing an empty coverage report. -{#v0-0-0-fixed} +{#v2-1-0-fixed} ### Fixed * (gazelle) `py_library` and `py_test` targets with missing source files can now be removed by Gazelle ([#3375](https://github.com/bazel-contrib/rules_python/issues/3375)). @@ -117,7 +117,7 @@ END_UNRELEASED_TEMPLATE * (coverage) handle nested coverage collection ([#3823](https://github.com/bazel-contrib/rules_python/pull/3823)) -{#v0-0-0-added} +{#v2-1-0-added} ### Added * (toolchain) Added {obj}`PyRuntimeInfo.interpreter_files_to_run` so action consumers can execute an in-build runtime interpreter with its runfiles. @@ -2475,4 +2475,4 @@ Breaking changes: * (pip) Create all_data_requirements alias * Expose Python C headers through the toolchain. -[0.24.0]: https://github.com/bazel-contrib/rules_python/releases/tag/0.24.0 +[0.24.0]: https://github.com/bazel-contrib/rules_python/releases/tag/0.24.0 \ No newline at end of file diff --git a/docs/api/rules_python/python/config_settings/index.md b/docs/api/rules_python/python/config_settings/index.md index d7bc296296..cd3cbc9829 100644 --- a/docs/api/rules_python/python/config_settings/index.md +++ b/docs/api/rules_python/python/config_settings/index.md @@ -375,7 +375,7 @@ is created. ## Removed Flags -:::{versionremoved} VERSION_NEXT_FEATURE +:::{versionremoved} 2.1.0 The following flags were removed: * `pip_whl` diff --git a/docs/environment-variables.md b/docs/environment-variables.md index 35792415cf..6c3444cba3 100644 --- a/docs/environment-variables.md +++ b/docs/environment-variables.md @@ -151,7 +151,7 @@ When `1`, debug information about coverage behavior is printed to stderr. ## Removed Environment Variables -:::{versionremoved} VERSION_NEXT_FEATURE +:::{versionremoved} 2.1.0 The following environment variables were removed: * `RULES_PYTHON_ENABLE_PYSTAR`: Used to enable the Starlark implementation of diff --git a/docs/toolchains.md b/docs/toolchains.md index 80884baedf..98534aee63 100644 --- a/docs/toolchains.md +++ b/docs/toolchains.md @@ -373,7 +373,7 @@ Notes: that match supported platforms. - Only runtimes matching known platforms in `rules_python` will be registered. -:::{versionadded} VERSION_NEXT_FEATURE +:::{versionadded} 2.1.0 Added support for registering runtimes from a manifest using `add_runtime_manifest_files`, `add_runtime_manifest_urls`, and `runtime_manifest_sha` in `python.override`. diff --git a/gazelle/docs/directives.md b/gazelle/docs/directives.md index dfa439358d..ce8c03b9cd 100644 --- a/gazelle/docs/directives.md +++ b/gazelle/docs/directives.md @@ -681,7 +681,7 @@ that are relative to the current package. {gh-pr}`3014` ::: -:::{versionchanged} VERSION_NEXT_FEATURE +:::{versionchanged} 2.1.0 The default was changed from `false` to `true`. {gh-pr}`3753` ::: @@ -725,7 +725,7 @@ not write `pyi_deps`. {gh-pr}`3356` ::: -:::{versionchanged} VERSION_NEXT_FEATURE +:::{versionchanged} 2.1.0 The default was changed from `false` to `true`. {gh-pr}`3753` ::: diff --git a/python/private/py_executable_info.bzl b/python/private/py_executable_info.bzl index 9e9baea421..7693fa950f 100644 --- a/python/private/py_executable_info.bzl +++ b/python/private/py_executable_info.bzl @@ -88,7 +88,7 @@ Only used with Windows for files that would have used `declare_symlink()` to create relative symlinks. These may overlap with paths in runfiles; it's up to the consumer to determine how to handle such overlaps. -:::{versionadded} VERSION_NEXT_FEATURE +:::{versionadded} 2.1.0 ::: """, "venv_interpreter_runfiles": """ diff --git a/python/private/py_runtime_info.bzl b/python/private/py_runtime_info.bzl index 64fef399e2..daf01e6c7e 100644 --- a/python/private/py_runtime_info.bzl +++ b/python/private/py_runtime_info.bzl @@ -261,7 +261,7 @@ interpreter in an action should use this field so Bazel can stage the interpreter together with its runfiles. This is `None` for platform runtimes using `interpreter_path` and for file-only interpreter targets. -:::{versionadded} VERSION_NEXT_FEATURE +:::{versionadded} 2.1.0 ::: """, "interpreter_path": """ diff --git a/python/private/python.bzl b/python/private/python.bzl index ac8d1111fb..23bae5d341 100644 --- a/python/private/python.bzl +++ b/python/private/python.bzl @@ -1231,7 +1231,7 @@ Example: [Manifest file format documentation](https://rules-python.readthedocs.io/en/latest/toolchains.html#manifest-file-format) ::: -:::{versionadded} VERSION_NEXT_FEATURE +:::{versionadded} 2.1.0 ::: """, ), @@ -1250,7 +1250,7 @@ Note that `/latest/` can be used in place of a specific release date (e.g., `202 [Manifest file format documentation](https://rules-python.readthedocs.io/en/latest/toolchains.html#manifest-file-format) ::: -:::{versionadded} VERSION_NEXT_FEATURE +:::{versionadded} 2.1.0 ::: """, ), @@ -1274,7 +1274,7 @@ These settings are appended to the `target_settings` of all toolchains registered by the extension, including any that already have settings from `python.single_version_platform_override`. -:::{versionadded} VERSION_NEXT_FEATURE +:::{versionadded} 2.1.0 ::: """, ), @@ -1328,7 +1328,7 @@ The values in this mapping override the default values and do not replace them. doc = """ SHA256 hash for the add_runtime_manifest_urls. -:::{versionadded} VERSION_NEXT_FEATURE +:::{versionadded} 2.1.0 ::: """, ), diff --git a/python/uv/private/lock.bzl b/python/uv/private/lock.bzl index 23f2eed467..7b2aa36098 100644 --- a/python/uv/private/lock.bzl +++ b/python/uv/private/lock.bzl @@ -306,7 +306,7 @@ shortest directory path is selected. This makes `uv` read `[tool.uv]` settings (e.g. `no-build-isolation`, `exclude-dependencies`) from that `pyproject.toml`. -:::{versionadded} VERSION_NEXT_FEATURE +:::{versionadded} 2.1.0 ::: """, ), @@ -529,7 +529,7 @@ def lock( `exclude-dependencies` from that `pyproject.toml`. If no `pyproject.toml` is in `srcs` and no `project` is given, the Bazel package directory is used as fallback. - {versionadded}VERSION_NEXT_FEATURE + {versionadded} 2.1.0 python_version: {type}`str | None` the python_version to transition to when locking the requirements. Defaults to the default python version configured by the {obj}`python` module extension. From 98b769ea97cc818e94860b4e1f3b3daac19cfaa9 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Thu, 18 Jun 2026 18:51:51 -0700 Subject: [PATCH 765/922] chore: generate release changelog from news entries (#3827) Currently, we update the change log directly when user-visible changes are made. This works, but makes backporting patches into release branches prone to unnecessary conflicts. These conflicts can sometimes be quite confusing, making it easy to accidentally merge incorrect content. Instead, what we can do is keep track of an upcoming release's change log entries in separate files under the `news/` directory. When creating a release, all the news entries are assembled into a section in the change log and the news entries deleted. When backporting a change, after cherry picking, the news entries are merged into the release's change log section. --- CHANGELOG.md | 23 +- CONTRIBUTING.md | 28 +- RELEASING.md | 23 +- news/.gitkeep | 1 + news/BUILD.bazel | 6 + tests/news/BUILD.bazel | 8 + tests/news/news_test.py | 68 ++++ tests/tools/private/release/release_test.py | 392 +++++++++++++++++++- tools/private/release/release.py | 359 +++++++++++++++++- 9 files changed, 848 insertions(+), 60 deletions(-) create mode 100644 news/.gitkeep create mode 100644 news/BUILD.bazel create mode 100644 tests/news/BUILD.bazel create mode 100644 tests/news/news_test.py diff --git a/CHANGELOG.md b/CHANGELOG.md index 5b9ef27d65..e1d371293a 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -28,24 +28,16 @@ BEGIN_UNRELEASED_TEMPLATE [0.0.0]: https://github.com/bazel-contrib/rules_python/releases/tag/0.0.0 -{#v0-0-0-removed} -### Removed -* Nothing removed. - -{#v0-0-0-changed} -### Changed -* Nothing changed. - -{#v0-0-0-fixed} -### Fixed -* Nothing fixed. - -{#v0-0-0-added} -### Added -* Nothing added. +Unreleased changes are tracked as individual files in the [news/](./news) directory. END_UNRELEASED_TEMPLATE --> +{#v0-0-0} +## Unreleased + +[0.0.0]: https://github.com/bazel-contrib/rules_python/releases/tag/0.0.0 + +Unreleased changes are tracked as individual files in the [news/](./news) directory. {#v2-1-0} ## [2.1.0] - 2026-06-17 @@ -138,6 +130,7 @@ END_UNRELEASED_TEMPLATE [#2054](https://github.com/bazel-contrib/rules_python/issues/2054). * (coverage) Add support for python 3.14 and bump `coverage.py` to 7.10.7. + {#v2-0-3} ## [2.0.3] - 2026-06-15 diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 917110978b..73ab6d6220 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -171,11 +171,31 @@ validation function. ### Documenting changes -Changes are documented in two places: CHANGELOG.md and API docs. +Changes are documented in two places: news entries and API docs. -CHANGELOG.md contains a brief, human friendly, description. This text is -intended for easy skimming so that, when people upgrade, they can quickly get a -sense of what's relevant to them. +Instead of modifying `CHANGELOG.md` directly, you should create a news entry file in the `news/` directory. These files are automatically assembled into `CHANGELOG.md` at release time. + +#### Creating a news entry + +Create a `.md` file in the `news/` directory. The filename must follow the format `..md`: + +* ``: A unique identifier, typically the GitHub Pull Request number or Issue number (e.g., `1234`). +* ``: The category of the change, which must be one of: + * `added`: For new features or behavior added in a backwards-compatible manner. + * `changed`: For changes in existing behavior. + * `fixed`: For bug fixes. + * `removed`: For removed features or behavior. + +The content of the file should be a brief, human-friendly description of the change. Do not include a leading bullet point (e.g. `*` or `-`), as this is automatically added during assembly. If your change is specific to a subsystem, prefix it with the subsystem in parentheses, e.g., `(gazelle) Fixed handling of...`. + +Example: `news/1234.fixed.md` +```markdown +(gazelle) Fixed handling of auto-included `__init__.py` files when generating `py_binary` targets. +``` + +Do not edit `CHANGELOG.md` directly for unreleased changes. + +#### API documentation API documentation are the doc strings for functions, fields, attributes, etc. When user-visible or notable behavior is added, changed, or removed, the diff --git a/RELEASING.md b/RELEASING.md index e4cf738f3d..a259c782c6 100644 --- a/RELEASING.md +++ b/RELEASING.md @@ -13,13 +13,23 @@ These are the steps for a regularly scheduled release from HEAD. ### Steps 1. Update the changelog and replace the version placeholders by running the - release tool. The next version number will by automatically determined - based on the presence of `VERSION_NEXT_*` placeholders and git tags. + release tool. The next version number will be automatically determined + based on the presence of `VERSION_NEXT_*` placeholders and git tags. The + tool will read all news entry files in the `news/` directory, assemble + them into the changelog, and delete the processed news files. ```shell bazel run //tools/private/release ``` + If you want to append news entries to an already existing release section in + the changelog (for example, to update a drafted release or a release + branch), you can specify the version explicitly: + + ```shell + bazel run //tools/private/release -- X.Y.Z + ``` + 1. Send these changes for review and get them merged. 1. Create a branch for the new release, named `release/X.Y` ``` @@ -72,11 +82,10 @@ gh workflow run release.yml --ref -f publish_to_pypi=false API changes and new features bump the minor, and those with only bug fixes and other minor changes bump the patch digit. -The release tool will automatically determine the next version number. To find -if there were any features added or incompatible changes made, review -[CHANGELOG.md](CHANGELOG.md) and the commit history. This can be done using -github by going to the url: -`https://github.com/bazel-contrib/rules_python/compare/...main`. +The release tool will automatically determine the next version number based on +the `VERSION_NEXT_*` placeholders in the codebase. To see what changes are +being accumulated for the next release, review the pending news entries in the +`news/` directory. ## Patch release with cherry picks diff --git a/news/.gitkeep b/news/.gitkeep new file mode 100644 index 0000000000..35e8e662aa --- /dev/null +++ b/news/.gitkeep @@ -0,0 +1 @@ +# Keep this directory even if empty so git tracks it. diff --git a/news/BUILD.bazel b/news/BUILD.bazel new file mode 100644 index 0000000000..79b48675ca --- /dev/null +++ b/news/BUILD.bazel @@ -0,0 +1,6 @@ +package(default_visibility = ["//:__subpackages__"]) + +filegroup( + name = "news_files", + srcs = glob(["**"]), +) diff --git a/tests/news/BUILD.bazel b/tests/news/BUILD.bazel new file mode 100644 index 0000000000..31abff1a8a --- /dev/null +++ b/tests/news/BUILD.bazel @@ -0,0 +1,8 @@ +load("//python:py_test.bzl", "py_test") + +py_test( + name = "news_test", + srcs = ["news_test.py"], + data = ["//news:news_files"], + deps = ["//python/runfiles"], +) diff --git a/tests/news/news_test.py b/tests/news/news_test.py new file mode 100644 index 0000000000..a8ed7a2849 --- /dev/null +++ b/tests/news/news_test.py @@ -0,0 +1,68 @@ +import pathlib +import unittest + +from python.runfiles import runfiles + + +def _get_news_dir(): + rf = runfiles.Create() + path = rf.Rlocation("rules_python/news") + if path: + return pathlib.Path(path) + return None + + +class NewsTest(unittest.TestCase): + def test_all_news_files_are_valid(self): + news_dir = _get_news_dir() + self.assertIsNotNone(news_dir, "Could not locate news directory in runfiles") + self.assertTrue(news_dir.exists(), "News directory does not exist in runfiles") + + allowed_categories = {"added", "changed", "fixed", "removed"} + + for p in news_dir.iterdir(): + if not p.is_file(): + continue + # Ignore BUILD files and .gitkeep if they are in the directory + if p.name in ("BUILD", "BUILD.bazel", ".gitkeep"): + continue + + filename = p.name + + # Collapse extension and filename check into a single assertRegex + self.assertRegex( + filename, + r"^[^.]+\.[^.]+\.md$", + f"News filename {filename} must follow ..md pattern", + ) + + parts = filename.split(".") + category = parts[1].lower() + + # Category must be valid + self.assertIn( + category, + allowed_categories, + f"News file {filename} has invalid category '{category}'. " + f"Must be one of {allowed_categories}", + ) + + # Must be readable as UTF-8 + try: + content = p.read_text(encoding="utf-8").strip() + except (IOError, UnicodeDecodeError) as e: + self.fail(f"Failed to read news file {filename} as UTF-8: {e}") + + # Content must not be empty + self.assertTrue(len(content) > 0, f"News file {filename} must not be empty") + + # Content must NOT start with bullet points (* or -) + self.assertFalse( + content.startswith("* ") or content.startswith("- "), + f"News file {filename} must not start with bullet points (* or -). " + "The release tool adds them automatically.", + ) + + +if __name__ == "__main__": + unittest.main() diff --git a/tests/tools/private/release/release_test.py b/tests/tools/private/release/release_test.py index 9f4f3b249b..3335a5f65f 100644 --- a/tests/tools/private/release/release_test.py +++ b/tests/tools/private/release/release_test.py @@ -16,21 +16,7 @@ [0.0.0]: https://github.com/bazel-contrib/rules_python/releases/tag/0.0.0 -{#v0-0-0-changed} -### Changed -* Nothing changed. - -{#v0-0-0-fixed} -### Fixed -* Nothing fixed. - -{#v0-0-0-added} -### Added -* Nothing added. - -{#v0-0-0-removed} -### Removed -* Nothing removed. +Unreleased changes are tracked as individual files in the [news/](./news) directory. END_UNRELEASED_TEMPLATE --> @@ -101,6 +87,319 @@ def test_update_changelog(self): self.assertIn("{#v1-23-4-added}", new_content) self.assertIn("{#v1-23-4-removed}", new_content) + def test_update_changelog_with_news(self): + # Arrange + changelog = f""" +# Changelog + +{_UNRELEASED_TEMPLATE} + +{{#v0-0-0}} +## Unreleased + +[0.0.0]: https://github.com/bazel-contrib/rules_python/releases/tag/0.0.0 + +{{#v0-0-0-removed}} +### Removed +* Nothing removed. + +{{#v0-0-0-changed}} +### Changed +* Nothing changed. + +{{#v0-0-0-fixed}} +### Fixed +* Nothing fixed. + +{{#v0-0-0-added}} +### Added +* Nothing added. + +{{#v2-0-2}} +## [2.0.2] - 2026-05-14 + +[2.0.2]: https://github.com/bazel-contrib/rules_python/releases/tag/2.0.2 + +{{#v2-0-2-added}} +### Added +* (toolchains) Some older change. +""" + changelog_path = self.tmpdir / "CHANGELOG.md" + changelog_path.write_text(changelog) + + news_dir = self.tmpdir / "news" + news_dir.mkdir() + + # Create news files + (news_dir / "123.fixed.md").write_text("Fixed a bug in the compiler") + # Test that it handles prefixing "* " if not present + (news_dir / "456.added.md").write_text("* Added a new feature for Python 3.13") + # Empty file should be ignored + (news_dir / "789.changed.md").write_text("") + # Invalid name should be ignored + (news_dir / "invalid_name.md").write_text("Should be ignored") + + # Act + releaser.update_changelog( + "3.0.0", + "2026-06-16", + changelog_path=changelog_path, + news_dir=news_dir, + ) + + # Assert + # 1. News files matching the pattern should be deleted (even empty ones) + self.assertFalse((news_dir / "123.fixed.md").exists()) + self.assertFalse((news_dir / "456.added.md").exists()) + self.assertFalse((news_dir / "789.changed.md").exists()) + # Invalid name does not match pattern -> NOT deleted + self.assertTrue((news_dir / "invalid_name.md").exists()) + + new_content = changelog_path.read_text() + + # 2. Unreleased template comment should still be there + self.assertIn( + _UNRELEASED_TEMPLATE, new_content, msg=f"ACTUAL:\n\n{new_content}\n\n" + ) + + # 3. A fresh active Unreleased section should be present + self.assertIn("{#v0-0-0}", new_content) + self.assertIn("## Unreleased", new_content) + self.assertIn( + "Unreleased changes are tracked as individual files in the [news/](./news) directory.", + new_content, + ) + + # 4. The new release section should be present + self.assertIn("{#v3-0-0}", new_content) + self.assertIn("## [3.0.0] - 2026-06-16", new_content) + self.assertIn( + "[3.0.0]: https://github.com/bazel-contrib/rules_python/releases/tag/3.0.0", + new_content, + ) + + # 5. Correct categories and content + self.assertIn( + "{#v3-0-0-fixed}\n### Fixed\n* Fixed a bug in the compiler", new_content + ) + self.assertIn( + "{#v3-0-0-added}\n### Added\n* Added a new feature for Python 3.13", + new_content, + ) + + # 6. Omitted categories should NOT be present in the new release + self.assertNotIn("{#v3-0-0-removed}", new_content) + self.assertNotIn("{#v3-0-0-changed}", new_content) + + # 7. Old release should still be there + self.assertIn("{#v2-0-2}", new_content) + self.assertIn("## [2.0.2] - 2026-05-14", new_content) + + def test_update_changelog_sorting(self): + # Arrange + changelog = f""" +# Changelog + +{_UNRELEASED_TEMPLATE} + +{{#v0-0-0}} +## Unreleased + +[0.0.0]: https://github.com/bazel-contrib/rules_python/releases/tag/0.0.0 + +Unreleased changes are tracked as individual files in the [news/](./news) directory. + +{{#v2-0-2}} +## [2.0.2] - 2026-05-14 + +[2.0.2]: https://github.com/bazel-contrib/rules_python/releases/tag/2.0.2 + +{{#v2-0-2-added}} +### Added +* (toolchains) Some older change. +""" + changelog_path = self.tmpdir / "CHANGELOG.md" + changelog_path.write_text(changelog) + + news_dir = self.tmpdir / "news" + news_dir.mkdir() + + # Create news files with different sub-categories and some without + (news_dir / "1.fixed.md").write_text("* (zebra) Zebra fix") + (news_dir / "2.fixed.md").write_text("* (apple) Apple fix") + (news_dir / "3.fixed.md").write_text("No subcategory B") + (news_dir / "4.fixed.md").write_text("* (apple) Another apple fix") + (news_dir / "5.fixed.md").write_text("No subcategory A") + + # Act + releaser.update_changelog( + "3.0.0", + "2026-06-16", + changelog_path=changelog_path, + news_dir=news_dir, + ) + + # Assert + new_content = changelog_path.read_text() + + # Expected order in Fixed section: + # 1. No subcategory A + # 2. No subcategory B + # 3. (apple) Another apple fix + # 4. (apple) Apple fix + # 5. (zebra) Zebra fix + + expected_fixed_section = ( + "### Fixed\n" + "* No subcategory A\n" + "* No subcategory B\n" + "* (apple) Another apple fix\n" + "* (apple) Apple fix\n" + "* (zebra) Zebra fix\n" + ) + + self.assertIn(expected_fixed_section, new_content) + + def test_update_changelog_read_failure(self): + # Arrange + original_read_text = pathlib.Path.read_text + + with patch("pathlib.Path.read_text", autospec=True) as mock_read_text: + + def side_effect(path_self, *args, **kwargs): + if "bad_file.fixed.md" in str(path_self): + raise IOError("Simulated read error") + return original_read_text(path_self, *args, **kwargs) + + mock_read_text.side_effect = side_effect + + changelog = f""" +# Changelog + +{_UNRELEASED_TEMPLATE} + +{{#v0-0-0}} +## Unreleased + +[0.0.0]: https://github.com/bazel-contrib/rules_python/releases/tag/0.0.0 + +Unreleased changes are tracked as individual files in the [news/](./news) directory. + +{{#v2-0-2}} +## [2.0.2] - 2026-05-14 + +[2.0.2]: https://github.com/bazel-contrib/rules_python/releases/tag/2.0.2 + +{{#v2-0-2-added}} +### Added +* (toolchains) Some older change. +""" + changelog_path = self.tmpdir / "CHANGELOG.md" + changelog_path.write_text(changelog) + + news_dir = self.tmpdir / "news" + news_dir.mkdir() + + # Create the bad file (must exist so it is found by iterdir) + bad_file = news_dir / "bad_file.fixed.md" + bad_file.write_text("some content that won't be read") + + # Create a good file too + good_file = news_dir / "good_file.fixed.md" + good_file.write_text("* (sub) Good fix") + + # Act & Assert + # It should raise IOError + with self.assertRaises(IOError): + releaser.update_changelog( + "3.0.0", + "2026-06-16", + changelog_path=changelog_path, + news_dir=news_dir, + ) + + # Both files should still exist (no deletion on failure!) + self.assertTrue(bad_file.exists()) + self.assertTrue(good_file.exists()) + + # Changelog should not be modified + new_content = changelog_path.read_text() + self.assertEqual(changelog, new_content) + + def test_update_changelog_merge_existing(self): + # Arrange + changelog = f""" +# Changelog + +{_UNRELEASED_TEMPLATE} + +{{#v0-0-0}} +## Unreleased + +[0.0.0]: https://github.com/bazel-contrib/rules_python/releases/tag/0.0.0 + +Unreleased changes are tracked as individual files in the [news/](./news) directory. + +{{#v2-0-3}} +## [2.0.3] - 2026-06-15 + +[2.0.3]: https://github.com/bazel-contrib/rules_python/releases/tag/2.0.3 + +{{#v2-0-3-fixed}} +### Fixed +* (pypi) Old fix + multi-line detail + * nested bullet item +* (pypi) Z old fix +""" + changelog_path = self.tmpdir / "CHANGELOG.md" + changelog_path.write_text(changelog) + + news_dir = self.tmpdir / "news" + news_dir.mkdir() + + # Create news files to merge + # 1. New fix in same category (should merge and sort) + (news_dir / "1.fixed.md").write_text("(pypi) New fix") + # 2. New entry in new category (should create category) + (news_dir / "2.added.md").write_text("(toolchains) New feature") + + # Act + releaser.update_changelog( + "2.0.3", + "2026-06-15", + changelog_path=changelog_path, + news_dir=news_dir, + ) + + # Assert + # News files should be deleted + self.assertFalse((news_dir / "1.fixed.md").exists()) + self.assertFalse((news_dir / "2.added.md").exists()) + + new_content = changelog_path.read_text() + + # Expected merged and sorted Fixed section: + # 1. (pypi) New fix (New < Old) + # 2. (pypi) Old fix (with its multi-line detail!) + # 3. (pypi) Z old fix + expected_fixed_section = ( + "### Fixed\n" + "* (pypi) New fix\n" + "* (pypi) Old fix\n" + " multi-line detail\n" + " * nested bullet item\n" + "* (pypi) Z old fix\n" + ) + self.assertIn(expected_fixed_section, new_content) + + # Expected created Added section: + expected_added_section = "### Added\n* (toolchains) New feature\n" + self.assertIn(expected_added_section, new_content) + + # Active Unreleased section should NOT be touched (should still be empty/pointing to news) + self.assertIn("Unreleased changes are tracked as individual files", new_content) + def test_replace_version_next(self): # Arrange mock_file_content = """ @@ -265,6 +564,69 @@ def test_both_markers(self): self.assertEqual(next_version, "1.3.0") + @patch("tools.private.release.release._get_current_branch") + @patch("tools.private.release.release._get_git_tags") + def test_determine_next_version_on_release_branch_with_existing_tags( + self, mock_get_tags, mock_get_branch + ): + mock_get_branch.return_value = "release/0.37" + mock_get_tags.return_value = ["0.37.0", "0.37.1", "0.36.0"] + + next_version = releaser.determine_next_version() + + self.assertEqual(next_version, "0.37.2") + + @patch("tools.private.release.release._get_current_branch") + @patch("tools.private.release.release._get_git_tags") + def test_determine_next_version_on_release_branch_no_tags( + self, mock_get_tags, mock_get_branch + ): + mock_get_branch.return_value = "release/0.38" + mock_get_tags.return_value = ["0.37.0"] # No 0.38.x tags + + next_version = releaser.determine_next_version() + + self.assertEqual(next_version, "0.38.0") + + @patch("tools.private.release.release._get_current_branch") + @patch("tools.private.release.release._get_git_tags") + def test_determine_next_version_on_release_branch_with_active_rc( + self, mock_get_tags, mock_get_branch + ): + mock_get_branch.return_value = "release/0.37" + # 0.37.0-rc0 and rc1 exist, but no stable 0.37.0 yet + mock_get_tags.return_value = ["0.37.0-rc0", "0.37.0-rc1", "0.36.0"] + + next_version = releaser.determine_next_version() + + # Should target 0.37.0, not 0.37.1 + self.assertEqual(next_version, "0.37.0") + + @patch("tools.private.release.release._get_current_branch") + @patch("tools.private.release.release._get_git_tags") + def test_determine_next_version_on_release_branch_with_stable_and_active_patch_rc( + self, mock_get_tags, mock_get_branch + ): + mock_get_branch.return_value = "release/0.37" + # 0.37.0 stable exists, and 0.37.1-rc0 exists (but no stable 0.37.1 yet) + mock_get_tags.return_value = ["0.37.0", "0.37.1-rc0", "0.36.0"] + + next_version = releaser.determine_next_version() + + # Should target 0.37.1, not 0.37.2 + self.assertEqual(next_version, "0.37.1") + + @patch("tools.private.release.release._get_current_branch") + def test_determine_next_version_on_main_branch_fallback(self, mock_get_branch): + mock_get_branch.return_value = "main" + # Should fallback to default behavior (which uses mock_get_latest_version from setUp) + self.mock_get_latest_version.return_value = "1.2.3" + (self.tmpdir / "mock_file.bzl").write_text("no markers here") + + next_version = releaser.determine_next_version() + + self.assertEqual(next_version, "1.2.4") + if __name__ == "__main__": unittest.main() diff --git a/tools/private/release/release.py b/tools/private/release/release.py index 6fce0ff3b0..4f956bd56c 100644 --- a/tools/private/release/release.py +++ b/tools/private/release/release.py @@ -91,8 +91,68 @@ def should_increment_minor(): return False -def determine_next_version(): - """Determines the next version based on git tags and placeholders.""" +def _get_current_branch(): + """Returns the current git branch name, or None if not in a git repo.""" + try: + return ( + subprocess.check_output( + ["git", "rev-parse", "--abbrev-ref", "HEAD"], + stderr=subprocess.DEVNULL, + ) + .decode("utf-8") + .strip() + ) + except subprocess.CalledProcessError: + return None + + +def determine_next_version(branch_name=None): + """Determines the next version based on git tags and the current branch.""" + if branch_name is None: + branch_name = _get_current_branch() + + if branch_name: + release_match = re.match(r"^release/(\d+)\.(\d+)$", branch_name) + if release_match: + branch_major = int(release_match.group(1)) + branch_minor = int(release_match.group(2)) + print( + f"Detected release branch: {branch_name} (targeting" + f" {branch_major}.{branch_minor}.x)" + ) + + # Find all stable tags matching this major.minor prefix. + # Crucially, we ignore release candidates (RCs) here. If an RC is active + # (e.g. 0.37.0-rc0 exists but 0.37.0 stable does not), we want to continue + # targeting 0.37.0, NOT increment to 0.37.1. + tags = _get_git_tags() + matching_patches = [] + for tag in tags: + tag = tag.strip() + m = re.match(rf"^{branch_major}\.{branch_minor}\.(\d+)$", tag) + if m: + matching_patches.append(int(m.group(1))) + + if matching_patches: + latest_patch = max(matching_patches) + next_version = f"{branch_major}.{branch_minor}.{latest_patch + 1}" + print( + f"Latest tag on this branch is" + f" {branch_major}.{branch_minor}.{latest_patch}. Next" + f" version: {next_version}" + ) + return next_version + else: + # No stable tags exist yet for this release branch (preparing X.Y.0, + # even if X.Y.0-rcN tags already exist) + next_version = f"{branch_major}.{branch_minor}.0" + print( + f"No stable tags found for {branch_major}.{branch_minor}.x." + f" Next version: {next_version}" + ) + return next_version + + # Fallback to default behavior (for main branch or other development branches) latest_version = get_latest_version() major, minor, patch = [int(n) for n in latest_version.split(".")] @@ -102,31 +162,292 @@ def determine_next_version(): return f"{major}.{minor}.{patch + 1}" -def update_changelog(version, release_date, changelog_path="CHANGELOG.md"): - """Performs the version replacements in CHANGELOG.md.""" +def _get_sub_category(content): + """Extracts the sub-category in parentheses from the entry content.""" + match = re.match(r"^(?:\*|-)\s*\(([^)]+)\)", content) + if match: + return match.group(1).lower() + return "" + + +def _get_news_files(news_dir): + """Returns a list of news files matching the ..md pattern.""" + news_path = pathlib.Path(news_dir) + if not news_path.exists(): + return [] + + valid_files = [] + for p in news_path.iterdir(): + if not p.is_file(): + continue + if p.suffix != ".md": + continue + parts = p.name.split(".") + if len(parts) < 3: + continue + valid_files.append(p) + + return valid_files + +def _parse_new_files(news_files): + """Parses news files and groups them by category.""" + entries = {} + for p in news_files: + parts = p.name.split(".") + category = parts[1].lower() + + content = p.read_text(encoding="utf-8").strip() + + if not content: + continue + + # Format as list item if not already + if not (content.startswith("* ") or content.startswith("- ")): + content = f"* {content}" + + if category not in entries: + entries[category] = [] + entries[category].append(content) + + return entries + + +def generate_release_block(version, release_date, news_entries): + """Generates the markdown block for the release.""" header_version = version.replace(".", "-") + lines = [ + f"{{#v{header_version}}}", + f"## [{version}] - {release_date}", + "", + f"[{version}]: https://github.com/bazel-contrib/rules_python/releases/tag/{version}", + "", + ] + + # Standard categories in preferred order + category_order = ["removed", "changed", "fixed", "added"] + # Add any other categories found + for cat in news_entries: + if cat not in category_order: + category_order.append(cat) + + for cat in category_order: + if cat in news_entries and news_entries[cat]: + lines.append(f"{{#v{header_version}-{cat}}}") + lines.append(f"### {cat.capitalize()}") + + # Sort entries by sub-category, then by content + sorted_entries = sorted( + news_entries[cat], key=lambda e: (_get_sub_category(e), e) + ) + + for entry in sorted_entries: + lines.append(entry) + lines.append("") + return "\n".join(lines) + + +def _add_news_to_changelog(changelog_path, version, entries, release_date): + """Adds or merges news entries into CHANGELOG.md.""" changelog_path_obj = pathlib.Path(changelog_path) - lines = changelog_path_obj.read_text().splitlines() + changelog_content = changelog_path_obj.read_text(encoding="utf-8") + + header_version = version.replace(".", "-") + version_anchor = f"{{#v{header_version}}}" + version_exists = version_anchor in changelog_content + + if version_exists: + if not entries: + print( + f"Version {version} already exists and no news entries found" + " to merge. Doing nothing." + ) + return + + print(f"Version {version} already exists in changelog. Merging news entries...") + # Extract the existing version block + # Match from the version anchor to the next version anchor (or end of file) + pattern = ( + r"(?P\{#v" + + re.escape(header_version) + + r"\})(?P.*?)(?=\n\s*\{#v(?!0-0-0)\d+-\d+-\d+\}|\Z)" + ) + match = re.search(pattern, changelog_content, re.DOTALL) + if not match: + raise RuntimeError( + f"Could not find content for existing version {version} in CHANGELOG.md" + ) + + content_block = match.group("content") + + # Split content_block into header and categories + category_anchor_pattern = ( + r"\{#v" + re.escape(header_version) + r"-(?P[a-z]+)\}" + ) + match_cat = re.search(category_anchor_pattern, content_block) + if match_cat: + header_end_idx = match_cat.start() + header_str = content_block[:header_end_idx] + categories_str = content_block[header_end_idx:] + else: + header_str = content_block + categories_str = "" + + # Parse existing categories + existing_entries = {} + if categories_str: + cat_matches = list(re.finditer(category_anchor_pattern, categories_str)) + for i, m in enumerate(cat_matches): + cat = m.group("cat") + start_idx = m.end() + end_idx = ( + cat_matches[i + 1].start() + if i + 1 < len(cat_matches) + else len(categories_str) + ) + cat_content = categories_str[start_idx:end_idx].strip() + + lines = cat_content.splitlines() + cat_entries = [] + current_entry = [] + for line in lines: + if not line.strip() or line.strip().startswith("### "): + continue + if line.startswith("* ") or line.startswith("- "): + if current_entry: + cat_entries.append("\n".join(current_entry)) + current_entry = [line] + else: + if current_entry: + current_entry.append(line) + if current_entry: + cat_entries.append("\n".join(current_entry)) + existing_entries[cat] = cat_entries + + # Merge news entries + merged_entries = dict(existing_entries) + for cat, cat_entries in entries.items(): + if cat not in merged_entries: + merged_entries[cat] = [] + merged_entries[cat].extend(cat_entries) + + # Reconstruct categories + reconstructed_lines = [] + category_order = ["removed", "changed", "fixed", "added"] + for cat in merged_entries: + if cat not in category_order: + category_order.append(cat) + + for cat in category_order: + if cat in merged_entries and merged_entries[cat]: + reconstructed_lines.append(f"{{#v{header_version}-{cat}}}") + reconstructed_lines.append(f"### {cat.capitalize()}") + + sorted_entries = sorted( + merged_entries[cat], key=lambda e: (_get_sub_category(e), e) + ) + + for entry in sorted_entries: + reconstructed_lines.append(entry) + reconstructed_lines.append("") + + new_categories_str = "\n".join(reconstructed_lines) + new_release_block = ( + header_str.rstrip() + "\n\n" + new_categories_str.strip() + "\n" + ) + + # Replace in changelog + new_content = re.sub( + pattern, + r"\g\n" + new_release_block.strip() + "\n", + changelog_content, + flags=re.DOTALL, + ) + changelog_path_obj.write_text(new_content, encoding="utf-8") + + else: + if entries: + print( + f"Version {version} does not exist in changelog. Creating new" + " release section from news entries..." + ) + # Extract template + template_match = re.search( + r"BEGIN_UNRELEASED_TEMPLATE\s*\n(.*?)\n\s*END_UNRELEASED_TEMPLATE", + changelog_content, + re.DOTALL, + ) + if not template_match: + raise RuntimeError( + "Could not find BEGIN_UNRELEASED_TEMPLATE in CHANGELOG.md" + ) + + unreleased_template = template_match.group(1).strip() + new_release_block = generate_release_block(version, release_date, entries) + + replacement = f"{unreleased_template}\n\n{new_release_block}\n" + + # Replace the active Unreleased section + pattern = r"(END_UNRELEASED_TEMPLATE\s*\n-->\s*\n)(.*?)(\n\s*\{#v(?!0-0-0)\d+-\d+-\d+\})" + + if not re.search(pattern, changelog_content, re.DOTALL): + raise RuntimeError( + "Could not find active Unreleased section to replace in" + " CHANGELOG.md" + ) + + new_content = re.sub( + pattern, + r"\g<1>" + replacement + r"\g<3>", + changelog_content, + flags=re.DOTALL, + ) + changelog_path_obj.write_text(new_content, encoding="utf-8") + else: + # Fallback to old behavior + print( + f"No news entries found and version {version} does not exist." + " Falling back to manual changelog update..." + ) + header_version = version.replace(".", "-") + lines = changelog_content.splitlines() + + new_lines = [] + after_template = False + before_already_released = True + for line in lines: + if "END_UNRELEASED_TEMPLATE" in line: + after_template = True + if re.match("#v[1-9]-", line): + before_already_released = False + + if after_template and before_already_released: + line = line.replace( + "## Unreleased", f"## [{version}] - {release_date}" + ) + line = line.replace("v0-0-0", f"v{header_version}") + line = line.replace("0.0.0", version) - new_lines = [] - after_template = False - before_already_released = True - for line in lines: - if "END_UNRELEASED_TEMPLATE" in line: - after_template = True - if re.match("#v[1-9]-", line): - before_already_released = False + new_lines.append(line) - if after_template and before_already_released: - line = line.replace("## Unreleased", f"## [{version}] - {release_date}") - line = line.replace("v0-0-0", f"v{header_version}") - line = line.replace("0.0.0", version) + changelog_path_obj.write_text("\n".join(new_lines), encoding="utf-8") + + +def update_changelog( + version, release_date, changelog_path="CHANGELOG.md", news_dir="news" +): + """Performs the version replacements in CHANGELOG.md.""" + news_files = _get_news_files(news_dir) + entries = _parse_new_files(news_files) - new_lines.append(line) + _add_news_to_changelog(changelog_path, version, entries, release_date) - changelog_path_obj.write_text("\n".join(new_lines)) + # Delete news files after successful update + for p in news_files: + p.unlink() + if news_files: + print(f"Removed {len(news_files)} processed news files.") def replace_version_next(version): From 105f6544efaac390ce2f2294d127139299c53781 Mon Sep 17 00:00:00 2001 From: Maximilian Birkner <99721567+maxbirkner@users.noreply.github.com> Date: Fri, 19 Jun 2026 18:57:36 +0200 Subject: [PATCH 766/922] fix(coverage): skip lcov report when no data was collected (#3832) Under `bazel coverage`, coverage.py raises `NoDataError` from `lcov_report()` when no instrumented Python code was executed, instead of writing an (empty) report. This error currently propagates out of the coverage bootstrap and fails an otherwise passing test. This is common and benign in a few setups: - a `py_binary` that only spawns another process and runs no Python itself - a test whose instrumented sources happen not to execute any Python **Before:** `bazel coverage //a:test` fails with `coverage.exceptions.NoDataError` even though `bazel test //a:test` passes. **After:** the `NoDataError` is caught and the empty report is skipped, so the test result is unchanged by whether coverage data happened to be collected. This mirrors the reporter's suggestion in #2762 to "ignore the return code of `coverage lcov`", applied in-process. As noted on the original report and in #3823, there is no in-repo harness for exercising real coverage collection, so this change does not add an automated test. Fixes #2762 --------- Co-authored-by: Richard Levasseur --- news/3832.fixed.md | 1 + python/private/stage2_bootstrap_template.py | 29 ++++++++++++++------- 2 files changed, 21 insertions(+), 9 deletions(-) create mode 100644 news/3832.fixed.md diff --git a/news/3832.fixed.md b/news/3832.fixed.md new file mode 100644 index 0000000000..f1dd32df06 --- /dev/null +++ b/news/3832.fixed.md @@ -0,0 +1 @@ +(coverage) Skip lcov report when no data was collected. diff --git a/python/private/stage2_bootstrap_template.py b/python/private/stage2_bootstrap_template.py index a66b6428b0..b11fc76093 100644 --- a/python/private/stage2_bootstrap_template.py +++ b/python/private/stage2_bootstrap_template.py @@ -357,6 +357,7 @@ def _maybe_collect_coverage(enable): print_verbose_coverage("Sources:\n" + "\n".join(unique_dirs)) import coverage + from coverage.exceptions import NoDataError coverage_dir = os.environ["COVERAGE_DIR"] unique_id = uuid.uuid4() @@ -408,15 +409,25 @@ def _maybe_collect_coverage(enable): cov.stop() lcov_path = os.path.join(coverage_dir, "pylcov_{}.dat".format(unique_id)) print_verbose_coverage("generating lcov from:", lcov_path) - cov.lcov_report( - outfile=lcov_path, - # Ignore errors because sometimes instrumented files aren't - # readable afterwards. e.g. if they come from /dev/fd or if - # they were transient code-under-test in /tmp - ignore_errors=True, - ) - if os.path.isfile(lcov_path): - unresolve_symlinks(lcov_path) + try: + cov.lcov_report( + outfile=lcov_path, + # Ignore errors because sometimes instrumented files aren't + # readable afterwards. e.g. if they come from /dev/fd or if + # they were transient code-under-test in /tmp + ignore_errors=True, + ) + except NoDataError: + # coverage.py raises NoDataError if no instrumented Python code ran + # (e.g. tests not running Python, or subprocess-only binaries). + # Skip the report to avoid failing otherwise passing tests. + # See https://github.com/bazel-contrib/rules_python/issues/2762. + print_verbose_coverage( + "no coverage data collected; skipping lcov report:", lcov_path + ) + else: + if os.path.isfile(lcov_path): + unresolve_symlinks(lcov_path) finally: try: os.unlink(rcfile_name) From fe24f8c3ea7b4d0a3c42d65fd10157eaeaaaaa9b Mon Sep 17 00:00:00 2001 From: Kris foster Date: Fri, 19 Jun 2026 18:07:51 +0100 Subject: [PATCH 767/922] fix(pypi): respect empty envsubst expansion in experimental_index_url (#3828) Resolve `pip.parse(experimental_index_url = ...)` through `envsubst` before checking whether the experimental index-url code path should be enabled. Previously, an unsubstituted template like `$RULES_PYTHON_PIP_INDEX_URL` is a non-empty string and therefore truthy, so `_set_get_index_urls` enabled the experimental index-url mode even when the env var was unset or expanded to empty. With this change, the value is expanded first (using `pip_attr.envsubst` and `module_ctx.os.environ.get`) and the truthiness check runs against the resolved string, matching the behavior users expect when they gate the index URL on an env var. --------- Co-authored-by: Richard Levasseur --- news/3828.fixed.md | 2 + python/private/pypi/BUILD.bazel | 1 + python/private/pypi/hub_builder.bzl | 16 +++++-- tests/pypi/hub_builder/hub_builder_tests.bzl | 50 ++++++++++++++++---- 4 files changed, 56 insertions(+), 13 deletions(-) create mode 100644 news/3828.fixed.md diff --git a/news/3828.fixed.md b/news/3828.fixed.md new file mode 100644 index 0000000000..64de2cc925 --- /dev/null +++ b/news/3828.fixed.md @@ -0,0 +1,2 @@ +(pypi) Fixed `experimental_index_url` checking truthiness before envsubst +expansion. diff --git a/python/private/pypi/BUILD.bazel b/python/private/pypi/BUILD.bazel index 7569eccac4..a9ec49ca79 100644 --- a/python/private/pypi/BUILD.bazel +++ b/python/private/pypi/BUILD.bazel @@ -197,6 +197,7 @@ bzl_library( ":requirements_files_by_platform_bzl", ":whl_config_setting_bzl", ":whl_repo_name_bzl", + "//python/private:envsubst_bzl", "//python/private:full_version_bzl", "//python/private:normalize_name_bzl", "//python/private:text_util_bzl", diff --git a/python/private/pypi/hub_builder.bzl b/python/private/pypi/hub_builder.bzl index d3be266bdf..6435e1656d 100644 --- a/python/private/pypi/hub_builder.bzl +++ b/python/private/pypi/hub_builder.bzl @@ -1,5 +1,6 @@ """A hub repository builder for incrementally building the hub configuration.""" +load("//python/private:envsubst.bzl", "envsubst") load("//python/private:full_version.bzl", "full_version") load("//python/private:normalize_name.bzl", "normalize_name") load("//python/private:repo_utils.bzl", "repo_utils") @@ -177,7 +178,7 @@ def _pip_parse(self, module_ctx, pip_attr): )) return - _set_get_index_urls(self, pip_attr) + _set_get_index_urls(self, module_ctx, pip_attr) self._platforms[python_version] = _platforms( module_ctx, python_version = full_python_version, @@ -345,8 +346,17 @@ def _add_whl_library(self, *, python_version, whl, repo): ### end of setters, below we have various functions to implement the public methods -def _set_get_index_urls(self, pip_attr): - default_index_url = pip_attr.experimental_index_url or self._config.index_url +def _set_get_index_urls(self, mctx, pip_attr): + # Resolve the index URL through envsubst so the ``$VAR`` / ``${VAR:-default}`` + # form is honored when deciding whether the experimental index-url mode is + # active. Without this, an unsubstituted template like ``$RULES_PYTHON_PIP_INDEX_URL`` + # is treated as truthy and the mode is forced on, even when the env var + # would expand to the empty string. + default_index_url = envsubst( + pip_attr.experimental_index_url, + pip_attr.envsubst, + mctx.getenv, + ) or self._config.index_url default_extra_index_urls = pip_attr.experimental_extra_index_urls or [] if not default_index_url: diff --git a/tests/pypi/hub_builder/hub_builder_tests.bzl b/tests/pypi/hub_builder/hub_builder_tests.bzl index d3ea704c27..854bbe856f 100644 --- a/tests/pypi/hub_builder/hub_builder_tests.bzl +++ b/tests/pypi/hub_builder/hub_builder_tests.bzl @@ -729,6 +729,31 @@ simple==0.0.1 --hash=sha256:deadb00f ], expect_url = "pypi.org/simple/", ), + # Regression: an unsubstituted ``$VAR`` template with the env var + # unset must expand to "" and fall back to the config default, + # rather than activating the experimental index-url path. + struct( + requirements_txt = "simple==0.0.1 --hash=sha256:deadb00f", + experimental_index_url = "$RULES_PYTHON_PIP_INDEX_URL", + experimental_extra_index_urls = [], + envsubst = ["RULES_PYTHON_PIP_INDEX_URL"], + environ = {}, + expect_index_url = "https://pypi.org/simple", + expect_extra_index_urls = [], + expect_url = "pypi.org/simple/", + ), + # When the env var is set, the resolved value drives the + # experimental index-url path. + struct( + requirements_txt = "simple==0.0.1 --hash=sha256:deadb00f", + experimental_index_url = "${RULES_PYTHON_PIP_INDEX_URL:-}", + experimental_extra_index_urls = [], + envsubst = ["RULES_PYTHON_PIP_INDEX_URL"], + environ = {"RULES_PYTHON_PIP_INDEX_URL": "https://from-env.example.com/simple"}, + expect_index_url = "https://from-env.example.com/simple", + expect_extra_index_urls = [], + expect_url = "from-env.example.com/simple/", + ), ]: got_kwargs = {} @@ -756,6 +781,7 @@ simple==0.0.1 --hash=sha256:deadb00f ) builder.pip_parse( _mock_mctx( + environ = getattr(test, "environ", {}), mock_files = { "requirements.txt": test.requirements_txt, }, @@ -765,6 +791,7 @@ simple==0.0.1 --hash=sha256:deadb00f python_version = "3.15", experimental_index_url = test.experimental_index_url, experimental_extra_index_urls = test.experimental_extra_index_urls, + envsubst = getattr(test, "envsubst", []), requirements_lock = "requirements.txt", target_platforms = [ "linux_x86_64", @@ -785,23 +812,26 @@ simple==0.0.1 --hash=sha256:deadb00f ], }, }) + want_whl_library = { + "config_load": "@pypi//:config.bzl", + "dep_template": "@pypi//{name}:{target}", + "filename": "simple-0.0.1-py3-none-any.whl", + "index_url": test.expect_index_url, + "requirement": "simple==0.0.1", + "sha256": "deadb00f", + "urls": [test.expect_url], + } + if getattr(test, "envsubst", []): + want_whl_library["envsubst"] = test.envsubst pypi.whl_libraries().contains_exactly({ - "pypi_315_simple_py3_none_any_deadb00f": { - "config_load": "@pypi//:config.bzl", - "dep_template": "@pypi//{name}:{target}", - "filename": "simple-0.0.1-py3-none-any.whl", - "index_url": test.expect_index_url, - "requirement": "simple==0.0.1", - "sha256": "deadb00f", - "urls": [test.expect_url], - }, + "pypi_315_simple_py3_none_any_deadb00f": want_whl_library, }) pypi.extra_aliases().contains_exactly({}) env.expect.that_dict(got_kwargs).contains_exactly({ "attr": struct( auth_patterns = {}, - envsubst = {}, + envsubst = getattr(test, "envsubst", []), extra_index_urls = test.expect_extra_index_urls, index_url = test.expect_index_url, index_url_overrides = {}, From a6541b5f6b2c9a26b735299148cbc6d70f153c42 Mon Sep 17 00:00:00 2001 From: Ignas Anikevicius <240938+aignas@users.noreply.github.com> Date: Sat, 20 Jun 2026 03:47:42 +0900 Subject: [PATCH 768/922] refactor(pypi): don't use Python to patch wheels (#3808) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit 1. Refactored patch_whl.bzl — Extracted fix_record_content() as a public pure function (no rctx dependency) so it can be unit tested directly. 2. Unit tests (patch_whl_tests.bzl) — 5 new rules_testing tests for fix_record_content: - Adds missing files to RECORD - Returns None when nothing is missing - Preserves quoting style of existing entries - Skips excluded files (INSTALLER, RECORD*, REQUESTED) - Skips .whl files 3. Integration test (patch_whl_patch_test.py) — End-to-end test that: - Creates a wheel from testdata/pkg/ via whl_from_dir_repo - Applies a patch via whl_library(whl_patches=...) - Verifies pkg.PATCHED is True (was False) and pkg.DATA is unchanged The integration test follows the established pattern: test data in tests/pypi/patch_whl/testdata/, registered in internal_dev_deps.bzl, wired into MODULE.bazel, and built as a py_test. All 205 pypi tests pass. With this we can potentially more easily separate whl and sdist path ways. --------- Co-authored-by: Richard Levasseur --- MODULE.bazel | 1 + python/private/BUILD.bazel | 1 - python/private/internal_dev_deps.bzl | 18 ++ python/private/pypi/BUILD.bazel | 7 - python/private/pypi/hub_builder.bzl | 2 +- python/private/pypi/patch_whl.bzl | 228 +++++++++++++----- python/private/pypi/repack_whl.ps1 | 7 + python/private/pypi/repack_whl.py | 200 --------------- python/private/pypi/whl_library.bzl | 8 +- tests/pypi/hub_builder/hub_builder_tests.bzl | 1 - tests/pypi/patch_whl/BUILD.bazel | 11 + tests/pypi/patch_whl/patch_whl_patch_test.py | 17 ++ tests/pypi/patch_whl/patch_whl_tests.bzl | 100 +++++++- tests/pypi/patch_whl/testdata/BUILD.bazel | 0 .../patch_whl/testdata/patches/BUILD.bazel | 1 + .../testdata/patches/modify_pkg.patch | 6 + tests/pypi/patch_whl/testdata/pkg/BUILD.bazel | 0 .../testdata/pkg/pkg-1.0.dist-info/METADATA | 2 + .../testdata/pkg/pkg-1.0.dist-info/RECORD | 4 + .../testdata/pkg/pkg-1.0.dist-info/WHEEL | 1 + tests/pypi/patch_whl/testdata/pkg/pkg.py | 2 + tests/pypi/repack_whl/BUILD.bazel | 8 - tests/pypi/repack_whl/repack_whl_test.py | 37 --- 23 files changed, 339 insertions(+), 323 deletions(-) create mode 100644 python/private/pypi/repack_whl.ps1 delete mode 100644 python/private/pypi/repack_whl.py create mode 100644 tests/pypi/patch_whl/patch_whl_patch_test.py create mode 100644 tests/pypi/patch_whl/testdata/BUILD.bazel create mode 100644 tests/pypi/patch_whl/testdata/patches/BUILD.bazel create mode 100644 tests/pypi/patch_whl/testdata/patches/modify_pkg.patch create mode 100644 tests/pypi/patch_whl/testdata/pkg/BUILD.bazel create mode 100644 tests/pypi/patch_whl/testdata/pkg/pkg-1.0.dist-info/METADATA create mode 100644 tests/pypi/patch_whl/testdata/pkg/pkg-1.0.dist-info/RECORD create mode 100644 tests/pypi/patch_whl/testdata/pkg/pkg-1.0.dist-info/WHEEL create mode 100644 tests/pypi/patch_whl/testdata/pkg/pkg.py delete mode 100644 tests/pypi/repack_whl/BUILD.bazel delete mode 100644 tests/pypi/repack_whl/repack_whl_test.py diff --git a/MODULE.bazel b/MODULE.bazel index ab1c41bc09..568c2732e5 100644 --- a/MODULE.bazel +++ b/MODULE.bazel @@ -106,6 +106,7 @@ use_repo( "buildkite_config", "implicit_namespace_ns_sub1", "implicit_namespace_ns_sub2", + "patch_whl_pkg", "pkgutil_nspkg1", "pkgutil_nspkg2", "rules_python_runtime_env_tc_info", diff --git a/python/private/BUILD.bazel b/python/private/BUILD.bazel index e4b38c2d1d..133c9c6444 100644 --- a/python/private/BUILD.bazel +++ b/python/private/BUILD.bazel @@ -774,7 +774,6 @@ bzl_library( exports_files( [ "coverage.patch", - "repack_whl.py", "py_package.bzl", "py_wheel.bzl", "version.bzl", diff --git a/python/private/internal_dev_deps.bzl b/python/private/internal_dev_deps.bzl index b0b06c31b2..5bbee232f9 100644 --- a/python/private/internal_dev_deps.bzl +++ b/python/private/internal_dev_deps.bzl @@ -131,6 +131,24 @@ def _internal_dev_deps_impl(mctx): config_load = "@rules_python//tests/pypi/whl_library/testdata:packages.bzl", ) + # Setup for //tests/pypi/patch_whl/patch_whl_patch_test.py + whl_from_dir_repo( + name = "patch_whl_pkg_whl", + root = "//tests/pypi/patch_whl/testdata/pkg:BUILD.bazel", + output = "pkg-1.0-any-none-any.whl", + ) + whl_library( + name = "patch_whl_pkg", + whl_file = "@patch_whl_pkg_whl//:pkg-1.0-any-none-any.whl", + requirement = "pkg", + whl_patches = { + "//tests/pypi/patch_whl/testdata/patches:modify_pkg.patch": json.encode({ + "patch_strip": 0, + "whls": ["pkg-1.0-any-none-any.whl"], + }), + }, + ) + def _whl_library_from_dir(*, name, output, root, **kwargs): whl_from_dir_repo( name = "{}_whl".format(name), diff --git a/python/private/pypi/BUILD.bazel b/python/private/pypi/BUILD.bazel index a9ec49ca79..b9a7a18aed 100644 --- a/python/private/pypi/BUILD.bazel +++ b/python/private/pypi/BUILD.bazel @@ -13,7 +13,6 @@ # limitations under the License. load("@bazel_skylib//:bzl_library.bzl", "bzl_library") -load("//python:py_library.bzl", "py_library") package(default_visibility = ["//:__subpackages__"]) @@ -416,12 +415,6 @@ bzl_library( ], ) -py_library( - name = "repack_whl", - srcs = ["repack_whl.py"], - deps = ["//tools:wheelmaker"], -) - bzl_library( name = "requirements_files_by_platform_bzl", srcs = ["requirements_files_by_platform.bzl"], diff --git a/python/private/pypi/hub_builder.bzl b/python/private/pypi/hub_builder.bzl index 6435e1656d..34b726d6f6 100644 --- a/python/private/pypi/hub_builder.bzl +++ b/python/private/pypi/hub_builder.bzl @@ -633,7 +633,7 @@ def _whl_repo( # need to pass the extra args there, so only pop this for whls args["extra_pip_args"] = src.extra_pip_args - if "whl_patches" in args or not (enable_pipstar_extract and is_whl): + if not (enable_pipstar_extract and is_whl): if interpreter.path: args["python_interpreter"] = interpreter.path if interpreter.target: diff --git a/python/private/pypi/patch_whl.bzl b/python/private/pypi/patch_whl.bzl index 71b46f6e7c..98a5ad49c8 100644 --- a/python/private/pypi/patch_whl.bzl +++ b/python/private/pypi/patch_whl.bzl @@ -12,27 +12,26 @@ # See the License for the specific language governing permissions and # limitations under the License. -"""A small utility to patch a file in the repository context and repackage it using a Python interpreter - -Note, because we are patching a wheel file and we need a new RECORD file, this -function will print a diff of the RECORD and will ask the user to include a -RECORD patch in their patches that they maintain. This is to ensure that we can -satisfy the following usecases: -* Patch an invalid RECORD file. -* Patch files within a wheel. - -If we were silently regenerating the RECORD file, we may be vulnerable to supply chain -attacks (it is a very small chance) and keeping the RECORD patches next to the -other patches ensures that the users have overview on exactly what has changed -within the wheel. +"""A small utility to patch a file in the repository context natively. + +This replaces the previous approach that depended on running Python to +repack patched wheels. Instead, after extracting and patching, we fix +the RECORD file natively in Starlark and repack using system zip. """ load("@rules_python_internal//:rules_python_config.bzl", rp_config = "config") load("//python/private:repo_utils.bzl", "repo_utils") load(":parse_whl_name.bzl", "parse_whl_name") -load(":pypi_repo_utils.bzl", "pypi_repo_utils") -_rules_python_root = Label("//:BUILD.bazel") +_DUMMY_SHA256 = "sha256=0" + +_RECORD_EXCLUDES = { + "INSTALLER": None, + "RECORD": None, + "RECORD.jws": None, + "RECORD.p7s": None, + "REQUESTED": None, +} def patched_whl_name(original_whl_name): """Return the new filename to output the patched wheel. @@ -47,14 +46,6 @@ def patched_whl_name(original_whl_name): version = parsed_whl.version suffix = "patched" if "+" in version: - # This already has some local version, so we just append one more - # identifier here. We comply with the spec and mark the file as patched - # by adding a local version identifier at the end. - # - # By doing this we can still install the package using most of the package - # managers - # - # See https://packaging.python.org/en/latest/specifications/version-specifiers/#local-version-identifiers version = "{}.{}".format(version, suffix) else: version = "{}+{}".format(version, suffix) @@ -67,23 +58,22 @@ def patched_whl_name(original_whl_name): platform_tag = parsed_whl.platform_tag, ) -def patch_whl(rctx, *, python_interpreter, whl_path, patches, **kwargs): - """Patch a whl file and repack it to ensure that the RECORD metadata stays correct. +def patch_whl(rctx, *, whl_path, patches): + """Patch a whl file and repack it natively. + + The wheel is extracted, patched, missing RECORD entries are added + with dummy sha256 and size values, and finally the wheel is + repacked using system zip. Args: rctx: repository_ctx - python_interpreter: the python interpreter to use. whl_path: The whl file name to be patched. patches: a label-keyed-int dict that has the patch files as keys and the patch_strip as the value. - **kwargs: extras passed to repo_utils.execute_checked. Returns: value of the repackaging action. """ - - # extract files into the current directory for patching as rctx.patch - # does not support patching in another directory. whl_input = rctx.path(whl_path) repo_utils.extract( @@ -98,43 +88,159 @@ def patch_whl(rctx, *, python_interpreter, whl_path, patches, **kwargs): for patch_file, patch_strip in patches.items(): rctx.patch(patch_file, strip = patch_strip) - record_patch = rctx.path("RECORD.patch") + _fix_record(rctx) + whl_patched = patched_whl_name(whl_input.basename) - pypi_repo_utils.execute_checked( + rctx.delete(whl_input) + _repack_whl(rctx, output = whl_patched) + + return rctx.path(whl_patched) + +def _fix_record(rctx): + """Add missing file entries to RECORD with dummy sha256 and file size.""" + for entry in rctx.path(".").readdir(): + if not entry.basename.endswith(".dist-info"): + continue + + record_path = entry.get_child("RECORD") + if not record_path.exists: + continue + + all_files = _collect_files(rctx) + record_rel = repo_utils.repo_root_relative_path(rctx, record_path) + + new_content = fix_record_content( + record_content = rctx.read(record_path), + all_files = all_files, + record_rel = record_rel, + ) + if new_content != None: + rctx.file(record_path, new_content) + +def fix_record_content(record_content, all_files, record_rel): + """Add missing file entries to RECORD content with dummy sha256 and size. + + Args: + record_content: {type}`str` The existing RECORD file content. + all_files: {type}`dict[str, bool]` All files in the directory, keys + are repo-root-relative paths. + record_rel: {type}`str` The repo-root-relative path to the RECORD file. + + Returns: + {type}`str | None` The new RECORD content if entries were added, + or None if no changes were needed. + """ + has_trailing_newline = record_content.endswith("\n") + lines = record_content.split("\n") + + is_all_quoted = True + has_content = False + existing = {} + for line in lines: + stripped = line.strip() + if not stripped: + continue + has_content = True + if not stripped.startswith('"'): + is_all_quoted = False + parts = stripped.split(",") + if len(parts) >= 1: + fname = parts[0].strip('"') + existing[fname] = True + + if not has_content: + is_all_quoted = False + + existing[record_rel] = True + + added = [] + for fpath in sorted(all_files.keys()): + if fpath in existing: + continue + basename = fpath.split("/")[-1] if "/" in fpath else fpath + if basename in _RECORD_EXCLUDES: + continue + if fpath.endswith(".whl"): + continue + added.append(fpath) + + if not added: + return None + + new_lines = list(lines) + if not has_trailing_newline: + new_lines.append("") + if is_all_quoted: + for fpath in added: + new_lines.append('"{file}",{sha256},0'.format( + file = fpath, + sha256 = _DUMMY_SHA256, + )) + else: + for fpath in added: + new_lines.append("{file},{sha256},0".format( + file = fpath, + sha256 = _DUMMY_SHA256, + )) + new_lines.append("") + + return "\n".join(new_lines) + +def _collect_files(rctx): + """Collect all file paths relative to the repo root (iterative).""" + result = {} + paths = [(rctx.path("."), "")] + for _ in range(10000000): + if not paths: + break + path, prefix = paths.pop() + for entry in path.readdir(): + rel = prefix + "/" + entry.basename if prefix else entry.basename + if entry.is_dir: + paths.append((entry, rel)) + else: + result[rel] = True + return result + +def _repack_whl(rctx, *, output): + """Repack the current directory into a wheel file using system zip.""" + os_name = repo_utils.get_platforms_os_name(rctx) + if os_name == "windows": + _repack_whl_windows(rctx, output = output) + else: + _repack_whl_unix(rctx, output = output) + +def _repack_whl_unix(rctx, *, output): + repo_utils.execute_checked( rctx, - python = python_interpreter, - srcs = [ - Label("//python/private/pypi:repack_whl.py"), - Label("//tools:wheelmaker.py"), - ], + op = "PatchWhl", arguments = [ - "-m", - "python.private.pypi.repack_whl", - "--record-patch", - record_patch, - whl_input, - whl_patched, + "zip", + "-0", + "-X", + str(output), + "-r", + ".", ], - environment = { - "PYTHONPATH": str(rctx.path(_rules_python_root).dirname), - }, - **kwargs ) - if record_patch.exists: - record_patch_contents = rctx.read(record_patch) - warning_msg = """WARNING: the resultant RECORD file of the patch wheel is different +def _repack_whl_windows(rctx, *, output): + powershell_exe = rctx.which("powershell.exe") or rctx.which("powershell") + if not powershell_exe: + fail("powershell not found on PATH") - If you are patching on Windows, you may see this warning because of - a known issue (bazel-contrib/rules_python#1639) with file endings. + script_path = rctx.path(Label("//python/private/pypi:repack_whl.ps1")) - If you would like to silence the warning, you can apply the patch that is stored in - {record_patch}. The contents of the file are below: -{record_patch_contents}""".format( - record_patch = record_patch, - record_patch_contents = record_patch_contents, - ) - print(warning_msg) # buildifier: disable=print - - return rctx.path(whl_patched) + repo_utils.execute_checked( + rctx, + op = "PatchWhl", + arguments = [ + powershell_exe, + "-NoProfile", + "-File", + str(script_path), + "-Output", + str(output), + ], + ) diff --git a/python/private/pypi/repack_whl.ps1 b/python/private/pypi/repack_whl.ps1 new file mode 100644 index 0000000000..2f3b4ada0f --- /dev/null +++ b/python/private/pypi/repack_whl.ps1 @@ -0,0 +1,7 @@ +param( + [string]$Output +) + +$files = Get-ChildItem -Path . -Exclude 'tmp.zip', $Output +Compress-Archive -Path $files -DestinationPath 'tmp.zip' -Force +Move-Item -Path 'tmp.zip' -Destination $Output -Force diff --git a/python/private/pypi/repack_whl.py b/python/private/pypi/repack_whl.py deleted file mode 100644 index 4bf554cdc9..0000000000 --- a/python/private/pypi/repack_whl.py +++ /dev/null @@ -1,200 +0,0 @@ -# Copyright 2023 The Bazel Authors. All rights reserved. -# -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -""" -Regenerate a whl file after patching and cleanup the patched contents. - -This script will take contents of the current directory and create a new wheel -out of it and will remove all files that were written to the wheel. -""" - -from __future__ import annotations - -import argparse -import csv -import difflib -import logging -import pathlib -import sys -import tempfile - -from tools.wheelmaker import _WhlFile - -# NOTE: Implement the following matching of what goes into the RECORD -# https://peps.python.org/pep-0491/#the-dist-info-directory -_EXCLUDES = [ - "RECORD", - "INSTALLER", - "RECORD.jws", - "RECORD.p7s", - "REQUESTED", -] - -_DISTINFO = "dist-info" - - -def _has_all_quoted_filenames(record_contents: str) -> bool: - """Check if all filenames in the RECORD are quoted. - - Some wheels (like torch) have all filenames quoted in their RECORD file. - We detect this to preserve the quoting style when repacking. - """ - lines = record_contents.splitlines() - return all(line.startswith('"') for line in lines) - - -def _unidiff_output(expected, actual, record): - """ - Helper function. Returns a string containing the unified diff of two - multiline strings. - """ - - expected = expected.splitlines(1) - actual = actual.splitlines(1) - - diff = difflib.unified_diff( - expected, actual, fromfile=f"a/{record}", tofile=f"b/{record}" - ) - - return "".join(diff) - - -def _files_to_pack(dir: pathlib.Path, want_record: str) -> list[pathlib.Path]: - """Check that the RECORD file entries are correct and print a unified diff on failure.""" - - # First get existing files by using the RECORD file - got_files = [] - got_distinfos = [] - for row in csv.reader(want_record.splitlines()): - rec = row[0] - path = dir / rec - - if not path.exists(): - # skip files that do not exist as they won't be present in the final - # RECORD file. - continue - - if not path.parent.name.endswith(_DISTINFO): - got_files.append(path) - elif path.name not in _EXCLUDES: - got_distinfos.append(path) - - # Then get extra files present in the directory but not in the RECORD file - extra_files = [] - extra_distinfos = [] - for path in dir.rglob("*"): - if path.is_dir(): - continue - - elif path.parent.name.endswith(_DISTINFO): - if path.name in _EXCLUDES: - # NOTE: we implement the following matching of what goes into the RECORD - # https://peps.python.org/pep-0491/#the-dist-info-directory - continue - elif path not in got_distinfos: - extra_distinfos.append(path) - - elif path not in got_files: - extra_files.append(path) - - # sort the extra files for reproducibility - extra_files.sort() - extra_distinfos.sort() - - # This order ensures that the structure of the RECORD file is always the - # same and ensures smaller patchsets to the RECORD file in general - return got_files + extra_files + got_distinfos + extra_distinfos - - -def main(sys_argv): - parser = argparse.ArgumentParser(description=__doc__) - parser.add_argument( - "whl_path", - type=pathlib.Path, - help="The original wheel file that we have patched.", - ) - parser.add_argument( - "--record-patch", - type=pathlib.Path, - help="The output path that we are going to write the RECORD file patch to.", - ) - parser.add_argument( - "output", - type=pathlib.Path, - help="The output path that we are going to write a new file to.", - ) - args = parser.parse_args(sys_argv) - - cwd = pathlib.Path.cwd() - logging.debug("=" * 80) - logging.debug("Repackaging the wheel") - logging.debug("=" * 80) - - with tempfile.TemporaryDirectory(dir=cwd) as tmpdir: - patched_wheel_dir = cwd / tmpdir - logging.debug(f"Created a tmpdir: {patched_wheel_dir}") - - excludes = [args.whl_path, patched_wheel_dir] - - logging.debug("Moving whl contents to the newly created tmpdir") - for p in cwd.glob("*"): - if p in excludes: - logging.debug(f"Ignoring: {p}") - continue - - rel_path = p.relative_to(cwd) - dst = p.rename(patched_wheel_dir / rel_path) - logging.debug(f"mv {p} -> {dst}") - - distinfo_dir = next(iter(patched_wheel_dir.glob("*dist-info"))) - logging.debug(f"Found dist-info dir: {distinfo_dir}") - record_path = distinfo_dir / "RECORD" - record_contents = record_path.read_text() if record_path.exists() else "" - quote_files = _has_all_quoted_filenames(record_contents) - distribution_prefix = distinfo_dir.with_suffix("").name - - with _WhlFile( - args.output, - mode="w", - distribution_prefix=distribution_prefix, - quote_all_filenames=quote_files, - ) as out: - for p in _files_to_pack(patched_wheel_dir, record_contents): - rel_path = p.relative_to(patched_wheel_dir) - out.add_file(str(rel_path), p) - - logging.debug("Writing RECORD file") - got_record = out.add_recordfile() - - if got_record == record_contents: - logging.info(f"Created a whl file: {args.output}") - return - - record_diff = _unidiff_output( - record_contents, - got_record, - out.distinfo_path("RECORD"), - ) - args.record_patch.write_text(record_diff) - logging.warning( - f"Please apply patch to the RECORD file ({args.record_patch}):\n{record_diff}" - ) - - -if __name__ == "__main__": - logging.basicConfig( - format="%(module)s: %(levelname)s: %(message)s", level=logging.DEBUG - ) - - sys.exit(main(sys.argv[1:])) diff --git a/python/private/pypi/whl_library.bzl b/python/private/pypi/whl_library.bzl index 966d25d04d..37cc36492e 100644 --- a/python/private/pypi/whl_library.bzl +++ b/python/private/pypi/whl_library.bzl @@ -359,10 +359,10 @@ def _whl_library_impl(rctx): # build deps from PyPI (e.g. `flit_core`) if they are missing. extra_pip_args.extend(["--find-links", "."]) - # When we already have a wheel and there are no patches, Python isn't used, + # When we already have a wheel, Python isn't used, # so there's no need to setup env vars to run Python, unless we need to # build an sdist or resolve a requirement. - if whl_path and not rctx.attr.whl_patches: + if whl_path: environment = {} args = [] python_interpreter = None @@ -420,12 +420,8 @@ def _whl_library_impl(rctx): if patches: whl_path = patch_whl( rctx, - op = "whl_library.PatchWhl({}, {})".format(rctx.attr.name, rctx.attr.requirement), - python_interpreter = python_interpreter, whl_path = whl_path, patches = patches, - quiet = rctx.attr.quiet, - timeout = rctx.attr.timeout, ) whl_extract(rctx, whl_path = whl_path, logger = logger) diff --git a/tests/pypi/hub_builder/hub_builder_tests.bzl b/tests/pypi/hub_builder/hub_builder_tests.bzl index 854bbe856f..60017593fb 100644 --- a/tests/pypi/hub_builder/hub_builder_tests.bzl +++ b/tests/pypi/hub_builder/hub_builder_tests.bzl @@ -1170,7 +1170,6 @@ git_dep @ git+https://git.server/repo/project@deadbeefdeadbeef "config_load": "@pypi//:config.bzl", "dep_template": "@pypi//{name}:{target}", "filename": "direct_without_sha-0.0.1-py3-none-any.whl", - "python_interpreter_target": "unit_test_interpreter_target", "requirement": "direct_without_sha==0.0.1", "sha256": "", "urls": ["example-direct.org/direct_without_sha-0.0.1-py3-none-any.whl"], diff --git a/tests/pypi/patch_whl/BUILD.bazel b/tests/pypi/patch_whl/BUILD.bazel index d6c4f47b36..cfdc3cc12a 100644 --- a/tests/pypi/patch_whl/BUILD.bazel +++ b/tests/pypi/patch_whl/BUILD.bazel @@ -1,3 +1,14 @@ +load("//python:py_test.bzl", "py_test") +load("//tests/support:support.bzl", "SUPPORTS_BZLMOD") load(":patch_whl_tests.bzl", "patch_whl_test_suite") patch_whl_test_suite(name = "patch_whl_tests") + +py_test( + name = "patch_whl_patch_test", + srcs = ["patch_whl_patch_test.py"], + target_compatible_with = SUPPORTS_BZLMOD, + deps = [ + "@patch_whl_pkg//:pkg", + ], +) diff --git a/tests/pypi/patch_whl/patch_whl_patch_test.py b/tests/pypi/patch_whl/patch_whl_patch_test.py new file mode 100644 index 0000000000..56dfe1aa55 --- /dev/null +++ b/tests/pypi/patch_whl/patch_whl_patch_test.py @@ -0,0 +1,17 @@ +"""Test that wheel patching works end-to-end.""" + +import unittest + +import pkg + + +class PatchWhlTest(unittest.TestCase): + def test_patched(self): + self.assertEqual(pkg.PATCHED, True) + + def test_data_unchanged(self): + self.assertEqual(pkg.DATA, "hello") + + +if __name__ == "__main__": + unittest.main() diff --git a/tests/pypi/patch_whl/patch_whl_tests.bzl b/tests/pypi/patch_whl/patch_whl_tests.bzl index f93fe459c9..5010f03dc0 100644 --- a/tests/pypi/patch_whl/patch_whl_tests.bzl +++ b/tests/pypi/patch_whl/patch_whl_tests.bzl @@ -15,7 +15,7 @@ "" load("@rules_testing//lib:test_suite.bzl", "test_suite") -load("//python/private/pypi:patch_whl.bzl", "patched_whl_name") # buildifier: disable=bzl-visibility +load("//python/private/pypi:patch_whl.bzl", "fix_record_content", "patched_whl_name") # buildifier: disable=bzl-visibility _tests = [] @@ -31,6 +31,104 @@ def _test_simple_local_version(env): _tests.append(_test_simple_local_version) +def _test_fix_record_adds_missing(env): + record = """\ +foo/__init__.py,sha256=abc,123 +foo-1.0.dist-info/RECORD,, +""" + all_files = { + "foo-1.0.dist-info/METADATA": True, + "foo-1.0.dist-info/RECORD": True, + "foo/__init__.py": True, + "foo/bar.py": True, + } + result = fix_record_content( + record_content = record, + all_files = all_files, + record_rel = "foo-1.0.dist-info/RECORD", + ) + env.expect.that_str(result).contains("foo/bar.py,sha256=0,0") + env.expect.that_str(result).contains("foo-1.0.dist-info/METADATA,sha256=0,0") + +_tests.append(_test_fix_record_adds_missing) + +def _test_fix_record_no_missing(env): + record = """\ +foo/__init__.py,sha256=abc,123 +foo/bar.py,sha256=def,456 +foo-1.0.dist-info/RECORD,, +""" + all_files = { + "foo-1.0.dist-info/RECORD": True, + "foo/__init__.py": True, + "foo/bar.py": True, + } + result = fix_record_content( + record_content = record, + all_files = all_files, + record_rel = "foo-1.0.dist-info/RECORD", + ) + env.expect.that_bool(result == None).equals(True) + +_tests.append(_test_fix_record_no_missing) + +def _test_fix_record_preserves_quoting(env): + record = '''\ +"foo/__init__.py",sha256=abc,123 +"foo-1.0.dist-info/RECORD",, +''' + all_files = { + "foo-1.0.dist-info/RECORD": True, + "foo/__init__.py": True, + "foo/bar.py": True, + } + result = fix_record_content( + record_content = record, + all_files = all_files, + record_rel = "foo-1.0.dist-info/RECORD", + ) + env.expect.that_str(result).contains('"foo/bar.py",sha256=0,0') + +_tests.append(_test_fix_record_preserves_quoting) + +def _test_fix_record_skips_excluded(env): + record = """\ +foo/__init__.py,sha256=abc,123 +foo-1.0.dist-info/RECORD,, +""" + all_files = { + "foo-1.0.dist-info/INSTALLER": True, + "foo-1.0.dist-info/RECORD": True, + "foo/__init__.py": True, + } + result = fix_record_content( + record_content = record, + all_files = all_files, + record_rel = "foo-1.0.dist-info/RECORD", + ) + env.expect.that_bool(result == None).equals(True) + +_tests.append(_test_fix_record_skips_excluded) + +def _test_fix_record_skips_whl_files(env): + record = """\ +foo/__init__.py,sha256=abc,123 +foo-1.0.dist-info/RECORD,, +""" + all_files = { + "foo-1.0.dist-info/RECORD": True, + "foo-1.0.whl": True, + "foo/__init__.py": True, + } + result = fix_record_content( + record_content = record, + all_files = all_files, + record_rel = "foo-1.0.dist-info/RECORD", + ) + env.expect.that_bool(result == None).equals(True) + +_tests.append(_test_fix_record_skips_whl_files) + def patch_whl_test_suite(name): """Create the test suite. diff --git a/tests/pypi/patch_whl/testdata/BUILD.bazel b/tests/pypi/patch_whl/testdata/BUILD.bazel new file mode 100644 index 0000000000..e69de29bb2 diff --git a/tests/pypi/patch_whl/testdata/patches/BUILD.bazel b/tests/pypi/patch_whl/testdata/patches/BUILD.bazel new file mode 100644 index 0000000000..85f0d05223 --- /dev/null +++ b/tests/pypi/patch_whl/testdata/patches/BUILD.bazel @@ -0,0 +1 @@ +exports_files(["modify_pkg.patch"]) diff --git a/tests/pypi/patch_whl/testdata/patches/modify_pkg.patch b/tests/pypi/patch_whl/testdata/patches/modify_pkg.patch new file mode 100644 index 0000000000..ba8cca54b5 --- /dev/null +++ b/tests/pypi/patch_whl/testdata/patches/modify_pkg.patch @@ -0,0 +1,6 @@ +--- pkg.py ++++ pkg.py +@@ -1,2 +1,2 @@ +-PATCHED = False ++PATCHED = True + DATA = "hello" diff --git a/tests/pypi/patch_whl/testdata/pkg/BUILD.bazel b/tests/pypi/patch_whl/testdata/pkg/BUILD.bazel new file mode 100644 index 0000000000..e69de29bb2 diff --git a/tests/pypi/patch_whl/testdata/pkg/pkg-1.0.dist-info/METADATA b/tests/pypi/patch_whl/testdata/pkg/pkg-1.0.dist-info/METADATA new file mode 100644 index 0000000000..a102dd73d4 --- /dev/null +++ b/tests/pypi/patch_whl/testdata/pkg/pkg-1.0.dist-info/METADATA @@ -0,0 +1,2 @@ +Name: pkg +Version: 1.0 diff --git a/tests/pypi/patch_whl/testdata/pkg/pkg-1.0.dist-info/RECORD b/tests/pypi/patch_whl/testdata/pkg/pkg-1.0.dist-info/RECORD new file mode 100644 index 0000000000..dba333a1c1 --- /dev/null +++ b/tests/pypi/patch_whl/testdata/pkg/pkg-1.0.dist-info/RECORD @@ -0,0 +1,4 @@ +pkg.py,sha256=abc,123 +pkg-1.0.dist-info/METADATA,sha256=def,456 +pkg-1.0.dist-info/WHEEL,sha256=ghi,789 +pkg-1.0.dist-info/RECORD,, diff --git a/tests/pypi/patch_whl/testdata/pkg/pkg-1.0.dist-info/WHEEL b/tests/pypi/patch_whl/testdata/pkg/pkg-1.0.dist-info/WHEEL new file mode 100644 index 0000000000..a64521a1cc --- /dev/null +++ b/tests/pypi/patch_whl/testdata/pkg/pkg-1.0.dist-info/WHEEL @@ -0,0 +1 @@ +Wheel-Version: 1.0 diff --git a/tests/pypi/patch_whl/testdata/pkg/pkg.py b/tests/pypi/patch_whl/testdata/pkg/pkg.py new file mode 100644 index 0000000000..0985d5a3dd --- /dev/null +++ b/tests/pypi/patch_whl/testdata/pkg/pkg.py @@ -0,0 +1,2 @@ +PATCHED = False +DATA = "hello" diff --git a/tests/pypi/repack_whl/BUILD.bazel b/tests/pypi/repack_whl/BUILD.bazel deleted file mode 100644 index 3f611a2e4f..0000000000 --- a/tests/pypi/repack_whl/BUILD.bazel +++ /dev/null @@ -1,8 +0,0 @@ -load("//python:py_test.bzl", "py_test") - -py_test( - name = "repack_whl_test", - size = "small", - srcs = ["repack_whl_test.py"], - deps = ["//python/private/pypi:repack_whl"], -) diff --git a/tests/pypi/repack_whl/repack_whl_test.py b/tests/pypi/repack_whl/repack_whl_test.py deleted file mode 100644 index 50781cc0e6..0000000000 --- a/tests/pypi/repack_whl/repack_whl_test.py +++ /dev/null @@ -1,37 +0,0 @@ -import unittest - -from python.private.pypi import repack_whl - - -class HasAllQuotedFilenamesTest(unittest.TestCase): - """Tests for _has_all_quoted_filenames detection logic.""" - - def test_all_quoted(self) -> None: - """Returns True when all lines start with quotes (torch-style).""" - record = """\ -"torch/__init__.py",sha256=abc,123 -"torch/utils.py",sha256=def,456 -"torch-2.0.0.dist-info/WHEEL",sha256=ghi,789 -""" - self.assertTrue(repack_whl._has_all_quoted_filenames(record)) - - def test_none_quoted(self) -> None: - """Returns False when no lines are quoted (standard style).""" - record = """\ -torch/__init__.py,sha256=abc,123 -torch/utils.py,sha256=def,456 -torch-2.0.0.dist-info/WHEEL,sha256=ghi,789 -""" - self.assertFalse(repack_whl._has_all_quoted_filenames(record)) - - def test_mixed_quoting(self) -> None: - """Returns False when only some lines are quoted.""" - record = """\ -"file,with,commas.py",sha256=abc,123 -normal_file.py,sha256=def,456 -""" - self.assertFalse(repack_whl._has_all_quoted_filenames(record)) - - -if __name__ == "__main__": - unittest.main() From de0405ad3aeac4bdaf45ab848b6897de8611d28d Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Fri, 19 Jun 2026 17:51:13 -0700 Subject: [PATCH 769/922] feat(py_info): expose VenvSymlinkEntry and VenvSymlinkKind as public APIs (#3835) Currently, `VenvSymlinkEntry` and `VenvSymlinkKind` are defined in the private `python/private/py_info.bzl` file, making them inaccessible to users who want to programmatically define virtual environment symlinks when using `PyInfo` (or at the least, having to load private file paths). To fix this, we load and re-export both symbols in the public `python/py_info.bzl` file. Along the way * Adds `VenvSymlinkEntryBuilder` to allow fluent construction of symlink entries via PyInfoBuilder. * Adds `features.loadable_symbols` to allow programmatic detection of these public symbols. --- docs/BUILD.bazel | 1 + news/expose-venv-symlink.added.md | 2 + news/loadable-symbols.added.md | 2 + python/features.bzl | 19 ++ python/private/py_info.bzl | 260 +++++++++++++++++- python/py_info.bzl | 13 +- tests/base_rules/py_info/py_info_tests.bzl | 23 +- .../app_files_building_tests.bzl | 3 +- 8 files changed, 318 insertions(+), 5 deletions(-) create mode 100644 news/expose-venv-symlink.added.md create mode 100644 news/loadable-symbols.added.md diff --git a/docs/BUILD.bazel b/docs/BUILD.bazel index 67f21b253b..94cb588018 100644 --- a/docs/BUILD.bazel +++ b/docs/BUILD.bazel @@ -94,6 +94,7 @@ sphinx_stardocs( "//python:py_exec_tools_info_bzl", "//python:py_exec_tools_toolchain_bzl", "//python:py_executable_info_bzl", + "//python:py_info_bzl", "//python:py_library_bzl", "//python:py_runtime_bzl", "//python:py_runtime_info_bzl", diff --git a/news/expose-venv-symlink.added.md b/news/expose-venv-symlink.added.md new file mode 100644 index 0000000000..e339bb5d3f --- /dev/null +++ b/news/expose-venv-symlink.added.md @@ -0,0 +1,2 @@ +Exposed {bzl:obj}`VenvSymlinkEntry` and {bzl:obj}`VenvSymlinkKind` in +{bzl:target}`//python:py_info.bzl`. diff --git a/news/loadable-symbols.added.md b/news/loadable-symbols.added.md new file mode 100644 index 0000000000..7d92df7aea --- /dev/null +++ b/news/loadable-symbols.added.md @@ -0,0 +1,2 @@ +Added {bzl:obj}`features.loadable_symbols` to allow detecting public symbols +exported by bzl files. diff --git a/python/features.bzl b/python/features.bzl index a2cb95f1ee..fab44385c8 100644 --- a/python/features.bzl +++ b/python/features.bzl @@ -81,6 +81,15 @@ def _features_typedef(): :::{versionadded} 1.9.0 :::: + + ::::{field} loadable_symbols + :type: dict[str, list[str]] + + A map of bzl paths to the list of public symbols they export. + + :::{versionadded} VERSION_NEXT_FEATURE + ::: + :::: """ _TARGETS = { @@ -90,10 +99,20 @@ _TARGETS = { "//python/cc:current_py_cc_headers_abi3": True, } +_LOADABLE_SYMBOLS = { + "//python:py_info.bzl": [ + # keep sorted + "PyInfo", + "VenvSymlinkEntry", + "VenvSymlinkKind", + ], +} + features = struct( TYPEDEF = _features_typedef, # keep sorted headers_abi3 = True, + loadable_symbols = _LOADABLE_SYMBOLS, precompile = True, py_info_venv_symlinks = True, targets = _TARGETS, diff --git a/python/private/py_info.bzl b/python/private/py_info.bzl index 551e0bb48c..da9421606a 100644 --- a/python/private/py_info.bzl +++ b/python/private/py_info.bzl @@ -116,6 +116,240 @@ PEP440 standard. }, ) +def _VenvSymlinkEntryBuilder_typedef(): + """Builder for VenvSymlinkEntry. + + To create an instance, use {obj}`PyInfoBuilder.add_venv_symlink()`. + + :::{field} files + :type: DepsetBuilder[File] + ::: + + :::{versionadded} VERSION_NEXT_FEATURE + ::: + """ + +def _VenvSymlinkEntryBuilder(): + # buildifier: disable=uninitialized + self = struct( + _state = { + "kind": None, + "link_to_file": None, + "link_to_path": None, + "package": None, + "venv_path": None, + "version": None, + }, + files = builders.DepsetBuilder(), + get_kind = lambda: _VenvSymlinkEntryBuilder_get_kind(self), + set_kind = lambda k: _VenvSymlinkEntryBuilder_set_kind(self, k), + get_link_to_file = lambda: _VenvSymlinkEntryBuilder_get_link_to_file(self), + set_link_to_file = lambda f: _VenvSymlinkEntryBuilder_set_link_to_file(self, f), + get_link_to_path = lambda: _VenvSymlinkEntryBuilder_get_link_to_path(self), + set_link_to_path = lambda p: _VenvSymlinkEntryBuilder_set_link_to_path(self, p), + get_package = lambda: _VenvSymlinkEntryBuilder_get_package(self), + set_package = lambda p: _VenvSymlinkEntryBuilder_set_package(self, p), + get_venv_path = lambda: _VenvSymlinkEntryBuilder_get_venv_path(self), + set_venv_path = lambda p: _VenvSymlinkEntryBuilder_set_venv_path(self, p), + get_version = lambda: _VenvSymlinkEntryBuilder_get_version(self), + set_version = lambda v: _VenvSymlinkEntryBuilder_set_version(self, v), + build = lambda: _VenvSymlinkEntryBuilder_build(self), + ) + return self + +def _VenvSymlinkEntryBuilder_get_kind(self): + """Get the kind of the symlink. + + Args: + self: implicitly added. + + Returns: + {type}`string` One of the {obj}`VenvSymlinkKind` values. + """ + return self._state["kind"] + +def _VenvSymlinkEntryBuilder_get_link_to_file(self): + """Get the file that the symlink points to. + + Args: + self: implicitly added. + + Returns: + {type}`File | None` + """ + return self._state["link_to_file"] + +def _VenvSymlinkEntryBuilder_get_link_to_path(self): + """Get the runfiles-root relative path that the symlink points to. + + Args: + self: implicitly added. + + Returns: + {type}`string | None` + """ + return self._state["link_to_path"] + +def _VenvSymlinkEntryBuilder_get_package(self): + """Get the PyPI package name that the code originates from. + + Args: + self: implicitly added. + + Returns: + {type}`string | None` + """ + return self._state["package"] + +def _VenvSymlinkEntryBuilder_get_venv_path(self): + """Get the path relative to the kind directory within the venv. + + Args: + self: implicitly added. + + Returns: + {type}`string` + """ + return self._state["venv_path"] + +def _VenvSymlinkEntryBuilder_get_version(self): + """Get the PyPI package version that the code originates from. + + Args: + self: implicitly added. + + Returns: + {type}`string | None` + """ + return self._state["version"] + +def _VenvSymlinkEntryBuilder_set_kind(self, kind): + """Set the kind of the symlink. + + Args: + self: implicitly added. + kind: {type}`string` One of the {obj}`VenvSymlinkKind` values. + + Returns: + {type}`VenvSymlinkEntryBuilder` self. + """ + _check_arg_type("kind", "string", kind) + self._state["kind"] = kind + return self + +def _VenvSymlinkEntryBuilder_set_link_to_file(self, link_to_file): + """Set the file that the symlink points to. + + Args: + self: implicitly added. + link_to_file: {type}`File | None` + + Returns: + {type}`VenvSymlinkEntryBuilder` self. + """ + if link_to_file != None: + _check_arg_type("link_to_file", "File", link_to_file) + self._state["link_to_file"] = link_to_file + return self + +def _VenvSymlinkEntryBuilder_set_link_to_path(self, link_to_path): + """Set the runfiles-root relative path that the symlink points to. + + Args: + self: implicitly added. + link_to_path: {type}`string | None` + + Returns: + {type}`VenvSymlinkEntryBuilder` self. + """ + if link_to_path != None: + _check_arg_type("link_to_path", "string", link_to_path) + self._state["link_to_path"] = link_to_path + return self + +def _VenvSymlinkEntryBuilder_set_package(self, package): + """Set the PyPI package name that the code originates from. + + Args: + self: implicitly added. + package: {type}`string | None` + + Returns: + {type}`VenvSymlinkEntryBuilder` self. + """ + if package != None: + _check_arg_type("package", "string", package) + self._state["package"] = package + return self + +def _VenvSymlinkEntryBuilder_set_venv_path(self, venv_path): + """Set the path relative to the kind directory within the venv. + + Args: + self: implicitly added. + venv_path: {type}`string` + + Returns: + {type}`VenvSymlinkEntryBuilder` self. + """ + _check_arg_type("venv_path", "string", venv_path) + self._state["venv_path"] = venv_path + return self + +def _VenvSymlinkEntryBuilder_set_version(self, version): + """Set the PyPI package version that the code originates from. + + Args: + self: implicitly added. + version: {type}`string | None` + + Returns: + {type}`VenvSymlinkEntryBuilder` self. + """ + if version != None: + _check_arg_type("version", "string", version) + self._state["version"] = version + return self + +def _VenvSymlinkEntryBuilder_build(self): + """Builds into a {obj}`VenvSymlinkEntry` object. + + Args: + self: implicitly added. + + Returns: + {type}`VenvSymlinkEntry` + """ + if not self._state["venv_path"]: + fail("venv_path must be set") + return VenvSymlinkEntry( + files = self.files.build(), + kind = self._state["kind"], + link_to_file = self._state["link_to_file"], + link_to_path = self._state["link_to_path"], + package = self._state["package"], + venv_path = self._state["venv_path"], + version = self._state["version"], + ) + +# buildifier: disable=name-conventions +VenvSymlinkEntryBuilder = struct( + TYPEDEF = _VenvSymlinkEntryBuilder_typedef, + build = _VenvSymlinkEntryBuilder_build, + get_kind = _VenvSymlinkEntryBuilder_get_kind, + set_kind = _VenvSymlinkEntryBuilder_set_kind, + get_link_to_file = _VenvSymlinkEntryBuilder_get_link_to_file, + set_link_to_file = _VenvSymlinkEntryBuilder_set_link_to_file, + get_link_to_path = _VenvSymlinkEntryBuilder_get_link_to_path, + set_link_to_path = _VenvSymlinkEntryBuilder_set_link_to_path, + get_package = _VenvSymlinkEntryBuilder_get_package, + set_package = _VenvSymlinkEntryBuilder_set_package, + get_venv_path = _VenvSymlinkEntryBuilder_get_venv_path, + set_venv_path = _VenvSymlinkEntryBuilder_set_venv_path, + get_version = _VenvSymlinkEntryBuilder_get_version, + set_version = _VenvSymlinkEntryBuilder_set_version, +) + def _check_arg_type(name, required_type, value): """Check that a value is of an expected type.""" value_type = type(value) @@ -409,6 +643,8 @@ def _PyInfoBuilder_new(): _has_py2_only_sources = [False], _has_py3_only_sources = [False], _uses_shared_libraries = [False], + _venv_symlink_builders = [], + add_venv_symlink = lambda *a, **k: _PyInfoBuilder_add_venv_symlink(self, *a, **k), build = lambda *a, **k: _PyInfoBuilder_build(self, *a, **k), build_builtin_py_info = lambda *a, **k: _PyInfoBuilder_build_builtin_py_info(self, *a, **k), direct_original_sources = builders.DepsetBuilder(), @@ -438,6 +674,22 @@ def _PyInfoBuilder_new(): ) return self +def _PyInfoBuilder_add_venv_symlink(self): + """Create and return a new VenvSymlinkEntryBuilder. + + :::{versionadded} VERSION_NEXT_FEATURE + ::: + + Args: + self: implicitly added. + + Returns: + {type}`VenvSymlinkEntryBuilder` + """ + entry_builder = _VenvSymlinkEntryBuilder() + self._venv_symlink_builders.append(entry_builder) + return entry_builder + def _PyInfoBuilder_get_has_py3_only_sources(self): """Get the `has_py3_only_sources` value. @@ -649,6 +901,11 @@ def _PyInfoBuilder_build(self): Returns: {type}`PyInfo` """ + venv_symlinks = depset( + direct = [b.build() for b in self._venv_symlink_builders], + transitive = [self.venv_symlinks.build()], + ) + return _EffectivePyInfo( has_py2_only_sources = self._has_py2_only_sources[0], has_py3_only_sources = self._has_py3_only_sources[0], @@ -663,7 +920,7 @@ def _PyInfoBuilder_build(self): transitive_original_sources = self.transitive_original_sources.build(), transitive_pyc_files = self.transitive_pyc_files.build(), transitive_pyi_files = self.transitive_pyi_files.build(), - venv_symlinks = self.venv_symlinks.build(), + venv_symlinks = venv_symlinks, ) def _PyInfoBuilder_build_builtin_py_info(self): @@ -692,6 +949,7 @@ def _PyInfoBuilder_build_builtin_py_info(self): PyInfoBuilder = struct( TYPEDEF = _PyInfoBuilder_typedef, new = _PyInfoBuilder_new, + add_venv_symlink = _PyInfoBuilder_add_venv_symlink, build = _PyInfoBuilder_build, build_builtin_py_info = _PyInfoBuilder_build_builtin_py_info, get_has_py2_only_sources = _PyInfoBuilder_get_has_py2_only_sources, diff --git a/python/py_info.bzl b/python/py_info.bzl index 350a4dbd9e..8e46926c53 100644 --- a/python/py_info.bzl +++ b/python/py_info.bzl @@ -14,6 +14,17 @@ """Public entry point for PyInfo.""" -load("//python/private:py_info.bzl", _PyInfo = "PyInfo") +load( + "//python/private:py_info.bzl", + _PyInfo = "PyInfo", + _VenvSymlinkEntry = "VenvSymlinkEntry", + _VenvSymlinkKind = "VenvSymlinkKind", +) PyInfo = _PyInfo + +# buildifier: disable=name-conventions +VenvSymlinkEntry = _VenvSymlinkEntry + +# buildifier: disable=name-conventions +VenvSymlinkKind = _VenvSymlinkKind diff --git a/tests/base_rules/py_info/py_info_tests.bzl b/tests/base_rules/py_info/py_info_tests.bzl index 273959b957..0e1a96548c 100644 --- a/tests/base_rules/py_info/py_info_tests.bzl +++ b/tests/base_rules/py_info/py_info_tests.bzl @@ -16,7 +16,7 @@ load("@rules_testing//lib:analysis_test.bzl", "analysis_test") load("@rules_testing//lib:test_suite.bzl", "test_suite") load("@rules_testing//lib:util.bzl", rt_util = "util") -load("//python:py_info.bzl", "PyInfo") +load("//python:py_info.bzl", "PyInfo", "VenvSymlinkKind") load("//python/private:py_info.bzl", "PyInfoBuilder") # buildifier: disable=bzl-visibility load("//python/private:reexports.bzl", "BuiltinPyInfo") # buildifier: disable=bzl-visibility load("//tests/support:py_info_subject.bzl", "py_info_subject") @@ -170,6 +170,12 @@ def _test_py_info_builder_impl(env, targets): builder.transitive_sources.add(trans) builder.merge_uses_shared_libraries(True) + symlink = builder.add_venv_symlink() + symlink.set_kind(VenvSymlinkKind.LIB) + symlink.set_venv_path("test_path") + symlink.set_link_to_path("test_target") + symlink.files.add(trans) + builder.merge_target(targets.py1) builder.merge_targets([targets.py2]) @@ -247,6 +253,21 @@ def _test_py_info_builder_impl(env, targets): "tests/base_rules/py_info/py6-trans.pyi", ]) + if hasattr(actual, "venv_symlinks"): + entries = actual.venv_symlinks.to_list() + env.expect.that_int(len(entries)).equals(1) + entry = entries[0] + env.expect.that_str(entry.venv_path).equals("test_path") + env.expect.that_str(entry.link_to_path).equals("test_target") + env.expect.that_str(entry.kind).equals(VenvSymlinkKind.LIB) + env.expect.that_collection(entry.files.to_list()).contains_exactly([trans]) + env.expect.that_bool(entry.package == None).equals(True) + env.expect.that_bool(entry.version == None).equals(True) + env.expect.that_bool(entry.link_to_file == None).equals(True) + + check(builder.build()) + + # Call build() again to verify it doesn't duplicate/leak state check(builder.build()) if BuiltinPyInfo != None: check(builder.build_builtin_py_info()) diff --git a/tests/venv_site_packages_libs/app_files_building/app_files_building_tests.bzl b/tests/venv_site_packages_libs/app_files_building/app_files_building_tests.bzl index 65124076c6..663ab6963e 100644 --- a/tests/venv_site_packages_libs/app_files_building/app_files_building_tests.bzl +++ b/tests/venv_site_packages_libs/app_files_building/app_files_building_tests.bzl @@ -2,10 +2,9 @@ load("@rules_testing//lib:analysis_test.bzl", "analysis_test") load("@rules_testing//lib:test_suite.bzl", "test_suite") -load("//python:py_info.bzl", "PyInfo") +load("//python:py_info.bzl", "PyInfo", "VenvSymlinkEntry", "VenvSymlinkKind") load("//python:py_library.bzl", "py_library") load("//python/private:common_labels.bzl", "labels") # buildifier: disable=bzl-visibility -load("//python/private:py_info.bzl", "VenvSymlinkEntry", "VenvSymlinkKind") # buildifier: disable=bzl-visibility load("//python/private:venv_runfiles.bzl", "build_link_map", "get_venv_symlinks") # buildifier: disable=bzl-visibility load("//tests/support:support.bzl", "SUPPORTS_BZLMOD") From 694cc060eb4e809f152770d37d4d46935b3d7e35 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Fri, 19 Jun 2026 20:26:15 -0700 Subject: [PATCH 770/922] doc: merge news into generated changelog (#3836) Currently, the generated changelog has a broken link to the news directory. Since we can process the files as part of docgen, fix the broken link by rendering the news as one would see it for a release. Along the way... * Delete defunct unreleased template handling, as we've switched to news file assembly instead. --- CHANGELOG.md | 20 +- docs/BUILD.bazel | 24 +- docs/merge_changelog.py | 38 +++ tests/tools/private/release/release_test.py | 268 ++++++++-------- tools/private/release/BUILD.bazel | 8 +- tools/private/release/changelog_news.py | 323 ++++++++++++++++++++ tools/private/release/release.py | 293 +----------------- 7 files changed, 542 insertions(+), 432 deletions(-) create mode 100644 docs/merge_changelog.py create mode 100644 tools/private/release/changelog_news.py diff --git a/CHANGELOG.md b/CHANGELOG.md index e1d371293a..21108ffda2 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -20,24 +20,14 @@ A brief description of the categories of changes: * Particular sub-systems are identified using parentheses, e.g. `(bzlmod)` or `(docs)`. - -{#v0-0-0} +{#unreleased} ## Unreleased -[0.0.0]: https://github.com/bazel-contrib/rules_python/releases/tag/0.0.0 +[unreleased]: https://github.com/bazel-contrib/rules_python/releases/tag/unreleased -Unreleased changes are tracked as individual files in the [news/](./news) directory. +Unreleased changes are tracked as individual files in the [news/](./news) +directory, or view the [latest generated +changelog](https://rules-python.readthedocs.io/en/latest/changelog.html). {#v2-1-0} ## [2.1.0] - 2026-06-17 diff --git a/docs/BUILD.bazel b/docs/BUILD.bazel index 94cb588018..315010c12d 100644 --- a/docs/BUILD.bazel +++ b/docs/BUILD.bazel @@ -17,6 +17,7 @@ load("@sphinxdocs//sphinxdocs:readthedocs.bzl", "readthedocs_install") load("@sphinxdocs//sphinxdocs:sphinx.bzl", "sphinx_build_binary", "sphinx_docs") load("@sphinxdocs//sphinxdocs:sphinx_docs_library.bzl", "sphinx_docs_library") load("@sphinxdocs//sphinxdocs:sphinx_stardoc.bzl", "sphinx_stardoc", "sphinx_stardocs") +load("//python:defs.bzl", "py_binary") load("//python/private:bzlmod_enabled.bzl", "BZLMOD_ENABLED") # buildifier: disable=bzl-visibility load("//python/private:common_labels.bzl", "labels") # buildifier: disable=bzl-visibility load("//python/uv:lock.bzl", "lock") # buildifier: disable=bzl-visibility @@ -37,6 +38,27 @@ _TARGET_COMPATIBLE_WITH = select({ "//conditions:default": ["@platforms//:incompatible"], }) if BZLMOD_ENABLED else ["@platforms//:incompatible"] +py_binary( + name = "merge_changelog", + srcs = ["merge_changelog.py"], + target_compatible_with = _TARGET_COMPATIBLE_WITH, + deps = [ + "//tools/private/release:changelog_news", + ], +) + +genrule( + name = "merged_changelog", + srcs = [ + "//:CHANGELOG.md", + "//news:news_files", + ], + outs = ["changelog.md"], + cmd = "$(location :merge_changelog) --changelog $(location //:CHANGELOG.md) --news-dir news --output $@", + target_compatible_with = _TARGET_COMPATIBLE_WITH, + tools = [":merge_changelog"], +) + # See README.md for instructions. Short version: # * `bazel run //docs:docs.serve` in a separate terminal # * `ibazel build //docs:docs` to automatically rebuild docs @@ -60,8 +82,8 @@ sphinx_docs( "html", ], renamed_srcs = { - "//:CHANGELOG.md": "changelog.md", "//:CONTRIBUTING.md": "contributing.md", + ":merged_changelog": "changelog.md", "@sphinxdocs//sphinxdocs/inventories:bazel_inventory": "bazel_inventory.inv", }, sphinx = ":sphinx-build", diff --git a/docs/merge_changelog.py b/docs/merge_changelog.py new file mode 100644 index 0000000000..47af98b2a2 --- /dev/null +++ b/docs/merge_changelog.py @@ -0,0 +1,38 @@ +"""A tool to merge news entries into CHANGELOG.md for documentation preview.""" + +import argparse +import pathlib + +from tools.private.release import changelog_news + + +def main(): + parser = argparse.ArgumentParser(description="Merge news entries into CHANGELOG.md") + parser.add_argument( + "--changelog", type=pathlib.Path, required=True, help="Path to CHANGELOG.md" + ) + parser.add_argument( + "--news-dir", type=pathlib.Path, required=True, help="Path to news directory" + ) + parser.add_argument( + "--output", + type=pathlib.Path, + required=True, + help="Path to output merged changelog", + ) + args = parser.parse_args() + + # Call the public merge_new_into_changelog function + # Using deterministic date "0000-00-00" and version "0.0.0" + changelog_news.merge_new_into_changelog( + changelog_path=args.changelog, + output_path=args.output, + news_dir=args.news_dir, + version="unreleased", + release_date="0000-00-00", + delete_news=False, + ) + + +if __name__ == "__main__": + main() diff --git a/tests/tools/private/release/release_test.py b/tests/tools/private/release/release_test.py index 3335a5f65f..84c6abeaf3 100644 --- a/tests/tools/private/release/release_test.py +++ b/tests/tools/private/release/release_test.py @@ -5,22 +5,7 @@ import unittest from unittest.mock import patch -from tools.private.release import release as releaser - -_UNRELEASED_TEMPLATE = """ - -""" +from tools.private.release import changelog_news, release as releaser class ReleaserTest(unittest.TestCase): @@ -33,94 +18,37 @@ def setUp(self): # NOTE: On windows, this must be done before files are deleted. self.addCleanup(os.chdir, self.original_cwd) - def test_update_changelog(self): - changelog = f""" -# Changelog - -{_UNRELEASED_TEMPLATE} - -{{#v0-0-0}} -## Unreleased - -[0.0.0]: https://github.com/bazel-contrib/rules_python/releases/tag/0.0.0 - -{{#v0-0-0-changed}} -### Changed -* Nothing changed - -{{#v0-0-0-fixed}} -### Fixed -* Nothing fixed - -{{#v0-0-0-added}} -### Added -* Nothing added - -{{#v0-0-0-removed}} -### Removed -* Nothing removed. -""" - changelog_path = self.tmpdir / "CHANGELOG.md" - changelog_path.write_text(changelog) - - # Act - releaser.update_changelog( - "1.23.4", - "2025-01-01", - changelog_path=changelog_path, - ) - - # Assert - new_content = changelog_path.read_text() - - self.assertIn( - _UNRELEASED_TEMPLATE, new_content, msg=f"ACTUAL:\n\n{new_content}\n\n" - ) - self.assertIn("## [1.23.4] - 2025-01-01", new_content) - self.assertIn( - "[1.23.4]: https://github.com/bazel-contrib/rules_python/releases/tag/1.23.4", - new_content, - ) - self.assertIn("{#v1-23-4}", new_content) - self.assertIn("{#v1-23-4-changed}", new_content) - self.assertIn("{#v1-23-4-fixed}", new_content) - self.assertIn("{#v1-23-4-added}", new_content) - self.assertIn("{#v1-23-4-removed}", new_content) - def test_update_changelog_with_news(self): # Arrange - changelog = f""" -# Changelog + changelog = """# Changelog -{_UNRELEASED_TEMPLATE} - -{{#v0-0-0}} +{#unreleased} ## Unreleased -[0.0.0]: https://github.com/bazel-contrib/rules_python/releases/tag/0.0.0 +[unreleased]: https://github.com/bazel-contrib/rules_python/releases/tag/unreleased -{{#v0-0-0-removed}} +{#unreleased-removed} ### Removed * Nothing removed. -{{#v0-0-0-changed}} +{#unreleased-changed} ### Changed * Nothing changed. -{{#v0-0-0-fixed}} +{#unreleased-fixed} ### Fixed * Nothing fixed. -{{#v0-0-0-added}} +{#unreleased-added} ### Added * Nothing added. -{{#v2-0-2}} +{#v2-0-2} ## [2.0.2] - 2026-05-14 [2.0.2]: https://github.com/bazel-contrib/rules_python/releases/tag/2.0.2 -{{#v2-0-2-added}} +{#v2-0-2-added} ### Added * (toolchains) Some older change. """ @@ -140,7 +68,7 @@ def test_update_changelog_with_news(self): (news_dir / "invalid_name.md").write_text("Should be ignored") # Act - releaser.update_changelog( + changelog_news.update_changelog( "3.0.0", "2026-06-16", changelog_path=changelog_path, @@ -157,20 +85,17 @@ def test_update_changelog_with_news(self): new_content = changelog_path.read_text() - # 2. Unreleased template comment should still be there - self.assertIn( - _UNRELEASED_TEMPLATE, new_content, msg=f"ACTUAL:\n\n{new_content}\n\n" - ) - - # 3. A fresh active Unreleased section should be present - self.assertIn("{#v0-0-0}", new_content) + # 2. A fresh active Unreleased section should be present + self.assertIn("{#unreleased}", new_content) self.assertIn("## Unreleased", new_content) self.assertIn( - "Unreleased changes are tracked as individual files in the [news/](./news) directory.", + "Unreleased changes are tracked as individual files in the [news/](./news)\n" + "directory, or view the [latest generated\n" + "changelog](https://rules-python.readthedocs.io/en/latest/changelog.html).", new_content, ) - # 4. The new release section should be present + # 3. The new release section should be present self.assertIn("{#v3-0-0}", new_content) self.assertIn("## [3.0.0] - 2026-06-16", new_content) self.assertIn( @@ -178,7 +103,7 @@ def test_update_changelog_with_news(self): new_content, ) - # 5. Correct categories and content + # 4. Correct categories and content self.assertIn( "{#v3-0-0-fixed}\n### Fixed\n* Fixed a bug in the compiler", new_content ) @@ -187,34 +112,33 @@ def test_update_changelog_with_news(self): new_content, ) - # 6. Omitted categories should NOT be present in the new release + # 5. Omitted categories should NOT be present in the new release self.assertNotIn("{#v3-0-0-removed}", new_content) self.assertNotIn("{#v3-0-0-changed}", new_content) - # 7. Old release should still be there + # 6. Old release should still be there self.assertIn("{#v2-0-2}", new_content) self.assertIn("## [2.0.2] - 2026-05-14", new_content) def test_update_changelog_sorting(self): # Arrange - changelog = f""" -# Changelog - -{_UNRELEASED_TEMPLATE} + changelog = """# Changelog -{{#v0-0-0}} +{#unreleased} ## Unreleased -[0.0.0]: https://github.com/bazel-contrib/rules_python/releases/tag/0.0.0 +[unreleased]: https://github.com/bazel-contrib/rules_python/releases/tag/unreleased -Unreleased changes are tracked as individual files in the [news/](./news) directory. +Unreleased changes are tracked as individual files in the [news/](./news) +directory, or view the [latest generated +changelog](https://rules-python.readthedocs.io/en/latest/changelog.html). -{{#v2-0-2}} +{#v2-0-2} ## [2.0.2] - 2026-05-14 [2.0.2]: https://github.com/bazel-contrib/rules_python/releases/tag/2.0.2 -{{#v2-0-2-added}} +{#v2-0-2-added} ### Added * (toolchains) Some older change. """ @@ -232,7 +156,7 @@ def test_update_changelog_sorting(self): (news_dir / "5.fixed.md").write_text("No subcategory A") # Act - releaser.update_changelog( + changelog_news.update_changelog( "3.0.0", "2026-06-16", changelog_path=changelog_path, @@ -273,24 +197,23 @@ def side_effect(path_self, *args, **kwargs): mock_read_text.side_effect = side_effect - changelog = f""" -# Changelog + changelog = """# Changelog -{_UNRELEASED_TEMPLATE} - -{{#v0-0-0}} +{#unreleased} ## Unreleased -[0.0.0]: https://github.com/bazel-contrib/rules_python/releases/tag/0.0.0 +[unreleased]: https://github.com/bazel-contrib/rules_python/releases/tag/unreleased -Unreleased changes are tracked as individual files in the [news/](./news) directory. +Unreleased changes are tracked as individual files in the [news/](./news) +directory, or view the [latest generated +changelog](https://rules-python.readthedocs.io/en/latest/changelog.html). -{{#v2-0-2}} +{#v2-0-2} ## [2.0.2] - 2026-05-14 [2.0.2]: https://github.com/bazel-contrib/rules_python/releases/tag/2.0.2 -{{#v2-0-2-added}} +{#v2-0-2-added} ### Added * (toolchains) Some older change. """ @@ -311,7 +234,7 @@ def side_effect(path_self, *args, **kwargs): # Act & Assert # It should raise IOError with self.assertRaises(IOError): - releaser.update_changelog( + changelog_news.update_changelog( "3.0.0", "2026-06-16", changelog_path=changelog_path, @@ -328,24 +251,23 @@ def side_effect(path_self, *args, **kwargs): def test_update_changelog_merge_existing(self): # Arrange - changelog = f""" -# Changelog - -{_UNRELEASED_TEMPLATE} + changelog = """# Changelog -{{#v0-0-0}} +{#unreleased} ## Unreleased -[0.0.0]: https://github.com/bazel-contrib/rules_python/releases/tag/0.0.0 +[unreleased]: https://github.com/bazel-contrib/rules_python/releases/tag/unreleased -Unreleased changes are tracked as individual files in the [news/](./news) directory. +Unreleased changes are tracked as individual files in the [news/](./news) +directory, or view the [latest generated +changelog](https://rules-python.readthedocs.io/en/latest/changelog.html). -{{#v2-0-3}} +{#v2-0-3} ## [2.0.3] - 2026-06-15 [2.0.3]: https://github.com/bazel-contrib/rules_python/releases/tag/2.0.3 -{{#v2-0-3-fixed}} +{#v2-0-3-fixed} ### Fixed * (pypi) Old fix multi-line detail @@ -365,7 +287,7 @@ def test_update_changelog_merge_existing(self): (news_dir / "2.added.md").write_text("(toolchains) New feature") # Act - releaser.update_changelog( + changelog_news.update_changelog( "2.0.3", "2026-06-15", changelog_path=changelog_path, @@ -400,6 +322,102 @@ def test_update_changelog_merge_existing(self): # Active Unreleased section should NOT be touched (should still be empty/pointing to news) self.assertIn("Unreleased changes are tracked as individual files", new_content) + def test_update_changelog_does_not_leak(self): + # Arrange + changelog = """# Changelog + +{#unreleased} +## Unreleased + +[unreleased]: https://github.com/bazel-contrib/rules_python/releases/tag/unreleased + +Unreleased changes are tracked as individual files in the [news/](./news) +directory, or view the [latest generated +changelog](https://rules-python.readthedocs.io/en/latest/changelog.html). + +{#v2-0-2} +## [2.0.2] - 2026-05-14 + +[2.0.2]: https://github.com/bazel-contrib/rules_python/releases/tag/2.0.2 + +This release body mentions the word unreleased and {#unreleased} anchor to test leaks. +""" + changelog_path = self.tmpdir / "CHANGELOG.md" + changelog_path.write_text(changelog) + + news_dir = self.tmpdir / "news" + news_dir.mkdir() + (news_dir / "1.fixed.md").write_text("Some fix") + + # Act + changelog_news.update_changelog( + "3.0.0", + "2026-06-16", + changelog_path=changelog_path, + news_dir=news_dir, + ) + + # Assert + new_content = changelog_path.read_text() + + # The 2.0.2 body should NOT be modified + self.assertIn( + "This release body mentions the word unreleased and {#unreleased} anchor to test leaks.", + new_content, + ) + + def test_update_changelog_empty_news(self): + # Arrange + changelog = """# Changelog + +{#unreleased} +## Unreleased + +[unreleased]: https://github.com/bazel-contrib/rules_python/releases/tag/unreleased + +Unreleased changes are tracked as individual files in the [news/](./news) +directory, or view the [latest generated +changelog](https://rules-python.readthedocs.io/en/latest/changelog.html). + +{#v2-0-2} +## [2.0.2] - 2026-05-14 + +[2.0.2]: https://github.com/bazel-contrib/rules_python/releases/tag/2.0.2 + +{#v2-0-2-added} +### Added +* (toolchains) Some older change. +""" + changelog_path = self.tmpdir / "CHANGELOG.md" + changelog_path.write_text(changelog) + + news_dir = self.tmpdir / "news" + news_dir.mkdir() + + # Act + changelog_news.update_changelog( + "3.0.0", + "2026-06-16", + changelog_path=changelog_path, + news_dir=news_dir, + ) + + # Assert + new_content = changelog_path.read_text() + + # The new release section should be present and contain "No notable changes." + self.assertIn("{#v3-0-0}", new_content) + self.assertIn("## [3.0.0] - 2026-06-16", new_content) + self.assertIn( + "[3.0.0]: https://github.com/bazel-contrib/rules_python/releases/tag/3.0.0", + new_content, + ) + self.assertIn("No notable changes.", new_content) + + # Verify that we didn't accidentally create any categories + self.assertNotIn("{#v3-0-0-fixed}", new_content) + self.assertNotIn("{#v3-0-0-added}", new_content) + def test_replace_version_next(self): # Arrange mock_file_content = """ diff --git a/tools/private/release/BUILD.bazel b/tools/private/release/BUILD.bazel index 31cc3a0239..0afb23962e 100644 --- a/tools/private/release/BUILD.bazel +++ b/tools/private/release/BUILD.bazel @@ -1,12 +1,18 @@ -load("@rules_python//python:defs.bzl", "py_binary") +load("@rules_python//python:defs.bzl", "py_binary", "py_library") package(default_visibility = ["//visibility:public"]) +py_library( + name = "changelog_news", + srcs = ["changelog_news.py"], +) + py_binary( name = "release", srcs = ["release.py"], main = "release.py", deps = [ + ":changelog_news", "@dev_pip//packaging", ], ) diff --git a/tools/private/release/changelog_news.py b/tools/private/release/changelog_news.py new file mode 100644 index 0000000000..b896af1f6b --- /dev/null +++ b/tools/private/release/changelog_news.py @@ -0,0 +1,323 @@ +"""Utility functions for handling news entries and merging them into CHANGELOG.md.""" + +import pathlib +import re + +_UNRELEASED_TEMPLATE_BODY = """## Unreleased + +[unreleased]: https://github.com/bazel-contrib/rules_python/releases/tag/unreleased + +Unreleased changes are tracked as individual files in the [news/](./news) +directory, or view the [latest generated +changelog](https://rules-python.readthedocs.io/en/latest/changelog.html).""" + + +def _get_sub_category(content): + """Extracts the sub-category in parentheses from the entry content.""" + match = re.match(r"^(?:\*|-)\s*\(([^)]+)\)", content) + if match: + return match.group(1).lower() + return "" + + +def is_news_file(path): + """Checks if a file path is a valid news file.""" + path = pathlib.Path(path) + if not path.is_file(): + return False + if path.suffix != ".md": + return False + parts = path.name.split(".") + if len(parts) < 3: + return False + return True + + +def _get_news_files(news_dir): + """Returns a list of news files matching the ..md pattern.""" + news_path = pathlib.Path(news_dir) + if not news_path.exists(): + return [] + + return [p for p in news_path.iterdir() if is_news_file(p)] + + +def _parse_new_files(news_files): + """Parses news files and groups them by category.""" + entries = {} + for p in news_files: + if not is_news_file(p): + continue + parts = p.name.split(".") + category = parts[1].lower() + + content = p.read_text(encoding="utf-8").strip() + + if not content: + continue + + # Format as list item if not already + if not (content.startswith("* ") or content.startswith("- ")): + content = f"* {content}" + + if category not in entries: + entries[category] = [] + entries[category].append(content) + + return entries + + +def generate_release_block(version, release_date, news_entries): + """Generates the markdown block for the release.""" + header_version = version.replace(".", "-") + lines = [ + f"{{#v{header_version}}}", + f"## [{version}] - {release_date}", + "", + f"[{version}]: https://github.com/bazel-contrib/rules_python/releases/tag/{version}", + "", + ] + + # Standard categories in preferred order + category_order = ["removed", "changed", "fixed", "added"] + # Add any other categories found + for cat in news_entries: + if cat not in category_order: + category_order.append(cat) + + has_entries = False + for cat in category_order: + if cat in news_entries and news_entries[cat]: + has_entries = True + lines.append(f"{{#v{header_version}-{cat}}}") + lines.append(f"### {cat.capitalize()}") + + # Sort entries by sub-category, then by content + sorted_entries = sorted( + news_entries[cat], key=lambda e: (_get_sub_category(e), e) + ) + + for entry in sorted_entries: + lines.append(entry) + lines.append("") + + if not has_entries: + lines.append("No notable changes.") + lines.append("") + + return "\n".join(lines) + + +def _add_news_to_changelog(input_path, output_path, version, entries, release_date): + """Adds or merges news entries into CHANGELOG.md.""" + input_path = pathlib.Path(input_path) + output_path = pathlib.Path(output_path) + changelog_content = input_path.read_text(encoding="utf-8") + + if version == "unreleased": + header_version = "unreleased" + version_anchor = "{#unreleased}" + category_anchor_fmt = "{{#unreleased-{cat}}}" + category_anchor_pattern = r"\{#unreleased-(?P[a-z]+)\}" + else: + header_version = version.replace(".", "-") + version_anchor = f"{{#v{header_version}}}" + category_anchor_fmt = "{{#v" + header_version + "-{cat}}}" + category_anchor_pattern = ( + r"\{#v" + re.escape(header_version) + r"-(?P[a-z]+)\}" + ) + + version_exists = version_anchor in changelog_content + + if version_exists: + if not entries and version != "unreleased": + print( + f"Version {version} already exists and no news entries found" + " to merge. Doing nothing." + ) + output_path.write_text(changelog_content, encoding="utf-8") + return + + print(f"Version {version} already exists in changelog. Merging news entries...") + # Extract the existing version block + pattern = ( + r"(?P" + + re.escape(version_anchor) + + r")(?P.*?)(?=\n\s*\{#v\d+-\d+-\d+\}|\Z)" + ) + match = re.search(pattern, changelog_content, re.DOTALL) + if not match: + raise RuntimeError( + f"Could not find content for existing version {version} in CHANGELOG.md" + ) + + content_block = match.group("content") + + # Strip the "Unreleased changes..." sentence for Unreleased preview + if version == "unreleased": + content_block = re.sub( + r"Unreleased\s+changes\s+are\s+tracked\s+as\s+individual\s+files\s+in\s+the\s+\[news/\]\(\./news\)\s+directory,\s+or\s+view\s+the\s+\[latest\s+generated\s+changelog\]\(https://rules-python\.readthedocs\.io/en/latest/changelog\.html\)\.\s*\n*", + "", + content_block, + ) + + # Parse existing categories + match_cat = re.search(category_anchor_pattern, content_block) + if match_cat: + header_end_idx = match_cat.start() + header_str = content_block[:header_end_idx] + categories_str = content_block[header_end_idx:] + else: + header_str = content_block + categories_str = "" + + existing_entries = {} + if categories_str: + cat_matches = list(re.finditer(category_anchor_pattern, categories_str)) + for i, m in enumerate(cat_matches): + cat = m.group("cat") + start_idx = m.end() + end_idx = ( + cat_matches[i + 1].start() + if i + 1 < len(cat_matches) + else len(categories_str) + ) + cat_content = categories_str[start_idx:end_idx].strip() + + lines = cat_content.splitlines() + cat_entries = [] + current_entry = [] + for line in lines: + if not line.strip() or line.strip().startswith("### "): + continue + if line.startswith("* ") or line.startswith("- "): + if current_entry: + cat_entries.append("\n".join(current_entry)) + current_entry = [line] + else: + if current_entry: + current_entry.append(line) + if current_entry: + cat_entries.append("\n".join(current_entry)) + existing_entries[cat] = cat_entries + + # Merge news entries + merged_entries = dict(existing_entries) + for cat, cat_entries in entries.items(): + if cat not in merged_entries: + merged_entries[cat] = [] + merged_entries[cat].extend(cat_entries) + + # Reconstruct categories + reconstructed_lines = [] + category_order = ["removed", "changed", "fixed", "added"] + for cat in merged_entries: + if cat not in category_order: + category_order.append(cat) + + for cat in category_order: + if cat in merged_entries and merged_entries[cat]: + reconstructed_lines.append(category_anchor_fmt.format(cat=cat)) + reconstructed_lines.append(f"### {cat.capitalize()}") + + sorted_entries = sorted( + merged_entries[cat], key=lambda e: (_get_sub_category(e), e) + ) + + for entry in sorted_entries: + reconstructed_lines.append(entry) + reconstructed_lines.append("") + + new_categories_str = "\n".join(reconstructed_lines) + new_release_block = ( + header_str.rstrip() + "\n\n" + new_categories_str.strip() + "\n" + ) + if version == "unreleased" and not new_categories_str.strip(): + new_release_block = ( + header_str.rstrip() + "\n\nNo notable unreleased changes.\n" + ) + + # Replace in changelog_content + new_content = re.sub( + pattern, + r"\g\n" + new_release_block.strip() + "\n", + changelog_content, + count=1, + flags=re.DOTALL, + ) + output_path.write_text(new_content, encoding="utf-8") + + else: + print( + f"Version {version} does not exist in changelog. Creating new" + " release section from news entries..." + ) + new_release_block = generate_release_block(version, release_date, entries) + replacement = ( + f"{{#unreleased}}\n{_UNRELEASED_TEMPLATE_BODY}\n\n{new_release_block}\n" + ) + + # Replace the active Unreleased section (from {#unreleased} to the first release anchor) + pattern = ( + r"(?P\{#unreleased\})(?P.*?)(?=\n\s*\{#v\d+-\d+-\d+\}|\Z)" + ) + + if not re.search(pattern, changelog_content, re.DOTALL): + raise RuntimeError( + "Could not find active Unreleased section to replace in CHANGELOG.md" + ) + + new_content = re.sub( + pattern, + replacement, + changelog_content, + count=1, + flags=re.DOTALL, + ) + output_path.write_text(new_content, encoding="utf-8") + + +def merge_new_into_changelog( + changelog_path, + output_path, + news_dir, + version, + release_date, + delete_news=False, +): + """Merges news entries from news_dir into changelog_path and writes to output_path.""" + news_files = _get_news_files(news_dir) + entries = _parse_new_files(news_files) + _add_news_to_changelog( + input_path=changelog_path, + output_path=output_path, + version=version, + entries=entries, + release_date=release_date, + ) + if delete_news: + for p in news_files: + p.unlink() + if news_files: + print(f"Removed {len(news_files)} processed news files.") + + +def update_changelog( + version, + release_date, + changelog_path="CHANGELOG.md", + output_path=None, + news_dir="news", + delete_news=True, +): + """Performs the version replacements in CHANGELOG.md.""" + if output_path is None: + output_path = changelog_path + merge_new_into_changelog( + changelog_path=changelog_path, + output_path=output_path, + news_dir=news_dir, + version=version, + release_date=release_date, + delete_news=delete_news, + ) diff --git a/tools/private/release/release.py b/tools/private/release/release.py index 4f956bd56c..9cd949ae99 100644 --- a/tools/private/release/release.py +++ b/tools/private/release/release.py @@ -4,12 +4,13 @@ import datetime import fnmatch import os -import pathlib import re import subprocess from packaging.version import parse as parse_version +from tools.private.release import changelog_news + _EXCLUDE_PATTERNS = [ "./.git/*", "./.github/*", @@ -162,294 +163,6 @@ def determine_next_version(branch_name=None): return f"{major}.{minor}.{patch + 1}" -def _get_sub_category(content): - """Extracts the sub-category in parentheses from the entry content.""" - match = re.match(r"^(?:\*|-)\s*\(([^)]+)\)", content) - if match: - return match.group(1).lower() - return "" - - -def _get_news_files(news_dir): - """Returns a list of news files matching the ..md pattern.""" - news_path = pathlib.Path(news_dir) - if not news_path.exists(): - return [] - - valid_files = [] - for p in news_path.iterdir(): - if not p.is_file(): - continue - if p.suffix != ".md": - continue - parts = p.name.split(".") - if len(parts) < 3: - continue - valid_files.append(p) - - return valid_files - - -def _parse_new_files(news_files): - """Parses news files and groups them by category.""" - entries = {} - for p in news_files: - parts = p.name.split(".") - category = parts[1].lower() - - content = p.read_text(encoding="utf-8").strip() - - if not content: - continue - - # Format as list item if not already - if not (content.startswith("* ") or content.startswith("- ")): - content = f"* {content}" - - if category not in entries: - entries[category] = [] - entries[category].append(content) - - return entries - - -def generate_release_block(version, release_date, news_entries): - """Generates the markdown block for the release.""" - header_version = version.replace(".", "-") - lines = [ - f"{{#v{header_version}}}", - f"## [{version}] - {release_date}", - "", - f"[{version}]: https://github.com/bazel-contrib/rules_python/releases/tag/{version}", - "", - ] - - # Standard categories in preferred order - category_order = ["removed", "changed", "fixed", "added"] - # Add any other categories found - for cat in news_entries: - if cat not in category_order: - category_order.append(cat) - - for cat in category_order: - if cat in news_entries and news_entries[cat]: - lines.append(f"{{#v{header_version}-{cat}}}") - lines.append(f"### {cat.capitalize()}") - - # Sort entries by sub-category, then by content - sorted_entries = sorted( - news_entries[cat], key=lambda e: (_get_sub_category(e), e) - ) - - for entry in sorted_entries: - lines.append(entry) - lines.append("") - - return "\n".join(lines) - - -def _add_news_to_changelog(changelog_path, version, entries, release_date): - """Adds or merges news entries into CHANGELOG.md.""" - changelog_path_obj = pathlib.Path(changelog_path) - changelog_content = changelog_path_obj.read_text(encoding="utf-8") - - header_version = version.replace(".", "-") - version_anchor = f"{{#v{header_version}}}" - version_exists = version_anchor in changelog_content - - if version_exists: - if not entries: - print( - f"Version {version} already exists and no news entries found" - " to merge. Doing nothing." - ) - return - - print(f"Version {version} already exists in changelog. Merging news entries...") - # Extract the existing version block - # Match from the version anchor to the next version anchor (or end of file) - pattern = ( - r"(?P\{#v" - + re.escape(header_version) - + r"\})(?P.*?)(?=\n\s*\{#v(?!0-0-0)\d+-\d+-\d+\}|\Z)" - ) - match = re.search(pattern, changelog_content, re.DOTALL) - if not match: - raise RuntimeError( - f"Could not find content for existing version {version} in CHANGELOG.md" - ) - - content_block = match.group("content") - - # Split content_block into header and categories - category_anchor_pattern = ( - r"\{#v" + re.escape(header_version) + r"-(?P[a-z]+)\}" - ) - match_cat = re.search(category_anchor_pattern, content_block) - if match_cat: - header_end_idx = match_cat.start() - header_str = content_block[:header_end_idx] - categories_str = content_block[header_end_idx:] - else: - header_str = content_block - categories_str = "" - - # Parse existing categories - existing_entries = {} - if categories_str: - cat_matches = list(re.finditer(category_anchor_pattern, categories_str)) - for i, m in enumerate(cat_matches): - cat = m.group("cat") - start_idx = m.end() - end_idx = ( - cat_matches[i + 1].start() - if i + 1 < len(cat_matches) - else len(categories_str) - ) - cat_content = categories_str[start_idx:end_idx].strip() - - lines = cat_content.splitlines() - cat_entries = [] - current_entry = [] - for line in lines: - if not line.strip() or line.strip().startswith("### "): - continue - if line.startswith("* ") or line.startswith("- "): - if current_entry: - cat_entries.append("\n".join(current_entry)) - current_entry = [line] - else: - if current_entry: - current_entry.append(line) - if current_entry: - cat_entries.append("\n".join(current_entry)) - existing_entries[cat] = cat_entries - - # Merge news entries - merged_entries = dict(existing_entries) - for cat, cat_entries in entries.items(): - if cat not in merged_entries: - merged_entries[cat] = [] - merged_entries[cat].extend(cat_entries) - - # Reconstruct categories - reconstructed_lines = [] - category_order = ["removed", "changed", "fixed", "added"] - for cat in merged_entries: - if cat not in category_order: - category_order.append(cat) - - for cat in category_order: - if cat in merged_entries and merged_entries[cat]: - reconstructed_lines.append(f"{{#v{header_version}-{cat}}}") - reconstructed_lines.append(f"### {cat.capitalize()}") - - sorted_entries = sorted( - merged_entries[cat], key=lambda e: (_get_sub_category(e), e) - ) - - for entry in sorted_entries: - reconstructed_lines.append(entry) - reconstructed_lines.append("") - - new_categories_str = "\n".join(reconstructed_lines) - new_release_block = ( - header_str.rstrip() + "\n\n" + new_categories_str.strip() + "\n" - ) - - # Replace in changelog - new_content = re.sub( - pattern, - r"\g\n" + new_release_block.strip() + "\n", - changelog_content, - flags=re.DOTALL, - ) - changelog_path_obj.write_text(new_content, encoding="utf-8") - - else: - if entries: - print( - f"Version {version} does not exist in changelog. Creating new" - " release section from news entries..." - ) - # Extract template - template_match = re.search( - r"BEGIN_UNRELEASED_TEMPLATE\s*\n(.*?)\n\s*END_UNRELEASED_TEMPLATE", - changelog_content, - re.DOTALL, - ) - if not template_match: - raise RuntimeError( - "Could not find BEGIN_UNRELEASED_TEMPLATE in CHANGELOG.md" - ) - - unreleased_template = template_match.group(1).strip() - new_release_block = generate_release_block(version, release_date, entries) - - replacement = f"{unreleased_template}\n\n{new_release_block}\n" - - # Replace the active Unreleased section - pattern = r"(END_UNRELEASED_TEMPLATE\s*\n-->\s*\n)(.*?)(\n\s*\{#v(?!0-0-0)\d+-\d+-\d+\})" - - if not re.search(pattern, changelog_content, re.DOTALL): - raise RuntimeError( - "Could not find active Unreleased section to replace in" - " CHANGELOG.md" - ) - - new_content = re.sub( - pattern, - r"\g<1>" + replacement + r"\g<3>", - changelog_content, - flags=re.DOTALL, - ) - changelog_path_obj.write_text(new_content, encoding="utf-8") - else: - # Fallback to old behavior - print( - f"No news entries found and version {version} does not exist." - " Falling back to manual changelog update..." - ) - header_version = version.replace(".", "-") - lines = changelog_content.splitlines() - - new_lines = [] - after_template = False - before_already_released = True - for line in lines: - if "END_UNRELEASED_TEMPLATE" in line: - after_template = True - if re.match("#v[1-9]-", line): - before_already_released = False - - if after_template and before_already_released: - line = line.replace( - "## Unreleased", f"## [{version}] - {release_date}" - ) - line = line.replace("v0-0-0", f"v{header_version}") - line = line.replace("0.0.0", version) - - new_lines.append(line) - - changelog_path_obj.write_text("\n".join(new_lines), encoding="utf-8") - - -def update_changelog( - version, release_date, changelog_path="CHANGELOG.md", news_dir="news" -): - """Performs the version replacements in CHANGELOG.md.""" - news_files = _get_news_files(news_dir) - entries = _parse_new_files(news_files) - - _add_news_to_changelog(changelog_path, version, entries, release_date) - - # Delete news files after successful update - for p in news_files: - p.unlink() - if news_files: - print(f"Removed {len(news_files)} processed news files.") - - def replace_version_next(version): """Replaces all VERSION_NEXT_* placeholders with the new version.""" for filepath in _iter_version_placeholder_files(): @@ -506,7 +219,7 @@ def main(): print("Updating changelog ...") release_date = datetime.date.today().strftime("%Y-%m-%d") - update_changelog(version, release_date) + changelog_news.update_changelog(version, release_date) print("Replacing VERSION_NEXT placeholders ...") replace_version_next(version) From ca869843c39d54883b77896f86f9948c2b827eaf Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Fri, 19 Jun 2026 21:47:59 -0700 Subject: [PATCH 771/922] ci: enable bazel-diff (#3838) Enable bazel-diff to improve CI performance by skipping targets that aren't affected by a PR --- .bazelci/presubmit.yml | 2 ++ 1 file changed, 2 insertions(+) diff --git a/.bazelci/presubmit.yml b/.bazelci/presubmit.yml index 6366741313..2cac5b961e 100644 --- a/.bazelci/presubmit.yml +++ b/.bazelci/presubmit.yml @@ -25,6 +25,8 @@ buildifier: bazel: 7.x skip_in_bazel_downstream_pipeline: "Bazel 7 required" .reusable_config: &reusable_config + environment: + USE_BAZEL_DIFF: "1" build_targets: - "--" - "..." From b16ce73f67acc07fe131a15700999d27400ce85b Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Sat, 20 Jun 2026 20:35:09 -0700 Subject: [PATCH 772/922] fix: run bazel-diff in cquery mode under bazelci (#3839) This PR fixes the CI breakage on main where bazel-diff ran in plain query mode, ignoring target_compatible_with and attempting to run platform-incompatible tests. It re-introduces the java wrapper to inject cquery mode targeting only non-manual tests, and re-applies the visibility fix. --- .bazelci/presubmit.yml | 1 + tests/uv/lock/pyproject_toml/BUILD.bazel | 1 + 2 files changed, 2 insertions(+) diff --git a/.bazelci/presubmit.yml b/.bazelci/presubmit.yml index 2cac5b961e..147f752724 100644 --- a/.bazelci/presubmit.yml +++ b/.bazelci/presubmit.yml @@ -27,6 +27,7 @@ buildifier: .reusable_config: &reusable_config environment: USE_BAZEL_DIFF: "1" + EXP_USE_CQUERY: "true" build_targets: - "--" - "..." diff --git a/tests/uv/lock/pyproject_toml/BUILD.bazel b/tests/uv/lock/pyproject_toml/BUILD.bazel index a64fe50d14..d73d206293 100644 --- a/tests/uv/lock/pyproject_toml/BUILD.bazel +++ b/tests/uv/lock/pyproject_toml/BUILD.bazel @@ -27,4 +27,5 @@ diff_test( timeout = "short", file1 = ":requirements", file2 = "requirements.txt", + visibility = ["//tests/uv/lock:__pkg__"], ) From e9ecdba91a389d3cac5cbd96e7ad27d59abf3884 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Sat, 20 Jun 2026 20:47:50 -0700 Subject: [PATCH 773/922] ci: disable bazel-diff due to platform compatibility conflicts (#3840) This PR disables the bazel-diff target filtering to repair the CI. Since we make heavy use of target_compatible_with to restrict tests to specific platforms, and Bazel CI's bazel-diff integration isn't able to handle those. --- .bazelci/presubmit.yml | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/.bazelci/presubmit.yml b/.bazelci/presubmit.yml index 147f752724..79d2dfbc30 100644 --- a/.bazelci/presubmit.yml +++ b/.bazelci/presubmit.yml @@ -25,9 +25,10 @@ buildifier: bazel: 7.x skip_in_bazel_downstream_pipeline: "Bazel 7 required" .reusable_config: &reusable_config - environment: - USE_BAZEL_DIFF: "1" - EXP_USE_CQUERY: "true" + # USE_BAZEL_DIFF is disabled because target_compatible_with is used heavily, + # but BazelCI's bazel-diff integration errors when handling them. + # environment: + # USE_BAZEL_DIFF: "1" build_targets: - "--" - "..." From 01903fcbc66721961bbabaeb37bc8fed176bf9a5 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Sun, 21 Jun 2026 09:10:52 -0700 Subject: [PATCH 774/922] chore: warn on implicit __init__.py creation (#3841) Implicit `__init__.py` creation is deprecated and will be disabled by default in a future release. We need to warn users when their targets rely on this behavior so they can transition to explicit `__init__.py` files before the default changes and their builds break. Work towards #2945 --- news/2945.changed.md | 1 + python/private/py_executable.bzl | 19 +++++++++++++++++++ 2 files changed, 20 insertions(+) create mode 100644 news/2945.changed.md diff --git a/news/2945.changed.md b/news/2945.changed.md new file mode 100644 index 0000000000..c0ff002661 --- /dev/null +++ b/news/2945.changed.md @@ -0,0 +1 @@ +(binaries/tests) Added a deprecation warning for targets relying on implicit `__init__.py` creation. diff --git a/python/private/py_executable.bzl b/python/private/py_executable.bzl index 198ff9d548..04a43be917 100644 --- a/python/private/py_executable.bzl +++ b/python/private/py_executable.bzl @@ -1485,6 +1485,25 @@ def _get_base_runfiles_for_binary( app_runfiles = app_runfiles.build(ctx) if _should_create_init_files(ctx): + # buildifier: disable=print + print( + """ +====================================================================== +WARNING: Target {} is using implicit __init__.py creation. + This diabolic behavior is deprecated and will be disabled by default in a + future release. + See https://github.com/bazel-contrib/rules_python/issues/2945 + + Ensure all __init__.py files are explicitly created and + added to the srcs or deps of your targets. + + Disable implicit creation by setting: + legacy_create_init = 0 + on the target, or globally by setting: + --incompatible_default_to_explicit_init_py +====================================================================== + """.rstrip().format(ctx.label), + ) app_runfiles = _py_builtins.merge_runfiles_with_generated_inits_empty_files_supplier( ctx = ctx, runfiles = app_runfiles, From 1a15c3b4c01940e41ec3ab7371884714e7f37914 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Mon, 22 Jun 2026 16:50:25 -0700 Subject: [PATCH 775/922] fix(bootstrap): use retry loop for win32 version lookup in site init template (#3842) Align the Windows path resolution in the site initialization template with the bootstrap template to prevent flaky platform lookup errors on Windows 2022. This is achieved by wrapping the win32 version retrieval in a retrying loop. Fixes https://github.com/bazel-contrib/rules_python/issues/3721 --- news/win32_version_lookup.fixed.md | 3 +++ python/private/site_init_template.py | 10 +++++++++- 2 files changed, 12 insertions(+), 1 deletion(-) create mode 100644 news/win32_version_lookup.fixed.md diff --git a/news/win32_version_lookup.fixed.md b/news/win32_version_lookup.fixed.md new file mode 100644 index 0000000000..6f01c9e73c --- /dev/null +++ b/news/win32_version_lookup.fixed.md @@ -0,0 +1,3 @@ +Fixed a flaky error on Windows 2022 when looking up the win32 version during +site initialization by retrying the lookup +([#3721](https://github.com/bazel-contrib/rules_python/issues/3721)). diff --git a/python/private/site_init_template.py b/python/private/site_init_template.py index af80b22a24..12be98eb57 100644 --- a/python/private/site_init_template.py +++ b/python/private/site_init_template.py @@ -98,7 +98,15 @@ def _get_windows_path_with_unc_prefix(path): # Related doc: https://docs.microsoft.com/en-us/windows/win32/fileio/maximum-file-path-limitation?tabs=cmd#enable-long-paths-in-windows-10-version-1607-and-later import platform - if platform.win32_ver()[1] >= "10.0.14393": + win32_version = None + # Windows 2022 with Python 3.12.8 gives flakey errors, so try a couple times. + for _ in range(3): + try: + win32_version = platform.win32_ver()[1] + break + except (ValueError, KeyError): + pass + if win32_version and win32_version >= "10.0.14393": return path # import sysconfig only now to maintain python 2.6 compatibility From dd53218006120bac2c51276c144bb9c4b5915c15 Mon Sep 17 00:00:00 2001 From: Gleb Kolobkov Date: Tue, 23 Jun 2026 20:26:25 -0700 Subject: [PATCH 776/922] fix(rules): disambiguate slash target venv outputs (#3845) Fixes venv output naming for `py_binary` and `py_test` targets whose names contain path separators. Before this change, the venv output prefix was derived from the executable basename. Distinct targets such as `//:foo/tool` and `//:bar/tool` both have executable basename `tool`, so they could both declare the same package-relative venv output directory: ```text bazel-out/.../bin/_tool.venv ``` After this change, the venv output prefix is derived from the full target name with `/` replaced by `_`, so those targets use separate venv output directories such as: ```text _foo_tool.venv _bar_tool.venv ``` Added regression coverage for slash-named executable targets in the shared `py_binary` / `py_test` base-rule tests. Fixes #3844 --- news/slash-target-venv-output.fixed.md | 3 ++ python/private/py_executable.bzl | 5 ++- tests/base_rules/py_executable_base_tests.bzl | 39 +++++++++++++++++++ 3 files changed, 46 insertions(+), 1 deletion(-) create mode 100644 news/slash-target-venv-output.fixed.md diff --git a/news/slash-target-venv-output.fixed.md b/news/slash-target-venv-output.fixed.md new file mode 100644 index 0000000000..780b498db7 --- /dev/null +++ b/news/slash-target-venv-output.fixed.md @@ -0,0 +1,3 @@ +(rules) Fixed venv output paths for `py_binary` and `py_test` targets whose +names contain path separators so distinct targets with the same basename no +longer share the same venv output directory. diff --git a/python/private/py_executable.bzl b/python/private/py_executable.bzl index 04a43be917..11246ec513 100644 --- a/python/private/py_executable.bzl +++ b/python/private/py_executable.bzl @@ -306,6 +306,9 @@ def _create_executable( else: base_executable_name = executable.basename + # Venv outputs are package-relative, so preserve the full target name to + # avoid collisions between targets like foo/tool, bar/tool, and foo_tool. + venv_output_prefix = ctx.label.name venv = None # The check for stage2_bootstrap_template is to support legacy @@ -318,7 +321,7 @@ def _create_executable( ): venv = _create_venv( ctx, - output_prefix = base_executable_name, + output_prefix = venv_output_prefix, imports = imports, runtime_details = runtime_details, add_runfiles_root_to_sys_path = ( diff --git a/tests/base_rules/py_executable_base_tests.bzl b/tests/base_rules/py_executable_base_tests.bzl index 7531de4c30..f6b3c9bb60 100644 --- a/tests/base_rules/py_executable_base_tests.bzl +++ b/tests/base_rules/py_executable_base_tests.bzl @@ -519,6 +519,45 @@ def _test_py_runtime_info_provided_impl(env, target): _tests.append(_test_py_runtime_info_provided) +def _test_venv_output_prefix_with_path_separators(name, config): + rt_util.helper_target( + config.rule, + name = name + "/foo/tool", + srcs = ["main.py"], + main = "main.py", + ) + rt_util.helper_target( + config.rule, + name = name + "/bar/tool", + srcs = ["main.py"], + main = "main.py", + ) + rt_util.helper_target( + config.rule, + name = name + "/foo_tool", + srcs = ["main.py"], + main = "main.py", + ) + analysis_test( + name = name, + impl = _test_venv_output_prefix_with_path_separators_impl, + targets = { + "bar": name + "/bar/tool", + "foo": name + "/foo/tool", + "foo_underscore": name + "/foo_tool", + }, + ) + +def _test_venv_output_prefix_with_path_separators_impl(env, targets): + for target in [targets.foo, targets.bar, targets.foo_underscore]: + target = env.expect.that_target(target) + venv_file = "*/_{}.venv/*site-packages/bazel.pth".format( + target.meta.format_str("{name}"), + ) + target.runfiles().contains_predicate(matching.str_matches(venv_file)) + +_tests.append(_test_venv_output_prefix_with_path_separators) + def _test_windows_target_with_path_separators(name, config): rt_util.helper_target( config.rule, From 4b19581b31ac30ff037b62f59741639847c73e20 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Tue, 23 Jun 2026 21:23:28 -0700 Subject: [PATCH 777/922] chore: automate releases (#3833) This introduces workflows and tools to automate releases. The basic idea it to have an issue with tasks, and workflows invoke tools that parse and update the tasks in the issue. The basic release steps are: 1. Prepare a release (update changelog via PR) 2. Create a release branch after preparation 3. Backport changes into the release branch 4. Tag RCs 5. Promote to final Each step has a workflow. For now, most of the workflows require a manual trigger, to help prevent workflows from running wild as we flush out bugs and edge cases. The workflows almost entirely delegate to the `release` tool. There's basically one command per workflow. This makes it easy to encode specialized logic and run it locally for testing or reproduction. --- .agents/plans/release_automation_plan.md | 186 ++++ .agents/rules/github_actions_workflows.md | 10 + .../release_tracking_template.md | 26 + .github/workflows/cut_release_branch.yml | 37 + .github/workflows/generate_rc.yml | 39 + .../on_prepare_release_pr_merged.yml | 35 + .github/workflows/prepare_release.yml | 36 + .github/workflows/process_backports.yml | 40 + .github/workflows/promote_rc.yml | 39 + tests/tools/private/release/release_test.py | 40 +- tools/private/release/BUILD.bazel | 7 +- tools/private/release/gh.py | 184 ++++ tools/private/release/git.py | 126 +++ tools/private/release/release.py | 843 ++++++++++++++++-- tools/private/release/utils.py | 28 + 15 files changed, 1601 insertions(+), 75 deletions(-) create mode 100644 .agents/plans/release_automation_plan.md create mode 100644 .agents/rules/github_actions_workflows.md create mode 100644 .github/ISSUE_TEMPLATE/release_tracking_template.md create mode 100644 .github/workflows/cut_release_branch.yml create mode 100644 .github/workflows/generate_rc.yml create mode 100644 .github/workflows/on_prepare_release_pr_merged.yml create mode 100644 .github/workflows/prepare_release.yml create mode 100644 .github/workflows/process_backports.yml create mode 100644 .github/workflows/promote_rc.yml create mode 100644 tools/private/release/gh.py create mode 100644 tools/private/release/git.py create mode 100644 tools/private/release/utils.py diff --git a/.agents/plans/release_automation_plan.md b/.agents/plans/release_automation_plan.md new file mode 100644 index 0000000000..90d87bc2a3 --- /dev/null +++ b/.agents/plans/release_automation_plan.md @@ -0,0 +1,186 @@ +# Plan for Better `rules_python` Release Automation + +The current release process (as described in `RELEASING.md`) has good +automation *after* a tag is pushed (`release.yml` handles GitHub Release, BCR +PR, and PyPI publishing). However, the steps *leading up* to the tag push are +largely manual. + +This plan outlines a **code-centric, reactive architecture** to automate the +manual preparation, branching, and tagging phases. By moving all core Git and +GitHub CLI logic out of the YAML workflow files and into the Python release +tool (`release.py`), we ensure the release process is robust, locally testable, and +independent of GitHub Actions runner syntax. + +## Prerequisite: Release Tracking Issue + +Every release is tracked by a dedicated GitHub Issue titled `Release ` +(e.g., `Release 0.38.0`), which acts as the single source of truth and state +controller. Release tracking issues are uniquely identified by the +`type:release` label. + +* **Role:** Contains a checklist of completed and remaining steps. +* **Automation Hub:** Workflows reactively trigger on tracking issue modifications. +* **Template File:** The issue body structure is decoupled from the codebase and + resides in `.github/ISSUE_TEMPLATE/release_tracking_template.md`. This is a + standard GitHub issue template equipped with frontmatter so maintainers can + easily create tracking issues manually from the GitHub web UI or let the + automation generate them programmatically. +* **Available Commands:** Placed at the very end of the issue body inside a + collapsed HTML `
` section to keep the main checklist clean. It + contains direct web links to the corresponding manual GitHub Action + workflows. + +### Tracking Issue Checklist Syntax + +The checklist uses a strict, machine-readable syntax using a pipe `|` separator +to attach space-separated metadata keys to tasks. + +#### Main Tasks Checklist + +```markdown +# Release tasks +- [ ] Prepare Release | status=pending pr=#1234 +- [ ] Create Release branch +- [ ] Tag RC0 +- [ ] Tag Final +``` + +* **Prepare Release**: + * Initial: `- [ ] Prepare Release | status=awaiting-preparation` + * Phase 1 PR created: `- [ ] Prepare Release | status=pending pr=#` + * PR merged (Phase 2): `- [x] Prepare Release | status=done pr=# commit=` +* **Create Release branch**: + * Initial: `- [ ] Create Release branch` + * Created: `- [x] Create Release branch | status=done branch=release/X.Y commit=` +* **Tag RC0**: + * Initial: `- [ ] Tag RC0` + * Tagged: `- [x] Tag RC0 | status=done tag=vX.Y.Z-rc0 commit=` +* **Tag Final**: + * Initial: `- [ ] Tag Final` + * Tagged: `- [x] Tag Final | status=done tag=vX.Y.Z commit=` + +#### Backports Checklist + +Maintainers list PRs to backport under the `## Backports` section: + +```markdown +- [x] #1234 | status=done rc=rc1 commit=deadbeef +- [ ] #2345 | status=merge-conflict +- [ ] #3456 +``` + +* **Pending:** `- [ ] #3456` (or `- [ ] #3456 | status=pending`) +* **Succeeded Cherry-pick:** `- [x] #1234 | status=done rc=rc commit=` (with checkbox marked `- [x]` to show it has been successfully completed). +* **Conflicting Cherry-pick:** `- [ ] #2345 | status=merge-conflict` (Unchecked, i.e., remains `- [ ]` to indicate it is not complete. Gates subsequent RC tags until resolved or removed). +* **Unmerged PR Error:** `- [ ] #3456 | status=unmerged-pr` (Unchecked, i.e., remains `- [ ]` to indicate it is not complete. Gates subsequent RC tags until resolved or removed). + +--- + +## Release Tool Commands + +The `release.py` script contains all execution logic. + +### 1. `determine-next-version` +* **Description:** Scans git tags and placeholders to determine the next release version. +* **Inputs:** None. +* **Outputs:** Prints the version string (e.g. `0.38.0`) to stdout. + +### 2. `create-release-issue` +* **Description:** Creates the tracking issue on GitHub using `release_tracking_template.md`. Exits with code `1` and prints a list of open tracking issue titles/URLs if a release is already in progress. +* **Inputs:** `--version ` (optional). + * *Version Resolution:* If not specified, determined automatically by calling `determine_next_version()`. + +### 3. `prepare` +* **Description:** Updates the changelog and placeholders. +* **Inputs:** `[version]` (optional), `--issue ` (optional). + * *Version Resolution:* If not specified, determined automatically by calling `determine_next_version()`. +* **Pre-checks:** + * If `--automation` is set, it first fetches upstream tags/commits and verifies that the workspace has **no uncommitted local edits**, exiting with code `1` if the workspace is dirty. +* **Automation Flag (`--automation`):** If set, it pushes to branch `prepare-{version}`, opens the preparation PR, and updates the tracking issue's `Prepare Release` task to `- [ ] Prepare Release | status=pending pr=#`. + +### 4. `complete-prepare` +* **Description:** Triggered when the prep PR merges. +* **Inputs:** `--pr ` (required), `--automation`. +* **Tracking Issue Resolution:** Automatically parses the tracking issue number + directly from the PR body (which links to the tracking issue). +* **State Updates:** + * Updates `Prepare Release` to: `status=done pr=# commit=` (checked). + +### 5. `create-release-branch` +* **Description:** Cuts the release branch. +* **Inputs:** `--issue ` (required), `--automation`. +* **State Updates:** + * Reads the `commit` SHA from the `Prepare Release` task. + * Cuts and pushes the `release/X.Y` branch. + * Updates `Create Release branch` to: `status=done branch=release/X.Y commit=` (checked). + +### 6. `process-backports` +* **Description:** Cherry-picks pending, merged backports. +* **Inputs:** `--issue ` (required), `--automation`. +* **Gating:** Resolves each backport PR. Unmerged PRs are marked as `status=unmerged-pr` (remains unchecked `- [ ]`) and the loop continues. Conflicting cherry-picks are marked as `status=merge-conflict` (remains unchecked `- [ ]`). If any PR is unmerged or cherry-pick fails with a conflict, the tool exits with code `1` at the end of the run. +* **State Updates:** + * Cherry-picks each pending, merged PR using `git cherry-pick -x` in chronological order. + * *Success:** Pushes to the release branch, updates the backport line to: `status=done rc=rc commit=` (with checkbox marked `- [x]` to show it has been successfully completed). + * *Conflict:* Aborts, updates the backport line to: `status=merge-conflict` (remains unchecked `- [ ]`). + * *Unmerged:* Updates the backport line to: `status=unmerged-pr` (remains unchecked `- [ ]`). + +### 7. `create-rc` +* **Description:** Tags the next RC. +* **Inputs:** `--issue ` (required), `--automation`. +* **Gating:** Fails if `Prepare Release` or `Create Release branch` are not `status=done`. Fails if any backport in the list is unchecked (`- [ ]`) or does not have `status=done`. +* **State Updates:** + * Queries git tags, increments to the next RC (e.g. `v0.38.0-rc0`), tags, and pushes. + * If tagging `rc0`, updates `Tag RC0` to: `status=done tag=vX.Y.Z-rc0 commit=` (checked). + * Announces the tag in an issue comment. + +### 8. `promote-rc` +* **Description:** Promotes the highest RC to final. +* **Inputs:** `[version]` (optional), `--issue ` (optional), `--automation`. + * *Version Resolution:* If not specified, determined automatically by finding the next version (which resolves to the active release version if it has not yet been tagged). + * *Issue Resolution:* If `--issue` is not specified, it searches for a single open tracking issue with the `type:release` label and matching version in the title. If zero or multiple tracking issues are found, the command errors and exits with code `1`. +* **State Updates:** + * Checks out the highest RC tag, tags `vX.Y.Z`, and pushes. + * Always attempts to update the tracking issue's `Tag Final` task to: `status=done tag=vX.Y.Z commit=` (checked), outputting a warning if the issue cannot be resolved. + +--- + +## GitHub Actions Workflows + +### 1. Prepare Release (`prepare_release.yml`) +* **Trigger:** Manual (`workflow_dispatch`). +* **Role:** Runs Phase 1. +* **Command:** `bazel run //tools/private/release -- --automation prepare`. +* **State Machine:** Transition: `Prepare Release` $\rightarrow$ `status=pending pr=#`. + +### 2. On PR Merged (`on_prepare_release_pr_merged.yml`) +* **Trigger:** `pull_request: [closed]` (merged, label `release-prepared`). +* **Role:** Completes Phase 1. +* **Command:** `bazel run //tools/private/release -- --automation complete-prepare --pr `. +* **State Machine:** Transition: `Prepare Release` $\rightarrow$ `status=done pr=# commit=` (checked). + +### 3. Cut Release Branch (`cut_release_branch.yml`) +* **Trigger:** `issues: [edited]` (filtered by label `type:release`). +* **Role:** Runs Phase 2 reactively. +* **Command:** `bazel run //tools/private/release -- --automation create-release-branch --issue `. +* **State Machine:** Transition: `Create Release branch` $\rightarrow$ `status=done branch=release/X.Y commit=` (checked). + +### 4. Process Backports (`process_backports.yml`) +* **Trigger:** Manual (`workflow_dispatch`), taking the tracking issue number. +* **Role:** Runs Phase 2.5 backport processing. +* **Command:** `bazel run //tools/private/release -- --automation process-backports --issue `. +* **State Machine:** + * Cherry-pick success: Backport PR $\rightarrow$ `status=done rc=rc commit=` (checked). + * Cherry-pick conflict: Backport PR $\rightarrow$ `status=merge-conflict` (unchecked). + * Unmerged PR error: Backport PR $\rightarrow$ `status=unmerged-pr` (unchecked). + +### 5. Generate RC Tag (`generate_rc.yml`) +* **Trigger:** Manual (`workflow_dispatch`), taking the tracking issue number. +* **Role:** Runs Phase 2.5 RC tagging. +* **Command:** `bazel run //tools/private/release -- --automation create-rc --issue `. +* **State Machine:** Transition: If tagging `rc0`, `Tag RC0` $\rightarrow$ `status=done tag=vX.Y.Z-rc0 commit=` (checked). + +### 6. Promote RC to Final (`promote_rc.yml`) +* **Trigger:** Manual (`workflow_dispatch`), taking the target version. +* **Role:** Runs Phase 3. +* **Command:** `bazel run //tools/private/release -- --automation promote-rc `. +* **State Machine:** Transition: `Tag Final` $\rightarrow$ `status=done tag=vX.Y.Z commit=` (checked). diff --git a/.agents/rules/github_actions_workflows.md b/.agents/rules/github_actions_workflows.md new file mode 100644 index 0000000000..29bf628a24 --- /dev/null +++ b/.agents/rules/github_actions_workflows.md @@ -0,0 +1,10 @@ +--- +name: github-actions-workflows +trigger: glob +globs: [".github/workflows/*.yml", ".github/workflows/*.yaml", ".github/*.yaml"] +--- + +# GitHub Actions Workflows Rule + +* When creating files in `.github/workflows/` (such as `.yml` or `.yaml` + files), always use the latest version of the referenced GitHub Actions. diff --git a/.github/ISSUE_TEMPLATE/release_tracking_template.md b/.github/ISSUE_TEMPLATE/release_tracking_template.md new file mode 100644 index 0000000000..66beea28b7 --- /dev/null +++ b/.github/ISSUE_TEMPLATE/release_tracking_template.md @@ -0,0 +1,26 @@ +--- +name: Release Tracking Issue +about: Checklist for tracking a new release of rules_python. +title: 'Release ' +labels: ['type:release'] +--- +# Release tasks +- [ ] Prepare Release | status=awaiting-preparation +- [ ] Create Release branch +- [ ] Tag RC0 +- [ ] Tag Final + +## Backports + + +--- +*Maintainers: Automation will react to changes on this issue.* + +
+Available Commands + +Maintainers can trigger automation by running manual workflows: +- [Process Backports Workflow](https://github.com/bazel-contrib/rules_python/actions/workflows/process_backports.yml) +- [Generate RC Tag Workflow](https://github.com/bazel-contrib/rules_python/actions/workflows/generate_rc.yml) +- [Promote RC to Final Release Workflow](https://github.com/bazel-contrib/rules_python/actions/workflows/promote_rc.yml) +
diff --git a/.github/workflows/cut_release_branch.yml b/.github/workflows/cut_release_branch.yml new file mode 100644 index 0000000000..b7c7118194 --- /dev/null +++ b/.github/workflows/cut_release_branch.yml @@ -0,0 +1,37 @@ +name: Cut Release Branch + +on: + issues: + types: [edited] + +permissions: + contents: write + issues: write + +jobs: + cut_branch: + # Run only if the issue has the type:release label + if: contains(github.event.issue.labels.*.name, 'type:release') + runs-on: ubuntu-latest + steps: + - name: Checkout repository + uses: actions/checkout@v7 + with: + fetch-depth: 0 + + - name: Setup Bazel + uses: bazel-contrib/setup-bazel@0.19.0 + with: + bazelisk-version: 1.20.0 + + - name: Configure Git Identity + run: | + git config --global user.name "github-actions[bot]" + git config --global user.email "41898282+github-actions[bot]@users.noreply.github.com" + + - name: Attempt Branch Creation + run: | + bazel run //tools/private/release -- \ + create-release-branch --issue ${{ github.event.issue.number }} + env: + GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} diff --git a/.github/workflows/generate_rc.yml b/.github/workflows/generate_rc.yml new file mode 100644 index 0000000000..26685c2f62 --- /dev/null +++ b/.github/workflows/generate_rc.yml @@ -0,0 +1,39 @@ +name: Generate RC Tag + +on: + workflow_dispatch: + inputs: + issue: + description: 'The Release Tracking Issue Number (e.g., 142)' + required: true + type: string + +permissions: + contents: write + issues: write + +jobs: + generate_rc: + runs-on: ubuntu-latest + steps: + - name: Checkout repository + uses: actions/checkout@v7 + with: + fetch-depth: 0 + + - name: Setup Bazel + uses: bazel-contrib/setup-bazel@0.19.0 + with: + bazelisk-version: 1.20.0 + + - name: Configure Git Identity + run: | + git config --global user.name "github-actions[bot]" + git config --global user.email "41898282+github-actions[bot]@users.noreply.github.com" + + - name: Attempt RC Tagging + run: | + bazel run //tools/private/release -- \ + create-rc --issue ${{ inputs.issue }} + env: + GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} diff --git a/.github/workflows/on_prepare_release_pr_merged.yml b/.github/workflows/on_prepare_release_pr_merged.yml new file mode 100644 index 0000000000..8ad4d1055c --- /dev/null +++ b/.github/workflows/on_prepare_release_pr_merged.yml @@ -0,0 +1,35 @@ +name: On PR Merged (Release Prepared) + +on: + pull_request: + types: [closed] + +permissions: + contents: write + issues: write + +jobs: + on_pr_merged: + # Run only if the release-prepared PR was merged + if: | + github.event.pull_request.merged == true && + contains(github.event.pull_request.labels.*.name, 'release-prepared') + runs-on: ubuntu-latest + steps: + - name: Checkout repository + uses: actions/checkout@v7 + with: + fetch-depth: 0 + + - name: Setup Bazel + uses: bazel-contrib/setup-bazel@0.19.0 + with: + bazelisk-version: 1.20.0 + + - name: Mark Prepare Release Complete + run: | + # Run the complete-prepare subcommand in the release tool to cleanly update checklist metadata + bazel run //tools/private/release -- \ + complete-prepare --pr ${{ github.event.pull_request.number }} + env: + GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} diff --git a/.github/workflows/prepare_release.yml b/.github/workflows/prepare_release.yml new file mode 100644 index 0000000000..7d5e93aa6b --- /dev/null +++ b/.github/workflows/prepare_release.yml @@ -0,0 +1,36 @@ +name: Prepare Release + +on: + workflow_dispatch: + # Allow manual triggering to prepare the release immediately + +permissions: + contents: write + issues: write + pull-requests: write + +jobs: + prepare_release: + runs-on: ubuntu-latest + steps: + - name: Checkout repository + uses: actions/checkout@v7 + with: + fetch-depth: 0 + + - name: Setup Bazel + uses: bazel-contrib/setup-bazel@0.19.0 + with: + bazelisk-version: 1.20.0 + + - name: Configure Git Identity + run: | + git config --global user.name "github-actions[bot]" + git config --global user.email "41898282+github-actions[bot]@users.noreply.github.com" + + - name: Run Release Preparation Pipeline + run: | + # Manual trigger: run full preparation + bazel run //tools/private/release -- prepare + env: + GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} diff --git a/.github/workflows/process_backports.yml b/.github/workflows/process_backports.yml new file mode 100644 index 0000000000..9a42b6c62c --- /dev/null +++ b/.github/workflows/process_backports.yml @@ -0,0 +1,40 @@ +name: Process Backports + +on: + workflow_dispatch: + inputs: + issue: + description: 'The Release Tracking Issue Number (e.g., 142)' + required: true + type: string + +permissions: + contents: write + issues: write + +jobs: + process_backports: + # Always gate GHA runs to ensure we are operating on a type:release labeled issue if metadata is queried + runs-on: ubuntu-latest + steps: + - name: Checkout repository + uses: actions/checkout@v7 + with: + fetch-depth: 0 + + - name: Setup Bazel + uses: bazel-contrib/setup-bazel@0.19.0 + with: + bazelisk-version: 1.20.0 + + - name: Configure Git Identity + run: | + git config --global user.name "github-actions[bot]" + git config --global user.email "41898282+github-actions[bot]@users.noreply.github.com" + + - name: Process Pending Backports + run: | + bazel run //tools/private/release -- \ + process-backports --issue ${{ inputs.issue }} + env: + GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} diff --git a/.github/workflows/promote_rc.yml b/.github/workflows/promote_rc.yml new file mode 100644 index 0000000000..d5e697f6d7 --- /dev/null +++ b/.github/workflows/promote_rc.yml @@ -0,0 +1,39 @@ +name: Promote RC to Final Release + +on: + workflow_dispatch: + inputs: + version: + description: 'The final version to release (e.g., 0.38.0)' + required: true + type: string + +permissions: + contents: write + issues: write + +jobs: + promote: + runs-on: ubuntu-latest + steps: + - name: Checkout repository + uses: actions/checkout@v7 + with: + fetch-depth: 0 + + - name: Setup Bazel + uses: bazel-contrib/setup-bazel@0.19.0 + with: + bazelisk-version: 1.20.0 + + - name: Configure Git Identity + run: | + git config --global user.name "github-actions[bot]" + git config --global user.email "41898282+github-actions[bot]@users.noreply.github.com" + + - name: Run Promote RC + run: | + bazel run //tools/private/release -- \ + promote-rc ${{ inputs.version }} + env: + GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} diff --git a/tests/tools/private/release/release_test.py b/tests/tools/private/release/release_test.py index 84c6abeaf3..9f9cd2e801 100644 --- a/tests/tools/private/release/release_test.py +++ b/tests/tools/private/release/release_test.py @@ -476,24 +476,26 @@ def test_replace_version_next_excludes_bazel_dirs(self): def test_valid_version(self): # These should not raise an exception - releaser.create_parser().parse_args(["0.28.0"]) - releaser.create_parser().parse_args(["1.0.0"]) - releaser.create_parser().parse_args(["1.2.3rc4"]) + releaser.create_parser().parse_args(["prepare", "0.28.0"]) + releaser.create_parser().parse_args(["promote-rc", "1.0.0"]) + releaser.create_parser().parse_args( + ["create-release-issue", "--version", "1.2.3rc4"] + ) def test_invalid_version(self): with self.assertRaises(SystemExit): - releaser.create_parser().parse_args(["0.28"]) + releaser.create_parser().parse_args(["prepare", "0.28"]) with self.assertRaises(SystemExit): - releaser.create_parser().parse_args(["a.b.c"]) + releaser.create_parser().parse_args(["prepare", "a.b.c"]) class GetLatestVersionTest(unittest.TestCase): - @patch("tools.private.release.release._get_git_tags") + @patch("tools.private.release.release.git.get_tags") def test_get_latest_version_success(self, mock_get_tags): mock_get_tags.return_value = ["0.1.0", "1.0.0", "0.2.0"] self.assertEqual(releaser.get_latest_version(), "1.0.0") - @patch("tools.private.release.release._get_git_tags") + @patch("tools.private.release.release.git.get_tags") def test_get_latest_version_rc_is_latest(self, mock_get_tags): mock_get_tags.return_value = ["0.1.0", "1.0.0", "1.1.0rc0"] with self.assertRaisesRegex( @@ -501,7 +503,7 @@ def test_get_latest_version_rc_is_latest(self, mock_get_tags): ): releaser.get_latest_version() - @patch("tools.private.release.release._get_git_tags") + @patch("tools.private.release.release.git.get_tags") def test_get_latest_version_no_tags(self, mock_get_tags): mock_get_tags.return_value = [] with self.assertRaisesRegex( @@ -509,7 +511,7 @@ def test_get_latest_version_no_tags(self, mock_get_tags): ): releaser.get_latest_version() - @patch("tools.private.release.release._get_git_tags") + @patch("tools.private.release.release.git.get_tags") def test_get_latest_version_no_matching_tags(self, mock_get_tags): mock_get_tags.return_value = ["v1.0", "latest"] with self.assertRaisesRegex( @@ -517,7 +519,7 @@ def test_get_latest_version_no_matching_tags(self, mock_get_tags): ): releaser.get_latest_version() - @patch("tools.private.release.release._get_git_tags") + @patch("tools.private.release.release.git.get_tags") def test_get_latest_version_only_rc_tags(self, mock_get_tags): mock_get_tags.return_value = ["1.0.0rc0", "1.1.0rc0"] with self.assertRaisesRegex( @@ -582,8 +584,8 @@ def test_both_markers(self): self.assertEqual(next_version, "1.3.0") - @patch("tools.private.release.release._get_current_branch") - @patch("tools.private.release.release._get_git_tags") + @patch("tools.private.release.release.git.get_current_branch") + @patch("tools.private.release.release.git.get_tags") def test_determine_next_version_on_release_branch_with_existing_tags( self, mock_get_tags, mock_get_branch ): @@ -594,8 +596,8 @@ def test_determine_next_version_on_release_branch_with_existing_tags( self.assertEqual(next_version, "0.37.2") - @patch("tools.private.release.release._get_current_branch") - @patch("tools.private.release.release._get_git_tags") + @patch("tools.private.release.release.git.get_current_branch") + @patch("tools.private.release.release.git.get_tags") def test_determine_next_version_on_release_branch_no_tags( self, mock_get_tags, mock_get_branch ): @@ -606,8 +608,8 @@ def test_determine_next_version_on_release_branch_no_tags( self.assertEqual(next_version, "0.38.0") - @patch("tools.private.release.release._get_current_branch") - @patch("tools.private.release.release._get_git_tags") + @patch("tools.private.release.release.git.get_current_branch") + @patch("tools.private.release.release.git.get_tags") def test_determine_next_version_on_release_branch_with_active_rc( self, mock_get_tags, mock_get_branch ): @@ -620,8 +622,8 @@ def test_determine_next_version_on_release_branch_with_active_rc( # Should target 0.37.0, not 0.37.1 self.assertEqual(next_version, "0.37.0") - @patch("tools.private.release.release._get_current_branch") - @patch("tools.private.release.release._get_git_tags") + @patch("tools.private.release.release.git.get_current_branch") + @patch("tools.private.release.release.git.get_tags") def test_determine_next_version_on_release_branch_with_stable_and_active_patch_rc( self, mock_get_tags, mock_get_branch ): @@ -634,7 +636,7 @@ def test_determine_next_version_on_release_branch_with_stable_and_active_patch_r # Should target 0.37.1, not 0.37.2 self.assertEqual(next_version, "0.37.1") - @patch("tools.private.release.release._get_current_branch") + @patch("tools.private.release.release.git.get_current_branch") def test_determine_next_version_on_main_branch_fallback(self, mock_get_branch): mock_get_branch.return_value = "main" # Should fallback to default behavior (which uses mock_get_latest_version from setUp) diff --git a/tools/private/release/BUILD.bazel b/tools/private/release/BUILD.bazel index 0afb23962e..747cb74e10 100644 --- a/tools/private/release/BUILD.bazel +++ b/tools/private/release/BUILD.bazel @@ -9,7 +9,12 @@ py_library( py_binary( name = "release", - srcs = ["release.py"], + srcs = [ + "gh.py", + "git.py", + "release.py", + "utils.py", + ], main = "release.py", deps = [ ":changelog_news", diff --git a/tools/private/release/gh.py b/tools/private/release/gh.py new file mode 100644 index 0000000000..9fa94eee20 --- /dev/null +++ b/tools/private/release/gh.py @@ -0,0 +1,184 @@ +"""GitHub CLI helper functions for the release tool.""" + +import json +import os +import tempfile + +from tools.private.release.utils import run_cmd + + +def get_open_tracking_issues(): + """Returns a list of open tracking issues with the 'type:release' label.""" + output = run_cmd( + "gh", + "issue", + "list", + "--label=type:release", + "--state=open", + "--json=number,title,url", + ) + return json.loads(output) if output else [] + + +def resolve_issue_number(version): + """Resolves the tracking issue number for a given version. + + Searches for an open issue with label 'type:release' and 'Release ' in the title. + Raises ValueError if 0 or multiple issues are found. + """ + matching_issues = [] + for issue in get_open_tracking_issues(): + if f"Release {version}" in issue["title"]: + matching_issues.append(issue) + + if not matching_issues: + raise ValueError(f"No open tracking issue found matching 'Release {version}'") + if len(matching_issues) > 1: + urls = [issue["url"] for issue in matching_issues] + raise ValueError( + f"Multiple open tracking issues found for version {version}:\n" + + "\n".join(urls) + ) + + return matching_issues[0]["number"] + + +def create_tracking_issue(version, template_content): + """Creates a new release tracking issue from template content (strips YAML frontmatter).""" + # Strip YAML frontmatter if present + issue_body = template_content + if template_content.startswith("---"): + parts = template_content.split("---", 2) + if len(parts) >= 3: + issue_body = parts[2].strip() + + # Write body to a secure temporary file to pass to the CLI + with tempfile.NamedTemporaryFile(mode="w", suffix=".md", delete=False) as f: + f.write(issue_body) + temp_path = f.name + + try: + output = run_cmd( + "gh", + "issue", + "create", + f"--title=Release {version}", + "--label=type:release", + f"--body-file={temp_path}", + ) + issue_url = output.strip() + issue_num = int(issue_url.split("/")[-1]) + return issue_num + finally: + if os.path.exists(temp_path): + os.unlink(temp_path) + + +def get_issue_body(issue_num): + """Fetches the body of a specific issue.""" + return run_cmd( + "gh", + "issue", + "view", + str(issue_num), + "--json=body", + "--jq=.body", + ) + + +def get_issue_title(issue_num): + """Fetches the title of a specific issue.""" + output = run_cmd( + "gh", + "issue", + "view", + str(issue_num), + "--json=title", + ) + return json.loads(output)["title"] if output else "" + + +def update_issue_body(issue_num, body): + """Updates the body of a specific issue.""" + with tempfile.NamedTemporaryFile(mode="w", suffix=".md", delete=False) as f: + f.write(body) + temp_path = f.name + try: + run_cmd( + "gh", + "issue", + "edit", + str(issue_num), + f"--body-file={temp_path}", + capture_output=False, + ) + finally: + if os.path.exists(temp_path): + os.unlink(temp_path) + + +def create_pr(version, branch, issue_num): + """Creates a pull request for release preparation.""" + return run_cmd( + "gh", + "pr", + "create", + f"--title=Prepare release v{version}", + f"--body=Work towards #{issue_num}", + f"--head={branch}", + "--base=main", + "--label=release-prepared", + ) + + +def get_pr_info(pr_num): + """Gets information about a PR, including state, merge commit, and body.""" + output = run_cmd( + "gh", + "pr", + "view", + str(pr_num), + "--json=state,mergeCommit,body", + ) + return json.loads(output) if output else {} + + +def post_issue_comment(issue_num, comment_body): + """Posts a comment to a specific issue.""" + run_cmd( + "gh", + "issue", + "comment", + str(issue_num), + f"--body={comment_body}", + capture_output=False, + ) + + +def resolve_backport_commits(pending_items): + """Resolves PR references in pending backports to their merge commit SHAs. + + Marks unmerged PRs or resolution failures with status='unmerged-pr'. + """ + resolved_items = [] + for item in pending_items: + pr_num = item["pr_ref"].lstrip("#") + print(f"Resolving PR #{pr_num} to merge commit...") + try: + pr_info = get_pr_info(pr_num) + if not pr_info or pr_info.get("state") != "MERGED": + state = pr_info.get("state", "UNKNOWN") + print(f"PR #{pr_num} is not merged (state: {state}). Gating.") + item["status"] = "unmerged-pr" + else: + merge_commit = pr_info.get("mergeCommit") + if merge_commit and "oid" in merge_commit: + item["commit"] = merge_commit["oid"] + else: + print(f"PR #{pr_num} has no merge commit SHA. Gating.") + item["status"] = "unmerged-pr" + except Exception as e: + print(f"Error resolving PR #{pr_num}: {e}. Gating.") + item["status"] = "unmerged-pr" + resolved_items.append(item) + return resolved_items diff --git a/tools/private/release/git.py b/tools/private/release/git.py new file mode 100644 index 0000000000..9bfd905109 --- /dev/null +++ b/tools/private/release/git.py @@ -0,0 +1,126 @@ +"""Git helper functions for the release tool.""" + +import subprocess + +from tools.private.release.utils import run_cmd + + +def get_tags(): + """Returns a list of all git tags in the repository.""" + output = run_cmd("git", "tag") + return output.splitlines() if output else [] + + +def checkout(ref, create_branch=False): + """Checks out a git reference (tag, branch, or commit).""" + if create_branch: + run_cmd("git", "checkout", "-b", ref, capture_output=False) + else: + run_cmd("git", "checkout", ref, capture_output=False) + + +def add(*files): + """Stages files for commit.""" + run_cmd("git", "add", *files, capture_output=False) + + +def commit(message, amend=False, no_edit=False): + """Commits staged changes, optionally amending the previous commit.""" + cmd = ["git", "commit"] + if amend: + cmd.append("--amend") + if no_edit: + cmd.append("--no-edit") + if message: + cmd.extend(["-m", message]) + run_cmd(*cmd, capture_output=False) + + +def push(remote, ref): + """Pushes a reference to a remote repository.""" + run_cmd("git", "push", remote, ref, capture_output=False) + + +def fetch(remote="origin", tags=False, force=False): + """Fetches updates from a remote repository.""" + cmd = ["git", "fetch", remote] + if tags: + cmd.append("--tags") + if force: + cmd.append("--force") + run_cmd(*cmd, capture_output=False) + + +def merge(commit_ref, ff_only=True): + """Merges a commit into the current branch.""" + cmd = ["git", "merge", commit_ref] + if ff_only: + cmd.append("--ff-only") + run_cmd(*cmd, capture_output=False) + + +def tag(tag_name): + """Creates a local tag pointing to HEAD.""" + run_cmd("git", "tag", tag_name, capture_output=False) + + +def cherry_pick(sha): + """Cherry-picks a commit using -x to append the original commit info.""" + run_cmd("git", "cherry-pick", "-x", sha, capture_output=False) + + +def cherry_pick_abort(): + """Aborts an in-progress cherry-pick operation.""" + run_cmd("git", "cherry-pick", "--abort", capture_output=False) + + +def status(): + """Returns the output of git status --porcelain.""" + return run_cmd("git", "status", "--porcelain") + + +def get_commit_sha(ref="HEAD", short=False): + """Returns the commit SHA of a given reference.""" + cmd = ["git", "rev-parse"] + if short: + cmd.append("--short") + cmd.append(ref) + return run_cmd(*cmd) + + +def branch_exists(branch_name): + """Returns True if a local branch exists.""" + try: + run_cmd("git", "show-ref", "--verify", f"refs/heads/{branch_name}") + return True + except subprocess.CalledProcessError: + return False + + +def tag_exists(tag_name): + """Returns True if a local tag exists.""" + try: + run_cmd("git", "show-ref", "--verify", f"refs/tags/{tag_name}") + return True + except subprocess.CalledProcessError: + return False + + +def sort_commits_chronologically(shas): + """Sorts a list of commit SHAs chronologically (oldest first).""" + output = run_cmd("git", "log", "--no-walk", "--reverse", "--format=%H", *shas) + return output.splitlines() if output else [] + + +def get_tags_at_head(): + """Returns a list of tags pointing at the current HEAD commit.""" + output = run_cmd("git", "tag", "--points-at", "HEAD") + return output.splitlines() if output else [] + + +def get_current_branch(): + """Returns the current git branch name, or None if not in a git repo.""" + try: + return run_cmd("git", "rev-parse", "--abbrev-ref", "HEAD") + except subprocess.CalledProcessError: + return None diff --git a/tools/private/release/release.py b/tools/private/release/release.py index 9cd949ae99..0e509f1bf3 100644 --- a/tools/private/release/release.py +++ b/tools/private/release/release.py @@ -4,12 +4,15 @@ import datetime import fnmatch import os +import pathlib import re -import subprocess +import sys from packaging.version import parse as parse_version -from tools.private.release import changelog_news +from tools.private.release import changelog_news, gh, git + +_REPO_URL = "https://github.com/bazel-contrib/rules_python" _EXCLUDE_PATTERNS = [ "./.git/*", @@ -23,6 +26,8 @@ "./tests/tools/private/release/*", ] +_RELEASE_TITLE_RE = re.compile(r"Release (\d+\.\d+\.\d+)", re.IGNORECASE) + def _iter_version_placeholder_files(): for root, dirs, files in os.walk(".", topdown=True): @@ -44,16 +49,9 @@ def _iter_version_placeholder_files(): yield filepath -def _get_git_tags(): - """Runs a git command and returns the output.""" - return subprocess.check_output(["git", "tag"]).decode("utf-8").splitlines() - - def get_latest_version(): """Gets the latest version from git tags.""" - tags = _get_git_tags() - # The packaging module can parse PEP440 versions, including RCs. - # It has a good understanding of version precedence. + tags = git.get_tags() versions = [ (tag, parse_version(tag)) for tag in tags @@ -68,15 +66,24 @@ def get_latest_version(): if latest_version.is_prerelease: raise ValueError(f"The latest version is a pre-release version: {latest_tag}") - # After all that, we only want to consider stable versions for the release. stable_versions = [tag for tag, version in versions if not version.is_prerelease] if not stable_versions: raise ValueError("No stable git tags found matching X.Y.Z format.") - # The versions are already sorted, so the last one is the latest. return stable_versions[-1] +def get_latest_rc_tag(version): + """Queries git tags and returns the highest RC tag for the version.""" + tags = git.get_tags() + pattern = rf"^v{re.escape(version)}-rc\d+$" + rc_tags = [tag.strip() for tag in tags if re.match(pattern, tag.strip())] + if not rc_tags: + return None + rc_tags.sort(key=parse_version) + return rc_tags[-1] + + def should_increment_minor(): """Checks if the minor version should be incremented.""" for filepath in _iter_version_placeholder_files(): @@ -84,7 +91,6 @@ def should_increment_minor(): with open(filepath, "r") as f: content = f.read() except (IOError, UnicodeDecodeError): - # Ignore binary files or files with read errors continue if "VERSION_NEXT_FEATURE" in content: @@ -92,25 +98,10 @@ def should_increment_minor(): return False -def _get_current_branch(): - """Returns the current git branch name, or None if not in a git repo.""" - try: - return ( - subprocess.check_output( - ["git", "rev-parse", "--abbrev-ref", "HEAD"], - stderr=subprocess.DEVNULL, - ) - .decode("utf-8") - .strip() - ) - except subprocess.CalledProcessError: - return None - - def determine_next_version(branch_name=None): """Determines the next version based on git tags and the current branch.""" if branch_name is None: - branch_name = _get_current_branch() + branch_name = git.get_current_branch() if branch_name: release_match = re.match(r"^release/(\d+)\.(\d+)$", branch_name) @@ -122,11 +113,7 @@ def determine_next_version(branch_name=None): f" {branch_major}.{branch_minor}.x)" ) - # Find all stable tags matching this major.minor prefix. - # Crucially, we ignore release candidates (RCs) here. If an RC is active - # (e.g. 0.37.0-rc0 exists but 0.37.0 stable does not), we want to continue - # targeting 0.37.0, NOT increment to 0.37.1. - tags = _get_git_tags() + tags = git.get_tags() matching_patches = [] for tag in tags: tag = tag.strip() @@ -144,8 +131,6 @@ def determine_next_version(branch_name=None): ) return next_version else: - # No stable tags exist yet for this release branch (preparing X.Y.0, - # even if X.Y.0-rcN tags already exist) next_version = f"{branch_major}.{branch_minor}.0" print( f"No stable tags found for {branch_major}.{branch_minor}.x." @@ -153,7 +138,6 @@ def determine_next_version(branch_name=None): ) return next_version - # Fallback to default behavior (for main branch or other development branches) latest_version = get_latest_version() major, minor, patch = [int(n) for n in latest_version.split(".")] @@ -170,7 +154,6 @@ def replace_version_next(version): with open(filepath, "r") as f: content = f.read() except (IOError, UnicodeDecodeError): - # Ignore binary files or files with read errors continue if "VERSION_NEXT_FEATURE" in content or "VERSION_NEXT_PATCH" in content: @@ -188,43 +171,793 @@ def _semver_type(value): return value +# ============================================================================== +# Checklist Parser and Formatter (Using new | key=value syntax) +# ============================================================================== + + +def parse_metadata_line(line): + """Parses a checklist line with optional | key=value metadata.""" + match = re.match(r"^\s*-\s*\[([ xX])\]\s+([^|]+)(?:\s*\|\s*(.*))?$", line) + if not match: + return None + + checked = match.group(1).lower() == "x" + name = match.group(2).strip() + metadata_str = match.group(3) + + metadata = {} + if metadata_str: + pairs = metadata_str.strip().split() + for pair in pairs: + if "=" in pair: + k, v = pair.split("=", 1) + metadata[k] = v + + return { + "checked": checked, + "name": name, + "metadata": metadata, + "original_line": line, + } + + +def format_metadata_line(checked, name, metadata): + """Formats a checklist line with space-separated key=value metadata.""" + check_str = "x" if checked else " " + if not metadata: + return f"- [{check_str}] {name}" + + metadata_str = " ".join(f"{k}={v}" for k, v in metadata.items()) + return f"- [{check_str}] {name} | {metadata_str}" + + +def update_task_in_body(body, task_name, checked, metadata): + """Updates a specific task's checked state and metadata in the issue body.""" + lines = body.splitlines() + updated_lines = [] + found = False + + for line in lines: + parsed = parse_metadata_line(line) + if parsed and parsed["name"].lower() == task_name.lower(): + updated_lines.append(format_metadata_line(checked, task_name, metadata)) + found = True + else: + updated_lines.append(line) + + if not found: + raise ValueError(f"Task '{task_name}' not found in issue body.") + + return "\n".join(updated_lines) + + +def parse_checklist_state(body): + """Parses the main checklist tasks and their metadata.""" + state = { + "prepare_release": { + "checked": False, + "status": None, + "pr": None, + "commit": None, + }, + "create_branch": { + "checked": False, + "status": None, + "branch": None, + "commit": None, + }, + "tag_final": {"checked": False, "status": None, "tag": None, "commit": None}, + "rc_tags": {}, # Dynamically mapped: int -> metadata dict + } + + lines = body.splitlines() + for line in lines: + parsed = parse_metadata_line(line) + if not parsed: + continue + + name = parsed["name"].strip() + meta = parsed["metadata"] + checked = parsed["checked"] + name_lower = name.lower() + + if "prepare release" in name_lower: + state["prepare_release"] = { + "checked": checked, + "status": meta.get("status"), + "pr": meta.get("pr"), + "commit": meta.get("commit"), + } + elif "create release branch" in name_lower: + state["create_branch"] = { + "checked": checked, + "status": meta.get("status"), + "branch": meta.get("branch"), + "commit": meta.get("commit"), + } + elif "tag final" in name_lower: + state["tag_final"] = { + "checked": checked, + "status": meta.get("status"), + "tag": meta.get("tag"), + "commit": meta.get("commit"), + } + else: + # Match Tag RC + rc_match = re.match(r"Tag RC(\d+)", name, re.IGNORECASE) + if rc_match: + rc_num = int(rc_match.group(1)) + state["rc_tags"][rc_num] = { + "checked": checked, + "status": meta.get("status"), + "tag": meta.get("tag"), + "commit": meta.get("commit"), + } + + return state + + +def parse_backports(body): + """Parses the ## Backports checklist section.""" + body = body.replace("\r\n", "\n") + match = re.search( + r"## Backports\n(.*?)(?=\n##|\n---|\Z)", body, re.DOTALL | re.IGNORECASE + ) + if not match: + return [] + + section_content = match.group(1) + items = [] + lines = section_content.splitlines() + + for line in lines: + parsed = parse_metadata_line(line) + if parsed: + items.append( + { + "pr_ref": parsed["name"], + "checked": parsed["checked"], + "status": parsed["metadata"].get("status", "PENDING"), + "rc": parsed["metadata"].get("rc"), + "commit": parsed["metadata"].get("commit"), + "metadata": parsed["metadata"], + } + ) + return items + + +# ============================================================================== +# Subcommand Execution Functions +# ============================================================================== + + +def cmd_determine_next_version(args): + """Executes the determine-next-version subcommand.""" + version = determine_next_version() + print(version) + return 0 + + +def cmd_create_release_issue(args): + """Executes the create-release-issue subcommand.""" + version = args.version + if version is None: + version = determine_next_version() + + # Concurrency check + open_issues = gh.get_open_tracking_issues() + if open_issues: + print("Error: A release is already in progress. Active tracking issues:") + for issue in open_issues: + print(f"- {issue['title']}: {issue['url']}") + return 1 + + template_path = pathlib.Path(".github/ISSUE_TEMPLATE/release_tracking_template.md") + if not template_path.exists(): + raise FileNotFoundError(f"Template file not found at {template_path}") + template_content = template_path.read_text(encoding="utf-8") + + issue_num = gh.create_tracking_issue(version, template_content) + print(f"Created tracking issue #{issue_num} for v{version}") + return 0 + + +def cmd_prepare(args): + """Executes the prepare subcommand.""" + print("Fetching upstream to verify fresh release history...") + git.fetch(tags=True, force=True) + + # Run pre-check: verify there are no local edits + status = git.status() + if status: + print( + "Error: Local edits detected. Workspace must be completely clean" + " before running release preparation." + ) + for line in status.splitlines(): + print(f" {line}") + return 1 + print("Pre-check passed: Workspace is clean.") + + version = args.version + if version is None: + version = determine_next_version() + + print(f"Running preparation pipeline for v{version}...") + + branch_name = f"prepare-{version}" + if git.branch_exists(branch_name): + print(f"Branch {branch_name} already exists. Checking it out...") + git.checkout(branch_name) + else: + git.checkout(branch_name, create_branch=True) + + print("Updating changelog and placeholders...") + release_date = datetime.date.today().strftime("%Y-%m-%d") + changelog_news.update_changelog(version, release_date) + replace_version_next(version) + + modified_files = git.status() + if not modified_files: + print("No files modified by the release tool. Nothing to commit.") + return 0 + + # Stage only modified files + for line in modified_files.splitlines(): + file_path = line.strip().split()[-1] + git.add(file_path) + + git.commit(f"Prepare release {version}") + git.push("origin", branch_name) + + issue_num = args.issue + if not issue_num: + open_issues = gh.get_open_tracking_issues() + for issue in open_issues: + if f"Release {version}" in issue["title"]: + issue_num = issue["number"] + break + + if not issue_num: + print( + f"No active tracking issue found for v{version}. Creating a new one..." + ) + template_path = pathlib.Path( + ".github/ISSUE_TEMPLATE/release_tracking_template.md" + ) + if not template_path.exists(): + raise FileNotFoundError(f"Template file not found at {template_path}") + template_content = template_path.read_text(encoding="utf-8") + issue_num = gh.create_tracking_issue(version, template_content) + + print(f"Using tracking issue #{issue_num}") + + pr_url = gh.create_pr(version, branch_name, issue_num) + pr_num = pr_url.split("/")[-1] + print(f"Created Pull Request: {pr_url} (PR #{pr_num})") + + print(f"Updating tracking issue #{issue_num} checklist status to PENDING...") + body = gh.get_issue_body(issue_num) + metadata = {"status": "pending", "pr": f"#{pr_num}"} + updated_body = update_task_in_body( + body, "Prepare Release", checked=False, metadata=metadata + ) + gh.update_issue_body(issue_num, updated_body) + print("Preparation pipeline completed successfully!") + return 0 + + +def cmd_complete_prepare(args): + """Executes the complete-prepare subcommand (Phase 2 PR merged).""" + print(f"Completing preparation for PR #{args.pr}...") + + pr_info = gh.get_pr_info(args.pr) + if not pr_info or pr_info.get("state") != "MERGED": + state = pr_info.get("state", "UNKNOWN") + print(f"Error: PR #{args.pr} is not merged yet (state: {state}).") + return 1 + + # Resolve issue number from PR body + pr_body = pr_info.get("body", "") + match = re.search(r"Work towards #(\d+)", pr_body) + if not match: + match = re.search(r"#(\d+)", pr_body) + if not match: + print( + f"Error: Could not determine tracking issue number from PR #{args.pr}" + f" body: {pr_body}" + ) + return 1 + + issue_num = int(match.group(1)) + print(f"Resolved tracking issue #{issue_num} from PR #{args.pr} body.") + + commit_sha = pr_info["mergeCommit"]["oid"] + short_commit = commit_sha[:8] + print(f"PR #{args.pr} merged at commit {commit_sha}. Updating tracking issue...") + + # Update checklist: mark Prepare Release as done (checked) and set SUCCESS + body = gh.get_issue_body(issue_num) + metadata = {"status": "done", "pr": f"#{args.pr}", "commit": short_commit} + updated_body = update_task_in_body( + body, "Prepare Release", checked=True, metadata=metadata + ) + gh.update_issue_body(issue_num, updated_body) + print("Prepare Release task marked complete successfully!") + return 0 + + +def cmd_create_release_branch(args): + """Executes the create-release-branch subcommand.""" + print(f"Evaluating branch creation for tracking issue #{args.issue}...") + body = gh.get_issue_body(args.issue) + state = parse_checklist_state(body) + + if ( + state["prepare_release"]["status"] != "done" + or not state["prepare_release"]["commit"] + ): + print( + "Error: Prepare Release task is not marked 'done' with a valid commit SHA." + ) + return 1 + + if state["create_branch"]["checked"]: + print("Release branch has already been created and checked. Skipping.") + return 0 + + # Extract version from issue title + issue_title = gh.get_issue_title(args.issue) + version_match = _RELEASE_TITLE_RE.search(issue_title) + if not version_match: + print(f"Error: Could not parse version from issue title: {issue_title}") + return 1 + + version = version_match.group(1) + branch_version = ".".join(version.split(".")[:2]) + branch_name = f"release/{branch_version}" + + commit_sha = state["prepare_release"]["commit"] + print(f"Cutting branch {branch_name} from commit {commit_sha}...") + + # Create and push branch + git.fetch("origin") + git.checkout(commit_sha) + + if not git.branch_exists(branch_name): + git.checkout(branch_name, create_branch=True) + else: + git.checkout(branch_name) + git.merge(commit_sha, ff_only=True) + + git.push("origin", branch_name) + print(f"Successfully pushed branch {branch_name}") + + # Update tracking issue checklist + print("Updating tracking issue checklist...") + metadata = {"status": "done", "branch": branch_name, "commit": commit_sha[:8]} + updated_body = update_task_in_body( + body, "Create Release branch", checked=True, metadata=metadata + ) + gh.update_issue_body(args.issue, updated_body) + print("Create Release branch task marked complete successfully!") + return 0 + + +def cmd_process_backports(args): + """Executes the process-backports subcommand.""" + body = gh.get_issue_body(args.issue) + items = parse_backports(body) + + pending_items = [ + item + for item in items + if not item["checked"] and item["status"] != "merge-conflict" + ] + + if not pending_items: + print("No pending backports found.") + return 0 + + print(f"Found {len(pending_items)} pending backports to process.") + + # Determine branch name from issue title + issue_title = gh.get_issue_title(args.issue) + version_match = _RELEASE_TITLE_RE.search(issue_title) + if not version_match: + print(f"Error: Could not parse version from issue title: {issue_title}") + return 1 + + version = version_match.group(1) + branch_version = ".".join(version.split(".")[:2]) + branch_name = f"release/{branch_version}" + + # Determine next RC tag to write to backport metadata + git.fetch("--tags", "--force") + latest_rc = get_latest_rc_tag(version) + if not latest_rc: + next_rc_suffix = "rc0" + else: + rc_num = int(latest_rc.split("-rc")[-1]) + next_rc_suffix = f"rc{rc_num + 1}" + + # Resolve PRs to merge commits using gh helper + resolved_items = gh.resolve_backport_commits(pending_items) + + shas = [] + sha_to_item = {} + any_failed = False + for item in resolved_items: + if item.get("commit"): + sha = item["commit"] + sha_to_item[sha] = item + shas.append(sha) + else: + any_failed = True + body = update_task_in_body( + body, + item["pr_ref"], + checked=False, + metadata={"status": item.get("status", "failed")}, + ) + gh.update_issue_body(args.issue, body) + + if not shas: + print("No valid merge commits to process.") + if any_failed: + return 1 + return 0 + + # Sort chronologically using git helper + sorted_shas = git.sort_commits_chronologically(shas) + + git.fetch("origin") + git.checkout(branch_name) + + for sha in sorted_shas: + item = sha_to_item[sha] + print(f"Cherry-picking {sha} (PR {item['pr_ref']})...") + try: + git.cherry_pick(sha) + + # Perform news processing (merging news/ files into the changelog) + print(f"Merging news fragments into changelog for PR {item['pr_ref']}...") + release_date = datetime.date.today().strftime("%Y-%m-%d") + changelog_news.update_changelog(version, release_date) + + # Stage changelog changes and news/ deletions + git.add("CHANGELOG.md", "news/") + + # Amend cherry-pick commit to include news merging and deletions + print(f"Amending cherry-pick commit for PR {item['pr_ref']}...") + git.commit("", amend=True, no_edit=True) + + # Push amended commit + git.push("origin", branch_name) + + new_sha = git.get_commit_sha("HEAD", short=True) + metadata = {"status": "done", "rc": next_rc_suffix, "commit": new_sha} + body = update_task_in_body( + body, item["pr_ref"], checked=True, metadata=metadata + ) + gh.update_issue_body(args.issue, body) + print(f"Applied: SUCCESS {new_sha}") + except Exception as e: + print(f"Conflict or error on {sha}: {e}. Aborting.") + try: + git.cherry_pick_abort() + except Exception: + pass + any_failed = True + + body = update_task_in_body( + body, + item["pr_ref"], + checked=False, + metadata={"status": "merge-conflict"}, + ) + gh.update_issue_body(args.issue, body) + print("Updated backport item to status=merge-conflict (unchecked)") + + if any_failed: + print("One or more cherry-picks/resolutions failed.") + return 1 + print("All backports successfully processed!") + return 0 + + +def cmd_create_rc(args): + """Executes the create-rc subcommand.""" + body = gh.get_issue_body(args.issue) + state = parse_checklist_state(body) + + if ( + state["prepare_release"]["status"] != "done" + or state["create_branch"]["status"] != "done" + ): + print( + "Error: Preconditions not met (release must be prepared and branch created)." + ) + return 1 + + # Gating: RC tagging is blocked if any backport is unchecked OR does not have status=done + backports = parse_backports(body) + conflicting_or_pending = [ + b for b in backports if not b["checked"] or b["status"] != "done" + ] + if conflicting_or_pending: + print( + f"Gating RC tagging: {len(conflicting_or_pending)} backports are still" + " unfinished, failed, or in conflict." + ) + return 1 + + # Resolve version and branch + issue_title = gh.get_issue_title(args.issue) + version_match = _RELEASE_TITLE_RE.search(issue_title) + if not version_match: + print(f"Error: Could not parse version from issue title: {issue_title}") + return 1 + + version = version_match.group(1) + branch_version = ".".join(version.split(".")[:2]) + branch_name = f"release/{branch_version}" + + # Determine next RC tag + git.fetch("--tags", "--force") + latest_rc = get_latest_rc_tag(version) + + if not latest_rc: + next_rc_num = 0 + next_rc = f"v{version}-rc0" + else: + rc_num = int(latest_rc.split("-rc")[-1]) + next_rc_num = rc_num + 1 + next_rc = f"v{version}-rc{next_rc_num}" + + # Precheck: next RC number must exist and be unchecked in the checklist + rc_tags = state.get("rc_tags", {}) + if next_rc_num not in rc_tags: + print( + f"Error: Checklist is missing required task 'Tag RC{next_rc_num}'" + f" to cut v{version}-rc{next_rc_num}." + ) + return 1 + + target_rc_task = rc_tags[next_rc_num] + if target_rc_task["checked"] or target_rc_task["status"] == "done": + print( + f"Error: Task 'Tag RC{next_rc_num}' is already marked done in the checklist." + ) + return 1 + + # Verify HEAD is not already tagged + git.checkout(branch_name) + head_tags = git.get_tags_at_head() + if any(tag.startswith(f"v{version}-rc") for tag in head_tags): + print(f"HEAD of {branch_name} is already tagged with an RC. Skipping.") + return 0 + + print(f"Tagging and pushing next RC: {next_rc}...") + git.tag(next_rc) + git.push("origin", next_rc) + + commit_sha = git.get_commit_sha("HEAD") + + # Check off the appropriate "Tag RC{N}" task in the checklist + print(f"Checking off Tag RC{next_rc_num} task...") + metadata = {"status": "done", "tag": next_rc, "commit": commit_sha[:8]} + task_name = f"Tag RC{next_rc_num}" + updated_body = update_task_in_body(body, task_name, checked=True, metadata=metadata) + gh.update_issue_body(args.issue, updated_body) + + tag_url = f"{_REPO_URL}/releases/tag/{next_rc}" + bcr_search_url = f"https://github.com/bazelbuild/bazel-central-registry/pulls?q=is%3Apr+rules_python+{version}" + comment_body = f"""🚀 **New Release Candidate Tagged!** + +Release Candidate **{next_rc}** has been successfully generated and tagged on branch `{branch_name}`. + +View Tag: [{next_rc}]({tag_url}) +Track BCR Progress: [Search BCR Pull Requests]({bcr_search_url})""" + gh.post_issue_comment(args.issue, comment_body) + print("RC creation completed successfully!") + return 0 + + +def cmd_promote_rc(args): + """Executes the promote-rc subcommand (Phase 3).""" + version = args.version + if version is None: + version = determine_next_version() + version = version.replace("v", "") + final_tag = f"v{version}" + + git.fetch("--tags", "--force") + latest_rc = get_latest_rc_tag(version) + if not latest_rc: + print(f"Error: No release candidate tags found matching v{version}-rc*") + return 1 + + print(f"Promoting {latest_rc} to final release {final_tag}...") + git.checkout(latest_rc) + + commit_sha = git.get_commit_sha("HEAD") + + if not git.tag_exists(final_tag): + git.tag(final_tag) + git.push("origin", final_tag) + else: + print(f"Final tag {final_tag} already exists.") + + # Resolve issue number + issue_num = args.issue + if not issue_num: + try: + issue_num = gh.resolve_issue_number(version) + except Exception as e: + print(f"Warning: Could not query GitHub to find tracking issue: {e}") + + if issue_num: + print(f"Updating tracking issue #{issue_num} checklist...") + body = gh.get_issue_body(issue_num) + metadata = {"status": "done", "tag": final_tag, "commit": commit_sha[:8]} + updated_body = update_task_in_body( + body, "Tag Final", checked=True, metadata=metadata + ) + gh.update_issue_body(issue_num, updated_body) + print("Checklist updated successfully.") + return 0 + else: + print( + "Error: No active tracking issue found or specified. Checklist was not updated." + ) + return 1 + + def create_parser(): - """Creates the argument parser.""" + """Creates the argument parser with subcommands.""" parser = argparse.ArgumentParser( description="Automate release steps for rules_python." ) - parser.add_argument( + + subparsers = parser.add_subparsers( + dest="command", required=True, help="Subcommands" + ) + + # Subcommand: determine-next-version + subparsers.add_parser( + "determine-next-version", + help="Determine the next version and print it, without making any changes.", + ) + + # Subcommand: create-release-issue + create_issue_parser = subparsers.add_parser( + "create-release-issue", + help="Search for open releases and create a new tracking issue.", + ) + create_issue_parser.add_argument( + "--version", + type=_semver_type, + help="The release version (e.g., 0.38.0). If not provided, determined automatically.", + ) + + # Subcommand: prepare + prepare_parser = subparsers.add_parser( + "prepare", + help="Prepare the release (updates changelog, placeholders).", + ) + prepare_parser.add_argument( "version", nargs="?", type=_semver_type, help="The new release version (e.g., 0.28.0). If not provided, " "it will be determined automatically.", ) + prepare_parser.add_argument( + "--issue", + type=int, + help="The tracking issue number (optional, triggers automated branch/PR pipeline).", + ) + + # Subcommand: complete-prepare + complete_prep_parser = subparsers.add_parser( + "complete-prepare", + help="Mark the Prepare Release task as complete in the tracking issue.", + ) + complete_prep_parser.add_argument( + "--pr", + type=int, + required=True, + help="The merged preparation PR number.", + ) + + # Subcommand: create-release-branch + create_branch_parser = subparsers.add_parser( + "create-release-branch", + help="Create the release branch pointing to the merged PR commit.", + ) + create_branch_parser.add_argument( + "--issue", + type=int, + required=True, + help="The tracking issue number (required).", + ) + + # Subcommand: process-backports + process_backports_parser = subparsers.add_parser( + "process-backports", + help="Cherry-pick pending backports listed in the tracking issue.", + ) + process_backports_parser.add_argument( + "--issue", + type=int, + required=True, + help="The tracking issue number (required).", + ) + + # Subcommand: create-rc + create_rc_parser = subparsers.add_parser( + "create-rc", + help="Tags the next RC on the release branch if no backports remain.", + ) + create_rc_parser.add_argument( + "--issue", + type=int, + required=True, + help="The tracking issue number (required).", + ) + + # Subcommand: promote-rc + promote_parser = subparsers.add_parser( + "promote-rc", + help="Promote the latest RC to final release.", + ) + promote_parser.add_argument( + "version", + nargs="?", + type=_semver_type, + help="The final version to release (e.g., 0.38.0).", + ) + promote_parser.add_argument( + "--issue", + type=int, + help="The tracking issue number (optional).", + ) + return parser def main(): - # Change to the workspace root so the script can be run using `bazel run` if "BUILD_WORKSPACE_DIRECTORY" in os.environ: os.chdir(os.environ["BUILD_WORKSPACE_DIRECTORY"]) parser = create_parser() args = parser.parse_args() - version = args.version - if version is None: - print("No version provided, determining next version automatically...") - version = determine_next_version() - print(f"Determined next version: {version}") - - print("Updating changelog ...") - release_date = datetime.date.today().strftime("%Y-%m-%d") - changelog_news.update_changelog(version, release_date) - - print("Replacing VERSION_NEXT placeholders ...") - replace_version_next(version) - - print("Done") + exit_code = 1 + try: + if args.command == "determine-next-version": + exit_code = cmd_determine_next_version(args) + elif args.command == "create-release-issue": + exit_code = cmd_create_release_issue(args) + elif args.command == "prepare": + exit_code = cmd_prepare(args) + elif args.command == "complete-prepare": + exit_code = cmd_complete_prepare(args) + elif args.command == "create-release-branch": + exit_code = cmd_create_release_branch(args) + elif args.command == "process-backports": + exit_code = cmd_process_backports(args) + elif args.command == "create-rc": + exit_code = cmd_create_rc(args) + elif args.command == "promote-rc": + exit_code = cmd_promote_rc(args) + except Exception as e: + print(f"Fatal error executing {args.command}: {e}", file=sys.stderr) + sys.exit(1) + + sys.exit(exit_code if exit_code is not None else 0) if __name__ == "__main__": diff --git a/tools/private/release/utils.py b/tools/private/release/utils.py new file mode 100644 index 0000000000..5a8411b657 --- /dev/null +++ b/tools/private/release/utils.py @@ -0,0 +1,28 @@ +"""Utility functions for the release tool.""" + +import subprocess + + +def run_cmd(*args, check=True, capture_output=True): + """Runs a command as a subprocess with separate arguments (prints command). + + If the command fails, it raises the CalledProcessError after attaching + a detailed note explaining the failure to preserve the stack trace. + """ + cmd = [str(arg) for arg in args] + print(f"Running: {' '.join(cmd)}") + try: + result = subprocess.run( + cmd, + check=check, + stdout=subprocess.PIPE if capture_output else None, + stderr=subprocess.PIPE if capture_output else None, + universal_newlines=True, + ) + return result.stdout.strip() if capture_output else None + except subprocess.CalledProcessError as e: + note = f"Error running command: {' '.join(cmd)}" + if capture_output: + note += f"\nStdout: {e.stdout}\nStderr: {e.stderr}" + e.add_note(note) + raise From 913201d0dc94dbc331b92a69e5d0e7aad33dcd65 Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Tue, 23 Jun 2026 21:46:00 -0700 Subject: [PATCH 778/922] build(deps): bump actions/checkout from 6 to 7 (#3843) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Bumps [actions/checkout](https://github.com/actions/checkout) from 6 to 7.
Release notes

Sourced from actions/checkout's releases.

v7.0.0

What's Changed

New Contributors

Full Changelog: https://github.com/actions/checkout/compare/v6.0.3...v7.0.0

v6.0.3

What's Changed

New Contributors

Full Changelog: https://github.com/actions/checkout/compare/v6...v6.0.3

v6.0.2

What's Changed

Full Changelog: https://github.com/actions/checkout/compare/v6.0.1...v6.0.2

v6.0.1

What's Changed

Full Changelog: https://github.com/actions/checkout/compare/v6...v6.0.1

Changelog

Sourced from actions/checkout's changelog.

Changelog

v7.0.0

v6.0.3

v6.0.2

v6.0.1

v6.0.0

v5.0.1

v5.0.0

v4.3.1

v4.3.0

v4.2.2

v4.2.1

... (truncated)

Commits

[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=actions/checkout&package-manager=github_actions&previous-version=6&new-version=7)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) ---
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot show ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> --- .github/workflows/ci.yaml | 4 ++-- .github/workflows/release.yml | 4 ++-- 2 files changed, 4 insertions(+), 4 deletions(-) diff --git a/.github/workflows/ci.yaml b/.github/workflows/ci.yaml index e324bfbf2a..daff8938e7 100644 --- a/.github/workflows/ci.yaml +++ b/.github/workflows/ci.yaml @@ -21,7 +21,7 @@ jobs: runs-on: ubuntu-latest steps: # Checkout the code - - uses: actions/checkout@v6 + - uses: actions/checkout@v7 - uses: jpetrucciani/mypy-check@master with: path: 'python/runfiles' @@ -31,7 +31,7 @@ jobs: ruff: runs-on: ubuntu-latest steps: - - uses: actions/checkout@v6 + - uses: actions/checkout@v7 - uses: astral-sh/ruff-action@v4.0.0 with: # Keep in sync with .pre-commit-config.yaml diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index 1e4f26981b..039e56a2b0 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -40,7 +40,7 @@ jobs: runs-on: ubuntu-latest steps: - name: Checkout - uses: actions/checkout@v6 + uses: actions/checkout@v7 with: ref: ${{ github.ref_name }} - name: Create release archive and notes @@ -71,7 +71,7 @@ jobs: runs-on: ubuntu-latest steps: - name: Checkout - uses: actions/checkout@v6 + uses: actions/checkout@v7 with: ref: ${{ github.tag_name || github.ref_name }} - if: github.event_name == 'push' || github.event.inputs.publish_to_pypi From 038467656bf1649b53200c95f431ada4e7475c2b Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Tue, 23 Jun 2026 22:26:28 -0700 Subject: [PATCH 779/922] chore: remove accidentally committed release automation plan (#3846) This file was not intended to be tracked in the repository, so it has been removed. --- .agents/plans/release_automation_plan.md | 186 ----------------------- 1 file changed, 186 deletions(-) delete mode 100644 .agents/plans/release_automation_plan.md diff --git a/.agents/plans/release_automation_plan.md b/.agents/plans/release_automation_plan.md deleted file mode 100644 index 90d87bc2a3..0000000000 --- a/.agents/plans/release_automation_plan.md +++ /dev/null @@ -1,186 +0,0 @@ -# Plan for Better `rules_python` Release Automation - -The current release process (as described in `RELEASING.md`) has good -automation *after* a tag is pushed (`release.yml` handles GitHub Release, BCR -PR, and PyPI publishing). However, the steps *leading up* to the tag push are -largely manual. - -This plan outlines a **code-centric, reactive architecture** to automate the -manual preparation, branching, and tagging phases. By moving all core Git and -GitHub CLI logic out of the YAML workflow files and into the Python release -tool (`release.py`), we ensure the release process is robust, locally testable, and -independent of GitHub Actions runner syntax. - -## Prerequisite: Release Tracking Issue - -Every release is tracked by a dedicated GitHub Issue titled `Release ` -(e.g., `Release 0.38.0`), which acts as the single source of truth and state -controller. Release tracking issues are uniquely identified by the -`type:release` label. - -* **Role:** Contains a checklist of completed and remaining steps. -* **Automation Hub:** Workflows reactively trigger on tracking issue modifications. -* **Template File:** The issue body structure is decoupled from the codebase and - resides in `.github/ISSUE_TEMPLATE/release_tracking_template.md`. This is a - standard GitHub issue template equipped with frontmatter so maintainers can - easily create tracking issues manually from the GitHub web UI or let the - automation generate them programmatically. -* **Available Commands:** Placed at the very end of the issue body inside a - collapsed HTML `
` section to keep the main checklist clean. It - contains direct web links to the corresponding manual GitHub Action - workflows. - -### Tracking Issue Checklist Syntax - -The checklist uses a strict, machine-readable syntax using a pipe `|` separator -to attach space-separated metadata keys to tasks. - -#### Main Tasks Checklist - -```markdown -# Release tasks -- [ ] Prepare Release | status=pending pr=#1234 -- [ ] Create Release branch -- [ ] Tag RC0 -- [ ] Tag Final -``` - -* **Prepare Release**: - * Initial: `- [ ] Prepare Release | status=awaiting-preparation` - * Phase 1 PR created: `- [ ] Prepare Release | status=pending pr=#` - * PR merged (Phase 2): `- [x] Prepare Release | status=done pr=# commit=` -* **Create Release branch**: - * Initial: `- [ ] Create Release branch` - * Created: `- [x] Create Release branch | status=done branch=release/X.Y commit=` -* **Tag RC0**: - * Initial: `- [ ] Tag RC0` - * Tagged: `- [x] Tag RC0 | status=done tag=vX.Y.Z-rc0 commit=` -* **Tag Final**: - * Initial: `- [ ] Tag Final` - * Tagged: `- [x] Tag Final | status=done tag=vX.Y.Z commit=` - -#### Backports Checklist - -Maintainers list PRs to backport under the `## Backports` section: - -```markdown -- [x] #1234 | status=done rc=rc1 commit=deadbeef -- [ ] #2345 | status=merge-conflict -- [ ] #3456 -``` - -* **Pending:** `- [ ] #3456` (or `- [ ] #3456 | status=pending`) -* **Succeeded Cherry-pick:** `- [x] #1234 | status=done rc=rc commit=` (with checkbox marked `- [x]` to show it has been successfully completed). -* **Conflicting Cherry-pick:** `- [ ] #2345 | status=merge-conflict` (Unchecked, i.e., remains `- [ ]` to indicate it is not complete. Gates subsequent RC tags until resolved or removed). -* **Unmerged PR Error:** `- [ ] #3456 | status=unmerged-pr` (Unchecked, i.e., remains `- [ ]` to indicate it is not complete. Gates subsequent RC tags until resolved or removed). - ---- - -## Release Tool Commands - -The `release.py` script contains all execution logic. - -### 1. `determine-next-version` -* **Description:** Scans git tags and placeholders to determine the next release version. -* **Inputs:** None. -* **Outputs:** Prints the version string (e.g. `0.38.0`) to stdout. - -### 2. `create-release-issue` -* **Description:** Creates the tracking issue on GitHub using `release_tracking_template.md`. Exits with code `1` and prints a list of open tracking issue titles/URLs if a release is already in progress. -* **Inputs:** `--version ` (optional). - * *Version Resolution:* If not specified, determined automatically by calling `determine_next_version()`. - -### 3. `prepare` -* **Description:** Updates the changelog and placeholders. -* **Inputs:** `[version]` (optional), `--issue ` (optional). - * *Version Resolution:* If not specified, determined automatically by calling `determine_next_version()`. -* **Pre-checks:** - * If `--automation` is set, it first fetches upstream tags/commits and verifies that the workspace has **no uncommitted local edits**, exiting with code `1` if the workspace is dirty. -* **Automation Flag (`--automation`):** If set, it pushes to branch `prepare-{version}`, opens the preparation PR, and updates the tracking issue's `Prepare Release` task to `- [ ] Prepare Release | status=pending pr=#`. - -### 4. `complete-prepare` -* **Description:** Triggered when the prep PR merges. -* **Inputs:** `--pr ` (required), `--automation`. -* **Tracking Issue Resolution:** Automatically parses the tracking issue number - directly from the PR body (which links to the tracking issue). -* **State Updates:** - * Updates `Prepare Release` to: `status=done pr=# commit=` (checked). - -### 5. `create-release-branch` -* **Description:** Cuts the release branch. -* **Inputs:** `--issue ` (required), `--automation`. -* **State Updates:** - * Reads the `commit` SHA from the `Prepare Release` task. - * Cuts and pushes the `release/X.Y` branch. - * Updates `Create Release branch` to: `status=done branch=release/X.Y commit=` (checked). - -### 6. `process-backports` -* **Description:** Cherry-picks pending, merged backports. -* **Inputs:** `--issue ` (required), `--automation`. -* **Gating:** Resolves each backport PR. Unmerged PRs are marked as `status=unmerged-pr` (remains unchecked `- [ ]`) and the loop continues. Conflicting cherry-picks are marked as `status=merge-conflict` (remains unchecked `- [ ]`). If any PR is unmerged or cherry-pick fails with a conflict, the tool exits with code `1` at the end of the run. -* **State Updates:** - * Cherry-picks each pending, merged PR using `git cherry-pick -x` in chronological order. - * *Success:** Pushes to the release branch, updates the backport line to: `status=done rc=rc commit=` (with checkbox marked `- [x]` to show it has been successfully completed). - * *Conflict:* Aborts, updates the backport line to: `status=merge-conflict` (remains unchecked `- [ ]`). - * *Unmerged:* Updates the backport line to: `status=unmerged-pr` (remains unchecked `- [ ]`). - -### 7. `create-rc` -* **Description:** Tags the next RC. -* **Inputs:** `--issue ` (required), `--automation`. -* **Gating:** Fails if `Prepare Release` or `Create Release branch` are not `status=done`. Fails if any backport in the list is unchecked (`- [ ]`) or does not have `status=done`. -* **State Updates:** - * Queries git tags, increments to the next RC (e.g. `v0.38.0-rc0`), tags, and pushes. - * If tagging `rc0`, updates `Tag RC0` to: `status=done tag=vX.Y.Z-rc0 commit=` (checked). - * Announces the tag in an issue comment. - -### 8. `promote-rc` -* **Description:** Promotes the highest RC to final. -* **Inputs:** `[version]` (optional), `--issue ` (optional), `--automation`. - * *Version Resolution:* If not specified, determined automatically by finding the next version (which resolves to the active release version if it has not yet been tagged). - * *Issue Resolution:* If `--issue` is not specified, it searches for a single open tracking issue with the `type:release` label and matching version in the title. If zero or multiple tracking issues are found, the command errors and exits with code `1`. -* **State Updates:** - * Checks out the highest RC tag, tags `vX.Y.Z`, and pushes. - * Always attempts to update the tracking issue's `Tag Final` task to: `status=done tag=vX.Y.Z commit=` (checked), outputting a warning if the issue cannot be resolved. - ---- - -## GitHub Actions Workflows - -### 1. Prepare Release (`prepare_release.yml`) -* **Trigger:** Manual (`workflow_dispatch`). -* **Role:** Runs Phase 1. -* **Command:** `bazel run //tools/private/release -- --automation prepare`. -* **State Machine:** Transition: `Prepare Release` $\rightarrow$ `status=pending pr=#`. - -### 2. On PR Merged (`on_prepare_release_pr_merged.yml`) -* **Trigger:** `pull_request: [closed]` (merged, label `release-prepared`). -* **Role:** Completes Phase 1. -* **Command:** `bazel run //tools/private/release -- --automation complete-prepare --pr `. -* **State Machine:** Transition: `Prepare Release` $\rightarrow$ `status=done pr=# commit=` (checked). - -### 3. Cut Release Branch (`cut_release_branch.yml`) -* **Trigger:** `issues: [edited]` (filtered by label `type:release`). -* **Role:** Runs Phase 2 reactively. -* **Command:** `bazel run //tools/private/release -- --automation create-release-branch --issue `. -* **State Machine:** Transition: `Create Release branch` $\rightarrow$ `status=done branch=release/X.Y commit=` (checked). - -### 4. Process Backports (`process_backports.yml`) -* **Trigger:** Manual (`workflow_dispatch`), taking the tracking issue number. -* **Role:** Runs Phase 2.5 backport processing. -* **Command:** `bazel run //tools/private/release -- --automation process-backports --issue `. -* **State Machine:** - * Cherry-pick success: Backport PR $\rightarrow$ `status=done rc=rc commit=` (checked). - * Cherry-pick conflict: Backport PR $\rightarrow$ `status=merge-conflict` (unchecked). - * Unmerged PR error: Backport PR $\rightarrow$ `status=unmerged-pr` (unchecked). - -### 5. Generate RC Tag (`generate_rc.yml`) -* **Trigger:** Manual (`workflow_dispatch`), taking the tracking issue number. -* **Role:** Runs Phase 2.5 RC tagging. -* **Command:** `bazel run //tools/private/release -- --automation create-rc --issue `. -* **State Machine:** Transition: If tagging `rc0`, `Tag RC0` $\rightarrow$ `status=done tag=vX.Y.Z-rc0 commit=` (checked). - -### 6. Promote RC to Final (`promote_rc.yml`) -* **Trigger:** Manual (`workflow_dispatch`), taking the target version. -* **Role:** Runs Phase 3. -* **Command:** `bazel run //tools/private/release -- --automation promote-rc `. -* **State Machine:** Transition: `Tag Final` $\rightarrow$ `status=done tag=vX.Y.Z commit=` (checked). From 63f77d3fc1cf0cd5f09a4d650998dd8d335b694c Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Wed, 24 Jun 2026 20:54:00 -0700 Subject: [PATCH 780/922] feat: unified pypi hub repository (#3837) This implements a pypi hub that is the union of all pypi hubs. The basic design is: * The `pip` extension _always_ creates a `@pypi` repo unless the name is already taken by another hub definition. * The `--pypi_hub` flag dispatches to one of the hubs. If not set, then it uses a default one (first, or as configured) The set of packages and targets the unified hub exposes is a union of all other hubs. If the unified hub routes to a hub that doesn't support such a target, then it points to a target that fails at execution time. This is to allow query and cquery to work even if some targets don't exist in some hubs. --- .agents/plans/pypi_hub_proxy_feature.md | 296 ++++++++++++++++++ .bazelignore | 1 + .bazelrc.deleted_packages | 1 + .../python/config_settings/index.md | 17 + docs/environment-variables.md | 13 + docs/pypi/download.md | 74 +++++ news/3837.added.md | 1 + python/config_settings/BUILD.bazel | 7 + python/features.bzl | 1 + python/private/common_labels.bzl | 1 + python/private/pypi/BUILD.bazel | 27 ++ python/private/pypi/extension.bzl | 115 ++++++- python/private/pypi/missing_package.bzl | 36 +++ python/private/pypi/unified_hub_repo.bzl | 81 +++++ python/private/pypi/unified_hub_setup.bzl | 110 +++++++ python/private/text_util.bzl | 12 + python/private/transition_labels.bzl | 1 + tests/integration/BUILD.bazel | 5 + .../bzlmod_lockfile/MODULE.bazel.lock | 2 +- tests/integration/unified_pypi/.bazelrc | 1 + tests/integration/unified_pypi/BUILD.bazel | 48 +++ tests/integration/unified_pypi/MODULE.bazel | 48 +++ tests/integration/unified_pypi/WORKSPACE | 1 + .../integration/unified_pypi/WORKSPACE.bzlmod | 1 + tests/integration/unified_pypi/bin_extra_b.py | 1 + tests/integration/unified_pypi/bin_six_a.py | 1 + .../unified_pypi/requirements_a.txt | 3 + .../unified_pypi/requirements_b.txt | 6 + tests/integration/unified_pypi/test_a.py | 3 + tests/integration/unified_pypi/test_cli.py | 3 + .../integration/unified_pypi/test_default.py | 3 + tests/integration/unified_pypi_test.py | 79 +++++ tests/pypi/extension/extension_tests.bzl | 152 +++++++++ 33 files changed, 1146 insertions(+), 5 deletions(-) create mode 100644 .agents/plans/pypi_hub_proxy_feature.md create mode 100644 news/3837.added.md create mode 100644 python/private/pypi/missing_package.bzl create mode 100644 python/private/pypi/unified_hub_repo.bzl create mode 100644 python/private/pypi/unified_hub_setup.bzl create mode 100644 tests/integration/unified_pypi/.bazelrc create mode 100644 tests/integration/unified_pypi/BUILD.bazel create mode 100644 tests/integration/unified_pypi/MODULE.bazel create mode 100644 tests/integration/unified_pypi/WORKSPACE create mode 100644 tests/integration/unified_pypi/WORKSPACE.bzlmod create mode 100644 tests/integration/unified_pypi/bin_extra_b.py create mode 100644 tests/integration/unified_pypi/bin_six_a.py create mode 100644 tests/integration/unified_pypi/requirements_a.txt create mode 100644 tests/integration/unified_pypi/requirements_b.txt create mode 100644 tests/integration/unified_pypi/test_a.py create mode 100644 tests/integration/unified_pypi/test_cli.py create mode 100644 tests/integration/unified_pypi/test_default.py create mode 100644 tests/integration/unified_pypi_test.py diff --git a/.agents/plans/pypi_hub_proxy_feature.md b/.agents/plans/pypi_hub_proxy_feature.md new file mode 100644 index 0000000000..b3360d9dbd --- /dev/null +++ b/.agents/plans/pypi_hub_proxy_feature.md @@ -0,0 +1,296 @@ +# Implementation Plan: Canonical Automatic PyPI Proxy Hub + +This document defines the locked, production-ready architectural, Starlark API, +and testing specifications for implementing dynamic PyPI dependency resolution in +`rules_python` using the `venv` flag. + +--- + +## 1. Architectural Strategy: The Canonical `@pypi` Proxy + +The `pip` bzlmod extension will automatically synthesize a canonical `@pypi` +proxy repository rule that orchestrates routing to underlying concrete hubs. + +### Bzlmod-Exclusive Scope + +The Unified PyPI Hub Proxy is an **exclusive feature of `bzlmod`**. Legacy +`WORKSPACE` evaluations using independent `pip_parse` repository macros are not +supported, as bzlmod's module extension architecture provides the required +centralized coordination to inspect and interlink cross-module hubs. + +### Automatic Proxy Construction & Collision Logic + +During the evaluation of the `pip` extension across the dependency graph: +1. **Unconditional Creation**: The extension will **always** synthesize a + proxy repository rule with the apparent name `pypi`, even if zero + `pip.parse` concrete hubs are defined in the dependency graph (in which + case the proxy is completely valid but empty). +2. **Collision Prevention**: If a user explicitly defines a concrete hub + named `pypi` (`pip.parse(hub_name = "pypi")`), the automatic proxy + synthesis is skipped so the user maintains absolute control over that + repository name. + +In `MODULE.bazel`: +```starlark +pip = use_extension("@rules_python//python/extensions:pip.bzl", "pip") + +# Concrete hubs defined for different execution contexts +pip.parse(hub_name = "pypi_a", ...) +pip.parse(hub_name = "pypi_b", ...) + +# Designate 'pypi_b' as the default hub for the unified '@pypi' repository +pip.default(default_hub = "pypi_b") + +# The canonical proxy is automatically created unconditionally: +use_repo(pip, "pypi") +``` + +### Unified PyPI Hub + +The canonical `@pypi` proxy repository matches exactly how concrete hubs create +their directory structure: a root package for shared configuration settings, and +a dedicated subdirectory (subpackage) for each PyPI package. + +Here is a complete, representative code example of what the generated files in +`@pypi` will look like when resolving packages between `pypi_a` and `pypi_b`: + +#### 1. `@pypi//BUILD.bazel` (Root Package) +The root package contains the shared `config_setting` targets following the +`_is_venv_` private naming convention. Leading underscores are strictly +applied because these configuration settings are an internal implementation +detail of the proxy repository and are not intended to be a public API. + +```starlark +package(default_visibility = ["//visibility:public"]) + +config_setting( + name = "_is_venv_pypi_a", + flag_values = { + "@rules_python//python/config_settings:venv": "pypi_a", + }, +) + +config_setting( + name = "_is_venv_pypi_b", + flag_values = { + "@rules_python//python/config_settings:venv": "pypi_b", + }, +) +``` + +#### 2. `@pypi//foo/BUILD.bazel` (PyPI Package Subpackage) +Each PyPI package subpackage defines the standard aliases (`pkg`, `whl`, `data`, +`dist_info`, `extracted_wheel_files`), plus a complete **union of all custom +`extra_hub_aliases`** defined across all concrete hubs. + +Each alias resolves dynamically to the active concrete hub based on the root +private configuration settings: + +```starlark +package(default_visibility = ["//visibility:public"]) + +alias( + name = "foo", + actual = ":pkg", +) + +alias( + name = "pkg", + actual = select({ + "//:_is_venv_pypi_a": "@pypi_a//foo:pkg", + "//:_is_venv_pypi_b": "@pypi_b//foo:pkg", + # When venv is "auto" (unset), it defaults to the designated fallback + # (or first defined concrete hub). + "//conditions:default": "@pypi_b//foo:pkg", + }), +) + +alias( + name = "whl", + actual = select({ + "//:_is_venv_pypi_a": "@pypi_a//foo:whl", + "//:_is_venv_pypi_b": "@pypi_b//foo:whl", + "//conditions:default": "@pypi_b//foo:whl", + }), +) + +# ... standard aliases for data, dist_info, extracted_wheel_files ... + +# 3. Unionized custom extra alias (defined in pypi_a but missing in pypi_b): +alias( + name = "my_custom_tool", + actual = select({ + "//:_is_venv_pypi_a": "@pypi_a//foo:my_custom_tool", + # Unrepresented branch routes to execution failure target: + "//:_is_venv_pypi_b": "//:_missing_package_error_pypi_b_foo", + "//conditions:default": "@pypi_a//foo:my_custom_tool", + }), +) +``` + +### Disjoint Hub Packages & Execution-Phase Failure + +If a package exists in one concrete hub but is missing in another (e.g., `scipy` +is in `pypi_b` but not `pypi_a`), our proxy synthesizes a package subpackage for +the union of all packages. + +To ensure that `bazel cquery` and `bazel query` successfully analyze over the +entire transitive build graph without failing, unrepresented select branches +must route to a dedicated **execution-phase error rule**. + +```starlark +# In @pypi//scipy/BUILD.bazel +alias( + name = "pkg", + actual = select({ + # Routes to execution-phase action failure target: + "//:_is_venv_pypi_a": "//:_missing_package_error_pypi_a_scipy", + "//:_is_venv_pypi_b": "@pypi_b//scipy:pkg", + "//conditions:default": "@pypi_b//scipy:pkg", + }), +) +``` + +The synthesized `//:_missing_package_error_XX` rule in `@pypi//BUILD.bazel` +returns standard Starlark Python providers so analysis/cquery passes, but +registers a build action that fails when executed: + +``` +Dependency Error: Third-party package 'scipy' is not available when building under PyPI hub 'pypi_a'. +``` + +### Fallback Hub Precedence (`"auto"`) + +When a target depends on `@pypi//foo` and the active build setting is `"auto"`, +the proxy resolves to a concrete hub using the following precedence: +1. **Designated Fallback**: If the user has explicitly designated a fallback + concrete hub via `pip.default(default_hub = "...")` in their root + `MODULE.bazel`, the proxy routes to it. +2. **First Defined Hub**: If no fallback is explicitly designated via + `pip.default()`, the proxy **automatically routes to the first defined + concrete hub** parsed during extension evaluation (e.g., `pypi_a`). + +```starlark +# Explicitly override the "auto" fallback hub +pip.default( + default_hub = "pypi_b", +) +``` + +--- + +## 2. Core Rule Integration: `config_settings` Transitions + +Users will switch active hubs using the standard, highly generic +`config_settings` transition attribute on executable targets. + +### Build Setting Definition + +In `python/config_settings/BUILD.bazel`: + +```starlark +string_flag( + name = "venv", + build_setting_default = "auto", # Default value is "auto" + visibility = ["//visibility:public"], +) +``` + +In `python/private/common_labels.bzl`: +```starlark + VENV = str(Label("//python/config_settings:venv")), +``` + +In `python/private/transition_labels.bzl`: +```starlark +_BASE_TRANSITION_LABELS = [ + # ... existing transition labels ... + labels.VENV, +] +``` + +Because `py_binary` and `py_test` implement an incoming transition +(`_transition_executable_impl`) that automatically processes any +`config_settings` keys matching `TRANSITION_LABELS`, **this provides complete +transition capabilities with zero changes to our core rule definitions**. + +### Usage in BUILD.bazel + +Libraries consume packages through the canonical proxy: + +```starlark +py_library( + name = "common", + deps = ["@pypi//foo"], # Apparent proxy repository +) +``` + +Binaries change the active hub by transitioning the build setting: + +```starlark +# Resolves @pypi -> pypi_b (default hub / designated fallback) +py_binary( + name = "bin_default", + deps = [":common"], +) + +# Resolves @pypi -> pypi_a via transition +py_binary( + name = "bin_a", + deps = [":common"], + config_settings = { + "//python/config_settings:venv": "pypi_a", + }, +) +``` + +### Analysis Cache & Memory Best Practices + +Because transitions fork the Bazel configuration, building targets with highly +diversified `config_settings` across large build graphs will result in +re-analysis and re-compilation of shared dependencies. + +We will include explicit documentation guidelines advising users to keep their +`venv` transition configurations localized and minimized to preserve Bazel +caching and memory efficiency. + +--- + +## 3. Integration Testing Specification + +We will construct a comprehensive Bazel-in-Bazel integration test suite in +`tests/integration/unified_pypi/` to guarantee correctness and verify +transitions. + +The integration test suite will assert: +1. **`"auto"` Precedence**: Author a test asserting `bazel run //:bin_default` + correctly inherits `"auto"` and resolves dependencies from the designated fallback. +2. **Transitional Resolution**: Author a test asserting two binary targets in + the same package with different `config_settings` successfully resolve + dependencies and execute against their respective concrete hubs (`pypi_a` + vs `pypi_b`). +3. **Command Line Override**: Author a test asserting + `bazel run --//python/config_settings:venv=pypi_a //:bin_default` + successfully forces the executable to run using imports resolved from + `pypi_a`. +4. **Disjoint Execution Failure**: Author a test asserting `bazel cquery` over + a target depending on an unrepresented missing package succeeds, while + `bazel run` on that target gracefully fails during execution with the exact + synthesized error message. +5. **Unionized Extra Hub Aliases**: Author a test asserting that a binary + successfully runs using a custom `extra_hub_aliases` target resolved + through the `@pypi proxy`. + +--- + +## 4. Execution Steps + +1. **Phase 1**: Define `venv` `string_flag` and register it in + `common_labels.bzl` and `transition_labels.bzl`. +2. **Phase 2**: Update `python/private/pypi/extension.bzl` to synthesize the + canonical `pypi` proxy repository rule. +3. **Phase 3**: Implement `missing_package_error` execution failure rule and + the `proxy_hub_repository` generation logic. +4. **Phase 4**: Author the Bazel-in-Bazel integration test suite in + `tests/integration/unified_pypi/`. +5. **Phase 5**: Run all tests and verify full pass before PR submission. diff --git a/.bazelignore b/.bazelignore index 2cf1523aef..5c3bb7caea 100644 --- a/.bazelignore +++ b/.bazelignore @@ -35,4 +35,5 @@ tests/integration/compile_pip_requirements/bazel-compile_pip_requirements tests/integration/local_toolchains/bazel-local_toolchains tests/integration/py_cc_toolchain_registered/bazel-py_cc_toolchain_registered tests/integration/toolchain_target_settings/bazel-module_under_test +tests/integration/unified_pypi/bazel-unified_pypi tests/integration/uv_lock/bazel-uv_lock diff --git a/.bazelrc.deleted_packages b/.bazelrc.deleted_packages index 407fd1cb48..ce42333e6f 100644 --- a/.bazelrc.deleted_packages +++ b/.bazelrc.deleted_packages @@ -40,6 +40,7 @@ common --deleted_packages=tests/integration/pip_parse_isolated common --deleted_packages=tests/integration/py_cc_toolchain_registered common --deleted_packages=tests/integration/runtime_manifests common --deleted_packages=tests/integration/toolchain_target_settings +common --deleted_packages=tests/integration/unified_pypi common --deleted_packages=tests/integration/uv_lock common --deleted_packages=tests/modules/another_module common --deleted_packages=tests/modules/other diff --git a/docs/api/rules_python/python/config_settings/index.md b/docs/api/rules_python/python/config_settings/index.md index cd3cbc9829..7908a28190 100644 --- a/docs/api/rules_python/python/config_settings/index.md +++ b/docs/api/rules_python/python/config_settings/index.md @@ -350,6 +350,22 @@ Values: :::: +::::{bzl:flag} venv +Determines which PyPI repository hub is used when resolving package dependencies. + +This flag is transitioned on automatically by executable targets (`py_binary`, `py_test`) +to select the appropriate concrete PyPI hub (e.g., when fallback or disjoint packages exist across multiple hubs). + +Values: +* `auto`: (default) Resolves dependencies using the fallback or first available hub. +* ``: Explicitly forces resolution of packages from the + specified concrete PyPI hub (corresponding to a + {obj}`pip.parse.hub_name` value). + +:::{versionadded} VERSION_NEXT_FEATURE +::: +:::: + ::::{bzl:flag} venvs_use_declare_symlink Determines if relative symlinks are created using `declare_symlink()` at build @@ -373,6 +389,7 @@ is created. ::: :::: + ## Removed Flags :::{versionremoved} 2.1.0 diff --git a/docs/environment-variables.md b/docs/environment-variables.md index 6c3444cba3..983ae3cb5f 100644 --- a/docs/environment-variables.md +++ b/docs/environment-variables.md @@ -118,6 +118,19 @@ which will effectively disable pyc caching. ::: +:::{envvar} RULES_PYTHON_PYPI_HUB_RESERVED + +When `1`, any PyPI hub named `"pypi"` will be renamed to `_pypi` +to prevent name collisions with the unified `@pypi` proxy repository, and +a warning is printed indicating that the renaming occurred. If not set (defaulting +to `0`), a warning is printed advising to rename the hub, and the collision +is not resolved. + +:::{versionadded} VERSION_NEXT_FEATURE +::: + +::: + :::{envvar} RULES_PYTHON_REPO_DEBUG When `1`, repository rules will print debug information about what they're diff --git a/docs/pypi/download.md b/docs/pypi/download.md index f0e70cf850..6705df1f3a 100644 --- a/docs/pypi/download.md +++ b/docs/pypi/download.md @@ -50,6 +50,79 @@ You can use the pip extension multiple times. This configuration will create multiple external repos that have no relation to one another and may result in downloading the same wheels numerous times. +(unified-pypi-hub)= +## Unified `@pypi` Hub for Multi-Hub Configurations + +:::{versionadded} VERSION_NEXT_FEATURE +Unified `@pypi` hub repository for Bzlmod multi-hub configurations. +::: + +When you call the `pip` extension multiple times with different `hub_name` +attributes, `rules_python` automatically generates a unified `@pypi` hub +repository (unless one of your concrete hubs is explicitly named `"pypi"`). + +This unified `@pypi` repository acts as a dynamic proxy that routes package +dependencies to the active concrete hub at build time. This is especially +useful in monorepos where shared library targets need to depend on PyPI +packages without knowing which specific hub or requirements lock file the +consuming binary is using. + +#### Reserved `"pypi"` Hub Name + +The hub name `"pypi"` is **reserved** for the automatically generated unified +hub repository. Defining a concrete hub named `"pypi"` will cause a collision. + +For details on how this collision is handled and resolved via environment +variables, see the {envvar}`RULES_PYTHON_PYPI_HUB_RESERVED` documentation. + +#### Configuring the Unified Hub + +To configure the unified hub, define your concrete hubs as usual, and +optionally designate a default hub using the `pip.default` tag's +`default_hub` attribute: + +```starlark +pip = use_extension("@rules_python//python/extensions:pip.bzl", "pip") + +# Define concrete hub 'pypi_a' +pip.parse( + hub_name = "pypi_a", + python_version = "3.11", + requirements_lock = "//:requirements_a.txt", +) + +# Define concrete hub 'pypi_b' +pip.parse( + hub_name = "pypi_b", + python_version = "3.11", + requirements_lock = "//:requirements_b.txt", +) + +# Designate 'pypi_b' as the default hub for the unified '@pypi' repository +pip.default(default_hub = "pypi_b") + +# Import the unified hub repository +use_repo(pip, "pypi") +``` + +#### Dynamic Routing at Build Time + +By default, the unified `@pypi` repository will resolve packages from the +designated `default_hub`. You can dynamically switch the active hub for a build +using the `--@rules_python//python/config_settings:venv` command-line flag +or via target transitions: + +```bash +# Build using packages from 'pypi_a' +bazel build --@rules_python//python/config_settings:venv=pypi_a //my:binary +``` + +Shared library targets can simply depend on the unified hub (e.g., +`@pypi//numpy`), and the dependency will automatically resolve to the correct +wheel version from the active hub during the build. + + + As with any repository rule or extension, if you would like to ensure that `pip_parse` is re-executed to pick up a non-hermetic change to your environment (e.g., updating your system `python` interpreter), you can force it to re-execute by running `bazel sync --only [pip_parse @@ -334,6 +407,7 @@ into whatever HTTP(S) request it performs against `example.com`. See the [Credential Helper Spec][cred-helper-spec] for more details. + [rfc7617]: https://datatracker.ietf.org/doc/html/rfc7617 [cred-helper-design]: https://github.com/bazelbuild/proposals/blob/main/designs/2022-06-07-bazel-credential-helpers.md [cred-helper-spec]: https://github.com/EngFlow/credential-helper-spec/blob/main/spec.md diff --git a/news/3837.added.md b/news/3837.added.md new file mode 100644 index 0000000000..6d3e4b5504 --- /dev/null +++ b/news/3837.added.md @@ -0,0 +1 @@ +(pypi) Added `@pypi` repo: a unified hub of `pip.parse` hubs. diff --git a/python/config_settings/BUILD.bazel b/python/config_settings/BUILD.bazel index 5b1317872f..369ce6de55 100644 --- a/python/config_settings/BUILD.bazel +++ b/python/config_settings/BUILD.bazel @@ -148,6 +148,13 @@ string_flag( # pip.parse related flags +string_flag( + name = "venv", + build_setting_default = "auto", + # NOTE: Only public because it is used in pip hub repos and executable transitions. + visibility = ["//visibility:public"], +) + string_flag( name = "pip_whl_osx_version", build_setting_default = "", diff --git a/python/features.bzl b/python/features.bzl index fab44385c8..f5d2c315cd 100644 --- a/python/features.bzl +++ b/python/features.bzl @@ -97,6 +97,7 @@ _TARGETS = { "//command_line_option:enable_runfiles": True, "//command_line_option:extra_toolchains": True, "//python/cc:current_py_cc_headers_abi3": True, + "//python/config_settings:venv": True, } _LOADABLE_SYMBOLS = { diff --git a/python/private/common_labels.bzl b/python/private/common_labels.bzl index a83ba2b462..135f8c0a1b 100644 --- a/python/private/common_labels.bzl +++ b/python/private/common_labels.bzl @@ -29,6 +29,7 @@ labels = struct( PY_FREETHREADED = str(Label("//python/config_settings:py_freethreaded")), PY_LINUX_LIBC = str(Label("//python/config_settings:py_linux_libc")), REPL_DEP = str(Label("//python/bin:repl_dep")), + VENV = str(Label("//python/config_settings:venv")), VENVS_SITE_PACKAGES = str(Label("//python/config_settings:venvs_site_packages")), VENVS_USE_DECLARE_SYMLINK = str(Label("//python/config_settings:venvs_use_declare_symlink")), VISIBLE_FOR_TESTING = str(Label("//python/private:visible_for_testing")), diff --git a/python/private/pypi/BUILD.bazel b/python/private/pypi/BUILD.bazel index b9a7a18aed..5e109a2a21 100644 --- a/python/private/pypi/BUILD.bazel +++ b/python/private/pypi/BUILD.bazel @@ -145,6 +145,7 @@ bzl_library( ":platform_bzl", ":pypi_cache_bzl", ":simpleapi_download_bzl", + ":unified_hub_repo_bzl", ":whl_library_bzl", "//python/private:auth_bzl", "//python/private:normalize_name_bzl", @@ -225,6 +226,15 @@ bzl_library( srcs = ["labels.bzl"], ) +bzl_library( + name = "missing_package_bzl", + srcs = ["missing_package.bzl"], + deps = [ + "//python/private:py_info_bzl", + "//python/private:reexports_bzl", + ], +) + bzl_library( name = "multi_pip_parse_bzl", srcs = ["multi_pip_parse.bzl"], @@ -445,6 +455,23 @@ bzl_library( ], ) +bzl_library( + name = "unified_hub_repo_bzl", + srcs = ["unified_hub_repo.bzl"], + deps = [ + "//python/private:text_util_bzl", + ], +) + +bzl_library( + name = "unified_hub_setup_bzl", + srcs = ["unified_hub_setup.bzl"], + deps = [ + ":labels_bzl", + ":missing_package_bzl", + ], +) + bzl_library( name = "urllib_bzl", srcs = ["urllib.bzl"], diff --git a/python/private/pypi/extension.bzl b/python/private/pypi/extension.bzl index a0b59e9ed1..5160b81ce8 100644 --- a/python/private/pypi/extension.bzl +++ b/python/private/pypi/extension.bzl @@ -28,6 +28,7 @@ load(":pip_repository_attrs.bzl", "ATTRS") load(":platform.bzl", _plat = "platform") load(":pypi_cache.bzl", "pypi_cache") load(":simpleapi_download.bzl", "simpleapi_download") +load(":unified_hub_repo.bzl", "unified_hub_repo") load(":whl_library.bzl", "whl_library") def _whl_mods_impl(whl_mods_dict): @@ -203,6 +204,7 @@ def build_config( Returns: A struct with the configuration. """ + default_hub = None defaults = { "platforms": default_platforms(), } @@ -211,6 +213,12 @@ def build_config( continue for tag in mod.tags.default: + if tag.default_hub: + if mod.is_root: + if default_hub: + fail("Duplicate pip.default tag: only one explicit default PyPI hub is allowed.") + default_hub = tag.default_hub + platform = tag.platform if platform: specific_config = defaults["platforms"].setdefault(platform, {}) @@ -244,6 +252,7 @@ def build_config( return struct( auth_patterns = defaults.get("auth_patterns", {}), + default_hub = default_hub, index_url = defaults.get("index_url", "https://pypi.org/simple").rstrip("/"), netrc = defaults.get("netrc", None), platforms = { @@ -343,9 +352,40 @@ You cannot use both the additive_build_content and additive_build_content_file a pip_hub_map = {} simpleapi_cache = pypi_cache(mctx = module_ctx) + is_pypi_hub_reserved = module_ctx.getenv("RULES_PYTHON_PYPI_HUB_RESERVED", "0") == "1" + renamed_default_hub = None + for mod in module_ctx.modules: for pip_attr in mod.tags.parse: hub_name = pip_attr.hub_name + if hub_name == "pypi": + if is_pypi_hub_reserved: + renamed_name = mod.name + "_pypi" + print( + ( + "WARNING: The PyPI hub name 'pypi' is reserved " + + "(module '{}'). The hub was renamed to '{}'. " + + "Please rename your hub." + ).format( + mod.name, + renamed_name, + ), + ) # buildifier: disable=print + hub_name = renamed_name + if not renamed_default_hub: + renamed_default_hub = hub_name + else: + print( + ( + "WARNING: The PyPI hub name 'pypi' is reserved " + + "(module '{}'). Please rename your hub, otherwise " + + "a future release will rename it to '{}_pypi'." + ).format( + mod.name, + mod.name, + ), + ) # buildifier: disable=print + if hub_name not in pip_hub_map: builder = hub_builder( name = hub_name, @@ -359,7 +399,7 @@ You cannot use both the additive_build_content and additive_build_content_file a available_interpreters = kwargs.get("available_interpreters", INTERPRETER_LABELS), logger = repo_utils.logger(module_ctx, "pypi:hub:" + hub_name, mod = mod), ) - pip_hub_map[pip_attr.hub_name] = builder + pip_hub_map[hub_name] = builder elif pip_hub_map[hub_name].module_name != mod.name: # We cannot have two hubs with the same name in different # modules. @@ -375,7 +415,7 @@ You cannot use both the additive_build_content and additive_build_content_file a )) else: - builder = pip_hub_map[pip_attr.hub_name] + builder = pip_hub_map[hub_name] builder.pip_parse( module_ctx, @@ -406,6 +446,7 @@ You cannot use both the additive_build_content and additive_build_content_file a return struct( config = config, + default_hub = config.default_hub or renamed_default_hub, exposed_packages = exposed_packages, extra_aliases = extra_aliases, facts = simpleapi_cache.get_facts(), @@ -422,6 +463,41 @@ You cannot use both the additive_build_content and additive_build_content_file a }, ) +def _create_unified_hub_repo(mods): + if "pypi" in mods.hub_whl_map: + return + + hubs = sorted(mods.hub_whl_map.keys()) + if mods.default_hub and mods.default_hub not in hubs: + fail("default_hub '%s' is not a defined PyPI hub. Available hubs: %s" % (mods.default_hub, ", ".join(hubs))) + + packages = {} + extra_aliases = {} + + for hub_name in hubs: + for pkg_name in mods.exposed_packages.get(hub_name, []): + norm_pkg = normalize_name(pkg_name) + if norm_pkg not in packages: + packages[norm_pkg] = [] + if hub_name not in packages[norm_pkg]: + packages[norm_pkg].append(hub_name) + + extra = mods.extra_aliases.get(hub_name, {}).get(norm_pkg, []) + for alias_name in extra: + qual_alias = "%s:%s" % (norm_pkg, alias_name) + if qual_alias not in extra_aliases: + extra_aliases[qual_alias] = [] + if hub_name not in extra_aliases[qual_alias]: + extra_aliases[qual_alias].append(hub_name) + + unified_hub_repo( + name = "pypi", + default_hub = mods.default_hub or (hubs[0] if hubs else ""), + extra_aliases = extra_aliases, + hubs = hubs, + packages = packages, + ) + def _pip_impl(module_ctx): """Implementation of a class tag that creates the pip hub and corresponding pip spoke whl repositories. @@ -513,6 +589,8 @@ def _pip_impl(module_ctx): groups = mods.hub_group_map.get(hub_name), ) + _create_unified_hub_repo(mods) + # The code is smart to not return facts if we don't support the mechanism for that. # Hence we should not pass it to the metadata if mods.facts: @@ -535,12 +613,14 @@ Either this or {attr}`env` `platform_machine` key should be specified. """, ), "config_settings": attr.label_list( - mandatory = True, doc = """\ The list of labels to `config_setting` targets that need to be matched for the platform to be -selected. +selected. Mandatory if platform is specified. """, ), + "default_hub": attr.string( + doc = "The name of the concrete PyPI hub to use by default when {flag}`--venv=auto`.", + ), "env": attr.string_dict( doc = """\ The values to use for environment markers when evaluating an expression. @@ -742,6 +822,11 @@ https://packaging.python.org/en/latest/specifications/simple-repository-api/ doc = """ The name of the repo pip dependencies will be accessible from. +The hub name `"pypi"` is reserved for the automatically generated +[Unified @pypi Hub](unified-pypi-hub) repository. Please choose a different name +for your concrete hubs. See [Unified @pypi Hub](unified-pypi-hub) for how to +handle collisions. + This name must be unique between modules; unless your module is guaranteed to always be the root module, it's highly recommended to include your module name in the hub name. Repo mapping, `use_repo(..., pip="my_modules_pip_deps")`, can @@ -757,6 +842,12 @@ is not required. Each hub is a separate resolution of pip dependencies. This means if different programs need different versions of some library, separate hubs can be created, and each program can use its respective hub's targets. Targets from different hubs should not be used together. + +:::{versionchanged} VERSION_NEXT_FEATURE +Using the hub name `"pypi"` is deprecated and is changed to +`{module_name}_pypi` depending on the +{envvar}`RULES_PYTHON_PYPI_HUB_RESERVED` environment variable. +::: """, ), "parallel_download": attr.bool( @@ -917,6 +1008,7 @@ other tags in this extension.""", ) pypi = module_extension( + environ = ["RULES_PYTHON_PYPI_HUB_RESERVED"], doc = """\ This extension is used to make dependencies from pip available. @@ -930,6 +1022,14 @@ can be made to configure different Python versions, and will be grouped by the `hub_name` argument. This allows the same logical name, e.g. `@pip//numpy` to automatically resolve to different, Python version-specific, libraries. +A unified `@pypi` proxy repository is always generated (unless a hub is +explicitly named "pypi") to route dependencies dynamically. See +[Unified @pypi Hub](unified-pypi-hub) for details. + +Environment Variables: +- `RULES_PYTHON_PYPI_HUB_RESERVED`: Enable fallback renaming for reserved hub name collisions. + See the {envvar}`RULES_PYTHON_PYPI_HUB_RESERVED` documentation for details. + pip.whl_mods: This tag class is used to help create JSON files to describe modifications to the BUILD files for wheels. @@ -941,6 +1041,10 @@ the BUILD files for wheels. doc = """\ This tag class allows for more customization of how the configuration for the hub repositories is built. +It can also be used to designate the default hub for the automatically +generated [Unified @pypi Hub](unified-pypi-hub) using the `default_hub` +attribute. + :::{seealso} The [environment markers][environment_markers] specification for the explanation of the @@ -961,6 +1065,9 @@ This tag class is used to create a pip hub and all of the spokes that are part o This tag class reuses most of the attributes found in {bzl:obj}`pip_parse`. The exception is it does not use the arg 'repo_prefix'. We set the repository prefix for the user and the alias arg is always True in bzlmod. + +You can use the automatically generated [Unified @pypi Hub](unified-pypi-hub) +repository to route package dependencies dynamically at build time. """, ), "whl_mods": tag_class( diff --git a/python/private/pypi/missing_package.bzl b/python/private/pypi/missing_package.bzl new file mode 100644 index 0000000000..c59f754b10 --- /dev/null +++ b/python/private/pypi/missing_package.bzl @@ -0,0 +1,36 @@ +"""Rule for generating an execution-phase action failure when a PyPI package is missing.""" + +load("//python/private:py_info.bzl", "PyInfo") +load("//python/private:reexports.bzl", "BuiltinPyInfo") + +def _missing_package_error_impl(ctx): + out = ctx.actions.declare_file(ctx.label.name + ".error") + + # Register an action that fails when Bazel attempts to stage/build this file + ctx.actions.run_shell( + outputs = [out], + command = "echo 'ERROR: PyPI package \"{pkg}\" is not available{hub_clause}.' >&2 && exit 1".format( + pkg = ctx.attr.package_name, + hub_clause = (' when building under PyPI hub "%s"' % ctx.attr.hub_name) if ctx.attr.hub_name else " because no PyPI hub or default hub is requested", + ), + ) + + maybe_builtin = [BuiltinPyInfo(transitive_sources = depset([out]))] if BuiltinPyInfo != None else [] + + return [ + DefaultInfo( + files = depset([out]), + data_runfiles = ctx.runfiles([out]), + ), + PyInfo( + transitive_sources = depset([out]), + ), + ] + maybe_builtin + +missing_package_error = rule( + implementation = _missing_package_error_impl, + attrs = { + "hub_name": attr.string(mandatory = True), + "package_name": attr.string(mandatory = True), + }, +) diff --git a/python/private/pypi/unified_hub_repo.bzl b/python/private/pypi/unified_hub_repo.bzl new file mode 100644 index 0000000000..cbe150575a --- /dev/null +++ b/python/private/pypi/unified_hub_repo.bzl @@ -0,0 +1,81 @@ +"""Repository rule for creating the Unified PyPI Hub.""" + +load("//python/private:text_util.bzl", "render") + +_ROOT_BUILD_TMPL = """\ +load("@rules_python//python/private/pypi:unified_hub_setup.bzl", "define_venv_flag_config_settings") + +package(default_visibility = ["//visibility:public"]) + +define_venv_flag_config_settings( + name = "venv_config_settings", + hubs = {hubs}, +) +""" + +_PKG_BUILD_TMPL = """\ +load("@rules_python//python/private/pypi:unified_hub_setup.bzl", "define_pypi_package_targets") + +package(default_visibility = ["//visibility:public"]) + +define_pypi_package_targets( + name = "{pkg_name}", + default_hub = {default_hub}, + extra_aliases = {extra_aliases}, + hubs = {hubs}, + pkg_hubs = {pkg_hubs}, +) +""" + +def _unified_hub_repo_impl(rctx): + hubs = rctx.attr.hubs + default_hub = rctx.attr.default_hub or None + + # 1. Generate Root BUILD.bazel with shared config settings + rctx.file( + "BUILD.bazel", + _ROOT_BUILD_TMPL.format(hubs = hubs), + ) + + # 2. Organize extra aliases by package + extra_aliases_by_pkg = {} + for qual_alias, alias_hubs in rctx.attr.extra_aliases.items(): + if ":" not in qual_alias: + fail("extra_aliases keys must be in 'pkg:alias' format.") + pkg, alias = qual_alias.split(":", 1) + extra_aliases_by_pkg.setdefault(pkg, {})[alias] = alias_hubs + + # 3. Generate package subpackages + for pkg_name, pkg_hubs in rctx.attr.packages.items(): + extra_aliases = extra_aliases_by_pkg.get(pkg_name, {}) + rctx.file( + pkg_name + "/BUILD.bazel", + _PKG_BUILD_TMPL.format( + default_hub = render.str(default_hub), + extra_aliases = extra_aliases, + hubs = hubs, + pkg_hubs = pkg_hubs, + pkg_name = pkg_name, + ), + ) + +unified_hub_repo = repository_rule( + implementation = _unified_hub_repo_impl, + attrs = { + "default_hub": attr.string( + doc = "The PyPI hub to use when no other hub's conditions match.", + ), + "extra_aliases": attr.string_list_dict( + doc = "Dictionary mapping 'package:alias' to a list of hubs that support it.", + ), + "hubs": attr.string_list( + mandatory = True, + doc = "List of all concrete PyPI hub names.", + ), + "packages": attr.string_list_dict( + mandatory = True, + doc = "Dictionary mapping package names to a list of hubs that contain them.", + ), + }, + doc = "Private repository rule creating the automatic Unified PyPI Hub.", +) diff --git a/python/private/pypi/unified_hub_setup.bzl b/python/private/pypi/unified_hub_setup.bzl new file mode 100644 index 0000000000..b19410fd4b --- /dev/null +++ b/python/private/pypi/unified_hub_setup.bzl @@ -0,0 +1,110 @@ +"""Helper functions for setting up targets within the Unified PyPI Hub repository.""" + +load( + "@rules_python//python/private/pypi:labels.bzl", + "DATA_LABEL", + "DIST_INFO_LABEL", + "EXTRACTED_WHEEL_FILES", + "PY_LIBRARY_PUBLIC_LABEL", + "WHEEL_FILE_PUBLIC_LABEL", +) +load("@rules_python//python/private/pypi:missing_package.bzl", "missing_package_error") + +def define_venv_flag_config_settings(name, hubs): + """Defines the root config_settings for each PyPI spoke hub. + + Args: + name: unused macro name required by buildifier. + hubs: list of concrete hub names. + """ + for hub in hubs: + native.config_setting( + name = "_is_venv_" + hub, + flag_values = {"@rules_python//python/config_settings:venv": hub}, + ) + +_STANDARD_ALIASES = [ + PY_LIBRARY_PUBLIC_LABEL, + WHEEL_FILE_PUBLIC_LABEL, + DATA_LABEL, + DIST_INFO_LABEL, + EXTRACTED_WHEEL_FILES, +] + +def define_pypi_package_targets(name, pkg_hubs, extra_aliases, hubs, default_hub = None): + """Define the targets for a PyPI package in the unified PyPI hub. + + Args: + name: normalized PyPI package name, serving as the main target name. + pkg_hubs: list of hubs that contain this package. + extra_aliases: dict mapping extra alias names to lists of hubs that support them. + hubs: list of all concrete hub names. + default_hub: the hub to use by default. + """ + pkg_name = name + + # Main apparent package target delegates to :pkg + native.alias( + name = pkg_name, + actual = ":pkg", + ) + + all_aliases = _STANDARD_ALIASES + sorted(extra_aliases.keys()) + missing_errors = {} + + for alias_name in all_aliases: + select_map = {} + for hub in hubs: + is_supported = ( + (alias_name in _STANDARD_ALIASES and hub in pkg_hubs) or + (alias_name not in _STANDARD_ALIASES and hub in extra_aliases.get(alias_name, [])) + ) + + if is_supported: + select_map["//:_is_venv_" + hub] = "@{hub}//{pkg}:{alias}".format( + hub = hub, + pkg = pkg_name, + alias = alias_name, + ) + else: + err_target = "_missing_{alias}_in_{hub}".format(alias = alias_name, hub = hub) + if err_target not in missing_errors: + missing_errors[err_target] = { + "hub_name": hub, + "package_name": pkg_name if alias_name in _STANDARD_ALIASES else (pkg_name + ":" + alias_name), + } + select_map["//:_is_venv_" + hub] = ":{}".format(err_target) + + # //conditions:default fallback + default_supported = ( + default_hub and + ((alias_name in _STANDARD_ALIASES and default_hub in pkg_hubs) or + (alias_name not in _STANDARD_ALIASES and default_hub in extra_aliases.get(alias_name, []))) + ) + + if default_supported: + select_map["//conditions:default"] = "@{hub}//{pkg}:{alias}".format( + hub = default_hub, + pkg = pkg_name, + alias = alias_name, + ) + else: + err_target = "_missing_{alias}_in_default".format(alias = alias_name) + if err_target not in missing_errors: + missing_errors[err_target] = { + "hub_name": default_hub or "", + "package_name": pkg_name if alias_name in _STANDARD_ALIASES else (pkg_name + ":" + alias_name), + } + select_map["//conditions:default"] = ":{}".format(err_target) + + native.alias( + name = alias_name, + actual = select(select_map), + ) + + # Generate missing package error targets + for err_name, err_args in missing_errors.items(): + missing_package_error( + name = err_name, + **err_args + ) diff --git a/python/private/text_util.bzl b/python/private/text_util.bzl index f725195978..eedf66009d 100644 --- a/python/private/text_util.bzl +++ b/python/private/text_util.bzl @@ -107,6 +107,18 @@ def _render_list(items, *, hanging_indent = "", value_repr = repr): return text def _render_str(value): + """Render a string value. + + If value is None, it is automatically rendered as the Starlark literal `None`. + + Args: + value: str or None. + + Returns: + The value represented as Starlark source text. + """ + if value == None: + return "None" return repr(value) def _render_string_list_dict(value): diff --git a/python/private/transition_labels.bzl b/python/private/transition_labels.bzl index 7a6531ed0f..d337044cb5 100644 --- a/python/private/transition_labels.bzl +++ b/python/private/transition_labels.bzl @@ -21,6 +21,7 @@ _BASE_TRANSITION_LABELS = [ labels.PYTHON_VERSION, labels.PY_FREETHREADED, labels.PY_LINUX_LIBC, + labels.VENV, labels.VENVS_SITE_PACKAGES, labels.VENVS_USE_DECLARE_SYMLINK, ] diff --git a/tests/integration/BUILD.bazel b/tests/integration/BUILD.bazel index 904fb4c247..9301e19590 100644 --- a/tests/integration/BUILD.bazel +++ b/tests/integration/BUILD.bazel @@ -118,6 +118,11 @@ rules_python_integration_test( py_main = "toolchain_target_settings_test.py", ) +rules_python_integration_test( + name = "unified_pypi_test", + py_main = "unified_pypi_test.py", +) + rules_python_integration_test( name = "uv_lock_test", py_deps = [ diff --git a/tests/integration/bzlmod_lockfile/MODULE.bazel.lock b/tests/integration/bzlmod_lockfile/MODULE.bazel.lock index 2a0bc7d76b..0408f791e1 100644 --- a/tests/integration/bzlmod_lockfile/MODULE.bazel.lock +++ b/tests/integration/bzlmod_lockfile/MODULE.bazel.lock @@ -250,7 +250,7 @@ }, "@@rules_python+//python/uv:uv.bzl%uv": { "general": { - "bzlTransitiveDigest": "46RcxJnhOapMeaxdcMm3RmVdNp1nPCewOOXoZyIbQ20=", + "bzlTransitiveDigest": "ELjwPp2kLku5M3S/gpjjVjy3TwT760/zVEQ70nJreHU=", "usagesDigest": "6yXGw7XDyXjOfqBL0SBu1YBEMMYPQzCE3jTzUCkxPgg=", "recordedInputs": [ "REPO_MAPPING:rules_python+,bazel_tools bazel_tools", diff --git a/tests/integration/unified_pypi/.bazelrc b/tests/integration/unified_pypi/.bazelrc new file mode 100644 index 0000000000..b3a24e8605 --- /dev/null +++ b/tests/integration/unified_pypi/.bazelrc @@ -0,0 +1 @@ +common --experimental_enable_bzlmod diff --git a/tests/integration/unified_pypi/BUILD.bazel b/tests/integration/unified_pypi/BUILD.bazel new file mode 100644 index 0000000000..8a37d4b153 --- /dev/null +++ b/tests/integration/unified_pypi/BUILD.bazel @@ -0,0 +1,48 @@ +load("@rules_python//python:py_binary.bzl", "py_binary") +load("@rules_python//python:py_test.bzl", "py_test") + +package(default_visibility = ["//visibility:public"]) + +py_test( + name = "test_default", + srcs = ["test_default.py"], + deps = ["@pypi//colorama"], +) + +py_test( + name = "test_cli", + srcs = ["test_cli.py"], + deps = ["@pypi//colorama"], +) + +py_test( + name = "test_a", + srcs = ["test_a.py"], + config_settings = { + "@rules_python//python/config_settings:venv": "pypi_a", + }, + deps = [ + "@pypi//colorama", + "@pypi//colorama:my_colorama", + ], +) + +# Sibling extra alias failure target (my_colorama is missing in pypi_b): +py_binary( + name = "bin_extra_b", + srcs = ["bin_extra_b.py"], + config_settings = { + "@rules_python//python/config_settings:venv": "pypi_b", + }, + deps = ["@pypi//colorama:my_colorama"], +) + +# Disjoint package failure target (six is missing in pypi_a): +py_binary( + name = "bin_six_a", + srcs = ["bin_six_a.py"], + config_settings = { + "@rules_python//python/config_settings:venv": "pypi_a", + }, + deps = ["@pypi//six"], +) diff --git a/tests/integration/unified_pypi/MODULE.bazel b/tests/integration/unified_pypi/MODULE.bazel new file mode 100644 index 0000000000..0d0f44f61c --- /dev/null +++ b/tests/integration/unified_pypi/MODULE.bazel @@ -0,0 +1,48 @@ +module(name = "unified_pypi") + +bazel_dep(name = "rules_python", version = "0.0.0") +local_path_override( + module_name = "rules_python", + path = "../../..", +) + +python = use_extension("@rules_python//python/extensions:python.bzl", "python") +python.toolchain(python_version = "3.11") + +pip = use_extension("@rules_python//python/extensions:pip.bzl", "pip") +pip.whl_mods( + additive_build_content = """\ +load("@rules_python//python:defs.bzl", "py_library") + +py_library( + name = "my_colorama", + deps = [":pkg"], +) +""", + hub_name = "whl_mods_hub", + whl_name = "colorama", +) +use_repo(pip, "whl_mods_hub") + +# pypi_a has colorama and an extra alias +pip.parse( + extra_hub_aliases = {"colorama": ["my_colorama"]}, + hub_name = "pypi_a", + python_version = "3.11", + requirements_lock = "//:requirements_a.txt", + whl_modifications = { + "@whl_mods_hub//:colorama.json": "colorama", + }, +) +use_repo(pip, "pypi_a") + +# pypi_b has colorama and six, and acts as designated fallback +pip.parse( + hub_name = "pypi_b", + python_version = "3.11", + requirements_lock = "//:requirements_b.txt", +) +use_repo(pip, "pypi_b") + +pip.default(default_hub = "pypi_b") +use_repo(pip, "pypi") diff --git a/tests/integration/unified_pypi/WORKSPACE b/tests/integration/unified_pypi/WORKSPACE new file mode 100644 index 0000000000..0a08afe832 --- /dev/null +++ b/tests/integration/unified_pypi/WORKSPACE @@ -0,0 +1 @@ +# Minimal WORKSPACE file diff --git a/tests/integration/unified_pypi/WORKSPACE.bzlmod b/tests/integration/unified_pypi/WORKSPACE.bzlmod new file mode 100644 index 0000000000..7bd1c969b9 --- /dev/null +++ b/tests/integration/unified_pypi/WORKSPACE.bzlmod @@ -0,0 +1 @@ +# Minimal WORKSPACE.bzlmod diff --git a/tests/integration/unified_pypi/bin_extra_b.py b/tests/integration/unified_pypi/bin_extra_b.py new file mode 100644 index 0000000000..f900d16fd2 --- /dev/null +++ b/tests/integration/unified_pypi/bin_extra_b.py @@ -0,0 +1 @@ +print("Should not be executed") diff --git a/tests/integration/unified_pypi/bin_six_a.py b/tests/integration/unified_pypi/bin_six_a.py new file mode 100644 index 0000000000..f900d16fd2 --- /dev/null +++ b/tests/integration/unified_pypi/bin_six_a.py @@ -0,0 +1 @@ +print("Should not be executed") diff --git a/tests/integration/unified_pypi/requirements_a.txt b/tests/integration/unified_pypi/requirements_a.txt new file mode 100644 index 0000000000..788f12f818 --- /dev/null +++ b/tests/integration/unified_pypi/requirements_a.txt @@ -0,0 +1,3 @@ +colorama==0.4.6 \ + --hash=sha256:08695f5cb7ed6e0531a20572697297273c47b8cae5a63ffc6d6ed5c201be6e44 \ + --hash=sha256:4f1d9991f5acc0ca119f9d443620b77f9d6b33703e51011c16baf57afb285fc6 diff --git a/tests/integration/unified_pypi/requirements_b.txt b/tests/integration/unified_pypi/requirements_b.txt new file mode 100644 index 0000000000..c69f3631b2 --- /dev/null +++ b/tests/integration/unified_pypi/requirements_b.txt @@ -0,0 +1,6 @@ +colorama==0.4.5 \ + --hash=sha256:854bf444933e37f5824ae7bfc1e98d5bce2ebe4160d46b5edf346a89358e99da \ + --hash=sha256:e6c6b4334fc50988a639d9b98ae42f5c90ec94cb1495b4fe76c5f72cf7f79435 +six==1.17.0 \ + --hash=sha256:4721f391ed90541fddacab5acf947aa0d3dc7d27b2e1e8eda2be8970586c3274 \ + --hash=sha256:ff70335d468e7eb6ec65b95b99d3a2836546063f63acc5171de367e834932a81 diff --git a/tests/integration/unified_pypi/test_a.py b/tests/integration/unified_pypi/test_a.py new file mode 100644 index 0000000000..a9127ba0c7 --- /dev/null +++ b/tests/integration/unified_pypi/test_a.py @@ -0,0 +1,3 @@ +import colorama + +assert colorama.__version__ == "0.4.6" diff --git a/tests/integration/unified_pypi/test_cli.py b/tests/integration/unified_pypi/test_cli.py new file mode 100644 index 0000000000..a9127ba0c7 --- /dev/null +++ b/tests/integration/unified_pypi/test_cli.py @@ -0,0 +1,3 @@ +import colorama + +assert colorama.__version__ == "0.4.6" diff --git a/tests/integration/unified_pypi/test_default.py b/tests/integration/unified_pypi/test_default.py new file mode 100644 index 0000000000..559df59961 --- /dev/null +++ b/tests/integration/unified_pypi/test_default.py @@ -0,0 +1,3 @@ +import colorama + +assert colorama.__version__ == "0.4.5" diff --git a/tests/integration/unified_pypi_test.py b/tests/integration/unified_pypi_test.py new file mode 100644 index 0000000000..707ab13444 --- /dev/null +++ b/tests/integration/unified_pypi_test.py @@ -0,0 +1,79 @@ +"""Integration test for Unified PyPI Hub dynamic dependency resolution.""" + +import contextlib +import unittest + +from tests.integration import runner + + +class UnifiedPypiTest(runner.TestCase): + def test_default_fallback_hub(self): + self.run_bazel("test", "//:test_default") + + def test_transitioned_hub(self): + self.run_bazel("test", "//:test_a") + + def test_cli_override(self): + self.run_bazel( + "run", + "--@rules_python//python/config_settings:venv=pypi_a", + "//:test_cli", + ) + + def test_disjoint_package_cquery_succeeds_but_build_fails(self): + self.run_bazel("cquery", "//:bin_six_a") + result = self.run_bazel("build", "//:bin_six_a", check=False) + self.assertNotEqual( + result.exit_code, + 0, + "Expected build to fail during execution phase", + ) + self.assert_result_matches( + result, + 'ERROR: PyPI package "six" is not available when building under PyPI hub "pypi_a".', + ) + + def test_sibling_extra_alias_cquery_succeeds_but_build_fails(self): + self.run_bazel("cquery", "//:bin_extra_b") + result = self.run_bazel("build", "//:bin_extra_b", check=False) + self.assertNotEqual( + result.exit_code, + 0, + "Expected build to fail during execution phase", + ) + self.assert_result_matches( + result, + 'ERROR: PyPI package "colorama:my_colorama" is not available when building under PyPI hub "pypi_b".', + ) + + @contextlib.contextmanager + def _temp_modify_file(self, path, new_content): + original_content = path.read_text() + path.write_text(new_content) + try: + yield + finally: + path.write_text(original_content) + + def test_invalid_default_hub_fails_evaluation(self): + module_bazel = self.repo_root / "MODULE.bazel" + invalid_content = module_bazel.read_text().replace( + 'pip.default(default_hub = "pypi_b")', + 'pip.default(default_hub = "invalid_hub")', + ) + with self._temp_modify_file(module_bazel, invalid_content): + # Run bazel cquery and expect it to fail during loading/extension phase + result = self.run_bazel("cquery", "//:test_default", check=False) + self.assertNotEqual( + result.exit_code, + 0, + "Expected extension evaluation to fail due to invalid default_hub", + ) + self.assert_result_matches( + result, + "default_hub 'invalid_hub' is not a defined PyPI hub", + ) + + +if __name__ == "__main__": + unittest.main() diff --git a/tests/pypi/extension/extension_tests.bzl b/tests/pypi/extension/extension_tests.bzl index 5a40714b64..bc4c0bcb5b 100644 --- a/tests/pypi/extension/extension_tests.bzl +++ b/tests/pypi/extension/extension_tests.bzl @@ -44,6 +44,7 @@ def _default( arch_name = None, auth_patterns = None, config_settings = None, + default_hub = "", env = None, index_url = None, marker = None, @@ -56,6 +57,7 @@ def _default( arch_name = arch_name, auth_patterns = auth_patterns or {}, config_settings = config_settings, + default_hub = default_hub, env = env or {}, index_url = index_url or "", marker = marker or "", @@ -104,6 +106,7 @@ def _parse_modules(env, **kwargs): return env.expect.that_struct( parse_modules(**kwargs), attrs = dict( + default_hub = subjects.str, exposed_packages = subjects.dict, hub_group_map = subjects.dict, hub_whl_map = subjects.dict, @@ -283,6 +286,155 @@ def _test_build_pipstar_platform(env): _tests.append(_test_build_pipstar_platform) +def _test_multiple_default_tags(env): + """Test that multiple pip.default tags do not trigger duplicate default hub failures. + + Only when multiple tags explicitly define default_hub should it fail. + """ + pypi = _parse_modules( + env, + module_ctx = _pypi_mock_mctx( + _mod( + name = "rules_python", + default = _default_tags_default + [ + _default(platform = "extra_custom_platform"), + ], + parse = [ + _parse( + hub_name = "pypi", + python_version = "3.15", + simpleapi_skip = ["simple"], + requirements_lock = "requirements.txt", + ), + ], + ), + os_name = "linux", + arch_name = "x86_64", + ), + available_interpreters = { + "python_3_15_host": "unit_test_interpreter_target", + }, + minor_mapping = {"3.15": "3.15.19"}, + ) + pypi.exposed_packages().contains_exactly({"pypi": ["simple"]}) + +_tests.append(_test_multiple_default_tags) + +def _test_name_collision_no_env(env): + """Test that a hub named 'pypi' is NOT renamed when the env var is not set.""" + pypi = _parse_modules( + env, + module_ctx = _pypi_mock_mctx( + _mod( + name = "rules_python", + parse = [ + _parse( + hub_name = "pypi", + python_version = "3.15", + simpleapi_skip = ["simple"], + requirements_lock = "requirements.txt", + ), + ], + ), + os_name = "linux", + arch_name = "x86_64", + environ = {}, # Env var NOT set + ), + available_interpreters = { + "python_3_15_host": "unit_test_interpreter_target", + }, + minor_mapping = {"3.15": "3.15.19"}, + ) + + # The hub name remains 'pypi' + pypi.exposed_packages().contains_exactly({"pypi": ["simple"]}) + pypi.default_hub().equals(None) + +_tests.append(_test_name_collision_no_env) + +def _test_name_collision_with_env(env): + """Test that a hub named 'pypi' is silently renamed to module_name_pypi and routed as default_hub when the env var is set.""" + pypi = _parse_modules( + env, + module_ctx = _pypi_mock_mctx( + _mod( + name = "rules_python", + parse = [ + _parse( + hub_name = "pypi", + python_version = "3.15", + simpleapi_skip = ["simple"], + requirements_lock = "requirements.txt", + ), + ], + ), + os_name = "linux", + arch_name = "x86_64", + environ = {"RULES_PYTHON_PYPI_HUB_RESERVED": "1"}, + ), + available_interpreters = { + "python_3_15_host": "unit_test_interpreter_target", + }, + minor_mapping = {"3.15": "3.15.19"}, + ) + + # The hub name is renamed to 'rules_python_pypi' + pypi.exposed_packages().contains_exactly({"rules_python_pypi": ["simple"]}) + + # It is used as the default_hub + pypi.default_hub().equals("rules_python_pypi") + +_tests.append(_test_name_collision_with_env) + +def _test_default_hub_precedence(env): + """Test that pip.default(default_hub = ...) has precedence over the fallback renamed default hub.""" + pypi = _parse_modules( + env, + module_ctx = _pypi_mock_mctx( + _mod( + name = "rules_python", + default = _default_tags_default + [ + _default( + platform = "extra_custom_platform", + default_hub = "other_pypi", + ), + ], + parse = [ + _parse( + hub_name = "pypi", + python_version = "3.15", + simpleapi_skip = ["simple"], + requirements_lock = "requirements.txt", + ), + _parse( + hub_name = "other_pypi", + python_version = "3.15", + simpleapi_skip = ["simple"], + requirements_lock = "requirements.txt", + ), + ], + ), + os_name = "linux", + arch_name = "x86_64", + environ = {"RULES_PYTHON_PYPI_HUB_RESERVED": "1"}, + ), + available_interpreters = { + "python_3_15_host": "unit_test_interpreter_target", + }, + minor_mapping = {"3.15": "3.15.19"}, + ) + + # The hub named 'pypi' is renamed to 'rules_python_pypi' + pypi.exposed_packages().contains_exactly({ + "other_pypi": ["simple"], + "rules_python_pypi": ["simple"], + }) + + # But the default_hub remains 'other_pypi' because pip.default has higher precedence! + pypi.default_hub().equals("other_pypi") + +_tests.append(_test_default_hub_precedence) + def extension_test_suite(name): """Create the test suite. From 12680c53a3a8740ff7c8ad5cadecbc1f0cf92d4c Mon Sep 17 00:00:00 2001 From: Ignas Anikevicius <240938+aignas@users.noreply.github.com> Date: Fri, 26 Jun 2026 07:58:14 +0900 Subject: [PATCH 781/922] feat(pypi): support importing uv.lock file (#3785) The strategy for this is: * First add a way for us to create a `uv.lock` file from the `lock` rule. * Then add a `uv.lock` reader via the bazel toml parser. * Then plug the code into the `parse_requirements` function so that we can reuse the most of code already there. * Add some sample docs for the `uv.lock`. Extra things that we could do: * Call the PyPI index to understand if the packages are yanked or not - lock file does not have that information. * Read the `pyproject.toml` file to get the index values for each package. * Add e2e tests from `rules_py` test suite. Closes #3557 Work towards #2787 --------- Co-authored-by: Richard Levasseur Co-authored-by: Richard Levasseur --- .gitattributes | 2 + MODULE.bazel | 2 + docs/BUILD.bazel | 9 + docs/requirements.txt | 60 +- docs/uv.lock | 1163 +++++++++++++++++ news/3785.added.md | 5 + python/private/py_repositories.bzl | 6 + python/private/pypi/BUILD.bazel | 2 + python/private/pypi/extension.bzl | 10 +- python/private/pypi/hub_builder.bzl | 2 + python/private/pypi/parse_requirements.bzl | 314 ++++- python/uv/private/BUILD.bazel | 43 +- python/uv/private/lock.bat | 7 - python/uv/private/lock.bzl | 415 +++--- python/uv/private/lock.sh | 9 - .../uv/private/{ => template}/lock_copier.py | 0 python/uv/private/template/uv_lock.bat | 21 + python/uv/private/template/uv_lock.sh | 32 + python/uv/private/template/uv_pip_compile.bat | 8 + python/uv/private/template/uv_pip_compile.sh | 15 + python/uv/private/uv_lock_to_requirements.bzl | 139 ++ .../bzlmod_lockfile/MODULE.bazel.lock | 2 + tests/pypi/extension/pip_parse.bzl | 2 + .../parse_requirements_tests.bzl | 636 +++++++++ tests/uv/lock/BUILD.bazel | 5 + tests/uv/lock/lock_run_test.py | 128 +- tests/uv/lock/lock_tests.bzl | 20 + tests/uv/lock/pyproject_toml/requirements.txt | 16 +- tests/uv/lock/testdata/constraints.txt | 4 + tests/uv/lock/testdata/pyproject.toml | 5 + tests/uv/lock/testdata/requirements.txt | 16 +- tests/uv/lock/testdata/uv_lock_expected.lock | 218 +++ .../uv/lock/uv_lock_to_requirements_tests.bzl | 289 ++++ 33 files changed, 3290 insertions(+), 315 deletions(-) create mode 100644 docs/uv.lock create mode 100644 news/3785.added.md delete mode 100755 python/uv/private/lock.bat delete mode 100755 python/uv/private/lock.sh rename python/uv/private/{ => template}/lock_copier.py (100%) create mode 100755 python/uv/private/template/uv_lock.bat create mode 100755 python/uv/private/template/uv_lock.sh create mode 100755 python/uv/private/template/uv_pip_compile.bat create mode 100755 python/uv/private/template/uv_pip_compile.sh create mode 100644 python/uv/private/uv_lock_to_requirements.bzl create mode 100644 tests/uv/lock/testdata/pyproject.toml create mode 100644 tests/uv/lock/testdata/uv_lock_expected.lock create mode 100644 tests/uv/lock/uv_lock_to_requirements_tests.bzl diff --git a/.gitattributes b/.gitattributes index fafafd001b..4f93d89d33 100644 --- a/.gitattributes +++ b/.gitattributes @@ -3,3 +3,5 @@ tools/publish/*.txt linguist-generated=true tests/uv/lock/testdata/requirements.txt text eol=lf python/private/runtimes_manifest_workspace.bzl text eol=lf python/private/runtimes_manifest.txt text eol=lf + +*.bat text eol=crlf diff --git a/MODULE.bazel b/MODULE.bazel index 568c2732e5..4de83f31be 100644 --- a/MODULE.bazel +++ b/MODULE.bazel @@ -9,6 +9,7 @@ bazel_dep(name = "bazel_skylib", version = "1.8.2") bazel_dep(name = "package_metadata", version = "0.0.7") bazel_dep(name = "platforms", version = "0.0.11") bazel_dep(name = "rules_cc", version = "0.2.17") +bazel_dep(name = "toml.bzl", version = "0.4.1") # Those are loaded only when using py_proto_library # Use py_proto_library directly from protobuf repository @@ -183,6 +184,7 @@ dev_pip = use_extension( "{os}_{arch}", "{os}_{arch}_freethreaded", ], + uv_lock = "//docs:uv.lock", ) for python_version in [ "3.9", diff --git a/docs/BUILD.bazel b/docs/BUILD.bazel index 315010c12d..043f3e4de7 100644 --- a/docs/BUILD.bazel +++ b/docs/BUILD.bazel @@ -223,3 +223,12 @@ lock( python_version = "3.9", visibility = ["//:__subpackages__"], ) + +# Run bazel run //docs:uv_lock.update +lock( + name = "uv_lock", + srcs = ["pyproject.toml"], + out = "uv.lock", + python_version = "3.9", + visibility = ["//:__subpackages__"], +) diff --git a/docs/requirements.txt b/docs/requirements.txt index 6397f0a7f1..468cd38c0d 100644 --- a/docs/requirements.txt +++ b/docs/requirements.txt @@ -30,9 +30,9 @@ babel==2.18.0 \ --hash=sha256:b80b99a14bd085fcacfa15c9165f651fbb3406e66cc603abf11c5750937c992d \ --hash=sha256:e2b422b277c2b9a9630c1d7903c2a00d0830c409c59ac8cae9081c92f1aeba35 # via sphinx -certifi==2026.2.25 \ - --hash=sha256:027692e4402ad994f1c42e52a4997a9763c646b73e4096e4d5d6db8af1d6f0fa \ - --hash=sha256:e887ab5cee78ea814d3472169153c2d12cd43b14bd03329a39a9c6e2e80bfba7 +certifi==2026.6.17 \ + --hash=sha256:024c88eeec92ca068db80f02b8b07c9cef7b9fe261d1d535abfd5abd6f6af432 \ + --hash=sha256:2227dcbaafe0d2f59279d1762ddddc37783ed4354594f194ffc31d20f41fc3db # via requests charset-normalizer==3.4.7 \ --hash=sha256:007d05ec7321d12a40227aae9e2bc6dca73f3cb21058999a1df9e193555a9dcc \ @@ -183,9 +183,9 @@ docutils==0.22.4 ; python_full_version >= '3.11' \ # myst-parser # sphinx # sphinx-rtd-theme -idna==3.11 \ - --hash=sha256:771a87f49d9defaf64091e6e6fe9c18d4833f140bd19464795bc32d966ca37ea \ - --hash=sha256:795dafcc9c04ed0c1fb032c2aa73654d8e8c5023a7df64a53f39190ada629902 +idna==3.18 \ + --hash=sha256:7f952cbe720b688055e3f87de14f5c3e5fdaa8bc3928985c4077ca689de849a2 \ + --hash=sha256:ffb385a7e039654cef1ab9ef32c6fafe283c0c0467bba1d9029738ce4a14a848 # via requests imagesize==1.5.0 ; python_full_version < '3.10' \ --hash=sha256:32677681b3f434c2cb496f00e89c5a291247b35b1f527589909e008057da5899 \ @@ -216,9 +216,9 @@ markdown-it-py==3.0.0 ; python_full_version < '3.11' \ # via # mdit-py-plugins # myst-parser -markdown-it-py==4.0.0 ; python_full_version >= '3.11' \ - --hash=sha256:87327c59b172c5011896038353a81343b6754500a08cd7a4973bb48c6d578147 \ - --hash=sha256:cb0a2b4aa34f932c007117b194e945bd74e0ec24133ceb5bac59009cda1cb9f3 +markdown-it-py==4.2.0 ; python_full_version >= '3.11' \ + --hash=sha256:04a21681d6fbb623de53f6f364d352309d4094dd4194040a10fd51833e418d49 \ + --hash=sha256:9f7ebbcd14fe59494226453aed97c1070d83f8d24b6fc3a3bcf9a38092641c4a # via # mdit-py-plugins # myst-parser @@ -319,9 +319,9 @@ mdit-py-plugins==0.4.2 ; python_full_version < '3.10' \ --hash=sha256:0c673c3f889399a33b95e88d2f0d111b4447bdfea7f237dab2d488f459835636 \ --hash=sha256:5f2cd1fdb606ddf152d37ec30e46101a60512bc0e5fa1a7002c36647b09e26b5 # via myst-parser -mdit-py-plugins==0.5.0 ; python_full_version >= '3.10' \ - --hash=sha256:07a08422fc1936a5d26d146759e9155ea466e842f5ab2f7d2266dd084c8dab1f \ - --hash=sha256:f4918cb50119f50446560513a8e311d574ff6aaed72606ddae6d35716fe809c6 +mdit-py-plugins==0.6.1 ; python_full_version >= '3.10' \ + --hash=sha256:214c82fb2ac524472ab6a5bcab1de80f73b50443e187f401bfd77efbc7c6481d \ + --hash=sha256:a2bca0f039f39dbd35fb74ae1b5f998608c437463371f0ff7f49a19a17a114d0 # via myst-parser mdurl==0.1.2 \ --hash=sha256:84008a41e51615a49fc9966191ff91509e3c40b939176e643fd50a5c2196b8f8 \ @@ -335,13 +335,13 @@ myst-parser==4.0.1 ; python_full_version == '3.10.*' \ --hash=sha256:5cfea715e4f3574138aecbf7d54132296bfd72bb614d31168f48c477a830a7c4 \ --hash=sha256:9134e88959ec3b5780aedf8a99680ea242869d012e8821db3126d427edc9c95d # via rules-python-docs (docs/pyproject.toml) -myst-parser==5.0.0 ; python_full_version >= '3.11' \ - --hash=sha256:ab31e516024918296e169139072b81592336f2fef55b8986aa31c9f04b5f7211 \ - --hash=sha256:f6f231452c56e8baa662cc352c548158f6a16fcbd6e3800fc594978002b94f3a +myst-parser==5.1.0 ; python_full_version >= '3.11' \ + --hash=sha256:9c91c52b3cdb4d94a6506e4fab4e2f296c7623a0da0dcbe6de1565c3dad67a8a \ + --hash=sha256:ab69322dc6719dcc7f296479dbb70181b66df6ed315064f92dbc85c0e1bf2f02 # via rules-python-docs (docs/pyproject.toml) -packaging==26.1 \ - --hash=sha256:5d9c0669c6285e491e0ced2eee587eaf67b670d94a19e94e3984a481aba6802f \ - --hash=sha256:f042152b681c4bfac5cae2742a55e103d27ab2ec0f3d88037136b6bfe7c9c5de +packaging==26.2 \ + --hash=sha256:5fc45236b9446107ff2415ce77c807cee2862cb6fac22b8a73826d0693b0980e \ + --hash=sha256:ff452ff5a3e828ce110190feff1178bb1f2ea2281fa2075aadb987c2fb221661 # via # readthedocs-sphinx-ext # sphinx @@ -349,10 +349,14 @@ pefile==2024.8.26 \ --hash=sha256:3ff6c5d8b43e8c37bb6e6dd5085658d658a7a0bdcd20b6a07b1fcfc1c4e9d632 \ --hash=sha256:76f8b485dcd3b1bb8166f1128d395fa3d87af26360c2358fb75b80019b957c6f # via rules-python-docs (docs/pyproject.toml) -pyelftools==0.32 \ +pyelftools==0.32 ; python_full_version < '3.10' \ --hash=sha256:013df952a006db5e138b1edf6d8a68ecc50630adbd0d83a2d41e7f846163d738 \ --hash=sha256:6de90ee7b8263e740c8715a925382d4099b354f29ac48ea40d840cf7aa14ace5 # via rules-python-docs (docs/pyproject.toml) +pyelftools==0.33 ; python_full_version >= '3.10' \ + --hash=sha256:660d82dcbeb8e83d1702bd97f223f761625da06111c0cc988eac6b8ab0c1b61f \ + --hash=sha256:f215ad5f47d3f1373a21496a6c9e0707c622840d0622f23ff7ce08678b020036 + # via rules-python-docs (docs/pyproject.toml) pygments==2.20.0 \ --hash=sha256:6757cd03768053ff99f3039c1a36d6c0aa0b263438fcab17520b30a303a82b5f \ --hash=sha256:81a9e26dd42fd28a23a2d169d86d7ac03b46e2f8b59ed4698fb4785f946d0176 @@ -442,9 +446,9 @@ requests==2.32.5 ; python_full_version < '3.10' \ # via # readthedocs-sphinx-ext # sphinx -requests==2.33.1 ; python_full_version >= '3.10' \ - --hash=sha256:18817f8c57c6263968bc123d237e3b8b08ac046f5456bd1e307ee8f4250d3517 \ - --hash=sha256:4e6d1ef462f3626a1f0a0a9c42dd93c63bad33f9f1c1937509b8c5c8718ab56a +requests==2.34.2 ; python_full_version >= '3.10' \ + --hash=sha256:2a0d60c172f83ac6ab31e4554906c0f3b3588d37b5cb939b1c061f4907e278e0 \ + --hash=sha256:f288924cae4e29463698d6d60bc6a4da69c89185ad1e0bcc4104f584e960b9ed # via # readthedocs-sphinx-ext # sphinx @@ -452,9 +456,9 @@ roman-numerals==4.1.0 ; python_full_version >= '3.11' \ --hash=sha256:1af8b147eb1405d5839e78aeb93131690495fe9da5c91856cb33ad55a7f1e5b2 \ --hash=sha256:647ba99caddc2cc1e55a51e4360689115551bf4476d90e8162cf8c345fe233c7 # via sphinx -snowballstemmer==3.0.1 \ - --hash=sha256:6cd7b3897da8d6c9ffb968a6781fa6532dce9c3618a4b127d920dab764a19064 \ - --hash=sha256:6d5eeeec8e9f84d4d56b847692bacf79bc2c8e90c7f80ca4444ff8b6f2e52895 +snowballstemmer==3.1.1 \ + --hash=sha256:7e207fa178741da09cdee59d3ecec3827ad5f92b1fc5c9ff3755b639f71f5752 \ + --hash=sha256:e07bbc54a0d798fe6010a12398422e62a8bfbba95c394fd0956ef58cb4d3e260 # via sphinx sphinx==7.4.7 ; python_full_version < '3.10' \ --hash=sha256:242f92a7ea7e6c5b406fdc2615413890ba9f699114a9c09192d7dfead2ee9cfe \ @@ -594,10 +598,14 @@ typing-extensions==4.15.0 \ # rules-python-docs (docs/pyproject.toml) # astroid # sphinx-autodoc2 -urllib3==2.6.3 \ +urllib3==2.6.3 ; python_full_version < '3.10' \ --hash=sha256:1b62b6884944a57dbe321509ab94fd4d3b307075e0c2eae991ac71ee15ad38ed \ --hash=sha256:bf272323e553dfb2e87d9bfd225ca7b0f467b919d7bbd355436d3fd37cb0acd4 # via requests +urllib3==2.7.0 ; python_full_version >= '3.10' \ + --hash=sha256:231e0ec3b63ceb14667c67be60f2f2c40a518cb38b03af60abc813da26505f4c \ + --hash=sha256:9fb4c81ebbb1ce9531cce37674bbc6f1360472bc18ca9a553ede278ef7276897 + # via requests zipp==3.23.1 ; python_full_version < '3.10' \ --hash=sha256:0b3596c50a5c700c9cb40ba8d86d9f2cc4807e9bedb06bcdf7fac85633e444dc \ --hash=sha256:32120e378d32cd9714ad503c1d024619063ec28aad2248dc6672ad13edfa5110 diff --git a/docs/uv.lock b/docs/uv.lock new file mode 100644 index 0000000000..72e8f307a9 --- /dev/null +++ b/docs/uv.lock @@ -0,0 +1,1163 @@ +version = 1 +revision = 3 +requires-python = ">=3.9" +resolution-markers = [ + "python_full_version >= '3.14'", + "python_full_version == '3.13.*'", + "python_full_version == '3.12.*'", + "python_full_version == '3.11.*'", + "python_full_version == '3.10.*'", + "python_full_version < '3.10'", +] + +[[package]] +name = "absl-py" +version = "2.3.1" +source = { registry = "https://pypi.org/simple" } +resolution-markers = [ + "python_full_version < '3.10'", +] +sdist = { url = "https://files.pythonhosted.org/packages/10/2a/c93173ffa1b39c1d0395b7e842bbdc62e556ca9d8d3b5572926f3e4ca752/absl_py-2.3.1.tar.gz", hash = "sha256:a97820526f7fbfd2ec1bce83f3f25e3a14840dac0d8e02a0b71cd75db3f77fc9", size = 116588, upload-time = "2025-07-03T09:31:44.05Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/8f/aa/ba0014cc4659328dc818a28827be78e6d97312ab0cb98105a770924dc11e/absl_py-2.3.1-py3-none-any.whl", hash = "sha256:eeecf07f0c2a93ace0772c92e596ace6d3d3996c042b2128459aaae2a76de11d", size = 135811, upload-time = "2025-07-03T09:31:42.253Z" }, +] + +[[package]] +name = "absl-py" +version = "2.4.0" +source = { registry = "https://pypi.org/simple" } +resolution-markers = [ + "python_full_version >= '3.14'", + "python_full_version == '3.13.*'", + "python_full_version == '3.12.*'", + "python_full_version == '3.11.*'", + "python_full_version == '3.10.*'", +] +sdist = { url = "https://files.pythonhosted.org/packages/64/c7/8de93764ad66968d19329a7e0c147a2bb3c7054c554d4a119111b8f9440f/absl_py-2.4.0.tar.gz", hash = "sha256:8c6af82722b35cf71e0f4d1d47dcaebfff286e27110a99fc359349b247dfb5d4", size = 116543, upload-time = "2026-01-28T10:17:05.322Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/18/a6/907a406bb7d359e6a63f99c313846d9eec4f7e6f7437809e03aa00fa3074/absl_py-2.4.0-py3-none-any.whl", hash = "sha256:88476fd881ca8aab94ffa78b7b6c632a782ab3ba1cd19c9bd423abc4fb4cd28d", size = 135750, upload-time = "2026-01-28T10:17:04.19Z" }, +] + +[[package]] +name = "alabaster" +version = "0.7.16" +source = { registry = "https://pypi.org/simple" } +resolution-markers = [ + "python_full_version < '3.10'", +] +sdist = { url = "https://files.pythonhosted.org/packages/c9/3e/13dd8e5ed9094e734ac430b5d0eb4f2bb001708a8b7856cbf8e084e001ba/alabaster-0.7.16.tar.gz", hash = "sha256:75a8b99c28a5dad50dd7f8ccdd447a121ddb3892da9e53d1ca5cca3106d58d65", size = 23776, upload-time = "2024-01-10T00:56:10.189Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/32/34/d4e1c02d3bee589efb5dfa17f88ea08bdb3e3eac12bc475462aec52ed223/alabaster-0.7.16-py3-none-any.whl", hash = "sha256:b46733c07dce03ae4e150330b975c75737fa60f0a7c591b6c8bf4928a28e2c92", size = 13511, upload-time = "2024-01-10T00:56:08.388Z" }, +] + +[[package]] +name = "alabaster" +version = "1.0.0" +source = { registry = "https://pypi.org/simple" } +resolution-markers = [ + "python_full_version >= '3.14'", + "python_full_version == '3.13.*'", + "python_full_version == '3.12.*'", + "python_full_version == '3.11.*'", + "python_full_version == '3.10.*'", +] +sdist = { url = "https://files.pythonhosted.org/packages/a6/f8/d9c74d0daf3f742840fd818d69cfae176fa332022fd44e3469487d5a9420/alabaster-1.0.0.tar.gz", hash = "sha256:c00dca57bca26fa62a6d7d0a9fcce65f3e026e9bfe33e9c538fd3fbb2144fd9e", size = 24210, upload-time = "2024-07-26T18:15:03.762Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/7e/b3/6b4067be973ae96ba0d615946e314c5ae35f9f993eca561b356540bb0c2b/alabaster-1.0.0-py3-none-any.whl", hash = "sha256:fc6786402dc3fcb2de3cabd5fe455a2db534b371124f1f21de8731783dec828b", size = 13929, upload-time = "2024-07-26T18:15:02.05Z" }, +] + +[[package]] +name = "altgraph" +version = "0.17.5" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/7e/f8/97fdf103f38fed6792a1601dbc16cc8aac56e7459a9fff08c812d8ae177a/altgraph-0.17.5.tar.gz", hash = "sha256:c87b395dd12fabde9c99573a9749d67da8d29ef9de0125c7f536699b4a9bc9e7", size = 48428, upload-time = "2025-11-21T20:35:50.583Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/a9/ba/000a1996d4308bc65120167c21241a3b205464a2e0b58deda26ae8ac21d1/altgraph-0.17.5-py2.py3-none-any.whl", hash = "sha256:f3a22400bce1b0c701683820ac4f3b159cd301acab067c51c653e06961600597", size = 21228, upload-time = "2025-11-21T20:35:49.444Z" }, +] + +[[package]] +name = "astroid" +version = "3.3.11" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "typing-extensions", marker = "python_full_version < '3.11'" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/18/74/dfb75f9ccd592bbedb175d4a32fc643cf569d7c218508bfbd6ea7ef9c091/astroid-3.3.11.tar.gz", hash = "sha256:1e5a5011af2920c7c67a53f65d536d65bfa7116feeaf2354d8b94f29573bb0ce", size = 400439, upload-time = "2025-07-13T18:04:23.177Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/af/0f/3b8fdc946b4d9cc8cc1e8af42c4e409468c84441b933d037e101b3d72d86/astroid-3.3.11-py3-none-any.whl", hash = "sha256:54c760ae8322ece1abd213057c4b5bba7c49818853fc901ef09719a60dbf9dec", size = 275612, upload-time = "2025-07-13T18:04:21.07Z" }, +] + +[[package]] +name = "babel" +version = "2.18.0" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/7d/b2/51899539b6ceeeb420d40ed3cd4b7a40519404f9baf3d4ac99dc413a834b/babel-2.18.0.tar.gz", hash = "sha256:b80b99a14bd085fcacfa15c9165f651fbb3406e66cc603abf11c5750937c992d", size = 9959554, upload-time = "2026-02-01T12:30:56.078Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/77/f5/21d2de20e8b8b0408f0681956ca2c69f1320a3848ac50e6e7f39c6159675/babel-2.18.0-py3-none-any.whl", hash = "sha256:e2b422b277c2b9a9630c1d7903c2a00d0830c409c59ac8cae9081c92f1aeba35", size = 10196845, upload-time = "2026-02-01T12:30:53.445Z" }, +] + +[[package]] +name = "certifi" +version = "2026.4.22" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/25/ee/6caf7a40c36a1220410afe15a1cc64993a1f864871f698c0f93acb72842a/certifi-2026.4.22.tar.gz", hash = "sha256:8d455352a37b71bf76a79caa83a3d6c25afee4a385d632127b6afb3963f1c580", size = 137077, upload-time = "2026-04-22T11:26:11.191Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/22/30/7cd8fdcdfbc5b869528b079bfb76dcdf6056b1a2097a662e5e8c04f42965/certifi-2026.4.22-py3-none-any.whl", hash = "sha256:3cb2210c8f88ba2318d29b0388d1023c8492ff72ecdde4ebdaddbb13a31b1c4a", size = 135707, upload-time = "2026-04-22T11:26:09.372Z" }, +] + +[[package]] +name = "charset-normalizer" +version = "3.4.7" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/e7/a1/67fe25fac3c7642725500a3f6cfe5821ad557c3abb11c9d20d12c7008d3e/charset_normalizer-3.4.7.tar.gz", hash = "sha256:ae89db9e5f98a11a4bf50407d4363e7b09b31e55bc117b4f7d80aab97ba009e5", size = 144271, upload-time = "2026-04-02T09:28:39.342Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/26/08/0f303cb0b529e456bb116f2d50565a482694fbb94340bf56d44677e7ed03/charset_normalizer-3.4.7-cp310-cp310-macosx_10_9_universal2.whl", hash = "sha256:cdd68a1fb318e290a2077696b7eb7a21a49163c455979c639bf5a5dcdc46617d", size = 315182, upload-time = "2026-04-02T09:25:40.673Z" }, + { url = "https://files.pythonhosted.org/packages/24/47/b192933e94b546f1b1fe4df9cc1f84fcdbf2359f8d1081d46dd029b50207/charset_normalizer-3.4.7-cp310-cp310-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:e17b8d5d6a8c47c85e68ca8379def1303fd360c3e22093a807cd34a71cd082b8", size = 209329, upload-time = "2026-04-02T09:25:42.354Z" }, + { url = "https://files.pythonhosted.org/packages/c2/b4/01fa81c5ca6141024d89a8fc15968002b71da7f825dd14113207113fabbd/charset_normalizer-3.4.7-cp310-cp310-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:511ef87c8aec0783e08ac18565a16d435372bc1ac25a91e6ac7f5ef2b0bff790", size = 231230, upload-time = "2026-04-02T09:25:44.281Z" }, + { url = "https://files.pythonhosted.org/packages/20/f7/7b991776844dfa058017e600e6e55ff01984a063290ca5622c0b63162f68/charset_normalizer-3.4.7-cp310-cp310-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:007d05ec7321d12a40227aae9e2bc6dca73f3cb21058999a1df9e193555a9dcc", size = 225890, upload-time = "2026-04-02T09:25:45.475Z" }, + { url = "https://files.pythonhosted.org/packages/20/e7/bed0024a0f4ab0c8a9c64d4445f39b30c99bd1acd228291959e3de664247/charset_normalizer-3.4.7-cp310-cp310-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:cf29836da5119f3c8a8a70667b0ef5fdca3bb12f80fd06487cfa575b3909b393", size = 216930, upload-time = "2026-04-02T09:25:46.58Z" }, + { url = "https://files.pythonhosted.org/packages/e2/ab/b18f0ab31cdd7b3ddb8bb76c4a414aeb8160c9810fdf1bc62f269a539d87/charset_normalizer-3.4.7-cp310-cp310-manylinux_2_31_armv7l.whl", hash = "sha256:12d8baf840cc7889b37c7c770f478adea7adce3dcb3944d02ec87508e2dcf153", size = 202109, upload-time = "2026-04-02T09:25:48.031Z" }, + { url = "https://files.pythonhosted.org/packages/82/e5/7e9440768a06dfb3075936490cb82dbf0ee20a133bf0dd8551fa096914ec/charset_normalizer-3.4.7-cp310-cp310-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:d560742f3c0d62afaccf9f41fe485ed69bd7661a241f86a3ef0f0fb8b1a397af", size = 214684, upload-time = "2026-04-02T09:25:49.245Z" }, + { url = "https://files.pythonhosted.org/packages/71/94/8c61d8da9f062fdf457c80acfa25060ec22bf1d34bbeaca4350f13bcfd07/charset_normalizer-3.4.7-cp310-cp310-musllinux_1_2_aarch64.whl", hash = "sha256:b14b2d9dac08e28bb8046a1a0434b1750eb221c8f5b87a68f4fa11a6f97b5e34", size = 212785, upload-time = "2026-04-02T09:25:50.671Z" }, + { url = "https://files.pythonhosted.org/packages/66/cd/6e9889c648e72c0ab2e5967528bb83508f354d706637bc7097190c874e13/charset_normalizer-3.4.7-cp310-cp310-musllinux_1_2_armv7l.whl", hash = "sha256:bc17a677b21b3502a21f66a8cc64f5bfad4df8a0b8434d661666f8ce90ac3af1", size = 203055, upload-time = "2026-04-02T09:25:51.802Z" }, + { url = "https://files.pythonhosted.org/packages/92/2e/7a951d6a08aefb7eb8e1b54cdfb580b1365afdd9dd484dc4bee9e5d8f258/charset_normalizer-3.4.7-cp310-cp310-musllinux_1_2_ppc64le.whl", hash = "sha256:750e02e074872a3fad7f233b47734166440af3cdea0add3e95163110816d6752", size = 232502, upload-time = "2026-04-02T09:25:53.388Z" }, + { url = "https://files.pythonhosted.org/packages/58/d5/abcf2d83bf8e0a1286df55cd0dc1d49af0da4282aa77e986df343e7de124/charset_normalizer-3.4.7-cp310-cp310-musllinux_1_2_riscv64.whl", hash = "sha256:4e5163c14bffd570ef2affbfdd77bba66383890797df43dc8b4cc7d6f500bf53", size = 214295, upload-time = "2026-04-02T09:25:54.765Z" }, + { url = "https://files.pythonhosted.org/packages/47/3a/7d4cd7ed54be99973a0dc176032cba5cb1f258082c31fa6df35cff46acfc/charset_normalizer-3.4.7-cp310-cp310-musllinux_1_2_s390x.whl", hash = "sha256:6ed74185b2db44f41ef35fd1617c5888e59792da9bbc9190d6c7300617182616", size = 227145, upload-time = "2026-04-02T09:25:55.904Z" }, + { url = "https://files.pythonhosted.org/packages/1d/98/3a45bf8247889cf28262ebd3d0872edff11565b2a1e3064ccb132db3fbb0/charset_normalizer-3.4.7-cp310-cp310-musllinux_1_2_x86_64.whl", hash = "sha256:94e1885b270625a9a828c9793b4d52a64445299baa1fea5a173bf1d3dd9a1a5a", size = 218884, upload-time = "2026-04-02T09:25:57.074Z" }, + { url = "https://files.pythonhosted.org/packages/ad/80/2e8b7f8915ed5c9ef13aa828d82738e33888c485b65ebf744d615040c7ea/charset_normalizer-3.4.7-cp310-cp310-win32.whl", hash = "sha256:6785f414ae0f3c733c437e0f3929197934f526d19dfaa75e18fdb4f94c6fb374", size = 148343, upload-time = "2026-04-02T09:25:58.199Z" }, + { url = "https://files.pythonhosted.org/packages/35/1b/3b8c8c77184af465ee9ad88b5aea46ea6b2e1f7b9dc9502891e37af21e30/charset_normalizer-3.4.7-cp310-cp310-win_amd64.whl", hash = "sha256:6696b7688f54f5af4462118f0bfa7c1621eeb87154f77fa04b9295ce7a8f2943", size = 159174, upload-time = "2026-04-02T09:25:59.322Z" }, + { url = "https://files.pythonhosted.org/packages/be/c1/feb40dca40dbb21e0a908801782d9288c64fc8d8e562c2098e9994c8c21b/charset_normalizer-3.4.7-cp310-cp310-win_arm64.whl", hash = "sha256:66671f93accb62ed07da56613636f3641f1a12c13046ce91ffc923721f23c008", size = 147805, upload-time = "2026-04-02T09:26:00.756Z" }, + { url = "https://files.pythonhosted.org/packages/c2/d7/b5b7020a0565c2e9fa8c09f4b5fa6232feb326b8c20081ccded47ea368fd/charset_normalizer-3.4.7-cp311-cp311-macosx_10_9_universal2.whl", hash = "sha256:7641bb8895e77f921102f72833904dcd9901df5d6d72a2ab8f31d04b7e51e4e7", size = 309705, upload-time = "2026-04-02T09:26:02.191Z" }, + { url = "https://files.pythonhosted.org/packages/5a/53/58c29116c340e5456724ecd2fff4196d236b98f3da97b404bc5e51ac3493/charset_normalizer-3.4.7-cp311-cp311-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:202389074300232baeb53ae2569a60901f7efadd4245cf3a3bf0617d60b439d7", size = 206419, upload-time = "2026-04-02T09:26:03.583Z" }, + { url = "https://files.pythonhosted.org/packages/b2/02/e8146dc6591a37a00e5144c63f29fb7c97a734ea8a111190783c0e60ab63/charset_normalizer-3.4.7-cp311-cp311-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:30b8d1d8c52a48c2c5690e152c169b673487a2a58de1ec7393196753063fcd5e", size = 227901, upload-time = "2026-04-02T09:26:04.738Z" }, + { url = "https://files.pythonhosted.org/packages/fb/73/77486c4cd58f1267bf17db420e930c9afa1b3be3fe8c8b8ebbebc9624359/charset_normalizer-3.4.7-cp311-cp311-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:532bc9bf33a68613fd7d65e4b1c71a6a38d7d42604ecf239c77392e9b4e8998c", size = 222742, upload-time = "2026-04-02T09:26:06.36Z" }, + { url = "https://files.pythonhosted.org/packages/a1/fa/f74eb381a7d94ded44739e9d94de18dc5edc9c17fb8c11f0a6890696c0a9/charset_normalizer-3.4.7-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:2fe249cb4651fd12605b7288b24751d8bfd46d35f12a20b1ba33dea122e690df", size = 214061, upload-time = "2026-04-02T09:26:08.347Z" }, + { url = "https://files.pythonhosted.org/packages/dc/92/42bd3cefcf7687253fb86694b45f37b733c97f59af3724f356fa92b8c344/charset_normalizer-3.4.7-cp311-cp311-manylinux_2_31_armv7l.whl", hash = "sha256:65bcd23054beab4d166035cabbc868a09c1a49d1efe458fe8e4361215df40265", size = 199239, upload-time = "2026-04-02T09:26:09.823Z" }, + { url = "https://files.pythonhosted.org/packages/4c/3d/069e7184e2aa3b3cddc700e3dd267413dc259854adc3380421c805c6a17d/charset_normalizer-3.4.7-cp311-cp311-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:08e721811161356f97b4059a9ba7bafb23ea5ee2255402c42881c214e173c6b4", size = 210173, upload-time = "2026-04-02T09:26:10.953Z" }, + { url = "https://files.pythonhosted.org/packages/62/51/9d56feb5f2e7074c46f93e0ebdbe61f0848ee246e2f0d89f8e20b89ebb8f/charset_normalizer-3.4.7-cp311-cp311-musllinux_1_2_aarch64.whl", hash = "sha256:e060d01aec0a910bdccb8be71faf34e7799ce36950f8294c8bf612cba65a2c9e", size = 209841, upload-time = "2026-04-02T09:26:12.142Z" }, + { url = "https://files.pythonhosted.org/packages/d2/59/893d8f99cc4c837dda1fe2f1139079703deb9f321aabcb032355de13b6c7/charset_normalizer-3.4.7-cp311-cp311-musllinux_1_2_armv7l.whl", hash = "sha256:38c0109396c4cfc574d502df99742a45c72c08eff0a36158b6f04000043dbf38", size = 200304, upload-time = "2026-04-02T09:26:13.711Z" }, + { url = "https://files.pythonhosted.org/packages/7d/1d/ee6f3be3464247578d1ed5c46de545ccc3d3ff933695395c402c21fa6b77/charset_normalizer-3.4.7-cp311-cp311-musllinux_1_2_ppc64le.whl", hash = "sha256:1c2a768fdd44ee4a9339a9b0b130049139b8ce3c01d2ce09f67f5a68048d477c", size = 229455, upload-time = "2026-04-02T09:26:14.941Z" }, + { url = "https://files.pythonhosted.org/packages/54/bb/8fb0a946296ea96a488928bdce8ef99023998c48e4713af533e9bb98ef07/charset_normalizer-3.4.7-cp311-cp311-musllinux_1_2_riscv64.whl", hash = "sha256:1a87ca9d5df6fe460483d9a5bbf2b18f620cbed41b432e2bddb686228282d10b", size = 210036, upload-time = "2026-04-02T09:26:16.478Z" }, + { url = "https://files.pythonhosted.org/packages/9a/bc/015b2387f913749f82afd4fcba07846d05b6d784dd16123cb66860e0237d/charset_normalizer-3.4.7-cp311-cp311-musllinux_1_2_s390x.whl", hash = "sha256:d635aab80466bc95771bb78d5370e74d36d1fe31467b6b29b8b57b2a3cd7d22c", size = 224739, upload-time = "2026-04-02T09:26:17.751Z" }, + { url = "https://files.pythonhosted.org/packages/17/ab/63133691f56baae417493cba6b7c641571a2130eb7bceba6773367ab9ec5/charset_normalizer-3.4.7-cp311-cp311-musllinux_1_2_x86_64.whl", hash = "sha256:ae196f021b5e7c78e918242d217db021ed2a6ace2bc6ae94c0fc596221c7f58d", size = 216277, upload-time = "2026-04-02T09:26:18.981Z" }, + { url = "https://files.pythonhosted.org/packages/06/6d/3be70e827977f20db77c12a97e6a9f973631a45b8d186c084527e53e77a4/charset_normalizer-3.4.7-cp311-cp311-win32.whl", hash = "sha256:adb2597b428735679446b46c8badf467b4ca5f5056aae4d51a19f9570301b1ad", size = 147819, upload-time = "2026-04-02T09:26:20.295Z" }, + { url = "https://files.pythonhosted.org/packages/20/d9/5f67790f06b735d7c7637171bbfd89882ad67201891b7275e51116ed8207/charset_normalizer-3.4.7-cp311-cp311-win_amd64.whl", hash = "sha256:8e385e4267ab76874ae30db04c627faaaf0b509e1ccc11a95b3fc3e83f855c00", size = 159281, upload-time = "2026-04-02T09:26:21.74Z" }, + { url = "https://files.pythonhosted.org/packages/ca/83/6413f36c5a34afead88ce6f66684d943d91f233d76dd083798f9602b75ae/charset_normalizer-3.4.7-cp311-cp311-win_arm64.whl", hash = "sha256:d4a48e5b3c2a489fae013b7589308a40146ee081f6f509e047e0e096084ceca1", size = 147843, upload-time = "2026-04-02T09:26:22.901Z" }, + { url = "https://files.pythonhosted.org/packages/0c/eb/4fc8d0a7110eb5fc9cc161723a34a8a6c200ce3b4fbf681bc86feee22308/charset_normalizer-3.4.7-cp312-cp312-macosx_10_13_universal2.whl", hash = "sha256:eca9705049ad3c7345d574e3510665cb2cf844c2f2dcfe675332677f081cbd46", size = 311328, upload-time = "2026-04-02T09:26:24.331Z" }, + { url = "https://files.pythonhosted.org/packages/f8/e3/0fadc706008ac9d7b9b5be6dc767c05f9d3e5df51744ce4cc9605de7b9f4/charset_normalizer-3.4.7-cp312-cp312-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:6178f72c5508bfc5fd446a5905e698c6212932f25bcdd4b47a757a50605a90e2", size = 208061, upload-time = "2026-04-02T09:26:25.568Z" }, + { url = "https://files.pythonhosted.org/packages/42/f0/3dd1045c47f4a4604df85ec18ad093912ae1344ac706993aff91d38773a2/charset_normalizer-3.4.7-cp312-cp312-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:e1421b502d83040e6d7fb2fb18dff63957f720da3d77b2fbd3187ceb63755d7b", size = 229031, upload-time = "2026-04-02T09:26:26.865Z" }, + { url = "https://files.pythonhosted.org/packages/dc/67/675a46eb016118a2fbde5a277a5d15f4f69d5f3f5f338e5ee2f8948fcf43/charset_normalizer-3.4.7-cp312-cp312-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:edac0f1ab77644605be2cbba52e6b7f630731fc42b34cb0f634be1a6eface56a", size = 225239, upload-time = "2026-04-02T09:26:28.044Z" }, + { url = "https://files.pythonhosted.org/packages/4b/f8/d0118a2f5f23b02cd166fa385c60f9b0d4f9194f574e2b31cef350ad7223/charset_normalizer-3.4.7-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:5649fd1c7bade02f320a462fdefd0b4bd3ce036065836d4f42e0de958038e116", size = 216589, upload-time = "2026-04-02T09:26:29.239Z" }, + { url = "https://files.pythonhosted.org/packages/b1/f1/6d2b0b261b6c4ceef0fcb0d17a01cc5bc53586c2d4796fa04b5c540bc13d/charset_normalizer-3.4.7-cp312-cp312-manylinux_2_31_armv7l.whl", hash = "sha256:203104ed3e428044fd943bc4bf45fa73c0730391f9621e37fe39ecf477b128cb", size = 202733, upload-time = "2026-04-02T09:26:30.5Z" }, + { url = "https://files.pythonhosted.org/packages/6f/c0/7b1f943f7e87cc3db9626ba17807d042c38645f0a1d4415c7a14afb5591f/charset_normalizer-3.4.7-cp312-cp312-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:298930cec56029e05497a76988377cbd7457ba864beeea92ad7e844fe74cd1f1", size = 212652, upload-time = "2026-04-02T09:26:31.709Z" }, + { url = "https://files.pythonhosted.org/packages/38/dd/5a9ab159fe45c6e72079398f277b7d2b523e7f716acc489726115a910097/charset_normalizer-3.4.7-cp312-cp312-musllinux_1_2_aarch64.whl", hash = "sha256:708838739abf24b2ceb208d0e22403dd018faeef86ddac04319a62ae884c4f15", size = 211229, upload-time = "2026-04-02T09:26:33.282Z" }, + { url = "https://files.pythonhosted.org/packages/d5/ff/531a1cad5ca855d1c1a8b69cb71abfd6d85c0291580146fda7c82857caa1/charset_normalizer-3.4.7-cp312-cp312-musllinux_1_2_armv7l.whl", hash = "sha256:0f7eb884681e3938906ed0434f20c63046eacd0111c4ba96f27b76084cd679f5", size = 203552, upload-time = "2026-04-02T09:26:34.845Z" }, + { url = "https://files.pythonhosted.org/packages/c1/4c/a5fb52d528a8ca41f7598cb619409ece30a169fbdf9cdce592e53b46c3a6/charset_normalizer-3.4.7-cp312-cp312-musllinux_1_2_ppc64le.whl", hash = "sha256:4dc1e73c36828f982bfe79fadf5919923f8a6f4df2860804db9a98c48824ce8d", size = 230806, upload-time = "2026-04-02T09:26:36.152Z" }, + { url = "https://files.pythonhosted.org/packages/59/7a/071feed8124111a32b316b33ae4de83d36923039ef8cf48120266844285b/charset_normalizer-3.4.7-cp312-cp312-musllinux_1_2_riscv64.whl", hash = "sha256:aed52fea0513bac0ccde438c188c8a471c4e0f457c2dd20cdbf6ea7a450046c7", size = 212316, upload-time = "2026-04-02T09:26:37.672Z" }, + { url = "https://files.pythonhosted.org/packages/fd/35/f7dba3994312d7ba508e041eaac39a36b120f32d4c8662b8814dab876431/charset_normalizer-3.4.7-cp312-cp312-musllinux_1_2_s390x.whl", hash = "sha256:fea24543955a6a729c45a73fe90e08c743f0b3334bbf3201e6c4bc1b0c7fa464", size = 227274, upload-time = "2026-04-02T09:26:38.93Z" }, + { url = "https://files.pythonhosted.org/packages/8a/2d/a572df5c9204ab7688ec1edc895a73ebded3b023bb07364710b05dd1c9be/charset_normalizer-3.4.7-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:bb6d88045545b26da47aa879dd4a89a71d1dce0f0e549b1abcb31dfe4a8eac49", size = 218468, upload-time = "2026-04-02T09:26:40.17Z" }, + { url = "https://files.pythonhosted.org/packages/86/eb/890922a8b03a568ca2f336c36585a4713c55d4d67bf0f0c78924be6315ca/charset_normalizer-3.4.7-cp312-cp312-win32.whl", hash = "sha256:2257141f39fe65a3fdf38aeccae4b953e5f3b3324f4ff0daf9f15b8518666a2c", size = 148460, upload-time = "2026-04-02T09:26:41.416Z" }, + { url = "https://files.pythonhosted.org/packages/35/d9/0e7dffa06c5ab081f75b1b786f0aefc88365825dfcd0ac544bdb7b2b6853/charset_normalizer-3.4.7-cp312-cp312-win_amd64.whl", hash = "sha256:5ed6ab538499c8644b8a3e18debabcd7ce684f3fa91cf867521a7a0279cab2d6", size = 159330, upload-time = "2026-04-02T09:26:42.554Z" }, + { url = "https://files.pythonhosted.org/packages/9e/5d/481bcc2a7c88ea6b0878c299547843b2521ccbc40980cb406267088bc701/charset_normalizer-3.4.7-cp312-cp312-win_arm64.whl", hash = "sha256:56be790f86bfb2c98fb742ce566dfb4816e5a83384616ab59c49e0604d49c51d", size = 147828, upload-time = "2026-04-02T09:26:44.075Z" }, + { url = "https://files.pythonhosted.org/packages/c1/3b/66777e39d3ae1ddc77ee606be4ec6d8cbd4c801f65e5a1b6f2b11b8346dd/charset_normalizer-3.4.7-cp313-cp313-macosx_10_13_universal2.whl", hash = "sha256:f496c9c3cc02230093d8330875c4c3cdfc3b73612a5fd921c65d39cbcef08063", size = 309627, upload-time = "2026-04-02T09:26:45.198Z" }, + { url = "https://files.pythonhosted.org/packages/2e/4e/b7f84e617b4854ade48a1b7915c8ccfadeba444d2a18c291f696e37f0d3b/charset_normalizer-3.4.7-cp313-cp313-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:0ea948db76d31190bf08bd371623927ee1339d5f2a0b4b1b4a4439a65298703c", size = 207008, upload-time = "2026-04-02T09:26:46.824Z" }, + { url = "https://files.pythonhosted.org/packages/c4/bb/ec73c0257c9e11b268f018f068f5d00aa0ef8c8b09f7753ebd5f2880e248/charset_normalizer-3.4.7-cp313-cp313-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:a277ab8928b9f299723bc1a2dabb1265911b1a76341f90a510368ca44ad9ab66", size = 228303, upload-time = "2026-04-02T09:26:48.397Z" }, + { url = "https://files.pythonhosted.org/packages/85/fb/32d1f5033484494619f701e719429c69b766bfc4dbc61aa9e9c8c166528b/charset_normalizer-3.4.7-cp313-cp313-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:3bec022aec2c514d9cf199522a802bd007cd588ab17ab2525f20f9c34d067c18", size = 224282, upload-time = "2026-04-02T09:26:49.684Z" }, + { url = "https://files.pythonhosted.org/packages/fa/07/330e3a0dda4c404d6da83b327270906e9654a24f6c546dc886a0eb0ffb23/charset_normalizer-3.4.7-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:e044c39e41b92c845bc815e5ae4230804e8e7bc29e399b0437d64222d92809dd", size = 215595, upload-time = "2026-04-02T09:26:50.915Z" }, + { url = "https://files.pythonhosted.org/packages/e3/7c/fc890655786e423f02556e0216d4b8c6bcb6bdfa890160dc66bf52dee468/charset_normalizer-3.4.7-cp313-cp313-manylinux_2_31_armv7l.whl", hash = "sha256:f495a1652cf3fbab2eb0639776dad966c2fb874d79d87ca07f9d5f059b8bd215", size = 201986, upload-time = "2026-04-02T09:26:52.197Z" }, + { url = "https://files.pythonhosted.org/packages/d8/97/bfb18b3db2aed3b90cf54dc292ad79fdd5ad65c4eae454099475cbeadd0d/charset_normalizer-3.4.7-cp313-cp313-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:e712b419df8ba5e42b226c510472b37bd57b38e897d3eca5e8cfd410a29fa859", size = 211711, upload-time = "2026-04-02T09:26:53.49Z" }, + { url = "https://files.pythonhosted.org/packages/6f/a5/a581c13798546a7fd557c82614a5c65a13df2157e9ad6373166d2a3e645d/charset_normalizer-3.4.7-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:7804338df6fcc08105c7745f1502ba68d900f45fd770d5bdd5288ddccb8a42d8", size = 210036, upload-time = "2026-04-02T09:26:54.975Z" }, + { url = "https://files.pythonhosted.org/packages/8c/bf/b3ab5bcb478e4193d517644b0fb2bf5497fbceeaa7a1bc0f4d5b50953861/charset_normalizer-3.4.7-cp313-cp313-musllinux_1_2_armv7l.whl", hash = "sha256:481551899c856c704d58119b5025793fa6730adda3571971af568f66d2424bb5", size = 202998, upload-time = "2026-04-02T09:26:56.303Z" }, + { url = "https://files.pythonhosted.org/packages/e7/4e/23efd79b65d314fa320ec6017b4b5834d5c12a58ba4610aa353af2e2f577/charset_normalizer-3.4.7-cp313-cp313-musllinux_1_2_ppc64le.whl", hash = "sha256:f59099f9b66f0d7145115e6f80dd8b1d847176df89b234a5a6b3f00437aa0832", size = 230056, upload-time = "2026-04-02T09:26:57.554Z" }, + { url = "https://files.pythonhosted.org/packages/b9/9f/1e1941bc3f0e01df116e68dc37a55c4d249df5e6fa77f008841aef68264f/charset_normalizer-3.4.7-cp313-cp313-musllinux_1_2_riscv64.whl", hash = "sha256:f59ad4c0e8f6bba240a9bb85504faa1ab438237199d4cce5f622761507b8f6a6", size = 211537, upload-time = "2026-04-02T09:26:58.843Z" }, + { url = "https://files.pythonhosted.org/packages/80/0f/088cbb3020d44428964a6c97fe1edfb1b9550396bf6d278330281e8b709c/charset_normalizer-3.4.7-cp313-cp313-musllinux_1_2_s390x.whl", hash = "sha256:3dedcc22d73ec993f42055eff4fcfed9318d1eeb9a6606c55892a26964964e48", size = 226176, upload-time = "2026-04-02T09:27:00.437Z" }, + { url = "https://files.pythonhosted.org/packages/6a/9f/130394f9bbe06f4f63e22641d32fc9b202b7e251c9aef4db044324dac493/charset_normalizer-3.4.7-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:64f02c6841d7d83f832cd97ccf8eb8a906d06eb95d5276069175c696b024b60a", size = 217723, upload-time = "2026-04-02T09:27:02.021Z" }, + { url = "https://files.pythonhosted.org/packages/73/55/c469897448a06e49f8fa03f6caae97074fde823f432a98f979cc42b90e69/charset_normalizer-3.4.7-cp313-cp313-win32.whl", hash = "sha256:4042d5c8f957e15221d423ba781e85d553722fc4113f523f2feb7b188cc34c5e", size = 148085, upload-time = "2026-04-02T09:27:03.192Z" }, + { url = "https://files.pythonhosted.org/packages/5d/78/1b74c5bbb3f99b77a1715c91b3e0b5bdb6fe302d95ace4f5b1bec37b0167/charset_normalizer-3.4.7-cp313-cp313-win_amd64.whl", hash = "sha256:3946fa46a0cf3e4c8cb1cc52f56bb536310d34f25f01ca9b6c16afa767dab110", size = 158819, upload-time = "2026-04-02T09:27:04.454Z" }, + { url = "https://files.pythonhosted.org/packages/68/86/46bd42279d323deb8687c4a5a811fd548cb7d1de10cf6535d099877a9a9f/charset_normalizer-3.4.7-cp313-cp313-win_arm64.whl", hash = "sha256:80d04837f55fc81da168b98de4f4b797ef007fc8a79ab71c6ec9bc4dd662b15b", size = 147915, upload-time = "2026-04-02T09:27:05.971Z" }, + { url = "https://files.pythonhosted.org/packages/97/c8/c67cb8c70e19ef1960b97b22ed2a1567711de46c4ddf19799923adc836c2/charset_normalizer-3.4.7-cp314-cp314-macosx_10_15_universal2.whl", hash = "sha256:c36c333c39be2dbca264d7803333c896ab8fa7d4d6f0ab7edb7dfd7aea6e98c0", size = 309234, upload-time = "2026-04-02T09:27:07.194Z" }, + { url = "https://files.pythonhosted.org/packages/99/85/c091fdee33f20de70d6c8b522743b6f831a2f1cd3ff86de4c6a827c48a76/charset_normalizer-3.4.7-cp314-cp314-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:1c2aed2e5e41f24ea8ef1590b8e848a79b56f3a5564a65ceec43c9d692dc7d8a", size = 208042, upload-time = "2026-04-02T09:27:08.749Z" }, + { url = "https://files.pythonhosted.org/packages/87/1c/ab2ce611b984d2fd5d86a5a8a19c1ae26acac6bad967da4967562c75114d/charset_normalizer-3.4.7-cp314-cp314-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:54523e136b8948060c0fa0bc7b1b50c32c186f2fceee897a495406bb6e311d2b", size = 228706, upload-time = "2026-04-02T09:27:09.951Z" }, + { url = "https://files.pythonhosted.org/packages/a8/29/2b1d2cb00bf085f59d29eb773ce58ec2d325430f8c216804a0a5cd83cbca/charset_normalizer-3.4.7-cp314-cp314-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:715479b9a2802ecac752a3b0efa2b0b60285cf962ee38414211abdfccc233b41", size = 224727, upload-time = "2026-04-02T09:27:11.175Z" }, + { url = "https://files.pythonhosted.org/packages/47/5c/032c2d5a07fe4d4855fea851209cca2b6f03ebeb6d4e3afdb3358386a684/charset_normalizer-3.4.7-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:bd6c2a1c7573c64738d716488d2cdd3c00e340e4835707d8fdb8dc1a66ef164e", size = 215882, upload-time = "2026-04-02T09:27:12.446Z" }, + { url = "https://files.pythonhosted.org/packages/2c/c2/356065d5a8b78ed04499cae5f339f091946a6a74f91e03476c33f0ab7100/charset_normalizer-3.4.7-cp314-cp314-manylinux_2_31_armv7l.whl", hash = "sha256:c45e9440fb78f8ddabcf714b68f936737a121355bf59f3907f4e17721b9d1aae", size = 200860, upload-time = "2026-04-02T09:27:13.721Z" }, + { url = "https://files.pythonhosted.org/packages/0c/cd/a32a84217ced5039f53b29f460962abb2d4420def55afabe45b1c3c7483d/charset_normalizer-3.4.7-cp314-cp314-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:3534e7dcbdcf757da6b85a0bbf5b6868786d5982dd959b065e65481644817a18", size = 211564, upload-time = "2026-04-02T09:27:15.272Z" }, + { url = "https://files.pythonhosted.org/packages/44/86/58e6f13ce26cc3b8f4a36b94a0f22ae2f00a72534520f4ae6857c4b81f89/charset_normalizer-3.4.7-cp314-cp314-musllinux_1_2_aarch64.whl", hash = "sha256:e8ac484bf18ce6975760921bb6148041faa8fef0547200386ea0b52b5d27bf7b", size = 211276, upload-time = "2026-04-02T09:27:16.834Z" }, + { url = "https://files.pythonhosted.org/packages/8f/fe/d17c32dc72e17e155e06883efa84514ca375f8a528ba2546bee73fc4df81/charset_normalizer-3.4.7-cp314-cp314-musllinux_1_2_armv7l.whl", hash = "sha256:a5fe03b42827c13cdccd08e6c0247b6a6d4b5e3cdc53fd1749f5896adcdc2356", size = 201238, upload-time = "2026-04-02T09:27:18.229Z" }, + { url = "https://files.pythonhosted.org/packages/6a/29/f33daa50b06525a237451cdb6c69da366c381a3dadcd833fa5676bc468b3/charset_normalizer-3.4.7-cp314-cp314-musllinux_1_2_ppc64le.whl", hash = "sha256:2d6eb928e13016cea4f1f21d1e10c1cebd5a421bc57ddf5b1142ae3f86824fab", size = 230189, upload-time = "2026-04-02T09:27:19.445Z" }, + { url = "https://files.pythonhosted.org/packages/b6/6e/52c84015394a6a0bdcd435210a7e944c5f94ea1055f5cc5d56c5fe368e7b/charset_normalizer-3.4.7-cp314-cp314-musllinux_1_2_riscv64.whl", hash = "sha256:e74327fb75de8986940def6e8dee4f127cc9752bee7355bb323cc5b2659b6d46", size = 211352, upload-time = "2026-04-02T09:27:20.79Z" }, + { url = "https://files.pythonhosted.org/packages/8c/d7/4353be581b373033fb9198bf1da3cf8f09c1082561e8e922aa7b39bf9fe8/charset_normalizer-3.4.7-cp314-cp314-musllinux_1_2_s390x.whl", hash = "sha256:d6038d37043bced98a66e68d3aa2b6a35505dc01328cd65217cefe82f25def44", size = 227024, upload-time = "2026-04-02T09:27:22.063Z" }, + { url = "https://files.pythonhosted.org/packages/30/45/99d18aa925bd1740098ccd3060e238e21115fffbfdcb8f3ece837d0ace6c/charset_normalizer-3.4.7-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:7579e913a5339fb8fa133f6bbcfd8e6749696206cf05acdbdca71a1b436d8e72", size = 217869, upload-time = "2026-04-02T09:27:23.486Z" }, + { url = "https://files.pythonhosted.org/packages/5c/05/5ee478aa53f4bb7996482153d4bfe1b89e0f087f0ab6b294fcf92d595873/charset_normalizer-3.4.7-cp314-cp314-win32.whl", hash = "sha256:5b77459df20e08151cd6f8b9ef8ef1f961ef73d85c21a555c7eed5b79410ec10", size = 148541, upload-time = "2026-04-02T09:27:25.146Z" }, + { url = "https://files.pythonhosted.org/packages/48/77/72dcb0921b2ce86420b2d79d454c7022bf5be40202a2a07906b9f2a35c97/charset_normalizer-3.4.7-cp314-cp314-win_amd64.whl", hash = "sha256:92a0a01ead5e668468e952e4238cccd7c537364eb7d851ab144ab6627dbbe12f", size = 159634, upload-time = "2026-04-02T09:27:26.642Z" }, + { url = "https://files.pythonhosted.org/packages/c6/a3/c2369911cd72f02386e4e340770f6e158c7980267da16af8f668217abaa0/charset_normalizer-3.4.7-cp314-cp314-win_arm64.whl", hash = "sha256:67f6279d125ca0046a7fd386d01b311c6363844deac3e5b069b514ba3e63c246", size = 148384, upload-time = "2026-04-02T09:27:28.271Z" }, + { url = "https://files.pythonhosted.org/packages/94/09/7e8a7f73d24dba1f0035fbbf014d2c36828fc1bf9c88f84093e57d315935/charset_normalizer-3.4.7-cp314-cp314t-macosx_10_15_universal2.whl", hash = "sha256:effc3f449787117233702311a1b7d8f59cba9ced946ba727bdc329ec69028e24", size = 330133, upload-time = "2026-04-02T09:27:29.474Z" }, + { url = "https://files.pythonhosted.org/packages/8d/da/96975ddb11f8e977f706f45cddd8540fd8242f71ecdb5d18a80723dcf62c/charset_normalizer-3.4.7-cp314-cp314t-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:fbccdc05410c9ee21bbf16a35f4c1d16123dcdeb8a1d38f33654fa21d0234f79", size = 216257, upload-time = "2026-04-02T09:27:30.793Z" }, + { url = "https://files.pythonhosted.org/packages/e5/e8/1d63bf8ef2d388e95c64b2098f45f84758f6d102a087552da1485912637b/charset_normalizer-3.4.7-cp314-cp314t-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:733784b6d6def852c814bce5f318d25da2ee65dd4839a0718641c696e09a2960", size = 234851, upload-time = "2026-04-02T09:27:32.44Z" }, + { url = "https://files.pythonhosted.org/packages/9b/40/e5ff04233e70da2681fa43969ad6f66ca5611d7e669be0246c4c7aaf6dc8/charset_normalizer-3.4.7-cp314-cp314t-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:a89c23ef8d2c6b27fd200a42aa4ac72786e7c60d40efdc76e6011260b6e949c4", size = 233393, upload-time = "2026-04-02T09:27:34.03Z" }, + { url = "https://files.pythonhosted.org/packages/be/c1/06c6c49d5a5450f76899992f1ee40b41d076aee9279b49cf9974d2f313d5/charset_normalizer-3.4.7-cp314-cp314t-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:6c114670c45346afedc0d947faf3c7f701051d2518b943679c8ff88befe14f8e", size = 223251, upload-time = "2026-04-02T09:27:35.369Z" }, + { url = "https://files.pythonhosted.org/packages/2b/9f/f2ff16fb050946169e3e1f82134d107e5d4ae72647ec8a1b1446c148480f/charset_normalizer-3.4.7-cp314-cp314t-manylinux_2_31_armv7l.whl", hash = "sha256:a180c5e59792af262bf263b21a3c49353f25945d8d9f70628e73de370d55e1e1", size = 206609, upload-time = "2026-04-02T09:27:36.661Z" }, + { url = "https://files.pythonhosted.org/packages/69/d5/a527c0cd8d64d2eab7459784fb4169a0ac76e5a6fc5237337982fd61347e/charset_normalizer-3.4.7-cp314-cp314t-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:3c9a494bc5ec77d43cea229c4f6db1e4d8fe7e1bbffa8b6f0f0032430ff8ab44", size = 220014, upload-time = "2026-04-02T09:27:38.019Z" }, + { url = "https://files.pythonhosted.org/packages/7e/80/8a7b8104a3e203074dc9aa2c613d4b726c0e136bad1cc734594b02867972/charset_normalizer-3.4.7-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:8d828b6667a32a728a1ad1d93957cdf37489c57b97ae6c4de2860fa749b8fc1e", size = 218979, upload-time = "2026-04-02T09:27:39.37Z" }, + { url = "https://files.pythonhosted.org/packages/02/9a/b759b503d507f375b2b5c153e4d2ee0a75aa215b7f2489cf314f4541f2c0/charset_normalizer-3.4.7-cp314-cp314t-musllinux_1_2_armv7l.whl", hash = "sha256:cf1493cd8607bec4d8a7b9b004e699fcf8f9103a9284cc94962cb73d20f9d4a3", size = 209238, upload-time = "2026-04-02T09:27:40.722Z" }, + { url = "https://files.pythonhosted.org/packages/c2/4e/0f3f5d47b86bdb79256e7290b26ac847a2832d9a4033f7eb2cd4bcf4bb5b/charset_normalizer-3.4.7-cp314-cp314t-musllinux_1_2_ppc64le.whl", hash = "sha256:0c96c3b819b5c3e9e165495db84d41914d6894d55181d2d108cc1a69bfc9cce0", size = 236110, upload-time = "2026-04-02T09:27:42.33Z" }, + { url = "https://files.pythonhosted.org/packages/96/23/bce28734eb3ed2c91dcf93abeb8a5cf393a7b2749725030bb630e554fdd8/charset_normalizer-3.4.7-cp314-cp314t-musllinux_1_2_riscv64.whl", hash = "sha256:752a45dc4a6934060b3b0dab47e04edc3326575f82be64bc4fc293914566503e", size = 219824, upload-time = "2026-04-02T09:27:43.924Z" }, + { url = "https://files.pythonhosted.org/packages/2c/6f/6e897c6984cc4d41af319b077f2f600fc8214eb2fe2d6bcb79141b882400/charset_normalizer-3.4.7-cp314-cp314t-musllinux_1_2_s390x.whl", hash = "sha256:8778f0c7a52e56f75d12dae53ae320fae900a8b9b4164b981b9c5ce059cd1fcb", size = 233103, upload-time = "2026-04-02T09:27:45.348Z" }, + { url = "https://files.pythonhosted.org/packages/76/22/ef7bd0fe480a0ae9b656189ec00744b60933f68b4f42a7bb06589f6f576a/charset_normalizer-3.4.7-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:ce3412fbe1e31eb81ea42f4169ed94861c56e643189e1e75f0041f3fe7020abe", size = 225194, upload-time = "2026-04-02T09:27:46.706Z" }, + { url = "https://files.pythonhosted.org/packages/c5/a7/0e0ab3e0b5bc1219bd80a6a0d4d72ca74d9250cb2382b7c699c147e06017/charset_normalizer-3.4.7-cp314-cp314t-win32.whl", hash = "sha256:c03a41a8784091e67a39648f70c5f97b5b6a37f216896d44d2cdcb82615339a0", size = 159827, upload-time = "2026-04-02T09:27:48.053Z" }, + { url = "https://files.pythonhosted.org/packages/7a/1d/29d32e0fb40864b1f878c7f5a0b343ae676c6e2b271a2d55cc3a152391da/charset_normalizer-3.4.7-cp314-cp314t-win_amd64.whl", hash = "sha256:03853ed82eeebbce3c2abfdbc98c96dc205f32a79627688ac9a27370ea61a49c", size = 174168, upload-time = "2026-04-02T09:27:49.795Z" }, + { url = "https://files.pythonhosted.org/packages/de/32/d92444ad05c7a6e41fb2036749777c163baf7a0301a040cb672d6b2b1ae9/charset_normalizer-3.4.7-cp314-cp314t-win_arm64.whl", hash = "sha256:c35abb8bfff0185efac5878da64c45dafd2b37fb0383add1be155a763c1f083d", size = 153018, upload-time = "2026-04-02T09:27:51.116Z" }, + { url = "https://files.pythonhosted.org/packages/01/1b/ef725f8eb19b5a261b30f78efa9252ef9d017985cb499102f6f49834cd12/charset_normalizer-3.4.7-cp39-cp39-macosx_10_9_universal2.whl", hash = "sha256:177a0ba5f0211d488e295aaf82707237e331c24788d8d76c96c5a41594723217", size = 299121, upload-time = "2026-04-02T09:28:14.372Z" }, + { url = "https://files.pythonhosted.org/packages/a3/22/2f12878fbc680fbbb52386cd39a379801f62eaca74fc8b323381325f0f04/charset_normalizer-3.4.7-cp39-cp39-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:6e0d51f618228538a3e8f46bd246f87a6cd030565e015803691603f55e12afb5", size = 200612, upload-time = "2026-04-02T09:28:16.162Z" }, + { url = "https://files.pythonhosted.org/packages/bc/b6/10c84e789126ca97d4a7228863a30481e786980a8b8cfcbf4f30658ca63c/charset_normalizer-3.4.7-cp39-cp39-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:14265bfe1f09498b9d8ec91e9ec9fa52775edf90fcbde092b25f4a33d444fea9", size = 221041, upload-time = "2026-04-02T09:28:17.554Z" }, + { url = "https://files.pythonhosted.org/packages/21/7b/c414866a138400b2e81973d006da7f694cfeaf895ef07d2cba9a8743841a/charset_normalizer-3.4.7-cp39-cp39-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:87fad7d9ba98c86bcb41b2dc8dbb326619be2562af1f8ff50776a39e55721c5a", size = 216323, upload-time = "2026-04-02T09:28:18.863Z" }, + { url = "https://files.pythonhosted.org/packages/2e/92/bdcf94997e06b223d826df3abed45a5ad6e17f609b7df9d25cd23b5bde30/charset_normalizer-3.4.7-cp39-cp39-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:f22dec1690b584cea26fade98b2435c132c1b5f68e39f5a0b7627cd7ae31f1dc", size = 208419, upload-time = "2026-04-02T09:28:20.332Z" }, + { url = "https://files.pythonhosted.org/packages/1a/64/3f9142293c88b1b10e199649ed1330f070c2a68e305335a5819fa7f25fa7/charset_normalizer-3.4.7-cp39-cp39-manylinux_2_31_armv7l.whl", hash = "sha256:d61f00a0869d77422d9b2aba989e2d24afa6ffd552af442e0e58de4f35ea6d00", size = 195016, upload-time = "2026-04-02T09:28:21.657Z" }, + { url = "https://files.pythonhosted.org/packages/c1/d1/d8a6b7dd5c5636b76ce0d080bc57d8e56c7bbd6bc2ac941529a35e41d84a/charset_normalizer-3.4.7-cp39-cp39-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:6370e8686f662e6a3941ee48ed4742317cafbe5707e36406e9df792cdb535776", size = 206115, upload-time = "2026-04-02T09:28:23.259Z" }, + { url = "https://files.pythonhosted.org/packages/dd/8c/60ebe912379627d023eb96995b40bc50308729f210f43d66109ca0a7bbd2/charset_normalizer-3.4.7-cp39-cp39-musllinux_1_2_aarch64.whl", hash = "sha256:a6c5863edfbe888d9eff9c8b8087354e27618d9da76425c119293f11712a6319", size = 204022, upload-time = "2026-04-02T09:28:24.779Z" }, + { url = "https://files.pythonhosted.org/packages/d5/2a/41816ceda78a551cbfdfbeab6f3891152b0e3f758ce6580c2c18c829f774/charset_normalizer-3.4.7-cp39-cp39-musllinux_1_2_armv7l.whl", hash = "sha256:ed065083d0898c9d5b4bbec7b026fd755ff7454e6e8b73a67f8c744b13986e24", size = 195914, upload-time = "2026-04-02T09:28:26.181Z" }, + { url = "https://files.pythonhosted.org/packages/8f/9b/7c7f4b7f11525fcbdfba752455314ac60646bae91cdd671d531c1f7a97c6/charset_normalizer-3.4.7-cp39-cp39-musllinux_1_2_ppc64le.whl", hash = "sha256:2cd4a60d0e2fb04537162c62bbbb4182f53541fe0ede35cdf270a1c1e723cc42", size = 222159, upload-time = "2026-04-02T09:28:27.504Z" }, + { url = "https://files.pythonhosted.org/packages/9f/57/301682e7469bdbfa2ce219a804f0668b2266ab8520570d85d3b3ef483ea3/charset_normalizer-3.4.7-cp39-cp39-musllinux_1_2_riscv64.whl", hash = "sha256:813c0e0132266c08eb87469a642cb30aaff57c5f426255419572aaeceeaa7bf4", size = 206154, upload-time = "2026-04-02T09:28:28.848Z" }, + { url = "https://files.pythonhosted.org/packages/20/ec/90339ff5cdc598b265748c1f231c7d7fbd9123a92cee10f757e0b1448de4/charset_normalizer-3.4.7-cp39-cp39-musllinux_1_2_s390x.whl", hash = "sha256:07d9e39b01743c3717745f4c530a6349eadbfa043c7577eef86c502c15df2c67", size = 217423, upload-time = "2026-04-02T09:28:30.248Z" }, + { url = "https://files.pythonhosted.org/packages/2e/e7/a7a6147f8e3375676309cf584b25c72a3bab784ea4085b0011fa07b23aeb/charset_normalizer-3.4.7-cp39-cp39-musllinux_1_2_x86_64.whl", hash = "sha256:c0f081d69a6e58272819b70288d3221a6ee64b98df852631c80f293514d3b274", size = 210604, upload-time = "2026-04-02T09:28:31.736Z" }, + { url = "https://files.pythonhosted.org/packages/1a/62/d9340c7a79c393e57807d7fb6c57e82060687891f81b74d3201958b919c1/charset_normalizer-3.4.7-cp39-cp39-win32.whl", hash = "sha256:8751d2787c9131302398b11e6c8068053dcb55d5a8964e114b6e196cf16cb366", size = 144631, upload-time = "2026-04-02T09:28:33.158Z" }, + { url = "https://files.pythonhosted.org/packages/21/e7/92901117e2ddc8facfe8235a3ecd4eb482185b2ad5d5b6606b37c1afea06/charset_normalizer-3.4.7-cp39-cp39-win_amd64.whl", hash = "sha256:12a6fff75f6bc66711b73a2f0addfc4c8c15a20e805146a02d147a318962c444", size = 154710, upload-time = "2026-04-02T09:28:34.557Z" }, + { url = "https://files.pythonhosted.org/packages/cc/4f/e1fb138201ad9a32499dd9a98aa4a5a5441fbf7f56b52b619a54b7ee8777/charset_normalizer-3.4.7-cp39-cp39-win_arm64.whl", hash = "sha256:bb8cc7534f51d9a017b93e3e85b260924f909601c3df002bcdb58ddb4dc41a5c", size = 143716, upload-time = "2026-04-02T09:28:35.908Z" }, + { url = "https://files.pythonhosted.org/packages/db/8f/61959034484a4a7c527811f4721e75d02d653a35afb0b6054474d8185d4c/charset_normalizer-3.4.7-py3-none-any.whl", hash = "sha256:3dce51d0f5e7951f8bb4900c257dad282f49190fdbebecd4ba99bcc41fef404d", size = 61958, upload-time = "2026-04-02T09:28:37.794Z" }, +] + +[[package]] +name = "colorama" +version = "0.4.6" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/d8/53/6f443c9a4a8358a93a6792e2acffb9d9d5cb0a5cfd8802644b7b1c9a02e4/colorama-0.4.6.tar.gz", hash = "sha256:08695f5cb7ed6e0531a20572697297273c47b8cae5a63ffc6d6ed5c201be6e44", size = 27697, upload-time = "2022-10-25T02:36:22.414Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/d1/d6/3965ed04c63042e047cb6a3e6ed1a63a35087b6a609aa3a15ed8ac56c221/colorama-0.4.6-py2.py3-none-any.whl", hash = "sha256:4f1d9991f5acc0ca119f9d443620b77f9d6b33703e51011c16baf57afb285fc6", size = 25335, upload-time = "2022-10-25T02:36:20.889Z" }, +] + +[[package]] +name = "docutils" +version = "0.21.2" +source = { registry = "https://pypi.org/simple" } +resolution-markers = [ + "python_full_version == '3.10.*'", + "python_full_version < '3.10'", +] +sdist = { url = "https://files.pythonhosted.org/packages/ae/ed/aefcc8cd0ba62a0560c3c18c33925362d46c6075480bfa4df87b28e169a9/docutils-0.21.2.tar.gz", hash = "sha256:3a6b18732edf182daa3cd12775bbb338cf5691468f91eeeb109deff6ebfa986f", size = 2204444, upload-time = "2024-04-23T18:57:18.24Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/8f/d7/9322c609343d929e75e7e5e6255e614fcc67572cfd083959cdef3b7aad79/docutils-0.21.2-py3-none-any.whl", hash = "sha256:dafca5b9e384f0e419294eb4d2ff9fa826435bf15f15b7bd45723e8ad76811b2", size = 587408, upload-time = "2024-04-23T18:57:14.835Z" }, +] + +[[package]] +name = "docutils" +version = "0.22.4" +source = { registry = "https://pypi.org/simple" } +resolution-markers = [ + "python_full_version >= '3.14'", + "python_full_version == '3.13.*'", + "python_full_version == '3.12.*'", + "python_full_version == '3.11.*'", +] +sdist = { url = "https://files.pythonhosted.org/packages/ae/b6/03bb70946330e88ffec97aefd3ea75ba575cb2e762061e0e62a213befee8/docutils-0.22.4.tar.gz", hash = "sha256:4db53b1fde9abecbb74d91230d32ab626d94f6badfc575d6db9194a49df29968", size = 2291750, upload-time = "2025-12-18T19:00:26.443Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/02/10/5da547df7a391dcde17f59520a231527b8571e6f46fc8efb02ccb370ab12/docutils-0.22.4-py3-none-any.whl", hash = "sha256:d0013f540772d1420576855455d050a2180186c91c15779301ac2ccb3eeb68de", size = 633196, upload-time = "2025-12-18T19:00:18.077Z" }, +] + +[[package]] +name = "idna" +version = "3.15" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/82/77/7b3966d0b9d1d31a36ddf1746926a11dface89a83409bf1483f0237aa758/idna-3.15.tar.gz", hash = "sha256:ca962446ea538f7092a95e057da437618e886f4d349216d2b1e294abfdb65fdc", size = 199245, upload-time = "2026-05-12T22:45:57.011Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/d2/23/408243171aa9aaba178d3e2559159c24c1171a641aa83b67bdd3394ead8e/idna-3.15-py3-none-any.whl", hash = "sha256:048adeaf8c2d788c40fee287673ccaa74c24ffd8dcf09ffa555a2fbb59f10ac8", size = 72340, upload-time = "2026-05-12T22:45:55.733Z" }, +] + +[[package]] +name = "imagesize" +version = "1.5.0" +source = { registry = "https://pypi.org/simple" } +resolution-markers = [ + "python_full_version < '3.10'", +] +sdist = { url = "https://files.pythonhosted.org/packages/cf/59/4b0dd64676aa6fb4986a755790cb6fc558559cf0084effad516820208ec3/imagesize-1.5.0.tar.gz", hash = "sha256:8bfc5363a7f2133a89f0098451e0bcb1cd71aba4dc02bbcecb39d99d40e1b94f", size = 1281127, upload-time = "2026-03-03T01:59:54.651Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/1e/b1/a0662b03103c66cf77101a187f396ea91167cd9b7d5d3a2e465ad2c7ee9b/imagesize-1.5.0-py2.py3-none-any.whl", hash = "sha256:32677681b3f434c2cb496f00e89c5a291247b35b1f527589909e008057da5899", size = 5763, upload-time = "2026-03-03T01:59:52.343Z" }, +] + +[[package]] +name = "imagesize" +version = "2.0.0" +source = { registry = "https://pypi.org/simple" } +resolution-markers = [ + "python_full_version >= '3.14'", + "python_full_version == '3.13.*'", + "python_full_version == '3.12.*'", + "python_full_version == '3.11.*'", + "python_full_version == '3.10.*'", +] +sdist = { url = "https://files.pythonhosted.org/packages/6c/e6/7bf14eeb8f8b7251141944835abd42eb20a658d89084b7e1f3e5fe394090/imagesize-2.0.0.tar.gz", hash = "sha256:8e8358c4a05c304f1fccf7ff96f036e7243a189e9e42e90851993c558cfe9ee3", size = 1773045, upload-time = "2026-03-03T14:18:29.941Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/5f/53/fb7122b71361a0d121b669dcf3d31244ef75badbbb724af388948de543e2/imagesize-2.0.0-py2.py3-none-any.whl", hash = "sha256:5667c5bbb57ab3f1fa4bc366f4fbc971db3d5ed011fd2715fd8001f782718d96", size = 9441, upload-time = "2026-03-03T14:18:27.892Z" }, +] + +[[package]] +name = "importlib-metadata" +version = "8.7.1" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "zipp", marker = "python_full_version < '3.10'" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/f3/49/3b30cad09e7771a4982d9975a8cbf64f00d4a1ececb53297f1d9a7be1b10/importlib_metadata-8.7.1.tar.gz", hash = "sha256:49fef1ae6440c182052f407c8d34a68f72efc36db9ca90dc0113398f2fdde8bb", size = 57107, upload-time = "2025-12-21T10:00:19.278Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/fa/5e/f8e9a1d23b9c20a551a8a02ea3637b4642e22c2626e3a13a9a29cdea99eb/importlib_metadata-8.7.1-py3-none-any.whl", hash = "sha256:5a1f80bf1daa489495071efbb095d75a634cf28a8bc299581244063b53176151", size = 27865, upload-time = "2025-12-21T10:00:18.329Z" }, +] + +[[package]] +name = "jinja2" +version = "3.1.6" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "markupsafe" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/df/bf/f7da0350254c0ed7c72f3e33cef02e048281fec7ecec5f032d4aac52226b/jinja2-3.1.6.tar.gz", hash = "sha256:0137fb05990d35f1275a587e9aee6d56da821fc83491a0fb838183be43f66d6d", size = 245115, upload-time = "2025-03-05T20:05:02.478Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/62/a1/3d680cbfd5f4b8f15abc1d571870c5fc3e594bb582bc3b64ea099db13e56/jinja2-3.1.6-py3-none-any.whl", hash = "sha256:85ece4451f492d0c13c5dd7c13a64681a86afae63a5f347908daf103ce6d2f67", size = 134899, upload-time = "2025-03-05T20:05:00.369Z" }, +] + +[[package]] +name = "macholib" +version = "1.16.4" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "altgraph" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/10/2f/97589876ea967487978071c9042518d28b958d87b17dceb7cdc1d881f963/macholib-1.16.4.tar.gz", hash = "sha256:f408c93ab2e995cd2c46e34fe328b130404be143469e41bc366c807448979362", size = 59427, upload-time = "2025-11-22T08:28:38.373Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/c7/d1/a9f36f8ecdf0fb7c9b1e78c8d7af12b8c8754e74851ac7b94a8305540fc7/macholib-1.16.4-py2.py3-none-any.whl", hash = "sha256:da1a3fa8266e30f0ce7e97c6a54eefaae8edd1e5f86f3eb8b95457cae90265ea", size = 38117, upload-time = "2025-11-22T08:28:36.939Z" }, +] + +[[package]] +name = "markdown-it-py" +version = "3.0.0" +source = { registry = "https://pypi.org/simple" } +resolution-markers = [ + "python_full_version == '3.10.*'", + "python_full_version < '3.10'", +] +dependencies = [ + { name = "mdurl", marker = "python_full_version < '3.11'" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/38/71/3b932df36c1a044d397a1f92d1cf91ee0a503d91e470cbd670aa66b07ed0/markdown-it-py-3.0.0.tar.gz", hash = "sha256:e3f60a94fa066dc52ec76661e37c851cb232d92f9886b15cb560aaada2df8feb", size = 74596, upload-time = "2023-06-03T06:41:14.443Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/42/d7/1ec15b46af6af88f19b8e5ffea08fa375d433c998b8a7639e76935c14f1f/markdown_it_py-3.0.0-py3-none-any.whl", hash = "sha256:355216845c60bd96232cd8d8c40e8f9765cc86f46880e43a8fd22dc1a1a8cab1", size = 87528, upload-time = "2023-06-03T06:41:11.019Z" }, +] + +[[package]] +name = "markdown-it-py" +version = "4.2.0" +source = { registry = "https://pypi.org/simple" } +resolution-markers = [ + "python_full_version >= '3.14'", + "python_full_version == '3.13.*'", + "python_full_version == '3.12.*'", + "python_full_version == '3.11.*'", +] +dependencies = [ + { name = "mdurl", marker = "python_full_version >= '3.11'" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/06/ff/7841249c247aa650a76b9ee4bbaeae59370dc8bfd2f6c01f3630c35eb134/markdown_it_py-4.2.0.tar.gz", hash = "sha256:04a21681d6fbb623de53f6f364d352309d4094dd4194040a10fd51833e418d49", size = 82454, upload-time = "2026-05-07T12:08:28.36Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/b3/81/4da04ced5a082363ecfa159c010d200ecbd959ae410c10c0264a38cac0f5/markdown_it_py-4.2.0-py3-none-any.whl", hash = "sha256:9f7ebbcd14fe59494226453aed97c1070d83f8d24b6fc3a3bcf9a38092641c4a", size = 91687, upload-time = "2026-05-07T12:08:27.182Z" }, +] + +[[package]] +name = "markupsafe" +version = "3.0.3" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/7e/99/7690b6d4034fffd95959cbe0c02de8deb3098cc577c67bb6a24fe5d7caa7/markupsafe-3.0.3.tar.gz", hash = "sha256:722695808f4b6457b320fdc131280796bdceb04ab50fe1795cd540799ebe1698", size = 80313, upload-time = "2025-09-27T18:37:40.426Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/e8/4b/3541d44f3937ba468b75da9eebcae497dcf67adb65caa16760b0a6807ebb/markupsafe-3.0.3-cp310-cp310-macosx_10_9_x86_64.whl", hash = "sha256:2f981d352f04553a7171b8e44369f2af4055f888dfb147d55e42d29e29e74559", size = 11631, upload-time = "2025-09-27T18:36:05.558Z" }, + { url = "https://files.pythonhosted.org/packages/98/1b/fbd8eed11021cabd9226c37342fa6ca4e8a98d8188a8d9b66740494960e4/markupsafe-3.0.3-cp310-cp310-macosx_11_0_arm64.whl", hash = "sha256:e1c1493fb6e50ab01d20a22826e57520f1284df32f2d8601fdd90b6304601419", size = 12057, upload-time = "2025-09-27T18:36:07.165Z" }, + { url = "https://files.pythonhosted.org/packages/40/01/e560d658dc0bb8ab762670ece35281dec7b6c1b33f5fbc09ebb57a185519/markupsafe-3.0.3-cp310-cp310-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:1ba88449deb3de88bd40044603fafffb7bc2b055d626a330323a9ed736661695", size = 22050, upload-time = "2025-09-27T18:36:08.005Z" }, + { url = "https://files.pythonhosted.org/packages/af/cd/ce6e848bbf2c32314c9b237839119c5a564a59725b53157c856e90937b7a/markupsafe-3.0.3-cp310-cp310-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:f42d0984e947b8adf7dd6dde396e720934d12c506ce84eea8476409563607591", size = 20681, upload-time = "2025-09-27T18:36:08.881Z" }, + { url = "https://files.pythonhosted.org/packages/c9/2a/b5c12c809f1c3045c4d580b035a743d12fcde53cf685dbc44660826308da/markupsafe-3.0.3-cp310-cp310-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:c0c0b3ade1c0b13b936d7970b1d37a57acde9199dc2aecc4c336773e1d86049c", size = 20705, upload-time = "2025-09-27T18:36:10.131Z" }, + { url = "https://files.pythonhosted.org/packages/cf/e3/9427a68c82728d0a88c50f890d0fc072a1484de2f3ac1ad0bfc1a7214fd5/markupsafe-3.0.3-cp310-cp310-musllinux_1_2_aarch64.whl", hash = "sha256:0303439a41979d9e74d18ff5e2dd8c43ed6c6001fd40e5bf2e43f7bd9bbc523f", size = 21524, upload-time = "2025-09-27T18:36:11.324Z" }, + { url = "https://files.pythonhosted.org/packages/bc/36/23578f29e9e582a4d0278e009b38081dbe363c5e7165113fad546918a232/markupsafe-3.0.3-cp310-cp310-musllinux_1_2_riscv64.whl", hash = "sha256:d2ee202e79d8ed691ceebae8e0486bd9a2cd4794cec4824e1c99b6f5009502f6", size = 20282, upload-time = "2025-09-27T18:36:12.573Z" }, + { url = "https://files.pythonhosted.org/packages/56/21/dca11354e756ebd03e036bd8ad58d6d7168c80ce1fe5e75218e4945cbab7/markupsafe-3.0.3-cp310-cp310-musllinux_1_2_x86_64.whl", hash = "sha256:177b5253b2834fe3678cb4a5f0059808258584c559193998be2601324fdeafb1", size = 20745, upload-time = "2025-09-27T18:36:13.504Z" }, + { url = "https://files.pythonhosted.org/packages/87/99/faba9369a7ad6e4d10b6a5fbf71fa2a188fe4a593b15f0963b73859a1bbd/markupsafe-3.0.3-cp310-cp310-win32.whl", hash = "sha256:2a15a08b17dd94c53a1da0438822d70ebcd13f8c3a95abe3a9ef9f11a94830aa", size = 14571, upload-time = "2025-09-27T18:36:14.779Z" }, + { url = "https://files.pythonhosted.org/packages/d6/25/55dc3ab959917602c96985cb1253efaa4ff42f71194bddeb61eb7278b8be/markupsafe-3.0.3-cp310-cp310-win_amd64.whl", hash = "sha256:c4ffb7ebf07cfe8931028e3e4c85f0357459a3f9f9490886198848f4fa002ec8", size = 15056, upload-time = "2025-09-27T18:36:16.125Z" }, + { url = "https://files.pythonhosted.org/packages/d0/9e/0a02226640c255d1da0b8d12e24ac2aa6734da68bff14c05dd53b94a0fc3/markupsafe-3.0.3-cp310-cp310-win_arm64.whl", hash = "sha256:e2103a929dfa2fcaf9bb4e7c091983a49c9ac3b19c9061b6d5427dd7d14d81a1", size = 13932, upload-time = "2025-09-27T18:36:17.311Z" }, + { url = "https://files.pythonhosted.org/packages/08/db/fefacb2136439fc8dd20e797950e749aa1f4997ed584c62cfb8ef7c2be0e/markupsafe-3.0.3-cp311-cp311-macosx_10_9_x86_64.whl", hash = "sha256:1cc7ea17a6824959616c525620e387f6dd30fec8cb44f649e31712db02123dad", size = 11631, upload-time = "2025-09-27T18:36:18.185Z" }, + { url = "https://files.pythonhosted.org/packages/e1/2e/5898933336b61975ce9dc04decbc0a7f2fee78c30353c5efba7f2d6ff27a/markupsafe-3.0.3-cp311-cp311-macosx_11_0_arm64.whl", hash = "sha256:4bd4cd07944443f5a265608cc6aab442e4f74dff8088b0dfc8238647b8f6ae9a", size = 12058, upload-time = "2025-09-27T18:36:19.444Z" }, + { url = "https://files.pythonhosted.org/packages/1d/09/adf2df3699d87d1d8184038df46a9c80d78c0148492323f4693df54e17bb/markupsafe-3.0.3-cp311-cp311-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:6b5420a1d9450023228968e7e6a9ce57f65d148ab56d2313fcd589eee96a7a50", size = 24287, upload-time = "2025-09-27T18:36:20.768Z" }, + { url = "https://files.pythonhosted.org/packages/30/ac/0273f6fcb5f42e314c6d8cd99effae6a5354604d461b8d392b5ec9530a54/markupsafe-3.0.3-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:0bf2a864d67e76e5c9a34dc26ec616a66b9888e25e7b9460e1c76d3293bd9dbf", size = 22940, upload-time = "2025-09-27T18:36:22.249Z" }, + { url = "https://files.pythonhosted.org/packages/19/ae/31c1be199ef767124c042c6c3e904da327a2f7f0cd63a0337e1eca2967a8/markupsafe-3.0.3-cp311-cp311-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:bc51efed119bc9cfdf792cdeaa4d67e8f6fcccab66ed4bfdd6bde3e59bfcbb2f", size = 21887, upload-time = "2025-09-27T18:36:23.535Z" }, + { url = "https://files.pythonhosted.org/packages/b2/76/7edcab99d5349a4532a459e1fe64f0b0467a3365056ae550d3bcf3f79e1e/markupsafe-3.0.3-cp311-cp311-musllinux_1_2_aarch64.whl", hash = "sha256:068f375c472b3e7acbe2d5318dea141359e6900156b5b2ba06a30b169086b91a", size = 23692, upload-time = "2025-09-27T18:36:24.823Z" }, + { url = "https://files.pythonhosted.org/packages/a4/28/6e74cdd26d7514849143d69f0bf2399f929c37dc2b31e6829fd2045b2765/markupsafe-3.0.3-cp311-cp311-musllinux_1_2_riscv64.whl", hash = "sha256:7be7b61bb172e1ed687f1754f8e7484f1c8019780f6f6b0786e76bb01c2ae115", size = 21471, upload-time = "2025-09-27T18:36:25.95Z" }, + { url = "https://files.pythonhosted.org/packages/62/7e/a145f36a5c2945673e590850a6f8014318d5577ed7e5920a4b3448e0865d/markupsafe-3.0.3-cp311-cp311-musllinux_1_2_x86_64.whl", hash = "sha256:f9e130248f4462aaa8e2552d547f36ddadbeaa573879158d721bbd33dfe4743a", size = 22923, upload-time = "2025-09-27T18:36:27.109Z" }, + { url = "https://files.pythonhosted.org/packages/0f/62/d9c46a7f5c9adbeeeda52f5b8d802e1094e9717705a645efc71b0913a0a8/markupsafe-3.0.3-cp311-cp311-win32.whl", hash = "sha256:0db14f5dafddbb6d9208827849fad01f1a2609380add406671a26386cdf15a19", size = 14572, upload-time = "2025-09-27T18:36:28.045Z" }, + { url = "https://files.pythonhosted.org/packages/83/8a/4414c03d3f891739326e1783338e48fb49781cc915b2e0ee052aa490d586/markupsafe-3.0.3-cp311-cp311-win_amd64.whl", hash = "sha256:de8a88e63464af587c950061a5e6a67d3632e36df62b986892331d4620a35c01", size = 15077, upload-time = "2025-09-27T18:36:29.025Z" }, + { url = "https://files.pythonhosted.org/packages/35/73/893072b42e6862f319b5207adc9ae06070f095b358655f077f69a35601f0/markupsafe-3.0.3-cp311-cp311-win_arm64.whl", hash = "sha256:3b562dd9e9ea93f13d53989d23a7e775fdfd1066c33494ff43f5418bc8c58a5c", size = 13876, upload-time = "2025-09-27T18:36:29.954Z" }, + { url = "https://files.pythonhosted.org/packages/5a/72/147da192e38635ada20e0a2e1a51cf8823d2119ce8883f7053879c2199b5/markupsafe-3.0.3-cp312-cp312-macosx_10_13_x86_64.whl", hash = "sha256:d53197da72cc091b024dd97249dfc7794d6a56530370992a5e1a08983ad9230e", size = 11615, upload-time = "2025-09-27T18:36:30.854Z" }, + { url = "https://files.pythonhosted.org/packages/9a/81/7e4e08678a1f98521201c3079f77db69fb552acd56067661f8c2f534a718/markupsafe-3.0.3-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:1872df69a4de6aead3491198eaf13810b565bdbeec3ae2dc8780f14458ec73ce", size = 12020, upload-time = "2025-09-27T18:36:31.971Z" }, + { url = "https://files.pythonhosted.org/packages/1e/2c/799f4742efc39633a1b54a92eec4082e4f815314869865d876824c257c1e/markupsafe-3.0.3-cp312-cp312-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:3a7e8ae81ae39e62a41ec302f972ba6ae23a5c5396c8e60113e9066ef893da0d", size = 24332, upload-time = "2025-09-27T18:36:32.813Z" }, + { url = "https://files.pythonhosted.org/packages/3c/2e/8d0c2ab90a8c1d9a24f0399058ab8519a3279d1bd4289511d74e909f060e/markupsafe-3.0.3-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:d6dd0be5b5b189d31db7cda48b91d7e0a9795f31430b7f271219ab30f1d3ac9d", size = 22947, upload-time = "2025-09-27T18:36:33.86Z" }, + { url = "https://files.pythonhosted.org/packages/2c/54/887f3092a85238093a0b2154bd629c89444f395618842e8b0c41783898ea/markupsafe-3.0.3-cp312-cp312-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:94c6f0bb423f739146aec64595853541634bde58b2135f27f61c1ffd1cd4d16a", size = 21962, upload-time = "2025-09-27T18:36:35.099Z" }, + { url = "https://files.pythonhosted.org/packages/c9/2f/336b8c7b6f4a4d95e91119dc8521402461b74a485558d8f238a68312f11c/markupsafe-3.0.3-cp312-cp312-musllinux_1_2_aarch64.whl", hash = "sha256:be8813b57049a7dc738189df53d69395eba14fb99345e0a5994914a3864c8a4b", size = 23760, upload-time = "2025-09-27T18:36:36.001Z" }, + { url = "https://files.pythonhosted.org/packages/32/43/67935f2b7e4982ffb50a4d169b724d74b62a3964bc1a9a527f5ac4f1ee2b/markupsafe-3.0.3-cp312-cp312-musllinux_1_2_riscv64.whl", hash = "sha256:83891d0e9fb81a825d9a6d61e3f07550ca70a076484292a70fde82c4b807286f", size = 21529, upload-time = "2025-09-27T18:36:36.906Z" }, + { url = "https://files.pythonhosted.org/packages/89/e0/4486f11e51bbba8b0c041098859e869e304d1c261e59244baa3d295d47b7/markupsafe-3.0.3-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:77f0643abe7495da77fb436f50f8dab76dbc6e5fd25d39589a0f1fe6548bfa2b", size = 23015, upload-time = "2025-09-27T18:36:37.868Z" }, + { url = "https://files.pythonhosted.org/packages/2f/e1/78ee7a023dac597a5825441ebd17170785a9dab23de95d2c7508ade94e0e/markupsafe-3.0.3-cp312-cp312-win32.whl", hash = "sha256:d88b440e37a16e651bda4c7c2b930eb586fd15ca7406cb39e211fcff3bf3017d", size = 14540, upload-time = "2025-09-27T18:36:38.761Z" }, + { url = "https://files.pythonhosted.org/packages/aa/5b/bec5aa9bbbb2c946ca2733ef9c4ca91c91b6a24580193e891b5f7dbe8e1e/markupsafe-3.0.3-cp312-cp312-win_amd64.whl", hash = "sha256:26a5784ded40c9e318cfc2bdb30fe164bdb8665ded9cd64d500a34fb42067b1c", size = 15105, upload-time = "2025-09-27T18:36:39.701Z" }, + { url = "https://files.pythonhosted.org/packages/e5/f1/216fc1bbfd74011693a4fd837e7026152e89c4bcf3e77b6692fba9923123/markupsafe-3.0.3-cp312-cp312-win_arm64.whl", hash = "sha256:35add3b638a5d900e807944a078b51922212fb3dedb01633a8defc4b01a3c85f", size = 13906, upload-time = "2025-09-27T18:36:40.689Z" }, + { url = "https://files.pythonhosted.org/packages/38/2f/907b9c7bbba283e68f20259574b13d005c121a0fa4c175f9bed27c4597ff/markupsafe-3.0.3-cp313-cp313-macosx_10_13_x86_64.whl", hash = "sha256:e1cf1972137e83c5d4c136c43ced9ac51d0e124706ee1c8aa8532c1287fa8795", size = 11622, upload-time = "2025-09-27T18:36:41.777Z" }, + { url = "https://files.pythonhosted.org/packages/9c/d9/5f7756922cdd676869eca1c4e3c0cd0df60ed30199ffd775e319089cb3ed/markupsafe-3.0.3-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:116bb52f642a37c115f517494ea5feb03889e04df47eeff5b130b1808ce7c219", size = 12029, upload-time = "2025-09-27T18:36:43.257Z" }, + { url = "https://files.pythonhosted.org/packages/00/07/575a68c754943058c78f30db02ee03a64b3c638586fba6a6dd56830b30a3/markupsafe-3.0.3-cp313-cp313-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:133a43e73a802c5562be9bbcd03d090aa5a1fe899db609c29e8c8d815c5f6de6", size = 24374, upload-time = "2025-09-27T18:36:44.508Z" }, + { url = "https://files.pythonhosted.org/packages/a9/21/9b05698b46f218fc0e118e1f8168395c65c8a2c750ae2bab54fc4bd4e0e8/markupsafe-3.0.3-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:ccfcd093f13f0f0b7fdd0f198b90053bf7b2f02a3927a30e63f3ccc9df56b676", size = 22980, upload-time = "2025-09-27T18:36:45.385Z" }, + { url = "https://files.pythonhosted.org/packages/7f/71/544260864f893f18b6827315b988c146b559391e6e7e8f7252839b1b846a/markupsafe-3.0.3-cp313-cp313-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:509fa21c6deb7a7a273d629cf5ec029bc209d1a51178615ddf718f5918992ab9", size = 21990, upload-time = "2025-09-27T18:36:46.916Z" }, + { url = "https://files.pythonhosted.org/packages/c2/28/b50fc2f74d1ad761af2f5dcce7492648b983d00a65b8c0e0cb457c82ebbe/markupsafe-3.0.3-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:a4afe79fb3de0b7097d81da19090f4df4f8d3a2b3adaa8764138aac2e44f3af1", size = 23784, upload-time = "2025-09-27T18:36:47.884Z" }, + { url = "https://files.pythonhosted.org/packages/ed/76/104b2aa106a208da8b17a2fb72e033a5a9d7073c68f7e508b94916ed47a9/markupsafe-3.0.3-cp313-cp313-musllinux_1_2_riscv64.whl", hash = "sha256:795e7751525cae078558e679d646ae45574b47ed6e7771863fcc079a6171a0fc", size = 21588, upload-time = "2025-09-27T18:36:48.82Z" }, + { url = "https://files.pythonhosted.org/packages/b5/99/16a5eb2d140087ebd97180d95249b00a03aa87e29cc224056274f2e45fd6/markupsafe-3.0.3-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:8485f406a96febb5140bfeca44a73e3ce5116b2501ac54fe953e488fb1d03b12", size = 23041, upload-time = "2025-09-27T18:36:49.797Z" }, + { url = "https://files.pythonhosted.org/packages/19/bc/e7140ed90c5d61d77cea142eed9f9c303f4c4806f60a1044c13e3f1471d0/markupsafe-3.0.3-cp313-cp313-win32.whl", hash = "sha256:bdd37121970bfd8be76c5fb069c7751683bdf373db1ed6c010162b2a130248ed", size = 14543, upload-time = "2025-09-27T18:36:51.584Z" }, + { url = "https://files.pythonhosted.org/packages/05/73/c4abe620b841b6b791f2edc248f556900667a5a1cf023a6646967ae98335/markupsafe-3.0.3-cp313-cp313-win_amd64.whl", hash = "sha256:9a1abfdc021a164803f4d485104931fb8f8c1efd55bc6b748d2f5774e78b62c5", size = 15113, upload-time = "2025-09-27T18:36:52.537Z" }, + { url = "https://files.pythonhosted.org/packages/f0/3a/fa34a0f7cfef23cf9500d68cb7c32dd64ffd58a12b09225fb03dd37d5b80/markupsafe-3.0.3-cp313-cp313-win_arm64.whl", hash = "sha256:7e68f88e5b8799aa49c85cd116c932a1ac15caaa3f5db09087854d218359e485", size = 13911, upload-time = "2025-09-27T18:36:53.513Z" }, + { url = "https://files.pythonhosted.org/packages/e4/d7/e05cd7efe43a88a17a37b3ae96e79a19e846f3f456fe79c57ca61356ef01/markupsafe-3.0.3-cp313-cp313t-macosx_10_13_x86_64.whl", hash = "sha256:218551f6df4868a8d527e3062d0fb968682fe92054e89978594c28e642c43a73", size = 11658, upload-time = "2025-09-27T18:36:54.819Z" }, + { url = "https://files.pythonhosted.org/packages/99/9e/e412117548182ce2148bdeacdda3bb494260c0b0184360fe0d56389b523b/markupsafe-3.0.3-cp313-cp313t-macosx_11_0_arm64.whl", hash = "sha256:3524b778fe5cfb3452a09d31e7b5adefeea8c5be1d43c4f810ba09f2ceb29d37", size = 12066, upload-time = "2025-09-27T18:36:55.714Z" }, + { url = "https://files.pythonhosted.org/packages/bc/e6/fa0ffcda717ef64a5108eaa7b4f5ed28d56122c9a6d70ab8b72f9f715c80/markupsafe-3.0.3-cp313-cp313t-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:4e885a3d1efa2eadc93c894a21770e4bc67899e3543680313b09f139e149ab19", size = 25639, upload-time = "2025-09-27T18:36:56.908Z" }, + { url = "https://files.pythonhosted.org/packages/96/ec/2102e881fe9d25fc16cb4b25d5f5cde50970967ffa5dddafdb771237062d/markupsafe-3.0.3-cp313-cp313t-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:8709b08f4a89aa7586de0aadc8da56180242ee0ada3999749b183aa23df95025", size = 23569, upload-time = "2025-09-27T18:36:57.913Z" }, + { url = "https://files.pythonhosted.org/packages/4b/30/6f2fce1f1f205fc9323255b216ca8a235b15860c34b6798f810f05828e32/markupsafe-3.0.3-cp313-cp313t-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:b8512a91625c9b3da6f127803b166b629725e68af71f8184ae7e7d54686a56d6", size = 23284, upload-time = "2025-09-27T18:36:58.833Z" }, + { url = "https://files.pythonhosted.org/packages/58/47/4a0ccea4ab9f5dcb6f79c0236d954acb382202721e704223a8aafa38b5c8/markupsafe-3.0.3-cp313-cp313t-musllinux_1_2_aarch64.whl", hash = "sha256:9b79b7a16f7fedff2495d684f2b59b0457c3b493778c9eed31111be64d58279f", size = 24801, upload-time = "2025-09-27T18:36:59.739Z" }, + { url = "https://files.pythonhosted.org/packages/6a/70/3780e9b72180b6fecb83a4814d84c3bf4b4ae4bf0b19c27196104149734c/markupsafe-3.0.3-cp313-cp313t-musllinux_1_2_riscv64.whl", hash = "sha256:12c63dfb4a98206f045aa9563db46507995f7ef6d83b2f68eda65c307c6829eb", size = 22769, upload-time = "2025-09-27T18:37:00.719Z" }, + { url = "https://files.pythonhosted.org/packages/98/c5/c03c7f4125180fc215220c035beac6b9cb684bc7a067c84fc69414d315f5/markupsafe-3.0.3-cp313-cp313t-musllinux_1_2_x86_64.whl", hash = "sha256:8f71bc33915be5186016f675cd83a1e08523649b0e33efdb898db577ef5bb009", size = 23642, upload-time = "2025-09-27T18:37:01.673Z" }, + { url = "https://files.pythonhosted.org/packages/80/d6/2d1b89f6ca4bff1036499b1e29a1d02d282259f3681540e16563f27ebc23/markupsafe-3.0.3-cp313-cp313t-win32.whl", hash = "sha256:69c0b73548bc525c8cb9a251cddf1931d1db4d2258e9599c28c07ef3580ef354", size = 14612, upload-time = "2025-09-27T18:37:02.639Z" }, + { url = "https://files.pythonhosted.org/packages/2b/98/e48a4bfba0a0ffcf9925fe2d69240bfaa19c6f7507b8cd09c70684a53c1e/markupsafe-3.0.3-cp313-cp313t-win_amd64.whl", hash = "sha256:1b4b79e8ebf6b55351f0d91fe80f893b4743f104bff22e90697db1590e47a218", size = 15200, upload-time = "2025-09-27T18:37:03.582Z" }, + { url = "https://files.pythonhosted.org/packages/0e/72/e3cc540f351f316e9ed0f092757459afbc595824ca724cbc5a5d4263713f/markupsafe-3.0.3-cp313-cp313t-win_arm64.whl", hash = "sha256:ad2cf8aa28b8c020ab2fc8287b0f823d0a7d8630784c31e9ee5edea20f406287", size = 13973, upload-time = "2025-09-27T18:37:04.929Z" }, + { url = "https://files.pythonhosted.org/packages/33/8a/8e42d4838cd89b7dde187011e97fe6c3af66d8c044997d2183fbd6d31352/markupsafe-3.0.3-cp314-cp314-macosx_10_13_x86_64.whl", hash = "sha256:eaa9599de571d72e2daf60164784109f19978b327a3910d3e9de8c97b5b70cfe", size = 11619, upload-time = "2025-09-27T18:37:06.342Z" }, + { url = "https://files.pythonhosted.org/packages/b5/64/7660f8a4a8e53c924d0fa05dc3a55c9cee10bbd82b11c5afb27d44b096ce/markupsafe-3.0.3-cp314-cp314-macosx_11_0_arm64.whl", hash = "sha256:c47a551199eb8eb2121d4f0f15ae0f923d31350ab9280078d1e5f12b249e0026", size = 12029, upload-time = "2025-09-27T18:37:07.213Z" }, + { url = "https://files.pythonhosted.org/packages/da/ef/e648bfd021127bef5fa12e1720ffed0c6cbb8310c8d9bea7266337ff06de/markupsafe-3.0.3-cp314-cp314-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:f34c41761022dd093b4b6896d4810782ffbabe30f2d443ff5f083e0cbbb8c737", size = 24408, upload-time = "2025-09-27T18:37:09.572Z" }, + { url = "https://files.pythonhosted.org/packages/41/3c/a36c2450754618e62008bf7435ccb0f88053e07592e6028a34776213d877/markupsafe-3.0.3-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:457a69a9577064c05a97c41f4e65148652db078a3a509039e64d3467b9e7ef97", size = 23005, upload-time = "2025-09-27T18:37:10.58Z" }, + { url = "https://files.pythonhosted.org/packages/bc/20/b7fdf89a8456b099837cd1dc21974632a02a999ec9bf7ca3e490aacd98e7/markupsafe-3.0.3-cp314-cp314-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:e8afc3f2ccfa24215f8cb28dcf43f0113ac3c37c2f0f0806d8c70e4228c5cf4d", size = 22048, upload-time = "2025-09-27T18:37:11.547Z" }, + { url = "https://files.pythonhosted.org/packages/9a/a7/591f592afdc734f47db08a75793a55d7fbcc6902a723ae4cfbab61010cc5/markupsafe-3.0.3-cp314-cp314-musllinux_1_2_aarch64.whl", hash = "sha256:ec15a59cf5af7be74194f7ab02d0f59a62bdcf1a537677ce67a2537c9b87fcda", size = 23821, upload-time = "2025-09-27T18:37:12.48Z" }, + { url = "https://files.pythonhosted.org/packages/7d/33/45b24e4f44195b26521bc6f1a82197118f74df348556594bd2262bda1038/markupsafe-3.0.3-cp314-cp314-musllinux_1_2_riscv64.whl", hash = "sha256:0eb9ff8191e8498cca014656ae6b8d61f39da5f95b488805da4bb029cccbfbaf", size = 21606, upload-time = "2025-09-27T18:37:13.485Z" }, + { url = "https://files.pythonhosted.org/packages/ff/0e/53dfaca23a69fbfbbf17a4b64072090e70717344c52eaaaa9c5ddff1e5f0/markupsafe-3.0.3-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:2713baf880df847f2bece4230d4d094280f4e67b1e813eec43b4c0e144a34ffe", size = 23043, upload-time = "2025-09-27T18:37:14.408Z" }, + { url = "https://files.pythonhosted.org/packages/46/11/f333a06fc16236d5238bfe74daccbca41459dcd8d1fa952e8fbd5dccfb70/markupsafe-3.0.3-cp314-cp314-win32.whl", hash = "sha256:729586769a26dbceff69f7a7dbbf59ab6572b99d94576a5592625d5b411576b9", size = 14747, upload-time = "2025-09-27T18:37:15.36Z" }, + { url = "https://files.pythonhosted.org/packages/28/52/182836104b33b444e400b14f797212f720cbc9ed6ba34c800639d154e821/markupsafe-3.0.3-cp314-cp314-win_amd64.whl", hash = "sha256:bdc919ead48f234740ad807933cdf545180bfbe9342c2bb451556db2ed958581", size = 15341, upload-time = "2025-09-27T18:37:16.496Z" }, + { url = "https://files.pythonhosted.org/packages/6f/18/acf23e91bd94fd7b3031558b1f013adfa21a8e407a3fdb32745538730382/markupsafe-3.0.3-cp314-cp314-win_arm64.whl", hash = "sha256:5a7d5dc5140555cf21a6fefbdbf8723f06fcd2f63ef108f2854de715e4422cb4", size = 14073, upload-time = "2025-09-27T18:37:17.476Z" }, + { url = "https://files.pythonhosted.org/packages/3c/f0/57689aa4076e1b43b15fdfa646b04653969d50cf30c32a102762be2485da/markupsafe-3.0.3-cp314-cp314t-macosx_10_13_x86_64.whl", hash = "sha256:1353ef0c1b138e1907ae78e2f6c63ff67501122006b0f9abad68fda5f4ffc6ab", size = 11661, upload-time = "2025-09-27T18:37:18.453Z" }, + { url = "https://files.pythonhosted.org/packages/89/c3/2e67a7ca217c6912985ec766c6393b636fb0c2344443ff9d91404dc4c79f/markupsafe-3.0.3-cp314-cp314t-macosx_11_0_arm64.whl", hash = "sha256:1085e7fbddd3be5f89cc898938f42c0b3c711fdcb37d75221de2666af647c175", size = 12069, upload-time = "2025-09-27T18:37:19.332Z" }, + { url = "https://files.pythonhosted.org/packages/f0/00/be561dce4e6ca66b15276e184ce4b8aec61fe83662cce2f7d72bd3249d28/markupsafe-3.0.3-cp314-cp314t-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:1b52b4fb9df4eb9ae465f8d0c228a00624de2334f216f178a995ccdcf82c4634", size = 25670, upload-time = "2025-09-27T18:37:20.245Z" }, + { url = "https://files.pythonhosted.org/packages/50/09/c419f6f5a92e5fadde27efd190eca90f05e1261b10dbd8cbcb39cd8ea1dc/markupsafe-3.0.3-cp314-cp314t-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:fed51ac40f757d41b7c48425901843666a6677e3e8eb0abcff09e4ba6e664f50", size = 23598, upload-time = "2025-09-27T18:37:21.177Z" }, + { url = "https://files.pythonhosted.org/packages/22/44/a0681611106e0b2921b3033fc19bc53323e0b50bc70cffdd19f7d679bb66/markupsafe-3.0.3-cp314-cp314t-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:f190daf01f13c72eac4efd5c430a8de82489d9cff23c364c3ea822545032993e", size = 23261, upload-time = "2025-09-27T18:37:22.167Z" }, + { url = "https://files.pythonhosted.org/packages/5f/57/1b0b3f100259dc9fffe780cfb60d4be71375510e435efec3d116b6436d43/markupsafe-3.0.3-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:e56b7d45a839a697b5eb268c82a71bd8c7f6c94d6fd50c3d577fa39a9f1409f5", size = 24835, upload-time = "2025-09-27T18:37:23.296Z" }, + { url = "https://files.pythonhosted.org/packages/26/6a/4bf6d0c97c4920f1597cc14dd720705eca0bf7c787aebc6bb4d1bead5388/markupsafe-3.0.3-cp314-cp314t-musllinux_1_2_riscv64.whl", hash = "sha256:f3e98bb3798ead92273dc0e5fd0f31ade220f59a266ffd8a4f6065e0a3ce0523", size = 22733, upload-time = "2025-09-27T18:37:24.237Z" }, + { url = "https://files.pythonhosted.org/packages/14/c7/ca723101509b518797fedc2fdf79ba57f886b4aca8a7d31857ba3ee8281f/markupsafe-3.0.3-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:5678211cb9333a6468fb8d8be0305520aa073f50d17f089b5b4b477ea6e67fdc", size = 23672, upload-time = "2025-09-27T18:37:25.271Z" }, + { url = "https://files.pythonhosted.org/packages/fb/df/5bd7a48c256faecd1d36edc13133e51397e41b73bb77e1a69deab746ebac/markupsafe-3.0.3-cp314-cp314t-win32.whl", hash = "sha256:915c04ba3851909ce68ccc2b8e2cd691618c4dc4c4232fb7982bca3f41fd8c3d", size = 14819, upload-time = "2025-09-27T18:37:26.285Z" }, + { url = "https://files.pythonhosted.org/packages/1a/8a/0402ba61a2f16038b48b39bccca271134be00c5c9f0f623208399333c448/markupsafe-3.0.3-cp314-cp314t-win_amd64.whl", hash = "sha256:4faffd047e07c38848ce017e8725090413cd80cbc23d86e55c587bf979e579c9", size = 15426, upload-time = "2025-09-27T18:37:27.316Z" }, + { url = "https://files.pythonhosted.org/packages/70/bc/6f1c2f612465f5fa89b95bead1f44dcb607670fd42891d8fdcd5d039f4f4/markupsafe-3.0.3-cp314-cp314t-win_arm64.whl", hash = "sha256:32001d6a8fc98c8cb5c947787c5d08b0a50663d139f1305bac5885d98d9b40fa", size = 14146, upload-time = "2025-09-27T18:37:28.327Z" }, + { url = "https://files.pythonhosted.org/packages/56/23/0d8c13a44bde9154821586520840643467aee574d8ce79a17da539ee7fed/markupsafe-3.0.3-cp39-cp39-macosx_10_9_x86_64.whl", hash = "sha256:15d939a21d546304880945ca1ecb8a039db6b4dc49b2c5a400387cdae6a62e26", size = 11623, upload-time = "2025-09-27T18:37:29.296Z" }, + { url = "https://files.pythonhosted.org/packages/fd/23/07a2cb9a8045d5f3f0890a8c3bc0859d7a47bfd9a560b563899bec7b72ed/markupsafe-3.0.3-cp39-cp39-macosx_11_0_arm64.whl", hash = "sha256:f71a396b3bf33ecaa1626c255855702aca4d3d9fea5e051b41ac59a9c1c41edc", size = 12049, upload-time = "2025-09-27T18:37:30.234Z" }, + { url = "https://files.pythonhosted.org/packages/bc/e4/6be85eb81503f8e11b61c0b6369b6e077dcf0a74adbd9ebf6b349937b4e9/markupsafe-3.0.3-cp39-cp39-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:0f4b68347f8c5eab4a13419215bdfd7f8c9b19f2b25520968adfad23eb0ce60c", size = 21923, upload-time = "2025-09-27T18:37:31.177Z" }, + { url = "https://files.pythonhosted.org/packages/6f/bc/4dc914ead3fe6ddaef035341fee0fc956949bbd27335b611829292b89ee2/markupsafe-3.0.3-cp39-cp39-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:e8fc20152abba6b83724d7ff268c249fa196d8259ff481f3b1476383f8f24e42", size = 20543, upload-time = "2025-09-27T18:37:32.168Z" }, + { url = "https://files.pythonhosted.org/packages/89/6e/5fe81fbcfba4aef4093d5f856e5c774ec2057946052d18d168219b7bd9f9/markupsafe-3.0.3-cp39-cp39-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:949b8d66bc381ee8b007cd945914c721d9aba8e27f71959d750a46f7c282b20b", size = 20585, upload-time = "2025-09-27T18:37:33.166Z" }, + { url = "https://files.pythonhosted.org/packages/f6/f6/e0e5a3d3ae9c4020f696cd055f940ef86b64fe88de26f3a0308b9d3d048c/markupsafe-3.0.3-cp39-cp39-musllinux_1_2_aarch64.whl", hash = "sha256:3537e01efc9d4dccdf77221fb1cb3b8e1a38d5428920e0657ce299b20324d758", size = 21387, upload-time = "2025-09-27T18:37:34.185Z" }, + { url = "https://files.pythonhosted.org/packages/c8/25/651753ef4dea08ea790f4fbb65146a9a44a014986996ca40102e237aa49a/markupsafe-3.0.3-cp39-cp39-musllinux_1_2_riscv64.whl", hash = "sha256:591ae9f2a647529ca990bc681daebdd52c8791ff06c2bfa05b65163e28102ef2", size = 20133, upload-time = "2025-09-27T18:37:35.138Z" }, + { url = "https://files.pythonhosted.org/packages/dc/0a/c3cf2b4fef5f0426e8a6d7fce3cb966a17817c568ce59d76b92a233fdbec/markupsafe-3.0.3-cp39-cp39-musllinux_1_2_x86_64.whl", hash = "sha256:a320721ab5a1aba0a233739394eb907f8c8da5c98c9181d1161e77a0c8e36f2d", size = 20588, upload-time = "2025-09-27T18:37:36.096Z" }, + { url = "https://files.pythonhosted.org/packages/cd/1b/a7782984844bd519ad4ffdbebbba2671ec5d0ebbeac34736c15fb86399e8/markupsafe-3.0.3-cp39-cp39-win32.whl", hash = "sha256:df2449253ef108a379b8b5d6b43f4b1a8e81a061d6537becd5582fba5f9196d7", size = 14566, upload-time = "2025-09-27T18:37:37.09Z" }, + { url = "https://files.pythonhosted.org/packages/18/1f/8d9c20e1c9440e215a44be5ab64359e207fcb4f675543f1cf9a2a7f648d0/markupsafe-3.0.3-cp39-cp39-win_amd64.whl", hash = "sha256:7c3fb7d25180895632e5d3148dbdc29ea38ccb7fd210aa27acbd1201a1902c6e", size = 15053, upload-time = "2025-09-27T18:37:38.054Z" }, + { url = "https://files.pythonhosted.org/packages/4e/d3/fe08482b5cd995033556d45041a4f4e76e7f0521112a9c9991d40d39825f/markupsafe-3.0.3-cp39-cp39-win_arm64.whl", hash = "sha256:38664109c14ffc9e7437e86b4dceb442b0096dfe3541d7864d9cbe1da4cf36c8", size = 13928, upload-time = "2025-09-27T18:37:39.037Z" }, +] + +[[package]] +name = "mdit-py-plugins" +version = "0.4.2" +source = { registry = "https://pypi.org/simple" } +resolution-markers = [ + "python_full_version < '3.10'", +] +dependencies = [ + { name = "markdown-it-py", version = "3.0.0", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.10'" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/19/03/a2ecab526543b152300717cf232bb4bb8605b6edb946c845016fa9c9c9fd/mdit_py_plugins-0.4.2.tar.gz", hash = "sha256:5f2cd1fdb606ddf152d37ec30e46101a60512bc0e5fa1a7002c36647b09e26b5", size = 43542, upload-time = "2024-09-09T20:27:49.564Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/a7/f7/7782a043553ee469c1ff49cfa1cdace2d6bf99a1f333cf38676b3ddf30da/mdit_py_plugins-0.4.2-py3-none-any.whl", hash = "sha256:0c673c3f889399a33b95e88d2f0d111b4447bdfea7f237dab2d488f459835636", size = 55316, upload-time = "2024-09-09T20:27:48.397Z" }, +] + +[[package]] +name = "mdit-py-plugins" +version = "0.6.1" +source = { registry = "https://pypi.org/simple" } +resolution-markers = [ + "python_full_version >= '3.14'", + "python_full_version == '3.13.*'", + "python_full_version == '3.12.*'", + "python_full_version == '3.11.*'", + "python_full_version == '3.10.*'", +] +dependencies = [ + { name = "markdown-it-py", version = "3.0.0", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version == '3.10.*'" }, + { name = "markdown-it-py", version = "4.2.0", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version >= '3.11'" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/59/fc/f8d0863f8862f25602c0404d75568e89fb6b4109804645e5cdfb1be5cf56/mdit_py_plugins-0.6.1.tar.gz", hash = "sha256:a2bca0f039f39dbd35fb74ae1b5f998608c437463371f0ff7f49a19a17a114d0", size = 56114, upload-time = "2026-05-13T09:03:38.91Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/a5/69/6da5581c6a7fede7dc261bf4e67d6adca4196f176b43288b55b3db395b6e/mdit_py_plugins-0.6.1-py3-none-any.whl", hash = "sha256:214c82fb2ac524472ab6a5bcab1de80f73b50443e187f401bfd77efbc7c6481d", size = 66663, upload-time = "2026-05-13T09:03:37.76Z" }, +] + +[[package]] +name = "mdurl" +version = "0.1.2" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/d6/54/cfe61301667036ec958cb99bd3efefba235e65cdeb9c84d24a8293ba1d90/mdurl-0.1.2.tar.gz", hash = "sha256:bb413d29f5eea38f31dd4754dd7377d4465116fb207585f97bf925588687c1ba", size = 8729, upload-time = "2022-08-14T12:40:10.846Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/b3/38/89ba8ad64ae25be8de66a6d463314cf1eb366222074cfda9ee839c56a4b4/mdurl-0.1.2-py3-none-any.whl", hash = "sha256:84008a41e51615a49fc9966191ff91509e3c40b939176e643fd50a5c2196b8f8", size = 9979, upload-time = "2022-08-14T12:40:09.779Z" }, +] + +[[package]] +name = "myst-parser" +version = "3.0.1" +source = { registry = "https://pypi.org/simple" } +resolution-markers = [ + "python_full_version < '3.10'", +] +dependencies = [ + { name = "docutils", version = "0.21.2", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.10'" }, + { name = "jinja2", marker = "python_full_version < '3.10'" }, + { name = "markdown-it-py", version = "3.0.0", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.10'" }, + { name = "mdit-py-plugins", version = "0.4.2", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.10'" }, + { name = "pyyaml", marker = "python_full_version < '3.10'" }, + { name = "sphinx", version = "7.4.7", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.10'" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/49/64/e2f13dac02f599980798c01156393b781aec983b52a6e4057ee58f07c43a/myst_parser-3.0.1.tar.gz", hash = "sha256:88f0cb406cb363b077d176b51c476f62d60604d68a8dcdf4832e080441301a87", size = 92392, upload-time = "2024-04-28T20:22:42.116Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/e2/de/21aa8394f16add8f7427f0a1326ccd2b3a2a8a3245c9252bc5ac034c6155/myst_parser-3.0.1-py3-none-any.whl", hash = "sha256:6457aaa33a5d474aca678b8ead9b3dc298e89c68e67012e73146ea6fd54babf1", size = 83163, upload-time = "2024-04-28T20:22:39.985Z" }, +] + +[[package]] +name = "myst-parser" +version = "4.0.1" +source = { registry = "https://pypi.org/simple" } +resolution-markers = [ + "python_full_version == '3.10.*'", +] +dependencies = [ + { name = "docutils", version = "0.21.2", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version == '3.10.*'" }, + { name = "jinja2", marker = "python_full_version == '3.10.*'" }, + { name = "markdown-it-py", version = "3.0.0", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version == '3.10.*'" }, + { name = "mdit-py-plugins", version = "0.6.1", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version == '3.10.*'" }, + { name = "pyyaml", marker = "python_full_version == '3.10.*'" }, + { name = "sphinx", version = "8.1.3", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version == '3.10.*'" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/66/a5/9626ba4f73555b3735ad86247a8077d4603aa8628537687c839ab08bfe44/myst_parser-4.0.1.tar.gz", hash = "sha256:5cfea715e4f3574138aecbf7d54132296bfd72bb614d31168f48c477a830a7c4", size = 93985, upload-time = "2025-02-12T10:53:03.833Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/5f/df/76d0321c3797b54b60fef9ec3bd6f4cfd124b9e422182156a1dd418722cf/myst_parser-4.0.1-py3-none-any.whl", hash = "sha256:9134e88959ec3b5780aedf8a99680ea242869d012e8821db3126d427edc9c95d", size = 84579, upload-time = "2025-02-12T10:53:02.078Z" }, +] + +[[package]] +name = "myst-parser" +version = "5.1.0" +source = { registry = "https://pypi.org/simple" } +resolution-markers = [ + "python_full_version >= '3.14'", + "python_full_version == '3.13.*'", + "python_full_version == '3.12.*'", + "python_full_version == '3.11.*'", +] +dependencies = [ + { name = "docutils", version = "0.22.4", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version >= '3.11'" }, + { name = "jinja2", marker = "python_full_version >= '3.11'" }, + { name = "markdown-it-py", version = "4.2.0", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version >= '3.11'" }, + { name = "mdit-py-plugins", version = "0.6.1", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version >= '3.11'" }, + { name = "pyyaml", marker = "python_full_version >= '3.11'" }, + { name = "sphinx", version = "9.0.4", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version == '3.11.*'" }, + { name = "sphinx", version = "9.1.0", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version >= '3.12'" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/21/dc/603751677fff302f34396e206b610f556a59d7fe58b9a2145f54e96b48e8/myst_parser-5.1.0.tar.gz", hash = "sha256:ab69322dc6719dcc7f296479dbb70181b66df6ed315064f92dbc85c0e1bf2f02", size = 101182, upload-time = "2026-05-13T09:38:19.361Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/09/dc/f3dfb7488b770f3f67e6545085bf2abea5172e88f57b8ad25ef860ca704c/myst_parser-5.1.0-py3-none-any.whl", hash = "sha256:9c91c52b3cdb4d94a6506e4fab4e2f296c7623a0da0dcbe6de1565c3dad67a8a", size = 85817, upload-time = "2026-05-13T09:38:17.904Z" }, +] + +[[package]] +name = "packaging" +version = "26.2" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/d7/f1/e7a6dd94a8d4a5626c03e4e99c87f241ba9e350cd9e6d75123f992427270/packaging-26.2.tar.gz", hash = "sha256:ff452ff5a3e828ce110190feff1178bb1f2ea2281fa2075aadb987c2fb221661", size = 228134, upload-time = "2026-04-24T20:15:23.917Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/df/b2/87e62e8c3e2f4b32e5fe99e0b86d576da1312593b39f47d8ceef365e95ed/packaging-26.2-py3-none-any.whl", hash = "sha256:5fc45236b9446107ff2415ce77c807cee2862cb6fac22b8a73826d0693b0980e", size = 100195, upload-time = "2026-04-24T20:15:22.081Z" }, +] + +[[package]] +name = "pefile" +version = "2024.8.26" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/03/4f/2750f7f6f025a1507cd3b7218691671eecfd0bbebebe8b39aa0fe1d360b8/pefile-2024.8.26.tar.gz", hash = "sha256:3ff6c5d8b43e8c37bb6e6dd5085658d658a7a0bdcd20b6a07b1fcfc1c4e9d632", size = 76008, upload-time = "2024-08-26T20:58:38.155Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/54/16/12b82f791c7f50ddec566873d5bdd245baa1491bac11d15ffb98aecc8f8b/pefile-2024.8.26-py3-none-any.whl", hash = "sha256:76f8b485dcd3b1bb8166f1128d395fa3d87af26360c2358fb75b80019b957c6f", size = 74766, upload-time = "2024-08-26T21:01:02.632Z" }, +] + +[[package]] +name = "pyelftools" +version = "0.32" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/b9/ab/33968940b2deb3d92f5b146bc6d4009a5f95d1d06c148ea2f9ee965071af/pyelftools-0.32.tar.gz", hash = "sha256:6de90ee7b8263e740c8715a925382d4099b354f29ac48ea40d840cf7aa14ace5", size = 15047199, upload-time = "2025-02-19T14:20:05.549Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/af/43/700932c4f0638c3421177144a2e86448c0d75dbaee2c7936bda3f9fd0878/pyelftools-0.32-py3-none-any.whl", hash = "sha256:013df952a006db5e138b1edf6d8a68ecc50630adbd0d83a2d41e7f846163d738", size = 188525, upload-time = "2025-02-19T14:19:59.919Z" }, +] + +[[package]] +name = "pygments" +version = "2.20.0" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/c3/b2/bc9c9196916376152d655522fdcebac55e66de6603a76a02bca1b6414f6c/pygments-2.20.0.tar.gz", hash = "sha256:6757cd03768053ff99f3039c1a36d6c0aa0b263438fcab17520b30a303a82b5f", size = 4955991, upload-time = "2026-03-29T13:29:33.898Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/f4/7e/a72dd26f3b0f4f2bf1dd8923c85f7ceb43172af56d63c7383eb62b332364/pygments-2.20.0-py3-none-any.whl", hash = "sha256:81a9e26dd42fd28a23a2d169d86d7ac03b46e2f8b59ed4698fb4785f946d0176", size = 1231151, upload-time = "2026-03-29T13:29:30.038Z" }, +] + +[[package]] +name = "pyyaml" +version = "6.0.3" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/05/8e/961c0007c59b8dd7729d542c61a4d537767a59645b82a0b521206e1e25c2/pyyaml-6.0.3.tar.gz", hash = "sha256:d76623373421df22fb4cf8817020cbb7ef15c725b9d5e45f17e189bfc384190f", size = 130960, upload-time = "2025-09-25T21:33:16.546Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/f4/a0/39350dd17dd6d6c6507025c0e53aef67a9293a6d37d3511f23ea510d5800/pyyaml-6.0.3-cp310-cp310-macosx_10_13_x86_64.whl", hash = "sha256:214ed4befebe12df36bcc8bc2b64b396ca31be9304b8f59e25c11cf94a4c033b", size = 184227, upload-time = "2025-09-25T21:31:46.04Z" }, + { url = "https://files.pythonhosted.org/packages/05/14/52d505b5c59ce73244f59c7a50ecf47093ce4765f116cdb98286a71eeca2/pyyaml-6.0.3-cp310-cp310-macosx_11_0_arm64.whl", hash = "sha256:02ea2dfa234451bbb8772601d7b8e426c2bfa197136796224e50e35a78777956", size = 174019, upload-time = "2025-09-25T21:31:47.706Z" }, + { url = "https://files.pythonhosted.org/packages/43/f7/0e6a5ae5599c838c696adb4e6330a59f463265bfa1e116cfd1fbb0abaaae/pyyaml-6.0.3-cp310-cp310-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:b30236e45cf30d2b8e7b3e85881719e98507abed1011bf463a8fa23e9c3e98a8", size = 740646, upload-time = "2025-09-25T21:31:49.21Z" }, + { url = "https://files.pythonhosted.org/packages/2f/3a/61b9db1d28f00f8fd0ae760459a5c4bf1b941baf714e207b6eb0657d2578/pyyaml-6.0.3-cp310-cp310-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:66291b10affd76d76f54fad28e22e51719ef9ba22b29e1d7d03d6777a9174198", size = 840793, upload-time = "2025-09-25T21:31:50.735Z" }, + { url = "https://files.pythonhosted.org/packages/7a/1e/7acc4f0e74c4b3d9531e24739e0ab832a5edf40e64fbae1a9c01941cabd7/pyyaml-6.0.3-cp310-cp310-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:9c7708761fccb9397fe64bbc0395abcae8c4bf7b0eac081e12b809bf47700d0b", size = 770293, upload-time = "2025-09-25T21:31:51.828Z" }, + { url = "https://files.pythonhosted.org/packages/8b/ef/abd085f06853af0cd59fa5f913d61a8eab65d7639ff2a658d18a25d6a89d/pyyaml-6.0.3-cp310-cp310-musllinux_1_2_aarch64.whl", hash = "sha256:418cf3f2111bc80e0933b2cd8cd04f286338bb88bdc7bc8e6dd775ebde60b5e0", size = 732872, upload-time = "2025-09-25T21:31:53.282Z" }, + { url = "https://files.pythonhosted.org/packages/1f/15/2bc9c8faf6450a8b3c9fc5448ed869c599c0a74ba2669772b1f3a0040180/pyyaml-6.0.3-cp310-cp310-musllinux_1_2_x86_64.whl", hash = "sha256:5e0b74767e5f8c593e8c9b5912019159ed0533c70051e9cce3e8b6aa699fcd69", size = 758828, upload-time = "2025-09-25T21:31:54.807Z" }, + { url = "https://files.pythonhosted.org/packages/a3/00/531e92e88c00f4333ce359e50c19b8d1de9fe8d581b1534e35ccfbc5f393/pyyaml-6.0.3-cp310-cp310-win32.whl", hash = "sha256:28c8d926f98f432f88adc23edf2e6d4921ac26fb084b028c733d01868d19007e", size = 142415, upload-time = "2025-09-25T21:31:55.885Z" }, + { url = "https://files.pythonhosted.org/packages/2a/fa/926c003379b19fca39dd4634818b00dec6c62d87faf628d1394e137354d4/pyyaml-6.0.3-cp310-cp310-win_amd64.whl", hash = "sha256:bdb2c67c6c1390b63c6ff89f210c8fd09d9a1217a465701eac7316313c915e4c", size = 158561, upload-time = "2025-09-25T21:31:57.406Z" }, + { url = "https://files.pythonhosted.org/packages/6d/16/a95b6757765b7b031c9374925bb718d55e0a9ba8a1b6a12d25962ea44347/pyyaml-6.0.3-cp311-cp311-macosx_10_13_x86_64.whl", hash = "sha256:44edc647873928551a01e7a563d7452ccdebee747728c1080d881d68af7b997e", size = 185826, upload-time = "2025-09-25T21:31:58.655Z" }, + { url = "https://files.pythonhosted.org/packages/16/19/13de8e4377ed53079ee996e1ab0a9c33ec2faf808a4647b7b4c0d46dd239/pyyaml-6.0.3-cp311-cp311-macosx_11_0_arm64.whl", hash = "sha256:652cb6edd41e718550aad172851962662ff2681490a8a711af6a4d288dd96824", size = 175577, upload-time = "2025-09-25T21:32:00.088Z" }, + { url = "https://files.pythonhosted.org/packages/0c/62/d2eb46264d4b157dae1275b573017abec435397aa59cbcdab6fc978a8af4/pyyaml-6.0.3-cp311-cp311-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:10892704fc220243f5305762e276552a0395f7beb4dbf9b14ec8fd43b57f126c", size = 775556, upload-time = "2025-09-25T21:32:01.31Z" }, + { url = "https://files.pythonhosted.org/packages/10/cb/16c3f2cf3266edd25aaa00d6c4350381c8b012ed6f5276675b9eba8d9ff4/pyyaml-6.0.3-cp311-cp311-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:850774a7879607d3a6f50d36d04f00ee69e7fc816450e5f7e58d7f17f1ae5c00", size = 882114, upload-time = "2025-09-25T21:32:03.376Z" }, + { url = "https://files.pythonhosted.org/packages/71/60/917329f640924b18ff085ab889a11c763e0b573da888e8404ff486657602/pyyaml-6.0.3-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:b8bb0864c5a28024fac8a632c443c87c5aa6f215c0b126c449ae1a150412f31d", size = 806638, upload-time = "2025-09-25T21:32:04.553Z" }, + { url = "https://files.pythonhosted.org/packages/dd/6f/529b0f316a9fd167281a6c3826b5583e6192dba792dd55e3203d3f8e655a/pyyaml-6.0.3-cp311-cp311-musllinux_1_2_aarch64.whl", hash = "sha256:1d37d57ad971609cf3c53ba6a7e365e40660e3be0e5175fa9f2365a379d6095a", size = 767463, upload-time = "2025-09-25T21:32:06.152Z" }, + { url = "https://files.pythonhosted.org/packages/f2/6a/b627b4e0c1dd03718543519ffb2f1deea4a1e6d42fbab8021936a4d22589/pyyaml-6.0.3-cp311-cp311-musllinux_1_2_x86_64.whl", hash = "sha256:37503bfbfc9d2c40b344d06b2199cf0e96e97957ab1c1b546fd4f87e53e5d3e4", size = 794986, upload-time = "2025-09-25T21:32:07.367Z" }, + { url = "https://files.pythonhosted.org/packages/45/91/47a6e1c42d9ee337c4839208f30d9f09caa9f720ec7582917b264defc875/pyyaml-6.0.3-cp311-cp311-win32.whl", hash = "sha256:8098f252adfa6c80ab48096053f512f2321f0b998f98150cea9bd23d83e1467b", size = 142543, upload-time = "2025-09-25T21:32:08.95Z" }, + { url = "https://files.pythonhosted.org/packages/da/e3/ea007450a105ae919a72393cb06f122f288ef60bba2dc64b26e2646fa315/pyyaml-6.0.3-cp311-cp311-win_amd64.whl", hash = "sha256:9f3bfb4965eb874431221a3ff3fdcddc7e74e3b07799e0e84ca4a0f867d449bf", size = 158763, upload-time = "2025-09-25T21:32:09.96Z" }, + { url = "https://files.pythonhosted.org/packages/d1/33/422b98d2195232ca1826284a76852ad5a86fe23e31b009c9886b2d0fb8b2/pyyaml-6.0.3-cp312-cp312-macosx_10_13_x86_64.whl", hash = "sha256:7f047e29dcae44602496db43be01ad42fc6f1cc0d8cd6c83d342306c32270196", size = 182063, upload-time = "2025-09-25T21:32:11.445Z" }, + { url = "https://files.pythonhosted.org/packages/89/a0/6cf41a19a1f2f3feab0e9c0b74134aa2ce6849093d5517a0c550fe37a648/pyyaml-6.0.3-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:fc09d0aa354569bc501d4e787133afc08552722d3ab34836a80547331bb5d4a0", size = 173973, upload-time = "2025-09-25T21:32:12.492Z" }, + { url = "https://files.pythonhosted.org/packages/ed/23/7a778b6bd0b9a8039df8b1b1d80e2e2ad78aa04171592c8a5c43a56a6af4/pyyaml-6.0.3-cp312-cp312-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:9149cad251584d5fb4981be1ecde53a1ca46c891a79788c0df828d2f166bda28", size = 775116, upload-time = "2025-09-25T21:32:13.652Z" }, + { url = "https://files.pythonhosted.org/packages/65/30/d7353c338e12baef4ecc1b09e877c1970bd3382789c159b4f89d6a70dc09/pyyaml-6.0.3-cp312-cp312-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:5fdec68f91a0c6739b380c83b951e2c72ac0197ace422360e6d5a959d8d97b2c", size = 844011, upload-time = "2025-09-25T21:32:15.21Z" }, + { url = "https://files.pythonhosted.org/packages/8b/9d/b3589d3877982d4f2329302ef98a8026e7f4443c765c46cfecc8858c6b4b/pyyaml-6.0.3-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:ba1cc08a7ccde2d2ec775841541641e4548226580ab850948cbfda66a1befcdc", size = 807870, upload-time = "2025-09-25T21:32:16.431Z" }, + { url = "https://files.pythonhosted.org/packages/05/c0/b3be26a015601b822b97d9149ff8cb5ead58c66f981e04fedf4e762f4bd4/pyyaml-6.0.3-cp312-cp312-musllinux_1_2_aarch64.whl", hash = "sha256:8dc52c23056b9ddd46818a57b78404882310fb473d63f17b07d5c40421e47f8e", size = 761089, upload-time = "2025-09-25T21:32:17.56Z" }, + { url = "https://files.pythonhosted.org/packages/be/8e/98435a21d1d4b46590d5459a22d88128103f8da4c2d4cb8f14f2a96504e1/pyyaml-6.0.3-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:41715c910c881bc081f1e8872880d3c650acf13dfa8214bad49ed4cede7c34ea", size = 790181, upload-time = "2025-09-25T21:32:18.834Z" }, + { url = "https://files.pythonhosted.org/packages/74/93/7baea19427dcfbe1e5a372d81473250b379f04b1bd3c4c5ff825e2327202/pyyaml-6.0.3-cp312-cp312-win32.whl", hash = "sha256:96b533f0e99f6579b3d4d4995707cf36df9100d67e0c8303a0c55b27b5f99bc5", size = 137658, upload-time = "2025-09-25T21:32:20.209Z" }, + { url = "https://files.pythonhosted.org/packages/86/bf/899e81e4cce32febab4fb42bb97dcdf66bc135272882d1987881a4b519e9/pyyaml-6.0.3-cp312-cp312-win_amd64.whl", hash = "sha256:5fcd34e47f6e0b794d17de1b4ff496c00986e1c83f7ab2fb8fcfe9616ff7477b", size = 154003, upload-time = "2025-09-25T21:32:21.167Z" }, + { url = "https://files.pythonhosted.org/packages/1a/08/67bd04656199bbb51dbed1439b7f27601dfb576fb864099c7ef0c3e55531/pyyaml-6.0.3-cp312-cp312-win_arm64.whl", hash = "sha256:64386e5e707d03a7e172c0701abfb7e10f0fb753ee1d773128192742712a98fd", size = 140344, upload-time = "2025-09-25T21:32:22.617Z" }, + { url = "https://files.pythonhosted.org/packages/d1/11/0fd08f8192109f7169db964b5707a2f1e8b745d4e239b784a5a1dd80d1db/pyyaml-6.0.3-cp313-cp313-macosx_10_13_x86_64.whl", hash = "sha256:8da9669d359f02c0b91ccc01cac4a67f16afec0dac22c2ad09f46bee0697eba8", size = 181669, upload-time = "2025-09-25T21:32:23.673Z" }, + { url = "https://files.pythonhosted.org/packages/b1/16/95309993f1d3748cd644e02e38b75d50cbc0d9561d21f390a76242ce073f/pyyaml-6.0.3-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:2283a07e2c21a2aa78d9c4442724ec1eb15f5e42a723b99cb3d822d48f5f7ad1", size = 173252, upload-time = "2025-09-25T21:32:25.149Z" }, + { url = "https://files.pythonhosted.org/packages/50/31/b20f376d3f810b9b2371e72ef5adb33879b25edb7a6d072cb7ca0c486398/pyyaml-6.0.3-cp313-cp313-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:ee2922902c45ae8ccada2c5b501ab86c36525b883eff4255313a253a3160861c", size = 767081, upload-time = "2025-09-25T21:32:26.575Z" }, + { url = "https://files.pythonhosted.org/packages/49/1e/a55ca81e949270d5d4432fbbd19dfea5321eda7c41a849d443dc92fd1ff7/pyyaml-6.0.3-cp313-cp313-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:a33284e20b78bd4a18c8c2282d549d10bc8408a2a7ff57653c0cf0b9be0afce5", size = 841159, upload-time = "2025-09-25T21:32:27.727Z" }, + { url = "https://files.pythonhosted.org/packages/74/27/e5b8f34d02d9995b80abcef563ea1f8b56d20134d8f4e5e81733b1feceb2/pyyaml-6.0.3-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:0f29edc409a6392443abf94b9cf89ce99889a1dd5376d94316ae5145dfedd5d6", size = 801626, upload-time = "2025-09-25T21:32:28.878Z" }, + { url = "https://files.pythonhosted.org/packages/f9/11/ba845c23988798f40e52ba45f34849aa8a1f2d4af4b798588010792ebad6/pyyaml-6.0.3-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:f7057c9a337546edc7973c0d3ba84ddcdf0daa14533c2065749c9075001090e6", size = 753613, upload-time = "2025-09-25T21:32:30.178Z" }, + { url = "https://files.pythonhosted.org/packages/3d/e0/7966e1a7bfc0a45bf0a7fb6b98ea03fc9b8d84fa7f2229e9659680b69ee3/pyyaml-6.0.3-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:eda16858a3cab07b80edaf74336ece1f986ba330fdb8ee0d6c0d68fe82bc96be", size = 794115, upload-time = "2025-09-25T21:32:31.353Z" }, + { url = "https://files.pythonhosted.org/packages/de/94/980b50a6531b3019e45ddeada0626d45fa85cbe22300844a7983285bed3b/pyyaml-6.0.3-cp313-cp313-win32.whl", hash = "sha256:d0eae10f8159e8fdad514efdc92d74fd8d682c933a6dd088030f3834bc8e6b26", size = 137427, upload-time = "2025-09-25T21:32:32.58Z" }, + { url = "https://files.pythonhosted.org/packages/97/c9/39d5b874e8b28845e4ec2202b5da735d0199dbe5b8fb85f91398814a9a46/pyyaml-6.0.3-cp313-cp313-win_amd64.whl", hash = "sha256:79005a0d97d5ddabfeeea4cf676af11e647e41d81c9a7722a193022accdb6b7c", size = 154090, upload-time = "2025-09-25T21:32:33.659Z" }, + { url = "https://files.pythonhosted.org/packages/73/e8/2bdf3ca2090f68bb3d75b44da7bbc71843b19c9f2b9cb9b0f4ab7a5a4329/pyyaml-6.0.3-cp313-cp313-win_arm64.whl", hash = "sha256:5498cd1645aa724a7c71c8f378eb29ebe23da2fc0d7a08071d89469bf1d2defb", size = 140246, upload-time = "2025-09-25T21:32:34.663Z" }, + { url = "https://files.pythonhosted.org/packages/9d/8c/f4bd7f6465179953d3ac9bc44ac1a8a3e6122cf8ada906b4f96c60172d43/pyyaml-6.0.3-cp314-cp314-macosx_10_13_x86_64.whl", hash = "sha256:8d1fab6bb153a416f9aeb4b8763bc0f22a5586065f86f7664fc23339fc1c1fac", size = 181814, upload-time = "2025-09-25T21:32:35.712Z" }, + { url = "https://files.pythonhosted.org/packages/bd/9c/4d95bb87eb2063d20db7b60faa3840c1b18025517ae857371c4dd55a6b3a/pyyaml-6.0.3-cp314-cp314-macosx_11_0_arm64.whl", hash = "sha256:34d5fcd24b8445fadc33f9cf348c1047101756fd760b4dacb5c3e99755703310", size = 173809, upload-time = "2025-09-25T21:32:36.789Z" }, + { url = "https://files.pythonhosted.org/packages/92/b5/47e807c2623074914e29dabd16cbbdd4bf5e9b2db9f8090fa64411fc5382/pyyaml-6.0.3-cp314-cp314-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:501a031947e3a9025ed4405a168e6ef5ae3126c59f90ce0cd6f2bfc477be31b7", size = 766454, upload-time = "2025-09-25T21:32:37.966Z" }, + { url = "https://files.pythonhosted.org/packages/02/9e/e5e9b168be58564121efb3de6859c452fccde0ab093d8438905899a3a483/pyyaml-6.0.3-cp314-cp314-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:b3bc83488de33889877a0f2543ade9f70c67d66d9ebb4ac959502e12de895788", size = 836355, upload-time = "2025-09-25T21:32:39.178Z" }, + { url = "https://files.pythonhosted.org/packages/88/f9/16491d7ed2a919954993e48aa941b200f38040928474c9e85ea9e64222c3/pyyaml-6.0.3-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:c458b6d084f9b935061bc36216e8a69a7e293a2f1e68bf956dcd9e6cbcd143f5", size = 794175, upload-time = "2025-09-25T21:32:40.865Z" }, + { url = "https://files.pythonhosted.org/packages/dd/3f/5989debef34dc6397317802b527dbbafb2b4760878a53d4166579111411e/pyyaml-6.0.3-cp314-cp314-musllinux_1_2_aarch64.whl", hash = "sha256:7c6610def4f163542a622a73fb39f534f8c101d690126992300bf3207eab9764", size = 755228, upload-time = "2025-09-25T21:32:42.084Z" }, + { url = "https://files.pythonhosted.org/packages/d7/ce/af88a49043cd2e265be63d083fc75b27b6ed062f5f9fd6cdc223ad62f03e/pyyaml-6.0.3-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:5190d403f121660ce8d1d2c1bb2ef1bd05b5f68533fc5c2ea899bd15f4399b35", size = 789194, upload-time = "2025-09-25T21:32:43.362Z" }, + { url = "https://files.pythonhosted.org/packages/23/20/bb6982b26a40bb43951265ba29d4c246ef0ff59c9fdcdf0ed04e0687de4d/pyyaml-6.0.3-cp314-cp314-win_amd64.whl", hash = "sha256:4a2e8cebe2ff6ab7d1050ecd59c25d4c8bd7e6f400f5f82b96557ac0abafd0ac", size = 156429, upload-time = "2025-09-25T21:32:57.844Z" }, + { url = "https://files.pythonhosted.org/packages/f4/f4/a4541072bb9422c8a883ab55255f918fa378ecf083f5b85e87fc2b4eda1b/pyyaml-6.0.3-cp314-cp314-win_arm64.whl", hash = "sha256:93dda82c9c22deb0a405ea4dc5f2d0cda384168e466364dec6255b293923b2f3", size = 143912, upload-time = "2025-09-25T21:32:59.247Z" }, + { url = "https://files.pythonhosted.org/packages/7c/f9/07dd09ae774e4616edf6cda684ee78f97777bdd15847253637a6f052a62f/pyyaml-6.0.3-cp314-cp314t-macosx_10_13_x86_64.whl", hash = "sha256:02893d100e99e03eda1c8fd5c441d8c60103fd175728e23e431db1b589cf5ab3", size = 189108, upload-time = "2025-09-25T21:32:44.377Z" }, + { url = "https://files.pythonhosted.org/packages/4e/78/8d08c9fb7ce09ad8c38ad533c1191cf27f7ae1effe5bb9400a46d9437fcf/pyyaml-6.0.3-cp314-cp314t-macosx_11_0_arm64.whl", hash = "sha256:c1ff362665ae507275af2853520967820d9124984e0f7466736aea23d8611fba", size = 183641, upload-time = "2025-09-25T21:32:45.407Z" }, + { url = "https://files.pythonhosted.org/packages/7b/5b/3babb19104a46945cf816d047db2788bcaf8c94527a805610b0289a01c6b/pyyaml-6.0.3-cp314-cp314t-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:6adc77889b628398debc7b65c073bcb99c4a0237b248cacaf3fe8a557563ef6c", size = 831901, upload-time = "2025-09-25T21:32:48.83Z" }, + { url = "https://files.pythonhosted.org/packages/8b/cc/dff0684d8dc44da4d22a13f35f073d558c268780ce3c6ba1b87055bb0b87/pyyaml-6.0.3-cp314-cp314t-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:a80cb027f6b349846a3bf6d73b5e95e782175e52f22108cfa17876aaeff93702", size = 861132, upload-time = "2025-09-25T21:32:50.149Z" }, + { url = "https://files.pythonhosted.org/packages/b1/5e/f77dc6b9036943e285ba76b49e118d9ea929885becb0a29ba8a7c75e29fe/pyyaml-6.0.3-cp314-cp314t-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:00c4bdeba853cc34e7dd471f16b4114f4162dc03e6b7afcc2128711f0eca823c", size = 839261, upload-time = "2025-09-25T21:32:51.808Z" }, + { url = "https://files.pythonhosted.org/packages/ce/88/a9db1376aa2a228197c58b37302f284b5617f56a5d959fd1763fb1675ce6/pyyaml-6.0.3-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:66e1674c3ef6f541c35191caae2d429b967b99e02040f5ba928632d9a7f0f065", size = 805272, upload-time = "2025-09-25T21:32:52.941Z" }, + { url = "https://files.pythonhosted.org/packages/da/92/1446574745d74df0c92e6aa4a7b0b3130706a4142b2d1a5869f2eaa423c6/pyyaml-6.0.3-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:16249ee61e95f858e83976573de0f5b2893b3677ba71c9dd36b9cf8be9ac6d65", size = 829923, upload-time = "2025-09-25T21:32:54.537Z" }, + { url = "https://files.pythonhosted.org/packages/f0/7a/1c7270340330e575b92f397352af856a8c06f230aa3e76f86b39d01b416a/pyyaml-6.0.3-cp314-cp314t-win_amd64.whl", hash = "sha256:4ad1906908f2f5ae4e5a8ddfce73c320c2a1429ec52eafd27138b7f1cbe341c9", size = 174062, upload-time = "2025-09-25T21:32:55.767Z" }, + { url = "https://files.pythonhosted.org/packages/f1/12/de94a39c2ef588c7e6455cfbe7343d3b2dc9d6b6b2f40c4c6565744c873d/pyyaml-6.0.3-cp314-cp314t-win_arm64.whl", hash = "sha256:ebc55a14a21cb14062aa4162f906cd962b28e2e9ea38f9b4391244cd8de4ae0b", size = 149341, upload-time = "2025-09-25T21:32:56.828Z" }, + { url = "https://files.pythonhosted.org/packages/9f/62/67fc8e68a75f738c9200422bf65693fb79a4cd0dc5b23310e5202e978090/pyyaml-6.0.3-cp39-cp39-macosx_10_13_x86_64.whl", hash = "sha256:b865addae83924361678b652338317d1bd7e79b1f4596f96b96c77a5a34b34da", size = 184450, upload-time = "2025-09-25T21:33:00.618Z" }, + { url = "https://files.pythonhosted.org/packages/ae/92/861f152ce87c452b11b9d0977952259aa7df792d71c1053365cc7b09cc08/pyyaml-6.0.3-cp39-cp39-macosx_11_0_arm64.whl", hash = "sha256:c3355370a2c156cffb25e876646f149d5d68f5e0a3ce86a5084dd0b64a994917", size = 174319, upload-time = "2025-09-25T21:33:02.086Z" }, + { url = "https://files.pythonhosted.org/packages/d0/cd/f0cfc8c74f8a030017a2b9c771b7f47e5dd702c3e28e5b2071374bda2948/pyyaml-6.0.3-cp39-cp39-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:3c5677e12444c15717b902a5798264fa7909e41153cdf9ef7ad571b704a63dd9", size = 737631, upload-time = "2025-09-25T21:33:03.25Z" }, + { url = "https://files.pythonhosted.org/packages/ef/b2/18f2bd28cd2055a79a46c9b0895c0b3d987ce40ee471cecf58a1a0199805/pyyaml-6.0.3-cp39-cp39-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:5ed875a24292240029e4483f9d4a4b8a1ae08843b9c54f43fcc11e404532a8a5", size = 836795, upload-time = "2025-09-25T21:33:05.014Z" }, + { url = "https://files.pythonhosted.org/packages/73/b9/793686b2d54b531203c160ef12bec60228a0109c79bae6c1277961026770/pyyaml-6.0.3-cp39-cp39-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:0150219816b6a1fa26fb4699fb7daa9caf09eb1999f3b70fb6e786805e80375a", size = 750767, upload-time = "2025-09-25T21:33:06.398Z" }, + { url = "https://files.pythonhosted.org/packages/a9/86/a137b39a611def2ed78b0e66ce2fe13ee701a07c07aebe55c340ed2a050e/pyyaml-6.0.3-cp39-cp39-musllinux_1_2_aarch64.whl", hash = "sha256:fa160448684b4e94d80416c0fa4aac48967a969efe22931448d853ada8baf926", size = 727982, upload-time = "2025-09-25T21:33:08.708Z" }, + { url = "https://files.pythonhosted.org/packages/dd/62/71c27c94f457cf4418ef8ccc71735324c549f7e3ea9d34aba50874563561/pyyaml-6.0.3-cp39-cp39-musllinux_1_2_x86_64.whl", hash = "sha256:27c0abcb4a5dac13684a37f76e701e054692a9b2d3064b70f5e4eb54810553d7", size = 755677, upload-time = "2025-09-25T21:33:09.876Z" }, + { url = "https://files.pythonhosted.org/packages/29/3d/6f5e0d58bd924fb0d06c3a6bad00effbdae2de5adb5cda5648006ffbd8d3/pyyaml-6.0.3-cp39-cp39-win32.whl", hash = "sha256:1ebe39cb5fc479422b83de611d14e2c0d3bb2a18bbcb01f229ab3cfbd8fee7a0", size = 142592, upload-time = "2025-09-25T21:33:10.983Z" }, + { url = "https://files.pythonhosted.org/packages/f0/0c/25113e0b5e103d7f1490c0e947e303fe4a696c10b501dea7a9f49d4e876c/pyyaml-6.0.3-cp39-cp39-win_amd64.whl", hash = "sha256:2e71d11abed7344e42a8849600193d15b6def118602c4c176f748e4583246007", size = 158777, upload-time = "2025-09-25T21:33:15.55Z" }, +] + +[[package]] +name = "readthedocs-sphinx-ext" +version = "2.2.5" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "jinja2" }, + { name = "packaging" }, + { name = "requests", version = "2.32.5", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.10'" }, + { name = "requests", version = "2.34.2", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version >= '3.10'" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/e8/ce/38130d8dec600bf5413eb89a3413dd38f204c7c728c4947e12ff8cb793b7/readthedocs-sphinx-ext-2.2.5.tar.gz", hash = "sha256:ee5fd5b99db9f0c180b2396cbce528aa36671951b9526bb0272dbfce5517bd27", size = 12303, upload-time = "2023-12-19T10:00:49.573Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/64/71/c89e7709a0d4f93af1848e9855112299a820b470d84f917b4dd5998bdd07/readthedocs_sphinx_ext-2.2.5-py2.py3-none-any.whl", hash = "sha256:f8c56184ea011c972dd45a90122568587cc85b0127bc9cf064d17c68bc809daa", size = 11332, upload-time = "2023-12-19T10:00:43.972Z" }, +] + +[[package]] +name = "requests" +version = "2.32.5" +source = { registry = "https://pypi.org/simple" } +resolution-markers = [ + "python_full_version < '3.10'", +] +dependencies = [ + { name = "certifi", marker = "python_full_version < '3.10'" }, + { name = "charset-normalizer", marker = "python_full_version < '3.10'" }, + { name = "idna", marker = "python_full_version < '3.10'" }, + { name = "urllib3", version = "2.6.3", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.10'" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/c9/74/b3ff8e6c8446842c3f5c837e9c3dfcfe2018ea6ecef224c710c85ef728f4/requests-2.32.5.tar.gz", hash = "sha256:dbba0bac56e100853db0ea71b82b4dfd5fe2bf6d3754a8893c3af500cec7d7cf", size = 134517, upload-time = "2025-08-18T20:46:02.573Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/1e/db/4254e3eabe8020b458f1a747140d32277ec7a271daf1d235b70dc0b4e6e3/requests-2.32.5-py3-none-any.whl", hash = "sha256:2462f94637a34fd532264295e186976db0f5d453d1cdd31473c85a6a161affb6", size = 64738, upload-time = "2025-08-18T20:46:00.542Z" }, +] + +[[package]] +name = "requests" +version = "2.34.2" +source = { registry = "https://pypi.org/simple" } +resolution-markers = [ + "python_full_version >= '3.14'", + "python_full_version == '3.13.*'", + "python_full_version == '3.12.*'", + "python_full_version == '3.11.*'", + "python_full_version == '3.10.*'", +] +dependencies = [ + { name = "certifi", marker = "python_full_version >= '3.10'" }, + { name = "charset-normalizer", marker = "python_full_version >= '3.10'" }, + { name = "idna", marker = "python_full_version >= '3.10'" }, + { name = "urllib3", version = "2.7.0", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version >= '3.10'" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/ac/c3/e2a2b89f2d3e2179abd6d00ebd70bff6273f37fb3e0cc209f48b39d00cbf/requests-2.34.2.tar.gz", hash = "sha256:f288924cae4e29463698d6d60bc6a4da69c89185ad1e0bcc4104f584e960b9ed", size = 142856, upload-time = "2026-05-14T19:25:27.735Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/a0/f4/c67b0b3f1b9245e8d266f0f112c500d50e5b4e83cb6f3b71b6528104182a/requests-2.34.2-py3-none-any.whl", hash = "sha256:2a0d60c172f83ac6ab31e4554906c0f3b3588d37b5cb939b1c061f4907e278e0", size = 73075, upload-time = "2026-05-14T19:25:26.443Z" }, +] + +[[package]] +name = "roman-numerals" +version = "4.1.0" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/ae/f9/41dc953bbeb056c17d5f7a519f50fdf010bd0553be2d630bc69d1e022703/roman_numerals-4.1.0.tar.gz", hash = "sha256:1af8b147eb1405d5839e78aeb93131690495fe9da5c91856cb33ad55a7f1e5b2", size = 9077, upload-time = "2025-12-17T18:25:34.381Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/04/54/6f679c435d28e0a568d8e8a7c0a93a09010818634c3c3907fc98d8983770/roman_numerals-4.1.0-py3-none-any.whl", hash = "sha256:647ba99caddc2cc1e55a51e4360689115551bf4476d90e8162cf8c345fe233c7", size = 7676, upload-time = "2025-12-17T18:25:33.098Z" }, +] + +[[package]] +name = "rules-python-docs" +version = "0.0.0" +source = { virtual = "." } +dependencies = [ + { name = "absl-py", version = "2.3.1", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.10'" }, + { name = "absl-py", version = "2.4.0", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version >= '3.10'" }, + { name = "macholib" }, + { name = "markupsafe" }, + { name = "myst-parser", version = "3.0.1", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.10'" }, + { name = "myst-parser", version = "4.0.1", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version == '3.10.*'" }, + { name = "myst-parser", version = "5.1.0", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version >= '3.11'" }, + { name = "pefile" }, + { name = "pyelftools" }, + { name = "readthedocs-sphinx-ext" }, + { name = "sphinx", version = "7.4.7", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.10'" }, + { name = "sphinx", version = "8.1.3", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version == '3.10.*'" }, + { name = "sphinx", version = "9.0.4", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version == '3.11.*'" }, + { name = "sphinx", version = "9.1.0", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version >= '3.12'" }, + { name = "sphinx-autodoc2" }, + { name = "sphinx-reredirects", version = "0.1.6", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.11'" }, + { name = "sphinx-reredirects", version = "1.1.0", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version >= '3.11'" }, + { name = "sphinx-rtd-theme" }, + { name = "typing-extensions" }, +] + +[package.metadata] +requires-dist = [ + { name = "absl-py" }, + { name = "macholib" }, + { name = "markupsafe" }, + { name = "myst-parser" }, + { name = "pefile" }, + { name = "pyelftools" }, + { name = "readthedocs-sphinx-ext" }, + { name = "sphinx" }, + { name = "sphinx-autodoc2" }, + { name = "sphinx-reredirects" }, + { name = "sphinx-rtd-theme", specifier = ">=2.0" }, + { name = "typing-extensions" }, +] + +[[package]] +name = "snowballstemmer" +version = "3.0.1" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/75/a7/9810d872919697c9d01295633f5d574fb416d47e535f258272ca1f01f447/snowballstemmer-3.0.1.tar.gz", hash = "sha256:6d5eeeec8e9f84d4d56b847692bacf79bc2c8e90c7f80ca4444ff8b6f2e52895", size = 105575, upload-time = "2025-05-09T16:34:51.843Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/c8/78/3565d011c61f5a43488987ee32b6f3f656e7f107ac2782dd57bdd7d91d9a/snowballstemmer-3.0.1-py3-none-any.whl", hash = "sha256:6cd7b3897da8d6c9ffb968a6781fa6532dce9c3618a4b127d920dab764a19064", size = 103274, upload-time = "2025-05-09T16:34:50.371Z" }, +] + +[[package]] +name = "sphinx" +version = "7.4.7" +source = { registry = "https://pypi.org/simple" } +resolution-markers = [ + "python_full_version < '3.10'", +] +dependencies = [ + { name = "alabaster", version = "0.7.16", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.10'" }, + { name = "babel", marker = "python_full_version < '3.10'" }, + { name = "colorama", marker = "python_full_version < '3.10' and sys_platform == 'win32'" }, + { name = "docutils", version = "0.21.2", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.10'" }, + { name = "imagesize", version = "1.5.0", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.10'" }, + { name = "importlib-metadata", marker = "python_full_version < '3.10'" }, + { name = "jinja2", marker = "python_full_version < '3.10'" }, + { name = "packaging", marker = "python_full_version < '3.10'" }, + { name = "pygments", marker = "python_full_version < '3.10'" }, + { name = "requests", version = "2.32.5", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.10'" }, + { name = "snowballstemmer", marker = "python_full_version < '3.10'" }, + { name = "sphinxcontrib-applehelp", marker = "python_full_version < '3.10'" }, + { name = "sphinxcontrib-devhelp", marker = "python_full_version < '3.10'" }, + { name = "sphinxcontrib-htmlhelp", marker = "python_full_version < '3.10'" }, + { name = "sphinxcontrib-jsmath", marker = "python_full_version < '3.10'" }, + { name = "sphinxcontrib-qthelp", marker = "python_full_version < '3.10'" }, + { name = "sphinxcontrib-serializinghtml", marker = "python_full_version < '3.10'" }, + { name = "tomli", marker = "python_full_version < '3.10'" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/5b/be/50e50cb4f2eff47df05673d361095cafd95521d2a22521b920c67a372dcb/sphinx-7.4.7.tar.gz", hash = "sha256:242f92a7ea7e6c5b406fdc2615413890ba9f699114a9c09192d7dfead2ee9cfe", size = 8067911, upload-time = "2024-07-20T14:46:56.059Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/0d/ef/153f6803c5d5f8917dbb7f7fcf6d34a871ede3296fa89c2c703f5f8a6c8e/sphinx-7.4.7-py3-none-any.whl", hash = "sha256:c2419e2135d11f1951cd994d6eb18a1835bd8fdd8429f9ca375dc1f3281bd239", size = 3401624, upload-time = "2024-07-20T14:46:52.142Z" }, +] + +[[package]] +name = "sphinx" +version = "8.1.3" +source = { registry = "https://pypi.org/simple" } +resolution-markers = [ + "python_full_version == '3.10.*'", +] +dependencies = [ + { name = "alabaster", version = "1.0.0", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version == '3.10.*'" }, + { name = "babel", marker = "python_full_version == '3.10.*'" }, + { name = "colorama", marker = "python_full_version == '3.10.*' and sys_platform == 'win32'" }, + { name = "docutils", version = "0.21.2", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version == '3.10.*'" }, + { name = "imagesize", version = "2.0.0", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version == '3.10.*'" }, + { name = "jinja2", marker = "python_full_version == '3.10.*'" }, + { name = "packaging", marker = "python_full_version == '3.10.*'" }, + { name = "pygments", marker = "python_full_version == '3.10.*'" }, + { name = "requests", version = "2.34.2", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version == '3.10.*'" }, + { name = "snowballstemmer", marker = "python_full_version == '3.10.*'" }, + { name = "sphinxcontrib-applehelp", marker = "python_full_version == '3.10.*'" }, + { name = "sphinxcontrib-devhelp", marker = "python_full_version == '3.10.*'" }, + { name = "sphinxcontrib-htmlhelp", marker = "python_full_version == '3.10.*'" }, + { name = "sphinxcontrib-jsmath", marker = "python_full_version == '3.10.*'" }, + { name = "sphinxcontrib-qthelp", marker = "python_full_version == '3.10.*'" }, + { name = "sphinxcontrib-serializinghtml", marker = "python_full_version == '3.10.*'" }, + { name = "tomli", marker = "python_full_version == '3.10.*'" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/6f/6d/be0b61178fe2cdcb67e2a92fc9ebb488e3c51c4f74a36a7824c0adf23425/sphinx-8.1.3.tar.gz", hash = "sha256:43c1911eecb0d3e161ad78611bc905d1ad0e523e4ddc202a58a821773dc4c927", size = 8184611, upload-time = "2024-10-13T20:27:13.93Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/26/60/1ddff83a56d33aaf6f10ec8ce84b4c007d9368b21008876fceda7e7381ef/sphinx-8.1.3-py3-none-any.whl", hash = "sha256:09719015511837b76bf6e03e42eb7595ac8c2e41eeb9c29c5b755c6b677992a2", size = 3487125, upload-time = "2024-10-13T20:27:10.448Z" }, +] + +[[package]] +name = "sphinx" +version = "9.0.4" +source = { registry = "https://pypi.org/simple" } +resolution-markers = [ + "python_full_version == '3.11.*'", +] +dependencies = [ + { name = "alabaster", version = "1.0.0", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version == '3.11.*'" }, + { name = "babel", marker = "python_full_version == '3.11.*'" }, + { name = "colorama", marker = "python_full_version == '3.11.*' and sys_platform == 'win32'" }, + { name = "docutils", version = "0.22.4", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version == '3.11.*'" }, + { name = "imagesize", version = "2.0.0", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version == '3.11.*'" }, + { name = "jinja2", marker = "python_full_version == '3.11.*'" }, + { name = "packaging", marker = "python_full_version == '3.11.*'" }, + { name = "pygments", marker = "python_full_version == '3.11.*'" }, + { name = "requests", version = "2.34.2", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version == '3.11.*'" }, + { name = "roman-numerals", marker = "python_full_version == '3.11.*'" }, + { name = "snowballstemmer", marker = "python_full_version == '3.11.*'" }, + { name = "sphinxcontrib-applehelp", marker = "python_full_version == '3.11.*'" }, + { name = "sphinxcontrib-devhelp", marker = "python_full_version == '3.11.*'" }, + { name = "sphinxcontrib-htmlhelp", marker = "python_full_version == '3.11.*'" }, + { name = "sphinxcontrib-jsmath", marker = "python_full_version == '3.11.*'" }, + { name = "sphinxcontrib-qthelp", marker = "python_full_version == '3.11.*'" }, + { name = "sphinxcontrib-serializinghtml", marker = "python_full_version == '3.11.*'" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/42/50/a8c6ccc36d5eacdfd7913ddccd15a9cee03ecafc5ee2bc40e1f168d85022/sphinx-9.0.4.tar.gz", hash = "sha256:594ef59d042972abbc581d8baa577404abe4e6c3b04ef61bd7fc2acbd51f3fa3", size = 8710502, upload-time = "2025-12-04T07:45:27.343Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/c6/3f/4bbd76424c393caead2e1eb89777f575dee5c8653e2d4b6afd7a564f5974/sphinx-9.0.4-py3-none-any.whl", hash = "sha256:5bebc595a5e943ea248b99c13814c1c5e10b3ece718976824ffa7959ff95fffb", size = 3917713, upload-time = "2025-12-04T07:45:24.944Z" }, +] + +[[package]] +name = "sphinx" +version = "9.1.0" +source = { registry = "https://pypi.org/simple" } +resolution-markers = [ + "python_full_version >= '3.14'", + "python_full_version == '3.13.*'", + "python_full_version == '3.12.*'", +] +dependencies = [ + { name = "alabaster", version = "1.0.0", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version >= '3.12'" }, + { name = "babel", marker = "python_full_version >= '3.12'" }, + { name = "colorama", marker = "python_full_version >= '3.12' and sys_platform == 'win32'" }, + { name = "docutils", version = "0.22.4", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version >= '3.12'" }, + { name = "imagesize", version = "2.0.0", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version >= '3.12'" }, + { name = "jinja2", marker = "python_full_version >= '3.12'" }, + { name = "packaging", marker = "python_full_version >= '3.12'" }, + { name = "pygments", marker = "python_full_version >= '3.12'" }, + { name = "requests", version = "2.34.2", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version >= '3.12'" }, + { name = "roman-numerals", marker = "python_full_version >= '3.12'" }, + { name = "snowballstemmer", marker = "python_full_version >= '3.12'" }, + { name = "sphinxcontrib-applehelp", marker = "python_full_version >= '3.12'" }, + { name = "sphinxcontrib-devhelp", marker = "python_full_version >= '3.12'" }, + { name = "sphinxcontrib-htmlhelp", marker = "python_full_version >= '3.12'" }, + { name = "sphinxcontrib-jsmath", marker = "python_full_version >= '3.12'" }, + { name = "sphinxcontrib-qthelp", marker = "python_full_version >= '3.12'" }, + { name = "sphinxcontrib-serializinghtml", marker = "python_full_version >= '3.12'" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/cd/bd/f08eb0f4eed5c83f1ba2a3bd18f7745a2b1525fad70660a1c00224ec468a/sphinx-9.1.0.tar.gz", hash = "sha256:7741722357dd75f8190766926071fed3bdc211c74dd2d7d4df5404da95930ddb", size = 8718324, upload-time = "2025-12-31T15:09:27.646Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/73/f7/b1884cb3188ab181fc81fa00c266699dab600f927a964df02ec3d5d1916a/sphinx-9.1.0-py3-none-any.whl", hash = "sha256:c84fdd4e782504495fe4f2c0b3413d6c2bf388589bb352d439b2a3bb99991978", size = 3921742, upload-time = "2025-12-31T15:09:25.561Z" }, +] + +[[package]] +name = "sphinx-autodoc2" +version = "0.5.0" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "astroid" }, + { name = "tomli", marker = "python_full_version < '3.11'" }, + { name = "typing-extensions" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/17/5f/5350046d1aa1a56b063ae08b9ad871025335c9d55fe2372896ea48711da9/sphinx_autodoc2-0.5.0.tar.gz", hash = "sha256:7d76044aa81d6af74447080182b6868c7eb066874edc835e8ddf810735b6565a", size = 115077, upload-time = "2023-11-27T07:27:51.407Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/19/e6/48d47961bbdae755ba9c17dfc65d89356312c67668dcb36c87cfadfa1964/sphinx_autodoc2-0.5.0-py3-none-any.whl", hash = "sha256:e867013b1512f9d6d7e6f6799f8b537d6884462acd118ef361f3f619a60b5c9e", size = 43385, upload-time = "2023-11-27T07:27:49.929Z" }, +] + +[[package]] +name = "sphinx-reredirects" +version = "0.1.6" +source = { registry = "https://pypi.org/simple" } +resolution-markers = [ + "python_full_version == '3.10.*'", + "python_full_version < '3.10'", +] +dependencies = [ + { name = "sphinx", version = "7.4.7", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.10'" }, + { name = "sphinx", version = "8.1.3", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version == '3.10.*'" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/16/6b/bcca2785de4071f604a722444d4d7ba8a9d40de3c14ad52fce93e6d92694/sphinx_reredirects-0.1.6.tar.gz", hash = "sha256:c491cba545f67be9697508727818d8626626366245ae64456fe29f37e9bbea64", size = 7080, upload-time = "2025-03-22T10:52:30.271Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/ac/6f/0b3625be30a1a50f9e4c2cb2ec147b08f15ed0e9f8444efcf274b751300b/sphinx_reredirects-0.1.6-py3-none-any.whl", hash = "sha256:efd50c766fbc5bf40cd5148e10c00f2c00d143027de5c5e48beece93cc40eeea", size = 5675, upload-time = "2025-03-22T10:52:29.113Z" }, +] + +[[package]] +name = "sphinx-reredirects" +version = "1.1.0" +source = { registry = "https://pypi.org/simple" } +resolution-markers = [ + "python_full_version >= '3.14'", + "python_full_version == '3.13.*'", + "python_full_version == '3.12.*'", + "python_full_version == '3.11.*'", +] +dependencies = [ + { name = "sphinx", version = "9.0.4", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version == '3.11.*'" }, + { name = "sphinx", version = "9.1.0", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version >= '3.12'" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/1b/8d/0e39fe2740d7d71417edf9a6424aa80ca2c27c17fc21282cdc39f90d5a40/sphinx_reredirects-1.1.0.tar.gz", hash = "sha256:fb9b195335ab14b43f8273287d0c7eeb637ba6c56c66581c11b47202f6718b29", size = 614624, upload-time = "2025-12-22T08:28:02.792Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/51/81/b5dd07067f3daac6d23687ec737b2d593740671ebcd145830c8f92d381c5/sphinx_reredirects-1.1.0-py3-none-any.whl", hash = "sha256:4b5692273c72cd2d4d917f4c6f87d5919e4d6114a752d4be033f7f5f6310efd9", size = 6351, upload-time = "2025-12-22T08:27:59.724Z" }, +] + +[[package]] +name = "sphinx-rtd-theme" +version = "3.1.0" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "docutils", version = "0.21.2", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.11'" }, + { name = "docutils", version = "0.22.4", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version >= '3.11'" }, + { name = "sphinx", version = "7.4.7", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.10'" }, + { name = "sphinx", version = "8.1.3", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version == '3.10.*'" }, + { name = "sphinx", version = "9.0.4", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version == '3.11.*'" }, + { name = "sphinx", version = "9.1.0", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version >= '3.12'" }, + { name = "sphinxcontrib-jquery" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/84/68/a1bfbf38c0f7bccc9b10bbf76b94606f64acb1552ae394f0b8285bfaea25/sphinx_rtd_theme-3.1.0.tar.gz", hash = "sha256:b44276f2c276e909239a4f6c955aa667aaafeb78597923b1c60babc76db78e4c", size = 7620915, upload-time = "2026-01-12T16:03:31.17Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/87/c7/b5c8015d823bfda1a346adb2c634a2101d50bb75d421eb6dcb31acd25ebc/sphinx_rtd_theme-3.1.0-py2.py3-none-any.whl", hash = "sha256:1785824ae8e6632060490f67cf3a72d404a85d2d9fc26bce3619944de5682b89", size = 7655617, upload-time = "2026-01-12T16:03:28.101Z" }, +] + +[[package]] +name = "sphinxcontrib-applehelp" +version = "2.0.0" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/ba/6e/b837e84a1a704953c62ef8776d45c3e8d759876b4a84fe14eba2859106fe/sphinxcontrib_applehelp-2.0.0.tar.gz", hash = "sha256:2f29ef331735ce958efa4734873f084941970894c6090408b079c61b2e1c06d1", size = 20053, upload-time = "2024-07-29T01:09:00.465Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/5d/85/9ebeae2f76e9e77b952f4b274c27238156eae7979c5421fba91a28f4970d/sphinxcontrib_applehelp-2.0.0-py3-none-any.whl", hash = "sha256:4cd3f0ec4ac5dd9c17ec65e9ab272c9b867ea77425228e68ecf08d6b28ddbdb5", size = 119300, upload-time = "2024-07-29T01:08:58.99Z" }, +] + +[[package]] +name = "sphinxcontrib-devhelp" +version = "2.0.0" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/f6/d2/5beee64d3e4e747f316bae86b55943f51e82bb86ecd325883ef65741e7da/sphinxcontrib_devhelp-2.0.0.tar.gz", hash = "sha256:411f5d96d445d1d73bb5d52133377b4248ec79db5c793ce7dbe59e074b4dd1ad", size = 12967, upload-time = "2024-07-29T01:09:23.417Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/35/7a/987e583882f985fe4d7323774889ec58049171828b58c2217e7f79cdf44e/sphinxcontrib_devhelp-2.0.0-py3-none-any.whl", hash = "sha256:aefb8b83854e4b0998877524d1029fd3e6879210422ee3780459e28a1f03a8a2", size = 82530, upload-time = "2024-07-29T01:09:21.945Z" }, +] + +[[package]] +name = "sphinxcontrib-htmlhelp" +version = "2.1.0" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/43/93/983afd9aa001e5201eab16b5a444ed5b9b0a7a010541e0ddfbbfd0b2470c/sphinxcontrib_htmlhelp-2.1.0.tar.gz", hash = "sha256:c9e2916ace8aad64cc13a0d233ee22317f2b9025b9cf3295249fa985cc7082e9", size = 22617, upload-time = "2024-07-29T01:09:37.889Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/0a/7b/18a8c0bcec9182c05a0b3ec2a776bba4ead82750a55ff798e8d406dae604/sphinxcontrib_htmlhelp-2.1.0-py3-none-any.whl", hash = "sha256:166759820b47002d22914d64a075ce08f4c46818e17cfc9470a9786b759b19f8", size = 98705, upload-time = "2024-07-29T01:09:36.407Z" }, +] + +[[package]] +name = "sphinxcontrib-jquery" +version = "4.1" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "sphinx", version = "7.4.7", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.10'" }, + { name = "sphinx", version = "8.1.3", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version == '3.10.*'" }, + { name = "sphinx", version = "9.0.4", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version == '3.11.*'" }, + { name = "sphinx", version = "9.1.0", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version >= '3.12'" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/de/f3/aa67467e051df70a6330fe7770894b3e4f09436dea6881ae0b4f3d87cad8/sphinxcontrib-jquery-4.1.tar.gz", hash = "sha256:1620739f04e36a2c779f1a131a2dfd49b2fd07351bf1968ced074365933abc7a", size = 122331, upload-time = "2023-03-14T15:01:01.944Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/76/85/749bd22d1a68db7291c89e2ebca53f4306c3f205853cf31e9de279034c3c/sphinxcontrib_jquery-4.1-py2.py3-none-any.whl", hash = "sha256:f936030d7d0147dd026a4f2b5a57343d233f1fc7b363f68b3d4f1cb0993878ae", size = 121104, upload-time = "2023-03-14T15:01:00.356Z" }, +] + +[[package]] +name = "sphinxcontrib-jsmath" +version = "1.0.1" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/b2/e8/9ed3830aeed71f17c026a07a5097edcf44b692850ef215b161b8ad875729/sphinxcontrib-jsmath-1.0.1.tar.gz", hash = "sha256:a9925e4a4587247ed2191a22df5f6970656cb8ca2bd6284309578f2153e0c4b8", size = 5787, upload-time = "2019-01-21T16:10:16.347Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/c2/42/4c8646762ee83602e3fb3fbe774c2fac12f317deb0b5dbeeedd2d3ba4b77/sphinxcontrib_jsmath-1.0.1-py2.py3-none-any.whl", hash = "sha256:2ec2eaebfb78f3f2078e73666b1415417a116cc848b72e5172e596c871103178", size = 5071, upload-time = "2019-01-21T16:10:14.333Z" }, +] + +[[package]] +name = "sphinxcontrib-qthelp" +version = "2.0.0" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/68/bc/9104308fc285eb3e0b31b67688235db556cd5b0ef31d96f30e45f2e51cae/sphinxcontrib_qthelp-2.0.0.tar.gz", hash = "sha256:4fe7d0ac8fc171045be623aba3e2a8f613f8682731f9153bb2e40ece16b9bbab", size = 17165, upload-time = "2024-07-29T01:09:56.435Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/27/83/859ecdd180cacc13b1f7e857abf8582a64552ea7a061057a6c716e790fce/sphinxcontrib_qthelp-2.0.0-py3-none-any.whl", hash = "sha256:b18a828cdba941ccd6ee8445dbe72ffa3ef8cbe7505d8cd1fa0d42d3f2d5f3eb", size = 88743, upload-time = "2024-07-29T01:09:54.885Z" }, +] + +[[package]] +name = "sphinxcontrib-serializinghtml" +version = "2.0.0" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/3b/44/6716b257b0aa6bfd51a1b31665d1c205fb12cb5ad56de752dfa15657de2f/sphinxcontrib_serializinghtml-2.0.0.tar.gz", hash = "sha256:e9d912827f872c029017a53f0ef2180b327c3f7fd23c87229f7a8e8b70031d4d", size = 16080, upload-time = "2024-07-29T01:10:09.332Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/52/a7/d2782e4e3f77c8450f727ba74a8f12756d5ba823d81b941f1b04da9d033a/sphinxcontrib_serializinghtml-2.0.0-py3-none-any.whl", hash = "sha256:6e2cb0eef194e10c27ec0023bfeb25badbbb5868244cf5bc5bdc04e4464bf331", size = 92072, upload-time = "2024-07-29T01:10:08.203Z" }, +] + +[[package]] +name = "tomli" +version = "2.4.1" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/22/de/48c59722572767841493b26183a0d1cc411d54fd759c5607c4590b6563a6/tomli-2.4.1.tar.gz", hash = "sha256:7c7e1a961a0b2f2472c1ac5b69affa0ae1132c39adcb67aba98568702b9cc23f", size = 17543, upload-time = "2026-03-25T20:22:03.828Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/f4/11/db3d5885d8528263d8adc260bb2d28ebf1270b96e98f0e0268d32b8d9900/tomli-2.4.1-cp311-cp311-macosx_10_9_x86_64.whl", hash = "sha256:f8f0fc26ec2cc2b965b7a3b87cd19c5c6b8c5e5f436b984e85f486d652285c30", size = 154704, upload-time = "2026-03-25T20:21:10.473Z" }, + { url = "https://files.pythonhosted.org/packages/6d/f7/675db52c7e46064a9aa928885a9b20f4124ecb9bc2e1ce74c9106648d202/tomli-2.4.1-cp311-cp311-macosx_11_0_arm64.whl", hash = "sha256:4ab97e64ccda8756376892c53a72bd1f964e519c77236368527f758fbc36a53a", size = 149454, upload-time = "2026-03-25T20:21:12.036Z" }, + { url = "https://files.pythonhosted.org/packages/61/71/81c50943cf953efa35bce7646caab3cf457a7d8c030b27cfb40d7235f9ee/tomli-2.4.1-cp311-cp311-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:96481a5786729fd470164b47cdb3e0e58062a496f455ee41b4403be77cb5a076", size = 237561, upload-time = "2026-03-25T20:21:13.098Z" }, + { url = "https://files.pythonhosted.org/packages/48/c1/f41d9cb618acccca7df82aaf682f9b49013c9397212cb9f53219e3abac37/tomli-2.4.1-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:5a881ab208c0baf688221f8cecc5401bd291d67e38a1ac884d6736cbcd8247e9", size = 243824, upload-time = "2026-03-25T20:21:14.569Z" }, + { url = "https://files.pythonhosted.org/packages/22/e4/5a816ecdd1f8ca51fb756ef684b90f2780afc52fc67f987e3c61d800a46d/tomli-2.4.1-cp311-cp311-musllinux_1_2_aarch64.whl", hash = "sha256:47149d5bd38761ac8be13a84864bf0b7b70bc051806bc3669ab1cbc56216b23c", size = 242227, upload-time = "2026-03-25T20:21:15.712Z" }, + { url = "https://files.pythonhosted.org/packages/6b/49/2b2a0ef529aa6eec245d25f0c703e020a73955ad7edf73e7f54ddc608aa5/tomli-2.4.1-cp311-cp311-musllinux_1_2_x86_64.whl", hash = "sha256:ec9bfaf3ad2df51ace80688143a6a4ebc09a248f6ff781a9945e51937008fcbc", size = 247859, upload-time = "2026-03-25T20:21:17.001Z" }, + { url = "https://files.pythonhosted.org/packages/83/bd/6c1a630eaca337e1e78c5903104f831bda934c426f9231429396ce3c3467/tomli-2.4.1-cp311-cp311-win32.whl", hash = "sha256:ff2983983d34813c1aeb0fa89091e76c3a22889ee83ab27c5eeb45100560c049", size = 97204, upload-time = "2026-03-25T20:21:18.079Z" }, + { url = "https://files.pythonhosted.org/packages/42/59/71461df1a885647e10b6bb7802d0b8e66480c61f3f43079e0dcd315b3954/tomli-2.4.1-cp311-cp311-win_amd64.whl", hash = "sha256:5ee18d9ebdb417e384b58fe414e8d6af9f4e7a0ae761519fb50f721de398dd4e", size = 108084, upload-time = "2026-03-25T20:21:18.978Z" }, + { url = "https://files.pythonhosted.org/packages/b8/83/dceca96142499c069475b790e7913b1044c1a4337e700751f48ed723f883/tomli-2.4.1-cp311-cp311-win_arm64.whl", hash = "sha256:c2541745709bad0264b7d4705ad453b76ccd191e64aa6f0fc66b69a293a45ece", size = 95285, upload-time = "2026-03-25T20:21:20.309Z" }, + { url = "https://files.pythonhosted.org/packages/c1/ba/42f134a3fe2b370f555f44b1d72feebb94debcab01676bf918d0cb70e9aa/tomli-2.4.1-cp312-cp312-macosx_10_13_x86_64.whl", hash = "sha256:c742f741d58a28940ce01d58f0ab2ea3ced8b12402f162f4d534dfe18ba1cd6a", size = 155924, upload-time = "2026-03-25T20:21:21.626Z" }, + { url = "https://files.pythonhosted.org/packages/dc/c7/62d7a17c26487ade21c5422b646110f2162f1fcc95980ef7f63e73c68f14/tomli-2.4.1-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:7f86fd587c4ed9dd76f318225e7d9b29cfc5a9d43de44e5754db8d1128487085", size = 150018, upload-time = "2026-03-25T20:21:23.002Z" }, + { url = "https://files.pythonhosted.org/packages/5c/05/79d13d7c15f13bdef410bdd49a6485b1c37d28968314eabee452c22a7fda/tomli-2.4.1-cp312-cp312-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:ff18e6a727ee0ab0388507b89d1bc6a22b138d1e2fa56d1ad494586d61d2eae9", size = 244948, upload-time = "2026-03-25T20:21:24.04Z" }, + { url = "https://files.pythonhosted.org/packages/10/90/d62ce007a1c80d0b2c93e02cab211224756240884751b94ca72df8a875ca/tomli-2.4.1-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:136443dbd7e1dee43c68ac2694fde36b2849865fa258d39bf822c10e8068eac5", size = 253341, upload-time = "2026-03-25T20:21:25.177Z" }, + { url = "https://files.pythonhosted.org/packages/1a/7e/caf6496d60152ad4ed09282c1885cca4eea150bfd007da84aea07bcc0a3e/tomli-2.4.1-cp312-cp312-musllinux_1_2_aarch64.whl", hash = "sha256:5e262d41726bc187e69af7825504c933b6794dc3fbd5945e41a79bb14c31f585", size = 248159, upload-time = "2026-03-25T20:21:26.364Z" }, + { url = "https://files.pythonhosted.org/packages/99/e7/c6f69c3120de34bbd882c6fba7975f3d7a746e9218e56ab46a1bc4b42552/tomli-2.4.1-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:5cb41aa38891e073ee49d55fbc7839cfdb2bc0e600add13874d048c94aadddd1", size = 253290, upload-time = "2026-03-25T20:21:27.46Z" }, + { url = "https://files.pythonhosted.org/packages/d6/2f/4a3c322f22c5c66c4b836ec58211641a4067364f5dcdd7b974b4c5da300c/tomli-2.4.1-cp312-cp312-win32.whl", hash = "sha256:da25dc3563bff5965356133435b757a795a17b17d01dbc0f42fb32447ddfd917", size = 98141, upload-time = "2026-03-25T20:21:28.492Z" }, + { url = "https://files.pythonhosted.org/packages/24/22/4daacd05391b92c55759d55eaee21e1dfaea86ce5c571f10083360adf534/tomli-2.4.1-cp312-cp312-win_amd64.whl", hash = "sha256:52c8ef851d9a240f11a88c003eacb03c31fc1c9c4ec64a99a0f922b93874fda9", size = 108847, upload-time = "2026-03-25T20:21:29.386Z" }, + { url = "https://files.pythonhosted.org/packages/68/fd/70e768887666ddd9e9f5d85129e84910f2db2796f9096aa02b721a53098d/tomli-2.4.1-cp312-cp312-win_arm64.whl", hash = "sha256:f758f1b9299d059cc3f6546ae2af89670cb1c4d48ea29c3cacc4fe7de3058257", size = 95088, upload-time = "2026-03-25T20:21:30.677Z" }, + { url = "https://files.pythonhosted.org/packages/07/06/b823a7e818c756d9a7123ba2cda7d07bc2dd32835648d1a7b7b7a05d848d/tomli-2.4.1-cp313-cp313-macosx_10_13_x86_64.whl", hash = "sha256:36d2bd2ad5fb9eaddba5226aa02c8ec3fa4f192631e347b3ed28186d43be6b54", size = 155866, upload-time = "2026-03-25T20:21:31.65Z" }, + { url = "https://files.pythonhosted.org/packages/14/6f/12645cf7f08e1a20c7eb8c297c6f11d31c1b50f316a7e7e1e1de6e2e7b7e/tomli-2.4.1-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:eb0dc4e38e6a1fd579e5d50369aa2e10acfc9cace504579b2faabb478e76941a", size = 149887, upload-time = "2026-03-25T20:21:33.028Z" }, + { url = "https://files.pythonhosted.org/packages/5c/e0/90637574e5e7212c09099c67ad349b04ec4d6020324539297b634a0192b0/tomli-2.4.1-cp313-cp313-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:c7f2c7f2b9ca6bdeef8f0fa897f8e05085923eb091721675170254cbc5b02897", size = 243704, upload-time = "2026-03-25T20:21:34.51Z" }, + { url = "https://files.pythonhosted.org/packages/10/8f/d3ddb16c5a4befdf31a23307f72828686ab2096f068eaf56631e136c1fdd/tomli-2.4.1-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:f3c6818a1a86dd6dca7ddcaaf76947d5ba31aecc28cb1b67009a5877c9a64f3f", size = 251628, upload-time = "2026-03-25T20:21:36.012Z" }, + { url = "https://files.pythonhosted.org/packages/e3/f1/dbeeb9116715abee2485bf0a12d07a8f31af94d71608c171c45f64c0469d/tomli-2.4.1-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:d312ef37c91508b0ab2cee7da26ec0b3ed2f03ce12bd87a588d771ae15dcf82d", size = 247180, upload-time = "2026-03-25T20:21:37.136Z" }, + { url = "https://files.pythonhosted.org/packages/d3/74/16336ffd19ed4da28a70959f92f506233bd7cfc2332b20bdb01591e8b1d1/tomli-2.4.1-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:51529d40e3ca50046d7606fa99ce3956a617f9b36380da3b7f0dd3dd28e68cb5", size = 251674, upload-time = "2026-03-25T20:21:38.298Z" }, + { url = "https://files.pythonhosted.org/packages/16/f9/229fa3434c590ddf6c0aa9af64d3af4b752540686cace29e6281e3458469/tomli-2.4.1-cp313-cp313-win32.whl", hash = "sha256:2190f2e9dd7508d2a90ded5ed369255980a1bcdd58e52f7fe24b8162bf9fedbd", size = 97976, upload-time = "2026-03-25T20:21:39.316Z" }, + { url = "https://files.pythonhosted.org/packages/6a/1e/71dfd96bcc1c775420cb8befe7a9d35f2e5b1309798f009dca17b7708c1e/tomli-2.4.1-cp313-cp313-win_amd64.whl", hash = "sha256:8d65a2fbf9d2f8352685bc1364177ee3923d6baf5e7f43ea4959d7d8bc326a36", size = 108755, upload-time = "2026-03-25T20:21:40.248Z" }, + { url = "https://files.pythonhosted.org/packages/83/7a/d34f422a021d62420b78f5c538e5b102f62bea616d1d75a13f0a88acb04a/tomli-2.4.1-cp313-cp313-win_arm64.whl", hash = "sha256:4b605484e43cdc43f0954ddae319fb75f04cc10dd80d830540060ee7cd0243cd", size = 95265, upload-time = "2026-03-25T20:21:41.219Z" }, + { url = "https://files.pythonhosted.org/packages/3c/fb/9a5c8d27dbab540869f7c1f8eb0abb3244189ce780ba9cd73f3770662072/tomli-2.4.1-cp314-cp314-macosx_10_15_x86_64.whl", hash = "sha256:fd0409a3653af6c147209d267a0e4243f0ae46b011aa978b1080359fddc9b6cf", size = 155726, upload-time = "2026-03-25T20:21:42.23Z" }, + { url = "https://files.pythonhosted.org/packages/62/05/d2f816630cc771ad836af54f5001f47a6f611d2d39535364f148b6a92d6b/tomli-2.4.1-cp314-cp314-macosx_11_0_arm64.whl", hash = "sha256:a120733b01c45e9a0c34aeef92bf0cf1d56cfe81ed9d47d562f9ed591a9828ac", size = 149859, upload-time = "2026-03-25T20:21:43.386Z" }, + { url = "https://files.pythonhosted.org/packages/ce/48/66341bdb858ad9bd0ceab5a86f90eddab127cf8b046418009f2125630ecb/tomli-2.4.1-cp314-cp314-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:559db847dc486944896521f68d8190be1c9e719fced785720d2216fe7022b662", size = 244713, upload-time = "2026-03-25T20:21:44.474Z" }, + { url = "https://files.pythonhosted.org/packages/df/6d/c5fad00d82b3c7a3ab6189bd4b10e60466f22cfe8a08a9394185c8a8111c/tomli-2.4.1-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:01f520d4f53ef97964a240a035ec2a869fe1a37dde002b57ebc4417a27ccd853", size = 252084, upload-time = "2026-03-25T20:21:45.62Z" }, + { url = "https://files.pythonhosted.org/packages/00/71/3a69e86f3eafe8c7a59d008d245888051005bd657760e96d5fbfb0b740c2/tomli-2.4.1-cp314-cp314-musllinux_1_2_aarch64.whl", hash = "sha256:7f94b27a62cfad8496c8d2513e1a222dd446f095fca8987fceef261225538a15", size = 247973, upload-time = "2026-03-25T20:21:46.937Z" }, + { url = "https://files.pythonhosted.org/packages/67/50/361e986652847fec4bd5e4a0208752fbe64689c603c7ae5ea7cb16b1c0ca/tomli-2.4.1-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:ede3e6487c5ef5d28634ba3f31f989030ad6af71edfb0055cbbd14189ff240ba", size = 256223, upload-time = "2026-03-25T20:21:48.467Z" }, + { url = "https://files.pythonhosted.org/packages/8c/9a/b4173689a9203472e5467217e0154b00e260621caa227b6fa01feab16998/tomli-2.4.1-cp314-cp314-win32.whl", hash = "sha256:3d48a93ee1c9b79c04bb38772ee1b64dcf18ff43085896ea460ca8dec96f35f6", size = 98973, upload-time = "2026-03-25T20:21:49.526Z" }, + { url = "https://files.pythonhosted.org/packages/14/58/640ac93bf230cd27d002462c9af0d837779f8773bc03dee06b5835208214/tomli-2.4.1-cp314-cp314-win_amd64.whl", hash = "sha256:88dceee75c2c63af144e456745e10101eb67361050196b0b6af5d717254dddf7", size = 109082, upload-time = "2026-03-25T20:21:50.506Z" }, + { url = "https://files.pythonhosted.org/packages/d5/2f/702d5e05b227401c1068f0d386d79a589bb12bf64c3d2c72ce0631e3bc49/tomli-2.4.1-cp314-cp314-win_arm64.whl", hash = "sha256:b8c198f8c1805dc42708689ed6864951fd2494f924149d3e4bce7710f8eb5232", size = 96490, upload-time = "2026-03-25T20:21:51.474Z" }, + { url = "https://files.pythonhosted.org/packages/45/4b/b877b05c8ba62927d9865dd980e34a755de541eb65fffba52b4cc495d4d2/tomli-2.4.1-cp314-cp314t-macosx_10_15_x86_64.whl", hash = "sha256:d4d8fe59808a54658fcc0160ecfb1b30f9089906c50b23bcb4c69eddc19ec2b4", size = 164263, upload-time = "2026-03-25T20:21:52.543Z" }, + { url = "https://files.pythonhosted.org/packages/24/79/6ab420d37a270b89f7195dec5448f79400d9e9c1826df982f3f8e97b24fd/tomli-2.4.1-cp314-cp314t-macosx_11_0_arm64.whl", hash = "sha256:7008df2e7655c495dd12d2a4ad038ff878d4ca4b81fccaf82b714e07eae4402c", size = 160736, upload-time = "2026-03-25T20:21:53.674Z" }, + { url = "https://files.pythonhosted.org/packages/02/e0/3630057d8eb170310785723ed5adcdfb7d50cb7e6455f85ba8a3deed642b/tomli-2.4.1-cp314-cp314t-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:1d8591993e228b0c930c4bb0db464bdad97b3289fb981255d6c9a41aedc84b2d", size = 270717, upload-time = "2026-03-25T20:21:55.129Z" }, + { url = "https://files.pythonhosted.org/packages/7a/b4/1613716072e544d1a7891f548d8f9ec6ce2faf42ca65acae01d76ea06bb0/tomli-2.4.1-cp314-cp314t-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:734e20b57ba95624ecf1841e72b53f6e186355e216e5412de414e3c51e5e3c41", size = 278461, upload-time = "2026-03-25T20:21:56.228Z" }, + { url = "https://files.pythonhosted.org/packages/05/38/30f541baf6a3f6df77b3df16b01ba319221389e2da59427e221ef417ac0c/tomli-2.4.1-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:8a650c2dbafa08d42e51ba0b62740dae4ecb9338eefa093aa5c78ceb546fcd5c", size = 274855, upload-time = "2026-03-25T20:21:57.653Z" }, + { url = "https://files.pythonhosted.org/packages/77/a3/ec9dd4fd2c38e98de34223b995a3b34813e6bdadf86c75314c928350ed14/tomli-2.4.1-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:504aa796fe0569bb43171066009ead363de03675276d2d121ac1a4572397870f", size = 283144, upload-time = "2026-03-25T20:21:59.089Z" }, + { url = "https://files.pythonhosted.org/packages/ef/be/605a6261cac79fba2ec0c9827e986e00323a1945700969b8ee0b30d85453/tomli-2.4.1-cp314-cp314t-win32.whl", hash = "sha256:b1d22e6e9387bf4739fbe23bfa80e93f6b0373a7f1b96c6227c32bef95a4d7a8", size = 108683, upload-time = "2026-03-25T20:22:00.214Z" }, + { url = "https://files.pythonhosted.org/packages/12/64/da524626d3b9cc40c168a13da8335fe1c51be12c0a63685cc6db7308daae/tomli-2.4.1-cp314-cp314t-win_amd64.whl", hash = "sha256:2c1c351919aca02858f740c6d33adea0c5deea37f9ecca1cc1ef9e884a619d26", size = 121196, upload-time = "2026-03-25T20:22:01.169Z" }, + { url = "https://files.pythonhosted.org/packages/5a/cd/e80b62269fc78fc36c9af5a6b89c835baa8af28ff5ad28c7028d60860320/tomli-2.4.1-cp314-cp314t-win_arm64.whl", hash = "sha256:eab21f45c7f66c13f2a9e0e1535309cee140182a9cdae1e041d02e47291e8396", size = 100393, upload-time = "2026-03-25T20:22:02.137Z" }, + { url = "https://files.pythonhosted.org/packages/7b/61/cceae43728b7de99d9b847560c262873a1f6c98202171fd5ed62640b494b/tomli-2.4.1-py3-none-any.whl", hash = "sha256:0d85819802132122da43cb86656f8d1f8c6587d54ae7dcaf30e90533028b49fe", size = 14583, upload-time = "2026-03-25T20:22:03.012Z" }, +] + +[[package]] +name = "typing-extensions" +version = "4.15.0" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/72/94/1a15dd82efb362ac84269196e94cf00f187f7ed21c242792a923cdb1c61f/typing_extensions-4.15.0.tar.gz", hash = "sha256:0cea48d173cc12fa28ecabc3b837ea3cf6f38c6d1136f85cbaaf598984861466", size = 109391, upload-time = "2025-08-25T13:49:26.313Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/18/67/36e9267722cc04a6b9f15c7f3441c2363321a3ea07da7ae0c0707beb2a9c/typing_extensions-4.15.0-py3-none-any.whl", hash = "sha256:f0fa19c6845758ab08074a0cfa8b7aecb71c999ca73d62883bc25cc018c4e548", size = 44614, upload-time = "2025-08-25T13:49:24.86Z" }, +] + +[[package]] +name = "urllib3" +version = "2.6.3" +source = { registry = "https://pypi.org/simple" } +resolution-markers = [ + "python_full_version < '3.10'", +] +sdist = { url = "https://files.pythonhosted.org/packages/c7/24/5f1b3bdffd70275f6661c76461e25f024d5a38a46f04aaca912426a2b1d3/urllib3-2.6.3.tar.gz", hash = "sha256:1b62b6884944a57dbe321509ab94fd4d3b307075e0c2eae991ac71ee15ad38ed", size = 435556, upload-time = "2026-01-07T16:24:43.925Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/39/08/aaaad47bc4e9dc8c725e68f9d04865dbcb2052843ff09c97b08904852d84/urllib3-2.6.3-py3-none-any.whl", hash = "sha256:bf272323e553dfb2e87d9bfd225ca7b0f467b919d7bbd355436d3fd37cb0acd4", size = 131584, upload-time = "2026-01-07T16:24:42.685Z" }, +] + +[[package]] +name = "urllib3" +version = "2.7.0" +source = { registry = "https://pypi.org/simple" } +resolution-markers = [ + "python_full_version >= '3.14'", + "python_full_version == '3.13.*'", + "python_full_version == '3.12.*'", + "python_full_version == '3.11.*'", + "python_full_version == '3.10.*'", +] +sdist = { url = "https://files.pythonhosted.org/packages/53/0c/06f8b233b8fd13b9e5ee11424ef85419ba0d8ba0b3138bf360be2ff56953/urllib3-2.7.0.tar.gz", hash = "sha256:231e0ec3b63ceb14667c67be60f2f2c40a518cb38b03af60abc813da26505f4c", size = 433602, upload-time = "2026-05-07T16:13:18.596Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/7f/3e/5db95bcf282c52709639744ca2a8b149baccf648e39c8cc87553df9eae0c/urllib3-2.7.0-py3-none-any.whl", hash = "sha256:9fb4c81ebbb1ce9531cce37674bbc6f1360472bc18ca9a553ede278ef7276897", size = 131087, upload-time = "2026-05-07T16:13:17.151Z" }, +] + +[[package]] +name = "zipp" +version = "3.23.1" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/30/21/093488dfc7cc8964ded15ab726fad40f25fd3d788fd741cc1c5a17d78ee8/zipp-3.23.1.tar.gz", hash = "sha256:32120e378d32cd9714ad503c1d024619063ec28aad2248dc6672ad13edfa5110", size = 25965, upload-time = "2026-04-13T23:21:46.6Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/08/8a/0861bec20485572fbddf3dfba2910e38fe249796cb73ecdeb74e07eeb8d3/zipp-3.23.1-py3-none-any.whl", hash = "sha256:0b3596c50a5c700c9cb40ba8d86d9f2cc4807e9bedb06bcdf7fac85633e444dc", size = 10378, upload-time = "2026-04-13T23:21:45.386Z" }, +] diff --git a/news/3785.added.md b/news/3785.added.md new file mode 100644 index 0000000000..db0ec5602d --- /dev/null +++ b/news/3785.added.md @@ -0,0 +1,5 @@ +(uv) Support for basic `uv.lock` generation via the `lock` rule +and basic support for importing the `uv.lock` file itself. Since this +may have bugs, please report this by creating new tickets. +Work towards [#2787](https://github.com/bazel-contrib/rules_python/issues/2787) +and [#1975](https://github.com/bazel-contrib/rules_python/issues/1975). diff --git a/python/private/py_repositories.bzl b/python/private/py_repositories.bzl index 9c4051ede9..3489613a6c 100644 --- a/python/private/py_repositories.bzl +++ b/python/private/py_repositories.bzl @@ -72,6 +72,12 @@ def py_repositories(transition_settings = []): strip_prefix = "rules_cc-0.1.5", urls = ["https://github.com/bazelbuild/rules_cc/releases/download/0.1.5/rules_cc-0.1.5.tar.gz"], ) + http_archive( + name = "toml.bzl", + sha256 = "63633c762bdb4d836add5a9a81deeeae93c4b2edbd62ac032733020b5652b90a", + strip_prefix = "toml.bzl-0.4.0", + url = "https://github.com/jvolkman/toml.bzl/releases/download/v0.4.0/toml.bzl-v0.4.0.tar.gz", + ) http_archive( name = "package_metadata", sha256 = "8f27dc7393e3f3bdc793bdc4ba36d67a63c22cc9d38cc65d3204654974ea4563", diff --git a/python/private/pypi/BUILD.bazel b/python/private/pypi/BUILD.bazel index 5e109a2a21..23f636c870 100644 --- a/python/private/pypi/BUILD.bazel +++ b/python/private/pypi/BUILD.bazel @@ -154,6 +154,7 @@ bzl_library( "@pythons_hub//:interpreters_bzl", "@pythons_hub//:versions_bzl", "@rules_python_internal//:rules_python_config_bzl", + "@toml.bzl//:toml", ], ) @@ -263,6 +264,7 @@ bzl_library( ":select_whl_bzl", "//python/private:normalize_name_bzl", "//python/private:repo_utils_bzl", + "//python/uv/private:uv_lock_to_requirements_bzl", ], ) diff --git a/python/private/pypi/extension.bzl b/python/private/pypi/extension.bzl index 5160b81ce8..62d2c8de16 100644 --- a/python/private/pypi/extension.bzl +++ b/python/private/pypi/extension.bzl @@ -17,6 +17,7 @@ load("@pythons_hub//:interpreters.bzl", "INTERPRETER_LABELS") load("@pythons_hub//:versions.bzl", "MINOR_MAPPING") load("@rules_python_internal//:rules_python_config.bzl", rp_config = "config") +load("@toml.bzl", "toml") load("//python/private:auth.bzl", "AUTH_ATTRS") load("//python/private:normalize_name.bzl", "normalize_name") load("//python/private:repo_utils.bzl", "repo_utils") @@ -35,7 +36,7 @@ def _whl_mods_impl(whl_mods_dict): """Implementation of the pip.whl_mods tag class. This creates the JSON files used to modify the creation of different wheels. -""" + """ for hub_name, whl_maps in whl_mods_dict.items(): whl_mods = {} @@ -260,6 +261,7 @@ def build_config( for name, values in defaults["platforms"].items() }, enable_pipstar_extract = enable_pipstar_extract, + toml_decode = toml.decode, ) def parse_modules( @@ -908,6 +910,12 @@ a string `"{os}_{arch}"` as the value here. You could also use `"{os}_{arch}_fre :::{versionadded} 1.8.0 ::: +""", + ), + "uv_lock": attr.label( + doc = """\ +(label, optional): A label pointing to the uv.lock file. If provided, +the uv.lock file will be used as the primary source for package metadata. """, ), "whl_modifications": attr.label_keyed_string_dict( diff --git a/python/private/pypi/hub_builder.bzl b/python/private/pypi/hub_builder.bzl index 34b726d6f6..7717f36731 100644 --- a/python/private/pypi/hub_builder.bzl +++ b/python/private/pypi/hub_builder.bzl @@ -503,9 +503,11 @@ def _create_whl_repos( ), logger = logger, ), + uv_lock = pip_attr.uv_lock, platforms = platforms, extra_pip_args = pip_attr.extra_pip_args, get_index_urls = self._get_index_urls.get(pip_attr.python_version), + toml_decode = getattr(self._config, "toml_decode", None), logger = logger, ) diff --git a/python/private/pypi/parse_requirements.bzl b/python/private/pypi/parse_requirements.bzl index c976e74bca..fd0399fc86 100644 --- a/python/private/pypi/parse_requirements.bzl +++ b/python/private/pypi/parse_requirements.bzl @@ -28,6 +28,7 @@ behavior. load("//python/private:normalize_name.bzl", "normalize_name") load("//python/private:repo_utils.bzl", "repo_utils") +load("//python/uv/private:uv_lock_to_requirements.bzl", "uv_lock_extras_map") # buildifier: disable=bzl-visibility load(":argparse.bzl", "argparse") load(":index_sources.bzl", "index_sources") load(":parse_requirements_txt.bzl", "parse_requirements_txt") @@ -40,15 +41,18 @@ def parse_requirements( *, requirements_by_platform = {}, extra_pip_args = [], - platforms = {}, + platforms, get_index_urls = None, extract_url_srcs = True, + uv_lock = None, + toml_decode = None, logger): """Get the requirements with platforms that the requirements apply to. Args: ctx: A context that has .read function that would read contents from a label. - platforms: The target platform descriptions. + platforms: The target platform descriptions. Cannot be empty and needs to have + at least the host platform and the definitions. requirements_by_platform (label_keyed_string_dict): a way to have different package versions (or different packages) for different os, arch combinations. @@ -59,28 +63,248 @@ def parse_requirements( distribution names to query. extract_url_srcs: A boolean to enable extracting URLs from requirement lines to enable using bazel downloader. + uv_lock: {type}`str | None` an optional label/file path to the uv.lock + file. The ctx.read function will be used to read the contents. + If provided, the function will use the uv.lock file as the primary + source for package metadata and perform a consistency check against + requirements files if both are provided. + toml_decode: {type}`callable | None` A function to decode TOML + content (e.g. `toml.decode`). Required when `uv_lock` is provided. logger: repo_utils.logger, a simple struct to log diagnostic messages. Returns: - {type}`dict[str, list[struct]]` where the key is the distribution name and the struct - contains the following attributes: - * `distribution`: {type}`str` The non-normalized distribution name. - * `srcs`: {type}`struct` The parsed requirement line for easier Simple - API downloading (see `index_sources` return value). - * `target_platforms`: {type}`list[str]` Target platforms that this package is for. - The format is `cp3{minor}_{os}_{arch}`. + {type}`list[struct]` where each struct contains the following attributes: + * `name`: {type}`str` The normalized distribution name. * `is_exposed`: {type}`bool` `True` if the package should be exposed via the hub repository. - * `extra_pip_args`: {type}`list[str]` pip args to use in case we are - not using the bazel downloader to download the archives. This should - be passed to {obj}`whl_library`. - * `whls`: {type}`list[struct]` The list of whl entries that can be - downloaded using the bazel downloader. - * `sdist`: {type}`list[struct]` The sdist that can be downloaded using - the bazel downloader. - - The second element is extra_pip_args should be passed to `whl_library`. + * `is_multiple_versions`: {type}`bool` `True` if multiple versions have been + specified for this package. + * `index_url`: {type}`str` The index URL used to download the package. + * `srcs`: {type}`list[struct]` A list of per-distribution source entries, each + containing: `distribution`, `extra_pip_args`, `requirement_line`, + `target_platforms`, `filename`, `sha256`, `url`, `yanked`. """ + if uv_lock and toml_decode: + uv_lock = toml_decode(ctx.read(uv_lock)) + return _parse_requirements_with_uv_lock( + ctx = ctx, + requirements_by_platform = requirements_by_platform, + extra_pip_args = extra_pip_args, + platforms = platforms, + get_index_urls = get_index_urls, + extract_url_srcs = extract_url_srcs, + uv_lock = uv_lock, + logger = logger, + ) + + return _parse_requirements_from_req_files( + ctx = ctx, + requirements_by_platform = requirements_by_platform, + extra_pip_args = extra_pip_args, + platforms = platforms, + get_index_urls = get_index_urls, + extract_url_srcs = extract_url_srcs, + logger = logger, + ) + +def _get_all_platforms(requirements_by_platform): + """Get the set of all platform names from requirements_by_platform.""" + all_platforms = {} + for plats in requirements_by_platform.values(): + for p in plats: + all_platforms[p] = None + return sorted(all_platforms) + +def _parse_requirements_with_uv_lock( + ctx, + *, + requirements_by_platform, + extra_pip_args, + platforms, + get_index_urls, + extract_url_srcs, + uv_lock, + logger): + """Parse requirements using uv.lock as the primary source.""" + if uv_lock: + return _parse_uv_lock_json( + uv_lock = uv_lock, + all_platforms = _get_all_platforms(requirements_by_platform) if requirements_by_platform else sorted(platforms.keys()), + platforms = platforms, + extra_pip_args = extra_pip_args, + logger = logger, + ) + + return _parse_requirements_from_req_files( + ctx = ctx, + requirements_by_platform = requirements_by_platform, + extra_pip_args = extra_pip_args, + platforms = platforms, + get_index_urls = get_index_urls, + extract_url_srcs = extract_url_srcs, + logger = logger, + ) + +def _parse_uv_lock_json(uv_lock, all_platforms, logger, extra_pip_args = None, platforms = {}): + """Parse uv.lock JSON and build the same return structs as parse_requirements. + + Args: + uv_lock: {type}`dict` The decoded uv.lock contents. + all_platforms: {type}`list[str]` The list of all platform names. + logger: {type}`struct` A logger for diagnostic messages. + extra_pip_args: {type}`list[str] | None` Extra pip arguments to pass through. + platforms: {type}`dict[str, struct]` A dict of platform name to platform info + (containing `.env` with PEP 508 marker environment). + + Returns: + {type}`list[struct]` The same format as {func}`parse_requirements`. + """ + extras_map = uv_lock_extras_map(uv_lock) + + uv_packages = {} + + if not platforms: + fail("BUG: platforms must be configured") + + for pkg in uv_lock["package"]: + name = pkg["name"] + version = pkg["version"] + norm_name = normalize_name(name) + entry = uv_packages.setdefault(norm_name, { + "distribution": name, + "resolved_srcs": [], + "versions": {}, + }) + entry["versions"][version] = None + + pkg_extras = sorted(extras_map.get(name, [])) + extra_str = "[{}]".format(",".join(pkg_extras)) if pkg_extras else "" + + markers = pkg.get("resolution-markers", []) + if markers: + marker_expr = " or ".join(markers) + pkg_platforms = [ + p + for p in all_platforms + if evaluate(marker_expr, env = platforms[p].env) + ] + else: + pkg_platforms = list(all_platforms) + + # Prepare candidates + candidates = [] + for wheel in pkg.get("wheels", []): + url = wheel["url"] + _, _, filename = url.rpartition("/") + sha256 = wheel.get("hash", "").replace("sha256:", "") + candidates.append(struct( + filename = filename, + url = url, + sha256 = sha256, + kind = "wheel", + )) + + sdist_struct = None + sdist = pkg.get("sdist", None) + if sdist: + url = sdist["url"] + _, _, filename = url.rpartition("/") + sha256 = sdist.get("hash", "").replace("sha256:", "") + sdist_struct = struct( + filename = filename, + url = url, + sha256 = sha256, + kind = "sdist", + ) + + git_struct = None + if pkg.get("source", {}).get("git"): + url = pkg["source"]["git"] + _, _, filename = url.rpartition("/") + git_struct = struct( + filename = filename, + url = url, + sha256 = "", + kind = "git", + source = pkg["source"], + ) + + plat_to_src = {} + for p in pkg_platforms: + platform = platforms.get(p) + if not platform: + continue + + best_wheel = None + if candidates: + best_wheel = select_whl( + whls = candidates, + python_version = platform.env.get("python_full_version", "3"), + whl_platform_tags = platform.whl_platform_tags, + whl_abi_tags = platform.whl_abi_tags, + implementation_name = platform.env.get("implementation_name", "cpython"), + limit = 1, + logger = logger, + ) + + if best_wheel: + plat_to_src[p] = best_wheel + elif sdist_struct: + plat_to_src[p] = sdist_struct + elif git_struct: + plat_to_src[p] = git_struct + + # Group platforms by resolved source + src_to_plats = {} + for p, src in plat_to_src.items(): + key = src.filename + src.sha256 + src_to_plats.setdefault(key, struct(src = src, plats = [])).plats.append(p) + + # Build resolved_srcs + for key, val in src_to_plats.items(): + src = val.src + plats = sorted(val.plats) + requirement_line = "{name}{extras}=={version}".format( + name = name, + extras = extra_str, + version = version, + ) + entry["resolved_srcs"].append(struct( + distribution = name, + extra_pip_args = extra_pip_args or [], + requirement_line = requirement_line, + target_platforms = plats, + filename = src.filename, + sha256 = src.sha256, + url = src.url, + yanked = None, + )) + + ret = [] + for norm_name, info in sorted(uv_packages.items()): + versions = sorted(info["versions"].keys()) + item = struct( + name = norm_name, + is_exposed = True, + is_multiple_versions = len(versions) > 1, + index_url = "", + srcs = info["resolved_srcs"], + ) + ret.append(item) + + logger.debug(lambda: "Parsed {} packages from uv.lock".format(len(ret))) + return ret + +def _parse_requirements_from_req_files( + ctx, + *, + requirements_by_platform, + extra_pip_args, + platforms, + get_index_urls, + extract_url_srcs, + logger): + """Parse requirements from requirements.txt files (existing behavior).""" options = {} requirements = {} all_files_parsed = {} @@ -90,14 +314,8 @@ def parse_requirements( logger.trace(lambda: "Using {} for {}".format(file, plats)) contents = ctx.read(file) - # Parse the requirements file directly in starlark to get the information - # needed for the whl_library declarations later. parse_result = parse_requirements_txt(contents) - # Save parsed results from ALL files, even those with no matching - # platforms. This ensures the distributions dict (used for index URL - # queries) includes packages from all platform files, making the - # lockfile facts platform-independent. if file not in all_files_parsed: all_files_parsed[file] = parse_result.requirements @@ -133,23 +351,18 @@ def parse_requirements( options[plat] = pip_args + index_url = argparse.index_url(pip_args, index_url) + extra_index_urls = argparse.extra_index_url(pip_args, []) + platform = argparse.platform(pip_args, []) + if platform: + get_index_urls = None + + reqs_by_name = {} requirements_by_platform = {} for plat, parse_results in requirements.items(): - # Replicate a surprising behavior that WORKSPACE builds allowed: - # Defining a repo with the same name multiple times, but only the last - # definition is respected. - # The requirement lines might have duplicate names because lines for extras - # are returned as just the base package name. e.g., `foo[bar]` results - # in an entry like `("foo", "foo[bar] == 1.0 ...")`. requirements_dict = {} for entry in sorted( parse_results, - # Get the longest match and fallback to original WORKSPACE sorting, - # which should get us the entry with most extras. - # - # FIXME @aignas 2024-05-13: The correct behaviour might be to get an - # entry with all aggregated extras, but it is unclear if we - # should do this now. key = lambda x: (len(x[1].partition("==")[0]), x), ): req_line = entry[1] @@ -157,32 +370,28 @@ def parse_requirements( requirements_dict[req.name] = entry - extra_pip_args = options[plat] + extra_pip_args_for_plat = options[plat] for distribution, requirement_line in requirements_dict.values(): - for_whl = requirements_by_platform.setdefault( + for_whl = reqs_by_name.setdefault( normalize_name(distribution), {}, ) for_req = for_whl.setdefault( - (requirement_line, ",".join(extra_pip_args)), + (requirement_line, ",".join(extra_pip_args_for_plat)), struct( distribution = distribution, srcs = index_sources(requirement_line), requirement_line = requirement_line, target_platforms = [], - extra_pip_args = extra_pip_args, + extra_pip_args = extra_pip_args_for_plat, ), ) for_req.target_platforms.append(plat) index_urls = {} if get_index_urls: - # Collect all distributions from all requirements files irrespective - # of python_version and platform markers. This ensures that the index - # is queried for all packages, not just those matching the current - # platform's markers. distributions = {} for entries in all_files_parsed.values(): for entry in entries: @@ -203,7 +412,7 @@ def parse_requirements( ) ret = [] - for name, reqs in sorted(requirements_by_platform.items()): + for name, reqs in sorted(reqs_by_name.items()): requirement_target_platforms = {} for r in reqs.values(): for p in r.target_platforms: @@ -219,22 +428,7 @@ def parse_requirements( logger = logger, ) - # FIXME @aignas 2025-11-24: we can get the list of target platforms here - # - # However it is likely that we may stop exposing packages like torch in here - # which do not have wheels for all osx platforms. - # - # If users specify the target platforms accurately, then it is a different - # (better) story, but we may not be able to guarantee this - # - # target_platforms = [ - # p - # for dist in package_srcs - # for p in dist.target_platforms - # ] - item = struct( - # Return normalized names name = normalize_name(name), is_exposed = len(requirement_target_platforms) == len(requirements), is_multiple_versions = len(reqs.values()) > 1, diff --git a/python/uv/private/BUILD.bazel b/python/uv/private/BUILD.bazel index 3e4d6c7baa..4ac75f4d45 100644 --- a/python/uv/private/BUILD.bazel +++ b/python/uv/private/BUILD.bazel @@ -15,13 +15,8 @@ load("@bazel_skylib//:bzl_library.bzl", "bzl_library") load("//python/private:bzlmod_enabled.bzl", "BZLMOD_ENABLED") # buildifier: disable=bzl-visibility -exports_files( - srcs = [ - "lock_copier.py", - ], - # only because this is used from a macro to template - visibility = ["//visibility:public"], -) +# public only because this is used from a macro to template +_NOT_REALLY_PUBLIC = ["//visibility:public"] filegroup( name = "distribution", @@ -68,6 +63,15 @@ bzl_library( ], ) +bzl_library( + name = "uv_lock_to_requirements_bzl", + srcs = ["uv_lock_to_requirements.bzl"], + visibility = [ + "//python/private:__subpackages__", + "//python/uv:__subpackages__", + ], +) + bzl_library( name = "uv_repository_bzl", srcs = ["uv_repository.bzl"], @@ -98,11 +102,28 @@ bzl_library( ) filegroup( - name = "lock_template", + name = "lock_copier_template", + srcs = ["template/lock_copier.py"], + target_compatible_with = [] if BZLMOD_ENABLED else ["@platforms//:incompatible"], + visibility = _NOT_REALLY_PUBLIC, +) + +filegroup( + name = "uv_lock_template", + srcs = select({ + "@platforms//os:windows": ["template/uv_lock.bat"], + "//conditions:default": ["template/uv_lock.sh"], + }), + target_compatible_with = [] if BZLMOD_ENABLED else ["@platforms//:incompatible"], + visibility = _NOT_REALLY_PUBLIC, +) + +filegroup( + name = "uv_pip_compile_template", srcs = select({ - "@platforms//os:windows": ["lock.bat"], - "//conditions:default": ["lock.sh"], + "@platforms//os:windows": ["template/uv_pip_compile.bat"], + "//conditions:default": ["template/uv_pip_compile.sh"], }), target_compatible_with = [] if BZLMOD_ENABLED else ["@platforms//:incompatible"], - visibility = ["//visibility:public"], + visibility = _NOT_REALLY_PUBLIC, ) diff --git a/python/uv/private/lock.bat b/python/uv/private/lock.bat deleted file mode 100755 index 5190ddf9eb..0000000000 --- a/python/uv/private/lock.bat +++ /dev/null @@ -1,7 +0,0 @@ -if defined BUILD_WORKSPACE_DIRECTORY ( - set "out=%BUILD_WORKSPACE_DIRECTORY%\{{src_out}}" -) else ( - exit /b 1 -) - -"{{args}}" --output-file "%out%" %* diff --git a/python/uv/private/lock.bzl b/python/uv/private/lock.bzl index 7b2aa36098..8caf0ed3de 100644 --- a/python/uv/private/lock.bzl +++ b/python/uv/private/lock.bzl @@ -30,6 +30,7 @@ _RunLockInfo = provider( "args": "The args passed to the `uv` by default when running the runnable target.", "env": "The env passed to the execution.", "srcs": "Source files required to run the runnable target.", + "template": "The template file for writing a script.", }, ) @@ -70,10 +71,11 @@ def _args(ctx): add_all = _add_all, ) -def _lock_impl(ctx): - srcs = [] + ctx.files.srcs - +def _common_lock(ctx, locker): fname = "{}.out".format(ctx.label.name) + + # TODO @aignas 2026-06-21: do not append python_version for uv.lock as it should work for all + # python versions python_version = ctx.attr.python_version if python_version: fname = "{}.{}.out".format( @@ -86,20 +88,20 @@ def _lock_impl(ctx): uv = toolchain_info.uv_toolchain_info.uv[DefaultInfo].files_to_run.executable args = _args(ctx) + args.add(uv) + + # The output params are: + # * srcs are the srcs for the locking command action inputs tracking + # * output_filename if set is to ensure that we can do special prep for uv.lock versus + # requirements.txt + # * mnemonic is for the action mnemonic + # * progress_message is the same + srcs, output_filename, mnemonic, progress_message = locker(args, output) + args.add_all([ - uv, - "pip", - "compile", "--no-python-downloads", "--no-cache", ]) - pkg = ctx.label.package - update_target = ctx.attr.update_target - args.add("--custom-compile-command", "bazel run //{}:{}".format(pkg, update_target)) - if ctx.attr.generate_hashes: - args.add("--generate-hashes") - if not ctx.attr.strip_extras: - args.add("--no-strip-extras") project = None if ctx.attr.project: @@ -116,13 +118,11 @@ def _lock_impl(ctx): project = src.dirname if project == None: - project = pkg + project = ctx.label.package if project: args.add_all([project], before_each = "--project") - args.add_all(ctx.files.build_constraints, before_each = "--build-constraints") - args.add_all(ctx.files.constraints, before_each = "--constraints") args.add_all(ctx.attr.args) exec_tools = ctx.toolchains[EXEC_TOOLS_TOOLCHAIN_TYPE].exec_tools @@ -130,90 +130,83 @@ def _lock_impl(ctx): python = runtime.interpreter or runtime.interpreter_path python_files = runtime.files or depset() args.add("--python", python) - args.add_all(srcs) - - args.run_shell.add("--output-file", output) # These arguments does not change behaviour, but it reduces the output from # the command, which is especially verbose in stderr. - args.run_shell.add("--no-progress") - args.run_shell.add("--quiet") + args.add("--no-progress") + args.add("--quiet") - # Generate a wrapper script that copies the existing output (if any) and - # then runs uv. On POSIX, args are forwarded via exec "$@". On Windows, - # the full command line is embedded in the .bat file with backslash paths - # (CMD doesn't recognize forward slashes in executable paths). - if ctx.attr.is_windows: - ext = ".bat" - lines = ["@echo off"] - else: - ext = ".sh" - lines = ["#!/usr/bin/env bash", "set -euo pipefail"] + if ctx.files.existing_output: + src_out = ctx.files.existing_output[0].path + elif output_filename: + # special case - the output filename has to be in the source tree and it has to have a + # special name, we use the project folder to determine this. - python_path = getattr(python, "path", python) + if not project: + fail("Cannot lock this if the project dir is unset or cannot be infered") - if ctx.files.existing_output: - python_cmd = "from shutil import copy; copy(\"{src}\", \"{dst}\")".format( - src = ctx.files.existing_output[0].path, - dst = output.path, + src_out = "{project}/{out_filename}".format( + project = project, + out_filename = output_filename, ) - if ctx.attr.is_windows: - # In batch files, use "" to escape internal double quotes. - lines.append( - "\"{py}\" -c \"from shutil import copy; copy(\"\"{src}\"\", \"\"{dst}\"\")\"".format( - py = python_path, - src = ctx.files.existing_output[0].path, - dst = output.path, - ), - ) - else: - lines.append("{py} -c '{cmd}'".format( - py = python_path, - cmd = python_cmd, - )) + else: + src_out = "" - if ctx.attr.is_windows: - # Build the command line with backslash paths for CMD. - # args.run_info has most args; add the output/progress/quiet - # args that were only added directly to args.run_shell. - def _quote(arg): + is_windows = ctx.attr.is_windows + if is_windows: + path_sep = "\\" + ext = ".bat" + else: + path_sep = "/" + ext = "" + + output_path = output.path.replace("/", path_sep) if is_windows else output.path + src_out_path = src_out.replace("/", path_sep) if is_windows else src_out + + # On Windows, all args must be embedded in the .bat script because + # arguments are not passed on the command line. + if is_windows: + args_parts = [] + for i, arg in enumerate(args.run_info): if hasattr(arg, "path"): - arg = arg.path.replace("/", "\\") - else: - arg = str(arg) - return '"' + arg.replace('"', '""') + '"' + arg = arg.path - bat_args = args.run_info + [ - "--output-file", - output, - "--no-progress", - "--quiet", - ] - lines.append(" ".join([_quote(a) for a in bat_args])) - - # Normalize CRLF line endings in the output on Windows. - lines.append( - "\"{py}\" -c \"import pathlib;p=pathlib.Path(r\"\"{dst}\"\");p.write_bytes(p.read_bytes().replace(b'\\r\\n', b'\\n'))\"".format( - py = python_path, - dst = output.path, - ), - ) + # Only use backslashes for the executable itself (first arg) + # to ensure CMD can run it, but keep forward slashes for arguments + # so that uv writes consistent paths in comments. + if i == 0: + a = arg.replace("/", "\\") + else: + a = arg + a = a.replace('"', '""') + args_parts.append('"' + a + '"') + + # uv pip compile adds --output-file to run_shell (not run_info). + # For the lock case, output_filename is "uv.lock" and uv lock + # writes to the project directory without --output-file. + if not output_filename: + args_parts.append('"--output-file"') + args_parts.append('"' + output_path + '"') + windows_args = " ".join(args_parts) else: - lines.append('exec "$@"') + windows_args = " ".join([]) script = ctx.actions.declare_file(ctx.label.name + "_lock" + ext) - if ctx.attr.is_windows: - content = "\r\n".join(lines) + "\r\n" - else: - content = "\n".join(lines) + "\n" - ctx.actions.write(output = script, content = content, is_executable = True) - - srcs = srcs + ctx.files.build_constraints + ctx.files.constraints + ctx.actions.expand_template( + template = ctx.files._template[0], + substitutions = { + '"{{args}}"': windows_args, + "{{out}}": output_path, + "{{src_out}}": src_out_path, + }, + output = script, + is_executable = True, + ) ctx.actions.run( executable = script, + mnemonic = mnemonic, inputs = srcs + ctx.files.existing_output, - mnemonic = "PyRequirementsLockUv", outputs = [output], # On Windows, the command line is embedded directly in the .bat # script (with backslash paths). On POSIX, args are forwarded via @@ -224,12 +217,13 @@ def _lock_impl(ctx): python_files, script, ], + # User reported being unable to add `--action_env` and get it to work. # Without this flag. # # Ref: https://app.slack.com/client/TA4K1KQ87/CA306CEV6 use_default_shell_env = True, - progress_message = "Creating a requirements.txt with uv: %{label}", + progress_message = progress_message, env = ctx.attr.env, ) @@ -242,9 +236,36 @@ def _lock_impl(ctx): srcs + [uv], transitive = [python_files], ), + template = ctx.files._template[0], ), ] +def _pip_compile_impl(ctx): + def _setup_args(args, output): + args.add_all(["pip", "compile"]) + pkg = ctx.label.package + update_target = ctx.attr.update_target + args.add("--custom-compile-command", "bazel run //{}:{}".format(pkg, update_target)) + + if ctx.attr.generate_hashes: + args.add("--generate-hashes") + if not ctx.attr.strip_extras: + args.add("--no-strip-extras") + + args.add_all(ctx.files.build_constraints, before_each = "--build-constraints") + args.add_all(ctx.files.constraints, before_each = "--constraints") + + args.run_shell.add("--output-file", output) + mnemonic = "PyRequirementsLockUv" + progress_message = "Creating a requirements.txt with uv: %{label}" + + args.add_all(ctx.files.srcs) + srcs = ctx.files.srcs + ctx.files.build_constraints + ctx.files.constraints + + return srcs, None, mnemonic, progress_message + + return _common_lock(ctx, _setup_args) + def _transition_impl(input_settings, attr): settings = { labels.PYTHON_VERSION: input_settings[labels.PYTHON_VERSION], @@ -259,65 +280,72 @@ _python_version_transition = transition( outputs = [labels.PYTHON_VERSION], ) -_lock = rule( - implementation = _lock_impl, - doc = """\ -The lock rule that does the locking in a build action (that makes it possible -to use RBE) and also prepares information for a `bazel run` executable rule. -""", - attrs = { - "args": attr.string_list( - doc = "Public, see the docs in the macro.", - ), - "build_constraints": attr.label_list( - allow_files = True, - doc = "Public, see the docs in the macro.", - ), - "constraints": attr.label_list( - allow_files = True, - doc = "Public, see the docs in the macro.", - ), - "env": attr.string_dict( - doc = "Public, see the docs in the macro.", - ), - "existing_output": attr.label( - mandatory = False, - allow_single_file = True, - doc = """\ +_common_attrs = { + "args": attr.string_list( + doc = "Public, see the docs in the macro.", + ), + "env": attr.string_dict( + doc = "Public, see the docs in the macro.", + ), + "existing_output": attr.label( + mandatory = False, + allow_single_file = True, + doc = """\ An already existing output file that is used as a basis for further modifications and the locking is not done from scratch. """, - ), - "generate_hashes": attr.bool( - doc = "Public, see the docs in the macro.", - default = True, - ), - "is_windows": attr.bool(mandatory = True), - "output": attr.string( - doc = "Public, see the docs in the macro.", - mandatory = True, - ), - "project": attr.string( - doc = """\ + ), + "is_windows": attr.bool(mandatory = True), + "output": attr.string( + doc = "Public, see the docs in the macro.", + mandatory = True, + ), + "project": attr.string( + doc = """\ Overrides the `--project` directory passed to `uv pip compile`. If not set, the project directory is auto-detected: when `pyproject.toml` files are in {obj}`lock.srcs`, the one with the shortest directory path is selected. This makes `uv` read `[tool.uv]` settings (e.g. `no-build-isolation`, `exclude-dependencies`) from that `pyproject.toml`. +""", + ), + "python_version": attr.string( + doc = "Public, see the docs in the macro.", + ), + "srcs": attr.label_list( + mandatory = True, + allow_files = True, + doc = "Public, see the docs in the macro.", + ), + "_allowlist_function_transition": attr.label( + default = "@bazel_tools//tools/allowlists/function_transition_allowlist", + ), +} + +_pip_compile = rule( + implementation = _pip_compile_impl, + doc = """\ +The lock rule that does the locking in a build action (that makes it possible +to use RBE) and also prepares information for a `bazel run` executable rule. -:::{versionadded} 2.1.0 +:::{versionchanged} 2.1.0 +Added the {attr}`project` to configure the project setting if autodetection fails. ::: """, - ), - "python_version": attr.string( + attrs = { + "build_constraints": attr.label_list( + allow_files = True, doc = "Public, see the docs in the macro.", ), - "srcs": attr.label_list( - mandatory = True, + "constraints": attr.label_list( allow_files = True, doc = "Public, see the docs in the macro.", ), + "generate_hashes": attr.bool( + doc = "Public, see the docs in the macro.", + default = True, + ), "strip_extras": attr.bool( doc = "Public, see the docs in the macro.", default = False, @@ -328,8 +356,46 @@ shortest directory path is selected. This makes `uv` read The string to input for the 'uv pip compile'. """, ), - "_allowlist_function_transition": attr.label( - default = "@bazel_tools//tools/allowlists/function_transition_allowlist", + "_template": attr.label( + default = "//python/uv/private:uv_pip_compile_template", + doc = """\ +The template to be used for 'uv pip compile'. This is either .bat or bash +script depending on what the target platform is executed on. +""", + ), + } | _common_attrs, + toolchains = [ + EXEC_TOOLS_TOOLCHAIN_TYPE, + UV_TOOLCHAIN_TYPE, + ], + cfg = _python_version_transition, +) + +def _lock_impl(ctx): + def _setup_args(args, _output): + args.add("lock") + mnemonic = "PyUvLock" + progress_message = "Creating a uv.lock with uv: %{label}" + + return ctx.files.srcs, "uv.lock", mnemonic, progress_message + + return _common_lock(ctx, _setup_args) + +_lock = rule( + implementation = _lock_impl, + doc = """\ +The lock rule that does the locking in a build action and also prepares information for a `bazel +run` executable rule. + +:::{versionadded} VERSION_NEXT_FEATURE +::: +""", + attrs = _common_attrs | { + "_template": attr.label( + default = "//python/uv/private:uv_lock_template", + doc = """\ +The template to be used for 'uv lock'. Used when output ends with '.lock'. +""", ), }, toolchains = [ @@ -339,7 +405,7 @@ The string to input for the 'uv pip compile'. cfg = _python_version_transition, ) -def _lock_run_impl(ctx): +def _run_impl(ctx): if ctx.attr.is_windows: path_sep = "\\" ext = ".bat" @@ -359,9 +425,10 @@ def _lock_run_impl(ctx): return shell.quote(arg) info = ctx.attr.lock[_RunLockInfo] + executable = ctx.actions.declare_file(ctx.label.name + ext) ctx.actions.expand_template( - template = ctx.files._template[0], + template = info.template, substitutions = { '"{{args}}"': " ".join([_maybe_path(arg) for arg in info.args]), "{{src_out}}": "{}/{}".format(ctx.label.package, ctx.attr.output).replace( @@ -383,8 +450,8 @@ def _lock_run_impl(ctx): ), ] -_lock_run = rule( - implementation = _lock_run_impl, +_run_locker = rule( + implementation = _run_impl, doc = """\ """, attrs = { @@ -397,13 +464,6 @@ _lock_run = rule( "output": attr.string( doc = """\ The output that we would be updated, relative to the package the macro is used in. -""", - ), - "_template": attr.label( - default = "//python/uv/private:lock_template", - doc = """\ -The template to be used for 'uv pip compile'. This is either .ps1 or bash -script depending on what the target platform is executed on. """, ), }, @@ -424,8 +484,10 @@ def _maybe_file(path): path: {type}`str` the file name. """ for p in native.glob([path], allow_empty = True): - if path == p: - return p + if path != p: + continue + + return p return None @@ -438,7 +500,7 @@ def _expand_template_impl(ctx): dst = "{}/{}".format(pkg, ctx.attr.output) if pkg else ctx.attr.output ctx.actions.expand_template( - template = ctx.files._template[0], + template = ctx.files._lock_copier_template[0], substitutions = { "{{dst}}": dst, "{{src}}": "{}".format(ctx.files.src[0].short_path), @@ -454,8 +516,8 @@ _expand_template = rule( "output": attr.string(mandatory = True), "src": attr.label(mandatory = True), "update_target": attr.string(mandatory = True), - "_template": attr.label( - default = "//python/uv/private:lock_copier.py", + "_lock_copier_template": attr.label( + default = "//python/uv/private:lock_copier_template", allow_single_file = True, ), }, @@ -513,8 +575,8 @@ def lock( build_constraints: {type}`list[Label]` The list of build constraints to use. constraints: {type}`list[Label]` The list of constraints files to use. generate_hashes: {type}`bool` Generate hashes for all of the - requirements. This is a must if you want to use - {attr}`pip.parse.experimental_index_url`. Defaults to `True`. + requirements. Only meaningful for `requirements.txt` style output. + Defaults to `True`. strip_extras: {type}`bool` whether to strip extras from the output. Currently `rules_python` requires `--no-strip-extras` to properly function, but sometimes one may want to not have the extras if you @@ -547,43 +609,52 @@ def lock( if not BZLMOD_ENABLED: kwargs["target_compatible_with"] = ["@platforms//:incompatible"] - _lock( - name = name, - args = args, - build_constraints = build_constraints, - constraints = constraints, - env = env, - existing_output = maybe_out, - generate_hashes = generate_hashes, - project = project, - is_windows = select({ + uv_kwargs = { + "is_windows": select({ "@platforms//os:windows": True, "//conditions:default": False, }), - python_version = python_version, - srcs = srcs, - strip_extras = strip_extras, - update_target = update_target, - output = out, - tags = [ + "output": out, + } | kwargs + + lock_target_kwargs = { + "args": args, + "env": env, + "existing_output": maybe_out, + "project": project, + "python_version": python_version, + "srcs": srcs, + "tags": [ "no-cache", "requires-network", ] + tags, - **kwargs - ) + } | uv_kwargs + + # NOTE @aignas 2026-06-20: if the user passes these args the command will fail + # with an error message instead of silently ignoring the args + if build_constraints: + lock_target_kwargs["build_constraints"] = build_constraints + if constraints: + lock_target_kwargs["constraints"] = constraints + + if out.endswith(".lock"): + _lock(name = name, **lock_target_kwargs) + else: + _pip_compile( + name = name, + generate_hashes = generate_hashes, + strip_extras = strip_extras, + update_target = update_target, + **lock_target_kwargs + ) # A target for updating the in-tree version directly by skipping the in-action - # uv pip compile. - _lock_run( + # uv pip compile or uv lock, depending what is defined for the locker_target. + _run_locker( name = locker_target, lock = name, - output = out, - is_windows = select({ - "@platforms//os:windows": True, - "//conditions:default": False, - }), tags = tags, - **kwargs + **uv_kwargs ) # FIXME @aignas 2025-03-20: is it possible to extend `py_binary` so that the diff --git a/python/uv/private/lock.sh b/python/uv/private/lock.sh deleted file mode 100755 index ffb19b2bea..0000000000 --- a/python/uv/private/lock.sh +++ /dev/null @@ -1,9 +0,0 @@ -#!/usr/bin/env bash -set -euo pipefail - -if [[ -n "${BUILD_WORKSPACE_DIRECTORY:-}" ]]; then - readonly out="${BUILD_WORKSPACE_DIRECTORY}/{{src_out}}" -else - exit 1 -fi -exec "{{args}}" --output-file "$out" "$@" diff --git a/python/uv/private/lock_copier.py b/python/uv/private/template/lock_copier.py similarity index 100% rename from python/uv/private/lock_copier.py rename to python/uv/private/template/lock_copier.py diff --git a/python/uv/private/template/uv_lock.bat b/python/uv/private/template/uv_lock.bat new file mode 100755 index 0000000000..d4f4558c53 --- /dev/null +++ b/python/uv/private/template/uv_lock.bat @@ -0,0 +1,21 @@ +@echo off +if not defined BUILD_WORKSPACE_DIRECTORY goto :not_in_workspace +"{{args}}" %* +exit /b %ERRORLEVEL% + +:not_in_workspace + +if not exist "{{src_out}}" goto :no_src_out +copy /y "{{src_out}}" "{{out}}" +del /f "{{src_out}}" +copy /y "{{out}}" "{{src_out}}" +"{{args}}" %* +set "exit_code=%ERRORLEVEL%" +copy /y "{{src_out}}" "{{out}}" +exit /b %exit_code% + +:no_src_out +"{{args}}" %* +set "exit_code=%ERRORLEVEL%" +copy /y "{{src_out}}" "{{out}}" +exit /b %exit_code% \ No newline at end of file diff --git a/python/uv/private/template/uv_lock.sh b/python/uv/private/template/uv_lock.sh new file mode 100755 index 0000000000..a6dd6d3c39 --- /dev/null +++ b/python/uv/private/template/uv_lock.sh @@ -0,0 +1,32 @@ +#!/usr/bin/env bash +set -euo pipefail + +if [[ -n "${BUILD_WORKSPACE_DIRECTORY:-}" ]]; then + exec "{{args}}" "$@" +fi + +# Build action mode +# +# If the uv.lock exists, remove because the existing uv.lock file is read-only, then symlink so +# that we can reuse the existing contents and not do a full relock all the time. If +# nothing exists, just symlink. +# +# On Windows we do it with file copies: +# 1. If the file exists: +# 1. Copy the current file to out. +# 2. Rm the existing file +# 3. Copy the contents back +# 4. Run uv +# 5. Copy the contents to out. +# 1. If the current uv.lock does not exist yet +# 1. Run uv +# 2. Copy the contents to out. +readonly out="{{out}}" +if [[ -f "{{src_out}}" ]]; then + cp "{{src_out}}" "$out" + rm "{{src_out}}" + ln -s "$(pwd)"/"$out" "{{src_out}}" +else + ln -s "$(pwd)"/"$out" "{{src_out}}" +fi +exec "$@" diff --git a/python/uv/private/template/uv_pip_compile.bat b/python/uv/private/template/uv_pip_compile.bat new file mode 100755 index 0000000000..2ea1b5a44a --- /dev/null +++ b/python/uv/private/template/uv_pip_compile.bat @@ -0,0 +1,8 @@ +@echo off +if not defined BUILD_WORKSPACE_DIRECTORY goto :else +set "out=%BUILD_WORKSPACE_DIRECTORY%\{{src_out}}" +"{{args}}" --output-file "%out%" %* +exit /b %ERRORLEVEL% + +:else +"{{args}}" %* diff --git a/python/uv/private/template/uv_pip_compile.sh b/python/uv/private/template/uv_pip_compile.sh new file mode 100755 index 0000000000..ba379ed0f9 --- /dev/null +++ b/python/uv/private/template/uv_pip_compile.sh @@ -0,0 +1,15 @@ +#!/usr/bin/env bash +set -euo pipefail + +if [[ -n "${BUILD_WORKSPACE_DIRECTORY:-}" ]]; then + readonly out="${BUILD_WORKSPACE_DIRECTORY}/{{src_out}}" + exec "{{args}}" --output-file "$out" "$@" +fi + +# Build action mode: seed the output with the source file, then run +# the full command (which includes --output-file from the action args). +readonly out="{{out}}" +if [[ -f "{{src_out}}" ]]; then + cp "{{src_out}}" "$out" +fi +exec "$@" diff --git a/python/uv/private/uv_lock_to_requirements.bzl b/python/uv/private/uv_lock_to_requirements.bzl new file mode 100644 index 0000000000..73d912bbfd --- /dev/null +++ b/python/uv/private/uv_lock_to_requirements.bzl @@ -0,0 +1,139 @@ +"""Convert a parsed uv.lock to requirements.txt format.""" + +def uv_lock_extras_map(uv_lock): + """Compute extras for each package from uv.lock data. + + Args: + uv_lock: a decoded JSON struct from a uv.lock file. + + Returns: + A dict of {package_name: [extra1, extra2, ...]} for packages with extras. + """ + extras_map = {} + for pkg in uv_lock.get("package", []): + pkg_name = pkg.get("name", "") + + for extra in pkg.get("provides-extras", pkg.get("extras", [])): + _add_extras(extras_map, pkg_name, [extra]) + + opt_deps = pkg.get("optional-dependencies", {}) + if opt_deps: + _add_extras(extras_map, pkg_name, _sorted(opt_deps.keys())) + + deps = pkg.get("dependencies", []) + for dep in deps: + dep_name = dep.get("name", "") + dep_extras_raw = dep.get("extra", []) + dep_extras = [dep_extras_raw] if type(dep_extras_raw) == "string" else dep_extras_raw + if dep_extras and dep_name != pkg_name: + _add_extras(extras_map, dep_name, dep_extras) + + metadata = pkg.get("metadata", {}) + for rd in metadata.get("requires-dist", []): + rd_name = rd.get("name", "") + rd_extras_raw = rd.get("extras", []) + rd_extras = [rd_extras_raw] if type(rd_extras_raw) == "string" else rd_extras_raw + if rd_extras and rd_name != pkg_name: + _add_extras(extras_map, rd_name, rd_extras) + + return extras_map + +def uv_lock_to_requirements(uv_lock): + """Convert a parsed uv.lock JSON struct to a requirements.txt formatted string. + + Args: + uv_lock: a decoded JSON struct from a uv.lock file. + + Returns: + A requirements.txt formatted string. + """ + packages = uv_lock.get("package", []) + extras_map = uv_lock_extras_map(uv_lock) + + dependents = {} + for pkg in packages: + pkg_name = pkg.get("name", "") + deps = pkg.get("dependencies", []) + for dep in deps: + dep_name = dep.get("name", "") + if dep_name != pkg_name: + _add_dependent(dependents, dep_name, pkg_name) + opt_deps = pkg.get("optional-dependencies", {}) + for _extra, deps in opt_deps.items(): + for dep in deps: + dep_name = dep.get("name", "") + if dep_name != pkg_name: + _add_dependent(dependents, dep_name, pkg_name) + + lines = [] + for pkg in packages: + source = pkg.get("source", {}) + if not source.get("registry"): + continue + + pkg_name = pkg.get("name", "") + version = pkg.get("version", "") + + markers = pkg.get("resolution-markers", []) + hashes = _collect_hashes(pkg) + + pkg_extras = extras_map.get(pkg_name, []) + if pkg_extras: + req = "{}[{}]=={}".format(pkg_name, ",".join(pkg_extras), version) + else: + req = "{}=={}".format(pkg_name, version) + if markers: + req += " ; " + " or ".join(markers) + + if hashes: + req += " \\" + lines.append(req) + + _emit_hashes(lines, hashes) + + dep_vias = _sorted(dependents.get(pkg_name, [])) + if dep_vias: + if len(dep_vias) == 1: + lines.append(" # via " + dep_vias[0]) + else: + lines.append(" # via") + for via in dep_vias: + lines.append(" # " + via) + + lines.append("") + + return "\n".join(lines) + +def _add_dependent(dependents, dep_name, dependent_name): + if dep_name not in dependents: + dependents[dep_name] = [] + if dependent_name not in dependents[dep_name]: + dependents[dep_name].append(dependent_name) + +def _add_extras(extras_map, pkg_name, extras): + existing = extras_map.get(pkg_name, []) + for extra in extras: + if extra not in existing: + existing.append(extra) + extras_map[pkg_name] = _sorted(existing) + +def _collect_hashes(pkg): + hashes = [] + for wheel in pkg.get("wheels", []): + whash = wheel.get("hash", "") + if whash.startswith("sha256:"): + hashes.append(whash[len("sha256:"):]) + sdist = pkg.get("sdist") + if sdist: + shash = sdist.get("hash", "") + if shash.startswith("sha256:"): + hashes.append(shash[len("sha256:"):]) + return _sorted(hashes) + +def _sorted(items): + return sorted(items) + +def _emit_hashes(lines, hashes): + for i, h in enumerate(hashes): + suffix = " \\" if i < len(hashes) - 1 else "" + lines.append(" --hash=sha256:{}{}".format(h, suffix)) diff --git a/tests/integration/bzlmod_lockfile/MODULE.bazel.lock b/tests/integration/bzlmod_lockfile/MODULE.bazel.lock index 0408f791e1..c7468217e9 100644 --- a/tests/integration/bzlmod_lockfile/MODULE.bazel.lock +++ b/tests/integration/bzlmod_lockfile/MODULE.bazel.lock @@ -161,6 +161,8 @@ "https://bcr.bazel.build/modules/swift_argument_parser/1.3.1.1/MODULE.bazel": "5e463fbfba7b1701d957555ed45097d7f984211330106ccd1352c6e0af0dcf91", "https://bcr.bazel.build/modules/swift_argument_parser/1.3.1.2/MODULE.bazel": "75aab2373a4bbe2a1260b9bf2a1ebbdbf872d3bd36f80bff058dccd82e89422f", "https://bcr.bazel.build/modules/swift_argument_parser/1.3.1.2/source.json": "5fba48bbe0ba48761f9e9f75f92876cafb5d07c0ce059cc7a8027416de94a05b", + "https://bcr.bazel.build/modules/toml.bzl/0.4.1/MODULE.bazel": "6bc0b938f03ade8d58c2fca0ad5c3fa12b4764e1e1927ad50b0c860286db2167", + "https://bcr.bazel.build/modules/toml.bzl/0.4.1/source.json": "86a90afd8b43c9b69ad31f5c03998c3adcf4b08175e621addb93e3a38eec538b", "https://bcr.bazel.build/modules/upb/0.0.0-20220923-a547704/MODULE.bazel": "7298990c00040a0e2f121f6c32544bab27d4452f80d9ce51349b1a28f3005c43", "https://bcr.bazel.build/modules/zlib/1.2.11/MODULE.bazel": "07b389abc85fdbca459b69e2ec656ae5622873af3f845e1c9d80fe179f3effa0", "https://bcr.bazel.build/modules/zlib/1.3.1.bcr.5/MODULE.bazel": "eec517b5bbe5492629466e11dae908d043364302283de25581e3eb944326c4ca", diff --git a/tests/pypi/extension/pip_parse.bzl b/tests/pypi/extension/pip_parse.bzl index 95cf666056..939639f2c5 100644 --- a/tests/pypi/extension/pip_parse.bzl +++ b/tests/pypi/extension/pip_parse.bzl @@ -30,6 +30,7 @@ def pip_parse( target_platforms = [], simpleapi_skip = [], timeout = 600, + uv_lock = None, whl_modifications = {}, **kwargs): """A simple helper for testing to simulate the PyPI extension parse tag class""" @@ -61,6 +62,7 @@ def pip_parse( requirements_lock = requirements_lock, requirements_windows = requirements_windows, timeout = timeout, + uv_lock = uv_lock, whl_modifications = whl_modifications, parallel_download = False, experimental_index_url_overrides = {}, diff --git a/tests/pypi/parse_requirements/parse_requirements_tests.bzl b/tests/pypi/parse_requirements/parse_requirements_tests.bzl index 2ef2f44764..8dd35c2260 100644 --- a/tests/pypi/parse_requirements/parse_requirements_tests.bzl +++ b/tests/pypi/parse_requirements/parse_requirements_tests.bzl @@ -97,6 +97,21 @@ foo==0.0.3 --hash=sha256:deadbaaf foo[extra]==0.0.2 --hash=sha256:deadbeef bar==0.0.1 --hash=sha256:deadb00f """, + "uv_lock_empty": """{"package":[]}""", + "uv_lock_foo": """{"package":[{"dependencies":[{"extra":"extra","name":"bar"}],"name":"foo","source":{"registry":"https://pypi.org/simple"},"version":"0.0.1","wheels":[{"hash":"sha256:deadbeef","url":"https://files.pythonhosted.org/packages/foo-0.0.1-py3-none-any.whl"}]}]}""", + "uv_lock_foo_bar": """{"package":[{"name":"bar","version":"0.0.1","source":{"registry":"https://pypi.org/simple"},"sdist":{"hash":"sha256:deadb00f","url":"https://files.pythonhosted.org/packages/bar-0.0.1.tar.gz"}},{"name":"foo","version":"0.0.1","source":{"registry":"https://pypi.org/simple"},"wheels":[{"hash":"sha256:deadbeef","url":"https://files.pythonhosted.org/packages/foo-0.0.1-py3-none-any.whl"}]}]}""", + "uv_lock_foo_dep_extra": """{"package":[{"name":"bar","version":"0.0.2","source":{"registry":"https://pypi.org/simple"},"wheels":[{"hash":"sha256:deadbeef","url":"https://files.pythonhosted.org/packages/bar-0.0.2-py3-none-any.whl"}]},{"name":"foo","version":"0.0.1","source":{"registry":"https://pypi.org/simple"},"dependencies":[{"name":"bar","extra":["extra1"]}],"wheels":[{"hash":"sha256:baadbeef","url":"https://files.pythonhosted.org/packages/foo-0.0.1-py3-none-any.whl"}]}]}""", + "uv_lock_foo_multi_versions": """{"package":[{"name":"foo","source":{"registry":"https://pypi.org/simple"},"version":"0.0.1","wheels":[{"hash":"sha256:deadbeef","url":"https://files.pythonhosted.org/packages/foo-0.0.1-py3-none-any.whl"}]},{"name":"foo","source":{"registry":"https://pypi.org/simple"},"version":"0.0.2","wheels":[{"hash":"sha256:deadb11f","url":"https://files.pythonhosted.org/packages/foo-0.0.2-py3-none-any.whl"}]}]}""", + "uv_lock_foo_multi_wheel_dedup": """{"package":[{"name":"foo","version":"0.0.1","source":{"registry":"https://pypi.org/simple"},"wheels":[{"hash":"sha256:aaa","url":"https://files.pythonhosted.org/packages/foo-0.0.1-cp39-cp39-manylinux_2_17_x86_64.whl"},{"hash":"sha256:bbb","url":"https://files.pythonhosted.org/packages/foo-0.0.1-py3-none-any.whl"}]}]}""", + "uv_lock_foo_only": """{"package":[{"name":"foo","source":{"registry":"https://pypi.org/simple"},"version":"0.0.2"}]}""", + "uv_lock_foo_optional_deps": """{"package":[{"name":"foo","version":"0.0.1","source":{"registry":"https://pypi.org/simple"},"optional-dependencies":{"extra1":[],"extra2":[]},"wheels":[{"hash":"sha256:deadbeef","url":"https://files.pythonhosted.org/packages/foo-0.0.1-py3-none-any.whl"}]}]}""", + "uv_lock_foo_requires_dist_extras": """{"package":[{"name":"foo","version":"0.0.1","source":{"registry":"https://pypi.org/simple"},"wheels":[{"hash":"sha256:deadbeef","url":"https://files.pythonhosted.org/packages/foo-0.0.1-py3-none-any.whl"}]},{"name":"root-pkg","source":{"virtual":"."},"version":"0.0.0","dependencies":[{"name":"foo"}],"metadata":{"requires-dist":[{"name":"foo","extras":["all"]}]}}]}""", + "uv_lock_foo_resolution_markers_dedup": """{"package":[{"name":"foo","source":{"registry":"https://pypi.org/simple"},"version":"0.0.1","resolution-markers":["sys_platform == 'linux'"],"wheels":[{"hash":"sha256:aaa","url":"https://files.pythonhosted.org/packages/foo-0.0.1-cp39-cp39-manylinux_2_17_x86_64.whl"},{"hash":"sha256:bbb","url":"https://files.pythonhosted.org/packages/foo-0.0.1-py3-none-any.whl"}]},{"name":"foo","source":{"registry":"https://pypi.org/simple"},"version":"0.0.2","resolution-markers":["sys_platform == 'darwin'"],"wheels":[{"hash":"sha256:ccc","url":"https://files.pythonhosted.org/packages/foo-0.0.2-cp39-cp39-macosx_11_0_arm64.whl"},{"hash":"sha256:ddd","url":"https://files.pythonhosted.org/packages/foo-0.0.2-py3-none-any.whl"}]}]}""", + "uv_lock_foo_sdist": """{"package":[{"name":"foo","sdist":{"hash":"sha256:feedcafe","url":"https://files.pythonhosted.org/packages/foo-0.0.1.tar.gz"},"source":{"registry":"https://pypi.org/simple"},"version":"0.0.1","wheels":[{"hash":"sha256:deadbeef","url":"https://files.pythonhosted.org/packages/foo-0.0.1-py3-none-any.whl"}]}]}""", + "uv_lock_foo_virtual": """{"package":[{"name":"foo","source":{"registry":"https://pypi.org/simple"},"version":"0.0.1","wheels":[{"hash":"sha256:deadbeef","url":"https://files.pythonhosted.org/packages/foo-0.0.1-py3-none-any.whl"}]},{"name":"virtual-pkg","source":{"virtual":true},"version":"0.0.0"}]}""", + "uv_lock_foo_with_extras": """{"package":[{"name":"foo","provides-extras":["extra"],"source":{"registry":"https://pypi.org/simple"},"version":"0.0.1","wheels":[{"hash":"sha256:deadbeef","url":"https://files.pythonhosted.org/packages/foo-0.0.1-py3-none-any.whl"}]}]}""", + "uv_lock_git_vcs": """{"package":[{"name":"foo","source":{"git":"https://github.com/org/foo.git"},"version":"0.1.0"}]}""", + "uv_lock_rules_python_pkg": """{"package":[{"name":"rules_python","source":{"registry":"https://pypi.org/simple"},"version":"0.0.1","wheels":[{"hash":"sha256:deadbeef","url":"https://files.pythonhosted.org/packages/rules_python-0.0.1-py3-none-any.whl"}]}]}""", } return mocks.mctx( @@ -107,7 +122,41 @@ bar==0.0.1 --hash=sha256:deadb00f _tests = [] +def _make_platforms(platform_names): + """Create minimal platform structs for testing, matching py3-none-any wheels.""" + platforms = {} + for name in platform_names: + platforms[name] = struct( + env = pep508_env(python_version = "3.11.0", os = "linux", arch = "x86_64"), + whl_abi_tags = ["none"], + whl_platform_tags = ["any"], + ) + return platforms + def parse_requirements(debug = False, **kwargs): + """Get requirements by calling the original parse_requirements. + + Args: + debug: If True, set verbosity to TRACE. + **kwargs: forwarded to the underlying function. + + Returns: + The result of the underlying parse_requirements call. + """ + kwargs.setdefault("toml_decode", json.decode) + + # Provide default platforms when not specified. + if "platforms" not in kwargs: + if "requirements_by_platform" in kwargs: + platform_names = {} + for _plats in kwargs["requirements_by_platform"].values(): + for _p in _plats: + platform_names[_p] = None + platform_names = sorted(platform_names) + kwargs["platforms"] = _make_platforms(platform_names) + elif "uv_lock" in kwargs: + kwargs["platforms"] = _make_platforms(["linux_x86_64"]) + return _parse_requirements( ctx = _mock_ctx(), logger = repo_utils.logger(struct( @@ -992,6 +1041,593 @@ def _test_get_index_urls_all_versions(env): _tests.append(_test_get_index_urls_all_versions) +def _test_uv_lock_consistent(env): + """Test that uv_lock with requirements_by_platform uses correct platforms.""" + got = parse_requirements( + requirements_by_platform = { + "requirements_lock": ["linux_x86_64", "windows_x86_64"], + }, + uv_lock = "uv_lock_foo_with_extras", + ) + env.expect.that_collection(got).contains_exactly([ + struct( + name = "foo", + index_url = "", + is_exposed = True, + is_multiple_versions = False, + srcs = [ + struct( + distribution = "foo", + extra_pip_args = [], + requirement_line = "foo[extra]==0.0.1", + target_platforms = ["linux_x86_64", "windows_x86_64"], + filename = "foo-0.0.1-py3-none-any.whl", + sha256 = "deadbeef", + url = "https://files.pythonhosted.org/packages/foo-0.0.1-py3-none-any.whl", + yanked = None, + ), + ], + ), + ]) + +_tests.append(_test_uv_lock_consistent) + +def _test_uv_lock_primary_source(env): + """Test that uv.lock can be used as the sole source without requirements files.""" + got = parse_requirements( + uv_lock = "uv_lock_foo_sdist", + ) + env.expect.that_collection(got).contains_exactly([ + struct( + name = "foo", + index_url = "", + is_exposed = True, + is_multiple_versions = False, + srcs = [ + struct( + distribution = "foo", + extra_pip_args = [], + requirement_line = "foo==0.0.1", + target_platforms = ["linux_x86_64"], + filename = "foo-0.0.1-py3-none-any.whl", + sha256 = "deadbeef", + url = "https://files.pythonhosted.org/packages/foo-0.0.1-py3-none-any.whl", + yanked = None, + ), + ], + ), + ]) + +_tests.append(_test_uv_lock_primary_source) + +def _test_uv_lock_primary_source_multiple_versions(env): + """Test that uv.lock with multiple versions of the same package works.""" + got = parse_requirements( + uv_lock = "uv_lock_foo_multi_versions", + ) + env.expect.that_collection(got).contains_exactly([ + struct( + name = "foo", + index_url = "", + is_exposed = True, + is_multiple_versions = True, + srcs = [ + struct( + distribution = "foo", + extra_pip_args = [], + requirement_line = "foo==0.0.1", + target_platforms = ["linux_x86_64"], + filename = "foo-0.0.1-py3-none-any.whl", + sha256 = "deadbeef", + url = "https://files.pythonhosted.org/packages/foo-0.0.1-py3-none-any.whl", + yanked = None, + ), + struct( + distribution = "foo", + extra_pip_args = [], + requirement_line = "foo==0.0.2", + target_platforms = ["linux_x86_64"], + filename = "foo-0.0.2-py3-none-any.whl", + sha256 = "deadb11f", + url = "https://files.pythonhosted.org/packages/foo-0.0.2-py3-none-any.whl", + yanked = None, + ), + ], + ), + ]) + +_tests.append(_test_uv_lock_primary_source_multiple_versions) + +def _test_uv_lock_primary_source_with_extras(env): + """Test that uv.lock extras are included in requirement lines.""" + got = parse_requirements( + uv_lock = "uv_lock_foo_with_extras", + ) + env.expect.that_collection(got).contains_exactly([ + struct( + name = "foo", + index_url = "", + is_exposed = True, + is_multiple_versions = False, + srcs = [ + struct( + distribution = "foo", + extra_pip_args = [], + requirement_line = "foo[extra]==0.0.1", + target_platforms = ["linux_x86_64"], + filename = "foo-0.0.1-py3-none-any.whl", + sha256 = "deadbeef", + url = "https://files.pythonhosted.org/packages/foo-0.0.1-py3-none-any.whl", + yanked = None, + ), + ], + ), + ]) + +_tests.append(_test_uv_lock_primary_source_with_extras) + +def _test_uv_lock_primary_source_includes_virtual(env): + """Test that virtual packages in uv.lock are included.""" + got = parse_requirements( + uv_lock = "uv_lock_foo_virtual", + ) + env.expect.that_collection(got).contains_exactly([ + struct( + name = "foo", + index_url = "", + is_exposed = True, + is_multiple_versions = False, + srcs = [ + struct( + distribution = "foo", + extra_pip_args = [], + requirement_line = "foo==0.0.1", + target_platforms = ["linux_x86_64"], + filename = "foo-0.0.1-py3-none-any.whl", + sha256 = "deadbeef", + url = "https://files.pythonhosted.org/packages/foo-0.0.1-py3-none-any.whl", + yanked = None, + ), + ], + ), + struct( + name = "virtual_pkg", + index_url = "", + is_exposed = True, + is_multiple_versions = False, + srcs = [], + ), + ]) + +_tests.append(_test_uv_lock_primary_source_includes_virtual) + +def _test_uv_lock_cross_consistent(env): + """Test that the uv.lock and requirements work together for cross-platform.""" + got = parse_requirements( + requirements_by_platform = { + "requirements_lock": ["linux_x86_64", "windows_x86_64"], + }, + uv_lock = "uv_lock_foo_with_extras", + ) + env.expect.that_collection(got).contains_exactly([ + struct( + name = "foo", + index_url = "", + is_exposed = True, + is_multiple_versions = False, + srcs = [ + struct( + distribution = "foo", + extra_pip_args = [], + requirement_line = "foo[extra]==0.0.1", + target_platforms = ["linux_x86_64", "windows_x86_64"], + filename = "foo-0.0.1-py3-none-any.whl", + sha256 = "deadbeef", + url = "https://files.pythonhosted.org/packages/foo-0.0.1-py3-none-any.whl", + yanked = None, + ), + ], + ), + ]) + +_tests.append(_test_uv_lock_cross_consistent) + +def _test_uv_lock_vcs_entry(env): + """Test that VCS entries in uv.lock are handled without crashing.""" + got = parse_requirements( + uv_lock = "uv_lock_git_vcs", + ) + env.expect.that_collection(got).contains_exactly([ + struct( + name = "foo", + index_url = "", + is_exposed = True, + is_multiple_versions = False, + srcs = [ + struct( + distribution = "foo", + extra_pip_args = [], + requirement_line = "foo==0.1.0", + target_platforms = ["linux_x86_64"], + filename = "foo.git", + sha256 = "", + url = "https://github.com/org/foo.git", + yanked = None, + ), + ], + ), + ]) + +_tests.append(_test_uv_lock_vcs_entry) + +def _test_uv_lock_rules_python_pkg_not_skipped(env): + """Test that 'rules_python' package is not skipped from uv.lock.""" + got = parse_requirements( + uv_lock = "uv_lock_rules_python_pkg", + ) + env.expect.that_collection(got).contains_exactly([ + struct( + name = "rules_python", + index_url = "", + is_exposed = True, + is_multiple_versions = False, + srcs = [ + struct( + distribution = "rules_python", + extra_pip_args = [], + requirement_line = "rules_python==0.0.1", + target_platforms = ["linux_x86_64"], + filename = "rules_python-0.0.1-py3-none-any.whl", + sha256 = "deadbeef", + url = "https://files.pythonhosted.org/packages/rules_python-0.0.1-py3-none-any.whl", + yanked = None, + ), + ], + ), + ]) + +_tests.append(_test_uv_lock_rules_python_pkg_not_skipped) + +def _test_uv_lock_no_consistency_check(env): + """Test that uv.lock is used as the primary source when both uv.lock and requirements exist.""" + got = parse_requirements( + requirements_by_platform = { + "requirements_lock": ["linux_x86_64"], + }, + uv_lock = "uv_lock_foo", + ) + + # The result comes from uv.lock (no extras since uv_lock_foo doesn't have provides-extras) + env.expect.that_collection(got).contains_exactly([ + struct( + name = "foo", + index_url = "", + is_exposed = True, + is_multiple_versions = False, + srcs = [ + struct( + distribution = "foo", + extra_pip_args = [], + requirement_line = "foo==0.0.1", + target_platforms = ["linux_x86_64"], + filename = "foo-0.0.1-py3-none-any.whl", + sha256 = "deadbeef", + url = "https://files.pythonhosted.org/packages/foo-0.0.1-py3-none-any.whl", + yanked = None, + ), + ], + ), + ]) + +_tests.append(_test_uv_lock_no_consistency_check) + +def _test_uv_lock_multiple_packages(env): + """Test that multiple packages from uv.lock are all returned.""" + got = parse_requirements( + uv_lock = "uv_lock_foo_bar", + ) + env.expect.that_collection(got).contains_exactly([ + struct( + name = "bar", + index_url = "", + is_exposed = True, + is_multiple_versions = False, + srcs = [ + struct( + distribution = "bar", + extra_pip_args = [], + requirement_line = "bar==0.0.1", + target_platforms = ["linux_x86_64"], + filename = "bar-0.0.1.tar.gz", + sha256 = "deadb00f", + url = "https://files.pythonhosted.org/packages/bar-0.0.1.tar.gz", + yanked = None, + ), + ], + ), + struct( + name = "foo", + index_url = "", + is_exposed = True, + is_multiple_versions = False, + srcs = [ + struct( + distribution = "foo", + extra_pip_args = [], + requirement_line = "foo==0.0.1", + target_platforms = ["linux_x86_64"], + filename = "foo-0.0.1-py3-none-any.whl", + sha256 = "deadbeef", + url = "https://files.pythonhosted.org/packages/foo-0.0.1-py3-none-any.whl", + yanked = None, + ), + ], + ), + ]) + +_tests.append(_test_uv_lock_multiple_packages) + +def _test_uv_lock_with_extra_pip_args(env): + """Test that extra_pip_args are passed through with uv.lock.""" + got = parse_requirements( + uv_lock = "uv_lock_foo", + extra_pip_args = ["--index-url=example.org"], + ) + env.expect.that_collection(got).contains_exactly([ + struct( + name = "foo", + index_url = "", + is_exposed = True, + is_multiple_versions = False, + srcs = [ + struct( + distribution = "foo", + extra_pip_args = ["--index-url=example.org"], + requirement_line = "foo==0.0.1", + target_platforms = ["linux_x86_64"], + filename = "foo-0.0.1-py3-none-any.whl", + sha256 = "deadbeef", + url = "https://files.pythonhosted.org/packages/foo-0.0.1-py3-none-any.whl", + yanked = None, + ), + ], + ), + ]) + +_tests.append(_test_uv_lock_with_extra_pip_args) + +def _test_uv_lock_multi_os_with_requirements(env): + """Test that uv.lock works with requirements_by_platform for multi-platform.""" + got = parse_requirements( + requirements_by_platform = { + "requirements_foo": ["linux_aarch64"], + "requirements_lock": ["linux_x86_64", "windows_x86_64"], + }, + uv_lock = "uv_lock_foo", + ) + env.expect.that_collection(got).contains_exactly([ + struct( + name = "foo", + index_url = "", + is_exposed = True, + is_multiple_versions = False, + srcs = [ + struct( + distribution = "foo", + extra_pip_args = [], + requirement_line = "foo==0.0.1", + target_platforms = ["linux_aarch64", "linux_x86_64", "windows_x86_64"], + filename = "foo-0.0.1-py3-none-any.whl", + sha256 = "deadbeef", + url = "https://files.pythonhosted.org/packages/foo-0.0.1-py3-none-any.whl", + yanked = None, + ), + ], + ), + ]) + +_tests.append(_test_uv_lock_multi_os_with_requirements) + +def _test_uv_lock_extras_optional_deps(env): + """Test that extras from optional-dependencies in uv.lock are included.""" + got = parse_requirements( + uv_lock = "uv_lock_foo_optional_deps", + ) + env.expect.that_collection(got).contains_exactly([ + struct( + name = "foo", + index_url = "", + is_exposed = True, + is_multiple_versions = False, + srcs = [ + struct( + distribution = "foo", + extra_pip_args = [], + requirement_line = "foo[extra1,extra2]==0.0.1", + target_platforms = ["linux_x86_64"], + filename = "foo-0.0.1-py3-none-any.whl", + sha256 = "deadbeef", + url = "https://files.pythonhosted.org/packages/foo-0.0.1-py3-none-any.whl", + yanked = None, + ), + ], + ), + ]) + +_tests.append(_test_uv_lock_extras_optional_deps) + +def _test_uv_lock_extras_dep_edge(env): + """Test that dep extra edges in uv.lock add extras to the dependency.""" + got = parse_requirements( + uv_lock = "uv_lock_foo_dep_extra", + ) + env.expect.that_collection(got).contains_exactly([ + struct( + name = "bar", + index_url = "", + is_exposed = True, + is_multiple_versions = False, + srcs = [ + struct( + distribution = "bar", + extra_pip_args = [], + requirement_line = "bar[extra1]==0.0.2", + target_platforms = ["linux_x86_64"], + filename = "bar-0.0.2-py3-none-any.whl", + sha256 = "deadbeef", + url = "https://files.pythonhosted.org/packages/bar-0.0.2-py3-none-any.whl", + yanked = None, + ), + ], + ), + struct( + name = "foo", + index_url = "", + is_exposed = True, + is_multiple_versions = False, + srcs = [ + struct( + distribution = "foo", + extra_pip_args = [], + requirement_line = "foo==0.0.1", + target_platforms = ["linux_x86_64"], + filename = "foo-0.0.1-py3-none-any.whl", + sha256 = "baadbeef", + url = "https://files.pythonhosted.org/packages/foo-0.0.1-py3-none-any.whl", + yanked = None, + ), + ], + ), + ]) + +_tests.append(_test_uv_lock_extras_dep_edge) + +def _test_uv_lock_wheel_dedup_single_version(env): + """Test that overlapping wheels for a single version are deduplicated to one per platform.""" + got = parse_requirements( + uv_lock = "uv_lock_foo_multi_wheel_dedup", + platforms = { + "cp39_linux_x86_64": struct( + env = pep508_env(python_version = "3.9.0", os = "linux", arch = "x86_64"), + whl_abi_tags = ["none", "abi3", "cp39"], + whl_platform_tags = ["any", "linux_x86_64", "manylinux_*_x86_64"], + ), + }, + ) + env.expect.that_collection(got).contains_exactly([ + struct( + name = "foo", + index_url = "", + is_exposed = True, + is_multiple_versions = False, + srcs = [ + struct( + distribution = "foo", + extra_pip_args = [], + requirement_line = "foo==0.0.1", + target_platforms = ["cp39_linux_x86_64"], + filename = "foo-0.0.1-cp39-cp39-manylinux_2_17_x86_64.whl", + sha256 = "aaa", + url = "https://files.pythonhosted.org/packages/foo-0.0.1-cp39-cp39-manylinux_2_17_x86_64.whl", + yanked = None, + ), + ], + ), + ]) + +_tests.append(_test_uv_lock_wheel_dedup_single_version) + +def _test_uv_lock_wheel_dedup_resolution_markers(env): + """Test that resolution-markers filtering and wheel dedup work together. + + Two versions of foo with resolution-markers for different platforms. + Each version has a platform-specific wheel and a generic py3-none-any wheel. + The dedup should pick the platform-specific wheel for each platform and + the resolution-markers should split versions across platforms. + """ + got = parse_requirements( + uv_lock = "uv_lock_foo_resolution_markers_dedup", + platforms = { + "cp39_linux_x86_64": struct( + env = pep508_env(python_version = "3.9.0", os = "linux", arch = "x86_64"), + whl_abi_tags = ["none", "abi3", "cp39"], + whl_platform_tags = ["any", "linux_x86_64", "manylinux_*_x86_64"], + ), + "cp39_osx_aarch64": struct( + env = pep508_env(python_version = "3.9.0", os = "osx", arch = "aarch64"), + whl_abi_tags = ["none", "abi3", "cp39"], + whl_platform_tags = ["any", "macosx_*_arm64"], + ), + }, + ) + env.expect.that_collection(got).contains_exactly([ + struct( + name = "foo", + index_url = "", + is_exposed = True, + is_multiple_versions = True, + srcs = [ + struct( + distribution = "foo", + extra_pip_args = [], + requirement_line = "foo==0.0.1", + target_platforms = ["cp39_linux_x86_64"], + filename = "foo-0.0.1-cp39-cp39-manylinux_2_17_x86_64.whl", + sha256 = "aaa", + url = "https://files.pythonhosted.org/packages/foo-0.0.1-cp39-cp39-manylinux_2_17_x86_64.whl", + yanked = None, + ), + struct( + distribution = "foo", + extra_pip_args = [], + requirement_line = "foo==0.0.2", + target_platforms = ["cp39_osx_aarch64"], + filename = "foo-0.0.2-cp39-cp39-macosx_11_0_arm64.whl", + sha256 = "ccc", + url = "https://files.pythonhosted.org/packages/foo-0.0.2-cp39-cp39-macosx_11_0_arm64.whl", + yanked = None, + ), + ], + ), + ]) + +_tests.append(_test_uv_lock_wheel_dedup_resolution_markers) + +def _test_uv_lock_requires_dist_extras(env): + """Test that extras from metadata.requires-dist appear in requirement_line.""" + got = parse_requirements( + uv_lock = "uv_lock_foo_requires_dist_extras", + ) + env.expect.that_collection(got).contains_exactly([ + struct( + name = "foo", + index_url = "", + is_exposed = True, + is_multiple_versions = False, + srcs = [ + struct( + distribution = "foo", + extra_pip_args = [], + requirement_line = "foo[all]==0.0.1", + target_platforms = ["linux_x86_64"], + filename = "foo-0.0.1-py3-none-any.whl", + sha256 = "deadbeef", + url = "https://files.pythonhosted.org/packages/foo-0.0.1-py3-none-any.whl", + yanked = None, + ), + ], + ), + struct( + name = "root_pkg", + index_url = "", + is_exposed = True, + is_multiple_versions = False, + srcs = [], + ), + ]) + +_tests.append(_test_uv_lock_requires_dist_extras) + def parse_requirements_test_suite(name): """Create the test suite. diff --git a/tests/uv/lock/BUILD.bazel b/tests/uv/lock/BUILD.bazel index 0b72f015b7..60d680bde8 100644 --- a/tests/uv/lock/BUILD.bazel +++ b/tests/uv/lock/BUILD.bazel @@ -1,4 +1,5 @@ load(":lock_tests.bzl", "lock_test_suite") +load(":uv_lock_to_requirements_tests.bzl", "uv_lock_to_requirements_test_suite") exports_files( glob(["testdata/*"]), @@ -8,3 +9,7 @@ exports_files( lock_test_suite( name = "lock_tests", ) + +uv_lock_to_requirements_test_suite( + name = "uv_lock_to_requirements_tests", +) diff --git a/tests/uv/lock/lock_run_test.py b/tests/uv/lock/lock_run_test.py index e2508161d5..6de5a96378 100644 --- a/tests/uv/lock/lock_run_test.py +++ b/tests/uv/lock/lock_run_test.py @@ -63,6 +63,86 @@ def _subprocess_env(self, workspace_dir: Path) -> dict[str, str]: env[key] = os.environ[key] return env + def test_requirements_run_script_for_new_file(self): + """Verify the requirements_new_file.run script has expected args.""" + run_script_path = _relative_rpath("requirements_new_file.run") + content = run_script_path.read_text() + + if os.name == "nt": + self.assertIn("@echo off", content) + else: + self.assertIn("#!/usr/bin/env bash", content) + self.assertIn("BUILD_WORKSPACE_DIRECTORY", content) + self.assertIn("--no-progress", content) + self.assertIn("--quiet", content) + self.assertIn("does_not_exist.txt", content) + + def test_uv_lock_run_script(self): + """Verify the uv_lock_test.run script has expected args.""" + run_script_path = _relative_rpath("uv_lock_test.run") + content = run_script_path.read_text() + + if os.name == "nt": + self.assertIn("@echo off", content) + else: + self.assertIn("#!/usr/bin/env bash", content) + self.assertIn("--no-progress", content) + self.assertIn("--quiet", content) + + def test_run_script_has_no_output_file_arg(self): + """Verify the uv lock .run script does NOT have --output-file (uv lock doesn't use it).""" + run_script_path = _relative_rpath("uv_lock_test.run") + content = run_script_path.read_text() + + self.assertNotIn("--output-file", content) + + def test_debug_requirements_diff(self): + """Temporary test to print diff of generated vs expected requirements on Windows.""" + expected_path = _relative_rpath("testdata/requirements.txt") + try: + # Pass "requirements.out" directly. We need to handle the case where + # _relative_rpath might append extensions on Windows. + # Actually, _relative_rpath has: + # exts = (".exe", ".bat", "") if os.name == "nt" else ... + # So on Windows it will try .exe, .bat, then empty. + # If "requirements.out" exists, the empty extension will match it. + generated_path = _relative_rpath("requirements.out") + except ValueError as e: + print(f"Could not find requirements.out: {e}") + # Dump runfiles directory to help debug if needed + runfiles_dir = os.environ.get("RUNFILES_DIR") + if runfiles_dir: + print(f"RUNFILES_DIR: {runfiles_dir}") + for root, _, files in os.walk(runfiles_dir): + for f in files: + if "requirements" in f: + print(os.path.join(root, f)) + raise + + expected = expected_path.read_text() + generated = generated_path.read_text() + + if expected != generated: + import difflib + + diff = list( + difflib.unified_diff( + expected.splitlines(keepends=True), + generated.splitlines(keepends=True), + fromfile="expected (testdata/requirements.txt)", + tofile="generated (requirements.out)", + ) + ) + print("\n=== DIFF START ===") + print("".join(diff)) + print("=== DIFF END ===\n") + + # Also print representation to see line endings + print(f"Expected line endings: {repr(expected[:100])}") + print(f"Generated line endings: {repr(generated[:100])}") + + self.assertEqual(expected, generated, "Files differ! See diff above.") + def test_requirements_updating_for_the_first_time(self): # Given copier_path = _relative_rpath("requirements_new_file.update") @@ -94,8 +174,6 @@ def test_requirements_updating_for_the_first_time(self): def test_requirements_updating(self): # Given copier_path = _relative_rpath("requirements.update") - existing_file = _relative_rpath("testdata/requirements.txt") - want_text = existing_file.read_text() # When with tempfile.TemporaryDirectory() as dir: @@ -109,9 +187,6 @@ def test_requirements_updating(self): / "requirements.txt" ) want_path.parent.mkdir(parents=True) - want_path.write_text( - want_text + "\n\n" - ) # Write something else to see that it is restored output = _run_binary( copier_path, @@ -126,7 +201,8 @@ def test_requirements_updating(self): "cp /tests/uv/lock/requirements", stdout, ) - self.assertEqual(want_path.read_text(), want_text) + self.assertTrue(want_path.exists(), "The path should exist after the test") + self.assertNotEqual(want_path.read_text(), "") def test_requirements_run_on_the_first_time(self): # Given @@ -141,7 +217,7 @@ def test_requirements_run_on_the_first_time(self): want_path.parent.mkdir(parents=True) self.assertFalse( - want_path.exists(), "The path should not exist after the test" + want_path.exists(), "The path should not exist before the test" ) output = _run_binary( copier_path, @@ -154,16 +230,33 @@ def test_requirements_run_on_the_first_time(self): self.assertTrue(want_path.exists(), "The path should exist after the test") got_contents = want_path.read_text() self.assertNotEqual(got_contents, "") - self.assertIn( - got_contents, - output.stdout.decode("utf-8"), - ) + # NOTE: stdout is typically empty because uv runs with --quiet --no-progress + + def test_requirements_run_script_has_expected_args(self): + """Verify the .run script template has expected args embedded.""" + run_script_path = _relative_rpath("requirements.run") + content = run_script_path.read_text() + + if os.name == "nt": + self.assertIn("@echo off", content) + self.assertIn("%*", content) + else: + self.assertIn("#!/usr/bin/env bash", content) + self.assertIn('"$@"', content) + self.assertIn("BUILD_WORKSPACE_DIRECTORY", content) + self.assertIn("--custom-compile-command", content) + self.assertIn("--generate-hashes", content) + self.assertIn("--no-strip-extras", content) + self.assertIn("--no-python-downloads", content) + self.assertIn("--no-cache", content) + self.assertIn("--no-progress", content) + self.assertIn("--quiet", content) + self.assertIn("--output-file", content) + self.assertIn("requirements.txt", content) def test_requirements_run(self): # Given copier_path = _relative_rpath("requirements.run") - existing_file = _relative_rpath("testdata/requirements.txt") - want_text = existing_file.read_text() # When with tempfile.TemporaryDirectory() as dir: @@ -176,11 +269,7 @@ def test_requirements_run(self): / "testdata" / "requirements.txt" ) - want_path.parent.mkdir(parents=True) - want_path.write_text( - want_text + "\n\n" - ) # Write something else to see that it is restored output = _run_binary( copier_path, @@ -193,10 +282,7 @@ def test_requirements_run(self): self.assertTrue(want_path.exists(), "The path should exist after the test") got_contents = want_path.read_text() self.assertNotEqual(got_contents, "") - self.assertIn( - got_contents, - output.stdout.decode("utf-8"), - ) + # NOTE: stdout is typically empty because uv runs with --quiet --no-progress if __name__ == "__main__": diff --git a/tests/uv/lock/lock_tests.bzl b/tests/uv/lock/lock_tests.bzl index 3e067f3e73..e3e035717f 100644 --- a/tests/uv/lock/lock_tests.bzl +++ b/tests/uv/lock/lock_tests.bzl @@ -15,6 +15,7 @@ "" load("@bazel_skylib//rules:diff_test.bzl", "diff_test") +load("@bazel_skylib//rules:native_binary.bzl", "native_test") load("//python/uv:lock.bzl", "lock") load("//tests/support:py_reconfig.bzl", "py_reconfig_test") @@ -65,6 +66,8 @@ def lock_test_suite(name): "requirements.update", "requirements.run", "testdata/requirements.txt", + "uv_lock_test.run", + ":requirements", ], main = "lock_run_test.py", tags = [ @@ -87,11 +90,28 @@ def lock_test_suite(name): file2 = "testdata/requirements.txt", ) + lock( + name = "uv_lock_test", + srcs = ["testdata/pyproject.toml"], + out = "testdata/uv_lock_expected.lock", + tags = ["no-remote-exec"], + ) + + native_test( + name = "uv_lock_test_check", + src = ":uv_lock_test.update", + target_compatible_with = select({ + "@platforms//os:windows": ["@platforms//:incompatible"], + "//conditions:default": [], + }), + ) + native.test_suite( name = name, tests = [ ":requirements_test", "//tests/uv/lock/pyproject_toml:requirements_test", ":requirements_run_tests", + ":uv_lock_test_check", ], ) diff --git a/tests/uv/lock/pyproject_toml/requirements.txt b/tests/uv/lock/pyproject_toml/requirements.txt index 4ea098b0e4..2f4330e1ae 100644 --- a/tests/uv/lock/pyproject_toml/requirements.txt +++ b/tests/uv/lock/pyproject_toml/requirements.txt @@ -3,16 +3,24 @@ certifi==2025.1.31 \ --hash=sha256:3d5da6925056f6f18f119200434a4780a94263f10d1c21d032a6f6b2baa20651 \ --hash=sha256:ca78db4565a652026a4db2bcdf68f2fb589ea80d0be70e03929ed730746b84fe - # via requests + # via + # -c tests/uv/lock/testdata/constraints.txt + # requests idna==3.10 \ --hash=sha256:12f65c9b470abda6dc35cf8e63cc574b1c52b11df2c86030af0ac09b01b13ea9 \ --hash=sha256:946d195a0d259cbba61165e88e65941f16e9b36ea6ddb97f00452bae8b1287d3 - # via requests + # via + # -c tests/uv/lock/testdata/constraints.txt + # requests requests==2.32.3 \ --hash=sha256:55365417734eb18255590a9ff9eb97e9e1da868d4ccd6402399eaf68af20a760 \ --hash=sha256:70761cfe03c773ceb22aa2f671b4757976145175cdfca038c02654d061d6dcc6 - # via test (tests/uv/lock/pyproject_toml/pyproject.toml) + # via + # -c tests/uv/lock/testdata/constraints.txt + # test (tests/uv/lock/pyproject_toml/pyproject.toml) urllib3==2.3.0 \ --hash=sha256:1cee9ad369867bfdbbb48b7dd50374c0967a0bb7710050facf0dd6911440e3df \ --hash=sha256:f8c5449b3cf0861679ce7e0503c7b44b5ec981bec0d1d3795a07f1ba96f0204d - # via requests + # via + # -c tests/uv/lock/testdata/constraints.txt + # requests diff --git a/tests/uv/lock/testdata/constraints.txt b/tests/uv/lock/testdata/constraints.txt index 18ade2c5b9..1d4f29173d 100644 --- a/tests/uv/lock/testdata/constraints.txt +++ b/tests/uv/lock/testdata/constraints.txt @@ -1 +1,5 @@ charset-normalizer==3.4.0 +certifi==2025.1.31 +requests==2.32.3 +idna==3.10 +urllib3==2.3.0 diff --git a/tests/uv/lock/testdata/pyproject.toml b/tests/uv/lock/testdata/pyproject.toml new file mode 100644 index 0000000000..d72efcf7c0 --- /dev/null +++ b/tests/uv/lock/testdata/pyproject.toml @@ -0,0 +1,5 @@ +[project] +name = "test-project" +version = "0.0.1" +dependencies = ["requests"] +requires-python = ">=3.9" diff --git a/tests/uv/lock/testdata/requirements.txt b/tests/uv/lock/testdata/requirements.txt index d02844636d..8b718fb145 100644 --- a/tests/uv/lock/testdata/requirements.txt +++ b/tests/uv/lock/testdata/requirements.txt @@ -3,7 +3,9 @@ certifi==2025.1.31 \ --hash=sha256:3d5da6925056f6f18f119200434a4780a94263f10d1c21d032a6f6b2baa20651 \ --hash=sha256:ca78db4565a652026a4db2bcdf68f2fb589ea80d0be70e03929ed730746b84fe - # via requests + # via + # -c tests/uv/lock/testdata/constraints.txt + # requests charset-normalizer==3.4.0 \ --hash=sha256:0099d79bdfcf5c1f0c2c72f91516702ebf8b0b8ddd8905f97a8aecf49712c621 \ --hash=sha256:0713f3adb9d03d49d365b70b84775d0a0d18e4ab08d12bc46baa6132ba78aaf6 \ @@ -117,12 +119,18 @@ charset-normalizer==3.4.0 \ idna==3.10 \ --hash=sha256:12f65c9b470abda6dc35cf8e63cc574b1c52b11df2c86030af0ac09b01b13ea9 \ --hash=sha256:946d195a0d259cbba61165e88e65941f16e9b36ea6ddb97f00452bae8b1287d3 - # via requests + # via + # -c tests/uv/lock/testdata/constraints.txt + # requests requests==2.32.3 \ --hash=sha256:55365417734eb18255590a9ff9eb97e9e1da868d4ccd6402399eaf68af20a760 \ --hash=sha256:70761cfe03c773ceb22aa2f671b4757976145175cdfca038c02654d061d6dcc6 - # via -r tests/uv/lock/testdata/requirements.in + # via + # -c tests/uv/lock/testdata/constraints.txt + # -r tests/uv/lock/testdata/requirements.in urllib3==2.3.0 \ --hash=sha256:1cee9ad369867bfdbbb48b7dd50374c0967a0bb7710050facf0dd6911440e3df \ --hash=sha256:f8c5449b3cf0861679ce7e0503c7b44b5ec981bec0d1d3795a07f1ba96f0204d - # via requests + # via + # -c tests/uv/lock/testdata/constraints.txt + # requests diff --git a/tests/uv/lock/testdata/uv_lock_expected.lock b/tests/uv/lock/testdata/uv_lock_expected.lock new file mode 100644 index 0000000000..fc027674ef --- /dev/null +++ b/tests/uv/lock/testdata/uv_lock_expected.lock @@ -0,0 +1,218 @@ +version = 1 +revision = 3 +requires-python = ">=3.9" +resolution-markers = [ + "python_full_version >= '3.10'", + "python_full_version < '3.10'", +] + +[[package]] +name = "certifi" +version = "2026.4.22" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/25/ee/6caf7a40c36a1220410afe15a1cc64993a1f864871f698c0f93acb72842a/certifi-2026.4.22.tar.gz", hash = "sha256:8d455352a37b71bf76a79caa83a3d6c25afee4a385d632127b6afb3963f1c580", size = 137077, upload-time = "2026-04-22T11:26:11.191Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/22/30/7cd8fdcdfbc5b869528b079bfb76dcdf6056b1a2097a662e5e8c04f42965/certifi-2026.4.22-py3-none-any.whl", hash = "sha256:3cb2210c8f88ba2318d29b0388d1023c8492ff72ecdde4ebdaddbb13a31b1c4a", size = 135707, upload-time = "2026-04-22T11:26:09.372Z" }, +] + +[[package]] +name = "charset-normalizer" +version = "3.4.7" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/e7/a1/67fe25fac3c7642725500a3f6cfe5821ad557c3abb11c9d20d12c7008d3e/charset_normalizer-3.4.7.tar.gz", hash = "sha256:ae89db9e5f98a11a4bf50407d4363e7b09b31e55bc117b4f7d80aab97ba009e5", size = 144271, upload-time = "2026-04-02T09:28:39.342Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/26/08/0f303cb0b529e456bb116f2d50565a482694fbb94340bf56d44677e7ed03/charset_normalizer-3.4.7-cp310-cp310-macosx_10_9_universal2.whl", hash = "sha256:cdd68a1fb318e290a2077696b7eb7a21a49163c455979c639bf5a5dcdc46617d", size = 315182, upload-time = "2026-04-02T09:25:40.673Z" }, + { url = "https://files.pythonhosted.org/packages/24/47/b192933e94b546f1b1fe4df9cc1f84fcdbf2359f8d1081d46dd029b50207/charset_normalizer-3.4.7-cp310-cp310-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:e17b8d5d6a8c47c85e68ca8379def1303fd360c3e22093a807cd34a71cd082b8", size = 209329, upload-time = "2026-04-02T09:25:42.354Z" }, + { url = "https://files.pythonhosted.org/packages/c2/b4/01fa81c5ca6141024d89a8fc15968002b71da7f825dd14113207113fabbd/charset_normalizer-3.4.7-cp310-cp310-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:511ef87c8aec0783e08ac18565a16d435372bc1ac25a91e6ac7f5ef2b0bff790", size = 231230, upload-time = "2026-04-02T09:25:44.281Z" }, + { url = "https://files.pythonhosted.org/packages/20/f7/7b991776844dfa058017e600e6e55ff01984a063290ca5622c0b63162f68/charset_normalizer-3.4.7-cp310-cp310-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:007d05ec7321d12a40227aae9e2bc6dca73f3cb21058999a1df9e193555a9dcc", size = 225890, upload-time = "2026-04-02T09:25:45.475Z" }, + { url = "https://files.pythonhosted.org/packages/20/e7/bed0024a0f4ab0c8a9c64d4445f39b30c99bd1acd228291959e3de664247/charset_normalizer-3.4.7-cp310-cp310-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:cf29836da5119f3c8a8a70667b0ef5fdca3bb12f80fd06487cfa575b3909b393", size = 216930, upload-time = "2026-04-02T09:25:46.58Z" }, + { url = "https://files.pythonhosted.org/packages/e2/ab/b18f0ab31cdd7b3ddb8bb76c4a414aeb8160c9810fdf1bc62f269a539d87/charset_normalizer-3.4.7-cp310-cp310-manylinux_2_31_armv7l.whl", hash = "sha256:12d8baf840cc7889b37c7c770f478adea7adce3dcb3944d02ec87508e2dcf153", size = 202109, upload-time = "2026-04-02T09:25:48.031Z" }, + { url = "https://files.pythonhosted.org/packages/82/e5/7e9440768a06dfb3075936490cb82dbf0ee20a133bf0dd8551fa096914ec/charset_normalizer-3.4.7-cp310-cp310-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:d560742f3c0d62afaccf9f41fe485ed69bd7661a241f86a3ef0f0fb8b1a397af", size = 214684, upload-time = "2026-04-02T09:25:49.245Z" }, + { url = "https://files.pythonhosted.org/packages/71/94/8c61d8da9f062fdf457c80acfa25060ec22bf1d34bbeaca4350f13bcfd07/charset_normalizer-3.4.7-cp310-cp310-musllinux_1_2_aarch64.whl", hash = "sha256:b14b2d9dac08e28bb8046a1a0434b1750eb221c8f5b87a68f4fa11a6f97b5e34", size = 212785, upload-time = "2026-04-02T09:25:50.671Z" }, + { url = "https://files.pythonhosted.org/packages/66/cd/6e9889c648e72c0ab2e5967528bb83508f354d706637bc7097190c874e13/charset_normalizer-3.4.7-cp310-cp310-musllinux_1_2_armv7l.whl", hash = "sha256:bc17a677b21b3502a21f66a8cc64f5bfad4df8a0b8434d661666f8ce90ac3af1", size = 203055, upload-time = "2026-04-02T09:25:51.802Z" }, + { url = "https://files.pythonhosted.org/packages/92/2e/7a951d6a08aefb7eb8e1b54cdfb580b1365afdd9dd484dc4bee9e5d8f258/charset_normalizer-3.4.7-cp310-cp310-musllinux_1_2_ppc64le.whl", hash = "sha256:750e02e074872a3fad7f233b47734166440af3cdea0add3e95163110816d6752", size = 232502, upload-time = "2026-04-02T09:25:53.388Z" }, + { url = "https://files.pythonhosted.org/packages/58/d5/abcf2d83bf8e0a1286df55cd0dc1d49af0da4282aa77e986df343e7de124/charset_normalizer-3.4.7-cp310-cp310-musllinux_1_2_riscv64.whl", hash = "sha256:4e5163c14bffd570ef2affbfdd77bba66383890797df43dc8b4cc7d6f500bf53", size = 214295, upload-time = "2026-04-02T09:25:54.765Z" }, + { url = "https://files.pythonhosted.org/packages/47/3a/7d4cd7ed54be99973a0dc176032cba5cb1f258082c31fa6df35cff46acfc/charset_normalizer-3.4.7-cp310-cp310-musllinux_1_2_s390x.whl", hash = "sha256:6ed74185b2db44f41ef35fd1617c5888e59792da9bbc9190d6c7300617182616", size = 227145, upload-time = "2026-04-02T09:25:55.904Z" }, + { url = "https://files.pythonhosted.org/packages/1d/98/3a45bf8247889cf28262ebd3d0872edff11565b2a1e3064ccb132db3fbb0/charset_normalizer-3.4.7-cp310-cp310-musllinux_1_2_x86_64.whl", hash = "sha256:94e1885b270625a9a828c9793b4d52a64445299baa1fea5a173bf1d3dd9a1a5a", size = 218884, upload-time = "2026-04-02T09:25:57.074Z" }, + { url = "https://files.pythonhosted.org/packages/ad/80/2e8b7f8915ed5c9ef13aa828d82738e33888c485b65ebf744d615040c7ea/charset_normalizer-3.4.7-cp310-cp310-win32.whl", hash = "sha256:6785f414ae0f3c733c437e0f3929197934f526d19dfaa75e18fdb4f94c6fb374", size = 148343, upload-time = "2026-04-02T09:25:58.199Z" }, + { url = "https://files.pythonhosted.org/packages/35/1b/3b8c8c77184af465ee9ad88b5aea46ea6b2e1f7b9dc9502891e37af21e30/charset_normalizer-3.4.7-cp310-cp310-win_amd64.whl", hash = "sha256:6696b7688f54f5af4462118f0bfa7c1621eeb87154f77fa04b9295ce7a8f2943", size = 159174, upload-time = "2026-04-02T09:25:59.322Z" }, + { url = "https://files.pythonhosted.org/packages/be/c1/feb40dca40dbb21e0a908801782d9288c64fc8d8e562c2098e9994c8c21b/charset_normalizer-3.4.7-cp310-cp310-win_arm64.whl", hash = "sha256:66671f93accb62ed07da56613636f3641f1a12c13046ce91ffc923721f23c008", size = 147805, upload-time = "2026-04-02T09:26:00.756Z" }, + { url = "https://files.pythonhosted.org/packages/c2/d7/b5b7020a0565c2e9fa8c09f4b5fa6232feb326b8c20081ccded47ea368fd/charset_normalizer-3.4.7-cp311-cp311-macosx_10_9_universal2.whl", hash = "sha256:7641bb8895e77f921102f72833904dcd9901df5d6d72a2ab8f31d04b7e51e4e7", size = 309705, upload-time = "2026-04-02T09:26:02.191Z" }, + { url = "https://files.pythonhosted.org/packages/5a/53/58c29116c340e5456724ecd2fff4196d236b98f3da97b404bc5e51ac3493/charset_normalizer-3.4.7-cp311-cp311-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:202389074300232baeb53ae2569a60901f7efadd4245cf3a3bf0617d60b439d7", size = 206419, upload-time = "2026-04-02T09:26:03.583Z" }, + { url = "https://files.pythonhosted.org/packages/b2/02/e8146dc6591a37a00e5144c63f29fb7c97a734ea8a111190783c0e60ab63/charset_normalizer-3.4.7-cp311-cp311-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:30b8d1d8c52a48c2c5690e152c169b673487a2a58de1ec7393196753063fcd5e", size = 227901, upload-time = "2026-04-02T09:26:04.738Z" }, + { url = "https://files.pythonhosted.org/packages/fb/73/77486c4cd58f1267bf17db420e930c9afa1b3be3fe8c8b8ebbebc9624359/charset_normalizer-3.4.7-cp311-cp311-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:532bc9bf33a68613fd7d65e4b1c71a6a38d7d42604ecf239c77392e9b4e8998c", size = 222742, upload-time = "2026-04-02T09:26:06.36Z" }, + { url = "https://files.pythonhosted.org/packages/a1/fa/f74eb381a7d94ded44739e9d94de18dc5edc9c17fb8c11f0a6890696c0a9/charset_normalizer-3.4.7-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:2fe249cb4651fd12605b7288b24751d8bfd46d35f12a20b1ba33dea122e690df", size = 214061, upload-time = "2026-04-02T09:26:08.347Z" }, + { url = "https://files.pythonhosted.org/packages/dc/92/42bd3cefcf7687253fb86694b45f37b733c97f59af3724f356fa92b8c344/charset_normalizer-3.4.7-cp311-cp311-manylinux_2_31_armv7l.whl", hash = "sha256:65bcd23054beab4d166035cabbc868a09c1a49d1efe458fe8e4361215df40265", size = 199239, upload-time = "2026-04-02T09:26:09.823Z" }, + { url = "https://files.pythonhosted.org/packages/4c/3d/069e7184e2aa3b3cddc700e3dd267413dc259854adc3380421c805c6a17d/charset_normalizer-3.4.7-cp311-cp311-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:08e721811161356f97b4059a9ba7bafb23ea5ee2255402c42881c214e173c6b4", size = 210173, upload-time = "2026-04-02T09:26:10.953Z" }, + { url = "https://files.pythonhosted.org/packages/62/51/9d56feb5f2e7074c46f93e0ebdbe61f0848ee246e2f0d89f8e20b89ebb8f/charset_normalizer-3.4.7-cp311-cp311-musllinux_1_2_aarch64.whl", hash = "sha256:e060d01aec0a910bdccb8be71faf34e7799ce36950f8294c8bf612cba65a2c9e", size = 209841, upload-time = "2026-04-02T09:26:12.142Z" }, + { url = "https://files.pythonhosted.org/packages/d2/59/893d8f99cc4c837dda1fe2f1139079703deb9f321aabcb032355de13b6c7/charset_normalizer-3.4.7-cp311-cp311-musllinux_1_2_armv7l.whl", hash = "sha256:38c0109396c4cfc574d502df99742a45c72c08eff0a36158b6f04000043dbf38", size = 200304, upload-time = "2026-04-02T09:26:13.711Z" }, + { url = "https://files.pythonhosted.org/packages/7d/1d/ee6f3be3464247578d1ed5c46de545ccc3d3ff933695395c402c21fa6b77/charset_normalizer-3.4.7-cp311-cp311-musllinux_1_2_ppc64le.whl", hash = "sha256:1c2a768fdd44ee4a9339a9b0b130049139b8ce3c01d2ce09f67f5a68048d477c", size = 229455, upload-time = "2026-04-02T09:26:14.941Z" }, + { url = "https://files.pythonhosted.org/packages/54/bb/8fb0a946296ea96a488928bdce8ef99023998c48e4713af533e9bb98ef07/charset_normalizer-3.4.7-cp311-cp311-musllinux_1_2_riscv64.whl", hash = "sha256:1a87ca9d5df6fe460483d9a5bbf2b18f620cbed41b432e2bddb686228282d10b", size = 210036, upload-time = "2026-04-02T09:26:16.478Z" }, + { url = "https://files.pythonhosted.org/packages/9a/bc/015b2387f913749f82afd4fcba07846d05b6d784dd16123cb66860e0237d/charset_normalizer-3.4.7-cp311-cp311-musllinux_1_2_s390x.whl", hash = "sha256:d635aab80466bc95771bb78d5370e74d36d1fe31467b6b29b8b57b2a3cd7d22c", size = 224739, upload-time = "2026-04-02T09:26:17.751Z" }, + { url = "https://files.pythonhosted.org/packages/17/ab/63133691f56baae417493cba6b7c641571a2130eb7bceba6773367ab9ec5/charset_normalizer-3.4.7-cp311-cp311-musllinux_1_2_x86_64.whl", hash = "sha256:ae196f021b5e7c78e918242d217db021ed2a6ace2bc6ae94c0fc596221c7f58d", size = 216277, upload-time = "2026-04-02T09:26:18.981Z" }, + { url = "https://files.pythonhosted.org/packages/06/6d/3be70e827977f20db77c12a97e6a9f973631a45b8d186c084527e53e77a4/charset_normalizer-3.4.7-cp311-cp311-win32.whl", hash = "sha256:adb2597b428735679446b46c8badf467b4ca5f5056aae4d51a19f9570301b1ad", size = 147819, upload-time = "2026-04-02T09:26:20.295Z" }, + { url = "https://files.pythonhosted.org/packages/20/d9/5f67790f06b735d7c7637171bbfd89882ad67201891b7275e51116ed8207/charset_normalizer-3.4.7-cp311-cp311-win_amd64.whl", hash = "sha256:8e385e4267ab76874ae30db04c627faaaf0b509e1ccc11a95b3fc3e83f855c00", size = 159281, upload-time = "2026-04-02T09:26:21.74Z" }, + { url = "https://files.pythonhosted.org/packages/ca/83/6413f36c5a34afead88ce6f66684d943d91f233d76dd083798f9602b75ae/charset_normalizer-3.4.7-cp311-cp311-win_arm64.whl", hash = "sha256:d4a48e5b3c2a489fae013b7589308a40146ee081f6f509e047e0e096084ceca1", size = 147843, upload-time = "2026-04-02T09:26:22.901Z" }, + { url = "https://files.pythonhosted.org/packages/0c/eb/4fc8d0a7110eb5fc9cc161723a34a8a6c200ce3b4fbf681bc86feee22308/charset_normalizer-3.4.7-cp312-cp312-macosx_10_13_universal2.whl", hash = "sha256:eca9705049ad3c7345d574e3510665cb2cf844c2f2dcfe675332677f081cbd46", size = 311328, upload-time = "2026-04-02T09:26:24.331Z" }, + { url = "https://files.pythonhosted.org/packages/f8/e3/0fadc706008ac9d7b9b5be6dc767c05f9d3e5df51744ce4cc9605de7b9f4/charset_normalizer-3.4.7-cp312-cp312-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:6178f72c5508bfc5fd446a5905e698c6212932f25bcdd4b47a757a50605a90e2", size = 208061, upload-time = "2026-04-02T09:26:25.568Z" }, + { url = "https://files.pythonhosted.org/packages/42/f0/3dd1045c47f4a4604df85ec18ad093912ae1344ac706993aff91d38773a2/charset_normalizer-3.4.7-cp312-cp312-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:e1421b502d83040e6d7fb2fb18dff63957f720da3d77b2fbd3187ceb63755d7b", size = 229031, upload-time = "2026-04-02T09:26:26.865Z" }, + { url = "https://files.pythonhosted.org/packages/dc/67/675a46eb016118a2fbde5a277a5d15f4f69d5f3f5f338e5ee2f8948fcf43/charset_normalizer-3.4.7-cp312-cp312-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:edac0f1ab77644605be2cbba52e6b7f630731fc42b34cb0f634be1a6eface56a", size = 225239, upload-time = "2026-04-02T09:26:28.044Z" }, + { url = "https://files.pythonhosted.org/packages/4b/f8/d0118a2f5f23b02cd166fa385c60f9b0d4f9194f574e2b31cef350ad7223/charset_normalizer-3.4.7-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:5649fd1c7bade02f320a462fdefd0b4bd3ce036065836d4f42e0de958038e116", size = 216589, upload-time = "2026-04-02T09:26:29.239Z" }, + { url = "https://files.pythonhosted.org/packages/b1/f1/6d2b0b261b6c4ceef0fcb0d17a01cc5bc53586c2d4796fa04b5c540bc13d/charset_normalizer-3.4.7-cp312-cp312-manylinux_2_31_armv7l.whl", hash = "sha256:203104ed3e428044fd943bc4bf45fa73c0730391f9621e37fe39ecf477b128cb", size = 202733, upload-time = "2026-04-02T09:26:30.5Z" }, + { url = "https://files.pythonhosted.org/packages/6f/c0/7b1f943f7e87cc3db9626ba17807d042c38645f0a1d4415c7a14afb5591f/charset_normalizer-3.4.7-cp312-cp312-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:298930cec56029e05497a76988377cbd7457ba864beeea92ad7e844fe74cd1f1", size = 212652, upload-time = "2026-04-02T09:26:31.709Z" }, + { url = "https://files.pythonhosted.org/packages/38/dd/5a9ab159fe45c6e72079398f277b7d2b523e7f716acc489726115a910097/charset_normalizer-3.4.7-cp312-cp312-musllinux_1_2_aarch64.whl", hash = "sha256:708838739abf24b2ceb208d0e22403dd018faeef86ddac04319a62ae884c4f15", size = 211229, upload-time = "2026-04-02T09:26:33.282Z" }, + { url = "https://files.pythonhosted.org/packages/d5/ff/531a1cad5ca855d1c1a8b69cb71abfd6d85c0291580146fda7c82857caa1/charset_normalizer-3.4.7-cp312-cp312-musllinux_1_2_armv7l.whl", hash = "sha256:0f7eb884681e3938906ed0434f20c63046eacd0111c4ba96f27b76084cd679f5", size = 203552, upload-time = "2026-04-02T09:26:34.845Z" }, + { url = "https://files.pythonhosted.org/packages/c1/4c/a5fb52d528a8ca41f7598cb619409ece30a169fbdf9cdce592e53b46c3a6/charset_normalizer-3.4.7-cp312-cp312-musllinux_1_2_ppc64le.whl", hash = "sha256:4dc1e73c36828f982bfe79fadf5919923f8a6f4df2860804db9a98c48824ce8d", size = 230806, upload-time = "2026-04-02T09:26:36.152Z" }, + { url = "https://files.pythonhosted.org/packages/59/7a/071feed8124111a32b316b33ae4de83d36923039ef8cf48120266844285b/charset_normalizer-3.4.7-cp312-cp312-musllinux_1_2_riscv64.whl", hash = "sha256:aed52fea0513bac0ccde438c188c8a471c4e0f457c2dd20cdbf6ea7a450046c7", size = 212316, upload-time = "2026-04-02T09:26:37.672Z" }, + { url = "https://files.pythonhosted.org/packages/fd/35/f7dba3994312d7ba508e041eaac39a36b120f32d4c8662b8814dab876431/charset_normalizer-3.4.7-cp312-cp312-musllinux_1_2_s390x.whl", hash = "sha256:fea24543955a6a729c45a73fe90e08c743f0b3334bbf3201e6c4bc1b0c7fa464", size = 227274, upload-time = "2026-04-02T09:26:38.93Z" }, + { url = "https://files.pythonhosted.org/packages/8a/2d/a572df5c9204ab7688ec1edc895a73ebded3b023bb07364710b05dd1c9be/charset_normalizer-3.4.7-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:bb6d88045545b26da47aa879dd4a89a71d1dce0f0e549b1abcb31dfe4a8eac49", size = 218468, upload-time = "2026-04-02T09:26:40.17Z" }, + { url = "https://files.pythonhosted.org/packages/86/eb/890922a8b03a568ca2f336c36585a4713c55d4d67bf0f0c78924be6315ca/charset_normalizer-3.4.7-cp312-cp312-win32.whl", hash = "sha256:2257141f39fe65a3fdf38aeccae4b953e5f3b3324f4ff0daf9f15b8518666a2c", size = 148460, upload-time = "2026-04-02T09:26:41.416Z" }, + { url = "https://files.pythonhosted.org/packages/35/d9/0e7dffa06c5ab081f75b1b786f0aefc88365825dfcd0ac544bdb7b2b6853/charset_normalizer-3.4.7-cp312-cp312-win_amd64.whl", hash = "sha256:5ed6ab538499c8644b8a3e18debabcd7ce684f3fa91cf867521a7a0279cab2d6", size = 159330, upload-time = "2026-04-02T09:26:42.554Z" }, + { url = "https://files.pythonhosted.org/packages/9e/5d/481bcc2a7c88ea6b0878c299547843b2521ccbc40980cb406267088bc701/charset_normalizer-3.4.7-cp312-cp312-win_arm64.whl", hash = "sha256:56be790f86bfb2c98fb742ce566dfb4816e5a83384616ab59c49e0604d49c51d", size = 147828, upload-time = "2026-04-02T09:26:44.075Z" }, + { url = "https://files.pythonhosted.org/packages/c1/3b/66777e39d3ae1ddc77ee606be4ec6d8cbd4c801f65e5a1b6f2b11b8346dd/charset_normalizer-3.4.7-cp313-cp313-macosx_10_13_universal2.whl", hash = "sha256:f496c9c3cc02230093d8330875c4c3cdfc3b73612a5fd921c65d39cbcef08063", size = 309627, upload-time = "2026-04-02T09:26:45.198Z" }, + { url = "https://files.pythonhosted.org/packages/2e/4e/b7f84e617b4854ade48a1b7915c8ccfadeba444d2a18c291f696e37f0d3b/charset_normalizer-3.4.7-cp313-cp313-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:0ea948db76d31190bf08bd371623927ee1339d5f2a0b4b1b4a4439a65298703c", size = 207008, upload-time = "2026-04-02T09:26:46.824Z" }, + { url = "https://files.pythonhosted.org/packages/c4/bb/ec73c0257c9e11b268f018f068f5d00aa0ef8c8b09f7753ebd5f2880e248/charset_normalizer-3.4.7-cp313-cp313-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:a277ab8928b9f299723bc1a2dabb1265911b1a76341f90a510368ca44ad9ab66", size = 228303, upload-time = "2026-04-02T09:26:48.397Z" }, + { url = "https://files.pythonhosted.org/packages/85/fb/32d1f5033484494619f701e719429c69b766bfc4dbc61aa9e9c8c166528b/charset_normalizer-3.4.7-cp313-cp313-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:3bec022aec2c514d9cf199522a802bd007cd588ab17ab2525f20f9c34d067c18", size = 224282, upload-time = "2026-04-02T09:26:49.684Z" }, + { url = "https://files.pythonhosted.org/packages/fa/07/330e3a0dda4c404d6da83b327270906e9654a24f6c546dc886a0eb0ffb23/charset_normalizer-3.4.7-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:e044c39e41b92c845bc815e5ae4230804e8e7bc29e399b0437d64222d92809dd", size = 215595, upload-time = "2026-04-02T09:26:50.915Z" }, + { url = "https://files.pythonhosted.org/packages/e3/7c/fc890655786e423f02556e0216d4b8c6bcb6bdfa890160dc66bf52dee468/charset_normalizer-3.4.7-cp313-cp313-manylinux_2_31_armv7l.whl", hash = "sha256:f495a1652cf3fbab2eb0639776dad966c2fb874d79d87ca07f9d5f059b8bd215", size = 201986, upload-time = "2026-04-02T09:26:52.197Z" }, + { url = "https://files.pythonhosted.org/packages/d8/97/bfb18b3db2aed3b90cf54dc292ad79fdd5ad65c4eae454099475cbeadd0d/charset_normalizer-3.4.7-cp313-cp313-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:e712b419df8ba5e42b226c510472b37bd57b38e897d3eca5e8cfd410a29fa859", size = 211711, upload-time = "2026-04-02T09:26:53.49Z" }, + { url = "https://files.pythonhosted.org/packages/6f/a5/a581c13798546a7fd557c82614a5c65a13df2157e9ad6373166d2a3e645d/charset_normalizer-3.4.7-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:7804338df6fcc08105c7745f1502ba68d900f45fd770d5bdd5288ddccb8a42d8", size = 210036, upload-time = "2026-04-02T09:26:54.975Z" }, + { url = "https://files.pythonhosted.org/packages/8c/bf/b3ab5bcb478e4193d517644b0fb2bf5497fbceeaa7a1bc0f4d5b50953861/charset_normalizer-3.4.7-cp313-cp313-musllinux_1_2_armv7l.whl", hash = "sha256:481551899c856c704d58119b5025793fa6730adda3571971af568f66d2424bb5", size = 202998, upload-time = "2026-04-02T09:26:56.303Z" }, + { url = "https://files.pythonhosted.org/packages/e7/4e/23efd79b65d314fa320ec6017b4b5834d5c12a58ba4610aa353af2e2f577/charset_normalizer-3.4.7-cp313-cp313-musllinux_1_2_ppc64le.whl", hash = "sha256:f59099f9b66f0d7145115e6f80dd8b1d847176df89b234a5a6b3f00437aa0832", size = 230056, upload-time = "2026-04-02T09:26:57.554Z" }, + { url = "https://files.pythonhosted.org/packages/b9/9f/1e1941bc3f0e01df116e68dc37a55c4d249df5e6fa77f008841aef68264f/charset_normalizer-3.4.7-cp313-cp313-musllinux_1_2_riscv64.whl", hash = "sha256:f59ad4c0e8f6bba240a9bb85504faa1ab438237199d4cce5f622761507b8f6a6", size = 211537, upload-time = "2026-04-02T09:26:58.843Z" }, + { url = "https://files.pythonhosted.org/packages/80/0f/088cbb3020d44428964a6c97fe1edfb1b9550396bf6d278330281e8b709c/charset_normalizer-3.4.7-cp313-cp313-musllinux_1_2_s390x.whl", hash = "sha256:3dedcc22d73ec993f42055eff4fcfed9318d1eeb9a6606c55892a26964964e48", size = 226176, upload-time = "2026-04-02T09:27:00.437Z" }, + { url = "https://files.pythonhosted.org/packages/6a/9f/130394f9bbe06f4f63e22641d32fc9b202b7e251c9aef4db044324dac493/charset_normalizer-3.4.7-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:64f02c6841d7d83f832cd97ccf8eb8a906d06eb95d5276069175c696b024b60a", size = 217723, upload-time = "2026-04-02T09:27:02.021Z" }, + { url = "https://files.pythonhosted.org/packages/73/55/c469897448a06e49f8fa03f6caae97074fde823f432a98f979cc42b90e69/charset_normalizer-3.4.7-cp313-cp313-win32.whl", hash = "sha256:4042d5c8f957e15221d423ba781e85d553722fc4113f523f2feb7b188cc34c5e", size = 148085, upload-time = "2026-04-02T09:27:03.192Z" }, + { url = "https://files.pythonhosted.org/packages/5d/78/1b74c5bbb3f99b77a1715c91b3e0b5bdb6fe302d95ace4f5b1bec37b0167/charset_normalizer-3.4.7-cp313-cp313-win_amd64.whl", hash = "sha256:3946fa46a0cf3e4c8cb1cc52f56bb536310d34f25f01ca9b6c16afa767dab110", size = 158819, upload-time = "2026-04-02T09:27:04.454Z" }, + { url = "https://files.pythonhosted.org/packages/68/86/46bd42279d323deb8687c4a5a811fd548cb7d1de10cf6535d099877a9a9f/charset_normalizer-3.4.7-cp313-cp313-win_arm64.whl", hash = "sha256:80d04837f55fc81da168b98de4f4b797ef007fc8a79ab71c6ec9bc4dd662b15b", size = 147915, upload-time = "2026-04-02T09:27:05.971Z" }, + { url = "https://files.pythonhosted.org/packages/97/c8/c67cb8c70e19ef1960b97b22ed2a1567711de46c4ddf19799923adc836c2/charset_normalizer-3.4.7-cp314-cp314-macosx_10_15_universal2.whl", hash = "sha256:c36c333c39be2dbca264d7803333c896ab8fa7d4d6f0ab7edb7dfd7aea6e98c0", size = 309234, upload-time = "2026-04-02T09:27:07.194Z" }, + { url = "https://files.pythonhosted.org/packages/99/85/c091fdee33f20de70d6c8b522743b6f831a2f1cd3ff86de4c6a827c48a76/charset_normalizer-3.4.7-cp314-cp314-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:1c2aed2e5e41f24ea8ef1590b8e848a79b56f3a5564a65ceec43c9d692dc7d8a", size = 208042, upload-time = "2026-04-02T09:27:08.749Z" }, + { url = "https://files.pythonhosted.org/packages/87/1c/ab2ce611b984d2fd5d86a5a8a19c1ae26acac6bad967da4967562c75114d/charset_normalizer-3.4.7-cp314-cp314-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:54523e136b8948060c0fa0bc7b1b50c32c186f2fceee897a495406bb6e311d2b", size = 228706, upload-time = "2026-04-02T09:27:09.951Z" }, + { url = "https://files.pythonhosted.org/packages/a8/29/2b1d2cb00bf085f59d29eb773ce58ec2d325430f8c216804a0a5cd83cbca/charset_normalizer-3.4.7-cp314-cp314-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:715479b9a2802ecac752a3b0efa2b0b60285cf962ee38414211abdfccc233b41", size = 224727, upload-time = "2026-04-02T09:27:11.175Z" }, + { url = "https://files.pythonhosted.org/packages/47/5c/032c2d5a07fe4d4855fea851209cca2b6f03ebeb6d4e3afdb3358386a684/charset_normalizer-3.4.7-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:bd6c2a1c7573c64738d716488d2cdd3c00e340e4835707d8fdb8dc1a66ef164e", size = 215882, upload-time = "2026-04-02T09:27:12.446Z" }, + { url = "https://files.pythonhosted.org/packages/2c/c2/356065d5a8b78ed04499cae5f339f091946a6a74f91e03476c33f0ab7100/charset_normalizer-3.4.7-cp314-cp314-manylinux_2_31_armv7l.whl", hash = "sha256:c45e9440fb78f8ddabcf714b68f936737a121355bf59f3907f4e17721b9d1aae", size = 200860, upload-time = "2026-04-02T09:27:13.721Z" }, + { url = "https://files.pythonhosted.org/packages/0c/cd/a32a84217ced5039f53b29f460962abb2d4420def55afabe45b1c3c7483d/charset_normalizer-3.4.7-cp314-cp314-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:3534e7dcbdcf757da6b85a0bbf5b6868786d5982dd959b065e65481644817a18", size = 211564, upload-time = "2026-04-02T09:27:15.272Z" }, + { url = "https://files.pythonhosted.org/packages/44/86/58e6f13ce26cc3b8f4a36b94a0f22ae2f00a72534520f4ae6857c4b81f89/charset_normalizer-3.4.7-cp314-cp314-musllinux_1_2_aarch64.whl", hash = "sha256:e8ac484bf18ce6975760921bb6148041faa8fef0547200386ea0b52b5d27bf7b", size = 211276, upload-time = "2026-04-02T09:27:16.834Z" }, + { url = "https://files.pythonhosted.org/packages/8f/fe/d17c32dc72e17e155e06883efa84514ca375f8a528ba2546bee73fc4df81/charset_normalizer-3.4.7-cp314-cp314-musllinux_1_2_armv7l.whl", hash = "sha256:a5fe03b42827c13cdccd08e6c0247b6a6d4b5e3cdc53fd1749f5896adcdc2356", size = 201238, upload-time = "2026-04-02T09:27:18.229Z" }, + { url = "https://files.pythonhosted.org/packages/6a/29/f33daa50b06525a237451cdb6c69da366c381a3dadcd833fa5676bc468b3/charset_normalizer-3.4.7-cp314-cp314-musllinux_1_2_ppc64le.whl", hash = "sha256:2d6eb928e13016cea4f1f21d1e10c1cebd5a421bc57ddf5b1142ae3f86824fab", size = 230189, upload-time = "2026-04-02T09:27:19.445Z" }, + { url = "https://files.pythonhosted.org/packages/b6/6e/52c84015394a6a0bdcd435210a7e944c5f94ea1055f5cc5d56c5fe368e7b/charset_normalizer-3.4.7-cp314-cp314-musllinux_1_2_riscv64.whl", hash = "sha256:e74327fb75de8986940def6e8dee4f127cc9752bee7355bb323cc5b2659b6d46", size = 211352, upload-time = "2026-04-02T09:27:20.79Z" }, + { url = "https://files.pythonhosted.org/packages/8c/d7/4353be581b373033fb9198bf1da3cf8f09c1082561e8e922aa7b39bf9fe8/charset_normalizer-3.4.7-cp314-cp314-musllinux_1_2_s390x.whl", hash = "sha256:d6038d37043bced98a66e68d3aa2b6a35505dc01328cd65217cefe82f25def44", size = 227024, upload-time = "2026-04-02T09:27:22.063Z" }, + { url = "https://files.pythonhosted.org/packages/30/45/99d18aa925bd1740098ccd3060e238e21115fffbfdcb8f3ece837d0ace6c/charset_normalizer-3.4.7-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:7579e913a5339fb8fa133f6bbcfd8e6749696206cf05acdbdca71a1b436d8e72", size = 217869, upload-time = "2026-04-02T09:27:23.486Z" }, + { url = "https://files.pythonhosted.org/packages/5c/05/5ee478aa53f4bb7996482153d4bfe1b89e0f087f0ab6b294fcf92d595873/charset_normalizer-3.4.7-cp314-cp314-win32.whl", hash = "sha256:5b77459df20e08151cd6f8b9ef8ef1f961ef73d85c21a555c7eed5b79410ec10", size = 148541, upload-time = "2026-04-02T09:27:25.146Z" }, + { url = "https://files.pythonhosted.org/packages/48/77/72dcb0921b2ce86420b2d79d454c7022bf5be40202a2a07906b9f2a35c97/charset_normalizer-3.4.7-cp314-cp314-win_amd64.whl", hash = "sha256:92a0a01ead5e668468e952e4238cccd7c537364eb7d851ab144ab6627dbbe12f", size = 159634, upload-time = "2026-04-02T09:27:26.642Z" }, + { url = "https://files.pythonhosted.org/packages/c6/a3/c2369911cd72f02386e4e340770f6e158c7980267da16af8f668217abaa0/charset_normalizer-3.4.7-cp314-cp314-win_arm64.whl", hash = "sha256:67f6279d125ca0046a7fd386d01b311c6363844deac3e5b069b514ba3e63c246", size = 148384, upload-time = "2026-04-02T09:27:28.271Z" }, + { url = "https://files.pythonhosted.org/packages/94/09/7e8a7f73d24dba1f0035fbbf014d2c36828fc1bf9c88f84093e57d315935/charset_normalizer-3.4.7-cp314-cp314t-macosx_10_15_universal2.whl", hash = "sha256:effc3f449787117233702311a1b7d8f59cba9ced946ba727bdc329ec69028e24", size = 330133, upload-time = "2026-04-02T09:27:29.474Z" }, + { url = "https://files.pythonhosted.org/packages/8d/da/96975ddb11f8e977f706f45cddd8540fd8242f71ecdb5d18a80723dcf62c/charset_normalizer-3.4.7-cp314-cp314t-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:fbccdc05410c9ee21bbf16a35f4c1d16123dcdeb8a1d38f33654fa21d0234f79", size = 216257, upload-time = "2026-04-02T09:27:30.793Z" }, + { url = "https://files.pythonhosted.org/packages/e5/e8/1d63bf8ef2d388e95c64b2098f45f84758f6d102a087552da1485912637b/charset_normalizer-3.4.7-cp314-cp314t-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:733784b6d6def852c814bce5f318d25da2ee65dd4839a0718641c696e09a2960", size = 234851, upload-time = "2026-04-02T09:27:32.44Z" }, + { url = "https://files.pythonhosted.org/packages/9b/40/e5ff04233e70da2681fa43969ad6f66ca5611d7e669be0246c4c7aaf6dc8/charset_normalizer-3.4.7-cp314-cp314t-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:a89c23ef8d2c6b27fd200a42aa4ac72786e7c60d40efdc76e6011260b6e949c4", size = 233393, upload-time = "2026-04-02T09:27:34.03Z" }, + { url = "https://files.pythonhosted.org/packages/be/c1/06c6c49d5a5450f76899992f1ee40b41d076aee9279b49cf9974d2f313d5/charset_normalizer-3.4.7-cp314-cp314t-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:6c114670c45346afedc0d947faf3c7f701051d2518b943679c8ff88befe14f8e", size = 223251, upload-time = "2026-04-02T09:27:35.369Z" }, + { url = "https://files.pythonhosted.org/packages/2b/9f/f2ff16fb050946169e3e1f82134d107e5d4ae72647ec8a1b1446c148480f/charset_normalizer-3.4.7-cp314-cp314t-manylinux_2_31_armv7l.whl", hash = "sha256:a180c5e59792af262bf263b21a3c49353f25945d8d9f70628e73de370d55e1e1", size = 206609, upload-time = "2026-04-02T09:27:36.661Z" }, + { url = "https://files.pythonhosted.org/packages/69/d5/a527c0cd8d64d2eab7459784fb4169a0ac76e5a6fc5237337982fd61347e/charset_normalizer-3.4.7-cp314-cp314t-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:3c9a494bc5ec77d43cea229c4f6db1e4d8fe7e1bbffa8b6f0f0032430ff8ab44", size = 220014, upload-time = "2026-04-02T09:27:38.019Z" }, + { url = "https://files.pythonhosted.org/packages/7e/80/8a7b8104a3e203074dc9aa2c613d4b726c0e136bad1cc734594b02867972/charset_normalizer-3.4.7-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:8d828b6667a32a728a1ad1d93957cdf37489c57b97ae6c4de2860fa749b8fc1e", size = 218979, upload-time = "2026-04-02T09:27:39.37Z" }, + { url = "https://files.pythonhosted.org/packages/02/9a/b759b503d507f375b2b5c153e4d2ee0a75aa215b7f2489cf314f4541f2c0/charset_normalizer-3.4.7-cp314-cp314t-musllinux_1_2_armv7l.whl", hash = "sha256:cf1493cd8607bec4d8a7b9b004e699fcf8f9103a9284cc94962cb73d20f9d4a3", size = 209238, upload-time = "2026-04-02T09:27:40.722Z" }, + { url = "https://files.pythonhosted.org/packages/c2/4e/0f3f5d47b86bdb79256e7290b26ac847a2832d9a4033f7eb2cd4bcf4bb5b/charset_normalizer-3.4.7-cp314-cp314t-musllinux_1_2_ppc64le.whl", hash = "sha256:0c96c3b819b5c3e9e165495db84d41914d6894d55181d2d108cc1a69bfc9cce0", size = 236110, upload-time = "2026-04-02T09:27:42.33Z" }, + { url = "https://files.pythonhosted.org/packages/96/23/bce28734eb3ed2c91dcf93abeb8a5cf393a7b2749725030bb630e554fdd8/charset_normalizer-3.4.7-cp314-cp314t-musllinux_1_2_riscv64.whl", hash = "sha256:752a45dc4a6934060b3b0dab47e04edc3326575f82be64bc4fc293914566503e", size = 219824, upload-time = "2026-04-02T09:27:43.924Z" }, + { url = "https://files.pythonhosted.org/packages/2c/6f/6e897c6984cc4d41af319b077f2f600fc8214eb2fe2d6bcb79141b882400/charset_normalizer-3.4.7-cp314-cp314t-musllinux_1_2_s390x.whl", hash = "sha256:8778f0c7a52e56f75d12dae53ae320fae900a8b9b4164b981b9c5ce059cd1fcb", size = 233103, upload-time = "2026-04-02T09:27:45.348Z" }, + { url = "https://files.pythonhosted.org/packages/76/22/ef7bd0fe480a0ae9b656189ec00744b60933f68b4f42a7bb06589f6f576a/charset_normalizer-3.4.7-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:ce3412fbe1e31eb81ea42f4169ed94861c56e643189e1e75f0041f3fe7020abe", size = 225194, upload-time = "2026-04-02T09:27:46.706Z" }, + { url = "https://files.pythonhosted.org/packages/c5/a7/0e0ab3e0b5bc1219bd80a6a0d4d72ca74d9250cb2382b7c699c147e06017/charset_normalizer-3.4.7-cp314-cp314t-win32.whl", hash = "sha256:c03a41a8784091e67a39648f70c5f97b5b6a37f216896d44d2cdcb82615339a0", size = 159827, upload-time = "2026-04-02T09:27:48.053Z" }, + { url = "https://files.pythonhosted.org/packages/7a/1d/29d32e0fb40864b1f878c7f5a0b343ae676c6e2b271a2d55cc3a152391da/charset_normalizer-3.4.7-cp314-cp314t-win_amd64.whl", hash = "sha256:03853ed82eeebbce3c2abfdbc98c96dc205f32a79627688ac9a27370ea61a49c", size = 174168, upload-time = "2026-04-02T09:27:49.795Z" }, + { url = "https://files.pythonhosted.org/packages/de/32/d92444ad05c7a6e41fb2036749777c163baf7a0301a040cb672d6b2b1ae9/charset_normalizer-3.4.7-cp314-cp314t-win_arm64.whl", hash = "sha256:c35abb8bfff0185efac5878da64c45dafd2b37fb0383add1be155a763c1f083d", size = 153018, upload-time = "2026-04-02T09:27:51.116Z" }, + { url = "https://files.pythonhosted.org/packages/01/1b/ef725f8eb19b5a261b30f78efa9252ef9d017985cb499102f6f49834cd12/charset_normalizer-3.4.7-cp39-cp39-macosx_10_9_universal2.whl", hash = "sha256:177a0ba5f0211d488e295aaf82707237e331c24788d8d76c96c5a41594723217", size = 299121, upload-time = "2026-04-02T09:28:14.372Z" }, + { url = "https://files.pythonhosted.org/packages/a3/22/2f12878fbc680fbbb52386cd39a379801f62eaca74fc8b323381325f0f04/charset_normalizer-3.4.7-cp39-cp39-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:6e0d51f618228538a3e8f46bd246f87a6cd030565e015803691603f55e12afb5", size = 200612, upload-time = "2026-04-02T09:28:16.162Z" }, + { url = "https://files.pythonhosted.org/packages/bc/b6/10c84e789126ca97d4a7228863a30481e786980a8b8cfcbf4f30658ca63c/charset_normalizer-3.4.7-cp39-cp39-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:14265bfe1f09498b9d8ec91e9ec9fa52775edf90fcbde092b25f4a33d444fea9", size = 221041, upload-time = "2026-04-02T09:28:17.554Z" }, + { url = "https://files.pythonhosted.org/packages/21/7b/c414866a138400b2e81973d006da7f694cfeaf895ef07d2cba9a8743841a/charset_normalizer-3.4.7-cp39-cp39-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:87fad7d9ba98c86bcb41b2dc8dbb326619be2562af1f8ff50776a39e55721c5a", size = 216323, upload-time = "2026-04-02T09:28:18.863Z" }, + { url = "https://files.pythonhosted.org/packages/2e/92/bdcf94997e06b223d826df3abed45a5ad6e17f609b7df9d25cd23b5bde30/charset_normalizer-3.4.7-cp39-cp39-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:f22dec1690b584cea26fade98b2435c132c1b5f68e39f5a0b7627cd7ae31f1dc", size = 208419, upload-time = "2026-04-02T09:28:20.332Z" }, + { url = "https://files.pythonhosted.org/packages/1a/64/3f9142293c88b1b10e199649ed1330f070c2a68e305335a5819fa7f25fa7/charset_normalizer-3.4.7-cp39-cp39-manylinux_2_31_armv7l.whl", hash = "sha256:d61f00a0869d77422d9b2aba989e2d24afa6ffd552af442e0e58de4f35ea6d00", size = 195016, upload-time = "2026-04-02T09:28:21.657Z" }, + { url = "https://files.pythonhosted.org/packages/c1/d1/d8a6b7dd5c5636b76ce0d080bc57d8e56c7bbd6bc2ac941529a35e41d84a/charset_normalizer-3.4.7-cp39-cp39-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:6370e8686f662e6a3941ee48ed4742317cafbe5707e36406e9df792cdb535776", size = 206115, upload-time = "2026-04-02T09:28:23.259Z" }, + { url = "https://files.pythonhosted.org/packages/dd/8c/60ebe912379627d023eb96995b40bc50308729f210f43d66109ca0a7bbd2/charset_normalizer-3.4.7-cp39-cp39-musllinux_1_2_aarch64.whl", hash = "sha256:a6c5863edfbe888d9eff9c8b8087354e27618d9da76425c119293f11712a6319", size = 204022, upload-time = "2026-04-02T09:28:24.779Z" }, + { url = "https://files.pythonhosted.org/packages/d5/2a/41816ceda78a551cbfdfbeab6f3891152b0e3f758ce6580c2c18c829f774/charset_normalizer-3.4.7-cp39-cp39-musllinux_1_2_armv7l.whl", hash = "sha256:ed065083d0898c9d5b4bbec7b026fd755ff7454e6e8b73a67f8c744b13986e24", size = 195914, upload-time = "2026-04-02T09:28:26.181Z" }, + { url = "https://files.pythonhosted.org/packages/8f/9b/7c7f4b7f11525fcbdfba752455314ac60646bae91cdd671d531c1f7a97c6/charset_normalizer-3.4.7-cp39-cp39-musllinux_1_2_ppc64le.whl", hash = "sha256:2cd4a60d0e2fb04537162c62bbbb4182f53541fe0ede35cdf270a1c1e723cc42", size = 222159, upload-time = "2026-04-02T09:28:27.504Z" }, + { url = "https://files.pythonhosted.org/packages/9f/57/301682e7469bdbfa2ce219a804f0668b2266ab8520570d85d3b3ef483ea3/charset_normalizer-3.4.7-cp39-cp39-musllinux_1_2_riscv64.whl", hash = "sha256:813c0e0132266c08eb87469a642cb30aaff57c5f426255419572aaeceeaa7bf4", size = 206154, upload-time = "2026-04-02T09:28:28.848Z" }, + { url = "https://files.pythonhosted.org/packages/20/ec/90339ff5cdc598b265748c1f231c7d7fbd9123a92cee10f757e0b1448de4/charset_normalizer-3.4.7-cp39-cp39-musllinux_1_2_s390x.whl", hash = "sha256:07d9e39b01743c3717745f4c530a6349eadbfa043c7577eef86c502c15df2c67", size = 217423, upload-time = "2026-04-02T09:28:30.248Z" }, + { url = "https://files.pythonhosted.org/packages/2e/e7/a7a6147f8e3375676309cf584b25c72a3bab784ea4085b0011fa07b23aeb/charset_normalizer-3.4.7-cp39-cp39-musllinux_1_2_x86_64.whl", hash = "sha256:c0f081d69a6e58272819b70288d3221a6ee64b98df852631c80f293514d3b274", size = 210604, upload-time = "2026-04-02T09:28:31.736Z" }, + { url = "https://files.pythonhosted.org/packages/1a/62/d9340c7a79c393e57807d7fb6c57e82060687891f81b74d3201958b919c1/charset_normalizer-3.4.7-cp39-cp39-win32.whl", hash = "sha256:8751d2787c9131302398b11e6c8068053dcb55d5a8964e114b6e196cf16cb366", size = 144631, upload-time = "2026-04-02T09:28:33.158Z" }, + { url = "https://files.pythonhosted.org/packages/21/e7/92901117e2ddc8facfe8235a3ecd4eb482185b2ad5d5b6606b37c1afea06/charset_normalizer-3.4.7-cp39-cp39-win_amd64.whl", hash = "sha256:12a6fff75f6bc66711b73a2f0addfc4c8c15a20e805146a02d147a318962c444", size = 154710, upload-time = "2026-04-02T09:28:34.557Z" }, + { url = "https://files.pythonhosted.org/packages/cc/4f/e1fb138201ad9a32499dd9a98aa4a5a5441fbf7f56b52b619a54b7ee8777/charset_normalizer-3.4.7-cp39-cp39-win_arm64.whl", hash = "sha256:bb8cc7534f51d9a017b93e3e85b260924f909601c3df002bcdb58ddb4dc41a5c", size = 143716, upload-time = "2026-04-02T09:28:35.908Z" }, + { url = "https://files.pythonhosted.org/packages/db/8f/61959034484a4a7c527811f4721e75d02d653a35afb0b6054474d8185d4c/charset_normalizer-3.4.7-py3-none-any.whl", hash = "sha256:3dce51d0f5e7951f8bb4900c257dad282f49190fdbebecd4ba99bcc41fef404d", size = 61958, upload-time = "2026-04-02T09:28:37.794Z" }, +] + +[[package]] +name = "idna" +version = "3.15" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/82/77/7b3966d0b9d1d31a36ddf1746926a11dface89a83409bf1483f0237aa758/idna-3.15.tar.gz", hash = "sha256:ca962446ea538f7092a95e057da437618e886f4d349216d2b1e294abfdb65fdc", size = 199245, upload-time = "2026-05-12T22:45:57.011Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/d2/23/408243171aa9aaba178d3e2559159c24c1171a641aa83b67bdd3394ead8e/idna-3.15-py3-none-any.whl", hash = "sha256:048adeaf8c2d788c40fee287673ccaa74c24ffd8dcf09ffa555a2fbb59f10ac8", size = 72340, upload-time = "2026-05-12T22:45:55.733Z" }, +] + +[[package]] +name = "requests" +version = "2.32.5" +source = { registry = "https://pypi.org/simple" } +resolution-markers = [ + "python_full_version < '3.10'", +] +dependencies = [ + { name = "certifi", marker = "python_full_version < '3.10'" }, + { name = "charset-normalizer", marker = "python_full_version < '3.10'" }, + { name = "idna", marker = "python_full_version < '3.10'" }, + { name = "urllib3", version = "2.6.3", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.10'" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/c9/74/b3ff8e6c8446842c3f5c837e9c3dfcfe2018ea6ecef224c710c85ef728f4/requests-2.32.5.tar.gz", hash = "sha256:dbba0bac56e100853db0ea71b82b4dfd5fe2bf6d3754a8893c3af500cec7d7cf", size = 134517, upload-time = "2025-08-18T20:46:02.573Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/1e/db/4254e3eabe8020b458f1a747140d32277ec7a271daf1d235b70dc0b4e6e3/requests-2.32.5-py3-none-any.whl", hash = "sha256:2462f94637a34fd532264295e186976db0f5d453d1cdd31473c85a6a161affb6", size = 64738, upload-time = "2025-08-18T20:46:00.542Z" }, +] + +[[package]] +name = "requests" +version = "2.34.2" +source = { registry = "https://pypi.org/simple" } +resolution-markers = [ + "python_full_version >= '3.10'", +] +dependencies = [ + { name = "certifi", marker = "python_full_version >= '3.10'" }, + { name = "charset-normalizer", marker = "python_full_version >= '3.10'" }, + { name = "idna", marker = "python_full_version >= '3.10'" }, + { name = "urllib3", version = "2.7.0", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version >= '3.10'" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/ac/c3/e2a2b89f2d3e2179abd6d00ebd70bff6273f37fb3e0cc209f48b39d00cbf/requests-2.34.2.tar.gz", hash = "sha256:f288924cae4e29463698d6d60bc6a4da69c89185ad1e0bcc4104f584e960b9ed", size = 142856, upload-time = "2026-05-14T19:25:27.735Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/a0/f4/c67b0b3f1b9245e8d266f0f112c500d50e5b4e83cb6f3b71b6528104182a/requests-2.34.2-py3-none-any.whl", hash = "sha256:2a0d60c172f83ac6ab31e4554906c0f3b3588d37b5cb939b1c061f4907e278e0", size = 73075, upload-time = "2026-05-14T19:25:26.443Z" }, +] + +[[package]] +name = "test-project" +version = "0.0.1" +source = { virtual = "." } +dependencies = [ + { name = "requests", version = "2.32.5", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.10'" }, + { name = "requests", version = "2.34.2", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version >= '3.10'" }, +] + +[package.metadata] +requires-dist = [{ name = "requests" }] + +[[package]] +name = "urllib3" +version = "2.6.3" +source = { registry = "https://pypi.org/simple" } +resolution-markers = [ + "python_full_version < '3.10'", +] +sdist = { url = "https://files.pythonhosted.org/packages/c7/24/5f1b3bdffd70275f6661c76461e25f024d5a38a46f04aaca912426a2b1d3/urllib3-2.6.3.tar.gz", hash = "sha256:1b62b6884944a57dbe321509ab94fd4d3b307075e0c2eae991ac71ee15ad38ed", size = 435556, upload-time = "2026-01-07T16:24:43.925Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/39/08/aaaad47bc4e9dc8c725e68f9d04865dbcb2052843ff09c97b08904852d84/urllib3-2.6.3-py3-none-any.whl", hash = "sha256:bf272323e553dfb2e87d9bfd225ca7b0f467b919d7bbd355436d3fd37cb0acd4", size = 131584, upload-time = "2026-01-07T16:24:42.685Z" }, +] + +[[package]] +name = "urllib3" +version = "2.7.0" +source = { registry = "https://pypi.org/simple" } +resolution-markers = [ + "python_full_version >= '3.10'", +] +sdist = { url = "https://files.pythonhosted.org/packages/53/0c/06f8b233b8fd13b9e5ee11424ef85419ba0d8ba0b3138bf360be2ff56953/urllib3-2.7.0.tar.gz", hash = "sha256:231e0ec3b63ceb14667c67be60f2f2c40a518cb38b03af60abc813da26505f4c", size = 433602, upload-time = "2026-05-07T16:13:18.596Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/7f/3e/5db95bcf282c52709639744ca2a8b149baccf648e39c8cc87553df9eae0c/urllib3-2.7.0-py3-none-any.whl", hash = "sha256:9fb4c81ebbb1ce9531cce37674bbc6f1360472bc18ca9a553ede278ef7276897", size = 131087, upload-time = "2026-05-07T16:13:17.151Z" }, +] diff --git a/tests/uv/lock/uv_lock_to_requirements_tests.bzl b/tests/uv/lock/uv_lock_to_requirements_tests.bzl new file mode 100644 index 0000000000..1b947877c9 --- /dev/null +++ b/tests/uv/lock/uv_lock_to_requirements_tests.bzl @@ -0,0 +1,289 @@ +"""Tests for uv_lock_to_requirements.""" + +load("@rules_testing//lib:test_suite.bzl", "test_suite") +load("//python/uv/private:uv_lock_to_requirements.bzl", "uv_lock_to_requirements") # buildifier: disable=bzl-visibility + +_tests = [] + +def _test_empty(env): + got = uv_lock_to_requirements(json.decode("""{"package":[]}""")) + env.expect.that_str(got).equals("") + +_tests.append(_test_empty) + +def _test_simple_package(env): + got = uv_lock_to_requirements(json.decode("""{"package":[ + {"name":"foo","version":"0.0.1","source":{"registry":"https://pypi.org/simple"},"wheels":[{"hash":"sha256:deadbeef","url":"https://example.org/foo.whl"}]} + ]}""")) + env.expect.that_str(got).equals("""\ +foo==0.0.1 \\ + --hash=sha256:deadbeef +""") + +_tests.append(_test_simple_package) + +def _test_package_with_markers(env): + got = uv_lock_to_requirements(json.decode("""{"package":[ + {"name":"foo","version":"0.0.1","source":{"registry":"https://pypi.org/simple"},"resolution-markers":["python_full_version < '3.10'"],"wheels":[{"hash":"sha256:deadbeef","url":"https://example.org/foo.whl"}]} + ]}""")) + env.expect.that_str(got).equals("""\ +foo==0.0.1 ; python_full_version < '3.10' \\ + --hash=sha256:deadbeef +""") + +_tests.append(_test_package_with_markers) + +def _test_package_with_multiple_markers(env): + got = uv_lock_to_requirements(json.decode("""{"package":[ + {"name":"foo","version":"0.0.1","source":{"registry":"https://pypi.org/simple"},"resolution-markers":["python_full_version == '3.10.*'","python_full_version >= '3.14'"],"wheels":[{"hash":"sha256:deadbeef","url":"https://example.org/foo.whl"}]} + ]}""")) + env.expect.that_str(got).equals("""\ +foo==0.0.1 ; python_full_version == '3.10.*' or python_full_version >= '3.14' \\ + --hash=sha256:deadbeef +""") + +_tests.append(_test_package_with_multiple_markers) + +def _test_package_with_deps(env): + got = uv_lock_to_requirements(json.decode("""{"package":[ + {"name":"bar","version":"0.0.1","source":{"registry":"https://pypi.org/simple"},"wheels":[{"hash":"sha256:baadbeef","url":"https://example.org/bar.whl"}]}, + {"name":"foo","version":"0.0.1","source":{"registry":"https://pypi.org/simple"},"dependencies":[{"name":"bar"}],"wheels":[{"hash":"sha256:deadbeef","url":"https://example.org/foo.whl"}]} + ]}""")) + env.expect.that_str(got).equals("""\ +bar==0.0.1 \\ + --hash=sha256:baadbeef + # via foo + +foo==0.0.1 \\ + --hash=sha256:deadbeef +""") + +_tests.append(_test_package_with_deps) + +def _test_package_with_optional_deps(env): + got = uv_lock_to_requirements(json.decode("""{"package":[ + {"name":"bar","version":"0.0.1","source":{"registry":"https://pypi.org/simple"},"wheels":[{"hash":"sha256:baadbeef","url":"https://example.org/bar.whl"}]}, + {"name":"foo","version":"0.0.1","source":{"registry":"https://pypi.org/simple"},"optional-dependencies":{"extra1":[{"name":"bar"}]},"wheels":[{"hash":"sha256:deadbeef","url":"https://example.org/foo.whl"}]} + ]}""")) + env.expect.that_str(got).equals("""\ +bar==0.0.1 \\ + --hash=sha256:baadbeef + # via foo + +foo[extra1]==0.0.1 \\ + --hash=sha256:deadbeef +""") + +_tests.append(_test_package_with_optional_deps) + +def _test_self_edge_excluded(env): + got = uv_lock_to_requirements(json.decode("""{"package":[ + {"name":"pydantic","version":"2.0.0","source":{"registry":"https://pypi.org/simple"},"dependencies":[{"name":"pydantic","extra":["email"]}],"wheels":[{"hash":"sha256:deadbeef","url":"https://example.org/pydantic.whl"}]} + ]}""")) + env.expect.that_str(got).equals("""\ +pydantic==2.0.0 \\ + --hash=sha256:deadbeef +""") + +_tests.append(_test_self_edge_excluded) + +def _test_multiple_dependents(env): + got = uv_lock_to_requirements(json.decode("""{"package":[ + {"name":"common","version":"1.0.0","source":{"registry":"https://pypi.org/simple"},"wheels":[{"hash":"sha256:aaaa","url":"https://example.org/common.whl"}]}, + {"name":"pkg_a","version":"0.1.0","source":{"registry":"https://pypi.org/simple"},"dependencies":[{"name":"common"}],"wheels":[{"hash":"sha256:bbbb","url":"https://example.org/a.whl"}]}, + {"name":"pkg_b","version":"0.2.0","source":{"registry":"https://pypi.org/simple"},"dependencies":[{"name":"common"}],"wheels":[{"hash":"sha256:cccc","url":"https://example.org/b.whl"}]} + ]}""")) + env.expect.that_str(got).equals("""\ +common==1.0.0 \\ + --hash=sha256:aaaa + # via + # pkg_a + # pkg_b + +pkg_a==0.1.0 \\ + --hash=sha256:bbbb + +pkg_b==0.2.0 \\ + --hash=sha256:cccc +""") + +_tests.append(_test_multiple_dependents) + +def _test_git_source_skipped(env): + got = uv_lock_to_requirements(json.decode("""{"package":[ + {"name":"foo","version":"0.1.0","source":{"git":"https://github.com/org/foo.git"}}, + {"name":"bar","version":"0.0.1","source":{"registry":"https://pypi.org/simple"},"wheels":[{"hash":"sha256:deadbeef","url":"https://example.org/bar.whl"}]} + ]}""")) + env.expect.that_str(got).equals("""\ +bar==0.0.1 \\ + --hash=sha256:deadbeef +""") + +_tests.append(_test_git_source_skipped) + +def _test_virtual_source_skipped(env): + got = uv_lock_to_requirements(json.decode("""{"package":[ + {"name":"virtual-pkg","version":"0.0.0","source":{"virtual":true}}, + {"name":"foo","version":"0.0.1","source":{"registry":"https://pypi.org/simple"},"wheels":[{"hash":"sha256:deadbeef","url":"https://example.org/foo.whl"}]} + ]}""")) + env.expect.that_str(got).equals("""\ +foo==0.0.1 \\ + --hash=sha256:deadbeef +""") + +_tests.append(_test_virtual_source_skipped) + +def _test_sdist_hash(env): + got = uv_lock_to_requirements(json.decode("""{"package":[ + {"name":"bar","version":"0.0.1","source":{"registry":"https://pypi.org/simple"},"sdist":{"hash":"sha256:deadb00f","url":"https://example.org/bar.tar.gz"}} + ]}""")) + env.expect.that_str(got).equals("""\ +bar==0.0.1 \\ + --hash=sha256:deadb00f +""") + +_tests.append(_test_sdist_hash) + +def _test_wheel_and_sdist_hashes(env): + got = uv_lock_to_requirements(json.decode("""{"package":[ + {"name":"foo","version":"0.0.1","source":{"registry":"https://pypi.org/simple"},"sdist":{"hash":"sha256:feedcafe","url":"https://example.org/foo.tar.gz"},"wheels":[{"hash":"sha256:deadbeef","url":"https://example.org/foo.whl"}]} + ]}""")) + env.expect.that_str(got).equals("""\ +foo==0.0.1 \\ + --hash=sha256:deadbeef \\ + --hash=sha256:feedcafe +""") + +_tests.append(_test_wheel_and_sdist_hashes) + +def _test_multiple_versions_same_package(env): + got = uv_lock_to_requirements(json.decode("""{"package":[ + {"name":"foo","version":"0.0.1","source":{"registry":"https://pypi.org/simple"},"resolution-markers":["python_full_version < '3.10'"],"wheels":[{"hash":"sha256:deadbeef","url":"https://example.org/foo-0.0.1.whl"}]}, + {"name":"foo","version":"0.0.2","source":{"registry":"https://pypi.org/simple"},"resolution-markers":["python_full_version >= '3.10'"],"wheels":[{"hash":"sha256:deadb11f","url":"https://example.org/foo-0.0.2.whl"}]} + ]}""")) + env.expect.that_str(got).equals("""\ +foo==0.0.1 ; python_full_version < '3.10' \\ + --hash=sha256:deadbeef + +foo==0.0.2 ; python_full_version >= '3.10' \\ + --hash=sha256:deadb11f +""") + +_tests.append(_test_multiple_versions_same_package) + +def _test_dep_with_extras(env): + got = uv_lock_to_requirements(json.decode("""{"package":[ + {"name":"foo","version":"0.0.1","source":{"registry":"https://pypi.org/simple"},"wheels":[{"hash":"sha256:deadbeef","url":"https://example.org/foo.whl"}]}, + {"name":"bar","version":"0.0.2","source":{"registry":"https://pypi.org/simple"},"dependencies":[{"name":"foo","extra":["extra1"]}],"wheels":[{"hash":"sha256:baadbeef","url":"https://example.org/bar.whl"}]} + ]}""")) + env.expect.that_str(got).equals("""\ +foo[extra1]==0.0.1 \\ + --hash=sha256:deadbeef + # via bar + +bar==0.0.2 \\ + --hash=sha256:baadbeef +""") + +_tests.append(_test_dep_with_extras) + +def _test_multiple_hashes_from_wheels(env): + got = uv_lock_to_requirements(json.decode("""{"package":[ + {"name":"foo","version":"0.0.1","source":{"registry":"https://pypi.org/simple"},"wheels":[ + {"hash":"sha256:aaaa","url":"https://example.org/foo-0.0.1-cp39.whl"}, + {"hash":"sha256:bbbb","url":"https://example.org/foo-0.0.1-py3-none-any.whl"} + ]} + ]}""")) + env.expect.that_str(got).equals("""\ +foo==0.0.1 \\ + --hash=sha256:aaaa \\ + --hash=sha256:bbbb +""") + +_tests.append(_test_multiple_hashes_from_wheels) + +def _test_package_no_hashes_no_deps(env): + got = uv_lock_to_requirements(json.decode("""{"package":[ + {"name":"foo","version":"0.0.1","source":{"registry":"https://pypi.org/simple"}} + ]}""")) + env.expect.that_str(got).equals("""\ +foo==0.0.1 +""") + +_tests.append(_test_package_no_hashes_no_deps) + +def _test_package_with_multiple_optional_deps(env): + got = uv_lock_to_requirements(json.decode("""{"package":[ + {"name":"bar","version":"0.0.1","source":{"registry":"https://pypi.org/simple"},"wheels":[{"hash":"sha256:baadbeef","url":"https://example.org/bar.whl"}]}, + {"name":"baz","version":"0.0.2","source":{"registry":"https://pypi.org/simple"},"wheels":[{"hash":"sha256:deadc0de","url":"https://example.org/baz.whl"}]}, + {"name":"foo","version":"0.0.3","source":{"registry":"https://pypi.org/simple"},"optional-dependencies":{"extra1":[{"name":"bar"}],"extra2":[{"name":"baz"}]},"wheels":[{"hash":"sha256:feedcafe","url":"https://example.org/foo.whl"}]} + ]}""")) + env.expect.that_str(got).equals("""\ +bar==0.0.1 \\ + --hash=sha256:baadbeef + # via foo + +baz==0.0.2 \\ + --hash=sha256:deadc0de + # via foo + +foo[extra1,extra2]==0.0.3 \\ + --hash=sha256:feedcafe +""") + +_tests.append(_test_package_with_multiple_optional_deps) + +def _test_dep_with_multiple_extras(env): + got = uv_lock_to_requirements(json.decode("""{"package":[ + {"name":"foo","version":"0.0.1","source":{"registry":"https://pypi.org/simple"},"wheels":[{"hash":"sha256:deadbeef","url":"https://example.org/foo.whl"}]}, + {"name":"bar","version":"0.0.2","source":{"registry":"https://pypi.org/simple"},"dependencies":[{"name":"foo","extra":["extra1","extra2"]}],"wheels":[{"hash":"sha256:baadbeef","url":"https://example.org/bar.whl"}]} + ]}""")) + env.expect.that_str(got).equals("""\ +foo[extra1,extra2]==0.0.1 \\ + --hash=sha256:deadbeef + # via bar + +bar==0.0.2 \\ + --hash=sha256:baadbeef +""") + +_tests.append(_test_dep_with_multiple_extras) + +def _test_extras_from_multiple_dependents(env): + got = uv_lock_to_requirements(json.decode("""{"package":[ + {"name":"common","version":"1.0.0","source":{"registry":"https://pypi.org/simple"},"wheels":[{"hash":"sha256:aaaa","url":"https://example.org/common.whl"}]}, + {"name":"pkg_a","version":"0.1.0","source":{"registry":"https://pypi.org/simple"},"dependencies":[{"name":"common","extra":["extra1"]}],"wheels":[{"hash":"sha256:bbbb","url":"https://example.org/a.whl"}]}, + {"name":"pkg_b","version":"0.2.0","source":{"registry":"https://pypi.org/simple"},"dependencies":[{"name":"common","extra":["extra2"]}],"wheels":[{"hash":"sha256:cccc","url":"https://example.org/b.whl"}]} + ]}""")) + env.expect.that_str(got).equals("""\ +common[extra1,extra2]==1.0.0 \\ + --hash=sha256:aaaa + # via + # pkg_a + # pkg_b + +pkg_a==0.1.0 \\ + --hash=sha256:bbbb + +pkg_b==0.2.0 \\ + --hash=sha256:cccc +""") + +_tests.append(_test_extras_from_multiple_dependents) + +def _test_requires_dist_extras(env): + """Test that extras from metadata.requires-dist are included.""" + got = uv_lock_to_requirements(json.decode("""{"package":[ + {"name":"pytest-bazel","version":"0.1.6","source":{"registry":"https://pypi.org/simple"},"wheels":[{"hash":"sha256:a29e80e1d67c3db801bdd4d0b6b742f2bfb48cd6841caa33401458e5c4e29c21","url":"https://example.org/pytest_bazel-0.1.6.whl"}]}, + {"name":"root-pkg","version":"0.0.0","source":{"virtual":"."},"dependencies":[{"name":"pytest-bazel"}],"metadata":{"requires-dist":[{"name":"pytest-bazel","extras":["all"]}]}} + ]}""")) + env.expect.that_str(got).equals("""\ +pytest-bazel[all]==0.1.6 \\ + --hash=sha256:a29e80e1d67c3db801bdd4d0b6b742f2bfb48cd6841caa33401458e5c4e29c21 + # via root-pkg +""") + +_tests.append(_test_requires_dist_extras) + +def uv_lock_to_requirements_test_suite(name): + test_suite(name = name, basic_tests = _tests) From ee53e46d38927fbccfc3436bb8cf19ad2ec033f3 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Fri, 26 Jun 2026 00:49:09 -0700 Subject: [PATCH 782/922] chore: add agent rule to base new branches on upstream/main (#3849) Add a workspace-specific agent rule to ensure that new conversations/sessions and branches are always based on the latest upstream code to avoid using outdated code states. This comes about after having several sessions go haywire at the start because they didn't see the latest changes. --- .agents/rules/workspace.md | 18 +++++++++++++++++ .agents/scripts/setup_triangle_branch.sh | 25 ++++++++++++++++++++++++ 2 files changed, 43 insertions(+) create mode 100644 .agents/rules/workspace.md create mode 100755 .agents/scripts/setup_triangle_branch.sh diff --git a/.agents/rules/workspace.md b/.agents/rules/workspace.md new file mode 100644 index 0000000000..b819925f83 --- /dev/null +++ b/.agents/rules/workspace.md @@ -0,0 +1,18 @@ +--- +trigger: always_on +--- + +# Workspace Rules + +To avoid confusion from using outdated code states, when starting a new +conversation/session or when first starting a new branch or worktree, unless +explicitly instructed otherwise, ensure the latest upstream code is used as the +basis: +* Fetch `upstream/main` (`git fetch upstream main`). +* Base any new branch or worktree upon `upstream/main` (e.g., + `git checkout -b upstream/main`). +* Run the workspace helper script to configure upstream tracking and safe + pushing: + ```bash + .agents/scripts/setup_triangle_branch.sh + ``` diff --git a/.agents/scripts/setup_triangle_branch.sh b/.agents/scripts/setup_triangle_branch.sh new file mode 100755 index 0000000000..b6bdc2bb20 --- /dev/null +++ b/.agents/scripts/setup_triangle_branch.sh @@ -0,0 +1,25 @@ +#!/bin/bash +# Helper script to set up upstream tracking for fork-and-PR workflow. +set -e + +# Detect current branch if not provided as argument +BRANCH_NAME="${1:-$(git symbolic-ref --short HEAD)}" + +if [ -z "$BRANCH_NAME" ]; then + echo "Error: Could not detect current branch name." >&2 + exit 1 +fi + +echo "Setting up upstream tracking for branch: $BRANCH_NAME" + +# 1. Set the pull/fetch behavior to track the canonical repo's main branch +git config --local "branch.${BRANCH_NAME}.remote" upstream +git config --local "branch.${BRANCH_NAME}.merge" refs/heads/main + +# 2. Override the destination remote for pushing +git config --local "branch.${BRANCH_NAME}.pushRemote" origin + +# 3. Defend against global push.default settings (ensure it pushes to current branch name) +git config --local push.default current + +echo "Successfully configured upstream/main tracking and origin pushRemote for $BRANCH_NAME!" From 9ed764141b8c5b5a389800a72517af12090ac72d Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Sat, 27 Jun 2026 23:56:58 -0700 Subject: [PATCH 783/922] build(ci): consolidate PR guardrails and block draft files (#3859) Create a workflow for lighter weight PR checks. These just check basic things about a PR, e.g. markers to prevent submit, blocking files that shouldn't be checked in. --- .../workflows/check_do_not_merge_label.yml | 20 ------- .github/workflows/pr-metadata-checks.yaml | 57 +++++++++++++++++++ 2 files changed, 57 insertions(+), 20 deletions(-) delete mode 100644 .github/workflows/check_do_not_merge_label.yml create mode 100644 .github/workflows/pr-metadata-checks.yaml diff --git a/.github/workflows/check_do_not_merge_label.yml b/.github/workflows/check_do_not_merge_label.yml deleted file mode 100644 index 97b91b156a..0000000000 --- a/.github/workflows/check_do_not_merge_label.yml +++ /dev/null @@ -1,20 +0,0 @@ -name: "Check 'do not merge' label" - -on: - pull_request_target: - types: - - opened - - synchronize - - reopened - - labeled - - unlabeled - -jobs: - block-do-not-merge: - runs-on: ubuntu-latest - steps: - - name: Check for "do not merge" label - if: "contains(github.event.pull_request.labels.*.name, 'do not merge')" - run: | - echo "This PR has the 'do not merge' label and cannot be merged." - exit 1 diff --git a/.github/workflows/pr-metadata-checks.yaml b/.github/workflows/pr-metadata-checks.yaml new file mode 100644 index 0000000000..003b4d99a5 --- /dev/null +++ b/.github/workflows/pr-metadata-checks.yaml @@ -0,0 +1,57 @@ +name: "PR Metadata Checks" + +on: + pull_request_target: + types: + - opened + - synchronize + - reopened + - labeled + - unlabeled + +jobs: + blocks-do-not-merge: + runs-on: ubuntu-latest + steps: + - name: Check for "do not merge" label + if: "contains(github.event.pull_request.labels.*.name, 'do not merge')" + run: | + echo "::error::This PR has the 'do not merge' label" \ + "and cannot be merged." + exit 1 + + - name: Check PR description for DO NOT SUBMIT/MERGE + env: + PR_BODY: ${{ github.event.pull_request.body }} + run: | + echo "Checking PR description..." + if echo "$PR_BODY" | grep -Ei "DO NOT SUBMIT|DO NOT MERGE"; then + echo "::error::This PR description contains" \ + "'DO NOT SUBMIT' or 'DO NOT MERGE'" \ + "and cannot be merged." + exit 1 + fi + echo "PR description is clean." + + block-transient-agent-files: + runs-on: ubuntu-latest + permissions: + pull-requests: read + steps: + - name: Check for blocked files + env: + GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} + PR_NUMBER: ${{ github.event.pull_request.number }} + run: | + echo "Checking for blocked files..." + changed_files=$(gh pr diff "$PR_NUMBER" --name-only) + blocked_files=$(echo "$changed_files" | grep -E '^.agents/(plans|scratch)(/|$)' || true) + if [ -n "$blocked_files" ]; then + echo "::error::Files in .agents/plans and" \ + ".agents/scratch are permitted in PRs to" \ + "facilitate discussion, but are not allowed" \ + "to be merged. Please remove them before merging:" + echo "$blocked_files" + exit 1 + fi + echo "No blocked files found." From de6ca1baa21ff295ef63c04ee8e3c2a21993af99 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Sun, 28 Jun 2026 01:07:31 -0700 Subject: [PATCH 784/922] refactor: use gazelle for bzl_library management (#3852) To make it easier to manage the many bzl_library targets we have, switch to using gazelle to do so. Unfortunately, gazelle has strong opinions about target names: it forces `{foo}` names and doesn't allow `{foo}_bzl` names. Trying to make it do so requires quite a bit of gazelle directives. Instead, rename all the internal targets, but create aliases for public targets. Along the way... * Delete unused py_args.bzl * Add agent rule to avoid inappropriate copyright * Add a skill for creating rules --- .agents/rules/when-to-use-copyright.md | 8 + .agents/skills/rule-creator/SKILL.md | 56 + .pre-commit-config.yaml | 6 + BUILD.bazel | 47 +- MODULE.bazel | 2 + WORKSPACE | 11 +- docs/BUILD.bazel | 104 +- gazelle/BUILD.bazel | 9 + gazelle/manifest/BUILD.bazel | 16 +- gazelle/modules_mapping/BUILD.bazel | 7 + gazelle/python/BUILD.bazel | 25 +- gazelle/python/private/BUILD.bazel | 1 + gazelle/pythonconfig/BUILD.bazel | 4 +- news/gazelle-bzl-library.changed.md | 3 + python/BUILD.bazel | 465 ++++--- python/api/BUILD.bazel | 39 +- python/cc/BUILD.bazel | 28 +- python/config_settings/BUILD.bazel | 5 + python/config_settings/private/py_args.bzl | 42 - python/entry_points/BUILD.bazel | 23 +- python/extensions/BUILD.bazel | 45 +- python/features.bzl | 44 +- python/local_toolchains/BUILD.bazel | 19 +- python/pip_install/BUILD.bazel | 40 +- python/private/BUILD.bazel | 1075 +++++++++-------- python/private/api/BUILD.bazel | 17 +- python/private/api/api.bzl | 2 +- python/private/internal_config_repo.bzl | 4 +- python/private/interpreter.bzl | 2 +- python/private/py_cc_toolchain_rule.bzl | 2 +- python/private/py_exec_tools_toolchain.bzl | 2 +- python/private/py_interpreter_program.bzl | 2 +- python/private/py_wheel.bzl | 2 +- python/private/pypi/BUILD.bazel | 540 +++++---- python/private/pythons_hub.bzl | 4 +- .../{sentinel.bzl => sentinel_impl.bzl} | 0 python/private/{stamp.bzl => stamp_impl.bzl} | 0 python/private/whl_filegroup/BUILD.bazel | 12 +- python/private/zipapp/BUILD.bazel | 37 +- python/uv/BUILD.bazel | 24 +- python/uv/private/BUILD.bazel | 129 +- python/zipapp/BUILD.bazel | 13 +- sphinxdocs/docs/BUILD.bazel | 10 +- sphinxdocs/sphinxdocs/BUILD.bazel | 42 +- sphinxdocs/sphinxdocs/private/BUILD.bazel | 32 +- .../sphinxdocs/private/sphinx_stardoc.bzl | 4 +- sphinxdocs/tests/sphinx_stardoc/BUILD.bazel | 2 +- tests/config_settings/transition/BUILD.bazel | 3 - .../transition/py_args_tests.bzl | 68 -- tools/build_defs/python/private/BUILD.bazel | 5 +- tools/private/BUILD.bazel | 8 + tools/private/gazelle/BUILD.bazel | 21 + tools/private/update_deps/BUILD.bazel | 4 +- workspace_bazel9.bzl | 10 + 54 files changed, 1741 insertions(+), 1384 deletions(-) create mode 100644 .agents/rules/when-to-use-copyright.md create mode 100644 .agents/skills/rule-creator/SKILL.md create mode 100644 news/gazelle-bzl-library.changed.md delete mode 100644 python/config_settings/private/py_args.bzl rename python/private/{sentinel.bzl => sentinel_impl.bzl} (100%) rename python/private/{stamp.bzl => stamp_impl.bzl} (100%) delete mode 100644 tests/config_settings/transition/py_args_tests.bzl create mode 100644 tools/private/gazelle/BUILD.bazel create mode 100644 workspace_bazel9.bzl diff --git a/.agents/rules/when-to-use-copyright.md b/.agents/rules/when-to-use-copyright.md new file mode 100644 index 0000000000..a73cbbfd70 --- /dev/null +++ b/.agents/rules/when-to-use-copyright.md @@ -0,0 +1,8 @@ +--- +trigger: always_on +--- + +# When to Use Copyright + +Unless directed by the user otherwise, do not add Bazel copyright to new or +existing files. diff --git a/.agents/skills/rule-creator/SKILL.md b/.agents/skills/rule-creator/SKILL.md new file mode 100644 index 0000000000..9ed7978120 --- /dev/null +++ b/.agents/skills/rule-creator/SKILL.md @@ -0,0 +1,56 @@ +--- +name: rule-creator +description: Create and format agent rules with proper front matter in the workspace +--- + +Use this skill when you need to create a new rule for the agent in the +workspace. + +### Rule File Location + +All workspace-specific rules must be created as individual Markdown files under +the `.agents/rules/` directory: +``` +.agents/rules/.md +``` + +### Rule Format + +Every rule file must start with a YAML front matter block defining the trigger +condition, followed by the rule content in Markdown. + +```yaml +--- +trigger: +--- + +# + + +``` + +#### Trigger Conditions +* `always_on`: The rule is always active and must be followed for all tasks. +* Custom triggers: You can specify other trigger conditions if the rule only + applies in certain contexts. + +### Formatting Guidelines +* **Line Wrapping:** Always wrap all text in the rule file (including the + title and description) to **80 columns** to ensure readability and + compatibility. +* **Clarity:** Write clear, actionable directives. + +### Example + +To create a rule that prevents adding copyrights: + +```markdown +--- +trigger: always_on +--- + +# No Copyrights Rule + +Unless directed by the user otherwise, do not add Bazel copyright to new or +existing files. +``` diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml index e4d65c8bb6..f4107880b2 100644 --- a/.pre-commit-config.yaml +++ b/.pre-commit-config.yaml @@ -65,3 +65,9 @@ repos: entry: ./tools/private/sync_downloader_configs.py files: downloader_config\.cfg$ pass_filenames: false + - id: gazelle + name: Run Gazelle + language: system + entry: bazel run //tools/private/gazelle + files: (\.bzl|\.bazel|BUILD|WORKSPACE(\.bzlmod)?)$ + pass_filenames: false diff --git a/BUILD.bazel b/BUILD.bazel index 7da18ebaa4..a52251393d 100644 --- a/BUILD.bazel +++ b/BUILD.bazel @@ -14,6 +14,40 @@ load("@bazel_skylib//:bzl_library.bzl", "bzl_library") +# Resolve rules_cc Starlark libraries to their correct targets +# gazelle:resolve starlark @rules_cc//cc/common:cc_info.bzl @rules_cc//cc/common:common +# gazelle:resolve starlark @rules_cc//cc/common:cc_common.bzl @rules_cc//cc/common:common + +# Resolve protobuf Starlark libraries +# gazelle:resolve starlark @com_google_protobuf//bazel:py_proto_library.bzl @com_google_protobuf//bazel:py_proto_library_bzl + +# Resolve bazel_tools repo rules to our internal wrapper to avoid transitive dependencies +# gazelle:resolve starlark @bazel_tools//tools/build_defs/repo:http.bzl //python/private:bazel_tools +# gazelle:resolve starlark @bazel_tools//tools/build_defs/repo:utils.bzl //python/private:bazel_tools + +# Prevent Gazelle from incorrectly stripping the .bzl suffix from the toml.bzl repo name +# gazelle:resolve starlark @toml.bzl//:toml.bzl @toml.bzl//:toml + +# Override Gazelle's incorrect default resolution for platforms host constraints Starlark library +# gazelle:resolve starlark @platforms//host:constraints.bzl @platforms//host:constraints_lib + +# Override Gazelle's incorrect default resolution for rules_cc Starlark libraries +# gazelle:resolve starlark @rules_cc//cc:cc_import.bzl @rules_cc//cc:core_rules +# gazelle:resolve starlark @rules_cc//cc:cc_library.bzl @rules_cc//cc:core_rules + +# Exclude directories that are separate packages/workspaces or only for testing/examples +# gazelle:exclude tests +# gazelle:exclude examples + +# Exclude internal development tools and dependencies that users don't need +# gazelle:exclude internal_dev_setup.bzl +# gazelle:exclude internal_dev_deps.bzl +# gazelle:exclude python/private/internal_dev_deps.bzl +# gazelle:exclude workspace_bazel9.bzl + +# Exclude legacy paths that are only kept for backwards compatibility +# gazelle:exclude python/private/common + package(default_visibility = ["//visibility:public"]) licenses(["notice"]) @@ -52,12 +86,6 @@ filegroup( ], ) -bzl_library( - name = "version_bzl", - srcs = ["version.bzl"], - visibility = ["//:__subpackages__"], -) - # Reexport of all bzl files used to allow downstream rules to generate docs # without shipping with a dependency on Skylib filegroup( @@ -73,3 +101,10 @@ filegroup( ], visibility = ["//visibility:public"], ) + +# keep +bzl_library( + name = "version", + srcs = ["version.bzl"], + visibility = ["//:__subpackages__"], +) diff --git a/MODULE.bazel b/MODULE.bazel index 4de83f31be..c4afa60640 100644 --- a/MODULE.bazel +++ b/MODULE.bazel @@ -96,6 +96,8 @@ bazel_dep(name = "another_module", version = "0", dev_dependency = True) # Extra gazelle plugin deps so that WORKSPACE.bzlmod can continue including it for e2e tests. # We use `WORKSPACE.bzlmod` because it is impossible to have dev-only local overrides. bazel_dep(name = "rules_go", version = "0.60.0", dev_dependency = True, repo_name = "io_bazel_rules_go") +bazel_dep(name = "gazelle", version = "0.40.0", dev_dependency = True, repo_name = "bazel_gazelle") +bazel_dep(name = "bazel_skylib_gazelle_plugin", version = "1.8.2", dev_dependency = True) internal_dev_deps = use_extension( "//python/private:internal_dev_deps.bzl", diff --git a/WORKSPACE b/WORKSPACE index 077ddb5e68..5a31274b21 100644 --- a/WORKSPACE +++ b/WORKSPACE @@ -17,13 +17,10 @@ workspace(name = "rules_python") # Everything below this line is used only for developing rules_python. Users # should not copy it to their WORKSPACE. -# Necessary so that Bazel 9 recognizes this as rules_python and doesn't try -# to load the version Bazel itself uses by default. -# buildifier: disable=duplicated-name -local_repository( - name = "rules_python", - path = ".", -) +# Workaround for Bazel 9 duplicate name issue in Gazelle. +load("//:workspace_bazel9.bzl", "bazel_9_workaround") + +bazel_9_workaround() load("//:internal_dev_deps.bzl", "rules_python_internal_deps") diff --git a/docs/BUILD.bazel b/docs/BUILD.bazel index 043f3e4de7..2e13da1296 100644 --- a/docs/BUILD.bazel +++ b/docs/BUILD.bazel @@ -106,59 +106,59 @@ build_test( sphinx_stardocs( name = "bzl_api_docs", srcs = [ - "//python:defs_bzl", - "//python:features_bzl", - "//python:packaging_bzl", - "//python:pip_bzl", - "//python:proto_bzl", - "//python:py_binary_bzl", - "//python:py_cc_link_params_info_bzl", - "//python:py_exec_tools_info_bzl", - "//python:py_exec_tools_toolchain_bzl", - "//python:py_executable_info_bzl", - "//python:py_info_bzl", - "//python:py_library_bzl", - "//python:py_runtime_bzl", - "//python:py_runtime_info_bzl", - "//python:py_test_bzl", - "//python:repositories_bzl", - "//python/api:api_bzl", - "//python/api:attr_builders_bzl", - "//python/api:executables_bzl", - "//python/api:libraries_bzl", - "//python/api:rule_builders_bzl", - "//python/cc:py_cc_toolchain_bzl", - "//python/cc:py_cc_toolchain_info_bzl", - "//python/entry_points:py_console_script_binary_bzl", - "//python/extensions:config_bzl", - "//python/extensions:python_bzl", - "//python/local_toolchains:repos_bzl", - "//python/private:attr_builders_bzl", - "//python/private:builders_bzl", - "//python/private:builders_util_bzl", - "//python/private:py_binary_rule_bzl", - "//python/private:py_cc_toolchain_rule_bzl", - "//python/private:py_info_bzl", - "//python/private:py_library_rule_bzl", - "//python/private:py_runtime_rule_bzl", - "//python/private:py_test_rule_bzl", - "//python/private:rule_builders_bzl", - "//python/private/api:py_common_api_bzl", - "//python/private/pypi:config_settings_bzl", - "//python/private/pypi:env_marker_info_bzl", - "//python/private/pypi:pkg_aliases_bzl", - "//python/private/pypi:whl_config_setting_bzl", - "//python/private/pypi:whl_library_bzl", - "//python/private/zipapp:py_zipapp_rule_bzl", - "//python/uv:lock_bzl", - "//python/uv:uv_bzl", - "//python/uv:uv_toolchain_bzl", - "//python/uv:uv_toolchain_info_bzl", - "//python/zipapp:py_zipapp_binary_bzl", - "//python/zipapp:py_zipapp_test_bzl", + "//python:defs", + "//python:features", + "//python:packaging", + "//python:pip", + "//python:proto", + "//python:py_binary", + "//python:py_cc_link_params_info", + "//python:py_exec_tools_info", + "//python:py_exec_tools_toolchain", + "//python:py_executable_info", + "//python:py_info", + "//python:py_library", + "//python:py_runtime", + "//python:py_runtime_info", + "//python:py_test", + "//python:repositories", + "//python/api", + "//python/api:attr_builders", + "//python/api:executables", + "//python/api:libraries", + "//python/api:rule_builders", + "//python/cc:py_cc_toolchain", + "//python/cc:py_cc_toolchain_info", + "//python/entry_points:py_console_script_binary", + "//python/extensions:config", + "//python/extensions:python", + "//python/local_toolchains:repos", + "//python/private:attr_builders", + "//python/private:builders", + "//python/private:builders_util", + "//python/private:py_binary_rule", + "//python/private:py_cc_toolchain_rule", + "//python/private:py_info", + "//python/private:py_library_rule", + "//python/private:py_runtime_rule", + "//python/private:py_test_rule", + "//python/private:rule_builders", + "//python/private/api:py_common_api", + "//python/private/pypi:config_settings", + "//python/private/pypi:env_marker_info", + "//python/private/pypi:pkg_aliases", + "//python/private/pypi:whl_config_setting", + "//python/private/pypi:whl_library", + "//python/private/zipapp:py_zipapp_rule", + "//python/uv", + "//python/uv:lock", + "//python/uv:uv_toolchain", + "//python/uv:uv_toolchain_info", + "//python/zipapp:py_zipapp_binary", + "//python/zipapp:py_zipapp_test", ] + ([ # This depends on @pythons_hub, which is only created under bzlmod, - "//python/extensions:pip_bzl", + "//python/extensions:pip", ] if BZLMOD_ENABLED else []), prefix = "api/rules_python/", tags = ["docs"], @@ -167,7 +167,7 @@ sphinx_stardocs( sphinx_stardoc( name = "py_runtime_pair", - src = "//python/private:py_runtime_pair_rule_bzl", + src = "//python/private:py_runtime_pair_rule", prefix = "api/rules_python/", tags = ["docs"], target_compatible_with = _TARGET_COMPATIBLE_WITH, diff --git a/gazelle/BUILD.bazel b/gazelle/BUILD.bazel index 0938be3dfc..30758181ed 100644 --- a/gazelle/BUILD.bazel +++ b/gazelle/BUILD.bazel @@ -1,9 +1,12 @@ load("@bazel_gazelle//:def.bzl", "gazelle") +load("@bazel_skylib//:bzl_library.bzl", "bzl_library") # Gazelle configuration options. # See https://github.com/bazelbuild/bazel-gazelle#running-gazelle-with-bazel # gazelle:prefix github.com/bazel-contrib/rules_python/gazelle # gazelle:exclude bazel-out +# gazelle:exclude deps.bzl +# gazelle:exclude internal_dev_deps.bzl gazelle( name = "gazelle", ) @@ -36,3 +39,9 @@ filegroup( ], visibility = ["@rules_python//:__pkg__"], ) + +bzl_library( + name = "def", + srcs = ["def.bzl"], + visibility = ["//visibility:public"], +) diff --git a/gazelle/manifest/BUILD.bazel b/gazelle/manifest/BUILD.bazel index ea81d85fbe..68f4e1be30 100644 --- a/gazelle/manifest/BUILD.bazel +++ b/gazelle/manifest/BUILD.bazel @@ -1,3 +1,4 @@ +load("@bazel_skylib//:bzl_library.bzl", "bzl_library") load("@io_bazel_rules_go//go:def.bzl", "go_library", "go_test") exports_files([ @@ -11,8 +12,8 @@ go_library( importpath = "github.com/bazel-contrib/rules_python/gazelle/manifest", visibility = ["//visibility:public"], deps = [ - "@com_github_emirpasic_gods//sets/treeset", - "@in_gopkg_yaml_v2//:yaml_v2", + "@com_github_emirpasic_gods//sets/treeset:go_default_library", + "@in_gopkg_yaml_v2//:go_default_library", ], ) @@ -32,3 +33,14 @@ filegroup( ], visibility = ["//:__pkg__"], ) + +bzl_library( + name = "defs", + srcs = ["defs.bzl"], + visibility = ["//visibility:public"], + deps = [ + "@bazel_skylib//rules:diff_test", + "@io_bazel_rules_go//go:def", + "@rules_python//python:defs_bzl", + ], +) diff --git a/gazelle/modules_mapping/BUILD.bazel b/gazelle/modules_mapping/BUILD.bazel index 3423f34e51..0e740f2636 100644 --- a/gazelle/modules_mapping/BUILD.bazel +++ b/gazelle/modules_mapping/BUILD.bazel @@ -1,3 +1,4 @@ +load("@bazel_skylib//:bzl_library.bzl", "bzl_library") load("@bazel_skylib//rules:copy_file.bzl", "copy_file") load("@rules_python//python:defs.bzl", "py_binary", "py_test") @@ -56,3 +57,9 @@ filegroup( srcs = glob(["**"]), visibility = ["//:__pkg__"], ) + +bzl_library( + name = "def", + srcs = ["def.bzl"], + visibility = ["//visibility:public"], +) diff --git a/gazelle/python/BUILD.bazel b/gazelle/python/BUILD.bazel index b988e493c7..8218d0b6d9 100644 --- a/gazelle/python/BUILD.bazel +++ b/gazelle/python/BUILD.bazel @@ -30,19 +30,19 @@ go_library( visibility = ["//visibility:public"], deps = [ "//pythonconfig", - "@bazel_gazelle//config:go_default_library", - "@bazel_gazelle//label:go_default_library", - "@bazel_gazelle//language:go_default_library", - "@bazel_gazelle//repo:go_default_library", - "@bazel_gazelle//resolve:go_default_library", - "@bazel_gazelle//rule:go_default_library", + "@bazel_gazelle//config", + "@bazel_gazelle//label", + "@bazel_gazelle//language", + "@bazel_gazelle//repo", + "@bazel_gazelle//resolve", + "@bazel_gazelle//rule", "@com_github_bazelbuild_buildtools//build", "@com_github_bmatcuk_doublestar_v4//:doublestar", - "@com_github_emirpasic_gods//lists/singlylinkedlist", - "@com_github_emirpasic_gods//sets/treeset", - "@com_github_emirpasic_gods//utils", - "@com_github_smacker_go_tree_sitter//:go-tree-sitter", - "@com_github_smacker_go_tree_sitter//python", + "@com_github_emirpasic_gods//lists/singlylinkedlist:go_default_library", + "@com_github_emirpasic_gods//sets/treeset:go_default_library", + "@com_github_emirpasic_gods//utils:go_default_library", + "@com_github_smacker_go_tree_sitter//:go_default_library", + "@com_github_smacker_go_tree_sitter//python:go_default_library", "@org_golang_x_sync//errgroup", ], ) @@ -65,6 +65,9 @@ copy_file( ) # gazelle:exclude testdata/ +# gazelle:exclude extensions.bzl +# Exclude test-only Starlark helper from Gazelle to avoid generating an unnecessary bzl_library +# gazelle:exclude gazelle_test.bzl gazelle_test( name = "python_test", diff --git a/gazelle/python/private/BUILD.bazel b/gazelle/python/private/BUILD.bazel index e69de29bb2..ccdc080664 100644 --- a/gazelle/python/private/BUILD.bazel +++ b/gazelle/python/private/BUILD.bazel @@ -0,0 +1 @@ +# gazelle:exclude extensions.bzl diff --git a/gazelle/pythonconfig/BUILD.bazel b/gazelle/pythonconfig/BUILD.bazel index 711bf2eb42..a82a7f19ca 100644 --- a/gazelle/pythonconfig/BUILD.bazel +++ b/gazelle/pythonconfig/BUILD.bazel @@ -10,8 +10,8 @@ go_library( visibility = ["//visibility:public"], deps = [ "//manifest", - "@bazel_gazelle//label:go_default_library", - "@com_github_emirpasic_gods//lists/singlylinkedlist", + "@bazel_gazelle//label", + "@com_github_emirpasic_gods//lists/singlylinkedlist:go_default_library", ], ) diff --git a/news/gazelle-bzl-library.changed.md b/news/gazelle-bzl-library.changed.md new file mode 100644 index 0000000000..0d602110e3 --- /dev/null +++ b/news/gazelle-bzl-library.changed.md @@ -0,0 +1,3 @@ +Renamed most public bzl_library targets from `{foo}_bzl` to `{foo}` to follow +gazelle naming conventions. Deprecated aliases are left for backwards +compatibility. diff --git a/python/BUILD.bazel b/python/BUILD.bazel index e940e9e94b..a4dd98c572 100644 --- a/python/BUILD.bazel +++ b/python/BUILD.bazel @@ -24,6 +24,10 @@ In an ideal renaming, we'd move the packaging rules to a different package so that @rules_python//python is only concerned with the core rules. """ +# TODO: make current_py_toolchain.bzl private +# gazelle:resolve starlark @rules_python//python:current_py_toolchain.bzl //python:current_py_toolchain_bzl +# gazelle:resolve starlark //python:current_py_toolchain.bzl //python:current_py_toolchain_bzl + load("@bazel_skylib//:bzl_library.bzl", "bzl_library") load(":current_py_toolchain.bzl", "current_py_toolchain") @@ -54,184 +58,6 @@ filegroup( # ========= bzl_library targets end ========= -bzl_library( - name = "current_py_toolchain_bzl", - srcs = ["current_py_toolchain.bzl"], - deps = ["//python/private:toolchain_types_bzl"], -) - -bzl_library( - name = "defs_bzl", - srcs = [ - "defs.bzl", - ], - visibility = ["//visibility:public"], - deps = [ - ":current_py_toolchain_bzl", - ":py_binary_bzl", - ":py_import_bzl", - ":py_info_bzl", - ":py_library_bzl", - ":py_runtime_bzl", - ":py_runtime_info_bzl", - ":py_runtime_pair_bzl", - ":py_test_bzl", - ], -) - -bzl_library( - name = "features_bzl", - srcs = ["features.bzl"], -) - -bzl_library( - name = "packaging_bzl", - srcs = ["packaging.bzl"], - deps = [ - "//python/entry_points:py_console_script_binary_bzl", - "//python/private:bzlmod_enabled_bzl", - "//python/private:py_package_bzl", - "//python/private:py_wheel_bzl", - "//python/private:util_bzl", - "@bazel_skylib//rules:native_binary", - ], -) - -bzl_library( - name = "pip_bzl", - srcs = ["pip.bzl"], - deps = [ - "//python/private:normalize_name_bzl", - "//python/private/pypi:multi_pip_parse_bzl", - "//python/private/pypi:package_annotation_bzl", - "//python/private/pypi:pip_compile_bzl", - "//python/private/pypi:pip_repository_bzl", - "//python/private/pypi:whl_library_alias_bzl", - "//python/private/whl_filegroup:whl_filegroup_bzl", - ], -) - -bzl_library( - name = "proto_bzl", - srcs = [ - "proto.bzl", - ], - visibility = ["//visibility:public"], - deps = [ - "@com_google_protobuf//bazel:py_proto_library_bzl", - ], -) - -bzl_library( - name = "py_binary_bzl", - srcs = ["py_binary.bzl"], - deps = [ - "//python/private:py_binary_macro_bzl", - ], -) - -bzl_library( - name = "py_cc_link_params_info_bzl", - srcs = ["py_cc_link_params_info.bzl"], - deps = [ - "//python/private:py_cc_link_params_info_bzl", - ], -) - -bzl_library( - name = "py_exec_tools_info_bzl", - srcs = ["py_exec_tools_info.bzl"], - deps = ["//python/private:py_exec_tools_info_bzl"], -) - -bzl_library( - name = "py_exec_tools_toolchain_bzl", - srcs = ["py_exec_tools_toolchain.bzl"], - deps = ["//python/private:py_exec_tools_toolchain_bzl"], -) - -bzl_library( - name = "py_executable_info_bzl", - srcs = ["py_executable_info.bzl"], - deps = ["//python/private:py_executable_info_bzl"], -) - -bzl_library( - name = "py_import_bzl", - srcs = ["py_import.bzl"], - deps = [":py_info_bzl"], -) - -bzl_library( - name = "py_info_bzl", - srcs = ["py_info.bzl"], - deps = [ - "//python/private:py_info_bzl", - ], -) - -bzl_library( - name = "py_library_bzl", - srcs = ["py_library.bzl"], - deps = [ - "//python/private:py_library_macro_bzl", - ], -) - -bzl_library( - name = "py_runtime_bzl", - srcs = ["py_runtime.bzl"], - deps = [ - "//python/private:py_runtime_macro_bzl", - ], -) - -bzl_library( - name = "py_runtime_pair_bzl", - srcs = ["py_runtime_pair.bzl"], - deps = [ - "//python/private:py_runtime_pair_macro_bzl", - ], -) - -bzl_library( - name = "py_runtime_info_bzl", - srcs = ["py_runtime_info.bzl"], - deps = [ - "//python/private:py_runtime_info_bzl", - ], -) - -bzl_library( - name = "py_test_bzl", - srcs = ["py_test.bzl"], - deps = [ - "//python/private:py_test_macro_bzl", - ], -) - -bzl_library( - name = "repositories_bzl", - srcs = ["repositories.bzl"], - deps = [ - "//python/private:is_standalone_interpreter_bzl", - "//python/private:py_repositories_bzl", - "//python/private:python_register_multi_toolchains_bzl", - "//python/private:python_register_toolchains_bzl", - "//python/private:python_repository_bzl", - ], -) - -bzl_library( - name = "versions_bzl", - srcs = ["versions.bzl"], - visibility = ["//:__subpackages__"], - deps = [ - "//python/private:platform_info_bzl", - "//python/private:runtimes_manifest_workspace_bzl", - ], -) - # NOTE: Remember to add bzl_library targets to //tests:bzl_libraries # ========= bzl_library targets end ========= @@ -351,3 +177,286 @@ exports_files([ current_py_toolchain( name = "current_py_toolchain", ) + +# Keep this target because it is a public API and Gazelle might otherwise +# remove it if it is not loaded by other bzl_library targets. +# keep +bzl_library( + name = "current_py_toolchain_bzl", + srcs = ["current_py_toolchain.bzl"], + visibility = ["//visibility:public"], +) + +bzl_library( + name = "defs", + srcs = ["defs.bzl"], + deps = [ + ":py_binary", + ":py_import", + ":py_info", + ":py_library", + ":py_runtime", + ":py_runtime_info", + ":py_runtime_pair", + ":py_test", + "//python:current_py_toolchain_bzl", + ], +) + +alias( + name = "defs_bzl", + actual = ":defs", + deprecation = "Use //python:defs instead", +) + +alias( + name = "features_bzl", + actual = ":features", + deprecation = "Use //python:features instead", +) + +bzl_library( + name = "packaging", + srcs = ["packaging.bzl"], + deps = [ + ":py_binary", + "//python/private:bzlmod_enabled", + "//python/private:py_package", + "//python/private:py_wheel", + "//python/private:util", + "@bazel_skylib//rules:native_binary", + ], +) + +alias( + name = "packaging_bzl", + actual = ":packaging", + deprecation = "Use //python:packaging instead", +) + +bzl_library( + name = "pip", + srcs = ["pip.bzl"], + deps = [ + "//python/private:normalize_name", + "//python/private/pypi:multi_pip_parse", + "//python/private/pypi:package_annotation", + "//python/private/pypi:pip_compile", + "//python/private/pypi:pip_repository", + "//python/private/pypi:whl_library_alias", + "//python/private/whl_filegroup", + ], +) + +alias( + name = "pip_bzl", + actual = ":pip", + deprecation = "Use //python:pip instead", +) + +bzl_library( + name = "proto", + srcs = ["proto.bzl"], + deps = ["@com_google_protobuf//bazel:py_proto_library_bzl"], +) + +alias( + name = "proto_bzl", + actual = ":proto", + deprecation = "Use //python:proto instead", +) + +bzl_library( + name = "py_binary", + srcs = ["py_binary.bzl"], + deps = ["//python/private:py_binary_macro"], +) + +alias( + name = "py_binary_bzl", + actual = ":py_binary", + deprecation = "Use //python:py_binary instead", +) + +bzl_library( + name = "py_cc_link_params_info", + srcs = ["py_cc_link_params_info.bzl"], + deps = ["//python/private:py_cc_link_params_info"], +) + +alias( + name = "py_cc_link_params_info_bzl", + actual = ":py_cc_link_params_info", + deprecation = "Use //python:py_cc_link_params_info instead", +) + +bzl_library( + name = "py_exec_tools_info", + srcs = ["py_exec_tools_info.bzl"], + deps = ["//python/private:py_exec_tools_info"], +) + +alias( + name = "py_exec_tools_info_bzl", + actual = ":py_exec_tools_info", + deprecation = "Use //python:py_exec_tools_info instead", +) + +bzl_library( + name = "py_exec_tools_toolchain", + srcs = ["py_exec_tools_toolchain.bzl"], + deps = ["//python/private:py_exec_tools_toolchain"], +) + +alias( + name = "py_exec_tools_toolchain_bzl", + actual = ":py_exec_tools_toolchain", + deprecation = "Use //python:py_exec_tools_toolchain instead", +) + +bzl_library( + name = "py_executable_info", + srcs = ["py_executable_info.bzl"], + deps = ["//python/private:py_executable_info"], +) + +alias( + name = "py_executable_info_bzl", + actual = ":py_executable_info", + deprecation = "Use //python:py_executable_info instead", +) + +bzl_library( + name = "py_import", + srcs = ["py_import.bzl"], + deps = [":py_info"], +) + +alias( + name = "py_import_bzl", + actual = ":py_import", + deprecation = "Use //python:py_import instead", +) + +bzl_library( + name = "py_info", + srcs = ["py_info.bzl"], + deps = ["//python/private:py_info"], +) + +alias( + name = "py_info_bzl", + actual = ":py_info", + deprecation = "Use //python:py_info instead", +) + +bzl_library( + name = "py_library", + srcs = ["py_library.bzl"], + deps = ["//python/private:py_library_macro"], +) + +alias( + name = "py_library_bzl", + actual = ":py_library", + deprecation = "Use //python:py_library instead", +) + +bzl_library( + name = "py_runtime", + srcs = ["py_runtime.bzl"], + deps = ["//python/private:py_runtime_macro"], +) + +alias( + name = "py_runtime_bzl", + actual = ":py_runtime", + deprecation = "Use //python:py_runtime instead", +) + +bzl_library( + name = "py_runtime_info", + srcs = ["py_runtime_info.bzl"], + deps = ["//python/private:py_runtime_info"], +) + +alias( + name = "py_runtime_info_bzl", + actual = ":py_runtime_info", + deprecation = "Use //python:py_runtime_info instead", +) + +bzl_library( + name = "py_runtime_pair", + srcs = ["py_runtime_pair.bzl"], + deps = ["//python/private:py_runtime_pair_macro"], +) + +alias( + name = "py_runtime_pair_bzl", + actual = ":py_runtime_pair", + deprecation = "Use //python:py_runtime_pair instead", +) + +# keep +bzl_library( + name = "py_test", + srcs = ["py_test.bzl"], + deps = ["//python/private:py_test_macro"], +) + +alias( + name = "py_test_bzl", + actual = ":py_test", + deprecation = "Use //python:py_test instead", +) + +alias( + name = "python_bzl", + actual = ":python", + deprecation = "Use //python:python instead", +) + +bzl_library( + name = "repositories", + srcs = ["repositories.bzl"], + deps = [ + "//python/private:is_standalone_interpreter", + "//python/private:py_repositories", + "//python/private:python_register_multi_toolchains", + "//python/private:python_register_toolchains", + "//python/private:python_repository", + ], +) + +alias( + name = "repositories_bzl", + actual = ":repositories", + deprecation = "Use //python:repositories instead", +) + +bzl_library( + name = "versions", + srcs = ["versions.bzl"], + deps = [ + "//python/private:pbs_manifest", + "//python/private:platform_info", + "//python/private:runtimes_manifest_workspace", + ], +) + +alias( + name = "versions_bzl", + actual = ":versions", + deprecation = "Use //python:versions instead", +) + +bzl_library( + name = "features", + srcs = ["features.bzl"], +) + +bzl_library( + name = "python", + srcs = ["python.bzl"], +) diff --git a/python/api/BUILD.bazel b/python/api/BUILD.bazel index 11fee103cb..93c70f4c8b 100644 --- a/python/api/BUILD.bazel +++ b/python/api/BUILD.bazel @@ -18,46 +18,41 @@ package( default_visibility = ["//:__subpackages__"], ) +filegroup( + name = "distribution", + srcs = glob(["**"]), +) + bzl_library( - name = "api_bzl", + name = "api", srcs = ["api.bzl"], - visibility = ["//visibility:public"], - deps = ["//python/private/api:api_bzl"], + deps = ["//python/private/api"], ) bzl_library( - name = "attr_builders_bzl", + name = "attr_builders", srcs = ["attr_builders.bzl"], - deps = ["//python/private:attr_builders_bzl"], + deps = ["//python/private:attr_builders"], ) bzl_library( - name = "executables_bzl", + name = "executables", srcs = ["executables.bzl"], - visibility = ["//visibility:public"], deps = [ - "//python/private:py_binary_rule_bzl", - "//python/private:py_executable_bzl", - "//python/private:py_test_rule_bzl", + "//python/private:py_binary_rule", + "//python/private:py_executable", + "//python/private:py_test_rule", ], ) bzl_library( - name = "libraries_bzl", + name = "libraries", srcs = ["libraries.bzl"], - visibility = ["//visibility:public"], - deps = [ - "//python/private:py_library_bzl", - ], + deps = ["//python/private:py_library"], ) bzl_library( - name = "rule_builders_bzl", + name = "rule_builders", srcs = ["rule_builders.bzl"], - deps = ["//python/private:rule_builders_bzl"], -) - -filegroup( - name = "distribution", - srcs = glob(["**"]), + deps = ["//python/private:rule_builders"], ) diff --git a/python/cc/BUILD.bazel b/python/cc/BUILD.bazel index f7686c41f6..f17805d92f 100644 --- a/python/cc/BUILD.bazel +++ b/python/cc/BUILD.bazel @@ -47,21 +47,35 @@ toolchain_type( visibility = ["//visibility:public"], ) +filegroup( + name = "distribution", + srcs = glob(["**"]), +) + bzl_library( - name = "py_cc_toolchain_bzl", + name = "py_cc_toolchain", srcs = ["py_cc_toolchain.bzl"], visibility = ["//visibility:public"], - deps = ["//python/private:py_cc_toolchain_macro_bzl"], + deps = ["//python/private:py_cc_toolchain_macro"], ) bzl_library( - name = "py_cc_toolchain_info_bzl", + name = "py_cc_toolchain_info", srcs = ["py_cc_toolchain_info.bzl"], visibility = ["//visibility:public"], - deps = ["//python/private:py_cc_toolchain_info_bzl"], + deps = ["//python/private:py_cc_toolchain_info"], ) -filegroup( - name = "distribution", - srcs = glob(["**"]), +alias( + name = "py_cc_toolchain_bzl", + actual = ":py_cc_toolchain", + deprecation = "Use //python/cc:py_cc_toolchain instead", + visibility = ["//visibility:public"], +) + +alias( + name = "py_cc_toolchain_info_bzl", + actual = ":py_cc_toolchain_info", + deprecation = "Use //python/cc:py_cc_toolchain_info instead", + visibility = ["//visibility:public"], ) diff --git a/python/config_settings/BUILD.bazel b/python/config_settings/BUILD.bazel index 369ce6de55..97761bea83 100644 --- a/python/config_settings/BUILD.bazel +++ b/python/config_settings/BUILD.bazel @@ -17,6 +17,11 @@ load( load("//python/private/pypi:flags.bzl", "define_pypi_internal_flags") load(":config_settings.bzl", "construct_config_settings") +# We don't generate bzl_library for these because they aren't public targets +# and should be moved +# gazelle:exclude config_settings.bzl +# gazelle:exclude transition.bzl + filegroup( name = "distribution", srcs = glob(["**"]) + [ diff --git a/python/config_settings/private/py_args.bzl b/python/config_settings/private/py_args.bzl deleted file mode 100644 index 09a26461b7..0000000000 --- a/python/config_settings/private/py_args.bzl +++ /dev/null @@ -1,42 +0,0 @@ -# Copyright 2023 The Bazel Authors. All rights reserved. -# -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -"""A helper to extract default args for the transition rule.""" - -def py_args(name, kwargs): - """A helper to extract common py_binary and py_test args - - See https://bazel.build/reference/be/python#py_binary and - https://bazel.build/reference/be/python#py_test for the list - that should be returned - - Args: - name: The name of the target. - kwargs: The kwargs to be extracted from; MODIFIED IN-PLACE. - - Returns: - A dict with the extracted arguments - """ - return dict( - args = kwargs.pop("args", None), - data = kwargs.pop("data", None), - env = kwargs.pop("env", None), - srcs = kwargs.pop("srcs", None), - deps = kwargs.pop("deps", None), - # See https://bazel.build/reference/be/python#py_binary.main - # for default logic. - # NOTE: This doesn't match the exact way a regular py_binary searches for - # it's main amongst the srcs, but is close enough for most cases. - main = kwargs.pop("main", name + ".py"), - ) diff --git a/python/entry_points/BUILD.bazel b/python/entry_points/BUILD.bazel index 46dbd9298b..7cfd0cdf25 100644 --- a/python/entry_points/BUILD.bazel +++ b/python/entry_points/BUILD.bazel @@ -21,17 +21,22 @@ exports_files( visibility = ["//docs:__subpackages__"], ) -bzl_library( - name = "py_console_script_binary_bzl", - srcs = [":py_console_script_binary.bzl"], - visibility = ["//visibility:public"], - deps = [ - "//python/private:py_console_script_binary_bzl", - ], -) - filegroup( name = "distribution", srcs = glob(["**"]), visibility = ["//python:__subpackages__"], ) + +bzl_library( + name = "py_console_script_binary", + srcs = ["py_console_script_binary.bzl"], + visibility = ["//visibility:public"], + deps = ["//python/private:py_console_script_binary"], +) + +alias( + name = "py_console_script_binary_bzl", + actual = ":py_console_script_binary", + deprecation = "Use //python/entry_points:py_console_script_binary instead", + visibility = ["//visibility:public"], +) diff --git a/python/extensions/BUILD.bazel b/python/extensions/BUILD.bazel index 50d2b3bbdf..af18fa15b7 100644 --- a/python/extensions/BUILD.bazel +++ b/python/extensions/BUILD.bazel @@ -25,28 +25,47 @@ filegroup( ) bzl_library( - name = "pip_bzl", + name = "config", + srcs = ["config.bzl"], + visibility = ["//:__subpackages__"], + deps = [ + "//python/private:internal_config_repo", + "//python/private/pypi:deps", + "@bazel_features//:features", + ], +) + +bzl_library( + name = "pip", srcs = ["pip.bzl"], visibility = ["//:__subpackages__"], - deps = ["//python/private/pypi:pip_bzl"], + deps = ["//python/private/pypi:pip"], ) bzl_library( - name = "python_bzl", + name = "python", srcs = ["python.bzl"], visibility = ["//:__subpackages__"], - deps = [ - "//python/private:python_bzl", - ], + deps = ["//python/private:python"], ) -bzl_library( +alias( name = "config_bzl", - srcs = ["config.bzl"], + actual = ":config", + deprecation = "Use //python/extensions:config instead", + visibility = ["//:__subpackages__"], +) + +alias( + name = "pip_bzl", + actual = ":pip", + deprecation = "Use //python/extensions:pip instead", + visibility = ["//:__subpackages__"], +) + +alias( + name = "python_bzl", + actual = ":python", + deprecation = "Use //python/extensions:python instead", visibility = ["//:__subpackages__"], - deps = [ - "//python/private:internal_config_repo_bzl", - "//python/private/pypi:deps_bzl", - "@bazel_features//:features", - ], ) diff --git a/python/features.bzl b/python/features.bzl index f5d2c315cd..ee850ade9e 100644 --- a/python/features.bzl +++ b/python/features.bzl @@ -49,6 +49,15 @@ def _features_typedef(): ::: :::: + ::::{field} loadable_symbols + :type: dict[str, list[str]] + + A map of bzl paths to the list of public symbols they export. + + :::{versionadded} VERSION_NEXT_FEATURE + ::: + :::: + ::::{field} py_info_venv_symlinks True if the `PyInfo.venv_symlinks` field is available. @@ -81,23 +90,40 @@ def _features_typedef(): :::{versionadded} 1.9.0 :::: - - ::::{field} loadable_symbols - :type: dict[str, list[str]] - - A map of bzl paths to the list of public symbols they export. - - :::{versionadded} VERSION_NEXT_FEATURE - ::: - :::: """ _TARGETS = { "//command_line_option:build_runfile_links": True, "//command_line_option:enable_runfiles": True, "//command_line_option:extra_toolchains": True, + "//python/api:api": True, + "//python/api:executables": True, + "//python/api:libraries": True, "//python/cc:current_py_cc_headers_abi3": True, + "//python/cc:py_cc_toolchain": True, + "//python/cc:py_cc_toolchain_info": True, "//python/config_settings:venv": True, + "//python/entry_points:py_console_script_binary": True, + "//python/local_toolchains:repos": True, + "//python:defs": True, + "//python:features": True, + "//python:packaging": True, + "//python:pip": True, + "//python:proto": True, + "//python:py_binary": True, + "//python:py_cc_link_params_info": True, + "//python:py_exec_tools_info": True, + "//python:py_exec_tools_toolchain": True, + "//python:py_executable_info": True, + "//python:py_import": True, + "//python:py_info": True, + "//python:py_library": True, + "//python:py_runtime": True, + "//python:py_runtime_info": True, + "//python:py_runtime_pair": True, + "//python:py_test": True, + "//python:repositories": True, + "//python:versions": True, } _LOADABLE_SYMBOLS = { diff --git a/python/local_toolchains/BUILD.bazel b/python/local_toolchains/BUILD.bazel index 211f3e21a7..256dffb337 100644 --- a/python/local_toolchains/BUILD.bazel +++ b/python/local_toolchains/BUILD.bazel @@ -2,17 +2,24 @@ load("@bazel_skylib//:bzl_library.bzl", "bzl_library") package(default_visibility = ["//:__subpackages__"]) +filegroup( + name = "distribution", + srcs = glob(["**"]), +) + bzl_library( - name = "repos_bzl", + name = "repos", srcs = ["repos.bzl"], visibility = ["//visibility:public"], deps = [ - "//python/private:local_runtime_repo_bzl", - "//python/private:local_runtime_toolchains_repo_bzl", + "//python/private:local_runtime_repo", + "//python/private:local_runtime_toolchains_repo", ], ) -filegroup( - name = "distribution", - srcs = glob(["**"]), +alias( + name = "repos_bzl", + actual = ":repos", + deprecation = "Use //python/local_toolchains:repos instead", + visibility = ["//visibility:public"], ) diff --git a/python/pip_install/BUILD.bazel b/python/pip_install/BUILD.bazel index 665375cc5b..b3c1ab1c03 100644 --- a/python/pip_install/BUILD.bazel +++ b/python/pip_install/BUILD.bazel @@ -18,23 +18,6 @@ package( default_visibility = ["//:__subpackages__"], ) -bzl_library( - name = "pip_repository_bzl", - srcs = ["pip_repository.bzl"], - deps = [ - "//python/private/pypi:package_annotation_bzl", - "//python/private/pypi:pip_repository_bzl", - "//python/private/pypi:whl_config_repo_bzl", - "//python/private/pypi:whl_library_bzl", - ], -) - -bzl_library( - name = "requirements_bzl", - srcs = ["requirements.bzl"], - deps = ["//python/private/pypi:pip_compile_bzl"], -) - filegroup( name = "distribution", srcs = glob(["**"]), @@ -51,3 +34,26 @@ exports_files( glob(["*.bzl"]), visibility = ["//docs:__pkg__"], ) + +bzl_library( + name = "pip_repository", + srcs = ["pip_repository.bzl"], + deps = [ + "//python/private/pypi:package_annotation", + "//python/private/pypi:pip_repository", + "//python/private/pypi:whl_config_repo", + "//python/private/pypi:whl_library", + ], +) + +bzl_library( + name = "requirements", + srcs = ["requirements.bzl"], + deps = ["//python/private/pypi:pip_compile"], +) + +bzl_library( + name = "requirements_parser", + srcs = ["requirements_parser.bzl"], + deps = ["//python/private/pypi:parse_requirements_txt"], +) diff --git a/python/private/BUILD.bazel b/python/private/BUILD.bazel index 133c9c6444..c3d60ffaf1 100644 --- a/python/private/BUILD.bazel +++ b/python/private/BUILD.bazel @@ -18,8 +18,8 @@ load("//python:py_binary.bzl", "py_binary") load("//python:py_library.bzl", "py_library") load(":bazel_config_mode.bzl", "bazel_config_mode") load(":py_exec_tools_toolchain.bzl", "current_interpreter_executable") -load(":sentinel.bzl", "sentinel") -load(":stamp.bzl", "stamp_build_setting") +load(":sentinel_impl.bzl", "sentinel") +load(":stamp_impl.bzl", "stamp_build_setting") load(":uncachable_version_file.bzl", "define_uncachable_version_file") package( @@ -50,7 +50,7 @@ filegroup( ) filegroup( - name = "coverage_deps", + name = "coverage_deps_filegroup", srcs = ["coverage_deps.bzl"], visibility = ["//tools/private/update_deps:__pkg__"], ) @@ -62,853 +62,926 @@ filegroup( visibility = ["//python:__pkg__"], ) +alias( + name = "build_data_writer", + actual = select({ + "@platforms//os:windows": ":build_data_writer.ps1", + "//conditions:default": ":build_data_writer.sh", + }), +) + +define_uncachable_version_file( + name = "uncachable_version_file", +) + +# Needed to define bzl_library targets for docgen. (We don't define the +# bzl_library target here because it'd give our users a transitive dependency +# on Skylib.) +exports_files( + [ + "coverage.patch", + "py_package.bzl", + "py_wheel.bzl", + "version.bzl", + "reexports.bzl", + "stamp_impl.bzl", + "util.bzl", + ], + visibility = ["//:__subpackages__"], +) + +exports_files( + ["python_bootstrap_template.txt"], + # Not actually public. Only public because it's an implicit dependency of + # py_runtime. + visibility = ["//visibility:public"], +) + +filegroup( + name = "stage1_bootstrap_template", + srcs = ["stage1_bootstrap_template.sh"], + # Not actually public. Only public because it's an implicit dependency of + # py_runtime. + visibility = ["//visibility:public"], +) + +filegroup( + name = "stage2_bootstrap_template", + srcs = ["stage2_bootstrap_template.py"], + # Not actually public. Only public because it's an implicit dependency of + # py_runtime. + visibility = ["//visibility:public"], +) + +filegroup( + name = "site_init_template", + srcs = ["site_init_template.py"], + # Not actually public. Only public because it's an implicit dependency of + # py_runtime. + visibility = ["//visibility:public"], +) + +# NOTE: Windows builds don't use this bootstrap. Instead, a native Windows +# program locates some Python exe and runs `python.exe foo.zip` which +# runs the __main__.py in the zip file. +alias( + name = "bootstrap_template", + actual = select({ + ":is_script_bootstrap_enabled": "stage1_bootstrap_template.sh", + "//conditions:default": "python_bootstrap_template.txt", + }), + # Not actually public. Only public because it's an implicit dependency of + # py_runtime. + visibility = ["//visibility:public"], +) + +# Used to determine the use of `--stamp` in Starlark rules +stamp_build_setting(name = "stamp") + +config_setting( + name = "is_script_bootstrap_enabled", + flag_values = { + "//python/config_settings:bootstrap_impl": "script", + }, +) + +config_setting( + name = "is_bazel_config_mode_target", + flag_values = { + "//python/private:bazel_config_mode": "target", + }, +) + +alias( + name = "debugger_if_target_config", + actual = select({ + ":is_bazel_config_mode_target": "//python/config_settings:debugger", + "//conditions:default": "//python/private:empty", + }), +) + +bazel_config_mode(name = "bazel_config_mode") + +# This should only be set by analysis tests to expose additional metadata to +# aid testing, so a setting instead of a flag. +bool_flag( + name = "visible_for_testing", + build_setting_default = False, + # This is only because it is an implicit dependency by the toolchains. + visibility = ["//visibility:public"], +) + +# Used for py_console_script_gen rule +py_binary( + name = "py_console_script_gen_py", + srcs = ["py_console_script_gen.py"], + main = "py_console_script_gen.py", + visibility = [ + "//visibility:public", + ], +) + +py_binary( + name = "py_wheel_dist", + srcs = ["py_wheel_dist.py"], + visibility = ["//visibility:public"], +) + +py_library( + name = "py_console_script_gen_lib", + srcs = ["py_console_script_gen.py"], + imports = ["../.."], + visibility = [ + "//tests/entry_points:__pkg__", + ], +) + +# The current toolchain's interpreter as an excutable, usable with +# executable=True attributes. +current_interpreter_executable( + name = "current_interpreter_executable", + # Not actually public. Only public because it's an implicit dependency of + # py_exec_tools_toolchain. + visibility = ["//visibility:public"], +) + +py_library( + name = "empty", +) + +sentinel( + name = "sentinel", +) + +py_binary( + name = "sync_runtimes_manifest_workspace", + srcs = ["tools/sync_runtimes_manifest_workspace.py"], + visibility = ["//:__subpackages__"], +) + bzl_library( - name = "attr_builders_bzl", + name = "attr_builders", srcs = ["attr_builders.bzl"], deps = [ - ":builders_util_bzl", + ":builders_util", "@bazel_skylib//lib:types", ], ) bzl_library( - name = "attributes_bzl", + name = "attributes", srcs = ["attributes.bzl"], deps = [ - ":attr_builders_bzl", - ":common_bzl", - ":common_labels_bzl", - ":enum_bzl", - ":flags_bzl", - ":py_info_bzl", - ":py_internal_bzl", - ":reexports_bzl", - ":rule_builders_bzl", - ":rules_cc_srcs_bzl", + ":attr_builders", + ":common_labels", + ":enum", + ":flags", + ":py_info", + ":reexports", + ":rule_builders", "@bazel_skylib//lib:dicts", "@bazel_skylib//rules:common_settings", + "@rules_cc//cc/common", ], ) bzl_library( - name = "auth_bzl", + name = "auth", srcs = ["auth.bzl"], - deps = [":bazel_tools_bzl"], + deps = ["//python/private:bazel_tools"], ) +# @bazel_tools can't define bzl_library itself, so we just put a wrapper around it. +# Keep this target because Gazelle does not automatically generate +# bzl_library targets for external repositories like @bazel_tools. +# keep bzl_library( - name = "runtime_env_toolchain_bzl", - srcs = ["runtime_env_toolchain.bzl"], - deps = [ - ":config_settings_bzl", - ":py_exec_tools_toolchain_bzl", - ":toolchain_types_bzl", - "//python:py_runtime_bzl", - "//python:py_runtime_pair_bzl", + name = "bazel_tools", + srcs = [ + # This set of sources is overly broad, but it's the only public + # target available across Bazel versions that has all the necessary + # sources. + "@bazel_tools//tools:bzl_srcs", ], ) -alias( - name = "build_data_writer", - actual = select({ - "@platforms//os:windows": ":build_data_writer.ps1", - "//conditions:default": ":build_data_writer.sh", - }), -) - bzl_library( - name = "builders_bzl", + name = "builders", srcs = ["builders.bzl"], - deps = [ - "@bazel_skylib//lib:types", - ], + deps = ["@bazel_skylib//lib:types"], ) bzl_library( - name = "builders_util_bzl", + name = "builders_util", srcs = ["builders_util.bzl"], deps = [ - ":bzlmod_enabled_bzl", + ":bzlmod_enabled", "@bazel_skylib//lib:types", ], ) bzl_library( - name = "bzlmod_enabled_bzl", - srcs = ["bzlmod_enabled.bzl"], -) - -bzl_library( - name = "cc_helper_bzl", + name = "cc_helper", srcs = ["cc_helper.bzl"], - deps = [":py_internal_bzl"], + deps = [":py_internal"], ) bzl_library( - name = "common_bzl", + name = "common", srcs = ["common.bzl"], deps = [ - ":cc_helper_bzl", - ":py_cc_link_params_info_bzl", - ":py_info_bzl", - ":py_internal_bzl", - ":reexports_bzl", - ":rules_cc_srcs_bzl", + ":builders", + ":cc_helper", + ":py_cc_link_params_info", + ":py_info", + ":py_internal", + ":py_interpreter_program", + ":reexports", + ":toolchain_types", "@bazel_skylib//lib:paths", + "@rules_cc//cc/common", + "@rules_python_internal//:rules_python_config", ], ) bzl_library( - name = "common_labels_bzl", - srcs = ["common_labels.bzl"], -) - -bzl_library( - name = "config_settings_bzl", + name = "config_settings", srcs = ["config_settings.bzl"], deps = [ - ":version_bzl", + ":text_util", + ":version", "@bazel_skylib//lib:selects", "@bazel_skylib//rules:common_settings", ], ) bzl_library( - name = "coverage_deps_bzl", + name = "coverage_deps", srcs = ["coverage_deps.bzl"], deps = [ - ":bazel_tools_bzl", - ":repo_utils_bzl", - ":version_label_bzl", + ":repo_utils", + ":version_label", + "//python/private:bazel_tools", ], ) bzl_library( - name = "deprecation_bzl", - srcs = ["deprecation.bzl"], + name = "current_py_cc_headers", + srcs = ["current_py_cc_headers.bzl"], deps = [ - "@rules_python_internal//:rules_python_config_bzl", + ":toolchain_types", + "@rules_cc//cc/common", ], ) bzl_library( - name = "enum_bzl", - srcs = ["enum.bzl"], + name = "current_py_cc_libs", + srcs = ["current_py_cc_libs.bzl"], + deps = ["@rules_cc//cc/common"], ) bzl_library( - name = "envsubst_bzl", - srcs = ["envsubst.bzl"], + name = "deprecation", + srcs = ["deprecation.bzl"], + deps = ["@rules_python_internal//:rules_python_config"], ) bzl_library( - name = "flags_bzl", + name = "flags", srcs = ["flags.bzl"], deps = [ - ":enum_bzl", + ":enum", "@bazel_skylib//rules:common_settings", ], ) bzl_library( - name = "full_version_bzl", - srcs = ["full_version.bzl"], -) - -bzl_library( - name = "internal_config_repo_bzl", - srcs = ["internal_config_repo.bzl"], - deps = [ - ":pbs_manifest_bzl", - ":repo_utils_bzl", - ":text_util_bzl", - ], -) - -bzl_library( - name = "is_standalone_interpreter_bzl", - srcs = ["is_standalone_interpreter.bzl"], - deps = [ - ":repo_utils_bzl", - ], -) - -bzl_library( - name = "local_runtime_repo_bzl", - srcs = ["local_runtime_repo.bzl"], + name = "hermetic_runtime_repo_setup", + srcs = ["hermetic_runtime_repo_setup.bzl"], deps = [ - ":enum_bzl", - ":repo_utils.bzl", + ":py_exec_tools_toolchain", + ":version", + "//python:py_runtime", + "//python:py_runtime_pair", + "//python/cc:py_cc_toolchain", + "@rules_cc//cc:core_rules", ], ) bzl_library( - name = "local_runtime_toolchains_repo_bzl", - srcs = ["local_runtime_toolchains_repo.bzl"], + name = "internal_config_repo", + srcs = ["internal_config_repo.bzl"], deps = [ - ":repo_utils.bzl", - ":text_util_bzl", + ":repo_utils", + ":text_util", ], ) bzl_library( - name = "normalize_name_bzl", - srcs = ["normalize_name.bzl"], -) - -bzl_library( - name = "pbs_manifest_bzl", - srcs = ["pbs_manifest.bzl"], -) - -bzl_library( - name = "precompile_bzl", - srcs = ["precompile.bzl"], + name = "interpreter", + srcs = ["interpreter.bzl"], deps = [ - ":attributes_bzl", - ":py_internal_bzl", - ":py_interpreter_program_bzl", - ":toolchain_types_bzl", + ":common", + ":sentinel_impl", + ":toolchain_types", + "//python:py_runtime_info", "@bazel_skylib//lib:paths", ], ) bzl_library( - name = "platform_info_bzl", - srcs = ["platform_info.bzl"], -) - -bzl_library( - name = "python_bzl", - srcs = ["python.bzl"], - deps = [ - ":full_version_bzl", - ":pbs_manifest_bzl", - ":platform_info_bzl", - ":python_register_toolchains_bzl", - ":pythons_hub_bzl", - ":repo_utils_bzl", - ":toolchains_repo_bzl", - ":version_bzl", - "@bazel_features//:features", - ], + name = "is_standalone_interpreter", + srcs = ["is_standalone_interpreter.bzl"], + deps = [":repo_utils"], ) bzl_library( - name = "python_register_toolchains_bzl", - srcs = ["python_register_toolchains.bzl"], + name = "local_runtime_repo", + srcs = ["local_runtime_repo.bzl"], deps = [ - ":auth_bzl", - ":bazel_tools_bzl", - ":coverage_deps_bzl", - ":full_version_bzl", - ":python_repository_bzl", - ":repo_utils_bzl", - ":toolchains_repo_bzl", - "//python:versions_bzl", - "//python/private/pypi:deps_bzl", + ":enum", + ":repo_utils", ], ) bzl_library( - name = "python_repository_bzl", - srcs = ["python_repository.bzl"], + name = "local_runtime_repo_setup", + srcs = ["local_runtime_repo_setup.bzl"], deps = [ - ":auth_bzl", - ":repo_utils_bzl", - ":text_util_bzl", - "//python:versions_bzl", + "@bazel_skylib//lib:selects", + "@rules_cc//cc:core_rules", + "@rules_python//python:py_runtime", + "@rules_python//python:py_runtime_pair", + "@rules_python//python/cc:py_cc_toolchain", + "@rules_python//python/private:py_exec_tools_toolchain", ], ) bzl_library( - name = "python_register_multi_toolchains_bzl", - srcs = ["python_register_multi_toolchains.bzl"], + name = "local_runtime_toolchains_repo", + srcs = ["local_runtime_toolchains_repo.bzl"], deps = [ - ":python_register_toolchains_bzl", - ":toolchains_repo_bzl", - "//python:versions_bzl", + ":repo_utils", + ":text_util", ], ) bzl_library( - name = "pythons_hub_bzl", - srcs = ["pythons_hub.bzl"], + name = "precompile", + srcs = ["precompile.bzl"], deps = [ - ":pbs_manifest_bzl", - ":py_toolchain_suite_bzl", - ":text_util_bzl", - "//python:versions_bzl", + ":attributes", + ":flags", + ":py_interpreter_program", + ":toolchain_types", + "@bazel_skylib//rules:common_settings", ], ) bzl_library( - name = "py_binary_macro_bzl", + name = "py_binary_macro", srcs = ["py_binary_macro.bzl"], deps = [ - ":py_binary_rule_bzl", - ":py_executable_bzl", + ":py_binary_rule", + ":py_executable", ], ) bzl_library( - name = "py_binary_rule_bzl", + name = "py_binary_rule", srcs = ["py_binary_rule.bzl"], deps = [ - ":attributes_bzl", - ":py_executable_bzl", - ":rule_builders_bzl", - "@bazel_skylib//lib:dicts", + ":attributes", + ":py_executable", ], ) bzl_library( - name = "py_cc_link_params_info_bzl", + name = "py_cc_link_params_info", srcs = ["py_cc_link_params_info.bzl"], - deps = [ - ":rules_cc_srcs_bzl", - ":util_bzl", - ], + deps = ["@rules_cc//cc/common"], ) bzl_library( - name = "py_cc_toolchain_macro_bzl", + name = "py_cc_toolchain_macro", srcs = ["py_cc_toolchain_macro.bzl"], deps = [ - ":py_cc_toolchain_rule_bzl", + ":py_cc_toolchain_rule", + ":util", ], ) bzl_library( - name = "py_cc_toolchain_rule_bzl", + name = "py_cc_toolchain_rule", srcs = ["py_cc_toolchain_rule.bzl"], deps = [ - ":common_labels_bzl", - ":py_cc_toolchain_info_bzl", - ":rules_cc_srcs_bzl", - ":sentinel_bzl", - ":util_bzl", + ":common_labels", + ":py_cc_toolchain_info", + ":sentinel_impl", "@bazel_skylib//rules:common_settings", + "@rules_cc//cc/common", ], ) bzl_library( - name = "py_cc_toolchain_info_bzl", - srcs = ["py_cc_toolchain_info.bzl"], -) - -bzl_library( - name = "py_console_script_binary_bzl", - srcs = [ - "py_console_script_binary.bzl", - "py_console_script_gen.bzl", - ], - visibility = ["//python/entry_points:__pkg__"], + name = "py_console_script_binary", + srcs = ["py_console_script_binary.bzl"], deps = [ - "//python:py_binary_bzl", + ":py_console_script_gen", + "//python:py_binary", ], ) bzl_library( - name = "py_exec_tools_info_bzl", - srcs = ["py_exec_tools_info.bzl"], -) - -bzl_library( - name = "py_exec_tools_toolchain_bzl", + name = "py_exec_tools_toolchain", srcs = ["py_exec_tools_toolchain.bzl"], deps = [ - ":common_bzl", - ":common_labels_bzl", - ":py_exec_tools_info_bzl", - ":sentinel_bzl", - ":toolchain_types_bzl", + ":common_labels", + ":py_exec_tools_info", + ":sentinel_impl", + ":toolchain_types", "@bazel_skylib//lib:paths", "@bazel_skylib//rules:common_settings", ], ) bzl_library( - name = "py_executable_bzl", + name = "py_executable", srcs = ["py_executable.bzl"], deps = [ - ":attributes_bzl", - ":cc_helper_bzl", - ":common_bzl", - ":common_labels_bzl", - ":flags_bzl", - ":precompile_bzl", - ":py_cc_link_params_info_bzl", - ":py_executable_info_bzl", - ":py_info_bzl", - ":py_internal_bzl", - ":py_runtime_info_bzl", - ":rules_cc_srcs_bzl", - ":toolchain_types_bzl", - ":transition_labels_bzl", - ":venv_runfiles_bzl", + ":attr_builders", + ":attributes", + ":builders", + ":cc_helper", + ":common", + ":common_labels", + ":flags", + ":precompile", + ":py_cc_link_params_info", + ":py_executable_info", + ":py_info", + ":py_internal", + ":py_runtime_info", + ":reexports", + ":rule_builders", + ":toolchain_types", + ":transition_labels", + ":venv_runfiles", "@bazel_skylib//lib:dicts", "@bazel_skylib//lib:paths", "@bazel_skylib//lib:structs", "@bazel_skylib//rules:common_settings", + "@rules_cc//cc/common", + "@rules_python_internal//:rules_python_config", ], ) bzl_library( - name = "py_executable_info_bzl", - srcs = ["py_executable_info.bzl"], -) - -bzl_library( - name = "py_info_bzl", + name = "py_info", srcs = ["py_info.bzl"], deps = [ - ":builders_bzl", - ":reexports_bzl", + ":builders", + ":reexports", ], ) bzl_library( - name = "py_internal_bzl", + name = "py_internal", srcs = ["py_internal.bzl"], - deps = ["//tools/build_defs/python/private:py_internal_renamed_bzl"], + deps = ["//tools/build_defs/python/private:py_internal_renamed"], ) bzl_library( - name = "py_interpreter_program_bzl", + name = "py_interpreter_program", srcs = ["py_interpreter_program.bzl"], deps = [ - ":sentinel_bzl", + ":sentinel_impl", "@bazel_skylib//rules:common_settings", ], ) bzl_library( - name = "py_library_bzl", + name = "py_library", srcs = ["py_library.bzl"], deps = [ - ":attributes_bzl", - ":common_bzl", - ":common_labels_bzl", - ":flags_bzl", - ":normalize_name_bzl", - ":precompile_bzl", - ":py_cc_link_params_info_bzl", - ":py_internal_bzl", - ":rule_builders_bzl", - ":toolchain_types_bzl", - ":venv_runfiles_bzl", - ":version_bzl", + ":attr_builders", + ":attributes", + ":builders", + ":common", + ":common_labels", + ":flags", + ":normalize_name", + ":precompile", + ":py_cc_link_params_info", + ":py_info", + ":reexports", + ":rule_builders", + ":toolchain_types", + ":venv_runfiles", + ":version", "@bazel_skylib//lib:dicts", + "@bazel_skylib//lib:paths", "@bazel_skylib//rules:common_settings", ], ) bzl_library( - name = "py_library_macro_bzl", + name = "py_library_macro", srcs = ["py_library_macro.bzl"], - deps = [":py_library_rule_bzl"], + deps = [":py_library_rule"], ) bzl_library( - name = "py_library_rule_bzl", + name = "py_library_rule", srcs = ["py_library_rule.bzl"], - deps = [ - ":common_labels_bzl", - ":py_library_bzl", - ], + deps = [":py_library"], ) bzl_library( - name = "py_package_bzl", + name = "py_package", srcs = ["py_package.bzl"], - visibility = ["//:__subpackages__"], deps = [ - ":builders_bzl", - ":py_info_bzl", + ":builders", + ":py_info", ], ) bzl_library( - name = "py_runtime_info_bzl", - srcs = ["py_runtime_info.bzl"], -) - -bzl_library( - name = "py_repositories_bzl", + name = "py_repositories", srcs = ["py_repositories.bzl"], deps = [ - ":bazel_tools_bzl", - ":internal_config_repo_bzl", - ":pythons_hub_bzl", - "//python:versions_bzl", - "//python/private/pypi:deps_bzl", + ":internal_config_repo", + ":pythons_hub", + "//python:versions", + "//python/private:bazel_tools", + "//python/private/pypi:deps", ], ) bzl_library( - name = "py_runtime_macro_bzl", + name = "py_runtime_macro", srcs = ["py_runtime_macro.bzl"], - deps = [":py_runtime_rule_bzl"], + deps = [":py_runtime_rule"], ) bzl_library( - name = "py_runtime_rule_bzl", - srcs = ["py_runtime_rule.bzl"], - deps = [ - ":attributes_bzl", - ":common_labels_bzl", - ":flags_bzl", - ":py_internal_bzl", - ":py_runtime_info_bzl", - ":reexports_bzl", - ":rule_builders_bzl", - ":version_bzl", - "@bazel_skylib//lib:dicts", - "@bazel_skylib//lib:paths", - "@bazel_skylib//rules:common_settings", - ], + name = "py_runtime_pair_macro", + srcs = ["py_runtime_pair_macro.bzl"], + deps = [":py_runtime_pair_rule"], ) bzl_library( - name = "py_runtime_pair_macro_bzl", - srcs = ["py_runtime_pair_macro.bzl"], - visibility = ["//:__subpackages__"], - deps = [":py_runtime_pair_rule_bzl"], + name = "py_runtime_pair_rule", + srcs = ["py_runtime_pair_rule.bzl"], + deps = [ + ":common_labels", + ":reexports", + "//python:py_runtime_info", + "@bazel_skylib//rules:common_settings", + ], ) bzl_library( - name = "py_runtime_pair_rule_bzl", - srcs = ["py_runtime_pair_rule.bzl"], + name = "py_runtime_rule", + srcs = ["py_runtime_rule.bzl"], deps = [ - ":common_labels_bzl", - "//python:py_runtime_bzl", - "//python:py_runtime_info_bzl", + ":common_labels", + ":flags", + ":py_internal", + ":py_runtime_info", + ":reexports", + ":version", + "@bazel_skylib//lib:dicts", + "@bazel_skylib//lib:paths", "@bazel_skylib//rules:common_settings", ], ) bzl_library( - name = "py_test_macro_bzl", + name = "py_test_macro", srcs = ["py_test_macro.bzl"], deps = [ - ":py_executable_bzl", - ":py_test_rule_bzl", + ":py_executable", + ":py_test_rule", ], ) bzl_library( - name = "py_test_rule_bzl", + name = "py_test_rule", srcs = ["py_test_rule.bzl"], deps = [ - ":attributes_bzl", - ":common_bzl", - ":py_executable_bzl", - ":rule_builders_bzl", - "@bazel_skylib//lib:dicts", + ":attributes", + ":common", + ":py_executable", ], ) bzl_library( - name = "py_toolchain_suite_bzl", + name = "py_toolchain_suite", srcs = ["py_toolchain_suite.bzl"], deps = [ - ":config_settings_bzl", - ":text_util_bzl", - ":toolchain_types_bzl", + ":text_util", + ":toolchain_types", "@bazel_skylib//lib:selects", + "@platforms//host:constraints_lib", ], ) bzl_library( - name = "py_wheel_bzl", + name = "py_wheel", srcs = ["py_wheel.bzl"], - visibility = ["//:__subpackages__"], deps = [ - ":py_package_bzl", - ":stamp_bzl", - ":transition_labels_bzl", - ":version_bzl", + ":attributes", + ":py_info", + ":py_package", + ":rule_builders", + ":stamp_impl", + ":transition_labels", + ":version", ], ) bzl_library( - name = "reexports_bzl", - srcs = ["reexports.bzl"], - visibility = [ - "//:__subpackages__", - ], + name = "python", + srcs = ["python.bzl"], deps = [ - ":bazel_tools_bzl", - "@rules_python_internal//:rules_python_config_bzl", + ":auth", + ":full_version", + ":pbs_manifest", + ":platform_info", + ":python_register_toolchains", + ":pythons_hub", + ":repo_utils", + ":toolchains_repo", + ":version", + "//python:versions", + "@bazel_features//:features", ], ) bzl_library( - name = "repo_utils_bzl", - srcs = ["repo_utils.bzl"], + name = "python_register_multi_toolchains", + srcs = ["python_register_multi_toolchains.bzl"], + deps = [ + ":python_register_toolchains", + ":toolchains_repo", + "//python:versions", + ], ) bzl_library( - name = "rule_builders_bzl", - srcs = ["rule_builders.bzl"], + name = "python_register_toolchains", + srcs = ["python_register_toolchains.bzl"], deps = [ - ":builders_bzl", - ":builders_util_bzl", - "@bazel_skylib//lib:types", + ":coverage_deps", + ":full_version", + ":python_repository", + ":repo_utils", + ":toolchains_repo", + "//python:versions", ], ) bzl_library( - name = "runtimes_manifest_workspace_bzl", - srcs = ["runtimes_manifest_workspace.bzl"], + name = "python_repository", + srcs = ["python_repository.bzl"], + deps = [ + ":auth", + ":repo_utils", + ":text_util", + "//python:versions", + ], ) bzl_library( - name = "sentinel_bzl", - srcs = ["sentinel.bzl"], + name = "pythons_hub", + srcs = ["pythons_hub.bzl"], + deps = [ + ":pbs_manifest", + ":text_util", + ":toolchains_repo", + "//python:versions", + ], ) bzl_library( - name = "stamp_bzl", - srcs = ["stamp.bzl"], - visibility = ["//:__subpackages__"], + name = "reexports", + srcs = ["reexports.bzl"], + deps = ["@rules_python_internal//:rules_python_config"], ) bzl_library( - name = "text_util_bzl", - srcs = ["text_util.bzl"], + name = "repl", + srcs = ["repl.bzl"], + deps = ["//python:py_binary"], ) bzl_library( - name = "toolchains_repo_bzl", - srcs = ["toolchains_repo.bzl"], + name = "rule_builders", + srcs = ["rule_builders.bzl"], deps = [ - ":repo_utils_bzl", - ":text_util_bzl", - "//python:versions_bzl", + ":builders_util", + "@bazel_skylib//lib:types", ], ) +# Keep this target because Gazelle does not automatically generate or +# maintain composite bzl_library targets. We need this to group external +# dependency sources. +# keep bzl_library( - name = "toolchain_types_bzl", - srcs = ["toolchain_types.bzl"], + name = "rules_cc_srcs", + srcs = [ + # rules_cc 0.0.13 and earlier load cc_proto_libary (and thus protobuf@), + # but their bzl srcs targets don't transitively refer to protobuf. + "@com_google_protobuf//:bzl_srcs", + # NOTE: As of rules_cc 0.10, cc:bzl_srcs no longer contains + # everything and sub-targets must be used instead + "@rules_cc//cc:bzl_srcs", + "@rules_cc//cc/common", + "@rules_cc//cc/toolchains:toolchain_rules", + ], + deps = [ + ":bazel_tools", + "@rules_cc//cc/common", + ], ) bzl_library( - name = "transition_labels_bzl", - srcs = ["transition_labels.bzl"], - deps = [ - ":common_labels_bzl", - "@bazel_skylib//lib:collections", - "@rules_python_internal//:extra_transition_settings_bzl", - ], + name = "runtime_env_repo", + srcs = ["runtime_env_repo.bzl"], + deps = [":repo_utils"], ) bzl_library( - name = "util_bzl", - srcs = ["util.bzl"], + name = "runtime_env_toolchain", + srcs = ["runtime_env_toolchain.bzl"], deps = [ - ":py_internal_bzl", - "@bazel_skylib//lib:types", + ":config_settings", + ":py_exec_tools_toolchain", + ":toolchain_types", + "//python:py_runtime", + "//python:py_runtime_pair", + "//python/cc:py_cc_toolchain", + "@rules_cc//cc:core_rules", ], ) -define_uncachable_version_file( - name = "uncachable_version_file", -) - bzl_library( - name = "version_bzl", - srcs = ["version.bzl"], + name = "toolchain_aliases", + srcs = ["toolchain_aliases.bzl"], + deps = [ + "//python:versions", + "@bazel_skylib//lib:selects", + ], ) bzl_library( - name = "version_label_bzl", - srcs = ["version_label.bzl"], + name = "toolchains_repo", + srcs = ["toolchains_repo.bzl"], + deps = [ + ":repo_utils", + ":text_util", + "//python:versions", + ], ) -# @bazel_tools can't define bzl_library itself, so we just put a wrapper around it. bzl_library( - name = "bazel_tools_bzl", - srcs = [ - # This set of sources is overly broad, but it's the only public - # target available across Bazel versions that has all the necessary - # sources. - "@bazel_tools//tools:bzl_srcs", + name = "transition_labels", + srcs = ["transition_labels.bzl"], + deps = [ + ":common_labels", + "@bazel_skylib//lib:collections", + "@rules_python_internal//:extra_transition_settings", ], ) bzl_library( - name = "rules_cc_srcs_bzl", - srcs = [ - # rules_cc 0.0.13 and earlier load cc_proto_libary (and thus protobuf@), - # but their bzl srcs targets don't transitively refer to protobuf. - "@com_google_protobuf//:bzl_srcs", - # NOTE: As of rules_cc 0.10, cc:bzl_srcs no longer contains - # everything and sub-targets must be used instead - "@rules_cc//cc:bzl_srcs", - "@rules_cc//cc/common", - "@rules_cc//cc/toolchains:toolchain_rules", - ], + name = "util", + srcs = ["util.bzl"], deps = [ - ":bazel_tools_bzl", - "@rules_cc//cc/common", + ":py_internal", + "@bazel_skylib//lib:types", ], ) bzl_library( - name = "venv_runfiles_bzl", + name = "venv_runfiles", srcs = ["venv_runfiles.bzl"], deps = [ - ":common_bzl", - ":py_info_bzl", - ":py_internal_bzl", + ":common", + ":py_info", + ":util", "@bazel_skylib//lib:paths", ], ) -# Needed to define bzl_library targets for docgen. (We don't define the -# bzl_library target here because it'd give our users a transitive dependency -# on Skylib.) -exports_files( - [ - "coverage.patch", - "py_package.bzl", - "py_wheel.bzl", - "version.bzl", - "reexports.bzl", - "stamp.bzl", - "util.bzl", - ], - visibility = ["//:__subpackages__"], +bzl_library( + name = "bzlmod_enabled", + srcs = ["bzlmod_enabled.bzl"], ) -exports_files( - ["python_bootstrap_template.txt"], - # Not actually public. Only public because it's an implicit dependency of - # py_runtime. - visibility = ["//visibility:public"], +bzl_library( + name = "common_labels", + srcs = ["common_labels.bzl"], ) -filegroup( - name = "stage1_bootstrap_template", - srcs = ["stage1_bootstrap_template.sh"], - # Not actually public. Only public because it's an implicit dependency of - # py_runtime. - visibility = ["//visibility:public"], +bzl_library( + name = "enum", + srcs = ["enum.bzl"], ) -filegroup( - name = "stage2_bootstrap_template", - srcs = ["stage2_bootstrap_template.py"], - # Not actually public. Only public because it's an implicit dependency of - # py_runtime. - visibility = ["//visibility:public"], +bzl_library( + name = "envsubst", + srcs = ["envsubst.bzl"], ) -filegroup( - name = "site_init_template", - srcs = ["site_init_template.py"], - # Not actually public. Only public because it's an implicit dependency of - # py_runtime. - visibility = ["//visibility:public"], +bzl_library( + name = "full_version", + srcs = ["full_version.bzl"], ) -# NOTE: Windows builds don't use this bootstrap. Instead, a native Windows -# program locates some Python exe and runs `python.exe foo.zip` which -# runs the __main__.py in the zip file. -alias( - name = "bootstrap_template", - actual = select({ - ":is_script_bootstrap_enabled": "stage1_bootstrap_template.sh", - "//conditions:default": "python_bootstrap_template.txt", - }), - # Not actually public. Only public because it's an implicit dependency of - # py_runtime. - visibility = ["//visibility:public"], +bzl_library( + name = "normalize_name", + srcs = ["normalize_name.bzl"], ) -# Used to determine the use of `--stamp` in Starlark rules -stamp_build_setting(name = "stamp") +bzl_library( + name = "pbs_manifest", + srcs = ["pbs_manifest.bzl"], +) -config_setting( - name = "is_script_bootstrap_enabled", - flag_values = { - "//python/config_settings:bootstrap_impl": "script", - }, +bzl_library( + name = "platform_info", + srcs = ["platform_info.bzl"], ) -config_setting( - name = "is_bazel_config_mode_target", - flag_values = { - "//python/private:bazel_config_mode": "target", - }, +bzl_library( + name = "py_cc_toolchain_info", + srcs = ["py_cc_toolchain_info.bzl"], ) -alias( - name = "debugger_if_target_config", - actual = select({ - ":is_bazel_config_mode_target": "//python/config_settings:debugger", - "//conditions:default": "//python/private:empty", - }), +bzl_library( + name = "py_console_script_gen", + srcs = ["py_console_script_gen.bzl"], ) -bazel_config_mode(name = "bazel_config_mode") +bzl_library( + name = "py_exec_tools_info", + srcs = ["py_exec_tools_info.bzl"], +) -# This should only be set by analysis tests to expose additional metadata to -# aid testing, so a setting instead of a flag. -bool_flag( - name = "visible_for_testing", - build_setting_default = False, - # This is only because it is an implicit dependency by the toolchains. - visibility = ["//visibility:public"], +bzl_library( + name = "py_executable_info", + srcs = ["py_executable_info.bzl"], ) -# Used for py_console_script_gen rule -py_binary( - name = "py_console_script_gen_py", - srcs = ["py_console_script_gen.py"], - main = "py_console_script_gen.py", - visibility = [ - "//visibility:public", - ], +bzl_library( + name = "py_runtime_info", + srcs = ["py_runtime_info.bzl"], ) -py_binary( - name = "py_wheel_dist", - srcs = ["py_wheel_dist.py"], - visibility = ["//visibility:public"], +bzl_library( + name = "repo_utils", + srcs = ["repo_utils.bzl"], ) -py_library( - name = "py_console_script_gen_lib", - srcs = ["py_console_script_gen.py"], - imports = ["../.."], - visibility = [ - "//tests/entry_points:__pkg__", - ], +bzl_library( + name = "runtimes_manifest_workspace", + srcs = ["runtimes_manifest_workspace.bzl"], ) -# The current toolchain's interpreter as an excutable, usable with -# executable=True attributes. -current_interpreter_executable( - name = "current_interpreter_executable", - # Not actually public. Only public because it's an implicit dependency of - # py_exec_tools_toolchain. - visibility = ["//visibility:public"], +bzl_library( + name = "sentinel_impl", + srcs = ["sentinel_impl.bzl"], ) -py_library( - name = "empty", +bzl_library( + name = "stamp_impl", + srcs = ["stamp_impl.bzl"], ) -sentinel( - name = "sentinel", +bzl_library( + name = "text_util", + srcs = ["text_util.bzl"], ) -py_binary( - name = "sync_runtimes_manifest_workspace", - srcs = ["tools/sync_runtimes_manifest_workspace.py"], - visibility = ["//:__subpackages__"], +bzl_library( + name = "toolchain_types", + srcs = ["toolchain_types.bzl"], +) + +bzl_library( + name = "version", + srcs = ["version.bzl"], +) + +bzl_library( + name = "version_label", + srcs = ["version_label.bzl"], +) + +bzl_library( + name = "visibility", + srcs = ["visibility.bzl"], ) diff --git a/python/private/api/BUILD.bazel b/python/private/api/BUILD.bazel index 0826b85d9b..b3cc7360a7 100644 --- a/python/private/api/BUILD.bazel +++ b/python/private/api/BUILD.bazel @@ -25,24 +25,21 @@ filegroup( ) py_common_api( - name = "py_common_api", + name = "py_common_api_impl", # NOTE: Not actually public. Implicit dependency of public rules. visibility = ["//visibility:public"], ) bzl_library( - name = "api_bzl", - srcs = ["api.bzl"], + name = "py_common_api", + srcs = ["py_common_api.bzl"], deps = [ - "//python/private:py_info_bzl", + ":api", + "//python/private:py_info", ], ) bzl_library( - name = "py_common_api_bzl", - srcs = ["py_common_api.bzl"], - deps = [ - ":api_bzl", - "//python/private:py_info_bzl", - ], + name = "api", + srcs = ["api.bzl"], ) diff --git a/python/private/api/api.bzl b/python/private/api/api.bzl index 44f9ab4e77..2a1f79fdf5 100644 --- a/python/private/api/api.bzl +++ b/python/private/api/api.bzl @@ -13,7 +13,7 @@ # limitations under the License. """Implementation of py_api.""" -_PY_COMMON_API_LABEL = Label("//python/private/api:py_common_api") +_PY_COMMON_API_LABEL = Label("//python/private/api:py_common_api_impl") ApiImplInfo = provider( doc = "Provider to hold an API implementation", diff --git a/python/private/internal_config_repo.bzl b/python/private/internal_config_repo.bzl index 72970cf100..9bae0a5821 100644 --- a/python/private/internal_config_repo.bzl +++ b/python/private/internal_config_repo.bzl @@ -49,12 +49,12 @@ package( ) bzl_library( - name = "extra_transition_settings_bzl", + name = "extra_transition_settings", srcs = ["extra_transition_settings.bzl"], ) bzl_library( - name = "rules_python_config_bzl", + name = "rules_python_config", srcs = ["rules_python_config.bzl"], ) """ diff --git a/python/private/interpreter.bzl b/python/private/interpreter.bzl index c66d3dc21e..b281be9639 100644 --- a/python/private/interpreter.bzl +++ b/python/private/interpreter.bzl @@ -17,7 +17,7 @@ load("@bazel_skylib//lib:paths.bzl", "paths") load("//python:py_runtime_info.bzl", "PyRuntimeInfo") load(":common.bzl", "runfiles_root_path") -load(":sentinel.bzl", "SentinelInfo") +load(":sentinel_impl.bzl", "SentinelInfo") load(":toolchain_types.bzl", "TARGET_TOOLCHAIN_TYPE") def _interpreter_binary_impl(ctx): diff --git a/python/private/py_cc_toolchain_rule.bzl b/python/private/py_cc_toolchain_rule.bzl index b89ea0e6b0..315de2d3f2 100644 --- a/python/private/py_cc_toolchain_rule.bzl +++ b/python/private/py_cc_toolchain_rule.bzl @@ -22,7 +22,7 @@ load("@bazel_skylib//rules:common_settings.bzl", "BuildSettingInfo") load("@rules_cc//cc/common:cc_info.bzl", "CcInfo") load(":common_labels.bzl", "labels") load(":py_cc_toolchain_info.bzl", "PyCcToolchainInfo") -load(":sentinel.bzl", "SentinelInfo") +load(":sentinel_impl.bzl", "SentinelInfo") def _py_cc_toolchain_impl(ctx): if ctx.attr.libs: diff --git a/python/private/py_exec_tools_toolchain.bzl b/python/private/py_exec_tools_toolchain.bzl index ec8d4e53d0..d126262033 100644 --- a/python/private/py_exec_tools_toolchain.bzl +++ b/python/private/py_exec_tools_toolchain.bzl @@ -18,7 +18,7 @@ load("@bazel_skylib//lib:paths.bzl", "paths") load("@bazel_skylib//rules:common_settings.bzl", "BuildSettingInfo") load(":common_labels.bzl", "labels") load(":py_exec_tools_info.bzl", "PyExecToolsInfo") -load(":sentinel.bzl", "SentinelInfo") +load(":sentinel_impl.bzl", "SentinelInfo") load(":toolchain_types.bzl", "TARGET_TOOLCHAIN_TYPE") def _py_exec_tools_toolchain_impl(ctx): diff --git a/python/private/py_interpreter_program.bzl b/python/private/py_interpreter_program.bzl index 7eb3e28bd9..1f1abb1bfe 100644 --- a/python/private/py_interpreter_program.bzl +++ b/python/private/py_interpreter_program.bzl @@ -15,7 +15,7 @@ """Internal only bootstrap level binary-like rule.""" load("@bazel_skylib//rules:common_settings.bzl", "BuildSettingInfo") -load("//python/private:sentinel.bzl", "SentinelInfo") +load("//python/private:sentinel_impl.bzl", "SentinelInfo") PyInterpreterProgramInfo = provider( doc = "Information about how to run a program with an external interpreter.", diff --git a/python/private/py_wheel.bzl b/python/private/py_wheel.bzl index 1ca344c086..b622411c56 100644 --- a/python/private/py_wheel.bzl +++ b/python/private/py_wheel.bzl @@ -18,7 +18,7 @@ load(":attributes.bzl", "CONFIG_SETTINGS_ATTR", "apply_config_settings_attr") load(":py_info.bzl", "PyInfo") load(":py_package.bzl", "py_package_lib") load(":rule_builders.bzl", "ruleb") -load(":stamp.bzl", "is_stamping_enabled") +load(":stamp_impl.bzl", "is_stamping_enabled") load(":transition_labels.bzl", "TRANSITION_LABELS") load(":version.bzl", "version") diff --git a/python/private/pypi/BUILD.bazel b/python/private/pypi/BUILD.bazel index 23f636c870..677154ee46 100644 --- a/python/private/pypi/BUILD.bazel +++ b/python/private/pypi/BUILD.bazel @@ -74,501 +74,507 @@ filegroup( visibility = ["//tools/private/update_deps:__pkg__"], ) -# Keep sorted by library name and keep the files named by the main symbol they export - -bzl_library( - name = "argparse_bzl", - srcs = ["argparse.bzl"], -) - -bzl_library( - name = "attrs_bzl", - srcs = ["attrs.bzl"], -) - bzl_library( - name = "config_settings_bzl", + name = "config_settings", srcs = ["config_settings.bzl"], - deps = [ - ":flags_bzl", - "//python/private:common_labels_bzl", - "//python/private:flags_bzl", - "@bazel_skylib//lib:selects", - ], + deps = ["@bazel_skylib//lib:selects"], ) bzl_library( - name = "deps_bzl", + name = "deps", srcs = ["deps.bzl"], - deps = [ - "//python/private:bazel_tools_bzl", - ], + deps = ["//python/private:bazel_tools"], ) bzl_library( - name = "env_marker_info_bzl", - srcs = ["env_marker_info.bzl"], -) - -bzl_library( - name = "env_marker_setting_bzl", + name = "env_marker_setting", srcs = ["env_marker_setting.bzl"], deps = [ - ":env_marker_info_bzl", - ":pep508_env_bzl", - ":pep508_evaluate_bzl", - "//python/private:common_labels_bzl", - "//python/private:toolchain_types_bzl", + ":env_marker_info", + ":pep508_env", + ":pep508_evaluate", + "//python/private:common_labels", + "//python/private:toolchain_types", "@bazel_skylib//rules:common_settings", ], ) bzl_library( - name = "evaluate_markers_bzl", + name = "evaluate_markers", srcs = ["evaluate_markers.bzl"], deps = [ - ":deps_bzl", - ":pep508_evaluate_bzl", - ":pep508_requirement_bzl", + ":pep508_evaluate", + ":pep508_requirement", ], ) +# keep bzl_library( - name = "extension_bzl", + name = "extension", srcs = ["extension.bzl"], deps = [ - ":hub_builder_bzl", - ":hub_repository_bzl", - ":parse_whl_name_bzl", - ":pep508_env_bzl", - ":pip_repository_attrs_bzl", - ":platform_bzl", - ":pypi_cache_bzl", - ":simpleapi_download_bzl", - ":unified_hub_repo_bzl", - ":whl_library_bzl", - "//python/private:auth_bzl", - "//python/private:normalize_name_bzl", - "//python/private:repo_utils_bzl", - "@bazel_features//:features", - "@pythons_hub//:interpreters_bzl", - "@pythons_hub//:versions_bzl", - "@rules_python_internal//:rules_python_config_bzl", + ":hub_builder", + ":hub_repository", + ":parse_whl_name", + ":pep508_env", + ":pip_repository_attrs", + ":platform", + ":pypi_cache", + ":simpleapi_download", + ":unified_hub_repo", + ":whl_library", + "//python/private:auth", + "//python/private:normalize_name", + "//python/private:repo_utils", + "@pythons_hub//:interpreters", + "@pythons_hub//:versions", + "@rules_python_internal//:rules_python_config", "@toml.bzl//:toml", ], ) bzl_library( - name = "flags_bzl", + name = "flags", srcs = ["flags.bzl"], deps = [ - ":env_marker_info.bzl", - ":pep508_env_bzl", + ":env_marker_info", + ":pep508_env", + "//python/private:common_labels", "@bazel_skylib//rules:common_settings", ], ) bzl_library( - name = "generate_whl_library_build_bazel_bzl", - srcs = ["generate_whl_library_build_bazel.bzl"], + name = "generate_group_library_build_bazel", + srcs = ["generate_group_library_build_bazel.bzl"], deps = [ - "//python/private:text_util_bzl", + ":labels", + "//python/private:normalize_name", + "//python/private:text_util", ], ) bzl_library( - name = "generate_group_library_build_bazel_bzl", - srcs = ["generate_group_library_build_bazel.bzl"], - deps = [ - ":labels_bzl", - "//python/private:normalize_name_bzl", - ], + name = "generate_whl_library_build_bazel", + srcs = ["generate_whl_library_build_bazel.bzl"], + deps = ["//python/private:text_util"], ) bzl_library( - name = "hub_builder_bzl", + name = "hub_builder", srcs = ["hub_builder.bzl"], - visibility = ["//:__subpackages__"], deps = [ - ":attrs_bzl", - ":parse_requirements_bzl", - ":pep508_env_bzl", - ":pep508_evaluate_bzl", - ":python_tag_bzl", - ":requirements_files_by_platform_bzl", - ":whl_config_setting_bzl", - ":whl_repo_name_bzl", - "//python/private:envsubst_bzl", - "//python/private:full_version_bzl", - "//python/private:normalize_name_bzl", - "//python/private:text_util_bzl", - "//python/private:version_bzl", - "//python/private:version_label_bzl", + ":attrs", + ":parse_requirements", + ":pep508_env", + ":pep508_evaluate", + ":python_tag", + ":requirements_files_by_platform", + ":whl_config_setting", + ":whl_repo_name", + "//python/private:envsubst", + "//python/private:full_version", + "//python/private:normalize_name", + "//python/private:repo_utils", + "//python/private:text_util", + "//python/private:version", + "//python/private:version_label", ], ) bzl_library( - name = "hub_repository_bzl", + name = "hub_repository", srcs = ["hub_repository.bzl"], - visibility = ["//:__subpackages__"], deps = [ - ":render_pkg_aliases_bzl", - "//python/private:text_util_bzl", + ":render_pkg_aliases", + ":whl_config_setting", + "//python/private:text_util", ], ) bzl_library( - name = "index_sources_bzl", - srcs = ["index_sources.bzl"], -) - -bzl_library( - name = "labels_bzl", - srcs = ["labels.bzl"], -) - -bzl_library( - name = "missing_package_bzl", + name = "missing_package", srcs = ["missing_package.bzl"], deps = [ - "//python/private:py_info_bzl", - "//python/private:reexports_bzl", + "//python/private:py_info", + "//python/private:reexports", ], ) bzl_library( - name = "multi_pip_parse_bzl", + name = "multi_pip_parse", srcs = ["multi_pip_parse.bzl"], deps = [ - ":pip_repository_bzl", - "//python/private:text_util_bzl", + ":pip_repository", + "//python/private:text_util", ], ) bzl_library( - name = "package_annotation_bzl", - srcs = ["package_annotation.bzl"], + name = "namespace_pkgs", + srcs = ["namespace_pkgs.bzl"], + deps = ["@bazel_skylib//rules:copy_file"], ) bzl_library( - name = "parse_requirements_bzl", + name = "parse_requirements", srcs = ["parse_requirements.bzl"], deps = [ - ":argparse_bzl", - ":index_sources_bzl", - ":parse_requirements_txt_bzl", - ":pep508_evaluate_bzl", - ":pep508_requirement_bzl", - ":pypi_repo_utils_bzl", - ":requirements_files_by_platform_bzl", - ":select_whl_bzl", - "//python/private:normalize_name_bzl", - "//python/private:repo_utils_bzl", - "//python/uv/private:uv_lock_to_requirements_bzl", + ":argparse", + ":index_sources", + ":parse_requirements_txt", + ":pep508_evaluate", + ":pep508_requirement", + ":select_whl", + "//python/private:normalize_name", + "//python/private:repo_utils", + "//python/uv/private:uv_lock_to_requirements", ], ) bzl_library( - name = "parse_requirements_txt_bzl", - srcs = ["parse_requirements_txt.bzl"], -) - -bzl_library( - name = "parse_simpleapi_html_bzl", + name = "parse_simpleapi_html", srcs = ["parse_simpleapi_html.bzl"], deps = [ - ":version_from_filename_bzl", - "//python/private:normalize_name_bzl", + ":version_from_filename", + "//python/private:normalize_name", ], ) bzl_library( - name = "parse_whl_name_bzl", - srcs = ["parse_whl_name.bzl"], -) - -bzl_library( - name = "patch_whl_bzl", + name = "patch_whl", srcs = ["patch_whl.bzl"], deps = [ - ":parse_whl_name_bzl", - "//python/private:repo_utils_bzl", - "@rules_python_internal//:rules_python_config_bzl", + ":parse_whl_name", + "//python/private:repo_utils", + "@rules_python_internal//:rules_python_config", ], ) bzl_library( - name = "pep508_deps_bzl", + name = "pep508_deps", srcs = ["pep508_deps.bzl"], deps = [ - ":pep508_env_bzl", - ":pep508_evaluate_bzl", - ":pep508_requirement_bzl", - "//python/private:normalize_name_bzl", + ":pep508_env", + ":pep508_evaluate", + ":pep508_requirement", + "//python/private:normalize_name", ], ) bzl_library( - name = "pep508_env_bzl", + name = "pep508_env", srcs = ["pep508_env.bzl"], deps = [ - "//python/private:normalize_name_bzl", - "//python/private:version_bzl", + "//python/private:normalize_name", + "//python/private:version", ], ) bzl_library( - name = "pep508_evaluate_bzl", + name = "pep508_evaluate", srcs = ["pep508_evaluate.bzl"], deps = [ - "//python/private:enum_bzl", - "//python/private:version_bzl", + "//python/private:enum", + "//python/private:version", ], ) bzl_library( - name = "pep508_requirement_bzl", + name = "pep508_requirement", srcs = ["pep508_requirement.bzl"], - deps = [ - "//python/private:normalize_name_bzl", - ], + deps = ["//python/private:normalize_name"], ) bzl_library( - name = "pip_bzl", + name = "pip", srcs = ["pip.bzl"], - deps = [ - ":extension_bzl", - ], + deps = [":extension"], ) bzl_library( - name = "pip_compile_bzl", + name = "pip_compile", srcs = ["pip_compile.bzl"], deps = [ - ":deps_bzl", - "//python:py_binary_bzl", - "//python:py_test_bzl", + "//python:py_binary", + "//python:py_test", ], ) bzl_library( - name = "pip_repository_bzl", + name = "pip_repository", srcs = ["pip_repository.bzl"], deps = [ - ":attrs_bzl", - ":parse_requirements_bzl", - ":pep508_env_bzl", - ":pip_repository_attrs_bzl", - ":pypi_repo_utils_bzl", - ":render_pkg_aliases_bzl", - ":whl_config_setting_bzl", - "//python/private:normalize_name_bzl", - "//python/private:repo_utils_bzl", - "//python/private:text_util_bzl", + ":parse_requirements", + ":pep508_env", + ":pip_repository_attrs", + ":pypi_repo_utils", + ":render_pkg_aliases", + ":requirements_files_by_platform", + "//python/private:normalize_name", + "//python/private:repo_utils", + "//python/private:text_util", "@bazel_skylib//lib:sets", ], ) bzl_library( - name = "pip_repository_attrs_bzl", + name = "pip_repository_attrs", srcs = ["pip_repository_attrs.bzl"], + deps = [":attrs"], ) bzl_library( - name = "pkg_aliases_bzl", + name = "pkg_aliases", srcs = ["pkg_aliases.bzl"], deps = [ - ":labels_bzl", - "//python/private:common_labels_bzl", - "//python/private:text_util_bzl", + ":labels", + "//python/private:common_labels", + "//python/private:text_util", "@bazel_skylib//lib:selects", ], ) bzl_library( - name = "platform_bzl", - srcs = ["platform.bzl"], -) - -bzl_library( - name = "pypi_cache_bzl", + name = "pypi_cache", srcs = ["pypi_cache.bzl"], - deps = [ - ":version_from_filename_bzl", - ], + deps = [":version_from_filename"], ) bzl_library( - name = "pypi_repo_utils_bzl", + name = "pypi_repo_utils", srcs = ["pypi_repo_utils.bzl"], deps = [ - "//python/private:repo_utils_bzl", + "//python/private:repo_utils", + "//python/private:util", "@bazel_skylib//lib:types", ], ) bzl_library( - name = "python_tag_bzl", + name = "python_tag", srcs = ["python_tag.bzl"], - deps = [ - "//python/private:version_bzl", - ], + deps = ["//python/private:version"], ) bzl_library( - name = "render_pkg_aliases_bzl", + name = "render_pkg_aliases", srcs = ["render_pkg_aliases.bzl"], deps = [ - ":generate_group_library_build_bazel_bzl", - ":whl_config_setting_bzl", - "//python/private:normalize_name_bzl", - "//python/private:text_util_bzl", + ":generate_group_library_build_bazel", + "//python/private:normalize_name", + "//python/private:text_util", ], ) bzl_library( - name = "requirements_files_by_platform_bzl", + name = "requirements_files_by_platform", srcs = ["requirements_files_by_platform.bzl"], deps = [ - ":argparse_bzl", - ":whl_target_platforms_bzl", + ":argparse", + ":whl_target_platforms", ], ) bzl_library( - name = "select_whl_bzl", + name = "select_whl", srcs = ["select_whl.bzl"], deps = [ - ":parse_whl_name_bzl", - ":python_tag_bzl", - "//python/private:version_bzl", + ":parse_whl_name", + ":python_tag", + "//python/private:version", ], ) bzl_library( - name = "simpleapi_download_bzl", + name = "simpleapi_download", srcs = ["simpleapi_download.bzl"], deps = [ - ":parse_simpleapi_html_bzl", - ":urllib_bzl", - "//python/private:auth_bzl", - "//python/private:normalize_name_bzl", + ":parse_simpleapi_html", + ":urllib", + "//python/private:auth", + "//python/private:envsubst", + "//python/private:normalize_name", ], ) bzl_library( - name = "unified_hub_repo_bzl", + name = "unified_hub_repo", srcs = ["unified_hub_repo.bzl"], - deps = [ - "//python/private:text_util_bzl", - ], + deps = ["//python/private:text_util"], ) bzl_library( - name = "unified_hub_setup_bzl", + name = "unified_hub_setup", srcs = ["unified_hub_setup.bzl"], deps = [ - ":labels_bzl", - ":missing_package_bzl", + "@rules_python//python/private/pypi:labels", + "@rules_python//python/private/pypi:missing_package", ], ) bzl_library( - name = "urllib_bzl", - srcs = ["urllib.bzl"], + name = "venv_entry_point", + srcs = ["venv_entry_point.bzl"], + deps = [ + "//python/private:attributes", + "//python/private:common", + "//python/private:rule_builders", + ], ) bzl_library( - name = "version_from_filename_bzl", - srcs = ["version_from_filename.bzl"], + name = "venv_rewrite_shebang", + srcs = ["venv_rewrite_shebang.bzl"], + deps = [ + "//python/private:attributes", + "//python/private:common", + "//python/private:py_info", + "//python/private:rule_builders", + ], ) bzl_library( - name = "whl_config_repo_bzl", + name = "whl_config_repo", srcs = ["whl_config_repo.bzl"], deps = [ - ":generate_group_library_build_bazel_bzl", - "//python/private:text_util_bzl", + ":generate_group_library_build_bazel", + "//python/private:text_util", ], ) bzl_library( - name = "whl_config_setting_bzl", - srcs = ["whl_config_setting.bzl"], + name = "whl_extract", + srcs = ["whl_extract.bzl"], + deps = [ + ":whl_metadata", + "//python/private:repo_utils", + "@rules_python_internal//:rules_python_config", + ], ) bzl_library( - name = "whl_extract_bzl", - srcs = ["whl_extract.bzl"], + name = "whl_library", + srcs = ["whl_library.bzl"], deps = [ - ":whl_metadata_bzl", - "//python/private:repo_utils_bzl", - "@rules_python_internal//:rules_python_config_bzl", + ":attrs", + ":deps", + ":generate_whl_library_build_bazel", + ":patch_whl", + ":pep508_requirement", + ":pypi_repo_utils", + ":urllib", + ":whl_extract", + ":whl_metadata", + "//python/private:auth", + "//python/private:envsubst", + "//python/private:is_standalone_interpreter", + "//python/private:normalize_name", + "//python/private:repo_utils", ], ) bzl_library( - name = "whl_library_alias_bzl", + name = "whl_library_alias", srcs = ["whl_library_alias.bzl"], deps = [ - ":render_pkg_aliases_bzl", - "//python/private:full_version_bzl", + ":render_pkg_aliases", + "//python/private:full_version", ], ) bzl_library( - name = "whl_library_bzl", - srcs = ["whl_library.bzl"], + name = "whl_library_targets", + srcs = ["whl_library_targets.bzl"], deps = [ - ":attrs_bzl", - ":deps_bzl", - ":generate_whl_library_build_bazel_bzl", - ":parse_whl_name_bzl", - ":patch_whl_bzl", - ":pep508_requirement_bzl", - ":pypi_repo_utils_bzl", - ":urllib_bzl", - ":whl_extract_bzl", - ":whl_metadata_bzl", - ":whl_target_platforms_bzl", - "//python/private:auth_bzl", - "//python/private:envsubst_bzl", - "//python/private:is_standalone_interpreter_bzl", - "//python/private:normalize_name_bzl", - "//python/private:repo_utils_bzl", - "//python/private:util_bzl", - "@rules_python_internal//:rules_python_config_bzl", - ], -) - -bzl_library( - name = "whl_metadata_bzl", - srcs = ["whl_metadata.bzl"], + ":env_marker_setting", + ":labels", + ":namespace_pkgs", + ":pep508_deps", + ":venv_entry_point", + ":venv_rewrite_shebang", + "//python:py_binary", + "//python:py_library", + "//python/private:normalize_name", + "@bazel_skylib//rules:copy_file", + ], ) bzl_library( - name = "whl_repo_name_bzl", + name = "whl_repo_name", srcs = ["whl_repo_name.bzl"], deps = [ - ":parse_whl_name_bzl", - "//python/private:normalize_name_bzl", + ":parse_whl_name", + "//python/private:normalize_name", ], ) bzl_library( - name = "whl_target_platforms_bzl", - srcs = ["whl_target_platforms.bzl"], + name = "argparse", + srcs = ["argparse.bzl"], ) bzl_library( - name = "venv_entry_point_bzl", - srcs = ["venv_entry_point.bzl"], - visibility = ["//visibility:public"], + name = "attrs", + srcs = ["attrs.bzl"], ) bzl_library( - name = "venv_rewrite_shebang_bzl", - srcs = ["venv_rewrite_shebang.bzl"], - visibility = ["//visibility:public"], + name = "env_marker_info", + srcs = ["env_marker_info.bzl"], +) + +bzl_library( + name = "index_sources", + srcs = ["index_sources.bzl"], +) + +bzl_library( + name = "labels", + srcs = ["labels.bzl"], +) + +bzl_library( + name = "package_annotation", + srcs = ["package_annotation.bzl"], +) + +bzl_library( + name = "parse_requirements_txt", + srcs = ["parse_requirements_txt.bzl"], +) + +bzl_library( + name = "parse_whl_name", + srcs = ["parse_whl_name.bzl"], +) + +bzl_library( + name = "platform", + srcs = ["platform.bzl"], +) + +bzl_library( + name = "urllib", + srcs = ["urllib.bzl"], +) + +bzl_library( + name = "version_from_filename", + srcs = ["version_from_filename.bzl"], +) + +bzl_library( + name = "whl_config_setting", + srcs = ["whl_config_setting.bzl"], +) + +bzl_library( + name = "whl_metadata", + srcs = ["whl_metadata.bzl"], +) + +bzl_library( + name = "whl_target_platforms", + srcs = ["whl_target_platforms.bzl"], ) diff --git a/python/private/pythons_hub.bzl b/python/private/pythons_hub.bzl index 173c811da6..c64abf9887 100644 --- a/python/private/pythons_hub.bzl +++ b/python/private/pythons_hub.bzl @@ -31,13 +31,13 @@ load("@@{rules_python}//python/private:py_toolchain_suite.bzl", "py_toolchain_su load("@bazel_skylib//:bzl_library.bzl", "bzl_library") bzl_library( - name = "interpreters_bzl", + name = "interpreters", srcs = ["interpreters.bzl"], visibility = ["@rules_python//:__subpackages__"], ) bzl_library( - name = "versions_bzl", + name = "versions", srcs = ["versions.bzl"], visibility = ["@rules_python//:__subpackages__"], ) diff --git a/python/private/sentinel.bzl b/python/private/sentinel_impl.bzl similarity index 100% rename from python/private/sentinel.bzl rename to python/private/sentinel_impl.bzl diff --git a/python/private/stamp.bzl b/python/private/stamp_impl.bzl similarity index 100% rename from python/private/stamp.bzl rename to python/private/stamp_impl.bzl diff --git a/python/private/whl_filegroup/BUILD.bazel b/python/private/whl_filegroup/BUILD.bazel index b4246ca080..4d0a096311 100644 --- a/python/private/whl_filegroup/BUILD.bazel +++ b/python/private/whl_filegroup/BUILD.bazel @@ -7,14 +7,14 @@ filegroup( visibility = ["//python/private:__pkg__"], ) -bzl_library( - name = "whl_filegroup_bzl", - srcs = ["whl_filegroup.bzl"], - visibility = ["//:__subpackages__"], -) - py_binary( name = "extract_wheel_files", srcs = ["extract_wheel_files.py"], visibility = ["//visibility:public"], ) + +bzl_library( + name = "whl_filegroup", + srcs = ["whl_filegroup.bzl"], + visibility = ["//python:__subpackages__"], +) diff --git a/python/private/zipapp/BUILD.bazel b/python/private/zipapp/BUILD.bazel index 543fe0a185..66f0b4f667 100644 --- a/python/private/zipapp/BUILD.bazel +++ b/python/private/zipapp/BUILD.bazel @@ -11,25 +11,6 @@ filegroup( srcs = glob(["**"]), ) -bzl_library( - name = "py_zipapp_rule_bzl", - srcs = ["py_zipapp_rule.bzl"], - deps = [ - "//python/private:attributes_bzl", - "//python/private:builders_bzl", - "//python/private:common_bzl", - "//python/private:common_labels_bzl", - "//python/private:py_executable_info_bzl", - "//python/private:py_info_bzl", - "//python/private:py_internal_bzl", - "//python/private:py_interpreter_program_bzl", - "//python/private:py_runtime_info_bzl", - "//python/private:toolchain_types_bzl", - "//python/private:transition_labels_bzl", - "@bazel_skylib//lib:paths", - ], -) - filegroup( name = "zip_main_template", srcs = ["zip_main_template.py"], @@ -47,3 +28,21 @@ filegroup( srcs = ["zipapp_stage2_bootstrap_template.py"], visibility = ["//visibility:public"], ) + +bzl_library( + name = "py_zipapp_rule", + srcs = ["py_zipapp_rule.bzl"], + deps = [ + "//python/private:attributes", + "//python/private:builders", + "//python/private:common", + "//python/private:common_labels", + "//python/private:py_executable_info", + "//python/private:py_internal", + "//python/private:py_runtime_info", + "//python/private:toolchain_types", + "//python/private:transition_labels", + "@bazel_skylib//lib:paths", + "@rules_python_internal//:rules_python_config", + ], +) diff --git a/python/uv/BUILD.bazel b/python/uv/BUILD.bazel index 7ce6ce0523..93aefbfbd7 100644 --- a/python/uv/BUILD.bazel +++ b/python/uv/BUILD.bazel @@ -45,33 +45,25 @@ current_toolchain( ) bzl_library( - name = "lock_bzl", + name = "lock", srcs = ["lock.bzl"], - # EXPERIMENTAL: Visibility is restricted to allow for changes. - visibility = ["//:__subpackages__"], - deps = ["//python/uv/private:lock_bzl"], + deps = ["//python/uv/private:lock"], ) bzl_library( - name = "uv_bzl", + name = "uv", srcs = ["uv.bzl"], - # EXPERIMENTAL: Visibility is restricted to allow for changes. - visibility = ["//:__subpackages__"], - deps = ["//python/uv/private:uv_bzl"], + deps = ["//python/uv/private:uv"], ) bzl_library( - name = "uv_toolchain_bzl", + name = "uv_toolchain", srcs = ["uv_toolchain.bzl"], - # EXPERIMENTAL: Visibility is restricted to allow for changes. - visibility = ["//:__subpackages__"], - deps = ["//python/uv/private:uv_toolchain_bzl"], + deps = ["//python/uv/private:uv_toolchain"], ) bzl_library( - name = "uv_toolchain_info_bzl", + name = "uv_toolchain_info", srcs = ["uv_toolchain_info.bzl"], - # EXPERIMENTAL: Visibility is restricted to allow for changes. - visibility = ["//:__subpackages__"], - deps = ["//python/uv/private:uv_toolchain_info_bzl"], + deps = ["//python/uv/private:uv_toolchain_info"], ) diff --git a/python/uv/private/BUILD.bazel b/python/uv/private/BUILD.bazel index 4ac75f4d45..ce0ce0ec81 100644 --- a/python/uv/private/BUILD.bazel +++ b/python/uv/private/BUILD.bazel @@ -24,106 +24,117 @@ filegroup( visibility = ["//python/uv:__pkg__"], ) +filegroup( + name = "lock_copier_template", + srcs = ["template/lock_copier.py"], + target_compatible_with = [] if BZLMOD_ENABLED else ["@platforms//:incompatible"], + visibility = _NOT_REALLY_PUBLIC, +) + +filegroup( + name = "uv_lock_template", + srcs = select({ + "@platforms//os:windows": ["template/uv_lock.bat"], + "//conditions:default": ["template/uv_lock.sh"], + }), + target_compatible_with = [] if BZLMOD_ENABLED else ["@platforms//:incompatible"], + visibility = _NOT_REALLY_PUBLIC, +) + +filegroup( + name = "uv_pip_compile_template", + srcs = select({ + "@platforms//os:windows": ["template/uv_pip_compile.bat"], + "//conditions:default": ["template/uv_pip_compile.sh"], + }), + target_compatible_with = [] if BZLMOD_ENABLED else ["@platforms//:incompatible"], + visibility = _NOT_REALLY_PUBLIC, +) + bzl_library( - name = "current_toolchain_bzl", + name = "current_toolchain", srcs = ["current_toolchain.bzl"], visibility = ["//python/uv:__subpackages__"], + deps = [":toolchain_types"], ) +# keep bzl_library( - name = "lock_bzl", + name = "lock", srcs = ["lock.bzl"], - visibility = ["//python/uv:__subpackages__"], + visibility = [ + "//python/uv:__subpackages__", + "//tools/private:__subpackages__", + ], deps = [ - ":toolchain_types_bzl", - "//python:py_binary_bzl", - "//python/private:bzlmod_enabled_bzl", - "//python/private:common_labels_bzl", - "//python/private:toolchain_types_bzl", + ":toolchain_types", + "//python:py_binary", + "//python/private:bzlmod_enabled", + "//python/private:common_labels", + "//python/private:toolchain_types", "@bazel_skylib//lib:shell", ], ) bzl_library( - name = "toolchain_types_bzl", - srcs = ["toolchain_types.bzl"], + name = "toolchains_hub", + srcs = ["toolchains_hub.bzl"], visibility = ["//python/uv:__subpackages__"], + deps = [":toolchain_types"], ) bzl_library( - name = "uv_bzl", + name = "uv", srcs = ["uv.bzl"], visibility = ["//python/uv:__subpackages__"], deps = [ - ":toolchain_types_bzl", - ":uv_repository_bzl", - ":uv_toolchains_repo_bzl", - "//python/private:auth_bzl", - "//python/private:common_labels_bzl", - ], -) - -bzl_library( - name = "uv_lock_to_requirements_bzl", - srcs = ["uv_lock_to_requirements.bzl"], - visibility = [ - "//python/private:__subpackages__", - "//python/uv:__subpackages__", + ":toolchain_types", + ":uv_repository", + ":uv_toolchains_repo", + "//python/private:auth", + "//python/private:common_labels", ], ) bzl_library( - name = "uv_repository_bzl", + name = "uv_repository", srcs = ["uv_repository.bzl"], visibility = ["//python/uv:__subpackages__"], - deps = ["//python/private:auth_bzl"], + deps = ["//python/private:auth"], ) bzl_library( - name = "uv_toolchain_bzl", + name = "uv_toolchain", srcs = ["uv_toolchain.bzl"], visibility = ["//python/uv:__subpackages__"], - deps = [":uv_toolchain_info_bzl"], + deps = [":uv_toolchain_info"], ) bzl_library( - name = "uv_toolchain_info_bzl", - srcs = ["uv_toolchain_info.bzl"], + name = "uv_toolchains_repo", + srcs = ["uv_toolchains_repo.bzl"], visibility = ["//python/uv:__subpackages__"], + deps = ["//python/private:text_util"], ) +# keep bzl_library( - name = "uv_toolchains_repo_bzl", - srcs = ["uv_toolchains_repo.bzl"], - visibility = ["//python/uv:__subpackages__"], - deps = [ - "//python/private:text_util_bzl", + name = "uv_lock_to_requirements", + srcs = ["uv_lock_to_requirements.bzl"], + visibility = [ + "//python/private:__subpackages__", + "//python/uv:__subpackages__", ], ) -filegroup( - name = "lock_copier_template", - srcs = ["template/lock_copier.py"], - target_compatible_with = [] if BZLMOD_ENABLED else ["@platforms//:incompatible"], - visibility = _NOT_REALLY_PUBLIC, -) - -filegroup( - name = "uv_lock_template", - srcs = select({ - "@platforms//os:windows": ["template/uv_lock.bat"], - "//conditions:default": ["template/uv_lock.sh"], - }), - target_compatible_with = [] if BZLMOD_ENABLED else ["@platforms//:incompatible"], - visibility = _NOT_REALLY_PUBLIC, +bzl_library( + name = "toolchain_types", + srcs = ["toolchain_types.bzl"], + visibility = ["//python/uv:__subpackages__"], ) -filegroup( - name = "uv_pip_compile_template", - srcs = select({ - "@platforms//os:windows": ["template/uv_pip_compile.bat"], - "//conditions:default": ["template/uv_pip_compile.sh"], - }), - target_compatible_with = [] if BZLMOD_ENABLED else ["@platforms//:incompatible"], - visibility = _NOT_REALLY_PUBLIC, +bzl_library( + name = "uv_toolchain_info", + srcs = ["uv_toolchain_info.bzl"], + visibility = ["//python/uv:__subpackages__"], ) diff --git a/python/zipapp/BUILD.bazel b/python/zipapp/BUILD.bazel index 71249ae45c..77a8752701 100644 --- a/python/zipapp/BUILD.bazel +++ b/python/zipapp/BUILD.bazel @@ -11,19 +11,20 @@ filegroup( ) bzl_library( - name = "py_zipapp_binary_bzl", + name = "py_zipapp_binary", srcs = ["py_zipapp_binary.bzl"], deps = [ - "//python/private:util_bzl", - "//python/private/zipapp:py_zipapp_rule_bzl", + "//python/private:util", + "//python/private/zipapp:py_zipapp_rule", ], ) +# keep bzl_library( - name = "py_zipapp_test_bzl", + name = "py_zipapp_test", srcs = ["py_zipapp_test.bzl"], deps = [ - "//python/private:util_bzl", - "//python/private/zipapp:py_zipapp_rule_bzl", + "//python/private:util", + "//python/private/zipapp:py_zipapp_rule", ], ) diff --git a/sphinxdocs/docs/BUILD.bazel b/sphinxdocs/docs/BUILD.bazel index 87771f14f1..58c9e7fc18 100644 --- a/sphinxdocs/docs/BUILD.bazel +++ b/sphinxdocs/docs/BUILD.bazel @@ -48,11 +48,11 @@ sphinx_docs_library( sphinx_stardocs( name = "bzl_docs", srcs = [ - "//sphinxdocs:readthedocs_bzl", - "//sphinxdocs:sphinx_bzl", - "//sphinxdocs:sphinx_docs_library_bzl", - "//sphinxdocs:sphinx_stardoc_bzl", - "//sphinxdocs/private:sphinx_docs_library_bzl", + "//sphinxdocs:readthedocs", + "//sphinxdocs:sphinx", + "//sphinxdocs:sphinx_docs_library", + "//sphinxdocs:sphinx_stardoc", + "//sphinxdocs/private:sphinx_docs_library", ], prefix = "api/sphinxdocs/", target_compatible_with = _TARGET_COMPATIBLE_WITH, diff --git a/sphinxdocs/sphinxdocs/BUILD.bazel b/sphinxdocs/sphinxdocs/BUILD.bazel index 5a498b197e..fbdd633a32 100644 --- a/sphinxdocs/sphinxdocs/BUILD.bazel +++ b/sphinxdocs/sphinxdocs/BUILD.bazel @@ -42,29 +42,55 @@ bool_flag( ) bzl_library( - name = "sphinx_bzl", + name = "sphinx", srcs = ["sphinx.bzl"], visibility = ["//visibility:public"], - deps = ["//sphinxdocs/private:sphinx_bzl"], + deps = ["//sphinxdocs/private:sphinx"], ) bzl_library( - name = "sphinx_docs_library_bzl", + name = "sphinx_docs_library", srcs = ["sphinx_docs_library.bzl"], visibility = ["//visibility:public"], - deps = ["//sphinxdocs/private:sphinx_docs_library_macro_bzl"], + deps = ["//sphinxdocs/private:sphinx_docs_library_macro"], ) bzl_library( - name = "sphinx_stardoc_bzl", + name = "sphinx_stardoc", srcs = ["sphinx_stardoc.bzl"], visibility = ["//visibility:public"], - deps = ["//sphinxdocs/private:sphinx_stardoc_bzl"], + deps = ["//sphinxdocs/private:sphinx_stardoc"], ) bzl_library( - name = "readthedocs_bzl", + name = "readthedocs", srcs = ["readthedocs.bzl"], visibility = ["//visibility:public"], - deps = ["//sphinxdocs/private:readthedocs_bzl"], + deps = ["//sphinxdocs/private:readthedocs"], +) + +# ========= Deprecated aliases for backwards compatibility ========= + +alias( + name = "sphinx_bzl", + actual = ":sphinx", + deprecation = "Use //sphinxdocs:sphinx instead", +) + +alias( + name = "sphinx_docs_library_bzl", + actual = ":sphinx_docs_library", + deprecation = "Use //sphinxdocs:sphinx_docs_library instead", +) + +alias( + name = "sphinx_stardoc_bzl", + actual = ":sphinx_stardoc", + deprecation = "Use //sphinxdocs:sphinx_stardoc instead", +) + +alias( + name = "readthedocs_bzl", + actual = ":readthedocs", + deprecation = "Use //sphinxdocs:readthedocs instead", ) diff --git a/sphinxdocs/sphinxdocs/private/BUILD.bazel b/sphinxdocs/sphinxdocs/private/BUILD.bazel index 785d2e074d..b054ed2611 100644 --- a/sphinxdocs/sphinxdocs/private/BUILD.bazel +++ b/sphinxdocs/sphinxdocs/private/BUILD.bazel @@ -35,7 +35,7 @@ exports_files( ) bzl_library( - name = "util_bzl", + name = "util", srcs = ["util.bzl"], deps = [ "@bazel_skylib//lib:types", @@ -43,31 +43,31 @@ bzl_library( ) bzl_library( - name = "sphinx_docs_library_macro_bzl", + name = "sphinx_docs_library_macro", srcs = ["sphinx_docs_library_macro.bzl"], deps = [ - ":sphinx_docs_library_bzl", - "//sphinxdocs/private:util_bzl", + ":sphinx_docs_library", + ":util", ], ) bzl_library( - name = "sphinx_docs_library_bzl", + name = "sphinx_docs_library", srcs = ["sphinx_docs_library.bzl"], - deps = [":sphinx_docs_library_info_bzl"], + deps = [":sphinx_docs_library_info"], ) bzl_library( - name = "sphinx_docs_library_info_bzl", + name = "sphinx_docs_library_info", srcs = ["sphinx_docs_library_info.bzl"], ) bzl_library( - name = "sphinx_bzl", + name = "sphinx", srcs = ["sphinx.bzl"], deps = [ - ":sphinx_docs_library_info_bzl", - ":util_bzl", + ":sphinx_docs_library_info", + ":util", "@bazel_skylib//:bzl_library", "@bazel_skylib//lib:paths", "@bazel_skylib//lib:types", @@ -79,12 +79,12 @@ bzl_library( ) bzl_library( - name = "sphinx_stardoc_bzl", + name = "sphinx_stardoc", srcs = ["sphinx_stardoc.bzl"], deps = [ - ":sphinx_docs_library_macro_bzl", - "//sphinxdocs:sphinx_bzl", - "//sphinxdocs/private:util_bzl", + ":sphinx_docs_library_macro", + ":util", + "//sphinxdocs:sphinx", "@bazel_skylib//:bzl_library", "@bazel_skylib//lib:paths", "@bazel_skylib//lib:types", @@ -94,10 +94,10 @@ bzl_library( ) bzl_library( - name = "readthedocs_bzl", + name = "readthedocs", srcs = ["readthedocs.bzl"], deps = [ - ":util_bzl", + ":util", "@rules_python//python:py_binary_bzl", ], ) diff --git a/sphinxdocs/sphinxdocs/private/sphinx_stardoc.bzl b/sphinxdocs/sphinxdocs/private/sphinx_stardoc.bzl index ac76219299..75a4daf4d0 100644 --- a/sphinxdocs/sphinxdocs/private/sphinx_stardoc.bzl +++ b/sphinxdocs/sphinxdocs/private/sphinx_stardoc.bzl @@ -171,7 +171,7 @@ def sphinx_stardoc( **common_kwargs ) - stardoc_name = internal_name + "_stardoc" + stardoc_name = internal_name + "__stardoc" # NOTE: The .binaryproto suffix is an optimization. It makes the stardoc() # call avoid performing a copy of the output to the desired name. @@ -186,7 +186,7 @@ def sphinx_stardoc( **common_kwargs ) - pb2md_name = internal_name + "_pb2md" + pb2md_name = internal_name + "__pb2md" _stardoc_proto_to_markdown( name = pb2md_name, src = stardoc_pb, diff --git a/sphinxdocs/tests/sphinx_stardoc/BUILD.bazel b/sphinxdocs/tests/sphinx_stardoc/BUILD.bazel index a5d402b809..2cbc773f77 100644 --- a/sphinxdocs/tests/sphinx_stardoc/BUILD.bazel +++ b/sphinxdocs/tests/sphinx_stardoc/BUILD.bazel @@ -69,7 +69,6 @@ sphinx_stardoc( deps = [":func_and_providers_bzl"], ) -# A bzl_library with multiple sources bzl_library( name = "func_and_providers_bzl", srcs = [ @@ -89,6 +88,7 @@ bzl_library( srcs = ["bzl_typedef.bzl"], ) +# A bzl_library with multiple sources sphinx_build_binary( name = "sphinx-build", tags = ["manual"], # Only needed as part of sphinx doc building diff --git a/tests/config_settings/transition/BUILD.bazel b/tests/config_settings/transition/BUILD.bazel index 19d4958669..093de50959 100644 --- a/tests/config_settings/transition/BUILD.bazel +++ b/tests/config_settings/transition/BUILD.bazel @@ -1,6 +1,3 @@ load(":multi_version_tests.bzl", "multi_version_test_suite") -load(":py_args_tests.bzl", "py_args_test_suite") - -py_args_test_suite(name = "py_args_tests") multi_version_test_suite(name = "multi_version_tests") diff --git a/tests/config_settings/transition/py_args_tests.bzl b/tests/config_settings/transition/py_args_tests.bzl deleted file mode 100644 index 4538c88a5c..0000000000 --- a/tests/config_settings/transition/py_args_tests.bzl +++ /dev/null @@ -1,68 +0,0 @@ -# Copyright 2023 The Bazel Authors. All rights reserved. -# -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -"" - -load("@rules_testing//lib:test_suite.bzl", "test_suite") -load("//python/config_settings/private:py_args.bzl", "py_args") # buildifier: disable=bzl-visibility - -_tests = [] - -def _test_py_args_default(env): - actual = py_args("foo", {}) - - want = { - "args": None, - "data": None, - "deps": None, - "env": None, - "main": "foo.py", - "srcs": None, - } - env.expect.that_dict(actual).contains_exactly(want) - -_tests.append(_test_py_args_default) - -def _test_kwargs_get_consumed(env): - kwargs = { - "args": ["some", "args"], - "data": ["data"], - "deps": ["deps"], - "env": {"key": "value"}, - "main": "__main__.py", - "srcs": ["__main__.py"], - "visibility": ["//visibility:public"], - } - actual = py_args("bar_bin", kwargs) - - want = { - "args": ["some", "args"], - "data": ["data"], - "deps": ["deps"], - "env": {"key": "value"}, - "main": "__main__.py", - "srcs": ["__main__.py"], - } - env.expect.that_dict(actual).contains_exactly(want) - env.expect.that_dict(kwargs).keys().contains_exactly(["visibility"]) - -_tests.append(_test_kwargs_get_consumed) - -def py_args_test_suite(name): - """Create the test suite. - - Args: - name: the name of the test suite - """ - test_suite(name = name, basic_tests = _tests) diff --git a/tools/build_defs/python/private/BUILD.bazel b/tools/build_defs/python/private/BUILD.bazel index 746545640d..e189b18880 100644 --- a/tools/build_defs/python/private/BUILD.bazel +++ b/tools/build_defs/python/private/BUILD.bazel @@ -20,8 +20,9 @@ filegroup( visibility = ["//python:__subpackages__"], ) +# keep bzl_library( - name = "py_internal_renamed_bzl", + name = "py_internal_renamed", srcs = ["py_internal_renamed.bzl"], - visibility = ["//python/private:__pkg__"], + visibility = ["//:__subpackages__"], ) diff --git a/tools/private/BUILD.bazel b/tools/private/BUILD.bazel index adc8de3b0f..ae6951c245 100644 --- a/tools/private/BUILD.bazel +++ b/tools/private/BUILD.bazel @@ -1,3 +1,5 @@ +load("@bazel_skylib//:bzl_library.bzl", "bzl_library") + package( default_visibility = ["//:__subpackages__"], ) @@ -8,3 +10,9 @@ filegroup( "//tools/private/zipapp:distribution", ], ) + +bzl_library( + name = "publish_deps", + srcs = ["publish_deps.bzl"], + deps = ["//python/uv/private:lock"], +) diff --git a/tools/private/gazelle/BUILD.bazel b/tools/private/gazelle/BUILD.bazel new file mode 100644 index 0000000000..2a38e8f3a1 --- /dev/null +++ b/tools/private/gazelle/BUILD.bazel @@ -0,0 +1,21 @@ +load("@bazel_gazelle//:def.bzl", "gazelle", "gazelle_binary") + +package( + default_visibility = ["//:__subpackages__"], +) + +gazelle_binary( + name = "gazelle_bin", + languages = [ + "@bazel_skylib_gazelle_plugin//bzl", + ], + # Marked manual to avoid building in CI, which fails due to copt settings. + tags = ["manual"], +) + +gazelle( + name = "gazelle", + gazelle = ":gazelle_bin", + # Marked manual to avoid building in CI, which fails due to copt settings. + tags = ["manual"], +) diff --git a/tools/private/update_deps/BUILD.bazel b/tools/private/update_deps/BUILD.bazel index beecf82189..c3f94c9814 100644 --- a/tools/private/update_deps/BUILD.bazel +++ b/tools/private/update_deps/BUILD.bazel @@ -34,10 +34,10 @@ py_binary( name = "update_coverage_deps", srcs = ["update_coverage_deps.py"], data = [ - "//python/private:coverage_deps", + "//python/private:coverage_deps_filegroup", ], env = { - "UPDATE_FILE": "$(rlocationpath //python/private:coverage_deps)", + "UPDATE_FILE": "$(rlocationpath //python/private:coverage_deps_filegroup)", }, imports = ["../../.."], deps = [ diff --git a/workspace_bazel9.bzl b/workspace_bazel9.bzl new file mode 100644 index 0000000000..dad59378a9 --- /dev/null +++ b/workspace_bazel9.bzl @@ -0,0 +1,10 @@ +"""Workaround for Bazel 9 duplicate name issue in Gazelle.""" + +def bazel_9_workaround(name = "bazel_9_workaround"): + # Necessary so that Bazel 9 recognizes this as rules_python and doesn't try + # to load the version Bazel itself uses by default. + # We hide this from Gazelle's WORKSPACE parser by putting it in a macro. + native.local_repository( + name = "rules_python", + path = ".", + ) From 959a103070bea87c5b19d24da06371955c9db9fb Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Sun, 28 Jun 2026 01:36:29 -0700 Subject: [PATCH 785/922] fix(ci): use --repo flag in gh pr diff to avoid checkout (#3861) The pr-metadata-checks workflow was failing because gh pr diff requires a git repository context when run without target repository information. We now explicitly pass the repository using the --repo flag, allowing the job to run successfully without needing a full repository checkout. --- .github/workflows/pr-metadata-checks.yaml | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/.github/workflows/pr-metadata-checks.yaml b/.github/workflows/pr-metadata-checks.yaml index 003b4d99a5..e49ed14810 100644 --- a/.github/workflows/pr-metadata-checks.yaml +++ b/.github/workflows/pr-metadata-checks.yaml @@ -44,7 +44,9 @@ jobs: PR_NUMBER: ${{ github.event.pull_request.number }} run: | echo "Checking for blocked files..." - changed_files=$(gh pr diff "$PR_NUMBER" --name-only) + # We use --repo to avoid needing actions/checkout. + # This keeps the job lightweight and avoids "not a git repository" errors. + changed_files=$(gh pr diff "$PR_NUMBER" --name-only --repo "${{ github.repository }}") blocked_files=$(echo "$changed_files" | grep -E '^.agents/(plans|scratch)(/|$)' || true) if [ -n "$blocked_files" ]; then echo "::error::Files in .agents/plans and" \ From d37c64951d8ca7418c8680e2fd9157a862f9a925 Mon Sep 17 00:00:00 2001 From: Ignas Anikevicius <240938+aignas@users.noreply.github.com> Date: Sun, 28 Jun 2026 17:41:50 +0900 Subject: [PATCH 786/922] feat: stop using chmod on Bazel 8.6+ (#3855) ## Summary Add an `extract_needs_chmod` config flag that controls whether `chmod` is run after extracting wheel files. The flag is `False` for Bazel >= 8.6 (where the fix for file permissions is built into Bazel) and `True` for older versions. Move `_maybe_fix_permissions` from `whl_extract.bzl` into `repo_utils.bzl` and have `_extract` call it conditionally based on the config flag. This ensures both `whl_extract` and `patch_whl` benefit from the chmod fix when needed. Whilst at it migrate tests to common mocks. Fixes #3585 Closes #3860 --- .agents/plans/pypi_hub_proxy_feature.md | 296 ------------------------ python/private/internal_config_repo.bzl | 3 + python/private/pypi/patch_whl.bzl | 1 + python/private/pypi/whl_extract.bzl | 24 +- python/private/repo_utils.bzl | 25 +- tests/repo_utils/repo_utils_test.bzl | 75 +++++- tests/support/mocks.bzl | 31 --- tests/support/mocks/mocks.bzl | 11 +- 8 files changed, 111 insertions(+), 355 deletions(-) delete mode 100644 .agents/plans/pypi_hub_proxy_feature.md delete mode 100644 tests/support/mocks.bzl diff --git a/.agents/plans/pypi_hub_proxy_feature.md b/.agents/plans/pypi_hub_proxy_feature.md deleted file mode 100644 index b3360d9dbd..0000000000 --- a/.agents/plans/pypi_hub_proxy_feature.md +++ /dev/null @@ -1,296 +0,0 @@ -# Implementation Plan: Canonical Automatic PyPI Proxy Hub - -This document defines the locked, production-ready architectural, Starlark API, -and testing specifications for implementing dynamic PyPI dependency resolution in -`rules_python` using the `venv` flag. - ---- - -## 1. Architectural Strategy: The Canonical `@pypi` Proxy - -The `pip` bzlmod extension will automatically synthesize a canonical `@pypi` -proxy repository rule that orchestrates routing to underlying concrete hubs. - -### Bzlmod-Exclusive Scope - -The Unified PyPI Hub Proxy is an **exclusive feature of `bzlmod`**. Legacy -`WORKSPACE` evaluations using independent `pip_parse` repository macros are not -supported, as bzlmod's module extension architecture provides the required -centralized coordination to inspect and interlink cross-module hubs. - -### Automatic Proxy Construction & Collision Logic - -During the evaluation of the `pip` extension across the dependency graph: -1. **Unconditional Creation**: The extension will **always** synthesize a - proxy repository rule with the apparent name `pypi`, even if zero - `pip.parse` concrete hubs are defined in the dependency graph (in which - case the proxy is completely valid but empty). -2. **Collision Prevention**: If a user explicitly defines a concrete hub - named `pypi` (`pip.parse(hub_name = "pypi")`), the automatic proxy - synthesis is skipped so the user maintains absolute control over that - repository name. - -In `MODULE.bazel`: -```starlark -pip = use_extension("@rules_python//python/extensions:pip.bzl", "pip") - -# Concrete hubs defined for different execution contexts -pip.parse(hub_name = "pypi_a", ...) -pip.parse(hub_name = "pypi_b", ...) - -# Designate 'pypi_b' as the default hub for the unified '@pypi' repository -pip.default(default_hub = "pypi_b") - -# The canonical proxy is automatically created unconditionally: -use_repo(pip, "pypi") -``` - -### Unified PyPI Hub - -The canonical `@pypi` proxy repository matches exactly how concrete hubs create -their directory structure: a root package for shared configuration settings, and -a dedicated subdirectory (subpackage) for each PyPI package. - -Here is a complete, representative code example of what the generated files in -`@pypi` will look like when resolving packages between `pypi_a` and `pypi_b`: - -#### 1. `@pypi//BUILD.bazel` (Root Package) -The root package contains the shared `config_setting` targets following the -`_is_venv_` private naming convention. Leading underscores are strictly -applied because these configuration settings are an internal implementation -detail of the proxy repository and are not intended to be a public API. - -```starlark -package(default_visibility = ["//visibility:public"]) - -config_setting( - name = "_is_venv_pypi_a", - flag_values = { - "@rules_python//python/config_settings:venv": "pypi_a", - }, -) - -config_setting( - name = "_is_venv_pypi_b", - flag_values = { - "@rules_python//python/config_settings:venv": "pypi_b", - }, -) -``` - -#### 2. `@pypi//foo/BUILD.bazel` (PyPI Package Subpackage) -Each PyPI package subpackage defines the standard aliases (`pkg`, `whl`, `data`, -`dist_info`, `extracted_wheel_files`), plus a complete **union of all custom -`extra_hub_aliases`** defined across all concrete hubs. - -Each alias resolves dynamically to the active concrete hub based on the root -private configuration settings: - -```starlark -package(default_visibility = ["//visibility:public"]) - -alias( - name = "foo", - actual = ":pkg", -) - -alias( - name = "pkg", - actual = select({ - "//:_is_venv_pypi_a": "@pypi_a//foo:pkg", - "//:_is_venv_pypi_b": "@pypi_b//foo:pkg", - # When venv is "auto" (unset), it defaults to the designated fallback - # (or first defined concrete hub). - "//conditions:default": "@pypi_b//foo:pkg", - }), -) - -alias( - name = "whl", - actual = select({ - "//:_is_venv_pypi_a": "@pypi_a//foo:whl", - "//:_is_venv_pypi_b": "@pypi_b//foo:whl", - "//conditions:default": "@pypi_b//foo:whl", - }), -) - -# ... standard aliases for data, dist_info, extracted_wheel_files ... - -# 3. Unionized custom extra alias (defined in pypi_a but missing in pypi_b): -alias( - name = "my_custom_tool", - actual = select({ - "//:_is_venv_pypi_a": "@pypi_a//foo:my_custom_tool", - # Unrepresented branch routes to execution failure target: - "//:_is_venv_pypi_b": "//:_missing_package_error_pypi_b_foo", - "//conditions:default": "@pypi_a//foo:my_custom_tool", - }), -) -``` - -### Disjoint Hub Packages & Execution-Phase Failure - -If a package exists in one concrete hub but is missing in another (e.g., `scipy` -is in `pypi_b` but not `pypi_a`), our proxy synthesizes a package subpackage for -the union of all packages. - -To ensure that `bazel cquery` and `bazel query` successfully analyze over the -entire transitive build graph without failing, unrepresented select branches -must route to a dedicated **execution-phase error rule**. - -```starlark -# In @pypi//scipy/BUILD.bazel -alias( - name = "pkg", - actual = select({ - # Routes to execution-phase action failure target: - "//:_is_venv_pypi_a": "//:_missing_package_error_pypi_a_scipy", - "//:_is_venv_pypi_b": "@pypi_b//scipy:pkg", - "//conditions:default": "@pypi_b//scipy:pkg", - }), -) -``` - -The synthesized `//:_missing_package_error_XX` rule in `@pypi//BUILD.bazel` -returns standard Starlark Python providers so analysis/cquery passes, but -registers a build action that fails when executed: - -``` -Dependency Error: Third-party package 'scipy' is not available when building under PyPI hub 'pypi_a'. -``` - -### Fallback Hub Precedence (`"auto"`) - -When a target depends on `@pypi//foo` and the active build setting is `"auto"`, -the proxy resolves to a concrete hub using the following precedence: -1. **Designated Fallback**: If the user has explicitly designated a fallback - concrete hub via `pip.default(default_hub = "...")` in their root - `MODULE.bazel`, the proxy routes to it. -2. **First Defined Hub**: If no fallback is explicitly designated via - `pip.default()`, the proxy **automatically routes to the first defined - concrete hub** parsed during extension evaluation (e.g., `pypi_a`). - -```starlark -# Explicitly override the "auto" fallback hub -pip.default( - default_hub = "pypi_b", -) -``` - ---- - -## 2. Core Rule Integration: `config_settings` Transitions - -Users will switch active hubs using the standard, highly generic -`config_settings` transition attribute on executable targets. - -### Build Setting Definition - -In `python/config_settings/BUILD.bazel`: - -```starlark -string_flag( - name = "venv", - build_setting_default = "auto", # Default value is "auto" - visibility = ["//visibility:public"], -) -``` - -In `python/private/common_labels.bzl`: -```starlark - VENV = str(Label("//python/config_settings:venv")), -``` - -In `python/private/transition_labels.bzl`: -```starlark -_BASE_TRANSITION_LABELS = [ - # ... existing transition labels ... - labels.VENV, -] -``` - -Because `py_binary` and `py_test` implement an incoming transition -(`_transition_executable_impl`) that automatically processes any -`config_settings` keys matching `TRANSITION_LABELS`, **this provides complete -transition capabilities with zero changes to our core rule definitions**. - -### Usage in BUILD.bazel - -Libraries consume packages through the canonical proxy: - -```starlark -py_library( - name = "common", - deps = ["@pypi//foo"], # Apparent proxy repository -) -``` - -Binaries change the active hub by transitioning the build setting: - -```starlark -# Resolves @pypi -> pypi_b (default hub / designated fallback) -py_binary( - name = "bin_default", - deps = [":common"], -) - -# Resolves @pypi -> pypi_a via transition -py_binary( - name = "bin_a", - deps = [":common"], - config_settings = { - "//python/config_settings:venv": "pypi_a", - }, -) -``` - -### Analysis Cache & Memory Best Practices - -Because transitions fork the Bazel configuration, building targets with highly -diversified `config_settings` across large build graphs will result in -re-analysis and re-compilation of shared dependencies. - -We will include explicit documentation guidelines advising users to keep their -`venv` transition configurations localized and minimized to preserve Bazel -caching and memory efficiency. - ---- - -## 3. Integration Testing Specification - -We will construct a comprehensive Bazel-in-Bazel integration test suite in -`tests/integration/unified_pypi/` to guarantee correctness and verify -transitions. - -The integration test suite will assert: -1. **`"auto"` Precedence**: Author a test asserting `bazel run //:bin_default` - correctly inherits `"auto"` and resolves dependencies from the designated fallback. -2. **Transitional Resolution**: Author a test asserting two binary targets in - the same package with different `config_settings` successfully resolve - dependencies and execute against their respective concrete hubs (`pypi_a` - vs `pypi_b`). -3. **Command Line Override**: Author a test asserting - `bazel run --//python/config_settings:venv=pypi_a //:bin_default` - successfully forces the executable to run using imports resolved from - `pypi_a`. -4. **Disjoint Execution Failure**: Author a test asserting `bazel cquery` over - a target depending on an unrepresented missing package succeeds, while - `bazel run` on that target gracefully fails during execution with the exact - synthesized error message. -5. **Unionized Extra Hub Aliases**: Author a test asserting that a binary - successfully runs using a custom `extra_hub_aliases` target resolved - through the `@pypi proxy`. - ---- - -## 4. Execution Steps - -1. **Phase 1**: Define `venv` `string_flag` and register it in - `common_labels.bzl` and `transition_labels.bzl`. -2. **Phase 2**: Update `python/private/pypi/extension.bzl` to synthesize the - canonical `pypi` proxy repository rule. -3. **Phase 3**: Implement `missing_package_error` execution failure rule and - the `proxy_hub_repository` generation logic. -4. **Phase 4**: Author the Bazel-in-Bazel integration test suite in - `tests/integration/unified_pypi/`. -5. **Phase 5**: Run all tests and verify full pass before PR submission. diff --git a/python/private/internal_config_repo.bzl b/python/private/internal_config_repo.bzl index 9bae0a5821..8ee6a2d017 100644 --- a/python/private/internal_config_repo.bzl +++ b/python/private/internal_config_repo.bzl @@ -30,6 +30,7 @@ config = struct( supports_whl_extraction = {supports_whl_extraction}, enable_pystar = True, enable_deprecation_warnings = {enable_deprecation_warnings}, + extract_needs_chmod = {extract_needs_chmod}, bazel_8_or_later = {bazel_8_or_later}, bazel_9_or_later = {bazel_9_or_later}, bazel_10_or_later = {bazel_10_or_later}, @@ -86,6 +87,7 @@ def _internal_config_repo_impl(rctx): bazel_minor_version = 99999 supports_whl_extraction = False + extract_needs_chmod = bazel_major_version < 8 or (bazel_major_version == 8 and bazel_minor_version < 6) if bazel_major_version >= 8: # Extracting .whl files requires Bazel 8.3.0 or later. if bazel_major_version > 8 or bazel_minor_version >= 3: @@ -105,6 +107,7 @@ def _internal_config_repo_impl(rctx): builtin_py_info_symbol = builtin_py_info_symbol, builtin_py_runtime_info_symbol = builtin_py_runtime_info_symbol, supports_whl_extraction = str(supports_whl_extraction), + extract_needs_chmod = str(extract_needs_chmod), builtin_py_cc_link_params_provider = builtin_py_cc_link_params_provider, bazel_8_or_later = str(bazel_major_version >= 8), bazel_9_or_later = str(bazel_major_version >= 9), diff --git a/python/private/pypi/patch_whl.bzl b/python/private/pypi/patch_whl.bzl index 98a5ad49c8..e8d76eeba5 100644 --- a/python/private/pypi/patch_whl.bzl +++ b/python/private/pypi/patch_whl.bzl @@ -80,6 +80,7 @@ def patch_whl(rctx, *, whl_path, patches): rctx, archive = whl_input, supports_whl_extraction = rp_config.supports_whl_extraction, + extract_needs_chmod = rp_config.extract_needs_chmod, ) if not patches: diff --git a/python/private/pypi/whl_extract.bzl b/python/private/pypi/whl_extract.bzl index 2ebb61a83a..0d61b9a07b 100644 --- a/python/private/pypi/whl_extract.bzl +++ b/python/private/pypi/whl_extract.bzl @@ -18,10 +18,9 @@ def whl_extract(rctx, *, whl_path, logger): archive = whl_path, output = install_dir_path, supports_whl_extraction = rp_config.supports_whl_extraction, + extract_needs_chmod = rp_config.extract_needs_chmod, ) - _maybe_fix_permissions(rctx, whl_path = whl_path, logger = logger) - metadata_file = find_whl_metadata( install_dir = install_dir_path, logger = logger, @@ -65,27 +64,6 @@ def whl_extract(rctx, *, whl_path, logger): # Ensure that there is no data dir left rctx.delete(data_dir) -# TODO: This can be removed when Bazel 8.6+ is the minimum supported version. -def _maybe_fix_permissions(rctx, *, whl_path, logger): - # Fix permissions on extracted files. Some wheels have files without read permissions set, - # which causes errors when trying to read them later. - # We apply this to the root directory to ensure that everything in bin/, site-packages/, - # etc. is readable and executable where appropriate. - os_name = repo_utils.get_platforms_os_name(rctx) - if os_name != "windows": - # On Unix-like systems, recursively add read permissions to all files - # and ensure directories are traversable (need execute permission) - result = repo_utils.execute_unchecked( - rctx, - op = "Fixing wheel permissions {}".format(whl_path), - arguments = ["chmod", "-R", "a+rX", "."], - logger = logger, - ) - if result.return_code != 0: - # It's possible chmod is not available or the filesystem doesn't support it. - # This is fine, we just want to try to fix permissions if possible. - logger.warn(lambda: "Failed to fix file permissions: {}".format(result.stderr)) - def merge_trees(src, dest): """Merge src into the destination path. diff --git a/python/private/repo_utils.bzl b/python/private/repo_utils.bzl index 6fa851b7b3..0e83fda28c 100644 --- a/python/private/repo_utils.bzl +++ b/python/private/repo_utils.bzl @@ -202,7 +202,7 @@ def _execute_internal( output = _outputs_to_str(result, log_stdout = log_stdout, log_stderr = log_stderr), )) - result_kwargs = {k: getattr(result, k) for k in dir(result)} + result_kwargs = {k: getattr(result, k) for k in dir(result) if k not in ["to_json", "to_proto"]} return struct( describe_failure = lambda: _execute_describe_failure( op = op, @@ -511,7 +511,7 @@ def _get_platforms_cpu_name(mrctx): return "riscv64" return arch -def _extract(mrctx, *, archive, supports_whl_extraction = False, **kwargs): +def _extract(mrctx, *, archive, supports_whl_extraction = False, extract_needs_chmod = False, **kwargs): """Extract an archive TODO: remove when the earliest supported bazel version is at least 8.3. @@ -533,6 +533,26 @@ def _extract(mrctx, *, archive, supports_whl_extraction = False, **kwargs): if not mrctx.delete(archive): fail("Failed to remove the symlink after extracting") + if extract_needs_chmod: + _maybe_fix_permissions(mrctx, whl_path = archive) + +def _maybe_fix_permissions(mrctx, *, whl_path, logger = None): + if not logger and hasattr(mrctx, "attr"): + logger = _logger(mrctx) + elif not logger: + fail("logger must be specified when using 'module_ctx'") + + os_name = _get_platforms_os_name(mrctx) + if os_name != "windows": + result = _execute_unchecked( + mrctx, + op = "Fixing wheel permissions {}".format(whl_path), + arguments = ["chmod", "-R", "a+rX", "."], + logger = logger, + ) + if result.return_code != 0: + logger.warn(lambda: "Failed to fix file permissions: {}".format(result.stderr)) + def _rename(mrctx, src, dest): """Rename a file or directory. @@ -575,6 +595,7 @@ repo_utils = struct( get_platforms_os_name = _get_platforms_os_name, is_repo_debug_enabled = _is_repo_debug_enabled, logger = _logger, + maybe_fix_permissions = _maybe_fix_permissions, mkdir = _mkdir, norm_path = _norm_path, relative_to = _relative_to, diff --git a/tests/repo_utils/repo_utils_test.bzl b/tests/repo_utils/repo_utils_test.bzl index ce9e48b5a6..56d125724d 100644 --- a/tests/repo_utils/repo_utils_test.bzl +++ b/tests/repo_utils/repo_utils_test.bzl @@ -2,10 +2,21 @@ load("@rules_testing//lib:test_suite.bzl", "test_suite") load("//python/private:repo_utils.bzl", "repo_utils") # buildifier: disable=bzl-visibility -load("//tests/support:mocks.bzl", "mocks") +load("//tests/support/mocks:mocks.bzl", "mocks") _tests = [] +def _make_rctx(*, os_name, mock_extracts = None, mock_files = None): + return mocks.rctx( + attr = { + "name": "unit", + "_rule_name": "test_rule", + }, + mock_files = mock_files, + mock_extracts = mock_extracts, + os_name = os_name, + ) + def _test_get_platforms_os_name(env): mock_mrctx = mocks.rctx(os_name = "Mac OS X") got = repo_utils.get_platforms_os_name(mock_mrctx) @@ -50,6 +61,68 @@ def _test_is_relative_to(env): _tests.append(_test_is_relative_to) +def _test_extract_calls_chmod_when_enabled(env): + mock_rctx = _make_rctx( + os_name = "linux", + mock_extracts = {"test.whl": {"f1": "c1"}}, + ) + + repo_utils.extract( + mock_rctx, + archive = mock_rctx.path("test.whl"), + output = "out", + supports_whl_extraction = True, + extract_needs_chmod = True, + ) + + env.expect.that_bool(len(mock_rctx.execute_calls) > 0).equals(True) + env.expect.that_str(mock_rctx.execute_calls[0][0]).equals("chmod") + +_tests.append(_test_extract_calls_chmod_when_enabled) + +def _test_extract_skips_chmod_when_disabled(env): + mock_rctx = _make_rctx( + os_name = "linux", + mock_extracts = {"test.whl": {"f1": "c1"}}, + ) + + repo_utils.extract( + mock_rctx, + archive = mock_rctx.path("test.whl"), + output = "out", + supports_whl_extraction = True, + extract_needs_chmod = False, + ) + + env.expect.that_collection(mock_rctx.execute_calls).contains_exactly([]) + +_tests.append(_test_extract_skips_chmod_when_disabled) + +def _test_maybe_fix_permissions_calls_chmod_on_linux(env): + mock_rctx = _make_rctx(os_name = "linux") + + repo_utils.maybe_fix_permissions( + mock_rctx, + whl_path = mock_rctx.path("test.whl"), + ) + + env.expect.that_bool(len(mock_rctx.execute_calls) > 0).equals(True) + env.expect.that_str(mock_rctx.execute_calls[0][0]).equals("chmod") + +_tests.append(_test_maybe_fix_permissions_calls_chmod_on_linux) + +def _test_maybe_fix_permissions_skips_on_windows(env): + mock_rctx = _make_rctx(os_name = "windows") + + repo_utils.maybe_fix_permissions( + mock_rctx, + whl_path = mock_rctx.path("test.whl"), + ) + + env.expect.that_collection(mock_rctx.execute_calls).contains_exactly([]) + +_tests.append(_test_maybe_fix_permissions_skips_on_windows) + def repo_utils_test_suite(name): """Create the test suite. diff --git a/tests/support/mocks.bzl b/tests/support/mocks.bzl deleted file mode 100644 index 2a4ccd0fc4..0000000000 --- a/tests/support/mocks.bzl +++ /dev/null @@ -1,31 +0,0 @@ -"""Mocks for testing.""" - -def _rctx(os_name = "linux", os_arch = "x86_64", environ = None, **kwargs): - """Creates a mock of repository_ctx or module_ctx. - - Args: - os_name: The OS name to mock (e.g., "linux", "Mac OS X", "windows"). - os_arch: The OS architecture to mock (e.g., "x86_64", "aarch64"). - environ: A dictionary representing the environment variables. - **kwargs: Additional attributes to add to the mock struct. - - Returns: - A struct mocking repository_ctx. - """ - if environ == None: - environ = {} - - attrs = { - "getenv": environ.get, - "os": struct( - name = os_name, - arch = os_arch, - ), - } - attrs.update(kwargs) - - return struct(**attrs) - -mocks = struct( - rctx = _rctx, -) diff --git a/tests/support/mocks/mocks.bzl b/tests/support/mocks/mocks.bzl index 84d39fa6e2..6c79b1f4ff 100644 --- a/tests/support/mocks/mocks.bzl +++ b/tests/support/mocks/mocks.bzl @@ -185,7 +185,7 @@ def _mctx_download( return struct(success = True, wait = lambda: struct(success = True)) return struct(success = False, wait = lambda: struct(success = False)) -def _mctx_report_progress(self, message): +def _mrctx_report_progress(self, message): self.report_progress_calls.append(message) return None @@ -259,7 +259,7 @@ def _mctx_new( path = lambda *a, **k: _mctx_path(self, *a, **k), read = lambda *a, **k: _mctx_read(self, *a, **k), download = lambda *a, **k: _mctx_download(self, *a, **k), - report_progress = lambda *a, **k: _mctx_report_progress(self, *a, **k), + report_progress = lambda *a, **k: _mrctx_report_progress(self, *a, **k), add_module = lambda **k: _mctx_add_module(self, **k), ) return self @@ -439,6 +439,7 @@ def _rctx_execute( environment, custom_reporter, ) # @unused + self.execute_calls.append(arguments) return struct(return_code = 0, stdout = "", stderr = "") def _rctx_symlink(self, target, link_name): @@ -485,7 +486,10 @@ def _rctx_new( mock_which = mock_which, mock_downloads = mock_downloads, mock_extracts = mock_extracts, + execute_calls = [], + report_progress_calls = [], attr = struct(**attr), + name = attr.get("name", "mock"), os = struct( name = os_name, arch = arch_name, @@ -494,10 +498,12 @@ def _rctx_new( path = lambda *a, **k: _rctx_path(self, *a, **k), read = lambda *a, **k: _rctx_read(self, *a, **k), file = lambda *a, **k: _rctx_file(self, *a, **k), + getenv = environ.get, template = lambda *a, **k: _rctx_template(self, *a, **k), which = lambda *a, **k: _rctx_which(self, *a, **k), download = lambda *a, **k: _rctx_download(self, *a, **k), extract = lambda *a, **k: _rctx_extract(self, *a, **k), + delete = lambda x: True, download_and_extract = lambda *a, **k: _rctx_download_and_extract( self, *a, @@ -505,6 +511,7 @@ def _rctx_new( ), execute = lambda *a, **k: _rctx_execute(self, *a, **k), symlink = lambda *a, **k: _rctx_symlink(self, *a, **k), + report_progress = lambda *a, **k: _mrctx_report_progress(self, *a, **k), ) return self From 20db9560c52878a3e5dc427430c225d3efa1a730 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Sun, 28 Jun 2026 01:47:08 -0700 Subject: [PATCH 787/922] ci: make pr-metadata-check run on PR body edit (#3862) Make the check run on edit so that it can detect adding/removing of the special marker text. --- .github/workflows/pr-metadata-checks.yaml | 1 + BUILD.bazel | 2 ++ 2 files changed, 3 insertions(+) diff --git a/.github/workflows/pr-metadata-checks.yaml b/.github/workflows/pr-metadata-checks.yaml index e49ed14810..8060ff921e 100644 --- a/.github/workflows/pr-metadata-checks.yaml +++ b/.github/workflows/pr-metadata-checks.yaml @@ -8,6 +8,7 @@ on: - reopened - labeled - unlabeled + - edited jobs: blocks-do-not-merge: diff --git a/BUILD.bazel b/BUILD.bazel index a52251393d..a25863708c 100644 --- a/BUILD.bazel +++ b/BUILD.bazel @@ -108,3 +108,5 @@ bzl_library( srcs = ["version.bzl"], visibility = ["//:__subpackages__"], ) + +# No-op change to verify PR metadata checks From 367190add274b22efe9992fd8b364d22396a2809 Mon Sep 17 00:00:00 2001 From: Ignas Anikevicius <240938+aignas@users.noreply.github.com> Date: Mon, 29 Jun 2026 09:00:43 +0900 Subject: [PATCH 788/922] chore(pypi): cleanup unused code (#3864) This is no longer used starting when we enabled pipstar by default and did a code cleanup where Python is no longer used to extract the wheels. Split from #3856 --- .../pypi/generate_whl_library_build_bazel.bzl | 47 +--- python/private/pypi/pkg_aliases.bzl | 1 + python/private/pypi/whl_library_targets.bzl | 107 +------- ...generate_whl_library_build_bazel_tests.bzl | 65 ----- .../whl_library_targets_tests.bzl | 247 ------------------ 5 files changed, 15 insertions(+), 452 deletions(-) diff --git a/python/private/pypi/generate_whl_library_build_bazel.bzl b/python/private/pypi/generate_whl_library_build_bazel.bzl index a9a29081f7..71eab26c0e 100644 --- a/python/private/pypi/generate_whl_library_build_bazel.bzl +++ b/python/private/pypi/generate_whl_library_build_bazel.bzl @@ -21,8 +21,6 @@ _RENDER = { "copy_files": render.dict, "data": render.list, "data_exclude": render.list, - "dependencies": render.list, - "dependencies_by_platform": lambda x: render.dict(x, value_repr = render.list), "entry_points": render.dict_dict, "extras": render.list, "group_deps": render.list, @@ -30,7 +28,6 @@ _RENDER = { "requires_dist": render.list, "srcs_exclude": render.list, "tags": render.list, - "target_platforms": render.list, } # NOTE @aignas 2024-10-25: We have to keep this so that files in @@ -55,15 +52,17 @@ package_metadata( def generate_whl_library_build_bazel( *, annotation = None, - default_python_version = None, + config_load, purl = None, + requires_dist = [], **kwargs): """Generate a BUILD file for an unzipped Wheel Args: annotation: The annotation for the build file. - default_python_version: The python version to use to parse the METADATA. + config_load: {type}`str` The location from where to load the config. purl: The purl. + requires_dist: {type}`list[str]` The list of dependencies from the METADATA file. **kwargs: Extra args serialized to be passed to the {obj}`whl_library_targets`. @@ -75,36 +74,14 @@ def generate_whl_library_build_bazel( """load("@package_metadata//rules:package_metadata.bzl", "package_metadata")""", ] - if kwargs.get("tags"): - fn = "whl_library_targets" - - # legacy path - unsupported_args = [ - "requires", - "metadata_name", - "metadata_version", - "packages", - "include", - ] + fn = "whl_library_targets_from_requires" + if not requires_dist: + # no deps, we can leave the extra loads out + pass else: - fn = "whl_library_targets_from_requires" - unsupported_args = [ - "dependencies", - "dependencies_by_platform", - "target_platforms", - "default_python_version", - ] - packages_load = kwargs.pop("config_load") - if not kwargs.get("requires_dist"): - # no deps, we can leave the extra loads out - pass - else: - loads.append("""load("{}", "{}")""".format(packages_load, "packages")) - kwargs["include"] = "packages" - - for arg in unsupported_args: - if kwargs.get(arg): - fail("BUG, unsupported arg: '{}'".format(arg)) + loads.append("""load("{}", "{}")""".format(config_load, "packages")) + kwargs["include"] = "packages" + kwargs["requires_dist"] = requires_dist loads.extend([ """load("@rules_python//python/private/pypi:whl_library_targets.bzl", "{}")""".format(fn), @@ -119,8 +96,6 @@ def generate_whl_library_build_bazel( kwargs["srcs_exclude"] = annotation.srcs_exclude_glob if annotation.additive_build_content: additional_content.append(annotation.additive_build_content) - if default_python_version: - kwargs["default_python_version"] = default_python_version contents = "\n".join( [ diff --git a/python/private/pypi/pkg_aliases.bzl b/python/private/pypi/pkg_aliases.bzl index 111a49d3c6..b1c29c95ee 100644 --- a/python/private/pypi/pkg_aliases.bzl +++ b/python/private/pypi/pkg_aliases.bzl @@ -201,6 +201,7 @@ def multiplatform_whl_aliases( ret[alias] = repo continue + # This is if we are using `whl_config_setting` struct config_settings = get_config_settings( target_platforms = alias.target_platforms, python_version = alias.version, diff --git a/python/private/pypi/whl_library_targets.bzl b/python/private/pypi/whl_library_targets.bzl index 01a89aadcc..933b529053 100644 --- a/python/private/pypi/whl_library_targets.bzl +++ b/python/private/pypi/whl_library_targets.bzl @@ -120,10 +120,8 @@ def whl_library_targets( tags = [], dependencies = [], filegroups = None, - dependencies_by_platform = {}, dependencies_with_markers = {}, entry_points = {}, - group_deps = [], group_name = "", data = [], copy_files = {}, @@ -151,8 +149,6 @@ def whl_library_targets( the filename of the sdist. tags: {type}`list[str]` The tags set on the `py_library`. dependencies: {type}`list[str]` A list of dependencies. - dependencies_by_platform: {type}`dict[str, list[str]]` A list of - dependencies by platform key. dependencies_with_markers: {type}`dict[str, str]` A marker to evaluate in order for the dep to be included. entry_points: {type}`list[dict]` A list of parsed entry point definitions. @@ -162,10 +158,6 @@ def whl_library_targets( contains this library. If set, this library will behave as a shim to group implementation rules which will provide simultaneously installed dependencies which would otherwise form a cycle. - group_deps: {type}`list[str]` names of fellow members of the group (if - any). These will be excluded from generated deps lists so as to avoid - direct cycles. These dependencies will be provided at runtime by the - group rules which wrap this library and its fellows together. copy_executables: {type}`dict[str, str]` The mapping between src and dest locations for the targets. copy_files: {type}`dict[str, str]` The mapping between src and @@ -183,10 +175,6 @@ def whl_library_targets( rules: {type}`struct` A struct with references to rules for creating targets. """ dependencies = sorted([normalize_name(d) for d in dependencies]) - dependencies_by_platform = { - platform: sorted([normalize_name(d) for d in deps]) - for platform, deps in dependencies_by_platform.items() - } tags = sorted(tags) data = [] + data @@ -267,9 +255,7 @@ def whl_library_targets( data.append(dest) _config_settings( - dependencies_by_platform = dependencies_by_platform.keys(), dependencies_with_markers = dependencies_with_markers, - native = native, rules = rules, visibility = ["//visibility:private"], ) @@ -278,25 +264,6 @@ def whl_library_targets( for d in dependencies_with_markers } - # Ensure this list is normalized - # Note: mapping used as set - group_deps = { - normalize_name(d): True - for d in group_deps - } - - dependencies = [ - d - for d in dependencies - if d not in group_deps - ] - dependencies_by_platform = { - p: deps - for p, deps in dependencies_by_platform.items() - for deps in [[d for d in deps if d not in group_deps]] - if deps - } - # If this library is a member of a group, its public label aliases need to # point to the group implementation rule not the implementation rules. We # also need to mark the implementation rules as visible to the group @@ -351,7 +318,6 @@ def whl_library_targets( srcs = [name], data = _deps( deps = dependencies, - deps_by_platform = dependencies_by_platform, deps_conditional = deps_conditional, tmpl = dep_template.format(name = "{}", target = WHEEL_FILE_PUBLIC_LABEL), ), @@ -418,7 +384,6 @@ def whl_library_targets( imports = ["site-packages"], deps = _deps( deps = dependencies, - deps_by_platform = dependencies_by_platform, deps_conditional = deps_conditional, tmpl = dep_template.format(name = "{}", target = PY_LIBRARY_PUBLIC_LABEL), ), @@ -428,22 +393,13 @@ def whl_library_targets( namespace_package_files = namespace_package_files, ) -def _config_settings(dependencies_by_platform, dependencies_with_markers, rules, native = native, **kwargs): +def _config_settings(dependencies_with_markers, rules, **kwargs): """Generate config settings for the targets. Args: - dependencies_by_platform: {type}`list[str]` platform keys, can be - one of the following formats: - * `//conditions:default` - * `@platforms//os:{value}` - * `@platforms//cpu:{value}` - * `@//python/config_settings:is_python_3.{minor_version}` - * `{os}_{cpu}` - * `cp3{minor_version}_{os}_{cpu}` dependencies_with_markers: {type}`dict[str, str]` The markers to evaluate by each dep. rules: used for testing - native: {type}`native` The native struct for overriding in tests. **kwargs: Extra kwargs to pass to the rule. """ for dep, expression in dependencies_with_markers.items(): @@ -453,46 +409,7 @@ def _config_settings(dependencies_by_platform, dependencies_with_markers, rules, **kwargs ) - for p in dependencies_by_platform: - if p.startswith("@") or p.endswith("default"): - continue - - # TODO @aignas 2025-04-20: add tests here - abi, _, tail = p.partition("_") - if not abi.startswith("cp"): - tail = p - abi = "" - os, _, arch = tail.partition("_") - - _kwargs = dict(kwargs) - _kwargs["constraint_values"] = [ - "@platforms//cpu:{}".format(arch), - "@platforms//os:{}".format(os), - ] - - if abi: - _kwargs["flag_values"] = { - Label("//python/config_settings:python_version"): "3.{}".format(abi[len("cp3"):]), - } - - native.config_setting( - name = "is_{name}".format( - name = p.replace("cp3", "python_3."), - ), - **_kwargs - ) - -def _plat_label(plat): - if plat.endswith("default"): - return plat - elif plat.startswith("@//"): - return Label(plat.strip("@")) - elif plat.startswith("@"): - return plat - else: - return ":is_" + plat.replace("cp3", "python_3.") - -def _deps(deps, deps_by_platform, deps_conditional, tmpl): +def _deps(deps, deps_conditional, tmpl): deps = [tmpl.format(d) for d in sorted(deps)] for dep, setting in deps_conditional.items(): @@ -501,22 +418,4 @@ def _deps(deps, deps_by_platform, deps_conditional, tmpl): "//conditions:default": [], }) - if not deps_by_platform: - return deps - - deps_by_platform = { - _plat_label(p): [ - tmpl.format(d) - for d in sorted(deps) - ] - for p, deps in sorted(deps_by_platform.items()) - } - - # Add the default, which means that we will be just using the dependencies in - # `deps` for platforms that are not handled in a special way by the packages - deps_by_platform.setdefault("//conditions:default", []) - - if not deps: - return select(deps_by_platform) - else: - return deps + select(deps_by_platform) + return deps diff --git a/tests/pypi/generate_whl_library_build_bazel/generate_whl_library_build_bazel_tests.bzl b/tests/pypi/generate_whl_library_build_bazel/generate_whl_library_build_bazel_tests.bzl index 9586581cad..1fd99205b1 100644 --- a/tests/pypi/generate_whl_library_build_bazel/generate_whl_library_build_bazel_tests.bzl +++ b/tests/pypi/generate_whl_library_build_bazel/generate_whl_library_build_bazel_tests.bzl @@ -19,71 +19,6 @@ load("//python/private/pypi:generate_whl_library_build_bazel.bzl", "generate_whl _tests = [] -def _test_all_legacy(env): - want = """\ -load("@package_metadata//rules:package_metadata.bzl", "package_metadata") -load("@rules_python//python/private/pypi:whl_library_targets.bzl", "whl_library_targets") - -package(default_visibility = ["//visibility:public"]) - -package_metadata( - name = "package_metadata", - purl = None, - visibility = ["//:__subpackages__"], -) - -whl_library_targets( - copy_executables = { - "exec_src": "exec_dest", - }, - copy_files = { - "file_src": "file_dest", - }, - data = ["extra_target"], - data_exclude = [ - "exclude_via_attr", - "data_exclude_all", - ], - dep_template = "@pypi_{name}//:{target}", - dependencies = ["foo"], - dependencies_by_platform = { - "baz": ["bar"], - }, - group_deps = [ - "foo", - "fox", - "qux", - ], - group_name = "qux", - name = "foo.whl", - srcs_exclude = ["srcs_exclude_all"], - tags = ["tag1"], -) - -# SOMETHING SPECIAL AT THE END -""" - actual = generate_whl_library_build_bazel( - dep_template = "@pypi_{name}//:{target}", - name = "foo.whl", - dependencies = ["foo"], - dependencies_by_platform = {"baz": ["bar"]}, - data_exclude = ["exclude_via_attr"], - annotation = struct( - copy_files = {"file_src": "file_dest"}, - copy_executables = {"exec_src": "exec_dest"}, - data = ["extra_target"], - data_exclude_glob = ["data_exclude_all"], - srcs_exclude_glob = ["srcs_exclude_all"], - additive_build_content = """# SOMETHING SPECIAL AT THE END""", - ), - group_name = "qux", - group_deps = ["foo", "fox", "qux"], - tags = ["tag1"], - ) - env.expect.that_str(actual.replace("@@", "@")).equals(want) - -_tests.append(_test_all_legacy) - def _test_all_workspace(env): want = """\ load("@package_metadata//rules:package_metadata.bzl", "package_metadata") diff --git a/tests/pypi/whl_library_targets/whl_library_targets_tests.bzl b/tests/pypi/whl_library_targets/whl_library_targets_tests.bzl index 91db15f296..5bd1d1f549 100644 --- a/tests/pypi/whl_library_targets/whl_library_targets_tests.bzl +++ b/tests/pypi/whl_library_targets/whl_library_targets_tests.bzl @@ -72,72 +72,12 @@ def _test_filegroups(env): _tests.append(_test_filegroups) -def _test_platforms(env): - calls = [] - - whl_library_targets( - name = "", - dep_template = None, - dependencies_by_platform = { - "@//python/config_settings:is_python_3.9": ["py39_dep"], - "@platforms//cpu:aarch64": ["arm_dep"], - "@platforms//os:windows": ["win_dep"], - "cp310.11_linux_ppc64le": ["full_version_dep"], - "cp310_linux_ppc64le": ["py310_linux_ppc64le_dep"], - "linux_x86_64": ["linux_intel_dep"], - }, - filegroups = {}, - native = struct( - config_setting = lambda **kwargs: calls.append(kwargs), - glob = lambda *args, **kwargs: [], - ), - rules = struct( - venv_rewrite_shebang = lambda **kwargs: None, - ), - ) - - env.expect.that_collection(calls).contains_exactly([ - { - "name": "is_python_3.10.11_linux_ppc64le", - "visibility": ["//visibility:private"], - "constraint_values": [ - "@platforms//cpu:ppc64le", - "@platforms//os:linux", - ], - "flag_values": { - Label("//python/config_settings:python_version"): "3.10.11", - }, - }, - { - "name": "is_python_3.10_linux_ppc64le", - "visibility": ["//visibility:private"], - "constraint_values": [ - "@platforms//cpu:ppc64le", - "@platforms//os:linux", - ], - "flag_values": { - Label("//python/config_settings:python_version"): "3.10", - }, - }, - { - "name": "is_linux_x86_64", - "visibility": ["//visibility:private"], - "constraint_values": [ - "@platforms//cpu:x86_64", - "@platforms//os:linux", - ], - }, - ]) # buildifier: @unsorted-dict-items - -_tests.append(_test_platforms) - def _test_copy(env): calls = [] whl_library_targets( name = "", dep_template = None, - dependencies_by_platform = {}, filegroups = {}, copy_files = {"file_src": "file_dest"}, copy_executables = {"exec_src": "exec_dest"}, @@ -289,193 +229,6 @@ def _test_whl_and_library_deps_from_requires(env): _tests.append(_test_whl_and_library_deps_from_requires) -def _test_whl_and_library_deps(env): - filegroup_calls = [] - py_library_calls = [] - m_glob = mocks.glob() - m_glob.results.append([]) # bin - m_glob.results.append([]) # rewrite-bin - m_glob.results.append(["site-packages/foo/SRCS.py"]) - m_glob.results.append(["site-packages/foo/DATA.txt"]) - m_glob.results.append(["site-packages/foo/PYI.pyi"]) - - whl_library_targets( - name = "foo.whl", - dep_template = "@pypi_{name}//:{target}", - dependencies = ["foo", "bar-baz"], - dependencies_by_platform = { - "@//python/config_settings:is_python_3.9": ["py39_dep"], - "@platforms//cpu:aarch64": ["arm_dep"], - "@platforms//os:windows": ["win_dep"], - "cp310_linux_ppc64le": ["py310_linux_ppc64le_dep"], - "cp39_anyos_aarch64": ["py39_arm_dep"], - "cp39_linux_anyarch": ["py39_linux_dep"], - "linux_x86_64": ["linux_intel_dep"], - }, - data_exclude = [], - tags = ["tag1", "tag2"], - # Overrides for testing - filegroups = {}, - native = struct( - filegroup = lambda **kwargs: filegroup_calls.append(kwargs), - config_setting = lambda **_: None, - glob = m_glob.glob, - ), - rules = struct( - py_library = lambda **kwargs: py_library_calls.append(kwargs), - create_inits = lambda **kwargs: ["_create_inits_target"], - venv_rewrite_shebang = lambda **kwargs: None, - ), - ) - - env.expect.that_collection(filegroup_calls).contains_exactly([ - { - "name": "whl", - "srcs": ["foo.whl"], - "data": [ - "@pypi_bar_baz//:whl", - "@pypi_foo//:whl", - ] + select( - { - Label("//python/config_settings:is_python_3.9"): ["@pypi_py39_dep//:whl"], - "@platforms//cpu:aarch64": ["@pypi_arm_dep//:whl"], - "@platforms//os:windows": ["@pypi_win_dep//:whl"], - ":is_python_3.10_linux_ppc64le": ["@pypi_py310_linux_ppc64le_dep//:whl"], - ":is_python_3.9_anyos_aarch64": ["@pypi_py39_arm_dep//:whl"], - ":is_python_3.9_linux_anyarch": ["@pypi_py39_linux_dep//:whl"], - ":is_linux_x86_64": ["@pypi_linux_intel_dep//:whl"], - "//conditions:default": [], - }, - ), - "visibility": ["//visibility:public"], - }, - ]) # buildifier: @unsorted-dict-items - - env.expect.that_collection(py_library_calls).has_size(1) - if len(py_library_calls) != 1: - return - env.expect.that_dict(py_library_calls[0]).contains_exactly({ - "name": "pkg", - "srcs": ["site-packages/foo/SRCS.py"] + select({ - Label("//python/config_settings:_is_venvs_site_packages_yes"): [], - "//conditions:default": ["_create_inits_target"], - }), - "pyi_srcs": ["site-packages/foo/PYI.pyi"], - "data": ["site-packages/foo/DATA.txt", "data"], - "imports": ["site-packages"], - "deps": [ - "@pypi_bar_baz//:pkg", - "@pypi_foo//:pkg", - ] + select( - { - Label("//python/config_settings:is_python_3.9"): ["@pypi_py39_dep//:pkg"], - "@platforms//cpu:aarch64": ["@pypi_arm_dep//:pkg"], - "@platforms//os:windows": ["@pypi_win_dep//:pkg"], - ":is_python_3.10_linux_ppc64le": ["@pypi_py310_linux_ppc64le_dep//:pkg"], - ":is_python_3.9_anyos_aarch64": ["@pypi_py39_arm_dep//:pkg"], - ":is_python_3.9_linux_anyarch": ["@pypi_py39_linux_dep//:pkg"], - ":is_linux_x86_64": ["@pypi_linux_intel_dep//:pkg"], - "//conditions:default": [], - }, - ), - "tags": ["tag1", "tag2"], - "visibility": ["//visibility:public"], - "experimental_venvs_site_packages": Label("//python/config_settings:venvs_site_packages"), - "namespace_package_files": [] + select({ - Label("//python/config_settings:_is_venvs_site_packages_yes"): [], - "//conditions:default": ["_create_inits_target"], - }), - }) # buildifier: @unsorted-dict-items - -_tests.append(_test_whl_and_library_deps) - -def _test_group(env): - alias_calls = [] - py_library_calls = [] - - m_glob = mocks.glob() - m_glob.results.append([]) # bin - m_glob.results.append([]) # rewrite-bin - m_glob.results.append(["site-packages/foo/srcs.py"]) - m_glob.results.append(["site-packages/foo/data.txt"]) - m_glob.results.append(["site-packages/foo/pyi.pyi"]) - - whl_library_targets( - name = "foo.whl", - dep_template = "@pypi_{name}//:{target}", - dependencies = ["foo", "bar-baz", "qux"], - dependencies_by_platform = { - "linux_x86_64": ["box", "box-amd64"], - "windows_x86_64": ["fox"], - "@platforms//os:linux": ["box"], # buildifier: disable=unsorted-dict-items to check that we sort inside the test - }, - tags = [], - data_exclude = [], - group_name = "qux", - group_deps = ["foo", "fox", "qux"], - # Overrides for testing - filegroups = {}, - native = struct( - config_setting = lambda **_: None, - glob = m_glob.glob, - alias = lambda **kwargs: alias_calls.append(kwargs), - ), - rules = struct( - py_library = lambda **kwargs: py_library_calls.append(kwargs), - create_inits = lambda **kwargs: ["_create_inits_target"], - venv_rewrite_shebang = lambda **kwargs: None, - ), - ) - - env.expect.that_collection(alias_calls).contains_exactly([ - {"name": "pkg", "actual": "@pypi__config//_groups:qux_pkg", "visibility": ["//visibility:public"]}, - {"name": "whl", "actual": "@pypi__config//_groups:qux_whl", "visibility": ["//visibility:public"]}, - ]) # buildifier: @unsorted-dict-items - - env.expect.that_collection(py_library_calls).has_size(1) - if len(py_library_calls) != 1: - return - - py_library_call = py_library_calls[0] - env.expect.where(case = "verify py library call").that_dict( - py_library_call, - ).contains_exactly({ - "name": "_pkg", - "srcs": ["site-packages/foo/srcs.py"] + select({ - Label("//python/config_settings:_is_venvs_site_packages_yes"): [], - "//conditions:default": ["_create_inits_target"], - }), - "pyi_srcs": ["site-packages/foo/pyi.pyi"], - "data": ["site-packages/foo/data.txt", "data"], - "imports": ["site-packages"], - "deps": ["@pypi_bar_baz//:pkg"] + select({ - "@platforms//os:linux": ["@pypi_box//:pkg"], - ":is_linux_x86_64": ["@pypi_box//:pkg", "@pypi_box_amd64//:pkg"], - "//conditions:default": [], - }), - "tags": [], - "visibility": ["@pypi__config//_groups:__pkg__"], - "experimental_venvs_site_packages": Label("//python/config_settings:venvs_site_packages"), - "namespace_package_files": [] + select({ - Label("//python/config_settings:_is_venvs_site_packages_yes"): [], - "//conditions:default": ["_create_inits_target"], - }), - }) # buildifier: @unsorted-dict-items - - env.expect.that_collection(m_glob.calls, expr = "glob calls").contains_exactly([ - mocks.glob_call(["bin/*"], allow_empty = True), - mocks.glob_call(["rewrite-bin/*"], allow_empty = True), - mocks.glob_call(["site-packages/**/*.py"], exclude = [], allow_empty = True), - mocks.glob_call(["site-packages/**/*"], exclude = [ - "**/*.py", - "**/*.pyc", - "**/*.pyc.*", - ], allow_empty = True), - mocks.glob_call(["site-packages/**/*.pyi"], allow_empty = True), - ]) - -_tests.append(_test_group) - def _test_sdist_excludes_record(env): py_library_calls = [] m_glob = mocks.glob() From 097f6a15c186f170a0471a0551e47fc910889693 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Sun, 28 Jun 2026 22:51:06 -0700 Subject: [PATCH 789/922] chore(release): correct version tagging and improve promotion flow (#3866) This change refactors the release promotion tool to ensure all pre-conditions are verified before making any modifications and automates post-promotion tracking updates. It also fixes a bug where the tool incorrectly assumed 'v' prefixes when identifying the latest release candidate and computing the final version tag. --- tests/tools/private/release/release_test.py | 406 +++++++++++++++++++- tools/private/release/gh.py | 71 ++-- tools/private/release/git.py | 6 +- tools/private/release/release.py | 125 ++++-- tools/private/release/utils.py | 5 +- 5 files changed, 541 insertions(+), 72 deletions(-) diff --git a/tests/tools/private/release/release_test.py b/tests/tools/private/release/release_test.py index 9f9cd2e801..4ec7de53ba 100644 --- a/tests/tools/private/release/release_test.py +++ b/tests/tools/private/release/release_test.py @@ -3,7 +3,7 @@ import shutil import tempfile import unittest -from unittest.mock import patch +from unittest.mock import MagicMock, patch from tools.private.release import changelog_news, release as releaser @@ -528,6 +528,33 @@ def test_get_latest_version_only_rc_tags(self, mock_get_tags): releaser.get_latest_version() +class GetLatestRcTagTest(unittest.TestCase): + @patch("tools.private.release.release.git.get_tags") + def test_get_latest_rc_tag_no_tags(self, mock_get_tags): + mock_get_tags.return_value = [] + self.assertIsNone(releaser.get_latest_rc_tag("2.0.0")) + + @patch("tools.private.release.release.git.get_tags") + def test_get_latest_rc_tag_no_matching_tags(self, mock_get_tags): + mock_get_tags.return_value = ["1.0.0", "2.0.0", "v2.0.0-rc0", "2.1.0-rc0"] + self.assertIsNone(releaser.get_latest_rc_tag("2.0.0")) + + @patch("tools.private.release.release.git.get_tags") + def test_get_latest_rc_tag_success(self, mock_get_tags): + mock_get_tags.return_value = [ + "2.0.0-rc0", + "2.0.0-rc2", + "2.0.0-rc1", + "2.1.0-rc0", + ] + self.assertEqual(releaser.get_latest_rc_tag("2.0.0"), "2.0.0-rc2") + + @patch("tools.private.release.release.git.get_tags") + def test_get_latest_rc_tag_ignores_v_prefix(self, mock_get_tags): + mock_get_tags.return_value = ["v2.0.0-rc0", "2.0.0-rc1"] + self.assertEqual(releaser.get_latest_rc_tag("2.0.0"), "2.0.0-rc1") + + class DetermineNextVersionTest(unittest.TestCase): def setUp(self): self.tmpdir = pathlib.Path(tempfile.mkdtemp()) @@ -648,5 +675,382 @@ def test_determine_next_version_on_main_branch_fallback(self, mock_get_branch): self.assertEqual(next_version, "1.2.4") +class CmdPrepareTest(unittest.TestCase): + def setUp(self): + self.mock_git = patch("tools.private.release.release.git").start() + self.mock_gh = patch("tools.private.release.release.gh").start() + self.addCleanup(patch.stopall) + + @patch("tools.private.release.release.pathlib.Path") + @patch("tools.private.release.release.changelog_news") + @patch("tools.private.release.release.replace_version_next") + def test_prepare_success_existing_issue( + self, mock_replace, mock_changelog, mock_path + ): + # Arrange + args = MagicMock(version="2.0.0", issue=None) + self.mock_git.status.side_effect = ["", "M foo"] + self.mock_git.branch_exists.return_value = False + self.mock_gh.get_release_tracking_issue.return_value = 123 + self.mock_gh.create_pr.return_value = "https://github.com/foo/bar/pull/456" + self.mock_gh.get_issue_body.return_value = "- [ ] Prepare Release" + + # Act + result = releaser.cmd_prepare(args) + + # Assert + self.assertEqual(result, 0) + self.mock_gh.get_release_tracking_issue.assert_called_once_with("2.0.0") + self.mock_gh.create_tracking_issue.assert_not_called() + self.mock_gh.create_pr.assert_called_once_with("2.0.0", "prepare-2.0.0", 123) + + @patch("tools.private.release.release.pathlib.Path") + @patch("tools.private.release.release.changelog_news") + @patch("tools.private.release.release.replace_version_next") + def test_prepare_success_create_issue( + self, mock_replace, mock_changelog, mock_path + ): + # Arrange + args = MagicMock(version="2.0.0", issue=None) + self.mock_git.status.side_effect = ["", "M foo"] + self.mock_git.branch_exists.return_value = False + self.mock_gh.get_release_tracking_issue.side_effect = ValueError("Not found") + self.mock_gh.create_tracking_issue.return_value = 123 + self.mock_gh.create_pr.return_value = "https://github.com/foo/bar/pull/456" + self.mock_gh.get_issue_body.return_value = "- [ ] Prepare Release" + + mock_template = MagicMock() + mock_template.exists.return_value = True + mock_template.read_text.return_value = "template content" + mock_path.return_value = mock_template + + # Act + result = releaser.cmd_prepare(args) + + # Assert + self.assertEqual(result, 0) + self.mock_gh.get_release_tracking_issue.assert_called_once_with("2.0.0") + self.mock_gh.create_tracking_issue.assert_called_once_with( + "2.0.0", "template content" + ) + self.mock_gh.create_pr.assert_called_once_with("2.0.0", "prepare-2.0.0", 123) + + @patch("tools.private.release.release.pathlib.Path") + @patch("tools.private.release.release.changelog_news") + @patch("tools.private.release.release.replace_version_next") + def test_prepare_ambiguous_issue(self, mock_replace, mock_changelog, mock_path): + # Arrange + args = MagicMock(version="2.0.0", issue=None) + self.mock_git.status.side_effect = ["", "M foo"] + self.mock_git.branch_exists.return_value = False + self.mock_gh.get_release_tracking_issue.side_effect = ValueError( + "Multiple open tracking issues" + ) + + # Act + result = releaser.cmd_prepare(args) + + # Assert + self.assertEqual(result, 1) + self.mock_gh.get_release_tracking_issue.assert_called_once_with("2.0.0") + self.mock_gh.create_tracking_issue.assert_not_called() + self.mock_gh.create_pr.assert_not_called() + + +class CmdCreateRcTest(unittest.TestCase): + def setUp(self): + self.mock_git = patch("tools.private.release.release.git").start() + self.mock_gh = patch("tools.private.release.release.gh").start() + self.addCleanup(patch.stopall) + + def test_create_rc_success_first_rc(self): + # Arrange + args = MagicMock(issue=123) + self.mock_gh.get_issue_title.return_value = "Release 2.0.0" + self.mock_gh.get_issue_body.return_value = """ +## Checklist +- [x] Prepare Release | status=done pr=#122 commit=abcdef12 +- [x] Create Release branch | status=done branch=release/2.0 commit=abcdef12 +- [ ] Tag RC0 | status=pending +""" + self.mock_git.get_tags.return_value = [] + self.mock_git.get_tags_at_head.return_value = [] + self.mock_git.get_commit_sha.return_value = "1234567890" + + # Act + result = releaser.cmd_create_rc(args) + + # Assert + self.assertEqual(result, 0) + self.mock_git.tag.assert_called_once_with("2.0.0-rc0", "HEAD") + self.mock_git.push.assert_called_once_with("origin", "2.0.0-rc0") + + self.mock_gh.update_issue_body.assert_called_once() + call_args = self.mock_gh.update_issue_body.call_args[0] + self.assertEqual(call_args[0], 123) + self.assertIn("tag=2.0.0-rc0", call_args[1]) + self.assertIn("commit=12345678", call_args[1]) + + self.mock_gh.post_issue_comment.assert_called_once() + + def test_create_rc_success_next_rc(self): + # Arrange + args = MagicMock(issue=123) + self.mock_gh.get_issue_title.return_value = "Release 2.0.0" + self.mock_gh.get_issue_body.return_value = """ +## Checklist +- [x] Prepare Release | status=done pr=#122 commit=abcdef12 +- [x] Create Release branch | status=done branch=release/2.0 commit=abcdef12 +- [x] Tag RC0 | status=done tag=2.0.0-rc0 commit=abcdef12 +- [ ] Tag RC1 | status=pending +""" + self.mock_git.get_tags.return_value = ["2.0.0-rc0"] + self.mock_git.get_tags_at_head.return_value = [] + self.mock_git.get_commit_sha.return_value = "1234567890" + + # Act + result = releaser.cmd_create_rc(args) + + # Assert + self.assertEqual(result, 0) + self.mock_git.tag.assert_called_once_with("2.0.0-rc1", "HEAD") + self.mock_git.push.assert_called_once_with("origin", "2.0.0-rc1") + + self.mock_gh.update_issue_body.assert_called_once() + call_args = self.mock_gh.update_issue_body.call_args[0] + self.assertEqual(call_args[0], 123) + self.assertIn("tag=2.0.0-rc1", call_args[1]) + + self.mock_gh.post_issue_comment.assert_called_once() + + def test_create_rc_already_tagged(self): + # Arrange + args = MagicMock(issue=123) + self.mock_gh.get_issue_title.return_value = "Release 2.0.0" + self.mock_gh.get_issue_body.return_value = """ +## Checklist +- [x] Prepare Release | status=done pr=#122 commit=abcdef12 +- [x] Create Release branch | status=done branch=release/2.0 commit=abcdef12 +- [ ] Tag RC0 | status=pending +""" + self.mock_git.get_tags.return_value = [] + self.mock_git.get_tags_at_head.return_value = ["2.0.0-rc0"] + + # Act + result = releaser.cmd_create_rc(args) + + # Assert + self.assertEqual(result, 0) + self.mock_git.tag.assert_not_called() + self.mock_git.push.assert_not_called() + self.mock_gh.update_issue_body.assert_not_called() + + +class CmdPromoteRcTest(unittest.TestCase): + def setUp(self): + self.mock_git = patch("tools.private.release.release.git").start() + self.mock_gh = patch("tools.private.release.release.gh").start() + self.addCleanup(patch.stopall) + + def test_promote_rc_success(self): + # Arrange + args = MagicMock(version="2.0.0", issue=123, dry_run=False) + self.mock_git.get_tags.return_value = ["2.0.0-rc0", "2.0.0-rc1"] + self.mock_git.get_commit_sha.return_value = "abcdef123456" + self.mock_git.tag_exists.return_value = False + initial_body = "- [ ] Tag Final" + self.mock_gh.get_issue_body.return_value = initial_body + + # Act + result = releaser.cmd_promote_rc(args) + + # Assert + self.assertEqual(result, 0) + self.mock_git.fetch.assert_called_once_with("upstream", tags=True, force=True) + self.mock_git.get_commit_sha.assert_called_once_with("2.0.0-rc1") + self.mock_git.checkout.assert_not_called() + self.mock_git.tag_exists.assert_called_once_with("2.0.0") + self.mock_git.tag.assert_called_once_with("2.0.0", "abcdef123456") + self.mock_git.push.assert_called_once_with("upstream", "2.0.0") + + # Verify issue update + self.mock_gh.get_issue_body.assert_called_once_with(123) + expected_updated_body = ( + "- [x] Tag Final | status=done tag=2.0.0 commit=abcdef12" + ) + self.mock_gh.update_issue_body.assert_called_once_with( + 123, expected_updated_body + ) + expected_comment = ( + "Version 2.0.0 has been tagged.\n\n" + "- **Release Page**: https://github.com/bazel-contrib/rules_python/releases/tag/2.0.0\n" + '- **BCR PR Search**: [is:pr ("bazel-contrib/rules_python" in:title) ("@2.0.0" in:title)](https://github.com/bazelbuild/bazel-central-registry/pulls?q=is%3Apr%20%28%22bazel-contrib/rules_python%22%20in%3Atitle%29%20%28%22%402.0.0%22%20in%3Atitle%29)' + ) + self.mock_gh.post_issue_comment.assert_called_once_with(123, expected_comment) + + def test_promote_rc_resolve_issue_success(self): + # Arrange + args = MagicMock(version="2.0.0", issue=None, dry_run=False) + self.mock_git.get_tags.return_value = ["2.0.0-rc1"] + self.mock_git.tag_exists.return_value = False + self.mock_gh.get_release_tracking_issue.return_value = 123 + self.mock_git.get_commit_sha.return_value = "abcdef123456" + initial_body = "- [ ] Tag Final" + self.mock_gh.get_issue_body.return_value = initial_body + + # Act + result = releaser.cmd_promote_rc(args) + + # Assert + self.assertEqual(result, 0) + self.mock_gh.get_release_tracking_issue.assert_called_once_with("2.0.0") + self.mock_git.get_commit_sha.assert_called_once_with("2.0.0-rc1") + self.mock_git.checkout.assert_not_called() + self.mock_git.tag.assert_called_once_with("2.0.0", "abcdef123456") + self.mock_git.push.assert_called_once_with("upstream", "2.0.0") + self.mock_gh.get_issue_body.assert_called_once_with(123) + expected_updated_body = ( + "- [x] Tag Final | status=done tag=2.0.0 commit=abcdef12" + ) + self.mock_gh.update_issue_body.assert_called_once_with( + 123, expected_updated_body + ) + expected_comment = ( + "Version 2.0.0 has been tagged.\n\n" + "- **Release Page**: https://github.com/bazel-contrib/rules_python/releases/tag/2.0.0\n" + '- **BCR PR Search**: [is:pr ("bazel-contrib/rules_python" in:title) ("@2.0.0" in:title)](https://github.com/bazelbuild/bazel-central-registry/pulls?q=is%3Apr%20%28%22bazel-contrib/rules_python%22%20in%3Atitle%29%20%28%22%402.0.0%22%20in%3Atitle%29)' + ) + self.mock_gh.post_issue_comment.assert_called_once_with(123, expected_comment) + + def test_promote_rc_defaults_to_determine_next_version(self): + # Arrange + args = MagicMock(version=None, issue=123, dry_run=False) + self.mock_git.get_current_branch.return_value = "release/2.0" + self.mock_git.get_tags.return_value = ["2.0.0", "2.0.1-rc0"] + self.mock_git.get_commit_sha.return_value = "12345678" + self.mock_git.tag_exists.return_value = False + initial_body = "- [ ] Tag Final" + self.mock_gh.get_issue_body.return_value = initial_body + + # Act + result = releaser.cmd_promote_rc(args) + + # Assert + self.assertEqual(result, 0) + self.mock_git.get_current_branch.assert_called_once() + self.assertTrue(self.mock_git.get_tags.call_count >= 2) + + self.mock_git.checkout.assert_not_called() + self.mock_git.get_commit_sha.assert_called_once_with("2.0.1-rc0") + self.mock_git.tag.assert_called_once_with("2.0.1", "12345678") + self.mock_git.push.assert_called_once_with("upstream", "2.0.1") + + expected_updated_body = ( + "- [x] Tag Final | status=done tag=2.0.1 commit=12345678" + ) + self.mock_gh.update_issue_body.assert_called_once_with( + 123, expected_updated_body + ) + expected_comment = ( + "Version 2.0.1 has been tagged.\n\n" + "- **Release Page**: https://github.com/bazel-contrib/rules_python/releases/tag/2.0.1\n" + '- **BCR PR Search**: [is:pr ("bazel-contrib/rules_python" in:title) ("@2.0.1" in:title)](https://github.com/bazelbuild/bazel-central-registry/pulls?q=is%3Apr%20%28%22bazel-contrib/rules_python%22%20in%3Atitle%29%20%28%22%402.0.1%22%20in%3Atitle%29)' + ) + self.mock_gh.post_issue_comment.assert_called_once_with(123, expected_comment) + + def test_promote_rc_dry_run_success(self): + # Arrange + args = MagicMock(version="2.0.0", issue=123, dry_run=True) + self.mock_git.get_tags.return_value = ["2.0.0-rc0", "2.0.0-rc1"] + self.mock_git.get_commit_sha.return_value = "abcdef123456" + self.mock_git.tag_exists.return_value = False + initial_body = "- [ ] Tag Final" + self.mock_gh.get_issue_body.return_value = initial_body + + # Act + result = releaser.cmd_promote_rc(args) + + # Assert + self.assertEqual(result, 0) + self.mock_git.fetch.assert_called_once_with("upstream", tags=True, force=True) + self.mock_git.get_commit_sha.assert_called_once_with("2.0.0-rc1") + self.mock_git.tag_exists.assert_called_once_with("2.0.0") + + # Core dry-run assertions: NO modifications + self.mock_git.tag.assert_not_called() + self.mock_git.push.assert_not_called() + self.mock_gh.update_issue_body.assert_not_called() + self.mock_gh.post_issue_comment.assert_not_called() + + def test_promote_rc_tag_already_exists(self): + # Arrange + args = MagicMock(version="2.0.0", issue=123) + self.mock_git.get_tags.return_value = ["2.0.0-rc1"] + self.mock_git.tag_exists.return_value = True + + # Act + result = releaser.cmd_promote_rc(args) + + # Assert + self.assertEqual(result, 1) + self.mock_git.checkout.assert_not_called() + self.mock_git.tag.assert_not_called() + self.mock_git.push.assert_not_called() + self.mock_gh.get_issue_body.assert_not_called() + self.mock_gh.update_issue_body.assert_not_called() + + def test_promote_rc_issue_not_found(self): + # Arrange + args = MagicMock(version="2.0.0", issue=None) + self.mock_git.get_tags.return_value = ["2.0.0-rc1"] + self.mock_git.tag_exists.return_value = False + self.mock_gh.get_release_tracking_issue.side_effect = ValueError("Not found") + + # Act + result = releaser.cmd_promote_rc(args) + + # Assert + self.assertEqual(result, 1) + self.mock_gh.get_release_tracking_issue.assert_called_once_with("2.0.0") + self.mock_git.checkout.assert_not_called() + self.mock_git.tag.assert_not_called() + self.mock_git.push.assert_not_called() + self.mock_gh.get_issue_body.assert_not_called() + + def test_promote_rc_issue_malformed(self): + # Arrange + args = MagicMock(version="2.0.0", issue=123) + self.mock_git.get_tags.return_value = ["2.0.0-rc1"] + self.mock_git.tag_exists.return_value = False + self.mock_git.get_commit_sha.return_value = "abcdef123456" + initial_body = "malformed body" + self.mock_gh.get_issue_body.return_value = initial_body + + # Act + result = releaser.cmd_promote_rc(args) + + # Assert + self.assertEqual(result, 1) + self.mock_gh.get_issue_body.assert_called_once_with(123) + self.mock_git.checkout.assert_not_called() + self.mock_git.tag.assert_not_called() + self.mock_git.push.assert_not_called() + self.mock_gh.update_issue_body.assert_not_called() + + def test_promote_rc_no_rc_found(self): + # Arrange + args = MagicMock(version="2.0.0", issue=123) + self.mock_git.get_tags.return_value = [] + + # Act + result = releaser.cmd_promote_rc(args) + + # Assert + self.assertEqual(result, 1) + self.mock_git.checkout.assert_not_called() + self.mock_git.tag.assert_not_called() + self.mock_gh.get_issue_body.assert_not_called() + + if __name__ == "__main__": unittest.main() diff --git a/tools/private/release/gh.py b/tools/private/release/gh.py index 9fa94eee20..09a0d3ae47 100644 --- a/tools/private/release/gh.py +++ b/tools/private/release/gh.py @@ -6,41 +6,62 @@ from tools.private.release.utils import run_cmd +_REPO = "bazel-contrib/rules_python" +_LABEL = "type: release" + + +def list_issues(*, fields, label=None, state=None, search=None): + """Helper to list issues using gh CLI.""" + cmd = ["gh", "issue", "list", f"--repo={_REPO}"] + if label: + cmd.append(f"--label={label}") + if state: + cmd.append(f"--state={state}") + if search: + cmd.append(f"--search={search}") + cmd.append(f"--json={fields}") + + output = run_cmd(*cmd) + return json.loads(output) if output else [] -def get_open_tracking_issues(): - """Returns a list of open tracking issues with the 'type:release' label.""" - output = run_cmd( - "gh", - "issue", - "list", - "--label=type:release", - "--state=open", - "--json=number,title,url", + +def get_open_tracking_issues(version=None): + """Returns a list of open tracking issues with the 'type: release' label.""" + search = f'"Release {version}" in:title' if version else None + return list_issues( + label=_LABEL, + state="open", + search=search, + fields="number,title,url", ) - return json.loads(output) if output else [] -def resolve_issue_number(version): +def get_release_tracking_issue(version): """Resolves the tracking issue number for a given version. - Searches for an open issue with label 'type:release' and 'Release ' in the title. + Searches for an open issue with label 'type: release' and 'Release ' in the title. Raises ValueError if 0 or multiple issues are found. """ - matching_issues = [] - for issue in get_open_tracking_issues(): - if f"Release {version}" in issue["title"]: - matching_issues.append(issue) - - if not matching_issues: - raise ValueError(f"No open tracking issue found matching 'Release {version}'") - if len(matching_issues) > 1: - urls = [issue["url"] for issue in matching_issues] + matching_issues = get_open_tracking_issues(version) + + exact_matches = [] + for issue in matching_issues: + if issue["title"] == f"Release {version}": + exact_matches.append(issue) + + if not exact_matches: + raise ValueError( + f"No open tracking issue found matching 'Release {version}' " + f"in repo {_REPO} with label '{_LABEL}'" + ) + if len(exact_matches) > 1: + urls = [issue["url"] for issue in exact_matches] raise ValueError( - f"Multiple open tracking issues found for version {version}:\n" - + "\n".join(urls) + f"Multiple open tracking issues found for version {version} " + f"in repo {_REPO} with label '{_LABEL}':\n" + "\n".join(urls) ) - return matching_issues[0]["number"] + return exact_matches[0]["number"] def create_tracking_issue(version, template_content): @@ -63,7 +84,7 @@ def create_tracking_issue(version, template_content): "issue", "create", f"--title=Release {version}", - "--label=type:release", + f"--label={_LABEL}", f"--body-file={temp_path}", ) issue_url = output.strip() diff --git a/tools/private/release/git.py b/tools/private/release/git.py index 9bfd905109..4e623e2f7a 100644 --- a/tools/private/release/git.py +++ b/tools/private/release/git.py @@ -59,9 +59,9 @@ def merge(commit_ref, ff_only=True): run_cmd(*cmd, capture_output=False) -def tag(tag_name): - """Creates a local tag pointing to HEAD.""" - run_cmd("git", "tag", tag_name, capture_output=False) +def tag(tag_name, commit_ref): + """Creates a local tag pointing to a specific commit.""" + run_cmd("git", "tag", tag_name, commit_ref, capture_output=False) def cherry_pick(sha): diff --git a/tools/private/release/release.py b/tools/private/release/release.py index 0e509f1bf3..3c2eed82ff 100644 --- a/tools/private/release/release.py +++ b/tools/private/release/release.py @@ -76,7 +76,7 @@ def get_latest_version(): def get_latest_rc_tag(version): """Queries git tags and returns the highest RC tag for the version.""" tags = git.get_tags() - pattern = rf"^v{re.escape(version)}-rc\d+$" + pattern = rf"^{re.escape(version)}-rc\d+$" rc_tags = [tag.strip() for tag in tags if re.match(pattern, tag.strip())] if not rc_tags: return None @@ -227,7 +227,10 @@ def update_task_in_body(body, task_name, checked, metadata): updated_lines.append(line) if not found: - raise ValueError(f"Task '{task_name}' not found in issue body.") + raise ValueError( + f"Task '{task_name}' not found in issue body. " + f"Expected format: '- [ ] {task_name}' or '- [x] {task_name}' (optionally followed by '| key=value')" + ) return "\n".join(updated_lines) @@ -413,13 +416,13 @@ def cmd_prepare(args): issue_num = args.issue if not issue_num: - open_issues = gh.get_open_tracking_issues() - for issue in open_issues: - if f"Release {version}" in issue["title"]: - issue_num = issue["number"] - break - - if not issue_num: + try: + issue_num = gh.get_release_tracking_issue(version) + print(f"Found active tracking issue #{issue_num} for v{version}") + except ValueError as e: + if "Multiple open tracking issues" in str(e): + print(f"Error: {e}") + return 1 print( f"No active tracking issue found for v{version}. Creating a new one..." ) @@ -710,18 +713,18 @@ def cmd_create_rc(args): if not latest_rc: next_rc_num = 0 - next_rc = f"v{version}-rc0" + next_rc = f"{version}-rc0" else: rc_num = int(latest_rc.split("-rc")[-1]) next_rc_num = rc_num + 1 - next_rc = f"v{version}-rc{next_rc_num}" + next_rc = f"{version}-rc{next_rc_num}" # Precheck: next RC number must exist and be unchecked in the checklist rc_tags = state.get("rc_tags", {}) if next_rc_num not in rc_tags: print( f"Error: Checklist is missing required task 'Tag RC{next_rc_num}'" - f" to cut v{version}-rc{next_rc_num}." + f" to cut {version}-rc{next_rc_num}." ) return 1 @@ -735,12 +738,12 @@ def cmd_create_rc(args): # Verify HEAD is not already tagged git.checkout(branch_name) head_tags = git.get_tags_at_head() - if any(tag.startswith(f"v{version}-rc") for tag in head_tags): + if any(tag.startswith(f"{version}-rc") for tag in head_tags): print(f"HEAD of {branch_name} is already tagged with an RC. Skipping.") return 0 print(f"Tagging and pushing next RC: {next_rc}...") - git.tag(next_rc) + git.tag(next_rc, "HEAD") git.push("origin", next_rc) commit_sha = git.get_commit_sha("HEAD") @@ -770,49 +773,83 @@ def cmd_promote_rc(args): version = args.version if version is None: version = determine_next_version() - version = version.replace("v", "") - final_tag = f"v{version}" - git.fetch("--tags", "--force") + # Fetch from upstream to ensure we have the latest tags + git.fetch("upstream", tags=True, force=True) latest_rc = get_latest_rc_tag(version) if not latest_rc: - print(f"Error: No release candidate tags found matching v{version}-rc*") + print(f"Error: No release candidate tags found matching {version}-rc*") return 1 - print(f"Promoting {latest_rc} to final release {final_tag}...") - git.checkout(latest_rc) - - commit_sha = git.get_commit_sha("HEAD") - - if not git.tag_exists(final_tag): - git.tag(final_tag) - git.push("origin", final_tag) - else: - print(f"Final tag {final_tag} already exists.") + # Verify final tag doesn't already exist + if git.tag_exists(version): + print(f"Error: Final tag {version} already exists.") + return 1 - # Resolve issue number + # Verify issue can be found issue_num = args.issue if not issue_num: try: - issue_num = gh.resolve_issue_number(version) + issue_num = gh.get_release_tracking_issue(version) + except ValueError as e: + print(f"Error: {e}") + return 1 except Exception as e: - print(f"Warning: Could not query GitHub to find tracking issue: {e}") + print(f"Error: Unexpected error finding tracking issue: {e}") + return 1 + + # Get commit SHA of the RC tag (which will be the same for the final tag) + commit_sha = git.get_commit_sha(latest_rc) - if issue_num: - print(f"Updating tracking issue #{issue_num} checklist...") - body = gh.get_issue_body(issue_num) - metadata = {"status": "done", "tag": final_tag, "commit": commit_sha[:8]} + # Verify issue is in the right format by trying to prepare the update + print(f"Verifying tracking issue #{issue_num} format...") + body = gh.get_issue_body(issue_num) + metadata = {"status": "done", "tag": version, "commit": commit_sha[:8]} + try: updated_body = update_task_in_body( body, "Tag Final", checked=True, metadata=metadata ) - gh.update_issue_body(issue_num, updated_body) - print("Checklist updated successfully.") - return 0 - else: + except ValueError as e: + print(f"Error: Tracking issue #{issue_num} is malformed: {e}") + return 1 + + # All pre-conditions met, perform modifications + if args.dry_run: print( - "Error: No active tracking issue found or specified. Checklist was not updated." + f"[DRY RUN] Pre-conditions passed successfully for promoting {latest_rc} to {version}." ) - return 1 + print(f"[DRY RUN] Would tag commit {commit_sha[:8]} as {version}") + print(f"[DRY RUN] Would push tag {version} to upstream") + print(f"[DRY RUN] Would update tracking issue #{issue_num} checklist") + print(f"[DRY RUN] Would post comment to tracking issue #{issue_num}") + return 0 + + print( + f"Promoting {latest_rc} to final release {version} (commit" + f" {commit_sha[:8]}) using tracking issue #{issue_num}..." + ) + + # Tag the specific commit without checkout, and push to upstream + git.tag(version, commit_sha) + git.push("upstream", version) + + print(f"Updating tracking issue #{issue_num} checklist...") + gh.update_issue_body(issue_num, updated_body) + + print(f"Posting comment to tracking issue #{issue_num}...") + import urllib.parse + + release_url = f"{_REPO_URL}/releases/tag/{version}" + bcr_query = f'is:pr ("bazel-contrib/rules_python" in:title) ("@{version}" in:title)' + bcr_search_url = f"https://github.com/bazelbuild/bazel-central-registry/pulls?q={urllib.parse.quote(bcr_query)}" + comment_body = ( + f"Version {version} has been tagged.\n\n" + f"- **Release Page**: {release_url}\n" + f"- **BCR PR Search**: [{bcr_query}]({bcr_search_url})" + ) + gh.post_issue_comment(issue_num, comment_body) + + return 0 def create_parser(): @@ -924,6 +961,12 @@ def create_parser(): type=int, help="The tracking issue number (optional).", ) + promote_parser.add_argument( + "--dry-run", + action=argparse.BooleanOptionalAction, + default=True, + help="Perform a dry run (default: True). Use --no-dry-run to actually execute.", + ) return parser diff --git a/tools/private/release/utils.py b/tools/private/release/utils.py index 5a8411b657..5eb7c43a71 100644 --- a/tools/private/release/utils.py +++ b/tools/private/release/utils.py @@ -1,5 +1,6 @@ """Utility functions for the release tool.""" +import shlex import subprocess @@ -10,7 +11,7 @@ def run_cmd(*args, check=True, capture_output=True): a detailed note explaining the failure to preserve the stack trace. """ cmd = [str(arg) for arg in args] - print(f"Running: {' '.join(cmd)}") + print(f"Running: {shlex.join(cmd)}") try: result = subprocess.run( cmd, @@ -21,7 +22,7 @@ def run_cmd(*args, check=True, capture_output=True): ) return result.stdout.strip() if capture_output else None except subprocess.CalledProcessError as e: - note = f"Error running command: {' '.join(cmd)}" + note = f"Error running command: {shlex.join(cmd)}" if capture_output: note += f"\nStdout: {e.stdout}\nStderr: {e.stderr}" e.add_note(note) From 9734b11bbd1ec2d04431bd0865b1a59778473397 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Mon, 29 Jun 2026 18:05:41 -0700 Subject: [PATCH 790/922] chore(release): add dry-run to release tool prepare command (#3869) To enable safe testing of the release pipeline, this change introduces a --dry-run flag to the prepare command that simulates all repository and GitHub actions. This also fixes a few issues with the prepare command not fully working end-to-end. Along the way, factor out the code a bit. release.py was over 1000 lines and becoming unwieldly. --- .github/workflows/prepare_release.yml | 2 +- tests/tools/private/release/release_test.py | 253 ++++++++++------ tools/private/release/BUILD.bazel | 3 + tools/private/release/gh.py | 22 +- tools/private/release/git.py | 15 +- tools/private/release/prepare.py | 144 +++++++++ tools/private/release/release.py | 314 ++------------------ tools/private/release/release_issue.py | 62 ++++ tools/private/release/shell.py | 29 ++ tools/private/release/utils.py | 182 ++++++++++-- 10 files changed, 601 insertions(+), 425 deletions(-) create mode 100644 tools/private/release/prepare.py create mode 100644 tools/private/release/release_issue.py create mode 100644 tools/private/release/shell.py diff --git a/.github/workflows/prepare_release.yml b/.github/workflows/prepare_release.yml index 7d5e93aa6b..b080f121eb 100644 --- a/.github/workflows/prepare_release.yml +++ b/.github/workflows/prepare_release.yml @@ -31,6 +31,6 @@ jobs: - name: Run Release Preparation Pipeline run: | # Manual trigger: run full preparation - bazel run //tools/private/release -- prepare + bazel run //tools/private/release -- prepare --no-dry-run env: GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} diff --git a/tests/tools/private/release/release_test.py b/tests/tools/private/release/release_test.py index 4ec7de53ba..534aa2a5fd 100644 --- a/tests/tools/private/release/release_test.py +++ b/tests/tools/private/release/release_test.py @@ -5,19 +5,48 @@ import unittest from unittest.mock import MagicMock, patch -from tools.private.release import changelog_news, release as releaser +from tools.private.release import changelog_news, release as releaser, utils +from tools.private.release.gh import MultipleTrackingIssuesError, NoTrackingIssueError -class ReleaserTest(unittest.TestCase): +def _mock_git_and_gh(test_case): + mock_git = MagicMock() + mock_gh = MagicMock() + test_case.mock_git = mock_git + test_case.mock_gh = mock_gh + + # Patch bindings in modules that import them at module level + patch("tools.private.release.release.git", new=mock_git).start() + patch("tools.private.release.prepare.git", new=mock_git).start() + patch("tools.private.release.utils.git", new=mock_git).start() + + patch("tools.private.release.release.gh", new=mock_gh).start() + patch("tools.private.release.prepare.gh", new=mock_gh).start() + mock_gh.MultipleTrackingIssuesError = MultipleTrackingIssuesError + mock_gh.NoTrackingIssueError = NoTrackingIssueError + + test_case.addCleanup(patch.stopall) + + # Apply safe defaults + mock_git.get_current_branch.return_value = None + mock_git.get_tags.return_value = [] + mock_git.get_tags_at_head.return_value = [] + mock_git.status.return_value = "" + mock_git.branch_exists.return_value = False + mock_git.tag_exists.return_value = False + mock_gh.get_release_tracking_issue.side_effect = NoTrackingIssueError("Not found") + + +class TempDirTestCase(unittest.TestCase): def setUp(self): self.tmpdir = pathlib.Path(tempfile.mkdtemp()) self.original_cwd = os.getcwd() self.addCleanup(shutil.rmtree, self.tmpdir) - os.chdir(self.tmpdir) - # NOTE: On windows, this must be done before files are deleted. self.addCleanup(os.chdir, self.original_cwd) + +class ReleaserTest(TempDirTestCase): def test_update_changelog_with_news(self): # Arrange changelog = """# Changelog @@ -431,7 +460,7 @@ def test_replace_version_next(self): """ (self.tmpdir / "mock_file.bzl").write_text(mock_file_content) - releaser.replace_version_next("0.28.0") + utils.replace_version_next("0.28.0") new_content = (self.tmpdir / "mock_file.bzl").read_text() @@ -462,7 +491,7 @@ def test_replace_version_next_excludes_bazel_dirs(self): version = "0.28.0" # Act - releaser.replace_version_next(version) + utils.replace_version_next(version) # Assert new_content = (bazel_dir / "mock_file.bzl").read_text() @@ -490,56 +519,56 @@ def test_invalid_version(self): class GetLatestVersionTest(unittest.TestCase): - @patch("tools.private.release.release.git.get_tags") + @patch("tools.private.release.git.get_tags") def test_get_latest_version_success(self, mock_get_tags): mock_get_tags.return_value = ["0.1.0", "1.0.0", "0.2.0"] - self.assertEqual(releaser.get_latest_version(), "1.0.0") + self.assertEqual(utils.get_latest_version(), "1.0.0") - @patch("tools.private.release.release.git.get_tags") + @patch("tools.private.release.git.get_tags") def test_get_latest_version_rc_is_latest(self, mock_get_tags): mock_get_tags.return_value = ["0.1.0", "1.0.0", "1.1.0rc0"] with self.assertRaisesRegex( ValueError, "The latest version is a pre-release version: 1.1.0rc0" ): - releaser.get_latest_version() + utils.get_latest_version() - @patch("tools.private.release.release.git.get_tags") + @patch("tools.private.release.git.get_tags") def test_get_latest_version_no_tags(self, mock_get_tags): mock_get_tags.return_value = [] with self.assertRaisesRegex( RuntimeError, "No git tags found matching X.Y.Z or X.Y.ZrcN format." ): - releaser.get_latest_version() + utils.get_latest_version() - @patch("tools.private.release.release.git.get_tags") + @patch("tools.private.release.git.get_tags") def test_get_latest_version_no_matching_tags(self, mock_get_tags): mock_get_tags.return_value = ["v1.0", "latest"] with self.assertRaisesRegex( RuntimeError, "No git tags found matching X.Y.Z or X.Y.ZrcN format." ): - releaser.get_latest_version() + utils.get_latest_version() - @patch("tools.private.release.release.git.get_tags") + @patch("tools.private.release.git.get_tags") def test_get_latest_version_only_rc_tags(self, mock_get_tags): mock_get_tags.return_value = ["1.0.0rc0", "1.1.0rc0"] with self.assertRaisesRegex( ValueError, "The latest version is a pre-release version: 1.1.0rc0" ): - releaser.get_latest_version() + utils.get_latest_version() class GetLatestRcTagTest(unittest.TestCase): - @patch("tools.private.release.release.git.get_tags") + @patch("tools.private.release.git.get_tags") def test_get_latest_rc_tag_no_tags(self, mock_get_tags): mock_get_tags.return_value = [] - self.assertIsNone(releaser.get_latest_rc_tag("2.0.0")) + self.assertIsNone(utils.get_latest_rc_tag("2.0.0")) - @patch("tools.private.release.release.git.get_tags") + @patch("tools.private.release.git.get_tags") def test_get_latest_rc_tag_no_matching_tags(self, mock_get_tags): mock_get_tags.return_value = ["1.0.0", "2.0.0", "v2.0.0-rc0", "2.1.0-rc0"] - self.assertIsNone(releaser.get_latest_rc_tag("2.0.0")) + self.assertIsNone(utils.get_latest_rc_tag("2.0.0")) - @patch("tools.private.release.release.git.get_tags") + @patch("tools.private.release.git.get_tags") def test_get_latest_rc_tag_success(self, mock_get_tags): mock_get_tags.return_value = [ "2.0.0-rc0", @@ -547,26 +576,19 @@ def test_get_latest_rc_tag_success(self, mock_get_tags): "2.0.0-rc1", "2.1.0-rc0", ] - self.assertEqual(releaser.get_latest_rc_tag("2.0.0"), "2.0.0-rc2") + self.assertEqual(utils.get_latest_rc_tag("2.0.0"), "2.0.0-rc2") - @patch("tools.private.release.release.git.get_tags") + @patch("tools.private.release.git.get_tags") def test_get_latest_rc_tag_ignores_v_prefix(self, mock_get_tags): mock_get_tags.return_value = ["v2.0.0-rc0", "2.0.0-rc1"] - self.assertEqual(releaser.get_latest_rc_tag("2.0.0"), "2.0.0-rc1") + self.assertEqual(utils.get_latest_rc_tag("2.0.0"), "2.0.0-rc1") -class DetermineNextVersionTest(unittest.TestCase): +class DetermineNextVersionTest(TempDirTestCase): def setUp(self): - self.tmpdir = pathlib.Path(tempfile.mkdtemp()) - self.original_cwd = os.getcwd() - self.addCleanup(shutil.rmtree, self.tmpdir) - - os.chdir(self.tmpdir) - # NOTE: On windows, this must be done before files are deleted. - self.addCleanup(os.chdir, self.original_cwd) - + super().setUp() self.mock_get_latest_version = patch( - "tools.private.release.release.get_latest_version" + "tools.private.release.utils.get_latest_version" ).start() self.addCleanup(patch.stopall) @@ -574,7 +596,7 @@ def test_no_markers(self): (self.tmpdir / "mock_file.bzl").write_text("no markers here") self.mock_get_latest_version.return_value = "1.2.3" - next_version = releaser.determine_next_version() + next_version = utils.determine_next_version() self.assertEqual(next_version, "1.2.4") @@ -584,7 +606,7 @@ def test_only_patch(self): ) self.mock_get_latest_version.return_value = "1.2.3" - next_version = releaser.determine_next_version() + next_version = utils.determine_next_version() self.assertEqual(next_version, "1.2.4") @@ -594,7 +616,7 @@ def test_only_feature(self): ) self.mock_get_latest_version.return_value = "1.2.3" - next_version = releaser.determine_next_version() + next_version = utils.determine_next_version() self.assertEqual(next_version, "1.3.0") @@ -607,36 +629,36 @@ def test_both_markers(self): ) self.mock_get_latest_version.return_value = "1.2.3" - next_version = releaser.determine_next_version() + next_version = utils.determine_next_version() self.assertEqual(next_version, "1.3.0") - @patch("tools.private.release.release.git.get_current_branch") - @patch("tools.private.release.release.git.get_tags") + @patch("tools.private.release.git.get_current_branch") + @patch("tools.private.release.git.get_tags") def test_determine_next_version_on_release_branch_with_existing_tags( self, mock_get_tags, mock_get_branch ): mock_get_branch.return_value = "release/0.37" mock_get_tags.return_value = ["0.37.0", "0.37.1", "0.36.0"] - next_version = releaser.determine_next_version() + next_version = utils.determine_next_version() self.assertEqual(next_version, "0.37.2") - @patch("tools.private.release.release.git.get_current_branch") - @patch("tools.private.release.release.git.get_tags") + @patch("tools.private.release.git.get_current_branch") + @patch("tools.private.release.git.get_tags") def test_determine_next_version_on_release_branch_no_tags( self, mock_get_tags, mock_get_branch ): mock_get_branch.return_value = "release/0.38" mock_get_tags.return_value = ["0.37.0"] # No 0.38.x tags - next_version = releaser.determine_next_version() + next_version = utils.determine_next_version() self.assertEqual(next_version, "0.38.0") - @patch("tools.private.release.release.git.get_current_branch") - @patch("tools.private.release.release.git.get_tags") + @patch("tools.private.release.git.get_current_branch") + @patch("tools.private.release.git.get_tags") def test_determine_next_version_on_release_branch_with_active_rc( self, mock_get_tags, mock_get_branch ): @@ -644,13 +666,13 @@ def test_determine_next_version_on_release_branch_with_active_rc( # 0.37.0-rc0 and rc1 exist, but no stable 0.37.0 yet mock_get_tags.return_value = ["0.37.0-rc0", "0.37.0-rc1", "0.36.0"] - next_version = releaser.determine_next_version() + next_version = utils.determine_next_version() # Should target 0.37.0, not 0.37.1 self.assertEqual(next_version, "0.37.0") - @patch("tools.private.release.release.git.get_current_branch") - @patch("tools.private.release.release.git.get_tags") + @patch("tools.private.release.git.get_current_branch") + @patch("tools.private.release.git.get_tags") def test_determine_next_version_on_release_branch_with_stable_and_active_patch_rc( self, mock_get_tags, mock_get_branch ): @@ -658,39 +680,36 @@ def test_determine_next_version_on_release_branch_with_stable_and_active_patch_r # 0.37.0 stable exists, and 0.37.1-rc0 exists (but no stable 0.37.1 yet) mock_get_tags.return_value = ["0.37.0", "0.37.1-rc0", "0.36.0"] - next_version = releaser.determine_next_version() + next_version = utils.determine_next_version() # Should target 0.37.1, not 0.37.2 self.assertEqual(next_version, "0.37.1") - @patch("tools.private.release.release.git.get_current_branch") + @patch("tools.private.release.git.get_current_branch") def test_determine_next_version_on_main_branch_fallback(self, mock_get_branch): mock_get_branch.return_value = "main" # Should fallback to default behavior (which uses mock_get_latest_version from setUp) self.mock_get_latest_version.return_value = "1.2.3" (self.tmpdir / "mock_file.bzl").write_text("no markers here") - next_version = releaser.determine_next_version() + next_version = utils.determine_next_version() self.assertEqual(next_version, "1.2.4") -class CmdPrepareTest(unittest.TestCase): +class CmdPrepareTest(TempDirTestCase): def setUp(self): - self.mock_git = patch("tools.private.release.release.git").start() - self.mock_gh = patch("tools.private.release.release.gh").start() - self.addCleanup(patch.stopall) + super().setUp() + _mock_git_and_gh(self) - @patch("tools.private.release.release.pathlib.Path") - @patch("tools.private.release.release.changelog_news") - @patch("tools.private.release.release.replace_version_next") - def test_prepare_success_existing_issue( - self, mock_replace, mock_changelog, mock_path - ): + @patch("tools.private.release.prepare.changelog_news") + @patch("tools.private.release.prepare.replace_version_next") + def test_prepare_success_existing_issue(self, mock_replace, mock_changelog): # Arrange - args = MagicMock(version="2.0.0", issue=None) + args = MagicMock(version="2.0.0", issue=None, dry_run=False) self.mock_git.status.side_effect = ["", "M foo"] self.mock_git.branch_exists.return_value = False + self.mock_gh.get_release_tracking_issue.side_effect = None self.mock_gh.get_release_tracking_issue.return_value = 123 self.mock_gh.create_pr.return_value = "https://github.com/foo/bar/pull/456" self.mock_gh.get_issue_body.return_value = "- [ ] Prepare Release" @@ -702,28 +721,28 @@ def test_prepare_success_existing_issue( self.assertEqual(result, 0) self.mock_gh.get_release_tracking_issue.assert_called_once_with("2.0.0") self.mock_gh.create_tracking_issue.assert_not_called() - self.mock_gh.create_pr.assert_called_once_with("2.0.0", "prepare-2.0.0", 123) + self.mock_gh.create_pr.assert_called_once_with("2.0.0", 123) + self.mock_git.add_modified_and_deleted.assert_called_once() - @patch("tools.private.release.release.pathlib.Path") - @patch("tools.private.release.release.changelog_news") - @patch("tools.private.release.release.replace_version_next") - def test_prepare_success_create_issue( - self, mock_replace, mock_changelog, mock_path - ): + @patch("tools.private.release.prepare.changelog_news") + @patch("tools.private.release.prepare.replace_version_next") + def test_prepare_success_create_issue(self, mock_replace, mock_changelog): # Arrange - args = MagicMock(version="2.0.0", issue=None) + template_dir = self.tmpdir / ".github" / "ISSUE_TEMPLATE" + template_dir.mkdir(parents=True, exist_ok=True) + template_file = template_dir / "release_tracking_template.md" + template_file.write_text("dummy template content") + + args = MagicMock(version="2.0.0", issue=None, dry_run=False) self.mock_git.status.side_effect = ["", "M foo"] self.mock_git.branch_exists.return_value = False - self.mock_gh.get_release_tracking_issue.side_effect = ValueError("Not found") + self.mock_gh.get_release_tracking_issue.side_effect = NoTrackingIssueError( + "Not found" + ) self.mock_gh.create_tracking_issue.return_value = 123 self.mock_gh.create_pr.return_value = "https://github.com/foo/bar/pull/456" self.mock_gh.get_issue_body.return_value = "- [ ] Prepare Release" - mock_template = MagicMock() - mock_template.exists.return_value = True - mock_template.read_text.return_value = "template content" - mock_path.return_value = mock_template - # Act result = releaser.cmd_prepare(args) @@ -731,20 +750,20 @@ def test_prepare_success_create_issue( self.assertEqual(result, 0) self.mock_gh.get_release_tracking_issue.assert_called_once_with("2.0.0") self.mock_gh.create_tracking_issue.assert_called_once_with( - "2.0.0", "template content" + "2.0.0", "dummy template content" ) - self.mock_gh.create_pr.assert_called_once_with("2.0.0", "prepare-2.0.0", 123) + self.mock_gh.create_pr.assert_called_once_with("2.0.0", 123) + self.mock_git.add_modified_and_deleted.assert_called_once() - @patch("tools.private.release.release.pathlib.Path") - @patch("tools.private.release.release.changelog_news") - @patch("tools.private.release.release.replace_version_next") - def test_prepare_ambiguous_issue(self, mock_replace, mock_changelog, mock_path): + @patch("tools.private.release.prepare.changelog_news") + @patch("tools.private.release.prepare.replace_version_next") + def test_prepare_ambiguous_issue(self, mock_replace, mock_changelog): # Arrange - args = MagicMock(version="2.0.0", issue=None) + args = MagicMock(version="2.0.0", issue=None, dry_run=False) self.mock_git.status.side_effect = ["", "M foo"] self.mock_git.branch_exists.return_value = False - self.mock_gh.get_release_tracking_issue.side_effect = ValueError( - "Multiple open tracking issues" + self.mock_gh.get_release_tracking_issue.side_effect = ( + MultipleTrackingIssuesError("Multiple open tracking issues") ) # Act @@ -755,13 +774,60 @@ def test_prepare_ambiguous_issue(self, mock_replace, mock_changelog, mock_path): self.mock_gh.get_release_tracking_issue.assert_called_once_with("2.0.0") self.mock_gh.create_tracking_issue.assert_not_called() self.mock_gh.create_pr.assert_not_called() + self.mock_git.add_modified_and_deleted.assert_not_called() + + @patch("tools.private.release.prepare.changelog_news") + @patch("tools.private.release.prepare.replace_version_next") + def test_prepare_dry_run(self, mock_replace, mock_changelog): + # Arrange + args = MagicMock(version="2.0.0", issue=None, dry_run=True) + self.mock_git.status.side_effect = [""] + self.mock_gh.get_release_tracking_issue.side_effect = None + self.mock_gh.get_release_tracking_issue.return_value = 123 + + # Act + result = releaser.cmd_prepare(args) + + # Assert + self.assertEqual(result, 0) + self.mock_git.checkout.assert_not_called() + self.mock_git.commit.assert_not_called() + self.mock_git.push.assert_not_called() + self.mock_gh.create_pr.assert_not_called() + self.mock_gh.update_issue_body.assert_not_called() + self.mock_git.fetch.assert_called_once() + self.mock_gh.get_release_tracking_issue.assert_called_once_with("2.0.0") + self.mock_git.add_modified_and_deleted.assert_not_called() + + @patch("tools.private.release.prepare.changelog_news") + @patch("tools.private.release.prepare.replace_version_next") + def test_prepare_dry_run_no_issue(self, mock_replace, mock_changelog): + # Arrange + template_dir = self.tmpdir / ".github" / "ISSUE_TEMPLATE" + template_dir.mkdir(parents=True, exist_ok=True) + template_file = template_dir / "release_tracking_template.md" + template_file.write_text("dummy template content") + + args = MagicMock(version="2.0.0", issue=None, dry_run=True) + self.mock_git.status.side_effect = [""] + self.mock_gh.get_release_tracking_issue.side_effect = NoTrackingIssueError( + "Not found" + ) + + # Act + result = releaser.cmd_prepare(args) + + # Assert + self.assertEqual(result, 0) + self.mock_git.checkout.assert_not_called() + self.mock_gh.create_tracking_issue.assert_not_called() + self.mock_gh.create_pr.assert_not_called() + self.mock_git.add_modified_and_deleted.assert_not_called() class CmdCreateRcTest(unittest.TestCase): def setUp(self): - self.mock_git = patch("tools.private.release.release.git").start() - self.mock_gh = patch("tools.private.release.release.gh").start() - self.addCleanup(patch.stopall) + _mock_git_and_gh(self) def test_create_rc_success_first_rc(self): # Arrange @@ -848,9 +914,7 @@ def test_create_rc_already_tagged(self): class CmdPromoteRcTest(unittest.TestCase): def setUp(self): - self.mock_git = patch("tools.private.release.release.git").start() - self.mock_gh = patch("tools.private.release.release.gh").start() - self.addCleanup(patch.stopall) + _mock_git_and_gh(self) def test_promote_rc_success(self): # Arrange @@ -893,6 +957,7 @@ def test_promote_rc_resolve_issue_success(self): args = MagicMock(version="2.0.0", issue=None, dry_run=False) self.mock_git.get_tags.return_value = ["2.0.0-rc1"] self.mock_git.tag_exists.return_value = False + self.mock_gh.get_release_tracking_issue.side_effect = None self.mock_gh.get_release_tracking_issue.return_value = 123 self.mock_git.get_commit_sha.return_value = "abcdef123456" initial_body = "- [ ] Tag Final" @@ -1004,7 +1069,9 @@ def test_promote_rc_issue_not_found(self): args = MagicMock(version="2.0.0", issue=None) self.mock_git.get_tags.return_value = ["2.0.0-rc1"] self.mock_git.tag_exists.return_value = False - self.mock_gh.get_release_tracking_issue.side_effect = ValueError("Not found") + self.mock_gh.get_release_tracking_issue.side_effect = NoTrackingIssueError( + "Not found" + ) # Act result = releaser.cmd_promote_rc(args) diff --git a/tools/private/release/BUILD.bazel b/tools/private/release/BUILD.bazel index 747cb74e10..96f864141d 100644 --- a/tools/private/release/BUILD.bazel +++ b/tools/private/release/BUILD.bazel @@ -12,7 +12,10 @@ py_binary( srcs = [ "gh.py", "git.py", + "prepare.py", "release.py", + "release_issue.py", + "shell.py", "utils.py", ], main = "release.py", diff --git a/tools/private/release/gh.py b/tools/private/release/gh.py index 09a0d3ae47..f37d87f396 100644 --- a/tools/private/release/gh.py +++ b/tools/private/release/gh.py @@ -4,12 +4,24 @@ import os import tempfile -from tools.private.release.utils import run_cmd +from tools.private.release.shell import run_cmd _REPO = "bazel-contrib/rules_python" _LABEL = "type: release" +class MultipleTrackingIssuesError(ValueError): + """Raised when multiple open tracking issues are found for a version.""" + + pass + + +class NoTrackingIssueError(ValueError): + """Raised when no open tracking issue is found for a version.""" + + pass + + def list_issues(*, fields, label=None, state=None, search=None): """Helper to list issues using gh CLI.""" cmd = ["gh", "issue", "list", f"--repo={_REPO}"] @@ -50,13 +62,13 @@ def get_release_tracking_issue(version): exact_matches.append(issue) if not exact_matches: - raise ValueError( + raise NoTrackingIssueError( f"No open tracking issue found matching 'Release {version}' " f"in repo {_REPO} with label '{_LABEL}'" ) if len(exact_matches) > 1: urls = [issue["url"] for issue in exact_matches] - raise ValueError( + raise MultipleTrackingIssuesError( f"Multiple open tracking issues found for version {version} " f"in repo {_REPO} with label '{_LABEL}':\n" + "\n".join(urls) ) @@ -138,7 +150,7 @@ def update_issue_body(issue_num, body): os.unlink(temp_path) -def create_pr(version, branch, issue_num): +def create_pr(version, issue_num): """Creates a pull request for release preparation.""" return run_cmd( "gh", @@ -146,9 +158,7 @@ def create_pr(version, branch, issue_num): "create", f"--title=Prepare release v{version}", f"--body=Work towards #{issue_num}", - f"--head={branch}", "--base=main", - "--label=release-prepared", ) diff --git a/tools/private/release/git.py b/tools/private/release/git.py index 4e623e2f7a..9c2662d4e6 100644 --- a/tools/private/release/git.py +++ b/tools/private/release/git.py @@ -2,7 +2,7 @@ import subprocess -from tools.private.release.utils import run_cmd +from tools.private.release.shell import run_cmd def get_tags(): @@ -24,6 +24,11 @@ def add(*files): run_cmd("git", "add", *files, capture_output=False) +def add_modified_and_deleted(): + """Stages all modified and deleted tracked files.""" + run_cmd("git", "add", "--update", capture_output=False) + + def commit(message, amend=False, no_edit=False): """Commits staged changes, optionally amending the previous commit.""" cmd = ["git", "commit"] @@ -36,9 +41,13 @@ def commit(message, amend=False, no_edit=False): run_cmd(*cmd, capture_output=False) -def push(remote, ref): +def push(remote, ref, set_upstream=False): """Pushes a reference to a remote repository.""" - run_cmd("git", "push", remote, ref, capture_output=False) + cmd = ["git", "push"] + if set_upstream: + cmd.append("-u") + cmd.extend([remote, ref]) + run_cmd(*cmd, capture_output=False) def fetch(remote="origin", tags=False, force=False): diff --git a/tools/private/release/prepare.py b/tools/private/release/prepare.py new file mode 100644 index 0000000000..1d946bda6c --- /dev/null +++ b/tools/private/release/prepare.py @@ -0,0 +1,144 @@ +import datetime +import pathlib + +from tools.private.release import changelog_news, gh, git +from tools.private.release.release_issue import update_task_in_body +from tools.private.release.utils import ( + determine_next_version, + replace_version_next, +) + + +def cmd_prepare(args): + """Executes the prepare subcommand.""" + print("Fetching upstream to verify fresh release history...") + git.fetch(tags=True, force=True) + + # Run pre-check: verify there are no local edits + status = git.status() + if status: + print( + "Error: Local edits detected. Workspace must be completely clean" + " before running release preparation." + ) + for line in status.splitlines(): + print(f" {line}") + return 1 + print("Pre-check passed: Workspace is clean.") + + version = args.version + if version is None: + version = determine_next_version() + + print(f"Running preparation pipeline for {version}...") + + # 1. Find or create tracking issue (EARLY) + # We do this before any write operations (branch creation, commit, push) + issue_num = args.issue + + if not issue_num: + try: + issue_num = gh.get_release_tracking_issue(version) + print(f"Tracking issue: #{issue_num}") + except gh.MultipleTrackingIssuesError as e: + print(f"Error: {e}") + return 1 + except gh.NoTrackingIssueError: + # Not found, we need the template + template_path = pathlib.Path( + ".github/ISSUE_TEMPLATE/release_tracking_template.md" + ) + if not template_path.exists(): + raise FileNotFoundError(f"Template file not found at {template_path}") + template_content = template_path.read_text(encoding="utf-8") + + if args.dry_run: + print( + f"[DRY RUN] No active tracking issue found for {version}. Would create a new one." + ) + print(f"[DRY RUN] Title: Release {version}\n{template_content}") + issue_num = None # Keep it None for dry-run prints later + else: + print( + f"No active tracking issue found for {version}. Creating a new one..." + ) + issue_num = gh.create_tracking_issue(version, template_content) + print(f"Tracking issue: #{issue_num}") + else: + print(f"Tracking issue: #{issue_num}") + + branch_name = f"prepare-{version}" + + # 2. Interleaved git and write operations + + # --- Branch selection/creation --- + if git.branch_exists(branch_name): + if args.dry_run: + print( + f"[DRY RUN] Branch {branch_name} already exists. Would checkout existing branch." + ) + else: + print(f"Branch {branch_name} already exists. Checking it out...") + git.checkout(branch_name) + else: + if args.dry_run: + print(f"[DRY RUN] Would create and checkout branch {branch_name}") + else: + git.checkout(branch_name, create_branch=True) + + # --- Update files --- + if args.dry_run: + print( + f"[DRY RUN] Would update CHANGELOG.md and version placeholders for {version}" + ) + else: + print("Updating changelog and placeholders...") + release_date = datetime.date.today().strftime("%Y-%m-%d") + changelog_news.update_changelog(version, release_date) + replace_version_next(version) + + # --- Commit and Push --- + if args.dry_run: + print(f"[DRY RUN] Would push branch {branch_name} to origin") + else: + modified_files = git.status() + if not modified_files: + print("No files modified by the release tool. Nothing to commit.") + return 0 + + # Stage all modified and deleted tracked files + git.add_modified_and_deleted() + + git.commit(f"Prepare release {version}") + git.push("origin", branch_name, set_upstream=True) + + # --- Create PR --- + if args.dry_run: + target_issue = f"#{issue_num}" if issue_num else "" + print( + f"[DRY RUN] Would create Pull Request for branch {branch_name} targeting issue {target_issue}" + ) + else: + pr_url = gh.create_pr(version, issue_num) + pr_num = pr_url.split("/")[-1] + print(f"Created Pull Request: {pr_url} (PR #{pr_num})") + + # --- Update checklist --- + if args.dry_run: + target_issue = f"#{issue_num}" if issue_num else "" + print( + f"[DRY RUN] Would update tracking issue {target_issue} checklist 'Prepare Release' task status to PENDING" + ) + else: + print( + f"Updating tracking issue #{issue_num} checklist 'Prepare Release' task status to PENDING..." + ) + body = gh.get_issue_body(issue_num) + metadata = {"status": "pending", "pr": f"#{pr_num}"} + updated_body = update_task_in_body( + body, "Prepare Release", checked=False, metadata=metadata + ) + gh.update_issue_body(issue_num, updated_body) + print("Preparation pipeline completed successfully!") + + return 0 diff --git a/tools/private/release/release.py b/tools/private/release/release.py index 3c2eed82ff..d9a225d5cf 100644 --- a/tools/private/release/release.py +++ b/tools/private/release/release.py @@ -2,167 +2,26 @@ import argparse import datetime -import fnmatch import os import pathlib import re import sys -from packaging.version import parse as parse_version - from tools.private.release import changelog_news, gh, git - -_REPO_URL = "https://github.com/bazel-contrib/rules_python" - -_EXCLUDE_PATTERNS = [ - "./.git/*", - "./.github/*", - "./.bazelci/*", - "./.bcr/*", - "./bazel-*/*", - "./CONTRIBUTING.md", - "./RELEASING.md", - "./tools/private/release/*", - "./tests/tools/private/release/*", -] +from tools.private.release.prepare import cmd_prepare +from tools.private.release.release_issue import ( + parse_metadata_line, + update_task_in_body, +) +from tools.private.release.utils import ( + _REPO_URL, + determine_next_version, + get_latest_rc_tag, +) _RELEASE_TITLE_RE = re.compile(r"Release (\d+\.\d+\.\d+)", re.IGNORECASE) -def _iter_version_placeholder_files(): - for root, dirs, files in os.walk(".", topdown=True): - # Filter directories - dirs[:] = [ - d - for d in dirs - if not any( - fnmatch.fnmatch(os.path.join(root, d), pattern) - for pattern in _EXCLUDE_PATTERNS - ) - ] - - for filename in files: - filepath = os.path.join(root, filename) - if any(fnmatch.fnmatch(filepath, pattern) for pattern in _EXCLUDE_PATTERNS): - continue - - yield filepath - - -def get_latest_version(): - """Gets the latest version from git tags.""" - tags = git.get_tags() - versions = [ - (tag, parse_version(tag)) - for tag in tags - if re.match(r"^\d+\.\d+\.\d+(rc\d+)?$", tag.strip()) - ] - if not versions: - raise RuntimeError("No git tags found matching X.Y.Z or X.Y.ZrcN format.") - - versions.sort(key=lambda v: v[1]) - latest_tag, latest_version = versions[-1] - - if latest_version.is_prerelease: - raise ValueError(f"The latest version is a pre-release version: {latest_tag}") - - stable_versions = [tag for tag, version in versions if not version.is_prerelease] - if not stable_versions: - raise ValueError("No stable git tags found matching X.Y.Z format.") - - return stable_versions[-1] - - -def get_latest_rc_tag(version): - """Queries git tags and returns the highest RC tag for the version.""" - tags = git.get_tags() - pattern = rf"^{re.escape(version)}-rc\d+$" - rc_tags = [tag.strip() for tag in tags if re.match(pattern, tag.strip())] - if not rc_tags: - return None - rc_tags.sort(key=parse_version) - return rc_tags[-1] - - -def should_increment_minor(): - """Checks if the minor version should be incremented.""" - for filepath in _iter_version_placeholder_files(): - try: - with open(filepath, "r") as f: - content = f.read() - except (IOError, UnicodeDecodeError): - continue - - if "VERSION_NEXT_FEATURE" in content: - return True - return False - - -def determine_next_version(branch_name=None): - """Determines the next version based on git tags and the current branch.""" - if branch_name is None: - branch_name = git.get_current_branch() - - if branch_name: - release_match = re.match(r"^release/(\d+)\.(\d+)$", branch_name) - if release_match: - branch_major = int(release_match.group(1)) - branch_minor = int(release_match.group(2)) - print( - f"Detected release branch: {branch_name} (targeting" - f" {branch_major}.{branch_minor}.x)" - ) - - tags = git.get_tags() - matching_patches = [] - for tag in tags: - tag = tag.strip() - m = re.match(rf"^{branch_major}\.{branch_minor}\.(\d+)$", tag) - if m: - matching_patches.append(int(m.group(1))) - - if matching_patches: - latest_patch = max(matching_patches) - next_version = f"{branch_major}.{branch_minor}.{latest_patch + 1}" - print( - f"Latest tag on this branch is" - f" {branch_major}.{branch_minor}.{latest_patch}. Next" - f" version: {next_version}" - ) - return next_version - else: - next_version = f"{branch_major}.{branch_minor}.0" - print( - f"No stable tags found for {branch_major}.{branch_minor}.x." - f" Next version: {next_version}" - ) - return next_version - - latest_version = get_latest_version() - major, minor, patch = [int(n) for n in latest_version.split(".")] - - if should_increment_minor(): - return f"{major}.{minor + 1}.0" - else: - return f"{major}.{minor}.{patch + 1}" - - -def replace_version_next(version): - """Replaces all VERSION_NEXT_* placeholders with the new version.""" - for filepath in _iter_version_placeholder_files(): - try: - with open(filepath, "r") as f: - content = f.read() - except (IOError, UnicodeDecodeError): - continue - - if "VERSION_NEXT_FEATURE" in content or "VERSION_NEXT_PATCH" in content: - new_content = content.replace("VERSION_NEXT_FEATURE", version) - new_content = new_content.replace("VERSION_NEXT_PATCH", version) - with open(filepath, "w") as f: - f.write(new_content) - - def _semver_type(value): if not re.match(r"^\d+\.\d+\.\d+(rc\d+)?$", value): raise argparse.ArgumentTypeError( @@ -176,65 +35,6 @@ def _semver_type(value): # ============================================================================== -def parse_metadata_line(line): - """Parses a checklist line with optional | key=value metadata.""" - match = re.match(r"^\s*-\s*\[([ xX])\]\s+([^|]+)(?:\s*\|\s*(.*))?$", line) - if not match: - return None - - checked = match.group(1).lower() == "x" - name = match.group(2).strip() - metadata_str = match.group(3) - - metadata = {} - if metadata_str: - pairs = metadata_str.strip().split() - for pair in pairs: - if "=" in pair: - k, v = pair.split("=", 1) - metadata[k] = v - - return { - "checked": checked, - "name": name, - "metadata": metadata, - "original_line": line, - } - - -def format_metadata_line(checked, name, metadata): - """Formats a checklist line with space-separated key=value metadata.""" - check_str = "x" if checked else " " - if not metadata: - return f"- [{check_str}] {name}" - - metadata_str = " ".join(f"{k}={v}" for k, v in metadata.items()) - return f"- [{check_str}] {name} | {metadata_str}" - - -def update_task_in_body(body, task_name, checked, metadata): - """Updates a specific task's checked state and metadata in the issue body.""" - lines = body.splitlines() - updated_lines = [] - found = False - - for line in lines: - parsed = parse_metadata_line(line) - if parsed and parsed["name"].lower() == task_name.lower(): - updated_lines.append(format_metadata_line(checked, task_name, metadata)) - found = True - else: - updated_lines.append(line) - - if not found: - raise ValueError( - f"Task '{task_name}' not found in issue body. " - f"Expected format: '- [ ] {task_name}' or '- [x] {task_name}' (optionally followed by '| key=value')" - ) - - return "\n".join(updated_lines) - - def parse_checklist_state(body): """Parses the main checklist tasks and their metadata.""" state = { @@ -366,91 +166,6 @@ def cmd_create_release_issue(args): return 0 -def cmd_prepare(args): - """Executes the prepare subcommand.""" - print("Fetching upstream to verify fresh release history...") - git.fetch(tags=True, force=True) - - # Run pre-check: verify there are no local edits - status = git.status() - if status: - print( - "Error: Local edits detected. Workspace must be completely clean" - " before running release preparation." - ) - for line in status.splitlines(): - print(f" {line}") - return 1 - print("Pre-check passed: Workspace is clean.") - - version = args.version - if version is None: - version = determine_next_version() - - print(f"Running preparation pipeline for v{version}...") - - branch_name = f"prepare-{version}" - if git.branch_exists(branch_name): - print(f"Branch {branch_name} already exists. Checking it out...") - git.checkout(branch_name) - else: - git.checkout(branch_name, create_branch=True) - - print("Updating changelog and placeholders...") - release_date = datetime.date.today().strftime("%Y-%m-%d") - changelog_news.update_changelog(version, release_date) - replace_version_next(version) - - modified_files = git.status() - if not modified_files: - print("No files modified by the release tool. Nothing to commit.") - return 0 - - # Stage only modified files - for line in modified_files.splitlines(): - file_path = line.strip().split()[-1] - git.add(file_path) - - git.commit(f"Prepare release {version}") - git.push("origin", branch_name) - - issue_num = args.issue - if not issue_num: - try: - issue_num = gh.get_release_tracking_issue(version) - print(f"Found active tracking issue #{issue_num} for v{version}") - except ValueError as e: - if "Multiple open tracking issues" in str(e): - print(f"Error: {e}") - return 1 - print( - f"No active tracking issue found for v{version}. Creating a new one..." - ) - template_path = pathlib.Path( - ".github/ISSUE_TEMPLATE/release_tracking_template.md" - ) - if not template_path.exists(): - raise FileNotFoundError(f"Template file not found at {template_path}") - template_content = template_path.read_text(encoding="utf-8") - issue_num = gh.create_tracking_issue(version, template_content) - - print(f"Using tracking issue #{issue_num}") - - pr_url = gh.create_pr(version, branch_name, issue_num) - pr_num = pr_url.split("/")[-1] - print(f"Created Pull Request: {pr_url} (PR #{pr_num})") - - print(f"Updating tracking issue #{issue_num} checklist status to PENDING...") - body = gh.get_issue_body(issue_num) - metadata = {"status": "pending", "pr": f"#{pr_num}"} - updated_body = update_task_in_body( - body, "Prepare Release", checked=False, metadata=metadata - ) - gh.update_issue_body(issue_num, updated_body) - print("Preparation pipeline completed successfully!") - return 0 - - def cmd_complete_prepare(args): """Executes the complete-prepare subcommand (Phase 2 PR merged).""" print(f"Completing preparation for PR #{args.pr}...") @@ -896,6 +611,12 @@ def create_parser(): type=int, help="The tracking issue number (optional, triggers automated branch/PR pipeline).", ) + prepare_parser.add_argument( + "--dry-run", + action=argparse.BooleanOptionalAction, + default=True, + help="Perform a dry run (default: True). Use --no-dry-run to actually execute.", + ) # Subcommand: complete-prepare complete_prep_parser = subparsers.add_parser( @@ -998,6 +719,9 @@ def main(): exit_code = cmd_promote_rc(args) except Exception as e: print(f"Fatal error executing {args.command}: {e}", file=sys.stderr) + if hasattr(e, "__notes__"): + for note in e.__notes__: + print(note, file=sys.stderr) sys.exit(1) sys.exit(exit_code if exit_code is not None else 0) diff --git a/tools/private/release/release_issue.py b/tools/private/release/release_issue.py new file mode 100644 index 0000000000..27bcf3d774 --- /dev/null +++ b/tools/private/release/release_issue.py @@ -0,0 +1,62 @@ +"""Helper functions for managing release tracking issues and checklists.""" + +import re + + +def parse_metadata_line(line): + """Parses a checklist line with optional | key=value metadata.""" + match = re.match(r"^\s*-\s*\[([ xX])\]\s+([^|]+)(?:\s*\|\s*(.*))?$", line) + if not match: + return None + + checked = match.group(1).lower() == "x" + name = match.group(2).strip() + metadata_str = match.group(3) + + metadata = {} + if metadata_str: + pairs = metadata_str.strip().split() + for pair in pairs: + if "=" in pair: + k, v = pair.split("=", 1) + metadata[k] = v + + return { + "checked": checked, + "name": name, + "metadata": metadata, + "original_line": line, + } + + +def format_metadata_line(checked, name, metadata): + """Formats a checklist line with space-separated key=value metadata.""" + check_str = "x" if checked else " " + if not metadata: + return f"- [{check_str}] {name}" + + metadata_str = " ".join(f"{k}={v}" for k, v in metadata.items()) + return f"- [{check_str}] {name} | {metadata_str}" + + +def update_task_in_body(body, task_name, checked, metadata): + """Updates a specific task's checked state and metadata in the issue body.""" + lines = body.splitlines() + updated_lines = [] + found = False + + for line in lines: + parsed = parse_metadata_line(line) + if parsed and parsed["name"].lower() == task_name.lower(): + updated_lines.append(format_metadata_line(checked, task_name, metadata)) + found = True + else: + updated_lines.append(line) + + if not found: + raise ValueError( + f"Task '{task_name}' not found in issue body. " + f"Expected format: '- [ ] {task_name}' or '- [x] {task_name}' (optionally followed by '| key=value')" + ) + + return "\n".join(updated_lines) diff --git a/tools/private/release/shell.py b/tools/private/release/shell.py new file mode 100644 index 0000000000..cfff53f4e4 --- /dev/null +++ b/tools/private/release/shell.py @@ -0,0 +1,29 @@ +"""Shell utility functions for the release tool.""" + +import shlex +import subprocess + + +def run_cmd(*args, check=True, capture_output=True): + """Runs a command as a subprocess with separate arguments (prints command). + + If the command fails, it raises the CalledProcessError after attaching + a detailed note explaining the failure to preserve the stack trace. + """ + cmd = [str(arg) for arg in args] + print(f"Running: {shlex.join(cmd)}") + try: + result = subprocess.run( + cmd, + check=check, + stdout=subprocess.PIPE if capture_output else None, + stderr=subprocess.PIPE if capture_output else None, + universal_newlines=True, + ) + return result.stdout.strip() if capture_output else None + except subprocess.CalledProcessError as e: + note = f"Error running command: {shlex.join(cmd)}" + if capture_output: + note += f"\nStdout: {e.stdout}\nStderr: {e.stderr}" + e.add_note(note) + raise diff --git a/tools/private/release/utils.py b/tools/private/release/utils.py index 5eb7c43a71..83ecbb9c7a 100644 --- a/tools/private/release/utils.py +++ b/tools/private/release/utils.py @@ -1,29 +1,157 @@ """Utility functions for the release tool.""" -import shlex -import subprocess - - -def run_cmd(*args, check=True, capture_output=True): - """Runs a command as a subprocess with separate arguments (prints command). - - If the command fails, it raises the CalledProcessError after attaching - a detailed note explaining the failure to preserve the stack trace. - """ - cmd = [str(arg) for arg in args] - print(f"Running: {shlex.join(cmd)}") - try: - result = subprocess.run( - cmd, - check=check, - stdout=subprocess.PIPE if capture_output else None, - stderr=subprocess.PIPE if capture_output else None, - universal_newlines=True, - ) - return result.stdout.strip() if capture_output else None - except subprocess.CalledProcessError as e: - note = f"Error running command: {shlex.join(cmd)}" - if capture_output: - note += f"\nStdout: {e.stdout}\nStderr: {e.stderr}" - e.add_note(note) - raise +import fnmatch +import os +import re + +from packaging.version import parse as parse_version + +from tools.private.release import git + +_REPO_URL = "https://github.com/bazel-contrib/rules_python" + +_EXCLUDE_PATTERNS = [ + "./.git/*", + "./.github/*", + "./.bazelci/*", + "./.bcr/*", + "./bazel-*/*", + "./CONTRIBUTING.md", + "./RELEASING.md", + "./tools/private/release/*", + "./tests/tools/private/release/*", +] + + +def _iter_version_placeholder_files(): + for root, dirs, files in os.walk(".", topdown=True): + # Filter directories + dirs[:] = [ + d + for d in dirs + if not any( + fnmatch.fnmatch(os.path.join(root, d), pattern) + for pattern in _EXCLUDE_PATTERNS + ) + ] + + for filename in files: + filepath = os.path.join(root, filename) + if any(fnmatch.fnmatch(filepath, pattern) for pattern in _EXCLUDE_PATTERNS): + continue + + yield filepath + + +def get_latest_version(): + """Gets the latest version from git tags.""" + tags = git.get_tags() + versions = [ + (tag, parse_version(tag)) + for tag in tags + if re.match(r"^\d+\.\d+\.\d+(rc\d+)?$", tag.strip()) + ] + if not versions: + raise RuntimeError("No git tags found matching X.Y.Z or X.Y.ZrcN format.") + + versions.sort(key=lambda v: v[1]) + latest_tag, latest_version = versions[-1] + + if latest_version.is_prerelease: + raise ValueError(f"The latest version is a pre-release version: {latest_tag}") + + stable_versions = [tag for tag, version in versions if not version.is_prerelease] + if not stable_versions: + raise ValueError("No stable git tags found matching X.Y.Z format.") + + return stable_versions[-1] + + +def get_latest_rc_tag(version): + """Queries git tags and returns the highest RC tag for the version.""" + tags = git.get_tags() + pattern = rf"^{re.escape(version)}-rc\d+$" + rc_tags = [tag.strip() for tag in tags if re.match(pattern, tag.strip())] + if not rc_tags: + return None + rc_tags.sort(key=parse_version) + return rc_tags[-1] + + +def should_increment_minor(): + """Checks if the minor version should be incremented.""" + for filepath in _iter_version_placeholder_files(): + try: + with open(filepath, "r") as f: + content = f.read() + except (IOError, UnicodeDecodeError): + continue + + if "VERSION_NEXT_FEATURE" in content: + return True + return False + + +def determine_next_version(branch_name=None): + """Determines the next version based on git tags and the current branch.""" + if branch_name is None: + branch_name = git.get_current_branch() + + if branch_name: + release_match = re.match(r"^release/(\d+)\.(\d+)$", branch_name) + if release_match: + branch_major = int(release_match.group(1)) + branch_minor = int(release_match.group(2)) + print( + f"Detected release branch: {branch_name} (targeting" + f" {branch_major}.{branch_minor}.x)" + ) + + tags = git.get_tags() + matching_patches = [] + for tag in tags: + tag = tag.strip() + m = re.match(rf"^{branch_major}\.{branch_minor}\.(\d+)$", tag) + if m: + matching_patches.append(int(m.group(1))) + + if matching_patches: + latest_patch = max(matching_patches) + next_version = f"{branch_major}.{branch_minor}.{latest_patch + 1}" + print( + f"Latest tag on this branch is" + f" {branch_major}.{branch_minor}.{latest_patch}. Next" + f" version: {next_version}" + ) + return next_version + else: + next_version = f"{branch_major}.{branch_minor}.0" + print( + f"No stable tags found for {branch_major}.{branch_minor}.x." + f" Next version: {next_version}" + ) + return next_version + + latest_version = get_latest_version() + major, minor, patch = [int(n) for n in latest_version.split(".")] + + if should_increment_minor(): + return f"{major}.{minor + 1}.0" + else: + return f"{major}.{minor}.{patch + 1}" + + +def replace_version_next(version): + """Replaces all VERSION_NEXT_* placeholders with the new version.""" + for filepath in _iter_version_placeholder_files(): + try: + with open(filepath, "r") as f: + content = f.read() + except (IOError, UnicodeDecodeError): + continue + + if "VERSION_NEXT_FEATURE" in content or "VERSION_NEXT_PATCH" in content: + new_content = content.replace("VERSION_NEXT_FEATURE", version) + new_content = new_content.replace("VERSION_NEXT_PATCH", version) + with open(filepath, "w") as f: + f.write(new_content) From 8a825aae648d3cfe0ef32c0f822d00e9e8f3d5c8 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Mon, 29 Jun 2026 19:16:14 -0700 Subject: [PATCH 791/922] chore(release): reuse existing PRs and avoid early exit in prep tool (#3870) Fix the release preparation tool to prevent it from exiting early when no local changes are present, ensuring the pipeline can always proceed. The tool now detects and reuses existing open branch PRs or PRs already associated with the tracking issue, avoiding the creation of duplicate PRs. --- tests/tools/private/release/release_test.py | 95 +++++++++++++++++++++ tools/private/release/gh.py | 16 ++++ tools/private/release/prepare.py | 50 ++++++++--- tools/private/release/release.py | 67 +-------------- tools/private/release/release_issue.py | 66 ++++++++++++++ 5 files changed, 215 insertions(+), 79 deletions(-) diff --git a/tests/tools/private/release/release_test.py b/tests/tools/private/release/release_test.py index 534aa2a5fd..a1467d8987 100644 --- a/tests/tools/private/release/release_test.py +++ b/tests/tools/private/release/release_test.py @@ -35,6 +35,7 @@ def _mock_git_and_gh(test_case): mock_git.branch_exists.return_value = False mock_git.tag_exists.return_value = False mock_gh.get_release_tracking_issue.side_effect = NoTrackingIssueError("Not found") + mock_gh.get_open_pr.return_value = None class TempDirTestCase(unittest.TestCase): @@ -799,6 +800,100 @@ def test_prepare_dry_run(self, mock_replace, mock_changelog): self.mock_gh.get_release_tracking_issue.assert_called_once_with("2.0.0") self.mock_git.add_modified_and_deleted.assert_not_called() + @patch("tools.private.release.prepare.changelog_news") + @patch("tools.private.release.prepare.replace_version_next") + def test_prepare_use_associated_pr_from_tracking_issue( + self, mock_replace, mock_changelog + ): + # Arrange + args = MagicMock(version="2.0.0", issue=None, dry_run=False) + self.mock_git.status.side_effect = ["", ""] + self.mock_git.branch_exists.return_value = True + self.mock_gh.get_release_tracking_issue.side_effect = None + self.mock_gh.get_release_tracking_issue.return_value = 123 + self.mock_gh.get_open_pr.return_value = None + # PR #456 is already associated in the tracking issue + self.mock_gh.get_issue_body.return_value = ( + "- [ ] Prepare Release | status=pending pr=#456" + ) + + # Act + result = releaser.cmd_prepare(args) + + # Assert + self.assertEqual(result, 0) + self.mock_git.checkout.assert_called_once_with("prepare-2.0.0") + self.mock_git.commit.assert_not_called() + self.mock_git.push.assert_called_once_with( + "origin", "prepare-2.0.0", set_upstream=True + ) + self.mock_gh.get_open_pr.assert_called_once_with("prepare-2.0.0") + self.mock_gh.create_pr.assert_not_called() # Should NOT create a new PR + self.mock_gh.update_issue_body.assert_called_once() + call_args = self.mock_gh.update_issue_body.call_args[0] + self.assertIn("pr=#456", call_args[1]) + + @patch("tools.private.release.prepare.changelog_news") + @patch("tools.private.release.prepare.replace_version_next") + def test_prepare_create_pr_when_none_associated(self, mock_replace, mock_changelog): + # Arrange + args = MagicMock(version="2.0.0", issue=None, dry_run=False) + self.mock_git.status.side_effect = ["", ""] + self.mock_git.branch_exists.return_value = True + self.mock_gh.get_release_tracking_issue.side_effect = None + self.mock_gh.get_release_tracking_issue.return_value = 123 + self.mock_gh.get_open_pr.return_value = None + # No PR associated in the tracking issue + self.mock_gh.get_issue_body.return_value = "- [ ] Prepare Release" + self.mock_gh.create_pr.return_value = "https://github.com/foo/bar/pull/789" + + # Act + result = releaser.cmd_prepare(args) + + # Assert + self.assertEqual(result, 0) + self.mock_git.checkout.assert_called_once_with("prepare-2.0.0") + self.mock_git.commit.assert_not_called() + self.mock_git.push.assert_called_once_with( + "origin", "prepare-2.0.0", set_upstream=True + ) + self.mock_gh.get_open_pr.assert_called_once_with("prepare-2.0.0") + self.mock_gh.create_pr.assert_called_once_with("2.0.0", 123) + self.mock_gh.update_issue_body.assert_called_once() + call_args = self.mock_gh.update_issue_body.call_args[0] + self.assertIn("pr=#789", call_args[1]) + + @patch("tools.private.release.prepare.changelog_news") + @patch("tools.private.release.prepare.replace_version_next") + def test_prepare_reuse_existing_pr(self, mock_replace, mock_changelog): + # Arrange + args = MagicMock(version="2.0.0", issue=None, dry_run=False) + self.mock_git.status.side_effect = ["", ""] + self.mock_git.branch_exists.return_value = True + self.mock_gh.get_release_tracking_issue.side_effect = None + self.mock_gh.get_release_tracking_issue.return_value = 123 + self.mock_gh.get_open_pr.return_value = { + "number": 456, + "url": "https://github.com/foo/bar/pull/456", + } + self.mock_gh.get_issue_body.return_value = "- [ ] Prepare Release" + + # Act + result = releaser.cmd_prepare(args) + + # Assert + self.assertEqual(result, 0) + self.mock_git.checkout.assert_called_once_with("prepare-2.0.0") + self.mock_git.commit.assert_not_called() + self.mock_git.push.assert_called_once_with( + "origin", "prepare-2.0.0", set_upstream=True + ) + self.mock_gh.get_open_pr.assert_called_once_with("prepare-2.0.0") + self.mock_gh.create_pr.assert_not_called() + self.mock_gh.update_issue_body.assert_called_once() + call_args = self.mock_gh.update_issue_body.call_args[0] + self.assertIn("pr=#456", call_args[1]) + @patch("tools.private.release.prepare.changelog_news") @patch("tools.private.release.prepare.replace_version_next") def test_prepare_dry_run_no_issue(self, mock_replace, mock_changelog): diff --git a/tools/private/release/gh.py b/tools/private/release/gh.py index f37d87f396..da86415650 100644 --- a/tools/private/release/gh.py +++ b/tools/private/release/gh.py @@ -162,6 +162,22 @@ def create_pr(version, issue_num): ) +def get_open_pr(branch_name): + """Returns PR info if an open PR exists for the given branch, else None.""" + cmd = [ + "gh", + "pr", + "list", + f"--repo={_REPO}", + f"--head={branch_name}", + "--state=open", + "--json=number,url", + ] + output = run_cmd(*cmd) + prs = json.loads(output) if output else [] + return prs[0] if prs else None + + def get_pr_info(pr_num): """Gets information about a PR, including state, merge commit, and body.""" output = run_cmd( diff --git a/tools/private/release/prepare.py b/tools/private/release/prepare.py index 1d946bda6c..c9012e0d04 100644 --- a/tools/private/release/prepare.py +++ b/tools/private/release/prepare.py @@ -2,7 +2,10 @@ import pathlib from tools.private.release import changelog_news, gh, git -from tools.private.release.release_issue import update_task_in_body +from tools.private.release.release_issue import ( + parse_checklist_state, + update_task_in_body, +) from tools.private.release.utils import ( determine_next_version, replace_version_next, @@ -102,26 +105,47 @@ def cmd_prepare(args): print(f"[DRY RUN] Would push branch {branch_name} to origin") else: modified_files = git.status() - if not modified_files: + if modified_files: + # Stage all modified and deleted tracked files + git.add_modified_and_deleted() + git.commit(f"Prepare release {version}") + else: print("No files modified by the release tool. Nothing to commit.") - return 0 - - # Stage all modified and deleted tracked files - git.add_modified_and_deleted() - git.commit(f"Prepare release {version}") + print(f"Pushing branch {branch_name} to origin...") git.push("origin", branch_name, set_upstream=True) # --- Create PR --- - if args.dry_run: - target_issue = f"#{issue_num}" if issue_num else "" + # Determine if we need to create a PR or reuse an existing one + open_pr = gh.get_open_pr(branch_name) + associated_pr = None + + if not open_pr and issue_num: + body = gh.get_issue_body(issue_num) + state = parse_checklist_state(body) + associated_pr = state["prepare_release"]["pr"] + + if open_pr: + pr_num = open_pr["number"] + pr_url = open_pr["url"] + print(f"Open Pull Request already exists: {pr_url} (PR #{pr_num})") + elif associated_pr: + pr_num = associated_pr.lstrip("#") + pr_url = f"https://github.com/bazel-contrib/rules_python/pull/{pr_num}" print( - f"[DRY RUN] Would create Pull Request for branch {branch_name} targeting issue {target_issue}" + f"PR #{pr_num} is already associated in tracking issue #{issue_num}. Using it." ) else: - pr_url = gh.create_pr(version, issue_num) - pr_num = pr_url.split("/")[-1] - print(f"Created Pull Request: {pr_url} (PR #{pr_num})") + if args.dry_run: + target_issue = f"#{issue_num}" if issue_num else "" + print( + f"[DRY RUN] Would create Pull Request for branch {branch_name} targeting issue {target_issue}" + ) + pr_num = "" + else: + pr_url = gh.create_pr(version, issue_num) + pr_num = pr_url.split("/")[-1] + print(f"Created Pull Request: {pr_url} (PR #{pr_num})") # --- Update checklist --- if args.dry_run: diff --git a/tools/private/release/release.py b/tools/private/release/release.py index d9a225d5cf..dab99e6d05 100644 --- a/tools/private/release/release.py +++ b/tools/private/release/release.py @@ -10,6 +10,7 @@ from tools.private.release import changelog_news, gh, git from tools.private.release.prepare import cmd_prepare from tools.private.release.release_issue import ( + parse_checklist_state, parse_metadata_line, update_task_in_body, ) @@ -35,72 +36,6 @@ def _semver_type(value): # ============================================================================== -def parse_checklist_state(body): - """Parses the main checklist tasks and their metadata.""" - state = { - "prepare_release": { - "checked": False, - "status": None, - "pr": None, - "commit": None, - }, - "create_branch": { - "checked": False, - "status": None, - "branch": None, - "commit": None, - }, - "tag_final": {"checked": False, "status": None, "tag": None, "commit": None}, - "rc_tags": {}, # Dynamically mapped: int -> metadata dict - } - - lines = body.splitlines() - for line in lines: - parsed = parse_metadata_line(line) - if not parsed: - continue - - name = parsed["name"].strip() - meta = parsed["metadata"] - checked = parsed["checked"] - name_lower = name.lower() - - if "prepare release" in name_lower: - state["prepare_release"] = { - "checked": checked, - "status": meta.get("status"), - "pr": meta.get("pr"), - "commit": meta.get("commit"), - } - elif "create release branch" in name_lower: - state["create_branch"] = { - "checked": checked, - "status": meta.get("status"), - "branch": meta.get("branch"), - "commit": meta.get("commit"), - } - elif "tag final" in name_lower: - state["tag_final"] = { - "checked": checked, - "status": meta.get("status"), - "tag": meta.get("tag"), - "commit": meta.get("commit"), - } - else: - # Match Tag RC - rc_match = re.match(r"Tag RC(\d+)", name, re.IGNORECASE) - if rc_match: - rc_num = int(rc_match.group(1)) - state["rc_tags"][rc_num] = { - "checked": checked, - "status": meta.get("status"), - "tag": meta.get("tag"), - "commit": meta.get("commit"), - } - - return state - - def parse_backports(body): """Parses the ## Backports checklist section.""" body = body.replace("\r\n", "\n") diff --git a/tools/private/release/release_issue.py b/tools/private/release/release_issue.py index 27bcf3d774..f96b917ab9 100644 --- a/tools/private/release/release_issue.py +++ b/tools/private/release/release_issue.py @@ -60,3 +60,69 @@ def update_task_in_body(body, task_name, checked, metadata): ) return "\n".join(updated_lines) + + +def parse_checklist_state(body): + """Parses the main checklist tasks and their metadata.""" + state = { + "prepare_release": { + "checked": False, + "status": None, + "pr": None, + "commit": None, + }, + "create_branch": { + "checked": False, + "status": None, + "branch": None, + "commit": None, + }, + "tag_final": {"checked": False, "status": None, "tag": None, "commit": None}, + "rc_tags": {}, # Dynamically mapped: int -> metadata dict + } + + lines = body.splitlines() + for line in lines: + parsed = parse_metadata_line(line) + if not parsed: + continue + + name = parsed["name"].strip() + meta = parsed["metadata"] + checked = parsed["checked"] + name_lower = name.lower() + + if "prepare release" in name_lower: + state["prepare_release"] = { + "checked": checked, + "status": meta.get("status"), + "pr": meta.get("pr"), + "commit": meta.get("commit"), + } + elif "create release branch" in name_lower: + state["create_branch"] = { + "checked": checked, + "status": meta.get("status"), + "branch": meta.get("branch"), + "commit": meta.get("commit"), + } + elif "tag final" in name_lower: + state["tag_final"] = { + "checked": checked, + "status": meta.get("status"), + "tag": meta.get("tag"), + "commit": meta.get("commit"), + } + else: + # Match Tag RC + rc_match = re.match(r"Tag RC(\d+)", name, re.IGNORECASE) + if rc_match: + rc_num = int(rc_match.group(1)) + state["rc_tags"][rc_num] = { + "checked": checked, + "status": meta.get("status"), + "tag": meta.get("tag"), + "commit": meta.get("commit"), + } + + return state From 9af23e51c614f82709dbab55120030cd3d66ac08 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Mon, 29 Jun 2026 20:53:55 -0700 Subject: [PATCH 792/922] chore(release): use force push and clean up git helpers in prep tool (#3873) This change fixes an edge case in the release preparation tool by using force push when pushing the preparation branch, resolving failures when the remote branch already exists. It also cleans up git helpers to use long option names and fail-fast on errors, with corresponding test updates. --- tests/tools/private/release/release_test.py | 10 +++++++--- tools/private/release/git.py | 13 ++++++------- tools/private/release/prepare.py | 3 ++- 3 files changed, 15 insertions(+), 11 deletions(-) diff --git a/tests/tools/private/release/release_test.py b/tests/tools/private/release/release_test.py index a1467d8987..293b82b277 100644 --- a/tests/tools/private/release/release_test.py +++ b/tests/tools/private/release/release_test.py @@ -591,6 +591,10 @@ def setUp(self): self.mock_get_latest_version = patch( "tools.private.release.utils.get_latest_version" ).start() + self.mock_get_current_branch = patch( + "tools.private.release.git.get_current_branch" + ).start() + self.mock_get_current_branch.return_value = "main" self.addCleanup(patch.stopall) def test_no_markers(self): @@ -825,7 +829,7 @@ def test_prepare_use_associated_pr_from_tracking_issue( self.mock_git.checkout.assert_called_once_with("prepare-2.0.0") self.mock_git.commit.assert_not_called() self.mock_git.push.assert_called_once_with( - "origin", "prepare-2.0.0", set_upstream=True + "origin", "prepare-2.0.0", set_upstream=True, force=True ) self.mock_gh.get_open_pr.assert_called_once_with("prepare-2.0.0") self.mock_gh.create_pr.assert_not_called() # Should NOT create a new PR @@ -855,7 +859,7 @@ def test_prepare_create_pr_when_none_associated(self, mock_replace, mock_changel self.mock_git.checkout.assert_called_once_with("prepare-2.0.0") self.mock_git.commit.assert_not_called() self.mock_git.push.assert_called_once_with( - "origin", "prepare-2.0.0", set_upstream=True + "origin", "prepare-2.0.0", set_upstream=True, force=True ) self.mock_gh.get_open_pr.assert_called_once_with("prepare-2.0.0") self.mock_gh.create_pr.assert_called_once_with("2.0.0", 123) @@ -886,7 +890,7 @@ def test_prepare_reuse_existing_pr(self, mock_replace, mock_changelog): self.mock_git.checkout.assert_called_once_with("prepare-2.0.0") self.mock_git.commit.assert_not_called() self.mock_git.push.assert_called_once_with( - "origin", "prepare-2.0.0", set_upstream=True + "origin", "prepare-2.0.0", set_upstream=True, force=True ) self.mock_gh.get_open_pr.assert_called_once_with("prepare-2.0.0") self.mock_gh.create_pr.assert_not_called() diff --git a/tools/private/release/git.py b/tools/private/release/git.py index 9c2662d4e6..ed6e637d3f 100644 --- a/tools/private/release/git.py +++ b/tools/private/release/git.py @@ -41,11 +41,13 @@ def commit(message, amend=False, no_edit=False): run_cmd(*cmd, capture_output=False) -def push(remote, ref, set_upstream=False): +def push(remote, ref, set_upstream=False, force=False): """Pushes a reference to a remote repository.""" cmd = ["git", "push"] if set_upstream: - cmd.append("-u") + cmd.append("--set-upstream") + if force: + cmd.append("--force") cmd.extend([remote, ref]) run_cmd(*cmd, capture_output=False) @@ -128,8 +130,5 @@ def get_tags_at_head(): def get_current_branch(): - """Returns the current git branch name, or None if not in a git repo.""" - try: - return run_cmd("git", "rev-parse", "--abbrev-ref", "HEAD") - except subprocess.CalledProcessError: - return None + """Returns the current git branch name.""" + return run_cmd("git", "rev-parse", "--abbrev-ref", "HEAD") diff --git a/tools/private/release/prepare.py b/tools/private/release/prepare.py index c9012e0d04..2d32ee4de0 100644 --- a/tools/private/release/prepare.py +++ b/tools/private/release/prepare.py @@ -113,7 +113,8 @@ def cmd_prepare(args): print("No files modified by the release tool. Nothing to commit.") print(f"Pushing branch {branch_name} to origin...") - git.push("origin", branch_name, set_upstream=True) + # Force push to overwrite the remote branch if it already exists (e.g. from a previous run) + git.push("origin", branch_name, set_upstream=True, force=True) # --- Create PR --- # Determine if we need to create a PR or reuse an existing one From ee0acbdcc9a010e6c3fe29df9474b5823821afd7 Mon Sep 17 00:00:00 2001 From: "github-actions[bot]" <41898282+github-actions[bot]@users.noreply.github.com> Date: Tue, 30 Jun 2026 04:05:05 +0000 Subject: [PATCH 793/922] Prepare release v2.2.0 (#3874) Work towards #3867 Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> --- CHANGELOG.md | 39 +++++++++++++++++++ .../python/config_settings/index.md | 2 +- docs/environment-variables.md | 2 +- docs/pypi/download.md | 2 +- news/2945.changed.md | 1 - news/3785.added.md | 5 --- news/3828.fixed.md | 2 - news/3832.fixed.md | 1 - news/3837.added.md | 1 - news/expose-venv-symlink.added.md | 2 - news/gazelle-bzl-library.changed.md | 3 -- news/loadable-symbols.added.md | 2 - news/slash-target-venv-output.fixed.md | 3 -- news/win32_version_lookup.fixed.md | 3 -- python/features.bzl | 2 +- python/private/py_info.bzl | 4 +- python/private/pypi/extension.bzl | 2 +- python/uv/private/lock.bzl | 2 +- 18 files changed, 47 insertions(+), 31 deletions(-) delete mode 100644 news/2945.changed.md delete mode 100644 news/3785.added.md delete mode 100644 news/3828.fixed.md delete mode 100644 news/3832.fixed.md delete mode 100644 news/3837.added.md delete mode 100644 news/expose-venv-symlink.added.md delete mode 100644 news/gazelle-bzl-library.changed.md delete mode 100644 news/loadable-symbols.added.md delete mode 100644 news/slash-target-venv-output.fixed.md delete mode 100644 news/win32_version_lookup.fixed.md diff --git a/CHANGELOG.md b/CHANGELOG.md index 21108ffda2..bcb6c6ddee 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -29,6 +29,45 @@ Unreleased changes are tracked as individual files in the [news/](./news) directory, or view the [latest generated changelog](https://rules-python.readthedocs.io/en/latest/changelog.html). +{#v2-2-0} +## [2.2.0] - 2026-06-30 + +[2.2.0]: https://github.com/bazel-contrib/rules_python/releases/tag/2.2.0 + +{#v2-2-0-changed} +### Changed +* Renamed most public bzl_library targets from `{foo}_bzl` to `{foo}` to follow +gazelle naming conventions. Deprecated aliases are left for backwards +compatibility. +* (binaries/tests) Added a deprecation warning for targets relying on implicit `__init__.py` creation. + +{#v2-2-0-fixed} +### Fixed +* Fixed a flaky error on Windows 2022 when looking up the win32 version during +site initialization by retrying the lookup +([#3721](https://github.com/bazel-contrib/rules_python/issues/3721)). +* (coverage) Skip lcov report when no data was collected. +* (pypi) Fixed `experimental_index_url` checking truthiness before envsubst +expansion. +* (rules) Fixed venv output paths for `py_binary` and `py_test` targets whose +names contain path separators so distinct targets with the same basename no +longer share the same venv output directory. + +{#v2-2-0-added} +### Added +* Added {bzl:obj}`features.loadable_symbols` to allow detecting public symbols +exported by bzl files. +* Exposed {bzl:obj}`VenvSymlinkEntry` and {bzl:obj}`VenvSymlinkKind` in +{bzl:target}`//python:py_info.bzl`. +* (pypi) Added `@pypi` repo: a unified hub of `pip.parse` hubs. +* (uv) Support for basic `uv.lock` generation via the `lock` rule +and basic support for importing the `uv.lock` file itself. Since this +may have bugs, please report this by creating new tickets. +Work towards [#2787](https://github.com/bazel-contrib/rules_python/issues/2787) +and [#1975](https://github.com/bazel-contrib/rules_python/issues/1975). + + + {#v2-1-0} ## [2.1.0] - 2026-06-17 diff --git a/docs/api/rules_python/python/config_settings/index.md b/docs/api/rules_python/python/config_settings/index.md index 7908a28190..6b7a150518 100644 --- a/docs/api/rules_python/python/config_settings/index.md +++ b/docs/api/rules_python/python/config_settings/index.md @@ -362,7 +362,7 @@ Values: specified concrete PyPI hub (corresponding to a {obj}`pip.parse.hub_name` value). -:::{versionadded} VERSION_NEXT_FEATURE +:::{versionadded} 2.2.0 ::: :::: diff --git a/docs/environment-variables.md b/docs/environment-variables.md index 983ae3cb5f..3eb62221c0 100644 --- a/docs/environment-variables.md +++ b/docs/environment-variables.md @@ -126,7 +126,7 @@ a warning is printed indicating that the renaming occurred. If not set (defaulti to `0`), a warning is printed advising to rename the hub, and the collision is not resolved. -:::{versionadded} VERSION_NEXT_FEATURE +:::{versionadded} 2.2.0 ::: ::: diff --git a/docs/pypi/download.md b/docs/pypi/download.md index 6705df1f3a..161a753645 100644 --- a/docs/pypi/download.md +++ b/docs/pypi/download.md @@ -53,7 +53,7 @@ downloading the same wheels numerous times. (unified-pypi-hub)= ## Unified `@pypi` Hub for Multi-Hub Configurations -:::{versionadded} VERSION_NEXT_FEATURE +:::{versionadded} 2.2.0 Unified `@pypi` hub repository for Bzlmod multi-hub configurations. ::: diff --git a/news/2945.changed.md b/news/2945.changed.md deleted file mode 100644 index c0ff002661..0000000000 --- a/news/2945.changed.md +++ /dev/null @@ -1 +0,0 @@ -(binaries/tests) Added a deprecation warning for targets relying on implicit `__init__.py` creation. diff --git a/news/3785.added.md b/news/3785.added.md deleted file mode 100644 index db0ec5602d..0000000000 --- a/news/3785.added.md +++ /dev/null @@ -1,5 +0,0 @@ -(uv) Support for basic `uv.lock` generation via the `lock` rule -and basic support for importing the `uv.lock` file itself. Since this -may have bugs, please report this by creating new tickets. -Work towards [#2787](https://github.com/bazel-contrib/rules_python/issues/2787) -and [#1975](https://github.com/bazel-contrib/rules_python/issues/1975). diff --git a/news/3828.fixed.md b/news/3828.fixed.md deleted file mode 100644 index 64de2cc925..0000000000 --- a/news/3828.fixed.md +++ /dev/null @@ -1,2 +0,0 @@ -(pypi) Fixed `experimental_index_url` checking truthiness before envsubst -expansion. diff --git a/news/3832.fixed.md b/news/3832.fixed.md deleted file mode 100644 index f1dd32df06..0000000000 --- a/news/3832.fixed.md +++ /dev/null @@ -1 +0,0 @@ -(coverage) Skip lcov report when no data was collected. diff --git a/news/3837.added.md b/news/3837.added.md deleted file mode 100644 index 6d3e4b5504..0000000000 --- a/news/3837.added.md +++ /dev/null @@ -1 +0,0 @@ -(pypi) Added `@pypi` repo: a unified hub of `pip.parse` hubs. diff --git a/news/expose-venv-symlink.added.md b/news/expose-venv-symlink.added.md deleted file mode 100644 index e339bb5d3f..0000000000 --- a/news/expose-venv-symlink.added.md +++ /dev/null @@ -1,2 +0,0 @@ -Exposed {bzl:obj}`VenvSymlinkEntry` and {bzl:obj}`VenvSymlinkKind` in -{bzl:target}`//python:py_info.bzl`. diff --git a/news/gazelle-bzl-library.changed.md b/news/gazelle-bzl-library.changed.md deleted file mode 100644 index 0d602110e3..0000000000 --- a/news/gazelle-bzl-library.changed.md +++ /dev/null @@ -1,3 +0,0 @@ -Renamed most public bzl_library targets from `{foo}_bzl` to `{foo}` to follow -gazelle naming conventions. Deprecated aliases are left for backwards -compatibility. diff --git a/news/loadable-symbols.added.md b/news/loadable-symbols.added.md deleted file mode 100644 index 7d92df7aea..0000000000 --- a/news/loadable-symbols.added.md +++ /dev/null @@ -1,2 +0,0 @@ -Added {bzl:obj}`features.loadable_symbols` to allow detecting public symbols -exported by bzl files. diff --git a/news/slash-target-venv-output.fixed.md b/news/slash-target-venv-output.fixed.md deleted file mode 100644 index 780b498db7..0000000000 --- a/news/slash-target-venv-output.fixed.md +++ /dev/null @@ -1,3 +0,0 @@ -(rules) Fixed venv output paths for `py_binary` and `py_test` targets whose -names contain path separators so distinct targets with the same basename no -longer share the same venv output directory. diff --git a/news/win32_version_lookup.fixed.md b/news/win32_version_lookup.fixed.md deleted file mode 100644 index 6f01c9e73c..0000000000 --- a/news/win32_version_lookup.fixed.md +++ /dev/null @@ -1,3 +0,0 @@ -Fixed a flaky error on Windows 2022 when looking up the win32 version during -site initialization by retrying the lookup -([#3721](https://github.com/bazel-contrib/rules_python/issues/3721)). diff --git a/python/features.bzl b/python/features.bzl index ee850ade9e..33323b8b65 100644 --- a/python/features.bzl +++ b/python/features.bzl @@ -54,7 +54,7 @@ def _features_typedef(): A map of bzl paths to the list of public symbols they export. - :::{versionadded} VERSION_NEXT_FEATURE + :::{versionadded} 2.2.0 ::: :::: diff --git a/python/private/py_info.bzl b/python/private/py_info.bzl index da9421606a..dac1ddeff3 100644 --- a/python/private/py_info.bzl +++ b/python/private/py_info.bzl @@ -125,7 +125,7 @@ def _VenvSymlinkEntryBuilder_typedef(): :type: DepsetBuilder[File] ::: - :::{versionadded} VERSION_NEXT_FEATURE + :::{versionadded} 2.2.0 ::: """ @@ -677,7 +677,7 @@ def _PyInfoBuilder_new(): def _PyInfoBuilder_add_venv_symlink(self): """Create and return a new VenvSymlinkEntryBuilder. - :::{versionadded} VERSION_NEXT_FEATURE + :::{versionadded} 2.2.0 ::: Args: diff --git a/python/private/pypi/extension.bzl b/python/private/pypi/extension.bzl index 62d2c8de16..02c5ad0c34 100644 --- a/python/private/pypi/extension.bzl +++ b/python/private/pypi/extension.bzl @@ -845,7 +845,7 @@ means if different programs need different versions of some library, separate hubs can be created, and each program can use its respective hub's targets. Targets from different hubs should not be used together. -:::{versionchanged} VERSION_NEXT_FEATURE +:::{versionchanged} 2.2.0 Using the hub name `"pypi"` is deprecated and is changed to `{module_name}_pypi` depending on the {envvar}`RULES_PYTHON_PYPI_HUB_RESERVED` environment variable. diff --git a/python/uv/private/lock.bzl b/python/uv/private/lock.bzl index 8caf0ed3de..695591e924 100644 --- a/python/uv/private/lock.bzl +++ b/python/uv/private/lock.bzl @@ -387,7 +387,7 @@ _lock = rule( The lock rule that does the locking in a build action and also prepares information for a `bazel run` executable rule. -:::{versionadded} VERSION_NEXT_FEATURE +:::{versionadded} 2.2.0 ::: """, attrs = _common_attrs | { From 098379f8ee632d951e6ced14885dfa3ee078a1a2 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Mon, 29 Jun 2026 23:29:00 -0700 Subject: [PATCH 794/922] chore(release): fix release label consistency in workflows and templates (#3876) This change fixes the release tracking and branch cutting workflows to use the correct "type: release" label with a space. This ensures the automation triggers correctly and matches the label expected by the release tool. --- .github/ISSUE_TEMPLATE/release_tracking_template.md | 2 +- .github/workflows/cut_release_branch.yml | 4 ++-- .github/workflows/process_backports.yml | 2 +- 3 files changed, 4 insertions(+), 4 deletions(-) diff --git a/.github/ISSUE_TEMPLATE/release_tracking_template.md b/.github/ISSUE_TEMPLATE/release_tracking_template.md index 66beea28b7..b7d355e210 100644 --- a/.github/ISSUE_TEMPLATE/release_tracking_template.md +++ b/.github/ISSUE_TEMPLATE/release_tracking_template.md @@ -2,7 +2,7 @@ name: Release Tracking Issue about: Checklist for tracking a new release of rules_python. title: 'Release ' -labels: ['type:release'] +labels: ['type: release'] --- # Release tasks - [ ] Prepare Release | status=awaiting-preparation diff --git a/.github/workflows/cut_release_branch.yml b/.github/workflows/cut_release_branch.yml index b7c7118194..9ae40d0700 100644 --- a/.github/workflows/cut_release_branch.yml +++ b/.github/workflows/cut_release_branch.yml @@ -10,8 +10,8 @@ permissions: jobs: cut_branch: - # Run only if the issue has the type:release label - if: contains(github.event.issue.labels.*.name, 'type:release') + # Run only if the issue has the type: release label + if: contains(github.event.issue.labels.*.name, 'type: release') runs-on: ubuntu-latest steps: - name: Checkout repository diff --git a/.github/workflows/process_backports.yml b/.github/workflows/process_backports.yml index 9a42b6c62c..4d8055d3c5 100644 --- a/.github/workflows/process_backports.yml +++ b/.github/workflows/process_backports.yml @@ -14,7 +14,7 @@ permissions: jobs: process_backports: - # Always gate GHA runs to ensure we are operating on a type:release labeled issue if metadata is queried + # Always gate GHA runs to ensure we are operating on a type: release labeled issue if metadata is queried runs-on: ubuntu-latest steps: - name: Checkout repository From fbb2ccf4ca4ac5d43f2e4181469b04dc14cfb092 Mon Sep 17 00:00:00 2001 From: Gleb Kolobkov Date: Tue, 30 Jun 2026 18:00:59 -0700 Subject: [PATCH 795/922] fix: avoid stage1 bootstrap stdlib shadowing (#3854) The stage 1 system-python bootstrap imports standard-library modules before it re-execs into the configured runtime. If the target output directory contains files named like standard-library modules, such as `shutil.py` or `types.py`, the bootstrap can import those files instead and fail before user code starts. This change removes Python's unsafe script-directory prepend before those early imports, while preserving `sys.path[0]` when Python has already suppressed that prepend through `-P`/`PYTHONSAFEPATH` or isolated mode. Stage 2 now uses the same isolated-mode guard when removing or recreating the main-directory path. It also pins the Windows venv entry point template to LF line endings so the compile-pip CI dirty-worktree check is stable across checkout settings. Before this change, a generated target output could shadow bootstrap stdlib imports, and isolated-mode runs on older Python versions could lose a required stdlib path. After this change, bootstrap stdlib imports resolve from the interpreter's standard library, isolated mode preserves the interpreter-provided path, and the line-ending-sensitive CI check stays clean. Tests: ```shell bazel test --config=fast-tests \ //tests/bootstrap_impls:stdlib_shadowing_system_python_test bazel test --config=fast-tests \ //tests/bootstrap_impls:stdlib_shadowing_system_python_test \ //tests/bootstrap_impls:interpreter_args_test \ //tests/bootstrap_impls:sys_path_order_bootstrap_script_test ``` --- .bazelci/presubmit.yml | 4 +++ news/3854.fixed.md | 2 ++ python/private/python_bootstrap_template.txt | 20 +++++++++++++- python/private/stage2_bootstrap_template.py | 15 ++++++++--- tests/bootstrap_impls/BUILD.bazel | 26 +++++++++++++++++++ .../bootstrap_impls/stdlib_shadowing_test.py | 20 ++++++++++++++ 6 files changed, 82 insertions(+), 5 deletions(-) create mode 100644 news/3854.fixed.md create mode 100644 tests/bootstrap_impls/stdlib_shadowing_test.py diff --git a/.bazelci/presubmit.yml b/.bazelci/presubmit.yml index 79d2dfbc30..b3e308c7c7 100644 --- a/.bazelci/presubmit.yml +++ b/.bazelci/presubmit.yml @@ -84,6 +84,9 @@ buildifier: coverage_targets: ["..."] .coverage_targets_example_bzlmod_build_file_generation: &coverage_targets_example_bzlmod_build_file_generation coverage_targets: ["//:bzlmod_build_file_generation_test"] +.coverage_targets_bootstrap: &coverage_targets_bootstrap + coverage_targets: + - //tests/bootstrap_impls:stdlib_shadowing_system_python_test .coverage_targets_example_multi_python: &coverage_targets_example_multi_python coverage_targets: - //tests:my_lib_3_10_test @@ -216,6 +219,7 @@ tasks: bazel: 7.x ubuntu: <<: *reusable_config + <<: *coverage_targets_bootstrap name: "Default: Ubuntu, Bazel {bazel}" platform: ubuntu2204 bazel: ${{ bazel }} diff --git a/news/3854.fixed.md b/news/3854.fixed.md new file mode 100644 index 0000000000..5c38b27642 --- /dev/null +++ b/news/3854.fixed.md @@ -0,0 +1,2 @@ +(bootstrap) Fixed stage 1 bootstrap imports when target outputs shadow standard +library modules. diff --git a/python/private/python_bootstrap_template.txt b/python/private/python_bootstrap_template.txt index 38c93ec5b5..482918c038 100644 --- a/python/private/python_bootstrap_template.txt +++ b/python/private/python_bootstrap_template.txt @@ -5,13 +5,31 @@ from __future__ import absolute_import from __future__ import division from __future__ import print_function +import sys + +# By default, Python prepends the directory containing this script to +# sys.path. The stage 1 bootstrap only needs stdlib modules before it +# re-execs into the configured runtime, so avoid resolving imports from the +# target's output or runfiles package directory. This matters when that +# directory contains files with stdlib names, such as shutil.py or types.py. +# +# Python 3.11 introduced PYTHONSAFEPATH (-P), which disables the unsafe prepend. +# Isolated mode (-I) also disables it, including on older interpreters without +# safe_path. In either case, sys.path[0] is not the script directory and should +# be preserved. +if ( + not getattr(sys.flags, "safe_path", False) and + not getattr(sys.flags, "isolated", False) and + sys.path +): + del sys.path[0] + # Generated file from @rules_python//python/private:python_bootstrap_template.txt from os.path import abspath, dirname, join, basename, normpath import os import shutil import subprocess -import sys # NOTE: The sentinel strings are split (e.g., "%stage2" + "_bootstrap%") so that # the substitution logic won't replace them. This allows runtime detection of diff --git a/python/private/stage2_bootstrap_template.py b/python/private/stage2_bootstrap_template.py index b11fc76093..f445ad2b6a 100644 --- a/python/private/stage2_bootstrap_template.py +++ b/python/private/stage2_bootstrap_template.py @@ -9,11 +9,16 @@ # and is a special case of #7091. # # Python 3.11 introduced an PYTHONSAFEPATH (-P) option that disables this -# behaviour, which we set in the stage 1 bootstrap. +# behaviour, which we set in the stage 1 bootstrap. Isolated mode (-I) also +# disables it, including on older interpreters without safe_path. # So the prepended entry needs to be removed only if the above option is either # unset or not supported by the interpreter. # NOTE: This can be removed when Python 3.10 and below is no longer supported -if not getattr(sys.flags, "safe_path", False): +if ( + not getattr(sys.flags, "safe_path", False) + and not getattr(sys.flags, "isolated", False) + and sys.path +): del sys.path[0] import contextlib @@ -539,8 +544,10 @@ def main(): # means only other generated files are importable (not source files). # # To replicate this behavior, we add main's directory within the runfiles - # when safe path isn't enabled. - if not getattr(sys.flags, "safe_path", False): + # when safe path or isolated mode isn't enabled. + if not getattr(sys.flags, "safe_path", False) and not getattr( + sys.flags, "isolated", False + ): prepend_path_entries = [ os.path.join(runfiles_root, os.path.dirname(main_rel_path)) ] diff --git a/tests/bootstrap_impls/BUILD.bazel b/tests/bootstrap_impls/BUILD.bazel index ab3148db00..89cd682a6a 100644 --- a/tests/bootstrap_impls/BUILD.bazel +++ b/tests/bootstrap_impls/BUILD.bazel @@ -136,6 +136,32 @@ py_reconfig_test( main = "sys_path_order_test.py", ) +genrule( + name = "stdlib_shadowing_outputs", + outs = [ + "shutil.py", + "types.py", + ], + cmd = """ +cat > $(@D)/shutil.py <<'PY' +raise RuntimeError("target output shutil.py shadowed the stdlib shutil module") +PY +cat > $(@D)/types.py <<'PY' +raise RuntimeError("target output types.py shadowed the stdlib types module") +PY +""", +) + +py_reconfig_test( + name = "stdlib_shadowing_system_python_test", + srcs = [ + "stdlib_shadowing_test.py", + ":stdlib_shadowing_outputs", + ], + bootstrap_impl = "system_python", + main = "stdlib_shadowing_test.py", +) + py_reconfig_test( name = "main_module_test", srcs = ["main_module.py"], diff --git a/tests/bootstrap_impls/stdlib_shadowing_test.py b/tests/bootstrap_impls/stdlib_shadowing_test.py new file mode 100644 index 0000000000..fc913857db --- /dev/null +++ b/tests/bootstrap_impls/stdlib_shadowing_test.py @@ -0,0 +1,20 @@ +# Copyright 2026 The Bazel Authors. All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +"""Verifies stage 1 bootstrap stdlib imports cannot be shadowed.""" + + +def test_bootstrap_reached_main(): + # If stage 1 imports target outputs such as shutil.py instead of stdlib + # modules, the process fails before this test module is executed. + pass From 865ed259b16fc77f38f08dc8a45e4bb11a44537c Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Tue, 30 Jun 2026 19:25:36 -0700 Subject: [PATCH 796/922] build(deps): bump bazel-contrib/publish-to-bcr/.github/workflows/publish.yaml from 1.1.0 to 1.4.1 (#3822) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Bumps [bazel-contrib/publish-to-bcr/.github/workflows/publish.yaml](https://github.com/bazel-contrib/publish-to-bcr) from 1.1.0 to 1.4.1.
Release notes

Sourced from bazel-contrib/publish-to-bcr/.github/workflows/publish.yaml's releases.

v1.4.1

What's Changed

Full Changelog: https://github.com/bazel-contrib/publish-to-bcr/compare/v1.4.0...v1.4.1

v1.4.0

What's Changed

New Contributors

Full Changelog: https://github.com/bazel-contrib/publish-to-bcr/compare/v1.3.0...v1.4.0

v1.3.0

What's Changed

Full Changelog: https://github.com/bazel-contrib/publish-to-bcr/compare/v1.2.0...v1.3.0

v1.2.0

What's Changed

... (truncated)

Commits

[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=bazel-contrib/publish-to-bcr/.github/workflows/publish.yaml&package-manager=github_actions&previous-version=1.1.0&new-version=1.4.1)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) ---
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot show ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> --- .github/workflows/publish.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.github/workflows/publish.yml b/.github/workflows/publish.yml index 5ef65e83a6..c5db587f18 100644 --- a/.github/workflows/publish.yml +++ b/.github/workflows/publish.yml @@ -21,7 +21,7 @@ on: type: string jobs: publish: - uses: bazel-contrib/publish-to-bcr/.github/workflows/publish.yaml@v1.1.0 + uses: bazel-contrib/publish-to-bcr/.github/workflows/publish.yaml@v1.4.1 with: tag_name: ${{ inputs.tag_name }} # GitHub repository which is a fork of the upstream where the Pull Request will be opened. From 3ab60b5598b0c9ef59a9dd1dd08e79d12aedd488 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Tue, 30 Jun 2026 19:29:19 -0700 Subject: [PATCH 797/922] chore(release): support custom remote in create-release-branch and refactor (#3877) - Add --remote flag to create-release-branch subcommand to support pushing to upstream locally and origin in GHA. - Factor cmd_create_release_branch out of release.py into create_release_branch.py. - Use branch_url instead of branch name in tracking issue metadata. - Update release tracking template with manual editing and backporting instructions. --- .../release_tracking_template.md | 19 ++++- .github/workflows/cut_release_branch.yml | 2 +- tests/tools/private/release/release_test.py | 73 ++++++++++++++++++ tools/private/release/BUILD.bazel | 1 + .../private/release/create_release_branch.py | 67 ++++++++++++++++ tools/private/release/release.py | 77 ++++--------------- tools/private/release/release_issue.py | 2 + tools/private/release/utils.py | 2 +- 8 files changed, 176 insertions(+), 67 deletions(-) create mode 100644 tools/private/release/create_release_branch.py diff --git a/.github/ISSUE_TEMPLATE/release_tracking_template.md b/.github/ISSUE_TEMPLATE/release_tracking_template.md index b7d355e210..f573fdfacf 100644 --- a/.github/ISSUE_TEMPLATE/release_tracking_template.md +++ b/.github/ISSUE_TEMPLATE/release_tracking_template.md @@ -11,11 +11,28 @@ labels: ['type: release'] - [ ] Tag Final ## Backports - + +
+How to add backports + +To request a backport: +1. Add a new checklist item under the `## Backports` section. +2. The format must be: `- [ ] #` (e.g., `- [ ] #1234`). +3. Trigger the [Process Backports Workflow](https://github.com/bazel-contrib/rules_python/actions/workflows/process_backports.yml). +
--- *Maintainers: Automation will react to changes on this issue.* +
+Manual Editing + +You can manually edit this issue to control the release flow. +The checklist items use metadata suffix: `| key=value key2=value2`. +- **Retry Prepare Release**: Reset to `- [ ] Prepare Release | status=awaiting-preparation`. +- **Force Task Done**: Check the box `- [x]` and add appropriate metadata (e.g. `status=done`). +
+
Available Commands diff --git a/.github/workflows/cut_release_branch.yml b/.github/workflows/cut_release_branch.yml index 9ae40d0700..f054f4cbc8 100644 --- a/.github/workflows/cut_release_branch.yml +++ b/.github/workflows/cut_release_branch.yml @@ -32,6 +32,6 @@ jobs: - name: Attempt Branch Creation run: | bazel run //tools/private/release -- \ - create-release-branch --issue ${{ github.event.issue.number }} + create-release-branch --issue ${{ github.event.issue.number }} --remote origin env: GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} diff --git a/tests/tools/private/release/release_test.py b/tests/tools/private/release/release_test.py index 293b82b277..d199eac99e 100644 --- a/tests/tools/private/release/release_test.py +++ b/tests/tools/private/release/release_test.py @@ -18,10 +18,12 @@ def _mock_git_and_gh(test_case): # Patch bindings in modules that import them at module level patch("tools.private.release.release.git", new=mock_git).start() patch("tools.private.release.prepare.git", new=mock_git).start() + patch("tools.private.release.create_release_branch.git", new=mock_git).start() patch("tools.private.release.utils.git", new=mock_git).start() patch("tools.private.release.release.gh", new=mock_gh).start() patch("tools.private.release.prepare.gh", new=mock_gh).start() + patch("tools.private.release.create_release_branch.gh", new=mock_gh).start() mock_gh.MultipleTrackingIssuesError = MultipleTrackingIssuesError mock_gh.NoTrackingIssueError = NoTrackingIssueError @@ -1218,5 +1220,76 @@ def test_promote_rc_no_rc_found(self): self.mock_gh.get_issue_body.assert_not_called() +class CmdCreateReleaseBranchTest(unittest.TestCase): + def setUp(self): + _mock_git_and_gh(self) + + def test_create_release_branch_success(self): + # Arrange + args = MagicMock(issue=123, remote="my-remote") + self.mock_gh.get_issue_title.return_value = "Release 2.0.0" + self.mock_gh.get_issue_body.return_value = """ +## Checklist +- [x] Prepare Release | status=done pr=#122 commit=abcdef12 +- [ ] Create Release branch | status=pending +""" + self.mock_git.branch_exists.return_value = False + + # Act + result = releaser.cmd_create_release_branch(args) + + # Assert + self.assertEqual(result, 0) + self.mock_git.fetch.assert_called_once_with("my-remote") + self.mock_git.checkout.assert_any_call("abcdef12") + self.mock_git.checkout.assert_any_call("release/2.0", create_branch=True) + self.mock_git.push.assert_called_once_with("my-remote", "release/2.0") + + self.mock_gh.update_issue_body.assert_called_once() + call_args = self.mock_gh.update_issue_body.call_args[0] + self.assertEqual(call_args[0], 123) + self.assertIn( + "branch_url=https://github.com/bazel-contrib/rules_python/tree/release/2.0", + call_args[1], + ) + self.assertIn("commit=abcdef12", call_args[1]) + + def test_create_release_branch_prepare_not_done(self): + # Arrange + args = MagicMock(issue=123, remote="my-remote") + self.mock_gh.get_issue_title.return_value = "Release 2.0.0" + self.mock_gh.get_issue_body.return_value = """ +## Checklist +- [ ] Prepare Release | status=pending +- [ ] Create Release branch | status=pending +""" + # Act + result = releaser.cmd_create_release_branch(args) + + # Assert + self.assertEqual(result, 1) + self.mock_git.fetch.assert_not_called() + self.mock_git.push.assert_not_called() + self.mock_gh.update_issue_body.assert_not_called() + + def test_create_release_branch_already_checked(self): + # Arrange + args = MagicMock(issue=123, remote="my-remote") + self.mock_gh.get_issue_title.return_value = "Release 2.0.0" + self.mock_gh.get_issue_body.return_value = """ +## Checklist +- [x] Prepare Release | status=done pr=#122 commit=abcdef12 +- [x] Create Release branch | status=done branch=release/2.0 commit=abcdef12 +""" + # Act + result = releaser.cmd_create_release_branch(args) + + # Assert + self.assertEqual(result, 0) + self.mock_git.fetch.assert_not_called() + self.mock_git.push.assert_not_called() + self.mock_gh.update_issue_body.assert_not_called() + + if __name__ == "__main__": unittest.main() diff --git a/tools/private/release/BUILD.bazel b/tools/private/release/BUILD.bazel index 96f864141d..de2aba4b71 100644 --- a/tools/private/release/BUILD.bazel +++ b/tools/private/release/BUILD.bazel @@ -10,6 +10,7 @@ py_library( py_binary( name = "release", srcs = [ + "create_release_branch.py", "gh.py", "git.py", "prepare.py", diff --git a/tools/private/release/create_release_branch.py b/tools/private/release/create_release_branch.py new file mode 100644 index 0000000000..45fec27196 --- /dev/null +++ b/tools/private/release/create_release_branch.py @@ -0,0 +1,67 @@ +"""Subcommand to create a release branch from a merged PR commit.""" + +from tools.private.release import gh, git +from tools.private.release.release_issue import ( + RELEASE_TITLE_RE, + parse_checklist_state, + update_task_in_body, +) +from tools.private.release.utils import REPO_URL + + +def cmd_create_release_branch(args): + """Executes the create-release-branch subcommand.""" + print(f"Evaluating branch creation for tracking issue #{args.issue}...") + body = gh.get_issue_body(args.issue) + state = parse_checklist_state(body) + + if ( + state["prepare_release"]["status"] != "done" + or not state["prepare_release"]["commit"] + ): + print( + "Error: Prepare Release task is not marked 'done' with a valid commit SHA." + ) + return 1 + + if state["create_branch"]["checked"]: + print("Release branch has already been created and checked. Skipping.") + return 0 + + # Extract version from issue title + issue_title = gh.get_issue_title(args.issue) + version_match = RELEASE_TITLE_RE.search(issue_title) + if not version_match: + print(f"Error: Could not parse version from issue title: {issue_title}") + return 1 + + version = version_match.group(1) + branch_version = ".".join(version.split(".")[:2]) + branch_name = f"release/{branch_version}" + + commit_sha = state["prepare_release"]["commit"] + print(f"Cutting branch {branch_name} from commit {commit_sha}...") + + # Create and push branch + git.fetch(args.remote) + git.checkout(commit_sha) + + if not git.branch_exists(branch_name): + git.checkout(branch_name, create_branch=True) + else: + git.checkout(branch_name) + git.merge(commit_sha, ff_only=True) + + git.push(args.remote, branch_name) + print(f"Successfully pushed branch {branch_name} to {args.remote}") + + # Update tracking issue checklist + print("Updating tracking issue checklist...") + branch_url = f"{REPO_URL}/tree/{branch_name}" + metadata = {"status": "done", "branch_url": branch_url, "commit": commit_sha[:8]} + updated_body = update_task_in_body( + body, "Create Release branch", checked=True, metadata=metadata + ) + gh.update_issue_body(args.issue, updated_body) + print("Create Release branch task marked complete successfully!") + return 0 diff --git a/tools/private/release/release.py b/tools/private/release/release.py index dab99e6d05..cae0053014 100644 --- a/tools/private/release/release.py +++ b/tools/private/release/release.py @@ -8,20 +8,20 @@ import sys from tools.private.release import changelog_news, gh, git +from tools.private.release.create_release_branch import cmd_create_release_branch from tools.private.release.prepare import cmd_prepare from tools.private.release.release_issue import ( + RELEASE_TITLE_RE, parse_checklist_state, parse_metadata_line, update_task_in_body, ) from tools.private.release.utils import ( - _REPO_URL, + REPO_URL, determine_next_version, get_latest_rc_tag, ) -_RELEASE_TITLE_RE = re.compile(r"Release (\d+\.\d+\.\d+)", re.IGNORECASE) - def _semver_type(value): if not re.match(r"^\d+\.\d+\.\d+(rc\d+)?$", value): @@ -141,63 +141,6 @@ def cmd_complete_prepare(args): return 0 -def cmd_create_release_branch(args): - """Executes the create-release-branch subcommand.""" - print(f"Evaluating branch creation for tracking issue #{args.issue}...") - body = gh.get_issue_body(args.issue) - state = parse_checklist_state(body) - - if ( - state["prepare_release"]["status"] != "done" - or not state["prepare_release"]["commit"] - ): - print( - "Error: Prepare Release task is not marked 'done' with a valid commit SHA." - ) - return 1 - - if state["create_branch"]["checked"]: - print("Release branch has already been created and checked. Skipping.") - return 0 - - # Extract version from issue title - issue_title = gh.get_issue_title(args.issue) - version_match = _RELEASE_TITLE_RE.search(issue_title) - if not version_match: - print(f"Error: Could not parse version from issue title: {issue_title}") - return 1 - - version = version_match.group(1) - branch_version = ".".join(version.split(".")[:2]) - branch_name = f"release/{branch_version}" - - commit_sha = state["prepare_release"]["commit"] - print(f"Cutting branch {branch_name} from commit {commit_sha}...") - - # Create and push branch - git.fetch("origin") - git.checkout(commit_sha) - - if not git.branch_exists(branch_name): - git.checkout(branch_name, create_branch=True) - else: - git.checkout(branch_name) - git.merge(commit_sha, ff_only=True) - - git.push("origin", branch_name) - print(f"Successfully pushed branch {branch_name}") - - # Update tracking issue checklist - print("Updating tracking issue checklist...") - metadata = {"status": "done", "branch": branch_name, "commit": commit_sha[:8]} - updated_body = update_task_in_body( - body, "Create Release branch", checked=True, metadata=metadata - ) - gh.update_issue_body(args.issue, updated_body) - print("Create Release branch task marked complete successfully!") - return 0 - - def cmd_process_backports(args): """Executes the process-backports subcommand.""" body = gh.get_issue_body(args.issue) @@ -217,7 +160,7 @@ def cmd_process_backports(args): # Determine branch name from issue title issue_title = gh.get_issue_title(args.issue) - version_match = _RELEASE_TITLE_RE.search(issue_title) + version_match = RELEASE_TITLE_RE.search(issue_title) if not version_match: print(f"Error: Could not parse version from issue title: {issue_title}") return 1 @@ -348,7 +291,7 @@ def cmd_create_rc(args): # Resolve version and branch issue_title = gh.get_issue_title(args.issue) - version_match = _RELEASE_TITLE_RE.search(issue_title) + version_match = RELEASE_TITLE_RE.search(issue_title) if not version_match: print(f"Error: Could not parse version from issue title: {issue_title}") return 1 @@ -405,7 +348,7 @@ def cmd_create_rc(args): updated_body = update_task_in_body(body, task_name, checked=True, metadata=metadata) gh.update_issue_body(args.issue, updated_body) - tag_url = f"{_REPO_URL}/releases/tag/{next_rc}" + tag_url = f"{REPO_URL}/releases/tag/{next_rc}" bcr_search_url = f"https://github.com/bazelbuild/bazel-central-registry/pulls?q=is%3Apr+rules_python+{version}" comment_body = f"""🚀 **New Release Candidate Tagged!** @@ -489,7 +432,7 @@ def cmd_promote_rc(args): print(f"Posting comment to tracking issue #{issue_num}...") import urllib.parse - release_url = f"{_REPO_URL}/releases/tag/{version}" + release_url = f"{REPO_URL}/releases/tag/{version}" bcr_query = f'is:pr ("bazel-contrib/rules_python" in:title) ("@{version}" in:title)' bcr_search_url = f"https://github.com/bazelbuild/bazel-central-registry/pulls?q={urllib.parse.quote(bcr_query)}" comment_body = ( @@ -576,6 +519,12 @@ def create_parser(): required=True, help="The tracking issue number (required).", ) + create_branch_parser.add_argument( + "--remote", + type=str, + required=True, + help="The git remote to create the branch on (required).", + ) # Subcommand: process-backports process_backports_parser = subparsers.add_parser( diff --git a/tools/private/release/release_issue.py b/tools/private/release/release_issue.py index f96b917ab9..c170217471 100644 --- a/tools/private/release/release_issue.py +++ b/tools/private/release/release_issue.py @@ -2,6 +2,8 @@ import re +RELEASE_TITLE_RE = re.compile(r"Release (\d+\.\d+\.\d+)", re.IGNORECASE) + def parse_metadata_line(line): """Parses a checklist line with optional | key=value metadata.""" diff --git a/tools/private/release/utils.py b/tools/private/release/utils.py index 83ecbb9c7a..6f7383f8ec 100644 --- a/tools/private/release/utils.py +++ b/tools/private/release/utils.py @@ -8,7 +8,7 @@ from tools.private.release import git -_REPO_URL = "https://github.com/bazel-contrib/rules_python" +REPO_URL = "https://github.com/bazel-contrib/rules_python" _EXCLUDE_PATTERNS = [ "./.git/*", From 2241f8b6a1175577c4403573bb5aabd5e5182f11 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Tue, 30 Jun 2026 19:59:41 -0700 Subject: [PATCH 798/922] chore(release): refactor create-rc and handle existing remote release branch (#3879) - Move cmd_create_rc logic from release.py to a new create_rc.py module. - Move parse_backports from release.py to release_issue.py. - Add required --remote flag to create-rc subcommand. - Update generate_rc.yml workflow to pass --remote origin. - In create-release-branch, detect if remote branch already exists and handle same-commit, fast-forward, and non-fast-forward cases. - Add remote_branch_exists and is_ancestor helpers to git.py. --- .github/workflows/generate_rc.yml | 2 +- tests/tools/private/release/release_test.py | 117 ++++++++++++++- tools/private/release/BUILD.bazel | 1 + tools/private/release/create_rc.py | 114 +++++++++++++++ .../private/release/create_release_branch.py | 35 +++-- tools/private/release/git.py | 18 +++ tools/private/release/release.py | 137 +----------------- tools/private/release/release_issue.py | 29 ++++ 8 files changed, 306 insertions(+), 147 deletions(-) create mode 100644 tools/private/release/create_rc.py diff --git a/.github/workflows/generate_rc.yml b/.github/workflows/generate_rc.yml index 26685c2f62..19a80bb1de 100644 --- a/.github/workflows/generate_rc.yml +++ b/.github/workflows/generate_rc.yml @@ -34,6 +34,6 @@ jobs: - name: Attempt RC Tagging run: | bazel run //tools/private/release -- \ - create-rc --issue ${{ inputs.issue }} + create-rc --issue ${{ inputs.issue }} --remote origin env: GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} diff --git a/tests/tools/private/release/release_test.py b/tests/tools/private/release/release_test.py index d199eac99e..636c9b2589 100644 --- a/tests/tools/private/release/release_test.py +++ b/tests/tools/private/release/release_test.py @@ -3,7 +3,7 @@ import shutil import tempfile import unittest -from unittest.mock import MagicMock, patch +from unittest.mock import MagicMock, call, patch from tools.private.release import changelog_news, release as releaser, utils from tools.private.release.gh import MultipleTrackingIssuesError, NoTrackingIssueError @@ -19,11 +19,13 @@ def _mock_git_and_gh(test_case): patch("tools.private.release.release.git", new=mock_git).start() patch("tools.private.release.prepare.git", new=mock_git).start() patch("tools.private.release.create_release_branch.git", new=mock_git).start() + patch("tools.private.release.create_rc.git", new=mock_git).start() patch("tools.private.release.utils.git", new=mock_git).start() patch("tools.private.release.release.gh", new=mock_gh).start() patch("tools.private.release.prepare.gh", new=mock_gh).start() patch("tools.private.release.create_release_branch.gh", new=mock_gh).start() + patch("tools.private.release.create_rc.gh", new=mock_gh).start() mock_gh.MultipleTrackingIssuesError = MultipleTrackingIssuesError mock_gh.NoTrackingIssueError = NoTrackingIssueError @@ -932,7 +934,7 @@ def setUp(self): def test_create_rc_success_first_rc(self): # Arrange - args = MagicMock(issue=123) + args = MagicMock(issue=123, remote="my-remote") self.mock_gh.get_issue_title.return_value = "Release 2.0.0" self.mock_gh.get_issue_body.return_value = """ ## Checklist @@ -949,8 +951,12 @@ def test_create_rc_success_first_rc(self): # Assert self.assertEqual(result, 0) + self.mock_git.fetch.assert_has_calls( + [call("my-remote"), call("my-remote", tags=True, force=True)] + ) + self.mock_git.checkout.assert_called_once_with("my-remote/release/2.0") self.mock_git.tag.assert_called_once_with("2.0.0-rc0", "HEAD") - self.mock_git.push.assert_called_once_with("origin", "2.0.0-rc0") + self.mock_git.push.assert_called_once_with("my-remote", "2.0.0-rc0") self.mock_gh.update_issue_body.assert_called_once() call_args = self.mock_gh.update_issue_body.call_args[0] @@ -959,10 +965,21 @@ def test_create_rc_success_first_rc(self): self.assertIn("commit=12345678", call_args[1]) self.mock_gh.post_issue_comment.assert_called_once() + comment_call_args = self.mock_gh.post_issue_comment.call_args[0] + self.assertEqual(comment_call_args[0], 123) + self.assertIn( + "**New Release Candidate Tagged!** 🐍🌿", + comment_call_args[1], + ) + self.assertIn( + "- Trigger Release Workflow: [Release Workflow](https://github.com/bazel-contrib/rules_python/actions/workflows/release.yml)", + comment_call_args[1], + ) + self.assertNotIn("🚀", comment_call_args[1]) def test_create_rc_success_next_rc(self): # Arrange - args = MagicMock(issue=123) + args = MagicMock(issue=123, remote="my-remote") self.mock_gh.get_issue_title.return_value = "Release 2.0.0" self.mock_gh.get_issue_body.return_value = """ ## Checklist @@ -980,8 +997,12 @@ def test_create_rc_success_next_rc(self): # Assert self.assertEqual(result, 0) + self.mock_git.fetch.assert_has_calls( + [call("my-remote"), call("my-remote", tags=True, force=True)] + ) + self.mock_git.checkout.assert_called_once_with("my-remote/release/2.0") self.mock_git.tag.assert_called_once_with("2.0.0-rc1", "HEAD") - self.mock_git.push.assert_called_once_with("origin", "2.0.0-rc1") + self.mock_git.push.assert_called_once_with("my-remote", "2.0.0-rc1") self.mock_gh.update_issue_body.assert_called_once() call_args = self.mock_gh.update_issue_body.call_args[0] @@ -989,6 +1010,17 @@ def test_create_rc_success_next_rc(self): self.assertIn("tag=2.0.0-rc1", call_args[1]) self.mock_gh.post_issue_comment.assert_called_once() + comment_call_args = self.mock_gh.post_issue_comment.call_args[0] + self.assertEqual(comment_call_args[0], 123) + self.assertIn( + "**New Release Candidate Tagged!** 🐍🌿", + comment_call_args[1], + ) + self.assertIn( + "- Trigger Release Workflow: [Release Workflow](https://github.com/bazel-contrib/rules_python/actions/workflows/release.yml)", + comment_call_args[1], + ) + self.assertNotIn("🚀", comment_call_args[1]) def test_create_rc_already_tagged(self): # Arrange @@ -1234,6 +1266,7 @@ def test_create_release_branch_success(self): - [ ] Create Release branch | status=pending """ self.mock_git.branch_exists.return_value = False + self.mock_git.remote_branch_exists.return_value = False # Act result = releaser.cmd_create_release_branch(args) @@ -1241,9 +1274,10 @@ def test_create_release_branch_success(self): # Assert self.assertEqual(result, 0) self.mock_git.fetch.assert_called_once_with("my-remote") - self.mock_git.checkout.assert_any_call("abcdef12") - self.mock_git.checkout.assert_any_call("release/2.0", create_branch=True) - self.mock_git.push.assert_called_once_with("my-remote", "release/2.0") + self.mock_git.checkout.assert_not_called() + self.mock_git.push.assert_called_once_with( + "my-remote", "abcdef12:refs/heads/release/2.0" + ) self.mock_gh.update_issue_body.assert_called_once() call_args = self.mock_gh.update_issue_body.call_args[0] @@ -1290,6 +1324,73 @@ def test_create_release_branch_already_checked(self): self.mock_git.push.assert_not_called() self.mock_gh.update_issue_body.assert_not_called() + def test_create_release_branch_already_exists_same_commit(self): + # Arrange + args = MagicMock(issue=123, remote="my-remote") + self.mock_gh.get_issue_title.return_value = "Release 2.0.0" + self.mock_gh.get_issue_body.return_value = """ +## Checklist +- [x] Prepare Release | status=done pr=#122 commit=abcdef12 +- [ ] Create Release branch | status=pending +""" + self.mock_git.remote_branch_exists.return_value = True + self.mock_git.get_commit_sha.return_value = "abcdef12" + + # Act + result = releaser.cmd_create_release_branch(args) + + # Assert + self.assertEqual(result, 0) + self.mock_git.fetch.assert_called_once_with("my-remote") + self.mock_git.push.assert_not_called() + self.mock_gh.update_issue_body.assert_called_once() # Should still update checklist + + def test_create_release_branch_already_exists_fast_forward(self): + # Arrange + args = MagicMock(issue=123, remote="my-remote") + self.mock_gh.get_issue_title.return_value = "Release 2.0.0" + self.mock_gh.get_issue_body.return_value = """ +## Checklist +- [x] Prepare Release | status=done pr=#122 commit=abcdef12 +- [ ] Create Release branch | status=pending +""" + self.mock_git.remote_branch_exists.return_value = True + self.mock_git.get_commit_sha.return_value = "oldcommit" + self.mock_git.is_ancestor.return_value = True + + # Act + result = releaser.cmd_create_release_branch(args) + + # Assert + self.assertEqual(result, 0) + self.mock_git.fetch.assert_called_once_with("my-remote") + self.mock_git.push.assert_called_once_with( + "my-remote", "abcdef12:refs/heads/release/2.0" + ) + self.mock_gh.update_issue_body.assert_called_once() + + def test_create_release_branch_already_exists_non_ff(self): + # Arrange + args = MagicMock(issue=123, remote="my-remote") + self.mock_gh.get_issue_title.return_value = "Release 2.0.0" + self.mock_gh.get_issue_body.return_value = """ +## Checklist +- [x] Prepare Release | status=done pr=#122 commit=abcdef12 +- [ ] Create Release branch | status=pending +""" + self.mock_git.remote_branch_exists.return_value = True + self.mock_git.get_commit_sha.return_value = "othercommit" + self.mock_git.is_ancestor.return_value = False + + # Act + result = releaser.cmd_create_release_branch(args) + + # Assert + self.assertEqual(result, 1) + self.mock_git.fetch.assert_called_once_with("my-remote") + self.mock_git.push.assert_not_called() + self.mock_gh.update_issue_body.assert_not_called() + if __name__ == "__main__": unittest.main() diff --git a/tools/private/release/BUILD.bazel b/tools/private/release/BUILD.bazel index de2aba4b71..ad2e1b4cdc 100644 --- a/tools/private/release/BUILD.bazel +++ b/tools/private/release/BUILD.bazel @@ -10,6 +10,7 @@ py_library( py_binary( name = "release", srcs = [ + "create_rc.py", "create_release_branch.py", "gh.py", "git.py", diff --git a/tools/private/release/create_rc.py b/tools/private/release/create_rc.py new file mode 100644 index 0000000000..9cd1a3b291 --- /dev/null +++ b/tools/private/release/create_rc.py @@ -0,0 +1,114 @@ +"""Subcommand to tag and push the next release candidate.""" + +from tools.private.release import gh, git +from tools.private.release.release_issue import ( + RELEASE_TITLE_RE, + parse_backports, + parse_checklist_state, + update_task_in_body, +) +from tools.private.release.utils import ( + REPO_URL, + get_latest_rc_tag, +) + + +def cmd_create_rc(args): + """Executes the create-rc subcommand.""" + body = gh.get_issue_body(args.issue) + state = parse_checklist_state(body) + + if ( + state["prepare_release"]["status"] != "done" + or state["create_branch"]["status"] != "done" + ): + print( + "Error: Preconditions not met (release must be prepared and branch created)." + ) + return 1 + + # Gating: RC tagging is blocked if any backport is unchecked OR does not have status=done + backports = parse_backports(body) + conflicting_or_pending = [ + b for b in backports if not b["checked"] or b["status"] != "done" + ] + if conflicting_or_pending: + print( + f"Gating RC tagging: {len(conflicting_or_pending)} backports are still" + " unfinished, failed, or in conflict." + ) + return 1 + + # Resolve version and branch + issue_title = gh.get_issue_title(args.issue) + version_match = RELEASE_TITLE_RE.search(issue_title) + if not version_match: + print(f"Error: Could not parse version from issue title: {issue_title}") + return 1 + + version = version_match.group(1) + branch_version = ".".join(version.split(".")[:2]) + branch_name = f"release/{branch_version}" + + # Determine next RC tag + git.fetch(args.remote) + git.fetch(args.remote, tags=True, force=True) + latest_rc = get_latest_rc_tag(version) + + if not latest_rc: + next_rc_num = 0 + next_rc = f"{version}-rc0" + else: + rc_num = int(latest_rc.split("-rc")[-1]) + next_rc_num = rc_num + 1 + next_rc = f"{version}-rc{next_rc_num}" + + # Precheck: next RC number must exist and be unchecked in the checklist + rc_tags = state.get("rc_tags", {}) + if next_rc_num not in rc_tags: + print( + f"Error: Checklist is missing required task 'Tag RC{next_rc_num}'" + f" to cut {version}-rc{next_rc_num}." + ) + return 1 + + target_rc_task = rc_tags[next_rc_num] + if target_rc_task["checked"] or target_rc_task["status"] == "done": + print( + f"Error: Task 'Tag RC{next_rc_num}' is already marked done in the checklist." + ) + return 1 + + # Verify HEAD is not already tagged + git.checkout(f"{args.remote}/{branch_name}") + head_tags = git.get_tags_at_head() + if any(tag.startswith(f"{version}-rc") for tag in head_tags): + print(f"HEAD of {branch_name} is already tagged with an RC. Skipping.") + return 0 + + print(f"Tagging and pushing next RC: {next_rc}...") + git.tag(next_rc, "HEAD") + git.push(args.remote, next_rc) + + commit_sha = git.get_commit_sha("HEAD") + + # Check off the appropriate "Tag RC{N}" task in the checklist + print(f"Checking off Tag RC{next_rc_num} task...") + metadata = {"status": "done", "tag": next_rc, "commit": commit_sha[:8]} + task_name = f"Tag RC{next_rc_num}" + updated_body = update_task_in_body(body, task_name, checked=True, metadata=metadata) + gh.update_issue_body(args.issue, updated_body) + + tag_url = f"{REPO_URL}/releases/tag/{next_rc}" + bcr_search_url = f"https://github.com/bazelbuild/bazel-central-registry/pulls?q=is%3Apr+rules_python+{version}" + release_workflow_url = f"{REPO_URL}/actions/workflows/release.yml" + comment_body = f"""**New Release Candidate Tagged!** 🐍🌿 + +Release Candidate **{next_rc}** has been successfully generated and tagged on branch `{branch_name}`. + +- View Tag: [{next_rc}]({tag_url}) +- Track BCR Progress: [Search BCR Pull Requests]({bcr_search_url}) +- Trigger Release Workflow: [Release Workflow]({release_workflow_url})""" + gh.post_issue_comment(args.issue, comment_body) + print("RC creation completed successfully!") + return 0 diff --git a/tools/private/release/create_release_branch.py b/tools/private/release/create_release_branch.py index 45fec27196..7b12465f4c 100644 --- a/tools/private/release/create_release_branch.py +++ b/tools/private/release/create_release_branch.py @@ -42,18 +42,35 @@ def cmd_create_release_branch(args): commit_sha = state["prepare_release"]["commit"] print(f"Cutting branch {branch_name} from commit {commit_sha}...") - # Create and push branch + # Create and push branch without affecting local checkout git.fetch(args.remote) - git.checkout(commit_sha) - if not git.branch_exists(branch_name): - git.checkout(branch_name, create_branch=True) + if git.remote_branch_exists(args.remote, branch_name): + remote_ref = f"{args.remote}/{branch_name}" + remote_sha = git.get_commit_sha(remote_ref) + if remote_sha == commit_sha: + print( + f"Branch {branch_name} already exists on {args.remote} and points to {commit_sha}. Skipping push." + ) + elif git.is_ancestor(remote_ref, commit_sha): + print( + f"Branch {branch_name} exists on {args.remote} but can be fast-forwarded to {commit_sha}. Pushing..." + ) + ref_spec = f"{commit_sha}:refs/heads/{branch_name}" + git.push(args.remote, ref_spec) + else: + print( + f"Error: Branch {branch_name} already exists on {args.remote} at {remote_sha[:8]}, " + f"which is not an ancestor of {commit_sha[:8]}. Cannot fast-forward." + ) + return 1 else: - git.checkout(branch_name) - git.merge(commit_sha, ff_only=True) - - git.push(args.remote, branch_name) - print(f"Successfully pushed branch {branch_name} to {args.remote}") + print(f"Branch {branch_name} does not exist on {args.remote}. Pushing...") + ref_spec = f"{commit_sha}:refs/heads/{branch_name}" + git.push(args.remote, ref_spec) + print( + f"Successfully pushed branch {branch_name} pointing to {commit_sha} to {args.remote}" + ) # Update tracking issue checklist print("Updating tracking issue checklist...") diff --git a/tools/private/release/git.py b/tools/private/release/git.py index ed6e637d3f..ce2bd8ca03 100644 --- a/tools/private/release/git.py +++ b/tools/private/release/git.py @@ -132,3 +132,21 @@ def get_tags_at_head(): def get_current_branch(): """Returns the current git branch name.""" return run_cmd("git", "rev-parse", "--abbrev-ref", "HEAD") + + +def remote_branch_exists(remote, branch_name): + """Returns True if a remote branch exists.""" + try: + run_cmd("git", "show-ref", "--verify", f"refs/remotes/{remote}/{branch_name}") + return True + except subprocess.CalledProcessError: + return False + + +def is_ancestor(ancestor, descendant): + """Returns True if ancestor is an ancestor of descendant (fast-forwardable).""" + try: + run_cmd("git", "merge-base", "--is-ancestor", ancestor, descendant) + return True + except subprocess.CalledProcessError: + return False diff --git a/tools/private/release/release.py b/tools/private/release/release.py index cae0053014..7bc6f9ba05 100644 --- a/tools/private/release/release.py +++ b/tools/private/release/release.py @@ -8,12 +8,12 @@ import sys from tools.private.release import changelog_news, gh, git +from tools.private.release.create_rc import cmd_create_rc from tools.private.release.create_release_branch import cmd_create_release_branch from tools.private.release.prepare import cmd_prepare from tools.private.release.release_issue import ( RELEASE_TITLE_RE, - parse_checklist_state, - parse_metadata_line, + parse_backports, update_task_in_body, ) from tools.private.release.utils import ( @@ -36,35 +36,6 @@ def _semver_type(value): # ============================================================================== -def parse_backports(body): - """Parses the ## Backports checklist section.""" - body = body.replace("\r\n", "\n") - match = re.search( - r"## Backports\n(.*?)(?=\n##|\n---|\Z)", body, re.DOTALL | re.IGNORECASE - ) - if not match: - return [] - - section_content = match.group(1) - items = [] - lines = section_content.splitlines() - - for line in lines: - parsed = parse_metadata_line(line) - if parsed: - items.append( - { - "pr_ref": parsed["name"], - "checked": parsed["checked"], - "status": parsed["metadata"].get("status", "PENDING"), - "rc": parsed["metadata"].get("rc"), - "commit": parsed["metadata"].get("commit"), - "metadata": parsed["metadata"], - } - ) - return items - - # ============================================================================== # Subcommand Execution Functions # ============================================================================== @@ -263,104 +234,6 @@ def cmd_process_backports(args): return 0 -def cmd_create_rc(args): - """Executes the create-rc subcommand.""" - body = gh.get_issue_body(args.issue) - state = parse_checklist_state(body) - - if ( - state["prepare_release"]["status"] != "done" - or state["create_branch"]["status"] != "done" - ): - print( - "Error: Preconditions not met (release must be prepared and branch created)." - ) - return 1 - - # Gating: RC tagging is blocked if any backport is unchecked OR does not have status=done - backports = parse_backports(body) - conflicting_or_pending = [ - b for b in backports if not b["checked"] or b["status"] != "done" - ] - if conflicting_or_pending: - print( - f"Gating RC tagging: {len(conflicting_or_pending)} backports are still" - " unfinished, failed, or in conflict." - ) - return 1 - - # Resolve version and branch - issue_title = gh.get_issue_title(args.issue) - version_match = RELEASE_TITLE_RE.search(issue_title) - if not version_match: - print(f"Error: Could not parse version from issue title: {issue_title}") - return 1 - - version = version_match.group(1) - branch_version = ".".join(version.split(".")[:2]) - branch_name = f"release/{branch_version}" - - # Determine next RC tag - git.fetch("--tags", "--force") - latest_rc = get_latest_rc_tag(version) - - if not latest_rc: - next_rc_num = 0 - next_rc = f"{version}-rc0" - else: - rc_num = int(latest_rc.split("-rc")[-1]) - next_rc_num = rc_num + 1 - next_rc = f"{version}-rc{next_rc_num}" - - # Precheck: next RC number must exist and be unchecked in the checklist - rc_tags = state.get("rc_tags", {}) - if next_rc_num not in rc_tags: - print( - f"Error: Checklist is missing required task 'Tag RC{next_rc_num}'" - f" to cut {version}-rc{next_rc_num}." - ) - return 1 - - target_rc_task = rc_tags[next_rc_num] - if target_rc_task["checked"] or target_rc_task["status"] == "done": - print( - f"Error: Task 'Tag RC{next_rc_num}' is already marked done in the checklist." - ) - return 1 - - # Verify HEAD is not already tagged - git.checkout(branch_name) - head_tags = git.get_tags_at_head() - if any(tag.startswith(f"{version}-rc") for tag in head_tags): - print(f"HEAD of {branch_name} is already tagged with an RC. Skipping.") - return 0 - - print(f"Tagging and pushing next RC: {next_rc}...") - git.tag(next_rc, "HEAD") - git.push("origin", next_rc) - - commit_sha = git.get_commit_sha("HEAD") - - # Check off the appropriate "Tag RC{N}" task in the checklist - print(f"Checking off Tag RC{next_rc_num} task...") - metadata = {"status": "done", "tag": next_rc, "commit": commit_sha[:8]} - task_name = f"Tag RC{next_rc_num}" - updated_body = update_task_in_body(body, task_name, checked=True, metadata=metadata) - gh.update_issue_body(args.issue, updated_body) - - tag_url = f"{REPO_URL}/releases/tag/{next_rc}" - bcr_search_url = f"https://github.com/bazelbuild/bazel-central-registry/pulls?q=is%3Apr+rules_python+{version}" - comment_body = f"""🚀 **New Release Candidate Tagged!** - -Release Candidate **{next_rc}** has been successfully generated and tagged on branch `{branch_name}`. - -View Tag: [{next_rc}]({tag_url}) -Track BCR Progress: [Search BCR Pull Requests]({bcr_search_url})""" - gh.post_issue_comment(args.issue, comment_body) - print("RC creation completed successfully!") - return 0 - - def cmd_promote_rc(args): """Executes the promote-rc subcommand (Phase 3).""" version = args.version @@ -549,6 +422,12 @@ def create_parser(): required=True, help="The tracking issue number (required).", ) + create_rc_parser.add_argument( + "--remote", + type=str, + required=True, + help="The git remote to push the RC tag to (required).", + ) # Subcommand: promote-rc promote_parser = subparsers.add_parser( diff --git a/tools/private/release/release_issue.py b/tools/private/release/release_issue.py index c170217471..eb7414e39a 100644 --- a/tools/private/release/release_issue.py +++ b/tools/private/release/release_issue.py @@ -128,3 +128,32 @@ def parse_checklist_state(body): } return state + + +def parse_backports(body): + """Parses the ## Backports checklist section.""" + body = body.replace("\r\n", "\n") + match = re.search( + r"## Backports\n(.*?)(?=\n##|\n---|\Z)", body, re.DOTALL | re.IGNORECASE + ) + if not match: + return [] + + section_content = match.group(1) + items = [] + lines = section_content.splitlines() + + for line in lines: + parsed = parse_metadata_line(line) + if parsed: + items.append( + { + "pr_ref": parsed["name"], + "checked": parsed["checked"], + "status": parsed["metadata"].get("status", "PENDING"), + "rc": parsed["metadata"].get("rc"), + "commit": parsed["metadata"].get("commit"), + "metadata": parsed["metadata"], + } + ) + return items From dc0d97765d63829a536879bc5c5ac6526ec4d6b7 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Tue, 30 Jun 2026 23:14:17 -0700 Subject: [PATCH 799/922] chore(release): various fixes to release tool and workflows (#3880) The release tool previously calculated incorrect RC versions or failed when local tags were out of sync with the remote repository. To resolve this: - Query remote tags directly via `git ls-remote` to determine the next RC version. - Fetch upstream tags before determining the next version in `promote-rc` to avoid stale local state. - Remove local checkout and the "already tagged" check in `create-rc` to simplify the tagging process. - Exclude release tool directories from the workflow version marker check. --- .github/workflows/check_version_markers.sh | 3 +- tests/tools/private/release/release_test.py | 72 +++++++++------------ tools/private/release/create_rc.py | 14 ++-- tools/private/release/git.py | 33 ++++++++-- tools/private/release/release.py | 11 ++-- tools/private/release/utils.py | 7 +- 6 files changed, 75 insertions(+), 65 deletions(-) diff --git a/.github/workflows/check_version_markers.sh b/.github/workflows/check_version_markers.sh index b8d35aec9d..15a0a67dc8 100755 --- a/.github/workflows/check_version_markers.sh +++ b/.github/workflows/check_version_markers.sh @@ -20,9 +20,8 @@ grep_exit_code=0 # Exclude CONTRIBUTING.md, RELEASING.md because they document how to use these strings. grep --exclude=CONTRIBUTING.md \ --exclude=RELEASING.md \ - --exclude=release.py \ - --exclude=release_test.py \ --exclude-dir=.* \ + --exclude-dir=release \ VERSION_NEXT_ -r || grep_exit_code=$? if [[ $grep_exit_code -eq 0 ]]; then diff --git a/tests/tools/private/release/release_test.py b/tests/tools/private/release/release_test.py index 636c9b2589..d06b9097a9 100644 --- a/tests/tools/private/release/release_test.py +++ b/tests/tools/private/release/release_test.py @@ -34,7 +34,8 @@ def _mock_git_and_gh(test_case): # Apply safe defaults mock_git.get_current_branch.return_value = None mock_git.get_tags.return_value = [] - mock_git.get_tags_at_head.return_value = [] + mock_git.get_remote_tags.return_value = [] + mock_git.status.return_value = "" mock_git.branch_exists.return_value = False mock_git.tag_exists.return_value = False @@ -588,6 +589,17 @@ def test_get_latest_rc_tag_ignores_v_prefix(self, mock_get_tags): mock_get_tags.return_value = ["v2.0.0-rc0", "2.0.0-rc1"] self.assertEqual(utils.get_latest_rc_tag("2.0.0"), "2.0.0-rc1") + @patch("tools.private.release.git.get_remote_tags") + def test_get_latest_rc_tag_remote_success(self, mock_get_remote_tags): + mock_get_remote_tags.return_value = [ + "2.0.0-rc0", + "2.0.0-rc2", + "2.0.0-rc1", + "2.1.0-rc0", + ] + self.assertEqual(utils.get_latest_rc_tag("2.0.0", remote="origin"), "2.0.0-rc2") + mock_get_remote_tags.assert_called_once_with("origin") + class DetermineNextVersionTest(TempDirTestCase): def setUp(self): @@ -942,8 +954,7 @@ def test_create_rc_success_first_rc(self): - [x] Create Release branch | status=done branch=release/2.0 commit=abcdef12 - [ ] Tag RC0 | status=pending """ - self.mock_git.get_tags.return_value = [] - self.mock_git.get_tags_at_head.return_value = [] + self.mock_git.get_remote_tags.return_value = [] self.mock_git.get_commit_sha.return_value = "1234567890" # Act @@ -954,9 +965,10 @@ def test_create_rc_success_first_rc(self): self.mock_git.fetch.assert_has_calls( [call("my-remote"), call("my-remote", tags=True, force=True)] ) - self.mock_git.checkout.assert_called_once_with("my-remote/release/2.0") - self.mock_git.tag.assert_called_once_with("2.0.0-rc0", "HEAD") + self.mock_git.checkout.assert_not_called() + self.mock_git.tag.assert_called_once_with("2.0.0-rc0", "my-remote/release/2.0") self.mock_git.push.assert_called_once_with("my-remote", "2.0.0-rc0") + self.mock_git.get_commit_sha.assert_called_once_with("my-remote/release/2.0") self.mock_gh.update_issue_body.assert_called_once() call_args = self.mock_gh.update_issue_body.call_args[0] @@ -988,8 +1000,7 @@ def test_create_rc_success_next_rc(self): - [x] Tag RC0 | status=done tag=2.0.0-rc0 commit=abcdef12 - [ ] Tag RC1 | status=pending """ - self.mock_git.get_tags.return_value = ["2.0.0-rc0"] - self.mock_git.get_tags_at_head.return_value = [] + self.mock_git.get_remote_tags.return_value = ["2.0.0-rc0"] self.mock_git.get_commit_sha.return_value = "1234567890" # Act @@ -1000,9 +1011,10 @@ def test_create_rc_success_next_rc(self): self.mock_git.fetch.assert_has_calls( [call("my-remote"), call("my-remote", tags=True, force=True)] ) - self.mock_git.checkout.assert_called_once_with("my-remote/release/2.0") - self.mock_git.tag.assert_called_once_with("2.0.0-rc1", "HEAD") + self.mock_git.checkout.assert_not_called() + self.mock_git.tag.assert_called_once_with("2.0.0-rc1", "my-remote/release/2.0") self.mock_git.push.assert_called_once_with("my-remote", "2.0.0-rc1") + self.mock_git.get_commit_sha.assert_called_once_with("my-remote/release/2.0") self.mock_gh.update_issue_body.assert_called_once() call_args = self.mock_gh.update_issue_body.call_args[0] @@ -1022,28 +1034,6 @@ def test_create_rc_success_next_rc(self): ) self.assertNotIn("🚀", comment_call_args[1]) - def test_create_rc_already_tagged(self): - # Arrange - args = MagicMock(issue=123) - self.mock_gh.get_issue_title.return_value = "Release 2.0.0" - self.mock_gh.get_issue_body.return_value = """ -## Checklist -- [x] Prepare Release | status=done pr=#122 commit=abcdef12 -- [x] Create Release branch | status=done branch=release/2.0 commit=abcdef12 -- [ ] Tag RC0 | status=pending -""" - self.mock_git.get_tags.return_value = [] - self.mock_git.get_tags_at_head.return_value = ["2.0.0-rc0"] - - # Act - result = releaser.cmd_create_rc(args) - - # Assert - self.assertEqual(result, 0) - self.mock_git.tag.assert_not_called() - self.mock_git.push.assert_not_called() - self.mock_gh.update_issue_body.assert_not_called() - class CmdPromoteRcTest(unittest.TestCase): def setUp(self): @@ -1052,7 +1042,7 @@ def setUp(self): def test_promote_rc_success(self): # Arrange args = MagicMock(version="2.0.0", issue=123, dry_run=False) - self.mock_git.get_tags.return_value = ["2.0.0-rc0", "2.0.0-rc1"] + self.mock_git.get_remote_tags.return_value = ["2.0.0-rc0", "2.0.0-rc1"] self.mock_git.get_commit_sha.return_value = "abcdef123456" self.mock_git.tag_exists.return_value = False initial_body = "- [ ] Tag Final" @@ -1088,7 +1078,7 @@ def test_promote_rc_success(self): def test_promote_rc_resolve_issue_success(self): # Arrange args = MagicMock(version="2.0.0", issue=None, dry_run=False) - self.mock_git.get_tags.return_value = ["2.0.0-rc1"] + self.mock_git.get_remote_tags.return_value = ["2.0.0-rc1"] self.mock_git.tag_exists.return_value = False self.mock_gh.get_release_tracking_issue.side_effect = None self.mock_gh.get_release_tracking_issue.return_value = 123 @@ -1124,7 +1114,8 @@ def test_promote_rc_defaults_to_determine_next_version(self): # Arrange args = MagicMock(version=None, issue=123, dry_run=False) self.mock_git.get_current_branch.return_value = "release/2.0" - self.mock_git.get_tags.return_value = ["2.0.0", "2.0.1-rc0"] + self.mock_git.get_tags.return_value = ["2.0.0"] + self.mock_git.get_remote_tags.return_value = ["2.0.1-rc0"] self.mock_git.get_commit_sha.return_value = "12345678" self.mock_git.tag_exists.return_value = False initial_body = "- [ ] Tag Final" @@ -1136,7 +1127,8 @@ def test_promote_rc_defaults_to_determine_next_version(self): # Assert self.assertEqual(result, 0) self.mock_git.get_current_branch.assert_called_once() - self.assertTrue(self.mock_git.get_tags.call_count >= 2) + self.mock_git.get_tags.assert_called_once() + self.mock_git.get_remote_tags.assert_called_once_with("upstream") self.mock_git.checkout.assert_not_called() self.mock_git.get_commit_sha.assert_called_once_with("2.0.1-rc0") @@ -1159,7 +1151,7 @@ def test_promote_rc_defaults_to_determine_next_version(self): def test_promote_rc_dry_run_success(self): # Arrange args = MagicMock(version="2.0.0", issue=123, dry_run=True) - self.mock_git.get_tags.return_value = ["2.0.0-rc0", "2.0.0-rc1"] + self.mock_git.get_remote_tags.return_value = ["2.0.0-rc0", "2.0.0-rc1"] self.mock_git.get_commit_sha.return_value = "abcdef123456" self.mock_git.tag_exists.return_value = False initial_body = "- [ ] Tag Final" @@ -1183,7 +1175,7 @@ def test_promote_rc_dry_run_success(self): def test_promote_rc_tag_already_exists(self): # Arrange args = MagicMock(version="2.0.0", issue=123) - self.mock_git.get_tags.return_value = ["2.0.0-rc1"] + self.mock_git.get_remote_tags.return_value = ["2.0.0-rc1"] self.mock_git.tag_exists.return_value = True # Act @@ -1200,7 +1192,7 @@ def test_promote_rc_tag_already_exists(self): def test_promote_rc_issue_not_found(self): # Arrange args = MagicMock(version="2.0.0", issue=None) - self.mock_git.get_tags.return_value = ["2.0.0-rc1"] + self.mock_git.get_remote_tags.return_value = ["2.0.0-rc1"] self.mock_git.tag_exists.return_value = False self.mock_gh.get_release_tracking_issue.side_effect = NoTrackingIssueError( "Not found" @@ -1220,7 +1212,7 @@ def test_promote_rc_issue_not_found(self): def test_promote_rc_issue_malformed(self): # Arrange args = MagicMock(version="2.0.0", issue=123) - self.mock_git.get_tags.return_value = ["2.0.0-rc1"] + self.mock_git.get_remote_tags.return_value = ["2.0.0-rc1"] self.mock_git.tag_exists.return_value = False self.mock_git.get_commit_sha.return_value = "abcdef123456" initial_body = "malformed body" @@ -1240,7 +1232,7 @@ def test_promote_rc_issue_malformed(self): def test_promote_rc_no_rc_found(self): # Arrange args = MagicMock(version="2.0.0", issue=123) - self.mock_git.get_tags.return_value = [] + self.mock_git.get_remote_tags.return_value = [] # Act result = releaser.cmd_promote_rc(args) diff --git a/tools/private/release/create_rc.py b/tools/private/release/create_rc.py index 9cd1a3b291..5273ca8195 100644 --- a/tools/private/release/create_rc.py +++ b/tools/private/release/create_rc.py @@ -53,7 +53,7 @@ def cmd_create_rc(args): # Determine next RC tag git.fetch(args.remote) git.fetch(args.remote, tags=True, force=True) - latest_rc = get_latest_rc_tag(version) + latest_rc = get_latest_rc_tag(version, remote=args.remote) if not latest_rc: next_rc_num = 0 @@ -79,19 +79,13 @@ def cmd_create_rc(args): ) return 1 - # Verify HEAD is not already tagged - git.checkout(f"{args.remote}/{branch_name}") - head_tags = git.get_tags_at_head() - if any(tag.startswith(f"{version}-rc") for tag in head_tags): - print(f"HEAD of {branch_name} is already tagged with an RC. Skipping.") - return 0 + target_ref = f"{args.remote}/{branch_name}" + commit_sha = git.get_commit_sha(target_ref) print(f"Tagging and pushing next RC: {next_rc}...") - git.tag(next_rc, "HEAD") + git.tag(next_rc, target_ref) git.push(args.remote, next_rc) - commit_sha = git.get_commit_sha("HEAD") - # Check off the appropriate "Tag RC{N}" task in the checklist print(f"Checking off Tag RC{next_rc_num} task...") metadata = {"status": "done", "tag": next_rc, "commit": commit_sha[:8]} diff --git a/tools/private/release/git.py b/tools/private/release/git.py index ce2bd8ca03..f7021b6f5c 100644 --- a/tools/private/release/git.py +++ b/tools/private/release/git.py @@ -123,12 +123,6 @@ def sort_commits_chronologically(shas): return output.splitlines() if output else [] -def get_tags_at_head(): - """Returns a list of tags pointing at the current HEAD commit.""" - output = run_cmd("git", "tag", "--points-at", "HEAD") - return output.splitlines() if output else [] - - def get_current_branch(): """Returns the current git branch name.""" return run_cmd("git", "rev-parse", "--abbrev-ref", "HEAD") @@ -150,3 +144,30 @@ def is_ancestor(ancestor, descendant): return True except subprocess.CalledProcessError: return False + + +def get_remote_tags(remote: str) -> list[str]: + """Returns a list of tags present on the specified remote repository. + + Args: + remote: The name of the git remote to query (e.g., 'origin', 'upstream'). + + Returns: + A list of tag names (strings) found on the remote, excluding peeled tags. + """ + output = run_cmd("git", "ls-remote", "--tags", remote) + tags = [] + for line in output.splitlines(): + if not line: + continue + parts = line.split() + if len(parts) < 2: + continue + ref = parts[1] + if ref.startswith("refs/tags/"): + tag = ref[len("refs/tags/") :] + # Skip peeled tags (e.g. tag^{}) to avoid + # duplicate tag names in the output. + if not tag.endswith("^{}"): + tags.append(tag) + return tags diff --git a/tools/private/release/release.py b/tools/private/release/release.py index 7bc6f9ba05..e1ebf8ec2e 100644 --- a/tools/private/release/release.py +++ b/tools/private/release/release.py @@ -141,8 +141,8 @@ def cmd_process_backports(args): branch_name = f"release/{branch_version}" # Determine next RC tag to write to backport metadata - git.fetch("--tags", "--force") - latest_rc = get_latest_rc_tag(version) + git.fetch("origin", tags=True, force=True) + latest_rc = get_latest_rc_tag(version, remote="origin") if not latest_rc: next_rc_suffix = "rc0" else: @@ -236,13 +236,14 @@ def cmd_process_backports(args): def cmd_promote_rc(args): """Executes the promote-rc subcommand (Phase 3).""" + # Fetch from upstream to ensure we have the latest tags + git.fetch("upstream", tags=True, force=True) + version = args.version if version is None: version = determine_next_version() - # Fetch from upstream to ensure we have the latest tags - git.fetch("upstream", tags=True, force=True) - latest_rc = get_latest_rc_tag(version) + latest_rc = get_latest_rc_tag(version, remote="upstream") if not latest_rc: print(f"Error: No release candidate tags found matching {version}-rc*") return 1 diff --git a/tools/private/release/utils.py b/tools/private/release/utils.py index 6f7383f8ec..2d050022b1 100644 --- a/tools/private/release/utils.py +++ b/tools/private/release/utils.py @@ -67,9 +67,12 @@ def get_latest_version(): return stable_versions[-1] -def get_latest_rc_tag(version): +def get_latest_rc_tag(version, remote=None): """Queries git tags and returns the highest RC tag for the version.""" - tags = git.get_tags() + if remote: + tags = git.get_remote_tags(remote) + else: + tags = git.get_tags() pattern = rf"^{re.escape(version)}-rc\d+$" rc_tags = [tag.strip() for tag in tags if re.match(pattern, tag.strip())] if not rc_tags: From 30a05831ff8cda98b7fe62fb48fdbf70a01faf09 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Wed, 1 Jul 2026 16:28:37 -0700 Subject: [PATCH 800/922] feat(pypi): add pip.dep to declare abstract pypi dependencies (#3850) Introduce the pip.dep tag class to allow modules to declare abstract PyPI dependencies. These declarations ensure the target structure for a PyPI package is created, but provide a no-op implementation that passes analysis time, but fails at execution time. Along the way, create an agent rule for `*.bzl` files to help guide it in creating better bzl files. --- .agents/rules/bzl.md | 10 ++ docs/pypi/download.md | 35 +++++++ news/pip-dep-tag-class.added.md | 4 + python/private/pypi/extension.bzl | 45 +++++++++ python/private/pypi/missing_package.bzl | 9 +- tests/integration/unified_pypi/BUILD.bazel | 13 +++ tests/integration/unified_pypi/MODULE.bazel | 4 + .../unified_pypi/bin_declared_only.py | 2 + tests/integration/unified_pypi_test.py | 25 ++++- tests/pypi/extension/extension_tests.bzl | 93 ++++++++++++++++++- 10 files changed, 236 insertions(+), 4 deletions(-) create mode 100644 .agents/rules/bzl.md create mode 100644 news/pip-dep-tag-class.added.md create mode 100644 tests/integration/unified_pypi/bin_declared_only.py diff --git a/.agents/rules/bzl.md b/.agents/rules/bzl.md new file mode 100644 index 0000000000..10aca72184 --- /dev/null +++ b/.agents/rules/bzl.md @@ -0,0 +1,10 @@ +--- +trigger: glob +description: Starlark / Bazel .bzl file coding style rules +globs: *.bzl +--- + +# Starlark Rules + +* Use triple-quoted strings for multi-line rule doc args. +* Don't use backslash line continuation in rule doc args. diff --git a/docs/pypi/download.md b/docs/pypi/download.md index 161a753645..88c2dd296d 100644 --- a/docs/pypi/download.md +++ b/docs/pypi/download.md @@ -121,6 +121,41 @@ Shared library targets can simply depend on the unified hub (e.g., `@pypi//numpy`), and the dependency will automatically resolve to the correct wheel version from the active hub during the build. +### Declaring Abstract Dependencies (pip.dep) + +:::{versionadded} VERSION_NEXT_FEATURE +Declaring abstract PyPI dependencies via `pip.dep` tags. +::: + +Sometimes a shared library target or a ruleset needs to depend on a PyPI +package (e.g., `@pypi//numpy`), but does not want to force a specific package +version or a concrete `requirements.txt` lock file on its consumers. + +Instead of calling `pip.parse()`, the module can declare its dependency using +the `pip.dep` tag: + +```starlark +pip = use_extension("@rules_python//python/extensions:pip.bzl", "pip") + +# Declare an abstract dependency on 'numpy' and specify extra targets that +# are expected to be available in the package. +pip.dep( + name = "numpy", + extra_targets = ["extra-alias"], +) +``` + +This ensures that the target structure `@pypi//numpy` (and +`@pypi//numpy:extra-alias`) exists in the unified `@pypi` hub repository, so the +declaring module can compile and analyze successfully without needing any local +requirements file. + +The actual concrete implementation and version of the package must be provided +by a downstream module calling `pip.parse`. + +If a downstream module attempts to build a target that depends on an abstract +dependency, but has not provided a concrete implementation for it via any +`pip.parse` call, the build will fail at execution time. As with any repository rule or extension, if you would like to ensure that `pip_parse` is diff --git a/news/pip-dep-tag-class.added.md b/news/pip-dep-tag-class.added.md new file mode 100644 index 0000000000..5306186081 --- /dev/null +++ b/news/pip-dep-tag-class.added.md @@ -0,0 +1,4 @@ +(pypi) Added a `dep` tag class to the `pip` bzlmod extension. This allows +modules to declare abstract PyPI dependencies, ensuring target structures +exist in the unified hub, while allowing other modules to provide the +concrete implementation via `pip.parse`. diff --git a/python/private/pypi/extension.bzl b/python/private/pypi/extension.bzl index 02c5ad0c34..094cac7f3f 100644 --- a/python/private/pypi/extension.bzl +++ b/python/private/pypi/extension.bzl @@ -424,6 +424,15 @@ You cannot use both the additive_build_content and additive_build_content_file a pip_attr = pip_attr, ) + # dict[str package, dict[str, None] extra_targets] + declared_deps = {} + for mod in module_ctx.modules: + for dep_attr in mod.tags.dep: + name = normalize_name(dep_attr.name) + targets = declared_deps.setdefault(name, {}) + for target in dep_attr.extra_targets: + targets[target] = None + # Keeps track of all the hub's whl repos across the different versions. # dict[hub, dict[whl, dict[version, str pip]]] # Where hub, whl, and pip are the repo names @@ -448,6 +457,7 @@ You cannot use both the additive_build_content and additive_build_content_file a return struct( config = config, + declared_deps = declared_deps, default_hub = config.default_hub or renamed_default_hub, exposed_packages = exposed_packages, extra_aliases = extra_aliases, @@ -492,6 +502,15 @@ def _create_unified_hub_repo(mods): if hub_name not in extra_aliases[qual_alias]: extra_aliases[qual_alias].append(hub_name) + for norm_pkg, extra_targets in mods.declared_deps.items(): + if norm_pkg not in packages: + packages[norm_pkg] = [] + + for target_name in extra_targets: + qual_alias = "%s:%s" % (norm_pkg, target_name) + if qual_alias not in extra_aliases: + extra_aliases[qual_alias] = [] + unified_hub_repo( name = "pypi", default_hub = mods.default_hub or (hubs[0] if hubs else ""), @@ -1015,6 +1034,31 @@ Apply any overrides (e.g. patches) to a given Python distribution defined by other tags in this extension.""", ) +_dep_tag = tag_class( + attrs = { + "extra_targets": attr.string_list( + doc = """\ +A list of extra target names in the package that are expected to be available. +See {obj}`pip.parse.extra_hub_aliases`. +""", + default = [], + ), + "name": attr.string( + doc = "The name of a pypi package. Note that the name is normalized.", + mandatory = True, + ), + }, + doc = """\ +Declare an abstract PyPI dependency to ensure its target structure exists in the unified hub. + +This is useful for targets or rules that need to depend on a package (e.g., `@pypi//numpy`) +but do not want to force a specific version or concrete requirements lock file on their +consumers. The concrete version and implementation must be provided by downstreams calling +`pip.parse`. If they are not, the target will still be defined, but it will result in an +execution-phase error when built. +""", +) + pypi = module_extension( environ = ["RULES_PYTHON_PYPI_HUB_RESERVED"], doc = """\ @@ -1065,6 +1109,7 @@ terms used in this extension. ::: """, ), + "dep": _dep_tag, "override": _override_tag, "parse": tag_class( attrs = _pip_parse_ext_attrs(), diff --git a/python/private/pypi/missing_package.bzl b/python/private/pypi/missing_package.bzl index c59f754b10..46823e813b 100644 --- a/python/private/pypi/missing_package.bzl +++ b/python/private/pypi/missing_package.bzl @@ -6,12 +6,19 @@ load("//python/private:reexports.bzl", "BuiltinPyInfo") def _missing_package_error_impl(ctx): out = ctx.actions.declare_file(ctx.label.name + ".error") + if ctx.attr.hub_name: + hub_clause = ' when building under PyPI hub "{hub}". Try adding it to the requirements of this hub (e.g. requirements_lock or requirements_by_platform in pip.parse)'.format( + hub = ctx.attr.hub_name, + ) + else: + hub_clause = ' because no default PyPI hub was configured. Try designating a default hub via pip.default(default_hub = "...") or select a hub using --@rules_python//python/config_settings:venv' + # Register an action that fails when Bazel attempts to stage/build this file ctx.actions.run_shell( outputs = [out], command = "echo 'ERROR: PyPI package \"{pkg}\" is not available{hub_clause}.' >&2 && exit 1".format( pkg = ctx.attr.package_name, - hub_clause = (' when building under PyPI hub "%s"' % ctx.attr.hub_name) if ctx.attr.hub_name else " because no PyPI hub or default hub is requested", + hub_clause = hub_clause, ), ) diff --git a/tests/integration/unified_pypi/BUILD.bazel b/tests/integration/unified_pypi/BUILD.bazel index 8a37d4b153..c0b9905fa6 100644 --- a/tests/integration/unified_pypi/BUILD.bazel +++ b/tests/integration/unified_pypi/BUILD.bazel @@ -46,3 +46,16 @@ py_binary( }, deps = ["@pypi//six"], ) + +py_binary( + name = "bin_declared_only", + srcs = ["bin_declared_only.py"], + deps = ["@pypi//declared_only_pkg"], +) + +py_binary( + name = "bin_declared_only_alias", + srcs = ["bin_declared_only.py"], + main = "bin_declared_only.py", + deps = ["@pypi//declared_only_pkg:declared-only-alias"], +) diff --git a/tests/integration/unified_pypi/MODULE.bazel b/tests/integration/unified_pypi/MODULE.bazel index 0d0f44f61c..6a4b87e015 100644 --- a/tests/integration/unified_pypi/MODULE.bazel +++ b/tests/integration/unified_pypi/MODULE.bazel @@ -45,4 +45,8 @@ pip.parse( use_repo(pip, "pypi_b") pip.default(default_hub = "pypi_b") +pip.dep( + name = "declared-only-pkg", + extra_targets = ["declared-only-alias"], +) use_repo(pip, "pypi") diff --git a/tests/integration/unified_pypi/bin_declared_only.py b/tests/integration/unified_pypi/bin_declared_only.py new file mode 100644 index 0000000000..0b03607e77 --- /dev/null +++ b/tests/integration/unified_pypi/bin_declared_only.py @@ -0,0 +1,2 @@ +# Dummy file for integration test +print("declared_only") diff --git a/tests/integration/unified_pypi_test.py b/tests/integration/unified_pypi_test.py index 707ab13444..a19a9fcfbe 100644 --- a/tests/integration/unified_pypi_test.py +++ b/tests/integration/unified_pypi_test.py @@ -30,7 +30,7 @@ def test_disjoint_package_cquery_succeeds_but_build_fails(self): ) self.assert_result_matches( result, - 'ERROR: PyPI package "six" is not available when building under PyPI hub "pypi_a".', + 'ERROR: PyPI package "six" is not available when building under PyPI hub "pypi_a"\\. Try adding it to the requirements of this hub', ) def test_sibling_extra_alias_cquery_succeeds_but_build_fails(self): @@ -43,7 +43,7 @@ def test_sibling_extra_alias_cquery_succeeds_but_build_fails(self): ) self.assert_result_matches( result, - 'ERROR: PyPI package "colorama:my_colorama" is not available when building under PyPI hub "pypi_b".', + 'ERROR: PyPI package "colorama:my_colorama" is not available when building under PyPI hub "pypi_b"\\. Try adding it to the requirements of this hub', ) @contextlib.contextmanager @@ -74,6 +74,27 @@ def test_invalid_default_hub_fails_evaluation(self): "default_hub 'invalid_hub' is not a defined PyPI hub", ) + def test_unimplemented_declared_dep_fails_build(self): + # Even though cquery succeeds: + self.run_bazel("cquery", "//:bin_declared_only") + + # Build must fail because the package is not implemented by any concrete hub + result = self.run_bazel("build", "//:bin_declared_only", check=False) + self.assertNotEqual(result.exit_code, 0) + self.assert_result_matches( + result, + 'ERROR: PyPI package "declared_only_pkg" is not available when building under PyPI hub "pypi_b"\\. Try adding it to the requirements of this hub', + ) + + def test_unimplemented_declared_dep_alias_fails_build(self): + # Build must fail for alias too + result = self.run_bazel("build", "//:bin_declared_only_alias", check=False) + self.assertNotEqual(result.exit_code, 0) + self.assert_result_matches( + result, + 'ERROR: PyPI package "declared_only_pkg:declared-only-alias" is not available when building under PyPI hub "pypi_b"\\. Try adding it to the requirements of this hub', + ) + if __name__ == "__main__": unittest.main() diff --git a/tests/pypi/extension/extension_tests.bzl b/tests/pypi/extension/extension_tests.bzl index bc4c0bcb5b..c2e0c8b60f 100644 --- a/tests/pypi/extension/extension_tests.bzl +++ b/tests/pypi/extension/extension_tests.bzl @@ -90,7 +90,13 @@ _default_tags_default = [ }.items() ] -def _mod(*, name, default = _default_tags_default, parse = [], override = [], whl_mods = [], is_root = True): +def _dep(*, name, extra_targets = []): + return struct( + name = name, + extra_targets = extra_targets, + ) + +def _mod(*, name, default = _default_tags_default, parse = [], override = [], whl_mods = [], dep = [], is_root = True): return struct( name = name, tags = struct( @@ -98,6 +104,7 @@ def _mod(*, name, default = _default_tags_default, parse = [], override = [], wh override = override, whl_mods = whl_mods, default = default, + dep = dep, ), is_root = is_root, ) @@ -106,6 +113,7 @@ def _parse_modules(env, **kwargs): return env.expect.that_struct( parse_modules(**kwargs), attrs = dict( + declared_deps = subjects.dict, default_hub = subjects.str, exposed_packages = subjects.dict, hub_group_map = subjects.dict, @@ -435,6 +443,89 @@ def _test_default_hub_precedence(env): _tests.append(_test_default_hub_precedence) +def _test_extension_dep(env): + pypi = _parse_modules( + env, + module_ctx = _pypi_mock_mctx( + _mod( + name = "my_module", + dep = [ + _dep( + name = "declared-pkg", + extra_targets = ["declared-alias"], + ), + ], + ), + os_name = "linux", + arch_name = "x86_64", + ), + available_interpreters = {}, + minor_mapping = {}, + ) + + pypi.declared_deps().contains_exactly({"declared_pkg": {"declared-alias": None}}) + pypi.exposed_packages().contains_exactly({}) + pypi.hub_group_map().contains_exactly({}) + pypi.hub_whl_map().contains_exactly({}) + pypi.whl_libraries().contains_exactly({}) + pypi.whl_mods().contains_exactly({}) + +_tests.append(_test_extension_dep) + +def _test_extension_dep_coexists_with_concrete_hub(env): + pypi = _parse_modules( + env, + module_ctx = _pypi_mock_mctx( + _mod( + name = "my_module", + parse = [ + _parse( + hub_name = "pypi_a", + python_version = "3.15", + simpleapi_skip = ["simple"], + requirements_lock = "requirements.txt", + ), + ], + dep = [ + _dep( + name = "simple", + extra_targets = ["extra-target"], + ), + ], + ), + os_name = "linux", + arch_name = "x86_64", + ), + available_interpreters = { + "python_3_15_host": "unit_test_interpreter_target", + }, + minor_mapping = {"3.15": "3.15.19"}, + ) + + pypi.declared_deps().contains_exactly({"simple": {"extra-target": None}}) + pypi.exposed_packages().contains_exactly({"pypi_a": ["simple"]}) + pypi.hub_group_map().contains_exactly({"pypi_a": {}}) + pypi.hub_whl_map().contains_exactly({"pypi_a": { + "simple": { + "pypi_a_315_simple": [ + whl_config_setting( + version = "3.15", + ), + ], + }, + }}) + pypi.whl_libraries().contains_exactly({ + "pypi_a_315_simple": { + "config_load": "@pypi_a//:config.bzl", + "dep_template": "@pypi_a//{name}:{target}", + "python_interpreter_target": "unit_test_interpreter_target", + "requirement": "simple==0.0.1 --hash=sha256:deadbeef --hash=sha256:deadbaaf", + }, + }) + pypi.whl_mods().contains_exactly({}) + +_tests.append(_test_extension_dep_coexists_with_concrete_hub) + def extension_test_suite(name): """Create the test suite. From cc799f17f5bf5ce8bb97f82f8bb649961138378d Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Wed, 1 Jul 2026 21:06:29 -0700 Subject: [PATCH 801/922] chore(release): refactor process-backports and various release tool fixes (#3881) This PR refactors the `process-backports` subcommand, improves its dry-run validation, and fixes a bug in `create-rc`. ### Why the changes are made - The `process-backports` logic was previously embedded in the main `release.py` script, making it difficult to maintain and extend. - `process-backports` lacked a robust dry-run validation to safely verify backport applicability without leaving the workspace dirty. - `process-backports` could fail with checkout ambiguity if the release branch existed on multiple remotes. - `create-rc` crashed with a fatal subscripting error when trying to access `BackportTask` objects as dictionaries after their recent refactoring. - `create-rc` tracking issue comments lacked a direct link to the BCR entry for the new version. --- tests/tools/private/release/release_test.py | 338 +++++++++++++++++++- tools/private/release/BUILD.bazel | 1 + tools/private/release/create_rc.py | 10 +- tools/private/release/gh.py | 45 ++- tools/private/release/git.py | 56 +++- tools/private/release/process_backports.py | 242 ++++++++++++++ tools/private/release/release.py | 145 +-------- tools/private/release/release_issue.py | 56 +++- 8 files changed, 724 insertions(+), 169 deletions(-) create mode 100644 tools/private/release/process_backports.py diff --git a/tests/tools/private/release/release_test.py b/tests/tools/private/release/release_test.py index d06b9097a9..1307f5e78b 100644 --- a/tests/tools/private/release/release_test.py +++ b/tests/tools/private/release/release_test.py @@ -1,3 +1,4 @@ +import datetime import os import pathlib import shutil @@ -5,7 +6,7 @@ import unittest from unittest.mock import MagicMock, call, patch -from tools.private.release import changelog_news, release as releaser, utils +from tools.private.release import changelog_news, git, release as releaser, utils from tools.private.release.gh import MultipleTrackingIssuesError, NoTrackingIssueError @@ -20,12 +21,14 @@ def _mock_git_and_gh(test_case): patch("tools.private.release.prepare.git", new=mock_git).start() patch("tools.private.release.create_release_branch.git", new=mock_git).start() patch("tools.private.release.create_rc.git", new=mock_git).start() + patch("tools.private.release.process_backports.git", new=mock_git).start() patch("tools.private.release.utils.git", new=mock_git).start() patch("tools.private.release.release.gh", new=mock_gh).start() patch("tools.private.release.prepare.gh", new=mock_gh).start() patch("tools.private.release.create_release_branch.gh", new=mock_gh).start() patch("tools.private.release.create_rc.gh", new=mock_gh).start() + patch("tools.private.release.process_backports.gh", new=mock_gh).start() mock_gh.MultipleTrackingIssuesError = MultipleTrackingIssuesError mock_gh.NoTrackingIssueError = NoTrackingIssueError @@ -984,7 +987,19 @@ def test_create_rc_success_first_rc(self): comment_call_args[1], ) self.assertIn( - "- Trigger Release Workflow: [Release Workflow](https://github.com/bazel-contrib/rules_python/actions/workflows/release.yml)", + "- [Github Release 2.0.0-rc0](https://github.com/bazel-contrib/rules_python/releases/tag/2.0.0-rc0)", + comment_call_args[1], + ) + self.assertIn( + "- BCR Entry: [rules_python@2.0.0](https://registry.bazel.build/modules/rules_python/2.0.0)", + comment_call_args[1], + ) + self.assertIn( + "- [BCR PRs](https://github.com/bazelbuild/bazel-central-registry/pulls?q=is%3Apr+rules_python+2.0.0)", + comment_call_args[1], + ) + self.assertIn( + "- [Release workflow status](https://github.com/bazel-contrib/rules_python/actions/workflows/release.yml)", comment_call_args[1], ) self.assertNotIn("🚀", comment_call_args[1]) @@ -1029,11 +1044,68 @@ def test_create_rc_success_next_rc(self): comment_call_args[1], ) self.assertIn( - "- Trigger Release Workflow: [Release Workflow](https://github.com/bazel-contrib/rules_python/actions/workflows/release.yml)", + "- [Github Release 2.0.0-rc1](https://github.com/bazel-contrib/rules_python/releases/tag/2.0.0-rc1)", + comment_call_args[1], + ) + self.assertIn( + "- BCR Entry: [rules_python@2.0.0](https://registry.bazel.build/modules/rules_python/2.0.0)", + comment_call_args[1], + ) + self.assertIn( + "- [BCR PRs](https://github.com/bazelbuild/bazel-central-registry/pulls?q=is%3Apr+rules_python+2.0.0)", + comment_call_args[1], + ) + self.assertIn( + "- [Release workflow status](https://github.com/bazel-contrib/rules_python/actions/workflows/release.yml)", comment_call_args[1], ) self.assertNotIn("🚀", comment_call_args[1]) + def test_create_rc_gating_on_backports(self): + # Arrange + args = MagicMock(issue=123, remote="my-remote") + self.mock_gh.get_issue_title.return_value = "Release 2.0.0" + self.mock_gh.get_issue_body.return_value = """ +## Checklist +- [x] Prepare Release | status=done pr=#122 commit=abcdef12 +- [x] Create Release branch | status=done branch=release/2.0 commit=abcdef12 +- [ ] Tag RC0 | status=pending + +## Backports +- [ ] #124 | status=pending +""" + # Act + result = releaser.cmd_create_rc(args) + + # Assert + self.assertEqual(result, 1) + self.mock_git.tag.assert_not_called() + self.mock_git.push.assert_not_called() + + def test_create_rc_with_finished_backports(self): + # Arrange + args = MagicMock(issue=123, remote="my-remote") + self.mock_gh.get_issue_title.return_value = "Release 2.0.0" + self.mock_gh.get_issue_body.return_value = """ +## Checklist +- [x] Prepare Release | status=done pr=#122 commit=abcdef12 +- [x] Create Release branch | status=done branch=release/2.0 commit=abcdef12 +- [ ] Tag RC0 | status=pending + +## Backports +- [x] #124 | status=done rc=rc0 commit=abcdef12 +""" + self.mock_git.get_remote_tags.return_value = [] + self.mock_git.get_commit_sha.return_value = "1234567890" + + # Act + result = releaser.cmd_create_rc(args) + + # Assert + self.assertEqual(result, 0) + self.mock_git.tag.assert_called_once_with("2.0.0-rc0", "my-remote/release/2.0") + self.mock_git.push.assert_called_once_with("my-remote", "2.0.0-rc0") + class CmdPromoteRcTest(unittest.TestCase): def setUp(self): @@ -1384,5 +1456,265 @@ def test_create_release_branch_already_exists_non_ff(self): self.mock_gh.update_issue_body.assert_not_called() +class CmdProcessBackportsTest(unittest.TestCase): + def setUp(self): + _mock_git_and_gh(self) + self.mock_changelog_news = patch( + "tools.private.release.process_backports.changelog_news" + ).start() + self.addCleanup(patch.stopall) + + def test_process_backports_no_pending(self): + args = MagicMock(issue=123, remote="origin", dry_run=False) + self.mock_gh.get_issue_body.return_value = "No backports here" + + result = releaser.cmd_process_backports(args) + + self.assertEqual(result, 0) + self.mock_gh.get_issue_body.assert_called_once_with(123) + self.mock_git.fetch.assert_not_called() + + @patch("tools.private.release.process_backports.datetime") + def test_process_backports_success(self, mock_datetime): + mock_datetime.date.today.return_value = datetime.date(2026, 7, 1) + args = MagicMock(issue=123, remote="origin", dry_run=False) + self.mock_gh.get_issue_title.return_value = "Release 2.0.0" + self.mock_gh.get_issue_body.return_value = """ +## Checklist +- [ ] Prepare Release +- [ ] Create Release branch + +## Backports +- [ ] #124 | status=pending +""" + self.mock_git.get_remote_tags.return_value = [] + + def mock_resolve(items): + for item in items: + if item.pr_ref == "#124": + item.commit = "abcdef12" + item.status = "done" + return items + + self.mock_gh.get_merge_commits_for_prs.side_effect = mock_resolve + + self.mock_git.sort_commits_chronologically.return_value = ["abcdef12"] + self.mock_git.get_commit_sha.return_value = "12345678" + self.mock_git.get_commit_message.return_value = 'Cherry-pick "fix bug"' + + result = releaser.cmd_process_backports(args) + + self.assertEqual(result, 0) + self.mock_git.fetch.assert_has_calls( + [call("origin", tags=True, force=True), call("origin")] + ) + self.mock_git.checkout.assert_called_once_with( + "release/2.0", track_remote="origin" + ) + self.mock_git.cherry_pick.assert_called_once_with("abcdef12") + self.mock_changelog_news.update_changelog.assert_called_once_with( + "2.0.0", "2026-07-01" + ) + self.mock_git.add.assert_called_once_with("CHANGELOG.md", "news/") + self.mock_git.commit.assert_called_once_with( + 'Cherry-pick "fix bug"\n\nWork towards #123', amend=True + ) + self.mock_git.push.assert_called_once_with("origin", "release/2.0") + + self.mock_gh.update_issue_body.assert_called_once() + call_args = self.mock_gh.update_issue_body.call_args[0] + self.assertEqual(call_args[0], 123) + self.assertIn("- [x] #124 | status=done rc=rc0 commit=12345678", call_args[1]) + + @patch("tools.private.release.process_backports.datetime") + def test_process_backports_dry_run(self, mock_datetime): + mock_datetime.date.today.return_value = datetime.date(2026, 7, 1) + args = MagicMock(issue=123, remote="origin", dry_run=True) + self.mock_gh.get_issue_title.return_value = "Release 2.0.0" + self.mock_gh.get_issue_body.return_value = """ +## Checklist +- [ ] Prepare Release +- [ ] Create Release branch + +## Backports +- [ ] #124 | status=pending +""" + self.mock_git.get_remote_tags.return_value = [] + + def mock_resolve(items): + for item in items: + if item.pr_ref == "#124": + item.commit = "abcdef12" + item.status = "done" + return items + + self.mock_gh.get_merge_commits_for_prs.side_effect = mock_resolve + + self.mock_git.sort_commits_chronologically.return_value = ["abcdef12"] + self.mock_git.get_commit_sha.return_value = "12345678" + self.mock_git.get_commit_message.return_value = 'Cherry-pick "fix bug"' + + result = releaser.cmd_process_backports(args) + + self.assertEqual(result, 0) + self.mock_git.fetch.assert_has_calls( + [call("origin", tags=True, force=True), call("origin")] + ) + self.mock_git.checkout.assert_called_once_with( + "release/2.0", track_remote="origin" + ) + self.mock_git.cherry_pick.assert_called_once_with("abcdef12") + self.mock_changelog_news.update_changelog.assert_called_once_with( + "2.0.0", "2026-07-01" + ) + self.mock_git.commit.assert_called_once_with( + 'Cherry-pick "fix bug"\n\nWork towards #123', amend=True + ) + self.mock_git.reset_hard.assert_called_once_with("12345678") + self.mock_git.push.assert_not_called() + self.mock_gh.update_issue_body.assert_not_called() + + def test_process_backports_ignored_and_failed_states(self): + args = MagicMock(issue=123, remote="origin", dry_run=False) + self.mock_gh.get_issue_title.return_value = "Release 2.0.0" + self.mock_gh.get_issue_body.return_value = """ +## Checklist +- [ ] Prepare Release +- [ ] Create Release branch + +## Backports +- [ ] #124 | status=pending +- [ ] #125 | status=pending +- [ ] #126 | status=pending +""" + self.mock_git.get_remote_tags.return_value = [] + + def mock_resolve(items): + for item in items: + if item.pr_ref == "#124": + item.status = "open-pr" + elif item.pr_ref == "#125": + item.status = "draft-pr" + elif item.pr_ref == "#126": + item.status = "error-closed-pr" + return items + + self.mock_gh.get_merge_commits_for_prs.side_effect = mock_resolve + + result = releaser.cmd_process_backports(args) + + self.assertEqual(result, 1) + self.mock_gh.update_issue_body.assert_called_once() + call_args = self.mock_gh.update_issue_body.call_args[0] + self.assertEqual(call_args[0], 123) + self.assertIn("- [ ] #126 | status=error-closed-pr", call_args[1]) + self.assertNotIn("status=open-pr", call_args[1]) + self.assertNotIn("status=draft-pr", call_args[1]) + self.mock_git.checkout.assert_not_called() + self.mock_git.cherry_pick.assert_not_called() + + def test_process_backports_ignored_error_status(self): + args = MagicMock(issue=123, remote="origin", dry_run=False) + self.mock_gh.get_issue_title.return_value = "Release 2.0.0" + self.mock_gh.get_issue_body.return_value = """ +## Checklist +- [ ] Prepare Release +- [ ] Create Release branch + +## Backports +- [ ] #124 | status=error-merge-conflict +- [ ] #125 | status=error-some-other-error +""" + self.mock_git.get_remote_tags.return_value = [] + self.mock_gh.get_merge_commits_for_prs.return_value = [] + + result = releaser.cmd_process_backports(args) + + self.assertEqual(result, 0) + self.mock_gh.get_merge_commits_for_prs.assert_not_called() + self.mock_git.checkout.assert_not_called() + + @patch("tools.private.release.process_backports.datetime") + def test_process_backports_cherry_pick_failed(self, mock_datetime): + mock_datetime.date.today.return_value = datetime.date(2026, 7, 1) + args = MagicMock(issue=123, remote="origin", dry_run=False) + self.mock_gh.get_issue_title.return_value = "Release 2.0.0" + self.mock_gh.get_issue_body.return_value = """ +## Checklist +- [ ] Prepare Release +- [ ] Create Release branch + +## Backports +- [ ] #124 | status=pending +""" + self.mock_git.get_remote_tags.return_value = [] + + def mock_resolve(items): + for item in items: + if item.pr_ref == "#124": + item.commit = "abcdef12" + item.status = "done" + return items + + self.mock_gh.get_merge_commits_for_prs.side_effect = mock_resolve + + self.mock_git.sort_commits_chronologically.return_value = ["abcdef12"] + self.mock_git.cherry_pick.side_effect = Exception("Cherry-pick conflict") + + result = releaser.cmd_process_backports(args) + + self.assertEqual(result, 1) + self.mock_git.checkout.assert_called_once_with( + "release/2.0", track_remote="origin" + ) + self.mock_git.cherry_pick.assert_called_once_with("abcdef12") + self.mock_git.cherry_pick_abort.assert_called_once() + + self.mock_gh.update_issue_body.assert_called_once() + call_args = self.mock_gh.update_issue_body.call_args[0] + self.assertEqual(call_args[0], 123) + self.assertIn("- [ ] #124 | status=error-merge-conflict", call_args[1]) + + self.mock_git.commit.assert_not_called() + self.mock_git.push.assert_not_called() + + +class GitCheckoutTest(unittest.TestCase): + @patch("tools.private.release.git.run_cmd") + def test_checkout_simple(self, mock_run_cmd): + git.checkout("my-branch") + mock_run_cmd.assert_called_once_with( + "git", "checkout", "my-branch", capture_output=False + ) + + @patch("tools.private.release.git.branch_exists") + @patch("tools.private.release.git.run_cmd") + def test_checkout_track_remote_new_branch(self, mock_run_cmd, mock_branch_exists): + mock_branch_exists.return_value = False + + git.checkout("my-branch", track_remote="origin") + + mock_branch_exists.assert_called_once_with("my-branch") + mock_run_cmd.assert_called_once_with( + "git", "checkout", "--track", "origin/my-branch", capture_output=False + ) + + @patch("tools.private.release.git.reset_hard") + @patch("tools.private.release.git.branch_exists") + @patch("tools.private.release.git.run_cmd") + def test_checkout_track_remote_existing_branch( + self, mock_run_cmd, mock_branch_exists, mock_reset_hard + ): + mock_branch_exists.return_value = True + + git.checkout("my-branch", track_remote="origin") + + mock_branch_exists.assert_called_once_with("my-branch") + mock_run_cmd.assert_called_once_with( + "git", "checkout", "my-branch", capture_output=False + ) + mock_reset_hard.assert_called_once_with("origin/my-branch") + + if __name__ == "__main__": unittest.main() diff --git a/tools/private/release/BUILD.bazel b/tools/private/release/BUILD.bazel index ad2e1b4cdc..1ae52cf37a 100644 --- a/tools/private/release/BUILD.bazel +++ b/tools/private/release/BUILD.bazel @@ -15,6 +15,7 @@ py_binary( "gh.py", "git.py", "prepare.py", + "process_backports.py", "release.py", "release_issue.py", "shell.py", diff --git a/tools/private/release/create_rc.py b/tools/private/release/create_rc.py index 5273ca8195..a4593323aa 100644 --- a/tools/private/release/create_rc.py +++ b/tools/private/release/create_rc.py @@ -30,7 +30,7 @@ def cmd_create_rc(args): # Gating: RC tagging is blocked if any backport is unchecked OR does not have status=done backports = parse_backports(body) conflicting_or_pending = [ - b for b in backports if not b["checked"] or b["status"] != "done" + b for b in backports if not b.checked or b.status != "done" ] if conflicting_or_pending: print( @@ -94,15 +94,17 @@ def cmd_create_rc(args): gh.update_issue_body(args.issue, updated_body) tag_url = f"{REPO_URL}/releases/tag/{next_rc}" + bcr_entry_url = f"https://registry.bazel.build/modules/rules_python/{version}" bcr_search_url = f"https://github.com/bazelbuild/bazel-central-registry/pulls?q=is%3Apr+rules_python+{version}" release_workflow_url = f"{REPO_URL}/actions/workflows/release.yml" comment_body = f"""**New Release Candidate Tagged!** 🐍🌿 Release Candidate **{next_rc}** has been successfully generated and tagged on branch `{branch_name}`. -- View Tag: [{next_rc}]({tag_url}) -- Track BCR Progress: [Search BCR Pull Requests]({bcr_search_url}) -- Trigger Release Workflow: [Release Workflow]({release_workflow_url})""" +- [Github Release {next_rc}]({tag_url}) +- BCR Entry: [rules_python@{version}]({bcr_entry_url}) +- [BCR PRs]({bcr_search_url}) +- [Release workflow status]({release_workflow_url})""" gh.post_issue_comment(args.issue, comment_body) print("RC creation completed successfully!") return 0 diff --git a/tools/private/release/gh.py b/tools/private/release/gh.py index da86415650..cec20ca2d6 100644 --- a/tools/private/release/gh.py +++ b/tools/private/release/gh.py @@ -4,6 +4,7 @@ import os import tempfile +from tools.private.release.release_issue import BackportTask from tools.private.release.shell import run_cmd _REPO = "bazel-contrib/rules_python" @@ -179,13 +180,13 @@ def get_open_pr(branch_name): def get_pr_info(pr_num): - """Gets information about a PR, including state, merge commit, and body.""" + """Gets information about a PR, including state, merge commit, body, and draft status.""" output = run_cmd( "gh", "pr", "view", str(pr_num), - "--json=state,mergeCommit,body", + "--json=state,mergeCommit,body,isDraft", ) return json.loads(output) if output else {} @@ -202,30 +203,44 @@ def post_issue_comment(issue_num, comment_body): ) -def resolve_backport_commits(pending_items): +def get_merge_commits_for_prs(pending_items: list[BackportTask]) -> list[BackportTask]: """Resolves PR references in pending backports to their merge commit SHAs. - Marks unmerged PRs or resolution failures with status='unmerged-pr'. + Updates item.status based on PR state if it cannot be resolved. """ resolved_items = [] for item in pending_items: - pr_num = item["pr_ref"].lstrip("#") + pr_num = item.pr_ref.lstrip("#") print(f"Resolving PR #{pr_num} to merge commit...") try: pr_info = get_pr_info(pr_num) - if not pr_info or pr_info.get("state") != "MERGED": - state = pr_info.get("state", "UNKNOWN") - print(f"PR #{pr_num} is not merged (state: {state}). Gating.") - item["status"] = "unmerged-pr" + if not pr_info: + print(f"PR #{pr_num} not found. Gating.") + item.status = "error-not-found" else: - merge_commit = pr_info.get("mergeCommit") - if merge_commit and "oid" in merge_commit: - item["commit"] = merge_commit["oid"] + state = pr_info.get("state") + is_draft = pr_info.get("isDraft", False) + if state == "OPEN" or is_draft: + print( + f"PR #{pr_num} is open or draft (state: {state}, draft: {is_draft}). Ignoring." + ) + item.status = "open-pr" if not is_draft else "draft-pr" + elif state == "CLOSED": + print(f"PR #{pr_num} is closed but not merged. Gating.") + item.status = "error-closed-pr" + elif state == "MERGED": + merge_commit = pr_info.get("mergeCommit") + if merge_commit and "oid" in merge_commit: + item.commit = merge_commit["oid"] + item.status = "resolved" + else: + print(f"PR #{pr_num} has no merge commit SHA. Gating.") + item.status = "error-no-merge-commit" else: - print(f"PR #{pr_num} has no merge commit SHA. Gating.") - item["status"] = "unmerged-pr" + print(f"PR #{pr_num} has unknown state: {state}. Gating.") + item.status = "error-unknown" except Exception as e: print(f"Error resolving PR #{pr_num}: {e}. Gating.") - item["status"] = "unmerged-pr" + item.status = "error-resolution-failed" resolved_items.append(item) return resolved_items diff --git a/tools/private/release/git.py b/tools/private/release/git.py index f7021b6f5c..795710fd8c 100644 --- a/tools/private/release/git.py +++ b/tools/private/release/git.py @@ -11,12 +11,34 @@ def get_tags(): return output.splitlines() if output else [] -def checkout(ref, create_branch=False): - """Checks out a git reference (tag, branch, or commit).""" +def checkout( + ref: str, create_branch: bool = False, track_remote: str | None = None +) -> None: + """Checks out a git reference (tag, branch, or commit). + + Args: + ref: The git reference (tag, branch, or commit) to checkout. + create_branch: If True, creates the branch before checking it out. + track_remote: If specified, checks out the branch tracking this remote's + corresponding branch. + """ + cmd = ["git", "checkout"] if create_branch: - run_cmd("git", "checkout", "-b", ref, capture_output=False) + cmd.append("-b") + + should_reset_hard = False + if track_remote: + if branch_exists(ref): + cmd.append(ref) + should_reset_hard = True + else: + cmd.extend(["--track", f"{track_remote}/{ref}"]) else: - run_cmd("git", "checkout", ref, capture_output=False) + cmd.append(ref) + run_cmd(*cmd, capture_output=False) + + if should_reset_hard: + reset_hard(f"{track_remote}/{ref}") def add(*files): @@ -75,8 +97,12 @@ def tag(tag_name, commit_ref): run_cmd("git", "tag", tag_name, commit_ref, capture_output=False) -def cherry_pick(sha): - """Cherry-picks a commit using -x to append the original commit info.""" +def cherry_pick(sha: str) -> None: + """Cherry-picks a commit. + + Args: + sha: The commit SHA to cherry-pick. + """ run_cmd("git", "cherry-pick", "-x", sha, capture_output=False) @@ -85,6 +111,15 @@ def cherry_pick_abort(): run_cmd("git", "cherry-pick", "--abort", capture_output=False) +def reset_hard(ref: str = "HEAD") -> None: + """Resets the index and working tree to a specific reference. + + Args: + ref: The git reference to reset to. Defaults to 'HEAD'. + """ + run_cmd("git", "reset", "--hard", ref, capture_output=False) + + def status(): """Returns the output of git status --porcelain.""" return run_cmd("git", "status", "--porcelain") @@ -99,6 +134,15 @@ def get_commit_sha(ref="HEAD", short=False): return run_cmd(*cmd) +def get_commit_message(ref: str = "HEAD") -> str: + """Returns the commit message of a given reference. + + Args: + ref: The git reference to get the message from. Defaults to 'HEAD'. + """ + return run_cmd("git", "log", "-1", "--format=%B", ref) + + def branch_exists(branch_name): """Returns True if a local branch exists.""" try: diff --git a/tools/private/release/process_backports.py b/tools/private/release/process_backports.py new file mode 100644 index 0000000000..373e1848ec --- /dev/null +++ b/tools/private/release/process_backports.py @@ -0,0 +1,242 @@ +"""Subcommand to process pending backports.""" + +import datetime +from typing import Any + +from tools.private.release import changelog_news, gh, git +from tools.private.release.release_issue import ( + RELEASE_TITLE_RE, + parse_backports, + update_task_in_body, +) +from tools.private.release.utils import get_latest_rc_tag + + +def _process_pr_commit_infos( + pr_commit_infos, body, issue, dry_run +) -> tuple[list[str], dict[str, Any], list[str], list[str], str]: + shas = [] + sha_to_item = {} + failed_prs = [] + ignored_prs = [] + for item in pr_commit_infos: + if item.commit: + sha = item.commit + sha_to_item[sha] = item + shas.append(sha) + elif item.status in ("open-pr", "draft-pr"): + print(f"PR {item.pr_ref} is open or draft. Ignoring.") + ignored_prs.append(item.pr_ref) + else: + failed_prs.append(item.pr_ref) + status_to_set = item.status or "error-unmerged-pr" + if dry_run: + print( + f"[DRY RUN] Would update tracking issue checklist for unresolved PR {item.pr_ref} to status={status_to_set}" + ) + else: + print( + f"Updating tracking issue checklist for unresolved PR {item.pr_ref}..." + ) + try: + body = update_task_in_body( + body, + item.pr_ref, + checked=False, + metadata={"status": status_to_set}, + ) + gh.update_issue_body(issue, body) + except Exception as e: + print( + f"ERROR: Failed to update tracking issue for unresolved PR {item.pr_ref}: {e}" + ) + return shas, sha_to_item, failed_prs, ignored_prs, body + + +def _cherry_pick_and_update_prs( + sorted_shas, + sha_to_item, + body, + issue, + remote, + dry_run, + version, + branch_name, + next_rc_suffix, +) -> tuple[list[str], str]: + failed_prs = [] + for sha in sorted_shas: + item = sha_to_item[sha] + print(f"Cherry-picking {item.pr_ref} / {sha}...") + try: + git.cherry_pick(sha) + + # Perform news processing (merging news/ files into the changelog) + print(f"Merging news fragments into changelog for PR {item.pr_ref}...") + release_date = datetime.date.today().strftime("%Y-%m-%d") + changelog_news.update_changelog(version, release_date) + + # Stage changelog changes and news/ deletions + git.add("CHANGELOG.md", "news/") + + # Amend cherry-pick commit to include news merging and deletions, + # and reference the release tracking issue. + print(f"Amending cherry-pick commit for PR {item.pr_ref}...") + current_msg = git.get_commit_message("HEAD") + new_msg = f"{current_msg.strip()}\n\nWork towards #{issue}" + git.commit(new_msg, amend=True) + + if not dry_run: + # Push amended commit + git.push(remote, branch_name) + + new_sha = git.get_commit_sha("HEAD", short=True) + metadata = {"status": "done", "rc": next_rc_suffix, "commit": new_sha} + print(f"Updating tracking issue checklist for PR {item.pr_ref}...") + try: + body = update_task_in_body( + body, item.pr_ref, checked=True, metadata=metadata + ) + gh.update_issue_body(issue, body) + except Exception as e: + print( + f"ERROR: Failed to update tracking issue for PR {item.pr_ref}: {e}" + ) + print(f"Success: backported {item.pr_ref} / {sha} to {branch_name}") + else: + print( + f"[DRY RUN] Success: {item.pr_ref} / {sha} can be backported without error." + ) + print( + f"[DRY RUN] Would update tracking issue checklist for PR {item.pr_ref} to status=done" + ) + except Exception as e: + print(f"ERROR: Conflict or error on {sha}: {e}. Aborting.") + try: + git.cherry_pick_abort() + except Exception: + pass + failed_prs.append(item.pr_ref) + + if dry_run: + print( + f"[DRY RUN] Would update tracking issue checklist for failed PR {item.pr_ref} to status=error-merge-conflict" + ) + else: + print( + f"Updating tracking issue checklist for failed PR {item.pr_ref}..." + ) + try: + body = update_task_in_body( + body, + item.pr_ref, + checked=False, + metadata={"status": "error-merge-conflict"}, + ) + gh.update_issue_body(issue, body) + print( + f"Updated back port of {item.pr_ref} to status=error-merge-conflict (unchecked)" + ) + except Exception as e: + print( + f"ERROR: Failed to update tracking issue for failed PR {item.pr_ref}: {e}" + ) + return failed_prs, body + + +def cmd_process_backports(args): + """Executes the process-backports subcommand.""" + body = gh.get_issue_body(args.issue) + items = parse_backports(body) + + pending_items = [ + item + for item in items + if not item.checked and not item.status.startswith("error-") + ] + + if not pending_items: + print("No pending backports found.") + return 0 + + print(f"Found {len(pending_items)} pending backports to process.") + + # Determine branch name from issue title + issue_title = gh.get_issue_title(args.issue) + version_match = RELEASE_TITLE_RE.search(issue_title) + if not version_match: + print(f"Error: Could not parse version from issue title: {issue_title}") + return 1 + + version = version_match.group(1) + branch_version = ".".join(version.split(".")[:2]) + branch_name = f"release/{branch_version}" + + # Determine next RC tag to write to backport metadata + git.fetch(args.remote, tags=True, force=True) + latest_rc = get_latest_rc_tag(version, remote=args.remote) + if not latest_rc: + next_rc_suffix = "rc0" + else: + rc_num = int(latest_rc.split("-rc")[-1]) + next_rc_suffix = f"rc{rc_num + 1}" + + # Resolve PRs to merge commits using gh helper. + pr_commit_infos = gh.get_merge_commits_for_prs(pending_items) + + shas, sha_to_item, failed_prs, ignored_prs, body = _process_pr_commit_infos( + pr_commit_infos, body, args.issue, args.dry_run + ) + + if not shas: + print("No valid merge commits to process.") + if failed_prs: + print("Failed PRs:") + for pr in failed_prs: + print(f"- {pr}") + return 1 + return 0 + + # Verify workspace is clean before proceeding + if git.status(): + print( + "ERROR: Git workspace is dirty. Please commit or stash changes before running backports." + ) + return 1 + + # Sort chronologically using git helper + sorted_shas = git.sort_commits_chronologically(shas) + + git.fetch(args.remote) + git.checkout(branch_name, track_remote=args.remote) + start_sha = git.get_commit_sha("HEAD") + + try: + new_failed_prs, body = _cherry_pick_and_update_prs( + sorted_shas, + sha_to_item, + body, + args.issue, + args.remote, + args.dry_run, + version, + branch_name, + next_rc_suffix, + ) + failed_prs.extend(new_failed_prs) + finally: + if args.dry_run: + print(f"[DRY RUN] Resetting branch {branch_name} to {start_sha}") + git.reset_hard(start_sha) + + if failed_prs: + print("ERROR: One or more cherry-picks/resolutions failed:") + for pr in failed_prs: + print(f"- {pr}") + return 1 + + if args.dry_run: + print("Dry run completed successfully. No errors found.") + else: + print("All backports successfully processed!") + return 0 diff --git a/tools/private/release/release.py b/tools/private/release/release.py index e1ebf8ec2e..a6c5b3f88b 100644 --- a/tools/private/release/release.py +++ b/tools/private/release/release.py @@ -1,19 +1,17 @@ """A tool to perform release steps.""" import argparse -import datetime import os import pathlib import re import sys -from tools.private.release import changelog_news, gh, git +from tools.private.release import gh, git from tools.private.release.create_rc import cmd_create_rc from tools.private.release.create_release_branch import cmd_create_release_branch from tools.private.release.prepare import cmd_prepare +from tools.private.release.process_backports import cmd_process_backports from tools.private.release.release_issue import ( - RELEASE_TITLE_RE, - parse_backports, update_task_in_body, ) from tools.private.release.utils import ( @@ -31,11 +29,6 @@ def _semver_type(value): return value -# ============================================================================== -# Checklist Parser and Formatter (Using new | key=value syntax) -# ============================================================================== - - # ============================================================================== # Subcommand Execution Functions # ============================================================================== @@ -112,128 +105,6 @@ def cmd_complete_prepare(args): return 0 -def cmd_process_backports(args): - """Executes the process-backports subcommand.""" - body = gh.get_issue_body(args.issue) - items = parse_backports(body) - - pending_items = [ - item - for item in items - if not item["checked"] and item["status"] != "merge-conflict" - ] - - if not pending_items: - print("No pending backports found.") - return 0 - - print(f"Found {len(pending_items)} pending backports to process.") - - # Determine branch name from issue title - issue_title = gh.get_issue_title(args.issue) - version_match = RELEASE_TITLE_RE.search(issue_title) - if not version_match: - print(f"Error: Could not parse version from issue title: {issue_title}") - return 1 - - version = version_match.group(1) - branch_version = ".".join(version.split(".")[:2]) - branch_name = f"release/{branch_version}" - - # Determine next RC tag to write to backport metadata - git.fetch("origin", tags=True, force=True) - latest_rc = get_latest_rc_tag(version, remote="origin") - if not latest_rc: - next_rc_suffix = "rc0" - else: - rc_num = int(latest_rc.split("-rc")[-1]) - next_rc_suffix = f"rc{rc_num + 1}" - - # Resolve PRs to merge commits using gh helper - resolved_items = gh.resolve_backport_commits(pending_items) - - shas = [] - sha_to_item = {} - any_failed = False - for item in resolved_items: - if item.get("commit"): - sha = item["commit"] - sha_to_item[sha] = item - shas.append(sha) - else: - any_failed = True - body = update_task_in_body( - body, - item["pr_ref"], - checked=False, - metadata={"status": item.get("status", "failed")}, - ) - gh.update_issue_body(args.issue, body) - - if not shas: - print("No valid merge commits to process.") - if any_failed: - return 1 - return 0 - - # Sort chronologically using git helper - sorted_shas = git.sort_commits_chronologically(shas) - - git.fetch("origin") - git.checkout(branch_name) - - for sha in sorted_shas: - item = sha_to_item[sha] - print(f"Cherry-picking {sha} (PR {item['pr_ref']})...") - try: - git.cherry_pick(sha) - - # Perform news processing (merging news/ files into the changelog) - print(f"Merging news fragments into changelog for PR {item['pr_ref']}...") - release_date = datetime.date.today().strftime("%Y-%m-%d") - changelog_news.update_changelog(version, release_date) - - # Stage changelog changes and news/ deletions - git.add("CHANGELOG.md", "news/") - - # Amend cherry-pick commit to include news merging and deletions - print(f"Amending cherry-pick commit for PR {item['pr_ref']}...") - git.commit("", amend=True, no_edit=True) - - # Push amended commit - git.push("origin", branch_name) - - new_sha = git.get_commit_sha("HEAD", short=True) - metadata = {"status": "done", "rc": next_rc_suffix, "commit": new_sha} - body = update_task_in_body( - body, item["pr_ref"], checked=True, metadata=metadata - ) - gh.update_issue_body(args.issue, body) - print(f"Applied: SUCCESS {new_sha}") - except Exception as e: - print(f"Conflict or error on {sha}: {e}. Aborting.") - try: - git.cherry_pick_abort() - except Exception: - pass - any_failed = True - - body = update_task_in_body( - body, - item["pr_ref"], - checked=False, - metadata={"status": "merge-conflict"}, - ) - gh.update_issue_body(args.issue, body) - print("Updated backport item to status=merge-conflict (unchecked)") - - if any_failed: - print("One or more cherry-picks/resolutions failed.") - return 1 - print("All backports successfully processed!") - return 0 - - def cmd_promote_rc(args): """Executes the promote-rc subcommand (Phase 3).""" # Fetch from upstream to ensure we have the latest tags @@ -411,6 +282,18 @@ def create_parser(): required=True, help="The tracking issue number (required).", ) + process_backports_parser.add_argument( + "--remote", + type=str, + required=True, + help="The git remote to push changes to (required).", + ) + process_backports_parser.add_argument( + "--dry-run", + action=argparse.BooleanOptionalAction, + default=True, + help="Perform a dry run (default: True). Use --no-dry-run to actually execute.", + ) # Subcommand: create-rc create_rc_parser = subparsers.add_parser( diff --git a/tools/private/release/release_issue.py b/tools/private/release/release_issue.py index eb7414e39a..adc0d88086 100644 --- a/tools/private/release/release_issue.py +++ b/tools/private/release/release_issue.py @@ -1,7 +1,43 @@ -"""Helper functions for managing release tracking issues and checklists.""" - import re + +class BackportTask: + """Represents a backport task from the tracking issue checklist.""" + + def __init__( + self, + pr_ref: str, + checked: bool, + status: str, + rc: str | None = None, + commit: str | None = None, + metadata: dict[str, str] | None = None, + ): + """Initializes a BackportTask. + + Args: + pr_ref: The PR reference (e.g. '#123'). + checked: Whether the checklist item is checked. + status: The status of the backport (e.g. 'pending', 'done', + 'error-merge-conflict'). + rc: The release candidate version this PR was backported to. + commit: The cherry-pick commit SHA. + metadata: Raw metadata parsed from the checklist line. + """ + self.pr_ref = pr_ref + self.checked = checked + self.status = status + self.rc = rc + self.commit = commit + self.metadata = metadata or {} + + def __repr__(self): + return ( + f"BackportTask(pr_ref={self.pr_ref!r}, checked={self.checked!r}, " + f"status={self.status!r}, rc={self.rc!r}, commit={self.commit!r})" + ) + + RELEASE_TITLE_RE = re.compile(r"Release (\d+\.\d+\.\d+)", re.IGNORECASE) @@ -147,13 +183,13 @@ def parse_backports(body): parsed = parse_metadata_line(line) if parsed: items.append( - { - "pr_ref": parsed["name"], - "checked": parsed["checked"], - "status": parsed["metadata"].get("status", "PENDING"), - "rc": parsed["metadata"].get("rc"), - "commit": parsed["metadata"].get("commit"), - "metadata": parsed["metadata"], - } + BackportTask( + pr_ref=parsed["name"], + checked=parsed["checked"], + status=parsed["metadata"].get("status", "pending"), + rc=parsed["metadata"].get("rc"), + commit=parsed["metadata"].get("commit"), + metadata=parsed["metadata"], + ) ) return items From 839deafd3cd87798ea142ec0c1012675f9cbbfb3 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Thu, 2 Jul 2026 00:30:15 -0700 Subject: [PATCH 802/922] chore(release): factor code, make testable, clarify workflow names (#3882) This refactors the release tool code to be more testable and broken up into smaller files. Workflows are also renamed to better match what they do and the command they run. --- ...rged.yml => release_complete_prepare.yaml} | 2 +- ...generate_rc.yml => release_create_rc.yaml} | 2 +- ...yml => release_create_release_branch.yaml} | 2 +- ...epare_release.yml => release_prepare.yaml} | 2 +- ...rts.yml => release_process_backports.yaml} | 2 +- ...promote_rc.yml => release_promote_rc.yaml} | 2 +- tests/tools/private/release/release_test.py | 172 +++--- tools/private/release/BUILD.bazel | 5 + tools/private/release/__init__.py | 1 + tools/private/release/complete_prepare.py | 81 +++ tools/private/release/create_rc.py | 216 ++++--- .../private/release/create_release_branch.py | 171 ++++-- tools/private/release/create_release_issue.py | 59 ++ .../private/release/determine_next_version.py | 30 + tools/private/release/gh.py | 547 +++++++++++------- tools/private/release/git.py | 543 ++++++++++------- tools/private/release/prepare.py | 321 +++++----- tools/private/release/process_backports.py | 470 ++++++++------- tools/private/release/promote_rc.py | 143 +++++ tools/private/release/release.py | 360 +----------- tools/private/release/release_issue.py | 124 ++-- tools/private/release/shell.py | 8 +- tools/private/release/utils.py | 16 +- 23 files changed, 1910 insertions(+), 1369 deletions(-) rename .github/workflows/{on_prepare_release_pr_merged.yml => release_complete_prepare.yaml} (96%) rename .github/workflows/{generate_rc.yml => release_create_rc.yaml} (97%) rename .github/workflows/{cut_release_branch.yml => release_create_release_branch.yaml} (96%) rename .github/workflows/{prepare_release.yml => release_prepare.yaml} (97%) rename .github/workflows/{process_backports.yml => release_process_backports.yaml} (96%) rename .github/workflows/{promote_rc.yml => release_promote_rc.yaml} (96%) create mode 100644 tools/private/release/__init__.py create mode 100644 tools/private/release/complete_prepare.py create mode 100644 tools/private/release/create_release_issue.py create mode 100644 tools/private/release/determine_next_version.py create mode 100644 tools/private/release/promote_rc.py diff --git a/.github/workflows/on_prepare_release_pr_merged.yml b/.github/workflows/release_complete_prepare.yaml similarity index 96% rename from .github/workflows/on_prepare_release_pr_merged.yml rename to .github/workflows/release_complete_prepare.yaml index 8ad4d1055c..6d8bc8fd03 100644 --- a/.github/workflows/on_prepare_release_pr_merged.yml +++ b/.github/workflows/release_complete_prepare.yaml @@ -1,4 +1,4 @@ -name: On PR Merged (Release Prepared) +name: "Release: Complete Prepare" on: pull_request: diff --git a/.github/workflows/generate_rc.yml b/.github/workflows/release_create_rc.yaml similarity index 97% rename from .github/workflows/generate_rc.yml rename to .github/workflows/release_create_rc.yaml index 19a80bb1de..2e22ed1413 100644 --- a/.github/workflows/generate_rc.yml +++ b/.github/workflows/release_create_rc.yaml @@ -1,4 +1,4 @@ -name: Generate RC Tag +name: "Release: Create RC" on: workflow_dispatch: diff --git a/.github/workflows/cut_release_branch.yml b/.github/workflows/release_create_release_branch.yaml similarity index 96% rename from .github/workflows/cut_release_branch.yml rename to .github/workflows/release_create_release_branch.yaml index f054f4cbc8..ea030a7085 100644 --- a/.github/workflows/cut_release_branch.yml +++ b/.github/workflows/release_create_release_branch.yaml @@ -1,4 +1,4 @@ -name: Cut Release Branch +name: "Release: Create Release Branch" on: issues: diff --git a/.github/workflows/prepare_release.yml b/.github/workflows/release_prepare.yaml similarity index 97% rename from .github/workflows/prepare_release.yml rename to .github/workflows/release_prepare.yaml index b080f121eb..5c9a0b8f49 100644 --- a/.github/workflows/prepare_release.yml +++ b/.github/workflows/release_prepare.yaml @@ -1,4 +1,4 @@ -name: Prepare Release +name: "Release: Prepare" on: workflow_dispatch: diff --git a/.github/workflows/process_backports.yml b/.github/workflows/release_process_backports.yaml similarity index 96% rename from .github/workflows/process_backports.yml rename to .github/workflows/release_process_backports.yaml index 4d8055d3c5..ac6ea99d80 100644 --- a/.github/workflows/process_backports.yml +++ b/.github/workflows/release_process_backports.yaml @@ -1,4 +1,4 @@ -name: Process Backports +name: "Release: Process Backports" on: workflow_dispatch: diff --git a/.github/workflows/promote_rc.yml b/.github/workflows/release_promote_rc.yaml similarity index 96% rename from .github/workflows/promote_rc.yml rename to .github/workflows/release_promote_rc.yaml index d5e697f6d7..b668a15338 100644 --- a/.github/workflows/promote_rc.yml +++ b/.github/workflows/release_promote_rc.yaml @@ -1,4 +1,4 @@ -name: Promote RC to Final Release +name: "Release: Promote RC" on: workflow_dispatch: diff --git a/tests/tools/private/release/release_test.py b/tests/tools/private/release/release_test.py index 1307f5e78b..96bb65280f 100644 --- a/tests/tools/private/release/release_test.py +++ b/tests/tools/private/release/release_test.py @@ -6,8 +6,17 @@ import unittest from unittest.mock import MagicMock, call, patch -from tools.private.release import changelog_news, git, release as releaser, utils -from tools.private.release.gh import MultipleTrackingIssuesError, NoTrackingIssueError +from tools.private.release import changelog_news, release as releaser, utils +from tools.private.release.create_rc import CreateRc +from tools.private.release.create_release_branch import CreateReleaseBranch +from tools.private.release.gh import ( + MultipleTrackingIssuesError, + NoTrackingIssueError, +) +from tools.private.release.git import Git +from tools.private.release.prepare import Prepare +from tools.private.release.process_backports import ProcessBackports +from tools.private.release.promote_rc import PromoteRc def _mock_git_and_gh(test_case): @@ -16,19 +25,9 @@ def _mock_git_and_gh(test_case): test_case.mock_git = mock_git test_case.mock_gh = mock_gh - # Patch bindings in modules that import them at module level - patch("tools.private.release.release.git", new=mock_git).start() - patch("tools.private.release.prepare.git", new=mock_git).start() - patch("tools.private.release.create_release_branch.git", new=mock_git).start() - patch("tools.private.release.create_rc.git", new=mock_git).start() - patch("tools.private.release.process_backports.git", new=mock_git).start() - patch("tools.private.release.utils.git", new=mock_git).start() - - patch("tools.private.release.release.gh", new=mock_gh).start() - patch("tools.private.release.prepare.gh", new=mock_gh).start() - patch("tools.private.release.create_release_branch.gh", new=mock_gh).start() - patch("tools.private.release.create_rc.gh", new=mock_gh).start() - patch("tools.private.release.process_backports.gh", new=mock_gh).start() + # Mock Git inside utils.py since it instantiates it locally + patch("tools.private.release.utils.Git", return_value=mock_git).start() + mock_gh.MultipleTrackingIssuesError = MultipleTrackingIssuesError mock_gh.NoTrackingIssueError = NoTrackingIssueError @@ -528,12 +527,12 @@ def test_invalid_version(self): class GetLatestVersionTest(unittest.TestCase): - @patch("tools.private.release.git.get_tags") + @patch("tools.private.release.git.Git.get_tags") def test_get_latest_version_success(self, mock_get_tags): mock_get_tags.return_value = ["0.1.0", "1.0.0", "0.2.0"] self.assertEqual(utils.get_latest_version(), "1.0.0") - @patch("tools.private.release.git.get_tags") + @patch("tools.private.release.git.Git.get_tags") def test_get_latest_version_rc_is_latest(self, mock_get_tags): mock_get_tags.return_value = ["0.1.0", "1.0.0", "1.1.0rc0"] with self.assertRaisesRegex( @@ -541,7 +540,7 @@ def test_get_latest_version_rc_is_latest(self, mock_get_tags): ): utils.get_latest_version() - @patch("tools.private.release.git.get_tags") + @patch("tools.private.release.git.Git.get_tags") def test_get_latest_version_no_tags(self, mock_get_tags): mock_get_tags.return_value = [] with self.assertRaisesRegex( @@ -549,7 +548,7 @@ def test_get_latest_version_no_tags(self, mock_get_tags): ): utils.get_latest_version() - @patch("tools.private.release.git.get_tags") + @patch("tools.private.release.git.Git.get_tags") def test_get_latest_version_no_matching_tags(self, mock_get_tags): mock_get_tags.return_value = ["v1.0", "latest"] with self.assertRaisesRegex( @@ -557,7 +556,7 @@ def test_get_latest_version_no_matching_tags(self, mock_get_tags): ): utils.get_latest_version() - @patch("tools.private.release.git.get_tags") + @patch("tools.private.release.git.Git.get_tags") def test_get_latest_version_only_rc_tags(self, mock_get_tags): mock_get_tags.return_value = ["1.0.0rc0", "1.1.0rc0"] with self.assertRaisesRegex( @@ -567,17 +566,17 @@ def test_get_latest_version_only_rc_tags(self, mock_get_tags): class GetLatestRcTagTest(unittest.TestCase): - @patch("tools.private.release.git.get_tags") + @patch("tools.private.release.git.Git.get_tags") def test_get_latest_rc_tag_no_tags(self, mock_get_tags): mock_get_tags.return_value = [] self.assertIsNone(utils.get_latest_rc_tag("2.0.0")) - @patch("tools.private.release.git.get_tags") + @patch("tools.private.release.git.Git.get_tags") def test_get_latest_rc_tag_no_matching_tags(self, mock_get_tags): mock_get_tags.return_value = ["1.0.0", "2.0.0", "v2.0.0-rc0", "2.1.0-rc0"] self.assertIsNone(utils.get_latest_rc_tag("2.0.0")) - @patch("tools.private.release.git.get_tags") + @patch("tools.private.release.git.Git.get_tags") def test_get_latest_rc_tag_success(self, mock_get_tags): mock_get_tags.return_value = [ "2.0.0-rc0", @@ -587,12 +586,12 @@ def test_get_latest_rc_tag_success(self, mock_get_tags): ] self.assertEqual(utils.get_latest_rc_tag("2.0.0"), "2.0.0-rc2") - @patch("tools.private.release.git.get_tags") + @patch("tools.private.release.git.Git.get_tags") def test_get_latest_rc_tag_ignores_v_prefix(self, mock_get_tags): mock_get_tags.return_value = ["v2.0.0-rc0", "2.0.0-rc1"] self.assertEqual(utils.get_latest_rc_tag("2.0.0"), "2.0.0-rc1") - @patch("tools.private.release.git.get_remote_tags") + @patch("tools.private.release.git.Git.get_remote_tags") def test_get_latest_rc_tag_remote_success(self, mock_get_remote_tags): mock_get_remote_tags.return_value = [ "2.0.0-rc0", @@ -611,7 +610,7 @@ def setUp(self): "tools.private.release.utils.get_latest_version" ).start() self.mock_get_current_branch = patch( - "tools.private.release.git.get_current_branch" + "tools.private.release.git.Git.get_current_branch" ).start() self.mock_get_current_branch.return_value = "main" self.addCleanup(patch.stopall) @@ -657,8 +656,8 @@ def test_both_markers(self): self.assertEqual(next_version, "1.3.0") - @patch("tools.private.release.git.get_current_branch") - @patch("tools.private.release.git.get_tags") + @patch("tools.private.release.git.Git.get_current_branch") + @patch("tools.private.release.git.Git.get_tags") def test_determine_next_version_on_release_branch_with_existing_tags( self, mock_get_tags, mock_get_branch ): @@ -669,8 +668,8 @@ def test_determine_next_version_on_release_branch_with_existing_tags( self.assertEqual(next_version, "0.37.2") - @patch("tools.private.release.git.get_current_branch") - @patch("tools.private.release.git.get_tags") + @patch("tools.private.release.git.Git.get_current_branch") + @patch("tools.private.release.git.Git.get_tags") def test_determine_next_version_on_release_branch_no_tags( self, mock_get_tags, mock_get_branch ): @@ -681,8 +680,8 @@ def test_determine_next_version_on_release_branch_no_tags( self.assertEqual(next_version, "0.38.0") - @patch("tools.private.release.git.get_current_branch") - @patch("tools.private.release.git.get_tags") + @patch("tools.private.release.git.Git.get_current_branch") + @patch("tools.private.release.git.Git.get_tags") def test_determine_next_version_on_release_branch_with_active_rc( self, mock_get_tags, mock_get_branch ): @@ -695,8 +694,8 @@ def test_determine_next_version_on_release_branch_with_active_rc( # Should target 0.37.0, not 0.37.1 self.assertEqual(next_version, "0.37.0") - @patch("tools.private.release.git.get_current_branch") - @patch("tools.private.release.git.get_tags") + @patch("tools.private.release.git.Git.get_current_branch") + @patch("tools.private.release.git.Git.get_tags") def test_determine_next_version_on_release_branch_with_stable_and_active_patch_rc( self, mock_get_tags, mock_get_branch ): @@ -709,7 +708,7 @@ def test_determine_next_version_on_release_branch_with_stable_and_active_patch_r # Should target 0.37.1, not 0.37.2 self.assertEqual(next_version, "0.37.1") - @patch("tools.private.release.git.get_current_branch") + @patch("tools.private.release.git.Git.get_current_branch") def test_determine_next_version_on_main_branch_fallback(self, mock_get_branch): mock_get_branch.return_value = "main" # Should fallback to default behavior (which uses mock_get_latest_version from setUp) @@ -739,7 +738,7 @@ def test_prepare_success_existing_issue(self, mock_replace, mock_changelog): self.mock_gh.get_issue_body.return_value = "- [ ] Prepare Release" # Act - result = releaser.cmd_prepare(args) + result = Prepare(args, self.mock_git, self.mock_gh).run() # Assert self.assertEqual(result, 0) @@ -768,7 +767,7 @@ def test_prepare_success_create_issue(self, mock_replace, mock_changelog): self.mock_gh.get_issue_body.return_value = "- [ ] Prepare Release" # Act - result = releaser.cmd_prepare(args) + result = Prepare(args, self.mock_git, self.mock_gh).run() # Assert self.assertEqual(result, 0) @@ -791,7 +790,7 @@ def test_prepare_ambiguous_issue(self, mock_replace, mock_changelog): ) # Act - result = releaser.cmd_prepare(args) + result = Prepare(args, self.mock_git, self.mock_gh).run() # Assert self.assertEqual(result, 1) @@ -810,7 +809,7 @@ def test_prepare_dry_run(self, mock_replace, mock_changelog): self.mock_gh.get_release_tracking_issue.return_value = 123 # Act - result = releaser.cmd_prepare(args) + result = Prepare(args, self.mock_git, self.mock_gh).run() # Assert self.assertEqual(result, 0) @@ -841,7 +840,7 @@ def test_prepare_use_associated_pr_from_tracking_issue( ) # Act - result = releaser.cmd_prepare(args) + result = Prepare(args, self.mock_git, self.mock_gh).run() # Assert self.assertEqual(result, 0) @@ -871,7 +870,7 @@ def test_prepare_create_pr_when_none_associated(self, mock_replace, mock_changel self.mock_gh.create_pr.return_value = "https://github.com/foo/bar/pull/789" # Act - result = releaser.cmd_prepare(args) + result = Prepare(args, self.mock_git, self.mock_gh).run() # Assert self.assertEqual(result, 0) @@ -902,7 +901,7 @@ def test_prepare_reuse_existing_pr(self, mock_replace, mock_changelog): self.mock_gh.get_issue_body.return_value = "- [ ] Prepare Release" # Act - result = releaser.cmd_prepare(args) + result = Prepare(args, self.mock_git, self.mock_gh).run() # Assert self.assertEqual(result, 0) @@ -933,7 +932,7 @@ def test_prepare_dry_run_no_issue(self, mock_replace, mock_changelog): ) # Act - result = releaser.cmd_prepare(args) + result = Prepare(args, self.mock_git, self.mock_gh).run() # Assert self.assertEqual(result, 0) @@ -961,7 +960,7 @@ def test_create_rc_success_first_rc(self): self.mock_git.get_commit_sha.return_value = "1234567890" # Act - result = releaser.cmd_create_rc(args) + result = CreateRc(args, self.mock_git, self.mock_gh).run() # Assert self.assertEqual(result, 0) @@ -1019,7 +1018,7 @@ def test_create_rc_success_next_rc(self): self.mock_git.get_commit_sha.return_value = "1234567890" # Act - result = releaser.cmd_create_rc(args) + result = CreateRc(args, self.mock_git, self.mock_gh).run() # Assert self.assertEqual(result, 0) @@ -1075,7 +1074,7 @@ def test_create_rc_gating_on_backports(self): - [ ] #124 | status=pending """ # Act - result = releaser.cmd_create_rc(args) + result = CreateRc(args, self.mock_git, self.mock_gh).run() # Assert self.assertEqual(result, 1) @@ -1099,7 +1098,7 @@ def test_create_rc_with_finished_backports(self): self.mock_git.get_commit_sha.return_value = "1234567890" # Act - result = releaser.cmd_create_rc(args) + result = CreateRc(args, self.mock_git, self.mock_gh).run() # Assert self.assertEqual(result, 0) @@ -1121,7 +1120,7 @@ def test_promote_rc_success(self): self.mock_gh.get_issue_body.return_value = initial_body # Act - result = releaser.cmd_promote_rc(args) + result = PromoteRc(args, self.mock_git, self.mock_gh).run() # Assert self.assertEqual(result, 0) @@ -1159,7 +1158,7 @@ def test_promote_rc_resolve_issue_success(self): self.mock_gh.get_issue_body.return_value = initial_body # Act - result = releaser.cmd_promote_rc(args) + result = PromoteRc(args, self.mock_git, self.mock_gh).run() # Assert self.assertEqual(result, 0) @@ -1194,7 +1193,7 @@ def test_promote_rc_defaults_to_determine_next_version(self): self.mock_gh.get_issue_body.return_value = initial_body # Act - result = releaser.cmd_promote_rc(args) + result = PromoteRc(args, self.mock_git, self.mock_gh).run() # Assert self.assertEqual(result, 0) @@ -1230,7 +1229,7 @@ def test_promote_rc_dry_run_success(self): self.mock_gh.get_issue_body.return_value = initial_body # Act - result = releaser.cmd_promote_rc(args) + result = PromoteRc(args, self.mock_git, self.mock_gh).run() # Assert self.assertEqual(result, 0) @@ -1251,7 +1250,7 @@ def test_promote_rc_tag_already_exists(self): self.mock_git.tag_exists.return_value = True # Act - result = releaser.cmd_promote_rc(args) + result = PromoteRc(args, self.mock_git, self.mock_gh).run() # Assert self.assertEqual(result, 1) @@ -1271,7 +1270,7 @@ def test_promote_rc_issue_not_found(self): ) # Act - result = releaser.cmd_promote_rc(args) + result = PromoteRc(args, self.mock_git, self.mock_gh).run() # Assert self.assertEqual(result, 1) @@ -1291,7 +1290,7 @@ def test_promote_rc_issue_malformed(self): self.mock_gh.get_issue_body.return_value = initial_body # Act - result = releaser.cmd_promote_rc(args) + result = PromoteRc(args, self.mock_git, self.mock_gh).run() # Assert self.assertEqual(result, 1) @@ -1307,7 +1306,7 @@ def test_promote_rc_no_rc_found(self): self.mock_git.get_remote_tags.return_value = [] # Act - result = releaser.cmd_promote_rc(args) + result = PromoteRc(args, self.mock_git, self.mock_gh).run() # Assert self.assertEqual(result, 1) @@ -1333,7 +1332,7 @@ def test_create_release_branch_success(self): self.mock_git.remote_branch_exists.return_value = False # Act - result = releaser.cmd_create_release_branch(args) + result = CreateReleaseBranch(args, self.mock_git, self.mock_gh).run() # Assert self.assertEqual(result, 0) @@ -1362,7 +1361,7 @@ def test_create_release_branch_prepare_not_done(self): - [ ] Create Release branch | status=pending """ # Act - result = releaser.cmd_create_release_branch(args) + result = CreateReleaseBranch(args, self.mock_git, self.mock_gh).run() # Assert self.assertEqual(result, 1) @@ -1380,7 +1379,7 @@ def test_create_release_branch_already_checked(self): - [x] Create Release branch | status=done branch=release/2.0 commit=abcdef12 """ # Act - result = releaser.cmd_create_release_branch(args) + result = CreateReleaseBranch(args, self.mock_git, self.mock_gh).run() # Assert self.assertEqual(result, 0) @@ -1401,7 +1400,7 @@ def test_create_release_branch_already_exists_same_commit(self): self.mock_git.get_commit_sha.return_value = "abcdef12" # Act - result = releaser.cmd_create_release_branch(args) + result = CreateReleaseBranch(args, self.mock_git, self.mock_gh).run() # Assert self.assertEqual(result, 0) @@ -1423,7 +1422,7 @@ def test_create_release_branch_already_exists_fast_forward(self): self.mock_git.is_ancestor.return_value = True # Act - result = releaser.cmd_create_release_branch(args) + result = CreateReleaseBranch(args, self.mock_git, self.mock_gh).run() # Assert self.assertEqual(result, 0) @@ -1447,7 +1446,7 @@ def test_create_release_branch_already_exists_non_ff(self): self.mock_git.is_ancestor.return_value = False # Act - result = releaser.cmd_create_release_branch(args) + result = CreateReleaseBranch(args, self.mock_git, self.mock_gh).run() # Assert self.assertEqual(result, 1) @@ -1468,7 +1467,7 @@ def test_process_backports_no_pending(self): args = MagicMock(issue=123, remote="origin", dry_run=False) self.mock_gh.get_issue_body.return_value = "No backports here" - result = releaser.cmd_process_backports(args) + result = ProcessBackports(args, self.mock_git, self.mock_gh).run() self.assertEqual(result, 0) self.mock_gh.get_issue_body.assert_called_once_with(123) @@ -1502,7 +1501,7 @@ def mock_resolve(items): self.mock_git.get_commit_sha.return_value = "12345678" self.mock_git.get_commit_message.return_value = 'Cherry-pick "fix bug"' - result = releaser.cmd_process_backports(args) + result = ProcessBackports(args, self.mock_git, self.mock_gh).run() self.assertEqual(result, 0) self.mock_git.fetch.assert_has_calls( @@ -1554,7 +1553,7 @@ def mock_resolve(items): self.mock_git.get_commit_sha.return_value = "12345678" self.mock_git.get_commit_message.return_value = 'Cherry-pick "fix bug"' - result = releaser.cmd_process_backports(args) + result = ProcessBackports(args, self.mock_git, self.mock_gh).run() self.assertEqual(result, 0) self.mock_git.fetch.assert_has_calls( @@ -1601,7 +1600,7 @@ def mock_resolve(items): self.mock_gh.get_merge_commits_for_prs.side_effect = mock_resolve - result = releaser.cmd_process_backports(args) + result = ProcessBackports(args, self.mock_git, self.mock_gh).run() self.assertEqual(result, 1) self.mock_gh.update_issue_body.assert_called_once() @@ -1628,7 +1627,7 @@ def test_process_backports_ignored_error_status(self): self.mock_git.get_remote_tags.return_value = [] self.mock_gh.get_merge_commits_for_prs.return_value = [] - result = releaser.cmd_process_backports(args) + result = ProcessBackports(args, self.mock_git, self.mock_gh).run() self.assertEqual(result, 0) self.mock_gh.get_merge_commits_for_prs.assert_not_called() @@ -1661,7 +1660,7 @@ def mock_resolve(items): self.mock_git.sort_commits_chronologically.return_value = ["abcdef12"] self.mock_git.cherry_pick.side_effect = Exception("Cherry-pick conflict") - result = releaser.cmd_process_backports(args) + result = ProcessBackports(args, self.mock_git, self.mock_gh).run() self.assertEqual(result, 1) self.mock_git.checkout.assert_called_once_with( @@ -1680,38 +1679,41 @@ def mock_resolve(items): class GitCheckoutTest(unittest.TestCase): - @patch("tools.private.release.git.run_cmd") - def test_checkout_simple(self, mock_run_cmd): - git.checkout("my-branch") - mock_run_cmd.assert_called_once_with( - "git", "checkout", "my-branch", capture_output=False + def setUp(self): + self.git = Git(".") + self.patcher = patch.object(self.git, "_run_git") + self.mock_run_git = self.patcher.start() + self.addCleanup(self.patcher.stop) + + def test_checkout_simple(self): + self.git.checkout("my-branch") + self.mock_run_git.assert_called_once_with( + "checkout", "my-branch", capture_output=False ) - @patch("tools.private.release.git.branch_exists") - @patch("tools.private.release.git.run_cmd") - def test_checkout_track_remote_new_branch(self, mock_run_cmd, mock_branch_exists): + @patch("tools.private.release.git.Git.branch_exists") + def test_checkout_track_remote_new_branch(self, mock_branch_exists): mock_branch_exists.return_value = False - git.checkout("my-branch", track_remote="origin") + self.git.checkout("my-branch", track_remote="origin") mock_branch_exists.assert_called_once_with("my-branch") - mock_run_cmd.assert_called_once_with( - "git", "checkout", "--track", "origin/my-branch", capture_output=False + self.mock_run_git.assert_called_once_with( + "checkout", "--track", "origin/my-branch", capture_output=False ) - @patch("tools.private.release.git.reset_hard") - @patch("tools.private.release.git.branch_exists") - @patch("tools.private.release.git.run_cmd") + @patch("tools.private.release.git.Git.reset_hard") + @patch("tools.private.release.git.Git.branch_exists") def test_checkout_track_remote_existing_branch( - self, mock_run_cmd, mock_branch_exists, mock_reset_hard + self, mock_branch_exists, mock_reset_hard ): mock_branch_exists.return_value = True - git.checkout("my-branch", track_remote="origin") + self.git.checkout("my-branch", track_remote="origin") mock_branch_exists.assert_called_once_with("my-branch") - mock_run_cmd.assert_called_once_with( - "git", "checkout", "my-branch", capture_output=False + self.mock_run_git.assert_called_once_with( + "checkout", "my-branch", capture_output=False ) mock_reset_hard.assert_called_once_with("origin/my-branch") diff --git a/tools/private/release/BUILD.bazel b/tools/private/release/BUILD.bazel index 1ae52cf37a..7ea3d2e535 100644 --- a/tools/private/release/BUILD.bazel +++ b/tools/private/release/BUILD.bazel @@ -10,12 +10,17 @@ py_library( py_binary( name = "release", srcs = [ + "__init__.py", + "complete_prepare.py", "create_rc.py", "create_release_branch.py", + "create_release_issue.py", + "determine_next_version.py", "gh.py", "git.py", "prepare.py", "process_backports.py", + "promote_rc.py", "release.py", "release_issue.py", "shell.py", diff --git a/tools/private/release/__init__.py b/tools/private/release/__init__.py new file mode 100644 index 0000000000..e68f0391f8 --- /dev/null +++ b/tools/private/release/__init__.py @@ -0,0 +1 @@ +"""Release tools package.""" diff --git a/tools/private/release/complete_prepare.py b/tools/private/release/complete_prepare.py new file mode 100644 index 0000000000..7657738aad --- /dev/null +++ b/tools/private/release/complete_prepare.py @@ -0,0 +1,81 @@ +"""Subcommand to mark preparation task as complete.""" + +import re + +from tools.private.release.gh import GitHub +from tools.private.release.release_issue import update_task_in_body + + +class CompletePrepare: + """Class to mark preparation task as complete.""" + + def __init__(self, args, gh: GitHub): + self.args = args + self.gh = gh + + def run(self) -> int: + """Executes the complete-prepare subcommand (Phase 2 PR merged).""" + args = self.args + print(f"Completing preparation for PR #{args.pr}...") + + pr_info = self.gh.get_pr_info(args.pr) + if not pr_info or pr_info.get("state") != "MERGED": + state = pr_info.get("state", "UNKNOWN") + print(f"Error: PR #{args.pr} is not merged yet (state: {state}).") + return 1 + + # Resolve issue number from PR body + pr_body = pr_info.get("body", "") + match = re.search(r"Work towards #(\d+)", pr_body) + if not match: + match = re.search(r"#(\d+)", pr_body) + if not match: + print( + f"Error: Could not determine tracking issue number from PR" + f" #{args.pr} body: {pr_body}" + ) + return 1 + + issue_num = int(match.group(1)) + print(f"Resolved tracking issue #{issue_num} from PR #{args.pr} body.") + + commit_sha = pr_info["mergeCommit"]["oid"] + short_commit = commit_sha[:8] + print( + f"PR #{args.pr} merged at commit {commit_sha}. Updating tracking issue..." + ) + + # Update checklist: mark Prepare Release as done (checked) and set SUCCESS + body = self.gh.get_issue_body(issue_num) + metadata = { + "status": "done", + "pr": f"#{args.pr}", + "commit": short_commit, + } + updated_body = update_task_in_body( + body, "Prepare Release", checked=True, metadata=metadata + ) + self.gh.update_issue_body(issue_num, updated_body) + print("Prepare Release task marked complete successfully!") + return 0 + + @classmethod + def add_parser(cls, subparsers): + """Adds parser for complete-prepare subcommand.""" + parser = subparsers.add_parser( + "complete-prepare", + help="Mark the Prepare Release task as complete in the tracking issue.", + ) + parser.add_argument( + "--pr", + type=int, + required=True, + help="The merged preparation PR number.", + ) + parser.set_defaults(command=cls.run_from_args) + + @classmethod + def run_from_args(cls, args): + """Instantiates and runs the command from parsed args.""" + gh = GitHub() + return cls(args, gh).run() diff --git a/tools/private/release/create_rc.py b/tools/private/release/create_rc.py index a4593323aa..ea4d8d7450 100644 --- a/tools/private/release/create_rc.py +++ b/tools/private/release/create_rc.py @@ -1,6 +1,7 @@ """Subcommand to tag and push the next release candidate.""" -from tools.private.release import gh, git +from tools.private.release.gh import GitHub +from tools.private.release.git import Git from tools.private.release.release_issue import ( RELEASE_TITLE_RE, parse_backports, @@ -13,91 +14,104 @@ ) -def cmd_create_rc(args): - """Executes the create-rc subcommand.""" - body = gh.get_issue_body(args.issue) - state = parse_checklist_state(body) - - if ( - state["prepare_release"]["status"] != "done" - or state["create_branch"]["status"] != "done" - ): - print( - "Error: Preconditions not met (release must be prepared and branch created)." - ) - return 1 - - # Gating: RC tagging is blocked if any backport is unchecked OR does not have status=done - backports = parse_backports(body) - conflicting_or_pending = [ - b for b in backports if not b.checked or b.status != "done" - ] - if conflicting_or_pending: - print( - f"Gating RC tagging: {len(conflicting_or_pending)} backports are still" - " unfinished, failed, or in conflict." - ) - return 1 - - # Resolve version and branch - issue_title = gh.get_issue_title(args.issue) - version_match = RELEASE_TITLE_RE.search(issue_title) - if not version_match: - print(f"Error: Could not parse version from issue title: {issue_title}") - return 1 - - version = version_match.group(1) - branch_version = ".".join(version.split(".")[:2]) - branch_name = f"release/{branch_version}" - - # Determine next RC tag - git.fetch(args.remote) - git.fetch(args.remote, tags=True, force=True) - latest_rc = get_latest_rc_tag(version, remote=args.remote) - - if not latest_rc: - next_rc_num = 0 - next_rc = f"{version}-rc0" - else: - rc_num = int(latest_rc.split("-rc")[-1]) - next_rc_num = rc_num + 1 - next_rc = f"{version}-rc{next_rc_num}" - - # Precheck: next RC number must exist and be unchecked in the checklist - rc_tags = state.get("rc_tags", {}) - if next_rc_num not in rc_tags: - print( - f"Error: Checklist is missing required task 'Tag RC{next_rc_num}'" - f" to cut {version}-rc{next_rc_num}." +class CreateRc: + """Class to tag and push the next release candidate.""" + + def __init__(self, args, git: Git, gh: GitHub): + self.args = args + self.git = git + self.gh = gh + + def run(self) -> int: + """Executes the create-rc subcommand.""" + args = self.args + body = self.gh.get_issue_body(args.issue) + state = parse_checklist_state(body) + + if ( + state["prepare_release"].status != "done" + or state["create_branch"].status != "done" + ): + print( + "Error: Preconditions not met (release must be prepared and" + " branch created)." + ) + return 1 + + # Gating: RC tagging is blocked if any backport is unchecked OR does not have status=done + backports = parse_backports(body) + conflicting_or_pending = [ + b for b in backports if not b.checked or b.status != "done" + ] + if conflicting_or_pending: + print( + f"Gating RC tagging: {len(conflicting_or_pending)} backports" + " are still unfinished, failed, or in conflict." + ) + return 1 + + # Resolve version and branch + issue_title = self.gh.get_issue_title(args.issue) + version_match = RELEASE_TITLE_RE.search(issue_title) + if not version_match: + print(f"Error: Could not parse version from issue title: {issue_title}") + return 1 + + version = version_match.group(1) + branch_version = ".".join(version.split(".")[:2]) + branch_name = f"release/{branch_version}" + + # Determine next RC tag + self.git.fetch(args.remote) + self.git.fetch(args.remote, tags=True, force=True) + latest_rc = get_latest_rc_tag(version, remote=args.remote) + + if not latest_rc: + next_rc_num = 0 + next_rc = f"{version}-rc0" + else: + rc_num = int(latest_rc.split("-rc")[-1]) + next_rc_num = rc_num + 1 + next_rc = f"{version}-rc{next_rc_num}" + + # Precheck: next RC number must exist and be unchecked in the checklist + rc_tags = state.get("rc_tags", {}) + if next_rc_num not in rc_tags: + print( + f"Error: Checklist is missing required task 'Tag RC{next_rc_num}'" + f" to cut {version}-rc{next_rc_num}." + ) + return 1 + + target_rc_task = rc_tags[next_rc_num] + if target_rc_task.checked or target_rc_task.status == "done": + print( + f"Error: Task 'Tag RC{next_rc_num}' is already marked done in" + " the checklist." + ) + return 1 + + target_ref = f"{args.remote}/{branch_name}" + commit_sha = self.git.get_commit_sha(target_ref) + + print(f"Tagging and pushing next RC: {next_rc}...") + self.git.tag(next_rc, target_ref) + self.git.push(args.remote, next_rc) + + # Check off the appropriate "Tag RC{N}" task in the checklist + print(f"Checking off Tag RC{next_rc_num} task...") + metadata = {"status": "done", "tag": next_rc, "commit": commit_sha[:8]} + task_name = f"Tag RC{next_rc_num}" + updated_body = update_task_in_body( + body, task_name, checked=True, metadata=metadata ) - return 1 - - target_rc_task = rc_tags[next_rc_num] - if target_rc_task["checked"] or target_rc_task["status"] == "done": - print( - f"Error: Task 'Tag RC{next_rc_num}' is already marked done in the checklist." - ) - return 1 - - target_ref = f"{args.remote}/{branch_name}" - commit_sha = git.get_commit_sha(target_ref) + self.gh.update_issue_body(args.issue, updated_body) - print(f"Tagging and pushing next RC: {next_rc}...") - git.tag(next_rc, target_ref) - git.push(args.remote, next_rc) - - # Check off the appropriate "Tag RC{N}" task in the checklist - print(f"Checking off Tag RC{next_rc_num} task...") - metadata = {"status": "done", "tag": next_rc, "commit": commit_sha[:8]} - task_name = f"Tag RC{next_rc_num}" - updated_body = update_task_in_body(body, task_name, checked=True, metadata=metadata) - gh.update_issue_body(args.issue, updated_body) - - tag_url = f"{REPO_URL}/releases/tag/{next_rc}" - bcr_entry_url = f"https://registry.bazel.build/modules/rules_python/{version}" - bcr_search_url = f"https://github.com/bazelbuild/bazel-central-registry/pulls?q=is%3Apr+rules_python+{version}" - release_workflow_url = f"{REPO_URL}/actions/workflows/release.yml" - comment_body = f"""**New Release Candidate Tagged!** 🐍🌿 + tag_url = f"{REPO_URL}/releases/tag/{next_rc}" + bcr_entry_url = f"https://registry.bazel.build/modules/rules_python/{version}" + bcr_search_url = f"https://github.com/bazelbuild/bazel-central-registry/pulls?q=is%3Apr+rules_python+{version}" + release_workflow_url = f"{REPO_URL}/actions/workflows/release.yml" + comment_body = f"""**New Release Candidate Tagged!** 🐍🌿 Release Candidate **{next_rc}** has been successfully generated and tagged on branch `{branch_name}`. @@ -105,6 +119,34 @@ def cmd_create_rc(args): - BCR Entry: [rules_python@{version}]({bcr_entry_url}) - [BCR PRs]({bcr_search_url}) - [Release workflow status]({release_workflow_url})""" - gh.post_issue_comment(args.issue, comment_body) - print("RC creation completed successfully!") - return 0 + self.gh.post_issue_comment(args.issue, comment_body) + print("RC creation completed successfully!") + return 0 + + @classmethod + def add_parser(cls, subparsers): + """Adds parser for create-rc subcommand.""" + parser = subparsers.add_parser( + "create-rc", + help="Tags the next RC on the release branch if no backports remain.", + ) + parser.add_argument( + "--issue", + type=int, + required=True, + help="The tracking issue number (required).", + ) + parser.add_argument( + "--remote", + type=str, + required=True, + help="The git remote to push the RC tag to (required).", + ) + parser.set_defaults(command=cls.run_from_args) + + @classmethod + def run_from_args(cls, args): + """Instantiates and runs the command from parsed args.""" + git = Git(".") + gh = GitHub() + return cls(args, git, gh).run() diff --git a/tools/private/release/create_release_branch.py b/tools/private/release/create_release_branch.py index 7b12465f4c..7a9d7eec37 100644 --- a/tools/private/release/create_release_branch.py +++ b/tools/private/release/create_release_branch.py @@ -1,6 +1,7 @@ """Subcommand to create a release branch from a merged PR commit.""" -from tools.private.release import gh, git +from tools.private.release.gh import GitHub +from tools.private.release.git import Git from tools.private.release.release_issue import ( RELEASE_TITLE_RE, parse_checklist_state, @@ -9,76 +10,122 @@ from tools.private.release.utils import REPO_URL -def cmd_create_release_branch(args): - """Executes the create-release-branch subcommand.""" - print(f"Evaluating branch creation for tracking issue #{args.issue}...") - body = gh.get_issue_body(args.issue) - state = parse_checklist_state(body) +class CreateReleaseBranch: + """Class to create a release branch from a merged PR commit.""" - if ( - state["prepare_release"]["status"] != "done" - or not state["prepare_release"]["commit"] - ): - print( - "Error: Prepare Release task is not marked 'done' with a valid commit SHA." - ) - return 1 + def __init__(self, args, git: Git, gh: GitHub): + self.args = args + self.git = git + self.gh = gh - if state["create_branch"]["checked"]: - print("Release branch has already been created and checked. Skipping.") - return 0 + def run(self) -> int: + """Executes the create-release-branch subcommand.""" + args = self.args + print(f"Evaluating branch creation for tracking issue #{args.issue}...") + body = self.gh.get_issue_body(args.issue) + state = parse_checklist_state(body) - # Extract version from issue title - issue_title = gh.get_issue_title(args.issue) - version_match = RELEASE_TITLE_RE.search(issue_title) - if not version_match: - print(f"Error: Could not parse version from issue title: {issue_title}") - return 1 + if ( + state["prepare_release"].status != "done" + or not state["prepare_release"].commit + ): + print( + "Error: Prepare Release task is not marked 'done' with a valid" + " commit SHA." + ) + return 1 - version = version_match.group(1) - branch_version = ".".join(version.split(".")[:2]) - branch_name = f"release/{branch_version}" + if state["create_branch"].checked: + print("Release branch has already been created and checked. Skipping.") + return 0 - commit_sha = state["prepare_release"]["commit"] - print(f"Cutting branch {branch_name} from commit {commit_sha}...") + # Extract version from issue title + issue_title = self.gh.get_issue_title(args.issue) + version_match = RELEASE_TITLE_RE.search(issue_title) + if not version_match: + print(f"Error: Could not parse version from issue title: {issue_title}") + return 1 - # Create and push branch without affecting local checkout - git.fetch(args.remote) + version = version_match.group(1) + branch_version = ".".join(version.split(".")[:2]) + branch_name = f"release/{branch_version}" - if git.remote_branch_exists(args.remote, branch_name): - remote_ref = f"{args.remote}/{branch_name}" - remote_sha = git.get_commit_sha(remote_ref) - if remote_sha == commit_sha: - print( - f"Branch {branch_name} already exists on {args.remote} and points to {commit_sha}. Skipping push." - ) - elif git.is_ancestor(remote_ref, commit_sha): - print( - f"Branch {branch_name} exists on {args.remote} but can be fast-forwarded to {commit_sha}. Pushing..." - ) - ref_spec = f"{commit_sha}:refs/heads/{branch_name}" - git.push(args.remote, ref_spec) + commit_sha = state["prepare_release"].commit + print(f"Cutting branch {branch_name} from commit {commit_sha}...") + + # Create and push branch without affecting local checkout + self.git.fetch(args.remote) + + if self.git.remote_branch_exists(args.remote, branch_name): + remote_ref = f"{args.remote}/{branch_name}" + remote_sha = self.git.get_commit_sha(remote_ref) + if remote_sha == commit_sha: + print( + f"Branch {branch_name} already exists on {args.remote} and" + f" points to {commit_sha}. Skipping push." + ) + elif self.git.is_ancestor(remote_ref, commit_sha): + print( + f"Branch {branch_name} exists on {args.remote} but can be" + f" fast-forwarded to {commit_sha}. Pushing..." + ) + ref_spec = f"{commit_sha}:refs/heads/{branch_name}" + self.git.push(args.remote, ref_spec) + else: + print( + f"Error: Branch {branch_name} already exists on" + f" {args.remote} at {remote_sha[:8]}, which is not an" + f" ancestor of {commit_sha[:8]}. Cannot fast-forward." + ) + return 1 else: + print(f"Branch {branch_name} does not exist on {args.remote}. Pushing...") + ref_spec = f"{commit_sha}:refs/heads/{branch_name}" + self.git.push(args.remote, ref_spec) print( - f"Error: Branch {branch_name} already exists on {args.remote} at {remote_sha[:8]}, " - f"which is not an ancestor of {commit_sha[:8]}. Cannot fast-forward." + f"Successfully pushed branch {branch_name} pointing to" + f" {commit_sha} to {args.remote}" ) - return 1 - else: - print(f"Branch {branch_name} does not exist on {args.remote}. Pushing...") - ref_spec = f"{commit_sha}:refs/heads/{branch_name}" - git.push(args.remote, ref_spec) - print( - f"Successfully pushed branch {branch_name} pointing to {commit_sha} to {args.remote}" + + # Update tracking issue checklist + print("Updating tracking issue checklist...") + branch_url = f"{REPO_URL}/tree/{branch_name}" + metadata = { + "status": "done", + "branch_url": branch_url, + "commit": commit_sha[:8], + } + updated_body = update_task_in_body( + body, "Create Release branch", checked=True, metadata=metadata + ) + self.gh.update_issue_body(args.issue, updated_body) + print("Create Release branch task marked complete successfully!") + return 0 + + @classmethod + def add_parser(cls, subparsers): + """Adds parser for create-release-branch subcommand.""" + parser = subparsers.add_parser( + "create-release-branch", + help="Create the release branch pointing to the merged PR commit.", + ) + parser.add_argument( + "--issue", + type=int, + required=True, + help="The tracking issue number (required).", + ) + parser.add_argument( + "--remote", + type=str, + required=True, + help="The git remote to create the branch on (required).", ) + parser.set_defaults(command=cls.run_from_args) - # Update tracking issue checklist - print("Updating tracking issue checklist...") - branch_url = f"{REPO_URL}/tree/{branch_name}" - metadata = {"status": "done", "branch_url": branch_url, "commit": commit_sha[:8]} - updated_body = update_task_in_body( - body, "Create Release branch", checked=True, metadata=metadata - ) - gh.update_issue_body(args.issue, updated_body) - print("Create Release branch task marked complete successfully!") - return 0 + @classmethod + def run_from_args(cls, args): + """Instantiates and runs the command from parsed args.""" + git = Git(".") + gh = GitHub() + return cls(args, git, gh).run() diff --git a/tools/private/release/create_release_issue.py b/tools/private/release/create_release_issue.py new file mode 100644 index 0000000000..2d00d7369a --- /dev/null +++ b/tools/private/release/create_release_issue.py @@ -0,0 +1,59 @@ +"""Subcommand to create a release tracking issue.""" + +import pathlib + +from tools.private.release.gh import GitHub +from tools.private.release.utils import determine_next_version, semver_type + + +class CreateReleaseIssue: + """Class to create a release tracking issue.""" + + def __init__(self, args, gh: GitHub): + self.args = args + self.gh = gh + + def run(self) -> int: + """Executes the create-release-issue subcommand.""" + version = self.args.version + if version is None: + version = determine_next_version() + + # Concurrency check + open_issues = self.gh.get_open_tracking_issues() + if open_issues: + print("Error: A release is already in progress. Active tracking issues:") + for issue in open_issues: + print(f"- {issue['title']}: {issue['url']}") + return 1 + + template_path = pathlib.Path( + ".github/ISSUE_TEMPLATE/release_tracking_template.md" + ) + if not template_path.exists(): + raise FileNotFoundError(f"Template file not found at {template_path}") + template_content = template_path.read_text(encoding="utf-8") + + issue_num = self.gh.create_tracking_issue(version, template_content) + print(f"Created tracking issue #{issue_num} for v{version}") + return 0 + + @classmethod + def add_parser(cls, subparsers): + """Adds parser for create-release-issue subcommand.""" + parser = subparsers.add_parser( + "create-release-issue", + help="Search for open releases and create a new tracking issue.", + ) + parser.add_argument( + "--version", + type=semver_type, + help="The release version (e.g., 0.38.0). If not provided, determined automatically.", + ) + parser.set_defaults(command=cls.run_from_args) + + @classmethod + def run_from_args(cls, args): + """Instantiates and runs the command from parsed args.""" + gh = GitHub() + return cls(args, gh).run() diff --git a/tools/private/release/determine_next_version.py b/tools/private/release/determine_next_version.py new file mode 100644 index 0000000000..541daddfe7 --- /dev/null +++ b/tools/private/release/determine_next_version.py @@ -0,0 +1,30 @@ +"""Subcommand to determine the next version.""" + +from tools.private.release.utils import determine_next_version + + +class DetermineNextVersion: + """Class to determine the next version.""" + + def __init__(self, args, git=None, gh=None): + self.args = args + + def run(self) -> int: + """Executes the determine-next-version subcommand.""" + version = determine_next_version() + print(version) + return 0 + + @classmethod + def add_parser(cls, subparsers): + """Adds parser for determine-next-version subcommand.""" + parser = subparsers.add_parser( + "determine-next-version", + help="Determine the next version and print it, without making any changes.", + ) + parser.set_defaults(command=cls.run_from_args) + + @classmethod + def run_from_args(cls, args): + """Instantiates and runs the command from parsed args.""" + return cls(args).run() diff --git a/tools/private/release/gh.py b/tools/private/release/gh.py index cec20ca2d6..8b938aecf6 100644 --- a/tools/private/release/gh.py +++ b/tools/private/release/gh.py @@ -7,9 +7,6 @@ from tools.private.release.release_issue import BackportTask from tools.private.release.shell import run_cmd -_REPO = "bazel-contrib/rules_python" -_LABEL = "type: release" - class MultipleTrackingIssuesError(ValueError): """Raised when multiple open tracking issues are found for a version.""" @@ -23,224 +20,352 @@ class NoTrackingIssueError(ValueError): pass -def list_issues(*, fields, label=None, state=None, search=None): - """Helper to list issues using gh CLI.""" - cmd = ["gh", "issue", "list", f"--repo={_REPO}"] - if label: - cmd.append(f"--label={label}") - if state: - cmd.append(f"--state={state}") - if search: - cmd.append(f"--search={search}") - cmd.append(f"--json={fields}") - - output = run_cmd(*cmd) - return json.loads(output) if output else [] - - -def get_open_tracking_issues(version=None): - """Returns a list of open tracking issues with the 'type: release' label.""" - search = f'"Release {version}" in:title' if version else None - return list_issues( - label=_LABEL, - state="open", - search=search, - fields="number,title,url", - ) - - -def get_release_tracking_issue(version): - """Resolves the tracking issue number for a given version. - - Searches for an open issue with label 'type: release' and 'Release ' in the title. - Raises ValueError if 0 or multiple issues are found. - """ - matching_issues = get_open_tracking_issues(version) - - exact_matches = [] - for issue in matching_issues: - if issue["title"] == f"Release {version}": - exact_matches.append(issue) - - if not exact_matches: - raise NoTrackingIssueError( - f"No open tracking issue found matching 'Release {version}' " - f"in repo {_REPO} with label '{_LABEL}'" +class GitHub: + """GitHub CLI helper class for the release tool.""" + + def __init__(self, repo: str = "bazel-contrib/rules_python"): + """Initializes the GitHub helper. + + Args: + repo: The GitHub repository to operate on. + """ + self.repo = repo + self.label = "type: release" + + def _run_gh( + self, *args: str, check: bool = True, capture_output: bool = True + ) -> str | None: + """Runs a 'gh' command. + + Args: + *args: Arguments for 'gh' (excluding 'gh'). + check: If True, raises CalledProcessError on failure. + capture_output: If True, captures and returns stdout. + + Returns: + The stdout of the command, stripped, or None. + """ + return run_cmd("gh", *args, check=check, capture_output=capture_output) + + def _gh_issue( + self, *args: str, check: bool = True, capture_output: bool = True + ) -> str | None: + """Runs a 'gh issue' command.""" + return self._run_gh( + "issue", + *args, + f"--repo={self.repo}", + check=check, + capture_output=capture_output, + ) + + def _gh_pr( + self, *args: str, check: bool = True, capture_output: bool = True + ) -> str | None: + """Runs a 'gh pr' command.""" + return self._run_gh( + "pr", + *args, + f"--repo={self.repo}", + check=check, + capture_output=capture_output, ) - if len(exact_matches) > 1: - urls = [issue["url"] for issue in exact_matches] - raise MultipleTrackingIssuesError( - f"Multiple open tracking issues found for version {version} " - f"in repo {_REPO} with label '{_LABEL}':\n" + "\n".join(urls) + + def list_issues( + self, + *, + fields: str, + label: str | None = None, + state: str | None = None, + search: str | None = None, + ) -> list[dict]: + """Helper to list issues using gh CLI. + + Args: + fields: Comma-separated list of fields to return. + label: Filter by label. + state: Filter by state (open, closed, all). + search: Search query. + + Returns: + A list of dictionaries representing the issues. + """ + cmd = ["list"] + if label: + cmd.append(f"--label={label}") + if state: + cmd.append(f"--state={state}") + if search: + cmd.append(f"--search={search}") + cmd.append(f"--json={fields}") + + output = self._gh_issue(*cmd) + return json.loads(output) if output else [] + + def get_open_tracking_issues(self, version: str | None = None) -> list[dict]: + """Returns a list of open tracking issues with the 'type: release' label. + + Args: + version: Optional version to filter by. + + Returns: + A list of open tracking issues. + """ + search = f'"Release {version}" in:title' if version else None + return self.list_issues( + label=self.label, + state="open", + search=search, + fields="number,title,url", ) - return exact_matches[0]["number"] + def get_release_tracking_issue(self, version: str) -> int: + """Resolves the tracking issue number for a given version. + + Searches for an open issue with label 'type: release' and 'Release + ' in the title. + + Args: + version: The version to find the tracking issue for. + + Returns: + The tracking issue number. + + Raises: + NoTrackingIssueError: If no open tracking issue is found. + MultipleTrackingIssuesError: If multiple open tracking issues are + found. + """ + matching_issues = self.get_open_tracking_issues(version) + + exact_matches = [] + for issue in matching_issues: + if issue["title"] == f"Release {version}": + exact_matches.append(issue) + + if not exact_matches: + raise NoTrackingIssueError( + f"No open tracking issue found matching 'Release {version}' " + f"in repo {self.repo} with label '{self.label}'" + ) + if len(exact_matches) > 1: + urls = [issue["url"] for issue in exact_matches] + raise MultipleTrackingIssuesError( + f"Multiple open tracking issues found for version {version} " + f"in repo {self.repo} with label '{self.label}':\n" + "\n".join(urls) + ) + + return exact_matches[0]["number"] + + def create_tracking_issue(self, version: str, template_content: str) -> int: + """Creates a new release tracking issue from template content. + + Strips YAML frontmatter if present. + + Args: + version: The version to create the tracking issue for. + template_content: The markdown template content for the issue body. + + Returns: + The created issue number. + """ + # Strip YAML frontmatter if present + issue_body = template_content + if template_content.startswith("---"): + parts = template_content.split("---", 2) + if len(parts) >= 3: + issue_body = parts[2].strip() + + # Write body to a secure temporary file to pass to the CLI + with tempfile.NamedTemporaryFile(mode="w", suffix=".md", delete=False) as f: + f.write(issue_body) + temp_path = f.name + try: + output = self._gh_issue( + "create", + f"--title=Release {version}", + f"--label={self.label}", + f"--body-file={temp_path}", + ) + if not output: + raise RuntimeError("Failed to get issue URL from gh issue create") + issue_url = output.strip() + issue_num = int(issue_url.split("/")[-1]) + return issue_num + finally: + if os.path.exists(temp_path): + os.unlink(temp_path) + + def get_issue_body(self, issue_num: int) -> str: + """Fetches the body of a specific issue. + + Args: + issue_num: The issue number. + + Returns: + The issue body markdown. + """ + output = self._gh_issue( + "view", + str(issue_num), + "--json=body", + "--jq=.body", + ) + return output if output else "" -def create_tracking_issue(version, template_content): - """Creates a new release tracking issue from template content (strips YAML frontmatter).""" - # Strip YAML frontmatter if present - issue_body = template_content - if template_content.startswith("---"): - parts = template_content.split("---", 2) - if len(parts) >= 3: - issue_body = parts[2].strip() + def get_issue_title(self, issue_num: int) -> str: + """Fetches the title of a specific issue. - # Write body to a secure temporary file to pass to the CLI - with tempfile.NamedTemporaryFile(mode="w", suffix=".md", delete=False) as f: - f.write(issue_body) - temp_path = f.name + Args: + issue_num: The issue number. - try: - output = run_cmd( - "gh", - "issue", + Returns: + The issue title. + """ + output = self._gh_issue( + "view", + str(issue_num), + "--json=title", + ) + return json.loads(output)["title"] if output else "" + + def update_issue_body(self, issue_num: int, body: str) -> None: + """Updates the body of a specific issue. + + Args: + issue_num: The issue number. + body: The new issue body markdown. + """ + with tempfile.NamedTemporaryFile(mode="w", suffix=".md", delete=False) as f: + f.write(body) + temp_path = f.name + try: + self._gh_issue( + "edit", + str(issue_num), + f"--body-file={temp_path}", + capture_output=False, + ) + finally: + if os.path.exists(temp_path): + os.unlink(temp_path) + + def create_pr(self, version: str, issue_num: int) -> str: + """Creates a pull request for release preparation. + + Args: + version: The version being prepared. + issue_num: The associated tracking issue number. + + Returns: + The URL of the created PR. + """ + output = self._gh_pr( "create", - f"--title=Release {version}", - f"--label={_LABEL}", - f"--body-file={temp_path}", + f"--title=Prepare release v{version}", + f"--body=Work towards #{issue_num}", + "--base=main", ) - issue_url = output.strip() - issue_num = int(issue_url.split("/")[-1]) - return issue_num - finally: - if os.path.exists(temp_path): - os.unlink(temp_path) - - -def get_issue_body(issue_num): - """Fetches the body of a specific issue.""" - return run_cmd( - "gh", - "issue", - "view", - str(issue_num), - "--json=body", - "--jq=.body", - ) - - -def get_issue_title(issue_num): - """Fetches the title of a specific issue.""" - output = run_cmd( - "gh", - "issue", - "view", - str(issue_num), - "--json=title", - ) - return json.loads(output)["title"] if output else "" - - -def update_issue_body(issue_num, body): - """Updates the body of a specific issue.""" - with tempfile.NamedTemporaryFile(mode="w", suffix=".md", delete=False) as f: - f.write(body) - temp_path = f.name - try: - run_cmd( - "gh", - "issue", - "edit", + return output if output else "" + + def get_open_pr(self, branch_name: str) -> dict | None: + """Returns PR info if an open PR exists for the given branch. + + Args: + branch_name: The head branch name of the PR. + + Returns: + A dictionary with 'number' and 'url' of the PR, or None. + """ + output = self._gh_pr( + "list", + f"--head={branch_name}", + "--state=open", + "--json=number,url", + ) + prs = json.loads(output) if output else [] + return prs[0] if prs else None + + def get_pr_info(self, pr_num: int) -> dict: + """Gets information about a PR. + + Includes state, merge commit, body, and draft status. + + Args: + pr_num: The PR number. + + Returns: + A dictionary containing the PR info. + """ + output = self._gh_pr( + "view", + str(pr_num), + "--json=state,mergeCommit,body,isDraft", + ) + return json.loads(output) if output else {} + + def post_issue_comment(self, issue_num: int, comment_body: str) -> None: + """Posts a comment to a specific issue. + + Args: + issue_num: The issue number. + comment_body: The comment body markdown. + """ + self._gh_issue( + "comment", str(issue_num), - f"--body-file={temp_path}", + f"--body={comment_body}", capture_output=False, ) - finally: - if os.path.exists(temp_path): - os.unlink(temp_path) - - -def create_pr(version, issue_num): - """Creates a pull request for release preparation.""" - return run_cmd( - "gh", - "pr", - "create", - f"--title=Prepare release v{version}", - f"--body=Work towards #{issue_num}", - "--base=main", - ) - - -def get_open_pr(branch_name): - """Returns PR info if an open PR exists for the given branch, else None.""" - cmd = [ - "gh", - "pr", - "list", - f"--repo={_REPO}", - f"--head={branch_name}", - "--state=open", - "--json=number,url", - ] - output = run_cmd(*cmd) - prs = json.loads(output) if output else [] - return prs[0] if prs else None - - -def get_pr_info(pr_num): - """Gets information about a PR, including state, merge commit, body, and draft status.""" - output = run_cmd( - "gh", - "pr", - "view", - str(pr_num), - "--json=state,mergeCommit,body,isDraft", - ) - return json.loads(output) if output else {} - - -def post_issue_comment(issue_num, comment_body): - """Posts a comment to a specific issue.""" - run_cmd( - "gh", - "issue", - "comment", - str(issue_num), - f"--body={comment_body}", - capture_output=False, - ) - - -def get_merge_commits_for_prs(pending_items: list[BackportTask]) -> list[BackportTask]: - """Resolves PR references in pending backports to their merge commit SHAs. - - Updates item.status based on PR state if it cannot be resolved. - """ - resolved_items = [] - for item in pending_items: - pr_num = item.pr_ref.lstrip("#") - print(f"Resolving PR #{pr_num} to merge commit...") - try: - pr_info = get_pr_info(pr_num) - if not pr_info: - print(f"PR #{pr_num} not found. Gating.") - item.status = "error-not-found" - else: - state = pr_info.get("state") - is_draft = pr_info.get("isDraft", False) - if state == "OPEN" or is_draft: - print( - f"PR #{pr_num} is open or draft (state: {state}, draft: {is_draft}). Ignoring." - ) - item.status = "open-pr" if not is_draft else "draft-pr" - elif state == "CLOSED": - print(f"PR #{pr_num} is closed but not merged. Gating.") - item.status = "error-closed-pr" - elif state == "MERGED": - merge_commit = pr_info.get("mergeCommit") - if merge_commit and "oid" in merge_commit: - item.commit = merge_commit["oid"] - item.status = "resolved" - else: - print(f"PR #{pr_num} has no merge commit SHA. Gating.") - item.status = "error-no-merge-commit" + + def get_merge_commits_for_prs( + self, pending_items: list[BackportTask] + ) -> list[BackportTask]: + """Resolves PR references in pending backports to their merge commit SHAs. + + Updates item.status based on PR state if it cannot be resolved. + + Args: + pending_items: A list of BackportTask items to resolve. + + Returns: + The list of resolved BackportTask items. + """ + resolved_items = [] + for item in pending_items: + pr_num = int(item.pr_ref.lstrip("#")) + print(f"Resolving PR #{pr_num} to merge commit...") + try: + pr_info = self.get_pr_info(pr_num) + if not pr_info: + print(f"PR #{pr_num} not found. Gating.") + item.status = "error-not-found" else: - print(f"PR #{pr_num} has unknown state: {state}. Gating.") - item.status = "error-unknown" - except Exception as e: - print(f"Error resolving PR #{pr_num}: {e}. Gating.") - item.status = "error-resolution-failed" - resolved_items.append(item) - return resolved_items + state = pr_info.get("state") + is_draft = pr_info.get("isDraft", False) + if state == "OPEN" or is_draft: + print( + f"PR #{pr_num} is open or draft (state: {state}," + f" draft: {is_draft}). Ignoring." + ) + item.status = "open-pr" if not is_draft else "draft-pr" + elif state == "CLOSED": + print(f"PR #{pr_num} is closed but not merged. Gating.") + item.status = "error-closed-pr" + elif state == "MERGED": + merge_commit = pr_info.get("mergeCommit") + if merge_commit and "oid" in merge_commit: + item.commit = merge_commit["oid"] + item.status = "resolved" + else: + print(f"PR #{pr_num} has no merge commit SHA. Gating.") + item.status = "error-no-merge-commit" + else: + print(f"PR #{pr_num} has unknown state: {state}. Gating.") + item.status = "error-unknown" + except Exception as e: + print(f"Error resolving PR #{pr_num}: {e}. Gating.") + item.status = "error-resolution-failed" + resolved_items.append(item) + return resolved_items diff --git a/tools/private/release/git.py b/tools/private/release/git.py index 795710fd8c..69edba3565 100644 --- a/tools/private/release/git.py +++ b/tools/private/release/git.py @@ -5,213 +5,344 @@ from tools.private.release.shell import run_cmd -def get_tags(): - """Returns a list of all git tags in the repository.""" - output = run_cmd("git", "tag") - return output.splitlines() if output else [] - - -def checkout( - ref: str, create_branch: bool = False, track_remote: str | None = None -) -> None: - """Checks out a git reference (tag, branch, or commit). - - Args: - ref: The git reference (tag, branch, or commit) to checkout. - create_branch: If True, creates the branch before checking it out. - track_remote: If specified, checks out the branch tracking this remote's - corresponding branch. +class Git: + """Git helper class for the release tool. + + Operates on a specific git repository path. """ - cmd = ["git", "checkout"] - if create_branch: - cmd.append("-b") - should_reset_hard = False - if track_remote: - if branch_exists(ref): - cmd.append(ref) - should_reset_hard = True + def __init__(self, repo: str): + """Initializes the Git helper. + + Args: + repo: The path to the git repository. + """ + self._repo = repo + + def _run_git( + self, *args: str, check: bool = True, capture_output: bool = True + ) -> str | None: + """Runs a git command in the repository directory. + + Args: + *args: Arguments passed to the git command. + check: If True, raises CalledProcessError on failure. + capture_output: If True, captures and returns stdout. + + Returns: + The stdout of the command, stripped, or None if capture_output is + False. + """ + return run_cmd( + "git", + *args, + check=check, + capture_output=capture_output, + cwd=self._repo, + ) + + def get_tags(self) -> list[str]: + """Returns a list of all git tags in the repository. + + Returns: + A list of tag names (strings). + """ + output = self._run_git("tag") + return output.splitlines() if output else [] + + def checkout( + self, + ref: str, + create_branch: bool = False, + track_remote: str | None = None, + ) -> None: + """Checks out a git reference (tag, branch, or commit). + + Args: + ref: The git reference (tag, branch, or commit) to checkout. + create_branch: If True, creates the branch before checking it out. + track_remote: If specified, checks out the branch tracking this + remote's corresponding branch. + """ + cmd = ["checkout"] + if create_branch: + cmd.append("-b") + + should_reset_hard = False + if track_remote: + if self.branch_exists(ref): + cmd.append(ref) + should_reset_hard = True + else: + cmd.extend(["--track", f"{track_remote}/{ref}"]) else: - cmd.extend(["--track", f"{track_remote}/{ref}"]) - else: + cmd.append(ref) + self._run_git(*cmd, capture_output=False) + + if should_reset_hard: + self.reset_hard(f"{track_remote}/{ref}") + + def add(self, *files: str) -> None: + """Stages files for commit. + + Args: + *files: Paths to files to stage. + """ + self._run_git("add", *files, capture_output=False) + + def add_modified_and_deleted(self) -> None: + """Stages all modified and deleted tracked files.""" + self._run_git("add", "--update", capture_output=False) + + def commit(self, message: str, amend: bool = False, no_edit: bool = False) -> None: + """Commits staged changes, optionally amending the previous commit. + + Args: + message: The commit message. + amend: If True, amends the previous commit. + no_edit: If True, uses the existing commit message without editing. + """ + cmd = ["commit"] + if amend: + cmd.append("--amend") + if no_edit: + cmd.append("--no-edit") + if message: + cmd.extend(["-m", message]) + self._run_git(*cmd, capture_output=False) + + def push( + self, + remote: str, + ref: str, + set_upstream: bool = False, + force: bool = False, + ) -> None: + """Pushes a reference to a remote repository. + + Args: + remote: The remote repository name (e.g., 'origin'). + ref: The reference to push (e.g., a branch name). + set_upstream: If True, sets the upstream tracking branch. + force: If True, force pushes the changes. + """ + cmd = ["push"] + if set_upstream: + cmd.append("--set-upstream") + if force: + cmd.append("--force") + cmd.extend([remote, ref]) + self._run_git(*cmd, capture_output=False) + + def fetch( + self, remote: str = "origin", tags: bool = False, force: bool = False + ) -> None: + """Fetches updates from a remote repository. + + Args: + remote: The remote repository name. Defaults to 'origin'. + tags: If True, fetches all tags. + force: If True, force fetches updates. + """ + cmd = ["fetch", remote] + if tags: + cmd.append("--tags") + if force: + cmd.append("--force") + self._run_git(*cmd, capture_output=False) + + def merge(self, commit_ref: str, ff_only: bool = True) -> None: + """Merges a commit into the current branch. + + Args: + commit_ref: The commit reference to merge. + ff_only: If True, only allows fast-forward merges. + """ + cmd = ["merge", commit_ref] + if ff_only: + cmd.append("--ff-only") + self._run_git(*cmd, capture_output=False) + + def tag(self, tag_name: str, commit_ref: str) -> None: + """Creates a local tag pointing to a specific commit. + + Args: + tag_name: The name of the tag to create. + commit_ref: The commit reference the tag should point to. + """ + self._run_git("tag", tag_name, commit_ref, capture_output=False) + + def cherry_pick(self, sha: str) -> None: + """Cherry-picks a commit. + + Args: + sha: The commit SHA to cherry-pick. + """ + self._run_git("cherry-pick", "-x", sha, capture_output=False) + + def cherry_pick_abort(self) -> None: + """Aborts an in-progress cherry-pick operation.""" + self._run_git("cherry-pick", "--abort", capture_output=False) + + def reset_hard(self, ref: str = "HEAD") -> None: + """Resets the index and working tree to a specific reference. + + Args: + ref: The git reference to reset to. Defaults to 'HEAD'. + """ + self._run_git("reset", "--hard", ref, capture_output=False) + + def status(self) -> str: + """Returns the output of git status --porcelain. + + Returns: + The porcelain status output. + """ + output = self._run_git("status", "--porcelain") + return output if output else "" + + def get_commit_sha(self, ref: str = "HEAD", short: bool = False) -> str: + """Returns the commit SHA of a given reference. + + Args: + ref: The git reference. Defaults to 'HEAD'. + short: If True, returns a short SHA. + + Returns: + The commit SHA. + """ + cmd = ["rev-parse"] + if short: + cmd.append("--short") cmd.append(ref) - run_cmd(*cmd, capture_output=False) - - if should_reset_hard: - reset_hard(f"{track_remote}/{ref}") - - -def add(*files): - """Stages files for commit.""" - run_cmd("git", "add", *files, capture_output=False) - - -def add_modified_and_deleted(): - """Stages all modified and deleted tracked files.""" - run_cmd("git", "add", "--update", capture_output=False) - - -def commit(message, amend=False, no_edit=False): - """Commits staged changes, optionally amending the previous commit.""" - cmd = ["git", "commit"] - if amend: - cmd.append("--amend") - if no_edit: - cmd.append("--no-edit") - if message: - cmd.extend(["-m", message]) - run_cmd(*cmd, capture_output=False) - - -def push(remote, ref, set_upstream=False, force=False): - """Pushes a reference to a remote repository.""" - cmd = ["git", "push"] - if set_upstream: - cmd.append("--set-upstream") - if force: - cmd.append("--force") - cmd.extend([remote, ref]) - run_cmd(*cmd, capture_output=False) - - -def fetch(remote="origin", tags=False, force=False): - """Fetches updates from a remote repository.""" - cmd = ["git", "fetch", remote] - if tags: - cmd.append("--tags") - if force: - cmd.append("--force") - run_cmd(*cmd, capture_output=False) - - -def merge(commit_ref, ff_only=True): - """Merges a commit into the current branch.""" - cmd = ["git", "merge", commit_ref] - if ff_only: - cmd.append("--ff-only") - run_cmd(*cmd, capture_output=False) - - -def tag(tag_name, commit_ref): - """Creates a local tag pointing to a specific commit.""" - run_cmd("git", "tag", tag_name, commit_ref, capture_output=False) - - -def cherry_pick(sha: str) -> None: - """Cherry-picks a commit. - - Args: - sha: The commit SHA to cherry-pick. - """ - run_cmd("git", "cherry-pick", "-x", sha, capture_output=False) - - -def cherry_pick_abort(): - """Aborts an in-progress cherry-pick operation.""" - run_cmd("git", "cherry-pick", "--abort", capture_output=False) - - -def reset_hard(ref: str = "HEAD") -> None: - """Resets the index and working tree to a specific reference. - - Args: - ref: The git reference to reset to. Defaults to 'HEAD'. - """ - run_cmd("git", "reset", "--hard", ref, capture_output=False) - - -def status(): - """Returns the output of git status --porcelain.""" - return run_cmd("git", "status", "--porcelain") - - -def get_commit_sha(ref="HEAD", short=False): - """Returns the commit SHA of a given reference.""" - cmd = ["git", "rev-parse"] - if short: - cmd.append("--short") - cmd.append(ref) - return run_cmd(*cmd) - - -def get_commit_message(ref: str = "HEAD") -> str: - """Returns the commit message of a given reference. - - Args: - ref: The git reference to get the message from. Defaults to 'HEAD'. - """ - return run_cmd("git", "log", "-1", "--format=%B", ref) - - -def branch_exists(branch_name): - """Returns True if a local branch exists.""" - try: - run_cmd("git", "show-ref", "--verify", f"refs/heads/{branch_name}") - return True - except subprocess.CalledProcessError: - return False - - -def tag_exists(tag_name): - """Returns True if a local tag exists.""" - try: - run_cmd("git", "show-ref", "--verify", f"refs/tags/{tag_name}") - return True - except subprocess.CalledProcessError: - return False - - -def sort_commits_chronologically(shas): - """Sorts a list of commit SHAs chronologically (oldest first).""" - output = run_cmd("git", "log", "--no-walk", "--reverse", "--format=%H", *shas) - return output.splitlines() if output else [] - - -def get_current_branch(): - """Returns the current git branch name.""" - return run_cmd("git", "rev-parse", "--abbrev-ref", "HEAD") - - -def remote_branch_exists(remote, branch_name): - """Returns True if a remote branch exists.""" - try: - run_cmd("git", "show-ref", "--verify", f"refs/remotes/{remote}/{branch_name}") - return True - except subprocess.CalledProcessError: - return False - - -def is_ancestor(ancestor, descendant): - """Returns True if ancestor is an ancestor of descendant (fast-forwardable).""" - try: - run_cmd("git", "merge-base", "--is-ancestor", ancestor, descendant) - return True - except subprocess.CalledProcessError: - return False - - -def get_remote_tags(remote: str) -> list[str]: - """Returns a list of tags present on the specified remote repository. - - Args: - remote: The name of the git remote to query (e.g., 'origin', 'upstream'). - - Returns: - A list of tag names (strings) found on the remote, excluding peeled tags. - """ - output = run_cmd("git", "ls-remote", "--tags", remote) - tags = [] - for line in output.splitlines(): - if not line: - continue - parts = line.split() - if len(parts) < 2: - continue - ref = parts[1] - if ref.startswith("refs/tags/"): - tag = ref[len("refs/tags/") :] - # Skip peeled tags (e.g. tag^{}) to avoid - # duplicate tag names in the output. - if not tag.endswith("^{}"): - tags.append(tag) - return tags + output = self._run_git(*cmd) + return output if output else "" + + def get_commit_message(self, ref: str = "HEAD") -> str: + """Returns the commit message of a given reference. + + Args: + ref: The git reference. Defaults to 'HEAD'. + + Returns: + The commit message. + """ + output = self._run_git("log", "-1", "--format=%B", ref) + return output if output else "" + + def branch_exists(self, branch_name: str) -> bool: + """Returns True if a local branch exists. + + Args: + branch_name: The name of the branch to check. + + Returns: + True if the branch exists, False otherwise. + """ + try: + self._run_git("show-ref", "--verify", f"refs/heads/{branch_name}") + return True + except subprocess.CalledProcessError: + return False + + def tag_exists(self, tag_name: str) -> bool: + """Returns True if a local tag exists. + + Args: + tag_name: The name of the tag to check. + + Returns: + True if the tag exists, False otherwise. + """ + try: + self._run_git("show-ref", "--verify", f"refs/tags/{tag_name}") + return True + except subprocess.CalledProcessError: + return False + + def sort_commits_chronologically(self, shas: list[str]) -> list[str]: + """Sorts a list of commit SHAs chronologically (oldest first). + + Args: + shas: A list of commit SHAs to sort. + + Returns: + The sorted list of commit SHAs. + """ + output = self._run_git("log", "--no-walk", "--reverse", "--format=%H", *shas) + return output.splitlines() if output else [] + + def get_current_branch(self) -> str: + """Returns the current git branch name. + + Returns: + The current branch name. + """ + output = self._run_git("rev-parse", "--abbrev-ref", "HEAD") + return output if output else "" + + def remote_branch_exists(self, remote: str, branch_name: str) -> bool: + """Returns True if a remote branch exists. + + Args: + remote: The name of the remote. + branch_name: The name of the branch. + + Returns: + True if the remote branch exists, False otherwise. + """ + try: + self._run_git( + "show-ref", + "--verify", + f"refs/remotes/{remote}/{branch_name}", + ) + return True + except subprocess.CalledProcessError: + return False + + def is_ancestor(self, ancestor: str, descendant: str) -> bool: + """Returns True if ancestor is an ancestor of descendant. + + Args: + ancestor: The commit reference that might be an ancestor. + descendant: The commit reference that might be a descendant. + + Returns: + True if ancestor is an ancestor of descendant, False otherwise. + """ + try: + self._run_git("merge-base", "--is-ancestor", ancestor, descendant) + return True + except subprocess.CalledProcessError: + return False + + def get_remote_tags(self, remote: str) -> list[str]: + """Returns a list of tags present on the specified remote repository. + + Args: + remote: The name of the git remote to query (e.g., 'origin', + 'upstream'). + + Returns: + A list of tag names (strings) found on the remote, excluding peeled + tags. + """ + output = self._run_git("ls-remote", "--tags", remote) + tags = [] + if not output: + return tags + for line in output.splitlines(): + if not line: + continue + parts = line.split() + if len(parts) < 2: + continue + ref = parts[1] + if ref.startswith("refs/tags/"): + tag = ref[len("refs/tags/") :] + # Skip peeled tags (e.g. tag^{}) to avoid + # duplicate tag names in the output. + if not tag.endswith("^{}"): + tags.append(tag) + return tags diff --git a/tools/private/release/prepare.py b/tools/private/release/prepare.py index 2d32ee4de0..610f2369e7 100644 --- a/tools/private/release/prepare.py +++ b/tools/private/release/prepare.py @@ -1,7 +1,16 @@ +"""Subcommand to prepare the release (updates changelog, placeholders).""" + +import argparse import datetime import pathlib -from tools.private.release import changelog_news, gh, git +from tools.private.release import changelog_news +from tools.private.release.gh import ( + GitHub, + MultipleTrackingIssuesError, + NoTrackingIssueError, +) +from tools.private.release.git import Git from tools.private.release.release_issue import ( parse_checklist_state, update_task_in_body, @@ -9,161 +18,215 @@ from tools.private.release.utils import ( determine_next_version, replace_version_next, + semver_type, ) -def cmd_prepare(args): - """Executes the prepare subcommand.""" - print("Fetching upstream to verify fresh release history...") - git.fetch(tags=True, force=True) +class Prepare: + """Class to prepare the release.""" - # Run pre-check: verify there are no local edits - status = git.status() - if status: - print( - "Error: Local edits detected. Workspace must be completely clean" - " before running release preparation." - ) - for line in status.splitlines(): - print(f" {line}") - return 1 - print("Pre-check passed: Workspace is clean.") + def __init__(self, args, git: Git, gh: GitHub): + self.args = args + self.git = git + self.gh = gh - version = args.version - if version is None: - version = determine_next_version() + def run(self) -> int: + """Executes the prepare subcommand.""" + args = self.args + print("Fetching upstream to verify fresh release history...") + self.git.fetch(tags=True, force=True) - print(f"Running preparation pipeline for {version}...") + # Run pre-check: verify there are no local edits + status = self.git.status() + if status: + print( + "Error: Local edits detected. Workspace must be completely clean" + " before running release preparation." + ) + for line in status.splitlines(): + print(f" {line}") + return 1 + print("Pre-check passed: Workspace is clean.") - # 1. Find or create tracking issue (EARLY) - # We do this before any write operations (branch creation, commit, push) - issue_num = args.issue + version = args.version + if version is None: + version = determine_next_version() - if not issue_num: - try: - issue_num = gh.get_release_tracking_issue(version) + print(f"Running preparation pipeline for {version}...") + + # 1. Find or create tracking issue (EARLY) + # We do this before any write operations (branch creation, commit, push) + issue_num = args.issue + + if not issue_num: + try: + issue_num = self.gh.get_release_tracking_issue(version) + print(f"Tracking issue: #{issue_num}") + except MultipleTrackingIssuesError as e: + print(f"Error: {e}") + return 1 + except NoTrackingIssueError: + # Not found, we need the template + template_path = pathlib.Path( + ".github/ISSUE_TEMPLATE/release_tracking_template.md" + ) + if not template_path.exists(): + raise FileNotFoundError( + f"Template file not found at {template_path}" + ) + template_content = template_path.read_text(encoding="utf-8") + + if args.dry_run: + print( + f"[DRY RUN] No active tracking issue found for" + f" {version}. Would create a new one." + ) + print(f"[DRY RUN] Title: Release {version}\n{template_content}") + issue_num = None # Keep it None for dry-run prints later + else: + print( + f"No active tracking issue found for {version}." + " Creating a new one..." + ) + issue_num = self.gh.create_tracking_issue(version, template_content) + print(f"Tracking issue: #{issue_num}") + else: print(f"Tracking issue: #{issue_num}") - except gh.MultipleTrackingIssuesError as e: - print(f"Error: {e}") - return 1 - except gh.NoTrackingIssueError: - # Not found, we need the template - template_path = pathlib.Path( - ".github/ISSUE_TEMPLATE/release_tracking_template.md" - ) - if not template_path.exists(): - raise FileNotFoundError(f"Template file not found at {template_path}") - template_content = template_path.read_text(encoding="utf-8") + branch_name = f"prepare-{version}" + + # 2. Interleaved git and write operations + + # --- Branch selection/creation --- + if self.git.branch_exists(branch_name): if args.dry_run: print( - f"[DRY RUN] No active tracking issue found for {version}. Would create a new one." + f"[DRY RUN] Branch {branch_name} already exists. Would" + " checkout existing branch." ) - print(f"[DRY RUN] Title: Release {version}\n{template_content}") - issue_num = None # Keep it None for dry-run prints later else: - print( - f"No active tracking issue found for {version}. Creating a new one..." - ) - issue_num = gh.create_tracking_issue(version, template_content) - print(f"Tracking issue: #{issue_num}") - else: - print(f"Tracking issue: #{issue_num}") - - branch_name = f"prepare-{version}" - - # 2. Interleaved git and write operations + print(f"Branch {branch_name} already exists. Checking it out...") + self.git.checkout(branch_name) + else: + if args.dry_run: + print(f"[DRY RUN] Would create and checkout branch {branch_name}") + else: + self.git.checkout(branch_name, create_branch=True) - # --- Branch selection/creation --- - if git.branch_exists(branch_name): + # --- Update files --- if args.dry_run: print( - f"[DRY RUN] Branch {branch_name} already exists. Would checkout existing branch." + f"[DRY RUN] Would update CHANGELOG.md and version placeholders" + f" for {version}" ) else: - print(f"Branch {branch_name} already exists. Checking it out...") - git.checkout(branch_name) - else: + print("Updating changelog and placeholders...") + release_date = datetime.date.today().strftime("%Y-%m-%d") + changelog_news.update_changelog(version, release_date) + replace_version_next(version) + + # --- Commit and Push --- if args.dry_run: - print(f"[DRY RUN] Would create and checkout branch {branch_name}") + print(f"[DRY RUN] Would push branch {branch_name} to origin") else: - git.checkout(branch_name, create_branch=True) - - # --- Update files --- - if args.dry_run: - print( - f"[DRY RUN] Would update CHANGELOG.md and version placeholders for {version}" - ) - else: - print("Updating changelog and placeholders...") - release_date = datetime.date.today().strftime("%Y-%m-%d") - changelog_news.update_changelog(version, release_date) - replace_version_next(version) - - # --- Commit and Push --- - if args.dry_run: - print(f"[DRY RUN] Would push branch {branch_name} to origin") - else: - modified_files = git.status() - if modified_files: - # Stage all modified and deleted tracked files - git.add_modified_and_deleted() - git.commit(f"Prepare release {version}") + modified_files = self.git.status() + if modified_files: + # Stage all modified and deleted tracked files + self.git.add_modified_and_deleted() + self.git.commit(f"Prepare release {version}") + else: + print("No files modified by the release tool. Nothing to commit.") + + print(f"Pushing branch {branch_name} to origin...") + # Force push to overwrite the remote branch if it already exists (e.g. from a previous run) + self.git.push("origin", branch_name, set_upstream=True, force=True) + + # --- Create PR --- + # Determine if we need to create a PR or reuse an existing one + open_pr = self.gh.get_open_pr(branch_name) + associated_pr = None + + if not open_pr and issue_num: + body = self.gh.get_issue_body(issue_num) + state = parse_checklist_state(body) + associated_pr = state["prepare_release"].pr + + if open_pr: + pr_num = open_pr["number"] + pr_url = open_pr["url"] + print(f"Open Pull Request already exists: {pr_url} (PR #{pr_num})") + elif associated_pr: + pr_num = associated_pr.lstrip("#") + pr_url = f"https://github.com/bazel-contrib/rules_python/pull/{pr_num}" + print( + f"PR #{pr_num} is already associated in tracking issue" + f" #{issue_num}. Using it." + ) else: - print("No files modified by the release tool. Nothing to commit.") - - print(f"Pushing branch {branch_name} to origin...") - # Force push to overwrite the remote branch if it already exists (e.g. from a previous run) - git.push("origin", branch_name, set_upstream=True, force=True) - - # --- Create PR --- - # Determine if we need to create a PR or reuse an existing one - open_pr = gh.get_open_pr(branch_name) - associated_pr = None - - if not open_pr and issue_num: - body = gh.get_issue_body(issue_num) - state = parse_checklist_state(body) - associated_pr = state["prepare_release"]["pr"] - - if open_pr: - pr_num = open_pr["number"] - pr_url = open_pr["url"] - print(f"Open Pull Request already exists: {pr_url} (PR #{pr_num})") - elif associated_pr: - pr_num = associated_pr.lstrip("#") - pr_url = f"https://github.com/bazel-contrib/rules_python/pull/{pr_num}" - print( - f"PR #{pr_num} is already associated in tracking issue #{issue_num}. Using it." - ) - else: + if args.dry_run: + target_issue = f"#{issue_num}" if issue_num else "" + print( + f"[DRY RUN] Would create Pull Request for branch" + f" {branch_name} targeting issue {target_issue}" + ) + pr_num = "" + else: + pr_url = self.gh.create_pr(version, issue_num) + pr_num = pr_url.split("/")[-1] + print(f"Created Pull Request: {pr_url} (PR #{pr_num})") + + # --- Update checklist --- if args.dry_run: target_issue = f"#{issue_num}" if issue_num else "" print( - f"[DRY RUN] Would create Pull Request for branch {branch_name} targeting issue {target_issue}" + f"[DRY RUN] Would update tracking issue {target_issue} checklist" + " 'Prepare Release' task status to PENDING" ) - pr_num = "" else: - pr_url = gh.create_pr(version, issue_num) - pr_num = pr_url.split("/")[-1] - print(f"Created Pull Request: {pr_url} (PR #{pr_num})") - - # --- Update checklist --- - if args.dry_run: - target_issue = f"#{issue_num}" if issue_num else "" - print( - f"[DRY RUN] Would update tracking issue {target_issue} checklist 'Prepare Release' task status to PENDING" + print( + f"Updating tracking issue #{issue_num} checklist 'Prepare" + " Release' task status to PENDING..." + ) + body = self.gh.get_issue_body(issue_num) + metadata = {"status": "pending", "pr": f"#{pr_num}"} + updated_body = update_task_in_body( + body, "Prepare Release", checked=False, metadata=metadata + ) + self.gh.update_issue_body(issue_num, updated_body) + print("Preparation pipeline completed successfully!") + + return 0 + + @classmethod + def add_parser(cls, subparsers): + """Adds parser for prepare subcommand.""" + parser = subparsers.add_parser( + "prepare", + help="Prepare the release (updates changelog, placeholders).", ) - else: - print( - f"Updating tracking issue #{issue_num} checklist 'Prepare Release' task status to PENDING..." + parser.add_argument( + "version", + nargs="?", + type=semver_type, + help="The new release version (e.g., 0.28.0). If not provided, " + "it will be determined automatically.", ) - body = gh.get_issue_body(issue_num) - metadata = {"status": "pending", "pr": f"#{pr_num}"} - updated_body = update_task_in_body( - body, "Prepare Release", checked=False, metadata=metadata + parser.add_argument( + "--issue", + type=int, + help="The tracking issue number (optional, triggers automated branch/PR pipeline).", ) - gh.update_issue_body(issue_num, updated_body) - print("Preparation pipeline completed successfully!") - - return 0 + parser.add_argument( + "--dry-run", + action=argparse.BooleanOptionalAction, + default=True, + help="Perform a dry run (default: True). Use --no-dry-run to actually execute.", + ) + parser.set_defaults(command=cls.run_from_args) + + @classmethod + def run_from_args(cls, args): + """Instantiates and runs the command from parsed args.""" + git = Git(".") + gh = GitHub() + return cls(args, git, gh).run() diff --git a/tools/private/release/process_backports.py b/tools/private/release/process_backports.py index 373e1848ec..21af2d5154 100644 --- a/tools/private/release/process_backports.py +++ b/tools/private/release/process_backports.py @@ -1,9 +1,12 @@ """Subcommand to process pending backports.""" +import argparse import datetime from typing import Any -from tools.private.release import changelog_news, gh, git +from tools.private.release import changelog_news +from tools.private.release.gh import GitHub +from tools.private.release.git import Git from tools.private.release.release_issue import ( RELEASE_TITLE_RE, parse_backports, @@ -12,231 +15,290 @@ from tools.private.release.utils import get_latest_rc_tag -def _process_pr_commit_infos( - pr_commit_infos, body, issue, dry_run -) -> tuple[list[str], dict[str, Any], list[str], list[str], str]: - shas = [] - sha_to_item = {} - failed_prs = [] - ignored_prs = [] - for item in pr_commit_infos: - if item.commit: - sha = item.commit - sha_to_item[sha] = item - shas.append(sha) - elif item.status in ("open-pr", "draft-pr"): - print(f"PR {item.pr_ref} is open or draft. Ignoring.") - ignored_prs.append(item.pr_ref) - else: - failed_prs.append(item.pr_ref) - status_to_set = item.status or "error-unmerged-pr" - if dry_run: - print( - f"[DRY RUN] Would update tracking issue checklist for unresolved PR {item.pr_ref} to status={status_to_set}" - ) +class ProcessBackports: + """Class to process pending backports.""" + + def __init__(self, args, git: Git, gh: GitHub): + self.args = args + self.git = git + self.gh = gh + + def _process_pr_commit_infos( + self, pr_commit_infos, body, issue, dry_run + ) -> tuple[list[str], dict[str, Any], list[str], list[str], str]: + shas = [] + sha_to_item = {} + failed_prs = [] + ignored_prs = [] + for item in pr_commit_infos: + if item.commit: + sha = item.commit + sha_to_item[sha] = item + shas.append(sha) + elif item.status in ("open-pr", "draft-pr"): + print(f"PR {item.pr_ref} is open or draft. Ignoring.") + ignored_prs.append(item.pr_ref) else: - print( - f"Updating tracking issue checklist for unresolved PR {item.pr_ref}..." - ) - try: - body = update_task_in_body( - body, - item.pr_ref, - checked=False, - metadata={"status": status_to_set}, + failed_prs.append(item.pr_ref) + status_to_set = item.status or "error-unmerged-pr" + if dry_run: + print( + f"[DRY RUN] Would update tracking issue checklist for" + f" unresolved PR {item.pr_ref} to status={status_to_set}" ) - gh.update_issue_body(issue, body) - except Exception as e: + else: print( - f"ERROR: Failed to update tracking issue for unresolved PR {item.pr_ref}: {e}" + f"Updating tracking issue checklist for unresolved PR" + f" {item.pr_ref}..." ) - return shas, sha_to_item, failed_prs, ignored_prs, body - - -def _cherry_pick_and_update_prs( - sorted_shas, - sha_to_item, - body, - issue, - remote, - dry_run, - version, - branch_name, - next_rc_suffix, -) -> tuple[list[str], str]: - failed_prs = [] - for sha in sorted_shas: - item = sha_to_item[sha] - print(f"Cherry-picking {item.pr_ref} / {sha}...") - try: - git.cherry_pick(sha) - - # Perform news processing (merging news/ files into the changelog) - print(f"Merging news fragments into changelog for PR {item.pr_ref}...") - release_date = datetime.date.today().strftime("%Y-%m-%d") - changelog_news.update_changelog(version, release_date) - - # Stage changelog changes and news/ deletions - git.add("CHANGELOG.md", "news/") - - # Amend cherry-pick commit to include news merging and deletions, - # and reference the release tracking issue. - print(f"Amending cherry-pick commit for PR {item.pr_ref}...") - current_msg = git.get_commit_message("HEAD") - new_msg = f"{current_msg.strip()}\n\nWork towards #{issue}" - git.commit(new_msg, amend=True) - - if not dry_run: - # Push amended commit - git.push(remote, branch_name) - - new_sha = git.get_commit_sha("HEAD", short=True) - metadata = {"status": "done", "rc": next_rc_suffix, "commit": new_sha} - print(f"Updating tracking issue checklist for PR {item.pr_ref}...") - try: - body = update_task_in_body( - body, item.pr_ref, checked=True, metadata=metadata + try: + body = update_task_in_body( + body, + item.pr_ref, + checked=False, + metadata={"status": status_to_set}, + ) + self.gh.update_issue_body(issue, body) + except Exception as e: + print( + f"ERROR: Failed to update tracking issue for" + f" unresolved PR {item.pr_ref}: {e}" + ) + return shas, sha_to_item, failed_prs, ignored_prs, body + + def _cherry_pick_and_update_prs( + self, + sorted_shas, + sha_to_item, + body, + issue, + remote, + dry_run, + version, + branch_name, + next_rc_suffix, + ) -> tuple[list[str], str]: + failed_prs = [] + for sha in sorted_shas: + item = sha_to_item[sha] + print(f"Cherry-picking {item.pr_ref} / {sha}...") + try: + self.git.cherry_pick(sha) + + # Perform news processing (merging news/ files into the changelog) + print(f"Merging news fragments into changelog for PR {item.pr_ref}...") + release_date = datetime.date.today().strftime("%Y-%m-%d") + changelog_news.update_changelog(version, release_date) + + # Stage changelog changes and news/ deletions + self.git.add("CHANGELOG.md", "news/") + + # Amend cherry-pick commit to include news merging and deletions, + # and reference the release tracking issue. + print(f"Amending cherry-pick commit for PR {item.pr_ref}...") + current_msg = self.git.get_commit_message("HEAD") + new_msg = f"{current_msg.strip()}\n\nWork towards #{issue}" + self.git.commit(new_msg, amend=True) + + if not dry_run: + # Push amended commit + self.git.push(remote, branch_name) + + new_sha = self.git.get_commit_sha("HEAD", short=True) + metadata = { + "status": "done", + "rc": next_rc_suffix, + "commit": new_sha, + } + print(f"Updating tracking issue checklist for PR {item.pr_ref}...") + try: + body = update_task_in_body( + body, item.pr_ref, checked=True, metadata=metadata + ) + self.gh.update_issue_body(issue, body) + except Exception as e: + print( + f"ERROR: Failed to update tracking issue for PR" + f" {item.pr_ref}: {e}" + ) + print(f"Success: backported {item.pr_ref} / {sha} to {branch_name}") + else: + print( + f"[DRY RUN] Success: {item.pr_ref} / {sha} can be" + f" backported without error." ) - gh.update_issue_body(issue, body) - except Exception as e: print( - f"ERROR: Failed to update tracking issue for PR {item.pr_ref}: {e}" + f"[DRY RUN] Would update tracking issue checklist for" + f" PR {item.pr_ref} to status=done" ) - print(f"Success: backported {item.pr_ref} / {sha} to {branch_name}") - else: - print( - f"[DRY RUN] Success: {item.pr_ref} / {sha} can be backported without error." - ) - print( - f"[DRY RUN] Would update tracking issue checklist for PR {item.pr_ref} to status=done" - ) - except Exception as e: - print(f"ERROR: Conflict or error on {sha}: {e}. Aborting.") - try: - git.cherry_pick_abort() - except Exception: - pass - failed_prs.append(item.pr_ref) - - if dry_run: - print( - f"[DRY RUN] Would update tracking issue checklist for failed PR {item.pr_ref} to status=error-merge-conflict" - ) - else: - print( - f"Updating tracking issue checklist for failed PR {item.pr_ref}..." - ) + except Exception as e: + print(f"ERROR: Conflict or error on {sha}: {e}. Aborting.") try: - body = update_task_in_body( - body, - item.pr_ref, - checked=False, - metadata={"status": "error-merge-conflict"}, - ) - gh.update_issue_body(issue, body) + self.git.cherry_pick_abort() + except Exception: + pass + failed_prs.append(item.pr_ref) + + if dry_run: print( - f"Updated back port of {item.pr_ref} to status=error-merge-conflict (unchecked)" + f"[DRY RUN] Would update tracking issue checklist for" + f" failed PR {item.pr_ref} to status=error-merge-conflict" ) - except Exception as e: + else: print( - f"ERROR: Failed to update tracking issue for failed PR {item.pr_ref}: {e}" + f"Updating tracking issue checklist for failed PR" + f" {item.pr_ref}..." ) - return failed_prs, body + try: + body = update_task_in_body( + body, + item.pr_ref, + checked=False, + metadata={"status": "error-merge-conflict"}, + ) + self.gh.update_issue_body(issue, body) + print( + f"Updated back port of {item.pr_ref} to" + f" status=error-merge-conflict (unchecked)" + ) + except Exception as e: + print( + f"ERROR: Failed to update tracking issue for" + f" failed PR {item.pr_ref}: {e}" + ) + return failed_prs, body + def run(self) -> int: + """Executes the process-backports subcommand.""" + args = self.args + body = self.gh.get_issue_body(args.issue) + items = parse_backports(body) -def cmd_process_backports(args): - """Executes the process-backports subcommand.""" - body = gh.get_issue_body(args.issue) - items = parse_backports(body) + pending_items = [ + item + for item in items + if not item.checked and not item.status.startswith("error-") + ] - pending_items = [ - item - for item in items - if not item.checked and not item.status.startswith("error-") - ] + if not pending_items: + print("No pending backports found.") + return 0 - if not pending_items: - print("No pending backports found.") - return 0 + print(f"Found {len(pending_items)} pending backports to process.") + + # Determine branch name from issue title + issue_title = self.gh.get_issue_title(args.issue) + version_match = RELEASE_TITLE_RE.search(issue_title) + if not version_match: + print(f"Error: Could not parse version from issue title: {issue_title}") + return 1 + + version = version_match.group(1) + branch_version = ".".join(version.split(".")[:2]) + branch_name = f"release/{branch_version}" + + # Determine next RC tag to write to backport metadata + self.git.fetch(args.remote, tags=True, force=True) + latest_rc = get_latest_rc_tag(version, remote=args.remote) + if not latest_rc: + next_rc_suffix = "rc0" + else: + rc_num = int(latest_rc.split("-rc")[-1]) + next_rc_suffix = f"rc{rc_num + 1}" + + # Resolve PRs to merge commits using gh helper. + pr_commit_infos = self.gh.get_merge_commits_for_prs(pending_items) + + shas, sha_to_item, failed_prs, ignored_prs, body = ( + self._process_pr_commit_infos( + pr_commit_infos, body, args.issue, args.dry_run + ) + ) + + if not shas: + print("No valid merge commits to process.") + if failed_prs: + print("Failed PRs:") + for pr in failed_prs: + print(f"- {pr}") + return 1 + return 0 + + # Verify workspace is clean before proceeding + if self.git.status(): + print( + "ERROR: Git workspace is dirty. Please commit or stash changes" + " before running backports." + ) + return 1 + + # Sort chronologically using git helper + sorted_shas = self.git.sort_commits_chronologically(shas) + + self.git.fetch(args.remote) + self.git.checkout(branch_name, track_remote=args.remote) + start_sha = self.git.get_commit_sha("HEAD") + + try: + new_failed_prs, body = self._cherry_pick_and_update_prs( + sorted_shas, + sha_to_item, + body, + args.issue, + args.remote, + args.dry_run, + version, + branch_name, + next_rc_suffix, + ) + failed_prs.extend(new_failed_prs) + finally: + if args.dry_run: + print(f"[DRY RUN] Resetting branch {branch_name} to {start_sha}") + self.git.reset_hard(start_sha) - print(f"Found {len(pending_items)} pending backports to process.") - - # Determine branch name from issue title - issue_title = gh.get_issue_title(args.issue) - version_match = RELEASE_TITLE_RE.search(issue_title) - if not version_match: - print(f"Error: Could not parse version from issue title: {issue_title}") - return 1 - - version = version_match.group(1) - branch_version = ".".join(version.split(".")[:2]) - branch_name = f"release/{branch_version}" - - # Determine next RC tag to write to backport metadata - git.fetch(args.remote, tags=True, force=True) - latest_rc = get_latest_rc_tag(version, remote=args.remote) - if not latest_rc: - next_rc_suffix = "rc0" - else: - rc_num = int(latest_rc.split("-rc")[-1]) - next_rc_suffix = f"rc{rc_num + 1}" - - # Resolve PRs to merge commits using gh helper. - pr_commit_infos = gh.get_merge_commits_for_prs(pending_items) - - shas, sha_to_item, failed_prs, ignored_prs, body = _process_pr_commit_infos( - pr_commit_infos, body, args.issue, args.dry_run - ) - - if not shas: - print("No valid merge commits to process.") if failed_prs: - print("Failed PRs:") + print("ERROR: One or more cherry-picks/resolutions failed:") for pr in failed_prs: print(f"- {pr}") return 1 + + if args.dry_run: + print("Dry run completed successfully. No errors found.") + else: + print("All backports successfully processed!") return 0 - # Verify workspace is clean before proceeding - if git.status(): - print( - "ERROR: Git workspace is dirty. Please commit or stash changes before running backports." + @classmethod + def add_parser(cls, subparsers): + """Adds parser for process-backports subcommand.""" + parser = subparsers.add_parser( + "process-backports", + help="Cherry-pick pending backports listed in the tracking issue.", ) - return 1 - - # Sort chronologically using git helper - sorted_shas = git.sort_commits_chronologically(shas) - - git.fetch(args.remote) - git.checkout(branch_name, track_remote=args.remote) - start_sha = git.get_commit_sha("HEAD") - - try: - new_failed_prs, body = _cherry_pick_and_update_prs( - sorted_shas, - sha_to_item, - body, - args.issue, - args.remote, - args.dry_run, - version, - branch_name, - next_rc_suffix, + parser.add_argument( + "--issue", + type=int, + required=True, + help="The tracking issue number (required).", ) - failed_prs.extend(new_failed_prs) - finally: - if args.dry_run: - print(f"[DRY RUN] Resetting branch {branch_name} to {start_sha}") - git.reset_hard(start_sha) - - if failed_prs: - print("ERROR: One or more cherry-picks/resolutions failed:") - for pr in failed_prs: - print(f"- {pr}") - return 1 - - if args.dry_run: - print("Dry run completed successfully. No errors found.") - else: - print("All backports successfully processed!") - return 0 + parser.add_argument( + "--remote", + type=str, + required=True, + help="The git remote to push changes to (required).", + ) + parser.add_argument( + "--dry-run", + action=argparse.BooleanOptionalAction, + default=True, + help="Perform a dry run (default: True). Use --no-dry-run to actually execute.", + ) + parser.set_defaults(command=cls.run_from_args) + + @classmethod + def run_from_args(cls, args): + """Instantiates and runs the command from parsed args.""" + git = Git(".") + gh = GitHub() + return cls(args, git, gh).run() diff --git a/tools/private/release/promote_rc.py b/tools/private/release/promote_rc.py new file mode 100644 index 0000000000..877a65b485 --- /dev/null +++ b/tools/private/release/promote_rc.py @@ -0,0 +1,143 @@ +"""Subcommand to promote a release candidate to final release.""" + +import argparse +import urllib.parse + +from tools.private.release.gh import GitHub +from tools.private.release.git import Git +from tools.private.release.release_issue import update_task_in_body +from tools.private.release.utils import ( + REPO_URL, + determine_next_version, + get_latest_rc_tag, + semver_type, +) + + +class PromoteRc: + """Class to promote a release candidate to final release.""" + + def __init__(self, args, git: Git, gh: GitHub): + self.args = args + self.git = git + self.gh = gh + + def run(self) -> int: + """Executes the promote-rc subcommand (Phase 3).""" + args = self.args + # Fetch from upstream to ensure we have the latest tags + self.git.fetch("upstream", tags=True, force=True) + + version = args.version + if version is None: + version = determine_next_version() + + latest_rc = get_latest_rc_tag(version, remote="upstream") + if not latest_rc: + print(f"Error: No release candidate tags found matching {version}-rc*") + return 1 + + # Verify final tag doesn't already exist + if self.git.tag_exists(version): + print(f"Error: Final tag {version} already exists.") + return 1 + + # Verify issue can be found + issue_num = args.issue + if not issue_num: + try: + issue_num = self.gh.get_release_tracking_issue(version) + except ValueError as e: + print(f"Error: {e}") + return 1 + except Exception as e: + print(f"Error: Unexpected error finding tracking issue: {e}") + return 1 + + # Get commit SHA of the RC tag (which will be the same for the final tag) + commit_sha = self.git.get_commit_sha(latest_rc) + + # Verify issue is in the right format by trying to prepare the update + print(f"Verifying tracking issue #{issue_num} format...") + body = self.gh.get_issue_body(issue_num) + metadata = {"status": "done", "tag": version, "commit": commit_sha[:8]} + try: + updated_body = update_task_in_body( + body, "Tag Final", checked=True, metadata=metadata + ) + except ValueError as e: + print(f"Error: Tracking issue #{issue_num} is malformed: {e}") + return 1 + + # All pre-conditions met, perform modifications + if args.dry_run: + print( + f"[DRY RUN] Pre-conditions passed successfully for promoting" + f" {latest_rc} to {version}." + ) + print(f"[DRY RUN] Would tag commit {commit_sha[:8]} as {version}") + print(f"[DRY RUN] Would push tag {version} to upstream") + print(f"[DRY RUN] Would update tracking issue #{issue_num} checklist") + print(f"[DRY RUN] Would post comment to tracking issue #{issue_num}") + return 0 + + print( + f"Promoting {latest_rc} to final release {version} (commit" + f" {commit_sha[:8]}) using tracking issue #{issue_num}..." + ) + + # Tag the specific commit without checkout, and push to upstream + self.git.tag(version, commit_sha) + self.git.push("upstream", version) + + print(f"Updating tracking issue #{issue_num} checklist...") + self.gh.update_issue_body(issue_num, updated_body) + + print(f"Posting comment to tracking issue #{issue_num}...") + + release_url = f"{REPO_URL}/releases/tag/{version}" + bcr_query = ( + f'is:pr ("bazel-contrib/rules_python" in:title) ("@{version}" in:title)' + ) + bcr_search_url = f"https://github.com/bazelbuild/bazel-central-registry/pulls?q={urllib.parse.quote(bcr_query)}" + comment_body = ( + f"Version {version} has been tagged.\n\n" + f"- **Release Page**: {release_url}\n" + f"- **BCR PR Search**: [{bcr_query}]({bcr_search_url})" + ) + self.gh.post_issue_comment(issue_num, comment_body) + + return 0 + + @classmethod + def add_parser(cls, subparsers): + """Adds parser for promote-rc subcommand.""" + parser = subparsers.add_parser( + "promote-rc", + help="Promote the latest RC to final release.", + ) + parser.add_argument( + "version", + nargs="?", + type=semver_type, + help="The final version to release (e.g., 0.38.0).", + ) + parser.add_argument( + "--issue", + type=int, + help="The tracking issue number (optional).", + ) + parser.add_argument( + "--dry-run", + action=argparse.BooleanOptionalAction, + default=True, + help="Perform a dry run (default: True). Use --no-dry-run to actually execute.", + ) + parser.set_defaults(command=cls.run_from_args) + + @classmethod + def run_from_args(cls, args): + """Instantiates and runs the command from parsed args.""" + git = Git(".") + gh = GitHub() + return cls(args, git, gh).run() diff --git a/tools/private/release/release.py b/tools/private/release/release.py index a6c5b3f88b..a6f3543e71 100644 --- a/tools/private/release/release.py +++ b/tools/private/release/release.py @@ -2,192 +2,27 @@ import argparse import os -import pathlib -import re import sys -from tools.private.release import gh, git -from tools.private.release.create_rc import cmd_create_rc -from tools.private.release.create_release_branch import cmd_create_release_branch -from tools.private.release.prepare import cmd_prepare -from tools.private.release.process_backports import cmd_process_backports -from tools.private.release.release_issue import ( - update_task_in_body, -) -from tools.private.release.utils import ( - REPO_URL, - determine_next_version, - get_latest_rc_tag, -) - - -def _semver_type(value): - if not re.match(r"^\d+\.\d+\.\d+(rc\d+)?$", value): - raise argparse.ArgumentTypeError( - f"'{value}' is not a valid semantic version (X.Y.Z or X.Y.ZrcN)" - ) - return value - - -# ============================================================================== -# Subcommand Execution Functions -# ============================================================================== - - -def cmd_determine_next_version(args): - """Executes the determine-next-version subcommand.""" - version = determine_next_version() - print(version) - return 0 - - -def cmd_create_release_issue(args): - """Executes the create-release-issue subcommand.""" - version = args.version - if version is None: - version = determine_next_version() - - # Concurrency check - open_issues = gh.get_open_tracking_issues() - if open_issues: - print("Error: A release is already in progress. Active tracking issues:") - for issue in open_issues: - print(f"- {issue['title']}: {issue['url']}") - return 1 - - template_path = pathlib.Path(".github/ISSUE_TEMPLATE/release_tracking_template.md") - if not template_path.exists(): - raise FileNotFoundError(f"Template file not found at {template_path}") - template_content = template_path.read_text(encoding="utf-8") - - issue_num = gh.create_tracking_issue(version, template_content) - print(f"Created tracking issue #{issue_num} for v{version}") - return 0 - - -def cmd_complete_prepare(args): - """Executes the complete-prepare subcommand (Phase 2 PR merged).""" - print(f"Completing preparation for PR #{args.pr}...") - - pr_info = gh.get_pr_info(args.pr) - if not pr_info or pr_info.get("state") != "MERGED": - state = pr_info.get("state", "UNKNOWN") - print(f"Error: PR #{args.pr} is not merged yet (state: {state}).") - return 1 - - # Resolve issue number from PR body - pr_body = pr_info.get("body", "") - match = re.search(r"Work towards #(\d+)", pr_body) - if not match: - match = re.search(r"#(\d+)", pr_body) - if not match: - print( - f"Error: Could not determine tracking issue number from PR #{args.pr}" - f" body: {pr_body}" - ) - return 1 - - issue_num = int(match.group(1)) - print(f"Resolved tracking issue #{issue_num} from PR #{args.pr} body.") - - commit_sha = pr_info["mergeCommit"]["oid"] - short_commit = commit_sha[:8] - print(f"PR #{args.pr} merged at commit {commit_sha}. Updating tracking issue...") - - # Update checklist: mark Prepare Release as done (checked) and set SUCCESS - body = gh.get_issue_body(issue_num) - metadata = {"status": "done", "pr": f"#{args.pr}", "commit": short_commit} - updated_body = update_task_in_body( - body, "Prepare Release", checked=True, metadata=metadata - ) - gh.update_issue_body(issue_num, updated_body) - print("Prepare Release task marked complete successfully!") - return 0 - - -def cmd_promote_rc(args): - """Executes the promote-rc subcommand (Phase 3).""" - # Fetch from upstream to ensure we have the latest tags - git.fetch("upstream", tags=True, force=True) - - version = args.version - if version is None: - version = determine_next_version() - - latest_rc = get_latest_rc_tag(version, remote="upstream") - if not latest_rc: - print(f"Error: No release candidate tags found matching {version}-rc*") - return 1 - - # Verify final tag doesn't already exist - if git.tag_exists(version): - print(f"Error: Final tag {version} already exists.") - return 1 - - # Verify issue can be found - issue_num = args.issue - if not issue_num: - try: - issue_num = gh.get_release_tracking_issue(version) - except ValueError as e: - print(f"Error: {e}") - return 1 - except Exception as e: - print(f"Error: Unexpected error finding tracking issue: {e}") - return 1 - - # Get commit SHA of the RC tag (which will be the same for the final tag) - commit_sha = git.get_commit_sha(latest_rc) - - # Verify issue is in the right format by trying to prepare the update - print(f"Verifying tracking issue #{issue_num} format...") - body = gh.get_issue_body(issue_num) - metadata = {"status": "done", "tag": version, "commit": commit_sha[:8]} - try: - updated_body = update_task_in_body( - body, "Tag Final", checked=True, metadata=metadata - ) - except ValueError as e: - print(f"Error: Tracking issue #{issue_num} is malformed: {e}") - return 1 - - # All pre-conditions met, perform modifications - if args.dry_run: - print( - f"[DRY RUN] Pre-conditions passed successfully for promoting {latest_rc} to {version}." - ) - print(f"[DRY RUN] Would tag commit {commit_sha[:8]} as {version}") - print(f"[DRY RUN] Would push tag {version} to upstream") - print(f"[DRY RUN] Would update tracking issue #{issue_num} checklist") - print(f"[DRY RUN] Would post comment to tracking issue #{issue_num}") - return 0 - - print( - f"Promoting {latest_rc} to final release {version} (commit" - f" {commit_sha[:8]}) using tracking issue #{issue_num}..." - ) - - # Tag the specific commit without checkout, and push to upstream - git.tag(version, commit_sha) - git.push("upstream", version) - - print(f"Updating tracking issue #{issue_num} checklist...") - gh.update_issue_body(issue_num, updated_body) - - print(f"Posting comment to tracking issue #{issue_num}...") - import urllib.parse - - release_url = f"{REPO_URL}/releases/tag/{version}" - bcr_query = f'is:pr ("bazel-contrib/rules_python" in:title) ("@{version}" in:title)' - bcr_search_url = f"https://github.com/bazelbuild/bazel-central-registry/pulls?q={urllib.parse.quote(bcr_query)}" - comment_body = ( - f"Version {version} has been tagged.\n\n" - f"- **Release Page**: {release_url}\n" - f"- **BCR PR Search**: [{bcr_query}]({bcr_search_url})" - ) - gh.post_issue_comment(issue_num, comment_body) - - return 0 +from tools.private.release.complete_prepare import CompletePrepare +from tools.private.release.create_rc import CreateRc +from tools.private.release.create_release_branch import CreateReleaseBranch +from tools.private.release.create_release_issue import CreateReleaseIssue +from tools.private.release.determine_next_version import DetermineNextVersion +from tools.private.release.prepare import Prepare +from tools.private.release.process_backports import ProcessBackports +from tools.private.release.promote_rc import PromoteRc + +cmds = [ + DetermineNextVersion, + CreateReleaseIssue, + Prepare, + CompletePrepare, + CreateReleaseBranch, + ProcessBackports, + CreateRc, + PromoteRc, +] def create_parser(): @@ -200,141 +35,8 @@ def create_parser(): dest="command", required=True, help="Subcommands" ) - # Subcommand: determine-next-version - subparsers.add_parser( - "determine-next-version", - help="Determine the next version and print it, without making any changes.", - ) - - # Subcommand: create-release-issue - create_issue_parser = subparsers.add_parser( - "create-release-issue", - help="Search for open releases and create a new tracking issue.", - ) - create_issue_parser.add_argument( - "--version", - type=_semver_type, - help="The release version (e.g., 0.38.0). If not provided, determined automatically.", - ) - - # Subcommand: prepare - prepare_parser = subparsers.add_parser( - "prepare", - help="Prepare the release (updates changelog, placeholders).", - ) - prepare_parser.add_argument( - "version", - nargs="?", - type=_semver_type, - help="The new release version (e.g., 0.28.0). If not provided, " - "it will be determined automatically.", - ) - prepare_parser.add_argument( - "--issue", - type=int, - help="The tracking issue number (optional, triggers automated branch/PR pipeline).", - ) - prepare_parser.add_argument( - "--dry-run", - action=argparse.BooleanOptionalAction, - default=True, - help="Perform a dry run (default: True). Use --no-dry-run to actually execute.", - ) - - # Subcommand: complete-prepare - complete_prep_parser = subparsers.add_parser( - "complete-prepare", - help="Mark the Prepare Release task as complete in the tracking issue.", - ) - complete_prep_parser.add_argument( - "--pr", - type=int, - required=True, - help="The merged preparation PR number.", - ) - - # Subcommand: create-release-branch - create_branch_parser = subparsers.add_parser( - "create-release-branch", - help="Create the release branch pointing to the merged PR commit.", - ) - create_branch_parser.add_argument( - "--issue", - type=int, - required=True, - help="The tracking issue number (required).", - ) - create_branch_parser.add_argument( - "--remote", - type=str, - required=True, - help="The git remote to create the branch on (required).", - ) - - # Subcommand: process-backports - process_backports_parser = subparsers.add_parser( - "process-backports", - help="Cherry-pick pending backports listed in the tracking issue.", - ) - process_backports_parser.add_argument( - "--issue", - type=int, - required=True, - help="The tracking issue number (required).", - ) - process_backports_parser.add_argument( - "--remote", - type=str, - required=True, - help="The git remote to push changes to (required).", - ) - process_backports_parser.add_argument( - "--dry-run", - action=argparse.BooleanOptionalAction, - default=True, - help="Perform a dry run (default: True). Use --no-dry-run to actually execute.", - ) - - # Subcommand: create-rc - create_rc_parser = subparsers.add_parser( - "create-rc", - help="Tags the next RC on the release branch if no backports remain.", - ) - create_rc_parser.add_argument( - "--issue", - type=int, - required=True, - help="The tracking issue number (required).", - ) - create_rc_parser.add_argument( - "--remote", - type=str, - required=True, - help="The git remote to push the RC tag to (required).", - ) - - # Subcommand: promote-rc - promote_parser = subparsers.add_parser( - "promote-rc", - help="Promote the latest RC to final release.", - ) - promote_parser.add_argument( - "version", - nargs="?", - type=_semver_type, - help="The final version to release (e.g., 0.38.0).", - ) - promote_parser.add_argument( - "--issue", - type=int, - help="The tracking issue number (optional).", - ) - promote_parser.add_argument( - "--dry-run", - action=argparse.BooleanOptionalAction, - default=True, - help="Perform a dry run (default: True). Use --no-dry-run to actually execute.", - ) + for cmd in cmds: + cmd.add_parser(subparsers) return parser @@ -348,24 +50,10 @@ def main(): exit_code = 1 try: - if args.command == "determine-next-version": - exit_code = cmd_determine_next_version(args) - elif args.command == "create-release-issue": - exit_code = cmd_create_release_issue(args) - elif args.command == "prepare": - exit_code = cmd_prepare(args) - elif args.command == "complete-prepare": - exit_code = cmd_complete_prepare(args) - elif args.command == "create-release-branch": - exit_code = cmd_create_release_branch(args) - elif args.command == "process-backports": - exit_code = cmd_process_backports(args) - elif args.command == "create-rc": - exit_code = cmd_create_rc(args) - elif args.command == "promote-rc": - exit_code = cmd_promote_rc(args) + # args.command is the run_from_args classmethod of the selected command + exit_code = args.command(args) except Exception as e: - print(f"Fatal error executing {args.command}: {e}", file=sys.stderr) + print(f"Fatal error: {e}", file=sys.stderr) if hasattr(e, "__notes__"): for note in e.__notes__: print(note, file=sys.stderr) diff --git a/tools/private/release/release_issue.py b/tools/private/release/release_issue.py index adc0d88086..276261bef6 100644 --- a/tools/private/release/release_issue.py +++ b/tools/private/release/release_issue.py @@ -38,6 +38,49 @@ def __repr__(self): ) +class ReleaseTask: + """Represents a release task from the tracking issue checklist.""" + + def __init__( + self, + name: str, + checked: bool, + status: str | None = None, + pr: str | None = None, + commit: str | None = None, + branch: str | None = None, + tag: str | None = None, + metadata: dict[str, str] | None = None, + ): + """Initializes a ReleaseTask. + + Args: + name: The name of the task (e.g. 'Prepare Release'). + checked: Whether the checklist item is checked. + status: The status of the task (e.g. 'pending', 'done'). + pr: The associated PR reference (e.g. '#123'). + commit: The associated commit SHA. + branch: The associated branch name. + tag: The associated tag name. + metadata: Raw metadata parsed from the checklist line. + """ + self.name = name + self.checked = checked + self.status = status + self.pr = pr + self.commit = commit + self.branch = branch + self.tag = tag + self.metadata = metadata or {} + + def __repr__(self): + return ( + f"ReleaseTask(name={self.name!r}, checked={self.checked!r}, " + f"status={self.status!r}, pr={self.pr!r}, commit={self.commit!r}, " + f"branch={self.branch!r}, tag={self.tag!r})" + ) + + RELEASE_TITLE_RE = re.compile(r"Release (\d+\.\d+\.\d+)", re.IGNORECASE) @@ -101,22 +144,17 @@ def update_task_in_body(body, task_name, checked, metadata): def parse_checklist_state(body): - """Parses the main checklist tasks and their metadata.""" + """Parses the main checklist tasks and their metadata. + + Returns: + A dict containing ReleaseTask objects for 'prepare_release', + 'create_branch', 'tag_final', and a dict of RC tags. + """ state = { - "prepare_release": { - "checked": False, - "status": None, - "pr": None, - "commit": None, - }, - "create_branch": { - "checked": False, - "status": None, - "branch": None, - "commit": None, - }, - "tag_final": {"checked": False, "status": None, "tag": None, "commit": None}, - "rc_tags": {}, # Dynamically mapped: int -> metadata dict + "prepare_release": ReleaseTask("Prepare Release", False), + "create_branch": ReleaseTask("Create Release branch", False), + "tag_final": ReleaseTask("Tag Final", False), + "rc_tags": {}, # Dynamically mapped: int -> ReleaseTask } lines = body.splitlines() @@ -131,37 +169,45 @@ def parse_checklist_state(body): name_lower = name.lower() if "prepare release" in name_lower: - state["prepare_release"] = { - "checked": checked, - "status": meta.get("status"), - "pr": meta.get("pr"), - "commit": meta.get("commit"), - } + state["prepare_release"] = ReleaseTask( + name=name, + checked=checked, + status=meta.get("status"), + pr=meta.get("pr"), + commit=meta.get("commit"), + metadata=meta, + ) elif "create release branch" in name_lower: - state["create_branch"] = { - "checked": checked, - "status": meta.get("status"), - "branch": meta.get("branch"), - "commit": meta.get("commit"), - } + state["create_branch"] = ReleaseTask( + name=name, + checked=checked, + status=meta.get("status"), + branch=meta.get("branch"), + commit=meta.get("commit"), + metadata=meta, + ) elif "tag final" in name_lower: - state["tag_final"] = { - "checked": checked, - "status": meta.get("status"), - "tag": meta.get("tag"), - "commit": meta.get("commit"), - } + state["tag_final"] = ReleaseTask( + name=name, + checked=checked, + status=meta.get("status"), + tag=meta.get("tag"), + commit=meta.get("commit"), + metadata=meta, + ) else: # Match Tag RC rc_match = re.match(r"Tag RC(\d+)", name, re.IGNORECASE) if rc_match: rc_num = int(rc_match.group(1)) - state["rc_tags"][rc_num] = { - "checked": checked, - "status": meta.get("status"), - "tag": meta.get("tag"), - "commit": meta.get("commit"), - } + state["rc_tags"][rc_num] = ReleaseTask( + name=name, + checked=checked, + status=meta.get("status"), + tag=meta.get("tag"), + commit=meta.get("commit"), + metadata=meta, + ) return state diff --git a/tools/private/release/shell.py b/tools/private/release/shell.py index cfff53f4e4..0093a28574 100644 --- a/tools/private/release/shell.py +++ b/tools/private/release/shell.py @@ -4,14 +4,15 @@ import subprocess -def run_cmd(*args, check=True, capture_output=True): +def run_cmd(*args, check=True, capture_output=True, cwd=None): """Runs a command as a subprocess with separate arguments (prints command). If the command fails, it raises the CalledProcessError after attaching a detailed note explaining the failure to preserve the stack trace. """ cmd = [str(arg) for arg in args] - print(f"Running: {shlex.join(cmd)}") + cwd_suffix = f" (cwd: {cwd})" if cwd else "" + print(f"> {shlex.join(cmd)}{cwd_suffix}") try: result = subprocess.run( cmd, @@ -19,10 +20,11 @@ def run_cmd(*args, check=True, capture_output=True): stdout=subprocess.PIPE if capture_output else None, stderr=subprocess.PIPE if capture_output else None, universal_newlines=True, + cwd=cwd, ) return result.stdout.strip() if capture_output else None except subprocess.CalledProcessError as e: - note = f"Error running command: {shlex.join(cmd)}" + note = f"Error running command: {shlex.join(cmd)}{cwd_suffix}" if capture_output: note += f"\nStdout: {e.stdout}\nStderr: {e.stderr}" e.add_note(note) diff --git a/tools/private/release/utils.py b/tools/private/release/utils.py index 2d050022b1..e774c80134 100644 --- a/tools/private/release/utils.py +++ b/tools/private/release/utils.py @@ -1,15 +1,26 @@ """Utility functions for the release tool.""" +import argparse import fnmatch import os import re from packaging.version import parse as parse_version -from tools.private.release import git +from tools.private.release.git import Git REPO_URL = "https://github.com/bazel-contrib/rules_python" + +def semver_type(value): + """Argparse type validator for semantic versions.""" + if not re.match(r"^\d+\.\d+\.\d+(rc\d+)?$", value): + raise argparse.ArgumentTypeError( + f"'{value}' is not a valid semantic version (X.Y.Z or X.Y.ZrcN)" + ) + return value + + _EXCLUDE_PATTERNS = [ "./.git/*", "./.github/*", @@ -45,6 +56,7 @@ def _iter_version_placeholder_files(): def get_latest_version(): """Gets the latest version from git tags.""" + git = Git(".") tags = git.get_tags() versions = [ (tag, parse_version(tag)) @@ -69,6 +81,7 @@ def get_latest_version(): def get_latest_rc_tag(version, remote=None): """Queries git tags and returns the highest RC tag for the version.""" + git = Git(".") if remote: tags = git.get_remote_tags(remote) else: @@ -97,6 +110,7 @@ def should_increment_minor(): def determine_next_version(branch_name=None): """Determines the next version based on git tags and the current branch.""" + git = Git(".") if branch_name is None: branch_name = git.get_current_branch() From 6570a9ec1db2bd61be1b4d3e92fdc85d24da0d1d Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Thu, 2 Jul 2026 00:51:46 -0700 Subject: [PATCH 803/922] chore(release): pass missing --remote argument to release workflows (#3883) Workflows using `process-backports` and `promote-rc` subcommands were failing due to missing or hardcoded remote arguments. - Added `--remote` parameter to `promote-rc` subcommand. - Updated `release_process_backports.yaml` and `release_promote_rc.yaml` to pass `--remote origin`. - Updated tests in `release_test.py` to support and verify the new argument. --- .../workflows/release_process_backports.yaml | 2 +- .github/workflows/release_promote_rc.yaml | 2 +- tests/tools/private/release/release_test.py | 34 +++++++++++-------- tools/private/release/promote_rc.py | 16 ++++++--- 4 files changed, 32 insertions(+), 22 deletions(-) diff --git a/.github/workflows/release_process_backports.yaml b/.github/workflows/release_process_backports.yaml index ac6ea99d80..a2ea7b2067 100644 --- a/.github/workflows/release_process_backports.yaml +++ b/.github/workflows/release_process_backports.yaml @@ -35,6 +35,6 @@ jobs: - name: Process Pending Backports run: | bazel run //tools/private/release -- \ - process-backports --issue ${{ inputs.issue }} + process-backports --issue ${{ inputs.issue }} --remote origin env: GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} diff --git a/.github/workflows/release_promote_rc.yaml b/.github/workflows/release_promote_rc.yaml index b668a15338..5a0ad72a80 100644 --- a/.github/workflows/release_promote_rc.yaml +++ b/.github/workflows/release_promote_rc.yaml @@ -34,6 +34,6 @@ jobs: - name: Run Promote RC run: | bazel run //tools/private/release -- \ - promote-rc ${{ inputs.version }} + promote-rc ${{ inputs.version }} --remote origin env: GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} diff --git a/tests/tools/private/release/release_test.py b/tests/tools/private/release/release_test.py index 96bb65280f..2a752ef1cc 100644 --- a/tests/tools/private/release/release_test.py +++ b/tests/tools/private/release/release_test.py @@ -514,7 +514,9 @@ def test_replace_version_next_excludes_bazel_dirs(self): def test_valid_version(self): # These should not raise an exception releaser.create_parser().parse_args(["prepare", "0.28.0"]) - releaser.create_parser().parse_args(["promote-rc", "1.0.0"]) + releaser.create_parser().parse_args( + ["promote-rc", "1.0.0", "--remote", "origin"] + ) releaser.create_parser().parse_args( ["create-release-issue", "--version", "1.2.3rc4"] ) @@ -1112,7 +1114,7 @@ def setUp(self): def test_promote_rc_success(self): # Arrange - args = MagicMock(version="2.0.0", issue=123, dry_run=False) + args = MagicMock(version="2.0.0", issue=123, dry_run=False, remote="my-remote") self.mock_git.get_remote_tags.return_value = ["2.0.0-rc0", "2.0.0-rc1"] self.mock_git.get_commit_sha.return_value = "abcdef123456" self.mock_git.tag_exists.return_value = False @@ -1124,12 +1126,12 @@ def test_promote_rc_success(self): # Assert self.assertEqual(result, 0) - self.mock_git.fetch.assert_called_once_with("upstream", tags=True, force=True) + self.mock_git.fetch.assert_called_once_with("my-remote", tags=True, force=True) self.mock_git.get_commit_sha.assert_called_once_with("2.0.0-rc1") self.mock_git.checkout.assert_not_called() self.mock_git.tag_exists.assert_called_once_with("2.0.0") self.mock_git.tag.assert_called_once_with("2.0.0", "abcdef123456") - self.mock_git.push.assert_called_once_with("upstream", "2.0.0") + self.mock_git.push.assert_called_once_with("my-remote", "2.0.0") # Verify issue update self.mock_gh.get_issue_body.assert_called_once_with(123) @@ -1148,7 +1150,7 @@ def test_promote_rc_success(self): def test_promote_rc_resolve_issue_success(self): # Arrange - args = MagicMock(version="2.0.0", issue=None, dry_run=False) + args = MagicMock(version="2.0.0", issue=None, dry_run=False, remote="my-remote") self.mock_git.get_remote_tags.return_value = ["2.0.0-rc1"] self.mock_git.tag_exists.return_value = False self.mock_gh.get_release_tracking_issue.side_effect = None @@ -1162,11 +1164,12 @@ def test_promote_rc_resolve_issue_success(self): # Assert self.assertEqual(result, 0) + self.mock_git.fetch.assert_called_once_with("my-remote", tags=True, force=True) self.mock_gh.get_release_tracking_issue.assert_called_once_with("2.0.0") self.mock_git.get_commit_sha.assert_called_once_with("2.0.0-rc1") self.mock_git.checkout.assert_not_called() self.mock_git.tag.assert_called_once_with("2.0.0", "abcdef123456") - self.mock_git.push.assert_called_once_with("upstream", "2.0.0") + self.mock_git.push.assert_called_once_with("my-remote", "2.0.0") self.mock_gh.get_issue_body.assert_called_once_with(123) expected_updated_body = ( "- [x] Tag Final | status=done tag=2.0.0 commit=abcdef12" @@ -1183,7 +1186,7 @@ def test_promote_rc_resolve_issue_success(self): def test_promote_rc_defaults_to_determine_next_version(self): # Arrange - args = MagicMock(version=None, issue=123, dry_run=False) + args = MagicMock(version=None, issue=123, dry_run=False, remote="my-remote") self.mock_git.get_current_branch.return_value = "release/2.0" self.mock_git.get_tags.return_value = ["2.0.0"] self.mock_git.get_remote_tags.return_value = ["2.0.1-rc0"] @@ -1197,14 +1200,15 @@ def test_promote_rc_defaults_to_determine_next_version(self): # Assert self.assertEqual(result, 0) + self.mock_git.fetch.assert_called_once_with("my-remote", tags=True, force=True) self.mock_git.get_current_branch.assert_called_once() self.mock_git.get_tags.assert_called_once() - self.mock_git.get_remote_tags.assert_called_once_with("upstream") + self.mock_git.get_remote_tags.assert_called_once_with("my-remote") self.mock_git.checkout.assert_not_called() self.mock_git.get_commit_sha.assert_called_once_with("2.0.1-rc0") self.mock_git.tag.assert_called_once_with("2.0.1", "12345678") - self.mock_git.push.assert_called_once_with("upstream", "2.0.1") + self.mock_git.push.assert_called_once_with("my-remote", "2.0.1") expected_updated_body = ( "- [x] Tag Final | status=done tag=2.0.1 commit=12345678" @@ -1221,7 +1225,7 @@ def test_promote_rc_defaults_to_determine_next_version(self): def test_promote_rc_dry_run_success(self): # Arrange - args = MagicMock(version="2.0.0", issue=123, dry_run=True) + args = MagicMock(version="2.0.0", issue=123, dry_run=True, remote="my-remote") self.mock_git.get_remote_tags.return_value = ["2.0.0-rc0", "2.0.0-rc1"] self.mock_git.get_commit_sha.return_value = "abcdef123456" self.mock_git.tag_exists.return_value = False @@ -1233,7 +1237,7 @@ def test_promote_rc_dry_run_success(self): # Assert self.assertEqual(result, 0) - self.mock_git.fetch.assert_called_once_with("upstream", tags=True, force=True) + self.mock_git.fetch.assert_called_once_with("my-remote", tags=True, force=True) self.mock_git.get_commit_sha.assert_called_once_with("2.0.0-rc1") self.mock_git.tag_exists.assert_called_once_with("2.0.0") @@ -1245,7 +1249,7 @@ def test_promote_rc_dry_run_success(self): def test_promote_rc_tag_already_exists(self): # Arrange - args = MagicMock(version="2.0.0", issue=123) + args = MagicMock(version="2.0.0", issue=123, remote="my-remote") self.mock_git.get_remote_tags.return_value = ["2.0.0-rc1"] self.mock_git.tag_exists.return_value = True @@ -1262,7 +1266,7 @@ def test_promote_rc_tag_already_exists(self): def test_promote_rc_issue_not_found(self): # Arrange - args = MagicMock(version="2.0.0", issue=None) + args = MagicMock(version="2.0.0", issue=None, remote="my-remote") self.mock_git.get_remote_tags.return_value = ["2.0.0-rc1"] self.mock_git.tag_exists.return_value = False self.mock_gh.get_release_tracking_issue.side_effect = NoTrackingIssueError( @@ -1282,7 +1286,7 @@ def test_promote_rc_issue_not_found(self): def test_promote_rc_issue_malformed(self): # Arrange - args = MagicMock(version="2.0.0", issue=123) + args = MagicMock(version="2.0.0", issue=123, remote="my-remote") self.mock_git.get_remote_tags.return_value = ["2.0.0-rc1"] self.mock_git.tag_exists.return_value = False self.mock_git.get_commit_sha.return_value = "abcdef123456" @@ -1302,7 +1306,7 @@ def test_promote_rc_issue_malformed(self): def test_promote_rc_no_rc_found(self): # Arrange - args = MagicMock(version="2.0.0", issue=123) + args = MagicMock(version="2.0.0", issue=123, remote="my-remote") self.mock_git.get_remote_tags.return_value = [] # Act diff --git a/tools/private/release/promote_rc.py b/tools/private/release/promote_rc.py index 877a65b485..74671daa71 100644 --- a/tools/private/release/promote_rc.py +++ b/tools/private/release/promote_rc.py @@ -25,14 +25,14 @@ def __init__(self, args, git: Git, gh: GitHub): def run(self) -> int: """Executes the promote-rc subcommand (Phase 3).""" args = self.args - # Fetch from upstream to ensure we have the latest tags - self.git.fetch("upstream", tags=True, force=True) + # Fetch from remote to ensure we have the latest tags + self.git.fetch(args.remote, tags=True, force=True) version = args.version if version is None: version = determine_next_version() - latest_rc = get_latest_rc_tag(version, remote="upstream") + latest_rc = get_latest_rc_tag(version, remote=args.remote) if not latest_rc: print(f"Error: No release candidate tags found matching {version}-rc*") return 1 @@ -86,9 +86,9 @@ def run(self) -> int: f" {commit_sha[:8]}) using tracking issue #{issue_num}..." ) - # Tag the specific commit without checkout, and push to upstream + # Tag the specific commit without checkout, and push to remote self.git.tag(version, commit_sha) - self.git.push("upstream", version) + self.git.push(args.remote, version) print(f"Updating tracking issue #{issue_num} checklist...") self.gh.update_issue_body(issue_num, updated_body) @@ -127,6 +127,12 @@ def add_parser(cls, subparsers): type=int, help="The tracking issue number (optional).", ) + parser.add_argument( + "--remote", + type=str, + required=True, + help="The git remote to push the final tag to (required).", + ) parser.add_argument( "--dry-run", action=argparse.BooleanOptionalAction, From e433644f7b67f9088a899594a42105cc790ba26c Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Thu, 2 Jul 2026 00:54:53 -0700 Subject: [PATCH 804/922] chore(release): enable execution in process-backports workflow (#3884) Passes `--no-dry-run` to the `process-backports` subcommand in the workflow, enabling it to actually perform the cherry-picks and push changes instead of only simulating them. --- .github/workflows/release_process_backports.yaml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.github/workflows/release_process_backports.yaml b/.github/workflows/release_process_backports.yaml index a2ea7b2067..822a337323 100644 --- a/.github/workflows/release_process_backports.yaml +++ b/.github/workflows/release_process_backports.yaml @@ -35,6 +35,6 @@ jobs: - name: Process Pending Backports run: | bazel run //tools/private/release -- \ - process-backports --issue ${{ inputs.issue }} --remote origin + process-backports --issue ${{ inputs.issue }} --remote origin --no-dry-run env: GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} From e51d82222ff769f1e94b22c40f9384c1d8d6e72b Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Thu, 2 Jul 2026 10:47:10 -0700 Subject: [PATCH 805/922] chore(release): release automation fixes and refactoring (#3885) - Fix promote_rc dry run message to use correct remote. - Support spaces in metadata parsing and format commit SHA with space for GitHub autolinking. - Trigger release publish workflow from create_rc workflow using workflow_call. - Replace version markers during backport processing. - Split release_test.py into separate files for each command. - Refactor release tool BUILD targets to use py_library and glob. --- .github/workflows/release_create_rc.yaml | 12 +- .../{release.yml => release_publish.yaml} | 17 +- RELEASING.md | 4 +- tests/tools/private/release/BUILD.bazel | 91 +- .../private/release/changelog_news_test.py | 413 ++++ tests/tools/private/release/create_rc_test.py | 181 ++ .../release/create_release_branch_test.py | 149 ++ tests/tools/private/release/git_test.py | 48 + tests/tools/private/release/prepare_test.py | 238 +++ .../private/release/process_backports_test.py | 239 +++ .../tools/private/release/promote_rc_test.py | 236 +++ .../private/release/release_issue_test.py | 70 + tests/tools/private/release/release_test.py | 1705 +---------------- .../private/release/release_test_helper.py | 46 + tests/tools/private/release/utils_test.py | 266 +++ tools/private/release/BUILD.bazel | 33 +- tools/private/release/create_rc.py | 8 +- tools/private/release/process_backports.py | 13 +- tools/private/release/promote_rc.py | 2 +- tools/private/release/release_issue.py | 18 +- 20 files changed, 2048 insertions(+), 1741 deletions(-) rename .github/workflows/{release.yml => release_publish.yaml} (87%) create mode 100644 tests/tools/private/release/changelog_news_test.py create mode 100644 tests/tools/private/release/create_rc_test.py create mode 100644 tests/tools/private/release/create_release_branch_test.py create mode 100644 tests/tools/private/release/git_test.py create mode 100644 tests/tools/private/release/prepare_test.py create mode 100644 tests/tools/private/release/process_backports_test.py create mode 100644 tests/tools/private/release/promote_rc_test.py create mode 100644 tests/tools/private/release/release_issue_test.py create mode 100644 tests/tools/private/release/release_test_helper.py create mode 100644 tests/tools/private/release/utils_test.py diff --git a/.github/workflows/release_create_rc.yaml b/.github/workflows/release_create_rc.yaml index 2e22ed1413..9a58e20c09 100644 --- a/.github/workflows/release_create_rc.yaml +++ b/.github/workflows/release_create_rc.yaml @@ -13,8 +13,10 @@ permissions: issues: write jobs: - generate_rc: + tag_rc: runs-on: ubuntu-latest + outputs: + tag_name: ${{ steps.tagger.outputs.tag_name }} steps: - name: Checkout repository uses: actions/checkout@v7 @@ -32,8 +34,16 @@ jobs: git config --global user.email "41898282+github-actions[bot]@users.noreply.github.com" - name: Attempt RC Tagging + id: tagger run: | bazel run //tools/private/release -- \ create-rc --issue ${{ inputs.issue }} --remote origin env: GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} + + call_release: + needs: tag_rc + uses: ./.github/workflows/release_publish.yaml + with: + tag_name: ${{ needs.tag_rc.outputs.tag_name }} + secrets: inherit diff --git a/.github/workflows/release.yml b/.github/workflows/release_publish.yaml similarity index 87% rename from .github/workflows/release.yml rename to .github/workflows/release_publish.yaml index 039e56a2b0..8379c3d46d 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release_publish.yaml @@ -13,7 +13,7 @@ # limitations under the License. # Cut a release whenever a new tag is pushed to the repo. -name: Release +name: "Release: Publish" on: push: @@ -33,6 +33,17 @@ on: secrets: publish_token: required: false + workflow_call: + inputs: + tag_name: + description: "release tag: tag that will be released" + required: true + type: string + publish_to_pypi: + description: 'Publish to PyPI' + required: false + type: boolean + default: true jobs: release: @@ -42,7 +53,7 @@ jobs: - name: Checkout uses: actions/checkout@v7 with: - ref: ${{ github.ref_name }} + ref: ${{ inputs.tag_name || github.ref_name }} - name: Create release archive and notes run: .github/workflows/create_archive_and_notes.sh ${{ inputs.tag_name || github.ref_name }} - name: Release @@ -73,7 +84,7 @@ jobs: - name: Checkout uses: actions/checkout@v7 with: - ref: ${{ github.tag_name || github.ref_name }} + ref: ${{ inputs.tag_name || github.ref_name }} - if: github.event_name == 'push' || github.event.inputs.publish_to_pypi env: # This special value tells pypi that the user identity is supplied within the token diff --git a/RELEASING.md b/RELEASING.md index a259c782c6..13eb4ddc0a 100644 --- a/RELEASING.md +++ b/RELEASING.md @@ -66,14 +66,14 @@ specific commit. To trigger the workflow, use the `gh workflow run` command: ```shell -gh workflow run release.yml --ref +gh workflow run release_publish.yaml --ref ``` By default, the workflow will publish the wheel to PyPI. To skip this step, you can set the `publish_to_pypi` input to `false`: ```shell -gh workflow run release.yml --ref -f publish_to_pypi=false +gh workflow run release_publish.yaml --ref -f publish_to_pypi=false ``` ### Determining Semantic Version diff --git a/tests/tools/private/release/BUILD.bazel b/tests/tools/private/release/BUILD.bazel index 9f3bc0542a..e68bbef7d5 100644 --- a/tests/tools/private/release/BUILD.bazel +++ b/tests/tools/private/release/BUILD.bazel @@ -1,10 +1,97 @@ -load("@rules_python//python:defs.bzl", "py_test") +load("@rules_python//python:defs.bzl", "py_library", "py_test") + +py_library( + name = "release_test_helper", + srcs = ["release_test_helper.py"], + deps = [ + "//tools/private/release:release_lib", + ], +) + +py_test( + name = "changelog_news_test", + srcs = ["changelog_news_test.py"], + deps = [ + ":release_test_helper", + "//tools/private/release:release_lib", + ], +) + +py_test( + name = "create_rc_test", + srcs = ["create_rc_test.py"], + deps = [ + ":release_test_helper", + "//tools/private/release:release_lib", + ], +) + +py_test( + name = "create_release_branch_test", + srcs = ["create_release_branch_test.py"], + deps = [ + ":release_test_helper", + "//tools/private/release:release_lib", + ], +) + +py_test( + name = "git_test", + srcs = ["git_test.py"], + deps = [ + "//tools/private/release:release_lib", + ], +) + +py_test( + name = "prepare_test", + srcs = ["prepare_test.py"], + deps = [ + ":release_test_helper", + "//tools/private/release:release_lib", + ], +) + +py_test( + name = "process_backports_test", + srcs = ["process_backports_test.py"], + deps = [ + ":release_test_helper", + "//tools/private/release:release_lib", + ], +) + +py_test( + name = "promote_rc_test", + srcs = ["promote_rc_test.py"], + deps = [ + ":release_test_helper", + "//tools/private/release:release_lib", + ], +) + +py_test( + name = "release_issue_test", + srcs = ["release_issue_test.py"], + deps = [ + "//tools/private/release:release_lib", + ], +) py_test( name = "release_test", srcs = ["release_test.py"], deps = [ - "//tools/private/release", + "//tools/private/release:release_lib", + ], +) + +py_test( + name = "utils_test", + srcs = ["utils_test.py"], + deps = [ + ":release_test_helper", + "//tools/private/release:release_lib", "@dev_pip//packaging", ], ) diff --git a/tests/tools/private/release/changelog_news_test.py b/tests/tools/private/release/changelog_news_test.py new file mode 100644 index 0000000000..f8bd141a55 --- /dev/null +++ b/tests/tools/private/release/changelog_news_test.py @@ -0,0 +1,413 @@ +import pathlib +import unittest +from unittest.mock import patch + +from tests.tools.private.release.release_test_helper import TempDirTestCase +from tools.private.release import changelog_news + + +class ChangelogNewsTest(TempDirTestCase): + def test_update_changelog_with_news(self): + # Arrange + changelog = """# Changelog + +{#unreleased} +## Unreleased + +[unreleased]: https://github.com/bazel-contrib/rules_python/releases/tag/unreleased + +{#unreleased-removed} +### Removed +* Nothing removed. + +{#unreleased-changed} +### Changed +* Nothing changed. + +{#unreleased-fixed} +### Fixed +* Nothing fixed. + +{#unreleased-added} +### Added +* Nothing added. + +{#v2-0-2} +## [2.0.2] - 2026-05-14 + +[2.0.2]: https://github.com/bazel-contrib/rules_python/releases/tag/2.0.2 + +{#v2-0-2-added} +### Added +* (toolchains) Some older change. +""" + changelog_path = self.tmpdir / "CHANGELOG.md" + changelog_path.write_text(changelog) + + news_dir = self.tmpdir / "news" + news_dir.mkdir() + + # Create news files + (news_dir / "123.fixed.md").write_text("Fixed a bug in the compiler") + # Test that it handles prefixing "* " if not present + (news_dir / "456.added.md").write_text("* Added a new feature for Python 3.13") + # Empty file should be ignored + (news_dir / "789.changed.md").write_text("") + # Invalid name should be ignored + (news_dir / "invalid_name.md").write_text("Should be ignored") + + # Act + changelog_news.update_changelog( + "3.0.0", + "2026-06-16", + changelog_path=changelog_path, + news_dir=news_dir, + ) + + # Assert + # 1. News files matching the pattern should be deleted (even empty ones) + self.assertFalse((news_dir / "123.fixed.md").exists()) + self.assertFalse((news_dir / "456.added.md").exists()) + self.assertFalse((news_dir / "789.changed.md").exists()) + # Invalid name does not match pattern -> NOT deleted + self.assertTrue((news_dir / "invalid_name.md").exists()) + + new_content = changelog_path.read_text() + + # 2. A fresh active Unreleased section should be present + self.assertIn("{#unreleased}", new_content) + self.assertIn("## Unreleased", new_content) + self.assertIn( + "Unreleased changes are tracked as individual files in the [news/](./news)\n" + "directory, or view the [latest generated\n" + "changelog](https://rules-python.readthedocs.io/en/latest/changelog.html).", + new_content, + ) + + # 3. The new release section should be present + self.assertIn("{#v3-0-0}", new_content) + self.assertIn("## [3.0.0] - 2026-06-16", new_content) + self.assertIn( + "[3.0.0]: https://github.com/bazel-contrib/rules_python/releases/tag/3.0.0", + new_content, + ) + + # 4. Correct categories and content + self.assertIn( + "{#v3-0-0-fixed}\n### Fixed\n* Fixed a bug in the compiler", + new_content, + ) + self.assertIn( + "{#v3-0-0-added}\n### Added\n* Added a new feature for Python 3.13", + new_content, + ) + + # 5. Omitted categories should NOT be present in the new release + self.assertNotIn("{#v3-0-0-removed}", new_content) + self.assertNotIn("{#v3-0-0-changed}", new_content) + + # 6. Old release should still be there + self.assertIn("{#v2-0-2}", new_content) + self.assertIn("## [2.0.2] - 2026-05-14", new_content) + + def test_update_changelog_sorting(self): + # Arrange + changelog = """# Changelog + +{#unreleased} +## Unreleased + +[unreleased]: https://github.com/bazel-contrib/rules_python/releases/tag/unreleased + +Unreleased changes are tracked as individual files in the [news/](./news) +directory, or view the [latest generated +changelog](https://rules-python.readthedocs.io/en/latest/changelog.html). + +{#v2-0-2} +## [2.0.2] - 2026-05-14 + +[2.0.2]: https://github.com/bazel-contrib/rules_python/releases/tag/2.0.2 + +{#v2-0-2-added} +### Added +* (toolchains) Some older change. +""" + changelog_path = self.tmpdir / "CHANGELOG.md" + changelog_path.write_text(changelog) + + news_dir = self.tmpdir / "news" + news_dir.mkdir() + + # Create news files with different sub-categories and some without + (news_dir / "1.fixed.md").write_text("* (zebra) Zebra fix") + (news_dir / "2.fixed.md").write_text("* (apple) Apple fix") + (news_dir / "3.fixed.md").write_text("No subcategory B") + (news_dir / "4.fixed.md").write_text("* (apple) Another apple fix") + (news_dir / "5.fixed.md").write_text("No subcategory A") + + # Act + changelog_news.update_changelog( + "3.0.0", + "2026-06-16", + changelog_path=changelog_path, + news_dir=news_dir, + ) + + # Assert + new_content = changelog_path.read_text() + + # Expected order in Fixed section: + # 1. No subcategory A + # 2. No subcategory B + # 3. (apple) Another apple fix + # 4. (apple) Apple fix + # 5. (zebra) Zebra fix + + expected_fixed_section = ( + "### Fixed\n" + "* No subcategory A\n" + "* No subcategory B\n" + "* (apple) Another apple fix\n" + "* (apple) Apple fix\n" + "* (zebra) Zebra fix\n" + ) + + self.assertIn(expected_fixed_section, new_content) + + def test_update_changelog_read_failure(self): + # Arrange + original_read_text = pathlib.Path.read_text + + with patch("pathlib.Path.read_text", autospec=True) as mock_read_text: + + def side_effect(path_self, *args, **kwargs): + if "bad_file.fixed.md" in str(path_self): + raise IOError("Simulated read error") + return original_read_text(path_self, *args, **kwargs) + + mock_read_text.side_effect = side_effect + + changelog = """# Changelog + +{#unreleased} +## Unreleased + +[unreleased]: https://github.com/bazel-contrib/rules_python/releases/tag/unreleased + +Unreleased changes are tracked as individual files in the [news/](./news) +directory, or view the [latest generated +changelog](https://rules-python.readthedocs.io/en/latest/changelog.html). + +{#v2-0-2} +## [2.0.2] - 2026-05-14 + +[2.0.2]: https://github.com/bazel-contrib/rules_python/releases/tag/2.0.2 + +{#v2-0-2-added} +### Added +* (toolchains) Some older change. +""" + changelog_path = self.tmpdir / "CHANGELOG.md" + changelog_path.write_text(changelog) + + news_dir = self.tmpdir / "news" + news_dir.mkdir() + + # Create the bad file (must exist so it is found by iterdir) + bad_file = news_dir / "bad_file.fixed.md" + bad_file.write_text("some content that won't be read") + + # Create a good file too + good_file = news_dir / "good_file.fixed.md" + good_file.write_text("* (sub) Good fix") + + # Act & Assert + # It should raise IOError + with self.assertRaises(IOError): + changelog_news.update_changelog( + "3.0.0", + "2026-06-16", + changelog_path=changelog_path, + news_dir=news_dir, + ) + + # Both files should still exist (no deletion on failure!) + self.assertTrue(bad_file.exists()) + self.assertTrue(good_file.exists()) + + # Changelog should not be modified + new_content = changelog_path.read_text() + self.assertEqual(changelog, new_content) + + def test_update_changelog_merge_existing(self): + # Arrange + changelog = """# Changelog + +{#unreleased} +## Unreleased + +[unreleased]: https://github.com/bazel-contrib/rules_python/releases/tag/unreleased + +Unreleased changes are tracked as individual files in the [news/](./news) +directory, or view the [latest generated +changelog](https://rules-python.readthedocs.io/en/latest/changelog.html). + +{#v2-0-3} +## [2.0.3] - 2026-06-15 + +[2.0.3]: https://github.com/bazel-contrib/rules_python/releases/tag/2.0.3 + +{#v2-0-3-fixed} +### Fixed +* (pypi) Old fix + multi-line detail + * nested bullet item +* (pypi) Z old fix +""" + changelog_path = self.tmpdir / "CHANGELOG.md" + changelog_path.write_text(changelog) + + news_dir = self.tmpdir / "news" + news_dir.mkdir() + + # Create news files to merge + # 1. New fix in same category (should merge and sort) + (news_dir / "1.fixed.md").write_text("(pypi) New fix") + # 2. New entry in new category (should create category) + (news_dir / "2.added.md").write_text("(toolchains) New feature") + + # Act + changelog_news.update_changelog( + "2.0.3", + "2026-06-15", + changelog_path=changelog_path, + news_dir=news_dir, + ) + + # Assert + # News files should be deleted + self.assertFalse((news_dir / "1.fixed.md").exists()) + self.assertFalse((news_dir / "2.added.md").exists()) + + new_content = changelog_path.read_text() + + # Expected merged and sorted Fixed section: + # 1. (pypi) New fix (New < Old) + # 2. (pypi) Old fix (with its multi-line detail!) + # 3. (pypi) Z old fix + expected_fixed_section = ( + "### Fixed\n" + "* (pypi) New fix\n" + "* (pypi) Old fix\n" + " multi-line detail\n" + " * nested bullet item\n" + "* (pypi) Z old fix\n" + ) + self.assertIn(expected_fixed_section, new_content) + + # Expected created Added section: + expected_added_section = "### Added\n* (toolchains) New feature\n" + self.assertIn(expected_added_section, new_content) + + # Active Unreleased section should NOT be touched (should still be empty/pointing to news) + self.assertIn("Unreleased changes are tracked as individual files", new_content) + + def test_update_changelog_does_not_leak(self): + # Arrange + changelog = """# Changelog + +{#unreleased} +## Unreleased + +[unreleased]: https://github.com/bazel-contrib/rules_python/releases/tag/unreleased + +Unreleased changes are tracked as individual files in the [news/](./news) +directory, or view the [latest generated +changelog](https://rules-python.readthedocs.io/en/latest/changelog.html). + +{#v2-0-2} +## [2.0.2] - 2026-05-14 + +[2.0.2]: https://github.com/bazel-contrib/rules_python/releases/tag/2.0.2 + +This release body mentions the word unreleased and {#unreleased} anchor to test leaks. +""" + changelog_path = self.tmpdir / "CHANGELOG.md" + changelog_path.write_text(changelog) + + news_dir = self.tmpdir / "news" + news_dir.mkdir() + (news_dir / "1.fixed.md").write_text("Some fix") + + # Act + changelog_news.update_changelog( + "3.0.0", + "2026-06-16", + changelog_path=changelog_path, + news_dir=news_dir, + ) + + # Assert + new_content = changelog_path.read_text() + + # The 2.0.2 body should NOT be modified + self.assertIn( + "This release body mentions the word unreleased and {#unreleased} anchor to test leaks.", + new_content, + ) + + def test_update_changelog_empty_news(self): + # Arrange + changelog = """# Changelog + +{#unreleased} +## Unreleased + +[unreleased]: https://github.com/bazel-contrib/rules_python/releases/tag/unreleased + +Unreleased changes are tracked as individual files in the [news/](./news) +directory, or view the [latest generated +changelog](https://rules-python.readthedocs.io/en/latest/changelog.html). + +{#v2-0-2} +## [2.0.2] - 2026-05-14 + +[2.0.2]: https://github.com/bazel-contrib/rules_python/releases/tag/2.0.2 + +{#v2-0-2-added} +### Added +* (toolchains) Some older change. +""" + changelog_path = self.tmpdir / "CHANGELOG.md" + changelog_path.write_text(changelog) + + news_dir = self.tmpdir / "news" + news_dir.mkdir() + + # Act + changelog_news.update_changelog( + "3.0.0", + "2026-06-16", + changelog_path=changelog_path, + news_dir=news_dir, + ) + + # Assert + new_content = changelog_path.read_text() + + # The new release section should be present and contain "No notable changes." + self.assertIn("{#v3-0-0}", new_content) + self.assertIn("## [3.0.0] - 2026-06-16", new_content) + self.assertIn( + "[3.0.0]: https://github.com/bazel-contrib/rules_python/releases/tag/3.0.0", + new_content, + ) + self.assertIn("No notable changes.", new_content) + + # Verify that we didn't accidentally create any categories + self.assertNotIn("{#v3-0-0-fixed}", new_content) + self.assertNotIn("{#v3-0-0-added}", new_content) + + +if __name__ == "__main__": + unittest.main() diff --git a/tests/tools/private/release/create_rc_test.py b/tests/tools/private/release/create_rc_test.py new file mode 100644 index 0000000000..2b4e40cc4e --- /dev/null +++ b/tests/tools/private/release/create_rc_test.py @@ -0,0 +1,181 @@ +import os +import pathlib +import tempfile +import unittest +from unittest.mock import MagicMock, call, patch + +from tests.tools.private.release.release_test_helper import _mock_git_and_gh +from tools.private.release.create_rc import CreateRc + + +class CmdCreateRcTest(unittest.TestCase): + def setUp(self): + _mock_git_and_gh(self) + + def test_create_rc_success_first_rc(self): + # Arrange + args = MagicMock(issue=123, remote="my-remote") + self.mock_gh.get_issue_title.return_value = "Release 2.0.0" + self.mock_gh.get_issue_body.return_value = """ +## Checklist +- [x] Prepare Release | status=done pr=#122 commit=abcdef12 +- [x] Create Release branch | status=done branch=release/2.0 commit=abcdef12 +- [ ] Tag RC0 | status=pending +""" + self.mock_git.get_remote_tags.return_value = [] + self.mock_git.get_commit_sha.return_value = "1234567890" + + # Act + with tempfile.TemporaryDirectory() as tmpdir: + github_output_file = pathlib.Path(tmpdir) / "github_output" + with patch.dict(os.environ, {"GITHUB_OUTPUT": str(github_output_file)}): + result = CreateRc(args, self.mock_git, self.mock_gh).run() + + # Assert + self.assertEqual(result, 0) + self.assertTrue(github_output_file.exists()) + self.assertEqual(github_output_file.read_text(), "tag_name=2.0.0-rc0\n") + self.mock_git.fetch.assert_has_calls( + [call("my-remote"), call("my-remote", tags=True, force=True)] + ) + self.mock_git.checkout.assert_not_called() + self.mock_git.tag.assert_called_once_with("2.0.0-rc0", "my-remote/release/2.0") + self.mock_git.push.assert_called_once_with("my-remote", "2.0.0-rc0") + self.mock_git.get_commit_sha.assert_called_once_with("my-remote/release/2.0") + + self.mock_gh.update_issue_body.assert_called_once() + call_args = self.mock_gh.update_issue_body.call_args[0] + self.assertEqual(call_args[0], 123) + self.assertIn("tag=2.0.0-rc0", call_args[1]) + self.assertIn("commit= 12345678", call_args[1]) + + self.mock_gh.post_issue_comment.assert_called_once() + comment_call_args = self.mock_gh.post_issue_comment.call_args[0] + self.assertEqual(comment_call_args[0], 123) + self.assertIn( + "**New Release Candidate Tagged!** 🐍🌿", + comment_call_args[1], + ) + self.assertIn( + "- [Github Release 2.0.0-rc0](https://github.com/bazel-contrib/rules_python/releases/tag/2.0.0-rc0)", + comment_call_args[1], + ) + self.assertIn( + "- BCR Entry: [rules_python@2.0.0](https://registry.bazel.build/modules/rules_python/2.0.0)", + comment_call_args[1], + ) + self.assertIn( + "- [BCR PRs](https://github.com/bazelbuild/bazel-central-registry/pulls?q=is%3Apr+rules_python+2.0.0)", + comment_call_args[1], + ) + self.assertIn( + "- [Release workflow status](https://github.com/bazel-contrib/rules_python/actions/workflows/release_publish.yaml)", + comment_call_args[1], + ) + self.assertNotIn("🚀", comment_call_args[1]) + + def test_create_rc_success_next_rc(self): + # Arrange + args = MagicMock(issue=123, remote="my-remote") + self.mock_gh.get_issue_title.return_value = "Release 2.0.0" + self.mock_gh.get_issue_body.return_value = """ +## Checklist +- [x] Prepare Release | status=done pr=#122 commit=abcdef12 +- [x] Create Release branch | status=done branch=release/2.0 commit=abcdef12 +- [x] Tag RC0 | status=done tag=2.0.0-rc0 commit=abcdef12 +- [ ] Tag RC1 | status=pending +""" + self.mock_git.get_remote_tags.return_value = ["2.0.0-rc0"] + self.mock_git.get_commit_sha.return_value = "1234567890" + + # Act + result = CreateRc(args, self.mock_git, self.mock_gh).run() + + # Assert + self.assertEqual(result, 0) + self.mock_git.fetch.assert_has_calls( + [call("my-remote"), call("my-remote", tags=True, force=True)] + ) + self.mock_git.checkout.assert_not_called() + self.mock_git.tag.assert_called_once_with("2.0.0-rc1", "my-remote/release/2.0") + self.mock_git.push.assert_called_once_with("my-remote", "2.0.0-rc1") + self.mock_git.get_commit_sha.assert_called_once_with("my-remote/release/2.0") + + self.mock_gh.update_issue_body.assert_called_once() + call_args = self.mock_gh.update_issue_body.call_args[0] + self.assertEqual(call_args[0], 123) + self.assertIn("tag=2.0.0-rc1", call_args[1]) + + self.mock_gh.post_issue_comment.assert_called_once() + comment_call_args = self.mock_gh.post_issue_comment.call_args[0] + self.assertEqual(comment_call_args[0], 123) + self.assertIn( + "**New Release Candidate Tagged!** 🐍🌿", + comment_call_args[1], + ) + self.assertIn( + "- [Github Release 2.0.0-rc1](https://github.com/bazel-contrib/rules_python/releases/tag/2.0.0-rc1)", + comment_call_args[1], + ) + self.assertIn( + "- BCR Entry: [rules_python@2.0.0](https://registry.bazel.build/modules/rules_python/2.0.0)", + comment_call_args[1], + ) + self.assertIn( + "- [BCR PRs](https://github.com/bazelbuild/bazel-central-registry/pulls?q=is%3Apr+rules_python+2.0.0)", + comment_call_args[1], + ) + self.assertIn( + "- [Release workflow status](https://github.com/bazel-contrib/rules_python/actions/workflows/release_publish.yaml)", + comment_call_args[1], + ) + self.assertNotIn("🚀", comment_call_args[1]) + + def test_create_rc_gating_on_backports(self): + # Arrange + args = MagicMock(issue=123, remote="my-remote") + self.mock_gh.get_issue_title.return_value = "Release 2.0.0" + self.mock_gh.get_issue_body.return_value = """ +## Checklist +- [x] Prepare Release | status=done pr=#122 commit=abcdef12 +- [x] Create Release branch | status=done branch=release/2.0 commit=abcdef12 +- [ ] Tag RC0 | status=pending + +## Backports +- [ ] #124 | status=pending +""" + # Act + result = CreateRc(args, self.mock_git, self.mock_gh).run() + + # Assert + self.assertEqual(result, 1) + self.mock_git.tag.assert_not_called() + self.mock_git.push.assert_not_called() + + def test_create_rc_with_finished_backports(self): + # Arrange + args = MagicMock(issue=123, remote="my-remote") + self.mock_gh.get_issue_title.return_value = "Release 2.0.0" + self.mock_gh.get_issue_body.return_value = """ +## Checklist +- [x] Prepare Release | status=done pr=#122 commit=abcdef12 +- [x] Create Release branch | status=done branch=release/2.0 commit=abcdef12 +- [ ] Tag RC0 | status=pending + +## Backports +- [x] #124 | status=done rc=rc0 commit=abcdef12 +""" + self.mock_git.get_remote_tags.return_value = [] + self.mock_git.get_commit_sha.return_value = "1234567890" + + # Act + result = CreateRc(args, self.mock_git, self.mock_gh).run() + + # Assert + self.assertEqual(result, 0) + self.mock_git.tag.assert_called_once_with("2.0.0-rc0", "my-remote/release/2.0") + self.mock_git.push.assert_called_once_with("my-remote", "2.0.0-rc0") + + +if __name__ == "__main__": + unittest.main() diff --git a/tests/tools/private/release/create_release_branch_test.py b/tests/tools/private/release/create_release_branch_test.py new file mode 100644 index 0000000000..07e3652c85 --- /dev/null +++ b/tests/tools/private/release/create_release_branch_test.py @@ -0,0 +1,149 @@ +import unittest +from unittest.mock import MagicMock + +from tests.tools.private.release.release_test_helper import _mock_git_and_gh +from tools.private.release.create_release_branch import CreateReleaseBranch + + +class CmdCreateReleaseBranchTest(unittest.TestCase): + def setUp(self): + _mock_git_and_gh(self) + + def test_create_release_branch_success(self): + # Arrange + args = MagicMock(issue=123, remote="my-remote") + self.mock_gh.get_issue_title.return_value = "Release 2.0.0" + self.mock_gh.get_issue_body.return_value = """ +## Checklist +- [x] Prepare Release | status=done pr=#122 commit=abcdef12 +- [ ] Create Release branch | status=pending +""" + self.mock_git.branch_exists.return_value = False + self.mock_git.remote_branch_exists.return_value = False + + # Act + result = CreateReleaseBranch(args, self.mock_git, self.mock_gh).run() + + # Assert + self.assertEqual(result, 0) + self.mock_git.fetch.assert_called_once_with("my-remote") + self.mock_git.checkout.assert_not_called() + self.mock_git.push.assert_called_once_with( + "my-remote", "abcdef12:refs/heads/release/2.0" + ) + + self.mock_gh.update_issue_body.assert_called_once() + call_args = self.mock_gh.update_issue_body.call_args[0] + self.assertEqual(call_args[0], 123) + self.assertIn( + "branch_url=https://github.com/bazel-contrib/rules_python/tree/release/2.0", + call_args[1], + ) + self.assertIn("commit= abcdef12", call_args[1]) + + def test_create_release_branch_prepare_not_done(self): + # Arrange + args = MagicMock(issue=123, remote="my-remote") + self.mock_gh.get_issue_title.return_value = "Release 2.0.0" + self.mock_gh.get_issue_body.return_value = """ +## Checklist +- [ ] Prepare Release | status=pending +- [ ] Create Release branch | status=pending +""" + # Act + result = CreateReleaseBranch(args, self.mock_git, self.mock_gh).run() + + # Assert + self.assertEqual(result, 1) + self.mock_git.fetch.assert_not_called() + self.mock_git.push.assert_not_called() + self.mock_gh.update_issue_body.assert_not_called() + + def test_create_release_branch_already_checked(self): + # Arrange + args = MagicMock(issue=123, remote="my-remote") + self.mock_gh.get_issue_title.return_value = "Release 2.0.0" + self.mock_gh.get_issue_body.return_value = """ +## Checklist +- [x] Prepare Release | status=done pr=#122 commit=abcdef12 +- [x] Create Release branch | status=done branch=release/2.0 commit=abcdef12 +""" + # Act + result = CreateReleaseBranch(args, self.mock_git, self.mock_gh).run() + + # Assert + self.assertEqual(result, 0) + self.mock_git.fetch.assert_not_called() + self.mock_git.push.assert_not_called() + self.mock_gh.update_issue_body.assert_not_called() + + def test_create_release_branch_already_exists_same_commit(self): + # Arrange + args = MagicMock(issue=123, remote="my-remote") + self.mock_gh.get_issue_title.return_value = "Release 2.0.0" + self.mock_gh.get_issue_body.return_value = """ +## Checklist +- [x] Prepare Release | status=done pr=#122 commit=abcdef12 +- [ ] Create Release branch | status=pending +""" + self.mock_git.remote_branch_exists.return_value = True + self.mock_git.get_commit_sha.return_value = "abcdef12" + + # Act + result = CreateReleaseBranch(args, self.mock_git, self.mock_gh).run() + + # Assert + self.assertEqual(result, 0) + self.mock_git.fetch.assert_called_once_with("my-remote") + self.mock_git.push.assert_not_called() + self.mock_gh.update_issue_body.assert_called_once() # Should still update checklist + + def test_create_release_branch_already_exists_fast_forward(self): + # Arrange + args = MagicMock(issue=123, remote="my-remote") + self.mock_gh.get_issue_title.return_value = "Release 2.0.0" + self.mock_gh.get_issue_body.return_value = """ +## Checklist +- [x] Prepare Release | status=done pr=#122 commit=abcdef12 +- [ ] Create Release branch | status=pending +""" + self.mock_git.remote_branch_exists.return_value = True + self.mock_git.get_commit_sha.return_value = "oldcommit" + self.mock_git.is_ancestor.return_value = True + + # Act + result = CreateReleaseBranch(args, self.mock_git, self.mock_gh).run() + + # Assert + self.assertEqual(result, 0) + self.mock_git.fetch.assert_called_once_with("my-remote") + self.mock_git.push.assert_called_once_with( + "my-remote", "abcdef12:refs/heads/release/2.0" + ) + self.mock_gh.update_issue_body.assert_called_once() + + def test_create_release_branch_already_exists_non_ff(self): + # Arrange + args = MagicMock(issue=123, remote="my-remote") + self.mock_gh.get_issue_title.return_value = "Release 2.0.0" + self.mock_gh.get_issue_body.return_value = """ +## Checklist +- [x] Prepare Release | status=done pr=#122 commit=abcdef12 +- [ ] Create Release branch | status=pending +""" + self.mock_git.remote_branch_exists.return_value = True + self.mock_git.get_commit_sha.return_value = "othercommit" + self.mock_git.is_ancestor.return_value = False + + # Act + result = CreateReleaseBranch(args, self.mock_git, self.mock_gh).run() + + # Assert + self.assertEqual(result, 1) + self.mock_git.fetch.assert_called_once_with("my-remote") + self.mock_git.push.assert_not_called() + self.mock_gh.update_issue_body.assert_not_called() + + +if __name__ == "__main__": + unittest.main() diff --git a/tests/tools/private/release/git_test.py b/tests/tools/private/release/git_test.py new file mode 100644 index 0000000000..59b5c31989 --- /dev/null +++ b/tests/tools/private/release/git_test.py @@ -0,0 +1,48 @@ +import unittest +from unittest.mock import patch + +from tools.private.release.git import Git + + +class GitCheckoutTest(unittest.TestCase): + def setUp(self): + self.git = Git(".") + self.patcher = patch.object(self.git, "_run_git") + self.mock_run_git = self.patcher.start() + self.addCleanup(self.patcher.stop) + + def test_checkout_simple(self): + self.git.checkout("my-branch") + self.mock_run_git.assert_called_once_with( + "checkout", "my-branch", capture_output=False + ) + + @patch("tools.private.release.git.Git.branch_exists") + def test_checkout_track_remote_new_branch(self, mock_branch_exists): + mock_branch_exists.return_value = False + + self.git.checkout("my-branch", track_remote="origin") + + mock_branch_exists.assert_called_once_with("my-branch") + self.mock_run_git.assert_called_once_with( + "checkout", "--track", "origin/my-branch", capture_output=False + ) + + @patch("tools.private.release.git.Git.reset_hard") + @patch("tools.private.release.git.Git.branch_exists") + def test_checkout_track_remote_existing_branch( + self, mock_branch_exists, mock_reset_hard + ): + mock_branch_exists.return_value = True + + self.git.checkout("my-branch", track_remote="origin") + + mock_branch_exists.assert_called_once_with("my-branch") + self.mock_run_git.assert_called_once_with( + "checkout", "my-branch", capture_output=False + ) + mock_reset_hard.assert_called_once_with("origin/my-branch") + + +if __name__ == "__main__": + unittest.main() diff --git a/tests/tools/private/release/prepare_test.py b/tests/tools/private/release/prepare_test.py new file mode 100644 index 0000000000..3d92b131e2 --- /dev/null +++ b/tests/tools/private/release/prepare_test.py @@ -0,0 +1,238 @@ +import unittest +from unittest.mock import MagicMock, patch + +from tests.tools.private.release.release_test_helper import ( + TempDirTestCase, + _mock_git_and_gh, +) +from tools.private.release.gh import ( + MultipleTrackingIssuesError, + NoTrackingIssueError, +) +from tools.private.release.prepare import Prepare + + +class CmdPrepareTest(TempDirTestCase): + def setUp(self): + super().setUp() + _mock_git_and_gh(self) + + @patch("tools.private.release.prepare.changelog_news") + @patch("tools.private.release.prepare.replace_version_next") + def test_prepare_success_existing_issue(self, mock_replace, mock_changelog): + # Arrange + args = MagicMock(version="2.0.0", issue=None, dry_run=False) + self.mock_git.status.side_effect = ["", "M foo"] + self.mock_git.branch_exists.return_value = False + self.mock_gh.get_release_tracking_issue.side_effect = None + self.mock_gh.get_release_tracking_issue.return_value = 123 + self.mock_gh.create_pr.return_value = "https://github.com/foo/bar/pull/456" + self.mock_gh.get_issue_body.return_value = "- [ ] Prepare Release" + + # Act + result = Prepare(args, self.mock_git, self.mock_gh).run() + + # Assert + self.assertEqual(result, 0) + self.mock_gh.get_release_tracking_issue.assert_called_once_with("2.0.0") + self.mock_gh.create_tracking_issue.assert_not_called() + self.mock_gh.create_pr.assert_called_once_with("2.0.0", 123) + self.mock_git.add_modified_and_deleted.assert_called_once() + + @patch("tools.private.release.prepare.changelog_news") + @patch("tools.private.release.prepare.replace_version_next") + def test_prepare_success_create_issue(self, mock_replace, mock_changelog): + # Arrange + template_dir = self.tmpdir / ".github" / "ISSUE_TEMPLATE" + template_dir.mkdir(parents=True, exist_ok=True) + template_file = template_dir / "release_tracking_template.md" + template_file.write_text("dummy template content") + + args = MagicMock(version="2.0.0", issue=None, dry_run=False) + self.mock_git.status.side_effect = ["", "M foo"] + self.mock_git.branch_exists.return_value = False + self.mock_gh.get_release_tracking_issue.side_effect = NoTrackingIssueError( + "Not found" + ) + self.mock_gh.create_tracking_issue.return_value = 123 + self.mock_gh.create_pr.return_value = "https://github.com/foo/bar/pull/456" + self.mock_gh.get_issue_body.return_value = "- [ ] Prepare Release" + + # Act + result = Prepare(args, self.mock_git, self.mock_gh).run() + + # Assert + self.assertEqual(result, 0) + self.mock_gh.get_release_tracking_issue.assert_called_once_with("2.0.0") + self.mock_gh.create_tracking_issue.assert_called_once_with( + "2.0.0", "dummy template content" + ) + self.mock_gh.create_pr.assert_called_once_with("2.0.0", 123) + self.mock_git.add_modified_and_deleted.assert_called_once() + + @patch("tools.private.release.prepare.changelog_news") + @patch("tools.private.release.prepare.replace_version_next") + def test_prepare_ambiguous_issue(self, mock_replace, mock_changelog): + # Arrange + args = MagicMock(version="2.0.0", issue=None, dry_run=False) + self.mock_git.status.side_effect = ["", "M foo"] + self.mock_git.branch_exists.return_value = False + self.mock_gh.get_release_tracking_issue.side_effect = ( + MultipleTrackingIssuesError("Multiple open tracking issues") + ) + + # Act + result = Prepare(args, self.mock_git, self.mock_gh).run() + + # Assert + self.assertEqual(result, 1) + self.mock_gh.get_release_tracking_issue.assert_called_once_with("2.0.0") + self.mock_gh.create_tracking_issue.assert_not_called() + self.mock_gh.create_pr.assert_not_called() + self.mock_git.add_modified_and_deleted.assert_not_called() + + @patch("tools.private.release.prepare.changelog_news") + @patch("tools.private.release.prepare.replace_version_next") + def test_prepare_dry_run(self, mock_replace, mock_changelog): + # Arrange + args = MagicMock(version="2.0.0", issue=None, dry_run=True) + self.mock_git.status.side_effect = [""] + self.mock_gh.get_release_tracking_issue.side_effect = None + self.mock_gh.get_release_tracking_issue.return_value = 123 + + # Act + result = Prepare(args, self.mock_git, self.mock_gh).run() + + # Assert + self.assertEqual(result, 0) + self.mock_git.checkout.assert_not_called() + self.mock_git.commit.assert_not_called() + self.mock_git.push.assert_not_called() + self.mock_gh.create_pr.assert_not_called() + self.mock_gh.update_issue_body.assert_not_called() + self.mock_git.fetch.assert_called_once() + self.mock_gh.get_release_tracking_issue.assert_called_once_with("2.0.0") + self.mock_git.add_modified_and_deleted.assert_not_called() + + @patch("tools.private.release.prepare.changelog_news") + @patch("tools.private.release.prepare.replace_version_next") + def test_prepare_use_associated_pr_from_tracking_issue( + self, mock_replace, mock_changelog + ): + # Arrange + args = MagicMock(version="2.0.0", issue=None, dry_run=False) + self.mock_git.status.side_effect = ["", ""] + self.mock_git.branch_exists.return_value = True + self.mock_gh.get_release_tracking_issue.side_effect = None + self.mock_gh.get_release_tracking_issue.return_value = 123 + self.mock_gh.get_open_pr.return_value = None + # PR #456 is already associated in the tracking issue + self.mock_gh.get_issue_body.return_value = ( + "- [ ] Prepare Release | status=pending pr=#456" + ) + + # Act + result = Prepare(args, self.mock_git, self.mock_gh).run() + + # Assert + self.assertEqual(result, 0) + self.mock_git.checkout.assert_called_once_with("prepare-2.0.0") + self.mock_git.commit.assert_not_called() + self.mock_git.push.assert_called_once_with( + "origin", "prepare-2.0.0", set_upstream=True, force=True + ) + self.mock_gh.get_open_pr.assert_called_once_with("prepare-2.0.0") + self.mock_gh.create_pr.assert_not_called() # Should NOT create a new PR + self.mock_gh.update_issue_body.assert_called_once() + call_args = self.mock_gh.update_issue_body.call_args[0] + self.assertIn("pr=#456", call_args[1]) + + @patch("tools.private.release.prepare.changelog_news") + @patch("tools.private.release.prepare.replace_version_next") + def test_prepare_create_pr_when_none_associated(self, mock_replace, mock_changelog): + # Arrange + args = MagicMock(version="2.0.0", issue=None, dry_run=False) + self.mock_git.status.side_effect = ["", ""] + self.mock_git.branch_exists.return_value = True + self.mock_gh.get_release_tracking_issue.side_effect = None + self.mock_gh.get_release_tracking_issue.return_value = 123 + self.mock_gh.get_open_pr.return_value = None + # No PR associated in the tracking issue + self.mock_gh.get_issue_body.return_value = "- [ ] Prepare Release" + self.mock_gh.create_pr.return_value = "https://github.com/foo/bar/pull/789" + + # Act + result = Prepare(args, self.mock_git, self.mock_gh).run() + + # Assert + self.assertEqual(result, 0) + self.mock_git.checkout.assert_called_once_with("prepare-2.0.0") + self.mock_git.commit.assert_not_called() + self.mock_git.push.assert_called_once_with( + "origin", "prepare-2.0.0", set_upstream=True, force=True + ) + self.mock_gh.get_open_pr.assert_called_once_with("prepare-2.0.0") + self.mock_gh.create_pr.assert_called_once_with("2.0.0", 123) + self.mock_gh.update_issue_body.assert_called_once() + call_args = self.mock_gh.update_issue_body.call_args[0] + self.assertIn("pr=#789", call_args[1]) + + @patch("tools.private.release.prepare.changelog_news") + @patch("tools.private.release.prepare.replace_version_next") + def test_prepare_reuse_existing_pr(self, mock_replace, mock_changelog): + # Arrange + args = MagicMock(version="2.0.0", issue=None, dry_run=False) + self.mock_git.status.side_effect = ["", ""] + self.mock_git.branch_exists.return_value = True + self.mock_gh.get_release_tracking_issue.side_effect = None + self.mock_gh.get_release_tracking_issue.return_value = 123 + self.mock_gh.get_open_pr.return_value = { + "number": 456, + "url": "https://github.com/foo/bar/pull/456", + } + self.mock_gh.get_issue_body.return_value = "- [ ] Prepare Release" + + # Act + result = Prepare(args, self.mock_git, self.mock_gh).run() + + # Assert + self.assertEqual(result, 0) + self.mock_git.checkout.assert_called_once_with("prepare-2.0.0") + self.mock_git.commit.assert_not_called() + self.mock_git.push.assert_called_once_with( + "origin", "prepare-2.0.0", set_upstream=True, force=True + ) + self.mock_gh.get_open_pr.assert_called_once_with("prepare-2.0.0") + self.mock_gh.create_pr.assert_not_called() + self.mock_gh.update_issue_body.assert_called_once() + call_args = self.mock_gh.update_issue_body.call_args[0] + self.assertIn("pr=#456", call_args[1]) + + @patch("tools.private.release.prepare.changelog_news") + @patch("tools.private.release.prepare.replace_version_next") + def test_prepare_dry_run_no_issue(self, mock_replace, mock_changelog): + # Arrange + template_dir = self.tmpdir / ".github" / "ISSUE_TEMPLATE" + template_dir.mkdir(parents=True, exist_ok=True) + template_file = template_dir / "release_tracking_template.md" + template_file.write_text("dummy template content") + + args = MagicMock(version="2.0.0", issue=None, dry_run=True) + self.mock_git.status.side_effect = [""] + self.mock_gh.get_release_tracking_issue.side_effect = NoTrackingIssueError( + "Not found" + ) + + # Act + result = Prepare(args, self.mock_git, self.mock_gh).run() + + # Assert + self.assertEqual(result, 0) + self.mock_git.checkout.assert_not_called() + self.mock_gh.create_tracking_issue.assert_not_called() + self.mock_gh.create_pr.assert_not_called() + self.mock_git.add_modified_and_deleted.assert_not_called() + + +if __name__ == "__main__": + unittest.main() diff --git a/tests/tools/private/release/process_backports_test.py b/tests/tools/private/release/process_backports_test.py new file mode 100644 index 0000000000..0ceb7b3210 --- /dev/null +++ b/tests/tools/private/release/process_backports_test.py @@ -0,0 +1,239 @@ +import datetime +import unittest +from unittest.mock import MagicMock, call, patch + +from tests.tools.private.release.release_test_helper import _mock_git_and_gh +from tools.private.release.process_backports import ProcessBackports + + +class CmdProcessBackportsTest(unittest.TestCase): + def setUp(self): + _mock_git_and_gh(self) + self.mock_changelog_news = patch( + "tools.private.release.process_backports.changelog_news" + ).start() + self.mock_replace_version_next = patch( + "tools.private.release.process_backports.replace_version_next" + ).start() + self.addCleanup(patch.stopall) + + def test_process_backports_no_pending(self): + args = MagicMock(issue=123, remote="origin", dry_run=False) + self.mock_gh.get_issue_body.return_value = "No backports here" + + result = ProcessBackports(args, self.mock_git, self.mock_gh).run() + + self.assertEqual(result, 0) + self.mock_gh.get_issue_body.assert_called_once_with(123) + self.mock_git.fetch.assert_not_called() + + @patch("tools.private.release.process_backports.datetime") + def test_process_backports_success(self, mock_datetime): + mock_datetime.date.today.return_value = datetime.date(2026, 7, 1) + args = MagicMock(issue=123, remote="origin", dry_run=False) + self.mock_gh.get_issue_title.return_value = "Release 2.0.0" + self.mock_gh.get_issue_body.return_value = """ +## Checklist +- [ ] Prepare Release +- [ ] Create Release branch + +## Backports +- [ ] #124 | status=pending +""" + self.mock_git.get_remote_tags.return_value = [] + + def mock_resolve(items): + for item in items: + if item.pr_ref == "#124": + item.commit = "abcdef12" + item.status = "done" + return items + + self.mock_gh.get_merge_commits_for_prs.side_effect = mock_resolve + + self.mock_git.sort_commits_chronologically.return_value = ["abcdef12"] + self.mock_git.get_commit_sha.return_value = "12345678" + self.mock_git.get_commit_message.return_value = 'Cherry-pick "fix bug"' + + result = ProcessBackports(args, self.mock_git, self.mock_gh).run() + + self.assertEqual(result, 0) + self.mock_git.fetch.assert_has_calls( + [call("origin", tags=True, force=True), call("origin")] + ) + self.mock_git.checkout.assert_called_once_with( + "release/2.0", track_remote="origin" + ) + self.mock_git.cherry_pick.assert_called_once_with("abcdef12") + self.mock_changelog_news.update_changelog.assert_called_once_with( + "2.0.0", "2026-07-01" + ) + self.mock_git.add_modified_and_deleted.assert_called_once() + self.mock_replace_version_next.assert_called_once_with("2.0.0") + self.mock_git.commit.assert_called_once_with( + 'Cherry-pick "fix bug"\n\nWork towards #123', amend=True + ) + self.mock_git.push.assert_called_once_with("origin", "release/2.0") + + self.mock_gh.update_issue_body.assert_called_once() + call_args = self.mock_gh.update_issue_body.call_args[0] + self.assertEqual(call_args[0], 123) + self.assertIn("- [x] #124 | status=done rc=rc0 commit= 12345678", call_args[1]) + + @patch("tools.private.release.process_backports.datetime") + def test_process_backports_dry_run(self, mock_datetime): + mock_datetime.date.today.return_value = datetime.date(2026, 7, 1) + args = MagicMock(issue=123, remote="origin", dry_run=True) + self.mock_gh.get_issue_title.return_value = "Release 2.0.0" + self.mock_gh.get_issue_body.return_value = """ +## Checklist +- [ ] Prepare Release +- [ ] Create Release branch + +## Backports +- [ ] #124 | status=pending +""" + self.mock_git.get_remote_tags.return_value = [] + + def mock_resolve(items): + for item in items: + if item.pr_ref == "#124": + item.commit = "abcdef12" + item.status = "done" + return items + + self.mock_gh.get_merge_commits_for_prs.side_effect = mock_resolve + + self.mock_git.sort_commits_chronologically.return_value = ["abcdef12"] + self.mock_git.get_commit_sha.return_value = "12345678" + self.mock_git.get_commit_message.return_value = 'Cherry-pick "fix bug"' + + result = ProcessBackports(args, self.mock_git, self.mock_gh).run() + + self.assertEqual(result, 0) + self.mock_git.fetch.assert_has_calls( + [call("origin", tags=True, force=True), call("origin")] + ) + self.mock_git.checkout.assert_called_once_with( + "release/2.0", track_remote="origin" + ) + self.mock_git.cherry_pick.assert_called_once_with("abcdef12") + self.mock_changelog_news.update_changelog.assert_called_once_with( + "2.0.0", "2026-07-01" + ) + self.mock_git.add_modified_and_deleted.assert_called_once() + self.mock_replace_version_next.assert_called_once_with("2.0.0") + self.mock_git.commit.assert_called_once_with( + 'Cherry-pick "fix bug"\n\nWork towards #123', amend=True + ) + self.mock_git.reset_hard.assert_called_once_with("12345678") + self.mock_git.push.assert_not_called() + self.mock_gh.update_issue_body.assert_not_called() + + def test_process_backports_ignored_and_failed_states(self): + args = MagicMock(issue=123, remote="origin", dry_run=False) + self.mock_gh.get_issue_title.return_value = "Release 2.0.0" + self.mock_gh.get_issue_body.return_value = """ +## Checklist +- [ ] Prepare Release +- [ ] Create Release branch + +## Backports +- [ ] #124 | status=pending +- [ ] #125 | status=pending +- [ ] #126 | status=pending +""" + self.mock_git.get_remote_tags.return_value = [] + + def mock_resolve(items): + for item in items: + if item.pr_ref == "#124": + item.status = "open-pr" + elif item.pr_ref == "#125": + item.status = "draft-pr" + elif item.pr_ref == "#126": + item.status = "error-closed-pr" + return items + + self.mock_gh.get_merge_commits_for_prs.side_effect = mock_resolve + + result = ProcessBackports(args, self.mock_git, self.mock_gh).run() + + self.assertEqual(result, 1) + self.mock_gh.update_issue_body.assert_called_once() + call_args = self.mock_gh.update_issue_body.call_args[0] + self.assertEqual(call_args[0], 123) + self.assertIn("- [ ] #126 | status=error-closed-pr", call_args[1]) + self.assertNotIn("status=open-pr", call_args[1]) + self.assertNotIn("status=draft-pr", call_args[1]) + self.mock_git.checkout.assert_not_called() + self.mock_git.cherry_pick.assert_not_called() + + def test_process_backports_ignored_error_status(self): + args = MagicMock(issue=123, remote="origin", dry_run=False) + self.mock_gh.get_issue_title.return_value = "Release 2.0.0" + self.mock_gh.get_issue_body.return_value = """ +## Checklist +- [ ] Prepare Release +- [ ] Create Release branch + +## Backports +- [ ] #124 | status=error-merge-conflict +- [ ] #125 | status=error-some-other-error +""" + self.mock_git.get_remote_tags.return_value = [] + self.mock_gh.get_merge_commits_for_prs.return_value = [] + + result = ProcessBackports(args, self.mock_git, self.mock_gh).run() + + self.assertEqual(result, 0) + self.mock_gh.get_merge_commits_for_prs.assert_not_called() + self.mock_git.checkout.assert_not_called() + + @patch("tools.private.release.process_backports.datetime") + def test_process_backports_cherry_pick_failed(self, mock_datetime): + mock_datetime.date.today.return_value = datetime.date(2026, 7, 1) + args = MagicMock(issue=123, remote="origin", dry_run=False) + self.mock_gh.get_issue_title.return_value = "Release 2.0.0" + self.mock_gh.get_issue_body.return_value = """ +## Checklist +- [ ] Prepare Release +- [ ] Create Release branch + +## Backports +- [ ] #124 | status=pending +""" + self.mock_git.get_remote_tags.return_value = [] + + def mock_resolve(items): + for item in items: + if item.pr_ref == "#124": + item.commit = "abcdef12" + item.status = "done" + return items + + self.mock_gh.get_merge_commits_for_prs.side_effect = mock_resolve + + self.mock_git.sort_commits_chronologically.return_value = ["abcdef12"] + self.mock_git.cherry_pick.side_effect = Exception("Cherry-pick conflict") + + result = ProcessBackports(args, self.mock_git, self.mock_gh).run() + + self.assertEqual(result, 1) + self.mock_git.checkout.assert_called_once_with( + "release/2.0", track_remote="origin" + ) + self.mock_git.cherry_pick.assert_called_once_with("abcdef12") + self.mock_git.cherry_pick_abort.assert_called_once() + + self.mock_gh.update_issue_body.assert_called_once() + call_args = self.mock_gh.update_issue_body.call_args[0] + self.assertEqual(call_args[0], 123) + self.assertIn("- [ ] #124 | status=error-merge-conflict", call_args[1]) + + self.mock_git.commit.assert_not_called() + self.mock_git.push.assert_not_called() + + +if __name__ == "__main__": + unittest.main() diff --git a/tests/tools/private/release/promote_rc_test.py b/tests/tools/private/release/promote_rc_test.py new file mode 100644 index 0000000000..71980dc1fd --- /dev/null +++ b/tests/tools/private/release/promote_rc_test.py @@ -0,0 +1,236 @@ +import unittest +from unittest.mock import MagicMock, call, patch + +from tests.tools.private.release.release_test_helper import _mock_git_and_gh +from tools.private.release.gh import NoTrackingIssueError +from tools.private.release.promote_rc import PromoteRc + + +class CmdPromoteRcTest(unittest.TestCase): + def setUp(self): + _mock_git_and_gh(self) + + def test_promote_rc_success(self): + # Arrange + args = MagicMock(version="2.0.0", issue=123, dry_run=False, remote="my-remote") + self.mock_git.get_remote_tags.return_value = ["2.0.0-rc0", "2.0.0-rc1"] + self.mock_git.get_commit_sha.return_value = "abcdef123456" + self.mock_git.tag_exists.return_value = False + initial_body = "- [ ] Tag Final" + self.mock_gh.get_issue_body.return_value = initial_body + + # Act + result = PromoteRc(args, self.mock_git, self.mock_gh).run() + + # Assert + self.assertEqual(result, 0) + self.mock_git.fetch.assert_called_once_with("my-remote", tags=True, force=True) + self.mock_git.get_commit_sha.assert_called_once_with("2.0.0-rc1") + self.mock_git.checkout.assert_not_called() + self.mock_git.tag_exists.assert_called_once_with("2.0.0") + self.mock_git.tag.assert_called_once_with("2.0.0", "abcdef123456") + self.mock_git.push.assert_called_once_with("my-remote", "2.0.0") + + # Verify issue update + self.mock_gh.get_issue_body.assert_called_once_with(123) + expected_updated_body = ( + "- [x] Tag Final | status=done tag=2.0.0 commit= abcdef12" + ) + self.mock_gh.update_issue_body.assert_called_once_with( + 123, expected_updated_body + ) + expected_comment = ( + "Version 2.0.0 has been tagged.\n\n" + "- **Release Page**: https://github.com/bazel-contrib/rules_python/releases/tag/2.0.0\n" + '- **BCR PR Search**: [is:pr ("bazel-contrib/rules_python" in:title) ("@2.0.0" in:title)](https://github.com/bazelbuild/bazel-central-registry/pulls?q=is%3Apr%20%28%22bazel-contrib/rules_python%22%20in%3Atitle%29%20%28%22%402.0.0%22%20in%3Atitle%29)' + ) + self.mock_gh.post_issue_comment.assert_called_once_with(123, expected_comment) + + def test_promote_rc_resolve_issue_success(self): + # Arrange + args = MagicMock(version="2.0.0", issue=None, dry_run=False, remote="my-remote") + self.mock_git.get_remote_tags.return_value = ["2.0.0-rc1"] + self.mock_git.tag_exists.return_value = False + self.mock_gh.get_release_tracking_issue.side_effect = None + self.mock_gh.get_release_tracking_issue.return_value = 123 + self.mock_git.get_commit_sha.return_value = "abcdef123456" + initial_body = "- [ ] Tag Final" + self.mock_gh.get_issue_body.return_value = initial_body + + # Act + result = PromoteRc(args, self.mock_git, self.mock_gh).run() + + # Assert + self.assertEqual(result, 0) + self.mock_git.fetch.assert_called_once_with("my-remote", tags=True, force=True) + self.mock_gh.get_release_tracking_issue.assert_called_once_with("2.0.0") + self.mock_git.get_commit_sha.assert_called_once_with("2.0.0-rc1") + self.mock_git.checkout.assert_not_called() + self.mock_git.tag.assert_called_once_with("2.0.0", "abcdef123456") + self.mock_git.push.assert_called_once_with("my-remote", "2.0.0") + self.mock_gh.get_issue_body.assert_called_once_with(123) + expected_updated_body = ( + "- [x] Tag Final | status=done tag=2.0.0 commit= abcdef12" + ) + self.mock_gh.update_issue_body.assert_called_once_with( + 123, expected_updated_body + ) + expected_comment = ( + "Version 2.0.0 has been tagged.\n\n" + "- **Release Page**: https://github.com/bazel-contrib/rules_python/releases/tag/2.0.0\n" + '- **BCR PR Search**: [is:pr ("bazel-contrib/rules_python" in:title) ("@2.0.0" in:title)](https://github.com/bazelbuild/bazel-central-registry/pulls?q=is%3Apr%20%28%22bazel-contrib/rules_python%22%20in%3Atitle%29%20%28%22%402.0.0%22%20in%3Atitle%29)' + ) + self.mock_gh.post_issue_comment.assert_called_once_with(123, expected_comment) + + def test_promote_rc_defaults_to_determine_next_version(self): + # Arrange + args = MagicMock(version=None, issue=123, dry_run=False, remote="my-remote") + self.mock_git.get_current_branch.return_value = "release/2.0" + self.mock_git.get_tags.return_value = ["2.0.0"] + self.mock_git.get_remote_tags.return_value = ["2.0.1-rc0"] + self.mock_git.get_commit_sha.return_value = "12345678" + self.mock_git.tag_exists.return_value = False + initial_body = "- [ ] Tag Final" + self.mock_gh.get_issue_body.return_value = initial_body + + # Act + result = PromoteRc(args, self.mock_git, self.mock_gh).run() + + # Assert + self.assertEqual(result, 0) + self.mock_git.fetch.assert_called_once_with("my-remote", tags=True, force=True) + self.mock_git.get_current_branch.assert_called_once() + self.mock_git.get_tags.assert_called_once() + self.mock_git.get_remote_tags.assert_called_once_with("my-remote") + + self.mock_git.checkout.assert_not_called() + self.mock_git.get_commit_sha.assert_called_once_with("2.0.1-rc0") + self.mock_git.tag.assert_called_once_with("2.0.1", "12345678") + self.mock_git.push.assert_called_once_with("my-remote", "2.0.1") + + expected_updated_body = ( + "- [x] Tag Final | status=done tag=2.0.1 commit= 12345678" + ) + self.mock_gh.update_issue_body.assert_called_once_with( + 123, expected_updated_body + ) + expected_comment = ( + "Version 2.0.1 has been tagged.\n\n" + "- **Release Page**: https://github.com/bazel-contrib/rules_python/releases/tag/2.0.1\n" + '- **BCR PR Search**: [is:pr ("bazel-contrib/rules_python" in:title) ("@2.0.1" in:title)](https://github.com/bazelbuild/bazel-central-registry/pulls?q=is%3Apr%20%28%22bazel-contrib/rules_python%22%20in%3Atitle%29%20%28%22%402.0.1%22%20in%3Atitle%29)' + ) + self.mock_gh.post_issue_comment.assert_called_once_with(123, expected_comment) + + @patch("builtins.print") + def test_promote_rc_dry_run_success(self, mock_print): + # Arrange + args = MagicMock(version="2.0.0", issue=123, dry_run=True, remote="my-remote") + self.mock_git.get_remote_tags.return_value = ["2.0.0-rc0", "2.0.0-rc1"] + self.mock_git.get_commit_sha.return_value = "abcdef123456" + self.mock_git.tag_exists.return_value = False + initial_body = "- [ ] Tag Final" + self.mock_gh.get_issue_body.return_value = initial_body + + # Act + result = PromoteRc(args, self.mock_git, self.mock_gh).run() + + # Assert + self.assertEqual(result, 0) + self.mock_git.fetch.assert_called_once_with("my-remote", tags=True, force=True) + self.mock_git.get_commit_sha.assert_called_once_with("2.0.0-rc1") + self.mock_git.tag_exists.assert_called_once_with("2.0.0") + + # Core dry-run assertions: NO modifications + self.mock_git.tag.assert_not_called() + self.mock_git.push.assert_not_called() + self.mock_gh.update_issue_body.assert_not_called() + self.mock_gh.post_issue_comment.assert_not_called() + + mock_print.assert_has_calls( + [ + call("Verifying tracking issue #123 format..."), + call( + "[DRY RUN] Pre-conditions passed successfully for promoting" + " 2.0.0-rc1 to 2.0.0." + ), + call("[DRY RUN] Would tag commit abcdef12 as 2.0.0"), + call("[DRY RUN] Would push tag 2.0.0 to my-remote"), + call("[DRY RUN] Would update tracking issue #123 checklist"), + call("[DRY RUN] Would post comment to tracking issue #123"), + ] + ) + + def test_promote_rc_tag_already_exists(self): + # Arrange + args = MagicMock(version="2.0.0", issue=123, remote="my-remote") + self.mock_git.get_remote_tags.return_value = ["2.0.0-rc1"] + self.mock_git.tag_exists.return_value = True + + # Act + result = PromoteRc(args, self.mock_git, self.mock_gh).run() + + # Assert + self.assertEqual(result, 1) + self.mock_git.checkout.assert_not_called() + self.mock_git.tag.assert_not_called() + self.mock_git.push.assert_not_called() + self.mock_gh.get_issue_body.assert_not_called() + self.mock_gh.update_issue_body.assert_not_called() + + def test_promote_rc_issue_not_found(self): + # Arrange + args = MagicMock(version="2.0.0", issue=None, remote="my-remote") + self.mock_git.get_remote_tags.return_value = ["2.0.0-rc1"] + self.mock_git.tag_exists.return_value = False + self.mock_gh.get_release_tracking_issue.side_effect = NoTrackingIssueError( + "Not found" + ) + + # Act + result = PromoteRc(args, self.mock_git, self.mock_gh).run() + + # Assert + self.assertEqual(result, 1) + self.mock_gh.get_release_tracking_issue.assert_called_once_with("2.0.0") + self.mock_git.checkout.assert_not_called() + self.mock_git.tag.assert_not_called() + self.mock_git.push.assert_not_called() + self.mock_gh.get_issue_body.assert_not_called() + + def test_promote_rc_issue_malformed(self): + # Arrange + args = MagicMock(version="2.0.0", issue=123, remote="my-remote") + self.mock_git.get_remote_tags.return_value = ["2.0.0-rc1"] + self.mock_git.tag_exists.return_value = False + self.mock_git.get_commit_sha.return_value = "abcdef123456" + initial_body = "malformed body" + self.mock_gh.get_issue_body.return_value = initial_body + + # Act + result = PromoteRc(args, self.mock_git, self.mock_gh).run() + + # Assert + self.assertEqual(result, 1) + self.mock_gh.get_issue_body.assert_called_once_with(123) + self.mock_git.checkout.assert_not_called() + self.mock_git.tag.assert_not_called() + self.mock_git.push.assert_not_called() + self.mock_gh.update_issue_body.assert_not_called() + + def test_promote_rc_no_rc_found(self): + # Arrange + args = MagicMock(version="2.0.0", issue=123, remote="my-remote") + self.mock_git.get_remote_tags.return_value = [] + + # Act + result = PromoteRc(args, self.mock_git, self.mock_gh).run() + + # Assert + self.assertEqual(result, 1) + self.mock_git.checkout.assert_not_called() + self.mock_git.tag.assert_not_called() + self.mock_gh.get_issue_body.assert_not_called() + + +if __name__ == "__main__": + unittest.main() diff --git a/tests/tools/private/release/release_issue_test.py b/tests/tools/private/release/release_issue_test.py new file mode 100644 index 0000000000..edf5f603f0 --- /dev/null +++ b/tests/tools/private/release/release_issue_test.py @@ -0,0 +1,70 @@ +import unittest + +from tools.private.release.release_issue import ( + format_metadata_line, + parse_metadata_line, +) + + +class ReleaseIssueTest(unittest.TestCase): + def test_parse_metadata_line_spaces(self): + # Test with spaces around '=' + line = "- [ ] Tag Final | tag = 2.0.0 commit = abcdef12" + expected = { + "checked": False, + "name": "Tag Final", + "metadata": {"tag": "2.0.0", "commit": "abcdef12"}, + "original_line": line, + } + self.assertEqual(parse_metadata_line(line), expected) + + # Test with spaces after '=' + line = "- [ ] Tag Final | tag= 2.0.0 commit= abcdef12" + expected = { + "checked": False, + "name": "Tag Final", + "metadata": {"tag": "2.0.0", "commit": "abcdef12"}, + "original_line": line, + } + self.assertEqual(parse_metadata_line(line), expected) + + # Test with standard format (no spaces) + line = "- [ ] Tag Final | tag=2.0.0 commit=abcdef12" + expected = { + "checked": False, + "name": "Tag Final", + "metadata": {"tag": "2.0.0", "commit": "abcdef12"}, + "original_line": line, + } + self.assertEqual(parse_metadata_line(line), expected) + + # Test with no metadata + line = "- [ ] Tag Final" + expected = { + "checked": False, + "name": "Tag Final", + "metadata": {}, + "original_line": line, + } + self.assertEqual(parse_metadata_line(line), expected) + + def test_format_metadata_line(self): + # Test with commit metadata (should have space) + metadata = {"status": "done", "tag": "2.0.0", "commit": "abcdef12"} + expected = "- [x] Tag Final | status=done tag=2.0.0 commit= abcdef12" + self.assertEqual(format_metadata_line(True, "Tag Final", metadata), expected) + + # Test with other metadata (should not have space) + metadata = {"status": "done", "pr": "#122"} + expected = "- [x] Prepare Release | status=done pr=#122" + self.assertEqual( + format_metadata_line(True, "Prepare Release", metadata), expected + ) + + # Test with no metadata + expected = "- [ ] Tag Final" + self.assertEqual(format_metadata_line(False, "Tag Final", {}), expected) + + +if __name__ == "__main__": + unittest.main() diff --git a/tests/tools/private/release/release_test.py b/tests/tools/private/release/release_test.py index 2a752ef1cc..c7ba1b17b7 100644 --- a/tests/tools/private/release/release_test.py +++ b/tests/tools/private/release/release_test.py @@ -1,516 +1,9 @@ -import datetime -import os -import pathlib -import shutil -import tempfile import unittest -from unittest.mock import MagicMock, call, patch -from tools.private.release import changelog_news, release as releaser, utils -from tools.private.release.create_rc import CreateRc -from tools.private.release.create_release_branch import CreateReleaseBranch -from tools.private.release.gh import ( - MultipleTrackingIssuesError, - NoTrackingIssueError, -) -from tools.private.release.git import Git -from tools.private.release.prepare import Prepare -from tools.private.release.process_backports import ProcessBackports -from tools.private.release.promote_rc import PromoteRc +from tools.private.release import release as releaser -def _mock_git_and_gh(test_case): - mock_git = MagicMock() - mock_gh = MagicMock() - test_case.mock_git = mock_git - test_case.mock_gh = mock_gh - - # Mock Git inside utils.py since it instantiates it locally - patch("tools.private.release.utils.Git", return_value=mock_git).start() - - mock_gh.MultipleTrackingIssuesError = MultipleTrackingIssuesError - mock_gh.NoTrackingIssueError = NoTrackingIssueError - - test_case.addCleanup(patch.stopall) - - # Apply safe defaults - mock_git.get_current_branch.return_value = None - mock_git.get_tags.return_value = [] - mock_git.get_remote_tags.return_value = [] - - mock_git.status.return_value = "" - mock_git.branch_exists.return_value = False - mock_git.tag_exists.return_value = False - mock_gh.get_release_tracking_issue.side_effect = NoTrackingIssueError("Not found") - mock_gh.get_open_pr.return_value = None - - -class TempDirTestCase(unittest.TestCase): - def setUp(self): - self.tmpdir = pathlib.Path(tempfile.mkdtemp()) - self.original_cwd = os.getcwd() - self.addCleanup(shutil.rmtree, self.tmpdir) - os.chdir(self.tmpdir) - self.addCleanup(os.chdir, self.original_cwd) - - -class ReleaserTest(TempDirTestCase): - def test_update_changelog_with_news(self): - # Arrange - changelog = """# Changelog - -{#unreleased} -## Unreleased - -[unreleased]: https://github.com/bazel-contrib/rules_python/releases/tag/unreleased - -{#unreleased-removed} -### Removed -* Nothing removed. - -{#unreleased-changed} -### Changed -* Nothing changed. - -{#unreleased-fixed} -### Fixed -* Nothing fixed. - -{#unreleased-added} -### Added -* Nothing added. - -{#v2-0-2} -## [2.0.2] - 2026-05-14 - -[2.0.2]: https://github.com/bazel-contrib/rules_python/releases/tag/2.0.2 - -{#v2-0-2-added} -### Added -* (toolchains) Some older change. -""" - changelog_path = self.tmpdir / "CHANGELOG.md" - changelog_path.write_text(changelog) - - news_dir = self.tmpdir / "news" - news_dir.mkdir() - - # Create news files - (news_dir / "123.fixed.md").write_text("Fixed a bug in the compiler") - # Test that it handles prefixing "* " if not present - (news_dir / "456.added.md").write_text("* Added a new feature for Python 3.13") - # Empty file should be ignored - (news_dir / "789.changed.md").write_text("") - # Invalid name should be ignored - (news_dir / "invalid_name.md").write_text("Should be ignored") - - # Act - changelog_news.update_changelog( - "3.0.0", - "2026-06-16", - changelog_path=changelog_path, - news_dir=news_dir, - ) - - # Assert - # 1. News files matching the pattern should be deleted (even empty ones) - self.assertFalse((news_dir / "123.fixed.md").exists()) - self.assertFalse((news_dir / "456.added.md").exists()) - self.assertFalse((news_dir / "789.changed.md").exists()) - # Invalid name does not match pattern -> NOT deleted - self.assertTrue((news_dir / "invalid_name.md").exists()) - - new_content = changelog_path.read_text() - - # 2. A fresh active Unreleased section should be present - self.assertIn("{#unreleased}", new_content) - self.assertIn("## Unreleased", new_content) - self.assertIn( - "Unreleased changes are tracked as individual files in the [news/](./news)\n" - "directory, or view the [latest generated\n" - "changelog](https://rules-python.readthedocs.io/en/latest/changelog.html).", - new_content, - ) - - # 3. The new release section should be present - self.assertIn("{#v3-0-0}", new_content) - self.assertIn("## [3.0.0] - 2026-06-16", new_content) - self.assertIn( - "[3.0.0]: https://github.com/bazel-contrib/rules_python/releases/tag/3.0.0", - new_content, - ) - - # 4. Correct categories and content - self.assertIn( - "{#v3-0-0-fixed}\n### Fixed\n* Fixed a bug in the compiler", new_content - ) - self.assertIn( - "{#v3-0-0-added}\n### Added\n* Added a new feature for Python 3.13", - new_content, - ) - - # 5. Omitted categories should NOT be present in the new release - self.assertNotIn("{#v3-0-0-removed}", new_content) - self.assertNotIn("{#v3-0-0-changed}", new_content) - - # 6. Old release should still be there - self.assertIn("{#v2-0-2}", new_content) - self.assertIn("## [2.0.2] - 2026-05-14", new_content) - - def test_update_changelog_sorting(self): - # Arrange - changelog = """# Changelog - -{#unreleased} -## Unreleased - -[unreleased]: https://github.com/bazel-contrib/rules_python/releases/tag/unreleased - -Unreleased changes are tracked as individual files in the [news/](./news) -directory, or view the [latest generated -changelog](https://rules-python.readthedocs.io/en/latest/changelog.html). - -{#v2-0-2} -## [2.0.2] - 2026-05-14 - -[2.0.2]: https://github.com/bazel-contrib/rules_python/releases/tag/2.0.2 - -{#v2-0-2-added} -### Added -* (toolchains) Some older change. -""" - changelog_path = self.tmpdir / "CHANGELOG.md" - changelog_path.write_text(changelog) - - news_dir = self.tmpdir / "news" - news_dir.mkdir() - - # Create news files with different sub-categories and some without - (news_dir / "1.fixed.md").write_text("* (zebra) Zebra fix") - (news_dir / "2.fixed.md").write_text("* (apple) Apple fix") - (news_dir / "3.fixed.md").write_text("No subcategory B") - (news_dir / "4.fixed.md").write_text("* (apple) Another apple fix") - (news_dir / "5.fixed.md").write_text("No subcategory A") - - # Act - changelog_news.update_changelog( - "3.0.0", - "2026-06-16", - changelog_path=changelog_path, - news_dir=news_dir, - ) - - # Assert - new_content = changelog_path.read_text() - - # Expected order in Fixed section: - # 1. No subcategory A - # 2. No subcategory B - # 3. (apple) Another apple fix - # 4. (apple) Apple fix - # 5. (zebra) Zebra fix - - expected_fixed_section = ( - "### Fixed\n" - "* No subcategory A\n" - "* No subcategory B\n" - "* (apple) Another apple fix\n" - "* (apple) Apple fix\n" - "* (zebra) Zebra fix\n" - ) - - self.assertIn(expected_fixed_section, new_content) - - def test_update_changelog_read_failure(self): - # Arrange - original_read_text = pathlib.Path.read_text - - with patch("pathlib.Path.read_text", autospec=True) as mock_read_text: - - def side_effect(path_self, *args, **kwargs): - if "bad_file.fixed.md" in str(path_self): - raise IOError("Simulated read error") - return original_read_text(path_self, *args, **kwargs) - - mock_read_text.side_effect = side_effect - - changelog = """# Changelog - -{#unreleased} -## Unreleased - -[unreleased]: https://github.com/bazel-contrib/rules_python/releases/tag/unreleased - -Unreleased changes are tracked as individual files in the [news/](./news) -directory, or view the [latest generated -changelog](https://rules-python.readthedocs.io/en/latest/changelog.html). - -{#v2-0-2} -## [2.0.2] - 2026-05-14 - -[2.0.2]: https://github.com/bazel-contrib/rules_python/releases/tag/2.0.2 - -{#v2-0-2-added} -### Added -* (toolchains) Some older change. -""" - changelog_path = self.tmpdir / "CHANGELOG.md" - changelog_path.write_text(changelog) - - news_dir = self.tmpdir / "news" - news_dir.mkdir() - - # Create the bad file (must exist so it is found by iterdir) - bad_file = news_dir / "bad_file.fixed.md" - bad_file.write_text("some content that won't be read") - - # Create a good file too - good_file = news_dir / "good_file.fixed.md" - good_file.write_text("* (sub) Good fix") - - # Act & Assert - # It should raise IOError - with self.assertRaises(IOError): - changelog_news.update_changelog( - "3.0.0", - "2026-06-16", - changelog_path=changelog_path, - news_dir=news_dir, - ) - - # Both files should still exist (no deletion on failure!) - self.assertTrue(bad_file.exists()) - self.assertTrue(good_file.exists()) - - # Changelog should not be modified - new_content = changelog_path.read_text() - self.assertEqual(changelog, new_content) - - def test_update_changelog_merge_existing(self): - # Arrange - changelog = """# Changelog - -{#unreleased} -## Unreleased - -[unreleased]: https://github.com/bazel-contrib/rules_python/releases/tag/unreleased - -Unreleased changes are tracked as individual files in the [news/](./news) -directory, or view the [latest generated -changelog](https://rules-python.readthedocs.io/en/latest/changelog.html). - -{#v2-0-3} -## [2.0.3] - 2026-06-15 - -[2.0.3]: https://github.com/bazel-contrib/rules_python/releases/tag/2.0.3 - -{#v2-0-3-fixed} -### Fixed -* (pypi) Old fix - multi-line detail - * nested bullet item -* (pypi) Z old fix -""" - changelog_path = self.tmpdir / "CHANGELOG.md" - changelog_path.write_text(changelog) - - news_dir = self.tmpdir / "news" - news_dir.mkdir() - - # Create news files to merge - # 1. New fix in same category (should merge and sort) - (news_dir / "1.fixed.md").write_text("(pypi) New fix") - # 2. New entry in new category (should create category) - (news_dir / "2.added.md").write_text("(toolchains) New feature") - - # Act - changelog_news.update_changelog( - "2.0.3", - "2026-06-15", - changelog_path=changelog_path, - news_dir=news_dir, - ) - - # Assert - # News files should be deleted - self.assertFalse((news_dir / "1.fixed.md").exists()) - self.assertFalse((news_dir / "2.added.md").exists()) - - new_content = changelog_path.read_text() - - # Expected merged and sorted Fixed section: - # 1. (pypi) New fix (New < Old) - # 2. (pypi) Old fix (with its multi-line detail!) - # 3. (pypi) Z old fix - expected_fixed_section = ( - "### Fixed\n" - "* (pypi) New fix\n" - "* (pypi) Old fix\n" - " multi-line detail\n" - " * nested bullet item\n" - "* (pypi) Z old fix\n" - ) - self.assertIn(expected_fixed_section, new_content) - - # Expected created Added section: - expected_added_section = "### Added\n* (toolchains) New feature\n" - self.assertIn(expected_added_section, new_content) - - # Active Unreleased section should NOT be touched (should still be empty/pointing to news) - self.assertIn("Unreleased changes are tracked as individual files", new_content) - - def test_update_changelog_does_not_leak(self): - # Arrange - changelog = """# Changelog - -{#unreleased} -## Unreleased - -[unreleased]: https://github.com/bazel-contrib/rules_python/releases/tag/unreleased - -Unreleased changes are tracked as individual files in the [news/](./news) -directory, or view the [latest generated -changelog](https://rules-python.readthedocs.io/en/latest/changelog.html). - -{#v2-0-2} -## [2.0.2] - 2026-05-14 - -[2.0.2]: https://github.com/bazel-contrib/rules_python/releases/tag/2.0.2 - -This release body mentions the word unreleased and {#unreleased} anchor to test leaks. -""" - changelog_path = self.tmpdir / "CHANGELOG.md" - changelog_path.write_text(changelog) - - news_dir = self.tmpdir / "news" - news_dir.mkdir() - (news_dir / "1.fixed.md").write_text("Some fix") - - # Act - changelog_news.update_changelog( - "3.0.0", - "2026-06-16", - changelog_path=changelog_path, - news_dir=news_dir, - ) - - # Assert - new_content = changelog_path.read_text() - - # The 2.0.2 body should NOT be modified - self.assertIn( - "This release body mentions the word unreleased and {#unreleased} anchor to test leaks.", - new_content, - ) - - def test_update_changelog_empty_news(self): - # Arrange - changelog = """# Changelog - -{#unreleased} -## Unreleased - -[unreleased]: https://github.com/bazel-contrib/rules_python/releases/tag/unreleased - -Unreleased changes are tracked as individual files in the [news/](./news) -directory, or view the [latest generated -changelog](https://rules-python.readthedocs.io/en/latest/changelog.html). - -{#v2-0-2} -## [2.0.2] - 2026-05-14 - -[2.0.2]: https://github.com/bazel-contrib/rules_python/releases/tag/2.0.2 - -{#v2-0-2-added} -### Added -* (toolchains) Some older change. -""" - changelog_path = self.tmpdir / "CHANGELOG.md" - changelog_path.write_text(changelog) - - news_dir = self.tmpdir / "news" - news_dir.mkdir() - - # Act - changelog_news.update_changelog( - "3.0.0", - "2026-06-16", - changelog_path=changelog_path, - news_dir=news_dir, - ) - - # Assert - new_content = changelog_path.read_text() - - # The new release section should be present and contain "No notable changes." - self.assertIn("{#v3-0-0}", new_content) - self.assertIn("## [3.0.0] - 2026-06-16", new_content) - self.assertIn( - "[3.0.0]: https://github.com/bazel-contrib/rules_python/releases/tag/3.0.0", - new_content, - ) - self.assertIn("No notable changes.", new_content) - - # Verify that we didn't accidentally create any categories - self.assertNotIn("{#v3-0-0-fixed}", new_content) - self.assertNotIn("{#v3-0-0-added}", new_content) - - def test_replace_version_next(self): - # Arrange - mock_file_content = """ -:::{versionadded} VERSION_NEXT_FEATURE -blabla -::: - -:::{versionchanged} VERSION_NEXT_PATCH -blabla -::: -""" - (self.tmpdir / "mock_file.bzl").write_text(mock_file_content) - - utils.replace_version_next("0.28.0") - - new_content = (self.tmpdir / "mock_file.bzl").read_text() - - self.assertIn(":::{versionadded} 0.28.0", new_content) - self.assertIn(":::{versionadded} 0.28.0", new_content) - self.assertNotIn("VERSION_NEXT_FEATURE", new_content) - self.assertNotIn("VERSION_NEXT_PATCH", new_content) - - def test_replace_version_next_excludes_bazel_dirs(self): - # Arrange - mock_file_content = """ -:::{versionadded} VERSION_NEXT_FEATURE -blabla -::: -""" - bazel_dir = self.tmpdir / "bazel-rules_python" - bazel_dir.mkdir() - (bazel_dir / "mock_file.bzl").write_text(mock_file_content) - - tools_dir = self.tmpdir / "tools" / "private" / "release" - tools_dir.mkdir(parents=True) - (tools_dir / "mock_file.bzl").write_text(mock_file_content) - - tests_dir = self.tmpdir / "tests" / "tools" / "private" / "release" - tests_dir.mkdir(parents=True) - (tests_dir / "mock_file.bzl").write_text(mock_file_content) - - version = "0.28.0" - - # Act - utils.replace_version_next(version) - - # Assert - new_content = (bazel_dir / "mock_file.bzl").read_text() - self.assertIn("VERSION_NEXT_FEATURE", new_content) - - new_content = (tools_dir / "mock_file.bzl").read_text() - self.assertIn("VERSION_NEXT_FEATURE", new_content) - - new_content = (tests_dir / "mock_file.bzl").read_text() - self.assertIn("VERSION_NEXT_FEATURE", new_content) - +class ReleaseCLITest(unittest.TestCase): def test_valid_version(self): # These should not raise an exception releaser.create_parser().parse_args(["prepare", "0.28.0"]) @@ -528,1199 +21,5 @@ def test_invalid_version(self): releaser.create_parser().parse_args(["prepare", "a.b.c"]) -class GetLatestVersionTest(unittest.TestCase): - @patch("tools.private.release.git.Git.get_tags") - def test_get_latest_version_success(self, mock_get_tags): - mock_get_tags.return_value = ["0.1.0", "1.0.0", "0.2.0"] - self.assertEqual(utils.get_latest_version(), "1.0.0") - - @patch("tools.private.release.git.Git.get_tags") - def test_get_latest_version_rc_is_latest(self, mock_get_tags): - mock_get_tags.return_value = ["0.1.0", "1.0.0", "1.1.0rc0"] - with self.assertRaisesRegex( - ValueError, "The latest version is a pre-release version: 1.1.0rc0" - ): - utils.get_latest_version() - - @patch("tools.private.release.git.Git.get_tags") - def test_get_latest_version_no_tags(self, mock_get_tags): - mock_get_tags.return_value = [] - with self.assertRaisesRegex( - RuntimeError, "No git tags found matching X.Y.Z or X.Y.ZrcN format." - ): - utils.get_latest_version() - - @patch("tools.private.release.git.Git.get_tags") - def test_get_latest_version_no_matching_tags(self, mock_get_tags): - mock_get_tags.return_value = ["v1.0", "latest"] - with self.assertRaisesRegex( - RuntimeError, "No git tags found matching X.Y.Z or X.Y.ZrcN format." - ): - utils.get_latest_version() - - @patch("tools.private.release.git.Git.get_tags") - def test_get_latest_version_only_rc_tags(self, mock_get_tags): - mock_get_tags.return_value = ["1.0.0rc0", "1.1.0rc0"] - with self.assertRaisesRegex( - ValueError, "The latest version is a pre-release version: 1.1.0rc0" - ): - utils.get_latest_version() - - -class GetLatestRcTagTest(unittest.TestCase): - @patch("tools.private.release.git.Git.get_tags") - def test_get_latest_rc_tag_no_tags(self, mock_get_tags): - mock_get_tags.return_value = [] - self.assertIsNone(utils.get_latest_rc_tag("2.0.0")) - - @patch("tools.private.release.git.Git.get_tags") - def test_get_latest_rc_tag_no_matching_tags(self, mock_get_tags): - mock_get_tags.return_value = ["1.0.0", "2.0.0", "v2.0.0-rc0", "2.1.0-rc0"] - self.assertIsNone(utils.get_latest_rc_tag("2.0.0")) - - @patch("tools.private.release.git.Git.get_tags") - def test_get_latest_rc_tag_success(self, mock_get_tags): - mock_get_tags.return_value = [ - "2.0.0-rc0", - "2.0.0-rc2", - "2.0.0-rc1", - "2.1.0-rc0", - ] - self.assertEqual(utils.get_latest_rc_tag("2.0.0"), "2.0.0-rc2") - - @patch("tools.private.release.git.Git.get_tags") - def test_get_latest_rc_tag_ignores_v_prefix(self, mock_get_tags): - mock_get_tags.return_value = ["v2.0.0-rc0", "2.0.0-rc1"] - self.assertEqual(utils.get_latest_rc_tag("2.0.0"), "2.0.0-rc1") - - @patch("tools.private.release.git.Git.get_remote_tags") - def test_get_latest_rc_tag_remote_success(self, mock_get_remote_tags): - mock_get_remote_tags.return_value = [ - "2.0.0-rc0", - "2.0.0-rc2", - "2.0.0-rc1", - "2.1.0-rc0", - ] - self.assertEqual(utils.get_latest_rc_tag("2.0.0", remote="origin"), "2.0.0-rc2") - mock_get_remote_tags.assert_called_once_with("origin") - - -class DetermineNextVersionTest(TempDirTestCase): - def setUp(self): - super().setUp() - self.mock_get_latest_version = patch( - "tools.private.release.utils.get_latest_version" - ).start() - self.mock_get_current_branch = patch( - "tools.private.release.git.Git.get_current_branch" - ).start() - self.mock_get_current_branch.return_value = "main" - self.addCleanup(patch.stopall) - - def test_no_markers(self): - (self.tmpdir / "mock_file.bzl").write_text("no markers here") - self.mock_get_latest_version.return_value = "1.2.3" - - next_version = utils.determine_next_version() - - self.assertEqual(next_version, "1.2.4") - - def test_only_patch(self): - (self.tmpdir / "mock_file.bzl").write_text( - ":::{versionchanged} VERSION_NEXT_PATCH" - ) - self.mock_get_latest_version.return_value = "1.2.3" - - next_version = utils.determine_next_version() - - self.assertEqual(next_version, "1.2.4") - - def test_only_feature(self): - (self.tmpdir / "mock_file.bzl").write_text( - ":::{versionadded} VERSION_NEXT_FEATURE" - ) - self.mock_get_latest_version.return_value = "1.2.3" - - next_version = utils.determine_next_version() - - self.assertEqual(next_version, "1.3.0") - - def test_both_markers(self): - (self.tmpdir / "mock_file_patch.bzl").write_text( - ":::{versionchanged} VERSION_NEXT_PATCH" - ) - (self.tmpdir / "mock_file_feature.bzl").write_text( - ":::{versionadded} VERSION_NEXT_FEATURE" - ) - self.mock_get_latest_version.return_value = "1.2.3" - - next_version = utils.determine_next_version() - - self.assertEqual(next_version, "1.3.0") - - @patch("tools.private.release.git.Git.get_current_branch") - @patch("tools.private.release.git.Git.get_tags") - def test_determine_next_version_on_release_branch_with_existing_tags( - self, mock_get_tags, mock_get_branch - ): - mock_get_branch.return_value = "release/0.37" - mock_get_tags.return_value = ["0.37.0", "0.37.1", "0.36.0"] - - next_version = utils.determine_next_version() - - self.assertEqual(next_version, "0.37.2") - - @patch("tools.private.release.git.Git.get_current_branch") - @patch("tools.private.release.git.Git.get_tags") - def test_determine_next_version_on_release_branch_no_tags( - self, mock_get_tags, mock_get_branch - ): - mock_get_branch.return_value = "release/0.38" - mock_get_tags.return_value = ["0.37.0"] # No 0.38.x tags - - next_version = utils.determine_next_version() - - self.assertEqual(next_version, "0.38.0") - - @patch("tools.private.release.git.Git.get_current_branch") - @patch("tools.private.release.git.Git.get_tags") - def test_determine_next_version_on_release_branch_with_active_rc( - self, mock_get_tags, mock_get_branch - ): - mock_get_branch.return_value = "release/0.37" - # 0.37.0-rc0 and rc1 exist, but no stable 0.37.0 yet - mock_get_tags.return_value = ["0.37.0-rc0", "0.37.0-rc1", "0.36.0"] - - next_version = utils.determine_next_version() - - # Should target 0.37.0, not 0.37.1 - self.assertEqual(next_version, "0.37.0") - - @patch("tools.private.release.git.Git.get_current_branch") - @patch("tools.private.release.git.Git.get_tags") - def test_determine_next_version_on_release_branch_with_stable_and_active_patch_rc( - self, mock_get_tags, mock_get_branch - ): - mock_get_branch.return_value = "release/0.37" - # 0.37.0 stable exists, and 0.37.1-rc0 exists (but no stable 0.37.1 yet) - mock_get_tags.return_value = ["0.37.0", "0.37.1-rc0", "0.36.0"] - - next_version = utils.determine_next_version() - - # Should target 0.37.1, not 0.37.2 - self.assertEqual(next_version, "0.37.1") - - @patch("tools.private.release.git.Git.get_current_branch") - def test_determine_next_version_on_main_branch_fallback(self, mock_get_branch): - mock_get_branch.return_value = "main" - # Should fallback to default behavior (which uses mock_get_latest_version from setUp) - self.mock_get_latest_version.return_value = "1.2.3" - (self.tmpdir / "mock_file.bzl").write_text("no markers here") - - next_version = utils.determine_next_version() - - self.assertEqual(next_version, "1.2.4") - - -class CmdPrepareTest(TempDirTestCase): - def setUp(self): - super().setUp() - _mock_git_and_gh(self) - - @patch("tools.private.release.prepare.changelog_news") - @patch("tools.private.release.prepare.replace_version_next") - def test_prepare_success_existing_issue(self, mock_replace, mock_changelog): - # Arrange - args = MagicMock(version="2.0.0", issue=None, dry_run=False) - self.mock_git.status.side_effect = ["", "M foo"] - self.mock_git.branch_exists.return_value = False - self.mock_gh.get_release_tracking_issue.side_effect = None - self.mock_gh.get_release_tracking_issue.return_value = 123 - self.mock_gh.create_pr.return_value = "https://github.com/foo/bar/pull/456" - self.mock_gh.get_issue_body.return_value = "- [ ] Prepare Release" - - # Act - result = Prepare(args, self.mock_git, self.mock_gh).run() - - # Assert - self.assertEqual(result, 0) - self.mock_gh.get_release_tracking_issue.assert_called_once_with("2.0.0") - self.mock_gh.create_tracking_issue.assert_not_called() - self.mock_gh.create_pr.assert_called_once_with("2.0.0", 123) - self.mock_git.add_modified_and_deleted.assert_called_once() - - @patch("tools.private.release.prepare.changelog_news") - @patch("tools.private.release.prepare.replace_version_next") - def test_prepare_success_create_issue(self, mock_replace, mock_changelog): - # Arrange - template_dir = self.tmpdir / ".github" / "ISSUE_TEMPLATE" - template_dir.mkdir(parents=True, exist_ok=True) - template_file = template_dir / "release_tracking_template.md" - template_file.write_text("dummy template content") - - args = MagicMock(version="2.0.0", issue=None, dry_run=False) - self.mock_git.status.side_effect = ["", "M foo"] - self.mock_git.branch_exists.return_value = False - self.mock_gh.get_release_tracking_issue.side_effect = NoTrackingIssueError( - "Not found" - ) - self.mock_gh.create_tracking_issue.return_value = 123 - self.mock_gh.create_pr.return_value = "https://github.com/foo/bar/pull/456" - self.mock_gh.get_issue_body.return_value = "- [ ] Prepare Release" - - # Act - result = Prepare(args, self.mock_git, self.mock_gh).run() - - # Assert - self.assertEqual(result, 0) - self.mock_gh.get_release_tracking_issue.assert_called_once_with("2.0.0") - self.mock_gh.create_tracking_issue.assert_called_once_with( - "2.0.0", "dummy template content" - ) - self.mock_gh.create_pr.assert_called_once_with("2.0.0", 123) - self.mock_git.add_modified_and_deleted.assert_called_once() - - @patch("tools.private.release.prepare.changelog_news") - @patch("tools.private.release.prepare.replace_version_next") - def test_prepare_ambiguous_issue(self, mock_replace, mock_changelog): - # Arrange - args = MagicMock(version="2.0.0", issue=None, dry_run=False) - self.mock_git.status.side_effect = ["", "M foo"] - self.mock_git.branch_exists.return_value = False - self.mock_gh.get_release_tracking_issue.side_effect = ( - MultipleTrackingIssuesError("Multiple open tracking issues") - ) - - # Act - result = Prepare(args, self.mock_git, self.mock_gh).run() - - # Assert - self.assertEqual(result, 1) - self.mock_gh.get_release_tracking_issue.assert_called_once_with("2.0.0") - self.mock_gh.create_tracking_issue.assert_not_called() - self.mock_gh.create_pr.assert_not_called() - self.mock_git.add_modified_and_deleted.assert_not_called() - - @patch("tools.private.release.prepare.changelog_news") - @patch("tools.private.release.prepare.replace_version_next") - def test_prepare_dry_run(self, mock_replace, mock_changelog): - # Arrange - args = MagicMock(version="2.0.0", issue=None, dry_run=True) - self.mock_git.status.side_effect = [""] - self.mock_gh.get_release_tracking_issue.side_effect = None - self.mock_gh.get_release_tracking_issue.return_value = 123 - - # Act - result = Prepare(args, self.mock_git, self.mock_gh).run() - - # Assert - self.assertEqual(result, 0) - self.mock_git.checkout.assert_not_called() - self.mock_git.commit.assert_not_called() - self.mock_git.push.assert_not_called() - self.mock_gh.create_pr.assert_not_called() - self.mock_gh.update_issue_body.assert_not_called() - self.mock_git.fetch.assert_called_once() - self.mock_gh.get_release_tracking_issue.assert_called_once_with("2.0.0") - self.mock_git.add_modified_and_deleted.assert_not_called() - - @patch("tools.private.release.prepare.changelog_news") - @patch("tools.private.release.prepare.replace_version_next") - def test_prepare_use_associated_pr_from_tracking_issue( - self, mock_replace, mock_changelog - ): - # Arrange - args = MagicMock(version="2.0.0", issue=None, dry_run=False) - self.mock_git.status.side_effect = ["", ""] - self.mock_git.branch_exists.return_value = True - self.mock_gh.get_release_tracking_issue.side_effect = None - self.mock_gh.get_release_tracking_issue.return_value = 123 - self.mock_gh.get_open_pr.return_value = None - # PR #456 is already associated in the tracking issue - self.mock_gh.get_issue_body.return_value = ( - "- [ ] Prepare Release | status=pending pr=#456" - ) - - # Act - result = Prepare(args, self.mock_git, self.mock_gh).run() - - # Assert - self.assertEqual(result, 0) - self.mock_git.checkout.assert_called_once_with("prepare-2.0.0") - self.mock_git.commit.assert_not_called() - self.mock_git.push.assert_called_once_with( - "origin", "prepare-2.0.0", set_upstream=True, force=True - ) - self.mock_gh.get_open_pr.assert_called_once_with("prepare-2.0.0") - self.mock_gh.create_pr.assert_not_called() # Should NOT create a new PR - self.mock_gh.update_issue_body.assert_called_once() - call_args = self.mock_gh.update_issue_body.call_args[0] - self.assertIn("pr=#456", call_args[1]) - - @patch("tools.private.release.prepare.changelog_news") - @patch("tools.private.release.prepare.replace_version_next") - def test_prepare_create_pr_when_none_associated(self, mock_replace, mock_changelog): - # Arrange - args = MagicMock(version="2.0.0", issue=None, dry_run=False) - self.mock_git.status.side_effect = ["", ""] - self.mock_git.branch_exists.return_value = True - self.mock_gh.get_release_tracking_issue.side_effect = None - self.mock_gh.get_release_tracking_issue.return_value = 123 - self.mock_gh.get_open_pr.return_value = None - # No PR associated in the tracking issue - self.mock_gh.get_issue_body.return_value = "- [ ] Prepare Release" - self.mock_gh.create_pr.return_value = "https://github.com/foo/bar/pull/789" - - # Act - result = Prepare(args, self.mock_git, self.mock_gh).run() - - # Assert - self.assertEqual(result, 0) - self.mock_git.checkout.assert_called_once_with("prepare-2.0.0") - self.mock_git.commit.assert_not_called() - self.mock_git.push.assert_called_once_with( - "origin", "prepare-2.0.0", set_upstream=True, force=True - ) - self.mock_gh.get_open_pr.assert_called_once_with("prepare-2.0.0") - self.mock_gh.create_pr.assert_called_once_with("2.0.0", 123) - self.mock_gh.update_issue_body.assert_called_once() - call_args = self.mock_gh.update_issue_body.call_args[0] - self.assertIn("pr=#789", call_args[1]) - - @patch("tools.private.release.prepare.changelog_news") - @patch("tools.private.release.prepare.replace_version_next") - def test_prepare_reuse_existing_pr(self, mock_replace, mock_changelog): - # Arrange - args = MagicMock(version="2.0.0", issue=None, dry_run=False) - self.mock_git.status.side_effect = ["", ""] - self.mock_git.branch_exists.return_value = True - self.mock_gh.get_release_tracking_issue.side_effect = None - self.mock_gh.get_release_tracking_issue.return_value = 123 - self.mock_gh.get_open_pr.return_value = { - "number": 456, - "url": "https://github.com/foo/bar/pull/456", - } - self.mock_gh.get_issue_body.return_value = "- [ ] Prepare Release" - - # Act - result = Prepare(args, self.mock_git, self.mock_gh).run() - - # Assert - self.assertEqual(result, 0) - self.mock_git.checkout.assert_called_once_with("prepare-2.0.0") - self.mock_git.commit.assert_not_called() - self.mock_git.push.assert_called_once_with( - "origin", "prepare-2.0.0", set_upstream=True, force=True - ) - self.mock_gh.get_open_pr.assert_called_once_with("prepare-2.0.0") - self.mock_gh.create_pr.assert_not_called() - self.mock_gh.update_issue_body.assert_called_once() - call_args = self.mock_gh.update_issue_body.call_args[0] - self.assertIn("pr=#456", call_args[1]) - - @patch("tools.private.release.prepare.changelog_news") - @patch("tools.private.release.prepare.replace_version_next") - def test_prepare_dry_run_no_issue(self, mock_replace, mock_changelog): - # Arrange - template_dir = self.tmpdir / ".github" / "ISSUE_TEMPLATE" - template_dir.mkdir(parents=True, exist_ok=True) - template_file = template_dir / "release_tracking_template.md" - template_file.write_text("dummy template content") - - args = MagicMock(version="2.0.0", issue=None, dry_run=True) - self.mock_git.status.side_effect = [""] - self.mock_gh.get_release_tracking_issue.side_effect = NoTrackingIssueError( - "Not found" - ) - - # Act - result = Prepare(args, self.mock_git, self.mock_gh).run() - - # Assert - self.assertEqual(result, 0) - self.mock_git.checkout.assert_not_called() - self.mock_gh.create_tracking_issue.assert_not_called() - self.mock_gh.create_pr.assert_not_called() - self.mock_git.add_modified_and_deleted.assert_not_called() - - -class CmdCreateRcTest(unittest.TestCase): - def setUp(self): - _mock_git_and_gh(self) - - def test_create_rc_success_first_rc(self): - # Arrange - args = MagicMock(issue=123, remote="my-remote") - self.mock_gh.get_issue_title.return_value = "Release 2.0.0" - self.mock_gh.get_issue_body.return_value = """ -## Checklist -- [x] Prepare Release | status=done pr=#122 commit=abcdef12 -- [x] Create Release branch | status=done branch=release/2.0 commit=abcdef12 -- [ ] Tag RC0 | status=pending -""" - self.mock_git.get_remote_tags.return_value = [] - self.mock_git.get_commit_sha.return_value = "1234567890" - - # Act - result = CreateRc(args, self.mock_git, self.mock_gh).run() - - # Assert - self.assertEqual(result, 0) - self.mock_git.fetch.assert_has_calls( - [call("my-remote"), call("my-remote", tags=True, force=True)] - ) - self.mock_git.checkout.assert_not_called() - self.mock_git.tag.assert_called_once_with("2.0.0-rc0", "my-remote/release/2.0") - self.mock_git.push.assert_called_once_with("my-remote", "2.0.0-rc0") - self.mock_git.get_commit_sha.assert_called_once_with("my-remote/release/2.0") - - self.mock_gh.update_issue_body.assert_called_once() - call_args = self.mock_gh.update_issue_body.call_args[0] - self.assertEqual(call_args[0], 123) - self.assertIn("tag=2.0.0-rc0", call_args[1]) - self.assertIn("commit=12345678", call_args[1]) - - self.mock_gh.post_issue_comment.assert_called_once() - comment_call_args = self.mock_gh.post_issue_comment.call_args[0] - self.assertEqual(comment_call_args[0], 123) - self.assertIn( - "**New Release Candidate Tagged!** 🐍🌿", - comment_call_args[1], - ) - self.assertIn( - "- [Github Release 2.0.0-rc0](https://github.com/bazel-contrib/rules_python/releases/tag/2.0.0-rc0)", - comment_call_args[1], - ) - self.assertIn( - "- BCR Entry: [rules_python@2.0.0](https://registry.bazel.build/modules/rules_python/2.0.0)", - comment_call_args[1], - ) - self.assertIn( - "- [BCR PRs](https://github.com/bazelbuild/bazel-central-registry/pulls?q=is%3Apr+rules_python+2.0.0)", - comment_call_args[1], - ) - self.assertIn( - "- [Release workflow status](https://github.com/bazel-contrib/rules_python/actions/workflows/release.yml)", - comment_call_args[1], - ) - self.assertNotIn("🚀", comment_call_args[1]) - - def test_create_rc_success_next_rc(self): - # Arrange - args = MagicMock(issue=123, remote="my-remote") - self.mock_gh.get_issue_title.return_value = "Release 2.0.0" - self.mock_gh.get_issue_body.return_value = """ -## Checklist -- [x] Prepare Release | status=done pr=#122 commit=abcdef12 -- [x] Create Release branch | status=done branch=release/2.0 commit=abcdef12 -- [x] Tag RC0 | status=done tag=2.0.0-rc0 commit=abcdef12 -- [ ] Tag RC1 | status=pending -""" - self.mock_git.get_remote_tags.return_value = ["2.0.0-rc0"] - self.mock_git.get_commit_sha.return_value = "1234567890" - - # Act - result = CreateRc(args, self.mock_git, self.mock_gh).run() - - # Assert - self.assertEqual(result, 0) - self.mock_git.fetch.assert_has_calls( - [call("my-remote"), call("my-remote", tags=True, force=True)] - ) - self.mock_git.checkout.assert_not_called() - self.mock_git.tag.assert_called_once_with("2.0.0-rc1", "my-remote/release/2.0") - self.mock_git.push.assert_called_once_with("my-remote", "2.0.0-rc1") - self.mock_git.get_commit_sha.assert_called_once_with("my-remote/release/2.0") - - self.mock_gh.update_issue_body.assert_called_once() - call_args = self.mock_gh.update_issue_body.call_args[0] - self.assertEqual(call_args[0], 123) - self.assertIn("tag=2.0.0-rc1", call_args[1]) - - self.mock_gh.post_issue_comment.assert_called_once() - comment_call_args = self.mock_gh.post_issue_comment.call_args[0] - self.assertEqual(comment_call_args[0], 123) - self.assertIn( - "**New Release Candidate Tagged!** 🐍🌿", - comment_call_args[1], - ) - self.assertIn( - "- [Github Release 2.0.0-rc1](https://github.com/bazel-contrib/rules_python/releases/tag/2.0.0-rc1)", - comment_call_args[1], - ) - self.assertIn( - "- BCR Entry: [rules_python@2.0.0](https://registry.bazel.build/modules/rules_python/2.0.0)", - comment_call_args[1], - ) - self.assertIn( - "- [BCR PRs](https://github.com/bazelbuild/bazel-central-registry/pulls?q=is%3Apr+rules_python+2.0.0)", - comment_call_args[1], - ) - self.assertIn( - "- [Release workflow status](https://github.com/bazel-contrib/rules_python/actions/workflows/release.yml)", - comment_call_args[1], - ) - self.assertNotIn("🚀", comment_call_args[1]) - - def test_create_rc_gating_on_backports(self): - # Arrange - args = MagicMock(issue=123, remote="my-remote") - self.mock_gh.get_issue_title.return_value = "Release 2.0.0" - self.mock_gh.get_issue_body.return_value = """ -## Checklist -- [x] Prepare Release | status=done pr=#122 commit=abcdef12 -- [x] Create Release branch | status=done branch=release/2.0 commit=abcdef12 -- [ ] Tag RC0 | status=pending - -## Backports -- [ ] #124 | status=pending -""" - # Act - result = CreateRc(args, self.mock_git, self.mock_gh).run() - - # Assert - self.assertEqual(result, 1) - self.mock_git.tag.assert_not_called() - self.mock_git.push.assert_not_called() - - def test_create_rc_with_finished_backports(self): - # Arrange - args = MagicMock(issue=123, remote="my-remote") - self.mock_gh.get_issue_title.return_value = "Release 2.0.0" - self.mock_gh.get_issue_body.return_value = """ -## Checklist -- [x] Prepare Release | status=done pr=#122 commit=abcdef12 -- [x] Create Release branch | status=done branch=release/2.0 commit=abcdef12 -- [ ] Tag RC0 | status=pending - -## Backports -- [x] #124 | status=done rc=rc0 commit=abcdef12 -""" - self.mock_git.get_remote_tags.return_value = [] - self.mock_git.get_commit_sha.return_value = "1234567890" - - # Act - result = CreateRc(args, self.mock_git, self.mock_gh).run() - - # Assert - self.assertEqual(result, 0) - self.mock_git.tag.assert_called_once_with("2.0.0-rc0", "my-remote/release/2.0") - self.mock_git.push.assert_called_once_with("my-remote", "2.0.0-rc0") - - -class CmdPromoteRcTest(unittest.TestCase): - def setUp(self): - _mock_git_and_gh(self) - - def test_promote_rc_success(self): - # Arrange - args = MagicMock(version="2.0.0", issue=123, dry_run=False, remote="my-remote") - self.mock_git.get_remote_tags.return_value = ["2.0.0-rc0", "2.0.0-rc1"] - self.mock_git.get_commit_sha.return_value = "abcdef123456" - self.mock_git.tag_exists.return_value = False - initial_body = "- [ ] Tag Final" - self.mock_gh.get_issue_body.return_value = initial_body - - # Act - result = PromoteRc(args, self.mock_git, self.mock_gh).run() - - # Assert - self.assertEqual(result, 0) - self.mock_git.fetch.assert_called_once_with("my-remote", tags=True, force=True) - self.mock_git.get_commit_sha.assert_called_once_with("2.0.0-rc1") - self.mock_git.checkout.assert_not_called() - self.mock_git.tag_exists.assert_called_once_with("2.0.0") - self.mock_git.tag.assert_called_once_with("2.0.0", "abcdef123456") - self.mock_git.push.assert_called_once_with("my-remote", "2.0.0") - - # Verify issue update - self.mock_gh.get_issue_body.assert_called_once_with(123) - expected_updated_body = ( - "- [x] Tag Final | status=done tag=2.0.0 commit=abcdef12" - ) - self.mock_gh.update_issue_body.assert_called_once_with( - 123, expected_updated_body - ) - expected_comment = ( - "Version 2.0.0 has been tagged.\n\n" - "- **Release Page**: https://github.com/bazel-contrib/rules_python/releases/tag/2.0.0\n" - '- **BCR PR Search**: [is:pr ("bazel-contrib/rules_python" in:title) ("@2.0.0" in:title)](https://github.com/bazelbuild/bazel-central-registry/pulls?q=is%3Apr%20%28%22bazel-contrib/rules_python%22%20in%3Atitle%29%20%28%22%402.0.0%22%20in%3Atitle%29)' - ) - self.mock_gh.post_issue_comment.assert_called_once_with(123, expected_comment) - - def test_promote_rc_resolve_issue_success(self): - # Arrange - args = MagicMock(version="2.0.0", issue=None, dry_run=False, remote="my-remote") - self.mock_git.get_remote_tags.return_value = ["2.0.0-rc1"] - self.mock_git.tag_exists.return_value = False - self.mock_gh.get_release_tracking_issue.side_effect = None - self.mock_gh.get_release_tracking_issue.return_value = 123 - self.mock_git.get_commit_sha.return_value = "abcdef123456" - initial_body = "- [ ] Tag Final" - self.mock_gh.get_issue_body.return_value = initial_body - - # Act - result = PromoteRc(args, self.mock_git, self.mock_gh).run() - - # Assert - self.assertEqual(result, 0) - self.mock_git.fetch.assert_called_once_with("my-remote", tags=True, force=True) - self.mock_gh.get_release_tracking_issue.assert_called_once_with("2.0.0") - self.mock_git.get_commit_sha.assert_called_once_with("2.0.0-rc1") - self.mock_git.checkout.assert_not_called() - self.mock_git.tag.assert_called_once_with("2.0.0", "abcdef123456") - self.mock_git.push.assert_called_once_with("my-remote", "2.0.0") - self.mock_gh.get_issue_body.assert_called_once_with(123) - expected_updated_body = ( - "- [x] Tag Final | status=done tag=2.0.0 commit=abcdef12" - ) - self.mock_gh.update_issue_body.assert_called_once_with( - 123, expected_updated_body - ) - expected_comment = ( - "Version 2.0.0 has been tagged.\n\n" - "- **Release Page**: https://github.com/bazel-contrib/rules_python/releases/tag/2.0.0\n" - '- **BCR PR Search**: [is:pr ("bazel-contrib/rules_python" in:title) ("@2.0.0" in:title)](https://github.com/bazelbuild/bazel-central-registry/pulls?q=is%3Apr%20%28%22bazel-contrib/rules_python%22%20in%3Atitle%29%20%28%22%402.0.0%22%20in%3Atitle%29)' - ) - self.mock_gh.post_issue_comment.assert_called_once_with(123, expected_comment) - - def test_promote_rc_defaults_to_determine_next_version(self): - # Arrange - args = MagicMock(version=None, issue=123, dry_run=False, remote="my-remote") - self.mock_git.get_current_branch.return_value = "release/2.0" - self.mock_git.get_tags.return_value = ["2.0.0"] - self.mock_git.get_remote_tags.return_value = ["2.0.1-rc0"] - self.mock_git.get_commit_sha.return_value = "12345678" - self.mock_git.tag_exists.return_value = False - initial_body = "- [ ] Tag Final" - self.mock_gh.get_issue_body.return_value = initial_body - - # Act - result = PromoteRc(args, self.mock_git, self.mock_gh).run() - - # Assert - self.assertEqual(result, 0) - self.mock_git.fetch.assert_called_once_with("my-remote", tags=True, force=True) - self.mock_git.get_current_branch.assert_called_once() - self.mock_git.get_tags.assert_called_once() - self.mock_git.get_remote_tags.assert_called_once_with("my-remote") - - self.mock_git.checkout.assert_not_called() - self.mock_git.get_commit_sha.assert_called_once_with("2.0.1-rc0") - self.mock_git.tag.assert_called_once_with("2.0.1", "12345678") - self.mock_git.push.assert_called_once_with("my-remote", "2.0.1") - - expected_updated_body = ( - "- [x] Tag Final | status=done tag=2.0.1 commit=12345678" - ) - self.mock_gh.update_issue_body.assert_called_once_with( - 123, expected_updated_body - ) - expected_comment = ( - "Version 2.0.1 has been tagged.\n\n" - "- **Release Page**: https://github.com/bazel-contrib/rules_python/releases/tag/2.0.1\n" - '- **BCR PR Search**: [is:pr ("bazel-contrib/rules_python" in:title) ("@2.0.1" in:title)](https://github.com/bazelbuild/bazel-central-registry/pulls?q=is%3Apr%20%28%22bazel-contrib/rules_python%22%20in%3Atitle%29%20%28%22%402.0.1%22%20in%3Atitle%29)' - ) - self.mock_gh.post_issue_comment.assert_called_once_with(123, expected_comment) - - def test_promote_rc_dry_run_success(self): - # Arrange - args = MagicMock(version="2.0.0", issue=123, dry_run=True, remote="my-remote") - self.mock_git.get_remote_tags.return_value = ["2.0.0-rc0", "2.0.0-rc1"] - self.mock_git.get_commit_sha.return_value = "abcdef123456" - self.mock_git.tag_exists.return_value = False - initial_body = "- [ ] Tag Final" - self.mock_gh.get_issue_body.return_value = initial_body - - # Act - result = PromoteRc(args, self.mock_git, self.mock_gh).run() - - # Assert - self.assertEqual(result, 0) - self.mock_git.fetch.assert_called_once_with("my-remote", tags=True, force=True) - self.mock_git.get_commit_sha.assert_called_once_with("2.0.0-rc1") - self.mock_git.tag_exists.assert_called_once_with("2.0.0") - - # Core dry-run assertions: NO modifications - self.mock_git.tag.assert_not_called() - self.mock_git.push.assert_not_called() - self.mock_gh.update_issue_body.assert_not_called() - self.mock_gh.post_issue_comment.assert_not_called() - - def test_promote_rc_tag_already_exists(self): - # Arrange - args = MagicMock(version="2.0.0", issue=123, remote="my-remote") - self.mock_git.get_remote_tags.return_value = ["2.0.0-rc1"] - self.mock_git.tag_exists.return_value = True - - # Act - result = PromoteRc(args, self.mock_git, self.mock_gh).run() - - # Assert - self.assertEqual(result, 1) - self.mock_git.checkout.assert_not_called() - self.mock_git.tag.assert_not_called() - self.mock_git.push.assert_not_called() - self.mock_gh.get_issue_body.assert_not_called() - self.mock_gh.update_issue_body.assert_not_called() - - def test_promote_rc_issue_not_found(self): - # Arrange - args = MagicMock(version="2.0.0", issue=None, remote="my-remote") - self.mock_git.get_remote_tags.return_value = ["2.0.0-rc1"] - self.mock_git.tag_exists.return_value = False - self.mock_gh.get_release_tracking_issue.side_effect = NoTrackingIssueError( - "Not found" - ) - - # Act - result = PromoteRc(args, self.mock_git, self.mock_gh).run() - - # Assert - self.assertEqual(result, 1) - self.mock_gh.get_release_tracking_issue.assert_called_once_with("2.0.0") - self.mock_git.checkout.assert_not_called() - self.mock_git.tag.assert_not_called() - self.mock_git.push.assert_not_called() - self.mock_gh.get_issue_body.assert_not_called() - - def test_promote_rc_issue_malformed(self): - # Arrange - args = MagicMock(version="2.0.0", issue=123, remote="my-remote") - self.mock_git.get_remote_tags.return_value = ["2.0.0-rc1"] - self.mock_git.tag_exists.return_value = False - self.mock_git.get_commit_sha.return_value = "abcdef123456" - initial_body = "malformed body" - self.mock_gh.get_issue_body.return_value = initial_body - - # Act - result = PromoteRc(args, self.mock_git, self.mock_gh).run() - - # Assert - self.assertEqual(result, 1) - self.mock_gh.get_issue_body.assert_called_once_with(123) - self.mock_git.checkout.assert_not_called() - self.mock_git.tag.assert_not_called() - self.mock_git.push.assert_not_called() - self.mock_gh.update_issue_body.assert_not_called() - - def test_promote_rc_no_rc_found(self): - # Arrange - args = MagicMock(version="2.0.0", issue=123, remote="my-remote") - self.mock_git.get_remote_tags.return_value = [] - - # Act - result = PromoteRc(args, self.mock_git, self.mock_gh).run() - - # Assert - self.assertEqual(result, 1) - self.mock_git.checkout.assert_not_called() - self.mock_git.tag.assert_not_called() - self.mock_gh.get_issue_body.assert_not_called() - - -class CmdCreateReleaseBranchTest(unittest.TestCase): - def setUp(self): - _mock_git_and_gh(self) - - def test_create_release_branch_success(self): - # Arrange - args = MagicMock(issue=123, remote="my-remote") - self.mock_gh.get_issue_title.return_value = "Release 2.0.0" - self.mock_gh.get_issue_body.return_value = """ -## Checklist -- [x] Prepare Release | status=done pr=#122 commit=abcdef12 -- [ ] Create Release branch | status=pending -""" - self.mock_git.branch_exists.return_value = False - self.mock_git.remote_branch_exists.return_value = False - - # Act - result = CreateReleaseBranch(args, self.mock_git, self.mock_gh).run() - - # Assert - self.assertEqual(result, 0) - self.mock_git.fetch.assert_called_once_with("my-remote") - self.mock_git.checkout.assert_not_called() - self.mock_git.push.assert_called_once_with( - "my-remote", "abcdef12:refs/heads/release/2.0" - ) - - self.mock_gh.update_issue_body.assert_called_once() - call_args = self.mock_gh.update_issue_body.call_args[0] - self.assertEqual(call_args[0], 123) - self.assertIn( - "branch_url=https://github.com/bazel-contrib/rules_python/tree/release/2.0", - call_args[1], - ) - self.assertIn("commit=abcdef12", call_args[1]) - - def test_create_release_branch_prepare_not_done(self): - # Arrange - args = MagicMock(issue=123, remote="my-remote") - self.mock_gh.get_issue_title.return_value = "Release 2.0.0" - self.mock_gh.get_issue_body.return_value = """ -## Checklist -- [ ] Prepare Release | status=pending -- [ ] Create Release branch | status=pending -""" - # Act - result = CreateReleaseBranch(args, self.mock_git, self.mock_gh).run() - - # Assert - self.assertEqual(result, 1) - self.mock_git.fetch.assert_not_called() - self.mock_git.push.assert_not_called() - self.mock_gh.update_issue_body.assert_not_called() - - def test_create_release_branch_already_checked(self): - # Arrange - args = MagicMock(issue=123, remote="my-remote") - self.mock_gh.get_issue_title.return_value = "Release 2.0.0" - self.mock_gh.get_issue_body.return_value = """ -## Checklist -- [x] Prepare Release | status=done pr=#122 commit=abcdef12 -- [x] Create Release branch | status=done branch=release/2.0 commit=abcdef12 -""" - # Act - result = CreateReleaseBranch(args, self.mock_git, self.mock_gh).run() - - # Assert - self.assertEqual(result, 0) - self.mock_git.fetch.assert_not_called() - self.mock_git.push.assert_not_called() - self.mock_gh.update_issue_body.assert_not_called() - - def test_create_release_branch_already_exists_same_commit(self): - # Arrange - args = MagicMock(issue=123, remote="my-remote") - self.mock_gh.get_issue_title.return_value = "Release 2.0.0" - self.mock_gh.get_issue_body.return_value = """ -## Checklist -- [x] Prepare Release | status=done pr=#122 commit=abcdef12 -- [ ] Create Release branch | status=pending -""" - self.mock_git.remote_branch_exists.return_value = True - self.mock_git.get_commit_sha.return_value = "abcdef12" - - # Act - result = CreateReleaseBranch(args, self.mock_git, self.mock_gh).run() - - # Assert - self.assertEqual(result, 0) - self.mock_git.fetch.assert_called_once_with("my-remote") - self.mock_git.push.assert_not_called() - self.mock_gh.update_issue_body.assert_called_once() # Should still update checklist - - def test_create_release_branch_already_exists_fast_forward(self): - # Arrange - args = MagicMock(issue=123, remote="my-remote") - self.mock_gh.get_issue_title.return_value = "Release 2.0.0" - self.mock_gh.get_issue_body.return_value = """ -## Checklist -- [x] Prepare Release | status=done pr=#122 commit=abcdef12 -- [ ] Create Release branch | status=pending -""" - self.mock_git.remote_branch_exists.return_value = True - self.mock_git.get_commit_sha.return_value = "oldcommit" - self.mock_git.is_ancestor.return_value = True - - # Act - result = CreateReleaseBranch(args, self.mock_git, self.mock_gh).run() - - # Assert - self.assertEqual(result, 0) - self.mock_git.fetch.assert_called_once_with("my-remote") - self.mock_git.push.assert_called_once_with( - "my-remote", "abcdef12:refs/heads/release/2.0" - ) - self.mock_gh.update_issue_body.assert_called_once() - - def test_create_release_branch_already_exists_non_ff(self): - # Arrange - args = MagicMock(issue=123, remote="my-remote") - self.mock_gh.get_issue_title.return_value = "Release 2.0.0" - self.mock_gh.get_issue_body.return_value = """ -## Checklist -- [x] Prepare Release | status=done pr=#122 commit=abcdef12 -- [ ] Create Release branch | status=pending -""" - self.mock_git.remote_branch_exists.return_value = True - self.mock_git.get_commit_sha.return_value = "othercommit" - self.mock_git.is_ancestor.return_value = False - - # Act - result = CreateReleaseBranch(args, self.mock_git, self.mock_gh).run() - - # Assert - self.assertEqual(result, 1) - self.mock_git.fetch.assert_called_once_with("my-remote") - self.mock_git.push.assert_not_called() - self.mock_gh.update_issue_body.assert_not_called() - - -class CmdProcessBackportsTest(unittest.TestCase): - def setUp(self): - _mock_git_and_gh(self) - self.mock_changelog_news = patch( - "tools.private.release.process_backports.changelog_news" - ).start() - self.addCleanup(patch.stopall) - - def test_process_backports_no_pending(self): - args = MagicMock(issue=123, remote="origin", dry_run=False) - self.mock_gh.get_issue_body.return_value = "No backports here" - - result = ProcessBackports(args, self.mock_git, self.mock_gh).run() - - self.assertEqual(result, 0) - self.mock_gh.get_issue_body.assert_called_once_with(123) - self.mock_git.fetch.assert_not_called() - - @patch("tools.private.release.process_backports.datetime") - def test_process_backports_success(self, mock_datetime): - mock_datetime.date.today.return_value = datetime.date(2026, 7, 1) - args = MagicMock(issue=123, remote="origin", dry_run=False) - self.mock_gh.get_issue_title.return_value = "Release 2.0.0" - self.mock_gh.get_issue_body.return_value = """ -## Checklist -- [ ] Prepare Release -- [ ] Create Release branch - -## Backports -- [ ] #124 | status=pending -""" - self.mock_git.get_remote_tags.return_value = [] - - def mock_resolve(items): - for item in items: - if item.pr_ref == "#124": - item.commit = "abcdef12" - item.status = "done" - return items - - self.mock_gh.get_merge_commits_for_prs.side_effect = mock_resolve - - self.mock_git.sort_commits_chronologically.return_value = ["abcdef12"] - self.mock_git.get_commit_sha.return_value = "12345678" - self.mock_git.get_commit_message.return_value = 'Cherry-pick "fix bug"' - - result = ProcessBackports(args, self.mock_git, self.mock_gh).run() - - self.assertEqual(result, 0) - self.mock_git.fetch.assert_has_calls( - [call("origin", tags=True, force=True), call("origin")] - ) - self.mock_git.checkout.assert_called_once_with( - "release/2.0", track_remote="origin" - ) - self.mock_git.cherry_pick.assert_called_once_with("abcdef12") - self.mock_changelog_news.update_changelog.assert_called_once_with( - "2.0.0", "2026-07-01" - ) - self.mock_git.add.assert_called_once_with("CHANGELOG.md", "news/") - self.mock_git.commit.assert_called_once_with( - 'Cherry-pick "fix bug"\n\nWork towards #123', amend=True - ) - self.mock_git.push.assert_called_once_with("origin", "release/2.0") - - self.mock_gh.update_issue_body.assert_called_once() - call_args = self.mock_gh.update_issue_body.call_args[0] - self.assertEqual(call_args[0], 123) - self.assertIn("- [x] #124 | status=done rc=rc0 commit=12345678", call_args[1]) - - @patch("tools.private.release.process_backports.datetime") - def test_process_backports_dry_run(self, mock_datetime): - mock_datetime.date.today.return_value = datetime.date(2026, 7, 1) - args = MagicMock(issue=123, remote="origin", dry_run=True) - self.mock_gh.get_issue_title.return_value = "Release 2.0.0" - self.mock_gh.get_issue_body.return_value = """ -## Checklist -- [ ] Prepare Release -- [ ] Create Release branch - -## Backports -- [ ] #124 | status=pending -""" - self.mock_git.get_remote_tags.return_value = [] - - def mock_resolve(items): - for item in items: - if item.pr_ref == "#124": - item.commit = "abcdef12" - item.status = "done" - return items - - self.mock_gh.get_merge_commits_for_prs.side_effect = mock_resolve - - self.mock_git.sort_commits_chronologically.return_value = ["abcdef12"] - self.mock_git.get_commit_sha.return_value = "12345678" - self.mock_git.get_commit_message.return_value = 'Cherry-pick "fix bug"' - - result = ProcessBackports(args, self.mock_git, self.mock_gh).run() - - self.assertEqual(result, 0) - self.mock_git.fetch.assert_has_calls( - [call("origin", tags=True, force=True), call("origin")] - ) - self.mock_git.checkout.assert_called_once_with( - "release/2.0", track_remote="origin" - ) - self.mock_git.cherry_pick.assert_called_once_with("abcdef12") - self.mock_changelog_news.update_changelog.assert_called_once_with( - "2.0.0", "2026-07-01" - ) - self.mock_git.commit.assert_called_once_with( - 'Cherry-pick "fix bug"\n\nWork towards #123', amend=True - ) - self.mock_git.reset_hard.assert_called_once_with("12345678") - self.mock_git.push.assert_not_called() - self.mock_gh.update_issue_body.assert_not_called() - - def test_process_backports_ignored_and_failed_states(self): - args = MagicMock(issue=123, remote="origin", dry_run=False) - self.mock_gh.get_issue_title.return_value = "Release 2.0.0" - self.mock_gh.get_issue_body.return_value = """ -## Checklist -- [ ] Prepare Release -- [ ] Create Release branch - -## Backports -- [ ] #124 | status=pending -- [ ] #125 | status=pending -- [ ] #126 | status=pending -""" - self.mock_git.get_remote_tags.return_value = [] - - def mock_resolve(items): - for item in items: - if item.pr_ref == "#124": - item.status = "open-pr" - elif item.pr_ref == "#125": - item.status = "draft-pr" - elif item.pr_ref == "#126": - item.status = "error-closed-pr" - return items - - self.mock_gh.get_merge_commits_for_prs.side_effect = mock_resolve - - result = ProcessBackports(args, self.mock_git, self.mock_gh).run() - - self.assertEqual(result, 1) - self.mock_gh.update_issue_body.assert_called_once() - call_args = self.mock_gh.update_issue_body.call_args[0] - self.assertEqual(call_args[0], 123) - self.assertIn("- [ ] #126 | status=error-closed-pr", call_args[1]) - self.assertNotIn("status=open-pr", call_args[1]) - self.assertNotIn("status=draft-pr", call_args[1]) - self.mock_git.checkout.assert_not_called() - self.mock_git.cherry_pick.assert_not_called() - - def test_process_backports_ignored_error_status(self): - args = MagicMock(issue=123, remote="origin", dry_run=False) - self.mock_gh.get_issue_title.return_value = "Release 2.0.0" - self.mock_gh.get_issue_body.return_value = """ -## Checklist -- [ ] Prepare Release -- [ ] Create Release branch - -## Backports -- [ ] #124 | status=error-merge-conflict -- [ ] #125 | status=error-some-other-error -""" - self.mock_git.get_remote_tags.return_value = [] - self.mock_gh.get_merge_commits_for_prs.return_value = [] - - result = ProcessBackports(args, self.mock_git, self.mock_gh).run() - - self.assertEqual(result, 0) - self.mock_gh.get_merge_commits_for_prs.assert_not_called() - self.mock_git.checkout.assert_not_called() - - @patch("tools.private.release.process_backports.datetime") - def test_process_backports_cherry_pick_failed(self, mock_datetime): - mock_datetime.date.today.return_value = datetime.date(2026, 7, 1) - args = MagicMock(issue=123, remote="origin", dry_run=False) - self.mock_gh.get_issue_title.return_value = "Release 2.0.0" - self.mock_gh.get_issue_body.return_value = """ -## Checklist -- [ ] Prepare Release -- [ ] Create Release branch - -## Backports -- [ ] #124 | status=pending -""" - self.mock_git.get_remote_tags.return_value = [] - - def mock_resolve(items): - for item in items: - if item.pr_ref == "#124": - item.commit = "abcdef12" - item.status = "done" - return items - - self.mock_gh.get_merge_commits_for_prs.side_effect = mock_resolve - - self.mock_git.sort_commits_chronologically.return_value = ["abcdef12"] - self.mock_git.cherry_pick.side_effect = Exception("Cherry-pick conflict") - - result = ProcessBackports(args, self.mock_git, self.mock_gh).run() - - self.assertEqual(result, 1) - self.mock_git.checkout.assert_called_once_with( - "release/2.0", track_remote="origin" - ) - self.mock_git.cherry_pick.assert_called_once_with("abcdef12") - self.mock_git.cherry_pick_abort.assert_called_once() - - self.mock_gh.update_issue_body.assert_called_once() - call_args = self.mock_gh.update_issue_body.call_args[0] - self.assertEqual(call_args[0], 123) - self.assertIn("- [ ] #124 | status=error-merge-conflict", call_args[1]) - - self.mock_git.commit.assert_not_called() - self.mock_git.push.assert_not_called() - - -class GitCheckoutTest(unittest.TestCase): - def setUp(self): - self.git = Git(".") - self.patcher = patch.object(self.git, "_run_git") - self.mock_run_git = self.patcher.start() - self.addCleanup(self.patcher.stop) - - def test_checkout_simple(self): - self.git.checkout("my-branch") - self.mock_run_git.assert_called_once_with( - "checkout", "my-branch", capture_output=False - ) - - @patch("tools.private.release.git.Git.branch_exists") - def test_checkout_track_remote_new_branch(self, mock_branch_exists): - mock_branch_exists.return_value = False - - self.git.checkout("my-branch", track_remote="origin") - - mock_branch_exists.assert_called_once_with("my-branch") - self.mock_run_git.assert_called_once_with( - "checkout", "--track", "origin/my-branch", capture_output=False - ) - - @patch("tools.private.release.git.Git.reset_hard") - @patch("tools.private.release.git.Git.branch_exists") - def test_checkout_track_remote_existing_branch( - self, mock_branch_exists, mock_reset_hard - ): - mock_branch_exists.return_value = True - - self.git.checkout("my-branch", track_remote="origin") - - mock_branch_exists.assert_called_once_with("my-branch") - self.mock_run_git.assert_called_once_with( - "checkout", "my-branch", capture_output=False - ) - mock_reset_hard.assert_called_once_with("origin/my-branch") - - if __name__ == "__main__": unittest.main() diff --git a/tests/tools/private/release/release_test_helper.py b/tests/tools/private/release/release_test_helper.py new file mode 100644 index 0000000000..c35cd0fa40 --- /dev/null +++ b/tests/tools/private/release/release_test_helper.py @@ -0,0 +1,46 @@ +import os +import pathlib +import shutil +import tempfile +import unittest +from unittest.mock import MagicMock, patch + +from tools.private.release.gh import ( + MultipleTrackingIssuesError, + NoTrackingIssueError, +) + + +def _mock_git_and_gh(test_case): + mock_git = MagicMock() + mock_gh = MagicMock() + test_case.mock_git = mock_git + test_case.mock_gh = mock_gh + + # Mock Git inside utils.py since it instantiates it locally + patch("tools.private.release.utils.Git", return_value=mock_git).start() + + mock_gh.MultipleTrackingIssuesError = MultipleTrackingIssuesError + mock_gh.NoTrackingIssueError = NoTrackingIssueError + + test_case.addCleanup(patch.stopall) + + # Apply safe defaults + mock_git.get_current_branch.return_value = None + mock_git.get_tags.return_value = [] + mock_git.get_remote_tags.return_value = [] + + mock_git.status.return_value = "" + mock_git.branch_exists.return_value = False + mock_git.tag_exists.return_value = False + mock_gh.get_release_tracking_issue.side_effect = NoTrackingIssueError("Not found") + mock_gh.get_open_pr.return_value = None + + +class TempDirTestCase(unittest.TestCase): + def setUp(self): + self.tmpdir = pathlib.Path(tempfile.mkdtemp()) + self.original_cwd = os.getcwd() + self.addCleanup(shutil.rmtree, self.tmpdir) + os.chdir(self.tmpdir) + self.addCleanup(os.chdir, self.original_cwd) diff --git a/tests/tools/private/release/utils_test.py b/tests/tools/private/release/utils_test.py new file mode 100644 index 0000000000..796bab1619 --- /dev/null +++ b/tests/tools/private/release/utils_test.py @@ -0,0 +1,266 @@ +import unittest +from unittest.mock import patch + +from tests.tools.private.release.release_test_helper import TempDirTestCase +from tools.private.release import utils + + +class GetLatestVersionTest(unittest.TestCase): + @patch("tools.private.release.git.Git.get_tags") + def test_get_latest_version_success(self, mock_get_tags): + mock_get_tags.return_value = ["0.1.0", "1.0.0", "0.2.0"] + self.assertEqual(utils.get_latest_version(), "1.0.0") + + @patch("tools.private.release.git.Git.get_tags") + def test_get_latest_version_rc_is_latest(self, mock_get_tags): + mock_get_tags.return_value = ["0.1.0", "1.0.0", "1.1.0rc0"] + with self.assertRaisesRegex( + ValueError, "The latest version is a pre-release version: 1.1.0rc0" + ): + utils.get_latest_version() + + @patch("tools.private.release.git.Git.get_tags") + def test_get_latest_version_no_tags(self, mock_get_tags): + mock_get_tags.return_value = [] + with self.assertRaisesRegex( + RuntimeError, "No git tags found matching X.Y.Z or X.Y.ZrcN format." + ): + utils.get_latest_version() + + @patch("tools.private.release.git.Git.get_tags") + def test_get_latest_version_no_matching_tags(self, mock_get_tags): + mock_get_tags.return_value = ["v1.0", "latest"] + with self.assertRaisesRegex( + RuntimeError, "No git tags found matching X.Y.Z or X.Y.ZrcN format." + ): + utils.get_latest_version() + + @patch("tools.private.release.git.Git.get_tags") + def test_get_latest_version_only_rc_tags(self, mock_get_tags): + mock_get_tags.return_value = ["1.0.0rc0", "1.1.0rc0"] + with self.assertRaisesRegex( + ValueError, "The latest version is a pre-release version: 1.1.0rc0" + ): + utils.get_latest_version() + + +class GetLatestRcTagTest(unittest.TestCase): + @patch("tools.private.release.git.Git.get_tags") + def test_get_latest_rc_tag_no_tags(self, mock_get_tags): + mock_get_tags.return_value = [] + self.assertIsNone(utils.get_latest_rc_tag("2.0.0")) + + @patch("tools.private.release.git.Git.get_tags") + def test_get_latest_rc_tag_no_matching_tags(self, mock_get_tags): + mock_get_tags.return_value = [ + "1.0.0", + "2.0.0", + "v2.0.0-rc0", + "2.1.0-rc0", + ] + self.assertIsNone(utils.get_latest_rc_tag("2.0.0")) + + @patch("tools.private.release.git.Git.get_tags") + def test_get_latest_rc_tag_success(self, mock_get_tags): + mock_get_tags.return_value = [ + "2.0.0-rc0", + "2.0.0-rc2", + "2.0.0-rc1", + "2.1.0-rc0", + ] + self.assertEqual(utils.get_latest_rc_tag("2.0.0"), "2.0.0-rc2") + + @patch("tools.private.release.git.Git.get_tags") + def test_get_latest_rc_tag_ignores_v_prefix(self, mock_get_tags): + mock_get_tags.return_value = ["v2.0.0-rc0", "2.0.0-rc1"] + self.assertEqual(utils.get_latest_rc_tag("2.0.0"), "2.0.0-rc1") + + @patch("tools.private.release.git.Git.get_remote_tags") + def test_get_latest_rc_tag_remote_success(self, mock_get_remote_tags): + mock_get_remote_tags.return_value = [ + "2.0.0-rc0", + "2.0.0-rc2", + "2.0.0-rc1", + "2.1.0-rc0", + ] + self.assertEqual(utils.get_latest_rc_tag("2.0.0", remote="origin"), "2.0.0-rc2") + mock_get_remote_tags.assert_called_once_with("origin") + + +class DetermineNextVersionTest(TempDirTestCase): + def setUp(self): + super().setUp() + self.mock_get_latest_version = patch( + "tools.private.release.utils.get_latest_version" + ).start() + self.mock_get_current_branch = patch( + "tools.private.release.git.Git.get_current_branch" + ).start() + self.mock_get_current_branch.return_value = "main" + self.addCleanup(patch.stopall) + + def test_no_markers(self): + (self.tmpdir / "mock_file.bzl").write_text("no markers here") + self.mock_get_latest_version.return_value = "1.2.3" + + next_version = utils.determine_next_version() + + self.assertEqual(next_version, "1.2.4") + + def test_only_patch(self): + (self.tmpdir / "mock_file.bzl").write_text( + ":::{versionchanged} VERSION_NEXT_PATCH" + ) + self.mock_get_latest_version.return_value = "1.2.3" + + next_version = utils.determine_next_version() + + self.assertEqual(next_version, "1.2.4") + + def test_only_feature(self): + (self.tmpdir / "mock_file.bzl").write_text( + ":::{versionadded} VERSION_NEXT_FEATURE" + ) + self.mock_get_latest_version.return_value = "1.2.3" + + next_version = utils.determine_next_version() + + self.assertEqual(next_version, "1.3.0") + + def test_both_markers(self): + (self.tmpdir / "mock_file_patch.bzl").write_text( + ":::{versionchanged} VERSION_NEXT_PATCH" + ) + (self.tmpdir / "mock_file_feature.bzl").write_text( + ":::{versionadded} VERSION_NEXT_FEATURE" + ) + self.mock_get_latest_version.return_value = "1.2.3" + + next_version = utils.determine_next_version() + + self.assertEqual(next_version, "1.3.0") + + @patch("tools.private.release.git.Git.get_current_branch") + @patch("tools.private.release.git.Git.get_tags") + def test_determine_next_version_on_release_branch_with_existing_tags( + self, mock_get_tags, mock_get_branch + ): + mock_get_branch.return_value = "release/0.37" + mock_get_tags.return_value = ["0.37.0", "0.37.1", "0.36.0"] + + next_version = utils.determine_next_version() + + self.assertEqual(next_version, "0.37.2") + + @patch("tools.private.release.git.Git.get_current_branch") + @patch("tools.private.release.git.Git.get_tags") + def test_determine_next_version_on_release_branch_no_tags( + self, mock_get_tags, mock_get_branch + ): + mock_get_branch.return_value = "release/0.38" + mock_get_tags.return_value = ["0.37.0"] # No 0.38.x tags + + next_version = utils.determine_next_version() + + self.assertEqual(next_version, "0.38.0") + + @patch("tools.private.release.git.Git.get_current_branch") + @patch("tools.private.release.git.Git.get_tags") + def test_determine_next_version_on_release_branch_with_active_rc( + self, mock_get_tags, mock_get_branch + ): + mock_get_branch.return_value = "release/0.37" + # 0.37.0-rc0 and rc1 exist, but no stable 0.37.0 yet + mock_get_tags.return_value = ["0.37.0-rc0", "0.37.0-rc1", "0.36.0"] + + next_version = utils.determine_next_version() + + # Should target 0.37.0, not 0.37.1 + self.assertEqual(next_version, "0.37.0") + + @patch("tools.private.release.git.Git.get_current_branch") + @patch("tools.private.release.git.Git.get_tags") + def test_determine_next_version_on_release_branch_with_stable_and_active_patch_rc( + self, mock_get_tags, mock_get_branch + ): + mock_get_branch.return_value = "release/0.37" + # 0.37.0 stable exists, and 0.37.1-rc0 exists (but no stable 0.37.1 yet) + mock_get_tags.return_value = ["0.37.0", "0.37.1-rc0", "0.36.0"] + + next_version = utils.determine_next_version() + + # Should target 0.37.1, not 0.37.2 + self.assertEqual(next_version, "0.37.1") + + @patch("tools.private.release.git.Git.get_current_branch") + def test_determine_next_version_on_main_branch_fallback(self, mock_get_branch): + mock_get_branch.return_value = "main" + # Should fallback to default behavior (which uses mock_get_latest_version from setUp) + self.mock_get_latest_version.return_value = "1.2.3" + (self.tmpdir / "mock_file.bzl").write_text("no markers here") + + next_version = utils.determine_next_version() + + self.assertEqual(next_version, "1.2.4") + + +class ReplaceVersionNextTest(TempDirTestCase): + def test_replace_version_next(self): + # Arrange + mock_file_content = """ +:::{versionadded} VERSION_NEXT_FEATURE +blabla +::: + +:::{versionchanged} VERSION_NEXT_PATCH +blabla +::: +""" + (self.tmpdir / "mock_file.bzl").write_text(mock_file_content) + + utils.replace_version_next("0.28.0") + + new_content = (self.tmpdir / "mock_file.bzl").read_text() + + self.assertIn(":::{versionadded} 0.28.0", new_content) + self.assertIn(":::{versionadded} 0.28.0", new_content) + self.assertNotIn("VERSION_NEXT_FEATURE", new_content) + self.assertNotIn("VERSION_NEXT_PATCH", new_content) + + def test_replace_version_next_excludes_bazel_dirs(self): + # Arrange + mock_file_content = """ +:::{versionadded} VERSION_NEXT_FEATURE +blabla +::: +""" + bazel_dir = self.tmpdir / "bazel-rules_python" + bazel_dir.mkdir() + (bazel_dir / "mock_file.bzl").write_text(mock_file_content) + + tools_dir = self.tmpdir / "tools" / "private" / "release" + tools_dir.mkdir(parents=True) + (tools_dir / "mock_file.bzl").write_text(mock_file_content) + + tests_dir = self.tmpdir / "tests" / "tools" / "private" / "release" + tests_dir.mkdir(parents=True) + (tests_dir / "mock_file.bzl").write_text(mock_file_content) + + version = "0.28.0" + + # Act + utils.replace_version_next(version) + + # Assert + new_content = (bazel_dir / "mock_file.bzl").read_text() + self.assertIn("VERSION_NEXT_FEATURE", new_content) + + new_content = (tools_dir / "mock_file.bzl").read_text() + self.assertIn("VERSION_NEXT_FEATURE", new_content) + + new_content = (tests_dir / "mock_file.bzl").read_text() + self.assertIn("VERSION_NEXT_FEATURE", new_content) + + +if __name__ == "__main__": + unittest.main() diff --git a/tools/private/release/BUILD.bazel b/tools/private/release/BUILD.bazel index 7ea3d2e535..a63d2111f7 100644 --- a/tools/private/release/BUILD.bazel +++ b/tools/private/release/BUILD.bazel @@ -7,28 +7,23 @@ py_library( srcs = ["changelog_news.py"], ) +py_library( + name = "release_lib", + srcs = glob( + ["*.py"], + exclude = ["changelog_news.py"], + ), + deps = [ + ":changelog_news", + "@dev_pip//packaging", + ], +) + py_binary( name = "release", - srcs = [ - "__init__.py", - "complete_prepare.py", - "create_rc.py", - "create_release_branch.py", - "create_release_issue.py", - "determine_next_version.py", - "gh.py", - "git.py", - "prepare.py", - "process_backports.py", - "promote_rc.py", - "release.py", - "release_issue.py", - "shell.py", - "utils.py", - ], + srcs = ["release.py"], main = "release.py", deps = [ - ":changelog_news", - "@dev_pip//packaging", + ":release_lib", ], ) diff --git a/tools/private/release/create_rc.py b/tools/private/release/create_rc.py index ea4d8d7450..4e8564a9b8 100644 --- a/tools/private/release/create_rc.py +++ b/tools/private/release/create_rc.py @@ -98,6 +98,12 @@ def run(self) -> int: self.git.tag(next_rc, target_ref) self.git.push(args.remote, next_rc) + import os + + if "GITHUB_OUTPUT" in os.environ: + with open(os.environ["GITHUB_OUTPUT"], "a") as f: + f.write(f"tag_name={next_rc}\n") + # Check off the appropriate "Tag RC{N}" task in the checklist print(f"Checking off Tag RC{next_rc_num} task...") metadata = {"status": "done", "tag": next_rc, "commit": commit_sha[:8]} @@ -110,7 +116,7 @@ def run(self) -> int: tag_url = f"{REPO_URL}/releases/tag/{next_rc}" bcr_entry_url = f"https://registry.bazel.build/modules/rules_python/{version}" bcr_search_url = f"https://github.com/bazelbuild/bazel-central-registry/pulls?q=is%3Apr+rules_python+{version}" - release_workflow_url = f"{REPO_URL}/actions/workflows/release.yml" + release_workflow_url = f"{REPO_URL}/actions/workflows/release_publish.yaml" comment_body = f"""**New Release Candidate Tagged!** 🐍🌿 Release Candidate **{next_rc}** has been successfully generated and tagged on branch `{branch_name}`. diff --git a/tools/private/release/process_backports.py b/tools/private/release/process_backports.py index 21af2d5154..83394bd27e 100644 --- a/tools/private/release/process_backports.py +++ b/tools/private/release/process_backports.py @@ -12,7 +12,10 @@ parse_backports, update_task_in_body, ) -from tools.private.release.utils import get_latest_rc_tag +from tools.private.release.utils import ( + get_latest_rc_tag, + replace_version_next, +) class ProcessBackports: @@ -90,8 +93,12 @@ def _cherry_pick_and_update_prs( release_date = datetime.date.today().strftime("%Y-%m-%d") changelog_news.update_changelog(version, release_date) - # Stage changelog changes and news/ deletions - self.git.add("CHANGELOG.md", "news/") + # Replace version markers that might have been introduced by the backport + print(f"Replacing version markers for PR {item.pr_ref}...") + replace_version_next(version) + + # Stage changelog changes, news/ deletions, and version placeholder updates + self.git.add_modified_and_deleted() # Amend cherry-pick commit to include news merging and deletions, # and reference the release tracking issue. diff --git a/tools/private/release/promote_rc.py b/tools/private/release/promote_rc.py index 74671daa71..80b4ac8065 100644 --- a/tools/private/release/promote_rc.py +++ b/tools/private/release/promote_rc.py @@ -76,7 +76,7 @@ def run(self) -> int: f" {latest_rc} to {version}." ) print(f"[DRY RUN] Would tag commit {commit_sha[:8]} as {version}") - print(f"[DRY RUN] Would push tag {version} to upstream") + print(f"[DRY RUN] Would push tag {version} to {args.remote}") print(f"[DRY RUN] Would update tracking issue #{issue_num} checklist") print(f"[DRY RUN] Would post comment to tracking issue #{issue_num}") return 0 diff --git a/tools/private/release/release_issue.py b/tools/private/release/release_issue.py index 276261bef6..32e0a0fee7 100644 --- a/tools/private/release/release_issue.py +++ b/tools/private/release/release_issue.py @@ -96,11 +96,8 @@ def parse_metadata_line(line): metadata = {} if metadata_str: - pairs = metadata_str.strip().split() - for pair in pairs: - if "=" in pair: - k, v = pair.split("=", 1) - metadata[k] = v + for k, v in re.findall(r"(\w+)\s*=\s*(\S+)", metadata_str): + metadata[k] = v return { "checked": checked, @@ -116,7 +113,16 @@ def format_metadata_line(checked, name, metadata): if not metadata: return f"- [{check_str}] {name}" - metadata_str = " ".join(f"{k}={v}" for k, v in metadata.items()) + metadata_pairs = [] + for k, v in metadata.items(): + if k == "commit": + # The 'commit' key is special-cased with a space after '=' so that + # GitHub autolinks the commit SHA. Autolinking requires certain + # characters to precede the value. + metadata_pairs.append(f"commit= {v}") + else: + metadata_pairs.append(f"{k}={v}") + metadata_str = " ".join(metadata_pairs) return f"- [{check_str}] {name} | {metadata_str}" From 8629003966d56fff8c919d7c9dc191bae9715c9e Mon Sep 17 00:00:00 2001 From: alex-the-third <139810351+alex-the-third@users.noreply.github.com> Date: Fri, 3 Jul 2026 00:31:51 +0200 Subject: [PATCH 806/922] fix: use locale-neutral Windows ACL for build data (#3887) On localized Windows installations, `build_data_writer.ps1` could fail when creating the output file ACL because it used the English localized `Everyone` principal name. This changes the ACL rule to use the language-neutral `WorldSid` identity instead, preserving the existing Everyone read permission without depending on the display language of Windows. Before: German and other localized Windows installs could fail with `IdentityMappedException` / `IdentityNotMappedException`. After: build data generation succeeds on localized Windows installs. Fixes #3886 Tests: `bazelisk test //tests/build_data:build_data_test --config=fast-tests` --------- Co-authored-by: Richard Levasseur --- news/3886.fixed.md | 1 + python/private/build_data_writer.ps1 | 11 ++++++++++- 2 files changed, 11 insertions(+), 1 deletion(-) create mode 100644 news/3886.fixed.md diff --git a/news/3886.fixed.md b/news/3886.fixed.md new file mode 100644 index 0000000000..2285813f8b --- /dev/null +++ b/news/3886.fixed.md @@ -0,0 +1 @@ +(windows) Fixed build data generation on localized Windows installations. diff --git a/python/private/build_data_writer.ps1 b/python/private/build_data_writer.ps1 index 0074e69d38..05c49fef9c 100644 --- a/python/private/build_data_writer.ps1 +++ b/python/private/build_data_writer.ps1 @@ -21,7 +21,16 @@ $Utf8NoBom = New-Object System.Text.UTF8Encoding $False [System.IO.File]::WriteAllLines($OutputPath, $Lines, $Utf8NoBom) $Acl = Get-Acl $OutputPath -$AccessRule = New-Object System.Security.AccessControl.FileSystemAccessRule("Everyone", "Read", "Allow") +# We use WorldSid because the "Everyone" name is locale-dependent. +$EveryoneSid = New-Object System.Security.Principal.SecurityIdentifier( + [System.Security.Principal.WellKnownSidType]::WorldSid, + $null +) +$AccessRule = New-Object System.Security.AccessControl.FileSystemAccessRule( + $EveryoneSid, + "Read", + "Allow" +) $Acl.SetAccessRule($AccessRule) Set-Acl $OutputPath $Acl From 08025e1a74310fd3c52afb2f48eb8210a964f7f3 Mon Sep 17 00:00:00 2001 From: Ted Kaplan Date: Thu, 2 Jul 2026 17:04:59 -0700 Subject: [PATCH 807/922] feat(py_test): add opt-in safeguard against accidental no-op tests (#3825) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit ## Summary Fixes #3824. `py_test` runs the main module directly and does not automatically invoke a test runner (e.g. `unittest`/`pytest`). A common pitfall is to define test classes/functions but forget to run them, so the test silently passes without executing anything. This adds a [validation action](https://bazel.build/extending/rules#validation_actions) to `py_test` that statically analyzes the main module with `ast` and fails the build if the module's top-level body is "inert" — only definitions, imports, assignments, and docstrings, with nothing that actually runs tests (such as an `if __name__ == "__main__":` guard invoking a runner). Because this is technically a breaking change, it is gated behind a new flag, `//python/config_settings:validate_test_main`, with values `auto`/`enabled`/ `disabled`. The default is `auto`, which currently resolves to `disabled`, and can be flipped to `enabled` in rules_python 3.0. Notes/limitations: * Only applies to `py_test` targets with a `main` source file; `main_module` targets are not checked. * Enabling requires the exec-tools toolchain (default for hermetic toolchains); if it's missing, the build fails with a message pointing at the disable flag. ## Test plan * Analysis tests (`tests/base_rules/py_test`) verify the `PyValidateTestMain` action and `_validation` output group are present when enabled and absent when disabled. * Unit test (`tests/validate_test_main`) covers the `ast` classification (inert vs. runs-something). * Bazel-in-bazel integration test (`tests/integration/validate_test_main`) builds real `py_test` targets with the flag and asserts an inert test fails the build with a descriptive error when enabled, while a correct test builds; and that an inert test builds when disabled / by default. --------- Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> Co-authored-by: Claude Sonnet 4.6 --- .bazelignore | 1 + .bazelrc.deleted_packages | 1 + .../python/config_settings/index.md | 38 +++ news/3825.added.md | 6 + python/config_settings/BUILD.bazel | 9 + python/private/BUILD.bazel | 20 ++ python/private/attributes.bzl | 8 + python/private/common_labels.bzl | 1 + python/private/flags.bzl | 21 ++ python/private/py_executable.bzl | 84 +++++- python/private/py_test_main_validator.py | 224 +++++++++++++++ tests/base_rules/py_test/py_test_tests.bzl | 58 +++- tests/integration/BUILD.bazel | 5 + .../bzlmod_lockfile/MODULE.bazel.lock | 2 +- .../validate_test_main/BUILD.bazel | 21 ++ .../validate_test_main/MODULE.bazel | 10 + .../integration/validate_test_main/WORKSPACE | 13 + .../validate_test_main/WORKSPACE.bzlmod | 0 .../validate_test_main/good_test.py | 10 + .../validate_test_main/import_only_test.py | 3 + .../validate_test_main/inert_test.py | 7 + tests/integration/validate_test_main_test.py | 40 +++ tests/validate_test_main/BUILD.bazel | 10 + .../validate_test_main_test.py | 261 ++++++++++++++++++ 24 files changed, 849 insertions(+), 4 deletions(-) create mode 100644 news/3825.added.md create mode 100644 python/private/py_test_main_validator.py create mode 100644 tests/integration/validate_test_main/BUILD.bazel create mode 100644 tests/integration/validate_test_main/MODULE.bazel create mode 100644 tests/integration/validate_test_main/WORKSPACE create mode 100644 tests/integration/validate_test_main/WORKSPACE.bzlmod create mode 100644 tests/integration/validate_test_main/good_test.py create mode 100644 tests/integration/validate_test_main/import_only_test.py create mode 100644 tests/integration/validate_test_main/inert_test.py create mode 100644 tests/integration/validate_test_main_test.py create mode 100644 tests/validate_test_main/BUILD.bazel create mode 100644 tests/validate_test_main/validate_test_main_test.py diff --git a/.bazelignore b/.bazelignore index 5c3bb7caea..88a4c32056 100644 --- a/.bazelignore +++ b/.bazelignore @@ -37,3 +37,4 @@ tests/integration/py_cc_toolchain_registered/bazel-py_cc_toolchain_registered tests/integration/toolchain_target_settings/bazel-module_under_test tests/integration/unified_pypi/bazel-unified_pypi tests/integration/uv_lock/bazel-uv_lock +tests/integration/validate_test_main/bazel-module_under_test diff --git a/.bazelrc.deleted_packages b/.bazelrc.deleted_packages index ce42333e6f..aca014bb56 100644 --- a/.bazelrc.deleted_packages +++ b/.bazelrc.deleted_packages @@ -42,6 +42,7 @@ common --deleted_packages=tests/integration/runtime_manifests common --deleted_packages=tests/integration/toolchain_target_settings common --deleted_packages=tests/integration/unified_pypi common --deleted_packages=tests/integration/uv_lock +common --deleted_packages=tests/integration/validate_test_main common --deleted_packages=tests/modules/another_module common --deleted_packages=tests/modules/other common --deleted_packages=tests/modules/other/nspkg_delta diff --git a/docs/api/rules_python/python/config_settings/index.md b/docs/api/rules_python/python/config_settings/index.md index 6b7a150518..78f8937a4d 100644 --- a/docs/api/rules_python/python/config_settings/index.md +++ b/docs/api/rules_python/python/config_settings/index.md @@ -242,6 +242,44 @@ The `auto` value The `omit_if_generated_source` value was removed :::: +::::{bzl:flag} validate_test_main +Determines if `py_test` runs a build-time validation that its main module +actually runs tests. + +A common `py_test` pitfall is to define test classes or functions but forget +to add code that runs them (for example, assuming `py_test` automatically +invokes `unittest` or `pytest`). When that happens, the test does nothing and +silently passes. + +When enabled, a validation action statically analyzes the main module and +fails the build if it defines test classes or functions but its top-level body +is "inert" -- i.e. it only contains definitions, imports, assignments, and +docstrings, with nothing that actually runs tests (such as an +`if __name__ == "__main__":` guard that invokes a test runner). + +A module that defines no classes or functions at all (for example, one that +only imports other modules) is always allowed, since it isn't the +"defined some tests but forgot to run them" case this check targets. + +This is only applicable to `py_test` targets that have a `main` source file; +targets using `main_module` are not checked. + +Values: + +* `auto`: (default) Automatically decide the effective value; the current + behavior is `disabled`. +* `enabled`: Run the validation action. +* `disabled`: Don't run the validation action. + +:::{note} +Enabling this requires the exec tools toolchain (with an exec interpreter) to +be registered, which is the case for the default hermetic toolchains. +::: + +:::{versionadded} VERSION_NEXT_FEATURE +::: +:::: + ::::{bzl:flag} py_linux_libc Set what libc is used for the target platform. This will affect which whl binaries will be pulled and what toolchain will be auto-detected. Currently `rules_python` only supplies toolchains compatible with `glibc`. diff --git a/news/3825.added.md b/news/3825.added.md new file mode 100644 index 0000000000..71b20aca91 --- /dev/null +++ b/news/3825.added.md @@ -0,0 +1,6 @@ +(py_test) Added an opt-in safeguard against `py_test` targets that silently +pass without running any tests. Set +{obj}`--@rules_python//python/config_settings:validate_test_main=enabled` to +fail the build when a test's main module only contains inert top-level +statements (definitions, imports, assignments) and never invokes a test +runner ([#3824](https://github.com/bazel-contrib/rules_python/issues/3824)). diff --git a/python/config_settings/BUILD.bazel b/python/config_settings/BUILD.bazel index 97761bea83..c7de1d00da 100644 --- a/python/config_settings/BUILD.bazel +++ b/python/config_settings/BUILD.bazel @@ -10,6 +10,7 @@ load( "LibcFlag", "PrecompileFlag", "PrecompileSourceRetentionFlag", + "ValidateTestMainFlag", "VenvsSitePackages", "VenvsUseDeclareSymlinkFlag", rp_string_flag = "string_flag", @@ -77,6 +78,14 @@ string_flag( visibility = ["//visibility:public"], ) +string_flag( + name = "validate_test_main", + build_setting_default = ValidateTestMainFlag.AUTO, + values = ValidateTestMainFlag.flag_values(), + # NOTE: Only public because it's an implicit dependency of py_test. + visibility = ["//visibility:public"], +) + string_flag( name = "precompile_source_retention", build_setting_default = PrecompileSourceRetentionFlag.AUTO, diff --git a/python/private/BUILD.bazel b/python/private/BUILD.bazel index c3d60ffaf1..9a80b6a6d6 100644 --- a/python/private/BUILD.bazel +++ b/python/private/BUILD.bazel @@ -18,6 +18,7 @@ load("//python:py_binary.bzl", "py_binary") load("//python:py_library.bzl", "py_library") load(":bazel_config_mode.bzl", "bazel_config_mode") load(":py_exec_tools_toolchain.bzl", "current_interpreter_executable") +load(":py_interpreter_program.bzl", "py_interpreter_program") load(":sentinel_impl.bzl", "sentinel") load(":stamp_impl.bzl", "stamp_build_setting") load(":uncachable_version_file.bzl", "define_uncachable_version_file") @@ -219,6 +220,25 @@ py_binary( visibility = ["//:__subpackages__"], ) +# Tool used by py_test's validation action to statically check that the main +# module actually runs tests. See the validate_test_main config setting. +py_interpreter_program( + name = "py_test_main_validator", + main = "py_test_main_validator.py", + # Not actually public. Only public because it's an implicit dependency of + # the py_test rule. + visibility = ["//visibility:public"], +) + +py_library( + name = "py_test_main_validator_lib", + srcs = ["py_test_main_validator.py"], + imports = ["../.."], + visibility = [ + "//tests/validate_test_main:__pkg__", + ], +) + bzl_library( name = "attr_builders", srcs = ["attr_builders.bzl"], diff --git a/python/private/attributes.bzl b/python/private/attributes.bzl index ad741d078d..61b41d527c 100644 --- a/python/private/attributes.bzl +++ b/python/private/attributes.bzl @@ -535,6 +535,14 @@ environment when the test is executed by bazel test. "@platforms//os:watchos", ], ), + "_validate_test_main": lambda: attrb.Label( + default = "//python/private:py_test_main_validator", + cfg = "exec", + ), + "_validate_test_main_flag": lambda: attrb.Label( + default = labels.VALIDATE_TEST_MAIN, + providers = [BuildSettingInfo], + ), }) # Attributes specific to Python test-equivalent executable rules. Such rules may diff --git a/python/private/common_labels.bzl b/python/private/common_labels.bzl index 135f8c0a1b..6cc42ecf8d 100644 --- a/python/private/common_labels.bzl +++ b/python/private/common_labels.bzl @@ -29,6 +29,7 @@ labels = struct( PY_FREETHREADED = str(Label("//python/config_settings:py_freethreaded")), PY_LINUX_LIBC = str(Label("//python/config_settings:py_linux_libc")), REPL_DEP = str(Label("//python/bin:repl_dep")), + VALIDATE_TEST_MAIN = str(Label("//python/config_settings:validate_test_main")), VENV = str(Label("//python/config_settings:venv")), VENVS_SITE_PACKAGES = str(Label("//python/config_settings:venvs_site_packages")), VENVS_USE_DECLARE_SYMLINK = str(Label("//python/config_settings:venvs_use_declare_symlink")), diff --git a/python/private/flags.bzl b/python/private/flags.bzl index d9e3aa41c3..32769e4084 100644 --- a/python/private/flags.bzl +++ b/python/private/flags.bzl @@ -87,6 +87,27 @@ AddSrcsToRunfilesFlag = FlagEnum( is_enabled = _AddSrcsToRunfilesFlag_is_enabled, ) +def _ValidateTestMainFlag_is_enabled(ctx): + value = ctx.attr._validate_test_main_flag[BuildSettingInfo].value + if value == ValidateTestMainFlag.AUTO: + # Default off; intended to be flipped to enabled in a future major + # version (e.g. rules_python 3.0). + value = ValidateTestMainFlag.DISABLED + return value == ValidateTestMainFlag.ENABLED + +# Determines if py_test runs a validation action that statically checks the +# main module actually runs tests (instead of silently passing). +# buildifier: disable=name-conventions +ValidateTestMainFlag = FlagEnum( + # Automatically decide the effective value; currently resolves to disabled. + AUTO = "auto", + # Run the validation action. + ENABLED = "enabled", + # Don't run the validation action. + DISABLED = "disabled", + is_enabled = _ValidateTestMainFlag_is_enabled, +) + def _string_flag_impl(ctx): if ctx.attr.override: value = ctx.attr.override diff --git a/python/private/py_executable.bzl b/python/private/py_executable.bzl index 11246ec513..c519af8c09 100644 --- a/python/private/py_executable.bzl +++ b/python/private/py_executable.bzl @@ -57,12 +57,13 @@ load( "runfiles_root_path", ) load(":common_labels.bzl", "labels") -load(":flags.bzl", "BootstrapImplFlag", "VenvsUseDeclareSymlinkFlag", "read_possibly_native_flag") +load(":flags.bzl", "BootstrapImplFlag", "ValidateTestMainFlag", "VenvsUseDeclareSymlinkFlag", "read_possibly_native_flag") load(":precompile.bzl", "maybe_precompile") load(":py_cc_link_params_info.bzl", "PyCcLinkParamsInfo") load(":py_executable_info.bzl", "PyExecutableInfo") load(":py_info.bzl", "PyInfo", "VenvSymlinkKind") load(":py_internal.bzl", "py_internal") +load(":py_interpreter_program.bzl", "PyInterpreterProgramInfo") load(":py_runtime_info.bzl", "DEFAULT_STUB_SHEBANG") load(":reexports.bzl", "BuiltinPyInfo", "BuiltinPyRuntimeInfo") load(":rule_builders.bzl", "ruleb") @@ -1170,6 +1171,11 @@ def py_executable_base_impl(ctx, *, semantics, is_test, inherited_environment = main_py = determine_main(ctx) else: main_py = None + + # Keep a reference to the main source file (before it may be replaced with a + # precompiled pyc below) so the test-main validation can statically analyze + # the original source. + main_py_source = main_py direct_sources = filter_to_py_srcs(ctx.files.srcs) precompile_result = maybe_precompile(ctx, direct_sources) @@ -1288,10 +1294,84 @@ def py_executable_base_impl(ctx, *, semantics, is_test, inherited_environment = implicit_pyc_source_files = implicit_pyc_source_files, imports = imports, ) - _add_provider_output_group_info(providers, py_info, exec_result.output_groups) + output_groups = dict(exec_result.output_groups) + if is_test: + _maybe_add_test_main_validation(ctx, main_py_source, output_groups) + _add_provider_output_group_info(providers, py_info, output_groups) return providers +def _maybe_add_test_main_validation(ctx, main_py, output_groups): + """Adds a validation action that checks the test main actually runs tests. + + This is a safeguard against the common pitfall of defining test classes or + functions but forgetting to invoke a test runner, which causes the test to + silently pass without running anything. See the + `//python/config_settings:validate_test_main` flag. + + Args: + ctx: Rule ctx. + main_py: File or None; the main entry point source file. None when the + target uses `main_module` (which can't be statically analyzed here). + output_groups: dict[str, depset[File]]; mutated in place to add the + `_validation` output group when a validation action is created. + """ + if not ValidateTestMainFlag.is_enabled(ctx): + return + + # `main_module` targets execute a module by name; there's no single source + # file to statically analyze, so the check doesn't apply. + if main_py == None: + return + + exec_tools_toolchain = ctx.toolchains[EXEC_TOOLS_TOOLCHAIN_TYPE] + if exec_tools_toolchain == None or exec_tools_toolchain.exec_tools.exec_interpreter == None: + fail( + "Validating py_test main modules requires the exec tools toolchain " + + "with an exec interpreter, but none was found. Either register one " + + "or set --@rules_python//python/config_settings:validate_test_main=disabled.", + ) + + exec_tools = exec_tools_toolchain.exec_tools + validator = ctx.attr._validate_test_main + program_info = validator[PyInterpreterProgramInfo] + interpreter = exec_tools.exec_interpreter[DefaultInfo].files_to_run + validator_files_to_run = validator[DefaultInfo].files_to_run + + validation_output = ctx.actions.declare_file(ctx.label.name + "_validate_test_main.txt") + + args = ctx.actions.args() + args.add_all(program_info.interpreter_args) + args.add(validator_files_to_run.executable) + args.add("--src", main_py) + args.add("--src_name", main_py.short_path) + args.add("--label", str(ctx.label)) + args.add("--output", validation_output) + + execution_requirements = {} + if testing.ExecutionInfo in validator: + execution_requirements = validator[testing.ExecutionInfo].requirements + + ctx.actions.run( + executable = interpreter, + arguments = [args], + inputs = [main_py], + outputs = [validation_output], + tools = [validator_files_to_run], + mnemonic = "PyValidateTestMain", + progress_message = "Validating py_test main %{label}", + env = program_info.env | { + "PYTHONNOUSERSITE": "1", + "PYTHONSAFEPATH": "1", + }, + execution_requirements = execution_requirements, + toolchain = EXEC_TOOLS_TOOLCHAIN_TYPE, + ) + if "_validation" in output_groups: + output_groups["_validation"] = depset([validation_output], transitive = [output_groups["_validation"]]) + else: + output_groups["_validation"] = depset([validation_output]) + def _get_build_info(ctx, cc_toolchain): build_info_files = py_internal.cc_toolchain_build_info_files(cc_toolchain) if cc_helper.is_stamping_enabled(ctx): diff --git a/python/private/py_test_main_validator.py b/python/private/py_test_main_validator.py new file mode 100644 index 0000000000..e3849c5f57 --- /dev/null +++ b/python/private/py_test_main_validator.py @@ -0,0 +1,224 @@ +"""Static check that a py_test main module actually runs something. + +A common ``py_test`` pitfall is to define test classes or functions but forget +to add any code that actually executes them (for example, assuming that +``py_test`` automatically invokes ``unittest`` or ``pytest``). When that +happens, running the test does nothing and the target silently passes. + +This validator parses the main module with :mod:`ast` and fails if the +module body is "inert", i.e. every top-level statement is one that does not +run anything (definitions, imports, assignments, docstrings, ``pass``). A +single active statement -- a bare call, a loop, or the conventional +``if __name__ == "__main__":`` guard -- is enough to consider the module able +to run tests. Top-level ``if`` and ``try`` blocks are inspected recursively, so +common guards like ``if TYPE_CHECKING:`` or ``try: import foo except +ImportError:`` (whose branches are themselves inert) don't bypass the check. + +As an exception, a module that defines no classes or functions at all (for +example, one that only imports other modules) is always allowed: it isn't the +"defined some tests but forgot to run them" case this check targets, and it +may legitimately rely on import side effects. +""" + +import argparse +import ast +import sys + +# Statement node types that never run any code on their own, regardless of +# their contents. A module whose top-level body consists solely of these (and +# inert assignments/expressions/guards, see below) is considered inert. +_INERT_NODE_TYPES = [ + ast.FunctionDef, + ast.AsyncFunctionDef, + ast.ClassDef, + ast.Import, + ast.ImportFrom, + ast.Global, + ast.Pass, +] + +# `ast.TypeAlias` (PEP 695, e.g. `type Alias = int`) only exists on Python +# 3.12+. Add it dynamically so the validator still imports on older versions. +if hasattr(ast, "TypeAlias"): + _INERT_NODE_TYPES.append(ast.TypeAlias) + +_INERT_NODE_TYPES = tuple(_INERT_NODE_TYPES) + +# `ast.TryStar` (PEP 654, `try/except*`) only exists on Python 3.11+. +_TRY_NODE_TYPES = (ast.Try, ast.TryStar) if hasattr(ast, "TryStar") else (ast.Try,) + +# Expression node types whose evaluation runs code (and thus may run tests). +# `await`/`yield` can't appear at the module top level, but are included for +# completeness when walking nested expressions. +_ACTIVE_EXPR_NODE_TYPES = (ast.Call, ast.Await, ast.Yield, ast.YieldFrom) + + +def _expression_runs_code(value) -> bool: + """Returns True if evaluating the expression would invoke/await/yield. + + Note this walks the expression as written; it does not try to model what + actually executes (e.g. a call inside a `lambda` body won't run until the + lambda is called). Erring toward "runs code" keeps the safeguard from + flagging valid tests. + """ + if value is None: + return False + return any(isinstance(child, _ACTIVE_EXPR_NODE_TYPES) for child in ast.walk(value)) + + +def _all_inert(nodes) -> bool: + return all(_is_inert_statement(node) for node in nodes) + + +def _is_inert_statement(node: ast.stmt) -> bool: + """Returns True if the top-level statement does not run any test code.""" + if isinstance(node, _INERT_NODE_TYPES): + return True + + # Assignments are inert unless their value runs code, e.g. + # `exit_code = unittest.main()` actually runs the tests. + if isinstance(node, (ast.Assign, ast.AnnAssign, ast.AugAssign)): + return not _expression_runs_code(node.value) + + # Bare expression statements: docstrings and other no-op expressions are + # inert; a call/await/yield (e.g. `unittest.main()`) runs code. + # Assert statements are inert unless their condition or message runs code. + if isinstance(node, ast.Assert): + return not (_expression_runs_code(node.test) or _expression_runs_code(node.msg)) + + if isinstance(node, ast.Expr): + return not _expression_runs_code(node.value) + + # An `if` whose condition and branches are entirely inert, e.g. the very + # common `if TYPE_CHECKING:` guard around typing-only imports. A call in the + # condition (e.g. `if feature_enabled():`) or a runner call in a branch body + # makes it active. + if isinstance(node, ast.If): + return ( + not _expression_runs_code(node.test) + and _all_inert(node.body) + and _all_inert(node.orelse) + ) + + # A `try` (or `try/except*`) whose body, handlers, else, and finally are all + # inert, e.g. the common `try: import foo except ImportError: ...` optional + # import pattern. + if isinstance(node, _TRY_NODE_TYPES): + if not ( + _all_inert(node.body) + and _all_inert(node.orelse) + and _all_inert(node.finalbody) + ): + return False + return all(_all_inert(handler.body) for handler in node.handlers) + + return False + + +def module_runs_something(tree: ast.Module) -> bool: + """Returns True if the module body has at least one active statement.""" + return any(not _is_inert_statement(node) for node in tree.body) + + +def _defines_test_code(tree: ast.Module) -> bool: + """Returns True if the module defines any top-level class or function.""" + return any( + isinstance(node, (ast.FunctionDef, ast.AsyncFunctionDef, ast.ClassDef)) + for node in tree.body + ) + + +def module_runs_tests(tree: ast.Module) -> bool: + """Returns True if the module appears able to run tests (or isn't checked). + + The check targets the specific pitfall of defining test classes/functions + but never running them. A module that defines no classes or functions at + all (for example, one that only imports other modules) is not subject to + the check and is always allowed, since it isn't the "defined but never run" + case and may legitimately rely on import side effects. + """ + if module_runs_something(tree): + return True + return not _defines_test_code(tree) + + +def _format_error(label: str, src_name: str) -> str: + target = label or src_name + return ( + "py_test target {target} will not run any tests.\n" + "\n" + "The main module '{src_name}' only contains inert top-level statements " + "(class/function definitions, imports, assignments). Running it does " + "nothing, so the test silently passes without executing any test " + "code.\n" + "\n" + "py_test runs the main module directly; it does not automatically " + "invoke a test runner such as unittest or pytest. Add code that runs " + "your tests, for example:\n" + "\n" + ' if __name__ == "__main__":\n' + " unittest.main()\n" + "\n" + "or use a main module that invokes a runner (e.g. pytest.main()).\n" + "\n" + "This check can be disabled by setting " + "--@rules_python//python/config_settings:validate_test_main=disabled." + ).format(target=target, src_name=src_name) + + +def main(args) -> int: + parser = argparse.ArgumentParser() + parser.add_argument( + "--src", + required=True, + help="Path to the main .py source file to analyze.", + ) + parser.add_argument( + "--src_name", + default="", + help="Human-friendly name of the source file, used in error messages.", + ) + parser.add_argument( + "--label", + default="", + help="The py_test target label, used in error messages.", + ) + parser.add_argument( + "--output", + required=True, + help="Path to the validation marker file to write on success.", + ) + options = parser.parse_args(args) + + src_name = options.src_name or options.src + + with open(options.src, "rb") as f: + source = f.read() + + try: + tree = ast.parse(source, filename=src_name) + except SyntaxError as e: + # A syntax error is surfaced by other actions (compilation/execution). + # The validator can't analyze the file, so don't fail here; treat it as + # passing to avoid duplicate or confusing errors. + sys.stderr.write( + "WARNING: py_test main validator could not parse {}: {}\n".format( + src_name, e + ) + ) + with open(options.output, "w") as out: + out.write("") + return 0 + + if not module_runs_tests(tree): + sys.stderr.write(_format_error(options.label, src_name) + "\n") + return 1 + + # Validation actions must produce their declared outputs on success. + with open(options.output, "w") as out: + out.write("") + return 0 + + +if __name__ == "__main__": + sys.exit(main(sys.argv[1:])) diff --git a/tests/base_rules/py_test/py_test_tests.bzl b/tests/base_rules/py_test/py_test_tests.bzl index fd284beffd..e16204e9f2 100644 --- a/tests/base_rules/py_test/py_test_tests.bzl +++ b/tests/base_rules/py_test/py_test_tests.bzl @@ -16,12 +16,13 @@ load("@rules_testing//lib:analysis_test.bzl", "analysis_test") load("@rules_testing//lib:util.bzl", rt_util = "util") load("//python:py_test.bzl", "py_test") +load("//python/private:common_labels.bzl", "labels") # buildifier: disable=bzl-visibility load( "//tests/base_rules:py_executable_base_tests.bzl", "create_executable_tests", ) load("//tests/base_rules:util.bzl", pt_util = "util") -load("//tests/support:support.bzl", "CC_TOOLCHAIN", "CROSSTOOL_TOP") +load("//tests/support:support.bzl", "CC_TOOLCHAIN", "CROSSTOOL_TOP", "PY_TOOLCHAINS") load("//tests/support/platforms:platforms.bzl", "platform_targets") # The Windows CI currently runs as root, which breaks when @@ -96,6 +97,61 @@ def _test_non_mac_doesnt_require_darwin_for_execution_impl(env, target): _tests.append(_test_non_mac_doesnt_require_darwin_for_execution) +_VALIDATE_TEST_MAIN_CONFIG_SETTINGS = { + "//command_line_option:extra_toolchains": [PY_TOOLCHAINS, CC_TOOLCHAIN], + labels.EXEC_TOOLS_TOOLCHAIN: "enabled", +} + +def _test_validate_test_main_enabled(name, config): + rt_util.helper_target( + config.rule, + name = name + "_subject", + srcs = [name + "_subject.py"], + ) + analysis_test( + name = name, + impl = _test_validate_test_main_enabled_impl, + target = name + "_subject", + config_settings = _VALIDATE_TEST_MAIN_CONFIG_SETTINGS | { + labels.VALIDATE_TEST_MAIN: "enabled", + }, + attr_values = _SKIP_WINDOWS, + ) + +def _test_validate_test_main_enabled_impl(env, target): + mnemonics = [a.mnemonic for a in target.actions] + env.expect.that_collection(mnemonics).contains("PyValidateTestMain") + env.expect.that_bool( + hasattr(target[OutputGroupInfo], "_validation"), + ).equals(True) + +_tests.append(_test_validate_test_main_enabled) + +def _test_validate_test_main_disabled(name, config): + rt_util.helper_target( + config.rule, + name = name + "_subject", + srcs = [name + "_subject.py"], + ) + analysis_test( + name = name, + impl = _test_validate_test_main_disabled_impl, + target = name + "_subject", + config_settings = _VALIDATE_TEST_MAIN_CONFIG_SETTINGS | { + labels.VALIDATE_TEST_MAIN: "disabled", + }, + attr_values = _SKIP_WINDOWS, + ) + +def _test_validate_test_main_disabled_impl(env, target): + mnemonics = [a.mnemonic for a in target.actions] + env.expect.that_collection(mnemonics).not_contains("PyValidateTestMain") + env.expect.that_bool( + hasattr(target[OutputGroupInfo], "_validation"), + ).equals(False) + +_tests.append(_test_validate_test_main_disabled) + def py_test_test_suite(name): config = struct(rule = py_test) native.test_suite( diff --git a/tests/integration/BUILD.bazel b/tests/integration/BUILD.bazel index 9301e19590..abdb37be57 100644 --- a/tests/integration/BUILD.bazel +++ b/tests/integration/BUILD.bazel @@ -118,6 +118,11 @@ rules_python_integration_test( py_main = "toolchain_target_settings_test.py", ) +rules_python_integration_test( + name = "validate_test_main_test", + py_main = "validate_test_main_test.py", +) + rules_python_integration_test( name = "unified_pypi_test", py_main = "unified_pypi_test.py", diff --git a/tests/integration/bzlmod_lockfile/MODULE.bazel.lock b/tests/integration/bzlmod_lockfile/MODULE.bazel.lock index c7468217e9..fd161a3965 100644 --- a/tests/integration/bzlmod_lockfile/MODULE.bazel.lock +++ b/tests/integration/bzlmod_lockfile/MODULE.bazel.lock @@ -252,7 +252,7 @@ }, "@@rules_python+//python/uv:uv.bzl%uv": { "general": { - "bzlTransitiveDigest": "ELjwPp2kLku5M3S/gpjjVjy3TwT760/zVEQ70nJreHU=", + "bzlTransitiveDigest": "OC4ZhWl8jX9wvFicn83AioC9J7Dx6Us2/+EzoGDlPzU=", "usagesDigest": "6yXGw7XDyXjOfqBL0SBu1YBEMMYPQzCE3jTzUCkxPgg=", "recordedInputs": [ "REPO_MAPPING:rules_python+,bazel_tools bazel_tools", diff --git a/tests/integration/validate_test_main/BUILD.bazel b/tests/integration/validate_test_main/BUILD.bazel new file mode 100644 index 0000000000..cf931d3303 --- /dev/null +++ b/tests/integration/validate_test_main/BUILD.bazel @@ -0,0 +1,21 @@ +load("@rules_python//python:py_test.bzl", "py_test") + +# A test whose main module only defines a test case but never runs it. With +# --validate_test_main=enabled this should fail to build. +py_test( + name = "inert_test", + srcs = ["inert_test.py"], +) + +# A test whose main module invokes a runner. This should always build. +py_test( + name = "good_test", + srcs = ["good_test.py"], +) + +# A test whose main module defines nothing and only imports. This is allowed +# even with validation enabled, since it isn't the "defined but never run" case. +py_test( + name = "import_only_test", + srcs = ["import_only_test.py"], +) diff --git a/tests/integration/validate_test_main/MODULE.bazel b/tests/integration/validate_test_main/MODULE.bazel new file mode 100644 index 0000000000..c154ac169e --- /dev/null +++ b/tests/integration/validate_test_main/MODULE.bazel @@ -0,0 +1,10 @@ +module(name = "module_under_test") + +bazel_dep(name = "rules_python", version = "0.0.0") +local_path_override( + module_name = "rules_python", + path = "../../..", +) + +python = use_extension("@rules_python//python/extensions:python.bzl", "python") +python.toolchain(python_version = "3.11") diff --git a/tests/integration/validate_test_main/WORKSPACE b/tests/integration/validate_test_main/WORKSPACE new file mode 100644 index 0000000000..de908549c0 --- /dev/null +++ b/tests/integration/validate_test_main/WORKSPACE @@ -0,0 +1,13 @@ +local_repository( + name = "rules_python", + path = "../../..", +) + +load("@rules_python//python:repositories.bzl", "py_repositories", "python_register_toolchains") + +py_repositories() + +python_register_toolchains( + name = "python_3_11", + python_version = "3.11", +) diff --git a/tests/integration/validate_test_main/WORKSPACE.bzlmod b/tests/integration/validate_test_main/WORKSPACE.bzlmod new file mode 100644 index 0000000000..e69de29bb2 diff --git a/tests/integration/validate_test_main/good_test.py b/tests/integration/validate_test_main/good_test.py new file mode 100644 index 0000000000..46107c96ec --- /dev/null +++ b/tests/integration/validate_test_main/good_test.py @@ -0,0 +1,10 @@ +import unittest + + +class GoodTest(unittest.TestCase): + def test_something(self): + self.assertTrue(True) + + +if __name__ == "__main__": + unittest.main() diff --git a/tests/integration/validate_test_main/import_only_test.py b/tests/integration/validate_test_main/import_only_test.py new file mode 100644 index 0000000000..1bd46dead7 --- /dev/null +++ b/tests/integration/validate_test_main/import_only_test.py @@ -0,0 +1,3 @@ +# This module defines no classes or functions; it only imports. The validation +# action allows it even when enabled. +import os # noqa: F401 diff --git a/tests/integration/validate_test_main/inert_test.py b/tests/integration/validate_test_main/inert_test.py new file mode 100644 index 0000000000..8decf676d4 --- /dev/null +++ b/tests/integration/validate_test_main/inert_test.py @@ -0,0 +1,7 @@ +import unittest + + +class InertTest(unittest.TestCase): + def test_nothing_runs(self): + # This test case is never executed because nothing invokes a runner. + self.assertTrue(True) diff --git a/tests/integration/validate_test_main_test.py b/tests/integration/validate_test_main_test.py new file mode 100644 index 0000000000..4d108411d5 --- /dev/null +++ b/tests/integration/validate_test_main_test.py @@ -0,0 +1,40 @@ +import unittest + +from tests.integration import runner + +_FLAG = "--@rules_python//python/config_settings:validate_test_main" + + +class ValidateTestMainTest(runner.TestCase): + def test_inert_test_fails_when_enabled(self): + """An inert test main should fail the build when validation is on.""" + result = self.run_bazel( + "build", + f"{_FLAG}=enabled", + "//:inert_test", + check=False, + ) + self.assertNotEqual(result.exit_code, 0, "Expected build to fail") + self.assert_result_matches(result, r"will not run any tests") + + def test_good_test_builds_when_enabled(self): + """A main that invokes a runner should build when validation is on.""" + self.run_bazel("build", f"{_FLAG}=enabled", "//:good_test") + + def test_import_only_test_builds_when_enabled(self): + """A main that only imports (defines nothing) is allowed when on.""" + self.run_bazel("build", f"{_FLAG}=enabled", "//:import_only_test") + + def test_inert_test_builds_when_disabled(self): + """Validation is off by default, so even an inert test builds.""" + self.run_bazel("build", f"{_FLAG}=disabled", "//:inert_test") + + def test_inert_test_builds_by_default(self): + """The default (auto) resolves to disabled, so an inert test builds.""" + self.run_bazel("build", "//:inert_test") + + +if __name__ == "__main__": + # Enabling this makes the runner log subprocesses as the test goes along. + # logging.basicConfig(level = "INFO") + unittest.main() diff --git a/tests/validate_test_main/BUILD.bazel b/tests/validate_test_main/BUILD.bazel new file mode 100644 index 0000000000..94163a2fde --- /dev/null +++ b/tests/validate_test_main/BUILD.bazel @@ -0,0 +1,10 @@ +load("//python:py_test.bzl", "py_test") + +py_test( + name = "validate_test_main_test", + srcs = ["validate_test_main_test.py"], + main = "validate_test_main_test.py", + deps = [ + "//python/private:py_test_main_validator_lib", + ], +) diff --git a/tests/validate_test_main/validate_test_main_test.py b/tests/validate_test_main/validate_test_main_test.py new file mode 100644 index 0000000000..4cbc55c937 --- /dev/null +++ b/tests/validate_test_main/validate_test_main_test.py @@ -0,0 +1,261 @@ +#!/usr/bin/env python3 +import ast +import textwrap +import unittest + +from python.private.py_test_main_validator import module_runs_tests + + +def _runs_tests(source: str) -> bool: + tree = ast.parse(textwrap.dedent(source)) + return module_runs_tests(tree) + + +class ModuleRunsTestsTest(unittest.TestCase): + def test_only_definitions_is_rejected(self): + self.assertFalse( + _runs_tests( + """ + import unittest + + class MyTest(unittest.TestCase): + def test_foo(self): + self.assertTrue(True) + """ + ) + ) + + def test_definitions_with_assignments_is_rejected(self): + self.assertFalse( + _runs_tests( + """ + import unittest + + CONSTANT = 5 + + class MyTest(unittest.TestCase): + def test_foo(self): + pass + + def helper(): pass + """ + ) + ) + + def test_global_statement_with_definition_is_rejected(self): + self.assertFalse( + _runs_tests( + """ + global x + + class MyTest: + def test_foo(self): + pass + """ + ) + ) + + def test_assert_statement_with_definition_is_rejected(self): + # A bare `assert` with an inert condition runs no tests. + self.assertFalse( + _runs_tests( + """ + assert True + + class MyTest: + def test_foo(self): + pass + """ + ) + ) + + def test_assert_statement_with_active_expression_runs_tests(self): + # An assert whose condition runs a call actually runs the tests. + self.assertTrue( + _runs_tests( + """ + import unittest + + class MyTest(unittest.TestCase): + def test_foo(self): + pass + + assert unittest.main() + """ + ) + ) + + @unittest.skipUnless( + hasattr(ast, "TypeAlias"), "PEP 695 type aliases require Python 3.12+" + ) + def test_type_alias_with_definition_is_rejected(self): + self.assertFalse( + _runs_tests( + """ + type Alias = int + + class MyTest: + def test_foo(self): + pass + """ + ) + ) + + def test_type_checking_block_with_definition_is_rejected(self): + # `if TYPE_CHECKING:` guarding typing-only imports is inert; with a test + # definition and no runner, the module is still rejected. + self.assertFalse( + _runs_tests( + """ + from typing import TYPE_CHECKING + import unittest + + if TYPE_CHECKING: + from typing import Any + + class MyTest(unittest.TestCase): + def test_foo(self): + pass + """ + ) + ) + + def test_try_except_import_error_with_definition_is_rejected(self): + # `try: import foo except ImportError:` optional imports are inert. + self.assertFalse( + _runs_tests( + """ + try: + import foo + except ImportError: + foo = None + + import unittest + + class MyTest(unittest.TestCase): + def test_foo(self): + pass + """ + ) + ) + + def test_definitions_with_active_assignment_runs_tests(self): + # An assignment whose value runs a call actually runs the tests. + self.assertTrue( + _runs_tests( + """ + import unittest + + class MyTest(unittest.TestCase): + def test_foo(self): + pass + + exit_code = unittest.main() + """ + ) + ) + + def test_definitions_with_active_expression_runs_tests(self): + self.assertTrue( + _runs_tests( + """ + import sys + import unittest + + class MyTest(unittest.TestCase): + def test_foo(self): + pass + + sys.exit(unittest.main()) + """ + ) + ) + + def test_if_block_invoking_runner_runs_tests(self): + # A runner call inside an `if` body must still count as active. + self.assertTrue( + _runs_tests( + """ + import sys + import unittest + + class MyTest(unittest.TestCase): + def test_foo(self): + pass + + if "--run" in sys.argv: + unittest.main() + """ + ) + ) + + def test_import_only_module_is_allowed(self): + # A module that defines nothing and only imports other modules is not + # the "defined but never run" case, so it is allowed. + self.assertTrue( + _runs_tests( + """ + import my_tests + from my_pkg.tests import suite + """ + ) + ) + + def test_imports_and_assignments_without_definitions_is_allowed(self): + self.assertTrue( + _runs_tests( + """ + import my_tests + + CONSTANT = 5 + """ + ) + ) + + def test_empty_module_is_allowed(self): + self.assertTrue(_runs_tests("")) + + def test_docstring_only_is_allowed(self): + self.assertTrue(_runs_tests('"""A module docstring."""')) + + def test_if_name_main_guard_runs_tests(self): + self.assertTrue( + _runs_tests( + """ + import unittest + + class MyTest(unittest.TestCase): + def test_foo(self): + pass + + if __name__ == "__main__": + unittest.main() + """ + ) + ) + + def test_bare_call_runs_tests(self): + self.assertTrue( + _runs_tests( + """ + import pytest + pytest.main() + """ + ) + ) + + def test_top_level_loop_runs_tests(self): + self.assertTrue( + _runs_tests( + """ + def f(): pass + + for _ in range(1): + f() + """ + ) + ) + + +if __name__ == "__main__": + unittest.main() From 7968ac919487a14b355c621d746ce93c02080a3b Mon Sep 17 00:00:00 2001 From: Jan Winkler <45763961+golithe@users.noreply.github.com> Date: Fri, 3 Jul 2026 02:08:04 +0200 Subject: [PATCH 808/922] feat: enable pyproject.toml as single source of truth for Python version (#3514) - Enables using pyproject.toml as single source of truth for Python version via `python.defaults(pyproject_toml=...)`, `pip.default(pyproject_toml=...)`, and `pip.parse(pyproject_toml=...)`. - The version is read from the requires-python field and used as the default toolchain version, so anything relying on the default (f.e. `py_binary`/`py_test`, `pip.parse`, `compile_pip_requirements`) picks it up with no explicit `python_version` in BUILD files: ```starlark # MODULE.bazel python.defaults(pyproject_toml = "//:pyproject.toml") pip.default(pyproject_toml = "//:pyproject.toml") # BUILD.bazel (version comes from default, no need to restate it) compile_pip_requirements( name="requirements", requirements_txt = "requirements.txt" ) ``` - Relates to #1708 (lifts the mandatory `python_version` restriction on `pip.parse`) Changes: - Added `pyproject_toml` attribute to `python.defaults()` and `pip.default()` and `pip.parse()`, reading the version from `requires-python` - Parses `pyproject.toml` with the pure-Starlark `@toml.bzl (toml.decode)` (no host Python interpreter required) - Auto-registers a toolchain for the default version if no explicit `python.toolchain()` call exists - Made `python_version` optional in `pip.parse()`: the version can come from `pip.parse(pyproject_toml=...)` directly (which also works for non-root modules) or fall back to `pip.default(pyproject_toml=...)` - Precedence order: `ENV > pyproject_toml > python_version_file > python_version` Requirements: - `requires-python = "==X.Y.Z"` format in pyproject.toml Thanks to @aignas and @rickeylev for the review and design guidance --------- Co-authored-by: Ignas Anikevicius <240938+aignas@users.noreply.github.com> --- news/3514.added.md | 1 + python/private/BUILD.bazel | 10 ++++ python/private/pypi/BUILD.bazel | 1 + python/private/pypi/extension.bzl | 61 +++++++++++++++++++++++- python/private/pypi/hub_builder.bzl | 27 ++++++----- python/private/pyproject_utils.bzl | 51 ++++++++++++++++++++ python/private/python.bzl | 46 +++++++++++++++++- tests/pypi/extension/extension_tests.bzl | 58 +++++++++++++++++++++- tests/pypi/extension/pip_parse.bzl | 4 +- 9 files changed, 242 insertions(+), 17 deletions(-) create mode 100644 news/3514.added.md create mode 100644 python/private/pyproject_utils.bzl diff --git a/news/3514.added.md b/news/3514.added.md new file mode 100644 index 0000000000..bf2a119380 --- /dev/null +++ b/news/3514.added.md @@ -0,0 +1 @@ +(pip,python) Added `pyproject_toml` attribute to {obj}`pip.default`, {obj}`pip.parse` and {obj}`python.defaults` to read the default Python version from the `requires-python` field of `pyproject.toml`. diff --git a/python/private/BUILD.bazel b/python/private/BUILD.bazel index 9a80b6a6d6..e4d06d4a89 100644 --- a/python/private/BUILD.bazel +++ b/python/private/BUILD.bazel @@ -729,6 +729,7 @@ bzl_library( ":full_version", ":pbs_manifest", ":platform_info", + ":pyproject_utils", ":python_register_toolchains", ":pythons_hub", ":repo_utils", @@ -896,6 +897,15 @@ bzl_library( ], ) +bzl_library( + name = "pyproject_utils", + srcs = ["pyproject_utils.bzl"], + deps = [ + ":version", + "@toml.bzl//:toml", + ], +) + bzl_library( name = "bzlmod_enabled", srcs = ["bzlmod_enabled.bzl"], diff --git a/python/private/pypi/BUILD.bazel b/python/private/pypi/BUILD.bazel index 677154ee46..8dd32afb9c 100644 --- a/python/private/pypi/BUILD.bazel +++ b/python/private/pypi/BUILD.bazel @@ -125,6 +125,7 @@ bzl_library( ":whl_library", "//python/private:auth", "//python/private:normalize_name", + "//python/private:pyproject_utils", "//python/private:repo_utils", "@pythons_hub//:interpreters", "@pythons_hub//:versions", diff --git a/python/private/pypi/extension.bzl b/python/private/pypi/extension.bzl index 094cac7f3f..b79d04c075 100644 --- a/python/private/pypi/extension.bzl +++ b/python/private/pypi/extension.bzl @@ -20,6 +20,7 @@ load("@rules_python_internal//:rules_python_config.bzl", rp_config = "config") load("@toml.bzl", "toml") load("//python/private:auth.bzl", "AUTH_ATTRS") load("//python/private:normalize_name.bzl", "normalize_name") +load("//python/private:pyproject_utils.bzl", "read_pyproject", "version_from_requires_python") load("//python/private:repo_utils.bzl", "repo_utils") load(":hub_builder.bzl", "hub_builder") load(":hub_repository.bzl", "hub_repository", "whl_config_settings_to_json") @@ -208,6 +209,7 @@ def build_config( default_hub = None defaults = { "platforms": default_platforms(), + "python_version": None, } for mod in module_ctx.modules: if not (mod.is_root or mod.name == "rules_python"): @@ -219,6 +221,11 @@ def build_config( if default_hub: fail("Duplicate pip.default tag: only one explicit default PyPI hub is allowed.") default_hub = tag.default_hub + pyproject_toml = tag.pyproject_toml + if pyproject_toml: + pyproject = read_pyproject(module_ctx, pyproject_toml) + if pyproject.requires_python: + defaults["python_version"] = version_from_requires_python(pyproject.requires_python) platform = tag.platform if platform: @@ -256,6 +263,7 @@ def build_config( default_hub = default_hub, index_url = defaults.get("index_url", "https://pypi.org/simple").rstrip("/"), netrc = defaults.get("netrc", None), + python_version = defaults.get("python_version", None), platforms = { name: _plat(**values) for name, values in defaults["platforms"].items() @@ -359,6 +367,15 @@ You cannot use both the additive_build_content and additive_build_content_file a for mod in module_ctx.modules: for pip_attr in mod.tags.parse: + python_version = pip_attr.python_version + if not python_version and pip_attr.pyproject_toml: + pyproject = read_pyproject(module_ctx, pip_attr.pyproject_toml) + if pyproject.requires_python: + python_version = version_from_requires_python(pyproject.requires_python) + python_version = python_version or config.python_version + if not python_version: + _fail("pip.parse() requires one of `python_version`, `pyproject_toml`, or `pip.default(pyproject_toml=...)` to be set") + hub_name = pip_attr.hub_name if hub_name == "pypi": if is_pypi_hub_reserved: @@ -422,6 +439,7 @@ You cannot use both the additive_build_content and additive_build_content_file a builder.pip_parse( module_ctx, pip_attr = pip_attr, + python_version = python_version, ) # dict[str package, dict[str, None] extra_targets] @@ -718,6 +736,23 @@ If you are defining custom platforms in your project and don't want things to cl [isolation] feature. [isolation]: https://bazel.build/rules/lib/globals/module#use_extension.isolate +""", + ), + "pyproject_toml": attr.label( + mandatory = False, + doc = """\ +Label pointing to pyproject.toml file to read the default Python version from. +When specified, reads the `requires-python` field from pyproject.toml and uses +it as the default python_version for all `pip.parse()` calls that don't +explicitly specify one. + +:::{note} +The version must be specified as `==X.Y.Z` (exact version with full semver). +This is designed to work with dependency management tools like Renovate. +::: + +:::{versionadded} VERSION_NEXT_FEATURE +::: """, ), "whl_abi_tags": attr.string_list( @@ -888,8 +923,23 @@ find in case extra indexes are specified. """, default = True, ), + "pyproject_toml": attr.label( + mandatory = False, + doc = """\ +Label pointing to a pyproject.toml file to read the Python version from. +When specified, the `requires-python` field is used as the `python_version` +for this `pip.parse()` call, unless `python_version` is set explicitly. + +:::{note} +The version must be specified as `==X.Y.Z` (exact version with full semver). +::: + +:::{versionadded} VERSION_NEXT_FEATURE +::: +""", + ), "python_version": attr.string( - mandatory = True, + mandatory = False, doc = """ The Python version the dependencies are targetting, in Major.Minor format (e.g., "3.11") or patch level granularity (e.g. "3.11.1"). @@ -897,6 +947,15 @@ The Python version the dependencies are targetting, in Major.Minor format If an interpreter isn't explicitly provided (using `python_interpreter` or `python_interpreter_target`), then the version specified here must have a corresponding `python.toolchain()` configured. + +:::{seealso} +The {obj}`pyproject_toml` attribute for getting the version from a project file. +::: + +:::{versionchanged} VERSION_NEXT_FEATURE +No longer mandatory if the {obj}`pyproject_toml` attribute or +{obj}`pip.default.pyproject_toml` is specified. +::: """, ), "simpleapi_skip": attr.string_list( diff --git a/python/private/pypi/hub_builder.bzl b/python/private/pypi/hub_builder.bzl index 7717f36731..01dc8494d6 100644 --- a/python/private/pypi/hub_builder.bzl +++ b/python/private/pypi/hub_builder.bzl @@ -144,8 +144,8 @@ def _build(self): whl_libraries = self._whl_libraries, ) -def _pip_parse(self, module_ctx, pip_attr): - python_version = pip_attr.python_version +def _pip_parse(self, module_ctx, pip_attr, python_version = None): + python_version = python_version or pip_attr.python_version if python_version in self._platforms: fail(( "Duplicate pip python version '{version}' for hub " + @@ -191,7 +191,8 @@ def _pip_parse(self, module_ctx, pip_attr): self, module_ctx, pip_attr = pip_attr, - enable_pipstar_extract = bool(self._config.enable_pipstar_extract or self._get_index_urls.get(pip_attr.python_version)), + python_version = python_version, + enable_pipstar_extract = bool(self._config.enable_pipstar_extract or self._get_index_urls.get(python_version)), ) ### end of PUBLIC methods @@ -393,11 +394,11 @@ def _set_get_index_urls(self, mctx, pip_attr): ) return True -def _detect_interpreter(self, pip_attr): +def _detect_interpreter(self, pip_attr, python_version): python_interpreter_target = pip_attr.python_interpreter_target if python_interpreter_target == None and not pip_attr.python_interpreter: python_name = "python_{}_host".format( - pip_attr.python_version.replace(".", "_"), + python_version.replace(".", "_"), ) if python_name not in self._available_interpreters: fail(( @@ -407,7 +408,7 @@ def _detect_interpreter(self, pip_attr): "Expected to find {python_name} among registered versions:\n {labels}" ).format( hub_name = self.name, - version = pip_attr.python_version, + version = python_version, python_name = python_name, labels = " \n".join(self._available_interpreters), )) @@ -476,6 +477,7 @@ def _create_whl_repos( module_ctx, *, pip_attr, + python_version, enable_pipstar_extract = False): """create all of the whl repositories @@ -483,10 +485,11 @@ def _create_whl_repos( self: the builder. module_ctx: {type}`module_ctx`. pip_attr: {type}`struct` - the struct that comes from the tag class iteration. + python_version: {type}`str` - the resolved python version for this pip.parse call. enable_pipstar_extract: {type}`bool` - enable the pipstar extraction or not. """ logger = self._logger - platforms = self._platforms[pip_attr.python_version] + platforms = self._platforms[python_version] requirements_by_platform = parse_requirements( module_ctx, requirements_by_platform = requirements_files_by_platform( @@ -498,7 +501,7 @@ def _create_whl_repos( extra_pip_args = pip_attr.extra_pip_args, platforms = sorted(platforms), # here we only need keys python_version = full_version( - version = pip_attr.python_version, + version = python_version, minor_mapping = self._minor_mapping, ), logger = logger, @@ -528,7 +531,7 @@ def _create_whl_repos( pip_attr = pip_attr, ) - interpreter = _detect_interpreter(self, pip_attr) + interpreter = _detect_interpreter(self, pip_attr, python_version) for whl in requirements_by_platform: whl_library_args = common_args | _whl_library_args( @@ -543,16 +546,16 @@ def _create_whl_repos( whl_library_args = whl_library_args, download_only = pip_attr.download_only, netrc = self._config.netrc or pip_attr.netrc, - use_downloader = src.url and _use_downloader(self, pip_attr.python_version, whl.name), + use_downloader = src.url and _use_downloader(self, python_version, whl.name), auth_patterns = self._config.auth_patterns or pip_attr.auth_patterns, - python_version = _major_minor_version(pip_attr.python_version), + python_version = _major_minor_version(python_version), is_multiple_versions = whl.is_multiple_versions, interpreter = interpreter, enable_pipstar_extract = enable_pipstar_extract, ) _add_whl_library( self, - python_version = pip_attr.python_version, + python_version = python_version, whl = whl, repo = repo, ) diff --git a/python/private/pyproject_utils.bzl b/python/private/pyproject_utils.bzl new file mode 100644 index 0000000000..e96ffd2023 --- /dev/null +++ b/python/private/pyproject_utils.bzl @@ -0,0 +1,51 @@ +"""Utilities for reading values from pyproject.toml.""" + +load("@toml.bzl", "toml") +load(":version.bzl", "version") + +def read_pyproject(module_ctx, pyproject): + """Read a pyproject.toml file and return the relevant fields. + + The file is parsed with a pure-Starlark TOML decoder; no Python + interpreter is required. The raw `requires-python` value is returned + as-is so that callers can decide how to interpret it. + + Args: + module_ctx: the module extension context (needs `path` and `read`). + pyproject: {type}`Label` pointing at the pyproject.toml file. + + Returns: + {type}`struct` with the attributes: + * `requires_python`: {type}`str | None` the raw `requires-python` + value (e.g. `"==3.13.9"`), or `None` if it is not set. + """ + data = toml.decode(module_ctx.read(module_ctx.path(pyproject), watch = "yes")) + return struct( + requires_python = data.get("project", {}).get("requires-python"), + ) + +def version_from_requires_python(requires_python): + """Derive a concrete Python version from a `requires-python` value. + + Currently only an exact `==X.Y.Z` specifier is supported. The value is + validated and normalized via {obj}`//python/private:version.bzl` so that + malformed input fails in a consistent way. Broader specifier support + (e.g. `>=`, `X.Y`) can be layered on here in the future. + + Args: + requires_python: {type}`str` the raw `requires-python` value. + + Returns: + {type}`str` the normalized version string (e.g. `"3.13.9"`). + """ + if not requires_python: + fail("`requires-python` must be specified") + + if not requires_python.startswith("=="): + fail("`requires-python` must pin an exact version with `==`, got: {}".format(requires_python)) + + bare_version = requires_python[len("=="):].strip() + + # Parse strictly so malformed versions fail cleanly, then normalize. + version.parse(bare_version, strict = True) + return version.normalize(bare_version) diff --git a/python/private/python.bzl b/python/private/python.bzl index 23bae5d341..f098a626fb 100644 --- a/python/private/python.bzl +++ b/python/private/python.bzl @@ -20,6 +20,7 @@ load(":auth.bzl", "AUTH_ATTRS") load(":full_version.bzl", "full_version") load(":pbs_manifest.bzl", "parse_runtime_manifest") load(":platform_info.bzl", "platform_info") +load(":pyproject_utils.bzl", "read_pyproject", "version_from_requires_python") load(":python_register_toolchains.bzl", "python_register_toolchains") load(":pythons_hub.bzl", "hub_repo") load(":repo_utils.bzl", "repo_utils") @@ -88,6 +89,7 @@ def parse_modules(*, module_ctx, logger = None, _fail = fail): mod = mod, seen_versions = seen_versions, config = config, + default_python_version = default_python_version, ) for toolchain_attr in toolchain_attr_structs: @@ -971,8 +973,15 @@ def _compute_default_python_version(mctx): defaults_attr_structs = _create_defaults_attr_structs(mod = mod) default_python_version_env = None default_python_version_file = None + pyproject_toml_label = None for defaults_attr in defaults_attr_structs: + pyproject_toml_label = _one_or_the_same( + pyproject_toml_label, + defaults_attr.pyproject_toml, + onerror = lambda: fail("Multiple pyproject.toml files specified in defaults"), + ) + default_python_version = _one_or_the_same( default_python_version, defaults_attr.python_version, @@ -988,11 +997,17 @@ def _compute_default_python_version(mctx): defaults_attr.python_version_file, onerror = _fail_multiple_defaults_python_version_file, ) + + # Priority order: ENV > pyproject_toml > python_version_file > python_version if default_python_version_file: default_python_version = _one_or_the_same( default_python_version, mctx.read(default_python_version_file, watch = "yes").strip(), ) + if pyproject_toml_label: + pyproject = read_pyproject(mctx, pyproject_toml_label) + if pyproject.requires_python: + default_python_version = version_from_requires_python(pyproject.requires_python) if default_python_version_env: default_python_version = mctx.getenv( default_python_version_env, @@ -1034,11 +1049,29 @@ def _create_defaults_attr_struct(*, tag): python_version = getattr(tag, "python_version", None), python_version_env = getattr(tag, "python_version_env", None), python_version_file = getattr(tag, "python_version_file", None), + pyproject_toml = getattr(tag, "pyproject_toml", None), ) -def _create_toolchain_attr_structs(*, mod, config, seen_versions): +def _create_toolchain_attr_structs(*, mod, config, seen_versions, default_python_version): arg_structs = [] + # Auto-register a toolchain for the default version if not already + # registered via an explicit python.toolchain() call. + # This works for any default source: pyproject_toml, python_version_file, + # python_version_env, or python_version. + has_explicit_toolchain = default_python_version and any([ + tag.python_version == default_python_version + for tag in mod.tags.toolchain + ]) + if (default_python_version and + default_python_version not in seen_versions and + mod.is_root and not has_explicit_toolchain): + arg_structs.append(_create_toolchain_attrs_struct( + python_version = default_python_version, + toolchain_tag_count = 1, + )) + seen_versions[default_python_version] = True + for tag in mod.tags.toolchain: arg_structs.append(_create_toolchain_attrs_struct( tag = tag, @@ -1078,6 +1111,17 @@ def _create_toolchain_attrs_struct( _defaults = tag_class( doc = """Tag class to specify the default Python version.""", attrs = { + "pyproject_toml": attr.label( + mandatory = False, + doc = """\ +Label pointing to pyproject.toml file to read the default Python version from. +When specified, reads the `requires-python` field from pyproject.toml. +The version must be specified as `==X.Y.Z` (exact version with full semver). + +:::{versionadded} VERSION_NEXT_FEATURE +::: +""", + ), "python_version": attr.string( mandatory = False, doc = """\ diff --git a/tests/pypi/extension/extension_tests.bzl b/tests/pypi/extension/extension_tests.bzl index c2e0c8b60f..97849e882d 100644 --- a/tests/pypi/extension/extension_tests.bzl +++ b/tests/pypi/extension/extension_tests.bzl @@ -24,7 +24,7 @@ load(":pip_parse.bzl", _parse = "pip_parse") _tests = [] -def _pypi_mock_mctx(*modules, os_name = "unittest", arch_name = "exotic", environ = {}, read = None): +def _pypi_mock_mctx(*modules, os_name = "unittest", arch_name = "exotic", environ = {}, read = None, mock_files = {}): _ = read # @unused return mocks.mctx( modules = list(modules), @@ -36,7 +36,7 @@ def _pypi_mock_mctx(*modules, os_name = "unittest", arch_name = "exotic", enviro simple==0.0.1 \ --hash=sha256:deadbeef \ --hash=sha256:deadbaaf""", - }, + } | mock_files, ) def _default( @@ -51,6 +51,7 @@ def _default( netrc = None, os_name = None, platform = None, + pyproject_toml = None, whl_platform_tags = None, whl_abi_tags = None): return struct( @@ -64,6 +65,7 @@ def _default( netrc = netrc, os_name = os_name, platform = platform, + pyproject_toml = pyproject_toml, whl_abi_tags = whl_abi_tags or [], whl_platform_tags = whl_platform_tags or [], ) @@ -183,6 +185,58 @@ def _test_simple(env): _tests.append(_test_simple) +def _test_pip_parse_pyproject_toml(env): + # pip.parse() reads the version from pyproject.toml's requires-python when + # python_version is not set explicitly. + pypi = _parse_modules( + env, + module_ctx = _pypi_mock_mctx( + _mod( + name = "rules_python", + parse = [ + _parse( + hub_name = "pypi", + pyproject_toml = "pyproject.toml", + simpleapi_skip = ["simple"], + requirements_lock = "requirements.txt", + ), + ], + ), + os_name = "linux", + arch_name = "x86_64", + mock_files = { + "pyproject.toml": "[project]\nrequires-python = \"==3.15.19\"\n", + }, + ), + available_interpreters = { + "python_3_15_19_host": "unit_test_interpreter_target", + }, + minor_mapping = {"3.15": "3.15.19"}, + ) + + # Resolves identically to passing python_version = "3.15.19" explicitly: + # the full version drives interpreter selection, hub naming uses major.minor. + pypi.exposed_packages().contains_exactly({"pypi": ["simple"]}) + pypi.hub_whl_map().contains_exactly({"pypi": { + "simple": { + "pypi_315_simple": [ + whl_config_setting( + version = "3.15", + ), + ], + }, + }}) + pypi.whl_libraries().contains_exactly({ + "pypi_315_simple": { + "config_load": "@pypi//:config.bzl", + "dep_template": "@pypi//{name}:{target}", + "python_interpreter_target": "unit_test_interpreter_target", + "requirement": "simple==0.0.1 --hash=sha256:deadbeef --hash=sha256:deadbaaf", + }, + }) + +_tests.append(_test_pip_parse_pyproject_toml) + def _test_simple_isolated(env): """Simulate `isolate = True` with parse_modules. diff --git a/tests/pypi/extension/pip_parse.bzl b/tests/pypi/extension/pip_parse.bzl index 939639f2c5..7b5bdfdfd6 100644 --- a/tests/pypi/extension/pip_parse.bzl +++ b/tests/pypi/extension/pip_parse.bzl @@ -3,7 +3,7 @@ def pip_parse( *, hub_name, - python_version, + python_version = None, add_libdir_to_library_search_path = False, auth_patterns = {}, download_only = False, @@ -19,6 +19,7 @@ def pip_parse( netrc = None, parse_all_requirements_files = True, pip_data_exclude = None, + pyproject_toml = None, python_interpreter = None, python_interpreter_target = None, quiet = True, @@ -52,6 +53,7 @@ def pip_parse( netrc = netrc, parse_all_requirements_files = parse_all_requirements_files, pip_data_exclude = pip_data_exclude, + pyproject_toml = pyproject_toml, python_interpreter = python_interpreter, python_interpreter_target = python_interpreter_target, python_version = python_version, From 8396c281295e1a051a6e0ea2f57934d93515e5bc Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Thu, 2 Jul 2026 20:00:06 -0700 Subject: [PATCH 809/922] Fix release tool links and improve workflow robustness (#3888) Fixes release tool bugs and improves workflow robustness. - Fix BCR entry link in create-rc to use RC version instead of final version. - Format BCR entry link as 'BCR Entry '. - Fix release workflow status link to use release_create_rc.yaml. - Use GITHUB_RUN_ID to link directly to the workflow run if available. - Link the branch name in the create-rc comment. - Add a always-running job to release_create_release_branch.yaml to suppress 'no jobs ran' errors. - Remove obsolete rocket assertions in create_rc tests. --- .../release_create_release_branch.yaml | 9 ++++ tests/tools/private/release/create_rc_test.py | 52 ++++++++++++++++--- tools/private/release/create_rc.py | 16 ++++-- 3 files changed, 64 insertions(+), 13 deletions(-) diff --git a/.github/workflows/release_create_release_branch.yaml b/.github/workflows/release_create_release_branch.yaml index ea030a7085..4c021814d0 100644 --- a/.github/workflows/release_create_release_branch.yaml +++ b/.github/workflows/release_create_release_branch.yaml @@ -35,3 +35,12 @@ jobs: create-release-branch --issue ${{ github.event.issue.number }} --remote origin env: GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} + + # A no-op job that always runs to prevent "no jobs ran" failures + # when the main job is skipped. + suppress-no-jobs-ran-error: + runs-on: ubuntu-latest + steps: + - name: Echo Success + run: echo "Success" + diff --git a/tests/tools/private/release/create_rc_test.py b/tests/tools/private/release/create_rc_test.py index 2b4e40cc4e..193f11561d 100644 --- a/tests/tools/private/release/create_rc_test.py +++ b/tests/tools/private/release/create_rc_test.py @@ -56,23 +56,56 @@ def test_create_rc_success_first_rc(self): "**New Release Candidate Tagged!** 🐍🌿", comment_call_args[1], ) + self.assertIn( + "tagged on branch [`release/2.0`](https://github.com/bazel-contrib/rules_python/tree/release/2.0)", + comment_call_args[1], + ) self.assertIn( "- [Github Release 2.0.0-rc0](https://github.com/bazel-contrib/rules_python/releases/tag/2.0.0-rc0)", comment_call_args[1], ) self.assertIn( - "- BCR Entry: [rules_python@2.0.0](https://registry.bazel.build/modules/rules_python/2.0.0)", + "- [BCR Entry 2.0.0-rc0](https://registry.bazel.build/modules/rules_python/2.0.0-rc0)", comment_call_args[1], ) self.assertIn( - "- [BCR PRs](https://github.com/bazelbuild/bazel-central-registry/pulls?q=is%3Apr+rules_python+2.0.0)", + "- [BCR PRs](https://github.com/bazelbuild/bazel-central-registry/pulls?q=is%3Apr+rules_python+2.0.0-rc0)", comment_call_args[1], ) self.assertIn( - "- [Release workflow status](https://github.com/bazel-contrib/rules_python/actions/workflows/release_publish.yaml)", + "- [Release workflow status](https://github.com/bazel-contrib/rules_python/actions/workflows/release_create_rc.yaml)", + comment_call_args[1], + ) + + def test_create_rc_success_with_run_id(self): + # Arrange + args = MagicMock(issue=123, remote="my-remote") + self.mock_gh.get_issue_title.return_value = "Release 2.0.0" + self.mock_gh.get_issue_body.return_value = """ +## Checklist +- [x] Prepare Release | status=done pr=#122 commit=abcdef12 +- [x] Create Release branch | status=done branch=release/2.0 commit=abcdef12 +- [ ] Tag RC0 | status=pending +""" + self.mock_git.get_remote_tags.return_value = [] + self.mock_git.get_commit_sha.return_value = "1234567890" + + # Act + with patch.dict(os.environ, {"GITHUB_RUN_ID": "987654321"}): + result = CreateRc(args, self.mock_git, self.mock_gh).run() + + # Assert + self.assertEqual(result, 0) + self.mock_gh.post_issue_comment.assert_called_once() + comment_call_args = self.mock_gh.post_issue_comment.call_args[0] + self.assertIn( + "- [Release workflow status](https://github.com/bazel-contrib/rules_python/actions/runs/987654321)", + comment_call_args[1], + ) + self.assertIn( + "tagged on branch [`release/2.0`](https://github.com/bazel-contrib/rules_python/tree/release/2.0)", comment_call_args[1], ) - self.assertNotIn("🚀", comment_call_args[1]) def test_create_rc_success_next_rc(self): # Arrange @@ -113,23 +146,26 @@ def test_create_rc_success_next_rc(self): "**New Release Candidate Tagged!** 🐍🌿", comment_call_args[1], ) + self.assertIn( + "tagged on branch [`release/2.0`](https://github.com/bazel-contrib/rules_python/tree/release/2.0)", + comment_call_args[1], + ) self.assertIn( "- [Github Release 2.0.0-rc1](https://github.com/bazel-contrib/rules_python/releases/tag/2.0.0-rc1)", comment_call_args[1], ) self.assertIn( - "- BCR Entry: [rules_python@2.0.0](https://registry.bazel.build/modules/rules_python/2.0.0)", + "- [BCR Entry 2.0.0-rc1](https://registry.bazel.build/modules/rules_python/2.0.0-rc1)", comment_call_args[1], ) self.assertIn( - "- [BCR PRs](https://github.com/bazelbuild/bazel-central-registry/pulls?q=is%3Apr+rules_python+2.0.0)", + "- [BCR PRs](https://github.com/bazelbuild/bazel-central-registry/pulls?q=is%3Apr+rules_python+2.0.0-rc1)", comment_call_args[1], ) self.assertIn( - "- [Release workflow status](https://github.com/bazel-contrib/rules_python/actions/workflows/release_publish.yaml)", + "- [Release workflow status](https://github.com/bazel-contrib/rules_python/actions/workflows/release_create_rc.yaml)", comment_call_args[1], ) - self.assertNotIn("🚀", comment_call_args[1]) def test_create_rc_gating_on_backports(self): # Arrange diff --git a/tools/private/release/create_rc.py b/tools/private/release/create_rc.py index 4e8564a9b8..260cbb8940 100644 --- a/tools/private/release/create_rc.py +++ b/tools/private/release/create_rc.py @@ -114,15 +114,21 @@ def run(self) -> int: self.gh.update_issue_body(args.issue, updated_body) tag_url = f"{REPO_URL}/releases/tag/{next_rc}" - bcr_entry_url = f"https://registry.bazel.build/modules/rules_python/{version}" - bcr_search_url = f"https://github.com/bazelbuild/bazel-central-registry/pulls?q=is%3Apr+rules_python+{version}" - release_workflow_url = f"{REPO_URL}/actions/workflows/release_publish.yaml" + bcr_entry_url = f"https://registry.bazel.build/modules/rules_python/{next_rc}" + bcr_search_url = f"https://github.com/bazelbuild/bazel-central-registry/pulls?q=is%3Apr+rules_python+{next_rc}" + if run_id := os.environ.get("GITHUB_RUN_ID"): + release_workflow_url = f"{REPO_URL}/actions/runs/{run_id}" + else: + release_workflow_url = ( + f"{REPO_URL}/actions/workflows/release_create_rc.yaml" + ) + branch_url = f"{REPO_URL}/tree/{branch_name}" comment_body = f"""**New Release Candidate Tagged!** 🐍🌿 -Release Candidate **{next_rc}** has been successfully generated and tagged on branch `{branch_name}`. +Release Candidate **{next_rc}** has been successfully generated and tagged on branch [`{branch_name}`]({branch_url}). - [Github Release {next_rc}]({tag_url}) -- BCR Entry: [rules_python@{version}]({bcr_entry_url}) +- [BCR Entry {next_rc}]({bcr_entry_url}) - [BCR PRs]({bcr_search_url}) - [Release workflow status]({release_workflow_url})""" self.gh.post_issue_comment(args.issue, comment_body) From f3fa8dab54e676560c1a886abafc8114a015f67f Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Thu, 2 Jul 2026 20:41:45 -0700 Subject: [PATCH 810/922] build(release): support triggering release workflows via comments (#3889) Allows maintainers to trigger release workflows (prepare, create-rc, process-backports) by commenting on the release tracking issue. This automates the release process further and reduces the need to manually trigger workflows from the GitHub Actions UI. - Created a central `on_issue_comment.yaml` workflow to parse comments. - Updated `release_prepare.yaml`, `release_create_rc.yaml`, and `release_process_backports.yaml` to be reusable workflows. - Updated the release tracking issue template to document the new comment commands. --- .../release_tracking_template.md | 24 +++++-- .github/workflows/on_issue_comment.yaml | 68 +++++++++++++++++++ .github/workflows/release_create_rc.yaml | 6 ++ .github/workflows/release_prepare.yaml | 16 ++++- .../workflows/release_process_backports.yaml | 6 ++ 5 files changed, 112 insertions(+), 8 deletions(-) create mode 100644 .github/workflows/on_issue_comment.yaml diff --git a/.github/ISSUE_TEMPLATE/release_tracking_template.md b/.github/ISSUE_TEMPLATE/release_tracking_template.md index f573fdfacf..6bae3f3c2c 100644 --- a/.github/ISSUE_TEMPLATE/release_tracking_template.md +++ b/.github/ISSUE_TEMPLATE/release_tracking_template.md @@ -18,7 +18,7 @@ labels: ['type: release'] To request a backport: 1. Add a new checklist item under the `## Backports` section. 2. The format must be: `- [ ] #` (e.g., `- [ ] #1234`). -3. Trigger the [Process Backports Workflow](https://github.com/bazel-contrib/rules_python/actions/workflows/process_backports.yml). +3. Trigger the [Process Backports Workflow][process_backports].
--- @@ -36,8 +36,22 @@ The checklist items use metadata suffix: `| key=value key2=value2`.
Available Commands -Maintainers can trigger automation by running manual workflows: -- [Process Backports Workflow](https://github.com/bazel-contrib/rules_python/actions/workflows/process_backports.yml) -- [Generate RC Tag Workflow](https://github.com/bazel-contrib/rules_python/actions/workflows/generate_rc.yml) -- [Promote RC to Final Release Workflow](https://github.com/bazel-contrib/rules_python/actions/workflows/promote_rc.yml) +Maintainers can trigger automation by: +- Running manual workflows: + - [Process Backports Workflow][process_backports] + - [Create RC Workflow][create_rc] + - [Promote RC to Final Release Workflow][promote_rc] +- Commenting on this issue (requires the issue to have the `type: release` + label): + - `/prepare` at the beginning of a line to trigger the Release Prepare + workflow. + - `/create-rc` at the beginning of a line to trigger the Create RC + workflow. + - `/process-backports` at the beginning of a line to trigger the Process + Backports workflow. +
+ +[process_backports]: https://github.com/bazel-contrib/rules_python/actions/workflows/release_process_backports.yaml +[create_rc]: https://github.com/bazel-contrib/rules_python/actions/workflows/release_create_rc.yaml +[promote_rc]: https://github.com/bazel-contrib/rules_python/actions/workflows/release_promote_rc.yaml diff --git a/.github/workflows/on_issue_comment.yaml b/.github/workflows/on_issue_comment.yaml new file mode 100644 index 0000000000..d37ae1516c --- /dev/null +++ b/.github/workflows/on_issue_comment.yaml @@ -0,0 +1,68 @@ +name: "On Issue Comment" + +on: + issue_comment: + types: [created] + +permissions: + contents: read + issues: read + +jobs: + # This job always runs to prevent GHA from marking the run as failed when + # all other jobs are skipped. + noop: + runs-on: ubuntu-latest + steps: + - run: echo "No-op" + + parse_comment: + runs-on: ubuntu-latest + if: | + github.event.issue.pull_request == null && + contains(github.event.issue.labels.*.name, 'type: release') && + (github.event.comment.author_association == 'OWNER' || + github.event.comment.author_association == 'MEMBER' || + github.event.comment.author_association == 'COLLABORATOR') + outputs: + command: ${{ steps.parse.outputs.command }} + issue_number: ${{ github.event.issue.number }} + steps: + - name: Parse comment + id: parse + env: + COMMENT_BODY: ${{ github.event.comment.body }} + run: | + if echo "$COMMENT_BODY" | grep -qE '^[[:space:]]*/create-rc([[:space:]]|$)'; then + echo "command=create-rc" >> "$GITHUB_OUTPUT" + elif echo "$COMMENT_BODY" | grep -qE '^[[:space:]]*/prepare([[:space:]]|$)'; then + echo "command=prepare" >> "$GITHUB_OUTPUT" + elif echo "$COMMENT_BODY" | grep -qE '^[[:space:]]*/process-backports([[:space:]]|$)'; then + echo "command=process-backports" >> "$GITHUB_OUTPUT" + else + echo "command=none" >> "$GITHUB_OUTPUT" + fi + + call_create_rc: + needs: parse_comment + if: needs.parse_comment.outputs.command == 'create-rc' + uses: ./.github/workflows/release_create_rc.yaml + with: + issue: ${{ needs.parse_comment.outputs.issue_number }} + secrets: inherit + + call_prepare: + needs: parse_comment + if: needs.parse_comment.outputs.command == 'prepare' + uses: ./.github/workflows/release_prepare.yaml + with: + issue: ${{ needs.parse_comment.outputs.issue_number }} + secrets: inherit + + call_process_backports: + needs: parse_comment + if: needs.parse_comment.outputs.command == 'process-backports' + uses: ./.github/workflows/release_process_backports.yaml + with: + issue: ${{ needs.parse_comment.outputs.issue_number }} + secrets: inherit diff --git a/.github/workflows/release_create_rc.yaml b/.github/workflows/release_create_rc.yaml index 9a58e20c09..eaf6250b85 100644 --- a/.github/workflows/release_create_rc.yaml +++ b/.github/workflows/release_create_rc.yaml @@ -7,6 +7,12 @@ on: description: 'The Release Tracking Issue Number (e.g., 142)' required: true type: string + workflow_call: + inputs: + issue: + description: 'The Release Tracking Issue Number (e.g., 142)' + required: true + type: string permissions: contents: write diff --git a/.github/workflows/release_prepare.yaml b/.github/workflows/release_prepare.yaml index 5c9a0b8f49..8ca38cb1c2 100644 --- a/.github/workflows/release_prepare.yaml +++ b/.github/workflows/release_prepare.yaml @@ -2,7 +2,17 @@ name: "Release: Prepare" on: workflow_dispatch: - # Allow manual triggering to prepare the release immediately + inputs: + issue: + description: 'The Release Tracking Issue Number (e.g., 142)' + required: false + type: string + workflow_call: + inputs: + issue: + description: 'The Release Tracking Issue Number (e.g., 142)' + required: false + type: string permissions: contents: write @@ -30,7 +40,7 @@ jobs: - name: Run Release Preparation Pipeline run: | - # Manual trigger: run full preparation - bazel run //tools/private/release -- prepare --no-dry-run + bazel run //tools/private/release -- \ + prepare ${{ inputs.issue && format('--issue={0}', inputs.issue) || '' }} --no-dry-run env: GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} diff --git a/.github/workflows/release_process_backports.yaml b/.github/workflows/release_process_backports.yaml index 822a337323..0812aa5248 100644 --- a/.github/workflows/release_process_backports.yaml +++ b/.github/workflows/release_process_backports.yaml @@ -7,6 +7,12 @@ on: description: 'The Release Tracking Issue Number (e.g., 142)' required: true type: string + workflow_call: + inputs: + issue: + description: 'The Release Tracking Issue Number (e.g., 142)' + required: true + type: string permissions: contents: write From b2d1bf0b26d3ff736019b48034b0bcee55ce9660 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Fri, 3 Jul 2026 00:57:00 -0700 Subject: [PATCH 811/922] chore(release): allow comments to trigger actions, update releasing docs (#3891) Updates the release documentation to reflect the new process using the Release Tracking Issue, comments (/prepare, /create-rc, /process-backports), and automated workflows. --- .../release_tracking_template.md | 43 ++----- .github/workflows/on_issue_comment.yaml | 37 +++++- .../workflows/release_process_backports.yaml | 31 ++++- .github/workflows/release_promote_rc.yaml | 23 +++- RELEASING.md | 113 ++++++++++-------- .../private/release/process_backports_test.py | 27 +++-- .../tools/private/release/promote_rc_test.py | 89 ++++++++++---- tools/private/release/gh.py | 33 +++++ tools/private/release/process_backports.py | 53 +++++++- tools/private/release/promote_rc.py | 72 ++++++++++- tools/private/release/release_issue.py | 49 +++++++- tools/private/release/utils.py | 22 ++++ 12 files changed, 468 insertions(+), 124 deletions(-) diff --git a/.github/ISSUE_TEMPLATE/release_tracking_template.md b/.github/ISSUE_TEMPLATE/release_tracking_template.md index 6bae3f3c2c..57a51a16ef 100644 --- a/.github/ISSUE_TEMPLATE/release_tracking_template.md +++ b/.github/ISSUE_TEMPLATE/release_tracking_template.md @@ -12,46 +12,21 @@ labels: ['type: release'] ## Backports -
-How to add backports - -To request a backport: -1. Add a new checklist item under the `## Backports` section. -2. The format must be: `- [ ] #` (e.g., `- [ ] #1234`). -3. Trigger the [Process Backports Workflow][process_backports]. -
+To request a backport, add it to the checklist below and process it. See [RELEASING.md: How to add backports](https://github.com/bazel-contrib/rules_python/blob/main/RELEASING.md#how-to-add-backports) for details. --- -*Maintainers: Automation will react to changes on this issue.* -
-Manual Editing - -You can manually edit this issue to control the release flow. -The checklist items use metadata suffix: `| key=value key2=value2`. -- **Retry Prepare Release**: Reset to `- [ ] Prepare Release | status=awaiting-preparation`. -- **Force Task Done**: Check the box `- [x]` and add appropriate metadata (e.g. `status=done`). -
+To manually control the release flow, see the [RELEASING.md: Manual Editing](https://github.com/bazel-contrib/rules_python/blob/main/RELEASING.md#manual-editing-of-tracking-issue) section.
Available Commands -Maintainers can trigger automation by: -- Running manual workflows: - - [Process Backports Workflow][process_backports] - - [Create RC Workflow][create_rc] - - [Promote RC to Final Release Workflow][promote_rc] -- Commenting on this issue (requires the issue to have the `type: release` - label): - - `/prepare` at the beginning of a line to trigger the Release Prepare - workflow. - - `/create-rc` at the beginning of a line to trigger the Create RC - workflow. - - `/process-backports` at the beginning of a line to trigger the Process - Backports workflow. +Comment commands: +- `/prepare`: Determines version, creates tracking issue and preparation PR. +- `/create-rc`: Tags and publishes a new release candidate (RC). +- `/process-backports`: Cherry-picks pending backports. +- `/add-backports `: Adds PRs to the backports and processes backports. +- `/promote`: Promotes the latest RC to final release. +See [RELEASING.md](https://github.com/bazel-contrib/rules_python/blob/main/RELEASING.md) for details on how to use them.
- -[process_backports]: https://github.com/bazel-contrib/rules_python/actions/workflows/release_process_backports.yaml -[create_rc]: https://github.com/bazel-contrib/rules_python/actions/workflows/release_create_rc.yaml -[promote_rc]: https://github.com/bazel-contrib/rules_python/actions/workflows/release_promote_rc.yaml diff --git a/.github/workflows/on_issue_comment.yaml b/.github/workflows/on_issue_comment.yaml index d37ae1516c..4be5906596 100644 --- a/.github/workflows/on_issue_comment.yaml +++ b/.github/workflows/on_issue_comment.yaml @@ -27,11 +27,13 @@ jobs: outputs: command: ${{ steps.parse.outputs.command }} issue_number: ${{ github.event.issue.number }} + backports: ${{ steps.parse.outputs.backports }} steps: - name: Parse comment id: parse env: COMMENT_BODY: ${{ github.event.comment.body }} + GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} run: | if echo "$COMMENT_BODY" | grep -qE '^[[:space:]]*/create-rc([[:space:]]|$)'; then echo "command=create-rc" >> "$GITHUB_OUTPUT" @@ -39,6 +41,27 @@ jobs: echo "command=prepare" >> "$GITHUB_OUTPUT" elif echo "$COMMENT_BODY" | grep -qE '^[[:space:]]*/process-backports([[:space:]]|$)'; then echo "command=process-backports" >> "$GITHUB_OUTPUT" + elif echo "$COMMENT_BODY" | grep -qE '^[[:space:]]*/add-backports([[:space:]]|$)'; then + args=$(echo "$COMMENT_BODY" | grep -E '^[[:space:]]*/add-backports([[:space:]]|$)' | sed -E 's/^[[:space:]]*\/add-backports[[:space:]]*//') + # Strip leading/trailing spaces and commas + args=$(echo "$args" | sed -e 's/^[[:space:],]*//' -e 's/[[:space:],]*$//') + # Replace internal spaces/commas with single comma + csv=$(echo "$args" | sed -E 's/[[:space:],]+/ /g' | tr ' ' ',') + if [ -n "$csv" ]; then + echo "command=add-backports" >> "$GITHUB_OUTPUT" + echo "backports=$csv" >> "$GITHUB_OUTPUT" + else + echo "command=none" >> "$GITHUB_OUTPUT" + echo "Error: No PRs specified for add-backports." >&2 + gh api \ + --method POST \ + -H "Accept: application/vnd.github+json" \ + -H "X-GitHub-Api-Version: 2022-11-28" \ + /repos/${{ github.repository }}/issues/comments/${{ github.event.comment.id }}/reactions \ + -f "content=-1" + fi + elif echo "$COMMENT_BODY" | grep -qE '^[[:space:]]*/promote([[:space:]]|$)'; then + echo "command=promote" >> "$GITHUB_OUTPUT" else echo "command=none" >> "$GITHUB_OUTPUT" fi @@ -61,8 +84,20 @@ jobs: call_process_backports: needs: parse_comment - if: needs.parse_comment.outputs.command == 'process-backports' + if: | + needs.parse_comment.outputs.command == 'process-backports' || + needs.parse_comment.outputs.command == 'add-backports' uses: ./.github/workflows/release_process_backports.yaml + with: + issue: ${{ needs.parse_comment.outputs.issue_number }} + add_backports: ${{ needs.parse_comment.outputs.command == 'add-backports' && needs.parse_comment.outputs.backports || '' }} + comment_id: "${{ github.event.comment.id }}" + secrets: inherit + + call_promote: + needs: parse_comment + if: needs.parse_comment.outputs.command == 'promote' + uses: ./.github/workflows/release_promote_rc.yaml with: issue: ${{ needs.parse_comment.outputs.issue_number }} secrets: inherit diff --git a/.github/workflows/release_process_backports.yaml b/.github/workflows/release_process_backports.yaml index 0812aa5248..394a5b0fe9 100644 --- a/.github/workflows/release_process_backports.yaml +++ b/.github/workflows/release_process_backports.yaml @@ -7,12 +7,28 @@ on: description: 'The Release Tracking Issue Number (e.g., 142)' required: true type: string + add_backports: + description: 'CSV list of PR numbers to add and process (optional)' + required: false + type: string + comment_id: + description: 'The ID of the comment that triggered this run (optional)' + required: false + type: string workflow_call: inputs: issue: description: 'The Release Tracking Issue Number (e.g., 142)' required: true type: string + add_backports: + description: 'CSV list of PR numbers to add and process (optional)' + required: false + type: string + comment_id: + description: 'The ID of the comment that triggered this run (optional)' + required: false + type: string permissions: contents: write @@ -40,7 +56,18 @@ jobs: - name: Process Pending Backports run: | - bazel run //tools/private/release -- \ - process-backports --issue ${{ inputs.issue }} --remote origin --no-dry-run + ARGS=() + if [ -n "${{ inputs.add_backports }}" ]; then + ARGS+=("--add=${{ inputs.add_backports }}") + fi + if [ -n "${{ inputs.comment_id }}" ]; then + ARGS+=("--triggering-comment=${{ inputs.comment_id }}") + fi + + bazel run //tools/private/release -- process-backports \ + --issue ${{ inputs.issue }} \ + --remote origin \ + --no-dry-run \ + "${ARGS[@]}" env: GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} diff --git a/.github/workflows/release_promote_rc.yaml b/.github/workflows/release_promote_rc.yaml index 5a0ad72a80..987583a1fd 100644 --- a/.github/workflows/release_promote_rc.yaml +++ b/.github/workflows/release_promote_rc.yaml @@ -7,6 +7,16 @@ on: description: 'The final version to release (e.g., 0.38.0)' required: true type: string + workflow_call: + inputs: + version: + description: 'The final version to release (e.g., 0.38.0)' + required: false + type: string + issue: + description: 'The tracking issue number' + required: true + type: string permissions: contents: write @@ -33,7 +43,16 @@ jobs: - name: Run Promote RC run: | - bazel run //tools/private/release -- \ - promote-rc ${{ inputs.version }} --remote origin + ARGS=() + if [ -n "${{ inputs.version }}" ]; then + ARGS+=("${{ inputs.version }}") + fi + if [ -n "${{ inputs.issue }}" ]; then + ARGS+=("--issue" "${{ inputs.issue }}") + fi + ARGS+=("--remote" "origin") + ARGS+=("--no-dry-run") + + bazel run //tools/private/release -- promote-rc "${ARGS[@]}" env: GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} diff --git a/RELEASING.md b/RELEASING.md index 13eb4ddc0a..528450f611 100644 --- a/RELEASING.md +++ b/RELEASING.md @@ -8,54 +8,40 @@ existing Bazel workspace to sanity check functionality. ## Releasing from HEAD -These are the steps for a regularly scheduled release from HEAD. +Releases are managed using a semi-automated process centered around a GitHub +Release Tracking Issue and automated workflows triggered by comments or issue edits. + +> [!NOTE] +> Comment-based commands must be posted by project maintainers (Owner, +> Member, or Collaborator) and must be on their own line (leading and trailing +> whitespace is ignored). ### Steps -1. Update the changelog and replace the version placeholders by running the - release tool. The next version number will be automatically determined - based on the presence of `VERSION_NEXT_*` placeholders and git tags. The - tool will read all news entry files in the `news/` directory, assemble - them into the changelog, and delete the processed news files. - - ```shell - bazel run //tools/private/release - ``` - - If you want to append news entries to an already existing release section in - the changelog (for example, to update a drafted release or a release - branch), you can specify the version explicitly: - - ```shell - bazel run //tools/private/release -- X.Y.Z - ``` - -1. Send these changes for review and get them merged. -1. Create a branch for the new release, named `release/X.Y` - ``` - git branch --no-track release/X.Y upstream/main && git push upstream release/X.Y - ``` - -The next step is to create tags to trigger release workflow, **however** -we start by using release candidate tags (`X.Y.Z-rcN`) before tagging the -final release (`X.Y.Z`). - -1. Create release candidate tag and push. The first RC uses `N=0`. Increment - `N` for each RC. - ``` - git tag X.Y.0-rcN upstream/release/X.Y && git push upstream tag X.Y.0-rcN - ``` -2. Announce the RC release: see [Announcing Releases] -3. Wait a week for feedback. - * Follow [Patch release with cherry picks] to pull bug fixes into the - release branch. - * Repeat the RC tagging step, incrementing `N`. -4. Finally, tag the final release tag: - ```shell - git tag X.Y.0 upstream/release/X.Y && git push upstream tag X.Y.0 - ``` - -Release automation will create a GitHub release and BCR pull request. +1. **Prepare the Release**: Run the [Release: Prepare](https://github.com/bazel-contrib/rules_python/actions/workflows/release_prepare.yaml) + workflow manually. You can trigger it from the GitHub Actions UI or using + the GitHub CLI: + ```shell + gh workflow run release_prepare.yaml --repo bazel-contrib/rules_python + ``` + This will automatically determine the next version, create a release tracking + issue, and send a preparation PR. + +2. **Approve and Merge**: Approve and merge the PR. Once merged, a release + branch will be created automatically. + +3. **Add Backports (if needed)**: If there are backports, add them following + the [How to add backports](#how-to-add-backports) steps. + +4. **Create an RC**: Comment `/create-rc` on the tracking issue. All pending + backports must be successfully processed before creating the RC. + +5. **Iterate**: Repeat steps 3 and 4 until backports and RCs are no longer + needed. + +6. **Finalize the Release**: Comment `/promote` on the tracking issue to + finalize the release. + ### Manually triggering the release workflow @@ -87,6 +73,31 @@ the `VERSION_NEXT_*` placeholders in the codebase. To see what changes are being accumulated for the next release, review the pending news entries in the `news/` directory. +## How to add backports + +To add backports to an active release, you can use one of the following +methods: + +### Method A: Manual Checklist Update +1. Manually add checklist items under the `## Backports` section of the + Release Tracking Issue. The format must be: `- [ ] #` (e.g., + `- [ ] #1234`). +2. When ready, comment `/process-backports` on the tracking issue to trigger + processing. + +### Method B: Comment Shortcut +1. Comment `/add-backports [ ...]` (space or comma + separated) on the tracking issue. This will automatically add the PRs to the + checklist and trigger processing. + +### Failure Behavior +If a backport fails to process (e.g., due to cherry-pick conflicts): +* The failed backport checklist item will remain unchecked with + `status=error-`. +* You must resolve the conflict manually: checkout the release branch, + cherry-pick the PR, resolve conflicts, push to remote, and manually check + the box on the tracking issue checklist with `status=done` metadata. + ## Patch release with cherry picks If a patch release from head would contain changes that aren't appropriate for @@ -105,8 +116,8 @@ The fix being included is commit `deadbeef`. If multiple commits need to be applied, repeat the `git cherry-pick` step for each. -Once the release branch is in the desired state, use `git tag` to tag it, as -done with a release from head. Release automation will do the rest. +Once the release branch is in the desired state, comment `/create-rc` on the +tracking issue to tag it, as done with a release from head. ### Announcing releases @@ -139,6 +150,14 @@ The two points of no return are: If release steps fail _prior_ to those steps, then its OK to change the tag. You may need to manually delete the GitHub release. +## Manual Editing of Tracking Issue + +You can manually edit the Release Tracking Issue to control the release flow. +The checklist items use metadata suffix: `| key=value key2=value2`. + +* **Retry Prepare Release**: Reset the task to `- [ ] Prepare Release | status=awaiting-preparation`. +* **Force Task Done**: Check the box `- [x]` and add appropriate metadata (e.g. `status=done`). + ## Secrets ### PyPI user rules-python diff --git a/tests/tools/private/release/process_backports_test.py b/tests/tools/private/release/process_backports_test.py index 0ceb7b3210..a0de7c8566 100644 --- a/tests/tools/private/release/process_backports_test.py +++ b/tests/tools/private/release/process_backports_test.py @@ -1,6 +1,7 @@ +import argparse import datetime import unittest -from unittest.mock import MagicMock, call, patch +from unittest.mock import call, patch from tests.tools.private.release.release_test_helper import _mock_git_and_gh from tools.private.release.process_backports import ProcessBackports @@ -18,7 +19,9 @@ def setUp(self): self.addCleanup(patch.stopall) def test_process_backports_no_pending(self): - args = MagicMock(issue=123, remote="origin", dry_run=False) + args = argparse.Namespace( + issue=123, remote="origin", dry_run=False, add=None, triggering_comment=None + ) self.mock_gh.get_issue_body.return_value = "No backports here" result = ProcessBackports(args, self.mock_git, self.mock_gh).run() @@ -30,7 +33,9 @@ def test_process_backports_no_pending(self): @patch("tools.private.release.process_backports.datetime") def test_process_backports_success(self, mock_datetime): mock_datetime.date.today.return_value = datetime.date(2026, 7, 1) - args = MagicMock(issue=123, remote="origin", dry_run=False) + args = argparse.Namespace( + issue=123, remote="origin", dry_run=False, add=None, triggering_comment=None + ) self.mock_gh.get_issue_title.return_value = "Release 2.0.0" self.mock_gh.get_issue_body.return_value = """ ## Checklist @@ -83,7 +88,9 @@ def mock_resolve(items): @patch("tools.private.release.process_backports.datetime") def test_process_backports_dry_run(self, mock_datetime): mock_datetime.date.today.return_value = datetime.date(2026, 7, 1) - args = MagicMock(issue=123, remote="origin", dry_run=True) + args = argparse.Namespace( + issue=123, remote="origin", dry_run=True, add=None, triggering_comment=None + ) self.mock_gh.get_issue_title.return_value = "Release 2.0.0" self.mock_gh.get_issue_body.return_value = """ ## Checklist @@ -131,7 +138,9 @@ def mock_resolve(items): self.mock_gh.update_issue_body.assert_not_called() def test_process_backports_ignored_and_failed_states(self): - args = MagicMock(issue=123, remote="origin", dry_run=False) + args = argparse.Namespace( + issue=123, remote="origin", dry_run=False, add=None, triggering_comment=None + ) self.mock_gh.get_issue_title.return_value = "Release 2.0.0" self.mock_gh.get_issue_body.return_value = """ ## Checklist @@ -170,7 +179,9 @@ def mock_resolve(items): self.mock_git.cherry_pick.assert_not_called() def test_process_backports_ignored_error_status(self): - args = MagicMock(issue=123, remote="origin", dry_run=False) + args = argparse.Namespace( + issue=123, remote="origin", dry_run=False, add=None, triggering_comment=None + ) self.mock_gh.get_issue_title.return_value = "Release 2.0.0" self.mock_gh.get_issue_body.return_value = """ ## Checklist @@ -193,7 +204,9 @@ def test_process_backports_ignored_error_status(self): @patch("tools.private.release.process_backports.datetime") def test_process_backports_cherry_pick_failed(self, mock_datetime): mock_datetime.date.today.return_value = datetime.date(2026, 7, 1) - args = MagicMock(issue=123, remote="origin", dry_run=False) + args = argparse.Namespace( + issue=123, remote="origin", dry_run=False, add=None, triggering_comment=None + ) self.mock_gh.get_issue_title.return_value = "Release 2.0.0" self.mock_gh.get_issue_body.return_value = """ ## Checklist diff --git a/tests/tools/private/release/promote_rc_test.py b/tests/tools/private/release/promote_rc_test.py index 71980dc1fd..68797e2e1e 100644 --- a/tests/tools/private/release/promote_rc_test.py +++ b/tests/tools/private/release/promote_rc_test.py @@ -1,5 +1,6 @@ +import argparse import unittest -from unittest.mock import MagicMock, call, patch +from unittest.mock import call, patch from tests.tools.private.release.release_test_helper import _mock_git_and_gh from tools.private.release.gh import NoTrackingIssueError @@ -12,7 +13,9 @@ def setUp(self): def test_promote_rc_success(self): # Arrange - args = MagicMock(version="2.0.0", issue=123, dry_run=False, remote="my-remote") + args = argparse.Namespace( + version="2.0.0", issue=123, dry_run=False, remote="my-remote" + ) self.mock_git.get_remote_tags.return_value = ["2.0.0-rc0", "2.0.0-rc1"] self.mock_git.get_commit_sha.return_value = "abcdef123456" self.mock_git.tag_exists.return_value = False @@ -24,8 +27,15 @@ def test_promote_rc_success(self): # Assert self.assertEqual(result, 0) - self.mock_git.fetch.assert_called_once_with("my-remote", tags=True, force=True) - self.mock_git.get_commit_sha.assert_called_once_with("2.0.0-rc1") + self.mock_git.fetch.assert_has_calls( + [ + call("my-remote", tags=True, force=True), + call("my-remote", refspec="release/2.0"), + ] + ) + self.mock_git.get_commit_sha.assert_has_calls( + [call("2.0.0-rc1"), call(remote_ref="my-remote/release/2.0")] + ) self.mock_git.checkout.assert_not_called() self.mock_git.tag_exists.assert_called_once_with("2.0.0") self.mock_git.tag.assert_called_once_with("2.0.0", "abcdef123456") @@ -48,7 +58,9 @@ def test_promote_rc_success(self): def test_promote_rc_resolve_issue_success(self): # Arrange - args = MagicMock(version="2.0.0", issue=None, dry_run=False, remote="my-remote") + args = argparse.Namespace( + version="2.0.0", issue=None, dry_run=False, remote="my-remote" + ) self.mock_git.get_remote_tags.return_value = ["2.0.0-rc1"] self.mock_git.tag_exists.return_value = False self.mock_gh.get_release_tracking_issue.side_effect = None @@ -62,9 +74,16 @@ def test_promote_rc_resolve_issue_success(self): # Assert self.assertEqual(result, 0) - self.mock_git.fetch.assert_called_once_with("my-remote", tags=True, force=True) + self.mock_git.fetch.assert_has_calls( + [ + call("my-remote", tags=True, force=True), + call("my-remote", refspec="release/2.0"), + ] + ) self.mock_gh.get_release_tracking_issue.assert_called_once_with("2.0.0") - self.mock_git.get_commit_sha.assert_called_once_with("2.0.0-rc1") + self.mock_git.get_commit_sha.assert_has_calls( + [call("2.0.0-rc1"), call(remote_ref="my-remote/release/2.0")] + ) self.mock_git.checkout.assert_not_called() self.mock_git.tag.assert_called_once_with("2.0.0", "abcdef123456") self.mock_git.push.assert_called_once_with("my-remote", "2.0.0") @@ -82,11 +101,12 @@ def test_promote_rc_resolve_issue_success(self): ) self.mock_gh.post_issue_comment.assert_called_once_with(123, expected_comment) - def test_promote_rc_defaults_to_determine_next_version(self): + def test_promote_rc_resolves_version_from_issue(self): # Arrange - args = MagicMock(version=None, issue=123, dry_run=False, remote="my-remote") - self.mock_git.get_current_branch.return_value = "release/2.0" - self.mock_git.get_tags.return_value = ["2.0.0"] + args = argparse.Namespace( + version=None, issue=123, dry_run=False, remote="my-remote" + ) + self.mock_gh.get_issue_title.return_value = "Release 2.0.1" self.mock_git.get_remote_tags.return_value = ["2.0.1-rc0"] self.mock_git.get_commit_sha.return_value = "12345678" self.mock_git.tag_exists.return_value = False @@ -98,13 +118,20 @@ def test_promote_rc_defaults_to_determine_next_version(self): # Assert self.assertEqual(result, 0) - self.mock_git.fetch.assert_called_once_with("my-remote", tags=True, force=True) - self.mock_git.get_current_branch.assert_called_once() - self.mock_git.get_tags.assert_called_once() + self.mock_git.fetch.assert_has_calls( + [ + call("my-remote", tags=True, force=True), + call("my-remote", refspec="release/2.0"), + ] + ) + self.mock_git.get_current_branch.assert_not_called() + self.mock_git.get_tags.assert_not_called() self.mock_git.get_remote_tags.assert_called_once_with("my-remote") self.mock_git.checkout.assert_not_called() - self.mock_git.get_commit_sha.assert_called_once_with("2.0.1-rc0") + self.mock_git.get_commit_sha.assert_has_calls( + [call("2.0.1-rc0"), call(remote_ref="my-remote/release/2.0")] + ) self.mock_git.tag.assert_called_once_with("2.0.1", "12345678") self.mock_git.push.assert_called_once_with("my-remote", "2.0.1") @@ -124,7 +151,9 @@ def test_promote_rc_defaults_to_determine_next_version(self): @patch("builtins.print") def test_promote_rc_dry_run_success(self, mock_print): # Arrange - args = MagicMock(version="2.0.0", issue=123, dry_run=True, remote="my-remote") + args = argparse.Namespace( + version="2.0.0", issue=123, dry_run=True, remote="my-remote" + ) self.mock_git.get_remote_tags.return_value = ["2.0.0-rc0", "2.0.0-rc1"] self.mock_git.get_commit_sha.return_value = "abcdef123456" self.mock_git.tag_exists.return_value = False @@ -136,8 +165,15 @@ def test_promote_rc_dry_run_success(self, mock_print): # Assert self.assertEqual(result, 0) - self.mock_git.fetch.assert_called_once_with("my-remote", tags=True, force=True) - self.mock_git.get_commit_sha.assert_called_once_with("2.0.0-rc1") + self.mock_git.fetch.assert_has_calls( + [ + call("my-remote", tags=True, force=True), + call("my-remote", refspec="release/2.0"), + ] + ) + self.mock_git.get_commit_sha.assert_has_calls( + [call("2.0.0-rc1"), call(remote_ref="my-remote/release/2.0")] + ) self.mock_git.tag_exists.assert_called_once_with("2.0.0") # Core dry-run assertions: NO modifications @@ -149,6 +185,7 @@ def test_promote_rc_dry_run_success(self, mock_print): mock_print.assert_has_calls( [ call("Verifying tracking issue #123 format..."), + call("Fetching remote branch my-remote/release/2.0..."), call( "[DRY RUN] Pre-conditions passed successfully for promoting" " 2.0.0-rc1 to 2.0.0." @@ -162,7 +199,9 @@ def test_promote_rc_dry_run_success(self, mock_print): def test_promote_rc_tag_already_exists(self): # Arrange - args = MagicMock(version="2.0.0", issue=123, remote="my-remote") + args = argparse.Namespace( + version="2.0.0", issue=123, dry_run=False, remote="my-remote" + ) self.mock_git.get_remote_tags.return_value = ["2.0.0-rc1"] self.mock_git.tag_exists.return_value = True @@ -179,7 +218,9 @@ def test_promote_rc_tag_already_exists(self): def test_promote_rc_issue_not_found(self): # Arrange - args = MagicMock(version="2.0.0", issue=None, remote="my-remote") + args = argparse.Namespace( + version="2.0.0", issue=None, dry_run=False, remote="my-remote" + ) self.mock_git.get_remote_tags.return_value = ["2.0.0-rc1"] self.mock_git.tag_exists.return_value = False self.mock_gh.get_release_tracking_issue.side_effect = NoTrackingIssueError( @@ -199,7 +240,9 @@ def test_promote_rc_issue_not_found(self): def test_promote_rc_issue_malformed(self): # Arrange - args = MagicMock(version="2.0.0", issue=123, remote="my-remote") + args = argparse.Namespace( + version="2.0.0", issue=123, dry_run=False, remote="my-remote" + ) self.mock_git.get_remote_tags.return_value = ["2.0.0-rc1"] self.mock_git.tag_exists.return_value = False self.mock_git.get_commit_sha.return_value = "abcdef123456" @@ -219,7 +262,9 @@ def test_promote_rc_issue_malformed(self): def test_promote_rc_no_rc_found(self): # Arrange - args = MagicMock(version="2.0.0", issue=123, remote="my-remote") + args = argparse.Namespace( + version="2.0.0", issue=123, dry_run=False, remote="my-remote" + ) self.mock_git.get_remote_tags.return_value = [] # Act diff --git a/tools/private/release/gh.py b/tools/private/release/gh.py index 8b938aecf6..e670fc98df 100644 --- a/tools/private/release/gh.py +++ b/tools/private/release/gh.py @@ -7,6 +7,17 @@ from tools.private.release.release_issue import BackportTask from tools.private.release.shell import run_cmd +# GitHub reaction types +# See: https://docs.github.com/en/rest/reactions/reactions?apiVersion=2022-11-28#about-reactions +GH_REACTION_THUMBS_UP = "+1" +GH_REACTION_THUMBS_DOWN = "-1" +GH_REACTION_LAUGH = "laugh" +GH_REACTION_CONFUSED = "confused" +GH_REACTION_HEART = "heart" +GH_REACTION_HOORAY = "hooray" +GH_REACTION_ROCKET = "rocket" +GH_REACTION_EYES = "eyes" + class MultipleTrackingIssuesError(ValueError): """Raised when multiple open tracking issues are found for a version.""" @@ -319,6 +330,28 @@ def post_issue_comment(self, issue_num: int, comment_body: str) -> None: capture_output=False, ) + def add_comment_reaction(self, comment_id: int, reaction: str) -> None: + """Adds a reaction to a comment. + + Args: + comment_id: The ID of the comment. + reaction: The reaction type (e.g. '+1', '-1', 'eyes', etc). + """ + path = f"/repos/{self.repo}/issues/comments/{comment_id}/reactions" + self._run_gh( + "api", + "--method", + "POST", + "-H", + "Accept: application/vnd.github+json", + "-H", + "X-GitHub-Api-Version: 2022-11-28", + path, + "-f", + f"content={reaction}", + capture_output=False, + ) + def get_merge_commits_for_prs( self, pending_items: list[BackportTask] ) -> list[BackportTask]: diff --git a/tools/private/release/process_backports.py b/tools/private/release/process_backports.py index 83394bd27e..f0308d7847 100644 --- a/tools/private/release/process_backports.py +++ b/tools/private/release/process_backports.py @@ -5,15 +5,17 @@ from typing import Any from tools.private.release import changelog_news -from tools.private.release.gh import GitHub +from tools.private.release.gh import GH_REACTION_THUMBS_DOWN, GitHub from tools.private.release.git import Git from tools.private.release.release_issue import ( RELEASE_TITLE_RE, + add_backports_to_body, parse_backports, update_task_in_body, ) from tools.private.release.utils import ( get_latest_rc_tag, + parse_pr_list, replace_version_next, ) @@ -178,7 +180,46 @@ def _cherry_pick_and_update_prs( def run(self) -> int: """Executes the process-backports subcommand.""" args = self.args + exit_code = 0 + try: + exit_code = self._run_internal() + except Exception as e: + print(f"Unexpected error: {e}") + exit_code = 1 + + if exit_code != 0 and args.triggering_comment: + print(f"Reacting with thumbs-down to comment {args.triggering_comment}...") + try: + self.gh.add_comment_reaction( + args.triggering_comment, GH_REACTION_THUMBS_DOWN + ) + except Exception as e: + print(f"Failed to add reaction to comment: {e}") + + return exit_code + + def _run_internal(self) -> int: + """Internal implementation of process-backports.""" + args = self.args body = self.gh.get_issue_body(args.issue) + + if args.add: + print(f"Adding backports {args.add} to tracking issue #{args.issue}...") + try: + body = add_backports_to_body(body, args.add) + except ValueError as e: + print(f"Error: {e}") + return 1 + + if not args.dry_run: + self.gh.update_issue_body(args.issue, body) + print("Successfully updated tracking issue checklist.") + else: + print( + "[DRY RUN] Would update tracking issue checklist with new" + " backports." + ) + items = parse_backports(body) pending_items = [ @@ -295,6 +336,16 @@ def add_parser(cls, subparsers): required=True, help="The git remote to push changes to (required).", ) + parser.add_argument( + "--add", + type=parse_pr_list, + help="PR numbers (comma or space separated) to add before processing.", + ) + parser.add_argument( + "--triggering-comment", + type=int, + help="The ID of the comment that triggered this run (optional).", + ) parser.add_argument( "--dry-run", action=argparse.BooleanOptionalAction, diff --git a/tools/private/release/promote_rc.py b/tools/private/release/promote_rc.py index 80b4ac8065..ac3a841526 100644 --- a/tools/private/release/promote_rc.py +++ b/tools/private/release/promote_rc.py @@ -5,7 +5,10 @@ from tools.private.release.gh import GitHub from tools.private.release.git import Git -from tools.private.release.release_issue import update_task_in_body +from tools.private.release.release_issue import ( + RELEASE_TITLE_RE, + update_task_in_body, +) from tools.private.release.utils import ( REPO_URL, determine_next_version, @@ -30,7 +33,23 @@ def run(self) -> int: version = args.version if version is None: - version = determine_next_version() + if args.issue: + issue_title = self.gh.get_issue_title(args.issue) + version_match = RELEASE_TITLE_RE.search(issue_title) + if version_match: + version = version_match.group(1) + print( + f"Resolved version {version} from tracking issue" + f" #{args.issue} title." + ) + else: + print( + f"Error: Could not parse version from issue title:" + f" {issue_title}" + ) + return 1 + else: + version = determine_next_version() latest_rc = get_latest_rc_tag(version, remote=args.remote) if not latest_rc: @@ -57,9 +76,56 @@ def run(self) -> int: # Get commit SHA of the RC tag (which will be the same for the final tag) commit_sha = self.git.get_commit_sha(latest_rc) - # Verify issue is in the right format by trying to prepare the update + # Verify issue can be found and read it early print(f"Verifying tracking issue #{issue_num} format...") body = self.gh.get_issue_body(issue_num) + + # Determine release branch name and verify it matches the RC tag commit + branch_version = ".".join(version.split(".")[:2]) + branch_name = f"release/{branch_version}" + remote_branch = f"{args.remote}/{branch_name}" + + print(f"Fetching remote branch {remote_branch}...") + self.git.fetch(args.remote, refspec=branch_name) + try: + branch_sha = self.git.get_commit_sha(remote_ref=remote_branch) + except Exception as e: + print( + f"Error: Could not get commit SHA for remote branch" + f" {remote_branch}: {e}" + ) + return 1 + + if commit_sha != branch_sha: + print( + f"Error: The latest RC tag {latest_rc} ({commit_sha[:8]}) is not at" + f" the head of release branch {remote_branch} ({branch_sha[:8]})." + ) + metadata = { + "status": "error-rc-tag-not-branch-head", + "rc": latest_rc, + "branch_commit": branch_sha[:8], + "tag_commit": commit_sha[:8], + } + try: + updated_body = update_task_in_body( + body, "Tag Final", checked=False, metadata=metadata + ) + except ValueError as e: + print(f"Error: Tracking issue #{issue_num} is malformed: {e}") + return 1 + + if not args.dry_run: + self.gh.update_issue_body(issue_num, updated_body) + print(f"Updated tracking issue #{issue_num} with error status.") + else: + print( + f"[DRY RUN] Would update tracking issue #{issue_num} with" + f" error status." + ) + return 1 + + # Verify issue is in the right format by trying to prepare the update (for success case) metadata = {"status": "done", "tag": version, "commit": commit_sha[:8]} try: updated_body = update_task_in_body( diff --git a/tools/private/release/release_issue.py b/tools/private/release/release_issue.py index 32e0a0fee7..75d00759ec 100644 --- a/tools/private/release/release_issue.py +++ b/tools/private/release/release_issue.py @@ -115,11 +115,11 @@ def format_metadata_line(checked, name, metadata): metadata_pairs = [] for k, v in metadata.items(): - if k == "commit": - # The 'commit' key is special-cased with a space after '=' so that - # GitHub autolinks the commit SHA. Autolinking requires certain - # characters to precede the value. - metadata_pairs.append(f"commit= {v}") + if k == "commit" or k.endswith("_commit"): + # The 'commit' key (and keys ending with '_commit') is special-cased with + # a space after '=' so that GitHub autolinks the commit SHA. Autolinking + # requires certain characters to precede the value. + metadata_pairs.append(f"{k}= {v}") else: metadata_pairs.append(f"{k}={v}") metadata_str = " ".join(metadata_pairs) @@ -245,3 +245,42 @@ def parse_backports(body): ) ) return items + + +def add_backports_to_body(body: str, prs: list[int]) -> str: + """Adds new backport checklist items to the ## Backports section.""" + body = body.replace("\r\n", "\n") + # Find the Backports section + pattern = r"(## Backports\n)(.*?)(?=\n##|\n---|\Z)" + match = re.search(pattern, body, re.DOTALL | re.IGNORECASE) + if not match: + raise ValueError("Could not find '## Backports' section in issue body.") + + section_content = match.group(2) + + # Parse existing backports to avoid duplicates + existing_items = parse_backports(body) + existing_prs = { + int(item.pr_ref.lstrip("#")) + for item in existing_items + if item.pr_ref.startswith("#") + } + + new_lines = [] + for pr in prs: + if pr in existing_prs: + print(f"PR #{pr} is already in the backports list. Skipping.") + continue + new_lines.append(f"- [ ] #{pr}") + + if not new_lines: + return body + + # Append new lines to the section content. + section_content_clean = section_content.rstrip("\n") + separator = "\n" if section_content_clean else "" + updated_section = section_content_clean + separator + "\n".join(new_lines) + "\n\n" + + # Replace the old section with the updated one + start, end = match.span(2) + return body[:start] + updated_section + body[end:] diff --git a/tools/private/release/utils.py b/tools/private/release/utils.py index e774c80134..db7b96a378 100644 --- a/tools/private/release/utils.py +++ b/tools/private/release/utils.py @@ -172,3 +172,25 @@ def replace_version_next(version): new_content = new_content.replace("VERSION_NEXT_PATCH", version) with open(filepath, "w") as f: f.write(new_content) + + +def parse_pr_list(value: str) -> list[int]: + """Parses a comma or space separated list of PR numbers. + + PR numbers can optionally be prefixed with '#'. + """ + if not value: + return [] + # Split by space and/or comma + pr_strings = [p for p in re.split(r"[\s,]+", value.strip()) if p] + prs = [] + for pr_str in pr_strings: + clean_pr_str = pr_str.lstrip("#") + try: + prs.append(int(clean_pr_str)) + except ValueError as e: + raise argparse.ArgumentTypeError( + f"Invalid PR reference '{pr_str}'. Must be integer optionally" + f" prefixed with '#'." + ) from e + return prs From 1f877845ae26f19e2e214d74990677da167c8b67 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Fri, 3 Jul 2026 01:00:36 -0700 Subject: [PATCH 812/922] chore(release): grant write permissions to on_issue_comment workflow (#3892) Update on_issue_comment.yaml to request contents: write and issues: write permissions, which are required for the called reusable workflows. --- .github/workflows/on_issue_comment.yaml | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/.github/workflows/on_issue_comment.yaml b/.github/workflows/on_issue_comment.yaml index 4be5906596..9c6a3eeb6d 100644 --- a/.github/workflows/on_issue_comment.yaml +++ b/.github/workflows/on_issue_comment.yaml @@ -5,8 +5,8 @@ on: types: [created] permissions: - contents: read - issues: read + contents: write + issues: write jobs: # This job always runs to prevent GHA from marking the run as failed when From e3c88670ae4706814b0ef339587074784c2bf07a Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Fri, 3 Jul 2026 01:04:30 -0700 Subject: [PATCH 813/922] chore(release): add pull-requests: write permission to on_issue_comment workflow (#3893) Update on_issue_comment.yaml to request pull-requests: write permission, which is required for release_prepare.yaml. --- .github/workflows/on_issue_comment.yaml | 1 + 1 file changed, 1 insertion(+) diff --git a/.github/workflows/on_issue_comment.yaml b/.github/workflows/on_issue_comment.yaml index 9c6a3eeb6d..02af5272e6 100644 --- a/.github/workflows/on_issue_comment.yaml +++ b/.github/workflows/on_issue_comment.yaml @@ -7,6 +7,7 @@ on: permissions: contents: write issues: write + pull-requests: write jobs: # This job always runs to prevent GHA from marking the run as failed when From 12ce31bb30e1bb3e6f347c92bbcb1dd237a590b5 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Fri, 3 Jul 2026 01:15:20 -0700 Subject: [PATCH 814/922] chore(release): auto-add RC tasks to checklist in create-rc (#3894) Modify create-rc subcommand to automatically insert missing Tag RC tasks into the tracking issue checklist before Tag Final, rather than failing. --- tests/tools/private/release/create_rc_test.py | 41 +++++++++++++ .../private/release/process_backports_test.py | 58 +++++++++++++++++++ tools/private/release/create_rc.py | 25 ++++---- tools/private/release/process_backports.py | 17 ++++++ tools/private/release/release_issue.py | 29 ++++++++++ 5 files changed, 157 insertions(+), 13 deletions(-) diff --git a/tests/tools/private/release/create_rc_test.py b/tests/tools/private/release/create_rc_test.py index 193f11561d..0937dfbac6 100644 --- a/tests/tools/private/release/create_rc_test.py +++ b/tests/tools/private/release/create_rc_test.py @@ -1,3 +1,4 @@ +import argparse import os import pathlib import tempfile @@ -212,6 +213,46 @@ def test_create_rc_with_finished_backports(self): self.mock_git.tag.assert_called_once_with("2.0.0-rc0", "my-remote/release/2.0") self.mock_git.push.assert_called_once_with("my-remote", "2.0.0-rc0") + def test_create_rc_auto_add_task(self): + # Arrange + args = argparse.Namespace(issue=123, remote="my-remote") + self.mock_gh.get_issue_title.return_value = "Release 2.0.0" + self.mock_gh.get_issue_body.return_value = """ +## Checklist +- [x] Prepare Release | status=done pr=#122 commit=abcdef12 +- [x] Create Release branch | status=done branch=release/2.0 commit=abcdef12 +- [x] Tag RC0 | status=done tag=2.0.0-rc0 commit=abcdef12 +- [ ] Tag Final +""" + self.mock_git.get_remote_tags.return_value = ["2.0.0-rc0"] + self.mock_git.get_commit_sha.return_value = "1234567890" + + # Act + result = CreateRc(args, self.mock_git, self.mock_gh).run() + + # Assert + self.assertEqual(result, 0) + self.mock_git.tag.assert_called_once_with("2.0.0-rc1", "my-remote/release/2.0") + self.mock_git.push.assert_called_once_with("my-remote", "2.0.0-rc1") + + self.assertEqual(self.mock_gh.update_issue_body.call_count, 2) + call1_args = self.mock_gh.update_issue_body.call_args_list[0][0] + call2_args = self.mock_gh.update_issue_body.call_args_list[1][0] + + self.assertEqual(call1_args[0], 123) + self.assertIn("- [ ] Tag RC1", call1_args[1]) + self.assertIn( + "- [x] Tag RC0 | status=done tag=2.0.0-rc0 commit=abcdef12\n- [ ]" + " Tag RC1\n- [ ] Tag Final", + call1_args[1].strip(), + ) + + self.assertEqual(call2_args[0], 123) + self.assertIn( + "- [x] Tag RC1 | status=done tag=2.0.0-rc1 commit= 12345678", + call2_args[1], + ) + if __name__ == "__main__": unittest.main() diff --git a/tests/tools/private/release/process_backports_test.py b/tests/tools/private/release/process_backports_test.py index a0de7c8566..639171f59f 100644 --- a/tests/tools/private/release/process_backports_test.py +++ b/tests/tools/private/release/process_backports_test.py @@ -247,6 +247,64 @@ def mock_resolve(items): self.mock_git.commit.assert_not_called() self.mock_git.push.assert_not_called() + @patch("tools.private.release.process_backports.datetime") + def test_process_backports_add_backports_and_auto_add_rc_task(self, mock_datetime): + mock_datetime.date.today.return_value = datetime.date(2026, 7, 1) + args = argparse.Namespace( + issue=123, + remote="origin", + dry_run=False, + add=[124], + triggering_comment=None, + ) + self.mock_gh.get_issue_title.return_value = "Release 2.0.0" + self.mock_gh.get_issue_body.return_value = """ +## Checklist +- [x] Prepare Release | status=done pr=#122 commit=abcdef12 +- [x] Create Release branch | status=done branch=release/2.0 commit=abcdef12 +- [x] Tag RC0 | status=done tag=2.0.0-rc0 commit=abcdef12 +- [ ] Tag Final + +## Backports +""" + self.mock_git.get_remote_tags.return_value = ["2.0.0-rc0"] + self.mock_git.get_commit_sha.return_value = "12345678" + self.mock_git.get_commit_message.return_value = 'Cherry-pick "fix bug"' + + def mock_resolve(items): + for item in items: + if item.pr_ref == "#124": + item.commit = "abcdef12" + item.status = "done" + return items + + self.mock_gh.get_merge_commits_for_prs.side_effect = mock_resolve + self.mock_git.sort_commits_chronologically.return_value = ["abcdef12"] + + result = ProcessBackports(args, self.mock_git, self.mock_gh).run() + + self.assertEqual(result, 0) + + # update_issue_body should be called twice: + # 1. When adding backports and auto-adding Tag RC1 task. + # 2. When updating the backport status to done. + self.assertEqual(self.mock_gh.update_issue_body.call_count, 2) + + call1_args = self.mock_gh.update_issue_body.call_args_list[0][0] + call2_args = self.mock_gh.update_issue_body.call_args_list[1][0] + + self.assertEqual(call1_args[0], 123) + self.assertIn("- [ ] #124", call1_args[1]) + self.assertIn("- [ ] Tag RC1", call1_args[1]) + self.assertIn( + "- [x] Tag RC0 | status=done tag=2.0.0-rc0 commit=abcdef12\n- [ ]" + " Tag RC1\n- [ ] Tag Final", + call1_args[1].strip(), + ) + + self.assertEqual(call2_args[0], 123) + self.assertIn("- [x] #124 | status=done rc=rc1 commit= 12345678", call2_args[1]) + if __name__ == "__main__": unittest.main() diff --git a/tools/private/release/create_rc.py b/tools/private/release/create_rc.py index 260cbb8940..abc91a8273 100644 --- a/tools/private/release/create_rc.py +++ b/tools/private/release/create_rc.py @@ -4,6 +4,7 @@ from tools.private.release.git import Git from tools.private.release.release_issue import ( RELEASE_TITLE_RE, + add_rc_task_to_body, parse_backports, parse_checklist_state, update_task_in_body, @@ -77,19 +78,17 @@ def run(self) -> int: # Precheck: next RC number must exist and be unchecked in the checklist rc_tags = state.get("rc_tags", {}) if next_rc_num not in rc_tags: - print( - f"Error: Checklist is missing required task 'Tag RC{next_rc_num}'" - f" to cut {version}-rc{next_rc_num}." - ) - return 1 - - target_rc_task = rc_tags[next_rc_num] - if target_rc_task.checked or target_rc_task.status == "done": - print( - f"Error: Task 'Tag RC{next_rc_num}' is already marked done in" - " the checklist." - ) - return 1 + print(f"Task 'Tag RC{next_rc_num}' not found in checklist. Adding it...") + body = add_rc_task_to_body(body, next_rc_num) + self.gh.update_issue_body(args.issue, body) + else: + target_rc_task = rc_tags[next_rc_num] + if target_rc_task.checked or target_rc_task.status == "done": + print( + f"Error: Task 'Tag RC{next_rc_num}' is already marked done in" + " the checklist." + ) + return 1 target_ref = f"{args.remote}/{branch_name}" commit_sha = self.git.get_commit_sha(target_ref) diff --git a/tools/private/release/process_backports.py b/tools/private/release/process_backports.py index f0308d7847..81dbe2bd5e 100644 --- a/tools/private/release/process_backports.py +++ b/tools/private/release/process_backports.py @@ -10,7 +10,9 @@ from tools.private.release.release_issue import ( RELEASE_TITLE_RE, add_backports_to_body, + add_rc_task_to_body, parse_backports, + parse_checklist_state, update_task_in_body, ) from tools.private.release.utils import ( @@ -207,6 +209,19 @@ def _run_internal(self) -> int: print(f"Adding backports {args.add} to tracking issue #{args.issue}...") try: body = add_backports_to_body(body, args.add) + state = parse_checklist_state(body) + rc_tags = state.get("rc_tags", {}) + has_pending_rc = any( + not task.checked and task.status != "done" + for task in rc_tags.values() + ) + next_rc_num = max(rc_tags.keys()) + 1 if rc_tags else 0 + if not has_pending_rc: + print( + f"No pending RC task found. Adding 'Tag" + f" RC{next_rc_num}' to checklist..." + ) + body = add_rc_task_to_body(body, next_rc_num) except ValueError as e: print(f"Error: {e}") return 1 @@ -219,6 +234,8 @@ def _run_internal(self) -> int: "[DRY RUN] Would update tracking issue checklist with new" " backports." ) + if not has_pending_rc: + print(f"[DRY RUN] Would add 'Tag RC{next_rc_num}' to checklist.") items = parse_backports(body) diff --git a/tools/private/release/release_issue.py b/tools/private/release/release_issue.py index 75d00759ec..2650b6607c 100644 --- a/tools/private/release/release_issue.py +++ b/tools/private/release/release_issue.py @@ -284,3 +284,32 @@ def add_backports_to_body(body: str, prs: list[int]) -> str: # Replace the old section with the updated one start, end = match.span(2) return body[:start] + updated_section + body[end:] + + +def add_rc_task_to_body(body: str, rc_num: int) -> str: + """Adds a new 'Tag RC' task to the checklist in the issue body.""" + body = body.replace("\r\n", "\n") + lines = body.splitlines() + + # Find the index of the last "Tag RC" line + last_rc_idx = -1 + for i, line in enumerate(lines): + parsed = parse_metadata_line(line) + if parsed and re.match(r"Tag RC\d+", parsed["name"], re.IGNORECASE): + last_rc_idx = i + + if last_rc_idx == -1: + # If no RC task found (unexpected, but fallback to before "Tag Final") + for i, line in enumerate(lines): + parsed = parse_metadata_line(line) + if parsed and parsed["name"].lower() == "tag final": + last_rc_idx = i - 1 + break + + if last_rc_idx == -1: + raise ValueError("Could not find a place to insert the new RC task.") + + new_task_line = f"- [ ] Tag RC{rc_num}" + lines.insert(last_rc_idx + 1, new_task_line) + + return "\n".join(lines) From d77d5a401ef474c51e56abb88c3bc03277dd208e Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Fri, 3 Jul 2026 01:30:14 -0700 Subject: [PATCH 815/922] fix(release): secure workflows against shell injection (#3895) Pass workflow inputs as environment variables instead of expanding them directly in run scripts to prevent shell injection. The shell expansions were guarded by collaborator checks, but best to prevent any type of injection. --- .github/workflows/release_create_rc.yaml | 7 ++++--- .github/workflows/release_prepare.yaml | 11 ++++++++--- .../workflows/release_process_backports.yaml | 17 ++++++++++------- .github/workflows/release_promote_rc.yaml | 14 ++++++++------ 4 files changed, 30 insertions(+), 19 deletions(-) diff --git a/.github/workflows/release_create_rc.yaml b/.github/workflows/release_create_rc.yaml index eaf6250b85..2c48dd8476 100644 --- a/.github/workflows/release_create_rc.yaml +++ b/.github/workflows/release_create_rc.yaml @@ -41,11 +41,12 @@ jobs: - name: Attempt RC Tagging id: tagger - run: | - bazel run //tools/private/release -- \ - create-rc --issue ${{ inputs.issue }} --remote origin env: GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} + ISSUE: ${{ inputs.issue }} + run: | + bazel run //tools/private/release -- \ + create-rc --issue "$ISSUE" --remote origin call_release: needs: tag_rc diff --git a/.github/workflows/release_prepare.yaml b/.github/workflows/release_prepare.yaml index 8ca38cb1c2..c83a7b74f9 100644 --- a/.github/workflows/release_prepare.yaml +++ b/.github/workflows/release_prepare.yaml @@ -39,8 +39,13 @@ jobs: git config --global user.email "41898282+github-actions[bot]@users.noreply.github.com" - name: Run Release Preparation Pipeline - run: | - bazel run //tools/private/release -- \ - prepare ${{ inputs.issue && format('--issue={0}', inputs.issue) || '' }} --no-dry-run env: GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} + ISSUE: ${{ inputs.issue }} + run: | + ARGS=() + if [ -n "$ISSUE" ]; then + ARGS+=("--issue=$ISSUE") + fi + bazel run //tools/private/release -- \ + prepare "${ARGS[@]}" --no-dry-run diff --git a/.github/workflows/release_process_backports.yaml b/.github/workflows/release_process_backports.yaml index 394a5b0fe9..ca719778df 100644 --- a/.github/workflows/release_process_backports.yaml +++ b/.github/workflows/release_process_backports.yaml @@ -55,19 +55,22 @@ jobs: git config --global user.email "41898282+github-actions[bot]@users.noreply.github.com" - name: Process Pending Backports + env: + GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} + ADD_BACKPORTS: ${{ inputs.add_backports }} + COMMENT_ID: ${{ inputs.comment_id }} + ISSUE: ${{ inputs.issue }} run: | ARGS=() - if [ -n "${{ inputs.add_backports }}" ]; then - ARGS+=("--add=${{ inputs.add_backports }}") + if [ -n "$ADD_BACKPORTS" ]; then + ARGS+=("--add=$ADD_BACKPORTS") fi - if [ -n "${{ inputs.comment_id }}" ]; then - ARGS+=("--triggering-comment=${{ inputs.comment_id }}") + if [ -n "$COMMENT_ID" ]; then + ARGS+=("--triggering-comment=$COMMENT_ID") fi bazel run //tools/private/release -- process-backports \ - --issue ${{ inputs.issue }} \ + --issue "$ISSUE" \ --remote origin \ --no-dry-run \ "${ARGS[@]}" - env: - GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} diff --git a/.github/workflows/release_promote_rc.yaml b/.github/workflows/release_promote_rc.yaml index 987583a1fd..7dc1c118b2 100644 --- a/.github/workflows/release_promote_rc.yaml +++ b/.github/workflows/release_promote_rc.yaml @@ -42,17 +42,19 @@ jobs: git config --global user.email "41898282+github-actions[bot]@users.noreply.github.com" - name: Run Promote RC + env: + GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} + VERSION: ${{ inputs.version }} + ISSUE: ${{ inputs.issue }} run: | ARGS=() - if [ -n "${{ inputs.version }}" ]; then - ARGS+=("${{ inputs.version }}") + if [ -n "$VERSION" ]; then + ARGS+=("$VERSION") fi - if [ -n "${{ inputs.issue }}" ]; then - ARGS+=("--issue" "${{ inputs.issue }}") + if [ -n "$ISSUE" ]; then + ARGS+=("--issue" "$ISSUE") fi ARGS+=("--remote" "origin") ARGS+=("--no-dry-run") bazel run //tools/private/release -- promote-rc "${ARGS[@]}" - env: - GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} From 1d99f70e5967da59ecd948c0e6775e1e6424d889 Mon Sep 17 00:00:00 2001 From: David Zbarsky Date: Fri, 3 Jul 2026 10:23:47 -0400 Subject: [PATCH 816/922] fix: run executable zip action with execution Python to better support windows building (#3890) The legacy self-executable zip action invokes `cat` through `run_shell`. The action does not declare `cat`, and a Windows-hosted Bazel invocation cannot use that action reliably when the selected execution platform is Linux. Run the existing `exe_zip_maker` through `actions_run` instead. This selects the helper and Python runtime from the execution configuration, preserves the prelude-plus-zip output format, and removes the ambient shell utility dependency. A news entry documents the fix. The Bazel integration that motivated this fix is [bazelbuild/bazel#30121](https://github.com/bazelbuild/bazel/pull/30121). --------- Co-authored-by: Richard Levasseur --- news/3890.fixed.md | 2 ++ python/private/py_executable.bzl | 22 ++++++++++++++-------- 2 files changed, 16 insertions(+), 8 deletions(-) create mode 100644 news/3890.fixed.md diff --git a/news/3890.fixed.md b/news/3890.fixed.md new file mode 100644 index 0000000000..59cd5fc5ca --- /dev/null +++ b/news/3890.fixed.md @@ -0,0 +1,2 @@ +(binaries) Fixed building of legacy zipapps on Windows execution platforms by +using a hermetic tool instead of host `cat`. diff --git a/python/private/py_executable.bzl b/python/private/py_executable.bzl index c519af8c09..242b09344e 100644 --- a/python/private/py_executable.bzl +++ b/python/private/py_executable.bzl @@ -38,6 +38,7 @@ load(":cc_helper.bzl", "cc_helper") load( ":common.bzl", "ExplicitSymlink", + "actions_run", "collect_cc_info", "collect_deps", "collect_imports", @@ -219,6 +220,10 @@ accepting arbitrary Python versions. default = "//python/private:debugger_if_target_config", providers = [PyInfo], ), + "_exe_zip_maker": lambda: attrb.Label( + cfg = "exec", + default = "//tools/private/zipapp:exe_zip_maker", + ), "_launcher": lambda: attrb.Label( cfg = "target", # NOTE: This is an executable, but is only used for Windows. It @@ -1095,15 +1100,16 @@ def _create_executable_zip_file( else: ctx.actions.write(prelude, "#!/usr/bin/env python3\n") - ctx.actions.run_shell( - command = "cat {prelude} {zip} > {output}".format( - prelude = prelude.path, - zip = zip_file.path, - output = output.path, - ), - inputs = [prelude, zip_file], + args = ctx.actions.args() + args.add(prelude) + args.add(zip_file) + args.add(output) + actions_run( + ctx, + executable = ctx.attr._exe_zip_maker, + arguments = [args], + inputs = depset([prelude, zip_file]), outputs = [output], - use_default_shell_env = True, mnemonic = "PyBuildExecutableZip", progress_message = "Build Python zip executable: %{label}", ) From f1db067d634e80a700bc1ce9e5fd3587c170e429 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Fri, 3 Jul 2026 08:44:28 -0700 Subject: [PATCH 817/922] chore(release): process backports as part of creating RC (#3896) Automatically process pending backports when `/create-rc` is called. If any backport fails to process (e.g. due to cherry-pick conflicts), the RC creation will abort. Additionally, support `--triggering-comment` option to react with a thumbs-down emoji on the triggering comment if the command fails. - Call `ProcessBackports` in `CreateRc` before proceeding with RC tagging. - Refactor `CreateRc.run` to handle exceptions and reactions. - Update GitHub Actions workflows to pass `comment_id` to `release_create_rc` workflow. - Add unit tests for the new integration and reaction behavior. --- .github/workflows/on_issue_comment.yaml | 1 + .github/workflows/release_create_rc.yaml | 15 ++- RELEASING.md | 5 +- news/process-backports-before-rc.changed.md | 3 + tests/tools/private/release/create_rc_test.py | 124 ++++++++++++++++++ tools/private/release/create_rc.py | 49 ++++++- 6 files changed, 193 insertions(+), 4 deletions(-) create mode 100644 news/process-backports-before-rc.changed.md diff --git a/.github/workflows/on_issue_comment.yaml b/.github/workflows/on_issue_comment.yaml index 02af5272e6..14248d0d7a 100644 --- a/.github/workflows/on_issue_comment.yaml +++ b/.github/workflows/on_issue_comment.yaml @@ -73,6 +73,7 @@ jobs: uses: ./.github/workflows/release_create_rc.yaml with: issue: ${{ needs.parse_comment.outputs.issue_number }} + comment_id: "${{ github.event.comment.id }}" secrets: inherit call_prepare: diff --git a/.github/workflows/release_create_rc.yaml b/.github/workflows/release_create_rc.yaml index 2c48dd8476..a96a04a0ec 100644 --- a/.github/workflows/release_create_rc.yaml +++ b/.github/workflows/release_create_rc.yaml @@ -7,12 +7,20 @@ on: description: 'The Release Tracking Issue Number (e.g., 142)' required: true type: string + comment_id: + description: 'The ID of the comment that triggered this run (optional)' + required: false + type: string workflow_call: inputs: issue: description: 'The Release Tracking Issue Number (e.g., 142)' required: true type: string + comment_id: + description: 'The ID of the comment that triggered this run (optional)' + required: false + type: string permissions: contents: write @@ -44,9 +52,14 @@ jobs: env: GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} ISSUE: ${{ inputs.issue }} + COMMENT_ID: ${{ inputs.comment_id }} run: | + ARGS=() + if [ -n "$COMMENT_ID" ]; then + ARGS+=("--triggering-comment=$COMMENT_ID") + fi bazel run //tools/private/release -- \ - create-rc --issue "$ISSUE" --remote origin + create-rc --issue "$ISSUE" --remote origin "${ARGS[@]}" call_release: needs: tag_rc diff --git a/RELEASING.md b/RELEASING.md index 528450f611..7846cef883 100644 --- a/RELEASING.md +++ b/RELEASING.md @@ -33,8 +33,9 @@ Release Tracking Issue and automated workflows triggered by comments or issue ed 3. **Add Backports (if needed)**: If there are backports, add them following the [How to add backports](#how-to-add-backports) steps. -4. **Create an RC**: Comment `/create-rc` on the tracking issue. All pending - backports must be successfully processed before creating the RC. +4. **Create an RC**: Comment `/create-rc` on the tracking issue. This will + automatically process pending backports before creating the RC. If any + backport fails, the RC creation will abort. 5. **Iterate**: Repeat steps 3 and 4 until backports and RCs are no longer needed. diff --git a/news/process-backports-before-rc.changed.md b/news/process-backports-before-rc.changed.md new file mode 100644 index 0000000000..5591b700ce --- /dev/null +++ b/news/process-backports-before-rc.changed.md @@ -0,0 +1,3 @@ +Changed `/create-rc` command to automatically process pending backports before +creating the RC, and react with a thumbs-down emoji on failure if triggered by +a comment. diff --git a/tests/tools/private/release/create_rc_test.py b/tests/tools/private/release/create_rc_test.py index 0937dfbac6..06cdb77345 100644 --- a/tests/tools/private/release/create_rc_test.py +++ b/tests/tools/private/release/create_rc_test.py @@ -253,6 +253,130 @@ def test_create_rc_auto_add_task(self): call2_args[1], ) + @patch("tools.private.release.create_rc.ProcessBackports") + def test_create_rc_calls_process_backports(self, mock_pb_class): + # Arrange + mock_pb = mock_pb_class.return_value + mock_pb.run.return_value = 0 + + args = MagicMock(issue=123, remote="my-remote") + self.mock_gh.get_issue_title.return_value = "Release 2.0.0" + self.mock_gh.get_issue_body.return_value = """ +## Checklist +- [x] Prepare Release | status=done pr=#122 commit=abcdef12 +- [x] Create Release branch | status=done branch=release/2.0 commit=abcdef12 +- [ ] Tag RC0 | status=pending +""" + self.mock_git.get_remote_tags.return_value = [] + self.mock_git.get_commit_sha.return_value = "1234567890" + + # Act + result = CreateRc(args, self.mock_git, self.mock_gh).run() + + # Assert + self.assertEqual(result, 0) + mock_pb_class.assert_called_once() + called_args = mock_pb_class.call_args[0][0] + self.assertEqual(called_args.issue, 123) + self.assertEqual(called_args.remote, "my-remote") + self.assertFalse(called_args.dry_run) + self.assertIsNone(called_args.add) + self.assertIsNone(called_args.triggering_comment) + mock_pb.run.assert_called_once() + + @patch("tools.private.release.create_rc.ProcessBackports") + def test_create_rc_aborts_on_process_backports_failure(self, mock_pb_class): + # Arrange + mock_pb = mock_pb_class.return_value + mock_pb.run.return_value = 1 + + args = MagicMock(issue=123, remote="my-remote") + + # Act + result = CreateRc(args, self.mock_git, self.mock_gh).run() + + # Assert + self.assertEqual(result, 1) + mock_pb_class.assert_called_once() + mock_pb.run.assert_called_once() + self.mock_gh.get_issue_body.assert_not_called() + self.mock_git.tag.assert_not_called() + + @patch("tools.private.release.create_rc.ProcessBackports") + def test_create_rc_failure_reacts_to_comment(self, mock_pb_class): + # Arrange + mock_pb = mock_pb_class.return_value + mock_pb.run.return_value = 1 # Simulate failure + + args = MagicMock(issue=123, remote="my-remote", triggering_comment=456) + + # Act + result = CreateRc(args, self.mock_git, self.mock_gh).run() + + # Assert + self.assertEqual(result, 1) + self.mock_gh.add_comment_reaction.assert_called_once_with(456, "-1") + + @patch("tools.private.release.create_rc.ProcessBackports") + def test_create_rc_failure_no_comment_no_reaction(self, mock_pb_class): + # Arrange + mock_pb = mock_pb_class.return_value + mock_pb.run.return_value = 1 # Simulate failure + + args = MagicMock(issue=123, remote="my-remote", triggering_comment=None) + + # Act + result = CreateRc(args, self.mock_git, self.mock_gh).run() + + # Assert + self.assertEqual(result, 1) + self.mock_gh.add_comment_reaction.assert_not_called() + + @patch("tools.private.release.create_rc.ProcessBackports") + def test_create_rc_success_with_comment_no_reaction(self, mock_pb_class): + # Arrange + mock_pb = mock_pb_class.return_value + mock_pb.run.return_value = 0 + + args = MagicMock(issue=123, remote="my-remote", triggering_comment=456) + self.mock_gh.get_issue_title.return_value = "Release 2.0.0" + self.mock_gh.get_issue_body.return_value = """ +## Checklist +- [x] Prepare Release | status=done pr=#122 commit=abcdef12 +- [x] Create Release branch | status=done branch=release/2.0 commit=abcdef12 +- [ ] Tag RC0 | status=pending +""" + self.mock_git.get_remote_tags.return_value = [] + self.mock_git.get_commit_sha.return_value = "1234567890" + + # Act + result = CreateRc(args, self.mock_git, self.mock_gh).run() + + # Assert + self.assertEqual(result, 0) + self.mock_gh.add_comment_reaction.assert_not_called() + + @patch("tools.private.release.create_rc.ProcessBackports") + def test_create_rc_precondition_failure_reacts_to_comment(self, mock_pb_class): + # Arrange + mock_pb = mock_pb_class.return_value + mock_pb.run.return_value = 0 # Backports succeed + + args = MagicMock(issue=123, remote="my-remote", triggering_comment=456) + self.mock_gh.get_issue_body.return_value = """ +## Checklist +- [ ] Prepare Release | status=pending +- [ ] Create Release branch | status=pending +- [ ] Tag RC0 | status=pending +""" + + # Act + result = CreateRc(args, self.mock_git, self.mock_gh).run() + + # Assert + self.assertEqual(result, 1) + self.mock_gh.add_comment_reaction.assert_called_once_with(456, "-1") + if __name__ == "__main__": unittest.main() diff --git a/tools/private/release/create_rc.py b/tools/private/release/create_rc.py index abc91a8273..0916377925 100644 --- a/tools/private/release/create_rc.py +++ b/tools/private/release/create_rc.py @@ -1,7 +1,11 @@ """Subcommand to tag and push the next release candidate.""" -from tools.private.release.gh import GitHub +import traceback +from argparse import Namespace + +from tools.private.release.gh import GH_REACTION_THUMBS_DOWN, GitHub from tools.private.release.git import Git +from tools.private.release.process_backports import ProcessBackports from tools.private.release.release_issue import ( RELEASE_TITLE_RE, add_rc_task_to_body, @@ -26,6 +30,44 @@ def __init__(self, args, git: Git, gh: GitHub): def run(self) -> int: """Executes the create-rc subcommand.""" args = self.args + exit_code = 0 + try: + exit_code = self._run_internal() + except Exception as e: + print(f"Unexpected error: {e}") + traceback.print_exc() + exit_code = 1 + + if exit_code != 0 and args.triggering_comment: + print(f"Reacting with thumbs-down to comment {args.triggering_comment}...") + try: + self.gh.add_comment_reaction( + args.triggering_comment, GH_REACTION_THUMBS_DOWN + ) + except Exception as e: + print(f"Failed to add reaction to comment: {e}") + + return exit_code + + def _run_internal(self) -> int: + """Internal implementation of create-rc.""" + args = self.args + + # Try to process pending backports first + print("Processing pending backports before creating RC...") + backport_args = Namespace( + issue=args.issue, + remote=args.remote, + add=None, + triggering_comment=None, + dry_run=False, + ) + pb = ProcessBackports(backport_args, self.git, self.gh) + backports_exit_code = pb.run() + if backports_exit_code != 0: + print("Error: Processing backports failed. Aborting RC creation.") + return backports_exit_code + body = self.gh.get_issue_body(args.issue) state = parse_checklist_state(body) @@ -153,6 +195,11 @@ def add_parser(cls, subparsers): required=True, help="The git remote to push the RC tag to (required).", ) + parser.add_argument( + "--triggering-comment", + type=int, + help="The ID of the comment that triggered this run (optional).", + ) parser.set_defaults(command=cls.run_from_args) @classmethod From 5fe64d8694a6c48c33a247a2b90ab37773d2ea84 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Fri, 3 Jul 2026 20:28:21 -0700 Subject: [PATCH 818/922] chore(release): automate backport processing on PR merge, and have add-backports accept hashtag and url refs (#3897) Currently, backports must be manually added to the release tracking issue. This change automates the process by allowing maintainers to comment `/backport` on a PR to add it to the active release's backport checklist, and automatically triggering the backport processing when the PR is merged. To support this, the following changes were made: - Added `add-backports` subcommand to the release tool to append PRs to the tracking issue. - Added `on-pr-merged` subcommand to verify `/backport` comment, find the tracking issue, and process backports. - Created `release_add_backports.yaml` reusable workflow to run the `add-backports` subcommand. - Updated `on_comment.yaml` to parse `/backport` comments on PRs and trigger the addition. - Created `on_pr_closed.yaml` to detect merged PRs and run the `on-pr-merged` subcommand. - Added unit tests for the new subcommands and URL resolution. --- .github/workflows/on_comment.yaml | 140 ++++++++++++++++++ .github/workflows/on_issue_comment.yaml | 105 ------------- .github/workflows/on_pr_closed.yaml | 83 +++++++++++ .github/workflows/release_add_backports.yaml | 58 ++++++++ RELEASING.md | 34 ++++- tests/tools/private/release/BUILD.bazel | 26 ++++ .../private/release/add_backports_test.py | 104 +++++++++++++ tests/tools/private/release/create_rc_test.py | 24 +++ tests/tools/private/release/gh_test.py | 61 ++++++++ .../private/release/on_pr_merged_test.py | 112 ++++++++++++++ .../private/release/process_backports_test.py | 50 ++++++- .../private/release/release_issue_test.py | 31 ++++ tools/private/release/add_backports.py | 115 ++++++++++++++ tools/private/release/complete_prepare.py | 2 +- tools/private/release/create_rc.py | 5 +- tools/private/release/gh.py | 48 ++++++ tools/private/release/on_pr_merged.py | 113 ++++++++++++++ tools/private/release/process_backports.py | 20 ++- tools/private/release/release.py | 4 + tools/private/release/release_issue.py | 34 +++-- tools/private/release/utils.py | 19 +-- 21 files changed, 1046 insertions(+), 142 deletions(-) create mode 100644 .github/workflows/on_comment.yaml delete mode 100644 .github/workflows/on_issue_comment.yaml create mode 100644 .github/workflows/on_pr_closed.yaml create mode 100644 .github/workflows/release_add_backports.yaml create mode 100644 tests/tools/private/release/add_backports_test.py create mode 100644 tests/tools/private/release/gh_test.py create mode 100644 tests/tools/private/release/on_pr_merged_test.py create mode 100644 tools/private/release/add_backports.py create mode 100644 tools/private/release/on_pr_merged.py diff --git a/.github/workflows/on_comment.yaml b/.github/workflows/on_comment.yaml new file mode 100644 index 0000000000..ed7813adac --- /dev/null +++ b/.github/workflows/on_comment.yaml @@ -0,0 +1,140 @@ +name: "On Comment" + +on: + issue_comment: + types: [created] + +permissions: + contents: write + issues: write + pull-requests: write + +jobs: + # This job always runs to prevent GHA from marking the run as failed when + # all other jobs are skipped. + noop: + runs-on: ubuntu-latest + steps: + - run: echo "No-op" + + parse_comment: + runs-on: ubuntu-latest + if: | + github.event.comment.author_association == 'OWNER' || + github.event.comment.author_association == 'MEMBER' || + github.event.comment.author_association == 'COLLABORATOR' + outputs: + command: ${{ steps.parse.outputs.command }} + issue_number: ${{ steps.parse.outputs.issue_number }} + pr_number: ${{ steps.parse.outputs.pr_number }} + backports: ${{ steps.parse.outputs.backports }} + steps: + - name: Parse comment + id: parse + env: + COMMENT_BODY: ${{ github.event.comment.body }} + IS_PR: "${{ github.event.issue.pull_request != null }}" + EVENT_NUMBER: "${{ github.event.issue.number }}" + HAS_RELEASE_LABEL: "${{ contains(github.event.issue.labels.*.name, 'type: release') }}" + GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} + run: | + if [ "$IS_PR" = "false" ] && [ "$HAS_RELEASE_LABEL" = "true" ]; then + issue_number=$EVENT_NUMBER + if echo "$COMMENT_BODY" | grep -qE '^[[:space:]]*/create-rc([[:space:]]|$)'; then + echo "command=create-rc" >> "$GITHUB_OUTPUT" + echo "issue_number=$issue_number" >> "$GITHUB_OUTPUT" + elif echo "$COMMENT_BODY" | grep -qE '^[[:space:]]*/prepare([[:space:]]|$)'; then + echo "command=prepare" >> "$GITHUB_OUTPUT" + echo "issue_number=$issue_number" >> "$GITHUB_OUTPUT" + elif echo "$COMMENT_BODY" | grep -qE '^[[:space:]]*/process-backports([[:space:]]|$)'; then + echo "command=process-backports" >> "$GITHUB_OUTPUT" + echo "issue_number=$issue_number" >> "$GITHUB_OUTPUT" + elif echo "$COMMENT_BODY" | grep -qE '^[[:space:]]*/add-backports([[:space:]]|$)'; then + args=$(echo "$COMMENT_BODY" | grep -E '^[[:space:]]*/add-backports([[:space:]]|$)' | sed -E 's/^[[:space:]]*\/add-backports[[:space:]]*//') + args=$(echo "$args" | sed -e 's/^[[:space:],]*//' -e 's/[[:space:],]*$//') + csv=$(echo "$args" | sed -E 's/[[:space:],]+/ /g' | tr ' ' ',') + if [ -n "$csv" ]; then + echo "command=add-backports" >> "$GITHUB_OUTPUT" + echo "backports=$csv" >> "$GITHUB_OUTPUT" + echo "issue_number=$issue_number" >> "$GITHUB_OUTPUT" + else + echo "command=none" >> "$GITHUB_OUTPUT" + echo "Error: No PRs specified for add-backports." >&2 + gh api \ + --method POST \ + -H "Accept: application/vnd.github+json" \ + -H "X-GitHub-Api-Version: 2022-11-28" \ + /repos/${{ github.repository }}/issues/comments/${{ github.event.comment.id }}/reactions \ + -f "content=-1" + fi + elif echo "$COMMENT_BODY" | grep -qE '^[[:space:]]*/promote([[:space:]]|$)'; then + echo "command=promote" >> "$GITHUB_OUTPUT" + echo "issue_number=$issue_number" >> "$GITHUB_OUTPUT" + else + echo "command=none" >> "$GITHUB_OUTPUT" + fi + elif [ "$IS_PR" = "true" ]; then + pr_number=$EVENT_NUMBER + if echo "$COMMENT_BODY" | grep -qE '^[[:space:]]*/backport([[:space:]]|$)'; then + echo "command=pr-backport" >> "$GITHUB_OUTPUT" + echo "pr_number=$pr_number" >> "$GITHUB_OUTPUT" + else + echo "command=none" >> "$GITHUB_OUTPUT" + fi + else + echo "command=none" >> "$GITHUB_OUTPUT" + fi + + call_create_rc: + needs: parse_comment + if: needs.parse_comment.outputs.command == 'create-rc' + uses: ./.github/workflows/release_create_rc.yaml + with: + issue: ${{ needs.parse_comment.outputs.issue_number }} + comment_id: "${{ github.event.comment.id }}" + secrets: inherit + + call_prepare: + needs: parse_comment + if: needs.parse_comment.outputs.command == 'prepare' + uses: ./.github/workflows/release_prepare.yaml + with: + issue: ${{ needs.parse_comment.outputs.issue_number }} + secrets: inherit + + call_add_backports: + needs: parse_comment + if: | + needs.parse_comment.outputs.command == 'add-backports' || + needs.parse_comment.outputs.command == 'pr-backport' + uses: ./.github/workflows/release_add_backports.yaml + with: + prs: ${{ needs.parse_comment.outputs.command == 'pr-backport' && needs.parse_comment.outputs.pr_number || needs.parse_comment.outputs.backports }} + issue: ${{ needs.parse_comment.outputs.issue_number }} + secrets: inherit + + call_process_backports_after_add: + needs: [parse_comment, call_add_backports] + if: needs.parse_comment.outputs.command == 'add-backports' + uses: ./.github/workflows/release_process_backports.yaml + with: + issue: ${{ needs.parse_comment.outputs.issue_number }} + comment_id: "${{ github.event.comment.id }}" + secrets: inherit + + call_process_backports_only: + needs: parse_comment + if: needs.parse_comment.outputs.command == 'process-backports' + uses: ./.github/workflows/release_process_backports.yaml + with: + issue: ${{ needs.parse_comment.outputs.issue_number }} + comment_id: "${{ github.event.comment.id }}" + secrets: inherit + + call_promote: + needs: parse_comment + if: needs.parse_comment.outputs.command == 'promote' + uses: ./.github/workflows/release_promote_rc.yaml + with: + issue: ${{ needs.parse_comment.outputs.issue_number }} + secrets: inherit \ No newline at end of file diff --git a/.github/workflows/on_issue_comment.yaml b/.github/workflows/on_issue_comment.yaml deleted file mode 100644 index 14248d0d7a..0000000000 --- a/.github/workflows/on_issue_comment.yaml +++ /dev/null @@ -1,105 +0,0 @@ -name: "On Issue Comment" - -on: - issue_comment: - types: [created] - -permissions: - contents: write - issues: write - pull-requests: write - -jobs: - # This job always runs to prevent GHA from marking the run as failed when - # all other jobs are skipped. - noop: - runs-on: ubuntu-latest - steps: - - run: echo "No-op" - - parse_comment: - runs-on: ubuntu-latest - if: | - github.event.issue.pull_request == null && - contains(github.event.issue.labels.*.name, 'type: release') && - (github.event.comment.author_association == 'OWNER' || - github.event.comment.author_association == 'MEMBER' || - github.event.comment.author_association == 'COLLABORATOR') - outputs: - command: ${{ steps.parse.outputs.command }} - issue_number: ${{ github.event.issue.number }} - backports: ${{ steps.parse.outputs.backports }} - steps: - - name: Parse comment - id: parse - env: - COMMENT_BODY: ${{ github.event.comment.body }} - GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} - run: | - if echo "$COMMENT_BODY" | grep -qE '^[[:space:]]*/create-rc([[:space:]]|$)'; then - echo "command=create-rc" >> "$GITHUB_OUTPUT" - elif echo "$COMMENT_BODY" | grep -qE '^[[:space:]]*/prepare([[:space:]]|$)'; then - echo "command=prepare" >> "$GITHUB_OUTPUT" - elif echo "$COMMENT_BODY" | grep -qE '^[[:space:]]*/process-backports([[:space:]]|$)'; then - echo "command=process-backports" >> "$GITHUB_OUTPUT" - elif echo "$COMMENT_BODY" | grep -qE '^[[:space:]]*/add-backports([[:space:]]|$)'; then - args=$(echo "$COMMENT_BODY" | grep -E '^[[:space:]]*/add-backports([[:space:]]|$)' | sed -E 's/^[[:space:]]*\/add-backports[[:space:]]*//') - # Strip leading/trailing spaces and commas - args=$(echo "$args" | sed -e 's/^[[:space:],]*//' -e 's/[[:space:],]*$//') - # Replace internal spaces/commas with single comma - csv=$(echo "$args" | sed -E 's/[[:space:],]+/ /g' | tr ' ' ',') - if [ -n "$csv" ]; then - echo "command=add-backports" >> "$GITHUB_OUTPUT" - echo "backports=$csv" >> "$GITHUB_OUTPUT" - else - echo "command=none" >> "$GITHUB_OUTPUT" - echo "Error: No PRs specified for add-backports." >&2 - gh api \ - --method POST \ - -H "Accept: application/vnd.github+json" \ - -H "X-GitHub-Api-Version: 2022-11-28" \ - /repos/${{ github.repository }}/issues/comments/${{ github.event.comment.id }}/reactions \ - -f "content=-1" - fi - elif echo "$COMMENT_BODY" | grep -qE '^[[:space:]]*/promote([[:space:]]|$)'; then - echo "command=promote" >> "$GITHUB_OUTPUT" - else - echo "command=none" >> "$GITHUB_OUTPUT" - fi - - call_create_rc: - needs: parse_comment - if: needs.parse_comment.outputs.command == 'create-rc' - uses: ./.github/workflows/release_create_rc.yaml - with: - issue: ${{ needs.parse_comment.outputs.issue_number }} - comment_id: "${{ github.event.comment.id }}" - secrets: inherit - - call_prepare: - needs: parse_comment - if: needs.parse_comment.outputs.command == 'prepare' - uses: ./.github/workflows/release_prepare.yaml - with: - issue: ${{ needs.parse_comment.outputs.issue_number }} - secrets: inherit - - call_process_backports: - needs: parse_comment - if: | - needs.parse_comment.outputs.command == 'process-backports' || - needs.parse_comment.outputs.command == 'add-backports' - uses: ./.github/workflows/release_process_backports.yaml - with: - issue: ${{ needs.parse_comment.outputs.issue_number }} - add_backports: ${{ needs.parse_comment.outputs.command == 'add-backports' && needs.parse_comment.outputs.backports || '' }} - comment_id: "${{ github.event.comment.id }}" - secrets: inherit - - call_promote: - needs: parse_comment - if: needs.parse_comment.outputs.command == 'promote' - uses: ./.github/workflows/release_promote_rc.yaml - with: - issue: ${{ needs.parse_comment.outputs.issue_number }} - secrets: inherit diff --git a/.github/workflows/on_pr_closed.yaml b/.github/workflows/on_pr_closed.yaml new file mode 100644 index 0000000000..5dfcb89748 --- /dev/null +++ b/.github/workflows/on_pr_closed.yaml @@ -0,0 +1,83 @@ +name: "On PR Closed" + +on: + pull_request: + types: [closed] + +permissions: + contents: read + issues: read + pull-requests: read + +jobs: + # This job always runs to prevent GHA from marking the run as failed when + # all other jobs are skipped. + noop: + runs-on: ubuntu-latest + steps: + - run: echo "No-op" + + check_if_backport: + runs-on: ubuntu-latest + if: github.event.pull_request.merged == true + outputs: + should_process: ${{ steps.check.outputs.should_process }} + steps: + - name: Check if PR is a backport candidate + id: check + env: + GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} + PR_NUMBER: ${{ github.event.pull_request.number }} + run: | + # Check if there is any active release issue + ACTIVE_ISSUES=$(gh issue list --repo ${{ github.repository }} --label "type: release" --state open --json number) + if [ "$ACTIVE_ISSUES" = "[]" ] || [ -z "$ACTIVE_ISSUES" ]; then + echo "No active release tracking issue found. Skipping." + echo "should_process=false" >> "$GITHUB_OUTPUT" + exit 0 + fi + + # Check if PR has "/backport" in comments (only comments, not body) + PR_DATA=$(gh pr view "$PR_NUMBER" --repo ${{ github.repository }} --json comments) + + if echo "$PR_DATA" | jq -r '.comments[].body' | grep -qE '^[[:space:]]*/backport([[:space:]]|$)'; then + echo "Found /backport comment. Proceeding." + echo "should_process=true" >> "$GITHUB_OUTPUT" + else + echo "No /backport comment found. Skipping." + echo "should_process=false" >> "$GITHUB_OUTPUT" + fi + + process_backports: + needs: check_if_backport + if: needs.check_if_backport.outputs.should_process == 'true' + runs-on: ubuntu-latest + permissions: + contents: write + issues: write + pull-requests: read + steps: + - name: Checkout repository + uses: actions/checkout@v7 + with: + fetch-depth: 0 + + - name: Setup Bazel + uses: bazel-contrib/setup-bazel@0.19.0 + with: + bazelisk-version: 1.20.0 + + - name: Configure Git Identity + run: | + git config --global user.name "github-actions[bot]" + git config --global user.email "41898282+github-actions[bot]@users.noreply.github.com" + + - name: Process Backports + env: + GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} + PR_NUMBER: ${{ github.event.pull_request.number }} + run: | + bazel run //tools/private/release -- on-pr-merged \ + "$PR_NUMBER" \ + --remote origin \ + --no-dry-run diff --git a/.github/workflows/release_add_backports.yaml b/.github/workflows/release_add_backports.yaml new file mode 100644 index 0000000000..d9d6d84e34 --- /dev/null +++ b/.github/workflows/release_add_backports.yaml @@ -0,0 +1,58 @@ +name: "Release: Add Backports" + +on: + workflow_dispatch: + inputs: + prs: + description: 'CSV list of PR numbers to add (e.g., 123,456)' + required: true + type: string + issue: + description: 'The Release Tracking Issue Number (optional)' + required: false + type: string + workflow_call: + inputs: + prs: + description: 'CSV list of PR numbers to add (e.g., 123,456)' + required: true + type: string + issue: + description: 'The Release Tracking Issue Number (optional)' + required: false + type: string + +permissions: + issues: write + +jobs: + add_backports: + runs-on: ubuntu-latest + steps: + - name: Checkout repository + uses: actions/checkout@v7 + with: + fetch-depth: 0 + + - name: Setup Bazel + uses: bazel-contrib/setup-bazel@0.19.0 + with: + bazelisk-version: 1.20.0 + + - name: Add Backports to Tracking Issue + env: + GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} + PRS: ${{ inputs.prs }} + ISSUE: ${{ inputs.issue }} + run: | + ARGS=() + if [ -n "$ISSUE" ]; then + ARGS+=("--issue=$ISSUE") + fi + + # Convert CSV to array + IFS=',' read -r -a pr_array <<< "$PRS" + + bazel run //tools/private/release -- add-backports \ + "${pr_array[@]}" \ + "${ARGS[@]}" diff --git a/RELEASING.md b/RELEASING.md index 7846cef883..2298b4c8a3 100644 --- a/RELEASING.md +++ b/RELEASING.md @@ -79,17 +79,41 @@ being accumulated for the next release, review the pending news entries in the To add backports to an active release, you can use one of the following methods: -### Method A: Manual Checklist Update +### Method A: Comment on the PR + +Comment `/backport` on the PR you wish to backport. This will automatically +add the PR to the active release's backports checklist. Once the PR is merged, +the backports will be automatically processed. + +> [!NOTE] +> Commenting `/backport` on an open PR will block further release publishing +> (like creating RCs or promoting) until the PR is merged or manually set to +> status=ignore in the checklist. + +### Method B: Comment on the Tracking Issue + +Comment `/add-backports [ ...]` (space or comma separated) on +the tracking issue. The `` can be a PR number (optionally prefixed with +`#`) or a PR URL (strictly for the configured repository). This will +automatically add the PRs to the checklist and trigger processing. + +### Method C: Manual Checklist Update 1. Manually add checklist items under the `## Backports` section of the Release Tracking Issue. The format must be: `- [ ] #` (e.g., `- [ ] #1234`). 2. When ready, comment `/process-backports` on the tracking issue to trigger processing. -### Method B: Comment Shortcut -1. Comment `/add-backports [ ...]` (space or comma - separated) on the tracking issue. This will automatically add the PRs to the - checklist and trigger processing. +### Method D: Release Tool CLI +You can use the release tool to add backports from your local checkout: +```shell +bazel run //tools/private/release -- add-backports [ ...] +``` +The `` can be: +* A PR number (e.g., `124` or `#124`) +* A PR URL (e.g., `https://github.com/bazel-contrib/rules_python/pull/124` + or `https://github.com/bazel-contrib/rules_python/pull/124/files`) +* Only URLs for the configured repository are accepted. ### Failure Behavior If a backport fails to process (e.g., due to cherry-pick conflicts): diff --git a/tests/tools/private/release/BUILD.bazel b/tests/tools/private/release/BUILD.bazel index e68bbef7d5..bc3b6f7e8b 100644 --- a/tests/tools/private/release/BUILD.bazel +++ b/tests/tools/private/release/BUILD.bazel @@ -8,6 +8,15 @@ py_library( ], ) +py_test( + name = "add_backports_test", + srcs = ["add_backports_test.py"], + deps = [ + ":release_test_helper", + "//tools/private/release:release_lib", + ], +) + py_test( name = "changelog_news_test", srcs = ["changelog_news_test.py"], @@ -35,6 +44,14 @@ py_test( ], ) +py_test( + name = "gh_test", + srcs = ["gh_test.py"], + deps = [ + "//tools/private/release:release_lib", + ], +) + py_test( name = "git_test", srcs = ["git_test.py"], @@ -43,6 +60,15 @@ py_test( ], ) +py_test( + name = "on_pr_merged_test", + srcs = ["on_pr_merged_test.py"], + deps = [ + ":release_test_helper", + "//tools/private/release:release_lib", + ], +) + py_test( name = "prepare_test", srcs = ["prepare_test.py"], diff --git a/tests/tools/private/release/add_backports_test.py b/tests/tools/private/release/add_backports_test.py new file mode 100644 index 0000000000..7a5d96a23b --- /dev/null +++ b/tests/tools/private/release/add_backports_test.py @@ -0,0 +1,104 @@ +import argparse +import unittest +from unittest.mock import patch + +from tests.tools.private.release.release_test_helper import _mock_git_and_gh +from tools.private.release.add_backports import AddBackports + + +class CmdAddBackportsTest(unittest.TestCase): + def setUp(self): + _mock_git_and_gh(self) + self.addCleanup(patch.stopall) + self.mock_gh.resolve_pr_number.side_effect = lambda x: int( + x.lstrip("#").split("/")[-1] + ) + + def test_add_backports_explicit_issue(self): + args = argparse.Namespace(issue=123, prs=["124", "125"]) + self.mock_gh.get_issue_body.return_value = """ +## Checklist +- [ ] Prepare Release +- [ ] Create Release branch +- [ ] Tag Final + +## Backports +""" + result = AddBackports(args, self.mock_gh).run() + + self.assertEqual(result, 0) + self.mock_gh.get_issue_body.assert_called_once_with(123) + self.mock_gh.update_issue_body.assert_called_once() + call_args = self.mock_gh.update_issue_body.call_args[0] + self.assertEqual(call_args[0], 123) + self.assertIn("- [ ] #124", call_args[1]) + self.assertIn("- [ ] #125", call_args[1]) + # Should also auto-add Tag RC0 + self.assertIn("- [ ] Tag RC0", call_args[1]) + + def test_add_backports_auto_discover_success(self): + args = argparse.Namespace(issue=None, prs=["124"]) + self.mock_gh.get_open_tracking_issues.return_value = [ + {"number": 456, "title": "Release 2.1.0", "url": "http://..."} + ] + self.mock_gh.get_issue_body.return_value = """ +## Checklist +- [ ] Prepare Release +- [ ] Create Release branch +- [ ] Tag Final + +## Backports +""" + result = AddBackports(args, self.mock_gh).run() + + self.assertEqual(result, 0) + self.mock_gh.get_open_tracking_issues.assert_called_once() + self.mock_gh.get_issue_body.assert_called_once_with(456) + self.mock_gh.update_issue_body.assert_called_once_with(456, unittest.mock.ANY) + + def test_add_backports_auto_discover_no_issues(self): + args = argparse.Namespace(issue=None, prs=["124"]) + self.mock_gh.get_open_tracking_issues.return_value = [] + + result = AddBackports(args, self.mock_gh).run() + + self.assertEqual(result, 1) + self.mock_gh.get_open_tracking_issues.assert_called_once() + self.mock_gh.get_issue_body.assert_not_called() + + def test_add_backports_auto_discover_multiple_issues(self): + args = argparse.Namespace(issue=None, prs=["124"]) + self.mock_gh.get_open_tracking_issues.return_value = [ + {"number": 456, "title": "Release 2.1.0", "url": "http://..."}, + {"number": 789, "title": "Release 2.2.0", "url": "http://..."}, + ] + + result = AddBackports(args, self.mock_gh).run() + + self.assertEqual(result, 1) + self.mock_gh.get_open_tracking_issues.assert_called_once() + self.mock_gh.get_issue_body.assert_not_called() + + def test_add_backports_no_auto_add_rc_if_pending(self): + args = argparse.Namespace(issue=123, prs=["124"]) + self.mock_gh.get_issue_body.return_value = """ +## Checklist +- [ ] Prepare Release +- [ ] Create Release branch +- [ ] Tag RC0 +- [ ] Tag Final + +## Backports +""" + result = AddBackports(args, self.mock_gh).run() + + self.assertEqual(result, 0) + self.mock_gh.update_issue_body.assert_called_once() + call_args = self.mock_gh.update_issue_body.call_args[0] + self.assertNotIn("Tag RC1", call_args[1]) + # Tag RC0 should still be there + self.assertIn("- [ ] Tag RC0", call_args[1]) + + +if __name__ == "__main__": + unittest.main() diff --git a/tests/tools/private/release/create_rc_test.py b/tests/tools/private/release/create_rc_test.py index 06cdb77345..8e4ed101e4 100644 --- a/tests/tools/private/release/create_rc_test.py +++ b/tests/tools/private/release/create_rc_test.py @@ -189,6 +189,30 @@ def test_create_rc_gating_on_backports(self): self.mock_git.tag.assert_not_called() self.mock_git.push.assert_not_called() + def test_create_rc_not_blocked_by_ignored_backports(self): + # Arrange + args = MagicMock(issue=123, remote="my-remote") + self.mock_gh.get_issue_title.return_value = "Release 2.0.0" + self.mock_gh.get_issue_body.return_value = """ +## Checklist +- [x] Prepare Release | status=done pr=#122 commit=abcdef12 +- [x] Create Release branch | status=done branch=release/2.0 commit=abcdef12 +- [ ] Tag RC0 | status=pending + +## Backports +- [ ] #124 | status=ignore +""" + self.mock_git.get_remote_tags.return_value = [] + self.mock_git.get_commit_sha.return_value = "1234567890" + + # Act + result = CreateRc(args, self.mock_git, self.mock_gh).run() + + # Assert + self.assertEqual(result, 0) + self.mock_git.tag.assert_called_once_with("2.0.0-rc0", "my-remote/release/2.0") + self.mock_git.push.assert_called_once_with("my-remote", "2.0.0-rc0") + def test_create_rc_with_finished_backports(self): # Arrange args = MagicMock(issue=123, remote="my-remote") diff --git a/tests/tools/private/release/gh_test.py b/tests/tools/private/release/gh_test.py new file mode 100644 index 0000000000..4010d2dca5 --- /dev/null +++ b/tests/tools/private/release/gh_test.py @@ -0,0 +1,61 @@ +import unittest +from unittest.mock import patch + +from tools.private.release.gh import GitHub + + +class GitHubTest(unittest.TestCase): + def setUp(self): + self.gh = GitHub("my-owner/my-repo") + + @patch("tools.private.release.gh.run_cmd") + def test_resolve_pr_number_digit(self, mock_run_cmd): + # 124 and #125 should resolve immediately without running command + self.assertEqual(self.gh.resolve_pr_number("124"), 124) + self.assertEqual(self.gh.resolve_pr_number("#125"), 125) + mock_run_cmd.assert_not_called() + + @patch("tools.private.release.gh.run_cmd") + def test_resolve_pr_number_url_simple(self, mock_run_cmd): + url = "https://github.com/my-owner/my-repo/pull/126" + # Should resolve via regex without calling gh + result = self.gh.resolve_pr_number(url) + self.assertEqual(result, 126) + mock_run_cmd.assert_not_called() + + @patch("tools.private.release.gh.run_cmd") + def test_resolve_pr_number_url_with_subpath(self, mock_run_cmd): + url = "https://github.com/my-owner/my-repo/pull/126/files" + # Should resolve via regex without calling gh + result = self.gh.resolve_pr_number(url) + self.assertEqual(result, 126) + mock_run_cmd.assert_not_called() + + @patch("tools.private.release.gh.run_cmd") + def test_resolve_pr_number_url_with_query(self, mock_run_cmd): + url = "https://github.com/my-owner/my-repo/pull/126/files?w=1" + # Should resolve via regex without calling gh + result = self.gh.resolve_pr_number(url) + self.assertEqual(result, 126) + mock_run_cmd.assert_not_called() + + @patch("tools.private.release.gh.run_cmd") + def test_resolve_pr_number_url_other_repo(self, mock_run_cmd): + # URL for a different repo should fail immediately without calling gh + url = "https://github.com/other-owner/other-repo/pull/126" + with self.assertRaises(ValueError) as ctx: + self.gh.resolve_pr_number(url) + self.assertIn("URL is not for the configured repository", str(ctx.exception)) + mock_run_cmd.assert_not_called() + + @patch("tools.private.release.gh.run_cmd") + def test_resolve_pr_number_invalid_ref(self, mock_run_cmd): + # Invalid reference (not number, not URL) should fail + with self.assertRaises(ValueError) as ctx: + self.gh.resolve_pr_number("invalid-ref") + self.assertIn("Could not resolve PR reference", str(ctx.exception)) + mock_run_cmd.assert_not_called() + + +if __name__ == "__main__": + unittest.main() diff --git a/tests/tools/private/release/on_pr_merged_test.py b/tests/tools/private/release/on_pr_merged_test.py new file mode 100644 index 0000000000..a5644c5705 --- /dev/null +++ b/tests/tools/private/release/on_pr_merged_test.py @@ -0,0 +1,112 @@ +import argparse +import unittest +from unittest.mock import MagicMock, patch + +from tests.tools.private.release.release_test_helper import _mock_git_and_gh +from tools.private.release.on_pr_merged import OnPrMerged + + +class CmdOnPrMergedTest(unittest.TestCase): + def setUp(self): + _mock_git_and_gh(self) + self.addCleanup(patch.stopall) + + # Mock ProcessBackports + self.mock_process_patcher = patch( + "tools.private.release.on_pr_merged.ProcessBackports" + ) + self.mock_process_class = self.mock_process_patcher.start() + self.mock_process_instance = MagicMock() + self.mock_process_class.return_value = self.mock_process_instance + + def test_on_pr_merged_no_comment(self): + args = argparse.Namespace(pr=124, remote="origin", dry_run=True) + self.mock_gh.get_pr_comments.return_value = [ + {"body": "Some comment"}, + {"body": "Another comment /backport_wrong"}, + ] + + result = OnPrMerged(args, self.mock_git, self.mock_gh).run() + + self.assertEqual(result, 1) + self.mock_gh.get_pr_comments.assert_called_once_with(124) + self.mock_gh.get_open_tracking_issues.assert_not_called() + self.mock_process_class.assert_not_called() + + def test_on_pr_merged_has_comment_no_active_release(self): + args = argparse.Namespace(pr=124, remote="origin", dry_run=True) + self.mock_gh.get_pr_comments.return_value = [ + {"body": "/backport"}, + ] + self.mock_gh.get_open_tracking_issues.return_value = [] + + result = OnPrMerged(args, self.mock_git, self.mock_gh).run() + + self.assertEqual(result, 1) + self.mock_gh.get_pr_comments.assert_called_once_with(124) + self.mock_gh.get_open_tracking_issues.assert_called_once() + self.mock_gh.get_issue_body.assert_not_called() + self.mock_process_class.assert_not_called() + + def test_on_pr_merged_has_comment_not_in_backports(self): + args = argparse.Namespace(pr=124, remote="origin", dry_run=True) + self.mock_gh.get_pr_comments.return_value = [ + {"body": " /backport "}, + ] + self.mock_gh.get_open_tracking_issues.return_value = [ + {"number": 456, "title": "Release 2.1.0", "url": "http://..."} + ] + self.mock_gh.get_issue_body.return_value = """ +## Checklist +- [ ] Prepare Release + +## Backports +- [ ] #125 | status=pending +""" + result = OnPrMerged(args, self.mock_git, self.mock_gh).run() + + self.assertEqual(result, 1) + self.mock_gh.get_pr_comments.assert_called_once_with(124) + self.mock_gh.get_open_tracking_issues.assert_called_once() + self.mock_gh.get_issue_body.assert_called_once_with(456) + self.mock_process_class.assert_not_called() + + def test_on_pr_merged_success(self): + args = argparse.Namespace(pr=124, remote="origin", dry_run=True) + self.mock_gh.get_pr_comments.return_value = [ + {"body": "/backport"}, + ] + self.mock_gh.get_open_tracking_issues.return_value = [ + {"number": 456, "title": "Release 2.1.0", "url": "http://..."} + ] + self.mock_gh.get_issue_body.return_value = """ +## Checklist +- [ ] Prepare Release + +## Backports +- [ ] #124 | status=pending +""" + self.mock_process_instance.run.return_value = 0 + + result = OnPrMerged(args, self.mock_git, self.mock_gh).run() + + self.assertEqual(result, 0) + self.mock_gh.get_pr_comments.assert_called_once_with(124) + self.mock_gh.get_open_tracking_issues.assert_called_once() + self.mock_gh.get_issue_body.assert_called_once_with(456) + + # Verify ProcessBackports was instantiated with correct args + self.mock_process_class.assert_called_once() + called_args = self.mock_process_class.call_args[0][0] + self.assertEqual(called_args.issue, 456) + self.assertEqual(called_args.remote, "origin") + self.assertEqual(called_args.dry_run, True) + self.assertIsNone(called_args.add) + self.assertIsNone(called_args.triggering_comment) + + # Verify ProcessBackports.run() was called + self.mock_process_instance.run.assert_called_once() + + +if __name__ == "__main__": + unittest.main() diff --git a/tests/tools/private/release/process_backports_test.py b/tests/tools/private/release/process_backports_test.py index 639171f59f..fe7bae256d 100644 --- a/tests/tools/private/release/process_backports_test.py +++ b/tests/tools/private/release/process_backports_test.py @@ -17,6 +17,9 @@ def setUp(self): "tools.private.release.process_backports.replace_version_next" ).start() self.addCleanup(patch.stopall) + self.mock_gh.resolve_pr_number.side_effect = lambda x: int( + x.lstrip("#").split("/")[-1] + ) def test_process_backports_no_pending(self): args = argparse.Namespace( @@ -254,7 +257,7 @@ def test_process_backports_add_backports_and_auto_add_rc_task(self, mock_datetim issue=123, remote="origin", dry_run=False, - add=[124], + add=["https://github.com/bazel-contrib/rules_python/pull/124"], triggering_comment=None, ) self.mock_gh.get_issue_title.return_value = "Release 2.0.0" @@ -305,6 +308,51 @@ def mock_resolve(items): self.assertEqual(call2_args[0], 123) self.assertIn("- [x] #124 | status=done rc=rc1 commit= 12345678", call2_args[1]) + @patch("tools.private.release.process_backports.datetime") + def test_process_backports_add_backports_marks_invalid(self, mock_datetime): + mock_datetime.date.today.return_value = datetime.date(2026, 7, 1) + args = argparse.Namespace( + issue=123, + remote="origin", + dry_run=False, + add=["124", "invalid", "125"], + triggering_comment=None, + ) + self.mock_gh.get_issue_title.return_value = "Release 2.0.0" + self.mock_gh.get_issue_body.return_value = """ +## Checklist +- [x] Prepare Release | status=done pr=#122 commit=abcdef12 +- [x] Create Release branch | status=done branch=release/2.0 commit=abcdef12 +- [x] Tag RC0 | status=done tag=2.0.0-rc0 commit=abcdef12 +- [ ] Tag Final + +## Backports +""" + self.mock_git.get_remote_tags.return_value = ["2.0.0-rc0"] + self.mock_git.get_commit_sha.return_value = "1234567890" + self.mock_git.get_commit_message.return_value = 'Cherry-pick "fix bug"' + + def mock_resolve(items): + # Both 124 and 125 should be processed, 'invalid' should be ignored (it has error status) + for item in items: + if item.pr_ref in ("#124", "#125"): + item.commit = "abcdef12" + item.status = "done" + return items + + self.mock_gh.get_merge_commits_for_prs.side_effect = mock_resolve + self.mock_git.sort_commits_chronologically.return_value = ["abcdef12"] + + result = ProcessBackports(args, self.mock_git, self.mock_gh).run() + + self.assertEqual(result, 0) + # Should have updated body to add 124, 125, and invalid + self.assertEqual(self.mock_gh.update_issue_body.call_count, 2) + call1_args = self.mock_gh.update_issue_body.call_args_list[0][0] + self.assertIn("- [ ] #124", call1_args[1]) + self.assertIn("- [ ] #125", call1_args[1]) + self.assertIn("- [ ] invalid | status=error-invalid-pr", call1_args[1]) + if __name__ == "__main__": unittest.main() diff --git a/tests/tools/private/release/release_issue_test.py b/tests/tools/private/release/release_issue_test.py index edf5f603f0..cfc6da6ae4 100644 --- a/tests/tools/private/release/release_issue_test.py +++ b/tests/tools/private/release/release_issue_test.py @@ -1,6 +1,7 @@ import unittest from tools.private.release.release_issue import ( + add_backports_to_body, format_metadata_line, parse_metadata_line, ) @@ -65,6 +66,36 @@ def test_format_metadata_line(self): expected = "- [ ] Tag Final" self.assertEqual(format_metadata_line(False, "Tag Final", {}), expected) + def test_add_backports_to_body(self): + body = """ +## Checklist +- [ ] Prepare Release +- [ ] Create Release branch +- [ ] Tag Final + +## Backports +- [ ] #123 | status=done +""" + items = [ + {"ref": "124"}, + {"ref": "#124"}, + {"ref": "125"}, + {"ref": "#123"}, + ] + updated_body = add_backports_to_body(body, items) + expected_body = """ +## Checklist +- [ ] Prepare Release +- [ ] Create Release branch +- [ ] Tag Final + +## Backports +- [ ] #123 | status=done +- [ ] #124 +- [ ] #125 +""" + self.assertEqual(updated_body.strip(), expected_body.strip()) + if __name__ == "__main__": unittest.main() diff --git a/tools/private/release/add_backports.py b/tools/private/release/add_backports.py new file mode 100644 index 0000000000..c25f509b9c --- /dev/null +++ b/tools/private/release/add_backports.py @@ -0,0 +1,115 @@ +"""Subcommand to add PRs to the release tracking issue backports checklist.""" + +from tools.private.release.gh import GitHub +from tools.private.release.release_issue import ( + add_backports_to_body, + add_rc_task_to_body, + parse_checklist_state, +) + + +class AddBackports: + """Class to add PRs to the release tracking issue.""" + + def __init__(self, args, gh: GitHub): + self.args = args + self.gh = gh + + def run(self) -> int: + """Executes the add-backports subcommand.""" + args = self.args + + issue_num = args.issue + if not issue_num: + print( + "No issue specified. Trying to auto-discover active release" + " tracking issue..." + ) + try: + open_issues = self.gh.get_open_tracking_issues() + if not open_issues: + print("Error: No open release tracking issues found.") + return 1 + if len(open_issues) > 1: + print( + "Error: Multiple open release tracking issues found." + " Cannot determine active one:" + ) + for issue in open_issues: + print(f"- #{issue['number']}: {issue['title']}") + return 1 + issue_num = open_issues[0]["number"] + print(f"Auto-discovered active release tracking issue: #{issue_num}") + except Exception as e: + print(f"Error auto-discovering tracking issue: {e}") + return 1 + + resolved_prs = [] + for pr_ref in args.prs: + try: + pr_num = self.gh.resolve_pr_number(pr_ref) + resolved_prs.append(pr_num) + except Exception as e: + print(f"Error resolving PR ref '{pr_ref}': {e}") + return 1 + + print( + f"Adding backports {resolved_prs} (resolved from {args.prs}) to tracking issue #{issue_num}..." + ) + try: + body = self.gh.get_issue_body(issue_num) + items_to_add = [{"ref": f"#{pr}"} for pr in resolved_prs] + body = add_backports_to_body(body, items_to_add) + state = parse_checklist_state(body) + rc_tags = state.get("rc_tags", {}) + has_pending_rc = any( + not task.checked and task.status != "done" for task in rc_tags.values() + ) + next_rc_num = max(rc_tags.keys()) + 1 if rc_tags else 0 + if not has_pending_rc: + print( + f"No pending RC task found. Adding 'Tag" + f" RC{next_rc_num}' to checklist..." + ) + body = add_rc_task_to_body(body, next_rc_num) + except ValueError as e: + print(f"Error: {e}") + return 1 + except Exception as e: + print(f"Failed to update tracking issue: {e}") + return 1 + + try: + self.gh.update_issue_body(issue_num, body) + print("Successfully updated tracking issue checklist.") + except Exception as e: + print(f"Failed to update tracking issue body: {e}") + return 1 + + return 0 + + @classmethod + def add_parser(cls, subparsers): + """Adds parser for add-backports subcommand.""" + parser = subparsers.add_parser( + "add-backports", + help="Add PRs to the release tracking issue backports checklist.", + ) + parser.add_argument( + "prs", + type=str, + nargs="+", + help="PR references (numbers, #numbers, or URLs) to add (positional, space-separated).", + ) + parser.add_argument( + "--issue", + type=int, + help="The tracking issue number. If omitted, will try to auto-discover the active release tracking issue.", + ) + parser.set_defaults(command=cls.run_from_args) + + @classmethod + def run_from_args(cls, args): + """Instantiates and runs the command from parsed args.""" + gh = GitHub() + return cls(args, gh).run() diff --git a/tools/private/release/complete_prepare.py b/tools/private/release/complete_prepare.py index 7657738aad..8a60cbf0f3 100644 --- a/tools/private/release/complete_prepare.py +++ b/tools/private/release/complete_prepare.py @@ -25,7 +25,7 @@ def run(self) -> int: return 1 # Resolve issue number from PR body - pr_body = pr_info.get("body", "") + pr_body = pr_info.get("body") or "" match = re.search(r"Work towards #(\d+)", pr_body) if not match: match = re.search(r"#(\d+)", pr_body) diff --git a/tools/private/release/create_rc.py b/tools/private/release/create_rc.py index 0916377925..2e61e7d80a 100644 --- a/tools/private/release/create_rc.py +++ b/tools/private/release/create_rc.py @@ -84,7 +84,10 @@ def _run_internal(self) -> int: # Gating: RC tagging is blocked if any backport is unchecked OR does not have status=done backports = parse_backports(body) conflicting_or_pending = [ - b for b in backports if not b.checked or b.status != "done" + b + for b in backports + if (b.checked and b.status != "done") + or (not b.checked and b.status != "ignore") ] if conflicting_or_pending: print( diff --git a/tools/private/release/gh.py b/tools/private/release/gh.py index e670fc98df..8990ef5abf 100644 --- a/tools/private/release/gh.py +++ b/tools/private/release/gh.py @@ -2,6 +2,7 @@ import json import os +import re import tempfile from tools.private.release.release_issue import BackportTask @@ -316,6 +317,53 @@ def get_pr_info(self, pr_num: int) -> dict: ) return json.loads(output) if output else {} + def get_pr_comments(self, pr_num: int) -> list[dict]: + """Gets comments for a PR. + + Args: + pr_num: The PR number. + + Returns: + A list of comments. + """ + output = self._gh_pr( + "view", + str(pr_num), + "--json=comments", + ) + return json.loads(output).get("comments") or [] + + def resolve_pr_number(self, pr_ref: str) -> int: + """Resolves a PR reference (number, #number, URL) to a PR number. + + Args: + pr_ref: The PR reference string. + + Returns: + The resolved PR number. + + Raises: + ValueError: If the reference cannot be resolved. + """ + # 1. Try number (e.g. "123" or "#123") + clean_ref = pr_ref.lstrip("#") + if clean_ref.isdigit(): + return int(clean_ref) + + # 2. Try URL (starts with http) + if pr_ref.startswith("http"): + # Try to extract PR number from URL using regex + # Pattern matches: github.com//pull/ followed by /, ?, or EOF + pattern = rf"github\.com/{re.escape(self.repo)}/pull/(\d+)(/|\?|\Z)" + match = re.search(pattern, pr_ref, re.IGNORECASE) + if match: + return int(match.group(1)) + raise ValueError( + f"URL is not for the configured repository ({self.repo}): {pr_ref}" + ) + + raise ValueError(f"Could not resolve PR reference: {pr_ref}") + def post_issue_comment(self, issue_num: int, comment_body: str) -> None: """Posts a comment to a specific issue. diff --git a/tools/private/release/on_pr_merged.py b/tools/private/release/on_pr_merged.py new file mode 100644 index 0000000000..8d451580fb --- /dev/null +++ b/tools/private/release/on_pr_merged.py @@ -0,0 +1,113 @@ +"""Subcommand to handle PR merge event by processing backports.""" + +import argparse +import re + +from tools.private.release.gh import GitHub +from tools.private.release.git import Git +from tools.private.release.process_backports import ProcessBackports +from tools.private.release.release_issue import parse_backports + + +class OnPrMerged: + """Class to handle PR merge event.""" + + def __init__(self, args, git: Git, gh: GitHub): + self.args = args + self.git = git + self.gh = gh + + def run(self) -> int: + """Executes the on-pr-merged subcommand.""" + args = self.args + pr_num = args.pr + pr_ref = f"#{pr_num}" + + print(f"Verifying PR {pr_ref} has backport comment...") + try: + comments = self.gh.get_pr_comments(pr_num) + has_comment = any( + re.match( + r"^\s*/backport(\s|$)", + comment.get("body") or "", + re.IGNORECASE, + ) + for comment in comments + ) + if not has_comment: + print(f"PR {pr_ref} does not have a /backport comment. Skipping.") + return 1 + except Exception as e: + print(f"Error checking PR comments: {e}") + return 1 + + print(f"Searching for active release tracking issue containing PR {pr_ref}...") + open_issues = self.gh.get_open_tracking_issues() + if not open_issues: + print("No open release tracking issues found.") + return 1 + + found_issue = None + for issue in open_issues: + issue_num = issue["number"] + body = self.gh.get_issue_body(issue_num) + backports = parse_backports(body) + + if any(item.pr_ref == pr_ref for item in backports): + if found_issue: + print( + f"Error: PR {pr_ref} found in multiple open release" + f" tracking issues: #{found_issue} and #{issue_num}" + ) + return 1 + found_issue = issue_num + + if not found_issue: + print(f"PR {pr_ref} not found in any active release tracking issue.") + return 1 + + print(f"Found PR {pr_ref} in tracking issue #{found_issue}") + + # Now run ProcessBackports for this issue + process_args = argparse.Namespace( + issue=found_issue, + remote=args.remote, + add=None, + triggering_comment=None, + dry_run=args.dry_run, + ) + print(f"Processing backports for issue #{found_issue}...") + return ProcessBackports(process_args, self.git, self.gh).run() + + @classmethod + def add_parser(cls, subparsers): + """Adds parser for on-pr-merged subcommand.""" + parser = subparsers.add_parser( + "on-pr-merged", + help="Handle PR merge event by processing backports.", + ) + parser.add_argument( + "pr", + type=int, + help="PR number that was merged.", + ) + parser.add_argument( + "--remote", + type=str, + required=True, + help="The git remote to push changes to (required).", + ) + parser.add_argument( + "--dry-run", + action=argparse.BooleanOptionalAction, + default=True, + help="Perform a dry run (default: True). Use --no-dry-run to actually execute.", + ) + parser.set_defaults(command=cls.run_from_args) + + @classmethod + def run_from_args(cls, args): + """Instantiates and runs the command from parsed args.""" + git = Git(".") + gh = GitHub() + return cls(args, git, gh).run() diff --git a/tools/private/release/process_backports.py b/tools/private/release/process_backports.py index 81dbe2bd5e..c8a936eab5 100644 --- a/tools/private/release/process_backports.py +++ b/tools/private/release/process_backports.py @@ -206,9 +206,23 @@ def _run_internal(self) -> int: body = self.gh.get_issue_body(args.issue) if args.add: - print(f"Adding backports {args.add} to tracking issue #{args.issue}...") + items_to_add = [] + for pr_ref in args.add: + try: + pr_num = self.gh.resolve_pr_number(pr_ref) + items_to_add.append({"ref": f"#{pr_num}"}) + except Exception as e: + print(f"Warning: PR ref '{pr_ref}' is invalid: {e}") + items_to_add.append( + { + "ref": pr_ref, + "metadata": {"status": "error-invalid-pr"}, + } + ) + + print(f"Adding backports {items_to_add} to tracking issue #{args.issue}...") try: - body = add_backports_to_body(body, args.add) + body = add_backports_to_body(body, items_to_add) state = parse_checklist_state(body) rc_tags = state.get("rc_tags", {}) has_pending_rc = any( @@ -356,7 +370,7 @@ def add_parser(cls, subparsers): parser.add_argument( "--add", type=parse_pr_list, - help="PR numbers (comma or space separated) to add before processing.", + help="PR references (numbers, #numbers, or URLs, comma/space separated) to add before processing.", ) parser.add_argument( "--triggering-comment", diff --git a/tools/private/release/release.py b/tools/private/release/release.py index a6f3543e71..6364c61b23 100644 --- a/tools/private/release/release.py +++ b/tools/private/release/release.py @@ -4,11 +4,13 @@ import os import sys +from tools.private.release.add_backports import AddBackports from tools.private.release.complete_prepare import CompletePrepare from tools.private.release.create_rc import CreateRc from tools.private.release.create_release_branch import CreateReleaseBranch from tools.private.release.create_release_issue import CreateReleaseIssue from tools.private.release.determine_next_version import DetermineNextVersion +from tools.private.release.on_pr_merged import OnPrMerged from tools.private.release.prepare import Prepare from tools.private.release.process_backports import ProcessBackports from tools.private.release.promote_rc import PromoteRc @@ -19,7 +21,9 @@ Prepare, CompletePrepare, CreateReleaseBranch, + AddBackports, ProcessBackports, + OnPrMerged, CreateRc, PromoteRc, ] diff --git a/tools/private/release/release_issue.py b/tools/private/release/release_issue.py index 2650b6607c..359249d11c 100644 --- a/tools/private/release/release_issue.py +++ b/tools/private/release/release_issue.py @@ -247,8 +247,14 @@ def parse_backports(body): return items -def add_backports_to_body(body: str, prs: list[int]) -> str: - """Adds new backport checklist items to the ## Backports section.""" +def add_backports_to_body(body: str, items: list[dict]) -> str: + """Adds new backport checklist items to the ## Backports section. + + Args: + body: The issue body. + items: A list of dicts, where each dict has a 'ref' key (str) and + optional 'metadata' key (dict). + """ body = body.replace("\r\n", "\n") # Find the Backports section pattern = r"(## Backports\n)(.*?)(?=\n##|\n---|\Z)" @@ -260,18 +266,24 @@ def add_backports_to_body(body: str, prs: list[int]) -> str: # Parse existing backports to avoid duplicates existing_items = parse_backports(body) - existing_prs = { - int(item.pr_ref.lstrip("#")) - for item in existing_items - if item.pr_ref.startswith("#") - } + existing_refs = {item.pr_ref for item in existing_items} new_lines = [] - for pr in prs: - if pr in existing_prs: - print(f"PR #{pr} is already in the backports list. Skipping.") + for item in items: + ref = item["ref"] + # Normalize numeric refs to #numeric + if not ref.startswith("#") and ref.isdigit(): + ref = f"#{ref}" + + if ref in existing_refs: + print(f"PR {ref} is already in the backports list. Skipping.") continue - new_lines.append(f"- [ ] #{pr}") + existing_refs.add(ref) + + metadata = item.get("metadata", {}) + new_lines.append( + format_metadata_line(checked=False, name=ref, metadata=metadata) + ) if not new_lines: return body diff --git a/tools/private/release/utils.py b/tools/private/release/utils.py index db7b96a378..cdd1b7724d 100644 --- a/tools/private/release/utils.py +++ b/tools/private/release/utils.py @@ -174,23 +174,12 @@ def replace_version_next(version): f.write(new_content) -def parse_pr_list(value: str) -> list[int]: - """Parses a comma or space separated list of PR numbers. +def parse_pr_list(value: str) -> list[str]: + """Parses a comma or space separated list of PR references. - PR numbers can optionally be prefixed with '#'. + PR references can be numbers (optionally prefixed with '#') or URLs. """ if not value: return [] # Split by space and/or comma - pr_strings = [p for p in re.split(r"[\s,]+", value.strip()) if p] - prs = [] - for pr_str in pr_strings: - clean_pr_str = pr_str.lstrip("#") - try: - prs.append(int(clean_pr_str)) - except ValueError as e: - raise argparse.ArgumentTypeError( - f"Invalid PR reference '{pr_str}'. Must be integer optionally" - f" prefixed with '#'." - ) from e - return prs + return [p for p in re.split(r"[\s,]+", value.strip()) if p] From 0896b4791e3d9e8785890a3798cfd8b5faf90e16 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Fri, 3 Jul 2026 21:01:10 -0700 Subject: [PATCH 819/922] chore(release): fix promote_rc arguments and align comments (#3898) chore(release): fix promote_rc arguments and align comments `promote_rc.py` was calling `Git.fetch` with `refspec` keyword argument which was not supported by `Git.fetch` signature, leading to fatal errors. The release promotion comment format was different from the release candidate creation comment format, leading to inconsistency. - Added `refspec` argument support to `Git.fetch` in `git.py`. - Added `sys.argv` printing at startup in `release.py` for debugging. - Updated `promote_rc.py` comment body to match `create_rc.py` style and added necessary helper variables. - Updated `git_test.py` to test `Git.fetch` with `refspec`. - Updated `promote_rc_test.py` to match the new comment format. --- tests/tools/private/release/git_test.py | 38 +++++++++++++++++++ .../tools/private/release/promote_rc_test.py | 27 ++++++++----- tools/private/release/git.py | 9 ++++- tools/private/release/promote_rc.py | 24 +++++++++--- tools/private/release/release.py | 1 + 5 files changed, 84 insertions(+), 15 deletions(-) diff --git a/tests/tools/private/release/git_test.py b/tests/tools/private/release/git_test.py index 59b5c31989..16667b8715 100644 --- a/tests/tools/private/release/git_test.py +++ b/tests/tools/private/release/git_test.py @@ -44,5 +44,43 @@ def test_checkout_track_remote_existing_branch( mock_reset_hard.assert_called_once_with("origin/my-branch") +class GitFetchTest(unittest.TestCase): + def setUp(self): + self.git = Git(".") + self.patcher = patch.object(self.git, "_run_git") + self.mock_run_git = self.patcher.start() + self.addCleanup(self.patcher.stop) + + def test_fetch_default(self): + self.git.fetch() + self.mock_run_git.assert_called_once_with( + "fetch", "origin", capture_output=False + ) + + def test_fetch_custom_remote(self): + self.git.fetch("upstream") + self.mock_run_git.assert_called_once_with( + "fetch", "upstream", capture_output=False + ) + + def test_fetch_with_refspec(self): + self.git.fetch("origin", refspec="my-branch") + self.mock_run_git.assert_called_once_with( + "fetch", "origin", "my-branch", capture_output=False + ) + + def test_fetch_with_tags_and_force(self): + self.git.fetch("origin", tags=True, force=True) + self.mock_run_git.assert_called_once_with( + "fetch", "origin", "--tags", "--force", capture_output=False + ) + + def test_fetch_all_options(self): + self.git.fetch("origin", refspec="my-branch", tags=True, force=True) + self.mock_run_git.assert_called_once_with( + "fetch", "origin", "my-branch", "--tags", "--force", capture_output=False + ) + + if __name__ == "__main__": unittest.main() diff --git a/tests/tools/private/release/promote_rc_test.py b/tests/tools/private/release/promote_rc_test.py index 68797e2e1e..f6fa2da970 100644 --- a/tests/tools/private/release/promote_rc_test.py +++ b/tests/tools/private/release/promote_rc_test.py @@ -50,9 +50,12 @@ def test_promote_rc_success(self): 123, expected_updated_body ) expected_comment = ( - "Version 2.0.0 has been tagged.\n\n" - "- **Release Page**: https://github.com/bazel-contrib/rules_python/releases/tag/2.0.0\n" - '- **BCR PR Search**: [is:pr ("bazel-contrib/rules_python" in:title) ("@2.0.0" in:title)](https://github.com/bazelbuild/bazel-central-registry/pulls?q=is%3Apr%20%28%22bazel-contrib/rules_python%22%20in%3Atitle%29%20%28%22%402.0.0%22%20in%3Atitle%29)' + "**New Release Tagged!** 🐍🌿\n\n" + "Version **2.0.0** has been successfully generated and tagged on branch [`release/2.0`](https://github.com/bazel-contrib/rules_python/tree/release/2.0).\n\n" + "- [Github Release 2.0.0](https://github.com/bazel-contrib/rules_python/releases/tag/2.0.0)\n" + "- [BCR Entry 2.0.0](https://registry.bazel.build/modules/rules_python/2.0.0)\n" + "- [BCR PRs](https://github.com/bazelbuild/bazel-central-registry/pulls?q=is%3Apr%20%28%22bazel-contrib/rules_python%22%20in%3Atitle%29%20%28%22%402.0.0%22%20in%3Atitle%29)\n" + "- [Release workflow status](https://github.com/bazel-contrib/rules_python/actions/workflows/release_promote_rc.yaml)" ) self.mock_gh.post_issue_comment.assert_called_once_with(123, expected_comment) @@ -95,9 +98,12 @@ def test_promote_rc_resolve_issue_success(self): 123, expected_updated_body ) expected_comment = ( - "Version 2.0.0 has been tagged.\n\n" - "- **Release Page**: https://github.com/bazel-contrib/rules_python/releases/tag/2.0.0\n" - '- **BCR PR Search**: [is:pr ("bazel-contrib/rules_python" in:title) ("@2.0.0" in:title)](https://github.com/bazelbuild/bazel-central-registry/pulls?q=is%3Apr%20%28%22bazel-contrib/rules_python%22%20in%3Atitle%29%20%28%22%402.0.0%22%20in%3Atitle%29)' + "**New Release Tagged!** 🐍🌿\n\n" + "Version **2.0.0** has been successfully generated and tagged on branch [`release/2.0`](https://github.com/bazel-contrib/rules_python/tree/release/2.0).\n\n" + "- [Github Release 2.0.0](https://github.com/bazel-contrib/rules_python/releases/tag/2.0.0)\n" + "- [BCR Entry 2.0.0](https://registry.bazel.build/modules/rules_python/2.0.0)\n" + "- [BCR PRs](https://github.com/bazelbuild/bazel-central-registry/pulls?q=is%3Apr%20%28%22bazel-contrib/rules_python%22%20in%3Atitle%29%20%28%22%402.0.0%22%20in%3Atitle%29)\n" + "- [Release workflow status](https://github.com/bazel-contrib/rules_python/actions/workflows/release_promote_rc.yaml)" ) self.mock_gh.post_issue_comment.assert_called_once_with(123, expected_comment) @@ -142,9 +148,12 @@ def test_promote_rc_resolves_version_from_issue(self): 123, expected_updated_body ) expected_comment = ( - "Version 2.0.1 has been tagged.\n\n" - "- **Release Page**: https://github.com/bazel-contrib/rules_python/releases/tag/2.0.1\n" - '- **BCR PR Search**: [is:pr ("bazel-contrib/rules_python" in:title) ("@2.0.1" in:title)](https://github.com/bazelbuild/bazel-central-registry/pulls?q=is%3Apr%20%28%22bazel-contrib/rules_python%22%20in%3Atitle%29%20%28%22%402.0.1%22%20in%3Atitle%29)' + "**New Release Tagged!** 🐍🌿\n\n" + "Version **2.0.1** has been successfully generated and tagged on branch [`release/2.0`](https://github.com/bazel-contrib/rules_python/tree/release/2.0).\n\n" + "- [Github Release 2.0.1](https://github.com/bazel-contrib/rules_python/releases/tag/2.0.1)\n" + "- [BCR Entry 2.0.1](https://registry.bazel.build/modules/rules_python/2.0.1)\n" + "- [BCR PRs](https://github.com/bazelbuild/bazel-central-registry/pulls?q=is%3Apr%20%28%22bazel-contrib/rules_python%22%20in%3Atitle%29%20%28%22%402.0.1%22%20in%3Atitle%29)\n" + "- [Release workflow status](https://github.com/bazel-contrib/rules_python/actions/workflows/release_promote_rc.yaml)" ) self.mock_gh.post_issue_comment.assert_called_once_with(123, expected_comment) diff --git a/tools/private/release/git.py b/tools/private/release/git.py index 69edba3565..3e7f77f39a 100644 --- a/tools/private/release/git.py +++ b/tools/private/release/git.py @@ -135,16 +135,23 @@ def push( self._run_git(*cmd, capture_output=False) def fetch( - self, remote: str = "origin", tags: bool = False, force: bool = False + self, + remote: str = "origin", + refspec: str | None = None, + tags: bool = False, + force: bool = False, ) -> None: """Fetches updates from a remote repository. Args: remote: The remote repository name. Defaults to 'origin'. + refspec: The refspec to fetch. tags: If True, fetches all tags. force: If True, force fetches updates. """ cmd = ["fetch", remote] + if refspec: + cmd.append(refspec) if tags: cmd.append("--tags") if force: diff --git a/tools/private/release/promote_rc.py b/tools/private/release/promote_rc.py index ac3a841526..4a261c3b84 100644 --- a/tools/private/release/promote_rc.py +++ b/tools/private/release/promote_rc.py @@ -1,6 +1,7 @@ """Subcommand to promote a release candidate to final release.""" import argparse +import os import urllib.parse from tools.private.release.gh import GitHub @@ -161,16 +162,29 @@ def run(self) -> int: print(f"Posting comment to tracking issue #{issue_num}...") + branch_url = f"{REPO_URL}/tree/{branch_name}" release_url = f"{REPO_URL}/releases/tag/{version}" + bcr_entry_url = f"https://registry.bazel.build/modules/rules_python/{version}" bcr_query = ( f'is:pr ("bazel-contrib/rules_python" in:title) ("@{version}" in:title)' ) bcr_search_url = f"https://github.com/bazelbuild/bazel-central-registry/pulls?q={urllib.parse.quote(bcr_query)}" - comment_body = ( - f"Version {version} has been tagged.\n\n" - f"- **Release Page**: {release_url}\n" - f"- **BCR PR Search**: [{bcr_query}]({bcr_search_url})" - ) + + if run_id := os.environ.get("GITHUB_RUN_ID"): + release_workflow_url = f"{REPO_URL}/actions/runs/{run_id}" + else: + release_workflow_url = ( + f"{REPO_URL}/actions/workflows/release_promote_rc.yaml" + ) + + comment_body = f"""**New Release Tagged!** 🐍🌿 + +Version **{version}** has been successfully generated and tagged on branch [`{branch_name}`]({branch_url}). + +- [Github Release {version}]({release_url}) +- [BCR Entry {version}]({bcr_entry_url}) +- [BCR PRs]({bcr_search_url}) +- [Release workflow status]({release_workflow_url})""" self.gh.post_issue_comment(issue_num, comment_body) return 0 diff --git a/tools/private/release/release.py b/tools/private/release/release.py index 6364c61b23..8ad9c2e33e 100644 --- a/tools/private/release/release.py +++ b/tools/private/release/release.py @@ -46,6 +46,7 @@ def create_parser(): def main(): + print(f"sys.argv: {sys.argv}") if "BUILD_WORKSPACE_DIRECTORY" in os.environ: os.chdir(os.environ["BUILD_WORKSPACE_DIRECTORY"]) From 5650cc8966ca42ed5dea1ca0c1b3e0c5549071af Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Fri, 3 Jul 2026 21:14:00 -0700 Subject: [PATCH 820/922] chore(release): fix promote-rc command and update workflow args (#3899) Currently, the `promote-rc` command fails because it calls `Git.get_commit_sha` with an invalid keyword argument `remote_ref`. Additionally, we want to align argument passing in the workflow with the `=` style. To fix this: - Remove the invalid `remote_ref` keyword argument from `Git.get_commit_sha` call in `promote_rc.py`. - Update `promote_rc_test.py` mock assertions to match the corrected call. - Update `release_promote_rc.yaml` to pass `--issue` and `--remote` arguments using `=`. --- .github/workflows/release_promote_rc.yaml | 4 ++-- tests/tools/private/release/promote_rc_test.py | 8 ++++---- tools/private/release/promote_rc.py | 2 +- 3 files changed, 7 insertions(+), 7 deletions(-) diff --git a/.github/workflows/release_promote_rc.yaml b/.github/workflows/release_promote_rc.yaml index 7dc1c118b2..6dd02f1a25 100644 --- a/.github/workflows/release_promote_rc.yaml +++ b/.github/workflows/release_promote_rc.yaml @@ -52,9 +52,9 @@ jobs: ARGS+=("$VERSION") fi if [ -n "$ISSUE" ]; then - ARGS+=("--issue" "$ISSUE") + ARGS+=("--issue=$ISSUE") fi - ARGS+=("--remote" "origin") + ARGS+=("--remote=origin") ARGS+=("--no-dry-run") bazel run //tools/private/release -- promote-rc "${ARGS[@]}" diff --git a/tests/tools/private/release/promote_rc_test.py b/tests/tools/private/release/promote_rc_test.py index f6fa2da970..5bfd122c4c 100644 --- a/tests/tools/private/release/promote_rc_test.py +++ b/tests/tools/private/release/promote_rc_test.py @@ -34,7 +34,7 @@ def test_promote_rc_success(self): ] ) self.mock_git.get_commit_sha.assert_has_calls( - [call("2.0.0-rc1"), call(remote_ref="my-remote/release/2.0")] + [call("2.0.0-rc1"), call("my-remote/release/2.0")] ) self.mock_git.checkout.assert_not_called() self.mock_git.tag_exists.assert_called_once_with("2.0.0") @@ -85,7 +85,7 @@ def test_promote_rc_resolve_issue_success(self): ) self.mock_gh.get_release_tracking_issue.assert_called_once_with("2.0.0") self.mock_git.get_commit_sha.assert_has_calls( - [call("2.0.0-rc1"), call(remote_ref="my-remote/release/2.0")] + [call("2.0.0-rc1"), call("my-remote/release/2.0")] ) self.mock_git.checkout.assert_not_called() self.mock_git.tag.assert_called_once_with("2.0.0", "abcdef123456") @@ -136,7 +136,7 @@ def test_promote_rc_resolves_version_from_issue(self): self.mock_git.checkout.assert_not_called() self.mock_git.get_commit_sha.assert_has_calls( - [call("2.0.1-rc0"), call(remote_ref="my-remote/release/2.0")] + [call("2.0.1-rc0"), call("my-remote/release/2.0")] ) self.mock_git.tag.assert_called_once_with("2.0.1", "12345678") self.mock_git.push.assert_called_once_with("my-remote", "2.0.1") @@ -181,7 +181,7 @@ def test_promote_rc_dry_run_success(self, mock_print): ] ) self.mock_git.get_commit_sha.assert_has_calls( - [call("2.0.0-rc1"), call(remote_ref="my-remote/release/2.0")] + [call("2.0.0-rc1"), call("my-remote/release/2.0")] ) self.mock_git.tag_exists.assert_called_once_with("2.0.0") diff --git a/tools/private/release/promote_rc.py b/tools/private/release/promote_rc.py index 4a261c3b84..ed59f0b76e 100644 --- a/tools/private/release/promote_rc.py +++ b/tools/private/release/promote_rc.py @@ -89,7 +89,7 @@ def run(self) -> int: print(f"Fetching remote branch {remote_branch}...") self.git.fetch(args.remote, refspec=branch_name) try: - branch_sha = self.git.get_commit_sha(remote_ref=remote_branch) + branch_sha = self.git.get_commit_sha(remote_branch) except Exception as e: print( f"Error: Could not get commit SHA for remote branch" From dcbdf1b36306c7aa73feac5dc5e42c2eab20919a Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Fri, 3 Jul 2026 23:33:38 -0700 Subject: [PATCH 821/922] chore(release): make promote_rc workflow call publish_release workflow to finish publishing (#3900) Why: The tagging done by promote_rc doesn't trigger the on-tag trigger of the publish workflow (due to GITHUB_TOKEN restriction). How: - Modified promote_rc.py to output the promoted version to GITHUB_OUTPUT. - Modified release_promote_rc.yaml to capture this version and chain the release_publish workflow. - Added unit tests for GITHUB_OUTPUT writing in promote_rc_test.py. --- .github/workflows/release_promote_rc.yaml | 11 ++++++++ .../tools/private/release/promote_rc_test.py | 27 +++++++++++++++++++ tools/private/release/promote_rc.py | 4 +++ 3 files changed, 42 insertions(+) diff --git a/.github/workflows/release_promote_rc.yaml b/.github/workflows/release_promote_rc.yaml index 6dd02f1a25..399e7b1fa2 100644 --- a/.github/workflows/release_promote_rc.yaml +++ b/.github/workflows/release_promote_rc.yaml @@ -25,6 +25,8 @@ permissions: jobs: promote: runs-on: ubuntu-latest + outputs: + version: ${{ steps.promote.outputs.version }} steps: - name: Checkout repository uses: actions/checkout@v7 @@ -42,6 +44,7 @@ jobs: git config --global user.email "41898282+github-actions[bot]@users.noreply.github.com" - name: Run Promote RC + id: promote env: GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} VERSION: ${{ inputs.version }} @@ -58,3 +61,11 @@ jobs: ARGS+=("--no-dry-run") bazel run //tools/private/release -- promote-rc "${ARGS[@]}" + + publish: + needs: promote + uses: ./.github/workflows/release_publish.yaml + with: + tag_name: ${{ needs.promote.outputs.version }} + publish_to_pypi: true + secrets: inherit diff --git a/tests/tools/private/release/promote_rc_test.py b/tests/tools/private/release/promote_rc_test.py index 5bfd122c4c..b5dff3e478 100644 --- a/tests/tools/private/release/promote_rc_test.py +++ b/tests/tools/private/release/promote_rc_test.py @@ -1,5 +1,8 @@ import argparse +import os +import tempfile import unittest +from pathlib import Path from unittest.mock import call, patch from tests.tools.private.release.release_test_helper import _mock_git_and_gh @@ -10,6 +13,8 @@ class CmdPromoteRcTest(unittest.TestCase): def setUp(self): _mock_git_and_gh(self) + self.test_dir = tempfile.TemporaryDirectory() + self.addCleanup(self.test_dir.cleanup) def test_promote_rc_success(self): # Arrange @@ -59,6 +64,28 @@ def test_promote_rc_success(self): ) self.mock_gh.post_issue_comment.assert_called_once_with(123, expected_comment) + def test_promote_rc_writes_github_output(self): + # Arrange + github_output_path = os.path.join(self.test_dir.name, "github_output") + args = argparse.Namespace( + version="2.0.0", issue=123, dry_run=False, remote="my-remote" + ) + self.mock_git.get_remote_tags.return_value = ["2.0.0-rc0", "2.0.0-rc1"] + self.mock_git.get_commit_sha.return_value = "abcdef123456" + self.mock_git.tag_exists.return_value = False + initial_body = "- [ ] Tag Final" + self.mock_gh.get_issue_body.return_value = initial_body + + # Act + with patch.dict("os.environ", {"GITHUB_OUTPUT": github_output_path}): + result = PromoteRc(args, self.mock_git, self.mock_gh).run() + + # Assert + self.assertEqual(result, 0) + self.assertTrue(os.path.exists(github_output_path)) + content = Path(github_output_path).read_text(encoding="utf-8") + self.assertEqual(content, "version=2.0.0\n") + def test_promote_rc_resolve_issue_success(self): # Arrange args = argparse.Namespace( diff --git a/tools/private/release/promote_rc.py b/tools/private/release/promote_rc.py index ed59f0b76e..34c8107af4 100644 --- a/tools/private/release/promote_rc.py +++ b/tools/private/release/promote_rc.py @@ -157,6 +157,10 @@ def run(self) -> int: self.git.tag(version, commit_sha) self.git.push(args.remote, version) + if github_output := os.environ.get("GITHUB_OUTPUT"): + with open(github_output, "a", encoding="utf-8") as f: + f.write(f"version={version}\n") + print(f"Updating tracking issue #{issue_num} checklist...") self.gh.update_issue_body(issue_num, updated_body) From 52c51be21d612a8b385438bebc594375f1cd5a09 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Sun, 5 Jul 2026 11:35:34 -0700 Subject: [PATCH 822/922] docs: sync 2.2 changelog and news to main (#3901) The 2.2 release was completed, but the changelog updates and news file deletions were only performed on the release branch. We need to sync these changes back to main so that they are reflected in the unreleased state and future releases. --- CHANGELOG.md | 9 +++++++++ docs/pypi/download.md | 2 +- news/3854.fixed.md | 2 -- news/3886.fixed.md | 1 - news/3890.fixed.md | 2 -- news/pip-dep-tag-class.added.md | 4 ---- 6 files changed, 10 insertions(+), 10 deletions(-) delete mode 100644 news/3854.fixed.md delete mode 100644 news/3886.fixed.md delete mode 100644 news/3890.fixed.md delete mode 100644 news/pip-dep-tag-class.added.md diff --git a/CHANGELOG.md b/CHANGELOG.md index bcb6c6ddee..17f176e5b2 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -46,12 +46,17 @@ compatibility. * Fixed a flaky error on Windows 2022 when looking up the win32 version during site initialization by retrying the lookup ([#3721](https://github.com/bazel-contrib/rules_python/issues/3721)). +* (binaries) Fixed building of legacy zipapps on Windows execution platforms by + using a hermetic tool instead of host `cat`. +* (bootstrap) Fixed stage 1 bootstrap imports when target outputs shadow + standard library modules. * (coverage) Skip lcov report when no data was collected. * (pypi) Fixed `experimental_index_url` checking truthiness before envsubst expansion. * (rules) Fixed venv output paths for `py_binary` and `py_test` targets whose names contain path separators so distinct targets with the same basename no longer share the same venv output directory. +* (windows) Fixed build data generation on localized Windows installations. {#v2-2-0-added} ### Added @@ -60,6 +65,10 @@ exported by bzl files. * Exposed {bzl:obj}`VenvSymlinkEntry` and {bzl:obj}`VenvSymlinkKind` in {bzl:target}`//python:py_info.bzl`. * (pypi) Added `@pypi` repo: a unified hub of `pip.parse` hubs. +* (pypi) Added a `dep` tag class to the `pip` bzlmod extension. This allows + modules to declare abstract PyPI dependencies, ensuring target structures + exist in the unified hub, while allowing other modules to provide the + concrete implementation via `pip.parse`. * (uv) Support for basic `uv.lock` generation via the `lock` rule and basic support for importing the `uv.lock` file itself. Since this may have bugs, please report this by creating new tickets. diff --git a/docs/pypi/download.md b/docs/pypi/download.md index 88c2dd296d..e819ed0791 100644 --- a/docs/pypi/download.md +++ b/docs/pypi/download.md @@ -123,7 +123,7 @@ wheel version from the active hub during the build. ### Declaring Abstract Dependencies (pip.dep) -:::{versionadded} VERSION_NEXT_FEATURE +:::{versionadded} 2.2.0 Declaring abstract PyPI dependencies via `pip.dep` tags. ::: diff --git a/news/3854.fixed.md b/news/3854.fixed.md deleted file mode 100644 index 5c38b27642..0000000000 --- a/news/3854.fixed.md +++ /dev/null @@ -1,2 +0,0 @@ -(bootstrap) Fixed stage 1 bootstrap imports when target outputs shadow standard -library modules. diff --git a/news/3886.fixed.md b/news/3886.fixed.md deleted file mode 100644 index 2285813f8b..0000000000 --- a/news/3886.fixed.md +++ /dev/null @@ -1 +0,0 @@ -(windows) Fixed build data generation on localized Windows installations. diff --git a/news/3890.fixed.md b/news/3890.fixed.md deleted file mode 100644 index 59cd5fc5ca..0000000000 --- a/news/3890.fixed.md +++ /dev/null @@ -1,2 +0,0 @@ -(binaries) Fixed building of legacy zipapps on Windows execution platforms by -using a hermetic tool instead of host `cat`. diff --git a/news/pip-dep-tag-class.added.md b/news/pip-dep-tag-class.added.md deleted file mode 100644 index 5306186081..0000000000 --- a/news/pip-dep-tag-class.added.md +++ /dev/null @@ -1,4 +0,0 @@ -(pypi) Added a `dep` tag class to the `pip` bzlmod extension. This allows -modules to declare abstract PyPI dependencies, ensuring target structures -exist in the unified hub, while allowing other modules to provide the -concrete implementation via `pip.parse`. From bcaf08a64d24cfb8f6d05f032fbf95cd314b2474 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Sun, 5 Jul 2026 19:15:46 -0700 Subject: [PATCH 823/922] chore(release): sync changelog and version markers to main during backports (#3903) This PR implements the changes to sync both CHANGELOG.md and version next markers to the `main` branch when backports are processed. Key changes: - Adds git `diff`, `apply`, and `apply_check` helpers to the release tool. - Updates the GitHub helper to support custom PR base and auto-merge. - Updates `changelog_news.py` to support selective news merging and correct semver-sorted insertion, while preserving the static `Unreleased` section. - Updates `process_backports.py` to collect version marker diffs during cherry-pick, check if they apply cleanly to main, apply them, and report any failures in the PR body. These changes ensure that the `main` branch's changelog and version markers remain in sync with the release branch during the backport process. --- .github/workflows/on_pr_closed.yaml | 30 ++ tests/tools/private/release/BUILD.bazel | 9 + .../private/release/add_backports_test.py | 3 + .../private/release/changelog_news_test.py | 141 +++++- .../release/complete_sync_changelog_test.py | 120 +++++ tests/tools/private/release/git_test.py | 105 +++- tests/tools/private/release/prepare_test.py | 18 +- .../private/release/process_backports_test.py | 467 ++++++++++++++++-- .../private/release/release_issue_test.py | 61 +++ tools/private/release/add_backports.py | 3 + tools/private/release/changelog_news.py | 72 +-- .../release/complete_sync_changelog.py | 97 ++++ tools/private/release/gh.py | 46 +- tools/private/release/git.py | 52 +- tools/private/release/prepare.py | 6 +- tools/private/release/process_backports.py | 256 +++++++++- tools/private/release/release.py | 2 + tools/private/release/release_issue.py | 61 +++ 18 files changed, 1442 insertions(+), 107 deletions(-) create mode 100644 tests/tools/private/release/complete_sync_changelog_test.py create mode 100644 tools/private/release/complete_sync_changelog.py diff --git a/.github/workflows/on_pr_closed.yaml b/.github/workflows/on_pr_closed.yaml index 5dfcb89748..94c49b6561 100644 --- a/.github/workflows/on_pr_closed.yaml +++ b/.github/workflows/on_pr_closed.yaml @@ -81,3 +81,33 @@ jobs: "$PR_NUMBER" \ --remote origin \ --no-dry-run + + complete_sync_changelog: + if: | + github.event.pull_request.merged == true && + contains(github.event.pull_request.labels.*.name, 'type: sync-changelog') + runs-on: ubuntu-latest + permissions: + contents: write + issues: write + pull-requests: read + steps: + - name: Checkout repository + uses: actions/checkout@v7 + with: + fetch-depth: 0 + + - name: Setup Bazel + uses: bazel-contrib/setup-bazel@0.19.0 + with: + bazelisk-version: 1.20.0 + + - name: Complete Sync Changelog + env: + GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} + PR_NUMBER: ${{ github.event.pull_request.number }} + run: | + bazel run //tools/private/release -- complete-sync-changelog \ + --pr "$PR_NUMBER" + + diff --git a/tests/tools/private/release/BUILD.bazel b/tests/tools/private/release/BUILD.bazel index bc3b6f7e8b..633cfa328f 100644 --- a/tests/tools/private/release/BUILD.bazel +++ b/tests/tools/private/release/BUILD.bazel @@ -26,6 +26,15 @@ py_test( ], ) +py_test( + name = "complete_sync_changelog_test", + srcs = ["complete_sync_changelog_test.py"], + deps = [ + ":release_test_helper", + "//tools/private/release:release_lib", + ], +) + py_test( name = "create_rc_test", srcs = ["create_rc_test.py"], diff --git a/tests/tools/private/release/add_backports_test.py b/tests/tools/private/release/add_backports_test.py index 7a5d96a23b..75729e87fe 100644 --- a/tests/tools/private/release/add_backports_test.py +++ b/tests/tools/private/release/add_backports_test.py @@ -35,6 +35,8 @@ def test_add_backports_explicit_issue(self): self.assertIn("- [ ] #125", call_args[1]) # Should also auto-add Tag RC0 self.assertIn("- [ ] Tag RC0", call_args[1]) + self.assertIn("- [ ] Sync Changelog #124", call_args[1]) + self.assertIn("- [ ] Sync Changelog #125", call_args[1]) def test_add_backports_auto_discover_success(self): args = argparse.Namespace(issue=None, prs=["124"]) @@ -98,6 +100,7 @@ def test_add_backports_no_auto_add_rc_if_pending(self): self.assertNotIn("Tag RC1", call_args[1]) # Tag RC0 should still be there self.assertIn("- [ ] Tag RC0", call_args[1]) + self.assertIn("- [ ] Sync Changelog #124", call_args[1]) if __name__ == "__main__": diff --git a/tests/tools/private/release/changelog_news_test.py b/tests/tools/private/release/changelog_news_test.py index f8bd141a55..83ab2cc959 100644 --- a/tests/tools/private/release/changelog_news_test.py +++ b/tests/tools/private/release/changelog_news_test.py @@ -16,21 +16,9 @@ def test_update_changelog_with_news(self): [unreleased]: https://github.com/bazel-contrib/rules_python/releases/tag/unreleased -{#unreleased-removed} -### Removed -* Nothing removed. - -{#unreleased-changed} -### Changed -* Nothing changed. - -{#unreleased-fixed} -### Fixed -* Nothing fixed. - -{#unreleased-added} -### Added -* Nothing added. +Unreleased changes are tracked as individual files in the [news/](./news) +directory, or view the [latest generated +changelog](https://rules-python.readthedocs.io/en/latest/changelog.html). {#v2-0-2} ## [2.0.2] - 2026-05-14 @@ -408,6 +396,129 @@ def test_update_changelog_empty_news(self): self.assertNotIn("{#v3-0-0-fixed}", new_content) self.assertNotIn("{#v3-0-0-added}", new_content) + def test_update_changelog_selective_news_files(self): + # Arrange + changelog = """# Changelog + +{#unreleased} +## Unreleased + +[unreleased]: https://github.com/bazel-contrib/rules_python/releases/tag/unreleased + +{#v2-0-2} +## [2.0.2] - 2026-05-14 + +[2.0.2]: https://github.com/bazel-contrib/rules_python/releases/tag/2.0.2 +""" + changelog_path = self.tmpdir / "CHANGELOG.md" + changelog_path.write_text(changelog) + + news_dir = self.tmpdir / "news" + news_dir.mkdir() + + # Create news files + (news_dir / "123.fixed.md").write_text("Fix A") + (news_dir / "456.fixed.md").write_text("Fix B") + + # Act: Only process 123.fixed.md + changelog_news.update_changelog( + "2.0.3", + "2026-06-16", + changelog_path=changelog_path, + news_dir=news_dir, + news_files=[news_dir / "123.fixed.md"], + ) + + # Assert + # 1. Only 123.fixed.md should be deleted + self.assertFalse((news_dir / "123.fixed.md").exists()) + self.assertTrue((news_dir / "456.fixed.md").exists()) + + new_content = changelog_path.read_text() + + # 2. Only Fix A should be in the changelog + self.assertIn("Fix A", new_content) + self.assertNotIn("Fix B", new_content) + + def test_update_changelog_insertion_point(self): + # Arrange + changelog = """# Changelog + +{#unreleased} +## Unreleased + +[unreleased]: https://github.com/bazel-contrib/rules_python/releases/tag/unreleased + +{#v2-2-0} +## [2.2.0] - 2026-06-30 + +[2.2.0]: https://github.com/bazel-contrib/rules_python/releases/tag/2.2.0 + +{#v2-0-0} +## [2.0.0] - 2026-04-09 + +[2.0.0]: https://github.com/bazel-contrib/rules_python/releases/tag/2.0.0 +""" + changelog_path = self.tmpdir / "CHANGELOG.md" + changelog_path.write_text(changelog) + + news_dir = self.tmpdir / "news" + news_dir.mkdir() + (news_dir / "123.fixed.md").write_text("Fix in 2.1.0") + + # Act: Insert 2.1.0 + changelog_news.update_changelog( + "2.1.0", + "2026-06-17", + changelog_path=changelog_path, + news_dir=news_dir, + ) + + # Assert + new_content = changelog_path.read_text() + + # Verify 2.1.0 is inserted BEFORE 2.0.0 but AFTER 2.2.0 + idx_2_2_0 = new_content.index("{#v2-2-0}") + idx_2_1_0 = new_content.index("{#v2-1-0}") + idx_2_0_0 = new_content.index("{#v2-0-0}") + + self.assertTrue(idx_2_2_0 < idx_2_1_0 < idx_2_0_0) + self.assertIn("Fix in 2.1.0", new_content) + + def test_update_changelog_insertion_point_too_small(self): + # Arrange + changelog = """# Changelog + +{#unreleased} +## Unreleased + +[unreleased]: https://github.com/bazel-contrib/rules_python/releases/tag/unreleased + +{#v2-0-0} +## [2.0.0] - 2026-04-09 + +[2.0.0]: https://github.com/bazel-contrib/rules_python/releases/tag/2.0.0 +""" + changelog_path = self.tmpdir / "CHANGELOG.md" + changelog_path.write_text(changelog) + + news_dir = self.tmpdir / "news" + news_dir.mkdir() + (news_dir / "123.fixed.md").write_text("Fix in 1.0.0") + + # Act & Assert + with self.assertRaises(ValueError) as ctx: + changelog_news.update_changelog( + "1.0.0", + "2026-01-01", + changelog_path=changelog_path, + news_dir=news_dir, + ) + self.assertIn( + "Could not find a version in CHANGELOG.md smaller than 1.0.0", + str(ctx.exception), + ) + if __name__ == "__main__": unittest.main() diff --git a/tests/tools/private/release/complete_sync_changelog_test.py b/tests/tools/private/release/complete_sync_changelog_test.py new file mode 100644 index 0000000000..66c40c01d3 --- /dev/null +++ b/tests/tools/private/release/complete_sync_changelog_test.py @@ -0,0 +1,120 @@ +import argparse +import unittest +from unittest.mock import patch + +from tests.tools.private.release.release_test_helper import _mock_git_and_gh +from tools.private.release.complete_sync_changelog import CompleteSyncChangelog + + +class CompleteSyncChangelogTest(unittest.TestCase): + def setUp(self): + _mock_git_and_gh(self) + self.addCleanup(patch.stopall) + + # Dynamic mock for issue body + self.issue_body = "" + + def mock_get_body(issue_num): + return self.issue_body + + def mock_update_body(issue_num, body): + self.issue_body = body + + self.mock_gh.get_issue_body.side_effect = mock_get_body + self.mock_gh.update_issue_body.side_effect = mock_update_body + + def test_complete_sync_changelog_success(self): + args = argparse.Namespace(pr=999) + self.mock_gh.get_pr_info.return_value = { + "state": "MERGED", + "body": "Updates CHANGELOG.md\n\nRelease-Tracking-Issue: #123", + "mergeCommit": {"oid": "abcdef1234567890"}, + } + self.issue_body = """ +## Checklist +- [ ] Prepare Release +- [ ] Create Release branch +- [ ] Sync Changelog #124 | status=pending pr=#999 +- [ ] Sync Changelog #125 | status=pending pr=#999 +- [ ] Sync Changelog #126 | status=pending pr=#888 +- [ ] Tag Final + +## Backports +""" + result = CompleteSyncChangelog(args, self.mock_gh).run() + + self.assertEqual(result, 0) + self.mock_gh.get_pr_info.assert_called_once_with(999) + self.mock_gh.get_issue_body.assert_called_once_with(123) + self.mock_gh.update_issue_body.assert_called_once() + + # Check that only tasks pointing to #999 were marked checked=True and status=done + self.assertIn( + "- [x] Sync Changelog #124 | status=done pr=#999 commit= abcdef12", + self.issue_body, + ) + self.assertIn( + "- [x] Sync Changelog #125 | status=done pr=#999 commit= abcdef12", + self.issue_body, + ) + # Task pointing to #888 should remain unchanged + self.assertIn( + "- [ ] Sync Changelog #126 | status=pending pr=#888", + self.issue_body, + ) + + def test_complete_sync_changelog_not_merged(self): + args = argparse.Namespace(pr=999) + self.mock_gh.get_pr_info.return_value = { + "state": "OPEN", + "body": "Updates CHANGELOG.md\n\nRelease-Tracking-Issue: #123", + } + + result = CompleteSyncChangelog(args, self.mock_gh).run() + + self.assertEqual(result, 1) + self.mock_gh.get_pr_info.assert_called_once_with(999) + self.mock_gh.update_issue_body.assert_not_called() + + def test_complete_sync_changelog_missing_tracking_issue_link(self): + args = argparse.Namespace(pr=999) + self.mock_gh.get_pr_info.return_value = { + "state": "MERGED", + "body": "Updates CHANGELOG.md without tracking issue link", + "mergeCommit": {"oid": "abcdef1234567890"}, + } + + result = CompleteSyncChangelog(args, self.mock_gh).run() + + self.assertEqual(result, 1) + self.mock_gh.get_pr_info.assert_called_once_with(999) + self.mock_gh.update_issue_body.assert_not_called() + + def test_complete_sync_changelog_no_matching_tasks(self): + args = argparse.Namespace(pr=999) + self.mock_gh.get_pr_info.return_value = { + "state": "MERGED", + "body": "Updates CHANGELOG.md\n\nRelease-Tracking-Issue: #123", + "mergeCommit": {"oid": "abcdef1234567890"}, + } + # Checklist has no tasks pointing to #999 + self.issue_body = """ +## Checklist +- [ ] Prepare Release +- [ ] Create Release branch +- [ ] Sync Changelog #124 | status=pending pr=#888 +- [ ] Tag Final + +## Backports +""" + result = CompleteSyncChangelog(args, self.mock_gh).run() + + # Should log warning but return 0 (success/noop) + self.assertEqual(result, 0) + self.mock_gh.get_pr_info.assert_called_once_with(999) + self.mock_gh.get_issue_body.assert_called_once_with(123) + self.mock_gh.update_issue_body.assert_not_called() + + +if __name__ == "__main__": + unittest.main() diff --git a/tests/tools/private/release/git_test.py b/tests/tools/private/release/git_test.py index 16667b8715..34643fc016 100644 --- a/tests/tools/private/release/git_test.py +++ b/tests/tools/private/release/git_test.py @@ -41,7 +41,7 @@ def test_checkout_track_remote_existing_branch( self.mock_run_git.assert_called_once_with( "checkout", "my-branch", capture_output=False ) - mock_reset_hard.assert_called_once_with("origin/my-branch") + mock_reset_hard.assert_called_once_with(reset_to="origin/my-branch") class GitFetchTest(unittest.TestCase): @@ -82,5 +82,108 @@ def test_fetch_all_options(self): ) +class GitGetModifiedFilesTest(unittest.TestCase): + def setUp(self): + self.git = Git(".") + self.patcher = patch.object(self.git, "_run_git") + self.mock_run_git = self.patcher.start() + self.addCleanup(self.patcher.stop) + + def test_get_modified_files(self): + self.mock_run_git.return_value = "file1.txt\nfile2.py\n\n" + files = self.git.get_modified_files("HEAD") + self.mock_run_git.assert_called_once_with( + "show", "--name-only", "--format=", "HEAD" + ) + self.assertEqual(files, ["file1.txt", "file2.py"]) + + def test_get_modified_files_empty(self): + self.mock_run_git.return_value = "" + files = self.git.get_modified_files("HEAD") + self.assertEqual(files, []) + + +class GitDiffTest(unittest.TestCase): + def setUp(self): + self.git = Git(".") + self.patcher = patch.object(self.git, "_run_git") + self.mock_run_git = self.patcher.start() + self.addCleanup(self.patcher.stop) + + def test_diff_has_changes(self): + self.mock_run_git.return_value = "some diff output" + output = self.git.diff() + self.mock_run_git.assert_called_once_with("diff") + self.assertEqual(output, "some diff output") + + def test_diff_empty(self): + self.mock_run_git.return_value = "" + output = self.git.diff() + self.mock_run_git.assert_called_once_with("diff") + self.assertEqual(output, "") + + +class GitApplyTest(unittest.TestCase): + def setUp(self): + self.git = Git(".") + self.patcher = patch.object(self.git, "_run_git") + self.mock_run_git = self.patcher.start() + self.addCleanup(self.patcher.stop) + + def test_apply(self): + self.git.apply("patch.patch") + self.mock_run_git.assert_called_once_with( + "apply", "patch.patch", capture_output=False + ) + + +class GitApplyCheckTest(unittest.TestCase): + def setUp(self): + self.git = Git(".") + self.patcher = patch.object(self.git, "_run_git") + self.mock_run_git = self.patcher.start() + self.addCleanup(self.patcher.stop) + + def test_apply_check_clean(self): + self.mock_run_git.return_value = "" + result = self.git.apply_check("patch.patch") + self.mock_run_git.assert_called_once_with( + "apply", "--check", "patch.patch", capture_output=False + ) + self.assertTrue(result) + + def test_apply_check_conflict(self): + import subprocess + + self.mock_run_git.side_effect = subprocess.CalledProcessError( + 1, ["git", "apply", "--check", "patch.patch"] + ) + result = self.git.apply_check("patch.patch") + self.mock_run_git.assert_called_once_with( + "apply", "--check", "patch.patch", capture_output=False + ) + self.assertFalse(result) + + +class GitResetHardTest(unittest.TestCase): + def setUp(self): + self.git = Git(".") + self.patcher = patch.object(self.git, "_run_git") + self.mock_run_git = self.patcher.start() + self.addCleanup(self.patcher.stop) + + def test_reset_hard_default(self): + self.git.reset_hard() + self.mock_run_git.assert_called_once_with( + "reset", "--hard", "HEAD", capture_output=False + ) + + def test_reset_hard_custom(self): + self.git.reset_hard(reset_to="my-commit") + self.mock_run_git.assert_called_once_with( + "reset", "--hard", "my-commit", capture_output=False + ) + + if __name__ == "__main__": unittest.main() diff --git a/tests/tools/private/release/prepare_test.py b/tests/tools/private/release/prepare_test.py index 3d92b131e2..45b92a636f 100644 --- a/tests/tools/private/release/prepare_test.py +++ b/tests/tools/private/release/prepare_test.py @@ -36,7 +36,11 @@ def test_prepare_success_existing_issue(self, mock_replace, mock_changelog): self.assertEqual(result, 0) self.mock_gh.get_release_tracking_issue.assert_called_once_with("2.0.0") self.mock_gh.create_tracking_issue.assert_not_called() - self.mock_gh.create_pr.assert_called_once_with("2.0.0", 123) + self.mock_gh.create_pr.assert_called_once_with( + title="Prepare release v2.0.0", + body="Work towards #123", + base="main", + ) self.mock_git.add_modified_and_deleted.assert_called_once() @patch("tools.private.release.prepare.changelog_news") @@ -67,7 +71,11 @@ def test_prepare_success_create_issue(self, mock_replace, mock_changelog): self.mock_gh.create_tracking_issue.assert_called_once_with( "2.0.0", "dummy template content" ) - self.mock_gh.create_pr.assert_called_once_with("2.0.0", 123) + self.mock_gh.create_pr.assert_called_once_with( + title="Prepare release v2.0.0", + body="Work towards #123", + base="main", + ) self.mock_git.add_modified_and_deleted.assert_called_once() @patch("tools.private.release.prepare.changelog_news") @@ -172,7 +180,11 @@ def test_prepare_create_pr_when_none_associated(self, mock_replace, mock_changel "origin", "prepare-2.0.0", set_upstream=True, force=True ) self.mock_gh.get_open_pr.assert_called_once_with("prepare-2.0.0") - self.mock_gh.create_pr.assert_called_once_with("2.0.0", 123) + self.mock_gh.create_pr.assert_called_once_with( + title="Prepare release v2.0.0", + body="Work towards #123", + base="main", + ) self.mock_gh.update_issue_body.assert_called_once() call_args = self.mock_gh.update_issue_body.call_args[0] self.assertIn("pr=#789", call_args[1]) diff --git a/tests/tools/private/release/process_backports_test.py b/tests/tools/private/release/process_backports_test.py index fe7bae256d..d5df43ae46 100644 --- a/tests/tools/private/release/process_backports_test.py +++ b/tests/tools/private/release/process_backports_test.py @@ -20,12 +20,26 @@ def setUp(self): self.mock_gh.resolve_pr_number.side_effect = lambda x: int( x.lstrip("#").split("/")[-1] ) + self.mock_git.diff.return_value = "" + self.mock_git.apply_check.return_value = True + + # Dynamic mock for issue body + self.issue_body = "" + + def mock_get_body(issue_num): + return self.issue_body + + def mock_update_body(issue_num, body): + self.issue_body = body + + self.mock_gh.get_issue_body.side_effect = mock_get_body + self.mock_gh.update_issue_body.side_effect = mock_update_body def test_process_backports_no_pending(self): args = argparse.Namespace( issue=123, remote="origin", dry_run=False, add=None, triggering_comment=None ) - self.mock_gh.get_issue_body.return_value = "No backports here" + self.issue_body = "No backports here" result = ProcessBackports(args, self.mock_git, self.mock_gh).run() @@ -40,10 +54,12 @@ def test_process_backports_success(self, mock_datetime): issue=123, remote="origin", dry_run=False, add=None, triggering_comment=None ) self.mock_gh.get_issue_title.return_value = "Release 2.0.0" - self.mock_gh.get_issue_body.return_value = """ + self.issue_body = """ ## Checklist - [ ] Prepare Release - [ ] Create Release branch +- [ ] Sync Changelog #124 +- [ ] Tag Final ## Backports - [ ] #124 | status=pending @@ -60,33 +76,204 @@ def mock_resolve(items): self.mock_gh.get_merge_commits_for_prs.side_effect = mock_resolve self.mock_git.sort_commits_chronologically.return_value = ["abcdef12"] - self.mock_git.get_commit_sha.return_value = "12345678" + self.mock_git.get_commit_sha.side_effect = ["12345678", "12345678", "main_sha"] self.mock_git.get_commit_message.return_value = 'Cherry-pick "fix bug"' + self.mock_git.get_modified_files.return_value = ["news/124.fixed.md"] + self.mock_git.diff.return_value = "version diff for 124" + self.mock_git.apply_check.return_value = True + self.mock_gh.create_pr.return_value = "https://github.com/foo/bar/pull/999" result = ProcessBackports(args, self.mock_git, self.mock_gh).run() self.assertEqual(result, 0) self.mock_git.fetch.assert_has_calls( - [call("origin", tags=True, force=True), call("origin")] + [ + call("origin", tags=True, force=True), + call("origin"), + call("origin", refspec="main"), + ] ) - self.mock_git.checkout.assert_called_once_with( - "release/2.0", track_remote="origin" + self.mock_git.checkout.assert_has_calls( + [ + call("release/2.0", track_remote="origin"), + call("main", track_remote="origin"), + call("prepare-2.0.0-backports-6affdae", create_branch=True), + call("release/2.0"), + ] ) self.mock_git.cherry_pick.assert_called_once_with("abcdef12") - self.mock_changelog_news.update_changelog.assert_called_once_with( - "2.0.0", "2026-07-01" + self.mock_git.diff.assert_called_once() + self.mock_git.apply_check.assert_called_once_with(unittest.mock.ANY) + self.mock_git.apply.assert_called_once_with(unittest.mock.ANY) + self.mock_changelog_news.update_changelog.assert_has_calls( + [ + call("2.0.0", "2026-07-01"), + call( + "2.0.0", + "2026-07-01", + news_files=["news/124.fixed.md"], + delete_news=True, + ), + ] ) - self.mock_git.add_modified_and_deleted.assert_called_once() + self.assertEqual(self.mock_git.add_modified_and_deleted.call_count, 2) self.mock_replace_version_next.assert_called_once_with("2.0.0") - self.mock_git.commit.assert_called_once_with( - 'Cherry-pick "fix bug"\n\nWork towards #123', amend=True + self.mock_git.commit.assert_has_calls( + [ + call('Cherry-pick "fix bug"\n\nWork towards #123', amend=True), + call("chore(release): sync changelog for v2.0.0 backports"), + ] + ) + self.mock_git.push.assert_has_calls( + [ + call("origin", "release/2.0"), + call( + "origin", + "prepare-2.0.0-backports-6affdae", + set_upstream=True, + force=True, + ), + ] ) - self.mock_git.push.assert_called_once_with("origin", "release/2.0") - self.mock_gh.update_issue_body.assert_called_once() - call_args = self.mock_gh.update_issue_body.call_args[0] - self.assertEqual(call_args[0], 123) - self.assertIn("- [x] #124 | status=done rc=rc0 commit= 12345678", call_args[1]) + self.mock_gh.create_pr.assert_called_once_with( + title="chore(release): sync changelog for v2.0.0 backports", + body="Updates CHANGELOG.md and removes news files for backports:\n- #124\n\nWork towards #123\nRelease-Tracking-Issue: #123", + base="main", + labels=["type: sync-changelog"], + ) + self.mock_gh.enable_auto_merge.assert_called_once_with(999) + + self.assertEqual(self.mock_gh.update_issue_body.call_count, 2) + call_args_list = self.mock_gh.update_issue_body.call_args_list + self.assertEqual(call_args_list[0][0][0], 123) + self.assertIn( + "- [x] #124 | status=done rc=rc0 commit= 12345678", call_args_list[0][0][1] + ) + self.assertEqual(call_args_list[1][0][0], 123) + self.assertIn( + "- [ ] Sync Changelog #124 | status=pending pr=#999", + call_args_list[1][0][1], + ) + + @patch("tools.private.release.process_backports.datetime") + def test_process_backports_sync_branch_exists(self, mock_datetime): + mock_datetime.date.today.return_value = datetime.date(2026, 7, 1) + args = argparse.Namespace( + issue=123, remote="origin", dry_run=False, add=None, triggering_comment=None + ) + self.mock_gh.get_issue_title.return_value = "Release 2.0.0" + self.issue_body = """ +## Checklist +- [ ] Prepare Release +- [ ] Create Release branch +- [ ] Sync Changelog #124 +- [ ] Tag Final + +## Backports +- [ ] #124 | status=pending +""" + self.mock_git.get_remote_tags.return_value = [] + + def mock_resolve(items): + for item in items: + if item.pr_ref == "#124": + item.commit = "abcdef12" + item.status = "done" + return items + + self.mock_gh.get_merge_commits_for_prs.side_effect = mock_resolve + + self.mock_git.sort_commits_chronologically.return_value = ["abcdef12"] + self.mock_git.get_commit_sha.side_effect = ["12345678", "12345678", "main_sha"] + self.mock_git.get_commit_message.return_value = 'Cherry-pick "fix bug"' + self.mock_git.get_modified_files.return_value = ["news/124.fixed.md"] + self.mock_git.diff.return_value = "version diff for 124" + self.mock_git.apply_check.return_value = True + self.mock_gh.create_pr.return_value = "https://github.com/foo/bar/pull/999" + + # Configure branch to exist + self.mock_git.branch_exists.return_value = True + + result = ProcessBackports(args, self.mock_git, self.mock_gh).run() + + self.assertEqual(result, 0) + self.mock_git.fetch.assert_has_calls( + [ + call("origin", tags=True, force=True), + call("origin"), + call("origin", refspec="main"), + ] + ) + self.mock_git.checkout.assert_has_calls( + [ + call("release/2.0", track_remote="origin"), + call("main", track_remote="origin"), + # Called without create_branch=True + call("prepare-2.0.0-backports-6affdae"), + call("release/2.0"), + ] + ) + # Verify reset_hard was called to reset the existing branch to main + self.mock_git.reset_hard.assert_has_calls( + [ + call(reset_to="main"), + ] + ) + self.mock_git.cherry_pick.assert_called_once_with("abcdef12") + self.mock_git.diff.assert_called_once() + self.mock_git.apply_check.assert_called_once_with(unittest.mock.ANY) + self.mock_git.apply.assert_called_once_with(unittest.mock.ANY) + self.mock_changelog_news.update_changelog.assert_has_calls( + [ + call("2.0.0", "2026-07-01"), + call( + "2.0.0", + "2026-07-01", + news_files=["news/124.fixed.md"], + delete_news=True, + ), + ] + ) + self.assertEqual(self.mock_git.add_modified_and_deleted.call_count, 2) + self.mock_replace_version_next.assert_called_once_with("2.0.0") + self.mock_git.commit.assert_has_calls( + [ + call('Cherry-pick "fix bug"\n\nWork towards #123', amend=True), + call("chore(release): sync changelog for v2.0.0 backports"), + ] + ) + self.mock_git.push.assert_has_calls( + [ + call("origin", "release/2.0"), + call( + "origin", + "prepare-2.0.0-backports-6affdae", + set_upstream=True, + force=True, + ), + ] + ) + + self.mock_gh.create_pr.assert_called_once_with( + title="chore(release): sync changelog for v2.0.0 backports", + body="Updates CHANGELOG.md and removes news files for backports:\n- #124\n\nWork towards #123\nRelease-Tracking-Issue: #123", + base="main", + labels=["type: sync-changelog"], + ) + self.mock_gh.enable_auto_merge.assert_called_once_with(999) + + self.assertEqual(self.mock_gh.update_issue_body.call_count, 2) + call_args_list = self.mock_gh.update_issue_body.call_args_list + self.assertEqual(call_args_list[0][0][0], 123) + self.assertIn( + "- [x] #124 | status=done rc=rc0 commit= 12345678", call_args_list[0][0][1] + ) + self.assertEqual(call_args_list[1][0][0], 123) + self.assertIn( + "- [ ] Sync Changelog #124 | status=pending pr=#999", + call_args_list[1][0][1], + ) @patch("tools.private.release.process_backports.datetime") def test_process_backports_dry_run(self, mock_datetime): @@ -95,10 +282,12 @@ def test_process_backports_dry_run(self, mock_datetime): issue=123, remote="origin", dry_run=True, add=None, triggering_comment=None ) self.mock_gh.get_issue_title.return_value = "Release 2.0.0" - self.mock_gh.get_issue_body.return_value = """ + self.issue_body = """ ## Checklist - [ ] Prepare Release - [ ] Create Release branch +- [ ] Sync Changelog #124 +- [ ] Tag Final ## Backports - [ ] #124 | status=pending @@ -115,28 +304,55 @@ def mock_resolve(items): self.mock_gh.get_merge_commits_for_prs.side_effect = mock_resolve self.mock_git.sort_commits_chronologically.return_value = ["abcdef12"] - self.mock_git.get_commit_sha.return_value = "12345678" + self.mock_git.get_commit_sha.side_effect = ["12345678", "main_sha"] self.mock_git.get_commit_message.return_value = 'Cherry-pick "fix bug"' + self.mock_git.get_modified_files.return_value = ["news/124.fixed.md"] + self.mock_git.diff.return_value = "version diff for 124" + self.mock_git.apply_check.return_value = True result = ProcessBackports(args, self.mock_git, self.mock_gh).run() self.assertEqual(result, 0) self.mock_git.fetch.assert_has_calls( - [call("origin", tags=True, force=True), call("origin")] + [ + call("origin", tags=True, force=True), + call("origin"), + call("origin", refspec="main"), + ] ) - self.mock_git.checkout.assert_called_once_with( - "release/2.0", track_remote="origin" + self.mock_git.checkout.assert_has_calls( + [ + call("release/2.0", track_remote="origin"), + call("main", track_remote="origin"), + call("release/2.0"), + ] ) self.mock_git.cherry_pick.assert_called_once_with("abcdef12") - self.mock_changelog_news.update_changelog.assert_called_once_with( - "2.0.0", "2026-07-01" + self.mock_git.diff.assert_called_once() + self.mock_git.apply_check.assert_called_once_with(unittest.mock.ANY) + self.mock_git.apply.assert_not_called() + self.mock_changelog_news.update_changelog.assert_has_calls( + [ + call("2.0.0", "2026-07-01"), + call( + "2.0.0", + "2026-07-01", + news_files=["news/124.fixed.md"], + delete_news=True, + ), + ] ) - self.mock_git.add_modified_and_deleted.assert_called_once() + self.assertEqual(self.mock_git.add_modified_and_deleted.call_count, 1) self.mock_replace_version_next.assert_called_once_with("2.0.0") self.mock_git.commit.assert_called_once_with( 'Cherry-pick "fix bug"\n\nWork towards #123', amend=True ) - self.mock_git.reset_hard.assert_called_once_with("12345678") + self.mock_git.reset_hard.assert_has_calls( + [ + call(reset_to="12345678"), + call(reset_to="main_sha"), + ] + ) self.mock_git.push.assert_not_called() self.mock_gh.update_issue_body.assert_not_called() @@ -145,7 +361,7 @@ def test_process_backports_ignored_and_failed_states(self): issue=123, remote="origin", dry_run=False, add=None, triggering_comment=None ) self.mock_gh.get_issue_title.return_value = "Release 2.0.0" - self.mock_gh.get_issue_body.return_value = """ + self.issue_body = """ ## Checklist - [ ] Prepare Release - [ ] Create Release branch @@ -186,7 +402,7 @@ def test_process_backports_ignored_error_status(self): issue=123, remote="origin", dry_run=False, add=None, triggering_comment=None ) self.mock_gh.get_issue_title.return_value = "Release 2.0.0" - self.mock_gh.get_issue_body.return_value = """ + self.issue_body = """ ## Checklist - [ ] Prepare Release - [ ] Create Release branch @@ -211,7 +427,7 @@ def test_process_backports_cherry_pick_failed(self, mock_datetime): issue=123, remote="origin", dry_run=False, add=None, triggering_comment=None ) self.mock_gh.get_issue_title.return_value = "Release 2.0.0" - self.mock_gh.get_issue_body.return_value = """ + self.issue_body = """ ## Checklist - [ ] Prepare Release - [ ] Create Release branch @@ -261,7 +477,7 @@ def test_process_backports_add_backports_and_auto_add_rc_task(self, mock_datetim triggering_comment=None, ) self.mock_gh.get_issue_title.return_value = "Release 2.0.0" - self.mock_gh.get_issue_body.return_value = """ + self.issue_body = """ ## Checklist - [x] Prepare Release | status=done pr=#122 commit=abcdef12 - [x] Create Release branch | status=done branch=release/2.0 commit=abcdef12 @@ -284,14 +500,18 @@ def mock_resolve(items): self.mock_gh.get_merge_commits_for_prs.side_effect = mock_resolve self.mock_git.sort_commits_chronologically.return_value = ["abcdef12"] + # Mock create_pr to return a string to avoid int(MagicMock) returning 1 + self.mock_gh.create_pr.return_value = "https://github.com/foo/bar/pull/999" + result = ProcessBackports(args, self.mock_git, self.mock_gh).run() self.assertEqual(result, 0) - # update_issue_body should be called twice: + # update_issue_body should be called 3 times: # 1. When adding backports and auto-adding Tag RC1 task. # 2. When updating the backport status to done. - self.assertEqual(self.mock_gh.update_issue_body.call_count, 2) + # 3. When updating the sync task status to pending. + self.assertEqual(self.mock_gh.update_issue_body.call_count, 3) call1_args = self.mock_gh.update_issue_body.call_args_list[0][0] call2_args = self.mock_gh.update_issue_body.call_args_list[1][0] @@ -299,15 +519,22 @@ def mock_resolve(items): self.assertEqual(call1_args[0], 123) self.assertIn("- [ ] #124", call1_args[1]) self.assertIn("- [ ] Tag RC1", call1_args[1]) + self.assertIn("- [ ] Sync Changelog #124", call1_args[1]) self.assertIn( "- [x] Tag RC0 | status=done tag=2.0.0-rc0 commit=abcdef12\n- [ ]" - " Tag RC1\n- [ ] Tag Final", + " Tag RC1\n- [ ] Sync Changelog #124\n- [ ] Tag Final", call1_args[1].strip(), ) self.assertEqual(call2_args[0], 123) self.assertIn("- [x] #124 | status=done rc=rc1 commit= 12345678", call2_args[1]) + call3_args = self.mock_gh.update_issue_body.call_args_list[2][0] + self.assertEqual(call3_args[0], 123) + self.assertIn( + "- [ ] Sync Changelog #124 | status=pending pr=#999", call3_args[1] + ) + @patch("tools.private.release.process_backports.datetime") def test_process_backports_add_backports_marks_invalid(self, mock_datetime): mock_datetime.date.today.return_value = datetime.date(2026, 7, 1) @@ -319,7 +546,7 @@ def test_process_backports_add_backports_marks_invalid(self, mock_datetime): triggering_comment=None, ) self.mock_gh.get_issue_title.return_value = "Release 2.0.0" - self.mock_gh.get_issue_body.return_value = """ + self.issue_body = """ ## Checklist - [x] Prepare Release | status=done pr=#122 commit=abcdef12 - [x] Create Release branch | status=done branch=release/2.0 commit=abcdef12 @@ -335,23 +562,189 @@ def test_process_backports_add_backports_marks_invalid(self, mock_datetime): def mock_resolve(items): # Both 124 and 125 should be processed, 'invalid' should be ignored (it has error status) for item in items: - if item.pr_ref in ("#124", "#125"): - item.commit = "abcdef12" + if item.pr_ref == "#124": + item.commit = "sha_124" + item.status = "done" + elif item.pr_ref == "#125": + item.commit = "sha_125" item.status = "done" return items self.mock_gh.get_merge_commits_for_prs.side_effect = mock_resolve - self.mock_git.sort_commits_chronologically.return_value = ["abcdef12"] + self.mock_git.sort_commits_chronologically.return_value = ["sha_124", "sha_125"] + self.mock_gh.create_pr.return_value = "https://github.com/foo/bar/pull/999" result = ProcessBackports(args, self.mock_git, self.mock_gh).run() self.assertEqual(result, 0) # Should have updated body to add 124, 125, and invalid - self.assertEqual(self.mock_gh.update_issue_body.call_count, 2) + # update_issue_body should be called 4 times: + # 1. When adding backports. + # 2. When updating 124 status to done. + # 3. When updating 125 status to done. + # 4. When updating sync tasks status to pending. + self.assertEqual(self.mock_gh.update_issue_body.call_count, 4) call1_args = self.mock_gh.update_issue_body.call_args_list[0][0] self.assertIn("- [ ] #124", call1_args[1]) self.assertIn("- [ ] #125", call1_args[1]) self.assertIn("- [ ] invalid | status=error-invalid-pr", call1_args[1]) + self.assertIn("- [ ] Sync Changelog #124", call1_args[1]) + self.assertIn("- [ ] Sync Changelog #125", call1_args[1]) + + call4_args = self.mock_gh.update_issue_body.call_args_list[3][0] + self.assertIn( + "- [ ] Sync Changelog #124 | status=pending pr=#999", call4_args[1] + ) + self.assertIn( + "- [ ] Sync Changelog #125 | status=pending pr=#999", call4_args[1] + ) + + @patch("tools.private.release.process_backports.datetime") + def test_process_backports_version_sync_failure(self, mock_datetime): + mock_datetime.date.today.return_value = datetime.date(2026, 7, 1) + args = argparse.Namespace( + issue=123, remote="origin", dry_run=False, add=None, triggering_comment=None + ) + self.mock_gh.get_issue_title.return_value = "Release 2.0.0" + self.issue_body = """ +## Checklist +- [ ] Prepare Release +- [ ] Create Release branch +- [ ] Sync Changelog #124 +- [ ] Sync Changelog #125 +- [ ] Tag Final + +## Backports +- [ ] #124 | status=pending +- [ ] #125 | status=pending +""" + self.mock_git.get_remote_tags.return_value = [] + + def mock_resolve(items): + for item in items: + if item.pr_ref in ("#124", "#125"): + item.commit = "sha_" + item.pr_ref.lstrip("#") + item.status = "done" + return items + + self.mock_gh.get_merge_commits_for_prs.side_effect = mock_resolve + + self.mock_git.sort_commits_chronologically.return_value = ["sha_124", "sha_125"] + self.mock_git.get_commit_sha.side_effect = [ + "12345678", + "sha_124_amended", + "sha_125_amended", + "main_sha", + ] + self.mock_git.get_commit_message.return_value = 'Cherry-pick "fix bug"' + self.mock_git.get_modified_files.side_effect = [ + ["news/124.fixed.md"], + ["news/125.fixed.md"], + ] + self.mock_git.diff.side_effect = ["diff 124", "diff 125"] + self.mock_git.apply_check.side_effect = [False, True] + self.mock_gh.create_pr.return_value = "https://github.com/foo/bar/pull/999" + + result = ProcessBackports(args, self.mock_git, self.mock_gh).run() + + self.assertEqual(result, 0) + self.mock_git.fetch.assert_has_calls( + [ + call("origin", tags=True, force=True), + call("origin"), + call("origin", refspec="main"), + ] + ) + self.mock_git.checkout.assert_has_calls( + [ + call("release/2.0", track_remote="origin"), + call("main", track_remote="origin"), + call("prepare-2.0.0-backports-b552a96", create_branch=True), + call("release/2.0"), + ] + ) + self.mock_git.cherry_pick.assert_has_calls( + [ + call("sha_124"), + call("sha_125"), + ] + ) + # diff should be called for each successful cherry-pick + self.assertEqual(self.mock_git.diff.call_count, 2) + # apply_check should be called for both patches + self.assertEqual(self.mock_git.apply_check.call_count, 2) + # apply should only be called for 125 (since 124 failed check) + self.mock_git.apply.assert_called_once_with(unittest.mock.ANY) + + self.mock_changelog_news.update_changelog.assert_has_calls( + [ + call("2.0.0", "2026-07-01"), + call("2.0.0", "2026-07-01"), + call( + "2.0.0", + "2026-07-01", + news_files=["news/124.fixed.md", "news/125.fixed.md"], + delete_news=True, + ), + ] + ) + # add_modified_and_deleted called: + # - once per cherry-pick (2) + # - once on main backport branch (1) + # Total = 3 + self.assertEqual(self.mock_git.add_modified_and_deleted.call_count, 3) + # replace_version_next called once per cherry-pick + self.assertEqual(self.mock_replace_version_next.call_count, 2) + + self.mock_git.commit.assert_has_calls( + [ + call('Cherry-pick "fix bug"\n\nWork towards #123', amend=True), + call('Cherry-pick "fix bug"\n\nWork towards #123', amend=True), + call("chore(release): sync changelog for v2.0.0 backports"), + ] + ) + self.mock_git.push.assert_has_calls( + [ + call("origin", "release/2.0"), + call("origin", "release/2.0"), + call( + "origin", + "prepare-2.0.0-backports-b552a96", + set_upstream=True, + force=True, + ), + ] + ) + + # PR body should contain warning about 124 + expected_body = ( + "Updates CHANGELOG.md and removes news files for backports:\n" + "- #124\n" + "- #125\n" + "\n" + "Warning: These PRs failed to update their version markers:\n" + "- #124\n" + "\n" + "Work towards #123\n" + "Release-Tracking-Issue: #123" + ) + self.mock_gh.create_pr.assert_called_once_with( + title="chore(release): sync changelog for v2.0.0 backports", + body=expected_body, + base="main", + labels=["type: sync-changelog"], + ) + self.mock_gh.enable_auto_merge.assert_called_once_with(999) + + # update_issue_body called 3 times (twice for backports, once for sync tasks) + self.assertEqual(self.mock_gh.update_issue_body.call_count, 3) + call3_args = self.mock_gh.update_issue_body.call_args_list[2][0] + self.assertIn( + "- [ ] Sync Changelog #124 | status=pending pr=#999", call3_args[1] + ) + self.assertIn( + "- [ ] Sync Changelog #125 | status=pending pr=#999", call3_args[1] + ) if __name__ == "__main__": diff --git a/tests/tools/private/release/release_issue_test.py b/tests/tools/private/release/release_issue_test.py index cfc6da6ae4..36f0dbd704 100644 --- a/tests/tools/private/release/release_issue_test.py +++ b/tests/tools/private/release/release_issue_test.py @@ -2,7 +2,9 @@ from tools.private.release.release_issue import ( add_backports_to_body, + add_sync_changelog_task_to_body, format_metadata_line, + parse_checklist_state, parse_metadata_line, ) @@ -96,6 +98,65 @@ def test_add_backports_to_body(self): """ self.assertEqual(updated_body.strip(), expected_body.strip()) + def test_add_sync_changelog_task_to_body(self): + body = """ +## Checklist +- [ ] Prepare Release +- [ ] Create Release branch +- [ ] Tag Final +""" + # Insert first task (should go before Tag Final) + body = add_sync_changelog_task_to_body(body, 124) + expected = """ +## Checklist +- [ ] Prepare Release +- [ ] Create Release branch +- [ ] Sync Changelog #124 +- [ ] Tag Final +""" + self.assertEqual(body.strip(), expected.strip()) + + # Insert second task (should go after the last Sync Changelog task) + body = add_sync_changelog_task_to_body(body, 125) + expected = """ +## Checklist +- [ ] Prepare Release +- [ ] Create Release branch +- [ ] Sync Changelog #124 +- [ ] Sync Changelog #125 +- [ ] Tag Final +""" + self.assertEqual(body.strip(), expected.strip()) + + # Insert duplicate (should be ignored) + body = add_sync_changelog_task_to_body(body, 124) + self.assertEqual(body.strip(), expected.strip()) + + def test_parse_checklist_state_with_sync_changelogs(self): + body = """ +## Checklist +- [x] Prepare Release | status=done pr=#122 commit=abcdef12 +- [x] Create Release branch | status=done branch=release/2.0 +- [ ] Sync Changelog #124 | status=pending pr=#125 +- [ ] Sync Changelog #126 +- [ ] Tag Final +""" + state = parse_checklist_state(body) + self.assertIn(124, state["sync_changelogs"]) + self.assertIn(126, state["sync_changelogs"]) + + task_124 = state["sync_changelogs"][124] + self.assertEqual(task_124.name, "Sync Changelog #124") + self.assertFalse(task_124.checked) + self.assertEqual(task_124.status, "pending") + self.assertEqual(task_124.pr, "#125") + + task_126 = state["sync_changelogs"][126] + self.assertEqual(task_126.name, "Sync Changelog #126") + self.assertFalse(task_126.checked) + self.assertIsNone(task_126.status) + self.assertIsNone(task_126.pr) + if __name__ == "__main__": unittest.main() diff --git a/tools/private/release/add_backports.py b/tools/private/release/add_backports.py index c25f509b9c..63c70af7d7 100644 --- a/tools/private/release/add_backports.py +++ b/tools/private/release/add_backports.py @@ -4,6 +4,7 @@ from tools.private.release.release_issue import ( add_backports_to_body, add_rc_task_to_body, + add_sync_changelog_task_to_body, parse_checklist_state, ) @@ -60,6 +61,8 @@ def run(self) -> int: body = self.gh.get_issue_body(issue_num) items_to_add = [{"ref": f"#{pr}"} for pr in resolved_prs] body = add_backports_to_body(body, items_to_add) + for pr in resolved_prs: + body = add_sync_changelog_task_to_body(body, pr) state = parse_checklist_state(body) rc_tags = state.get("rc_tags", {}) has_pending_rc = any( diff --git a/tools/private/release/changelog_news.py b/tools/private/release/changelog_news.py index b896af1f6b..e09b082ade 100644 --- a/tools/private/release/changelog_news.py +++ b/tools/private/release/changelog_news.py @@ -3,14 +3,6 @@ import pathlib import re -_UNRELEASED_TEMPLATE_BODY = """## Unreleased - -[unreleased]: https://github.com/bazel-contrib/rules_python/releases/tag/unreleased - -Unreleased changes are tracked as individual files in the [news/](./news) -directory, or view the [latest generated -changelog](https://rules-python.readthedocs.io/en/latest/changelog.html).""" - def _get_sub_category(content): """Extracts the sub-category in parentheses from the entry content.""" @@ -108,6 +100,34 @@ def generate_release_block(version, release_date, news_entries): return "\n".join(lines) +def _parse_simple_version(ver_str): + """Parses a version string (X-Y-Z or X.Y.Z) into a tuple of ints.""" + normalized = ver_str.replace("-", ".") + return tuple(int(x) for x in normalized.split(".")) + + +def _find_insertion_point(changelog_content, new_version): + """Finds the character index to insert the new version block. + + It should be inserted before the first version in the changelog that is + smaller than new_version. + """ + # Find all version anchors: {#vX-Y-Z} + matches = list(re.finditer(r"\{#v(?P\d+-\d+-\d+)\}", changelog_content)) + + parsed_new_ver = _parse_simple_version(new_version) + + for m in matches: + ver_str = m.group("ver") + parsed_ver = _parse_simple_version(ver_str) + if parsed_ver < parsed_new_ver: + return m.start() + + raise ValueError( + f"Could not find a version in CHANGELOG.md smaller than {new_version} to insert before." + ) + + def _add_news_to_changelog(input_path, output_path, version, entries, release_date): """Adds or merges news entries into CHANGELOG.md.""" input_path = pathlib.Path(input_path) @@ -253,26 +273,15 @@ def _add_news_to_changelog(input_path, output_path, version, entries, release_da " release section from news entries..." ) new_release_block = generate_release_block(version, release_date, entries) - replacement = ( - f"{{#unreleased}}\n{_UNRELEASED_TEMPLATE_BODY}\n\n{new_release_block}\n" - ) - - # Replace the active Unreleased section (from {#unreleased} to the first release anchor) - pattern = ( - r"(?P\{#unreleased\})(?P.*?)(?=\n\s*\{#v\d+-\d+-\d+\}|\Z)" - ) - if not re.search(pattern, changelog_content, re.DOTALL): - raise RuntimeError( - "Could not find active Unreleased section to replace in CHANGELOG.md" - ) + # Find insertion point + insertion_point = _find_insertion_point(changelog_content, version) - new_content = re.sub( - pattern, - replacement, - changelog_content, - count=1, - flags=re.DOTALL, + new_content = ( + changelog_content[:insertion_point] + + new_release_block + + "\n\n" + + changelog_content[insertion_point:] ) output_path.write_text(new_content, encoding="utf-8") @@ -284,9 +293,13 @@ def merge_new_into_changelog( version, release_date, delete_news=False, + news_files=None, ): """Merges news entries from news_dir into changelog_path and writes to output_path.""" - news_files = _get_news_files(news_dir) + if news_files is None: + news_files = _get_news_files(news_dir) + else: + news_files = [pathlib.Path(f) for f in news_files] entries = _parse_new_files(news_files) _add_news_to_changelog( input_path=changelog_path, @@ -297,7 +310,8 @@ def merge_new_into_changelog( ) if delete_news: for p in news_files: - p.unlink() + if p.exists(): + p.unlink() if news_files: print(f"Removed {len(news_files)} processed news files.") @@ -309,6 +323,7 @@ def update_changelog( output_path=None, news_dir="news", delete_news=True, + news_files=None, ): """Performs the version replacements in CHANGELOG.md.""" if output_path is None: @@ -320,4 +335,5 @@ def update_changelog( version=version, release_date=release_date, delete_news=delete_news, + news_files=news_files, ) diff --git a/tools/private/release/complete_sync_changelog.py b/tools/private/release/complete_sync_changelog.py new file mode 100644 index 0000000000..9c8d9028f0 --- /dev/null +++ b/tools/private/release/complete_sync_changelog.py @@ -0,0 +1,97 @@ +"""Subcommand to mark sync changelog tasks as complete.""" + +import re + +from tools.private.release.gh import GitHub +from tools.private.release.release_issue import ( + parse_checklist_state, + update_task_in_body, +) + + +class CompleteSyncChangelog: + """Class to mark sync changelog tasks as complete.""" + + def __init__(self, args, gh: GitHub): + self.args = args + self.gh = gh + + def run(self) -> int: + """Executes the complete-sync-changelog subcommand.""" + args = self.args + print(f"Completing sync changelog for PR #{args.pr}...") + + pr_info = self.gh.get_pr_info(args.pr) + if not pr_info or pr_info.get("state") != "MERGED": + state = pr_info.get("state", "UNKNOWN") + print(f"Error: PR #{args.pr} is not merged yet (state: {state}).") + return 1 + + # Resolve issue number from PR body using Release-Tracking-Issue: # + pr_body = pr_info.get("body") or "" + match = re.search(r"Release-Tracking-Issue:\s*#(\d+)", pr_body) + if not match: + print( + f"Error: Could not find 'Release-Tracking-Issue: #' in" + f" PR #{args.pr} body: {pr_body}" + ) + return 1 + + issue_num = int(match.group(1)) + print(f"Resolved tracking issue #{issue_num} from PR #{args.pr} body.") + + commit_sha = pr_info["mergeCommit"]["oid"] + short_commit = commit_sha[:8] + print( + f"PR #{args.pr} merged at commit {commit_sha}. Updating tracking issue..." + ) + + # Update checklist: mark all Sync Changelog tasks pointing to this PR as done + body = self.gh.get_issue_body(issue_num) + state = parse_checklist_state(body) + sync_changelogs = state.get("sync_changelogs", {}) + + updated_any = False + for pr_num, task in sync_changelogs.items(): + # Check if this task points to our merged PR + task_pr = task.metadata.get("pr") + if task_pr == f"#{args.pr}": + print(f"Marking task '{task.name}' as complete...") + metadata = { + "status": "done", + "pr": f"#{args.pr}", + "commit": short_commit, + } + body = update_task_in_body( + body, task.name, checked=True, metadata=metadata + ) + updated_any = True + + if not updated_any: + print(f"Warning: No 'Sync Changelog' tasks found pointing to PR #{args.pr}") + return 0 + + self.gh.update_issue_body(issue_num, body) + print("Sync changelog tasks marked complete successfully!") + return 0 + + @classmethod + def add_parser(cls, subparsers): + """Adds parser for complete-sync-changelog subcommand.""" + parser = subparsers.add_parser( + "complete-sync-changelog", + help="Mark the Sync Changelog tasks as complete in the tracking issue.", + ) + parser.add_argument( + "--pr", + type=int, + required=True, + help="The merged sync changelog PR number.", + ) + parser.set_defaults(command=cls.run_from_args) + + @classmethod + def run_from_args(cls, args): + """Instantiates and runs the command from parsed args.""" + gh = GitHub() + return cls(args, gh).run() diff --git a/tools/private/release/gh.py b/tools/private/release/gh.py index 8990ef5abf..92cc3ca5c4 100644 --- a/tools/private/release/gh.py +++ b/tools/private/release/gh.py @@ -263,24 +263,52 @@ def update_issue_body(self, issue_num: int, body: str) -> None: if os.path.exists(temp_path): os.unlink(temp_path) - def create_pr(self, version: str, issue_num: int) -> str: - """Creates a pull request for release preparation. + def create_pr( + self, + title: str, + body: str, + base: str = "main", + labels: list[str] | None = None, + ) -> str: + """Creates a pull request. Args: - version: The version being prepared. - issue_num: The associated tracking issue number. + title: The title of the PR. + body: The body of the PR. + base: The base branch to merge into (default: 'main'). + labels: Optional list of labels to add to the PR. Returns: The URL of the created PR. """ - output = self._gh_pr( + cmd = [ "create", - f"--title=Prepare release v{version}", - f"--body=Work towards #{issue_num}", - "--base=main", - ) + f"--title={title}", + f"--body={body}", + f"--base={base}", + ] + if labels: + for label in labels: + cmd.append(f"--label={label}") + output = self._gh_pr(*cmd) return output if output else "" + def enable_auto_merge(self, pr_num: int, method: str = "squash") -> None: + """Enables auto-merge for a PR. + + Args: + pr_num: The PR number. + method: The merge method ('squash', 'rebase', or 'merge'). + """ + cmd = ["merge", str(pr_num), "--auto"] + if method == "squash": + cmd.append("--squash") + elif method == "rebase": + cmd.append("--rebase") + elif method == "merge": + cmd.append("--merge") + self._gh_pr(*cmd, capture_output=False) + def get_open_pr(self, branch_name: str) -> dict | None: """Returns PR info if an open PR exists for the given branch. diff --git a/tools/private/release/git.py b/tools/private/release/git.py index 3e7f77f39a..4305de855f 100644 --- a/tools/private/release/git.py +++ b/tools/private/release/git.py @@ -80,7 +80,7 @@ def checkout( self._run_git(*cmd, capture_output=False) if should_reset_hard: - self.reset_hard(f"{track_remote}/{ref}") + self.reset_hard(reset_to=f"{track_remote}/{ref}") def add(self, *files: str) -> None: """Stages files for commit. @@ -191,13 +191,13 @@ def cherry_pick_abort(self) -> None: """Aborts an in-progress cherry-pick operation.""" self._run_git("cherry-pick", "--abort", capture_output=False) - def reset_hard(self, ref: str = "HEAD") -> None: + def reset_hard(self, *, reset_to: str = "HEAD") -> None: """Resets the index and working tree to a specific reference. Args: - ref: The git reference to reset to. Defaults to 'HEAD'. + reset_to: The git reference to reset to. Defaults to 'HEAD'. """ - self._run_git("reset", "--hard", ref, capture_output=False) + self._run_git("reset", "--hard", reset_to, capture_output=False) def status(self) -> str: """Returns the output of git status --porcelain. @@ -353,3 +353,47 @@ def get_remote_tags(self, remote: str) -> list[str]: if not tag.endswith("^{}"): tags.append(tag) return tags + + def get_modified_files(self, ref: str) -> list[str]: + """Returns a list of files modified in a given reference. + + Args: + ref: The git reference. + + Returns: + A list of file paths. + """ + output = self._run_git("show", "--name-only", "--format=", ref) + return [line for line in output.splitlines() if line.strip()] if output else [] + + def diff(self) -> str: + """Returns the diff of unstaged changes. + + Returns: + The diff output as a string. + """ + output = self._run_git("diff") + return output if output else "" + + def apply(self, patch_file: str) -> None: + """Applies a patch file. + + Args: + patch_file: The path to the patch file. + """ + self._run_git("apply", patch_file, capture_output=False) + + def apply_check(self, patch_file: str) -> bool: + """Verifies if a patch can be applied cleanly. + + Args: + patch_file: The path to the patch file. + + Returns: + True if the patch can be applied cleanly, False otherwise. + """ + try: + self._run_git("apply", "--check", patch_file, capture_output=False) + return True + except subprocess.CalledProcessError: + return False diff --git a/tools/private/release/prepare.py b/tools/private/release/prepare.py index 610f2369e7..fd19362bec 100644 --- a/tools/private/release/prepare.py +++ b/tools/private/release/prepare.py @@ -171,7 +171,11 @@ def run(self) -> int: ) pr_num = "" else: - pr_url = self.gh.create_pr(version, issue_num) + pr_url = self.gh.create_pr( + title=f"Prepare release v{version}", + body=f"Work towards #{issue_num}", + base="main", + ) pr_num = pr_url.split("/")[-1] print(f"Created Pull Request: {pr_url} (PR #{pr_num})") diff --git a/tools/private/release/process_backports.py b/tools/private/release/process_backports.py index c8a936eab5..27eb3226b1 100644 --- a/tools/private/release/process_backports.py +++ b/tools/private/release/process_backports.py @@ -2,6 +2,10 @@ import argparse import datetime +import hashlib +import os +import tempfile +from dataclasses import dataclass from typing import Any from tools.private.release import changelog_news @@ -11,6 +15,7 @@ RELEASE_TITLE_RE, add_backports_to_body, add_rc_task_to_body, + add_sync_changelog_task_to_body, parse_backports, parse_checklist_state, update_task_in_body, @@ -22,6 +27,20 @@ ) +@dataclass +class CherryPickAndUpdatePrsResult: + # List of PR references that failed to cherry-pick. + failed_prs: list[str] + # List of news files collected from the successful cherry-picks. + collected_news_files: list[str] + # List of PR numbers that were successfully cherry-picked. + successful_pr_nums: list[int] + # List of tuples mapping successful PR numbers to their version marker diffs. + collected_diffs: list[tuple[int, str]] + # The updated checklist body for the release tracking issue. + body: str + + class ProcessBackports: """Class to process pending backports.""" @@ -84,23 +103,35 @@ def _cherry_pick_and_update_prs( version, branch_name, next_rc_suffix, - ) -> tuple[list[str], str]: + ) -> CherryPickAndUpdatePrsResult: failed_prs = [] + collected_news_files = [] + successful_pr_nums = [] + collected_diffs = [] for sha in sorted_shas: item = sha_to_item[sha] print(f"Cherry-picking {item.pr_ref} / {sha}...") try: self.git.cherry_pick(sha) + # Collect news files before they are deleted by update_changelog + modified_files = self.git.get_modified_files("HEAD") + for f in modified_files: + if changelog_news.is_news_file(f): + collected_news_files.append(f) + + # Replace version markers FIRST to isolate diff + print(f"Replacing version markers for PR {item.pr_ref}...") + replace_version_next(version) + + # Get diff of unstaged changes (version marker replacement) + diff_content = self.git.diff() + # Perform news processing (merging news/ files into the changelog) print(f"Merging news fragments into changelog for PR {item.pr_ref}...") release_date = datetime.date.today().strftime("%Y-%m-%d") changelog_news.update_changelog(version, release_date) - # Replace version markers that might have been introduced by the backport - print(f"Replacing version markers for PR {item.pr_ref}...") - replace_version_next(version) - # Stage changelog changes, news/ deletions, and version placeholder updates self.git.add_modified_and_deleted() @@ -111,6 +142,16 @@ def _cherry_pick_and_update_prs( new_msg = f"{current_msg.strip()}\n\nWork towards #{issue}" self.git.commit(new_msg, amend=True) + try: + pr_num = self.gh.resolve_pr_number(item.pr_ref) + if diff_content: + collected_diffs.append((pr_num, diff_content)) + successful_pr_nums.append(pr_num) + except Exception as e: + print( + f"Warning: Failed to resolve PR number for {item.pr_ref}: {e}" + ) + if not dry_run: # Push amended commit self.git.push(remote, branch_name) @@ -177,7 +218,180 @@ def _cherry_pick_and_update_prs( f"ERROR: Failed to update tracking issue for" f" failed PR {item.pr_ref}: {e}" ) - return failed_prs, body + return CherryPickAndUpdatePrsResult( + failed_prs=failed_prs, + collected_news_files=collected_news_files, + successful_pr_nums=successful_pr_nums, + collected_diffs=collected_diffs, + body=body, + ) + + def _sync_changelog_to_main( + self, + version: str, + collected_news_files: list[str], + successful_pr_nums: list[int], + collected_diffs: list[tuple[int, str]], + release_branch: str, + ) -> None: + args = self.args + sorted_prs = sorted(successful_pr_nums) + prs_str = ",".join(str(n) for n in sorted_prs) + prs_hash = hashlib.sha256(prs_str.encode()).hexdigest()[:7] + + main_branch = "main" + backport_branch = f"prepare-{version}-backports-{prs_hash}" + + print(f"Syncing changelog to {main_branch} via branch {backport_branch}...") + + self.git.fetch(args.remote, refspec=main_branch) + self.git.checkout(main_branch, track_remote=args.remote) + main_start_sha = self.git.get_commit_sha("HEAD") + + failed_version_sync_prs = [] + try: + if args.dry_run: + print( + f"[DRY RUN] Would create and checkout branch {backport_branch} from {main_branch}" + ) + else: + if self.git.branch_exists(backport_branch): + self.git.checkout(backport_branch) + self.git.reset_hard(reset_to=main_branch) + else: + self.git.checkout(backport_branch, create_branch=True) + + print( + f"Updating CHANGELOG.md and removing news files on {backport_branch}..." + ) + release_date = datetime.date.today().strftime("%Y-%m-%d") + changelog_news.update_changelog( + version, + release_date, + news_files=collected_news_files, + delete_news=True, + ) + + # Apply version marker diffs + failed_version_sync_prs = self._apply_version_marker_diffs(collected_diffs) + + if args.dry_run: + print( + f"[DRY RUN] Would commit: 'chore(release): sync changelog for v{version} backports'" + ) + print(f"[DRY RUN] Would push {backport_branch} to {args.remote}") + print( + f"[DRY RUN] Would create PR to {main_branch} with label 'type: sync-changelog'" + ) + print( + f"[DRY RUN] Would update tracking issue #{args.issue} checklist tasks 'Sync Changelog #' to PENDING" + ) + print("[DRY RUN] Diff of changes:") + print(self.git.status()) + else: + self.git.add_modified_and_deleted() + self.git.commit( + f"chore(release): sync changelog for v{version} backports" + ) + self.git.push( + args.remote, backport_branch, set_upstream=True, force=True + ) + + pr_title = f"chore(release): sync changelog for v{version} backports" + pr_body_lines = [ + "Updates CHANGELOG.md and removes news files for backports:", + ] + for pr_num in sorted_prs: + pr_body_lines.append(f"- #{pr_num}") + + if failed_version_sync_prs: + pr_body_lines.append("") + pr_body_lines.append( + "Warning: These PRs failed to update their version markers:" + ) + for pr_num in sorted(failed_version_sync_prs): + pr_body_lines.append(f"- #{pr_num}") + + pr_body_lines.append("") + pr_body_lines.append(f"Work towards #{args.issue}") + pr_body_lines.append(f"Release-Tracking-Issue: #{args.issue}") + pr_body = "\n".join(pr_body_lines) + + print(f"Creating PR to {main_branch}...") + pr_url = self.gh.create_pr( + title=pr_title, + body=pr_body, + base=main_branch, + labels=["type: sync-changelog"], + ) + print(f"Created PR: {pr_url}") + + try: + pr_num = int(pr_url.split("/")[-1]) + print(f"Enabling auto-merge for PR #{pr_num}...") + self.gh.enable_auto_merge(pr_num) + + print( + f"Updating tracking issue #{args.issue} checklist with" + " Sync Changelog tasks..." + ) + issue_body = self.gh.get_issue_body(args.issue) + for pr in successful_pr_nums: + task_name = f"Sync Changelog #{pr}" + metadata = {"status": "pending", "pr": f"#{pr_num}"} + issue_body = update_task_in_body( + issue_body, + task_name, + checked=False, + metadata=metadata, + ) + self.gh.update_issue_body(args.issue, issue_body) + except Exception as e: + print( + f"Warning: Failed to update tracking issue or enable" + f" auto-merge: {e}" + ) + finally: + if args.dry_run: + self.git.reset_hard(reset_to=main_start_sha) + self.git.checkout(release_branch) + + def _apply_version_marker_diffs( + self, + collected_diffs: list[tuple[int, str]], + ) -> list[int]: + """Applies version marker diffs on main branch and returns failed PR numbers.""" + args = self.args + failed_version_sync_prs = [] + if not collected_diffs: + return failed_version_sync_prs + + with tempfile.TemporaryDirectory() as temp_dir: + print(f"Applying {len(collected_diffs)} version marker patches...") + for pr_num, diff_content in collected_diffs: + if args.dry_run: + print( + f"[DRY RUN] Would check and apply version marker patch for PR #{pr_num}" + ) + + patch_filepath = os.path.join(temp_dir, f"{pr_num}.patch") + with open(patch_filepath, "w", encoding="utf-8") as f: + f.write(diff_content) + + if self.git.apply_check(patch_filepath): + if args.dry_run: + print( + f"[DRY RUN] Version marker patch for PR #{pr_num} applies cleanly." + ) + else: + print(f"Applying version marker patch for PR #{pr_num}...") + self.git.apply(patch_filepath) + else: + print( + f"Warning: Version marker patch for PR #{pr_num} could not be applied cleanly to main. Skipping." + ) + failed_version_sync_prs.append(pr_num) + return failed_version_sync_prs def run(self) -> int: """Executes the process-backports subcommand.""" @@ -223,6 +437,14 @@ def _run_internal(self) -> int: print(f"Adding backports {items_to_add} to tracking issue #{args.issue}...") try: body = add_backports_to_body(body, items_to_add) + for item in items_to_add: + if ( + "metadata" in item + and item["metadata"].get("status") == "error-invalid-pr" + ): + continue + pr_num = int(item["ref"].lstrip("#")) + body = add_sync_changelog_task_to_body(body, pr_num) state = parse_checklist_state(body) rc_tags = state.get("rc_tags", {}) has_pending_rc = any( @@ -318,8 +540,11 @@ def _run_internal(self) -> int: self.git.checkout(branch_name, track_remote=args.remote) start_sha = self.git.get_commit_sha("HEAD") + collected_news_files = [] + successful_pr_nums = [] + collected_diffs = [] try: - new_failed_prs, body = self._cherry_pick_and_update_prs( + result = self._cherry_pick_and_update_prs( sorted_shas, sha_to_item, body, @@ -330,11 +555,24 @@ def _run_internal(self) -> int: branch_name, next_rc_suffix, ) - failed_prs.extend(new_failed_prs) + failed_prs.extend(result.failed_prs) + collected_news_files.extend(result.collected_news_files) + successful_pr_nums.extend(result.successful_pr_nums) + collected_diffs.extend(result.collected_diffs) + body = result.body finally: if args.dry_run: print(f"[DRY RUN] Resetting branch {branch_name} to {start_sha}") - self.git.reset_hard(start_sha) + self.git.reset_hard(reset_to=start_sha) + + if successful_pr_nums: + self._sync_changelog_to_main( + version, + collected_news_files, + successful_pr_nums, + collected_diffs, + branch_name, + ) if failed_prs: print("ERROR: One or more cherry-picks/resolutions failed:") diff --git a/tools/private/release/release.py b/tools/private/release/release.py index 8ad9c2e33e..f008c093b1 100644 --- a/tools/private/release/release.py +++ b/tools/private/release/release.py @@ -6,6 +6,7 @@ from tools.private.release.add_backports import AddBackports from tools.private.release.complete_prepare import CompletePrepare +from tools.private.release.complete_sync_changelog import CompleteSyncChangelog from tools.private.release.create_rc import CreateRc from tools.private.release.create_release_branch import CreateReleaseBranch from tools.private.release.create_release_issue import CreateReleaseIssue @@ -20,6 +21,7 @@ CreateReleaseIssue, Prepare, CompletePrepare, + CompleteSyncChangelog, CreateReleaseBranch, AddBackports, ProcessBackports, diff --git a/tools/private/release/release_issue.py b/tools/private/release/release_issue.py index 359249d11c..220348bd47 100644 --- a/tools/private/release/release_issue.py +++ b/tools/private/release/release_issue.py @@ -161,6 +161,7 @@ def parse_checklist_state(body): "create_branch": ReleaseTask("Create Release branch", False), "tag_final": ReleaseTask("Tag Final", False), "rc_tags": {}, # Dynamically mapped: int -> ReleaseTask + "sync_changelogs": {}, # Dynamically mapped: int -> ReleaseTask } lines = body.splitlines() @@ -214,6 +215,19 @@ def parse_checklist_state(body): commit=meta.get("commit"), metadata=meta, ) + else: + # Match Sync Changelog # + sync_match = re.match(r"Sync Changelog #(\d+)", name, re.IGNORECASE) + if sync_match: + pr_num = int(sync_match.group(1)) + state["sync_changelogs"][pr_num] = ReleaseTask( + name=name, + checked=checked, + status=meta.get("status"), + pr=meta.get("pr"), + commit=meta.get("commit"), + metadata=meta, + ) return state @@ -325,3 +339,50 @@ def add_rc_task_to_body(body: str, rc_num: int) -> str: lines.insert(last_rc_idx + 1, new_task_line) return "\n".join(lines) + + +def add_sync_changelog_task_to_body(body: str, pr_num: int) -> str: + """Adds a new 'Sync Changelog #' task to the checklist in the issue body.""" + body = body.replace("\r\n", "\n") + + # Check if already exists + task_name = f"Sync Changelog #{pr_num}" + lines = body.splitlines() + for line in lines: + parsed = parse_metadata_line(line) + if parsed and parsed["name"].lower() == task_name.lower(): + print(f"Task '{task_name}' already exists. Skipping.") + return body + + # Find the index of the last "Sync Changelog #" line + last_sync_idx = -1 + for i, line in enumerate(lines): + parsed = parse_metadata_line(line) + if parsed and re.match(r"Sync Changelog #\d+", parsed["name"], re.IGNORECASE): + last_sync_idx = i + + if last_sync_idx == -1: + # If no Sync Changelog task found, insert before "Tag Final" + for i, line in enumerate(lines): + parsed = parse_metadata_line(line) + if parsed and parsed["name"].lower() == "tag final": + last_sync_idx = i - 1 + break + + if last_sync_idx == -1: + # If "Tag Final" not found, insert after "Create Release branch" + for i, line in enumerate(lines): + parsed = parse_metadata_line(line) + if parsed and parsed["name"].lower() == "create release branch": + last_sync_idx = i + break + + if last_sync_idx == -1: + raise ValueError( + "Could not find a place to insert the new Sync Changelog task." + ) + + new_task_line = f"- [ ] Sync Changelog #{pr_num}" + lines.insert(last_sync_idx + 1, new_task_line) + + return "\n".join(lines) From b5d754e82b5b8003a9374bf7df655d593e7b9ee3 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Sun, 5 Jul 2026 21:28:07 -0700 Subject: [PATCH 824/922] chore: remove stale process-backports-before-rc news file (#3904) This news file was left over from a previous release process and is no longer needed. Removed the file `news/process-backports-before-rc.changed.md`. --- news/process-backports-before-rc.changed.md | 3 --- 1 file changed, 3 deletions(-) delete mode 100644 news/process-backports-before-rc.changed.md diff --git a/news/process-backports-before-rc.changed.md b/news/process-backports-before-rc.changed.md deleted file mode 100644 index 5591b700ce..0000000000 --- a/news/process-backports-before-rc.changed.md +++ /dev/null @@ -1,3 +0,0 @@ -Changed `/create-rc` command to automatically process pending backports before -creating the RC, and react with a thumbs-down emoji on failure if triggered by -a comment. From 89eba163c82ad944880c273d9eb882ceea614382 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Mon, 6 Jul 2026 09:46:49 -0700 Subject: [PATCH 825/922] chore(release): fix invalid workflow file syntax (#3908) Quote the 'if' condition in release_create_release_branch.yaml to fix YAML syntax error. --- .github/workflows/release_create_release_branch.yaml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.github/workflows/release_create_release_branch.yaml b/.github/workflows/release_create_release_branch.yaml index 4c021814d0..4cbabc7d12 100644 --- a/.github/workflows/release_create_release_branch.yaml +++ b/.github/workflows/release_create_release_branch.yaml @@ -11,7 +11,7 @@ permissions: jobs: cut_branch: # Run only if the issue has the type: release label - if: contains(github.event.issue.labels.*.name, 'type: release') + if: "contains(github.event.issue.labels.*.name, 'type: release')" runs-on: ubuntu-latest steps: - name: Checkout repository From b443ec787b5f7c24e10b5be487756a0a160ddb45 Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Tue, 7 Jul 2026 10:43:41 +0900 Subject: [PATCH 826/922] build(deps): bump astral-sh/ruff-action from 4.0.0 to 4.1.0 (#3910) Bumps [astral-sh/ruff-action](https://github.com/astral-sh/ruff-action) from 4.0.0 to 4.1.0.
Commits

[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=astral-sh/ruff-action&package-manager=github_actions&previous-version=4.0.0&new-version=4.1.0)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) ---
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot show ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> --- .github/workflows/ci.yaml | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/.github/workflows/ci.yaml b/.github/workflows/ci.yaml index daff8938e7..e7b2a4d956 100644 --- a/.github/workflows/ci.yaml +++ b/.github/workflows/ci.yaml @@ -32,12 +32,12 @@ jobs: runs-on: ubuntu-latest steps: - uses: actions/checkout@v7 - - uses: astral-sh/ruff-action@v4.0.0 + - uses: astral-sh/ruff-action@v4.1.0 with: # Keep in sync with .pre-commit-config.yaml version: 0.15.14 args: check --extend-exclude testdata - - uses: astral-sh/ruff-action@v4.0.0 + - uses: astral-sh/ruff-action@v4.1.0 with: version: 0.15.14 args: format --check --exclude testdata From 00922ad2dcc7cf7aa0c2fa0979e67cd89c07df51 Mon Sep 17 00:00:00 2001 From: Ignas Anikevicius <240938+aignas@users.noreply.github.com> Date: Thu, 9 Jul 2026 11:17:18 +0900 Subject: [PATCH 827/922] fix(pip): parse the index_url from uv.lock file (#3906) This is so that the purl can be generated correctly and we pass the right index_url parameter to the whl_library file. Work towards #1975 --------- Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> Co-authored-by: Richard Levasseur --- news/3906.fixed.md | 3 ++ python/private/pypi/parse_requirements.bzl | 9 ++++- .../parse_requirements_tests.bzl | 36 +++++++++---------- 3 files changed, 29 insertions(+), 19 deletions(-) create mode 100644 news/3906.fixed.md diff --git a/news/3906.fixed.md b/news/3906.fixed.md new file mode 100644 index 0000000000..64a16fd9c8 --- /dev/null +++ b/news/3906.fixed.md @@ -0,0 +1,3 @@ +(pypi) correctly parse the `index_url` for each wheel so that the source registry is forwarded to +the {obj}`whl_library`. This is so that the `purl` for `package_metadata` can be correctly +constructed. diff --git a/python/private/pypi/parse_requirements.bzl b/python/private/pypi/parse_requirements.bzl index fd0399fc86..be3ad9a5e5 100644 --- a/python/private/pypi/parse_requirements.bzl +++ b/python/private/pypi/parse_requirements.bzl @@ -176,6 +176,13 @@ def _parse_uv_lock_json(uv_lock, all_platforms, logger, extra_pip_args = None, p "versions": {}, }) entry["versions"][version] = None + source = pkg.get("source") or {} + registry = (source.get("registry") or "").rstrip("/") + if registry.startswith("http://") or registry.startswith("https://"): + index_url = "{}/{}".format(registry, norm_name.replace("_", "-")) + else: + index_url = "" + entry["index_url"] = index_url pkg_extras = sorted(extras_map.get(name, [])) extra_str = "[{}]".format(",".join(pkg_extras)) if pkg_extras else "" @@ -287,7 +294,7 @@ def _parse_uv_lock_json(uv_lock, all_platforms, logger, extra_pip_args = None, p name = norm_name, is_exposed = True, is_multiple_versions = len(versions) > 1, - index_url = "", + index_url = info["index_url"], srcs = info["resolved_srcs"], ) ret.append(item) diff --git a/tests/pypi/parse_requirements/parse_requirements_tests.bzl b/tests/pypi/parse_requirements/parse_requirements_tests.bzl index 8dd35c2260..576a31ae5d 100644 --- a/tests/pypi/parse_requirements/parse_requirements_tests.bzl +++ b/tests/pypi/parse_requirements/parse_requirements_tests.bzl @@ -1052,7 +1052,7 @@ def _test_uv_lock_consistent(env): env.expect.that_collection(got).contains_exactly([ struct( name = "foo", - index_url = "", + index_url = "https://pypi.org/simple/foo", is_exposed = True, is_multiple_versions = False, srcs = [ @@ -1080,7 +1080,7 @@ def _test_uv_lock_primary_source(env): env.expect.that_collection(got).contains_exactly([ struct( name = "foo", - index_url = "", + index_url = "https://pypi.org/simple/foo", is_exposed = True, is_multiple_versions = False, srcs = [ @@ -1108,7 +1108,7 @@ def _test_uv_lock_primary_source_multiple_versions(env): env.expect.that_collection(got).contains_exactly([ struct( name = "foo", - index_url = "", + index_url = "https://pypi.org/simple/foo", is_exposed = True, is_multiple_versions = True, srcs = [ @@ -1146,7 +1146,7 @@ def _test_uv_lock_primary_source_with_extras(env): env.expect.that_collection(got).contains_exactly([ struct( name = "foo", - index_url = "", + index_url = "https://pypi.org/simple/foo", is_exposed = True, is_multiple_versions = False, srcs = [ @@ -1174,7 +1174,7 @@ def _test_uv_lock_primary_source_includes_virtual(env): env.expect.that_collection(got).contains_exactly([ struct( name = "foo", - index_url = "", + index_url = "https://pypi.org/simple/foo", is_exposed = True, is_multiple_versions = False, srcs = [ @@ -1212,7 +1212,7 @@ def _test_uv_lock_cross_consistent(env): env.expect.that_collection(got).contains_exactly([ struct( name = "foo", - index_url = "", + index_url = "https://pypi.org/simple/foo", is_exposed = True, is_multiple_versions = False, srcs = [ @@ -1268,7 +1268,7 @@ def _test_uv_lock_rules_python_pkg_not_skipped(env): env.expect.that_collection(got).contains_exactly([ struct( name = "rules_python", - index_url = "", + index_url = "https://pypi.org/simple/rules-python", is_exposed = True, is_multiple_versions = False, srcs = [ @@ -1301,7 +1301,7 @@ def _test_uv_lock_no_consistency_check(env): env.expect.that_collection(got).contains_exactly([ struct( name = "foo", - index_url = "", + index_url = "https://pypi.org/simple/foo", is_exposed = True, is_multiple_versions = False, srcs = [ @@ -1329,7 +1329,7 @@ def _test_uv_lock_multiple_packages(env): env.expect.that_collection(got).contains_exactly([ struct( name = "bar", - index_url = "", + index_url = "https://pypi.org/simple/bar", is_exposed = True, is_multiple_versions = False, srcs = [ @@ -1347,7 +1347,7 @@ def _test_uv_lock_multiple_packages(env): ), struct( name = "foo", - index_url = "", + index_url = "https://pypi.org/simple/foo", is_exposed = True, is_multiple_versions = False, srcs = [ @@ -1376,7 +1376,7 @@ def _test_uv_lock_with_extra_pip_args(env): env.expect.that_collection(got).contains_exactly([ struct( name = "foo", - index_url = "", + index_url = "https://pypi.org/simple/foo", is_exposed = True, is_multiple_versions = False, srcs = [ @@ -1408,7 +1408,7 @@ def _test_uv_lock_multi_os_with_requirements(env): env.expect.that_collection(got).contains_exactly([ struct( name = "foo", - index_url = "", + index_url = "https://pypi.org/simple/foo", is_exposed = True, is_multiple_versions = False, srcs = [ @@ -1436,7 +1436,7 @@ def _test_uv_lock_extras_optional_deps(env): env.expect.that_collection(got).contains_exactly([ struct( name = "foo", - index_url = "", + index_url = "https://pypi.org/simple/foo", is_exposed = True, is_multiple_versions = False, srcs = [ @@ -1464,7 +1464,7 @@ def _test_uv_lock_extras_dep_edge(env): env.expect.that_collection(got).contains_exactly([ struct( name = "bar", - index_url = "", + index_url = "https://pypi.org/simple/bar", is_exposed = True, is_multiple_versions = False, srcs = [ @@ -1482,7 +1482,7 @@ def _test_uv_lock_extras_dep_edge(env): ), struct( name = "foo", - index_url = "", + index_url = "https://pypi.org/simple/foo", is_exposed = True, is_multiple_versions = False, srcs = [ @@ -1517,7 +1517,7 @@ def _test_uv_lock_wheel_dedup_single_version(env): env.expect.that_collection(got).contains_exactly([ struct( name = "foo", - index_url = "", + index_url = "https://pypi.org/simple/foo", is_exposed = True, is_multiple_versions = False, srcs = [ @@ -1563,7 +1563,7 @@ def _test_uv_lock_wheel_dedup_resolution_markers(env): env.expect.that_collection(got).contains_exactly([ struct( name = "foo", - index_url = "", + index_url = "https://pypi.org/simple/foo", is_exposed = True, is_multiple_versions = True, srcs = [ @@ -1601,7 +1601,7 @@ def _test_uv_lock_requires_dist_extras(env): env.expect.that_collection(got).contains_exactly([ struct( name = "foo", - index_url = "", + index_url = "https://pypi.org/simple/foo", is_exposed = True, is_multiple_versions = False, srcs = [ From fc18dc15138296efc0a3ac5ccc5b1586717382b7 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Wed, 8 Jul 2026 20:23:24 -0700 Subject: [PATCH 828/922] chore: automated code review using antigravity (#3905) Free gemini code assist is being disabled in a few weeks, and I've found the ai reviews useful enough to want to keep them, so replace it with a custom agent based upon the antigravity sdk that does code reviews. I've added a Gemini API key to the project for it to use. It's free-tier level quota, so is relatively limited, hence reviews are limited to project maintainer PRsor when a maintainer comments `/review`. --- .github/workflows/automated_pr_review.yaml | 61 +++++++++++++++++++ tools/private/reviewbot/antigravity_review.py | 48 +++++++++++++++ tools/private/reviewbot/prompt.txt | 3 + .../reviewbot/skills/review-pr/SKILL.md | 52 ++++++++++++++++ 4 files changed, 164 insertions(+) create mode 100644 .github/workflows/automated_pr_review.yaml create mode 100644 tools/private/reviewbot/antigravity_review.py create mode 100644 tools/private/reviewbot/prompt.txt create mode 100644 tools/private/reviewbot/skills/review-pr/SKILL.md diff --git a/.github/workflows/automated_pr_review.yaml b/.github/workflows/automated_pr_review.yaml new file mode 100644 index 0000000000..23d4369b2a --- /dev/null +++ b/.github/workflows/automated_pr_review.yaml @@ -0,0 +1,61 @@ +name: Automated Code Review + +# TODO: Eventually, use pull_request_target instead of pull_request. +# pull_request_target runs in the base branch context and has access +# to secrets (like GEMINI_API_KEY) even for fork PRs. +# Using pull_request for now during setup/testing. +on: + pull_request: + types: [opened] + pull_request_review_comment: + types: [created] + +permissions: + contents: read + pull-requests: read + +jobs: + review: + runs-on: ubuntu-latest + # Trigger only if: + # 1. It is a pull_request event, it is NOT a draft, and the author is a maintainer + # (OWNER, MEMBER, or COLLABORATOR). + # 2. OR it is a pull_request_review_comment event, the comment body has a line starting with "/review", + # and the commenter is a maintainer. + if: > + (github.event_name == 'pull_request' && !github.event.pull_request.draft && + contains(fromJson('["OWNER", "MEMBER", "COLLABORATOR"]'), github.event.pull_request.author_association)) || + (github.event_name == 'pull_request_review_comment' && + (startsWith(github.event.comment.body, '/review') || contains(github.event.comment.body, '\n/review')) && + contains(fromJson('["OWNER", "MEMBER", "COLLABORATOR"]'), github.event.comment.author_association)) + steps: + - name: Checkout PR Branch + uses: actions/checkout@v7 + with: + ref: ${{ github.event.pull_request.head.sha }} + persist-credentials: false + + - name: Checkout Reviewbot (Base Branch) + uses: actions/checkout@v7 + with: + sparse-checkout: | + tools/private/reviewbot + path: reviewbot + + - name: Install uv + uses: astral-sh/setup-uv@v8.3.0 + + - name: Set up Python + run: uv python install 3.13 + + - name: Install Dependencies + run: | + uv pip install google-antigravity requests + + - name: Run Antigravity Review + env: + GEMINI_API_KEY: ${{ secrets.GEMINI_API_KEY }} + GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} + run: | + uv run reviewbot/tools/private/reviewbot/antigravity_review.py \ + --prompt reviewbot/tools/private/reviewbot/prompt.txt diff --git a/tools/private/reviewbot/antigravity_review.py b/tools/private/reviewbot/antigravity_review.py new file mode 100644 index 0000000000..1bb23283be --- /dev/null +++ b/tools/private/reviewbot/antigravity_review.py @@ -0,0 +1,48 @@ +import argparse +import asyncio +from pathlib import Path + +from google.antigravity import Agent, CapabilitiesConfig, LocalAgentConfig + + +def parse_args(): + parser = argparse.ArgumentParser() + parser.add_argument("--prompt", required=True, help="Path to prompt file") + return parser.parse_args() + + +async def main(): + args = parse_args() + + # Read prompt file + prompt = Path(args.prompt).read_text() + + # Initialize the Antigravity Agent in read-only mode for security. + # Register the review-pr skill from the local reviewbot folder. + config = LocalAgentConfig( + skills_paths=["tools/private/reviewbot/skills/review-pr"], + capabilities=CapabilitiesConfig( + allow_filesystem_read=True, + allow_filesystem_write=False, + allow_network=False, + ), + ) + + # General coordinator instructions for the reviewer agent. + system_instructions = ( + "You are a code review assistant. Use your available skills to perform " + "reviews on pull requests." + ) + + async with Agent(config, system_instructions=system_instructions) as agent: + response = await agent.chat(prompt) + report = await response.text() + + print("--- REVIEW REPORT GENERATED ---") + print(report) + + # TODO: Use GITHUB_TOKEN to post the report back to the PR comments. + + +if __name__ == "__main__": + asyncio.run(main()) diff --git a/tools/private/reviewbot/prompt.txt b/tools/private/reviewbot/prompt.txt new file mode 100644 index 0000000000..e406164af1 --- /dev/null +++ b/tools/private/reviewbot/prompt.txt @@ -0,0 +1,3 @@ +Use the review-pr skill to review the files modified in this pull request. +Summarize your findings and suggest specific, actionable improvements. +Group your findings into clear, descriptive nits or suggestions. diff --git a/tools/private/reviewbot/skills/review-pr/SKILL.md b/tools/private/reviewbot/skills/review-pr/SKILL.md new file mode 100644 index 0000000000..18c998f9ed --- /dev/null +++ b/tools/private/reviewbot/skills/review-pr/SKILL.md @@ -0,0 +1,52 @@ +--- +name: review-pr +description: Perform a read-only code review on a pull request. +--- + +# review-pr + +You are an expert Starlark, Python, and Bazel code reviewer. Analyze the changed files for +correctness, edge cases, and performance. Focus strictly on logical +correctness, concurrency safety, system architecture, performance bottlenecks, +and resource management. Do not comment on style nits or formatting issues +that an automated formatter can handle. Be constructive and concise. + +For every issue or improvement you identify, you MUST output the finding in the +GitHub Actions workflow command warning format. Specify the exact file path +and line numbers that the comment applies to. + +Format each finding exactly as a single line to stdout matching this template: +`::warning file={file_path},line={line_number},endLine={end_line},title={category}::{comment_body}` + +Where: +* `file_path` is the relative file path from the repository root. +* `line_number` is the starting line number in the file where the comment applies. +* `end_line` is the ending line number in the file where the comment applies (equal to line_number if the issue is on a single line). +* `category` is a short tag for the type of issue (e.g., "Error Handling", "Correctness", "Performance"). +* `comment_body` is your constructive and concise feedback. + +Do not write any markdown commentary outside of these GHA command formatted lines. + +Follow these checklists during your review: + +### General Quality & Architecture Checklist +* **PR Description Audit**: Verify the description contains the Why + (business/technical reason), a brief high level overview of changes, + Issue/Bug Link, and explicit Testing Evidence. +* **Separation of Concerns**: Suggest extracting large hardcoded data structures + (e.g., massive templates, complex regexes) to resource files. +* **Logic Correctness**: Verify calculations, negative values, division-by-zero, + and null safety before member access. +* **Error Handling**: Flag silent failures (e.g., empty except blocks) and + unconditional defaults that override configs. +* **Deterministic Operations**: Sort collections/keys to guarantee + reproducible/deterministic execution. + +### Skeptical Critic (Adversarial Specialist Review) +* **Dynamic Filtering**: Filter the PR diff and run only the specialist checks + that have relevant files changed (e.g. skip the C++ checks if only Python + files are modified). +* **Specialist Review Pillars**: Run parallel audits focusing on: + 1. Crash Regression: Null safety and resource lifecycle. + 2. Performance & Latency: Thread bottlenecks, locks, and network calls. + 3. Test Integrity: Coverage validity, change detectors defense. From 84d313bdc5bf09929a00ec242cce4fb0bc890cd8 Mon Sep 17 00:00:00 2001 From: Artemy Granat Date: Thu, 9 Jul 2026 09:55:44 +0300 Subject: [PATCH 829/922] fix: add explicit `data` attribute to `compile_pip_requirements` and forward it to the generated `py_binary` (#3865) As stated in #3858, the `compile_pip_requirements` macro accepts a `data` attribute, but this attribute is not forwarded to the internal `py_binary` target generated for the `.update` target. As a result, using `$(location :pip_cert)` inside `extra_args` fails during analysis because the generated `py_binary` does not declare `:pip_cert` as a prerequisite. Before: ``` ERROR: /.../BUILD.bazel:8:25: in args attribute of py_binary rule //:requirements.update: label '//:pip_cert' in $(location) expression is not a declared prerequisite of this rule. Since this rule was created by the macro 'pip_compile', the error might have been caused by the macro implementation ERROR: /.../BUILD.bazel:8:25: Analysis of target '//:requirements.update' (config: 7f8856b) failed ERROR: Analysis of target '//:requirements.update' failed; build aborted ``` After: ``` INFO: Analyzed target //:requirements.update (87 packages loaded, 4078 targets configured). INFO: Found 1 target... Target //:requirements.update up-to-date: bazel-bin/requirements.update INFO: Elapsed time: 6.240s, Critical Path: 0.11s INFO: 1 process: 11 action cache hit, 1 internal. INFO: Build completed successfully, 1 total action INFO: Running command line: bazel-bin/requirements.update '--src=rules_python_pip_parse_example/requirements.in' rules_python_pip_parse_example/requirements_lock.txt //:requirements '--resolver=backtracking' --allow-unsafe --generate-hashes '--cert=./pip.cert' ``` Closes #3858 --------- Co-authored-by: Ignas Anikevicius <240938+aignas@users.noreply.github.com> Co-authored-by: Richard Levasseur --- news/3858.fixed.md | 3 +++ python/private/pypi/pip_compile.bzl | 17 +++++++++++++++-- .../compile_pip_requirements/BUILD.bazel | 13 +++++++++++++ .../compile_pip_requirements/WORKSPACE | 5 +++++ 4 files changed, 36 insertions(+), 2 deletions(-) create mode 100644 news/3858.fixed.md diff --git a/news/3858.fixed.md b/news/3858.fixed.md new file mode 100644 index 0000000000..dd991e4654 --- /dev/null +++ b/news/3858.fixed.md @@ -0,0 +1,3 @@ +(compile_pip_requirements) Add the explicit `data` attribute and forward it +directly to the generated `py_binary`, so files passed via `data` can be +referenced from `extra_args` using `$(location ...)`. diff --git a/python/private/pypi/pip_compile.bzl b/python/private/pypi/pip_compile.bzl index 3ef2cdb39c..58f7ba3a59 100644 --- a/python/private/pypi/pip_compile.bzl +++ b/python/private/pypi/pip_compile.bzl @@ -22,6 +22,15 @@ make it possible to have multiple tools inside the `pypi` directory load("//python:py_binary.bzl", _py_binary = "py_binary") load("//python:py_test.bzl", _py_test = "py_test") +# The `data` attribute does not allow duplicate labels, but user-provided `data` +# can overlap with labels added from attributes like `src`. +def _dedupe_data(data): + res = [] + for d in data: + if d not in res: + res.append(d) + return res + def pip_compile( name, srcs = None, @@ -39,6 +48,7 @@ def pip_compile( visibility = ["//visibility:private"], tags = None, constraints = [], + data = [], **kwargs): """Generates targets for managing pip dependencies with pip-compile (piptools). @@ -81,6 +91,7 @@ def pip_compile( tags: tagging attribute common to all build rules, passed to both the _test and .update rules. visibility: passed to both the _test and .update rules. constraints: a list of files containing constraints to pass to pip-compile with `--constraint`. + data: A list of labels to include as part of the `data` attribute in the generated `py_binary`. **kwargs: other bazel attributes passed to the "_test" rule. """ if len([x for x in [srcs, src, requirements_in] if x != None]) > 1: @@ -95,16 +106,18 @@ def pip_compile( requirements_txt = name + ".txt" if requirements_txt == None else requirements_txt + data = data or [] + # "Default" target produced by this macro # Allow a compile_pip_requirements rule to include another one in the data # for a requirements file that does `-r ../other/requirements.txt` native.filegroup( name = name, - srcs = kwargs.pop("data", []) + [requirements_txt], + srcs = data + [requirements_txt], visibility = visibility, ) - data = [name, requirements_txt] + srcs + [f for f in (requirements_linux, requirements_darwin, requirements_windows) if f != None] + constraints + data = _dedupe_data(data + [name, requirements_txt] + srcs + [f for f in (requirements_linux, requirements_darwin, requirements_windows) if f != None] + constraints) # Use the Label constructor so this is expanded in the context of the file # where it appears, which is to say, in @rules_python diff --git a/tests/integration/compile_pip_requirements/BUILD.bazel b/tests/integration/compile_pip_requirements/BUILD.bazel index 6df46b8372..b1f398c4a8 100644 --- a/tests/integration/compile_pip_requirements/BUILD.bazel +++ b/tests/integration/compile_pip_requirements/BUILD.bazel @@ -29,6 +29,19 @@ compile_pip_requirements( requirements_txt = "requirements_lock.txt", ) +genquery( + name = "requirements_update_data", + expression = """ +some(labels(data, @compile_pip_requirements//:requirements.update) intersect @compile_pip_requirements//:requirements.in) union +some(labels(data, @compile_pip_requirements//:requirements.update) intersect @compile_pip_requirements//:requirements_extra.in) +""", + scope = [ + "@compile_pip_requirements//:requirements.update", + "@compile_pip_requirements//:requirements.in", + "@compile_pip_requirements//:requirements_extra.in", + ], +) + compile_pip_requirements( name = "requirements_nohashes", src = "requirements.txt", diff --git a/tests/integration/compile_pip_requirements/WORKSPACE b/tests/integration/compile_pip_requirements/WORKSPACE index 0eeab2067c..397d828261 100644 --- a/tests/integration/compile_pip_requirements/WORKSPACE +++ b/tests/integration/compile_pip_requirements/WORKSPACE @@ -3,6 +3,11 @@ local_repository( path = "../../..", ) +local_repository( + name = "compile_pip_requirements", + path = ".", +) + load("@rules_python//python:repositories.bzl", "py_repositories", "python_register_toolchains") py_repositories() From 966329b0644d6377eefc59b2b86d80d9c12a6f83 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Fri, 10 Jul 2026 19:40:03 -0700 Subject: [PATCH 830/922] chore(pr-review): trigger automated review workflow on issue comments (#3918) Currently, the automated PR review workflow requires a diff review comment (`pull_request_review_comment`) to trigger on-demand reviews via `/review`. This prevents maintainers from triggering reviews from regular conversation comments on pull requests. To fix, change the event trigger from `pull_request_review_comment` to `issue_comment`, ensure the event is on a pull request by checking `github.event.issue.pull_request != null`, and check out `refs/pull/$NUMBER/head` so the PR branch is retrieved during comment events. * Also checks `\r\n/review` when parsing comment body for multiline `/review` commands. --- .github/workflows/automated_pr_review.yaml | 21 ++++++++++++++------- 1 file changed, 14 insertions(+), 7 deletions(-) diff --git a/.github/workflows/automated_pr_review.yaml b/.github/workflows/automated_pr_review.yaml index 23d4369b2a..a512ae2427 100644 --- a/.github/workflows/automated_pr_review.yaml +++ b/.github/workflows/automated_pr_review.yaml @@ -7,7 +7,7 @@ name: Automated Code Review on: pull_request: types: [opened] - pull_request_review_comment: + issue_comment: types: [created] permissions: @@ -20,19 +20,24 @@ jobs: # Trigger only if: # 1. It is a pull_request event, it is NOT a draft, and the author is a maintainer # (OWNER, MEMBER, or COLLABORATOR). - # 2. OR it is a pull_request_review_comment event, the comment body has a line starting with "/review", + # 2. OR it is a regular conversation comment on a pull request, the comment body has a line starting with "/review", # and the commenter is a maintainer. if: > (github.event_name == 'pull_request' && !github.event.pull_request.draft && contains(fromJson('["OWNER", "MEMBER", "COLLABORATOR"]'), github.event.pull_request.author_association)) || - (github.event_name == 'pull_request_review_comment' && - (startsWith(github.event.comment.body, '/review') || contains(github.event.comment.body, '\n/review')) && + (github.event_name == 'issue_comment' && github.event.issue.pull_request != null && + (startsWith(github.event.comment.body, '/review') || + contains(github.event.comment.body, '\n/review') || + contains(github.event.comment.body, '\r\n/review')) && contains(fromJson('["OWNER", "MEMBER", "COLLABORATOR"]'), github.event.comment.author_association)) steps: - name: Checkout PR Branch uses: actions/checkout@v7 with: - ref: ${{ github.event.pull_request.head.sha }} + # Note: In GHA, during an issue_comment event on a PR, github.event.pull_request is null + # and the PR number is placed inside github.event.issue.number (because every PR is an issue). + # Therefore, github.event.issue.number is the PR number when checking out refs/pull//head. + ref: refs/pull/${{ github.event.pull_request.number || github.event.issue.number }}/head persist-credentials: false - name: Checkout Reviewbot (Base Branch) @@ -45,8 +50,10 @@ jobs: - name: Install uv uses: astral-sh/setup-uv@v8.3.0 - - name: Set up Python - run: uv python install 3.13 + - name: Set up Python and Virtual Environment + run: | + uv python install 3.13 + uv venv --python 3.13 - name: Install Dependencies run: | From cf152cd9dfef51915146e02e543d136100ae1030 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Sat, 11 Jul 2026 11:59:30 -0700 Subject: [PATCH 831/922] chore(reviewbot): fix antigravity_review execution and configuration (#3921) Currently, running `antigravity_review.py` with `uv run` fails because it lacks PEP 723 dependency metadata and passes `system_instructions` to `Agent` instead of `LocalAgentConfig`. Also, `skills_paths` points to the skill directory rather than the `SKILL.md` file. To fix, add inline PEP 723 script metadata declaring dependencies, pass `system_instructions` to `LocalAgentConfig`, and target `SKILL.md` directly in `skills_paths`. --- .github/workflows/automated_pr_review.yaml | 9 ------- tools/private/reviewbot/antigravity_review.py | 24 ++++++++++++------- 2 files changed, 16 insertions(+), 17 deletions(-) diff --git a/.github/workflows/automated_pr_review.yaml b/.github/workflows/automated_pr_review.yaml index a512ae2427..af2da7e0ca 100644 --- a/.github/workflows/automated_pr_review.yaml +++ b/.github/workflows/automated_pr_review.yaml @@ -50,15 +50,6 @@ jobs: - name: Install uv uses: astral-sh/setup-uv@v8.3.0 - - name: Set up Python and Virtual Environment - run: | - uv python install 3.13 - uv venv --python 3.13 - - - name: Install Dependencies - run: | - uv pip install google-antigravity requests - - name: Run Antigravity Review env: GEMINI_API_KEY: ${{ secrets.GEMINI_API_KEY }} diff --git a/tools/private/reviewbot/antigravity_review.py b/tools/private/reviewbot/antigravity_review.py index 1bb23283be..e415e42a91 100644 --- a/tools/private/reviewbot/antigravity_review.py +++ b/tools/private/reviewbot/antigravity_review.py @@ -1,3 +1,10 @@ +# /// script +# requires-python = ">=3.11" +# dependencies = [ +# "google-antigravity", +# "requests", +# ] +# /// import argparse import asyncio from pathlib import Path @@ -17,10 +24,17 @@ async def main(): # Read prompt file prompt = Path(args.prompt).read_text() + # General coordinator instructions for the reviewer agent. + system_instructions = ( + "You are a code review assistant. Use your available skills to perform " + "reviews on pull requests." + ) + # Initialize the Antigravity Agent in read-only mode for security. # Register the review-pr skill from the local reviewbot folder. config = LocalAgentConfig( - skills_paths=["tools/private/reviewbot/skills/review-pr"], + system_instructions=system_instructions, + skills_paths=[str(Path(__file__).parent / "skills" / "review-pr" / "SKILL.md")], capabilities=CapabilitiesConfig( allow_filesystem_read=True, allow_filesystem_write=False, @@ -28,13 +42,7 @@ async def main(): ), ) - # General coordinator instructions for the reviewer agent. - system_instructions = ( - "You are a code review assistant. Use your available skills to perform " - "reviews on pull requests." - ) - - async with Agent(config, system_instructions=system_instructions) as agent: + async with Agent(config) as agent: response = await agent.chat(prompt) report = await response.text() From a31feb9a32daf8acb44c8479b4831eb4dea52288 Mon Sep 17 00:00:00 2001 From: Kayce Basques Date: Sat, 11 Jul 2026 12:13:06 -0700 Subject: [PATCH 832/922] fix(sphinxdocs): invalidate cache on branch change (#3917) We had to disable the `sphinxdocs` persistent worker in Pigweed (https://pwbug.dev/493749800) because the cache invalidation logic seemed incomplete with regards to Git branches. E.g. in branch `feat` you create a new doc `foo.rst`. You switch back to branch `main` and try to build Sphinx, but Sphinx warns about needing to ignore unreadable document `foo.rst`. Because Pigweed builds Sphinx with `--fail-on-warning` this would break our docs build. This PR adds the bug fix and also starts a suite of tests that will be focused on persistent worker correctness. --------- Co-authored-by: Richard Levasseur --- sphinxdocs/.bazelignore | 4 + sphinxdocs/.bazelrc | 8 ++ sphinxdocs/.bazelrc.deleted_packages | 1 + sphinxdocs/BUILD.bazel | 16 +++ sphinxdocs/MODULE.bazel | 18 ++++ sphinxdocs/integration_tests/BUILD.bazel | 8 ++ sphinxdocs/integration_tests/bazel_from_env | 6 ++ .../integration_tests/integration_test.bzl | 68 +++++++++++++ .../persistent_worker/BUILD.bazel | 8 ++ .../persistent_worker_test.py | 54 ++++++++++ .../persistent_worker/workspace/.bazelrc | 8 ++ .../persistent_worker/workspace/BUILD.bazel | 17 ++++ .../persistent_worker/workspace/MODULE.bazel | 23 +++++ .../persistent_worker/workspace/WORKSPACE | 1 + .../persistent_worker/workspace/conf.py | 1 + .../workspace/downloader_config.cfg | 21 ++++ .../persistent_worker/workspace/index.md | 7 ++ sphinxdocs/integration_tests/runner.py | 99 +++++++++++++++++++ sphinxdocs/sphinxdocs/BUILD.bazel | 8 ++ sphinxdocs/sphinxdocs/private/BUILD.bazel | 6 ++ sphinxdocs/sphinxdocs/private/sphinx_build.py | 63 ++++++++++-- 21 files changed, 437 insertions(+), 8 deletions(-) create mode 100644 sphinxdocs/.bazelignore create mode 100644 sphinxdocs/BUILD.bazel create mode 100644 sphinxdocs/integration_tests/BUILD.bazel create mode 100755 sphinxdocs/integration_tests/bazel_from_env create mode 100644 sphinxdocs/integration_tests/integration_test.bzl create mode 100644 sphinxdocs/integration_tests/persistent_worker/BUILD.bazel create mode 100644 sphinxdocs/integration_tests/persistent_worker/persistent_worker_test.py create mode 100644 sphinxdocs/integration_tests/persistent_worker/workspace/.bazelrc create mode 100644 sphinxdocs/integration_tests/persistent_worker/workspace/BUILD.bazel create mode 100644 sphinxdocs/integration_tests/persistent_worker/workspace/MODULE.bazel create mode 100644 sphinxdocs/integration_tests/persistent_worker/workspace/WORKSPACE create mode 100644 sphinxdocs/integration_tests/persistent_worker/workspace/conf.py create mode 100644 sphinxdocs/integration_tests/persistent_worker/workspace/downloader_config.cfg create mode 100644 sphinxdocs/integration_tests/persistent_worker/workspace/index.md create mode 100644 sphinxdocs/integration_tests/runner.py diff --git a/sphinxdocs/.bazelignore b/sphinxdocs/.bazelignore new file mode 100644 index 0000000000..80f2d3514f --- /dev/null +++ b/sphinxdocs/.bazelignore @@ -0,0 +1,4 @@ +bazel-bin +bazel-out +bazel-testlogs +bazel-sphinxdocs diff --git a/sphinxdocs/.bazelrc b/sphinxdocs/.bazelrc index 65c996c678..ce4d782113 100644 --- a/sphinxdocs/.bazelrc +++ b/sphinxdocs/.bazelrc @@ -21,5 +21,13 @@ common --experimental_repository_downloader_retries=10 common --incompatible_python_disallow_native_rules common --incompatible_no_implicit_file_export +# Pass host PATH to build and test actions so bazel_from_env (used by rules_bazel_integration_test +# for local 'self' testing) can locate the user's bazel binary without disabling strict action env. +common --action_env=PATH +common --test_env=PATH build --lockfile_mode=update + +common:fast-tests --build_tests_only=true +common:fast-tests --build_tag_filters=-large,-enormous,-integration-test +common:fast-tests --test_tag_filters=-large,-enormous,-integration-test diff --git a/sphinxdocs/.bazelrc.deleted_packages b/sphinxdocs/.bazelrc.deleted_packages index 442c80e960..9a90ef21be 100644 --- a/sphinxdocs/.bazelrc.deleted_packages +++ b/sphinxdocs/.bazelrc.deleted_packages @@ -1 +1,2 @@ common --deleted_packages=integration_tests/bcr +common --deleted_packages=integration_tests/persistent_worker/workspace diff --git a/sphinxdocs/BUILD.bazel b/sphinxdocs/BUILD.bazel new file mode 100644 index 0000000000..8ff9bc956f --- /dev/null +++ b/sphinxdocs/BUILD.bazel @@ -0,0 +1,16 @@ +package(default_visibility = ["//visibility:public"]) + +filegroup( + name = "distribution", + srcs = glob( + ["**/*"], + exclude = [ + "bazel-*/**", + "integration_tests/**", + "tests/**", + ], + ) + [ + "//sphinxdocs:distribution", + ], + visibility = ["//visibility:public"], +) diff --git a/sphinxdocs/MODULE.bazel b/sphinxdocs/MODULE.bazel index 30fb2196dc..3d4caf2bb2 100644 --- a/sphinxdocs/MODULE.bazel +++ b/sphinxdocs/MODULE.bazel @@ -21,3 +21,21 @@ dev_pip.parse( requirements_lock = "@rules_python//docs:requirements.txt", ) use_repo(dev_pip, "dev_pip") + +bazel_dep(name = "rules_bazel_integration_test", version = "0.37.1", dev_dependency = True) + +bazel_binaries = use_extension( + "@rules_bazel_integration_test//:extensions.bzl", + "bazel_binaries", + dev_dependency = True, +) +bazel_binaries.local( + name = "self", + path = "integration_tests/bazel_from_env", +) +use_repo( + bazel_binaries, + "bazel_binaries", + "bazel_binaries_bazelisk", + "build_bazel_bazel_self", +) diff --git a/sphinxdocs/integration_tests/BUILD.bazel b/sphinxdocs/integration_tests/BUILD.bazel new file mode 100644 index 0000000000..45e894fc2e --- /dev/null +++ b/sphinxdocs/integration_tests/BUILD.bazel @@ -0,0 +1,8 @@ +load("@rules_python//python:py_library.bzl", "py_library") + +package(default_visibility = ["//visibility:public"]) + +py_library( + name = "runner_lib", + srcs = ["runner.py"], +) diff --git a/sphinxdocs/integration_tests/bazel_from_env b/sphinxdocs/integration_tests/bazel_from_env new file mode 100755 index 0000000000..a372736f32 --- /dev/null +++ b/sphinxdocs/integration_tests/bazel_from_env @@ -0,0 +1,6 @@ +#!/usr/bin/env bash +# +# A simple wrapper so rules_bazel_integration_test can use the +# bazel version inherited from the environment. + +bazel "$@" diff --git a/sphinxdocs/integration_tests/integration_test.bzl b/sphinxdocs/integration_tests/integration_test.bzl new file mode 100644 index 0000000000..b6786b9c6b --- /dev/null +++ b/sphinxdocs/integration_tests/integration_test.bzl @@ -0,0 +1,68 @@ +"""Helpers for running bazel-in-bazel integration tests for sphinxdocs.""" + +load( + "@rules_bazel_integration_test//bazel_integration_test:defs.bzl", + "bazel_integration_test", + "integration_test_utils", +) +load("@rules_python//python:py_test.bzl", "py_test") + +def _test_runner(*, name, bazel_version, py_main, py_deps): + test_runner = "{}_bazel_{}_py_runner".format(name, bazel_version) + py_test( + name = test_runner, + srcs = [py_main], + main = py_main, + deps = ["//integration_tests:runner_lib"] + py_deps, + tags = ["manual"], + ) + return test_runner + +def sphinxdocs_integration_test( + name, + workspace_path = "workspace", + tags = None, + py_main = None, + py_deps = None, + bazel_versions = None, + **kwargs): + """Runs a bazel-in-bazel integration test for sphinxdocs. + + Args: + name: Name of the test. + workspace_path: The directory name of the sub-workspace. + tags: Test tags. + py_main: Main Python test runner script. + py_deps: Dependencies for py_main. + bazel_versions: List of bazel versions to test. + **kwargs: Passed to bazel_integration_test. + """ + workspace_files = integration_test_utils.glob_workspace_files(workspace_path) + native.filegroup( + name = name + "_workspace_files", + srcs = workspace_files + [ + "//:distribution", + ], + ) + kwargs.setdefault("size", "large") + for bazel_version in bazel_versions or ["self"]: + test_runner = _test_runner( + name = name, + bazel_version = bazel_version, + py_main = py_main, + py_deps = py_deps or [], + ) + bazel_integration_test( + name = "{}_bazel_{}".format(name, bazel_version), + workspace_path = workspace_path, + test_runner = test_runner, + bazel_version = bazel_version, + workspace_files = [name + "_workspace_files"], + tags = (tags or []) + [ + "exclusive", + "no-sandbox", + "no-remote-exec", + "integration-test", + ], + **kwargs + ) diff --git a/sphinxdocs/integration_tests/persistent_worker/BUILD.bazel b/sphinxdocs/integration_tests/persistent_worker/BUILD.bazel new file mode 100644 index 0000000000..60fb4cbc6c --- /dev/null +++ b/sphinxdocs/integration_tests/persistent_worker/BUILD.bazel @@ -0,0 +1,8 @@ +load("//integration_tests:integration_test.bzl", "sphinxdocs_integration_test") + +package(default_visibility = ["//visibility:public"]) + +sphinxdocs_integration_test( + name = "persistent_worker_test", + py_main = "persistent_worker_test.py", +) diff --git a/sphinxdocs/integration_tests/persistent_worker/persistent_worker_test.py b/sphinxdocs/integration_tests/persistent_worker/persistent_worker_test.py new file mode 100644 index 0000000000..30a8b46c00 --- /dev/null +++ b/sphinxdocs/integration_tests/persistent_worker/persistent_worker_test.py @@ -0,0 +1,54 @@ +import unittest + +from integration_tests import runner + + +class PersistentWorkerTest(runner.TestCase): + def _check_index_html(self, text: str, should_exist: bool): + index_html = ( + self.repo_root / "bazel-bin" / "docs" / "_build" / "html" / "index.html" + ) + if not index_html.exists(): + self.fail(f"Could not find index.html at {index_html}") + content = index_html.read_text() + if should_exist: + self.assertIn( + text, + content, + f"Expected '{text}' in index.html after build, but not found.", + ) + else: + self.assertNotIn( + text, + content, + f"Expected '{text}' NOT to be in index.html after build, but found it.", + ) + + def test_incremental_add_and_remove_files(self): + # 1. Initial build + result = self.run_bazel("build", "//:docs") + self.assert_result_matches(result, "bazel-bin") + index_html = ( + self.repo_root / "bazel-bin" / "docs" / "_build" / "html" / "index.html" + ) + self.assertTrue( + index_html.exists(), "index.html should exist after initial build" + ) + + # 2. Add a new markdown file and verify it is included across incremental build + page2_md = self.repo_root / "page2.md" + page2_md.write_text("# Page 2\n\nThis is a newly added page.\n") + result = self.run_bazel("build", "//:docs") + self.assert_result_matches(result, "bazel-bin") + self._check_index_html("page2.html", should_exist=True) + + # 3. Remove the added markdown file and verify the persistent worker cleans up + # stale source files and invalidates toctrees without errors or warnings. + page2_md.unlink() + result = self.run_bazel("build", "//:docs") + self.assert_result_matches(result, "bazel-bin") + self._check_index_html("page2.html", should_exist=False) + + +if __name__ == "__main__": + unittest.main() diff --git a/sphinxdocs/integration_tests/persistent_worker/workspace/.bazelrc b/sphinxdocs/integration_tests/persistent_worker/workspace/.bazelrc new file mode 100644 index 0000000000..f3162b9547 --- /dev/null +++ b/sphinxdocs/integration_tests/persistent_worker/workspace/.bazelrc @@ -0,0 +1,8 @@ +# Disable disk caching and remote action caching so Bazel is forced to dispatch builds +# directly to the running persistent worker instead of restoring pre-built HTML cache results across test steps. +common --disk_cache= +build --noremote_accept_cached +common --http_timeout_scaling=10.0 +common --experimental_repository_downloader_retries=10 +common --experimental_downloader_config=downloader_config.cfg +common --lockfile_mode=off diff --git a/sphinxdocs/integration_tests/persistent_worker/workspace/BUILD.bazel b/sphinxdocs/integration_tests/persistent_worker/workspace/BUILD.bazel new file mode 100644 index 0000000000..60e66004fd --- /dev/null +++ b/sphinxdocs/integration_tests/persistent_worker/workspace/BUILD.bazel @@ -0,0 +1,17 @@ +load("@sphinxdocs//sphinxdocs:sphinx.bzl", "sphinx_build_binary", "sphinx_docs") + +sphinx_docs( + name = "docs", + srcs = glob(["*.md"]), + config = "conf.py", + formats = ["html"], + sphinx = ":sphinx-build", +) + +sphinx_build_binary( + name = "sphinx-build", + deps = [ + "@dev_pip//myst_parser", + "@dev_pip//sphinx", + ], +) diff --git a/sphinxdocs/integration_tests/persistent_worker/workspace/MODULE.bazel b/sphinxdocs/integration_tests/persistent_worker/workspace/MODULE.bazel new file mode 100644 index 0000000000..4e2c25e8e4 --- /dev/null +++ b/sphinxdocs/integration_tests/persistent_worker/workspace/MODULE.bazel @@ -0,0 +1,23 @@ +module(version = "0.0.0") + +bazel_dep(name = "sphinxdocs", version = "0.0.0") +local_path_override( + module_name = "sphinxdocs", + path = "../../..", +) + +bazel_dep(name = "rules_python", version = "1.8.5") + +dev_pip = use_extension( + "@rules_python//python/extensions:pip.bzl", + "pip", + dev_dependency = True, +) +dev_pip.parse( + hub_name = "dev_pip", + python_version = "3.11", + requirements_lock = "@rules_python//docs:requirements.txt", +) +use_repo(dev_pip, "dev_pip") + +bazel_dep(name = "bazel_skylib", version = "1.8.2") diff --git a/sphinxdocs/integration_tests/persistent_worker/workspace/WORKSPACE b/sphinxdocs/integration_tests/persistent_worker/workspace/WORKSPACE new file mode 100644 index 0000000000..4be37fab94 --- /dev/null +++ b/sphinxdocs/integration_tests/persistent_worker/workspace/WORKSPACE @@ -0,0 +1 @@ +# Bzlmod-enabled workspace; this empty WORKSPACE file satisfies rules_bazel_integration_test checks. diff --git a/sphinxdocs/integration_tests/persistent_worker/workspace/conf.py b/sphinxdocs/integration_tests/persistent_worker/workspace/conf.py new file mode 100644 index 0000000000..de33bde101 --- /dev/null +++ b/sphinxdocs/integration_tests/persistent_worker/workspace/conf.py @@ -0,0 +1 @@ +extensions = ["myst_parser"] diff --git a/sphinxdocs/integration_tests/persistent_worker/workspace/downloader_config.cfg b/sphinxdocs/integration_tests/persistent_worker/workspace/downloader_config.cfg new file mode 100644 index 0000000000..3fa6264eda --- /dev/null +++ b/sphinxdocs/integration_tests/persistent_worker/workspace/downloader_config.cfg @@ -0,0 +1,21 @@ +# Try GitHub first (primary) +rewrite ^github\.com/bazel-contrib/bazel_features/(.*) github.com/bazel-contrib/bazel_features/$1 +rewrite ^github\.com/bazel-contrib/rules_go/(.*) github.com/bazel-contrib/rules_go/$1 +rewrite ^github\.com/bazelbuild/bazel-skylib/(.*) github.com/bazelbuild/bazel-skylib/$1 +rewrite ^github\.com/bazelbuild/platforms/(.*) github.com/bazelbuild/platforms/$1 +rewrite ^github\.com/bazelbuild/rules_kotlin/(.*) github.com/bazelbuild/rules_kotlin/$1 +rewrite ^github\.com/bazelbuild/rules_shell/(.*) github.com/bazelbuild/rules_shell/$1 +rewrite ^github\.com/bazelbuild/rules_java/(.*) github.com/bazelbuild/rules_java/$1 +rewrite ^github\.com/bazelbuild/stardoc/(.*) github.com/bazelbuild/stardoc/$1 + + +# Fall back to mirror (secondary) +# Tracking upstream BCR mirror addition: https://github.com/bazelbuild/platforms/issues/139 +rewrite ^github\.com/bazel-contrib/bazel_features/(.*) mirror.bazel.build/github.com/bazel-contrib/bazel_features/$1 +rewrite ^github\.com/bazel-contrib/rules_go/(.*) mirror.bazel.build/github.com/bazel-contrib/rules_go/$1 +rewrite ^github\.com/bazelbuild/bazel-skylib/(.*) mirror.bazel.build/github.com/bazelbuild/bazel-skylib/$1 +rewrite ^github\.com/bazelbuild/platforms/(.*) mirror.bazel.build/github.com/bazelbuild/platforms/$1 +rewrite ^github\.com/bazelbuild/rules_kotlin/(.*) mirror.bazel.build/github.com/bazelbuild/rules_kotlin/$1 +rewrite ^github\.com/bazelbuild/rules_shell/(.*) mirror.bazel.build/github.com/bazelbuild/rules_shell/$1 +rewrite ^github\.com/bazelbuild/rules_java/(.*) mirror.bazel.build/github.com/bazelbuild/rules_java/$1 +rewrite ^github\.com/bazelbuild/stardoc/(.*) mirror.bazel.build/github.com/bazelbuild/stardoc/$1 diff --git a/sphinxdocs/integration_tests/persistent_worker/workspace/index.md b/sphinxdocs/integration_tests/persistent_worker/workspace/index.md new file mode 100644 index 0000000000..d1dca92350 --- /dev/null +++ b/sphinxdocs/integration_tests/persistent_worker/workspace/index.md @@ -0,0 +1,7 @@ +# Test Documentation + +```{toctree} +:glob: true + +* +``` diff --git a/sphinxdocs/integration_tests/runner.py b/sphinxdocs/integration_tests/runner.py new file mode 100644 index 0000000000..cab9730bb8 --- /dev/null +++ b/sphinxdocs/integration_tests/runner.py @@ -0,0 +1,99 @@ +import logging +import os +import os.path +import pathlib +import re +import shlex +import subprocess +import unittest + +_logger = logging.getLogger(__name__) + + +class ExecuteError(Exception): + def __init__(self, result): + self.result = result + + def __str__(self): + return self.result.describe() + + +class ExecuteResult: + def __init__( + self, + args: list[str], + env: dict[str, str], + cwd: pathlib.Path, + proc_result: subprocess.CompletedProcess, + ): + self.args = args + self.env = env + self.cwd = cwd + self.exit_code = proc_result.returncode + self.stdout = proc_result.stdout + self.stderr = proc_result.stderr + + def describe(self) -> str: + env_lines = [ + " " + shlex.quote(f"{key}={value}") + for key, value in sorted(self.env.items()) + ] + env = " \\\n".join(env_lines) + args = shlex.join(self.args) + maybe_stdout_nl = "" if self.stdout.endswith("\n") else "\n" + maybe_stderr_nl = "" if self.stderr.endswith("\n") else "\n" + return f"""\ +COMMAND: +cd {self.cwd} && \\ +env \\ +{env} \\ + {args} +RESULT: exit_code: {self.exit_code} +===== STDOUT START ===== +{self.stdout}{maybe_stdout_nl}===== STDOUT END ===== +===== STDERR START ===== +{self.stderr}{maybe_stderr_nl}===== STDERR END ===== +""" + + +class TestCase(unittest.TestCase): + def setUp(self): + super().setUp() + self.repo_root = pathlib.Path(os.environ["BIT_WORKSPACE_DIR"]) + self.bazel = pathlib.Path(os.environ["BIT_BAZEL_BINARY"]) + outer_test_tmpdir = pathlib.Path(os.environ["TEST_TMPDIR"]) + self.test_tmp_dir = outer_test_tmpdir / "bit_test_tmp" + self.tmp_dir = outer_test_tmpdir / "bit_tmp" + self.bazel_env = { + "PATH": os.environ["PATH"], + "TEST_TMPDIR": str(self.test_tmp_dir), + "TMP": str(self.tmp_dir), + "RUNFILES_DIR": os.environ["TEST_SRCDIR"], + } + + def run_bazel(self, *args: str, check: bool = True) -> ExecuteResult: + args = [str(self.bazel), *args] + env = self.bazel_env + _logger.info("executing: %s", shlex.join(args)) + cwd = self.repo_root + proc_result = subprocess.run( + args=args, + text=True, + capture_output=True, + cwd=cwd, + env=env, + check=False, + ) + exec_result = ExecuteResult(args, env, cwd, proc_result) + if check and exec_result.exit_code: + raise ExecuteError(exec_result) + else: + return exec_result + + def assert_result_matches(self, result: ExecuteResult, regex: str) -> None: + if not re.search(regex, result.stdout + result.stderr): + self.fail( + "Bazel output did not match expected pattern\n" + + f"expected pattern: {regex}\n" + + f"invocation details:\n{result.describe()}" + ) diff --git a/sphinxdocs/sphinxdocs/BUILD.bazel b/sphinxdocs/sphinxdocs/BUILD.bazel index fbdd633a32..9a871f5268 100644 --- a/sphinxdocs/sphinxdocs/BUILD.bazel +++ b/sphinxdocs/sphinxdocs/BUILD.bazel @@ -20,6 +20,14 @@ package( default_visibility = ["//:__subpackages__"], ) +filegroup( + name = "distribution", + srcs = glob(["**/*"]) + [ + "//sphinxdocs/private:distribution", + ], + visibility = ["//:__pkg__"], +) + # Additional -D values to add to every Sphinx build. # This is usually used to override the version when building repeated_string_list_flag( diff --git a/sphinxdocs/sphinxdocs/private/BUILD.bazel b/sphinxdocs/sphinxdocs/private/BUILD.bazel index b054ed2611..0fe0ba4152 100644 --- a/sphinxdocs/sphinxdocs/private/BUILD.bazel +++ b/sphinxdocs/sphinxdocs/private/BUILD.bazel @@ -34,6 +34,12 @@ exports_files( visibility = ["//visibility:public"], ) +filegroup( + name = "distribution", + srcs = glob(["**/*"]), + visibility = ["//sphinxdocs:__pkg__"], +) + bzl_library( name = "util", srcs = ["util.bzl"], diff --git a/sphinxdocs/sphinxdocs/private/sphinx_build.py b/sphinxdocs/sphinxdocs/private/sphinx_build.py index 3605b4579d..07a123b496 100644 --- a/sphinxdocs/sphinxdocs/private/sphinx_build.py +++ b/sphinxdocs/sphinxdocs/private/sphinx_build.py @@ -144,6 +144,30 @@ def _prepare_sphinx(self, request): logger.info("path %s changed", path) changed_paths.append(path) + # Remove any source files that were tracked in the previous request (`current_digests`) + # but are missing from the current `request["inputs"]` (`incoming_digests`). + # Across incremental branch switches or file removals, if these stale symlinks + # remain in `srcdir` on disk, Sphinx will discover broken/unreadable files during + # `find_files()` and abort with "WARNING: Ignored unreadable document" (fatal with -W). + for path in set(current_digests) - set(incoming_digests): + removed_path = os.path.join(srcdir, path) + if os.path.exists(removed_path) or os.path.islink(removed_path): + logger.info("removing stale source file %s", removed_path) + try: + if os.path.islink(removed_path): + try: + os.remove(removed_path) + except OSError: + os.rmdir(removed_path) + elif os.path.isdir(removed_path): + shutil.rmtree(removed_path) + else: + os.remove(removed_path) + except OSError as e: + logger.warning( + "failed to remove stale source %s: %s", removed_path, e + ) + self._digests[srcdir] = incoming_digests self._extension.changed_paths = changed_paths request_info["changed_sources"] = changed_paths @@ -188,6 +212,12 @@ def _process_request(self, request: "WorkRequest") -> "WorkResponse | None": stdout.truncate(0) stderr.seek(0) stderr.truncate(0) + # If Sphinx cache (`--doctree-dir`) becomes corrupted across incremental + # updates or branch checkouts, exit code 2 is returned. Wiping out the cached + # doctrees before retrying allows Sphinx to recover cleanly from scratch. + for arg in sphinx_args: + if arg.startswith("--doctree-dir="): + shutil.rmtree(arg.split("=", 1)[1], ignore_errors=True) exit_code = main(sphinx_args) if exit_code: @@ -247,14 +277,31 @@ def setup(self, app): return {"parallel_read_safe": True, "parallel_write_safe": True} def _handle_env_get_outdated(self, app, env, added, changed, removed): - changed = { - # NOTE: path2doc returns None if it's not a doc path - env.path2doc(p) - for p in self.changed_paths - } - - logger.info("changed docs: %s", changed) - return changed + changed_docs = set() + for p in self.changed_paths: + # Try multiple path resolutions because depending on how Sphinx and Bazel + # represent inputs (`p`), `env.path2doc` may require relative, srcdir-joined, + # or absolute paths to successfully resolve the document name. + doc = ( + env.path2doc(p) + or env.path2doc(os.path.join(env.srcdir, p)) + or env.path2doc(os.path.abspath(os.path.join(env.srcdir, p))) + ) + if doc: + changed_docs.add(doc) + + # When documents are added or removed across incremental builds or branch checkouts, + # parent documents whose `toctree` includes them (especially via glob patterns or + # explicit references to removed docs) must be invalidated and re-read. Otherwise, + # Sphinx retains stale table of contents entries or throws unresolvable reference errors. + if added or removed: + glob_toctrees = getattr(env, "glob_toctrees", set()) + for doc, includes in getattr(env, "toctree_includes", {}).items(): + if doc in glob_toctrees or not removed.isdisjoint(includes): + changed_docs.add(doc) + + logger.info("changed docs: %s", changed_docs) + return changed_docs def _worker_main(stdin, stdout, exec_root): From 5bdec9fd2173c57fc31dfffe4fe37ee029577f38 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Sat, 11 Jul 2026 20:18:32 -0700 Subject: [PATCH 833/922] refactor: use centralized NOT_ACTUALLY_PUBLIC across internal targets (#3920) Currently, several internal macros, toolchain definitions, and config settings either hardcode \`["//visibility:public"]\` or define private copies like \`_NOT_ACTUALLY_PUBLIC\`. This scatters visibility workarounds across the codebase and makes developer intent unclear. To clean this up, transition internal rules across \`python/private\` to use \`NOT_ACTUALLY_PUBLIC\` from \`python/private/visibility.bzl\`, and update corresponding \`bzl_library\` dependencies to match. --- python/config_settings/BUILD.bazel | 25 +++++++++-------- python/private/BUILD.bazel | 29 +++++++++++--------- python/private/api/BUILD.bazel | 3 +- python/private/config_settings.bzl | 21 ++++++-------- python/private/stamp_impl.bzl | 4 ++- sphinxdocs/sphinxdocs/private/BUILD.bazel | 12 ++++++-- sphinxdocs/sphinxdocs/private/visibility.bzl | 7 +++++ tools/precompiler/BUILD.bazel | 9 +++--- tools/private/zipapp/BUILD.bazel | 25 +++++++---------- 9 files changed, 73 insertions(+), 62 deletions(-) create mode 100644 sphinxdocs/sphinxdocs/private/visibility.bzl diff --git a/python/config_settings/BUILD.bazel b/python/config_settings/BUILD.bazel index c7de1d00da..d92aa4261c 100644 --- a/python/config_settings/BUILD.bazel +++ b/python/config_settings/BUILD.bazel @@ -15,6 +15,7 @@ load( "VenvsUseDeclareSymlinkFlag", rp_string_flag = "string_flag", ) +load("//python/private:visibility.bzl", "NOT_ACTUALLY_PUBLIC") # buildifier: disable=bzl-visibility load("//python/private/pypi:flags.bzl", "define_pypi_internal_flags") load(":config_settings.bzl", "construct_config_settings") @@ -48,7 +49,7 @@ string_flag( build_setting_default = AddSrcsToRunfilesFlag.AUTO, values = AddSrcsToRunfilesFlag.flag_values(), # NOTE: Only public because it is dependency of public rules. - visibility = ["//visibility:public"], + visibility = NOT_ACTUALLY_PUBLIC, ) string_flag( @@ -67,7 +68,7 @@ config_setting( }, # NOTE: Only public because it is used in py_toolchain_suite from toolchain # repositories - visibility = ["//visibility:public"], + visibility = NOT_ACTUALLY_PUBLIC, ) string_flag( @@ -75,7 +76,7 @@ string_flag( build_setting_default = PrecompileFlag.AUTO, values = sorted(PrecompileFlag.__members__.values()), # NOTE: Only public because it's an implicit dependency - visibility = ["//visibility:public"], + visibility = NOT_ACTUALLY_PUBLIC, ) string_flag( @@ -83,7 +84,7 @@ string_flag( build_setting_default = ValidateTestMainFlag.AUTO, values = ValidateTestMainFlag.flag_values(), # NOTE: Only public because it's an implicit dependency of py_test. - visibility = ["//visibility:public"], + visibility = NOT_ACTUALLY_PUBLIC, ) string_flag( @@ -91,7 +92,7 @@ string_flag( build_setting_default = PrecompileSourceRetentionFlag.AUTO, values = sorted(PrecompileSourceRetentionFlag.__members__.values()), # NOTE: Only public because it's an implicit dependency - visibility = ["//visibility:public"], + visibility = NOT_ACTUALLY_PUBLIC, ) rp_string_flag( @@ -104,7 +105,7 @@ rp_string_flag( }), values = sorted(BootstrapImplFlag.__members__.values()), # NOTE: Only public because it's an implicit dependency - visibility = ["//visibility:public"], + visibility = NOT_ACTUALLY_PUBLIC, ) label_flag( @@ -127,7 +128,7 @@ string_flag( build_setting_default = LibcFlag.GLIBC, values = LibcFlag.flag_values(), # NOTE: Only public because it is used in pip hub and toolchain repos. - visibility = ["//visibility:public"], + visibility = NOT_ACTUALLY_PUBLIC, ) string_flag( @@ -166,21 +167,21 @@ string_flag( name = "venv", build_setting_default = "auto", # NOTE: Only public because it is used in pip hub repos and executable transitions. - visibility = ["//visibility:public"], + visibility = NOT_ACTUALLY_PUBLIC, ) string_flag( name = "pip_whl_osx_version", build_setting_default = "", # NOTE: Only public because it is used in pip hub repos. - visibility = ["//visibility:public"], + visibility = NOT_ACTUALLY_PUBLIC, ) string_flag( name = "venvs_site_packages", build_setting_default = VenvsSitePackages.NO, # NOTE: Only public because it is used in pip hub repos. - visibility = ["//visibility:public"], + visibility = NOT_ACTUALLY_PUBLIC, ) config_setting( @@ -189,7 +190,7 @@ config_setting( ":venvs_site_packages": VenvsSitePackages.YES, }, # NOTE: Only public because it is used in whl_library repos. - visibility = ["//visibility:public"], + visibility = NOT_ACTUALLY_PUBLIC, ) define_pypi_internal_flags( @@ -200,7 +201,7 @@ label_flag( name = "pip_env_marker_config", build_setting_default = ":_pip_env_marker_default_config", # NOTE: Only public because it is used in pip hub repos. - visibility = ["//visibility:public"], + visibility = NOT_ACTUALLY_PUBLIC, ) bool_flag( diff --git a/python/private/BUILD.bazel b/python/private/BUILD.bazel index e4d06d4a89..5f51b7650c 100644 --- a/python/private/BUILD.bazel +++ b/python/private/BUILD.bazel @@ -22,6 +22,7 @@ load(":py_interpreter_program.bzl", "py_interpreter_program") load(":sentinel_impl.bzl", "sentinel") load(":stamp_impl.bzl", "stamp_build_setting") load(":uncachable_version_file.bzl", "define_uncachable_version_file") +load(":visibility.bzl", "NOT_ACTUALLY_PUBLIC") package( default_visibility = [ @@ -95,7 +96,7 @@ exports_files( ["python_bootstrap_template.txt"], # Not actually public. Only public because it's an implicit dependency of # py_runtime. - visibility = ["//visibility:public"], + visibility = NOT_ACTUALLY_PUBLIC, ) filegroup( @@ -103,7 +104,7 @@ filegroup( srcs = ["stage1_bootstrap_template.sh"], # Not actually public. Only public because it's an implicit dependency of # py_runtime. - visibility = ["//visibility:public"], + visibility = NOT_ACTUALLY_PUBLIC, ) filegroup( @@ -111,7 +112,7 @@ filegroup( srcs = ["stage2_bootstrap_template.py"], # Not actually public. Only public because it's an implicit dependency of # py_runtime. - visibility = ["//visibility:public"], + visibility = NOT_ACTUALLY_PUBLIC, ) filegroup( @@ -119,7 +120,7 @@ filegroup( srcs = ["site_init_template.py"], # Not actually public. Only public because it's an implicit dependency of # py_runtime. - visibility = ["//visibility:public"], + visibility = NOT_ACTUALLY_PUBLIC, ) # NOTE: Windows builds don't use this bootstrap. Instead, a native Windows @@ -133,7 +134,7 @@ alias( }), # Not actually public. Only public because it's an implicit dependency of # py_runtime. - visibility = ["//visibility:public"], + visibility = NOT_ACTUALLY_PUBLIC, ) # Used to determine the use of `--stamp` in Starlark rules @@ -169,7 +170,7 @@ bool_flag( name = "visible_for_testing", build_setting_default = False, # This is only because it is an implicit dependency by the toolchains. - visibility = ["//visibility:public"], + visibility = NOT_ACTUALLY_PUBLIC, ) # Used for py_console_script_gen rule @@ -203,7 +204,7 @@ current_interpreter_executable( name = "current_interpreter_executable", # Not actually public. Only public because it's an implicit dependency of # py_exec_tools_toolchain. - visibility = ["//visibility:public"], + visibility = NOT_ACTUALLY_PUBLIC, ) py_library( @@ -227,7 +228,7 @@ py_interpreter_program( main = "py_test_main_validator.py", # Not actually public. Only public because it's an implicit dependency of # the py_test rule. - visibility = ["//visibility:public"], + visibility = NOT_ACTUALLY_PUBLIC, ) py_library( @@ -330,6 +331,7 @@ bzl_library( deps = [ ":text_util", ":version", + ":visibility", "@bazel_skylib//lib:selects", "@bazel_skylib//rules:common_settings", ], @@ -906,6 +908,12 @@ bzl_library( ], ) +bzl_library( + name = "stamp_impl", + srcs = ["stamp_impl.bzl"], + deps = [":visibility"], +) + bzl_library( name = "bzlmod_enabled", srcs = ["bzlmod_enabled.bzl"], @@ -986,11 +994,6 @@ bzl_library( srcs = ["sentinel_impl.bzl"], ) -bzl_library( - name = "stamp_impl", - srcs = ["stamp_impl.bzl"], -) - bzl_library( name = "text_util", srcs = ["text_util.bzl"], diff --git a/python/private/api/BUILD.bazel b/python/private/api/BUILD.bazel index b3cc7360a7..cd7eda1bd2 100644 --- a/python/private/api/BUILD.bazel +++ b/python/private/api/BUILD.bazel @@ -13,6 +13,7 @@ # limitations under the License. load("@bazel_skylib//:bzl_library.bzl", "bzl_library") +load("//python/private:visibility.bzl", "NOT_ACTUALLY_PUBLIC") load(":py_common_api.bzl", "py_common_api") package( @@ -27,7 +28,7 @@ filegroup( py_common_api( name = "py_common_api_impl", # NOTE: Not actually public. Implicit dependency of public rules. - visibility = ["//visibility:public"], + visibility = NOT_ACTUALLY_PUBLIC, ) bzl_library( diff --git a/python/private/config_settings.bzl b/python/private/config_settings.bzl index 91fbbba8cb..9cd729aff2 100644 --- a/python/private/config_settings.bzl +++ b/python/private/config_settings.bzl @@ -17,8 +17,9 @@ load("@bazel_skylib//lib:selects.bzl", "selects") load("@bazel_skylib//rules:common_settings.bzl", "BuildSettingInfo") -load("//python/private:text_util.bzl", "render") +load(":text_util.bzl", "render") load(":version.bzl", "version") +load(":visibility.bzl", "NOT_ACTUALLY_PUBLIC") _PYTHON_VERSION_FLAG = Label("//python/config_settings:python_version") _PYTHON_VERSION_MAJOR_MINOR_FLAG = Label("//python/config_settings:python_version_major_minor") @@ -31,10 +32,6 @@ If the value is missing, then the default value is being used, see documentation {docs_url}/python/config_settings """ -# Indicates something needs public visibility so that other generated code can -# access it, but it's not intended for general public usage. -_NOT_ACTUALLY_PUBLIC = ["//visibility:public"] - def construct_config_settings( *, name, @@ -93,7 +90,7 @@ def construct_config_settings( native.config_setting( name = "_" + name, flag_values = {":python_version": ver}, - visibility = ["//visibility:public"], + visibility = NOT_ACTUALLY_PUBLIC, ) # An alias pointing to an underscore-prefixed config_setting_group @@ -111,7 +108,7 @@ def construct_config_settings( native.alias( name = name, actual = "_{}_group".format(name), - visibility = ["//visibility:public"], + visibility = NOT_ACTUALLY_PUBLIC, ) # This matches the raw flag value, e.g. --//python/config_settings:python_version=3.8 @@ -156,30 +153,30 @@ def construct_config_settings( # `whl_library` in the hub repo created by `pip.parse`. flag_values = {"current_config": "will-never-match"}, # Only public so that PyPI hub repo can access it - visibility = _NOT_ACTUALLY_PUBLIC, + visibility = NOT_ACTUALLY_PUBLIC, ) libc = Label("//python/config_settings:py_linux_libc") native.config_setting( name = "_is_py_linux_libc_glibc", flag_values = {libc: "glibc"}, - visibility = _NOT_ACTUALLY_PUBLIC, + visibility = NOT_ACTUALLY_PUBLIC, ) native.config_setting( name = "_is_py_linux_libc_musl", flag_values = {libc: "musl"}, - visibility = _NOT_ACTUALLY_PUBLIC, + visibility = NOT_ACTUALLY_PUBLIC, ) freethreaded = Label("//python/config_settings:py_freethreaded") native.config_setting( name = "_is_py_freethreaded_yes", flag_values = {freethreaded: "yes"}, - visibility = _NOT_ACTUALLY_PUBLIC, + visibility = NOT_ACTUALLY_PUBLIC, ) native.config_setting( name = "_is_py_freethreaded_no", flag_values = {freethreaded: "no"}, - visibility = _NOT_ACTUALLY_PUBLIC, + visibility = NOT_ACTUALLY_PUBLIC, ) def _python_version_flag_impl(ctx): diff --git a/python/private/stamp_impl.bzl b/python/private/stamp_impl.bzl index 6bc0cd9d23..77e3da1f72 100644 --- a/python/private/stamp_impl.bzl +++ b/python/private/stamp_impl.bzl @@ -18,6 +18,8 @@ This module can be removed likely after the following PRs ar addressed: - https://github.com/bazelbuild/bazel/issues/11164 """ +load(":visibility.bzl", "NOT_ACTUALLY_PUBLIC") + StampSettingInfo = provider( doc = "Information about the `--stamp` command line flag", fields = { @@ -50,7 +52,7 @@ Stamped binaries are not rebuilt unless their dependencies change. }, ) -def stamp_build_setting(name, visibility = ["//visibility:public"]): +def stamp_build_setting(name, visibility = NOT_ACTUALLY_PUBLIC): native.config_setting( name = "stamp_detect", values = {"stamp": "1"}, diff --git a/sphinxdocs/sphinxdocs/private/BUILD.bazel b/sphinxdocs/sphinxdocs/private/BUILD.bazel index 0fe0ba4152..fa5ded15f1 100644 --- a/sphinxdocs/sphinxdocs/private/BUILD.bazel +++ b/sphinxdocs/sphinxdocs/private/BUILD.bazel @@ -16,6 +16,7 @@ load("@bazel_skylib//:bzl_library.bzl", "bzl_library") load("@com_google_protobuf//bazel:py_proto_library.bzl", "py_proto_library") load("@rules_python//python:py_binary.bzl", "py_binary") load("@rules_python//python:py_library.bzl", "py_library") +load(":visibility.bzl", "NOT_ACTUALLY_PUBLIC") package( default_visibility = ["//:__subpackages__"], @@ -108,18 +109,23 @@ bzl_library( ], ) +bzl_library( + name = "visibility", + srcs = ["visibility.bzl"], +) + py_binary( name = "inventory_builder", srcs = ["inventory_builder.py"], # Only public because it's an implicit attribute - visibility = ["//visibility:public"], + visibility = NOT_ACTUALLY_PUBLIC, ) py_binary( name = "proto_to_markdown", srcs = ["proto_to_markdown.py"], # Only public because it's an implicit attribute - visibility = ["//visibility:public"], + visibility = NOT_ACTUALLY_PUBLIC, deps = [":proto_to_markdown_lib"], ) @@ -127,7 +133,7 @@ py_library( name = "proto_to_markdown_lib", srcs = ["proto_to_markdown.py"], # Only public because it's an implicit attribute - visibility = ["//visibility:public"], + visibility = NOT_ACTUALLY_PUBLIC, deps = [ ":stardoc_output_proto_py_pb2", ], diff --git a/sphinxdocs/sphinxdocs/private/visibility.bzl b/sphinxdocs/sphinxdocs/private/visibility.bzl new file mode 100644 index 0000000000..706be45a3b --- /dev/null +++ b/sphinxdocs/sphinxdocs/private/visibility.bzl @@ -0,0 +1,7 @@ +"""Shared code for use with visibility specs.""" + +# Use when a target isn't actually public, but needs public +# visibility to keep Bazel happy. +# Such cases are typically for defaults of rule attributes or macro args that +# get used outside of sphinxdocs itself. +NOT_ACTUALLY_PUBLIC = ["//visibility:public"] diff --git a/tools/precompiler/BUILD.bazel b/tools/precompiler/BUILD.bazel index 268f41b032..e055980c72 100644 --- a/tools/precompiler/BUILD.bazel +++ b/tools/precompiler/BUILD.bazel @@ -14,6 +14,7 @@ load("@bazel_skylib//rules:common_settings.bzl", "string_list_flag") load("//python/private:py_interpreter_program.bzl", "py_interpreter_program") # buildifier: disable=bzl-visibility +load("//python/private:visibility.bzl", "NOT_ACTUALLY_PUBLIC") # buildifier: disable=bzl-visibility filegroup( name = "distribution", @@ -25,11 +26,9 @@ py_interpreter_program( name = "precompiler", execution_requirements = ":execution_requirements", main = "precompiler.py", - visibility = [ - # Not actually public. Only public so rules_python-generated toolchains - # are able to reference it. - "//visibility:public", - ], + # Not actually public. Only public so rules_python-generated toolchains + # are able to reference it. + visibility = NOT_ACTUALLY_PUBLIC, ) string_list_flag( diff --git a/tools/private/zipapp/BUILD.bazel b/tools/private/zipapp/BUILD.bazel index 7420776ef3..45bd126c6e 100644 --- a/tools/private/zipapp/BUILD.bazel +++ b/tools/private/zipapp/BUILD.bazel @@ -1,5 +1,6 @@ load("//python:py_library.bzl", "py_library") load("//python/private:py_interpreter_program.bzl", "py_interpreter_program") # buildifier: disable=bzl-visibility +load("//python/private:visibility.bzl", "NOT_ACTUALLY_PUBLIC") # buildifier: disable=bzl-visibility package( default_visibility = ["//:__subpackages__"], @@ -8,11 +9,9 @@ package( py_interpreter_program( name = "zipper", main = "zipper.py", - visibility = [ - # Not actually public. Only public so rules_python-generated toolchains - # are able to reference it. - "//visibility:public", - ], + # Not actually public. Only public so rules_python-generated toolchains + # are able to reference it. + visibility = NOT_ACTUALLY_PUBLIC, ) py_library( @@ -23,11 +22,9 @@ py_library( py_interpreter_program( name = "exe_zip_maker", main = "exe_zip_maker.py", - visibility = [ - # Not actually public. Only public so rules_python-generated toolchains - # are able to reference it. - "//visibility:public", - ], + # Not actually public. Only public so rules_python-generated toolchains + # are able to reference it. + visibility = NOT_ACTUALLY_PUBLIC, ) py_library( @@ -38,11 +35,9 @@ py_library( py_interpreter_program( name = "zip_main_maker", main = "zip_main_maker.py", - visibility = [ - # Not actually public. Only public so rules_python-generated toolchains - # are able to reference it. - "//visibility:public", - ], + # Not actually public. Only public so rules_python-generated toolchains + # are able to reference it. + visibility = NOT_ACTUALLY_PUBLIC, ) py_library( From 19ffd215cff201eccbe12b2a26c4a287413b902d Mon Sep 17 00:00:00 2001 From: 13steinj <13steinj@users.noreply.github.com> Date: Sat, 11 Jul 2026 22:18:58 -0500 Subject: [PATCH 834/922] fix: make executable implicit defaults public so other modules can use rule builders (#3919) `create_executable_rule_builder()` (used by `py_binary_rule_builder()` / `py_test_rule_builder()`, the public `python/api/executables.bzl` API) bundles three implicit attrs whose defaults point at private targets under `//python/private`: `build_data_writer`, `debugger_if_target_config`, and `uncachable_version_file`. These targets had no explicit visibility, so they inherited the package's `default_visibility` (`//:__subpackages__`), which only covers packages inside the rules_python repo itself. Because the builder API is designed to let external modules call `rule()` themselves (via `builder.build()`), Bazel checks visibility of these attr defaults from the *calling* module's package, not from rules_python's. Any external repo constructing a rule via `py_binary_rule_builder()` / `py_test_rule_builder()` therefore fails at analysis time with a visibility error. This has been broken since the builder API's introduction; there's prior art in this same file for exactly this situation (e.g. `stage1_bootstrap_template`, among others) Co-authored-by: Richard Levasseur --- .bazelrc.deleted_packages | 1 + .../other_module/rule_builder/BUILD.bazel | 13 +++++++++++++ .../other_module/rule_builder/app.py | 1 + .../other_module/rule_builder/rule.bzl | 17 +++++++++++++++++ examples/bzlmod/tests/other_module/BUILD.bazel | 11 +++++++++++ news/3919.fixed.md | 3 +++ python/private/BUILD.bazel | 17 +++++++++++++++++ python/private/uncachable_version_file.bzl | 3 ++- 8 files changed, 65 insertions(+), 1 deletion(-) create mode 100644 examples/bzlmod/other_module/other_module/rule_builder/BUILD.bazel create mode 100644 examples/bzlmod/other_module/other_module/rule_builder/app.py create mode 100644 examples/bzlmod/other_module/other_module/rule_builder/rule.bzl create mode 100644 news/3919.fixed.md diff --git a/.bazelrc.deleted_packages b/.bazelrc.deleted_packages index aca014bb56..db42040b5c 100644 --- a/.bazelrc.deleted_packages +++ b/.bazelrc.deleted_packages @@ -7,6 +7,7 @@ common --deleted_packages=examples/bzlmod/entry_points/tests common --deleted_packages=examples/bzlmod/libs/my_lib common --deleted_packages=examples/bzlmod/other_module common --deleted_packages=examples/bzlmod/other_module/other_module/pkg +common --deleted_packages=examples/bzlmod/other_module/other_module/rule_builder common --deleted_packages=examples/bzlmod/patches common --deleted_packages=examples/bzlmod/runfiles common --deleted_packages=examples/bzlmod/tests diff --git a/examples/bzlmod/other_module/other_module/rule_builder/BUILD.bazel b/examples/bzlmod/other_module/other_module/rule_builder/BUILD.bazel new file mode 100644 index 0000000000..40f66b3322 --- /dev/null +++ b/examples/bzlmod/other_module/other_module/rule_builder/BUILD.bazel @@ -0,0 +1,13 @@ +load(":rule.bzl", "custom_py_binary") + +# This target's mere existence is the regression test: analyzing it exercises +# the implicit attr defaults (build_data_writer, debugger_if_target_config, +# uncachable_version_file) that py_binary_rule_builder() bundles from +# rules_python's private package, from a rule() call made in this external +# module. +custom_py_binary( + name = "app", + srcs = ["app.py"], + main = "app.py", + visibility = ["//visibility:public"], +) diff --git a/examples/bzlmod/other_module/other_module/rule_builder/app.py b/examples/bzlmod/other_module/other_module/rule_builder/app.py new file mode 100644 index 0000000000..db51494ed6 --- /dev/null +++ b/examples/bzlmod/other_module/other_module/rule_builder/app.py @@ -0,0 +1 @@ +print("hello from a rule built via py_binary_rule_builder()") diff --git a/examples/bzlmod/other_module/other_module/rule_builder/rule.bzl b/examples/bzlmod/other_module/other_module/rule_builder/rule.bzl new file mode 100644 index 0000000000..7b1505987a --- /dev/null +++ b/examples/bzlmod/other_module/other_module/rule_builder/rule.bzl @@ -0,0 +1,17 @@ +"""A minimal custom py_binary built via the executables rule builder API. + +Regression coverage for https://github.com/bazel-contrib/rules_python/pull/3919: +py_binary_rule_builder()'s implicit attr defaults (build_data_writer, +debugger_if_target_config, uncachable_version_file) previously had no +visibility outside of rules_python, so any external module (like this one) +constructing a rule via the builder failed at analysis time with a +visibility error. +""" + +load("@rules_python//python/api:executables.bzl", "executables") + +def _make_rule(): + builder = executables.py_binary_rule_builder() + return builder.build() + +custom_py_binary = _make_rule() diff --git a/examples/bzlmod/tests/other_module/BUILD.bazel b/examples/bzlmod/tests/other_module/BUILD.bazel index 24231e651a..f4756e7c3e 100644 --- a/examples/bzlmod/tests/other_module/BUILD.bazel +++ b/examples/bzlmod/tests/other_module/BUILD.bazel @@ -14,6 +14,17 @@ build_test( ], ) +# Regression test for https://github.com/bazel-contrib/rules_python/pull/3919: py_binary_rule_builder()'s implicit +# attr defaults previously had no visibility outside of rules_python, so +# building this target (defined via the builder from an external module) +# failed at analysis time with a visibility error. +build_test( + name = "other_module_rule_builder_build_test", + targets = [ + "@our_other_module//other_module/rule_builder:app", + ], +) + py_test( name = "other_module_import_test", srcs = ["other_module_import_test.py"], diff --git a/news/3919.fixed.md b/news/3919.fixed.md new file mode 100644 index 0000000000..0e45d3ce9c --- /dev/null +++ b/news/3919.fixed.md @@ -0,0 +1,3 @@ +Fixed `py_binary_rule_builder()` / `py_test_rule_builder()` (from `python/api/executables.bzl`) +failing at analysis time with a visibility error when used to construct a custom rule from an +external module. diff --git a/python/private/BUILD.bazel b/python/private/BUILD.bazel index 5f51b7650c..17d18fef6f 100644 --- a/python/private/BUILD.bazel +++ b/python/private/BUILD.bazel @@ -70,10 +70,16 @@ alias( "@platforms//os:windows": ":build_data_writer.ps1", "//conditions:default": ":build_data_writer.sh", }), + # Not actually public. Only public because it's an implicit dependency of + # rules built via py_binary_rule_builder() / py_test_rule_builder(). + visibility = ["//visibility:public"], ) define_uncachable_version_file( name = "uncachable_version_file", + # Not actually public. Only public because it's an implicit dependency of + # rules built via py_binary_rule_builder() / py_test_rule_builder(). + visibility = ["//visibility:public"], ) # Needed to define bzl_library targets for docgen. (We don't define the @@ -160,6 +166,9 @@ alias( ":is_bazel_config_mode_target": "//python/config_settings:debugger", "//conditions:default": "//python/private:empty", }), + # Not actually public. Only public because it's an implicit dependency of + # rules built via py_binary_rule_builder() / py_test_rule_builder(). + visibility = ["//visibility:public"], ) bazel_config_mode(name = "bazel_config_mode") @@ -209,10 +218,18 @@ current_interpreter_executable( py_library( name = "empty", + # Not actually public. Only public because it's the resolved default of + # debugger_if_target_config, an implicit dependency of rules built via + # py_binary_rule_builder() / py_test_rule_builder(). + visibility = ["//visibility:public"], ) sentinel( name = "sentinel", + # Not actually public. Only public because it's the resolved default of + # uncachable_version_file, an implicit dependency of rules built via + # py_binary_rule_builder() / py_test_rule_builder(). + visibility = ["//visibility:public"], ) py_binary( diff --git a/python/private/uncachable_version_file.bzl b/python/private/uncachable_version_file.bzl index 9b1d65a469..0f58df8043 100644 --- a/python/private/uncachable_version_file.bzl +++ b/python/private/uncachable_version_file.bzl @@ -28,12 +28,13 @@ actions depending on this file will always re-run. implementation = _uncachable_version_file_impl, ) -def define_uncachable_version_file(name): +def define_uncachable_version_file(name, visibility = None): native.alias( name = name, actual = select({ ":stamp_detect": ":uncachable_version_file_impl", "//conditions:default": ":sentinel", }), + visibility = visibility, ) uncachable_version_file(name = "uncachable_version_file_impl") From 68d08786c9299b768f72a436e07cee332bd99f7c Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Sat, 11 Jul 2026 23:26:06 -0700 Subject: [PATCH 835/922] chore(reviewbot): expand multi-model cascade with fallback endpoints (#3922) Currently, `LocalAgentConfig` only configures a single default model without an explicit endpoint attached when passing `ModelTarget` objects, which causes startup errors (`must have an endpoint configured`) or rate-limit failures when quota runs short. To fix, import `GeminiAPIEndpoint` and `ModelTarget`, create an explicit endpoint (`GeminiAPIEndpoint()`), and configure a prioritized multi-model cascade across all major Gemini 3 and Gemini 2 models so the agent automatically falls back to whichever model has open quota and full function calling capabilities. --- .github/workflows/automated_pr_review.yaml | 48 +++++++++++++------ tools/private/reviewbot/antigravity_review.py | 41 +++++++++++++--- .../reviewbot/skills/review-pr/SKILL.md | 2 + 3 files changed, 71 insertions(+), 20 deletions(-) diff --git a/.github/workflows/automated_pr_review.yaml b/.github/workflows/automated_pr_review.yaml index af2da7e0ca..927adbbd4c 100644 --- a/.github/workflows/automated_pr_review.yaml +++ b/.github/workflows/automated_pr_review.yaml @@ -5,8 +5,6 @@ name: Automated Code Review # to secrets (like GEMINI_API_KEY) even for fork PRs. # Using pull_request for now during setup/testing. on: - pull_request: - types: [opened] issue_comment: types: [created] @@ -15,21 +13,24 @@ permissions: pull-requests: read jobs: + # Always runs so the workflow run exits cleanly without a "No jobs ran" error + # when the review job's if-condition evaluates to false. + noop: + runs-on: ubuntu-latest + steps: + - name: Workflow trigger check + run: echo "Workflow triggered successfully." + review: runs-on: ubuntu-latest - # Trigger only if: - # 1. It is a pull_request event, it is NOT a draft, and the author is a maintainer - # (OWNER, MEMBER, or COLLABORATOR). - # 2. OR it is a regular conversation comment on a pull request, the comment body has a line starting with "/review", - # and the commenter is a maintainer. + # Trigger only if it is a regular comment on a pull request, the comment body has a line + # starting with "/review", and the commenter is a maintainer (OWNER, MEMBER, or COLLABORATOR). if: > - (github.event_name == 'pull_request' && !github.event.pull_request.draft && - contains(fromJson('["OWNER", "MEMBER", "COLLABORATOR"]'), github.event.pull_request.author_association)) || - (github.event_name == 'issue_comment' && github.event.issue.pull_request != null && - (startsWith(github.event.comment.body, '/review') || - contains(github.event.comment.body, '\n/review') || - contains(github.event.comment.body, '\r\n/review')) && - contains(fromJson('["OWNER", "MEMBER", "COLLABORATOR"]'), github.event.comment.author_association)) + github.event_name == 'issue_comment' && github.event.issue.pull_request != null && + (startsWith(github.event.comment.body, '/review') || + contains(github.event.comment.body, '\n/review') || + contains(github.event.comment.body, '\r\n/review')) && + contains(fromJson('["OWNER", "MEMBER", "COLLABORATOR"]'), github.event.comment.author_association) steps: - name: Checkout PR Branch uses: actions/checkout@v7 @@ -40,6 +41,25 @@ jobs: ref: refs/pull/${{ github.event.pull_request.number || github.event.issue.number }}/head persist-credentials: false + - name: Fetch Base Branch and Negotiate Minimal Diff History + env: + PR_COMMITS: ${{ github.event.pull_request.commits }} + run: | + # 1. Fetch the tip of main + git fetch origin main:refs/remotes/origin/main --depth=1 + + # 2. Check if we already have the merge base (common ancestor) + if ! git merge-base origin/main HEAD >/dev/null 2>&1; then + # If we know the exact number of commits in the PR, deepen by (PR_COMMITS + 10) + if [ -n "$PR_COMMITS" ] && [ "$PR_COMMITS" != "null" ]; then + git fetch --deepen="$((PR_COMMITS + 10))" + fi + # 3. If it's an issue_comment event (where PR_COMMITS is null) or still shallow, unshallow/deepen + if ! git merge-base origin/main HEAD >/dev/null 2>&1; then + git fetch --unshallow || git fetch --deepen=50 + fi + fi + - name: Checkout Reviewbot (Base Branch) uses: actions/checkout@v7 with: diff --git a/tools/private/reviewbot/antigravity_review.py b/tools/private/reviewbot/antigravity_review.py index e415e42a91..3aef480f3b 100644 --- a/tools/private/reviewbot/antigravity_review.py +++ b/tools/private/reviewbot/antigravity_review.py @@ -2,14 +2,16 @@ # requires-python = ">=3.11" # dependencies = [ # "google-antigravity", -# "requests", # ] # /// import argparse import asyncio +import subprocess from pathlib import Path from google.antigravity import Agent, CapabilitiesConfig, LocalAgentConfig +from google.antigravity.models import GeminiAPIEndpoint, ModelTarget +from google.antigravity.types import BuiltinTools def parse_args(): @@ -18,11 +20,26 @@ def parse_args(): return parser.parse_args() +def get_pr_diff() -> str: + """Fetches the git diff for the current pull request against origin/main.""" + try: + return subprocess.check_output( + ["git", "diff", "origin/main...HEAD"], text=True, stderr=subprocess.DEVNULL + ) + except Exception: + return "No diff could be automatically extracted via git commands." + + async def main(): args = parse_args() - # Read prompt file - prompt = Path(args.prompt).read_text() + # Read prompt file and pre-hydrate with the exact PR code diff + base_prompt = Path(args.prompt).read_text() + diff_text = get_pr_diff() + prompt = ( + f"{base_prompt}\n\n## Pull Request Git Diff\n" + f"Here is the exact code diff for this pull request:\n```diff\n{diff_text}\n```" + ) # General coordinator instructions for the reviewer agent. system_instructions = ( @@ -30,15 +47,27 @@ async def main(): "reviews on pull requests." ) + # Create the default endpoint picking up GEMINI_API_KEY from the environment. + endpoint = GeminiAPIEndpoint() + # Initialize the Antigravity Agent in read-only mode for security. # Register the review-pr skill from the local reviewbot folder. + # Provide a comprehensive prioritized cascade across Gemini 3 models + # to automatically fall back if any model hits free-tier quota limits (429) + # or temporary unavailability. config = LocalAgentConfig( + models=[ + ModelTarget(name="gemini-3.5-flash", endpoint=endpoint), + ModelTarget(name="gemini-3.1-pro-preview", endpoint=endpoint), + ModelTarget(name="gemini-3.1-flash-lite", endpoint=endpoint), + ModelTarget(name="gemini-3-pro-preview", endpoint=endpoint), + ModelTarget(name="gemini-flash-latest", endpoint=endpoint), + ModelTarget(name="gemini-pro-latest", endpoint=endpoint), + ], system_instructions=system_instructions, skills_paths=[str(Path(__file__).parent / "skills" / "review-pr" / "SKILL.md")], capabilities=CapabilitiesConfig( - allow_filesystem_read=True, - allow_filesystem_write=False, - allow_network=False, + enabled_tools=BuiltinTools.read_only(), ), ) diff --git a/tools/private/reviewbot/skills/review-pr/SKILL.md b/tools/private/reviewbot/skills/review-pr/SKILL.md index 18c998f9ed..915a526686 100644 --- a/tools/private/reviewbot/skills/review-pr/SKILL.md +++ b/tools/private/reviewbot/skills/review-pr/SKILL.md @@ -5,6 +5,8 @@ description: Perform a read-only code review on a pull request. # review-pr +IMPORTANT: The exact git diff for the pull request is pre-provided right in your prompt. Analyze this provided diff directly in a single pass without calling exploratory directory listing or file reading tools unless you specifically need surrounding lines of context from a modified file. + You are an expert Starlark, Python, and Bazel code reviewer. Analyze the changed files for correctness, edge cases, and performance. Focus strictly on logical correctness, concurrency safety, system architecture, performance bottlenecks, From 8c5d11a50236baa56c8110267adac34a26979a74 Mon Sep 17 00:00:00 2001 From: MartinNeudecker Date: Sat, 18 Jul 2026 00:51:24 +0200 Subject: [PATCH 836/922] fix(sphinxdocs): clean stale worker output dir and doctrees on first request (#3933) The Sphinx worker keeps doctrees outside declared outputs so that incremental builds survive across invocations. A newly started worker has no digest history, so it treats every input as changed and re-reads all docs. Re-reading against the stale environment left by a previous worker can produce spurious warnings, e.g. duplicate labels, which become build failures when `--fail-on-warning` is enabled. On the first request, delete the worker output directory and doctree directory so the worker starts from a clean Sphinx environment. --- sphinxdocs/sphinxdocs/private/sphinx_build.py | 13 +++++++++++++ 1 file changed, 13 insertions(+) diff --git a/sphinxdocs/sphinxdocs/private/sphinx_build.py b/sphinxdocs/sphinxdocs/private/sphinx_build.py index 07a123b496..790d4dff84 100644 --- a/sphinxdocs/sphinxdocs/private/sphinx_build.py +++ b/sphinxdocs/sphinxdocs/private/sphinx_build.py @@ -128,6 +128,7 @@ def _prepare_sphinx(self, request): incoming_digests = {} current_digests = self._digests.setdefault(srcdir, {}) + is_first_request = not current_digests changed_paths = [] request_info = {"exec_root": self._exec_root, "inputs": request["inputs"]} for entry in request["inputs"]: @@ -174,6 +175,18 @@ def _prepare_sphinx(self, request): bazel_outdir = sphinx_args[1] worker_outdir = bazel_outdir + ".worker-out.d" + # The doctree dir deliberately lives outside the declared outputs so + # it survives between invocations (that is what makes worker builds + # incremental). A new worker has no digest history: it reports every + # file as changed and re-reads all docs. Doing that against the stale + # Sphinx environment of a previous worker produces spurious warnings + # (e.g. duplicate labels), which --fail-on-warning turns into build + # failures. So on the first request start from a clean slate. + if is_first_request: + shutil.rmtree(worker_outdir, ignore_errors=True) + for arg in sphinx_args: + if arg.startswith("--doctree-dir="): + shutil.rmtree(arg.partition("=")[2], ignore_errors=True) self._worker_outdirs.add(worker_outdir) sphinx_args[1] = worker_outdir From 1faaa82e081c13e193b0cae096af458f358d5c0b Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Fri, 17 Jul 2026 17:38:54 -0700 Subject: [PATCH 837/922] build(deps): bump myst-parser from 4.0.1 to 5.1.0 in /docs (#3927) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Bumps [myst-parser](https://github.com/executablebooks/MyST-Parser) from 4.0.1 to 5.1.0.
Release notes

Sourced from myst-parser's releases.

v5.1.0

✨ New Features

👌 Improvements

  • 👌 Update myst_gfm_only mode to use the unified gfm_plugin, which now includes GFM autolinks, alerts, and improved strikethrough/tasklist handling by @​chrisjsewell in #1128
  • 👌 Improve MathJax 4 compatibility for Sphinx 9 by @​chrisjsewell in #1110
  • 👌 Stop directive-option parsing at colon fences, fixing nested colon fence directives by @​chrisjsewell in #1133

🐛 Bug Fixes

⬆️ Dependency Upgrades

New Contributors

Full Changelog: https://github.com/executablebooks/MyST-Parser/compare/v5.0.0...v5.1.0

v5.0.0

MyST-Parser 5.0.0

Release Date: 2026-01-15

This release significantly bumps the supported versions of core dependencies:

‼️ Breaking Changes

This release updates the minimum supported versions:

  • Python: >=3.11 (dropped Python 3.10, tests up to 3.14)
  • Sphinx: >=8,<10 (dropped Sphinx 7, added Sphinx 9)
  • Docutils: >=0.20,<0.23 (dropped docutils 0.19, added docutils 0.22)
  • markdown-it-py: ~=4.0 (upgraded from v3)

... (truncated)

Changelog

Sourced from myst-parser's changelog.

5.1.0 - 2026-05-13

✨ New Features

👌 Improvements

🐛 Bug Fixes

⬆️ Dependency Upgrades

Full Changelog: v5.0.0...v5.1.0

5.0.0 - 2026-01-15

This release significantly bumps the supported versions of core dependencies:

‼️ Breaking Changes

This release updates the minimum supported versions:

  • Python: >=3.11 (dropped Python 3.10, tests up to 3.14)
  • Sphinx: >=8,<10 (dropped Sphinx 7, added Sphinx 9)
  • Docutils: >=0.20,<0.23 (dropped docutils 0.19, added docutils 0.22)
  • markdown-it-py: ~=4.0 (upgraded from v3)

⬆️ Dependency Upgrades

... (truncated)

Commits
  • 2871eb9 🚀 Release v5.1.0 (#1135)
  • cc5db37 🐛 FIX: Pin mdit-py-plugins>=0.6.1 for nested field list fix (#1134)
  • 4ce57f9 👌 Stop directive-option parsing at colon fences (#1133)
  • cfcc327 ⬆️ Bump mypy from 2.0.0 to 2.1.0 (#1131)
  • 691738c ⬆️ Bump ruff from 0.15.10 to 0.15.12 (#1132)
  • 0fb1ae9 👌 IMPROVE: MathJax 4 compatibility (Sphinx 9) (#1110)
  • f153b4b ⬆️ Bump actions/setup-python from 5 to 6 (#1092)
  • 93acf8d [pre-commit.ci] pre-commit autoupdate (#1095)
  • a5f1d69 ⬆️ Update pygments requirement from <2.20 to <2.21 (#1117)
  • 8381296 🐛 FIX: Use docname instead of source path in warning locations (#1114)
  • Additional commits viewable in compare view

[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=myst-parser&package-manager=pip&previous-version=4.0.1&new-version=5.1.0)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) ---
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot show ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> --- docs/requirements.txt | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/requirements.txt b/docs/requirements.txt index 468cd38c0d..24782bac8f 100644 --- a/docs/requirements.txt +++ b/docs/requirements.txt @@ -331,9 +331,9 @@ myst-parser==3.0.1 ; python_full_version < '3.10' \ --hash=sha256:6457aaa33a5d474aca678b8ead9b3dc298e89c68e67012e73146ea6fd54babf1 \ --hash=sha256:88f0cb406cb363b077d176b51c476f62d60604d68a8dcdf4832e080441301a87 # via rules-python-docs (docs/pyproject.toml) -myst-parser==4.0.1 ; python_full_version == '3.10.*' \ - --hash=sha256:5cfea715e4f3574138aecbf7d54132296bfd72bb614d31168f48c477a830a7c4 \ - --hash=sha256:9134e88959ec3b5780aedf8a99680ea242869d012e8821db3126d427edc9c95d +myst-parser==5.1.0 ; python_full_version == '3.10.*' \ + --hash=sha256:9c91c52b3cdb4d94a6506e4fab4e2f296c7623a0da0dcbe6de1565c3dad67a8a \ + --hash=sha256:ab69322dc6719dcc7f296479dbb70181b66df6ed315064f92dbc85c0e1bf2f02 # via rules-python-docs (docs/pyproject.toml) myst-parser==5.1.0 ; python_full_version >= '3.11' \ --hash=sha256:9c91c52b3cdb4d94a6506e4fab4e2f296c7623a0da0dcbe6de1565c3dad67a8a \ From 8fed9f37afa4d70ddc9259c2bbfe5628e725e78b Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Fri, 17 Jul 2026 17:39:56 -0700 Subject: [PATCH 838/922] build(deps): bump astral-sh/setup-uv from 8.3.0 to 8.3.2 (#3924) Bumps [astral-sh/setup-uv](https://github.com/astral-sh/setup-uv) from 8.3.0 to 8.3.2.
Commits
  • 11f9893 chore: roll up Dependabot updates (#948)
  • f798556 docs: update version references to v8.3.1 (#946)
  • e80544d chore: update known checksums for 0.11.28 (#947)
  • f98e069 Change update-docs PR labels from 'update-docs' to 'documentation' (#945)
  • cd46263 chore: update known checksums for 0.11.27 (#944)
  • 11245c7 docs: update version references to v8.3.0 (#939)
  • See full diff in compare view

[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=astral-sh/setup-uv&package-manager=github_actions&previous-version=8.3.0&new-version=8.3.2)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) ---
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot show ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> --- .github/workflows/automated_pr_review.yaml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.github/workflows/automated_pr_review.yaml b/.github/workflows/automated_pr_review.yaml index 927adbbd4c..1076101db6 100644 --- a/.github/workflows/automated_pr_review.yaml +++ b/.github/workflows/automated_pr_review.yaml @@ -68,7 +68,7 @@ jobs: path: reviewbot - name: Install uv - uses: astral-sh/setup-uv@v8.3.0 + uses: astral-sh/setup-uv@v8.3.2 - name: Run Antigravity Review env: From d0f0f35bd878c5672670f56226df857f66bc4208 Mon Sep 17 00:00:00 2001 From: Ignas Anikevicius <240938+aignas@users.noreply.github.com> Date: Sat, 18 Jul 2026 10:31:15 +0900 Subject: [PATCH 839/922] fix(pypi): handle absolute file URLs in wheel downloader (#3935) This fixes the normalization function to return correctly normalized file URLs which is the minimum that we need to get absolute file URLs handled correctly. Related #3312 --------- Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> --- news/3935.fixed.md | 4 ++++ python/private/pypi/BUILD.bazel | 11 ++++++----- python/private/pypi/urllib.bzl | 21 +++++++++++++++++---- python/private/pypi/whl_library.bzl | 4 +++- tests/pypi/urllib/urllib_tests.bzl | 12 ++++++++++-- 5 files changed, 40 insertions(+), 12 deletions(-) create mode 100644 news/3935.fixed.md diff --git a/news/3935.fixed.md b/news/3935.fixed.md new file mode 100644 index 0000000000..83cd0473e8 --- /dev/null +++ b/news/3935.fixed.md @@ -0,0 +1,4 @@ +(pypi) fixed the URL normalization function to correctly handle local paths +enabling wheel sources files to point to an absolute path. Currently it supports +the `file://` for linux and windows like paths. We also support +envsubst for the said paths from now on. diff --git a/python/private/pypi/BUILD.bazel b/python/private/pypi/BUILD.bazel index 8dd32afb9c..07d2300ce7 100644 --- a/python/private/pypi/BUILD.bazel +++ b/python/private/pypi/BUILD.bazel @@ -510,6 +510,12 @@ bzl_library( ], ) +bzl_library( + name = "urllib", + srcs = ["urllib.bzl"], + deps = ["//python/private:envsubst"], +) + bzl_library( name = "argparse", srcs = ["argparse.bzl"], @@ -555,11 +561,6 @@ bzl_library( srcs = ["platform.bzl"], ) -bzl_library( - name = "urllib", - srcs = ["urllib.bzl"], -) - bzl_library( name = "version_from_filename", srcs = ["version_from_filename.bzl"], diff --git a/python/private/pypi/urllib.bzl b/python/private/pypi/urllib.bzl index ea4cd32cc9..0754334de9 100644 --- a/python/private/pypi/urllib.bzl +++ b/python/private/pypi/urllib.bzl @@ -1,5 +1,7 @@ """Utilities for getting an absolute URL from index_url and the URL we find on PyPI index.""" +load("//python/private:envsubst.bzl", _envsubst = "envsubst") + def _get_root_directory(url): scheme_end = url.find("://") if scheme_end == -1: @@ -53,6 +55,12 @@ def _absolute_url(index_url, candidate): # relative path without up-references return "{}/{}".format(index_url.rstrip("/"), candidate) +def _with_envsubst(url, envsubst = None, getenv = None): + if not url or not envsubst or not getenv: + return url + + return _envsubst(url, envsubst, getenv) + def _strip_empty_path_segments(url): """Removes empty path segments from a URL. Does nothing for urls with no scheme. @@ -65,18 +73,23 @@ def _strip_empty_path_segments(url): The url with empty path segments removed and any trailing slash preserved. If the url had no scheme it is returned unchanged. """ - scheme, _, rest = url.partition("://") + sep = "://" + scheme, _, rest = url.partition(sep) + if scheme.lower() == "file" and rest.startswith("/"): + sep = sep + "/" + rest = rest[1:] + if rest == "": return url stripped = "/".join([p for p in rest.split("/") if p]) if url.endswith("/"): - return "{}://{}/".format(scheme, stripped) + return "{}{}{}/".format(scheme, sep, stripped) else: - return "{}://{}".format(scheme, stripped) + return "{}{}{}".format(scheme, sep, stripped) urllib = struct( is_absolute = _is_downloadable, # Ensure that we strip empty path segments when making an absolute URL - absolute_url = lambda index_url, candidate: _strip_empty_path_segments(_absolute_url(index_url, candidate)), + absolute_url = lambda index_url, candidate, *, envsubst = None, getenv = None: _strip_empty_path_segments(_with_envsubst(_absolute_url(index_url, candidate), envsubst, getenv)), strip_empty_path_segments = _strip_empty_path_segments, ) diff --git a/python/private/pypi/whl_library.bzl b/python/private/pypi/whl_library.bzl index 37cc36492e..1e75d69c8d 100644 --- a/python/private/pypi/whl_library.bzl +++ b/python/private/pypi/whl_library.bzl @@ -327,8 +327,10 @@ def _whl_library_impl(rctx): urls = rctx.attr.urls urls = [ urllib.absolute_url( - envsubst(rctx.attr.index_url, rctx.attr.envsubst, rctx.getenv), + rctx.attr.index_url, url, + envsubst = rctx.attr.envsubst, + getenv = rctx.getenv, ) for url in urls ] diff --git a/tests/pypi/urllib/urllib_tests.bzl b/tests/pypi/urllib/urllib_tests.bzl index 40c48dc854..1a175dc120 100644 --- a/tests/pypi/urllib/urllib_tests.bzl +++ b/tests/pypi/urllib/urllib_tests.bzl @@ -8,7 +8,8 @@ _tests = [] def _test_absolute_url(env): # Already absolute for already_absolute in [ - "file://foo", + "file:///foo", + "file:///c:/foo", "https://foo.com", "http://foo.com", ]: @@ -25,7 +26,13 @@ def _test_absolute_url(env): env.expect.that_str(urllib.absolute_url("https://example.com/relative/", "../relative/file.whl")).equals("https://example.com/relative/file.whl") # Relative URL for files - env.expect.that_str(urllib.absolute_url("file://{PYPI_BAZEL_WORKSPACE_ROOT}", "vendor/distro/file.whl")).equals("file://{PYPI_BAZEL_WORKSPACE_ROOT}/vendor/distro/file.whl") + env.expect.that_str(urllib.absolute_url("file://${PYPI_BAZEL_WORKSPACE_ROOT}", "vendor/distro/file.whl")).equals("file://${PYPI_BAZEL_WORKSPACE_ROOT}/vendor/distro/file.whl") + env.expect.that_str(urllib.absolute_url( + "file://${PYPI_BAZEL_WORKSPACE_ROOT}", + "vendor/distro/file.whl", + envsubst = ["PYPI_BAZEL_WORKSPACE_ROOT"], + getenv = {"PYPI_BAZEL_WORKSPACE_ROOT": "/some/dir"}.get, + )).equals("file:///some/dir/vendor/distro/file.whl") _tests.append(_test_absolute_url) @@ -36,6 +43,7 @@ def _test_strip_empty_path_segments(env): env.expect.that_str(urllib.strip_empty_path_segments("scheme://with///multiple//empty/segments")).equals("scheme://with/multiple/empty/segments") env.expect.that_str(urllib.strip_empty_path_segments("scheme://with//trailing/slash/")).equals("scheme://with/trailing/slash/") env.expect.that_str(urllib.strip_empty_path_segments("scheme://with/trailing/slashes///")).equals("scheme://with/trailing/slashes/") + env.expect.that_str(urllib.strip_empty_path_segments("file:///home/user//foo")).equals("file:///home/user/foo") _tests.append(_test_strip_empty_path_segments) From d92af74f2f719a608e588d71e70bf2256a962f6a Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Fri, 17 Jul 2026 20:21:34 -0700 Subject: [PATCH 840/922] chore(release): automate multi-version backports and support direct patch promotion (#3936) Adds automated workflow and tooling support for multi-version backports and support direct patch release promotion. Previously, backports required manual preparation and tracking across multiple branches, and patch releases were promoted as RCs instead of direct patch releases. This change: * Adds backport_prepare and backport_create_releases scripts and GitHub Action workflows to track and automate cherry-picking PRs across multiple minor release branches. * Renames promote_rc to promote to support promoting patch releases directly. * Refactors release tools to use BackportMetadata dataclass and propagate errors with context (add_note). * Updates RELEASING.md documentation. --- .../workflows/backport_create_releases.yaml | 47 +++ .github/workflows/backport_prepare.yaml | 76 +++++ .github/workflows/on_comment.yaml | 98 ++++-- ...e_promote_rc.yaml => release_promote.yaml} | 4 +- RELEASING.md | 64 +++- tests/tools/private/release/BUILD.bazel | 23 +- .../release/backport_create_releases_test.py | 176 ++++++++++ .../private/release/backport_prepare_test.py | 247 ++++++++++++++ .../{promote_rc_test.py => promote_test.py} | 42 ++- tests/tools/private/release/release_test.py | 4 +- .../private/release/release_test_helper.py | 44 ++- tools/private/release/BUILD.bazel | 13 +- .../release/backport_create_releases.py | 237 ++++++++++++++ tools/private/release/backport_prepare.py | 305 ++++++++++++++++++ tools/private/release/create_release_issue.py | 8 + tools/private/release/gh.py | 55 +++- tools/private/release/git.py | 22 ++ tools/private/release/mock_gh.py | 84 +++++ .../release/{promote_rc.py => promote.py} | 93 +++--- tools/private/release/release.py | 8 +- 20 files changed, 1527 insertions(+), 123 deletions(-) create mode 100644 .github/workflows/backport_create_releases.yaml create mode 100644 .github/workflows/backport_prepare.yaml rename .github/workflows/{release_promote_rc.yaml => release_promote.yaml} (94%) create mode 100644 tests/tools/private/release/backport_create_releases_test.py create mode 100644 tests/tools/private/release/backport_prepare_test.py rename tests/tools/private/release/{promote_rc_test.py => promote_test.py} (90%) create mode 100644 tools/private/release/backport_create_releases.py create mode 100644 tools/private/release/backport_prepare.py create mode 100644 tools/private/release/mock_gh.py rename tools/private/release/{promote_rc.py => promote.py} (75%) diff --git a/.github/workflows/backport_create_releases.yaml b/.github/workflows/backport_create_releases.yaml new file mode 100644 index 0000000000..346bfdbbdb --- /dev/null +++ b/.github/workflows/backport_create_releases.yaml @@ -0,0 +1,47 @@ +name: "Backport: Create Releases" + +on: + workflow_dispatch: + inputs: + issue: + description: 'The Backport Tracking Issue Number' + required: true + type: string + workflow_call: + inputs: + issue: + description: 'The Backport Tracking Issue Number' + required: true + type: string + +permissions: + contents: write + issues: write + pull-requests: write + +jobs: + backport_create_releases: + runs-on: ubuntu-latest + steps: + - name: Checkout repository + uses: actions/checkout@v7 + with: + fetch-depth: 0 + + - name: Setup Bazel + uses: bazel-contrib/setup-bazel@0.19.0 + with: + bazelisk-version: 1.20.0 + + - name: Configure Git Identity + run: | + git config --global user.name "github-actions[bot]" + git config --global user.email "41898282+github-actions[bot]@users.noreply.github.com" + + - name: Run Backport Create Releases Pipeline + env: + GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} + ISSUE: ${{ inputs.issue }} + run: | + bazel run //tools/private/release -- \ + backport-create-releases --issue=$ISSUE --no-dry-run diff --git a/.github/workflows/backport_prepare.yaml b/.github/workflows/backport_prepare.yaml new file mode 100644 index 0000000000..587c0fc357 --- /dev/null +++ b/.github/workflows/backport_prepare.yaml @@ -0,0 +1,76 @@ +name: "Backport: Prepare" + +on: + workflow_dispatch: + inputs: + issue: + description: 'The Backport Tracking Issue Number' + required: false + type: string + pr: + description: 'PR to backport (e.g. #123) (if no issue)' + required: false + type: string + from_minor: + description: 'From minor version (e.g. 1.7) (if no issue)' + required: false + type: string + to_minor: + description: 'To minor version (e.g. 1.9) (if no issue, optional)' + required: false + type: string + workflow_call: + inputs: + issue: + description: 'The Backport Tracking Issue Number' + required: false + type: string + +permissions: + contents: write + issues: write + pull-requests: write + +jobs: + backport_prepare: + runs-on: ubuntu-latest + steps: + - name: Checkout repository + uses: actions/checkout@v7 + with: + fetch-depth: 0 + + - name: Setup Bazel + uses: bazel-contrib/setup-bazel@0.19.0 + with: + bazelisk-version: 1.20.0 + + - name: Configure Git Identity + run: | + git config --global user.name "github-actions[bot]" + git config --global user.email "41898282+github-actions[bot]@users.noreply.github.com" + + - name: Run Backport Preparation Pipeline + env: + GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} + ISSUE: ${{ inputs.issue }} + PR: ${{ inputs.pr }} + FROM_MINOR: ${{ inputs.from_minor }} + TO_MINOR: ${{ inputs.to_minor }} + run: | + ARGS=() + if [ -n "$ISSUE" ]; then + ARGS+=("--issue=$ISSUE") + else + if [ -n "$PR" ]; then + ARGS+=("--pr=$PR") + fi + if [ -n "$FROM_MINOR" ]; then + ARGS+=("--from-minor=$FROM_MINOR") + fi + if [ -n "$TO_MINOR" ]; then + ARGS+=("--to-minor=$TO_MINOR") + fi + fi + bazel run //tools/private/release -- \ + backport-prepare "${ARGS[@]}" --no-dry-run diff --git a/.github/workflows/on_comment.yaml b/.github/workflows/on_comment.yaml index ed7813adac..7e4645f3fe 100644 --- a/.github/workflows/on_comment.yaml +++ b/.github/workflows/on_comment.yaml @@ -36,45 +36,67 @@ jobs: IS_PR: "${{ github.event.issue.pull_request != null }}" EVENT_NUMBER: "${{ github.event.issue.number }}" HAS_RELEASE_LABEL: "${{ contains(github.event.issue.labels.*.name, 'type: release') }}" + HAS_BACKPORT_LABEL: "${{ contains(github.event.issue.labels.*.name, 'type: backport-pr') }}" GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} run: | - if [ "$IS_PR" = "false" ] && [ "$HAS_RELEASE_LABEL" = "true" ]; then + if [ "$IS_PR" = "false" ]; then + # Set issue number for non-PR issues (release or backport tracking issues) issue_number=$EVENT_NUMBER - if echo "$COMMENT_BODY" | grep -qE '^[[:space:]]*/create-rc([[:space:]]|$)'; then - echo "command=create-rc" >> "$GITHUB_OUTPUT" - echo "issue_number=$issue_number" >> "$GITHUB_OUTPUT" - elif echo "$COMMENT_BODY" | grep -qE '^[[:space:]]*/prepare([[:space:]]|$)'; then - echo "command=prepare" >> "$GITHUB_OUTPUT" - echo "issue_number=$issue_number" >> "$GITHUB_OUTPUT" - elif echo "$COMMENT_BODY" | grep -qE '^[[:space:]]*/process-backports([[:space:]]|$)'; then - echo "command=process-backports" >> "$GITHUB_OUTPUT" - echo "issue_number=$issue_number" >> "$GITHUB_OUTPUT" - elif echo "$COMMENT_BODY" | grep -qE '^[[:space:]]*/add-backports([[:space:]]|$)'; then - args=$(echo "$COMMENT_BODY" | grep -E '^[[:space:]]*/add-backports([[:space:]]|$)' | sed -E 's/^[[:space:]]*\/add-backports[[:space:]]*//') - args=$(echo "$args" | sed -e 's/^[[:space:],]*//' -e 's/[[:space:],]*$//') - csv=$(echo "$args" | sed -E 's/[[:space:],]+/ /g' | tr ' ' ',') - if [ -n "$csv" ]; then - echo "command=add-backports" >> "$GITHUB_OUTPUT" - echo "backports=$csv" >> "$GITHUB_OUTPUT" - echo "issue_number=$issue_number" >> "$GITHUB_OUTPUT" + echo "issue_number=$issue_number" >> "$GITHUB_OUTPUT" + echo "issue_number=$issue_number" >> "$GITHUB_ENV" + + # Check if it's a release tracking issue + if [ "$HAS_RELEASE_LABEL" = "true" ]; then + # Handle /create-rc comment + if echo "$COMMENT_BODY" | grep -qE '^[[:space:]]*/create-rc([[:space:]]|$)'; then + echo "command=create-rc" >> "$GITHUB_OUTPUT" + # Handle /prepare comment + elif echo "$COMMENT_BODY" | grep -qE '^[[:space:]]*/prepare([[:space:]]|$)'; then + echo "command=prepare" >> "$GITHUB_OUTPUT" + # Handle /process-backports comment + elif echo "$COMMENT_BODY" | grep -qE '^[[:space:]]*/process-backports([[:space:]]|$)'; then + echo "command=process-backports" >> "$GITHUB_OUTPUT" + # Handle /add-backports comment + elif echo "$COMMENT_BODY" | grep -qE '^[[:space:]]*/add-backports([[:space:]]|$)'; then + args=$(echo "$COMMENT_BODY" | grep -E '^[[:space:]]*/add-backports([[:space:]]|$)' | sed -E 's/^[[:space:]]*\/add-backports[[:space:]]*//') + args=$(echo "$args" | sed -e 's/^[[:space:],]*//' -e 's/[[:space:],]*$//') + csv=$(echo "$args" | sed -E 's/[[:space:],]+/ /g' | tr ' ' ',') + if [ -n "$csv" ]; then + echo "command=add-backports" >> "$GITHUB_OUTPUT" + echo "backports=$csv" >> "$GITHUB_OUTPUT" + else + echo "command=none" >> "$GITHUB_OUTPUT" + echo "Error: No PRs specified for add-backports." >&2 + gh api \ + --method POST \ + -H "Accept: application/vnd.github+json" \ + -H "X-GitHub-Api-Version: 2022-11-28" \ + /repos/${{ github.repository }}/issues/comments/${{ github.event.comment.id }}/reactions \ + -f "content=-1" + fi + # Handle /promote comment + elif echo "$COMMENT_BODY" | grep -qE '^[[:space:]]*/promote([[:space:]]|$)'; then + echo "command=promote" >> "$GITHUB_OUTPUT" + else + echo "command=none" >> "$GITHUB_OUTPUT" + fi + # Check if it's a backport tracking issue + elif [ "$HAS_BACKPORT_LABEL" = "true" ]; then + # Handle /prepare comment for backports + if echo "$COMMENT_BODY" | grep -qE '^[[:space:]]*/prepare([[:space:]]|$)'; then + echo "command=backport-prepare" >> "$GITHUB_OUTPUT" + # Handle /create-releases comment for backports + elif echo "$COMMENT_BODY" | grep -qE '^[[:space:]]*/create-releases([[:space:]]|$)'; then + echo "command=backport-create-releases" >> "$GITHUB_OUTPUT" else echo "command=none" >> "$GITHUB_OUTPUT" - echo "Error: No PRs specified for add-backports." >&2 - gh api \ - --method POST \ - -H "Accept: application/vnd.github+json" \ - -H "X-GitHub-Api-Version: 2022-11-28" \ - /repos/${{ github.repository }}/issues/comments/${{ github.event.comment.id }}/reactions \ - -f "content=-1" fi - elif echo "$COMMENT_BODY" | grep -qE '^[[:space:]]*/promote([[:space:]]|$)'; then - echo "command=promote" >> "$GITHUB_OUTPUT" - echo "issue_number=$issue_number" >> "$GITHUB_OUTPUT" else echo "command=none" >> "$GITHUB_OUTPUT" fi elif [ "$IS_PR" = "true" ]; then pr_number=$EVENT_NUMBER + # Handle /backport comment on PR if echo "$COMMENT_BODY" | grep -qE '^[[:space:]]*/backport([[:space:]]|$)'; then echo "command=pr-backport" >> "$GITHUB_OUTPUT" echo "pr_number=$pr_number" >> "$GITHUB_OUTPUT" @@ -134,7 +156,23 @@ jobs: call_promote: needs: parse_comment if: needs.parse_comment.outputs.command == 'promote' - uses: ./.github/workflows/release_promote_rc.yaml + uses: ./.github/workflows/release_promote.yaml with: issue: ${{ needs.parse_comment.outputs.issue_number }} - secrets: inherit \ No newline at end of file + secrets: inherit + + call_backport_prepare: + needs: parse_comment + if: needs.parse_comment.outputs.command == 'backport-prepare' + uses: ./.github/workflows/backport_prepare.yaml + with: + issue: ${{ needs.parse_comment.outputs.issue_number }} + secrets: inherit + + call_backport_create_releases: + needs: parse_comment + if: needs.parse_comment.outputs.command == 'backport-create-releases' + uses: ./.github/workflows/backport_create_releases.yaml + with: + issue: ${{ needs.parse_comment.outputs.issue_number }} + secrets: inherit diff --git a/.github/workflows/release_promote_rc.yaml b/.github/workflows/release_promote.yaml similarity index 94% rename from .github/workflows/release_promote_rc.yaml rename to .github/workflows/release_promote.yaml index 399e7b1fa2..93ec23a882 100644 --- a/.github/workflows/release_promote_rc.yaml +++ b/.github/workflows/release_promote.yaml @@ -1,4 +1,4 @@ -name: "Release: Promote RC" +name: "Release: Promote" on: workflow_dispatch: @@ -60,7 +60,7 @@ jobs: ARGS+=("--remote=origin") ARGS+=("--no-dry-run") - bazel run //tools/private/release -- promote-rc "${ARGS[@]}" + bazel run //tools/private/release -- promote "${ARGS[@]}" publish: needs: promote diff --git a/RELEASING.md b/RELEASING.md index 2298b4c8a3..a03cd83652 100644 --- a/RELEASING.md +++ b/RELEASING.md @@ -123,7 +123,69 @@ If a backport fails to process (e.g., due to cherry-pick conflicts): cherry-pick the PR, resolve conflicts, push to remote, and manually check the box on the tracking issue checklist with `status=done` metadata. -## Patch release with cherry picks +## Automated patch releases to multiple versions + +If you need to backport a PR to multiple older active release branches (e.g., +backporting a fix to `1.7`, `1.8`, and `1.9` when the latest release is +`2.2.0`), you can automate the creation of release tracking issues and +verification of cherry-picks using a Backport Tracking Issue. + +### Steps + +1. **Create a Backport Tracking Issue**: Create a new issue on GitHub with + the label `type: backport-pr`. + * The title should describe the backport, e.g., `Backport: #1234` + (referencing the PR to backport). + * The body must contain the target range in the following format: + ```markdown + * PR: #1234 + * From version: 1.7 + * To version: 2.2 + ``` + This tells the tool to target all active release branches between + `release/1.7` and `release/2.2` inclusive. + +2. **Prepare the Backports**: Trigger the preparation by commenting `/prepare` + on the backport tracking issue, or by manually running the [Backport: + Prepare](https://github.com/bazel-contrib/rules_python/actions/workflows/backport_prepare.yaml) + workflow with the issue number. + * The workflow will checkout each release branch in the range, attempt to + cherry-pick the PR, and verify if the changelog can be updated. + * It will then update the backport tracking issue body with a list of + tasks: + * `Verify apply `: Automatically checked if the + cherry-pick succeeded, or left unchecked with + `status=failed-conflict` or `status=failed-changelog` if it failed. + * `Release issue `: A task to initiate the release for + each target version. + +3. **Resolve Conflicts (if any)**: If any `Verify apply` task failed: + * You must resolve the conflict manually on that specific release branch + (checkout branch, cherry-pick, resolve conflicts, push to remote). + * Once resolved, manually check the corresponding `Verify apply` task box + on the tracking issue and set its metadata to `status=success` (e.g., + `- [x] Verify apply 1.8 | status=success`). + +4. **Initiate Releases**: Once all `Verify apply` tasks are successful + (either automatically or after manual resolution), comment + `/create-releases` on the backport tracking issue, or manually run the + [Backport: Create + Releases](https://github.com/bazel-contrib/rules_python/actions/workflows/backport_create_releases.yaml) + workflow. + * This will automatically create a standard Release Tracking Issue for + each target version (e.g., `Release 1.7.1`, `Release 1.8.1`, etc.). + * For patch releases, the created release tracking issues will have `Tag + RC` tasks automatically removed, as release candidates are not + required for patch releases. + * The backport PR will be automatically added to the checklist of each + created release tracking issue. + +5. **Execute Releases**: Follow the standard release process for each created + release tracking issue. Since these are patch releases, you can skip the + `/create-rc` step and comment `/promote` directly to tag and publish the + release from the release branch head. + +## Manual patch release with cherry picks If a patch release from head would contain changes that aren't appropriate for a patch release, then the patch release needs to be based on the original diff --git a/tests/tools/private/release/BUILD.bazel b/tests/tools/private/release/BUILD.bazel index 633cfa328f..d49d8fd221 100644 --- a/tests/tools/private/release/BUILD.bazel +++ b/tests/tools/private/release/BUILD.bazel @@ -4,6 +4,7 @@ py_library( name = "release_test_helper", srcs = ["release_test_helper.py"], deps = [ + "//tools/private/release:mock_gh", "//tools/private/release:release_lib", ], ) @@ -97,8 +98,8 @@ py_test( ) py_test( - name = "promote_rc_test", - srcs = ["promote_rc_test.py"], + name = "promote_test", + srcs = ["promote_test.py"], deps = [ ":release_test_helper", "//tools/private/release:release_lib", @@ -130,3 +131,21 @@ py_test( "@dev_pip//packaging", ], ) + +py_test( + name = "backport_prepare_test", + srcs = ["backport_prepare_test.py"], + deps = [ + ":release_test_helper", + "//tools/private/release:release_lib", + ], +) + +py_test( + name = "backport_create_releases_test", + srcs = ["backport_create_releases_test.py"], + deps = [ + ":release_test_helper", + "//tools/private/release:release_lib", + ], +) diff --git a/tests/tools/private/release/backport_create_releases_test.py b/tests/tools/private/release/backport_create_releases_test.py new file mode 100644 index 0000000000..f1bdfe477f --- /dev/null +++ b/tests/tools/private/release/backport_create_releases_test.py @@ -0,0 +1,176 @@ +import argparse +import unittest + +from tests.tools.private.release.release_test_helper import ReleaseToolTestCase +from tools.private.release.backport_create_releases import BackportCreateReleases + + +class CmdBackportCreateReleasesTest(ReleaseToolTestCase): + def test_create_releases_all_success(self): + # Arrange + args = argparse.Namespace(issue=123, dry_run=False) + gh = self.gh + + # Setup backport issue in mock GH + backport_body = """* PR: #456 +* From version: 1.7 +* To version: 1.9 + +## Tasks + +- [x] Verify apply 1.7 | status=success +- [x] Verify apply 1.8 | status=success +- [x] Verify apply 1.9 | status=success +- [ ] Track Release 1.7.2 +- [ ] Track Release 1.8.1 +- [ ] Track Release 1.9.0""" + + gh.issues[123] = { + "title": "Backport: #456", + "body": backport_body, + "labels": ["type:backport-pr"], + "number": 123, + "url": "https://github.com/.../issues/123", + } + + # Act + result = BackportCreateReleases(args, gh).run() + + # Assert + self.assertEqual(result, 0) + + # Verify release issues created (IDs 1001, 1002, 1003) + self.assertIn(1001, gh.issues) + self.assertIn(1002, gh.issues) + self.assertIn(1003, gh.issues) + + # 1.7.2 (patch) should not have Tag RC0 + self.assertEqual(gh.issues[1001]["title"], "Release 1.7.2") + self.assertNotIn("Tag RC0", gh.issues[1001]["body"]) + self.assertIn("## Backports\n- [ ] #456", gh.issues[1001]["body"]) + + # 1.9.0 (minor) should have Tag RC0 + self.assertEqual(gh.issues[1003]["title"], "Release 1.9.0") + self.assertIn("Tag RC0", gh.issues[1003]["body"]) + + # Verify backport issue updated + expected_updated_backport_body = """* PR: #456 +* From version: 1.7 +* To version: 1.9 + +## Tasks + +- [x] Verify apply 1.7 | status=success +- [x] Verify apply 1.8 | status=success +- [x] Verify apply 1.9 | status=success +- [x] Track Release 1.7.2 | status=success release_issue=#1001 +- [x] Track Release 1.8.1 | status=success release_issue=#1002 +- [x] Track Release 1.9.0 | status=success release_issue=#1003""" + + self.assertEqual(gh.issues[123]["body"], expected_updated_backport_body) + + def test_create_releases_dependency_blocking(self): + # Arrange + args = argparse.Namespace(issue=123, dry_run=False) + gh = self.gh + + # 1.8 failed, 1.7 and 1.9 succeeded + backport_body = """* PR: #456 +* From version: 1.7 +* To version: 1.9 + +## Tasks + +- [x] Verify apply 1.7 | status=success +- [ ] Verify apply 1.8 | status=failed-conflict +- [x] Verify apply 1.9 | status=success +- [ ] Track Release 1.7.2 +- [ ] Track Release 1.8.1 +- [ ] Track Release 1.9.0""" + + gh.issues[123] = { + "title": "Backport: #456", + "body": backport_body, + "labels": ["type:backport-pr"], + "number": 123, + "url": "https://github.com/.../issues/123", + } + + # Act + result = BackportCreateReleases(args, gh).run() + + # Assert + self.assertEqual(result, 0) + + # Only 1.9.0 (ID 1001) should be created. 1.7.2 and 1.8.1 are blocked by 1.8 failure. + self.assertEqual(len(gh.issues), 2) # Backport issue + 1 release issue + self.assertIn(1001, gh.issues) + self.assertEqual(gh.issues[1001]["title"], "Release 1.9.0") + + # Verify backport issue updated with block statuses + expected_updated_backport_body = """* PR: #456 +* From version: 1.7 +* To version: 1.9 + +## Tasks + +- [x] Verify apply 1.7 | status=success +- [ ] Verify apply 1.8 | status=failed-conflict +- [x] Verify apply 1.9 | status=success +- [ ] Track Release 1.7.2 | status=error-later-release-did-not-apply +- [ ] Track Release 1.8.1 | status=error-later-release-did-not-apply +- [x] Track Release 1.9.0 | status=success release_issue=#1001""" + + self.assertEqual(gh.issues[123]["body"], expected_updated_backport_body) + + def test_create_releases_already_exists(self): + # Arrange + args = argparse.Namespace(issue=123, dry_run=False) + gh = self.gh + + backport_body = """* PR: #456 +* From version: 1.7 +* To version: 1.7 + +## Tasks + +- [x] Verify apply 1.7 | status=success +- [ ] Track Release 1.7.2""" + + gh.issues[123] = { + "title": "Backport: #456", + "body": backport_body, + "labels": ["type:backport-pr"], + "number": 123, + "url": "https://github.com/.../issues/123", + } + + # Pre-create the release issue to simulate it already existing + gh.create_issue( + title="Release 1.7.2", + body="existing body\n## Backports\n", + labels=["type: release"], + ) # Will get ID 1001 + + # Act + result = BackportCreateReleases(args, gh).run() + + # Assert + self.assertEqual(result, 0) + self.assertEqual(len(gh.issues), 2) + + # Should link existing issue 1001 + expected_updated_backport_body = """* PR: #456 +* From version: 1.7 +* To version: 1.7 + +## Tasks + +- [x] Verify apply 1.7 | status=success +- [x] Track Release 1.7.2 | status=success release_issue=#1001""" + + self.assertEqual(gh.issues[123]["body"], expected_updated_backport_body) + + +if __name__ == "__main__": + unittest.main() diff --git a/tests/tools/private/release/backport_prepare_test.py b/tests/tools/private/release/backport_prepare_test.py new file mode 100644 index 0000000000..26883b78b3 --- /dev/null +++ b/tests/tools/private/release/backport_prepare_test.py @@ -0,0 +1,247 @@ +import argparse +import unittest +from unittest.mock import call, patch + +from tests.tools.private.release.release_test_helper import ( + ReleaseToolTestCase, + _mock_git, +) +from tools.private.release.backport_prepare import BackportPrepare +from tools.private.release.gh import BACKPORT_LABEL + + +class CmdBackportPrepareTest(ReleaseToolTestCase): + def setUp(self): + super().setUp() + _mock_git(self) + # Mock changelog_news and determine_next_version + self.patcher_news = patch( + "tools.private.release.backport_prepare.changelog_news" + ) + self.mock_news = self.patcher_news.start() + + self.patcher_det = patch( + "tools.private.release.backport_prepare.determine_next_version" + ) + self.mock_det = self.patcher_det.start() + + self.addCleanup(self.patcher_news.stop) + self.addCleanup(self.patcher_det.stop) + + def test_prepare_from_issue_success(self): + # Arrange + args = argparse.Namespace( + issue=123, + pr=None, + from_minor=None, + to_minor=None, + remote="my-remote", + dry_run=False, + ) + + # Setup backport issue in mock GH + backport_body = "* PR: #456\n* From version: 1.7\n* To version: 1.9\n" + self.gh.issues[123] = { + "title": "Backport: #456", + "body": backport_body, + "labels": ["type: backport-pr"], + "number": 123, + "url": "https://github.com/.../issues/123", + } + + # Setup PR info in mock GH + self.gh.prs[456] = { + "state": "MERGED", + "mergeCommit": {"oid": "pr_merge_sha_12345"}, + } + + # Mock remote branches + self.mock_git.get_remote_branches.return_value = [ + "main", + "release/1.6", + "release/1.7", + "release/1.8", + "release/1.9", + "release/2.0", + ] + self.mock_git.get_current_branch.return_value = "work-branch" + + # Mock next versions + self.mock_det.side_effect = ["1.7.2", "1.8.1", "1.9.0"] + + # Act + result = BackportPrepare(args, self.mock_git, self.gh).run() + + # Assert + self.assertEqual(result, 0) + self.mock_git.fetch.assert_called_once_with("my-remote", tags=True, force=True) + + # Verify checkouts and cherry-picks + self.mock_git.checkout.assert_has_calls( + [ + call("release/1.7", track_remote="my-remote"), + call("release/1.8", track_remote="my-remote"), + call("release/1.9", track_remote="my-remote"), + call("work-branch"), # Restored branch + ] + ) + + self.mock_git.cherry_pick.assert_has_calls( + [ + call("pr_merge_sha_12345"), + call("pr_merge_sha_12345"), + call("pr_merge_sha_12345"), + ] + ) + + # Verify changelog updates + self.mock_news.update_changelog.assert_has_calls( + [ + call("1.7.2", unittest.mock.ANY), + call("1.8.1", unittest.mock.ANY), + call("1.9.0", unittest.mock.ANY), + ] + ) + + # Verify issue body update + expected_body = ( + "* PR: #456\n" + "* From version: 1.7\n" + "* To version: 1.9\n" + "\n" + "## Tasks\n" + "\n" + "- [x] Verify apply 1.7 | status=success\n" + "- [x] Verify apply 1.8 | status=success\n" + "- [x] Verify apply 1.9 | status=success\n" + "- [ ] Track Release 1.7.2\n" + "- [ ] Track Release 1.8.1\n" + "- [ ] Track Release 1.9.0" + ) + self.assertEqual(self.gh.issues[123]["body"], expected_body) + + def test_prepare_manual_success(self): + # Arrange + args = argparse.Namespace( + issue=None, + pr="#456", + from_minor="1.7", + to_minor="1.8", + remote="my-remote", + dry_run=False, + ) + self.gh.prs[456] = { + "state": "MERGED", + "mergeCommit": {"oid": "pr_merge_sha_12345"}, + } + self.mock_git.get_remote_branches.return_value = [ + "release/1.7", + "release/1.8", + ] + self.mock_git.get_current_branch.return_value = "work-branch" + self.mock_det.side_effect = ["1.7.2", "1.8.1"] + + # Act + result = BackportPrepare(args, self.mock_git, self.gh).run() + + # Assert + self.assertEqual(result, 0) + self.assertIn(1001, self.gh.issues) + issue = self.gh.issues[1001] + self.assertEqual(issue["title"], "Backport: #456") + self.assertEqual(issue["labels"], [BACKPORT_LABEL]) + body = issue["body"] + self.assertIn("- [x] Verify apply 1.7 | status=success", body) + self.assertIn("- [x] Verify apply 1.8 | status=success", body) + self.assertIn("- [ ] Track Release 1.7.2", body) + self.assertIn("- [ ] Track Release 1.8.1", body) + + def test_prepare_manual_with_patch_versions(self): + # Arrange + args = argparse.Namespace( + issue=None, + pr="#456", + from_minor="1.7.0", + to_minor="1.8.0", + remote="my-remote", + dry_run=False, + ) + self.gh.prs[456] = { + "state": "MERGED", + "mergeCommit": {"oid": "pr_merge_sha_12345"}, + } + self.mock_git.get_remote_branches.return_value = [ + "release/1.7", + "release/1.8", + ] + self.mock_git.get_current_branch.return_value = "work-branch" + self.mock_det.side_effect = ["1.7.2", "1.8.1"] + + # Act + result = BackportPrepare(args, self.mock_git, self.gh).run() + + # Assert + self.assertEqual(result, 0) + self.assertIn(1001, self.gh.issues) + body = self.gh.issues[1001]["body"] + self.assertIn("- [x] Verify apply 1.7 | status=success", body) + self.assertIn("- [x] Verify apply 1.8 | status=success", body) + + def test_prepare_verify_failed(self): + # Arrange + args = argparse.Namespace( + issue=123, + pr=None, + from_minor=None, + to_minor=None, + remote="my-remote", + dry_run=False, + ) + issue_body = "* PR: #456\n* From version: 1.7\n* To version: 1.8\n" + self.gh.issues[123] = { + "title": "Backport: #456", + "body": issue_body, + "labels": ["type: backport-pr"], + "number": 123, + "url": "https://github.com/.../issues/123", + } + self.gh.prs[456] = { + "state": "MERGED", + "mergeCommit": {"oid": "pr_merge_sha_12345"}, + } + self.mock_git.get_remote_branches.return_value = [ + "release/1.7", + "release/1.8", + ] + self.mock_git.get_current_branch.return_value = "work-branch" + self.mock_det.side_effect = ["1.7.2", "1.8.1"] + + # Mock cherry-pick failure on 1.7 and changelog failure on 1.8 + self.mock_git.cherry_pick.side_effect = [Exception("Conflict"), None] + self.mock_news.update_changelog.side_effect = [Exception("Changelog error")] + + # Act + result = BackportPrepare(args, self.mock_git, self.gh).run() + + # Assert + self.assertEqual( + result, 0 + ) # Returns 0 even if verification fails, it just updates tasks + + expected_body = ( + "* PR: #456\n" + "* From version: 1.7\n" + "* To version: 1.8\n" + "\n" + "## Tasks\n" + "\n" + "- [ ] Verify apply 1.7 | status=failed-conflict\n" + "- [ ] Verify apply 1.8 | status=failed-changelog\n" + "- [ ] Track Release 1.7.2\n" + "- [ ] Track Release 1.8.1" + ) + self.assertEqual(self.gh.issues[123]["body"], expected_body) + + +if __name__ == "__main__": + unittest.main() diff --git a/tests/tools/private/release/promote_rc_test.py b/tests/tools/private/release/promote_test.py similarity index 90% rename from tests/tools/private/release/promote_rc_test.py rename to tests/tools/private/release/promote_test.py index b5dff3e478..8d1d2cfb29 100644 --- a/tests/tools/private/release/promote_rc_test.py +++ b/tests/tools/private/release/promote_test.py @@ -7,10 +7,10 @@ from tests.tools.private.release.release_test_helper import _mock_git_and_gh from tools.private.release.gh import NoTrackingIssueError -from tools.private.release.promote_rc import PromoteRc +from tools.private.release.promote import Promote -class CmdPromoteRcTest(unittest.TestCase): +class CmdPromoteTest(unittest.TestCase): def setUp(self): _mock_git_and_gh(self) self.test_dir = tempfile.TemporaryDirectory() @@ -28,7 +28,7 @@ def test_promote_rc_success(self): self.mock_gh.get_issue_body.return_value = initial_body # Act - result = PromoteRc(args, self.mock_git, self.mock_gh).run() + result = Promote(args, self.mock_git, self.mock_gh).run() # Assert self.assertEqual(result, 0) @@ -60,7 +60,7 @@ def test_promote_rc_success(self): "- [Github Release 2.0.0](https://github.com/bazel-contrib/rules_python/releases/tag/2.0.0)\n" "- [BCR Entry 2.0.0](https://registry.bazel.build/modules/rules_python/2.0.0)\n" "- [BCR PRs](https://github.com/bazelbuild/bazel-central-registry/pulls?q=is%3Apr%20%28%22bazel-contrib/rules_python%22%20in%3Atitle%29%20%28%22%402.0.0%22%20in%3Atitle%29)\n" - "- [Release workflow status](https://github.com/bazel-contrib/rules_python/actions/workflows/release_promote_rc.yaml)" + "- [Release workflow status](https://github.com/bazel-contrib/rules_python/actions/workflows/release_promote.yaml)" ) self.mock_gh.post_issue_comment.assert_called_once_with(123, expected_comment) @@ -78,7 +78,7 @@ def test_promote_rc_writes_github_output(self): # Act with patch.dict("os.environ", {"GITHUB_OUTPUT": github_output_path}): - result = PromoteRc(args, self.mock_git, self.mock_gh).run() + result = Promote(args, self.mock_git, self.mock_gh).run() # Assert self.assertEqual(result, 0) @@ -100,7 +100,7 @@ def test_promote_rc_resolve_issue_success(self): self.mock_gh.get_issue_body.return_value = initial_body # Act - result = PromoteRc(args, self.mock_git, self.mock_gh).run() + result = Promote(args, self.mock_git, self.mock_gh).run() # Assert self.assertEqual(result, 0) @@ -130,24 +130,22 @@ def test_promote_rc_resolve_issue_success(self): "- [Github Release 2.0.0](https://github.com/bazel-contrib/rules_python/releases/tag/2.0.0)\n" "- [BCR Entry 2.0.0](https://registry.bazel.build/modules/rules_python/2.0.0)\n" "- [BCR PRs](https://github.com/bazelbuild/bazel-central-registry/pulls?q=is%3Apr%20%28%22bazel-contrib/rules_python%22%20in%3Atitle%29%20%28%22%402.0.0%22%20in%3Atitle%29)\n" - "- [Release workflow status](https://github.com/bazel-contrib/rules_python/actions/workflows/release_promote_rc.yaml)" + "- [Release workflow status](https://github.com/bazel-contrib/rules_python/actions/workflows/release_promote.yaml)" ) self.mock_gh.post_issue_comment.assert_called_once_with(123, expected_comment) - def test_promote_rc_resolves_version_from_issue(self): + def test_promote_patch_success(self): # Arrange args = argparse.Namespace( - version=None, issue=123, dry_run=False, remote="my-remote" + version="2.0.1", issue=123, dry_run=False, remote="my-remote" ) - self.mock_gh.get_issue_title.return_value = "Release 2.0.1" - self.mock_git.get_remote_tags.return_value = ["2.0.1-rc0"] - self.mock_git.get_commit_sha.return_value = "12345678" self.mock_git.tag_exists.return_value = False + self.mock_git.get_commit_sha.return_value = "12345678" initial_body = "- [ ] Tag Final" self.mock_gh.get_issue_body.return_value = initial_body # Act - result = PromoteRc(args, self.mock_git, self.mock_gh).run() + result = Promote(args, self.mock_git, self.mock_gh).run() # Assert self.assertEqual(result, 0) @@ -159,12 +157,10 @@ def test_promote_rc_resolves_version_from_issue(self): ) self.mock_git.get_current_branch.assert_not_called() self.mock_git.get_tags.assert_not_called() - self.mock_git.get_remote_tags.assert_called_once_with("my-remote") + self.mock_git.get_remote_tags.assert_not_called() self.mock_git.checkout.assert_not_called() - self.mock_git.get_commit_sha.assert_has_calls( - [call("2.0.1-rc0"), call("my-remote/release/2.0")] - ) + self.mock_git.get_commit_sha.assert_called_once_with("my-remote/release/2.0") self.mock_git.tag.assert_called_once_with("2.0.1", "12345678") self.mock_git.push.assert_called_once_with("my-remote", "2.0.1") @@ -180,7 +176,7 @@ def test_promote_rc_resolves_version_from_issue(self): "- [Github Release 2.0.1](https://github.com/bazel-contrib/rules_python/releases/tag/2.0.1)\n" "- [BCR Entry 2.0.1](https://registry.bazel.build/modules/rules_python/2.0.1)\n" "- [BCR PRs](https://github.com/bazelbuild/bazel-central-registry/pulls?q=is%3Apr%20%28%22bazel-contrib/rules_python%22%20in%3Atitle%29%20%28%22%402.0.1%22%20in%3Atitle%29)\n" - "- [Release workflow status](https://github.com/bazel-contrib/rules_python/actions/workflows/release_promote_rc.yaml)" + "- [Release workflow status](https://github.com/bazel-contrib/rules_python/actions/workflows/release_promote.yaml)" ) self.mock_gh.post_issue_comment.assert_called_once_with(123, expected_comment) @@ -197,7 +193,7 @@ def test_promote_rc_dry_run_success(self, mock_print): self.mock_gh.get_issue_body.return_value = initial_body # Act - result = PromoteRc(args, self.mock_git, self.mock_gh).run() + result = Promote(args, self.mock_git, self.mock_gh).run() # Assert self.assertEqual(result, 0) @@ -242,7 +238,7 @@ def test_promote_rc_tag_already_exists(self): self.mock_git.tag_exists.return_value = True # Act - result = PromoteRc(args, self.mock_git, self.mock_gh).run() + result = Promote(args, self.mock_git, self.mock_gh).run() # Assert self.assertEqual(result, 1) @@ -264,7 +260,7 @@ def test_promote_rc_issue_not_found(self): ) # Act - result = PromoteRc(args, self.mock_git, self.mock_gh).run() + result = Promote(args, self.mock_git, self.mock_gh).run() # Assert self.assertEqual(result, 1) @@ -286,7 +282,7 @@ def test_promote_rc_issue_malformed(self): self.mock_gh.get_issue_body.return_value = initial_body # Act - result = PromoteRc(args, self.mock_git, self.mock_gh).run() + result = Promote(args, self.mock_git, self.mock_gh).run() # Assert self.assertEqual(result, 1) @@ -304,7 +300,7 @@ def test_promote_rc_no_rc_found(self): self.mock_git.get_remote_tags.return_value = [] # Act - result = PromoteRc(args, self.mock_git, self.mock_gh).run() + result = Promote(args, self.mock_git, self.mock_gh).run() # Assert self.assertEqual(result, 1) diff --git a/tests/tools/private/release/release_test.py b/tests/tools/private/release/release_test.py index c7ba1b17b7..f0cdbdfe1d 100644 --- a/tests/tools/private/release/release_test.py +++ b/tests/tools/private/release/release_test.py @@ -7,9 +7,7 @@ class ReleaseCLITest(unittest.TestCase): def test_valid_version(self): # These should not raise an exception releaser.create_parser().parse_args(["prepare", "0.28.0"]) - releaser.create_parser().parse_args( - ["promote-rc", "1.0.0", "--remote", "origin"] - ) + releaser.create_parser().parse_args(["promote", "1.0.0", "--remote", "origin"]) releaser.create_parser().parse_args( ["create-release-issue", "--version", "1.2.3rc4"] ) diff --git a/tests/tools/private/release/release_test_helper.py b/tests/tools/private/release/release_test_helper.py index c35cd0fa40..12945193ef 100644 --- a/tests/tools/private/release/release_test_helper.py +++ b/tests/tools/private/release/release_test_helper.py @@ -9,30 +9,36 @@ MultipleTrackingIssuesError, NoTrackingIssueError, ) +from tools.private.release.mock_gh import MockGitHub -def _mock_git_and_gh(test_case): +def _mock_git(test_case): mock_git = MagicMock() - mock_gh = MagicMock() test_case.mock_git = mock_git - test_case.mock_gh = mock_gh # Mock Git inside utils.py since it instantiates it locally patch("tools.private.release.utils.Git", return_value=mock_git).start() - mock_gh.MultipleTrackingIssuesError = MultipleTrackingIssuesError - mock_gh.NoTrackingIssueError = NoTrackingIssueError - test_case.addCleanup(patch.stopall) # Apply safe defaults mock_git.get_current_branch.return_value = None mock_git.get_tags.return_value = [] mock_git.get_remote_tags.return_value = [] - mock_git.status.return_value = "" mock_git.branch_exists.return_value = False mock_git.tag_exists.return_value = False + return mock_git + + +def _mock_git_and_gh(test_case): + _mock_git(test_case) + mock_gh = MagicMock() + test_case.mock_gh = mock_gh + + mock_gh.MultipleTrackingIssuesError = MultipleTrackingIssuesError + mock_gh.NoTrackingIssueError = NoTrackingIssueError + mock_gh.get_release_tracking_issue.side_effect = NoTrackingIssueError("Not found") mock_gh.get_open_pr.return_value = None @@ -44,3 +50,27 @@ def setUp(self): self.addCleanup(shutil.rmtree, self.tmpdir) os.chdir(self.tmpdir) self.addCleanup(os.chdir, self.original_cwd) + + +DEFAULT_RELEASE_TEMPLATE_CONTENT = ( + "template content\n" + "- [ ] Prepare Release\n" + "- [ ] Tag RC0\n" + "- [ ] Tag Final\n" + "\n" + "## Backports\n" +) + + +class ReleaseToolTestCase(TempDirTestCase): + def setUp(self): + super().setUp() + self.gh = MockGitHub() + self.setUpReleaseTemplate() + + def setUpReleaseTemplate(self): + template_dir = self.tmpdir / ".github/ISSUE_TEMPLATE" + template_dir.mkdir(parents=True, exist_ok=True) + self.template_file = template_dir / "release_tracking_template.md" + self.template_content = DEFAULT_RELEASE_TEMPLATE_CONTENT + self.template_file.write_text(self.template_content, encoding="utf-8") diff --git a/tools/private/release/BUILD.bazel b/tools/private/release/BUILD.bazel index a63d2111f7..5f80d1130a 100644 --- a/tools/private/release/BUILD.bazel +++ b/tools/private/release/BUILD.bazel @@ -11,7 +11,10 @@ py_library( name = "release_lib", srcs = glob( ["*.py"], - exclude = ["changelog_news.py"], + exclude = [ + "changelog_news.py", + "mock_gh.py", + ], ), deps = [ ":changelog_news", @@ -19,6 +22,14 @@ py_library( ], ) +py_library( + name = "mock_gh", + srcs = ["mock_gh.py"], + deps = [ + ":release_lib", + ], +) + py_binary( name = "release", srcs = ["release.py"], diff --git a/tools/private/release/backport_create_releases.py b/tools/private/release/backport_create_releases.py new file mode 100644 index 0000000000..662ef96eb1 --- /dev/null +++ b/tools/private/release/backport_create_releases.py @@ -0,0 +1,237 @@ +"""Subcommand to initiate releases for verified backports.""" + +import argparse +import pathlib +import re +from dataclasses import dataclass + +from tools.private.release.backport_prepare import parse_backport_metadata +from tools.private.release.gh import GitHub +from tools.private.release.release_issue import ( + add_backports_to_body, + add_sync_changelog_task_to_body, + parse_metadata_line, + update_task_in_body, +) + + +@dataclass +class BackportTasksState: + # Map of minor version (e.g. "1.7") to verification status (e.g. "success", "failed-conflict") + verify: dict[str, str] + # Map of full version (e.g. "1.7.1") to tuple of (status, release_issue_number_or_None) + release: dict[str, tuple[str, str | None]] + + +def parse_backport_tasks(body) -> BackportTasksState: + """Parses tasks from backport issue body.""" + verify_tasks = {} + release_tasks = {} + + lines = body.splitlines() + for line in lines: + parsed = parse_metadata_line(line) + if not parsed: + continue + name = parsed["name"] + meta = parsed["metadata"] + + verify_match = re.match(r"Verify apply (\d+\.\d+)", name, re.IGNORECASE) + if verify_match: + minor = verify_match.group(1) + verify_tasks[minor] = meta.get("status", "pending") + continue + + release_match = re.match(r"Track Release (\d+\.\d+\.\d+)", name, re.IGNORECASE) + if release_match: + version = release_match.group(1) + status = meta.get("status", "pending") + issue_num = meta.get("release_issue") + release_tasks[version] = (status, issue_num) + + return BackportTasksState(verify=verify_tasks, release=release_tasks) + + +def is_release_eligible(version, target_minors, verify_statuses): + """Checks if a release version is eligible based on verification statuses.""" + version_minor = ".".join(version.split(".")[:2]) + v_minor_parsed = [int(x) for x in version_minor.split(".")] + + for minor in target_minors: + minor_parsed = [int(x) for x in minor.split(".")] + if minor_parsed >= v_minor_parsed: + status = verify_statuses.get(minor) + if status != "success": + return ( + False, + f"Blocked by verification failure on {minor} (status: {status})", + ) + + return True, "Eligible" + + +def _load_release_template() -> str: + """Loads the release tracking issue template.""" + template_path = pathlib.Path(".github/ISSUE_TEMPLATE/release_tracking_template.md") + if not template_path.exists(): + raise FileNotFoundError(f"Template file not found at {template_path}") + return template_path.read_text(encoding="utf-8") + + +class BackportCreateReleases: + """Class to initiate releases for verified backports.""" + + def __init__(self, args, gh: GitHub): + self._args = args + self._gh = gh + + def run(self) -> int: + """Executes the backport-create-releases subcommand.""" + args = self._args + issue_num = args.issue + if not issue_num: + raise ValueError("--issue is required.") + + print(f"Reading backport issue #{issue_num}...") + body = self._gh.get_issue_body(issue_num) + try: + metadata = parse_backport_metadata(body) + except ValueError as e: + e.add_note(f"Failed to parse backport metadata from issue #{issue_num}") + raise + pr_ref = metadata.pr + + pr_num = self._gh.resolve_pr_number(pr_ref) + + # Parse tasks + tasks_state = parse_backport_tasks(body) + verify_statuses = tasks_state.verify + release_tasks = tasks_state.release + + target_minors = sorted( + list(verify_statuses.keys()), key=lambda m: [int(x) for x in m.split(".")] + ) + + # We need the templates for release issues + template_content = _load_release_template() + + updated_body = body + changes_made = False + + for version, (status, release_issue) in release_tasks.items(): + # If status is success, we already created the issue. + # We check the actual checklist item status, but the helper returns it. + # The status in metadata is 'success' when done. + if status == "success": + print(f"Release for {version} already initiated: {release_issue}") + continue + + eligible, reason = is_release_eligible( + version, target_minors, verify_statuses + ) + + task_name = f"Track Release {version}" + + if eligible: + print(f"Initiating release for {version}...") + + # Check if release issue already exists + existing_issues = self._gh.get_open_tracking_issues(version) + if existing_issues: + new_issue_num = existing_issues[0]["number"] + print( + f"Release issue for {version} already exists: #{new_issue_num}. Reusing it." + ) + else: + # Create the issue + is_first_release = version.endswith(".0") + if is_first_release: + issue_template = template_content + else: + lines = template_content.splitlines() + lines = [ + line for line in lines if not re.search(r"Tag RC\d+", line) + ] + issue_template = "\n".join(lines) + if template_content.endswith("\n"): + issue_template += "\n" + + if args.dry_run: + print( + f"[DRY RUN] Would create release tracking issue for {version} (without RC tasks)" + ) + new_issue_num = "" + else: + new_issue_num = self._gh.create_tracking_issue( + version, issue_template + ) + print( + f"Created release tracking issue #{new_issue_num} for {version}" + ) + + # Add the backport PR to the new release issue + print( + f"Adding PR #{pr_num} to release issue #{new_issue_num} checklist..." + ) + rel_body = self._gh.get_issue_body(new_issue_num) + rel_body = add_backports_to_body( + rel_body, [{"ref": f"#{pr_num}"}] + ) + rel_body = add_sync_changelog_task_to_body(rel_body, pr_num) + self._gh.update_issue_body(new_issue_num, rel_body) + + # Update task in backport issue + metadata = {"status": "success", "release_issue": f"#{new_issue_num}"} + updated_body = update_task_in_body( + updated_body, task_name, checked=True, metadata=metadata + ) + changes_made = True + else: + print(f"Release for {version} is not eligible: {reason}") + metadata = {"status": "error-later-release-did-not-apply"} + + if status != "error-later-release-did-not-apply": + updated_body = update_task_in_body( + updated_body, task_name, checked=False, metadata=metadata + ) + changes_made = True + + if changes_made: + if args.dry_run: + print( + f"[DRY RUN] Would update backport issue #{issue_num} body:\n{updated_body}" + ) + else: + self._gh.update_issue_body(issue_num, updated_body) + print(f"Updated backport issue #{issue_num}") + else: + print("No changes needed for backport issue.") + + return 0 + + @classmethod + def add_parser(cls, subparsers): + """Adds parser for backport-create-releases subcommand.""" + parser = subparsers.add_parser( + "backport-create-releases", + help="Initiate releases for verified backports.", + ) + parser.add_argument( + "--issue", + type=int, + required=True, + help="The backport tracking issue number (required).", + ) + parser.add_argument( + "--dry-run", + action=argparse.BooleanOptionalAction, + default=True, + help="Perform a dry run (default: True). Use --no-dry-run to actually execute.", + ) + parser.set_defaults(command=cls.run_from_args) + + @classmethod + def run_from_args(cls, args): + """Instantiates and runs the command from parsed args.""" + gh = GitHub() + return cls(args, gh).run() diff --git a/tools/private/release/backport_prepare.py b/tools/private/release/backport_prepare.py new file mode 100644 index 0000000000..459e91f703 --- /dev/null +++ b/tools/private/release/backport_prepare.py @@ -0,0 +1,305 @@ +"""Subcommand to prepare backport tracking issue and verify cherry-picks.""" + +import argparse +import datetime +import re +from dataclasses import dataclass + +from tools.private.release import changelog_news +from tools.private.release.gh import BACKPORT_LABEL, GitHub +from tools.private.release.git import Git +from tools.private.release.utils import determine_next_version + + +@dataclass +class BackportMetadata: + pr: str + from_minor: str + to_minor: str + + +def parse_backport_metadata(body) -> BackportMetadata: + """Parses backport metadata from issue body.""" + pr_match = re.search(r"^\s*\*\s*PR:\s*(\S+)", body, re.MULTILINE | re.IGNORECASE) + from_match = re.search( + r"^\s*\*\s*From version:\s*(\S+)", body, re.MULTILINE | re.IGNORECASE + ) + to_match = re.search( + r"^\s*\*\s*To version:\s*(\S+)", body, re.MULTILINE | re.IGNORECASE + ) + + if not pr_match or not from_match or not to_match: + raise ValueError( + "Missing metadata in issue body. Need PR, From version, and To version." + ) + + return BackportMetadata( + pr=pr_match.group(1), + from_minor=from_match.group(1), + to_minor=to_match.group(1), + ) + + +def get_target_branches(git, remote, from_minor, to_minor): + """Identifies target release branches in the given range.""" + branches = git.get_remote_branches(remote) + target_branches = [] + + # Parse version strings to compare (only major and minor) + from_v = [int(x) for x in from_minor.split(".")][:2] + to_v = [int(x) for x in to_minor.split(".")][:2] + + for branch in branches: + match = re.match(r"^release/(\d+)\.(\d+)$", branch) + if match: + major = int(match.group(1)) + minor = int(match.group(2)) + v = [major, minor] + if v >= from_v and v <= to_v: + target_branches.append(branch) + + # Sort branches by version + target_branches.sort(key=lambda b: [int(x) for x in b.split("/")[1].split(".")]) + return target_branches + + +def get_latest_release_branch(git, remote): + """Determines the latest release branch.""" + branches = git.get_remote_branches(remote) + release_branches = [] + for branch in branches: + match = re.match(r"^release/(\d+)\.(\d+)$", branch) + if match: + release_branches.append(branch) + if not release_branches: + return None + release_branches.sort(key=lambda b: [int(x) for x in b.split("/")[1].split(".")]) + return release_branches[-1] + + +class BackportPrepare: + """Class to prepare backport tracking issue and verify cherry-picks.""" + + def __init__(self, args, git: Git, gh: GitHub): + self.args = args + self.git = git + self.gh = gh + + def run(self) -> int: + """Executes the backport-prepare subcommand.""" + args = self.args + print("Fetching remote to verify release branches...") + self.git.fetch(args.remote, tags=True, force=True) + + issue_num = args.issue + + if issue_num: + # Triggered by comment, read from issue + print(f"Reading metadata from issue #{issue_num}...") + body = self.gh.get_issue_body(issue_num) + try: + metadata = parse_backport_metadata(body) + except ValueError as e: + e.add_note(f"Failed to parse backport metadata from issue #{issue_num}") + raise + pr_ref = metadata.pr + from_minor = metadata.from_minor + to_minor = metadata.to_minor + else: + # Triggered manually, use args + pr_ref = args.pr + from_minor = args.from_minor + to_minor = args.to_minor + + if not pr_ref or not from_minor: + raise ValueError( + "PR and From version are required if not running from an issue." + ) + + if not to_minor: + latest_branch = get_latest_release_branch(self.git, args.remote) + if not latest_branch: + raise RuntimeError( + "Could not determine latest release branch and --to-minor was not provided." + ) + to_minor = latest_branch.split("/")[1] + print(f"Auto-determined latest minor version: {to_minor}") + + # Resolve PR to merge commit + try: + pr_num = self.gh.resolve_pr_number(pr_ref) + print(f"Resolving PR #{pr_num} info...") + pr_info = self.gh.get_pr_info(pr_num) + except Exception as e: + e.add_note(f"Failed to resolve PR info for {pr_ref}") + raise + + if pr_info.get("state") != "MERGED": + raise ValueError( + f"PR #{pr_num} is not merged (state: {pr_info.get('state')})." + ) + merge_commit = pr_info.get("mergeCommit") + if not merge_commit or "oid" not in merge_commit: + raise ValueError(f"PR #{pr_num} has no merge commit SHA.") + pr_sha = merge_commit["oid"] + print(f"Resolved PR #{pr_num} to merge commit {pr_sha[:8]}") + + target_branches = get_target_branches( + self.git, args.remote, from_minor, to_minor + ) + if not target_branches: + raise ValueError( + f"No release branches found in range {from_minor} to {to_minor}" + ) + + print(f"Identified target branches: {target_branches}") + + # Verify workspace is clean + if self.git.status(): + raise RuntimeError("Workspace is dirty. Aborting.") + + current_branch = self.git.get_current_branch() + verify_results = {} # branch -> (success, reason) + version_map = {} # branch -> next_version + + try: + for branch in target_branches: + minor_ver = branch.split("/")[1] + print(f"Verifying application on {branch}...") + self.git.checkout(branch, track_remote=args.remote) + + # Determine next version + next_version = determine_next_version(branch) + version_map[branch] = next_version + + # Verify apply (cherry-pick + news) + try: + self.git.cherry_pick(pr_sha) + + # Try news update + try: + release_date = datetime.date.today().strftime("%Y-%m-%d") + changelog_news.update_changelog(next_version, release_date) + verify_results[branch] = (True, "Success") + print(f"Verification successful for {branch}") + except Exception as e: + verify_results[branch] = ( + False, + f"Changelog update failed: {e}", + ) + print( + f"Verification failed for {branch} (changelog update): {e}" + ) + except Exception as e: + verify_results[branch] = (False, f"Cherry-pick failed: {e}") + print(f"Verification failed for {branch} (cherry-pick): {e}") + finally: + # Always abort/reset + try: + self.git.cherry_pick_abort() + except Exception: + pass + self.git.reset_hard(reset_to=f"{args.remote}/{branch}") + finally: + # Restore original branch + if current_branch: + self.git.checkout(current_branch) + + # Generate issue content + body_lines = [ + f"* PR: #{pr_num}", + f"* From version: {from_minor}", + f"* To version: {to_minor}", + "", + "## Tasks", + "", + ] + + # Add Verify tasks + for branch in target_branches: + minor_ver = branch.split("/")[1] + success, reason = verify_results[branch] + if success: + body_lines.append(f"- [x] Verify apply {minor_ver} | status=success") + else: + status = "failed-conflict" + if "Changelog" in reason: + status = "failed-changelog" + body_lines.append(f"- [ ] Verify apply {minor_ver} | status={status}") + + # Add Release tasks + for branch in target_branches: + next_version = version_map[branch] + body_lines.append(f"- [ ] Track Release {next_version}") + + new_body = "\n".join(body_lines) + + if issue_num: + if args.dry_run: + print( + f"[DRY RUN] Would update issue #{issue_num} with body:\n{new_body}" + ) + else: + self.gh.update_issue_body(issue_num, new_body) + print(f"Successfully updated issue #{issue_num}") + else: + title = f"Backport: #{pr_num}" + if args.dry_run: + print( + f"[DRY RUN] Would create issue with title '{title}' and body:\n{new_body}" + ) + else: + new_issue_num = self.gh.create_issue( + title, new_body, labels=[BACKPORT_LABEL] + ) + print(f"Created backport tracking issue #{new_issue_num}") + + return 0 + + @classmethod + def add_parser(cls, subparsers): + """Adds parser for backport-prepare subcommand.""" + parser = subparsers.add_parser( + "backport-prepare", + help="Prepare backport tracking issue and verify cherry-picks.", + ) + parser.add_argument( + "--issue", + type=int, + help="The backport tracking issue number (if running from an existing issue).", + ) + parser.add_argument( + "--pr", + type=str, + help="PR reference to backport (required if not running from issue).", + ) + parser.add_argument( + "--from-minor", + type=str, + help="Oldest minor version to target (inclusive) (e.g. 1.7) (required if not running from issue).", + ) + parser.add_argument( + "--to-minor", + type=str, + help="Newest minor version to target (inclusive) (optional, defaults to latest).", + ) + parser.add_argument( + "--remote", + type=str, + default="origin", + help="The git remote (default: origin).", + ) + parser.add_argument( + "--dry-run", + action=argparse.BooleanOptionalAction, + default=True, + help="Perform a dry run (default: True). Use --no-dry-run to actually execute.", + ) + parser.set_defaults(command=cls.run_from_args) + + @classmethod + def run_from_args(cls, args): + """Instantiates and runs the command from parsed args.""" + git = Git(".") + gh = GitHub() + return cls(args, git, gh).run() diff --git a/tools/private/release/create_release_issue.py b/tools/private/release/create_release_issue.py index 2d00d7369a..1c28e86c7d 100644 --- a/tools/private/release/create_release_issue.py +++ b/tools/private/release/create_release_issue.py @@ -1,6 +1,7 @@ """Subcommand to create a release tracking issue.""" import pathlib +import re from tools.private.release.gh import GitHub from tools.private.release.utils import determine_next_version, semver_type @@ -34,6 +35,13 @@ def run(self) -> int: raise FileNotFoundError(f"Template file not found at {template_path}") template_content = template_path.read_text(encoding="utf-8") + is_first_release = version.endswith(".0") + if not is_first_release: + # Patch release: remove RC tasks + lines = template_content.splitlines() + lines = [line for line in lines if not re.search(r"Tag RC\d+", line)] + template_content = "\n".join(lines) + issue_num = self.gh.create_tracking_issue(version, template_content) print(f"Created tracking issue #{issue_num} for v{version}") return 0 diff --git a/tools/private/release/gh.py b/tools/private/release/gh.py index 92cc3ca5c4..27206f1ffe 100644 --- a/tools/private/release/gh.py +++ b/tools/private/release/gh.py @@ -8,6 +8,10 @@ from tools.private.release.release_issue import BackportTask from tools.private.release.shell import run_cmd +# GitHub label types +RELEASE_LABEL = "type: release" +BACKPORT_LABEL = "type: backport-pr" + # GitHub reaction types # See: https://docs.github.com/en/rest/reactions/reactions?apiVersion=2022-11-28#about-reactions GH_REACTION_THUMBS_UP = "+1" @@ -42,7 +46,6 @@ def __init__(self, repo: str = "bazel-contrib/rules_python"): repo: The GitHub repository to operate on. """ self.repo = repo - self.label = "type: release" def _run_gh( self, *args: str, check: bool = True, capture_output: bool = True @@ -125,7 +128,7 @@ def get_open_tracking_issues(self, version: str | None = None) -> list[dict]: """ search = f'"Release {version}" in:title' if version else None return self.list_issues( - label=self.label, + label=RELEASE_LABEL, state="open", search=search, fields="number,title,url", @@ -158,13 +161,13 @@ def get_release_tracking_issue(self, version: str) -> int: if not exact_matches: raise NoTrackingIssueError( f"No open tracking issue found matching 'Release {version}' " - f"in repo {self.repo} with label '{self.label}'" + f"in repo {self.repo} with label '{RELEASE_LABEL}'" ) if len(exact_matches) > 1: urls = [issue["url"] for issue in exact_matches] raise MultipleTrackingIssuesError( f"Multiple open tracking issues found for version {version} " - f"in repo {self.repo} with label '{self.label}':\n" + "\n".join(urls) + f"in repo {self.repo} with label '{RELEASE_LABEL}':\n" + "\n".join(urls) ) return exact_matches[0]["number"] @@ -188,16 +191,15 @@ def create_tracking_issue(self, version: str, template_content: str) -> int: if len(parts) >= 3: issue_body = parts[2].strip() - # Write body to a secure temporary file to pass to the CLI - with tempfile.NamedTemporaryFile(mode="w", suffix=".md", delete=False) as f: + with tempfile.NamedTemporaryFile(mode="w", suffix=".md") as f: f.write(issue_body) + f.flush() temp_path = f.name - try: output = self._gh_issue( "create", f"--title=Release {version}", - f"--label={self.label}", + f"--label={RELEASE_LABEL}", f"--body-file={temp_path}", ) if not output: @@ -205,9 +207,40 @@ def create_tracking_issue(self, version: str, template_content: str) -> int: issue_url = output.strip() issue_num = int(issue_url.split("/")[-1]) return issue_num - finally: - if os.path.exists(temp_path): - os.unlink(temp_path) + + def create_issue( + self, title: str, body: str, labels: list[str] | None = None + ) -> int: + """Creates a generic issue. + + Args: + title: The title of the issue. + body: The body of the issue. + labels: Optional list of labels to add. + + Returns: + The created issue number. + """ + with tempfile.NamedTemporaryFile(mode="w", suffix=".md") as f: + f.write(body) + f.flush() + temp_path = f.name + + cmd = [ + "create", + f"--title={title}", + f"--body-file={temp_path}", + ] + if labels: + for label in labels: + cmd.append(f"--label={label}") + + output = self._gh_issue(*cmd) + if not output: + raise RuntimeError("Failed to get issue URL from gh issue create") + issue_url = output.strip() + issue_num = int(issue_url.split("/")[-1]) + return issue_num def get_issue_body(self, issue_num: int) -> str: """Fetches the body of a specific issue. diff --git a/tools/private/release/git.py b/tools/private/release/git.py index 4305de855f..3e7e3cb893 100644 --- a/tools/private/release/git.py +++ b/tools/private/release/git.py @@ -397,3 +397,25 @@ def apply_check(self, patch_file: str) -> bool: return True except subprocess.CalledProcessError: return False + + def get_remote_branches(self, remote: str = "origin") -> list[str]: + """Returns a list of remote branches. + + Args: + remote: The name of the remote. + + Returns: + A list of branch names (without the remote prefix). + """ + output = self._run_git("branch", "-r") + branches = [] + if not output: + return branches + for line in output.splitlines(): + line = line.strip() + if "->" in line: + continue + parts = line.split("/") + if len(parts) >= 2 and parts[0] == remote: + branches.append("/".join(parts[1:])) + return branches diff --git a/tools/private/release/mock_gh.py b/tools/private/release/mock_gh.py new file mode 100644 index 0000000000..3a5a8dde48 --- /dev/null +++ b/tools/private/release/mock_gh.py @@ -0,0 +1,84 @@ +"""In-memory fake for GitHub API.""" + +import re + +from tools.private.release.gh import RELEASE_LABEL + + +class MockGitHub: + def __init__(self, repo: str = "bazel-contrib/rules_python"): + self.repo = repo + self.issues = {} + self.next_issue_num = 1001 + self.prs = {} # num -> pr_info + + def create_issue( + self, title: str, body: str, labels: list[str] | None = None + ) -> int: + issue_num = self.next_issue_num + self.next_issue_num += 1 + self.issues[issue_num] = { + "title": title, + "body": body, + "labels": labels or [], + "number": issue_num, + "url": f"https://github.com/{self.repo}/issues/{issue_num}", + } + return issue_num + + def create_tracking_issue(self, version: str, template_content: str) -> int: + # Strip YAML frontmatter if present (simplified copy from gh.py) + issue_body = template_content + if template_content.startswith("---"): + parts = template_content.split("---", 2) + if len(parts) >= 3: + issue_body = parts[2].strip() + + return self.create_issue( + title=f"Release {version}", body=issue_body, labels=[RELEASE_LABEL] + ) + + def get_issue_body(self, issue_num: int) -> str: + if issue_num not in self.issues: + raise ValueError(f"Issue #{issue_num} not found in MockGitHub") + return self.issues[issue_num]["body"] + + def update_issue_body(self, issue_num: int, body: str): + if issue_num not in self.issues: + raise ValueError(f"Issue #{issue_num} not found in MockGitHub") + self.issues[issue_num]["body"] = body + + def resolve_pr_number(self, pr_ref: str) -> int: + # Real algorithm copy (doesn't require RPCs) + clean_ref = pr_ref.lstrip("#") + if clean_ref.isdigit(): + return int(clean_ref) + + if pr_ref.startswith("http"): + pattern = rf"github\.com/{re.escape(self.repo)}/pull/(\d+)(/|\?|\Z)" + match = re.search(pattern, pr_ref, re.IGNORECASE) + if match: + return int(match.group(1)) + raise ValueError( + f"URL is not for the configured repository ({self.repo}): {pr_ref}" + ) + raise ValueError(f"Could not resolve PR ref: {pr_ref}") + + def get_open_tracking_issues(self, version: str | None = None) -> list[dict]: + results = [] + for issue in self.issues.values(): + if RELEASE_LABEL in issue["labels"]: + if version: + if issue["title"] == f"Release {version}": + results.append(issue) + else: + results.append(issue) + return results + + def get_pr_info(self, pr_num: int) -> dict: + if pr_num in self.prs: + return self.prs[pr_num] + return { + "state": "MERGED", + "mergeCommit": {"oid": f"mock_merge_sha_{pr_num}"}, + } diff --git a/tools/private/release/promote_rc.py b/tools/private/release/promote.py similarity index 75% rename from tools/private/release/promote_rc.py rename to tools/private/release/promote.py index 34c8107af4..66d5b74324 100644 --- a/tools/private/release/promote_rc.py +++ b/tools/private/release/promote.py @@ -18,7 +18,7 @@ ) -class PromoteRc: +class Promote: """Class to promote a release candidate to final release.""" def __init__(self, args, git: Git, gh: GitHub): @@ -52,16 +52,13 @@ def run(self) -> int: else: version = determine_next_version() - latest_rc = get_latest_rc_tag(version, remote=args.remote) - if not latest_rc: - print(f"Error: No release candidate tags found matching {version}-rc*") - return 1 - # Verify final tag doesn't already exist if self.git.tag_exists(version): print(f"Error: Final tag {version} already exists.") return 1 + is_first_release = version.endswith(".0") + # Verify issue can be found issue_num = args.issue if not issue_num: @@ -74,14 +71,20 @@ def run(self) -> int: print(f"Error: Unexpected error finding tracking issue: {e}") return 1 - # Get commit SHA of the RC tag (which will be the same for the final tag) - commit_sha = self.git.get_commit_sha(latest_rc) + if is_first_release: + latest_rc = get_latest_rc_tag(version, remote=args.remote) + if not latest_rc: + print(f"Error: No release candidate tags found matching {version}-rc*") + return 1 + commit_sha = self.git.get_commit_sha(latest_rc) + else: + latest_rc = None # Verify issue can be found and read it early print(f"Verifying tracking issue #{issue_num} format...") body = self.gh.get_issue_body(issue_num) - # Determine release branch name and verify it matches the RC tag commit + # Determine release branch name branch_version = ".".join(version.split(".")[:2]) branch_name = f"release/{branch_version}" remote_branch = f"{args.remote}/{branch_name}" @@ -97,34 +100,39 @@ def run(self) -> int: ) return 1 - if commit_sha != branch_sha: - print( - f"Error: The latest RC tag {latest_rc} ({commit_sha[:8]}) is not at" - f" the head of release branch {remote_branch} ({branch_sha[:8]})." - ) - metadata = { - "status": "error-rc-tag-not-branch-head", - "rc": latest_rc, - "branch_commit": branch_sha[:8], - "tag_commit": commit_sha[:8], - } - try: - updated_body = update_task_in_body( - body, "Tag Final", checked=False, metadata=metadata - ) - except ValueError as e: - print(f"Error: Tracking issue #{issue_num} is malformed: {e}") - return 1 - - if not args.dry_run: - self.gh.update_issue_body(issue_num, updated_body) - print(f"Updated tracking issue #{issue_num} with error status.") - else: + if is_first_release: + if commit_sha != branch_sha: print( - f"[DRY RUN] Would update tracking issue #{issue_num} with" - f" error status." + f"Error: The latest RC tag {latest_rc} ({commit_sha[:8]}) is not at" + f" the head of release branch {remote_branch} ({branch_sha[:8]})." ) - return 1 + metadata = { + "status": "error-rc-tag-not-branch-head", + "rc": latest_rc, + "branch_commit": branch_sha[:8], + "tag_commit": commit_sha[:8], + } + try: + updated_body = update_task_in_body( + body, "Tag Final", checked=False, metadata=metadata + ) + except ValueError as e: + print(f"Error: Tracking issue #{issue_num} is malformed: {e}") + return 1 + + if not args.dry_run: + self.gh.update_issue_body(issue_num, updated_body) + print(f"Updated tracking issue #{issue_num} with error status.") + else: + print( + f"[DRY RUN] Would update tracking issue #{issue_num} with" + f" error status." + ) + return 1 + else: + # Patch release: tag branch head directly + commit_sha = branch_sha + latest_rc = None # Verify issue is in the right format by trying to prepare the update (for success case) metadata = {"status": "done", "tag": version, "commit": commit_sha[:8]} @@ -137,10 +145,15 @@ def run(self) -> int: return 1 # All pre-conditions met, perform modifications + promote_source = ( + latest_rc + if is_first_release + else f"head of {branch_name} ({commit_sha[:8]})" + ) if args.dry_run: print( f"[DRY RUN] Pre-conditions passed successfully for promoting" - f" {latest_rc} to {version}." + f" {promote_source} to {version}." ) print(f"[DRY RUN] Would tag commit {commit_sha[:8]} as {version}") print(f"[DRY RUN] Would push tag {version} to {args.remote}") @@ -149,7 +162,7 @@ def run(self) -> int: return 0 print( - f"Promoting {latest_rc} to final release {version} (commit" + f"Promoting {promote_source} to final release {version} (commit" f" {commit_sha[:8]}) using tracking issue #{issue_num}..." ) @@ -177,9 +190,7 @@ def run(self) -> int: if run_id := os.environ.get("GITHUB_RUN_ID"): release_workflow_url = f"{REPO_URL}/actions/runs/{run_id}" else: - release_workflow_url = ( - f"{REPO_URL}/actions/workflows/release_promote_rc.yaml" - ) + release_workflow_url = f"{REPO_URL}/actions/workflows/release_promote.yaml" comment_body = f"""**New Release Tagged!** 🐍🌿 @@ -197,7 +208,7 @@ def run(self) -> int: def add_parser(cls, subparsers): """Adds parser for promote-rc subcommand.""" parser = subparsers.add_parser( - "promote-rc", + "promote", help="Promote the latest RC to final release.", ) parser.add_argument( diff --git a/tools/private/release/release.py b/tools/private/release/release.py index f008c093b1..72c427a392 100644 --- a/tools/private/release/release.py +++ b/tools/private/release/release.py @@ -5,6 +5,8 @@ import sys from tools.private.release.add_backports import AddBackports +from tools.private.release.backport_create_releases import BackportCreateReleases +from tools.private.release.backport_prepare import BackportPrepare from tools.private.release.complete_prepare import CompletePrepare from tools.private.release.complete_sync_changelog import CompleteSyncChangelog from tools.private.release.create_rc import CreateRc @@ -14,7 +16,7 @@ from tools.private.release.on_pr_merged import OnPrMerged from tools.private.release.prepare import Prepare from tools.private.release.process_backports import ProcessBackports -from tools.private.release.promote_rc import PromoteRc +from tools.private.release.promote import Promote cmds = [ DetermineNextVersion, @@ -27,7 +29,9 @@ ProcessBackports, OnPrMerged, CreateRc, - PromoteRc, + Promote, + BackportPrepare, + BackportCreateReleases, ] From 5fd7d8f17773c906b62fa6027823cb265eff6c25 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Sat, 18 Jul 2026 10:25:00 -0700 Subject: [PATCH 841/922] tests: implement pytest_test rule for internal test usage (#3931) This implements a pytest_test rule based upon the pytest_bazel library. The core implementation of this is fairly simple: a file is generated that calls `pytest.main()` with the `srcs` to test. This generated file is the file that is run by `py_test`. For now, this is kept under tests/support soas to use it for our own tests while some design decisions are figured out. Work towards https://github.com/bazel-contrib/rules_python/issues/3594 --- .bazelversion | 2 +- AGENTS.md | 1 - BUILD.bazel | 1 + MODULE.bazel | 2 +- docs/BUILD.bazel | 1 + docs/pyproject.toml | 2 + docs/requirements.txt | 278 +++++++++--------- docs/uv.lock | 112 +++++++ tests/pytest_test/BUILD.bazel | 22 ++ tests/pytest_test/basic_test.py | 2 + tests/support/pytest_test/BUILD.bazel | 30 ++ .../pytest_test/pytest_bootstrap_template.py | 8 + tests/support/pytest_test/pytest_test.bzl | 74 +++++ 13 files changed, 394 insertions(+), 141 deletions(-) create mode 100644 tests/pytest_test/BUILD.bazel create mode 100644 tests/pytest_test/basic_test.py create mode 100644 tests/support/pytest_test/BUILD.bazel create mode 100644 tests/support/pytest_test/pytest_bootstrap_template.py create mode 100644 tests/support/pytest_test/pytest_test.bzl diff --git a/.bazelversion b/.bazelversion index 512e4c889e..44931da266 100644 --- a/.bazelversion +++ b/.bazelversion @@ -1 +1 @@ -9.x +9.1.1 diff --git a/AGENTS.md b/AGENTS.md index e6e1733c1d..6f6e6fe76a 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -121,7 +121,6 @@ The `repository_ctx` API docs are at: https://bazel.build/rules/lib/builtins/rep e.g. given `load("//foo:bar.bzl", ...)`, the target is `//foo:bar_bzl`. * For files outside rules_python: remove the `.bzl` suffix. e.g. given `load("@foo//foo:bar.bzl", ...)`, the target is `@foo//foo:bar`. -* `bzl_library()` targets should be kept in alphabetical order by name. Example: diff --git a/BUILD.bazel b/BUILD.bazel index a25863708c..f978126da7 100644 --- a/BUILD.bazel +++ b/BUILD.bazel @@ -38,6 +38,7 @@ load("@bazel_skylib//:bzl_library.bzl", "bzl_library") # Exclude directories that are separate packages/workspaces or only for testing/examples # gazelle:exclude tests # gazelle:exclude examples +# gazelle:exclude gazelle # Exclude internal development tools and dependencies that users don't need # gazelle:exclude internal_dev_setup.bzl diff --git a/MODULE.bazel b/MODULE.bazel index c4afa60640..e4c4e08e89 100644 --- a/MODULE.bazel +++ b/MODULE.bazel @@ -214,7 +214,7 @@ dev_pip.parse( python_version = "3.11", requirements_lock = "//tests/multi_pypi/beta:requirements.txt", ) -use_repo(dev_pip, "dev_pip", "pypi_alpha", "pypi_beta", "pypiserver") +use_repo(dev_pip, "dev_pip", "pypi", "pypi_alpha", "pypi_beta", "pypiserver") # Bazel integration test setup below diff --git a/docs/BUILD.bazel b/docs/BUILD.bazel index 2e13da1296..9a9327aadf 100644 --- a/docs/BUILD.bazel +++ b/docs/BUILD.bazel @@ -198,6 +198,7 @@ sphinx_build_binary( target_compatible_with = _TARGET_COMPATIBLE_WITH, deps = [ "@dev_pip//myst_parser", + "@dev_pip//pytest", "@dev_pip//readthedocs_sphinx_ext", "@dev_pip//sphinx", "@dev_pip//sphinx_autodoc2", diff --git a/docs/pyproject.toml b/docs/pyproject.toml index f0ca3928ff..ea8a544241 100644 --- a/docs/pyproject.toml +++ b/docs/pyproject.toml @@ -17,4 +17,6 @@ dependencies = [ "pyelftools", "macholib", "markupsafe", + "pytest", + "pytest-bazel", ] diff --git a/docs/requirements.txt b/docs/requirements.txt index 24782bac8f..a0224b9283 100644 --- a/docs/requirements.txt +++ b/docs/requirements.txt @@ -6,9 +6,9 @@ absl-py==2.3.1 ; python_full_version < '3.10' \ --hash=sha256:a97820526f7fbfd2ec1bce83f3f25e3a14840dac0d8e02a0b71cd75db3f77fc9 \ --hash=sha256:eeecf07f0c2a93ace0772c92e596ace6d3d3996c042b2128459aaae2a76de11d # via rules-python-docs (docs/pyproject.toml) -absl-py==2.4.0 ; python_full_version >= '3.10' \ - --hash=sha256:88476fd881ca8aab94ffa78b7b6c632a782ab3ba1cd19c9bd423abc4fb4cd28d \ - --hash=sha256:8c6af82722b35cf71e0f4d1d47dcaebfff286e27110a99fc359349b247dfb5d4 +absl-py==2.5.0 ; python_full_version >= '3.10' \ + --hash=sha256:0c996f25c0490700fadabe6351630f6111534fa0ae252cc6d2014ea3b141135f \ + --hash=sha256:0f17b89f2a4eaaedc4f28c622998aa690564b3012a396a4ffad0821007fe03ba # via rules-python-docs (docs/pyproject.toml) alabaster==0.7.16 ; python_full_version < '3.10' \ --hash=sha256:75a8b99c28a5dad50dd7f8ccdd447a121ddb3892da9e53d1ca5cca3106d58d65 \ @@ -34,141 +34,107 @@ certifi==2026.6.17 \ --hash=sha256:024c88eeec92ca068db80f02b8b07c9cef7b9fe261d1d535abfd5abd6f6af432 \ --hash=sha256:2227dcbaafe0d2f59279d1762ddddc37783ed4354594f194ffc31d20f41fc3db # via requests -charset-normalizer==3.4.7 \ - --hash=sha256:007d05ec7321d12a40227aae9e2bc6dca73f3cb21058999a1df9e193555a9dcc \ - --hash=sha256:03853ed82eeebbce3c2abfdbc98c96dc205f32a79627688ac9a27370ea61a49c \ - --hash=sha256:07d9e39b01743c3717745f4c530a6349eadbfa043c7577eef86c502c15df2c67 \ - --hash=sha256:08e721811161356f97b4059a9ba7bafb23ea5ee2255402c42881c214e173c6b4 \ - --hash=sha256:0c96c3b819b5c3e9e165495db84d41914d6894d55181d2d108cc1a69bfc9cce0 \ - --hash=sha256:0ea948db76d31190bf08bd371623927ee1339d5f2a0b4b1b4a4439a65298703c \ - --hash=sha256:0f7eb884681e3938906ed0434f20c63046eacd0111c4ba96f27b76084cd679f5 \ - --hash=sha256:12a6fff75f6bc66711b73a2f0addfc4c8c15a20e805146a02d147a318962c444 \ - --hash=sha256:12d8baf840cc7889b37c7c770f478adea7adce3dcb3944d02ec87508e2dcf153 \ - --hash=sha256:14265bfe1f09498b9d8ec91e9ec9fa52775edf90fcbde092b25f4a33d444fea9 \ - --hash=sha256:16d971e29578a5e97d7117866d15889a4a07befe0e87e703ed63cd90cb348c01 \ - --hash=sha256:177a0ba5f0211d488e295aaf82707237e331c24788d8d76c96c5a41594723217 \ - --hash=sha256:1a87ca9d5df6fe460483d9a5bbf2b18f620cbed41b432e2bddb686228282d10b \ - --hash=sha256:1c2a768fdd44ee4a9339a9b0b130049139b8ce3c01d2ce09f67f5a68048d477c \ - --hash=sha256:1c2aed2e5e41f24ea8ef1590b8e848a79b56f3a5564a65ceec43c9d692dc7d8a \ - --hash=sha256:1dc8b0ea451d6e69735094606991f32867807881400f808a106ee1d963c46a83 \ - --hash=sha256:1efde3cae86c8c273f1eb3b287be7d8499420cf2fe7585c41d370d3e790054a5 \ - --hash=sha256:202389074300232baeb53ae2569a60901f7efadd4245cf3a3bf0617d60b439d7 \ - --hash=sha256:203104ed3e428044fd943bc4bf45fa73c0730391f9621e37fe39ecf477b128cb \ - --hash=sha256:2257141f39fe65a3fdf38aeccae4b953e5f3b3324f4ff0daf9f15b8518666a2c \ - --hash=sha256:298930cec56029e05497a76988377cbd7457ba864beeea92ad7e844fe74cd1f1 \ - --hash=sha256:2cd4a60d0e2fb04537162c62bbbb4182f53541fe0ede35cdf270a1c1e723cc42 \ - --hash=sha256:2d6eb928e13016cea4f1f21d1e10c1cebd5a421bc57ddf5b1142ae3f86824fab \ - --hash=sha256:2fe249cb4651fd12605b7288b24751d8bfd46d35f12a20b1ba33dea122e690df \ - --hash=sha256:30b8d1d8c52a48c2c5690e152c169b673487a2a58de1ec7393196753063fcd5e \ - --hash=sha256:320ade88cfb846b8cd6b4ddf5ee9e80ee0c1f52401f2456b84ae1ae6a1a5f207 \ - --hash=sha256:3534e7dcbdcf757da6b85a0bbf5b6868786d5982dd959b065e65481644817a18 \ - --hash=sha256:36836d6ff945a00b88ba1e4572d721e60b5b8c98c155d465f56ad19d68f23734 \ - --hash=sha256:38c0109396c4cfc574d502df99742a45c72c08eff0a36158b6f04000043dbf38 \ - --hash=sha256:3946fa46a0cf3e4c8cb1cc52f56bb536310d34f25f01ca9b6c16afa767dab110 \ - --hash=sha256:3bec022aec2c514d9cf199522a802bd007cd588ab17ab2525f20f9c34d067c18 \ - --hash=sha256:3c9a494bc5ec77d43cea229c4f6db1e4d8fe7e1bbffa8b6f0f0032430ff8ab44 \ - --hash=sha256:3dce51d0f5e7951f8bb4900c257dad282f49190fdbebecd4ba99bcc41fef404d \ - --hash=sha256:3dedcc22d73ec993f42055eff4fcfed9318d1eeb9a6606c55892a26964964e48 \ - --hash=sha256:4042d5c8f957e15221d423ba781e85d553722fc4113f523f2feb7b188cc34c5e \ - --hash=sha256:481551899c856c704d58119b5025793fa6730adda3571971af568f66d2424bb5 \ - --hash=sha256:4dc1e73c36828f982bfe79fadf5919923f8a6f4df2860804db9a98c48824ce8d \ - --hash=sha256:4e5163c14bffd570ef2affbfdd77bba66383890797df43dc8b4cc7d6f500bf53 \ - --hash=sha256:511ef87c8aec0783e08ac18565a16d435372bc1ac25a91e6ac7f5ef2b0bff790 \ - --hash=sha256:532bc9bf33a68613fd7d65e4b1c71a6a38d7d42604ecf239c77392e9b4e8998c \ - --hash=sha256:54523e136b8948060c0fa0bc7b1b50c32c186f2fceee897a495406bb6e311d2b \ - --hash=sha256:5649fd1c7bade02f320a462fdefd0b4bd3ce036065836d4f42e0de958038e116 \ - --hash=sha256:56be790f86bfb2c98fb742ce566dfb4816e5a83384616ab59c49e0604d49c51d \ - --hash=sha256:5b77459df20e08151cd6f8b9ef8ef1f961ef73d85c21a555c7eed5b79410ec10 \ - --hash=sha256:5ed6ab538499c8644b8a3e18debabcd7ce684f3fa91cf867521a7a0279cab2d6 \ - --hash=sha256:6178f72c5508bfc5fd446a5905e698c6212932f25bcdd4b47a757a50605a90e2 \ - --hash=sha256:6370e8686f662e6a3941ee48ed4742317cafbe5707e36406e9df792cdb535776 \ - --hash=sha256:64f02c6841d7d83f832cd97ccf8eb8a906d06eb95d5276069175c696b024b60a \ - --hash=sha256:65bcd23054beab4d166035cabbc868a09c1a49d1efe458fe8e4361215df40265 \ - --hash=sha256:66671f93accb62ed07da56613636f3641f1a12c13046ce91ffc923721f23c008 \ - --hash=sha256:6696b7688f54f5af4462118f0bfa7c1621eeb87154f77fa04b9295ce7a8f2943 \ - --hash=sha256:6785f414ae0f3c733c437e0f3929197934f526d19dfaa75e18fdb4f94c6fb374 \ - --hash=sha256:67f6279d125ca0046a7fd386d01b311c6363844deac3e5b069b514ba3e63c246 \ - --hash=sha256:6c114670c45346afedc0d947faf3c7f701051d2518b943679c8ff88befe14f8e \ - --hash=sha256:6e0d51f618228538a3e8f46bd246f87a6cd030565e015803691603f55e12afb5 \ - --hash=sha256:6ed74185b2db44f41ef35fd1617c5888e59792da9bbc9190d6c7300617182616 \ - --hash=sha256:708838739abf24b2ceb208d0e22403dd018faeef86ddac04319a62ae884c4f15 \ - --hash=sha256:715479b9a2802ecac752a3b0efa2b0b60285cf962ee38414211abdfccc233b41 \ - --hash=sha256:733784b6d6def852c814bce5f318d25da2ee65dd4839a0718641c696e09a2960 \ - --hash=sha256:750e02e074872a3fad7f233b47734166440af3cdea0add3e95163110816d6752 \ - --hash=sha256:752a45dc4a6934060b3b0dab47e04edc3326575f82be64bc4fc293914566503e \ - --hash=sha256:7579e913a5339fb8fa133f6bbcfd8e6749696206cf05acdbdca71a1b436d8e72 \ - --hash=sha256:7641bb8895e77f921102f72833904dcd9901df5d6d72a2ab8f31d04b7e51e4e7 \ - --hash=sha256:7804338df6fcc08105c7745f1502ba68d900f45fd770d5bdd5288ddccb8a42d8 \ - --hash=sha256:80d04837f55fc81da168b98de4f4b797ef007fc8a79ab71c6ec9bc4dd662b15b \ - --hash=sha256:813c0e0132266c08eb87469a642cb30aaff57c5f426255419572aaeceeaa7bf4 \ - --hash=sha256:82b271f5137d07749f7bf32f70b17ab6eaabedd297e75dce75081a24f76eb545 \ - --hash=sha256:84c018e49c3bf790f9c2771c45e9313a08c2c2a6342b162cd650258b57817706 \ - --hash=sha256:8751d2787c9131302398b11e6c8068053dcb55d5a8964e114b6e196cf16cb366 \ - --hash=sha256:8778f0c7a52e56f75d12dae53ae320fae900a8b9b4164b981b9c5ce059cd1fcb \ - --hash=sha256:87fad7d9ba98c86bcb41b2dc8dbb326619be2562af1f8ff50776a39e55721c5a \ - --hash=sha256:8d828b6667a32a728a1ad1d93957cdf37489c57b97ae6c4de2860fa749b8fc1e \ - --hash=sha256:8e385e4267ab76874ae30db04c627faaaf0b509e1ccc11a95b3fc3e83f855c00 \ - --hash=sha256:92a0a01ead5e668468e952e4238cccd7c537364eb7d851ab144ab6627dbbe12f \ - --hash=sha256:94e1885b270625a9a828c9793b4d52a64445299baa1fea5a173bf1d3dd9a1a5a \ - --hash=sha256:a180c5e59792af262bf263b21a3c49353f25945d8d9f70628e73de370d55e1e1 \ - --hash=sha256:a277ab8928b9f299723bc1a2dabb1265911b1a76341f90a510368ca44ad9ab66 \ - --hash=sha256:a5fe03b42827c13cdccd08e6c0247b6a6d4b5e3cdc53fd1749f5896adcdc2356 \ - --hash=sha256:a6c5863edfbe888d9eff9c8b8087354e27618d9da76425c119293f11712a6319 \ - --hash=sha256:a89c23ef8d2c6b27fd200a42aa4ac72786e7c60d40efdc76e6011260b6e949c4 \ - --hash=sha256:adb2597b428735679446b46c8badf467b4ca5f5056aae4d51a19f9570301b1ad \ - --hash=sha256:ae196f021b5e7c78e918242d217db021ed2a6ace2bc6ae94c0fc596221c7f58d \ - --hash=sha256:ae89db9e5f98a11a4bf50407d4363e7b09b31e55bc117b4f7d80aab97ba009e5 \ - --hash=sha256:aed52fea0513bac0ccde438c188c8a471c4e0f457c2dd20cdbf6ea7a450046c7 \ - --hash=sha256:aef65cd602a6d0e0ff6f9930fcb1c8fec60dd2cfcb6facaf4bdb0e5873042db0 \ - --hash=sha256:af21eb4409a119e365397b2adbaca4c9ccab56543a65d5dbd9f920d6ac29f686 \ - --hash=sha256:b14b2d9dac08e28bb8046a1a0434b1750eb221c8f5b87a68f4fa11a6f97b5e34 \ - --hash=sha256:bb6d88045545b26da47aa879dd4a89a71d1dce0f0e549b1abcb31dfe4a8eac49 \ - --hash=sha256:bb8cc7534f51d9a017b93e3e85b260924f909601c3df002bcdb58ddb4dc41a5c \ - --hash=sha256:bc17a677b21b3502a21f66a8cc64f5bfad4df8a0b8434d661666f8ce90ac3af1 \ - --hash=sha256:bd6c2a1c7573c64738d716488d2cdd3c00e340e4835707d8fdb8dc1a66ef164e \ - --hash=sha256:bd9b23791fe793e4968dba0c447e12f78e425c59fc0e3b97f6450f4781f3ee60 \ - --hash=sha256:c03a41a8784091e67a39648f70c5f97b5b6a37f216896d44d2cdcb82615339a0 \ - --hash=sha256:c0f081d69a6e58272819b70288d3221a6ee64b98df852631c80f293514d3b274 \ - --hash=sha256:c35abb8bfff0185efac5878da64c45dafd2b37fb0383add1be155a763c1f083d \ - --hash=sha256:c36c333c39be2dbca264d7803333c896ab8fa7d4d6f0ab7edb7dfd7aea6e98c0 \ - --hash=sha256:c45e9440fb78f8ddabcf714b68f936737a121355bf59f3907f4e17721b9d1aae \ - --hash=sha256:c593052c465475e64bbfe5dbd81680f64a67fdc752c56d7a0ae205dc8aeefe0f \ - --hash=sha256:cdd68a1fb318e290a2077696b7eb7a21a49163c455979c639bf5a5dcdc46617d \ - --hash=sha256:ce3412fbe1e31eb81ea42f4169ed94861c56e643189e1e75f0041f3fe7020abe \ - --hash=sha256:cf1493cd8607bec4d8a7b9b004e699fcf8f9103a9284cc94962cb73d20f9d4a3 \ - --hash=sha256:cf29836da5119f3c8a8a70667b0ef5fdca3bb12f80fd06487cfa575b3909b393 \ - --hash=sha256:d4a48e5b3c2a489fae013b7589308a40146ee081f6f509e047e0e096084ceca1 \ - --hash=sha256:d560742f3c0d62afaccf9f41fe485ed69bd7661a241f86a3ef0f0fb8b1a397af \ - --hash=sha256:d6038d37043bced98a66e68d3aa2b6a35505dc01328cd65217cefe82f25def44 \ - --hash=sha256:d61f00a0869d77422d9b2aba989e2d24afa6ffd552af442e0e58de4f35ea6d00 \ - --hash=sha256:d635aab80466bc95771bb78d5370e74d36d1fe31467b6b29b8b57b2a3cd7d22c \ - --hash=sha256:dca4bbc466a95ba9c0234ef56d7dd9509f63da22274589ebd4ed7f1f4d4c54e3 \ - --hash=sha256:dd915403e231e6b1809fe9b6d9fc55cf8fb5e02765ac625d9cd623342a7905d7 \ - --hash=sha256:e044c39e41b92c845bc815e5ae4230804e8e7bc29e399b0437d64222d92809dd \ - --hash=sha256:e060d01aec0a910bdccb8be71faf34e7799ce36950f8294c8bf612cba65a2c9e \ - --hash=sha256:e1421b502d83040e6d7fb2fb18dff63957f720da3d77b2fbd3187ceb63755d7b \ - --hash=sha256:e17b8d5d6a8c47c85e68ca8379def1303fd360c3e22093a807cd34a71cd082b8 \ - --hash=sha256:e5f4d355f0a2b1a31bc3edec6795b46324349c9cb25eed068049e4f472fb4259 \ - --hash=sha256:e712b419df8ba5e42b226c510472b37bd57b38e897d3eca5e8cfd410a29fa859 \ - --hash=sha256:e74327fb75de8986940def6e8dee4f127cc9752bee7355bb323cc5b2659b6d46 \ - --hash=sha256:e80c8378d8f3d83cd3164da1ad2df9e37a666cdde7b1cb2298ed0b558064be30 \ - --hash=sha256:e8ac484bf18ce6975760921bb6148041faa8fef0547200386ea0b52b5d27bf7b \ - --hash=sha256:eca9705049ad3c7345d574e3510665cb2cf844c2f2dcfe675332677f081cbd46 \ - --hash=sha256:ed065083d0898c9d5b4bbec7b026fd755ff7454e6e8b73a67f8c744b13986e24 \ - --hash=sha256:edac0f1ab77644605be2cbba52e6b7f630731fc42b34cb0f634be1a6eface56a \ - --hash=sha256:effc3f449787117233702311a1b7d8f59cba9ced946ba727bdc329ec69028e24 \ - --hash=sha256:f22dec1690b584cea26fade98b2435c132c1b5f68e39f5a0b7627cd7ae31f1dc \ - --hash=sha256:f495a1652cf3fbab2eb0639776dad966c2fb874d79d87ca07f9d5f059b8bd215 \ - --hash=sha256:f496c9c3cc02230093d8330875c4c3cdfc3b73612a5fd921c65d39cbcef08063 \ - --hash=sha256:f59099f9b66f0d7145115e6f80dd8b1d847176df89b234a5a6b3f00437aa0832 \ - --hash=sha256:f59ad4c0e8f6bba240a9bb85504faa1ab438237199d4cce5f622761507b8f6a6 \ - --hash=sha256:fbccdc05410c9ee21bbf16a35f4c1d16123dcdeb8a1d38f33654fa21d0234f79 \ - --hash=sha256:fea24543955a6a729c45a73fe90e08c743f0b3334bbf3201e6c4bc1b0c7fa464 +charset-normalizer==3.4.9 \ + --hash=sha256:0327fcd59a935777d83410750c50600ee9571af2846f71ce40f25b13da1ef380 \ + --hash=sha256:03d07803992c6c7bbc976327f34b18b6160327fc81cb82c9d504720ac0be3b62 \ + --hash=sha256:04ce310cb89c15df659582aee80a0603788732a5e017d5bd5c81158106ce249c \ + --hash=sha256:0d861473f743244d349b50f850d10eb87aeb22bbdcc8e64f79273c94af5a8226 \ + --hash=sha256:0e94703ec9684807f20cfb5eed95c70f67f2a8f21ad620146d7b5a13677b93e5 \ + --hash=sha256:0fa1aec2d32bcc03c8fa0f6f1712caad1adc38509f31142112e5c9daf5b9c833 \ + --hash=sha256:16b65ea0f2465b6fb52aa22de5eca612aa964ddfec00a912e26f4656cbef890b \ + --hash=sha256:16d10d789dd9bcca1173c95af82c58433122564b7bc39385124be735a35cbe99 \ + --hash=sha256:19ac87f93086ce37b86e098888555c4b4bc48102279bae3350098c0ed664b501 \ + --hash=sha256:1d22856ffbe153a602df38e4a5464f0b748a54002e0d69ac6d2ad0a197cc99ec \ + --hash=sha256:21e764fd1e70b6a3e205a0e46f3051701f98a8cb3fad66eeb80e48bb502f8698 \ + --hash=sha256:231ddcbb35e2ff8973e1365db41fe0572662893b99a05deb183b68ad4c0c8bd4 \ + --hash=sha256:253a4a220747e8b5faf57ec320c4f5efb0cef05f647420bf267143ec15dba10a \ + --hash=sha256:280081916dc341820640489a66e4696049401ef1cf6dd672f672e70ad915aca3 \ + --hash=sha256:2a441ea71902098ffe78c5abe6c494f44160b4af614ed16c3d9a3b1d17fd8ee2 \ + --hash=sha256:304b13570067b2547562e308af560b3963857b1fa90bd6afd978130130fe2d6a \ + --hash=sha256:32286a2c8d167e897177b673176c1e3e00d4057caf5d2b64eef9a3666b03018e \ + --hash=sha256:33bdcc2a32c0a0e861f60841a512c8acc658c87c2ac59d89e3a46dacf7d866e4 \ + --hash=sha256:375b83ed0aecfce76c16d198fbc21f3b11b337d68662bea0a995046682a11419 \ + --hash=sha256:3c09a49d6cde137258beb3d551994a2927fd35ad5cf96aed573f61bbd67c5f84 \ + --hash=sha256:3d92613ec25e43b05f042302531ec0f00b8445190e43325880cbd6ab7c2581da \ + --hash=sha256:40a126142a56b2dfc0aacbad1de8310cbf60da7656db0e6b16eebd48e3e93519 \ + --hash=sha256:416c229f77e5ea25b3dfd4b582f8d73d7e43c22320302b9ab128a2d3a0b38efe \ + --hash=sha256:432786d3561e69aeeae6c7e8648964ce0ad05736120135601f87ac26b9c83381 \ + --hash=sha256:43b9e366a31fdd1c87d0eb08f579b4a82b723ea54338f040d6b4e518a026ea29 \ + --hash=sha256:440eede837960000d74978f0eba527be106b5b9aee0daf779d395276ed0b0614 \ + --hash=sha256:45b0cc4e3556cd875e09102988d1ab8356c998b596c9fced84547c8138b487a0 \ + --hash=sha256:476743fe6dfe14a2da12e3ac79125dc84a3b2cf8094369a47a1529b0cd8549fe \ + --hash=sha256:4773092f8019072343a7447203308b176e10199920eb02d6195e81bbb3274c29 \ + --hash=sha256:4b3dac63058cc36820b0dd072f89898604e2d39686fe05321729d00d8ac185a0 \ + --hash=sha256:4d1c96a7a18b9690a4d46df09e3e3382406ae3213727cd1019ebade1c4a81917 \ + --hash=sha256:51307f5c71007673a2bf8232ad973483d281e74cb99c8c5a990af1eefa6277d9 \ + --hash=sha256:51447e9aa2684679af07ca5021c3db526e0284347ebf4ffcec1154c3350cfe32 \ + --hash=sha256:58150c9f9b9a552505912d182ccdf26f6396fb6094816ceebcbb20eecabaed94 \ + --hash=sha256:5b10cd92fc5c498b35a8635df6d5a100207f88b63a4dc1de7ef9a548e1e2cd63 \ + --hash=sha256:5e226f6218febc71f6c1fc2fafb91c226f75bdc1d8fb12d66823716e891608fd \ + --hash=sha256:609b3ba8fcc0fb5ab7af00719d0fb6ad0cb518e48e7712d12fd68f1327951198 \ + --hash=sha256:60f44ade2cf573dad7a277e6f8ca9a51a21dda572b13bd7d8539bb3cd5dbedde \ + --hash=sha256:611057cc5d5c0afc743ba8be6bd828c17e0aaa8643f9d0a9b9bb7dea80eb8012 \ + --hash=sha256:6366a16e1a25018694d6a5d784d09b046edc9eac40ea2b54065c3052672516a1 \ + --hash=sha256:65a7ff3f705e57d392f7261b6d0550fe137c3019477431f1c355e0db0a7d3e15 \ + --hash=sha256:673611bbd43f0810bec0b0f028ddeaaa501190339cac411f347ac76917c3ae7b \ + --hash=sha256:67830fc78e67501f47bb950471b2dcb9b35b140084429318e862895a8e89c993 \ + --hash=sha256:68ce9f4d6b26d5ccbf7fd4459bf75f74a0a146677ebba80597df60cbdb20e6f4 \ + --hash=sha256:68e5f26a1ad57ded6d1cfb85331d1c1a195314756471d97758c48498bb4dcdf5 \ + --hash=sha256:69b157c5d3292bcd443faca052f3096f637f1e074b98212a933c074ae23dc3b8 \ + --hash=sha256:75286256590a6320cf106a0d28970d3560aad9ee09aa7b34fb40524792436d35 \ + --hash=sha256:78841cccf1af7b40f6f716338d50c0902dbe88d9f800b3c973b7a9a0a693a642 \ + --hash=sha256:78fa18e436a1a0e58dbd7e02fc4473f3f32cceb12df9dfca542d075961c307d2 \ + --hash=sha256:79580094b00d1789d1f93ea55bc43cb2f611910c72235b7657f3482ddcc1b22d \ + --hash=sha256:7b86a2b16095d250c6f58b3d9b2eee6f4147754344f3dab0922f7c9bf7d226c9 \ + --hash=sha256:83aed2c10721ddd90f68140685391b50811a880af20654c59af6b6c66c40513c \ + --hash=sha256:84fd18bcc17526fc2b3c1af7d2b9217d32c9c04448c16ec693b9b4f1985c3d33 \ + --hash=sha256:871ff67ea1aad4dfd91736464934d56b32dac49f9fbe16cddba36198a7b3a0db \ + --hash=sha256:898f0e9068ca27d37f8e83a5b962821df851532e6c4a7d615c1c033f9da6eedf \ + --hash=sha256:8a79d9f4d8001473a30c163556b3c3bfebec837495a412dde78b51672f6134f9 \ + --hash=sha256:8c041122946b7ba21bb32c45b1aa57b1be35527690aeb3c5c234521085632eee \ + --hash=sha256:90c44bc373b7687f6948b693cceaea1348ae0975d7474746559494468e3c1d84 \ + --hash=sha256:9104ed0bd76a429d46f9ec0dbc9b08ad1d2dcdf2b00a5a0daa1c145329b35b44 \ + --hash=sha256:920079c3f7456fa213e0829ed2073aaa727fd39d889ead5b4f35d0de5460d04f \ + --hash=sha256:93d59d504b230e83c7a843251681959a0b6a9cd76f6e146ce1b8a80eb8739af9 \ + --hash=sha256:9b2aff1c7b3884512b9512c3eaadd9bab39fb45042ffaaa1dd08ff2b9f8109d9 \ + --hash=sha256:9b8e0f3107e2200b76f6054de99016eac3ee6762713587b36baaa7e4bd2ae177 \ + --hash=sha256:9bb41182d93ea91f60b4bc8fbf4c820c69ef8a12ab2d917f3f1834f1acad07e8 \ + --hash=sha256:9cdef90ae47919cae358d8ab15797a800ed41da7aba5d72419fb510729e2ed4b \ + --hash=sha256:a1786910334ed46ab1dd73222f2cd1e05c2c3bb39f6dddb4f8b36fc382058a39 \ + --hash=sha256:a4cfde78a9f2880208d16a93b795726a3017d5977e08d1e162a7a31322479c41 \ + --hash=sha256:a4fbdde9dd4a9ce5fd52c2b3a347bb50cc89483ef783f1cb00d408c13f7a96c0 \ + --hash=sha256:aa99adc8f081b475a12843953db36831eaf83ec33eb46a90629ca6a5de45a616 \ + --hash=sha256:ac351b3b8014eead140e77e9717e2992c6bbe30b63bc3422422eb84865412e3d \ + --hash=sha256:ad41ba96094304aa090f5a30cb6e4fb3b3f1c264c523394b4c39bbacc4dc92ba \ + --hash=sha256:b5314963fce9b0b12743891de876e724997864ee22aa496f903f426c7e2fa5b2 \ + --hash=sha256:bcf74c1df76758a395bf0af608c04c82257523f55c9868b334f06270d0f2112b \ + --hash=sha256:bd47ba7fc3ca94896759ea0109775132d3e7ab921fbf54038e1bab2e46c313c9 \ + --hash=sha256:c0323c9daef75ef2e5083624b4585018a0c9d5e3b40f607eed81a311270b934b \ + --hash=sha256:c1225416b463483160e4af85d5fc3a9690ccb53fd4b1865a6437825f5ede3209 \ + --hash=sha256:c1c948747b03be832dceed96ca815cef7360de9aa19d37c730f8e3f6101aca48 \ + --hash=sha256:c25fe15c70c59eb7c5ce8c06a1f3fa1da0ecc5ea1e7a5922c40fd2fa9b0d5046 \ + --hash=sha256:cc1b0fff8ead343dae06305f954eb8468ba0ec1a97881f42489d198e4ce3c632 \ + --hash=sha256:cd6280cf040f233bd7d3407b743b4b4c74f70e8e1c4199cb112a62c941c0772a \ + --hash=sha256:cd6c3d4b783c556fa00bf540854e42f135e2f256abd29669fcd0da0f2dec79c2 \ + --hash=sha256:d4d6fcde76f94f5cb9e43e9e9a61f16dacefd228cbbf6f1a09bd9b219a92f1a1 \ + --hash=sha256:ddf4af30b417d9fe16481e9b81c27ab2a7cde1ff7ba3e85653b02db7d145dc7b \ + --hash=sha256:df115d4d83168fdf2cae48ef1ff6d1cb4c466364e30861b37121de0f3bf1b990 \ + --hash=sha256:df7276909358e5635ae203673ab7e509ddd224225a8d6b0790bf13eb2bde1cc5 \ + --hash=sha256:e4fd89cc178bced6ad29cb3e6dd4aa63fa5017c3524dbd0b25998fb64a87cc8b \ + --hash=sha256:e9701d0049d92c16703a42771b98d560b95248949f23f8cf7b4eddd201814fb9 \ + --hash=sha256:ee2f2a527e3c1a6e6411eb4209642e138b544a2d72fe5d0d76daf77b24063534 \ + --hash=sha256:f7fb7d750cfa0a070d2c24e831fd3481019a60dd317ea2b39acbcebc08b6ed81 \ + --hash=sha256:f840ed6d8ecba8255df8c42b87fadeda98ddfc6eeec05e2dc66e26d46dd6f58a \ + --hash=sha256:f86c6358749bd4fda175388691e3ba8c46e24c5347d0afd20f9b7edfc9faf07d \ + --hash=sha256:fa36ec09ef71d158186bc79e359ff5fdd6e7996fe8ab638f00d6b93139ba4fcf \ + --hash=sha256:fe2c7201c642b7c308f1675355ad7ff7b66acfe3541625efe5a3ad38f29d6115 # via requests colorama==0.4.6 ; sys_platform == 'win32' \ --hash=sha256:08695f5cb7ed6e0531a20572697297273c47b8cae5a63ffc6d6ed5c201be6e44 \ --hash=sha256:4f1d9991f5acc0ca119f9d443620b77f9d6b33703e51011c16baf57afb285fc6 - # via sphinx + # via + # pytest + # sphinx docutils==0.21.2 ; python_full_version < '3.11' \ --hash=sha256:3a6b18732edf182daa3cd12775bbb338cf5691468f91eeeb109deff6ebfa986f \ --hash=sha256:dafca5b9e384f0e419294eb4d2ff9fa826435bf15f15b7bd45723e8ad76811b2 @@ -183,6 +149,10 @@ docutils==0.22.4 ; python_full_version >= '3.11' \ # myst-parser # sphinx # sphinx-rtd-theme +exceptiongroup==1.3.1 ; python_full_version < '3.11' \ + --hash=sha256:8b412432c6055b0b7d14c310000ae93352ed6754f70fa8f7c34141f91c4e3219 \ + --hash=sha256:a7a39a3bd276781e98394987d3a5701d0c4edffb633bb7a5144577f82c773598 + # via pytest idna==3.18 \ --hash=sha256:7f952cbe720b688055e3f87de14f5c3e5fdaa8bc3928985c4077ca689de849a2 \ --hash=sha256:ffb385a7e039654cef1ab9ef32c6fafe283c0c0467bba1d9029738ce4a14a848 @@ -199,6 +169,14 @@ importlib-metadata==8.7.1 ; python_full_version < '3.10' \ --hash=sha256:49fef1ae6440c182052f407c8d34a68f72efc36db9ca90dc0113398f2fdde8bb \ --hash=sha256:5a1f80bf1daa489495071efbb095d75a634cf28a8bc299581244063b53176151 # via sphinx +iniconfig==2.1.0 ; python_full_version < '3.10' \ + --hash=sha256:3abbd2e30b36733fee78f9c7f7308f2d0050e88f0087fd25c2645f63c773e1c7 \ + --hash=sha256:9deba5723312380e77435581c6bf4935c94cbfab9b1ed33ef8d238ea168eb760 + # via pytest +iniconfig==2.3.0 ; python_full_version >= '3.10' \ + --hash=sha256:c76315c77db068650d49c5b56314774a7804df16fee4402c1f19d6d15d8c4730 \ + --hash=sha256:f631c04d2c48c52b84d0d0549c99ff3859c98df65b3101406327ecc7d53fbf12 + # via pytest jinja2==3.1.6 \ --hash=sha256:0137fb05990d35f1275a587e9aee6d56da821fc83491a0fb838183be43f66d6d \ --hash=sha256:85ece4451f492d0c13c5dd7c13a64681a86afae63a5f347908daf103ce6d2f67 @@ -343,12 +321,17 @@ packaging==26.2 \ --hash=sha256:5fc45236b9446107ff2415ce77c807cee2862cb6fac22b8a73826d0693b0980e \ --hash=sha256:ff452ff5a3e828ce110190feff1178bb1f2ea2281fa2075aadb987c2fb221661 # via + # pytest # readthedocs-sphinx-ext # sphinx pefile==2024.8.26 \ --hash=sha256:3ff6c5d8b43e8c37bb6e6dd5085658d658a7a0bdcd20b6a07b1fcfc1c4e9d632 \ --hash=sha256:76f8b485dcd3b1bb8166f1128d395fa3d87af26360c2358fb75b80019b957c6f # via rules-python-docs (docs/pyproject.toml) +pluggy==1.6.0 \ + --hash=sha256:7dcc130b76258d33b90f61b658791dede3486c3e6bfb003ee5c9bfb396dd22f3 \ + --hash=sha256:e920276dd6813095e9377c0bc5566d94c932c33b27a3e3945d8389c374dd4746 + # via pytest pyelftools==0.32 ; python_full_version < '3.10' \ --hash=sha256:013df952a006db5e138b1edf6d8a68ecc50630adbd0d83a2d41e7f846163d738 \ --hash=sha256:6de90ee7b8263e740c8715a925382d4099b354f29ac48ea40d840cf7aa14ace5 @@ -360,7 +343,24 @@ pyelftools==0.33 ; python_full_version >= '3.10' \ pygments==2.20.0 \ --hash=sha256:6757cd03768053ff99f3039c1a36d6c0aa0b263438fcab17520b30a303a82b5f \ --hash=sha256:81a9e26dd42fd28a23a2d169d86d7ac03b46e2f8b59ed4698fb4785f946d0176 - # via sphinx + # via + # pytest + # sphinx +pytest==8.4.2 ; python_full_version < '3.10' \ + --hash=sha256:86c0d0b93306b961d58d62a4db4879f27fe25513d4b969df351abdddb3c30e01 \ + --hash=sha256:872f880de3fc3a5bdc88a11b39c9710c3497a547cfa9320bc3c5e62fbf272e79 + # via + # rules-python-docs (docs/pyproject.toml) + # pytest-bazel +pytest==9.1.1 ; python_full_version >= '3.10' \ + --hash=sha256:1088fbde8f2b49d95a549a195707afa7a76a3ce9bcadc26b6d71f0ffda5fe313 \ + --hash=sha256:37a86b45efb9a47a61a36449063e8e18d0cab3161329fc099eb21783169c4f0c + # via + # rules-python-docs (docs/pyproject.toml) + # pytest-bazel +pytest-bazel==0.1.6 \ + --hash=sha256:a29e80e1d67c3db801bdd4d0b6b742f2bfb48cd6841caa33401458e5c4e29c21 + # via rules-python-docs (docs/pyproject.toml) pyyaml==6.0.3 \ --hash=sha256:00c4bdeba853cc34e7dd471f16b4114f4162dc03e6b7afcc2128711f0eca823c \ --hash=sha256:0150219816b6a1fa26fb4699fb7daa9caf09eb1999f3b70fb6e786805e80375a \ @@ -589,14 +589,16 @@ tomli==2.4.1 ; python_full_version < '3.11' \ --hash=sha256:ff18e6a727ee0ab0388507b89d1bc6a22b138d1e2fa56d1ad494586d61d2eae9 \ --hash=sha256:ff2983983d34813c1aeb0fa89091e76c3a22889ee83ab27c5eeb45100560c049 # via + # pytest # sphinx # sphinx-autodoc2 -typing-extensions==4.15.0 \ - --hash=sha256:0cea48d173cc12fa28ecabc3b837ea3cf6f38c6d1136f85cbaaf598984861466 \ - --hash=sha256:f0fa19c6845758ab08074a0cfa8b7aecb71c999ca73d62883bc25cc018c4e548 +typing-extensions==4.16.0 \ + --hash=sha256:481caa481374e813c1b176ada14e97f1f67a4539ce9cfeb3f350d78d6370c2e8 \ + --hash=sha256:dc983d19a509c94dba722ee6abd33940f7c05a89e243c47e907eb4db6f1a43e5 # via # rules-python-docs (docs/pyproject.toml) # astroid + # exceptiongroup # sphinx-autodoc2 urllib3==2.6.3 ; python_full_version < '3.10' \ --hash=sha256:1b62b6884944a57dbe321509ab94fd4d3b307075e0c2eae991ac71ee15ad38ed \ diff --git a/docs/uv.lock b/docs/uv.lock index 72e8f307a9..c86b4cd18e 100644 --- a/docs/uv.lock +++ b/docs/uv.lock @@ -263,6 +263,18 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/02/10/5da547df7a391dcde17f59520a231527b8571e6f46fc8efb02ccb370ab12/docutils-0.22.4-py3-none-any.whl", hash = "sha256:d0013f540772d1420576855455d050a2180186c91c15779301ac2ccb3eeb68de", size = 633196, upload-time = "2025-12-18T19:00:18.077Z" }, ] +[[package]] +name = "exceptiongroup" +version = "1.3.1" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "typing-extensions", marker = "python_full_version < '3.11'" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/50/79/66800aadf48771f6b62f7eb014e352e5d06856655206165d775e675a02c9/exceptiongroup-1.3.1.tar.gz", hash = "sha256:8b412432c6055b0b7d14c310000ae93352ed6754f70fa8f7c34141f91c4e3219", size = 30371, upload-time = "2025-11-21T23:01:54.787Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/8a/0e/97c33bf5009bdbac74fd2beace167cab3f978feb69cc36f1ef79360d6c4e/exceptiongroup-1.3.1-py3-none-any.whl", hash = "sha256:a7a39a3bd276781e98394987d3a5701d0c4edffb633bb7a5144577f82c773598", size = 16740, upload-time = "2025-11-21T23:01:53.443Z" }, +] + [[package]] name = "idna" version = "3.15" @@ -312,6 +324,34 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/fa/5e/f8e9a1d23b9c20a551a8a02ea3637b4642e22c2626e3a13a9a29cdea99eb/importlib_metadata-8.7.1-py3-none-any.whl", hash = "sha256:5a1f80bf1daa489495071efbb095d75a634cf28a8bc299581244063b53176151", size = 27865, upload-time = "2025-12-21T10:00:18.329Z" }, ] +[[package]] +name = "iniconfig" +version = "2.1.0" +source = { registry = "https://pypi.org/simple" } +resolution-markers = [ + "python_full_version < '3.10'", +] +sdist = { url = "https://files.pythonhosted.org/packages/f2/97/ebf4da567aa6827c909642694d71c9fcf53e5b504f2d96afea02718862f3/iniconfig-2.1.0.tar.gz", hash = "sha256:3abbd2e30b36733fee78f9c7f7308f2d0050e88f0087fd25c2645f63c773e1c7", size = 4793, upload-time = "2025-03-19T20:09:59.721Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/2c/e1/e6716421ea10d38022b952c159d5161ca1193197fb744506875fbb87ea7b/iniconfig-2.1.0-py3-none-any.whl", hash = "sha256:9deba5723312380e77435581c6bf4935c94cbfab9b1ed33ef8d238ea168eb760", size = 6050, upload-time = "2025-03-19T20:10:01.071Z" }, +] + +[[package]] +name = "iniconfig" +version = "2.3.0" +source = { registry = "https://pypi.org/simple" } +resolution-markers = [ + "python_full_version >= '3.14'", + "python_full_version == '3.13.*'", + "python_full_version == '3.12.*'", + "python_full_version == '3.11.*'", + "python_full_version == '3.10.*'", +] +sdist = { url = "https://files.pythonhosted.org/packages/72/34/14ca021ce8e5dfedc35312d08ba8bf51fdd999c576889fc2c24cb97f4f10/iniconfig-2.3.0.tar.gz", hash = "sha256:c76315c77db068650d49c5b56314774a7804df16fee4402c1f19d6d15d8c4730", size = 20503, upload-time = "2025-10-18T21:55:43.219Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/cb/b1/3846dd7f199d53cb17f49cba7e651e9ce294d8497c8c150530ed11865bb8/iniconfig-2.3.0-py3-none-any.whl", hash = "sha256:f631c04d2c48c52b84d0d0549c99ff3859c98df65b3101406327ecc7d53fbf12", size = 7484, upload-time = "2025-10-18T21:55:41.639Z" }, +] + [[package]] name = "jinja2" version = "3.1.6" @@ -592,6 +632,15 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/54/16/12b82f791c7f50ddec566873d5bdd245baa1491bac11d15ffb98aecc8f8b/pefile-2024.8.26-py3-none-any.whl", hash = "sha256:76f8b485dcd3b1bb8166f1128d395fa3d87af26360c2358fb75b80019b957c6f", size = 74766, upload-time = "2024-08-26T21:01:02.632Z" }, ] +[[package]] +name = "pluggy" +version = "1.6.0" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/f9/e2/3e91f31a7d2b083fe6ef3fa267035b518369d9511ffab804f839851d2779/pluggy-1.6.0.tar.gz", hash = "sha256:7dcc130b76258d33b90f61b658791dede3486c3e6bfb003ee5c9bfb396dd22f3", size = 69412, upload-time = "2025-05-15T12:30:07.975Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/54/20/4d324d65cc6d9205fabedc306948156824eb9f0ee1633355a8f7ec5c66bf/pluggy-1.6.0-py3-none-any.whl", hash = "sha256:e920276dd6813095e9377c0bc5566d94c932c33b27a3e3945d8389c374dd4746", size = 20538, upload-time = "2025-05-15T12:30:06.134Z" }, +] + [[package]] name = "pyelftools" version = "0.32" @@ -610,6 +659,64 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/f4/7e/a72dd26f3b0f4f2bf1dd8923c85f7ceb43172af56d63c7383eb62b332364/pygments-2.20.0-py3-none-any.whl", hash = "sha256:81a9e26dd42fd28a23a2d169d86d7ac03b46e2f8b59ed4698fb4785f946d0176", size = 1231151, upload-time = "2026-03-29T13:29:30.038Z" }, ] +[[package]] +name = "pytest" +version = "8.4.2" +source = { registry = "https://pypi.org/simple" } +resolution-markers = [ + "python_full_version < '3.10'", +] +dependencies = [ + { name = "colorama", marker = "python_full_version < '3.10' and sys_platform == 'win32'" }, + { name = "exceptiongroup", marker = "python_full_version < '3.10'" }, + { name = "iniconfig", version = "2.1.0", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.10'" }, + { name = "packaging", marker = "python_full_version < '3.10'" }, + { name = "pluggy", marker = "python_full_version < '3.10'" }, + { name = "pygments", marker = "python_full_version < '3.10'" }, + { name = "tomli", marker = "python_full_version < '3.10'" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/a3/5c/00a0e072241553e1a7496d638deababa67c5058571567b92a7eaa258397c/pytest-8.4.2.tar.gz", hash = "sha256:86c0d0b93306b961d58d62a4db4879f27fe25513d4b969df351abdddb3c30e01", size = 1519618, upload-time = "2025-09-04T14:34:22.711Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/a8/a4/20da314d277121d6534b3a980b29035dcd51e6744bd79075a6ce8fa4eb8d/pytest-8.4.2-py3-none-any.whl", hash = "sha256:872f880de3fc3a5bdc88a11b39c9710c3497a547cfa9320bc3c5e62fbf272e79", size = 365750, upload-time = "2025-09-04T14:34:20.226Z" }, +] + +[[package]] +name = "pytest" +version = "9.1.1" +source = { registry = "https://pypi.org/simple" } +resolution-markers = [ + "python_full_version >= '3.14'", + "python_full_version == '3.13.*'", + "python_full_version == '3.12.*'", + "python_full_version == '3.11.*'", + "python_full_version == '3.10.*'", +] +dependencies = [ + { name = "colorama", marker = "python_full_version >= '3.10' and sys_platform == 'win32'" }, + { name = "exceptiongroup", marker = "python_full_version == '3.10.*'" }, + { name = "iniconfig", version = "2.3.0", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version >= '3.10'" }, + { name = "packaging", marker = "python_full_version >= '3.10'" }, + { name = "pluggy", marker = "python_full_version >= '3.10'" }, + { name = "pygments", marker = "python_full_version >= '3.10'" }, + { name = "tomli", marker = "python_full_version == '3.10.*'" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/e4/47/b9efed96c114afcfa3c9d3fe98a76a1d14c74a9e266d397cf6eb64be5e01/pytest-9.1.1.tar.gz", hash = "sha256:1088fbde8f2b49d95a549a195707afa7a76a3ce9bcadc26b6d71f0ffda5fe313", size = 1636369, upload-time = "2026-06-19T10:58:32.857Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/24/25/1de2678b631f5a49215c6c96fff41ba892b0a34df68d6d80292b1b48aa7f/pytest-9.1.1-py3-none-any.whl", hash = "sha256:37a86b45efb9a47a61a36449063e8e18d0cab3161329fc099eb21783169c4f0c", size = 386536, upload-time = "2026-06-19T10:58:31.347Z" }, +] + +[[package]] +name = "pytest-bazel" +version = "0.1.6" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "pytest", version = "8.4.2", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.10'" }, + { name = "pytest", version = "9.1.1", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version >= '3.10'" }, +] +wheels = [ + { url = "https://files.pythonhosted.org/packages/c2/90/4726a39728fb5a5f77a773c5e9d7bfd3a0120aa2029963a88fd06e2c5fb2/pytest_bazel-0.1.6-py3-none-any.whl", hash = "sha256:a29e80e1d67c3db801bdd4d0b6b742f2bfb48cd6841caa33401458e5c4e29c21", size = 10126, upload-time = "2025-10-31T08:41:17.165Z" }, +] + [[package]] name = "pyyaml" version = "6.0.3" @@ -761,6 +868,9 @@ dependencies = [ { name = "myst-parser", version = "5.1.0", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version >= '3.11'" }, { name = "pefile" }, { name = "pyelftools" }, + { name = "pytest", version = "8.4.2", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.10'" }, + { name = "pytest", version = "9.1.1", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version >= '3.10'" }, + { name = "pytest-bazel" }, { name = "readthedocs-sphinx-ext" }, { name = "sphinx", version = "7.4.7", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.10'" }, { name = "sphinx", version = "8.1.3", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version == '3.10.*'" }, @@ -781,6 +891,8 @@ requires-dist = [ { name = "myst-parser" }, { name = "pefile" }, { name = "pyelftools" }, + { name = "pytest" }, + { name = "pytest-bazel" }, { name = "readthedocs-sphinx-ext" }, { name = "sphinx" }, { name = "sphinx-autodoc2" }, diff --git a/tests/pytest_test/BUILD.bazel b/tests/pytest_test/BUILD.bazel new file mode 100644 index 0000000000..b15094a615 --- /dev/null +++ b/tests/pytest_test/BUILD.bazel @@ -0,0 +1,22 @@ +load("//tests/support:support.bzl", "SUPPORTS_BZLMOD") +load("//tests/support/pytest_test:pytest_test.bzl", "pytest_test") + +pytest_test( + name = "pytest_script_venv_test", + srcs = [ + "basic_test.py", + ], + config_settings = { + "@rules_python//python/config_settings:bootstrap_impl": "script", + "@rules_python//python/config_settings:venvs_site_packages": "yes", + }, + target_compatible_with = SUPPORTS_BZLMOD, +) + +pytest_test( + name = "pytest_default_test", + srcs = [ + "basic_test.py", + ], + target_compatible_with = SUPPORTS_BZLMOD, +) diff --git a/tests/pytest_test/basic_test.py b/tests/pytest_test/basic_test.py new file mode 100644 index 0000000000..30e2c41ab9 --- /dev/null +++ b/tests/pytest_test/basic_test.py @@ -0,0 +1,2 @@ +def test_foo(): + assert True diff --git a/tests/support/pytest_test/BUILD.bazel b/tests/support/pytest_test/BUILD.bazel new file mode 100644 index 0000000000..50ad59bf0e --- /dev/null +++ b/tests/support/pytest_test/BUILD.bazel @@ -0,0 +1,30 @@ +load("@bazel_skylib//:bzl_library.bzl", "bzl_library") +load("//python/private:bzlmod_enabled.bzl", "BZLMOD_ENABLED") # buildifier: disable=bzl-visibility + +package(default_visibility = ["//:__subpackages__"]) + +filegroup( + name = "bootstrap_template", + srcs = ["pytest_bootstrap_template.py"], +) + +# keep +bzl_library( + name = "pytest_test", + srcs = ["pytest_test.bzl"], + deps = [ + "//python:py_test", + ], +) + +# These aliases are used to avoid duplicate targets in the deps list +alias( + name = "default_pytest", + actual = "@pypi//pytest" if BZLMOD_ENABLED else "//python/private:empty", +) + +# These aliases are used to avoid duplicate targets in the deps list +alias( + name = "default_pytest_bazel", + actual = "@pypi//pytest_bazel" if BZLMOD_ENABLED else "//python/private:empty", +) diff --git a/tests/support/pytest_test/pytest_bootstrap_template.py b/tests/support/pytest_test/pytest_bootstrap_template.py new file mode 100644 index 0000000000..9769531f47 --- /dev/null +++ b/tests/support/pytest_test/pytest_bootstrap_template.py @@ -0,0 +1,8 @@ +import sys + +import pytest_bazel + +TEST_FILES = """%TEST_FILES%""".splitlines() + +args = sys.argv[1:] + TEST_FILES +sys.exit(pytest_bazel.main(args)) diff --git a/tests/support/pytest_test/pytest_test.bzl b/tests/support/pytest_test/pytest_test.bzl new file mode 100644 index 0000000000..893b22e5ce --- /dev/null +++ b/tests/support/pytest_test/pytest_test.bzl @@ -0,0 +1,74 @@ +"""pytest_test rule implementation.""" + +load("//python:py_test.bzl", "py_test") + +_DEFAULT_PYTEST = Label("//tests/support/pytest_test:default_pytest") +_DEFAULT_PYTEST_BAZEL = Label("//tests/support/pytest_test:default_pytest_bazel") + +def pytest_test( + *, + name, + srcs, + pytest = None, + pytest_bazel = None, + **kwargs): + """Run pytest tests. + + Args: + name: A unique name for this target. + srcs: List of source files (test files). These are the files that + pytest will run as tests. + pytest: The pytest target to use. Defaults to @pypi//pytest. + pytest_bazel: The pytest-bazel target to use. Defaults to + @pypi//pytest_bazel. + **kwargs: Additional arguments passed to py_test. Note that `main` is + not a supported argument. + """ + if pytest == None: + pytest = _DEFAULT_PYTEST + if pytest_bazel == None: + pytest_bazel = _DEFAULT_PYTEST_BAZEL + + bootstrap_target = name + "_bootstrap" + main_file = name + "_boot.py" + _write_pytest_bootstrap( + name = bootstrap_target, + srcs = srcs, + output_name = main_file, + ) + + py_test( + name = name, + main = main_file, + srcs = [bootstrap_target] + srcs, + deps = kwargs.pop("deps", []) + [ + pytest, + pytest_bazel, + ], + **kwargs + ) + +def _write_pytest_bootstrap_impl(ctx): + output = ctx.actions.declare_file(ctx.attr.output_name) + test_files = "\n".join([f.short_path for f in ctx.files.srcs]) + + ctx.actions.expand_template( + output = output, + template = ctx.file._bootstrap_template, + substitutions = { + "%TEST_FILES%": test_files, + }, + ) + return [DefaultInfo(files = depset([output]))] + +_write_pytest_bootstrap = rule( + implementation = _write_pytest_bootstrap_impl, + attrs = { + "output_name": attr.string(mandatory = True), + "srcs": attr.label_list(allow_files = True), + "_bootstrap_template": attr.label( + default = "//tests/support/pytest_test:bootstrap_template", + allow_single_file = True, + ), + }, +) From f0badb1295e81da64c388f1c76ce0354424dad42 Mon Sep 17 00:00:00 2001 From: Greg Date: Sat, 18 Jul 2026 22:24:15 -0400 Subject: [PATCH 842/922] feat: add flag aliases to Starlark-defined flags (#3932) This was breaking rules_python CI with Bazel 9.0.0. @aranguyen and I made some changes that we thought would resolve in 9.1.0 or 9.2.0. The repro I tried on 9.0.0 now passes: Tested: ``` USE_BAZEL_VERSION=9.0.0 bazelisk build --nobuild //tests/py_zipapp:system_python_zipapp ERROR: Analysis of target '//tests/py_zipapp:system_python_zipapp' failed; build aborted: no such package '@@rules_python//python/config_settings': The repository '@@rules_python' could not be resolved: Repository '@@rules_python' is not defined ``` ``` USE_BAZEL_VERSION=9.2.0 bazelisk build --nobuild //tests/py_zipapp:system_python_zipapp INFO: Build completed successfully, 0 total actions ``` References: - https://github.com/bazel-contrib/rules_python/pull/3450#issuecomment-3974932245 - https://github.com/bazel-contrib/rules_python/pull/3450#issuecomment-3986798612 --------- Co-authored-by: Richard Levasseur --- MODULE.bazel | 35 ++++++++++++++++------------------- news/3932.added.md | 3 +++ 2 files changed, 19 insertions(+), 19 deletions(-) create mode 100644 news/3932.added.md diff --git a/MODULE.bazel b/MODULE.bazel index e4c4e08e89..2df354fe61 100644 --- a/MODULE.bazel +++ b/MODULE.bazel @@ -326,25 +326,22 @@ uv_dev.configure( version = "0.11.2", ) -# Temporarily comment out these flag aliases because they break Bazel 9 -# when transitions are also used with a target. -# -# flag_alias( -# name = "build_python_zip", -# starlark_flag = "//python/config_settings:build_python_zip", -# ) +flag_alias( + name = "build_python_zip", + starlark_flag = "//python/config_settings:build_python_zip", +) -# flag_alias( -# name = "incompatible_default_to_explicit_init_py", -# starlark_flag = "//python/config_settings:incompatible_default_to_explicit_init_py", -# ) +flag_alias( + name = "incompatible_default_to_explicit_init_py", + starlark_flag = "//python/config_settings:incompatible_default_to_explicit_init_py", +) -# flag_alias( -# name = "python_path", -# starlark_flag = "//python/config_settings:python_path", -# ) +flag_alias( + name = "python_path", + starlark_flag = "//python/config_settings:python_path", +) -# flag_alias( -# name = "experimental_python_import_all_repositories", -# starlark_flag = "//python/config_settings:experimental_python_import_all_repositories", -# ) +flag_alias( + name = "experimental_python_import_all_repositories", + starlark_flag = "//python/config_settings:experimental_python_import_all_repositories", +) diff --git a/news/3932.added.md b/news/3932.added.md new file mode 100644 index 0000000000..579d1b5f59 --- /dev/null +++ b/news/3932.added.md @@ -0,0 +1,3 @@ +(bzlmod) Added MODULE.bazel flag aliases for Starlark-defined flags: +`build_python_zip`, `incompatible_default_to_explicit_init_py`, +`python_path`, and `experimental_python_import_all_repositories`. From c271d3f93d255e10970efdc9a7496b2768c1ff72 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Sun, 19 Jul 2026 22:40:30 -0700 Subject: [PATCH 843/922] refactor(tools): migrate private release tool tests to pytest and pytest-mock (#3937) Migrates release tool unit tests under `tests/tools/private/release/` to Pytest and Pytest-mock (`mocker` fixture). - Added `pytest-mock` dependency to `docs/pyproject.toml`, updated `docs/uv.lock` & `docs/requirements.txt`, and added `@pypi//pytest_mock` to release test helper `deps`. - Converted release test targets from standard unittest to pytest fixtures. - Refactored tests to use the standard `mocker` fixture from `pytest-mock`. --- AGENTS.md | 10 +- docs/pyproject.toml | 1 + docs/requirements.txt | 12 +- docs/uv.lock | 15 + tests/tools/private/release/BUILD.bazel | 111 +- .../private/release/add_backports_test.py | 142 +-- .../release/backport_create_releases_test.py | 182 ++- .../private/release/backport_prepare_test.py | 469 ++++--- .../private/release/changelog_news_test.py | 631 +++++----- .../release/complete_sync_changelog_test.py | 177 ++- tests/tools/private/release/create_rc_test.py | 702 +++++------ .../release/create_release_branch_test.py | 260 ++-- tests/tools/private/release/gh_test.py | 114 +- tests/tools/private/release/git_test.py | 335 +++-- .../private/release/on_pr_merged_test.py | 154 +-- tests/tools/private/release/prepare_test.py | 453 ++++--- .../private/release/process_backports_test.py | 1092 +++++++---------- tests/tools/private/release/promote_test.py | 582 +++++---- .../private/release/release_issue_test.py | 207 ++-- tests/tools/private/release/release_test.py | 30 +- .../private/release/release_test_helper.py | 94 +- tests/tools/private/release/utils_test.py | 412 ++++--- .../release/backport_create_releases.py | 2 +- tools/private/release/create_release_issue.py | 2 +- tools/private/release/gh.py | 506 ++++---- tools/private/release/mock_gh.py | 89 +- tools/private/release/prepare.py | 4 +- 27 files changed, 3260 insertions(+), 3528 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index 6f6e6fe76a..170aa49c8e 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -42,6 +42,14 @@ because it interferes with code review comments. Follow the advice in `CONTRIBUTING.md` for PR descriptions. PR descriptions become the commit message upon merge. +### Python pytest conventions + +* When registering pytest fixtures from helper modules in test files, use + `pytest_plugins = [""]`. +* Name fixture functions with a `fixture_` prefix (e.g. `def fixture_foo():`), + and pass the public fixture name using the `name` parameter in + `@pytest.fixture(name="foo")`. + ### Starlark style For doc strings, using triple quoted strings when the doc string is more than @@ -98,7 +106,6 @@ def foo_test_suite(name): test_suite(name=name, tests=_tests) ``` - #### Repository rules The function argument `rctx` is a hint that the function is a repository rule, @@ -157,7 +164,6 @@ This repository contains 3 Bazel bzlmod modules. `tests/support/` contains utility code and helpers for testing. - `python/config_settings/BUILD.bazel` contains build flags that are part of the public API. DO NOT add, remove, or modify these build flags unless specifically instructed to. diff --git a/docs/pyproject.toml b/docs/pyproject.toml index ea8a544241..e54fd53504 100644 --- a/docs/pyproject.toml +++ b/docs/pyproject.toml @@ -19,4 +19,5 @@ dependencies = [ "markupsafe", "pytest", "pytest-bazel", + "pytest-mock", ] diff --git a/docs/requirements.txt b/docs/requirements.txt index a0224b9283..5ac2d2b323 100644 --- a/docs/requirements.txt +++ b/docs/requirements.txt @@ -309,9 +309,9 @@ myst-parser==3.0.1 ; python_full_version < '3.10' \ --hash=sha256:6457aaa33a5d474aca678b8ead9b3dc298e89c68e67012e73146ea6fd54babf1 \ --hash=sha256:88f0cb406cb363b077d176b51c476f62d60604d68a8dcdf4832e080441301a87 # via rules-python-docs (docs/pyproject.toml) -myst-parser==5.1.0 ; python_full_version == '3.10.*' \ - --hash=sha256:9c91c52b3cdb4d94a6506e4fab4e2f296c7623a0da0dcbe6de1565c3dad67a8a \ - --hash=sha256:ab69322dc6719dcc7f296479dbb70181b66df6ed315064f92dbc85c0e1bf2f02 +myst-parser==4.0.1 ; python_full_version == '3.10.*' \ + --hash=sha256:5cfea715e4f3574138aecbf7d54132296bfd72bb614d31168f48c477a830a7c4 \ + --hash=sha256:9134e88959ec3b5780aedf8a99680ea242869d012e8821db3126d427edc9c95d # via rules-python-docs (docs/pyproject.toml) myst-parser==5.1.0 ; python_full_version >= '3.11' \ --hash=sha256:9c91c52b3cdb4d94a6506e4fab4e2f296c7623a0da0dcbe6de1565c3dad67a8a \ @@ -352,15 +352,21 @@ pytest==8.4.2 ; python_full_version < '3.10' \ # via # rules-python-docs (docs/pyproject.toml) # pytest-bazel + # pytest-mock pytest==9.1.1 ; python_full_version >= '3.10' \ --hash=sha256:1088fbde8f2b49d95a549a195707afa7a76a3ce9bcadc26b6d71f0ffda5fe313 \ --hash=sha256:37a86b45efb9a47a61a36449063e8e18d0cab3161329fc099eb21783169c4f0c # via # rules-python-docs (docs/pyproject.toml) # pytest-bazel + # pytest-mock pytest-bazel==0.1.6 \ --hash=sha256:a29e80e1d67c3db801bdd4d0b6b742f2bfb48cd6841caa33401458e5c4e29c21 # via rules-python-docs (docs/pyproject.toml) +pytest-mock==3.15.1 \ + --hash=sha256:0a25e2eb88fe5168d535041d09a4529a188176ae608a6d249ee65abc0949630d \ + --hash=sha256:1849a238f6f396da19762269de72cb1814ab44416fa73a8686deac10b0d87a0f + # via rules-python-docs (docs/pyproject.toml) pyyaml==6.0.3 \ --hash=sha256:00c4bdeba853cc34e7dd471f16b4114f4162dc03e6b7afcc2128711f0eca823c \ --hash=sha256:0150219816b6a1fa26fb4699fb7daa9caf09eb1999f3b70fb6e786805e80375a \ diff --git a/docs/uv.lock b/docs/uv.lock index c86b4cd18e..d01b79b784 100644 --- a/docs/uv.lock +++ b/docs/uv.lock @@ -717,6 +717,19 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/c2/90/4726a39728fb5a5f77a773c5e9d7bfd3a0120aa2029963a88fd06e2c5fb2/pytest_bazel-0.1.6-py3-none-any.whl", hash = "sha256:a29e80e1d67c3db801bdd4d0b6b742f2bfb48cd6841caa33401458e5c4e29c21", size = 10126, upload-time = "2025-10-31T08:41:17.165Z" }, ] +[[package]] +name = "pytest-mock" +version = "3.15.1" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "pytest", version = "8.4.2", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.10'" }, + { name = "pytest", version = "9.1.1", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version >= '3.10'" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/68/14/eb014d26be205d38ad5ad20d9a80f7d201472e08167f0bb4361e251084a9/pytest_mock-3.15.1.tar.gz", hash = "sha256:1849a238f6f396da19762269de72cb1814ab44416fa73a8686deac10b0d87a0f", size = 34036, upload-time = "2025-09-16T16:37:27.081Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/5a/cc/06253936f4a7fa2e0f48dfe6d851d9c56df896a9ab09ac019d70b760619c/pytest_mock-3.15.1-py3-none-any.whl", hash = "sha256:0a25e2eb88fe5168d535041d09a4529a188176ae608a6d249ee65abc0949630d", size = 10095, upload-time = "2025-09-16T16:37:25.734Z" }, +] + [[package]] name = "pyyaml" version = "6.0.3" @@ -871,6 +884,7 @@ dependencies = [ { name = "pytest", version = "8.4.2", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.10'" }, { name = "pytest", version = "9.1.1", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version >= '3.10'" }, { name = "pytest-bazel" }, + { name = "pytest-mock" }, { name = "readthedocs-sphinx-ext" }, { name = "sphinx", version = "7.4.7", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.10'" }, { name = "sphinx", version = "8.1.3", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version == '3.10.*'" }, @@ -893,6 +907,7 @@ requires-dist = [ { name = "pyelftools" }, { name = "pytest" }, { name = "pytest-bazel" }, + { name = "pytest-mock" }, { name = "readthedocs-sphinx-ext" }, { name = "sphinx" }, { name = "sphinx-autodoc2" }, diff --git a/tests/tools/private/release/BUILD.bazel b/tests/tools/private/release/BUILD.bazel index d49d8fd221..5ac912a955 100644 --- a/tests/tools/private/release/BUILD.bazel +++ b/tests/tools/private/release/BUILD.bazel @@ -1,149 +1,172 @@ -load("@rules_python//python:defs.bzl", "py_library", "py_test") +load("//python:py_library.bzl", "py_library") +load("//tests/support:support.bzl", "SUPPORTS_BZLMOD") +load("//tests/support/pytest_test:pytest_test.bzl", "pytest_test") py_library( name = "release_test_helper", srcs = ["release_test_helper.py"], + target_compatible_with = SUPPORTS_BZLMOD, deps = [ "//tools/private/release:mock_gh", "//tools/private/release:release_lib", + "@pypi//pytest_mock", ], ) -py_test( +pytest_test( name = "add_backports_test", srcs = ["add_backports_test.py"], + target_compatible_with = SUPPORTS_BZLMOD, deps = [ ":release_test_helper", "//tools/private/release:release_lib", ], ) -py_test( +pytest_test( name = "changelog_news_test", srcs = ["changelog_news_test.py"], + target_compatible_with = SUPPORTS_BZLMOD, deps = [ ":release_test_helper", "//tools/private/release:release_lib", ], ) -py_test( +pytest_test( name = "complete_sync_changelog_test", srcs = ["complete_sync_changelog_test.py"], + target_compatible_with = SUPPORTS_BZLMOD, deps = [ ":release_test_helper", "//tools/private/release:release_lib", ], ) -py_test( - name = "create_rc_test", - srcs = ["create_rc_test.py"], +pytest_test( + name = "create_release_branch_test", + srcs = ["create_release_branch_test.py"], + target_compatible_with = SUPPORTS_BZLMOD, deps = [ ":release_test_helper", "//tools/private/release:release_lib", ], ) -py_test( - name = "create_release_branch_test", - srcs = ["create_release_branch_test.py"], +pytest_test( + name = "git_test", + srcs = ["git_test.py"], + target_compatible_with = SUPPORTS_BZLMOD, deps = [ ":release_test_helper", "//tools/private/release:release_lib", ], ) -py_test( - name = "gh_test", - srcs = ["gh_test.py"], +pytest_test( + name = "on_pr_merged_test", + srcs = ["on_pr_merged_test.py"], + target_compatible_with = SUPPORTS_BZLMOD, deps = [ + ":release_test_helper", "//tools/private/release:release_lib", ], ) -py_test( - name = "git_test", - srcs = ["git_test.py"], +pytest_test( + name = "promote_test", + srcs = ["promote_test.py"], + target_compatible_with = SUPPORTS_BZLMOD, deps = [ + ":release_test_helper", "//tools/private/release:release_lib", ], ) -py_test( - name = "on_pr_merged_test", - srcs = ["on_pr_merged_test.py"], +pytest_test( + name = "release_issue_test", + srcs = ["release_issue_test.py"], + target_compatible_with = SUPPORTS_BZLMOD, deps = [ ":release_test_helper", "//tools/private/release:release_lib", ], ) -py_test( - name = "prepare_test", - srcs = ["prepare_test.py"], +pytest_test( + name = "release_test", + srcs = ["release_test.py"], + target_compatible_with = SUPPORTS_BZLMOD, deps = [ ":release_test_helper", "//tools/private/release:release_lib", ], ) -py_test( - name = "process_backports_test", - srcs = ["process_backports_test.py"], +pytest_test( + name = "backport_create_releases_test", + srcs = ["backport_create_releases_test.py"], + target_compatible_with = SUPPORTS_BZLMOD, deps = [ ":release_test_helper", "//tools/private/release:release_lib", ], ) -py_test( - name = "promote_test", - srcs = ["promote_test.py"], +pytest_test( + name = "prepare_test", + srcs = ["prepare_test.py"], + target_compatible_with = SUPPORTS_BZLMOD, deps = [ ":release_test_helper", "//tools/private/release:release_lib", ], ) -py_test( - name = "release_issue_test", - srcs = ["release_issue_test.py"], +pytest_test( + name = "gh_test", + srcs = ["gh_test.py"], + target_compatible_with = SUPPORTS_BZLMOD, deps = [ + ":release_test_helper", "//tools/private/release:release_lib", ], ) -py_test( - name = "release_test", - srcs = ["release_test.py"], +pytest_test( + name = "backport_prepare_test", + srcs = ["backport_prepare_test.py"], + target_compatible_with = SUPPORTS_BZLMOD, deps = [ + ":release_test_helper", "//tools/private/release:release_lib", ], ) -py_test( - name = "utils_test", - srcs = ["utils_test.py"], +pytest_test( + name = "process_backports_test", + srcs = ["process_backports_test.py"], + target_compatible_with = SUPPORTS_BZLMOD, deps = [ ":release_test_helper", "//tools/private/release:release_lib", - "@dev_pip//packaging", ], ) -py_test( - name = "backport_prepare_test", - srcs = ["backport_prepare_test.py"], +pytest_test( + name = "create_rc_test", + srcs = ["create_rc_test.py"], + target_compatible_with = SUPPORTS_BZLMOD, deps = [ ":release_test_helper", "//tools/private/release:release_lib", ], ) -py_test( - name = "backport_create_releases_test", - srcs = ["backport_create_releases_test.py"], +pytest_test( + name = "utils_test", + srcs = ["utils_test.py"], + target_compatible_with = SUPPORTS_BZLMOD, deps = [ ":release_test_helper", "//tools/private/release:release_lib", diff --git a/tests/tools/private/release/add_backports_test.py b/tests/tools/private/release/add_backports_test.py index 75729e87fe..c1c7b5cc0a 100644 --- a/tests/tools/private/release/add_backports_test.py +++ b/tests/tools/private/release/add_backports_test.py @@ -1,89 +1,85 @@ import argparse -import unittest -from unittest.mock import patch -from tests.tools.private.release.release_test_helper import _mock_git_and_gh from tools.private.release.add_backports import AddBackports +pytest_plugins = ["tests.tools.private.release.release_test_helper"] -class CmdAddBackportsTest(unittest.TestCase): - def setUp(self): - _mock_git_and_gh(self) - self.addCleanup(patch.stopall) - self.mock_gh.resolve_pr_number.side_effect = lambda x: int( - x.lstrip("#").split("/")[-1] - ) - def test_add_backports_explicit_issue(self): - args = argparse.Namespace(issue=123, prs=["124", "125"]) - self.mock_gh.get_issue_body.return_value = """ +def test_add_backports_explicit_issue(mock_gh): + args = argparse.Namespace(issue=123, prs=["124", "125"]) + mock_gh.issues[123] = { + "title": "Release 2.1.0", + "body": """ ## Checklist - [ ] Prepare Release - [ ] Create Release branch - [ ] Tag Final ## Backports -""" - result = AddBackports(args, self.mock_gh).run() - - self.assertEqual(result, 0) - self.mock_gh.get_issue_body.assert_called_once_with(123) - self.mock_gh.update_issue_body.assert_called_once() - call_args = self.mock_gh.update_issue_body.call_args[0] - self.assertEqual(call_args[0], 123) - self.assertIn("- [ ] #124", call_args[1]) - self.assertIn("- [ ] #125", call_args[1]) - # Should also auto-add Tag RC0 - self.assertIn("- [ ] Tag RC0", call_args[1]) - self.assertIn("- [ ] Sync Changelog #124", call_args[1]) - self.assertIn("- [ ] Sync Changelog #125", call_args[1]) - - def test_add_backports_auto_discover_success(self): - args = argparse.Namespace(issue=None, prs=["124"]) - self.mock_gh.get_open_tracking_issues.return_value = [ - {"number": 456, "title": "Release 2.1.0", "url": "http://..."} - ] - self.mock_gh.get_issue_body.return_value = """ +""", + "labels": ["type: release"], + "number": 123, + "url": "https://github.com/bazel-contrib/rules_python/issues/123", + } + result = AddBackports(args, mock_gh).run() + + assert result == 0 + updated_body = mock_gh.get_issue_body(123) + assert "- [ ] #124" in updated_body + assert "- [ ] #125" in updated_body + assert "- [ ] Tag RC0" in updated_body + assert "- [ ] Sync Changelog #124" in updated_body + assert "- [ ] Sync Changelog #125" in updated_body + + +def test_add_backports_auto_discover_success(mock_gh): + args = argparse.Namespace(issue=None, prs=["124"]) + issue_num = mock_gh.create_issue( + title="Release 2.1.0", + body=""" ## Checklist - [ ] Prepare Release - [ ] Create Release branch - [ ] Tag Final ## Backports -""" - result = AddBackports(args, self.mock_gh).run() +""", + labels=["type: release"], + ) + result = AddBackports(args, mock_gh).run() - self.assertEqual(result, 0) - self.mock_gh.get_open_tracking_issues.assert_called_once() - self.mock_gh.get_issue_body.assert_called_once_with(456) - self.mock_gh.update_issue_body.assert_called_once_with(456, unittest.mock.ANY) + assert result == 0 + updated_body = mock_gh.get_issue_body(issue_num) + assert "- [ ] #124" in updated_body - def test_add_backports_auto_discover_no_issues(self): - args = argparse.Namespace(issue=None, prs=["124"]) - self.mock_gh.get_open_tracking_issues.return_value = [] - result = AddBackports(args, self.mock_gh).run() +def test_add_backports_auto_discover_no_issues(mock_gh): + args = argparse.Namespace(issue=None, prs=["124"]) - self.assertEqual(result, 1) - self.mock_gh.get_open_tracking_issues.assert_called_once() - self.mock_gh.get_issue_body.assert_not_called() + result = AddBackports(args, mock_gh).run() - def test_add_backports_auto_discover_multiple_issues(self): - args = argparse.Namespace(issue=None, prs=["124"]) - self.mock_gh.get_open_tracking_issues.return_value = [ - {"number": 456, "title": "Release 2.1.0", "url": "http://..."}, - {"number": 789, "title": "Release 2.2.0", "url": "http://..."}, - ] + assert result == 1 - result = AddBackports(args, self.mock_gh).run() - self.assertEqual(result, 1) - self.mock_gh.get_open_tracking_issues.assert_called_once() - self.mock_gh.get_issue_body.assert_not_called() +def test_add_backports_auto_discover_multiple_issues(mock_gh): + args = argparse.Namespace(issue=None, prs=["124"]) + mock_gh.create_issue( + title="Release 2.1.0", body="## Backports\n", labels=["type: release"] + ) + mock_gh.create_issue( + title="Release 2.2.0", body="## Backports\n", labels=["type: release"] + ) - def test_add_backports_no_auto_add_rc_if_pending(self): - args = argparse.Namespace(issue=123, prs=["124"]) - self.mock_gh.get_issue_body.return_value = """ + result = AddBackports(args, mock_gh).run() + + assert result == 1 + + +def test_add_backports_no_auto_add_rc_if_pending(mock_gh): + args = argparse.Namespace(issue=123, prs=["124"]) + mock_gh.issues[123] = { + "title": "Release 2.1.0", + "body": """ ## Checklist - [ ] Prepare Release - [ ] Create Release branch @@ -91,17 +87,15 @@ def test_add_backports_no_auto_add_rc_if_pending(self): - [ ] Tag Final ## Backports -""" - result = AddBackports(args, self.mock_gh).run() - - self.assertEqual(result, 0) - self.mock_gh.update_issue_body.assert_called_once() - call_args = self.mock_gh.update_issue_body.call_args[0] - self.assertNotIn("Tag RC1", call_args[1]) - # Tag RC0 should still be there - self.assertIn("- [ ] Tag RC0", call_args[1]) - self.assertIn("- [ ] Sync Changelog #124", call_args[1]) - - -if __name__ == "__main__": - unittest.main() +""", + "labels": ["type: release"], + "number": 123, + "url": "https://github.com/bazel-contrib/rules_python/issues/123", + } + result = AddBackports(args, mock_gh).run() + + assert result == 0 + updated_body = mock_gh.get_issue_body(123) + assert "Tag RC1" not in updated_body + assert "- [ ] Tag RC0" in updated_body + assert "- [ ] Sync Changelog #124" in updated_body diff --git a/tests/tools/private/release/backport_create_releases_test.py b/tests/tools/private/release/backport_create_releases_test.py index f1bdfe477f..81dc7e00ce 100644 --- a/tests/tools/private/release/backport_create_releases_test.py +++ b/tests/tools/private/release/backport_create_releases_test.py @@ -1,18 +1,18 @@ import argparse -import unittest -from tests.tools.private.release.release_test_helper import ReleaseToolTestCase from tools.private.release.backport_create_releases import BackportCreateReleases +# Register pytest fixtures (such as release_tool_env) from release_test_helper +pytest_plugins = ["tests.tools.private.release.release_test_helper"] -class CmdBackportCreateReleasesTest(ReleaseToolTestCase): - def test_create_releases_all_success(self): - # Arrange - args = argparse.Namespace(issue=123, dry_run=False) - gh = self.gh - # Setup backport issue in mock GH - backport_body = """* PR: #456 +def test_create_releases_all_success(release_tool_env, mock_gh): + # Arrange + args = argparse.Namespace(issue=123, dry_run=False) + gh = mock_gh + + # Setup backport issue in mock GH + backport_body = """* PR: #456 * From version: 1.7 * To version: 1.9 @@ -25,36 +25,36 @@ def test_create_releases_all_success(self): - [ ] Track Release 1.8.1 - [ ] Track Release 1.9.0""" - gh.issues[123] = { - "title": "Backport: #456", - "body": backport_body, - "labels": ["type:backport-pr"], - "number": 123, - "url": "https://github.com/.../issues/123", - } + gh.issues[123] = { + "title": "Backport: #456", + "body": backport_body, + "labels": ["type:backport-pr"], + "number": 123, + "url": "https://github.com/.../issues/123", + } - # Act - result = BackportCreateReleases(args, gh).run() + # Act + result = BackportCreateReleases(args, gh).run() - # Assert - self.assertEqual(result, 0) + # Assert + assert result == 0 - # Verify release issues created (IDs 1001, 1002, 1003) - self.assertIn(1001, gh.issues) - self.assertIn(1002, gh.issues) - self.assertIn(1003, gh.issues) + # Verify release issues created (IDs 1001, 1002, 1003) + assert 1001 in gh.issues + assert 1002 in gh.issues + assert 1003 in gh.issues - # 1.7.2 (patch) should not have Tag RC0 - self.assertEqual(gh.issues[1001]["title"], "Release 1.7.2") - self.assertNotIn("Tag RC0", gh.issues[1001]["body"]) - self.assertIn("## Backports\n- [ ] #456", gh.issues[1001]["body"]) + # 1.7.2 (patch) should not have Tag RC0 + assert gh.issues[1001]["title"] == "Release 1.7.2" + assert "Tag RC0" not in gh.issues[1001]["body"] + assert "## Backports\n- [ ] #456" in gh.issues[1001]["body"] - # 1.9.0 (minor) should have Tag RC0 - self.assertEqual(gh.issues[1003]["title"], "Release 1.9.0") - self.assertIn("Tag RC0", gh.issues[1003]["body"]) + # 1.9.0 (minor) should have Tag RC0 + assert gh.issues[1003]["title"] == "Release 1.9.0" + assert "Tag RC0" in gh.issues[1003]["body"] - # Verify backport issue updated - expected_updated_backport_body = """* PR: #456 + # Verify backport issue updated + expected_updated_backport_body = """* PR: #456 * From version: 1.7 * To version: 1.9 @@ -67,15 +67,16 @@ def test_create_releases_all_success(self): - [x] Track Release 1.8.1 | status=success release_issue=#1002 - [x] Track Release 1.9.0 | status=success release_issue=#1003""" - self.assertEqual(gh.issues[123]["body"], expected_updated_backport_body) + assert gh.issues[123]["body"] == expected_updated_backport_body + - def test_create_releases_dependency_blocking(self): - # Arrange - args = argparse.Namespace(issue=123, dry_run=False) - gh = self.gh +def test_create_releases_dependency_blocking(release_tool_env, mock_gh): + # Arrange + args = argparse.Namespace(issue=123, dry_run=False) + gh = mock_gh - # 1.8 failed, 1.7 and 1.9 succeeded - backport_body = """* PR: #456 + # 1.8 failed, 1.7 and 1.9 succeeded + backport_body = """* PR: #456 * From version: 1.7 * To version: 1.9 @@ -88,27 +89,27 @@ def test_create_releases_dependency_blocking(self): - [ ] Track Release 1.8.1 - [ ] Track Release 1.9.0""" - gh.issues[123] = { - "title": "Backport: #456", - "body": backport_body, - "labels": ["type:backport-pr"], - "number": 123, - "url": "https://github.com/.../issues/123", - } + gh.issues[123] = { + "title": "Backport: #456", + "body": backport_body, + "labels": ["type:backport-pr"], + "number": 123, + "url": "https://github.com/.../issues/123", + } - # Act - result = BackportCreateReleases(args, gh).run() + # Act + result = BackportCreateReleases(args, gh).run() - # Assert - self.assertEqual(result, 0) + # Assert + assert result == 0 - # Only 1.9.0 (ID 1001) should be created. 1.7.2 and 1.8.1 are blocked by 1.8 failure. - self.assertEqual(len(gh.issues), 2) # Backport issue + 1 release issue - self.assertIn(1001, gh.issues) - self.assertEqual(gh.issues[1001]["title"], "Release 1.9.0") + # Only 1.9.0 (ID 1001) should be created. 1.7.2 and 1.8.1 are blocked by 1.8 failure. + assert len(gh.issues) == 2 # Backport issue + 1 release issue + assert 1001 in gh.issues + assert gh.issues[1001]["title"] == "Release 1.9.0" - # Verify backport issue updated with block statuses - expected_updated_backport_body = """* PR: #456 + # Verify backport issue updated with block statuses + expected_updated_backport_body = """* PR: #456 * From version: 1.7 * To version: 1.9 @@ -121,14 +122,15 @@ def test_create_releases_dependency_blocking(self): - [ ] Track Release 1.8.1 | status=error-later-release-did-not-apply - [x] Track Release 1.9.0 | status=success release_issue=#1001""" - self.assertEqual(gh.issues[123]["body"], expected_updated_backport_body) + assert gh.issues[123]["body"] == expected_updated_backport_body + - def test_create_releases_already_exists(self): - # Arrange - args = argparse.Namespace(issue=123, dry_run=False) - gh = self.gh +def test_create_releases_already_exists(release_tool_env, mock_gh): + # Arrange + args = argparse.Namespace(issue=123, dry_run=False) + gh = mock_gh - backport_body = """* PR: #456 + backport_body = """* PR: #456 * From version: 1.7 * To version: 1.7 @@ -137,30 +139,30 @@ def test_create_releases_already_exists(self): - [x] Verify apply 1.7 | status=success - [ ] Track Release 1.7.2""" - gh.issues[123] = { - "title": "Backport: #456", - "body": backport_body, - "labels": ["type:backport-pr"], - "number": 123, - "url": "https://github.com/.../issues/123", - } - - # Pre-create the release issue to simulate it already existing - gh.create_issue( - title="Release 1.7.2", - body="existing body\n## Backports\n", - labels=["type: release"], - ) # Will get ID 1001 - - # Act - result = BackportCreateReleases(args, gh).run() - - # Assert - self.assertEqual(result, 0) - self.assertEqual(len(gh.issues), 2) - - # Should link existing issue 1001 - expected_updated_backport_body = """* PR: #456 + gh.issues[123] = { + "title": "Backport: #456", + "body": backport_body, + "labels": ["type:backport-pr"], + "number": 123, + "url": "https://github.com/.../issues/123", + } + + # Pre-create the release issue to simulate it already existing + gh.create_issue( + title="Release 1.7.2", + body="existing body\n## Backports\n", + labels=["type: release"], + ) # Will get ID 1001 + + # Act + result = BackportCreateReleases(args, gh).run() + + # Assert + assert result == 0 + assert len(gh.issues) == 2 + + # Should link existing issue 1001 + expected_updated_backport_body = """* PR: #456 * From version: 1.7 * To version: 1.7 @@ -169,8 +171,4 @@ def test_create_releases_already_exists(self): - [x] Verify apply 1.7 | status=success - [x] Track Release 1.7.2 | status=success release_issue=#1001""" - self.assertEqual(gh.issues[123]["body"], expected_updated_backport_body) - - -if __name__ == "__main__": - unittest.main() + assert gh.issues[123]["body"] == expected_updated_backport_body diff --git a/tests/tools/private/release/backport_prepare_test.py b/tests/tools/private/release/backport_prepare_test.py index 26883b78b3..ddef4885b9 100644 --- a/tests/tools/private/release/backport_prepare_test.py +++ b/tests/tools/private/release/backport_prepare_test.py @@ -1,247 +1,238 @@ import argparse -import unittest -from unittest.mock import call, patch +from unittest.mock import ANY, call -from tests.tools.private.release.release_test_helper import ( - ReleaseToolTestCase, - _mock_git, -) from tools.private.release.backport_prepare import BackportPrepare from tools.private.release.gh import BACKPORT_LABEL - -class CmdBackportPrepareTest(ReleaseToolTestCase): - def setUp(self): - super().setUp() - _mock_git(self) - # Mock changelog_news and determine_next_version - self.patcher_news = patch( - "tools.private.release.backport_prepare.changelog_news" - ) - self.mock_news = self.patcher_news.start() - - self.patcher_det = patch( - "tools.private.release.backport_prepare.determine_next_version" - ) - self.mock_det = self.patcher_det.start() - - self.addCleanup(self.patcher_news.stop) - self.addCleanup(self.patcher_det.stop) - - def test_prepare_from_issue_success(self): - # Arrange - args = argparse.Namespace( - issue=123, - pr=None, - from_minor=None, - to_minor=None, - remote="my-remote", - dry_run=False, - ) - - # Setup backport issue in mock GH - backport_body = "* PR: #456\n* From version: 1.7\n* To version: 1.9\n" - self.gh.issues[123] = { - "title": "Backport: #456", - "body": backport_body, - "labels": ["type: backport-pr"], - "number": 123, - "url": "https://github.com/.../issues/123", - } - - # Setup PR info in mock GH - self.gh.prs[456] = { - "state": "MERGED", - "mergeCommit": {"oid": "pr_merge_sha_12345"}, - } - - # Mock remote branches - self.mock_git.get_remote_branches.return_value = [ - "main", - "release/1.6", - "release/1.7", - "release/1.8", - "release/1.9", - "release/2.0", - ] - self.mock_git.get_current_branch.return_value = "work-branch" - - # Mock next versions - self.mock_det.side_effect = ["1.7.2", "1.8.1", "1.9.0"] - - # Act - result = BackportPrepare(args, self.mock_git, self.gh).run() - - # Assert - self.assertEqual(result, 0) - self.mock_git.fetch.assert_called_once_with("my-remote", tags=True, force=True) - - # Verify checkouts and cherry-picks - self.mock_git.checkout.assert_has_calls( - [ - call("release/1.7", track_remote="my-remote"), - call("release/1.8", track_remote="my-remote"), - call("release/1.9", track_remote="my-remote"), - call("work-branch"), # Restored branch - ] - ) - - self.mock_git.cherry_pick.assert_has_calls( - [ - call("pr_merge_sha_12345"), - call("pr_merge_sha_12345"), - call("pr_merge_sha_12345"), - ] - ) - - # Verify changelog updates - self.mock_news.update_changelog.assert_has_calls( - [ - call("1.7.2", unittest.mock.ANY), - call("1.8.1", unittest.mock.ANY), - call("1.9.0", unittest.mock.ANY), - ] - ) - - # Verify issue body update - expected_body = ( - "* PR: #456\n" - "* From version: 1.7\n" - "* To version: 1.9\n" - "\n" - "## Tasks\n" - "\n" - "- [x] Verify apply 1.7 | status=success\n" - "- [x] Verify apply 1.8 | status=success\n" - "- [x] Verify apply 1.9 | status=success\n" - "- [ ] Track Release 1.7.2\n" - "- [ ] Track Release 1.8.1\n" - "- [ ] Track Release 1.9.0" - ) - self.assertEqual(self.gh.issues[123]["body"], expected_body) - - def test_prepare_manual_success(self): - # Arrange - args = argparse.Namespace( - issue=None, - pr="#456", - from_minor="1.7", - to_minor="1.8", - remote="my-remote", - dry_run=False, - ) - self.gh.prs[456] = { - "state": "MERGED", - "mergeCommit": {"oid": "pr_merge_sha_12345"}, - } - self.mock_git.get_remote_branches.return_value = [ - "release/1.7", - "release/1.8", +pytest_plugins = ["tests.tools.private.release.release_test_helper"] + + +def test_prepare_from_issue_success(mocker, mock_git, mock_gh): + # Arrange + args = argparse.Namespace( + issue=123, + pr=None, + from_minor=None, + to_minor=None, + remote="my-remote", + dry_run=False, + ) + + # Setup backport issue in mock GH + backport_body = "* PR: #456\n* From version: 1.7\n* To version: 1.9\n" + mock_gh.issues[123] = { + "title": "Backport: #456", + "body": backport_body, + "labels": ["type: backport-pr"], + "number": 123, + "url": "https://github.com/.../issues/123", + } + + # Setup PR info in mock GH + mock_gh.prs[456] = { + "state": "MERGED", + "mergeCommit": {"oid": "pr_merge_sha_12345"}, + } + + # Mock remote branches + mock_git.get_remote_branches.return_value = [ + "main", + "release/1.6", + "release/1.7", + "release/1.8", + "release/1.9", + "release/2.0", + ] + mock_git.get_current_branch.return_value = "work-branch" + + mock_news = mocker.patch("tools.private.release.backport_prepare.changelog_news") + mock_det = mocker.patch( + "tools.private.release.backport_prepare.determine_next_version" + ) + mock_det.side_effect = ["1.7.2", "1.8.1", "1.9.0"] + + # Act + result = BackportPrepare(args, mock_git, mock_gh).run() + + # Assert + assert result == 0 + mock_git.fetch.assert_called_once_with("my-remote", tags=True, force=True) + + # Verify checkouts and cherry-picks + mock_git.checkout.assert_has_calls( + [ + call("release/1.7", track_remote="my-remote"), + call("release/1.8", track_remote="my-remote"), + call("release/1.9", track_remote="my-remote"), + call("work-branch"), # Restored branch ] - self.mock_git.get_current_branch.return_value = "work-branch" - self.mock_det.side_effect = ["1.7.2", "1.8.1"] - - # Act - result = BackportPrepare(args, self.mock_git, self.gh).run() - - # Assert - self.assertEqual(result, 0) - self.assertIn(1001, self.gh.issues) - issue = self.gh.issues[1001] - self.assertEqual(issue["title"], "Backport: #456") - self.assertEqual(issue["labels"], [BACKPORT_LABEL]) - body = issue["body"] - self.assertIn("- [x] Verify apply 1.7 | status=success", body) - self.assertIn("- [x] Verify apply 1.8 | status=success", body) - self.assertIn("- [ ] Track Release 1.7.2", body) - self.assertIn("- [ ] Track Release 1.8.1", body) - - def test_prepare_manual_with_patch_versions(self): - # Arrange - args = argparse.Namespace( - issue=None, - pr="#456", - from_minor="1.7.0", - to_minor="1.8.0", - remote="my-remote", - dry_run=False, - ) - self.gh.prs[456] = { - "state": "MERGED", - "mergeCommit": {"oid": "pr_merge_sha_12345"}, - } - self.mock_git.get_remote_branches.return_value = [ - "release/1.7", - "release/1.8", + ) + + mock_git.cherry_pick.assert_has_calls( + [ + call("pr_merge_sha_12345"), + call("pr_merge_sha_12345"), + call("pr_merge_sha_12345"), ] - self.mock_git.get_current_branch.return_value = "work-branch" - self.mock_det.side_effect = ["1.7.2", "1.8.1"] - - # Act - result = BackportPrepare(args, self.mock_git, self.gh).run() - - # Assert - self.assertEqual(result, 0) - self.assertIn(1001, self.gh.issues) - body = self.gh.issues[1001]["body"] - self.assertIn("- [x] Verify apply 1.7 | status=success", body) - self.assertIn("- [x] Verify apply 1.8 | status=success", body) - - def test_prepare_verify_failed(self): - # Arrange - args = argparse.Namespace( - issue=123, - pr=None, - from_minor=None, - to_minor=None, - remote="my-remote", - dry_run=False, - ) - issue_body = "* PR: #456\n* From version: 1.7\n* To version: 1.8\n" - self.gh.issues[123] = { - "title": "Backport: #456", - "body": issue_body, - "labels": ["type: backport-pr"], - "number": 123, - "url": "https://github.com/.../issues/123", - } - self.gh.prs[456] = { - "state": "MERGED", - "mergeCommit": {"oid": "pr_merge_sha_12345"}, - } - self.mock_git.get_remote_branches.return_value = [ - "release/1.7", - "release/1.8", + ) + + # Verify changelog updates + mock_news.update_changelog.assert_has_calls( + [ + call("1.7.2", ANY), + call("1.8.1", ANY), + call("1.9.0", ANY), ] - self.mock_git.get_current_branch.return_value = "work-branch" - self.mock_det.side_effect = ["1.7.2", "1.8.1"] - - # Mock cherry-pick failure on 1.7 and changelog failure on 1.8 - self.mock_git.cherry_pick.side_effect = [Exception("Conflict"), None] - self.mock_news.update_changelog.side_effect = [Exception("Changelog error")] - - # Act - result = BackportPrepare(args, self.mock_git, self.gh).run() - - # Assert - self.assertEqual( - result, 0 - ) # Returns 0 even if verification fails, it just updates tasks - - expected_body = ( - "* PR: #456\n" - "* From version: 1.7\n" - "* To version: 1.8\n" - "\n" - "## Tasks\n" - "\n" - "- [ ] Verify apply 1.7 | status=failed-conflict\n" - "- [ ] Verify apply 1.8 | status=failed-changelog\n" - "- [ ] Track Release 1.7.2\n" - "- [ ] Track Release 1.8.1" - ) - self.assertEqual(self.gh.issues[123]["body"], expected_body) - - -if __name__ == "__main__": - unittest.main() + ) + + # Verify issue body update + expected_body = ( + "* PR: #456\n" + "* From version: 1.7\n" + "* To version: 1.9\n" + "\n" + "## Tasks\n" + "\n" + "- [x] Verify apply 1.7 | status=success\n" + "- [x] Verify apply 1.8 | status=success\n" + "- [x] Verify apply 1.9 | status=success\n" + "- [ ] Track Release 1.7.2\n" + "- [ ] Track Release 1.8.1\n" + "- [ ] Track Release 1.9.0" + ) + assert mock_gh.issues[123]["body"] == expected_body + + +def test_prepare_manual_success(mocker, mock_git, mock_gh): + # Arrange + args = argparse.Namespace( + issue=None, + pr="#456", + from_minor="1.7", + to_minor="1.8", + remote="my-remote", + dry_run=False, + ) + mock_gh.prs[456] = { + "state": "MERGED", + "mergeCommit": {"oid": "pr_merge_sha_12345"}, + } + mock_git.get_remote_branches.return_value = [ + "release/1.7", + "release/1.8", + ] + mock_git.get_current_branch.return_value = "work-branch" + + mocker.patch("tools.private.release.backport_prepare.changelog_news") + mock_det = mocker.patch( + "tools.private.release.backport_prepare.determine_next_version" + ) + mock_det.side_effect = ["1.7.2", "1.8.1"] + + # Act + result = BackportPrepare(args, mock_git, mock_gh).run() + + # Assert + assert result == 0 + assert 1001 in mock_gh.issues + issue = mock_gh.issues[1001] + assert issue["title"] == "Backport: #456" + assert issue["labels"] == [BACKPORT_LABEL] + body = issue["body"] + assert "- [x] Verify apply 1.7 | status=success" in body + assert "- [x] Verify apply 1.8 | status=success" in body + assert "- [ ] Track Release 1.7.2" in body + assert "- [ ] Track Release 1.8.1" in body + + +def test_prepare_manual_with_patch_versions(mocker, mock_git, mock_gh): + # Arrange + args = argparse.Namespace( + issue=None, + pr="#456", + from_minor="1.7.0", + to_minor="1.8.0", + remote="my-remote", + dry_run=False, + ) + mock_gh.prs[456] = { + "state": "MERGED", + "mergeCommit": {"oid": "pr_merge_sha_12345"}, + } + mock_git.get_remote_branches.return_value = [ + "release/1.7", + "release/1.8", + ] + mock_git.get_current_branch.return_value = "work-branch" + + mocker.patch("tools.private.release.backport_prepare.changelog_news") + mock_det = mocker.patch( + "tools.private.release.backport_prepare.determine_next_version" + ) + mock_det.side_effect = ["1.7.2", "1.8.1"] + + # Act + result = BackportPrepare(args, mock_git, mock_gh).run() + + # Assert + assert result == 0 + assert 1001 in mock_gh.issues + body = mock_gh.issues[1001]["body"] + assert "- [x] Verify apply 1.7 | status=success" in body + assert "- [x] Verify apply 1.8 | status=success" in body + + +def test_prepare_verify_failed(mocker, mock_git, mock_gh): + # Arrange + args = argparse.Namespace( + issue=123, + pr=None, + from_minor=None, + to_minor=None, + remote="my-remote", + dry_run=False, + ) + issue_body = "* PR: #456\n* From version: 1.7\n* To version: 1.8\n" + mock_gh.issues[123] = { + "title": "Backport: #456", + "body": issue_body, + "labels": ["type: backport-pr"], + "number": 123, + "url": "https://github.com/.../issues/123", + } + mock_gh.prs[456] = { + "state": "MERGED", + "mergeCommit": {"oid": "pr_merge_sha_12345"}, + } + mock_git.get_remote_branches.return_value = [ + "release/1.7", + "release/1.8", + ] + mock_git.get_current_branch.return_value = "work-branch" + + mock_news = mocker.patch("tools.private.release.backport_prepare.changelog_news") + mock_det = mocker.patch( + "tools.private.release.backport_prepare.determine_next_version" + ) + mock_det.side_effect = ["1.7.2", "1.8.1"] + mock_git.cherry_pick.side_effect = [Exception("Conflict"), None] + mock_news.update_changelog.side_effect = [Exception("Changelog error")] + + # Act + result = BackportPrepare(args, mock_git, mock_gh).run() + + # Assert + assert result == 0 + expected_body = ( + "* PR: #456\n" + "* From version: 1.7\n" + "* To version: 1.8\n" + "\n" + "## Tasks\n" + "\n" + "- [ ] Verify apply 1.7 | status=failed-conflict\n" + "- [ ] Verify apply 1.8 | status=failed-changelog\n" + "- [ ] Track Release 1.7.2\n" + "- [ ] Track Release 1.8.1" + ) + assert mock_gh.issues[123]["body"] == expected_body diff --git a/tests/tools/private/release/changelog_news_test.py b/tests/tools/private/release/changelog_news_test.py index 83ab2cc959..38bba1eecf 100644 --- a/tests/tools/private/release/changelog_news_test.py +++ b/tests/tools/private/release/changelog_news_test.py @@ -1,15 +1,13 @@ import pathlib -import unittest -from unittest.mock import patch -from tests.tools.private.release.release_test_helper import TempDirTestCase +import pytest + from tools.private.release import changelog_news -class ChangelogNewsTest(TempDirTestCase): - def test_update_changelog_with_news(self): - # Arrange - changelog = """# Changelog +def test_update_changelog_with_news(tmp_path): + # Arrange + changelog = """# Changelog {#unreleased} ## Unreleased @@ -29,78 +27,76 @@ def test_update_changelog_with_news(self): ### Added * (toolchains) Some older change. """ - changelog_path = self.tmpdir / "CHANGELOG.md" - changelog_path.write_text(changelog) - - news_dir = self.tmpdir / "news" - news_dir.mkdir() - - # Create news files - (news_dir / "123.fixed.md").write_text("Fixed a bug in the compiler") - # Test that it handles prefixing "* " if not present - (news_dir / "456.added.md").write_text("* Added a new feature for Python 3.13") - # Empty file should be ignored - (news_dir / "789.changed.md").write_text("") - # Invalid name should be ignored - (news_dir / "invalid_name.md").write_text("Should be ignored") - - # Act - changelog_news.update_changelog( - "3.0.0", - "2026-06-16", - changelog_path=changelog_path, - news_dir=news_dir, - ) - - # Assert - # 1. News files matching the pattern should be deleted (even empty ones) - self.assertFalse((news_dir / "123.fixed.md").exists()) - self.assertFalse((news_dir / "456.added.md").exists()) - self.assertFalse((news_dir / "789.changed.md").exists()) - # Invalid name does not match pattern -> NOT deleted - self.assertTrue((news_dir / "invalid_name.md").exists()) - - new_content = changelog_path.read_text() - - # 2. A fresh active Unreleased section should be present - self.assertIn("{#unreleased}", new_content) - self.assertIn("## Unreleased", new_content) - self.assertIn( - "Unreleased changes are tracked as individual files in the [news/](./news)\n" - "directory, or view the [latest generated\n" - "changelog](https://rules-python.readthedocs.io/en/latest/changelog.html).", - new_content, - ) - - # 3. The new release section should be present - self.assertIn("{#v3-0-0}", new_content) - self.assertIn("## [3.0.0] - 2026-06-16", new_content) - self.assertIn( - "[3.0.0]: https://github.com/bazel-contrib/rules_python/releases/tag/3.0.0", - new_content, - ) - - # 4. Correct categories and content - self.assertIn( - "{#v3-0-0-fixed}\n### Fixed\n* Fixed a bug in the compiler", - new_content, - ) - self.assertIn( - "{#v3-0-0-added}\n### Added\n* Added a new feature for Python 3.13", - new_content, - ) - - # 5. Omitted categories should NOT be present in the new release - self.assertNotIn("{#v3-0-0-removed}", new_content) - self.assertNotIn("{#v3-0-0-changed}", new_content) - - # 6. Old release should still be there - self.assertIn("{#v2-0-2}", new_content) - self.assertIn("## [2.0.2] - 2026-05-14", new_content) - - def test_update_changelog_sorting(self): - # Arrange - changelog = """# Changelog + changelog_path = tmp_path / "CHANGELOG.md" + changelog_path.write_text(changelog) + + news_dir = tmp_path / "news" + news_dir.mkdir() + + # Create news files + (news_dir / "123.fixed.md").write_text("Fixed a bug in the compiler") + # Test that it handles prefixing "* " if not present + (news_dir / "456.added.md").write_text("* Added a new feature for Python 3.13") + # Empty file should be ignored + (news_dir / "789.changed.md").write_text("") + # Invalid name should be ignored + (news_dir / "invalid_name.md").write_text("Should be ignored") + + # Act + changelog_news.update_changelog( + "3.0.0", + "2026-06-16", + changelog_path=changelog_path, + news_dir=news_dir, + ) + + # Assert + # 1. News files matching the pattern should be deleted (even empty ones) + assert not (news_dir / "123.fixed.md").exists() + assert not (news_dir / "456.added.md").exists() + assert not (news_dir / "789.changed.md").exists() + # Invalid name does not match pattern -> NOT deleted + assert (news_dir / "invalid_name.md").exists() + + new_content = changelog_path.read_text() + + # 2. A fresh active Unreleased section should be present + assert "{#unreleased}" in new_content + assert "## Unreleased" in new_content + assert ( + "Unreleased changes are tracked as individual files in the [news/](./news)\n" + "directory, or view the [latest generated\n" + "changelog](https://rules-python.readthedocs.io/en/latest/changelog.html)." + in new_content + ) + + # 3. The new release section should be present + assert "{#v3-0-0}" in new_content + assert "## [3.0.0] - 2026-06-16" in new_content + assert ( + "[3.0.0]: https://github.com/bazel-contrib/rules_python/releases/tag/3.0.0" + in new_content + ) + + # 4. Correct categories and content + assert "{#v3-0-0-fixed}\n### Fixed\n* Fixed a bug in the compiler" in new_content + assert ( + "{#v3-0-0-added}\n### Added\n* Added a new feature for Python 3.13" + in new_content + ) + + # 5. Omitted categories should NOT be present in the new release + assert "{#v3-0-0-removed}" not in new_content + assert "{#v3-0-0-changed}" not in new_content + + # 6. Old release should still be there + assert "{#v2-0-2}" in new_content + assert "## [2.0.2] - 2026-05-14" in new_content + + +def test_update_changelog_sorting(tmp_path): + # Arrange + changelog = """# Changelog {#unreleased} ## Unreleased @@ -120,62 +116,59 @@ def test_update_changelog_sorting(self): ### Added * (toolchains) Some older change. """ - changelog_path = self.tmpdir / "CHANGELOG.md" - changelog_path.write_text(changelog) + changelog_path = tmp_path / "CHANGELOG.md" + changelog_path.write_text(changelog) - news_dir = self.tmpdir / "news" - news_dir.mkdir() + news_dir = tmp_path / "news" + news_dir.mkdir() - # Create news files with different sub-categories and some without - (news_dir / "1.fixed.md").write_text("* (zebra) Zebra fix") - (news_dir / "2.fixed.md").write_text("* (apple) Apple fix") - (news_dir / "3.fixed.md").write_text("No subcategory B") - (news_dir / "4.fixed.md").write_text("* (apple) Another apple fix") - (news_dir / "5.fixed.md").write_text("No subcategory A") + # Create news files with different sub-categories and some without + (news_dir / "1.fixed.md").write_text("* (zebra) Zebra fix") + (news_dir / "2.fixed.md").write_text("* (apple) Apple fix") + (news_dir / "3.fixed.md").write_text("No subcategory B") + (news_dir / "4.fixed.md").write_text("* (apple) Another apple fix") + (news_dir / "5.fixed.md").write_text("No subcategory A") - # Act - changelog_news.update_changelog( - "3.0.0", - "2026-06-16", - changelog_path=changelog_path, - news_dir=news_dir, - ) + # Act + changelog_news.update_changelog( + "3.0.0", + "2026-06-16", + changelog_path=changelog_path, + news_dir=news_dir, + ) + + # Assert + new_content = changelog_path.read_text() + + expected_fixed_section = ( + "### Fixed\n" + "* No subcategory A\n" + "* No subcategory B\n" + "* (apple) Another apple fix\n" + "* (apple) Apple fix\n" + "* (zebra) Zebra fix\n" + ) + + assert expected_fixed_section in new_content - # Assert - new_content = changelog_path.read_text() - - # Expected order in Fixed section: - # 1. No subcategory A - # 2. No subcategory B - # 3. (apple) Another apple fix - # 4. (apple) Apple fix - # 5. (zebra) Zebra fix - - expected_fixed_section = ( - "### Fixed\n" - "* No subcategory A\n" - "* No subcategory B\n" - "* (apple) Another apple fix\n" - "* (apple) Apple fix\n" - "* (zebra) Zebra fix\n" - ) - self.assertIn(expected_fixed_section, new_content) +pytest_plugins = ["tests.tools.private.release.release_test_helper"] - def test_update_changelog_read_failure(self): - # Arrange - original_read_text = pathlib.Path.read_text - with patch("pathlib.Path.read_text", autospec=True) as mock_read_text: +def test_update_changelog_read_failure(mocker, tmp_path): + # Arrange + original_read_text = pathlib.Path.read_text - def side_effect(path_self, *args, **kwargs): - if "bad_file.fixed.md" in str(path_self): - raise IOError("Simulated read error") - return original_read_text(path_self, *args, **kwargs) + mock_read_text = mocker.patch.object(pathlib.Path, "read_text", autospec=True) - mock_read_text.side_effect = side_effect + def side_effect(path_self, *args, **kwargs): + if "bad_file.fixed.md" in str(path_self): + raise IOError("Simulated read error") + return original_read_text(path_self, *args, **kwargs) - changelog = """# Changelog + mock_read_text.side_effect = side_effect + + changelog = """# Changelog {#unreleased} ## Unreleased @@ -195,41 +188,41 @@ def side_effect(path_self, *args, **kwargs): ### Added * (toolchains) Some older change. """ - changelog_path = self.tmpdir / "CHANGELOG.md" - changelog_path.write_text(changelog) - - news_dir = self.tmpdir / "news" - news_dir.mkdir() - - # Create the bad file (must exist so it is found by iterdir) - bad_file = news_dir / "bad_file.fixed.md" - bad_file.write_text("some content that won't be read") - - # Create a good file too - good_file = news_dir / "good_file.fixed.md" - good_file.write_text("* (sub) Good fix") - - # Act & Assert - # It should raise IOError - with self.assertRaises(IOError): - changelog_news.update_changelog( - "3.0.0", - "2026-06-16", - changelog_path=changelog_path, - news_dir=news_dir, - ) - - # Both files should still exist (no deletion on failure!) - self.assertTrue(bad_file.exists()) - self.assertTrue(good_file.exists()) - - # Changelog should not be modified - new_content = changelog_path.read_text() - self.assertEqual(changelog, new_content) - - def test_update_changelog_merge_existing(self): - # Arrange - changelog = """# Changelog + changelog_path = tmp_path / "CHANGELOG.md" + changelog_path.write_text(changelog) + + news_dir = tmp_path / "news" + news_dir.mkdir() + + # Create the bad file (must exist so it is found by iterdir) + bad_file = news_dir / "bad_file.fixed.md" + bad_file.write_text("some content that won't be read") + + # Create a good file too + good_file = news_dir / "good_file.fixed.md" + good_file.write_text("* (sub) Good fix") + + # Act & Assert + with pytest.raises(IOError): + changelog_news.update_changelog( + "3.0.0", + "2026-06-16", + changelog_path=changelog_path, + news_dir=news_dir, + ) + + # Both files should still exist (no deletion on failure!) + assert bad_file.exists() + assert good_file.exists() + + # Changelog should not be modified + new_content = changelog_path.read_text() + assert changelog == new_content + + +def test_update_changelog_merge_existing(tmp_path): + # Arrange + changelog = """# Changelog {#unreleased} ## Unreleased @@ -252,57 +245,48 @@ def test_update_changelog_merge_existing(self): * nested bullet item * (pypi) Z old fix """ - changelog_path = self.tmpdir / "CHANGELOG.md" - changelog_path.write_text(changelog) + changelog_path = tmp_path / "CHANGELOG.md" + changelog_path.write_text(changelog) - news_dir = self.tmpdir / "news" - news_dir.mkdir() + news_dir = tmp_path / "news" + news_dir.mkdir() - # Create news files to merge - # 1. New fix in same category (should merge and sort) - (news_dir / "1.fixed.md").write_text("(pypi) New fix") - # 2. New entry in new category (should create category) - (news_dir / "2.added.md").write_text("(toolchains) New feature") + # Create news files to merge + (news_dir / "1.fixed.md").write_text("(pypi) New fix") + (news_dir / "2.added.md").write_text("(toolchains) New feature") - # Act - changelog_news.update_changelog( - "2.0.3", - "2026-06-15", - changelog_path=changelog_path, - news_dir=news_dir, - ) + # Act + changelog_news.update_changelog( + "2.0.3", + "2026-06-15", + changelog_path=changelog_path, + news_dir=news_dir, + ) - # Assert - # News files should be deleted - self.assertFalse((news_dir / "1.fixed.md").exists()) - self.assertFalse((news_dir / "2.added.md").exists()) - - new_content = changelog_path.read_text() - - # Expected merged and sorted Fixed section: - # 1. (pypi) New fix (New < Old) - # 2. (pypi) Old fix (with its multi-line detail!) - # 3. (pypi) Z old fix - expected_fixed_section = ( - "### Fixed\n" - "* (pypi) New fix\n" - "* (pypi) Old fix\n" - " multi-line detail\n" - " * nested bullet item\n" - "* (pypi) Z old fix\n" - ) - self.assertIn(expected_fixed_section, new_content) + # Assert + assert not (news_dir / "1.fixed.md").exists() + assert not (news_dir / "2.added.md").exists() + + new_content = changelog_path.read_text() - # Expected created Added section: - expected_added_section = "### Added\n* (toolchains) New feature\n" - self.assertIn(expected_added_section, new_content) + expected_fixed_section = ( + "### Fixed\n" + "* (pypi) New fix\n" + "* (pypi) Old fix\n" + " multi-line detail\n" + " * nested bullet item\n" + "* (pypi) Z old fix\n" + ) + assert expected_fixed_section in new_content - # Active Unreleased section should NOT be touched (should still be empty/pointing to news) - self.assertIn("Unreleased changes are tracked as individual files", new_content) + expected_added_section = "### Added\n* (toolchains) New feature\n" + assert expected_added_section in new_content + assert "Unreleased changes are tracked as individual files" in new_content - def test_update_changelog_does_not_leak(self): - # Arrange - changelog = """# Changelog + +def test_update_changelog_does_not_leak(tmp_path): + # Arrange + changelog = """# Changelog {#unreleased} ## Unreleased @@ -320,33 +304,32 @@ def test_update_changelog_does_not_leak(self): This release body mentions the word unreleased and {#unreleased} anchor to test leaks. """ - changelog_path = self.tmpdir / "CHANGELOG.md" - changelog_path.write_text(changelog) + changelog_path = tmp_path / "CHANGELOG.md" + changelog_path.write_text(changelog) - news_dir = self.tmpdir / "news" - news_dir.mkdir() - (news_dir / "1.fixed.md").write_text("Some fix") + news_dir = tmp_path / "news" + news_dir.mkdir() + (news_dir / "1.fixed.md").write_text("Some fix") - # Act - changelog_news.update_changelog( - "3.0.0", - "2026-06-16", - changelog_path=changelog_path, - news_dir=news_dir, - ) + # Act + changelog_news.update_changelog( + "3.0.0", + "2026-06-16", + changelog_path=changelog_path, + news_dir=news_dir, + ) - # Assert - new_content = changelog_path.read_text() + # Assert + new_content = changelog_path.read_text() + assert ( + "This release body mentions the word unreleased and {#unreleased} anchor to test leaks." + in new_content + ) - # The 2.0.2 body should NOT be modified - self.assertIn( - "This release body mentions the word unreleased and {#unreleased} anchor to test leaks.", - new_content, - ) - def test_update_changelog_empty_news(self): - # Arrange - changelog = """# Changelog +def test_update_changelog_empty_news(tmp_path): + # Arrange + changelog = """# Changelog {#unreleased} ## Unreleased @@ -366,39 +349,37 @@ def test_update_changelog_empty_news(self): ### Added * (toolchains) Some older change. """ - changelog_path = self.tmpdir / "CHANGELOG.md" - changelog_path.write_text(changelog) - - news_dir = self.tmpdir / "news" - news_dir.mkdir() - - # Act - changelog_news.update_changelog( - "3.0.0", - "2026-06-16", - changelog_path=changelog_path, - news_dir=news_dir, - ) - - # Assert - new_content = changelog_path.read_text() - - # The new release section should be present and contain "No notable changes." - self.assertIn("{#v3-0-0}", new_content) - self.assertIn("## [3.0.0] - 2026-06-16", new_content) - self.assertIn( - "[3.0.0]: https://github.com/bazel-contrib/rules_python/releases/tag/3.0.0", - new_content, - ) - self.assertIn("No notable changes.", new_content) - - # Verify that we didn't accidentally create any categories - self.assertNotIn("{#v3-0-0-fixed}", new_content) - self.assertNotIn("{#v3-0-0-added}", new_content) - - def test_update_changelog_selective_news_files(self): - # Arrange - changelog = """# Changelog + changelog_path = tmp_path / "CHANGELOG.md" + changelog_path.write_text(changelog) + + news_dir = tmp_path / "news" + news_dir.mkdir() + + # Act + changelog_news.update_changelog( + "3.0.0", + "2026-06-16", + changelog_path=changelog_path, + news_dir=news_dir, + ) + + # Assert + new_content = changelog_path.read_text() + + assert "{#v3-0-0}" in new_content + assert "## [3.0.0] - 2026-06-16" in new_content + assert ( + "[3.0.0]: https://github.com/bazel-contrib/rules_python/releases/tag/3.0.0" + in new_content + ) + assert "No notable changes." in new_content + assert "{#v3-0-0-fixed}" not in new_content + assert "{#v3-0-0-added}" not in new_content + + +def test_update_changelog_selective_news_files(tmp_path): + # Arrange + changelog = """# Changelog {#unreleased} ## Unreleased @@ -410,39 +391,37 @@ def test_update_changelog_selective_news_files(self): [2.0.2]: https://github.com/bazel-contrib/rules_python/releases/tag/2.0.2 """ - changelog_path = self.tmpdir / "CHANGELOG.md" - changelog_path.write_text(changelog) + changelog_path = tmp_path / "CHANGELOG.md" + changelog_path.write_text(changelog) - news_dir = self.tmpdir / "news" - news_dir.mkdir() + news_dir = tmp_path / "news" + news_dir.mkdir() - # Create news files - (news_dir / "123.fixed.md").write_text("Fix A") - (news_dir / "456.fixed.md").write_text("Fix B") + # Create news files + (news_dir / "123.fixed.md").write_text("Fix A") + (news_dir / "456.fixed.md").write_text("Fix B") - # Act: Only process 123.fixed.md - changelog_news.update_changelog( - "2.0.3", - "2026-06-16", - changelog_path=changelog_path, - news_dir=news_dir, - news_files=[news_dir / "123.fixed.md"], - ) + # Act: Only process 123.fixed.md + changelog_news.update_changelog( + "2.0.3", + "2026-06-16", + changelog_path=changelog_path, + news_dir=news_dir, + news_files=[news_dir / "123.fixed.md"], + ) - # Assert - # 1. Only 123.fixed.md should be deleted - self.assertFalse((news_dir / "123.fixed.md").exists()) - self.assertTrue((news_dir / "456.fixed.md").exists()) + # Assert + assert not (news_dir / "123.fixed.md").exists() + assert (news_dir / "456.fixed.md").exists() - new_content = changelog_path.read_text() + new_content = changelog_path.read_text() + assert "Fix A" in new_content + assert "Fix B" not in new_content - # 2. Only Fix A should be in the changelog - self.assertIn("Fix A", new_content) - self.assertNotIn("Fix B", new_content) - def test_update_changelog_insertion_point(self): - # Arrange - changelog = """# Changelog +def test_update_changelog_insertion_point(tmp_path): + # Arrange + changelog = """# Changelog {#unreleased} ## Unreleased @@ -459,35 +438,35 @@ def test_update_changelog_insertion_point(self): [2.0.0]: https://github.com/bazel-contrib/rules_python/releases/tag/2.0.0 """ - changelog_path = self.tmpdir / "CHANGELOG.md" - changelog_path.write_text(changelog) + changelog_path = tmp_path / "CHANGELOG.md" + changelog_path.write_text(changelog) - news_dir = self.tmpdir / "news" - news_dir.mkdir() - (news_dir / "123.fixed.md").write_text("Fix in 2.1.0") + news_dir = tmp_path / "news" + news_dir.mkdir() + (news_dir / "123.fixed.md").write_text("Fix in 2.1.0") - # Act: Insert 2.1.0 - changelog_news.update_changelog( - "2.1.0", - "2026-06-17", - changelog_path=changelog_path, - news_dir=news_dir, - ) + # Act: Insert 2.1.0 + changelog_news.update_changelog( + "2.1.0", + "2026-06-17", + changelog_path=changelog_path, + news_dir=news_dir, + ) + + # Assert + new_content = changelog_path.read_text() - # Assert - new_content = changelog_path.read_text() + idx_2_2_0 = new_content.index("{#v2-2-0}") + idx_2_1_0 = new_content.index("{#v2-1-0}") + idx_2_0_0 = new_content.index("{#v2-0-0}") - # Verify 2.1.0 is inserted BEFORE 2.0.0 but AFTER 2.2.0 - idx_2_2_0 = new_content.index("{#v2-2-0}") - idx_2_1_0 = new_content.index("{#v2-1-0}") - idx_2_0_0 = new_content.index("{#v2-0-0}") + assert idx_2_2_0 < idx_2_1_0 < idx_2_0_0 + assert "Fix in 2.1.0" in new_content - self.assertTrue(idx_2_2_0 < idx_2_1_0 < idx_2_0_0) - self.assertIn("Fix in 2.1.0", new_content) - def test_update_changelog_insertion_point_too_small(self): - # Arrange - changelog = """# Changelog +def test_update_changelog_insertion_point_too_small(tmp_path): + # Arrange + changelog = """# Changelog {#unreleased} ## Unreleased @@ -499,26 +478,20 @@ def test_update_changelog_insertion_point_too_small(self): [2.0.0]: https://github.com/bazel-contrib/rules_python/releases/tag/2.0.0 """ - changelog_path = self.tmpdir / "CHANGELOG.md" - changelog_path.write_text(changelog) - - news_dir = self.tmpdir / "news" - news_dir.mkdir() - (news_dir / "123.fixed.md").write_text("Fix in 1.0.0") - - # Act & Assert - with self.assertRaises(ValueError) as ctx: - changelog_news.update_changelog( - "1.0.0", - "2026-01-01", - changelog_path=changelog_path, - news_dir=news_dir, - ) - self.assertIn( - "Could not find a version in CHANGELOG.md smaller than 1.0.0", - str(ctx.exception), - ) + changelog_path = tmp_path / "CHANGELOG.md" + changelog_path.write_text(changelog) + news_dir = tmp_path / "news" + news_dir.mkdir() + (news_dir / "123.fixed.md").write_text("Fix in 1.0.0") -if __name__ == "__main__": - unittest.main() + # Act & Assert + with pytest.raises( + ValueError, match="Could not find a version in CHANGELOG.md smaller than 1.0.0" + ): + changelog_news.update_changelog( + "1.0.0", + "2026-01-01", + changelog_path=changelog_path, + news_dir=news_dir, + ) diff --git a/tests/tools/private/release/complete_sync_changelog_test.py b/tests/tools/private/release/complete_sync_changelog_test.py index 66c40c01d3..d0eb148b57 100644 --- a/tests/tools/private/release/complete_sync_changelog_test.py +++ b/tests/tools/private/release/complete_sync_changelog_test.py @@ -1,36 +1,18 @@ import argparse -import unittest -from unittest.mock import patch -from tests.tools.private.release.release_test_helper import _mock_git_and_gh from tools.private.release.complete_sync_changelog import CompleteSyncChangelog +pytest_plugins = ["tests.tools.private.release.release_test_helper"] -class CompleteSyncChangelogTest(unittest.TestCase): - def setUp(self): - _mock_git_and_gh(self) - self.addCleanup(patch.stopall) - # Dynamic mock for issue body - self.issue_body = "" - - def mock_get_body(issue_num): - return self.issue_body - - def mock_update_body(issue_num, body): - self.issue_body = body - - self.mock_gh.get_issue_body.side_effect = mock_get_body - self.mock_gh.update_issue_body.side_effect = mock_update_body - - def test_complete_sync_changelog_success(self): - args = argparse.Namespace(pr=999) - self.mock_gh.get_pr_info.return_value = { - "state": "MERGED", - "body": "Updates CHANGELOG.md\n\nRelease-Tracking-Issue: #123", - "mergeCommit": {"oid": "abcdef1234567890"}, - } - self.issue_body = """ +def test_complete_sync_changelog_success(mock_gh): + args = argparse.Namespace(pr=999) + mock_gh.prs[999] = { + "state": "MERGED", + "body": "Updates CHANGELOG.md\n\nRelease-Tracking-Issue: #123", + "mergeCommit": {"oid": "abcdef1234567890"}, + } + issue_body = """ ## Checklist - [ ] Prepare Release - [ ] Create Release branch @@ -41,64 +23,66 @@ def test_complete_sync_changelog_success(self): ## Backports """ - result = CompleteSyncChangelog(args, self.mock_gh).run() - - self.assertEqual(result, 0) - self.mock_gh.get_pr_info.assert_called_once_with(999) - self.mock_gh.get_issue_body.assert_called_once_with(123) - self.mock_gh.update_issue_body.assert_called_once() - - # Check that only tasks pointing to #999 were marked checked=True and status=done - self.assertIn( - "- [x] Sync Changelog #124 | status=done pr=#999 commit= abcdef12", - self.issue_body, - ) - self.assertIn( - "- [x] Sync Changelog #125 | status=done pr=#999 commit= abcdef12", - self.issue_body, - ) - # Task pointing to #888 should remain unchanged - self.assertIn( - "- [ ] Sync Changelog #126 | status=pending pr=#888", - self.issue_body, - ) - - def test_complete_sync_changelog_not_merged(self): - args = argparse.Namespace(pr=999) - self.mock_gh.get_pr_info.return_value = { - "state": "OPEN", - "body": "Updates CHANGELOG.md\n\nRelease-Tracking-Issue: #123", - } - - result = CompleteSyncChangelog(args, self.mock_gh).run() - - self.assertEqual(result, 1) - self.mock_gh.get_pr_info.assert_called_once_with(999) - self.mock_gh.update_issue_body.assert_not_called() - - def test_complete_sync_changelog_missing_tracking_issue_link(self): - args = argparse.Namespace(pr=999) - self.mock_gh.get_pr_info.return_value = { - "state": "MERGED", - "body": "Updates CHANGELOG.md without tracking issue link", - "mergeCommit": {"oid": "abcdef1234567890"}, - } - - result = CompleteSyncChangelog(args, self.mock_gh).run() - - self.assertEqual(result, 1) - self.mock_gh.get_pr_info.assert_called_once_with(999) - self.mock_gh.update_issue_body.assert_not_called() - - def test_complete_sync_changelog_no_matching_tasks(self): - args = argparse.Namespace(pr=999) - self.mock_gh.get_pr_info.return_value = { - "state": "MERGED", - "body": "Updates CHANGELOG.md\n\nRelease-Tracking-Issue: #123", - "mergeCommit": {"oid": "abcdef1234567890"}, - } - # Checklist has no tasks pointing to #999 - self.issue_body = """ + mock_gh.issues[123] = { + "title": "Release 2.1.0", + "body": issue_body, + "labels": ["type: release"], + "number": 123, + "url": "https://github.com/bazel-contrib/rules_python/issues/123", + } + + result = CompleteSyncChangelog(args, mock_gh).run() + + assert result == 0 + updated_body = mock_gh.get_issue_body(123) + + # Check that only tasks pointing to #999 were marked checked=True and status=done + assert ( + "- [x] Sync Changelog #124 | status=done pr=#999 commit= abcdef12" + in updated_body + ) + assert ( + "- [x] Sync Changelog #125 | status=done pr=#999 commit= abcdef12" + in updated_body + ) + # Task pointing to #888 should remain unchanged + assert "- [ ] Sync Changelog #126 | status=pending pr=#888" in updated_body + + +def test_complete_sync_changelog_not_merged(mock_gh): + args = argparse.Namespace(pr=999) + mock_gh.prs[999] = { + "state": "OPEN", + "body": "Updates CHANGELOG.md\n\nRelease-Tracking-Issue: #123", + } + + result = CompleteSyncChangelog(args, mock_gh).run() + + assert result == 1 + + +def test_complete_sync_changelog_missing_tracking_issue_link(mock_gh): + args = argparse.Namespace(pr=999) + mock_gh.prs[999] = { + "state": "MERGED", + "body": "Updates CHANGELOG.md without tracking issue link", + "mergeCommit": {"oid": "abcdef1234567890"}, + } + + result = CompleteSyncChangelog(args, mock_gh).run() + + assert result == 1 + + +def test_complete_sync_changelog_no_matching_tasks(mock_gh): + args = argparse.Namespace(pr=999) + mock_gh.prs[999] = { + "state": "MERGED", + "body": "Updates CHANGELOG.md\n\nRelease-Tracking-Issue: #123", + "mergeCommit": {"oid": "abcdef1234567890"}, + } + # Checklist has no tasks pointing to #999 + issue_body = """ ## Checklist - [ ] Prepare Release - [ ] Create Release branch @@ -107,14 +91,15 @@ def test_complete_sync_changelog_no_matching_tasks(self): ## Backports """ - result = CompleteSyncChangelog(args, self.mock_gh).run() - - # Should log warning but return 0 (success/noop) - self.assertEqual(result, 0) - self.mock_gh.get_pr_info.assert_called_once_with(999) - self.mock_gh.get_issue_body.assert_called_once_with(123) - self.mock_gh.update_issue_body.assert_not_called() - - -if __name__ == "__main__": - unittest.main() + mock_gh.issues[123] = { + "title": "Release 2.1.0", + "body": issue_body, + "labels": ["type: release"], + "number": 123, + "url": "https://github.com/bazel-contrib/rules_python/issues/123", + } + + result = CompleteSyncChangelog(args, mock_gh).run() + + assert result == 0 + assert mock_gh.get_issue_body(123) == issue_body diff --git a/tests/tools/private/release/create_rc_test.py b/tests/tools/private/release/create_rc_test.py index 8e4ed101e4..6aee746dbb 100644 --- a/tests/tools/private/release/create_rc_test.py +++ b/tests/tools/private/release/create_rc_test.py @@ -2,177 +2,163 @@ import os import pathlib import tempfile -import unittest -from unittest.mock import MagicMock, call, patch +from unittest.mock import call -from tests.tools.private.release.release_test_helper import _mock_git_and_gh from tools.private.release.create_rc import CreateRc +pytest_plugins = ["tests.tools.private.release.release_test_helper"] -class CmdCreateRcTest(unittest.TestCase): - def setUp(self): - _mock_git_and_gh(self) - def test_create_rc_success_first_rc(self): - # Arrange - args = MagicMock(issue=123, remote="my-remote") - self.mock_gh.get_issue_title.return_value = "Release 2.0.0" - self.mock_gh.get_issue_body.return_value = """ +def test_create_rc_success_first_rc(mocker, mock_git, mock_gh): + # Arrange + args = argparse.Namespace( + issue=123, remote="my-remote", triggering_comment=None, dry_run=False + ) + mock_gh.issues[123] = { + "title": "Release 2.0.0", + "body": """ ## Checklist - [x] Prepare Release | status=done pr=#122 commit=abcdef12 - [x] Create Release branch | status=done branch=release/2.0 commit=abcdef12 - [ ] Tag RC0 | status=pending -""" - self.mock_git.get_remote_tags.return_value = [] - self.mock_git.get_commit_sha.return_value = "1234567890" - - # Act - with tempfile.TemporaryDirectory() as tmpdir: - github_output_file = pathlib.Path(tmpdir) / "github_output" - with patch.dict(os.environ, {"GITHUB_OUTPUT": str(github_output_file)}): - result = CreateRc(args, self.mock_git, self.mock_gh).run() - - # Assert - self.assertEqual(result, 0) - self.assertTrue(github_output_file.exists()) - self.assertEqual(github_output_file.read_text(), "tag_name=2.0.0-rc0\n") - self.mock_git.fetch.assert_has_calls( - [call("my-remote"), call("my-remote", tags=True, force=True)] - ) - self.mock_git.checkout.assert_not_called() - self.mock_git.tag.assert_called_once_with("2.0.0-rc0", "my-remote/release/2.0") - self.mock_git.push.assert_called_once_with("my-remote", "2.0.0-rc0") - self.mock_git.get_commit_sha.assert_called_once_with("my-remote/release/2.0") - - self.mock_gh.update_issue_body.assert_called_once() - call_args = self.mock_gh.update_issue_body.call_args[0] - self.assertEqual(call_args[0], 123) - self.assertIn("tag=2.0.0-rc0", call_args[1]) - self.assertIn("commit= 12345678", call_args[1]) - - self.mock_gh.post_issue_comment.assert_called_once() - comment_call_args = self.mock_gh.post_issue_comment.call_args[0] - self.assertEqual(comment_call_args[0], 123) - self.assertIn( - "**New Release Candidate Tagged!** 🐍🌿", - comment_call_args[1], - ) - self.assertIn( - "tagged on branch [`release/2.0`](https://github.com/bazel-contrib/rules_python/tree/release/2.0)", - comment_call_args[1], - ) - self.assertIn( - "- [Github Release 2.0.0-rc0](https://github.com/bazel-contrib/rules_python/releases/tag/2.0.0-rc0)", - comment_call_args[1], - ) - self.assertIn( - "- [BCR Entry 2.0.0-rc0](https://registry.bazel.build/modules/rules_python/2.0.0-rc0)", - comment_call_args[1], - ) - self.assertIn( - "- [BCR PRs](https://github.com/bazelbuild/bazel-central-registry/pulls?q=is%3Apr+rules_python+2.0.0-rc0)", - comment_call_args[1], - ) - self.assertIn( - "- [Release workflow status](https://github.com/bazel-contrib/rules_python/actions/workflows/release_create_rc.yaml)", - comment_call_args[1], - ) - - def test_create_rc_success_with_run_id(self): - # Arrange - args = MagicMock(issue=123, remote="my-remote") - self.mock_gh.get_issue_title.return_value = "Release 2.0.0" - self.mock_gh.get_issue_body.return_value = """ +""", + "labels": ["type: release"], + } + mock_git.get_remote_tags.return_value = [] + mock_git.get_commit_sha.return_value = "1234567890" + + # Act + with tempfile.TemporaryDirectory() as tmpdir: + github_output_file = pathlib.Path(tmpdir) / "github_output" + mocker.patch.dict(os.environ, {"GITHUB_OUTPUT": str(github_output_file)}) + result = CreateRc(args, mock_git, mock_gh).run() + + # Assert + assert result == 0 + assert github_output_file.exists() + assert github_output_file.read_text() == "tag_name=2.0.0-rc0\n" + + mock_git.fetch.assert_has_calls( + [call("my-remote"), call("my-remote", tags=True, force=True)] + ) + mock_git.checkout.assert_not_called() + mock_git.tag.assert_called_once_with("2.0.0-rc0", "my-remote/release/2.0") + mock_git.push.assert_called_once_with("my-remote", "2.0.0-rc0") + mock_git.get_commit_sha.assert_called_once_with("my-remote/release/2.0") + + updated_body = mock_gh.get_issue_body(123) + assert "tag=2.0.0-rc0" in updated_body + assert "commit= 12345678" in updated_body + + assert 123 in mock_gh.issue_comments + comment_text = mock_gh.issue_comments[123][0] + assert "**New Release Candidate Tagged!** 🐍🌿" in comment_text + assert ( + "tagged on branch [`release/2.0`](https://github.com/bazel-contrib/rules_python/tree/release/2.0)" + in comment_text + ) + assert ( + "- [Github Release 2.0.0-rc0](https://github.com/bazel-contrib/rules_python/releases/tag/2.0.0-rc0)" + in comment_text + ) + assert ( + "- [BCR Entry 2.0.0-rc0](https://registry.bazel.build/modules/rules_python/2.0.0-rc0)" + in comment_text + ) + assert ( + "- [BCR PRs](https://github.com/bazelbuild/bazel-central-registry/pulls?q=is%3Apr+rules_python+2.0.0-rc0)" + in comment_text + ) + assert ( + "- [Release workflow status](https://github.com/bazel-contrib/rules_python/actions/workflows/release_create_rc.yaml)" + in comment_text + ) + + +def test_create_rc_success_with_run_id(mocker, mock_git, mock_gh): + # Arrange + args = argparse.Namespace( + issue=123, remote="my-remote", triggering_comment=None, dry_run=False + ) + mock_gh.issues[123] = { + "title": "Release 2.0.0", + "body": """ ## Checklist - [x] Prepare Release | status=done pr=#122 commit=abcdef12 - [x] Create Release branch | status=done branch=release/2.0 commit=abcdef12 - [ ] Tag RC0 | status=pending -""" - self.mock_git.get_remote_tags.return_value = [] - self.mock_git.get_commit_sha.return_value = "1234567890" - - # Act - with patch.dict(os.environ, {"GITHUB_RUN_ID": "987654321"}): - result = CreateRc(args, self.mock_git, self.mock_gh).run() - - # Assert - self.assertEqual(result, 0) - self.mock_gh.post_issue_comment.assert_called_once() - comment_call_args = self.mock_gh.post_issue_comment.call_args[0] - self.assertIn( - "- [Release workflow status](https://github.com/bazel-contrib/rules_python/actions/runs/987654321)", - comment_call_args[1], - ) - self.assertIn( - "tagged on branch [`release/2.0`](https://github.com/bazel-contrib/rules_python/tree/release/2.0)", - comment_call_args[1], - ) - - def test_create_rc_success_next_rc(self): - # Arrange - args = MagicMock(issue=123, remote="my-remote") - self.mock_gh.get_issue_title.return_value = "Release 2.0.0" - self.mock_gh.get_issue_body.return_value = """ +""", + "labels": ["type: release"], + } + mock_git.get_remote_tags.return_value = [] + mock_git.get_commit_sha.return_value = "1234567890" + + # Act + mocker.patch.dict(os.environ, {"GITHUB_RUN_ID": "987654321"}) + result = CreateRc(args, mock_git, mock_gh).run() + + # Assert + assert result == 0 + assert 123 in mock_gh.issue_comments + comment_text = mock_gh.issue_comments[123][0] + assert ( + "- [Release workflow status](https://github.com/bazel-contrib/rules_python/actions/runs/987654321)" + in comment_text + ) + assert ( + "tagged on branch [`release/2.0`](https://github.com/bazel-contrib/rules_python/tree/release/2.0)" + in comment_text + ) + + +def test_create_rc_success_next_rc(mock_git, mock_gh): + # Arrange + args = argparse.Namespace( + issue=123, remote="my-remote", triggering_comment=None, dry_run=False + ) + mock_gh.issues[123] = { + "title": "Release 2.0.0", + "body": """ ## Checklist - [x] Prepare Release | status=done pr=#122 commit=abcdef12 - [x] Create Release branch | status=done branch=release/2.0 commit=abcdef12 - [x] Tag RC0 | status=done tag=2.0.0-rc0 commit=abcdef12 - [ ] Tag RC1 | status=pending -""" - self.mock_git.get_remote_tags.return_value = ["2.0.0-rc0"] - self.mock_git.get_commit_sha.return_value = "1234567890" - - # Act - result = CreateRc(args, self.mock_git, self.mock_gh).run() - - # Assert - self.assertEqual(result, 0) - self.mock_git.fetch.assert_has_calls( - [call("my-remote"), call("my-remote", tags=True, force=True)] - ) - self.mock_git.checkout.assert_not_called() - self.mock_git.tag.assert_called_once_with("2.0.0-rc1", "my-remote/release/2.0") - self.mock_git.push.assert_called_once_with("my-remote", "2.0.0-rc1") - self.mock_git.get_commit_sha.assert_called_once_with("my-remote/release/2.0") - - self.mock_gh.update_issue_body.assert_called_once() - call_args = self.mock_gh.update_issue_body.call_args[0] - self.assertEqual(call_args[0], 123) - self.assertIn("tag=2.0.0-rc1", call_args[1]) - - self.mock_gh.post_issue_comment.assert_called_once() - comment_call_args = self.mock_gh.post_issue_comment.call_args[0] - self.assertEqual(comment_call_args[0], 123) - self.assertIn( - "**New Release Candidate Tagged!** 🐍🌿", - comment_call_args[1], - ) - self.assertIn( - "tagged on branch [`release/2.0`](https://github.com/bazel-contrib/rules_python/tree/release/2.0)", - comment_call_args[1], - ) - self.assertIn( - "- [Github Release 2.0.0-rc1](https://github.com/bazel-contrib/rules_python/releases/tag/2.0.0-rc1)", - comment_call_args[1], - ) - self.assertIn( - "- [BCR Entry 2.0.0-rc1](https://registry.bazel.build/modules/rules_python/2.0.0-rc1)", - comment_call_args[1], - ) - self.assertIn( - "- [BCR PRs](https://github.com/bazelbuild/bazel-central-registry/pulls?q=is%3Apr+rules_python+2.0.0-rc1)", - comment_call_args[1], - ) - self.assertIn( - "- [Release workflow status](https://github.com/bazel-contrib/rules_python/actions/workflows/release_create_rc.yaml)", - comment_call_args[1], - ) - - def test_create_rc_gating_on_backports(self): - # Arrange - args = MagicMock(issue=123, remote="my-remote") - self.mock_gh.get_issue_title.return_value = "Release 2.0.0" - self.mock_gh.get_issue_body.return_value = """ +""", + "labels": ["type: release"], + } + mock_git.get_remote_tags.return_value = ["2.0.0-rc0"] + mock_git.get_commit_sha.return_value = "1234567890" + + # Act + result = CreateRc(args, mock_git, mock_gh).run() + + # Assert + assert result == 0 + mock_git.fetch.assert_has_calls( + [call("my-remote"), call("my-remote", tags=True, force=True)] + ) + mock_git.checkout.assert_not_called() + mock_git.tag.assert_called_once_with("2.0.0-rc1", "my-remote/release/2.0") + mock_git.push.assert_called_once_with("my-remote", "2.0.0-rc1") + mock_git.get_commit_sha.assert_called_once_with("my-remote/release/2.0") + + updated_body = mock_gh.get_issue_body(123) + assert "tag=2.0.0-rc1" in updated_body + + assert 123 in mock_gh.issue_comments + comment_text = mock_gh.issue_comments[123][0] + assert "**New Release Candidate Tagged!** 🐍🌿" in comment_text + + +def test_create_rc_gating_on_backports(mock_git, mock_gh): + # Arrange + args = argparse.Namespace( + issue=123, remote="my-remote", triggering_comment=None, dry_run=False + ) + mock_gh.issues[123] = { + "title": "Release 2.0.0", + "body": """ ## Checklist - [x] Prepare Release | status=done pr=#122 commit=abcdef12 - [x] Create Release branch | status=done branch=release/2.0 commit=abcdef12 @@ -180,20 +166,26 @@ def test_create_rc_gating_on_backports(self): ## Backports - [ ] #124 | status=pending -""" - # Act - result = CreateRc(args, self.mock_git, self.mock_gh).run() - - # Assert - self.assertEqual(result, 1) - self.mock_git.tag.assert_not_called() - self.mock_git.push.assert_not_called() - - def test_create_rc_not_blocked_by_ignored_backports(self): - # Arrange - args = MagicMock(issue=123, remote="my-remote") - self.mock_gh.get_issue_title.return_value = "Release 2.0.0" - self.mock_gh.get_issue_body.return_value = """ +""", + "labels": ["type: release"], + } + # Act + result = CreateRc(args, mock_git, mock_gh).run() + + # Assert + assert result == 1 + mock_git.tag.assert_not_called() + mock_git.push.assert_not_called() + + +def test_create_rc_not_blocked_by_ignored_backports(mock_git, mock_gh): + # Arrange + args = argparse.Namespace( + issue=123, remote="my-remote", triggering_comment=None, dry_run=False + ) + mock_gh.issues[123] = { + "title": "Release 2.0.0", + "body": """ ## Checklist - [x] Prepare Release | status=done pr=#122 commit=abcdef12 - [x] Create Release branch | status=done branch=release/2.0 commit=abcdef12 @@ -201,23 +193,29 @@ def test_create_rc_not_blocked_by_ignored_backports(self): ## Backports - [ ] #124 | status=ignore -""" - self.mock_git.get_remote_tags.return_value = [] - self.mock_git.get_commit_sha.return_value = "1234567890" - - # Act - result = CreateRc(args, self.mock_git, self.mock_gh).run() - - # Assert - self.assertEqual(result, 0) - self.mock_git.tag.assert_called_once_with("2.0.0-rc0", "my-remote/release/2.0") - self.mock_git.push.assert_called_once_with("my-remote", "2.0.0-rc0") - - def test_create_rc_with_finished_backports(self): - # Arrange - args = MagicMock(issue=123, remote="my-remote") - self.mock_gh.get_issue_title.return_value = "Release 2.0.0" - self.mock_gh.get_issue_body.return_value = """ +""", + "labels": ["type: release"], + } + mock_git.get_remote_tags.return_value = [] + mock_git.get_commit_sha.return_value = "1234567890" + + # Act + result = CreateRc(args, mock_git, mock_gh).run() + + # Assert + assert result == 0 + mock_git.tag.assert_called_once_with("2.0.0-rc0", "my-remote/release/2.0") + mock_git.push.assert_called_once_with("my-remote", "2.0.0-rc0") + + +def test_create_rc_with_finished_backports(mock_git, mock_gh): + # Arrange + args = argparse.Namespace( + issue=123, remote="my-remote", triggering_comment=None, dry_run=False + ) + mock_gh.issues[123] = { + "title": "Release 2.0.0", + "body": """ ## Checklist - [x] Prepare Release | status=done pr=#122 commit=abcdef12 - [x] Create Release branch | status=done branch=release/2.0 commit=abcdef12 @@ -225,182 +223,198 @@ def test_create_rc_with_finished_backports(self): ## Backports - [x] #124 | status=done rc=rc0 commit=abcdef12 -""" - self.mock_git.get_remote_tags.return_value = [] - self.mock_git.get_commit_sha.return_value = "1234567890" - - # Act - result = CreateRc(args, self.mock_git, self.mock_gh).run() - - # Assert - self.assertEqual(result, 0) - self.mock_git.tag.assert_called_once_with("2.0.0-rc0", "my-remote/release/2.0") - self.mock_git.push.assert_called_once_with("my-remote", "2.0.0-rc0") - - def test_create_rc_auto_add_task(self): - # Arrange - args = argparse.Namespace(issue=123, remote="my-remote") - self.mock_gh.get_issue_title.return_value = "Release 2.0.0" - self.mock_gh.get_issue_body.return_value = """ +""", + "labels": ["type: release"], + } + mock_git.get_remote_tags.return_value = [] + mock_git.get_commit_sha.return_value = "1234567890" + + # Act + result = CreateRc(args, mock_git, mock_gh).run() + + # Assert + assert result == 0 + mock_git.tag.assert_called_once_with("2.0.0-rc0", "my-remote/release/2.0") + mock_git.push.assert_called_once_with("my-remote", "2.0.0-rc0") + + +def test_create_rc_auto_add_task(mock_git, mock_gh): + # Arrange + args = argparse.Namespace( + issue=123, remote="my-remote", triggering_comment=None, dry_run=False + ) + mock_gh.issues[123] = { + "title": "Release 2.0.0", + "body": """ ## Checklist - [x] Prepare Release | status=done pr=#122 commit=abcdef12 - [x] Create Release branch | status=done branch=release/2.0 commit=abcdef12 - [x] Tag RC0 | status=done tag=2.0.0-rc0 commit=abcdef12 - [ ] Tag Final -""" - self.mock_git.get_remote_tags.return_value = ["2.0.0-rc0"] - self.mock_git.get_commit_sha.return_value = "1234567890" - - # Act - result = CreateRc(args, self.mock_git, self.mock_gh).run() - - # Assert - self.assertEqual(result, 0) - self.mock_git.tag.assert_called_once_with("2.0.0-rc1", "my-remote/release/2.0") - self.mock_git.push.assert_called_once_with("my-remote", "2.0.0-rc1") - - self.assertEqual(self.mock_gh.update_issue_body.call_count, 2) - call1_args = self.mock_gh.update_issue_body.call_args_list[0][0] - call2_args = self.mock_gh.update_issue_body.call_args_list[1][0] - - self.assertEqual(call1_args[0], 123) - self.assertIn("- [ ] Tag RC1", call1_args[1]) - self.assertIn( - "- [x] Tag RC0 | status=done tag=2.0.0-rc0 commit=abcdef12\n- [ ]" - " Tag RC1\n- [ ] Tag Final", - call1_args[1].strip(), - ) - - self.assertEqual(call2_args[0], 123) - self.assertIn( - "- [x] Tag RC1 | status=done tag=2.0.0-rc1 commit= 12345678", - call2_args[1], - ) - - @patch("tools.private.release.create_rc.ProcessBackports") - def test_create_rc_calls_process_backports(self, mock_pb_class): - # Arrange - mock_pb = mock_pb_class.return_value - mock_pb.run.return_value = 0 - - args = MagicMock(issue=123, remote="my-remote") - self.mock_gh.get_issue_title.return_value = "Release 2.0.0" - self.mock_gh.get_issue_body.return_value = """ +""", + "labels": ["type: release"], + } + mock_git.get_remote_tags.return_value = ["2.0.0-rc0"] + mock_git.get_commit_sha.return_value = "1234567890" + + # Act + result = CreateRc(args, mock_git, mock_gh).run() + + # Assert + assert result == 0 + mock_git.tag.assert_called_once_with("2.0.0-rc1", "my-remote/release/2.0") + mock_git.push.assert_called_once_with("my-remote", "2.0.0-rc1") + + updated_body = mock_gh.get_issue_body(123) + assert "- [x] Tag RC1 | status=done tag=2.0.0-rc1 commit= 12345678" in updated_body + + +def test_create_rc_calls_process_backports(mocker, mock_git, mock_gh): + # Arrange + mock_pb_class = mocker.patch("tools.private.release.create_rc.ProcessBackports") + mock_pb = mock_pb_class.return_value + mock_pb.run.return_value = 0 + + args = argparse.Namespace( + issue=123, remote="my-remote", triggering_comment=None, dry_run=False + ) + mock_gh.issues[123] = { + "title": "Release 2.0.0", + "body": """ ## Checklist - [x] Prepare Release | status=done pr=#122 commit=abcdef12 - [x] Create Release branch | status=done branch=release/2.0 commit=abcdef12 - [ ] Tag RC0 | status=pending -""" - self.mock_git.get_remote_tags.return_value = [] - self.mock_git.get_commit_sha.return_value = "1234567890" - - # Act - result = CreateRc(args, self.mock_git, self.mock_gh).run() - - # Assert - self.assertEqual(result, 0) - mock_pb_class.assert_called_once() - called_args = mock_pb_class.call_args[0][0] - self.assertEqual(called_args.issue, 123) - self.assertEqual(called_args.remote, "my-remote") - self.assertFalse(called_args.dry_run) - self.assertIsNone(called_args.add) - self.assertIsNone(called_args.triggering_comment) - mock_pb.run.assert_called_once() - - @patch("tools.private.release.create_rc.ProcessBackports") - def test_create_rc_aborts_on_process_backports_failure(self, mock_pb_class): - # Arrange - mock_pb = mock_pb_class.return_value - mock_pb.run.return_value = 1 - - args = MagicMock(issue=123, remote="my-remote") - - # Act - result = CreateRc(args, self.mock_git, self.mock_gh).run() - - # Assert - self.assertEqual(result, 1) - mock_pb_class.assert_called_once() - mock_pb.run.assert_called_once() - self.mock_gh.get_issue_body.assert_not_called() - self.mock_git.tag.assert_not_called() - - @patch("tools.private.release.create_rc.ProcessBackports") - def test_create_rc_failure_reacts_to_comment(self, mock_pb_class): - # Arrange - mock_pb = mock_pb_class.return_value - mock_pb.run.return_value = 1 # Simulate failure - - args = MagicMock(issue=123, remote="my-remote", triggering_comment=456) - - # Act - result = CreateRc(args, self.mock_git, self.mock_gh).run() - - # Assert - self.assertEqual(result, 1) - self.mock_gh.add_comment_reaction.assert_called_once_with(456, "-1") - - @patch("tools.private.release.create_rc.ProcessBackports") - def test_create_rc_failure_no_comment_no_reaction(self, mock_pb_class): - # Arrange - mock_pb = mock_pb_class.return_value - mock_pb.run.return_value = 1 # Simulate failure - - args = MagicMock(issue=123, remote="my-remote", triggering_comment=None) - - # Act - result = CreateRc(args, self.mock_git, self.mock_gh).run() - - # Assert - self.assertEqual(result, 1) - self.mock_gh.add_comment_reaction.assert_not_called() - - @patch("tools.private.release.create_rc.ProcessBackports") - def test_create_rc_success_with_comment_no_reaction(self, mock_pb_class): - # Arrange - mock_pb = mock_pb_class.return_value - mock_pb.run.return_value = 0 - - args = MagicMock(issue=123, remote="my-remote", triggering_comment=456) - self.mock_gh.get_issue_title.return_value = "Release 2.0.0" - self.mock_gh.get_issue_body.return_value = """ +""", + "labels": ["type: release"], + } + mock_git.get_remote_tags.return_value = [] + mock_git.get_commit_sha.return_value = "1234567890" + + # Act + result = CreateRc(args, mock_git, mock_gh).run() + + # Assert + assert result == 0 + mock_pb_class.assert_called_once() + called_args = mock_pb_class.call_args[0][0] + assert called_args.issue == 123 + assert called_args.remote == "my-remote" + assert not called_args.dry_run + assert called_args.add is None + assert called_args.triggering_comment is None + mock_pb.run.assert_called_once() + + +def test_create_rc_aborts_on_process_backports_failure(mocker, mock_git, mock_gh): + # Arrange + mock_pb_class = mocker.patch("tools.private.release.create_rc.ProcessBackports") + mock_pb = mock_pb_class.return_value + mock_pb.run.return_value = 1 + + args = argparse.Namespace( + issue=123, remote="my-remote", triggering_comment=None, dry_run=False + ) + + # Act + result = CreateRc(args, mock_git, mock_gh).run() + + # Assert + assert result == 1 + mock_pb_class.assert_called_once() + mock_pb.run.assert_called_once() + mock_git.tag.assert_not_called() + + +def test_create_rc_failure_reacts_to_comment(mocker, mock_git, mock_gh): + # Arrange + mock_pb_class = mocker.patch("tools.private.release.create_rc.ProcessBackports") + mock_pb = mock_pb_class.return_value + mock_pb.run.return_value = 1 # Simulate failure + + args = argparse.Namespace( + issue=123, remote="my-remote", triggering_comment=456, dry_run=False + ) + + # Act + result = CreateRc(args, mock_git, mock_gh).run() + + # Assert + assert result == 1 + assert mock_gh.reactions.get(456) == ["-1"] + + +def test_create_rc_failure_no_comment_no_reaction(mocker, mock_git, mock_gh): + # Arrange + mock_pb_class = mocker.patch("tools.private.release.create_rc.ProcessBackports") + mock_pb = mock_pb_class.return_value + mock_pb.run.return_value = 1 # Simulate failure + + args = argparse.Namespace( + issue=123, remote="my-remote", triggering_comment=None, dry_run=False + ) + + # Act + result = CreateRc(args, mock_git, mock_gh).run() + + # Assert + assert result == 1 + assert 456 not in mock_gh.reactions + + +def test_create_rc_success_with_comment_no_reaction(mocker, mock_git, mock_gh): + # Arrange + mock_pb_class = mocker.patch("tools.private.release.create_rc.ProcessBackports") + mock_pb = mock_pb_class.return_value + mock_pb.run.return_value = 0 + + args = argparse.Namespace( + issue=123, remote="my-remote", triggering_comment=456, dry_run=False + ) + mock_gh.issues[123] = { + "title": "Release 2.0.0", + "body": """ ## Checklist - [x] Prepare Release | status=done pr=#122 commit=abcdef12 - [x] Create Release branch | status=done branch=release/2.0 commit=abcdef12 - [ ] Tag RC0 | status=pending -""" - self.mock_git.get_remote_tags.return_value = [] - self.mock_git.get_commit_sha.return_value = "1234567890" - - # Act - result = CreateRc(args, self.mock_git, self.mock_gh).run() - - # Assert - self.assertEqual(result, 0) - self.mock_gh.add_comment_reaction.assert_not_called() - - @patch("tools.private.release.create_rc.ProcessBackports") - def test_create_rc_precondition_failure_reacts_to_comment(self, mock_pb_class): - # Arrange - mock_pb = mock_pb_class.return_value - mock_pb.run.return_value = 0 # Backports succeed - - args = MagicMock(issue=123, remote="my-remote", triggering_comment=456) - self.mock_gh.get_issue_body.return_value = """ +""", + "labels": ["type: release"], + } + mock_git.get_remote_tags.return_value = [] + mock_git.get_commit_sha.return_value = "1234567890" + + # Act + result = CreateRc(args, mock_git, mock_gh).run() + + # Assert + assert result == 0 + assert 456 not in mock_gh.reactions + + +def test_create_rc_precondition_failure_reacts_to_comment(mocker, mock_git, mock_gh): + # Arrange + mock_pb_class = mocker.patch("tools.private.release.create_rc.ProcessBackports") + mock_pb = mock_pb_class.return_value + mock_pb.run.return_value = 0 # Backports succeed + + args = argparse.Namespace( + issue=123, remote="my-remote", triggering_comment=456, dry_run=False + ) + mock_gh.issues[123] = { + "title": "Release 2.0.0", + "body": """ ## Checklist - [ ] Prepare Release | status=pending - [ ] Create Release branch | status=pending - [ ] Tag RC0 | status=pending -""" - - # Act - result = CreateRc(args, self.mock_git, self.mock_gh).run() - - # Assert - self.assertEqual(result, 1) - self.mock_gh.add_comment_reaction.assert_called_once_with(456, "-1") +""", + "labels": ["type: release"], + } + # Act + result = CreateRc(args, mock_git, mock_gh).run() -if __name__ == "__main__": - unittest.main() + # Assert + assert result == 1 + assert mock_gh.reactions.get(456) == ["-1"] diff --git a/tests/tools/private/release/create_release_branch_test.py b/tests/tools/private/release/create_release_branch_test.py index 07e3652c85..332ffec851 100644 --- a/tests/tools/private/release/create_release_branch_test.py +++ b/tests/tools/private/release/create_release_branch_test.py @@ -1,149 +1,157 @@ -import unittest -from unittest.mock import MagicMock +import argparse -from tests.tools.private.release.release_test_helper import _mock_git_and_gh from tools.private.release.create_release_branch import CreateReleaseBranch +pytest_plugins = ["tests.tools.private.release.release_test_helper"] -class CmdCreateReleaseBranchTest(unittest.TestCase): - def setUp(self): - _mock_git_and_gh(self) - def test_create_release_branch_success(self): - # Arrange - args = MagicMock(issue=123, remote="my-remote") - self.mock_gh.get_issue_title.return_value = "Release 2.0.0" - self.mock_gh.get_issue_body.return_value = """ +def test_create_release_branch_success(mock_git, mock_gh): + # Arrange + args = argparse.Namespace(issue=123, remote="my-remote") + mock_gh.issues[123] = { + "title": "Release 2.0.0", + "body": """ ## Checklist - [x] Prepare Release | status=done pr=#122 commit=abcdef12 - [ ] Create Release branch | status=pending -""" - self.mock_git.branch_exists.return_value = False - self.mock_git.remote_branch_exists.return_value = False - - # Act - result = CreateReleaseBranch(args, self.mock_git, self.mock_gh).run() - - # Assert - self.assertEqual(result, 0) - self.mock_git.fetch.assert_called_once_with("my-remote") - self.mock_git.checkout.assert_not_called() - self.mock_git.push.assert_called_once_with( - "my-remote", "abcdef12:refs/heads/release/2.0" - ) - - self.mock_gh.update_issue_body.assert_called_once() - call_args = self.mock_gh.update_issue_body.call_args[0] - self.assertEqual(call_args[0], 123) - self.assertIn( - "branch_url=https://github.com/bazel-contrib/rules_python/tree/release/2.0", - call_args[1], - ) - self.assertIn("commit= abcdef12", call_args[1]) - - def test_create_release_branch_prepare_not_done(self): - # Arrange - args = MagicMock(issue=123, remote="my-remote") - self.mock_gh.get_issue_title.return_value = "Release 2.0.0" - self.mock_gh.get_issue_body.return_value = """ +""", + "labels": ["type: release"], + } + mock_git.branch_exists.return_value = False + mock_git.remote_branch_exists.return_value = False + + # Act + result = CreateReleaseBranch(args, mock_git, mock_gh).run() + + # Assert + assert result == 0 + mock_git.fetch.assert_called_once_with("my-remote") + mock_git.checkout.assert_not_called() + mock_git.push.assert_called_once_with( + "my-remote", "abcdef12:refs/heads/release/2.0" + ) + + updated_body = mock_gh.get_issue_body(123) + assert ( + "branch_url=https://github.com/bazel-contrib/rules_python/tree/release/2.0" + in updated_body + ) + assert "commit= abcdef12" in updated_body + + +def test_create_release_branch_prepare_not_done(mock_git, mock_gh): + # Arrange + args = argparse.Namespace(issue=123, remote="my-remote") + mock_gh.issues[123] = { + "title": "Release 2.0.0", + "body": """ ## Checklist - [ ] Prepare Release | status=pending - [ ] Create Release branch | status=pending -""" - # Act - result = CreateReleaseBranch(args, self.mock_git, self.mock_gh).run() - - # Assert - self.assertEqual(result, 1) - self.mock_git.fetch.assert_not_called() - self.mock_git.push.assert_not_called() - self.mock_gh.update_issue_body.assert_not_called() - - def test_create_release_branch_already_checked(self): - # Arrange - args = MagicMock(issue=123, remote="my-remote") - self.mock_gh.get_issue_title.return_value = "Release 2.0.0" - self.mock_gh.get_issue_body.return_value = """ +""", + "labels": ["type: release"], + } + # Act + result = CreateReleaseBranch(args, mock_git, mock_gh).run() + + # Assert + assert result == 1 + mock_git.fetch.assert_not_called() + mock_git.push.assert_not_called() + + +def test_create_release_branch_already_checked(mock_git, mock_gh): + # Arrange + args = argparse.Namespace(issue=123, remote="my-remote") + mock_gh.issues[123] = { + "title": "Release 2.0.0", + "body": """ ## Checklist - [x] Prepare Release | status=done pr=#122 commit=abcdef12 - [x] Create Release branch | status=done branch=release/2.0 commit=abcdef12 -""" - # Act - result = CreateReleaseBranch(args, self.mock_git, self.mock_gh).run() - - # Assert - self.assertEqual(result, 0) - self.mock_git.fetch.assert_not_called() - self.mock_git.push.assert_not_called() - self.mock_gh.update_issue_body.assert_not_called() - - def test_create_release_branch_already_exists_same_commit(self): - # Arrange - args = MagicMock(issue=123, remote="my-remote") - self.mock_gh.get_issue_title.return_value = "Release 2.0.0" - self.mock_gh.get_issue_body.return_value = """ +""", + "labels": ["type: release"], + } + # Act + result = CreateReleaseBranch(args, mock_git, mock_gh).run() + + # Assert + assert result == 0 + mock_git.fetch.assert_not_called() + mock_git.push.assert_not_called() + + +def test_create_release_branch_already_exists_same_commit(mock_git, mock_gh): + # Arrange + args = argparse.Namespace(issue=123, remote="my-remote") + mock_gh.issues[123] = { + "title": "Release 2.0.0", + "body": """ ## Checklist - [x] Prepare Release | status=done pr=#122 commit=abcdef12 - [ ] Create Release branch | status=pending -""" - self.mock_git.remote_branch_exists.return_value = True - self.mock_git.get_commit_sha.return_value = "abcdef12" - - # Act - result = CreateReleaseBranch(args, self.mock_git, self.mock_gh).run() - - # Assert - self.assertEqual(result, 0) - self.mock_git.fetch.assert_called_once_with("my-remote") - self.mock_git.push.assert_not_called() - self.mock_gh.update_issue_body.assert_called_once() # Should still update checklist - - def test_create_release_branch_already_exists_fast_forward(self): - # Arrange - args = MagicMock(issue=123, remote="my-remote") - self.mock_gh.get_issue_title.return_value = "Release 2.0.0" - self.mock_gh.get_issue_body.return_value = """ +""", + "labels": ["type: release"], + } + mock_git.remote_branch_exists.return_value = True + mock_git.get_commit_sha.return_value = "abcdef12" + + # Act + result = CreateReleaseBranch(args, mock_git, mock_gh).run() + + # Assert + assert result == 0 + mock_git.fetch.assert_called_once_with("my-remote") + mock_git.push.assert_not_called() + + +def test_create_release_branch_already_exists_fast_forward(mock_git, mock_gh): + # Arrange + args = argparse.Namespace(issue=123, remote="my-remote") + mock_gh.issues[123] = { + "title": "Release 2.0.0", + "body": """ ## Checklist - [x] Prepare Release | status=done pr=#122 commit=abcdef12 - [ ] Create Release branch | status=pending -""" - self.mock_git.remote_branch_exists.return_value = True - self.mock_git.get_commit_sha.return_value = "oldcommit" - self.mock_git.is_ancestor.return_value = True - - # Act - result = CreateReleaseBranch(args, self.mock_git, self.mock_gh).run() - - # Assert - self.assertEqual(result, 0) - self.mock_git.fetch.assert_called_once_with("my-remote") - self.mock_git.push.assert_called_once_with( - "my-remote", "abcdef12:refs/heads/release/2.0" - ) - self.mock_gh.update_issue_body.assert_called_once() - - def test_create_release_branch_already_exists_non_ff(self): - # Arrange - args = MagicMock(issue=123, remote="my-remote") - self.mock_gh.get_issue_title.return_value = "Release 2.0.0" - self.mock_gh.get_issue_body.return_value = """ +""", + "labels": ["type: release"], + } + mock_git.remote_branch_exists.return_value = True + mock_git.get_commit_sha.return_value = "oldcommit" + mock_git.is_ancestor.return_value = True + + # Act + result = CreateReleaseBranch(args, mock_git, mock_gh).run() + + # Assert + assert result == 0 + mock_git.fetch.assert_called_once_with("my-remote") + mock_git.push.assert_called_once_with( + "my-remote", "abcdef12:refs/heads/release/2.0" + ) + + +def test_create_release_branch_already_exists_non_ff(mock_git, mock_gh): + # Arrange + args = argparse.Namespace(issue=123, remote="my-remote") + mock_gh.issues[123] = { + "title": "Release 2.0.0", + "body": """ ## Checklist - [x] Prepare Release | status=done pr=#122 commit=abcdef12 - [ ] Create Release branch | status=pending -""" - self.mock_git.remote_branch_exists.return_value = True - self.mock_git.get_commit_sha.return_value = "othercommit" - self.mock_git.is_ancestor.return_value = False - - # Act - result = CreateReleaseBranch(args, self.mock_git, self.mock_gh).run() - - # Assert - self.assertEqual(result, 1) - self.mock_git.fetch.assert_called_once_with("my-remote") - self.mock_git.push.assert_not_called() - self.mock_gh.update_issue_body.assert_not_called() - - -if __name__ == "__main__": - unittest.main() +""", + "labels": ["type: release"], + } + mock_git.remote_branch_exists.return_value = True + mock_git.get_commit_sha.return_value = "othercommit" + mock_git.is_ancestor.return_value = False + + # Act + result = CreateReleaseBranch(args, mock_git, mock_gh).run() + + # Assert + assert result == 1 + mock_git.fetch.assert_called_once_with("my-remote") + mock_git.push.assert_not_called() diff --git a/tests/tools/private/release/gh_test.py b/tests/tools/private/release/gh_test.py index 4010d2dca5..ccc61e7f21 100644 --- a/tests/tools/private/release/gh_test.py +++ b/tests/tools/private/release/gh_test.py @@ -1,61 +1,61 @@ -import unittest -from unittest.mock import patch +import pytest from tools.private.release.gh import GitHub +pytest_plugins = ["tests.tools.private.release.release_test_helper"] -class GitHubTest(unittest.TestCase): - def setUp(self): - self.gh = GitHub("my-owner/my-repo") - - @patch("tools.private.release.gh.run_cmd") - def test_resolve_pr_number_digit(self, mock_run_cmd): - # 124 and #125 should resolve immediately without running command - self.assertEqual(self.gh.resolve_pr_number("124"), 124) - self.assertEqual(self.gh.resolve_pr_number("#125"), 125) - mock_run_cmd.assert_not_called() - - @patch("tools.private.release.gh.run_cmd") - def test_resolve_pr_number_url_simple(self, mock_run_cmd): - url = "https://github.com/my-owner/my-repo/pull/126" - # Should resolve via regex without calling gh - result = self.gh.resolve_pr_number(url) - self.assertEqual(result, 126) - mock_run_cmd.assert_not_called() - - @patch("tools.private.release.gh.run_cmd") - def test_resolve_pr_number_url_with_subpath(self, mock_run_cmd): - url = "https://github.com/my-owner/my-repo/pull/126/files" - # Should resolve via regex without calling gh - result = self.gh.resolve_pr_number(url) - self.assertEqual(result, 126) - mock_run_cmd.assert_not_called() - - @patch("tools.private.release.gh.run_cmd") - def test_resolve_pr_number_url_with_query(self, mock_run_cmd): - url = "https://github.com/my-owner/my-repo/pull/126/files?w=1" - # Should resolve via regex without calling gh - result = self.gh.resolve_pr_number(url) - self.assertEqual(result, 126) - mock_run_cmd.assert_not_called() - - @patch("tools.private.release.gh.run_cmd") - def test_resolve_pr_number_url_other_repo(self, mock_run_cmd): - # URL for a different repo should fail immediately without calling gh - url = "https://github.com/other-owner/other-repo/pull/126" - with self.assertRaises(ValueError) as ctx: - self.gh.resolve_pr_number(url) - self.assertIn("URL is not for the configured repository", str(ctx.exception)) - mock_run_cmd.assert_not_called() - - @patch("tools.private.release.gh.run_cmd") - def test_resolve_pr_number_invalid_ref(self, mock_run_cmd): - # Invalid reference (not number, not URL) should fail - with self.assertRaises(ValueError) as ctx: - self.gh.resolve_pr_number("invalid-ref") - self.assertIn("Could not resolve PR reference", str(ctx.exception)) - mock_run_cmd.assert_not_called() - - -if __name__ == "__main__": - unittest.main() + +@pytest.fixture(name="gh") +def fixture_gh(): + return GitHub("my-owner/my-repo") + + +def test_resolve_pr_number_digit(mocker, gh): + mock_run_cmd = mocker.patch("tools.private.release.gh.run_cmd") + # 124 and #125 should resolve immediately without running command + assert gh.resolve_pr_number("124") == 124 + assert gh.resolve_pr_number("#125") == 125 + mock_run_cmd.assert_not_called() + + +def test_resolve_pr_number_url_simple(mocker, gh): + mock_run_cmd = mocker.patch("tools.private.release.gh.run_cmd") + url = "https://github.com/my-owner/my-repo/pull/126" + # Should resolve via regex without calling gh + result = gh.resolve_pr_number(url) + assert result == 126 + mock_run_cmd.assert_not_called() + + +def test_resolve_pr_number_url_with_subpath(mocker, gh): + mock_run_cmd = mocker.patch("tools.private.release.gh.run_cmd") + url = "https://github.com/my-owner/my-repo/pull/126/files" + # Should resolve via regex without calling gh + result = gh.resolve_pr_number(url) + assert result == 126 + mock_run_cmd.assert_not_called() + + +def test_resolve_pr_number_url_with_query(mocker, gh): + mock_run_cmd = mocker.patch("tools.private.release.gh.run_cmd") + url = "https://github.com/my-owner/my-repo/pull/126/files?w=1" + # Should resolve via regex without calling gh + result = gh.resolve_pr_number(url) + assert result == 126 + mock_run_cmd.assert_not_called() + + +def test_resolve_pr_number_url_other_repo(mocker, gh): + mock_run_cmd = mocker.patch("tools.private.release.gh.run_cmd") + # URL for a different repo should fail immediately without calling gh + url = "https://github.com/other-owner/other-repo/pull/126" + with pytest.raises(ValueError, match="URL is not for the configured repository"): + gh.resolve_pr_number(url) + mock_run_cmd.assert_not_called() + + +def test_resolve_pr_number_invalid(mocker, gh): + mock_run_cmd = mocker.patch("tools.private.release.gh.run_cmd") + with pytest.raises(ValueError, match="Could not resolve PR reference"): + gh.resolve_pr_number("invalid-ref") + mock_run_cmd.assert_not_called() diff --git a/tests/tools/private/release/git_test.py b/tests/tools/private/release/git_test.py index 34643fc016..4a8cada4cc 100644 --- a/tests/tools/private/release/git_test.py +++ b/tests/tools/private/release/git_test.py @@ -1,189 +1,154 @@ -import unittest -from unittest.mock import patch +import subprocess + +import pytest from tools.private.release.git import Git +pytest_plugins = ["tests.tools.private.release.release_test_helper"] + + +@pytest.fixture(name="git_obj") +def fixture_git_obj(mocker): + git = Git(".") + git.mock_run_git = mocker.patch.object(git, "_run_git") + return git + + +def test_checkout_simple(git_obj): + git_obj.checkout("my-branch") + git_obj.mock_run_git.assert_called_once_with( + "checkout", "my-branch", capture_output=False + ) + + +def test_checkout_track_remote_new_branch(mocker, git_obj): + mock_branch_exists = mocker.patch( + "tools.private.release.git.Git.branch_exists", return_value=False + ) + + git_obj.checkout("my-branch", track_remote="origin") + + mock_branch_exists.assert_called_once_with("my-branch") + git_obj.mock_run_git.assert_called_once_with( + "checkout", "--track", "origin/my-branch", capture_output=False + ) + + +def test_checkout_track_remote_existing_branch(mocker, git_obj): + mock_branch_exists = mocker.patch( + "tools.private.release.git.Git.branch_exists", return_value=True + ) + mock_reset_hard = mocker.patch("tools.private.release.git.Git.reset_hard") + + git_obj.checkout("my-branch", track_remote="origin") + + mock_branch_exists.assert_called_once_with("my-branch") + git_obj.mock_run_git.assert_called_once_with( + "checkout", "my-branch", capture_output=False + ) + mock_reset_hard.assert_called_once_with(reset_to="origin/my-branch") + + +def test_fetch_default(git_obj): + git_obj.fetch() + git_obj.mock_run_git.assert_called_once_with( + "fetch", "origin", capture_output=False + ) + + +def test_fetch_custom_remote(git_obj): + git_obj.fetch("upstream") + git_obj.mock_run_git.assert_called_once_with( + "fetch", "upstream", capture_output=False + ) + + +def test_fetch_with_refspec(git_obj): + git_obj.fetch("origin", refspec="my-branch") + git_obj.mock_run_git.assert_called_once_with( + "fetch", "origin", "my-branch", capture_output=False + ) + + +def test_fetch_with_tags_and_force(git_obj): + git_obj.fetch("origin", tags=True, force=True) + git_obj.mock_run_git.assert_called_once_with( + "fetch", "origin", "--tags", "--force", capture_output=False + ) + + +def test_fetch_all_options(git_obj): + git_obj.fetch("origin", refspec="my-branch", tags=True, force=True) + git_obj.mock_run_git.assert_called_once_with( + "fetch", "origin", "my-branch", "--tags", "--force", capture_output=False + ) + + +def test_get_modified_files(git_obj): + git_obj.mock_run_git.return_value = "file1.txt\nfile2.py\n\n" + files = git_obj.get_modified_files("HEAD") + git_obj.mock_run_git.assert_called_once_with( + "show", "--name-only", "--format=", "HEAD" + ) + assert files == ["file1.txt", "file2.py"] + + +def test_get_modified_files_empty(git_obj): + git_obj.mock_run_git.return_value = "" + files = git_obj.get_modified_files("HEAD") + assert files == [] + + +def test_diff_has_changes(git_obj): + git_obj.mock_run_git.return_value = "some diff output" + output = git_obj.diff() + git_obj.mock_run_git.assert_called_once_with("diff") + assert output == "some diff output" + + +def test_diff_empty(git_obj): + git_obj.mock_run_git.return_value = "" + output = git_obj.diff() + git_obj.mock_run_git.assert_called_once_with("diff") + assert output == "" + + +def test_apply(git_obj): + git_obj.apply("patch.patch") + git_obj.mock_run_git.assert_called_once_with( + "apply", "patch.patch", capture_output=False + ) + + +def test_apply_check_clean(git_obj): + git_obj.mock_run_git.return_value = "" + result = git_obj.apply_check("patch.patch") + git_obj.mock_run_git.assert_called_once_with( + "apply", "--check", "patch.patch", capture_output=False + ) + assert result is True + + +def test_apply_check_conflict(git_obj): + git_obj.mock_run_git.side_effect = subprocess.CalledProcessError( + 1, ["git", "apply", "--check", "patch.patch"] + ) + result = git_obj.apply_check("patch.patch") + git_obj.mock_run_git.assert_called_once_with( + "apply", "--check", "patch.patch", capture_output=False + ) + assert result is False + + +def test_reset_hard_default(git_obj): + git_obj.reset_hard() + git_obj.mock_run_git.assert_called_once_with( + "reset", "--hard", "HEAD", capture_output=False + ) + -class GitCheckoutTest(unittest.TestCase): - def setUp(self): - self.git = Git(".") - self.patcher = patch.object(self.git, "_run_git") - self.mock_run_git = self.patcher.start() - self.addCleanup(self.patcher.stop) - - def test_checkout_simple(self): - self.git.checkout("my-branch") - self.mock_run_git.assert_called_once_with( - "checkout", "my-branch", capture_output=False - ) - - @patch("tools.private.release.git.Git.branch_exists") - def test_checkout_track_remote_new_branch(self, mock_branch_exists): - mock_branch_exists.return_value = False - - self.git.checkout("my-branch", track_remote="origin") - - mock_branch_exists.assert_called_once_with("my-branch") - self.mock_run_git.assert_called_once_with( - "checkout", "--track", "origin/my-branch", capture_output=False - ) - - @patch("tools.private.release.git.Git.reset_hard") - @patch("tools.private.release.git.Git.branch_exists") - def test_checkout_track_remote_existing_branch( - self, mock_branch_exists, mock_reset_hard - ): - mock_branch_exists.return_value = True - - self.git.checkout("my-branch", track_remote="origin") - - mock_branch_exists.assert_called_once_with("my-branch") - self.mock_run_git.assert_called_once_with( - "checkout", "my-branch", capture_output=False - ) - mock_reset_hard.assert_called_once_with(reset_to="origin/my-branch") - - -class GitFetchTest(unittest.TestCase): - def setUp(self): - self.git = Git(".") - self.patcher = patch.object(self.git, "_run_git") - self.mock_run_git = self.patcher.start() - self.addCleanup(self.patcher.stop) - - def test_fetch_default(self): - self.git.fetch() - self.mock_run_git.assert_called_once_with( - "fetch", "origin", capture_output=False - ) - - def test_fetch_custom_remote(self): - self.git.fetch("upstream") - self.mock_run_git.assert_called_once_with( - "fetch", "upstream", capture_output=False - ) - - def test_fetch_with_refspec(self): - self.git.fetch("origin", refspec="my-branch") - self.mock_run_git.assert_called_once_with( - "fetch", "origin", "my-branch", capture_output=False - ) - - def test_fetch_with_tags_and_force(self): - self.git.fetch("origin", tags=True, force=True) - self.mock_run_git.assert_called_once_with( - "fetch", "origin", "--tags", "--force", capture_output=False - ) - - def test_fetch_all_options(self): - self.git.fetch("origin", refspec="my-branch", tags=True, force=True) - self.mock_run_git.assert_called_once_with( - "fetch", "origin", "my-branch", "--tags", "--force", capture_output=False - ) - - -class GitGetModifiedFilesTest(unittest.TestCase): - def setUp(self): - self.git = Git(".") - self.patcher = patch.object(self.git, "_run_git") - self.mock_run_git = self.patcher.start() - self.addCleanup(self.patcher.stop) - - def test_get_modified_files(self): - self.mock_run_git.return_value = "file1.txt\nfile2.py\n\n" - files = self.git.get_modified_files("HEAD") - self.mock_run_git.assert_called_once_with( - "show", "--name-only", "--format=", "HEAD" - ) - self.assertEqual(files, ["file1.txt", "file2.py"]) - - def test_get_modified_files_empty(self): - self.mock_run_git.return_value = "" - files = self.git.get_modified_files("HEAD") - self.assertEqual(files, []) - - -class GitDiffTest(unittest.TestCase): - def setUp(self): - self.git = Git(".") - self.patcher = patch.object(self.git, "_run_git") - self.mock_run_git = self.patcher.start() - self.addCleanup(self.patcher.stop) - - def test_diff_has_changes(self): - self.mock_run_git.return_value = "some diff output" - output = self.git.diff() - self.mock_run_git.assert_called_once_with("diff") - self.assertEqual(output, "some diff output") - - def test_diff_empty(self): - self.mock_run_git.return_value = "" - output = self.git.diff() - self.mock_run_git.assert_called_once_with("diff") - self.assertEqual(output, "") - - -class GitApplyTest(unittest.TestCase): - def setUp(self): - self.git = Git(".") - self.patcher = patch.object(self.git, "_run_git") - self.mock_run_git = self.patcher.start() - self.addCleanup(self.patcher.stop) - - def test_apply(self): - self.git.apply("patch.patch") - self.mock_run_git.assert_called_once_with( - "apply", "patch.patch", capture_output=False - ) - - -class GitApplyCheckTest(unittest.TestCase): - def setUp(self): - self.git = Git(".") - self.patcher = patch.object(self.git, "_run_git") - self.mock_run_git = self.patcher.start() - self.addCleanup(self.patcher.stop) - - def test_apply_check_clean(self): - self.mock_run_git.return_value = "" - result = self.git.apply_check("patch.patch") - self.mock_run_git.assert_called_once_with( - "apply", "--check", "patch.patch", capture_output=False - ) - self.assertTrue(result) - - def test_apply_check_conflict(self): - import subprocess - - self.mock_run_git.side_effect = subprocess.CalledProcessError( - 1, ["git", "apply", "--check", "patch.patch"] - ) - result = self.git.apply_check("patch.patch") - self.mock_run_git.assert_called_once_with( - "apply", "--check", "patch.patch", capture_output=False - ) - self.assertFalse(result) - - -class GitResetHardTest(unittest.TestCase): - def setUp(self): - self.git = Git(".") - self.patcher = patch.object(self.git, "_run_git") - self.mock_run_git = self.patcher.start() - self.addCleanup(self.patcher.stop) - - def test_reset_hard_default(self): - self.git.reset_hard() - self.mock_run_git.assert_called_once_with( - "reset", "--hard", "HEAD", capture_output=False - ) - - def test_reset_hard_custom(self): - self.git.reset_hard(reset_to="my-commit") - self.mock_run_git.assert_called_once_with( - "reset", "--hard", "my-commit", capture_output=False - ) - - -if __name__ == "__main__": - unittest.main() +def test_reset_hard_custom(git_obj): + git_obj.reset_hard(reset_to="my-commit") + git_obj.mock_run_git.assert_called_once_with( + "reset", "--hard", "my-commit", capture_output=False + ) diff --git a/tests/tools/private/release/on_pr_merged_test.py b/tests/tools/private/release/on_pr_merged_test.py index a5644c5705..df3698d1dd 100644 --- a/tests/tools/private/release/on_pr_merged_test.py +++ b/tests/tools/private/release/on_pr_merged_test.py @@ -1,112 +1,92 @@ import argparse -import unittest -from unittest.mock import MagicMock, patch +from unittest.mock import MagicMock -from tests.tools.private.release.release_test_helper import _mock_git_and_gh from tools.private.release.on_pr_merged import OnPrMerged +pytest_plugins = ["tests.tools.private.release.release_test_helper"] -class CmdOnPrMergedTest(unittest.TestCase): - def setUp(self): - _mock_git_and_gh(self) - self.addCleanup(patch.stopall) - # Mock ProcessBackports - self.mock_process_patcher = patch( - "tools.private.release.on_pr_merged.ProcessBackports" - ) - self.mock_process_class = self.mock_process_patcher.start() - self.mock_process_instance = MagicMock() - self.mock_process_class.return_value = self.mock_process_instance - - def test_on_pr_merged_no_comment(self): - args = argparse.Namespace(pr=124, remote="origin", dry_run=True) - self.mock_gh.get_pr_comments.return_value = [ +def test_on_pr_merged_no_comment(mocker, mock_git, mock_gh): + args = argparse.Namespace(pr=124, remote="origin", dry_run=True) + mock_gh.pr_comments = { + 124: [ {"body": "Some comment"}, {"body": "Another comment /backport_wrong"}, ] + } - result = OnPrMerged(args, self.mock_git, self.mock_gh).run() + mock_pb = mocker.patch("tools.private.release.on_pr_merged.ProcessBackports") + result = OnPrMerged(args, mock_git, mock_gh).run() - self.assertEqual(result, 1) - self.mock_gh.get_pr_comments.assert_called_once_with(124) - self.mock_gh.get_open_tracking_issues.assert_not_called() - self.mock_process_class.assert_not_called() + assert result == 1 + mock_pb.assert_not_called() - def test_on_pr_merged_has_comment_no_active_release(self): - args = argparse.Namespace(pr=124, remote="origin", dry_run=True) - self.mock_gh.get_pr_comments.return_value = [ - {"body": "/backport"}, - ] - self.mock_gh.get_open_tracking_issues.return_value = [] - result = OnPrMerged(args, self.mock_git, self.mock_gh).run() +def test_on_pr_merged_has_comment_no_active_release(mocker, mock_git, mock_gh): + args = argparse.Namespace(pr=124, remote="origin", dry_run=True) + mock_gh.pr_comments = {124: [{"body": "/backport"}]} - self.assertEqual(result, 1) - self.mock_gh.get_pr_comments.assert_called_once_with(124) - self.mock_gh.get_open_tracking_issues.assert_called_once() - self.mock_gh.get_issue_body.assert_not_called() - self.mock_process_class.assert_not_called() + mock_pb = mocker.patch("tools.private.release.on_pr_merged.ProcessBackports") + result = OnPrMerged(args, mock_git, mock_gh).run() - def test_on_pr_merged_has_comment_not_in_backports(self): - args = argparse.Namespace(pr=124, remote="origin", dry_run=True) - self.mock_gh.get_pr_comments.return_value = [ - {"body": " /backport "}, - ] - self.mock_gh.get_open_tracking_issues.return_value = [ - {"number": 456, "title": "Release 2.1.0", "url": "http://..."} - ] - self.mock_gh.get_issue_body.return_value = """ -## Checklist -- [ ] Prepare Release + assert result == 1 + mock_pb.assert_not_called() -## Backports -- [ ] #125 | status=pending -""" - result = OnPrMerged(args, self.mock_git, self.mock_gh).run() - - self.assertEqual(result, 1) - self.mock_gh.get_pr_comments.assert_called_once_with(124) - self.mock_gh.get_open_tracking_issues.assert_called_once() - self.mock_gh.get_issue_body.assert_called_once_with(456) - self.mock_process_class.assert_not_called() - - def test_on_pr_merged_success(self): - args = argparse.Namespace(pr=124, remote="origin", dry_run=True) - self.mock_gh.get_pr_comments.return_value = [ - {"body": "/backport"}, - ] - self.mock_gh.get_open_tracking_issues.return_value = [ - {"number": 456, "title": "Release 2.1.0", "url": "http://..."} - ] - self.mock_gh.get_issue_body.return_value = """ + +def test_on_pr_merged_has_comment_not_in_backports(mocker, mock_git, mock_gh): + args = argparse.Namespace(pr=124, remote="origin", dry_run=True) + mock_gh.pr_comments = {124: [{"body": " /backport "}]} + mock_gh.create_issue( + title="Release 2.1.0", + body=""" ## Checklist - [ ] Prepare Release ## Backports -- [ ] #124 | status=pending -""" - self.mock_process_instance.run.return_value = 0 - - result = OnPrMerged(args, self.mock_git, self.mock_gh).run() +- [ ] #125 | status=pending +""", + labels=["type: release"], + ) - self.assertEqual(result, 0) - self.mock_gh.get_pr_comments.assert_called_once_with(124) - self.mock_gh.get_open_tracking_issues.assert_called_once() - self.mock_gh.get_issue_body.assert_called_once_with(456) + mock_pb = mocker.patch("tools.private.release.on_pr_merged.ProcessBackports") + result = OnPrMerged(args, mock_git, mock_gh).run() - # Verify ProcessBackports was instantiated with correct args - self.mock_process_class.assert_called_once() - called_args = self.mock_process_class.call_args[0][0] - self.assertEqual(called_args.issue, 456) - self.assertEqual(called_args.remote, "origin") - self.assertEqual(called_args.dry_run, True) - self.assertIsNone(called_args.add) - self.assertIsNone(called_args.triggering_comment) + assert result == 1 + mock_pb.assert_not_called() - # Verify ProcessBackports.run() was called - self.mock_process_instance.run.assert_called_once() +def test_on_pr_merged_success(mocker, mock_git, mock_gh): + args = argparse.Namespace(pr=124, remote="origin", dry_run=True) + mock_gh.pr_comments = {124: [{"body": "/backport"}]} + issue_num = mock_gh.create_issue( + title="Release 2.1.0", + body=""" +## Checklist +- [ ] Prepare Release -if __name__ == "__main__": - unittest.main() +## Backports +- [ ] #124 | status=pending +""", + labels=["type: release"], + ) + + mock_pb_class = mocker.patch("tools.private.release.on_pr_merged.ProcessBackports") + mock_pb_instance = MagicMock() + mock_pb_instance.run.return_value = 0 + mock_pb_class.return_value = mock_pb_instance + + result = OnPrMerged(args, mock_git, mock_gh).run() + + assert result == 0 + + # Verify ProcessBackports was instantiated with correct args + mock_pb_class.assert_called_once() + called_args = mock_pb_class.call_args[0][0] + assert called_args.issue == issue_num + assert called_args.remote == "origin" + assert called_args.dry_run is True + assert called_args.add is None + assert called_args.triggering_comment is None + + # Verify ProcessBackports.run() was called + mock_pb_instance.run.assert_called_once() diff --git a/tests/tools/private/release/prepare_test.py b/tests/tools/private/release/prepare_test.py index 45b92a636f..3a3e886f45 100644 --- a/tests/tools/private/release/prepare_test.py +++ b/tests/tools/private/release/prepare_test.py @@ -1,250 +1,207 @@ -import unittest -from unittest.mock import MagicMock, patch - -from tests.tools.private.release.release_test_helper import ( - TempDirTestCase, - _mock_git_and_gh, -) -from tools.private.release.gh import ( - MultipleTrackingIssuesError, - NoTrackingIssueError, -) -from tools.private.release.prepare import Prepare +import argparse +from tools.private.release.prepare import Prepare -class CmdPrepareTest(TempDirTestCase): - def setUp(self): - super().setUp() - _mock_git_and_gh(self) - - @patch("tools.private.release.prepare.changelog_news") - @patch("tools.private.release.prepare.replace_version_next") - def test_prepare_success_existing_issue(self, mock_replace, mock_changelog): - # Arrange - args = MagicMock(version="2.0.0", issue=None, dry_run=False) - self.mock_git.status.side_effect = ["", "M foo"] - self.mock_git.branch_exists.return_value = False - self.mock_gh.get_release_tracking_issue.side_effect = None - self.mock_gh.get_release_tracking_issue.return_value = 123 - self.mock_gh.create_pr.return_value = "https://github.com/foo/bar/pull/456" - self.mock_gh.get_issue_body.return_value = "- [ ] Prepare Release" - - # Act - result = Prepare(args, self.mock_git, self.mock_gh).run() - - # Assert - self.assertEqual(result, 0) - self.mock_gh.get_release_tracking_issue.assert_called_once_with("2.0.0") - self.mock_gh.create_tracking_issue.assert_not_called() - self.mock_gh.create_pr.assert_called_once_with( - title="Prepare release v2.0.0", - body="Work towards #123", - base="main", - ) - self.mock_git.add_modified_and_deleted.assert_called_once() - - @patch("tools.private.release.prepare.changelog_news") - @patch("tools.private.release.prepare.replace_version_next") - def test_prepare_success_create_issue(self, mock_replace, mock_changelog): - # Arrange - template_dir = self.tmpdir / ".github" / "ISSUE_TEMPLATE" - template_dir.mkdir(parents=True, exist_ok=True) - template_file = template_dir / "release_tracking_template.md" - template_file.write_text("dummy template content") - - args = MagicMock(version="2.0.0", issue=None, dry_run=False) - self.mock_git.status.side_effect = ["", "M foo"] - self.mock_git.branch_exists.return_value = False - self.mock_gh.get_release_tracking_issue.side_effect = NoTrackingIssueError( - "Not found" - ) - self.mock_gh.create_tracking_issue.return_value = 123 - self.mock_gh.create_pr.return_value = "https://github.com/foo/bar/pull/456" - self.mock_gh.get_issue_body.return_value = "- [ ] Prepare Release" - - # Act - result = Prepare(args, self.mock_git, self.mock_gh).run() - - # Assert - self.assertEqual(result, 0) - self.mock_gh.get_release_tracking_issue.assert_called_once_with("2.0.0") - self.mock_gh.create_tracking_issue.assert_called_once_with( - "2.0.0", "dummy template content" - ) - self.mock_gh.create_pr.assert_called_once_with( - title="Prepare release v2.0.0", - body="Work towards #123", - base="main", - ) - self.mock_git.add_modified_and_deleted.assert_called_once() - - @patch("tools.private.release.prepare.changelog_news") - @patch("tools.private.release.prepare.replace_version_next") - def test_prepare_ambiguous_issue(self, mock_replace, mock_changelog): - # Arrange - args = MagicMock(version="2.0.0", issue=None, dry_run=False) - self.mock_git.status.side_effect = ["", "M foo"] - self.mock_git.branch_exists.return_value = False - self.mock_gh.get_release_tracking_issue.side_effect = ( - MultipleTrackingIssuesError("Multiple open tracking issues") - ) - - # Act - result = Prepare(args, self.mock_git, self.mock_gh).run() - - # Assert - self.assertEqual(result, 1) - self.mock_gh.get_release_tracking_issue.assert_called_once_with("2.0.0") - self.mock_gh.create_tracking_issue.assert_not_called() - self.mock_gh.create_pr.assert_not_called() - self.mock_git.add_modified_and_deleted.assert_not_called() - - @patch("tools.private.release.prepare.changelog_news") - @patch("tools.private.release.prepare.replace_version_next") - def test_prepare_dry_run(self, mock_replace, mock_changelog): - # Arrange - args = MagicMock(version="2.0.0", issue=None, dry_run=True) - self.mock_git.status.side_effect = [""] - self.mock_gh.get_release_tracking_issue.side_effect = None - self.mock_gh.get_release_tracking_issue.return_value = 123 - - # Act - result = Prepare(args, self.mock_git, self.mock_gh).run() - - # Assert - self.assertEqual(result, 0) - self.mock_git.checkout.assert_not_called() - self.mock_git.commit.assert_not_called() - self.mock_git.push.assert_not_called() - self.mock_gh.create_pr.assert_not_called() - self.mock_gh.update_issue_body.assert_not_called() - self.mock_git.fetch.assert_called_once() - self.mock_gh.get_release_tracking_issue.assert_called_once_with("2.0.0") - self.mock_git.add_modified_and_deleted.assert_not_called() - - @patch("tools.private.release.prepare.changelog_news") - @patch("tools.private.release.prepare.replace_version_next") - def test_prepare_use_associated_pr_from_tracking_issue( - self, mock_replace, mock_changelog - ): - # Arrange - args = MagicMock(version="2.0.0", issue=None, dry_run=False) - self.mock_git.status.side_effect = ["", ""] - self.mock_git.branch_exists.return_value = True - self.mock_gh.get_release_tracking_issue.side_effect = None - self.mock_gh.get_release_tracking_issue.return_value = 123 - self.mock_gh.get_open_pr.return_value = None - # PR #456 is already associated in the tracking issue - self.mock_gh.get_issue_body.return_value = ( - "- [ ] Prepare Release | status=pending pr=#456" - ) - - # Act - result = Prepare(args, self.mock_git, self.mock_gh).run() - - # Assert - self.assertEqual(result, 0) - self.mock_git.checkout.assert_called_once_with("prepare-2.0.0") - self.mock_git.commit.assert_not_called() - self.mock_git.push.assert_called_once_with( - "origin", "prepare-2.0.0", set_upstream=True, force=True - ) - self.mock_gh.get_open_pr.assert_called_once_with("prepare-2.0.0") - self.mock_gh.create_pr.assert_not_called() # Should NOT create a new PR - self.mock_gh.update_issue_body.assert_called_once() - call_args = self.mock_gh.update_issue_body.call_args[0] - self.assertIn("pr=#456", call_args[1]) - - @patch("tools.private.release.prepare.changelog_news") - @patch("tools.private.release.prepare.replace_version_next") - def test_prepare_create_pr_when_none_associated(self, mock_replace, mock_changelog): - # Arrange - args = MagicMock(version="2.0.0", issue=None, dry_run=False) - self.mock_git.status.side_effect = ["", ""] - self.mock_git.branch_exists.return_value = True - self.mock_gh.get_release_tracking_issue.side_effect = None - self.mock_gh.get_release_tracking_issue.return_value = 123 - self.mock_gh.get_open_pr.return_value = None - # No PR associated in the tracking issue - self.mock_gh.get_issue_body.return_value = "- [ ] Prepare Release" - self.mock_gh.create_pr.return_value = "https://github.com/foo/bar/pull/789" - - # Act - result = Prepare(args, self.mock_git, self.mock_gh).run() - - # Assert - self.assertEqual(result, 0) - self.mock_git.checkout.assert_called_once_with("prepare-2.0.0") - self.mock_git.commit.assert_not_called() - self.mock_git.push.assert_called_once_with( - "origin", "prepare-2.0.0", set_upstream=True, force=True - ) - self.mock_gh.get_open_pr.assert_called_once_with("prepare-2.0.0") - self.mock_gh.create_pr.assert_called_once_with( - title="Prepare release v2.0.0", - body="Work towards #123", - base="main", - ) - self.mock_gh.update_issue_body.assert_called_once() - call_args = self.mock_gh.update_issue_body.call_args[0] - self.assertIn("pr=#789", call_args[1]) - - @patch("tools.private.release.prepare.changelog_news") - @patch("tools.private.release.prepare.replace_version_next") - def test_prepare_reuse_existing_pr(self, mock_replace, mock_changelog): - # Arrange - args = MagicMock(version="2.0.0", issue=None, dry_run=False) - self.mock_git.status.side_effect = ["", ""] - self.mock_git.branch_exists.return_value = True - self.mock_gh.get_release_tracking_issue.side_effect = None - self.mock_gh.get_release_tracking_issue.return_value = 123 - self.mock_gh.get_open_pr.return_value = { - "number": 456, - "url": "https://github.com/foo/bar/pull/456", - } - self.mock_gh.get_issue_body.return_value = "- [ ] Prepare Release" - - # Act - result = Prepare(args, self.mock_git, self.mock_gh).run() - - # Assert - self.assertEqual(result, 0) - self.mock_git.checkout.assert_called_once_with("prepare-2.0.0") - self.mock_git.commit.assert_not_called() - self.mock_git.push.assert_called_once_with( - "origin", "prepare-2.0.0", set_upstream=True, force=True - ) - self.mock_gh.get_open_pr.assert_called_once_with("prepare-2.0.0") - self.mock_gh.create_pr.assert_not_called() - self.mock_gh.update_issue_body.assert_called_once() - call_args = self.mock_gh.update_issue_body.call_args[0] - self.assertIn("pr=#456", call_args[1]) - - @patch("tools.private.release.prepare.changelog_news") - @patch("tools.private.release.prepare.replace_version_next") - def test_prepare_dry_run_no_issue(self, mock_replace, mock_changelog): - # Arrange - template_dir = self.tmpdir / ".github" / "ISSUE_TEMPLATE" - template_dir.mkdir(parents=True, exist_ok=True) - template_file = template_dir / "release_tracking_template.md" - template_file.write_text("dummy template content") - - args = MagicMock(version="2.0.0", issue=None, dry_run=True) - self.mock_git.status.side_effect = [""] - self.mock_gh.get_release_tracking_issue.side_effect = NoTrackingIssueError( - "Not found" - ) - - # Act - result = Prepare(args, self.mock_git, self.mock_gh).run() - - # Assert - self.assertEqual(result, 0) - self.mock_git.checkout.assert_not_called() - self.mock_gh.create_tracking_issue.assert_not_called() - self.mock_gh.create_pr.assert_not_called() - self.mock_git.add_modified_and_deleted.assert_not_called() - - -if __name__ == "__main__": - unittest.main() +pytest_plugins = ["tests.tools.private.release.release_test_helper"] + + +def test_prepare_success_existing_issue(mocker, release_tool_env, mock_git, mock_gh): + mocker.patch("tools.private.release.prepare.replace_version_next") + mocker.patch("tools.private.release.prepare.changelog_news") + + # Arrange + args = argparse.Namespace(version="2.0.0", issue=None, dry_run=False) + mock_gh.create_issue( + title="Release 2.0.0", + body="- [ ] Prepare Release", + labels=["type: release"], + ) # Assigns issue 1001 + mock_git.status.side_effect = ["", "M foo"] + mock_git.branch_exists.return_value = False + + # Act + result = Prepare(args, mock_git, mock_gh).run() + + # Assert + assert result == 0 + assert 1002 in mock_gh.prs + assert mock_gh.prs[1002]["title"] == "Prepare release v2.0.0" + assert mock_gh.prs[1002]["body"] == "Work towards #1001" + mock_git.add_modified_and_deleted.assert_called_once() + + +def test_prepare_success_create_issue(mocker, release_tool_env, mock_git, mock_gh): + mocker.patch("tools.private.release.prepare.replace_version_next") + mocker.patch("tools.private.release.prepare.changelog_news") + + # Arrange: release_tool_env sets up template_file automatically + args = argparse.Namespace(version="2.0.0", issue=None, dry_run=False) + mock_git.status.side_effect = ["", "M foo"] + mock_git.branch_exists.return_value = False + + # Act + result = Prepare(args, mock_git, mock_gh).run() + + # Assert + assert result == 0 + assert 1001 in mock_gh.issues + assert mock_gh.issues[1001]["title"] == "Release 2.0.0" + assert 1002 in mock_gh.prs + assert mock_gh.prs[1002]["title"] == "Prepare release v2.0.0" + assert mock_gh.prs[1002]["body"] == "Work towards #1001" + mock_git.add_modified_and_deleted.assert_called_once() + + +def test_prepare_ambiguous_issue(mocker, release_tool_env, mock_git, mock_gh): + mocker.patch("tools.private.release.prepare.replace_version_next") + mocker.patch("tools.private.release.prepare.changelog_news") + + # Arrange + args = argparse.Namespace(version="2.0.0", issue=None, dry_run=False) + mock_gh.create_issue( + title="Release 2.0.0", body="issue 1", labels=["type: release"] + ) + mock_gh.create_issue( + title="Release 2.0.0", body="issue 2", labels=["type: release"] + ) + mock_git.status.side_effect = ["", "M foo"] + mock_git.branch_exists.return_value = False + + # Act + result = Prepare(args, mock_git, mock_gh).run() + + # Assert + assert result == 1 + mock_git.add_modified_and_deleted.assert_not_called() + + +def test_prepare_dry_run(mocker, release_tool_env, mock_git, mock_gh): + mocker.patch("tools.private.release.prepare.replace_version_next") + mocker.patch("tools.private.release.prepare.changelog_news") + + # Arrange + args = argparse.Namespace(version="2.0.0", issue=None, dry_run=True) + mock_gh.create_issue(title="Release 2.0.0", body="body", labels=["type: release"]) + mock_git.status.side_effect = [""] + + # Act + result = Prepare(args, mock_git, mock_gh).run() + + # Assert + assert result == 0 + mock_git.checkout.assert_not_called() + mock_git.commit.assert_not_called() + mock_git.push.assert_not_called() + mock_git.fetch.assert_called_once() + mock_git.add_modified_and_deleted.assert_not_called() + + +def test_prepare_use_associated_pr_from_tracking_issue( + mocker, release_tool_env, mock_git, mock_gh +): + mocker.patch("tools.private.release.prepare.replace_version_next") + mocker.patch("tools.private.release.prepare.changelog_news") + + # Arrange + args = argparse.Namespace(version="2.0.0", issue=None, dry_run=False) + mock_gh.create_issue( + title="Release 2.0.0", + body="- [ ] Prepare Release | status=pending pr=#456", + labels=["type: release"], + ) + mock_git.status.side_effect = ["", ""] + mock_git.branch_exists.return_value = True + + # Act + result = Prepare(args, mock_git, mock_gh).run() + + # Assert + assert result == 0 + mock_git.checkout.assert_called_once_with("prepare-2.0.0") + mock_git.commit.assert_not_called() + mock_git.push.assert_called_once_with( + "origin", "prepare-2.0.0", set_upstream=True, force=True + ) + updated_body = mock_gh.get_issue_body(1001) + assert "pr=#456" in updated_body + + +def test_prepare_create_pr_when_none_associated( + mocker, release_tool_env, mock_git, mock_gh +): + mocker.patch("tools.private.release.prepare.replace_version_next") + mocker.patch("tools.private.release.prepare.changelog_news") + + # Arrange + args = argparse.Namespace(version="2.0.0", issue=None, dry_run=False) + mock_gh.create_issue( + title="Release 2.0.0", + body="- [ ] Prepare Release", + labels=["type: release"], + ) + mock_git.status.side_effect = ["", ""] + mock_git.branch_exists.return_value = True + + # Act + result = Prepare(args, mock_git, mock_gh).run() + + # Assert + assert result == 0 + mock_git.checkout.assert_called_once_with("prepare-2.0.0") + mock_git.commit.assert_not_called() + mock_git.push.assert_called_once_with( + "origin", "prepare-2.0.0", set_upstream=True, force=True + ) + updated_body = mock_gh.get_issue_body(1001) + assert "pr=#1002" in updated_body + + +def test_prepare_reuse_existing_pr(mocker, release_tool_env, mock_git, mock_gh): + mocker.patch("tools.private.release.prepare.replace_version_next") + mocker.patch("tools.private.release.prepare.changelog_news") + + # Arrange + args = argparse.Namespace(version="2.0.0", issue=None, dry_run=False) + mock_gh.create_issue( + title="Release 2.0.0", + body="- [ ] Prepare Release", + labels=["type: release"], + ) + mock_gh.prs[456] = { + "number": 456, + "head": "prepare-2.0.0", + "state": "OPEN", + "url": "https://github.com/foo/bar/pull/456", + } + mock_git.status.side_effect = ["", ""] + mock_git.branch_exists.return_value = True + + # Act + result = Prepare(args, mock_git, mock_gh).run() + + # Assert + assert result == 0 + mock_git.checkout.assert_called_once_with("prepare-2.0.0") + mock_git.commit.assert_not_called() + mock_git.push.assert_called_once_with( + "origin", "prepare-2.0.0", set_upstream=True, force=True + ) + updated_body = mock_gh.get_issue_body(1001) + assert "pr=#456" in updated_body + + +def test_prepare_dry_run_no_issue(mocker, release_tool_env, mock_git, mock_gh): + mocker.patch("tools.private.release.prepare.replace_version_next") + mocker.patch("tools.private.release.prepare.changelog_news") + + # Arrange + args = argparse.Namespace(version="2.0.0", issue=None, dry_run=True) + mock_git.status.side_effect = [""] + + # Act + result = Prepare(args, mock_git, mock_gh).run() + + # Assert + assert result == 0 + mock_git.checkout.assert_not_called() + mock_git.add_modified_and_deleted.assert_not_called() diff --git a/tests/tools/private/release/process_backports_test.py b/tests/tools/private/release/process_backports_test.py index d5df43ae46..5c37e63a97 100644 --- a/tests/tools/private/release/process_backports_test.py +++ b/tests/tools/private/release/process_backports_test.py @@ -1,60 +1,44 @@ import argparse import datetime -import unittest -from unittest.mock import call, patch +from unittest.mock import ANY, call -from tests.tools.private.release.release_test_helper import _mock_git_and_gh from tools.private.release.process_backports import ProcessBackports - -class CmdProcessBackportsTest(unittest.TestCase): - def setUp(self): - _mock_git_and_gh(self) - self.mock_changelog_news = patch( - "tools.private.release.process_backports.changelog_news" - ).start() - self.mock_replace_version_next = patch( - "tools.private.release.process_backports.replace_version_next" - ).start() - self.addCleanup(patch.stopall) - self.mock_gh.resolve_pr_number.side_effect = lambda x: int( - x.lstrip("#").split("/")[-1] - ) - self.mock_git.diff.return_value = "" - self.mock_git.apply_check.return_value = True - - # Dynamic mock for issue body - self.issue_body = "" - - def mock_get_body(issue_num): - return self.issue_body - - def mock_update_body(issue_num, body): - self.issue_body = body - - self.mock_gh.get_issue_body.side_effect = mock_get_body - self.mock_gh.update_issue_body.side_effect = mock_update_body - - def test_process_backports_no_pending(self): - args = argparse.Namespace( - issue=123, remote="origin", dry_run=False, add=None, triggering_comment=None - ) - self.issue_body = "No backports here" - - result = ProcessBackports(args, self.mock_git, self.mock_gh).run() - - self.assertEqual(result, 0) - self.mock_gh.get_issue_body.assert_called_once_with(123) - self.mock_git.fetch.assert_not_called() - - @patch("tools.private.release.process_backports.datetime") - def test_process_backports_success(self, mock_datetime): - mock_datetime.date.today.return_value = datetime.date(2026, 7, 1) - args = argparse.Namespace( - issue=123, remote="origin", dry_run=False, add=None, triggering_comment=None - ) - self.mock_gh.get_issue_title.return_value = "Release 2.0.0" - self.issue_body = """ +pytest_plugins = ["tests.tools.private.release.release_test_helper"] + + +def test_process_backports_no_pending(mock_git, mock_gh): + args = argparse.Namespace( + issue=123, remote="origin", dry_run=False, add=None, triggering_comment=None + ) + mock_gh.issues[123] = { + "title": "Release 2.0.0", + "body": "No backports here", + "labels": ["type: release"], + } + + result = ProcessBackports(args, mock_git, mock_gh).run() + + assert result == 0 + mock_git.fetch.assert_not_called() + + +def test_process_backports_success(mocker, mock_git, mock_gh): + mock_changelog = mocker.patch( + "tools.private.release.process_backports.changelog_news" + ) + mock_replace = mocker.patch( + "tools.private.release.process_backports.replace_version_next" + ) + mock_datetime = mocker.patch("tools.private.release.process_backports.datetime") + mock_datetime.date.today.return_value = datetime.date(2026, 7, 1) + + args = argparse.Namespace( + issue=123, remote="origin", dry_run=False, add=None, triggering_comment=None + ) + mock_gh.issues[123] = { + "title": "Release 2.0.0", + "body": """ ## Checklist - [ ] Prepare Release - [ ] Create Release branch @@ -63,107 +47,80 @@ def test_process_backports_success(self, mock_datetime): ## Backports - [ ] #124 | status=pending -""" - self.mock_git.get_remote_tags.return_value = [] - - def mock_resolve(items): - for item in items: - if item.pr_ref == "#124": - item.commit = "abcdef12" - item.status = "done" - return items - - self.mock_gh.get_merge_commits_for_prs.side_effect = mock_resolve - - self.mock_git.sort_commits_chronologically.return_value = ["abcdef12"] - self.mock_git.get_commit_sha.side_effect = ["12345678", "12345678", "main_sha"] - self.mock_git.get_commit_message.return_value = 'Cherry-pick "fix bug"' - self.mock_git.get_modified_files.return_value = ["news/124.fixed.md"] - self.mock_git.diff.return_value = "version diff for 124" - self.mock_git.apply_check.return_value = True - self.mock_gh.create_pr.return_value = "https://github.com/foo/bar/pull/999" - - result = ProcessBackports(args, self.mock_git, self.mock_gh).run() - - self.assertEqual(result, 0) - self.mock_git.fetch.assert_has_calls( - [ - call("origin", tags=True, force=True), - call("origin"), - call("origin", refspec="main"), - ] - ) - self.mock_git.checkout.assert_has_calls( - [ - call("release/2.0", track_remote="origin"), - call("main", track_remote="origin"), - call("prepare-2.0.0-backports-6affdae", create_branch=True), - call("release/2.0"), - ] - ) - self.mock_git.cherry_pick.assert_called_once_with("abcdef12") - self.mock_git.diff.assert_called_once() - self.mock_git.apply_check.assert_called_once_with(unittest.mock.ANY) - self.mock_git.apply.assert_called_once_with(unittest.mock.ANY) - self.mock_changelog_news.update_changelog.assert_has_calls( - [ - call("2.0.0", "2026-07-01"), - call( - "2.0.0", - "2026-07-01", - news_files=["news/124.fixed.md"], - delete_news=True, - ), - ] - ) - self.assertEqual(self.mock_git.add_modified_and_deleted.call_count, 2) - self.mock_replace_version_next.assert_called_once_with("2.0.0") - self.mock_git.commit.assert_has_calls( - [ - call('Cherry-pick "fix bug"\n\nWork towards #123', amend=True), - call("chore(release): sync changelog for v2.0.0 backports"), - ] - ) - self.mock_git.push.assert_has_calls( - [ - call("origin", "release/2.0"), - call( - "origin", - "prepare-2.0.0-backports-6affdae", - set_upstream=True, - force=True, - ), - ] - ) - - self.mock_gh.create_pr.assert_called_once_with( - title="chore(release): sync changelog for v2.0.0 backports", - body="Updates CHANGELOG.md and removes news files for backports:\n- #124\n\nWork towards #123\nRelease-Tracking-Issue: #123", - base="main", - labels=["type: sync-changelog"], - ) - self.mock_gh.enable_auto_merge.assert_called_once_with(999) - - self.assertEqual(self.mock_gh.update_issue_body.call_count, 2) - call_args_list = self.mock_gh.update_issue_body.call_args_list - self.assertEqual(call_args_list[0][0][0], 123) - self.assertIn( - "- [x] #124 | status=done rc=rc0 commit= 12345678", call_args_list[0][0][1] - ) - self.assertEqual(call_args_list[1][0][0], 123) - self.assertIn( - "- [ ] Sync Changelog #124 | status=pending pr=#999", - call_args_list[1][0][1], - ) - - @patch("tools.private.release.process_backports.datetime") - def test_process_backports_sync_branch_exists(self, mock_datetime): - mock_datetime.date.today.return_value = datetime.date(2026, 7, 1) - args = argparse.Namespace( - issue=123, remote="origin", dry_run=False, add=None, triggering_comment=None - ) - self.mock_gh.get_issue_title.return_value = "Release 2.0.0" - self.issue_body = """ +""", + "labels": ["type: release"], + } + mock_gh.prs[124] = { + "state": "MERGED", + "mergeCommit": {"oid": "abcdef12"}, + } + mock_git.get_remote_tags.return_value = [] + mock_git.sort_commits_chronologically.return_value = ["abcdef12"] + mock_git.get_commit_sha.side_effect = ["12345678", "12345678", "main_sha"] + mock_git.get_commit_message.return_value = 'Cherry-pick "fix bug"' + mock_git.get_modified_files.return_value = ["news/124.fixed.md"] + mock_git.diff.return_value = "version diff for 124" + mock_git.apply_check.return_value = True + + result = ProcessBackports(args, mock_git, mock_gh).run() + + assert result == 0 + mock_git.fetch.assert_has_calls( + [ + call("origin", tags=True, force=True), + call("origin"), + call("origin", refspec="main"), + ] + ) + mock_git.checkout.assert_has_calls( + [ + call("release/2.0", track_remote="origin"), + call("main", track_remote="origin"), + call("prepare-2.0.0-backports-6affdae", create_branch=True), + call("release/2.0"), + ] + ) + mock_git.cherry_pick.assert_called_once_with("abcdef12") + mock_git.diff.assert_called_once() + mock_git.apply_check.assert_called_once_with(ANY) + mock_git.apply.assert_called_once_with(ANY) + mock_changelog.update_changelog.assert_has_calls( + [ + call("2.0.0", "2026-07-01"), + call( + "2.0.0", + "2026-07-01", + news_files=["news/124.fixed.md"], + delete_news=True, + ), + ] + ) + assert mock_git.add_modified_and_deleted.call_count == 2 + mock_replace.assert_called_once_with("2.0.0") + mock_git.commit.assert_has_calls( + [ + call('Cherry-pick "fix bug"\n\nWork towards #123', amend=True), + call("chore(release): sync changelog for v2.0.0 backports"), + ] + ) + + updated_body = mock_gh.get_issue_body(123) + assert "- [x] #124 | status=done rc=rc0 commit= 12345678" in updated_body + assert "- [ ] Sync Changelog #124 | status=pending pr=#1001" in updated_body + + +def test_process_backports_sync_branch_exists(mocker, mock_git, mock_gh): + mocker.patch("tools.private.release.process_backports.changelog_news") + mocker.patch("tools.private.release.process_backports.replace_version_next") + mock_datetime = mocker.patch("tools.private.release.process_backports.datetime") + mock_datetime.date.today.return_value = datetime.date(2026, 7, 1) + + args = argparse.Namespace( + issue=123, remote="origin", dry_run=False, add=None, triggering_comment=None + ) + mock_gh.issues[123] = { + "title": "Release 2.0.0", + "body": """ ## Checklist - [ ] Prepare Release - [ ] Create Release branch @@ -172,117 +129,48 @@ def test_process_backports_sync_branch_exists(self, mock_datetime): ## Backports - [ ] #124 | status=pending -""" - self.mock_git.get_remote_tags.return_value = [] - - def mock_resolve(items): - for item in items: - if item.pr_ref == "#124": - item.commit = "abcdef12" - item.status = "done" - return items - - self.mock_gh.get_merge_commits_for_prs.side_effect = mock_resolve - - self.mock_git.sort_commits_chronologically.return_value = ["abcdef12"] - self.mock_git.get_commit_sha.side_effect = ["12345678", "12345678", "main_sha"] - self.mock_git.get_commit_message.return_value = 'Cherry-pick "fix bug"' - self.mock_git.get_modified_files.return_value = ["news/124.fixed.md"] - self.mock_git.diff.return_value = "version diff for 124" - self.mock_git.apply_check.return_value = True - self.mock_gh.create_pr.return_value = "https://github.com/foo/bar/pull/999" - - # Configure branch to exist - self.mock_git.branch_exists.return_value = True - - result = ProcessBackports(args, self.mock_git, self.mock_gh).run() - - self.assertEqual(result, 0) - self.mock_git.fetch.assert_has_calls( - [ - call("origin", tags=True, force=True), - call("origin"), - call("origin", refspec="main"), - ] - ) - self.mock_git.checkout.assert_has_calls( - [ - call("release/2.0", track_remote="origin"), - call("main", track_remote="origin"), - # Called without create_branch=True - call("prepare-2.0.0-backports-6affdae"), - call("release/2.0"), - ] - ) - # Verify reset_hard was called to reset the existing branch to main - self.mock_git.reset_hard.assert_has_calls( - [ - call(reset_to="main"), - ] - ) - self.mock_git.cherry_pick.assert_called_once_with("abcdef12") - self.mock_git.diff.assert_called_once() - self.mock_git.apply_check.assert_called_once_with(unittest.mock.ANY) - self.mock_git.apply.assert_called_once_with(unittest.mock.ANY) - self.mock_changelog_news.update_changelog.assert_has_calls( - [ - call("2.0.0", "2026-07-01"), - call( - "2.0.0", - "2026-07-01", - news_files=["news/124.fixed.md"], - delete_news=True, - ), - ] - ) - self.assertEqual(self.mock_git.add_modified_and_deleted.call_count, 2) - self.mock_replace_version_next.assert_called_once_with("2.0.0") - self.mock_git.commit.assert_has_calls( - [ - call('Cherry-pick "fix bug"\n\nWork towards #123', amend=True), - call("chore(release): sync changelog for v2.0.0 backports"), - ] - ) - self.mock_git.push.assert_has_calls( - [ - call("origin", "release/2.0"), - call( - "origin", - "prepare-2.0.0-backports-6affdae", - set_upstream=True, - force=True, - ), - ] - ) - - self.mock_gh.create_pr.assert_called_once_with( - title="chore(release): sync changelog for v2.0.0 backports", - body="Updates CHANGELOG.md and removes news files for backports:\n- #124\n\nWork towards #123\nRelease-Tracking-Issue: #123", - base="main", - labels=["type: sync-changelog"], - ) - self.mock_gh.enable_auto_merge.assert_called_once_with(999) - - self.assertEqual(self.mock_gh.update_issue_body.call_count, 2) - call_args_list = self.mock_gh.update_issue_body.call_args_list - self.assertEqual(call_args_list[0][0][0], 123) - self.assertIn( - "- [x] #124 | status=done rc=rc0 commit= 12345678", call_args_list[0][0][1] - ) - self.assertEqual(call_args_list[1][0][0], 123) - self.assertIn( - "- [ ] Sync Changelog #124 | status=pending pr=#999", - call_args_list[1][0][1], - ) - - @patch("tools.private.release.process_backports.datetime") - def test_process_backports_dry_run(self, mock_datetime): - mock_datetime.date.today.return_value = datetime.date(2026, 7, 1) - args = argparse.Namespace( - issue=123, remote="origin", dry_run=True, add=None, triggering_comment=None - ) - self.mock_gh.get_issue_title.return_value = "Release 2.0.0" - self.issue_body = """ +""", + "labels": ["type: release"], + } + mock_gh.prs[124] = { + "state": "MERGED", + "mergeCommit": {"oid": "abcdef12"}, + } + mock_git.get_remote_tags.return_value = [] + mock_git.sort_commits_chronologically.return_value = ["abcdef12"] + mock_git.get_commit_sha.side_effect = ["12345678", "12345678", "main_sha"] + mock_git.get_commit_message.return_value = 'Cherry-pick "fix bug"' + mock_git.get_modified_files.return_value = ["news/124.fixed.md"] + mock_git.diff.return_value = "version diff for 124" + mock_git.apply_check.return_value = True + mock_git.branch_exists.return_value = True + + result = ProcessBackports(args, mock_git, mock_gh).run() + + assert result == 0 + mock_git.checkout.assert_has_calls( + [ + call("release/2.0", track_remote="origin"), + call("main", track_remote="origin"), + call("prepare-2.0.0-backports-6affdae"), + call("release/2.0"), + ] + ) + mock_git.reset_hard.assert_has_calls([call(reset_to="main")]) + + +def test_process_backports_dry_run(mocker, mock_git, mock_gh): + mocker.patch("tools.private.release.process_backports.changelog_news") + mocker.patch("tools.private.release.process_backports.replace_version_next") + mock_datetime = mocker.patch("tools.private.release.process_backports.datetime") + mock_datetime.date.today.return_value = datetime.date(2026, 7, 1) + + args = argparse.Namespace( + issue=123, remote="origin", dry_run=True, add=None, triggering_comment=None + ) + mock_gh.issues[123] = { + "title": "Release 2.0.0", + "body": """ ## Checklist - [ ] Prepare Release - [ ] Create Release branch @@ -291,77 +179,35 @@ def test_process_backports_dry_run(self, mock_datetime): ## Backports - [ ] #124 | status=pending -""" - self.mock_git.get_remote_tags.return_value = [] - - def mock_resolve(items): - for item in items: - if item.pr_ref == "#124": - item.commit = "abcdef12" - item.status = "done" - return items - - self.mock_gh.get_merge_commits_for_prs.side_effect = mock_resolve - - self.mock_git.sort_commits_chronologically.return_value = ["abcdef12"] - self.mock_git.get_commit_sha.side_effect = ["12345678", "main_sha"] - self.mock_git.get_commit_message.return_value = 'Cherry-pick "fix bug"' - self.mock_git.get_modified_files.return_value = ["news/124.fixed.md"] - self.mock_git.diff.return_value = "version diff for 124" - self.mock_git.apply_check.return_value = True - - result = ProcessBackports(args, self.mock_git, self.mock_gh).run() - - self.assertEqual(result, 0) - self.mock_git.fetch.assert_has_calls( - [ - call("origin", tags=True, force=True), - call("origin"), - call("origin", refspec="main"), - ] - ) - self.mock_git.checkout.assert_has_calls( - [ - call("release/2.0", track_remote="origin"), - call("main", track_remote="origin"), - call("release/2.0"), - ] - ) - self.mock_git.cherry_pick.assert_called_once_with("abcdef12") - self.mock_git.diff.assert_called_once() - self.mock_git.apply_check.assert_called_once_with(unittest.mock.ANY) - self.mock_git.apply.assert_not_called() - self.mock_changelog_news.update_changelog.assert_has_calls( - [ - call("2.0.0", "2026-07-01"), - call( - "2.0.0", - "2026-07-01", - news_files=["news/124.fixed.md"], - delete_news=True, - ), - ] - ) - self.assertEqual(self.mock_git.add_modified_and_deleted.call_count, 1) - self.mock_replace_version_next.assert_called_once_with("2.0.0") - self.mock_git.commit.assert_called_once_with( - 'Cherry-pick "fix bug"\n\nWork towards #123', amend=True - ) - self.mock_git.reset_hard.assert_has_calls( - [ - call(reset_to="12345678"), - call(reset_to="main_sha"), - ] - ) - self.mock_git.push.assert_not_called() - self.mock_gh.update_issue_body.assert_not_called() - - def test_process_backports_ignored_and_failed_states(self): - args = argparse.Namespace( - issue=123, remote="origin", dry_run=False, add=None, triggering_comment=None - ) - self.mock_gh.get_issue_title.return_value = "Release 2.0.0" - self.issue_body = """ +""", + "labels": ["type: release"], + } + mock_gh.prs[124] = { + "state": "MERGED", + "mergeCommit": {"oid": "abcdef12"}, + } + mock_git.get_remote_tags.return_value = [] + mock_git.sort_commits_chronologically.return_value = ["abcdef12"] + mock_git.get_commit_sha.side_effect = ["12345678", "main_sha"] + mock_git.get_commit_message.return_value = 'Cherry-pick "fix bug"' + mock_git.get_modified_files.return_value = ["news/124.fixed.md"] + mock_git.diff.return_value = "version diff for 124" + mock_git.apply_check.return_value = True + + result = ProcessBackports(args, mock_git, mock_gh).run() + + assert result == 0 + mock_git.apply.assert_not_called() + mock_git.push.assert_not_called() + + +def test_process_backports_ignored_and_failed_states(mock_git, mock_gh): + args = argparse.Namespace( + issue=123, remote="origin", dry_run=False, add=None, triggering_comment=None + ) + mock_gh.issues[123] = { + "title": "Release 2.0.0", + "body": """ ## Checklist - [ ] Prepare Release - [ ] Create Release branch @@ -370,39 +216,31 @@ def test_process_backports_ignored_and_failed_states(self): - [ ] #124 | status=pending - [ ] #125 | status=pending - [ ] #126 | status=pending -""" - self.mock_git.get_remote_tags.return_value = [] - - def mock_resolve(items): - for item in items: - if item.pr_ref == "#124": - item.status = "open-pr" - elif item.pr_ref == "#125": - item.status = "draft-pr" - elif item.pr_ref == "#126": - item.status = "error-closed-pr" - return items - - self.mock_gh.get_merge_commits_for_prs.side_effect = mock_resolve - - result = ProcessBackports(args, self.mock_git, self.mock_gh).run() - - self.assertEqual(result, 1) - self.mock_gh.update_issue_body.assert_called_once() - call_args = self.mock_gh.update_issue_body.call_args[0] - self.assertEqual(call_args[0], 123) - self.assertIn("- [ ] #126 | status=error-closed-pr", call_args[1]) - self.assertNotIn("status=open-pr", call_args[1]) - self.assertNotIn("status=draft-pr", call_args[1]) - self.mock_git.checkout.assert_not_called() - self.mock_git.cherry_pick.assert_not_called() - - def test_process_backports_ignored_error_status(self): - args = argparse.Namespace( - issue=123, remote="origin", dry_run=False, add=None, triggering_comment=None - ) - self.mock_gh.get_issue_title.return_value = "Release 2.0.0" - self.issue_body = """ +""", + "labels": ["type: release"], + } + mock_git.get_remote_tags.return_value = [] + + mock_gh.prs[124] = {"state": "OPEN"} + mock_gh.prs[125] = {"state": "OPEN", "isDraft": True} + mock_gh.prs[126] = {"state": "CLOSED"} + + result = ProcessBackports(args, mock_git, mock_gh).run() + + assert result == 1 + updated_body = mock_gh.get_issue_body(123) + assert "- [ ] #126 | status=error-closed-pr" in updated_body + mock_git.checkout.assert_not_called() + mock_git.cherry_pick.assert_not_called() + + +def test_process_backports_ignored_error_status(mock_git, mock_gh): + args = argparse.Namespace( + issue=123, remote="origin", dry_run=False, add=None, triggering_comment=None + ) + mock_gh.issues[123] = { + "title": "Release 2.0.0", + "body": """ ## Checklist - [ ] Prepare Release - [ ] Create Release branch @@ -410,74 +248,73 @@ def test_process_backports_ignored_error_status(self): ## Backports - [ ] #124 | status=error-merge-conflict - [ ] #125 | status=error-some-other-error -""" - self.mock_git.get_remote_tags.return_value = [] - self.mock_gh.get_merge_commits_for_prs.return_value = [] - - result = ProcessBackports(args, self.mock_git, self.mock_gh).run() - - self.assertEqual(result, 0) - self.mock_gh.get_merge_commits_for_prs.assert_not_called() - self.mock_git.checkout.assert_not_called() - - @patch("tools.private.release.process_backports.datetime") - def test_process_backports_cherry_pick_failed(self, mock_datetime): - mock_datetime.date.today.return_value = datetime.date(2026, 7, 1) - args = argparse.Namespace( - issue=123, remote="origin", dry_run=False, add=None, triggering_comment=None - ) - self.mock_gh.get_issue_title.return_value = "Release 2.0.0" - self.issue_body = """ +""", + "labels": ["type: release"], + } + mock_git.get_remote_tags.return_value = [] + + result = ProcessBackports(args, mock_git, mock_gh).run() + + assert result == 0 + mock_git.checkout.assert_not_called() + + +def test_process_backports_cherry_pick_failed(mocker, mock_git, mock_gh): + mock_datetime = mocker.patch("tools.private.release.process_backports.datetime") + mock_datetime.date.today.return_value = datetime.date(2026, 7, 1) + args = argparse.Namespace( + issue=123, remote="origin", dry_run=False, add=None, triggering_comment=None + ) + mock_gh.issues[123] = { + "title": "Release 2.0.0", + "body": """ ## Checklist - [ ] Prepare Release - [ ] Create Release branch ## Backports - [ ] #124 | status=pending -""" - self.mock_git.get_remote_tags.return_value = [] - - def mock_resolve(items): - for item in items: - if item.pr_ref == "#124": - item.commit = "abcdef12" - item.status = "done" - return items - - self.mock_gh.get_merge_commits_for_prs.side_effect = mock_resolve - - self.mock_git.sort_commits_chronologically.return_value = ["abcdef12"] - self.mock_git.cherry_pick.side_effect = Exception("Cherry-pick conflict") - - result = ProcessBackports(args, self.mock_git, self.mock_gh).run() - - self.assertEqual(result, 1) - self.mock_git.checkout.assert_called_once_with( - "release/2.0", track_remote="origin" - ) - self.mock_git.cherry_pick.assert_called_once_with("abcdef12") - self.mock_git.cherry_pick_abort.assert_called_once() - - self.mock_gh.update_issue_body.assert_called_once() - call_args = self.mock_gh.update_issue_body.call_args[0] - self.assertEqual(call_args[0], 123) - self.assertIn("- [ ] #124 | status=error-merge-conflict", call_args[1]) - - self.mock_git.commit.assert_not_called() - self.mock_git.push.assert_not_called() - - @patch("tools.private.release.process_backports.datetime") - def test_process_backports_add_backports_and_auto_add_rc_task(self, mock_datetime): - mock_datetime.date.today.return_value = datetime.date(2026, 7, 1) - args = argparse.Namespace( - issue=123, - remote="origin", - dry_run=False, - add=["https://github.com/bazel-contrib/rules_python/pull/124"], - triggering_comment=None, - ) - self.mock_gh.get_issue_title.return_value = "Release 2.0.0" - self.issue_body = """ +""", + "labels": ["type: release"], + } + mock_gh.prs[124] = { + "state": "MERGED", + "mergeCommit": {"oid": "abcdef12"}, + } + mock_git.get_remote_tags.return_value = [] + mock_git.sort_commits_chronologically.return_value = ["abcdef12"] + mock_git.cherry_pick.side_effect = Exception("Cherry-pick conflict") + + result = ProcessBackports(args, mock_git, mock_gh).run() + + assert result == 1 + mock_git.checkout.assert_called_once_with("release/2.0", track_remote="origin") + mock_git.cherry_pick.assert_called_once_with("abcdef12") + mock_git.cherry_pick_abort.assert_called_once() + + updated_body = mock_gh.get_issue_body(123) + assert "- [ ] #124 | status=error-merge-conflict" in updated_body + mock_git.commit.assert_not_called() + mock_git.push.assert_not_called() + + +def test_process_backports_add_backports_and_auto_add_rc_task( + mocker, mock_git, mock_gh +): + mocker.patch("tools.private.release.process_backports.changelog_news") + mocker.patch("tools.private.release.process_backports.replace_version_next") + mock_datetime = mocker.patch("tools.private.release.process_backports.datetime") + mock_datetime.date.today.return_value = datetime.date(2026, 7, 1) + args = argparse.Namespace( + issue=123, + remote="origin", + dry_run=False, + add=["https://github.com/bazel-contrib/rules_python/pull/124"], + triggering_comment=None, + ) + mock_gh.issues[123] = { + "title": "Release 2.0.0", + "body": """ ## Checklist - [x] Prepare Release | status=done pr=#122 commit=abcdef12 - [x] Create Release branch | status=done branch=release/2.0 commit=abcdef12 @@ -485,68 +322,43 @@ def test_process_backports_add_backports_and_auto_add_rc_task(self, mock_datetim - [ ] Tag Final ## Backports -""" - self.mock_git.get_remote_tags.return_value = ["2.0.0-rc0"] - self.mock_git.get_commit_sha.return_value = "12345678" - self.mock_git.get_commit_message.return_value = 'Cherry-pick "fix bug"' - - def mock_resolve(items): - for item in items: - if item.pr_ref == "#124": - item.commit = "abcdef12" - item.status = "done" - return items - - self.mock_gh.get_merge_commits_for_prs.side_effect = mock_resolve - self.mock_git.sort_commits_chronologically.return_value = ["abcdef12"] - - # Mock create_pr to return a string to avoid int(MagicMock) returning 1 - self.mock_gh.create_pr.return_value = "https://github.com/foo/bar/pull/999" - - result = ProcessBackports(args, self.mock_git, self.mock_gh).run() - - self.assertEqual(result, 0) - - # update_issue_body should be called 3 times: - # 1. When adding backports and auto-adding Tag RC1 task. - # 2. When updating the backport status to done. - # 3. When updating the sync task status to pending. - self.assertEqual(self.mock_gh.update_issue_body.call_count, 3) - - call1_args = self.mock_gh.update_issue_body.call_args_list[0][0] - call2_args = self.mock_gh.update_issue_body.call_args_list[1][0] - - self.assertEqual(call1_args[0], 123) - self.assertIn("- [ ] #124", call1_args[1]) - self.assertIn("- [ ] Tag RC1", call1_args[1]) - self.assertIn("- [ ] Sync Changelog #124", call1_args[1]) - self.assertIn( - "- [x] Tag RC0 | status=done tag=2.0.0-rc0 commit=abcdef12\n- [ ]" - " Tag RC1\n- [ ] Sync Changelog #124\n- [ ] Tag Final", - call1_args[1].strip(), - ) - - self.assertEqual(call2_args[0], 123) - self.assertIn("- [x] #124 | status=done rc=rc1 commit= 12345678", call2_args[1]) - - call3_args = self.mock_gh.update_issue_body.call_args_list[2][0] - self.assertEqual(call3_args[0], 123) - self.assertIn( - "- [ ] Sync Changelog #124 | status=pending pr=#999", call3_args[1] - ) - - @patch("tools.private.release.process_backports.datetime") - def test_process_backports_add_backports_marks_invalid(self, mock_datetime): - mock_datetime.date.today.return_value = datetime.date(2026, 7, 1) - args = argparse.Namespace( - issue=123, - remote="origin", - dry_run=False, - add=["124", "invalid", "125"], - triggering_comment=None, - ) - self.mock_gh.get_issue_title.return_value = "Release 2.0.0" - self.issue_body = """ +""", + "labels": ["type: release"], + } + mock_gh.prs[124] = { + "state": "MERGED", + "mergeCommit": {"oid": "abcdef12"}, + } + mock_git.get_remote_tags.return_value = ["2.0.0-rc0"] + mock_git.get_commit_sha.return_value = "12345678" + mock_git.get_commit_message.return_value = 'Cherry-pick "fix bug"' + mock_git.sort_commits_chronologically.return_value = ["abcdef12"] + mock_git.diff.return_value = "version diff" + mock_git.apply_check.return_value = True + + result = ProcessBackports(args, mock_git, mock_gh).run() + + assert result == 0 + updated_body = mock_gh.get_issue_body(123) + assert "- [x] #124 | status=done rc=rc1 commit= 12345678" in updated_body + assert "- [ ] Sync Changelog #124 | status=pending pr=#1001" in updated_body + + +def test_process_backports_add_backports_marks_invalid(mocker, mock_git, mock_gh): + mocker.patch("tools.private.release.process_backports.changelog_news") + mocker.patch("tools.private.release.process_backports.replace_version_next") + mock_datetime = mocker.patch("tools.private.release.process_backports.datetime") + mock_datetime.date.today.return_value = datetime.date(2026, 7, 1) + args = argparse.Namespace( + issue=123, + remote="origin", + dry_run=False, + add=["124", "invalid", "125"], + triggering_comment=None, + ) + mock_gh.issues[123] = { + "title": "Release 2.0.0", + "body": """ ## Checklist - [x] Prepare Release | status=done pr=#122 commit=abcdef12 - [x] Create Release branch | status=done branch=release/2.0 commit=abcdef12 @@ -554,59 +366,48 @@ def test_process_backports_add_backports_marks_invalid(self, mock_datetime): - [ ] Tag Final ## Backports -""" - self.mock_git.get_remote_tags.return_value = ["2.0.0-rc0"] - self.mock_git.get_commit_sha.return_value = "1234567890" - self.mock_git.get_commit_message.return_value = 'Cherry-pick "fix bug"' - - def mock_resolve(items): - # Both 124 and 125 should be processed, 'invalid' should be ignored (it has error status) - for item in items: - if item.pr_ref == "#124": - item.commit = "sha_124" - item.status = "done" - elif item.pr_ref == "#125": - item.commit = "sha_125" - item.status = "done" - return items - - self.mock_gh.get_merge_commits_for_prs.side_effect = mock_resolve - self.mock_git.sort_commits_chronologically.return_value = ["sha_124", "sha_125"] - self.mock_gh.create_pr.return_value = "https://github.com/foo/bar/pull/999" - - result = ProcessBackports(args, self.mock_git, self.mock_gh).run() - - self.assertEqual(result, 0) - # Should have updated body to add 124, 125, and invalid - # update_issue_body should be called 4 times: - # 1. When adding backports. - # 2. When updating 124 status to done. - # 3. When updating 125 status to done. - # 4. When updating sync tasks status to pending. - self.assertEqual(self.mock_gh.update_issue_body.call_count, 4) - call1_args = self.mock_gh.update_issue_body.call_args_list[0][0] - self.assertIn("- [ ] #124", call1_args[1]) - self.assertIn("- [ ] #125", call1_args[1]) - self.assertIn("- [ ] invalid | status=error-invalid-pr", call1_args[1]) - self.assertIn("- [ ] Sync Changelog #124", call1_args[1]) - self.assertIn("- [ ] Sync Changelog #125", call1_args[1]) - - call4_args = self.mock_gh.update_issue_body.call_args_list[3][0] - self.assertIn( - "- [ ] Sync Changelog #124 | status=pending pr=#999", call4_args[1] - ) - self.assertIn( - "- [ ] Sync Changelog #125 | status=pending pr=#999", call4_args[1] - ) - - @patch("tools.private.release.process_backports.datetime") - def test_process_backports_version_sync_failure(self, mock_datetime): - mock_datetime.date.today.return_value = datetime.date(2026, 7, 1) - args = argparse.Namespace( - issue=123, remote="origin", dry_run=False, add=None, triggering_comment=None - ) - self.mock_gh.get_issue_title.return_value = "Release 2.0.0" - self.issue_body = """ +""", + "labels": ["type: release"], + } + mock_gh.prs[124] = { + "state": "MERGED", + "mergeCommit": {"oid": "sha_124"}, + } + mock_gh.prs[125] = { + "state": "MERGED", + "mergeCommit": {"oid": "sha_125"}, + } + mock_git.get_remote_tags.return_value = ["2.0.0-rc0"] + mock_git.get_commit_sha.return_value = "1234567890" + mock_git.get_commit_message.return_value = 'Cherry-pick "fix bug"' + mock_git.sort_commits_chronologically.return_value = ["sha_124", "sha_125"] + mock_git.diff.return_value = "version diff" + mock_git.apply_check.return_value = True + + result = ProcessBackports(args, mock_git, mock_gh).run() + + assert result == 0 + updated_body = mock_gh.get_issue_body(123) + assert "- [ ] invalid | status=error-invalid-pr" in updated_body + assert "- [ ] Sync Changelog #124 | status=pending pr=#1001" in updated_body + assert "- [ ] Sync Changelog #125 | status=pending pr=#1001" in updated_body + + +def test_process_backports_version_sync_failure(mocker, mock_git, mock_gh): + mock_changelog = mocker.patch( + "tools.private.release.process_backports.changelog_news" + ) + mock_replace = mocker.patch( + "tools.private.release.process_backports.replace_version_next" + ) + mock_datetime = mocker.patch("tools.private.release.process_backports.datetime") + mock_datetime.date.today.return_value = datetime.date(2026, 7, 1) + args = argparse.Namespace( + issue=123, remote="origin", dry_run=False, add=None, triggering_comment=None + ) + mock_gh.issues[123] = { + "title": "Release 2.0.0", + "body": """ ## Checklist - [ ] Prepare Release - [ ] Create Release branch @@ -617,135 +418,76 @@ def test_process_backports_version_sync_failure(self, mock_datetime): ## Backports - [ ] #124 | status=pending - [ ] #125 | status=pending -""" - self.mock_git.get_remote_tags.return_value = [] - - def mock_resolve(items): - for item in items: - if item.pr_ref in ("#124", "#125"): - item.commit = "sha_" + item.pr_ref.lstrip("#") - item.status = "done" - return items - - self.mock_gh.get_merge_commits_for_prs.side_effect = mock_resolve - - self.mock_git.sort_commits_chronologically.return_value = ["sha_124", "sha_125"] - self.mock_git.get_commit_sha.side_effect = [ - "12345678", - "sha_124_amended", - "sha_125_amended", - "main_sha", +""", + "labels": ["type: release"], + } + mock_gh.prs[124] = { + "state": "MERGED", + "mergeCommit": {"oid": "sha_124"}, + } + mock_gh.prs[125] = { + "state": "MERGED", + "mergeCommit": {"oid": "sha_125"}, + } + mock_git.get_remote_tags.return_value = [] + mock_git.sort_commits_chronologically.return_value = ["sha_124", "sha_125"] + mock_git.get_commit_sha.side_effect = [ + "12345678", + "sha_124_amended", + "sha_125_amended", + "main_sha", + ] + mock_git.get_commit_message.return_value = 'Cherry-pick "fix bug"' + mock_git.get_modified_files.side_effect = [ + ["news/124.fixed.md"], + ["news/125.fixed.md"], + ] + mock_git.diff.return_value = "diff content" + mock_git.apply_check.side_effect = [False, True] + + result = ProcessBackports(args, mock_git, mock_gh).run() + + assert result == 0 + mock_git.checkout.assert_has_calls( + [ + call("release/2.0", track_remote="origin"), + call("main", track_remote="origin"), + call("prepare-2.0.0-backports-b552a96", create_branch=True), + call("release/2.0"), + ] + ) + mock_git.cherry_pick.assert_has_calls( + [ + call("sha_124"), + call("sha_125"), ] - self.mock_git.get_commit_message.return_value = 'Cherry-pick "fix bug"' - self.mock_git.get_modified_files.side_effect = [ - ["news/124.fixed.md"], - ["news/125.fixed.md"], + ) + assert mock_git.diff.call_count == 2 + assert mock_git.apply_check.call_count == 2 + mock_git.apply.assert_called_once_with(ANY) + + mock_changelog.update_changelog.assert_has_calls( + [ + call("2.0.0", "2026-07-01"), + call("2.0.0", "2026-07-01"), + call( + "2.0.0", + "2026-07-01", + news_files=["news/124.fixed.md", "news/125.fixed.md"], + delete_news=True, + ), ] - self.mock_git.diff.side_effect = ["diff 124", "diff 125"] - self.mock_git.apply_check.side_effect = [False, True] - self.mock_gh.create_pr.return_value = "https://github.com/foo/bar/pull/999" - - result = ProcessBackports(args, self.mock_git, self.mock_gh).run() - - self.assertEqual(result, 0) - self.mock_git.fetch.assert_has_calls( - [ - call("origin", tags=True, force=True), - call("origin"), - call("origin", refspec="main"), - ] - ) - self.mock_git.checkout.assert_has_calls( - [ - call("release/2.0", track_remote="origin"), - call("main", track_remote="origin"), - call("prepare-2.0.0-backports-b552a96", create_branch=True), - call("release/2.0"), - ] - ) - self.mock_git.cherry_pick.assert_has_calls( - [ - call("sha_124"), - call("sha_125"), - ] - ) - # diff should be called for each successful cherry-pick - self.assertEqual(self.mock_git.diff.call_count, 2) - # apply_check should be called for both patches - self.assertEqual(self.mock_git.apply_check.call_count, 2) - # apply should only be called for 125 (since 124 failed check) - self.mock_git.apply.assert_called_once_with(unittest.mock.ANY) - - self.mock_changelog_news.update_changelog.assert_has_calls( - [ - call("2.0.0", "2026-07-01"), - call("2.0.0", "2026-07-01"), - call( - "2.0.0", - "2026-07-01", - news_files=["news/124.fixed.md", "news/125.fixed.md"], - delete_news=True, - ), - ] - ) - # add_modified_and_deleted called: - # - once per cherry-pick (2) - # - once on main backport branch (1) - # Total = 3 - self.assertEqual(self.mock_git.add_modified_and_deleted.call_count, 3) - # replace_version_next called once per cherry-pick - self.assertEqual(self.mock_replace_version_next.call_count, 2) - - self.mock_git.commit.assert_has_calls( - [ - call('Cherry-pick "fix bug"\n\nWork towards #123', amend=True), - call('Cherry-pick "fix bug"\n\nWork towards #123', amend=True), - call("chore(release): sync changelog for v2.0.0 backports"), - ] - ) - self.mock_git.push.assert_has_calls( - [ - call("origin", "release/2.0"), - call("origin", "release/2.0"), - call( - "origin", - "prepare-2.0.0-backports-b552a96", - set_upstream=True, - force=True, - ), - ] - ) - - # PR body should contain warning about 124 - expected_body = ( - "Updates CHANGELOG.md and removes news files for backports:\n" - "- #124\n" - "- #125\n" - "\n" - "Warning: These PRs failed to update their version markers:\n" - "- #124\n" - "\n" - "Work towards #123\n" - "Release-Tracking-Issue: #123" - ) - self.mock_gh.create_pr.assert_called_once_with( - title="chore(release): sync changelog for v2.0.0 backports", - body=expected_body, - base="main", - labels=["type: sync-changelog"], - ) - self.mock_gh.enable_auto_merge.assert_called_once_with(999) - - # update_issue_body called 3 times (twice for backports, once for sync tasks) - self.assertEqual(self.mock_gh.update_issue_body.call_count, 3) - call3_args = self.mock_gh.update_issue_body.call_args_list[2][0] - self.assertIn( - "- [ ] Sync Changelog #124 | status=pending pr=#999", call3_args[1] - ) - self.assertIn( - "- [ ] Sync Changelog #125 | status=pending pr=#999", call3_args[1] - ) - - -if __name__ == "__main__": - unittest.main() + ) + assert mock_git.add_modified_and_deleted.call_count == 3 + assert mock_replace.call_count == 2 + + assert 1001 in mock_gh.prs + pr = mock_gh.prs[1001] + assert ( + "Warning: These PRs failed to update their version markers:\n- #124" + in pr["body"] + ) + + updated_body = mock_gh.get_issue_body(123) + assert "- [ ] Sync Changelog #124 | status=pending pr=#1001" in updated_body + assert "- [ ] Sync Changelog #125 | status=pending pr=#1001" in updated_body diff --git a/tests/tools/private/release/promote_test.py b/tests/tools/private/release/promote_test.py index 8d1d2cfb29..b7fe9c9848 100644 --- a/tests/tools/private/release/promote_test.py +++ b/tests/tools/private/release/promote_test.py @@ -1,313 +1,281 @@ import argparse -import os -import tempfile -import unittest from pathlib import Path from unittest.mock import call, patch -from tests.tools.private.release.release_test_helper import _mock_git_and_gh -from tools.private.release.gh import NoTrackingIssueError from tools.private.release.promote import Promote - -class CmdPromoteTest(unittest.TestCase): - def setUp(self): - _mock_git_and_gh(self) - self.test_dir = tempfile.TemporaryDirectory() - self.addCleanup(self.test_dir.cleanup) - - def test_promote_rc_success(self): - # Arrange - args = argparse.Namespace( - version="2.0.0", issue=123, dry_run=False, remote="my-remote" - ) - self.mock_git.get_remote_tags.return_value = ["2.0.0-rc0", "2.0.0-rc1"] - self.mock_git.get_commit_sha.return_value = "abcdef123456" - self.mock_git.tag_exists.return_value = False - initial_body = "- [ ] Tag Final" - self.mock_gh.get_issue_body.return_value = initial_body - - # Act - result = Promote(args, self.mock_git, self.mock_gh).run() - - # Assert - self.assertEqual(result, 0) - self.mock_git.fetch.assert_has_calls( - [ - call("my-remote", tags=True, force=True), - call("my-remote", refspec="release/2.0"), - ] - ) - self.mock_git.get_commit_sha.assert_has_calls( - [call("2.0.0-rc1"), call("my-remote/release/2.0")] - ) - self.mock_git.checkout.assert_not_called() - self.mock_git.tag_exists.assert_called_once_with("2.0.0") - self.mock_git.tag.assert_called_once_with("2.0.0", "abcdef123456") - self.mock_git.push.assert_called_once_with("my-remote", "2.0.0") - - # Verify issue update - self.mock_gh.get_issue_body.assert_called_once_with(123) - expected_updated_body = ( - "- [x] Tag Final | status=done tag=2.0.0 commit= abcdef12" - ) - self.mock_gh.update_issue_body.assert_called_once_with( - 123, expected_updated_body - ) - expected_comment = ( - "**New Release Tagged!** 🐍🌿\n\n" - "Version **2.0.0** has been successfully generated and tagged on branch [`release/2.0`](https://github.com/bazel-contrib/rules_python/tree/release/2.0).\n\n" - "- [Github Release 2.0.0](https://github.com/bazel-contrib/rules_python/releases/tag/2.0.0)\n" - "- [BCR Entry 2.0.0](https://registry.bazel.build/modules/rules_python/2.0.0)\n" - "- [BCR PRs](https://github.com/bazelbuild/bazel-central-registry/pulls?q=is%3Apr%20%28%22bazel-contrib/rules_python%22%20in%3Atitle%29%20%28%22%402.0.0%22%20in%3Atitle%29)\n" - "- [Release workflow status](https://github.com/bazel-contrib/rules_python/actions/workflows/release_promote.yaml)" - ) - self.mock_gh.post_issue_comment.assert_called_once_with(123, expected_comment) - - def test_promote_rc_writes_github_output(self): - # Arrange - github_output_path = os.path.join(self.test_dir.name, "github_output") - args = argparse.Namespace( - version="2.0.0", issue=123, dry_run=False, remote="my-remote" - ) - self.mock_git.get_remote_tags.return_value = ["2.0.0-rc0", "2.0.0-rc1"] - self.mock_git.get_commit_sha.return_value = "abcdef123456" - self.mock_git.tag_exists.return_value = False - initial_body = "- [ ] Tag Final" - self.mock_gh.get_issue_body.return_value = initial_body - - # Act - with patch.dict("os.environ", {"GITHUB_OUTPUT": github_output_path}): - result = Promote(args, self.mock_git, self.mock_gh).run() - - # Assert - self.assertEqual(result, 0) - self.assertTrue(os.path.exists(github_output_path)) - content = Path(github_output_path).read_text(encoding="utf-8") - self.assertEqual(content, "version=2.0.0\n") - - def test_promote_rc_resolve_issue_success(self): - # Arrange - args = argparse.Namespace( - version="2.0.0", issue=None, dry_run=False, remote="my-remote" - ) - self.mock_git.get_remote_tags.return_value = ["2.0.0-rc1"] - self.mock_git.tag_exists.return_value = False - self.mock_gh.get_release_tracking_issue.side_effect = None - self.mock_gh.get_release_tracking_issue.return_value = 123 - self.mock_git.get_commit_sha.return_value = "abcdef123456" - initial_body = "- [ ] Tag Final" - self.mock_gh.get_issue_body.return_value = initial_body - - # Act - result = Promote(args, self.mock_git, self.mock_gh).run() - - # Assert - self.assertEqual(result, 0) - self.mock_git.fetch.assert_has_calls( - [ - call("my-remote", tags=True, force=True), - call("my-remote", refspec="release/2.0"), - ] - ) - self.mock_gh.get_release_tracking_issue.assert_called_once_with("2.0.0") - self.mock_git.get_commit_sha.assert_has_calls( - [call("2.0.0-rc1"), call("my-remote/release/2.0")] - ) - self.mock_git.checkout.assert_not_called() - self.mock_git.tag.assert_called_once_with("2.0.0", "abcdef123456") - self.mock_git.push.assert_called_once_with("my-remote", "2.0.0") - self.mock_gh.get_issue_body.assert_called_once_with(123) - expected_updated_body = ( - "- [x] Tag Final | status=done tag=2.0.0 commit= abcdef12" - ) - self.mock_gh.update_issue_body.assert_called_once_with( - 123, expected_updated_body - ) - expected_comment = ( - "**New Release Tagged!** 🐍🌿\n\n" - "Version **2.0.0** has been successfully generated and tagged on branch [`release/2.0`](https://github.com/bazel-contrib/rules_python/tree/release/2.0).\n\n" - "- [Github Release 2.0.0](https://github.com/bazel-contrib/rules_python/releases/tag/2.0.0)\n" - "- [BCR Entry 2.0.0](https://registry.bazel.build/modules/rules_python/2.0.0)\n" - "- [BCR PRs](https://github.com/bazelbuild/bazel-central-registry/pulls?q=is%3Apr%20%28%22bazel-contrib/rules_python%22%20in%3Atitle%29%20%28%22%402.0.0%22%20in%3Atitle%29)\n" - "- [Release workflow status](https://github.com/bazel-contrib/rules_python/actions/workflows/release_promote.yaml)" - ) - self.mock_gh.post_issue_comment.assert_called_once_with(123, expected_comment) - - def test_promote_patch_success(self): - # Arrange - args = argparse.Namespace( - version="2.0.1", issue=123, dry_run=False, remote="my-remote" - ) - self.mock_git.tag_exists.return_value = False - self.mock_git.get_commit_sha.return_value = "12345678" - initial_body = "- [ ] Tag Final" - self.mock_gh.get_issue_body.return_value = initial_body - - # Act - result = Promote(args, self.mock_git, self.mock_gh).run() - - # Assert - self.assertEqual(result, 0) - self.mock_git.fetch.assert_has_calls( - [ - call("my-remote", tags=True, force=True), - call("my-remote", refspec="release/2.0"), - ] - ) - self.mock_git.get_current_branch.assert_not_called() - self.mock_git.get_tags.assert_not_called() - self.mock_git.get_remote_tags.assert_not_called() - - self.mock_git.checkout.assert_not_called() - self.mock_git.get_commit_sha.assert_called_once_with("my-remote/release/2.0") - self.mock_git.tag.assert_called_once_with("2.0.1", "12345678") - self.mock_git.push.assert_called_once_with("my-remote", "2.0.1") - - expected_updated_body = ( - "- [x] Tag Final | status=done tag=2.0.1 commit= 12345678" - ) - self.mock_gh.update_issue_body.assert_called_once_with( - 123, expected_updated_body - ) - expected_comment = ( - "**New Release Tagged!** 🐍🌿\n\n" - "Version **2.0.1** has been successfully generated and tagged on branch [`release/2.0`](https://github.com/bazel-contrib/rules_python/tree/release/2.0).\n\n" - "- [Github Release 2.0.1](https://github.com/bazel-contrib/rules_python/releases/tag/2.0.1)\n" - "- [BCR Entry 2.0.1](https://registry.bazel.build/modules/rules_python/2.0.1)\n" - "- [BCR PRs](https://github.com/bazelbuild/bazel-central-registry/pulls?q=is%3Apr%20%28%22bazel-contrib/rules_python%22%20in%3Atitle%29%20%28%22%402.0.1%22%20in%3Atitle%29)\n" - "- [Release workflow status](https://github.com/bazel-contrib/rules_python/actions/workflows/release_promote.yaml)" - ) - self.mock_gh.post_issue_comment.assert_called_once_with(123, expected_comment) - - @patch("builtins.print") - def test_promote_rc_dry_run_success(self, mock_print): - # Arrange - args = argparse.Namespace( - version="2.0.0", issue=123, dry_run=True, remote="my-remote" - ) - self.mock_git.get_remote_tags.return_value = ["2.0.0-rc0", "2.0.0-rc1"] - self.mock_git.get_commit_sha.return_value = "abcdef123456" - self.mock_git.tag_exists.return_value = False - initial_body = "- [ ] Tag Final" - self.mock_gh.get_issue_body.return_value = initial_body - - # Act - result = Promote(args, self.mock_git, self.mock_gh).run() - - # Assert - self.assertEqual(result, 0) - self.mock_git.fetch.assert_has_calls( - [ - call("my-remote", tags=True, force=True), - call("my-remote", refspec="release/2.0"), - ] - ) - self.mock_git.get_commit_sha.assert_has_calls( - [call("2.0.0-rc1"), call("my-remote/release/2.0")] - ) - self.mock_git.tag_exists.assert_called_once_with("2.0.0") - - # Core dry-run assertions: NO modifications - self.mock_git.tag.assert_not_called() - self.mock_git.push.assert_not_called() - self.mock_gh.update_issue_body.assert_not_called() - self.mock_gh.post_issue_comment.assert_not_called() - - mock_print.assert_has_calls( - [ - call("Verifying tracking issue #123 format..."), - call("Fetching remote branch my-remote/release/2.0..."), - call( - "[DRY RUN] Pre-conditions passed successfully for promoting" - " 2.0.0-rc1 to 2.0.0." - ), - call("[DRY RUN] Would tag commit abcdef12 as 2.0.0"), - call("[DRY RUN] Would push tag 2.0.0 to my-remote"), - call("[DRY RUN] Would update tracking issue #123 checklist"), - call("[DRY RUN] Would post comment to tracking issue #123"), - ] - ) - - def test_promote_rc_tag_already_exists(self): - # Arrange - args = argparse.Namespace( - version="2.0.0", issue=123, dry_run=False, remote="my-remote" - ) - self.mock_git.get_remote_tags.return_value = ["2.0.0-rc1"] - self.mock_git.tag_exists.return_value = True - - # Act - result = Promote(args, self.mock_git, self.mock_gh).run() - - # Assert - self.assertEqual(result, 1) - self.mock_git.checkout.assert_not_called() - self.mock_git.tag.assert_not_called() - self.mock_git.push.assert_not_called() - self.mock_gh.get_issue_body.assert_not_called() - self.mock_gh.update_issue_body.assert_not_called() - - def test_promote_rc_issue_not_found(self): - # Arrange - args = argparse.Namespace( - version="2.0.0", issue=None, dry_run=False, remote="my-remote" - ) - self.mock_git.get_remote_tags.return_value = ["2.0.0-rc1"] - self.mock_git.tag_exists.return_value = False - self.mock_gh.get_release_tracking_issue.side_effect = NoTrackingIssueError( - "Not found" - ) - - # Act - result = Promote(args, self.mock_git, self.mock_gh).run() - - # Assert - self.assertEqual(result, 1) - self.mock_gh.get_release_tracking_issue.assert_called_once_with("2.0.0") - self.mock_git.checkout.assert_not_called() - self.mock_git.tag.assert_not_called() - self.mock_git.push.assert_not_called() - self.mock_gh.get_issue_body.assert_not_called() - - def test_promote_rc_issue_malformed(self): - # Arrange - args = argparse.Namespace( - version="2.0.0", issue=123, dry_run=False, remote="my-remote" - ) - self.mock_git.get_remote_tags.return_value = ["2.0.0-rc1"] - self.mock_git.tag_exists.return_value = False - self.mock_git.get_commit_sha.return_value = "abcdef123456" - initial_body = "malformed body" - self.mock_gh.get_issue_body.return_value = initial_body - - # Act - result = Promote(args, self.mock_git, self.mock_gh).run() - - # Assert - self.assertEqual(result, 1) - self.mock_gh.get_issue_body.assert_called_once_with(123) - self.mock_git.checkout.assert_not_called() - self.mock_git.tag.assert_not_called() - self.mock_git.push.assert_not_called() - self.mock_gh.update_issue_body.assert_not_called() - - def test_promote_rc_no_rc_found(self): - # Arrange - args = argparse.Namespace( - version="2.0.0", issue=123, dry_run=False, remote="my-remote" - ) - self.mock_git.get_remote_tags.return_value = [] - - # Act - result = Promote(args, self.mock_git, self.mock_gh).run() - - # Assert - self.assertEqual(result, 1) - self.mock_git.checkout.assert_not_called() - self.mock_git.tag.assert_not_called() - self.mock_gh.get_issue_body.assert_not_called() - - -if __name__ == "__main__": - unittest.main() +pytest_plugins = ["tests.tools.private.release.release_test_helper"] + + +def test_promote_rc_success(mock_git, mock_gh): + # Arrange + issue_num = mock_gh.create_issue( + title="Release 2.0.0", + body="- [ ] Tag Final", + labels=["type: release"], + ) + args = argparse.Namespace( + version="2.0.0", issue=issue_num, dry_run=False, remote="my-remote" + ) + mock_git.get_remote_tags.return_value = ["2.0.0-rc0", "2.0.0-rc1"] + mock_git.get_commit_sha.return_value = "abcdef123456" + mock_git.tag_exists.return_value = False + + # Act + result = Promote(args, mock_git, mock_gh).run() + + # Assert + assert result == 0 + mock_git.fetch.assert_has_calls( + [ + call("my-remote", tags=True, force=True), + call("my-remote", refspec="release/2.0"), + ] + ) + mock_git.get_commit_sha.assert_has_calls( + [call("2.0.0-rc1"), call("my-remote/release/2.0")] + ) + mock_git.checkout.assert_not_called() + mock_git.tag_exists.assert_called_once_with("2.0.0") + mock_git.tag.assert_called_once_with("2.0.0", "abcdef123456") + mock_git.push.assert_called_once_with("my-remote", "2.0.0") + + # Verify issue update + expected_updated_body = "- [x] Tag Final | status=done tag=2.0.0 commit= abcdef12" + assert mock_gh.get_issue_body(issue_num) == expected_updated_body + + expected_comment = ( + "**New Release Tagged!** 🐍🌿\n\n" + "Version **2.0.0** has been successfully generated and tagged on branch [`release/2.0`](https://github.com/bazel-contrib/rules_python/tree/release/2.0).\n\n" + "- [Github Release 2.0.0](https://github.com/bazel-contrib/rules_python/releases/tag/2.0.0)\n" + "- [BCR Entry 2.0.0](https://registry.bazel.build/modules/rules_python/2.0.0)\n" + "- [BCR PRs](https://github.com/bazelbuild/bazel-central-registry/pulls?q=is%3Apr%20%28%22bazel-contrib/rules_python%22%20in%3Atitle%29%20%28%22%402.0.0%22%20in%3Atitle%29)\n" + "- [Release workflow status](https://github.com/bazel-contrib/rules_python/actions/workflows/release_promote.yaml)" + ) + assert mock_gh.issue_comments[issue_num] == [expected_comment] + + +def test_promote_rc_writes_github_output(tmp_path, monkeypatch, mock_git, mock_gh): + # Arrange + github_output_path = str(tmp_path / "github_output") + monkeypatch.setenv("GITHUB_OUTPUT", github_output_path) + issue_num = mock_gh.create_issue( + title="Release 2.0.0", + body="- [ ] Tag Final", + labels=["type: release"], + ) + args = argparse.Namespace( + version="2.0.0", issue=issue_num, dry_run=False, remote="my-remote" + ) + mock_git.get_remote_tags.return_value = ["2.0.0-rc0", "2.0.0-rc1"] + mock_git.get_commit_sha.return_value = "abcdef123456" + mock_git.tag_exists.return_value = False + + # Act + result = Promote(args, mock_git, mock_gh).run() + + # Assert + assert result == 0 + assert Path(github_output_path).exists() + content = Path(github_output_path).read_text(encoding="utf-8") + assert content == "version=2.0.0\n" + + +def test_promote_rc_resolve_issue_success(mock_git, mock_gh): + # Arrange + args = argparse.Namespace( + version="2.0.0", issue=None, dry_run=False, remote="my-remote" + ) + issue_num = mock_gh.create_issue( + title="Release 2.0.0", + body="- [ ] Tag Final", + labels=["type: release"], + ) + mock_git.get_remote_tags.return_value = ["2.0.0-rc1"] + mock_git.tag_exists.return_value = False + mock_git.get_commit_sha.return_value = "abcdef123456" + + # Act + result = Promote(args, mock_git, mock_gh).run() + + # Assert + assert result == 0 + mock_git.fetch.assert_has_calls( + [ + call("my-remote", tags=True, force=True), + call("my-remote", refspec="release/2.0"), + ] + ) + mock_git.get_commit_sha.assert_has_calls( + [call("2.0.0-rc1"), call("my-remote/release/2.0")] + ) + mock_git.checkout.assert_not_called() + mock_git.tag.assert_called_once_with("2.0.0", "abcdef123456") + mock_git.push.assert_called_once_with("my-remote", "2.0.0") + + expected_updated_body = "- [x] Tag Final | status=done tag=2.0.0 commit= abcdef12" + assert mock_gh.get_issue_body(issue_num) == expected_updated_body + + +def test_promote_patch_success(mock_git, mock_gh): + # Arrange + issue_num = mock_gh.create_issue( + title="Release 2.0.1", + body="- [ ] Tag Final", + labels=["type: release"], + ) + args = argparse.Namespace( + version="2.0.1", issue=issue_num, dry_run=False, remote="my-remote" + ) + mock_git.tag_exists.return_value = False + mock_git.get_commit_sha.return_value = "12345678" + + # Act + result = Promote(args, mock_git, mock_gh).run() + + # Assert + assert result == 0 + mock_git.fetch.assert_has_calls( + [ + call("my-remote", tags=True, force=True), + call("my-remote", refspec="release/2.0"), + ] + ) + mock_git.get_current_branch.assert_not_called() + mock_git.get_tags.assert_not_called() + mock_git.get_remote_tags.assert_not_called() + + mock_git.checkout.assert_not_called() + mock_git.get_commit_sha.assert_called_once_with("my-remote/release/2.0") + mock_git.tag.assert_called_once_with("2.0.1", "12345678") + mock_git.push.assert_called_once_with("my-remote", "2.0.1") + + expected_updated_body = "- [x] Tag Final | status=done tag=2.0.1 commit= 12345678" + assert mock_gh.get_issue_body(issue_num) == expected_updated_body + + +@patch("builtins.print") +def test_promote_rc_dry_run_success(mock_print, mock_git, mock_gh): + # Arrange + issue_num = mock_gh.create_issue( + title="Release 2.0.0", + body="- [ ] Tag Final", + labels=["type: release"], + ) + args = argparse.Namespace( + version="2.0.0", issue=issue_num, dry_run=True, remote="my-remote" + ) + mock_git.get_remote_tags.return_value = ["2.0.0-rc0", "2.0.0-rc1"] + mock_git.get_commit_sha.return_value = "abcdef123456" + mock_git.tag_exists.return_value = False + + # Act + result = Promote(args, mock_git, mock_gh).run() + + # Assert + assert result == 0 + mock_git.fetch.assert_has_calls( + [ + call("my-remote", tags=True, force=True), + call("my-remote", refspec="release/2.0"), + ] + ) + mock_git.get_commit_sha.assert_has_calls( + [call("2.0.0-rc1"), call("my-remote/release/2.0")] + ) + mock_git.tag_exists.assert_called_once_with("2.0.0") + + # Core dry-run assertions: NO modifications + mock_git.tag.assert_not_called() + mock_git.push.assert_not_called() + + mock_print.assert_has_calls( + [ + call(f"Verifying tracking issue #{issue_num} format..."), + call("Fetching remote branch my-remote/release/2.0..."), + call( + "[DRY RUN] Pre-conditions passed successfully for promoting" + " 2.0.0-rc1 to 2.0.0." + ), + call("[DRY RUN] Would tag commit abcdef12 as 2.0.0"), + call("[DRY RUN] Would push tag 2.0.0 to my-remote"), + call(f"[DRY RUN] Would update tracking issue #{issue_num} checklist"), + call(f"[DRY RUN] Would post comment to tracking issue #{issue_num}"), + ] + ) + + +def test_promote_rc_tag_already_exists(mock_git, mock_gh): + # Arrange + args = argparse.Namespace( + version="2.0.0", issue=123, dry_run=False, remote="my-remote" + ) + mock_git.get_remote_tags.return_value = ["2.0.0-rc1"] + mock_git.tag_exists.return_value = True + + # Act + result = Promote(args, mock_git, mock_gh).run() + + # Assert + assert result == 1 + mock_git.checkout.assert_not_called() + mock_git.tag.assert_not_called() + mock_git.push.assert_not_called() + + +def test_promote_rc_issue_not_found(mock_git, mock_gh): + # Arrange + args = argparse.Namespace( + version="2.0.0", issue=None, dry_run=False, remote="my-remote" + ) + mock_git.get_remote_tags.return_value = ["2.0.0-rc1"] + mock_git.tag_exists.return_value = False + + # Act + result = Promote(args, mock_git, mock_gh).run() + + # Assert + assert result == 1 + mock_git.checkout.assert_not_called() + mock_git.tag.assert_not_called() + mock_git.push.assert_not_called() + + +def test_promote_rc_issue_malformed(mock_git, mock_gh): + # Arrange + issue_num = mock_gh.create_issue( + title="Release 2.0.0", + body="malformed body", + labels=["type: release"], + ) + args = argparse.Namespace( + version="2.0.0", issue=issue_num, dry_run=False, remote="my-remote" + ) + mock_git.get_remote_tags.return_value = ["2.0.0-rc1"] + mock_git.tag_exists.return_value = False + mock_git.get_commit_sha.return_value = "abcdef123456" + + # Act + result = Promote(args, mock_git, mock_gh).run() + + # Assert + assert result == 1 + mock_git.checkout.assert_not_called() + mock_git.tag.assert_not_called() + mock_git.push.assert_not_called() + + +def test_promote_rc_no_rc_found(mock_git, mock_gh): + # Arrange + args = argparse.Namespace( + version="2.0.0", issue=123, dry_run=False, remote="my-remote" + ) + mock_git.get_remote_tags.return_value = [] + + # Act + result = Promote(args, mock_git, mock_gh).run() + + # Assert + assert result == 1 + mock_git.checkout.assert_not_called() + mock_git.tag.assert_not_called() diff --git a/tests/tools/private/release/release_issue_test.py b/tests/tools/private/release/release_issue_test.py index 36f0dbd704..869a303ebf 100644 --- a/tests/tools/private/release/release_issue_test.py +++ b/tests/tools/private/release/release_issue_test.py @@ -1,5 +1,3 @@ -import unittest - from tools.private.release.release_issue import ( add_backports_to_body, add_sync_changelog_task_to_body, @@ -9,67 +7,66 @@ ) -class ReleaseIssueTest(unittest.TestCase): - def test_parse_metadata_line_spaces(self): - # Test with spaces around '=' - line = "- [ ] Tag Final | tag = 2.0.0 commit = abcdef12" - expected = { - "checked": False, - "name": "Tag Final", - "metadata": {"tag": "2.0.0", "commit": "abcdef12"}, - "original_line": line, - } - self.assertEqual(parse_metadata_line(line), expected) - - # Test with spaces after '=' - line = "- [ ] Tag Final | tag= 2.0.0 commit= abcdef12" - expected = { - "checked": False, - "name": "Tag Final", - "metadata": {"tag": "2.0.0", "commit": "abcdef12"}, - "original_line": line, - } - self.assertEqual(parse_metadata_line(line), expected) - - # Test with standard format (no spaces) - line = "- [ ] Tag Final | tag=2.0.0 commit=abcdef12" - expected = { - "checked": False, - "name": "Tag Final", - "metadata": {"tag": "2.0.0", "commit": "abcdef12"}, - "original_line": line, - } - self.assertEqual(parse_metadata_line(line), expected) - - # Test with no metadata - line = "- [ ] Tag Final" - expected = { - "checked": False, - "name": "Tag Final", - "metadata": {}, - "original_line": line, - } - self.assertEqual(parse_metadata_line(line), expected) - - def test_format_metadata_line(self): - # Test with commit metadata (should have space) - metadata = {"status": "done", "tag": "2.0.0", "commit": "abcdef12"} - expected = "- [x] Tag Final | status=done tag=2.0.0 commit= abcdef12" - self.assertEqual(format_metadata_line(True, "Tag Final", metadata), expected) - - # Test with other metadata (should not have space) - metadata = {"status": "done", "pr": "#122"} - expected = "- [x] Prepare Release | status=done pr=#122" - self.assertEqual( - format_metadata_line(True, "Prepare Release", metadata), expected - ) - - # Test with no metadata - expected = "- [ ] Tag Final" - self.assertEqual(format_metadata_line(False, "Tag Final", {}), expected) - - def test_add_backports_to_body(self): - body = """ +def test_parse_metadata_line_spaces(): + # Test with spaces around '=' + line = "- [ ] Tag Final | tag = 2.0.0 commit = abcdef12" + expected = { + "checked": False, + "name": "Tag Final", + "metadata": {"tag": "2.0.0", "commit": "abcdef12"}, + "original_line": line, + } + assert parse_metadata_line(line) == expected + + # Test with spaces after '=' + line = "- [ ] Tag Final | tag= 2.0.0 commit= abcdef12" + expected = { + "checked": False, + "name": "Tag Final", + "metadata": {"tag": "2.0.0", "commit": "abcdef12"}, + "original_line": line, + } + assert parse_metadata_line(line) == expected + + # Test with standard format (no spaces) + line = "- [ ] Tag Final | tag=2.0.0 commit=abcdef12" + expected = { + "checked": False, + "name": "Tag Final", + "metadata": {"tag": "2.0.0", "commit": "abcdef12"}, + "original_line": line, + } + assert parse_metadata_line(line) == expected + + # Test with no metadata + line = "- [ ] Tag Final" + expected = { + "checked": False, + "name": "Tag Final", + "metadata": {}, + "original_line": line, + } + assert parse_metadata_line(line) == expected + + +def test_format_metadata_line(): + # Test with commit metadata (should have space) + metadata = {"status": "done", "tag": "2.0.0", "commit": "abcdef12"} + expected = "- [x] Tag Final | status=done tag=2.0.0 commit= abcdef12" + assert format_metadata_line(True, "Tag Final", metadata) == expected + + # Test with other metadata (should not have space) + metadata = {"status": "done", "pr": "#122"} + expected = "- [x] Prepare Release | status=done pr=#122" + assert format_metadata_line(True, "Prepare Release", metadata) == expected + + # Test with no metadata + expected = "- [ ] Tag Final" + assert format_metadata_line(False, "Tag Final", {}) == expected + + +def test_add_backports_to_body(): + body = """ ## Checklist - [ ] Prepare Release - [ ] Create Release branch @@ -78,14 +75,14 @@ def test_add_backports_to_body(self): ## Backports - [ ] #123 | status=done """ - items = [ - {"ref": "124"}, - {"ref": "#124"}, - {"ref": "125"}, - {"ref": "#123"}, - ] - updated_body = add_backports_to_body(body, items) - expected_body = """ + items = [ + {"ref": "124"}, + {"ref": "#124"}, + {"ref": "125"}, + {"ref": "#123"}, + ] + updated_body = add_backports_to_body(body, items) + expected_body = """ ## Checklist - [ ] Prepare Release - [ ] Create Release branch @@ -96,29 +93,30 @@ def test_add_backports_to_body(self): - [ ] #124 - [ ] #125 """ - self.assertEqual(updated_body.strip(), expected_body.strip()) + assert updated_body.strip() == expected_body.strip() + - def test_add_sync_changelog_task_to_body(self): - body = """ +def test_add_sync_changelog_task_to_body(): + body = """ ## Checklist - [ ] Prepare Release - [ ] Create Release branch - [ ] Tag Final """ - # Insert first task (should go before Tag Final) - body = add_sync_changelog_task_to_body(body, 124) - expected = """ + # Insert first task (should go before Tag Final) + body = add_sync_changelog_task_to_body(body, 124) + expected = """ ## Checklist - [ ] Prepare Release - [ ] Create Release branch - [ ] Sync Changelog #124 - [ ] Tag Final """ - self.assertEqual(body.strip(), expected.strip()) + assert body.strip() == expected.strip() - # Insert second task (should go after the last Sync Changelog task) - body = add_sync_changelog_task_to_body(body, 125) - expected = """ + # Insert second task (should go after the last Sync Changelog task) + body = add_sync_changelog_task_to_body(body, 125) + expected = """ ## Checklist - [ ] Prepare Release - [ ] Create Release branch @@ -126,14 +124,15 @@ def test_add_sync_changelog_task_to_body(self): - [ ] Sync Changelog #125 - [ ] Tag Final """ - self.assertEqual(body.strip(), expected.strip()) + assert body.strip() == expected.strip() - # Insert duplicate (should be ignored) - body = add_sync_changelog_task_to_body(body, 124) - self.assertEqual(body.strip(), expected.strip()) + # Insert duplicate (should be ignored) + body = add_sync_changelog_task_to_body(body, 124) + assert body.strip() == expected.strip() - def test_parse_checklist_state_with_sync_changelogs(self): - body = """ + +def test_parse_checklist_state_with_sync_changelogs(): + body = """ ## Checklist - [x] Prepare Release | status=done pr=#122 commit=abcdef12 - [x] Create Release branch | status=done branch=release/2.0 @@ -141,22 +140,18 @@ def test_parse_checklist_state_with_sync_changelogs(self): - [ ] Sync Changelog #126 - [ ] Tag Final """ - state = parse_checklist_state(body) - self.assertIn(124, state["sync_changelogs"]) - self.assertIn(126, state["sync_changelogs"]) - - task_124 = state["sync_changelogs"][124] - self.assertEqual(task_124.name, "Sync Changelog #124") - self.assertFalse(task_124.checked) - self.assertEqual(task_124.status, "pending") - self.assertEqual(task_124.pr, "#125") - - task_126 = state["sync_changelogs"][126] - self.assertEqual(task_126.name, "Sync Changelog #126") - self.assertFalse(task_126.checked) - self.assertIsNone(task_126.status) - self.assertIsNone(task_126.pr) - - -if __name__ == "__main__": - unittest.main() + state = parse_checklist_state(body) + assert 124 in state["sync_changelogs"] + assert 126 in state["sync_changelogs"] + + task_124 = state["sync_changelogs"][124] + assert task_124.name == "Sync Changelog #124" + assert not task_124.checked + assert task_124.status == "pending" + assert task_124.pr == "#125" + + task_126 = state["sync_changelogs"][126] + assert task_126.name == "Sync Changelog #126" + assert not task_126.checked + assert task_126.status is None + assert task_126.pr is None diff --git a/tests/tools/private/release/release_test.py b/tests/tools/private/release/release_test.py index f0cdbdfe1d..651af24c2f 100644 --- a/tests/tools/private/release/release_test.py +++ b/tests/tools/private/release/release_test.py @@ -1,23 +1,19 @@ -import unittest +import pytest from tools.private.release import release as releaser -class ReleaseCLITest(unittest.TestCase): - def test_valid_version(self): - # These should not raise an exception - releaser.create_parser().parse_args(["prepare", "0.28.0"]) - releaser.create_parser().parse_args(["promote", "1.0.0", "--remote", "origin"]) - releaser.create_parser().parse_args( - ["create-release-issue", "--version", "1.2.3rc4"] - ) +def test_valid_version(): + # These should not raise an exception + releaser.create_parser().parse_args(["prepare", "0.28.0"]) + releaser.create_parser().parse_args(["promote", "1.0.0", "--remote", "origin"]) + releaser.create_parser().parse_args( + ["create-release-issue", "--version", "1.2.3rc4"] + ) - def test_invalid_version(self): - with self.assertRaises(SystemExit): - releaser.create_parser().parse_args(["prepare", "0.28"]) - with self.assertRaises(SystemExit): - releaser.create_parser().parse_args(["prepare", "a.b.c"]) - -if __name__ == "__main__": - unittest.main() +def test_invalid_version(): + with pytest.raises(SystemExit): + releaser.create_parser().parse_args(["prepare", "0.28"]) + with pytest.raises(SystemExit): + releaser.create_parser().parse_args(["prepare", "a.b.c"]) diff --git a/tests/tools/private/release/release_test_helper.py b/tests/tools/private/release/release_test_helper.py index 12945193ef..c738d834f9 100644 --- a/tests/tools/private/release/release_test_helper.py +++ b/tests/tools/private/release/release_test_helper.py @@ -1,55 +1,21 @@ -import os -import pathlib -import shutil -import tempfile -import unittest +import dataclasses +from pathlib import Path from unittest.mock import MagicMock, patch -from tools.private.release.gh import ( - MultipleTrackingIssuesError, - NoTrackingIssueError, -) -from tools.private.release.mock_gh import MockGitHub - - -def _mock_git(test_case): - mock_git = MagicMock() - test_case.mock_git = mock_git - - # Mock Git inside utils.py since it instantiates it locally - patch("tools.private.release.utils.Git", return_value=mock_git).start() - - test_case.addCleanup(patch.stopall) - - # Apply safe defaults - mock_git.get_current_branch.return_value = None - mock_git.get_tags.return_value = [] - mock_git.get_remote_tags.return_value = [] - mock_git.status.return_value = "" - mock_git.branch_exists.return_value = False - mock_git.tag_exists.return_value = False - return mock_git +import pytest +from tools.private.release.mock_gh import MockGitHub -def _mock_git_and_gh(test_case): - _mock_git(test_case) - mock_gh = MagicMock() - test_case.mock_gh = mock_gh - - mock_gh.MultipleTrackingIssuesError = MultipleTrackingIssuesError - mock_gh.NoTrackingIssueError = NoTrackingIssueError - mock_gh.get_release_tracking_issue.side_effect = NoTrackingIssueError("Not found") - mock_gh.get_open_pr.return_value = None +@dataclasses.dataclass +class ReleaseToolEnv: + """Environment setup for testing release tools. + Attributes: + git_root: The root path of the temporary Git repository workspace. + """ -class TempDirTestCase(unittest.TestCase): - def setUp(self): - self.tmpdir = pathlib.Path(tempfile.mkdtemp()) - self.original_cwd = os.getcwd() - self.addCleanup(shutil.rmtree, self.tmpdir) - os.chdir(self.tmpdir) - self.addCleanup(os.chdir, self.original_cwd) + git_root: Path DEFAULT_RELEASE_TEMPLATE_CONTENT = ( @@ -62,15 +28,31 @@ def setUp(self): ) -class ReleaseToolTestCase(TempDirTestCase): - def setUp(self): - super().setUp() - self.gh = MockGitHub() - self.setUpReleaseTemplate() +@pytest.fixture(name="mock_git") +def fixture_mock_git(): + mock_git_inst = MagicMock() + mock_git_inst.get_current_branch.return_value = None + mock_git_inst.get_tags.return_value = [] + mock_git_inst.get_remote_tags.return_value = [] + mock_git_inst.status.return_value = "" + mock_git_inst.branch_exists.return_value = False + mock_git_inst.tag_exists.return_value = False + + with patch("tools.private.release.utils.Git", return_value=mock_git_inst): + yield mock_git_inst + + +@pytest.fixture(name="mock_gh") +def fixture_mock_gh(): + return MockGitHub() + - def setUpReleaseTemplate(self): - template_dir = self.tmpdir / ".github/ISSUE_TEMPLATE" - template_dir.mkdir(parents=True, exist_ok=True) - self.template_file = template_dir / "release_tracking_template.md" - self.template_content = DEFAULT_RELEASE_TEMPLATE_CONTENT - self.template_file.write_text(self.template_content, encoding="utf-8") +@pytest.fixture(name="release_tool_env") +def fixture_release_tool_env(tmp_path, monkeypatch): + """Fixture providing a temp cwd with release template set up.""" + monkeypatch.chdir(tmp_path) + template_dir = tmp_path / ".github" / "ISSUE_TEMPLATE" + template_dir.mkdir(parents=True, exist_ok=True) + template_file = template_dir / "release_tracking_template.md" + template_file.write_text(DEFAULT_RELEASE_TEMPLATE_CONTENT, encoding="utf-8") + yield ReleaseToolEnv(git_root=tmp_path) diff --git a/tests/tools/private/release/utils_test.py b/tests/tools/private/release/utils_test.py index 796bab1619..f161308b60 100644 --- a/tests/tools/private/release/utils_test.py +++ b/tests/tools/private/release/utils_test.py @@ -1,213 +1,243 @@ -import unittest -from unittest.mock import patch +import pytest -from tests.tools.private.release.release_test_helper import TempDirTestCase from tools.private.release import utils +pytest_plugins = ["tests.tools.private.release.release_test_helper"] -class GetLatestVersionTest(unittest.TestCase): - @patch("tools.private.release.git.Git.get_tags") - def test_get_latest_version_success(self, mock_get_tags): - mock_get_tags.return_value = ["0.1.0", "1.0.0", "0.2.0"] - self.assertEqual(utils.get_latest_version(), "1.0.0") - - @patch("tools.private.release.git.Git.get_tags") - def test_get_latest_version_rc_is_latest(self, mock_get_tags): - mock_get_tags.return_value = ["0.1.0", "1.0.0", "1.1.0rc0"] - with self.assertRaisesRegex( - ValueError, "The latest version is a pre-release version: 1.1.0rc0" - ): - utils.get_latest_version() - - @patch("tools.private.release.git.Git.get_tags") - def test_get_latest_version_no_tags(self, mock_get_tags): - mock_get_tags.return_value = [] - with self.assertRaisesRegex( - RuntimeError, "No git tags found matching X.Y.Z or X.Y.ZrcN format." - ): - utils.get_latest_version() - - @patch("tools.private.release.git.Git.get_tags") - def test_get_latest_version_no_matching_tags(self, mock_get_tags): - mock_get_tags.return_value = ["v1.0", "latest"] - with self.assertRaisesRegex( - RuntimeError, "No git tags found matching X.Y.Z or X.Y.ZrcN format." - ): - utils.get_latest_version() - - @patch("tools.private.release.git.Git.get_tags") - def test_get_latest_version_only_rc_tags(self, mock_get_tags): - mock_get_tags.return_value = ["1.0.0rc0", "1.1.0rc0"] - with self.assertRaisesRegex( - ValueError, "The latest version is a pre-release version: 1.1.0rc0" - ): - utils.get_latest_version() - - -class GetLatestRcTagTest(unittest.TestCase): - @patch("tools.private.release.git.Git.get_tags") - def test_get_latest_rc_tag_no_tags(self, mock_get_tags): - mock_get_tags.return_value = [] - self.assertIsNone(utils.get_latest_rc_tag("2.0.0")) - - @patch("tools.private.release.git.Git.get_tags") - def test_get_latest_rc_tag_no_matching_tags(self, mock_get_tags): - mock_get_tags.return_value = [ + +def test_get_latest_version_success(mocker): + mocker.patch( + "tools.private.release.git.Git.get_tags", + return_value=["0.1.0", "1.0.0", "0.2.0"], + ) + assert utils.get_latest_version() == "1.0.0" + + +def test_get_latest_version_rc_is_latest(mocker): + mocker.patch( + "tools.private.release.git.Git.get_tags", + return_value=["0.1.0", "1.0.0", "1.1.0rc0"], + ) + with pytest.raises( + ValueError, match="The latest version is a pre-release version: 1.1.0rc0" + ): + utils.get_latest_version() + + +def test_get_latest_version_no_tags(mocker): + mocker.patch("tools.private.release.git.Git.get_tags", return_value=[]) + with pytest.raises( + RuntimeError, match="No git tags found matching X.Y.Z or X.Y.ZrcN format." + ): + utils.get_latest_version() + + +def test_get_latest_version_no_matching_tags(mocker): + mocker.patch( + "tools.private.release.git.Git.get_tags", return_value=["v1.0", "latest"] + ) + with pytest.raises( + RuntimeError, match="No git tags found matching X.Y.Z or X.Y.ZrcN format." + ): + utils.get_latest_version() + + +def test_get_latest_version_only_rc_tags(mocker): + mocker.patch( + "tools.private.release.git.Git.get_tags", return_value=["1.0.0rc0", "1.1.0rc0"] + ) + with pytest.raises( + ValueError, match="The latest version is a pre-release version: 1.1.0rc0" + ): + utils.get_latest_version() + + +def test_get_latest_rc_tag_no_tags(mocker): + mocker.patch("tools.private.release.git.Git.get_tags", return_value=[]) + assert utils.get_latest_rc_tag("2.0.0") is None + + +def test_get_latest_rc_tag_no_matching_tags(mocker): + mocker.patch( + "tools.private.release.git.Git.get_tags", + return_value=[ "1.0.0", "2.0.0", "v2.0.0-rc0", "2.1.0-rc0", - ] - self.assertIsNone(utils.get_latest_rc_tag("2.0.0")) + ], + ) + assert utils.get_latest_rc_tag("2.0.0") is None - @patch("tools.private.release.git.Git.get_tags") - def test_get_latest_rc_tag_success(self, mock_get_tags): - mock_get_tags.return_value = [ + +def test_get_latest_rc_tag_success(mocker): + mocker.patch( + "tools.private.release.git.Git.get_tags", + return_value=[ "2.0.0-rc0", "2.0.0-rc2", "2.0.0-rc1", "2.1.0-rc0", - ] - self.assertEqual(utils.get_latest_rc_tag("2.0.0"), "2.0.0-rc2") + ], + ) + assert utils.get_latest_rc_tag("2.0.0") == "2.0.0-rc2" + + +def test_get_latest_rc_tag_ignores_v_prefix(mocker): + mocker.patch( + "tools.private.release.git.Git.get_tags", + return_value=["v2.0.0-rc0", "2.0.0-rc1"], + ) + assert utils.get_latest_rc_tag("2.0.0") == "2.0.0-rc1" - @patch("tools.private.release.git.Git.get_tags") - def test_get_latest_rc_tag_ignores_v_prefix(self, mock_get_tags): - mock_get_tags.return_value = ["v2.0.0-rc0", "2.0.0-rc1"] - self.assertEqual(utils.get_latest_rc_tag("2.0.0"), "2.0.0-rc1") - @patch("tools.private.release.git.Git.get_remote_tags") - def test_get_latest_rc_tag_remote_success(self, mock_get_remote_tags): - mock_get_remote_tags.return_value = [ +def test_get_latest_rc_tag_remote_success(mocker): + mock_get_remote_tags = mocker.patch( + "tools.private.release.git.Git.get_remote_tags", + return_value=[ "2.0.0-rc0", "2.0.0-rc2", "2.0.0-rc1", "2.1.0-rc0", - ] - self.assertEqual(utils.get_latest_rc_tag("2.0.0", remote="origin"), "2.0.0-rc2") - mock_get_remote_tags.assert_called_once_with("origin") + ], + ) + assert utils.get_latest_rc_tag("2.0.0", remote="origin") == "2.0.0-rc2" + mock_get_remote_tags.assert_called_once_with("origin") -class DetermineNextVersionTest(TempDirTestCase): - def setUp(self): - super().setUp() - self.mock_get_latest_version = patch( - "tools.private.release.utils.get_latest_version" - ).start() - self.mock_get_current_branch = patch( - "tools.private.release.git.Git.get_current_branch" - ).start() - self.mock_get_current_branch.return_value = "main" - self.addCleanup(patch.stopall) +def test_determine_next_version_no_markers(mocker, release_tool_env): + mocker.patch( + "tools.private.release.git.Git.get_current_branch", return_value="main" + ) + mocker.patch("tools.private.release.utils.get_latest_version", return_value="1.2.3") + (release_tool_env.git_root / "mock_file.bzl").write_text("no markers here") - def test_no_markers(self): - (self.tmpdir / "mock_file.bzl").write_text("no markers here") - self.mock_get_latest_version.return_value = "1.2.3" + next_version = utils.determine_next_version() - next_version = utils.determine_next_version() + assert next_version == "1.2.4" - self.assertEqual(next_version, "1.2.4") - def test_only_patch(self): - (self.tmpdir / "mock_file.bzl").write_text( - ":::{versionchanged} VERSION_NEXT_PATCH" - ) - self.mock_get_latest_version.return_value = "1.2.3" +def test_determine_next_version_only_patch(mocker, release_tool_env): + mocker.patch( + "tools.private.release.git.Git.get_current_branch", return_value="main" + ) + mocker.patch("tools.private.release.utils.get_latest_version", return_value="1.2.3") + (release_tool_env.git_root / "mock_file.bzl").write_text( + ":::{versionchanged} VERSION_NEXT_PATCH" + ) - next_version = utils.determine_next_version() + next_version = utils.determine_next_version() - self.assertEqual(next_version, "1.2.4") + assert next_version == "1.2.4" - def test_only_feature(self): - (self.tmpdir / "mock_file.bzl").write_text( - ":::{versionadded} VERSION_NEXT_FEATURE" - ) - self.mock_get_latest_version.return_value = "1.2.3" - next_version = utils.determine_next_version() +def test_determine_next_version_only_feature(mocker, release_tool_env): + mocker.patch( + "tools.private.release.git.Git.get_current_branch", return_value="main" + ) + mocker.patch("tools.private.release.utils.get_latest_version", return_value="1.2.3") + (release_tool_env.git_root / "mock_file.bzl").write_text( + ":::{versionadded} VERSION_NEXT_FEATURE" + ) - self.assertEqual(next_version, "1.3.0") + next_version = utils.determine_next_version() - def test_both_markers(self): - (self.tmpdir / "mock_file_patch.bzl").write_text( - ":::{versionchanged} VERSION_NEXT_PATCH" - ) - (self.tmpdir / "mock_file_feature.bzl").write_text( - ":::{versionadded} VERSION_NEXT_FEATURE" - ) - self.mock_get_latest_version.return_value = "1.2.3" + assert next_version == "1.3.0" - next_version = utils.determine_next_version() - self.assertEqual(next_version, "1.3.0") +def test_determine_next_version_both_markers(mocker, release_tool_env): + mocker.patch( + "tools.private.release.git.Git.get_current_branch", return_value="main" + ) + mocker.patch("tools.private.release.utils.get_latest_version", return_value="1.2.3") + (release_tool_env.git_root / "mock_file_patch.bzl").write_text( + ":::{versionchanged} VERSION_NEXT_PATCH" + ) + (release_tool_env.git_root / "mock_file_feature.bzl").write_text( + ":::{versionadded} VERSION_NEXT_FEATURE" + ) - @patch("tools.private.release.git.Git.get_current_branch") - @patch("tools.private.release.git.Git.get_tags") - def test_determine_next_version_on_release_branch_with_existing_tags( - self, mock_get_tags, mock_get_branch - ): - mock_get_branch.return_value = "release/0.37" - mock_get_tags.return_value = ["0.37.0", "0.37.1", "0.36.0"] + next_version = utils.determine_next_version() - next_version = utils.determine_next_version() + assert next_version == "1.3.0" - self.assertEqual(next_version, "0.37.2") - @patch("tools.private.release.git.Git.get_current_branch") - @patch("tools.private.release.git.Git.get_tags") - def test_determine_next_version_on_release_branch_no_tags( - self, mock_get_tags, mock_get_branch - ): - mock_get_branch.return_value = "release/0.38" - mock_get_tags.return_value = ["0.37.0"] # No 0.38.x tags +def test_determine_next_version_on_release_branch_with_existing_tags(mocker): + mocker.patch( + "tools.private.release.git.Git.get_current_branch", return_value="release/0.37" + ) + mocker.patch( + "tools.private.release.git.Git.get_tags", + return_value=["0.37.0", "0.37.1", "0.36.0"], + ) - next_version = utils.determine_next_version() + next_version = utils.determine_next_version() - self.assertEqual(next_version, "0.38.0") + assert next_version == "0.37.2" - @patch("tools.private.release.git.Git.get_current_branch") - @patch("tools.private.release.git.Git.get_tags") - def test_determine_next_version_on_release_branch_with_active_rc( - self, mock_get_tags, mock_get_branch - ): - mock_get_branch.return_value = "release/0.37" - # 0.37.0-rc0 and rc1 exist, but no stable 0.37.0 yet - mock_get_tags.return_value = ["0.37.0-rc0", "0.37.0-rc1", "0.36.0"] - next_version = utils.determine_next_version() +def test_determine_next_version_on_release_branch_no_tags(mocker): + mocker.patch( + "tools.private.release.git.Git.get_current_branch", return_value="release/0.38" + ) + mocker.patch( + "tools.private.release.git.Git.get_tags", return_value=["0.37.0"] + ) # No 0.38.x tags - # Should target 0.37.0, not 0.37.1 - self.assertEqual(next_version, "0.37.0") + next_version = utils.determine_next_version() - @patch("tools.private.release.git.Git.get_current_branch") - @patch("tools.private.release.git.Git.get_tags") - def test_determine_next_version_on_release_branch_with_stable_and_active_patch_rc( - self, mock_get_tags, mock_get_branch - ): - mock_get_branch.return_value = "release/0.37" - # 0.37.0 stable exists, and 0.37.1-rc0 exists (but no stable 0.37.1 yet) - mock_get_tags.return_value = ["0.37.0", "0.37.1-rc0", "0.36.0"] + assert next_version == "0.38.0" + + +def test_determine_next_version_on_release_branch_with_active_rc(mocker): + mocker.patch( + "tools.private.release.git.Git.get_current_branch", return_value="release/0.37" + ) + # 0.37.0-rc0 and rc1 exist, but no stable 0.37.0 yet + mocker.patch( + "tools.private.release.git.Git.get_tags", + return_value=["0.37.0-rc0", "0.37.0-rc1", "0.36.0"], + ) - next_version = utils.determine_next_version() + next_version = utils.determine_next_version() - # Should target 0.37.1, not 0.37.2 - self.assertEqual(next_version, "0.37.1") + # Should target 0.37.0, not 0.37.1 + assert next_version == "0.37.0" - @patch("tools.private.release.git.Git.get_current_branch") - def test_determine_next_version_on_main_branch_fallback(self, mock_get_branch): - mock_get_branch.return_value = "main" - # Should fallback to default behavior (which uses mock_get_latest_version from setUp) - self.mock_get_latest_version.return_value = "1.2.3" - (self.tmpdir / "mock_file.bzl").write_text("no markers here") - next_version = utils.determine_next_version() +def test_determine_next_version_on_release_branch_with_stable_and_active_patch_rc( + mocker, +): + mocker.patch( + "tools.private.release.git.Git.get_current_branch", return_value="release/0.37" + ) + # 0.37.0 stable exists, and 0.37.1-rc0 exists (but no stable 0.37.1 yet) + mocker.patch( + "tools.private.release.git.Git.get_tags", + return_value=["0.37.0", "0.37.1-rc0", "0.36.0"], + ) - self.assertEqual(next_version, "1.2.4") + next_version = utils.determine_next_version() + # Should target 0.37.1, not 0.37.2 + assert next_version == "0.37.1" -class ReplaceVersionNextTest(TempDirTestCase): - def test_replace_version_next(self): - # Arrange - mock_file_content = """ + +def test_determine_next_version_on_main_branch_fallback(mocker, release_tool_env): + mocker.patch( + "tools.private.release.git.Git.get_current_branch", return_value="main" + ) + mocker.patch("tools.private.release.utils.get_latest_version", return_value="1.2.3") + (release_tool_env.git_root / "mock_file.bzl").write_text("no markers here") + + next_version = utils.determine_next_version() + + assert next_version == "1.2.4" + + +def test_replace_version_next(release_tool_env): + # Arrange + mock_file_content = """ :::{versionadded} VERSION_NEXT_FEATURE blabla ::: @@ -216,51 +246,47 @@ def test_replace_version_next(self): blabla ::: """ - (self.tmpdir / "mock_file.bzl").write_text(mock_file_content) + (release_tool_env.git_root / "mock_file.bzl").write_text(mock_file_content) + + utils.replace_version_next("0.28.0") - utils.replace_version_next("0.28.0") + new_content = (release_tool_env.git_root / "mock_file.bzl").read_text() - new_content = (self.tmpdir / "mock_file.bzl").read_text() + assert ":::{versionadded} 0.28.0" in new_content + assert "VERSION_NEXT_FEATURE" not in new_content + assert "VERSION_NEXT_PATCH" not in new_content - self.assertIn(":::{versionadded} 0.28.0", new_content) - self.assertIn(":::{versionadded} 0.28.0", new_content) - self.assertNotIn("VERSION_NEXT_FEATURE", new_content) - self.assertNotIn("VERSION_NEXT_PATCH", new_content) - def test_replace_version_next_excludes_bazel_dirs(self): - # Arrange - mock_file_content = """ +def test_replace_version_next_excludes_bazel_dirs(release_tool_env): + # Arrange + mock_file_content = """ :::{versionadded} VERSION_NEXT_FEATURE blabla ::: """ - bazel_dir = self.tmpdir / "bazel-rules_python" - bazel_dir.mkdir() - (bazel_dir / "mock_file.bzl").write_text(mock_file_content) - - tools_dir = self.tmpdir / "tools" / "private" / "release" - tools_dir.mkdir(parents=True) - (tools_dir / "mock_file.bzl").write_text(mock_file_content) - - tests_dir = self.tmpdir / "tests" / "tools" / "private" / "release" - tests_dir.mkdir(parents=True) - (tests_dir / "mock_file.bzl").write_text(mock_file_content) + bazel_dir = release_tool_env.git_root / "bazel-rules_python" + bazel_dir.mkdir() + (bazel_dir / "mock_file.bzl").write_text(mock_file_content) - version = "0.28.0" + tools_dir = release_tool_env.git_root / "tools" / "private" / "release" + tools_dir.mkdir(parents=True) + (tools_dir / "mock_file.bzl").write_text(mock_file_content) - # Act - utils.replace_version_next(version) + tests_dir = release_tool_env.git_root / "tests" / "tools" / "private" / "release" + tests_dir.mkdir(parents=True) + (tests_dir / "mock_file.bzl").write_text(mock_file_content) - # Assert - new_content = (bazel_dir / "mock_file.bzl").read_text() - self.assertIn("VERSION_NEXT_FEATURE", new_content) + version = "0.28.0" - new_content = (tools_dir / "mock_file.bzl").read_text() - self.assertIn("VERSION_NEXT_FEATURE", new_content) + # Act + utils.replace_version_next(version) - new_content = (tests_dir / "mock_file.bzl").read_text() - self.assertIn("VERSION_NEXT_FEATURE", new_content) + # Assert + new_content = (bazel_dir / "mock_file.bzl").read_text() + assert "VERSION_NEXT_FEATURE" in new_content + new_content = (tools_dir / "mock_file.bzl").read_text() + assert "VERSION_NEXT_FEATURE" in new_content -if __name__ == "__main__": - unittest.main() + new_content = (tests_dir / "mock_file.bzl").read_text() + assert "VERSION_NEXT_FEATURE" in new_content diff --git a/tools/private/release/backport_create_releases.py b/tools/private/release/backport_create_releases.py index 662ef96eb1..a6d2beabdf 100644 --- a/tools/private/release/backport_create_releases.py +++ b/tools/private/release/backport_create_releases.py @@ -162,7 +162,7 @@ def run(self) -> int: ) new_issue_num = "" else: - new_issue_num = self._gh.create_tracking_issue( + new_issue_num = self._gh.create_release_tracking_issue( version, issue_template ) print( diff --git a/tools/private/release/create_release_issue.py b/tools/private/release/create_release_issue.py index 1c28e86c7d..72b71aa716 100644 --- a/tools/private/release/create_release_issue.py +++ b/tools/private/release/create_release_issue.py @@ -42,7 +42,7 @@ def run(self) -> int: lines = [line for line in lines if not re.search(r"Tag RC\d+", line)] template_content = "\n".join(lines) - issue_num = self.gh.create_tracking_issue(version, template_content) + issue_num = self.gh.create_release_tracking_issue(version, template_content) print(f"Created tracking issue #{issue_num} for v{version}") return 0 diff --git a/tools/private/release/gh.py b/tools/private/release/gh.py index 27206f1ffe..9d157f31d6 100644 --- a/tools/private/release/gh.py +++ b/tools/private/release/gh.py @@ -1,9 +1,11 @@ """GitHub CLI helper functions for the release tool.""" +import enum import json import os import re import tempfile +from typing import TypedDict from tools.private.release.release_issue import BackportTask from tools.private.release.shell import run_cmd @@ -24,6 +26,71 @@ GH_REACTION_EYES = "eyes" +class BackportTaskStatus(str, enum.Enum): + """Status strings for backport tasks on a release tracking issue.""" + + PENDING = "pending" + DONE = "done" + RESOLVED = "resolved" + OPEN_PR = "open-pr" + DRAFT_PR = "draft-pr" + ERROR_NOT_FOUND = "error-not-found" + ERROR_CLOSED_PR = "error-closed-pr" + ERROR_NO_MERGE_COMMIT = "error-no-merge-commit" + ERROR_UNKNOWN = "error-unknown" + ERROR_RESOLUTION_FAILED = "error-resolution-failed" + ERROR_MERGE_CONFLICT = "error-merge-conflict" + ERROR_INVALID_PR = "error-invalid-pr" + IGNORE = "ignore" + + def __str__(self) -> str: + return self.value + + +class IssueDict(TypedDict, total=False): + """In-memory representation of a GitHub Issue object. + + See GitHub API docs: + https://docs.github.com/en/rest/issues/issues#get-an-issue + """ + + number: int + title: str + body: str + labels: list[str] + url: str + + +class AutoMergeDict(TypedDict, total=False): + """Representation of auto-merge status on a Pull Request. + + See GitHub API docs: + https://docs.github.com/en/rest/pulls/pulls#get-a-pull-request + """ + + merge_method: str + + +class PrDict(TypedDict, total=False): + """In-memory representation of a GitHub Pull Request object. + + See GitHub API docs: + https://docs.github.com/en/rest/pulls/pulls#get-a-pull-request + """ + + number: int + title: str + body: str + base: str + head: str + labels: list[str] + url: str + state: str + isDraft: bool + mergeCommit: dict[str, str] + auto_merge: AutoMergeDict | None + + class MultipleTrackingIssuesError(ValueError): """Raised when multiple open tracking issues are found for a version.""" @@ -93,97 +160,107 @@ def list_issues( label: str | None = None, state: str | None = None, search: str | None = None, - ) -> list[dict]: + ) -> list[IssueDict]: """Helper to list issues using gh CLI. Args: fields: Comma-separated list of fields to return. label: Filter by label. - state: Filter by state (open, closed, all). + state: Filter by state ('open', 'closed', 'all'). search: Search query. Returns: - A list of dictionaries representing the issues. + A list of issue dictionaries. """ - cmd = ["list"] + cmd = ["list", f"--json={fields}"] if label: cmd.append(f"--label={label}") if state: cmd.append(f"--state={state}") if search: cmd.append(f"--search={search}") - cmd.append(f"--json={fields}") output = self._gh_issue(*cmd) return json.loads(output) if output else [] - def get_open_tracking_issues(self, version: str | None = None) -> list[dict]: - """Returns a list of open tracking issues with the 'type: release' label. + def get_open_tracking_issues(self, version: str | None = None) -> list[IssueDict]: + """Finds open tracking issues for release. Args: - version: Optional version to filter by. + version: Optional specific version to match (e.g., "1.0.0"). Returns: - A list of open tracking issues. + List of matching open release tracking issue dictionaries. """ - search = f'"Release {version}" in:title' if version else None + search = f"Release {version}" if version else None return self.list_issues( + fields="number,title,url", label=RELEASE_LABEL, state="open", search=search, - fields="number,title,url", ) def get_release_tracking_issue(self, version: str) -> int: - """Resolves the tracking issue number for a given version. - - Searches for an open issue with label 'type: release' and 'Release - ' in the title. + """Finds the single open tracking issue for a given version. Args: - version: The version to find the tracking issue for. + version: Version string (e.g. "1.0.0"). Returns: - The tracking issue number. + The issue number. Raises: NoTrackingIssueError: If no open tracking issue is found. - MultipleTrackingIssuesError: If multiple open tracking issues are - found. + MultipleTrackingIssuesError: If multiple open tracking issues are found. """ - matching_issues = self.get_open_tracking_issues(version) - - exact_matches = [] - for issue in matching_issues: - if issue["title"] == f"Release {version}": - exact_matches.append(issue) - - if not exact_matches: + issues = self.get_open_tracking_issues(version) + matching = [i for i in issues if i["title"] == f"Release {version}"] + if not matching: raise NoTrackingIssueError( - f"No open tracking issue found matching 'Release {version}' " - f"in repo {self.repo} with label '{RELEASE_LABEL}'" + f"No open tracking issue found for Release {version}" ) - if len(exact_matches) > 1: - urls = [issue["url"] for issue in exact_matches] + if len(matching) > 1: raise MultipleTrackingIssuesError( - f"Multiple open tracking issues found for version {version} " - f"in repo {self.repo} with label '{RELEASE_LABEL}':\n" + "\n".join(urls) + f"Multiple open tracking issues found for Release {version}: " + + ", ".join(str(i["number"]) for i in matching) ) + return matching[0]["number"] - return exact_matches[0]["number"] + def create_issue( + self, title: str, body: str, labels: list[str] | None = None + ) -> int: + """Creates an issue using gh CLI. - def create_tracking_issue(self, version: str, template_content: str) -> int: - """Creates a new release tracking issue from template content. + Args: + title: Title of the issue. + body: Body of the issue. + labels: List of labels to add. - Strips YAML frontmatter if present. + Returns: + The issue number. + """ + cmd = ["create", f"--title={title}", f"--body={body}"] + if labels: + for label in labels: + cmd.append(f"--label={label}") + + output = self._gh_issue(*cmd) + if not output: + raise RuntimeError("gh issue create returned no output") + # output is URL: https://github.com/owner/repo/issues/123 + return int(output.rstrip("/").split("/")[-1]) + + def create_release_tracking_issue(self, version: str, template_content: str) -> int: + """Creates a release tracking issue from a template. Args: - version: The version to create the tracking issue for. - template_content: The markdown template content for the issue body. + version: Release version string (e.g., "1.0.0"). + template_content: Content of the issue template markdown file. Returns: The created issue number. """ + title = f"Release {version}" # Strip YAML frontmatter if present issue_body = template_content if template_content.startswith("---"): @@ -191,110 +268,111 @@ def create_tracking_issue(self, version: str, template_content: str) -> int: if len(parts) >= 3: issue_body = parts[2].strip() - with tempfile.NamedTemporaryFile(mode="w", suffix=".md") as f: - f.write(issue_body) - f.flush() - temp_path = f.name - - output = self._gh_issue( - "create", - f"--title=Release {version}", - f"--label={RELEASE_LABEL}", - f"--body-file={temp_path}", - ) - if not output: - raise RuntimeError("Failed to get issue URL from gh issue create") - issue_url = output.strip() - issue_num = int(issue_url.split("/")[-1]) - return issue_num - - def create_issue( - self, title: str, body: str, labels: list[str] | None = None - ) -> int: - """Creates a generic issue. - - Args: - title: The title of the issue. - body: The body of the issue. - labels: Optional list of labels to add. - - Returns: - The created issue number. - """ - with tempfile.NamedTemporaryFile(mode="w", suffix=".md") as f: - f.write(body) - f.flush() - temp_path = f.name - - cmd = [ - "create", - f"--title={title}", - f"--body-file={temp_path}", - ] - if labels: - for label in labels: - cmd.append(f"--label={label}") - - output = self._gh_issue(*cmd) - if not output: - raise RuntimeError("Failed to get issue URL from gh issue create") - issue_url = output.strip() - issue_num = int(issue_url.split("/")[-1]) - return issue_num + return self.create_issue(title=title, body=issue_body, labels=[RELEASE_LABEL]) def get_issue_body(self, issue_num: int) -> str: - """Fetches the body of a specific issue. + """Gets the body content of an issue. Args: issue_num: The issue number. Returns: - The issue body markdown. + The body string of the issue. """ - output = self._gh_issue( - "view", - str(issue_num), - "--json=body", - "--jq=.body", - ) - return output if output else "" + output = self._gh_issue("view", str(issue_num), "--json=body") + if not output: + return "" + data = json.loads(output) + return data.get("body", "") def get_issue_title(self, issue_num: int) -> str: - """Fetches the title of a specific issue. + """Gets the title of an issue. Args: issue_num: The issue number. Returns: - The issue title. + The title string of the issue. """ - output = self._gh_issue( - "view", - str(issue_num), - "--json=title", - ) - return json.loads(output)["title"] if output else "" + output = self._gh_issue("view", str(issue_num), "--json=title") + if not output: + return "" + data = json.loads(output) + return data.get("title", "") def update_issue_body(self, issue_num: int, body: str) -> None: - """Updates the body of a specific issue. + """Updates the body of an issue. Args: issue_num: The issue number. - body: The new issue body markdown. + body: The new body content. """ - with tempfile.NamedTemporaryFile(mode="w", suffix=".md", delete=False) as f: + with tempfile.NamedTemporaryFile("w", delete=False, mode="w") as f: f.write(body) + f.flush() temp_path = f.name + try: self._gh_issue( - "edit", - str(issue_num), - f"--body-file={temp_path}", - capture_output=False, + "edit", str(issue_num), f"--body-file={temp_path}", capture_output=False ) finally: if os.path.exists(temp_path): - os.unlink(temp_path) + os.remove(temp_path) + + def resolve_pr_number(self, pr_ref: str) -> int: + """Resolves a PR reference (number, #number, or GitHub URL) to a PR number. + + Args: + pr_ref: PR number string (e.g., "123", "#123") or URL. + + Returns: + The integer PR number. + + Raises: + ValueError: If the PR reference cannot be resolved or is for another repo. + """ + clean_ref = pr_ref.lstrip("#") + if clean_ref.isdigit(): + return int(clean_ref) + + if pr_ref.startswith("http"): + pattern = rf"github\.com/{re.escape(self.repo)}/pull/(\d+)(/|\?|\Z)" + match = re.search(pattern, pr_ref, re.IGNORECASE) + if match: + return int(match.group(1)) + raise ValueError( + f"URL is not for the configured repository ({self.repo}): {pr_ref}" + ) + + raise ValueError(f"Could not resolve PR reference: {pr_ref}") + + def get_pr_info(self, pr_num: int) -> PrDict: + """Gets info about a PR using gh CLI. + + Args: + pr_num: The PR number. + + Returns: + Dictionary containing PR fields (state, isDraft, mergeCommit, etc.). + """ + output = self._gh_pr("view", str(pr_num), "--json=state,isDraft,mergeCommit") + return json.loads(output) if output else {} + + def get_pr_comments(self, pr_num: int) -> list[dict]: + """Gets all comments for a PR using gh CLI. + + Args: + pr_num: The PR number. + + Returns: + List of comment objects (with body, author, etc.). + """ + output = self._gh_pr("view", str(pr_num), "--json=comments") + if not output: + return [] + data = json.loads(output) + return data.get("comments", []) def create_pr( self, @@ -306,10 +384,10 @@ def create_pr( """Creates a pull request. Args: - title: The title of the PR. - body: The body of the PR. - base: The base branch to merge into (default: 'main'). - labels: Optional list of labels to add to the PR. + title: Title of the PR. + body: Body of the PR. + base: Base branch to merge into (default: "main"). + labels: Optional list of labels to add. Returns: The URL of the created PR. @@ -342,95 +420,31 @@ def enable_auto_merge(self, pr_num: int, method: str = "squash") -> None: cmd.append("--merge") self._gh_pr(*cmd, capture_output=False) - def get_open_pr(self, branch_name: str) -> dict | None: - """Returns PR info if an open PR exists for the given branch. + def get_open_pr(self, branch_name: str) -> PrDict | None: + """Finds an open PR for the given branch. Args: - branch_name: The head branch name of the PR. + branch_name: The head branch name to search for. Returns: - A dictionary with 'number' and 'url' of the PR, or None. + Dictionary with 'number' and 'url' if an open PR exists, else None. """ - output = self._gh_pr( + cmd = [ "list", f"--head={branch_name}", "--state=open", "--json=number,url", - ) + ] + output = self._gh_pr(*cmd) prs = json.loads(output) if output else [] return prs[0] if prs else None - def get_pr_info(self, pr_num: int) -> dict: - """Gets information about a PR. - - Includes state, merge commit, body, and draft status. - - Args: - pr_num: The PR number. - - Returns: - A dictionary containing the PR info. - """ - output = self._gh_pr( - "view", - str(pr_num), - "--json=state,mergeCommit,body,isDraft", - ) - return json.loads(output) if output else {} - - def get_pr_comments(self, pr_num: int) -> list[dict]: - """Gets comments for a PR. - - Args: - pr_num: The PR number. - - Returns: - A list of comments. - """ - output = self._gh_pr( - "view", - str(pr_num), - "--json=comments", - ) - return json.loads(output).get("comments") or [] - - def resolve_pr_number(self, pr_ref: str) -> int: - """Resolves a PR reference (number, #number, URL) to a PR number. - - Args: - pr_ref: The PR reference string. - - Returns: - The resolved PR number. - - Raises: - ValueError: If the reference cannot be resolved. - """ - # 1. Try number (e.g. "123" or "#123") - clean_ref = pr_ref.lstrip("#") - if clean_ref.isdigit(): - return int(clean_ref) - - # 2. Try URL (starts with http) - if pr_ref.startswith("http"): - # Try to extract PR number from URL using regex - # Pattern matches: github.com//pull/ followed by /, ?, or EOF - pattern = rf"github\.com/{re.escape(self.repo)}/pull/(\d+)(/|\?|\Z)" - match = re.search(pattern, pr_ref, re.IGNORECASE) - if match: - return int(match.group(1)) - raise ValueError( - f"URL is not for the configured repository ({self.repo}): {pr_ref}" - ) - - raise ValueError(f"Could not resolve PR reference: {pr_ref}") - def post_issue_comment(self, issue_num: int, comment_body: str) -> None: - """Posts a comment to a specific issue. + """Posts a comment on an issue or PR. Args: - issue_num: The issue number. - comment_body: The comment body markdown. + issue_num: The issue or PR number. + comment_body: The body content of the comment. """ self._gh_issue( "comment", @@ -440,22 +454,15 @@ def post_issue_comment(self, issue_num: int, comment_body: str) -> None: ) def add_comment_reaction(self, comment_id: int, reaction: str) -> None: - """Adds a reaction to a comment. + """Adds a reaction to an issue or PR comment. Args: - comment_id: The ID of the comment. - reaction: The reaction type (e.g. '+1', '-1', 'eyes', etc). + comment_id: The comment ID (note: gh api endpoint needed for comment reactions). + reaction: The reaction type (e.g., "+1", "-1", "rocket"). """ - path = f"/repos/{self.repo}/issues/comments/{comment_id}/reactions" self._run_gh( "api", - "--method", - "POST", - "-H", - "Accept: application/vnd.github+json", - "-H", - "X-GitHub-Api-Version: 2022-11-28", - path, + f"repos/{self.repo}/issues/comments/{comment_id}/reactions", "-f", f"content={reaction}", capture_output=False, @@ -474,40 +481,61 @@ def get_merge_commits_for_prs( Returns: The list of resolved BackportTask items. """ - resolved_items = [] - for item in pending_items: - pr_num = int(item.pr_ref.lstrip("#")) - print(f"Resolving PR #{pr_num} to merge commit...") - try: - pr_info = self.get_pr_info(pr_num) - if not pr_info: - print(f"PR #{pr_num} not found. Gating.") - item.status = "error-not-found" - else: - state = pr_info.get("state") - is_draft = pr_info.get("isDraft", False) - if state == "OPEN" or is_draft: - print( - f"PR #{pr_num} is open or draft (state: {state}," - f" draft: {is_draft}). Ignoring." - ) - item.status = "open-pr" if not is_draft else "draft-pr" - elif state == "CLOSED": - print(f"PR #{pr_num} is closed but not merged. Gating.") - item.status = "error-closed-pr" - elif state == "MERGED": - merge_commit = pr_info.get("mergeCommit") - if merge_commit and "oid" in merge_commit: - item.commit = merge_commit["oid"] - item.status = "resolved" - else: - print(f"PR #{pr_num} has no merge commit SHA. Gating.") - item.status = "error-no-merge-commit" + return resolve_merge_commits_for_prs(self, pending_items) + + +def resolve_merge_commits_for_prs( + gh_client: GitHub, pending_items: list[BackportTask] +) -> list[BackportTask]: + """Resolves PR references in pending backports to their merge commit SHAs. + + Updates item.status based on PR state if it cannot be resolved. + + Args: + gh_client: The GitHub client. + pending_items: A list of BackportTask items to resolve. + + Returns: + The list of resolved BackportTask items. + """ + resolved_items = [] + for item in pending_items: + pr_num = int(item.pr_ref.lstrip("#")) + print(f"Resolving PR #{pr_num} to merge commit...") + try: + pr_info = gh_client.get_pr_info(pr_num) + if not pr_info: + print(f"PR #{pr_num} not found. Gating.") + item.status = BackportTaskStatus.ERROR_NOT_FOUND + else: + state = pr_info.get("state") + is_draft = pr_info.get("isDraft", False) + if state == "OPEN" or is_draft: + print( + f"PR #{pr_num} is open or draft (state: {state}," + f" draft: {is_draft}). Ignoring." + ) + item.status = ( + BackportTaskStatus.OPEN_PR + if not is_draft + else BackportTaskStatus.DRAFT_PR + ) + elif state == "CLOSED": + print(f"PR #{pr_num} is closed but not merged. Gating.") + item.status = BackportTaskStatus.ERROR_CLOSED_PR + elif state == "MERGED": + merge_commit = pr_info.get("mergeCommit") + if merge_commit and "oid" in merge_commit: + item.commit = merge_commit["oid"] + item.status = BackportTaskStatus.RESOLVED else: - print(f"PR #{pr_num} has unknown state: {state}. Gating.") - item.status = "error-unknown" - except Exception as e: - print(f"Error resolving PR #{pr_num}: {e}. Gating.") - item.status = "error-resolution-failed" - resolved_items.append(item) - return resolved_items + print(f"PR #{pr_num} has no merge commit SHA. Gating.") + item.status = BackportTaskStatus.ERROR_NO_MERGE_COMMIT + else: + print(f"PR #{pr_num} has unknown state: {state}. Gating.") + item.status = BackportTaskStatus.ERROR_UNKNOWN + except Exception as e: + print(f"Error resolving PR #{pr_num}: {e}. Gating.") + item.status = BackportTaskStatus.ERROR_RESOLUTION_FAILED + resolved_items.append(item) + return resolved_items diff --git a/tools/private/release/mock_gh.py b/tools/private/release/mock_gh.py index 3a5a8dde48..e5def53799 100644 --- a/tools/private/release/mock_gh.py +++ b/tools/private/release/mock_gh.py @@ -2,15 +2,36 @@ import re -from tools.private.release.gh import RELEASE_LABEL +from tools.private.release.gh import ( + RELEASE_LABEL, + IssueDict, + MultipleTrackingIssuesError, + NoTrackingIssueError, + PrDict, + resolve_merge_commits_for_prs, +) class MockGitHub: def __init__(self, repo: str = "bazel-contrib/rules_python"): self.repo = repo - self.issues = {} + self.issues: dict[int, IssueDict] = {} self.next_issue_num = 1001 - self.prs = {} # num -> pr_info + self.prs: dict[int, PrDict] = {} # num -> pr_info + self.issue_comments: dict[int, list[str]] = {} + self.reactions: dict[int, list[str]] = {} + self.pr_comments: dict[int, list[dict]] = {} + + def post_issue_comment(self, issue_num: int, comment_body: str) -> None: + self.issue_comments.setdefault(issue_num, []).append(comment_body) + + def add_comment_reaction(self, comment_id: int, reaction: str) -> None: + self.reactions.setdefault(comment_id, []).append(reaction) + + def enable_auto_merge(self, pr_num: int, method: str = "squash") -> None: + if pr_num not in self.prs: + self.create_pr(title="", body="") + self.prs[pr_num]["auto_merge"] = {"merge_method": method} def create_issue( self, title: str, body: str, labels: list[str] | None = None @@ -26,7 +47,7 @@ def create_issue( } return issue_num - def create_tracking_issue(self, version: str, template_content: str) -> int: + def create_release_tracking_issue(self, version: str, template_content: str) -> int: # Strip YAML frontmatter if present (simplified copy from gh.py) issue_body = template_content if template_content.startswith("---"): @@ -43,6 +64,11 @@ def get_issue_body(self, issue_num: int) -> str: raise ValueError(f"Issue #{issue_num} not found in MockGitHub") return self.issues[issue_num]["body"] + def get_issue_title(self, issue_num: int) -> str: + if issue_num not in self.issues: + raise ValueError(f"Issue #{issue_num} not found in MockGitHub") + return self.issues[issue_num]["title"] + def update_issue_body(self, issue_num: int, body: str): if issue_num not in self.issues: raise ValueError(f"Issue #{issue_num} not found in MockGitHub") @@ -64,7 +90,52 @@ def resolve_pr_number(self, pr_ref: str) -> int: ) raise ValueError(f"Could not resolve PR ref: {pr_ref}") - def get_open_tracking_issues(self, version: str | None = None) -> list[dict]: + def get_release_tracking_issue(self, version: str) -> int: + search_title = f"Release {version}" + matching = [ + num + for num, issue in self.issues.items() + if issue["title"] == search_title + and RELEASE_LABEL in issue.get("labels", []) + ] + if not matching: + raise NoTrackingIssueError( + f"No open tracking issue found for Release {version}" + ) + if len(matching) > 1: + raise MultipleTrackingIssuesError( + f"Multiple open tracking issues found for Release {version}" + ) + return matching[0] + + def create_pr( + self, + title: str, + body: str, + base: str = "main", + labels: list[str] | None = None, + ) -> str: + pr_num = self.next_issue_num + self.next_issue_num += 1 + url = f"https://github.com/{self.repo}/pull/{pr_num}" + self.prs[pr_num] = { + "title": title, + "body": body, + "base": base, + "labels": labels or [], + "number": pr_num, + "url": url, + "state": "OPEN", + } + return url + + def get_open_pr(self, branch_name: str) -> PrDict | None: + for pr in self.prs.values(): + if pr.get("head") == branch_name and pr.get("state") == "OPEN": + return pr + return None + + def get_open_tracking_issues(self, version: str | None = None) -> list[IssueDict]: results = [] for issue in self.issues.values(): if RELEASE_LABEL in issue["labels"]: @@ -75,10 +146,16 @@ def get_open_tracking_issues(self, version: str | None = None) -> list[dict]: results.append(issue) return results - def get_pr_info(self, pr_num: int) -> dict: + def get_pr_info(self, pr_num: int) -> PrDict: if pr_num in self.prs: return self.prs[pr_num] return { "state": "MERGED", "mergeCommit": {"oid": f"mock_merge_sha_{pr_num}"}, } + + def get_pr_comments(self, pr_num: int) -> list[dict]: + return self.pr_comments.get(pr_num, []) + + def get_merge_commits_for_prs(self, pending_items: list) -> list: + return resolve_merge_commits_for_prs(self, pending_items) diff --git a/tools/private/release/prepare.py b/tools/private/release/prepare.py index fd19362bec..b01f135ff8 100644 --- a/tools/private/release/prepare.py +++ b/tools/private/release/prepare.py @@ -88,7 +88,9 @@ def run(self) -> int: f"No active tracking issue found for {version}." " Creating a new one..." ) - issue_num = self.gh.create_tracking_issue(version, template_content) + issue_num = self.gh.create_release_tracking_issue( + version, template_content + ) print(f"Tracking issue: #{issue_num}") else: print(f"Tracking issue: #{issue_num}") From 7fce328b34a7296e97548e8f3f24937027e65ae4 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Mon, 20 Jul 2026 11:17:46 -0700 Subject: [PATCH 844/922] refactor(release): extract pypi publishing to standalone workflow (#3940) Factor out PyPI publishing from release_publish.yaml into a new standalone workflow publish_pypi.yaml. This enables PyPI publishing to be manually executed via workflow_dispatch while maintaining the ability to be invoked by release_publish.yaml during release execution. --- .github/workflows/publish_pypi.yaml | 32 ++++++++++++++++++++++++++ .github/workflows/release_publish.yaml | 20 ++++------------ RELEASING.md | 10 ++++++++ 3 files changed, 47 insertions(+), 15 deletions(-) create mode 100644 .github/workflows/publish_pypi.yaml diff --git a/.github/workflows/publish_pypi.yaml b/.github/workflows/publish_pypi.yaml new file mode 100644 index 0000000000..44f0a1d9fb --- /dev/null +++ b/.github/workflows/publish_pypi.yaml @@ -0,0 +1,32 @@ +name: Publish to PyPI + +on: + workflow_call: + inputs: + tag_name: + description: "release tag: tag that will be released" + required: true + type: string + workflow_dispatch: + inputs: + tag_name: + description: "release tag: tag that will be released" + required: true + type: string + +jobs: + publish_pypi: + name: Publish runfiles to PyPI + runs-on: ubuntu-latest + steps: + - name: Checkout + uses: actions/checkout@v7 + with: + ref: ${{ inputs.tag_name || github.ref_name }} + - env: + # This special value tells pypi that the user identity is supplied within the token + TWINE_USERNAME: __token__ + # Note, the PYPI_API_TOKEN is for the rules-python pypi user, added by @rickylev on + # https://github.com/bazel-contrib/rules_python/settings/secrets/actions + TWINE_PASSWORD: ${{ secrets.PYPI_API_TOKEN }} + run: bazel run --stamp --embed_label=${{ inputs.tag_name || github.ref_name }} //python/runfiles:wheel.publish diff --git a/.github/workflows/release_publish.yaml b/.github/workflows/release_publish.yaml index 8379c3d46d..eedf653fc1 100644 --- a/.github/workflows/release_publish.yaml +++ b/.github/workflows/release_publish.yaml @@ -77,19 +77,9 @@ jobs: publish_pypi: # We just want publish_pypi last, since once uploaded, it can't be changed. - name: Publish runfiles to PyPI needs: publish_bcr - runs-on: ubuntu-latest - steps: - - name: Checkout - uses: actions/checkout@v7 - with: - ref: ${{ inputs.tag_name || github.ref_name }} - - if: github.event_name == 'push' || github.event.inputs.publish_to_pypi - env: - # This special value tells pypi that the user identity is supplied within the token - TWINE_USERNAME: __token__ - # Note, the PYPI_API_TOKEN is for the rules-python pypi user, added by @rickylev on - # https://github.com/bazel-contrib/rules_python/settings/secrets/actions - TWINE_PASSWORD: ${{ secrets.PYPI_API_TOKEN }} - run: bazel run --stamp --embed_label=${{ inputs.tag_name || github.ref_name }} //python/runfiles:wheel.publish + if: github.event_name == 'push' || inputs.publish_to_pypi + uses: ./.github/workflows/publish_pypi.yaml + with: + tag_name: ${{ inputs.tag_name || github.ref_name }} + secrets: inherit diff --git a/RELEASING.md b/RELEASING.md index a03cd83652..1942553538 100644 --- a/RELEASING.md +++ b/RELEASING.md @@ -63,6 +63,16 @@ you can set the `publish_to_pypi` input to `false`: gh workflow run release_publish.yaml --ref -f publish_to_pypi=false ``` +### Manually publishing to PyPI + +If PyPI publishing failed or was skipped during the main release, the PyPI +publishing workflow can be triggered manually using the GitHub CLI (`gh`) or +via the [GitHub Actions UI](https://github.com/bazel-contrib/rules_python/actions/workflows/publish_pypi.yaml): + +```shell +gh workflow run publish_pypi.yaml --ref -f tag_name= +``` + ### Determining Semantic Version **rules_python** uses [semantic version](https://semver.org), so releases with From a57eff9129a67d034d4bf3a6f4b22914196572b4 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Mon, 20 Jul 2026 11:47:56 -0700 Subject: [PATCH 845/922] docs(releasing): update PyPI account owner information (#3941) Update RELEASING.md to clarify that rickeylev manages the PyPI account, while Google retains recovery access via recovery codes and the rules-python-pypi@google.com email address. --- RELEASING.md | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/RELEASING.md b/RELEASING.md index 1942553538..2b3d5859c1 100644 --- a/RELEASING.md +++ b/RELEASING.md @@ -260,5 +260,6 @@ The checklist items use metadata suffix: `| key=value key2=value2`. ### PyPI user rules-python Part of the release process uploads packages to PyPI as the user `rules-python`. -This account is managed by Google; contact rules-python-pyi@google.com if -something needs to be done with the PyPI account. +This account is managed by `rickeylev`. Note that Google also has recovery access +and can be contacted at rules-python-pypi@google.com if something needs to be +done with the PyPI account. From ced895d0d59eefc24d1d5522a04861dae6d01e98 Mon Sep 17 00:00:00 2001 From: Logan Pulley Date: Mon, 20 Jul 2026 18:51:06 -0500 Subject: [PATCH 846/922] fix(pypi): don't expose source-less uv.lock packages (#3938) > [!NOTE] > I created this PR with AI. Fixes #3934. ## Problem `pip.parse(uv_lock = ...)` exposes every `[[package]]` in the lock, including uv workspace/root members with `source = { virtual = "." }` (and editable installs). These resolve to no wheel/sdist (`srcs = []`) but were still marked `is_exposed = True`, so the hub adds them to `all_requirements` / `all_whl_requirements` and creates an alias to a subpackage that doesn't exist. Anything enumerating the full set then fails analysis, e.g. `modules_mapping(wheels = all_whl_requirements)`: ``` ERROR: no such package '@@rules_python++pip+pip//myproject': BUILD file not found ... and referenced by '//:modules_map' ``` ## Fix `_parse_uv_lock_json` in `python/private/pypi/parse_requirements.bzl` set `is_exposed = True` unconditionally. This gates it on whether the package actually resolved to any sources: ```starlark is_exposed = bool(info["resolved_srcs"]), ``` The entry is still kept; it just isn't exposed when nothing resolved. This mirrors the requirements path, which already gates `is_exposed`. ## Tests As called out in the issue, this flips the expectations in two existing tests, both of which assert on source-less packages: - `_test_uv_lock_primary_source_includes_virtual` (the `virtual_pkg` entry) - `_test_uv_lock_requires_dist_extras` (the `root_pkg` entry) Both now expect `is_exposed = False`. A `news/3934.fixed.md` fragment is included. --- news/3934.fixed.md | 7 +++++++ python/private/pypi/parse_requirements.bzl | 9 ++++++++- .../pypi/parse_requirements/parse_requirements_tests.bzl | 4 ++-- 3 files changed, 17 insertions(+), 3 deletions(-) create mode 100644 news/3934.fixed.md diff --git a/news/3934.fixed.md b/news/3934.fixed.md new file mode 100644 index 0000000000..a30f475770 --- /dev/null +++ b/news/3934.fixed.md @@ -0,0 +1,7 @@ +(pypi) `pip.parse(uv_lock = ...)` no longer exposes uv workspace/root members +that resolve to no wheel or sdist (e.g. `source = { virtual = "." }` or editable +installs). Previously these source-less packages were added to the hub's +`all_requirements` / `all_whl_requirements` with an alias to a subpackage that +does not exist, breaking analysis for anything enumerating the full set such as +`modules_mapping(wheels = all_whl_requirements)` +([#3934](https://github.com/bazel-contrib/rules_python/issues/3934)). diff --git a/python/private/pypi/parse_requirements.bzl b/python/private/pypi/parse_requirements.bzl index be3ad9a5e5..648ec9e65c 100644 --- a/python/private/pypi/parse_requirements.bzl +++ b/python/private/pypi/parse_requirements.bzl @@ -292,7 +292,14 @@ def _parse_uv_lock_json(uv_lock, all_platforms, logger, extra_pip_args = None, p versions = sorted(info["versions"].keys()) item = struct( name = norm_name, - is_exposed = True, + # Only expose packages that resolved to at least one source. uv + # workspace/root members (e.g. `source = { virtual = "." }` or + # editable installs) resolve to no wheel/sdist, so exposing them + # would add a dangling entry to the hub's `all_requirements` / + # `all_whl_requirements` and create an alias to a subpackage that + # doesn't exist. This mirrors the requirements path, which also + # gates `is_exposed`. + is_exposed = bool(info["resolved_srcs"]), is_multiple_versions = len(versions) > 1, index_url = info["index_url"], srcs = info["resolved_srcs"], diff --git a/tests/pypi/parse_requirements/parse_requirements_tests.bzl b/tests/pypi/parse_requirements/parse_requirements_tests.bzl index 576a31ae5d..57b3ea5d3e 100644 --- a/tests/pypi/parse_requirements/parse_requirements_tests.bzl +++ b/tests/pypi/parse_requirements/parse_requirements_tests.bzl @@ -1193,7 +1193,7 @@ def _test_uv_lock_primary_source_includes_virtual(env): struct( name = "virtual_pkg", index_url = "", - is_exposed = True, + is_exposed = False, is_multiple_versions = False, srcs = [], ), @@ -1620,7 +1620,7 @@ def _test_uv_lock_requires_dist_extras(env): struct( name = "root_pkg", index_url = "", - is_exposed = True, + is_exposed = False, is_multiple_versions = False, srcs = [], ), From 8cb7d1cdc8b79912b917c90ab97497a407347f9a Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Tue, 21 Jul 2026 19:20:36 -0700 Subject: [PATCH 847/922] refactor(reviewbot): use separate jobs to protect access to credentials (#3943) automated_pr_review.yaml previously checked out untrusted pull request code into the workflow root and ran uv run directly from that directory while exposing GEMINI_API_KEY. Because uv run by default searches $PWD for project build files (pyproject.toml, setup.py), an untrusted PR could trigger arbitrary code execution during workflow runs. To fix: - Split the workflow into two separate jobs: `prepare_diff` (runs in an unprivileged context without access to secrets to safely extract the git diff from untrusted PR code) and `review` (runs in the trusted base branch context with secrets, consuming only the text diff artifact without checking out untrusted PR head onto disk). - Check out the untrusted PR branch into a dedicated `untrusted_pr_head` subdirectory in `prepare_diff` so it is treated strictly as input data. - Run `uv` with `--no-project --directory .` inside `working-directory: reviewbot` to prevent `uv` from searching `$PWD` or discovering untrusted project configuration files. --- .github/workflows/automated_pr_review.yaml | 45 ++++++++++++++----- tools/private/reviewbot/antigravity_review.py | 7 ++- 2 files changed, 40 insertions(+), 12 deletions(-) diff --git a/.github/workflows/automated_pr_review.yaml b/.github/workflows/automated_pr_review.yaml index 1076101db6..3fe93db7f3 100644 --- a/.github/workflows/automated_pr_review.yaml +++ b/.github/workflows/automated_pr_review.yaml @@ -21,10 +21,10 @@ jobs: - name: Workflow trigger check run: echo "Workflow triggered successfully." - review: + # Job 1: Runs without secrets to extract the git diff from untrusted PR code. + # Keeps GEMINI_API_KEY away from any environment that checks out untrusted PR files. + prepare_diff: runs-on: ubuntu-latest - # Trigger only if it is a regular comment on a pull request, the comment body has a line - # starting with "/review", and the commenter is a maintainer (OWNER, MEMBER, or COLLABORATOR). if: > github.event_name == 'issue_comment' && github.event.issue.pull_request != null && (startsWith(github.event.comment.body, '/review') || @@ -32,19 +32,18 @@ jobs: contains(github.event.comment.body, '\r\n/review')) && contains(fromJson('["OWNER", "MEMBER", "COLLABORATOR"]'), github.event.comment.author_association) steps: - - name: Checkout PR Branch + - name: Checkout PR Branch (Data Only) uses: actions/checkout@v7 with: - # Note: In GHA, during an issue_comment event on a PR, github.event.pull_request is null - # and the PR number is placed inside github.event.issue.number (because every PR is an issue). - # Therefore, github.event.issue.number is the PR number when checking out refs/pull//head. ref: refs/pull/${{ github.event.pull_request.number || github.event.issue.number }}/head + path: untrusted_pr_head persist-credentials: false - name: Fetch Base Branch and Negotiate Minimal Diff History env: PR_COMMITS: ${{ github.event.pull_request.commits }} run: | + cd untrusted_pr_head # 1. Fetch the tip of main git fetch origin main:refs/remotes/origin/main --depth=1 @@ -59,8 +58,27 @@ jobs: git fetch --unshallow || git fetch --deepen=50 fi fi + git diff origin/main...HEAD > ../pr_diff.txt + + # Upload extracted diff as artifact to pass to Job 2 safely as text data. + - name: Upload Diff Artifact + uses: actions/upload-artifact@v4 + with: + name: pr_diff + path: pr_diff.txt + + # Job 2: Runs with secrets in base branch context. + review: + needs: prepare_diff + runs-on: ubuntu-latest + steps: + - name: Download Diff Artifact + uses: actions/download-artifact@v4 + with: + name: pr_diff - - name: Checkout Reviewbot (Base Branch) + # Check out reviewbot into a separate directory to isolate base branch tool code. + - name: Checkout Reviewbot (Base Branch Only) uses: actions/checkout@v7 with: sparse-checkout: | @@ -71,9 +89,16 @@ jobs: uses: astral-sh/setup-uv@v8.3.2 - name: Run Antigravity Review + # Run inside reviewbot directory so execution context is the trusted base branch. + # This also helps prevent uv from looking for config files in locations it shouldn't, + # i.e. by default, uv will look in $PWD for a pyproject file to build. + working-directory: reviewbot env: GEMINI_API_KEY: ${{ secrets.GEMINI_API_KEY }} GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} run: | - uv run reviewbot/tools/private/reviewbot/antigravity_review.py \ - --prompt reviewbot/tools/private/reviewbot/prompt.txt + # Use --no-project to prevent uv from discovering or building pyproject.toml/setup.py + # in the workspace, ensuring only standalone script dependencies are resolved. + uv run --no-project --directory . tools/private/reviewbot/antigravity_review.py \ + --prompt tools/private/reviewbot/prompt.txt \ + --diff-file ../pr_diff.txt diff --git a/tools/private/reviewbot/antigravity_review.py b/tools/private/reviewbot/antigravity_review.py index 3aef480f3b..4ae0ef1690 100644 --- a/tools/private/reviewbot/antigravity_review.py +++ b/tools/private/reviewbot/antigravity_review.py @@ -17,11 +17,14 @@ def parse_args(): parser = argparse.ArgumentParser() parser.add_argument("--prompt", required=True, help="Path to prompt file") + parser.add_argument("--diff-file", help="Path to pre-computed diff file") return parser.parse_args() -def get_pr_diff() -> str: +def get_pr_diff(diff_file: str | None = None) -> str: """Fetches the git diff for the current pull request against origin/main.""" + if diff_file and Path(diff_file).exists(): + return Path(diff_file).read_text() try: return subprocess.check_output( ["git", "diff", "origin/main...HEAD"], text=True, stderr=subprocess.DEVNULL @@ -35,7 +38,7 @@ async def main(): # Read prompt file and pre-hydrate with the exact PR code diff base_prompt = Path(args.prompt).read_text() - diff_text = get_pr_diff() + diff_text = get_pr_diff(args.diff_file) prompt = ( f"{base_prompt}\n\n## Pull Request Git Diff\n" f"Here is the exact code diff for this pull request:\n```diff\n{diff_text}\n```" From d21d3f073f70f57838b7828142d37c6de7e7d188 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Wed, 22 Jul 2026 10:34:27 -0700 Subject: [PATCH 848/922] test(tools): auto-patch release CLI helpers in conftest.py (#3939) Add an autouse fixture in conftest.py for tests/tools/private/release that automatically patches run_cmd, Git._run_git, and GitHub._run_gh using pytest-mock. This prevents tests from executing command line tools that could have side-effects. --- tests/tools/private/release/BUILD.bazel | 5 ++- tests/tools/private/release/conftest.py | 50 +++++++++++++++++++++++++ tests/tools/private/release/gh_test.py | 18 +++++++++ 3 files changed, 72 insertions(+), 1 deletion(-) create mode 100644 tests/tools/private/release/conftest.py diff --git a/tests/tools/private/release/BUILD.bazel b/tests/tools/private/release/BUILD.bazel index 5ac912a955..317a4626c9 100644 --- a/tests/tools/private/release/BUILD.bazel +++ b/tests/tools/private/release/BUILD.bazel @@ -4,7 +4,10 @@ load("//tests/support/pytest_test:pytest_test.bzl", "pytest_test") py_library( name = "release_test_helper", - srcs = ["release_test_helper.py"], + srcs = [ + "conftest.py", + "release_test_helper.py", + ], target_compatible_with = SUPPORTS_BZLMOD, deps = [ "//tools/private/release:mock_gh", diff --git a/tests/tools/private/release/conftest.py b/tests/tools/private/release/conftest.py new file mode 100644 index 0000000000..f8798c11c7 --- /dev/null +++ b/tests/tools/private/release/conftest.py @@ -0,0 +1,50 @@ +import dataclasses +from unittest.mock import MagicMock + +import pytest + +pytest_plugins = ["tests.tools.private.release.release_test_helper"] + + +@dataclasses.dataclass +class AutoPatchCmdHelpers: + """Dataclass holding mocked command helpers.""" + + run_cmd: MagicMock + run_git: MagicMock + run_gh: MagicMock + + +@pytest.fixture(name="mock_run_cmd") +def fixture_mock_run_cmd(mocker): + """Fixture to patch shell.run_cmd and its imports in git and gh modules.""" + mock = mocker.patch("tools.private.release.shell.run_cmd") + mocker.patch("tools.private.release.git.run_cmd", mock) + mocker.patch("tools.private.release.gh.run_cmd", mock) + return mock + + +@pytest.fixture(name="mock_run_git") +def fixture_mock_run_git(mocker): + """Fixture to patch Git._run_git.""" + return mocker.patch("tools.private.release.git.Git._run_git") + + +@pytest.fixture(name="mock_run_gh") +def fixture_mock_run_gh(mocker): + """Fixture to patch GitHub._run_gh.""" + return mocker.patch("tools.private.release.gh.GitHub._run_gh") + + +@pytest.fixture(name="auto_patch_cmd_helpers", autouse=True) +def fixture_auto_patch_cmd_helpers(mock_run_cmd, mock_run_git, mock_run_gh): + """Automatically patches run_cmd, Git, and GitHub CLI helpers. + + This prevents tests from executing command line tools that could have + side-effects. + """ + return AutoPatchCmdHelpers( + run_cmd=mock_run_cmd, + run_git=mock_run_git, + run_gh=mock_run_gh, + ) diff --git a/tests/tools/private/release/gh_test.py b/tests/tools/private/release/gh_test.py index ccc61e7f21..b8c346004b 100644 --- a/tests/tools/private/release/gh_test.py +++ b/tests/tools/private/release/gh_test.py @@ -1,6 +1,8 @@ import pytest +from tools.private.release import shell from tools.private.release.gh import GitHub +from tools.private.release.git import Git pytest_plugins = ["tests.tools.private.release.release_test_helper"] @@ -59,3 +61,19 @@ def test_resolve_pr_number_invalid(mocker, gh): with pytest.raises(ValueError, match="Could not resolve PR reference"): gh.resolve_pr_number("invalid-ref") mock_run_cmd.assert_not_called() + + +def test_auto_patched_helpers_prevent_real_execution(auto_patch_cmd_helpers): + # Calling run_cmd directly hits the mock + shell.run_cmd("echo", "test") + auto_patch_cmd_helpers.run_cmd.assert_called_with("echo", "test") + + # Git._run_git hits the mock + git = Git(".") + git._run_git("status") + auto_patch_cmd_helpers.run_git.assert_called_with("status") + + # GitHub._run_gh hits the mock + gh_obj = GitHub("foo/bar") + gh_obj._run_gh("issue", "list") + auto_patch_cmd_helpers.run_gh.assert_called_with("issue", "list") From c6d1c916f83fcea25cdf4c3477c3d54484c3c730 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Wed, 22 Jul 2026 19:09:08 -0700 Subject: [PATCH 849/922] tests: add workspace-compatible unified pypi hub so @pypi refs work (#3946) Currently, the unified pypi hub is bzlmod only. While this is WAI, it makes using the references in tests that would be good to run under workspace unable to run. To fix, have our workspace config create a unified hub repo with the underlying pypi hubs that it needs to map to. This is kept as an internal-only helper because it doesn't generalize well to arbitrary projects. --- WORKSPACE | 24 +++++++++++++--- python/private/pypi/BUILD.bazel | 5 +++- python/private/pypi/unified_hub_repo.bzl | 30 ++++++++++++++++++++ tests/support/pytest_test/BUILD.bazel | 5 ++-- tests/tools/private/release/BUILD.bazel | 36 ++++++++++++------------ 5 files changed, 74 insertions(+), 26 deletions(-) diff --git a/WORKSPACE b/WORKSPACE index 5a31274b21..59b8489760 100644 --- a/WORKSPACE +++ b/WORKSPACE @@ -138,7 +138,7 @@ pip_parse( requirements_lock = "//examples/wheel:requirements_server.txt", ) -load("@pypiserver//:requirements.bzl", install_pypiserver = "install_deps") +load("@pypiserver//:requirements.bzl", install_pypiserver = "install_deps", pypiserver_requirements = "all_requirements") install_pypiserver() @@ -151,7 +151,7 @@ pip_parse( requirements_lock = "//docs:requirements.txt", ) -load("@dev_pip//:requirements.bzl", docs_install_deps = "install_deps") +load("@dev_pip//:requirements.bzl", dev_pip_requirements = "all_requirements", docs_install_deps = "install_deps") docs_install_deps() @@ -164,7 +164,7 @@ pip_parse( requirements_lock = "//tests/multi_pypi/alpha:requirements.txt", ) -load("@pypi_alpha//:requirements.bzl", pypi_alpha_install_deps = "install_deps") +load("@pypi_alpha//:requirements.bzl", pypi_alpha_install_deps = "install_deps", pypi_alpha_requirements = "all_requirements") pypi_alpha_install_deps() @@ -174,6 +174,22 @@ pip_parse( requirements_lock = "//tests/multi_pypi/beta:requirements.txt", ) -load("@pypi_beta//:requirements.bzl", pypi_beta_install_deps = "install_deps") +load("@pypi_beta//:requirements.bzl", pypi_beta_install_deps = "install_deps", pypi_beta_requirements = "all_requirements") pypi_beta_install_deps() + +load( + "//python/private/pypi:unified_hub_repo.bzl", + "unified_workspace_hub_repo", +) # buildifier: disable=bzl-visibility + +unified_workspace_hub_repo( + name = "pypi", + default_hub = "dev_pip", + hubs = { + "dev_pip": dev_pip_requirements, + "pypi_alpha": pypi_alpha_requirements, + "pypi_beta": pypi_beta_requirements, + "pypiserver": pypiserver_requirements, + }, +) diff --git a/python/private/pypi/BUILD.bazel b/python/private/pypi/BUILD.bazel index 07d2300ce7..09ba5ef135 100644 --- a/python/private/pypi/BUILD.bazel +++ b/python/private/pypi/BUILD.bazel @@ -402,7 +402,10 @@ bzl_library( bzl_library( name = "unified_hub_repo", srcs = ["unified_hub_repo.bzl"], - deps = ["//python/private:text_util"], + deps = [ + "//python/private:normalize_name", + "//python/private:text_util", + ], ) bzl_library( diff --git a/python/private/pypi/unified_hub_repo.bzl b/python/private/pypi/unified_hub_repo.bzl index cbe150575a..f88db75f19 100644 --- a/python/private/pypi/unified_hub_repo.bzl +++ b/python/private/pypi/unified_hub_repo.bzl @@ -1,5 +1,6 @@ """Repository rule for creating the Unified PyPI Hub.""" +load("//python/private:normalize_name.bzl", "normalize_name") load("//python/private:text_util.bzl", "render") _ROOT_BUILD_TMPL = """\ @@ -79,3 +80,32 @@ unified_hub_repo = repository_rule( }, doc = "Private repository rule creating the automatic Unified PyPI Hub.", ) + +def unified_workspace_hub_repo(name, hubs, default_hub = None, extra_aliases = {}): + """Creates a Unified PyPI Hub repository for WORKSPACE mode by loading requirements from hubs. + + Args: + name: Name of the repository rule (e.g. "pypi"). + hubs: Dict mapping hub name to its `all_requirements` list or dict. + e.g. {"dev_pip": dev_pip_requirements, "pypi_alpha": pypi_alpha_requirements} + default_hub: Optional default hub name. + extra_aliases: Dictionary mapping 'package:alias' to a list of hubs that support it. + """ + packages = {} + for hub_name, req_map in hubs.items(): + req_list = req_map.keys() if type(req_map) == type({}) else req_map + for req in req_list: + pkg_name = req.split("//")[-1].split(":")[0] + norm_pkg = normalize_name(pkg_name) + if norm_pkg not in packages: + packages[norm_pkg] = [] + if hub_name not in packages[norm_pkg]: + packages[norm_pkg].append(hub_name) + + unified_hub_repo( + name = name, + default_hub = default_hub, + extra_aliases = extra_aliases, + hubs = sorted(hubs.keys()), + packages = packages, + ) diff --git a/tests/support/pytest_test/BUILD.bazel b/tests/support/pytest_test/BUILD.bazel index 50ad59bf0e..4e6f6dd168 100644 --- a/tests/support/pytest_test/BUILD.bazel +++ b/tests/support/pytest_test/BUILD.bazel @@ -1,5 +1,4 @@ load("@bazel_skylib//:bzl_library.bzl", "bzl_library") -load("//python/private:bzlmod_enabled.bzl", "BZLMOD_ENABLED") # buildifier: disable=bzl-visibility package(default_visibility = ["//:__subpackages__"]) @@ -20,11 +19,11 @@ bzl_library( # These aliases are used to avoid duplicate targets in the deps list alias( name = "default_pytest", - actual = "@pypi//pytest" if BZLMOD_ENABLED else "//python/private:empty", + actual = "@pypi//pytest", ) # These aliases are used to avoid duplicate targets in the deps list alias( name = "default_pytest_bazel", - actual = "@pypi//pytest_bazel" if BZLMOD_ENABLED else "//python/private:empty", + actual = "@pypi//pytest_bazel", ) diff --git a/tests/tools/private/release/BUILD.bazel b/tests/tools/private/release/BUILD.bazel index 317a4626c9..6a4b2f8710 100644 --- a/tests/tools/private/release/BUILD.bazel +++ b/tests/tools/private/release/BUILD.bazel @@ -1,5 +1,5 @@ load("//python:py_library.bzl", "py_library") -load("//tests/support:support.bzl", "SUPPORTS_BZLMOD") +load("//tests/support:support.bzl", "NOT_WINDOWS") load("//tests/support/pytest_test:pytest_test.bzl", "pytest_test") py_library( @@ -8,7 +8,7 @@ py_library( "conftest.py", "release_test_helper.py", ], - target_compatible_with = SUPPORTS_BZLMOD, + target_compatible_with = NOT_WINDOWS, deps = [ "//tools/private/release:mock_gh", "//tools/private/release:release_lib", @@ -19,7 +19,7 @@ py_library( pytest_test( name = "add_backports_test", srcs = ["add_backports_test.py"], - target_compatible_with = SUPPORTS_BZLMOD, + target_compatible_with = NOT_WINDOWS, deps = [ ":release_test_helper", "//tools/private/release:release_lib", @@ -29,7 +29,7 @@ pytest_test( pytest_test( name = "changelog_news_test", srcs = ["changelog_news_test.py"], - target_compatible_with = SUPPORTS_BZLMOD, + target_compatible_with = NOT_WINDOWS, deps = [ ":release_test_helper", "//tools/private/release:release_lib", @@ -39,7 +39,7 @@ pytest_test( pytest_test( name = "complete_sync_changelog_test", srcs = ["complete_sync_changelog_test.py"], - target_compatible_with = SUPPORTS_BZLMOD, + target_compatible_with = NOT_WINDOWS, deps = [ ":release_test_helper", "//tools/private/release:release_lib", @@ -49,7 +49,7 @@ pytest_test( pytest_test( name = "create_release_branch_test", srcs = ["create_release_branch_test.py"], - target_compatible_with = SUPPORTS_BZLMOD, + target_compatible_with = NOT_WINDOWS, deps = [ ":release_test_helper", "//tools/private/release:release_lib", @@ -59,7 +59,7 @@ pytest_test( pytest_test( name = "git_test", srcs = ["git_test.py"], - target_compatible_with = SUPPORTS_BZLMOD, + target_compatible_with = NOT_WINDOWS, deps = [ ":release_test_helper", "//tools/private/release:release_lib", @@ -69,7 +69,7 @@ pytest_test( pytest_test( name = "on_pr_merged_test", srcs = ["on_pr_merged_test.py"], - target_compatible_with = SUPPORTS_BZLMOD, + target_compatible_with = NOT_WINDOWS, deps = [ ":release_test_helper", "//tools/private/release:release_lib", @@ -79,7 +79,7 @@ pytest_test( pytest_test( name = "promote_test", srcs = ["promote_test.py"], - target_compatible_with = SUPPORTS_BZLMOD, + target_compatible_with = NOT_WINDOWS, deps = [ ":release_test_helper", "//tools/private/release:release_lib", @@ -89,7 +89,7 @@ pytest_test( pytest_test( name = "release_issue_test", srcs = ["release_issue_test.py"], - target_compatible_with = SUPPORTS_BZLMOD, + target_compatible_with = NOT_WINDOWS, deps = [ ":release_test_helper", "//tools/private/release:release_lib", @@ -99,7 +99,7 @@ pytest_test( pytest_test( name = "release_test", srcs = ["release_test.py"], - target_compatible_with = SUPPORTS_BZLMOD, + target_compatible_with = NOT_WINDOWS, deps = [ ":release_test_helper", "//tools/private/release:release_lib", @@ -109,7 +109,7 @@ pytest_test( pytest_test( name = "backport_create_releases_test", srcs = ["backport_create_releases_test.py"], - target_compatible_with = SUPPORTS_BZLMOD, + target_compatible_with = NOT_WINDOWS, deps = [ ":release_test_helper", "//tools/private/release:release_lib", @@ -119,7 +119,7 @@ pytest_test( pytest_test( name = "prepare_test", srcs = ["prepare_test.py"], - target_compatible_with = SUPPORTS_BZLMOD, + target_compatible_with = NOT_WINDOWS, deps = [ ":release_test_helper", "//tools/private/release:release_lib", @@ -129,7 +129,7 @@ pytest_test( pytest_test( name = "gh_test", srcs = ["gh_test.py"], - target_compatible_with = SUPPORTS_BZLMOD, + target_compatible_with = NOT_WINDOWS, deps = [ ":release_test_helper", "//tools/private/release:release_lib", @@ -139,7 +139,7 @@ pytest_test( pytest_test( name = "backport_prepare_test", srcs = ["backport_prepare_test.py"], - target_compatible_with = SUPPORTS_BZLMOD, + target_compatible_with = NOT_WINDOWS, deps = [ ":release_test_helper", "//tools/private/release:release_lib", @@ -149,7 +149,7 @@ pytest_test( pytest_test( name = "process_backports_test", srcs = ["process_backports_test.py"], - target_compatible_with = SUPPORTS_BZLMOD, + target_compatible_with = NOT_WINDOWS, deps = [ ":release_test_helper", "//tools/private/release:release_lib", @@ -159,7 +159,7 @@ pytest_test( pytest_test( name = "create_rc_test", srcs = ["create_rc_test.py"], - target_compatible_with = SUPPORTS_BZLMOD, + target_compatible_with = NOT_WINDOWS, deps = [ ":release_test_helper", "//tools/private/release:release_lib", @@ -169,7 +169,7 @@ pytest_test( pytest_test( name = "utils_test", srcs = ["utils_test.py"], - target_compatible_with = SUPPORTS_BZLMOD, + target_compatible_with = NOT_WINDOWS, deps = [ ":release_test_helper", "//tools/private/release:release_lib", From cdb717fbf63e8133a8eeac596d4f08e86de2f375 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Wed, 22 Jul 2026 19:11:06 -0700 Subject: [PATCH 850/922] tests: convert private tool tests to pytest_test and pytest style (#3945) Convert private tool tests (`tests/tools/zipapp/` and `tools/private/update_deps/`) to `pytest_test` and pytest style to be more idiomatic Python tests. Also converts to use pytest fixtures. --- tests/tools/zipapp/BUILD.bazel | 12 +- tests/tools/zipapp/exe_zip_maker_test.py | 87 +-- tests/tools/zipapp/zip_main_maker_test.py | 169 +++-- tests/tools/zipapp/zipper_test.py | 588 ++++++++---------- tools/private/update_deps/BUILD.bazel | 6 +- tools/private/update_deps/update_file_test.py | 112 ++-- 6 files changed, 447 insertions(+), 527 deletions(-) diff --git a/tests/tools/zipapp/BUILD.bazel b/tests/tools/zipapp/BUILD.bazel index b71e9b2589..97c8096ce9 100644 --- a/tests/tools/zipapp/BUILD.bazel +++ b/tests/tools/zipapp/BUILD.bazel @@ -1,19 +1,23 @@ -load("//python:py_test.bzl", "py_test") +load("//tests/support:support.bzl", "SUPPORTS_BZLMOD") +load("//tests/support/pytest_test:pytest_test.bzl", "pytest_test") -py_test( +pytest_test( name = "zipper_test", srcs = ["zipper_test.py"], + target_compatible_with = SUPPORTS_BZLMOD, deps = ["//tools/private/zipapp:zipper_lib"], ) -py_test( +pytest_test( name = "exe_zip_maker_test", srcs = ["exe_zip_maker_test.py"], + target_compatible_with = SUPPORTS_BZLMOD, deps = ["//tools/private/zipapp:exe_zip_maker_lib"], ) -py_test( +pytest_test( name = "zip_main_maker_test", srcs = ["zip_main_maker_test.py"], + target_compatible_with = SUPPORTS_BZLMOD, deps = ["//tools/private/zipapp:zip_main_maker_lib"], ) diff --git a/tests/tools/zipapp/exe_zip_maker_test.py b/tests/tools/zipapp/exe_zip_maker_test.py index 73c509bdbe..97df258e92 100644 --- a/tests/tools/zipapp/exe_zip_maker_test.py +++ b/tests/tools/zipapp/exe_zip_maker_test.py @@ -1,71 +1,46 @@ import hashlib -import pathlib -import shutil import stat -import tempfile -import unittest from tools.private.zipapp import exe_zip_maker -class ExeZipMakerTest(unittest.TestCase): - def setUp(self): - self.test_dir = pathlib.Path(tempfile.mkdtemp()) - self.preamble_path = self.test_dir / "preamble.txt" - self.zip_path = self.test_dir / "data.zip" - self.output_path = self.test_dir / "output.exe" +def test_create_exe_zip(tmp_path): + preamble_path = tmp_path / "preamble.txt" + zip_path = tmp_path / "data.zip" + output_path = tmp_path / "output.exe" - def tearDown(self): - shutil.rmtree(self.test_dir) + # Create dummy zip file + zip_content = b"PK\x03\x04dummyzipcontent" + zip_path.write_bytes(zip_content) - def assertStartsWith(self, actual, expected): - if not actual.startswith(expected): - self.fail(f"{actual!r} does not start with {expected!r}") + # Calculate expected hash + expected_hash = hashlib.sha256(zip_content).hexdigest().encode("utf-8") - def test_create_exe_zip(self): - # Create dummy zip file - zip_content = b"PK\x03\x04dummyzipcontent" - self.zip_path.write_bytes(zip_content) + # Create preamble with placeholder + preamble_text = b"#!/bin/bash\nEXPECTED_HASH='%ZIP_HASH%'\n# ... logic ...\n" + preamble_path.write_bytes(preamble_text) - # Calculate expected hash - expected_hash = hashlib.sha256(zip_content).hexdigest().encode("utf-8") + # Call create_exe_zip directly + exe_zip_maker.create_exe_zip(str(preamble_path), str(zip_path), str(output_path)) - # Create preamble with placeholder - preamble_text = b"#!/bin/bash\nEXPECTED_HASH='%ZIP_HASH%'\n# ... logic ...\n" - self.preamble_path.write_bytes(preamble_text) + # Verify output exists + assert output_path.exists(), f"Output path '{output_path}' should exist" - # Call create_exe_zip directly - exe_zip_maker.create_exe_zip( - str(self.preamble_path), str(self.zip_path), str(self.output_path) - ) + # Verify executable bit + st = output_path.stat() + assert st.st_mode & stat.S_IEXEC, ( + f"Output path '{output_path}' should be executable" + ) - # Verify output exists - self.assertTrue( - self.output_path.exists(), - msg=f"Output path '{self.output_path}' should exist", - ) + # Verify content + content = output_path.read_bytes() - # Verify executable bit - st = self.output_path.stat() - self.assertTrue( - st.st_mode & stat.S_IEXEC, - msg=f"Output path '{self.output_path}' should be executable", - ) + # Split content back into preamble and zip + # We know the preamble text length after substitution. + expected_preamble = preamble_text.replace(b"%ZIP_HASH%", expected_hash) - # Verify content - content = self.output_path.read_bytes() - - # Split content back into preamble and zip - # We know the preamble text length after substitution. - expected_preamble = preamble_text.replace(b"%ZIP_HASH%", expected_hash) - - self.assertStartsWith(content, expected_preamble) - self.assertTrue( - content.endswith(zip_content), - msg="Output content should end with the zip content", - ) - self.assertEqual(len(content), len(expected_preamble) + len(zip_content)) - - -if __name__ == "__main__": - unittest.main() + assert content.startswith(expected_preamble) + assert content.endswith(zip_content), ( + "Output content should end with the zip content" + ) + assert len(content) == len(expected_preamble) + len(zip_content) diff --git a/tests/tools/zipapp/zip_main_maker_test.py b/tests/tools/zipapp/zip_main_maker_test.py index dd8e8e8029..5c7f57c590 100644 --- a/tests/tools/zipapp/zip_main_maker_test.py +++ b/tests/tools/zipapp/zip_main_maker_test.py @@ -1,101 +1,90 @@ import hashlib import os -import tempfile -import unittest -from unittest import mock from tools.private.zipapp import zip_main_maker -class ZipMainMakerTest(unittest.TestCase): - def setUp(self): - self.temp_dir = tempfile.TemporaryDirectory() - self.addCleanup(self.temp_dir.cleanup) - - def test_creates_zip_main(self): - template_path = os.path.join(self.temp_dir.name, "template.py") - with open(template_path, "w", encoding="utf-8") as f: - f.write("hash=%APP_HASH%\nfoo=%FOO%\n") - - output_path = os.path.join(self.temp_dir.name, "output.py") - - file1_path = os.path.join(self.temp_dir.name, "file1.txt") - with open(file1_path, "wb") as f: - f.write(b"content1") - - file2_path = os.path.join(self.temp_dir.name, "file2.txt") - with open(file2_path, "wb") as f: - f.write(b"content2") - - # Add a symlink to test symlink hashing - symlink_path = os.path.join(self.temp_dir.name, "symlink.txt") - os.symlink(file1_path, symlink_path) - - manifest_path = os.path.join(self.temp_dir.name, "manifest.txt") - with open(manifest_path, "w", encoding="utf-8") as f: - f.write(f"rf-file|0|file1.txt|{file1_path}\n") - f.write(f"rf-file|0|file2.txt|{file2_path}\n") - f.write(f"rf-symlink|1|symlink.txt|{symlink_path}\n") - f.write("rf-empty|empty_file.txt\n") - - argv = [ - "zip_main_maker.py", - "--template", - template_path, - "--output", - output_path, - "--substitution", - "%FOO%=bar", - "--hash_files_manifest", - manifest_path, - ] - - with mock.patch("sys.argv", argv): - zip_main_maker.main() - - # Calculate expected hash - h = hashlib.sha256() - line1 = f"rf-file|0|file1.txt|{file1_path}" - line2 = f"rf-file|0|file2.txt|{file2_path}" - line3 = f"rf-symlink|1|symlink.txt|{symlink_path}" - line4 = "rf-empty|empty_file.txt" - - # Sort lines like the program does - lines = sorted([line1, line2, line3, line4]) - for line in lines: - parts = line.split("|") - if len(parts) > 1: - _, rest = line.split("|", 1) - h.update(rest.encode("utf-8")) - else: - h.update(line.encode("utf-8")) - - type_ = parts[0] - if type_ == "rf-empty": +def test_creates_zip_main(tmp_path, monkeypatch): + temp_dir = str(tmp_path) + template_path = os.path.join(temp_dir, "template.py") + with open(template_path, "w", encoding="utf-8") as f: + f.write("hash=%APP_HASH%\nfoo=%FOO%\n") + + output_path = os.path.join(temp_dir, "output.py") + + file1_path = os.path.join(temp_dir, "file1.txt") + with open(file1_path, "wb") as f: + f.write(b"content1") + + file2_path = os.path.join(temp_dir, "file2.txt") + with open(file2_path, "wb") as f: + f.write(b"content2") + + # Add a symlink to test symlink hashing + symlink_path = os.path.join(temp_dir, "symlink.txt") + os.symlink(file1_path, symlink_path) + + manifest_path = os.path.join(temp_dir, "manifest.txt") + with open(manifest_path, "w", encoding="utf-8") as f: + f.write(f"rf-file|0|file1.txt|{file1_path}\n") + f.write(f"rf-file|0|file2.txt|{file2_path}\n") + f.write(f"rf-symlink|1|symlink.txt|{symlink_path}\n") + f.write("rf-empty|empty_file.txt\n") + + argv = [ + "zip_main_maker.py", + "--template", + template_path, + "--output", + output_path, + "--substitution", + "%FOO%=bar", + "--hash_files_manifest", + manifest_path, + ] + + monkeypatch.setattr("sys.argv", argv) + zip_main_maker.main() + + # Calculate expected hash + h = hashlib.sha256() + line1 = f"rf-file|0|file1.txt|{file1_path}" + line2 = f"rf-file|0|file2.txt|{file2_path}" + line3 = f"rf-symlink|1|symlink.txt|{symlink_path}" + line4 = "rf-empty|empty_file.txt" + + # Sort lines like the program does + lines = sorted([line1, line2, line3, line4]) + for line in lines: + parts = line.split("|") + if len(parts) > 1: + _, rest = line.split("|", 1) + h.update(rest.encode("utf-8")) + else: + h.update(line.encode("utf-8")) + + type_ = parts[0] + if type_ == "rf-empty": + continue + if len(parts) >= 4: + is_symlink_str = parts[1] + path = parts[-1] + if not path: continue - if len(parts) >= 4: - is_symlink_str = parts[1] - path = parts[-1] - if not path: - continue - if is_symlink_str == "-1": - is_symlink = not os.path.exists(path) - else: - is_symlink = is_symlink_str == "1" - - if is_symlink: - h.update(os.readlink(path).encode("utf-8")) - else: - with open(path, "rb") as f: - h.update(f.read()) - - expected_hash = h.hexdigest() + if is_symlink_str == "-1": + is_symlink = not os.path.exists(path) + else: + is_symlink = is_symlink_str == "1" - with open(output_path, "r", encoding="utf-8") as f: - content = f.read() + if is_symlink: + h.update(os.readlink(path).encode("utf-8")) + else: + with open(path, "rb") as f: + h.update(f.read()) - self.assertEqual(content, f"hash={expected_hash}\nfoo=bar\n") + expected_hash = h.hexdigest() + with open(output_path, "r", encoding="utf-8") as f: + content = f.read() -if __name__ == "__main__": - unittest.main() + assert content == f"hash={expected_hash}\nfoo=bar\n" diff --git a/tests/tools/zipapp/zipper_test.py b/tests/tools/zipapp/zipper_test.py index ac70917c30..bbd85b6536 100644 --- a/tests/tools/zipapp/zipper_test.py +++ b/tests/tools/zipapp/zipper_test.py @@ -1,8 +1,5 @@ import os -import pathlib import shutil -import tempfile -import unittest import zipfile from tools.private.zipapp import zipper @@ -12,333 +9,294 @@ def symlink_target_path(p): return p.replace("/", os.sep) -class ZipperTest(unittest.TestCase): - def setUp(self): - self.test_dir = pathlib.Path(tempfile.mkdtemp()) - self.manifest_path = self.test_dir / "manifest.txt" - self.output_zip = self.test_dir / "output.zip" - - def tearDown(self): - shutil.rmtree(self.test_dir) - - def _create_zip(self, **kwargs): - defaults = { - "manifest_path": self.manifest_path, - "output_zip": self.output_zip, - "compress_level": 0, - "workspace_name": "my_ws", - "legacy_external_runfiles": False, - "runfiles_dir": "runfiles", - # We need to generate paths for the platform we're running on. - "platform_pathsep": os.sep, +def is_symlink(zip_info): + # Check upper 4 bits of external_attr for S_IFLNK + # S_IFLNK is 0o120000 = 0xA000 + attr = zip_info.external_attr >> 16 + return (attr & 0xF000) == 0xA000 + + +def assert_zip_file_content(zf, path, content=None, is_symlink_file=False, target=None): + info = zf.getinfo(path) + if is_symlink_file: + assert is_symlink(info), f"{path} should be a symlink but is not" + assert zf.read(path).decode() == target + else: + assert not is_symlink(info), f"{path} should NOT be a symlink but is" + assert zf.read(path).decode() == content + + +def create_zip(manifest_path, output_zip, **kwargs): + defaults = { + "manifest_path": manifest_path, + "output_zip": output_zip, + "compress_level": 0, + "workspace_name": "my_ws", + "legacy_external_runfiles": False, + "runfiles_dir": "runfiles", + "platform_pathsep": os.sep, + } + defaults.update(kwargs) + zipper.create_zip(**defaults) + + +def extract_zip(zip_path, extract_dir): + # Manually extract to preserve symlinks + with zipfile.ZipFile(zip_path, "r") as zf: + for info in zf.infolist(): + extract_path = extract_dir / info.filename + extract_path.parent.mkdir(parents=True, exist_ok=True) + if is_symlink(info): + target = zf.read(info).decode() + os.symlink(target, extract_path) + else: + with zf.open(info) as src, open(extract_path, "wb") as dst: + shutil.copyfileobj(src, dst) + + +def test_create_zip_with_files_and_symlinks(tmp_path): + manifest_path = tmp_path / "manifest.txt" + output_zip = tmp_path / "output.zip" + + file1_path = tmp_path / "file1.txt" + file1_path.write_text("content1") + + link_target_path = "target.txt" + symlink_path = tmp_path / "symlink_source" + symlink_path.symlink_to(link_target_path) + + manifest_content = [ + f"regular|0|file1.txt|{file1_path}", + f"rf-file|0|foo/bar.txt|{file1_path}", + f"rf-symlink|1|link1|{symlink_path}", + f"rf-root-symlink|0|root_file|{file1_path}", + "rf-empty|empty_file", + ] + manifest_path.write_text("\n".join(manifest_content)) + + create_zip(manifest_path, output_zip) + + assert output_zip.exists() + + with zipfile.ZipFile(output_zip, "r") as zf: + assert set(zf.namelist()) == { + "file1.txt", + "runfiles/my_ws/foo/bar.txt", + "runfiles/my_ws/link1", + "runfiles/root_file", + "runfiles/my_ws/empty_file", } - defaults.update(kwargs) - zipper.create_zip(**defaults) - - def assertZipFileContent( - self, zf, path, content=None, is_symlink=False, target=None - ): - info = zf.getinfo(path) - if is_symlink: - self.assertTrue( - self.is_symlink(info), - f"{path} should be a symlink but is not", - ) - self.assertEqual(zf.read(path).decode(), target) - else: - self.assertFalse( - self.is_symlink(info), - f"{path} should NOT be a symlink but is", - ) - self.assertEqual(zf.read(path).decode(), content) - - def test_create_zip_with_files_and_symlinks(self): - file1_path = self.test_dir / "file1.txt" - file1_path.write_text("content1") - - link_target_path = "target.txt" # Relative target - symlink_path = self.test_dir / "symlink_source" - symlink_path.symlink_to(link_target_path) - - manifest_content = [ - f"regular|0|file1.txt|{file1_path}", - f"rf-file|0|foo/bar.txt|{file1_path}", - f"rf-symlink|1|link1|{symlink_path}", # Should read target 'target.txt' - f"rf-root-symlink|0|root_file|{file1_path}", - "rf-empty|empty_file", - ] - self.manifest_path.write_text("\n".join(manifest_content)) - - self._create_zip() - - self.assertTrue(self.output_zip.exists()) - - with zipfile.ZipFile(self.output_zip, "r") as zf: - self.assertEqual( - set(zf.namelist()), - { - "file1.txt", - "runfiles/my_ws/foo/bar.txt", - "runfiles/my_ws/link1", - "runfiles/root_file", - "runfiles/my_ws/empty_file", - }, - ) - - self.assertZipFileContent(zf, "file1.txt", content="content1") - self.assertZipFileContent( - zf, "runfiles/my_ws/foo/bar.txt", content="content1" - ) - self.assertZipFileContent( - zf, "runfiles/my_ws/link1", is_symlink=True, target="target.txt" - ) - self.assertZipFileContent(zf, "runfiles/root_file", content="content1") - self.assertZipFileContent(zf, "runfiles/my_ws/empty_file", content="") - - def test_create_zip_with_direct_symlink(self): - # Test the 'symlink' manifest entry type - manifest_content = [ - "symlink|path/to/link|target/path", - ] - self.manifest_path.write_text("\n".join(manifest_content)) - - self._create_zip() - - with zipfile.ZipFile(self.output_zip, "r") as zf: - self.assertEqual(zf.namelist(), ["runfiles/path/to/link"]) - self.assertZipFileContent( - zf, - "runfiles/path/to/link", - is_symlink=True, - target=symlink_target_path("../../target/path"), - ) - - def test_pathsep_normalization(self): - # Test that pathsep="\\" normalizes paths - file1_path = self.test_dir / "file1.txt" - file1_path.write_text("content1") - - manifest_content = [ - f"regular|0|dir/file.txt|{file1_path}", - "symlink|link/path|target/path", - ] - self.manifest_path.write_text("\n".join(manifest_content)) - - # Use backslash as platform_pathsep - self._create_zip(platform_pathsep="\\") - - with zipfile.ZipFile(self.output_zip, "r") as zf: - # zipfile.namelist() always returns with forward slashes - # But the content of the symlink should be normalized if it was passed through path_norm - self.assertEqual( - set(zf.namelist()), - {"dir/file.txt", "runfiles/link/path"}, - ) - # The target of the symlink should have backslashes - self.assertZipFileContent( - zf, - "runfiles/link/path", - is_symlink=True, - target="..\\target\\path", - ) - - def test_symlink_precedence(self): - # Test that 'symlink' entries take precedence over others for the same path - file1_path = self.test_dir / "file1.txt" - file1_path.write_text("content1") - - manifest_content = [ - # Same zip path: runfiles/my_ws/path/to/file - f"rf-file|0|path/to/file|{file1_path}", - "symlink|my_ws/path/to/file|symlink/target", - ] - self.manifest_path.write_text("\n".join(manifest_content)) - - self._create_zip() - - with zipfile.ZipFile(self.output_zip, "r") as zf: - self.assertEqual(zf.namelist(), ["runfiles/my_ws/path/to/file"]) - # It should be the symlink, not the file - self.assertZipFileContent( - zf, - "runfiles/my_ws/path/to/file", - is_symlink=True, - target=symlink_target_path("../../../symlink/target"), - ) - - def test_timestamps_are_deterministic(self): - # Create a content file with a specific recent timestamp - file1_path = self.test_dir / "file1.txt" - file1_path.write_text("content1") - - # Set mtime to something recent (e.g. now) - os.utime(file1_path, None) - - manifest_content = [ - f"regular|0|file1.txt|{file1_path}", - ] - self.manifest_path.write_text("\n".join(manifest_content)) + assert_zip_file_content(zf, "file1.txt", content="content1") + assert_zip_file_content(zf, "runfiles/my_ws/foo/bar.txt", content="content1") + assert_zip_file_content( + zf, "runfiles/my_ws/link1", is_symlink_file=True, target="target.txt" + ) + assert_zip_file_content(zf, "runfiles/root_file", content="content1") + assert_zip_file_content(zf, "runfiles/my_ws/empty_file", content="") - self._create_zip() - with zipfile.ZipFile(self.output_zip, "r") as zf: - info = zf.getinfo("file1.txt") - # DOS epoch is 1980-01-01 00:00:00 - expected_date_time = (1980, 1, 1, 0, 0, 0) - self.assertEqual(info.date_time, expected_date_time) +def test_create_zip_with_direct_symlink(tmp_path): + manifest_path = tmp_path / "manifest.txt" + output_zip = tmp_path / "output.zip" - def test_runfiles_mapping_with_cross_repo_paths(self): - # Create content file - file1_path = self.test_dir / "file1.txt" - file1_path.write_text("content1") + manifest_content = ["symlink|path/to/link|target/path"] + manifest_path.write_text("\n".join(manifest_content)) - manifest_content = [ - f"rf-file|0|../other_repo/foo.txt|{file1_path}", - "rf-empty|../other_repo/empty_file", - ] + create_zip(manifest_path, output_zip) - self.manifest_path.write_text("\n".join(manifest_content)) - - self._create_zip(workspace_name="my_ws") - - with zipfile.ZipFile(self.output_zip, "r") as zf: - self.assertEqual( - set(zf.namelist()), - { - "runfiles/other_repo/foo.txt", - "runfiles/other_repo/empty_file", - }, - ) - self.assertZipFileContent( - zf, "runfiles/other_repo/foo.txt", content="content1" - ) - self.assertZipFileContent(zf, "runfiles/other_repo/empty_file", content="") - - def test_runfiles_mapping_with_legacy_external_paths(self): - file1_path = self.test_dir / "file1.txt" - file1_path.write_text("content1") - - manifest_content = [ - f"rf-file|0|external/other_repo/foo.txt|{file1_path}", - "rf-empty|external/other_repo/empty_file", - ] + with zipfile.ZipFile(output_zip, "r") as zf: + assert zf.namelist() == ["runfiles/path/to/link"] + assert_zip_file_content( + zf, + "runfiles/path/to/link", + is_symlink_file=True, + target=symlink_target_path("../../target/path"), + ) - self.manifest_path.write_text("\n".join(manifest_content)) - - self._create_zip(workspace_name="my_ws", legacy_external_runfiles=True) - - with zipfile.ZipFile(self.output_zip, "r") as zf: - self.assertEqual( - set(zf.namelist()), - { - "runfiles/other_repo/foo.txt", - "runfiles/other_repo/empty_file", - }, - ) - self.assertZipFileContent( - zf, "runfiles/other_repo/foo.txt", content="content1" - ) - self.assertZipFileContent(zf, "runfiles/other_repo/empty_file", content="") - - def test_output_deterministic(self): - # Create files - file1 = self.test_dir / "file1" - file1.write_text("1") - file2 = self.test_dir / "file2" - file2.write_text("2") - file3 = self.test_dir / "file3" - file3.write_text("3") - - # Manifest entries mixed up - # We want the final order to be: - # 1. a/regular (regular) - # 2. runfiles/a_root_link (rf-root-symlink) - # 3. runfiles/my_ws/b_rf_file (rf-file) - # 4. runfiles/my_ws/c_rf_link (rf-symlink) - # 5. runfiles/my_ws/d_rf_empty (rf-empty) - # 6. z/regular (regular) - - manifest_content = [ - f"regular|0|z/regular|{file1}", - f"rf-file|0|b_rf_file|{file2}", # -> runfiles/my_ws/b_rf_file - f"rf-root-symlink|0|a_root_link|{file3}", # -> runfiles/a_root_link - f"regular|0|a/regular|{file3}", - "rf-empty|d_rf_empty", # -> runfiles/my_ws/d_rf_empty - f"rf-symlink|0|c_rf_link|{file3}", # -> runfiles/my_ws/c_rf_link - ] - self.manifest_path.write_text("\n".join(manifest_content)) - - self._create_zip(workspace_name="my_ws") - - with zipfile.ZipFile(self.output_zip, "r") as zf: - self.assertEqual( - zf.namelist(), - [ - "a/regular", - "runfiles/a_root_link", - "runfiles/my_ws/b_rf_file", - "runfiles/my_ws/c_rf_link", - "runfiles/my_ws/d_rf_empty", - "z/regular", - ], - ) - - def _extract_zip(self, zip_path, extract_dir): - # Manually extract to preserve symlinks - with zipfile.ZipFile(zip_path, "r") as zf: - for info in zf.infolist(): - extract_path = extract_dir / info.filename - extract_path.parent.mkdir(parents=True, exist_ok=True) - if self.is_symlink(info): - target = zf.read(info).decode() - # On Windows, relative symlinks must use backslashes to be readable - os.symlink(target, extract_path) - else: - with zf.open(info) as src, open(extract_path, "wb") as dst: - shutil.copyfileobj(src, dst) - - def test_symlink_extraction(self): - # Test that 'symlink' entries extract correctly as relative symlinks - # Create a file that the symlink will point to - target_file = self.test_dir / "target_file.txt" - target_file.write_text("target content") - - manifest_content = [ - f"rf-file|0|target/path|{target_file}", - "symlink|my_ws/path/to/link|my_ws/target/path", - f"rf-file|0|same_dir_target|{target_file}", - "symlink|my_ws/same_dir_link|my_ws/same_dir_target", - ] - self.manifest_path.write_text("\n".join(manifest_content)) +def test_pathsep_normalization(tmp_path): + manifest_path = tmp_path / "manifest.txt" + output_zip = tmp_path / "output.zip" - self._create_zip(workspace_name="my_ws") + file1_path = tmp_path / "file1.txt" + file1_path.write_text("content1") - extract_dir = self.test_dir / "extract" - extract_dir.mkdir() + manifest_content = [ + f"regular|0|dir/file.txt|{file1_path}", + "symlink|link/path|target/path", + ] + manifest_path.write_text("\n".join(manifest_content)) - self._extract_zip(self.output_zip, extract_dir) + create_zip(manifest_path, output_zip, platform_pathsep="\\") - link_path = extract_dir / "runfiles/my_ws/path/to/link" - self.assertTrue(link_path.is_symlink(), f"{link_path} should be a symlink") - self.assertEqual( - os.readlink(link_path), "../../target/path".replace("/", os.path.sep) + with zipfile.ZipFile(output_zip, "r") as zf: + assert set(zf.namelist()) == {"dir/file.txt", "runfiles/link/path"} + assert_zip_file_content( + zf, + "runfiles/link/path", + is_symlink_file=True, + target="..\\target\\path", ) - self.assertEqual(link_path.read_text(), "target content") - link2_path = extract_dir / "runfiles/my_ws/same_dir_link" - self.assertTrue(link2_path.is_symlink(), f"{link2_path} should be a symlink") - # Relative path from runfiles/my_ws/ to runfiles/my_ws/same_dir_target is just same_dir_target - self.assertEqual(os.readlink(link2_path), "same_dir_target") - self.assertEqual(link2_path.read_text(), "target content") - def is_symlink(self, zip_info): - # Check upper 4 bits of external_attr for S_IFLNK - # S_IFLNK is 0o120000 = 0xA000 - attr = zip_info.external_attr >> 16 - return (attr & 0xF000) == 0xA000 +def test_symlink_precedence(tmp_path): + manifest_path = tmp_path / "manifest.txt" + output_zip = tmp_path / "output.zip" + + file1_path = tmp_path / "file1.txt" + file1_path.write_text("content1") + + manifest_content = [ + f"rf-file|0|path/to/file|{file1_path}", + "symlink|my_ws/path/to/file|symlink/target", + ] + manifest_path.write_text("\n".join(manifest_content)) + + create_zip(manifest_path, output_zip) + + with zipfile.ZipFile(output_zip, "r") as zf: + assert zf.namelist() == ["runfiles/my_ws/path/to/file"] + assert_zip_file_content( + zf, + "runfiles/my_ws/path/to/file", + is_symlink_file=True, + target=symlink_target_path("../../../symlink/target"), + ) + + +def test_timestamps_are_deterministic(tmp_path): + manifest_path = tmp_path / "manifest.txt" + output_zip = tmp_path / "output.zip" + + file1_path = tmp_path / "file1.txt" + file1_path.write_text("content1") + os.utime(file1_path, None) + + manifest_content = [f"regular|0|file1.txt|{file1_path}"] + manifest_path.write_text("\n".join(manifest_content)) + + create_zip(manifest_path, output_zip) + + with zipfile.ZipFile(output_zip, "r") as zf: + info = zf.getinfo("file1.txt") + expected_date_time = (1980, 1, 1, 0, 0, 0) + assert info.date_time == expected_date_time + + +def test_runfiles_mapping_with_cross_repo_paths(tmp_path): + manifest_path = tmp_path / "manifest.txt" + output_zip = tmp_path / "output.zip" + + file1_path = tmp_path / "file1.txt" + file1_path.write_text("content1") + + manifest_content = [ + f"rf-file|0|../other_repo/foo.txt|{file1_path}", + "rf-empty|../other_repo/empty_file", + ] + manifest_path.write_text("\n".join(manifest_content)) + + create_zip(manifest_path, output_zip, workspace_name="my_ws") + + with zipfile.ZipFile(output_zip, "r") as zf: + assert set(zf.namelist()) == { + "runfiles/other_repo/foo.txt", + "runfiles/other_repo/empty_file", + } + assert_zip_file_content(zf, "runfiles/other_repo/foo.txt", content="content1") + assert_zip_file_content(zf, "runfiles/other_repo/empty_file", content="") + + +def test_runfiles_mapping_with_legacy_external_paths(tmp_path): + manifest_path = tmp_path / "manifest.txt" + output_zip = tmp_path / "output.zip" + + file1_path = tmp_path / "file1.txt" + file1_path.write_text("content1") + + manifest_content = [ + f"rf-file|0|external/other_repo/foo.txt|{file1_path}", + "rf-empty|external/other_repo/empty_file", + ] + manifest_path.write_text("\n".join(manifest_content)) + + create_zip( + manifest_path, output_zip, workspace_name="my_ws", legacy_external_runfiles=True + ) + + with zipfile.ZipFile(output_zip, "r") as zf: + assert set(zf.namelist()) == { + "runfiles/other_repo/foo.txt", + "runfiles/other_repo/empty_file", + } + assert_zip_file_content(zf, "runfiles/other_repo/foo.txt", content="content1") + assert_zip_file_content(zf, "runfiles/other_repo/empty_file", content="") + + +def test_output_deterministic(tmp_path): + manifest_path = tmp_path / "manifest.txt" + output_zip = tmp_path / "output.zip" + + file1 = tmp_path / "file1" + file1.write_text("1") + file2 = tmp_path / "file2" + file2.write_text("2") + file3 = tmp_path / "file3" + file3.write_text("3") + + manifest_content = [ + f"regular|0|z/regular|{file1}", + f"rf-file|0|b_rf_file|{file2}", + f"rf-root-symlink|0|a_root_link|{file3}", + f"regular|0|a/regular|{file3}", + "rf-empty|d_rf_empty", + f"rf-symlink|0|c_rf_link|{file3}", + ] + + manifest_path.write_text("\n".join(manifest_content)) + + create_zip(manifest_path, output_zip, workspace_name="my_ws") + + with zipfile.ZipFile(output_zip, "r") as zf: + assert zf.namelist() == [ + "a/regular", + "runfiles/a_root_link", + "runfiles/my_ws/b_rf_file", + "runfiles/my_ws/c_rf_link", + "runfiles/my_ws/d_rf_empty", + "z/regular", + ] + + +def test_symlink_extraction(tmp_path): + manifest_path = tmp_path / "manifest.txt" + output_zip = tmp_path / "output.zip" + + target_file = tmp_path / "target_file.txt" + target_file.write_text("target content") + + manifest_content = [ + f"rf-file|0|target/path|{target_file}", + "symlink|my_ws/path/to/link|my_ws/target/path", + f"rf-file|0|same_dir_target|{target_file}", + "symlink|my_ws/same_dir_link|my_ws/same_dir_target", + ] + manifest_path.write_text("\n".join(manifest_content)) + + create_zip(manifest_path, output_zip, workspace_name="my_ws") + + extract_dir = tmp_path / "extract" + extract_dir.mkdir() + + extract_zip(output_zip, extract_dir) + link_path = extract_dir / "runfiles/my_ws/path/to/link" + assert link_path.is_symlink(), f"{link_path} should be a symlink" + assert os.readlink(link_path) == "../../target/path".replace("/", os.path.sep) + assert link_path.read_text() == "target content" -if __name__ == "__main__": - unittest.main() + link2_path = extract_dir / "runfiles/my_ws/same_dir_link" + assert link2_path.is_symlink(), f"{link2_path} should be a symlink" + assert os.readlink(link2_path) == "same_dir_target" + assert link2_path.read_text() == "target content" diff --git a/tools/private/update_deps/BUILD.bazel b/tools/private/update_deps/BUILD.bazel index c3f94c9814..8746d82c17 100644 --- a/tools/private/update_deps/BUILD.bazel +++ b/tools/private/update_deps/BUILD.bazel @@ -13,7 +13,8 @@ # limitations under the License. load("//python:py_binary.bzl", "py_binary") load("//python:py_library.bzl", "py_library") -load("//python:py_test.bzl", "py_test") +load("//tests/support:support.bzl", "SUPPORTS_BZLMOD") +load("//tests/support/pytest_test:pytest_test.bzl", "pytest_test") licenses(["notice"]) @@ -65,10 +66,11 @@ py_binary( ], ) -py_test( +pytest_test( name = "update_file_test", srcs = ["update_file_test.py"], imports = ["../../.."], + target_compatible_with = SUPPORTS_BZLMOD, deps = [ ":update_file", ], diff --git a/tools/private/update_deps/update_file_test.py b/tools/private/update_deps/update_file_test.py index a3cb1c0a6d..b1d6545ead 100644 --- a/tools/private/update_deps/update_file_test.py +++ b/tools/private/update_deps/update_file_test.py @@ -12,14 +12,13 @@ # See the License for the specific language governing permissions and # limitations under the License. -import unittest +import pytest from tools.private.update_deps.update_file import replace_snippet, unified_diff -class TestReplaceSnippet(unittest.TestCase): - def test_replace_simple(self): - current = """\ +def test_replace_simple(): + current = """\ Before the snippet # Start marker @@ -30,15 +29,14 @@ def test_replace_simple(self): After the snippet """ - snippet = "Replaced" # noqa: F841 - got = replace_snippet( - current=current, - snippet="Replaced", - start_marker="# Start marker", - end_marker="# End marker", - ) - - want = """\ + got = replace_snippet( + current=current, + snippet="Replaced", + start_marker="# Start marker", + end_marker="# End marker", + ) + + want = """\ Before the snippet # Start marker @@ -47,10 +45,11 @@ def test_replace_simple(self): After the snippet """ - self.assertEqual(want, got) + assert got == want - def test_replace_indented(self): - current = """\ + +def test_replace_indented(): + current = """\ Before the snippet # Start marker @@ -59,14 +58,14 @@ def test_replace_indented(self): After the snippet """ - got = replace_snippet( - current=current, - snippet=" Replaced", - start_marker="# Start marker", - end_marker="# End marker", - ) - - want = """\ + got = replace_snippet( + current=current, + snippet=" Replaced", + start_marker="# Start marker", + end_marker="# End marker", + ) + + want = """\ Before the snippet # Start marker @@ -75,45 +74,42 @@ def test_replace_indented(self): After the snippet """ - self.assertEqual(want, got) - - def test_raises_if_start_is_not_found(self): - with self.assertRaises(RuntimeError) as exc: - replace_snippet( - current="foo", - snippet="", - start_marker="start", - end_marker="end", - ) - - self.assertEqual(exc.exception.args[0], "Start marker 'start' was not found") - - def test_raises_if_end_is_not_found(self): - with self.assertRaises(RuntimeError) as exc: - replace_snippet( - current="start", - snippet="", - start_marker="start", - end_marker="end", - ) - - self.assertEqual(exc.exception.args[0], "End marker 'end' was not found") - - -class TestUnifiedDiff(unittest.TestCase): - def test_diff(self): - give_a = """\ + assert got == want + + +def test_raises_if_start_is_not_found(): + with pytest.raises(RuntimeError, match="Start marker 'start' was not found"): + replace_snippet( + current="foo", + snippet="", + start_marker="start", + end_marker="end", + ) + + +def test_raises_if_end_is_not_found(): + with pytest.raises(RuntimeError, match="End marker 'end' was not found"): + replace_snippet( + current="start", + snippet="", + start_marker="start", + end_marker="end", + ) + + +def test_diff(): + give_a = """\ First line second line Third line """ - give_b = """\ + give_b = """\ First line Second line Third line """ - got = unified_diff("filename", give_a, give_b) - want = """\ + got = unified_diff("filename", give_a, give_b) + want = """\ --- a/filename +++ b/filename @@ -1,3 +1,3 @@ @@ -121,8 +117,4 @@ def test_diff(self): -second line +Second line Third line""" - self.assertEqual(want, got) - - -if __name__ == "__main__": - unittest.main() + assert got == want From 4f231ac5baf7949a7477c86e91550471ebc8d5b6 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Thu, 23 Jul 2026 01:40:22 -0700 Subject: [PATCH 851/922] refactor(tests): create conftest py_library target (#3947) Refactor test/tools/private/release tests such that conftest.py is its own py_library target, and add it as a dependency to targets that directly need it. --- tests/tools/private/release/BUILD.bazel | 34 +++++++++++++++++++++---- 1 file changed, 29 insertions(+), 5 deletions(-) diff --git a/tests/tools/private/release/BUILD.bazel b/tests/tools/private/release/BUILD.bazel index 6a4b2f8710..b666cf8b54 100644 --- a/tests/tools/private/release/BUILD.bazel +++ b/tests/tools/private/release/BUILD.bazel @@ -3,16 +3,24 @@ load("//tests/support:support.bzl", "NOT_WINDOWS") load("//tests/support/pytest_test:pytest_test.bzl", "pytest_test") py_library( - name = "release_test_helper", - srcs = [ - "conftest.py", - "release_test_helper.py", + name = "conftest", + testonly = True, + srcs = ["conftest.py"], + target_compatible_with = NOT_WINDOWS, + deps = [ + ":release_test_helper", + "//tools/private/release:release_lib", + "@pypi//pytest_mock", ], +) + +py_library( + name = "release_test_helper", + srcs = ["release_test_helper.py"], target_compatible_with = NOT_WINDOWS, deps = [ "//tools/private/release:mock_gh", "//tools/private/release:release_lib", - "@pypi//pytest_mock", ], ) @@ -21,6 +29,7 @@ pytest_test( srcs = ["add_backports_test.py"], target_compatible_with = NOT_WINDOWS, deps = [ + ":conftest", ":release_test_helper", "//tools/private/release:release_lib", ], @@ -31,6 +40,7 @@ pytest_test( srcs = ["changelog_news_test.py"], target_compatible_with = NOT_WINDOWS, deps = [ + ":conftest", ":release_test_helper", "//tools/private/release:release_lib", ], @@ -41,6 +51,7 @@ pytest_test( srcs = ["complete_sync_changelog_test.py"], target_compatible_with = NOT_WINDOWS, deps = [ + ":conftest", ":release_test_helper", "//tools/private/release:release_lib", ], @@ -51,6 +62,7 @@ pytest_test( srcs = ["create_release_branch_test.py"], target_compatible_with = NOT_WINDOWS, deps = [ + ":conftest", ":release_test_helper", "//tools/private/release:release_lib", ], @@ -61,6 +73,7 @@ pytest_test( srcs = ["git_test.py"], target_compatible_with = NOT_WINDOWS, deps = [ + ":conftest", ":release_test_helper", "//tools/private/release:release_lib", ], @@ -71,6 +84,7 @@ pytest_test( srcs = ["on_pr_merged_test.py"], target_compatible_with = NOT_WINDOWS, deps = [ + ":conftest", ":release_test_helper", "//tools/private/release:release_lib", ], @@ -81,6 +95,7 @@ pytest_test( srcs = ["promote_test.py"], target_compatible_with = NOT_WINDOWS, deps = [ + ":conftest", ":release_test_helper", "//tools/private/release:release_lib", ], @@ -91,6 +106,7 @@ pytest_test( srcs = ["release_issue_test.py"], target_compatible_with = NOT_WINDOWS, deps = [ + ":conftest", ":release_test_helper", "//tools/private/release:release_lib", ], @@ -101,6 +117,7 @@ pytest_test( srcs = ["release_test.py"], target_compatible_with = NOT_WINDOWS, deps = [ + ":conftest", ":release_test_helper", "//tools/private/release:release_lib", ], @@ -111,6 +128,7 @@ pytest_test( srcs = ["backport_create_releases_test.py"], target_compatible_with = NOT_WINDOWS, deps = [ + ":conftest", ":release_test_helper", "//tools/private/release:release_lib", ], @@ -121,6 +139,7 @@ pytest_test( srcs = ["prepare_test.py"], target_compatible_with = NOT_WINDOWS, deps = [ + ":conftest", ":release_test_helper", "//tools/private/release:release_lib", ], @@ -131,6 +150,7 @@ pytest_test( srcs = ["gh_test.py"], target_compatible_with = NOT_WINDOWS, deps = [ + ":conftest", ":release_test_helper", "//tools/private/release:release_lib", ], @@ -141,6 +161,7 @@ pytest_test( srcs = ["backport_prepare_test.py"], target_compatible_with = NOT_WINDOWS, deps = [ + ":conftest", ":release_test_helper", "//tools/private/release:release_lib", ], @@ -151,6 +172,7 @@ pytest_test( srcs = ["process_backports_test.py"], target_compatible_with = NOT_WINDOWS, deps = [ + ":conftest", ":release_test_helper", "//tools/private/release:release_lib", ], @@ -161,6 +183,7 @@ pytest_test( srcs = ["create_rc_test.py"], target_compatible_with = NOT_WINDOWS, deps = [ + ":conftest", ":release_test_helper", "//tools/private/release:release_lib", ], @@ -171,6 +194,7 @@ pytest_test( srcs = ["utils_test.py"], target_compatible_with = NOT_WINDOWS, deps = [ + ":conftest", ":release_test_helper", "//tools/private/release:release_lib", ], From c6b667177fe907db14d976b8e3631801bf55efd3 Mon Sep 17 00:00:00 2001 From: Ignas Anikevicius <240938+aignas@users.noreply.github.com> Date: Sat, 25 Jul 2026 03:45:08 +0900 Subject: [PATCH 852/922] refactor(pypi): make whl_library a macro and split impl into whl_archive and pip_archive (#3948) Before this PR the `whl_library` would be a do-all repository rule. Whilst it is convenient to reuse the code, it is actually really difficult to maintain and make it more performant. Side effect here is that the python dependencies (like `setuptools`, etc) will no longer be downloaded for whl-only extracts, it makes it a tiny bit faster. With this split we can drop certain dependencies from the whl extraction and optimize the common path - whl extraction where the URL for downloading the wheel is known. This also allows us to start handling the sdists in an entirely different way. In a followup PR I plan to split the part which just extracts the wheel to lay a more surgical foundation to #3856. Foundation work for #2410. Split out of #3856. Work towards #2948. --------- Co-authored-by: Richard Levasseur --- python/private/pypi/attrs.bzl | 73 ----- python/private/pypi/pip_repository_attrs.bzl | 73 +++++ python/private/pypi/whl_library.bzl | 324 ++++++++++++------- 3 files changed, 284 insertions(+), 186 deletions(-) diff --git a/python/private/pypi/attrs.bzl b/python/private/pypi/attrs.bzl index 57bd93f40a..271c17e35c 100644 --- a/python/private/pypi/attrs.bzl +++ b/python/private/pypi/attrs.bzl @@ -64,79 +64,6 @@ here do not cause packages to be re-fetched. Don't fetch different things based on the value of these variables. """, ), - "experimental_requirement_cycles": attr.string_list_dict( - default = {}, - doc = """\ -A mapping of dependency cycle names to a list of requirements which form that cycle. - -Requirements which form cycles will be installed together and taken as -dependencies together in order to ensure that the cycle is always satisified. - -Example: - `sphinx` depends on `sphinxcontrib-serializinghtml` - When listing both as requirements, ala - - ``` - py_binary( - name = "doctool", - ... - deps = [ - "@pypi//sphinx:pkg", - "@pypi//sphinxcontrib_serializinghtml", - ] - ) - ``` - - Will produce a Bazel error such as - - ``` - ERROR: .../external/pypi_sphinxcontrib_serializinghtml/BUILD.bazel:44:6: in alias rule @pypi_sphinxcontrib_serializinghtml//:pkg: cycle in dependency graph: - //:doctool (...) - @pypi//sphinxcontrib_serializinghtml:pkg (...) - .-> @pypi_sphinxcontrib_serializinghtml//:pkg (...) - | @pypi_sphinxcontrib_serializinghtml//:_pkg (...) - | @pypi_sphinx//:pkg (...) - | @pypi_sphinx//:_pkg (...) - `-- @pypi_sphinxcontrib_serializinghtml//:pkg (...) - ``` - - Which we can resolve by configuring these two requirements to be installed together as a cycle - - ``` - pip_parse( - ... - experimental_requirement_cycles = { - "sphinx": [ - "sphinx", - "sphinxcontrib-serializinghtml", - ] - }, - ) - ``` - -Warning: - If a dependency participates in multiple cycles, all of those cycles must be - collapsed down to one. For instance `a <-> b` and `a <-> c` cannot be listed - as two separate cycles. -""", - ), - "extra_hub_aliases": attr.string_list_dict( - doc = """\ -Extra aliases to make for specific wheels in the hub repo. This is useful when -paired with the {attr}`whl_modifications`. - -:::{versionadded} 0.38.0 - -For `pip.parse` with bzlmod -::: - -:::{versionadded} 1.0.0 - -For `pip_parse` with workspace. -::: -""", - mandatory = False, - ), "extra_pip_args": attr.string_list( doc = """Extra arguments to pass on to pip. Must not contain spaces. diff --git a/python/private/pypi/pip_repository_attrs.bzl b/python/private/pypi/pip_repository_attrs.bzl index 23000869e9..4f0b26448b 100644 --- a/python/private/pypi/pip_repository_attrs.bzl +++ b/python/private/pypi/pip_repository_attrs.bzl @@ -21,6 +21,79 @@ repositories.""" load(":attrs.bzl", COMMON_ATTRS = "ATTRS") ATTRS = { + "experimental_requirement_cycles": attr.string_list_dict( + default = {}, + doc = """\ +A mapping of dependency cycle names to a list of requirements which form that cycle. + +Requirements which form cycles will be installed together and taken as +dependencies together in order to ensure that the cycle is always satisified. + +Example: + `sphinx` depends on `sphinxcontrib-serializinghtml` + When listing both as requirements, ala + + ``` + py_binary( + name = "doctool", + ... + deps = [ + "@pypi//sphinx:pkg", + "@pypi//sphinxcontrib_serializinghtml", + ] + ) + ``` + + Will produce a Bazel error such as + + ``` + ERROR: .../external/pypi_sphinxcontrib_serializinghtml/BUILD.bazel:44:6: in alias rule @pypi_sphinxcontrib_serializinghtml//:pkg: cycle in dependency graph: + //:doctool (...) + @pypi//sphinxcontrib_serializinghtml:pkg (...) + .-> @pypi_sphinxcontrib_serializinghtml//:pkg (...) + | @pypi_sphinxcontrib_serializinghtml//:_pkg (...) + | @pypi_sphinx//:pkg (...) + | @pypi_sphinx//:_pkg (...) + `-- @pypi_sphinxcontrib_serializinghtml//:pkg (...) + ``` + + Which we can resolve by configuring these two requirements to be installed together as a cycle + + ``` + pip_parse( + ... + experimental_requirement_cycles = { + "sphinx": [ + "sphinx", + "sphinxcontrib-serializinghtml", + ] + }, + ) + ``` + +Warning: + If a dependency participates in multiple cycles, all of those cycles must be + collapsed down to one. For instance `a <-> b` and `a <-> c` cannot be listed + as two separate cycles. +""", + ), + "extra_hub_aliases": attr.string_list_dict( + doc = """\ +Extra aliases to make for specific wheels in the hub repo. This is useful when +paired with the {attr}`whl_modifications`. + +:::{versionadded} 0.38.0 + +For `pip.parse` with bzlmod +::: + +:::{versionadded} 1.0.0 + +For `pip_parse` with workspace. +::: +""", + mandatory = False, + ), "requirements_by_platform": attr.label_keyed_string_dict( doc = """\ The requirements files and the comma delimited list of target platforms as values. diff --git a/python/private/pypi/whl_library.bzl b/python/private/pypi/whl_library.bzl index 1e75d69c8d..b66e502681 100644 --- a/python/private/pypi/whl_library.bzl +++ b/python/private/pypi/whl_library.bzl @@ -308,110 +308,8 @@ def _to_purl(*, index, metadata, filename): return "pkg:pypi/{}@{}?{}".format(name, metadata.version, "&".join(["{}={}".format(key, val) for key, val in qualifiers.items()])) -def _whl_library_impl(rctx): - logger = repo_utils.logger(rctx) - - whl_path = None - sdist_filename = None - extra_pip_args = [] - extra_pip_args.extend(rctx.attr.extra_pip_args) - if rctx.attr.whl_file: - rctx.watch(rctx.attr.whl_file) - whl_path = rctx.path(rctx.attr.whl_file) - - # Simulate the behaviour where the whl is present in the current directory. - rctx.symlink(whl_path, whl_path.basename) - whl_path = rctx.path(whl_path.basename) - elif rctx.attr.urls and rctx.attr.filename: - filename = rctx.attr.filename - urls = rctx.attr.urls - urls = [ - urllib.absolute_url( - rctx.attr.index_url, - url, - envsubst = rctx.attr.envsubst, - getenv = rctx.getenv, - ) - for url in urls - ] - result = rctx.download( - url = urls, - output = filename, - sha256 = rctx.attr.sha256, - auth = get_auth(rctx, urls), - ) - if not rctx.attr.sha256: - # this is only seen when there is a direct URL reference without sha256 - logger.warn("Please update the requirement line to include the hash:\n{} \\\n --hash=sha256:{}".format( - rctx.attr.requirement, - result.sha256, - )) - - if not result.success: - fail("could not download the '{}' from {}:\n{}".format(filename, urls, result)) - - if filename.endswith(".whl"): - whl_path = rctx.path(filename) - else: - sdist_filename = filename - - # It is an sdist and we need to tell PyPI to use a file in this directory - # and, allow getting build dependencies from PYTHONPATH, which we - # setup in this repository rule, but still download any necessary - # build deps from PyPI (e.g. `flit_core`) if they are missing. - extra_pip_args.extend(["--find-links", "."]) - - # When we already have a wheel, Python isn't used, - # so there's no need to setup env vars to run Python, unless we need to - # build an sdist or resolve a requirement. - if whl_path: - environment = {} - args = [] - python_interpreter = None - else: - python_interpreter = pypi_repo_utils.resolve_python_interpreter( - rctx, - python_interpreter = rctx.attr.python_interpreter, - python_interpreter_target = rctx.attr.python_interpreter_target, - ) - args = [ - "-m", - "python.private.pypi.whl_installer.wheel_installer", - "--requirement", - rctx.attr.requirement, - ] - args = _parse_optional_attrs(rctx, args, extra_pip_args) - - # Manually construct the PYTHONPATH since we cannot use the toolchain here - environment = _create_repository_execution_environment(rctx, python_interpreter, logger = logger) - - if not whl_path: - if rctx.attr.urls: - op_tmpl = "whl_library.BuildWheelFromSource({name}, {requirement})" - elif rctx.attr.download_only: - op_tmpl = "whl_library.DownloadWheel({name}, {requirement})" - else: - op_tmpl = "whl_library.ResolveRequirement({name}, {requirement})" - - pypi_repo_utils.execute_checked( - rctx, - # truncate the requirement value when logging it / reporting - # progress since it may contain several ' --hash=sha256:... - # --hash=sha256:...' substrings that fill up the console - python = python_interpreter, - op = op_tmpl.format(name = rctx.attr.name, requirement = rctx.attr.requirement.split(" ", 1)[0]), - arguments = args, - environment = environment, - srcs = rctx.attr._python_srcs, - quiet = rctx.attr.quiet, - timeout = rctx.attr.timeout, - logger = logger, - ) - - whl_path = rctx.path(json.decode(rctx.read("whl_file.json"))["whl_file"]) - if not rctx.delete("whl_file.json"): - fail("failed to delete the whl_file.json file") - +def _whl_extract(rctx, *, whl_path, logger, sdist_filename = None): + """Extract the wheel, apply patches and generate BUILD.bazel files.""" if rctx.attr.whl_patches: patches = {} for patch_file, json_args in rctx.attr.whl_patches.items(): @@ -441,10 +339,10 @@ def _whl_library_impl(rctx): build_file_contents = generate_whl_library_build_bazel( name = whl_path.basename, - sdist_filename = sdist_filename, dep_template = rctx.attr.dep_template or "@{}{{name}}//:{{target}}".format( rctx.attr.repo_prefix, ), + sdist_filename = sdist_filename, config_load = rctx.attr.config_load, metadata_name = metadata.name, metadata_version = metadata.version, @@ -488,6 +386,144 @@ repo( return None +def _whl_archive_impl(rctx): + logger = repo_utils.logger(rctx) + + whl_path = None + if rctx.attr.whl_file: + rctx.watch(rctx.attr.whl_file) + whl_path = rctx.path(rctx.attr.whl_file) + + # Simulate the behaviour where the whl is present in the current directory. + rctx.symlink(whl_path, whl_path.basename) + whl_path = rctx.path(whl_path.basename) + elif rctx.attr.urls and rctx.attr.filename: + filename = rctx.attr.filename + urls = rctx.attr.urls + urls = [ + urllib.absolute_url( + rctx.attr.index_url, + url, + envsubst = rctx.attr.envsubst, + getenv = rctx.getenv, + ) + for url in urls + ] + result = rctx.download( + url = urls, + output = filename, + sha256 = rctx.attr.sha256, + auth = get_auth(rctx, urls), + ) + if not rctx.attr.sha256: + # this is only seen when there is a direct URL reference without sha256 + logger.warn("Please update the requirement line to include the hash:\n{} \\\n --hash=sha256:{}".format( + rctx.attr.requirement, + result.sha256, + )) + + if not result.success: + fail("could not download the '{}' from {}:\n{}".format(filename, urls, result)) + + if filename.endswith(".whl"): + whl_path = rctx.path(filename) + else: + fail("Only wheels are supported") + + return _whl_extract(rctx, whl_path = whl_path, logger = logger) + +def _pip_archive_impl(rctx): + logger = repo_utils.logger(rctx) + + sdist_filename = None + extra_pip_args = [] + extra_pip_args.extend(rctx.attr.extra_pip_args) + if rctx.attr.urls and rctx.attr.filename: + filename = rctx.attr.filename + urls = rctx.attr.urls + urls = [ + urllib.absolute_url( + rctx.attr.index_url, + url, + envsubst = rctx.attr.envsubst, + getenv = rctx.getenv, + ) + for url in urls + ] + result = rctx.download( + url = urls, + output = filename, + sha256 = rctx.attr.sha256, + auth = get_auth(rctx, urls), + ) + if not rctx.attr.sha256: + # this is only seen when there is a direct URL reference without sha256 + logger.warn("Please update the requirement line to include the hash:\n{} \\\n --hash=sha256:{}".format( + rctx.attr.requirement, + result.sha256, + )) + + if not result.success: + fail("could not download the '{}' from {}:\n{}".format(filename, urls, result)) + + if filename.endswith(".whl"): + fail("Only sdists are supported") + else: + sdist_filename = filename + + # It is an sdist and we need to tell PyPI to use a file in this directory + # and, allow getting build dependencies from PYTHONPATH, which we + # setup in this repository rule, but still download any necessary + # build deps from PyPI (e.g. `flit_core`) if they are missing. + extra_pip_args.extend(["--find-links", "."]) + + # When we already have a wheel, Python isn't used, + # so there's no need to setup env vars to run Python, unless we need to + # build an sdist or resolve a requirement. + python_interpreter = pypi_repo_utils.resolve_python_interpreter( + rctx, + python_interpreter = rctx.attr.python_interpreter, + python_interpreter_target = rctx.attr.python_interpreter_target, + ) + args = [ + "-m", + "python.private.pypi.whl_installer.wheel_installer", + "--requirement", + rctx.attr.requirement, + ] + args = _parse_optional_attrs(rctx, args, extra_pip_args) + + # Manually construct the PYTHONPATH since we cannot use the toolchain here + environment = _create_repository_execution_environment(rctx, python_interpreter, logger = logger) + + if rctx.attr.urls: + op_tmpl = "whl_library.BuildWheelFromSource({name}, {requirement})" + elif rctx.attr.download_only: + op_tmpl = "whl_library.DownloadWheel({name}, {requirement})" + else: + op_tmpl = "whl_library.ResolveRequirement({name}, {requirement})" + + pypi_repo_utils.execute_checked( + rctx, + # truncate the requirement value when logging it / reporting + # progress since it may contain several ' --hash=sha256:... + # --hash=sha256:...' substrings that fill up the console + python = python_interpreter, + op = op_tmpl.format(name = rctx.attr.name, requirement = rctx.attr.requirement.split(" ", 1)[0]), + arguments = args, + environment = environment, + srcs = rctx.attr._python_srcs, + quiet = rctx.attr.quiet, + timeout = rctx.attr.timeout, + logger = logger, + ) + + whl_path = rctx.path(json.decode(rctx.read("whl_file.json"))["whl_file"]) + if not rctx.delete("whl_file.json"): + fail("failed to delete the whl_file.json file") + + return _whl_extract(rctx, whl_path = whl_path, logger = logger, sdist_filename = sdist_filename) + def _remove_files(rctx, *basenames): paths = list(rctx.path(".").readdir()) for _ in range(10000000): @@ -501,7 +537,7 @@ def _remove_files(rctx, *basenames): paths.extend(path.readdir()) # NOTE @aignas 2024-03-21: The usage of dict({}, **common) ensures that all args to `dict` are unique -whl_library_attrs = dict({ +_pip_archive_attrs = dict({ "annotation": attr.label( doc = ( "Optional json encoded file containing annotation to apply to the extracted wheel. " + @@ -557,9 +593,6 @@ DEPRECATED. Only left for people who vendor requirements.bzl. The list of urls of the whl to be downloaded using bazel downloader. Using this attr makes `extra_pip_args` and `download_only` ignored.""", ), - "whl_file": attr.label( - doc = "The whl file that should be used instead of downloading or building the whl.", - ), "whl_patches": attr.label_keyed_string_dict( doc = """ A label-keyed-string dict with patch files as keys and json-strings as values. @@ -607,10 +640,10 @@ way to define whl_library and move whl patching to a separate place. INTERNAL US ), "_rule_name": attr.string(default = "whl_library"), }, **ATTRS) -whl_library_attrs.update(AUTH_ATTRS) +_pip_archive_attrs.update(AUTH_ATTRS) -whl_library = repository_rule( - attrs = whl_library_attrs, +pip_archive = repository_rule( + attrs = _pip_archive_attrs, doc = """ Download and extracts a single wheel based into a bazel repo based on the requirement string passed in. Instantiated from pip_repository and inherits config options from there. @@ -619,10 +652,75 @@ Instantiated from pip_repository and inherits config options from there. The `whl_library` is marked as reproducible if using starlark to extract and parse the wheel contents without building an `sdist` first. ::: + +:::{versionchanged} VERSION_NEXT_FEATURE +The whl-only pure Starlark operations have been refactored into {obj}`whl_archive` and the +previously named {obj}`whl_library` repository became renamed to `pip_archive`. +::: """, - implementation = _whl_library_impl, + implementation = _pip_archive_impl, environ = [ "RULES_PYTHON_PIP_ISOLATED", REPO_DEBUG_ENV_VAR, ], ) + +whl_archive = repository_rule( + attrs = { + k: _pip_archive_attrs[k] + for k in [ + "annotation", + "config_load", + "dep_template", + "filename", + "group_deps", + "group_name", + "index_url", + "repo", + "repo_prefix", + "requirement", + "sha256", + "urls", + "whl_patches", + # common attrs + "enable_implicit_namespace_pkgs", + "envsubst", + "pip_data_exclude", + ] + } | { + "whl_file": attr.label( + doc = "The whl file that should be used instead of downloading or building the whl.", + ), + } | AUTH_ATTRS, + doc = """ +Download and extracts a single wheel based into a bazel repo based on the requirement string passed in. + +Does not depend on any python. +""", + implementation = _whl_archive_impl, + environ = [ + REPO_DEBUG_ENV_VAR, + ], +) + +def whl_library(name, **kwargs): + """Create a whl_library. + + This proxies to one of the underlying implementations: + * {obj}`whl_archive` + * {obj}`pip_archive` + + Args: + name: {type}`str` The name of the repo. + **kwargs: The args passed to the underlying implementation. + + Returns: + the repo metadata. + """ + whl_file = kwargs.get("whl_file") + urls = kwargs.get("urls", []) + filename = kwargs.get("filename") + if whl_file or (urls and filename and filename.endswith(".whl")): + return whl_archive(name = name, **kwargs) + + return pip_archive(name = name, **kwargs) From e44e24dcac7d428392456913b6f335ebf964a22b Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Sat, 25 Jul 2026 12:29:40 -0700 Subject: [PATCH 853/922] build(deps): bump myst-parser from 4.0.1 to 5.1.0 in /docs (#3942) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Bumps [myst-parser](https://github.com/executablebooks/MyST-Parser) from 4.0.1 to 5.1.0.
Release notes

Sourced from myst-parser's releases.

v5.1.0

✨ New Features

👌 Improvements

  • 👌 Update myst_gfm_only mode to use the unified gfm_plugin, which now includes GFM autolinks, alerts, and improved strikethrough/tasklist handling by @​chrisjsewell in #1128
  • 👌 Improve MathJax 4 compatibility for Sphinx 9 by @​chrisjsewell in #1110
  • 👌 Stop directive-option parsing at colon fences, fixing nested colon fence directives by @​chrisjsewell in #1133

🐛 Bug Fixes

⬆️ Dependency Upgrades

New Contributors

Full Changelog: https://github.com/executablebooks/MyST-Parser/compare/v5.0.0...v5.1.0

v5.0.0

MyST-Parser 5.0.0

Release Date: 2026-01-15

This release significantly bumps the supported versions of core dependencies:

‼️ Breaking Changes

This release updates the minimum supported versions:

  • Python: >=3.11 (dropped Python 3.10, tests up to 3.14)
  • Sphinx: >=8,<10 (dropped Sphinx 7, added Sphinx 9)
  • Docutils: >=0.20,<0.23 (dropped docutils 0.19, added docutils 0.22)
  • markdown-it-py: ~=4.0 (upgraded from v3)

... (truncated)

Changelog

Sourced from myst-parser's changelog.

5.1.0 - 2026-05-13

✨ New Features

👌 Improvements

🐛 Bug Fixes

⬆️ Dependency Upgrades

Full Changelog: v5.0.0...v5.1.0

5.0.0 - 2026-01-15

This release significantly bumps the supported versions of core dependencies:

‼️ Breaking Changes

This release updates the minimum supported versions:

  • Python: >=3.11 (dropped Python 3.10, tests up to 3.14)
  • Sphinx: >=8,<10 (dropped Sphinx 7, added Sphinx 9)
  • Docutils: >=0.20,<0.23 (dropped docutils 0.19, added docutils 0.22)
  • markdown-it-py: ~=4.0 (upgraded from v3)

⬆️ Dependency Upgrades

... (truncated)

Commits
  • 2871eb9 🚀 Release v5.1.0 (#1135)
  • cc5db37 🐛 FIX: Pin mdit-py-plugins>=0.6.1 for nested field list fix (#1134)
  • 4ce57f9 👌 Stop directive-option parsing at colon fences (#1133)
  • cfcc327 ⬆️ Bump mypy from 2.0.0 to 2.1.0 (#1131)
  • 691738c ⬆️ Bump ruff from 0.15.10 to 0.15.12 (#1132)
  • 0fb1ae9 👌 IMPROVE: MathJax 4 compatibility (Sphinx 9) (#1110)
  • f153b4b ⬆️ Bump actions/setup-python from 5 to 6 (#1092)
  • 93acf8d [pre-commit.ci] pre-commit autoupdate (#1095)
  • a5f1d69 ⬆️ Update pygments requirement from <2.20 to <2.21 (#1117)
  • 8381296 🐛 FIX: Use docname instead of source path in warning locations (#1114)
  • Additional commits viewable in compare view

[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=myst-parser&package-manager=pip&previous-version=4.0.1&new-version=5.1.0)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) ---
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot show ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> --- docs/requirements.txt | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/requirements.txt b/docs/requirements.txt index 5ac2d2b323..9a266e9644 100644 --- a/docs/requirements.txt +++ b/docs/requirements.txt @@ -309,9 +309,9 @@ myst-parser==3.0.1 ; python_full_version < '3.10' \ --hash=sha256:6457aaa33a5d474aca678b8ead9b3dc298e89c68e67012e73146ea6fd54babf1 \ --hash=sha256:88f0cb406cb363b077d176b51c476f62d60604d68a8dcdf4832e080441301a87 # via rules-python-docs (docs/pyproject.toml) -myst-parser==4.0.1 ; python_full_version == '3.10.*' \ - --hash=sha256:5cfea715e4f3574138aecbf7d54132296bfd72bb614d31168f48c477a830a7c4 \ - --hash=sha256:9134e88959ec3b5780aedf8a99680ea242869d012e8821db3126d427edc9c95d +myst-parser==5.1.0 ; python_full_version == '3.10.*' \ + --hash=sha256:9c91c52b3cdb4d94a6506e4fab4e2f296c7623a0da0dcbe6de1565c3dad67a8a \ + --hash=sha256:ab69322dc6719dcc7f296479dbb70181b66df6ed315064f92dbc85c0e1bf2f02 # via rules-python-docs (docs/pyproject.toml) myst-parser==5.1.0 ; python_full_version >= '3.11' \ --hash=sha256:9c91c52b3cdb4d94a6506e4fab4e2f296c7623a0da0dcbe6de1565c3dad67a8a \ From 87a6f0c77d5d18a9c9ce5d6e1b19aa53cbda7bcb Mon Sep 17 00:00:00 2001 From: Ignas Anikevicius <240938+aignas@users.noreply.github.com> Date: Sun, 26 Jul 2026 07:52:35 +0900 Subject: [PATCH 854/922] refactor(pypi): a seam for source vs library targets (#3949) With this we are starting to separate whl_library_targets into 2 parts - one for sources only (without deps) and another one is just the deps parts. Next PR I'll create a way to create 2 separate instances. Split out of #3856 Work towards #2948 --- python/private/pypi/labels.bzl | 2 + python/private/pypi/whl_library.bzl | 1 - python/private/pypi/whl_library_targets.bzl | 299 ++++++++++++------ .../whl_library_targets_tests.bzl | 28 +- 4 files changed, 217 insertions(+), 113 deletions(-) diff --git a/python/private/pypi/labels.bzl b/python/private/pypi/labels.bzl index 8f91a03b4c..c77167b593 100644 --- a/python/private/pypi/labels.bzl +++ b/python/private/pypi/labels.bzl @@ -17,8 +17,10 @@ EXTRACTED_WHEEL_FILES = "extracted_whl_files" WHEEL_FILE_PUBLIC_LABEL = "whl" WHEEL_FILE_IMPL_LABEL = "_whl" +WHEEL_FILE = "whl_file" PY_LIBRARY_PUBLIC_LABEL = "pkg" PY_LIBRARY_IMPL_LABEL = "_pkg" +PY_SRCS_LABEL = "srcs" DATA_LABEL = "data" DIST_INFO_LABEL = "dist_info" NODEPS_LABEL = "no_deps" diff --git a/python/private/pypi/whl_library.bzl b/python/private/pypi/whl_library.bzl index b66e502681..9a150db9e8 100644 --- a/python/private/pypi/whl_library.bzl +++ b/python/private/pypi/whl_library.bzl @@ -682,7 +682,6 @@ whl_archive = repository_rule( "sha256", "urls", "whl_patches", - # common attrs "enable_implicit_namespace_pkgs", "envsubst", "pip_data_exclude", diff --git a/python/private/pypi/whl_library_targets.bzl b/python/private/pypi/whl_library_targets.bzl index 933b529053..217210e782 100644 --- a/python/private/pypi/whl_library_targets.bzl +++ b/python/private/pypi/whl_library_targets.bzl @@ -26,6 +26,8 @@ load( "EXTRACTED_WHEEL_FILES", "PY_LIBRARY_IMPL_LABEL", "PY_LIBRARY_PUBLIC_LABEL", + "PY_SRCS_LABEL", + "WHEEL_FILE", "WHEEL_FILE_IMPL_LABEL", "WHEEL_FILE_PUBLIC_LABEL", ) @@ -57,6 +59,19 @@ def whl_library_targets_from_requires( entry_points = {}, include = [], group_deps = [], + group_name = None, + dep_template = None, + data_exclude = [], + enable_implicit_namespace_pkgs = False, + sdist_filename = None, + namespace_package_files = [], + filegroups = None, + copy_files = {}, + copy_executables = {}, + srcs_exclude = [], + data = [], + visibility = ["//visibility:public"], + tags = [], **kwargs): """The macro to create whl targets from the METADATA. @@ -73,8 +88,44 @@ def whl_library_targets_from_requires( extras: {type}`list[str]` The list of requested extras. This essentially includes extra transitive dependencies in the final targets depending on the wheel `METADATA`. entry_points: {type}`list[dict]` A list of parsed entry point definitions. include: {type}`list[str]` The list of packages to include. - **kwargs: Extra args passed to the {obj}`whl_library_targets` + group_name: {type}`str | None` name of the dependency group (if any). + dep_template: {type}`str | None` The dep_template to use. + data_exclude: {type}`list[str]` The globs for data attribute exclusion. + enable_implicit_namespace_pkgs: {type}`boolean` generate __init__.py files for namespace pkgs. + sdist_filename: {type}`str | None` The filename of the sdist. + namespace_package_files: {type}`list[str]` A list of labels of files whose directories are namespace packages. + filegroups: {type}`dict[str, list[str]] | None` A dictionary of the target names and the glob matches. + copy_files: {type}`dict[str, str]` The mapping between src and dest locations. + copy_executables: {type}`dict[str, str]` The mapping between src and dest locations for executables. + srcs_exclude: {type}`list[str]` The globs for srcs attribute exclusion. + data: {type}`list[str]` A list of labels to include as part of the `data` attribute. + visibility: {type}`list[str]` The visibility of the targets. + tags: {type}`list[str]` The tags set on the targets. + **kwargs: Extra args passed to the {obj}`whl_library_targets` and {obj}`whl_library_srcs`. """ + pypi_tags = [ + "pypi_name={}".format(metadata_name), + "pypi_version={}".format(metadata_version), + ] + all_tags = sorted(tags + pypi_tags) + + whl_library_srcs( + name = name, + sdist_filename = sdist_filename, + data_exclude = data_exclude, + srcs_exclude = srcs_exclude, + tags = all_tags, + filegroups = filegroups, + entry_points = entry_points, + visibility = visibility, + data = data, + copy_files = copy_files, + copy_executables = copy_executables, + enable_implicit_namespace_pkgs = enable_implicit_namespace_pkgs, + namespace_package_files = namespace_package_files, + **kwargs + ) + package_deps = _parse_requires_dist( name = metadata_name, requires_dist = requires_dist, @@ -87,42 +138,22 @@ def whl_library_targets_from_requires( name = name, dependencies = package_deps.deps, dependencies_with_markers = package_deps.deps_select, - entry_points = entry_points, - tags = [ - "pypi_name={}".format(metadata_name), - "pypi_version={}".format(metadata_version), - ], + group_name = group_name, + dep_template = dep_template, + tags = all_tags, **kwargs ) -def _parse_requires_dist( +def whl_library_srcs( *, name, - requires_dist, - excludes, - include, - extras): - return deps( - name = normalize_name(name), - requires_dist = requires_dist, - excludes = excludes, - include = include, - extras = extras, - ) - -def whl_library_targets( - *, - name, - dep_template, sdist_filename = None, data_exclude = [], srcs_exclude = [], tags = [], - dependencies = [], filegroups = None, - dependencies_with_markers = {}, entry_points = {}, - group_name = "", + visibility = ["//visibility:public"], data = [], copy_files = {}, copy_executables = {}, @@ -143,21 +174,13 @@ def whl_library_targets( Args: name: {type}`str` The file to match for including it into the `whl` filegroup. This may be also parsed to generate extra metadata. - dep_template: {type}`str` The dep_template to use for dependency - interpolation. sdist_filename: {type}`str | None` If the wheel was built from an sdist, the filename of the sdist. + visibility: {type}`list[str]` The visibility of the source targets. tags: {type}`list[str]` The tags set on the `py_library`. - dependencies: {type}`list[str]` A list of dependencies. - dependencies_with_markers: {type}`dict[str, str]` A marker to evaluate - in order for the dep to be included. entry_points: {type}`list[dict]` A list of parsed entry point definitions. filegroups: {type}`dict[str, list[str]] | None` A dictionary of the target names and the glob matches. If `None`, defaults will be used. - group_name: {type}`str` name of the dependency group (if any) which - contains this library. If set, this library will behave as a shim - to group implementation rules which will provide simultaneously - installed dependencies which would otherwise form a cycle. copy_executables: {type}`dict[str, str]` The mapping between src and dest locations for the targets. copy_files: {type}`dict[str, str]` The mapping between src and @@ -174,7 +197,6 @@ def whl_library_targets( directories are namespace packages. rules: {type}`struct` A struct with references to rules for creating targets. """ - dependencies = sorted([normalize_name(d) for d in dependencies]) tags = sorted(tags) data = [] + data @@ -233,7 +255,7 @@ def whl_library_targets( native.filegroup( name = filegroup_name, srcs = srcs, - visibility = ["//visibility:public"], + visibility = visibility, ) for src, dest in copy_files.items(): @@ -241,7 +263,7 @@ def whl_library_targets( name = dest + ".copy", src = src, out = dest, - visibility = ["//visibility:public"], + visibility = visibility, ) data.append(dest) for src, dest in copy_executables.items(): @@ -250,10 +272,137 @@ def whl_library_targets( src = src, out = dest, is_executable = True, - visibility = ["//visibility:public"], + visibility = visibility, ) data.append(dest) + if hasattr(native, "filegroup"): + native.filegroup( + name = WHEEL_FILE, + srcs = [name], + visibility = visibility, + ) + + if hasattr(rules, "py_library"): + srcs = native.glob( + ["site-packages/**/*.py"], + exclude = srcs_exclude, + # Empty sources are allowed to support wheels that don't have any + # pure-Python code, e.g. pymssql, which is written in Cython. + allow_empty = True, + ) + + # NOTE: pyi files should probably be excluded because they're carried + # by the pyi_srcs attribute. However, historical behavior included + # them in data and some tools currently rely on that. + _data_exclude = [ + "**/*.py", + "**/*.pyc", + "**/*.pyc.*", # During pyc creation, temp files named *.pyc.NNNN are created + ] + if sdist_filename: + _data_exclude.append("**/*.dist-info/RECORD") + for item in data_exclude: + if item not in _data_exclude: + _data_exclude.append(item) + + data = data + native.glob( + ["site-packages/**/*"], + exclude = _data_exclude, + allow_empty = True, + ) + + pyi_srcs = native.glob( + ["site-packages/**/*.pyi"], + allow_empty = True, + ) + + if not enable_implicit_namespace_pkgs: + generated_namespace_package_files = select({ + _IS_VENV_SITE_PACKAGES_YES: [], + "//conditions:default": rules.create_inits( + srcs = srcs + data + pyi_srcs, + ignored_dirnames = [], # If you need to ignore certain folders, you can patch rules_python here to do so. + root = "site-packages", + ), + }) + namespace_package_files += generated_namespace_package_files + srcs = srcs + generated_namespace_package_files + + # This is done after create_inits() is called so that the data scheme + # files don't have such files created in their directories. + data = data + [DATA_LABEL] + + rules.py_library( + name = PY_SRCS_LABEL, + srcs = srcs, + pyi_srcs = pyi_srcs, + data = data, + # This makes this directory a top-level in the python import + # search path for anything that depends on this. + imports = ["site-packages"], + tags = tags, + visibility = visibility, + experimental_venvs_site_packages = _VENV_SITE_PACKAGES_FLAG, + namespace_package_files = namespace_package_files, + ) + +def _parse_requires_dist( + *, + name, + requires_dist, + excludes, + include, + extras): + return deps( + name = normalize_name(name), + requires_dist = requires_dist, + excludes = excludes, + include = include, + extras = extras, + ) + +def whl_library_targets( + *, + name, + dep_template, + tags = [], + dependencies = [], + dependencies_with_markers = {}, + group_name = "", + native = native, + rules = struct( + copy_file = copy_file, + py_binary = py_binary, + py_library = py_library, + venv_entry_point = venv_entry_point, + venv_rewrite_shebang = venv_rewrite_shebang, + env_marker_setting = env_marker_setting, + create_inits = _create_inits, + ), + **_kwargs): + """Create all of the whl_library targets. + + Args: + name: {type}`str` The file to match for including it into the `whl` + filegroup. This may be also parsed to generate extra metadata. + dep_template: {type}`str` The dep_template to use for dependency + interpolation. + tags: {type}`list[str]` The tags set on the `py_library`. + dependencies: {type}`list[str]` A list of dependencies. + dependencies_with_markers: {type}`dict[str, str]` A marker to evaluate + in order for the dep to be included. + group_name: {type}`str` name of the dependency group (if any) which + contains this library. If set, this library will behave as a shim + to group implementation rules which will provide simultaneously + installed dependencies which would otherwise form a cycle. + native: {type}`native` The native struct for overriding in tests. + rules: {type}`struct` A struct with references to rules for creating targets. + **_kwargs: ignored args that are not needed. + """ + dependencies = sorted([normalize_name(d) for d in dependencies]) + tags = sorted(tags) + _config_settings( dependencies_with_markers = dependencies_with_markers, rules = rules, @@ -315,8 +464,9 @@ def whl_library_targets( if hasattr(native, "filegroup"): native.filegroup( name = whl_file_label, - srcs = [name], - data = _deps( + data = [ + WHEEL_FILE, + ] + _deps( deps = dependencies, deps_conditional = deps_conditional, tmpl = dep_template.format(name = "{}", target = WHEEL_FILE_PUBLIC_LABEL), @@ -325,72 +475,23 @@ def whl_library_targets( ) if hasattr(rules, "py_library"): - srcs = native.glob( - ["site-packages/**/*.py"], - exclude = srcs_exclude, - # Empty sources are allowed to support wheels that don't have any - # pure-Python code, e.g. pymssql, which is written in Cython. - allow_empty = True, - ) - - # NOTE: pyi files should probably be excluded because they're carried - # by the pyi_srcs attribute. However, historical behavior included - # them in data and some tools currently rely on that. - _data_exclude = [ - "**/*.py", - "**/*.pyc", - "**/*.pyc.*", # During pyc creation, temp files named *.pyc.NNNN are created - ] - if sdist_filename: - _data_exclude.append("**/*.dist-info/RECORD") - for item in data_exclude: - if item not in _data_exclude: - _data_exclude.append(item) - - data = data + native.glob( - ["site-packages/**/*"], - exclude = _data_exclude, - allow_empty = True, - ) - - pyi_srcs = native.glob( - ["site-packages/**/*.pyi"], - allow_empty = True, - ) - - if not enable_implicit_namespace_pkgs: - generated_namespace_package_files = select({ - _IS_VENV_SITE_PACKAGES_YES: [], - "//conditions:default": rules.create_inits( - srcs = srcs + data + pyi_srcs, - ignored_dirnames = [], # If you need to ignore certain folders, you can patch rules_python here to do so. - root = "site-packages", - ), - }) - namespace_package_files += generated_namespace_package_files - srcs = srcs + generated_namespace_package_files - - # This is done after create_inits() is called so that the data scheme - # files don't have such files created in their directories. - data = data + [DATA_LABEL] - rules.py_library( name = py_library_label, - srcs = srcs, - pyi_srcs = pyi_srcs, - data = data, - # This makes this directory a top-level in the python import - # search path for anything that depends on this. - imports = ["site-packages"], - deps = _deps( + srcs = [ + # We include as srcs to ensure that the (locations :pkg) works as expected. + PY_SRCS_LABEL, + ], + deps = [ + # We include as deps, so that `PyInfo` and friends get propagated as deps. + # not sure if just including it as `srcs` is enough. + PY_SRCS_LABEL, + ] + _deps( deps = dependencies, deps_conditional = deps_conditional, tmpl = dep_template.format(name = "{}", target = PY_LIBRARY_PUBLIC_LABEL), ), tags = tags, visibility = impl_vis, - experimental_venvs_site_packages = _VENV_SITE_PACKAGES_FLAG, - namespace_package_files = namespace_package_files, ) def _config_settings(dependencies_with_markers, rules, **kwargs): diff --git a/tests/pypi/whl_library_targets/whl_library_targets_tests.bzl b/tests/pypi/whl_library_targets/whl_library_targets_tests.bzl index 5bd1d1f549..1f4aac1ba8 100644 --- a/tests/pypi/whl_library_targets/whl_library_targets_tests.bzl +++ b/tests/pypi/whl_library_targets/whl_library_targets_tests.bzl @@ -17,7 +17,7 @@ load("@rules_testing//lib:test_suite.bzl", "test_suite") load( "//python/private/pypi:whl_library_targets.bzl", - "whl_library_targets", + "whl_library_srcs", "whl_library_targets_from_requires", ) # buildifier: disable=bzl-visibility load("//tests/support/mocks:mocks.bzl", "mocks") @@ -34,9 +34,8 @@ def _test_filegroups(env): return [] return include - whl_library_targets( + whl_library_srcs( name = "", - dep_template = "", native = struct( filegroup = lambda **kwargs: calls.append(kwargs), glob = glob, @@ -63,9 +62,8 @@ def _test_filegroups(env): "visibility": ["//visibility:public"], }, { - "name": "whl", + "name": "whl_file", "srcs": [""], - "data": [], "visibility": ["//visibility:public"], }, ]) # buildifier: @unsorted-dict-items @@ -75,9 +73,8 @@ _tests.append(_test_filegroups) def _test_copy(env): calls = [] - whl_library_targets( + whl_library_srcs( name = "", - dep_template = None, filegroups = {}, copy_files = {"file_src": "file_dest"}, copy_executables = {"exec_src": "exec_dest"}, @@ -151,17 +148,23 @@ def _test_whl_and_library_deps_from_requires(env): env.expect.that_collection(filegroup_calls).contains_exactly([ { - "name": "whl", + "name": "whl_file", "srcs": ["foo-0-py3-none-any.whl"], - "data": ["@pypi//bar:whl"] + select({ + "visibility": ["//visibility:public"], + }, + { + "name": "whl", + # NOTE @aignas 2026-07-25: depending on the brackets position one may get different + # results in the expectation. + "data": ["whl_file"] + (["@pypi//bar:whl"] + select({ ":is_include_bar_baz_true": ["@pypi//bar_baz:whl"], "//conditions:default": [], - }), + })), "visibility": ["//visibility:public"], }, ]) # buildifier: @unsorted-dict-items - env.expect.that_collection(py_library_calls).has_size(1) + env.expect.that_collection(py_library_calls).has_size(2) if len(py_library_calls) != 1: return py_library_call = py_library_calls[0] @@ -238,9 +241,8 @@ def _test_sdist_excludes_record(env): m_glob.results.append([]) # data m_glob.results.append([]) # pyi - whl_library_targets( + whl_library_srcs( name = "foo.whl", - dep_template = "@pypi_{name}//:{target}", sdist_filename = "foo.tar.gz", filegroups = {}, native = struct( From 2ab469ed3936dc314f7c9246705163fc631eb9d4 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Sun, 26 Jul 2026 19:18:38 -0700 Subject: [PATCH 855/922] docs(pypi): add versionadded 2.2.0 directive to uv_lock attribute (#3953) The uv_lock argument for pip.parse was introduced in version 2.2.0, but its attribute documentation did not indicate when it was added. To fix this, add the :::{versionadded} 2.2.0::: directive to the uv_lock attribute docstring in python/private/pypi/extension.bzl. --- python/private/pypi/extension.bzl | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/python/private/pypi/extension.bzl b/python/private/pypi/extension.bzl index b79d04c075..412b03c21f 100644 --- a/python/private/pypi/extension.bzl +++ b/python/private/pypi/extension.bzl @@ -991,9 +991,12 @@ a string `"{os}_{arch}"` as the value here. You could also use `"{os}_{arch}_fre """, ), "uv_lock": attr.label( - doc = """\ + doc = """ (label, optional): A label pointing to the uv.lock file. If provided, the uv.lock file will be used as the primary source for package metadata. + +:::{versionadded} 2.2.0 +::: """, ), "whl_modifications": attr.label_keyed_string_dict( From 0b6c475b35b2991e7f5d3b86f4e3b078df122fd0 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Sun, 26 Jul 2026 19:34:03 -0700 Subject: [PATCH 856/922] agents: clarify fast-tests config flag usage for building (#3954) Currently, AGENTS.md instructs agents to add --config=fast-tests when building. However, --config=fast-tests enables --build_tests_only=true, which silently ignores non-test targets like //docs:docs and reports 0 targets found. To fix this, clarify that --config=fast-tests should only be used when building test targets, and must not be used when building non-test targets. --- AGENTS.md | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/AGENTS.md b/AGENTS.md index 170aa49c8e..d66e18ca11 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -149,7 +149,10 @@ Tests are under the `tests/` directory. When testing, add `--config=fast-tests`. -When building, add `--config=fast-tests`. +When building test targets, add `--config=fast-tests`. Do NOT use +`--config=fast-tests` when building non-test targets (such as `//docs:docs`), +because `--config=fast-tests` enables `--build_tests_only=true`, which causes +non-test targets to be silently ignored (finding 0 targets). The `--config=fast-tests` flag avoids running expensive and slow tests can that freeze the host machine or cause flakiness. From 44a0ff793c272d549ef04f3de4e636bd5378e099 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Sun, 26 Jul 2026 19:35:13 -0700 Subject: [PATCH 857/922] sphinxdocs: create sphinxdocs-specific pyproject and requirements (#3951) To further separate sphinxdocs from rules_python, give it its own pyproject and requirements files. Also switch it to using uv.lock instead of requirements.lock --- sphinxdocs/.bazelignore | 8 + sphinxdocs/BUILD.bazel | 3 + sphinxdocs/MODULE.bazel | 5 +- sphinxdocs/dev/BUILD.bazel | 30 + sphinxdocs/dev/requirements.txt | 556 +++++++++ sphinxdocs/dev/uv.lock | 1073 +++++++++++++++++ sphinxdocs/integration_tests/bcr/MODULE.bazel | 2 +- .../persistent_worker/workspace/MODULE.bazel | 2 +- sphinxdocs/pyproject.toml | 15 + 9 files changed, 1690 insertions(+), 4 deletions(-) create mode 100644 sphinxdocs/dev/BUILD.bazel create mode 100644 sphinxdocs/dev/requirements.txt create mode 100644 sphinxdocs/dev/uv.lock create mode 100644 sphinxdocs/pyproject.toml diff --git a/sphinxdocs/.bazelignore b/sphinxdocs/.bazelignore index 80f2d3514f..48237cb876 100644 --- a/sphinxdocs/.bazelignore +++ b/sphinxdocs/.bazelignore @@ -2,3 +2,11 @@ bazel-bin bazel-out bazel-testlogs bazel-sphinxdocs +integration_tests/bcr/bazel-bin +integration_tests/bcr/bazel-bcr +integration_tests/bcr/bazel-out +integration_tests/bcr/bazel-testlogs +integration_tests/persistent_worker/workspace/bazel-bin +integration_tests/persistent_worker/workspace/bazel-out +integration_tests/persistent_worker/workspace/bazel-testlogs +integration_tests/persistent_worker/workspace/bazel-workspace diff --git a/sphinxdocs/BUILD.bazel b/sphinxdocs/BUILD.bazel index 8ff9bc956f..d1eec30a09 100644 --- a/sphinxdocs/BUILD.bazel +++ b/sphinxdocs/BUILD.bazel @@ -1,5 +1,7 @@ package(default_visibility = ["//visibility:public"]) +exports_files(["pyproject.toml"]) + filegroup( name = "distribution", srcs = glob( @@ -10,6 +12,7 @@ filegroup( "tests/**", ], ) + [ + "//dev:distribution", "//sphinxdocs:distribution", ], visibility = ["//visibility:public"], diff --git a/sphinxdocs/MODULE.bazel b/sphinxdocs/MODULE.bazel index 3d4caf2bb2..51de5e94f9 100644 --- a/sphinxdocs/MODULE.bazel +++ b/sphinxdocs/MODULE.bazel @@ -8,7 +8,7 @@ bazel_dep(name = "bazel_skylib", version = "1.8.2") bazel_dep(name = "stardoc", version = "0.7.2", repo_name = "io_bazel_stardoc") bazel_dep(name = "platforms", version = "0.0.11") bazel_dep(name = "protobuf", version = "29.0-rc2", repo_name = "com_google_protobuf") -bazel_dep(name = "rules_python", version = "1.8.5") +bazel_dep(name = "rules_python", version = "2.2.0") dev_pip = use_extension( "@rules_python//python/extensions:pip.bzl", @@ -18,7 +18,8 @@ dev_pip = use_extension( dev_pip.parse( hub_name = "dev_pip", python_version = "3.11", - requirements_lock = "@rules_python//docs:requirements.txt", + requirements_lock = "//dev:requirements.txt", + uv_lock = "//dev:uv.lock", ) use_repo(dev_pip, "dev_pip") diff --git a/sphinxdocs/dev/BUILD.bazel b/sphinxdocs/dev/BUILD.bazel new file mode 100644 index 0000000000..41e88f2c59 --- /dev/null +++ b/sphinxdocs/dev/BUILD.bazel @@ -0,0 +1,30 @@ +load("@rules_python//python/uv:lock.bzl", "lock") + +package(default_visibility = ["//:__subpackages__"]) + +filegroup( + name = "distribution", + srcs = glob(["**/*"]), + visibility = ["//:__pkg__"], +) + +# Run bazel run //dev:uv_lock.update +lock( + name = "uv_lock", + srcs = ["//:pyproject.toml"], + out = "uv.lock", + python_version = "3.11", +) + +# Run bazel run //dev:requirements.update +lock( + name = "requirements", + srcs = ["//:pyproject.toml"], + out = "requirements.txt", + args = [ + "--emit-index-url", + "--universal", + "--upgrade", + ], + python_version = "3.11", +) diff --git a/sphinxdocs/dev/requirements.txt b/sphinxdocs/dev/requirements.txt new file mode 100644 index 0000000000..e24834fe1e --- /dev/null +++ b/sphinxdocs/dev/requirements.txt @@ -0,0 +1,556 @@ +# This file was autogenerated by uv via the following command: +# bazel run //dev:requirements.update +--index-url https://pypi.org/simple + +absl-py==2.3.1 ; python_full_version < '3.10' \ + --hash=sha256:a97820526f7fbfd2ec1bce83f3f25e3a14840dac0d8e02a0b71cd75db3f77fc9 \ + --hash=sha256:eeecf07f0c2a93ace0772c92e596ace6d3d3996c042b2128459aaae2a76de11d + # via rules-python-sphinxdocs-dev (dev/pyproject.toml) +absl-py==2.5.0 ; python_full_version >= '3.10' \ + --hash=sha256:0c996f25c0490700fadabe6351630f6111534fa0ae252cc6d2014ea3b141135f \ + --hash=sha256:0f17b89f2a4eaaedc4f28c622998aa690564b3012a396a4ffad0821007fe03ba + # via rules-python-sphinxdocs-dev (dev/pyproject.toml) +alabaster==0.7.16 ; python_full_version < '3.10' \ + --hash=sha256:75a8b99c28a5dad50dd7f8ccdd447a121ddb3892da9e53d1ca5cca3106d58d65 \ + --hash=sha256:b46733c07dce03ae4e150330b975c75737fa60f0a7c591b6c8bf4928a28e2c92 + # via sphinx +alabaster==1.0.0 ; python_full_version >= '3.10' \ + --hash=sha256:c00dca57bca26fa62a6d7d0a9fcce65f3e026e9bfe33e9c538fd3fbb2144fd9e \ + --hash=sha256:fc6786402dc3fcb2de3cabd5fe455a2db534b371124f1f21de8731783dec828b + # via sphinx +astroid==3.3.11 \ + --hash=sha256:1e5a5011af2920c7c67a53f65d536d65bfa7116feeaf2354d8b94f29573bb0ce \ + --hash=sha256:54c760ae8322ece1abd213057c4b5bba7c49818853fc901ef09719a60dbf9dec + # via sphinx-autodoc2 +babel==2.18.0 \ + --hash=sha256:b80b99a14bd085fcacfa15c9165f651fbb3406e66cc603abf11c5750937c992d \ + --hash=sha256:e2b422b277c2b9a9630c1d7903c2a00d0830c409c59ac8cae9081c92f1aeba35 + # via sphinx +certifi==2026.7.22 \ + --hash=sha256:62f22742b58a1a33014a2b6b706588a8d7e2a88ae7bd1a6ebe8c992928483775 \ + --hash=sha256:741e2c3b351ddf169a738da9f2c048608ff7f2c5cc02f1ebc6b118bb090d5d55 + # via requests +charset-normalizer==3.4.9 \ + --hash=sha256:0327fcd59a935777d83410750c50600ee9571af2846f71ce40f25b13da1ef380 \ + --hash=sha256:03d07803992c6c7bbc976327f34b18b6160327fc81cb82c9d504720ac0be3b62 \ + --hash=sha256:04ce310cb89c15df659582aee80a0603788732a5e017d5bd5c81158106ce249c \ + --hash=sha256:0d861473f743244d349b50f850d10eb87aeb22bbdcc8e64f79273c94af5a8226 \ + --hash=sha256:0e94703ec9684807f20cfb5eed95c70f67f2a8f21ad620146d7b5a13677b93e5 \ + --hash=sha256:0fa1aec2d32bcc03c8fa0f6f1712caad1adc38509f31142112e5c9daf5b9c833 \ + --hash=sha256:16b65ea0f2465b6fb52aa22de5eca612aa964ddfec00a912e26f4656cbef890b \ + --hash=sha256:16d10d789dd9bcca1173c95af82c58433122564b7bc39385124be735a35cbe99 \ + --hash=sha256:19ac87f93086ce37b86e098888555c4b4bc48102279bae3350098c0ed664b501 \ + --hash=sha256:1d22856ffbe153a602df38e4a5464f0b748a54002e0d69ac6d2ad0a197cc99ec \ + --hash=sha256:21e764fd1e70b6a3e205a0e46f3051701f98a8cb3fad66eeb80e48bb502f8698 \ + --hash=sha256:231ddcbb35e2ff8973e1365db41fe0572662893b99a05deb183b68ad4c0c8bd4 \ + --hash=sha256:253a4a220747e8b5faf57ec320c4f5efb0cef05f647420bf267143ec15dba10a \ + --hash=sha256:280081916dc341820640489a66e4696049401ef1cf6dd672f672e70ad915aca3 \ + --hash=sha256:2a441ea71902098ffe78c5abe6c494f44160b4af614ed16c3d9a3b1d17fd8ee2 \ + --hash=sha256:304b13570067b2547562e308af560b3963857b1fa90bd6afd978130130fe2d6a \ + --hash=sha256:32286a2c8d167e897177b673176c1e3e00d4057caf5d2b64eef9a3666b03018e \ + --hash=sha256:33bdcc2a32c0a0e861f60841a512c8acc658c87c2ac59d89e3a46dacf7d866e4 \ + --hash=sha256:375b83ed0aecfce76c16d198fbc21f3b11b337d68662bea0a995046682a11419 \ + --hash=sha256:3c09a49d6cde137258beb3d551994a2927fd35ad5cf96aed573f61bbd67c5f84 \ + --hash=sha256:3d92613ec25e43b05f042302531ec0f00b8445190e43325880cbd6ab7c2581da \ + --hash=sha256:40a126142a56b2dfc0aacbad1de8310cbf60da7656db0e6b16eebd48e3e93519 \ + --hash=sha256:416c229f77e5ea25b3dfd4b582f8d73d7e43c22320302b9ab128a2d3a0b38efe \ + --hash=sha256:432786d3561e69aeeae6c7e8648964ce0ad05736120135601f87ac26b9c83381 \ + --hash=sha256:43b9e366a31fdd1c87d0eb08f579b4a82b723ea54338f040d6b4e518a026ea29 \ + --hash=sha256:440eede837960000d74978f0eba527be106b5b9aee0daf779d395276ed0b0614 \ + --hash=sha256:45b0cc4e3556cd875e09102988d1ab8356c998b596c9fced84547c8138b487a0 \ + --hash=sha256:476743fe6dfe14a2da12e3ac79125dc84a3b2cf8094369a47a1529b0cd8549fe \ + --hash=sha256:4773092f8019072343a7447203308b176e10199920eb02d6195e81bbb3274c29 \ + --hash=sha256:4b3dac63058cc36820b0dd072f89898604e2d39686fe05321729d00d8ac185a0 \ + --hash=sha256:4d1c96a7a18b9690a4d46df09e3e3382406ae3213727cd1019ebade1c4a81917 \ + --hash=sha256:51307f5c71007673a2bf8232ad973483d281e74cb99c8c5a990af1eefa6277d9 \ + --hash=sha256:51447e9aa2684679af07ca5021c3db526e0284347ebf4ffcec1154c3350cfe32 \ + --hash=sha256:58150c9f9b9a552505912d182ccdf26f6396fb6094816ceebcbb20eecabaed94 \ + --hash=sha256:5b10cd92fc5c498b35a8635df6d5a100207f88b63a4dc1de7ef9a548e1e2cd63 \ + --hash=sha256:5e226f6218febc71f6c1fc2fafb91c226f75bdc1d8fb12d66823716e891608fd \ + --hash=sha256:609b3ba8fcc0fb5ab7af00719d0fb6ad0cb518e48e7712d12fd68f1327951198 \ + --hash=sha256:60f44ade2cf573dad7a277e6f8ca9a51a21dda572b13bd7d8539bb3cd5dbedde \ + --hash=sha256:611057cc5d5c0afc743ba8be6bd828c17e0aaa8643f9d0a9b9bb7dea80eb8012 \ + --hash=sha256:6366a16e1a25018694d6a5d784d09b046edc9eac40ea2b54065c3052672516a1 \ + --hash=sha256:65a7ff3f705e57d392f7261b6d0550fe137c3019477431f1c355e0db0a7d3e15 \ + --hash=sha256:673611bbd43f0810bec0b0f028ddeaaa501190339cac411f347ac76917c3ae7b \ + --hash=sha256:67830fc78e67501f47bb950471b2dcb9b35b140084429318e862895a8e89c993 \ + --hash=sha256:68ce9f4d6b26d5ccbf7fd4459bf75f74a0a146677ebba80597df60cbdb20e6f4 \ + --hash=sha256:68e5f26a1ad57ded6d1cfb85331d1c1a195314756471d97758c48498bb4dcdf5 \ + --hash=sha256:69b157c5d3292bcd443faca052f3096f637f1e074b98212a933c074ae23dc3b8 \ + --hash=sha256:75286256590a6320cf106a0d28970d3560aad9ee09aa7b34fb40524792436d35 \ + --hash=sha256:78841cccf1af7b40f6f716338d50c0902dbe88d9f800b3c973b7a9a0a693a642 \ + --hash=sha256:78fa18e436a1a0e58dbd7e02fc4473f3f32cceb12df9dfca542d075961c307d2 \ + --hash=sha256:79580094b00d1789d1f93ea55bc43cb2f611910c72235b7657f3482ddcc1b22d \ + --hash=sha256:7b86a2b16095d250c6f58b3d9b2eee6f4147754344f3dab0922f7c9bf7d226c9 \ + --hash=sha256:83aed2c10721ddd90f68140685391b50811a880af20654c59af6b6c66c40513c \ + --hash=sha256:84fd18bcc17526fc2b3c1af7d2b9217d32c9c04448c16ec693b9b4f1985c3d33 \ + --hash=sha256:871ff67ea1aad4dfd91736464934d56b32dac49f9fbe16cddba36198a7b3a0db \ + --hash=sha256:898f0e9068ca27d37f8e83a5b962821df851532e6c4a7d615c1c033f9da6eedf \ + --hash=sha256:8a79d9f4d8001473a30c163556b3c3bfebec837495a412dde78b51672f6134f9 \ + --hash=sha256:8c041122946b7ba21bb32c45b1aa57b1be35527690aeb3c5c234521085632eee \ + --hash=sha256:90c44bc373b7687f6948b693cceaea1348ae0975d7474746559494468e3c1d84 \ + --hash=sha256:9104ed0bd76a429d46f9ec0dbc9b08ad1d2dcdf2b00a5a0daa1c145329b35b44 \ + --hash=sha256:920079c3f7456fa213e0829ed2073aaa727fd39d889ead5b4f35d0de5460d04f \ + --hash=sha256:93d59d504b230e83c7a843251681959a0b6a9cd76f6e146ce1b8a80eb8739af9 \ + --hash=sha256:9b2aff1c7b3884512b9512c3eaadd9bab39fb45042ffaaa1dd08ff2b9f8109d9 \ + --hash=sha256:9b8e0f3107e2200b76f6054de99016eac3ee6762713587b36baaa7e4bd2ae177 \ + --hash=sha256:9bb41182d93ea91f60b4bc8fbf4c820c69ef8a12ab2d917f3f1834f1acad07e8 \ + --hash=sha256:9cdef90ae47919cae358d8ab15797a800ed41da7aba5d72419fb510729e2ed4b \ + --hash=sha256:a1786910334ed46ab1dd73222f2cd1e05c2c3bb39f6dddb4f8b36fc382058a39 \ + --hash=sha256:a4cfde78a9f2880208d16a93b795726a3017d5977e08d1e162a7a31322479c41 \ + --hash=sha256:a4fbdde9dd4a9ce5fd52c2b3a347bb50cc89483ef783f1cb00d408c13f7a96c0 \ + --hash=sha256:aa99adc8f081b475a12843953db36831eaf83ec33eb46a90629ca6a5de45a616 \ + --hash=sha256:ac351b3b8014eead140e77e9717e2992c6bbe30b63bc3422422eb84865412e3d \ + --hash=sha256:ad41ba96094304aa090f5a30cb6e4fb3b3f1c264c523394b4c39bbacc4dc92ba \ + --hash=sha256:b5314963fce9b0b12743891de876e724997864ee22aa496f903f426c7e2fa5b2 \ + --hash=sha256:bcf74c1df76758a395bf0af608c04c82257523f55c9868b334f06270d0f2112b \ + --hash=sha256:bd47ba7fc3ca94896759ea0109775132d3e7ab921fbf54038e1bab2e46c313c9 \ + --hash=sha256:c0323c9daef75ef2e5083624b4585018a0c9d5e3b40f607eed81a311270b934b \ + --hash=sha256:c1225416b463483160e4af85d5fc3a9690ccb53fd4b1865a6437825f5ede3209 \ + --hash=sha256:c1c948747b03be832dceed96ca815cef7360de9aa19d37c730f8e3f6101aca48 \ + --hash=sha256:c25fe15c70c59eb7c5ce8c06a1f3fa1da0ecc5ea1e7a5922c40fd2fa9b0d5046 \ + --hash=sha256:cc1b0fff8ead343dae06305f954eb8468ba0ec1a97881f42489d198e4ce3c632 \ + --hash=sha256:cd6280cf040f233bd7d3407b743b4b4c74f70e8e1c4199cb112a62c941c0772a \ + --hash=sha256:cd6c3d4b783c556fa00bf540854e42f135e2f256abd29669fcd0da0f2dec79c2 \ + --hash=sha256:d4d6fcde76f94f5cb9e43e9e9a61f16dacefd228cbbf6f1a09bd9b219a92f1a1 \ + --hash=sha256:ddf4af30b417d9fe16481e9b81c27ab2a7cde1ff7ba3e85653b02db7d145dc7b \ + --hash=sha256:df115d4d83168fdf2cae48ef1ff6d1cb4c466364e30861b37121de0f3bf1b990 \ + --hash=sha256:df7276909358e5635ae203673ab7e509ddd224225a8d6b0790bf13eb2bde1cc5 \ + --hash=sha256:e4fd89cc178bced6ad29cb3e6dd4aa63fa5017c3524dbd0b25998fb64a87cc8b \ + --hash=sha256:e9701d0049d92c16703a42771b98d560b95248949f23f8cf7b4eddd201814fb9 \ + --hash=sha256:ee2f2a527e3c1a6e6411eb4209642e138b544a2d72fe5d0d76daf77b24063534 \ + --hash=sha256:f7fb7d750cfa0a070d2c24e831fd3481019a60dd317ea2b39acbcebc08b6ed81 \ + --hash=sha256:f840ed6d8ecba8255df8c42b87fadeda98ddfc6eeec05e2dc66e26d46dd6f58a \ + --hash=sha256:f86c6358749bd4fda175388691e3ba8c46e24c5347d0afd20f9b7edfc9faf07d \ + --hash=sha256:fa36ec09ef71d158186bc79e359ff5fdd6e7996fe8ab638f00d6b93139ba4fcf \ + --hash=sha256:fe2c7201c642b7c308f1675355ad7ff7b66acfe3541625efe5a3ad38f29d6115 + # via requests +colorama==0.4.6 ; sys_platform == 'win32' \ + --hash=sha256:08695f5cb7ed6e0531a20572697297273c47b8cae5a63ffc6d6ed5c201be6e44 \ + --hash=sha256:4f1d9991f5acc0ca119f9d443620b77f9d6b33703e51011c16baf57afb285fc6 + # via sphinx +docutils==0.21.2 ; python_full_version < '3.11' \ + --hash=sha256:3a6b18732edf182daa3cd12775bbb338cf5691468f91eeeb109deff6ebfa986f \ + --hash=sha256:dafca5b9e384f0e419294eb4d2ff9fa826435bf15f15b7bd45723e8ad76811b2 + # via + # myst-parser + # sphinx + # sphinx-rtd-theme +docutils==0.22.4 ; python_full_version >= '3.11' \ + --hash=sha256:4db53b1fde9abecbb74d91230d32ab626d94f6badfc575d6db9194a49df29968 \ + --hash=sha256:d0013f540772d1420576855455d050a2180186c91c15779301ac2ccb3eeb68de + # via + # myst-parser + # sphinx + # sphinx-rtd-theme +idna==3.18 \ + --hash=sha256:7f952cbe720b688055e3f87de14f5c3e5fdaa8bc3928985c4077ca689de849a2 \ + --hash=sha256:ffb385a7e039654cef1ab9ef32c6fafe283c0c0467bba1d9029738ce4a14a848 + # via requests +imagesize==1.5.0 ; python_full_version < '3.10' \ + --hash=sha256:32677681b3f434c2cb496f00e89c5a291247b35b1f527589909e008057da5899 \ + --hash=sha256:8bfc5363a7f2133a89f0098451e0bcb1cd71aba4dc02bbcecb39d99d40e1b94f + # via sphinx +imagesize==2.0.0 ; python_full_version >= '3.10' \ + --hash=sha256:5667c5bbb57ab3f1fa4bc366f4fbc971db3d5ed011fd2715fd8001f782718d96 \ + --hash=sha256:8e8358c4a05c304f1fccf7ff96f036e7243a189e9e42e90851993c558cfe9ee3 + # via sphinx +importlib-metadata==8.7.1 ; python_full_version < '3.10' \ + --hash=sha256:49fef1ae6440c182052f407c8d34a68f72efc36db9ca90dc0113398f2fdde8bb \ + --hash=sha256:5a1f80bf1daa489495071efbb095d75a634cf28a8bc299581244063b53176151 + # via sphinx +jinja2==3.1.6 \ + --hash=sha256:0137fb05990d35f1275a587e9aee6d56da821fc83491a0fb838183be43f66d6d \ + --hash=sha256:85ece4451f492d0c13c5dd7c13a64681a86afae63a5f347908daf103ce6d2f67 + # via + # myst-parser + # readthedocs-sphinx-ext + # sphinx +markdown-it-py==3.0.0 ; python_full_version < '3.11' \ + --hash=sha256:355216845c60bd96232cd8d8c40e8f9765cc86f46880e43a8fd22dc1a1a8cab1 \ + --hash=sha256:e3f60a94fa066dc52ec76661e37c851cb232d92f9886b15cb560aaada2df8feb + # via + # mdit-py-plugins + # myst-parser +markdown-it-py==4.2.0 ; python_full_version >= '3.11' \ + --hash=sha256:04a21681d6fbb623de53f6f364d352309d4094dd4194040a10fd51833e418d49 \ + --hash=sha256:9f7ebbcd14fe59494226453aed97c1070d83f8d24b6fc3a3bcf9a38092641c4a + # via + # mdit-py-plugins + # myst-parser +markupsafe==3.0.3 \ + --hash=sha256:0303439a41979d9e74d18ff5e2dd8c43ed6c6001fd40e5bf2e43f7bd9bbc523f \ + --hash=sha256:068f375c472b3e7acbe2d5318dea141359e6900156b5b2ba06a30b169086b91a \ + --hash=sha256:0bf2a864d67e76e5c9a34dc26ec616a66b9888e25e7b9460e1c76d3293bd9dbf \ + --hash=sha256:0db14f5dafddbb6d9208827849fad01f1a2609380add406671a26386cdf15a19 \ + --hash=sha256:0eb9ff8191e8498cca014656ae6b8d61f39da5f95b488805da4bb029cccbfbaf \ + --hash=sha256:0f4b68347f8c5eab4a13419215bdfd7f8c9b19f2b25520968adfad23eb0ce60c \ + --hash=sha256:1085e7fbddd3be5f89cc898938f42c0b3c711fdcb37d75221de2666af647c175 \ + --hash=sha256:116bb52f642a37c115f517494ea5feb03889e04df47eeff5b130b1808ce7c219 \ + --hash=sha256:12c63dfb4a98206f045aa9563db46507995f7ef6d83b2f68eda65c307c6829eb \ + --hash=sha256:133a43e73a802c5562be9bbcd03d090aa5a1fe899db609c29e8c8d815c5f6de6 \ + --hash=sha256:1353ef0c1b138e1907ae78e2f6c63ff67501122006b0f9abad68fda5f4ffc6ab \ + --hash=sha256:15d939a21d546304880945ca1ecb8a039db6b4dc49b2c5a400387cdae6a62e26 \ + --hash=sha256:177b5253b2834fe3678cb4a5f0059808258584c559193998be2601324fdeafb1 \ + --hash=sha256:1872df69a4de6aead3491198eaf13810b565bdbeec3ae2dc8780f14458ec73ce \ + --hash=sha256:1b4b79e8ebf6b55351f0d91fe80f893b4743f104bff22e90697db1590e47a218 \ + --hash=sha256:1b52b4fb9df4eb9ae465f8d0c228a00624de2334f216f178a995ccdcf82c4634 \ + --hash=sha256:1ba88449deb3de88bd40044603fafffb7bc2b055d626a330323a9ed736661695 \ + --hash=sha256:1cc7ea17a6824959616c525620e387f6dd30fec8cb44f649e31712db02123dad \ + --hash=sha256:218551f6df4868a8d527e3062d0fb968682fe92054e89978594c28e642c43a73 \ + --hash=sha256:26a5784ded40c9e318cfc2bdb30fe164bdb8665ded9cd64d500a34fb42067b1c \ + --hash=sha256:2713baf880df847f2bece4230d4d094280f4e67b1e813eec43b4c0e144a34ffe \ + --hash=sha256:2a15a08b17dd94c53a1da0438822d70ebcd13f8c3a95abe3a9ef9f11a94830aa \ + --hash=sha256:2f981d352f04553a7171b8e44369f2af4055f888dfb147d55e42d29e29e74559 \ + --hash=sha256:32001d6a8fc98c8cb5c947787c5d08b0a50663d139f1305bac5885d98d9b40fa \ + --hash=sha256:3524b778fe5cfb3452a09d31e7b5adefeea8c5be1d43c4f810ba09f2ceb29d37 \ + --hash=sha256:3537e01efc9d4dccdf77221fb1cb3b8e1a38d5428920e0657ce299b20324d758 \ + --hash=sha256:35add3b638a5d900e807944a078b51922212fb3dedb01633a8defc4b01a3c85f \ + --hash=sha256:38664109c14ffc9e7437e86b4dceb442b0096dfe3541d7864d9cbe1da4cf36c8 \ + --hash=sha256:3a7e8ae81ae39e62a41ec302f972ba6ae23a5c5396c8e60113e9066ef893da0d \ + --hash=sha256:3b562dd9e9ea93f13d53989d23a7e775fdfd1066c33494ff43f5418bc8c58a5c \ + --hash=sha256:457a69a9577064c05a97c41f4e65148652db078a3a509039e64d3467b9e7ef97 \ + --hash=sha256:4bd4cd07944443f5a265608cc6aab442e4f74dff8088b0dfc8238647b8f6ae9a \ + --hash=sha256:4e885a3d1efa2eadc93c894a21770e4bc67899e3543680313b09f139e149ab19 \ + --hash=sha256:4faffd047e07c38848ce017e8725090413cd80cbc23d86e55c587bf979e579c9 \ + --hash=sha256:509fa21c6deb7a7a273d629cf5ec029bc209d1a51178615ddf718f5918992ab9 \ + --hash=sha256:5678211cb9333a6468fb8d8be0305520aa073f50d17f089b5b4b477ea6e67fdc \ + --hash=sha256:591ae9f2a647529ca990bc681daebdd52c8791ff06c2bfa05b65163e28102ef2 \ + --hash=sha256:5a7d5dc5140555cf21a6fefbdbf8723f06fcd2f63ef108f2854de715e4422cb4 \ + --hash=sha256:69c0b73548bc525c8cb9a251cddf1931d1db4d2258e9599c28c07ef3580ef354 \ + --hash=sha256:6b5420a1d9450023228968e7e6a9ce57f65d148ab56d2313fcd589eee96a7a50 \ + --hash=sha256:722695808f4b6457b320fdc131280796bdceb04ab50fe1795cd540799ebe1698 \ + --hash=sha256:729586769a26dbceff69f7a7dbbf59ab6572b99d94576a5592625d5b411576b9 \ + --hash=sha256:77f0643abe7495da77fb436f50f8dab76dbc6e5fd25d39589a0f1fe6548bfa2b \ + --hash=sha256:795e7751525cae078558e679d646ae45574b47ed6e7771863fcc079a6171a0fc \ + --hash=sha256:7be7b61bb172e1ed687f1754f8e7484f1c8019780f6f6b0786e76bb01c2ae115 \ + --hash=sha256:7c3fb7d25180895632e5d3148dbdc29ea38ccb7fd210aa27acbd1201a1902c6e \ + --hash=sha256:7e68f88e5b8799aa49c85cd116c932a1ac15caaa3f5db09087854d218359e485 \ + --hash=sha256:83891d0e9fb81a825d9a6d61e3f07550ca70a076484292a70fde82c4b807286f \ + --hash=sha256:8485f406a96febb5140bfeca44a73e3ce5116b2501ac54fe953e488fb1d03b12 \ + --hash=sha256:8709b08f4a89aa7586de0aadc8da56180242ee0ada3999749b183aa23df95025 \ + --hash=sha256:8f71bc33915be5186016f675cd83a1e08523649b0e33efdb898db577ef5bb009 \ + --hash=sha256:915c04ba3851909ce68ccc2b8e2cd691618c4dc4c4232fb7982bca3f41fd8c3d \ + --hash=sha256:949b8d66bc381ee8b007cd945914c721d9aba8e27f71959d750a46f7c282b20b \ + --hash=sha256:94c6f0bb423f739146aec64595853541634bde58b2135f27f61c1ffd1cd4d16a \ + --hash=sha256:9a1abfdc021a164803f4d485104931fb8f8c1efd55bc6b748d2f5774e78b62c5 \ + --hash=sha256:9b79b7a16f7fedff2495d684f2b59b0457c3b493778c9eed31111be64d58279f \ + --hash=sha256:a320721ab5a1aba0a233739394eb907f8c8da5c98c9181d1161e77a0c8e36f2d \ + --hash=sha256:a4afe79fb3de0b7097d81da19090f4df4f8d3a2b3adaa8764138aac2e44f3af1 \ + --hash=sha256:ad2cf8aa28b8c020ab2fc8287b0f823d0a7d8630784c31e9ee5edea20f406287 \ + --hash=sha256:b8512a91625c9b3da6f127803b166b629725e68af71f8184ae7e7d54686a56d6 \ + --hash=sha256:bc51efed119bc9cfdf792cdeaa4d67e8f6fcccab66ed4bfdd6bde3e59bfcbb2f \ + --hash=sha256:bdc919ead48f234740ad807933cdf545180bfbe9342c2bb451556db2ed958581 \ + --hash=sha256:bdd37121970bfd8be76c5fb069c7751683bdf373db1ed6c010162b2a130248ed \ + --hash=sha256:be8813b57049a7dc738189df53d69395eba14fb99345e0a5994914a3864c8a4b \ + --hash=sha256:c0c0b3ade1c0b13b936d7970b1d37a57acde9199dc2aecc4c336773e1d86049c \ + --hash=sha256:c47a551199eb8eb2121d4f0f15ae0f923d31350ab9280078d1e5f12b249e0026 \ + --hash=sha256:c4ffb7ebf07cfe8931028e3e4c85f0357459a3f9f9490886198848f4fa002ec8 \ + --hash=sha256:ccfcd093f13f0f0b7fdd0f198b90053bf7b2f02a3927a30e63f3ccc9df56b676 \ + --hash=sha256:d2ee202e79d8ed691ceebae8e0486bd9a2cd4794cec4824e1c99b6f5009502f6 \ + --hash=sha256:d53197da72cc091b024dd97249dfc7794d6a56530370992a5e1a08983ad9230e \ + --hash=sha256:d6dd0be5b5b189d31db7cda48b91d7e0a9795f31430b7f271219ab30f1d3ac9d \ + --hash=sha256:d88b440e37a16e651bda4c7c2b930eb586fd15ca7406cb39e211fcff3bf3017d \ + --hash=sha256:de8a88e63464af587c950061a5e6a67d3632e36df62b986892331d4620a35c01 \ + --hash=sha256:df2449253ef108a379b8b5d6b43f4b1a8e81a061d6537becd5582fba5f9196d7 \ + --hash=sha256:e1c1493fb6e50ab01d20a22826e57520f1284df32f2d8601fdd90b6304601419 \ + --hash=sha256:e1cf1972137e83c5d4c136c43ced9ac51d0e124706ee1c8aa8532c1287fa8795 \ + --hash=sha256:e2103a929dfa2fcaf9bb4e7c091983a49c9ac3b19c9061b6d5427dd7d14d81a1 \ + --hash=sha256:e56b7d45a839a697b5eb268c82a71bd8c7f6c94d6fd50c3d577fa39a9f1409f5 \ + --hash=sha256:e8afc3f2ccfa24215f8cb28dcf43f0113ac3c37c2f0f0806d8c70e4228c5cf4d \ + --hash=sha256:e8fc20152abba6b83724d7ff268c249fa196d8259ff481f3b1476383f8f24e42 \ + --hash=sha256:eaa9599de571d72e2daf60164784109f19978b327a3910d3e9de8c97b5b70cfe \ + --hash=sha256:ec15a59cf5af7be74194f7ab02d0f59a62bdcf1a537677ce67a2537c9b87fcda \ + --hash=sha256:f190daf01f13c72eac4efd5c430a8de82489d9cff23c364c3ea822545032993e \ + --hash=sha256:f34c41761022dd093b4b6896d4810782ffbabe30f2d443ff5f083e0cbbb8c737 \ + --hash=sha256:f3e98bb3798ead92273dc0e5fd0f31ade220f59a266ffd8a4f6065e0a3ce0523 \ + --hash=sha256:f42d0984e947b8adf7dd6dde396e720934d12c506ce84eea8476409563607591 \ + --hash=sha256:f71a396b3bf33ecaa1626c255855702aca4d3d9fea5e051b41ac59a9c1c41edc \ + --hash=sha256:f9e130248f4462aaa8e2552d547f36ddadbeaa573879158d721bbd33dfe4743a \ + --hash=sha256:fed51ac40f757d41b7c48425901843666a6677e3e8eb0abcff09e4ba6e664f50 + # via + # rules-python-sphinxdocs-dev (dev/pyproject.toml) + # jinja2 +mdit-py-plugins==0.4.2 ; python_full_version < '3.10' \ + --hash=sha256:0c673c3f889399a33b95e88d2f0d111b4447bdfea7f237dab2d488f459835636 \ + --hash=sha256:5f2cd1fdb606ddf152d37ec30e46101a60512bc0e5fa1a7002c36647b09e26b5 + # via myst-parser +mdit-py-plugins==0.6.1 ; python_full_version >= '3.10' \ + --hash=sha256:214c82fb2ac524472ab6a5bcab1de80f73b50443e187f401bfd77efbc7c6481d \ + --hash=sha256:a2bca0f039f39dbd35fb74ae1b5f998608c437463371f0ff7f49a19a17a114d0 + # via myst-parser +mdurl==0.1.2 \ + --hash=sha256:84008a41e51615a49fc9966191ff91509e3c40b939176e643fd50a5c2196b8f8 \ + --hash=sha256:bb413d29f5eea38f31dd4754dd7377d4465116fb207585f97bf925588687c1ba + # via markdown-it-py +myst-parser==3.0.1 ; python_full_version < '3.10' \ + --hash=sha256:6457aaa33a5d474aca678b8ead9b3dc298e89c68e67012e73146ea6fd54babf1 \ + --hash=sha256:88f0cb406cb363b077d176b51c476f62d60604d68a8dcdf4832e080441301a87 + # via rules-python-sphinxdocs-dev (dev/pyproject.toml) +myst-parser==4.0.1 ; python_full_version == '3.10.*' \ + --hash=sha256:5cfea715e4f3574138aecbf7d54132296bfd72bb614d31168f48c477a830a7c4 \ + --hash=sha256:9134e88959ec3b5780aedf8a99680ea242869d012e8821db3126d427edc9c95d + # via rules-python-sphinxdocs-dev (dev/pyproject.toml) +myst-parser==5.1.0 ; python_full_version >= '3.11' \ + --hash=sha256:9c91c52b3cdb4d94a6506e4fab4e2f296c7623a0da0dcbe6de1565c3dad67a8a \ + --hash=sha256:ab69322dc6719dcc7f296479dbb70181b66df6ed315064f92dbc85c0e1bf2f02 + # via rules-python-sphinxdocs-dev (dev/pyproject.toml) +packaging==26.2 \ + --hash=sha256:5fc45236b9446107ff2415ce77c807cee2862cb6fac22b8a73826d0693b0980e \ + --hash=sha256:ff452ff5a3e828ce110190feff1178bb1f2ea2281fa2075aadb987c2fb221661 + # via + # readthedocs-sphinx-ext + # sphinx +pygments==2.20.0 \ + --hash=sha256:6757cd03768053ff99f3039c1a36d6c0aa0b263438fcab17520b30a303a82b5f \ + --hash=sha256:81a9e26dd42fd28a23a2d169d86d7ac03b46e2f8b59ed4698fb4785f946d0176 + # via sphinx +pyyaml==6.0.3 \ + --hash=sha256:00c4bdeba853cc34e7dd471f16b4114f4162dc03e6b7afcc2128711f0eca823c \ + --hash=sha256:0150219816b6a1fa26fb4699fb7daa9caf09eb1999f3b70fb6e786805e80375a \ + --hash=sha256:02893d100e99e03eda1c8fd5c441d8c60103fd175728e23e431db1b589cf5ab3 \ + --hash=sha256:02ea2dfa234451bbb8772601d7b8e426c2bfa197136796224e50e35a78777956 \ + --hash=sha256:0f29edc409a6392443abf94b9cf89ce99889a1dd5376d94316ae5145dfedd5d6 \ + --hash=sha256:10892704fc220243f5305762e276552a0395f7beb4dbf9b14ec8fd43b57f126c \ + --hash=sha256:16249ee61e95f858e83976573de0f5b2893b3677ba71c9dd36b9cf8be9ac6d65 \ + --hash=sha256:1d37d57ad971609cf3c53ba6a7e365e40660e3be0e5175fa9f2365a379d6095a \ + --hash=sha256:1ebe39cb5fc479422b83de611d14e2c0d3bb2a18bbcb01f229ab3cfbd8fee7a0 \ + --hash=sha256:214ed4befebe12df36bcc8bc2b64b396ca31be9304b8f59e25c11cf94a4c033b \ + --hash=sha256:2283a07e2c21a2aa78d9c4442724ec1eb15f5e42a723b99cb3d822d48f5f7ad1 \ + --hash=sha256:22ba7cfcad58ef3ecddc7ed1db3409af68d023b7f940da23c6c2a1890976eda6 \ + --hash=sha256:27c0abcb4a5dac13684a37f76e701e054692a9b2d3064b70f5e4eb54810553d7 \ + --hash=sha256:28c8d926f98f432f88adc23edf2e6d4921ac26fb084b028c733d01868d19007e \ + --hash=sha256:2e71d11abed7344e42a8849600193d15b6def118602c4c176f748e4583246007 \ + --hash=sha256:34d5fcd24b8445fadc33f9cf348c1047101756fd760b4dacb5c3e99755703310 \ + --hash=sha256:37503bfbfc9d2c40b344d06b2199cf0e96e97957ab1c1b546fd4f87e53e5d3e4 \ + --hash=sha256:3c5677e12444c15717b902a5798264fa7909e41153cdf9ef7ad571b704a63dd9 \ + --hash=sha256:3ff07ec89bae51176c0549bc4c63aa6202991da2d9a6129d7aef7f1407d3f295 \ + --hash=sha256:41715c910c881bc081f1e8872880d3c650acf13dfa8214bad49ed4cede7c34ea \ + --hash=sha256:418cf3f2111bc80e0933b2cd8cd04f286338bb88bdc7bc8e6dd775ebde60b5e0 \ + --hash=sha256:44edc647873928551a01e7a563d7452ccdebee747728c1080d881d68af7b997e \ + --hash=sha256:4a2e8cebe2ff6ab7d1050ecd59c25d4c8bd7e6f400f5f82b96557ac0abafd0ac \ + --hash=sha256:4ad1906908f2f5ae4e5a8ddfce73c320c2a1429ec52eafd27138b7f1cbe341c9 \ + --hash=sha256:501a031947e3a9025ed4405a168e6ef5ae3126c59f90ce0cd6f2bfc477be31b7 \ + --hash=sha256:5190d403f121660ce8d1d2c1bb2ef1bd05b5f68533fc5c2ea899bd15f4399b35 \ + --hash=sha256:5498cd1645aa724a7c71c8f378eb29ebe23da2fc0d7a08071d89469bf1d2defb \ + --hash=sha256:5cf4e27da7e3fbed4d6c3d8e797387aaad68102272f8f9752883bc32d61cb87b \ + --hash=sha256:5e0b74767e5f8c593e8c9b5912019159ed0533c70051e9cce3e8b6aa699fcd69 \ + --hash=sha256:5ed875a24292240029e4483f9d4a4b8a1ae08843b9c54f43fcc11e404532a8a5 \ + --hash=sha256:5fcd34e47f6e0b794d17de1b4ff496c00986e1c83f7ab2fb8fcfe9616ff7477b \ + --hash=sha256:5fdec68f91a0c6739b380c83b951e2c72ac0197ace422360e6d5a959d8d97b2c \ + --hash=sha256:6344df0d5755a2c9a276d4473ae6b90647e216ab4757f8426893b5dd2ac3f369 \ + --hash=sha256:64386e5e707d03a7e172c0701abfb7e10f0fb753ee1d773128192742712a98fd \ + --hash=sha256:652cb6edd41e718550aad172851962662ff2681490a8a711af6a4d288dd96824 \ + --hash=sha256:66291b10affd76d76f54fad28e22e51719ef9ba22b29e1d7d03d6777a9174198 \ + --hash=sha256:66e1674c3ef6f541c35191caae2d429b967b99e02040f5ba928632d9a7f0f065 \ + --hash=sha256:6adc77889b628398debc7b65c073bcb99c4a0237b248cacaf3fe8a557563ef6c \ + --hash=sha256:79005a0d97d5ddabfeeea4cf676af11e647e41d81c9a7722a193022accdb6b7c \ + --hash=sha256:7c6610def4f163542a622a73fb39f534f8c101d690126992300bf3207eab9764 \ + --hash=sha256:7f047e29dcae44602496db43be01ad42fc6f1cc0d8cd6c83d342306c32270196 \ + --hash=sha256:8098f252adfa6c80ab48096053f512f2321f0b998f98150cea9bd23d83e1467b \ + --hash=sha256:850774a7879607d3a6f50d36d04f00ee69e7fc816450e5f7e58d7f17f1ae5c00 \ + --hash=sha256:8d1fab6bb153a416f9aeb4b8763bc0f22a5586065f86f7664fc23339fc1c1fac \ + --hash=sha256:8da9669d359f02c0b91ccc01cac4a67f16afec0dac22c2ad09f46bee0697eba8 \ + --hash=sha256:8dc52c23056b9ddd46818a57b78404882310fb473d63f17b07d5c40421e47f8e \ + --hash=sha256:9149cad251584d5fb4981be1ecde53a1ca46c891a79788c0df828d2f166bda28 \ + --hash=sha256:93dda82c9c22deb0a405ea4dc5f2d0cda384168e466364dec6255b293923b2f3 \ + --hash=sha256:96b533f0e99f6579b3d4d4995707cf36df9100d67e0c8303a0c55b27b5f99bc5 \ + --hash=sha256:9c57bb8c96f6d1808c030b1687b9b5fb476abaa47f0db9c0101f5e9f394e97f4 \ + --hash=sha256:9c7708761fccb9397fe64bbc0395abcae8c4bf7b0eac081e12b809bf47700d0b \ + --hash=sha256:9f3bfb4965eb874431221a3ff3fdcddc7e74e3b07799e0e84ca4a0f867d449bf \ + --hash=sha256:a33284e20b78bd4a18c8c2282d549d10bc8408a2a7ff57653c0cf0b9be0afce5 \ + --hash=sha256:a80cb027f6b349846a3bf6d73b5e95e782175e52f22108cfa17876aaeff93702 \ + --hash=sha256:b30236e45cf30d2b8e7b3e85881719e98507abed1011bf463a8fa23e9c3e98a8 \ + --hash=sha256:b3bc83488de33889877a0f2543ade9f70c67d66d9ebb4ac959502e12de895788 \ + --hash=sha256:b865addae83924361678b652338317d1bd7e79b1f4596f96b96c77a5a34b34da \ + --hash=sha256:b8bb0864c5a28024fac8a632c443c87c5aa6f215c0b126c449ae1a150412f31d \ + --hash=sha256:ba1cc08a7ccde2d2ec775841541641e4548226580ab850948cbfda66a1befcdc \ + --hash=sha256:bdb2c67c6c1390b63c6ff89f210c8fd09d9a1217a465701eac7316313c915e4c \ + --hash=sha256:c1ff362665ae507275af2853520967820d9124984e0f7466736aea23d8611fba \ + --hash=sha256:c2514fceb77bc5e7a2f7adfaa1feb2fb311607c9cb518dbc378688ec73d8292f \ + --hash=sha256:c3355370a2c156cffb25e876646f149d5d68f5e0a3ce86a5084dd0b64a994917 \ + --hash=sha256:c458b6d084f9b935061bc36216e8a69a7e293a2f1e68bf956dcd9e6cbcd143f5 \ + --hash=sha256:d0eae10f8159e8fdad514efdc92d74fd8d682c933a6dd088030f3834bc8e6b26 \ + --hash=sha256:d76623373421df22fb4cf8817020cbb7ef15c725b9d5e45f17e189bfc384190f \ + --hash=sha256:ebc55a14a21cb14062aa4162f906cd962b28e2e9ea38f9b4391244cd8de4ae0b \ + --hash=sha256:eda16858a3cab07b80edaf74336ece1f986ba330fdb8ee0d6c0d68fe82bc96be \ + --hash=sha256:ee2922902c45ae8ccada2c5b501ab86c36525b883eff4255313a253a3160861c \ + --hash=sha256:efd7b85f94a6f21e4932043973a7ba2613b059c4a000551892ac9f1d11f5baf3 \ + --hash=sha256:f7057c9a337546edc7973c0d3ba84ddcdf0daa14533c2065749c9075001090e6 \ + --hash=sha256:fa160448684b4e94d80416c0fa4aac48967a969efe22931448d853ada8baf926 \ + --hash=sha256:fc09d0aa354569bc501d4e787133afc08552722d3ab34836a80547331bb5d4a0 + # via myst-parser +readthedocs-sphinx-ext==2.2.5 \ + --hash=sha256:ee5fd5b99db9f0c180b2396cbce528aa36671951b9526bb0272dbfce5517bd27 \ + --hash=sha256:f8c56184ea011c972dd45a90122568587cc85b0127bc9cf064d17c68bc809daa + # via rules-python-sphinxdocs-dev (dev/pyproject.toml) +requests==2.32.5 ; python_full_version < '3.10' \ + --hash=sha256:2462f94637a34fd532264295e186976db0f5d453d1cdd31473c85a6a161affb6 \ + --hash=sha256:dbba0bac56e100853db0ea71b82b4dfd5fe2bf6d3754a8893c3af500cec7d7cf + # via + # readthedocs-sphinx-ext + # sphinx +requests==2.34.2 ; python_full_version >= '3.10' \ + --hash=sha256:2a0d60c172f83ac6ab31e4554906c0f3b3588d37b5cb939b1c061f4907e278e0 \ + --hash=sha256:f288924cae4e29463698d6d60bc6a4da69c89185ad1e0bcc4104f584e960b9ed + # via + # readthedocs-sphinx-ext + # sphinx +roman-numerals==4.1.0 ; python_full_version >= '3.11' \ + --hash=sha256:1af8b147eb1405d5839e78aeb93131690495fe9da5c91856cb33ad55a7f1e5b2 \ + --hash=sha256:647ba99caddc2cc1e55a51e4360689115551bf4476d90e8162cf8c345fe233c7 + # via sphinx +snowballstemmer==3.1.1 \ + --hash=sha256:7e207fa178741da09cdee59d3ecec3827ad5f92b1fc5c9ff3755b639f71f5752 \ + --hash=sha256:e07bbc54a0d798fe6010a12398422e62a8bfbba95c394fd0956ef58cb4d3e260 + # via sphinx +sphinx==7.4.7 ; python_full_version < '3.10' \ + --hash=sha256:242f92a7ea7e6c5b406fdc2615413890ba9f699114a9c09192d7dfead2ee9cfe \ + --hash=sha256:c2419e2135d11f1951cd994d6eb18a1835bd8fdd8429f9ca375dc1f3281bd239 + # via + # rules-python-sphinxdocs-dev (dev/pyproject.toml) + # myst-parser + # sphinx-reredirects + # sphinx-rtd-theme + # sphinxcontrib-jquery +sphinx==8.1.3 ; python_full_version == '3.10.*' \ + --hash=sha256:09719015511837b76bf6e03e42eb7595ac8c2e41eeb9c29c5b755c6b677992a2 \ + --hash=sha256:43c1911eecb0d3e161ad78611bc905d1ad0e523e4ddc202a58a821773dc4c927 + # via + # rules-python-sphinxdocs-dev (dev/pyproject.toml) + # myst-parser + # sphinx-reredirects + # sphinx-rtd-theme + # sphinxcontrib-jquery +sphinx==9.0.4 ; python_full_version == '3.11.*' \ + --hash=sha256:594ef59d042972abbc581d8baa577404abe4e6c3b04ef61bd7fc2acbd51f3fa3 \ + --hash=sha256:5bebc595a5e943ea248b99c13814c1c5e10b3ece718976824ffa7959ff95fffb + # via + # rules-python-sphinxdocs-dev (dev/pyproject.toml) + # myst-parser + # sphinx-reredirects + # sphinx-rtd-theme + # sphinxcontrib-jquery +sphinx==9.1.0 ; python_full_version >= '3.12' \ + --hash=sha256:7741722357dd75f8190766926071fed3bdc211c74dd2d7d4df5404da95930ddb \ + --hash=sha256:c84fdd4e782504495fe4f2c0b3413d6c2bf388589bb352d439b2a3bb99991978 + # via + # rules-python-sphinxdocs-dev (dev/pyproject.toml) + # myst-parser + # sphinx-reredirects + # sphinx-rtd-theme + # sphinxcontrib-jquery +sphinx-autodoc2==0.5.0 \ + --hash=sha256:7d76044aa81d6af74447080182b6868c7eb066874edc835e8ddf810735b6565a \ + --hash=sha256:e867013b1512f9d6d7e6f6799f8b537d6884462acd118ef361f3f619a60b5c9e + # via rules-python-sphinxdocs-dev (dev/pyproject.toml) +sphinx-reredirects==0.1.6 ; python_full_version < '3.11' \ + --hash=sha256:c491cba545f67be9697508727818d8626626366245ae64456fe29f37e9bbea64 \ + --hash=sha256:efd50c766fbc5bf40cd5148e10c00f2c00d143027de5c5e48beece93cc40eeea + # via rules-python-sphinxdocs-dev (dev/pyproject.toml) +sphinx-reredirects==1.1.0 ; python_full_version >= '3.11' \ + --hash=sha256:4b5692273c72cd2d4d917f4c6f87d5919e4d6114a752d4be033f7f5f6310efd9 \ + --hash=sha256:fb9b195335ab14b43f8273287d0c7eeb637ba6c56c66581c11b47202f6718b29 + # via rules-python-sphinxdocs-dev (dev/pyproject.toml) +sphinx-rtd-theme==3.1.0 \ + --hash=sha256:1785824ae8e6632060490f67cf3a72d404a85d2d9fc26bce3619944de5682b89 \ + --hash=sha256:b44276f2c276e909239a4f6c955aa667aaafeb78597923b1c60babc76db78e4c + # via rules-python-sphinxdocs-dev (dev/pyproject.toml) +sphinxcontrib-applehelp==2.0.0 \ + --hash=sha256:2f29ef331735ce958efa4734873f084941970894c6090408b079c61b2e1c06d1 \ + --hash=sha256:4cd3f0ec4ac5dd9c17ec65e9ab272c9b867ea77425228e68ecf08d6b28ddbdb5 + # via sphinx +sphinxcontrib-devhelp==2.0.0 \ + --hash=sha256:411f5d96d445d1d73bb5d52133377b4248ec79db5c793ce7dbe59e074b4dd1ad \ + --hash=sha256:aefb8b83854e4b0998877524d1029fd3e6879210422ee3780459e28a1f03a8a2 + # via sphinx +sphinxcontrib-htmlhelp==2.1.0 \ + --hash=sha256:166759820b47002d22914d64a075ce08f4c46818e17cfc9470a9786b759b19f8 \ + --hash=sha256:c9e2916ace8aad64cc13a0d233ee22317f2b9025b9cf3295249fa985cc7082e9 + # via sphinx +sphinxcontrib-jquery==4.1 \ + --hash=sha256:1620739f04e36a2c779f1a131a2dfd49b2fd07351bf1968ced074365933abc7a \ + --hash=sha256:f936030d7d0147dd026a4f2b5a57343d233f1fc7b363f68b3d4f1cb0993878ae + # via sphinx-rtd-theme +sphinxcontrib-jsmath==1.0.1 \ + --hash=sha256:2ec2eaebfb78f3f2078e73666b1415417a116cc848b72e5172e596c871103178 \ + --hash=sha256:a9925e4a4587247ed2191a22df5f6970656cb8ca2bd6284309578f2153e0c4b8 + # via sphinx +sphinxcontrib-qthelp==2.0.0 \ + --hash=sha256:4fe7d0ac8fc171045be623aba3e2a8f613f8682731f9153bb2e40ece16b9bbab \ + --hash=sha256:b18a828cdba941ccd6ee8445dbe72ffa3ef8cbe7505d8cd1fa0d42d3f2d5f3eb + # via sphinx +sphinxcontrib-serializinghtml==2.0.0 \ + --hash=sha256:6e2cb0eef194e10c27ec0023bfeb25badbbb5868244cf5bc5bdc04e4464bf331 \ + --hash=sha256:e9d912827f872c029017a53f0ef2180b327c3f7fd23c87229f7a8e8b70031d4d + # via sphinx +tomli==2.4.1 ; python_full_version < '3.11' \ + --hash=sha256:01f520d4f53ef97964a240a035ec2a869fe1a37dde002b57ebc4417a27ccd853 \ + --hash=sha256:0d85819802132122da43cb86656f8d1f8c6587d54ae7dcaf30e90533028b49fe \ + --hash=sha256:136443dbd7e1dee43c68ac2694fde36b2849865fa258d39bf822c10e8068eac5 \ + --hash=sha256:1d8591993e228b0c930c4bb0db464bdad97b3289fb981255d6c9a41aedc84b2d \ + --hash=sha256:2190f2e9dd7508d2a90ded5ed369255980a1bcdd58e52f7fe24b8162bf9fedbd \ + --hash=sha256:2c1c351919aca02858f740c6d33adea0c5deea37f9ecca1cc1ef9e884a619d26 \ + --hash=sha256:36d2bd2ad5fb9eaddba5226aa02c8ec3fa4f192631e347b3ed28186d43be6b54 \ + --hash=sha256:3d48a93ee1c9b79c04bb38772ee1b64dcf18ff43085896ea460ca8dec96f35f6 \ + --hash=sha256:47149d5bd38761ac8be13a84864bf0b7b70bc051806bc3669ab1cbc56216b23c \ + --hash=sha256:4ab97e64ccda8756376892c53a72bd1f964e519c77236368527f758fbc36a53a \ + --hash=sha256:4b605484e43cdc43f0954ddae319fb75f04cc10dd80d830540060ee7cd0243cd \ + --hash=sha256:504aa796fe0569bb43171066009ead363de03675276d2d121ac1a4572397870f \ + --hash=sha256:51529d40e3ca50046d7606fa99ce3956a617f9b36380da3b7f0dd3dd28e68cb5 \ + --hash=sha256:52c8ef851d9a240f11a88c003eacb03c31fc1c9c4ec64a99a0f922b93874fda9 \ + --hash=sha256:559db847dc486944896521f68d8190be1c9e719fced785720d2216fe7022b662 \ + --hash=sha256:5a881ab208c0baf688221f8cecc5401bd291d67e38a1ac884d6736cbcd8247e9 \ + --hash=sha256:5cb41aa38891e073ee49d55fbc7839cfdb2bc0e600add13874d048c94aadddd1 \ + --hash=sha256:5e262d41726bc187e69af7825504c933b6794dc3fbd5945e41a79bb14c31f585 \ + --hash=sha256:5ee18d9ebdb417e384b58fe414e8d6af9f4e7a0ae761519fb50f721de398dd4e \ + --hash=sha256:7008df2e7655c495dd12d2a4ad038ff878d4ca4b81fccaf82b714e07eae4402c \ + --hash=sha256:734e20b57ba95624ecf1841e72b53f6e186355e216e5412de414e3c51e5e3c41 \ + --hash=sha256:7c7e1a961a0b2f2472c1ac5b69affa0ae1132c39adcb67aba98568702b9cc23f \ + --hash=sha256:7f86fd587c4ed9dd76f318225e7d9b29cfc5a9d43de44e5754db8d1128487085 \ + --hash=sha256:7f94b27a62cfad8496c8d2513e1a222dd446f095fca8987fceef261225538a15 \ + --hash=sha256:88dceee75c2c63af144e456745e10101eb67361050196b0b6af5d717254dddf7 \ + --hash=sha256:8a650c2dbafa08d42e51ba0b62740dae4ecb9338eefa093aa5c78ceb546fcd5c \ + --hash=sha256:8d65a2fbf9d2f8352685bc1364177ee3923d6baf5e7f43ea4959d7d8bc326a36 \ + --hash=sha256:96481a5786729fd470164b47cdb3e0e58062a496f455ee41b4403be77cb5a076 \ + --hash=sha256:a120733b01c45e9a0c34aeef92bf0cf1d56cfe81ed9d47d562f9ed591a9828ac \ + --hash=sha256:b1d22e6e9387bf4739fbe23bfa80e93f6b0373a7f1b96c6227c32bef95a4d7a8 \ + --hash=sha256:b8c198f8c1805dc42708689ed6864951fd2494f924149d3e4bce7710f8eb5232 \ + --hash=sha256:c2541745709bad0264b7d4705ad453b76ccd191e64aa6f0fc66b69a293a45ece \ + --hash=sha256:c742f741d58a28940ce01d58f0ab2ea3ced8b12402f162f4d534dfe18ba1cd6a \ + --hash=sha256:c7f2c7f2b9ca6bdeef8f0fa897f8e05085923eb091721675170254cbc5b02897 \ + --hash=sha256:d312ef37c91508b0ab2cee7da26ec0b3ed2f03ce12bd87a588d771ae15dcf82d \ + --hash=sha256:d4d8fe59808a54658fcc0160ecfb1b30f9089906c50b23bcb4c69eddc19ec2b4 \ + --hash=sha256:da25dc3563bff5965356133435b757a795a17b17d01dbc0f42fb32447ddfd917 \ + --hash=sha256:eab21f45c7f66c13f2a9e0e1535309cee140182a9cdae1e041d02e47291e8396 \ + --hash=sha256:eb0dc4e38e6a1fd579e5d50369aa2e10acfc9cace504579b2faabb478e76941a \ + --hash=sha256:ec9bfaf3ad2df51ace80688143a6a4ebc09a248f6ff781a9945e51937008fcbc \ + --hash=sha256:ede3e6487c5ef5d28634ba3f31f989030ad6af71edfb0055cbbd14189ff240ba \ + --hash=sha256:f3c6818a1a86dd6dca7ddcaaf76947d5ba31aecc28cb1b67009a5877c9a64f3f \ + --hash=sha256:f758f1b9299d059cc3f6546ae2af89670cb1c4d48ea29c3cacc4fe7de3058257 \ + --hash=sha256:f8f0fc26ec2cc2b965b7a3b87cd19c5c6b8c5e5f436b984e85f486d652285c30 \ + --hash=sha256:fd0409a3653af6c147209d267a0e4243f0ae46b011aa978b1080359fddc9b6cf \ + --hash=sha256:ff18e6a727ee0ab0388507b89d1bc6a22b138d1e2fa56d1ad494586d61d2eae9 \ + --hash=sha256:ff2983983d34813c1aeb0fa89091e76c3a22889ee83ab27c5eeb45100560c049 + # via + # sphinx + # sphinx-autodoc2 +typing-extensions==4.16.0 \ + --hash=sha256:481caa481374e813c1b176ada14e97f1f67a4539ce9cfeb3f350d78d6370c2e8 \ + --hash=sha256:dc983d19a509c94dba722ee6abd33940f7c05a89e243c47e907eb4db6f1a43e5 + # via + # rules-python-sphinxdocs-dev (dev/pyproject.toml) + # astroid + # sphinx-autodoc2 +urllib3==2.6.3 ; python_full_version < '3.10' \ + --hash=sha256:1b62b6884944a57dbe321509ab94fd4d3b307075e0c2eae991ac71ee15ad38ed \ + --hash=sha256:bf272323e553dfb2e87d9bfd225ca7b0f467b919d7bbd355436d3fd37cb0acd4 + # via requests +urllib3==2.7.0 ; python_full_version >= '3.10' \ + --hash=sha256:231e0ec3b63ceb14667c67be60f2f2c40a518cb38b03af60abc813da26505f4c \ + --hash=sha256:9fb4c81ebbb1ce9531cce37674bbc6f1360472bc18ca9a553ede278ef7276897 + # via requests +zipp==3.23.1 ; python_full_version < '3.10' \ + --hash=sha256:0b3596c50a5c700c9cb40ba8d86d9f2cc4807e9bedb06bcdf7fac85633e444dc \ + --hash=sha256:32120e378d32cd9714ad503c1d024619063ec28aad2248dc6672ad13edfa5110 + # via importlib-metadata diff --git a/sphinxdocs/dev/uv.lock b/sphinxdocs/dev/uv.lock new file mode 100644 index 0000000000..39ce81556d --- /dev/null +++ b/sphinxdocs/dev/uv.lock @@ -0,0 +1,1073 @@ +version = 1 +revision = 3 +requires-python = ">=3.9" +resolution-markers = [ + "python_full_version >= '3.12'", + "python_full_version == '3.11.*'", + "python_full_version == '3.10.*'", + "python_full_version < '3.10'", +] + +[[package]] +name = "absl-py" +version = "2.3.1" +source = { registry = "https://pypi.org/simple" } +resolution-markers = [ + "python_full_version < '3.10'", +] +sdist = { url = "https://files.pythonhosted.org/packages/10/2a/c93173ffa1b39c1d0395b7e842bbdc62e556ca9d8d3b5572926f3e4ca752/absl_py-2.3.1.tar.gz", hash = "sha256:a97820526f7fbfd2ec1bce83f3f25e3a14840dac0d8e02a0b71cd75db3f77fc9", size = 116588, upload-time = "2025-07-03T09:31:44.05Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/8f/aa/ba0014cc4659328dc818a28827be78e6d97312ab0cb98105a770924dc11e/absl_py-2.3.1-py3-none-any.whl", hash = "sha256:eeecf07f0c2a93ace0772c92e596ace6d3d3996c042b2128459aaae2a76de11d", size = 135811, upload-time = "2025-07-03T09:31:42.253Z" }, +] + +[[package]] +name = "absl-py" +version = "2.5.0" +source = { registry = "https://pypi.org/simple" } +resolution-markers = [ + "python_full_version >= '3.12'", + "python_full_version == '3.11.*'", + "python_full_version == '3.10.*'", +] +sdist = { url = "https://files.pythonhosted.org/packages/d0/4f/d79676ab82f2e42fc3611618139f13a9c4c31d0cff4b486982047679a802/absl_py-2.5.0.tar.gz", hash = "sha256:0c996f25c0490700fadabe6351630f6111534fa0ae252cc6d2014ea3b141135f", size = 118119, upload-time = "2026-07-03T10:57:48.157Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/58/0a/a10b45aab35b175aded078a462dc8d0c698f5b13946e7cb0869097b78bb6/absl_py-2.5.0-py3-none-any.whl", hash = "sha256:0f17b89f2a4eaaedc4f28c622998aa690564b3012a396a4ffad0821007fe03ba", size = 137410, upload-time = "2026-07-03T10:57:46.735Z" }, +] + +[[package]] +name = "alabaster" +version = "0.7.16" +source = { registry = "https://pypi.org/simple" } +resolution-markers = [ + "python_full_version < '3.10'", +] +sdist = { url = "https://files.pythonhosted.org/packages/c9/3e/13dd8e5ed9094e734ac430b5d0eb4f2bb001708a8b7856cbf8e084e001ba/alabaster-0.7.16.tar.gz", hash = "sha256:75a8b99c28a5dad50dd7f8ccdd447a121ddb3892da9e53d1ca5cca3106d58d65", size = 23776, upload-time = "2024-01-10T00:56:10.189Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/32/34/d4e1c02d3bee589efb5dfa17f88ea08bdb3e3eac12bc475462aec52ed223/alabaster-0.7.16-py3-none-any.whl", hash = "sha256:b46733c07dce03ae4e150330b975c75737fa60f0a7c591b6c8bf4928a28e2c92", size = 13511, upload-time = "2024-01-10T00:56:08.388Z" }, +] + +[[package]] +name = "alabaster" +version = "1.0.0" +source = { registry = "https://pypi.org/simple" } +resolution-markers = [ + "python_full_version >= '3.12'", + "python_full_version == '3.11.*'", + "python_full_version == '3.10.*'", +] +sdist = { url = "https://files.pythonhosted.org/packages/a6/f8/d9c74d0daf3f742840fd818d69cfae176fa332022fd44e3469487d5a9420/alabaster-1.0.0.tar.gz", hash = "sha256:c00dca57bca26fa62a6d7d0a9fcce65f3e026e9bfe33e9c538fd3fbb2144fd9e", size = 24210, upload-time = "2024-07-26T18:15:03.762Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/7e/b3/6b4067be973ae96ba0d615946e314c5ae35f9f993eca561b356540bb0c2b/alabaster-1.0.0-py3-none-any.whl", hash = "sha256:fc6786402dc3fcb2de3cabd5fe455a2db534b371124f1f21de8731783dec828b", size = 13929, upload-time = "2024-07-26T18:15:02.05Z" }, +] + +[[package]] +name = "astroid" +version = "3.3.11" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "typing-extensions", marker = "python_full_version < '3.11'" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/18/74/dfb75f9ccd592bbedb175d4a32fc643cf569d7c218508bfbd6ea7ef9c091/astroid-3.3.11.tar.gz", hash = "sha256:1e5a5011af2920c7c67a53f65d536d65bfa7116feeaf2354d8b94f29573bb0ce", size = 400439, upload-time = "2025-07-13T18:04:23.177Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/af/0f/3b8fdc946b4d9cc8cc1e8af42c4e409468c84441b933d037e101b3d72d86/astroid-3.3.11-py3-none-any.whl", hash = "sha256:54c760ae8322ece1abd213057c4b5bba7c49818853fc901ef09719a60dbf9dec", size = 275612, upload-time = "2025-07-13T18:04:21.07Z" }, +] + +[[package]] +name = "babel" +version = "2.18.0" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/7d/b2/51899539b6ceeeb420d40ed3cd4b7a40519404f9baf3d4ac99dc413a834b/babel-2.18.0.tar.gz", hash = "sha256:b80b99a14bd085fcacfa15c9165f651fbb3406e66cc603abf11c5750937c992d", size = 9959554, upload-time = "2026-02-01T12:30:56.078Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/77/f5/21d2de20e8b8b0408f0681956ca2c69f1320a3848ac50e6e7f39c6159675/babel-2.18.0-py3-none-any.whl", hash = "sha256:e2b422b277c2b9a9630c1d7903c2a00d0830c409c59ac8cae9081c92f1aeba35", size = 10196845, upload-time = "2026-02-01T12:30:53.445Z" }, +] + +[[package]] +name = "certifi" +version = "2026.7.22" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/a3/c2/24167ea9858356b47a87a50d39908bfdb72ceeefe0041586e704e5376b3a/certifi-2026.7.22.tar.gz", hash = "sha256:741e2c3b351ddf169a738da9f2c048608ff7f2c5cc02f1ebc6b118bb090d5d55", size = 138112, upload-time = "2026-07-22T03:35:12.644Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/0b/a7/71ac2cff56fec219ed242bb11b8efb69fcc4bec75db06fb7bfe35de520e6/certifi-2026.7.22-py3-none-any.whl", hash = "sha256:62f22742b58a1a33014a2b6b706588a8d7e2a88ae7bd1a6ebe8c992928483775", size = 136983, upload-time = "2026-07-22T03:35:11.276Z" }, +] + +[[package]] +name = "charset-normalizer" +version = "3.4.9" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/bd/2a/23f34ec9d04624958e137efdc394888716353190e75f25dd22c7a2c7a8aa/charset_normalizer-3.4.9.tar.gz", hash = "sha256:673611bbd43f0810bec0b0f028ddeaaa501190339cac411f347ac76917c3ae7b", size = 152439, upload-time = "2026-07-07T14:34:58.454Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/ad/81/8e983840c6e5b93b33c2ba81aa3d52c2e42f0e9a690ce7607a2e61da4a5c/charset_normalizer-3.4.9-cp310-cp310-macosx_10_9_universal2.whl", hash = "sha256:cd6280cf040f233bd7d3407b743b4b4c74f70e8e1c4199cb112a62c941c0772a", size = 322240, upload-time = "2026-07-07T14:32:36.236Z" }, + { url = "https://files.pythonhosted.org/packages/de/d1/b4319dc3229d8272fba305e206fc0a148e2de8d4087917ce62ae6382f359/charset_normalizer-3.4.9-cp310-cp310-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:aa99adc8f081b475a12843953db36831eaf83ec33eb46a90629ca6a5de45a616", size = 216475, upload-time = "2026-07-07T14:32:38.142Z" }, + { url = "https://files.pythonhosted.org/packages/80/33/6c99c1b3e6b8bf730e1bc809b9a2608f224145069114c479a2e9e1494346/charset_normalizer-3.4.9-cp310-cp310-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:c1225416b463483160e4af85d5fc3a9690ccb53fd4b1865a6437825f5ede3209", size = 238670, upload-time = "2026-07-07T14:32:39.658Z" }, + { url = "https://files.pythonhosted.org/packages/7f/f4/ffbb83546e1f198ecc70ecd372b65cf2b50f9068b380abd67640f17a8e18/charset_normalizer-3.4.9-cp310-cp310-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:16d10d789dd9bcca1173c95af82c58433122564b7bc39385124be735a35cbe99", size = 233476, upload-time = "2026-07-07T14:32:41.155Z" }, + { url = "https://files.pythonhosted.org/packages/e8/5f/b98b8da398637b551e427e7be922bdec19177dc54d6811dcdaa503f23aac/charset_normalizer-3.4.9-cp310-cp310-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:9bb41182d93ea91f60b4bc8fbf4c820c69ef8a12ab2d917f3f1834f1acad07e8", size = 223817, upload-time = "2026-07-07T14:32:42.592Z" }, + { url = "https://files.pythonhosted.org/packages/36/31/a276bb2e66243072a3fd06fdcab9cbb61a305b02143d70d2bda21d888fa8/charset_normalizer-3.4.9-cp310-cp310-manylinux_2_31_armv7l.whl", hash = "sha256:bcf74c1df76758a395bf0af608c04c82257523f55c9868b334f06270d0f2112b", size = 207974, upload-time = "2026-07-07T14:32:44.258Z" }, + { url = "https://files.pythonhosted.org/packages/5e/be/7ee4453d7e88dfbc4104ccd34900b9f2c7c17dac22881865fe0e82424a25/charset_normalizer-3.4.9-cp310-cp310-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:b5314963fce9b0b12743891de876e724997864ee22aa496f903f426c7e2fa5b2", size = 221655, upload-time = "2026-07-07T14:32:45.64Z" }, + { url = "https://files.pythonhosted.org/packages/1d/85/181c652953eb5276d198f375b1dd641047392050098100a3a02d6534f657/charset_normalizer-3.4.9-cp310-cp310-musllinux_1_2_aarch64.whl", hash = "sha256:e9701d0049d92c16703a42771b98d560b95248949f23f8cf7b4eddd201814fb9", size = 219229, upload-time = "2026-07-07T14:32:47.376Z" }, + { url = "https://files.pythonhosted.org/packages/0c/e7/aaf6da33fc9f4691cda8f7efbc9f69179d3d39ec8a4799baf273ee1d8db0/charset_normalizer-3.4.9-cp310-cp310-musllinux_1_2_armv7l.whl", hash = "sha256:65a7ff3f705e57d392f7261b6d0550fe137c3019477431f1c355e0db0a7d3e15", size = 209704, upload-time = "2026-07-07T14:32:48.855Z" }, + { url = "https://files.pythonhosted.org/packages/63/01/f2fb3bd3a73be48b173ee0c6aa8d2497af97d5663a8c4c4b491de4c62f7a/charset_normalizer-3.4.9-cp310-cp310-musllinux_1_2_x86_64.whl", hash = "sha256:79580094b00d1789d1f93ea55bc43cb2f611910c72235b7657f3482ddcc1b22d", size = 226243, upload-time = "2026-07-07T14:32:50.239Z" }, + { url = "https://files.pythonhosted.org/packages/c4/02/c57a22739fe05246b0b5783b3bfb6afaac4eebb46f3ececdfb2f048f780e/charset_normalizer-3.4.9-cp310-cp310-win32.whl", hash = "sha256:432786d3561e69aeeae6c7e8648964ce0ad05736120135601f87ac26b9c83381", size = 150935, upload-time = "2026-07-07T14:32:51.676Z" }, + { url = "https://files.pythonhosted.org/packages/37/8d/ca39a7559a4797505530d084fd3a49a2c959efbbbff146302fb7be4e3b35/charset_normalizer-3.4.9-cp310-cp310-win_amd64.whl", hash = "sha256:8c041122946b7ba21bb32c45b1aa57b1be35527690aeb3c5c234521085632eee", size = 162314, upload-time = "2026-07-07T14:32:53.193Z" }, + { url = "https://files.pythonhosted.org/packages/01/da/a44bd7a13d426e69e4894557106cd58669097bfad4a8681123b618fbfc5d/charset_normalizer-3.4.9-cp310-cp310-win_arm64.whl", hash = "sha256:375b83ed0aecfce76c16d198fbc21f3b11b337d68662bea0a995046682a11419", size = 153075, upload-time = "2026-07-07T14:32:54.554Z" }, + { url = "https://files.pythonhosted.org/packages/0b/e3/85ec501f206fb049259288c1f3506e53876937fb00edb47009348e66756b/charset_normalizer-3.4.9-cp311-cp311-macosx_10_9_universal2.whl", hash = "sha256:0e94703ec9684807f20cfb5eed95c70f67f2a8f21ad620146d7b5a13677b93e5", size = 317075, upload-time = "2026-07-07T14:32:56.021Z" }, + { url = "https://files.pythonhosted.org/packages/c3/69/2a5385192e67175f7d8bd5ce4f57c24bc956439adeae5c13a99aa28a53d1/charset_normalizer-3.4.9-cp311-cp311-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:2a441ea71902098ffe78c5abe6c494f44160b4af614ed16c3d9a3b1d17fd8ee2", size = 213837, upload-time = "2026-07-07T14:32:57.78Z" }, + { url = "https://files.pythonhosted.org/packages/b3/46/03ddc7da576d814fe0a36dd1f0fd3258e95404b4b2e3c026b7923d7e133f/charset_normalizer-3.4.9-cp311-cp311-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:304b13570067b2547562e308af560b3963857b1fa90bd6afd978130130fe2d6a", size = 235503, upload-time = "2026-07-07T14:32:59.205Z" }, + { url = "https://files.pythonhosted.org/packages/4e/6e/de0229a7ef40f6f9d28a837eebf4ec47bdca5dab4e900c84f22919af636a/charset_normalizer-3.4.9-cp311-cp311-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:4773092f8019072343a7447203308b176e10199920eb02d6195e81bbb3274c29", size = 229944, upload-time = "2026-07-07T14:33:00.803Z" }, + { url = "https://files.pythonhosted.org/packages/a5/34/49b9060e8418b14fb5cba9cf6bfb383111e2538a03a1fb18e66a95aeb3d5/charset_normalizer-3.4.9-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:04ce310cb89c15df659582aee80a0603788732a5e017d5bd5c81158106ce249c", size = 221276, upload-time = "2026-07-07T14:33:02.199Z" }, + { url = "https://files.pythonhosted.org/packages/44/95/80282cce0fae9c3061203d723ee87da996aed79679e65d8935050ee7ca1f/charset_normalizer-3.4.9-cp311-cp311-manylinux_2_31_armv7l.whl", hash = "sha256:c0323c9daef75ef2e5083624b4585018a0c9d5e3b40f607eed81a311270b934b", size = 205260, upload-time = "2026-07-07T14:33:03.698Z" }, + { url = "https://files.pythonhosted.org/packages/0c/74/2f62c8821b969ea3bd67cc2e6976834f48ca5d12664d2559ebcd9bcfbed7/charset_normalizer-3.4.9-cp311-cp311-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:871ff67ea1aad4dfd91736464934d56b32dac49f9fbe16cddba36198a7b3a0db", size = 217786, upload-time = "2026-07-07T14:33:05.12Z" }, + { url = "https://files.pythonhosted.org/packages/d9/8d/feabb82cb49fcad14515b1d7d1ca4787b0da7fc723a212bf89bc9e0fac52/charset_normalizer-3.4.9-cp311-cp311-musllinux_1_2_aarch64.whl", hash = "sha256:67830fc78e67501f47bb950471b2dcb9b35b140084429318e862895a8e89c993", size = 216798, upload-time = "2026-07-07T14:33:06.629Z" }, + { url = "https://files.pythonhosted.org/packages/a5/ff/c946d63bc3786d5b84d960b0f7ab7e25b828486a946b5aa997625bcaf6a6/charset_normalizer-3.4.9-cp311-cp311-musllinux_1_2_armv7l.whl", hash = "sha256:3d92613ec25e43b05f042302531ec0f00b8445190e43325880cbd6ab7c2581da", size = 206429, upload-time = "2026-07-07T14:33:08.006Z" }, + { url = "https://files.pythonhosted.org/packages/af/ba/5e5007c370702f85d2ef75791fac7943ed41e080364a673b20142e430e3e/charset_normalizer-3.4.9-cp311-cp311-musllinux_1_2_x86_64.whl", hash = "sha256:280081916dc341820640489a66e4696049401ef1cf6dd672f672e70ad915aca3", size = 223066, upload-time = "2026-07-07T14:33:09.783Z" }, + { url = "https://files.pythonhosted.org/packages/83/d5/9096aa3cf532dfad237861544eb47a0f20d5adbf1039760fed8eaae935d9/charset_normalizer-3.4.9-cp311-cp311-win32.whl", hash = "sha256:ac351b3b8014eead140e77e9717e2992c6bbe30b63bc3422422eb84865412e3d", size = 150456, upload-time = "2026-07-07T14:33:11.217Z" }, + { url = "https://files.pythonhosted.org/packages/ed/a1/e29995109e455dc8eff8d0fac6ae509be39561318a7cfeac5d33ad029213/charset_normalizer-3.4.9-cp311-cp311-win_amd64.whl", hash = "sha256:6366a16e1a25018694d6a5d784d09b046edc9eac40ea2b54065c3052672516a1", size = 161410, upload-time = "2026-07-07T14:33:12.743Z" }, + { url = "https://files.pythonhosted.org/packages/4f/8d/1569f4d0032d6ba2a4fe4591c35bf87868c600c41a71eb5c2e1ffa8464c2/charset_normalizer-3.4.9-cp311-cp311-win_arm64.whl", hash = "sha256:1d22856ffbe153a602df38e4a5464f0b748a54002e0d69ac6d2ad0a197cc99ec", size = 152649, upload-time = "2026-07-07T14:33:14.173Z" }, + { url = "https://files.pythonhosted.org/packages/70/4a/ecbd131485c07fcdfad54e28946d513e3da22ef3b4bd854dcafae54ec739/charset_normalizer-3.4.9-cp312-cp312-macosx_10_13_universal2.whl", hash = "sha256:45b0cc4e3556cd875e09102988d1ab8356c998b596c9fced84547c8138b487a0", size = 319300, upload-time = "2026-07-07T14:33:15.666Z" }, + { url = "https://files.pythonhosted.org/packages/ec/96/5d9364e3342d69f3a045e1777bc47c85c383e6e9466d561b33fdb419d1f9/charset_normalizer-3.4.9-cp312-cp312-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:9b2aff1c7b3884512b9512c3eaadd9bab39fb45042ffaaa1dd08ff2b9f8109d9", size = 215802, upload-time = "2026-07-07T14:33:17.031Z" }, + { url = "https://files.pythonhosted.org/packages/4b/4c/5361f9aa7f2cb58d94f2ab831b3d493f69efb1d239654b4744e3c09527cb/charset_normalizer-3.4.9-cp312-cp312-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:9104ed0bd76a429d46f9ec0dbc9b08ad1d2dcdf2b00a5a0daa1c145329b35b44", size = 237171, upload-time = "2026-07-07T14:33:18.576Z" }, + { url = "https://files.pythonhosted.org/packages/50/78/ce342ca4ff30b2eb49fe6d9578df85974f90c67d294113e94efdd9664cbd/charset_normalizer-3.4.9-cp312-cp312-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:7b86a2b16095d250c6f58b3d9b2eee6f4147754344f3dab0922f7c9bf7d226c9", size = 233075, upload-time = "2026-07-07T14:33:20.084Z" }, + { url = "https://files.pythonhosted.org/packages/01/c4/4fa4c8b3097a11f3c5f09a35b72ed6855fb1d332469504962ab7bafcc702/charset_normalizer-3.4.9-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:5e226f6218febc71f6c1fc2fafb91c226f75bdc1d8fb12d66823716e891608fd", size = 224256, upload-time = "2026-07-07T14:33:21.747Z" }, + { url = "https://files.pythonhosted.org/packages/87/3a/ad914516df7e358a81aae018caa5e0470ba827fa6d763b1d2e87d920a5f6/charset_normalizer-3.4.9-cp312-cp312-manylinux_2_31_armv7l.whl", hash = "sha256:90c44bc373b7687f6948b693cceaea1348ae0975d7474746559494468e3c1d84", size = 208784, upload-time = "2026-07-07T14:33:23.313Z" }, + { url = "https://files.pythonhosted.org/packages/d7/74/3c12f9755717dfe5c5c87da63f35d765fa0c00382ec26bf23f7fae34f2ba/charset_normalizer-3.4.9-cp312-cp312-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:9cdef90ae47919cae358d8ab15797a800ed41da7aba5d72419fb510729e2ed4b", size = 219928, upload-time = "2026-07-07T14:33:24.814Z" }, + { url = "https://files.pythonhosted.org/packages/33/9a/895095b83e7907abd6d3d99aad3a38ad0d9686cc186cb0c94c24320fe63e/charset_normalizer-3.4.9-cp312-cp312-musllinux_1_2_aarch64.whl", hash = "sha256:60f44ade2cf573dad7a277e6f8ca9a51a21dda572b13bd7d8539bb3cd5dbedde", size = 218489, upload-time = "2026-07-07T14:33:26.42Z" }, + { url = "https://files.pythonhosted.org/packages/a1/34/ef5c05f412f42520d7709b7d3784d19640839eb7366ded1755511585429f/charset_normalizer-3.4.9-cp312-cp312-musllinux_1_2_armv7l.whl", hash = "sha256:a1786910334ed46ab1dd73222f2cd1e05c2c3bb39f6dddb4f8b36fc382058a39", size = 210267, upload-time = "2026-07-07T14:33:27.952Z" }, + { url = "https://files.pythonhosted.org/packages/83/dc/9b29fa4412b318bf3bfea985c35d67eb55e04b59a7c3f2237168b0e0be6f/charset_normalizer-3.4.9-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:03d07803992c6c7bbc976327f34b18b6160327fc81cb82c9d504720ac0be3b62", size = 226030, upload-time = "2026-07-07T14:33:29.397Z" }, + { url = "https://files.pythonhosted.org/packages/0e/42/6dbc00b8cd16011691203e33570fa42ed5746599a2e878112d16eab403a3/charset_normalizer-3.4.9-cp312-cp312-win32.whl", hash = "sha256:78841cccf1af7b40f6f716338d50c0902dbe88d9f800b3c973b7a9a0a693a642", size = 151185, upload-time = "2026-07-07T14:33:30.781Z" }, + { url = "https://files.pythonhosted.org/packages/80/cc/f920afd1a23c58ccd53c1d36085a71893a4737ff5e66e0371efab6809850/charset_normalizer-3.4.9-cp312-cp312-win_amd64.whl", hash = "sha256:4b3dac63058cc36820b0dd072f89898604e2d39686fe05321729d00d8ac185a0", size = 162557, upload-time = "2026-07-07T14:33:32.176Z" }, + { url = "https://files.pythonhosted.org/packages/f0/e6/0386d43a261ff4e4b30c5857af7df877254b46bec7b9d1b74b6bf969a90b/charset_normalizer-3.4.9-cp312-cp312-win_arm64.whl", hash = "sha256:78fa18e436a1a0e58dbd7e02fc4473f3f32cceb12df9dfca542d075961c307d2", size = 152665, upload-time = "2026-07-07T14:33:33.711Z" }, + { url = "https://files.pythonhosted.org/packages/b2/06/97ec2aeae780b31d742b6352218b43841a6871e2564578ca522dce4a45c3/charset_normalizer-3.4.9-cp313-cp313-macosx_10_13_universal2.whl", hash = "sha256:440eede837960000d74978f0eba527be106b5b9aee0daf779d395276ed0b0614", size = 317688, upload-time = "2026-07-07T14:33:35.408Z" }, + { url = "https://files.pythonhosted.org/packages/d0/39/8ff066c672434225f8d25f8b739f992af250944392173dcc88362681c9bf/charset_normalizer-3.4.9-cp313-cp313-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:21e764fd1e70b6a3e205a0e46f3051701f98a8cb3fad66eeb80e48bb502f8698", size = 214982, upload-time = "2026-07-07T14:33:36.996Z" }, + { url = "https://files.pythonhosted.org/packages/92/8f/3a47a3667c83c2df9483d91644c6c107de3bf8874aa1793da9d3012eb986/charset_normalizer-3.4.9-cp313-cp313-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:e4fd89cc178bced6ad29cb3e6dd4aa63fa5017c3524dbd0b25998fb64a87cc8b", size = 236460, upload-time = "2026-07-07T14:33:38.536Z" }, + { url = "https://files.pythonhosted.org/packages/f1/60/b22cdbee7e4013dab8b0d7647fc6181120fbbbc8f7025c226d15bd5a47fc/charset_normalizer-3.4.9-cp313-cp313-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:bd47ba7fc3ca94896759ea0109775132d3e7ab921fbf54038e1bab2e46c313c9", size = 232003, upload-time = "2026-07-07T14:33:40.059Z" }, + { url = "https://files.pythonhosted.org/packages/ea/f8/72eb13dcabe7257035cea8aefd922caad2f110d252bf9f67c4c2ca763aee/charset_normalizer-3.4.9-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:84fd18bcc17526fc2b3c1af7d2b9217d32c9c04448c16ec693b9b4f1985c3d33", size = 223149, upload-time = "2026-07-07T14:33:41.631Z" }, + { url = "https://files.pythonhosted.org/packages/b0/3e/faee8f9de92b14ee1198e9163252bb15efee7301b31256a3b6d9ebfdd0dd/charset_normalizer-3.4.9-cp313-cp313-manylinux_2_31_armv7l.whl", hash = "sha256:5b10cd92fc5c498b35a8635df6d5a100207f88b63a4dc1de7ef9a548e1e2cd63", size = 207901, upload-time = "2026-07-07T14:33:43.209Z" }, + { url = "https://files.pythonhosted.org/packages/3a/25/45f30093ae27dd7b92a793b61882a38685f993700113ca36e0c9c14965e1/charset_normalizer-3.4.9-cp313-cp313-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:a4fbdde9dd4a9ce5fd52c2b3a347bb50cc89483ef783f1cb00d408c13f7a96c0", size = 219176, upload-time = "2026-07-07T14:33:44.725Z" }, + { url = "https://files.pythonhosted.org/packages/48/18/c8f397329c35e32f6a837e488986f4ae03bd2abebc453b48714991630c2f/charset_normalizer-3.4.9-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:416c229f77e5ea25b3dfd4b582f8d73d7e43c22320302b9ab128a2d3a0b38efe", size = 217356, upload-time = "2026-07-07T14:33:46.192Z" }, + { url = "https://files.pythonhosted.org/packages/86/7e/5ce0bba863470fd1902d5e5843968951bddf38abe4742fc97116ef4598b3/charset_normalizer-3.4.9-cp313-cp313-musllinux_1_2_armv7l.whl", hash = "sha256:75286256590a6320cf106a0d28970d3560aad9ee09aa7b34fb40524792436d35", size = 209614, upload-time = "2026-07-07T14:33:47.705Z" }, + { url = "https://files.pythonhosted.org/packages/6c/ef/2473d3c4d869155be4af1191111d59c4d5c4e0173026f7e85b176e23bf65/charset_normalizer-3.4.9-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:69b157c5d3292bcd443faca052f3096f637f1e074b98212a933c074ae23dc3b8", size = 224991, upload-time = "2026-07-07T14:33:49.238Z" }, + { url = "https://files.pythonhosted.org/packages/d0/a3/53ddae3db108a088156aa8ddfafd411ebbc1340f48c5573f697b27f69a39/charset_normalizer-3.4.9-cp313-cp313-win32.whl", hash = "sha256:51307f5c71007673a2bf8232ad973483d281e74cb99c8c5a990af1eefa6277d9", size = 150622, upload-time = "2026-07-07T14:33:50.711Z" }, + { url = "https://files.pythonhosted.org/packages/e8/ef/6953a77c7cf2c2ff9998e6f575ab3e380119f100223381565a4f94c1f836/charset_normalizer-3.4.9-cp313-cp313-win_amd64.whl", hash = "sha256:fe2c7201c642b7c308f1675355ad7ff7b66acfe3541625efe5a3ad38f29d6115", size = 161947, upload-time = "2026-07-07T14:33:52.197Z" }, + { url = "https://files.pythonhosted.org/packages/6e/fb/d560d1d1555debbfe7849d9cac6145c1b537709d79576bf22557ed803b82/charset_normalizer-3.4.9-cp313-cp313-win_arm64.whl", hash = "sha256:611057cc5d5c0afc743ba8be6bd828c17e0aaa8643f9d0a9b9bb7dea80eb8012", size = 152594, upload-time = "2026-07-07T14:33:53.486Z" }, + { url = "https://files.pythonhosted.org/packages/7e/8d/496817fa0944239ecae662dd57ea765cfeaec6a735f9f025d4b7b72e7143/charset_normalizer-3.4.9-cp314-cp314-macosx_10_15_universal2.whl", hash = "sha256:0327fcd59a935777d83410750c50600ee9571af2846f71ce40f25b13da1ef380", size = 317253, upload-time = "2026-07-07T14:33:54.994Z" }, + { url = "https://files.pythonhosted.org/packages/2b/f9/ef4a69ea338ad3c0deceea0f5f7d2380ae8b52132b06d652cb0d2cd86706/charset_normalizer-3.4.9-cp314-cp314-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:8a79d9f4d8001473a30c163556b3c3bfebec837495a412dde78b51672f6134f9", size = 215898, upload-time = "2026-07-07T14:33:56.334Z" }, + { url = "https://files.pythonhosted.org/packages/8c/e7/5ddfd76fc061eb52de219658a4aa431cbacadf0a0219c8854f00da50d289/charset_normalizer-3.4.9-cp314-cp314-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:33bdcc2a32c0a0e861f60841a512c8acc658c87c2ac59d89e3a46dacf7d866e4", size = 236718, upload-time = "2026-07-07T14:33:57.9Z" }, + { url = "https://files.pythonhosted.org/packages/49/ba/768fa3f36048d81c477a0ce61f813bc1454d80917ccfe550abd9f44f5e24/charset_normalizer-3.4.9-cp314-cp314-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:f840ed6d8ecba8255df8c42b87fadeda98ddfc6eeec05e2dc66e26d46dd6f58a", size = 232519, upload-time = "2026-07-07T14:33:59.811Z" }, + { url = "https://files.pythonhosted.org/packages/f4/c4/b3e049d2aa3766180c78507110543d9d50894cc97f57de543f1be521dcdc/charset_normalizer-3.4.9-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:c25fe15c70c59eb7c5ce8c06a1f3fa1da0ecc5ea1e7a5922c40fd2fa9b0d5046", size = 223143, upload-time = "2026-07-07T14:34:01.517Z" }, + { url = "https://files.pythonhosted.org/packages/19/79/55c32d06d76ae4feafe053f061f3e3ab70bcf19f4007797ce8c3efda7830/charset_normalizer-3.4.9-cp314-cp314-manylinux_2_31_armv7l.whl", hash = "sha256:f7fb7d750cfa0a070d2c24e831fd3481019a60dd317ea2b39acbcebc08b6ed81", size = 206742, upload-time = "2026-07-07T14:34:03.04Z" }, + { url = "https://files.pythonhosted.org/packages/10/e0/47c079dd82d217c807479cd59ffd30af56307ea31c108b75758970459ad3/charset_normalizer-3.4.9-cp314-cp314-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:4d1c96a7a18b9690a4d46df09e3e3382406ae3213727cd1019ebade1c4a81917", size = 219191, upload-time = "2026-07-07T14:34:04.657Z" }, + { url = "https://files.pythonhosted.org/packages/42/ab/b9bc2e77d6b44a7e46ef62ec5cac1c9a6ba7b9135a5d560f002696ec9995/charset_normalizer-3.4.9-cp314-cp314-musllinux_1_2_aarch64.whl", hash = "sha256:a4cfde78a9f2880208d16a93b795726a3017d5977e08d1e162a7a31322479c41", size = 218328, upload-time = "2026-07-07T14:34:06.115Z" }, + { url = "https://files.pythonhosted.org/packages/f1/78/c9c71d599f5aa2d42bcdd35cbbd46d7f535351a57e40ff7d8e5a7e219401/charset_normalizer-3.4.9-cp314-cp314-musllinux_1_2_armv7l.whl", hash = "sha256:d4d6fcde76f94f5cb9e43e9e9a61f16dacefd228cbbf6f1a09bd9b219a92f1a1", size = 207406, upload-time = "2026-07-07T14:34:07.554Z" }, + { url = "https://files.pythonhosted.org/packages/f6/39/c914445c321a845097ce4f6ac7de9a18228a77b766272125a1ce00d851eb/charset_normalizer-3.4.9-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:898f0e9068ca27d37f8e83a5b962821df851532e6c4a7d615c1c033f9da6eedf", size = 225157, upload-time = "2026-07-07T14:34:09.061Z" }, + { url = "https://files.pythonhosted.org/packages/9b/f2/c0d4b8508565a36bc5c624e88ed297f5b0b1095011034d7f5b83a69908b5/charset_normalizer-3.4.9-cp314-cp314-win32.whl", hash = "sha256:c1c948747b03be832dceed96ca815cef7360de9aa19d37c730f8e3f6101aca48", size = 151095, upload-time = "2026-07-07T14:34:10.901Z" }, + { url = "https://files.pythonhosted.org/packages/49/fd/a1d26144398c67486422a72bf5812cda22cb4ccfcd95a290fb41ceb4b8e2/charset_normalizer-3.4.9-cp314-cp314-win_amd64.whl", hash = "sha256:16b65ea0f2465b6fb52aa22de5eca612aa964ddfec00a912e26f4656cbef890b", size = 162796, upload-time = "2026-07-07T14:34:12.47Z" }, + { url = "https://files.pythonhosted.org/packages/20/95/d75e82f8ce9fd323ebf059c16c9aadefb22a1ecde13b7840b35835e4886c/charset_normalizer-3.4.9-cp314-cp314-win_arm64.whl", hash = "sha256:40a126142a56b2dfc0aacbad1de8310cbf60da7656db0e6b16eebd48e3e93519", size = 153334, upload-time = "2026-07-07T14:34:14.044Z" }, + { url = "https://files.pythonhosted.org/packages/00/5e/17398df3a139985ba9d11ed072531986f408c8fca952835ef1ab1820c02b/charset_normalizer-3.4.9-cp314-cp314t-macosx_10_15_universal2.whl", hash = "sha256:609b3ba8fcc0fb5ab7af00719d0fb6ad0cb518e48e7712d12fd68f1327951198", size = 338848, upload-time = "2026-07-07T14:34:15.688Z" }, + { url = "https://files.pythonhosted.org/packages/cd/91/7253a32e86b7e1d1239b1b36ba6dd0f021a21107ab33054b53119cc083b9/charset_normalizer-3.4.9-cp314-cp314t-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:51447e9aa2684679af07ca5021c3db526e0284347ebf4ffcec1154c3350cfe32", size = 223022, upload-time = "2026-07-07T14:34:17.248Z" }, + { url = "https://files.pythonhosted.org/packages/cb/32/2e64bd2be10e89c61e57ebe6a93fd98ae88eb7ebe414b5121f22c96c69eb/charset_normalizer-3.4.9-cp314-cp314t-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:cc1b0fff8ead343dae06305f954eb8468ba0ec1a97881f42489d198e4ce3c632", size = 241590, upload-time = "2026-07-07T14:34:18.813Z" }, + { url = "https://files.pythonhosted.org/packages/3d/ef/d96ec496cfea0c21db43b0ad03891308b02388d054cc902cf0e5a1ad6a88/charset_normalizer-3.4.9-cp314-cp314t-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:fa36ec09ef71d158186bc79e359ff5fdd6e7996fe8ab638f00d6b93139ba4fcf", size = 239584, upload-time = "2026-07-07T14:34:20.52Z" }, + { url = "https://files.pythonhosted.org/packages/d4/ce/9af95f7876194bd7a14e3dfe4a4de2e0bff02666a3910d72beafd06cc297/charset_normalizer-3.4.9-cp314-cp314t-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:df115d4d83168fdf2cae48ef1ff6d1cb4c466364e30861b37121de0f3bf1b990", size = 230224, upload-time = "2026-07-07T14:34:22.189Z" }, + { url = "https://files.pythonhosted.org/packages/52/94/af74dde74a3996bd959c350709bfe50e297823d70a8c1cbd54b838880863/charset_normalizer-3.4.9-cp314-cp314t-manylinux_2_31_armv7l.whl", hash = "sha256:f86c6358749bd4fda175388691e3ba8c46e24c5347d0afd20f9b7edfc9faf07d", size = 212667, upload-time = "2026-07-07T14:34:23.857Z" }, + { url = "https://files.pythonhosted.org/packages/ee/f0/f1c4fe746c395922961b5916ed1d7d6e7d4c84851d19ed43cc89980ec953/charset_normalizer-3.4.9-cp314-cp314t-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:32286a2c8d167e897177b673176c1e3e00d4057caf5d2b64eef9a3666b03018e", size = 227179, upload-time = "2026-07-07T14:34:25.586Z" }, + { url = "https://files.pythonhosted.org/packages/e4/56/6c745619ac397e8871e2bcd3cea1eec86b877488f33888b3aef5c3ed506e/charset_normalizer-3.4.9-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:83aed2c10721ddd90f68140685391b50811a880af20654c59af6b6c66c40513c", size = 225372, upload-time = "2026-07-07T14:34:27.212Z" }, + { url = "https://files.pythonhosted.org/packages/78/ad/98aae8630ac71f16711968e38a5acfecce41b778bf2f0312851020f565a8/charset_normalizer-3.4.9-cp314-cp314t-musllinux_1_2_armv7l.whl", hash = "sha256:cd6c3d4b783c556fa00bf540854e42f135e2f256abd29669fcd0da0f2dec79c2", size = 215222, upload-time = "2026-07-07T14:34:28.774Z" }, + { url = "https://files.pythonhosted.org/packages/f7/40/9593d54209765207a7f11073c06494c1721e4ca4a0a426c597679bf7f91e/charset_normalizer-3.4.9-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:ee2f2a527e3c1a6e6411eb4209642e138b544a2d72fe5d0d76daf77b24063534", size = 231958, upload-time = "2026-07-07T14:34:30.345Z" }, + { url = "https://files.pythonhosted.org/packages/b1/27/693ee5e8a18191eb38647360c51cd505013e2bd3b366aa43fd5344c21e3c/charset_normalizer-3.4.9-cp314-cp314t-win32.whl", hash = "sha256:0d861473f743244d349b50f850d10eb87aeb22bbdcc8e64f79273c94af5a8226", size = 155580, upload-time = "2026-07-07T14:34:31.884Z" }, + { url = "https://files.pythonhosted.org/packages/80/3f/bd97d3d9c613013d07cb7733d299385b41df37f0471310f5a73dc359f0b8/charset_normalizer-3.4.9-cp314-cp314t-win_amd64.whl", hash = "sha256:9b8e0f3107e2200b76f6054de99016eac3ee6762713587b36baaa7e4bd2ae177", size = 167620, upload-time = "2026-07-07T14:34:33.438Z" }, + { url = "https://files.pythonhosted.org/packages/3d/c6/eee9dca4439b1061f76373f06ea855678cc4a64c1c3c90b50e479edbb8eb/charset_normalizer-3.4.9-cp314-cp314t-win_arm64.whl", hash = "sha256:19ac87f93086ce37b86e098888555c4b4bc48102279bae3350098c0ed664b501", size = 158037, upload-time = "2026-07-07T14:34:35.018Z" }, + { url = "https://files.pythonhosted.org/packages/a6/ec/81e22253f4b7091eca6515bb3da5e45d05a663f7f567bb745695dc60f892/charset_normalizer-3.4.9-cp39-cp39-macosx_10_9_universal2.whl", hash = "sha256:253a4a220747e8b5faf57ec320c4f5efb0cef05f647420bf267143ec15dba10a", size = 306122, upload-time = "2026-07-07T14:34:36.607Z" }, + { url = "https://files.pythonhosted.org/packages/c8/53/a8c042eb9eee4716f4d42a0f5a571eb32a09ec429be9fb0b8b9d765393ba/charset_normalizer-3.4.9-cp39-cp39-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:68ce9f4d6b26d5ccbf7fd4459bf75f74a0a146677ebba80597df60cbdb20e6f4", size = 206284, upload-time = "2026-07-07T14:34:38.166Z" }, + { url = "https://files.pythonhosted.org/packages/14/cb/1db8b96547ee3186cd2dd7f2e59dd560a9b80748f3604171f3c153d62811/charset_normalizer-3.4.9-cp39-cp39-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:58150c9f9b9a552505912d182ccdf26f6396fb6094816ceebcbb20eecabaed94", size = 226837, upload-time = "2026-07-07T14:34:39.77Z" }, + { url = "https://files.pythonhosted.org/packages/6a/05/c94d5cd23396289c54c93b02e0273b4dd8921641d9968c4828caf9bbaad9/charset_normalizer-3.4.9-cp39-cp39-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:df7276909358e5635ae203673ab7e509ddd224225a8d6b0790bf13eb2bde1cc5", size = 222199, upload-time = "2026-07-07T14:34:41.391Z" }, + { url = "https://files.pythonhosted.org/packages/6d/46/79847edd07244a4a2d443c6655a7b6ee94203c21539414b059f32713c357/charset_normalizer-3.4.9-cp39-cp39-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:3c09a49d6cde137258beb3d551994a2927fd35ad5cf96aed573f61bbd67c5f84", size = 214344, upload-time = "2026-07-07T14:34:42.986Z" }, + { url = "https://files.pythonhosted.org/packages/ec/b4/ef5a49b2e77c00deb43bb3256592b115ba9e4346016e82c516b8d215bf68/charset_normalizer-3.4.9-cp39-cp39-manylinux_2_31_armv7l.whl", hash = "sha256:231ddcbb35e2ff8973e1365db41fe0572662893b99a05deb183b68ad4c0c8bd4", size = 199988, upload-time = "2026-07-07T14:34:44.685Z" }, + { url = "https://files.pythonhosted.org/packages/3d/ca/ad1d7c7d3077dab873f539d3e1d083c0845a762cb0bafdfbe3ef93add598/charset_normalizer-3.4.9-cp39-cp39-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:920079c3f7456fa213e0829ed2073aaa727fd39d889ead5b4f35d0de5460d04f", size = 211908, upload-time = "2026-07-07T14:34:46.227Z" }, + { url = "https://files.pythonhosted.org/packages/ed/61/710738687f90d01c06a04ed52d6ca1e62dd9b1d8cc2567098167c4691034/charset_normalizer-3.4.9-cp39-cp39-musllinux_1_2_aarch64.whl", hash = "sha256:0fa1aec2d32bcc03c8fa0f6f1712caad1adc38509f31142112e5c9daf5b9c833", size = 209320, upload-time = "2026-07-07T14:34:47.753Z" }, + { url = "https://files.pythonhosted.org/packages/5f/c0/6eec7bdabe6cbbcc274ec04596f6d93865751a0541d33d60d1ce179bd372/charset_normalizer-3.4.9-cp39-cp39-musllinux_1_2_armv7l.whl", hash = "sha256:ad41ba96094304aa090f5a30cb6e4fb3b3f1c264c523394b4c39bbacc4dc92ba", size = 200980, upload-time = "2026-07-07T14:34:49.362Z" }, + { url = "https://files.pythonhosted.org/packages/eb/78/59344ff9a4a7b5f6530bf7bec2c980047cc42c3a616596cdbd8cb5c1a1af/charset_normalizer-3.4.9-cp39-cp39-musllinux_1_2_x86_64.whl", hash = "sha256:43b9e366a31fdd1c87d0eb08f579b4a82b723ea54338f040d6b4e518a026ea29", size = 216545, upload-time = "2026-07-07T14:34:50.98Z" }, + { url = "https://files.pythonhosted.org/packages/17/6d/bff78a4bacc4891bc63ec5bdc6776d8c85e47fab93d0d5f6223068fad0a4/charset_normalizer-3.4.9-cp39-cp39-win32.whl", hash = "sha256:93d59d504b230e83c7a843251681959a0b6a9cd76f6e146ce1b8a80eb8739af9", size = 146256, upload-time = "2026-07-07T14:34:52.509Z" }, + { url = "https://files.pythonhosted.org/packages/a2/55/86048bde1c9d0352940bd7b87d825091a52aef67d01cde6c6f7342c5b552/charset_normalizer-3.4.9-cp39-cp39-win_amd64.whl", hash = "sha256:ddf4af30b417d9fe16481e9b81c27ab2a7cde1ff7ba3e85653b02db7d145dc7b", size = 156413, upload-time = "2026-07-07T14:34:54.117Z" }, + { url = "https://files.pythonhosted.org/packages/28/e9/9fb6099b868c82a40698a748ae0fbd4f31ccc13844c176a07158ba2abbfd/charset_normalizer-3.4.9-cp39-cp39-win_arm64.whl", hash = "sha256:476743fe6dfe14a2da12e3ac79125dc84a3b2cf8094369a47a1529b0cd8549fe", size = 147887, upload-time = "2026-07-07T14:34:55.51Z" }, + { url = "https://files.pythonhosted.org/packages/98/2b/f97f1c193fb855c345d678f5077d6926034db0722df74c8f057020e05a25/charset_normalizer-3.4.9-py3-none-any.whl", hash = "sha256:68e5f26a1ad57ded6d1cfb85331d1c1a195314756471d97758c48498bb4dcdf5", size = 64538, upload-time = "2026-07-07T14:34:56.993Z" }, +] + +[[package]] +name = "colorama" +version = "0.4.6" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/d8/53/6f443c9a4a8358a93a6792e2acffb9d9d5cb0a5cfd8802644b7b1c9a02e4/colorama-0.4.6.tar.gz", hash = "sha256:08695f5cb7ed6e0531a20572697297273c47b8cae5a63ffc6d6ed5c201be6e44", size = 27697, upload-time = "2022-10-25T02:36:22.414Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/d1/d6/3965ed04c63042e047cb6a3e6ed1a63a35087b6a609aa3a15ed8ac56c221/colorama-0.4.6-py2.py3-none-any.whl", hash = "sha256:4f1d9991f5acc0ca119f9d443620b77f9d6b33703e51011c16baf57afb285fc6", size = 25335, upload-time = "2022-10-25T02:36:20.889Z" }, +] + +[[package]] +name = "docutils" +version = "0.21.2" +source = { registry = "https://pypi.org/simple" } +resolution-markers = [ + "python_full_version == '3.10.*'", + "python_full_version < '3.10'", +] +sdist = { url = "https://files.pythonhosted.org/packages/ae/ed/aefcc8cd0ba62a0560c3c18c33925362d46c6075480bfa4df87b28e169a9/docutils-0.21.2.tar.gz", hash = "sha256:3a6b18732edf182daa3cd12775bbb338cf5691468f91eeeb109deff6ebfa986f", size = 2204444, upload-time = "2024-04-23T18:57:18.24Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/8f/d7/9322c609343d929e75e7e5e6255e614fcc67572cfd083959cdef3b7aad79/docutils-0.21.2-py3-none-any.whl", hash = "sha256:dafca5b9e384f0e419294eb4d2ff9fa826435bf15f15b7bd45723e8ad76811b2", size = 587408, upload-time = "2024-04-23T18:57:14.835Z" }, +] + +[[package]] +name = "docutils" +version = "0.22.4" +source = { registry = "https://pypi.org/simple" } +resolution-markers = [ + "python_full_version >= '3.12'", + "python_full_version == '3.11.*'", +] +sdist = { url = "https://files.pythonhosted.org/packages/ae/b6/03bb70946330e88ffec97aefd3ea75ba575cb2e762061e0e62a213befee8/docutils-0.22.4.tar.gz", hash = "sha256:4db53b1fde9abecbb74d91230d32ab626d94f6badfc575d6db9194a49df29968", size = 2291750, upload-time = "2025-12-18T19:00:26.443Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/02/10/5da547df7a391dcde17f59520a231527b8571e6f46fc8efb02ccb370ab12/docutils-0.22.4-py3-none-any.whl", hash = "sha256:d0013f540772d1420576855455d050a2180186c91c15779301ac2ccb3eeb68de", size = 633196, upload-time = "2025-12-18T19:00:18.077Z" }, +] + +[[package]] +name = "idna" +version = "3.18" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/cd/63/9496c57188a2ee585e0f1db071d75089a11e98aa86eb99d9d7618fc1edce/idna-3.18.tar.gz", hash = "sha256:ffb385a7e039654cef1ab9ef32c6fafe283c0c0467bba1d9029738ce4a14a848", size = 196711, upload-time = "2026-06-02T14:34:07.794Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/1e/5e/d4e9f1a599fb8e573b7b87160658329fbf28d19eac2718f51fc3def3aa5a/idna-3.18-py3-none-any.whl", hash = "sha256:7f952cbe720b688055e3f87de14f5c3e5fdaa8bc3928985c4077ca689de849a2", size = 65455, upload-time = "2026-06-02T14:34:06.319Z" }, +] + +[[package]] +name = "imagesize" +version = "1.5.0" +source = { registry = "https://pypi.org/simple" } +resolution-markers = [ + "python_full_version < '3.10'", +] +sdist = { url = "https://files.pythonhosted.org/packages/cf/59/4b0dd64676aa6fb4986a755790cb6fc558559cf0084effad516820208ec3/imagesize-1.5.0.tar.gz", hash = "sha256:8bfc5363a7f2133a89f0098451e0bcb1cd71aba4dc02bbcecb39d99d40e1b94f", size = 1281127, upload-time = "2026-03-03T01:59:54.651Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/1e/b1/a0662b03103c66cf77101a187f396ea91167cd9b7d5d3a2e465ad2c7ee9b/imagesize-1.5.0-py2.py3-none-any.whl", hash = "sha256:32677681b3f434c2cb496f00e89c5a291247b35b1f527589909e008057da5899", size = 5763, upload-time = "2026-03-03T01:59:52.343Z" }, +] + +[[package]] +name = "imagesize" +version = "2.0.0" +source = { registry = "https://pypi.org/simple" } +resolution-markers = [ + "python_full_version >= '3.12'", + "python_full_version == '3.11.*'", + "python_full_version == '3.10.*'", +] +sdist = { url = "https://files.pythonhosted.org/packages/6c/e6/7bf14eeb8f8b7251141944835abd42eb20a658d89084b7e1f3e5fe394090/imagesize-2.0.0.tar.gz", hash = "sha256:8e8358c4a05c304f1fccf7ff96f036e7243a189e9e42e90851993c558cfe9ee3", size = 1773045, upload-time = "2026-03-03T14:18:29.941Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/5f/53/fb7122b71361a0d121b669dcf3d31244ef75badbbb724af388948de543e2/imagesize-2.0.0-py2.py3-none-any.whl", hash = "sha256:5667c5bbb57ab3f1fa4bc366f4fbc971db3d5ed011fd2715fd8001f782718d96", size = 9441, upload-time = "2026-03-03T14:18:27.892Z" }, +] + +[[package]] +name = "importlib-metadata" +version = "8.7.1" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "zipp", marker = "python_full_version < '3.10'" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/f3/49/3b30cad09e7771a4982d9975a8cbf64f00d4a1ececb53297f1d9a7be1b10/importlib_metadata-8.7.1.tar.gz", hash = "sha256:49fef1ae6440c182052f407c8d34a68f72efc36db9ca90dc0113398f2fdde8bb", size = 57107, upload-time = "2025-12-21T10:00:19.278Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/fa/5e/f8e9a1d23b9c20a551a8a02ea3637b4642e22c2626e3a13a9a29cdea99eb/importlib_metadata-8.7.1-py3-none-any.whl", hash = "sha256:5a1f80bf1daa489495071efbb095d75a634cf28a8bc299581244063b53176151", size = 27865, upload-time = "2025-12-21T10:00:18.329Z" }, +] + +[[package]] +name = "jinja2" +version = "3.1.6" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "markupsafe" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/df/bf/f7da0350254c0ed7c72f3e33cef02e048281fec7ecec5f032d4aac52226b/jinja2-3.1.6.tar.gz", hash = "sha256:0137fb05990d35f1275a587e9aee6d56da821fc83491a0fb838183be43f66d6d", size = 245115, upload-time = "2025-03-05T20:05:02.478Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/62/a1/3d680cbfd5f4b8f15abc1d571870c5fc3e594bb582bc3b64ea099db13e56/jinja2-3.1.6-py3-none-any.whl", hash = "sha256:85ece4451f492d0c13c5dd7c13a64681a86afae63a5f347908daf103ce6d2f67", size = 134899, upload-time = "2025-03-05T20:05:00.369Z" }, +] + +[[package]] +name = "markdown-it-py" +version = "3.0.0" +source = { registry = "https://pypi.org/simple" } +resolution-markers = [ + "python_full_version == '3.10.*'", + "python_full_version < '3.10'", +] +dependencies = [ + { name = "mdurl", marker = "python_full_version < '3.11'" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/38/71/3b932df36c1a044d397a1f92d1cf91ee0a503d91e470cbd670aa66b07ed0/markdown-it-py-3.0.0.tar.gz", hash = "sha256:e3f60a94fa066dc52ec76661e37c851cb232d92f9886b15cb560aaada2df8feb", size = 74596, upload-time = "2023-06-03T06:41:14.443Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/42/d7/1ec15b46af6af88f19b8e5ffea08fa375d433c998b8a7639e76935c14f1f/markdown_it_py-3.0.0-py3-none-any.whl", hash = "sha256:355216845c60bd96232cd8d8c40e8f9765cc86f46880e43a8fd22dc1a1a8cab1", size = 87528, upload-time = "2023-06-03T06:41:11.019Z" }, +] + +[[package]] +name = "markdown-it-py" +version = "4.2.0" +source = { registry = "https://pypi.org/simple" } +resolution-markers = [ + "python_full_version >= '3.12'", + "python_full_version == '3.11.*'", +] +dependencies = [ + { name = "mdurl", marker = "python_full_version >= '3.11'" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/06/ff/7841249c247aa650a76b9ee4bbaeae59370dc8bfd2f6c01f3630c35eb134/markdown_it_py-4.2.0.tar.gz", hash = "sha256:04a21681d6fbb623de53f6f364d352309d4094dd4194040a10fd51833e418d49", size = 82454, upload-time = "2026-05-07T12:08:28.36Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/b3/81/4da04ced5a082363ecfa159c010d200ecbd959ae410c10c0264a38cac0f5/markdown_it_py-4.2.0-py3-none-any.whl", hash = "sha256:9f7ebbcd14fe59494226453aed97c1070d83f8d24b6fc3a3bcf9a38092641c4a", size = 91687, upload-time = "2026-05-07T12:08:27.182Z" }, +] + +[[package]] +name = "markupsafe" +version = "3.0.3" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/7e/99/7690b6d4034fffd95959cbe0c02de8deb3098cc577c67bb6a24fe5d7caa7/markupsafe-3.0.3.tar.gz", hash = "sha256:722695808f4b6457b320fdc131280796bdceb04ab50fe1795cd540799ebe1698", size = 80313, upload-time = "2025-09-27T18:37:40.426Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/e8/4b/3541d44f3937ba468b75da9eebcae497dcf67adb65caa16760b0a6807ebb/markupsafe-3.0.3-cp310-cp310-macosx_10_9_x86_64.whl", hash = "sha256:2f981d352f04553a7171b8e44369f2af4055f888dfb147d55e42d29e29e74559", size = 11631, upload-time = "2025-09-27T18:36:05.558Z" }, + { url = "https://files.pythonhosted.org/packages/98/1b/fbd8eed11021cabd9226c37342fa6ca4e8a98d8188a8d9b66740494960e4/markupsafe-3.0.3-cp310-cp310-macosx_11_0_arm64.whl", hash = "sha256:e1c1493fb6e50ab01d20a22826e57520f1284df32f2d8601fdd90b6304601419", size = 12057, upload-time = "2025-09-27T18:36:07.165Z" }, + { url = "https://files.pythonhosted.org/packages/40/01/e560d658dc0bb8ab762670ece35281dec7b6c1b33f5fbc09ebb57a185519/markupsafe-3.0.3-cp310-cp310-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:1ba88449deb3de88bd40044603fafffb7bc2b055d626a330323a9ed736661695", size = 22050, upload-time = "2025-09-27T18:36:08.005Z" }, + { url = "https://files.pythonhosted.org/packages/af/cd/ce6e848bbf2c32314c9b237839119c5a564a59725b53157c856e90937b7a/markupsafe-3.0.3-cp310-cp310-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:f42d0984e947b8adf7dd6dde396e720934d12c506ce84eea8476409563607591", size = 20681, upload-time = "2025-09-27T18:36:08.881Z" }, + { url = "https://files.pythonhosted.org/packages/c9/2a/b5c12c809f1c3045c4d580b035a743d12fcde53cf685dbc44660826308da/markupsafe-3.0.3-cp310-cp310-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:c0c0b3ade1c0b13b936d7970b1d37a57acde9199dc2aecc4c336773e1d86049c", size = 20705, upload-time = "2025-09-27T18:36:10.131Z" }, + { url = "https://files.pythonhosted.org/packages/cf/e3/9427a68c82728d0a88c50f890d0fc072a1484de2f3ac1ad0bfc1a7214fd5/markupsafe-3.0.3-cp310-cp310-musllinux_1_2_aarch64.whl", hash = "sha256:0303439a41979d9e74d18ff5e2dd8c43ed6c6001fd40e5bf2e43f7bd9bbc523f", size = 21524, upload-time = "2025-09-27T18:36:11.324Z" }, + { url = "https://files.pythonhosted.org/packages/bc/36/23578f29e9e582a4d0278e009b38081dbe363c5e7165113fad546918a232/markupsafe-3.0.3-cp310-cp310-musllinux_1_2_riscv64.whl", hash = "sha256:d2ee202e79d8ed691ceebae8e0486bd9a2cd4794cec4824e1c99b6f5009502f6", size = 20282, upload-time = "2025-09-27T18:36:12.573Z" }, + { url = "https://files.pythonhosted.org/packages/56/21/dca11354e756ebd03e036bd8ad58d6d7168c80ce1fe5e75218e4945cbab7/markupsafe-3.0.3-cp310-cp310-musllinux_1_2_x86_64.whl", hash = "sha256:177b5253b2834fe3678cb4a5f0059808258584c559193998be2601324fdeafb1", size = 20745, upload-time = "2025-09-27T18:36:13.504Z" }, + { url = "https://files.pythonhosted.org/packages/87/99/faba9369a7ad6e4d10b6a5fbf71fa2a188fe4a593b15f0963b73859a1bbd/markupsafe-3.0.3-cp310-cp310-win32.whl", hash = "sha256:2a15a08b17dd94c53a1da0438822d70ebcd13f8c3a95abe3a9ef9f11a94830aa", size = 14571, upload-time = "2025-09-27T18:36:14.779Z" }, + { url = "https://files.pythonhosted.org/packages/d6/25/55dc3ab959917602c96985cb1253efaa4ff42f71194bddeb61eb7278b8be/markupsafe-3.0.3-cp310-cp310-win_amd64.whl", hash = "sha256:c4ffb7ebf07cfe8931028e3e4c85f0357459a3f9f9490886198848f4fa002ec8", size = 15056, upload-time = "2025-09-27T18:36:16.125Z" }, + { url = "https://files.pythonhosted.org/packages/d0/9e/0a02226640c255d1da0b8d12e24ac2aa6734da68bff14c05dd53b94a0fc3/markupsafe-3.0.3-cp310-cp310-win_arm64.whl", hash = "sha256:e2103a929dfa2fcaf9bb4e7c091983a49c9ac3b19c9061b6d5427dd7d14d81a1", size = 13932, upload-time = "2025-09-27T18:36:17.311Z" }, + { url = "https://files.pythonhosted.org/packages/08/db/fefacb2136439fc8dd20e797950e749aa1f4997ed584c62cfb8ef7c2be0e/markupsafe-3.0.3-cp311-cp311-macosx_10_9_x86_64.whl", hash = "sha256:1cc7ea17a6824959616c525620e387f6dd30fec8cb44f649e31712db02123dad", size = 11631, upload-time = "2025-09-27T18:36:18.185Z" }, + { url = "https://files.pythonhosted.org/packages/e1/2e/5898933336b61975ce9dc04decbc0a7f2fee78c30353c5efba7f2d6ff27a/markupsafe-3.0.3-cp311-cp311-macosx_11_0_arm64.whl", hash = "sha256:4bd4cd07944443f5a265608cc6aab442e4f74dff8088b0dfc8238647b8f6ae9a", size = 12058, upload-time = "2025-09-27T18:36:19.444Z" }, + { url = "https://files.pythonhosted.org/packages/1d/09/adf2df3699d87d1d8184038df46a9c80d78c0148492323f4693df54e17bb/markupsafe-3.0.3-cp311-cp311-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:6b5420a1d9450023228968e7e6a9ce57f65d148ab56d2313fcd589eee96a7a50", size = 24287, upload-time = "2025-09-27T18:36:20.768Z" }, + { url = "https://files.pythonhosted.org/packages/30/ac/0273f6fcb5f42e314c6d8cd99effae6a5354604d461b8d392b5ec9530a54/markupsafe-3.0.3-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:0bf2a864d67e76e5c9a34dc26ec616a66b9888e25e7b9460e1c76d3293bd9dbf", size = 22940, upload-time = "2025-09-27T18:36:22.249Z" }, + { url = "https://files.pythonhosted.org/packages/19/ae/31c1be199ef767124c042c6c3e904da327a2f7f0cd63a0337e1eca2967a8/markupsafe-3.0.3-cp311-cp311-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:bc51efed119bc9cfdf792cdeaa4d67e8f6fcccab66ed4bfdd6bde3e59bfcbb2f", size = 21887, upload-time = "2025-09-27T18:36:23.535Z" }, + { url = "https://files.pythonhosted.org/packages/b2/76/7edcab99d5349a4532a459e1fe64f0b0467a3365056ae550d3bcf3f79e1e/markupsafe-3.0.3-cp311-cp311-musllinux_1_2_aarch64.whl", hash = "sha256:068f375c472b3e7acbe2d5318dea141359e6900156b5b2ba06a30b169086b91a", size = 23692, upload-time = "2025-09-27T18:36:24.823Z" }, + { url = "https://files.pythonhosted.org/packages/a4/28/6e74cdd26d7514849143d69f0bf2399f929c37dc2b31e6829fd2045b2765/markupsafe-3.0.3-cp311-cp311-musllinux_1_2_riscv64.whl", hash = "sha256:7be7b61bb172e1ed687f1754f8e7484f1c8019780f6f6b0786e76bb01c2ae115", size = 21471, upload-time = "2025-09-27T18:36:25.95Z" }, + { url = "https://files.pythonhosted.org/packages/62/7e/a145f36a5c2945673e590850a6f8014318d5577ed7e5920a4b3448e0865d/markupsafe-3.0.3-cp311-cp311-musllinux_1_2_x86_64.whl", hash = "sha256:f9e130248f4462aaa8e2552d547f36ddadbeaa573879158d721bbd33dfe4743a", size = 22923, upload-time = "2025-09-27T18:36:27.109Z" }, + { url = "https://files.pythonhosted.org/packages/0f/62/d9c46a7f5c9adbeeeda52f5b8d802e1094e9717705a645efc71b0913a0a8/markupsafe-3.0.3-cp311-cp311-win32.whl", hash = "sha256:0db14f5dafddbb6d9208827849fad01f1a2609380add406671a26386cdf15a19", size = 14572, upload-time = "2025-09-27T18:36:28.045Z" }, + { url = "https://files.pythonhosted.org/packages/83/8a/4414c03d3f891739326e1783338e48fb49781cc915b2e0ee052aa490d586/markupsafe-3.0.3-cp311-cp311-win_amd64.whl", hash = "sha256:de8a88e63464af587c950061a5e6a67d3632e36df62b986892331d4620a35c01", size = 15077, upload-time = "2025-09-27T18:36:29.025Z" }, + { url = "https://files.pythonhosted.org/packages/35/73/893072b42e6862f319b5207adc9ae06070f095b358655f077f69a35601f0/markupsafe-3.0.3-cp311-cp311-win_arm64.whl", hash = "sha256:3b562dd9e9ea93f13d53989d23a7e775fdfd1066c33494ff43f5418bc8c58a5c", size = 13876, upload-time = "2025-09-27T18:36:29.954Z" }, + { url = "https://files.pythonhosted.org/packages/5a/72/147da192e38635ada20e0a2e1a51cf8823d2119ce8883f7053879c2199b5/markupsafe-3.0.3-cp312-cp312-macosx_10_13_x86_64.whl", hash = "sha256:d53197da72cc091b024dd97249dfc7794d6a56530370992a5e1a08983ad9230e", size = 11615, upload-time = "2025-09-27T18:36:30.854Z" }, + { url = "https://files.pythonhosted.org/packages/9a/81/7e4e08678a1f98521201c3079f77db69fb552acd56067661f8c2f534a718/markupsafe-3.0.3-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:1872df69a4de6aead3491198eaf13810b565bdbeec3ae2dc8780f14458ec73ce", size = 12020, upload-time = "2025-09-27T18:36:31.971Z" }, + { url = "https://files.pythonhosted.org/packages/1e/2c/799f4742efc39633a1b54a92eec4082e4f815314869865d876824c257c1e/markupsafe-3.0.3-cp312-cp312-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:3a7e8ae81ae39e62a41ec302f972ba6ae23a5c5396c8e60113e9066ef893da0d", size = 24332, upload-time = "2025-09-27T18:36:32.813Z" }, + { url = "https://files.pythonhosted.org/packages/3c/2e/8d0c2ab90a8c1d9a24f0399058ab8519a3279d1bd4289511d74e909f060e/markupsafe-3.0.3-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:d6dd0be5b5b189d31db7cda48b91d7e0a9795f31430b7f271219ab30f1d3ac9d", size = 22947, upload-time = "2025-09-27T18:36:33.86Z" }, + { url = "https://files.pythonhosted.org/packages/2c/54/887f3092a85238093a0b2154bd629c89444f395618842e8b0c41783898ea/markupsafe-3.0.3-cp312-cp312-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:94c6f0bb423f739146aec64595853541634bde58b2135f27f61c1ffd1cd4d16a", size = 21962, upload-time = "2025-09-27T18:36:35.099Z" }, + { url = "https://files.pythonhosted.org/packages/c9/2f/336b8c7b6f4a4d95e91119dc8521402461b74a485558d8f238a68312f11c/markupsafe-3.0.3-cp312-cp312-musllinux_1_2_aarch64.whl", hash = "sha256:be8813b57049a7dc738189df53d69395eba14fb99345e0a5994914a3864c8a4b", size = 23760, upload-time = "2025-09-27T18:36:36.001Z" }, + { url = "https://files.pythonhosted.org/packages/32/43/67935f2b7e4982ffb50a4d169b724d74b62a3964bc1a9a527f5ac4f1ee2b/markupsafe-3.0.3-cp312-cp312-musllinux_1_2_riscv64.whl", hash = "sha256:83891d0e9fb81a825d9a6d61e3f07550ca70a076484292a70fde82c4b807286f", size = 21529, upload-time = "2025-09-27T18:36:36.906Z" }, + { url = "https://files.pythonhosted.org/packages/89/e0/4486f11e51bbba8b0c041098859e869e304d1c261e59244baa3d295d47b7/markupsafe-3.0.3-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:77f0643abe7495da77fb436f50f8dab76dbc6e5fd25d39589a0f1fe6548bfa2b", size = 23015, upload-time = "2025-09-27T18:36:37.868Z" }, + { url = "https://files.pythonhosted.org/packages/2f/e1/78ee7a023dac597a5825441ebd17170785a9dab23de95d2c7508ade94e0e/markupsafe-3.0.3-cp312-cp312-win32.whl", hash = "sha256:d88b440e37a16e651bda4c7c2b930eb586fd15ca7406cb39e211fcff3bf3017d", size = 14540, upload-time = "2025-09-27T18:36:38.761Z" }, + { url = "https://files.pythonhosted.org/packages/aa/5b/bec5aa9bbbb2c946ca2733ef9c4ca91c91b6a24580193e891b5f7dbe8e1e/markupsafe-3.0.3-cp312-cp312-win_amd64.whl", hash = "sha256:26a5784ded40c9e318cfc2bdb30fe164bdb8665ded9cd64d500a34fb42067b1c", size = 15105, upload-time = "2025-09-27T18:36:39.701Z" }, + { url = "https://files.pythonhosted.org/packages/e5/f1/216fc1bbfd74011693a4fd837e7026152e89c4bcf3e77b6692fba9923123/markupsafe-3.0.3-cp312-cp312-win_arm64.whl", hash = "sha256:35add3b638a5d900e807944a078b51922212fb3dedb01633a8defc4b01a3c85f", size = 13906, upload-time = "2025-09-27T18:36:40.689Z" }, + { url = "https://files.pythonhosted.org/packages/38/2f/907b9c7bbba283e68f20259574b13d005c121a0fa4c175f9bed27c4597ff/markupsafe-3.0.3-cp313-cp313-macosx_10_13_x86_64.whl", hash = "sha256:e1cf1972137e83c5d4c136c43ced9ac51d0e124706ee1c8aa8532c1287fa8795", size = 11622, upload-time = "2025-09-27T18:36:41.777Z" }, + { url = "https://files.pythonhosted.org/packages/9c/d9/5f7756922cdd676869eca1c4e3c0cd0df60ed30199ffd775e319089cb3ed/markupsafe-3.0.3-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:116bb52f642a37c115f517494ea5feb03889e04df47eeff5b130b1808ce7c219", size = 12029, upload-time = "2025-09-27T18:36:43.257Z" }, + { url = "https://files.pythonhosted.org/packages/00/07/575a68c754943058c78f30db02ee03a64b3c638586fba6a6dd56830b30a3/markupsafe-3.0.3-cp313-cp313-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:133a43e73a802c5562be9bbcd03d090aa5a1fe899db609c29e8c8d815c5f6de6", size = 24374, upload-time = "2025-09-27T18:36:44.508Z" }, + { url = "https://files.pythonhosted.org/packages/a9/21/9b05698b46f218fc0e118e1f8168395c65c8a2c750ae2bab54fc4bd4e0e8/markupsafe-3.0.3-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:ccfcd093f13f0f0b7fdd0f198b90053bf7b2f02a3927a30e63f3ccc9df56b676", size = 22980, upload-time = "2025-09-27T18:36:45.385Z" }, + { url = "https://files.pythonhosted.org/packages/7f/71/544260864f893f18b6827315b988c146b559391e6e7e8f7252839b1b846a/markupsafe-3.0.3-cp313-cp313-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:509fa21c6deb7a7a273d629cf5ec029bc209d1a51178615ddf718f5918992ab9", size = 21990, upload-time = "2025-09-27T18:36:46.916Z" }, + { url = "https://files.pythonhosted.org/packages/c2/28/b50fc2f74d1ad761af2f5dcce7492648b983d00a65b8c0e0cb457c82ebbe/markupsafe-3.0.3-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:a4afe79fb3de0b7097d81da19090f4df4f8d3a2b3adaa8764138aac2e44f3af1", size = 23784, upload-time = "2025-09-27T18:36:47.884Z" }, + { url = "https://files.pythonhosted.org/packages/ed/76/104b2aa106a208da8b17a2fb72e033a5a9d7073c68f7e508b94916ed47a9/markupsafe-3.0.3-cp313-cp313-musllinux_1_2_riscv64.whl", hash = "sha256:795e7751525cae078558e679d646ae45574b47ed6e7771863fcc079a6171a0fc", size = 21588, upload-time = "2025-09-27T18:36:48.82Z" }, + { url = "https://files.pythonhosted.org/packages/b5/99/16a5eb2d140087ebd97180d95249b00a03aa87e29cc224056274f2e45fd6/markupsafe-3.0.3-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:8485f406a96febb5140bfeca44a73e3ce5116b2501ac54fe953e488fb1d03b12", size = 23041, upload-time = "2025-09-27T18:36:49.797Z" }, + { url = "https://files.pythonhosted.org/packages/19/bc/e7140ed90c5d61d77cea142eed9f9c303f4c4806f60a1044c13e3f1471d0/markupsafe-3.0.3-cp313-cp313-win32.whl", hash = "sha256:bdd37121970bfd8be76c5fb069c7751683bdf373db1ed6c010162b2a130248ed", size = 14543, upload-time = "2025-09-27T18:36:51.584Z" }, + { url = "https://files.pythonhosted.org/packages/05/73/c4abe620b841b6b791f2edc248f556900667a5a1cf023a6646967ae98335/markupsafe-3.0.3-cp313-cp313-win_amd64.whl", hash = "sha256:9a1abfdc021a164803f4d485104931fb8f8c1efd55bc6b748d2f5774e78b62c5", size = 15113, upload-time = "2025-09-27T18:36:52.537Z" }, + { url = "https://files.pythonhosted.org/packages/f0/3a/fa34a0f7cfef23cf9500d68cb7c32dd64ffd58a12b09225fb03dd37d5b80/markupsafe-3.0.3-cp313-cp313-win_arm64.whl", hash = "sha256:7e68f88e5b8799aa49c85cd116c932a1ac15caaa3f5db09087854d218359e485", size = 13911, upload-time = "2025-09-27T18:36:53.513Z" }, + { url = "https://files.pythonhosted.org/packages/e4/d7/e05cd7efe43a88a17a37b3ae96e79a19e846f3f456fe79c57ca61356ef01/markupsafe-3.0.3-cp313-cp313t-macosx_10_13_x86_64.whl", hash = "sha256:218551f6df4868a8d527e3062d0fb968682fe92054e89978594c28e642c43a73", size = 11658, upload-time = "2025-09-27T18:36:54.819Z" }, + { url = "https://files.pythonhosted.org/packages/99/9e/e412117548182ce2148bdeacdda3bb494260c0b0184360fe0d56389b523b/markupsafe-3.0.3-cp313-cp313t-macosx_11_0_arm64.whl", hash = "sha256:3524b778fe5cfb3452a09d31e7b5adefeea8c5be1d43c4f810ba09f2ceb29d37", size = 12066, upload-time = "2025-09-27T18:36:55.714Z" }, + { url = "https://files.pythonhosted.org/packages/bc/e6/fa0ffcda717ef64a5108eaa7b4f5ed28d56122c9a6d70ab8b72f9f715c80/markupsafe-3.0.3-cp313-cp313t-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:4e885a3d1efa2eadc93c894a21770e4bc67899e3543680313b09f139e149ab19", size = 25639, upload-time = "2025-09-27T18:36:56.908Z" }, + { url = "https://files.pythonhosted.org/packages/96/ec/2102e881fe9d25fc16cb4b25d5f5cde50970967ffa5dddafdb771237062d/markupsafe-3.0.3-cp313-cp313t-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:8709b08f4a89aa7586de0aadc8da56180242ee0ada3999749b183aa23df95025", size = 23569, upload-time = "2025-09-27T18:36:57.913Z" }, + { url = "https://files.pythonhosted.org/packages/4b/30/6f2fce1f1f205fc9323255b216ca8a235b15860c34b6798f810f05828e32/markupsafe-3.0.3-cp313-cp313t-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:b8512a91625c9b3da6f127803b166b629725e68af71f8184ae7e7d54686a56d6", size = 23284, upload-time = "2025-09-27T18:36:58.833Z" }, + { url = "https://files.pythonhosted.org/packages/58/47/4a0ccea4ab9f5dcb6f79c0236d954acb382202721e704223a8aafa38b5c8/markupsafe-3.0.3-cp313-cp313t-musllinux_1_2_aarch64.whl", hash = "sha256:9b79b7a16f7fedff2495d684f2b59b0457c3b493778c9eed31111be64d58279f", size = 24801, upload-time = "2025-09-27T18:36:59.739Z" }, + { url = "https://files.pythonhosted.org/packages/6a/70/3780e9b72180b6fecb83a4814d84c3bf4b4ae4bf0b19c27196104149734c/markupsafe-3.0.3-cp313-cp313t-musllinux_1_2_riscv64.whl", hash = "sha256:12c63dfb4a98206f045aa9563db46507995f7ef6d83b2f68eda65c307c6829eb", size = 22769, upload-time = "2025-09-27T18:37:00.719Z" }, + { url = "https://files.pythonhosted.org/packages/98/c5/c03c7f4125180fc215220c035beac6b9cb684bc7a067c84fc69414d315f5/markupsafe-3.0.3-cp313-cp313t-musllinux_1_2_x86_64.whl", hash = "sha256:8f71bc33915be5186016f675cd83a1e08523649b0e33efdb898db577ef5bb009", size = 23642, upload-time = "2025-09-27T18:37:01.673Z" }, + { url = "https://files.pythonhosted.org/packages/80/d6/2d1b89f6ca4bff1036499b1e29a1d02d282259f3681540e16563f27ebc23/markupsafe-3.0.3-cp313-cp313t-win32.whl", hash = "sha256:69c0b73548bc525c8cb9a251cddf1931d1db4d2258e9599c28c07ef3580ef354", size = 14612, upload-time = "2025-09-27T18:37:02.639Z" }, + { url = "https://files.pythonhosted.org/packages/2b/98/e48a4bfba0a0ffcf9925fe2d69240bfaa19c6f7507b8cd09c70684a53c1e/markupsafe-3.0.3-cp313-cp313t-win_amd64.whl", hash = "sha256:1b4b79e8ebf6b55351f0d91fe80f893b4743f104bff22e90697db1590e47a218", size = 15200, upload-time = "2025-09-27T18:37:03.582Z" }, + { url = "https://files.pythonhosted.org/packages/0e/72/e3cc540f351f316e9ed0f092757459afbc595824ca724cbc5a5d4263713f/markupsafe-3.0.3-cp313-cp313t-win_arm64.whl", hash = "sha256:ad2cf8aa28b8c020ab2fc8287b0f823d0a7d8630784c31e9ee5edea20f406287", size = 13973, upload-time = "2025-09-27T18:37:04.929Z" }, + { url = "https://files.pythonhosted.org/packages/33/8a/8e42d4838cd89b7dde187011e97fe6c3af66d8c044997d2183fbd6d31352/markupsafe-3.0.3-cp314-cp314-macosx_10_13_x86_64.whl", hash = "sha256:eaa9599de571d72e2daf60164784109f19978b327a3910d3e9de8c97b5b70cfe", size = 11619, upload-time = "2025-09-27T18:37:06.342Z" }, + { url = "https://files.pythonhosted.org/packages/b5/64/7660f8a4a8e53c924d0fa05dc3a55c9cee10bbd82b11c5afb27d44b096ce/markupsafe-3.0.3-cp314-cp314-macosx_11_0_arm64.whl", hash = "sha256:c47a551199eb8eb2121d4f0f15ae0f923d31350ab9280078d1e5f12b249e0026", size = 12029, upload-time = "2025-09-27T18:37:07.213Z" }, + { url = "https://files.pythonhosted.org/packages/da/ef/e648bfd021127bef5fa12e1720ffed0c6cbb8310c8d9bea7266337ff06de/markupsafe-3.0.3-cp314-cp314-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:f34c41761022dd093b4b6896d4810782ffbabe30f2d443ff5f083e0cbbb8c737", size = 24408, upload-time = "2025-09-27T18:37:09.572Z" }, + { url = "https://files.pythonhosted.org/packages/41/3c/a36c2450754618e62008bf7435ccb0f88053e07592e6028a34776213d877/markupsafe-3.0.3-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:457a69a9577064c05a97c41f4e65148652db078a3a509039e64d3467b9e7ef97", size = 23005, upload-time = "2025-09-27T18:37:10.58Z" }, + { url = "https://files.pythonhosted.org/packages/bc/20/b7fdf89a8456b099837cd1dc21974632a02a999ec9bf7ca3e490aacd98e7/markupsafe-3.0.3-cp314-cp314-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:e8afc3f2ccfa24215f8cb28dcf43f0113ac3c37c2f0f0806d8c70e4228c5cf4d", size = 22048, upload-time = "2025-09-27T18:37:11.547Z" }, + { url = "https://files.pythonhosted.org/packages/9a/a7/591f592afdc734f47db08a75793a55d7fbcc6902a723ae4cfbab61010cc5/markupsafe-3.0.3-cp314-cp314-musllinux_1_2_aarch64.whl", hash = "sha256:ec15a59cf5af7be74194f7ab02d0f59a62bdcf1a537677ce67a2537c9b87fcda", size = 23821, upload-time = "2025-09-27T18:37:12.48Z" }, + { url = "https://files.pythonhosted.org/packages/7d/33/45b24e4f44195b26521bc6f1a82197118f74df348556594bd2262bda1038/markupsafe-3.0.3-cp314-cp314-musllinux_1_2_riscv64.whl", hash = "sha256:0eb9ff8191e8498cca014656ae6b8d61f39da5f95b488805da4bb029cccbfbaf", size = 21606, upload-time = "2025-09-27T18:37:13.485Z" }, + { url = "https://files.pythonhosted.org/packages/ff/0e/53dfaca23a69fbfbbf17a4b64072090e70717344c52eaaaa9c5ddff1e5f0/markupsafe-3.0.3-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:2713baf880df847f2bece4230d4d094280f4e67b1e813eec43b4c0e144a34ffe", size = 23043, upload-time = "2025-09-27T18:37:14.408Z" }, + { url = "https://files.pythonhosted.org/packages/46/11/f333a06fc16236d5238bfe74daccbca41459dcd8d1fa952e8fbd5dccfb70/markupsafe-3.0.3-cp314-cp314-win32.whl", hash = "sha256:729586769a26dbceff69f7a7dbbf59ab6572b99d94576a5592625d5b411576b9", size = 14747, upload-time = "2025-09-27T18:37:15.36Z" }, + { url = "https://files.pythonhosted.org/packages/28/52/182836104b33b444e400b14f797212f720cbc9ed6ba34c800639d154e821/markupsafe-3.0.3-cp314-cp314-win_amd64.whl", hash = "sha256:bdc919ead48f234740ad807933cdf545180bfbe9342c2bb451556db2ed958581", size = 15341, upload-time = "2025-09-27T18:37:16.496Z" }, + { url = "https://files.pythonhosted.org/packages/6f/18/acf23e91bd94fd7b3031558b1f013adfa21a8e407a3fdb32745538730382/markupsafe-3.0.3-cp314-cp314-win_arm64.whl", hash = "sha256:5a7d5dc5140555cf21a6fefbdbf8723f06fcd2f63ef108f2854de715e4422cb4", size = 14073, upload-time = "2025-09-27T18:37:17.476Z" }, + { url = "https://files.pythonhosted.org/packages/3c/f0/57689aa4076e1b43b15fdfa646b04653969d50cf30c32a102762be2485da/markupsafe-3.0.3-cp314-cp314t-macosx_10_13_x86_64.whl", hash = "sha256:1353ef0c1b138e1907ae78e2f6c63ff67501122006b0f9abad68fda5f4ffc6ab", size = 11661, upload-time = "2025-09-27T18:37:18.453Z" }, + { url = "https://files.pythonhosted.org/packages/89/c3/2e67a7ca217c6912985ec766c6393b636fb0c2344443ff9d91404dc4c79f/markupsafe-3.0.3-cp314-cp314t-macosx_11_0_arm64.whl", hash = "sha256:1085e7fbddd3be5f89cc898938f42c0b3c711fdcb37d75221de2666af647c175", size = 12069, upload-time = "2025-09-27T18:37:19.332Z" }, + { url = "https://files.pythonhosted.org/packages/f0/00/be561dce4e6ca66b15276e184ce4b8aec61fe83662cce2f7d72bd3249d28/markupsafe-3.0.3-cp314-cp314t-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:1b52b4fb9df4eb9ae465f8d0c228a00624de2334f216f178a995ccdcf82c4634", size = 25670, upload-time = "2025-09-27T18:37:20.245Z" }, + { url = "https://files.pythonhosted.org/packages/50/09/c419f6f5a92e5fadde27efd190eca90f05e1261b10dbd8cbcb39cd8ea1dc/markupsafe-3.0.3-cp314-cp314t-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:fed51ac40f757d41b7c48425901843666a6677e3e8eb0abcff09e4ba6e664f50", size = 23598, upload-time = "2025-09-27T18:37:21.177Z" }, + { url = "https://files.pythonhosted.org/packages/22/44/a0681611106e0b2921b3033fc19bc53323e0b50bc70cffdd19f7d679bb66/markupsafe-3.0.3-cp314-cp314t-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:f190daf01f13c72eac4efd5c430a8de82489d9cff23c364c3ea822545032993e", size = 23261, upload-time = "2025-09-27T18:37:22.167Z" }, + { url = "https://files.pythonhosted.org/packages/5f/57/1b0b3f100259dc9fffe780cfb60d4be71375510e435efec3d116b6436d43/markupsafe-3.0.3-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:e56b7d45a839a697b5eb268c82a71bd8c7f6c94d6fd50c3d577fa39a9f1409f5", size = 24835, upload-time = "2025-09-27T18:37:23.296Z" }, + { url = "https://files.pythonhosted.org/packages/26/6a/4bf6d0c97c4920f1597cc14dd720705eca0bf7c787aebc6bb4d1bead5388/markupsafe-3.0.3-cp314-cp314t-musllinux_1_2_riscv64.whl", hash = "sha256:f3e98bb3798ead92273dc0e5fd0f31ade220f59a266ffd8a4f6065e0a3ce0523", size = 22733, upload-time = "2025-09-27T18:37:24.237Z" }, + { url = "https://files.pythonhosted.org/packages/14/c7/ca723101509b518797fedc2fdf79ba57f886b4aca8a7d31857ba3ee8281f/markupsafe-3.0.3-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:5678211cb9333a6468fb8d8be0305520aa073f50d17f089b5b4b477ea6e67fdc", size = 23672, upload-time = "2025-09-27T18:37:25.271Z" }, + { url = "https://files.pythonhosted.org/packages/fb/df/5bd7a48c256faecd1d36edc13133e51397e41b73bb77e1a69deab746ebac/markupsafe-3.0.3-cp314-cp314t-win32.whl", hash = "sha256:915c04ba3851909ce68ccc2b8e2cd691618c4dc4c4232fb7982bca3f41fd8c3d", size = 14819, upload-time = "2025-09-27T18:37:26.285Z" }, + { url = "https://files.pythonhosted.org/packages/1a/8a/0402ba61a2f16038b48b39bccca271134be00c5c9f0f623208399333c448/markupsafe-3.0.3-cp314-cp314t-win_amd64.whl", hash = "sha256:4faffd047e07c38848ce017e8725090413cd80cbc23d86e55c587bf979e579c9", size = 15426, upload-time = "2025-09-27T18:37:27.316Z" }, + { url = "https://files.pythonhosted.org/packages/70/bc/6f1c2f612465f5fa89b95bead1f44dcb607670fd42891d8fdcd5d039f4f4/markupsafe-3.0.3-cp314-cp314t-win_arm64.whl", hash = "sha256:32001d6a8fc98c8cb5c947787c5d08b0a50663d139f1305bac5885d98d9b40fa", size = 14146, upload-time = "2025-09-27T18:37:28.327Z" }, + { url = "https://files.pythonhosted.org/packages/56/23/0d8c13a44bde9154821586520840643467aee574d8ce79a17da539ee7fed/markupsafe-3.0.3-cp39-cp39-macosx_10_9_x86_64.whl", hash = "sha256:15d939a21d546304880945ca1ecb8a039db6b4dc49b2c5a400387cdae6a62e26", size = 11623, upload-time = "2025-09-27T18:37:29.296Z" }, + { url = "https://files.pythonhosted.org/packages/fd/23/07a2cb9a8045d5f3f0890a8c3bc0859d7a47bfd9a560b563899bec7b72ed/markupsafe-3.0.3-cp39-cp39-macosx_11_0_arm64.whl", hash = "sha256:f71a396b3bf33ecaa1626c255855702aca4d3d9fea5e051b41ac59a9c1c41edc", size = 12049, upload-time = "2025-09-27T18:37:30.234Z" }, + { url = "https://files.pythonhosted.org/packages/bc/e4/6be85eb81503f8e11b61c0b6369b6e077dcf0a74adbd9ebf6b349937b4e9/markupsafe-3.0.3-cp39-cp39-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:0f4b68347f8c5eab4a13419215bdfd7f8c9b19f2b25520968adfad23eb0ce60c", size = 21923, upload-time = "2025-09-27T18:37:31.177Z" }, + { url = "https://files.pythonhosted.org/packages/6f/bc/4dc914ead3fe6ddaef035341fee0fc956949bbd27335b611829292b89ee2/markupsafe-3.0.3-cp39-cp39-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:e8fc20152abba6b83724d7ff268c249fa196d8259ff481f3b1476383f8f24e42", size = 20543, upload-time = "2025-09-27T18:37:32.168Z" }, + { url = "https://files.pythonhosted.org/packages/89/6e/5fe81fbcfba4aef4093d5f856e5c774ec2057946052d18d168219b7bd9f9/markupsafe-3.0.3-cp39-cp39-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:949b8d66bc381ee8b007cd945914c721d9aba8e27f71959d750a46f7c282b20b", size = 20585, upload-time = "2025-09-27T18:37:33.166Z" }, + { url = "https://files.pythonhosted.org/packages/f6/f6/e0e5a3d3ae9c4020f696cd055f940ef86b64fe88de26f3a0308b9d3d048c/markupsafe-3.0.3-cp39-cp39-musllinux_1_2_aarch64.whl", hash = "sha256:3537e01efc9d4dccdf77221fb1cb3b8e1a38d5428920e0657ce299b20324d758", size = 21387, upload-time = "2025-09-27T18:37:34.185Z" }, + { url = "https://files.pythonhosted.org/packages/c8/25/651753ef4dea08ea790f4fbb65146a9a44a014986996ca40102e237aa49a/markupsafe-3.0.3-cp39-cp39-musllinux_1_2_riscv64.whl", hash = "sha256:591ae9f2a647529ca990bc681daebdd52c8791ff06c2bfa05b65163e28102ef2", size = 20133, upload-time = "2025-09-27T18:37:35.138Z" }, + { url = "https://files.pythonhosted.org/packages/dc/0a/c3cf2b4fef5f0426e8a6d7fce3cb966a17817c568ce59d76b92a233fdbec/markupsafe-3.0.3-cp39-cp39-musllinux_1_2_x86_64.whl", hash = "sha256:a320721ab5a1aba0a233739394eb907f8c8da5c98c9181d1161e77a0c8e36f2d", size = 20588, upload-time = "2025-09-27T18:37:36.096Z" }, + { url = "https://files.pythonhosted.org/packages/cd/1b/a7782984844bd519ad4ffdbebbba2671ec5d0ebbeac34736c15fb86399e8/markupsafe-3.0.3-cp39-cp39-win32.whl", hash = "sha256:df2449253ef108a379b8b5d6b43f4b1a8e81a061d6537becd5582fba5f9196d7", size = 14566, upload-time = "2025-09-27T18:37:37.09Z" }, + { url = "https://files.pythonhosted.org/packages/18/1f/8d9c20e1c9440e215a44be5ab64359e207fcb4f675543f1cf9a2a7f648d0/markupsafe-3.0.3-cp39-cp39-win_amd64.whl", hash = "sha256:7c3fb7d25180895632e5d3148dbdc29ea38ccb7fd210aa27acbd1201a1902c6e", size = 15053, upload-time = "2025-09-27T18:37:38.054Z" }, + { url = "https://files.pythonhosted.org/packages/4e/d3/fe08482b5cd995033556d45041a4f4e76e7f0521112a9c9991d40d39825f/markupsafe-3.0.3-cp39-cp39-win_arm64.whl", hash = "sha256:38664109c14ffc9e7437e86b4dceb442b0096dfe3541d7864d9cbe1da4cf36c8", size = 13928, upload-time = "2025-09-27T18:37:39.037Z" }, +] + +[[package]] +name = "mdit-py-plugins" +version = "0.4.2" +source = { registry = "https://pypi.org/simple" } +resolution-markers = [ + "python_full_version < '3.10'", +] +dependencies = [ + { name = "markdown-it-py", version = "3.0.0", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.10'" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/19/03/a2ecab526543b152300717cf232bb4bb8605b6edb946c845016fa9c9c9fd/mdit_py_plugins-0.4.2.tar.gz", hash = "sha256:5f2cd1fdb606ddf152d37ec30e46101a60512bc0e5fa1a7002c36647b09e26b5", size = 43542, upload-time = "2024-09-09T20:27:49.564Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/a7/f7/7782a043553ee469c1ff49cfa1cdace2d6bf99a1f333cf38676b3ddf30da/mdit_py_plugins-0.4.2-py3-none-any.whl", hash = "sha256:0c673c3f889399a33b95e88d2f0d111b4447bdfea7f237dab2d488f459835636", size = 55316, upload-time = "2024-09-09T20:27:48.397Z" }, +] + +[[package]] +name = "mdit-py-plugins" +version = "0.6.1" +source = { registry = "https://pypi.org/simple" } +resolution-markers = [ + "python_full_version >= '3.12'", + "python_full_version == '3.11.*'", + "python_full_version == '3.10.*'", +] +dependencies = [ + { name = "markdown-it-py", version = "3.0.0", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version == '3.10.*'" }, + { name = "markdown-it-py", version = "4.2.0", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version >= '3.11'" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/59/fc/f8d0863f8862f25602c0404d75568e89fb6b4109804645e5cdfb1be5cf56/mdit_py_plugins-0.6.1.tar.gz", hash = "sha256:a2bca0f039f39dbd35fb74ae1b5f998608c437463371f0ff7f49a19a17a114d0", size = 56114, upload-time = "2026-05-13T09:03:38.91Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/a5/69/6da5581c6a7fede7dc261bf4e67d6adca4196f176b43288b55b3db395b6e/mdit_py_plugins-0.6.1-py3-none-any.whl", hash = "sha256:214c82fb2ac524472ab6a5bcab1de80f73b50443e187f401bfd77efbc7c6481d", size = 66663, upload-time = "2026-05-13T09:03:37.76Z" }, +] + +[[package]] +name = "mdurl" +version = "0.1.2" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/d6/54/cfe61301667036ec958cb99bd3efefba235e65cdeb9c84d24a8293ba1d90/mdurl-0.1.2.tar.gz", hash = "sha256:bb413d29f5eea38f31dd4754dd7377d4465116fb207585f97bf925588687c1ba", size = 8729, upload-time = "2022-08-14T12:40:10.846Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/b3/38/89ba8ad64ae25be8de66a6d463314cf1eb366222074cfda9ee839c56a4b4/mdurl-0.1.2-py3-none-any.whl", hash = "sha256:84008a41e51615a49fc9966191ff91509e3c40b939176e643fd50a5c2196b8f8", size = 9979, upload-time = "2022-08-14T12:40:09.779Z" }, +] + +[[package]] +name = "myst-parser" +version = "3.0.1" +source = { registry = "https://pypi.org/simple" } +resolution-markers = [ + "python_full_version < '3.10'", +] +dependencies = [ + { name = "docutils", version = "0.21.2", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.10'" }, + { name = "jinja2", marker = "python_full_version < '3.10'" }, + { name = "markdown-it-py", version = "3.0.0", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.10'" }, + { name = "mdit-py-plugins", version = "0.4.2", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.10'" }, + { name = "pyyaml", marker = "python_full_version < '3.10'" }, + { name = "sphinx", version = "7.4.7", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.10'" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/49/64/e2f13dac02f599980798c01156393b781aec983b52a6e4057ee58f07c43a/myst_parser-3.0.1.tar.gz", hash = "sha256:88f0cb406cb363b077d176b51c476f62d60604d68a8dcdf4832e080441301a87", size = 92392, upload-time = "2024-04-28T20:22:42.116Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/e2/de/21aa8394f16add8f7427f0a1326ccd2b3a2a8a3245c9252bc5ac034c6155/myst_parser-3.0.1-py3-none-any.whl", hash = "sha256:6457aaa33a5d474aca678b8ead9b3dc298e89c68e67012e73146ea6fd54babf1", size = 83163, upload-time = "2024-04-28T20:22:39.985Z" }, +] + +[[package]] +name = "myst-parser" +version = "4.0.1" +source = { registry = "https://pypi.org/simple" } +resolution-markers = [ + "python_full_version == '3.10.*'", +] +dependencies = [ + { name = "docutils", version = "0.21.2", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version == '3.10.*'" }, + { name = "jinja2", marker = "python_full_version == '3.10.*'" }, + { name = "markdown-it-py", version = "3.0.0", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version == '3.10.*'" }, + { name = "mdit-py-plugins", version = "0.6.1", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version == '3.10.*'" }, + { name = "pyyaml", marker = "python_full_version == '3.10.*'" }, + { name = "sphinx", version = "8.1.3", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version == '3.10.*'" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/66/a5/9626ba4f73555b3735ad86247a8077d4603aa8628537687c839ab08bfe44/myst_parser-4.0.1.tar.gz", hash = "sha256:5cfea715e4f3574138aecbf7d54132296bfd72bb614d31168f48c477a830a7c4", size = 93985, upload-time = "2025-02-12T10:53:03.833Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/5f/df/76d0321c3797b54b60fef9ec3bd6f4cfd124b9e422182156a1dd418722cf/myst_parser-4.0.1-py3-none-any.whl", hash = "sha256:9134e88959ec3b5780aedf8a99680ea242869d012e8821db3126d427edc9c95d", size = 84579, upload-time = "2025-02-12T10:53:02.078Z" }, +] + +[[package]] +name = "myst-parser" +version = "5.1.0" +source = { registry = "https://pypi.org/simple" } +resolution-markers = [ + "python_full_version >= '3.12'", + "python_full_version == '3.11.*'", +] +dependencies = [ + { name = "docutils", version = "0.22.4", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version >= '3.11'" }, + { name = "jinja2", marker = "python_full_version >= '3.11'" }, + { name = "markdown-it-py", version = "4.2.0", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version >= '3.11'" }, + { name = "mdit-py-plugins", version = "0.6.1", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version >= '3.11'" }, + { name = "pyyaml", marker = "python_full_version >= '3.11'" }, + { name = "sphinx", version = "9.0.4", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version == '3.11.*'" }, + { name = "sphinx", version = "9.1.0", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version >= '3.12'" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/21/dc/603751677fff302f34396e206b610f556a59d7fe58b9a2145f54e96b48e8/myst_parser-5.1.0.tar.gz", hash = "sha256:ab69322dc6719dcc7f296479dbb70181b66df6ed315064f92dbc85c0e1bf2f02", size = 101182, upload-time = "2026-05-13T09:38:19.361Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/09/dc/f3dfb7488b770f3f67e6545085bf2abea5172e88f57b8ad25ef860ca704c/myst_parser-5.1.0-py3-none-any.whl", hash = "sha256:9c91c52b3cdb4d94a6506e4fab4e2f296c7623a0da0dcbe6de1565c3dad67a8a", size = 85817, upload-time = "2026-05-13T09:38:17.904Z" }, +] + +[[package]] +name = "packaging" +version = "26.2" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/d7/f1/e7a6dd94a8d4a5626c03e4e99c87f241ba9e350cd9e6d75123f992427270/packaging-26.2.tar.gz", hash = "sha256:ff452ff5a3e828ce110190feff1178bb1f2ea2281fa2075aadb987c2fb221661", size = 228134, upload-time = "2026-04-24T20:15:23.917Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/df/b2/87e62e8c3e2f4b32e5fe99e0b86d576da1312593b39f47d8ceef365e95ed/packaging-26.2-py3-none-any.whl", hash = "sha256:5fc45236b9446107ff2415ce77c807cee2862cb6fac22b8a73826d0693b0980e", size = 100195, upload-time = "2026-04-24T20:15:22.081Z" }, +] + +[[package]] +name = "pygments" +version = "2.20.0" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/c3/b2/bc9c9196916376152d655522fdcebac55e66de6603a76a02bca1b6414f6c/pygments-2.20.0.tar.gz", hash = "sha256:6757cd03768053ff99f3039c1a36d6c0aa0b263438fcab17520b30a303a82b5f", size = 4955991, upload-time = "2026-03-29T13:29:33.898Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/f4/7e/a72dd26f3b0f4f2bf1dd8923c85f7ceb43172af56d63c7383eb62b332364/pygments-2.20.0-py3-none-any.whl", hash = "sha256:81a9e26dd42fd28a23a2d169d86d7ac03b46e2f8b59ed4698fb4785f946d0176", size = 1231151, upload-time = "2026-03-29T13:29:30.038Z" }, +] + +[[package]] +name = "pyyaml" +version = "6.0.3" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/05/8e/961c0007c59b8dd7729d542c61a4d537767a59645b82a0b521206e1e25c2/pyyaml-6.0.3.tar.gz", hash = "sha256:d76623373421df22fb4cf8817020cbb7ef15c725b9d5e45f17e189bfc384190f", size = 130960, upload-time = "2025-09-25T21:33:16.546Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/f4/a0/39350dd17dd6d6c6507025c0e53aef67a9293a6d37d3511f23ea510d5800/pyyaml-6.0.3-cp310-cp310-macosx_10_13_x86_64.whl", hash = "sha256:214ed4befebe12df36bcc8bc2b64b396ca31be9304b8f59e25c11cf94a4c033b", size = 184227, upload-time = "2025-09-25T21:31:46.04Z" }, + { url = "https://files.pythonhosted.org/packages/05/14/52d505b5c59ce73244f59c7a50ecf47093ce4765f116cdb98286a71eeca2/pyyaml-6.0.3-cp310-cp310-macosx_11_0_arm64.whl", hash = "sha256:02ea2dfa234451bbb8772601d7b8e426c2bfa197136796224e50e35a78777956", size = 174019, upload-time = "2025-09-25T21:31:47.706Z" }, + { url = "https://files.pythonhosted.org/packages/43/f7/0e6a5ae5599c838c696adb4e6330a59f463265bfa1e116cfd1fbb0abaaae/pyyaml-6.0.3-cp310-cp310-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:b30236e45cf30d2b8e7b3e85881719e98507abed1011bf463a8fa23e9c3e98a8", size = 740646, upload-time = "2025-09-25T21:31:49.21Z" }, + { url = "https://files.pythonhosted.org/packages/2f/3a/61b9db1d28f00f8fd0ae760459a5c4bf1b941baf714e207b6eb0657d2578/pyyaml-6.0.3-cp310-cp310-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:66291b10affd76d76f54fad28e22e51719ef9ba22b29e1d7d03d6777a9174198", size = 840793, upload-time = "2025-09-25T21:31:50.735Z" }, + { url = "https://files.pythonhosted.org/packages/7a/1e/7acc4f0e74c4b3d9531e24739e0ab832a5edf40e64fbae1a9c01941cabd7/pyyaml-6.0.3-cp310-cp310-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:9c7708761fccb9397fe64bbc0395abcae8c4bf7b0eac081e12b809bf47700d0b", size = 770293, upload-time = "2025-09-25T21:31:51.828Z" }, + { url = "https://files.pythonhosted.org/packages/8b/ef/abd085f06853af0cd59fa5f913d61a8eab65d7639ff2a658d18a25d6a89d/pyyaml-6.0.3-cp310-cp310-musllinux_1_2_aarch64.whl", hash = "sha256:418cf3f2111bc80e0933b2cd8cd04f286338bb88bdc7bc8e6dd775ebde60b5e0", size = 732872, upload-time = "2025-09-25T21:31:53.282Z" }, + { url = "https://files.pythonhosted.org/packages/1f/15/2bc9c8faf6450a8b3c9fc5448ed869c599c0a74ba2669772b1f3a0040180/pyyaml-6.0.3-cp310-cp310-musllinux_1_2_x86_64.whl", hash = "sha256:5e0b74767e5f8c593e8c9b5912019159ed0533c70051e9cce3e8b6aa699fcd69", size = 758828, upload-time = "2025-09-25T21:31:54.807Z" }, + { url = "https://files.pythonhosted.org/packages/a3/00/531e92e88c00f4333ce359e50c19b8d1de9fe8d581b1534e35ccfbc5f393/pyyaml-6.0.3-cp310-cp310-win32.whl", hash = "sha256:28c8d926f98f432f88adc23edf2e6d4921ac26fb084b028c733d01868d19007e", size = 142415, upload-time = "2025-09-25T21:31:55.885Z" }, + { url = "https://files.pythonhosted.org/packages/2a/fa/926c003379b19fca39dd4634818b00dec6c62d87faf628d1394e137354d4/pyyaml-6.0.3-cp310-cp310-win_amd64.whl", hash = "sha256:bdb2c67c6c1390b63c6ff89f210c8fd09d9a1217a465701eac7316313c915e4c", size = 158561, upload-time = "2025-09-25T21:31:57.406Z" }, + { url = "https://files.pythonhosted.org/packages/6d/16/a95b6757765b7b031c9374925bb718d55e0a9ba8a1b6a12d25962ea44347/pyyaml-6.0.3-cp311-cp311-macosx_10_13_x86_64.whl", hash = "sha256:44edc647873928551a01e7a563d7452ccdebee747728c1080d881d68af7b997e", size = 185826, upload-time = "2025-09-25T21:31:58.655Z" }, + { url = "https://files.pythonhosted.org/packages/16/19/13de8e4377ed53079ee996e1ab0a9c33ec2faf808a4647b7b4c0d46dd239/pyyaml-6.0.3-cp311-cp311-macosx_11_0_arm64.whl", hash = "sha256:652cb6edd41e718550aad172851962662ff2681490a8a711af6a4d288dd96824", size = 175577, upload-time = "2025-09-25T21:32:00.088Z" }, + { url = "https://files.pythonhosted.org/packages/0c/62/d2eb46264d4b157dae1275b573017abec435397aa59cbcdab6fc978a8af4/pyyaml-6.0.3-cp311-cp311-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:10892704fc220243f5305762e276552a0395f7beb4dbf9b14ec8fd43b57f126c", size = 775556, upload-time = "2025-09-25T21:32:01.31Z" }, + { url = "https://files.pythonhosted.org/packages/10/cb/16c3f2cf3266edd25aaa00d6c4350381c8b012ed6f5276675b9eba8d9ff4/pyyaml-6.0.3-cp311-cp311-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:850774a7879607d3a6f50d36d04f00ee69e7fc816450e5f7e58d7f17f1ae5c00", size = 882114, upload-time = "2025-09-25T21:32:03.376Z" }, + { url = "https://files.pythonhosted.org/packages/71/60/917329f640924b18ff085ab889a11c763e0b573da888e8404ff486657602/pyyaml-6.0.3-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:b8bb0864c5a28024fac8a632c443c87c5aa6f215c0b126c449ae1a150412f31d", size = 806638, upload-time = "2025-09-25T21:32:04.553Z" }, + { url = "https://files.pythonhosted.org/packages/dd/6f/529b0f316a9fd167281a6c3826b5583e6192dba792dd55e3203d3f8e655a/pyyaml-6.0.3-cp311-cp311-musllinux_1_2_aarch64.whl", hash = "sha256:1d37d57ad971609cf3c53ba6a7e365e40660e3be0e5175fa9f2365a379d6095a", size = 767463, upload-time = "2025-09-25T21:32:06.152Z" }, + { url = "https://files.pythonhosted.org/packages/f2/6a/b627b4e0c1dd03718543519ffb2f1deea4a1e6d42fbab8021936a4d22589/pyyaml-6.0.3-cp311-cp311-musllinux_1_2_x86_64.whl", hash = "sha256:37503bfbfc9d2c40b344d06b2199cf0e96e97957ab1c1b546fd4f87e53e5d3e4", size = 794986, upload-time = "2025-09-25T21:32:07.367Z" }, + { url = "https://files.pythonhosted.org/packages/45/91/47a6e1c42d9ee337c4839208f30d9f09caa9f720ec7582917b264defc875/pyyaml-6.0.3-cp311-cp311-win32.whl", hash = "sha256:8098f252adfa6c80ab48096053f512f2321f0b998f98150cea9bd23d83e1467b", size = 142543, upload-time = "2025-09-25T21:32:08.95Z" }, + { url = "https://files.pythonhosted.org/packages/da/e3/ea007450a105ae919a72393cb06f122f288ef60bba2dc64b26e2646fa315/pyyaml-6.0.3-cp311-cp311-win_amd64.whl", hash = "sha256:9f3bfb4965eb874431221a3ff3fdcddc7e74e3b07799e0e84ca4a0f867d449bf", size = 158763, upload-time = "2025-09-25T21:32:09.96Z" }, + { url = "https://files.pythonhosted.org/packages/d1/33/422b98d2195232ca1826284a76852ad5a86fe23e31b009c9886b2d0fb8b2/pyyaml-6.0.3-cp312-cp312-macosx_10_13_x86_64.whl", hash = "sha256:7f047e29dcae44602496db43be01ad42fc6f1cc0d8cd6c83d342306c32270196", size = 182063, upload-time = "2025-09-25T21:32:11.445Z" }, + { url = "https://files.pythonhosted.org/packages/89/a0/6cf41a19a1f2f3feab0e9c0b74134aa2ce6849093d5517a0c550fe37a648/pyyaml-6.0.3-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:fc09d0aa354569bc501d4e787133afc08552722d3ab34836a80547331bb5d4a0", size = 173973, upload-time = "2025-09-25T21:32:12.492Z" }, + { url = "https://files.pythonhosted.org/packages/ed/23/7a778b6bd0b9a8039df8b1b1d80e2e2ad78aa04171592c8a5c43a56a6af4/pyyaml-6.0.3-cp312-cp312-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:9149cad251584d5fb4981be1ecde53a1ca46c891a79788c0df828d2f166bda28", size = 775116, upload-time = "2025-09-25T21:32:13.652Z" }, + { url = "https://files.pythonhosted.org/packages/65/30/d7353c338e12baef4ecc1b09e877c1970bd3382789c159b4f89d6a70dc09/pyyaml-6.0.3-cp312-cp312-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:5fdec68f91a0c6739b380c83b951e2c72ac0197ace422360e6d5a959d8d97b2c", size = 844011, upload-time = "2025-09-25T21:32:15.21Z" }, + { url = "https://files.pythonhosted.org/packages/8b/9d/b3589d3877982d4f2329302ef98a8026e7f4443c765c46cfecc8858c6b4b/pyyaml-6.0.3-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:ba1cc08a7ccde2d2ec775841541641e4548226580ab850948cbfda66a1befcdc", size = 807870, upload-time = "2025-09-25T21:32:16.431Z" }, + { url = "https://files.pythonhosted.org/packages/05/c0/b3be26a015601b822b97d9149ff8cb5ead58c66f981e04fedf4e762f4bd4/pyyaml-6.0.3-cp312-cp312-musllinux_1_2_aarch64.whl", hash = "sha256:8dc52c23056b9ddd46818a57b78404882310fb473d63f17b07d5c40421e47f8e", size = 761089, upload-time = "2025-09-25T21:32:17.56Z" }, + { url = "https://files.pythonhosted.org/packages/be/8e/98435a21d1d4b46590d5459a22d88128103f8da4c2d4cb8f14f2a96504e1/pyyaml-6.0.3-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:41715c910c881bc081f1e8872880d3c650acf13dfa8214bad49ed4cede7c34ea", size = 790181, upload-time = "2025-09-25T21:32:18.834Z" }, + { url = "https://files.pythonhosted.org/packages/74/93/7baea19427dcfbe1e5a372d81473250b379f04b1bd3c4c5ff825e2327202/pyyaml-6.0.3-cp312-cp312-win32.whl", hash = "sha256:96b533f0e99f6579b3d4d4995707cf36df9100d67e0c8303a0c55b27b5f99bc5", size = 137658, upload-time = "2025-09-25T21:32:20.209Z" }, + { url = "https://files.pythonhosted.org/packages/86/bf/899e81e4cce32febab4fb42bb97dcdf66bc135272882d1987881a4b519e9/pyyaml-6.0.3-cp312-cp312-win_amd64.whl", hash = "sha256:5fcd34e47f6e0b794d17de1b4ff496c00986e1c83f7ab2fb8fcfe9616ff7477b", size = 154003, upload-time = "2025-09-25T21:32:21.167Z" }, + { url = "https://files.pythonhosted.org/packages/1a/08/67bd04656199bbb51dbed1439b7f27601dfb576fb864099c7ef0c3e55531/pyyaml-6.0.3-cp312-cp312-win_arm64.whl", hash = "sha256:64386e5e707d03a7e172c0701abfb7e10f0fb753ee1d773128192742712a98fd", size = 140344, upload-time = "2025-09-25T21:32:22.617Z" }, + { url = "https://files.pythonhosted.org/packages/d1/11/0fd08f8192109f7169db964b5707a2f1e8b745d4e239b784a5a1dd80d1db/pyyaml-6.0.3-cp313-cp313-macosx_10_13_x86_64.whl", hash = "sha256:8da9669d359f02c0b91ccc01cac4a67f16afec0dac22c2ad09f46bee0697eba8", size = 181669, upload-time = "2025-09-25T21:32:23.673Z" }, + { url = "https://files.pythonhosted.org/packages/b1/16/95309993f1d3748cd644e02e38b75d50cbc0d9561d21f390a76242ce073f/pyyaml-6.0.3-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:2283a07e2c21a2aa78d9c4442724ec1eb15f5e42a723b99cb3d822d48f5f7ad1", size = 173252, upload-time = "2025-09-25T21:32:25.149Z" }, + { url = "https://files.pythonhosted.org/packages/50/31/b20f376d3f810b9b2371e72ef5adb33879b25edb7a6d072cb7ca0c486398/pyyaml-6.0.3-cp313-cp313-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:ee2922902c45ae8ccada2c5b501ab86c36525b883eff4255313a253a3160861c", size = 767081, upload-time = "2025-09-25T21:32:26.575Z" }, + { url = "https://files.pythonhosted.org/packages/49/1e/a55ca81e949270d5d4432fbbd19dfea5321eda7c41a849d443dc92fd1ff7/pyyaml-6.0.3-cp313-cp313-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:a33284e20b78bd4a18c8c2282d549d10bc8408a2a7ff57653c0cf0b9be0afce5", size = 841159, upload-time = "2025-09-25T21:32:27.727Z" }, + { url = "https://files.pythonhosted.org/packages/74/27/e5b8f34d02d9995b80abcef563ea1f8b56d20134d8f4e5e81733b1feceb2/pyyaml-6.0.3-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:0f29edc409a6392443abf94b9cf89ce99889a1dd5376d94316ae5145dfedd5d6", size = 801626, upload-time = "2025-09-25T21:32:28.878Z" }, + { url = "https://files.pythonhosted.org/packages/f9/11/ba845c23988798f40e52ba45f34849aa8a1f2d4af4b798588010792ebad6/pyyaml-6.0.3-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:f7057c9a337546edc7973c0d3ba84ddcdf0daa14533c2065749c9075001090e6", size = 753613, upload-time = "2025-09-25T21:32:30.178Z" }, + { url = "https://files.pythonhosted.org/packages/3d/e0/7966e1a7bfc0a45bf0a7fb6b98ea03fc9b8d84fa7f2229e9659680b69ee3/pyyaml-6.0.3-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:eda16858a3cab07b80edaf74336ece1f986ba330fdb8ee0d6c0d68fe82bc96be", size = 794115, upload-time = "2025-09-25T21:32:31.353Z" }, + { url = "https://files.pythonhosted.org/packages/de/94/980b50a6531b3019e45ddeada0626d45fa85cbe22300844a7983285bed3b/pyyaml-6.0.3-cp313-cp313-win32.whl", hash = "sha256:d0eae10f8159e8fdad514efdc92d74fd8d682c933a6dd088030f3834bc8e6b26", size = 137427, upload-time = "2025-09-25T21:32:32.58Z" }, + { url = "https://files.pythonhosted.org/packages/97/c9/39d5b874e8b28845e4ec2202b5da735d0199dbe5b8fb85f91398814a9a46/pyyaml-6.0.3-cp313-cp313-win_amd64.whl", hash = "sha256:79005a0d97d5ddabfeeea4cf676af11e647e41d81c9a7722a193022accdb6b7c", size = 154090, upload-time = "2025-09-25T21:32:33.659Z" }, + { url = "https://files.pythonhosted.org/packages/73/e8/2bdf3ca2090f68bb3d75b44da7bbc71843b19c9f2b9cb9b0f4ab7a5a4329/pyyaml-6.0.3-cp313-cp313-win_arm64.whl", hash = "sha256:5498cd1645aa724a7c71c8f378eb29ebe23da2fc0d7a08071d89469bf1d2defb", size = 140246, upload-time = "2025-09-25T21:32:34.663Z" }, + { url = "https://files.pythonhosted.org/packages/9d/8c/f4bd7f6465179953d3ac9bc44ac1a8a3e6122cf8ada906b4f96c60172d43/pyyaml-6.0.3-cp314-cp314-macosx_10_13_x86_64.whl", hash = "sha256:8d1fab6bb153a416f9aeb4b8763bc0f22a5586065f86f7664fc23339fc1c1fac", size = 181814, upload-time = "2025-09-25T21:32:35.712Z" }, + { url = "https://files.pythonhosted.org/packages/bd/9c/4d95bb87eb2063d20db7b60faa3840c1b18025517ae857371c4dd55a6b3a/pyyaml-6.0.3-cp314-cp314-macosx_11_0_arm64.whl", hash = "sha256:34d5fcd24b8445fadc33f9cf348c1047101756fd760b4dacb5c3e99755703310", size = 173809, upload-time = "2025-09-25T21:32:36.789Z" }, + { url = "https://files.pythonhosted.org/packages/92/b5/47e807c2623074914e29dabd16cbbdd4bf5e9b2db9f8090fa64411fc5382/pyyaml-6.0.3-cp314-cp314-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:501a031947e3a9025ed4405a168e6ef5ae3126c59f90ce0cd6f2bfc477be31b7", size = 766454, upload-time = "2025-09-25T21:32:37.966Z" }, + { url = "https://files.pythonhosted.org/packages/02/9e/e5e9b168be58564121efb3de6859c452fccde0ab093d8438905899a3a483/pyyaml-6.0.3-cp314-cp314-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:b3bc83488de33889877a0f2543ade9f70c67d66d9ebb4ac959502e12de895788", size = 836355, upload-time = "2025-09-25T21:32:39.178Z" }, + { url = "https://files.pythonhosted.org/packages/88/f9/16491d7ed2a919954993e48aa941b200f38040928474c9e85ea9e64222c3/pyyaml-6.0.3-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:c458b6d084f9b935061bc36216e8a69a7e293a2f1e68bf956dcd9e6cbcd143f5", size = 794175, upload-time = "2025-09-25T21:32:40.865Z" }, + { url = "https://files.pythonhosted.org/packages/dd/3f/5989debef34dc6397317802b527dbbafb2b4760878a53d4166579111411e/pyyaml-6.0.3-cp314-cp314-musllinux_1_2_aarch64.whl", hash = "sha256:7c6610def4f163542a622a73fb39f534f8c101d690126992300bf3207eab9764", size = 755228, upload-time = "2025-09-25T21:32:42.084Z" }, + { url = "https://files.pythonhosted.org/packages/d7/ce/af88a49043cd2e265be63d083fc75b27b6ed062f5f9fd6cdc223ad62f03e/pyyaml-6.0.3-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:5190d403f121660ce8d1d2c1bb2ef1bd05b5f68533fc5c2ea899bd15f4399b35", size = 789194, upload-time = "2025-09-25T21:32:43.362Z" }, + { url = "https://files.pythonhosted.org/packages/23/20/bb6982b26a40bb43951265ba29d4c246ef0ff59c9fdcdf0ed04e0687de4d/pyyaml-6.0.3-cp314-cp314-win_amd64.whl", hash = "sha256:4a2e8cebe2ff6ab7d1050ecd59c25d4c8bd7e6f400f5f82b96557ac0abafd0ac", size = 156429, upload-time = "2025-09-25T21:32:57.844Z" }, + { url = "https://files.pythonhosted.org/packages/f4/f4/a4541072bb9422c8a883ab55255f918fa378ecf083f5b85e87fc2b4eda1b/pyyaml-6.0.3-cp314-cp314-win_arm64.whl", hash = "sha256:93dda82c9c22deb0a405ea4dc5f2d0cda384168e466364dec6255b293923b2f3", size = 143912, upload-time = "2025-09-25T21:32:59.247Z" }, + { url = "https://files.pythonhosted.org/packages/7c/f9/07dd09ae774e4616edf6cda684ee78f97777bdd15847253637a6f052a62f/pyyaml-6.0.3-cp314-cp314t-macosx_10_13_x86_64.whl", hash = "sha256:02893d100e99e03eda1c8fd5c441d8c60103fd175728e23e431db1b589cf5ab3", size = 189108, upload-time = "2025-09-25T21:32:44.377Z" }, + { url = "https://files.pythonhosted.org/packages/4e/78/8d08c9fb7ce09ad8c38ad533c1191cf27f7ae1effe5bb9400a46d9437fcf/pyyaml-6.0.3-cp314-cp314t-macosx_11_0_arm64.whl", hash = "sha256:c1ff362665ae507275af2853520967820d9124984e0f7466736aea23d8611fba", size = 183641, upload-time = "2025-09-25T21:32:45.407Z" }, + { url = "https://files.pythonhosted.org/packages/7b/5b/3babb19104a46945cf816d047db2788bcaf8c94527a805610b0289a01c6b/pyyaml-6.0.3-cp314-cp314t-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:6adc77889b628398debc7b65c073bcb99c4a0237b248cacaf3fe8a557563ef6c", size = 831901, upload-time = "2025-09-25T21:32:48.83Z" }, + { url = "https://files.pythonhosted.org/packages/8b/cc/dff0684d8dc44da4d22a13f35f073d558c268780ce3c6ba1b87055bb0b87/pyyaml-6.0.3-cp314-cp314t-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:a80cb027f6b349846a3bf6d73b5e95e782175e52f22108cfa17876aaeff93702", size = 861132, upload-time = "2025-09-25T21:32:50.149Z" }, + { url = "https://files.pythonhosted.org/packages/b1/5e/f77dc6b9036943e285ba76b49e118d9ea929885becb0a29ba8a7c75e29fe/pyyaml-6.0.3-cp314-cp314t-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:00c4bdeba853cc34e7dd471f16b4114f4162dc03e6b7afcc2128711f0eca823c", size = 839261, upload-time = "2025-09-25T21:32:51.808Z" }, + { url = "https://files.pythonhosted.org/packages/ce/88/a9db1376aa2a228197c58b37302f284b5617f56a5d959fd1763fb1675ce6/pyyaml-6.0.3-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:66e1674c3ef6f541c35191caae2d429b967b99e02040f5ba928632d9a7f0f065", size = 805272, upload-time = "2025-09-25T21:32:52.941Z" }, + { url = "https://files.pythonhosted.org/packages/da/92/1446574745d74df0c92e6aa4a7b0b3130706a4142b2d1a5869f2eaa423c6/pyyaml-6.0.3-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:16249ee61e95f858e83976573de0f5b2893b3677ba71c9dd36b9cf8be9ac6d65", size = 829923, upload-time = "2025-09-25T21:32:54.537Z" }, + { url = "https://files.pythonhosted.org/packages/f0/7a/1c7270340330e575b92f397352af856a8c06f230aa3e76f86b39d01b416a/pyyaml-6.0.3-cp314-cp314t-win_amd64.whl", hash = "sha256:4ad1906908f2f5ae4e5a8ddfce73c320c2a1429ec52eafd27138b7f1cbe341c9", size = 174062, upload-time = "2025-09-25T21:32:55.767Z" }, + { url = "https://files.pythonhosted.org/packages/f1/12/de94a39c2ef588c7e6455cfbe7343d3b2dc9d6b6b2f40c4c6565744c873d/pyyaml-6.0.3-cp314-cp314t-win_arm64.whl", hash = "sha256:ebc55a14a21cb14062aa4162f906cd962b28e2e9ea38f9b4391244cd8de4ae0b", size = 149341, upload-time = "2025-09-25T21:32:56.828Z" }, + { url = "https://files.pythonhosted.org/packages/9f/62/67fc8e68a75f738c9200422bf65693fb79a4cd0dc5b23310e5202e978090/pyyaml-6.0.3-cp39-cp39-macosx_10_13_x86_64.whl", hash = "sha256:b865addae83924361678b652338317d1bd7e79b1f4596f96b96c77a5a34b34da", size = 184450, upload-time = "2025-09-25T21:33:00.618Z" }, + { url = "https://files.pythonhosted.org/packages/ae/92/861f152ce87c452b11b9d0977952259aa7df792d71c1053365cc7b09cc08/pyyaml-6.0.3-cp39-cp39-macosx_11_0_arm64.whl", hash = "sha256:c3355370a2c156cffb25e876646f149d5d68f5e0a3ce86a5084dd0b64a994917", size = 174319, upload-time = "2025-09-25T21:33:02.086Z" }, + { url = "https://files.pythonhosted.org/packages/d0/cd/f0cfc8c74f8a030017a2b9c771b7f47e5dd702c3e28e5b2071374bda2948/pyyaml-6.0.3-cp39-cp39-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:3c5677e12444c15717b902a5798264fa7909e41153cdf9ef7ad571b704a63dd9", size = 737631, upload-time = "2025-09-25T21:33:03.25Z" }, + { url = "https://files.pythonhosted.org/packages/ef/b2/18f2bd28cd2055a79a46c9b0895c0b3d987ce40ee471cecf58a1a0199805/pyyaml-6.0.3-cp39-cp39-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:5ed875a24292240029e4483f9d4a4b8a1ae08843b9c54f43fcc11e404532a8a5", size = 836795, upload-time = "2025-09-25T21:33:05.014Z" }, + { url = "https://files.pythonhosted.org/packages/73/b9/793686b2d54b531203c160ef12bec60228a0109c79bae6c1277961026770/pyyaml-6.0.3-cp39-cp39-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:0150219816b6a1fa26fb4699fb7daa9caf09eb1999f3b70fb6e786805e80375a", size = 750767, upload-time = "2025-09-25T21:33:06.398Z" }, + { url = "https://files.pythonhosted.org/packages/a9/86/a137b39a611def2ed78b0e66ce2fe13ee701a07c07aebe55c340ed2a050e/pyyaml-6.0.3-cp39-cp39-musllinux_1_2_aarch64.whl", hash = "sha256:fa160448684b4e94d80416c0fa4aac48967a969efe22931448d853ada8baf926", size = 727982, upload-time = "2025-09-25T21:33:08.708Z" }, + { url = "https://files.pythonhosted.org/packages/dd/62/71c27c94f457cf4418ef8ccc71735324c549f7e3ea9d34aba50874563561/pyyaml-6.0.3-cp39-cp39-musllinux_1_2_x86_64.whl", hash = "sha256:27c0abcb4a5dac13684a37f76e701e054692a9b2d3064b70f5e4eb54810553d7", size = 755677, upload-time = "2025-09-25T21:33:09.876Z" }, + { url = "https://files.pythonhosted.org/packages/29/3d/6f5e0d58bd924fb0d06c3a6bad00effbdae2de5adb5cda5648006ffbd8d3/pyyaml-6.0.3-cp39-cp39-win32.whl", hash = "sha256:1ebe39cb5fc479422b83de611d14e2c0d3bb2a18bbcb01f229ab3cfbd8fee7a0", size = 142592, upload-time = "2025-09-25T21:33:10.983Z" }, + { url = "https://files.pythonhosted.org/packages/f0/0c/25113e0b5e103d7f1490c0e947e303fe4a696c10b501dea7a9f49d4e876c/pyyaml-6.0.3-cp39-cp39-win_amd64.whl", hash = "sha256:2e71d11abed7344e42a8849600193d15b6def118602c4c176f748e4583246007", size = 158777, upload-time = "2025-09-25T21:33:15.55Z" }, +] + +[[package]] +name = "readthedocs-sphinx-ext" +version = "2.2.5" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "jinja2" }, + { name = "packaging" }, + { name = "requests", version = "2.32.5", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.10'" }, + { name = "requests", version = "2.34.2", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version >= '3.10'" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/e8/ce/38130d8dec600bf5413eb89a3413dd38f204c7c728c4947e12ff8cb793b7/readthedocs-sphinx-ext-2.2.5.tar.gz", hash = "sha256:ee5fd5b99db9f0c180b2396cbce528aa36671951b9526bb0272dbfce5517bd27", size = 12303, upload-time = "2023-12-19T10:00:49.573Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/64/71/c89e7709a0d4f93af1848e9855112299a820b470d84f917b4dd5998bdd07/readthedocs_sphinx_ext-2.2.5-py2.py3-none-any.whl", hash = "sha256:f8c56184ea011c972dd45a90122568587cc85b0127bc9cf064d17c68bc809daa", size = 11332, upload-time = "2023-12-19T10:00:43.972Z" }, +] + +[[package]] +name = "requests" +version = "2.32.5" +source = { registry = "https://pypi.org/simple" } +resolution-markers = [ + "python_full_version < '3.10'", +] +dependencies = [ + { name = "certifi", marker = "python_full_version < '3.10'" }, + { name = "charset-normalizer", marker = "python_full_version < '3.10'" }, + { name = "idna", marker = "python_full_version < '3.10'" }, + { name = "urllib3", version = "2.6.3", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.10'" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/c9/74/b3ff8e6c8446842c3f5c837e9c3dfcfe2018ea6ecef224c710c85ef728f4/requests-2.32.5.tar.gz", hash = "sha256:dbba0bac56e100853db0ea71b82b4dfd5fe2bf6d3754a8893c3af500cec7d7cf", size = 134517, upload-time = "2025-08-18T20:46:02.573Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/1e/db/4254e3eabe8020b458f1a747140d32277ec7a271daf1d235b70dc0b4e6e3/requests-2.32.5-py3-none-any.whl", hash = "sha256:2462f94637a34fd532264295e186976db0f5d453d1cdd31473c85a6a161affb6", size = 64738, upload-time = "2025-08-18T20:46:00.542Z" }, +] + +[[package]] +name = "requests" +version = "2.34.2" +source = { registry = "https://pypi.org/simple" } +resolution-markers = [ + "python_full_version >= '3.12'", + "python_full_version == '3.11.*'", + "python_full_version == '3.10.*'", +] +dependencies = [ + { name = "certifi", marker = "python_full_version >= '3.10'" }, + { name = "charset-normalizer", marker = "python_full_version >= '3.10'" }, + { name = "idna", marker = "python_full_version >= '3.10'" }, + { name = "urllib3", version = "2.7.0", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version >= '3.10'" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/ac/c3/e2a2b89f2d3e2179abd6d00ebd70bff6273f37fb3e0cc209f48b39d00cbf/requests-2.34.2.tar.gz", hash = "sha256:f288924cae4e29463698d6d60bc6a4da69c89185ad1e0bcc4104f584e960b9ed", size = 142856, upload-time = "2026-05-14T19:25:27.735Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/a0/f4/c67b0b3f1b9245e8d266f0f112c500d50e5b4e83cb6f3b71b6528104182a/requests-2.34.2-py3-none-any.whl", hash = "sha256:2a0d60c172f83ac6ab31e4554906c0f3b3588d37b5cb939b1c061f4907e278e0", size = 73075, upload-time = "2026-05-14T19:25:26.443Z" }, +] + +[[package]] +name = "roman-numerals" +version = "4.1.0" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/ae/f9/41dc953bbeb056c17d5f7a519f50fdf010bd0553be2d630bc69d1e022703/roman_numerals-4.1.0.tar.gz", hash = "sha256:1af8b147eb1405d5839e78aeb93131690495fe9da5c91856cb33ad55a7f1e5b2", size = 9077, upload-time = "2025-12-17T18:25:34.381Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/04/54/6f679c435d28e0a568d8e8a7c0a93a09010818634c3c3907fc98d8983770/roman_numerals-4.1.0-py3-none-any.whl", hash = "sha256:647ba99caddc2cc1e55a51e4360689115551bf4476d90e8162cf8c345fe233c7", size = 7676, upload-time = "2025-12-17T18:25:33.098Z" }, +] + +[[package]] +name = "rules-python-sphinxdocs-dev" +version = "0.0.0" +source = { virtual = "." } +dependencies = [ + { name = "absl-py", version = "2.3.1", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.10'" }, + { name = "absl-py", version = "2.5.0", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version >= '3.10'" }, + { name = "markupsafe" }, + { name = "myst-parser", version = "3.0.1", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.10'" }, + { name = "myst-parser", version = "4.0.1", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version == '3.10.*'" }, + { name = "myst-parser", version = "5.1.0", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version >= '3.11'" }, + { name = "readthedocs-sphinx-ext" }, + { name = "sphinx", version = "7.4.7", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.10'" }, + { name = "sphinx", version = "8.1.3", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version == '3.10.*'" }, + { name = "sphinx", version = "9.0.4", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version == '3.11.*'" }, + { name = "sphinx", version = "9.1.0", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version >= '3.12'" }, + { name = "sphinx-autodoc2" }, + { name = "sphinx-reredirects", version = "0.1.6", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.11'" }, + { name = "sphinx-reredirects", version = "1.1.0", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version >= '3.11'" }, + { name = "sphinx-rtd-theme" }, + { name = "typing-extensions" }, +] + +[package.metadata] +requires-dist = [ + { name = "absl-py" }, + { name = "markupsafe" }, + { name = "myst-parser" }, + { name = "readthedocs-sphinx-ext" }, + { name = "sphinx" }, + { name = "sphinx-autodoc2" }, + { name = "sphinx-reredirects" }, + { name = "sphinx-rtd-theme", specifier = ">=2.0" }, + { name = "typing-extensions" }, +] + +[[package]] +name = "snowballstemmer" +version = "3.1.1" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/43/f8/0a71edf031f03c40db17503cb8ca78a69a171254e568e7db241b0ab57ea1/snowballstemmer-3.1.1.tar.gz", hash = "sha256:e07bbc54a0d798fe6010a12398422e62a8bfbba95c394fd0956ef58cb4d3e260", size = 123314, upload-time = "2026-06-03T00:56:40.194Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/4c/07/2ebca9b11fb9be7340a818d8d6f63feaebb146be2c4afbd6061701d6df6e/snowballstemmer-3.1.1-py3-none-any.whl", hash = "sha256:7e207fa178741da09cdee59d3ecec3827ad5f92b1fc5c9ff3755b639f71f5752", size = 104164, upload-time = "2026-06-03T00:56:38.614Z" }, +] + +[[package]] +name = "sphinx" +version = "7.4.7" +source = { registry = "https://pypi.org/simple" } +resolution-markers = [ + "python_full_version < '3.10'", +] +dependencies = [ + { name = "alabaster", version = "0.7.16", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.10'" }, + { name = "babel", marker = "python_full_version < '3.10'" }, + { name = "colorama", marker = "python_full_version < '3.10' and sys_platform == 'win32'" }, + { name = "docutils", version = "0.21.2", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.10'" }, + { name = "imagesize", version = "1.5.0", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.10'" }, + { name = "importlib-metadata", marker = "python_full_version < '3.10'" }, + { name = "jinja2", marker = "python_full_version < '3.10'" }, + { name = "packaging", marker = "python_full_version < '3.10'" }, + { name = "pygments", marker = "python_full_version < '3.10'" }, + { name = "requests", version = "2.32.5", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.10'" }, + { name = "snowballstemmer", marker = "python_full_version < '3.10'" }, + { name = "sphinxcontrib-applehelp", marker = "python_full_version < '3.10'" }, + { name = "sphinxcontrib-devhelp", marker = "python_full_version < '3.10'" }, + { name = "sphinxcontrib-htmlhelp", marker = "python_full_version < '3.10'" }, + { name = "sphinxcontrib-jsmath", marker = "python_full_version < '3.10'" }, + { name = "sphinxcontrib-qthelp", marker = "python_full_version < '3.10'" }, + { name = "sphinxcontrib-serializinghtml", marker = "python_full_version < '3.10'" }, + { name = "tomli", marker = "python_full_version < '3.10'" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/5b/be/50e50cb4f2eff47df05673d361095cafd95521d2a22521b920c67a372dcb/sphinx-7.4.7.tar.gz", hash = "sha256:242f92a7ea7e6c5b406fdc2615413890ba9f699114a9c09192d7dfead2ee9cfe", size = 8067911, upload-time = "2024-07-20T14:46:56.059Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/0d/ef/153f6803c5d5f8917dbb7f7fcf6d34a871ede3296fa89c2c703f5f8a6c8e/sphinx-7.4.7-py3-none-any.whl", hash = "sha256:c2419e2135d11f1951cd994d6eb18a1835bd8fdd8429f9ca375dc1f3281bd239", size = 3401624, upload-time = "2024-07-20T14:46:52.142Z" }, +] + +[[package]] +name = "sphinx" +version = "8.1.3" +source = { registry = "https://pypi.org/simple" } +resolution-markers = [ + "python_full_version == '3.10.*'", +] +dependencies = [ + { name = "alabaster", version = "1.0.0", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version == '3.10.*'" }, + { name = "babel", marker = "python_full_version == '3.10.*'" }, + { name = "colorama", marker = "python_full_version == '3.10.*' and sys_platform == 'win32'" }, + { name = "docutils", version = "0.21.2", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version == '3.10.*'" }, + { name = "imagesize", version = "2.0.0", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version == '3.10.*'" }, + { name = "jinja2", marker = "python_full_version == '3.10.*'" }, + { name = "packaging", marker = "python_full_version == '3.10.*'" }, + { name = "pygments", marker = "python_full_version == '3.10.*'" }, + { name = "requests", version = "2.34.2", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version == '3.10.*'" }, + { name = "snowballstemmer", marker = "python_full_version == '3.10.*'" }, + { name = "sphinxcontrib-applehelp", marker = "python_full_version == '3.10.*'" }, + { name = "sphinxcontrib-devhelp", marker = "python_full_version == '3.10.*'" }, + { name = "sphinxcontrib-htmlhelp", marker = "python_full_version == '3.10.*'" }, + { name = "sphinxcontrib-jsmath", marker = "python_full_version == '3.10.*'" }, + { name = "sphinxcontrib-qthelp", marker = "python_full_version == '3.10.*'" }, + { name = "sphinxcontrib-serializinghtml", marker = "python_full_version == '3.10.*'" }, + { name = "tomli", marker = "python_full_version == '3.10.*'" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/6f/6d/be0b61178fe2cdcb67e2a92fc9ebb488e3c51c4f74a36a7824c0adf23425/sphinx-8.1.3.tar.gz", hash = "sha256:43c1911eecb0d3e161ad78611bc905d1ad0e523e4ddc202a58a821773dc4c927", size = 8184611, upload-time = "2024-10-13T20:27:13.93Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/26/60/1ddff83a56d33aaf6f10ec8ce84b4c007d9368b21008876fceda7e7381ef/sphinx-8.1.3-py3-none-any.whl", hash = "sha256:09719015511837b76bf6e03e42eb7595ac8c2e41eeb9c29c5b755c6b677992a2", size = 3487125, upload-time = "2024-10-13T20:27:10.448Z" }, +] + +[[package]] +name = "sphinx" +version = "9.0.4" +source = { registry = "https://pypi.org/simple" } +resolution-markers = [ + "python_full_version == '3.11.*'", +] +dependencies = [ + { name = "alabaster", version = "1.0.0", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version == '3.11.*'" }, + { name = "babel", marker = "python_full_version == '3.11.*'" }, + { name = "colorama", marker = "python_full_version == '3.11.*' and sys_platform == 'win32'" }, + { name = "docutils", version = "0.22.4", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version == '3.11.*'" }, + { name = "imagesize", version = "2.0.0", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version == '3.11.*'" }, + { name = "jinja2", marker = "python_full_version == '3.11.*'" }, + { name = "packaging", marker = "python_full_version == '3.11.*'" }, + { name = "pygments", marker = "python_full_version == '3.11.*'" }, + { name = "requests", version = "2.34.2", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version == '3.11.*'" }, + { name = "roman-numerals", marker = "python_full_version == '3.11.*'" }, + { name = "snowballstemmer", marker = "python_full_version == '3.11.*'" }, + { name = "sphinxcontrib-applehelp", marker = "python_full_version == '3.11.*'" }, + { name = "sphinxcontrib-devhelp", marker = "python_full_version == '3.11.*'" }, + { name = "sphinxcontrib-htmlhelp", marker = "python_full_version == '3.11.*'" }, + { name = "sphinxcontrib-jsmath", marker = "python_full_version == '3.11.*'" }, + { name = "sphinxcontrib-qthelp", marker = "python_full_version == '3.11.*'" }, + { name = "sphinxcontrib-serializinghtml", marker = "python_full_version == '3.11.*'" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/42/50/a8c6ccc36d5eacdfd7913ddccd15a9cee03ecafc5ee2bc40e1f168d85022/sphinx-9.0.4.tar.gz", hash = "sha256:594ef59d042972abbc581d8baa577404abe4e6c3b04ef61bd7fc2acbd51f3fa3", size = 8710502, upload-time = "2025-12-04T07:45:27.343Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/c6/3f/4bbd76424c393caead2e1eb89777f575dee5c8653e2d4b6afd7a564f5974/sphinx-9.0.4-py3-none-any.whl", hash = "sha256:5bebc595a5e943ea248b99c13814c1c5e10b3ece718976824ffa7959ff95fffb", size = 3917713, upload-time = "2025-12-04T07:45:24.944Z" }, +] + +[[package]] +name = "sphinx" +version = "9.1.0" +source = { registry = "https://pypi.org/simple" } +resolution-markers = [ + "python_full_version >= '3.12'", +] +dependencies = [ + { name = "alabaster", version = "1.0.0", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version >= '3.12'" }, + { name = "babel", marker = "python_full_version >= '3.12'" }, + { name = "colorama", marker = "python_full_version >= '3.12' and sys_platform == 'win32'" }, + { name = "docutils", version = "0.22.4", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version >= '3.12'" }, + { name = "imagesize", version = "2.0.0", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version >= '3.12'" }, + { name = "jinja2", marker = "python_full_version >= '3.12'" }, + { name = "packaging", marker = "python_full_version >= '3.12'" }, + { name = "pygments", marker = "python_full_version >= '3.12'" }, + { name = "requests", version = "2.34.2", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version >= '3.12'" }, + { name = "roman-numerals", marker = "python_full_version >= '3.12'" }, + { name = "snowballstemmer", marker = "python_full_version >= '3.12'" }, + { name = "sphinxcontrib-applehelp", marker = "python_full_version >= '3.12'" }, + { name = "sphinxcontrib-devhelp", marker = "python_full_version >= '3.12'" }, + { name = "sphinxcontrib-htmlhelp", marker = "python_full_version >= '3.12'" }, + { name = "sphinxcontrib-jsmath", marker = "python_full_version >= '3.12'" }, + { name = "sphinxcontrib-qthelp", marker = "python_full_version >= '3.12'" }, + { name = "sphinxcontrib-serializinghtml", marker = "python_full_version >= '3.12'" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/cd/bd/f08eb0f4eed5c83f1ba2a3bd18f7745a2b1525fad70660a1c00224ec468a/sphinx-9.1.0.tar.gz", hash = "sha256:7741722357dd75f8190766926071fed3bdc211c74dd2d7d4df5404da95930ddb", size = 8718324, upload-time = "2025-12-31T15:09:27.646Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/73/f7/b1884cb3188ab181fc81fa00c266699dab600f927a964df02ec3d5d1916a/sphinx-9.1.0-py3-none-any.whl", hash = "sha256:c84fdd4e782504495fe4f2c0b3413d6c2bf388589bb352d439b2a3bb99991978", size = 3921742, upload-time = "2025-12-31T15:09:25.561Z" }, +] + +[[package]] +name = "sphinx-autodoc2" +version = "0.5.0" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "astroid" }, + { name = "tomli", marker = "python_full_version < '3.11'" }, + { name = "typing-extensions" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/17/5f/5350046d1aa1a56b063ae08b9ad871025335c9d55fe2372896ea48711da9/sphinx_autodoc2-0.5.0.tar.gz", hash = "sha256:7d76044aa81d6af74447080182b6868c7eb066874edc835e8ddf810735b6565a", size = 115077, upload-time = "2023-11-27T07:27:51.407Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/19/e6/48d47961bbdae755ba9c17dfc65d89356312c67668dcb36c87cfadfa1964/sphinx_autodoc2-0.5.0-py3-none-any.whl", hash = "sha256:e867013b1512f9d6d7e6f6799f8b537d6884462acd118ef361f3f619a60b5c9e", size = 43385, upload-time = "2023-11-27T07:27:49.929Z" }, +] + +[[package]] +name = "sphinx-reredirects" +version = "0.1.6" +source = { registry = "https://pypi.org/simple" } +resolution-markers = [ + "python_full_version == '3.10.*'", + "python_full_version < '3.10'", +] +dependencies = [ + { name = "sphinx", version = "7.4.7", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.10'" }, + { name = "sphinx", version = "8.1.3", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version == '3.10.*'" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/16/6b/bcca2785de4071f604a722444d4d7ba8a9d40de3c14ad52fce93e6d92694/sphinx_reredirects-0.1.6.tar.gz", hash = "sha256:c491cba545f67be9697508727818d8626626366245ae64456fe29f37e9bbea64", size = 7080, upload-time = "2025-03-22T10:52:30.271Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/ac/6f/0b3625be30a1a50f9e4c2cb2ec147b08f15ed0e9f8444efcf274b751300b/sphinx_reredirects-0.1.6-py3-none-any.whl", hash = "sha256:efd50c766fbc5bf40cd5148e10c00f2c00d143027de5c5e48beece93cc40eeea", size = 5675, upload-time = "2025-03-22T10:52:29.113Z" }, +] + +[[package]] +name = "sphinx-reredirects" +version = "1.1.0" +source = { registry = "https://pypi.org/simple" } +resolution-markers = [ + "python_full_version >= '3.12'", + "python_full_version == '3.11.*'", +] +dependencies = [ + { name = "sphinx", version = "9.0.4", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version == '3.11.*'" }, + { name = "sphinx", version = "9.1.0", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version >= '3.12'" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/1b/8d/0e39fe2740d7d71417edf9a6424aa80ca2c27c17fc21282cdc39f90d5a40/sphinx_reredirects-1.1.0.tar.gz", hash = "sha256:fb9b195335ab14b43f8273287d0c7eeb637ba6c56c66581c11b47202f6718b29", size = 614624, upload-time = "2025-12-22T08:28:02.792Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/51/81/b5dd07067f3daac6d23687ec737b2d593740671ebcd145830c8f92d381c5/sphinx_reredirects-1.1.0-py3-none-any.whl", hash = "sha256:4b5692273c72cd2d4d917f4c6f87d5919e4d6114a752d4be033f7f5f6310efd9", size = 6351, upload-time = "2025-12-22T08:27:59.724Z" }, +] + +[[package]] +name = "sphinx-rtd-theme" +version = "3.1.0" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "docutils", version = "0.21.2", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.11'" }, + { name = "docutils", version = "0.22.4", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version >= '3.11'" }, + { name = "sphinx", version = "7.4.7", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.10'" }, + { name = "sphinx", version = "8.1.3", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version == '3.10.*'" }, + { name = "sphinx", version = "9.0.4", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version == '3.11.*'" }, + { name = "sphinx", version = "9.1.0", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version >= '3.12'" }, + { name = "sphinxcontrib-jquery" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/84/68/a1bfbf38c0f7bccc9b10bbf76b94606f64acb1552ae394f0b8285bfaea25/sphinx_rtd_theme-3.1.0.tar.gz", hash = "sha256:b44276f2c276e909239a4f6c955aa667aaafeb78597923b1c60babc76db78e4c", size = 7620915, upload-time = "2026-01-12T16:03:31.17Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/87/c7/b5c8015d823bfda1a346adb2c634a2101d50bb75d421eb6dcb31acd25ebc/sphinx_rtd_theme-3.1.0-py2.py3-none-any.whl", hash = "sha256:1785824ae8e6632060490f67cf3a72d404a85d2d9fc26bce3619944de5682b89", size = 7655617, upload-time = "2026-01-12T16:03:28.101Z" }, +] + +[[package]] +name = "sphinxcontrib-applehelp" +version = "2.0.0" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/ba/6e/b837e84a1a704953c62ef8776d45c3e8d759876b4a84fe14eba2859106fe/sphinxcontrib_applehelp-2.0.0.tar.gz", hash = "sha256:2f29ef331735ce958efa4734873f084941970894c6090408b079c61b2e1c06d1", size = 20053, upload-time = "2024-07-29T01:09:00.465Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/5d/85/9ebeae2f76e9e77b952f4b274c27238156eae7979c5421fba91a28f4970d/sphinxcontrib_applehelp-2.0.0-py3-none-any.whl", hash = "sha256:4cd3f0ec4ac5dd9c17ec65e9ab272c9b867ea77425228e68ecf08d6b28ddbdb5", size = 119300, upload-time = "2024-07-29T01:08:58.99Z" }, +] + +[[package]] +name = "sphinxcontrib-devhelp" +version = "2.0.0" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/f6/d2/5beee64d3e4e747f316bae86b55943f51e82bb86ecd325883ef65741e7da/sphinxcontrib_devhelp-2.0.0.tar.gz", hash = "sha256:411f5d96d445d1d73bb5d52133377b4248ec79db5c793ce7dbe59e074b4dd1ad", size = 12967, upload-time = "2024-07-29T01:09:23.417Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/35/7a/987e583882f985fe4d7323774889ec58049171828b58c2217e7f79cdf44e/sphinxcontrib_devhelp-2.0.0-py3-none-any.whl", hash = "sha256:aefb8b83854e4b0998877524d1029fd3e6879210422ee3780459e28a1f03a8a2", size = 82530, upload-time = "2024-07-29T01:09:21.945Z" }, +] + +[[package]] +name = "sphinxcontrib-htmlhelp" +version = "2.1.0" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/43/93/983afd9aa001e5201eab16b5a444ed5b9b0a7a010541e0ddfbbfd0b2470c/sphinxcontrib_htmlhelp-2.1.0.tar.gz", hash = "sha256:c9e2916ace8aad64cc13a0d233ee22317f2b9025b9cf3295249fa985cc7082e9", size = 22617, upload-time = "2024-07-29T01:09:37.889Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/0a/7b/18a8c0bcec9182c05a0b3ec2a776bba4ead82750a55ff798e8d406dae604/sphinxcontrib_htmlhelp-2.1.0-py3-none-any.whl", hash = "sha256:166759820b47002d22914d64a075ce08f4c46818e17cfc9470a9786b759b19f8", size = 98705, upload-time = "2024-07-29T01:09:36.407Z" }, +] + +[[package]] +name = "sphinxcontrib-jquery" +version = "4.1" +source = { registry = "https://pypi.org/simple" } +dependencies = [ + { name = "sphinx", version = "7.4.7", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.10'" }, + { name = "sphinx", version = "8.1.3", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version == '3.10.*'" }, + { name = "sphinx", version = "9.0.4", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version == '3.11.*'" }, + { name = "sphinx", version = "9.1.0", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version >= '3.12'" }, +] +sdist = { url = "https://files.pythonhosted.org/packages/de/f3/aa67467e051df70a6330fe7770894b3e4f09436dea6881ae0b4f3d87cad8/sphinxcontrib-jquery-4.1.tar.gz", hash = "sha256:1620739f04e36a2c779f1a131a2dfd49b2fd07351bf1968ced074365933abc7a", size = 122331, upload-time = "2023-03-14T15:01:01.944Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/76/85/749bd22d1a68db7291c89e2ebca53f4306c3f205853cf31e9de279034c3c/sphinxcontrib_jquery-4.1-py2.py3-none-any.whl", hash = "sha256:f936030d7d0147dd026a4f2b5a57343d233f1fc7b363f68b3d4f1cb0993878ae", size = 121104, upload-time = "2023-03-14T15:01:00.356Z" }, +] + +[[package]] +name = "sphinxcontrib-jsmath" +version = "1.0.1" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/b2/e8/9ed3830aeed71f17c026a07a5097edcf44b692850ef215b161b8ad875729/sphinxcontrib-jsmath-1.0.1.tar.gz", hash = "sha256:a9925e4a4587247ed2191a22df5f6970656cb8ca2bd6284309578f2153e0c4b8", size = 5787, upload-time = "2019-01-21T16:10:16.347Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/c2/42/4c8646762ee83602e3fb3fbe774c2fac12f317deb0b5dbeeedd2d3ba4b77/sphinxcontrib_jsmath-1.0.1-py2.py3-none-any.whl", hash = "sha256:2ec2eaebfb78f3f2078e73666b1415417a116cc848b72e5172e596c871103178", size = 5071, upload-time = "2019-01-21T16:10:14.333Z" }, +] + +[[package]] +name = "sphinxcontrib-qthelp" +version = "2.0.0" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/68/bc/9104308fc285eb3e0b31b67688235db556cd5b0ef31d96f30e45f2e51cae/sphinxcontrib_qthelp-2.0.0.tar.gz", hash = "sha256:4fe7d0ac8fc171045be623aba3e2a8f613f8682731f9153bb2e40ece16b9bbab", size = 17165, upload-time = "2024-07-29T01:09:56.435Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/27/83/859ecdd180cacc13b1f7e857abf8582a64552ea7a061057a6c716e790fce/sphinxcontrib_qthelp-2.0.0-py3-none-any.whl", hash = "sha256:b18a828cdba941ccd6ee8445dbe72ffa3ef8cbe7505d8cd1fa0d42d3f2d5f3eb", size = 88743, upload-time = "2024-07-29T01:09:54.885Z" }, +] + +[[package]] +name = "sphinxcontrib-serializinghtml" +version = "2.0.0" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/3b/44/6716b257b0aa6bfd51a1b31665d1c205fb12cb5ad56de752dfa15657de2f/sphinxcontrib_serializinghtml-2.0.0.tar.gz", hash = "sha256:e9d912827f872c029017a53f0ef2180b327c3f7fd23c87229f7a8e8b70031d4d", size = 16080, upload-time = "2024-07-29T01:10:09.332Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/52/a7/d2782e4e3f77c8450f727ba74a8f12756d5ba823d81b941f1b04da9d033a/sphinxcontrib_serializinghtml-2.0.0-py3-none-any.whl", hash = "sha256:6e2cb0eef194e10c27ec0023bfeb25badbbb5868244cf5bc5bdc04e4464bf331", size = 92072, upload-time = "2024-07-29T01:10:08.203Z" }, +] + +[[package]] +name = "tomli" +version = "2.4.1" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/22/de/48c59722572767841493b26183a0d1cc411d54fd759c5607c4590b6563a6/tomli-2.4.1.tar.gz", hash = "sha256:7c7e1a961a0b2f2472c1ac5b69affa0ae1132c39adcb67aba98568702b9cc23f", size = 17543, upload-time = "2026-03-25T20:22:03.828Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/f4/11/db3d5885d8528263d8adc260bb2d28ebf1270b96e98f0e0268d32b8d9900/tomli-2.4.1-cp311-cp311-macosx_10_9_x86_64.whl", hash = "sha256:f8f0fc26ec2cc2b965b7a3b87cd19c5c6b8c5e5f436b984e85f486d652285c30", size = 154704, upload-time = "2026-03-25T20:21:10.473Z" }, + { url = "https://files.pythonhosted.org/packages/6d/f7/675db52c7e46064a9aa928885a9b20f4124ecb9bc2e1ce74c9106648d202/tomli-2.4.1-cp311-cp311-macosx_11_0_arm64.whl", hash = "sha256:4ab97e64ccda8756376892c53a72bd1f964e519c77236368527f758fbc36a53a", size = 149454, upload-time = "2026-03-25T20:21:12.036Z" }, + { url = "https://files.pythonhosted.org/packages/61/71/81c50943cf953efa35bce7646caab3cf457a7d8c030b27cfb40d7235f9ee/tomli-2.4.1-cp311-cp311-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:96481a5786729fd470164b47cdb3e0e58062a496f455ee41b4403be77cb5a076", size = 237561, upload-time = "2026-03-25T20:21:13.098Z" }, + { url = "https://files.pythonhosted.org/packages/48/c1/f41d9cb618acccca7df82aaf682f9b49013c9397212cb9f53219e3abac37/tomli-2.4.1-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:5a881ab208c0baf688221f8cecc5401bd291d67e38a1ac884d6736cbcd8247e9", size = 243824, upload-time = "2026-03-25T20:21:14.569Z" }, + { url = "https://files.pythonhosted.org/packages/22/e4/5a816ecdd1f8ca51fb756ef684b90f2780afc52fc67f987e3c61d800a46d/tomli-2.4.1-cp311-cp311-musllinux_1_2_aarch64.whl", hash = "sha256:47149d5bd38761ac8be13a84864bf0b7b70bc051806bc3669ab1cbc56216b23c", size = 242227, upload-time = "2026-03-25T20:21:15.712Z" }, + { url = "https://files.pythonhosted.org/packages/6b/49/2b2a0ef529aa6eec245d25f0c703e020a73955ad7edf73e7f54ddc608aa5/tomli-2.4.1-cp311-cp311-musllinux_1_2_x86_64.whl", hash = "sha256:ec9bfaf3ad2df51ace80688143a6a4ebc09a248f6ff781a9945e51937008fcbc", size = 247859, upload-time = "2026-03-25T20:21:17.001Z" }, + { url = "https://files.pythonhosted.org/packages/83/bd/6c1a630eaca337e1e78c5903104f831bda934c426f9231429396ce3c3467/tomli-2.4.1-cp311-cp311-win32.whl", hash = "sha256:ff2983983d34813c1aeb0fa89091e76c3a22889ee83ab27c5eeb45100560c049", size = 97204, upload-time = "2026-03-25T20:21:18.079Z" }, + { url = "https://files.pythonhosted.org/packages/42/59/71461df1a885647e10b6bb7802d0b8e66480c61f3f43079e0dcd315b3954/tomli-2.4.1-cp311-cp311-win_amd64.whl", hash = "sha256:5ee18d9ebdb417e384b58fe414e8d6af9f4e7a0ae761519fb50f721de398dd4e", size = 108084, upload-time = "2026-03-25T20:21:18.978Z" }, + { url = "https://files.pythonhosted.org/packages/b8/83/dceca96142499c069475b790e7913b1044c1a4337e700751f48ed723f883/tomli-2.4.1-cp311-cp311-win_arm64.whl", hash = "sha256:c2541745709bad0264b7d4705ad453b76ccd191e64aa6f0fc66b69a293a45ece", size = 95285, upload-time = "2026-03-25T20:21:20.309Z" }, + { url = "https://files.pythonhosted.org/packages/c1/ba/42f134a3fe2b370f555f44b1d72feebb94debcab01676bf918d0cb70e9aa/tomli-2.4.1-cp312-cp312-macosx_10_13_x86_64.whl", hash = "sha256:c742f741d58a28940ce01d58f0ab2ea3ced8b12402f162f4d534dfe18ba1cd6a", size = 155924, upload-time = "2026-03-25T20:21:21.626Z" }, + { url = "https://files.pythonhosted.org/packages/dc/c7/62d7a17c26487ade21c5422b646110f2162f1fcc95980ef7f63e73c68f14/tomli-2.4.1-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:7f86fd587c4ed9dd76f318225e7d9b29cfc5a9d43de44e5754db8d1128487085", size = 150018, upload-time = "2026-03-25T20:21:23.002Z" }, + { url = "https://files.pythonhosted.org/packages/5c/05/79d13d7c15f13bdef410bdd49a6485b1c37d28968314eabee452c22a7fda/tomli-2.4.1-cp312-cp312-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:ff18e6a727ee0ab0388507b89d1bc6a22b138d1e2fa56d1ad494586d61d2eae9", size = 244948, upload-time = "2026-03-25T20:21:24.04Z" }, + { url = "https://files.pythonhosted.org/packages/10/90/d62ce007a1c80d0b2c93e02cab211224756240884751b94ca72df8a875ca/tomli-2.4.1-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:136443dbd7e1dee43c68ac2694fde36b2849865fa258d39bf822c10e8068eac5", size = 253341, upload-time = "2026-03-25T20:21:25.177Z" }, + { url = "https://files.pythonhosted.org/packages/1a/7e/caf6496d60152ad4ed09282c1885cca4eea150bfd007da84aea07bcc0a3e/tomli-2.4.1-cp312-cp312-musllinux_1_2_aarch64.whl", hash = "sha256:5e262d41726bc187e69af7825504c933b6794dc3fbd5945e41a79bb14c31f585", size = 248159, upload-time = "2026-03-25T20:21:26.364Z" }, + { url = "https://files.pythonhosted.org/packages/99/e7/c6f69c3120de34bbd882c6fba7975f3d7a746e9218e56ab46a1bc4b42552/tomli-2.4.1-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:5cb41aa38891e073ee49d55fbc7839cfdb2bc0e600add13874d048c94aadddd1", size = 253290, upload-time = "2026-03-25T20:21:27.46Z" }, + { url = "https://files.pythonhosted.org/packages/d6/2f/4a3c322f22c5c66c4b836ec58211641a4067364f5dcdd7b974b4c5da300c/tomli-2.4.1-cp312-cp312-win32.whl", hash = "sha256:da25dc3563bff5965356133435b757a795a17b17d01dbc0f42fb32447ddfd917", size = 98141, upload-time = "2026-03-25T20:21:28.492Z" }, + { url = "https://files.pythonhosted.org/packages/24/22/4daacd05391b92c55759d55eaee21e1dfaea86ce5c571f10083360adf534/tomli-2.4.1-cp312-cp312-win_amd64.whl", hash = "sha256:52c8ef851d9a240f11a88c003eacb03c31fc1c9c4ec64a99a0f922b93874fda9", size = 108847, upload-time = "2026-03-25T20:21:29.386Z" }, + { url = "https://files.pythonhosted.org/packages/68/fd/70e768887666ddd9e9f5d85129e84910f2db2796f9096aa02b721a53098d/tomli-2.4.1-cp312-cp312-win_arm64.whl", hash = "sha256:f758f1b9299d059cc3f6546ae2af89670cb1c4d48ea29c3cacc4fe7de3058257", size = 95088, upload-time = "2026-03-25T20:21:30.677Z" }, + { url = "https://files.pythonhosted.org/packages/07/06/b823a7e818c756d9a7123ba2cda7d07bc2dd32835648d1a7b7b7a05d848d/tomli-2.4.1-cp313-cp313-macosx_10_13_x86_64.whl", hash = "sha256:36d2bd2ad5fb9eaddba5226aa02c8ec3fa4f192631e347b3ed28186d43be6b54", size = 155866, upload-time = "2026-03-25T20:21:31.65Z" }, + { url = "https://files.pythonhosted.org/packages/14/6f/12645cf7f08e1a20c7eb8c297c6f11d31c1b50f316a7e7e1e1de6e2e7b7e/tomli-2.4.1-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:eb0dc4e38e6a1fd579e5d50369aa2e10acfc9cace504579b2faabb478e76941a", size = 149887, upload-time = "2026-03-25T20:21:33.028Z" }, + { url = "https://files.pythonhosted.org/packages/5c/e0/90637574e5e7212c09099c67ad349b04ec4d6020324539297b634a0192b0/tomli-2.4.1-cp313-cp313-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:c7f2c7f2b9ca6bdeef8f0fa897f8e05085923eb091721675170254cbc5b02897", size = 243704, upload-time = "2026-03-25T20:21:34.51Z" }, + { url = "https://files.pythonhosted.org/packages/10/8f/d3ddb16c5a4befdf31a23307f72828686ab2096f068eaf56631e136c1fdd/tomli-2.4.1-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:f3c6818a1a86dd6dca7ddcaaf76947d5ba31aecc28cb1b67009a5877c9a64f3f", size = 251628, upload-time = "2026-03-25T20:21:36.012Z" }, + { url = "https://files.pythonhosted.org/packages/e3/f1/dbeeb9116715abee2485bf0a12d07a8f31af94d71608c171c45f64c0469d/tomli-2.4.1-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:d312ef37c91508b0ab2cee7da26ec0b3ed2f03ce12bd87a588d771ae15dcf82d", size = 247180, upload-time = "2026-03-25T20:21:37.136Z" }, + { url = "https://files.pythonhosted.org/packages/d3/74/16336ffd19ed4da28a70959f92f506233bd7cfc2332b20bdb01591e8b1d1/tomli-2.4.1-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:51529d40e3ca50046d7606fa99ce3956a617f9b36380da3b7f0dd3dd28e68cb5", size = 251674, upload-time = "2026-03-25T20:21:38.298Z" }, + { url = "https://files.pythonhosted.org/packages/16/f9/229fa3434c590ddf6c0aa9af64d3af4b752540686cace29e6281e3458469/tomli-2.4.1-cp313-cp313-win32.whl", hash = "sha256:2190f2e9dd7508d2a90ded5ed369255980a1bcdd58e52f7fe24b8162bf9fedbd", size = 97976, upload-time = "2026-03-25T20:21:39.316Z" }, + { url = "https://files.pythonhosted.org/packages/6a/1e/71dfd96bcc1c775420cb8befe7a9d35f2e5b1309798f009dca17b7708c1e/tomli-2.4.1-cp313-cp313-win_amd64.whl", hash = "sha256:8d65a2fbf9d2f8352685bc1364177ee3923d6baf5e7f43ea4959d7d8bc326a36", size = 108755, upload-time = "2026-03-25T20:21:40.248Z" }, + { url = "https://files.pythonhosted.org/packages/83/7a/d34f422a021d62420b78f5c538e5b102f62bea616d1d75a13f0a88acb04a/tomli-2.4.1-cp313-cp313-win_arm64.whl", hash = "sha256:4b605484e43cdc43f0954ddae319fb75f04cc10dd80d830540060ee7cd0243cd", size = 95265, upload-time = "2026-03-25T20:21:41.219Z" }, + { url = "https://files.pythonhosted.org/packages/3c/fb/9a5c8d27dbab540869f7c1f8eb0abb3244189ce780ba9cd73f3770662072/tomli-2.4.1-cp314-cp314-macosx_10_15_x86_64.whl", hash = "sha256:fd0409a3653af6c147209d267a0e4243f0ae46b011aa978b1080359fddc9b6cf", size = 155726, upload-time = "2026-03-25T20:21:42.23Z" }, + { url = "https://files.pythonhosted.org/packages/62/05/d2f816630cc771ad836af54f5001f47a6f611d2d39535364f148b6a92d6b/tomli-2.4.1-cp314-cp314-macosx_11_0_arm64.whl", hash = "sha256:a120733b01c45e9a0c34aeef92bf0cf1d56cfe81ed9d47d562f9ed591a9828ac", size = 149859, upload-time = "2026-03-25T20:21:43.386Z" }, + { url = "https://files.pythonhosted.org/packages/ce/48/66341bdb858ad9bd0ceab5a86f90eddab127cf8b046418009f2125630ecb/tomli-2.4.1-cp314-cp314-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:559db847dc486944896521f68d8190be1c9e719fced785720d2216fe7022b662", size = 244713, upload-time = "2026-03-25T20:21:44.474Z" }, + { url = "https://files.pythonhosted.org/packages/df/6d/c5fad00d82b3c7a3ab6189bd4b10e60466f22cfe8a08a9394185c8a8111c/tomli-2.4.1-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:01f520d4f53ef97964a240a035ec2a869fe1a37dde002b57ebc4417a27ccd853", size = 252084, upload-time = "2026-03-25T20:21:45.62Z" }, + { url = "https://files.pythonhosted.org/packages/00/71/3a69e86f3eafe8c7a59d008d245888051005bd657760e96d5fbfb0b740c2/tomli-2.4.1-cp314-cp314-musllinux_1_2_aarch64.whl", hash = "sha256:7f94b27a62cfad8496c8d2513e1a222dd446f095fca8987fceef261225538a15", size = 247973, upload-time = "2026-03-25T20:21:46.937Z" }, + { url = "https://files.pythonhosted.org/packages/67/50/361e986652847fec4bd5e4a0208752fbe64689c603c7ae5ea7cb16b1c0ca/tomli-2.4.1-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:ede3e6487c5ef5d28634ba3f31f989030ad6af71edfb0055cbbd14189ff240ba", size = 256223, upload-time = "2026-03-25T20:21:48.467Z" }, + { url = "https://files.pythonhosted.org/packages/8c/9a/b4173689a9203472e5467217e0154b00e260621caa227b6fa01feab16998/tomli-2.4.1-cp314-cp314-win32.whl", hash = "sha256:3d48a93ee1c9b79c04bb38772ee1b64dcf18ff43085896ea460ca8dec96f35f6", size = 98973, upload-time = "2026-03-25T20:21:49.526Z" }, + { url = "https://files.pythonhosted.org/packages/14/58/640ac93bf230cd27d002462c9af0d837779f8773bc03dee06b5835208214/tomli-2.4.1-cp314-cp314-win_amd64.whl", hash = "sha256:88dceee75c2c63af144e456745e10101eb67361050196b0b6af5d717254dddf7", size = 109082, upload-time = "2026-03-25T20:21:50.506Z" }, + { url = "https://files.pythonhosted.org/packages/d5/2f/702d5e05b227401c1068f0d386d79a589bb12bf64c3d2c72ce0631e3bc49/tomli-2.4.1-cp314-cp314-win_arm64.whl", hash = "sha256:b8c198f8c1805dc42708689ed6864951fd2494f924149d3e4bce7710f8eb5232", size = 96490, upload-time = "2026-03-25T20:21:51.474Z" }, + { url = "https://files.pythonhosted.org/packages/45/4b/b877b05c8ba62927d9865dd980e34a755de541eb65fffba52b4cc495d4d2/tomli-2.4.1-cp314-cp314t-macosx_10_15_x86_64.whl", hash = "sha256:d4d8fe59808a54658fcc0160ecfb1b30f9089906c50b23bcb4c69eddc19ec2b4", size = 164263, upload-time = "2026-03-25T20:21:52.543Z" }, + { url = "https://files.pythonhosted.org/packages/24/79/6ab420d37a270b89f7195dec5448f79400d9e9c1826df982f3f8e97b24fd/tomli-2.4.1-cp314-cp314t-macosx_11_0_arm64.whl", hash = "sha256:7008df2e7655c495dd12d2a4ad038ff878d4ca4b81fccaf82b714e07eae4402c", size = 160736, upload-time = "2026-03-25T20:21:53.674Z" }, + { url = "https://files.pythonhosted.org/packages/02/e0/3630057d8eb170310785723ed5adcdfb7d50cb7e6455f85ba8a3deed642b/tomli-2.4.1-cp314-cp314t-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:1d8591993e228b0c930c4bb0db464bdad97b3289fb981255d6c9a41aedc84b2d", size = 270717, upload-time = "2026-03-25T20:21:55.129Z" }, + { url = "https://files.pythonhosted.org/packages/7a/b4/1613716072e544d1a7891f548d8f9ec6ce2faf42ca65acae01d76ea06bb0/tomli-2.4.1-cp314-cp314t-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:734e20b57ba95624ecf1841e72b53f6e186355e216e5412de414e3c51e5e3c41", size = 278461, upload-time = "2026-03-25T20:21:56.228Z" }, + { url = "https://files.pythonhosted.org/packages/05/38/30f541baf6a3f6df77b3df16b01ba319221389e2da59427e221ef417ac0c/tomli-2.4.1-cp314-cp314t-musllinux_1_2_aarch64.whl", hash = "sha256:8a650c2dbafa08d42e51ba0b62740dae4ecb9338eefa093aa5c78ceb546fcd5c", size = 274855, upload-time = "2026-03-25T20:21:57.653Z" }, + { url = "https://files.pythonhosted.org/packages/77/a3/ec9dd4fd2c38e98de34223b995a3b34813e6bdadf86c75314c928350ed14/tomli-2.4.1-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:504aa796fe0569bb43171066009ead363de03675276d2d121ac1a4572397870f", size = 283144, upload-time = "2026-03-25T20:21:59.089Z" }, + { url = "https://files.pythonhosted.org/packages/ef/be/605a6261cac79fba2ec0c9827e986e00323a1945700969b8ee0b30d85453/tomli-2.4.1-cp314-cp314t-win32.whl", hash = "sha256:b1d22e6e9387bf4739fbe23bfa80e93f6b0373a7f1b96c6227c32bef95a4d7a8", size = 108683, upload-time = "2026-03-25T20:22:00.214Z" }, + { url = "https://files.pythonhosted.org/packages/12/64/da524626d3b9cc40c168a13da8335fe1c51be12c0a63685cc6db7308daae/tomli-2.4.1-cp314-cp314t-win_amd64.whl", hash = "sha256:2c1c351919aca02858f740c6d33adea0c5deea37f9ecca1cc1ef9e884a619d26", size = 121196, upload-time = "2026-03-25T20:22:01.169Z" }, + { url = "https://files.pythonhosted.org/packages/5a/cd/e80b62269fc78fc36c9af5a6b89c835baa8af28ff5ad28c7028d60860320/tomli-2.4.1-cp314-cp314t-win_arm64.whl", hash = "sha256:eab21f45c7f66c13f2a9e0e1535309cee140182a9cdae1e041d02e47291e8396", size = 100393, upload-time = "2026-03-25T20:22:02.137Z" }, + { url = "https://files.pythonhosted.org/packages/7b/61/cceae43728b7de99d9b847560c262873a1f6c98202171fd5ed62640b494b/tomli-2.4.1-py3-none-any.whl", hash = "sha256:0d85819802132122da43cb86656f8d1f8c6587d54ae7dcaf30e90533028b49fe", size = 14583, upload-time = "2026-03-25T20:22:03.012Z" }, +] + +[[package]] +name = "typing-extensions" +version = "4.16.0" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/f6/cc/6253133b5bb138fc3306cebfbda2c520f545d36b5be2c7255cc528bb45d6/typing_extensions-4.16.0.tar.gz", hash = "sha256:dc983d19a509c94dba722ee6abd33940f7c05a89e243c47e907eb4db6f1a43e5", size = 113555, upload-time = "2026-07-02T08:40:05.92Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/49/d3/b8441a820a491ddfc024b0b0cf0393375b75ea13866d9c66727e54c2fc80/typing_extensions-4.16.0-py3-none-any.whl", hash = "sha256:481caa481374e813c1b176ada14e97f1f67a4539ce9cfeb3f350d78d6370c2e8", size = 45571, upload-time = "2026-07-02T08:40:04.659Z" }, +] + +[[package]] +name = "urllib3" +version = "2.6.3" +source = { registry = "https://pypi.org/simple" } +resolution-markers = [ + "python_full_version < '3.10'", +] +sdist = { url = "https://files.pythonhosted.org/packages/c7/24/5f1b3bdffd70275f6661c76461e25f024d5a38a46f04aaca912426a2b1d3/urllib3-2.6.3.tar.gz", hash = "sha256:1b62b6884944a57dbe321509ab94fd4d3b307075e0c2eae991ac71ee15ad38ed", size = 435556, upload-time = "2026-01-07T16:24:43.925Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/39/08/aaaad47bc4e9dc8c725e68f9d04865dbcb2052843ff09c97b08904852d84/urllib3-2.6.3-py3-none-any.whl", hash = "sha256:bf272323e553dfb2e87d9bfd225ca7b0f467b919d7bbd355436d3fd37cb0acd4", size = 131584, upload-time = "2026-01-07T16:24:42.685Z" }, +] + +[[package]] +name = "urllib3" +version = "2.7.0" +source = { registry = "https://pypi.org/simple" } +resolution-markers = [ + "python_full_version >= '3.12'", + "python_full_version == '3.11.*'", + "python_full_version == '3.10.*'", +] +sdist = { url = "https://files.pythonhosted.org/packages/53/0c/06f8b233b8fd13b9e5ee11424ef85419ba0d8ba0b3138bf360be2ff56953/urllib3-2.7.0.tar.gz", hash = "sha256:231e0ec3b63ceb14667c67be60f2f2c40a518cb38b03af60abc813da26505f4c", size = 433602, upload-time = "2026-05-07T16:13:18.596Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/7f/3e/5db95bcf282c52709639744ca2a8b149baccf648e39c8cc87553df9eae0c/urllib3-2.7.0-py3-none-any.whl", hash = "sha256:9fb4c81ebbb1ce9531cce37674bbc6f1360472bc18ca9a553ede278ef7276897", size = 131087, upload-time = "2026-05-07T16:13:17.151Z" }, +] + +[[package]] +name = "zipp" +version = "3.23.1" +source = { registry = "https://pypi.org/simple" } +sdist = { url = "https://files.pythonhosted.org/packages/30/21/093488dfc7cc8964ded15ab726fad40f25fd3d788fd741cc1c5a17d78ee8/zipp-3.23.1.tar.gz", hash = "sha256:32120e378d32cd9714ad503c1d024619063ec28aad2248dc6672ad13edfa5110", size = 25965, upload-time = "2026-04-13T23:21:46.6Z" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/08/8a/0861bec20485572fbddf3dfba2910e38fe249796cb73ecdeb74e07eeb8d3/zipp-3.23.1-py3-none-any.whl", hash = "sha256:0b3596c50a5c700c9cb40ba8d86d9f2cc4807e9bedb06bcdf7fac85633e444dc", size = 10378, upload-time = "2026-04-13T23:21:45.386Z" }, +] diff --git a/sphinxdocs/integration_tests/bcr/MODULE.bazel b/sphinxdocs/integration_tests/bcr/MODULE.bazel index ae1c752513..451f2e920d 100644 --- a/sphinxdocs/integration_tests/bcr/MODULE.bazel +++ b/sphinxdocs/integration_tests/bcr/MODULE.bazel @@ -19,7 +19,7 @@ dev_pip = use_extension( dev_pip.parse( hub_name = "dev_pip", python_version = "3.11", - requirements_lock = "@rules_python//docs:requirements.txt", + requirements_lock = "@sphinxdocs//dev:requirements.txt", ) use_repo(dev_pip, "dev_pip") diff --git a/sphinxdocs/integration_tests/persistent_worker/workspace/MODULE.bazel b/sphinxdocs/integration_tests/persistent_worker/workspace/MODULE.bazel index 4e2c25e8e4..976718f312 100644 --- a/sphinxdocs/integration_tests/persistent_worker/workspace/MODULE.bazel +++ b/sphinxdocs/integration_tests/persistent_worker/workspace/MODULE.bazel @@ -16,7 +16,7 @@ dev_pip = use_extension( dev_pip.parse( hub_name = "dev_pip", python_version = "3.11", - requirements_lock = "@rules_python//docs:requirements.txt", + requirements_lock = "@sphinxdocs//dev:requirements.txt", ) use_repo(dev_pip, "dev_pip") diff --git a/sphinxdocs/pyproject.toml b/sphinxdocs/pyproject.toml new file mode 100644 index 0000000000..89330debc6 --- /dev/null +++ b/sphinxdocs/pyproject.toml @@ -0,0 +1,15 @@ +[project] +name = "sphinxdocs" +version = "0.0.0" + +dependencies = [ + "absl-py", + "markupsafe", + "myst-parser", + "readthedocs-sphinx-ext", + "sphinx", + "sphinx-autodoc2", + "sphinx-reredirects", + "sphinx_rtd_theme >=2.0", # uv insists on downgrading for some reason + "typing-extensions", +] From 109866a0f06ac93b48c93b40e68e7e6199aefcc0 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Sun, 26 Jul 2026 20:44:51 -0700 Subject: [PATCH 858/922] agents: emphasize trailing ampersand in CI monitor skill invocation (#3956) When launching `monitor_remote_ci.py` via agent tool calls without a trailing `&`, the script remains attached as a child process wrapper and blocks foreground execution. To fix, update `SKILL.md` with an explicit note emphasizing that tool invocations must always include the trailing `&` to ensure the monitoring script runs as a detached background task. --- .agents/skills/monitor-ci-results/SKILL.md | 1 + 1 file changed, 1 insertion(+) diff --git a/.agents/skills/monitor-ci-results/SKILL.md b/.agents/skills/monitor-ci-results/SKILL.md index 8f6429d7ce..c5f8de309f 100644 --- a/.agents/skills/monitor-ci-results/SKILL.md +++ b/.agents/skills/monitor-ci-results/SKILL.md @@ -18,3 +18,4 @@ When any CI job completes with errors or returns a non-zero exit code: ```bash ./scripts/monitor_remote_ci.py 3812 "0be435bd-96aa-4e1b-9c6f-727b31e80fa0" & ``` +*Note: Always include the trailing `&` when launching the monitoring script via tool calls to ensure it runs as a detached background task without blocking foreground execution.* From 28d9fca949d0df2f3f4856eecef0d50eaafe79a3 Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Mon, 27 Jul 2026 03:39:04 +0000 Subject: [PATCH 859/922] build(deps): bump charset-normalizer from 3.4.3 to 3.4.9 in /tools/publish (#3928) Bumps [charset-normalizer](https://github.com/jawah/charset_normalizer) from 3.4.3 to 3.4.9.
Release notes

Sourced from charset-normalizer's releases.

Version 3.4.9

3.4.9 (2026-07-07)

Fixed

  • Regression in our fallback path leading to a decode error. (#771) We've yanked 3.4.8 as a result of that bug.

Version 3.4.8

3.4.8 (2026-07-06)

Fixed

  • Wall import time due to cascade codec imports for our multibyte first sort of iana supported codecs (#742)
  • Unnecessary json import at runtime (#753)
  • Inverse capitalization not seen by noise detector (#731)

Changed

  • No longer holding a global cache for our noise / coherence measurements. Relax RSS memory usage.
  • Micro-optimizations in our noise / coherence measurements.
  • No longer using regex search by default for our preemptive charset mark algorithm.
  • Raised upperbound of setuptools to v83.
  • Raised upperbound of mypy(c) to v2.1.

Removed

  • Redundant UTF7 BOM marker (#730)

Version 3.4.7

3.4.7 (2026-04-02)

Changed

  • Pre-built optimized version using mypy[c] v1.20.
  • Relax setuptools constraint to setuptools>=68,<82.1.

Fixed

  • Correctly remove SIG remnant in utf-7 decoded string. (#718) (#716)

Version 3.4.6

3.4.6 (2026-03-15)

Changed

  • Flattened the logic in charset_normalizer.md for higher performance. Removed eligible(..) and feed(...) in favor of feed_info(...).
  • Raised upper bound for mypy[c] to 1.20, for our optimized version.
  • Updated UNICODE_RANGES_COMBINED using Unicode blocks v17.

Fixed

  • Edge case where noise difference between two candidates can be almost insignificant. (#672)
  • CLI --normalize writing to wrong path when passing multiple files in. (#702)

Misc

  • Freethreaded pre-built wheels now shipped in PyPI starting with 3.14t. (#616)

... (truncated)

Changelog

Sourced from charset-normalizer's changelog.

3.4.9 (2026-07-07)

Fixed

  • Regression in our fallback path leading to a decode error. (#771) We've yanked 3.4.8 as a result of that bug.

3.4.8 (2026-07-06)

Fixed

  • Wall import time due to cascade codec imports for our multibyte first sort of iana supported codecs (#742)
  • Unnecessary json import at runtime (#753)
  • Inverse capitalization not seen by noise detector (#731)

Changed

  • No longer holding a global cache for our noise / coherence measurements. Relax RSS memory usage.
  • Micro-optimizations in our noise / coherence measurements.
  • No longer using regex search by default for our preemptive charset mark algorithm.
  • Raised upperbound of setuptools to v83.
  • Raised upperbound of mypy(c) to v2.1.

Removed

  • Redundant UTF7 BOM marker (#730)

3.4.7 (2026-04-02)

Changed

  • Pre-built optimized version using mypy[c] v1.20.
  • Relax setuptools constraint to setuptools>=68,<82.1.

Fixed

  • Correctly remove SIG remnant in utf-7 decoded string. (#718) (#716)

3.4.6 (2026-03-15)

Changed

  • Flattened the logic in charset_normalizer.md for higher performance. Removed eligible(..) and feed(...) in favor of feed_info(...).
  • Raised upper bound for mypy[c] to 1.20, for our optimized version.
  • Updated UNICODE_RANGES_COMBINED using Unicode blocks v17.

Fixed

  • Edge case where noise difference between two candidates can be almost insignificant. (#672)
  • CLI --normalize writing to wrong path when passing multiple files in. (#702)

Misc

  • Freethreaded pre-built wheels now shipped in PyPI starting with 3.14t. (#616)

3.4.5 (2026-03-06)

Changed

... (truncated)

Commits
  • cc68407 Merge pull request #772 from jawah/fix-regression-fallback-path
  • 152b923 chore: release 3.4.9
  • 2bc2607 fix: unicodedecodeerror in fallback path
  • be252d7 chore(deps): bump docker/setup-qemu-action from 4.1.0 to 4.2.0 (#767)
  • 71c7bdd chore(deps): bump actions/setup-python from 6.2.0 to 6.3.0 (#768)
  • aeea391 chore(deps): bump pypa/cibuildwheel from 3.4.1 to 4.1.0 (#758)
  • a6f8feb Merge pull request #770 from jawah/unblock-ci
  • 528e16c chore: add osv-scanner.toml
  • 5993498 chore: ast_serialize musl missing prebuilt riscv,s390x,ppc64le
  • aa2ddd8 Release 3.4.8 (#766)
  • Additional commits viewable in compare view

--------- Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Richard Levasseur --- .../bzlmod_lockfile/MODULE.bazel.lock | 172 +++++++++-------- tools/publish/requirements_darwin.txt | 174 ++++++++++-------- tools/publish/requirements_linux.txt | 174 ++++++++++-------- tools/publish/requirements_universal.txt | 174 ++++++++++-------- tools/publish/requirements_windows.txt | 174 ++++++++++-------- 5 files changed, 469 insertions(+), 399 deletions(-) diff --git a/tests/integration/bzlmod_lockfile/MODULE.bazel.lock b/tests/integration/bzlmod_lockfile/MODULE.bazel.lock index fd161a3965..92d1f6be7b 100644 --- a/tests/integration/bzlmod_lockfile/MODULE.bazel.lock +++ b/tests/integration/bzlmod_lockfile/MODULE.bazel.lock @@ -380,85 +380,99 @@ "https://files.pythonhosted.org/packages/ff/df/a4f0fbd47331ceeba3d37c2e51e9dfc9722498becbeec2bd8bc856c9538a/cffi-2.0.0-cp312-cp312-manylinux1_i686.manylinux2014_i686.manylinux_2_17_i686.manylinux_2_5_i686.whl": "21d1152871b019407d8ac3985f6775c079416c282e431a4da6afe7aefd2bccbe" }, "charset-normalizer": { - "https://files.pythonhosted.org/packages/00/bd/ef9c88464b126fa176f4ef4a317ad9b6f4d30b2cffbc43386062367c3e2c/charset_normalizer-3.4.3-cp38-cp38-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl": "8999f965f922ae054125286faf9f11bc6932184b93011d138925a1773830bbe9", - "https://files.pythonhosted.org/packages/02/f7/3611b32318b30974131db62b4043f335861d4d9b49adc6d57c1149cc49d4/charset_normalizer-3.4.3-cp314-cp314-musllinux_1_2_aarch64.whl": "ccf600859c183d70eb47e05a44cd80a4ce77394d1ac0f79dbd2dd90a69a3a049", - "https://files.pythonhosted.org/packages/04/9a/914d294daa4809c57667b77470533e65def9c0be1ef8b4c1183a99170e9d/charset_normalizer-3.4.3-cp39-cp39-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl": "fb731e5deb0c7ef82d698b0f4c5bb724633ee2a489401594c5c88b02e6cb15f7", - "https://files.pythonhosted.org/packages/05/35/bb59b1cd012d7196fc81c2f5879113971efc226a63812c9cf7f89fe97c40/charset_normalizer-3.4.3-cp38-cp38-win_amd64.whl": "5d8d01eac18c423815ed4f4a2ec3b439d654e55ee4ad610e153cf02faf67ea40", - "https://files.pythonhosted.org/packages/05/6b/e2539a0a4be302b481e8cafb5af8792da8093b486885a1ae4d15d452bcec/charset_normalizer-3.4.3-cp312-cp312-musllinux_1_2_ppc64le.whl": "42e5088973e56e31e4fa58eb6bd709e42fc03799c11c42929592889a2e54c491", - "https://files.pythonhosted.org/packages/06/57/84722eefdd338c04cf3030ada66889298eaedf3e7a30a624201e0cbe424a/charset_normalizer-3.4.3-cp314-cp314-musllinux_1_2_s390x.whl": "30a96e1e1f865f78b030d65241c1ee850cdf422d869e9028e2fc1d5e4db73b92", - "https://files.pythonhosted.org/packages/0c/52/8b0c6c3e53f7e546a5e49b9edb876f379725914e1130297f3b423c7b71c5/charset_normalizer-3.4.3-cp38-cp38-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl": "c60e092517a73c632ec38e290eba714e9627abe9d301c8c8a12ec32c314a2a4b", - "https://files.pythonhosted.org/packages/16/ab/0233c3231af734f5dfcf0844aa9582d5a1466c985bbed6cedab85af9bfe3/charset_normalizer-3.4.3-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl": "1606f4a55c0fd363d754049cdf400175ee96c992b1f8018b993941f221221c5f", - "https://files.pythonhosted.org/packages/17/e5/5e67ab85e6d22b04641acb5399c8684f4d37caf7558a53859f0283a650e9/charset_normalizer-3.4.3-cp314-cp314-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl": "2001a39612b241dae17b4687898843f254f8748b796a2e16f1051a17078d991d", - "https://files.pythonhosted.org/packages/1a/79/ae516e678d6e32df2e7e740a7be51dc80b700e2697cb70054a0f1ac2c955/charset_normalizer-3.4.3-cp38-cp38-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl": "3653fad4fe3ed447a596ae8638b437f827234f01a8cd801842e43f3d0a6b281b", - "https://files.pythonhosted.org/packages/20/30/5f64fe3981677fe63fa987b80e6c01042eb5ff653ff7cec1b7bd9268e54e/charset_normalizer-3.4.3-cp39-cp39-musllinux_1_2_ppc64le.whl": "2c322db9c8c89009a990ef07c3bcc9f011a3269bc06782f916cd3d9eed7c9312", - "https://files.pythonhosted.org/packages/21/40/5188be1e3118c82dcb7c2a5ba101b783822cfb413a0268ed3be0468532de/charset_normalizer-3.4.3-cp310-cp310-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl": "cc9370a2da1ac13f0153780040f465839e6cccb4a1e44810124b4e22483c93fe", - "https://files.pythonhosted.org/packages/22/82/63a45bfc36f73efe46731a3a71cb84e2112f7e0b049507025ce477f0f052/charset_normalizer-3.4.3-cp38-cp38-macosx_10_9_universal2.whl": "0f2be7e0cf7754b9a30eb01f4295cc3d4358a479843b31f328afd210e2c7598c", - "https://files.pythonhosted.org/packages/2a/91/26c3036e62dfe8de8061182d33be5025e2424002125c9500faff74a6735e/charset_normalizer-3.4.3-cp310-cp310-win32.whl": "d79c198e27580c8e958906f803e63cddb77653731be08851c7df0b1a14a8fc0f", - "https://files.pythonhosted.org/packages/2f/36/77da9c6a328c54d17b960c89eccacfab8271fdaaa228305330915b88afa9/charset_normalizer-3.4.3-cp311-cp311-musllinux_1_2_x86_64.whl": "1e8ac75d72fa3775e0b7cb7e4629cec13b7514d928d15ef8ea06bca03ef01cae", - "https://files.pythonhosted.org/packages/31/e7/883ee5676a2ef217a40ce0bffcc3d0dfbf9e64cbcfbdf822c52981c3304b/charset_normalizer-3.4.3-cp312-cp312-musllinux_1_2_s390x.whl": "cc34f233c9e71701040d772aa7490318673aa7164a0efe3172b2981218c26d93", - "https://files.pythonhosted.org/packages/33/9e/eca49d35867ca2db336b6ca27617deed4653b97ebf45dfc21311ce473c37/charset_normalizer-3.4.3-cp310-cp310-musllinux_1_2_x86_64.whl": "78deba4d8f9590fe4dae384aeff04082510a709957e968753ff3c48399f6f92a", - "https://files.pythonhosted.org/packages/37/60/5d0d74bc1e1380f0b72c327948d9c2aca14b46a9efd87604e724260f384c/charset_normalizer-3.4.3-cp310-cp310-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl": "07a0eae9e2787b586e129fdcbe1af6997f8d0e5abaa0bc98c0e20e124d67e601", - "https://files.pythonhosted.org/packages/39/c6/99271dc37243a4f925b09090493fb96c9333d7992c6187f5cfe5312008d2/charset_normalizer-3.4.3-cp39-cp39-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl": "23b6b24d74478dc833444cbd927c338349d6ae852ba53a0d02a2de1fce45b96e", - "https://files.pythonhosted.org/packages/39/f5/3b3836ca6064d0992c58c7561c6b6eee1b3892e9665d650c803bd5614522/charset_normalizer-3.4.3-cp312-cp312-win_amd64.whl": "86df271bf921c2ee3818f0522e9a5b8092ca2ad8b065ece5d7d9d0e9f4849bcc", - "https://files.pythonhosted.org/packages/3a/a4/b3b6c76e7a635748c4421d2b92c7b8f90a432f98bda5082049af37ffc8e3/charset_normalizer-3.4.3-cp311-cp311-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl": "00237675befef519d9af72169d8604a067d92755e84fe76492fef5441db05b91", - "https://files.pythonhosted.org/packages/3b/38/20a1f44e4851aa1c9105d6e7110c9d020e093dfa5836d712a5f074a12bf7/charset_normalizer-3.4.3-cp310-cp310-musllinux_1_2_ppc64le.whl": "4ca4c094de7771a98d7fbd67d9e5dbf1eb73efa4f744a730437d8a3a5cf994f0", - "https://files.pythonhosted.org/packages/45/8c/dcef87cfc2b3f002a6478f38906f9040302c68aebe21468090e39cde1445/charset_normalizer-3.4.3-cp39-cp39-musllinux_1_2_x86_64.whl": "88ab34806dea0671532d3f82d82b85e8fc23d7b2dd12fa837978dad9bb392a34", - "https://files.pythonhosted.org/packages/4c/92/27dbe365d34c68cfe0ca76f1edd70e8705d82b378cb54ebbaeabc2e3029d/charset_normalizer-3.4.3-cp311-cp311-musllinux_1_2_ppc64le.whl": "939578d9d8fd4299220161fdd76e86c6a251987476f5243e8864a7844476ba14", - "https://files.pythonhosted.org/packages/50/10/c117806094d2c956ba88958dab680574019abc0c02bcf57b32287afca544/charset_normalizer-3.4.3-cp38-cp38-musllinux_1_2_x86_64.whl": "a2d08ac246bb48479170408d6c19f6385fa743e7157d716e144cad849b2dd94b", - "https://files.pythonhosted.org/packages/50/ee/f4704bad8201de513fdc8aac1cabc87e38c5818c93857140e06e772b5892/charset_normalizer-3.4.3-cp312-cp312-win32.whl": "fb6fecfd65564f208cbf0fba07f107fb661bcd1a7c389edbced3f7a493f70e37", - "https://files.pythonhosted.org/packages/59/c0/a74f3bd167d311365e7973990243f32c35e7a94e45103125275b9e6c479f/charset_normalizer-3.4.3-cp38-cp38-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl": "252098c8c7a873e17dd696ed98bbe91dbacd571da4b87df3736768efa7a792e4", - "https://files.pythonhosted.org/packages/60/f5/4659a4cb3c4ec146bec80c32d8bb16033752574c20b1252ee842a95d1a1e/charset_normalizer-3.4.3-cp313-cp313-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl": "1bb60174149316da1c35fa5233681f7c0f9f514509b8e399ab70fea5f17e45c9", - "https://files.pythonhosted.org/packages/61/c5/dc3ba772489c453621ffc27e8978a98fe7e41a93e787e5e5bde797f1dddb/charset_normalizer-3.4.3-cp38-cp38-win32.whl": "ec557499516fc90fd374bf2e32349a2887a876fbf162c160e3c01b6849eaf557", - "https://files.pythonhosted.org/packages/61/f1/190d9977e0084d3f1dc169acd060d479bbbc71b90bf3e7bf7b9927dec3eb/charset_normalizer-3.4.3-cp311-cp311-musllinux_1_2_aarch64.whl": "96b2b3d1a83ad55310de8c7b4a2d04d9277d5591f40761274856635acc5fcb30", - "https://files.pythonhosted.org/packages/63/86/9cbd533bd37883d467fcd1bd491b3547a3532d0fbb46de2b99feeebf185e/charset_normalizer-3.4.3-cp39-cp39-win32.whl": "16a8770207946ac75703458e2c743631c79c59c5890c80011d536248f8eaa432", - "https://files.pythonhosted.org/packages/64/d1/f9d141c893ef5d4243bc75c130e95af8fd4bc355beff06e9b1e941daad6e/charset_normalizer-3.4.3-cp38-cp38-musllinux_1_2_ppc64le.whl": "5b413b0b1bfd94dbf4023ad6945889f374cd24e3f62de58d6bb102c4d9ae534a", - "https://files.pythonhosted.org/packages/64/d4/9eb4ff2c167edbbf08cdd28e19078bf195762e9bd63371689cab5ecd3d0d/charset_normalizer-3.4.3-cp311-cp311-win32.whl": "6cf8fd4c04756b6b60146d98cd8a77d0cdae0e1ca20329da2ac85eed779b6849", - "https://files.pythonhosted.org/packages/65/1a/7425c952944a6521a9cfa7e675343f83fd82085b8af2b1373a2409c683dc/charset_normalizer-3.4.3-cp310-cp310-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl": "d0e909868420b7049dafd3a31d45125b31143eec59235311fc4c57ea26a4acd2", - "https://files.pythonhosted.org/packages/65/ca/2135ac97709b400c7654b4b764daf5c5567c2da45a30cdd20f9eefe2d658/charset_normalizer-3.4.3-cp313-cp313-macosx_10_13_universal2.whl": "14c2a87c65b351109f6abfc424cab3927b3bdece6f706e4d12faaf3d52ee5efe", - "https://files.pythonhosted.org/packages/70/99/f1c3bdcfaa9c45b3ce96f70b14f070411366fa19549c1d4832c935d8e2c3/charset_normalizer-3.4.3-cp313-cp313-musllinux_1_2_x86_64.whl": "18343b2d246dc6761a249ba1fb13f9ee9a2bcd95decc767319506056ea4ad4dc", - "https://files.pythonhosted.org/packages/71/11/98a04c3c97dd34e49c7d247083af03645ca3730809a5509443f3c37f7c99/charset_normalizer-3.4.3-cp313-cp313-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl": "41d1fc408ff5fdfb910200ec0e74abc40387bccb3252f3f27c0676731df2b2c8", - "https://files.pythonhosted.org/packages/72/2a/aff5dd112b2f14bcc3462c312dce5445806bfc8ab3a7328555da95330e4b/charset_normalizer-3.4.3-cp314-cp314-musllinux_1_2_x86_64.whl": "d716a916938e03231e86e43782ca7878fb602a125a91e7acb8b5112e2e96ac16", - "https://files.pythonhosted.org/packages/77/d9/cbcf1a2a5c7d7856f11e7ac2d782aec12bdfea60d104e60e0aa1c97849dc/charset_normalizer-3.4.3-cp313-cp313-musllinux_1_2_ppc64le.whl": "fdabf8315679312cfa71302f9bd509ded4f2f263fb5b765cf1433b39106c3cc9", - "https://files.pythonhosted.org/packages/7a/03/cbb6fac9d3e57f7e07ce062712ee80d80a5ab46614684078461917426279/charset_normalizer-3.4.3-cp38-cp38-musllinux_1_2_aarch64.whl": "d95bfb53c211b57198bb91c46dd5a2d8018b3af446583aab40074bf7988401cb", - "https://files.pythonhosted.org/packages/7d/a8/c6ec5d389672521f644505a257f50544c074cf5fc292d5390331cd6fc9c3/charset_normalizer-3.4.3-cp312-cp312-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl": "0cacf8f7297b0c4fcb74227692ca46b4a5852f8f4f24b3c766dd94a1075c4884", - "https://files.pythonhosted.org/packages/7e/61/19b36f4bd67f2793ab6a99b979b4e4f3d8fc754cbdffb805335df4337126/charset_normalizer-3.4.3-cp314-cp314-musllinux_1_2_ppc64le.whl": "53cd68b185d98dde4ad8990e56a58dea83a4162161b1ea9272e5c9182ce415e0", - "https://files.pythonhosted.org/packages/7e/95/42aa2156235cbc8fa61208aded06ef46111c4d3f0de233107b3f38631803/charset_normalizer-3.4.3-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl": "416175faf02e4b0810f1f38bcb54682878a4af94059a1cd63b8747244420801f", - "https://files.pythonhosted.org/packages/7f/b5/991245018615474a60965a7c9cd2b4efbaabd16d582a5547c47ee1c7730b/charset_normalizer-3.4.3-cp311-cp311-macosx_10_9_universal2.whl": "b256ee2e749283ef3ddcff51a675ff43798d92d746d1a6e4631bf8c707d22d0b", - "https://files.pythonhosted.org/packages/82/10/0fd19f20c624b278dddaf83b8464dcddc2456cb4b02bb902a6da126b87a1/charset_normalizer-3.4.3-cp312-cp312-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl": "3cfb2aad70f2c6debfbcb717f23b7eb55febc0bb23dcffc0f076009da10c6392", - "https://files.pythonhosted.org/packages/83/2d/5fd176ceb9b2fc619e63405525573493ca23441330fcdaee6bef9460e924/charset_normalizer-3.4.3.tar.gz": "6fce4b8500244f6fcb71465d4a4930d132ba9ab8e71a7859e6a5d59851068d14", - "https://files.pythonhosted.org/packages/85/9a/d891f63722d9158688de58d050c59dc3da560ea7f04f4c53e769de5140f5/charset_normalizer-3.4.3-cp310-cp310-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl": "74d77e25adda8581ffc1c720f1c81ca082921329452eba58b16233ab1842141c", - "https://files.pythonhosted.org/packages/86/9e/f552f7a00611f168b9a5865a1414179b2c6de8235a4fa40189f6f79a1753/charset_normalizer-3.4.3-cp313-cp313-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl": "30d006f98569de3459c2fc1f2acde170b7b2bd265dc1943e87e1a4efe1b67c31", - "https://files.pythonhosted.org/packages/87/df/b7737ff046c974b183ea9aa111b74185ac8c3a326c6262d413bd5a1b8c69/charset_normalizer-3.4.3-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl": "0e78314bdc32fa80696f72fa16dc61168fda4d6a0c014e0380f9d02f0e5d8a07", - "https://files.pythonhosted.org/packages/8a/1f/f041989e93b001bc4e44bb1669ccdcf54d3f00e628229a85b08d330615c5/charset_normalizer-3.4.3-py3-none-any.whl": "ce571ab16d890d23b5c278547ba694193a45011ff86a9162a71307ed9f86759a", - "https://files.pythonhosted.org/packages/8e/91/b5a06ad970ddc7a0e513112d40113e834638f4ca1120eb727a249fb2715e/charset_normalizer-3.4.3-cp314-cp314-macosx_10_13_universal2.whl": "3cd35b7e8aedeb9e34c41385fda4f73ba609e561faedfae0a9e75e44ac558a15", - "https://files.pythonhosted.org/packages/99/04/baae2a1ea1893a01635d475b9261c889a18fd48393634b6270827869fa34/charset_normalizer-3.4.3-cp311-cp311-musllinux_1_2_s390x.whl": "fd10de089bcdcd1be95a2f73dbe6254798ec1bda9f450d5828c96f93e2536b9c", - "https://files.pythonhosted.org/packages/9a/8f/ae790790c7b64f925e5c953b924aaa42a243fb778fed9e41f147b2a5715a/charset_normalizer-3.4.3-cp313-cp313-win_amd64.whl": "cf1ebb7d78e1ad8ec2a8c4732c7be2e736f6e5123a4146c5b89c9d1f585f8cef", - "https://files.pythonhosted.org/packages/a0/e4/5a075de8daa3ec0745a9a3b54467e0c2967daaaf2cec04c845f73493e9a1/charset_normalizer-3.4.3-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl": "18b97b8404387b96cdbd30ad660f6407799126d26a39ca65729162fd810a99aa", - "https://files.pythonhosted.org/packages/a3/ad/b0081f2f99a4b194bcbb1934ef3b12aa4d9702ced80a37026b7607c72e58/charset_normalizer-3.4.3-cp313-cp313-win32.whl": "6fb70de56f1859a3f71261cbe41005f56a7842cc348d3aeb26237560bfa5e0ce", - "https://files.pythonhosted.org/packages/a4/fa/384d2c0f57edad03d7bec3ebefb462090d8905b4ff5a2d2525f3bb711fac/charset_normalizer-3.4.3-cp310-cp310-musllinux_1_2_s390x.whl": "02425242e96bcf29a49711b0ca9f37e451da7c70562bc10e8ed992a5a7a25cc0", - "https://files.pythonhosted.org/packages/ae/02/e29e22b4e02839a0e4a06557b1999d0a47db3567e82989b5bb21f3fbbd9f/charset_normalizer-3.4.3-cp312-cp312-musllinux_1_2_aarch64.whl": "027b776c26d38b7f15b26a5da1044f376455fb3766df8fc38563b4efbc515154", - "https://files.pythonhosted.org/packages/b0/a8/6f5bcf1bcf63cb45625f7c5cadca026121ff8a6c8a3256d8d8cd59302663/charset_normalizer-3.4.3-cp39-cp39-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl": "257f26fed7d7ff59921b78244f3cd93ed2af1800ff048c33f624c87475819dd7", - "https://files.pythonhosted.org/packages/b7/8c/9839225320046ed279c6e839d51f028342eb77c91c89b8ef2549f951f3ec/charset_normalizer-3.4.3-cp314-cp314-win32.whl": "c6dbd0ccdda3a2ba7c2ecd9d77b37f3b5831687d8dc1b6ca5f56a4880cc7b7ce", - "https://files.pythonhosted.org/packages/c1/35/6525b21aa0db614cf8b5792d232021dca3df7f90a1944db934efa5d20bb1/charset_normalizer-3.4.3-cp312-cp312-musllinux_1_2_x86_64.whl": "320e8e66157cc4e247d9ddca8e21f427efc7a04bbd0ac8a9faf56583fa543f9f", - "https://files.pythonhosted.org/packages/c2/a9/3865b02c56f300a6f94fc631ef54f0a8a29da74fb45a773dfd3dcd380af7/charset_normalizer-3.4.3-cp313-cp313-musllinux_1_2_aarch64.whl": "6aab0f181c486f973bc7262a97f5aca3ee7e1437011ef0c2ec04b5a11d16c927", - "https://files.pythonhosted.org/packages/c2/ca/9a0983dd5c8e9733565cf3db4df2b0a2e9a82659fd8aa2a868ac6e4a991f/charset_normalizer-3.4.3-cp39-cp39-macosx_10_9_universal2.whl": "70bfc5f2c318afece2f5838ea5e4c3febada0be750fcf4775641052bbba14d05", - "https://files.pythonhosted.org/packages/c4/72/d3d0e9592f4e504f9dea08b8db270821c909558c353dc3b457ed2509f2fb/charset_normalizer-3.4.3-cp39-cp39-musllinux_1_2_aarch64.whl": "1ef99f0456d3d46a50945c98de1774da86f8e992ab5c77865ea8b8195341fc19", - "https://files.pythonhosted.org/packages/c5/35/9c99739250742375167bc1b1319cd1cec2bf67438a70d84b2e1ec4c9daa3/charset_normalizer-3.4.3-cp38-cp38-musllinux_1_2_s390x.whl": "b5e3b2d152e74e100a9e9573837aba24aab611d39428ded46f4e4022ea7d1942", - "https://files.pythonhosted.org/packages/c7/2a/ae245c41c06299ec18262825c1569c5d3298fc920e4ddf56ab011b417efd/charset_normalizer-3.4.3-cp311-cp311-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl": "13faeacfe61784e2559e690fc53fa4c5ae97c6fcedb8eb6fb8d0a15b475d2c64", - "https://files.pythonhosted.org/packages/ce/d6/7e805c8e5c46ff9729c49950acc4ee0aeb55efb8b3a56687658ad10c3216/charset_normalizer-3.4.3-cp39-cp39-win_amd64.whl": "d22dbedd33326a4a5190dd4fe9e9e693ef12160c77382d9e87919bce54f3d4ca", - "https://files.pythonhosted.org/packages/ce/ec/1edc30a377f0a02689342f214455c3f6c2fbedd896a1d2f856c002fc3062/charset_normalizer-3.4.3-cp314-cp314-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl": "b89bc04de1d83006373429975f8ef9e7932534b8cc9ca582e4db7d20d91816db", - "https://files.pythonhosted.org/packages/d6/98/f3b8013223728a99b908c9344da3aa04ee6e3fa235f19409033eda92fb78/charset_normalizer-3.4.3-cp310-cp310-macosx_10_9_universal2.whl": "fb7f67a1bfa6e40b438170ebdc8158b78dc465a5a67b6dde178a46987b244a72", - "https://files.pythonhosted.org/packages/e1/ef/dd08b2cac9284fd59e70f7d97382c33a3d0a926e45b15fc21b3308324ffd/charset_normalizer-3.4.3-cp39-cp39-musllinux_1_2_s390x.whl": "511729f456829ef86ac41ca78c63a5cb55240ed23b4b737faca0eb1abb1c41bc", - "https://files.pythonhosted.org/packages/e2/c6/f05db471f81af1fa01839d44ae2a8bfeec8d2a8b4590f16c4e7393afd323/charset_normalizer-3.4.3-cp310-cp310-win_amd64.whl": "c6e490913a46fa054e03699c70019ab869e990270597018cef1d8562132c2669", - "https://files.pythonhosted.org/packages/e2/e6/63bb0e10f90a8243c5def74b5b105b3bbbfb3e7bb753915fe333fb0c11ea/charset_normalizer-3.4.3-cp311-cp311-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl": "585f3b2a80fbd26b048a0be90c5aae8f06605d3c92615911c3a2b03a8a3b796f", - "https://files.pythonhosted.org/packages/e4/69/132eab043356bba06eb333cc2cc60c6340857d0a2e4ca6dc2b51312886b3/charset_normalizer-3.4.3-cp39-cp39-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl": "34a7f768e3f985abdb42841e20e17b330ad3aaf4bb7e7aeeb73db2e70f077b99", - "https://files.pythonhosted.org/packages/e9/5e/14c94999e418d9b87682734589404a25854d5f5d0408df68bc15b6ff54bb/charset_normalizer-3.4.3-cp312-cp312-macosx_10_13_universal2.whl": "e28e334d3ff134e88989d90ba04b47d84382a828c061d0d1027b1b12a62b39b1", - "https://files.pythonhosted.org/packages/ee/7a/36fbcf646e41f710ce0a563c1c9a343c6edf9be80786edeb15b6f62e17db/charset_normalizer-3.4.3-cp314-cp314-win_amd64.whl": "73dc19b562516fc9bcf6e5d6e596df0b4eb98d87e4f79f3ae71840e6ed21361c", - "https://files.pythonhosted.org/packages/f0/c9/a2c9c2a355a8594ce2446085e2ec97fd44d323c684ff32042e2a6b718e1d/charset_normalizer-3.4.3-cp310-cp310-musllinux_1_2_aarch64.whl": "c6f162aabe9a91a309510d74eeb6507fab5fff92337a15acbe77753d88d9dcf0", - "https://files.pythonhosted.org/packages/f1/e5/38421987f6c697ee3722981289d554957c4be652f963d71c5e46a262e135/charset_normalizer-3.4.3-cp314-cp314-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl": "8dcfc373f888e4fb39a7bc57e93e3b845e7f462dacc008d9749568b1c4ece096", - "https://files.pythonhosted.org/packages/f4/9c/996a4a028222e7761a96634d1820de8a744ff4327a00ada9c8942033089b/charset_normalizer-3.4.3-cp311-cp311-win_amd64.whl": "31a9a6f775f9bcd865d88ee350f0ffb0e25936a7f930ca98995c05abf1faf21c", - "https://files.pythonhosted.org/packages/f6/42/6f45efee8697b89fda4d50580f292b8f7f9306cb2971d4b53f8914e4d890/charset_normalizer-3.4.3-cp313-cp313-musllinux_1_2_s390x.whl": "bd28b817ea8c70215401f657edef3a8aa83c29d447fb0b622c35403780ba11d5", - "https://files.pythonhosted.org/packages/fc/eb/a2ffb08547f4e1e5415fb69eb7db25932c52a52bed371429648db4d84fb1/charset_normalizer-3.4.3-cp312-cp312-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl": "c6fd51128a41297f5409deab284fecbe5305ebd7e5a1f959bee1c054622b7018" + "https://files.pythonhosted.org/packages/00/5e/17398df3a139985ba9d11ed072531986f408c8fca952835ef1ab1820c02b/charset_normalizer-3.4.9-cp314-cp314t-macosx_10_15_universal2.whl": "609b3ba8fcc0fb5ab7af00719d0fb6ad0cb518e48e7712d12fd68f1327951198", + "https://files.pythonhosted.org/packages/01/c4/4fa4c8b3097a11f3c5f09a35b72ed6855fb1d332469504962ab7bafcc702/charset_normalizer-3.4.9-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl": "5e226f6218febc71f6c1fc2fafb91c226f75bdc1d8fb12d66823716e891608fd", + "https://files.pythonhosted.org/packages/01/da/a44bd7a13d426e69e4894557106cd58669097bfad4a8681123b618fbfc5d/charset_normalizer-3.4.9-cp310-cp310-win_arm64.whl": "375b83ed0aecfce76c16d198fbc21f3b11b337d68662bea0a995046682a11419", + "https://files.pythonhosted.org/packages/0b/e3/85ec501f206fb049259288c1f3506e53876937fb00edb47009348e66756b/charset_normalizer-3.4.9-cp311-cp311-macosx_10_9_universal2.whl": "0e94703ec9684807f20cfb5eed95c70f67f2a8f21ad620146d7b5a13677b93e5", + "https://files.pythonhosted.org/packages/0c/74/2f62c8821b969ea3bd67cc2e6976834f48ca5d12664d2559ebcd9bcfbed7/charset_normalizer-3.4.9-cp311-cp311-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl": "871ff67ea1aad4dfd91736464934d56b32dac49f9fbe16cddba36198a7b3a0db", + "https://files.pythonhosted.org/packages/0c/e7/aaf6da33fc9f4691cda8f7efbc9f69179d3d39ec8a4799baf273ee1d8db0/charset_normalizer-3.4.9-cp310-cp310-musllinux_1_2_armv7l.whl": "65a7ff3f705e57d392f7261b6d0550fe137c3019477431f1c355e0db0a7d3e15", + "https://files.pythonhosted.org/packages/0e/42/6dbc00b8cd16011691203e33570fa42ed5746599a2e878112d16eab403a3/charset_normalizer-3.4.9-cp312-cp312-win32.whl": "78841cccf1af7b40f6f716338d50c0902dbe88d9f800b3c973b7a9a0a693a642", + "https://files.pythonhosted.org/packages/10/e0/47c079dd82d217c807479cd59ffd30af56307ea31c108b75758970459ad3/charset_normalizer-3.4.9-cp314-cp314-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl": "4d1c96a7a18b9690a4d46df09e3e3382406ae3213727cd1019ebade1c4a81917", + "https://files.pythonhosted.org/packages/14/cb/1db8b96547ee3186cd2dd7f2e59dd560a9b80748f3604171f3c153d62811/charset_normalizer-3.4.9-cp39-cp39-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl": "58150c9f9b9a552505912d182ccdf26f6396fb6094816ceebcbb20eecabaed94", + "https://files.pythonhosted.org/packages/17/6d/bff78a4bacc4891bc63ec5bdc6776d8c85e47fab93d0d5f6223068fad0a4/charset_normalizer-3.4.9-cp39-cp39-win32.whl": "93d59d504b230e83c7a843251681959a0b6a9cd76f6e146ce1b8a80eb8739af9", + "https://files.pythonhosted.org/packages/19/79/55c32d06d76ae4feafe053f061f3e3ab70bcf19f4007797ce8c3efda7830/charset_normalizer-3.4.9-cp314-cp314-manylinux_2_31_armv7l.whl": "f7fb7d750cfa0a070d2c24e831fd3481019a60dd317ea2b39acbcebc08b6ed81", + "https://files.pythonhosted.org/packages/1d/85/181c652953eb5276d198f375b1dd641047392050098100a3a02d6534f657/charset_normalizer-3.4.9-cp310-cp310-musllinux_1_2_aarch64.whl": "e9701d0049d92c16703a42771b98d560b95248949f23f8cf7b4eddd201814fb9", + "https://files.pythonhosted.org/packages/20/95/d75e82f8ce9fd323ebf059c16c9aadefb22a1ecde13b7840b35835e4886c/charset_normalizer-3.4.9-cp314-cp314-win_arm64.whl": "40a126142a56b2dfc0aacbad1de8310cbf60da7656db0e6b16eebd48e3e93519", + "https://files.pythonhosted.org/packages/28/e9/9fb6099b868c82a40698a748ae0fbd4f31ccc13844c176a07158ba2abbfd/charset_normalizer-3.4.9-cp39-cp39-win_arm64.whl": "476743fe6dfe14a2da12e3ac79125dc84a3b2cf8094369a47a1529b0cd8549fe", + "https://files.pythonhosted.org/packages/2b/f9/ef4a69ea338ad3c0deceea0f5f7d2380ae8b52132b06d652cb0d2cd86706/charset_normalizer-3.4.9-cp314-cp314-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl": "8a79d9f4d8001473a30c163556b3c3bfebec837495a412dde78b51672f6134f9", + "https://files.pythonhosted.org/packages/33/9a/895095b83e7907abd6d3d99aad3a38ad0d9686cc186cb0c94c24320fe63e/charset_normalizer-3.4.9-cp312-cp312-musllinux_1_2_aarch64.whl": "60f44ade2cf573dad7a277e6f8ca9a51a21dda572b13bd7d8539bb3cd5dbedde", + "https://files.pythonhosted.org/packages/36/31/a276bb2e66243072a3fd06fdcab9cbb61a305b02143d70d2bda21d888fa8/charset_normalizer-3.4.9-cp310-cp310-manylinux_2_31_armv7l.whl": "bcf74c1df76758a395bf0af608c04c82257523f55c9868b334f06270d0f2112b", + "https://files.pythonhosted.org/packages/37/8d/ca39a7559a4797505530d084fd3a49a2c959efbbbff146302fb7be4e3b35/charset_normalizer-3.4.9-cp310-cp310-win_amd64.whl": "8c041122946b7ba21bb32c45b1aa57b1be35527690aeb3c5c234521085632eee", + "https://files.pythonhosted.org/packages/3a/25/45f30093ae27dd7b92a793b61882a38685f993700113ca36e0c9c14965e1/charset_normalizer-3.4.9-cp313-cp313-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl": "a4fbdde9dd4a9ce5fd52c2b3a347bb50cc89483ef783f1cb00d408c13f7a96c0", + "https://files.pythonhosted.org/packages/3d/c6/eee9dca4439b1061f76373f06ea855678cc4a64c1c3c90b50e479edbb8eb/charset_normalizer-3.4.9-cp314-cp314t-win_arm64.whl": "19ac87f93086ce37b86e098888555c4b4bc48102279bae3350098c0ed664b501", + "https://files.pythonhosted.org/packages/3d/ca/ad1d7c7d3077dab873f539d3e1d083c0845a762cb0bafdfbe3ef93add598/charset_normalizer-3.4.9-cp39-cp39-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl": "920079c3f7456fa213e0829ed2073aaa727fd39d889ead5b4f35d0de5460d04f", + "https://files.pythonhosted.org/packages/3d/ef/d96ec496cfea0c21db43b0ad03891308b02388d054cc902cf0e5a1ad6a88/charset_normalizer-3.4.9-cp314-cp314t-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl": "fa36ec09ef71d158186bc79e359ff5fdd6e7996fe8ab638f00d6b93139ba4fcf", + "https://files.pythonhosted.org/packages/42/ab/b9bc2e77d6b44a7e46ef62ec5cac1c9a6ba7b9135a5d560f002696ec9995/charset_normalizer-3.4.9-cp314-cp314-musllinux_1_2_aarch64.whl": "a4cfde78a9f2880208d16a93b795726a3017d5977e08d1e162a7a31322479c41", + "https://files.pythonhosted.org/packages/44/95/80282cce0fae9c3061203d723ee87da996aed79679e65d8935050ee7ca1f/charset_normalizer-3.4.9-cp311-cp311-manylinux_2_31_armv7l.whl": "c0323c9daef75ef2e5083624b4585018a0c9d5e3b40f607eed81a311270b934b", + "https://files.pythonhosted.org/packages/48/18/c8f397329c35e32f6a837e488986f4ae03bd2abebc453b48714991630c2f/charset_normalizer-3.4.9-cp313-cp313-musllinux_1_2_aarch64.whl": "416c229f77e5ea25b3dfd4b582f8d73d7e43c22320302b9ab128a2d3a0b38efe", + "https://files.pythonhosted.org/packages/49/ba/768fa3f36048d81c477a0ce61f813bc1454d80917ccfe550abd9f44f5e24/charset_normalizer-3.4.9-cp314-cp314-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl": "f840ed6d8ecba8255df8c42b87fadeda98ddfc6eeec05e2dc66e26d46dd6f58a", + "https://files.pythonhosted.org/packages/49/fd/a1d26144398c67486422a72bf5812cda22cb4ccfcd95a290fb41ceb4b8e2/charset_normalizer-3.4.9-cp314-cp314-win_amd64.whl": "16b65ea0f2465b6fb52aa22de5eca612aa964ddfec00a912e26f4656cbef890b", + "https://files.pythonhosted.org/packages/4b/4c/5361f9aa7f2cb58d94f2ab831b3d493f69efb1d239654b4744e3c09527cb/charset_normalizer-3.4.9-cp312-cp312-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl": "9104ed0bd76a429d46f9ec0dbc9b08ad1d2dcdf2b00a5a0daa1c145329b35b44", + "https://files.pythonhosted.org/packages/4e/6e/de0229a7ef40f6f9d28a837eebf4ec47bdca5dab4e900c84f22919af636a/charset_normalizer-3.4.9-cp311-cp311-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl": "4773092f8019072343a7447203308b176e10199920eb02d6195e81bbb3274c29", + "https://files.pythonhosted.org/packages/4f/8d/1569f4d0032d6ba2a4fe4591c35bf87868c600c41a71eb5c2e1ffa8464c2/charset_normalizer-3.4.9-cp311-cp311-win_arm64.whl": "1d22856ffbe153a602df38e4a5464f0b748a54002e0d69ac6d2ad0a197cc99ec", + "https://files.pythonhosted.org/packages/50/78/ce342ca4ff30b2eb49fe6d9578df85974f90c67d294113e94efdd9664cbd/charset_normalizer-3.4.9-cp312-cp312-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl": "7b86a2b16095d250c6f58b3d9b2eee6f4147754344f3dab0922f7c9bf7d226c9", + "https://files.pythonhosted.org/packages/52/94/af74dde74a3996bd959c350709bfe50e297823d70a8c1cbd54b838880863/charset_normalizer-3.4.9-cp314-cp314t-manylinux_2_31_armv7l.whl": "f86c6358749bd4fda175388691e3ba8c46e24c5347d0afd20f9b7edfc9faf07d", + "https://files.pythonhosted.org/packages/5e/be/7ee4453d7e88dfbc4104ccd34900b9f2c7c17dac22881865fe0e82424a25/charset_normalizer-3.4.9-cp310-cp310-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl": "b5314963fce9b0b12743891de876e724997864ee22aa496f903f426c7e2fa5b2", + "https://files.pythonhosted.org/packages/5f/c0/6eec7bdabe6cbbcc274ec04596f6d93865751a0541d33d60d1ce179bd372/charset_normalizer-3.4.9-cp39-cp39-musllinux_1_2_armv7l.whl": "ad41ba96094304aa090f5a30cb6e4fb3b3f1c264c523394b4c39bbacc4dc92ba", + "https://files.pythonhosted.org/packages/63/01/f2fb3bd3a73be48b173ee0c6aa8d2497af97d5663a8c4c4b491de4c62f7a/charset_normalizer-3.4.9-cp310-cp310-musllinux_1_2_x86_64.whl": "79580094b00d1789d1f93ea55bc43cb2f611910c72235b7657f3482ddcc1b22d", + "https://files.pythonhosted.org/packages/6a/05/c94d5cd23396289c54c93b02e0273b4dd8921641d9968c4828caf9bbaad9/charset_normalizer-3.4.9-cp39-cp39-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl": "df7276909358e5635ae203673ab7e509ddd224225a8d6b0790bf13eb2bde1cc5", + "https://files.pythonhosted.org/packages/6c/ef/2473d3c4d869155be4af1191111d59c4d5c4e0173026f7e85b176e23bf65/charset_normalizer-3.4.9-cp313-cp313-musllinux_1_2_x86_64.whl": "69b157c5d3292bcd443faca052f3096f637f1e074b98212a933c074ae23dc3b8", + "https://files.pythonhosted.org/packages/6d/46/79847edd07244a4a2d443c6655a7b6ee94203c21539414b059f32713c357/charset_normalizer-3.4.9-cp39-cp39-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl": "3c09a49d6cde137258beb3d551994a2927fd35ad5cf96aed573f61bbd67c5f84", + "https://files.pythonhosted.org/packages/6e/fb/d560d1d1555debbfe7849d9cac6145c1b537709d79576bf22557ed803b82/charset_normalizer-3.4.9-cp313-cp313-win_arm64.whl": "611057cc5d5c0afc743ba8be6bd828c17e0aaa8643f9d0a9b9bb7dea80eb8012", + "https://files.pythonhosted.org/packages/70/4a/ecbd131485c07fcdfad54e28946d513e3da22ef3b4bd854dcafae54ec739/charset_normalizer-3.4.9-cp312-cp312-macosx_10_13_universal2.whl": "45b0cc4e3556cd875e09102988d1ab8356c998b596c9fced84547c8138b487a0", + "https://files.pythonhosted.org/packages/78/ad/98aae8630ac71f16711968e38a5acfecce41b778bf2f0312851020f565a8/charset_normalizer-3.4.9-cp314-cp314t-musllinux_1_2_armv7l.whl": "cd6c3d4b783c556fa00bf540854e42f135e2f256abd29669fcd0da0f2dec79c2", + "https://files.pythonhosted.org/packages/7e/8d/496817fa0944239ecae662dd57ea765cfeaec6a735f9f025d4b7b72e7143/charset_normalizer-3.4.9-cp314-cp314-macosx_10_15_universal2.whl": "0327fcd59a935777d83410750c50600ee9571af2846f71ce40f25b13da1ef380", + "https://files.pythonhosted.org/packages/7f/f4/ffbb83546e1f198ecc70ecd372b65cf2b50f9068b380abd67640f17a8e18/charset_normalizer-3.4.9-cp310-cp310-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl": "16d10d789dd9bcca1173c95af82c58433122564b7bc39385124be735a35cbe99", + "https://files.pythonhosted.org/packages/80/33/6c99c1b3e6b8bf730e1bc809b9a2608f224145069114c479a2e9e1494346/charset_normalizer-3.4.9-cp310-cp310-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl": "c1225416b463483160e4af85d5fc3a9690ccb53fd4b1865a6437825f5ede3209", + "https://files.pythonhosted.org/packages/80/3f/bd97d3d9c613013d07cb7733d299385b41df37f0471310f5a73dc359f0b8/charset_normalizer-3.4.9-cp314-cp314t-win_amd64.whl": "9b8e0f3107e2200b76f6054de99016eac3ee6762713587b36baaa7e4bd2ae177", + "https://files.pythonhosted.org/packages/80/cc/f920afd1a23c58ccd53c1d36085a71893a4737ff5e66e0371efab6809850/charset_normalizer-3.4.9-cp312-cp312-win_amd64.whl": "4b3dac63058cc36820b0dd072f89898604e2d39686fe05321729d00d8ac185a0", + "https://files.pythonhosted.org/packages/83/d5/9096aa3cf532dfad237861544eb47a0f20d5adbf1039760fed8eaae935d9/charset_normalizer-3.4.9-cp311-cp311-win32.whl": "ac351b3b8014eead140e77e9717e2992c6bbe30b63bc3422422eb84865412e3d", + "https://files.pythonhosted.org/packages/83/dc/9b29fa4412b318bf3bfea985c35d67eb55e04b59a7c3f2237168b0e0be6f/charset_normalizer-3.4.9-cp312-cp312-musllinux_1_2_x86_64.whl": "03d07803992c6c7bbc976327f34b18b6160327fc81cb82c9d504720ac0be3b62", + "https://files.pythonhosted.org/packages/86/7e/5ce0bba863470fd1902d5e5843968951bddf38abe4742fc97116ef4598b3/charset_normalizer-3.4.9-cp313-cp313-musllinux_1_2_armv7l.whl": "75286256590a6320cf106a0d28970d3560aad9ee09aa7b34fb40524792436d35", + "https://files.pythonhosted.org/packages/87/3a/ad914516df7e358a81aae018caa5e0470ba827fa6d763b1d2e87d920a5f6/charset_normalizer-3.4.9-cp312-cp312-manylinux_2_31_armv7l.whl": "90c44bc373b7687f6948b693cceaea1348ae0975d7474746559494468e3c1d84", + "https://files.pythonhosted.org/packages/8c/e7/5ddfd76fc061eb52de219658a4aa431cbacadf0a0219c8854f00da50d289/charset_normalizer-3.4.9-cp314-cp314-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl": "33bdcc2a32c0a0e861f60841a512c8acc658c87c2ac59d89e3a46dacf7d866e4", + "https://files.pythonhosted.org/packages/92/8f/3a47a3667c83c2df9483d91644c6c107de3bf8874aa1793da9d3012eb986/charset_normalizer-3.4.9-cp313-cp313-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl": "e4fd89cc178bced6ad29cb3e6dd4aa63fa5017c3524dbd0b25998fb64a87cc8b", + "https://files.pythonhosted.org/packages/98/2b/f97f1c193fb855c345d678f5077d6926034db0722df74c8f057020e05a25/charset_normalizer-3.4.9-py3-none-any.whl": "68e5f26a1ad57ded6d1cfb85331d1c1a195314756471d97758c48498bb4dcdf5", + "https://files.pythonhosted.org/packages/9b/f2/c0d4b8508565a36bc5c624e88ed297f5b0b1095011034d7f5b83a69908b5/charset_normalizer-3.4.9-cp314-cp314-win32.whl": "c1c948747b03be832dceed96ca815cef7360de9aa19d37c730f8e3f6101aca48", + "https://files.pythonhosted.org/packages/a1/34/ef5c05f412f42520d7709b7d3784d19640839eb7366ded1755511585429f/charset_normalizer-3.4.9-cp312-cp312-musllinux_1_2_armv7l.whl": "a1786910334ed46ab1dd73222f2cd1e05c2c3bb39f6dddb4f8b36fc382058a39", + "https://files.pythonhosted.org/packages/a2/55/86048bde1c9d0352940bd7b87d825091a52aef67d01cde6c6f7342c5b552/charset_normalizer-3.4.9-cp39-cp39-win_amd64.whl": "ddf4af30b417d9fe16481e9b81c27ab2a7cde1ff7ba3e85653b02db7d145dc7b", + "https://files.pythonhosted.org/packages/a5/34/49b9060e8418b14fb5cba9cf6bfb383111e2538a03a1fb18e66a95aeb3d5/charset_normalizer-3.4.9-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl": "04ce310cb89c15df659582aee80a0603788732a5e017d5bd5c81158106ce249c", + "https://files.pythonhosted.org/packages/a5/ff/c946d63bc3786d5b84d960b0f7ab7e25b828486a946b5aa997625bcaf6a6/charset_normalizer-3.4.9-cp311-cp311-musllinux_1_2_armv7l.whl": "3d92613ec25e43b05f042302531ec0f00b8445190e43325880cbd6ab7c2581da", + "https://files.pythonhosted.org/packages/a6/ec/81e22253f4b7091eca6515bb3da5e45d05a663f7f567bb745695dc60f892/charset_normalizer-3.4.9-cp39-cp39-macosx_10_9_universal2.whl": "253a4a220747e8b5faf57ec320c4f5efb0cef05f647420bf267143ec15dba10a", + "https://files.pythonhosted.org/packages/ad/81/8e983840c6e5b93b33c2ba81aa3d52c2e42f0e9a690ce7607a2e61da4a5c/charset_normalizer-3.4.9-cp310-cp310-macosx_10_9_universal2.whl": "cd6280cf040f233bd7d3407b743b4b4c74f70e8e1c4199cb112a62c941c0772a", + "https://files.pythonhosted.org/packages/af/ba/5e5007c370702f85d2ef75791fac7943ed41e080364a673b20142e430e3e/charset_normalizer-3.4.9-cp311-cp311-musllinux_1_2_x86_64.whl": "280081916dc341820640489a66e4696049401ef1cf6dd672f672e70ad915aca3", + "https://files.pythonhosted.org/packages/b0/3e/faee8f9de92b14ee1198e9163252bb15efee7301b31256a3b6d9ebfdd0dd/charset_normalizer-3.4.9-cp313-cp313-manylinux_2_31_armv7l.whl": "5b10cd92fc5c498b35a8635df6d5a100207f88b63a4dc1de7ef9a548e1e2cd63", + "https://files.pythonhosted.org/packages/b1/27/693ee5e8a18191eb38647360c51cd505013e2bd3b366aa43fd5344c21e3c/charset_normalizer-3.4.9-cp314-cp314t-win32.whl": "0d861473f743244d349b50f850d10eb87aeb22bbdcc8e64f79273c94af5a8226", + "https://files.pythonhosted.org/packages/b2/06/97ec2aeae780b31d742b6352218b43841a6871e2564578ca522dce4a45c3/charset_normalizer-3.4.9-cp313-cp313-macosx_10_13_universal2.whl": "440eede837960000d74978f0eba527be106b5b9aee0daf779d395276ed0b0614", + "https://files.pythonhosted.org/packages/b3/46/03ddc7da576d814fe0a36dd1f0fd3258e95404b4b2e3c026b7923d7e133f/charset_normalizer-3.4.9-cp311-cp311-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl": "304b13570067b2547562e308af560b3963857b1fa90bd6afd978130130fe2d6a", + "https://files.pythonhosted.org/packages/bd/2a/23f34ec9d04624958e137efdc394888716353190e75f25dd22c7a2c7a8aa/charset_normalizer-3.4.9.tar.gz": "673611bbd43f0810bec0b0f028ddeaaa501190339cac411f347ac76917c3ae7b", + "https://files.pythonhosted.org/packages/c3/69/2a5385192e67175f7d8bd5ce4f57c24bc956439adeae5c13a99aa28a53d1/charset_normalizer-3.4.9-cp311-cp311-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl": "2a441ea71902098ffe78c5abe6c494f44160b4af614ed16c3d9a3b1d17fd8ee2", + "https://files.pythonhosted.org/packages/c4/02/c57a22739fe05246b0b5783b3bfb6afaac4eebb46f3ececdfb2f048f780e/charset_normalizer-3.4.9-cp310-cp310-win32.whl": "432786d3561e69aeeae6c7e8648964ce0ad05736120135601f87ac26b9c83381", + "https://files.pythonhosted.org/packages/c8/53/a8c042eb9eee4716f4d42a0f5a571eb32a09ec429be9fb0b8b9d765393ba/charset_normalizer-3.4.9-cp39-cp39-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl": "68ce9f4d6b26d5ccbf7fd4459bf75f74a0a146677ebba80597df60cbdb20e6f4", + "https://files.pythonhosted.org/packages/cb/32/2e64bd2be10e89c61e57ebe6a93fd98ae88eb7ebe414b5121f22c96c69eb/charset_normalizer-3.4.9-cp314-cp314t-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl": "cc1b0fff8ead343dae06305f954eb8468ba0ec1a97881f42489d198e4ce3c632", + "https://files.pythonhosted.org/packages/cd/91/7253a32e86b7e1d1239b1b36ba6dd0f021a21107ab33054b53119cc083b9/charset_normalizer-3.4.9-cp314-cp314t-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl": "51447e9aa2684679af07ca5021c3db526e0284347ebf4ffcec1154c3350cfe32", + "https://files.pythonhosted.org/packages/d0/39/8ff066c672434225f8d25f8b739f992af250944392173dcc88362681c9bf/charset_normalizer-3.4.9-cp313-cp313-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl": "21e764fd1e70b6a3e205a0e46f3051701f98a8cb3fad66eeb80e48bb502f8698", + "https://files.pythonhosted.org/packages/d0/a3/53ddae3db108a088156aa8ddfafd411ebbc1340f48c5573f697b27f69a39/charset_normalizer-3.4.9-cp313-cp313-win32.whl": "51307f5c71007673a2bf8232ad973483d281e74cb99c8c5a990af1eefa6277d9", + "https://files.pythonhosted.org/packages/d4/ce/9af95f7876194bd7a14e3dfe4a4de2e0bff02666a3910d72beafd06cc297/charset_normalizer-3.4.9-cp314-cp314t-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl": "df115d4d83168fdf2cae48ef1ff6d1cb4c466364e30861b37121de0f3bf1b990", + "https://files.pythonhosted.org/packages/d7/74/3c12f9755717dfe5c5c87da63f35d765fa0c00382ec26bf23f7fae34f2ba/charset_normalizer-3.4.9-cp312-cp312-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl": "9cdef90ae47919cae358d8ab15797a800ed41da7aba5d72419fb510729e2ed4b", + "https://files.pythonhosted.org/packages/d9/8d/feabb82cb49fcad14515b1d7d1ca4787b0da7fc723a212bf89bc9e0fac52/charset_normalizer-3.4.9-cp311-cp311-musllinux_1_2_aarch64.whl": "67830fc78e67501f47bb950471b2dcb9b35b140084429318e862895a8e89c993", + "https://files.pythonhosted.org/packages/de/d1/b4319dc3229d8272fba305e206fc0a148e2de8d4087917ce62ae6382f359/charset_normalizer-3.4.9-cp310-cp310-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl": "aa99adc8f081b475a12843953db36831eaf83ec33eb46a90629ca6a5de45a616", + "https://files.pythonhosted.org/packages/e4/56/6c745619ac397e8871e2bcd3cea1eec86b877488f33888b3aef5c3ed506e/charset_normalizer-3.4.9-cp314-cp314t-musllinux_1_2_aarch64.whl": "83aed2c10721ddd90f68140685391b50811a880af20654c59af6b6c66c40513c", + "https://files.pythonhosted.org/packages/e8/5f/b98b8da398637b551e427e7be922bdec19177dc54d6811dcdaa503f23aac/charset_normalizer-3.4.9-cp310-cp310-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl": "9bb41182d93ea91f60b4bc8fbf4c820c69ef8a12ab2d917f3f1834f1acad07e8", + "https://files.pythonhosted.org/packages/e8/ef/6953a77c7cf2c2ff9998e6f575ab3e380119f100223381565a4f94c1f836/charset_normalizer-3.4.9-cp313-cp313-win_amd64.whl": "fe2c7201c642b7c308f1675355ad7ff7b66acfe3541625efe5a3ad38f29d6115", + "https://files.pythonhosted.org/packages/ea/f8/72eb13dcabe7257035cea8aefd922caad2f110d252bf9f67c4c2ca763aee/charset_normalizer-3.4.9-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl": "84fd18bcc17526fc2b3c1af7d2b9217d32c9c04448c16ec693b9b4f1985c3d33", + "https://files.pythonhosted.org/packages/eb/78/59344ff9a4a7b5f6530bf7bec2c980047cc42c3a616596cdbd8cb5c1a1af/charset_normalizer-3.4.9-cp39-cp39-musllinux_1_2_x86_64.whl": "43b9e366a31fdd1c87d0eb08f579b4a82b723ea54338f040d6b4e518a026ea29", + "https://files.pythonhosted.org/packages/ec/96/5d9364e3342d69f3a045e1777bc47c85c383e6e9466d561b33fdb419d1f9/charset_normalizer-3.4.9-cp312-cp312-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl": "9b2aff1c7b3884512b9512c3eaadd9bab39fb45042ffaaa1dd08ff2b9f8109d9", + "https://files.pythonhosted.org/packages/ec/b4/ef5a49b2e77c00deb43bb3256592b115ba9e4346016e82c516b8d215bf68/charset_normalizer-3.4.9-cp39-cp39-manylinux_2_31_armv7l.whl": "231ddcbb35e2ff8973e1365db41fe0572662893b99a05deb183b68ad4c0c8bd4", + "https://files.pythonhosted.org/packages/ed/61/710738687f90d01c06a04ed52d6ca1e62dd9b1d8cc2567098167c4691034/charset_normalizer-3.4.9-cp39-cp39-musllinux_1_2_aarch64.whl": "0fa1aec2d32bcc03c8fa0f6f1712caad1adc38509f31142112e5c9daf5b9c833", + "https://files.pythonhosted.org/packages/ed/a1/e29995109e455dc8eff8d0fac6ae509be39561318a7cfeac5d33ad029213/charset_normalizer-3.4.9-cp311-cp311-win_amd64.whl": "6366a16e1a25018694d6a5d784d09b046edc9eac40ea2b54065c3052672516a1", + "https://files.pythonhosted.org/packages/ee/f0/f1c4fe746c395922961b5916ed1d7d6e7d4c84851d19ed43cc89980ec953/charset_normalizer-3.4.9-cp314-cp314t-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl": "32286a2c8d167e897177b673176c1e3e00d4057caf5d2b64eef9a3666b03018e", + "https://files.pythonhosted.org/packages/f0/e6/0386d43a261ff4e4b30c5857af7df877254b46bec7b9d1b74b6bf969a90b/charset_normalizer-3.4.9-cp312-cp312-win_arm64.whl": "78fa18e436a1a0e58dbd7e02fc4473f3f32cceb12df9dfca542d075961c307d2", + "https://files.pythonhosted.org/packages/f1/60/b22cdbee7e4013dab8b0d7647fc6181120fbbbc8f7025c226d15bd5a47fc/charset_normalizer-3.4.9-cp313-cp313-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl": "bd47ba7fc3ca94896759ea0109775132d3e7ab921fbf54038e1bab2e46c313c9", + "https://files.pythonhosted.org/packages/f1/78/c9c71d599f5aa2d42bcdd35cbbd46d7f535351a57e40ff7d8e5a7e219401/charset_normalizer-3.4.9-cp314-cp314-musllinux_1_2_armv7l.whl": "d4d6fcde76f94f5cb9e43e9e9a61f16dacefd228cbbf6f1a09bd9b219a92f1a1", + "https://files.pythonhosted.org/packages/f4/c4/b3e049d2aa3766180c78507110543d9d50894cc97f57de543f1be521dcdc/charset_normalizer-3.4.9-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl": "c25fe15c70c59eb7c5ce8c06a1f3fa1da0ecc5ea1e7a5922c40fd2fa9b0d5046", + "https://files.pythonhosted.org/packages/f6/39/c914445c321a845097ce4f6ac7de9a18228a77b766272125a1ce00d851eb/charset_normalizer-3.4.9-cp314-cp314-musllinux_1_2_x86_64.whl": "898f0e9068ca27d37f8e83a5b962821df851532e6c4a7d615c1c033f9da6eedf", + "https://files.pythonhosted.org/packages/f7/40/9593d54209765207a7f11073c06494c1721e4ca4a0a426c597679bf7f91e/charset_normalizer-3.4.9-cp314-cp314t-musllinux_1_2_x86_64.whl": "ee2f2a527e3c1a6e6411eb4209642e138b544a2d72fe5d0d76daf77b24063534" }, "cryptography": { "https://files.pythonhosted.org/packages/03/11/5e395f961d6868269835dee1bafec6a1ac176505a167f68b7d8818431068/cryptography-46.0.7-cp314-cp314t-manylinux2014_x86_64.manylinux_2_17_x86_64.whl": "ebd6daf519b9f189f85c479427bbd6e9c9037862cf8fe89ee35503bd209ed902", diff --git a/tools/publish/requirements_darwin.txt b/tools/publish/requirements_darwin.txt index 6d8de8b4c9..dd292fbaee 100644 --- a/tools/publish/requirements_darwin.txt +++ b/tools/publish/requirements_darwin.txt @@ -10,86 +10,100 @@ certifi==2025.10.5 \ --hash=sha256:0f212c2744a9bb6de0c56639a6f68afe01ecd92d91f14ae897c4fe7bbeeef0de \ --hash=sha256:47c09d31ccf2acf0be3f701ea53595ee7e0b8fa08801c6624be771df09ae7b43 # via requests -charset-normalizer==3.4.3 \ - --hash=sha256:00237675befef519d9af72169d8604a067d92755e84fe76492fef5441db05b91 \ - --hash=sha256:02425242e96bcf29a49711b0ca9f37e451da7c70562bc10e8ed992a5a7a25cc0 \ - --hash=sha256:027b776c26d38b7f15b26a5da1044f376455fb3766df8fc38563b4efbc515154 \ - --hash=sha256:07a0eae9e2787b586e129fdcbe1af6997f8d0e5abaa0bc98c0e20e124d67e601 \ - --hash=sha256:0cacf8f7297b0c4fcb74227692ca46b4a5852f8f4f24b3c766dd94a1075c4884 \ - --hash=sha256:0e78314bdc32fa80696f72fa16dc61168fda4d6a0c014e0380f9d02f0e5d8a07 \ - --hash=sha256:0f2be7e0cf7754b9a30eb01f4295cc3d4358a479843b31f328afd210e2c7598c \ - --hash=sha256:13faeacfe61784e2559e690fc53fa4c5ae97c6fcedb8eb6fb8d0a15b475d2c64 \ - --hash=sha256:14c2a87c65b351109f6abfc424cab3927b3bdece6f706e4d12faaf3d52ee5efe \ - --hash=sha256:1606f4a55c0fd363d754049cdf400175ee96c992b1f8018b993941f221221c5f \ - --hash=sha256:16a8770207946ac75703458e2c743631c79c59c5890c80011d536248f8eaa432 \ - --hash=sha256:18343b2d246dc6761a249ba1fb13f9ee9a2bcd95decc767319506056ea4ad4dc \ - --hash=sha256:18b97b8404387b96cdbd30ad660f6407799126d26a39ca65729162fd810a99aa \ - --hash=sha256:1bb60174149316da1c35fa5233681f7c0f9f514509b8e399ab70fea5f17e45c9 \ - --hash=sha256:1e8ac75d72fa3775e0b7cb7e4629cec13b7514d928d15ef8ea06bca03ef01cae \ - --hash=sha256:1ef99f0456d3d46a50945c98de1774da86f8e992ab5c77865ea8b8195341fc19 \ - --hash=sha256:2001a39612b241dae17b4687898843f254f8748b796a2e16f1051a17078d991d \ - --hash=sha256:23b6b24d74478dc833444cbd927c338349d6ae852ba53a0d02a2de1fce45b96e \ - --hash=sha256:252098c8c7a873e17dd696ed98bbe91dbacd571da4b87df3736768efa7a792e4 \ - --hash=sha256:257f26fed7d7ff59921b78244f3cd93ed2af1800ff048c33f624c87475819dd7 \ - --hash=sha256:2c322db9c8c89009a990ef07c3bcc9f011a3269bc06782f916cd3d9eed7c9312 \ - --hash=sha256:30a96e1e1f865f78b030d65241c1ee850cdf422d869e9028e2fc1d5e4db73b92 \ - --hash=sha256:30d006f98569de3459c2fc1f2acde170b7b2bd265dc1943e87e1a4efe1b67c31 \ - --hash=sha256:31a9a6f775f9bcd865d88ee350f0ffb0e25936a7f930ca98995c05abf1faf21c \ - --hash=sha256:320e8e66157cc4e247d9ddca8e21f427efc7a04bbd0ac8a9faf56583fa543f9f \ - --hash=sha256:34a7f768e3f985abdb42841e20e17b330ad3aaf4bb7e7aeeb73db2e70f077b99 \ - --hash=sha256:3653fad4fe3ed447a596ae8638b437f827234f01a8cd801842e43f3d0a6b281b \ - --hash=sha256:3cd35b7e8aedeb9e34c41385fda4f73ba609e561faedfae0a9e75e44ac558a15 \ - --hash=sha256:3cfb2aad70f2c6debfbcb717f23b7eb55febc0bb23dcffc0f076009da10c6392 \ - --hash=sha256:416175faf02e4b0810f1f38bcb54682878a4af94059a1cd63b8747244420801f \ - --hash=sha256:41d1fc408ff5fdfb910200ec0e74abc40387bccb3252f3f27c0676731df2b2c8 \ - --hash=sha256:42e5088973e56e31e4fa58eb6bd709e42fc03799c11c42929592889a2e54c491 \ - --hash=sha256:4ca4c094de7771a98d7fbd67d9e5dbf1eb73efa4f744a730437d8a3a5cf994f0 \ - --hash=sha256:511729f456829ef86ac41ca78c63a5cb55240ed23b4b737faca0eb1abb1c41bc \ - --hash=sha256:53cd68b185d98dde4ad8990e56a58dea83a4162161b1ea9272e5c9182ce415e0 \ - --hash=sha256:585f3b2a80fbd26b048a0be90c5aae8f06605d3c92615911c3a2b03a8a3b796f \ - --hash=sha256:5b413b0b1bfd94dbf4023ad6945889f374cd24e3f62de58d6bb102c4d9ae534a \ - --hash=sha256:5d8d01eac18c423815ed4f4a2ec3b439d654e55ee4ad610e153cf02faf67ea40 \ - --hash=sha256:6aab0f181c486f973bc7262a97f5aca3ee7e1437011ef0c2ec04b5a11d16c927 \ - --hash=sha256:6cf8fd4c04756b6b60146d98cd8a77d0cdae0e1ca20329da2ac85eed779b6849 \ - --hash=sha256:6fb70de56f1859a3f71261cbe41005f56a7842cc348d3aeb26237560bfa5e0ce \ - --hash=sha256:6fce4b8500244f6fcb71465d4a4930d132ba9ab8e71a7859e6a5d59851068d14 \ - --hash=sha256:70bfc5f2c318afece2f5838ea5e4c3febada0be750fcf4775641052bbba14d05 \ - --hash=sha256:73dc19b562516fc9bcf6e5d6e596df0b4eb98d87e4f79f3ae71840e6ed21361c \ - --hash=sha256:74d77e25adda8581ffc1c720f1c81ca082921329452eba58b16233ab1842141c \ - --hash=sha256:78deba4d8f9590fe4dae384aeff04082510a709957e968753ff3c48399f6f92a \ - --hash=sha256:86df271bf921c2ee3818f0522e9a5b8092ca2ad8b065ece5d7d9d0e9f4849bcc \ - --hash=sha256:88ab34806dea0671532d3f82d82b85e8fc23d7b2dd12fa837978dad9bb392a34 \ - --hash=sha256:8999f965f922ae054125286faf9f11bc6932184b93011d138925a1773830bbe9 \ - --hash=sha256:8dcfc373f888e4fb39a7bc57e93e3b845e7f462dacc008d9749568b1c4ece096 \ - --hash=sha256:939578d9d8fd4299220161fdd76e86c6a251987476f5243e8864a7844476ba14 \ - --hash=sha256:96b2b3d1a83ad55310de8c7b4a2d04d9277d5591f40761274856635acc5fcb30 \ - --hash=sha256:a2d08ac246bb48479170408d6c19f6385fa743e7157d716e144cad849b2dd94b \ - --hash=sha256:b256ee2e749283ef3ddcff51a675ff43798d92d746d1a6e4631bf8c707d22d0b \ - --hash=sha256:b5e3b2d152e74e100a9e9573837aba24aab611d39428ded46f4e4022ea7d1942 \ - --hash=sha256:b89bc04de1d83006373429975f8ef9e7932534b8cc9ca582e4db7d20d91816db \ - --hash=sha256:bd28b817ea8c70215401f657edef3a8aa83c29d447fb0b622c35403780ba11d5 \ - --hash=sha256:c60e092517a73c632ec38e290eba714e9627abe9d301c8c8a12ec32c314a2a4b \ - --hash=sha256:c6dbd0ccdda3a2ba7c2ecd9d77b37f3b5831687d8dc1b6ca5f56a4880cc7b7ce \ - --hash=sha256:c6e490913a46fa054e03699c70019ab869e990270597018cef1d8562132c2669 \ - --hash=sha256:c6f162aabe9a91a309510d74eeb6507fab5fff92337a15acbe77753d88d9dcf0 \ - --hash=sha256:c6fd51128a41297f5409deab284fecbe5305ebd7e5a1f959bee1c054622b7018 \ - --hash=sha256:cc34f233c9e71701040d772aa7490318673aa7164a0efe3172b2981218c26d93 \ - --hash=sha256:cc9370a2da1ac13f0153780040f465839e6cccb4a1e44810124b4e22483c93fe \ - --hash=sha256:ccf600859c183d70eb47e05a44cd80a4ce77394d1ac0f79dbd2dd90a69a3a049 \ - --hash=sha256:ce571ab16d890d23b5c278547ba694193a45011ff86a9162a71307ed9f86759a \ - --hash=sha256:cf1ebb7d78e1ad8ec2a8c4732c7be2e736f6e5123a4146c5b89c9d1f585f8cef \ - --hash=sha256:d0e909868420b7049dafd3a31d45125b31143eec59235311fc4c57ea26a4acd2 \ - --hash=sha256:d22dbedd33326a4a5190dd4fe9e9e693ef12160c77382d9e87919bce54f3d4ca \ - --hash=sha256:d716a916938e03231e86e43782ca7878fb602a125a91e7acb8b5112e2e96ac16 \ - --hash=sha256:d79c198e27580c8e958906f803e63cddb77653731be08851c7df0b1a14a8fc0f \ - --hash=sha256:d95bfb53c211b57198bb91c46dd5a2d8018b3af446583aab40074bf7988401cb \ - --hash=sha256:e28e334d3ff134e88989d90ba04b47d84382a828c061d0d1027b1b12a62b39b1 \ - --hash=sha256:ec557499516fc90fd374bf2e32349a2887a876fbf162c160e3c01b6849eaf557 \ - --hash=sha256:fb6fecfd65564f208cbf0fba07f107fb661bcd1a7c389edbced3f7a493f70e37 \ - --hash=sha256:fb731e5deb0c7ef82d698b0f4c5bb724633ee2a489401594c5c88b02e6cb15f7 \ - --hash=sha256:fb7f67a1bfa6e40b438170ebdc8158b78dc465a5a67b6dde178a46987b244a72 \ - --hash=sha256:fd10de089bcdcd1be95a2f73dbe6254798ec1bda9f450d5828c96f93e2536b9c \ - --hash=sha256:fdabf8315679312cfa71302f9bd509ded4f2f263fb5b765cf1433b39106c3cc9 +charset-normalizer==3.4.9 \ + --hash=sha256:0327fcd59a935777d83410750c50600ee9571af2846f71ce40f25b13da1ef380 \ + --hash=sha256:03d07803992c6c7bbc976327f34b18b6160327fc81cb82c9d504720ac0be3b62 \ + --hash=sha256:04ce310cb89c15df659582aee80a0603788732a5e017d5bd5c81158106ce249c \ + --hash=sha256:0d861473f743244d349b50f850d10eb87aeb22bbdcc8e64f79273c94af5a8226 \ + --hash=sha256:0e94703ec9684807f20cfb5eed95c70f67f2a8f21ad620146d7b5a13677b93e5 \ + --hash=sha256:0fa1aec2d32bcc03c8fa0f6f1712caad1adc38509f31142112e5c9daf5b9c833 \ + --hash=sha256:16b65ea0f2465b6fb52aa22de5eca612aa964ddfec00a912e26f4656cbef890b \ + --hash=sha256:16d10d789dd9bcca1173c95af82c58433122564b7bc39385124be735a35cbe99 \ + --hash=sha256:19ac87f93086ce37b86e098888555c4b4bc48102279bae3350098c0ed664b501 \ + --hash=sha256:1d22856ffbe153a602df38e4a5464f0b748a54002e0d69ac6d2ad0a197cc99ec \ + --hash=sha256:21e764fd1e70b6a3e205a0e46f3051701f98a8cb3fad66eeb80e48bb502f8698 \ + --hash=sha256:231ddcbb35e2ff8973e1365db41fe0572662893b99a05deb183b68ad4c0c8bd4 \ + --hash=sha256:253a4a220747e8b5faf57ec320c4f5efb0cef05f647420bf267143ec15dba10a \ + --hash=sha256:280081916dc341820640489a66e4696049401ef1cf6dd672f672e70ad915aca3 \ + --hash=sha256:2a441ea71902098ffe78c5abe6c494f44160b4af614ed16c3d9a3b1d17fd8ee2 \ + --hash=sha256:304b13570067b2547562e308af560b3963857b1fa90bd6afd978130130fe2d6a \ + --hash=sha256:32286a2c8d167e897177b673176c1e3e00d4057caf5d2b64eef9a3666b03018e \ + --hash=sha256:33bdcc2a32c0a0e861f60841a512c8acc658c87c2ac59d89e3a46dacf7d866e4 \ + --hash=sha256:375b83ed0aecfce76c16d198fbc21f3b11b337d68662bea0a995046682a11419 \ + --hash=sha256:3c09a49d6cde137258beb3d551994a2927fd35ad5cf96aed573f61bbd67c5f84 \ + --hash=sha256:3d92613ec25e43b05f042302531ec0f00b8445190e43325880cbd6ab7c2581da \ + --hash=sha256:40a126142a56b2dfc0aacbad1de8310cbf60da7656db0e6b16eebd48e3e93519 \ + --hash=sha256:416c229f77e5ea25b3dfd4b582f8d73d7e43c22320302b9ab128a2d3a0b38efe \ + --hash=sha256:432786d3561e69aeeae6c7e8648964ce0ad05736120135601f87ac26b9c83381 \ + --hash=sha256:43b9e366a31fdd1c87d0eb08f579b4a82b723ea54338f040d6b4e518a026ea29 \ + --hash=sha256:440eede837960000d74978f0eba527be106b5b9aee0daf779d395276ed0b0614 \ + --hash=sha256:45b0cc4e3556cd875e09102988d1ab8356c998b596c9fced84547c8138b487a0 \ + --hash=sha256:476743fe6dfe14a2da12e3ac79125dc84a3b2cf8094369a47a1529b0cd8549fe \ + --hash=sha256:4773092f8019072343a7447203308b176e10199920eb02d6195e81bbb3274c29 \ + --hash=sha256:4b3dac63058cc36820b0dd072f89898604e2d39686fe05321729d00d8ac185a0 \ + --hash=sha256:4d1c96a7a18b9690a4d46df09e3e3382406ae3213727cd1019ebade1c4a81917 \ + --hash=sha256:51307f5c71007673a2bf8232ad973483d281e74cb99c8c5a990af1eefa6277d9 \ + --hash=sha256:51447e9aa2684679af07ca5021c3db526e0284347ebf4ffcec1154c3350cfe32 \ + --hash=sha256:58150c9f9b9a552505912d182ccdf26f6396fb6094816ceebcbb20eecabaed94 \ + --hash=sha256:5b10cd92fc5c498b35a8635df6d5a100207f88b63a4dc1de7ef9a548e1e2cd63 \ + --hash=sha256:5e226f6218febc71f6c1fc2fafb91c226f75bdc1d8fb12d66823716e891608fd \ + --hash=sha256:609b3ba8fcc0fb5ab7af00719d0fb6ad0cb518e48e7712d12fd68f1327951198 \ + --hash=sha256:60f44ade2cf573dad7a277e6f8ca9a51a21dda572b13bd7d8539bb3cd5dbedde \ + --hash=sha256:611057cc5d5c0afc743ba8be6bd828c17e0aaa8643f9d0a9b9bb7dea80eb8012 \ + --hash=sha256:6366a16e1a25018694d6a5d784d09b046edc9eac40ea2b54065c3052672516a1 \ + --hash=sha256:65a7ff3f705e57d392f7261b6d0550fe137c3019477431f1c355e0db0a7d3e15 \ + --hash=sha256:673611bbd43f0810bec0b0f028ddeaaa501190339cac411f347ac76917c3ae7b \ + --hash=sha256:67830fc78e67501f47bb950471b2dcb9b35b140084429318e862895a8e89c993 \ + --hash=sha256:68ce9f4d6b26d5ccbf7fd4459bf75f74a0a146677ebba80597df60cbdb20e6f4 \ + --hash=sha256:68e5f26a1ad57ded6d1cfb85331d1c1a195314756471d97758c48498bb4dcdf5 \ + --hash=sha256:69b157c5d3292bcd443faca052f3096f637f1e074b98212a933c074ae23dc3b8 \ + --hash=sha256:75286256590a6320cf106a0d28970d3560aad9ee09aa7b34fb40524792436d35 \ + --hash=sha256:78841cccf1af7b40f6f716338d50c0902dbe88d9f800b3c973b7a9a0a693a642 \ + --hash=sha256:78fa18e436a1a0e58dbd7e02fc4473f3f32cceb12df9dfca542d075961c307d2 \ + --hash=sha256:79580094b00d1789d1f93ea55bc43cb2f611910c72235b7657f3482ddcc1b22d \ + --hash=sha256:7b86a2b16095d250c6f58b3d9b2eee6f4147754344f3dab0922f7c9bf7d226c9 \ + --hash=sha256:83aed2c10721ddd90f68140685391b50811a880af20654c59af6b6c66c40513c \ + --hash=sha256:84fd18bcc17526fc2b3c1af7d2b9217d32c9c04448c16ec693b9b4f1985c3d33 \ + --hash=sha256:871ff67ea1aad4dfd91736464934d56b32dac49f9fbe16cddba36198a7b3a0db \ + --hash=sha256:898f0e9068ca27d37f8e83a5b962821df851532e6c4a7d615c1c033f9da6eedf \ + --hash=sha256:8a79d9f4d8001473a30c163556b3c3bfebec837495a412dde78b51672f6134f9 \ + --hash=sha256:8c041122946b7ba21bb32c45b1aa57b1be35527690aeb3c5c234521085632eee \ + --hash=sha256:90c44bc373b7687f6948b693cceaea1348ae0975d7474746559494468e3c1d84 \ + --hash=sha256:9104ed0bd76a429d46f9ec0dbc9b08ad1d2dcdf2b00a5a0daa1c145329b35b44 \ + --hash=sha256:920079c3f7456fa213e0829ed2073aaa727fd39d889ead5b4f35d0de5460d04f \ + --hash=sha256:93d59d504b230e83c7a843251681959a0b6a9cd76f6e146ce1b8a80eb8739af9 \ + --hash=sha256:9b2aff1c7b3884512b9512c3eaadd9bab39fb45042ffaaa1dd08ff2b9f8109d9 \ + --hash=sha256:9b8e0f3107e2200b76f6054de99016eac3ee6762713587b36baaa7e4bd2ae177 \ + --hash=sha256:9bb41182d93ea91f60b4bc8fbf4c820c69ef8a12ab2d917f3f1834f1acad07e8 \ + --hash=sha256:9cdef90ae47919cae358d8ab15797a800ed41da7aba5d72419fb510729e2ed4b \ + --hash=sha256:a1786910334ed46ab1dd73222f2cd1e05c2c3bb39f6dddb4f8b36fc382058a39 \ + --hash=sha256:a4cfde78a9f2880208d16a93b795726a3017d5977e08d1e162a7a31322479c41 \ + --hash=sha256:a4fbdde9dd4a9ce5fd52c2b3a347bb50cc89483ef783f1cb00d408c13f7a96c0 \ + --hash=sha256:aa99adc8f081b475a12843953db36831eaf83ec33eb46a90629ca6a5de45a616 \ + --hash=sha256:ac351b3b8014eead140e77e9717e2992c6bbe30b63bc3422422eb84865412e3d \ + --hash=sha256:ad41ba96094304aa090f5a30cb6e4fb3b3f1c264c523394b4c39bbacc4dc92ba \ + --hash=sha256:b5314963fce9b0b12743891de876e724997864ee22aa496f903f426c7e2fa5b2 \ + --hash=sha256:bcf74c1df76758a395bf0af608c04c82257523f55c9868b334f06270d0f2112b \ + --hash=sha256:bd47ba7fc3ca94896759ea0109775132d3e7ab921fbf54038e1bab2e46c313c9 \ + --hash=sha256:c0323c9daef75ef2e5083624b4585018a0c9d5e3b40f607eed81a311270b934b \ + --hash=sha256:c1225416b463483160e4af85d5fc3a9690ccb53fd4b1865a6437825f5ede3209 \ + --hash=sha256:c1c948747b03be832dceed96ca815cef7360de9aa19d37c730f8e3f6101aca48 \ + --hash=sha256:c25fe15c70c59eb7c5ce8c06a1f3fa1da0ecc5ea1e7a5922c40fd2fa9b0d5046 \ + --hash=sha256:cc1b0fff8ead343dae06305f954eb8468ba0ec1a97881f42489d198e4ce3c632 \ + --hash=sha256:cd6280cf040f233bd7d3407b743b4b4c74f70e8e1c4199cb112a62c941c0772a \ + --hash=sha256:cd6c3d4b783c556fa00bf540854e42f135e2f256abd29669fcd0da0f2dec79c2 \ + --hash=sha256:d4d6fcde76f94f5cb9e43e9e9a61f16dacefd228cbbf6f1a09bd9b219a92f1a1 \ + --hash=sha256:ddf4af30b417d9fe16481e9b81c27ab2a7cde1ff7ba3e85653b02db7d145dc7b \ + --hash=sha256:df115d4d83168fdf2cae48ef1ff6d1cb4c466364e30861b37121de0f3bf1b990 \ + --hash=sha256:df7276909358e5635ae203673ab7e509ddd224225a8d6b0790bf13eb2bde1cc5 \ + --hash=sha256:e4fd89cc178bced6ad29cb3e6dd4aa63fa5017c3524dbd0b25998fb64a87cc8b \ + --hash=sha256:e9701d0049d92c16703a42771b98d560b95248949f23f8cf7b4eddd201814fb9 \ + --hash=sha256:ee2f2a527e3c1a6e6411eb4209642e138b544a2d72fe5d0d76daf77b24063534 \ + --hash=sha256:f7fb7d750cfa0a070d2c24e831fd3481019a60dd317ea2b39acbcebc08b6ed81 \ + --hash=sha256:f840ed6d8ecba8255df8c42b87fadeda98ddfc6eeec05e2dc66e26d46dd6f58a \ + --hash=sha256:f86c6358749bd4fda175388691e3ba8c46e24c5347d0afd20f9b7edfc9faf07d \ + --hash=sha256:fa36ec09ef71d158186bc79e359ff5fdd6e7996fe8ab638f00d6b93139ba4fcf \ + --hash=sha256:fe2c7201c642b7c308f1675355ad7ff7b66acfe3541625efe5a3ad38f29d6115 # via requests docutils==0.22.2 \ --hash=sha256:9fdb771707c8784c8f2728b67cb2c691305933d68137ef95a75db5f4dfbc213d \ diff --git a/tools/publish/requirements_linux.txt b/tools/publish/requirements_linux.txt index 8865eb3ecb..cfba2aceea 100644 --- a/tools/publish/requirements_linux.txt +++ b/tools/publish/requirements_linux.txt @@ -96,86 +96,100 @@ cffi==2.0.0 \ --hash=sha256:fc7de24befaeae77ba923797c7c87834c73648a05a4bde34b3b7e5588973a453 \ --hash=sha256:fe562eb1a64e67dd297ccc4f5addea2501664954f2692b69a76449ec7913ecbf # via cryptography -charset-normalizer==3.4.3 \ - --hash=sha256:00237675befef519d9af72169d8604a067d92755e84fe76492fef5441db05b91 \ - --hash=sha256:02425242e96bcf29a49711b0ca9f37e451da7c70562bc10e8ed992a5a7a25cc0 \ - --hash=sha256:027b776c26d38b7f15b26a5da1044f376455fb3766df8fc38563b4efbc515154 \ - --hash=sha256:07a0eae9e2787b586e129fdcbe1af6997f8d0e5abaa0bc98c0e20e124d67e601 \ - --hash=sha256:0cacf8f7297b0c4fcb74227692ca46b4a5852f8f4f24b3c766dd94a1075c4884 \ - --hash=sha256:0e78314bdc32fa80696f72fa16dc61168fda4d6a0c014e0380f9d02f0e5d8a07 \ - --hash=sha256:0f2be7e0cf7754b9a30eb01f4295cc3d4358a479843b31f328afd210e2c7598c \ - --hash=sha256:13faeacfe61784e2559e690fc53fa4c5ae97c6fcedb8eb6fb8d0a15b475d2c64 \ - --hash=sha256:14c2a87c65b351109f6abfc424cab3927b3bdece6f706e4d12faaf3d52ee5efe \ - --hash=sha256:1606f4a55c0fd363d754049cdf400175ee96c992b1f8018b993941f221221c5f \ - --hash=sha256:16a8770207946ac75703458e2c743631c79c59c5890c80011d536248f8eaa432 \ - --hash=sha256:18343b2d246dc6761a249ba1fb13f9ee9a2bcd95decc767319506056ea4ad4dc \ - --hash=sha256:18b97b8404387b96cdbd30ad660f6407799126d26a39ca65729162fd810a99aa \ - --hash=sha256:1bb60174149316da1c35fa5233681f7c0f9f514509b8e399ab70fea5f17e45c9 \ - --hash=sha256:1e8ac75d72fa3775e0b7cb7e4629cec13b7514d928d15ef8ea06bca03ef01cae \ - --hash=sha256:1ef99f0456d3d46a50945c98de1774da86f8e992ab5c77865ea8b8195341fc19 \ - --hash=sha256:2001a39612b241dae17b4687898843f254f8748b796a2e16f1051a17078d991d \ - --hash=sha256:23b6b24d74478dc833444cbd927c338349d6ae852ba53a0d02a2de1fce45b96e \ - --hash=sha256:252098c8c7a873e17dd696ed98bbe91dbacd571da4b87df3736768efa7a792e4 \ - --hash=sha256:257f26fed7d7ff59921b78244f3cd93ed2af1800ff048c33f624c87475819dd7 \ - --hash=sha256:2c322db9c8c89009a990ef07c3bcc9f011a3269bc06782f916cd3d9eed7c9312 \ - --hash=sha256:30a96e1e1f865f78b030d65241c1ee850cdf422d869e9028e2fc1d5e4db73b92 \ - --hash=sha256:30d006f98569de3459c2fc1f2acde170b7b2bd265dc1943e87e1a4efe1b67c31 \ - --hash=sha256:31a9a6f775f9bcd865d88ee350f0ffb0e25936a7f930ca98995c05abf1faf21c \ - --hash=sha256:320e8e66157cc4e247d9ddca8e21f427efc7a04bbd0ac8a9faf56583fa543f9f \ - --hash=sha256:34a7f768e3f985abdb42841e20e17b330ad3aaf4bb7e7aeeb73db2e70f077b99 \ - --hash=sha256:3653fad4fe3ed447a596ae8638b437f827234f01a8cd801842e43f3d0a6b281b \ - --hash=sha256:3cd35b7e8aedeb9e34c41385fda4f73ba609e561faedfae0a9e75e44ac558a15 \ - --hash=sha256:3cfb2aad70f2c6debfbcb717f23b7eb55febc0bb23dcffc0f076009da10c6392 \ - --hash=sha256:416175faf02e4b0810f1f38bcb54682878a4af94059a1cd63b8747244420801f \ - --hash=sha256:41d1fc408ff5fdfb910200ec0e74abc40387bccb3252f3f27c0676731df2b2c8 \ - --hash=sha256:42e5088973e56e31e4fa58eb6bd709e42fc03799c11c42929592889a2e54c491 \ - --hash=sha256:4ca4c094de7771a98d7fbd67d9e5dbf1eb73efa4f744a730437d8a3a5cf994f0 \ - --hash=sha256:511729f456829ef86ac41ca78c63a5cb55240ed23b4b737faca0eb1abb1c41bc \ - --hash=sha256:53cd68b185d98dde4ad8990e56a58dea83a4162161b1ea9272e5c9182ce415e0 \ - --hash=sha256:585f3b2a80fbd26b048a0be90c5aae8f06605d3c92615911c3a2b03a8a3b796f \ - --hash=sha256:5b413b0b1bfd94dbf4023ad6945889f374cd24e3f62de58d6bb102c4d9ae534a \ - --hash=sha256:5d8d01eac18c423815ed4f4a2ec3b439d654e55ee4ad610e153cf02faf67ea40 \ - --hash=sha256:6aab0f181c486f973bc7262a97f5aca3ee7e1437011ef0c2ec04b5a11d16c927 \ - --hash=sha256:6cf8fd4c04756b6b60146d98cd8a77d0cdae0e1ca20329da2ac85eed779b6849 \ - --hash=sha256:6fb70de56f1859a3f71261cbe41005f56a7842cc348d3aeb26237560bfa5e0ce \ - --hash=sha256:6fce4b8500244f6fcb71465d4a4930d132ba9ab8e71a7859e6a5d59851068d14 \ - --hash=sha256:70bfc5f2c318afece2f5838ea5e4c3febada0be750fcf4775641052bbba14d05 \ - --hash=sha256:73dc19b562516fc9bcf6e5d6e596df0b4eb98d87e4f79f3ae71840e6ed21361c \ - --hash=sha256:74d77e25adda8581ffc1c720f1c81ca082921329452eba58b16233ab1842141c \ - --hash=sha256:78deba4d8f9590fe4dae384aeff04082510a709957e968753ff3c48399f6f92a \ - --hash=sha256:86df271bf921c2ee3818f0522e9a5b8092ca2ad8b065ece5d7d9d0e9f4849bcc \ - --hash=sha256:88ab34806dea0671532d3f82d82b85e8fc23d7b2dd12fa837978dad9bb392a34 \ - --hash=sha256:8999f965f922ae054125286faf9f11bc6932184b93011d138925a1773830bbe9 \ - --hash=sha256:8dcfc373f888e4fb39a7bc57e93e3b845e7f462dacc008d9749568b1c4ece096 \ - --hash=sha256:939578d9d8fd4299220161fdd76e86c6a251987476f5243e8864a7844476ba14 \ - --hash=sha256:96b2b3d1a83ad55310de8c7b4a2d04d9277d5591f40761274856635acc5fcb30 \ - --hash=sha256:a2d08ac246bb48479170408d6c19f6385fa743e7157d716e144cad849b2dd94b \ - --hash=sha256:b256ee2e749283ef3ddcff51a675ff43798d92d746d1a6e4631bf8c707d22d0b \ - --hash=sha256:b5e3b2d152e74e100a9e9573837aba24aab611d39428ded46f4e4022ea7d1942 \ - --hash=sha256:b89bc04de1d83006373429975f8ef9e7932534b8cc9ca582e4db7d20d91816db \ - --hash=sha256:bd28b817ea8c70215401f657edef3a8aa83c29d447fb0b622c35403780ba11d5 \ - --hash=sha256:c60e092517a73c632ec38e290eba714e9627abe9d301c8c8a12ec32c314a2a4b \ - --hash=sha256:c6dbd0ccdda3a2ba7c2ecd9d77b37f3b5831687d8dc1b6ca5f56a4880cc7b7ce \ - --hash=sha256:c6e490913a46fa054e03699c70019ab869e990270597018cef1d8562132c2669 \ - --hash=sha256:c6f162aabe9a91a309510d74eeb6507fab5fff92337a15acbe77753d88d9dcf0 \ - --hash=sha256:c6fd51128a41297f5409deab284fecbe5305ebd7e5a1f959bee1c054622b7018 \ - --hash=sha256:cc34f233c9e71701040d772aa7490318673aa7164a0efe3172b2981218c26d93 \ - --hash=sha256:cc9370a2da1ac13f0153780040f465839e6cccb4a1e44810124b4e22483c93fe \ - --hash=sha256:ccf600859c183d70eb47e05a44cd80a4ce77394d1ac0f79dbd2dd90a69a3a049 \ - --hash=sha256:ce571ab16d890d23b5c278547ba694193a45011ff86a9162a71307ed9f86759a \ - --hash=sha256:cf1ebb7d78e1ad8ec2a8c4732c7be2e736f6e5123a4146c5b89c9d1f585f8cef \ - --hash=sha256:d0e909868420b7049dafd3a31d45125b31143eec59235311fc4c57ea26a4acd2 \ - --hash=sha256:d22dbedd33326a4a5190dd4fe9e9e693ef12160c77382d9e87919bce54f3d4ca \ - --hash=sha256:d716a916938e03231e86e43782ca7878fb602a125a91e7acb8b5112e2e96ac16 \ - --hash=sha256:d79c198e27580c8e958906f803e63cddb77653731be08851c7df0b1a14a8fc0f \ - --hash=sha256:d95bfb53c211b57198bb91c46dd5a2d8018b3af446583aab40074bf7988401cb \ - --hash=sha256:e28e334d3ff134e88989d90ba04b47d84382a828c061d0d1027b1b12a62b39b1 \ - --hash=sha256:ec557499516fc90fd374bf2e32349a2887a876fbf162c160e3c01b6849eaf557 \ - --hash=sha256:fb6fecfd65564f208cbf0fba07f107fb661bcd1a7c389edbced3f7a493f70e37 \ - --hash=sha256:fb731e5deb0c7ef82d698b0f4c5bb724633ee2a489401594c5c88b02e6cb15f7 \ - --hash=sha256:fb7f67a1bfa6e40b438170ebdc8158b78dc465a5a67b6dde178a46987b244a72 \ - --hash=sha256:fd10de089bcdcd1be95a2f73dbe6254798ec1bda9f450d5828c96f93e2536b9c \ - --hash=sha256:fdabf8315679312cfa71302f9bd509ded4f2f263fb5b765cf1433b39106c3cc9 +charset-normalizer==3.4.9 \ + --hash=sha256:0327fcd59a935777d83410750c50600ee9571af2846f71ce40f25b13da1ef380 \ + --hash=sha256:03d07803992c6c7bbc976327f34b18b6160327fc81cb82c9d504720ac0be3b62 \ + --hash=sha256:04ce310cb89c15df659582aee80a0603788732a5e017d5bd5c81158106ce249c \ + --hash=sha256:0d861473f743244d349b50f850d10eb87aeb22bbdcc8e64f79273c94af5a8226 \ + --hash=sha256:0e94703ec9684807f20cfb5eed95c70f67f2a8f21ad620146d7b5a13677b93e5 \ + --hash=sha256:0fa1aec2d32bcc03c8fa0f6f1712caad1adc38509f31142112e5c9daf5b9c833 \ + --hash=sha256:16b65ea0f2465b6fb52aa22de5eca612aa964ddfec00a912e26f4656cbef890b \ + --hash=sha256:16d10d789dd9bcca1173c95af82c58433122564b7bc39385124be735a35cbe99 \ + --hash=sha256:19ac87f93086ce37b86e098888555c4b4bc48102279bae3350098c0ed664b501 \ + --hash=sha256:1d22856ffbe153a602df38e4a5464f0b748a54002e0d69ac6d2ad0a197cc99ec \ + --hash=sha256:21e764fd1e70b6a3e205a0e46f3051701f98a8cb3fad66eeb80e48bb502f8698 \ + --hash=sha256:231ddcbb35e2ff8973e1365db41fe0572662893b99a05deb183b68ad4c0c8bd4 \ + --hash=sha256:253a4a220747e8b5faf57ec320c4f5efb0cef05f647420bf267143ec15dba10a \ + --hash=sha256:280081916dc341820640489a66e4696049401ef1cf6dd672f672e70ad915aca3 \ + --hash=sha256:2a441ea71902098ffe78c5abe6c494f44160b4af614ed16c3d9a3b1d17fd8ee2 \ + --hash=sha256:304b13570067b2547562e308af560b3963857b1fa90bd6afd978130130fe2d6a \ + --hash=sha256:32286a2c8d167e897177b673176c1e3e00d4057caf5d2b64eef9a3666b03018e \ + --hash=sha256:33bdcc2a32c0a0e861f60841a512c8acc658c87c2ac59d89e3a46dacf7d866e4 \ + --hash=sha256:375b83ed0aecfce76c16d198fbc21f3b11b337d68662bea0a995046682a11419 \ + --hash=sha256:3c09a49d6cde137258beb3d551994a2927fd35ad5cf96aed573f61bbd67c5f84 \ + --hash=sha256:3d92613ec25e43b05f042302531ec0f00b8445190e43325880cbd6ab7c2581da \ + --hash=sha256:40a126142a56b2dfc0aacbad1de8310cbf60da7656db0e6b16eebd48e3e93519 \ + --hash=sha256:416c229f77e5ea25b3dfd4b582f8d73d7e43c22320302b9ab128a2d3a0b38efe \ + --hash=sha256:432786d3561e69aeeae6c7e8648964ce0ad05736120135601f87ac26b9c83381 \ + --hash=sha256:43b9e366a31fdd1c87d0eb08f579b4a82b723ea54338f040d6b4e518a026ea29 \ + --hash=sha256:440eede837960000d74978f0eba527be106b5b9aee0daf779d395276ed0b0614 \ + --hash=sha256:45b0cc4e3556cd875e09102988d1ab8356c998b596c9fced84547c8138b487a0 \ + --hash=sha256:476743fe6dfe14a2da12e3ac79125dc84a3b2cf8094369a47a1529b0cd8549fe \ + --hash=sha256:4773092f8019072343a7447203308b176e10199920eb02d6195e81bbb3274c29 \ + --hash=sha256:4b3dac63058cc36820b0dd072f89898604e2d39686fe05321729d00d8ac185a0 \ + --hash=sha256:4d1c96a7a18b9690a4d46df09e3e3382406ae3213727cd1019ebade1c4a81917 \ + --hash=sha256:51307f5c71007673a2bf8232ad973483d281e74cb99c8c5a990af1eefa6277d9 \ + --hash=sha256:51447e9aa2684679af07ca5021c3db526e0284347ebf4ffcec1154c3350cfe32 \ + --hash=sha256:58150c9f9b9a552505912d182ccdf26f6396fb6094816ceebcbb20eecabaed94 \ + --hash=sha256:5b10cd92fc5c498b35a8635df6d5a100207f88b63a4dc1de7ef9a548e1e2cd63 \ + --hash=sha256:5e226f6218febc71f6c1fc2fafb91c226f75bdc1d8fb12d66823716e891608fd \ + --hash=sha256:609b3ba8fcc0fb5ab7af00719d0fb6ad0cb518e48e7712d12fd68f1327951198 \ + --hash=sha256:60f44ade2cf573dad7a277e6f8ca9a51a21dda572b13bd7d8539bb3cd5dbedde \ + --hash=sha256:611057cc5d5c0afc743ba8be6bd828c17e0aaa8643f9d0a9b9bb7dea80eb8012 \ + --hash=sha256:6366a16e1a25018694d6a5d784d09b046edc9eac40ea2b54065c3052672516a1 \ + --hash=sha256:65a7ff3f705e57d392f7261b6d0550fe137c3019477431f1c355e0db0a7d3e15 \ + --hash=sha256:673611bbd43f0810bec0b0f028ddeaaa501190339cac411f347ac76917c3ae7b \ + --hash=sha256:67830fc78e67501f47bb950471b2dcb9b35b140084429318e862895a8e89c993 \ + --hash=sha256:68ce9f4d6b26d5ccbf7fd4459bf75f74a0a146677ebba80597df60cbdb20e6f4 \ + --hash=sha256:68e5f26a1ad57ded6d1cfb85331d1c1a195314756471d97758c48498bb4dcdf5 \ + --hash=sha256:69b157c5d3292bcd443faca052f3096f637f1e074b98212a933c074ae23dc3b8 \ + --hash=sha256:75286256590a6320cf106a0d28970d3560aad9ee09aa7b34fb40524792436d35 \ + --hash=sha256:78841cccf1af7b40f6f716338d50c0902dbe88d9f800b3c973b7a9a0a693a642 \ + --hash=sha256:78fa18e436a1a0e58dbd7e02fc4473f3f32cceb12df9dfca542d075961c307d2 \ + --hash=sha256:79580094b00d1789d1f93ea55bc43cb2f611910c72235b7657f3482ddcc1b22d \ + --hash=sha256:7b86a2b16095d250c6f58b3d9b2eee6f4147754344f3dab0922f7c9bf7d226c9 \ + --hash=sha256:83aed2c10721ddd90f68140685391b50811a880af20654c59af6b6c66c40513c \ + --hash=sha256:84fd18bcc17526fc2b3c1af7d2b9217d32c9c04448c16ec693b9b4f1985c3d33 \ + --hash=sha256:871ff67ea1aad4dfd91736464934d56b32dac49f9fbe16cddba36198a7b3a0db \ + --hash=sha256:898f0e9068ca27d37f8e83a5b962821df851532e6c4a7d615c1c033f9da6eedf \ + --hash=sha256:8a79d9f4d8001473a30c163556b3c3bfebec837495a412dde78b51672f6134f9 \ + --hash=sha256:8c041122946b7ba21bb32c45b1aa57b1be35527690aeb3c5c234521085632eee \ + --hash=sha256:90c44bc373b7687f6948b693cceaea1348ae0975d7474746559494468e3c1d84 \ + --hash=sha256:9104ed0bd76a429d46f9ec0dbc9b08ad1d2dcdf2b00a5a0daa1c145329b35b44 \ + --hash=sha256:920079c3f7456fa213e0829ed2073aaa727fd39d889ead5b4f35d0de5460d04f \ + --hash=sha256:93d59d504b230e83c7a843251681959a0b6a9cd76f6e146ce1b8a80eb8739af9 \ + --hash=sha256:9b2aff1c7b3884512b9512c3eaadd9bab39fb45042ffaaa1dd08ff2b9f8109d9 \ + --hash=sha256:9b8e0f3107e2200b76f6054de99016eac3ee6762713587b36baaa7e4bd2ae177 \ + --hash=sha256:9bb41182d93ea91f60b4bc8fbf4c820c69ef8a12ab2d917f3f1834f1acad07e8 \ + --hash=sha256:9cdef90ae47919cae358d8ab15797a800ed41da7aba5d72419fb510729e2ed4b \ + --hash=sha256:a1786910334ed46ab1dd73222f2cd1e05c2c3bb39f6dddb4f8b36fc382058a39 \ + --hash=sha256:a4cfde78a9f2880208d16a93b795726a3017d5977e08d1e162a7a31322479c41 \ + --hash=sha256:a4fbdde9dd4a9ce5fd52c2b3a347bb50cc89483ef783f1cb00d408c13f7a96c0 \ + --hash=sha256:aa99adc8f081b475a12843953db36831eaf83ec33eb46a90629ca6a5de45a616 \ + --hash=sha256:ac351b3b8014eead140e77e9717e2992c6bbe30b63bc3422422eb84865412e3d \ + --hash=sha256:ad41ba96094304aa090f5a30cb6e4fb3b3f1c264c523394b4c39bbacc4dc92ba \ + --hash=sha256:b5314963fce9b0b12743891de876e724997864ee22aa496f903f426c7e2fa5b2 \ + --hash=sha256:bcf74c1df76758a395bf0af608c04c82257523f55c9868b334f06270d0f2112b \ + --hash=sha256:bd47ba7fc3ca94896759ea0109775132d3e7ab921fbf54038e1bab2e46c313c9 \ + --hash=sha256:c0323c9daef75ef2e5083624b4585018a0c9d5e3b40f607eed81a311270b934b \ + --hash=sha256:c1225416b463483160e4af85d5fc3a9690ccb53fd4b1865a6437825f5ede3209 \ + --hash=sha256:c1c948747b03be832dceed96ca815cef7360de9aa19d37c730f8e3f6101aca48 \ + --hash=sha256:c25fe15c70c59eb7c5ce8c06a1f3fa1da0ecc5ea1e7a5922c40fd2fa9b0d5046 \ + --hash=sha256:cc1b0fff8ead343dae06305f954eb8468ba0ec1a97881f42489d198e4ce3c632 \ + --hash=sha256:cd6280cf040f233bd7d3407b743b4b4c74f70e8e1c4199cb112a62c941c0772a \ + --hash=sha256:cd6c3d4b783c556fa00bf540854e42f135e2f256abd29669fcd0da0f2dec79c2 \ + --hash=sha256:d4d6fcde76f94f5cb9e43e9e9a61f16dacefd228cbbf6f1a09bd9b219a92f1a1 \ + --hash=sha256:ddf4af30b417d9fe16481e9b81c27ab2a7cde1ff7ba3e85653b02db7d145dc7b \ + --hash=sha256:df115d4d83168fdf2cae48ef1ff6d1cb4c466364e30861b37121de0f3bf1b990 \ + --hash=sha256:df7276909358e5635ae203673ab7e509ddd224225a8d6b0790bf13eb2bde1cc5 \ + --hash=sha256:e4fd89cc178bced6ad29cb3e6dd4aa63fa5017c3524dbd0b25998fb64a87cc8b \ + --hash=sha256:e9701d0049d92c16703a42771b98d560b95248949f23f8cf7b4eddd201814fb9 \ + --hash=sha256:ee2f2a527e3c1a6e6411eb4209642e138b544a2d72fe5d0d76daf77b24063534 \ + --hash=sha256:f7fb7d750cfa0a070d2c24e831fd3481019a60dd317ea2b39acbcebc08b6ed81 \ + --hash=sha256:f840ed6d8ecba8255df8c42b87fadeda98ddfc6eeec05e2dc66e26d46dd6f58a \ + --hash=sha256:f86c6358749bd4fda175388691e3ba8c46e24c5347d0afd20f9b7edfc9faf07d \ + --hash=sha256:fa36ec09ef71d158186bc79e359ff5fdd6e7996fe8ab638f00d6b93139ba4fcf \ + --hash=sha256:fe2c7201c642b7c308f1675355ad7ff7b66acfe3541625efe5a3ad38f29d6115 # via requests cryptography==46.0.7 \ --hash=sha256:04959522f938493042d595a736e7dbdff6eb6cc2339c11465b3ff89343b65f65 \ diff --git a/tools/publish/requirements_universal.txt b/tools/publish/requirements_universal.txt index 4dbcc92abc..4ab1911981 100644 --- a/tools/publish/requirements_universal.txt +++ b/tools/publish/requirements_universal.txt @@ -79,86 +79,100 @@ cffi==2.0.0 ; platform_python_implementation != 'PyPy' and sys_platform == 'linu --hash=sha256:f7f5baafcc48261359e14bcd6d9bff6d4b28d9103847c9e136694cb0501aef87 \ --hash=sha256:fc48c783f9c87e60831201f2cce7f3b2e4846bf4d8728eabe54d60700b318a0b # via cryptography -charset-normalizer==3.4.3 \ - --hash=sha256:00237675befef519d9af72169d8604a067d92755e84fe76492fef5441db05b91 \ - --hash=sha256:02425242e96bcf29a49711b0ca9f37e451da7c70562bc10e8ed992a5a7a25cc0 \ - --hash=sha256:027b776c26d38b7f15b26a5da1044f376455fb3766df8fc38563b4efbc515154 \ - --hash=sha256:07a0eae9e2787b586e129fdcbe1af6997f8d0e5abaa0bc98c0e20e124d67e601 \ - --hash=sha256:0cacf8f7297b0c4fcb74227692ca46b4a5852f8f4f24b3c766dd94a1075c4884 \ - --hash=sha256:0e78314bdc32fa80696f72fa16dc61168fda4d6a0c014e0380f9d02f0e5d8a07 \ - --hash=sha256:0f2be7e0cf7754b9a30eb01f4295cc3d4358a479843b31f328afd210e2c7598c \ - --hash=sha256:13faeacfe61784e2559e690fc53fa4c5ae97c6fcedb8eb6fb8d0a15b475d2c64 \ - --hash=sha256:14c2a87c65b351109f6abfc424cab3927b3bdece6f706e4d12faaf3d52ee5efe \ - --hash=sha256:1606f4a55c0fd363d754049cdf400175ee96c992b1f8018b993941f221221c5f \ - --hash=sha256:16a8770207946ac75703458e2c743631c79c59c5890c80011d536248f8eaa432 \ - --hash=sha256:18343b2d246dc6761a249ba1fb13f9ee9a2bcd95decc767319506056ea4ad4dc \ - --hash=sha256:18b97b8404387b96cdbd30ad660f6407799126d26a39ca65729162fd810a99aa \ - --hash=sha256:1bb60174149316da1c35fa5233681f7c0f9f514509b8e399ab70fea5f17e45c9 \ - --hash=sha256:1e8ac75d72fa3775e0b7cb7e4629cec13b7514d928d15ef8ea06bca03ef01cae \ - --hash=sha256:1ef99f0456d3d46a50945c98de1774da86f8e992ab5c77865ea8b8195341fc19 \ - --hash=sha256:2001a39612b241dae17b4687898843f254f8748b796a2e16f1051a17078d991d \ - --hash=sha256:23b6b24d74478dc833444cbd927c338349d6ae852ba53a0d02a2de1fce45b96e \ - --hash=sha256:252098c8c7a873e17dd696ed98bbe91dbacd571da4b87df3736768efa7a792e4 \ - --hash=sha256:257f26fed7d7ff59921b78244f3cd93ed2af1800ff048c33f624c87475819dd7 \ - --hash=sha256:2c322db9c8c89009a990ef07c3bcc9f011a3269bc06782f916cd3d9eed7c9312 \ - --hash=sha256:30a96e1e1f865f78b030d65241c1ee850cdf422d869e9028e2fc1d5e4db73b92 \ - --hash=sha256:30d006f98569de3459c2fc1f2acde170b7b2bd265dc1943e87e1a4efe1b67c31 \ - --hash=sha256:31a9a6f775f9bcd865d88ee350f0ffb0e25936a7f930ca98995c05abf1faf21c \ - --hash=sha256:320e8e66157cc4e247d9ddca8e21f427efc7a04bbd0ac8a9faf56583fa543f9f \ - --hash=sha256:34a7f768e3f985abdb42841e20e17b330ad3aaf4bb7e7aeeb73db2e70f077b99 \ - --hash=sha256:3653fad4fe3ed447a596ae8638b437f827234f01a8cd801842e43f3d0a6b281b \ - --hash=sha256:3cd35b7e8aedeb9e34c41385fda4f73ba609e561faedfae0a9e75e44ac558a15 \ - --hash=sha256:3cfb2aad70f2c6debfbcb717f23b7eb55febc0bb23dcffc0f076009da10c6392 \ - --hash=sha256:416175faf02e4b0810f1f38bcb54682878a4af94059a1cd63b8747244420801f \ - --hash=sha256:41d1fc408ff5fdfb910200ec0e74abc40387bccb3252f3f27c0676731df2b2c8 \ - --hash=sha256:42e5088973e56e31e4fa58eb6bd709e42fc03799c11c42929592889a2e54c491 \ - --hash=sha256:4ca4c094de7771a98d7fbd67d9e5dbf1eb73efa4f744a730437d8a3a5cf994f0 \ - --hash=sha256:511729f456829ef86ac41ca78c63a5cb55240ed23b4b737faca0eb1abb1c41bc \ - --hash=sha256:53cd68b185d98dde4ad8990e56a58dea83a4162161b1ea9272e5c9182ce415e0 \ - --hash=sha256:585f3b2a80fbd26b048a0be90c5aae8f06605d3c92615911c3a2b03a8a3b796f \ - --hash=sha256:5b413b0b1bfd94dbf4023ad6945889f374cd24e3f62de58d6bb102c4d9ae534a \ - --hash=sha256:5d8d01eac18c423815ed4f4a2ec3b439d654e55ee4ad610e153cf02faf67ea40 \ - --hash=sha256:6aab0f181c486f973bc7262a97f5aca3ee7e1437011ef0c2ec04b5a11d16c927 \ - --hash=sha256:6cf8fd4c04756b6b60146d98cd8a77d0cdae0e1ca20329da2ac85eed779b6849 \ - --hash=sha256:6fb70de56f1859a3f71261cbe41005f56a7842cc348d3aeb26237560bfa5e0ce \ - --hash=sha256:6fce4b8500244f6fcb71465d4a4930d132ba9ab8e71a7859e6a5d59851068d14 \ - --hash=sha256:70bfc5f2c318afece2f5838ea5e4c3febada0be750fcf4775641052bbba14d05 \ - --hash=sha256:73dc19b562516fc9bcf6e5d6e596df0b4eb98d87e4f79f3ae71840e6ed21361c \ - --hash=sha256:74d77e25adda8581ffc1c720f1c81ca082921329452eba58b16233ab1842141c \ - --hash=sha256:78deba4d8f9590fe4dae384aeff04082510a709957e968753ff3c48399f6f92a \ - --hash=sha256:86df271bf921c2ee3818f0522e9a5b8092ca2ad8b065ece5d7d9d0e9f4849bcc \ - --hash=sha256:88ab34806dea0671532d3f82d82b85e8fc23d7b2dd12fa837978dad9bb392a34 \ - --hash=sha256:8999f965f922ae054125286faf9f11bc6932184b93011d138925a1773830bbe9 \ - --hash=sha256:8dcfc373f888e4fb39a7bc57e93e3b845e7f462dacc008d9749568b1c4ece096 \ - --hash=sha256:939578d9d8fd4299220161fdd76e86c6a251987476f5243e8864a7844476ba14 \ - --hash=sha256:96b2b3d1a83ad55310de8c7b4a2d04d9277d5591f40761274856635acc5fcb30 \ - --hash=sha256:a2d08ac246bb48479170408d6c19f6385fa743e7157d716e144cad849b2dd94b \ - --hash=sha256:b256ee2e749283ef3ddcff51a675ff43798d92d746d1a6e4631bf8c707d22d0b \ - --hash=sha256:b5e3b2d152e74e100a9e9573837aba24aab611d39428ded46f4e4022ea7d1942 \ - --hash=sha256:b89bc04de1d83006373429975f8ef9e7932534b8cc9ca582e4db7d20d91816db \ - --hash=sha256:bd28b817ea8c70215401f657edef3a8aa83c29d447fb0b622c35403780ba11d5 \ - --hash=sha256:c60e092517a73c632ec38e290eba714e9627abe9d301c8c8a12ec32c314a2a4b \ - --hash=sha256:c6dbd0ccdda3a2ba7c2ecd9d77b37f3b5831687d8dc1b6ca5f56a4880cc7b7ce \ - --hash=sha256:c6e490913a46fa054e03699c70019ab869e990270597018cef1d8562132c2669 \ - --hash=sha256:c6f162aabe9a91a309510d74eeb6507fab5fff92337a15acbe77753d88d9dcf0 \ - --hash=sha256:c6fd51128a41297f5409deab284fecbe5305ebd7e5a1f959bee1c054622b7018 \ - --hash=sha256:cc34f233c9e71701040d772aa7490318673aa7164a0efe3172b2981218c26d93 \ - --hash=sha256:cc9370a2da1ac13f0153780040f465839e6cccb4a1e44810124b4e22483c93fe \ - --hash=sha256:ccf600859c183d70eb47e05a44cd80a4ce77394d1ac0f79dbd2dd90a69a3a049 \ - --hash=sha256:ce571ab16d890d23b5c278547ba694193a45011ff86a9162a71307ed9f86759a \ - --hash=sha256:cf1ebb7d78e1ad8ec2a8c4732c7be2e736f6e5123a4146c5b89c9d1f585f8cef \ - --hash=sha256:d0e909868420b7049dafd3a31d45125b31143eec59235311fc4c57ea26a4acd2 \ - --hash=sha256:d22dbedd33326a4a5190dd4fe9e9e693ef12160c77382d9e87919bce54f3d4ca \ - --hash=sha256:d716a916938e03231e86e43782ca7878fb602a125a91e7acb8b5112e2e96ac16 \ - --hash=sha256:d79c198e27580c8e958906f803e63cddb77653731be08851c7df0b1a14a8fc0f \ - --hash=sha256:d95bfb53c211b57198bb91c46dd5a2d8018b3af446583aab40074bf7988401cb \ - --hash=sha256:e28e334d3ff134e88989d90ba04b47d84382a828c061d0d1027b1b12a62b39b1 \ - --hash=sha256:ec557499516fc90fd374bf2e32349a2887a876fbf162c160e3c01b6849eaf557 \ - --hash=sha256:fb6fecfd65564f208cbf0fba07f107fb661bcd1a7c389edbced3f7a493f70e37 \ - --hash=sha256:fb731e5deb0c7ef82d698b0f4c5bb724633ee2a489401594c5c88b02e6cb15f7 \ - --hash=sha256:fb7f67a1bfa6e40b438170ebdc8158b78dc465a5a67b6dde178a46987b244a72 \ - --hash=sha256:fd10de089bcdcd1be95a2f73dbe6254798ec1bda9f450d5828c96f93e2536b9c \ - --hash=sha256:fdabf8315679312cfa71302f9bd509ded4f2f263fb5b765cf1433b39106c3cc9 +charset-normalizer==3.4.9 \ + --hash=sha256:0327fcd59a935777d83410750c50600ee9571af2846f71ce40f25b13da1ef380 \ + --hash=sha256:03d07803992c6c7bbc976327f34b18b6160327fc81cb82c9d504720ac0be3b62 \ + --hash=sha256:04ce310cb89c15df659582aee80a0603788732a5e017d5bd5c81158106ce249c \ + --hash=sha256:0d861473f743244d349b50f850d10eb87aeb22bbdcc8e64f79273c94af5a8226 \ + --hash=sha256:0e94703ec9684807f20cfb5eed95c70f67f2a8f21ad620146d7b5a13677b93e5 \ + --hash=sha256:0fa1aec2d32bcc03c8fa0f6f1712caad1adc38509f31142112e5c9daf5b9c833 \ + --hash=sha256:16b65ea0f2465b6fb52aa22de5eca612aa964ddfec00a912e26f4656cbef890b \ + --hash=sha256:16d10d789dd9bcca1173c95af82c58433122564b7bc39385124be735a35cbe99 \ + --hash=sha256:19ac87f93086ce37b86e098888555c4b4bc48102279bae3350098c0ed664b501 \ + --hash=sha256:1d22856ffbe153a602df38e4a5464f0b748a54002e0d69ac6d2ad0a197cc99ec \ + --hash=sha256:21e764fd1e70b6a3e205a0e46f3051701f98a8cb3fad66eeb80e48bb502f8698 \ + --hash=sha256:231ddcbb35e2ff8973e1365db41fe0572662893b99a05deb183b68ad4c0c8bd4 \ + --hash=sha256:253a4a220747e8b5faf57ec320c4f5efb0cef05f647420bf267143ec15dba10a \ + --hash=sha256:280081916dc341820640489a66e4696049401ef1cf6dd672f672e70ad915aca3 \ + --hash=sha256:2a441ea71902098ffe78c5abe6c494f44160b4af614ed16c3d9a3b1d17fd8ee2 \ + --hash=sha256:304b13570067b2547562e308af560b3963857b1fa90bd6afd978130130fe2d6a \ + --hash=sha256:32286a2c8d167e897177b673176c1e3e00d4057caf5d2b64eef9a3666b03018e \ + --hash=sha256:33bdcc2a32c0a0e861f60841a512c8acc658c87c2ac59d89e3a46dacf7d866e4 \ + --hash=sha256:375b83ed0aecfce76c16d198fbc21f3b11b337d68662bea0a995046682a11419 \ + --hash=sha256:3c09a49d6cde137258beb3d551994a2927fd35ad5cf96aed573f61bbd67c5f84 \ + --hash=sha256:3d92613ec25e43b05f042302531ec0f00b8445190e43325880cbd6ab7c2581da \ + --hash=sha256:40a126142a56b2dfc0aacbad1de8310cbf60da7656db0e6b16eebd48e3e93519 \ + --hash=sha256:416c229f77e5ea25b3dfd4b582f8d73d7e43c22320302b9ab128a2d3a0b38efe \ + --hash=sha256:432786d3561e69aeeae6c7e8648964ce0ad05736120135601f87ac26b9c83381 \ + --hash=sha256:43b9e366a31fdd1c87d0eb08f579b4a82b723ea54338f040d6b4e518a026ea29 \ + --hash=sha256:440eede837960000d74978f0eba527be106b5b9aee0daf779d395276ed0b0614 \ + --hash=sha256:45b0cc4e3556cd875e09102988d1ab8356c998b596c9fced84547c8138b487a0 \ + --hash=sha256:476743fe6dfe14a2da12e3ac79125dc84a3b2cf8094369a47a1529b0cd8549fe \ + --hash=sha256:4773092f8019072343a7447203308b176e10199920eb02d6195e81bbb3274c29 \ + --hash=sha256:4b3dac63058cc36820b0dd072f89898604e2d39686fe05321729d00d8ac185a0 \ + --hash=sha256:4d1c96a7a18b9690a4d46df09e3e3382406ae3213727cd1019ebade1c4a81917 \ + --hash=sha256:51307f5c71007673a2bf8232ad973483d281e74cb99c8c5a990af1eefa6277d9 \ + --hash=sha256:51447e9aa2684679af07ca5021c3db526e0284347ebf4ffcec1154c3350cfe32 \ + --hash=sha256:58150c9f9b9a552505912d182ccdf26f6396fb6094816ceebcbb20eecabaed94 \ + --hash=sha256:5b10cd92fc5c498b35a8635df6d5a100207f88b63a4dc1de7ef9a548e1e2cd63 \ + --hash=sha256:5e226f6218febc71f6c1fc2fafb91c226f75bdc1d8fb12d66823716e891608fd \ + --hash=sha256:609b3ba8fcc0fb5ab7af00719d0fb6ad0cb518e48e7712d12fd68f1327951198 \ + --hash=sha256:60f44ade2cf573dad7a277e6f8ca9a51a21dda572b13bd7d8539bb3cd5dbedde \ + --hash=sha256:611057cc5d5c0afc743ba8be6bd828c17e0aaa8643f9d0a9b9bb7dea80eb8012 \ + --hash=sha256:6366a16e1a25018694d6a5d784d09b046edc9eac40ea2b54065c3052672516a1 \ + --hash=sha256:65a7ff3f705e57d392f7261b6d0550fe137c3019477431f1c355e0db0a7d3e15 \ + --hash=sha256:673611bbd43f0810bec0b0f028ddeaaa501190339cac411f347ac76917c3ae7b \ + --hash=sha256:67830fc78e67501f47bb950471b2dcb9b35b140084429318e862895a8e89c993 \ + --hash=sha256:68ce9f4d6b26d5ccbf7fd4459bf75f74a0a146677ebba80597df60cbdb20e6f4 \ + --hash=sha256:68e5f26a1ad57ded6d1cfb85331d1c1a195314756471d97758c48498bb4dcdf5 \ + --hash=sha256:69b157c5d3292bcd443faca052f3096f637f1e074b98212a933c074ae23dc3b8 \ + --hash=sha256:75286256590a6320cf106a0d28970d3560aad9ee09aa7b34fb40524792436d35 \ + --hash=sha256:78841cccf1af7b40f6f716338d50c0902dbe88d9f800b3c973b7a9a0a693a642 \ + --hash=sha256:78fa18e436a1a0e58dbd7e02fc4473f3f32cceb12df9dfca542d075961c307d2 \ + --hash=sha256:79580094b00d1789d1f93ea55bc43cb2f611910c72235b7657f3482ddcc1b22d \ + --hash=sha256:7b86a2b16095d250c6f58b3d9b2eee6f4147754344f3dab0922f7c9bf7d226c9 \ + --hash=sha256:83aed2c10721ddd90f68140685391b50811a880af20654c59af6b6c66c40513c \ + --hash=sha256:84fd18bcc17526fc2b3c1af7d2b9217d32c9c04448c16ec693b9b4f1985c3d33 \ + --hash=sha256:871ff67ea1aad4dfd91736464934d56b32dac49f9fbe16cddba36198a7b3a0db \ + --hash=sha256:898f0e9068ca27d37f8e83a5b962821df851532e6c4a7d615c1c033f9da6eedf \ + --hash=sha256:8a79d9f4d8001473a30c163556b3c3bfebec837495a412dde78b51672f6134f9 \ + --hash=sha256:8c041122946b7ba21bb32c45b1aa57b1be35527690aeb3c5c234521085632eee \ + --hash=sha256:90c44bc373b7687f6948b693cceaea1348ae0975d7474746559494468e3c1d84 \ + --hash=sha256:9104ed0bd76a429d46f9ec0dbc9b08ad1d2dcdf2b00a5a0daa1c145329b35b44 \ + --hash=sha256:920079c3f7456fa213e0829ed2073aaa727fd39d889ead5b4f35d0de5460d04f \ + --hash=sha256:93d59d504b230e83c7a843251681959a0b6a9cd76f6e146ce1b8a80eb8739af9 \ + --hash=sha256:9b2aff1c7b3884512b9512c3eaadd9bab39fb45042ffaaa1dd08ff2b9f8109d9 \ + --hash=sha256:9b8e0f3107e2200b76f6054de99016eac3ee6762713587b36baaa7e4bd2ae177 \ + --hash=sha256:9bb41182d93ea91f60b4bc8fbf4c820c69ef8a12ab2d917f3f1834f1acad07e8 \ + --hash=sha256:9cdef90ae47919cae358d8ab15797a800ed41da7aba5d72419fb510729e2ed4b \ + --hash=sha256:a1786910334ed46ab1dd73222f2cd1e05c2c3bb39f6dddb4f8b36fc382058a39 \ + --hash=sha256:a4cfde78a9f2880208d16a93b795726a3017d5977e08d1e162a7a31322479c41 \ + --hash=sha256:a4fbdde9dd4a9ce5fd52c2b3a347bb50cc89483ef783f1cb00d408c13f7a96c0 \ + --hash=sha256:aa99adc8f081b475a12843953db36831eaf83ec33eb46a90629ca6a5de45a616 \ + --hash=sha256:ac351b3b8014eead140e77e9717e2992c6bbe30b63bc3422422eb84865412e3d \ + --hash=sha256:ad41ba96094304aa090f5a30cb6e4fb3b3f1c264c523394b4c39bbacc4dc92ba \ + --hash=sha256:b5314963fce9b0b12743891de876e724997864ee22aa496f903f426c7e2fa5b2 \ + --hash=sha256:bcf74c1df76758a395bf0af608c04c82257523f55c9868b334f06270d0f2112b \ + --hash=sha256:bd47ba7fc3ca94896759ea0109775132d3e7ab921fbf54038e1bab2e46c313c9 \ + --hash=sha256:c0323c9daef75ef2e5083624b4585018a0c9d5e3b40f607eed81a311270b934b \ + --hash=sha256:c1225416b463483160e4af85d5fc3a9690ccb53fd4b1865a6437825f5ede3209 \ + --hash=sha256:c1c948747b03be832dceed96ca815cef7360de9aa19d37c730f8e3f6101aca48 \ + --hash=sha256:c25fe15c70c59eb7c5ce8c06a1f3fa1da0ecc5ea1e7a5922c40fd2fa9b0d5046 \ + --hash=sha256:cc1b0fff8ead343dae06305f954eb8468ba0ec1a97881f42489d198e4ce3c632 \ + --hash=sha256:cd6280cf040f233bd7d3407b743b4b4c74f70e8e1c4199cb112a62c941c0772a \ + --hash=sha256:cd6c3d4b783c556fa00bf540854e42f135e2f256abd29669fcd0da0f2dec79c2 \ + --hash=sha256:d4d6fcde76f94f5cb9e43e9e9a61f16dacefd228cbbf6f1a09bd9b219a92f1a1 \ + --hash=sha256:ddf4af30b417d9fe16481e9b81c27ab2a7cde1ff7ba3e85653b02db7d145dc7b \ + --hash=sha256:df115d4d83168fdf2cae48ef1ff6d1cb4c466364e30861b37121de0f3bf1b990 \ + --hash=sha256:df7276909358e5635ae203673ab7e509ddd224225a8d6b0790bf13eb2bde1cc5 \ + --hash=sha256:e4fd89cc178bced6ad29cb3e6dd4aa63fa5017c3524dbd0b25998fb64a87cc8b \ + --hash=sha256:e9701d0049d92c16703a42771b98d560b95248949f23f8cf7b4eddd201814fb9 \ + --hash=sha256:ee2f2a527e3c1a6e6411eb4209642e138b544a2d72fe5d0d76daf77b24063534 \ + --hash=sha256:f7fb7d750cfa0a070d2c24e831fd3481019a60dd317ea2b39acbcebc08b6ed81 \ + --hash=sha256:f840ed6d8ecba8255df8c42b87fadeda98ddfc6eeec05e2dc66e26d46dd6f58a \ + --hash=sha256:f86c6358749bd4fda175388691e3ba8c46e24c5347d0afd20f9b7edfc9faf07d \ + --hash=sha256:fa36ec09ef71d158186bc79e359ff5fdd6e7996fe8ab638f00d6b93139ba4fcf \ + --hash=sha256:fe2c7201c642b7c308f1675355ad7ff7b66acfe3541625efe5a3ad38f29d6115 # via requests cryptography==46.0.7 ; sys_platform == 'linux' \ --hash=sha256:04959522f938493042d595a736e7dbdff6eb6cc2339c11465b3ff89343b65f65 \ diff --git a/tools/publish/requirements_windows.txt b/tools/publish/requirements_windows.txt index 029c6bef4d..1cbd8197fe 100644 --- a/tools/publish/requirements_windows.txt +++ b/tools/publish/requirements_windows.txt @@ -10,86 +10,100 @@ certifi==2025.10.5 \ --hash=sha256:0f212c2744a9bb6de0c56639a6f68afe01ecd92d91f14ae897c4fe7bbeeef0de \ --hash=sha256:47c09d31ccf2acf0be3f701ea53595ee7e0b8fa08801c6624be771df09ae7b43 # via requests -charset-normalizer==3.4.3 \ - --hash=sha256:00237675befef519d9af72169d8604a067d92755e84fe76492fef5441db05b91 \ - --hash=sha256:02425242e96bcf29a49711b0ca9f37e451da7c70562bc10e8ed992a5a7a25cc0 \ - --hash=sha256:027b776c26d38b7f15b26a5da1044f376455fb3766df8fc38563b4efbc515154 \ - --hash=sha256:07a0eae9e2787b586e129fdcbe1af6997f8d0e5abaa0bc98c0e20e124d67e601 \ - --hash=sha256:0cacf8f7297b0c4fcb74227692ca46b4a5852f8f4f24b3c766dd94a1075c4884 \ - --hash=sha256:0e78314bdc32fa80696f72fa16dc61168fda4d6a0c014e0380f9d02f0e5d8a07 \ - --hash=sha256:0f2be7e0cf7754b9a30eb01f4295cc3d4358a479843b31f328afd210e2c7598c \ - --hash=sha256:13faeacfe61784e2559e690fc53fa4c5ae97c6fcedb8eb6fb8d0a15b475d2c64 \ - --hash=sha256:14c2a87c65b351109f6abfc424cab3927b3bdece6f706e4d12faaf3d52ee5efe \ - --hash=sha256:1606f4a55c0fd363d754049cdf400175ee96c992b1f8018b993941f221221c5f \ - --hash=sha256:16a8770207946ac75703458e2c743631c79c59c5890c80011d536248f8eaa432 \ - --hash=sha256:18343b2d246dc6761a249ba1fb13f9ee9a2bcd95decc767319506056ea4ad4dc \ - --hash=sha256:18b97b8404387b96cdbd30ad660f6407799126d26a39ca65729162fd810a99aa \ - --hash=sha256:1bb60174149316da1c35fa5233681f7c0f9f514509b8e399ab70fea5f17e45c9 \ - --hash=sha256:1e8ac75d72fa3775e0b7cb7e4629cec13b7514d928d15ef8ea06bca03ef01cae \ - --hash=sha256:1ef99f0456d3d46a50945c98de1774da86f8e992ab5c77865ea8b8195341fc19 \ - --hash=sha256:2001a39612b241dae17b4687898843f254f8748b796a2e16f1051a17078d991d \ - --hash=sha256:23b6b24d74478dc833444cbd927c338349d6ae852ba53a0d02a2de1fce45b96e \ - --hash=sha256:252098c8c7a873e17dd696ed98bbe91dbacd571da4b87df3736768efa7a792e4 \ - --hash=sha256:257f26fed7d7ff59921b78244f3cd93ed2af1800ff048c33f624c87475819dd7 \ - --hash=sha256:2c322db9c8c89009a990ef07c3bcc9f011a3269bc06782f916cd3d9eed7c9312 \ - --hash=sha256:30a96e1e1f865f78b030d65241c1ee850cdf422d869e9028e2fc1d5e4db73b92 \ - --hash=sha256:30d006f98569de3459c2fc1f2acde170b7b2bd265dc1943e87e1a4efe1b67c31 \ - --hash=sha256:31a9a6f775f9bcd865d88ee350f0ffb0e25936a7f930ca98995c05abf1faf21c \ - --hash=sha256:320e8e66157cc4e247d9ddca8e21f427efc7a04bbd0ac8a9faf56583fa543f9f \ - --hash=sha256:34a7f768e3f985abdb42841e20e17b330ad3aaf4bb7e7aeeb73db2e70f077b99 \ - --hash=sha256:3653fad4fe3ed447a596ae8638b437f827234f01a8cd801842e43f3d0a6b281b \ - --hash=sha256:3cd35b7e8aedeb9e34c41385fda4f73ba609e561faedfae0a9e75e44ac558a15 \ - --hash=sha256:3cfb2aad70f2c6debfbcb717f23b7eb55febc0bb23dcffc0f076009da10c6392 \ - --hash=sha256:416175faf02e4b0810f1f38bcb54682878a4af94059a1cd63b8747244420801f \ - --hash=sha256:41d1fc408ff5fdfb910200ec0e74abc40387bccb3252f3f27c0676731df2b2c8 \ - --hash=sha256:42e5088973e56e31e4fa58eb6bd709e42fc03799c11c42929592889a2e54c491 \ - --hash=sha256:4ca4c094de7771a98d7fbd67d9e5dbf1eb73efa4f744a730437d8a3a5cf994f0 \ - --hash=sha256:511729f456829ef86ac41ca78c63a5cb55240ed23b4b737faca0eb1abb1c41bc \ - --hash=sha256:53cd68b185d98dde4ad8990e56a58dea83a4162161b1ea9272e5c9182ce415e0 \ - --hash=sha256:585f3b2a80fbd26b048a0be90c5aae8f06605d3c92615911c3a2b03a8a3b796f \ - --hash=sha256:5b413b0b1bfd94dbf4023ad6945889f374cd24e3f62de58d6bb102c4d9ae534a \ - --hash=sha256:5d8d01eac18c423815ed4f4a2ec3b439d654e55ee4ad610e153cf02faf67ea40 \ - --hash=sha256:6aab0f181c486f973bc7262a97f5aca3ee7e1437011ef0c2ec04b5a11d16c927 \ - --hash=sha256:6cf8fd4c04756b6b60146d98cd8a77d0cdae0e1ca20329da2ac85eed779b6849 \ - --hash=sha256:6fb70de56f1859a3f71261cbe41005f56a7842cc348d3aeb26237560bfa5e0ce \ - --hash=sha256:6fce4b8500244f6fcb71465d4a4930d132ba9ab8e71a7859e6a5d59851068d14 \ - --hash=sha256:70bfc5f2c318afece2f5838ea5e4c3febada0be750fcf4775641052bbba14d05 \ - --hash=sha256:73dc19b562516fc9bcf6e5d6e596df0b4eb98d87e4f79f3ae71840e6ed21361c \ - --hash=sha256:74d77e25adda8581ffc1c720f1c81ca082921329452eba58b16233ab1842141c \ - --hash=sha256:78deba4d8f9590fe4dae384aeff04082510a709957e968753ff3c48399f6f92a \ - --hash=sha256:86df271bf921c2ee3818f0522e9a5b8092ca2ad8b065ece5d7d9d0e9f4849bcc \ - --hash=sha256:88ab34806dea0671532d3f82d82b85e8fc23d7b2dd12fa837978dad9bb392a34 \ - --hash=sha256:8999f965f922ae054125286faf9f11bc6932184b93011d138925a1773830bbe9 \ - --hash=sha256:8dcfc373f888e4fb39a7bc57e93e3b845e7f462dacc008d9749568b1c4ece096 \ - --hash=sha256:939578d9d8fd4299220161fdd76e86c6a251987476f5243e8864a7844476ba14 \ - --hash=sha256:96b2b3d1a83ad55310de8c7b4a2d04d9277d5591f40761274856635acc5fcb30 \ - --hash=sha256:a2d08ac246bb48479170408d6c19f6385fa743e7157d716e144cad849b2dd94b \ - --hash=sha256:b256ee2e749283ef3ddcff51a675ff43798d92d746d1a6e4631bf8c707d22d0b \ - --hash=sha256:b5e3b2d152e74e100a9e9573837aba24aab611d39428ded46f4e4022ea7d1942 \ - --hash=sha256:b89bc04de1d83006373429975f8ef9e7932534b8cc9ca582e4db7d20d91816db \ - --hash=sha256:bd28b817ea8c70215401f657edef3a8aa83c29d447fb0b622c35403780ba11d5 \ - --hash=sha256:c60e092517a73c632ec38e290eba714e9627abe9d301c8c8a12ec32c314a2a4b \ - --hash=sha256:c6dbd0ccdda3a2ba7c2ecd9d77b37f3b5831687d8dc1b6ca5f56a4880cc7b7ce \ - --hash=sha256:c6e490913a46fa054e03699c70019ab869e990270597018cef1d8562132c2669 \ - --hash=sha256:c6f162aabe9a91a309510d74eeb6507fab5fff92337a15acbe77753d88d9dcf0 \ - --hash=sha256:c6fd51128a41297f5409deab284fecbe5305ebd7e5a1f959bee1c054622b7018 \ - --hash=sha256:cc34f233c9e71701040d772aa7490318673aa7164a0efe3172b2981218c26d93 \ - --hash=sha256:cc9370a2da1ac13f0153780040f465839e6cccb4a1e44810124b4e22483c93fe \ - --hash=sha256:ccf600859c183d70eb47e05a44cd80a4ce77394d1ac0f79dbd2dd90a69a3a049 \ - --hash=sha256:ce571ab16d890d23b5c278547ba694193a45011ff86a9162a71307ed9f86759a \ - --hash=sha256:cf1ebb7d78e1ad8ec2a8c4732c7be2e736f6e5123a4146c5b89c9d1f585f8cef \ - --hash=sha256:d0e909868420b7049dafd3a31d45125b31143eec59235311fc4c57ea26a4acd2 \ - --hash=sha256:d22dbedd33326a4a5190dd4fe9e9e693ef12160c77382d9e87919bce54f3d4ca \ - --hash=sha256:d716a916938e03231e86e43782ca7878fb602a125a91e7acb8b5112e2e96ac16 \ - --hash=sha256:d79c198e27580c8e958906f803e63cddb77653731be08851c7df0b1a14a8fc0f \ - --hash=sha256:d95bfb53c211b57198bb91c46dd5a2d8018b3af446583aab40074bf7988401cb \ - --hash=sha256:e28e334d3ff134e88989d90ba04b47d84382a828c061d0d1027b1b12a62b39b1 \ - --hash=sha256:ec557499516fc90fd374bf2e32349a2887a876fbf162c160e3c01b6849eaf557 \ - --hash=sha256:fb6fecfd65564f208cbf0fba07f107fb661bcd1a7c389edbced3f7a493f70e37 \ - --hash=sha256:fb731e5deb0c7ef82d698b0f4c5bb724633ee2a489401594c5c88b02e6cb15f7 \ - --hash=sha256:fb7f67a1bfa6e40b438170ebdc8158b78dc465a5a67b6dde178a46987b244a72 \ - --hash=sha256:fd10de089bcdcd1be95a2f73dbe6254798ec1bda9f450d5828c96f93e2536b9c \ - --hash=sha256:fdabf8315679312cfa71302f9bd509ded4f2f263fb5b765cf1433b39106c3cc9 +charset-normalizer==3.4.9 \ + --hash=sha256:0327fcd59a935777d83410750c50600ee9571af2846f71ce40f25b13da1ef380 \ + --hash=sha256:03d07803992c6c7bbc976327f34b18b6160327fc81cb82c9d504720ac0be3b62 \ + --hash=sha256:04ce310cb89c15df659582aee80a0603788732a5e017d5bd5c81158106ce249c \ + --hash=sha256:0d861473f743244d349b50f850d10eb87aeb22bbdcc8e64f79273c94af5a8226 \ + --hash=sha256:0e94703ec9684807f20cfb5eed95c70f67f2a8f21ad620146d7b5a13677b93e5 \ + --hash=sha256:0fa1aec2d32bcc03c8fa0f6f1712caad1adc38509f31142112e5c9daf5b9c833 \ + --hash=sha256:16b65ea0f2465b6fb52aa22de5eca612aa964ddfec00a912e26f4656cbef890b \ + --hash=sha256:16d10d789dd9bcca1173c95af82c58433122564b7bc39385124be735a35cbe99 \ + --hash=sha256:19ac87f93086ce37b86e098888555c4b4bc48102279bae3350098c0ed664b501 \ + --hash=sha256:1d22856ffbe153a602df38e4a5464f0b748a54002e0d69ac6d2ad0a197cc99ec \ + --hash=sha256:21e764fd1e70b6a3e205a0e46f3051701f98a8cb3fad66eeb80e48bb502f8698 \ + --hash=sha256:231ddcbb35e2ff8973e1365db41fe0572662893b99a05deb183b68ad4c0c8bd4 \ + --hash=sha256:253a4a220747e8b5faf57ec320c4f5efb0cef05f647420bf267143ec15dba10a \ + --hash=sha256:280081916dc341820640489a66e4696049401ef1cf6dd672f672e70ad915aca3 \ + --hash=sha256:2a441ea71902098ffe78c5abe6c494f44160b4af614ed16c3d9a3b1d17fd8ee2 \ + --hash=sha256:304b13570067b2547562e308af560b3963857b1fa90bd6afd978130130fe2d6a \ + --hash=sha256:32286a2c8d167e897177b673176c1e3e00d4057caf5d2b64eef9a3666b03018e \ + --hash=sha256:33bdcc2a32c0a0e861f60841a512c8acc658c87c2ac59d89e3a46dacf7d866e4 \ + --hash=sha256:375b83ed0aecfce76c16d198fbc21f3b11b337d68662bea0a995046682a11419 \ + --hash=sha256:3c09a49d6cde137258beb3d551994a2927fd35ad5cf96aed573f61bbd67c5f84 \ + --hash=sha256:3d92613ec25e43b05f042302531ec0f00b8445190e43325880cbd6ab7c2581da \ + --hash=sha256:40a126142a56b2dfc0aacbad1de8310cbf60da7656db0e6b16eebd48e3e93519 \ + --hash=sha256:416c229f77e5ea25b3dfd4b582f8d73d7e43c22320302b9ab128a2d3a0b38efe \ + --hash=sha256:432786d3561e69aeeae6c7e8648964ce0ad05736120135601f87ac26b9c83381 \ + --hash=sha256:43b9e366a31fdd1c87d0eb08f579b4a82b723ea54338f040d6b4e518a026ea29 \ + --hash=sha256:440eede837960000d74978f0eba527be106b5b9aee0daf779d395276ed0b0614 \ + --hash=sha256:45b0cc4e3556cd875e09102988d1ab8356c998b596c9fced84547c8138b487a0 \ + --hash=sha256:476743fe6dfe14a2da12e3ac79125dc84a3b2cf8094369a47a1529b0cd8549fe \ + --hash=sha256:4773092f8019072343a7447203308b176e10199920eb02d6195e81bbb3274c29 \ + --hash=sha256:4b3dac63058cc36820b0dd072f89898604e2d39686fe05321729d00d8ac185a0 \ + --hash=sha256:4d1c96a7a18b9690a4d46df09e3e3382406ae3213727cd1019ebade1c4a81917 \ + --hash=sha256:51307f5c71007673a2bf8232ad973483d281e74cb99c8c5a990af1eefa6277d9 \ + --hash=sha256:51447e9aa2684679af07ca5021c3db526e0284347ebf4ffcec1154c3350cfe32 \ + --hash=sha256:58150c9f9b9a552505912d182ccdf26f6396fb6094816ceebcbb20eecabaed94 \ + --hash=sha256:5b10cd92fc5c498b35a8635df6d5a100207f88b63a4dc1de7ef9a548e1e2cd63 \ + --hash=sha256:5e226f6218febc71f6c1fc2fafb91c226f75bdc1d8fb12d66823716e891608fd \ + --hash=sha256:609b3ba8fcc0fb5ab7af00719d0fb6ad0cb518e48e7712d12fd68f1327951198 \ + --hash=sha256:60f44ade2cf573dad7a277e6f8ca9a51a21dda572b13bd7d8539bb3cd5dbedde \ + --hash=sha256:611057cc5d5c0afc743ba8be6bd828c17e0aaa8643f9d0a9b9bb7dea80eb8012 \ + --hash=sha256:6366a16e1a25018694d6a5d784d09b046edc9eac40ea2b54065c3052672516a1 \ + --hash=sha256:65a7ff3f705e57d392f7261b6d0550fe137c3019477431f1c355e0db0a7d3e15 \ + --hash=sha256:673611bbd43f0810bec0b0f028ddeaaa501190339cac411f347ac76917c3ae7b \ + --hash=sha256:67830fc78e67501f47bb950471b2dcb9b35b140084429318e862895a8e89c993 \ + --hash=sha256:68ce9f4d6b26d5ccbf7fd4459bf75f74a0a146677ebba80597df60cbdb20e6f4 \ + --hash=sha256:68e5f26a1ad57ded6d1cfb85331d1c1a195314756471d97758c48498bb4dcdf5 \ + --hash=sha256:69b157c5d3292bcd443faca052f3096f637f1e074b98212a933c074ae23dc3b8 \ + --hash=sha256:75286256590a6320cf106a0d28970d3560aad9ee09aa7b34fb40524792436d35 \ + --hash=sha256:78841cccf1af7b40f6f716338d50c0902dbe88d9f800b3c973b7a9a0a693a642 \ + --hash=sha256:78fa18e436a1a0e58dbd7e02fc4473f3f32cceb12df9dfca542d075961c307d2 \ + --hash=sha256:79580094b00d1789d1f93ea55bc43cb2f611910c72235b7657f3482ddcc1b22d \ + --hash=sha256:7b86a2b16095d250c6f58b3d9b2eee6f4147754344f3dab0922f7c9bf7d226c9 \ + --hash=sha256:83aed2c10721ddd90f68140685391b50811a880af20654c59af6b6c66c40513c \ + --hash=sha256:84fd18bcc17526fc2b3c1af7d2b9217d32c9c04448c16ec693b9b4f1985c3d33 \ + --hash=sha256:871ff67ea1aad4dfd91736464934d56b32dac49f9fbe16cddba36198a7b3a0db \ + --hash=sha256:898f0e9068ca27d37f8e83a5b962821df851532e6c4a7d615c1c033f9da6eedf \ + --hash=sha256:8a79d9f4d8001473a30c163556b3c3bfebec837495a412dde78b51672f6134f9 \ + --hash=sha256:8c041122946b7ba21bb32c45b1aa57b1be35527690aeb3c5c234521085632eee \ + --hash=sha256:90c44bc373b7687f6948b693cceaea1348ae0975d7474746559494468e3c1d84 \ + --hash=sha256:9104ed0bd76a429d46f9ec0dbc9b08ad1d2dcdf2b00a5a0daa1c145329b35b44 \ + --hash=sha256:920079c3f7456fa213e0829ed2073aaa727fd39d889ead5b4f35d0de5460d04f \ + --hash=sha256:93d59d504b230e83c7a843251681959a0b6a9cd76f6e146ce1b8a80eb8739af9 \ + --hash=sha256:9b2aff1c7b3884512b9512c3eaadd9bab39fb45042ffaaa1dd08ff2b9f8109d9 \ + --hash=sha256:9b8e0f3107e2200b76f6054de99016eac3ee6762713587b36baaa7e4bd2ae177 \ + --hash=sha256:9bb41182d93ea91f60b4bc8fbf4c820c69ef8a12ab2d917f3f1834f1acad07e8 \ + --hash=sha256:9cdef90ae47919cae358d8ab15797a800ed41da7aba5d72419fb510729e2ed4b \ + --hash=sha256:a1786910334ed46ab1dd73222f2cd1e05c2c3bb39f6dddb4f8b36fc382058a39 \ + --hash=sha256:a4cfde78a9f2880208d16a93b795726a3017d5977e08d1e162a7a31322479c41 \ + --hash=sha256:a4fbdde9dd4a9ce5fd52c2b3a347bb50cc89483ef783f1cb00d408c13f7a96c0 \ + --hash=sha256:aa99adc8f081b475a12843953db36831eaf83ec33eb46a90629ca6a5de45a616 \ + --hash=sha256:ac351b3b8014eead140e77e9717e2992c6bbe30b63bc3422422eb84865412e3d \ + --hash=sha256:ad41ba96094304aa090f5a30cb6e4fb3b3f1c264c523394b4c39bbacc4dc92ba \ + --hash=sha256:b5314963fce9b0b12743891de876e724997864ee22aa496f903f426c7e2fa5b2 \ + --hash=sha256:bcf74c1df76758a395bf0af608c04c82257523f55c9868b334f06270d0f2112b \ + --hash=sha256:bd47ba7fc3ca94896759ea0109775132d3e7ab921fbf54038e1bab2e46c313c9 \ + --hash=sha256:c0323c9daef75ef2e5083624b4585018a0c9d5e3b40f607eed81a311270b934b \ + --hash=sha256:c1225416b463483160e4af85d5fc3a9690ccb53fd4b1865a6437825f5ede3209 \ + --hash=sha256:c1c948747b03be832dceed96ca815cef7360de9aa19d37c730f8e3f6101aca48 \ + --hash=sha256:c25fe15c70c59eb7c5ce8c06a1f3fa1da0ecc5ea1e7a5922c40fd2fa9b0d5046 \ + --hash=sha256:cc1b0fff8ead343dae06305f954eb8468ba0ec1a97881f42489d198e4ce3c632 \ + --hash=sha256:cd6280cf040f233bd7d3407b743b4b4c74f70e8e1c4199cb112a62c941c0772a \ + --hash=sha256:cd6c3d4b783c556fa00bf540854e42f135e2f256abd29669fcd0da0f2dec79c2 \ + --hash=sha256:d4d6fcde76f94f5cb9e43e9e9a61f16dacefd228cbbf6f1a09bd9b219a92f1a1 \ + --hash=sha256:ddf4af30b417d9fe16481e9b81c27ab2a7cde1ff7ba3e85653b02db7d145dc7b \ + --hash=sha256:df115d4d83168fdf2cae48ef1ff6d1cb4c466364e30861b37121de0f3bf1b990 \ + --hash=sha256:df7276909358e5635ae203673ab7e509ddd224225a8d6b0790bf13eb2bde1cc5 \ + --hash=sha256:e4fd89cc178bced6ad29cb3e6dd4aa63fa5017c3524dbd0b25998fb64a87cc8b \ + --hash=sha256:e9701d0049d92c16703a42771b98d560b95248949f23f8cf7b4eddd201814fb9 \ + --hash=sha256:ee2f2a527e3c1a6e6411eb4209642e138b544a2d72fe5d0d76daf77b24063534 \ + --hash=sha256:f7fb7d750cfa0a070d2c24e831fd3481019a60dd317ea2b39acbcebc08b6ed81 \ + --hash=sha256:f840ed6d8ecba8255df8c42b87fadeda98ddfc6eeec05e2dc66e26d46dd6f58a \ + --hash=sha256:f86c6358749bd4fda175388691e3ba8c46e24c5347d0afd20f9b7edfc9faf07d \ + --hash=sha256:fa36ec09ef71d158186bc79e359ff5fdd6e7996fe8ab638f00d6b93139ba4fcf \ + --hash=sha256:fe2c7201c642b7c308f1675355ad7ff7b66acfe3541625efe5a3ad38f29d6115 # via requests docutils==0.22.2 \ --hash=sha256:9fdb771707c8784c8f2728b67cb2c691305933d68137ef95a75db5f4dfbc213d \ From 989776774f1ddcd200017ac455e79c0e8b996c85 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Sun, 26 Jul 2026 21:31:16 -0700 Subject: [PATCH 860/922] fix(pip.parse): allow uv_lock to be set without requirements_lock (#3952) Currently, calling `pip.parse(uv_lock = ...)` without setting `requirements_lock` causes repository generation to fail during platform lockfile validation because mandatory requirement files are enforced. This prevents using `uv_lock` as a standalone lockfile in `pip.parse`. To fix, accept `uv_lock` in platform requirement validation and exempt callers from mandatory requirement file checks when a `uv_lock` is present. Related #1975 Related #2787 --- news/3952.fixed.md | 3 +++ python/private/pypi/hub_builder.bzl | 1 + .../pypi/requirements_files_by_platform.bzl | 16 ++++++++---- tests/pypi/hub_builder/hub_builder_tests.bzl | 25 +++++++++++++++++++ .../requirements_files_by_platform_tests.bzl | 21 +++++++++++++++- 5 files changed, 60 insertions(+), 6 deletions(-) create mode 100644 news/3952.fixed.md diff --git a/news/3952.fixed.md b/news/3952.fixed.md new file mode 100644 index 0000000000..70558dd986 --- /dev/null +++ b/news/3952.fixed.md @@ -0,0 +1,3 @@ +(pypi) Allow `uv_lock` to be specified in `pip.parse` without requiring +`requirements_lock` (or other os-specific requirement file attributes) to be +set. diff --git a/python/private/pypi/hub_builder.bzl b/python/private/pypi/hub_builder.bzl index 01dc8494d6..d7974d39ad 100644 --- a/python/private/pypi/hub_builder.bzl +++ b/python/private/pypi/hub_builder.bzl @@ -498,6 +498,7 @@ def _create_whl_repos( requirements_lock = pip_attr.requirements_lock, requirements_osx = pip_attr.requirements_darwin, requirements_windows = pip_attr.requirements_windows, + uv_lock = pip_attr.uv_lock, extra_pip_args = pip_attr.extra_pip_args, platforms = sorted(platforms), # here we only need keys python_version = full_version( diff --git a/python/private/pypi/requirements_files_by_platform.bzl b/python/private/pypi/requirements_files_by_platform.bzl index dcdb6128a7..c6f53ae611 100644 --- a/python/private/pypi/requirements_files_by_platform.bzl +++ b/python/private/pypi/requirements_files_by_platform.bzl @@ -73,6 +73,7 @@ def requirements_files_by_platform( requirements_linux = None, requirements_lock = None, requirements_windows = None, + uv_lock = None, platforms, extra_pip_args = None, python_version = None, @@ -88,6 +89,7 @@ def requirements_files_by_platform( requirements_linux (label): The requirements file for the linux OS. requirements_lock (label): The requirements file for all OSes, or used as a fallback. requirements_windows (label): The requirements file for windows OS. + uv_lock (label): The uv.lock file, or used as primary source. extra_pip_args (string list): Extra pip arguments to perform extra validations and to be joined with args fined in files. python_version: str or None. This is needed when the get_index_urls is @@ -106,10 +108,11 @@ def requirements_files_by_platform( requirements_linux or requirements_osx or requirements_windows or - requirements_by_platform + requirements_by_platform or + uv_lock ): fail_fn( - "A 'requirements_lock' attribute must be specified, a platform-specific lockfiles " + + "A 'requirements_lock' or 'uv_lock' attribute must be specified, a platform-specific lockfiles " + "via 'requirements_by_platform' or an os-specific lockfiles must be specified " + "via 'requirements_*' attributes", ) @@ -143,9 +146,12 @@ def requirements_files_by_platform( fail_fn("only a single 'requirements_lock' file can be used when using '--platform' pip argument, consider specifying it via 'requirements_lock' attribute") return None - files_by_platform = [ - (lock_files[0], platforms_from_args), - ] + if not lock_files: + files_by_platform = [] + else: + files_by_platform = [ + (lock_files[0], platforms_from_args), + ] if logger: logger.debug(lambda: "Files by platform with the platform set in the args: {}".format(files_by_platform)) else: diff --git a/tests/pypi/hub_builder/hub_builder_tests.bzl b/tests/pypi/hub_builder/hub_builder_tests.bzl index 60017593fb..4651342dd4 100644 --- a/tests/pypi/hub_builder/hub_builder_tests.bzl +++ b/tests/pypi/hub_builder/hub_builder_tests.bzl @@ -79,6 +79,7 @@ def hub_builder( }, netrc = None, auth_patterns = None, + toml_decode = json.decode, ), whl_overrides = whl_overrides, minor_mapping = minor_mapping or {"3.15": "3.15.19"}, @@ -151,6 +152,30 @@ def _test_simple(env): _tests.append(_test_simple) +def _test_uv_lock_only(env): + builder = hub_builder(env) + builder.pip_parse( + _mock_mctx( + os_name = "osx", + arch_name = "aarch64", + mock_files = { + "uv.lock": """{"package":[{"name":"simple","source":{"registry":"https://pypi.org/simple"},"version":"0.0.1","wheels":[{"hash":"sha256:deadbeef","url":"https://files.pythonhosted.org/packages/simple-0.0.1-py3-none-any.whl"}]}]}""", + }, + ), + _parse( + hub_name = "pypi", + python_version = "3.15", + requirements_lock = None, + uv_lock = "uv.lock", + ), + ) + pypi = builder.build() + + pypi.exposed_packages().contains_exactly(["simple"]) + pypi.group_map().contains_exactly({}) + +_tests.append(_test_uv_lock_only) + def _test_simple_multiple_requirements(env): sub_tests = { ("osx", "aarch64"): "simple==0.0.2 --hash=sha256:deadb00f", diff --git a/tests/pypi/requirements_files_by_platform/requirements_files_by_platform_tests.bzl b/tests/pypi/requirements_files_by_platform/requirements_files_by_platform_tests.bzl index b1176e6a15..7f4260081b 100644 --- a/tests/pypi/requirements_files_by_platform/requirements_files_by_platform_tests.bzl +++ b/tests/pypi/requirements_files_by_platform/requirements_files_by_platform_tests.bzl @@ -43,7 +43,7 @@ def _test_fail_no_requirements(env): fail_fn = errors.append, ) env.expect.that_str(errors[0]).equals("""\ -A 'requirements_lock' attribute must be specified, a platform-specific lockfiles via 'requirements_by_platform' or an os-specific lockfiles must be specified via 'requirements_*' attributes""") +A 'requirements_lock' or 'uv_lock' attribute must be specified, a platform-specific lockfiles via 'requirements_by_platform' or an os-specific lockfiles must be specified via 'requirements_*' attributes""") _tests.append(_test_fail_no_requirements) @@ -319,6 +319,25 @@ def _test_host_only_os_with_fallback(env): _tests.append(_test_host_only_os_with_fallback) +def _test_uv_lock_only(env): + """Verify that using only uv_lock without requirements_lock succeeds.""" + got = requirements_files_by_platform( + uv_lock = "uv.lock", + ) + env.expect.that_dict(got).contains_exactly({}) + +_tests.append(_test_uv_lock_only) + +def _test_uv_lock_with_platform_arg(env): + """Verify that using uv_lock with --platform argument succeeds.""" + got = requirements_files_by_platform( + uv_lock = "uv.lock", + extra_pip_args = ["--platform", "linux_x86_64"], + ) + env.expect.that_dict(got).contains_exactly({}) + +_tests.append(_test_uv_lock_with_platform_arg) + def requirements_files_by_platform_test_suite(name): """Create the test suite. From e3211ac0f77e68cbec4cb9a9dcb1dbda228c1056 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Sun, 26 Jul 2026 23:39:46 -0700 Subject: [PATCH 861/922] agents: add rule for PR descriptions and news entries (#3959) Currently, AI agents may draft pull request descriptions or news entries without consulting repository style guides, leading to format regressions such as verbose diff lists or markdown section headers. To prevent this, add an agent rule that mandates reading and adhering to `CONTRIBUTING.md` before drafting pull request descriptions or news entries. --- .agents/rules/pr_and_news.md | 12 ++++++++++++ 1 file changed, 12 insertions(+) create mode 100644 .agents/rules/pr_and_news.md diff --git a/.agents/rules/pr_and_news.md b/.agents/rules/pr_and_news.md new file mode 100644 index 0000000000..0eb7726efc --- /dev/null +++ b/.agents/rules/pr_and_news.md @@ -0,0 +1,12 @@ +--- +trigger: model_decision +description: Apply when drafting pull request descriptions or news entries. +--- + +# PR Descriptions and News Entries + +Before drafting any pull request description or news entry, you MUST read +`CONTRIBUTING.md` (specifically the sections on **Commit messages and PR +descriptions** and **Documenting changes**) and strictly adhere to its style, +formatting, and structure rules. Do not generate PR descriptions or news +entries without consulting `CONTRIBUTING.md` first. From c1b85e92fd0762b60e816d3b71a1482afa98e612 Mon Sep 17 00:00:00 2001 From: Joshua Yanchar Date: Sun, 26 Jul 2026 23:27:20 -0700 Subject: [PATCH 862/922] fix(coverage): warn about a missing coverage wheel only for the selected runtime (#3955) Previously, configuring coverage checked every registered platform in `PLATFORMS` for a bundled `coverage.py` wheel and warned once per platform lacking one, even for platforms never used by the build and when coverage was not being collected. This change moves the warning to `py_runtime` during analysis so that: - It is only checked for the runtime actually selected by toolchain resolution. - It is only emitted when coverage is enabled (`ctx.configuration.coverage_enabled`). As a result, `coverage_dep` now silently returns `None` when no wheel is found, and `module_ctx` is no longer needed in `python_register_toolchains` for logging. Fixes #3950. --------- Co-authored-by: Claude Opus 5 Co-authored-by: Richard Levasseur --- docs/coverage.md | 13 +++-- news/3950.fixed.md | 3 ++ python/private/BUILD.bazel | 2 - python/private/coverage_deps.bzl | 28 +++-------- python/private/py_runtime_rule.bzl | 48 ++++++++++++++++++ python/private/python.bzl | 1 - python/private/python_register_toolchains.bzl | 15 ------ tests/coverage_deps/coverage_deps_test.bzl | 49 ++++--------------- tests/py_runtime/py_runtime_tests.bzl | 41 ++++++++++++++++ 9 files changed, 119 insertions(+), 81 deletions(-) create mode 100644 news/3950.fixed.md diff --git a/docs/coverage.md b/docs/coverage.md index 41c95a1b75..8c162d86fd 100644 --- a/docs/coverage.md +++ b/docs/coverage.md @@ -39,9 +39,16 @@ need to manually configure coverage (see below). :::{note} The bundled `coverage` wheel set covers CPython 3.9 through 3.14 (with -freethreaded variants for 3.13+). For Python versions outside that range, -`configure_coverage_tool = True` is a silent no-op and `bazel coverage` will -produce empty lcov data; manually configure coverage (see below) instead. +freethreaded variants for 3.13+), and not every platform within that range. +When the interpreter that `bazel coverage` actually selects has no bundled +wheel, `configure_coverage_tool = True` produces no coverage tool and +`bazel coverage` emits empty lcov data; manually configure coverage (see +below) instead. + +`py_runtime` warns when this happens. The warning is emitted during analysis, +for the runtime toolchain resolution selected and only when coverage is being +collected, so it does not fire for the many platforms toolchains are +registered for but a given build never uses. ::: ## Manually configuring coverage diff --git a/news/3950.fixed.md b/news/3950.fixed.md new file mode 100644 index 0000000000..4684beb890 --- /dev/null +++ b/news/3950.fixed.md @@ -0,0 +1,3 @@ +(coverage) The warning about a missing bundled `coverage.py` wheel is only +emitted once per affected and used toolchain when coverage is being collected +([#3950](https://github.com/bazel-contrib/rules_python/issues/3950)). diff --git a/python/private/BUILD.bazel b/python/private/BUILD.bazel index 17d18fef6f..a5f4d2da3e 100644 --- a/python/private/BUILD.bazel +++ b/python/private/BUILD.bazel @@ -358,7 +358,6 @@ bzl_library( name = "coverage_deps", srcs = ["coverage_deps.bzl"], deps = [ - ":repo_utils", ":version_label", "//python/private:bazel_tools", ], @@ -776,7 +775,6 @@ bzl_library( ":coverage_deps", ":full_version", ":python_repository", - ":repo_utils", ":toolchains_repo", "//python:versions", ], diff --git a/python/private/coverage_deps.bzl b/python/private/coverage_deps.bzl index a32b2e3f97..a3f211ba2a 100644 --- a/python/private/coverage_deps.bzl +++ b/python/private/coverage_deps.bzl @@ -17,7 +17,6 @@ load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive") load("@bazel_tools//tools/build_defs/repo:utils.bzl", "maybe") -load("//python/private:repo_utils.bzl", "repo_utils") load("//python/private:version_label.bzl", "version_label") # START: maintained by 'bazel run //tools/private/update_deps:update_coverage_deps ' @@ -167,7 +166,7 @@ _coverage_deps = { _coverage_patch = Label("//python/private:coverage.patch") -def coverage_dep(name, python_version, platform, visibility, logger = None): +def coverage_dep(name, python_version, platform, visibility): """Register a single coverage dependency based on the python version and platform. Args: @@ -175,37 +174,24 @@ def coverage_dep(name, python_version, platform, visibility, logger = None): python_version: The full python version. platform: The platform, which can be found in //python:versions.bzl PLATFORMS dict. visibility: The visibility of the coverage tool. - logger: {type}`repo_utils.logger | None` Optional logger used to emit a - warning when no wheel is available for the (python_version, - platform) pair. If not supplied, a default logger is constructed. Returns: The label of the coverage tool if the platform is supported, otherwise - None. """ - if logger == None: - logger = repo_utils.logger( - struct(getenv = lambda _: None), - name = "coverage_dep", - ) - if "windows" in platform: # NOTE @aignas 2023-01-19: currently we do not support windows as the - # upstream coverage wrapper is written in shell. Do not log any warning - # for now as it is not actionable. + # upstream coverage wrapper is written in shell. return None abi = "cp" + version_label(python_version) url, sha256 = _coverage_deps.get(abi, {}).get(platform, (None, "")) if url == None: - logger.warn(lambda: ( - "rules_python's bundled coverage tool has no wheel for " + - "python_version={}, platform={}. `bazel coverage` will produce " + - "empty lcov for py_test targets in this configuration. Either " + - "pin python_version to a version in the bundled set (see " + - "python/private/coverage_deps.bzl), or configure coverage " + - "manually via py_runtime.coverage_tool. See docs/coverage.md." - ).format(python_version, platform)) + # Toolchains are registered for every platform in PLATFORMS, most of + # which a given build never resolves, so warning here is noise. The + # empty-lcov outcome is reported by py_runtime instead, which is + # analyzed only for the runtime actually selected. See + # https://github.com/bazel-contrib/rules_python/issues/3950. return None maybe( diff --git a/python/private/py_runtime_rule.bzl b/python/private/py_runtime_rule.bzl index 48637389bf..b3399a24fb 100644 --- a/python/private/py_runtime_rule.bzl +++ b/python/private/py_runtime_rule.bzl @@ -25,6 +25,41 @@ load(":version.bzl", "version") _py_builtins = py_internal +def coverage_tool_missing_message(*, coverage_enabled, coverage_tool, label): + """Build the warning for a selected runtime that cannot produce coverage. + + Kept separate from the rule implementation so the decision is unit testable + without having to capture analysis-phase output. + + Args: + coverage_enabled: {type}`bool` whether the build is collecting coverage. + coverage_tool: {type}`File | None` the runtime's coverage entry point. + label: {type}`Label` the `py_runtime` being analyzed. + + Returns: + {type}`str | None` the message to print, or `None` when no warning is + warranted. + """ + if not coverage_enabled or coverage_tool: + return None + + return """ +====================================================================== +WARNING: Python runtime {label} has no coverage_tool. + `bazel coverage` will produce empty lcov data for py_test targets that + resolve to this runtime. + + For rules_python's hermetic toolchains, enable the bundled coverage.py: + python.toolchain(configure_coverage_tool = True) # bzlmod + python_register_toolchains(register_coverage_tool = True) # WORKSPACE + A bundled wheel must exist for this interpreter's version and platform; + python/private/coverage_deps.bzl lists what ships with rules_python. + + Otherwise, set py_runtime.coverage_tool directly. See + https://rules-python.readthedocs.io/en/latest/coverage.html +====================================================================== +""".format(label = label) + def _py_runtime_impl(ctx): interpreter_path = ctx.attr.interpreter_path or None # Convert empty string to None interpreter = ctx.attr.interpreter @@ -108,6 +143,19 @@ def _py_runtime_impl(ctx): coverage_tool = None coverage_files = None + # Reported here rather than where the toolchains are registered: this rule is + # analyzed once per configuration, and only for the runtime that toolchain + # resolution actually selected, so the empty-lcov outcome is real rather than + # hypothetical. See https://github.com/bazel-contrib/rules_python/issues/3950. + coverage_warning = coverage_tool_missing_message( + coverage_enabled = ctx.configuration.coverage_enabled, + coverage_tool = coverage_tool, + label = ctx.label, + ) + if coverage_warning: + # buildifier: disable=print + print(coverage_warning) + python_version = ctx.attr.python_version interpreter_version_info = ctx.attr.interpreter_version_info diff --git a/python/private/python.bzl b/python/private/python.bzl index f098a626fb..0d7d2baf40 100644 --- a/python/private/python.bzl +++ b/python/private/python.bzl @@ -278,7 +278,6 @@ def _python_impl(module_ctx): register_result = python_register_toolchains( name = toolchain_info.name, _internal_bzlmod_toolchain_call = True, - _internal_module_ctx = module_ctx, **kwargs ) if not register_result.impl_repos: diff --git a/python/private/python_register_toolchains.bzl b/python/private/python_register_toolchains.bzl index 5a2f96857b..c6f827be22 100644 --- a/python/private/python_register_toolchains.bzl +++ b/python/private/python_register_toolchains.bzl @@ -26,7 +26,6 @@ load( load(":coverage_deps.bzl", "coverage_dep") load(":full_version.bzl", "full_version") load(":python_repository.bzl", "python_repository") -load(":repo_utils.bzl", "repo_utils") load( ":toolchains_repo.bzl", "host_compatible_python_repo", @@ -90,19 +89,6 @@ def python_register_toolchains( if bzlmod_toolchain_call: register_toolchains = False - # When invoked from the bzlmod python extension, a module_ctx is plumbed in - # so the coverage_dep logger can attribute warnings to the right module and - # honor module-root filtering. In the WORKSPACE/macro path no module_ctx is - # available; a minimal stand-in struct gives the logger what it needs. - module_ctx = kwargs.pop("_internal_module_ctx", None) - if module_ctx != None: - coverage_logger = repo_utils.logger(module_ctx, name = "coverage_dep") - else: - coverage_logger = repo_utils.logger( - struct(getenv = lambda _: None), - name = "coverage_dep", - ) - base_urls = kwargs.pop("base_urls", DEFAULT_RELEASE_BASE_URLS) tool_versions = tool_versions or TOOL_VERSIONS minor_mapping = minor_mapping or MINOR_MAPPING @@ -140,7 +126,6 @@ def python_register_toolchains( ), python_version = python_version, platform = platform, - logger = coverage_logger, visibility = ["@{name}_{platform}//:__subpackages__".format( name = name, platform = platform, diff --git a/tests/coverage_deps/coverage_deps_test.bzl b/tests/coverage_deps/coverage_deps_test.bzl index 12351affde..2d84d7b919 100644 --- a/tests/coverage_deps/coverage_deps_test.bzl +++ b/tests/coverage_deps/coverage_deps_test.bzl @@ -12,68 +12,39 @@ # See the License for the specific language governing permissions and # limitations under the License. -"Tests for the warning emitted by coverage_dep when no wheel is available." +"Tests for coverage_dep's handling of platforms with no bundled wheel." load("@rules_testing//lib:test_suite.bzl", "test_suite") load("//python/private:coverage_deps.bzl", "coverage_dep") # buildifier: disable=bzl-visibility -load("//python/private:repo_utils.bzl", "REPO_DEBUG_ENV_VAR", "REPO_VERBOSITY_ENV_VAR", "repo_utils") # buildifier: disable=bzl-visibility _tests = [] -def _capturing_logger(): - """Build a (logger, captured_messages_list) pair. - - The logger has its verbosity set to INFO so WARN messages are captured but - nothing noisier than necessary is emitted. The printer collects the second - positional argument from each printer invocation (the formatted message). - """ - captured = [] - logger = repo_utils.logger( - struct( - getenv = { - REPO_DEBUG_ENV_VAR: None, - REPO_VERBOSITY_ENV_VAR: "INFO", - }.get, - ), - name = "unit-test", - printer = lambda _key, message: captured.append(message), - ) - return logger, captured - -def _test_unsupported_python_version_warns(env): - # cp37 is not in the bundled wheel set; coverage_dep should return None - # and emit a warning describing the misconfiguration. - logger, captured = _capturing_logger() +def _test_unsupported_python_version_returns_none(env): + # cp37 is not in the bundled wheel set, so there is no coverage tool to + # attach to the runtime. Reporting that is py_runtime's job -- registration + # covers every platform in PLATFORMS, most of which are never selected. result = coverage_dep( name = "unused_for_test", python_version = "3.7", platform = "aarch64-apple-darwin", visibility = ["//visibility:public"], - logger = logger, ) env.expect.that_bool(result == None).equals(True) - env.expect.that_int(len(captured)).equals(1) - env.expect.that_str(captured[0]).contains("no wheel for") - env.expect.that_str(captured[0]).contains("python_version=3.7") - env.expect.that_str(captured[0]).contains("platform=aarch64-apple-darwin") -_tests.append(_test_unsupported_python_version_warns) +_tests.append(_test_unsupported_python_version_returns_none) -def _test_windows_platform_is_silent(env): - # Windows is intentionally unsupported and not actionable; coverage_dep - # must return None without logging anything. - logger, captured = _capturing_logger() +def _test_windows_platform_returns_none(env): + # Windows is intentionally unsupported: the upstream coverage wrapper is + # written in shell. result = coverage_dep( name = "unused_for_test", python_version = "3.10", platform = "x86_64-pc-windows-msvc", visibility = ["//visibility:public"], - logger = logger, ) env.expect.that_bool(result == None).equals(True) - env.expect.that_int(len(captured)).equals(0) -_tests.append(_test_windows_platform_is_silent) +_tests.append(_test_windows_platform_returns_none) # NOTE: there is intentionally no unit test for the supported-wheel path # (where coverage_dep returns a non-None label and emits no warning). diff --git a/tests/py_runtime/py_runtime_tests.bzl b/tests/py_runtime/py_runtime_tests.bzl index ac80c8556d..47cd97f142 100644 --- a/tests/py_runtime/py_runtime_tests.bzl +++ b/tests/py_runtime/py_runtime_tests.bzl @@ -20,9 +20,49 @@ load("@rules_testing//lib:util.bzl", rt_util = "util") load("//python:py_runtime.bzl", "py_runtime") load("//python:py_runtime_info.bzl", "PyRuntimeInfo") load("//python/private:common_labels.bzl", "labels") # buildifier: disable=bzl-visibility +load("//python/private:py_runtime_rule.bzl", "coverage_tool_missing_message") # buildifier: disable=bzl-visibility load("//tests/support:py_runtime_info_subject.bzl", "py_runtime_info_subject") _tests = [] +_basic_tests = [] + +def _test_coverage_warning_when_enabled_and_no_tool(env): + # The only case that warrants a warning: coverage is being collected and the + # runtime that was selected cannot produce any. + msg = coverage_tool_missing_message( + coverage_enabled = True, + coverage_tool = None, + label = Label("//fake:runtime"), + ) + env.expect.that_bool(msg == None).equals(False) + env.expect.that_str(msg).contains("has no coverage_tool") + env.expect.that_str(msg).contains("//fake:runtime") + +_basic_tests.append(_test_coverage_warning_when_enabled_and_no_tool) + +def _test_no_coverage_warning_when_coverage_disabled(env): + # Without `bazel coverage` the missing tool has no observable effect, so + # saying anything would be noise on every ordinary build. + msg = coverage_tool_missing_message( + coverage_enabled = False, + coverage_tool = None, + label = Label("//fake:runtime"), + ) + env.expect.that_bool(msg == None).equals(True) + +_basic_tests.append(_test_no_coverage_warning_when_coverage_disabled) + +def _test_no_coverage_warning_when_tool_present(env): + # A configured coverage tool is the working case, whether it came from the + # bundled wheel set or from py_runtime.coverage_tool directly. + msg = coverage_tool_missing_message( + coverage_enabled = True, + coverage_tool = "some-coverage-tool-file", + label = Label("//fake:runtime"), + ) + env.expect.that_bool(msg == None).equals(True) + +_basic_tests.append(_test_no_coverage_warning_when_tool_present) def _simple_binary_impl(ctx): executable = ctx.actions.declare_file(ctx.label.name) @@ -557,5 +597,6 @@ _tests.append(_test_version_info_from_flag) def py_runtime_test_suite(name): test_suite( name = name, + basic_tests = _basic_tests, tests = _tests, ) From c0fec11c63a13b864cfba0038d43575af7adbee4 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Mon, 27 Jul 2026 10:00:31 -0700 Subject: [PATCH 863/922] build: move requirements and lock files out of docs/ to dev/ (#3957) build(deps): move root requirements and lock files out of docs/ to dev/ Currently, the root development requirements and lock files reside in the `docs/` directory, which is confusing as they contain development and testing dependencies for the entire repository, not just documentation. To fix this, create a `dev/` directory and move `pyproject.toml`, `requirements.txt`, `uv.lock`, and their associated lock build rules into it. Update repository references and regenerate the lockfiles accordingly. --- CONTRIBUTING.md | 2 +- MODULE.bazel | 4 +-- WORKSPACE | 4 +-- dev/BUILD.bazel | 28 +++++++++++++++ {docs => dev}/pyproject.toml | 2 +- {docs => dev}/requirements.txt | 62 +++++++++++++++++----------------- {docs => dev}/uv.lock | 2 +- docs/BUILD.bazel | 26 -------------- private/BUILD.bazel | 2 +- 9 files changed, 67 insertions(+), 65 deletions(-) create mode 100644 dev/BUILD.bazel rename {docs => dev}/pyproject.toml (95%) rename {docs => dev}/requirements.txt (96%) rename {docs => dev}/uv.lock (99%) diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 73ab6d6220..9efce5a148 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -276,7 +276,7 @@ merged: * **requirements lock files**: These are usually generated by a `compile_pip_requirements` update target, which is usually in the same directory. - e.g. `bazel run //docs:requirements.update` + e.g. `bazel run //dev:requirements.update` ## Binary artifacts diff --git a/MODULE.bazel b/MODULE.bazel index 2df354fe61..1c2042152e 100644 --- a/MODULE.bazel +++ b/MODULE.bazel @@ -180,13 +180,13 @@ dev_pip = use_extension( hub_name = "dev_pip", parallel_download = False, python_version = python_version, - requirements_lock = "//docs:requirements.txt", + requirements_lock = "//dev:requirements.txt", # Ensure that we are setting up the following platforms target_platforms = [ "{os}_{arch}", "{os}_{arch}_freethreaded", ], - uv_lock = "//docs:uv.lock", + uv_lock = "//dev:uv.lock", ) for python_version in [ "3.9", diff --git a/WORKSPACE b/WORKSPACE index 59b8489760..a692f1b289 100644 --- a/WORKSPACE +++ b/WORKSPACE @@ -143,12 +143,12 @@ load("@pypiserver//:requirements.bzl", install_pypiserver = "install_deps", pypi install_pypiserver() ##################### -# Install sphinx for doc generation. +# Install dev dependencies. pip_parse( name = "dev_pip", python_interpreter_target = interpreter, - requirements_lock = "//docs:requirements.txt", + requirements_lock = "//dev:requirements.txt", ) load("@dev_pip//:requirements.bzl", dev_pip_requirements = "all_requirements", docs_install_deps = "install_deps") diff --git a/dev/BUILD.bazel b/dev/BUILD.bazel new file mode 100644 index 0000000000..91b2a9f566 --- /dev/null +++ b/dev/BUILD.bazel @@ -0,0 +1,28 @@ +load("//python/uv:lock.bzl", "lock") # buildifier: disable=bzl-visibility + +licenses(["notice"]) + +# Run bazel run //dev:requirements.update +lock( + name = "requirements", + srcs = ["pyproject.toml"], + out = "requirements.txt", + args = [ + "--emit-index-url", + "--universal", + "--upgrade", + ], + # NOTE @aignas 2025-08-17: here we select the lowest actively supported version so that the + # requirements file is generated to be compatible with Python version 3.9 or greater. + python_version = "3.9", + visibility = ["//:__subpackages__"], +) + +# Run bazel run //dev:uv_lock.update +lock( + name = "uv_lock", + srcs = ["pyproject.toml"], + out = "uv.lock", + python_version = "3.9", + visibility = ["//:__subpackages__"], +) diff --git a/docs/pyproject.toml b/dev/pyproject.toml similarity index 95% rename from docs/pyproject.toml rename to dev/pyproject.toml index e54fd53504..cdc00b902c 100644 --- a/docs/pyproject.toml +++ b/dev/pyproject.toml @@ -1,5 +1,5 @@ [project] -name = "rules_python_docs" +name = "rules_python_dev" version = "0.0.0" dependencies = [ diff --git a/docs/requirements.txt b/dev/requirements.txt similarity index 96% rename from docs/requirements.txt rename to dev/requirements.txt index 9a266e9644..943fce2dcf 100644 --- a/docs/requirements.txt +++ b/dev/requirements.txt @@ -1,15 +1,15 @@ # This file was autogenerated by uv via the following command: -# bazel run //docs:requirements.update +# bazel run //dev:requirements.update --index-url https://pypi.org/simple absl-py==2.3.1 ; python_full_version < '3.10' \ --hash=sha256:a97820526f7fbfd2ec1bce83f3f25e3a14840dac0d8e02a0b71cd75db3f77fc9 \ --hash=sha256:eeecf07f0c2a93ace0772c92e596ace6d3d3996c042b2128459aaae2a76de11d - # via rules-python-docs (docs/pyproject.toml) + # via rules-python-dev (dev/pyproject.toml) absl-py==2.5.0 ; python_full_version >= '3.10' \ --hash=sha256:0c996f25c0490700fadabe6351630f6111534fa0ae252cc6d2014ea3b141135f \ --hash=sha256:0f17b89f2a4eaaedc4f28c622998aa690564b3012a396a4ffad0821007fe03ba - # via rules-python-docs (docs/pyproject.toml) + # via rules-python-dev (dev/pyproject.toml) alabaster==0.7.16 ; python_full_version < '3.10' \ --hash=sha256:75a8b99c28a5dad50dd7f8ccdd447a121ddb3892da9e53d1ca5cca3106d58d65 \ --hash=sha256:b46733c07dce03ae4e150330b975c75737fa60f0a7c591b6c8bf4928a28e2c92 @@ -30,9 +30,9 @@ babel==2.18.0 \ --hash=sha256:b80b99a14bd085fcacfa15c9165f651fbb3406e66cc603abf11c5750937c992d \ --hash=sha256:e2b422b277c2b9a9630c1d7903c2a00d0830c409c59ac8cae9081c92f1aeba35 # via sphinx -certifi==2026.6.17 \ - --hash=sha256:024c88eeec92ca068db80f02b8b07c9cef7b9fe261d1d535abfd5abd6f6af432 \ - --hash=sha256:2227dcbaafe0d2f59279d1762ddddc37783ed4354594f194ffc31d20f41fc3db +certifi==2026.7.22 \ + --hash=sha256:62f22742b58a1a33014a2b6b706588a8d7e2a88ae7bd1a6ebe8c992928483775 \ + --hash=sha256:741e2c3b351ddf169a738da9f2c048608ff7f2c5cc02f1ebc6b118bb090d5d55 # via requests charset-normalizer==3.4.9 \ --hash=sha256:0327fcd59a935777d83410750c50600ee9571af2846f71ce40f25b13da1ef380 \ @@ -187,7 +187,7 @@ jinja2==3.1.6 \ macholib==1.16.4 \ --hash=sha256:da1a3fa8266e30f0ce7e97c6a54eefaae8edd1e5f86f3eb8b95457cae90265ea \ --hash=sha256:f408c93ab2e995cd2c46e34fe328b130404be143469e41bc366c807448979362 - # via rules-python-docs (docs/pyproject.toml) + # via rules-python-dev (dev/pyproject.toml) markdown-it-py==3.0.0 ; python_full_version < '3.11' \ --hash=sha256:355216845c60bd96232cd8d8c40e8f9765cc86f46880e43a8fd22dc1a1a8cab1 \ --hash=sha256:e3f60a94fa066dc52ec76661e37c851cb232d92f9886b15cb560aaada2df8feb @@ -291,7 +291,7 @@ markupsafe==3.0.3 \ --hash=sha256:f9e130248f4462aaa8e2552d547f36ddadbeaa573879158d721bbd33dfe4743a \ --hash=sha256:fed51ac40f757d41b7c48425901843666a6677e3e8eb0abcff09e4ba6e664f50 # via - # rules-python-docs (docs/pyproject.toml) + # rules-python-dev (dev/pyproject.toml) # jinja2 mdit-py-plugins==0.4.2 ; python_full_version < '3.10' \ --hash=sha256:0c673c3f889399a33b95e88d2f0d111b4447bdfea7f237dab2d488f459835636 \ @@ -308,15 +308,15 @@ mdurl==0.1.2 \ myst-parser==3.0.1 ; python_full_version < '3.10' \ --hash=sha256:6457aaa33a5d474aca678b8ead9b3dc298e89c68e67012e73146ea6fd54babf1 \ --hash=sha256:88f0cb406cb363b077d176b51c476f62d60604d68a8dcdf4832e080441301a87 - # via rules-python-docs (docs/pyproject.toml) -myst-parser==5.1.0 ; python_full_version == '3.10.*' \ - --hash=sha256:9c91c52b3cdb4d94a6506e4fab4e2f296c7623a0da0dcbe6de1565c3dad67a8a \ - --hash=sha256:ab69322dc6719dcc7f296479dbb70181b66df6ed315064f92dbc85c0e1bf2f02 - # via rules-python-docs (docs/pyproject.toml) + # via rules-python-dev (dev/pyproject.toml) +myst-parser==4.0.1 ; python_full_version == '3.10.*' \ + --hash=sha256:5cfea715e4f3574138aecbf7d54132296bfd72bb614d31168f48c477a830a7c4 \ + --hash=sha256:9134e88959ec3b5780aedf8a99680ea242869d012e8821db3126d427edc9c95d + # via rules-python-dev (dev/pyproject.toml) myst-parser==5.1.0 ; python_full_version >= '3.11' \ --hash=sha256:9c91c52b3cdb4d94a6506e4fab4e2f296c7623a0da0dcbe6de1565c3dad67a8a \ --hash=sha256:ab69322dc6719dcc7f296479dbb70181b66df6ed315064f92dbc85c0e1bf2f02 - # via rules-python-docs (docs/pyproject.toml) + # via rules-python-dev (dev/pyproject.toml) packaging==26.2 \ --hash=sha256:5fc45236b9446107ff2415ce77c807cee2862cb6fac22b8a73826d0693b0980e \ --hash=sha256:ff452ff5a3e828ce110190feff1178bb1f2ea2281fa2075aadb987c2fb221661 @@ -327,7 +327,7 @@ packaging==26.2 \ pefile==2024.8.26 \ --hash=sha256:3ff6c5d8b43e8c37bb6e6dd5085658d658a7a0bdcd20b6a07b1fcfc1c4e9d632 \ --hash=sha256:76f8b485dcd3b1bb8166f1128d395fa3d87af26360c2358fb75b80019b957c6f - # via rules-python-docs (docs/pyproject.toml) + # via rules-python-dev (dev/pyproject.toml) pluggy==1.6.0 \ --hash=sha256:7dcc130b76258d33b90f61b658791dede3486c3e6bfb003ee5c9bfb396dd22f3 \ --hash=sha256:e920276dd6813095e9377c0bc5566d94c932c33b27a3e3945d8389c374dd4746 @@ -335,11 +335,11 @@ pluggy==1.6.0 \ pyelftools==0.32 ; python_full_version < '3.10' \ --hash=sha256:013df952a006db5e138b1edf6d8a68ecc50630adbd0d83a2d41e7f846163d738 \ --hash=sha256:6de90ee7b8263e740c8715a925382d4099b354f29ac48ea40d840cf7aa14ace5 - # via rules-python-docs (docs/pyproject.toml) + # via rules-python-dev (dev/pyproject.toml) pyelftools==0.33 ; python_full_version >= '3.10' \ --hash=sha256:660d82dcbeb8e83d1702bd97f223f761625da06111c0cc988eac6b8ab0c1b61f \ --hash=sha256:f215ad5f47d3f1373a21496a6c9e0707c622840d0622f23ff7ce08678b020036 - # via rules-python-docs (docs/pyproject.toml) + # via rules-python-dev (dev/pyproject.toml) pygments==2.20.0 \ --hash=sha256:6757cd03768053ff99f3039c1a36d6c0aa0b263438fcab17520b30a303a82b5f \ --hash=sha256:81a9e26dd42fd28a23a2d169d86d7ac03b46e2f8b59ed4698fb4785f946d0176 @@ -350,23 +350,23 @@ pytest==8.4.2 ; python_full_version < '3.10' \ --hash=sha256:86c0d0b93306b961d58d62a4db4879f27fe25513d4b969df351abdddb3c30e01 \ --hash=sha256:872f880de3fc3a5bdc88a11b39c9710c3497a547cfa9320bc3c5e62fbf272e79 # via - # rules-python-docs (docs/pyproject.toml) + # rules-python-dev (dev/pyproject.toml) # pytest-bazel # pytest-mock pytest==9.1.1 ; python_full_version >= '3.10' \ --hash=sha256:1088fbde8f2b49d95a549a195707afa7a76a3ce9bcadc26b6d71f0ffda5fe313 \ --hash=sha256:37a86b45efb9a47a61a36449063e8e18d0cab3161329fc099eb21783169c4f0c # via - # rules-python-docs (docs/pyproject.toml) + # rules-python-dev (dev/pyproject.toml) # pytest-bazel # pytest-mock pytest-bazel==0.1.6 \ --hash=sha256:a29e80e1d67c3db801bdd4d0b6b742f2bfb48cd6841caa33401458e5c4e29c21 - # via rules-python-docs (docs/pyproject.toml) + # via rules-python-dev (dev/pyproject.toml) pytest-mock==3.15.1 \ --hash=sha256:0a25e2eb88fe5168d535041d09a4529a188176ae608a6d249ee65abc0949630d \ --hash=sha256:1849a238f6f396da19762269de72cb1814ab44416fa73a8686deac10b0d87a0f - # via rules-python-docs (docs/pyproject.toml) + # via rules-python-dev (dev/pyproject.toml) pyyaml==6.0.3 \ --hash=sha256:00c4bdeba853cc34e7dd471f16b4114f4162dc03e6b7afcc2128711f0eca823c \ --hash=sha256:0150219816b6a1fa26fb4699fb7daa9caf09eb1999f3b70fb6e786805e80375a \ @@ -445,7 +445,7 @@ pyyaml==6.0.3 \ readthedocs-sphinx-ext==2.2.5 \ --hash=sha256:ee5fd5b99db9f0c180b2396cbce528aa36671951b9526bb0272dbfce5517bd27 \ --hash=sha256:f8c56184ea011c972dd45a90122568587cc85b0127bc9cf064d17c68bc809daa - # via rules-python-docs (docs/pyproject.toml) + # via rules-python-dev (dev/pyproject.toml) requests==2.32.5 ; python_full_version < '3.10' \ --hash=sha256:2462f94637a34fd532264295e186976db0f5d453d1cdd31473c85a6a161affb6 \ --hash=sha256:dbba0bac56e100853db0ea71b82b4dfd5fe2bf6d3754a8893c3af500cec7d7cf @@ -470,7 +470,7 @@ sphinx==7.4.7 ; python_full_version < '3.10' \ --hash=sha256:242f92a7ea7e6c5b406fdc2615413890ba9f699114a9c09192d7dfead2ee9cfe \ --hash=sha256:c2419e2135d11f1951cd994d6eb18a1835bd8fdd8429f9ca375dc1f3281bd239 # via - # rules-python-docs (docs/pyproject.toml) + # rules-python-dev (dev/pyproject.toml) # myst-parser # sphinx-reredirects # sphinx-rtd-theme @@ -479,7 +479,7 @@ sphinx==8.1.3 ; python_full_version == '3.10.*' \ --hash=sha256:09719015511837b76bf6e03e42eb7595ac8c2e41eeb9c29c5b755c6b677992a2 \ --hash=sha256:43c1911eecb0d3e161ad78611bc905d1ad0e523e4ddc202a58a821773dc4c927 # via - # rules-python-docs (docs/pyproject.toml) + # rules-python-dev (dev/pyproject.toml) # myst-parser # sphinx-reredirects # sphinx-rtd-theme @@ -488,7 +488,7 @@ sphinx==9.0.4 ; python_full_version == '3.11.*' \ --hash=sha256:594ef59d042972abbc581d8baa577404abe4e6c3b04ef61bd7fc2acbd51f3fa3 \ --hash=sha256:5bebc595a5e943ea248b99c13814c1c5e10b3ece718976824ffa7959ff95fffb # via - # rules-python-docs (docs/pyproject.toml) + # rules-python-dev (dev/pyproject.toml) # myst-parser # sphinx-reredirects # sphinx-rtd-theme @@ -497,7 +497,7 @@ sphinx==9.1.0 ; python_full_version >= '3.12' \ --hash=sha256:7741722357dd75f8190766926071fed3bdc211c74dd2d7d4df5404da95930ddb \ --hash=sha256:c84fdd4e782504495fe4f2c0b3413d6c2bf388589bb352d439b2a3bb99991978 # via - # rules-python-docs (docs/pyproject.toml) + # rules-python-dev (dev/pyproject.toml) # myst-parser # sphinx-reredirects # sphinx-rtd-theme @@ -505,19 +505,19 @@ sphinx==9.1.0 ; python_full_version >= '3.12' \ sphinx-autodoc2==0.5.0 \ --hash=sha256:7d76044aa81d6af74447080182b6868c7eb066874edc835e8ddf810735b6565a \ --hash=sha256:e867013b1512f9d6d7e6f6799f8b537d6884462acd118ef361f3f619a60b5c9e - # via rules-python-docs (docs/pyproject.toml) + # via rules-python-dev (dev/pyproject.toml) sphinx-reredirects==0.1.6 ; python_full_version < '3.11' \ --hash=sha256:c491cba545f67be9697508727818d8626626366245ae64456fe29f37e9bbea64 \ --hash=sha256:efd50c766fbc5bf40cd5148e10c00f2c00d143027de5c5e48beece93cc40eeea - # via rules-python-docs (docs/pyproject.toml) + # via rules-python-dev (dev/pyproject.toml) sphinx-reredirects==1.1.0 ; python_full_version >= '3.11' \ --hash=sha256:4b5692273c72cd2d4d917f4c6f87d5919e4d6114a752d4be033f7f5f6310efd9 \ --hash=sha256:fb9b195335ab14b43f8273287d0c7eeb637ba6c56c66581c11b47202f6718b29 - # via rules-python-docs (docs/pyproject.toml) + # via rules-python-dev (dev/pyproject.toml) sphinx-rtd-theme==3.1.0 \ --hash=sha256:1785824ae8e6632060490f67cf3a72d404a85d2d9fc26bce3619944de5682b89 \ --hash=sha256:b44276f2c276e909239a4f6c955aa667aaafeb78597923b1c60babc76db78e4c - # via rules-python-docs (docs/pyproject.toml) + # via rules-python-dev (dev/pyproject.toml) sphinxcontrib-applehelp==2.0.0 \ --hash=sha256:2f29ef331735ce958efa4734873f084941970894c6090408b079c61b2e1c06d1 \ --hash=sha256:4cd3f0ec4ac5dd9c17ec65e9ab272c9b867ea77425228e68ecf08d6b28ddbdb5 @@ -602,7 +602,7 @@ typing-extensions==4.16.0 \ --hash=sha256:481caa481374e813c1b176ada14e97f1f67a4539ce9cfeb3f350d78d6370c2e8 \ --hash=sha256:dc983d19a509c94dba722ee6abd33940f7c05a89e243c47e907eb4db6f1a43e5 # via - # rules-python-docs (docs/pyproject.toml) + # rules-python-dev (dev/pyproject.toml) # astroid # exceptiongroup # sphinx-autodoc2 diff --git a/docs/uv.lock b/dev/uv.lock similarity index 99% rename from docs/uv.lock rename to dev/uv.lock index d01b79b784..842e831dec 100644 --- a/docs/uv.lock +++ b/dev/uv.lock @@ -868,7 +868,7 @@ wheels = [ ] [[package]] -name = "rules-python-docs" +name = "rules-python-dev" version = "0.0.0" source = { virtual = "." } dependencies = [ diff --git a/docs/BUILD.bazel b/docs/BUILD.bazel index 9a9327aadf..dd304c410a 100644 --- a/docs/BUILD.bazel +++ b/docs/BUILD.bazel @@ -20,7 +20,6 @@ load("@sphinxdocs//sphinxdocs:sphinx_stardoc.bzl", "sphinx_stardoc", "sphinx_sta load("//python:defs.bzl", "py_binary") load("//python/private:bzlmod_enabled.bzl", "BZLMOD_ENABLED") # buildifier: disable=bzl-visibility load("//python/private:common_labels.bzl", "labels") # buildifier: disable=bzl-visibility -load("//python/uv:lock.bzl", "lock") # buildifier: disable=bzl-visibility package(default_visibility = ["//:__subpackages__"]) @@ -208,28 +207,3 @@ sphinx_build_binary( "@sphinxdocs//sphinxdocs/src/sphinx_bzl", ], ) - -# Run bazel run //docs:requirements.update -lock( - name = "requirements", - srcs = ["pyproject.toml"], - out = "requirements.txt", - args = [ - "--emit-index-url", - "--universal", - "--upgrade", - ], - # NOTE @aignas 2025-08-17: here we select the lowest actively supported version so that the - # requirements file is generated to be compatible with Python version 3.9 or greater. - python_version = "3.9", - visibility = ["//:__subpackages__"], -) - -# Run bazel run //docs:uv_lock.update -lock( - name = "uv_lock", - srcs = ["pyproject.toml"], - out = "uv.lock", - python_version = "3.9", - visibility = ["//:__subpackages__"], -) diff --git a/private/BUILD.bazel b/private/BUILD.bazel index ef5652b826..c6bfe6bd9f 100644 --- a/private/BUILD.bazel +++ b/private/BUILD.bazel @@ -13,7 +13,7 @@ multirun( "requirements_linux", ] ] + [ - "//docs:requirements.update", + "//dev:requirements.update", ], tags = ["manual"], ) From 872da66a0884612b40442628e48f1601937627ed Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Tue, 28 Jul 2026 16:47:39 +0000 Subject: [PATCH 864/922] build(deps): bump actions/download-artifact from 4 to 8 (#3963) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Bumps [actions/download-artifact](https://github.com/actions/download-artifact) from 4 to 8.
Release notes

Sourced from actions/download-artifact's releases.

v8.0.0

v8 - What's new

[!IMPORTANT] actions/download-artifact@v8 has been migrated to an ESM module. This should be transparent to the caller but forks might need to make significant changes.

[!IMPORTANT] Hash mismatches will now error by default. Users can override this behavior with a setting change (see below).

Direct downloads

To support direct uploads in actions/upload-artifact, the action will no longer attempt to unzip all downloaded files. Instead, the action checks the Content-Type header ahead of unzipping and skips non-zipped files. Callers wishing to download a zipped file as-is can also set the new skip-decompress parameter to true.

Enforced checks (breaking)

A previous release introduced digest checks on the download. If a download hash didn't match the expected hash from the server, the action would log a warning. Callers can now configure the behavior on mismatch with the digest-mismatch parameter. To be secure by default, we are now defaulting the behavior to error which will fail the workflow run.

ESM

To support new versions of the @actions/* packages, we've upgraded the package to ESM.

What's Changed

Full Changelog: https://github.com/actions/download-artifact/compare/v7...v8.0.0

v7.0.0

v7 - What's new

[!IMPORTANT] actions/download-artifact@v7 now runs on Node.js 24 (runs.using: node24) and requires a minimum Actions Runner version of 2.327.1. If you are using self-hosted runners, ensure they are updated before upgrading.

Node.js 24

This release updates the runtime to Node.js 24. v6 had preliminary support for Node 24, however this action was by default still running on Node.js 20. Now this action by default will run on Node.js 24.

What's Changed

New Contributors

Full Changelog: https://github.com/actions/download-artifact/compare/v6.0.0...v7.0.0

v6.0.0

... (truncated)

Commits
  • 3e5f45b Add regression tests for CJK characters (#471)
  • e6d03f6 Add a regression test for artifact name + content-type mismatches (#472)
  • 70fc10c Merge pull request #461 from actions/danwkennedy/digest-mismatch-behavior
  • f258da9 Add change docs
  • ccc058e Fix linting issues
  • bd7976b Add a setting to specify what to do on hash mismatch and default it to error
  • ac21fcf Merge pull request #460 from actions/danwkennedy/download-no-unzip
  • 15999bf Add note about package bumps
  • 974686e Bump the version to v8 and add release notes
  • fbe48b1 Update test names to make it clearer what they do
  • Additional commits viewable in compare view

[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=actions/download-artifact&package-manager=github_actions&previous-version=4&new-version=8)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) ---
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot show ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> --- .github/workflows/automated_pr_review.yaml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.github/workflows/automated_pr_review.yaml b/.github/workflows/automated_pr_review.yaml index 3fe93db7f3..179b1dcd4b 100644 --- a/.github/workflows/automated_pr_review.yaml +++ b/.github/workflows/automated_pr_review.yaml @@ -73,7 +73,7 @@ jobs: runs-on: ubuntu-latest steps: - name: Download Diff Artifact - uses: actions/download-artifact@v4 + uses: actions/download-artifact@v8 with: name: pr_diff From 3f396ddf2dfaa188f7fd1ba080656240cd210643 Mon Sep 17 00:00:00 2001 From: Ignas Anikevicius <240938+aignas@users.noreply.github.com> Date: Wed, 29 Jul 2026 02:41:31 +0900 Subject: [PATCH 865/922] feat(coverage): fallback to pure Python wheel (#3961) This is an alternative way to solve the warning issue that was reported in the ticket. This means that `rules_python` from now on actually can register `coverage` for any UNIXy toolchain no matter the toolchain build. Fixes #3950 --- news/3950.fixed.md | 4 ++-- python/private/coverage_deps.bzl | 15 +++++++-------- tests/coverage_deps/coverage_deps_test.bzl | 14 -------------- .../private/update_deps/update_coverage_deps.py | 17 ++++++++++++++--- 4 files changed, 23 insertions(+), 27 deletions(-) diff --git a/news/3950.fixed.md b/news/3950.fixed.md index 4684beb890..87e0bd91a6 100644 --- a/news/3950.fixed.md +++ b/news/3950.fixed.md @@ -1,3 +1,3 @@ -(coverage) The warning about a missing bundled `coverage.py` wheel is only -emitted once per affected and used toolchain when coverage is being collected +(coverage) The warning about a missing bundled `coverage.py` wheel is no longer +emitted as we are now falling back to a pure python wheel ([#3950](https://github.com/bazel-contrib/rules_python/issues/3950)). diff --git a/python/private/coverage_deps.bzl b/python/private/coverage_deps.bzl index a3f211ba2a..001146c3d4 100644 --- a/python/private/coverage_deps.bzl +++ b/python/private/coverage_deps.bzl @@ -20,6 +20,10 @@ load("@bazel_tools//tools/build_defs/repo:utils.bzl", "maybe") load("//python/private:version_label.bzl", "version_label") # START: maintained by 'bazel run //tools/private/update_deps:update_coverage_deps ' +_default = ( + "https://files.pythonhosted.org/packages/ec/16/114df1c291c22cac3b0c127a73e0af5c12ed7bbb6558d310429a0ae24023/coverage-7.10.7-py3-none-any.whl", + "f7941f6f2fe6dd6807a1208737b8a0cbcf1cc6d7b07d24998ad2d63590868260", +) _coverage_deps = { "cp310": { "aarch64-apple-darwin": ( @@ -184,15 +188,10 @@ def coverage_dep(name, python_version, platform, visibility): return None abi = "cp" + version_label(python_version) - url, sha256 = _coverage_deps.get(abi, {}).get(platform, (None, "")) + url, sha256 = _coverage_deps.get(abi, {}).get(platform, _default) - if url == None: - # Toolchains are registered for every platform in PLATFORMS, most of - # which a given build never resolves, so warning here is noise. The - # empty-lcov outcome is reported by py_runtime instead, which is - # analyzed only for the runtime actually selected. See - # https://github.com/bazel-contrib/rules_python/issues/3950. - return None + # NOTE @aignas 2026-07-27: if the default is matched, then the same file may be extracted + # multiple times. The wheel is small enough to not matter in most cases. maybe( http_archive, diff --git a/tests/coverage_deps/coverage_deps_test.bzl b/tests/coverage_deps/coverage_deps_test.bzl index 2d84d7b919..ddb82cd6c5 100644 --- a/tests/coverage_deps/coverage_deps_test.bzl +++ b/tests/coverage_deps/coverage_deps_test.bzl @@ -19,20 +19,6 @@ load("//python/private:coverage_deps.bzl", "coverage_dep") # buildifier: disabl _tests = [] -def _test_unsupported_python_version_returns_none(env): - # cp37 is not in the bundled wheel set, so there is no coverage tool to - # attach to the runtime. Reporting that is py_runtime's job -- registration - # covers every platform in PLATFORMS, most of which are never selected. - result = coverage_dep( - name = "unused_for_test", - python_version = "3.7", - platform = "aarch64-apple-darwin", - visibility = ["//visibility:public"], - ) - env.expect.that_bool(result == None).equals(True) - -_tests.append(_test_unsupported_python_version_returns_none) - def _test_windows_platform_returns_none(env): # Windows is intentionally unsupported: the upstream coverage wrapper is # written in shell. diff --git a/tools/private/update_deps/update_coverage_deps.py b/tools/private/update_deps/update_coverage_deps.py index fcb44fcc7c..8a4ccb41ba 100755 --- a/tools/private/update_deps/update_coverage_deps.py +++ b/tools/private/update_deps/update_coverage_deps.py @@ -115,12 +115,12 @@ def _map( platform: str, **kwargs: Any, ): - if platform not in _supported_platforms: + if platform and platform not in _supported_platforms: return None return Dep( name=name, - platform=_supported_platforms[platform], + platform=_supported_platforms[platform] if platform else "", python=python_version, url=url, sha256=digests["sha256"], @@ -170,6 +170,7 @@ def main(): data = json.loads(response.read().decode("utf-8")) urls = [] + default_url = None for u in data["urls"]: if u["yanked"]: continue @@ -177,6 +178,10 @@ def main(): if not u["filename"].endswith(".whl"): continue + if u["filename"].endswith("py3-none-any.whl"): + default_url = _map(name=args.name, platform="", **u) + continue + if u["python_version"] not in args.py: continue @@ -196,7 +201,13 @@ def main(): # Update the coverage_deps, which are used to register deps update_file( path=args.update_file, - snippet=f"_coverage_deps = {repr(Deps(urls))}\n", + snippet="\n".join( + [ + f"_default = {repr(default_url)}", + f"_coverage_deps = {repr(Deps(urls))}", + "", + ] + ), start_marker="# START: maintained by 'bazel run //tools/private/update_deps:update_coverage_deps '", end_marker="# END: maintained by 'bazel run //tools/private/update_deps:update_coverage_deps '", dry_run=args.dry_run, From 758c8e2db2dd2c95a0872809b7aef7bbee38615a Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Tue, 28 Jul 2026 12:21:44 -0700 Subject: [PATCH 866/922] agents: add merge-pr skill for shepherding pull request merges (#3969) Add a standalone `merge-pr` skill that orchestrates pull request merges and merge queue monitoring. When invoked, the skill deploys a background `Merge PR Shepherd` subagent that uses `monitor-ci-results` to watch checks and analyze failure logs, automatically retries transient network flakes via `retry_buildkite_jobs.py`, and re-enqueues the pull request if it is ever ejected from the merge queue. --- .agents/skills/merge-pr/SKILL.md | 14 ++++++++++++++ 1 file changed, 14 insertions(+) create mode 100644 .agents/skills/merge-pr/SKILL.md diff --git a/.agents/skills/merge-pr/SKILL.md b/.agents/skills/merge-pr/SKILL.md new file mode 100644 index 0000000000..73086759ed --- /dev/null +++ b/.agents/skills/merge-pr/SKILL.md @@ -0,0 +1,14 @@ +--- +name: merge-pr +description: Merge a pull request into main, monitoring the merge queue, retrying CI flakes, and re-enqueuing if necessary +--- + +When the user asks to merge a pull request (e.g., "merge PR ", "merge this PR", or monitor its merge): + +1. **Enqueue for Merge**: Run `gh pr merge --auto --squash` to enable auto-merge or add the pull request to the merge queue. +2. **Invoke a Background Shepherd**: Launch a background subagent with the role `Merge PR Shepherd` to continuously watch the PR until it merges. +3. **Leverage Existing CI Skills**: + - Have the subagent use the **`monitor-ci-results`** skill to watch for CI check failures and generate analysis reports. + - Have the subagent use the **`buildkite-retry-job`** skill (`retry_buildkite_jobs.py `) to automatically retry any transient network flakes (e.g., HTTP 504 gateway timeouts, downloader errors). +4. **Queue Shepherding**: Periodically check `gh pr view --json state,autoMergeRequest`. While `state` is `"OPEN"`, ensure auto-merge is enabled / queued by running `gh pr merge --auto --squash`. If `autoMergeRequest` is null (e.g., ejected from the merge queue due to a CI flake in the temporary queue branch), re-enqueue it for merge by running `gh pr merge --auto --squash` once checks are retried or green. +5. **Completion Notification**: Once `state` becomes `"MERGED"`, send a high-priority message back to the parent conversation. From 7868f5d2f92b5d82bc999b3583f64e673b8a407f Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Tue, 28 Jul 2026 14:16:51 -0700 Subject: [PATCH 867/922] build(deps): bump actions/upload-artifact from 4 to 7 (#3964) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Bumps [actions/upload-artifact](https://github.com/actions/upload-artifact) from 4 to 7.
Release notes

Sourced from actions/upload-artifact's releases.

v7.0.0

v7 What's new

Direct Uploads

Adds support for uploading single files directly (unzipped). Callers can set the new archive parameter to false to skip zipping the file during upload. Right now, we only support single files. The action will fail if the glob passed resolves to multiple files. The name parameter is also ignored with this setting. Instead, the name of the artifact will be the name of the uploaded file.

ESM

To support new versions of the @actions/* packages, we've upgraded the package to ESM.

What's Changed

New Contributors

Full Changelog: https://github.com/actions/upload-artifact/compare/v6...v7.0.0

v6.0.0

v6 - What's new

[!IMPORTANT] actions/upload-artifact@v6 now runs on Node.js 24 (runs.using: node24) and requires a minimum Actions Runner version of 2.327.1. If you are using self-hosted runners, ensure they are updated before upgrading.

Node.js 24

This release updates the runtime to Node.js 24. v5 had preliminary support for Node.js 24, however this action was by default still running on Node.js 20. Now this action by default will run on Node.js 24.

What's Changed

Full Changelog: https://github.com/actions/upload-artifact/compare/v5.0.0...v6.0.0

v5.0.0

What's Changed

BREAKING CHANGE: this update supports Node v24.x. This is not a breaking change per-se but we're treating it as such.

... (truncated)

Commits
  • 043fb46 Merge pull request #797 from actions/yacaovsnc/update-dependency
  • 634250c Include changes in typespec/ts-http-runtime 0.3.5
  • e454baa Readme: bump all the example versions to v7 (#796)
  • 74fad66 Update the readme with direct upload details (#795)
  • bbbca2d Support direct file uploads (#764)
  • 589182c Upgrade the module to ESM and bump dependencies (#762)
  • 47309c9 Merge pull request #754 from actions/Link-/add-proxy-integration-tests
  • 02a8460 Add proxy integration test
  • b7c566a Merge pull request #745 from actions/upload-artifact-v6-release
  • e516bc8 docs: correct description of Node.js 24 support in README
  • Additional commits viewable in compare view

[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=actions/upload-artifact&package-manager=github_actions&previous-version=4&new-version=7)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) ---
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot show ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> --- .github/workflows/automated_pr_review.yaml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.github/workflows/automated_pr_review.yaml b/.github/workflows/automated_pr_review.yaml index 179b1dcd4b..dbfffa9083 100644 --- a/.github/workflows/automated_pr_review.yaml +++ b/.github/workflows/automated_pr_review.yaml @@ -62,7 +62,7 @@ jobs: # Upload extracted diff as artifact to pass to Job 2 safely as text data. - name: Upload Diff Artifact - uses: actions/upload-artifact@v4 + uses: actions/upload-artifact@v7 with: name: pr_diff path: pr_diff.txt From ca783803a869425b95111f5f7290ec74442fec68 Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Tue, 28 Jul 2026 20:11:20 -0700 Subject: [PATCH 868/922] build(deps): bump bazel-contrib/publish-to-bcr/.github/workflows/publish.yaml from 1.4.1 to 1.4.2 (#3925) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Bumps [bazel-contrib/publish-to-bcr/.github/workflows/publish.yaml](https://github.com/bazel-contrib/publish-to-bcr) from 1.4.1 to 1.4.2.
Release notes

Sourced from bazel-contrib/publish-to-bcr/.github/workflows/publish.yaml's releases.

v1.4.2

What's Changed

New Contributors

Full Changelog: https://github.com/bazel-contrib/publish-to-bcr/compare/v1.4.1...v1.4.2

Commits
  • ad6879f ci: make CI pass for fork PRs (#405)
  • 005a51c ci: pin actions in publish.yaml to full-length commit SHAs (#404)
  • 63d4406 chore(deps): update dependency @​types/nodemailer to v7.0.12 (#402)
  • 6b034ce chore(deps): update dependency globby to v16 (#355)
  • d4e8c21 chore(deps): update dependency typescript-eslint to v8.62.1 (#400)
  • 3650e3c chore(deps): update dependency buildifier_prebuilt to v8.5.1.2 (#399)
  • 832b5d7 chore(deps): update dependency mailparser to v3.9.3 [security] (#371)
  • ab8074b fix(deps): update dependency nodemailer to v8 [security] (#382)
  • 1c91c56 fix: update bazel lockfile when autopatching (#398)
  • 7fccf69 chore(deps): update dependency yaml to v2.8.3 [security] (#381)
  • Additional commits viewable in compare view

[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=bazel-contrib/publish-to-bcr/.github/workflows/publish.yaml&package-manager=github_actions&previous-version=1.4.1&new-version=1.4.2)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) ---
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot show ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Richard Levasseur --- .github/workflows/publish.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.github/workflows/publish.yml b/.github/workflows/publish.yml index c5db587f18..7b358956d9 100644 --- a/.github/workflows/publish.yml +++ b/.github/workflows/publish.yml @@ -21,7 +21,7 @@ on: type: string jobs: publish: - uses: bazel-contrib/publish-to-bcr/.github/workflows/publish.yaml@v1.4.1 + uses: bazel-contrib/publish-to-bcr/.github/workflows/publish.yaml@v1.4.2 with: tag_name: ${{ inputs.tag_name }} # GitHub repository which is a fork of the upstream where the Pull Request will be opened. From f87a4a7b7d6b29c0e2695028cee5c2f40a05bbb7 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Tue, 28 Jul 2026 21:20:15 -0700 Subject: [PATCH 869/922] build(agents): handle blocked Buildkite jobs in CI monitoring and update create-pr skill (#3975) build(agents): handle blocked Buildkite jobs in CI monitoring and update create-pr skill When remote Buildkite CI jobs enter a blocked state waiting for user confirmation, the CI monitoring script did not explicitly track or notify about blocked jobs, leading to unhandled paused builds. In addition, the `create-pr` skill lacked instructions for handling proposal-only workflows where subagents should draft PR information without executing `gh pr create`. To address this: * Updated `monitor_remote_ci.py` to identify `blocked` and `blocked_failed` job states, track blocked job counts separately, and send real-time notifications via `agentapi send-message` to alert users when a job is waiting for approval. * Documented blocked job orchestration in `.agents/skills/monitor-ci-results/SKILL.md`. * Updated `.agents/skills/create-pr/SKILL.md` to guide subagents on drafting PR proposals via `pr_info.md` artifacts when running in proposal mode rather than calling `gh pr create`. --- .agents/skills/create-pr/SKILL.md | 34 ++++++++ .agents/skills/monitor-ci-results/SKILL.md | 30 ++++--- .../scripts/monitor_remote_ci.py | 83 ++++++++++++++++--- 3 files changed, 126 insertions(+), 21 deletions(-) create mode 100644 .agents/skills/create-pr/SKILL.md diff --git a/.agents/skills/create-pr/SKILL.md b/.agents/skills/create-pr/SKILL.md new file mode 100644 index 0000000000..010d112fb8 --- /dev/null +++ b/.agents/skills/create-pr/SKILL.md @@ -0,0 +1,34 @@ +--- +name: create-pr +description: Create a pull request by delegating to a subagent +--- + +When creating a Pull Request for local changes or a branch, invoke a subagent +to handle PR creation or description drafting. + +### Instructions + +1. Launch a subagent using `invoke_subagent` with `TypeName: "self"` (or + `agentapi new-conversation`). +2. Provide a prompt to the subagent directing it to: + - Read `CONTRIBUTING.md` (specifically the sections on **Commit messages + and PR descriptions** and **Documenting changes**) before drafting. + - Strictly adhere to `CONTRIBUTING.md` rules for: + - **PR Title**: Follow conventional commit style and title formatting. + - **PR Body**: Include rationale, high-level summary, and structure. + - **Formatting**: Follow repository style guidelines and structure. + - Create a Markdown artifact (`pr_info.md`) containing the PR title, body, + and link/metadata so the user can review and comment on it. + - **Propose vs. Create**: If the user requested to propose or draft a PR + description, **do not** run `gh pr create`—just create the `pr_info.md` + artifact for the user to review. Otherwise, execute `gh pr create` with + the formatted title and body. +3. **Return Status**: Direct the subagent to communicate the PR number or draft + status back using `send_message` (or `agentapi send-message`) with the + parent conversation ID, or include it in its final completion response. +4. **Publish Artifact**: Upon receiving the subagent completion message, the + main agent must publish `pr_info.md` to display the artifact directly in + the primary user UI. +5. **Interactive Actions**: To present custom action choices to the user + (e.g., "Create PR", "Create Draft PR"), the main agent can use the + `ask_question` tool with custom options. diff --git a/.agents/skills/monitor-ci-results/SKILL.md b/.agents/skills/monitor-ci-results/SKILL.md index c5f8de309f..2b4bc5a460 100644 --- a/.agents/skills/monitor-ci-results/SKILL.md +++ b/.agents/skills/monitor-ci-results/SKILL.md @@ -1,21 +1,31 @@ --- name: monitor-ci-results -description: Monitor remote CI results for a PR and autonomously trigger log analysis upon failures +description: Monitor CI for PRs and notify of status --- -When the user requests to monitor remote CI results or watch a pull request, invoke `scripts/monitor_remote_ci.py `. +When the user requests to monitor remote CI results or watch a pull request, +invoke `scripts/monitor_remote_ci.py `. -This long-running monitoring service runs in the background and continuously polls both GitHub PR checks and Buildkite workflow executions. +This long-running monitoring service runs in the background and continuously +polls both GitHub PR checks and Buildkite workflow executions. -### ✨ Autonomous Failure Orchestration -When any CI job completes with errors or returns a non-zero exit code: -1. It automatically downloads the raw CI log file to `ci_logs/`. -2. It launches an independent background analyzer script (`analyze_ci_failure.py`). -3. It authors a beautifully structured Markdown suggested plan for how to fix the failure. -4. It natively dispatches a high-priority notification message back to your active agent conversation (containing the downloaded log path and fix plan) using `agentapi send-message`! +### ✨ Autonomous Failure and Blocked Job Orchestration +1. **Blocked Jobs**: When a Buildkite job or GitHub check is in a blocked + state waiting for user confirmation, it dispatches a notification via + `agentapi send-message` so the user is alerted to confirm running the job. +2. **Failure Analysis**: When any CI job completes with errors or returns a + non-zero exit code: + - It automatically downloads the raw CI log file to `ci_logs/`. + - It launches an independent background analyzer script + (`analyze_ci_failure.py`). + - It authors a structured Markdown plan to fix the failure. + - It natively dispatches a high-priority notification back to your active + agent conversation using `agentapi send-message`! ### Example Invocation ```bash ./scripts/monitor_remote_ci.py 3812 "0be435bd-96aa-4e1b-9c6f-727b31e80fa0" & ``` -*Note: Always include the trailing `&` when launching the monitoring script via tool calls to ensure it runs as a detached background task without blocking foreground execution.* +*Note: Always include the trailing `&` when launching the monitoring script via +tool calls to ensure it runs as a detached background task without blocking +foreground execution.* diff --git a/.agents/skills/monitor-ci-results/scripts/monitor_remote_ci.py b/.agents/skills/monitor-ci-results/scripts/monitor_remote_ci.py index fc1f8955d3..40be4d5d26 100755 --- a/.agents/skills/monitor-ci-results/scripts/monitor_remote_ci.py +++ b/.agents/skills/monitor-ci-results/scripts/monitor_remote_ci.py @@ -111,24 +111,32 @@ def main(): passed = 0 failed = 0 running = 0 + blocked = 0 other = 0 for job in jobs: jstate = job.get("state", "unknown") exit_status = job.get("exit_status") is_soft_failed = job.get("soft_failed") is True + is_blocked = jstate in ["blocked", "blocked_failed"] is_failed = ( - jstate in ["failed", "failing"] - or (exit_status != 0 and exit_status is not None) - ) and not is_soft_failed + ( + jstate in ["failed", "failing"] + or (exit_status != 0 and exit_status is not None) + ) + and not is_soft_failed + and not is_blocked + ) is_passed = ( jstate in ["passed", "success"] or (jstate == "finished" and exit_status == 0) or is_soft_failed - ) - is_running = jstate in ["running", "scheduled"] + ) and not is_blocked + is_running = jstate in ["running", "scheduled"] and not is_blocked - if is_failed: + if is_blocked: + blocked += 1 + elif is_failed: failed += 1 elif is_passed: passed += 1 @@ -140,7 +148,7 @@ def main(): build_id = link.split("/")[-1].split("#")[0] print( f"Buildkite #{build_id}: {len(jobs)} total jobs " - f"(Passed: {passed}, Failed: {failed}, Running: {running}, Other: {other})" + f"(Passed: {passed}, Failed: {failed}, Running: {running}, Blocked: {blocked}, Other: {other})" ) for job in jobs: @@ -151,12 +159,42 @@ def main(): exit_status = job.get("exit_status") is_soft_failed = job.get("soft_failed") is True + is_blocked = jstate in ["blocked", "blocked_failed"] is_failed = ( - jstate in ["failed", "failing"] - or (exit_status != 0 and exit_status is not None) - ) and not is_soft_failed + ( + jstate in ["failed", "failing"] + or (exit_status != 0 and exit_status is not None) + ) + and not is_soft_failed + and not is_blocked + ) - if is_failed and jkey not in monitored: + if is_blocked: + jkey_blocked = f"bk_blocked_{jid}" + if jkey_blocked not in monitored: + print( + f"⏸️ Notifying blocked state for Buildkite job '{jname}' (ID: {jid})..." + ) + msg = ( + f"⚠️ Remote CI Buildkite Job '{jname}' is blocked!\n\n" + f"Build ID: {build_id} | Job ID: {jid}\n" + f"Log URL: {job.get('log_url', link)}\n\n" + f"The user must confirm whether to run the CI jobs." + ) + subprocess.run( + [ + "agentapi", + "send-message", + "--title=CI Job Blocked", + args.conv_id, + msg, + ] + ) + monitored[jkey_blocked] = time.time() + with open(state_file, "w") as f: + json.dump(monitored, f) + + elif is_failed and jkey not in monitored: print( f"🚨 Notifying failure for Buildkite job '{jname}' (ID: {jid})..." ) @@ -179,6 +217,29 @@ def main(): with open(state_file, "w") as f: json.dump(monitored, f) + elif ( + state.upper() in ["ACTION_REQUIRED", "BLOCKED"] + and f"gh_blocked_{name}" not in monitored + ): + print(f"⏸️ Notifying blocked state for GitHub check '{name}'...") + msg = ( + f"⚠️ Remote CI GitHub Check '{name}' is blocked!\n\n" + f"Link: {link}\n\n" + f"The user must confirm whether to run the CI jobs." + ) + subprocess.run( + [ + "agentapi", + "send-message", + "--title=CI Check Blocked", + args.conv_id, + msg, + ] + ) + monitored[f"gh_blocked_{name}"] = time.time() + with open(state_file, "w") as f: + json.dump(monitored, f) + elif state in ["FAILURE", "failed"] and name not in monitored: print(f"🚨 Notifying failure for GitHub check '{name}'...") msg = ( From b16f7267140be0e889625044e1bbf836415e7a97 Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Wed, 29 Jul 2026 03:36:55 +0000 Subject: [PATCH 870/922] build(deps): bump nh3 from 0.3.0 to 0.3.6 in /tools/publish (#3966) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Bumps [nh3](https://github.com/messense/nh3) from 0.3.0 to 0.3.6.
Release notes

Sourced from nh3's releases.

v0.3.6

What's Changed

Full Changelog: https://github.com/messense/nh3/compare/v0.3.5...v0.3.6

v0.3.5

What's Changed

Full Changelog: https://github.com/messense/nh3/compare/v0.3.4...v0.3.5

v0.3.4

What's Changed

New Contributors

Full Changelog: https://github.com/messense/nh3/compare/v0.3.3...v0.3.4

v0.3.3

What's Changed

New Contributors

Full Changelog: https://github.com/messense/nh3/compare/v0.3.2...v0.3.3

v0.3.2

What's Changed

... (truncated)

Commits
  • bea70df Bump version to 0.3.6
  • 788cee2 Expose ammonia's url_relative policy via url_relative kwarg (#131)
  • e76a81a Bump pyo3 from 0.28.3 to 0.29.0 (#130)
  • 8ec45a3 Add nh3.escape alias for clean_text (#127)
  • a1e0226 Document tag_attribute_values as alternate to attributes (#126)
  • 389fb79 Validate clean_content_tags conflict with tags (#125)
  • 61aad5a Bump uraimo/run-on-arch-action in the github-actions group (#124)
  • 9e78e6a Bump version to 0.3.5
  • 5225ec2 Add tags parameter to clean_text (#122)
  • 129df52 Bump pyo3 from 0.28.2 to 0.28.3 (#123)
  • Additional commits viewable in compare view

[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=nh3&package-manager=pip&previous-version=0.3.0&new-version=0.3.6)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) ---
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot show ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
--------- Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Richard Levasseur --- .../bzlmod_lockfile/MODULE.bazel.lock | 416 ++++++++-------- tools/publish/requirements_darwin.txt | 166 ++++--- tools/publish/requirements_linux.txt | 461 +++++++++--------- tools/publish/requirements_universal.txt | 452 +++++++++-------- tools/publish/requirements_windows.txt | 166 ++++--- 5 files changed, 868 insertions(+), 793 deletions(-) diff --git a/tests/integration/bzlmod_lockfile/MODULE.bazel.lock b/tests/integration/bzlmod_lockfile/MODULE.bazel.lock index 92d1f6be7b..5fac194583 100644 --- a/tests/integration/bzlmod_lockfile/MODULE.bazel.lock +++ b/tests/integration/bzlmod_lockfile/MODULE.bazel.lock @@ -290,94 +290,110 @@ "https://files.pythonhosted.org/packages/b9/fa/123043af240e49752f1c4bd24da5053b6bd00cad78c2be53c0d1e8b975bc/backports.tarfile-1.2.0-py3-none-any.whl": "77e284d754527b01fb1e6fa8a1afe577858ebe4e9dad8919e34c862cb399bc34" }, "certifi": { - "https://files.pythonhosted.org/packages/4c/5b/b6ce21586237c77ce67d01dc5507039d444b630dd76611bbca2d8e5dcd91/certifi-2025.10.5.tar.gz": "47c09d31ccf2acf0be3f701ea53595ee7e0b8fa08801c6624be771df09ae7b43", - "https://files.pythonhosted.org/packages/e4/37/af0d2ef3967ac0d6113837b44a4f0bfe1328c2b9763bd5b1744520e5cfed/certifi-2025.10.5-py3-none-any.whl": "0f212c2744a9bb6de0c56639a6f68afe01ecd92d91f14ae897c4fe7bbeeef0de" + "https://files.pythonhosted.org/packages/0b/a7/71ac2cff56fec219ed242bb11b8efb69fcc4bec75db06fb7bfe35de520e6/certifi-2026.7.22-py3-none-any.whl": "62f22742b58a1a33014a2b6b706588a8d7e2a88ae7bd1a6ebe8c992928483775", + "https://files.pythonhosted.org/packages/a3/c2/24167ea9858356b47a87a50d39908bfdb72ceeefe0041586e704e5376b3a/certifi-2026.7.22.tar.gz": "741e2c3b351ddf169a738da9f2c048608ff7f2c5cc02f1ebc6b118bb090d5d55" }, "cffi": { - "https://files.pythonhosted.org/packages/05/eb/b86f2a2645b62adcfff53b0dd97e8dfafb5c8aa864bd0d9a2c2049a0d551/cffi-2.0.0-cp310-cp310-manylinux2014_ppc64le.manylinux_2_17_ppc64le.whl": "5eda85d6d1879e692d546a078b44251cdd08dd1cfb98dfb77b670c97cee49ea0", - "https://files.pythonhosted.org/packages/07/e0/267e57e387b4ca276b90f0434ff88b2c2241ad72b16d31836adddfd6031b/cffi-2.0.0-cp312-cp312-musllinux_1_2_aarch64.whl": "3925dd22fa2b7699ed2617149842d2e6adde22b262fcbfada50e3d195e4b3a94", - "https://files.pythonhosted.org/packages/0b/28/dd0967a76aab36731b6ebfe64dec4e981aff7e0608f60c2d46b46982607d/cffi-2.0.0-cp311-cp311-musllinux_1_2_x86_64.whl": "5fed36fccc0612a53f1d4d9a816b50a36702c28a2aa880cb8a122b3466638743", - "https://files.pythonhosted.org/packages/12/4a/3dfd5f7850cbf0d06dc84ba9aa00db766b52ca38d8b86e3a38314d52498c/cffi-2.0.0-cp311-cp311-macosx_10_13_x86_64.whl": "b4c854ef3adc177950a8dfc81a86f5115d2abd545751a304c5bcf2c2c7283cfe", - "https://files.pythonhosted.org/packages/15/12/a7a79bd0df4c3bff744b2d7e52cc1b68d5e7e427b384252c42366dc1ecbc/cffi-2.0.0-cp39-cp39-manylinux2014_aarch64.manylinux_2_17_aarch64.whl": "3f4d46d8b35698056ec29bca21546e1551a205058ae1a181d871e278b0b28165", - "https://files.pythonhosted.org/packages/1f/74/cc4096ce66f5939042ae094e2e96f53426a979864aa1f96a621ad128be27/cffi-2.0.0-cp39-cp39-manylinux2014_x86_64.manylinux_2_17_x86_64.whl": "61d028e90346df14fedc3d1e5441df818d095f3b87d286825dfcbd6459b7ef63", - "https://files.pythonhosted.org/packages/21/7a/13b24e70d2f90a322f2900c5d8e1f14fa7e2a6b3332b7309ba7b2ba51a5a/cffi-2.0.0-cp310-cp310-musllinux_1_2_aarch64.whl": "cf364028c016c03078a23b503f02058f1814320a56ad535686f90565636a9495", - "https://files.pythonhosted.org/packages/25/8e/342a504ff018a2825d395d44d63a767dd8ebc927ebda557fecdaca3ac33a/cffi-2.0.0-cp314-cp314t-musllinux_1_2_aarch64.whl": "7553fb2090d71822f02c629afe6042c299edf91ba1bf94951165613553984512", - "https://files.pythonhosted.org/packages/2b/0f/1f177e3683aead2bb00f7679a16451d302c436b5cbf2505f0ea8146ef59e/cffi-2.0.0-cp314-cp314-musllinux_1_2_aarch64.whl": "737fe7d37e1a1bffe70bd5754ea763a62a066dc5913ca57e957824b72a85e205", - "https://files.pythonhosted.org/packages/2b/c0/015b25184413d7ab0a410775fdb4a50fca20f5589b5dab1dbbfa3baad8ce/cffi-2.0.0-cp311-cp311-win32.whl": "c649e3a33450ec82378822b3dad03cc228b8f5963c0c12fc3b1e0ab940f768a5", - "https://files.pythonhosted.org/packages/2b/e7/7c769804eb75e4c4b35e658dba01de1640a351a9653c3d49ca89d16ccc91/cffi-2.0.0-cp39-cp39-musllinux_1_2_x86_64.whl": "89472c9762729b5ae1ad974b777416bfda4ac5642423fa93bd57a09204712322", - "https://files.pythonhosted.org/packages/2c/ea/5f76bce7cf6fcd0ab1a1058b5af899bfbef198bea4d5686da88471ea0336/cffi-2.0.0-cp314-cp314t-macosx_11_0_arm64.whl": "7a66c7204d8869299919db4d5069a82f1561581af12b11b3c9f48c584eb8743d", - "https://files.pythonhosted.org/packages/32/f2/81b63e288295928739d715d00952c8c6034cb6c6a516b17d37e0c8be5600/cffi-2.0.0-cp39-cp39-manylinux2014_s390x.manylinux_2_17_s390x.whl": "cb527a79772e5ef98fb1d700678fe031e353e765d1ca2d409c92263c6d43e09f", - "https://files.pythonhosted.org/packages/33/fa/072dd15ae27fbb4e06b437eb6e944e75b068deb09e2a2826039e49ee2045/cffi-2.0.0-cp310-cp310-win_amd64.whl": "b18a3ed7d5b3bd8d9ef7a8cb226502c6bf8308df1525e1cc676c3680e7176739", - "https://files.pythonhosted.org/packages/36/54/0362578dd2c9e557a28ac77698ed67323ed5b9775ca9d3fe73fe191bb5d8/cffi-2.0.0-cp313-cp313-musllinux_1_2_x86_64.whl": "6d50360be4546678fc1b79ffe7a66265e28667840010348dd69a314145807a1b", - "https://files.pythonhosted.org/packages/37/18/6519e1ee6f5a1e579e04b9ddb6f1676c17368a7aba48299c3759bbc3c8b3/cffi-2.0.0-cp313-cp313-win_amd64.whl": "19f705ada2530c1167abacb171925dd886168931e0a7b78f5bffcae5c6b5be75", - "https://files.pythonhosted.org/packages/3a/c8/15cb9ada8895957ea171c62dc78ff3e99159ee7adb13c0123c001a2546c1/cffi-2.0.0-cp312-cp312-manylinux2014_s390x.manylinux_2_17_s390x.whl": "81afed14892743bbe14dacb9e36d9e0e504cd204e0b165062c488942b9718037", - "https://files.pythonhosted.org/packages/3d/de/38d9726324e127f727b4ecc376bc85e505bfe61ef130eaf3f290c6847dd4/cffi-2.0.0-cp39-cp39-macosx_11_0_arm64.whl": "de8dad4425a6ca6e4e5e297b27b5c824ecc7581910bf9aee86cb6835e6812aa7", - "https://files.pythonhosted.org/packages/3e/61/c768e4d548bfa607abcda77423448df8c471f25dbe64fb2ef6d555eae006/cffi-2.0.0-cp314-cp314t-macosx_10_13_x86_64.whl": "9a67fc9e8eb39039280526379fb3a70023d77caec1852002b4da7e8b270c4dd9", - "https://files.pythonhosted.org/packages/3e/aa/df335faa45b395396fcbc03de2dfcab242cd61a9900e914fe682a59170b1/cffi-2.0.0-cp314-cp314-win32.whl": "087067fa8953339c723661eda6b54bc98c5625757ea62e95eb4898ad5e776e9f", - "https://files.pythonhosted.org/packages/44/64/58f6255b62b101093d5df22dcb752596066c7e89dd725e0afaed242a61be/cffi-2.0.0-cp311-cp311-musllinux_1_2_aarch64.whl": "a05d0c237b3349096d3981b727493e22147f934b20f6f125a3eba8f994bec4a9", - "https://files.pythonhosted.org/packages/47/d9/d83e293854571c877a92da46fdec39158f8d7e68da75bf73581225d28e90/cffi-2.0.0-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.whl": "afb8db5439b81cf9c9d0c80404b60c3cc9c3add93e114dcae767f1477cb53775", - "https://files.pythonhosted.org/packages/49/72/ff2d12dbf21aca1b32a40ed792ee6b40f6dc3a9cf1644bd7ef6e95e0ac5e/cffi-2.0.0-cp310-cp310-musllinux_1_2_x86_64.whl": "8ea985900c5c95ce9db1745f7933eeef5d314f0565b27625d9a10ec9881e1bfb", - "https://files.pythonhosted.org/packages/4a/d2/a6c0296814556c68ee32009d9c2ad4f85f2707cdecfd7727951ec228005d/cffi-2.0.0-cp313-cp313-macosx_11_0_arm64.whl": "45d5e886156860dc35862657e1494b9bae8dfa63bf56796f2fb56e1679fc0bca", - "https://files.pythonhosted.org/packages/4b/8d/a0a47a0c9e413a658623d014e91e74a50cdd2c423f7ccfd44086ef767f90/cffi-2.0.0-cp313-cp313-macosx_10_13_x86_64.whl": "00bdf7acc5f795150faa6957054fbbca2439db2f775ce831222b66f192f03beb", - "https://files.pythonhosted.org/packages/4f/27/6933a8b2562d7bd1fb595074cf99cc81fc3789f6a6c05cdabb46284a3188/cffi-2.0.0-cp310-cp310-manylinux2014_aarch64.manylinux_2_17_aarch64.whl": "3e837e369566884707ddaf85fc1744b47575005c0a229de3327f8f9a20f4efeb", - "https://files.pythonhosted.org/packages/4f/8b/f0e4c441227ba756aafbe78f117485b25bb26b1c059d01f137fa6d14896b/cffi-2.0.0-cp311-cp311-macosx_11_0_arm64.whl": "2de9a304e27f7596cd03d16f1b7c72219bd944e99cc52b84d0145aefb07cbd3c", - "https://files.pythonhosted.org/packages/50/bd/b1a6362b80628111e6653c961f987faa55262b4002fcec42308cad1db680/cffi-2.0.0-cp310-cp310-manylinux1_i686.manylinux2014_i686.manylinux_2_17_i686.manylinux_2_5_i686.whl": "53f77cbe57044e88bbd5ed26ac1d0514d2acf0591dd6bb02a3ae37f76811b80c", - "https://files.pythonhosted.org/packages/50/e1/a969e687fcf9ea58e6e2a928ad5e2dd88cc12f6f0ab477e9971f2309b57c/cffi-2.0.0-cp313-cp313-musllinux_1_2_aarch64.whl": "d9b29c1f0ae438d5ee9acb31cadee00a58c46cc9c0b2f9038c6b0b3470877a8c", - "https://files.pythonhosted.org/packages/54/8f/a1e836f82d8e32a97e6b29cc8f641779181ac7363734f12df27db803ebda/cffi-2.0.0-cp39-cp39-win_amd64.whl": "b882b3df248017dba09d6b16defe9b5c407fe32fc7c65a9c69798e6175601be9", - "https://files.pythonhosted.org/packages/59/dd/27e9fa567a23931c838c6b02d0764611c62290062a6d4e8ff7863daf9730/cffi-2.0.0-cp314-cp314-macosx_11_0_arm64.whl": "c654de545946e0db659b3400168c9ad31b5d29593291482c43e3564effbcee13", - "https://files.pythonhosted.org/packages/60/99/c9dc110974c59cc981b1f5b66e1d8af8af764e00f0293266824d9c4254bc/cffi-2.0.0-cp310-cp310-musllinux_1_2_i686.whl": "e11e82b744887154b182fd3e7e8512418446501191994dbf9c9fc1f32cc8efd5", - "https://files.pythonhosted.org/packages/78/2d/7fa73dfa841b5ac06c7b8855cfc18622132e365f5b81d02230333ff26e9e/cffi-2.0.0-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.whl": "3e17ed538242334bf70832644a32a7aae3d83b57567f9fd60a26257e992b79ba", - "https://files.pythonhosted.org/packages/7b/2b/2b6435f76bfeb6bbf055596976da087377ede68df465419d192acf00c437/cffi-2.0.0-cp312-cp312-win32.whl": "da902562c3e9c550df360bfa53c035b2f241fed6d9aef119048073680ace4a18", - "https://files.pythonhosted.org/packages/84/ef/a7b77c8bdc0f77adc3b46888f1ad54be8f3b7821697a7b89126e829e676a/cffi-2.0.0-cp311-cp311-manylinux2014_s390x.manylinux_2_17_s390x.whl": "9de40a7b0323d889cf8d23d1ef214f565ab154443c42737dfe52ff82cf857664", - "https://files.pythonhosted.org/packages/92/c4/3ce07396253a83250ee98564f8d7e9789fab8e58858f35d07a9a2c78de9f/cffi-2.0.0-cp314-cp314-macosx_10_13_x86_64.whl": "fc33c5141b55ed366cfaad382df24fe7dcbc686de5be719b207bb248e3053dc5", - "https://files.pythonhosted.org/packages/93/d7/516d984057745a6cd96575eea814fe1edd6646ee6efd552fb7b0921dec83/cffi-2.0.0-cp310-cp310-macosx_10_13_x86_64.whl": "0cf2d91ecc3fcc0625c2c530fe004f82c110405f101548512cce44322fa8ac44", - "https://files.pythonhosted.org/packages/95/31/9f7f93ad2f8eff1dbc1c3656d7ca5bfd8fb52c9d786b4dcf19b2d02217fa/cffi-2.0.0-cp312-cp312-win_arm64.whl": "4671d9dd5ec934cb9a73e7ee9676f9362aba54f7f34910956b84d727b0d73fb6", - "https://files.pythonhosted.org/packages/95/5c/1b493356429f9aecfd56bc171285a4c4ac8697f76e9bbbbb105e537853a1/cffi-2.0.0-cp311-cp311-win_arm64.whl": "c6638687455baf640e37344fe26d37c404db8b80d037c3d29f58fe8d1c3b194d", - "https://files.pythonhosted.org/packages/98/29/9b366e70e243eb3d14a5cb488dfd3a0b6b2f1fb001a203f653b93ccfac88/cffi-2.0.0-cp310-cp310-manylinux2014_x86_64.manylinux_2_17_x86_64.whl": "fc7de24befaeae77ba923797c7c87834c73648a05a4bde34b3b7e5588973a453", - "https://files.pythonhosted.org/packages/98/df/0a1755e750013a2081e863e7cd37e0cdd02664372c754e5560099eb7aa44/cffi-2.0.0-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.whl": "c8d3b5532fc71b7a77c09192b4a5a200ea992702734a2e9279a37f2478236f26", - "https://files.pythonhosted.org/packages/9b/13/c92e36358fbcc39cf0962e83223c9522154ee8630e1df7c0b3a39a8124e2/cffi-2.0.0-cp39-cp39-manylinux1_i686.manylinux2014_i686.manylinux_2_17_i686.manylinux_2_5_i686.whl": "4647afc2f90d1ddd33441e5b0e85b16b12ddec4fca55f0d9671fef036ecca27c", - "https://files.pythonhosted.org/packages/9e/84/ad6a0b408daa859246f57c03efd28e5dd1b33c21737c2db84cae8c237aa5/cffi-2.0.0-cp310-cp310-macosx_11_0_arm64.whl": "f73b96c41e3b2adedc34a7356e64c8eb96e03a3782b535e043a986276ce12a49", - "https://files.pythonhosted.org/packages/9f/2c/98ece204b9d35a7366b5b2c6539c350313ca13932143e79dc133ba757104/cffi-2.0.0-cp314-cp314-win_arm64.whl": "dbd5c7a25a7cb98f5ca55d258b103a2054f859a46ae11aaf23134f9cc0d356ad", - "https://files.pythonhosted.org/packages/9f/e0/6cbe77a53acf5acc7c08cc186c9928864bd7c005f9efd0d126884858a5fe/cffi-2.0.0-cp310-cp310-manylinux2014_s390x.manylinux_2_17_s390x.whl": "9332088d75dc3241c702d852d4671613136d90fa6881da7d770a483fd05248b4", - "https://files.pythonhosted.org/packages/a0/1d/ec1a60bd1a10daa292d3cd6bb0b359a81607154fb8165f3ec95fe003b85c/cffi-2.0.0-cp314-cp314t-win32.whl": "1fc9ea04857caf665289b7a75923f2c6ed559b8298a1b8c49e59f7dd95c8481e", - "https://files.pythonhosted.org/packages/a3/ad/5c51c1c7600bdd7ed9a24a203ec255dccdd0ebf4527f7b922a0bde2fb6ed/cffi-2.0.0-cp39-cp39-manylinux2014_ppc64le.manylinux_2_17_ppc64le.whl": "e6e73b9e02893c764e7e8d5bb5ce277f1a009cd5243f8228f75f842bf937c534", - "https://files.pythonhosted.org/packages/a9/f5/a2c23eb03b61a0b8747f211eb716446c826ad66818ddc7810cc2cc19b3f2/cffi-2.0.0-cp313-cp313-manylinux2014_aarch64.manylinux_2_17_aarch64.whl": "d48a880098c96020b02d5a1f7d9251308510ce8858940e6fa99ece33f610838b", - "https://files.pythonhosted.org/packages/aa/d9/6218d78f920dcd7507fc16a766b5ef8f3b913cc7aa938e7fc80b9978d089/cffi-2.0.0-cp39-cp39-win32.whl": "2081580ebb843f759b9f617314a24ed5738c51d2aee65d31e02f6f7a2b97707a", - "https://files.pythonhosted.org/packages/ab/49/fa72cebe2fd8a55fbe14956f9970fe8eb1ac59e5df042f603ef7c8ba0adc/cffi-2.0.0-cp311-cp311-musllinux_1_2_i686.whl": "94698a9c5f91f9d138526b48fe26a199609544591f859c870d477351dc7b2414", - "https://files.pythonhosted.org/packages/ae/3a/dbeec9d1ee0844c679f6bb5d6ad4e9f198b1224f4e7a32825f47f6192b0c/cffi-2.0.0-cp314-cp314t-win_arm64.whl": "0a1527a803f0a659de1af2e1fd700213caba79377e27e4693648c2923da066f9", - "https://files.pythonhosted.org/packages/ae/8f/dc5531155e7070361eb1b7e4c1a9d896d0cb21c49f807a6c03fd63fc877e/cffi-2.0.0-cp311-cp311-win_amd64.whl": "66f011380d0e49ed280c789fbd08ff0d40968ee7b665575489afa95c98196ab5", - "https://files.pythonhosted.org/packages/b0/1e/d22cc63332bd59b06481ceaac49d6c507598642e2230f201649058a7e704/cffi-2.0.0-cp313-cp313-manylinux1_i686.manylinux2014_i686.manylinux_2_17_i686.manylinux_2_5_i686.whl": "07b271772c100085dd28b74fa0cd81c8fb1a3ba18b21e03d7c27f3436a10606b", - "https://files.pythonhosted.org/packages/b1/b7/1200d354378ef52ec227395d95c2576330fd22a869f7a70e88e1447eb234/cffi-2.0.0-cp311-cp311-manylinux1_i686.manylinux2014_i686.manylinux_2_17_i686.manylinux_2_5_i686.whl": "baf5215e0ab74c16e2dd324e8ec067ef59e41125d3eade2b863d294fd5035c92", - "https://files.pythonhosted.org/packages/b4/89/76799151d9c2d2d1ead63c2429da9ea9d7aac304603de0c6e8764e6e8e70/cffi-2.0.0-cp314-cp314-manylinux2014_ppc64le.manylinux_2_17_ppc64le.whl": "12873ca6cb9b0f0d3a0da705d6086fe911591737a59f28b7936bdfed27c0d47c", - "https://files.pythonhosted.org/packages/b6/75/1f2747525e06f53efbd878f4d03bac5b859cbc11c633d0fb81432d98a795/cffi-2.0.0-cp312-cp312-musllinux_1_2_x86_64.whl": "2c8f814d84194c9ea681642fd164267891702542f028a15fc97d4674b6206187", - "https://files.pythonhosted.org/packages/b8/56/6033f5e86e8cc9bb629f0077ba71679508bdf54a9a5e112a3c0b91870332/cffi-2.0.0-cp311-cp311-manylinux2014_aarch64.manylinux_2_17_aarch64.whl": "730cacb21e1bdff3ce90babf007d0a0917cc3e6492f336c2f0134101e0944f93", - "https://files.pythonhosted.org/packages/bb/92/882c2d30831744296ce713f0feb4c1cd30f346ef747b530b5318715cc367/cffi-2.0.0-cp314-cp314-win_amd64.whl": "203a48d1fb583fc7d78a4c6655692963b860a417c0528492a6bc21f1aaefab25", - "https://files.pythonhosted.org/packages/bb/dd/3465b14bb9e24ee24cb88c9e3730f6de63111fffe513492bf8c808a3547e/cffi-2.0.0-cp314-cp314-manylinux2014_s390x.manylinux_2_17_s390x.whl": "d9b97165e8aed9272a6bb17c01e3cc5871a594a446ebedc996e2397a1c1ea8ef", - "https://files.pythonhosted.org/packages/be/b4/c56878d0d1755cf9caa54ba71e5d049479c52f9e4afc230f06822162ab2f/cffi-2.0.0-cp314-cp314t-manylinux2014_aarch64.manylinux_2_17_aarch64.whl": "7cc09976e8b56f8cebd752f7113ad07752461f48a58cbba644139015ac24954c", - "https://files.pythonhosted.org/packages/bf/41/4c1168c74fac325c0c8156f04b6749c8b6a8f405bbf91413ba088359f60d/cffi-2.0.0-cp314-cp314t-win_amd64.whl": "d68b6cef7827e8641e8ef16f4494edda8b36104d79773a334beaa1e3521430f6", - "https://files.pythonhosted.org/packages/c0/cc/08ed5a43f2996a16b462f64a7055c6e962803534924b9b2f1371d8c00b7b/cffi-2.0.0-cp39-cp39-macosx_10_13_x86_64.whl": "fe562eb1a64e67dd297ccc4f5addea2501664954f2692b69a76449ec7913ecbf", - "https://files.pythonhosted.org/packages/c2/95/7a135d52a50dfa7c882ab0ac17e8dc11cec9d55d2c18dda414c051c5e69e/cffi-2.0.0-cp312-cp312-manylinux2014_ppc64le.manylinux_2_17_ppc64le.whl": "1e3a615586f05fc4065a8b22b8152f0c1b00cdbc60596d187c2a74f9e3036e4e", - "https://files.pythonhosted.org/packages/c6/0f/cafacebd4b040e3119dcb32fed8bdef8dfe94da653155f9d0b9dc660166e/cffi-2.0.0-cp314-cp314-musllinux_1_2_x86_64.whl": "38100abb9d1b1435bc4cc340bb4489635dc2f0da7456590877030c9b3d40b0c1", - "https://files.pythonhosted.org/packages/cb/0e/02ceeec9a7d6ee63bb596121c2c8e9b3a9e150936f4fbef6ca1943e6137c/cffi-2.0.0-cp313-cp313-win_arm64.whl": "256f80b80ca3853f90c21b23ee78cd008713787b1b1e93eae9f3d6a7134abd91", - "https://files.pythonhosted.org/packages/cb/1e/a5a1bd6f1fb30f22573f76533de12a00bf274abcdc55c8edab639078abb6/cffi-2.0.0-cp313-cp313-manylinux2014_s390x.manylinux_2_17_s390x.whl": "dd4f05f54a52fb558f1ba9f528228066954fee3ebe629fc1660d874d040ae5a3", - "https://files.pythonhosted.org/packages/d0/44/681604464ed9541673e486521497406fadcc15b5217c3e326b061696899a/cffi-2.0.0-cp314-cp314t-manylinux2014_x86_64.manylinux_2_17_x86_64.whl": "28a3a209b96630bca57cce802da70c266eb08c6e97e5afd61a75611ee6c64592", - "https://files.pythonhosted.org/packages/d5/72/12b5f8d3865bf0f87cf1404d8c374e7487dcf097a1c91c436e72e6badd83/cffi-2.0.0-cp312-cp312-manylinux2014_aarch64.manylinux_2_17_aarch64.whl": "b21e08af67b8a103c71a250401c78d5e0893beff75e28c53c98f4de42f774062", - "https://files.pythonhosted.org/packages/d6/43/0e822876f87ea8a4ef95442c3d766a06a51fc5298823f884ef87aaad168c/cffi-2.0.0-cp314-cp314-manylinux2014_aarch64.manylinux_2_17_aarch64.whl": "24b6f81f1983e6df8db3adc38562c83f7d4a0c36162885ec7f7b77c7dcbec97b", - "https://files.pythonhosted.org/packages/d7/91/500d892b2bf36529a75b77958edfcd5ad8e2ce4064ce2ecfeab2125d72d1/cffi-2.0.0-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.whl": "8941aaadaf67246224cee8c3803777eed332a19d909b47e29c9842ef1e79ac26", - "https://files.pythonhosted.org/packages/d8/19/3c435d727b368ca475fb8742ab97c9cb13a0de600ce86f62eab7fa3eea60/cffi-2.0.0-cp314-cp314t-manylinux2014_s390x.manylinux_2_17_s390x.whl": "b1e74d11748e7e98e2f426ab176d4ed720a64412b6a15054378afdb71e0f37dc", - "https://files.pythonhosted.org/packages/dc/7f/55fecd70f7ece178db2f26128ec41430d8720f2d12ca97bf8f0a628207d5/cffi-2.0.0-cp311-cp311-manylinux2014_ppc64le.manylinux_2_17_ppc64le.whl": "6824f87845e3396029f3820c206e459ccc91760e8fa24422f8b0c3d1731cbec5", - "https://files.pythonhosted.org/packages/df/a2/781b623f57358e360d62cdd7a8c681f074a71d445418a776eef0aadb4ab4/cffi-2.0.0-cp312-cp312-macosx_11_0_arm64.whl": "8eca2a813c1cb7ad4fb74d368c2ffbbb4789d377ee5bb8df98373c2cc0dee76c", - "https://files.pythonhosted.org/packages/e0/0d/eb704606dfe8033e7128df5e90fee946bbcb64a04fcdaa97321309004000/cffi-2.0.0-cp314-cp314t-manylinux2014_ppc64le.manylinux_2_17_ppc64le.whl": "92b68146a71df78564e4ef48af17551a5ddd142e5190cdf2c5624d0c3ff5b2e8", - "https://files.pythonhosted.org/packages/e1/5e/b666bacbbc60fbf415ba9988324a132c9a7a0448a9a8f125074671c0f2c3/cffi-2.0.0-cp314-cp314t-musllinux_1_2_x86_64.whl": "6c6c373cfc5c83a975506110d17457138c8c63016b563cc9ed6e056a82f13ce4", - "https://files.pythonhosted.org/packages/e2/cc/027d7fb82e58c48ea717149b03bcadcbdc293553edb283af792bd4bcbb3f/cffi-2.0.0-cp310-cp310-win32.whl": "1f72fb8906754ac8a2cc3f9f5aaa298070652a0ffae577e0ea9bd480dc3c931a", - "https://files.pythonhosted.org/packages/e8/be/f6424d1dc46b1091ffcc8964fa7c0ab0cd36839dd2761b49c90481a6ba1b/cffi-2.0.0-cp39-cp39-musllinux_1_2_aarch64.whl": "0f6084a0ea23d05d20c3edcda20c3d006f9b6f3fefeac38f59262e10cef47ee2", - "https://files.pythonhosted.org/packages/ea/47/4f61023ea636104d4f16ab488e268b93008c3d0bb76893b1b31db1f96802/cffi-2.0.0-cp312-cp312-macosx_10_13_x86_64.whl": "6d02d6655b0e54f54c4ef0b94eb6be0607b70853c45ce98bd278dc7de718be5d", - "https://files.pythonhosted.org/packages/eb/56/b1ba7935a17738ae8453301356628e8147c79dbb825bcbc73dc7401f9846/cffi-2.0.0.tar.gz": "44d1b5909021139fe36001ae048dbdde8214afa20200eda0f64c068cac5d5529", - "https://files.pythonhosted.org/packages/eb/6d/bf9bda840d5f1dfdbf0feca87fbdb64a918a69bca42cfa0ba7b137c48cb8/cffi-2.0.0-cp313-cp313-win32.whl": "74a03b9698e198d47562765773b4a8309919089150a0bb17d829ad7b44b60d27", - "https://files.pythonhosted.org/packages/f2/7f/e6647792fc5850d634695bc0e6ab4111ae88e89981d35ac269956605feba/cffi-2.0.0-cp313-cp313-manylinux2014_ppc64le.manylinux_2_17_ppc64le.whl": "f93fd8e5c8c0a4aa1f424d6173f14a892044054871c771f8566e4008eaa359d2", - "https://files.pythonhosted.org/packages/f7/e0/dda537c2309817edf60109e39265f24f24aa7f050767e22c98c53fe7f48b/cffi-2.0.0-cp39-cp39-musllinux_1_2_i686.whl": "1cd13c99ce269b3ed80b417dcd591415d3372bcac067009b6e0f59c7d4015e65", - "https://files.pythonhosted.org/packages/f8/ed/13bd4418627013bec4ed6e54283b1959cf6db888048c7cf4b4c3b5b36002/cffi-2.0.0-cp312-cp312-win_amd64.whl": "da68248800ad6320861f129cd9c1bf96ca849a2771a59e0344e88681905916f5", - "https://files.pythonhosted.org/packages/ff/df/a4f0fbd47331ceeba3d37c2e51e9dfc9722498becbeec2bd8bc856c9538a/cffi-2.0.0-cp312-cp312-manylinux1_i686.manylinux2014_i686.manylinux_2_17_i686.manylinux_2_5_i686.whl": "21d1152871b019407d8ac3985f6775c079416c282e431a4da6afe7aefd2bccbe" + "https://files.pythonhosted.org/packages/04/8c/b925975448cf20634a9fbd5efceb807219db452653648d2897c0989cab2d/cffi-2.1.0-cp311-cp311-musllinux_1_2_x86_64.whl": "89095c1968b4ba8285840e131bf2891b09ae137fe2146905acae0354fbce1b5e", + "https://files.pythonhosted.org/packages/05/ef/6cd4f8c671517162379dc79cfae5aea9106bc38abb89628d5c16adf6a838/cffi-2.1.0-cp315-cp315-win_arm64.whl": "8d35c139744adb3e727cd51b1a18324bbe44b8bd41bf8322bca4d41289f48eda", + "https://files.pythonhosted.org/packages/0f/6f/ade5ce9863a57992a6ea3d0d10d7e29b8749fc127204b3d493d667b2815f/cffi-2.1.0-cp314-cp314-win32.whl": "1854b724d00f6654c742097d5387569021be12d3a0f770eae1df8f8acfcc6acd", + "https://files.pythonhosted.org/packages/11/b6/12fc55092817a5faa26fb8c40c7f9d662e11a46ee248c137aafc42517d92/cffi-2.1.0-cp315-cp315t-macosx_10_15_x86_64.whl": "f9912624a0c0b834b7520d7769b3644453aabc0a7e1c839da7359f050750e9bc", + "https://files.pythonhosted.org/packages/14/d0/117dcd9209255ad8571fbc8c92ef32593a1d294dcec91ddc4e4db50606f2/cffi-2.1.0-cp314-cp314t-manylinux2014_aarch64.manylinux_2_17_aarch64.whl": "eb4e8997a49aa2c08a3e43c9045d224448b8941d88e7ac163c7d383e560cbf98", + "https://files.pythonhosted.org/packages/14/f0/134c00ce0779ec86dea2aa1aac69339c2741a8045072676763512363a2ea/cffi-2.1.0-cp314-cp314t-macosx_10_15_x86_64.whl": "7ea6b3e2c4250ff1de21c630fe72d0f63eb95c2c32ffbf64a358cf4a8836d714", + "https://files.pythonhosted.org/packages/19/e5/d3cc82a4a0be7902af279c04181ad038449c096734464a5ae1de3e1401bd/cffi-2.1.0-cp315-cp315t-manylinux2014_x86_64.manylinux_2_17_x86_64.whl": "0611e7ebf90573a535ebdc33ae9da222d037853983e13359f580fab781ca017f", + "https://files.pythonhosted.org/packages/1b/dc/5620cf930688be01f2d673804291de757a934c90b946dbdc3d84130c2ea4/cffi-2.1.0-cp312-cp312-manylinux2014_aarch64.manylinux_2_17_aarch64.whl": "b6422532152adf4e59b110cb2808cee7a033800952f5c036b4af047ee43199e7", + "https://files.pythonhosted.org/packages/1e/85/990925db5df586ec90beb97529c853497e7f85ba0234830447faf41c3057/cffi-2.1.0-cp312-cp312-macosx_10_15_x86_64.whl": "df2b82571a1b30f58a87bf4e5a9e78d2b1eff6c6ce8fd3aa3757221f93f0863f", + "https://files.pythonhosted.org/packages/20/71/7c8372d30e42415602ed9f268f7cfd66f1b855fed881ecd168bcb45dbc0b/cffi-2.1.0-cp314-cp314-macosx_10_15_x86_64.whl": "1ff3456eab0d889592d1936d6125bbfbc7ae4d3354a700f8bd80450a66445d4d", + "https://files.pythonhosted.org/packages/22/d7/1a74539db16d8bfd839ff1515948948efbb162e574650fd3d846896eea95/cffi-2.1.0-cp311-cp311-manylinux2014_aarch64.manylinux_2_17_aarch64.whl": "88023dfe18799507b73f1dbb0d14326a17465de1bc9c9c7655c22845e9ddc3a2", + "https://files.pythonhosted.org/packages/22/f0/a2fc43084c0433caf7f461bccc013e28f848d04ee1c5ed7fce71423cf4d9/cffi-2.1.0-cp311-cp311-musllinux_1_2_i686.whl": "7762faa47e8ff7eb80bd261d9a7d8eea2d8baa69de5e95b70c1f338bbe712f02", + "https://files.pythonhosted.org/packages/28/3b/fad54de07260b93ddeef4b96d0131d57ea900675df1d410ae1deee52d7a6/cffi-2.1.0-cp314-cp314t-manylinux2014_s390x.manylinux_2_17_s390x.whl": "33eb1ad83ebe8f313e0df035c406227d55a79456704a863fad9842136af5ad7d", + "https://files.pythonhosted.org/packages/28/ed/c127d3ac36e899c965e3361357c3befacd6578c03f40125183e41c3b219e/cffi-2.1.0-cp314-cp314-manylinux2014_s390x.manylinux_2_17_s390x.whl": "6ca4919c6e4f89aa99c42510b42cf54596892c00b3f9077f6bdd1505e24b9c8d", + "https://files.pythonhosted.org/packages/2c/0e/fac738d73728c6cea2a88a2883dca54892496cbba88a1dc1f2909cb8a6f5/cffi-2.1.0-cp315-cp315-manylinux2014_s390x.manylinux_2_17_s390x.whl": "2b71d409cccee78310ab5dec549aed052aaea483346e282c7b02362596e01bb0", + "https://files.pythonhosted.org/packages/2c/d8/772b8259bf75749adffb1c546828978381fb516f60cf701f6c83daf60c85/cffi-2.1.0-cp315-cp315-win32.whl": "0a42c688d19fca6e095a53c6a6e2295a5b050a8b289f109adab02a9e61a25de6", + "https://files.pythonhosted.org/packages/2e/1a/cc6ae6c2913a03aab8898eee57963cf1035b8df5872ed8b9115fcc7e2be8/cffi-2.1.0-cp314-cp314-win_arm64.whl": "7d28dff1db6764108bc30788d85d61c876beff416d9a49cb9dd7c5a9f34f5804", + "https://files.pythonhosted.org/packages/2e/d2/065fcae1c73979fac8e054462478d0ff8a29c40cdc2ed7ea5676a061df53/cffi-2.1.0-cp314-cp314-manylinux2014_aarch64.manylinux_2_17_aarch64.whl": "276f20fffd7b396e12516ba8edf9509210ac248cbbc5acbc39cd512f9f59ebe6", + "https://files.pythonhosted.org/packages/2f/dd/afa2191fc6d57fedd26e5844a2fe2fcc0bbfa00961bbaa5a41e4921e7cca/cffi-2.1.0-cp315-cp315-win_amd64.whl": "bccbbb5ee76a61f9d99b5bf3846a51d7fca4b6a732fe46f89295610edaf41853", + "https://files.pythonhosted.org/packages/38/37/04f54b8e63a02f3d908332c9effbf8c366167c6f733ed8a3d4f79b7e2a1e/cffi-2.1.0-cp313-cp313-musllinux_1_2_aarch64.whl": "961be50688f7fba2fa65f63712d3b9b341a22311f5253460ce933f52f0de1c8c", + "https://files.pythonhosted.org/packages/38/66/04781a77b411f0bb5b234d62c1814754ab75ebe455ccff1b08e8d7aae98f/cffi-2.1.0-cp310-cp310-manylinux2014_x86_64.manylinux_2_17_x86_64.whl": "4d433a51f1870e43a13b6732f92aaf540ff77c2015097c78556f75a2d6c030e0", + "https://files.pythonhosted.org/packages/3a/a6/e879bb68cc23a2bc9ba8f4b7d8019f0c2694bad2ab6c4a3701d429439f58/cffi-2.1.0-cp313-cp313-manylinux2014_aarch64.manylinux_2_17_aarch64.whl": "ff067a8d8d880e7809e4ac88eb009bb848870115317b306666502ccad30b147f", + "https://files.pythonhosted.org/packages/3b/30/c806937ed5e4c2c7ac30d9d6b76b5dc57ff8b75d83800d9bb11a8253cf2a/cffi-2.1.0-cp310-cp310-manylinux2014_aarch64.manylinux_2_17_aarch64.whl": "a016194dbe13d14ee9556e734b772d8d67b947092b268d757fd4290e3ba2dfc2", + "https://files.pythonhosted.org/packages/41/aa/3c1409cdd26094efacd1c36c66e0a6eb9d4296e4fd4f9901b8b2042f4323/cffi-2.1.0-cp310-cp310-musllinux_1_2_i686.whl": "c5f5df567f6eb216de69be06ce55c8b714090fae02b18a3b40da8163b8c5fa9c", + "https://files.pythonhosted.org/packages/41/de/92b9eeed4ae4a21d6fd9b2a2c8505cbed573299902ea73981cc13f7ff62c/cffi-2.1.0-cp314-cp314-win_amd64.whl": "1b96bfe2c4bd825681b7d311ad6d9b7280a091f43e8f63da5729638083cd3bfb", + "https://files.pythonhosted.org/packages/45/ca/f91641185cdd90c36d317a9dc7f85e88ef8682d8b300977baff5e23c35d8/cffi-2.1.0-cp310-cp310-manylinux2014_s390x.manylinux_2_17_s390x.whl": "19c54ac121cad98450b4896fa9a43ee0180d57bc4bc911a33db6cab1efab6cd3", + "https://files.pythonhosted.org/packages/4b/92/e7bb136ad6b5352603732cf907ef862ca103f20f2031c1735a46300c20c9/cffi-2.1.0-cp312-cp312-macosx_11_0_arm64.whl": "78474632761faa0fb96f30b1c928c84ebcf68713cbb80d15bab09dfe61640fde", + "https://files.pythonhosted.org/packages/4b/a4/77b53abbf7a1e0beb9637edbef2a94d15f9c822f591e85d439ffd91519a6/cffi-2.1.0-cp312-cp312-manylinux2014_ppc64le.manylinux_2_17_ppc64le.whl": "46b1c8db8f6122420f32d02fffb924c2fe9bc772d228c7c711748fff56aabb2b", + "https://files.pythonhosted.org/packages/50/d8/3b86aba791cb610d24e8a3e1b2cd529e71fa15096b04e4d4e360049d4a4c/cffi-2.1.0-cp314-cp314t-macosx_11_0_arm64.whl": "6af371f3767faeffc6ac1ef57cdfd25844403e9d3f476c5537caee499de96376", + "https://files.pythonhosted.org/packages/55/c7/8c8c50cb11c6750051daf12164098a9a6f027ac4356967fd4d800a07f242/cffi-2.1.0-cp315-cp315-ios_13_0_arm64_iphoneos.whl": "2e9dabb9abcb7ad15938c7196ad5c1718a4e6d33cc79b4c0209bdb64c4a54a5c", + "https://files.pythonhosted.org/packages/57/5f/ff100cae70ebe9d8df1c01a00e510e45d9adb5c1fdda84791b199141de97/cffi-2.1.0.tar.gz": "efc1cdd798b1aaf39b4610bba7aad28c9bea9b910f25c784ccf9ec1fa719d1f9", + "https://files.pythonhosted.org/packages/58/0c/f528df19cc94b675087324d4760d9e6d5bfae97d6217aa4fac43de4f5fcc/cffi-2.1.0-cp312-cp312-manylinux2014_s390x.manylinux_2_17_s390x.whl": "d9fafc5aa2e2a39aaf7f8cc0c1f044a9b07fca12e558dca53a3cc5c654ad67a7", + "https://files.pythonhosted.org/packages/58/85/7ae00d5c8dd6266f4e944c3db630f3c5c9a98b61d469c714d848b1d8138a/cffi-2.1.0-cp313-cp313-ios_13_0_arm64_iphonesimulator.whl": "a95b05f9baf29b91171b3a8bd2020b028835243e7b0ff6bb23e2a3c228518b1b", + "https://files.pythonhosted.org/packages/5a/02/d5e6c43ea85c41bda2a184a3418f195fe7cf602967a8d2b94e085b83deef/cffi-2.1.0-cp315-cp315-musllinux_1_2_x86_64.whl": "af5e2915d41fe6c961694d7bfdc8562942638200f3ce2765dfb8b745cf997629", + "https://files.pythonhosted.org/packages/5a/47/59eb7975cb0e4ef0afa764ea945b29a5bb4537a9f771cb7d6c8a5dd74c95/cffi-2.1.0-cp313-cp313-win_amd64.whl": "8e74a6135550c4748af665b1b1118b6aab33b1fc6a16f9aff630af107c3b4512", + "https://files.pythonhosted.org/packages/5a/67/9e6e09409336d9e515c58367e7cfcf4f89df06ad25252675595a58eb59d5/cffi-2.1.0-cp315-cp315t-manylinux2014_s390x.manylinux_2_17_s390x.whl": "762f99479dcb369f60ab9017ad4ab97a36a1dd7c1ee5a3b15db0f4b8659120cd", + "https://files.pythonhosted.org/packages/5a/af/34fee85c48f8d94efc8597bc09470c9dd274c145f1c12e0fbc6ab6d38d74/cffi-2.1.0-cp313-cp313-win_arm64.whl": "2282cd5e38aa8accd03e99d1256af8411c84cdbee6a89d841b563fdbd1f3e50f", + "https://files.pythonhosted.org/packages/5d/7c/b7379a5704c79eda57ce075869ba70a0368d1c850f803b3c0d078d39dcaf/cffi-2.1.0-cp315-cp315-musllinux_1_2_aarch64.whl": "8f9ec95b8a043d3dfbc74d9abc6f7baf524dd27a8dc160b0a32ff9cdab650c28", + "https://files.pythonhosted.org/packages/62/f2/c9522a81c32132799a1972c39f5c5f8b4c8b9f00488a23feaa6c06f07741/cffi-2.1.0-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.whl": "1e9f50d192a3e525b15a75ab5114e442d83d657b7ec29182a991bc9a88fd3a66", + "https://files.pythonhosted.org/packages/65/68/9f3ef890cf3c6ab97bd531c5677f67613d302165d16f8142b2811782a614/cffi-2.1.0-cp311-cp311-manylinux1_i686.manylinux2014_i686.manylinux_2_17_i686.manylinux_2_5_i686.whl": "30b65779d598c370374fefabf138d456fd6f3216bfa7bedfab1ba82025b0cd93", + "https://files.pythonhosted.org/packages/68/5a/e536c528bc8057496c360c0978559a2dc45653f89dd6151078aa7d8fca1a/cffi-2.1.0-cp315-cp315t-win32.whl": "cb96698e3c7413d906ce83f8ffd245ec1bd94707541f299d0ce4d6b0193e982b", + "https://files.pythonhosted.org/packages/69/aa/24580a278de21fd7322635556334d9b535f1cbc00b0a3919447cdf464c65/cffi-2.1.0-cp310-cp310-macosx_11_0_arm64.whl": "164bff1657b2a74f0b6d54e11c9b375bc97b931f2ca9c43fcf875838da1570dd", + "https://files.pythonhosted.org/packages/6c/d0/47e338384ab6b1004241002fa616301020cea4fc95f283506565d252f276/cffi-2.1.0-cp314-cp314t-musllinux_1_2_aarch64.whl": "c16914df9fb7f500e440e6875fa23ff5e0b31db01fa9c06af98d59a91f0dc2e4", + "https://files.pythonhosted.org/packages/6e/28/bd53988b9833e8f8ad539d26f4c07a6b3f6bcb1e9e02e7ca038250b3428d/cffi-2.1.0-cp312-cp312-musllinux_1_2_aarch64.whl": "98fff996e983a36d3aa2eca83af40c5821202e7e6f32d13ae94e3d2286f10cfe", + "https://files.pythonhosted.org/packages/6f/08/f2e7d62c460faae0926f2d6e423694aa409ced3bc1fe2927a0a6e5f05416/cffi-2.1.0-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.whl": "799416bae98336e400981ff6e532d67d5c709cfb30afb79865a1315f94b0e224", + "https://files.pythonhosted.org/packages/70/25/65bd5b58ea4bfdfc15cde02cb5365f89ef8ab8b2adfb8fe5c4bd4233382f/cffi-2.1.0-cp314-cp314t-musllinux_1_2_x86_64.whl": "5ecbd0499275d57506d397eebe1981cee87b47fcd9ef5c22cab7ed7644a39a94", + "https://files.pythonhosted.org/packages/70/b6/9003c33a3e7d2c1306f5962e646457dcfe5a8cd8fce6bbe02d7af25db783/cffi-2.1.0-cp310-cp310-win32.whl": "9d72af0cf10a76a600a9690078fe31c63b9588c8e86bf9fd353f713c84b5db0f", + "https://files.pythonhosted.org/packages/79/99/0d0fd37f055224085f42bbb2c022d002e17dde4a97972822327b07d84101/cffi-2.1.0-cp312-cp312-musllinux_1_2_x86_64.whl": "379de10ce1ba048b1448599d1b37b24caee16309d1ac98d3982fc997f768700b", + "https://files.pythonhosted.org/packages/7d/95/8de304305cd9204974b0ca051b86d307cafca13aa575a0ef1b44d92c0d8c/cffi-2.1.0-cp314-cp314-ios_13_0_arm64_iphonesimulator.whl": "702c436735fbe99d59ada02a1f65cfc0d31c0ee8b7290912f8fbc5cd1e4b16c3", + "https://files.pythonhosted.org/packages/84/4c/82f132cb4418ee6d953d982b19191e87e2a6372c8a4ce36e50b69d6ade4a/cffi-2.1.0-cp313-cp313-macosx_11_0_arm64.whl": "716ff8ec22f20b4d988b12884086bcef0fc99737043e503f7a3935a6be99b1ea", + "https://files.pythonhosted.org/packages/88/a9/02cae418ec4beb282ace11958d9d4737793439d561fadc7e6d56f2e2b354/cffi-2.1.0-cp310-cp310-manylinux1_i686.manylinux2014_i686.manylinux_2_17_i686.manylinux_2_5_i686.whl": "c941bb58d5a6e1c3892d86e42927ed6c180302f07e6d395d08c416e594b98b46", + "https://files.pythonhosted.org/packages/88/f6/01890cfd63c08f8eb96a8319b0443690197d240a8bd6346048cf7bde9190/cffi-2.1.0-cp313-cp313-manylinux2014_ppc64le.manylinux_2_17_ppc64le.whl": "3b926723c13eba9f81d2ef3820d63aeceec3b2d4639906047bf675cb8a7a500d", + "https://files.pythonhosted.org/packages/8a/26/710688310447531c7a22f857c7f79d9855ec18b03e04494ced723fb37e2f/cffi-2.1.0-cp310-cp310-win_amd64.whl": "fb62edb5bb52cca65fab91a63afa7561607120d26090a7e8fda6fb9f064726da", + "https://files.pythonhosted.org/packages/8b/31/e115c985105dd7ffb32444505f18ceb874bb42d992af05d5dced7ecf1980/cffi-2.1.0-cp315-cp315t-manylinux2014_ppc64le.manylinux_2_17_ppc64le.whl": "3681e031db29958a7502f5c0c9d6bbc4c36cb20f7b104086fa642d1799631ff8", + "https://files.pythonhosted.org/packages/8c/e9/45c3a76ad8d43ad9261f4c95436da61128d3ca545d72b9612c0ab5be0b1c/cffi-2.1.0-cp313-cp313-macosx_10_15_x86_64.whl": "15faec4adfff450819f3aee0e2e02c812de6edb88203aa58807955db2003472a", + "https://files.pythonhosted.org/packages/8d/2e/cdac88979f295fde5daa69622c7d2111e56e7ceb94f211357fbe452339e4/cffi-2.1.0-cp315-cp315t-macosx_11_0_arm64.whl": "df92f2aba50eb4d96718b68ef76f2e57a57b54f2fa62333496d16c6d585a85ca", + "https://files.pythonhosted.org/packages/96/88/a996879e2eeccb815f6e3a5967b12a308257412acec882039d386bd2aa7b/cffi-2.1.0-cp313-cp313-ios_13_0_arm64_iphoneos.whl": "10537b1df4967ca26d21e5072d7d54188354483b91dc75058968d3f0cf13fbda", + "https://files.pythonhosted.org/packages/99/e2/67680bf19a6b60d2bb7ff83baefa2a4c3d2d7dc0f3277034b802e1fc504c/cffi-2.1.0-cp315-cp315-ios_13_0_arm64_iphonesimulator.whl": "37f525a7e7e50c017fdebe58b787be310ad59357ae43a053943a6e1a6c526001", + "https://files.pythonhosted.org/packages/9e/4e/e8d7cb5783f1841a3c8fb3a7735838d7484d08ec08c9f984b14cac1ac0e9/cffi-2.1.0-cp311-cp311-win_arm64.whl": "35aaea0c7ee0e58a5cd8c2fd1a48fdf7ece0d2699b7ecdda08194e9ce5dd9b3d", + "https://files.pythonhosted.org/packages/a0/17/1073b53b68c9b5ca6914adf5f8bf55aacc2d3be102418c90700160ea8605/cffi-2.1.0-cp315-cp315t-win_arm64.whl": "cbb7640ce37159548d2147b5b8c241f962143d4c71231431820783f4dc78f210", + "https://files.pythonhosted.org/packages/a0/1c/4ed5a0e5bdca6cbc275556de3328dd1b76fd0c11cc13c88fe66d1d8715f2/cffi-2.1.0-cp313-cp313-manylinux1_i686.manylinux2014_i686.manylinux_2_17_i686.manylinux_2_5_i686.whl": "63960549e4f8dc41e31accb97b975abaecfc44c03e396c093a6436763c2ea7db", + "https://files.pythonhosted.org/packages/a2/a5/d4fe77b589e5e82d43ebc809bf2e6474afe8e48e32ea050b9357645b6471/cffi-2.1.0-cp311-cp311-musllinux_1_2_aarch64.whl": "9d8272c0e483b024e1b9ad029821470ed8ec65631dbd90217469da0e7cd89f1c", + "https://files.pythonhosted.org/packages/a6/cf/2b684132056f438567b61e19d690dd31cd0921ace051e0a458be6074369e/cffi-2.1.0-cp313-cp313-manylinux2014_s390x.manylinux_2_17_s390x.whl": "47ff3a8bfd8cb9da1af7524b965127095055654c177fcfc7578debcb015eecd0", + "https://files.pythonhosted.org/packages/a8/eb/f636456ff21a83fc13c032b58cc5dde061691546ac79efa284b2989b7982/cffi-2.1.0-cp312-cp312-win_amd64.whl": "c97f080ea627e2863524c5af3836e2270b5f5dfff1f104392b959f8df0c5d384", + "https://files.pythonhosted.org/packages/a9/d6/c72eecca433cd3e681c65ed313ab4835d9d4a379704d0f628a6a05f51c2e/cffi-2.1.0-cp313-cp313-musllinux_1_2_x86_64.whl": "bf5c6cf48238b0eb4c086978c492ad1cbc22373fc5b2d7353b3a598ce6db887a", + "https://files.pythonhosted.org/packages/b0/80/c138990aa2a70b1a269f6e06348729836d733d6f970867943f61d367f8cc/cffi-2.1.0-cp312-cp312-win32.whl": "9b8f0f26ca4e7513c534d351eca551947d053fac438f2a04ac96d882909b0d3a", + "https://files.pythonhosted.org/packages/b3/c1/6dbd291ee2ae5a50a034aa057207081f545923bbf15dad4511e985aafff5/cffi-2.1.0-cp314-cp314-musllinux_1_2_x86_64.whl": "dbf7c7a88e2bac086f06d14577332760bdeecc42bdec8ac4077f6260557d9326", + "https://files.pythonhosted.org/packages/b5/9f/d4dc66ca651eb1145a133314cda721abf13cfac3d28c4a0402263ae6ad75/cffi-2.1.0-cp315-cp315t-musllinux_1_2_x86_64.whl": "ba00f661f8ba35d075c937174e27c2c421cec3942fd2e0ea3e66996757c0fdd9", + "https://files.pythonhosted.org/packages/b6/3d/f20f8b886b254e3ad10e15cd4186d3aed49f3e6a35ab37aab9f8f25f7c03/cffi-2.1.0-cp314-cp314t-manylinux2014_ppc64le.manylinux_2_17_ppc64le.whl": "bf01d8c84cbea96b944c73b22182e6c7c432b3475632b8111dbfdc95ddad6e13", + "https://files.pythonhosted.org/packages/b9/26/d00496b22de4d4228f32dde94ad996f350c8aad676d63bcca0743c8dea4d/cffi-2.1.0-cp314-cp314t-win_amd64.whl": "0582a58f3051372229ca8e7f5f589f9e5632678208d8636fea3676711fdf7fe5", + "https://files.pythonhosted.org/packages/b9/65/b434abc97ce7cecc2c640fde160507c0ecc7e21544b483ba3325d2e2ea17/cffi-2.1.0-cp315-cp315t-musllinux_1_2_aarch64.whl": "86cf8755a791f72c85dc287128cc62d4f24d392e3f1e15837245623f4a33cccc", + "https://files.pythonhosted.org/packages/ba/b0/e131a9c41f10607926278453d9596163594fe1c4ebc46efe3b5e5b34eb84/cffi-2.1.0-cp311-cp311-manylinux2014_s390x.manylinux_2_17_s390x.whl": "a5781494d4d400a3f47f8f1da94b324f6e6b440a53387774002890a2a2f4b50f", + "https://files.pythonhosted.org/packages/c0/e9/6d7724983b3d5a0908dbf74f64038ade77c18646ff6636ec7894fd392ce1/cffi-2.1.0-cp310-cp310-macosx_10_15_x86_64.whl": "b65f590ef2a44640f9a05dbb548a429b4ade77913ce683ac8b1480777658a6c0", + "https://files.pythonhosted.org/packages/c3/c0/d1ec30ffb370f748f2fb54425972bfef9871e0132e82fb589c46b6676049/cffi-2.1.0-cp312-cp312-manylinux1_i686.manylinux2014_i686.manylinux_2_17_i686.manylinux_2_5_i686.whl": "5972433ad71a9e46516584ef60a0fda12d9dc459938d1539c3ddecf9bdc1368d", + "https://files.pythonhosted.org/packages/c6/4b/e706f67279140f92939da3475ad610df18bfd52d50f14953a8e5fede71d5/cffi-2.1.0-cp313-cp313-win32.whl": "db3eb7d46527159a878ec3460e9d40615bc25ba337d477db681aea6e4f05c5d2", + "https://files.pythonhosted.org/packages/cc/82/3d5c705acb7abbba9bbd7d79b8e62e0f25b6120eb7ae6ac49f1b721722fe/cffi-2.1.0-cp314-cp314t-manylinux2014_x86_64.manylinux_2_17_x86_64.whl": "ac0f1a2d0cfa7eea3f2aaf006ab6e70e8feeb16b75d65b7e5939982ca2f11056", + "https://files.pythonhosted.org/packages/cc/d7/97d3136f81db489ec8d1d67748c110d6c994268fd7528014aa9f2b085e4e/cffi-2.1.0-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.whl": "d53d10f7da99ae46f7373b9150393e9c5eab9b224909982b43832668de4779f5", + "https://files.pythonhosted.org/packages/d0/9a/bb1d5ed9c3fcae158e9f6391bf309c95d98c2ac37ed56573228471d0af5e/cffi-2.1.0-cp310-cp310-musllinux_1_2_aarch64.whl": "3d7f118b5adbfdfead90c25822690b02bc8074fba949bb7858bec4ebd55adb43", + "https://files.pythonhosted.org/packages/d3/0b/0ffe8b82d3875bced5fa1e7986a7a46b748262a40ab7f60b475eb9fb1bb3/cffi-2.1.0-cp315-cp315t-win_amd64.whl": "f146d154428a2523f9cc7936c02353c2459b8f6cf07d3cd1ee1c0a611109c5d5", + "https://files.pythonhosted.org/packages/d3/27/93195977168ee63aed233a1a0993a2178798654d1f4bddcdd321d6fd3b21/cffi-2.1.0-cp314-cp314-musllinux_1_2_aarch64.whl": "c351efb95e832a853a29361675f33a7ce53de1a109cd73fd47af0712213aa4ce", + "https://files.pythonhosted.org/packages/d3/67/85c89a59ba36a671e79638f44d466749f08179266a57e4f2ffdf92174072/cffi-2.1.0-cp311-cp311-macosx_10_15_x86_64.whl": "02cb7ff33ded4f1532476731f89ede53e2e488a8e6205515a82144246ffa7dcc", + "https://files.pythonhosted.org/packages/d5/dd/0c7dbf815a579ff005008a2d815a55d6bb047c349eef536d9dc53d3f0a8d/cffi-2.1.0-cp314-cp314t-win_arm64.whl": "510aeeeac94811b138077451da1fb18b308a5feab47dd2b603af55804155e1c8", + "https://files.pythonhosted.org/packages/d6/5c/584e626835f0375c928176c04137c96927165cb8733cdb3150ec04e5ee5e/cffi-2.1.0-cp314-cp314-macosx_11_0_arm64.whl": "c4165821e131d6d4ca444347c2b694e2311bcfa3fe5a861cc72968f28867beac", + "https://files.pythonhosted.org/packages/d8/f0/81478e482afa03f6d18dc8f2afb5edc45b3080853b634b5ed91961be0998/cffi-2.1.0-cp314-cp314-ios_13_0_arm64_iphoneos.whl": "d2117334c3af3bdcb9a88522b844a2bdb5efdc4f71c6c822df55486ae1c3347a", + "https://files.pythonhosted.org/packages/dc/78/aa01ac599a8a4322533d45a1f9bc93b338276d2d59dabbe7c6d92a775c81/cffi-2.1.0-cp314-cp314t-win32.whl": "7d034dcffa09e9a46c93fa3a3be402096cb5354ac6e41ab8e5cc9cd8b642ad76", + "https://files.pythonhosted.org/packages/dd/2c/400ea43e721727dca8a65c4521390e9196757caba4a45643acb2b63271b8/cffi-2.1.0-cp312-cp312-win_arm64.whl": "6d194185eabd279f1c05ebe3504265ddfc5ad2b58d0714f7db9f01da592e9eb6", + "https://files.pythonhosted.org/packages/e0/27/1d0b408497e41a74795af122d7b603c418c5fed0171450f899afd04e594f/cffi-2.1.0-cp315-cp315t-manylinux2014_aarch64.manylinux_2_17_aarch64.whl": "0520e1f4c35f44e209cbbb421b67eec42e6a157f59444dfb6058874ff3610e5d", + "https://files.pythonhosted.org/packages/e5/4b/1f4c36ab273980d7aa75bb126ea4f8971f24a96108acad3a0a084028c57b/cffi-2.1.0-cp315-cp315-macosx_11_0_arm64.whl": "cdf2448aab5f661c9315308ec8b93f4e8a1a67a3c733f8631067a2b67d5913dc", + "https://files.pythonhosted.org/packages/e6/3f/0b04a700dd64f465c93020253a793a82c9b4dff9961f48facd0df945d9b8/cffi-2.1.0-cp315-cp315-manylinux2014_x86_64.manylinux_2_17_x86_64.whl": "7d3538f9c0e50670f4deb93dbb696576e60590369cae2faf7de681e597a8a1f1", + "https://files.pythonhosted.org/packages/ea/dd/e3b0baa2d3d6a857ac72b7efbf18e32e487c9cdafcc13049ad765495b15e/cffi-2.1.0-cp311-cp311-macosx_11_0_arm64.whl": "f5bce581e6b8c235e566a14768a943b172ada3ed73537bb0c0be1edee312d4e7", + "https://files.pythonhosted.org/packages/eb/d8/df4543cc087245044ed02ef3ad8e0a26619d0075ac7a77a12dc81177851b/cffi-2.1.0-cp315-cp315-manylinux2014_ppc64le.manylinux_2_17_ppc64le.whl": "6274dcb2d15cef48daa73ed1be5a40d501d74dccd0cd6db364776d12cb6ba022", + "https://files.pythonhosted.org/packages/eb/da/5c4918a2d61d86fa927d716cb3d8e4626ef8dc8f605a599d32f33897f59a/cffi-2.1.0-cp311-cp311-win32.whl": "64c753a0f87a256020004f37a1c8c02c480e725f910f0b2a0f3f07debd1b2479", + "https://files.pythonhosted.org/packages/ec/d1/9a5b7169499e8e8d8e636de70b97ac7c9447104d2ff1a2cd94790cea5162/cffi-2.1.0-cp311-cp311-manylinux2014_ppc64le.manylinux_2_17_ppc64le.whl": "0a96b74cda968eebbad56d973efe5098974f0a9fb323865bf99ea1fd24e3e64c", + "https://files.pythonhosted.org/packages/ed/a5/e8bbb1ce5b3ac2f53ad6a10bde44318a5a8d99d4f4a000d44a6e39aeb3e4/cffi-2.1.0-cp314-cp314-manylinux2014_ppc64le.manylinux_2_17_ppc64le.whl": "7d5980a3433d4b71a5e120f9dd551403d7824e31e2e67124fe2769c404c06913", + "https://files.pythonhosted.org/packages/ed/da/4bbe583a3b3a5c8c60892124fe17f3fa3656523faf0d3484eae90f091853/cffi-2.1.0-cp315-cp315-macosx_10_15_x86_64.whl": "95f2954c2c9473d892eca6e0409f3568b37ab62a8eedb122461f73cc273476e3", + "https://files.pythonhosted.org/packages/ef/c3/ad299dc38f3583f8d916b299f028af418a9ec98bc695fcbebeae7420691c/cffi-2.1.0-cp315-cp315-manylinux2014_aarch64.manylinux_2_17_aarch64.whl": "90bec57cf82089383bd06a605b3eb8daebf7e5a668520beaf6e327a83a947699", + "https://files.pythonhosted.org/packages/f9/c8/6c2de1d55cf35ef8b92885d5ef280790f0fb9634d87ea1cc315176aecd61/cffi-2.1.0-cp311-cp311-win_amd64.whl": "4f26194e3d95e06501b942642855aed4f953d55e95d7d01b7c4483db3ecff458", + "https://files.pythonhosted.org/packages/f9/cf/398272b8bbfd58aa314fda5a7f1cdbb26d1d78ae324a11211521315dd1f0/cffi-2.1.0-cp310-cp310-manylinux2014_ppc64le.manylinux_2_17_ppc64le.whl": "03e9810d18c646077e501f661b682fbf5dee4676048527ca3cffe66faa9960dd", + "https://files.pythonhosted.org/packages/fa/75/74dfb7c3fc6ebbd408038476bd4c1d7e925c62614e7b9c534ecc34218288/cffi-2.1.0-cp310-cp310-musllinux_1_2_x86_64.whl": "11b3fb55f4f8ad92274ed26705f65d8f91457de71f5380061eb6d125a768fecd", + "https://files.pythonhosted.org/packages/fb/d2/4398416cd699b35167947c6e22aca52c47e69ad5695073c9f1f2c52e04aa/cffi-2.1.0-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.whl": "aa7a1b53a2a4452ada2d1b5dade9960b2522f1e61293a811a077439e39029565" }, "charset-normalizer": { "https://files.pythonhosted.org/packages/00/5e/17398df3a139985ba9d11ed072531986f408c8fca952835ef1ab1820c02b/charset_normalizer-3.4.9-cp314-cp314t-macosx_10_15_universal2.whl": "609b3ba8fcc0fb5ab7af00719d0fb6ad0cb518e48e7712d12fd68f1327951198", @@ -475,153 +491,153 @@ "https://files.pythonhosted.org/packages/f7/40/9593d54209765207a7f11073c06494c1721e4ca4a0a426c597679bf7f91e/charset_normalizer-3.4.9-cp314-cp314t-musllinux_1_2_x86_64.whl": "ee2f2a527e3c1a6e6411eb4209642e138b544a2d72fe5d0d76daf77b24063534" }, "cryptography": { - "https://files.pythonhosted.org/packages/03/11/5e395f961d6868269835dee1bafec6a1ac176505a167f68b7d8818431068/cryptography-46.0.7-cp314-cp314t-manylinux2014_x86_64.manylinux_2_17_x86_64.whl": "ebd6daf519b9f189f85c479427bbd6e9c9037862cf8fe89ee35503bd209ed902", - "https://files.pythonhosted.org/packages/0b/5d/4a8f770695d73be252331e60e526291e3df0c9b27556a90a6b47bccca4c2/cryptography-46.0.7-cp311-abi3-macosx_10_9_universal2.whl": "ea42cbe97209df307fdc3b155f1b6fa2577c0defa8f1f7d3be7d31d189108ad4", - "https://files.pythonhosted.org/packages/0f/54/6bbbfc5efe86f9d71041827b793c24811a017c6ac0fd12883e4caa86b8ed/cryptography-46.0.7-cp311-abi3-manylinux_2_28_ppc64le.whl": "cbd5fb06b62bd0721e1170273d3f4d5a277044c47ca27ee257025146c34cbdd1", - "https://files.pythonhosted.org/packages/10/f2/19ceb3b3dc14009373432af0c13f46aa08e3ce334ec6eff13492e1812ccd/cryptography-46.0.7-cp311-abi3-musllinux_1_2_x86_64.whl": "5d1c02a14ceb9148cc7816249f64f623fbfee39e8c03b3650d842ad3f34d637e", - "https://files.pythonhosted.org/packages/16/01/0cd51dd86ab5b9befe0d031e276510491976c3a80e9f6e31810cce46c4ad/cryptography-46.0.7-cp38-abi3-manylinux_2_31_armv7l.whl": "cdfbe22376065ffcf8be74dc9a909f032df19bc58a699456a21712d6e5eabfd0", - "https://files.pythonhosted.org/packages/1a/bb/a5c213c19ee94b15dfccc48f363738633a493812687f5567addbcbba9f6f/cryptography-46.0.7-cp311-abi3-win32.whl": "d23c8ca48e44ee015cd0a54aeccdf9f09004eba9fc96f38c911011d9ff1bd457", - "https://files.pythonhosted.org/packages/20/2a/1b016902351a523aa2bd446b50a5bc1175d7a7d1cf90fe2ef904f9b84ebc/cryptography-46.0.7-pp311-pypy311_pp73-win_amd64.whl": "258514877e15963bd43b558917bc9f54cf7cf866c38aa576ebf47a77ddbc43a4", - "https://files.pythonhosted.org/packages/28/17/b59a741645822ec6d04732b43c5d35e4ef58be7bfa84a81e5ae6f05a1d33/cryptography-46.0.7-cp314-cp314t-musllinux_1_2_aarch64.whl": "fcd8eac50d9138c1d7fc53a653ba60a2bee81a505f9f8850b6b2888555a45d0e", - "https://files.pythonhosted.org/packages/2b/02/7788f9fefa1d060ca68717c3901ae7fffa21ee087a90b7f23c7a603c32ae/cryptography-46.0.7-cp311-abi3-win_amd64.whl": "397655da831414d165029da9bc483bed2fe0e75dde6a1523ec2fe63f3c46046b", - "https://files.pythonhosted.org/packages/2d/cf/054b9d8220f81509939599c8bdbc0c408dbd2bdd41688616a20731371fe0/cryptography-46.0.7-cp311-abi3-manylinux_2_28_x86_64.whl": "420b1e4109cc95f0e5700eed79908cef9268265c773d3a66f7af1eef53d409ef", - "https://files.pythonhosted.org/packages/32/a8/9f0e4ed57ec9cebe506e58db11ae472972ecb0c659e4d52bbaee80ca340a/cryptography-46.0.7-cp314-cp314t-win_amd64.whl": "e06acf3c99be55aa3b516397fe42f5855597f430add9c17fa46bf2e0fb34c9bb", - "https://files.pythonhosted.org/packages/36/5f/313586c3be5a2fbe87e4c9a254207b860155a8e1f3cca99f9910008e7d08/cryptography-46.0.7-cp311-abi3-manylinux_2_34_aarch64.whl": "8a469028a86f12eb7d2fe97162d0634026d92a21f3ae0ac87ed1c4a447886c83", - "https://files.pythonhosted.org/packages/3a/ea/075aac6a84b7c271578d81a2f9968acb6e273002408729f2ddff517fed4a/cryptography-46.0.7-pp311-pypy311_pp73-manylinux_2_28_aarch64.whl": "d3b99c535a9de0adced13d159c5a9cf65c325601aa30f4be08afd680643e9c15", - "https://files.pythonhosted.org/packages/3d/4c/7d258f169ae71230f25d9f3d06caabcff8c3baf0978e2b7d65e0acac3827/cryptography-46.0.7-cp314-cp314t-manylinux_2_31_armv7l.whl": "60627cf07e0d9274338521205899337c5d18249db56865f943cbe753aa96f40f", - "https://files.pythonhosted.org/packages/40/53/8ed1cf4c3b9c8e611e7122fb56f1c32d09e1fff0f1d77e78d9ff7c82653e/cryptography-46.0.7-cp314-cp314t-manylinux_2_28_aarch64.whl": "b7b412817be92117ec5ed95f880defe9cf18a832e8cafacf0a22337dc1981b4d", - "https://files.pythonhosted.org/packages/41/3d/fe14df95a83319af25717677e956567a105bb6ab25641acaa093db79975d/cryptography-46.0.7-cp314-cp314t-manylinux_2_34_ppc64le.whl": "c5b1ccd1239f48b7151a65bc6dd54bcfcc15e028c8ac126d3fada09db0e07ef1", - "https://files.pythonhosted.org/packages/41/52/a8908dcb1a389a459a29008c29966c1d552588d4ae6d43f3a1a4512e0ebe/cryptography-46.0.7-cp38-abi3-musllinux_1_2_x86_64.whl": "a1529d614f44b863a7b480c6d000fe93b59acee9c82ffa027cfadc77521a9f5e", - "https://files.pythonhosted.org/packages/47/93/ac8f3d5ff04d54bc814e961a43ae5b0b146154c89c61b47bb07557679b18/cryptography-46.0.7.tar.gz": "e4cfd68c5f3e0bfdad0d38e023239b96a2fe84146481852dffbcca442c245aa5", - "https://files.pythonhosted.org/packages/4a/9a/1765afe9f572e239c3469f2cb429f3ba7b31878c893b246b4b2994ffe2fe/cryptography-46.0.7-cp311-abi3-manylinux2014_x86_64.manylinux_2_17_x86_64.whl": "5ad9ef796328c5e3c4ceed237a183f5d41d21150f972455a9d926593a1dcb308", - "https://files.pythonhosted.org/packages/4b/fa/f0ab06238e899cc3fb332623f337a7364f36f4bb3f2534c2bb95a35b132c/cryptography-46.0.7-cp38-abi3-win32.whl": "f247c8c1a1fb45e12586afbb436ef21ff1e80670b2861a90353d9b025583d246", - "https://files.pythonhosted.org/packages/50/46/cf71e26025c2e767c5609162c866a78e8a2915bbcfa408b7ca495c6140c4/cryptography-46.0.7-cp314-cp314t-manylinux_2_28_ppc64le.whl": "fbfd0e5f273877695cb93baf14b185f4878128b250cc9f8e617ea0c025dfb022", - "https://files.pythonhosted.org/packages/59/6a/bb2e166d6d0e0955f1e9ff70f10ec4b2824c9cfcdb4da772c7dd69cc7d80/cryptography-46.0.7-cp314-cp314t-musllinux_1_2_x86_64.whl": "65814c60f8cc400c63131584e3e1fad01235edba2614b61fbfbfa954082db0ee", - "https://files.pythonhosted.org/packages/5f/45/6d80dc379b0bbc1f9d1e429f42e4cb9e1d319c7a8201beffd967c516ea01/cryptography-46.0.7-cp311-abi3-manylinux2014_aarch64.manylinux_2_17_aarch64.whl": "b36a4695e29fe69215d75960b22577197aca3f7a25b9cf9d165dcfe9d80bc325", - "https://files.pythonhosted.org/packages/63/0c/dca8abb64e7ca4f6b2978769f6fea5ad06686a190cec381f0a796fdcaaba/cryptography-46.0.7-pp311-pypy311_pp73-macosx_11_0_arm64.whl": "fc9ab8856ae6cf7c9358430e49b368f3108f050031442eaeb6b9d87e4dcf4e4f", - "https://files.pythonhosted.org/packages/69/33/60dfc4595f334a2082749673386a4d05e4f0cf4df8248e63b2c3437585f2/cryptography-46.0.7-cp311-abi3-manylinux_2_34_ppc64le.whl": "9694078c5d44c157ef3162e3bf3946510b857df5a3955458381d1c7cfc143ddb", - "https://files.pythonhosted.org/packages/6c/7b/1c55db7242b5e5612b29fc7a630e91ee7a6e3c8e7bf5406d22e206875fbd/cryptography-46.0.7-pp311-pypy311_pp73-manylinux_2_28_x86_64.whl": "d02c738dacda7dc2a74d1b2b3177042009d5cab7c7079db74afc19e56ca1b455", - "https://files.pythonhosted.org/packages/74/66/e3ce040721b0b5599e175ba91ab08884c75928fbeb74597dd10ef13505d2/cryptography-46.0.7-cp314-cp314t-manylinux2014_aarch64.manylinux_2_17_aarch64.whl": "db0f493b9181c7820c8134437eb8b0b4792085d37dbb24da050476ccb664e59c", - "https://files.pythonhosted.org/packages/7b/56/15619b210e689c5403bb0540e4cb7dbf11a6bf42e483b7644e471a2812b3/cryptography-46.0.7-cp314-cp314t-macosx_10_9_universal2.whl": "d151173275e1728cf7839aaa80c34fe550c04ddb27b34f48c232193df8db5842", - "https://files.pythonhosted.org/packages/80/07/ad9b3c56ebb95ed2473d46df0847357e01583f4c52a85754d1a55e29e4d0/cryptography-46.0.7-cp38-abi3-manylinux_2_34_ppc64le.whl": "935ce7e3cfdb53e3536119a542b839bb94ec1ad081013e9ab9b7cfd478b05006", - "https://files.pythonhosted.org/packages/8a/6c/1a42450f464dda6ffbe578a911f773e54dd48c10f9895a23a7e88b3e7db5/cryptography-46.0.7-cp38-abi3-manylinux2014_x86_64.manylinux_2_17_x86_64.whl": "128c5edfe5e5938b86b03941e94fac9ee793a94452ad1365c9fc3f4f62216832", - "https://files.pythonhosted.org/packages/8f/3e/af9246aaf23cd4ee060699adab1e47ced3f5f7e7a8ffdd339f817b446462/cryptography-46.0.7-cp311-abi3-manylinux_2_28_aarch64.whl": "73510b83623e080a2c35c62c15298096e2a5dc8d51c3b4e1740211839d0dea77", - "https://files.pythonhosted.org/packages/92/49/819d6ed3a7d9349c2939f81b500a738cb733ab62fbecdbc1e38e83d45e12/cryptography-46.0.7-cp38-abi3-manylinux_2_34_aarch64.whl": "abad9dac36cbf55de6eb49badd4016806b3165d396f64925bf2999bcb67837ba", - "https://files.pythonhosted.org/packages/95/b6/3da51d48415bcb63b00dc17c2eff3a651b7c4fed484308d0f19b30e8cb2c/cryptography-46.0.7-cp314-cp314t-win32.whl": "fdd1736fed309b4300346f88f74cd120c27c56852c3838cab416e7a166f67298", - "https://files.pythonhosted.org/packages/9a/92/4ed714dbe93a066dc1f4b4581a464d2d7dbec9046f7c8b7016f5286329e2/cryptography-46.0.7-cp38-abi3-manylinux_2_28_aarch64.whl": "5e51be372b26ef4ba3de3c167cd3d1022934bc838ae9eaad7e644986d2a3d163", - "https://files.pythonhosted.org/packages/9c/59/4a479e0f36f8f378d397f4eab4c850b4ffb79a2f0d58704b8fa0703ddc11/cryptography-46.0.7-cp314-cp314t-manylinux_2_34_x86_64.whl": "d5f7520159cd9c2154eb61eb67548ca05c5774d39e9c2c4339fd793fe7d097b2", - "https://files.pythonhosted.org/packages/a5/d0/36a49f0262d2319139d2829f773f1b97ef8aef7f97e6e5bd21455e5a8fb5/cryptography-46.0.7-cp38-abi3-manylinux2014_aarch64.manylinux_2_17_aarch64.whl": "84d4cced91f0f159a7ddacad249cc077e63195c36aac40b4150e7a57e84fffe7", - "https://files.pythonhosted.org/packages/a5/ef/649750cbf96f3033c3c976e112265c33906f8e462291a33d77f90356548c/cryptography-46.0.7-cp38-abi3-musllinux_1_2_aarch64.whl": "7bbc6ccf49d05ac8f7d7b5e2e2c33830d4fe2061def88210a126d130d7f71a85", - "https://files.pythonhosted.org/packages/a7/7f/cd42fc3614386bc0c12f0cb3c4ae1fc2bbca5c9662dfed031514911d513d/cryptography-46.0.7-cp38-abi3-macosx_10_9_universal2.whl": "462ad5cb1c148a22b2e3bcc5ad52504dff325d17daf5df8d88c17dda1f75f2a4", - "https://files.pythonhosted.org/packages/b5/2a/2ea0767cad19e71b3530e4cad9605d0b5e338b6a1e72c37c9c1ceb86c333/cryptography-46.0.7-cp314-cp314t-manylinux_2_34_aarch64.whl": "80406c3065e2c55d7f49a9550fe0c49b3f12e5bfff5dedb727e319e1afb9bf99", - "https://files.pythonhosted.org/packages/b7/e6/a26b84096eddd51494bba19111f8fffe976f6a09f132706f8f1bf03f51f7/cryptography-46.0.7-cp38-abi3-manylinux_2_28_ppc64le.whl": "cdf1a610ef82abb396451862739e3fc93b071c844399e15b90726ef7470eeaf2", - "https://files.pythonhosted.org/packages/b8/c7/201d3d58f30c4c2bdbe9b03844c291feb77c20511cc3586daf7edc12a47b/cryptography-46.0.7-cp38-abi3-manylinux_2_34_x86_64.whl": "35719dc79d4730d30f1c2b6474bd6acda36ae2dfae1e3c16f2051f215df33ce0", - "https://files.pythonhosted.org/packages/c0/ea/01276740375bac6249d0a971ebdf6b4dc9ead0ee0a34ef3b5a88c1a9b0d4/cryptography-46.0.7-cp314-cp314t-manylinux_2_28_x86_64.whl": "ffca7aa1d00cf7d6469b988c581598f2259e46215e0140af408966a24cf086ce", - "https://files.pythonhosted.org/packages/c7/08/ffd537b605568a148543ac3c2b239708ae0bd635064bab41359252ef88ed/cryptography-46.0.7-cp38-abi3-manylinux_2_28_x86_64.whl": "1d25aee46d0c6f1a501adcddb2d2fee4b979381346a78558ed13e50aa8a59067", - "https://files.pythonhosted.org/packages/c7/0b/333ddab4270c4f5b972f980adef4faa66951a4aaf646ca067af597f15563/cryptography-46.0.7-cp311-abi3-manylinux_2_34_x86_64.whl": "42a1e5f98abb6391717978baf9f90dc28a743b7d9be7f0751a6f56a75d14065b", - "https://files.pythonhosted.org/packages/cb/da/9870eec4b69c63ef5925bf7d8342b7e13bc2ee3d47791461c4e49ca212f4/cryptography-46.0.7-pp311-pypy311_pp73-manylinux_2_34_aarch64.whl": "04959522f938493042d595a736e7dbdff6eb6cc2339c11465b3ff89343b65f65", - "https://files.pythonhosted.org/packages/d2/14/633913398b43b75f1234834170947957c6b623d1701ffc7a9600da907e89/cryptography-46.0.7-cp311-abi3-musllinux_1_2_aarch64.whl": "91bbcb08347344f810cbe49065914fe048949648f6bd5c2519f34619142bbe85", - "https://files.pythonhosted.org/packages/d2/f1/00ce3bde3ca542d1acd8f8cfa38e446840945aa6363f9b74746394b14127/cryptography-46.0.7-cp38-abi3-win_amd64.whl": "506c4ff91eff4f82bdac7633318a526b1d1309fc07ca76a3ad182cb5b686d6d3", - "https://files.pythonhosted.org/packages/f4/72/05aa5832b82dd341969e9a734d1812a6aadb088d9eb6f0430fc337cc5a8f/cryptography-46.0.7-pp311-pypy311_pp73-manylinux_2_34_x86_64.whl": "3986ac1dee6def53797289999eabe84798ad7817f3e97779b5061a95b0ee4968", - "https://files.pythonhosted.org/packages/f9/46/4e4e9c6040fb01c7467d47217d2f882daddeb8828f7df800cb806d8a2288/cryptography-46.0.7-cp311-abi3-manylinux_2_31_armv7l.whl": "24402210aa54baae71d99441d15bb5a1919c195398a87b563df84468160a65de" + "https://files.pythonhosted.org/packages/09/41/3797cfaf69cae04a13ee78ebd83f0678d9c02b4779d21ce24445326f1a69/cryptography-49.0.0-cp311-abi3-manylinux2014_aarch64.manylinux_2_17_aarch64.whl": "36d1709f992593689b45bda411498d62c6e365f2ca00b84657d4dadd24de16db", + "https://files.pythonhosted.org/packages/11/2d/5e1fb307cb5931881516b464c98774b3f2c36b5d4bb9a2830253cf553cad/cryptography-49.0.0-cp314-cp314t-manylinux_2_34_x86_64.whl": "d8ecde755e2e91bf773fc94e8c9d730cd7f2007004cb492263a794ec3899a1c8", + "https://files.pythonhosted.org/packages/17/50/983e838c7fd0d87fd8c969bcdd328edaf5f756e38df5281637424c155873/cryptography-49.0.0-cp314-cp314t-manylinux_2_31_armv7l.whl": "07cab27cc7b7e0fd28e5e26bb9eeedde5c135c868b46de4a27845abe94af6122", + "https://files.pythonhosted.org/packages/19/2a/5bb823f5bedcf80718cea7fbc95ec5515cca3769633c4b01a32be7f30e7c/cryptography-49.0.0-cp39-abi3-macosx_11_0_arm64.whl": "ec5e529fb80935c94fe7b729f9972b50e351a0e6b50aa294fd5cabb109fcc29a", + "https://files.pythonhosted.org/packages/1f/09/f42b1d190c5ba75f72062a387f8030d1d75f6ab035788f1d9c4b01de6525/cryptography-49.0.0-cp311-abi3-win_amd64.whl": "e5dfc1e64de5677cec922ffa8da89c546d0415bf6efdf081842e5d44c84e1f0e", + "https://files.pythonhosted.org/packages/1f/99/d1c90d6041656cc6ee229dc99cd67fd0cd5aec3c5f7d72fffc27cc750054/cryptography-49.0.0.tar.gz": "f89660a348f4f78a92366240a61404e337586ef7f5909a2fef59ca88ef505493", + "https://files.pythonhosted.org/packages/20/2c/0622f20ff02b2ef32558733443805dc82fd4c275be01b2d19d14676f3a1b/cryptography-49.0.0-cp311-abi3-manylinux_2_28_x86_64.whl": "2afe9051da7ae7bd5905da5a949280c7d2bb75682e188f650a9d0f2756b834c6", + "https://files.pythonhosted.org/packages/24/01/186c825898477d77e2324d5360fefe622ff1d8d1963ec0554e2cada8ec77/cryptography-49.0.0-cp39-abi3-manylinux_2_28_ppc64le.whl": "9e82dcc8e56052715fb18b2429e3bca4823b1629136a2084fc45a9a5cecb9b64", + "https://files.pythonhosted.org/packages/2c/99/2d13299eb3dd27b02dcfaafcc91d6b5cb3329f7cbd6d8f51921acd566c1a/cryptography-49.0.0-cp39-abi3-manylinux2014_x86_64.manylinux_2_17_x86_64.whl": "35b151772baff2c74cba7fa290ceaff4c3b11c0c881eb93eb5dbc05a7cfbba18", + "https://files.pythonhosted.org/packages/3d/df/40577043ca124e17012f408ddddaeb213b856336ac82ddb3bc915f39e29f/cryptography-49.0.0-cp39-abi3-manylinux2014_aarch64.manylinux_2_17_aarch64.whl": "f78ff2c9ed8dc2d036b0f4d640e22522213d047c1b14e61205a7e55c80a494d4", + "https://files.pythonhosted.org/packages/47/f1/1d3eaa243bfc5de4a187b22aa8c048b3e4980bfbe830ac46e6bac2e66947/cryptography-49.0.0-cp314-cp314t-manylinux_2_28_ppc64le.whl": "f37d847238971164fdbc68ade6f6574aecc9c0af714190e2083429ff68f4ce9d", + "https://files.pythonhosted.org/packages/4a/91/01ce7303a4579e6d3a6abef01bd322848e9ea7a219adcabc5048b9033571/cryptography-49.0.0-cp311-abi3-manylinux_2_28_aarch64.whl": "53ecee2e23f7169b6117e99fc8a944e5e50f79e69758a83b52a00cb98ab2b2d2", + "https://files.pythonhosted.org/packages/4c/fe/93ecac273d3738939d023612ad12cca9a3740a5345d69fda04134c43fd96/cryptography-49.0.0-cp314-cp314t-win_amd64.whl": "33cd0565932807baddb67b96dbee92f2c374b5c89dee09fd74079aeb8c8dba61", + "https://files.pythonhosted.org/packages/4f/01/339573cf1023163a400b0b5d16f6d507de413b9f60be6fd1b77feeaf6737/cryptography-49.0.0-cp311-abi3-musllinux_1_2_aarch64.whl": "b87e65d263b3e5d3bb92a57e2a6638e2f31110fa7aa890c7b2dbba42248d0a3f", + "https://files.pythonhosted.org/packages/58/39/2d51306721330c486495853eda1c567880ff036de15a14c4b74f399934af/cryptography-49.0.0-cp314-cp314t-manylinux_2_28_x86_64.whl": "c2bc30226390d60ea19d9f82b19db005fe0452154a23c1c410c12ea801e43561", + "https://files.pythonhosted.org/packages/62/99/a2c95cf8293f07491e9e27c20cc4dcd18176d944e674679adeb1d0173fd6/cryptography-49.0.0-cp311-abi3-manylinux_2_28_ppc64le.whl": "2eda353d8a27bcbcaa4cbed18994a74ab4d19a2ca897db188ea269ab9b71419b", + "https://files.pythonhosted.org/packages/63/d3/4a83af35d65e3fad632c926fad684c193ea4398569ccb0bbbc7fe8f5dc9a/cryptography-49.0.0-pp311-pypy311_pp73-macosx_11_0_arm64.whl": "fc1e275c2f1d97b1a6450b8b0ea3ebfa6e087a611c2b26cb2404d48588abab7b", + "https://files.pythonhosted.org/packages/67/d0/a5fcd3515f0bae49a7b6d0413cc1bdccdcc1fc0047037a0d480642cdc5d6/cryptography-49.0.0-cp39-abi3-manylinux_2_34_ppc64le.whl": "6fc361c34fb6aac015ce19435876635e5c6d21db31998b0920f675f131e043b8", + "https://files.pythonhosted.org/packages/68/28/8a3ad4653662c93fc44dc4e5d8fd374c25c42e07b34bbfbadf49cf57a5a8/cryptography-49.0.0-cp314-cp314t-manylinux2014_x86_64.manylinux_2_17_x86_64.whl": "7abcee80084cda3f7691f3eb1ce480d8df49cec637b429aa35986c1de71738aa", + "https://files.pythonhosted.org/packages/6c/72/3e798c064bc39e471008075d0f9bc9daf77a80879c092e4a8e170c585ed4/cryptography-49.0.0-cp39-abi3-manylinux_2_31_armv7l.whl": "8c25ceb16df5b9435f3f6a9829204985b0e0cbee3b48aacd432c7d2c850b44d9", + "https://files.pythonhosted.org/packages/6c/a0/db537264e234f7273a73ec020873d6d6b39dfd8a53db78b550ca8320440e/cryptography-49.0.0-cp39-abi3-musllinux_1_2_aarch64.whl": "67e1d20ad9ef3a563c59ef22e7a8a0b8210bd26604369ea4a30a7c66aefe504e", + "https://files.pythonhosted.org/packages/6d/88/05563c7fe2e914e87d1a536d06fe83e66b4e1d95cb593e05aea375531da8/cryptography-49.0.0-cp311-abi3-manylinux_2_34_aarch64.whl": "ccac2bfebc306b862133e3bb71f3f6ee8bb525240089b2d952e4144b3a6d5da7", + "https://files.pythonhosted.org/packages/71/fd/577302e213a1be9468f92d1afef66fcf1ef83d516819d9992ca547f592bd/cryptography-49.0.0-cp311-abi3-musllinux_1_2_x86_64.whl": "66ec79c3904820572d7e987abdf304281f141d37ad9a489b8e97066e7b9b6459", + "https://files.pythonhosted.org/packages/86/12/c48a424f38db03027be9f7ed5c7dc5de9933dbee992865f98b13727a009d/cryptography-49.0.0-cp314-cp314t-manylinux2014_aarch64.manylinux_2_17_aarch64.whl": "196ecd6a36e4e9aa10270393bb98d8df88fccee0bf1e5128b91ae4eb4375896d", + "https://files.pythonhosted.org/packages/93/77/8df9eb486495979bccecd1062e2eaf435250e84437040295b57d09048b0b/cryptography-49.0.0-cp39-abi3-musllinux_1_2_x86_64.whl": "42b0684e0e40cf26122427802486f6d93aea593612603a94fbf260c7eb1e9c1b", + "https://files.pythonhosted.org/packages/94/64/2923570ac1c0bd3a737aa366ac3abbbbde273042308b8cde95e2364a6e6a/cryptography-49.0.0-pp311-pypy311_pp73-manylinux_2_34_aarch64.whl": "b47db11c2c3525083296069b98ac5221907455e989ae0c2e3008bde851921615", + "https://files.pythonhosted.org/packages/9b/22/adf66990e63584a68dfb50c24f48a125c07b1699899381c8151e63ed458c/cryptography-49.0.0-cp311-abi3-macosx_11_0_arm64.whl": "966fe0e9c67490071f14c0d2b1cb2dfb3023c5ce39457343931415f08382f2db", + "https://files.pythonhosted.org/packages/a0/84/84fe36f19caf857d61cb7fc9c63035a47ffabd84ea12d1d393148efa3615/cryptography-49.0.0-cp39-abi3-manylinux_2_34_x86_64.whl": "2400ef9c9e2299a25614eb1dea3db54a69b1349efd043bfac9c67630d136df36", + "https://files.pythonhosted.org/packages/a3/5b/c5246635d5fd3b64e0d45ae10e99fd32fe9676a79915ccfe5a61ba9af1a5/cryptography-49.0.0-cp311-abi3-manylinux_2_31_armv7l.whl": "0b82e28ee398a386f0807bba7884d30f25218855690f45115831bcce5d90822c", + "https://files.pythonhosted.org/packages/a5/4d/9c0cd02f95e2602dd5e563da149ee0830abef3537be8b34dc56281ebe27a/cryptography-49.0.0-cp39-abi3-manylinux_2_28_aarch64.whl": "0f21641cf4b30fca7aee061ced0ec7ad7b073518088b7c9969a297c0ae796c69", + "https://files.pythonhosted.org/packages/a7/f5/8f571d7e27c55bce9f76f026143bcb1e040a4233149ecca0bea5fa5dd5f7/cryptography-49.0.0-cp314-cp314t-manylinux_2_34_aarch64.whl": "b20133d204d2bb56ba047642199603876c872026ca53e79c35b83772ab2cc505", + "https://files.pythonhosted.org/packages/a8/b2/2193fc74f81aee4f9b62733133b73b5176718932ed8f2e4b03fa040480a6/cryptography-49.0.0-cp314-cp314t-manylinux_2_28_aarch64.whl": "4ae387c9cb68ea569ca17e490d66d8142b81c3cc814bf179974b7d146e490bbb", + "https://files.pythonhosted.org/packages/a9/3c/f3ad17eecc1a57b0ba236dc01f90e783c51f4a2f35f64777cc4f47a184b2/cryptography-49.0.0-cp311-abi3-manylinux_2_34_x86_64.whl": "cbc77da8c523d5abd028635ba850a6966fcee2c82e2bf65a41d1d8afe0f98be9", + "https://files.pythonhosted.org/packages/aa/50/a9caea39ad19c431c1a3f8a31114df65b260cdfe67786b6c7e7c040c4c44/cryptography-49.0.0-pp311-pypy311_pp73-win_amd64.whl": "be9fcb48a55f023493482827d4f459bd263cc20efde64f204b97c123201850c6", + "https://files.pythonhosted.org/packages/ab/f8/614dc7e051418cfe53d55173c1e24c6b0085e89996fe90508c2fdf769aef/cryptography-49.0.0-pp311-pypy311_pp73-manylinux_2_34_x86_64.whl": "084ef1af862eb07ec46d25f68689f2102a9fc0e05ce7b80f14f5fe51e4eef0f6", + "https://files.pythonhosted.org/packages/b8/7b/62cbbab75d0659865bf0273790031544a0b16c8072d258f9428dcd8190dc/cryptography-49.0.0-cp39-abi3-manylinux_2_28_x86_64.whl": "6f2debedf9ca60cf1d5bd466475638af5130f89965605cd818484d19987d3a21", + "https://files.pythonhosted.org/packages/b9/26/814681d14248d95d73d5c3eea0c39a94eb8302df966f670a2c60de90974b/cryptography-49.0.0-cp314-cp314t-musllinux_1_2_x86_64.whl": "32703d93296f5c1f4b53349ad3a250c2cae0fdecd3a3dd5d47e616d8d616af27", + "https://files.pythonhosted.org/packages/c2/e6/f60198ea8d9dfa15fff9ed4ca02ce362f6eadd9ba757dcc50634c4257b63/cryptography-49.0.0-cp39-abi3-win_amd64.whl": "026ac7423e6fa66872d3bf889be5974507da3944f866f704fa200eadacd00001", + "https://files.pythonhosted.org/packages/c4/b6/d7696e4e890d6ae1469935164c9e5215c557671cb78d6e3f458ccceaa632/cryptography-49.0.0-cp311-abi3-manylinux_2_34_ppc64le.whl": "d0527ce944105f257f605a827d6ebead966c752038b6e8656abb9c5edee6fc68", + "https://files.pythonhosted.org/packages/d6/a7/f9dac0ab7f80368c56993a7bf638ef9935f825c91902798481fac0898138/cryptography-49.0.0-pp311-pypy311_pp73-manylinux_2_28_aarch64.whl": "c83782480a4a9da4d0feb51950131ba32e12e70813848b3343f6e18c28a66838", + "https://files.pythonhosted.org/packages/d7/70/2ba3769dd0ae167e2f33dfa9592d45db6ff9a61d62ca1a5b3d1bdd09068f/cryptography-49.0.0-pp311-pypy311_pp73-manylinux_2_28_x86_64.whl": "b39efa323140595abd3ecca8529d321ae50f55f3aa3ba9cc81ea56a6011953d5", + "https://files.pythonhosted.org/packages/e4/c0/bff5a02ee731d207d6a1ed51732549d8c53d2bc8da1d10ec6f2844201d68/cryptography-49.0.0-cp314-cp314t-musllinux_1_2_aarch64.whl": "e3fb64c420688e5319ae25113a354015abbd8dffbfbc41781a1ea66fc7622ac3", + "https://files.pythonhosted.org/packages/e6/8b/43011f7ebe515a8aa20d61f290a326cd890c2e738e16e59eaff8d9c3a412/cryptography-49.0.0-cp311-abi3-manylinux2014_x86_64.manylinux_2_17_x86_64.whl": "0e959b578856a3924bc0cbb710fc12c387b9412a951389f3ca61704a9e25f325", + "https://files.pythonhosted.org/packages/e7/84/0e27016a6fc5a0886f797018b26aa42f40c09a82332bff77822a451deaaa/cryptography-49.0.0-cp314-cp314t-manylinux_2_34_ppc64le.whl": "b970c6da94d5bb18629db453d14f2a1300f6bf59b61e9b82377931ef95504866", + "https://files.pythonhosted.org/packages/ec/9e/db72b3ae7fc9cfad53e630e56c6ae83b9b6ff0bf3718ffb8012d20b3aabf/cryptography-49.0.0-cp314-cp314t-macosx_11_0_arm64.whl": "73a205dce83953d131a4aa1e0fd917a2fd1c5b1eef251e9d7152efefcbf5caf7", + "https://files.pythonhosted.org/packages/f0/ee/6fca21d1ac73e06f8bef71940abfd4d2f6472b4bca284d770f32bd4086f6/cryptography-49.0.0-cp39-abi3-manylinux_2_34_aarch64.whl": "28d8b15e6275f12c8a207dc309dfa957903c927d08d0cc937ee3f63f200693cc" }, "docutils": { - "https://files.pythonhosted.org/packages/4a/c0/89fe6215b443b919cb98a5002e107cb5026854ed1ccb6b5833e0768419d1/docutils-0.22.2.tar.gz": "9fdb771707c8784c8f2728b67cb2c691305933d68137ef95a75db5f4dfbc213d", - "https://files.pythonhosted.org/packages/66/dd/f95350e853a4468ec37478414fc04ae2d61dad7a947b3015c3dcc51a09b9/docutils-0.22.2-py3-none-any.whl": "b0e98d679283fc3bb0ead8a5da7f501baa632654e7056e9c5846842213d674d8" + "https://files.pythonhosted.org/packages/32/91/30151a39f7570f448ed84529390628a651d7f27c87d73c9b887f8189695e/docutils-0.23-py3-none-any.whl": "25d013af9bf23bc1c7b2b093dff4208166c53a94786c9e447808335ef1185fea", + "https://files.pythonhosted.org/packages/39/a4/5180d9afc57e8fca05601dd652bdff19604c218814037fe90ffc7625a50a/docutils-0.23.tar.gz": "746f5060322511280a1e50eb76846ed6bf2342984b2ac04dc42caa1a8d78799e" + }, + "id": { + "https://files.pythonhosted.org/packages/42/77/de194443bf38daed9452139e960c632b0ef9f9a5dd9ce605fdf18ca9f1b1/id-1.6.1-py3-none-any.whl": "f5ec41ed2629a508f5d0988eda142e190c9c6da971100612c4de9ad9f9b237ca", + "https://files.pythonhosted.org/packages/6d/04/c2156091427636080787aac190019dc64096e56a23b7364d3c1764ee3a06/id-1.6.1.tar.gz": "d0732d624fb46fd4e7bc4e5152f00214450953b9e772c182c1c22964def1a069" }, "idna": { - "https://files.pythonhosted.org/packages/76/c6/c88e154df9c4e1a2a66ccf0005a88dfb2650c1dffb6f5ce603dfbd452ce3/idna-3.10-py3-none-any.whl": "946d195a0d259cbba61165e88e65941f16e9b36ea6ddb97f00452bae8b1287d3", - "https://files.pythonhosted.org/packages/f1/70/7703c29685631f5a7590aa73f1f1d3fa9a380e654b86af429e0934a32f7d/idna-3.10.tar.gz": "12f65c9b470abda6dc35cf8e63cc574b1c52b11df2c86030af0ac09b01b13ea9" + "https://files.pythonhosted.org/packages/1e/5e/d4e9f1a599fb8e573b7b87160658329fbf28d19eac2718f51fc3def3aa5a/idna-3.18-py3-none-any.whl": "7f952cbe720b688055e3f87de14f5c3e5fdaa8bc3928985c4077ca689de849a2", + "https://files.pythonhosted.org/packages/cd/63/9496c57188a2ee585e0f1db071d75089a11e98aa86eb99d9d7618fc1edce/idna-3.18.tar.gz": "ffb385a7e039654cef1ab9ef32c6fafe283c0c0467bba1d9029738ce4a14a848" }, "importlib-metadata": { - "https://files.pythonhosted.org/packages/20/b0/36bd937216ec521246249be3bf9855081de4c5e06a0c9b4219dbeda50373/importlib_metadata-8.7.0-py3-none-any.whl": "e5dd1551894c77868a30651cef00984d50e1002d06942a7101d34870c5f02afd", - "https://files.pythonhosted.org/packages/76/66/650a33bd90f786193e4de4b3ad86ea60b53c89b669a5c7be931fac31cdb0/importlib_metadata-8.7.0.tar.gz": "d13b81ad223b890aa16c5471f2ac3056cf76c5f10f82d6f9292f0b415f389000" + "https://files.pythonhosted.org/packages/38/3d/2d244233ac4f76e38533cfcb2991c9eb4c7bf688ae0a036d30725b8faafe/importlib_metadata-9.0.0-py3-none-any.whl": "2d21d1cc5a017bd0559e36150c21c830ab1dc304dedd1b7ea85d20f45ef3edd7", + "https://files.pythonhosted.org/packages/a9/01/15bb152d77b21318514a96f43af312635eb2500c96b55398d020c93d86ea/importlib_metadata-9.0.0.tar.gz": "a4f57ab599e6a2e3016d7595cfd72eb4661a5106e787a95bcc90c7105b831efc" }, "jaraco-classes": { "https://files.pythonhosted.org/packages/06/c0/ed4a27bc5571b99e3cff68f8a9fa5b56ff7df1c2251cc715a652ddd26402/jaraco.classes-3.4.0.tar.gz": "47a024b51d0239c0dd8c8540c6c7f484be3b8fcf0b2d85c13825780d3b3f3acd", "https://files.pythonhosted.org/packages/7f/66/b15ce62552d84bbfcec9a4873ab79d993a1dd4edb922cbfccae192bd5b5f/jaraco.classes-3.4.0-py3-none-any.whl": "f662826b6bed8cace05e7ff873ce0f9283b5c924470fe664fff1c2f00f581790" }, "jaraco-context": { - "https://files.pythonhosted.org/packages/df/ad/f3777b81bf0b6e7bc7514a1656d3e637b2e8e15fab2ce3235730b3e7a4e6/jaraco_context-6.0.1.tar.gz": "9bae4ea555cf0b14938dc0aee7c9f32ed303aa20a3b73e7dc80111628792d1b3", - "https://files.pythonhosted.org/packages/ff/db/0c52c4cf5e4bd9f5d7135ec7669a3a767af21b3a308e1ed3674881e52b62/jaraco.context-6.0.1-py3-none-any.whl": "f797fc481b490edb305122c9181830a3a5b76d84ef6d1aef2fb9b47ab956f9e4" + "https://files.pythonhosted.org/packages/af/50/4763cd07e722bb6285316d390a164bc7e479db9d90daa769f22578f698b4/jaraco_context-6.1.2.tar.gz": "f1a6c9d391e661cc5b8d39861ff077a7dc24dc23833ccee564b234b81c82dfe3", + "https://files.pythonhosted.org/packages/f2/58/bc8954bda5fcda97bd7c19be11b85f91973d67a706ed4a3aec33e7de22db/jaraco_context-6.1.2-py3-none-any.whl": "bf8150b79a2d5d91ae48629d8b427a8f7ba0e1097dd6202a9059f29a36379535" }, "jaraco-functools": { - "https://files.pythonhosted.org/packages/b4/09/726f168acad366b11e420df31bf1c702a54d373a83f968d94141a8c3fde0/jaraco_functools-4.3.0-py3-none-any.whl": "227ff8ed6f7b8f62c56deff101545fa7543cf2c8e7b82a7c2116e672f29c26e8", - "https://files.pythonhosted.org/packages/f7/ed/1aa2d585304ec07262e1a83a9889880701079dde796ac7b1d1826f40c63d/jaraco_functools-4.3.0.tar.gz": "cfd13ad0dd2c47a3600b439ef72d8615d482cedcff1632930d6f28924d92f294" + "https://files.pythonhosted.org/packages/02/36/ecc85bc96c273dc8a11273ed4782272975e6338d4a3e9228621175edf0e3/jaraco_functools-4.6.0-py3-none-any.whl": "99e3dc0060c5cbe8fcd1cdb36258e2a65ca40f1566b2033b12abb1bb44dd3c30", + "https://files.pythonhosted.org/packages/6c/1f/c23395957d41ccf27c4e535c3d334c4051e5395b3752057ba4cbaec35c56/jaraco_functools-4.6.0.tar.gz": "880c577ec9720b3a052d5bc611fb9f2269b3d87902ef42440df443b88e443280" }, "jeepney": { "https://files.pythonhosted.org/packages/7b/6f/357efd7602486741aa73ffc0617fb310a29b588ed0fd69c2399acbb85b0c/jeepney-0.9.0.tar.gz": "cf0e9e845622b81e4a28df94c40345400256ec608d0e55bb8a3feaa9163f5732", "https://files.pythonhosted.org/packages/b2/a3/e137168c9c44d18eff0376253da9f1e9234d0239e0ee230d2fee6cea8e55/jeepney-0.9.0-py3-none-any.whl": "97e5714520c16fc0a45695e5365a2e11b81ea79bba796e26f9f1d178cb182683" }, "keyring": { - "https://files.pythonhosted.org/packages/70/09/d904a6e96f76ff214be59e7aa6ef7190008f52a0ab6689760a98de0bf37d/keyring-25.6.0.tar.gz": "0b39998aa941431eb3d9b0d4b2460bc773b9df6fed7621c2dfb291a7e0187a66", - "https://files.pythonhosted.org/packages/d3/32/da7f44bcb1105d3e88a0b74ebdca50c59121d2ddf71c9e34ba47df7f3a56/keyring-25.6.0-py3-none-any.whl": "552a3f7af126ece7ed5c89753650eec89c7eaae8617d0aa4d9ad2b75111266bd" + "https://files.pythonhosted.org/packages/43/4b/674af6ef2f97d56f0ab5153bf0bfa28ccb6c3ed4d1babf4305449668807b/keyring-25.7.0.tar.gz": "fe01bd85eb3f8fb3dd0405defdeac9a5b4f6f0439edbb3149577f244a2e8245b", + "https://files.pythonhosted.org/packages/81/db/e655086b7f3a705df045bf0933bdd9c2f79bb3c97bfef1384598bb79a217/keyring-25.7.0-py3-none-any.whl": "be4a0b195f149690c166e850609a477c532ddbfbaed96a404d4e43f8d5e2689f" }, "markdown-it-py": { - "https://files.pythonhosted.org/packages/5b/f5/4ec618ed16cc4f8fb3b701563655a69816155e79e24a17b651541804721d/markdown_it_py-4.0.0.tar.gz": "cb0a2b4aa34f932c007117b194e945bd74e0ec24133ceb5bac59009cda1cb9f3", - "https://files.pythonhosted.org/packages/94/54/e7d793b573f298e1c9013b8c4dade17d481164aa517d1d7148619c2cedbf/markdown_it_py-4.0.0-py3-none-any.whl": "87327c59b172c5011896038353a81343b6754500a08cd7a4973bb48c6d578147" + "https://files.pythonhosted.org/packages/06/ff/7841249c247aa650a76b9ee4bbaeae59370dc8bfd2f6c01f3630c35eb134/markdown_it_py-4.2.0.tar.gz": "04a21681d6fbb623de53f6f364d352309d4094dd4194040a10fd51833e418d49", + "https://files.pythonhosted.org/packages/b3/81/4da04ced5a082363ecfa159c010d200ecbd959ae410c10c0264a38cac0f5/markdown_it_py-4.2.0-py3-none-any.whl": "9f7ebbcd14fe59494226453aed97c1070d83f8d24b6fc3a3bcf9a38092641c4a" }, "mdurl": { "https://files.pythonhosted.org/packages/b3/38/89ba8ad64ae25be8de66a6d463314cf1eb366222074cfda9ee839c56a4b4/mdurl-0.1.2-py3-none-any.whl": "84008a41e51615a49fc9966191ff91509e3c40b939176e643fd50a5c2196b8f8", "https://files.pythonhosted.org/packages/d6/54/cfe61301667036ec958cb99bd3efefba235e65cdeb9c84d24a8293ba1d90/mdurl-0.1.2.tar.gz": "bb413d29f5eea38f31dd4754dd7377d4465116fb207585f97bf925588687c1ba" }, "more-itertools": { - "https://files.pythonhosted.org/packages/a4/8e/469e5a4a2f5855992e425f3cb33804cc07bf18d48f2db061aec61ce50270/more_itertools-10.8.0-py3-none-any.whl": "52d4362373dcf7c52546bc4af9a86ee7c4579df9a8dc268be0a2f949d376cc9b", - "https://files.pythonhosted.org/packages/ea/5d/38b681d3fce7a266dd9ab73c66959406d565b3e85f21d5e66e1181d93721/more_itertools-10.8.0.tar.gz": "f638ddf8a1a0d134181275fb5d58b086ead7c6a72429ad725c67503f13ba30bd" + "https://files.pythonhosted.org/packages/de/1d/f4da6f02cdffe04d6362210b807146a26044c88d839208aec273bb0d9184/more_itertools-11.1.0.tar.gz": "48e8f4d9e7e5878571ecf6f2b4e57634f93cd474cc8cfbd2376f2d11b396e30d", + "https://files.pythonhosted.org/packages/e8/3d/1087453384dbde46a8c7f9356eead2c58be8a7bf156bca40243377c85715/more_itertools-11.1.0-py3-none-any.whl": "4b65538ae22f6fed0ce4874efd317463a7489796a0939fa66824dd542125a192" }, "nh3": { - "https://files.pythonhosted.org/packages/0c/e0/cf1543e798ba86d838952e8be4cb8d18e22999be2a24b112a671f1c04fd6/nh3-0.3.0-cp38-abi3-macosx_10_12_x86_64.macosx_11_0_arm64.macosx_10_12_universal2.whl": "ec6cfdd2e0399cb79ba4dcffb2332b94d9696c52272ff9d48a630c5dca5e325a", - "https://files.pythonhosted.org/packages/10/71/2fb1834c10fab6d9291d62c95192ea2f4c7518bd32ad6c46aab5d095cb87/nh3-0.3.0-cp313-cp313t-musllinux_1_2_i686.whl": "0649464ac8eee018644aacbc103874ccbfac80e3035643c3acaab4287e36e7f5", - "https://files.pythonhosted.org/packages/23/1e/80a8c517655dd40bb13363fc4d9e66b2f13245763faab1a20f1df67165a7/nh3-0.3.0-cp313-cp313t-win_amd64.whl": "423201bbdf3164a9e09aa01e540adbb94c9962cc177d5b1cbb385f5e1e79216e", - "https://files.pythonhosted.org/packages/2f/d6/f1c6e091cbe8700401c736c2bc3980c46dca770a2cf6a3b48a175114058e/nh3-0.3.0-cp313-cp313t-win32.whl": "7275fdffaab10cc5801bf026e3c089d8de40a997afc9e41b981f7ac48c5aa7d5", - "https://files.pythonhosted.org/packages/33/c1/8f8ccc2492a000b6156dce68a43253fcff8b4ce70ab4216d08f90a2ac998/nh3-0.3.0-cp313-cp313t-musllinux_1_2_x86_64.whl": "1adeb1062a1c2974bc75b8d1ecb014c5fd4daf2df646bbe2831f7c23659793f9", - "https://files.pythonhosted.org/packages/39/2c/6394301428b2017a9d5644af25f487fa557d06bc8a491769accec7524d9a/nh3-0.3.0-cp38-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl": "f416c35efee3e6a6c9ab7716d9e57aa0a49981be915963a82697952cba1353e1", - "https://files.pythonhosted.org/packages/4c/3c/cba7b26ccc0ef150c81646478aa32f9c9535234f54845603c838a1dc955c/nh3-0.3.0-cp313-cp313t-musllinux_1_2_aarch64.whl": "80fe20171c6da69c7978ecba33b638e951b85fb92059259edd285ff108b82a6d", - "https://files.pythonhosted.org/packages/4e/9a/344b9f9c4bd1c2413a397f38ee6a3d5db30f1a507d4976e046226f12b297/nh3-0.3.0-cp38-abi3-manylinux_2_5_i686.manylinux1_i686.whl": "37d3003d98dedca6cd762bf88f2e70b67f05100f6b949ffe540e189cc06887f9", - "https://files.pythonhosted.org/packages/5b/76/3165e84e5266d146d967a6cc784ff2fbf6ddd00985a55ec006b72bc39d5d/nh3-0.3.0-cp38-abi3-win_arm64.whl": "d97d3efd61404af7e5721a0e74d81cdbfc6e5f97e11e731bb6d090e30a7b62b2", - "https://files.pythonhosted.org/packages/5c/86/a96b1453c107b815f9ab8fac5412407c33cc5c7580a4daf57aabeb41b774/nh3-0.3.0-cp38-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl": "ce5e7185599f89b0e391e2f29cc12dc2e206167380cea49b33beda4891be2fe1", - "https://files.pythonhosted.org/packages/63/da/c5fd472b700ba37d2df630a9e0d8cc156033551ceb8b4c49cc8a5f606b68/nh3-0.3.0-cp313-cp313t-manylinux_2_5_i686.manylinux1_i686.whl": "ba0caa8aa184196daa6e574d997a33867d6d10234018012d35f86d46024a2a95", - "https://files.pythonhosted.org/packages/66/3f/cd37f76c8ca277b02a84aa20d7bd60fbac85b4e2cbdae77cb759b22de58b/nh3-0.3.0-cp38-abi3-musllinux_1_2_aarch64.whl": "634e34e6162e0408e14fb61d5e69dbaea32f59e847cfcfa41b66100a6b796f62", - "https://files.pythonhosted.org/packages/6a/1b/b15bd1ce201a1a610aeb44afd478d55ac018b4475920a3118ffd806e2483/nh3-0.3.0-cp38-abi3-manylinux_2_17_ppc64.manylinux2014_ppc64.whl": "e9e6a7e4d38f7e8dda9edd1433af5170c597336c1a74b4693c5cb75ab2b30f2a", - "https://files.pythonhosted.org/packages/8c/ae/324b165d904dc1672eee5f5661c0a68d4bab5b59fbb07afb6d8d19a30b45/nh3-0.3.0-cp38-abi3-win_amd64.whl": "bae63772408fd63ad836ec569a7c8f444dd32863d0c67f6e0b25ebbd606afa95", - "https://files.pythonhosted.org/packages/8f/14/079670fb2e848c4ba2476c5a7a2d1319826053f4f0368f61fca9bb4227ae/nh3-0.3.0-cp38-abi3-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl": "7852f038a054e0096dac12b8141191e02e93e0b4608c4b993ec7d4ffafea4e49", - "https://files.pythonhosted.org/packages/97/03/03f79f7e5178eb1ad5083af84faff471e866801beb980cc72943a4397368/nh3-0.3.0-cp38-abi3-musllinux_1_2_i686.whl": "c7a32a7f0d89f7d30cb8f4a84bdbd56d1eb88b78a2434534f62c71dac538c450", - "https://files.pythonhosted.org/packages/97/33/11e7273b663839626f714cb68f6eb49899da5a0d9b6bc47b41fe870259c2/nh3-0.3.0-cp38-abi3-manylinux_2_17_armv7l.manylinux2014_armv7l.whl": "389d93d59b8214d51c400fb5b07866c2a4f79e4e14b071ad66c92184fec3a392", - "https://files.pythonhosted.org/packages/9a/e0/af86d2a974c87a4ba7f19bc3b44a8eaa3da480de264138fec82fe17b340b/nh3-0.3.0-cp313-cp313t-win_arm64.whl": "16f8670201f7e8e0e05ed1a590eb84bfa51b01a69dd5caf1d3ea57733de6a52f", - "https://files.pythonhosted.org/packages/a3/e5/ac7fc565f5d8bce7f979d1afd68e8cb415020d62fa6507133281c7d49f91/nh3-0.3.0-cp38-abi3-manylinux_2_17_s390x.manylinux2014_s390x.whl": "af5aa8127f62bbf03d68f67a956627b1bd0469703a35b3dad28d0c1195e6c7fb", - "https://files.pythonhosted.org/packages/ad/7f/7c6b8358cf1222921747844ab0eef81129e9970b952fcb814df417159fb9/nh3-0.3.0-cp313-cp313t-manylinux_2_17_x86_64.manylinux2014_x86_64.whl": "7c915060a2c8131bef6a29f78debc29ba40859b6dbe2362ef9e5fd44f11487c2", - "https://files.pythonhosted.org/packages/b4/11/340b7a551916a4b2b68c54799d710f86cf3838a4abaad8e74d35360343bb/nh3-0.3.0-cp313-cp313t-macosx_10_12_x86_64.macosx_11_0_arm64.macosx_10_12_universal2.whl": "a537ece1bf513e5a88d8cff8a872e12fe8d0f42ef71dd15a5e7520fecd191bbb", - "https://files.pythonhosted.org/packages/c3/a4/96cff0977357f60f06ec4368c4c7a7a26cccfe7c9fcd54f5378bf0428fd3/nh3-0.3.0.tar.gz": "d8ba24cb31525492ea71b6aac11a4adac91d828aadeff7c4586541bf5dc34d2f", - "https://files.pythonhosted.org/packages/c9/50/76936ec021fe1f3270c03278b8af5f2079038116b5d0bfe8538ffe699d69/nh3-0.3.0-cp38-abi3-win32.whl": "6d68fa277b4a3cf04e5c4b84dd0c6149ff7d56c12b3e3fab304c525b850f613d", - "https://files.pythonhosted.org/packages/ce/55/1974bcc16884a397ee699cebd3914e1f59be64ab305533347ca2d983756f/nh3-0.3.0-cp38-abi3-musllinux_1_2_x86_64.whl": "3f1b4f8a264a0c86ea01da0d0c390fe295ea0bcacc52c2103aca286f6884f518", - "https://files.pythonhosted.org/packages/ee/db/7aa11b44bae4e7474feb1201d8dee04fabe5651c7cb51409ebda94a4ed67/nh3-0.3.0-cp38-abi3-musllinux_1_2_armv7l.whl": "b0612ccf5de8a480cf08f047b08f9d3fecc12e63d2ee91769cb19d7290614c23", - "https://files.pythonhosted.org/packages/f3/ba/59e204d90727c25b253856e456ea61265ca810cda8ee802c35f3fadaab00/nh3-0.3.0-cp313-cp313t-musllinux_1_2_armv7l.whl": "e90883f9f85288f423c77b3f5a6f4486375636f25f793165112679a7b6363b35" - }, - "pkginfo": { - "https://files.pythonhosted.org/packages/24/03/e26bf3d6453b7fda5bd2b84029a426553bb373d6277ef6b5ac8863421f87/pkginfo-1.12.1.2.tar.gz": "5cd957824ac36f140260964eba3c6be6442a8359b8c48f4adf90210f33a04b7b", - "https://files.pythonhosted.org/packages/fa/3d/f4f2ba829efb54b6cd2d91349c7463316a9cc55a43fc980447416c88540f/pkginfo-1.12.1.2-py3-none-any.whl": "c783ac885519cab2c34927ccfa6bf64b5a704d7c69afaea583dd9b7afe969343" + "https://files.pythonhosted.org/packages/09/a1/ea83abe738a3fbaa203dfdb836ca7cbab0e7e9609faaee4fe1d4652599c0/nh3-0.3.6-cp314-cp314t-musllinux_1_2_i686.whl": "edb2b4a1a27523e6cc7c417f8d21ce3d005243548b93e56b762b66b0c7f589f9", + "https://files.pythonhosted.org/packages/0a/13/6f1e302ca674ac74362e150848ad56a1be5145391204f74facdb8e94df12/nh3-0.3.6-cp38-abi3-manylinux_2_17_s390x.manylinux2014_s390x.whl": "455469a29951edc92bc48b47ac2281c3f2609e6c4f6a047056449f8c2c23facf", + "https://files.pythonhosted.org/packages/0e/24/a0d80182a18919665fefd19c1c06f1d1df1c9a6455d0252de40c034a0bc3/nh3-0.3.6-cp38-abi3-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl": "e6b7beece07525dc6e6b0fc2f104442de2ba328360ad00e50cbe2e1fd620447d", + "https://files.pythonhosted.org/packages/11/f9/3966c61455668c08853bf5e33b4bed93c421f3194ce4de896dc248d6f6ce/nh3-0.3.6-cp38-abi3-musllinux_1_2_armv7l.whl": "f5ed5fe84aee7f39db95c214a7421bf0499fbf500fec6d86a4e29bfc37971438", + "https://files.pythonhosted.org/packages/17/0c/6cdb5ee1e127be50dc8391e54bddc1f64e87bf4bfad0c55633320e2e02db/nh3-0.3.6-cp38-abi3-musllinux_1_2_x86_64.whl": "36d06341bd501240d320f5942481ed5e6846136b666e1ba4faf802b78ebc875f", + "https://files.pythonhosted.org/packages/19/d3/479cb4ae440424825735d60525b53e3c77fd60fd6e6afc0e984f00eb0178/nh3-0.3.6-cp38-abi3-musllinux_1_2_i686.whl": "082675ff87b9385ec430ffe6d5847ba7456cc39b73720cd4add472f9f4cffd56", + "https://files.pythonhosted.org/packages/25/bb/431615ba1d1d3eb63cde0f974f2114edf863a8a3f6049a12fed23fc241d3/nh3-0.3.6-cp38-abi3-manylinux_2_17_ppc64.manylinux2014_ppc64.whl": "44673b27010051ab5a5e438a86ec31bbda61d4a77d7e900af6b7be3037c1abae", + "https://files.pythonhosted.org/packages/30/a8/fb2c38845efb703a9173bffdfc745fc64d2b0e55cfc73a3647d2f028250c/nh3-0.3.6-cp38-abi3-manylinux_2_5_i686.manylinux1_i686.whl": "2f90d9a0cfdbee218994fdaaeeb5a0fde62d08f35e4eef0378ec1e2200172fd0", + "https://files.pythonhosted.org/packages/36/ea/5542f3c45da4c00290d9d67a65e996702e23e613c4b627de3e09cb9fe357/nh3-0.3.6-cp314-cp314t-win_amd64.whl": "4713502748f564fee0633b37b3403783ce0a3af3a3d148ad91025a5bdadb7bc6", + "https://files.pythonhosted.org/packages/3c/a6/bfaa00046e58603507dcfc266c4778e3ab7adf68a5dedd73b6274b8d9314/nh3-0.3.6-cp38-abi3-manylinux_2_31_riscv64.whl": "25c733bee928530556b1db0ea46c52cf5aa686146e38e60a6fc7cb801ef91cec", + "https://files.pythonhosted.org/packages/41/21/e1084ab18eb589506335c7c7576f2d4643e9a0c0e33983ef0e549a256b96/nh3-0.3.6-cp38-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl": "e1b160831c9cdb06a6c79c2f9cdb11386602938f9af260d1c457a85add4f6f69", + "https://files.pythonhosted.org/packages/49/09/0d8e3101636d9ad88cdefb2914e764cb8e876ebdbb4286bfc251277d9c67/nh3-0.3.6-cp314-cp314t-musllinux_1_2_armv7l.whl": "889932a97fb4abb6f95fef1914c0d269ebfb60011e67121c1163059b9449dbb4", + "https://files.pythonhosted.org/packages/4b/4a/526f199626bfcb496bc01a268051b44737962005553b158e985ed7e64865/nh3-0.3.6-cp314-cp314t-musllinux_1_2_aarch64.whl": "f2f14b7ae1fca99c4a66c981aac3974e7fbc1ca30a12673d223ae1df76680917", + "https://files.pythonhosted.org/packages/59/62/5b6108bedaef2b2637fed04c87bdbcb5967b9961758b41f0e466ef22a022/nh3-0.3.6-cp314-cp314t-manylinux_2_5_i686.manylinux1_i686.whl": "34d2b0d934156b87ee114f599a3ba9b8b9e17b5d79652ba3a13fa50903de965e", + "https://files.pythonhosted.org/packages/5b/67/314f6151bad77a93d751978a344033e1fc890822f05f0416079338e34231/nh3-0.3.6-cp38-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl": "905f877dc66dd7aea4a76e54bcb26acb5ff8216f720c0017ccf63e0e6035698e", + "https://files.pythonhosted.org/packages/5e/1b/ef84624f14954d270f74060a19fc550dd4f06656399447569afb584d8c06/nh3-0.3.6.tar.gz": "f3736c9dd3d1856f80cd031715b84ca75cda2bbb1ac802c3da26bfce590838d7", + "https://files.pythonhosted.org/packages/66/35/26bd47e6af5915a628281dccdac354ddf4e32f7397047894270acd8c9870/nh3-0.3.6-cp314-cp314t-win_arm64.whl": "69bbb92865a693d909db3a700d3c01537533844d0948c1e9323561ce06ecda41", + "https://files.pythonhosted.org/packages/66/69/0654482b8635012fbae67826bd6c381abb05d841ac7388b9b4666300fdad/nh3-0.3.6-cp314-cp314t-musllinux_1_2_x86_64.whl": "43bc1ed3fa0716295fabee29ba42b2667e4a51d140b0a68e092170a765474fa6", + "https://files.pythonhosted.org/packages/68/17/06e72a18ee9b572914447338237ca7eb164c0df901f141bc10d1282247a2/nh3-0.3.6-cp38-abi3-musllinux_1_2_aarch64.whl": "82ca5bf427ad1b216b65ede1a2e2d87dc49bec417ceba0f297213107d3cd9d78", + "https://files.pythonhosted.org/packages/7b/e5/7cafee2f0413ca4cb0ef3bd111e94d408a48810008b283ad8aee00dd1809/nh3-0.3.6-cp38-abi3-win_arm64.whl": "69f365963f63a1e9bff53bdbb3c542c7c2efed3e163c9d5d83a772a2ac468c21", + "https://files.pythonhosted.org/packages/82/fa/2b5d684e3edf1e81bfd02d298c78c3e3da77ca1d8a2be3183a79544a7548/nh3-0.3.6-cp38-abi3-win_amd64.whl": "f338ac7d594c067679f1e99b4f5ec3906842979560f9d8f15d6bdfa39a353b10", + "https://files.pythonhosted.org/packages/99/3e/6506aa4f23dc7b7993a2d0a45dca3ce864ec48380adfe15a173e643c63e8/nh3-0.3.6-cp314-cp314t-macosx_10_12_x86_64.macosx_11_0_arm64.macosx_10_12_universal2.whl": "2411e8c3cee81a1ddd62c2a5d50585c28aa5566d373ad1db92536b95ddb24ef2", + "https://files.pythonhosted.org/packages/b0/94/f48d08e6f72a406300fa11d8acd929fea1a80d4bf750fa292cb10785f126/nh3-0.3.6-cp38-abi3-manylinux_2_17_armv7l.manylinux2014_armv7l.whl": "d14bf7982e7a77c0c775634c29c07ce08b38a046df73e1c1f139b3e82f18a38e", + "https://files.pythonhosted.org/packages/e3/e1/e96e7864a7a53bd6b6fab7e9632467382a2a2c1f3fed951918ad131542fb/nh3-0.3.6-cp314-cp314t-manylinux_2_17_x86_64.manylinux2014_x86_64.whl": "e196fa70c2ff2eb4de7d3df3108f8f358c1d69dff20d45b11f20a5aa227ffb6d", + "https://files.pythonhosted.org/packages/e9/55/9de666ad975d6ccd77d799ea0add55ee2347aa81286ce21b2a97c070746b/nh3-0.3.6-cp38-abi3-win32.whl": "5276ef17bdba9ad8040575c74072008b13aae429436e9d0429e718bb5f90f4da", + "https://files.pythonhosted.org/packages/ed/a6/1f7285ffadc8307c4dbeb08d21b920536d5117785056d1079e998c4dfa44/nh3-0.3.6-cp314-cp314t-win32.whl": "597a8e843bea00b2eb5520658dc24a9bb032e7fc9e7c2c0c4cd29420220c9796", + "https://files.pythonhosted.org/packages/f3/ab/a7653bce9a3b204be6a6931767a9e23595807bb84790ce6685e4d7e5bd08/nh3-0.3.6-cp38-abi3-macosx_10_12_x86_64.macosx_11_0_arm64.macosx_10_12_universal2.whl": "a43ebd7543555c3ac1bc353023d0794e75cb76f6f18f19c32e95441496c0cc25" + }, + "packaging": { + "https://files.pythonhosted.org/packages/d7/f1/e7a6dd94a8d4a5626c03e4e99c87f241ba9e350cd9e6d75123f992427270/packaging-26.2.tar.gz": "ff452ff5a3e828ce110190feff1178bb1f2ea2281fa2075aadb987c2fb221661", + "https://files.pythonhosted.org/packages/df/b2/87e62e8c3e2f4b32e5fe99e0b86d576da1312593b39f47d8ceef365e95ed/packaging-26.2-py3-none-any.whl": "5fc45236b9446107ff2415ce77c807cee2862cb6fac22b8a73826d0693b0980e" }, "pycparser": { "https://files.pythonhosted.org/packages/0c/c3/44f3fbbfa403ea2a7c779186dc20772604442dde72947e7d01069cbe98e3/pycparser-3.0-py3-none-any.whl": "b727414169a36b7d524c1c3e31839a521725078d7b2ff038656844266160a992", - "https://files.pythonhosted.org/packages/1b/7d/92392ff7815c21062bea51aa7b87d45576f649f16458d78b7cf94b9ab2e6/pycparser-3.0.tar.gz": "600f49d217304a5902ac3c37e1281c9fe94e4d0489de643a9504c5cdfdfc6b29", - "https://files.pythonhosted.org/packages/a0/e3/59cd50310fc9b59512193629e1984c1f95e5c8ae6e5d8c69532ccc65a7fe/pycparser-2.23-py3-none-any.whl": "e5c6e8d3fbad53479cab09ac03729e0a9faf2bee3db8208a550daf5af81a5934", - "https://files.pythonhosted.org/packages/fe/cf/d2d3b9f5699fb1e4615c8e32ff220203e43b248e1dfcc6736ad9057731ca/pycparser-2.23.tar.gz": "78816d4f24add8f10a06d6f05b4d424ad9e96cfebf68a4ddc99c65c0720d00c2" + "https://files.pythonhosted.org/packages/1b/7d/92392ff7815c21062bea51aa7b87d45576f649f16458d78b7cf94b9ab2e6/pycparser-3.0.tar.gz": "600f49d217304a5902ac3c37e1281c9fe94e4d0489de643a9504c5cdfdfc6b29" }, "pygments": { - "https://files.pythonhosted.org/packages/b0/77/a5b8c569bf593b0140bde72ea885a803b82086995367bf2037de0159d924/pygments-2.19.2.tar.gz": "636cb2477cec7f8952536970bc533bc43743542f70392ae026374600add5b887", - "https://files.pythonhosted.org/packages/c7/21/705964c7812476f378728bdf590ca4b771ec72385c533964653c68e86bdc/pygments-2.19.2-py3-none-any.whl": "86540386c03d588bb81d44bc3928634ff26449851e99741617ecb9037ee5ec0b" + "https://files.pythonhosted.org/packages/c3/b2/bc9c9196916376152d655522fdcebac55e66de6603a76a02bca1b6414f6c/pygments-2.20.0.tar.gz": "6757cd03768053ff99f3039c1a36d6c0aa0b263438fcab17520b30a303a82b5f", + "https://files.pythonhosted.org/packages/f4/7e/a72dd26f3b0f4f2bf1dd8923c85f7ceb43172af56d63c7383eb62b332364/pygments-2.20.0-py3-none-any.whl": "81a9e26dd42fd28a23a2d169d86d7ac03b46e2f8b59ed4698fb4785f946d0176" }, "pywin32-ctypes": { "https://files.pythonhosted.org/packages/85/9f/01a1a99704853cb63f253eea009390c88e7131c67e66a0a02099a8c917cb/pywin32-ctypes-0.2.3.tar.gz": "d162dc04946d704503b2edc4d55f3dba5c1d539ead017afa00142c38b9885755", "https://files.pythonhosted.org/packages/de/3d/8161f7711c017e01ac9f008dfddd9410dff3674334c233bde66e7ba65bbf/pywin32_ctypes-0.2.3-py3-none-any.whl": "8a1513379d709975552d202d942d9837758905c8d01eb82b8bcc30918929e7b8" }, "readme-renderer": { - "https://files.pythonhosted.org/packages/5a/a9/104ec9234c8448c4379768221ea6df01260cd6c2ce13182d4eac531c8342/readme_renderer-44.0.tar.gz": "8712034eabbfa6805cacf1402b4eeb2a73028f72d1166d6f5cb7f9c047c5d1e1", - "https://files.pythonhosted.org/packages/e1/67/921ec3024056483db83953ae8e48079ad62b92db7880013ca77632921dd0/readme_renderer-44.0-py3-none-any.whl": "2fbca89b81a08526aadf1357a8c2ae889ec05fb03f5da67f9769c9a592166151" + "https://files.pythonhosted.org/packages/02/51/d3a6ea424652c60f05600d8c2e01a55c913755e7cdad64afabbd1aa16f44/readme_renderer-45.0.tar.gz": "030a8fac74904f8fba11ad1bb6964e3f76e896dc7e5e71f16af190c9056696d1", + "https://files.pythonhosted.org/packages/97/1b/295bf2fa3e740131778065e5ffa2c481f0e7210182d408e9a2c244ff5b0c/readme_renderer-45.0-py3-none-any.whl": "3385ed220117104a2bceb4a9dac8c5fdf6d1f96890d7ea2a9c7174fd5c84091f" }, "requests": { - "https://files.pythonhosted.org/packages/34/64/8860370b167a9721e8956ae116825caff829224fbca0ca6e7bf8ddef8430/requests-2.33.0.tar.gz": "c7ebc5e8b0f21837386ad0e1c8fe8b829fa5f544d8df3b2253bff14ef29d7652", - "https://files.pythonhosted.org/packages/56/5d/c814546c2333ceea4ba42262d8c4d55763003e767fa169adc693bd524478/requests-2.33.0-py3-none-any.whl": "3324635456fa185245e24865e810cecec7b4caf933d7eb133dcde67d48cee69b" + "https://files.pythonhosted.org/packages/a0/f4/c67b0b3f1b9245e8d266f0f112c500d50e5b4e83cb6f3b71b6528104182a/requests-2.34.2-py3-none-any.whl": "2a0d60c172f83ac6ab31e4554906c0f3b3588d37b5cb939b1c061f4907e278e0", + "https://files.pythonhosted.org/packages/ac/c3/e2a2b89f2d3e2179abd6d00ebd70bff6273f37fb3e0cc209f48b39d00cbf/requests-2.34.2.tar.gz": "f288924cae4e29463698d6d60bc6a4da69c89185ad1e0bcc4104f584e960b9ed" }, "requests-toolbelt": { "https://files.pythonhosted.org/packages/3f/51/d4db610ef29373b879047326cbf6fa98b6c1969d6f6dc423279de2b1be2c/requests_toolbelt-1.0.0-py2.py3-none-any.whl": "cccfdd665f0a24fcf4726e690f65639d272bb0637b9b92dfd91a5568ccf6bd06", @@ -632,28 +648,28 @@ "https://files.pythonhosted.org/packages/ff/9a/9afaade874b2fa6c752c36f1548f718b5b83af81ed9b76628329dab81c1b/rfc3986-2.0.0-py2.py3-none-any.whl": "50b1502b60e289cb37883f3dfd34532b8873c7de9f49bb546641ce9cbd256ebd" }, "rich": { - "https://files.pythonhosted.org/packages/e3/30/3c4d035596d3cf444529e0b2953ad0466f6049528a879d27534700580395/rich-14.1.0-py3-none-any.whl": "536f5f1785986d6dbdea3c75205c473f970777b4a0d6c6dd1b696aa05a3fa04f", - "https://files.pythonhosted.org/packages/fe/75/af448d8e52bf1d8fa6a9d089ca6c07ff4453d86c65c145d0a300bb073b9b/rich-14.1.0.tar.gz": "e497a48b844b0320d45007cdebfeaeed8db2a4f4bcf49f15e455cfc4af11eaa8" + "https://files.pythonhosted.org/packages/82/3b/64d4899d73f91ba49a8c18a8ff3f0ea8f1c1d75481760df8c68ef5235bf5/rich-15.0.0-py3-none-any.whl": "33bd4ef74232fb73fe9279a257718407f169c09b78a87ad3d296f548e27de0bb", + "https://files.pythonhosted.org/packages/c0/8f/0722ca900cc807c13a6a0c696dacf35430f72e0ec571c4275d2371fca3e9/rich-15.0.0.tar.gz": "edd07a4824c6b40189fb7ac9bc4c52536e9780fbbfbddf6f1e2502c31b068c36" }, "secretstorage": { - "https://files.pythonhosted.org/packages/53/a4/f48c9d79cb507ed1373477dbceaba7401fd8a23af63b837fa61f1dcd3691/SecretStorage-3.3.3.tar.gz": "2403533ef369eca6d2ba81718576c5e0f564d5cca1b58f73a8b23e7d4eeebd77", - "https://files.pythonhosted.org/packages/54/24/b4293291fa1dd830f353d2cb163295742fa87f179fcc8a20a306a81978b7/SecretStorage-3.3.3-py3-none-any.whl": "f356e6628222568e3af06f2eba8df495efa13b3b63081dafd4f7d9a7b7bc9f99" + "https://files.pythonhosted.org/packages/1c/03/e834bcd866f2f8a49a85eaff47340affa3bfa391ee9912a952a1faa68c7b/secretstorage-3.5.0.tar.gz": "f04b8e4689cbce351744d5537bf6b1329c6fc68f91fa666f60a380edddcd11be", + "https://files.pythonhosted.org/packages/b7/46/f5af3402b579fd5e11573ce652019a67074317e18c1935cc0b4ba9b35552/secretstorage-3.5.0-py3-none-any.whl": "0ce65888c0725fcb2c5bc0fdb8e5438eece02c523557ea40ce0703c266248137" }, "six": { "https://files.pythonhosted.org/packages/94/e7/b2c673351809dca68a0e064b6af791aa332cf192da575fd474ed7d6f16a2/six-1.17.0.tar.gz": "ff70335d468e7eb6ec65b95b99d3a2836546063f63acc5171de367e834932a81", "https://files.pythonhosted.org/packages/b7/ce/149a00dd41f10bc29e5921b496af8b574d8413afcd5e30dfa0ed46c2cc5e/six-1.17.0-py2.py3-none-any.whl": "4721f391ed90541fddacab5acf947aa0d3dc7d27b2e1e8eda2be8970586c3274" }, "twine": { - "https://files.pythonhosted.org/packages/5d/ec/00f9d5fd040ae29867355e559a94e9a8429225a0284a3f5f091a3878bfc0/twine-5.1.1-py3-none-any.whl": "215dbe7b4b94c2c50a7315c0275d2258399280fbb7d04182c7e55e24b5f93997", - "https://files.pythonhosted.org/packages/77/68/bd982e5e949ef8334e6f7dcf76ae40922a8750aa2e347291ae1477a4782b/twine-5.1.1.tar.gz": "9aa0825139c02b3434d913545c7b847a21c835e11597f5255842d457da2322db" + "https://files.pythonhosted.org/packages/92/3c/58f808a359700f39a967dffede33efeac809262c03303fa3eec6afff8f49/twine-7.0.0.tar.gz": "85cdb29c518efef867360ae4acd4b0dfd61c8654a22fca08e6f8539f05022177", + "https://files.pythonhosted.org/packages/96/08/ddcdc06225eaad6de0e48e1002b06d919dbde20582d0662c7af51308e5d6/twine-7.0.0-py3-none-any.whl": "b854164df26db268af05f49aa5c0344b10e27a494343ff05b1e0bad3b135f5a7" }, "urllib3": { - "https://files.pythonhosted.org/packages/39/08/aaaad47bc4e9dc8c725e68f9d04865dbcb2052843ff09c97b08904852d84/urllib3-2.6.3-py3-none-any.whl": "bf272323e553dfb2e87d9bfd225ca7b0f467b919d7bbd355436d3fd37cb0acd4", - "https://files.pythonhosted.org/packages/c7/24/5f1b3bdffd70275f6661c76461e25f024d5a38a46f04aaca912426a2b1d3/urllib3-2.6.3.tar.gz": "1b62b6884944a57dbe321509ab94fd4d3b307075e0c2eae991ac71ee15ad38ed" + "https://files.pythonhosted.org/packages/53/0c/06f8b233b8fd13b9e5ee11424ef85419ba0d8ba0b3138bf360be2ff56953/urllib3-2.7.0.tar.gz": "231e0ec3b63ceb14667c67be60f2f2c40a518cb38b03af60abc813da26505f4c", + "https://files.pythonhosted.org/packages/7f/3e/5db95bcf282c52709639744ca2a8b149baccf648e39c8cc87553df9eae0c/urllib3-2.7.0-py3-none-any.whl": "9fb4c81ebbb1ce9531cce37674bbc6f1360472bc18ca9a553ede278ef7276897" }, "zipp": { - "https://files.pythonhosted.org/packages/2e/54/647ade08bf0db230bfea292f893923872fd20be6ac6f53b2b936ba839d75/zipp-3.23.0-py3-none-any.whl": "071652d6115ed432f5ce1d34c336c0adfd6a884660d1e9712a256d3d3bd4b14e", - "https://files.pythonhosted.org/packages/e3/02/0f2892c661036d50ede074e376733dca2ae7c6eb617489437771209d4180/zipp-3.23.0.tar.gz": "a07157588a12518c9d4034df3fbbee09c814741a33ff63c05fa29d26a2404166" + "https://files.pythonhosted.org/packages/3a/13/547360d81e6d88d58492968ffda9f9542854f11310ee556fef14260cc886/zipp-4.1.0-py3-none-any.whl": "25ad4e16390cd314347dd8f1de67a2ac538ae658ed4ab9db16029c07c188e97f", + "https://files.pythonhosted.org/packages/b9/d8/eab98a517c14134c0b2eb4e2387bc5f457334293ec5d2dd3857ec2966802/zipp-4.1.0.tar.gz": "4cb57381f544315db7688e976e922a2b18cdb513d21cc194eb42232ba2a3e602" } } }, diff --git a/tools/publish/requirements_darwin.txt b/tools/publish/requirements_darwin.txt index dd292fbaee..5022924d80 100644 --- a/tools/publish/requirements_darwin.txt +++ b/tools/publish/requirements_darwin.txt @@ -6,9 +6,9 @@ backports-tarfile==1.2.0 \ --hash=sha256:77e284d754527b01fb1e6fa8a1afe577858ebe4e9dad8919e34c862cb399bc34 \ --hash=sha256:d75e02c268746e1b8144c278978b6e98e85de6ad16f8e4b0844a154557eca991 # via jaraco-context -certifi==2025.10.5 \ - --hash=sha256:0f212c2744a9bb6de0c56639a6f68afe01ecd92d91f14ae897c4fe7bbeeef0de \ - --hash=sha256:47c09d31ccf2acf0be3f701ea53595ee7e0b8fa08801c6624be771df09ae7b43 +certifi==2026.7.22 \ + --hash=sha256:62f22742b58a1a33014a2b6b706588a8d7e2a88ae7bd1a6ebe8c992928483775 \ + --hash=sha256:741e2c3b351ddf169a738da9f2c048608ff7f2c5cc02f1ebc6b118bb090d5d55 # via requests charset-normalizer==3.4.9 \ --hash=sha256:0327fcd59a935777d83410750c50600ee9571af2846f71ce40f25b13da1ef380 \ @@ -105,95 +105,98 @@ charset-normalizer==3.4.9 \ --hash=sha256:fa36ec09ef71d158186bc79e359ff5fdd6e7996fe8ab638f00d6b93139ba4fcf \ --hash=sha256:fe2c7201c642b7c308f1675355ad7ff7b66acfe3541625efe5a3ad38f29d6115 # via requests -docutils==0.22.2 \ - --hash=sha256:9fdb771707c8784c8f2728b67cb2c691305933d68137ef95a75db5f4dfbc213d \ - --hash=sha256:b0e98d679283fc3bb0ead8a5da7f501baa632654e7056e9c5846842213d674d8 +docutils==0.23 \ + --hash=sha256:25d013af9bf23bc1c7b2b093dff4208166c53a94786c9e447808335ef1185fea \ + --hash=sha256:746f5060322511280a1e50eb76846ed6bf2342984b2ac04dc42caa1a8d78799e # via readme-renderer -idna==3.10 \ - --hash=sha256:12f65c9b470abda6dc35cf8e63cc574b1c52b11df2c86030af0ac09b01b13ea9 \ - --hash=sha256:946d195a0d259cbba61165e88e65941f16e9b36ea6ddb97f00452bae8b1287d3 +id==1.6.1 \ + --hash=sha256:d0732d624fb46fd4e7bc4e5152f00214450953b9e772c182c1c22964def1a069 \ + --hash=sha256:f5ec41ed2629a508f5d0988eda142e190c9c6da971100612c4de9ad9f9b237ca + # via twine +idna==3.18 \ + --hash=sha256:7f952cbe720b688055e3f87de14f5c3e5fdaa8bc3928985c4077ca689de849a2 \ + --hash=sha256:ffb385a7e039654cef1ab9ef32c6fafe283c0c0467bba1d9029738ce4a14a848 # via requests -importlib-metadata==8.7.0 \ - --hash=sha256:d13b81ad223b890aa16c5471f2ac3056cf76c5f10f82d6f9292f0b415f389000 \ - --hash=sha256:e5dd1551894c77868a30651cef00984d50e1002d06942a7101d34870c5f02afd - # via - # keyring - # twine +importlib-metadata==9.0.0 \ + --hash=sha256:2d21d1cc5a017bd0559e36150c21c830ab1dc304dedd1b7ea85d20f45ef3edd7 \ + --hash=sha256:a4f57ab599e6a2e3016d7595cfd72eb4661a5106e787a95bcc90c7105b831efc + # via keyring jaraco-classes==3.4.0 \ --hash=sha256:47a024b51d0239c0dd8c8540c6c7f484be3b8fcf0b2d85c13825780d3b3f3acd \ --hash=sha256:f662826b6bed8cace05e7ff873ce0f9283b5c924470fe664fff1c2f00f581790 # via keyring -jaraco-context==6.0.1 \ - --hash=sha256:9bae4ea555cf0b14938dc0aee7c9f32ed303aa20a3b73e7dc80111628792d1b3 \ - --hash=sha256:f797fc481b490edb305122c9181830a3a5b76d84ef6d1aef2fb9b47ab956f9e4 +jaraco-context==6.1.2 \ + --hash=sha256:bf8150b79a2d5d91ae48629d8b427a8f7ba0e1097dd6202a9059f29a36379535 \ + --hash=sha256:f1a6c9d391e661cc5b8d39861ff077a7dc24dc23833ccee564b234b81c82dfe3 # via keyring -jaraco-functools==4.3.0 \ - --hash=sha256:227ff8ed6f7b8f62c56deff101545fa7543cf2c8e7b82a7c2116e672f29c26e8 \ - --hash=sha256:cfd13ad0dd2c47a3600b439ef72d8615d482cedcff1632930d6f28924d92f294 +jaraco-functools==4.6.0 \ + --hash=sha256:880c577ec9720b3a052d5bc611fb9f2269b3d87902ef42440df443b88e443280 \ + --hash=sha256:99e3dc0060c5cbe8fcd1cdb36258e2a65ca40f1566b2033b12abb1bb44dd3c30 # via keyring -keyring==25.6.0 \ - --hash=sha256:0b39998aa941431eb3d9b0d4b2460bc773b9df6fed7621c2dfb291a7e0187a66 \ - --hash=sha256:552a3f7af126ece7ed5c89753650eec89c7eaae8617d0aa4d9ad2b75111266bd +keyring==25.7.0 \ + --hash=sha256:be4a0b195f149690c166e850609a477c532ddbfbaed96a404d4e43f8d5e2689f \ + --hash=sha256:fe01bd85eb3f8fb3dd0405defdeac9a5b4f6f0439edbb3149577f244a2e8245b # via twine -markdown-it-py==4.0.0 \ - --hash=sha256:87327c59b172c5011896038353a81343b6754500a08cd7a4973bb48c6d578147 \ - --hash=sha256:cb0a2b4aa34f932c007117b194e945bd74e0ec24133ceb5bac59009cda1cb9f3 +markdown-it-py==4.2.0 \ + --hash=sha256:04a21681d6fbb623de53f6f364d352309d4094dd4194040a10fd51833e418d49 \ + --hash=sha256:9f7ebbcd14fe59494226453aed97c1070d83f8d24b6fc3a3bcf9a38092641c4a # via rich mdurl==0.1.2 \ --hash=sha256:84008a41e51615a49fc9966191ff91509e3c40b939176e643fd50a5c2196b8f8 \ --hash=sha256:bb413d29f5eea38f31dd4754dd7377d4465116fb207585f97bf925588687c1ba # via markdown-it-py -more-itertools==10.8.0 \ - --hash=sha256:52d4362373dcf7c52546bc4af9a86ee7c4579df9a8dc268be0a2f949d376cc9b \ - --hash=sha256:f638ddf8a1a0d134181275fb5d58b086ead7c6a72429ad725c67503f13ba30bd +more-itertools==11.1.0 \ + --hash=sha256:48e8f4d9e7e5878571ecf6f2b4e57634f93cd474cc8cfbd2376f2d11b396e30d \ + --hash=sha256:4b65538ae22f6fed0ce4874efd317463a7489796a0939fa66824dd542125a192 # via # jaraco-classes # jaraco-functools -nh3==0.3.0 \ - --hash=sha256:0649464ac8eee018644aacbc103874ccbfac80e3035643c3acaab4287e36e7f5 \ - --hash=sha256:16f8670201f7e8e0e05ed1a590eb84bfa51b01a69dd5caf1d3ea57733de6a52f \ - --hash=sha256:1adeb1062a1c2974bc75b8d1ecb014c5fd4daf2df646bbe2831f7c23659793f9 \ - --hash=sha256:37d3003d98dedca6cd762bf88f2e70b67f05100f6b949ffe540e189cc06887f9 \ - --hash=sha256:389d93d59b8214d51c400fb5b07866c2a4f79e4e14b071ad66c92184fec3a392 \ - --hash=sha256:3f1b4f8a264a0c86ea01da0d0c390fe295ea0bcacc52c2103aca286f6884f518 \ - --hash=sha256:423201bbdf3164a9e09aa01e540adbb94c9962cc177d5b1cbb385f5e1e79216e \ - --hash=sha256:634e34e6162e0408e14fb61d5e69dbaea32f59e847cfcfa41b66100a6b796f62 \ - --hash=sha256:6d68fa277b4a3cf04e5c4b84dd0c6149ff7d56c12b3e3fab304c525b850f613d \ - --hash=sha256:7275fdffaab10cc5801bf026e3c089d8de40a997afc9e41b981f7ac48c5aa7d5 \ - --hash=sha256:7852f038a054e0096dac12b8141191e02e93e0b4608c4b993ec7d4ffafea4e49 \ - --hash=sha256:7c915060a2c8131bef6a29f78debc29ba40859b6dbe2362ef9e5fd44f11487c2 \ - --hash=sha256:80fe20171c6da69c7978ecba33b638e951b85fb92059259edd285ff108b82a6d \ - --hash=sha256:a537ece1bf513e5a88d8cff8a872e12fe8d0f42ef71dd15a5e7520fecd191bbb \ - --hash=sha256:af5aa8127f62bbf03d68f67a956627b1bd0469703a35b3dad28d0c1195e6c7fb \ - --hash=sha256:b0612ccf5de8a480cf08f047b08f9d3fecc12e63d2ee91769cb19d7290614c23 \ - --hash=sha256:ba0caa8aa184196daa6e574d997a33867d6d10234018012d35f86d46024a2a95 \ - --hash=sha256:bae63772408fd63ad836ec569a7c8f444dd32863d0c67f6e0b25ebbd606afa95 \ - --hash=sha256:c7a32a7f0d89f7d30cb8f4a84bdbd56d1eb88b78a2434534f62c71dac538c450 \ - --hash=sha256:ce5e7185599f89b0e391e2f29cc12dc2e206167380cea49b33beda4891be2fe1 \ - --hash=sha256:d8ba24cb31525492ea71b6aac11a4adac91d828aadeff7c4586541bf5dc34d2f \ - --hash=sha256:d97d3efd61404af7e5721a0e74d81cdbfc6e5f97e11e731bb6d090e30a7b62b2 \ - --hash=sha256:e90883f9f85288f423c77b3f5a6f4486375636f25f793165112679a7b6363b35 \ - --hash=sha256:e9e6a7e4d38f7e8dda9edd1433af5170c597336c1a74b4693c5cb75ab2b30f2a \ - --hash=sha256:ec6cfdd2e0399cb79ba4dcffb2332b94d9696c52272ff9d48a630c5dca5e325a \ - --hash=sha256:f416c35efee3e6a6c9ab7716d9e57aa0a49981be915963a82697952cba1353e1 +nh3==0.3.6 \ + --hash=sha256:082675ff87b9385ec430ffe6d5847ba7456cc39b73720cd4add472f9f4cffd56 \ + --hash=sha256:2411e8c3cee81a1ddd62c2a5d50585c28aa5566d373ad1db92536b95ddb24ef2 \ + --hash=sha256:25c733bee928530556b1db0ea46c52cf5aa686146e38e60a6fc7cb801ef91cec \ + --hash=sha256:2f90d9a0cfdbee218994fdaaeeb5a0fde62d08f35e4eef0378ec1e2200172fd0 \ + --hash=sha256:34d2b0d934156b87ee114f599a3ba9b8b9e17b5d79652ba3a13fa50903de965e \ + --hash=sha256:36d06341bd501240d320f5942481ed5e6846136b666e1ba4faf802b78ebc875f \ + --hash=sha256:43bc1ed3fa0716295fabee29ba42b2667e4a51d140b0a68e092170a765474fa6 \ + --hash=sha256:44673b27010051ab5a5e438a86ec31bbda61d4a77d7e900af6b7be3037c1abae \ + --hash=sha256:455469a29951edc92bc48b47ac2281c3f2609e6c4f6a047056449f8c2c23facf \ + --hash=sha256:4713502748f564fee0633b37b3403783ce0a3af3a3d148ad91025a5bdadb7bc6 \ + --hash=sha256:5276ef17bdba9ad8040575c74072008b13aae429436e9d0429e718bb5f90f4da \ + --hash=sha256:597a8e843bea00b2eb5520658dc24a9bb032e7fc9e7c2c0c4cd29420220c9796 \ + --hash=sha256:69bbb92865a693d909db3a700d3c01537533844d0948c1e9323561ce06ecda41 \ + --hash=sha256:69f365963f63a1e9bff53bdbb3c542c7c2efed3e163c9d5d83a772a2ac468c21 \ + --hash=sha256:82ca5bf427ad1b216b65ede1a2e2d87dc49bec417ceba0f297213107d3cd9d78 \ + --hash=sha256:889932a97fb4abb6f95fef1914c0d269ebfb60011e67121c1163059b9449dbb4 \ + --hash=sha256:905f877dc66dd7aea4a76e54bcb26acb5ff8216f720c0017ccf63e0e6035698e \ + --hash=sha256:a43ebd7543555c3ac1bc353023d0794e75cb76f6f18f19c32e95441496c0cc25 \ + --hash=sha256:d14bf7982e7a77c0c775634c29c07ce08b38a046df73e1c1f139b3e82f18a38e \ + --hash=sha256:e196fa70c2ff2eb4de7d3df3108f8f358c1d69dff20d45b11f20a5aa227ffb6d \ + --hash=sha256:e1b160831c9cdb06a6c79c2f9cdb11386602938f9af260d1c457a85add4f6f69 \ + --hash=sha256:e6b7beece07525dc6e6b0fc2f104442de2ba328360ad00e50cbe2e1fd620447d \ + --hash=sha256:edb2b4a1a27523e6cc7c417f8d21ce3d005243548b93e56b762b66b0c7f589f9 \ + --hash=sha256:f2f14b7ae1fca99c4a66c981aac3974e7fbc1ca30a12673d223ae1df76680917 \ + --hash=sha256:f338ac7d594c067679f1e99b4f5ec3906842979560f9d8f15d6bdfa39a353b10 \ + --hash=sha256:f3736c9dd3d1856f80cd031715b84ca75cda2bbb1ac802c3da26bfce590838d7 \ + --hash=sha256:f5ed5fe84aee7f39db95c214a7421bf0499fbf500fec6d86a4e29bfc37971438 # via readme-renderer -pkginfo==1.12.1.2 \ - --hash=sha256:5cd957824ac36f140260964eba3c6be6442a8359b8c48f4adf90210f33a04b7b \ - --hash=sha256:c783ac885519cab2c34927ccfa6bf64b5a704d7c69afaea583dd9b7afe969343 +packaging==26.2 \ + --hash=sha256:5fc45236b9446107ff2415ce77c807cee2862cb6fac22b8a73826d0693b0980e \ + --hash=sha256:ff452ff5a3e828ce110190feff1178bb1f2ea2281fa2075aadb987c2fb221661 # via twine -pygments==2.19.2 \ - --hash=sha256:636cb2477cec7f8952536970bc533bc43743542f70392ae026374600add5b887 \ - --hash=sha256:86540386c03d588bb81d44bc3928634ff26449851e99741617ecb9037ee5ec0b +pygments==2.20.0 \ + --hash=sha256:6757cd03768053ff99f3039c1a36d6c0aa0b263438fcab17520b30a303a82b5f \ + --hash=sha256:81a9e26dd42fd28a23a2d169d86d7ac03b46e2f8b59ed4698fb4785f946d0176 # via # readme-renderer # rich -readme-renderer==44.0 \ - --hash=sha256:2fbca89b81a08526aadf1357a8c2ae889ec05fb03f5da67f9769c9a592166151 \ - --hash=sha256:8712034eabbfa6805cacf1402b4eeb2a73028f72d1166d6f5cb7f9c047c5d1e1 +readme-renderer==45.0 \ + --hash=sha256:030a8fac74904f8fba11ad1bb6964e3f76e896dc7e5e71f16af190c9056696d1 \ + --hash=sha256:3385ed220117104a2bceb4a9dac8c5fdf6d1f96890d7ea2a9c7174fd5c84091f # via twine -requests==2.33.0 \ - --hash=sha256:3324635456fa185245e24865e810cecec7b4caf933d7eb133dcde67d48cee69b \ - --hash=sha256:c7ebc5e8b0f21837386ad0e1c8fe8b829fa5f544d8df3b2253bff14ef29d7652 +requests==2.34.2 \ + --hash=sha256:2a0d60c172f83ac6ab31e4554906c0f3b3588d37b5cb939b1c061f4907e278e0 \ + --hash=sha256:f288924cae4e29463698d6d60bc6a4da69c89185ad1e0bcc4104f584e960b9ed # via # requests-toolbelt # twine @@ -205,21 +208,22 @@ rfc3986==2.0.0 \ --hash=sha256:50b1502b60e289cb37883f3dfd34532b8873c7de9f49bb546641ce9cbd256ebd \ --hash=sha256:97aacf9dbd4bfd829baad6e6309fa6573aaf1be3f6fa735c8ab05e46cecb261c # via twine -rich==14.1.0 \ - --hash=sha256:536f5f1785986d6dbdea3c75205c473f970777b4a0d6c6dd1b696aa05a3fa04f \ - --hash=sha256:e497a48b844b0320d45007cdebfeaeed8db2a4f4bcf49f15e455cfc4af11eaa8 +rich==15.0.0 \ + --hash=sha256:33bd4ef74232fb73fe9279a257718407f169c09b78a87ad3d296f548e27de0bb \ + --hash=sha256:edd07a4824c6b40189fb7ac9bc4c52536e9780fbbfbddf6f1e2502c31b068c36 # via twine -twine==5.1.1 \ - --hash=sha256:215dbe7b4b94c2c50a7315c0275d2258399280fbb7d04182c7e55e24b5f93997 \ - --hash=sha256:9aa0825139c02b3434d913545c7b847a21c835e11597f5255842d457da2322db +twine==7.0.0 \ + --hash=sha256:85cdb29c518efef867360ae4acd4b0dfd61c8654a22fca08e6f8539f05022177 \ + --hash=sha256:b854164df26db268af05f49aa5c0344b10e27a494343ff05b1e0bad3b135f5a7 # via -r tools/publish/requirements.in -urllib3==2.6.3 \ - --hash=sha256:1b62b6884944a57dbe321509ab94fd4d3b307075e0c2eae991ac71ee15ad38ed \ - --hash=sha256:bf272323e553dfb2e87d9bfd225ca7b0f467b919d7bbd355436d3fd37cb0acd4 +urllib3==2.7.0 \ + --hash=sha256:231e0ec3b63ceb14667c67be60f2f2c40a518cb38b03af60abc813da26505f4c \ + --hash=sha256:9fb4c81ebbb1ce9531cce37674bbc6f1360472bc18ca9a553ede278ef7276897 # via + # id # requests # twine -zipp==3.23.0 \ - --hash=sha256:071652d6115ed432f5ce1d34c336c0adfd6a884660d1e9712a256d3d3bd4b14e \ - --hash=sha256:a07157588a12518c9d4034df3fbbee09c814741a33ff63c05fa29d26a2404166 +zipp==4.1.0 \ + --hash=sha256:25ad4e16390cd314347dd8f1de67a2ac538ae658ed4ab9db16029c07c188e97f \ + --hash=sha256:4cb57381f544315db7688e976e922a2b18cdb513d21cc194eb42232ba2a3e602 # via importlib-metadata diff --git a/tools/publish/requirements_linux.txt b/tools/publish/requirements_linux.txt index cfba2aceea..41b0755b12 100644 --- a/tools/publish/requirements_linux.txt +++ b/tools/publish/requirements_linux.txt @@ -6,95 +6,111 @@ backports-tarfile==1.2.0 \ --hash=sha256:77e284d754527b01fb1e6fa8a1afe577858ebe4e9dad8919e34c862cb399bc34 \ --hash=sha256:d75e02c268746e1b8144c278978b6e98e85de6ad16f8e4b0844a154557eca991 # via jaraco-context -certifi==2025.10.5 \ - --hash=sha256:0f212c2744a9bb6de0c56639a6f68afe01ecd92d91f14ae897c4fe7bbeeef0de \ - --hash=sha256:47c09d31ccf2acf0be3f701ea53595ee7e0b8fa08801c6624be771df09ae7b43 +certifi==2026.7.22 \ + --hash=sha256:62f22742b58a1a33014a2b6b706588a8d7e2a88ae7bd1a6ebe8c992928483775 \ + --hash=sha256:741e2c3b351ddf169a738da9f2c048608ff7f2c5cc02f1ebc6b118bb090d5d55 # via requests -cffi==2.0.0 \ - --hash=sha256:00bdf7acc5f795150faa6957054fbbca2439db2f775ce831222b66f192f03beb \ - --hash=sha256:07b271772c100085dd28b74fa0cd81c8fb1a3ba18b21e03d7c27f3436a10606b \ - --hash=sha256:087067fa8953339c723661eda6b54bc98c5625757ea62e95eb4898ad5e776e9f \ - --hash=sha256:0a1527a803f0a659de1af2e1fd700213caba79377e27e4693648c2923da066f9 \ - --hash=sha256:0cf2d91ecc3fcc0625c2c530fe004f82c110405f101548512cce44322fa8ac44 \ - --hash=sha256:0f6084a0ea23d05d20c3edcda20c3d006f9b6f3fefeac38f59262e10cef47ee2 \ - --hash=sha256:12873ca6cb9b0f0d3a0da705d6086fe911591737a59f28b7936bdfed27c0d47c \ - --hash=sha256:19f705ada2530c1167abacb171925dd886168931e0a7b78f5bffcae5c6b5be75 \ - --hash=sha256:1cd13c99ce269b3ed80b417dcd591415d3372bcac067009b6e0f59c7d4015e65 \ - --hash=sha256:1e3a615586f05fc4065a8b22b8152f0c1b00cdbc60596d187c2a74f9e3036e4e \ - --hash=sha256:1f72fb8906754ac8a2cc3f9f5aaa298070652a0ffae577e0ea9bd480dc3c931a \ - --hash=sha256:1fc9ea04857caf665289b7a75923f2c6ed559b8298a1b8c49e59f7dd95c8481e \ - --hash=sha256:203a48d1fb583fc7d78a4c6655692963b860a417c0528492a6bc21f1aaefab25 \ - --hash=sha256:2081580ebb843f759b9f617314a24ed5738c51d2aee65d31e02f6f7a2b97707a \ - --hash=sha256:21d1152871b019407d8ac3985f6775c079416c282e431a4da6afe7aefd2bccbe \ - --hash=sha256:24b6f81f1983e6df8db3adc38562c83f7d4a0c36162885ec7f7b77c7dcbec97b \ - --hash=sha256:256f80b80ca3853f90c21b23ee78cd008713787b1b1e93eae9f3d6a7134abd91 \ - --hash=sha256:28a3a209b96630bca57cce802da70c266eb08c6e97e5afd61a75611ee6c64592 \ - --hash=sha256:2c8f814d84194c9ea681642fd164267891702542f028a15fc97d4674b6206187 \ - --hash=sha256:2de9a304e27f7596cd03d16f1b7c72219bd944e99cc52b84d0145aefb07cbd3c \ - --hash=sha256:38100abb9d1b1435bc4cc340bb4489635dc2f0da7456590877030c9b3d40b0c1 \ - --hash=sha256:3925dd22fa2b7699ed2617149842d2e6adde22b262fcbfada50e3d195e4b3a94 \ - --hash=sha256:3e17ed538242334bf70832644a32a7aae3d83b57567f9fd60a26257e992b79ba \ - --hash=sha256:3e837e369566884707ddaf85fc1744b47575005c0a229de3327f8f9a20f4efeb \ - --hash=sha256:3f4d46d8b35698056ec29bca21546e1551a205058ae1a181d871e278b0b28165 \ - --hash=sha256:44d1b5909021139fe36001ae048dbdde8214afa20200eda0f64c068cac5d5529 \ - --hash=sha256:45d5e886156860dc35862657e1494b9bae8dfa63bf56796f2fb56e1679fc0bca \ - --hash=sha256:4647afc2f90d1ddd33441e5b0e85b16b12ddec4fca55f0d9671fef036ecca27c \ - --hash=sha256:4671d9dd5ec934cb9a73e7ee9676f9362aba54f7f34910956b84d727b0d73fb6 \ - --hash=sha256:53f77cbe57044e88bbd5ed26ac1d0514d2acf0591dd6bb02a3ae37f76811b80c \ - --hash=sha256:5eda85d6d1879e692d546a078b44251cdd08dd1cfb98dfb77b670c97cee49ea0 \ - --hash=sha256:5fed36fccc0612a53f1d4d9a816b50a36702c28a2aa880cb8a122b3466638743 \ - --hash=sha256:61d028e90346df14fedc3d1e5441df818d095f3b87d286825dfcbd6459b7ef63 \ - --hash=sha256:66f011380d0e49ed280c789fbd08ff0d40968ee7b665575489afa95c98196ab5 \ - --hash=sha256:6824f87845e3396029f3820c206e459ccc91760e8fa24422f8b0c3d1731cbec5 \ - --hash=sha256:6c6c373cfc5c83a975506110d17457138c8c63016b563cc9ed6e056a82f13ce4 \ - --hash=sha256:6d02d6655b0e54f54c4ef0b94eb6be0607b70853c45ce98bd278dc7de718be5d \ - --hash=sha256:6d50360be4546678fc1b79ffe7a66265e28667840010348dd69a314145807a1b \ - --hash=sha256:730cacb21e1bdff3ce90babf007d0a0917cc3e6492f336c2f0134101e0944f93 \ - --hash=sha256:737fe7d37e1a1bffe70bd5754ea763a62a066dc5913ca57e957824b72a85e205 \ - --hash=sha256:74a03b9698e198d47562765773b4a8309919089150a0bb17d829ad7b44b60d27 \ - --hash=sha256:7553fb2090d71822f02c629afe6042c299edf91ba1bf94951165613553984512 \ - --hash=sha256:7a66c7204d8869299919db4d5069a82f1561581af12b11b3c9f48c584eb8743d \ - --hash=sha256:7cc09976e8b56f8cebd752f7113ad07752461f48a58cbba644139015ac24954c \ - --hash=sha256:81afed14892743bbe14dacb9e36d9e0e504cd204e0b165062c488942b9718037 \ - --hash=sha256:8941aaadaf67246224cee8c3803777eed332a19d909b47e29c9842ef1e79ac26 \ - --hash=sha256:89472c9762729b5ae1ad974b777416bfda4ac5642423fa93bd57a09204712322 \ - --hash=sha256:8ea985900c5c95ce9db1745f7933eeef5d314f0565b27625d9a10ec9881e1bfb \ - --hash=sha256:8eca2a813c1cb7ad4fb74d368c2ffbbb4789d377ee5bb8df98373c2cc0dee76c \ - --hash=sha256:92b68146a71df78564e4ef48af17551a5ddd142e5190cdf2c5624d0c3ff5b2e8 \ - --hash=sha256:9332088d75dc3241c702d852d4671613136d90fa6881da7d770a483fd05248b4 \ - --hash=sha256:94698a9c5f91f9d138526b48fe26a199609544591f859c870d477351dc7b2414 \ - --hash=sha256:9a67fc9e8eb39039280526379fb3a70023d77caec1852002b4da7e8b270c4dd9 \ - --hash=sha256:9de40a7b0323d889cf8d23d1ef214f565ab154443c42737dfe52ff82cf857664 \ - --hash=sha256:a05d0c237b3349096d3981b727493e22147f934b20f6f125a3eba8f994bec4a9 \ - --hash=sha256:afb8db5439b81cf9c9d0c80404b60c3cc9c3add93e114dcae767f1477cb53775 \ - --hash=sha256:b18a3ed7d5b3bd8d9ef7a8cb226502c6bf8308df1525e1cc676c3680e7176739 \ - --hash=sha256:b1e74d11748e7e98e2f426ab176d4ed720a64412b6a15054378afdb71e0f37dc \ - --hash=sha256:b21e08af67b8a103c71a250401c78d5e0893beff75e28c53c98f4de42f774062 \ - --hash=sha256:b4c854ef3adc177950a8dfc81a86f5115d2abd545751a304c5bcf2c2c7283cfe \ - --hash=sha256:b882b3df248017dba09d6b16defe9b5c407fe32fc7c65a9c69798e6175601be9 \ - --hash=sha256:baf5215e0ab74c16e2dd324e8ec067ef59e41125d3eade2b863d294fd5035c92 \ - --hash=sha256:c649e3a33450ec82378822b3dad03cc228b8f5963c0c12fc3b1e0ab940f768a5 \ - --hash=sha256:c654de545946e0db659b3400168c9ad31b5d29593291482c43e3564effbcee13 \ - --hash=sha256:c6638687455baf640e37344fe26d37c404db8b80d037c3d29f58fe8d1c3b194d \ - --hash=sha256:c8d3b5532fc71b7a77c09192b4a5a200ea992702734a2e9279a37f2478236f26 \ - --hash=sha256:cb527a79772e5ef98fb1d700678fe031e353e765d1ca2d409c92263c6d43e09f \ - --hash=sha256:cf364028c016c03078a23b503f02058f1814320a56ad535686f90565636a9495 \ - --hash=sha256:d48a880098c96020b02d5a1f7d9251308510ce8858940e6fa99ece33f610838b \ - --hash=sha256:d68b6cef7827e8641e8ef16f4494edda8b36104d79773a334beaa1e3521430f6 \ - --hash=sha256:d9b29c1f0ae438d5ee9acb31cadee00a58c46cc9c0b2f9038c6b0b3470877a8c \ - --hash=sha256:d9b97165e8aed9272a6bb17c01e3cc5871a594a446ebedc996e2397a1c1ea8ef \ - --hash=sha256:da68248800ad6320861f129cd9c1bf96ca849a2771a59e0344e88681905916f5 \ - --hash=sha256:da902562c3e9c550df360bfa53c035b2f241fed6d9aef119048073680ace4a18 \ - --hash=sha256:dbd5c7a25a7cb98f5ca55d258b103a2054f859a46ae11aaf23134f9cc0d356ad \ - --hash=sha256:dd4f05f54a52fb558f1ba9f528228066954fee3ebe629fc1660d874d040ae5a3 \ - --hash=sha256:de8dad4425a6ca6e4e5e297b27b5c824ecc7581910bf9aee86cb6835e6812aa7 \ - --hash=sha256:e11e82b744887154b182fd3e7e8512418446501191994dbf9c9fc1f32cc8efd5 \ - --hash=sha256:e6e73b9e02893c764e7e8d5bb5ce277f1a009cd5243f8228f75f842bf937c534 \ - --hash=sha256:f73b96c41e3b2adedc34a7356e64c8eb96e03a3782b535e043a986276ce12a49 \ - --hash=sha256:f93fd8e5c8c0a4aa1f424d6173f14a892044054871c771f8566e4008eaa359d2 \ - --hash=sha256:fc33c5141b55ed366cfaad382df24fe7dcbc686de5be719b207bb248e3053dc5 \ - --hash=sha256:fc7de24befaeae77ba923797c7c87834c73648a05a4bde34b3b7e5588973a453 \ - --hash=sha256:fe562eb1a64e67dd297ccc4f5addea2501664954f2692b69a76449ec7913ecbf +cffi==2.1.0 \ + --hash=sha256:02cb7ff33ded4f1532476731f89ede53e2e488a8e6205515a82144246ffa7dcc \ + --hash=sha256:03e9810d18c646077e501f661b682fbf5dee4676048527ca3cffe66faa9960dd \ + --hash=sha256:0520e1f4c35f44e209cbbb421b67eec42e6a157f59444dfb6058874ff3610e5d \ + --hash=sha256:0582a58f3051372229ca8e7f5f589f9e5632678208d8636fea3676711fdf7fe5 \ + --hash=sha256:0611e7ebf90573a535ebdc33ae9da222d037853983e13359f580fab781ca017f \ + --hash=sha256:0a42c688d19fca6e095a53c6a6e2295a5b050a8b289f109adab02a9e61a25de6 \ + --hash=sha256:0a96b74cda968eebbad56d973efe5098974f0a9fb323865bf99ea1fd24e3e64c \ + --hash=sha256:10537b1df4967ca26d21e5072d7d54188354483b91dc75058968d3f0cf13fbda \ + --hash=sha256:11b3fb55f4f8ad92274ed26705f65d8f91457de71f5380061eb6d125a768fecd \ + --hash=sha256:15faec4adfff450819f3aee0e2e02c812de6edb88203aa58807955db2003472a \ + --hash=sha256:164bff1657b2a74f0b6d54e11c9b375bc97b931f2ca9c43fcf875838da1570dd \ + --hash=sha256:1854b724d00f6654c742097d5387569021be12d3a0f770eae1df8f8acfcc6acd \ + --hash=sha256:19c54ac121cad98450b4896fa9a43ee0180d57bc4bc911a33db6cab1efab6cd3 \ + --hash=sha256:1b96bfe2c4bd825681b7d311ad6d9b7280a091f43e8f63da5729638083cd3bfb \ + --hash=sha256:1e9f50d192a3e525b15a75ab5114e442d83d657b7ec29182a991bc9a88fd3a66 \ + --hash=sha256:1ff3456eab0d889592d1936d6125bbfbc7ae4d3354a700f8bd80450a66445d4d \ + --hash=sha256:2282cd5e38aa8accd03e99d1256af8411c84cdbee6a89d841b563fdbd1f3e50f \ + --hash=sha256:276f20fffd7b396e12516ba8edf9509210ac248cbbc5acbc39cd512f9f59ebe6 \ + --hash=sha256:2b71d409cccee78310ab5dec549aed052aaea483346e282c7b02362596e01bb0 \ + --hash=sha256:2e9dabb9abcb7ad15938c7196ad5c1718a4e6d33cc79b4c0209bdb64c4a54a5c \ + --hash=sha256:30b65779d598c370374fefabf138d456fd6f3216bfa7bedfab1ba82025b0cd93 \ + --hash=sha256:33eb1ad83ebe8f313e0df035c406227d55a79456704a863fad9842136af5ad7d \ + --hash=sha256:35aaea0c7ee0e58a5cd8c2fd1a48fdf7ece0d2699b7ecdda08194e9ce5dd9b3d \ + --hash=sha256:3681e031db29958a7502f5c0c9d6bbc4c36cb20f7b104086fa642d1799631ff8 \ + --hash=sha256:379de10ce1ba048b1448599d1b37b24caee16309d1ac98d3982fc997f768700b \ + --hash=sha256:37f525a7e7e50c017fdebe58b787be310ad59357ae43a053943a6e1a6c526001 \ + --hash=sha256:3b926723c13eba9f81d2ef3820d63aeceec3b2d4639906047bf675cb8a7a500d \ + --hash=sha256:3d7f118b5adbfdfead90c25822690b02bc8074fba949bb7858bec4ebd55adb43 \ + --hash=sha256:46b1c8db8f6122420f32d02fffb924c2fe9bc772d228c7c711748fff56aabb2b \ + --hash=sha256:47ff3a8bfd8cb9da1af7524b965127095055654c177fcfc7578debcb015eecd0 \ + --hash=sha256:4d433a51f1870e43a13b6732f92aaf540ff77c2015097c78556f75a2d6c030e0 \ + --hash=sha256:4f26194e3d95e06501b942642855aed4f953d55e95d7d01b7c4483db3ecff458 \ + --hash=sha256:510aeeeac94811b138077451da1fb18b308a5feab47dd2b603af55804155e1c8 \ + --hash=sha256:5972433ad71a9e46516584ef60a0fda12d9dc459938d1539c3ddecf9bdc1368d \ + --hash=sha256:5ecbd0499275d57506d397eebe1981cee87b47fcd9ef5c22cab7ed7644a39a94 \ + --hash=sha256:6274dcb2d15cef48daa73ed1be5a40d501d74dccd0cd6db364776d12cb6ba022 \ + --hash=sha256:63960549e4f8dc41e31accb97b975abaecfc44c03e396c093a6436763c2ea7db \ + --hash=sha256:64c753a0f87a256020004f37a1c8c02c480e725f910f0b2a0f3f07debd1b2479 \ + --hash=sha256:6af371f3767faeffc6ac1ef57cdfd25844403e9d3f476c5537caee499de96376 \ + --hash=sha256:6ca4919c6e4f89aa99c42510b42cf54596892c00b3f9077f6bdd1505e24b9c8d \ + --hash=sha256:6d194185eabd279f1c05ebe3504265ddfc5ad2b58d0714f7db9f01da592e9eb6 \ + --hash=sha256:702c436735fbe99d59ada02a1f65cfc0d31c0ee8b7290912f8fbc5cd1e4b16c3 \ + --hash=sha256:716ff8ec22f20b4d988b12884086bcef0fc99737043e503f7a3935a6be99b1ea \ + --hash=sha256:762f99479dcb369f60ab9017ad4ab97a36a1dd7c1ee5a3b15db0f4b8659120cd \ + --hash=sha256:7762faa47e8ff7eb80bd261d9a7d8eea2d8baa69de5e95b70c1f338bbe712f02 \ + --hash=sha256:78474632761faa0fb96f30b1c928c84ebcf68713cbb80d15bab09dfe61640fde \ + --hash=sha256:799416bae98336e400981ff6e532d67d5c709cfb30afb79865a1315f94b0e224 \ + --hash=sha256:7d034dcffa09e9a46c93fa3a3be402096cb5354ac6e41ab8e5cc9cd8b642ad76 \ + --hash=sha256:7d28dff1db6764108bc30788d85d61c876beff416d9a49cb9dd7c5a9f34f5804 \ + --hash=sha256:7d3538f9c0e50670f4deb93dbb696576e60590369cae2faf7de681e597a8a1f1 \ + --hash=sha256:7d5980a3433d4b71a5e120f9dd551403d7824e31e2e67124fe2769c404c06913 \ + --hash=sha256:7ea6b3e2c4250ff1de21c630fe72d0f63eb95c2c32ffbf64a358cf4a8836d714 \ + --hash=sha256:86cf8755a791f72c85dc287128cc62d4f24d392e3f1e15837245623f4a33cccc \ + --hash=sha256:88023dfe18799507b73f1dbb0d14326a17465de1bc9c9c7655c22845e9ddc3a2 \ + --hash=sha256:89095c1968b4ba8285840e131bf2891b09ae137fe2146905acae0354fbce1b5e \ + --hash=sha256:8d35c139744adb3e727cd51b1a18324bbe44b8bd41bf8322bca4d41289f48eda \ + --hash=sha256:8e74a6135550c4748af665b1b1118b6aab33b1fc6a16f9aff630af107c3b4512 \ + --hash=sha256:8f9ec95b8a043d3dfbc74d9abc6f7baf524dd27a8dc160b0a32ff9cdab650c28 \ + --hash=sha256:90bec57cf82089383bd06a605b3eb8daebf7e5a668520beaf6e327a83a947699 \ + --hash=sha256:95f2954c2c9473d892eca6e0409f3568b37ab62a8eedb122461f73cc273476e3 \ + --hash=sha256:961be50688f7fba2fa65f63712d3b9b341a22311f5253460ce933f52f0de1c8c \ + --hash=sha256:98fff996e983a36d3aa2eca83af40c5821202e7e6f32d13ae94e3d2286f10cfe \ + --hash=sha256:9b8f0f26ca4e7513c534d351eca551947d053fac438f2a04ac96d882909b0d3a \ + --hash=sha256:9d72af0cf10a76a600a9690078fe31c63b9588c8e86bf9fd353f713c84b5db0f \ + --hash=sha256:9d8272c0e483b024e1b9ad029821470ed8ec65631dbd90217469da0e7cd89f1c \ + --hash=sha256:a016194dbe13d14ee9556e734b772d8d67b947092b268d757fd4290e3ba2dfc2 \ + --hash=sha256:a5781494d4d400a3f47f8f1da94b324f6e6b440a53387774002890a2a2f4b50f \ + --hash=sha256:a95b05f9baf29b91171b3a8bd2020b028835243e7b0ff6bb23e2a3c228518b1b \ + --hash=sha256:aa7a1b53a2a4452ada2d1b5dade9960b2522f1e61293a811a077439e39029565 \ + --hash=sha256:ac0f1a2d0cfa7eea3f2aaf006ab6e70e8feeb16b75d65b7e5939982ca2f11056 \ + --hash=sha256:af5e2915d41fe6c961694d7bfdc8562942638200f3ce2765dfb8b745cf997629 \ + --hash=sha256:b6422532152adf4e59b110cb2808cee7a033800952f5c036b4af047ee43199e7 \ + --hash=sha256:b65f590ef2a44640f9a05dbb548a429b4ade77913ce683ac8b1480777658a6c0 \ + --hash=sha256:ba00f661f8ba35d075c937174e27c2c421cec3942fd2e0ea3e66996757c0fdd9 \ + --hash=sha256:bccbbb5ee76a61f9d99b5bf3846a51d7fca4b6a732fe46f89295610edaf41853 \ + --hash=sha256:bf01d8c84cbea96b944c73b22182e6c7c432b3475632b8111dbfdc95ddad6e13 \ + --hash=sha256:bf5c6cf48238b0eb4c086978c492ad1cbc22373fc5b2d7353b3a598ce6db887a \ + --hash=sha256:c16914df9fb7f500e440e6875fa23ff5e0b31db01fa9c06af98d59a91f0dc2e4 \ + --hash=sha256:c351efb95e832a853a29361675f33a7ce53de1a109cd73fd47af0712213aa4ce \ + --hash=sha256:c4165821e131d6d4ca444347c2b694e2311bcfa3fe5a861cc72968f28867beac \ + --hash=sha256:c5f5df567f6eb216de69be06ce55c8b714090fae02b18a3b40da8163b8c5fa9c \ + --hash=sha256:c941bb58d5a6e1c3892d86e42927ed6c180302f07e6d395d08c416e594b98b46 \ + --hash=sha256:c97f080ea627e2863524c5af3836e2270b5f5dfff1f104392b959f8df0c5d384 \ + --hash=sha256:cb96698e3c7413d906ce83f8ffd245ec1bd94707541f299d0ce4d6b0193e982b \ + --hash=sha256:cbb7640ce37159548d2147b5b8c241f962143d4c71231431820783f4dc78f210 \ + --hash=sha256:cdf2448aab5f661c9315308ec8b93f4e8a1a67a3c733f8631067a2b67d5913dc \ + --hash=sha256:d2117334c3af3bdcb9a88522b844a2bdb5efdc4f71c6c822df55486ae1c3347a \ + --hash=sha256:d53d10f7da99ae46f7373b9150393e9c5eab9b224909982b43832668de4779f5 \ + --hash=sha256:d9fafc5aa2e2a39aaf7f8cc0c1f044a9b07fca12e558dca53a3cc5c654ad67a7 \ + --hash=sha256:db3eb7d46527159a878ec3460e9d40615bc25ba337d477db681aea6e4f05c5d2 \ + --hash=sha256:dbf7c7a88e2bac086f06d14577332760bdeecc42bdec8ac4077f6260557d9326 \ + --hash=sha256:df2b82571a1b30f58a87bf4e5a9e78d2b1eff6c6ce8fd3aa3757221f93f0863f \ + --hash=sha256:df92f2aba50eb4d96718b68ef76f2e57a57b54f2fa62333496d16c6d585a85ca \ + --hash=sha256:eb4e8997a49aa2c08a3e43c9045d224448b8941d88e7ac163c7d383e560cbf98 \ + --hash=sha256:efc1cdd798b1aaf39b4610bba7aad28c9bea9b910f25c784ccf9ec1fa719d1f9 \ + --hash=sha256:f146d154428a2523f9cc7936c02353c2459b8f6cf07d3cd1ee1c0a611109c5d5 \ + --hash=sha256:f5bce581e6b8c235e566a14768a943b172ada3ed73537bb0c0be1edee312d4e7 \ + --hash=sha256:f9912624a0c0b834b7520d7769b3644453aabc0a7e1c839da7359f050750e9bc \ + --hash=sha256:fb62edb5bb52cca65fab91a63afa7561607120d26090a7e8fda6fb9f064726da \ + --hash=sha256:ff067a8d8d880e7809e4ac88eb009bb848870115317b306666502ccad30b147f # via cryptography charset-normalizer==3.4.9 \ --hash=sha256:0327fcd59a935777d83410750c50600ee9571af2846f71ce40f25b13da1ef380 \ @@ -191,82 +207,81 @@ charset-normalizer==3.4.9 \ --hash=sha256:fa36ec09ef71d158186bc79e359ff5fdd6e7996fe8ab638f00d6b93139ba4fcf \ --hash=sha256:fe2c7201c642b7c308f1675355ad7ff7b66acfe3541625efe5a3ad38f29d6115 # via requests -cryptography==46.0.7 \ - --hash=sha256:04959522f938493042d595a736e7dbdff6eb6cc2339c11465b3ff89343b65f65 \ - --hash=sha256:128c5edfe5e5938b86b03941e94fac9ee793a94452ad1365c9fc3f4f62216832 \ - --hash=sha256:1d25aee46d0c6f1a501adcddb2d2fee4b979381346a78558ed13e50aa8a59067 \ - --hash=sha256:24402210aa54baae71d99441d15bb5a1919c195398a87b563df84468160a65de \ - --hash=sha256:258514877e15963bd43b558917bc9f54cf7cf866c38aa576ebf47a77ddbc43a4 \ - --hash=sha256:35719dc79d4730d30f1c2b6474bd6acda36ae2dfae1e3c16f2051f215df33ce0 \ - --hash=sha256:397655da831414d165029da9bc483bed2fe0e75dde6a1523ec2fe63f3c46046b \ - --hash=sha256:3986ac1dee6def53797289999eabe84798ad7817f3e97779b5061a95b0ee4968 \ - --hash=sha256:420b1e4109cc95f0e5700eed79908cef9268265c773d3a66f7af1eef53d409ef \ - --hash=sha256:42a1e5f98abb6391717978baf9f90dc28a743b7d9be7f0751a6f56a75d14065b \ - --hash=sha256:462ad5cb1c148a22b2e3bcc5ad52504dff325d17daf5df8d88c17dda1f75f2a4 \ - --hash=sha256:506c4ff91eff4f82bdac7633318a526b1d1309fc07ca76a3ad182cb5b686d6d3 \ - --hash=sha256:5ad9ef796328c5e3c4ceed237a183f5d41d21150f972455a9d926593a1dcb308 \ - --hash=sha256:5d1c02a14ceb9148cc7816249f64f623fbfee39e8c03b3650d842ad3f34d637e \ - --hash=sha256:5e51be372b26ef4ba3de3c167cd3d1022934bc838ae9eaad7e644986d2a3d163 \ - --hash=sha256:60627cf07e0d9274338521205899337c5d18249db56865f943cbe753aa96f40f \ - --hash=sha256:65814c60f8cc400c63131584e3e1fad01235edba2614b61fbfbfa954082db0ee \ - --hash=sha256:73510b83623e080a2c35c62c15298096e2a5dc8d51c3b4e1740211839d0dea77 \ - --hash=sha256:7bbc6ccf49d05ac8f7d7b5e2e2c33830d4fe2061def88210a126d130d7f71a85 \ - --hash=sha256:80406c3065e2c55d7f49a9550fe0c49b3f12e5bfff5dedb727e319e1afb9bf99 \ - --hash=sha256:84d4cced91f0f159a7ddacad249cc077e63195c36aac40b4150e7a57e84fffe7 \ - --hash=sha256:8a469028a86f12eb7d2fe97162d0634026d92a21f3ae0ac87ed1c4a447886c83 \ - --hash=sha256:91bbcb08347344f810cbe49065914fe048949648f6bd5c2519f34619142bbe85 \ - --hash=sha256:935ce7e3cfdb53e3536119a542b839bb94ec1ad081013e9ab9b7cfd478b05006 \ - --hash=sha256:9694078c5d44c157ef3162e3bf3946510b857df5a3955458381d1c7cfc143ddb \ - --hash=sha256:a1529d614f44b863a7b480c6d000fe93b59acee9c82ffa027cfadc77521a9f5e \ - --hash=sha256:abad9dac36cbf55de6eb49badd4016806b3165d396f64925bf2999bcb67837ba \ - --hash=sha256:b36a4695e29fe69215d75960b22577197aca3f7a25b9cf9d165dcfe9d80bc325 \ - --hash=sha256:b7b412817be92117ec5ed95f880defe9cf18a832e8cafacf0a22337dc1981b4d \ - --hash=sha256:c5b1ccd1239f48b7151a65bc6dd54bcfcc15e028c8ac126d3fada09db0e07ef1 \ - --hash=sha256:cbd5fb06b62bd0721e1170273d3f4d5a277044c47ca27ee257025146c34cbdd1 \ - --hash=sha256:cdf1a610ef82abb396451862739e3fc93b071c844399e15b90726ef7470eeaf2 \ - --hash=sha256:cdfbe22376065ffcf8be74dc9a909f032df19bc58a699456a21712d6e5eabfd0 \ - --hash=sha256:d02c738dacda7dc2a74d1b2b3177042009d5cab7c7079db74afc19e56ca1b455 \ - --hash=sha256:d151173275e1728cf7839aaa80c34fe550c04ddb27b34f48c232193df8db5842 \ - --hash=sha256:d23c8ca48e44ee015cd0a54aeccdf9f09004eba9fc96f38c911011d9ff1bd457 \ - --hash=sha256:d3b99c535a9de0adced13d159c5a9cf65c325601aa30f4be08afd680643e9c15 \ - --hash=sha256:d5f7520159cd9c2154eb61eb67548ca05c5774d39e9c2c4339fd793fe7d097b2 \ - --hash=sha256:db0f493b9181c7820c8134437eb8b0b4792085d37dbb24da050476ccb664e59c \ - --hash=sha256:e06acf3c99be55aa3b516397fe42f5855597f430add9c17fa46bf2e0fb34c9bb \ - --hash=sha256:e4cfd68c5f3e0bfdad0d38e023239b96a2fe84146481852dffbcca442c245aa5 \ - --hash=sha256:ea42cbe97209df307fdc3b155f1b6fa2577c0defa8f1f7d3be7d31d189108ad4 \ - --hash=sha256:ebd6daf519b9f189f85c479427bbd6e9c9037862cf8fe89ee35503bd209ed902 \ - --hash=sha256:f247c8c1a1fb45e12586afbb436ef21ff1e80670b2861a90353d9b025583d246 \ - --hash=sha256:fbfd0e5f273877695cb93baf14b185f4878128b250cc9f8e617ea0c025dfb022 \ - --hash=sha256:fc9ab8856ae6cf7c9358430e49b368f3108f050031442eaeb6b9d87e4dcf4e4f \ - --hash=sha256:fcd8eac50d9138c1d7fc53a653ba60a2bee81a505f9f8850b6b2888555a45d0e \ - --hash=sha256:fdd1736fed309b4300346f88f74cd120c27c56852c3838cab416e7a166f67298 \ - --hash=sha256:ffca7aa1d00cf7d6469b988c581598f2259e46215e0140af408966a24cf086ce +cryptography==49.0.0 \ + --hash=sha256:026ac7423e6fa66872d3bf889be5974507da3944f866f704fa200eadacd00001 \ + --hash=sha256:07cab27cc7b7e0fd28e5e26bb9eeedde5c135c868b46de4a27845abe94af6122 \ + --hash=sha256:084ef1af862eb07ec46d25f68689f2102a9fc0e05ce7b80f14f5fe51e4eef0f6 \ + --hash=sha256:0b82e28ee398a386f0807bba7884d30f25218855690f45115831bcce5d90822c \ + --hash=sha256:0e959b578856a3924bc0cbb710fc12c387b9412a951389f3ca61704a9e25f325 \ + --hash=sha256:0f21641cf4b30fca7aee061ced0ec7ad7b073518088b7c9969a297c0ae796c69 \ + --hash=sha256:196ecd6a36e4e9aa10270393bb98d8df88fccee0bf1e5128b91ae4eb4375896d \ + --hash=sha256:2400ef9c9e2299a25614eb1dea3db54a69b1349efd043bfac9c67630d136df36 \ + --hash=sha256:28d8b15e6275f12c8a207dc309dfa957903c927d08d0cc937ee3f63f200693cc \ + --hash=sha256:2afe9051da7ae7bd5905da5a949280c7d2bb75682e188f650a9d0f2756b834c6 \ + --hash=sha256:2eda353d8a27bcbcaa4cbed18994a74ab4d19a2ca897db188ea269ab9b71419b \ + --hash=sha256:32703d93296f5c1f4b53349ad3a250c2cae0fdecd3a3dd5d47e616d8d616af27 \ + --hash=sha256:33cd0565932807baddb67b96dbee92f2c374b5c89dee09fd74079aeb8c8dba61 \ + --hash=sha256:35b151772baff2c74cba7fa290ceaff4c3b11c0c881eb93eb5dbc05a7cfbba18 \ + --hash=sha256:36d1709f992593689b45bda411498d62c6e365f2ca00b84657d4dadd24de16db \ + --hash=sha256:42b0684e0e40cf26122427802486f6d93aea593612603a94fbf260c7eb1e9c1b \ + --hash=sha256:4ae387c9cb68ea569ca17e490d66d8142b81c3cc814bf179974b7d146e490bbb \ + --hash=sha256:53ecee2e23f7169b6117e99fc8a944e5e50f79e69758a83b52a00cb98ab2b2d2 \ + --hash=sha256:66ec79c3904820572d7e987abdf304281f141d37ad9a489b8e97066e7b9b6459 \ + --hash=sha256:67e1d20ad9ef3a563c59ef22e7a8a0b8210bd26604369ea4a30a7c66aefe504e \ + --hash=sha256:6f2debedf9ca60cf1d5bd466475638af5130f89965605cd818484d19987d3a21 \ + --hash=sha256:6fc361c34fb6aac015ce19435876635e5c6d21db31998b0920f675f131e043b8 \ + --hash=sha256:73a205dce83953d131a4aa1e0fd917a2fd1c5b1eef251e9d7152efefcbf5caf7 \ + --hash=sha256:7abcee80084cda3f7691f3eb1ce480d8df49cec637b429aa35986c1de71738aa \ + --hash=sha256:8c25ceb16df5b9435f3f6a9829204985b0e0cbee3b48aacd432c7d2c850b44d9 \ + --hash=sha256:966fe0e9c67490071f14c0d2b1cb2dfb3023c5ce39457343931415f08382f2db \ + --hash=sha256:9e82dcc8e56052715fb18b2429e3bca4823b1629136a2084fc45a9a5cecb9b64 \ + --hash=sha256:b20133d204d2bb56ba047642199603876c872026ca53e79c35b83772ab2cc505 \ + --hash=sha256:b39efa323140595abd3ecca8529d321ae50f55f3aa3ba9cc81ea56a6011953d5 \ + --hash=sha256:b47db11c2c3525083296069b98ac5221907455e989ae0c2e3008bde851921615 \ + --hash=sha256:b87e65d263b3e5d3bb92a57e2a6638e2f31110fa7aa890c7b2dbba42248d0a3f \ + --hash=sha256:b970c6da94d5bb18629db453d14f2a1300f6bf59b61e9b82377931ef95504866 \ + --hash=sha256:be9fcb48a55f023493482827d4f459bd263cc20efde64f204b97c123201850c6 \ + --hash=sha256:c2bc30226390d60ea19d9f82b19db005fe0452154a23c1c410c12ea801e43561 \ + --hash=sha256:c83782480a4a9da4d0feb51950131ba32e12e70813848b3343f6e18c28a66838 \ + --hash=sha256:cbc77da8c523d5abd028635ba850a6966fcee2c82e2bf65a41d1d8afe0f98be9 \ + --hash=sha256:ccac2bfebc306b862133e3bb71f3f6ee8bb525240089b2d952e4144b3a6d5da7 \ + --hash=sha256:d0527ce944105f257f605a827d6ebead966c752038b6e8656abb9c5edee6fc68 \ + --hash=sha256:d8ecde755e2e91bf773fc94e8c9d730cd7f2007004cb492263a794ec3899a1c8 \ + --hash=sha256:e3fb64c420688e5319ae25113a354015abbd8dffbfbc41781a1ea66fc7622ac3 \ + --hash=sha256:e5dfc1e64de5677cec922ffa8da89c546d0415bf6efdf081842e5d44c84e1f0e \ + --hash=sha256:ec5e529fb80935c94fe7b729f9972b50e351a0e6b50aa294fd5cabb109fcc29a \ + --hash=sha256:f37d847238971164fdbc68ade6f6574aecc9c0af714190e2083429ff68f4ce9d \ + --hash=sha256:f78ff2c9ed8dc2d036b0f4d640e22522213d047c1b14e61205a7e55c80a494d4 \ + --hash=sha256:f89660a348f4f78a92366240a61404e337586ef7f5909a2fef59ca88ef505493 \ + --hash=sha256:fc1e275c2f1d97b1a6450b8b0ea3ebfa6e087a611c2b26cb2404d48588abab7b # via secretstorage -docutils==0.22.2 \ - --hash=sha256:9fdb771707c8784c8f2728b67cb2c691305933d68137ef95a75db5f4dfbc213d \ - --hash=sha256:b0e98d679283fc3bb0ead8a5da7f501baa632654e7056e9c5846842213d674d8 +docutils==0.23 \ + --hash=sha256:25d013af9bf23bc1c7b2b093dff4208166c53a94786c9e447808335ef1185fea \ + --hash=sha256:746f5060322511280a1e50eb76846ed6bf2342984b2ac04dc42caa1a8d78799e # via readme-renderer -idna==3.10 \ - --hash=sha256:12f65c9b470abda6dc35cf8e63cc574b1c52b11df2c86030af0ac09b01b13ea9 \ - --hash=sha256:946d195a0d259cbba61165e88e65941f16e9b36ea6ddb97f00452bae8b1287d3 +id==1.6.1 \ + --hash=sha256:d0732d624fb46fd4e7bc4e5152f00214450953b9e772c182c1c22964def1a069 \ + --hash=sha256:f5ec41ed2629a508f5d0988eda142e190c9c6da971100612c4de9ad9f9b237ca + # via twine +idna==3.18 \ + --hash=sha256:7f952cbe720b688055e3f87de14f5c3e5fdaa8bc3928985c4077ca689de849a2 \ + --hash=sha256:ffb385a7e039654cef1ab9ef32c6fafe283c0c0467bba1d9029738ce4a14a848 # via requests -importlib-metadata==8.7.0 \ - --hash=sha256:d13b81ad223b890aa16c5471f2ac3056cf76c5f10f82d6f9292f0b415f389000 \ - --hash=sha256:e5dd1551894c77868a30651cef00984d50e1002d06942a7101d34870c5f02afd - # via - # keyring - # twine +importlib-metadata==9.0.0 \ + --hash=sha256:2d21d1cc5a017bd0559e36150c21c830ab1dc304dedd1b7ea85d20f45ef3edd7 \ + --hash=sha256:a4f57ab599e6a2e3016d7595cfd72eb4661a5106e787a95bcc90c7105b831efc + # via keyring jaraco-classes==3.4.0 \ --hash=sha256:47a024b51d0239c0dd8c8540c6c7f484be3b8fcf0b2d85c13825780d3b3f3acd \ --hash=sha256:f662826b6bed8cace05e7ff873ce0f9283b5c924470fe664fff1c2f00f581790 # via keyring -jaraco-context==6.0.1 \ - --hash=sha256:9bae4ea555cf0b14938dc0aee7c9f32ed303aa20a3b73e7dc80111628792d1b3 \ - --hash=sha256:f797fc481b490edb305122c9181830a3a5b76d84ef6d1aef2fb9b47ab956f9e4 +jaraco-context==6.1.2 \ + --hash=sha256:bf8150b79a2d5d91ae48629d8b427a8f7ba0e1097dd6202a9059f29a36379535 \ + --hash=sha256:f1a6c9d391e661cc5b8d39861ff077a7dc24dc23833ccee564b234b81c82dfe3 # via keyring -jaraco-functools==4.3.0 \ - --hash=sha256:227ff8ed6f7b8f62c56deff101545fa7543cf2c8e7b82a7c2116e672f29c26e8 \ - --hash=sha256:cfd13ad0dd2c47a3600b439ef72d8615d482cedcff1632930d6f28924d92f294 +jaraco-functools==4.6.0 \ + --hash=sha256:880c577ec9720b3a052d5bc611fb9f2269b3d87902ef42440df443b88e443280 \ + --hash=sha256:99e3dc0060c5cbe8fcd1cdb36258e2a65ca40f1566b2033b12abb1bb44dd3c30 # via keyring jeepney==0.9.0 \ --hash=sha256:97e5714520c16fc0a45695e5365a2e11b81ea79bba796e26f9f1d178cb182683 \ @@ -274,73 +289,74 @@ jeepney==0.9.0 \ # via # keyring # secretstorage -keyring==25.6.0 \ - --hash=sha256:0b39998aa941431eb3d9b0d4b2460bc773b9df6fed7621c2dfb291a7e0187a66 \ - --hash=sha256:552a3f7af126ece7ed5c89753650eec89c7eaae8617d0aa4d9ad2b75111266bd +keyring==25.7.0 \ + --hash=sha256:be4a0b195f149690c166e850609a477c532ddbfbaed96a404d4e43f8d5e2689f \ + --hash=sha256:fe01bd85eb3f8fb3dd0405defdeac9a5b4f6f0439edbb3149577f244a2e8245b # via twine -markdown-it-py==4.0.0 \ - --hash=sha256:87327c59b172c5011896038353a81343b6754500a08cd7a4973bb48c6d578147 \ - --hash=sha256:cb0a2b4aa34f932c007117b194e945bd74e0ec24133ceb5bac59009cda1cb9f3 +markdown-it-py==4.2.0 \ + --hash=sha256:04a21681d6fbb623de53f6f364d352309d4094dd4194040a10fd51833e418d49 \ + --hash=sha256:9f7ebbcd14fe59494226453aed97c1070d83f8d24b6fc3a3bcf9a38092641c4a # via rich mdurl==0.1.2 \ --hash=sha256:84008a41e51615a49fc9966191ff91509e3c40b939176e643fd50a5c2196b8f8 \ --hash=sha256:bb413d29f5eea38f31dd4754dd7377d4465116fb207585f97bf925588687c1ba # via markdown-it-py -more-itertools==10.8.0 \ - --hash=sha256:52d4362373dcf7c52546bc4af9a86ee7c4579df9a8dc268be0a2f949d376cc9b \ - --hash=sha256:f638ddf8a1a0d134181275fb5d58b086ead7c6a72429ad725c67503f13ba30bd +more-itertools==11.1.0 \ + --hash=sha256:48e8f4d9e7e5878571ecf6f2b4e57634f93cd474cc8cfbd2376f2d11b396e30d \ + --hash=sha256:4b65538ae22f6fed0ce4874efd317463a7489796a0939fa66824dd542125a192 # via # jaraco-classes # jaraco-functools -nh3==0.3.0 \ - --hash=sha256:0649464ac8eee018644aacbc103874ccbfac80e3035643c3acaab4287e36e7f5 \ - --hash=sha256:16f8670201f7e8e0e05ed1a590eb84bfa51b01a69dd5caf1d3ea57733de6a52f \ - --hash=sha256:1adeb1062a1c2974bc75b8d1ecb014c5fd4daf2df646bbe2831f7c23659793f9 \ - --hash=sha256:37d3003d98dedca6cd762bf88f2e70b67f05100f6b949ffe540e189cc06887f9 \ - --hash=sha256:389d93d59b8214d51c400fb5b07866c2a4f79e4e14b071ad66c92184fec3a392 \ - --hash=sha256:3f1b4f8a264a0c86ea01da0d0c390fe295ea0bcacc52c2103aca286f6884f518 \ - --hash=sha256:423201bbdf3164a9e09aa01e540adbb94c9962cc177d5b1cbb385f5e1e79216e \ - --hash=sha256:634e34e6162e0408e14fb61d5e69dbaea32f59e847cfcfa41b66100a6b796f62 \ - --hash=sha256:6d68fa277b4a3cf04e5c4b84dd0c6149ff7d56c12b3e3fab304c525b850f613d \ - --hash=sha256:7275fdffaab10cc5801bf026e3c089d8de40a997afc9e41b981f7ac48c5aa7d5 \ - --hash=sha256:7852f038a054e0096dac12b8141191e02e93e0b4608c4b993ec7d4ffafea4e49 \ - --hash=sha256:7c915060a2c8131bef6a29f78debc29ba40859b6dbe2362ef9e5fd44f11487c2 \ - --hash=sha256:80fe20171c6da69c7978ecba33b638e951b85fb92059259edd285ff108b82a6d \ - --hash=sha256:a537ece1bf513e5a88d8cff8a872e12fe8d0f42ef71dd15a5e7520fecd191bbb \ - --hash=sha256:af5aa8127f62bbf03d68f67a956627b1bd0469703a35b3dad28d0c1195e6c7fb \ - --hash=sha256:b0612ccf5de8a480cf08f047b08f9d3fecc12e63d2ee91769cb19d7290614c23 \ - --hash=sha256:ba0caa8aa184196daa6e574d997a33867d6d10234018012d35f86d46024a2a95 \ - --hash=sha256:bae63772408fd63ad836ec569a7c8f444dd32863d0c67f6e0b25ebbd606afa95 \ - --hash=sha256:c7a32a7f0d89f7d30cb8f4a84bdbd56d1eb88b78a2434534f62c71dac538c450 \ - --hash=sha256:ce5e7185599f89b0e391e2f29cc12dc2e206167380cea49b33beda4891be2fe1 \ - --hash=sha256:d8ba24cb31525492ea71b6aac11a4adac91d828aadeff7c4586541bf5dc34d2f \ - --hash=sha256:d97d3efd61404af7e5721a0e74d81cdbfc6e5f97e11e731bb6d090e30a7b62b2 \ - --hash=sha256:e90883f9f85288f423c77b3f5a6f4486375636f25f793165112679a7b6363b35 \ - --hash=sha256:e9e6a7e4d38f7e8dda9edd1433af5170c597336c1a74b4693c5cb75ab2b30f2a \ - --hash=sha256:ec6cfdd2e0399cb79ba4dcffb2332b94d9696c52272ff9d48a630c5dca5e325a \ - --hash=sha256:f416c35efee3e6a6c9ab7716d9e57aa0a49981be915963a82697952cba1353e1 +nh3==0.3.6 \ + --hash=sha256:082675ff87b9385ec430ffe6d5847ba7456cc39b73720cd4add472f9f4cffd56 \ + --hash=sha256:2411e8c3cee81a1ddd62c2a5d50585c28aa5566d373ad1db92536b95ddb24ef2 \ + --hash=sha256:25c733bee928530556b1db0ea46c52cf5aa686146e38e60a6fc7cb801ef91cec \ + --hash=sha256:2f90d9a0cfdbee218994fdaaeeb5a0fde62d08f35e4eef0378ec1e2200172fd0 \ + --hash=sha256:34d2b0d934156b87ee114f599a3ba9b8b9e17b5d79652ba3a13fa50903de965e \ + --hash=sha256:36d06341bd501240d320f5942481ed5e6846136b666e1ba4faf802b78ebc875f \ + --hash=sha256:43bc1ed3fa0716295fabee29ba42b2667e4a51d140b0a68e092170a765474fa6 \ + --hash=sha256:44673b27010051ab5a5e438a86ec31bbda61d4a77d7e900af6b7be3037c1abae \ + --hash=sha256:455469a29951edc92bc48b47ac2281c3f2609e6c4f6a047056449f8c2c23facf \ + --hash=sha256:4713502748f564fee0633b37b3403783ce0a3af3a3d148ad91025a5bdadb7bc6 \ + --hash=sha256:5276ef17bdba9ad8040575c74072008b13aae429436e9d0429e718bb5f90f4da \ + --hash=sha256:597a8e843bea00b2eb5520658dc24a9bb032e7fc9e7c2c0c4cd29420220c9796 \ + --hash=sha256:69bbb92865a693d909db3a700d3c01537533844d0948c1e9323561ce06ecda41 \ + --hash=sha256:69f365963f63a1e9bff53bdbb3c542c7c2efed3e163c9d5d83a772a2ac468c21 \ + --hash=sha256:82ca5bf427ad1b216b65ede1a2e2d87dc49bec417ceba0f297213107d3cd9d78 \ + --hash=sha256:889932a97fb4abb6f95fef1914c0d269ebfb60011e67121c1163059b9449dbb4 \ + --hash=sha256:905f877dc66dd7aea4a76e54bcb26acb5ff8216f720c0017ccf63e0e6035698e \ + --hash=sha256:a43ebd7543555c3ac1bc353023d0794e75cb76f6f18f19c32e95441496c0cc25 \ + --hash=sha256:d14bf7982e7a77c0c775634c29c07ce08b38a046df73e1c1f139b3e82f18a38e \ + --hash=sha256:e196fa70c2ff2eb4de7d3df3108f8f358c1d69dff20d45b11f20a5aa227ffb6d \ + --hash=sha256:e1b160831c9cdb06a6c79c2f9cdb11386602938f9af260d1c457a85add4f6f69 \ + --hash=sha256:e6b7beece07525dc6e6b0fc2f104442de2ba328360ad00e50cbe2e1fd620447d \ + --hash=sha256:edb2b4a1a27523e6cc7c417f8d21ce3d005243548b93e56b762b66b0c7f589f9 \ + --hash=sha256:f2f14b7ae1fca99c4a66c981aac3974e7fbc1ca30a12673d223ae1df76680917 \ + --hash=sha256:f338ac7d594c067679f1e99b4f5ec3906842979560f9d8f15d6bdfa39a353b10 \ + --hash=sha256:f3736c9dd3d1856f80cd031715b84ca75cda2bbb1ac802c3da26bfce590838d7 \ + --hash=sha256:f5ed5fe84aee7f39db95c214a7421bf0499fbf500fec6d86a4e29bfc37971438 # via readme-renderer -pkginfo==1.12.1.2 \ - --hash=sha256:5cd957824ac36f140260964eba3c6be6442a8359b8c48f4adf90210f33a04b7b \ - --hash=sha256:c783ac885519cab2c34927ccfa6bf64b5a704d7c69afaea583dd9b7afe969343 +packaging==26.2 \ + --hash=sha256:5fc45236b9446107ff2415ce77c807cee2862cb6fac22b8a73826d0693b0980e \ + --hash=sha256:ff452ff5a3e828ce110190feff1178bb1f2ea2281fa2075aadb987c2fb221661 # via twine -pycparser==2.23 \ - --hash=sha256:78816d4f24add8f10a06d6f05b4d424ad9e96cfebf68a4ddc99c65c0720d00c2 \ - --hash=sha256:e5c6e8d3fbad53479cab09ac03729e0a9faf2bee3db8208a550daf5af81a5934 +pycparser==3.0 \ + --hash=sha256:600f49d217304a5902ac3c37e1281c9fe94e4d0489de643a9504c5cdfdfc6b29 \ + --hash=sha256:b727414169a36b7d524c1c3e31839a521725078d7b2ff038656844266160a992 # via cffi -pygments==2.19.2 \ - --hash=sha256:636cb2477cec7f8952536970bc533bc43743542f70392ae026374600add5b887 \ - --hash=sha256:86540386c03d588bb81d44bc3928634ff26449851e99741617ecb9037ee5ec0b +pygments==2.20.0 \ + --hash=sha256:6757cd03768053ff99f3039c1a36d6c0aa0b263438fcab17520b30a303a82b5f \ + --hash=sha256:81a9e26dd42fd28a23a2d169d86d7ac03b46e2f8b59ed4698fb4785f946d0176 # via # readme-renderer # rich -readme-renderer==44.0 \ - --hash=sha256:2fbca89b81a08526aadf1357a8c2ae889ec05fb03f5da67f9769c9a592166151 \ - --hash=sha256:8712034eabbfa6805cacf1402b4eeb2a73028f72d1166d6f5cb7f9c047c5d1e1 +readme-renderer==45.0 \ + --hash=sha256:030a8fac74904f8fba11ad1bb6964e3f76e896dc7e5e71f16af190c9056696d1 \ + --hash=sha256:3385ed220117104a2bceb4a9dac8c5fdf6d1f96890d7ea2a9c7174fd5c84091f # via twine -requests==2.33.0 \ - --hash=sha256:3324635456fa185245e24865e810cecec7b4caf933d7eb133dcde67d48cee69b \ - --hash=sha256:c7ebc5e8b0f21837386ad0e1c8fe8b829fa5f544d8df3b2253bff14ef29d7652 +requests==2.34.2 \ + --hash=sha256:2a0d60c172f83ac6ab31e4554906c0f3b3588d37b5cb939b1c061f4907e278e0 \ + --hash=sha256:f288924cae4e29463698d6d60bc6a4da69c89185ad1e0bcc4104f584e960b9ed # via # requests-toolbelt # twine @@ -352,25 +368,26 @@ rfc3986==2.0.0 \ --hash=sha256:50b1502b60e289cb37883f3dfd34532b8873c7de9f49bb546641ce9cbd256ebd \ --hash=sha256:97aacf9dbd4bfd829baad6e6309fa6573aaf1be3f6fa735c8ab05e46cecb261c # via twine -rich==14.1.0 \ - --hash=sha256:536f5f1785986d6dbdea3c75205c473f970777b4a0d6c6dd1b696aa05a3fa04f \ - --hash=sha256:e497a48b844b0320d45007cdebfeaeed8db2a4f4bcf49f15e455cfc4af11eaa8 +rich==15.0.0 \ + --hash=sha256:33bd4ef74232fb73fe9279a257718407f169c09b78a87ad3d296f548e27de0bb \ + --hash=sha256:edd07a4824c6b40189fb7ac9bc4c52536e9780fbbfbddf6f1e2502c31b068c36 # via twine -secretstorage==3.3.3 \ - --hash=sha256:2403533ef369eca6d2ba81718576c5e0f564d5cca1b58f73a8b23e7d4eeebd77 \ - --hash=sha256:f356e6628222568e3af06f2eba8df495efa13b3b63081dafd4f7d9a7b7bc9f99 +secretstorage==3.5.0 \ + --hash=sha256:0ce65888c0725fcb2c5bc0fdb8e5438eece02c523557ea40ce0703c266248137 \ + --hash=sha256:f04b8e4689cbce351744d5537bf6b1329c6fc68f91fa666f60a380edddcd11be # via keyring -twine==5.1.1 \ - --hash=sha256:215dbe7b4b94c2c50a7315c0275d2258399280fbb7d04182c7e55e24b5f93997 \ - --hash=sha256:9aa0825139c02b3434d913545c7b847a21c835e11597f5255842d457da2322db +twine==7.0.0 \ + --hash=sha256:85cdb29c518efef867360ae4acd4b0dfd61c8654a22fca08e6f8539f05022177 \ + --hash=sha256:b854164df26db268af05f49aa5c0344b10e27a494343ff05b1e0bad3b135f5a7 # via -r tools/publish/requirements.in -urllib3==2.6.3 \ - --hash=sha256:1b62b6884944a57dbe321509ab94fd4d3b307075e0c2eae991ac71ee15ad38ed \ - --hash=sha256:bf272323e553dfb2e87d9bfd225ca7b0f467b919d7bbd355436d3fd37cb0acd4 +urllib3==2.7.0 \ + --hash=sha256:231e0ec3b63ceb14667c67be60f2f2c40a518cb38b03af60abc813da26505f4c \ + --hash=sha256:9fb4c81ebbb1ce9531cce37674bbc6f1360472bc18ca9a553ede278ef7276897 # via + # id # requests # twine -zipp==3.23.0 \ - --hash=sha256:071652d6115ed432f5ce1d34c336c0adfd6a884660d1e9712a256d3d3bd4b14e \ - --hash=sha256:a07157588a12518c9d4034df3fbbee09c814741a33ff63c05fa29d26a2404166 +zipp==4.1.0 \ + --hash=sha256:25ad4e16390cd314347dd8f1de67a2ac538ae658ed4ab9db16029c07c188e97f \ + --hash=sha256:4cb57381f544315db7688e976e922a2b18cdb513d21cc194eb42232ba2a3e602 # via importlib-metadata diff --git a/tools/publish/requirements_universal.txt b/tools/publish/requirements_universal.txt index 4ab1911981..bd17c90a54 100644 --- a/tools/publish/requirements_universal.txt +++ b/tools/publish/requirements_universal.txt @@ -2,82 +2,115 @@ # bazel run //tools/publish:requirements_universal.update --index-url https://pypi.org/simple -backports-tarfile==1.2.0 ; python_full_version < '3.12' \ +backports-tarfile==1.2.0 ; python_full_version < '3.12' and platform_machine != 'ppc64le' and platform_machine != 's390x' \ --hash=sha256:77e284d754527b01fb1e6fa8a1afe577858ebe4e9dad8919e34c862cb399bc34 \ --hash=sha256:d75e02c268746e1b8144c278978b6e98e85de6ad16f8e4b0844a154557eca991 # via jaraco-context -certifi==2025.10.5 \ - --hash=sha256:0f212c2744a9bb6de0c56639a6f68afe01ecd92d91f14ae897c4fe7bbeeef0de \ - --hash=sha256:47c09d31ccf2acf0be3f701ea53595ee7e0b8fa08801c6624be771df09ae7b43 +certifi==2026.7.22 \ + --hash=sha256:62f22742b58a1a33014a2b6b706588a8d7e2a88ae7bd1a6ebe8c992928483775 \ + --hash=sha256:741e2c3b351ddf169a738da9f2c048608ff7f2c5cc02f1ebc6b118bb090d5d55 # via requests -cffi==2.0.0 ; platform_python_implementation != 'PyPy' and sys_platform == 'linux' \ - --hash=sha256:045d61c734659cc045141be4bae381a41d89b741f795af1dd018bfb532fd0df8 \ - --hash=sha256:0984a4925a435b1da406122d4d7968dd861c1385afe3b45ba82b750f229811e2 \ - --hash=sha256:0e2b1fac190ae3ebfe37b979cc1ce69c81f4e4fe5746bb401dca63a9062cdaf1 \ - --hash=sha256:0f048dcf80db46f0098ccac01132761580d28e28bc0f78ae0d58048063317e15 \ - --hash=sha256:1257bdabf294dceb59f5e70c64a3e2f462c30c7ad68092d01bbbfb1c16b1ba36 \ - --hash=sha256:1c39c6016c32bc48dd54561950ebd6836e1670f2ae46128f67cf49e789c52824 \ - --hash=sha256:1d599671f396c4723d016dbddb72fe8e0397082b0a77a4fab8028923bec050e8 \ - --hash=sha256:28b16024becceed8c6dfbc75629e27788d8a3f9030691a1dbf9821a128b22c36 \ - --hash=sha256:2bb1a08b8008b281856e5971307cc386a8e9c5b625ac297e853d36da6efe9c17 \ - --hash=sha256:30c5e0cb5ae493c04c8b42916e52ca38079f1b235c2f8ae5f4527b963c401caf \ - --hash=sha256:31000ec67d4221a71bd3f67df918b1f88f676f1c3b535a7eb473255fdc0b83fc \ - --hash=sha256:386c8bf53c502fff58903061338ce4f4950cbdcb23e2902d86c0f722b786bbe3 \ - --hash=sha256:3edc8d958eb099c634dace3c7e16560ae474aa3803a5df240542b305d14e14ed \ - --hash=sha256:45398b671ac6d70e67da8e4224a065cec6a93541bb7aebe1b198a61b58c7b702 \ - --hash=sha256:46bf43160c1a35f7ec506d254e5c890f3c03648a4dbac12d624e4490a7046cd1 \ - --hash=sha256:4ceb10419a9adf4460ea14cfd6bc43d08701f0835e979bf821052f1805850fe8 \ - --hash=sha256:51392eae71afec0d0c8fb1a53b204dbb3bcabcb3c9b807eedf3e1e6ccf2de903 \ - --hash=sha256:5da5719280082ac6bd9aa7becb3938dc9f9cbd57fac7d2871717b1feb0902ab6 \ - --hash=sha256:610faea79c43e44c71e1ec53a554553fa22321b65fae24889706c0a84d4ad86d \ - --hash=sha256:636062ea65bd0195bc012fea9321aca499c0504409f413dc88af450b57ffd03b \ - --hash=sha256:6883e737d7d9e4899a8a695e00ec36bd4e5e4f18fabe0aca0efe0a4b44cdb13e \ - --hash=sha256:6b8b4a92e1c65048ff98cfe1f735ef8f1ceb72e3d5f0c25fdb12087a23da22be \ - --hash=sha256:6f17be4345073b0a7b8ea599688f692ac3ef23ce28e5df79c04de519dbc4912c \ - --hash=sha256:706510fe141c86a69c8ddc029c7910003a17353970cff3b904ff0686a5927683 \ - --hash=sha256:72e72408cad3d5419375fc87d289076ee319835bdfa2caad331e377589aebba9 \ - --hash=sha256:733e99bc2df47476e3848417c5a4540522f234dfd4ef3ab7fafdf555b082ec0c \ - --hash=sha256:7596d6620d3fa590f677e9ee430df2958d2d6d6de2feeae5b20e82c00b76fbf8 \ - --hash=sha256:78122be759c3f8a014ce010908ae03364d00a1f81ab5c7f4a7a5120607ea56e1 \ - --hash=sha256:805b4371bf7197c329fcb3ead37e710d1bca9da5d583f5073b799d5c5bd1eee4 \ - --hash=sha256:85a950a4ac9c359340d5963966e3e0a94a676bd6245a4b55bc43949eee26a655 \ - --hash=sha256:8f2cdc858323644ab277e9bb925ad72ae0e67f69e804f4898c070998d50b1a67 \ - --hash=sha256:9755e4345d1ec879e3849e62222a18c7174d65a6a92d5b346b1863912168b595 \ - --hash=sha256:98e3969bcff97cae1b2def8ba499ea3d6f31ddfdb7635374834cf89a1a08ecf0 \ - --hash=sha256:a08d7e755f8ed21095a310a693525137cfe756ce62d066e53f502a83dc550f65 \ - --hash=sha256:a1ed2dd2972641495a3ec98445e09766f077aee98a1c896dcb4ad0d303628e41 \ - --hash=sha256:a24ed04c8ffd54b0729c07cee15a81d964e6fee0e3d4d342a27b020d22959dc6 \ - --hash=sha256:a45e3c6913c5b87b3ff120dcdc03f6131fa0065027d0ed7ee6190736a74cd401 \ - --hash=sha256:a9b15d491f3ad5d692e11f6b71f7857e7835eb677955c00cc0aefcd0669adaf6 \ - --hash=sha256:ad9413ccdeda48c5afdae7e4fa2192157e991ff761e7ab8fdd8926f40b160cc3 \ - --hash=sha256:b2ab587605f4ba0bf81dc0cb08a41bd1c0a5906bd59243d56bad7668a6fc6c16 \ - --hash=sha256:b62ce867176a75d03a665bad002af8e6d54644fad99a3c70905c543130e39d93 \ - --hash=sha256:c03e868a0b3bc35839ba98e74211ed2b05d2119be4e8a0f224fba9384f1fe02e \ - --hash=sha256:c59d6e989d07460165cc5ad3c61f9fd8f1b4796eacbd81cee78957842b834af4 \ - --hash=sha256:c7eac2ef9b63c79431bc4b25f1cd649d7f061a28808cbc6c47b534bd789ef964 \ - --hash=sha256:c9c3d058ebabb74db66e431095118094d06abf53284d9c81f27300d0e0d8bc7c \ - --hash=sha256:ca74b8dbe6e8e8263c0ffd60277de77dcee6c837a3d0881d8c1ead7268c9e576 \ - --hash=sha256:caaf0640ef5f5517f49bc275eca1406b0ffa6aa184892812030f04c2abf589a0 \ - --hash=sha256:cdf5ce3acdfd1661132f2a9c19cac174758dc2352bfe37d98aa7512c6b7178b3 \ - --hash=sha256:d016c76bdd850f3c626af19b0542c9677ba156e4ee4fccfdd7848803533ef662 \ - --hash=sha256:d01b12eeeb4427d3110de311e1774046ad344f5b1a7403101878976ecd7a10f3 \ - --hash=sha256:d63afe322132c194cf832bfec0dc69a99fb9bb6bbd550f161a49e9e855cc78ff \ - --hash=sha256:da95af8214998d77a98cc14e3a3bd00aa191526343078b530ceb0bd710fb48a5 \ - --hash=sha256:dd398dbc6773384a17fe0d3e7eeb8d1a21c2200473ee6806bb5e6a8e62bb73dd \ - --hash=sha256:de2ea4b5833625383e464549fec1bc395c1bdeeb5f25c4a3a82b5a8c756ec22f \ - --hash=sha256:de55b766c7aa2e2a3092c51e0483d700341182f08e67c63630d5b6f200bb28e5 \ - --hash=sha256:df8b1c11f177bc2313ec4b2d46baec87a5f3e71fc8b45dab2ee7cae86d9aba14 \ - --hash=sha256:e03eab0a8677fa80d646b5ddece1cbeaf556c313dcfac435ba11f107ba117b5d \ - --hash=sha256:e221cf152cff04059d011ee126477f0d9588303eb57e88923578ace7baad17f9 \ - --hash=sha256:e31ae45bc2e29f6b2abd0de1cc3b9d5205aa847cafaecb8af1476a609a2f6eb7 \ - --hash=sha256:edae79245293e15384b51f88b00613ba9f7198016a5948b5dddf4917d4d26382 \ - --hash=sha256:f1e22e8c4419538cb197e4dd60acc919d7696e5ef98ee4da4e01d3f8cfa4cc5a \ - --hash=sha256:f3a2b4222ce6b60e2e8b337bb9596923045681d71e5a082783484d845390938e \ - --hash=sha256:f6a16c31041f09ead72d69f583767292f750d24913dadacf5756b966aacb3f1a \ - --hash=sha256:f75c7ab1f9e4aca5414ed4d8e5c0e303a34f4421f8a0d47a4d019ceff0ab6af4 \ - --hash=sha256:f79fc4fc25f1c8698ff97788206bb3c2598949bfe0fef03d299eb1b5356ada99 \ - --hash=sha256:f7f5baafcc48261359e14bcd6d9bff6d4b28d9103847c9e136694cb0501aef87 \ - --hash=sha256:fc48c783f9c87e60831201f2cce7f3b2e4846bf4d8728eabe54d60700b318a0b +cffi==2.1.0 ; platform_machine != 'ppc64le' and platform_machine != 's390x' and platform_python_implementation != 'PyPy' and sys_platform == 'linux' \ + --hash=sha256:02cb7ff33ded4f1532476731f89ede53e2e488a8e6205515a82144246ffa7dcc \ + --hash=sha256:03e9810d18c646077e501f661b682fbf5dee4676048527ca3cffe66faa9960dd \ + --hash=sha256:0520e1f4c35f44e209cbbb421b67eec42e6a157f59444dfb6058874ff3610e5d \ + --hash=sha256:0582a58f3051372229ca8e7f5f589f9e5632678208d8636fea3676711fdf7fe5 \ + --hash=sha256:0611e7ebf90573a535ebdc33ae9da222d037853983e13359f580fab781ca017f \ + --hash=sha256:0a42c688d19fca6e095a53c6a6e2295a5b050a8b289f109adab02a9e61a25de6 \ + --hash=sha256:0a96b74cda968eebbad56d973efe5098974f0a9fb323865bf99ea1fd24e3e64c \ + --hash=sha256:10537b1df4967ca26d21e5072d7d54188354483b91dc75058968d3f0cf13fbda \ + --hash=sha256:11b3fb55f4f8ad92274ed26705f65d8f91457de71f5380061eb6d125a768fecd \ + --hash=sha256:15faec4adfff450819f3aee0e2e02c812de6edb88203aa58807955db2003472a \ + --hash=sha256:164bff1657b2a74f0b6d54e11c9b375bc97b931f2ca9c43fcf875838da1570dd \ + --hash=sha256:1854b724d00f6654c742097d5387569021be12d3a0f770eae1df8f8acfcc6acd \ + --hash=sha256:19c54ac121cad98450b4896fa9a43ee0180d57bc4bc911a33db6cab1efab6cd3 \ + --hash=sha256:1b96bfe2c4bd825681b7d311ad6d9b7280a091f43e8f63da5729638083cd3bfb \ + --hash=sha256:1e9f50d192a3e525b15a75ab5114e442d83d657b7ec29182a991bc9a88fd3a66 \ + --hash=sha256:1ff3456eab0d889592d1936d6125bbfbc7ae4d3354a700f8bd80450a66445d4d \ + --hash=sha256:2282cd5e38aa8accd03e99d1256af8411c84cdbee6a89d841b563fdbd1f3e50f \ + --hash=sha256:276f20fffd7b396e12516ba8edf9509210ac248cbbc5acbc39cd512f9f59ebe6 \ + --hash=sha256:2b71d409cccee78310ab5dec549aed052aaea483346e282c7b02362596e01bb0 \ + --hash=sha256:2e9dabb9abcb7ad15938c7196ad5c1718a4e6d33cc79b4c0209bdb64c4a54a5c \ + --hash=sha256:30b65779d598c370374fefabf138d456fd6f3216bfa7bedfab1ba82025b0cd93 \ + --hash=sha256:33eb1ad83ebe8f313e0df035c406227d55a79456704a863fad9842136af5ad7d \ + --hash=sha256:35aaea0c7ee0e58a5cd8c2fd1a48fdf7ece0d2699b7ecdda08194e9ce5dd9b3d \ + --hash=sha256:3681e031db29958a7502f5c0c9d6bbc4c36cb20f7b104086fa642d1799631ff8 \ + --hash=sha256:379de10ce1ba048b1448599d1b37b24caee16309d1ac98d3982fc997f768700b \ + --hash=sha256:37f525a7e7e50c017fdebe58b787be310ad59357ae43a053943a6e1a6c526001 \ + --hash=sha256:3b926723c13eba9f81d2ef3820d63aeceec3b2d4639906047bf675cb8a7a500d \ + --hash=sha256:3d7f118b5adbfdfead90c25822690b02bc8074fba949bb7858bec4ebd55adb43 \ + --hash=sha256:46b1c8db8f6122420f32d02fffb924c2fe9bc772d228c7c711748fff56aabb2b \ + --hash=sha256:47ff3a8bfd8cb9da1af7524b965127095055654c177fcfc7578debcb015eecd0 \ + --hash=sha256:4d433a51f1870e43a13b6732f92aaf540ff77c2015097c78556f75a2d6c030e0 \ + --hash=sha256:4f26194e3d95e06501b942642855aed4f953d55e95d7d01b7c4483db3ecff458 \ + --hash=sha256:510aeeeac94811b138077451da1fb18b308a5feab47dd2b603af55804155e1c8 \ + --hash=sha256:5972433ad71a9e46516584ef60a0fda12d9dc459938d1539c3ddecf9bdc1368d \ + --hash=sha256:5ecbd0499275d57506d397eebe1981cee87b47fcd9ef5c22cab7ed7644a39a94 \ + --hash=sha256:6274dcb2d15cef48daa73ed1be5a40d501d74dccd0cd6db364776d12cb6ba022 \ + --hash=sha256:63960549e4f8dc41e31accb97b975abaecfc44c03e396c093a6436763c2ea7db \ + --hash=sha256:64c753a0f87a256020004f37a1c8c02c480e725f910f0b2a0f3f07debd1b2479 \ + --hash=sha256:6af371f3767faeffc6ac1ef57cdfd25844403e9d3f476c5537caee499de96376 \ + --hash=sha256:6ca4919c6e4f89aa99c42510b42cf54596892c00b3f9077f6bdd1505e24b9c8d \ + --hash=sha256:6d194185eabd279f1c05ebe3504265ddfc5ad2b58d0714f7db9f01da592e9eb6 \ + --hash=sha256:702c436735fbe99d59ada02a1f65cfc0d31c0ee8b7290912f8fbc5cd1e4b16c3 \ + --hash=sha256:716ff8ec22f20b4d988b12884086bcef0fc99737043e503f7a3935a6be99b1ea \ + --hash=sha256:762f99479dcb369f60ab9017ad4ab97a36a1dd7c1ee5a3b15db0f4b8659120cd \ + --hash=sha256:7762faa47e8ff7eb80bd261d9a7d8eea2d8baa69de5e95b70c1f338bbe712f02 \ + --hash=sha256:78474632761faa0fb96f30b1c928c84ebcf68713cbb80d15bab09dfe61640fde \ + --hash=sha256:799416bae98336e400981ff6e532d67d5c709cfb30afb79865a1315f94b0e224 \ + --hash=sha256:7d034dcffa09e9a46c93fa3a3be402096cb5354ac6e41ab8e5cc9cd8b642ad76 \ + --hash=sha256:7d28dff1db6764108bc30788d85d61c876beff416d9a49cb9dd7c5a9f34f5804 \ + --hash=sha256:7d3538f9c0e50670f4deb93dbb696576e60590369cae2faf7de681e597a8a1f1 \ + --hash=sha256:7d5980a3433d4b71a5e120f9dd551403d7824e31e2e67124fe2769c404c06913 \ + --hash=sha256:7ea6b3e2c4250ff1de21c630fe72d0f63eb95c2c32ffbf64a358cf4a8836d714 \ + --hash=sha256:86cf8755a791f72c85dc287128cc62d4f24d392e3f1e15837245623f4a33cccc \ + --hash=sha256:88023dfe18799507b73f1dbb0d14326a17465de1bc9c9c7655c22845e9ddc3a2 \ + --hash=sha256:89095c1968b4ba8285840e131bf2891b09ae137fe2146905acae0354fbce1b5e \ + --hash=sha256:8d35c139744adb3e727cd51b1a18324bbe44b8bd41bf8322bca4d41289f48eda \ + --hash=sha256:8e74a6135550c4748af665b1b1118b6aab33b1fc6a16f9aff630af107c3b4512 \ + --hash=sha256:8f9ec95b8a043d3dfbc74d9abc6f7baf524dd27a8dc160b0a32ff9cdab650c28 \ + --hash=sha256:90bec57cf82089383bd06a605b3eb8daebf7e5a668520beaf6e327a83a947699 \ + --hash=sha256:95f2954c2c9473d892eca6e0409f3568b37ab62a8eedb122461f73cc273476e3 \ + --hash=sha256:961be50688f7fba2fa65f63712d3b9b341a22311f5253460ce933f52f0de1c8c \ + --hash=sha256:98fff996e983a36d3aa2eca83af40c5821202e7e6f32d13ae94e3d2286f10cfe \ + --hash=sha256:9b8f0f26ca4e7513c534d351eca551947d053fac438f2a04ac96d882909b0d3a \ + --hash=sha256:9d72af0cf10a76a600a9690078fe31c63b9588c8e86bf9fd353f713c84b5db0f \ + --hash=sha256:9d8272c0e483b024e1b9ad029821470ed8ec65631dbd90217469da0e7cd89f1c \ + --hash=sha256:a016194dbe13d14ee9556e734b772d8d67b947092b268d757fd4290e3ba2dfc2 \ + --hash=sha256:a5781494d4d400a3f47f8f1da94b324f6e6b440a53387774002890a2a2f4b50f \ + --hash=sha256:a95b05f9baf29b91171b3a8bd2020b028835243e7b0ff6bb23e2a3c228518b1b \ + --hash=sha256:aa7a1b53a2a4452ada2d1b5dade9960b2522f1e61293a811a077439e39029565 \ + --hash=sha256:ac0f1a2d0cfa7eea3f2aaf006ab6e70e8feeb16b75d65b7e5939982ca2f11056 \ + --hash=sha256:af5e2915d41fe6c961694d7bfdc8562942638200f3ce2765dfb8b745cf997629 \ + --hash=sha256:b6422532152adf4e59b110cb2808cee7a033800952f5c036b4af047ee43199e7 \ + --hash=sha256:b65f590ef2a44640f9a05dbb548a429b4ade77913ce683ac8b1480777658a6c0 \ + --hash=sha256:ba00f661f8ba35d075c937174e27c2c421cec3942fd2e0ea3e66996757c0fdd9 \ + --hash=sha256:bccbbb5ee76a61f9d99b5bf3846a51d7fca4b6a732fe46f89295610edaf41853 \ + --hash=sha256:bf01d8c84cbea96b944c73b22182e6c7c432b3475632b8111dbfdc95ddad6e13 \ + --hash=sha256:bf5c6cf48238b0eb4c086978c492ad1cbc22373fc5b2d7353b3a598ce6db887a \ + --hash=sha256:c16914df9fb7f500e440e6875fa23ff5e0b31db01fa9c06af98d59a91f0dc2e4 \ + --hash=sha256:c351efb95e832a853a29361675f33a7ce53de1a109cd73fd47af0712213aa4ce \ + --hash=sha256:c4165821e131d6d4ca444347c2b694e2311bcfa3fe5a861cc72968f28867beac \ + --hash=sha256:c5f5df567f6eb216de69be06ce55c8b714090fae02b18a3b40da8163b8c5fa9c \ + --hash=sha256:c941bb58d5a6e1c3892d86e42927ed6c180302f07e6d395d08c416e594b98b46 \ + --hash=sha256:c97f080ea627e2863524c5af3836e2270b5f5dfff1f104392b959f8df0c5d384 \ + --hash=sha256:cb96698e3c7413d906ce83f8ffd245ec1bd94707541f299d0ce4d6b0193e982b \ + --hash=sha256:cbb7640ce37159548d2147b5b8c241f962143d4c71231431820783f4dc78f210 \ + --hash=sha256:cdf2448aab5f661c9315308ec8b93f4e8a1a67a3c733f8631067a2b67d5913dc \ + --hash=sha256:d2117334c3af3bdcb9a88522b844a2bdb5efdc4f71c6c822df55486ae1c3347a \ + --hash=sha256:d53d10f7da99ae46f7373b9150393e9c5eab9b224909982b43832668de4779f5 \ + --hash=sha256:d9fafc5aa2e2a39aaf7f8cc0c1f044a9b07fca12e558dca53a3cc5c654ad67a7 \ + --hash=sha256:db3eb7d46527159a878ec3460e9d40615bc25ba337d477db681aea6e4f05c5d2 \ + --hash=sha256:dbf7c7a88e2bac086f06d14577332760bdeecc42bdec8ac4077f6260557d9326 \ + --hash=sha256:df2b82571a1b30f58a87bf4e5a9e78d2b1eff6c6ce8fd3aa3757221f93f0863f \ + --hash=sha256:df92f2aba50eb4d96718b68ef76f2e57a57b54f2fa62333496d16c6d585a85ca \ + --hash=sha256:eb4e8997a49aa2c08a3e43c9045d224448b8941d88e7ac163c7d383e560cbf98 \ + --hash=sha256:efc1cdd798b1aaf39b4610bba7aad28c9bea9b910f25c784ccf9ec1fa719d1f9 \ + --hash=sha256:f146d154428a2523f9cc7936c02353c2459b8f6cf07d3cd1ee1c0a611109c5d5 \ + --hash=sha256:f5bce581e6b8c235e566a14768a943b172ada3ed73537bb0c0be1edee312d4e7 \ + --hash=sha256:f9912624a0c0b834b7520d7769b3644453aabc0a7e1c839da7359f050750e9bc \ + --hash=sha256:fb62edb5bb52cca65fab91a63afa7561607120d26090a7e8fda6fb9f064726da \ + --hash=sha256:ff067a8d8d880e7809e4ac88eb009bb848870115317b306666502ccad30b147f # via cryptography charset-normalizer==3.4.9 \ --hash=sha256:0327fcd59a935777d83410750c50600ee9571af2846f71ce40f25b13da1ef380 \ @@ -174,160 +207,160 @@ charset-normalizer==3.4.9 \ --hash=sha256:fa36ec09ef71d158186bc79e359ff5fdd6e7996fe8ab638f00d6b93139ba4fcf \ --hash=sha256:fe2c7201c642b7c308f1675355ad7ff7b66acfe3541625efe5a3ad38f29d6115 # via requests -cryptography==46.0.7 ; sys_platform == 'linux' \ - --hash=sha256:04959522f938493042d595a736e7dbdff6eb6cc2339c11465b3ff89343b65f65 \ - --hash=sha256:128c5edfe5e5938b86b03941e94fac9ee793a94452ad1365c9fc3f4f62216832 \ - --hash=sha256:1d25aee46d0c6f1a501adcddb2d2fee4b979381346a78558ed13e50aa8a59067 \ - --hash=sha256:24402210aa54baae71d99441d15bb5a1919c195398a87b563df84468160a65de \ - --hash=sha256:258514877e15963bd43b558917bc9f54cf7cf866c38aa576ebf47a77ddbc43a4 \ - --hash=sha256:35719dc79d4730d30f1c2b6474bd6acda36ae2dfae1e3c16f2051f215df33ce0 \ - --hash=sha256:397655da831414d165029da9bc483bed2fe0e75dde6a1523ec2fe63f3c46046b \ - --hash=sha256:3986ac1dee6def53797289999eabe84798ad7817f3e97779b5061a95b0ee4968 \ - --hash=sha256:420b1e4109cc95f0e5700eed79908cef9268265c773d3a66f7af1eef53d409ef \ - --hash=sha256:42a1e5f98abb6391717978baf9f90dc28a743b7d9be7f0751a6f56a75d14065b \ - --hash=sha256:462ad5cb1c148a22b2e3bcc5ad52504dff325d17daf5df8d88c17dda1f75f2a4 \ - --hash=sha256:506c4ff91eff4f82bdac7633318a526b1d1309fc07ca76a3ad182cb5b686d6d3 \ - --hash=sha256:5ad9ef796328c5e3c4ceed237a183f5d41d21150f972455a9d926593a1dcb308 \ - --hash=sha256:5d1c02a14ceb9148cc7816249f64f623fbfee39e8c03b3650d842ad3f34d637e \ - --hash=sha256:5e51be372b26ef4ba3de3c167cd3d1022934bc838ae9eaad7e644986d2a3d163 \ - --hash=sha256:60627cf07e0d9274338521205899337c5d18249db56865f943cbe753aa96f40f \ - --hash=sha256:65814c60f8cc400c63131584e3e1fad01235edba2614b61fbfbfa954082db0ee \ - --hash=sha256:73510b83623e080a2c35c62c15298096e2a5dc8d51c3b4e1740211839d0dea77 \ - --hash=sha256:7bbc6ccf49d05ac8f7d7b5e2e2c33830d4fe2061def88210a126d130d7f71a85 \ - --hash=sha256:80406c3065e2c55d7f49a9550fe0c49b3f12e5bfff5dedb727e319e1afb9bf99 \ - --hash=sha256:84d4cced91f0f159a7ddacad249cc077e63195c36aac40b4150e7a57e84fffe7 \ - --hash=sha256:8a469028a86f12eb7d2fe97162d0634026d92a21f3ae0ac87ed1c4a447886c83 \ - --hash=sha256:91bbcb08347344f810cbe49065914fe048949648f6bd5c2519f34619142bbe85 \ - --hash=sha256:935ce7e3cfdb53e3536119a542b839bb94ec1ad081013e9ab9b7cfd478b05006 \ - --hash=sha256:9694078c5d44c157ef3162e3bf3946510b857df5a3955458381d1c7cfc143ddb \ - --hash=sha256:a1529d614f44b863a7b480c6d000fe93b59acee9c82ffa027cfadc77521a9f5e \ - --hash=sha256:abad9dac36cbf55de6eb49badd4016806b3165d396f64925bf2999bcb67837ba \ - --hash=sha256:b36a4695e29fe69215d75960b22577197aca3f7a25b9cf9d165dcfe9d80bc325 \ - --hash=sha256:b7b412817be92117ec5ed95f880defe9cf18a832e8cafacf0a22337dc1981b4d \ - --hash=sha256:c5b1ccd1239f48b7151a65bc6dd54bcfcc15e028c8ac126d3fada09db0e07ef1 \ - --hash=sha256:cbd5fb06b62bd0721e1170273d3f4d5a277044c47ca27ee257025146c34cbdd1 \ - --hash=sha256:cdf1a610ef82abb396451862739e3fc93b071c844399e15b90726ef7470eeaf2 \ - --hash=sha256:cdfbe22376065ffcf8be74dc9a909f032df19bc58a699456a21712d6e5eabfd0 \ - --hash=sha256:d02c738dacda7dc2a74d1b2b3177042009d5cab7c7079db74afc19e56ca1b455 \ - --hash=sha256:d151173275e1728cf7839aaa80c34fe550c04ddb27b34f48c232193df8db5842 \ - --hash=sha256:d23c8ca48e44ee015cd0a54aeccdf9f09004eba9fc96f38c911011d9ff1bd457 \ - --hash=sha256:d3b99c535a9de0adced13d159c5a9cf65c325601aa30f4be08afd680643e9c15 \ - --hash=sha256:d5f7520159cd9c2154eb61eb67548ca05c5774d39e9c2c4339fd793fe7d097b2 \ - --hash=sha256:db0f493b9181c7820c8134437eb8b0b4792085d37dbb24da050476ccb664e59c \ - --hash=sha256:e06acf3c99be55aa3b516397fe42f5855597f430add9c17fa46bf2e0fb34c9bb \ - --hash=sha256:e4cfd68c5f3e0bfdad0d38e023239b96a2fe84146481852dffbcca442c245aa5 \ - --hash=sha256:ea42cbe97209df307fdc3b155f1b6fa2577c0defa8f1f7d3be7d31d189108ad4 \ - --hash=sha256:ebd6daf519b9f189f85c479427bbd6e9c9037862cf8fe89ee35503bd209ed902 \ - --hash=sha256:f247c8c1a1fb45e12586afbb436ef21ff1e80670b2861a90353d9b025583d246 \ - --hash=sha256:fbfd0e5f273877695cb93baf14b185f4878128b250cc9f8e617ea0c025dfb022 \ - --hash=sha256:fc9ab8856ae6cf7c9358430e49b368f3108f050031442eaeb6b9d87e4dcf4e4f \ - --hash=sha256:fcd8eac50d9138c1d7fc53a653ba60a2bee81a505f9f8850b6b2888555a45d0e \ - --hash=sha256:fdd1736fed309b4300346f88f74cd120c27c56852c3838cab416e7a166f67298 \ - --hash=sha256:ffca7aa1d00cf7d6469b988c581598f2259e46215e0140af408966a24cf086ce +cryptography==49.0.0 ; platform_machine != 'ppc64le' and platform_machine != 's390x' and sys_platform == 'linux' \ + --hash=sha256:026ac7423e6fa66872d3bf889be5974507da3944f866f704fa200eadacd00001 \ + --hash=sha256:07cab27cc7b7e0fd28e5e26bb9eeedde5c135c868b46de4a27845abe94af6122 \ + --hash=sha256:084ef1af862eb07ec46d25f68689f2102a9fc0e05ce7b80f14f5fe51e4eef0f6 \ + --hash=sha256:0b82e28ee398a386f0807bba7884d30f25218855690f45115831bcce5d90822c \ + --hash=sha256:0e959b578856a3924bc0cbb710fc12c387b9412a951389f3ca61704a9e25f325 \ + --hash=sha256:0f21641cf4b30fca7aee061ced0ec7ad7b073518088b7c9969a297c0ae796c69 \ + --hash=sha256:196ecd6a36e4e9aa10270393bb98d8df88fccee0bf1e5128b91ae4eb4375896d \ + --hash=sha256:2400ef9c9e2299a25614eb1dea3db54a69b1349efd043bfac9c67630d136df36 \ + --hash=sha256:28d8b15e6275f12c8a207dc309dfa957903c927d08d0cc937ee3f63f200693cc \ + --hash=sha256:2afe9051da7ae7bd5905da5a949280c7d2bb75682e188f650a9d0f2756b834c6 \ + --hash=sha256:2eda353d8a27bcbcaa4cbed18994a74ab4d19a2ca897db188ea269ab9b71419b \ + --hash=sha256:32703d93296f5c1f4b53349ad3a250c2cae0fdecd3a3dd5d47e616d8d616af27 \ + --hash=sha256:33cd0565932807baddb67b96dbee92f2c374b5c89dee09fd74079aeb8c8dba61 \ + --hash=sha256:35b151772baff2c74cba7fa290ceaff4c3b11c0c881eb93eb5dbc05a7cfbba18 \ + --hash=sha256:36d1709f992593689b45bda411498d62c6e365f2ca00b84657d4dadd24de16db \ + --hash=sha256:42b0684e0e40cf26122427802486f6d93aea593612603a94fbf260c7eb1e9c1b \ + --hash=sha256:4ae387c9cb68ea569ca17e490d66d8142b81c3cc814bf179974b7d146e490bbb \ + --hash=sha256:53ecee2e23f7169b6117e99fc8a944e5e50f79e69758a83b52a00cb98ab2b2d2 \ + --hash=sha256:66ec79c3904820572d7e987abdf304281f141d37ad9a489b8e97066e7b9b6459 \ + --hash=sha256:67e1d20ad9ef3a563c59ef22e7a8a0b8210bd26604369ea4a30a7c66aefe504e \ + --hash=sha256:6f2debedf9ca60cf1d5bd466475638af5130f89965605cd818484d19987d3a21 \ + --hash=sha256:6fc361c34fb6aac015ce19435876635e5c6d21db31998b0920f675f131e043b8 \ + --hash=sha256:73a205dce83953d131a4aa1e0fd917a2fd1c5b1eef251e9d7152efefcbf5caf7 \ + --hash=sha256:7abcee80084cda3f7691f3eb1ce480d8df49cec637b429aa35986c1de71738aa \ + --hash=sha256:8c25ceb16df5b9435f3f6a9829204985b0e0cbee3b48aacd432c7d2c850b44d9 \ + --hash=sha256:966fe0e9c67490071f14c0d2b1cb2dfb3023c5ce39457343931415f08382f2db \ + --hash=sha256:9e82dcc8e56052715fb18b2429e3bca4823b1629136a2084fc45a9a5cecb9b64 \ + --hash=sha256:b20133d204d2bb56ba047642199603876c872026ca53e79c35b83772ab2cc505 \ + --hash=sha256:b39efa323140595abd3ecca8529d321ae50f55f3aa3ba9cc81ea56a6011953d5 \ + --hash=sha256:b47db11c2c3525083296069b98ac5221907455e989ae0c2e3008bde851921615 \ + --hash=sha256:b87e65d263b3e5d3bb92a57e2a6638e2f31110fa7aa890c7b2dbba42248d0a3f \ + --hash=sha256:b970c6da94d5bb18629db453d14f2a1300f6bf59b61e9b82377931ef95504866 \ + --hash=sha256:be9fcb48a55f023493482827d4f459bd263cc20efde64f204b97c123201850c6 \ + --hash=sha256:c2bc30226390d60ea19d9f82b19db005fe0452154a23c1c410c12ea801e43561 \ + --hash=sha256:c83782480a4a9da4d0feb51950131ba32e12e70813848b3343f6e18c28a66838 \ + --hash=sha256:cbc77da8c523d5abd028635ba850a6966fcee2c82e2bf65a41d1d8afe0f98be9 \ + --hash=sha256:ccac2bfebc306b862133e3bb71f3f6ee8bb525240089b2d952e4144b3a6d5da7 \ + --hash=sha256:d0527ce944105f257f605a827d6ebead966c752038b6e8656abb9c5edee6fc68 \ + --hash=sha256:d8ecde755e2e91bf773fc94e8c9d730cd7f2007004cb492263a794ec3899a1c8 \ + --hash=sha256:e3fb64c420688e5319ae25113a354015abbd8dffbfbc41781a1ea66fc7622ac3 \ + --hash=sha256:e5dfc1e64de5677cec922ffa8da89c546d0415bf6efdf081842e5d44c84e1f0e \ + --hash=sha256:ec5e529fb80935c94fe7b729f9972b50e351a0e6b50aa294fd5cabb109fcc29a \ + --hash=sha256:f37d847238971164fdbc68ade6f6574aecc9c0af714190e2083429ff68f4ce9d \ + --hash=sha256:f78ff2c9ed8dc2d036b0f4d640e22522213d047c1b14e61205a7e55c80a494d4 \ + --hash=sha256:f89660a348f4f78a92366240a61404e337586ef7f5909a2fef59ca88ef505493 \ + --hash=sha256:fc1e275c2f1d97b1a6450b8b0ea3ebfa6e087a611c2b26cb2404d48588abab7b # via secretstorage -docutils==0.22.2 \ - --hash=sha256:9fdb771707c8784c8f2728b67cb2c691305933d68137ef95a75db5f4dfbc213d \ - --hash=sha256:b0e98d679283fc3bb0ead8a5da7f501baa632654e7056e9c5846842213d674d8 +docutils==0.23 \ + --hash=sha256:25d013af9bf23bc1c7b2b093dff4208166c53a94786c9e447808335ef1185fea \ + --hash=sha256:746f5060322511280a1e50eb76846ed6bf2342984b2ac04dc42caa1a8d78799e # via readme-renderer -idna==3.10 \ - --hash=sha256:12f65c9b470abda6dc35cf8e63cc574b1c52b11df2c86030af0ac09b01b13ea9 \ - --hash=sha256:946d195a0d259cbba61165e88e65941f16e9b36ea6ddb97f00452bae8b1287d3 +id==1.6.1 \ + --hash=sha256:d0732d624fb46fd4e7bc4e5152f00214450953b9e772c182c1c22964def1a069 \ + --hash=sha256:f5ec41ed2629a508f5d0988eda142e190c9c6da971100612c4de9ad9f9b237ca + # via twine +idna==3.18 \ + --hash=sha256:7f952cbe720b688055e3f87de14f5c3e5fdaa8bc3928985c4077ca689de849a2 \ + --hash=sha256:ffb385a7e039654cef1ab9ef32c6fafe283c0c0467bba1d9029738ce4a14a848 # via requests -importlib-metadata==8.7.0 \ - --hash=sha256:d13b81ad223b890aa16c5471f2ac3056cf76c5f10f82d6f9292f0b415f389000 \ - --hash=sha256:e5dd1551894c77868a30651cef00984d50e1002d06942a7101d34870c5f02afd - # via - # keyring - # twine -jaraco-classes==3.4.0 \ +importlib-metadata==9.0.0 ; python_full_version < '3.12' and platform_machine != 'ppc64le' and platform_machine != 's390x' \ + --hash=sha256:2d21d1cc5a017bd0559e36150c21c830ab1dc304dedd1b7ea85d20f45ef3edd7 \ + --hash=sha256:a4f57ab599e6a2e3016d7595cfd72eb4661a5106e787a95bcc90c7105b831efc + # via keyring +jaraco-classes==3.4.0 ; platform_machine != 'ppc64le' and platform_machine != 's390x' \ --hash=sha256:47a024b51d0239c0dd8c8540c6c7f484be3b8fcf0b2d85c13825780d3b3f3acd \ --hash=sha256:f662826b6bed8cace05e7ff873ce0f9283b5c924470fe664fff1c2f00f581790 # via keyring -jaraco-context==6.0.1 \ - --hash=sha256:9bae4ea555cf0b14938dc0aee7c9f32ed303aa20a3b73e7dc80111628792d1b3 \ - --hash=sha256:f797fc481b490edb305122c9181830a3a5b76d84ef6d1aef2fb9b47ab956f9e4 +jaraco-context==6.1.2 ; platform_machine != 'ppc64le' and platform_machine != 's390x' \ + --hash=sha256:bf8150b79a2d5d91ae48629d8b427a8f7ba0e1097dd6202a9059f29a36379535 \ + --hash=sha256:f1a6c9d391e661cc5b8d39861ff077a7dc24dc23833ccee564b234b81c82dfe3 # via keyring -jaraco-functools==4.3.0 \ - --hash=sha256:227ff8ed6f7b8f62c56deff101545fa7543cf2c8e7b82a7c2116e672f29c26e8 \ - --hash=sha256:cfd13ad0dd2c47a3600b439ef72d8615d482cedcff1632930d6f28924d92f294 +jaraco-functools==4.6.0 ; platform_machine != 'ppc64le' and platform_machine != 's390x' \ + --hash=sha256:880c577ec9720b3a052d5bc611fb9f2269b3d87902ef42440df443b88e443280 \ + --hash=sha256:99e3dc0060c5cbe8fcd1cdb36258e2a65ca40f1566b2033b12abb1bb44dd3c30 # via keyring -jeepney==0.9.0 ; sys_platform == 'linux' \ +jeepney==0.9.0 ; platform_machine != 'ppc64le' and platform_machine != 's390x' and sys_platform == 'linux' \ --hash=sha256:97e5714520c16fc0a45695e5365a2e11b81ea79bba796e26f9f1d178cb182683 \ --hash=sha256:cf0e9e845622b81e4a28df94c40345400256ec608d0e55bb8a3feaa9163f5732 # via # keyring # secretstorage -keyring==25.6.0 \ - --hash=sha256:0b39998aa941431eb3d9b0d4b2460bc773b9df6fed7621c2dfb291a7e0187a66 \ - --hash=sha256:552a3f7af126ece7ed5c89753650eec89c7eaae8617d0aa4d9ad2b75111266bd +keyring==25.7.0 ; platform_machine != 'ppc64le' and platform_machine != 's390x' \ + --hash=sha256:be4a0b195f149690c166e850609a477c532ddbfbaed96a404d4e43f8d5e2689f \ + --hash=sha256:fe01bd85eb3f8fb3dd0405defdeac9a5b4f6f0439edbb3149577f244a2e8245b # via twine -markdown-it-py==4.0.0 \ - --hash=sha256:87327c59b172c5011896038353a81343b6754500a08cd7a4973bb48c6d578147 \ - --hash=sha256:cb0a2b4aa34f932c007117b194e945bd74e0ec24133ceb5bac59009cda1cb9f3 +markdown-it-py==4.2.0 \ + --hash=sha256:04a21681d6fbb623de53f6f364d352309d4094dd4194040a10fd51833e418d49 \ + --hash=sha256:9f7ebbcd14fe59494226453aed97c1070d83f8d24b6fc3a3bcf9a38092641c4a # via rich mdurl==0.1.2 \ --hash=sha256:84008a41e51615a49fc9966191ff91509e3c40b939176e643fd50a5c2196b8f8 \ --hash=sha256:bb413d29f5eea38f31dd4754dd7377d4465116fb207585f97bf925588687c1ba # via markdown-it-py -more-itertools==10.8.0 \ - --hash=sha256:52d4362373dcf7c52546bc4af9a86ee7c4579df9a8dc268be0a2f949d376cc9b \ - --hash=sha256:f638ddf8a1a0d134181275fb5d58b086ead7c6a72429ad725c67503f13ba30bd +more-itertools==11.1.0 ; platform_machine != 'ppc64le' and platform_machine != 's390x' \ + --hash=sha256:48e8f4d9e7e5878571ecf6f2b4e57634f93cd474cc8cfbd2376f2d11b396e30d \ + --hash=sha256:4b65538ae22f6fed0ce4874efd317463a7489796a0939fa66824dd542125a192 # via # jaraco-classes # jaraco-functools -nh3==0.3.0 \ - --hash=sha256:0649464ac8eee018644aacbc103874ccbfac80e3035643c3acaab4287e36e7f5 \ - --hash=sha256:16f8670201f7e8e0e05ed1a590eb84bfa51b01a69dd5caf1d3ea57733de6a52f \ - --hash=sha256:1adeb1062a1c2974bc75b8d1ecb014c5fd4daf2df646bbe2831f7c23659793f9 \ - --hash=sha256:37d3003d98dedca6cd762bf88f2e70b67f05100f6b949ffe540e189cc06887f9 \ - --hash=sha256:389d93d59b8214d51c400fb5b07866c2a4f79e4e14b071ad66c92184fec3a392 \ - --hash=sha256:3f1b4f8a264a0c86ea01da0d0c390fe295ea0bcacc52c2103aca286f6884f518 \ - --hash=sha256:423201bbdf3164a9e09aa01e540adbb94c9962cc177d5b1cbb385f5e1e79216e \ - --hash=sha256:634e34e6162e0408e14fb61d5e69dbaea32f59e847cfcfa41b66100a6b796f62 \ - --hash=sha256:6d68fa277b4a3cf04e5c4b84dd0c6149ff7d56c12b3e3fab304c525b850f613d \ - --hash=sha256:7275fdffaab10cc5801bf026e3c089d8de40a997afc9e41b981f7ac48c5aa7d5 \ - --hash=sha256:7852f038a054e0096dac12b8141191e02e93e0b4608c4b993ec7d4ffafea4e49 \ - --hash=sha256:7c915060a2c8131bef6a29f78debc29ba40859b6dbe2362ef9e5fd44f11487c2 \ - --hash=sha256:80fe20171c6da69c7978ecba33b638e951b85fb92059259edd285ff108b82a6d \ - --hash=sha256:a537ece1bf513e5a88d8cff8a872e12fe8d0f42ef71dd15a5e7520fecd191bbb \ - --hash=sha256:af5aa8127f62bbf03d68f67a956627b1bd0469703a35b3dad28d0c1195e6c7fb \ - --hash=sha256:b0612ccf5de8a480cf08f047b08f9d3fecc12e63d2ee91769cb19d7290614c23 \ - --hash=sha256:ba0caa8aa184196daa6e574d997a33867d6d10234018012d35f86d46024a2a95 \ - --hash=sha256:bae63772408fd63ad836ec569a7c8f444dd32863d0c67f6e0b25ebbd606afa95 \ - --hash=sha256:c7a32a7f0d89f7d30cb8f4a84bdbd56d1eb88b78a2434534f62c71dac538c450 \ - --hash=sha256:ce5e7185599f89b0e391e2f29cc12dc2e206167380cea49b33beda4891be2fe1 \ - --hash=sha256:d8ba24cb31525492ea71b6aac11a4adac91d828aadeff7c4586541bf5dc34d2f \ - --hash=sha256:d97d3efd61404af7e5721a0e74d81cdbfc6e5f97e11e731bb6d090e30a7b62b2 \ - --hash=sha256:e90883f9f85288f423c77b3f5a6f4486375636f25f793165112679a7b6363b35 \ - --hash=sha256:e9e6a7e4d38f7e8dda9edd1433af5170c597336c1a74b4693c5cb75ab2b30f2a \ - --hash=sha256:ec6cfdd2e0399cb79ba4dcffb2332b94d9696c52272ff9d48a630c5dca5e325a \ - --hash=sha256:f416c35efee3e6a6c9ab7716d9e57aa0a49981be915963a82697952cba1353e1 +nh3==0.3.6 \ + --hash=sha256:082675ff87b9385ec430ffe6d5847ba7456cc39b73720cd4add472f9f4cffd56 \ + --hash=sha256:2411e8c3cee81a1ddd62c2a5d50585c28aa5566d373ad1db92536b95ddb24ef2 \ + --hash=sha256:25c733bee928530556b1db0ea46c52cf5aa686146e38e60a6fc7cb801ef91cec \ + --hash=sha256:2f90d9a0cfdbee218994fdaaeeb5a0fde62d08f35e4eef0378ec1e2200172fd0 \ + --hash=sha256:34d2b0d934156b87ee114f599a3ba9b8b9e17b5d79652ba3a13fa50903de965e \ + --hash=sha256:36d06341bd501240d320f5942481ed5e6846136b666e1ba4faf802b78ebc875f \ + --hash=sha256:43bc1ed3fa0716295fabee29ba42b2667e4a51d140b0a68e092170a765474fa6 \ + --hash=sha256:44673b27010051ab5a5e438a86ec31bbda61d4a77d7e900af6b7be3037c1abae \ + --hash=sha256:455469a29951edc92bc48b47ac2281c3f2609e6c4f6a047056449f8c2c23facf \ + --hash=sha256:4713502748f564fee0633b37b3403783ce0a3af3a3d148ad91025a5bdadb7bc6 \ + --hash=sha256:5276ef17bdba9ad8040575c74072008b13aae429436e9d0429e718bb5f90f4da \ + --hash=sha256:597a8e843bea00b2eb5520658dc24a9bb032e7fc9e7c2c0c4cd29420220c9796 \ + --hash=sha256:69bbb92865a693d909db3a700d3c01537533844d0948c1e9323561ce06ecda41 \ + --hash=sha256:69f365963f63a1e9bff53bdbb3c542c7c2efed3e163c9d5d83a772a2ac468c21 \ + --hash=sha256:82ca5bf427ad1b216b65ede1a2e2d87dc49bec417ceba0f297213107d3cd9d78 \ + --hash=sha256:889932a97fb4abb6f95fef1914c0d269ebfb60011e67121c1163059b9449dbb4 \ + --hash=sha256:905f877dc66dd7aea4a76e54bcb26acb5ff8216f720c0017ccf63e0e6035698e \ + --hash=sha256:a43ebd7543555c3ac1bc353023d0794e75cb76f6f18f19c32e95441496c0cc25 \ + --hash=sha256:d14bf7982e7a77c0c775634c29c07ce08b38a046df73e1c1f139b3e82f18a38e \ + --hash=sha256:e196fa70c2ff2eb4de7d3df3108f8f358c1d69dff20d45b11f20a5aa227ffb6d \ + --hash=sha256:e1b160831c9cdb06a6c79c2f9cdb11386602938f9af260d1c457a85add4f6f69 \ + --hash=sha256:e6b7beece07525dc6e6b0fc2f104442de2ba328360ad00e50cbe2e1fd620447d \ + --hash=sha256:edb2b4a1a27523e6cc7c417f8d21ce3d005243548b93e56b762b66b0c7f589f9 \ + --hash=sha256:f2f14b7ae1fca99c4a66c981aac3974e7fbc1ca30a12673d223ae1df76680917 \ + --hash=sha256:f338ac7d594c067679f1e99b4f5ec3906842979560f9d8f15d6bdfa39a353b10 \ + --hash=sha256:f3736c9dd3d1856f80cd031715b84ca75cda2bbb1ac802c3da26bfce590838d7 \ + --hash=sha256:f5ed5fe84aee7f39db95c214a7421bf0499fbf500fec6d86a4e29bfc37971438 # via readme-renderer -pkginfo==1.12.1.2 \ - --hash=sha256:5cd957824ac36f140260964eba3c6be6442a8359b8c48f4adf90210f33a04b7b \ - --hash=sha256:c783ac885519cab2c34927ccfa6bf64b5a704d7c69afaea583dd9b7afe969343 +packaging==26.2 \ + --hash=sha256:5fc45236b9446107ff2415ce77c807cee2862cb6fac22b8a73826d0693b0980e \ + --hash=sha256:ff452ff5a3e828ce110190feff1178bb1f2ea2281fa2075aadb987c2fb221661 # via twine -pycparser==2.23 ; platform_python_implementation != 'PyPy' and sys_platform == 'linux' \ - --hash=sha256:491c8be9c040f5390f5bf44a5b07752bd07f56edf992381b05c701439eec10f6 \ - --hash=sha256:c3702b6d3dd8c7abc1afa565d7e63d53a1d0bd86cdc24edd75470f4de499cfcc +pycparser==3.0 ; implementation_name != 'PyPy' and platform_machine != 'ppc64le' and platform_machine != 's390x' and platform_python_implementation != 'PyPy' and sys_platform == 'linux' \ + --hash=sha256:600f49d217304a5902ac3c37e1281c9fe94e4d0489de643a9504c5cdfdfc6b29 \ + --hash=sha256:b727414169a36b7d524c1c3e31839a521725078d7b2ff038656844266160a992 # via cffi -pygments==2.19.2 \ - --hash=sha256:636cb2477cec7f8952536970bc533bc43743542f70392ae026374600add5b887 \ - --hash=sha256:86540386c03d588bb81d44bc3928634ff26449851e99741617ecb9037ee5ec0b +pygments==2.20.0 \ + --hash=sha256:6757cd03768053ff99f3039c1a36d6c0aa0b263438fcab17520b30a303a82b5f \ + --hash=sha256:81a9e26dd42fd28a23a2d169d86d7ac03b46e2f8b59ed4698fb4785f946d0176 # via # readme-renderer # rich -pywin32-ctypes==0.2.3 ; sys_platform == 'win32' \ +pywin32-ctypes==0.2.3 ; platform_machine != 'ppc64le' and platform_machine != 's390x' and sys_platform == 'win32' \ --hash=sha256:8a1513379d709975552d202d942d9837758905c8d01eb82b8bcc30918929e7b8 \ --hash=sha256:d162dc04946d704503b2edc4d55f3dba5c1d539ead017afa00142c38b9885755 # via keyring -readme-renderer==44.0 \ - --hash=sha256:2fbca89b81a08526aadf1357a8c2ae889ec05fb03f5da67f9769c9a592166151 \ - --hash=sha256:8712034eabbfa6805cacf1402b4eeb2a73028f72d1166d6f5cb7f9c047c5d1e1 +readme-renderer==45.0 \ + --hash=sha256:030a8fac74904f8fba11ad1bb6964e3f76e896dc7e5e71f16af190c9056696d1 \ + --hash=sha256:3385ed220117104a2bceb4a9dac8c5fdf6d1f96890d7ea2a9c7174fd5c84091f # via twine -requests==2.33.0 \ - --hash=sha256:3324635456fa185245e24865e810cecec7b4caf933d7eb133dcde67d48cee69b \ - --hash=sha256:c7ebc5e8b0f21837386ad0e1c8fe8b829fa5f544d8df3b2253bff14ef29d7652 +requests==2.34.2 \ + --hash=sha256:2a0d60c172f83ac6ab31e4554906c0f3b3588d37b5cb939b1c061f4907e278e0 \ + --hash=sha256:f288924cae4e29463698d6d60bc6a4da69c89185ad1e0bcc4104f584e960b9ed # via # requests-toolbelt # twine @@ -339,25 +372,26 @@ rfc3986==2.0.0 \ --hash=sha256:50b1502b60e289cb37883f3dfd34532b8873c7de9f49bb546641ce9cbd256ebd \ --hash=sha256:97aacf9dbd4bfd829baad6e6309fa6573aaf1be3f6fa735c8ab05e46cecb261c # via twine -rich==14.1.0 \ - --hash=sha256:536f5f1785986d6dbdea3c75205c473f970777b4a0d6c6dd1b696aa05a3fa04f \ - --hash=sha256:e497a48b844b0320d45007cdebfeaeed8db2a4f4bcf49f15e455cfc4af11eaa8 +rich==15.0.0 \ + --hash=sha256:33bd4ef74232fb73fe9279a257718407f169c09b78a87ad3d296f548e27de0bb \ + --hash=sha256:edd07a4824c6b40189fb7ac9bc4c52536e9780fbbfbddf6f1e2502c31b068c36 # via twine -secretstorage==3.3.3 ; sys_platform == 'linux' \ - --hash=sha256:2403533ef369eca6d2ba81718576c5e0f564d5cca1b58f73a8b23e7d4eeebd77 \ - --hash=sha256:f356e6628222568e3af06f2eba8df495efa13b3b63081dafd4f7d9a7b7bc9f99 +secretstorage==3.5.0 ; platform_machine != 'ppc64le' and platform_machine != 's390x' and sys_platform == 'linux' \ + --hash=sha256:0ce65888c0725fcb2c5bc0fdb8e5438eece02c523557ea40ce0703c266248137 \ + --hash=sha256:f04b8e4689cbce351744d5537bf6b1329c6fc68f91fa666f60a380edddcd11be # via keyring -twine==5.1.1 \ - --hash=sha256:215dbe7b4b94c2c50a7315c0275d2258399280fbb7d04182c7e55e24b5f93997 \ - --hash=sha256:9aa0825139c02b3434d913545c7b847a21c835e11597f5255842d457da2322db +twine==7.0.0 \ + --hash=sha256:85cdb29c518efef867360ae4acd4b0dfd61c8654a22fca08e6f8539f05022177 \ + --hash=sha256:b854164df26db268af05f49aa5c0344b10e27a494343ff05b1e0bad3b135f5a7 # via -r tools/publish/requirements.in -urllib3==2.6.3 \ - --hash=sha256:1b62b6884944a57dbe321509ab94fd4d3b307075e0c2eae991ac71ee15ad38ed \ - --hash=sha256:bf272323e553dfb2e87d9bfd225ca7b0f467b919d7bbd355436d3fd37cb0acd4 +urllib3==2.7.0 \ + --hash=sha256:231e0ec3b63ceb14667c67be60f2f2c40a518cb38b03af60abc813da26505f4c \ + --hash=sha256:9fb4c81ebbb1ce9531cce37674bbc6f1360472bc18ca9a553ede278ef7276897 # via + # id # requests # twine -zipp==3.23.0 \ - --hash=sha256:071652d6115ed432f5ce1d34c336c0adfd6a884660d1e9712a256d3d3bd4b14e \ - --hash=sha256:a07157588a12518c9d4034df3fbbee09c814741a33ff63c05fa29d26a2404166 +zipp==4.1.0 ; python_full_version < '3.12' and platform_machine != 'ppc64le' and platform_machine != 's390x' \ + --hash=sha256:25ad4e16390cd314347dd8f1de67a2ac538ae658ed4ab9db16029c07c188e97f \ + --hash=sha256:4cb57381f544315db7688e976e922a2b18cdb513d21cc194eb42232ba2a3e602 # via importlib-metadata diff --git a/tools/publish/requirements_windows.txt b/tools/publish/requirements_windows.txt index 1cbd8197fe..b235d95a52 100644 --- a/tools/publish/requirements_windows.txt +++ b/tools/publish/requirements_windows.txt @@ -6,9 +6,9 @@ backports-tarfile==1.2.0 \ --hash=sha256:77e284d754527b01fb1e6fa8a1afe577858ebe4e9dad8919e34c862cb399bc34 \ --hash=sha256:d75e02c268746e1b8144c278978b6e98e85de6ad16f8e4b0844a154557eca991 # via jaraco-context -certifi==2025.10.5 \ - --hash=sha256:0f212c2744a9bb6de0c56639a6f68afe01ecd92d91f14ae897c4fe7bbeeef0de \ - --hash=sha256:47c09d31ccf2acf0be3f701ea53595ee7e0b8fa08801c6624be771df09ae7b43 +certifi==2026.7.22 \ + --hash=sha256:62f22742b58a1a33014a2b6b706588a8d7e2a88ae7bd1a6ebe8c992928483775 \ + --hash=sha256:741e2c3b351ddf169a738da9f2c048608ff7f2c5cc02f1ebc6b118bb090d5d55 # via requests charset-normalizer==3.4.9 \ --hash=sha256:0327fcd59a935777d83410750c50600ee9571af2846f71ce40f25b13da1ef380 \ @@ -105,85 +105,88 @@ charset-normalizer==3.4.9 \ --hash=sha256:fa36ec09ef71d158186bc79e359ff5fdd6e7996fe8ab638f00d6b93139ba4fcf \ --hash=sha256:fe2c7201c642b7c308f1675355ad7ff7b66acfe3541625efe5a3ad38f29d6115 # via requests -docutils==0.22.2 \ - --hash=sha256:9fdb771707c8784c8f2728b67cb2c691305933d68137ef95a75db5f4dfbc213d \ - --hash=sha256:b0e98d679283fc3bb0ead8a5da7f501baa632654e7056e9c5846842213d674d8 +docutils==0.23 \ + --hash=sha256:25d013af9bf23bc1c7b2b093dff4208166c53a94786c9e447808335ef1185fea \ + --hash=sha256:746f5060322511280a1e50eb76846ed6bf2342984b2ac04dc42caa1a8d78799e # via readme-renderer -idna==3.10 \ - --hash=sha256:12f65c9b470abda6dc35cf8e63cc574b1c52b11df2c86030af0ac09b01b13ea9 \ - --hash=sha256:946d195a0d259cbba61165e88e65941f16e9b36ea6ddb97f00452bae8b1287d3 +id==1.6.1 \ + --hash=sha256:d0732d624fb46fd4e7bc4e5152f00214450953b9e772c182c1c22964def1a069 \ + --hash=sha256:f5ec41ed2629a508f5d0988eda142e190c9c6da971100612c4de9ad9f9b237ca + # via twine +idna==3.18 \ + --hash=sha256:7f952cbe720b688055e3f87de14f5c3e5fdaa8bc3928985c4077ca689de849a2 \ + --hash=sha256:ffb385a7e039654cef1ab9ef32c6fafe283c0c0467bba1d9029738ce4a14a848 # via requests -importlib-metadata==8.7.0 \ - --hash=sha256:d13b81ad223b890aa16c5471f2ac3056cf76c5f10f82d6f9292f0b415f389000 \ - --hash=sha256:e5dd1551894c77868a30651cef00984d50e1002d06942a7101d34870c5f02afd - # via - # keyring - # twine +importlib-metadata==9.0.0 \ + --hash=sha256:2d21d1cc5a017bd0559e36150c21c830ab1dc304dedd1b7ea85d20f45ef3edd7 \ + --hash=sha256:a4f57ab599e6a2e3016d7595cfd72eb4661a5106e787a95bcc90c7105b831efc + # via keyring jaraco-classes==3.4.0 \ --hash=sha256:47a024b51d0239c0dd8c8540c6c7f484be3b8fcf0b2d85c13825780d3b3f3acd \ --hash=sha256:f662826b6bed8cace05e7ff873ce0f9283b5c924470fe664fff1c2f00f581790 # via keyring -jaraco-context==6.0.1 \ - --hash=sha256:9bae4ea555cf0b14938dc0aee7c9f32ed303aa20a3b73e7dc80111628792d1b3 \ - --hash=sha256:f797fc481b490edb305122c9181830a3a5b76d84ef6d1aef2fb9b47ab956f9e4 +jaraco-context==6.1.2 \ + --hash=sha256:bf8150b79a2d5d91ae48629d8b427a8f7ba0e1097dd6202a9059f29a36379535 \ + --hash=sha256:f1a6c9d391e661cc5b8d39861ff077a7dc24dc23833ccee564b234b81c82dfe3 # via keyring -jaraco-functools==4.3.0 \ - --hash=sha256:227ff8ed6f7b8f62c56deff101545fa7543cf2c8e7b82a7c2116e672f29c26e8 \ - --hash=sha256:cfd13ad0dd2c47a3600b439ef72d8615d482cedcff1632930d6f28924d92f294 +jaraco-functools==4.6.0 \ + --hash=sha256:880c577ec9720b3a052d5bc611fb9f2269b3d87902ef42440df443b88e443280 \ + --hash=sha256:99e3dc0060c5cbe8fcd1cdb36258e2a65ca40f1566b2033b12abb1bb44dd3c30 # via keyring -keyring==25.6.0 \ - --hash=sha256:0b39998aa941431eb3d9b0d4b2460bc773b9df6fed7621c2dfb291a7e0187a66 \ - --hash=sha256:552a3f7af126ece7ed5c89753650eec89c7eaae8617d0aa4d9ad2b75111266bd +keyring==25.7.0 \ + --hash=sha256:be4a0b195f149690c166e850609a477c532ddbfbaed96a404d4e43f8d5e2689f \ + --hash=sha256:fe01bd85eb3f8fb3dd0405defdeac9a5b4f6f0439edbb3149577f244a2e8245b # via twine -markdown-it-py==4.0.0 \ - --hash=sha256:87327c59b172c5011896038353a81343b6754500a08cd7a4973bb48c6d578147 \ - --hash=sha256:cb0a2b4aa34f932c007117b194e945bd74e0ec24133ceb5bac59009cda1cb9f3 +markdown-it-py==4.2.0 \ + --hash=sha256:04a21681d6fbb623de53f6f364d352309d4094dd4194040a10fd51833e418d49 \ + --hash=sha256:9f7ebbcd14fe59494226453aed97c1070d83f8d24b6fc3a3bcf9a38092641c4a # via rich mdurl==0.1.2 \ --hash=sha256:84008a41e51615a49fc9966191ff91509e3c40b939176e643fd50a5c2196b8f8 \ --hash=sha256:bb413d29f5eea38f31dd4754dd7377d4465116fb207585f97bf925588687c1ba # via markdown-it-py -more-itertools==10.8.0 \ - --hash=sha256:52d4362373dcf7c52546bc4af9a86ee7c4579df9a8dc268be0a2f949d376cc9b \ - --hash=sha256:f638ddf8a1a0d134181275fb5d58b086ead7c6a72429ad725c67503f13ba30bd +more-itertools==11.1.0 \ + --hash=sha256:48e8f4d9e7e5878571ecf6f2b4e57634f93cd474cc8cfbd2376f2d11b396e30d \ + --hash=sha256:4b65538ae22f6fed0ce4874efd317463a7489796a0939fa66824dd542125a192 # via # jaraco-classes # jaraco-functools -nh3==0.3.0 \ - --hash=sha256:0649464ac8eee018644aacbc103874ccbfac80e3035643c3acaab4287e36e7f5 \ - --hash=sha256:16f8670201f7e8e0e05ed1a590eb84bfa51b01a69dd5caf1d3ea57733de6a52f \ - --hash=sha256:1adeb1062a1c2974bc75b8d1ecb014c5fd4daf2df646bbe2831f7c23659793f9 \ - --hash=sha256:37d3003d98dedca6cd762bf88f2e70b67f05100f6b949ffe540e189cc06887f9 \ - --hash=sha256:389d93d59b8214d51c400fb5b07866c2a4f79e4e14b071ad66c92184fec3a392 \ - --hash=sha256:3f1b4f8a264a0c86ea01da0d0c390fe295ea0bcacc52c2103aca286f6884f518 \ - --hash=sha256:423201bbdf3164a9e09aa01e540adbb94c9962cc177d5b1cbb385f5e1e79216e \ - --hash=sha256:634e34e6162e0408e14fb61d5e69dbaea32f59e847cfcfa41b66100a6b796f62 \ - --hash=sha256:6d68fa277b4a3cf04e5c4b84dd0c6149ff7d56c12b3e3fab304c525b850f613d \ - --hash=sha256:7275fdffaab10cc5801bf026e3c089d8de40a997afc9e41b981f7ac48c5aa7d5 \ - --hash=sha256:7852f038a054e0096dac12b8141191e02e93e0b4608c4b993ec7d4ffafea4e49 \ - --hash=sha256:7c915060a2c8131bef6a29f78debc29ba40859b6dbe2362ef9e5fd44f11487c2 \ - --hash=sha256:80fe20171c6da69c7978ecba33b638e951b85fb92059259edd285ff108b82a6d \ - --hash=sha256:a537ece1bf513e5a88d8cff8a872e12fe8d0f42ef71dd15a5e7520fecd191bbb \ - --hash=sha256:af5aa8127f62bbf03d68f67a956627b1bd0469703a35b3dad28d0c1195e6c7fb \ - --hash=sha256:b0612ccf5de8a480cf08f047b08f9d3fecc12e63d2ee91769cb19d7290614c23 \ - --hash=sha256:ba0caa8aa184196daa6e574d997a33867d6d10234018012d35f86d46024a2a95 \ - --hash=sha256:bae63772408fd63ad836ec569a7c8f444dd32863d0c67f6e0b25ebbd606afa95 \ - --hash=sha256:c7a32a7f0d89f7d30cb8f4a84bdbd56d1eb88b78a2434534f62c71dac538c450 \ - --hash=sha256:ce5e7185599f89b0e391e2f29cc12dc2e206167380cea49b33beda4891be2fe1 \ - --hash=sha256:d8ba24cb31525492ea71b6aac11a4adac91d828aadeff7c4586541bf5dc34d2f \ - --hash=sha256:d97d3efd61404af7e5721a0e74d81cdbfc6e5f97e11e731bb6d090e30a7b62b2 \ - --hash=sha256:e90883f9f85288f423c77b3f5a6f4486375636f25f793165112679a7b6363b35 \ - --hash=sha256:e9e6a7e4d38f7e8dda9edd1433af5170c597336c1a74b4693c5cb75ab2b30f2a \ - --hash=sha256:ec6cfdd2e0399cb79ba4dcffb2332b94d9696c52272ff9d48a630c5dca5e325a \ - --hash=sha256:f416c35efee3e6a6c9ab7716d9e57aa0a49981be915963a82697952cba1353e1 +nh3==0.3.6 \ + --hash=sha256:082675ff87b9385ec430ffe6d5847ba7456cc39b73720cd4add472f9f4cffd56 \ + --hash=sha256:2411e8c3cee81a1ddd62c2a5d50585c28aa5566d373ad1db92536b95ddb24ef2 \ + --hash=sha256:25c733bee928530556b1db0ea46c52cf5aa686146e38e60a6fc7cb801ef91cec \ + --hash=sha256:2f90d9a0cfdbee218994fdaaeeb5a0fde62d08f35e4eef0378ec1e2200172fd0 \ + --hash=sha256:34d2b0d934156b87ee114f599a3ba9b8b9e17b5d79652ba3a13fa50903de965e \ + --hash=sha256:36d06341bd501240d320f5942481ed5e6846136b666e1ba4faf802b78ebc875f \ + --hash=sha256:43bc1ed3fa0716295fabee29ba42b2667e4a51d140b0a68e092170a765474fa6 \ + --hash=sha256:44673b27010051ab5a5e438a86ec31bbda61d4a77d7e900af6b7be3037c1abae \ + --hash=sha256:455469a29951edc92bc48b47ac2281c3f2609e6c4f6a047056449f8c2c23facf \ + --hash=sha256:4713502748f564fee0633b37b3403783ce0a3af3a3d148ad91025a5bdadb7bc6 \ + --hash=sha256:5276ef17bdba9ad8040575c74072008b13aae429436e9d0429e718bb5f90f4da \ + --hash=sha256:597a8e843bea00b2eb5520658dc24a9bb032e7fc9e7c2c0c4cd29420220c9796 \ + --hash=sha256:69bbb92865a693d909db3a700d3c01537533844d0948c1e9323561ce06ecda41 \ + --hash=sha256:69f365963f63a1e9bff53bdbb3c542c7c2efed3e163c9d5d83a772a2ac468c21 \ + --hash=sha256:82ca5bf427ad1b216b65ede1a2e2d87dc49bec417ceba0f297213107d3cd9d78 \ + --hash=sha256:889932a97fb4abb6f95fef1914c0d269ebfb60011e67121c1163059b9449dbb4 \ + --hash=sha256:905f877dc66dd7aea4a76e54bcb26acb5ff8216f720c0017ccf63e0e6035698e \ + --hash=sha256:a43ebd7543555c3ac1bc353023d0794e75cb76f6f18f19c32e95441496c0cc25 \ + --hash=sha256:d14bf7982e7a77c0c775634c29c07ce08b38a046df73e1c1f139b3e82f18a38e \ + --hash=sha256:e196fa70c2ff2eb4de7d3df3108f8f358c1d69dff20d45b11f20a5aa227ffb6d \ + --hash=sha256:e1b160831c9cdb06a6c79c2f9cdb11386602938f9af260d1c457a85add4f6f69 \ + --hash=sha256:e6b7beece07525dc6e6b0fc2f104442de2ba328360ad00e50cbe2e1fd620447d \ + --hash=sha256:edb2b4a1a27523e6cc7c417f8d21ce3d005243548b93e56b762b66b0c7f589f9 \ + --hash=sha256:f2f14b7ae1fca99c4a66c981aac3974e7fbc1ca30a12673d223ae1df76680917 \ + --hash=sha256:f338ac7d594c067679f1e99b4f5ec3906842979560f9d8f15d6bdfa39a353b10 \ + --hash=sha256:f3736c9dd3d1856f80cd031715b84ca75cda2bbb1ac802c3da26bfce590838d7 \ + --hash=sha256:f5ed5fe84aee7f39db95c214a7421bf0499fbf500fec6d86a4e29bfc37971438 # via readme-renderer -pkginfo==1.12.1.2 \ - --hash=sha256:5cd957824ac36f140260964eba3c6be6442a8359b8c48f4adf90210f33a04b7b \ - --hash=sha256:c783ac885519cab2c34927ccfa6bf64b5a704d7c69afaea583dd9b7afe969343 +packaging==26.2 \ + --hash=sha256:5fc45236b9446107ff2415ce77c807cee2862cb6fac22b8a73826d0693b0980e \ + --hash=sha256:ff452ff5a3e828ce110190feff1178bb1f2ea2281fa2075aadb987c2fb221661 # via twine -pygments==2.19.2 \ - --hash=sha256:636cb2477cec7f8952536970bc533bc43743542f70392ae026374600add5b887 \ - --hash=sha256:86540386c03d588bb81d44bc3928634ff26449851e99741617ecb9037ee5ec0b +pygments==2.20.0 \ + --hash=sha256:6757cd03768053ff99f3039c1a36d6c0aa0b263438fcab17520b30a303a82b5f \ + --hash=sha256:81a9e26dd42fd28a23a2d169d86d7ac03b46e2f8b59ed4698fb4785f946d0176 # via # readme-renderer # rich @@ -191,13 +194,13 @@ pywin32-ctypes==0.2.3 \ --hash=sha256:8a1513379d709975552d202d942d9837758905c8d01eb82b8bcc30918929e7b8 \ --hash=sha256:d162dc04946d704503b2edc4d55f3dba5c1d539ead017afa00142c38b9885755 # via keyring -readme-renderer==44.0 \ - --hash=sha256:2fbca89b81a08526aadf1357a8c2ae889ec05fb03f5da67f9769c9a592166151 \ - --hash=sha256:8712034eabbfa6805cacf1402b4eeb2a73028f72d1166d6f5cb7f9c047c5d1e1 +readme-renderer==45.0 \ + --hash=sha256:030a8fac74904f8fba11ad1bb6964e3f76e896dc7e5e71f16af190c9056696d1 \ + --hash=sha256:3385ed220117104a2bceb4a9dac8c5fdf6d1f96890d7ea2a9c7174fd5c84091f # via twine -requests==2.33.0 \ - --hash=sha256:3324635456fa185245e24865e810cecec7b4caf933d7eb133dcde67d48cee69b \ - --hash=sha256:c7ebc5e8b0f21837386ad0e1c8fe8b829fa5f544d8df3b2253bff14ef29d7652 +requests==2.34.2 \ + --hash=sha256:2a0d60c172f83ac6ab31e4554906c0f3b3588d37b5cb939b1c061f4907e278e0 \ + --hash=sha256:f288924cae4e29463698d6d60bc6a4da69c89185ad1e0bcc4104f584e960b9ed # via # requests-toolbelt # twine @@ -209,21 +212,22 @@ rfc3986==2.0.0 \ --hash=sha256:50b1502b60e289cb37883f3dfd34532b8873c7de9f49bb546641ce9cbd256ebd \ --hash=sha256:97aacf9dbd4bfd829baad6e6309fa6573aaf1be3f6fa735c8ab05e46cecb261c # via twine -rich==14.1.0 \ - --hash=sha256:536f5f1785986d6dbdea3c75205c473f970777b4a0d6c6dd1b696aa05a3fa04f \ - --hash=sha256:e497a48b844b0320d45007cdebfeaeed8db2a4f4bcf49f15e455cfc4af11eaa8 +rich==15.0.0 \ + --hash=sha256:33bd4ef74232fb73fe9279a257718407f169c09b78a87ad3d296f548e27de0bb \ + --hash=sha256:edd07a4824c6b40189fb7ac9bc4c52536e9780fbbfbddf6f1e2502c31b068c36 # via twine -twine==5.1.1 \ - --hash=sha256:215dbe7b4b94c2c50a7315c0275d2258399280fbb7d04182c7e55e24b5f93997 \ - --hash=sha256:9aa0825139c02b3434d913545c7b847a21c835e11597f5255842d457da2322db +twine==7.0.0 \ + --hash=sha256:85cdb29c518efef867360ae4acd4b0dfd61c8654a22fca08e6f8539f05022177 \ + --hash=sha256:b854164df26db268af05f49aa5c0344b10e27a494343ff05b1e0bad3b135f5a7 # via -r tools/publish/requirements.in -urllib3==2.6.3 \ - --hash=sha256:1b62b6884944a57dbe321509ab94fd4d3b307075e0c2eae991ac71ee15ad38ed \ - --hash=sha256:bf272323e553dfb2e87d9bfd225ca7b0f467b919d7bbd355436d3fd37cb0acd4 +urllib3==2.7.0 \ + --hash=sha256:231e0ec3b63ceb14667c67be60f2f2c40a518cb38b03af60abc813da26505f4c \ + --hash=sha256:9fb4c81ebbb1ce9531cce37674bbc6f1360472bc18ca9a553ede278ef7276897 # via + # id # requests # twine -zipp==3.23.0 \ - --hash=sha256:071652d6115ed432f5ce1d34c336c0adfd6a884660d1e9712a256d3d3bd4b14e \ - --hash=sha256:a07157588a12518c9d4034df3fbbee09c814741a33ff63c05fa29d26a2404166 +zipp==4.1.0 \ + --hash=sha256:25ad4e16390cd314347dd8f1de67a2ac538ae658ed4ab9db16029c07c188e97f \ + --hash=sha256:4cb57381f544315db7688e976e922a2b18cdb513d21cc194eb42232ba2a3e602 # via importlib-metadata From be47c220ea68f025dc9aca2102d86275e6b38fec Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Wed, 29 Jul 2026 03:37:03 +0000 Subject: [PATCH 871/922] build(deps): bump secretstorage from 3.3.3 to 3.5.0 in /tools/publish (#3967) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Bumps [secretstorage](https://github.com/mitya57/secretstorage) from 3.3.3 to 3.5.0.
Changelog

Sourced from secretstorage's changelog.

SecretStorage 3.5.0, 2025-11-23

  • Added timeout argument to the unlock() methods of Collection and Item [[#33](https://github.com/mitya57/secretstorage/issues/33)_].
  • Removed int_to_bytes() function in favor of the built-in method.

.. _[#33](https://github.com/mitya57/secretstorage/issues/33): mitya57/secretstorage#33

SecretStorage 3.4.1, 2025-11-11

  • Make sure public key length is exactly 128 bytes [[#48](https://github.com/mitya57/secretstorage/issues/48)_]. This fixes Client public key size is invalid error from KWallet.

.. _[#48](https://github.com/mitya57/secretstorage/issues/48): mitya57/secretstorage#48

SecretStorage 3.4.0, 2025-09-09

  • Handle D-Bus UnknownObject error when no collection is found [[#43](https://github.com/mitya57/secretstorage/issues/43)_]. Thanks to Renato Alencar for the pull request!
  • Added __repr__ methods to Collection and Item classes [[#47](https://github.com/mitya57/secretstorage/issues/47)_].
  • Moved project metadata to pyproject.toml.
  • Python ≥ 3.10 and setuptools ≥ 77.0 are now required.
  • Various code modernizations. Thanks to Hugo van Kemenade and Tomasz Kłoczko for the pull requests!

.. _[#43](https://github.com/mitya57/secretstorage/issues/43): mitya57/secretstorage#43 .. _[#47](https://github.com/mitya57/secretstorage/issues/47): mitya57/secretstorage#47

Commits
  • 3a3c006 Releasing version 3.5.0
  • fed7361 docs: Replace broken KeePassXC link with a working one
  • a97aa4b Update copyright years, again
  • d8d1ea0 Add timeout argument to unlock() methods
  • 2db9b67 util: Correct documentation for the exec_prompt() function
  • cc1e18d Get rid of int_to_bytes() function
  • d036431 Releasing version 3.4.1
  • d583dc6 Run the tests with Python 3.14
  • 070e560 Update copyright years
  • bff61ca Make sure my_public_key length is exactly 128 bytes
  • Additional commits viewable in compare view

[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=secretstorage&package-manager=pip&previous-version=3.3.3&new-version=3.5.0)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) ---
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot show ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
--------- Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Richard Levasseur From 905a1c1d061cedfb31b058fb866d9261710f9469 Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Wed, 29 Jul 2026 03:37:23 +0000 Subject: [PATCH 872/922] build(deps): bump the pip group across 1 directory with 3 updates (#3912) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Bumps the pip group with 3 updates in the /tools/publish directory: [cryptography](https://github.com/pyca/cryptography), [idna](https://github.com/kjd/idna) and [urllib3](https://github.com/urllib3/urllib3). Updates `cryptography` from 46.0.7 to 48.0.1
Changelog

Sourced from cryptography's changelog.

48.0.1 - 2026-06-09


* Updated Windows, macOS, and Linux wheels to be compiled with OpenSSL
4.0.1.

.. _v48-0-0:

48.0.0 - 2026-05-04

  • BACKWARDS INCOMPATIBLE: Support for Python 3.8 has been removed. cryptography now requires Python 3.9 or later.

  • BACKWARDS INCOMPATIBLE: Loading an X.509 CRL whose inner TBSCertList.signature algorithm does not match the outer signatureAlgorithm now raises ValueError. Previously, such CRLs were parsed successfully and only rejected during signature validation.

  • Added support for :doc:/hazmat/primitives/asymmetric/mlkem and :doc:/hazmat/primitives/asymmetric/mldsa when using OpenSSL 3.5.0 or later, in addition to the existing AWS-LC and BoringSSL support. This means post-quantum algorithms are now available to users of our wheels.

    • Note: Going forward, we do not guarantee that all functionality in cryptography will be available when building against OpenSSL. See :doc:/statements/state-of-openssl for more information.

.. _v47-0-0:

47.0.0 - 2026-04-24


* Support for Python 3.8 is deprecated and will be removed in the next
  ``cryptography`` release.
* **BACKWARDS INCOMPATIBLE:** Support for binary elliptic curves
  (``SECT*`` classes) has been removed. These curves are rarely used and
  have additional security considerations that make them undesirable.
* **BACKWARDS INCOMPATIBLE:** Support for OpenSSL 1.1.x has been
removed.
OpenSSL 3.0.0 or later is now required. LibreSSL, BoringSSL, and AWS-LC
  continue to be supported.
* **BACKWARDS INCOMPATIBLE:** Dropped support for LibreSSL < 4.1.
* **BACKWARDS INCOMPATIBLE:** Loading keys with unsupported algorithms
or
  keys with unsupported explicit curve encodings now raises
  :class:`~cryptography.exceptions.UnsupportedAlgorithm` instead of
  ``ValueError``. This change affects

:func:`~cryptography.hazmat.primitives.serialization.load_pem_private_key`,

:func:`~cryptography.hazmat.primitives.serialization.load_der_private_key`,

:func:`~cryptography.hazmat.primitives.serialization.load_pem_public_key`,

:func:`~cryptography.hazmat.primitives.serialization.load_der_public_key`,
  and :meth:`~cryptography.x509.Certificate.public_key` when called on
  certificates with unsupported public key algorithms.
</tr></table>

... (truncated)

Commits

Updates `idna` from 3.10 to 3.15
Changelog

Sourced from idna's changelog.

3.15 (2026-05-12)

  • Enforce DNS-length cap on individual labels early in check_label, short-circuiting contextual-rule processing for oversized input while staying compatible with UTS 46 usage.
  • Tidy core helpers: hoist bidi category sets to module-level frozensets (avoiding per-codepoint list construction), simplify length checks, and reuse the shared _unicode_dots_re from idna.core in the codec module.
  • Use raise ... from err for proper exception chaining and switch internal string formatting to f-strings.
  • Allow flit_core 4.x in the build backend.
  • Expand the ruff lint set (flake8-bugbear, flake8-simplify, pyupgrade, perflint) and apply the surfaced fixes; pin lint CI to Python 3.14.
  • Add Dependabot configuration for GitHub Actions.
  • Convert README and HISTORY from reStructuredText to Markdown.
  • Reference CVE-2026-45409 for the 3.14 advisory in place of the initial GHSA identifier.

Thanks to Felix Yan, Stan Ulbrych, and metsw24-max for contributions to this release.

3.14 (2026-05-10)

  • Removed opportunity to process long inputs into quadratic time by rejecting oversize inputs up-front. Closes a bypass of the CVE-2024-3651 mitigation. [CVE-2026-45409]

Thanks to Stan Ulbrych for reporting the issue.

3.13 (2026-04-22)

  • Correct classification error for codepoint U+A7F1

3.12 (2026-04-21)

  • Update to Unicode 17.0.0.
  • Issue a deprecation warning for the transitional argument.
  • Added lazy-loading to provide some performance improvements.
  • Removed vestiges of code related to Python 2 support, including segmentation of data structures specific to Jython.

Thanks to Rodrigo Nogueira for contributions to this release.

3.11 (2025-10-12)

  • Update to Unicode 16.0.0, including significant changes to UTS46 processing. As a result of Unicode ending support for it, transitional processing no longer has an effect and returns the same result.

... (truncated)

Commits
  • af30a09 Release 3.15
  • 30314d4 Pre-release 3.15rc0
  • 05d4b21 Merge pull request #237 from kjd/convert-docs-to-markdown
  • 2987fdb Convert README and HISTORY from reStructuredText to Markdown
  • 59fa800 Merge pull request #236 from kjd/dependabot/github_actions/actions-f3e34333ea
  • def6983 Merge branch 'master' into dependabot/github_actions/actions-f3e34333ea
  • bbd8004 Merge pull request #234 from StanFromIreland/patch-1
  • edd07c0 Bump github/codeql-action from 3.35.2 to 4.35.2 in the actions group
  • 5557db0 Merge branch 'master' into patch-1
  • f11746c Merge pull request #235 from StanFromIreland/patch-2
  • Additional commits viewable in compare view

Updates `urllib3` from 2.6.3 to 2.7.0
Release notes

Sourced from urllib3's releases.

2.7.0

🚀 urllib3 is fundraising for HTTP/2 support

urllib3 is raising ~$40,000 USD to release HTTP/2 support and ensure long-term sustainable maintenance of the project after a sharp decline in financial support. If your company or organization uses Python and would benefit from HTTP/2 support in Requests, pip, cloud SDKs, and thousands of other projects please consider contributing financially to ensure HTTP/2 support is developed sustainably and maintained for the long-haul.

Thank you for your support.

Security

Addressed high-severity security issues. Impact was limited to specific use cases detailed in the accompanying advisories; overall user exposure was estimated to be marginal.

  • Decompression-bomb safeguards of the streaming API were bypassed:

    1. When HTTPResponse.drain_conn() was called after the response had been read and decompressed partially. (Reported by @​Cycloctane)
    2. During the second HTTPResponse.read(amt=N) or HTTPResponse.stream(amt=N) call when the response was decompressed using the official Brotli library. (Reported by @​kimkou2024)

    See GHSA-mf9v-mfxr-j63j for details.

  • HTTP pools created using ProxyManager.connection_from_url did not strip sensitive headers specified in Retry.remove_headers_on_redirect when redirecting to a different host. (GHSA-qccp-gfcp-xxvc reported by @​christos-spearbit)

Deprecations and Removals

  • Used FutureWarning instead of DeprecationWarning for better visibility of existing deprecation notices. Rescheduled the removal of deprecated features to version 3.0. (urllib3/urllib3#3763)
  • Removed support for end-of-life Python 3.9. (urllib3/urllib3#3720)
  • Removed support for end-of-life PyPy3.10. (urllib3/urllib3#4979)
  • Bumped the minimum supported pyOpenSSL version to 19.0.0. (urllib3/urllib3#3777)

Bugfixes

  • Fixed a bug where HTTPResponse.read(amt=None) was ignoring decompressed data buffered from previous partial reads. (urllib3/urllib3#3636)
  • Fixed a bug where HTTPResponse.read() could cache only part of the response after a partial read when cache_content=True. (urllib3/urllib3#4967)
  • Fixed HTTPResponse.stream() and HTTPResponse.read_chunked() to handle amt=0. (urllib3/urllib3#3793)
  • Updated _TYPE_BODY type alias to include missing Iterable[str], matching the documented and runtime behavior of chunked request bodies. (urllib3/urllib3#3798)
  • Fixed LocationParseError when paths resembling schemeless URIs were passed to HTTPConnectionPool.urlopen(). (urllib3/urllib3#3352)
  • Fixed BaseHTTPResponse.readinto() type annotation to accept memoryview in addition to bytearray, matching the io.RawIOBase.readinto contract and enabling use with io.BufferedReader without type errors. (urllib3/urllib3#3764)
Changelog

Sourced from urllib3's changelog.

2.7.0 (2026-05-07)

Security

Addressed high-severity security issues. Impact was limited to specific use cases detailed in the accompanying advisories; overall user exposure was estimated to be marginal.

  • Decompression-bomb safeguards of the streaming API were bypassed:

    1. When HTTPResponse.drain_conn() was called after the response had been read and decompressed partially.
    2. During the second HTTPResponse.read(amt=N) or HTTPResponse.stream(amt=N) call when the response was decompressed using the official Brotli <https://pypi.org/project/brotli/>__ library.

    See GHSA-mf9v-mfxr-j63j <https://github.com/urllib3/urllib3/security/advisories/GHSA-mf9v-mfxr-j63j>__ for details.

  • HTTP pools created using ProxyManager.connection_from_url did not strip sensitive headers specified in Retry.remove_headers_on_redirect when redirecting to a different host. (GHSA-qccp-gfcp-xxvc <https://github.com/urllib3/urllib3/security/advisories/GHSA-qccp-gfcp-xxvc>__)

Deprecations and Removals

  • Used FutureWarning instead of DeprecationWarning for better visibility of existing deprecation notices. Rescheduled the removal of deprecated features to version 3.0. ([#3763](https://github.com/urllib3/urllib3/issues/3763) <https://github.com/urllib3/urllib3/issues/3763>__)
  • Removed support for end-of-life Python 3.9. ([#3720](https://github.com/urllib3/urllib3/issues/3720) <https://github.com/urllib3/urllib3/issues/3720>__)
  • Removed support for end-of-life PyPy3.10. ([#4979](https://github.com/urllib3/urllib3/issues/4979) <https://github.com/urllib3/urllib3/issues/4979>__)
  • Bumped the minimum supported pyOpenSSL version to 19.0.0. ([#3777](https://github.com/urllib3/urllib3/issues/3777) <https://github.com/urllib3/urllib3/issues/3777>__)

Bugfixes

  • Fixed a bug where HTTPResponse.read(amt=None) was ignoring decompressed data buffered from previous partial reads. ([#3636](https://github.com/urllib3/urllib3/issues/3636) <https://github.com/urllib3/urllib3/issues/3636>__)
  • Fixed a bug where HTTPResponse.read() could cache only part of the response after a partial read when cache_content=True.

... (truncated)

Commits

--------- Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Richard Levasseur From 66d00b3ae7381a17260f7c6bc8ca1cd58afc7b0e Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Tue, 28 Jul 2026 21:40:07 -0700 Subject: [PATCH 873/922] build(deps): bump astral-sh/setup-uv from 8.3.2 to 9.0.0 (#3965) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Bumps [astral-sh/setup-uv](https://github.com/astral-sh/setup-uv) from 8.3.2 to 9.0.0.
Release notes

Sourced from astral-sh/setup-uv's releases.

v9.0.0 🌈 Change prune-cache default to false

Changes

This release disables the default cache cache pruning to ease the load on the PyPi infrastructure. Since users might experience more GitHub Actions cache usage which might result in higher costs this is marked as a breaking change. To read more on why we did this (now) you can read the detailed analysis and reasoning in #967

Besides this big breaking change we also have a small bugfix while building caches for linux distributions that behave a big different than the "big ones" and a speed up in version resolution by only reading the version manifest until a matching version is found saving runtime and network bandwith.

🚨 Breaking changes

🐛 Bug fixes

  • fix: fall back to distribution ID when os-release has no version field @​cxzhong (#961)

🚀 Enhancements

🧰 Maintenance

📚 Documentation

⬆️ Dependency updates

Commits
  • c771a70 chore(deps): roll up Dependabot updates (#970)
  • 2f537ca chore: update known checksums for 0.11.30 (#968)
  • 2269552 Speed up version client by partial response reads (#807)
  • 47a7f4f Change prune-cache default to false (#967)
  • 71966ef chore(deps): roll up Dependabot updates (#962)
  • f12b1f0 fix: fall back to distribution ID when os-release has no version field (#961)
  • ecd24dd chore: update known checksums for 0.11.29 (#960)
  • 6a19136 docs: update version references to v8.3.2 (#949)
  • See full diff in compare view

Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> --- .github/workflows/automated_pr_review.yaml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.github/workflows/automated_pr_review.yaml b/.github/workflows/automated_pr_review.yaml index dbfffa9083..01b63e42a4 100644 --- a/.github/workflows/automated_pr_review.yaml +++ b/.github/workflows/automated_pr_review.yaml @@ -86,7 +86,7 @@ jobs: path: reviewbot - name: Install uv - uses: astral-sh/setup-uv@v8.3.2 + uses: astral-sh/setup-uv@v9.0.0 - name: Run Antigravity Review # Run inside reviewbot directory so execution context is the trusted base branch. From 41665f5662719c25c730ab3349e89837e8a99a91 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Tue, 28 Jul 2026 22:36:51 -0700 Subject: [PATCH 874/922] feat(ci): support GitHub Actions log retrieval and subagent failure analysis (#3976) Previously, CI failure monitoring and log analysis were tied strictly to Buildkite jobs, preventing automated analysis of failed GitHub Actions checks. Furthermore, failure notifications were hardcoded without triggering subagent delegation. To address this, update `monitor_remote_ci.py` to report failed GitHub Actions checks back to the active conversation for subagent delegation. Additionally, enhance `analyze_ci_failure.py` to fetch job logs via the `gh` CLI and parse GitHub Actions error signatures. * Also updates documentation in `monitor-ci-results` and `analyze-ci-failure` skills to reflect the subagent workflow. --- .agents/skills/analyze-ci-failure/SKILL.md | 25 +++++++---- .../scripts/analyze_ci_failure.py | 23 +++++++++++ .agents/skills/monitor-ci-results/SKILL.md | 41 ++++++++----------- .../scripts/monitor_remote_ci.py | 11 +++-- 4 files changed, 66 insertions(+), 34 deletions(-) diff --git a/.agents/skills/analyze-ci-failure/SKILL.md b/.agents/skills/analyze-ci-failure/SKILL.md index 2019c5b5f9..6bda9d5d6b 100644 --- a/.agents/skills/analyze-ci-failure/SKILL.md +++ b/.agents/skills/analyze-ci-failure/SKILL.md @@ -1,16 +1,25 @@ --- name: analyze-ci-failure -description: Download and analyze a CI failure log to construct an actionable suggested fix plan and report back +description: Download and analyze a CI failure log to construct an actionable + suggested fix plan and report back --- -When a CI monitoring workflow alerts you to a failed Buildkite job or GitHub check, invoke this skill by running: +When a CI monitoring workflow alerts you to a failed Buildkite job or GitHub +check, invoke this skill by running: ```bash -./.agents/skills/analyze-ci-failure/scripts/analyze_ci_failure.py "" "" "" "" +./.agents/skills/analyze-ci-failure/scripts/analyze_ci_failure.py \ + "" "" "" "" ``` ### ✨ What this Skill Does -1. **Resolves Log**: Automatically resolves the Buildkite job download URL or locates existing local log artifacts. -2. **Downloads & Ingests**: Fetches the full raw CI log file and saves it locally. -3. **Smart Error Extraction**: Scans the log lines for critical failure signatures (`Traceback`, `ERROR:`, `FAILED:`, missing packages, compiler aborts). -4. **Fix Plan Synthesis**: Constructs a beautifully structured Markdown suggested plan on how to resolve the root cause. -5. **Natively Notifies**: Dispatches a high-priority summary notification message back to your active agent conversation via `agentapi send-message`! +1. **Resolves Log**: Automatically resolves the Buildkite job download URL or + fetches GitHub Actions logs via `gh` CLI. +2. **Downloads & Ingests**: Fetches the full raw CI log file and saves it + locally. +3. **Smart Error Extraction**: Scans log lines for critical failure signatures + (`Traceback`, `ERROR:`, `FAILED:`, missing packages, compiler aborts, linter + errors). +4. **Fix Plan Synthesis**: Constructs a beautifully structured Markdown + suggested plan on how to resolve the root cause. +5. **Natively Notifies**: Dispatches a high-priority summary notification + message back to your active agent conversation via `agentapi send-message`. diff --git a/.agents/skills/analyze-ci-failure/scripts/analyze_ci_failure.py b/.agents/skills/analyze-ci-failure/scripts/analyze_ci_failure.py index 3b975acfb3..20212f6d23 100644 --- a/.agents/skills/analyze-ci-failure/scripts/analyze_ci_failure.py +++ b/.agents/skills/analyze-ci-failure/scripts/analyze_ci_failure.py @@ -16,6 +16,25 @@ def fetch_log(build_id, job_id, output_path): else: log_url = f"https://buildkite.com/organizations/bazel/pipelines/rules-python-python/builds/{build_id}/jobs/{job_id}/download.txt" + # Check if this is a GitHub Actions job + gh_match = re.search(r"github\.com/.*/job/(\d+)", log_url) or re.search( + r"^(\d+)$", job_id + ) + if gh_match: + gh_job_id = gh_match.group(1) + print(f"📥 Fetching GitHub Action log for job {gh_job_id} using gh CLI...") + cmd = ["gh", "run", "view", "--job", gh_job_id, "--log"] + try: + res = subprocess.run(cmd, capture_output=True, text=True, check=True) + with open(output_path, "w") as f: + f.write(res.stdout) + return True + except Exception as e: + print( + f"⚠️ Failed to fetch GitHub log via gh CLI for job {gh_job_id}: {e}", + file=sys.stderr, + ) + if not log_url.endswith("/download.txt") and "buildkite.com" in log_url: log_url = re.sub(r"/log$", "/download.txt", log_url) @@ -55,6 +74,10 @@ def parse_log(log_path): "no such package", "no such target", "exit code", + "##[error]", + "Would reformat:", + "would be reformatted", + "error]", ] ): errors.append(line.strip()) diff --git a/.agents/skills/monitor-ci-results/SKILL.md b/.agents/skills/monitor-ci-results/SKILL.md index 2b4bc5a460..1cef8293f4 100644 --- a/.agents/skills/monitor-ci-results/SKILL.md +++ b/.agents/skills/monitor-ci-results/SKILL.md @@ -1,31 +1,26 @@ --- name: monitor-ci-results -description: Monitor CI for PRs and notify of status +description: Monitor remote CI results for a PR and autonomously launch + subagents to analyze CI failures --- When the user requests to monitor remote CI results or watch a pull request, -invoke `scripts/monitor_remote_ci.py `. - -This long-running monitoring service runs in the background and continuously -polls both GitHub PR checks and Buildkite workflow executions. +launch the monitoring script in the background: +```bash +./.agents/skills/monitor-ci-results/scripts/monitor_remote_ci.py \ + "" & +``` -### ✨ Autonomous Failure and Blocked Job Orchestration -1. **Blocked Jobs**: When a Buildkite job or GitHub check is in a blocked +### ✨ Autonomous Subagent Orchestration +1. **Background Polling**: `monitor_remote_ci.py` continuously polls both + GitHub PR checks and Buildkite workflow executions in the background. +2. **Blocked Jobs**: When a Buildkite job or GitHub check is in a blocked state waiting for user confirmation, it dispatches a notification via `agentapi send-message` so the user is alerted to confirm running the job. -2. **Failure Analysis**: When any CI job completes with errors or returns a - non-zero exit code: - - It automatically downloads the raw CI log file to `ci_logs/`. - - It launches an independent background analyzer script - (`analyze_ci_failure.py`). - - It authors a structured Markdown plan to fix the failure. - - It natively dispatches a high-priority notification back to your active - agent conversation using `agentapi send-message`! - -### Example Invocation -```bash -./scripts/monitor_remote_ci.py 3812 "0be435bd-96aa-4e1b-9c6f-727b31e80fa0" & -``` -*Note: Always include the trailing `&` when launching the monitoring script via -tool calls to ensure it runs as a detached background task without blocking -foreground execution.* +3. **Failure Reporting**: When any GitHub check or Buildkite job completes + with errors, `monitor_remote_ci.py` dispatches a high-priority notification + message reporting the failed check back to your conversation. +4. **Subagent Analysis**: Upon receiving a failure notification message from + the monitoring script, immediately launch a separate subagent using the + `invoke_subagent` tool with the role "CI Failure Analyzer" to run the + `analyze-ci-failure` skill on the reported failure. diff --git a/.agents/skills/monitor-ci-results/scripts/monitor_remote_ci.py b/.agents/skills/monitor-ci-results/scripts/monitor_remote_ci.py index 40be4d5d26..f668ed96f5 100755 --- a/.agents/skills/monitor-ci-results/scripts/monitor_remote_ci.py +++ b/.agents/skills/monitor-ci-results/scripts/monitor_remote_ci.py @@ -104,6 +104,7 @@ def main(): name = check.get("name", "unknown") state = check.get("state", "UNKNOWN") link = check.get("link", "") + bucket = check.get("bucket", "") if "buildkite" in name.lower() and link: jobs = get_buildkite_jobs(link) @@ -202,7 +203,7 @@ def main(): f"⚠️ Remote CI Buildkite Job '{jname}' completed with errors!\n\n" f"Build ID: {build_id} | Job ID: {jid}\n" f"Log URL: {job.get('log_url', link)}\n\n" - f"Start subagent: run analyze-ci-failure skill on this failure" + f"Action required: Start a subagent to run the analyze-ci-failure skill for '{jname}' with build ID '{build_id}' and job ID '{jid}'." ) subprocess.run( [ @@ -240,12 +241,16 @@ def main(): with open(state_file, "w") as f: json.dump(monitored, f) - elif state in ["FAILURE", "failed"] and name not in monitored: + elif ( + state.upper() in ["FAILURE", "FAIL", "CANCELLED", "TIMED_OUT"] + or bucket == "fail" + ) and name not in monitored: print(f"🚨 Notifying failure for GitHub check '{name}'...") + job_id = link.split("/")[-1] if "/" in link else link msg = ( f"⚠️ Remote CI GitHub Check '{name}' completed with errors!\n\n" f"Link: {link}\n\n" - f"Start subagent: run analyze-ci-failure skill on this failure" + f"Action required: Start a subagent to run the analyze-ci-failure skill for '{name}' with link '{link}' and job ID '{job_id}'." ) subprocess.run( [ From fd25d1f3107dfa9032cf39ee8262b1cde0338fc7 Mon Sep 17 00:00:00 2001 From: Ignas Anikevicius <240938+aignas@users.noreply.github.com> Date: Sat, 1 Aug 2026 11:59:24 +0900 Subject: [PATCH 875/922] refactor(pypi): add whl_deps_library repo rule (#3960) Summary: - Add a new repo rule to just read metadata.json - Add integration tests for the repository rules in `whl_library.bzl` file. - Make some of the arguments optional in the BUILD.bazel code generation. No changelog, because the rule is not yet exposed to the user in any way. Split out of #3856 Work towards #2948 Fixes #3071 --------- Co-authored-by: Richard Levasseur --- .bazelrc.deleted_packages | 1 + .../pypi/generate_whl_library_build_bazel.bzl | 32 ++- python/private/pypi/pep508_deps.bzl | 6 + python/private/pypi/whl_library.bzl | 90 +++++- python/private/pypi/whl_library_targets.bzl | 257 ++++++++++-------- .../py_library/py_library_tests.bzl | 59 ++++ tests/integration/BUILD.bazel | 4 + .../integration/bzlmod_lockfile/MODULE.bazel | 4 +- .../MODULE.bazel | 2 +- .../pip_parse_isolated/MODULE.bazel | 4 +- tests/integration/whl_library/.bazelrc | 2 + tests/integration/whl_library/BUILD.bazel | 65 +++++ tests/integration/whl_library/MODULE.bazel | 57 ++++ tests/integration/whl_library/WORKSPACE | 0 .../integration/whl_library/test_contents.py | 179 ++++++++++++ ...generate_whl_library_build_bazel_tests.bzl | 72 +++-- .../whl_library_targets_tests.bzl | 123 +++++---- 17 files changed, 736 insertions(+), 221 deletions(-) create mode 100644 tests/integration/whl_library/.bazelrc create mode 100644 tests/integration/whl_library/BUILD.bazel create mode 100644 tests/integration/whl_library/MODULE.bazel create mode 100644 tests/integration/whl_library/WORKSPACE create mode 100644 tests/integration/whl_library/test_contents.py diff --git a/.bazelrc.deleted_packages b/.bazelrc.deleted_packages index db42040b5c..9c4745c178 100644 --- a/.bazelrc.deleted_packages +++ b/.bazelrc.deleted_packages @@ -44,6 +44,7 @@ common --deleted_packages=tests/integration/toolchain_target_settings common --deleted_packages=tests/integration/unified_pypi common --deleted_packages=tests/integration/uv_lock common --deleted_packages=tests/integration/validate_test_main +common --deleted_packages=tests/integration/whl_library common --deleted_packages=tests/modules/another_module common --deleted_packages=tests/modules/other common --deleted_packages=tests/modules/other/nspkg_delta diff --git a/python/private/pypi/generate_whl_library_build_bazel.bzl b/python/private/pypi/generate_whl_library_build_bazel.bzl index 71eab26c0e..ceee319cd7 100644 --- a/python/private/pypi/generate_whl_library_build_bazel.bzl +++ b/python/private/pypi/generate_whl_library_build_bazel.bzl @@ -25,6 +25,7 @@ _RENDER = { "extras": render.list, "group_deps": render.list, "include": str, + "repo": lambda maybe_label: repr(str(maybe_label)), "requires_dist": render.list, "srcs_exclude": render.list, "tags": render.list, @@ -38,19 +39,22 @@ _TEMPLATE = """\ package(default_visibility = ["//visibility:public"]) +{fn}( +{kwargs} +) +""" + +_PURL = """\ package_metadata( name = "package_metadata", purl = {purl}, visibility = ["//:__subpackages__"], ) - -{fn}( -{kwargs} -) """ def generate_whl_library_build_bazel( *, + metadata_version, annotation = None, config_load, purl = None, @@ -59,6 +63,7 @@ def generate_whl_library_build_bazel( """Generate a BUILD file for an unzipped Wheel Args: + metadata_version: The version to use for tag generation. annotation: The annotation for the build file. config_load: {type}`str` The location from where to load the config. purl: The purl. @@ -74,14 +79,26 @@ def generate_whl_library_build_bazel( """load("@package_metadata//rules:package_metadata.bzl", "package_metadata")""", ] - fn = "whl_library_targets_from_requires" + if kwargs.get("repo"): + fn = "whl_library_deps_targets" + else: + fn = "whl_library_targets" + + tags = [ + "pypi_name={}".format(kwargs.get("metadata_name")), + "pypi_version={}".format(metadata_version), + ] + kwargs["tags"] = tags + if not requires_dist: # no deps, we can leave the extra loads out pass - else: + elif config_load: loads.append("""load("{}", "{}")""".format(config_load, "packages")) kwargs["include"] = "packages" kwargs["requires_dist"] = requires_dist + else: + kwargs["requires_dist"] = requires_dist loads.extend([ """load("@rules_python//python/private/pypi:whl_library_targets.bzl", "{}")""".format(fn), @@ -106,9 +123,8 @@ def generate_whl_library_build_bazel( "{} = {},".format(k, _RENDER.get(k, repr)(v)) for k, v in sorted(kwargs.items()) ])), - purl = repr(purl), ), - ] + additional_content, + ] + ([_PURL.format(purl = repr(purl))] if purl else []) + additional_content, ) # NOTE: Ensure that we terminate with a new line diff --git a/python/private/pypi/pep508_deps.bzl b/python/private/pypi/pep508_deps.bzl index c004334d96..fd6d961bf5 100644 --- a/python/private/pypi/pep508_deps.bzl +++ b/python/private/pypi/pep508_deps.bzl @@ -44,6 +44,12 @@ def deps( * deps_select: {type}`dict[str, list[str]]` dependencies to include on particular subset of target platforms. """ + if not requires_dist: + return struct( + deps = [], + deps_select = {}, + ) + reqs = sorted( [requirement(r) for r in requires_dist], key = lambda x: "{}:{}:".format(x.name, sorted(x.extras), x.marker), diff --git a/python/private/pypi/whl_library.bzl b/python/private/pypi/whl_library.bzl index 9a150db9e8..53edd62efb 100644 --- a/python/private/pypi/whl_library.bzl +++ b/python/private/pypi/whl_library.bzl @@ -332,6 +332,12 @@ def _whl_extract(rctx, *, whl_path, logger, sdist_filename = None): read_fn = rctx.read, logger = logger, ) + rctx.file("metadata.json", json.encode_indent({ + "name": metadata.name, + "provides_extra": metadata.provides_extra, + "requires_dist": metadata.requires_dist, + "version": metadata.version, + })) namespace_package_files = pypi_repo_utils.find_namespace_package_files(rctx, install_dir_path) entry_points = _get_entry_points(rctx, install_dir_path, metadata) @@ -429,6 +435,8 @@ def _whl_archive_impl(rctx): whl_path = rctx.path(filename) else: fail("Only wheels are supported") + else: + fail("Either 'whl_file' or 'urls' and 'filename' needs to be specified") return _whl_extract(rctx, whl_path = whl_path, logger = logger) @@ -571,9 +579,6 @@ For example if your whl depends on `numpy` and your Python package repo is named "index_url": attr.string( doc = "The index_url that the package will be downloaded from.", ), - "repo": attr.string( - doc = "Pointer to parent repo name. Used to make these rules rerun if the parent repo changes.", - ), "repo_prefix": attr.string( doc = """ Prefix for the generated packages will be of the form `@//...` @@ -676,7 +681,6 @@ whl_archive = repository_rule( "group_deps", "group_name", "index_url", - "repo", "repo_prefix", "requirement", "sha256", @@ -702,7 +706,74 @@ Does not depend on any python. ], ) -def whl_library(name, **kwargs): +def _whl_deps_library_impl(rctx): + logger = repo_utils.logger(rctx) + + if rctx.attr.metadata_file and rctx.attr.metadata: + logger.fail("Only one of 'metadata_file' and 'metadata' can be specified") + return + if not (rctx.attr.metadata_file or rctx.attr.metadata): + logger.fail("At least one of 'metadata_file' and 'metadata' must be specified") + return + + if rctx.attr.metadata_file: + metadata_contents = rctx.read(rctx.attr.metadata_file) + else: + metadata_contents = rctx.attr.metadata + + metadata = struct(**json.decode(metadata_contents)) + + build_file_contents = generate_whl_library_build_bazel( + dep_template = rctx.attr.dep_template or "@{}{{name}}//:{{target}}".format( + rctx.attr.repo_prefix, + ), + config_load = rctx.attr.config_load, + metadata_name = metadata.name, + metadata_version = metadata.version, + requires_dist = metadata.requires_dist, + group_deps = rctx.attr.group_deps, + group_name = rctx.attr.group_name, + repo = rctx.attr.repo or ( + str(rctx.attr.metadata_file) if rctx.attr.metadata_file else None + ), + extras = requirement(rctx.attr.requirement).extras, + ) + rctx.file("BUILD.bazel", build_file_contents) + +whl_deps_library = repository_rule( + attrs = { + k: _pip_archive_attrs[k] + for k in [ + "config_load", + "dep_template", + "group_deps", + "group_name", + "requirement", + ] + } | { + "metadata": attr.string( + doc = """ +The subset of the METADATA contents that is needed for generation of the dependencies. +* name: {type}`str` +* version: {type}`str` +* provides_extra: {type}`list[str]` +* requires_dist: {type}`list[str]` +""", + ), + "metadata_file": attr.label(doc = "An alternative way to pass {attr}`metadata` but as a file."), + "repo": attr.label(doc = "A label at the root of the repo to get stuff from."), + }, + doc = """ +A repo rule that reuses the sources from a different place and then creates the necessary targets +so that this can be used in the repo. + +Does not depend on any python. +""", + implementation = _whl_deps_library_impl, + environ = [REPO_DEBUG_ENV_VAR], +) + +def whl_library(name, repo = None, **kwargs): """Create a whl_library. This proxies to one of the underlying implementations: @@ -711,15 +782,18 @@ def whl_library(name, **kwargs): Args: name: {type}`str` The name of the repo. + repo: Unused, will be dropped in the next major release. **kwargs: The args passed to the underlying implementation. Returns: the repo metadata. """ + _ = repo # buildifier: disable=unused-variable + whl_file = kwargs.get("whl_file") urls = kwargs.get("urls", []) filename = kwargs.get("filename") if whl_file or (urls and filename and filename.endswith(".whl")): - return whl_archive(name = name, **kwargs) - - return pip_archive(name = name, **kwargs) + whl_archive(name = name, **kwargs) + else: + pip_archive(name = name, **kwargs) diff --git a/python/private/pypi/whl_library_targets.bzl b/python/private/pypi/whl_library_targets.bzl index 217210e782..737a739145 100644 --- a/python/private/pypi/whl_library_targets.bzl +++ b/python/private/pypi/whl_library_targets.bzl @@ -49,11 +49,10 @@ _BAZEL_REPO_FILE_GLOBS = [ _IS_VENV_SITE_PACKAGES_YES = Label("//python/config_settings:_is_venvs_site_packages_yes") _VENV_SITE_PACKAGES_FLAG = Label("//python/config_settings:venvs_site_packages") -def whl_library_targets_from_requires( +def whl_library_targets( *, name, metadata_name = "", - metadata_version = "", requires_dist = [], extras = [], entry_points = {}, @@ -71,14 +70,12 @@ def whl_library_targets_from_requires( srcs_exclude = [], data = [], visibility = ["//visibility:public"], - tags = [], **kwargs): """The macro to create whl targets from the METADATA. Args: name: {type}`str` The wheel filename metadata_name: {type}`str` The package name as written in wheel `METADATA`. - metadata_version: {type}`str` The package version as written in wheel `METADATA`. group_deps: {type}`list[str]` names of fellow members of the group (if any). These will be excluded from generated deps lists so as to avoid direct cycles. These dependencies will be provided at runtime by the @@ -100,21 +97,14 @@ def whl_library_targets_from_requires( srcs_exclude: {type}`list[str]` The globs for srcs attribute exclusion. data: {type}`list[str]` A list of labels to include as part of the `data` attribute. visibility: {type}`list[str]` The visibility of the targets. - tags: {type}`list[str]` The tags set on the targets. - **kwargs: Extra args passed to the {obj}`whl_library_targets` and {obj}`whl_library_srcs`. + **kwargs: Extra args passed to the {obj}`whl_library_deps_targets` and {obj}`whl_library_srcs`. """ - pypi_tags = [ - "pypi_name={}".format(metadata_name), - "pypi_version={}".format(metadata_version), - ] - all_tags = sorted(tags + pypi_tags) - + create_extra_targets = bool(requires_dist or group_name) and dep_template whl_library_srcs( name = name, sdist_filename = sdist_filename, data_exclude = data_exclude, srcs_exclude = srcs_exclude, - tags = all_tags, filegroups = filegroups, entry_points = entry_points, visibility = visibility, @@ -123,26 +113,27 @@ def whl_library_targets_from_requires( copy_executables = copy_executables, enable_implicit_namespace_pkgs = enable_implicit_namespace_pkgs, namespace_package_files = namespace_package_files, + # If there are no dependencies, then let's create the targets with public labels. + # Note, we are not supporting grouping the packages in this case, but that is fine. + whl_name = WHEEL_FILE if create_extra_targets else WHEEL_FILE_PUBLIC_LABEL, + pkg_name = PY_SRCS_LABEL if create_extra_targets else PY_LIBRARY_PUBLIC_LABEL, **kwargs ) - package_deps = _parse_requires_dist( - name = metadata_name, - requires_dist = requires_dist, - excludes = group_deps, - extras = extras, - include = include, - ) - - whl_library_targets( - name = name, - dependencies = package_deps.deps, - dependencies_with_markers = package_deps.deps_select, - group_name = group_name, - dep_template = dep_template, - tags = all_tags, - **kwargs - ) + if create_extra_targets: + whl_library_deps_targets( + name = name, + metadata_name = metadata_name, + requires_dist = requires_dist, + group_deps = group_deps, # only needed if requires_dist is present + extras = extras, # only needed if requires_dist is present + include = include, # only needed if requires_dist is present + group_name = group_name, # only needed if requires_dist is present + dep_template = dep_template, # only needed if requires_dist is present + repo = None, # set aliases in the same repo + aliases = {}, + **kwargs + ) def whl_library_srcs( *, @@ -153,13 +144,15 @@ def whl_library_srcs( tags = [], filegroups = None, entry_points = {}, - visibility = ["//visibility:public"], data = [], copy_files = {}, copy_executables = {}, native = native, enable_implicit_namespace_pkgs = False, namespace_package_files = [], + whl_name = WHEEL_FILE, + pkg_name = PY_SRCS_LABEL, + visibility = ["//visibility:public"], rules = struct( copy_file = copy_file, py_binary = py_binary, @@ -192,9 +185,11 @@ def whl_library_srcs( data: {type}`list[str]` A list of labels to include as part of the `data` attribute in `py_library`. enable_implicit_namespace_pkgs: {type}`boolean` generate __init__.py files for namespace pkgs. - native: {type}`native` The native struct for overriding in tests. namespace_package_files: {type}`list[str]` A list of labels of files whose directories are namespace packages. + whl_name: {type}`str` The label name to use for the wheel filegroup target. + pkg_name: {type}`str` The label name to use for the py_library target. + native: {type}`native` The native struct for overriding in tests. rules: {type}`struct` A struct with references to rules for creating targets. """ tags = sorted(tags) @@ -278,7 +273,7 @@ def whl_library_srcs( if hasattr(native, "filegroup"): native.filegroup( - name = WHEEL_FILE, + name = whl_name, srcs = [name], visibility = visibility, ) @@ -334,7 +329,7 @@ def whl_library_srcs( data = data + [DATA_LABEL] rules.py_library( - name = PY_SRCS_LABEL, + name = pkg_name, srcs = srcs, pyi_srcs = pyi_srcs, data = data, @@ -347,29 +342,20 @@ def whl_library_srcs( namespace_package_files = namespace_package_files, ) -def _parse_requires_dist( +def whl_library_deps_targets( *, - name, + name = None, + repo, + aliases = None, + metadata_name, requires_dist, - excludes, - include, - extras): - return deps( - name = normalize_name(name), - requires_dist = requires_dist, - excludes = excludes, - include = include, - extras = extras, - ) - -def whl_library_targets( - *, - name, + extras, + include = [], + group_deps = [], + group_name = None, dep_template, tags = [], - dependencies = [], - dependencies_with_markers = {}, - group_name = "", + visibility = ["//visibility:public"], native = native, rules = struct( copy_file = copy_file, @@ -379,39 +365,38 @@ def whl_library_targets( venv_rewrite_shebang = venv_rewrite_shebang, env_marker_setting = env_marker_setting, create_inits = _create_inits, - ), - **_kwargs): + )): """Create all of the whl_library targets. Args: - name: {type}`str` The file to match for including it into the `whl` - filegroup. This may be also parsed to generate extra metadata. - dep_template: {type}`str` The dep_template to use for dependency - interpolation. - tags: {type}`list[str]` The tags set on the `py_library`. - dependencies: {type}`list[str]` A list of dependencies. - dependencies_with_markers: {type}`dict[str, str]` A marker to evaluate - in order for the dep to be included. - group_name: {type}`str` name of the dependency group (if any) which - contains this library. If set, this library will behave as a shim - to group implementation rules which will provide simultaneously - installed dependencies which would otherwise form a cycle. + name: {type}`str` The wheel filename + metadata_name: {type}`str` The package name as written in wheel `METADATA`. + group_deps: {type}`list[str]` names of fellow members of the group (if + any). These will be excluded from generated deps lists so as to avoid + direct cycles. These dependencies will be provided at runtime by the + group rules which wrap this library and its fellows together. + requires_dist: {type}`list[str]` The list of `Requires-Dist` values from + the whl `METADATA`. + extras: {type}`list[str]` The list of requested extras. This essentially includes extra transitive dependencies in the final targets depending on the wheel `METADATA`. + include: {type}`list[str]` The list of packages to include. + group_name: {type}`str | None` name of the dependency group (if any). + dep_template: {type}`str | None` The dep_template to use. + tags: {type}`list[str]` The tags set on the targets. + repo: {type}`str | Label | None` The BUILD.bazel label to the parent repo that has the + sources. If none, then will take the targets from the current dir. + aliases: {type}`dict[str, str] | None` The list of aliases to create in the parent repo. If None, will create + the default values. Empty list means no aliases. + visibility: {type}`list[str]` The visibility of the targets. native: {type}`native` The native struct for overriding in tests. rules: {type}`struct` A struct with references to rules for creating targets. - **_kwargs: ignored args that are not needed. """ - dependencies = sorted([normalize_name(d) for d in dependencies]) - tags = sorted(tags) - - _config_settings( - dependencies_with_markers = dependencies_with_markers, - rules = rules, - visibility = ["//visibility:private"], - ) - deps_conditional = { - d: "is_include_{}_true".format(d) - for d in dependencies_with_markers - } + repo_label = Label(repo).same_package_label if repo else (lambda x: x) + if aliases == None: + aliases = { + EXTRACTED_WHEEL_FILES: repo_label(EXTRACTED_WHEEL_FILES), + DIST_INFO_LABEL: repo_label(DIST_INFO_LABEL), + DATA_LABEL: repo_label(DATA_LABEL), + } # If this library is a member of a group, its public label aliases need to # point to the group implementation rule not the implementation rules. We @@ -430,6 +415,10 @@ def whl_library_targets( "//:", "//_groups:", ) + aliases = aliases | { + PY_LIBRARY_PUBLIC_LABEL: label_tmpl.format(PY_LIBRARY_PUBLIC_LABEL), + WHEEL_FILE_PUBLIC_LABEL: label_tmpl.format(WHEEL_FILE_PUBLIC_LABEL), + } impl_vis = [dep_template.format( name = "_config", target = "__pkg__", @@ -438,37 +427,59 @@ def whl_library_targets( "//_groups:", )] - native.alias( - name = PY_LIBRARY_PUBLIC_LABEL, - actual = label_tmpl.format(PY_LIBRARY_PUBLIC_LABEL), - visibility = ["//visibility:public"], - ) - native.alias( - name = WHEEL_FILE_PUBLIC_LABEL, - actual = label_tmpl.format(WHEEL_FILE_PUBLIC_LABEL), - visibility = ["//visibility:public"], - ) py_library_label = PY_LIBRARY_IMPL_LABEL whl_file_label = WHEEL_FILE_IMPL_LABEL - - elif group_name: - py_library_label = PY_LIBRARY_PUBLIC_LABEL - whl_file_label = WHEEL_FILE_PUBLIC_LABEL - impl_vis = [dep_template.format(name = "", target = "__subpackages__")] - else: py_library_label = PY_LIBRARY_PUBLIC_LABEL whl_file_label = WHEEL_FILE_PUBLIC_LABEL - impl_vis = ["//visibility:public"] + if group_name: + impl_vis = [dep_template.format(name = "", target = "__subpackages__")] + else: + impl_vis = visibility + + if not requires_dist: + # If the package is in a group but has no deps, we still need the public labels to + # point at the srcs targets so that the group implementation can use them. We don't + # need any of the extra targets, so just create the aliases. + aliases = aliases | { + py_library_label: repo_label(PY_SRCS_LABEL), + whl_file_label: repo_label(WHEEL_FILE), + } + + for alias, actual in aliases.items(): + native.alias( + name = alias, + actual = actual, + visibility = visibility, + ) + + if not requires_dist: + # If there are extras, then they will be visible in requires_dist. + return + + package_deps = _parse_requires_dist( + name = metadata_name, + requires_dist = requires_dist, + excludes = group_deps, + extras = extras, + include = include, + ) + + _config_settings( + dependencies_with_markers = package_deps.deps_select, + rules = rules, + visibility = ["//visibility:private"], + ) if hasattr(native, "filegroup"): + # We include the whl file as srcs so that `$(location :whl)` expands to the whl file. + # The transitive dependencies are available via the `data` attribute. native.filegroup( name = whl_file_label, - data = [ - WHEEL_FILE, - ] + _deps( - deps = dependencies, - deps_conditional = deps_conditional, + srcs = [repo_label(WHEEL_FILE)], + data = _deps( + deps = [], + package_deps = package_deps, tmpl = dep_template.format(name = "{}", target = WHEEL_FILE_PUBLIC_LABEL), ), visibility = impl_vis, @@ -477,23 +488,35 @@ def whl_library_targets( if hasattr(rules, "py_library"): rules.py_library( name = py_library_label, - srcs = [ - # We include as srcs to ensure that the (locations :pkg) works as expected. - PY_SRCS_LABEL, - ], - deps = [ - # We include as deps, so that `PyInfo` and friends get propagated as deps. - # not sure if just including it as `srcs` is enough. - PY_SRCS_LABEL, - ] + _deps( - deps = dependencies, - deps_conditional = deps_conditional, + # We include as srcs to ensure that the (locations :pkg) works as expected. + srcs = [repo_label(PY_SRCS_LABEL)], + deps = _deps( + # We include as deps, so that `PyInfo` and friends (e.g. `pyi_srcs`) get + # propagated. Just passing the target as `srcs` is not enough to propagate + # `pyi_srcs`, see `tests/base_rules/py_library`. + deps = [repo_label(PY_SRCS_LABEL)], + package_deps = package_deps, tmpl = dep_template.format(name = "{}", target = PY_LIBRARY_PUBLIC_LABEL), ), tags = tags, visibility = impl_vis, ) +def _parse_requires_dist( + *, + name, + requires_dist, + excludes, + include, + extras): + return deps( + name = normalize_name(name), + requires_dist = requires_dist, + excludes = excludes, + include = include, + extras = extras, + ) + def _config_settings(dependencies_with_markers, rules, **kwargs): """Generate config settings for the targets. @@ -510,12 +533,12 @@ def _config_settings(dependencies_with_markers, rules, **kwargs): **kwargs ) -def _deps(deps, deps_conditional, tmpl): - deps = [tmpl.format(d) for d in sorted(deps)] +def _deps(deps, package_deps, tmpl): + deps = [] + deps + [tmpl.format(d) for d in sorted(package_deps.deps)] - for dep, setting in deps_conditional.items(): + for dep in package_deps.deps_select: deps = deps + select({ - ":{}".format(setting): [tmpl.format(dep)], + ":is_include_{}_true".format(dep): [tmpl.format(dep)], "//conditions:default": [], }) diff --git a/tests/base_rules/py_library/py_library_tests.bzl b/tests/base_rules/py_library/py_library_tests.bzl index 3726ff1f41..c375e72794 100644 --- a/tests/base_rules/py_library/py_library_tests.bzl +++ b/tests/base_rules/py_library/py_library_tests.bzl @@ -3,10 +3,12 @@ load("@rules_testing//lib:analysis_test.bzl", "analysis_test") load("@rules_testing//lib:truth.bzl", "matching") load("@rules_testing//lib:util.bzl", rt_util = "util") +load("//python:py_info.bzl", "PyInfo") load("//python:py_library.bzl", "py_library") load("//python:py_runtime_info.bzl", "PyRuntimeInfo") load("//tests/base_rules:base_tests.bzl", "create_base_tests") load("//tests/base_rules:util.bzl", pt_util = "util") +load("//tests/support:py_info_subject.bzl", "py_info_subject") _tests = [] @@ -141,6 +143,63 @@ def _test_files_to_compile_impl(env, target): _tests.append(_test_files_to_compile) +def _test_pyi_srcs_not_propagated_via_srcs(name, config): + rt_util.helper_target( + config.rule, + name = name + "_lib", + srcs = ["lib.py"], + pyi_srcs = ["lib.pyi"], + ) + rt_util.helper_target( + config.rule, + name = name + "_subject", + srcs = [name + "_lib"], + ) + analysis_test( + name = name, + target = name + "_subject", + impl = _test_pyi_srcs_not_propagated_via_srcs_impl, + ) + +def _test_pyi_srcs_not_propagated_via_srcs_impl(env, target): + info = env.expect.that_target(target).provider( + PyInfo, + factory = py_info_subject, + ) + info.transitive_pyi_files().contains_exactly([]) + +_tests.append(_test_pyi_srcs_not_propagated_via_srcs) + +def _test_pyi_srcs_propagated_via_deps(name, config): + rt_util.helper_target( + config.rule, + name = name + "_lib", + srcs = ["lib.py"], + pyi_srcs = ["lib.pyi"], + ) + rt_util.helper_target( + config.rule, + name = name + "_subject", + srcs = [name + "_lib"], + deps = [name + "_lib"], + ) + analysis_test( + name = name, + target = name + "_subject", + impl = _test_pyi_srcs_propagated_via_deps_impl, + ) + +def _test_pyi_srcs_propagated_via_deps_impl(env, target): + info = env.expect.that_target(target).provider( + PyInfo, + factory = py_info_subject, + ) + info.transitive_pyi_files().contains_exactly([ + "{package}/lib.pyi", + ]) + +_tests.append(_test_pyi_srcs_propagated_via_deps) + def py_library_test_suite(name): config = struct(rule = py_library, base_test_rule = py_library) native.test_suite( diff --git a/tests/integration/BUILD.bazel b/tests/integration/BUILD.bazel index abdb37be57..ef7681ebe3 100644 --- a/tests/integration/BUILD.bazel +++ b/tests/integration/BUILD.bazel @@ -137,6 +137,10 @@ rules_python_integration_test( py_main = "uv_lock_test.py", ) +rules_python_integration_test( + name = "whl_library_test", +) + py_library( name = "runner_lib", srcs = ["runner.py"], diff --git a/tests/integration/bzlmod_lockfile/MODULE.bazel b/tests/integration/bzlmod_lockfile/MODULE.bazel index 6c074bcdd4..bfd43cb895 100644 --- a/tests/integration/bzlmod_lockfile/MODULE.bazel +++ b/tests/integration/bzlmod_lockfile/MODULE.bazel @@ -12,8 +12,8 @@ python.toolchain(python_version = "3.13") # TODO: This test module should also verify that isolate = True works, will do in a followup PR. pip = use_extension("@rules_python//python/extensions:pip.bzl", "pip") pip.parse( - hub_name = "pypi", + hub_name = "pypi_lockfile", python_version = "3.13", requirements_lock = "//:requirements_lock.txt", ) -use_repo(pip, "pypi") +use_repo(pip, pypi = "pypi_lockfile") diff --git a/tests/integration/compile_pip_requirements_test_from_external_repo/MODULE.bazel b/tests/integration/compile_pip_requirements_test_from_external_repo/MODULE.bazel index 596a0bcfc8..201cb4a1ab 100644 --- a/tests/integration/compile_pip_requirements_test_from_external_repo/MODULE.bazel +++ b/tests/integration/compile_pip_requirements_test_from_external_repo/MODULE.bazel @@ -19,7 +19,7 @@ local_path_override( pip = use_extension("@rules_python//python/extensions:pip.bzl", "pip") pip.parse( - hub_name = "pypi", + hub_name = "pypi_compile", python_version = "3.9", requirements_lock = "@compile_pip_requirements//:requirements_lock.txt", ) diff --git a/tests/integration/pip_parse_isolated/MODULE.bazel b/tests/integration/pip_parse_isolated/MODULE.bazel index 6c44257acb..c25e9dc024 100644 --- a/tests/integration/pip_parse_isolated/MODULE.bazel +++ b/tests/integration/pip_parse_isolated/MODULE.bazel @@ -12,8 +12,8 @@ python.toolchain(python_version = "3.13") # This test module verifies that dependencies can be used with `isolate = True`. pip = use_extension("@rules_python//python/extensions:pip.bzl", "pip", isolate = True) pip.parse( - hub_name = "pypi", + hub_name = "pypi_isolated", python_version = "3.13", requirements_lock = "//:requirements_lock.txt", ) -use_repo(pip, "pypi") +use_repo(pip, pypi = "pypi_isolated") diff --git a/tests/integration/whl_library/.bazelrc b/tests/integration/whl_library/.bazelrc new file mode 100644 index 0000000000..eee47a8f19 --- /dev/null +++ b/tests/integration/whl_library/.bazelrc @@ -0,0 +1,2 @@ +common --incompatible_default_to_explicit_init_py +test --test_output=errors diff --git a/tests/integration/whl_library/BUILD.bazel b/tests/integration/whl_library/BUILD.bazel new file mode 100644 index 0000000000..0c26fee4cf --- /dev/null +++ b/tests/integration/whl_library/BUILD.bazel @@ -0,0 +1,65 @@ +load("@rules_python//python:py_test.bzl", "py_test") + +[ + alias( + name = "{}_pkg".format(pkg), + actual = "@rules_python//python:none", + ) + for pkg in [ + "charset_normalizer", + "certifi", + "idna", + "urllib3", + ] +] + +[ + filegroup( + name = "{}_whl".format(pkg), + srcs = [], + visibility = ["//visibility:public"], + ) + for pkg in [ + "charset_normalizer", + "certifi", + "idna", + "urllib3", + ] +] + +# Extract all deps +genquery( + name = "whl_target_deps", + expression = "deps(@whl_archive//:pkg)", + scope = ["@whl_archive//:pkg"], +) + +# Extract all deps +genquery( + name = "whl_deps_target_deps", + expression = "deps(@whl_deps_library//:pkg)", + scope = ["@whl_deps_library//:pkg"], +) + +py_test( + name = "test_contents", + srcs = ["test_contents.py"], + data = [ + ":whl_deps_target_deps", + ":whl_target_deps", + "@pip_archive//:srcs", + "@pip_sdist_archive//:srcs", + "@whl_archive//:srcs", + "@whl_archive//:whl", + "@whl_deps_library//:whl", + ], + env = { + "SDIST_SRC_FILES": "$(locations @pip_sdist_archive//:srcs)", + "SRC_FILES": "$(locations @pip_archive//:srcs)", + "WHL_DEPS": "$(location :whl_target_deps)", + "WHL_DEPS_LOCATION": "$(location @whl_deps_library//:whl)", + "WHL_FILES": "$(locations @whl_archive//:srcs)", + "WHL_LOCATION": "$(location @whl_archive//:whl)", + "WHL_TARGET_DEPS": "$(location :whl_deps_target_deps)", + }, +) diff --git a/tests/integration/whl_library/MODULE.bazel b/tests/integration/whl_library/MODULE.bazel new file mode 100644 index 0000000000..39a579f79e --- /dev/null +++ b/tests/integration/whl_library/MODULE.bazel @@ -0,0 +1,57 @@ +module(name = "integration_test") + +bazel_dep(name = "package_metadata", version = "0.0.7") +bazel_dep(name = "rules_python", version = "0.0.0") +local_path_override( + module_name = "rules_python", + path = "../../..", +) + +python = use_extension("@rules_python//python/extensions:python.bzl", "python") +python.toolchain( + python_version = "3.14", +) +use_repo(python, pbs_host = "python_3_14_host") + +pip_archive = use_repo_rule("@rules_python//python/private/pypi:whl_library.bzl", "pip_archive") + +pip_archive( + name = "pip_sdist_archive", + filename = "requests-2.34.2.tar.gz", + python_interpreter_target = "@pbs_host//:python", + requirement = "requests", + # https://pypi.org/project/requests/#requests-2.34.2.tar.gz + sha256 = "f288924cae4e29463698d6d60bc6a4da69c89185ad1e0bcc4104f584e960b9ed", + urls = [ + "https://files.pythonhosted.org/packages/ac/c3/e2a2b89f2d3e2179abd6d00ebd70bff6273f37fb3e0cc209f48b39d00cbf/requests-2.34.2.tar.gz", + ], +) + +pip_archive( + name = "pip_archive", + python_interpreter_target = "@pbs_host//:python", + requirement = "requests", +) + +whl_archive = use_repo_rule("@rules_python//python/private/pypi:whl_library.bzl", "whl_archive") + +whl_archive( + name = "whl_archive", + dep_template = "@integration_test//:{name}_{target}", + filename = "requests-2.34.2-py3-none-any.whl", + # https://pypi.org/project/requests/#requests-2.34.2-py3-none-any.whl + requirement = "requests", + sha256 = "2a0d60c172f83ac6ab31e4554906c0f3b3588d37b5cb939b1c061f4907e278e0", + urls = [ + "https://files.pythonhosted.org/packages/a0/f4/c67b0b3f1b9245e8d266f0f112c500d50e5b4e83cb6f3b71b6528104182a/requests-2.34.2-py3-none-any.whl", + ], +) + +whl_deps_library = use_repo_rule("@rules_python//python/private/pypi:whl_library.bzl", "whl_deps_library") + +whl_deps_library( + name = "whl_deps_library", + dep_template = "@integration_test//:{name}_{target}", + metadata_file = "@whl_archive//:metadata.json", + requirement = "requests", +) diff --git a/tests/integration/whl_library/WORKSPACE b/tests/integration/whl_library/WORKSPACE new file mode 100644 index 0000000000..e69de29bb2 diff --git a/tests/integration/whl_library/test_contents.py b/tests/integration/whl_library/test_contents.py new file mode 100644 index 0000000000..ab352a5d98 --- /dev/null +++ b/tests/integration/whl_library/test_contents.py @@ -0,0 +1,179 @@ +import os +import re +import unittest +from pathlib import Path + + +class TestContents(unittest.TestCase): + maxDiff = None + + @staticmethod + def _get_files(env_var: str) -> list[str]: + return [ + f.partition("site-packages/")[-1] for f in os.environ[env_var].split(" ") + ] + + def test_sdist_srcs(self): + self.assertEqual( + self._get_files("SDIST_SRC_FILES"), + [ + "requests/__init__.py", + "requests/__version__.py", + "requests/_internal_utils.py", + "requests/_types.py", + "requests/adapters.py", + "requests/api.py", + "requests/auth.py", + "requests/certs.py", + "requests/compat.py", + "requests/cookies.py", + "requests/exceptions.py", + "requests/help.py", + "requests/hooks.py", + "requests/models.py", + "requests/packages.py", + "requests/sessions.py", + "requests/status_codes.py", + "requests/structures.py", + "requests/utils.py", + ], + ) + + def test_srcs(self): + self.assertEqual( + self._get_files("SRC_FILES"), + [ + "requests/__init__.py", + "requests/__version__.py", + "requests/_internal_utils.py", + "requests/_types.py", + "requests/adapters.py", + "requests/api.py", + "requests/auth.py", + "requests/certs.py", + "requests/compat.py", + "requests/cookies.py", + "requests/exceptions.py", + "requests/help.py", + "requests/hooks.py", + "requests/models.py", + "requests/packages.py", + "requests/sessions.py", + "requests/status_codes.py", + "requests/structures.py", + "requests/utils.py", + ], + ) + + def test_whl_srcs(self): + self.assertEqual( + self._get_files("WHL_FILES"), + [ + "requests/__init__.py", + "requests/__version__.py", + "requests/_internal_utils.py", + "requests/_types.py", + "requests/adapters.py", + "requests/api.py", + "requests/auth.py", + "requests/certs.py", + "requests/compat.py", + "requests/cookies.py", + "requests/exceptions.py", + "requests/help.py", + "requests/hooks.py", + "requests/models.py", + "requests/packages.py", + "requests/sessions.py", + "requests/status_codes.py", + "requests/structures.py", + "requests/utils.py", + ], + ) + + def test_whl_location(self): + self.assertTrue( + os.environ["WHL_LOCATION"].endswith("requests-2.34.2-py3-none-any.whl"), + msg=os.environ["WHL_LOCATION"], + ) + self.assertTrue( + os.environ["WHL_DEPS_LOCATION"].endswith( + "requests-2.34.2-py3-none-any.whl" + ), + msg=os.environ["WHL_DEPS_LOCATION"], + ) + + @staticmethod + def _read_file(env_var: str) -> list[str]: + return set(Path(os.environ[env_var]).read_text().splitlines()) + + @staticmethod + def _normalize_label(label: str) -> str: + if not label.startswith("@@"): + return label + repo, _, rest = label.partition("//") + parts = [p for p in re.split(r"[~+]", repo[2:]) if p] + if parts: + return f"@{parts[-1]}//{rest}" + return label + + def test_whl_deps_ar_the_same(self): + for var, main_dep in { + "WHL_DEPS": "@whl_archive//:pkg", + "WHL_TARGET_DEPS": "@whl_deps_library//:pkg", + }.items(): + self.assertEqual( + { + self._normalize_label(x) + for x in self._read_file(var) + if not x.endswith("toolchain_type") + }, + { + main_dep, + "//:certifi_pkg", + "//:charset_normalizer_pkg", + "//:idna_pkg", + "//:urllib3_pkg", + "@whl_archive//:data", + "@whl_archive//:package_metadata", + "@whl_archive//:site-packages/requests-2.34.2.dist-info/INSTALLER", + "@whl_archive//:site-packages/requests-2.34.2.dist-info/METADATA", + "@whl_archive//:site-packages/requests-2.34.2.dist-info/RECORD", + "@whl_archive//:site-packages/requests-2.34.2.dist-info/WHEEL", + "@whl_archive//:site-packages/requests-2.34.2.dist-info/licenses/LICENSE", + "@whl_archive//:site-packages/requests-2.34.2.dist-info/licenses/NOTICE", + "@whl_archive//:site-packages/requests-2.34.2.dist-info/top_level.txt", + "@whl_archive//:site-packages/requests/__init__.py", + "@whl_archive//:site-packages/requests/__version__.py", + "@whl_archive//:site-packages/requests/_internal_utils.py", + "@whl_archive//:site-packages/requests/_types.py", + "@whl_archive//:site-packages/requests/adapters.py", + "@whl_archive//:site-packages/requests/api.py", + "@whl_archive//:site-packages/requests/auth.py", + "@whl_archive//:site-packages/requests/certs.py", + "@whl_archive//:site-packages/requests/compat.py", + "@whl_archive//:site-packages/requests/cookies.py", + "@whl_archive//:site-packages/requests/exceptions.py", + "@whl_archive//:site-packages/requests/help.py", + "@whl_archive//:site-packages/requests/hooks.py", + "@whl_archive//:site-packages/requests/models.py", + "@whl_archive//:site-packages/requests/packages.py", + "@whl_archive//:site-packages/requests/py.typed", + "@whl_archive//:site-packages/requests/sessions.py", + "@whl_archive//:site-packages/requests/status_codes.py", + "@whl_archive//:site-packages/requests/structures.py", + "@whl_archive//:site-packages/requests/utils.py", + "@whl_archive//:srcs", + "@rules_python//python:none", + "@rules_python//python/config_settings:_is_venvs_site_packages_yes", + "@rules_python//python/config_settings:add_srcs_to_runfiles", + "@rules_python//python/config_settings:precompile", + "@rules_python//python/config_settings:precompile_source_retention", + "@rules_python//python/config_settings:venvs_site_packages", + "@rules_python//python/private:sentinel", + }, + ) + + +if __name__ == "__main__": + unittest.main() diff --git a/tests/pypi/generate_whl_library_build_bazel/generate_whl_library_build_bazel_tests.bzl b/tests/pypi/generate_whl_library_build_bazel/generate_whl_library_build_bazel_tests.bzl index 1fd99205b1..bb03d9a589 100644 --- a/tests/pypi/generate_whl_library_build_bazel/generate_whl_library_build_bazel_tests.bzl +++ b/tests/pypi/generate_whl_library_build_bazel/generate_whl_library_build_bazel_tests.bzl @@ -23,17 +23,11 @@ def _test_all_workspace(env): want = """\ load("@package_metadata//rules:package_metadata.bzl", "package_metadata") load("@pypi//:config.bzl", "packages") -load("@rules_python//python/private/pypi:whl_library_targets.bzl", "whl_library_targets_from_requires") +load("@rules_python//python/private/pypi:whl_library_targets.bzl", "whl_library_targets") package(default_visibility = ["//visibility:public"]) -package_metadata( - name = "package_metadata", - purl = None, - visibility = ["//:__subpackages__"], -) - -whl_library_targets_from_requires( +whl_library_targets( copy_executables = { "exec_src": "exec_dest", }, @@ -53,6 +47,7 @@ whl_library_targets_from_requires( ], group_name = "qux", include = packages, + metadata_name = "foo", name = "foo.whl", requires_dist = [ "foo", @@ -60,11 +55,23 @@ whl_library_targets_from_requires( "qux", ], srcs_exclude = ["srcs_exclude_all"], + tags = [ + "pypi_name=foo", + "pypi_version=0", + ], +) + +package_metadata( + name = "package_metadata", + purl = "foo", + visibility = ["//:__subpackages__"], ) # SOMETHING SPECIAL AT THE END """ actual = generate_whl_library_build_bazel( + metadata_version = "0", + metadata_name = "foo", dep_template = "@pypi//{name}:{target}", name = "foo.whl", requires_dist = ["foo", "bar-baz", "qux"], @@ -80,6 +87,7 @@ whl_library_targets_from_requires( config_load = "@pypi//:config.bzl", group_name = "qux", group_deps = ["foo", "fox", "qux"], + purl = "foo", ) env.expect.that_str(actual.replace("@@", "@")).equals(want) @@ -89,17 +97,11 @@ def _test_all(env): want = """\ load("@package_metadata//rules:package_metadata.bzl", "package_metadata") load("@pypi//:config.bzl", "packages") -load("@rules_python//python/private/pypi:whl_library_targets.bzl", "whl_library_targets_from_requires") +load("@rules_python//python/private/pypi:whl_library_targets.bzl", "whl_library_targets") package(default_visibility = ["//visibility:public"]) -package_metadata( - name = "package_metadata", - purl = None, - visibility = ["//:__subpackages__"], -) - -whl_library_targets_from_requires( +whl_library_targets( copy_executables = { "exec_src": "exec_dest", }, @@ -119,6 +121,7 @@ whl_library_targets_from_requires( ], group_name = "qux", include = packages, + metadata_name = "foo", name = "foo.whl", requires_dist = [ "foo", @@ -126,11 +129,23 @@ whl_library_targets_from_requires( "qux", ], srcs_exclude = ["srcs_exclude_all"], + tags = [ + "pypi_name=foo", + "pypi_version=0", + ], +) + +package_metadata( + name = "package_metadata", + purl = "foo", + visibility = ["//:__subpackages__"], ) # SOMETHING SPECIAL AT THE END """ actual = generate_whl_library_build_bazel( + metadata_version = "0", + metadata_name = "foo", dep_template = "@pypi//{name}:{target}", name = "foo.whl", requires_dist = ["foo", "bar-baz", "qux"], @@ -145,6 +160,7 @@ whl_library_targets_from_requires( ), config_load = "@pypi//:config.bzl", group_name = "qux", + purl = "foo", group_deps = ["foo", "fox", "qux"], ) env.expect.that_str(actual.replace("@@", "@")).equals(want) @@ -155,17 +171,11 @@ def _test_all_with_loads(env): want = """\ load("@package_metadata//rules:package_metadata.bzl", "package_metadata") load("@pypi//:config.bzl", "packages") -load("@rules_python//python/private/pypi:whl_library_targets.bzl", "whl_library_targets_from_requires") +load("@rules_python//python/private/pypi:whl_library_targets.bzl", "whl_library_targets") package(default_visibility = ["//visibility:public"]) -package_metadata( - name = "package_metadata", - purl = None, - visibility = ["//:__subpackages__"], -) - -whl_library_targets_from_requires( +whl_library_targets( copy_executables = { "exec_src": "exec_dest", }, @@ -185,6 +195,7 @@ whl_library_targets_from_requires( ], group_name = "qux", include = packages, + metadata_name = "foo", name = "foo.whl", requires_dist = [ "foo", @@ -192,11 +203,23 @@ whl_library_targets_from_requires( "qux", ], srcs_exclude = ["srcs_exclude_all"], + tags = [ + "pypi_name=foo", + "pypi_version=0", + ], +) + +package_metadata( + name = "package_metadata", + purl = "foo", + visibility = ["//:__subpackages__"], ) # SOMETHING SPECIAL AT THE END """ actual = generate_whl_library_build_bazel( + metadata_version = "0", + metadata_name = "foo", dep_template = "@pypi//{name}:{target}", name = "foo.whl", requires_dist = ["foo", "bar-baz", "qux"], @@ -212,6 +235,7 @@ whl_library_targets_from_requires( group_name = "qux", config_load = "@pypi//:config.bzl", group_deps = ["foo", "fox", "qux"], + purl = "foo", ) env.expect.that_str(actual.replace("@@", "@")).equals(want) diff --git a/tests/pypi/whl_library_targets/whl_library_targets_tests.bzl b/tests/pypi/whl_library_targets/whl_library_targets_tests.bzl index 1f4aac1ba8..0a059c9d6a 100644 --- a/tests/pypi/whl_library_targets/whl_library_targets_tests.bzl +++ b/tests/pypi/whl_library_targets/whl_library_targets_tests.bzl @@ -17,8 +17,8 @@ load("@rules_testing//lib:test_suite.bzl", "test_suite") load( "//python/private/pypi:whl_library_targets.bzl", + "whl_library_deps_targets", "whl_library_srcs", - "whl_library_targets_from_requires", ) # buildifier: disable=bzl-visibility load("//tests/support/mocks:mocks.bzl", "mocks") @@ -105,7 +105,7 @@ def _test_copy(env): _tests.append(_test_copy) -def _test_whl_and_library_deps_from_requires(env): +def _test_whl_library_deps_targets(env): filegroup_calls = [] py_library_calls = [] env_marker_setting_calls = [] @@ -118,10 +118,9 @@ def _test_whl_and_library_deps_from_requires(env): m_glob.results.append(["site-packages/foo/DATA.txt"]) # data m_glob.results.append(["site-packages/foo/PYI.pyi"]) # pyi - whl_library_targets_from_requires( + whl_library_deps_targets( name = "foo-0-py3-none-any.whl", metadata_name = "Foo", - metadata_version = "0", dep_template = "@pypi//{name}:{target}", requires_dist = [ "foo", # this self-edge will be ignored @@ -130,11 +129,13 @@ def _test_whl_and_library_deps_from_requires(env): "booo", # this is effectively excluded due to the list below ], include = ["foo", "bar", "bar_baz"], - data_exclude = [], # Overrides for testing - filegroups = {}, + repo = None, + aliases = None, + extras = [], native = struct( filegroup = lambda **kwargs: filegroup_calls.append(kwargs), + alias = lambda **kwargs: None, config_setting = lambda **_: None, glob = m_glob.glob, ), @@ -147,80 +148,36 @@ def _test_whl_and_library_deps_from_requires(env): ) env.expect.that_collection(filegroup_calls).contains_exactly([ - { - "name": "whl_file", - "srcs": ["foo-0-py3-none-any.whl"], - "visibility": ["//visibility:public"], - }, { "name": "whl", # NOTE @aignas 2026-07-25: depending on the brackets position one may get different # results in the expectation. - "data": ["whl_file"] + (["@pypi//bar:whl"] + select({ + "srcs": ["whl_file"], + "data": ["@pypi//bar:whl"] + select({ ":is_include_bar_baz_true": ["@pypi//bar_baz:whl"], "//conditions:default": [], - })), + }), "visibility": ["//visibility:public"], }, ]) # buildifier: @unsorted-dict-items - env.expect.that_collection(py_library_calls).has_size(2) + env.expect.that_collection(py_library_calls).has_size(1) if len(py_library_calls) != 1: return py_library_call = py_library_calls[0] env.expect.that_dict(py_library_call).contains_exactly({ "name": "pkg", - "srcs": ["site-packages/foo/SRCS.py"] + select({ - Label("//python/config_settings:_is_venvs_site_packages_yes"): [], - "//conditions:default": ["_create_inits_target"], - }), - "pyi_srcs": ["site-packages/foo/PYI.pyi"], - "data": ["site-packages/foo/DATA.txt", "data"], - "imports": ["site-packages"], - "deps": ["@pypi//bar:pkg"] + select({ + "srcs": ["srcs"], + "deps": ["srcs", "@pypi//bar:pkg"] + select({ ":is_include_bar_baz_true": ["@pypi//bar_baz:pkg"], "//conditions:default": [], }), - "tags": ["pypi_name=Foo", "pypi_version=0"], + "tags": [], "visibility": ["//visibility:public"], - "experimental_venvs_site_packages": Label("//python/config_settings:venvs_site_packages"), - "namespace_package_files": [] + select({ - Label("//python/config_settings:_is_venvs_site_packages_yes"): [], - "//conditions:default": ["_create_inits_target"], - }), }) # buildifier: @unsorted-dict-items - env.expect.that_collection(m_glob.calls).contains_exactly([ - # bin call - mocks.glob_call( - ["bin/*"], - allow_empty = True, - ), - # rewrite-bin call - mocks.glob_call( - ["rewrite-bin/*"], - allow_empty = True, - ), - # srcs call - mocks.glob_call( - ["site-packages/**/*.py"], - exclude = [], - allow_empty = True, - ), - # data call - mocks.glob_call( - ["site-packages/**/*"], - exclude = [ - "**/*.py", - "**/*.pyc", - "**/*.pyc.*", - ], - allow_empty = True, - ), - # pyi call - mocks.glob_call(["site-packages/**/*.pyi"], allow_empty = True), - ]) + env.expect.that_collection(m_glob.calls).contains_exactly([]) env.expect.that_collection(env_marker_setting_calls).contains_exactly([ { @@ -230,7 +187,55 @@ def _test_whl_and_library_deps_from_requires(env): }, ]) # buildifier: @unsorted-dict-items -_tests.append(_test_whl_and_library_deps_from_requires) +_tests.append(_test_whl_library_deps_targets) + +def _test_whl_library_deps_targets_no_deps(env): + alias_calls = [] + filegroup_calls = [] + py_library_calls = [] + env_marker_setting_calls = [] + + whl_library_deps_targets( + name = "foo-0-py3-none-any.whl", + metadata_name = "Foo", + dep_template = "@pypi//{name}:{target}", + requires_dist = [], + group_name = "qux", + repo = None, + aliases = {}, + extras = [], + native = struct( + filegroup = lambda **kwargs: filegroup_calls.append(kwargs), + alias = lambda **kwargs: alias_calls.append(kwargs), + config_setting = lambda **_: None, + glob = lambda **_: [], + ), + rules = struct( + py_library = lambda **kwargs: py_library_calls.append(kwargs), + env_marker_setting = lambda **kwargs: env_marker_setting_calls.append(kwargs), + ), + ) + + # If the package is in a group but has no deps, then the public labels should be aliases + # to the srcs targets and no other targets should be created. + env.expect.that_collection(alias_calls).contains_exactly([ + { + "name": "pkg", + "actual": "srcs", + "visibility": ["//visibility:public"], + }, + { + "name": "whl", + "actual": "whl_file", + "visibility": ["//visibility:public"], + }, + ]) # buildifier: @unsorted-dict-items + + env.expect.that_collection(filegroup_calls).contains_exactly([]) + env.expect.that_collection(py_library_calls).contains_exactly([]) + env.expect.that_collection(env_marker_setting_calls).contains_exactly([]) + +_tests.append(_test_whl_library_deps_targets_no_deps) def _test_sdist_excludes_record(env): py_library_calls = [] From b6bd27bf35ef58482121599dbc3216a4194629bf Mon Sep 17 00:00:00 2001 From: Ignas Anikevicius <240938+aignas@users.noreply.github.com> Date: Sat, 1 Aug 2026 16:02:27 +0900 Subject: [PATCH 876/922] tests(pypi): add test for excluding bazel files from wheels (#3979) This adds a test to verify that bazel files are excluded from the files matched from an extracted wheel. Followup to #3960 Work towards #2948 --- python/private/pypi/whl_library_targets.bzl | 5 +- .../whl_library_targets_tests.bzl | 54 +++++++++++++++++++ 2 files changed, 57 insertions(+), 2 deletions(-) diff --git a/python/private/pypi/whl_library_targets.bzl b/python/private/pypi/whl_library_targets.bzl index 737a739145..ee5c781b3d 100644 --- a/python/private/pypi/whl_library_targets.bzl +++ b/python/private/pypi/whl_library_targets.bzl @@ -229,8 +229,9 @@ def whl_library_srcs( filegroups = { EXTRACTED_WHEEL_FILES: dict( include = ["**"], - exclude = ( - _BAZEL_REPO_FILE_GLOBS + + # The Bazel repo files are always excluded; only the sdist + # filename is conditional on `sdist_filename`. + exclude = _BAZEL_REPO_FILE_GLOBS + ( [sdist_filename] if sdist_filename else [] ), ), diff --git a/tests/pypi/whl_library_targets/whl_library_targets_tests.bzl b/tests/pypi/whl_library_targets/whl_library_targets_tests.bzl index 0a059c9d6a..d752159b32 100644 --- a/tests/pypi/whl_library_targets/whl_library_targets_tests.bzl +++ b/tests/pypi/whl_library_targets/whl_library_targets_tests.bzl @@ -277,6 +277,60 @@ def _test_sdist_excludes_record(env): _tests.append(_test_sdist_excludes_record) +def _test_exclude_bazel_files(env): + # Regression test: the `extracted_whl_files` glob must always exclude the + # Bazel repo files, even when the wheel is not built from an sdist. + for sdist_filename in [None, "foo.tar.gz"]: + m_glob = mocks.glob() + m_glob.results.append([]) # bin + m_glob.results.append([]) # rewrite-bin + m_glob.results.append([]) # extracted_whl_files + m_glob.results.append([]) # dist_info + m_glob.results.append([]) # data + + whl_library_srcs( + name = "foo.whl", + sdist_filename = sdist_filename, + native = struct( + filegroup = lambda **_: None, + glob = m_glob.glob, + ), + rules = struct( + venv_rewrite_shebang = lambda **kwargs: None, + ), + ) + + expected_exclude = [ + "BUILD", + "BUILD.bazel", + "REPO.bazel", + "WORKSPACE", + "WORKSPACE.bzlmod", + "WORKSPACE.bazel", + ] + if sdist_filename: + expected_exclude.append(sdist_filename) + + env.expect.that_collection(m_glob.calls).contains_exactly([ + mocks.glob_call(["bin/*"], allow_empty = True), + mocks.glob_call(["rewrite-bin/*"], allow_empty = True), + mocks.glob_call( + include = ["**"], + exclude = expected_exclude, + allow_empty = True, + ), + mocks.glob_call( + include = ["site-packages/*.dist-info/**"], + allow_empty = True, + ), + mocks.glob_call( + include = ["data/**", "bin/**", "include/**"], + allow_empty = True, + ), + ]) + +_tests.append(_test_exclude_bazel_files) + def whl_library_targets_test_suite(name): """create the test suite. From 700ce58f4f438c8ec348043e7cc235017ee5a8ba Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Sat, 1 Aug 2026 00:19:00 -0700 Subject: [PATCH 877/922] agents: relocate analyze-ci-failure logs to skill scratch directory (#3980) Previously, `analyze_ci_failure.py` output fetched CI logs and generated repair plans into a `ci_logs/` subfolder located directly inside the script directory (`.agents/skills/analyze-ci-failure/scripts/`). Storing runtime logs within the scripts directory pollutes the source tree. To resolve this: * Update `analyze_ci_failure.py` to target a dedicated `scratch/` directory at the skill root level (`.agents/skills/analyze-ci-failure/scratch/`). * Add `.agents/**/scratch` to `.gitignore` so all skill-generated scratch files remain untracked across all agent skills. --- .../analyze-ci-failure/scripts/analyze_ci_failure.py | 10 +++++----- .gitignore | 3 +++ 2 files changed, 8 insertions(+), 5 deletions(-) diff --git a/.agents/skills/analyze-ci-failure/scripts/analyze_ci_failure.py b/.agents/skills/analyze-ci-failure/scripts/analyze_ci_failure.py index 20212f6d23..1f976f3e79 100644 --- a/.agents/skills/analyze-ci-failure/scripts/analyze_ci_failure.py +++ b/.agents/skills/analyze-ci-failure/scripts/analyze_ci_failure.py @@ -121,12 +121,12 @@ def main(): parser.add_argument("conv_id", help="Conversation ID to report back to") args = parser.parse_args() - skill_dir = os.path.abspath(os.path.dirname(__file__)) - logs_dir = os.path.join(skill_dir, "ci_logs") - os.makedirs(logs_dir, exist_ok=True) + skill_dir = os.path.abspath(os.path.join(os.path.dirname(__file__), "..")) + scratch_dir = os.path.join(skill_dir, "scratch") + os.makedirs(scratch_dir, exist_ok=True) safe_jname = re.sub(r"[^a-zA-Z0-9]", "_", args.job_name) - log_path = os.path.join(logs_dir, f"ci_{safe_jname}_{args.job_id}.log") + log_path = os.path.join(scratch_dir, f"ci_{safe_jname}_{args.job_id}.log") fetch_log(args.build_id, args.job_id, log_path) @@ -134,7 +134,7 @@ def main(): errors = parse_log(log_path) plan = create_plan(args.job_name, log_path, errors) - plan_file = os.path.join(logs_dir, f"ci_plan_{safe_jname}.md") + plan_file = os.path.join(scratch_dir, f"ci_plan_{safe_jname}.md") with open(plan_file, "w") as f: f.write(plan) diff --git a/.gitignore b/.gitignore index efce592aa0..cb5bdec980 100644 --- a/.gitignore +++ b/.gitignore @@ -58,3 +58,6 @@ MODULE.bazel.lock # Buildkite logs *Windows*.log +# Agent scratch directories +.agents/**/scratch + From 3a64c8b3abadda8eea687ad2c0fdcdd060e6fe45 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Sat, 1 Aug 2026 00:25:32 -0700 Subject: [PATCH 878/922] agents: update create-pr and analyze-ci-failure skills (#3981) Update the create-pr skill to specify --repo bazel-contrib/rules_python and enforce using the agents: title prefix for agent-related pull requests. Also update analyze-ci-failure skill documentation to clarify the scratch directory path for downloaded logs. * Updates create-pr SKILL.md with upstream repo targeting rules and title prefix guidelines. * Updates analyze-ci-failure SKILL.md to clarify log download scratch directory path. --- .agents/skills/analyze-ci-failure/SKILL.md | 2 +- .agents/skills/create-pr/SKILL.md | 4 ++++ 2 files changed, 5 insertions(+), 1 deletion(-) diff --git a/.agents/skills/analyze-ci-failure/SKILL.md b/.agents/skills/analyze-ci-failure/SKILL.md index 6bda9d5d6b..0af8a92fb1 100644 --- a/.agents/skills/analyze-ci-failure/SKILL.md +++ b/.agents/skills/analyze-ci-failure/SKILL.md @@ -15,7 +15,7 @@ check, invoke this skill by running: 1. **Resolves Log**: Automatically resolves the Buildkite job download URL or fetches GitHub Actions logs via `gh` CLI. 2. **Downloads & Ingests**: Fetches the full raw CI log file and saves it - locally. + locally to `.agents/skills/analyze-ci-failure/scratch/`. 3. **Smart Error Extraction**: Scans log lines for critical failure signatures (`Traceback`, `ERROR:`, `FAILED:`, missing packages, compiler aborts, linter errors). diff --git a/.agents/skills/create-pr/SKILL.md b/.agents/skills/create-pr/SKILL.md index 010d112fb8..822ae1bdf8 100644 --- a/.agents/skills/create-pr/SKILL.md +++ b/.agents/skills/create-pr/SKILL.md @@ -15,6 +15,7 @@ to handle PR creation or description drafting. and PR descriptions** and **Documenting changes**) before drafting. - Strictly adhere to `CONTRIBUTING.md` rules for: - **PR Title**: Follow conventional commit style and title formatting. + For agent rules, skills, and system updates, use `agents:` prefix. - **PR Body**: Include rationale, high-level summary, and structure. - **Formatting**: Follow repository style guidelines and structure. - Create a Markdown artifact (`pr_info.md`) containing the PR title, body, @@ -23,6 +24,9 @@ to handle PR creation or description drafting. description, **do not** run `gh pr create`—just create the `pr_info.md` artifact for the user to review. Otherwise, execute `gh pr create` with the formatted title and body. + - **Targeting Upstream Repo**: When executing `gh pr create`, always target + the upstream repository by passing `--repo bazel-contrib/rules_python` and + `--head :`. 3. **Return Status**: Direct the subagent to communicate the PR number or draft status back using `send_message` (or `agentapi send-message`) with the parent conversation ID, or include it in its final completion response. From c11e5e84dad0209bd4073661bdccb7cc44d17cc0 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Sat, 1 Aug 2026 00:53:40 -0700 Subject: [PATCH 879/922] agents: update buildkite-retry-job skill to support job-level retries and rebuilds (#3983) Previously, the Buildkite retry functionality lacked granular job-level retry controls and explicit full-build rebuild options, defaulting to basic retries without flexible CLI flags for targeting specific failed jobs or executing pipeline rebuilds. To address this: * Update `retry_buildkite_jobs.py` to support `--jobs` for matching job names/patterns, `--job-id` for targeting specific job UUIDs, and `--rebuild` for full pipeline rebuilds via `bk build rebuild`. * Update default retry behavior to inspect build job states and automatically attempt job-level retries for failed jobs before falling back to full rebuilds. --- .agents/skills/buildkite-retry-job/SKILL.md | 28 ++-- .../scripts/retry_buildkite_jobs.py | 150 +++++++++++++++--- 2 files changed, 148 insertions(+), 30 deletions(-) diff --git a/.agents/skills/buildkite-retry-job/SKILL.md b/.agents/skills/buildkite-retry-job/SKILL.md index 3f43846da9..4b822e6222 100644 --- a/.agents/skills/buildkite-retry-job/SKILL.md +++ b/.agents/skills/buildkite-retry-job/SKILL.md @@ -1,17 +1,27 @@ --- name: buildkite-retry-job -description: Retry a failed build kite job +description: Retry failed Buildkite jobs individually or rebuild full builds --- -Use `scripts/retry_buildkite_jobs.py` to retry a job. This is best used -when there are network failures. +Use `scripts/retry_buildkite_jobs.py` to retry individual jobs or rebuild full +Buildkite builds. +### Retrying Modes -example: +1. **Job-Level Retry (Default for failed jobs)**: + Retries only specific failing jobs via `bk job retry `: + ```bash + ./.agents/skills/buildkite-retry-job/scripts/retry_buildkite_jobs.py + ``` -``` -retry_buildkite_jobs.py org pipeline build -``` -You can also simply pass a PR number or a direct Buildkite build URL. +2. **Retry Specific Jobs by Name or ID**: + ```bash + ./.agents/skills/buildkite-retry-job/scripts/retry_buildkite_jobs.py --jobs "compile_pip_requirements" + ./.agents/skills/buildkite-retry-job/scripts/retry_buildkite_jobs.py --job-id + ``` -The `--jobs` flag can be used to retry specific jobs. +3. **Rebuild Full Build**: + Re-runs the entire pipeline build from scratch via `bk build rebuild`: + ```bash + ./.agents/skills/buildkite-retry-job/scripts/retry_buildkite_jobs.py --rebuild + ``` diff --git a/.agents/skills/buildkite-retry-job/scripts/retry_buildkite_jobs.py b/.agents/skills/buildkite-retry-job/scripts/retry_buildkite_jobs.py index d501ce8f14..b0a154ac0c 100755 --- a/.agents/skills/buildkite-retry-job/scripts/retry_buildkite_jobs.py +++ b/.agents/skills/buildkite-retry-job/scripts/retry_buildkite_jobs.py @@ -43,39 +43,97 @@ def get_build_url_from_pr(pr_number): def normalize_build_target(target): # Transforms https://buildkite.com/bazel/rules-python-python/builds/15707 - # into bazel/rules-python-python/15707 - m = re.search(r"buildkite\.com/([^/]+)/([^/]+)/builds/(\d+)", target) + # into (org/pipeline, 15707) + m = re.search(r"buildkite\.com/([^/]+/[^/]+)/builds/(\d+)", target) if m: - return f"{m.group(1)}/{m.group(2)}/{m.group(3)}" - return target + return m.group(1), m.group(2) + parts = target.split("/") + if len(parts) == 3: + return f"{parts[0]}/{parts[1]}", parts[2] + elif len(parts) == 2: + return parts[0], parts[1] + return "bazel/rules-python-python", target + + +def retry_job_by_uuid(job_id): + check_cli("bk", "https://github.com/buildkite/cli") + print(f"🚀 Retrying individual job UUID: {job_id}") + return subprocess.run(["bk", "job", "retry", job_id]) + + +def retry_jobs_by_name(pipeline, build_number, job_pattern): + check_cli("bk", "https://github.com/buildkite/cli") + cmd = ["bk", "build", "view", str(build_number), "-p", pipeline, "--json"] + try: + res = subprocess.run(cmd, capture_output=True, text=True, check=True) + data = json.loads(res.stdout) + jobs = data.get("jobs", []) + matched = [] + for j in jobs: + name = j.get("name", "") or j.get("label", "") + state = j.get("state", "") + jid = j.get("id") + if jid and ( + re.search(job_pattern, name, re.IGNORECASE) + or job_pattern.lower() in name.lower() + ): + matched.append((jid, name, state)) + + if not matched: + print( + f"⚠️ No jobs found matching pattern '{job_pattern}' in build #{build_number}", + file=sys.stderr, + ) + sys.exit(1) + + for jid, name, state in matched: + print(f"🚀 Retrying job '{name}' (ID: {jid}, status: {state})...") + subprocess.run(["bk", "job", "retry", jid]) + except Exception as e: + print(f"❌ Error fetching build jobs: {e}", file=sys.stderr) + sys.exit(1) def main(): parser = argparse.ArgumentParser( - description="Retry failed Buildkite jobs using the 'bk' CLI." + description="Retry Buildkite jobs individually using 'bk job retry' or rebuild the whole build using 'bk build rebuild'." ) parser.add_argument( "args", nargs="+", - help="Target build (org pipeline build OR a single PR# / URL / ID)", + help="Target build (PR#, Buildkite URL, or org/pipeline/build)", ) parser.add_argument( "--jobs", "--job-name", dest="job_name", - help="Specific job name or pattern to retry", + help="Specific job name or regex pattern to retry individually via 'bk job retry'", + ) + parser.add_argument( + "--job-id", + dest="job_id", + help="Specific job UUID to retry individually via 'bk job retry'", + ) + parser.add_argument( + "--rebuild", + action="store_true", + help="Rebuild the entire build afresh via 'bk build rebuild'", ) args = parser.parse_args() check_cli("bk", "https://github.com/buildkite/cli") + if args.job_id: + res = retry_job_by_uuid(args.job_id) + sys.exit(res.returncode) + if len(args.args) == 3: target = f"{args.args[0]}/{args.args[1]}/{args.args[2]}" elif len(args.args) == 1: target = args.args[0] else: print( - "❌ Error: Invalid arguments. Provide either 'org pipeline build' or a single target (PR#, URL, or org/pipeline/build).", + "❌ Error: Invalid arguments. Provide a single target (PR#, URL, or org/pipeline/build).", file=sys.stderr, ) sys.exit(1) @@ -84,23 +142,73 @@ def main(): print(f"🔍 Inspecting PR #{target} via gh to find Buildkite URL...") target = get_build_url_from_pr(target) - build_id = normalize_build_target(target) + pipeline, build_number = normalize_build_target(target) if args.job_name: - print(f"🚀 Retrying jobs matching '{args.job_name}' in build: {build_id}") - res = subprocess.run(["bk", "build", "retry", build_id, "--failed"]) - else: - print(f"🚀 Retrying all failed jobs in build: {build_id}") - res = subprocess.run(["bk", "build", "retry", build_id, "--failed"]) - - if res.returncode != 0: - print( - f"❌ Failed to retry build '{build_id}' via 'bk' CLI.", - file=sys.stderr, + retry_jobs_by_name(pipeline, build_number, args.job_name) + elif args.rebuild: + print(f"🚀 Rebuilding entire build #{build_number} for pipeline: {pipeline}") + res = subprocess.run( + ["bk", "build", "rebuild", build_number, "-p", pipeline, "--yes"] ) sys.exit(res.returncode) - - print(f"🎉 Successfully triggered retry for build: {build_id}") + else: + # Default behavior: try job-level retries of failed jobs if any, otherwise rebuild + cmd = [ + "bk", + "build", + "view", + str(build_number), + "-p", + pipeline, + "--json", + ] + try: + res = subprocess.run(cmd, capture_output=True, text=True, check=True) + data = json.loads(res.stdout) + failed_jobs = [ + j + for j in data.get("jobs", []) + if j.get("state") in ("failed", "broken", "timed_out") and j.get("id") + ] + if failed_jobs: + for j in failed_jobs: + jid = j["id"] + jname = j.get("name") or j.get("label") or jid + print(f"🚀 Retrying failed job '{jname}' (ID: {jid})...") + subprocess.run(["bk", "job", "retry", jid]) + else: + print( + f"🚀 Rebuilding entire build #{build_number} for pipeline: {pipeline}" + ) + res = subprocess.run( + [ + "bk", + "build", + "rebuild", + build_number, + "-p", + pipeline, + "--yes", + ] + ) + sys.exit(res.returncode) + except Exception: + print( + f"🚀 Rebuilding entire build #{build_number} for pipeline: {pipeline}" + ) + res = subprocess.run( + [ + "bk", + "build", + "rebuild", + build_number, + "-p", + pipeline, + "--yes", + ] + ) + sys.exit(res.returncode) if __name__ == "__main__": From 216f0f55132eb58ffec398626db0e7e09f321315 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Sat, 1 Aug 2026 02:16:39 -0700 Subject: [PATCH 880/922] agents: prevent duplicate CI monitoring processes in monitor-ci-results skill (#3984) Previously, launching multiple CI monitoring workflows or updating PRs could spawn duplicate `monitor_remote_ci.py` background processes for the same pull request, causing redundant polling and notification noise. To prevent duplication: * Implement non-blocking file locking (`fcntl.flock`) in `monitor_remote_ci.py` using a lockfile in the skill's `scratch/` directory (`.monitored_pr_.lock`). * Automatically exit gracefully with an informative message if another process is already monitoring the target PR. * Update `SKILL.md` guidance to instruct agents to check for existing monitoring processes before launching new ones. * Relocate monitoring state files to the skill `scratch/` directory to keep the skill directory clean. --- .agents/skills/monitor-ci-results/SKILL.md | 21 ++++++--- .../scripts/monitor_remote_ci.py | 45 ++++++++++++++++++- 2 files changed, 59 insertions(+), 7 deletions(-) diff --git a/.agents/skills/monitor-ci-results/SKILL.md b/.agents/skills/monitor-ci-results/SKILL.md index 1cef8293f4..3a1fab866b 100644 --- a/.agents/skills/monitor-ci-results/SKILL.md +++ b/.agents/skills/monitor-ci-results/SKILL.md @@ -5,7 +5,17 @@ description: Monitor remote CI results for a PR and autonomously launch --- When the user requests to monitor remote CI results or watch a pull request, -launch the monitoring script in the background: +or when monitoring CI after PR updates: + +> **Note**: `monitor_remote_ci.py` is a single long-running continuous task. +> It continuously watches all current and future CI runs for the PR. Do NOT +> launch duplicate monitoring jobs for the same PR. + +1. **Check Existing Process**: Check if a monitor script is already running for + the PR (e.g., `pgrep -f "monitor_remote_ci.py "`). If one is + already running, do not start another instance. +2. **Launch Monitoring Script**: If no monitor process is active for + ``, launch the script in the background: ```bash ./.agents/skills/monitor-ci-results/scripts/monitor_remote_ci.py \ "" & @@ -13,10 +23,11 @@ launch the monitoring script in the background: ### ✨ Autonomous Subagent Orchestration 1. **Background Polling**: `monitor_remote_ci.py` continuously polls both - GitHub PR checks and Buildkite workflow executions in the background. -2. **Blocked Jobs**: When a Buildkite job or GitHub check is in a blocked - state waiting for user confirmation, it dispatches a notification via - `agentapi send-message` so the user is alerted to confirm running the job. + GitHub PR checks and Buildkite workflow executions in the background across + new commits and CI re-runs. +2. **Blocked Jobs**: When a Buildkite job or GitHub check is in a blocked state + waiting for user confirmation, it dispatches a notification via `agentapi + send-message` so the user is alerted to confirm running the job. 3. **Failure Reporting**: When any GitHub check or Buildkite job completes with errors, `monitor_remote_ci.py` dispatches a high-priority notification message reporting the failed check back to your conversation. diff --git a/.agents/skills/monitor-ci-results/scripts/monitor_remote_ci.py b/.agents/skills/monitor-ci-results/scripts/monitor_remote_ci.py index f668ed96f5..9b5970757e 100755 --- a/.agents/skills/monitor-ci-results/scripts/monitor_remote_ci.py +++ b/.agents/skills/monitor-ci-results/scripts/monitor_remote_ci.py @@ -1,6 +1,7 @@ #!/usr/bin/env python3 import argparse +import fcntl import json import os import subprocess @@ -79,9 +80,49 @@ def main(): ) args = parser.parse_args() - skill_dir = os.path.abspath(os.path.dirname(__file__)) + skill_dir = os.path.abspath(os.path.join(os.path.dirname(__file__), "..")) + scratch_dir = os.path.join(skill_dir, "scratch") + os.makedirs(scratch_dir, exist_ok=True) - state_file = os.path.join(skill_dir, f"monitored_state_pr_{args.pr}.json") + # Acquire lock for PR to avoid duplicate monitoring processes + lock_file_path = os.path.join(scratch_dir, f".monitored_pr_{args.pr}.lock") + lock_file = open(lock_file_path, "a+") + running_agent_conv_id = os.environ.get("ANTIGRAVITY_CONVERSATION_ID", "") + try: + fcntl.flock(lock_file.fileno(), fcntl.LOCK_EX | fcntl.LOCK_NB) + lock_file.seek(0) + lock_file.truncate(0) + lock_data = { + "pid": os.getpid(), + "agent_conv_id": running_agent_conv_id, + "notify_conv_id": args.conv_id, + } + lock_file.write(json.dumps(lock_data) + "\n") + lock_file.flush() + except (BlockingIOError, OSError): + existing_agent_conv = "" + try: + lock_file.seek(0) + content = lock_file.read().strip() + if content: + data = json.loads(content) + existing_agent_conv = data.get("agent_conv_id", "") or data.get( + "conv_id", "" + ) + except Exception: + pass + + subagent_str = ( + f" under agent/subagent conversation '{existing_agent_conv}'" + if existing_agent_conv + else "" + ) + print( + f"ℹ️ Continuous remote CI monitoring for PR #{args.pr} is already running in another process{subagent_str}. Exiting." + ) + sys.exit(0) + + state_file = os.path.join(scratch_dir, f"monitored_state_pr_{args.pr}.json") monitored = {} if os.path.exists(state_file): try: From ddc675c9a5235d03aa98aae8637860e51ab30752 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Sat, 1 Aug 2026 05:38:09 -0700 Subject: [PATCH 881/922] test: improve error reporting and artifact generation for lockfile test (#3982) When `bzlmod_lockfile_test` fails due to an out-of-date `MODULE.bazel.lock` file (e.g. under `--lockfile_mode=error`), Bazel previously output a generic failure message without indicating what diff changed or providing an updated lockfile artifact. This made debugging CI lockfile failures tedious and manual. To improve the developer experience: * On test failure, automatically run `bazel mod deps --lockfile_mode=update` to compute the expected lockfile changes. * Output a unified diff between the checked-in and expected `MODULE.bazel.lock` directly in the test failure logs along with actionable update commands. * Export the updated `MODULE.bazel.lock` and `MODULE.bazel.lock.diff` to `TEST_UNDECLARED_OUTPUTS_DIR` as build artifacts, allowing them to be downloaded directly in the Buildkite UI. --- tests/integration/BUILD.bazel | 1 + tests/integration/bzlmod_lockfile_test.py | 59 +++++++++++++++++++++++ tests/integration/runner.py | 3 ++ 3 files changed, 63 insertions(+) create mode 100644 tests/integration/bzlmod_lockfile_test.py diff --git a/tests/integration/BUILD.bazel b/tests/integration/BUILD.bazel index ef7681ebe3..13b9c2e855 100644 --- a/tests/integration/BUILD.bazel +++ b/tests/integration/BUILD.bazel @@ -42,6 +42,7 @@ default_test_runner( rules_python_integration_test( name = "bzlmod_lockfile_test", bazel_versions = ["9.1.0"], + py_main = "bzlmod_lockfile_test.py", ) test_suite( diff --git a/tests/integration/bzlmod_lockfile_test.py b/tests/integration/bzlmod_lockfile_test.py new file mode 100644 index 0000000000..650f132f0f --- /dev/null +++ b/tests/integration/bzlmod_lockfile_test.py @@ -0,0 +1,59 @@ +import difflib +import os +import pathlib +import unittest + +from tests.integration import runner + + +class BzlmodLockfileTest(runner.TestCase): + def test_bzlmod_lockfile(self): + lockfile_path = self.repo_root / "MODULE.bazel.lock" + self.assertTrue( + lockfile_path.exists(), + f"Expected lockfile at {lockfile_path}", + ) + original_lockfile = lockfile_path.read_text() + + res = self.run_bazel("test", "//...", check=False) + if res.exit_code == 0: + return + + # Generate updated lockfile to compare diff and export artifact + self.run_bazel("mod", "deps", "--lockfile_mode=update", check=False) + updated_lockfile = lockfile_path.read_text() if lockfile_path.exists() else "" + + undeclared_outputs_dir = os.environ.get("TEST_UNDECLARED_OUTPUTS_DIR") + if undeclared_outputs_dir: + out_dir = pathlib.Path(undeclared_outputs_dir) + (out_dir / "MODULE.bazel.lock").write_text(updated_lockfile) + + if original_lockfile != updated_lockfile: + diff_lines = list( + difflib.unified_diff( + original_lockfile.splitlines(keepends=True), + updated_lockfile.splitlines(keepends=True), + fromfile="MODULE.bazel.lock (checked-in)", + tofile="MODULE.bazel.lock (expected/updated)", + ) + ) + diff_str = "".join(diff_lines) + + if undeclared_outputs_dir: + (out_dir / "MODULE.bazel.lock.diff").write_text(diff_str) + + msg = ( + f"MODULE.bazel.lock is out of date.\n\n" + f"--- DIFF ---\n{diff_str}\n" + f"--- END DIFF ---\n\n" + f"To update the lockfile, run:\n" + f" bazel mod deps --lockfile_mode=update\n" + f"inside tests/integration/bzlmod_lockfile, or copy the updated MODULE.bazel.lock artifact.\n" + ) + self.fail(msg) + else: + self.fail(f"bazel test //... failed:\n{res.describe()}") + + +if __name__ == "__main__": + unittest.main() diff --git a/tests/integration/runner.py b/tests/integration/runner.py index 2534ab2d90..c187623b3c 100644 --- a/tests/integration/runner.py +++ b/tests/integration/runner.py @@ -80,10 +80,13 @@ def setUp(self): # Put the global tmp not under the test tmp to better match how a real # execution has entirely different directories for these. self.tmp_dir = outer_test_tmpdir / "bit_tmp" + self.test_tmp_dir.mkdir(parents=True, exist_ok=True) + self.tmp_dir.mkdir(parents=True, exist_ok=True) self.bazel_env = { "PATH": os.environ["PATH"], "TEST_TMPDIR": str(self.test_tmp_dir), "TMP": str(self.tmp_dir), + "TEMP": str(self.tmp_dir), # For some reason, this is necessary for Bazel 6.4 to work. # If not present, it can't find some bash helpers in @bazel_tools "RUNFILES_DIR": os.environ["TEST_SRCDIR"], From 70734baa7dcc9e93be71682e1642711eb1729327 Mon Sep 17 00:00:00 2001 From: Lequn Chen Date: Sat, 1 Aug 2026 06:14:45 -0700 Subject: [PATCH 882/922] feat(pypi): support non-sha256 --hash pins and index digests (#3974) Previously only `--hash=sha256:` pins and `#sha256=` URL fragments were recognized. Pins using any other hash algorithm were silently dropped: the requirement was treated as hash-less, and in the pip fallback the hashes were stripped from the requirement line entirely, so the artifact was no longer verified against what was locked. Now a distribution digest is carried through the code as one canonical `:` string (the same shape as pip `--hash` and uv.lock `hash` values) for any `hashlib` algorithm: - a new `hash.bzl` module (single `hash` symbol) centralizes digest parsing, validation and conversion to SRI; - `parse_simpleapi_html` accepts any PEP 503 `#=` fragment and keys `whls`/`sdists` by the canonical digest; `sha256s_by_version` is renamed to `hashes_by_version`; - matching pins against the index is a direct dict lookup; when the index only advertises a different algorithm the requirement falls back to pip with the pins kept in the requirement line, so nothing installs unverified (pip verifies sha256/384/512 pins and rejects requirement lines pinned only with other algorithms); - `whl_library` gains an `integrity` attribute, and the extension now always passes the digest as an SRI `integrity` value instead of `sha256`, so the lock entries use one format for all algorithms; a malformed (non-hex) sha256/384/512 digest fails loudly at extension evaluation instead of silently disabling download verification; - the lock file facts store `:` values (`_FACT_VERSION` bumped to `v2`; old facts are refetched once); this also fixes the latent bug where `dist_filenames` was written keyed by URL but read back keyed by digest, so stored filenames were never found; - uv.lock `hash` values pass through for any algorithm instead of being mangled by a `sha256:` string replace. Repo names are unchanged for sha256 digests, and sha256 stays the preferred algorithm when a direct-URL requirement has several pins. Digests whose algorithm has no SRI form (e.g. md5) still match the index but are downloaded without downloader-side verification, with a warning. Fixes #3972 --------- Co-authored-by: Claude Fable 5 Co-authored-by: Ignas Anikevicius <240938+aignas@users.noreply.github.com> --- docs/pypi/download.md | 5 +- news/3972.fixed.md | 11 + python/private/pypi/BUILD.bazel | 9 + python/private/pypi/extension.bzl | 4 +- python/private/pypi/hash.bzl | 189 +++++ python/private/pypi/hub_builder.bzl | 10 +- python/private/pypi/index_sources.bzl | 33 +- python/private/pypi/parse_requirements.bzl | 53 +- python/private/pypi/parse_simpleapi_html.bzl | 32 +- python/private/pypi/pypi_cache.bzl | 52 +- python/private/pypi/simpleapi_download.bzl | 4 +- python/private/pypi/whl_library.bzl | 24 +- python/private/pypi/whl_repo_name.bzl | 16 +- .../bzlmod_lockfile/MODULE.bazel.lock | 646 +++++++++--------- tests/pypi/hash/BUILD.bazel | 3 + tests/pypi/hash/hash_tests.bzl | 102 +++ tests/pypi/hub_builder/hub_builder_tests.bzl | 129 +++- .../index_sources/index_sources_tests.bzl | 82 ++- .../parse_requirements_tests.bzl | 210 ++++-- .../parse_simpleapi_html_tests.bzl | 94 ++- tests/pypi/pypi_cache/pypi_cache_tests.bzl | 195 ++++-- .../simpleapi_download_tests.bzl | 16 +- .../whl_repo_name/whl_repo_name_tests.bzl | 12 + 23 files changed, 1333 insertions(+), 598 deletions(-) create mode 100644 news/3972.fixed.md create mode 100644 python/private/pypi/hash.bzl create mode 100644 tests/pypi/hash/BUILD.bazel create mode 100644 tests/pypi/hash/hash_tests.bzl diff --git a/docs/pypi/download.md b/docs/pypi/download.md index e819ed0791..fb7f38dc24 100644 --- a/docs/pypi/download.md +++ b/docs/pypi/download.md @@ -342,8 +342,9 @@ This does not mean that `rules_python` is fetching the wheels eagerly; rather, it means that it is calling the PyPI server to get the Simple API response to get the list of all available source and wheel distributions. Once it has gotten all of the available distributions, it will select the right ones depending -on the `sha256` values in your `requirements_lock.txt` file. If `sha256` hashes -are not present in the requirements file, we will fall back to matching by version +on the `--hash` values in your `requirements_lock.txt` file (any hash algorithm +advertised by the index can be matched, not only `sha256`). If hashes are not +present in the requirements file, we will fall back to matching by version specified in the lock file. Fetching the distribution information from the PyPI allows `rules_python` to diff --git a/news/3972.fixed.md b/news/3972.fixed.md new file mode 100644 index 0000000000..f96f803988 --- /dev/null +++ b/news/3972.fixed.md @@ -0,0 +1,11 @@ +(pypi) Requirement `--hash=:` pins and Simple API +`#=` URL fragments are now parsed for all hash algorithms +instead of silently dropping everything except `sha256`. Non-sha256 pins are +matched against the digests advertised by the index and downloads are verified +using the corresponding Subresource Integrity value, and the pins are kept in +the requirement line when falling back to `pip` +([#3972](https://github.com/bazel-contrib/rules_python/issues/3972)). +As part of this, `whl_library` repos created by `pip.parse` now always pass +the digest via the `integrity` attribute (SRI format) instead of `sha256`, +and the lock file facts store digests as `:` values (the facts +version was bumped, so cached index information is refreshed once). diff --git a/python/private/pypi/BUILD.bazel b/python/private/pypi/BUILD.bazel index 09ba5ef135..775aa0d9d2 100644 --- a/python/private/pypi/BUILD.bazel +++ b/python/private/pypi/BUILD.bazel @@ -166,6 +166,7 @@ bzl_library( srcs = ["hub_builder.bzl"], deps = [ ":attrs", + ":hash", ":parse_requirements", ":pep508_env", ":pep508_evaluate", @@ -222,6 +223,7 @@ bzl_library( srcs = ["parse_requirements.bzl"], deps = [ ":argparse", + ":hash", ":index_sources", ":parse_requirements_txt", ":pep508_evaluate", @@ -237,6 +239,7 @@ bzl_library( name = "parse_simpleapi_html", srcs = ["parse_simpleapi_html.bzl"], deps = [ + ":hash", ":version_from_filename", "//python/private:normalize_name", ], @@ -534,9 +537,15 @@ bzl_library( srcs = ["env_marker_info.bzl"], ) +bzl_library( + name = "hash", + srcs = ["hash.bzl"], +) + bzl_library( name = "index_sources", srcs = ["index_sources.bzl"], + deps = [":hash"], ) bzl_library( diff --git a/python/private/pypi/extension.bzl b/python/private/pypi/extension.bzl index 412b03c21f..210e169d4d 100644 --- a/python/private/pypi/extension.bzl +++ b/python/private/pypi/extension.bzl @@ -692,8 +692,8 @@ This value is going to be subject to `envsubst` substitutions if necessary, look The indexes must support Simple API as described here: https://packaging.python.org/en/latest/specifications/simple-repository-api/ -Index metadata will be used to get `sha256` values for packages even if the -`sha256` values are not present in the requirements.txt lock file. +Index metadata will be used to get the hash digest values for packages even +if the `--hash` values are not present in the requirements.txt lock file. Defaults to `https://pypi.org/simple`. diff --git a/python/private/pypi/hash.bzl b/python/private/pypi/hash.bzl new file mode 100644 index 0000000000..bcdfaecf00 --- /dev/null +++ b/python/private/pypi/hash.bzl @@ -0,0 +1,189 @@ +# Copyright 2026 The Bazel Authors. All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +"""Helpers for working with distribution hash digests. + +Both the requirement `--hash=:` option (PEP 508 tooling) and the +Simple API `#=` URL fragments (PEP 503) or `hashes` dicts +(PEP 691) may use any algorithm from `hashlib.algorithms_guaranteed`, even +though `sha256` is the most common one. + +Internally a single digest is always represented as the canonical +`:` string (the same shape as the pip `--hash` values and +the uv.lock `hash` values) and only converted to the Subresource Integrity +format at the `ctx.download(integrity = ...)` boundary. +""" + +# The names from `hashlib.algorithms_guaranteed`, which PEP 691 uses as the +# set of valid hash names and PEP 503 strongly recommends fragments to come +# from. +_ALGOS = [ + "blake2b", + "blake2s", + "md5", + "sha1", + "sha224", + "sha256", + "sha384", + "sha3_224", + "sha3_256", + "sha3_384", + "sha3_512", + "sha512", + "shake_128", + "shake_256", +] + +# The algorithms supported by the Subresource Integrity format understood by +# `ctx.download(integrity = ...)`, strongest first. +_SRI_ALGOS = ["sha512", "sha384", "sha256"] + +# The order used to pick the digest identifying an artifact when several are +# available: `sha256` first so that repo names stay stable for the common +# case, then the remaining algorithms that the bazel downloader can verify. +_PREFERRED_ALGOS = ["sha256", "sha512", "sha384"] + +_HEX = "0123456789abcdef" +_B64 = "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/" + +def _hex_to_b64(hex_digest): + if len(hex_digest) % 2: + return None + + data = [] + for i in range(0, len(hex_digest), 2): + hi = _HEX.find(hex_digest[i]) + lo = _HEX.find(hex_digest[i + 1]) + if hi < 0 or lo < 0: + return None + data.append(hi * 16 + lo) + + out = [] + for i in range(0, len(data) // 3 * 3, 3): + n = (data[i] << 16) | (data[i + 1] << 8) | data[i + 2] + out.append(_B64[n >> 18]) + out.append(_B64[(n >> 12) & 63]) + out.append(_B64[(n >> 6) & 63]) + out.append(_B64[n & 63]) + + rem = len(data) % 3 + if rem == 1: + n = data[-1] << 16 + out.append(_B64[n >> 18]) + out.append(_B64[(n >> 12) & 63]) + out.append("==") + elif rem == 2: + n = (data[-2] << 16) | (data[-1] << 8) + out.append(_B64[n >> 18]) + out.append(_B64[(n >> 12) & 63]) + out.append(_B64[(n >> 6) & 63]) + out.append("=") + + return "".join(out) + +def _digest(algo, hex_digest): + """Construct a canonical `:` digest string. + + Args: + algo: {type}`str` the hash algorithm name, e.g. `sha256`. + hex_digest: {type}`str` the hex encoded digest. + + Returns: + {type}`str` the canonical digest string or an empty string if the + algorithm is not a known `hashlib` algorithm or the digest is empty. + """ + algo = algo.lower() + if not hex_digest or algo not in _ALGOS: + return "" + + return "{}:{}".format(algo, hex_digest) + +def _hex_to_sri(algo, hex_digest): + """Convert a hex digest to a Subresource Integrity value. + + Args: + algo: {type}`str` the hash algorithm name, e.g. `sha512`. + hex_digest: {type}`str` the hex encoded digest. + + Returns: + {type}`str` the SRI value (e.g. `sha512-...`) that can be passed to + `ctx.download(integrity = ...)` or an empty string if the algorithm + cannot be expressed as SRI or the digest is not valid hex. + """ + if algo not in _SRI_ALGOS or not hex_digest: + return "" + + b64 = _hex_to_b64(hex_digest.lower()) + if b64 == None: + return "" + + return "{}-{}".format(algo, b64) + +def _integrity(digest): + """Get the SRI value for a canonical `:` string. + + Args: + digest: {type}`str` the canonical digest string. + + Returns: + {type}`str` the SRI value (e.g. `sha256-...`) that can be passed to + `ctx.download(integrity = ...)` or an empty string if the algorithm + cannot be expressed as SRI. Fails if the algorithm is SRI supported + but the digest is not valid hex, because silently returning an empty + string would disable the download verification for an artifact that + should be verifiable. + """ + algo, _, hex_digest = digest.partition(":") + if algo not in _SRI_ALGOS or not hex_digest: + return "" + + sri = _hex_to_sri(algo, hex_digest) + if not sri: + fail("Invalid {} digest: {}".format(algo, hex_digest)) + + return sri + +def _preferred_digest(digests): + """Pick the digest that identifies an artifact, e.g. for repo naming. + + Args: + digests: {type}`list[str]` canonical `:` strings. + + Returns: + {type}`str` the canonical digest string of the most preferred + algorithm present or an empty string if there are no digests. + """ + by_algo = {} + for digest in digests: + if not digest: + continue + algo, _, _ = digest.partition(":") + by_algo.setdefault(algo, digest) + + if not by_algo: + return "" + + for algo in _PREFERRED_ALGOS: + if algo in by_algo: + return by_algo[algo] + + return by_algo[sorted(by_algo.keys())[0]] + +hash = struct( + ALGOS = _ALGOS, + digest = _digest, + hex_to_sri = _hex_to_sri, + integrity = _integrity, + preferred_digest = _preferred_digest, +) diff --git a/python/private/pypi/hub_builder.bzl b/python/private/pypi/hub_builder.bzl index d7974d39ad..596cfb2cbe 100644 --- a/python/private/pypi/hub_builder.bzl +++ b/python/private/pypi/hub_builder.bzl @@ -8,6 +8,7 @@ load("//python/private:text_util.bzl", "render") load("//python/private:version.bzl", "version") load("//python/private:version_label.bzl", "version_label") load(":attrs.bzl", "use_isolated") +load(":hash.bzl", "hash") load(":parse_requirements.bzl", "parse_requirements") load(":pep508_env.bzl", "env") load(":pep508_evaluate.bzl", "evaluate") @@ -676,7 +677,12 @@ def _whl_repo( args["index_url"] = index_url args["urls"] = [src.url] - args["sha256"] = src.sha256 + + # The digest is always passed in the SRI format that the bazel downloader + # supports. It may be empty if the digest uses a hash algorithm that + # cannot be expressed as SRI (e.g. `md5`), in which case `whl_library` + # will download without verification and warn. + args["integrity"] = hash.integrity(src.digest) args["filename"] = src.filename # TODO @aignas 2025-11-02: once we have pipstar enabled we can add extra @@ -685,7 +691,7 @@ def _whl_repo( target_platforms = src.target_platforms if is_multiple_versions else [] return struct( - repo_name = whl_repo_name(src.filename, src.sha256, *target_platforms), + repo_name = whl_repo_name(src.filename, src.digest, *target_platforms), args = args, config_setting = whl_config_setting( version = python_version, diff --git a/python/private/pypi/index_sources.bzl b/python/private/pypi/index_sources.bzl index 1998e4fb33..4353f5051a 100644 --- a/python/private/pypi/index_sources.bzl +++ b/python/private/pypi/index_sources.bzl @@ -16,6 +16,8 @@ A file that houses private functions used in the `bzlmod` extension with the same name. """ +load(":hash.bzl", "hash") + # Just list them here and me super conservative _KNOWN_EXTS = [ # Note, the following source in pip has more extensions @@ -43,8 +45,10 @@ def index_sources(line): line(str): The requirements.txt entry. Returns: - A struct with shas attribute containing: - * `shas` - list[str]; shas to download from pypi_index. + A struct with hashes attribute containing: + * `hashes` - list[str]; `:` hashes of the artifacts + to download from pypi_index. Note that any hash algorithm from + {obj}`hash.ALGOS` is accepted, not only `sha256`. * `version` - str; version of the package. * `marker` - str; the marker expression, as per PEP508 spec. * `requirement` - str; a requirement line without the marker. This can @@ -58,10 +62,12 @@ def index_sources(line): marker, _, _ = maybe_hashes.partition("--hash=") maybe_hashes = maybe_hashes or line - shas = [ - sha.strip() - for sha in maybe_hashes.split("--hash=sha256:")[1:] - ] + hashes = [] + for h in maybe_hashes.split("--hash=")[1:]: + algo, _, hex_digest = h.strip().partition(" ")[0].partition(":") + digest = hash.digest(algo, hex_digest) + if digest: + hashes.append(digest) marker = marker.strip() if head == line: @@ -71,7 +77,7 @@ def index_sources(line): requirement_line = "{} {}".format( requirement, - " ".join(["--hash=sha256:{}".format(sha) for sha in shas]), + " ".join(["--hash={}".format(h) for h in hashes]), ).strip() url = "" @@ -80,9 +86,14 @@ def index_sources(line): maybe_requirement, _, url_and_rest = requirement.partition("@") url = url_and_rest.strip().partition(" ")[0].strip() - url, _, sha256 = url.partition("#sha256=") - if sha256: - shas.append(sha256) + url, _, fragment = url.partition("#") + algo, _, hex_digest = fragment.partition("=") + digest = hash.digest(algo, hex_digest) + if digest: + hashes.append(digest) + elif fragment: + # Not a hash fragment (e.g. `#egg=`), keep it as part of the URL. + url = "{}#{}".format(url, fragment) _, _, filename = url.rpartition("/") # Replace URL encoded characters and luckily there is only one case @@ -104,7 +115,7 @@ def index_sources(line): requirement = requirement, requirement_line = requirement_line, version = version, - shas = sorted(shas), + hashes = sorted(hashes), marker = marker, url = url, filename = filename, diff --git a/python/private/pypi/parse_requirements.bzl b/python/private/pypi/parse_requirements.bzl index 648ec9e65c..4874c7ac79 100644 --- a/python/private/pypi/parse_requirements.bzl +++ b/python/private/pypi/parse_requirements.bzl @@ -30,6 +30,7 @@ load("//python/private:normalize_name.bzl", "normalize_name") load("//python/private:repo_utils.bzl", "repo_utils") load("//python/uv/private:uv_lock_to_requirements.bzl", "uv_lock_extras_map") # buildifier: disable=bzl-visibility load(":argparse.bzl", "argparse") +load(":hash.bzl", "hash") load(":index_sources.bzl", "index_sources") load(":parse_requirements_txt.bzl", "parse_requirements_txt") load(":pep508_evaluate.bzl", "evaluate") @@ -82,7 +83,8 @@ def parse_requirements( * `index_url`: {type}`str` The index URL used to download the package. * `srcs`: {type}`list[struct]` A list of per-distribution source entries, each containing: `distribution`, `extra_pip_args`, `requirement_line`, - `target_platforms`, `filename`, `sha256`, `url`, `yanked`. + `target_platforms`, `filename`, `digest`, `url`, `yanked`. The `digest` + is a `:` string or empty if the artifact digest is unknown. """ if uv_lock and toml_decode: uv_lock = toml_decode(ctx.read(uv_lock)) @@ -203,11 +205,10 @@ def _parse_uv_lock_json(uv_lock, all_platforms, logger, extra_pip_args = None, p for wheel in pkg.get("wheels", []): url = wheel["url"] _, _, filename = url.rpartition("/") - sha256 = wheel.get("hash", "").replace("sha256:", "") candidates.append(struct( filename = filename, url = url, - sha256 = sha256, + digest = _parse_uv_lock_hash(wheel.get("hash", "")), kind = "wheel", )) @@ -216,11 +217,10 @@ def _parse_uv_lock_json(uv_lock, all_platforms, logger, extra_pip_args = None, p if sdist: url = sdist["url"] _, _, filename = url.rpartition("/") - sha256 = sdist.get("hash", "").replace("sha256:", "") sdist_struct = struct( filename = filename, url = url, - sha256 = sha256, + digest = _parse_uv_lock_hash(sdist.get("hash", "")), kind = "sdist", ) @@ -231,7 +231,7 @@ def _parse_uv_lock_json(uv_lock, all_platforms, logger, extra_pip_args = None, p git_struct = struct( filename = filename, url = url, - sha256 = "", + digest = "", kind = "git", source = pkg["source"], ) @@ -264,7 +264,7 @@ def _parse_uv_lock_json(uv_lock, all_platforms, logger, extra_pip_args = None, p # Group platforms by resolved source src_to_plats = {} for p, src in plat_to_src.items(): - key = src.filename + src.sha256 + key = src.filename + src.digest src_to_plats.setdefault(key, struct(src = src, plats = [])).plats.append(p) # Build resolved_srcs @@ -282,7 +282,7 @@ def _parse_uv_lock_json(uv_lock, all_platforms, logger, extra_pip_args = None, p requirement_line = requirement_line, target_platforms = plats, filename = src.filename, - sha256 = src.sha256, + digest = src.digest, url = src.url, yanked = None, )) @@ -309,6 +309,15 @@ def _parse_uv_lock_json(uv_lock, all_platforms, logger, extra_pip_args = None, p logger.debug(lambda: "Parsed {} packages from uv.lock".format(len(ret))) return ret +def _parse_uv_lock_hash(hash_str): + """Parse a uv.lock `hash` value of the form `:`. + + The algorithm is not necessarily `sha256` because uv records the strongest + digest that the index offers. + """ + algo, _, hex_digest = hash_str.partition(":") + return hash.digest(algo, hex_digest) + def _parse_requirements_from_req_files( ctx, *, @@ -495,7 +504,7 @@ def _package_srcs( dist = struct( url = "", filename = "", - sha256 = "", + digest = "", yanked = None, ) req_line = r.srcs.requirement_line @@ -515,7 +524,7 @@ def _package_srcs( requirement_line = req_line, target_platforms = [], filename = dist.filename, - sha256 = dist.sha256, + digest = dist.digest, url = dist.url, yanked = dist.yanked, ), @@ -607,7 +616,7 @@ def _add_dists(*, requirement, index_urls, target_platform, logger = None): dist = struct( url = requirement.srcs.url, filename = requirement.srcs.filename, - sha256 = requirement.srcs.shas[0] if requirement.srcs.shas else "", + digest = hash.preferred_digest(requirement.srcs.hashes), yanked = None, ) @@ -619,29 +628,29 @@ def _add_dists(*, requirement, index_urls, target_platform, logger = None): whls = [] sdist = None - # First try to find distributions by SHA256 if provided - shas_to_use = requirement.srcs.shas - if not shas_to_use: + # First try to find distributions by the hashes if provided + hashes_to_use = requirement.srcs.hashes + if not hashes_to_use: version = requirement.srcs.version - shas_to_use = index_urls.sha256s_by_version.get(version, []) - logger.warn(lambda: "requirement file has been generated without hashes, will use all hashes for the given version {} that could find on the index:\n {}".format(version, shas_to_use)) + hashes_to_use = index_urls.hashes_by_version.get(version, []) + logger.warn(lambda: "requirement file has been generated without hashes, will use all hashes for the given version {} that could find on the index:\n {}".format(version, hashes_to_use)) - for sha256 in shas_to_use: + for digest in hashes_to_use: # For now if the artifact is marked as yanked we just ignore it. # # See https://packaging.python.org/en/latest/specifications/simple-repository-api/#adding-yank-support-to-the-simple-api - maybe_whl = index_urls.whls.get(sha256) + maybe_whl = index_urls.whls.get(digest) if maybe_whl and maybe_whl.yanked == None: whls.append(maybe_whl) continue - maybe_sdist = index_urls.sdists.get(sha256) + maybe_sdist = index_urls.sdists.get(digest) if maybe_sdist and maybe_sdist.yanked == None: sdist = maybe_sdist continue - logger.warn(lambda: "Could not find a whl or an sdist with sha256={}".format(sha256)) + logger.warn(lambda: "Could not find a whl or an sdist with hash {}; note that the index may be advertising digests calculated with a different hash algorithm".format(digest)) yanked = {} for dist in whls + [sdist]: @@ -662,8 +671,8 @@ def _add_dists(*, requirement, index_urls, target_platform, logger = None): if not whls and not sdist: # If there are no suitable wheels to handle for now allow fallback to pip, it # may be a little bit more helpful when debugging? Most likely something is - # going a bit wrong here, should we raise an error because the sha256 have most - # likely mismatched? We are already printing a warning above. + # going a bit wrong here, should we raise an error because the digests have + # most likely mismatched? We are already printing a warning above. return None, True # Select a single wheel that can work on the target_platform diff --git a/python/private/pypi/parse_simpleapi_html.bzl b/python/private/pypi/parse_simpleapi_html.bzl index 7f0d2776d7..e919c8ca55 100644 --- a/python/private/pypi/parse_simpleapi_html.bzl +++ b/python/private/pypi/parse_simpleapi_html.bzl @@ -17,10 +17,11 @@ Parse SimpleAPI HTML in Starlark. """ load("//python/private:normalize_name.bzl", "normalize_name") +load(":hash.bzl", "hash") load(":version_from_filename.bzl", "version_from_filename") def parse_simpleapi_html(*, content, parse_index = False): - """Get the package URLs for given shas by parsing the Simple API HTML. + """Get the package URLs for given digests by parsing the Simple API HTML. Args: content: {type}`str` The Simple API HTML content. @@ -29,11 +30,15 @@ def parse_simpleapi_html(*, content, parse_index = False): Returns: If it is the index page, return the map of package to URL it can be queried from. - Otherwise, a list of structs with: + Otherwise, a struct with `whls` and `sdists` dicts keyed by the `:` + value advertised in the URL fragment (PEP 503, most commonly `sha256`, but any + algorithm from {obj}`hash.ALGOS` is accepted) and `hashes_by_version` mapping each + version to the digests of its artifacts. The dict values are structs with: * filename: {type}`str` The filename of the artifact. * version: {type}`str` The version of the artifact. * url: {type}`str` The URL to download the artifact. - * sha256: {type}`str` The sha256 of the artifact. + * digest: {type}`str` The `:` value of the artifact, may be empty + if the index does not advertise any digest. * metadata_sha256: {type}`str` The whl METADATA sha256 if we can download it. If this is present, then the 'metadata_url' is also present. Defaults to "". * metadata_url: {type}`str` The URL for the METADATA if we can download it. Defaults to "". @@ -43,7 +48,7 @@ def parse_simpleapi_html(*, content, parse_index = False): """ sdists = {} whls = {} - sha256s_by_version = {} + hashes_by_version = {} # 1. Faster Version Extraction # Search only the first 2KB for versioning metadata instead of splitting everything @@ -101,7 +106,14 @@ def parse_simpleapi_html(*, content, parse_index = False): continue # 3. Efficient Attribute Parsing - dist_url, _, sha256 = href.partition("#sha256=") + # PEP 503 says the URL fragment SHOULD be `#=` where the + # hash name may be any algorithm from `hashlib`, not only `sha256`. + dist_url, _, fragment = href.partition("#") + algo, _, hex_digest = fragment.partition("=") + digest = hash.digest(algo, hex_digest) + if not digest and fragment: + # Not a hash fragment, keep it as part of the URL. + dist_url = "{}#{}".format(dist_url, fragment) # Handle Yanked status yanked = None @@ -109,7 +121,7 @@ def parse_simpleapi_html(*, content, parse_index = False): yanked = _unescape_pypi_html(attrs["data-yanked"]) version = version_from_filename(filename) - sha256s_by_version.setdefault(version, []).append(sha256) + hashes_by_version.setdefault(version, []).append(digest) # 4. Optimized Metadata Check (PEP 714) metadata_sha256 = "" @@ -126,16 +138,16 @@ def parse_simpleapi_html(*, content, parse_index = False): filename = filename, version = version, url = dist_url, - sha256 = sha256, + digest = digest, metadata_sha256 = metadata_sha256, metadata_url = metadata_url, yanked = yanked, ) if filename.endswith(".whl"): - whls[sha256] = dist + whls[digest] = dist else: - sdists[sha256] = dist + sdists[digest] = dist if parse_index: return packages @@ -143,7 +155,7 @@ def parse_simpleapi_html(*, content, parse_index = False): return struct( sdists = sdists, whls = whls, - sha256s_by_version = sha256s_by_version, + hashes_by_version = hashes_by_version, ) def _parse_attrs(attr_string): diff --git a/python/private/pypi/pypi_cache.bzl b/python/private/pypi/pypi_cache.bzl index d3a3034a79..4f6fa9ac37 100644 --- a/python/private/pypi/pypi_cache.bzl +++ b/python/private/pypi/pypi_cache.bzl @@ -13,7 +13,10 @@ load(":version_from_filename.bzl", "version_from_filename") # This value should be changed whenever the storage format changes. # Changing it simply means the information cached in the lockfile has to be # recomputed. -_FACT_VERSION = "v1" +# +# v2: the `dist_hashes` values and the `dist_yanked` keys are canonical +# `:` strings instead of bare sha256 hex digests. +_FACT_VERSION = "v2" def pypi_cache(mctx = None, store = None): """The cache for PyPI index queries. @@ -135,23 +138,23 @@ def _filter_packages(dists, requested_versions): } return result if result else None - sha256s_by_version = {} + hashes_by_version = {} whls = {} sdists = {} - for sha256, d in dists.sdists.items(): + for digest, d in dists.sdists.items(): if d.version not in requested_versions: continue - sdists[sha256] = d - sha256s_by_version.setdefault(d.version, []).append(sha256) + sdists[digest] = d + hashes_by_version.setdefault(d.version, []).append(digest) - for sha256, d in dists.whls.items(): + for digest, d in dists.whls.items(): if d.version not in requested_versions: continue - whls[sha256] = d - sha256s_by_version.setdefault(d.version, []).append(sha256) + whls[digest] = d + hashes_by_version.setdefault(d.version, []).append(digest) if not whls and not sdists: # TODO @aignas 2026-03-08: add logging @@ -161,9 +164,9 @@ def _filter_packages(dists, requested_versions): return struct( whls = whls, sdists = sdists, - sha256s_by_version = { + hashes_by_version = { k: sorted(v) - for k, v in sha256s_by_version.items() + for k, v in hashes_by_version.items() }, ) @@ -232,14 +235,14 @@ def _get_from_facts(facts, known_facts, index_url, requested_versions, facts_ver retrieved_versions = {} - for url, sha256 in known_facts.get("dist_hashes", {}).get(root_url, {}).get(distribution, {}).items(): - filename = known_facts.get("dist_filenames", {}).get(root_url, {}).get(distribution, {}).get(sha256) + for url, digest in known_facts.get("dist_hashes", {}).get(root_url, {}).get(distribution, {}).items(): + filename = known_facts.get("dist_filenames", {}).get(root_url, {}).get(distribution, {}).get(url) if not filename: _, _, filename = url.rpartition("/") version = version_from_filename(filename) if version not in requested_versions: - # TODO @aignas 2026-01-21: do the check by requested shas at some point + # TODO @aignas 2026-01-21: do the check by requested digests at some point # We don't have sufficient info in the lock file, need to call the API # continue @@ -251,16 +254,16 @@ def _get_from_facts(facts, known_facts, index_url, requested_versions, facts_ver else: dists = known_sources.setdefault("sdists", {}) - known_sources.setdefault("sha256s_by_version", {}).setdefault(version, []).append(sha256) + known_sources.setdefault("hashes_by_version", {}).setdefault(version, []).append(digest) - dists.setdefault(sha256, struct( - sha256 = sha256, + dists.setdefault(digest, struct( + digest = digest, filename = filename, version = version, metadata_url = "", metadata_sha256 = "", url = url, - yanked = known_facts.get("dist_yanked", {}).get(root_url, {}).get(distribution, {}).get(sha256), + yanked = known_facts.get("dist_yanked", {}).get(root_url, {}).get(distribution, {}).get(digest), )) if not known_sources: @@ -275,9 +278,9 @@ def _get_from_facts(facts, known_facts, index_url, requested_versions, facts_ver output = struct( whls = known_sources.get("whls", {}), sdists = known_sources.get("sdists", {}), - sha256s_by_version = { + hashes_by_version = { k: sorted(v) - for k, v in known_sources.get("sha256s_by_version", {}).items() + for k, v in known_sources.get("hashes_by_version", {}).items() }, ) @@ -318,7 +321,8 @@ def _store_facts(facts, fact_version, index_url, value): # "dist_hashes": { # "": { # "": { - # "": "", + # # An empty string if the index does not advertise any digest. + # "": ":", # }, # }, # }, @@ -332,16 +336,16 @@ def _store_facts(facts, fact_version, index_url, value): # "dist_yanked": { # "": { # "": { - # "": "", # if the package is yanked + # ":": "", # if the package is yanked # }, # }, # }, # }, - for sha256, d in (value.sdists | value.whls).items(): - facts.setdefault("dist_hashes", {}).setdefault(root_url, {}).setdefault(distribution, {}).setdefault(d.url, sha256) + for digest, d in (value.sdists | value.whls).items(): + facts.setdefault("dist_hashes", {}).setdefault(root_url, {}).setdefault(distribution, {}).setdefault(d.url, digest) if not d.url.endswith(d.filename): facts.setdefault("dist_filenames", {}).setdefault(root_url, {}).setdefault(distribution, {}).setdefault(d.url, d.filename) if d.yanked != None: - facts.setdefault("dist_yanked", {}).setdefault(root_url, {}).setdefault(distribution, {}).setdefault(sha256, d.yanked) + facts.setdefault("dist_yanked", {}).setdefault(root_url, {}).setdefault(distribution, {}).setdefault(digest, d.yanked) return value diff --git a/python/private/pypi/simpleapi_download.bzl b/python/private/pypi/simpleapi_download.bzl index 5377a08093..b5f7032b0a 100644 --- a/python/private/pypi/simpleapi_download.bzl +++ b/python/private/pypi/simpleapi_download.bzl @@ -34,7 +34,7 @@ def simpleapi_download( """Download Simple API HTML. First it queries all of the indexes for available packages and then it downloads the contents of - the per-package URLs and sha256 values. This is to enable us to use bazel_downloader with + the per-package URLs and hash digest values. This is to enable us to use bazel_downloader with `requirements.txt` files. As a side effect we also are able to "cross-compile" by fetching the right wheel for the right target platform through the information that we retrieve here. @@ -294,6 +294,6 @@ def _with_index_url(index_url, values): return struct( sdists = values.sdists, whls = values.whls, - sha256s_by_version = values.sha256s_by_version, + hashes_by_version = values.hashes_by_version, index_url = index_url, ) diff --git a/python/private/pypi/whl_library.bzl b/python/private/pypi/whl_library.bzl index 53edd62efb..640a6518a6 100644 --- a/python/private/pypi/whl_library.bzl +++ b/python/private/pypi/whl_library.bzl @@ -419,10 +419,11 @@ def _whl_archive_impl(rctx): url = urls, output = filename, sha256 = rctx.attr.sha256, + integrity = rctx.attr.integrity if not rctx.attr.sha256 else "", auth = get_auth(rctx, urls), ) - if not rctx.attr.sha256: - # this is only seen when there is a direct URL reference without sha256 + if not rctx.attr.sha256 and not rctx.attr.integrity: + # this is only seen when there is a direct URL reference without a hash logger.warn("Please update the requirement line to include the hash:\n{} \\\n --hash=sha256:{}".format( rctx.attr.requirement, result.sha256, @@ -462,10 +463,11 @@ def _pip_archive_impl(rctx): url = urls, output = filename, sha256 = rctx.attr.sha256, + integrity = rctx.attr.integrity if not rctx.attr.sha256 else "", auth = get_auth(rctx, urls), ) - if not rctx.attr.sha256: - # this is only seen when there is a direct URL reference without sha256 + if not rctx.attr.sha256 and not rctx.attr.integrity: + # this is only seen when there is a direct URL reference without a hash logger.warn("Please update the requirement line to include the hash:\n{} \\\n --hash=sha256:{}".format( rctx.attr.requirement, result.sha256, @@ -579,6 +581,19 @@ For example if your whl depends on `numpy` and your Python package repo is named "index_url": attr.string( doc = "The index_url that the package will be downloaded from.", ), + "integrity": attr.string( + doc = """\ +The expected checksum of the downloaded whl in Subresource Integrity format +(e.g. `sha256-...` or `sha512-...`). Only used when `urls` is passed. If +`sha256` is also set, it takes precedence over this attribute. + +:::{versionadded} VERSION_NEXT_FEATURE +::: +""", + ), + "repo": attr.string( + doc = "Pointer to parent repo name. Used to make these rules rerun if the parent repo changes.", + ), "repo_prefix": attr.string( doc = """ Prefix for the generated packages will be of the form `@//...` @@ -681,6 +696,7 @@ whl_archive = repository_rule( "group_deps", "group_name", "index_url", + "integrity", "repo_prefix", "requirement", "sha256", diff --git a/python/private/pypi/whl_repo_name.bzl b/python/private/pypi/whl_repo_name.bzl index 29d774c361..739ff2a4a1 100644 --- a/python/private/pypi/whl_repo_name.bzl +++ b/python/private/pypi/whl_repo_name.bzl @@ -18,25 +18,31 @@ load("//python/private:normalize_name.bzl", "normalize_name") load(":parse_whl_name.bzl", "parse_whl_name") -def whl_repo_name(filename, sha256, *target_platforms): +def whl_repo_name(filename, digest, *target_platforms): """Return a valid whl_library repo name given a distribution filename. Args: filename: {type}`str` the filename of the distribution. - sha256: {type}`str` the sha256 of the distribution. + digest: {type}`str` the digest of the distribution, either as a + canonical `:` string or a bare hex digest. *target_platforms: {type}`list[str]` the extra suffixes to append. Only used when we need to support different extras per version. Returns: a string that can be used in {obj}`whl_library`. """ + + # Strip the `:` prefix so that the name only contains the hex digest + # and stays the same as it has always been for bare sha256 values. + digest = digest.rpartition(":")[2] + parts = [] if not filename.endswith(".whl"): # Then the filename is basically foo-3.2.1. name, _, tail = filename.rpartition("-") parts.append(normalize_name(name)) - if sha256: + if digest: parts.append("sdist") version = "" else: @@ -56,8 +62,8 @@ def whl_repo_name(filename, sha256, *target_platforms): parts.append(abi_tag) parts.append(platform_tag) - if sha256: - parts.append(sha256[:8]) + if digest: + parts.append(digest[:8]) elif version: parts.insert(1, version) diff --git a/tests/integration/bzlmod_lockfile/MODULE.bazel.lock b/tests/integration/bzlmod_lockfile/MODULE.bazel.lock index 5fac194583..686ee0ee40 100644 --- a/tests/integration/bzlmod_lockfile/MODULE.bazel.lock +++ b/tests/integration/bzlmod_lockfile/MODULE.bazel.lock @@ -286,394 +286,394 @@ "dist_hashes": { "https://pypi.org/simple": { "backports-tarfile": { - "https://files.pythonhosted.org/packages/86/72/cd9b395f25e290e633655a100af28cb253e4393396264a98bd5f5951d50f/backports_tarfile-1.2.0.tar.gz": "d75e02c268746e1b8144c278978b6e98e85de6ad16f8e4b0844a154557eca991", - "https://files.pythonhosted.org/packages/b9/fa/123043af240e49752f1c4bd24da5053b6bd00cad78c2be53c0d1e8b975bc/backports.tarfile-1.2.0-py3-none-any.whl": "77e284d754527b01fb1e6fa8a1afe577858ebe4e9dad8919e34c862cb399bc34" + "https://files.pythonhosted.org/packages/86/72/cd9b395f25e290e633655a100af28cb253e4393396264a98bd5f5951d50f/backports_tarfile-1.2.0.tar.gz": "sha256:d75e02c268746e1b8144c278978b6e98e85de6ad16f8e4b0844a154557eca991", + "https://files.pythonhosted.org/packages/b9/fa/123043af240e49752f1c4bd24da5053b6bd00cad78c2be53c0d1e8b975bc/backports.tarfile-1.2.0-py3-none-any.whl": "sha256:77e284d754527b01fb1e6fa8a1afe577858ebe4e9dad8919e34c862cb399bc34" }, "certifi": { - "https://files.pythonhosted.org/packages/0b/a7/71ac2cff56fec219ed242bb11b8efb69fcc4bec75db06fb7bfe35de520e6/certifi-2026.7.22-py3-none-any.whl": "62f22742b58a1a33014a2b6b706588a8d7e2a88ae7bd1a6ebe8c992928483775", - "https://files.pythonhosted.org/packages/a3/c2/24167ea9858356b47a87a50d39908bfdb72ceeefe0041586e704e5376b3a/certifi-2026.7.22.tar.gz": "741e2c3b351ddf169a738da9f2c048608ff7f2c5cc02f1ebc6b118bb090d5d55" + "https://files.pythonhosted.org/packages/0b/a7/71ac2cff56fec219ed242bb11b8efb69fcc4bec75db06fb7bfe35de520e6/certifi-2026.7.22-py3-none-any.whl": "sha256:62f22742b58a1a33014a2b6b706588a8d7e2a88ae7bd1a6ebe8c992928483775", + "https://files.pythonhosted.org/packages/a3/c2/24167ea9858356b47a87a50d39908bfdb72ceeefe0041586e704e5376b3a/certifi-2026.7.22.tar.gz": "sha256:741e2c3b351ddf169a738da9f2c048608ff7f2c5cc02f1ebc6b118bb090d5d55" }, "cffi": { - "https://files.pythonhosted.org/packages/04/8c/b925975448cf20634a9fbd5efceb807219db452653648d2897c0989cab2d/cffi-2.1.0-cp311-cp311-musllinux_1_2_x86_64.whl": "89095c1968b4ba8285840e131bf2891b09ae137fe2146905acae0354fbce1b5e", - "https://files.pythonhosted.org/packages/05/ef/6cd4f8c671517162379dc79cfae5aea9106bc38abb89628d5c16adf6a838/cffi-2.1.0-cp315-cp315-win_arm64.whl": "8d35c139744adb3e727cd51b1a18324bbe44b8bd41bf8322bca4d41289f48eda", - "https://files.pythonhosted.org/packages/0f/6f/ade5ce9863a57992a6ea3d0d10d7e29b8749fc127204b3d493d667b2815f/cffi-2.1.0-cp314-cp314-win32.whl": "1854b724d00f6654c742097d5387569021be12d3a0f770eae1df8f8acfcc6acd", - "https://files.pythonhosted.org/packages/11/b6/12fc55092817a5faa26fb8c40c7f9d662e11a46ee248c137aafc42517d92/cffi-2.1.0-cp315-cp315t-macosx_10_15_x86_64.whl": "f9912624a0c0b834b7520d7769b3644453aabc0a7e1c839da7359f050750e9bc", - "https://files.pythonhosted.org/packages/14/d0/117dcd9209255ad8571fbc8c92ef32593a1d294dcec91ddc4e4db50606f2/cffi-2.1.0-cp314-cp314t-manylinux2014_aarch64.manylinux_2_17_aarch64.whl": "eb4e8997a49aa2c08a3e43c9045d224448b8941d88e7ac163c7d383e560cbf98", - "https://files.pythonhosted.org/packages/14/f0/134c00ce0779ec86dea2aa1aac69339c2741a8045072676763512363a2ea/cffi-2.1.0-cp314-cp314t-macosx_10_15_x86_64.whl": "7ea6b3e2c4250ff1de21c630fe72d0f63eb95c2c32ffbf64a358cf4a8836d714", - "https://files.pythonhosted.org/packages/19/e5/d3cc82a4a0be7902af279c04181ad038449c096734464a5ae1de3e1401bd/cffi-2.1.0-cp315-cp315t-manylinux2014_x86_64.manylinux_2_17_x86_64.whl": "0611e7ebf90573a535ebdc33ae9da222d037853983e13359f580fab781ca017f", - "https://files.pythonhosted.org/packages/1b/dc/5620cf930688be01f2d673804291de757a934c90b946dbdc3d84130c2ea4/cffi-2.1.0-cp312-cp312-manylinux2014_aarch64.manylinux_2_17_aarch64.whl": "b6422532152adf4e59b110cb2808cee7a033800952f5c036b4af047ee43199e7", - "https://files.pythonhosted.org/packages/1e/85/990925db5df586ec90beb97529c853497e7f85ba0234830447faf41c3057/cffi-2.1.0-cp312-cp312-macosx_10_15_x86_64.whl": "df2b82571a1b30f58a87bf4e5a9e78d2b1eff6c6ce8fd3aa3757221f93f0863f", - "https://files.pythonhosted.org/packages/20/71/7c8372d30e42415602ed9f268f7cfd66f1b855fed881ecd168bcb45dbc0b/cffi-2.1.0-cp314-cp314-macosx_10_15_x86_64.whl": "1ff3456eab0d889592d1936d6125bbfbc7ae4d3354a700f8bd80450a66445d4d", - "https://files.pythonhosted.org/packages/22/d7/1a74539db16d8bfd839ff1515948948efbb162e574650fd3d846896eea95/cffi-2.1.0-cp311-cp311-manylinux2014_aarch64.manylinux_2_17_aarch64.whl": "88023dfe18799507b73f1dbb0d14326a17465de1bc9c9c7655c22845e9ddc3a2", - "https://files.pythonhosted.org/packages/22/f0/a2fc43084c0433caf7f461bccc013e28f848d04ee1c5ed7fce71423cf4d9/cffi-2.1.0-cp311-cp311-musllinux_1_2_i686.whl": "7762faa47e8ff7eb80bd261d9a7d8eea2d8baa69de5e95b70c1f338bbe712f02", - "https://files.pythonhosted.org/packages/28/3b/fad54de07260b93ddeef4b96d0131d57ea900675df1d410ae1deee52d7a6/cffi-2.1.0-cp314-cp314t-manylinux2014_s390x.manylinux_2_17_s390x.whl": "33eb1ad83ebe8f313e0df035c406227d55a79456704a863fad9842136af5ad7d", - "https://files.pythonhosted.org/packages/28/ed/c127d3ac36e899c965e3361357c3befacd6578c03f40125183e41c3b219e/cffi-2.1.0-cp314-cp314-manylinux2014_s390x.manylinux_2_17_s390x.whl": "6ca4919c6e4f89aa99c42510b42cf54596892c00b3f9077f6bdd1505e24b9c8d", - "https://files.pythonhosted.org/packages/2c/0e/fac738d73728c6cea2a88a2883dca54892496cbba88a1dc1f2909cb8a6f5/cffi-2.1.0-cp315-cp315-manylinux2014_s390x.manylinux_2_17_s390x.whl": "2b71d409cccee78310ab5dec549aed052aaea483346e282c7b02362596e01bb0", - "https://files.pythonhosted.org/packages/2c/d8/772b8259bf75749adffb1c546828978381fb516f60cf701f6c83daf60c85/cffi-2.1.0-cp315-cp315-win32.whl": "0a42c688d19fca6e095a53c6a6e2295a5b050a8b289f109adab02a9e61a25de6", - "https://files.pythonhosted.org/packages/2e/1a/cc6ae6c2913a03aab8898eee57963cf1035b8df5872ed8b9115fcc7e2be8/cffi-2.1.0-cp314-cp314-win_arm64.whl": "7d28dff1db6764108bc30788d85d61c876beff416d9a49cb9dd7c5a9f34f5804", - "https://files.pythonhosted.org/packages/2e/d2/065fcae1c73979fac8e054462478d0ff8a29c40cdc2ed7ea5676a061df53/cffi-2.1.0-cp314-cp314-manylinux2014_aarch64.manylinux_2_17_aarch64.whl": "276f20fffd7b396e12516ba8edf9509210ac248cbbc5acbc39cd512f9f59ebe6", - "https://files.pythonhosted.org/packages/2f/dd/afa2191fc6d57fedd26e5844a2fe2fcc0bbfa00961bbaa5a41e4921e7cca/cffi-2.1.0-cp315-cp315-win_amd64.whl": "bccbbb5ee76a61f9d99b5bf3846a51d7fca4b6a732fe46f89295610edaf41853", - "https://files.pythonhosted.org/packages/38/37/04f54b8e63a02f3d908332c9effbf8c366167c6f733ed8a3d4f79b7e2a1e/cffi-2.1.0-cp313-cp313-musllinux_1_2_aarch64.whl": "961be50688f7fba2fa65f63712d3b9b341a22311f5253460ce933f52f0de1c8c", - "https://files.pythonhosted.org/packages/38/66/04781a77b411f0bb5b234d62c1814754ab75ebe455ccff1b08e8d7aae98f/cffi-2.1.0-cp310-cp310-manylinux2014_x86_64.manylinux_2_17_x86_64.whl": "4d433a51f1870e43a13b6732f92aaf540ff77c2015097c78556f75a2d6c030e0", - "https://files.pythonhosted.org/packages/3a/a6/e879bb68cc23a2bc9ba8f4b7d8019f0c2694bad2ab6c4a3701d429439f58/cffi-2.1.0-cp313-cp313-manylinux2014_aarch64.manylinux_2_17_aarch64.whl": "ff067a8d8d880e7809e4ac88eb009bb848870115317b306666502ccad30b147f", - "https://files.pythonhosted.org/packages/3b/30/c806937ed5e4c2c7ac30d9d6b76b5dc57ff8b75d83800d9bb11a8253cf2a/cffi-2.1.0-cp310-cp310-manylinux2014_aarch64.manylinux_2_17_aarch64.whl": "a016194dbe13d14ee9556e734b772d8d67b947092b268d757fd4290e3ba2dfc2", - "https://files.pythonhosted.org/packages/41/aa/3c1409cdd26094efacd1c36c66e0a6eb9d4296e4fd4f9901b8b2042f4323/cffi-2.1.0-cp310-cp310-musllinux_1_2_i686.whl": "c5f5df567f6eb216de69be06ce55c8b714090fae02b18a3b40da8163b8c5fa9c", - "https://files.pythonhosted.org/packages/41/de/92b9eeed4ae4a21d6fd9b2a2c8505cbed573299902ea73981cc13f7ff62c/cffi-2.1.0-cp314-cp314-win_amd64.whl": "1b96bfe2c4bd825681b7d311ad6d9b7280a091f43e8f63da5729638083cd3bfb", - "https://files.pythonhosted.org/packages/45/ca/f91641185cdd90c36d317a9dc7f85e88ef8682d8b300977baff5e23c35d8/cffi-2.1.0-cp310-cp310-manylinux2014_s390x.manylinux_2_17_s390x.whl": "19c54ac121cad98450b4896fa9a43ee0180d57bc4bc911a33db6cab1efab6cd3", - "https://files.pythonhosted.org/packages/4b/92/e7bb136ad6b5352603732cf907ef862ca103f20f2031c1735a46300c20c9/cffi-2.1.0-cp312-cp312-macosx_11_0_arm64.whl": "78474632761faa0fb96f30b1c928c84ebcf68713cbb80d15bab09dfe61640fde", - "https://files.pythonhosted.org/packages/4b/a4/77b53abbf7a1e0beb9637edbef2a94d15f9c822f591e85d439ffd91519a6/cffi-2.1.0-cp312-cp312-manylinux2014_ppc64le.manylinux_2_17_ppc64le.whl": "46b1c8db8f6122420f32d02fffb924c2fe9bc772d228c7c711748fff56aabb2b", - "https://files.pythonhosted.org/packages/50/d8/3b86aba791cb610d24e8a3e1b2cd529e71fa15096b04e4d4e360049d4a4c/cffi-2.1.0-cp314-cp314t-macosx_11_0_arm64.whl": "6af371f3767faeffc6ac1ef57cdfd25844403e9d3f476c5537caee499de96376", - "https://files.pythonhosted.org/packages/55/c7/8c8c50cb11c6750051daf12164098a9a6f027ac4356967fd4d800a07f242/cffi-2.1.0-cp315-cp315-ios_13_0_arm64_iphoneos.whl": "2e9dabb9abcb7ad15938c7196ad5c1718a4e6d33cc79b4c0209bdb64c4a54a5c", - "https://files.pythonhosted.org/packages/57/5f/ff100cae70ebe9d8df1c01a00e510e45d9adb5c1fdda84791b199141de97/cffi-2.1.0.tar.gz": "efc1cdd798b1aaf39b4610bba7aad28c9bea9b910f25c784ccf9ec1fa719d1f9", - "https://files.pythonhosted.org/packages/58/0c/f528df19cc94b675087324d4760d9e6d5bfae97d6217aa4fac43de4f5fcc/cffi-2.1.0-cp312-cp312-manylinux2014_s390x.manylinux_2_17_s390x.whl": "d9fafc5aa2e2a39aaf7f8cc0c1f044a9b07fca12e558dca53a3cc5c654ad67a7", - "https://files.pythonhosted.org/packages/58/85/7ae00d5c8dd6266f4e944c3db630f3c5c9a98b61d469c714d848b1d8138a/cffi-2.1.0-cp313-cp313-ios_13_0_arm64_iphonesimulator.whl": "a95b05f9baf29b91171b3a8bd2020b028835243e7b0ff6bb23e2a3c228518b1b", - "https://files.pythonhosted.org/packages/5a/02/d5e6c43ea85c41bda2a184a3418f195fe7cf602967a8d2b94e085b83deef/cffi-2.1.0-cp315-cp315-musllinux_1_2_x86_64.whl": "af5e2915d41fe6c961694d7bfdc8562942638200f3ce2765dfb8b745cf997629", - "https://files.pythonhosted.org/packages/5a/47/59eb7975cb0e4ef0afa764ea945b29a5bb4537a9f771cb7d6c8a5dd74c95/cffi-2.1.0-cp313-cp313-win_amd64.whl": "8e74a6135550c4748af665b1b1118b6aab33b1fc6a16f9aff630af107c3b4512", - "https://files.pythonhosted.org/packages/5a/67/9e6e09409336d9e515c58367e7cfcf4f89df06ad25252675595a58eb59d5/cffi-2.1.0-cp315-cp315t-manylinux2014_s390x.manylinux_2_17_s390x.whl": "762f99479dcb369f60ab9017ad4ab97a36a1dd7c1ee5a3b15db0f4b8659120cd", - "https://files.pythonhosted.org/packages/5a/af/34fee85c48f8d94efc8597bc09470c9dd274c145f1c12e0fbc6ab6d38d74/cffi-2.1.0-cp313-cp313-win_arm64.whl": "2282cd5e38aa8accd03e99d1256af8411c84cdbee6a89d841b563fdbd1f3e50f", - "https://files.pythonhosted.org/packages/5d/7c/b7379a5704c79eda57ce075869ba70a0368d1c850f803b3c0d078d39dcaf/cffi-2.1.0-cp315-cp315-musllinux_1_2_aarch64.whl": "8f9ec95b8a043d3dfbc74d9abc6f7baf524dd27a8dc160b0a32ff9cdab650c28", - "https://files.pythonhosted.org/packages/62/f2/c9522a81c32132799a1972c39f5c5f8b4c8b9f00488a23feaa6c06f07741/cffi-2.1.0-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.whl": "1e9f50d192a3e525b15a75ab5114e442d83d657b7ec29182a991bc9a88fd3a66", - "https://files.pythonhosted.org/packages/65/68/9f3ef890cf3c6ab97bd531c5677f67613d302165d16f8142b2811782a614/cffi-2.1.0-cp311-cp311-manylinux1_i686.manylinux2014_i686.manylinux_2_17_i686.manylinux_2_5_i686.whl": "30b65779d598c370374fefabf138d456fd6f3216bfa7bedfab1ba82025b0cd93", - "https://files.pythonhosted.org/packages/68/5a/e536c528bc8057496c360c0978559a2dc45653f89dd6151078aa7d8fca1a/cffi-2.1.0-cp315-cp315t-win32.whl": "cb96698e3c7413d906ce83f8ffd245ec1bd94707541f299d0ce4d6b0193e982b", - "https://files.pythonhosted.org/packages/69/aa/24580a278de21fd7322635556334d9b535f1cbc00b0a3919447cdf464c65/cffi-2.1.0-cp310-cp310-macosx_11_0_arm64.whl": "164bff1657b2a74f0b6d54e11c9b375bc97b931f2ca9c43fcf875838da1570dd", - "https://files.pythonhosted.org/packages/6c/d0/47e338384ab6b1004241002fa616301020cea4fc95f283506565d252f276/cffi-2.1.0-cp314-cp314t-musllinux_1_2_aarch64.whl": "c16914df9fb7f500e440e6875fa23ff5e0b31db01fa9c06af98d59a91f0dc2e4", - "https://files.pythonhosted.org/packages/6e/28/bd53988b9833e8f8ad539d26f4c07a6b3f6bcb1e9e02e7ca038250b3428d/cffi-2.1.0-cp312-cp312-musllinux_1_2_aarch64.whl": "98fff996e983a36d3aa2eca83af40c5821202e7e6f32d13ae94e3d2286f10cfe", - "https://files.pythonhosted.org/packages/6f/08/f2e7d62c460faae0926f2d6e423694aa409ced3bc1fe2927a0a6e5f05416/cffi-2.1.0-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.whl": "799416bae98336e400981ff6e532d67d5c709cfb30afb79865a1315f94b0e224", - "https://files.pythonhosted.org/packages/70/25/65bd5b58ea4bfdfc15cde02cb5365f89ef8ab8b2adfb8fe5c4bd4233382f/cffi-2.1.0-cp314-cp314t-musllinux_1_2_x86_64.whl": "5ecbd0499275d57506d397eebe1981cee87b47fcd9ef5c22cab7ed7644a39a94", - "https://files.pythonhosted.org/packages/70/b6/9003c33a3e7d2c1306f5962e646457dcfe5a8cd8fce6bbe02d7af25db783/cffi-2.1.0-cp310-cp310-win32.whl": "9d72af0cf10a76a600a9690078fe31c63b9588c8e86bf9fd353f713c84b5db0f", - "https://files.pythonhosted.org/packages/79/99/0d0fd37f055224085f42bbb2c022d002e17dde4a97972822327b07d84101/cffi-2.1.0-cp312-cp312-musllinux_1_2_x86_64.whl": "379de10ce1ba048b1448599d1b37b24caee16309d1ac98d3982fc997f768700b", - "https://files.pythonhosted.org/packages/7d/95/8de304305cd9204974b0ca051b86d307cafca13aa575a0ef1b44d92c0d8c/cffi-2.1.0-cp314-cp314-ios_13_0_arm64_iphonesimulator.whl": "702c436735fbe99d59ada02a1f65cfc0d31c0ee8b7290912f8fbc5cd1e4b16c3", - "https://files.pythonhosted.org/packages/84/4c/82f132cb4418ee6d953d982b19191e87e2a6372c8a4ce36e50b69d6ade4a/cffi-2.1.0-cp313-cp313-macosx_11_0_arm64.whl": "716ff8ec22f20b4d988b12884086bcef0fc99737043e503f7a3935a6be99b1ea", - "https://files.pythonhosted.org/packages/88/a9/02cae418ec4beb282ace11958d9d4737793439d561fadc7e6d56f2e2b354/cffi-2.1.0-cp310-cp310-manylinux1_i686.manylinux2014_i686.manylinux_2_17_i686.manylinux_2_5_i686.whl": "c941bb58d5a6e1c3892d86e42927ed6c180302f07e6d395d08c416e594b98b46", - "https://files.pythonhosted.org/packages/88/f6/01890cfd63c08f8eb96a8319b0443690197d240a8bd6346048cf7bde9190/cffi-2.1.0-cp313-cp313-manylinux2014_ppc64le.manylinux_2_17_ppc64le.whl": "3b926723c13eba9f81d2ef3820d63aeceec3b2d4639906047bf675cb8a7a500d", - "https://files.pythonhosted.org/packages/8a/26/710688310447531c7a22f857c7f79d9855ec18b03e04494ced723fb37e2f/cffi-2.1.0-cp310-cp310-win_amd64.whl": "fb62edb5bb52cca65fab91a63afa7561607120d26090a7e8fda6fb9f064726da", - "https://files.pythonhosted.org/packages/8b/31/e115c985105dd7ffb32444505f18ceb874bb42d992af05d5dced7ecf1980/cffi-2.1.0-cp315-cp315t-manylinux2014_ppc64le.manylinux_2_17_ppc64le.whl": "3681e031db29958a7502f5c0c9d6bbc4c36cb20f7b104086fa642d1799631ff8", - "https://files.pythonhosted.org/packages/8c/e9/45c3a76ad8d43ad9261f4c95436da61128d3ca545d72b9612c0ab5be0b1c/cffi-2.1.0-cp313-cp313-macosx_10_15_x86_64.whl": "15faec4adfff450819f3aee0e2e02c812de6edb88203aa58807955db2003472a", - "https://files.pythonhosted.org/packages/8d/2e/cdac88979f295fde5daa69622c7d2111e56e7ceb94f211357fbe452339e4/cffi-2.1.0-cp315-cp315t-macosx_11_0_arm64.whl": "df92f2aba50eb4d96718b68ef76f2e57a57b54f2fa62333496d16c6d585a85ca", - "https://files.pythonhosted.org/packages/96/88/a996879e2eeccb815f6e3a5967b12a308257412acec882039d386bd2aa7b/cffi-2.1.0-cp313-cp313-ios_13_0_arm64_iphoneos.whl": "10537b1df4967ca26d21e5072d7d54188354483b91dc75058968d3f0cf13fbda", - "https://files.pythonhosted.org/packages/99/e2/67680bf19a6b60d2bb7ff83baefa2a4c3d2d7dc0f3277034b802e1fc504c/cffi-2.1.0-cp315-cp315-ios_13_0_arm64_iphonesimulator.whl": "37f525a7e7e50c017fdebe58b787be310ad59357ae43a053943a6e1a6c526001", - "https://files.pythonhosted.org/packages/9e/4e/e8d7cb5783f1841a3c8fb3a7735838d7484d08ec08c9f984b14cac1ac0e9/cffi-2.1.0-cp311-cp311-win_arm64.whl": "35aaea0c7ee0e58a5cd8c2fd1a48fdf7ece0d2699b7ecdda08194e9ce5dd9b3d", - "https://files.pythonhosted.org/packages/a0/17/1073b53b68c9b5ca6914adf5f8bf55aacc2d3be102418c90700160ea8605/cffi-2.1.0-cp315-cp315t-win_arm64.whl": "cbb7640ce37159548d2147b5b8c241f962143d4c71231431820783f4dc78f210", - "https://files.pythonhosted.org/packages/a0/1c/4ed5a0e5bdca6cbc275556de3328dd1b76fd0c11cc13c88fe66d1d8715f2/cffi-2.1.0-cp313-cp313-manylinux1_i686.manylinux2014_i686.manylinux_2_17_i686.manylinux_2_5_i686.whl": "63960549e4f8dc41e31accb97b975abaecfc44c03e396c093a6436763c2ea7db", - "https://files.pythonhosted.org/packages/a2/a5/d4fe77b589e5e82d43ebc809bf2e6474afe8e48e32ea050b9357645b6471/cffi-2.1.0-cp311-cp311-musllinux_1_2_aarch64.whl": "9d8272c0e483b024e1b9ad029821470ed8ec65631dbd90217469da0e7cd89f1c", - "https://files.pythonhosted.org/packages/a6/cf/2b684132056f438567b61e19d690dd31cd0921ace051e0a458be6074369e/cffi-2.1.0-cp313-cp313-manylinux2014_s390x.manylinux_2_17_s390x.whl": "47ff3a8bfd8cb9da1af7524b965127095055654c177fcfc7578debcb015eecd0", - "https://files.pythonhosted.org/packages/a8/eb/f636456ff21a83fc13c032b58cc5dde061691546ac79efa284b2989b7982/cffi-2.1.0-cp312-cp312-win_amd64.whl": "c97f080ea627e2863524c5af3836e2270b5f5dfff1f104392b959f8df0c5d384", - "https://files.pythonhosted.org/packages/a9/d6/c72eecca433cd3e681c65ed313ab4835d9d4a379704d0f628a6a05f51c2e/cffi-2.1.0-cp313-cp313-musllinux_1_2_x86_64.whl": "bf5c6cf48238b0eb4c086978c492ad1cbc22373fc5b2d7353b3a598ce6db887a", - "https://files.pythonhosted.org/packages/b0/80/c138990aa2a70b1a269f6e06348729836d733d6f970867943f61d367f8cc/cffi-2.1.0-cp312-cp312-win32.whl": "9b8f0f26ca4e7513c534d351eca551947d053fac438f2a04ac96d882909b0d3a", - "https://files.pythonhosted.org/packages/b3/c1/6dbd291ee2ae5a50a034aa057207081f545923bbf15dad4511e985aafff5/cffi-2.1.0-cp314-cp314-musllinux_1_2_x86_64.whl": "dbf7c7a88e2bac086f06d14577332760bdeecc42bdec8ac4077f6260557d9326", - "https://files.pythonhosted.org/packages/b5/9f/d4dc66ca651eb1145a133314cda721abf13cfac3d28c4a0402263ae6ad75/cffi-2.1.0-cp315-cp315t-musllinux_1_2_x86_64.whl": "ba00f661f8ba35d075c937174e27c2c421cec3942fd2e0ea3e66996757c0fdd9", - "https://files.pythonhosted.org/packages/b6/3d/f20f8b886b254e3ad10e15cd4186d3aed49f3e6a35ab37aab9f8f25f7c03/cffi-2.1.0-cp314-cp314t-manylinux2014_ppc64le.manylinux_2_17_ppc64le.whl": "bf01d8c84cbea96b944c73b22182e6c7c432b3475632b8111dbfdc95ddad6e13", - "https://files.pythonhosted.org/packages/b9/26/d00496b22de4d4228f32dde94ad996f350c8aad676d63bcca0743c8dea4d/cffi-2.1.0-cp314-cp314t-win_amd64.whl": "0582a58f3051372229ca8e7f5f589f9e5632678208d8636fea3676711fdf7fe5", - "https://files.pythonhosted.org/packages/b9/65/b434abc97ce7cecc2c640fde160507c0ecc7e21544b483ba3325d2e2ea17/cffi-2.1.0-cp315-cp315t-musllinux_1_2_aarch64.whl": "86cf8755a791f72c85dc287128cc62d4f24d392e3f1e15837245623f4a33cccc", - "https://files.pythonhosted.org/packages/ba/b0/e131a9c41f10607926278453d9596163594fe1c4ebc46efe3b5e5b34eb84/cffi-2.1.0-cp311-cp311-manylinux2014_s390x.manylinux_2_17_s390x.whl": "a5781494d4d400a3f47f8f1da94b324f6e6b440a53387774002890a2a2f4b50f", - "https://files.pythonhosted.org/packages/c0/e9/6d7724983b3d5a0908dbf74f64038ade77c18646ff6636ec7894fd392ce1/cffi-2.1.0-cp310-cp310-macosx_10_15_x86_64.whl": "b65f590ef2a44640f9a05dbb548a429b4ade77913ce683ac8b1480777658a6c0", - "https://files.pythonhosted.org/packages/c3/c0/d1ec30ffb370f748f2fb54425972bfef9871e0132e82fb589c46b6676049/cffi-2.1.0-cp312-cp312-manylinux1_i686.manylinux2014_i686.manylinux_2_17_i686.manylinux_2_5_i686.whl": "5972433ad71a9e46516584ef60a0fda12d9dc459938d1539c3ddecf9bdc1368d", - "https://files.pythonhosted.org/packages/c6/4b/e706f67279140f92939da3475ad610df18bfd52d50f14953a8e5fede71d5/cffi-2.1.0-cp313-cp313-win32.whl": "db3eb7d46527159a878ec3460e9d40615bc25ba337d477db681aea6e4f05c5d2", - "https://files.pythonhosted.org/packages/cc/82/3d5c705acb7abbba9bbd7d79b8e62e0f25b6120eb7ae6ac49f1b721722fe/cffi-2.1.0-cp314-cp314t-manylinux2014_x86_64.manylinux_2_17_x86_64.whl": "ac0f1a2d0cfa7eea3f2aaf006ab6e70e8feeb16b75d65b7e5939982ca2f11056", - "https://files.pythonhosted.org/packages/cc/d7/97d3136f81db489ec8d1d67748c110d6c994268fd7528014aa9f2b085e4e/cffi-2.1.0-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.whl": "d53d10f7da99ae46f7373b9150393e9c5eab9b224909982b43832668de4779f5", - "https://files.pythonhosted.org/packages/d0/9a/bb1d5ed9c3fcae158e9f6391bf309c95d98c2ac37ed56573228471d0af5e/cffi-2.1.0-cp310-cp310-musllinux_1_2_aarch64.whl": "3d7f118b5adbfdfead90c25822690b02bc8074fba949bb7858bec4ebd55adb43", - "https://files.pythonhosted.org/packages/d3/0b/0ffe8b82d3875bced5fa1e7986a7a46b748262a40ab7f60b475eb9fb1bb3/cffi-2.1.0-cp315-cp315t-win_amd64.whl": "f146d154428a2523f9cc7936c02353c2459b8f6cf07d3cd1ee1c0a611109c5d5", - "https://files.pythonhosted.org/packages/d3/27/93195977168ee63aed233a1a0993a2178798654d1f4bddcdd321d6fd3b21/cffi-2.1.0-cp314-cp314-musllinux_1_2_aarch64.whl": "c351efb95e832a853a29361675f33a7ce53de1a109cd73fd47af0712213aa4ce", - "https://files.pythonhosted.org/packages/d3/67/85c89a59ba36a671e79638f44d466749f08179266a57e4f2ffdf92174072/cffi-2.1.0-cp311-cp311-macosx_10_15_x86_64.whl": "02cb7ff33ded4f1532476731f89ede53e2e488a8e6205515a82144246ffa7dcc", - "https://files.pythonhosted.org/packages/d5/dd/0c7dbf815a579ff005008a2d815a55d6bb047c349eef536d9dc53d3f0a8d/cffi-2.1.0-cp314-cp314t-win_arm64.whl": "510aeeeac94811b138077451da1fb18b308a5feab47dd2b603af55804155e1c8", - "https://files.pythonhosted.org/packages/d6/5c/584e626835f0375c928176c04137c96927165cb8733cdb3150ec04e5ee5e/cffi-2.1.0-cp314-cp314-macosx_11_0_arm64.whl": "c4165821e131d6d4ca444347c2b694e2311bcfa3fe5a861cc72968f28867beac", - "https://files.pythonhosted.org/packages/d8/f0/81478e482afa03f6d18dc8f2afb5edc45b3080853b634b5ed91961be0998/cffi-2.1.0-cp314-cp314-ios_13_0_arm64_iphoneos.whl": "d2117334c3af3bdcb9a88522b844a2bdb5efdc4f71c6c822df55486ae1c3347a", - "https://files.pythonhosted.org/packages/dc/78/aa01ac599a8a4322533d45a1f9bc93b338276d2d59dabbe7c6d92a775c81/cffi-2.1.0-cp314-cp314t-win32.whl": "7d034dcffa09e9a46c93fa3a3be402096cb5354ac6e41ab8e5cc9cd8b642ad76", - "https://files.pythonhosted.org/packages/dd/2c/400ea43e721727dca8a65c4521390e9196757caba4a45643acb2b63271b8/cffi-2.1.0-cp312-cp312-win_arm64.whl": "6d194185eabd279f1c05ebe3504265ddfc5ad2b58d0714f7db9f01da592e9eb6", - "https://files.pythonhosted.org/packages/e0/27/1d0b408497e41a74795af122d7b603c418c5fed0171450f899afd04e594f/cffi-2.1.0-cp315-cp315t-manylinux2014_aarch64.manylinux_2_17_aarch64.whl": "0520e1f4c35f44e209cbbb421b67eec42e6a157f59444dfb6058874ff3610e5d", - "https://files.pythonhosted.org/packages/e5/4b/1f4c36ab273980d7aa75bb126ea4f8971f24a96108acad3a0a084028c57b/cffi-2.1.0-cp315-cp315-macosx_11_0_arm64.whl": "cdf2448aab5f661c9315308ec8b93f4e8a1a67a3c733f8631067a2b67d5913dc", - "https://files.pythonhosted.org/packages/e6/3f/0b04a700dd64f465c93020253a793a82c9b4dff9961f48facd0df945d9b8/cffi-2.1.0-cp315-cp315-manylinux2014_x86_64.manylinux_2_17_x86_64.whl": "7d3538f9c0e50670f4deb93dbb696576e60590369cae2faf7de681e597a8a1f1", - "https://files.pythonhosted.org/packages/ea/dd/e3b0baa2d3d6a857ac72b7efbf18e32e487c9cdafcc13049ad765495b15e/cffi-2.1.0-cp311-cp311-macosx_11_0_arm64.whl": "f5bce581e6b8c235e566a14768a943b172ada3ed73537bb0c0be1edee312d4e7", - "https://files.pythonhosted.org/packages/eb/d8/df4543cc087245044ed02ef3ad8e0a26619d0075ac7a77a12dc81177851b/cffi-2.1.0-cp315-cp315-manylinux2014_ppc64le.manylinux_2_17_ppc64le.whl": "6274dcb2d15cef48daa73ed1be5a40d501d74dccd0cd6db364776d12cb6ba022", - "https://files.pythonhosted.org/packages/eb/da/5c4918a2d61d86fa927d716cb3d8e4626ef8dc8f605a599d32f33897f59a/cffi-2.1.0-cp311-cp311-win32.whl": "64c753a0f87a256020004f37a1c8c02c480e725f910f0b2a0f3f07debd1b2479", - "https://files.pythonhosted.org/packages/ec/d1/9a5b7169499e8e8d8e636de70b97ac7c9447104d2ff1a2cd94790cea5162/cffi-2.1.0-cp311-cp311-manylinux2014_ppc64le.manylinux_2_17_ppc64le.whl": "0a96b74cda968eebbad56d973efe5098974f0a9fb323865bf99ea1fd24e3e64c", - "https://files.pythonhosted.org/packages/ed/a5/e8bbb1ce5b3ac2f53ad6a10bde44318a5a8d99d4f4a000d44a6e39aeb3e4/cffi-2.1.0-cp314-cp314-manylinux2014_ppc64le.manylinux_2_17_ppc64le.whl": "7d5980a3433d4b71a5e120f9dd551403d7824e31e2e67124fe2769c404c06913", - "https://files.pythonhosted.org/packages/ed/da/4bbe583a3b3a5c8c60892124fe17f3fa3656523faf0d3484eae90f091853/cffi-2.1.0-cp315-cp315-macosx_10_15_x86_64.whl": "95f2954c2c9473d892eca6e0409f3568b37ab62a8eedb122461f73cc273476e3", - "https://files.pythonhosted.org/packages/ef/c3/ad299dc38f3583f8d916b299f028af418a9ec98bc695fcbebeae7420691c/cffi-2.1.0-cp315-cp315-manylinux2014_aarch64.manylinux_2_17_aarch64.whl": "90bec57cf82089383bd06a605b3eb8daebf7e5a668520beaf6e327a83a947699", - "https://files.pythonhosted.org/packages/f9/c8/6c2de1d55cf35ef8b92885d5ef280790f0fb9634d87ea1cc315176aecd61/cffi-2.1.0-cp311-cp311-win_amd64.whl": "4f26194e3d95e06501b942642855aed4f953d55e95d7d01b7c4483db3ecff458", - "https://files.pythonhosted.org/packages/f9/cf/398272b8bbfd58aa314fda5a7f1cdbb26d1d78ae324a11211521315dd1f0/cffi-2.1.0-cp310-cp310-manylinux2014_ppc64le.manylinux_2_17_ppc64le.whl": "03e9810d18c646077e501f661b682fbf5dee4676048527ca3cffe66faa9960dd", - "https://files.pythonhosted.org/packages/fa/75/74dfb7c3fc6ebbd408038476bd4c1d7e925c62614e7b9c534ecc34218288/cffi-2.1.0-cp310-cp310-musllinux_1_2_x86_64.whl": "11b3fb55f4f8ad92274ed26705f65d8f91457de71f5380061eb6d125a768fecd", - "https://files.pythonhosted.org/packages/fb/d2/4398416cd699b35167947c6e22aca52c47e69ad5695073c9f1f2c52e04aa/cffi-2.1.0-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.whl": "aa7a1b53a2a4452ada2d1b5dade9960b2522f1e61293a811a077439e39029565" + "https://files.pythonhosted.org/packages/04/8c/b925975448cf20634a9fbd5efceb807219db452653648d2897c0989cab2d/cffi-2.1.0-cp311-cp311-musllinux_1_2_x86_64.whl": "sha256:89095c1968b4ba8285840e131bf2891b09ae137fe2146905acae0354fbce1b5e", + "https://files.pythonhosted.org/packages/05/ef/6cd4f8c671517162379dc79cfae5aea9106bc38abb89628d5c16adf6a838/cffi-2.1.0-cp315-cp315-win_arm64.whl": "sha256:8d35c139744adb3e727cd51b1a18324bbe44b8bd41bf8322bca4d41289f48eda", + "https://files.pythonhosted.org/packages/0f/6f/ade5ce9863a57992a6ea3d0d10d7e29b8749fc127204b3d493d667b2815f/cffi-2.1.0-cp314-cp314-win32.whl": "sha256:1854b724d00f6654c742097d5387569021be12d3a0f770eae1df8f8acfcc6acd", + "https://files.pythonhosted.org/packages/11/b6/12fc55092817a5faa26fb8c40c7f9d662e11a46ee248c137aafc42517d92/cffi-2.1.0-cp315-cp315t-macosx_10_15_x86_64.whl": "sha256:f9912624a0c0b834b7520d7769b3644453aabc0a7e1c839da7359f050750e9bc", + "https://files.pythonhosted.org/packages/14/d0/117dcd9209255ad8571fbc8c92ef32593a1d294dcec91ddc4e4db50606f2/cffi-2.1.0-cp314-cp314t-manylinux2014_aarch64.manylinux_2_17_aarch64.whl": "sha256:eb4e8997a49aa2c08a3e43c9045d224448b8941d88e7ac163c7d383e560cbf98", + "https://files.pythonhosted.org/packages/14/f0/134c00ce0779ec86dea2aa1aac69339c2741a8045072676763512363a2ea/cffi-2.1.0-cp314-cp314t-macosx_10_15_x86_64.whl": "sha256:7ea6b3e2c4250ff1de21c630fe72d0f63eb95c2c32ffbf64a358cf4a8836d714", + "https://files.pythonhosted.org/packages/19/e5/d3cc82a4a0be7902af279c04181ad038449c096734464a5ae1de3e1401bd/cffi-2.1.0-cp315-cp315t-manylinux2014_x86_64.manylinux_2_17_x86_64.whl": "sha256:0611e7ebf90573a535ebdc33ae9da222d037853983e13359f580fab781ca017f", + "https://files.pythonhosted.org/packages/1b/dc/5620cf930688be01f2d673804291de757a934c90b946dbdc3d84130c2ea4/cffi-2.1.0-cp312-cp312-manylinux2014_aarch64.manylinux_2_17_aarch64.whl": "sha256:b6422532152adf4e59b110cb2808cee7a033800952f5c036b4af047ee43199e7", + "https://files.pythonhosted.org/packages/1e/85/990925db5df586ec90beb97529c853497e7f85ba0234830447faf41c3057/cffi-2.1.0-cp312-cp312-macosx_10_15_x86_64.whl": "sha256:df2b82571a1b30f58a87bf4e5a9e78d2b1eff6c6ce8fd3aa3757221f93f0863f", + "https://files.pythonhosted.org/packages/20/71/7c8372d30e42415602ed9f268f7cfd66f1b855fed881ecd168bcb45dbc0b/cffi-2.1.0-cp314-cp314-macosx_10_15_x86_64.whl": "sha256:1ff3456eab0d889592d1936d6125bbfbc7ae4d3354a700f8bd80450a66445d4d", + "https://files.pythonhosted.org/packages/22/d7/1a74539db16d8bfd839ff1515948948efbb162e574650fd3d846896eea95/cffi-2.1.0-cp311-cp311-manylinux2014_aarch64.manylinux_2_17_aarch64.whl": "sha256:88023dfe18799507b73f1dbb0d14326a17465de1bc9c9c7655c22845e9ddc3a2", + "https://files.pythonhosted.org/packages/22/f0/a2fc43084c0433caf7f461bccc013e28f848d04ee1c5ed7fce71423cf4d9/cffi-2.1.0-cp311-cp311-musllinux_1_2_i686.whl": "sha256:7762faa47e8ff7eb80bd261d9a7d8eea2d8baa69de5e95b70c1f338bbe712f02", + "https://files.pythonhosted.org/packages/28/3b/fad54de07260b93ddeef4b96d0131d57ea900675df1d410ae1deee52d7a6/cffi-2.1.0-cp314-cp314t-manylinux2014_s390x.manylinux_2_17_s390x.whl": "sha256:33eb1ad83ebe8f313e0df035c406227d55a79456704a863fad9842136af5ad7d", + "https://files.pythonhosted.org/packages/28/ed/c127d3ac36e899c965e3361357c3befacd6578c03f40125183e41c3b219e/cffi-2.1.0-cp314-cp314-manylinux2014_s390x.manylinux_2_17_s390x.whl": "sha256:6ca4919c6e4f89aa99c42510b42cf54596892c00b3f9077f6bdd1505e24b9c8d", + "https://files.pythonhosted.org/packages/2c/0e/fac738d73728c6cea2a88a2883dca54892496cbba88a1dc1f2909cb8a6f5/cffi-2.1.0-cp315-cp315-manylinux2014_s390x.manylinux_2_17_s390x.whl": "sha256:2b71d409cccee78310ab5dec549aed052aaea483346e282c7b02362596e01bb0", + "https://files.pythonhosted.org/packages/2c/d8/772b8259bf75749adffb1c546828978381fb516f60cf701f6c83daf60c85/cffi-2.1.0-cp315-cp315-win32.whl": "sha256:0a42c688d19fca6e095a53c6a6e2295a5b050a8b289f109adab02a9e61a25de6", + "https://files.pythonhosted.org/packages/2e/1a/cc6ae6c2913a03aab8898eee57963cf1035b8df5872ed8b9115fcc7e2be8/cffi-2.1.0-cp314-cp314-win_arm64.whl": "sha256:7d28dff1db6764108bc30788d85d61c876beff416d9a49cb9dd7c5a9f34f5804", + "https://files.pythonhosted.org/packages/2e/d2/065fcae1c73979fac8e054462478d0ff8a29c40cdc2ed7ea5676a061df53/cffi-2.1.0-cp314-cp314-manylinux2014_aarch64.manylinux_2_17_aarch64.whl": "sha256:276f20fffd7b396e12516ba8edf9509210ac248cbbc5acbc39cd512f9f59ebe6", + "https://files.pythonhosted.org/packages/2f/dd/afa2191fc6d57fedd26e5844a2fe2fcc0bbfa00961bbaa5a41e4921e7cca/cffi-2.1.0-cp315-cp315-win_amd64.whl": "sha256:bccbbb5ee76a61f9d99b5bf3846a51d7fca4b6a732fe46f89295610edaf41853", + "https://files.pythonhosted.org/packages/38/37/04f54b8e63a02f3d908332c9effbf8c366167c6f733ed8a3d4f79b7e2a1e/cffi-2.1.0-cp313-cp313-musllinux_1_2_aarch64.whl": "sha256:961be50688f7fba2fa65f63712d3b9b341a22311f5253460ce933f52f0de1c8c", + "https://files.pythonhosted.org/packages/38/66/04781a77b411f0bb5b234d62c1814754ab75ebe455ccff1b08e8d7aae98f/cffi-2.1.0-cp310-cp310-manylinux2014_x86_64.manylinux_2_17_x86_64.whl": "sha256:4d433a51f1870e43a13b6732f92aaf540ff77c2015097c78556f75a2d6c030e0", + "https://files.pythonhosted.org/packages/3a/a6/e879bb68cc23a2bc9ba8f4b7d8019f0c2694bad2ab6c4a3701d429439f58/cffi-2.1.0-cp313-cp313-manylinux2014_aarch64.manylinux_2_17_aarch64.whl": "sha256:ff067a8d8d880e7809e4ac88eb009bb848870115317b306666502ccad30b147f", + "https://files.pythonhosted.org/packages/3b/30/c806937ed5e4c2c7ac30d9d6b76b5dc57ff8b75d83800d9bb11a8253cf2a/cffi-2.1.0-cp310-cp310-manylinux2014_aarch64.manylinux_2_17_aarch64.whl": "sha256:a016194dbe13d14ee9556e734b772d8d67b947092b268d757fd4290e3ba2dfc2", + "https://files.pythonhosted.org/packages/41/aa/3c1409cdd26094efacd1c36c66e0a6eb9d4296e4fd4f9901b8b2042f4323/cffi-2.1.0-cp310-cp310-musllinux_1_2_i686.whl": "sha256:c5f5df567f6eb216de69be06ce55c8b714090fae02b18a3b40da8163b8c5fa9c", + "https://files.pythonhosted.org/packages/41/de/92b9eeed4ae4a21d6fd9b2a2c8505cbed573299902ea73981cc13f7ff62c/cffi-2.1.0-cp314-cp314-win_amd64.whl": "sha256:1b96bfe2c4bd825681b7d311ad6d9b7280a091f43e8f63da5729638083cd3bfb", + "https://files.pythonhosted.org/packages/45/ca/f91641185cdd90c36d317a9dc7f85e88ef8682d8b300977baff5e23c35d8/cffi-2.1.0-cp310-cp310-manylinux2014_s390x.manylinux_2_17_s390x.whl": "sha256:19c54ac121cad98450b4896fa9a43ee0180d57bc4bc911a33db6cab1efab6cd3", + "https://files.pythonhosted.org/packages/4b/92/e7bb136ad6b5352603732cf907ef862ca103f20f2031c1735a46300c20c9/cffi-2.1.0-cp312-cp312-macosx_11_0_arm64.whl": "sha256:78474632761faa0fb96f30b1c928c84ebcf68713cbb80d15bab09dfe61640fde", + "https://files.pythonhosted.org/packages/4b/a4/77b53abbf7a1e0beb9637edbef2a94d15f9c822f591e85d439ffd91519a6/cffi-2.1.0-cp312-cp312-manylinux2014_ppc64le.manylinux_2_17_ppc64le.whl": "sha256:46b1c8db8f6122420f32d02fffb924c2fe9bc772d228c7c711748fff56aabb2b", + "https://files.pythonhosted.org/packages/50/d8/3b86aba791cb610d24e8a3e1b2cd529e71fa15096b04e4d4e360049d4a4c/cffi-2.1.0-cp314-cp314t-macosx_11_0_arm64.whl": "sha256:6af371f3767faeffc6ac1ef57cdfd25844403e9d3f476c5537caee499de96376", + "https://files.pythonhosted.org/packages/55/c7/8c8c50cb11c6750051daf12164098a9a6f027ac4356967fd4d800a07f242/cffi-2.1.0-cp315-cp315-ios_13_0_arm64_iphoneos.whl": "sha256:2e9dabb9abcb7ad15938c7196ad5c1718a4e6d33cc79b4c0209bdb64c4a54a5c", + "https://files.pythonhosted.org/packages/57/5f/ff100cae70ebe9d8df1c01a00e510e45d9adb5c1fdda84791b199141de97/cffi-2.1.0.tar.gz": "sha256:efc1cdd798b1aaf39b4610bba7aad28c9bea9b910f25c784ccf9ec1fa719d1f9", + "https://files.pythonhosted.org/packages/58/0c/f528df19cc94b675087324d4760d9e6d5bfae97d6217aa4fac43de4f5fcc/cffi-2.1.0-cp312-cp312-manylinux2014_s390x.manylinux_2_17_s390x.whl": "sha256:d9fafc5aa2e2a39aaf7f8cc0c1f044a9b07fca12e558dca53a3cc5c654ad67a7", + "https://files.pythonhosted.org/packages/58/85/7ae00d5c8dd6266f4e944c3db630f3c5c9a98b61d469c714d848b1d8138a/cffi-2.1.0-cp313-cp313-ios_13_0_arm64_iphonesimulator.whl": "sha256:a95b05f9baf29b91171b3a8bd2020b028835243e7b0ff6bb23e2a3c228518b1b", + "https://files.pythonhosted.org/packages/5a/02/d5e6c43ea85c41bda2a184a3418f195fe7cf602967a8d2b94e085b83deef/cffi-2.1.0-cp315-cp315-musllinux_1_2_x86_64.whl": "sha256:af5e2915d41fe6c961694d7bfdc8562942638200f3ce2765dfb8b745cf997629", + "https://files.pythonhosted.org/packages/5a/47/59eb7975cb0e4ef0afa764ea945b29a5bb4537a9f771cb7d6c8a5dd74c95/cffi-2.1.0-cp313-cp313-win_amd64.whl": "sha256:8e74a6135550c4748af665b1b1118b6aab33b1fc6a16f9aff630af107c3b4512", + "https://files.pythonhosted.org/packages/5a/67/9e6e09409336d9e515c58367e7cfcf4f89df06ad25252675595a58eb59d5/cffi-2.1.0-cp315-cp315t-manylinux2014_s390x.manylinux_2_17_s390x.whl": "sha256:762f99479dcb369f60ab9017ad4ab97a36a1dd7c1ee5a3b15db0f4b8659120cd", + "https://files.pythonhosted.org/packages/5a/af/34fee85c48f8d94efc8597bc09470c9dd274c145f1c12e0fbc6ab6d38d74/cffi-2.1.0-cp313-cp313-win_arm64.whl": "sha256:2282cd5e38aa8accd03e99d1256af8411c84cdbee6a89d841b563fdbd1f3e50f", + "https://files.pythonhosted.org/packages/5d/7c/b7379a5704c79eda57ce075869ba70a0368d1c850f803b3c0d078d39dcaf/cffi-2.1.0-cp315-cp315-musllinux_1_2_aarch64.whl": "sha256:8f9ec95b8a043d3dfbc74d9abc6f7baf524dd27a8dc160b0a32ff9cdab650c28", + "https://files.pythonhosted.org/packages/62/f2/c9522a81c32132799a1972c39f5c5f8b4c8b9f00488a23feaa6c06f07741/cffi-2.1.0-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.whl": "sha256:1e9f50d192a3e525b15a75ab5114e442d83d657b7ec29182a991bc9a88fd3a66", + "https://files.pythonhosted.org/packages/65/68/9f3ef890cf3c6ab97bd531c5677f67613d302165d16f8142b2811782a614/cffi-2.1.0-cp311-cp311-manylinux1_i686.manylinux2014_i686.manylinux_2_17_i686.manylinux_2_5_i686.whl": "sha256:30b65779d598c370374fefabf138d456fd6f3216bfa7bedfab1ba82025b0cd93", + "https://files.pythonhosted.org/packages/68/5a/e536c528bc8057496c360c0978559a2dc45653f89dd6151078aa7d8fca1a/cffi-2.1.0-cp315-cp315t-win32.whl": "sha256:cb96698e3c7413d906ce83f8ffd245ec1bd94707541f299d0ce4d6b0193e982b", + "https://files.pythonhosted.org/packages/69/aa/24580a278de21fd7322635556334d9b535f1cbc00b0a3919447cdf464c65/cffi-2.1.0-cp310-cp310-macosx_11_0_arm64.whl": "sha256:164bff1657b2a74f0b6d54e11c9b375bc97b931f2ca9c43fcf875838da1570dd", + "https://files.pythonhosted.org/packages/6c/d0/47e338384ab6b1004241002fa616301020cea4fc95f283506565d252f276/cffi-2.1.0-cp314-cp314t-musllinux_1_2_aarch64.whl": "sha256:c16914df9fb7f500e440e6875fa23ff5e0b31db01fa9c06af98d59a91f0dc2e4", + "https://files.pythonhosted.org/packages/6e/28/bd53988b9833e8f8ad539d26f4c07a6b3f6bcb1e9e02e7ca038250b3428d/cffi-2.1.0-cp312-cp312-musllinux_1_2_aarch64.whl": "sha256:98fff996e983a36d3aa2eca83af40c5821202e7e6f32d13ae94e3d2286f10cfe", + "https://files.pythonhosted.org/packages/6f/08/f2e7d62c460faae0926f2d6e423694aa409ced3bc1fe2927a0a6e5f05416/cffi-2.1.0-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.whl": "sha256:799416bae98336e400981ff6e532d67d5c709cfb30afb79865a1315f94b0e224", + "https://files.pythonhosted.org/packages/70/25/65bd5b58ea4bfdfc15cde02cb5365f89ef8ab8b2adfb8fe5c4bd4233382f/cffi-2.1.0-cp314-cp314t-musllinux_1_2_x86_64.whl": "sha256:5ecbd0499275d57506d397eebe1981cee87b47fcd9ef5c22cab7ed7644a39a94", + "https://files.pythonhosted.org/packages/70/b6/9003c33a3e7d2c1306f5962e646457dcfe5a8cd8fce6bbe02d7af25db783/cffi-2.1.0-cp310-cp310-win32.whl": "sha256:9d72af0cf10a76a600a9690078fe31c63b9588c8e86bf9fd353f713c84b5db0f", + "https://files.pythonhosted.org/packages/79/99/0d0fd37f055224085f42bbb2c022d002e17dde4a97972822327b07d84101/cffi-2.1.0-cp312-cp312-musllinux_1_2_x86_64.whl": "sha256:379de10ce1ba048b1448599d1b37b24caee16309d1ac98d3982fc997f768700b", + "https://files.pythonhosted.org/packages/7d/95/8de304305cd9204974b0ca051b86d307cafca13aa575a0ef1b44d92c0d8c/cffi-2.1.0-cp314-cp314-ios_13_0_arm64_iphonesimulator.whl": "sha256:702c436735fbe99d59ada02a1f65cfc0d31c0ee8b7290912f8fbc5cd1e4b16c3", + "https://files.pythonhosted.org/packages/84/4c/82f132cb4418ee6d953d982b19191e87e2a6372c8a4ce36e50b69d6ade4a/cffi-2.1.0-cp313-cp313-macosx_11_0_arm64.whl": "sha256:716ff8ec22f20b4d988b12884086bcef0fc99737043e503f7a3935a6be99b1ea", + "https://files.pythonhosted.org/packages/88/a9/02cae418ec4beb282ace11958d9d4737793439d561fadc7e6d56f2e2b354/cffi-2.1.0-cp310-cp310-manylinux1_i686.manylinux2014_i686.manylinux_2_17_i686.manylinux_2_5_i686.whl": "sha256:c941bb58d5a6e1c3892d86e42927ed6c180302f07e6d395d08c416e594b98b46", + "https://files.pythonhosted.org/packages/88/f6/01890cfd63c08f8eb96a8319b0443690197d240a8bd6346048cf7bde9190/cffi-2.1.0-cp313-cp313-manylinux2014_ppc64le.manylinux_2_17_ppc64le.whl": "sha256:3b926723c13eba9f81d2ef3820d63aeceec3b2d4639906047bf675cb8a7a500d", + "https://files.pythonhosted.org/packages/8a/26/710688310447531c7a22f857c7f79d9855ec18b03e04494ced723fb37e2f/cffi-2.1.0-cp310-cp310-win_amd64.whl": "sha256:fb62edb5bb52cca65fab91a63afa7561607120d26090a7e8fda6fb9f064726da", + "https://files.pythonhosted.org/packages/8b/31/e115c985105dd7ffb32444505f18ceb874bb42d992af05d5dced7ecf1980/cffi-2.1.0-cp315-cp315t-manylinux2014_ppc64le.manylinux_2_17_ppc64le.whl": "sha256:3681e031db29958a7502f5c0c9d6bbc4c36cb20f7b104086fa642d1799631ff8", + "https://files.pythonhosted.org/packages/8c/e9/45c3a76ad8d43ad9261f4c95436da61128d3ca545d72b9612c0ab5be0b1c/cffi-2.1.0-cp313-cp313-macosx_10_15_x86_64.whl": "sha256:15faec4adfff450819f3aee0e2e02c812de6edb88203aa58807955db2003472a", + "https://files.pythonhosted.org/packages/8d/2e/cdac88979f295fde5daa69622c7d2111e56e7ceb94f211357fbe452339e4/cffi-2.1.0-cp315-cp315t-macosx_11_0_arm64.whl": "sha256:df92f2aba50eb4d96718b68ef76f2e57a57b54f2fa62333496d16c6d585a85ca", + "https://files.pythonhosted.org/packages/96/88/a996879e2eeccb815f6e3a5967b12a308257412acec882039d386bd2aa7b/cffi-2.1.0-cp313-cp313-ios_13_0_arm64_iphoneos.whl": "sha256:10537b1df4967ca26d21e5072d7d54188354483b91dc75058968d3f0cf13fbda", + "https://files.pythonhosted.org/packages/99/e2/67680bf19a6b60d2bb7ff83baefa2a4c3d2d7dc0f3277034b802e1fc504c/cffi-2.1.0-cp315-cp315-ios_13_0_arm64_iphonesimulator.whl": "sha256:37f525a7e7e50c017fdebe58b787be310ad59357ae43a053943a6e1a6c526001", + "https://files.pythonhosted.org/packages/9e/4e/e8d7cb5783f1841a3c8fb3a7735838d7484d08ec08c9f984b14cac1ac0e9/cffi-2.1.0-cp311-cp311-win_arm64.whl": "sha256:35aaea0c7ee0e58a5cd8c2fd1a48fdf7ece0d2699b7ecdda08194e9ce5dd9b3d", + "https://files.pythonhosted.org/packages/a0/17/1073b53b68c9b5ca6914adf5f8bf55aacc2d3be102418c90700160ea8605/cffi-2.1.0-cp315-cp315t-win_arm64.whl": "sha256:cbb7640ce37159548d2147b5b8c241f962143d4c71231431820783f4dc78f210", + "https://files.pythonhosted.org/packages/a0/1c/4ed5a0e5bdca6cbc275556de3328dd1b76fd0c11cc13c88fe66d1d8715f2/cffi-2.1.0-cp313-cp313-manylinux1_i686.manylinux2014_i686.manylinux_2_17_i686.manylinux_2_5_i686.whl": "sha256:63960549e4f8dc41e31accb97b975abaecfc44c03e396c093a6436763c2ea7db", + "https://files.pythonhosted.org/packages/a2/a5/d4fe77b589e5e82d43ebc809bf2e6474afe8e48e32ea050b9357645b6471/cffi-2.1.0-cp311-cp311-musllinux_1_2_aarch64.whl": "sha256:9d8272c0e483b024e1b9ad029821470ed8ec65631dbd90217469da0e7cd89f1c", + "https://files.pythonhosted.org/packages/a6/cf/2b684132056f438567b61e19d690dd31cd0921ace051e0a458be6074369e/cffi-2.1.0-cp313-cp313-manylinux2014_s390x.manylinux_2_17_s390x.whl": "sha256:47ff3a8bfd8cb9da1af7524b965127095055654c177fcfc7578debcb015eecd0", + "https://files.pythonhosted.org/packages/a8/eb/f636456ff21a83fc13c032b58cc5dde061691546ac79efa284b2989b7982/cffi-2.1.0-cp312-cp312-win_amd64.whl": "sha256:c97f080ea627e2863524c5af3836e2270b5f5dfff1f104392b959f8df0c5d384", + "https://files.pythonhosted.org/packages/a9/d6/c72eecca433cd3e681c65ed313ab4835d9d4a379704d0f628a6a05f51c2e/cffi-2.1.0-cp313-cp313-musllinux_1_2_x86_64.whl": "sha256:bf5c6cf48238b0eb4c086978c492ad1cbc22373fc5b2d7353b3a598ce6db887a", + "https://files.pythonhosted.org/packages/b0/80/c138990aa2a70b1a269f6e06348729836d733d6f970867943f61d367f8cc/cffi-2.1.0-cp312-cp312-win32.whl": "sha256:9b8f0f26ca4e7513c534d351eca551947d053fac438f2a04ac96d882909b0d3a", + "https://files.pythonhosted.org/packages/b3/c1/6dbd291ee2ae5a50a034aa057207081f545923bbf15dad4511e985aafff5/cffi-2.1.0-cp314-cp314-musllinux_1_2_x86_64.whl": "sha256:dbf7c7a88e2bac086f06d14577332760bdeecc42bdec8ac4077f6260557d9326", + "https://files.pythonhosted.org/packages/b5/9f/d4dc66ca651eb1145a133314cda721abf13cfac3d28c4a0402263ae6ad75/cffi-2.1.0-cp315-cp315t-musllinux_1_2_x86_64.whl": "sha256:ba00f661f8ba35d075c937174e27c2c421cec3942fd2e0ea3e66996757c0fdd9", + "https://files.pythonhosted.org/packages/b6/3d/f20f8b886b254e3ad10e15cd4186d3aed49f3e6a35ab37aab9f8f25f7c03/cffi-2.1.0-cp314-cp314t-manylinux2014_ppc64le.manylinux_2_17_ppc64le.whl": "sha256:bf01d8c84cbea96b944c73b22182e6c7c432b3475632b8111dbfdc95ddad6e13", + "https://files.pythonhosted.org/packages/b9/26/d00496b22de4d4228f32dde94ad996f350c8aad676d63bcca0743c8dea4d/cffi-2.1.0-cp314-cp314t-win_amd64.whl": "sha256:0582a58f3051372229ca8e7f5f589f9e5632678208d8636fea3676711fdf7fe5", + "https://files.pythonhosted.org/packages/b9/65/b434abc97ce7cecc2c640fde160507c0ecc7e21544b483ba3325d2e2ea17/cffi-2.1.0-cp315-cp315t-musllinux_1_2_aarch64.whl": "sha256:86cf8755a791f72c85dc287128cc62d4f24d392e3f1e15837245623f4a33cccc", + "https://files.pythonhosted.org/packages/ba/b0/e131a9c41f10607926278453d9596163594fe1c4ebc46efe3b5e5b34eb84/cffi-2.1.0-cp311-cp311-manylinux2014_s390x.manylinux_2_17_s390x.whl": "sha256:a5781494d4d400a3f47f8f1da94b324f6e6b440a53387774002890a2a2f4b50f", + "https://files.pythonhosted.org/packages/c0/e9/6d7724983b3d5a0908dbf74f64038ade77c18646ff6636ec7894fd392ce1/cffi-2.1.0-cp310-cp310-macosx_10_15_x86_64.whl": "sha256:b65f590ef2a44640f9a05dbb548a429b4ade77913ce683ac8b1480777658a6c0", + "https://files.pythonhosted.org/packages/c3/c0/d1ec30ffb370f748f2fb54425972bfef9871e0132e82fb589c46b6676049/cffi-2.1.0-cp312-cp312-manylinux1_i686.manylinux2014_i686.manylinux_2_17_i686.manylinux_2_5_i686.whl": "sha256:5972433ad71a9e46516584ef60a0fda12d9dc459938d1539c3ddecf9bdc1368d", + "https://files.pythonhosted.org/packages/c6/4b/e706f67279140f92939da3475ad610df18bfd52d50f14953a8e5fede71d5/cffi-2.1.0-cp313-cp313-win32.whl": "sha256:db3eb7d46527159a878ec3460e9d40615bc25ba337d477db681aea6e4f05c5d2", + "https://files.pythonhosted.org/packages/cc/82/3d5c705acb7abbba9bbd7d79b8e62e0f25b6120eb7ae6ac49f1b721722fe/cffi-2.1.0-cp314-cp314t-manylinux2014_x86_64.manylinux_2_17_x86_64.whl": "sha256:ac0f1a2d0cfa7eea3f2aaf006ab6e70e8feeb16b75d65b7e5939982ca2f11056", + "https://files.pythonhosted.org/packages/cc/d7/97d3136f81db489ec8d1d67748c110d6c994268fd7528014aa9f2b085e4e/cffi-2.1.0-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.whl": "sha256:d53d10f7da99ae46f7373b9150393e9c5eab9b224909982b43832668de4779f5", + "https://files.pythonhosted.org/packages/d0/9a/bb1d5ed9c3fcae158e9f6391bf309c95d98c2ac37ed56573228471d0af5e/cffi-2.1.0-cp310-cp310-musllinux_1_2_aarch64.whl": "sha256:3d7f118b5adbfdfead90c25822690b02bc8074fba949bb7858bec4ebd55adb43", + "https://files.pythonhosted.org/packages/d3/0b/0ffe8b82d3875bced5fa1e7986a7a46b748262a40ab7f60b475eb9fb1bb3/cffi-2.1.0-cp315-cp315t-win_amd64.whl": "sha256:f146d154428a2523f9cc7936c02353c2459b8f6cf07d3cd1ee1c0a611109c5d5", + "https://files.pythonhosted.org/packages/d3/27/93195977168ee63aed233a1a0993a2178798654d1f4bddcdd321d6fd3b21/cffi-2.1.0-cp314-cp314-musllinux_1_2_aarch64.whl": "sha256:c351efb95e832a853a29361675f33a7ce53de1a109cd73fd47af0712213aa4ce", + "https://files.pythonhosted.org/packages/d3/67/85c89a59ba36a671e79638f44d466749f08179266a57e4f2ffdf92174072/cffi-2.1.0-cp311-cp311-macosx_10_15_x86_64.whl": "sha256:02cb7ff33ded4f1532476731f89ede53e2e488a8e6205515a82144246ffa7dcc", + "https://files.pythonhosted.org/packages/d5/dd/0c7dbf815a579ff005008a2d815a55d6bb047c349eef536d9dc53d3f0a8d/cffi-2.1.0-cp314-cp314t-win_arm64.whl": "sha256:510aeeeac94811b138077451da1fb18b308a5feab47dd2b603af55804155e1c8", + "https://files.pythonhosted.org/packages/d6/5c/584e626835f0375c928176c04137c96927165cb8733cdb3150ec04e5ee5e/cffi-2.1.0-cp314-cp314-macosx_11_0_arm64.whl": "sha256:c4165821e131d6d4ca444347c2b694e2311bcfa3fe5a861cc72968f28867beac", + "https://files.pythonhosted.org/packages/d8/f0/81478e482afa03f6d18dc8f2afb5edc45b3080853b634b5ed91961be0998/cffi-2.1.0-cp314-cp314-ios_13_0_arm64_iphoneos.whl": "sha256:d2117334c3af3bdcb9a88522b844a2bdb5efdc4f71c6c822df55486ae1c3347a", + "https://files.pythonhosted.org/packages/dc/78/aa01ac599a8a4322533d45a1f9bc93b338276d2d59dabbe7c6d92a775c81/cffi-2.1.0-cp314-cp314t-win32.whl": "sha256:7d034dcffa09e9a46c93fa3a3be402096cb5354ac6e41ab8e5cc9cd8b642ad76", + "https://files.pythonhosted.org/packages/dd/2c/400ea43e721727dca8a65c4521390e9196757caba4a45643acb2b63271b8/cffi-2.1.0-cp312-cp312-win_arm64.whl": "sha256:6d194185eabd279f1c05ebe3504265ddfc5ad2b58d0714f7db9f01da592e9eb6", + "https://files.pythonhosted.org/packages/e0/27/1d0b408497e41a74795af122d7b603c418c5fed0171450f899afd04e594f/cffi-2.1.0-cp315-cp315t-manylinux2014_aarch64.manylinux_2_17_aarch64.whl": "sha256:0520e1f4c35f44e209cbbb421b67eec42e6a157f59444dfb6058874ff3610e5d", + "https://files.pythonhosted.org/packages/e5/4b/1f4c36ab273980d7aa75bb126ea4f8971f24a96108acad3a0a084028c57b/cffi-2.1.0-cp315-cp315-macosx_11_0_arm64.whl": "sha256:cdf2448aab5f661c9315308ec8b93f4e8a1a67a3c733f8631067a2b67d5913dc", + "https://files.pythonhosted.org/packages/e6/3f/0b04a700dd64f465c93020253a793a82c9b4dff9961f48facd0df945d9b8/cffi-2.1.0-cp315-cp315-manylinux2014_x86_64.manylinux_2_17_x86_64.whl": "sha256:7d3538f9c0e50670f4deb93dbb696576e60590369cae2faf7de681e597a8a1f1", + "https://files.pythonhosted.org/packages/ea/dd/e3b0baa2d3d6a857ac72b7efbf18e32e487c9cdafcc13049ad765495b15e/cffi-2.1.0-cp311-cp311-macosx_11_0_arm64.whl": "sha256:f5bce581e6b8c235e566a14768a943b172ada3ed73537bb0c0be1edee312d4e7", + "https://files.pythonhosted.org/packages/eb/d8/df4543cc087245044ed02ef3ad8e0a26619d0075ac7a77a12dc81177851b/cffi-2.1.0-cp315-cp315-manylinux2014_ppc64le.manylinux_2_17_ppc64le.whl": "sha256:6274dcb2d15cef48daa73ed1be5a40d501d74dccd0cd6db364776d12cb6ba022", + "https://files.pythonhosted.org/packages/eb/da/5c4918a2d61d86fa927d716cb3d8e4626ef8dc8f605a599d32f33897f59a/cffi-2.1.0-cp311-cp311-win32.whl": "sha256:64c753a0f87a256020004f37a1c8c02c480e725f910f0b2a0f3f07debd1b2479", + "https://files.pythonhosted.org/packages/ec/d1/9a5b7169499e8e8d8e636de70b97ac7c9447104d2ff1a2cd94790cea5162/cffi-2.1.0-cp311-cp311-manylinux2014_ppc64le.manylinux_2_17_ppc64le.whl": "sha256:0a96b74cda968eebbad56d973efe5098974f0a9fb323865bf99ea1fd24e3e64c", + "https://files.pythonhosted.org/packages/ed/a5/e8bbb1ce5b3ac2f53ad6a10bde44318a5a8d99d4f4a000d44a6e39aeb3e4/cffi-2.1.0-cp314-cp314-manylinux2014_ppc64le.manylinux_2_17_ppc64le.whl": "sha256:7d5980a3433d4b71a5e120f9dd551403d7824e31e2e67124fe2769c404c06913", + "https://files.pythonhosted.org/packages/ed/da/4bbe583a3b3a5c8c60892124fe17f3fa3656523faf0d3484eae90f091853/cffi-2.1.0-cp315-cp315-macosx_10_15_x86_64.whl": "sha256:95f2954c2c9473d892eca6e0409f3568b37ab62a8eedb122461f73cc273476e3", + "https://files.pythonhosted.org/packages/ef/c3/ad299dc38f3583f8d916b299f028af418a9ec98bc695fcbebeae7420691c/cffi-2.1.0-cp315-cp315-manylinux2014_aarch64.manylinux_2_17_aarch64.whl": "sha256:90bec57cf82089383bd06a605b3eb8daebf7e5a668520beaf6e327a83a947699", + "https://files.pythonhosted.org/packages/f9/c8/6c2de1d55cf35ef8b92885d5ef280790f0fb9634d87ea1cc315176aecd61/cffi-2.1.0-cp311-cp311-win_amd64.whl": "sha256:4f26194e3d95e06501b942642855aed4f953d55e95d7d01b7c4483db3ecff458", + "https://files.pythonhosted.org/packages/f9/cf/398272b8bbfd58aa314fda5a7f1cdbb26d1d78ae324a11211521315dd1f0/cffi-2.1.0-cp310-cp310-manylinux2014_ppc64le.manylinux_2_17_ppc64le.whl": "sha256:03e9810d18c646077e501f661b682fbf5dee4676048527ca3cffe66faa9960dd", + "https://files.pythonhosted.org/packages/fa/75/74dfb7c3fc6ebbd408038476bd4c1d7e925c62614e7b9c534ecc34218288/cffi-2.1.0-cp310-cp310-musllinux_1_2_x86_64.whl": "sha256:11b3fb55f4f8ad92274ed26705f65d8f91457de71f5380061eb6d125a768fecd", + "https://files.pythonhosted.org/packages/fb/d2/4398416cd699b35167947c6e22aca52c47e69ad5695073c9f1f2c52e04aa/cffi-2.1.0-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.whl": "sha256:aa7a1b53a2a4452ada2d1b5dade9960b2522f1e61293a811a077439e39029565" }, "charset-normalizer": { - "https://files.pythonhosted.org/packages/00/5e/17398df3a139985ba9d11ed072531986f408c8fca952835ef1ab1820c02b/charset_normalizer-3.4.9-cp314-cp314t-macosx_10_15_universal2.whl": "609b3ba8fcc0fb5ab7af00719d0fb6ad0cb518e48e7712d12fd68f1327951198", - "https://files.pythonhosted.org/packages/01/c4/4fa4c8b3097a11f3c5f09a35b72ed6855fb1d332469504962ab7bafcc702/charset_normalizer-3.4.9-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl": "5e226f6218febc71f6c1fc2fafb91c226f75bdc1d8fb12d66823716e891608fd", - "https://files.pythonhosted.org/packages/01/da/a44bd7a13d426e69e4894557106cd58669097bfad4a8681123b618fbfc5d/charset_normalizer-3.4.9-cp310-cp310-win_arm64.whl": "375b83ed0aecfce76c16d198fbc21f3b11b337d68662bea0a995046682a11419", - "https://files.pythonhosted.org/packages/0b/e3/85ec501f206fb049259288c1f3506e53876937fb00edb47009348e66756b/charset_normalizer-3.4.9-cp311-cp311-macosx_10_9_universal2.whl": "0e94703ec9684807f20cfb5eed95c70f67f2a8f21ad620146d7b5a13677b93e5", - "https://files.pythonhosted.org/packages/0c/74/2f62c8821b969ea3bd67cc2e6976834f48ca5d12664d2559ebcd9bcfbed7/charset_normalizer-3.4.9-cp311-cp311-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl": "871ff67ea1aad4dfd91736464934d56b32dac49f9fbe16cddba36198a7b3a0db", - "https://files.pythonhosted.org/packages/0c/e7/aaf6da33fc9f4691cda8f7efbc9f69179d3d39ec8a4799baf273ee1d8db0/charset_normalizer-3.4.9-cp310-cp310-musllinux_1_2_armv7l.whl": "65a7ff3f705e57d392f7261b6d0550fe137c3019477431f1c355e0db0a7d3e15", - "https://files.pythonhosted.org/packages/0e/42/6dbc00b8cd16011691203e33570fa42ed5746599a2e878112d16eab403a3/charset_normalizer-3.4.9-cp312-cp312-win32.whl": "78841cccf1af7b40f6f716338d50c0902dbe88d9f800b3c973b7a9a0a693a642", - "https://files.pythonhosted.org/packages/10/e0/47c079dd82d217c807479cd59ffd30af56307ea31c108b75758970459ad3/charset_normalizer-3.4.9-cp314-cp314-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl": "4d1c96a7a18b9690a4d46df09e3e3382406ae3213727cd1019ebade1c4a81917", - "https://files.pythonhosted.org/packages/14/cb/1db8b96547ee3186cd2dd7f2e59dd560a9b80748f3604171f3c153d62811/charset_normalizer-3.4.9-cp39-cp39-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl": "58150c9f9b9a552505912d182ccdf26f6396fb6094816ceebcbb20eecabaed94", - "https://files.pythonhosted.org/packages/17/6d/bff78a4bacc4891bc63ec5bdc6776d8c85e47fab93d0d5f6223068fad0a4/charset_normalizer-3.4.9-cp39-cp39-win32.whl": "93d59d504b230e83c7a843251681959a0b6a9cd76f6e146ce1b8a80eb8739af9", - "https://files.pythonhosted.org/packages/19/79/55c32d06d76ae4feafe053f061f3e3ab70bcf19f4007797ce8c3efda7830/charset_normalizer-3.4.9-cp314-cp314-manylinux_2_31_armv7l.whl": "f7fb7d750cfa0a070d2c24e831fd3481019a60dd317ea2b39acbcebc08b6ed81", - "https://files.pythonhosted.org/packages/1d/85/181c652953eb5276d198f375b1dd641047392050098100a3a02d6534f657/charset_normalizer-3.4.9-cp310-cp310-musllinux_1_2_aarch64.whl": "e9701d0049d92c16703a42771b98d560b95248949f23f8cf7b4eddd201814fb9", - "https://files.pythonhosted.org/packages/20/95/d75e82f8ce9fd323ebf059c16c9aadefb22a1ecde13b7840b35835e4886c/charset_normalizer-3.4.9-cp314-cp314-win_arm64.whl": "40a126142a56b2dfc0aacbad1de8310cbf60da7656db0e6b16eebd48e3e93519", - "https://files.pythonhosted.org/packages/28/e9/9fb6099b868c82a40698a748ae0fbd4f31ccc13844c176a07158ba2abbfd/charset_normalizer-3.4.9-cp39-cp39-win_arm64.whl": "476743fe6dfe14a2da12e3ac79125dc84a3b2cf8094369a47a1529b0cd8549fe", - "https://files.pythonhosted.org/packages/2b/f9/ef4a69ea338ad3c0deceea0f5f7d2380ae8b52132b06d652cb0d2cd86706/charset_normalizer-3.4.9-cp314-cp314-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl": "8a79d9f4d8001473a30c163556b3c3bfebec837495a412dde78b51672f6134f9", - "https://files.pythonhosted.org/packages/33/9a/895095b83e7907abd6d3d99aad3a38ad0d9686cc186cb0c94c24320fe63e/charset_normalizer-3.4.9-cp312-cp312-musllinux_1_2_aarch64.whl": "60f44ade2cf573dad7a277e6f8ca9a51a21dda572b13bd7d8539bb3cd5dbedde", - "https://files.pythonhosted.org/packages/36/31/a276bb2e66243072a3fd06fdcab9cbb61a305b02143d70d2bda21d888fa8/charset_normalizer-3.4.9-cp310-cp310-manylinux_2_31_armv7l.whl": "bcf74c1df76758a395bf0af608c04c82257523f55c9868b334f06270d0f2112b", - "https://files.pythonhosted.org/packages/37/8d/ca39a7559a4797505530d084fd3a49a2c959efbbbff146302fb7be4e3b35/charset_normalizer-3.4.9-cp310-cp310-win_amd64.whl": "8c041122946b7ba21bb32c45b1aa57b1be35527690aeb3c5c234521085632eee", - "https://files.pythonhosted.org/packages/3a/25/45f30093ae27dd7b92a793b61882a38685f993700113ca36e0c9c14965e1/charset_normalizer-3.4.9-cp313-cp313-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl": "a4fbdde9dd4a9ce5fd52c2b3a347bb50cc89483ef783f1cb00d408c13f7a96c0", - "https://files.pythonhosted.org/packages/3d/c6/eee9dca4439b1061f76373f06ea855678cc4a64c1c3c90b50e479edbb8eb/charset_normalizer-3.4.9-cp314-cp314t-win_arm64.whl": "19ac87f93086ce37b86e098888555c4b4bc48102279bae3350098c0ed664b501", - "https://files.pythonhosted.org/packages/3d/ca/ad1d7c7d3077dab873f539d3e1d083c0845a762cb0bafdfbe3ef93add598/charset_normalizer-3.4.9-cp39-cp39-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl": "920079c3f7456fa213e0829ed2073aaa727fd39d889ead5b4f35d0de5460d04f", - "https://files.pythonhosted.org/packages/3d/ef/d96ec496cfea0c21db43b0ad03891308b02388d054cc902cf0e5a1ad6a88/charset_normalizer-3.4.9-cp314-cp314t-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl": "fa36ec09ef71d158186bc79e359ff5fdd6e7996fe8ab638f00d6b93139ba4fcf", - "https://files.pythonhosted.org/packages/42/ab/b9bc2e77d6b44a7e46ef62ec5cac1c9a6ba7b9135a5d560f002696ec9995/charset_normalizer-3.4.9-cp314-cp314-musllinux_1_2_aarch64.whl": "a4cfde78a9f2880208d16a93b795726a3017d5977e08d1e162a7a31322479c41", - "https://files.pythonhosted.org/packages/44/95/80282cce0fae9c3061203d723ee87da996aed79679e65d8935050ee7ca1f/charset_normalizer-3.4.9-cp311-cp311-manylinux_2_31_armv7l.whl": "c0323c9daef75ef2e5083624b4585018a0c9d5e3b40f607eed81a311270b934b", - "https://files.pythonhosted.org/packages/48/18/c8f397329c35e32f6a837e488986f4ae03bd2abebc453b48714991630c2f/charset_normalizer-3.4.9-cp313-cp313-musllinux_1_2_aarch64.whl": "416c229f77e5ea25b3dfd4b582f8d73d7e43c22320302b9ab128a2d3a0b38efe", - "https://files.pythonhosted.org/packages/49/ba/768fa3f36048d81c477a0ce61f813bc1454d80917ccfe550abd9f44f5e24/charset_normalizer-3.4.9-cp314-cp314-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl": "f840ed6d8ecba8255df8c42b87fadeda98ddfc6eeec05e2dc66e26d46dd6f58a", - "https://files.pythonhosted.org/packages/49/fd/a1d26144398c67486422a72bf5812cda22cb4ccfcd95a290fb41ceb4b8e2/charset_normalizer-3.4.9-cp314-cp314-win_amd64.whl": "16b65ea0f2465b6fb52aa22de5eca612aa964ddfec00a912e26f4656cbef890b", - "https://files.pythonhosted.org/packages/4b/4c/5361f9aa7f2cb58d94f2ab831b3d493f69efb1d239654b4744e3c09527cb/charset_normalizer-3.4.9-cp312-cp312-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl": "9104ed0bd76a429d46f9ec0dbc9b08ad1d2dcdf2b00a5a0daa1c145329b35b44", - "https://files.pythonhosted.org/packages/4e/6e/de0229a7ef40f6f9d28a837eebf4ec47bdca5dab4e900c84f22919af636a/charset_normalizer-3.4.9-cp311-cp311-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl": "4773092f8019072343a7447203308b176e10199920eb02d6195e81bbb3274c29", - "https://files.pythonhosted.org/packages/4f/8d/1569f4d0032d6ba2a4fe4591c35bf87868c600c41a71eb5c2e1ffa8464c2/charset_normalizer-3.4.9-cp311-cp311-win_arm64.whl": "1d22856ffbe153a602df38e4a5464f0b748a54002e0d69ac6d2ad0a197cc99ec", - "https://files.pythonhosted.org/packages/50/78/ce342ca4ff30b2eb49fe6d9578df85974f90c67d294113e94efdd9664cbd/charset_normalizer-3.4.9-cp312-cp312-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl": "7b86a2b16095d250c6f58b3d9b2eee6f4147754344f3dab0922f7c9bf7d226c9", - "https://files.pythonhosted.org/packages/52/94/af74dde74a3996bd959c350709bfe50e297823d70a8c1cbd54b838880863/charset_normalizer-3.4.9-cp314-cp314t-manylinux_2_31_armv7l.whl": "f86c6358749bd4fda175388691e3ba8c46e24c5347d0afd20f9b7edfc9faf07d", - "https://files.pythonhosted.org/packages/5e/be/7ee4453d7e88dfbc4104ccd34900b9f2c7c17dac22881865fe0e82424a25/charset_normalizer-3.4.9-cp310-cp310-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl": "b5314963fce9b0b12743891de876e724997864ee22aa496f903f426c7e2fa5b2", - "https://files.pythonhosted.org/packages/5f/c0/6eec7bdabe6cbbcc274ec04596f6d93865751a0541d33d60d1ce179bd372/charset_normalizer-3.4.9-cp39-cp39-musllinux_1_2_armv7l.whl": "ad41ba96094304aa090f5a30cb6e4fb3b3f1c264c523394b4c39bbacc4dc92ba", - "https://files.pythonhosted.org/packages/63/01/f2fb3bd3a73be48b173ee0c6aa8d2497af97d5663a8c4c4b491de4c62f7a/charset_normalizer-3.4.9-cp310-cp310-musllinux_1_2_x86_64.whl": "79580094b00d1789d1f93ea55bc43cb2f611910c72235b7657f3482ddcc1b22d", - "https://files.pythonhosted.org/packages/6a/05/c94d5cd23396289c54c93b02e0273b4dd8921641d9968c4828caf9bbaad9/charset_normalizer-3.4.9-cp39-cp39-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl": "df7276909358e5635ae203673ab7e509ddd224225a8d6b0790bf13eb2bde1cc5", - "https://files.pythonhosted.org/packages/6c/ef/2473d3c4d869155be4af1191111d59c4d5c4e0173026f7e85b176e23bf65/charset_normalizer-3.4.9-cp313-cp313-musllinux_1_2_x86_64.whl": "69b157c5d3292bcd443faca052f3096f637f1e074b98212a933c074ae23dc3b8", - "https://files.pythonhosted.org/packages/6d/46/79847edd07244a4a2d443c6655a7b6ee94203c21539414b059f32713c357/charset_normalizer-3.4.9-cp39-cp39-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl": "3c09a49d6cde137258beb3d551994a2927fd35ad5cf96aed573f61bbd67c5f84", - "https://files.pythonhosted.org/packages/6e/fb/d560d1d1555debbfe7849d9cac6145c1b537709d79576bf22557ed803b82/charset_normalizer-3.4.9-cp313-cp313-win_arm64.whl": "611057cc5d5c0afc743ba8be6bd828c17e0aaa8643f9d0a9b9bb7dea80eb8012", - "https://files.pythonhosted.org/packages/70/4a/ecbd131485c07fcdfad54e28946d513e3da22ef3b4bd854dcafae54ec739/charset_normalizer-3.4.9-cp312-cp312-macosx_10_13_universal2.whl": "45b0cc4e3556cd875e09102988d1ab8356c998b596c9fced84547c8138b487a0", - "https://files.pythonhosted.org/packages/78/ad/98aae8630ac71f16711968e38a5acfecce41b778bf2f0312851020f565a8/charset_normalizer-3.4.9-cp314-cp314t-musllinux_1_2_armv7l.whl": "cd6c3d4b783c556fa00bf540854e42f135e2f256abd29669fcd0da0f2dec79c2", - "https://files.pythonhosted.org/packages/7e/8d/496817fa0944239ecae662dd57ea765cfeaec6a735f9f025d4b7b72e7143/charset_normalizer-3.4.9-cp314-cp314-macosx_10_15_universal2.whl": "0327fcd59a935777d83410750c50600ee9571af2846f71ce40f25b13da1ef380", - "https://files.pythonhosted.org/packages/7f/f4/ffbb83546e1f198ecc70ecd372b65cf2b50f9068b380abd67640f17a8e18/charset_normalizer-3.4.9-cp310-cp310-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl": "16d10d789dd9bcca1173c95af82c58433122564b7bc39385124be735a35cbe99", - "https://files.pythonhosted.org/packages/80/33/6c99c1b3e6b8bf730e1bc809b9a2608f224145069114c479a2e9e1494346/charset_normalizer-3.4.9-cp310-cp310-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl": "c1225416b463483160e4af85d5fc3a9690ccb53fd4b1865a6437825f5ede3209", - "https://files.pythonhosted.org/packages/80/3f/bd97d3d9c613013d07cb7733d299385b41df37f0471310f5a73dc359f0b8/charset_normalizer-3.4.9-cp314-cp314t-win_amd64.whl": "9b8e0f3107e2200b76f6054de99016eac3ee6762713587b36baaa7e4bd2ae177", - "https://files.pythonhosted.org/packages/80/cc/f920afd1a23c58ccd53c1d36085a71893a4737ff5e66e0371efab6809850/charset_normalizer-3.4.9-cp312-cp312-win_amd64.whl": "4b3dac63058cc36820b0dd072f89898604e2d39686fe05321729d00d8ac185a0", - "https://files.pythonhosted.org/packages/83/d5/9096aa3cf532dfad237861544eb47a0f20d5adbf1039760fed8eaae935d9/charset_normalizer-3.4.9-cp311-cp311-win32.whl": "ac351b3b8014eead140e77e9717e2992c6bbe30b63bc3422422eb84865412e3d", - "https://files.pythonhosted.org/packages/83/dc/9b29fa4412b318bf3bfea985c35d67eb55e04b59a7c3f2237168b0e0be6f/charset_normalizer-3.4.9-cp312-cp312-musllinux_1_2_x86_64.whl": "03d07803992c6c7bbc976327f34b18b6160327fc81cb82c9d504720ac0be3b62", - "https://files.pythonhosted.org/packages/86/7e/5ce0bba863470fd1902d5e5843968951bddf38abe4742fc97116ef4598b3/charset_normalizer-3.4.9-cp313-cp313-musllinux_1_2_armv7l.whl": "75286256590a6320cf106a0d28970d3560aad9ee09aa7b34fb40524792436d35", - "https://files.pythonhosted.org/packages/87/3a/ad914516df7e358a81aae018caa5e0470ba827fa6d763b1d2e87d920a5f6/charset_normalizer-3.4.9-cp312-cp312-manylinux_2_31_armv7l.whl": "90c44bc373b7687f6948b693cceaea1348ae0975d7474746559494468e3c1d84", - "https://files.pythonhosted.org/packages/8c/e7/5ddfd76fc061eb52de219658a4aa431cbacadf0a0219c8854f00da50d289/charset_normalizer-3.4.9-cp314-cp314-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl": "33bdcc2a32c0a0e861f60841a512c8acc658c87c2ac59d89e3a46dacf7d866e4", - "https://files.pythonhosted.org/packages/92/8f/3a47a3667c83c2df9483d91644c6c107de3bf8874aa1793da9d3012eb986/charset_normalizer-3.4.9-cp313-cp313-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl": "e4fd89cc178bced6ad29cb3e6dd4aa63fa5017c3524dbd0b25998fb64a87cc8b", - "https://files.pythonhosted.org/packages/98/2b/f97f1c193fb855c345d678f5077d6926034db0722df74c8f057020e05a25/charset_normalizer-3.4.9-py3-none-any.whl": "68e5f26a1ad57ded6d1cfb85331d1c1a195314756471d97758c48498bb4dcdf5", - "https://files.pythonhosted.org/packages/9b/f2/c0d4b8508565a36bc5c624e88ed297f5b0b1095011034d7f5b83a69908b5/charset_normalizer-3.4.9-cp314-cp314-win32.whl": "c1c948747b03be832dceed96ca815cef7360de9aa19d37c730f8e3f6101aca48", - "https://files.pythonhosted.org/packages/a1/34/ef5c05f412f42520d7709b7d3784d19640839eb7366ded1755511585429f/charset_normalizer-3.4.9-cp312-cp312-musllinux_1_2_armv7l.whl": "a1786910334ed46ab1dd73222f2cd1e05c2c3bb39f6dddb4f8b36fc382058a39", - "https://files.pythonhosted.org/packages/a2/55/86048bde1c9d0352940bd7b87d825091a52aef67d01cde6c6f7342c5b552/charset_normalizer-3.4.9-cp39-cp39-win_amd64.whl": "ddf4af30b417d9fe16481e9b81c27ab2a7cde1ff7ba3e85653b02db7d145dc7b", - "https://files.pythonhosted.org/packages/a5/34/49b9060e8418b14fb5cba9cf6bfb383111e2538a03a1fb18e66a95aeb3d5/charset_normalizer-3.4.9-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl": "04ce310cb89c15df659582aee80a0603788732a5e017d5bd5c81158106ce249c", - "https://files.pythonhosted.org/packages/a5/ff/c946d63bc3786d5b84d960b0f7ab7e25b828486a946b5aa997625bcaf6a6/charset_normalizer-3.4.9-cp311-cp311-musllinux_1_2_armv7l.whl": "3d92613ec25e43b05f042302531ec0f00b8445190e43325880cbd6ab7c2581da", - "https://files.pythonhosted.org/packages/a6/ec/81e22253f4b7091eca6515bb3da5e45d05a663f7f567bb745695dc60f892/charset_normalizer-3.4.9-cp39-cp39-macosx_10_9_universal2.whl": "253a4a220747e8b5faf57ec320c4f5efb0cef05f647420bf267143ec15dba10a", - "https://files.pythonhosted.org/packages/ad/81/8e983840c6e5b93b33c2ba81aa3d52c2e42f0e9a690ce7607a2e61da4a5c/charset_normalizer-3.4.9-cp310-cp310-macosx_10_9_universal2.whl": "cd6280cf040f233bd7d3407b743b4b4c74f70e8e1c4199cb112a62c941c0772a", - "https://files.pythonhosted.org/packages/af/ba/5e5007c370702f85d2ef75791fac7943ed41e080364a673b20142e430e3e/charset_normalizer-3.4.9-cp311-cp311-musllinux_1_2_x86_64.whl": "280081916dc341820640489a66e4696049401ef1cf6dd672f672e70ad915aca3", - "https://files.pythonhosted.org/packages/b0/3e/faee8f9de92b14ee1198e9163252bb15efee7301b31256a3b6d9ebfdd0dd/charset_normalizer-3.4.9-cp313-cp313-manylinux_2_31_armv7l.whl": "5b10cd92fc5c498b35a8635df6d5a100207f88b63a4dc1de7ef9a548e1e2cd63", - "https://files.pythonhosted.org/packages/b1/27/693ee5e8a18191eb38647360c51cd505013e2bd3b366aa43fd5344c21e3c/charset_normalizer-3.4.9-cp314-cp314t-win32.whl": "0d861473f743244d349b50f850d10eb87aeb22bbdcc8e64f79273c94af5a8226", - "https://files.pythonhosted.org/packages/b2/06/97ec2aeae780b31d742b6352218b43841a6871e2564578ca522dce4a45c3/charset_normalizer-3.4.9-cp313-cp313-macosx_10_13_universal2.whl": "440eede837960000d74978f0eba527be106b5b9aee0daf779d395276ed0b0614", - "https://files.pythonhosted.org/packages/b3/46/03ddc7da576d814fe0a36dd1f0fd3258e95404b4b2e3c026b7923d7e133f/charset_normalizer-3.4.9-cp311-cp311-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl": "304b13570067b2547562e308af560b3963857b1fa90bd6afd978130130fe2d6a", - "https://files.pythonhosted.org/packages/bd/2a/23f34ec9d04624958e137efdc394888716353190e75f25dd22c7a2c7a8aa/charset_normalizer-3.4.9.tar.gz": "673611bbd43f0810bec0b0f028ddeaaa501190339cac411f347ac76917c3ae7b", - "https://files.pythonhosted.org/packages/c3/69/2a5385192e67175f7d8bd5ce4f57c24bc956439adeae5c13a99aa28a53d1/charset_normalizer-3.4.9-cp311-cp311-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl": "2a441ea71902098ffe78c5abe6c494f44160b4af614ed16c3d9a3b1d17fd8ee2", - "https://files.pythonhosted.org/packages/c4/02/c57a22739fe05246b0b5783b3bfb6afaac4eebb46f3ececdfb2f048f780e/charset_normalizer-3.4.9-cp310-cp310-win32.whl": "432786d3561e69aeeae6c7e8648964ce0ad05736120135601f87ac26b9c83381", - "https://files.pythonhosted.org/packages/c8/53/a8c042eb9eee4716f4d42a0f5a571eb32a09ec429be9fb0b8b9d765393ba/charset_normalizer-3.4.9-cp39-cp39-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl": "68ce9f4d6b26d5ccbf7fd4459bf75f74a0a146677ebba80597df60cbdb20e6f4", - "https://files.pythonhosted.org/packages/cb/32/2e64bd2be10e89c61e57ebe6a93fd98ae88eb7ebe414b5121f22c96c69eb/charset_normalizer-3.4.9-cp314-cp314t-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl": "cc1b0fff8ead343dae06305f954eb8468ba0ec1a97881f42489d198e4ce3c632", - "https://files.pythonhosted.org/packages/cd/91/7253a32e86b7e1d1239b1b36ba6dd0f021a21107ab33054b53119cc083b9/charset_normalizer-3.4.9-cp314-cp314t-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl": "51447e9aa2684679af07ca5021c3db526e0284347ebf4ffcec1154c3350cfe32", - "https://files.pythonhosted.org/packages/d0/39/8ff066c672434225f8d25f8b739f992af250944392173dcc88362681c9bf/charset_normalizer-3.4.9-cp313-cp313-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl": "21e764fd1e70b6a3e205a0e46f3051701f98a8cb3fad66eeb80e48bb502f8698", - "https://files.pythonhosted.org/packages/d0/a3/53ddae3db108a088156aa8ddfafd411ebbc1340f48c5573f697b27f69a39/charset_normalizer-3.4.9-cp313-cp313-win32.whl": "51307f5c71007673a2bf8232ad973483d281e74cb99c8c5a990af1eefa6277d9", - "https://files.pythonhosted.org/packages/d4/ce/9af95f7876194bd7a14e3dfe4a4de2e0bff02666a3910d72beafd06cc297/charset_normalizer-3.4.9-cp314-cp314t-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl": "df115d4d83168fdf2cae48ef1ff6d1cb4c466364e30861b37121de0f3bf1b990", - "https://files.pythonhosted.org/packages/d7/74/3c12f9755717dfe5c5c87da63f35d765fa0c00382ec26bf23f7fae34f2ba/charset_normalizer-3.4.9-cp312-cp312-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl": "9cdef90ae47919cae358d8ab15797a800ed41da7aba5d72419fb510729e2ed4b", - "https://files.pythonhosted.org/packages/d9/8d/feabb82cb49fcad14515b1d7d1ca4787b0da7fc723a212bf89bc9e0fac52/charset_normalizer-3.4.9-cp311-cp311-musllinux_1_2_aarch64.whl": "67830fc78e67501f47bb950471b2dcb9b35b140084429318e862895a8e89c993", - "https://files.pythonhosted.org/packages/de/d1/b4319dc3229d8272fba305e206fc0a148e2de8d4087917ce62ae6382f359/charset_normalizer-3.4.9-cp310-cp310-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl": "aa99adc8f081b475a12843953db36831eaf83ec33eb46a90629ca6a5de45a616", - "https://files.pythonhosted.org/packages/e4/56/6c745619ac397e8871e2bcd3cea1eec86b877488f33888b3aef5c3ed506e/charset_normalizer-3.4.9-cp314-cp314t-musllinux_1_2_aarch64.whl": "83aed2c10721ddd90f68140685391b50811a880af20654c59af6b6c66c40513c", - "https://files.pythonhosted.org/packages/e8/5f/b98b8da398637b551e427e7be922bdec19177dc54d6811dcdaa503f23aac/charset_normalizer-3.4.9-cp310-cp310-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl": "9bb41182d93ea91f60b4bc8fbf4c820c69ef8a12ab2d917f3f1834f1acad07e8", - "https://files.pythonhosted.org/packages/e8/ef/6953a77c7cf2c2ff9998e6f575ab3e380119f100223381565a4f94c1f836/charset_normalizer-3.4.9-cp313-cp313-win_amd64.whl": "fe2c7201c642b7c308f1675355ad7ff7b66acfe3541625efe5a3ad38f29d6115", - "https://files.pythonhosted.org/packages/ea/f8/72eb13dcabe7257035cea8aefd922caad2f110d252bf9f67c4c2ca763aee/charset_normalizer-3.4.9-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl": "84fd18bcc17526fc2b3c1af7d2b9217d32c9c04448c16ec693b9b4f1985c3d33", - "https://files.pythonhosted.org/packages/eb/78/59344ff9a4a7b5f6530bf7bec2c980047cc42c3a616596cdbd8cb5c1a1af/charset_normalizer-3.4.9-cp39-cp39-musllinux_1_2_x86_64.whl": "43b9e366a31fdd1c87d0eb08f579b4a82b723ea54338f040d6b4e518a026ea29", - "https://files.pythonhosted.org/packages/ec/96/5d9364e3342d69f3a045e1777bc47c85c383e6e9466d561b33fdb419d1f9/charset_normalizer-3.4.9-cp312-cp312-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl": "9b2aff1c7b3884512b9512c3eaadd9bab39fb45042ffaaa1dd08ff2b9f8109d9", - "https://files.pythonhosted.org/packages/ec/b4/ef5a49b2e77c00deb43bb3256592b115ba9e4346016e82c516b8d215bf68/charset_normalizer-3.4.9-cp39-cp39-manylinux_2_31_armv7l.whl": "231ddcbb35e2ff8973e1365db41fe0572662893b99a05deb183b68ad4c0c8bd4", - "https://files.pythonhosted.org/packages/ed/61/710738687f90d01c06a04ed52d6ca1e62dd9b1d8cc2567098167c4691034/charset_normalizer-3.4.9-cp39-cp39-musllinux_1_2_aarch64.whl": "0fa1aec2d32bcc03c8fa0f6f1712caad1adc38509f31142112e5c9daf5b9c833", - "https://files.pythonhosted.org/packages/ed/a1/e29995109e455dc8eff8d0fac6ae509be39561318a7cfeac5d33ad029213/charset_normalizer-3.4.9-cp311-cp311-win_amd64.whl": "6366a16e1a25018694d6a5d784d09b046edc9eac40ea2b54065c3052672516a1", - "https://files.pythonhosted.org/packages/ee/f0/f1c4fe746c395922961b5916ed1d7d6e7d4c84851d19ed43cc89980ec953/charset_normalizer-3.4.9-cp314-cp314t-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl": "32286a2c8d167e897177b673176c1e3e00d4057caf5d2b64eef9a3666b03018e", - "https://files.pythonhosted.org/packages/f0/e6/0386d43a261ff4e4b30c5857af7df877254b46bec7b9d1b74b6bf969a90b/charset_normalizer-3.4.9-cp312-cp312-win_arm64.whl": "78fa18e436a1a0e58dbd7e02fc4473f3f32cceb12df9dfca542d075961c307d2", - "https://files.pythonhosted.org/packages/f1/60/b22cdbee7e4013dab8b0d7647fc6181120fbbbc8f7025c226d15bd5a47fc/charset_normalizer-3.4.9-cp313-cp313-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl": "bd47ba7fc3ca94896759ea0109775132d3e7ab921fbf54038e1bab2e46c313c9", - "https://files.pythonhosted.org/packages/f1/78/c9c71d599f5aa2d42bcdd35cbbd46d7f535351a57e40ff7d8e5a7e219401/charset_normalizer-3.4.9-cp314-cp314-musllinux_1_2_armv7l.whl": "d4d6fcde76f94f5cb9e43e9e9a61f16dacefd228cbbf6f1a09bd9b219a92f1a1", - "https://files.pythonhosted.org/packages/f4/c4/b3e049d2aa3766180c78507110543d9d50894cc97f57de543f1be521dcdc/charset_normalizer-3.4.9-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl": "c25fe15c70c59eb7c5ce8c06a1f3fa1da0ecc5ea1e7a5922c40fd2fa9b0d5046", - "https://files.pythonhosted.org/packages/f6/39/c914445c321a845097ce4f6ac7de9a18228a77b766272125a1ce00d851eb/charset_normalizer-3.4.9-cp314-cp314-musllinux_1_2_x86_64.whl": "898f0e9068ca27d37f8e83a5b962821df851532e6c4a7d615c1c033f9da6eedf", - "https://files.pythonhosted.org/packages/f7/40/9593d54209765207a7f11073c06494c1721e4ca4a0a426c597679bf7f91e/charset_normalizer-3.4.9-cp314-cp314t-musllinux_1_2_x86_64.whl": "ee2f2a527e3c1a6e6411eb4209642e138b544a2d72fe5d0d76daf77b24063534" + "https://files.pythonhosted.org/packages/00/5e/17398df3a139985ba9d11ed072531986f408c8fca952835ef1ab1820c02b/charset_normalizer-3.4.9-cp314-cp314t-macosx_10_15_universal2.whl": "sha256:609b3ba8fcc0fb5ab7af00719d0fb6ad0cb518e48e7712d12fd68f1327951198", + "https://files.pythonhosted.org/packages/01/c4/4fa4c8b3097a11f3c5f09a35b72ed6855fb1d332469504962ab7bafcc702/charset_normalizer-3.4.9-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl": "sha256:5e226f6218febc71f6c1fc2fafb91c226f75bdc1d8fb12d66823716e891608fd", + "https://files.pythonhosted.org/packages/01/da/a44bd7a13d426e69e4894557106cd58669097bfad4a8681123b618fbfc5d/charset_normalizer-3.4.9-cp310-cp310-win_arm64.whl": "sha256:375b83ed0aecfce76c16d198fbc21f3b11b337d68662bea0a995046682a11419", + "https://files.pythonhosted.org/packages/0b/e3/85ec501f206fb049259288c1f3506e53876937fb00edb47009348e66756b/charset_normalizer-3.4.9-cp311-cp311-macosx_10_9_universal2.whl": "sha256:0e94703ec9684807f20cfb5eed95c70f67f2a8f21ad620146d7b5a13677b93e5", + "https://files.pythonhosted.org/packages/0c/74/2f62c8821b969ea3bd67cc2e6976834f48ca5d12664d2559ebcd9bcfbed7/charset_normalizer-3.4.9-cp311-cp311-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl": "sha256:871ff67ea1aad4dfd91736464934d56b32dac49f9fbe16cddba36198a7b3a0db", + "https://files.pythonhosted.org/packages/0c/e7/aaf6da33fc9f4691cda8f7efbc9f69179d3d39ec8a4799baf273ee1d8db0/charset_normalizer-3.4.9-cp310-cp310-musllinux_1_2_armv7l.whl": "sha256:65a7ff3f705e57d392f7261b6d0550fe137c3019477431f1c355e0db0a7d3e15", + "https://files.pythonhosted.org/packages/0e/42/6dbc00b8cd16011691203e33570fa42ed5746599a2e878112d16eab403a3/charset_normalizer-3.4.9-cp312-cp312-win32.whl": "sha256:78841cccf1af7b40f6f716338d50c0902dbe88d9f800b3c973b7a9a0a693a642", + "https://files.pythonhosted.org/packages/10/e0/47c079dd82d217c807479cd59ffd30af56307ea31c108b75758970459ad3/charset_normalizer-3.4.9-cp314-cp314-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl": "sha256:4d1c96a7a18b9690a4d46df09e3e3382406ae3213727cd1019ebade1c4a81917", + "https://files.pythonhosted.org/packages/14/cb/1db8b96547ee3186cd2dd7f2e59dd560a9b80748f3604171f3c153d62811/charset_normalizer-3.4.9-cp39-cp39-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl": "sha256:58150c9f9b9a552505912d182ccdf26f6396fb6094816ceebcbb20eecabaed94", + "https://files.pythonhosted.org/packages/17/6d/bff78a4bacc4891bc63ec5bdc6776d8c85e47fab93d0d5f6223068fad0a4/charset_normalizer-3.4.9-cp39-cp39-win32.whl": "sha256:93d59d504b230e83c7a843251681959a0b6a9cd76f6e146ce1b8a80eb8739af9", + "https://files.pythonhosted.org/packages/19/79/55c32d06d76ae4feafe053f061f3e3ab70bcf19f4007797ce8c3efda7830/charset_normalizer-3.4.9-cp314-cp314-manylinux_2_31_armv7l.whl": "sha256:f7fb7d750cfa0a070d2c24e831fd3481019a60dd317ea2b39acbcebc08b6ed81", + "https://files.pythonhosted.org/packages/1d/85/181c652953eb5276d198f375b1dd641047392050098100a3a02d6534f657/charset_normalizer-3.4.9-cp310-cp310-musllinux_1_2_aarch64.whl": "sha256:e9701d0049d92c16703a42771b98d560b95248949f23f8cf7b4eddd201814fb9", + "https://files.pythonhosted.org/packages/20/95/d75e82f8ce9fd323ebf059c16c9aadefb22a1ecde13b7840b35835e4886c/charset_normalizer-3.4.9-cp314-cp314-win_arm64.whl": "sha256:40a126142a56b2dfc0aacbad1de8310cbf60da7656db0e6b16eebd48e3e93519", + "https://files.pythonhosted.org/packages/28/e9/9fb6099b868c82a40698a748ae0fbd4f31ccc13844c176a07158ba2abbfd/charset_normalizer-3.4.9-cp39-cp39-win_arm64.whl": "sha256:476743fe6dfe14a2da12e3ac79125dc84a3b2cf8094369a47a1529b0cd8549fe", + "https://files.pythonhosted.org/packages/2b/f9/ef4a69ea338ad3c0deceea0f5f7d2380ae8b52132b06d652cb0d2cd86706/charset_normalizer-3.4.9-cp314-cp314-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl": "sha256:8a79d9f4d8001473a30c163556b3c3bfebec837495a412dde78b51672f6134f9", + "https://files.pythonhosted.org/packages/33/9a/895095b83e7907abd6d3d99aad3a38ad0d9686cc186cb0c94c24320fe63e/charset_normalizer-3.4.9-cp312-cp312-musllinux_1_2_aarch64.whl": "sha256:60f44ade2cf573dad7a277e6f8ca9a51a21dda572b13bd7d8539bb3cd5dbedde", + "https://files.pythonhosted.org/packages/36/31/a276bb2e66243072a3fd06fdcab9cbb61a305b02143d70d2bda21d888fa8/charset_normalizer-3.4.9-cp310-cp310-manylinux_2_31_armv7l.whl": "sha256:bcf74c1df76758a395bf0af608c04c82257523f55c9868b334f06270d0f2112b", + "https://files.pythonhosted.org/packages/37/8d/ca39a7559a4797505530d084fd3a49a2c959efbbbff146302fb7be4e3b35/charset_normalizer-3.4.9-cp310-cp310-win_amd64.whl": "sha256:8c041122946b7ba21bb32c45b1aa57b1be35527690aeb3c5c234521085632eee", + "https://files.pythonhosted.org/packages/3a/25/45f30093ae27dd7b92a793b61882a38685f993700113ca36e0c9c14965e1/charset_normalizer-3.4.9-cp313-cp313-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl": "sha256:a4fbdde9dd4a9ce5fd52c2b3a347bb50cc89483ef783f1cb00d408c13f7a96c0", + "https://files.pythonhosted.org/packages/3d/c6/eee9dca4439b1061f76373f06ea855678cc4a64c1c3c90b50e479edbb8eb/charset_normalizer-3.4.9-cp314-cp314t-win_arm64.whl": "sha256:19ac87f93086ce37b86e098888555c4b4bc48102279bae3350098c0ed664b501", + "https://files.pythonhosted.org/packages/3d/ca/ad1d7c7d3077dab873f539d3e1d083c0845a762cb0bafdfbe3ef93add598/charset_normalizer-3.4.9-cp39-cp39-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl": "sha256:920079c3f7456fa213e0829ed2073aaa727fd39d889ead5b4f35d0de5460d04f", + "https://files.pythonhosted.org/packages/3d/ef/d96ec496cfea0c21db43b0ad03891308b02388d054cc902cf0e5a1ad6a88/charset_normalizer-3.4.9-cp314-cp314t-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl": "sha256:fa36ec09ef71d158186bc79e359ff5fdd6e7996fe8ab638f00d6b93139ba4fcf", + "https://files.pythonhosted.org/packages/42/ab/b9bc2e77d6b44a7e46ef62ec5cac1c9a6ba7b9135a5d560f002696ec9995/charset_normalizer-3.4.9-cp314-cp314-musllinux_1_2_aarch64.whl": "sha256:a4cfde78a9f2880208d16a93b795726a3017d5977e08d1e162a7a31322479c41", + "https://files.pythonhosted.org/packages/44/95/80282cce0fae9c3061203d723ee87da996aed79679e65d8935050ee7ca1f/charset_normalizer-3.4.9-cp311-cp311-manylinux_2_31_armv7l.whl": "sha256:c0323c9daef75ef2e5083624b4585018a0c9d5e3b40f607eed81a311270b934b", + "https://files.pythonhosted.org/packages/48/18/c8f397329c35e32f6a837e488986f4ae03bd2abebc453b48714991630c2f/charset_normalizer-3.4.9-cp313-cp313-musllinux_1_2_aarch64.whl": "sha256:416c229f77e5ea25b3dfd4b582f8d73d7e43c22320302b9ab128a2d3a0b38efe", + "https://files.pythonhosted.org/packages/49/ba/768fa3f36048d81c477a0ce61f813bc1454d80917ccfe550abd9f44f5e24/charset_normalizer-3.4.9-cp314-cp314-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl": "sha256:f840ed6d8ecba8255df8c42b87fadeda98ddfc6eeec05e2dc66e26d46dd6f58a", + "https://files.pythonhosted.org/packages/49/fd/a1d26144398c67486422a72bf5812cda22cb4ccfcd95a290fb41ceb4b8e2/charset_normalizer-3.4.9-cp314-cp314-win_amd64.whl": "sha256:16b65ea0f2465b6fb52aa22de5eca612aa964ddfec00a912e26f4656cbef890b", + "https://files.pythonhosted.org/packages/4b/4c/5361f9aa7f2cb58d94f2ab831b3d493f69efb1d239654b4744e3c09527cb/charset_normalizer-3.4.9-cp312-cp312-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl": "sha256:9104ed0bd76a429d46f9ec0dbc9b08ad1d2dcdf2b00a5a0daa1c145329b35b44", + "https://files.pythonhosted.org/packages/4e/6e/de0229a7ef40f6f9d28a837eebf4ec47bdca5dab4e900c84f22919af636a/charset_normalizer-3.4.9-cp311-cp311-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl": "sha256:4773092f8019072343a7447203308b176e10199920eb02d6195e81bbb3274c29", + "https://files.pythonhosted.org/packages/4f/8d/1569f4d0032d6ba2a4fe4591c35bf87868c600c41a71eb5c2e1ffa8464c2/charset_normalizer-3.4.9-cp311-cp311-win_arm64.whl": "sha256:1d22856ffbe153a602df38e4a5464f0b748a54002e0d69ac6d2ad0a197cc99ec", + "https://files.pythonhosted.org/packages/50/78/ce342ca4ff30b2eb49fe6d9578df85974f90c67d294113e94efdd9664cbd/charset_normalizer-3.4.9-cp312-cp312-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl": "sha256:7b86a2b16095d250c6f58b3d9b2eee6f4147754344f3dab0922f7c9bf7d226c9", + "https://files.pythonhosted.org/packages/52/94/af74dde74a3996bd959c350709bfe50e297823d70a8c1cbd54b838880863/charset_normalizer-3.4.9-cp314-cp314t-manylinux_2_31_armv7l.whl": "sha256:f86c6358749bd4fda175388691e3ba8c46e24c5347d0afd20f9b7edfc9faf07d", + "https://files.pythonhosted.org/packages/5e/be/7ee4453d7e88dfbc4104ccd34900b9f2c7c17dac22881865fe0e82424a25/charset_normalizer-3.4.9-cp310-cp310-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl": "sha256:b5314963fce9b0b12743891de876e724997864ee22aa496f903f426c7e2fa5b2", + "https://files.pythonhosted.org/packages/5f/c0/6eec7bdabe6cbbcc274ec04596f6d93865751a0541d33d60d1ce179bd372/charset_normalizer-3.4.9-cp39-cp39-musllinux_1_2_armv7l.whl": "sha256:ad41ba96094304aa090f5a30cb6e4fb3b3f1c264c523394b4c39bbacc4dc92ba", + "https://files.pythonhosted.org/packages/63/01/f2fb3bd3a73be48b173ee0c6aa8d2497af97d5663a8c4c4b491de4c62f7a/charset_normalizer-3.4.9-cp310-cp310-musllinux_1_2_x86_64.whl": "sha256:79580094b00d1789d1f93ea55bc43cb2f611910c72235b7657f3482ddcc1b22d", + "https://files.pythonhosted.org/packages/6a/05/c94d5cd23396289c54c93b02e0273b4dd8921641d9968c4828caf9bbaad9/charset_normalizer-3.4.9-cp39-cp39-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl": "sha256:df7276909358e5635ae203673ab7e509ddd224225a8d6b0790bf13eb2bde1cc5", + "https://files.pythonhosted.org/packages/6c/ef/2473d3c4d869155be4af1191111d59c4d5c4e0173026f7e85b176e23bf65/charset_normalizer-3.4.9-cp313-cp313-musllinux_1_2_x86_64.whl": "sha256:69b157c5d3292bcd443faca052f3096f637f1e074b98212a933c074ae23dc3b8", + "https://files.pythonhosted.org/packages/6d/46/79847edd07244a4a2d443c6655a7b6ee94203c21539414b059f32713c357/charset_normalizer-3.4.9-cp39-cp39-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl": "sha256:3c09a49d6cde137258beb3d551994a2927fd35ad5cf96aed573f61bbd67c5f84", + "https://files.pythonhosted.org/packages/6e/fb/d560d1d1555debbfe7849d9cac6145c1b537709d79576bf22557ed803b82/charset_normalizer-3.4.9-cp313-cp313-win_arm64.whl": "sha256:611057cc5d5c0afc743ba8be6bd828c17e0aaa8643f9d0a9b9bb7dea80eb8012", + "https://files.pythonhosted.org/packages/70/4a/ecbd131485c07fcdfad54e28946d513e3da22ef3b4bd854dcafae54ec739/charset_normalizer-3.4.9-cp312-cp312-macosx_10_13_universal2.whl": "sha256:45b0cc4e3556cd875e09102988d1ab8356c998b596c9fced84547c8138b487a0", + "https://files.pythonhosted.org/packages/78/ad/98aae8630ac71f16711968e38a5acfecce41b778bf2f0312851020f565a8/charset_normalizer-3.4.9-cp314-cp314t-musllinux_1_2_armv7l.whl": "sha256:cd6c3d4b783c556fa00bf540854e42f135e2f256abd29669fcd0da0f2dec79c2", + "https://files.pythonhosted.org/packages/7e/8d/496817fa0944239ecae662dd57ea765cfeaec6a735f9f025d4b7b72e7143/charset_normalizer-3.4.9-cp314-cp314-macosx_10_15_universal2.whl": "sha256:0327fcd59a935777d83410750c50600ee9571af2846f71ce40f25b13da1ef380", + "https://files.pythonhosted.org/packages/7f/f4/ffbb83546e1f198ecc70ecd372b65cf2b50f9068b380abd67640f17a8e18/charset_normalizer-3.4.9-cp310-cp310-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl": "sha256:16d10d789dd9bcca1173c95af82c58433122564b7bc39385124be735a35cbe99", + "https://files.pythonhosted.org/packages/80/33/6c99c1b3e6b8bf730e1bc809b9a2608f224145069114c479a2e9e1494346/charset_normalizer-3.4.9-cp310-cp310-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl": "sha256:c1225416b463483160e4af85d5fc3a9690ccb53fd4b1865a6437825f5ede3209", + "https://files.pythonhosted.org/packages/80/3f/bd97d3d9c613013d07cb7733d299385b41df37f0471310f5a73dc359f0b8/charset_normalizer-3.4.9-cp314-cp314t-win_amd64.whl": "sha256:9b8e0f3107e2200b76f6054de99016eac3ee6762713587b36baaa7e4bd2ae177", + "https://files.pythonhosted.org/packages/80/cc/f920afd1a23c58ccd53c1d36085a71893a4737ff5e66e0371efab6809850/charset_normalizer-3.4.9-cp312-cp312-win_amd64.whl": "sha256:4b3dac63058cc36820b0dd072f89898604e2d39686fe05321729d00d8ac185a0", + "https://files.pythonhosted.org/packages/83/d5/9096aa3cf532dfad237861544eb47a0f20d5adbf1039760fed8eaae935d9/charset_normalizer-3.4.9-cp311-cp311-win32.whl": "sha256:ac351b3b8014eead140e77e9717e2992c6bbe30b63bc3422422eb84865412e3d", + "https://files.pythonhosted.org/packages/83/dc/9b29fa4412b318bf3bfea985c35d67eb55e04b59a7c3f2237168b0e0be6f/charset_normalizer-3.4.9-cp312-cp312-musllinux_1_2_x86_64.whl": "sha256:03d07803992c6c7bbc976327f34b18b6160327fc81cb82c9d504720ac0be3b62", + "https://files.pythonhosted.org/packages/86/7e/5ce0bba863470fd1902d5e5843968951bddf38abe4742fc97116ef4598b3/charset_normalizer-3.4.9-cp313-cp313-musllinux_1_2_armv7l.whl": "sha256:75286256590a6320cf106a0d28970d3560aad9ee09aa7b34fb40524792436d35", + "https://files.pythonhosted.org/packages/87/3a/ad914516df7e358a81aae018caa5e0470ba827fa6d763b1d2e87d920a5f6/charset_normalizer-3.4.9-cp312-cp312-manylinux_2_31_armv7l.whl": "sha256:90c44bc373b7687f6948b693cceaea1348ae0975d7474746559494468e3c1d84", + "https://files.pythonhosted.org/packages/8c/e7/5ddfd76fc061eb52de219658a4aa431cbacadf0a0219c8854f00da50d289/charset_normalizer-3.4.9-cp314-cp314-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl": "sha256:33bdcc2a32c0a0e861f60841a512c8acc658c87c2ac59d89e3a46dacf7d866e4", + "https://files.pythonhosted.org/packages/92/8f/3a47a3667c83c2df9483d91644c6c107de3bf8874aa1793da9d3012eb986/charset_normalizer-3.4.9-cp313-cp313-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl": "sha256:e4fd89cc178bced6ad29cb3e6dd4aa63fa5017c3524dbd0b25998fb64a87cc8b", + "https://files.pythonhosted.org/packages/98/2b/f97f1c193fb855c345d678f5077d6926034db0722df74c8f057020e05a25/charset_normalizer-3.4.9-py3-none-any.whl": "sha256:68e5f26a1ad57ded6d1cfb85331d1c1a195314756471d97758c48498bb4dcdf5", + "https://files.pythonhosted.org/packages/9b/f2/c0d4b8508565a36bc5c624e88ed297f5b0b1095011034d7f5b83a69908b5/charset_normalizer-3.4.9-cp314-cp314-win32.whl": "sha256:c1c948747b03be832dceed96ca815cef7360de9aa19d37c730f8e3f6101aca48", + "https://files.pythonhosted.org/packages/a1/34/ef5c05f412f42520d7709b7d3784d19640839eb7366ded1755511585429f/charset_normalizer-3.4.9-cp312-cp312-musllinux_1_2_armv7l.whl": "sha256:a1786910334ed46ab1dd73222f2cd1e05c2c3bb39f6dddb4f8b36fc382058a39", + "https://files.pythonhosted.org/packages/a2/55/86048bde1c9d0352940bd7b87d825091a52aef67d01cde6c6f7342c5b552/charset_normalizer-3.4.9-cp39-cp39-win_amd64.whl": "sha256:ddf4af30b417d9fe16481e9b81c27ab2a7cde1ff7ba3e85653b02db7d145dc7b", + "https://files.pythonhosted.org/packages/a5/34/49b9060e8418b14fb5cba9cf6bfb383111e2538a03a1fb18e66a95aeb3d5/charset_normalizer-3.4.9-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl": "sha256:04ce310cb89c15df659582aee80a0603788732a5e017d5bd5c81158106ce249c", + "https://files.pythonhosted.org/packages/a5/ff/c946d63bc3786d5b84d960b0f7ab7e25b828486a946b5aa997625bcaf6a6/charset_normalizer-3.4.9-cp311-cp311-musllinux_1_2_armv7l.whl": "sha256:3d92613ec25e43b05f042302531ec0f00b8445190e43325880cbd6ab7c2581da", + "https://files.pythonhosted.org/packages/a6/ec/81e22253f4b7091eca6515bb3da5e45d05a663f7f567bb745695dc60f892/charset_normalizer-3.4.9-cp39-cp39-macosx_10_9_universal2.whl": "sha256:253a4a220747e8b5faf57ec320c4f5efb0cef05f647420bf267143ec15dba10a", + "https://files.pythonhosted.org/packages/ad/81/8e983840c6e5b93b33c2ba81aa3d52c2e42f0e9a690ce7607a2e61da4a5c/charset_normalizer-3.4.9-cp310-cp310-macosx_10_9_universal2.whl": "sha256:cd6280cf040f233bd7d3407b743b4b4c74f70e8e1c4199cb112a62c941c0772a", + "https://files.pythonhosted.org/packages/af/ba/5e5007c370702f85d2ef75791fac7943ed41e080364a673b20142e430e3e/charset_normalizer-3.4.9-cp311-cp311-musllinux_1_2_x86_64.whl": "sha256:280081916dc341820640489a66e4696049401ef1cf6dd672f672e70ad915aca3", + "https://files.pythonhosted.org/packages/b0/3e/faee8f9de92b14ee1198e9163252bb15efee7301b31256a3b6d9ebfdd0dd/charset_normalizer-3.4.9-cp313-cp313-manylinux_2_31_armv7l.whl": "sha256:5b10cd92fc5c498b35a8635df6d5a100207f88b63a4dc1de7ef9a548e1e2cd63", + "https://files.pythonhosted.org/packages/b1/27/693ee5e8a18191eb38647360c51cd505013e2bd3b366aa43fd5344c21e3c/charset_normalizer-3.4.9-cp314-cp314t-win32.whl": "sha256:0d861473f743244d349b50f850d10eb87aeb22bbdcc8e64f79273c94af5a8226", + "https://files.pythonhosted.org/packages/b2/06/97ec2aeae780b31d742b6352218b43841a6871e2564578ca522dce4a45c3/charset_normalizer-3.4.9-cp313-cp313-macosx_10_13_universal2.whl": "sha256:440eede837960000d74978f0eba527be106b5b9aee0daf779d395276ed0b0614", + "https://files.pythonhosted.org/packages/b3/46/03ddc7da576d814fe0a36dd1f0fd3258e95404b4b2e3c026b7923d7e133f/charset_normalizer-3.4.9-cp311-cp311-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl": "sha256:304b13570067b2547562e308af560b3963857b1fa90bd6afd978130130fe2d6a", + "https://files.pythonhosted.org/packages/bd/2a/23f34ec9d04624958e137efdc394888716353190e75f25dd22c7a2c7a8aa/charset_normalizer-3.4.9.tar.gz": "sha256:673611bbd43f0810bec0b0f028ddeaaa501190339cac411f347ac76917c3ae7b", + "https://files.pythonhosted.org/packages/c3/69/2a5385192e67175f7d8bd5ce4f57c24bc956439adeae5c13a99aa28a53d1/charset_normalizer-3.4.9-cp311-cp311-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl": "sha256:2a441ea71902098ffe78c5abe6c494f44160b4af614ed16c3d9a3b1d17fd8ee2", + "https://files.pythonhosted.org/packages/c4/02/c57a22739fe05246b0b5783b3bfb6afaac4eebb46f3ececdfb2f048f780e/charset_normalizer-3.4.9-cp310-cp310-win32.whl": "sha256:432786d3561e69aeeae6c7e8648964ce0ad05736120135601f87ac26b9c83381", + "https://files.pythonhosted.org/packages/c8/53/a8c042eb9eee4716f4d42a0f5a571eb32a09ec429be9fb0b8b9d765393ba/charset_normalizer-3.4.9-cp39-cp39-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl": "sha256:68ce9f4d6b26d5ccbf7fd4459bf75f74a0a146677ebba80597df60cbdb20e6f4", + "https://files.pythonhosted.org/packages/cb/32/2e64bd2be10e89c61e57ebe6a93fd98ae88eb7ebe414b5121f22c96c69eb/charset_normalizer-3.4.9-cp314-cp314t-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl": "sha256:cc1b0fff8ead343dae06305f954eb8468ba0ec1a97881f42489d198e4ce3c632", + "https://files.pythonhosted.org/packages/cd/91/7253a32e86b7e1d1239b1b36ba6dd0f021a21107ab33054b53119cc083b9/charset_normalizer-3.4.9-cp314-cp314t-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl": "sha256:51447e9aa2684679af07ca5021c3db526e0284347ebf4ffcec1154c3350cfe32", + "https://files.pythonhosted.org/packages/d0/39/8ff066c672434225f8d25f8b739f992af250944392173dcc88362681c9bf/charset_normalizer-3.4.9-cp313-cp313-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl": "sha256:21e764fd1e70b6a3e205a0e46f3051701f98a8cb3fad66eeb80e48bb502f8698", + "https://files.pythonhosted.org/packages/d0/a3/53ddae3db108a088156aa8ddfafd411ebbc1340f48c5573f697b27f69a39/charset_normalizer-3.4.9-cp313-cp313-win32.whl": "sha256:51307f5c71007673a2bf8232ad973483d281e74cb99c8c5a990af1eefa6277d9", + "https://files.pythonhosted.org/packages/d4/ce/9af95f7876194bd7a14e3dfe4a4de2e0bff02666a3910d72beafd06cc297/charset_normalizer-3.4.9-cp314-cp314t-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl": "sha256:df115d4d83168fdf2cae48ef1ff6d1cb4c466364e30861b37121de0f3bf1b990", + "https://files.pythonhosted.org/packages/d7/74/3c12f9755717dfe5c5c87da63f35d765fa0c00382ec26bf23f7fae34f2ba/charset_normalizer-3.4.9-cp312-cp312-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl": "sha256:9cdef90ae47919cae358d8ab15797a800ed41da7aba5d72419fb510729e2ed4b", + "https://files.pythonhosted.org/packages/d9/8d/feabb82cb49fcad14515b1d7d1ca4787b0da7fc723a212bf89bc9e0fac52/charset_normalizer-3.4.9-cp311-cp311-musllinux_1_2_aarch64.whl": "sha256:67830fc78e67501f47bb950471b2dcb9b35b140084429318e862895a8e89c993", + "https://files.pythonhosted.org/packages/de/d1/b4319dc3229d8272fba305e206fc0a148e2de8d4087917ce62ae6382f359/charset_normalizer-3.4.9-cp310-cp310-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl": "sha256:aa99adc8f081b475a12843953db36831eaf83ec33eb46a90629ca6a5de45a616", + "https://files.pythonhosted.org/packages/e4/56/6c745619ac397e8871e2bcd3cea1eec86b877488f33888b3aef5c3ed506e/charset_normalizer-3.4.9-cp314-cp314t-musllinux_1_2_aarch64.whl": "sha256:83aed2c10721ddd90f68140685391b50811a880af20654c59af6b6c66c40513c", + "https://files.pythonhosted.org/packages/e8/5f/b98b8da398637b551e427e7be922bdec19177dc54d6811dcdaa503f23aac/charset_normalizer-3.4.9-cp310-cp310-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl": "sha256:9bb41182d93ea91f60b4bc8fbf4c820c69ef8a12ab2d917f3f1834f1acad07e8", + "https://files.pythonhosted.org/packages/e8/ef/6953a77c7cf2c2ff9998e6f575ab3e380119f100223381565a4f94c1f836/charset_normalizer-3.4.9-cp313-cp313-win_amd64.whl": "sha256:fe2c7201c642b7c308f1675355ad7ff7b66acfe3541625efe5a3ad38f29d6115", + "https://files.pythonhosted.org/packages/ea/f8/72eb13dcabe7257035cea8aefd922caad2f110d252bf9f67c4c2ca763aee/charset_normalizer-3.4.9-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl": "sha256:84fd18bcc17526fc2b3c1af7d2b9217d32c9c04448c16ec693b9b4f1985c3d33", + "https://files.pythonhosted.org/packages/eb/78/59344ff9a4a7b5f6530bf7bec2c980047cc42c3a616596cdbd8cb5c1a1af/charset_normalizer-3.4.9-cp39-cp39-musllinux_1_2_x86_64.whl": "sha256:43b9e366a31fdd1c87d0eb08f579b4a82b723ea54338f040d6b4e518a026ea29", + "https://files.pythonhosted.org/packages/ec/96/5d9364e3342d69f3a045e1777bc47c85c383e6e9466d561b33fdb419d1f9/charset_normalizer-3.4.9-cp312-cp312-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl": "sha256:9b2aff1c7b3884512b9512c3eaadd9bab39fb45042ffaaa1dd08ff2b9f8109d9", + "https://files.pythonhosted.org/packages/ec/b4/ef5a49b2e77c00deb43bb3256592b115ba9e4346016e82c516b8d215bf68/charset_normalizer-3.4.9-cp39-cp39-manylinux_2_31_armv7l.whl": "sha256:231ddcbb35e2ff8973e1365db41fe0572662893b99a05deb183b68ad4c0c8bd4", + "https://files.pythonhosted.org/packages/ed/61/710738687f90d01c06a04ed52d6ca1e62dd9b1d8cc2567098167c4691034/charset_normalizer-3.4.9-cp39-cp39-musllinux_1_2_aarch64.whl": "sha256:0fa1aec2d32bcc03c8fa0f6f1712caad1adc38509f31142112e5c9daf5b9c833", + "https://files.pythonhosted.org/packages/ed/a1/e29995109e455dc8eff8d0fac6ae509be39561318a7cfeac5d33ad029213/charset_normalizer-3.4.9-cp311-cp311-win_amd64.whl": "sha256:6366a16e1a25018694d6a5d784d09b046edc9eac40ea2b54065c3052672516a1", + "https://files.pythonhosted.org/packages/ee/f0/f1c4fe746c395922961b5916ed1d7d6e7d4c84851d19ed43cc89980ec953/charset_normalizer-3.4.9-cp314-cp314t-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl": "sha256:32286a2c8d167e897177b673176c1e3e00d4057caf5d2b64eef9a3666b03018e", + "https://files.pythonhosted.org/packages/f0/e6/0386d43a261ff4e4b30c5857af7df877254b46bec7b9d1b74b6bf969a90b/charset_normalizer-3.4.9-cp312-cp312-win_arm64.whl": "sha256:78fa18e436a1a0e58dbd7e02fc4473f3f32cceb12df9dfca542d075961c307d2", + "https://files.pythonhosted.org/packages/f1/60/b22cdbee7e4013dab8b0d7647fc6181120fbbbc8f7025c226d15bd5a47fc/charset_normalizer-3.4.9-cp313-cp313-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl": "sha256:bd47ba7fc3ca94896759ea0109775132d3e7ab921fbf54038e1bab2e46c313c9", + "https://files.pythonhosted.org/packages/f1/78/c9c71d599f5aa2d42bcdd35cbbd46d7f535351a57e40ff7d8e5a7e219401/charset_normalizer-3.4.9-cp314-cp314-musllinux_1_2_armv7l.whl": "sha256:d4d6fcde76f94f5cb9e43e9e9a61f16dacefd228cbbf6f1a09bd9b219a92f1a1", + "https://files.pythonhosted.org/packages/f4/c4/b3e049d2aa3766180c78507110543d9d50894cc97f57de543f1be521dcdc/charset_normalizer-3.4.9-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl": "sha256:c25fe15c70c59eb7c5ce8c06a1f3fa1da0ecc5ea1e7a5922c40fd2fa9b0d5046", + "https://files.pythonhosted.org/packages/f6/39/c914445c321a845097ce4f6ac7de9a18228a77b766272125a1ce00d851eb/charset_normalizer-3.4.9-cp314-cp314-musllinux_1_2_x86_64.whl": "sha256:898f0e9068ca27d37f8e83a5b962821df851532e6c4a7d615c1c033f9da6eedf", + "https://files.pythonhosted.org/packages/f7/40/9593d54209765207a7f11073c06494c1721e4ca4a0a426c597679bf7f91e/charset_normalizer-3.4.9-cp314-cp314t-musllinux_1_2_x86_64.whl": "sha256:ee2f2a527e3c1a6e6411eb4209642e138b544a2d72fe5d0d76daf77b24063534" }, "cryptography": { - "https://files.pythonhosted.org/packages/09/41/3797cfaf69cae04a13ee78ebd83f0678d9c02b4779d21ce24445326f1a69/cryptography-49.0.0-cp311-abi3-manylinux2014_aarch64.manylinux_2_17_aarch64.whl": "36d1709f992593689b45bda411498d62c6e365f2ca00b84657d4dadd24de16db", - "https://files.pythonhosted.org/packages/11/2d/5e1fb307cb5931881516b464c98774b3f2c36b5d4bb9a2830253cf553cad/cryptography-49.0.0-cp314-cp314t-manylinux_2_34_x86_64.whl": "d8ecde755e2e91bf773fc94e8c9d730cd7f2007004cb492263a794ec3899a1c8", - "https://files.pythonhosted.org/packages/17/50/983e838c7fd0d87fd8c969bcdd328edaf5f756e38df5281637424c155873/cryptography-49.0.0-cp314-cp314t-manylinux_2_31_armv7l.whl": "07cab27cc7b7e0fd28e5e26bb9eeedde5c135c868b46de4a27845abe94af6122", - "https://files.pythonhosted.org/packages/19/2a/5bb823f5bedcf80718cea7fbc95ec5515cca3769633c4b01a32be7f30e7c/cryptography-49.0.0-cp39-abi3-macosx_11_0_arm64.whl": "ec5e529fb80935c94fe7b729f9972b50e351a0e6b50aa294fd5cabb109fcc29a", - "https://files.pythonhosted.org/packages/1f/09/f42b1d190c5ba75f72062a387f8030d1d75f6ab035788f1d9c4b01de6525/cryptography-49.0.0-cp311-abi3-win_amd64.whl": "e5dfc1e64de5677cec922ffa8da89c546d0415bf6efdf081842e5d44c84e1f0e", - "https://files.pythonhosted.org/packages/1f/99/d1c90d6041656cc6ee229dc99cd67fd0cd5aec3c5f7d72fffc27cc750054/cryptography-49.0.0.tar.gz": "f89660a348f4f78a92366240a61404e337586ef7f5909a2fef59ca88ef505493", - "https://files.pythonhosted.org/packages/20/2c/0622f20ff02b2ef32558733443805dc82fd4c275be01b2d19d14676f3a1b/cryptography-49.0.0-cp311-abi3-manylinux_2_28_x86_64.whl": "2afe9051da7ae7bd5905da5a949280c7d2bb75682e188f650a9d0f2756b834c6", - "https://files.pythonhosted.org/packages/24/01/186c825898477d77e2324d5360fefe622ff1d8d1963ec0554e2cada8ec77/cryptography-49.0.0-cp39-abi3-manylinux_2_28_ppc64le.whl": "9e82dcc8e56052715fb18b2429e3bca4823b1629136a2084fc45a9a5cecb9b64", - "https://files.pythonhosted.org/packages/2c/99/2d13299eb3dd27b02dcfaafcc91d6b5cb3329f7cbd6d8f51921acd566c1a/cryptography-49.0.0-cp39-abi3-manylinux2014_x86_64.manylinux_2_17_x86_64.whl": "35b151772baff2c74cba7fa290ceaff4c3b11c0c881eb93eb5dbc05a7cfbba18", - "https://files.pythonhosted.org/packages/3d/df/40577043ca124e17012f408ddddaeb213b856336ac82ddb3bc915f39e29f/cryptography-49.0.0-cp39-abi3-manylinux2014_aarch64.manylinux_2_17_aarch64.whl": "f78ff2c9ed8dc2d036b0f4d640e22522213d047c1b14e61205a7e55c80a494d4", - "https://files.pythonhosted.org/packages/47/f1/1d3eaa243bfc5de4a187b22aa8c048b3e4980bfbe830ac46e6bac2e66947/cryptography-49.0.0-cp314-cp314t-manylinux_2_28_ppc64le.whl": "f37d847238971164fdbc68ade6f6574aecc9c0af714190e2083429ff68f4ce9d", - "https://files.pythonhosted.org/packages/4a/91/01ce7303a4579e6d3a6abef01bd322848e9ea7a219adcabc5048b9033571/cryptography-49.0.0-cp311-abi3-manylinux_2_28_aarch64.whl": "53ecee2e23f7169b6117e99fc8a944e5e50f79e69758a83b52a00cb98ab2b2d2", - "https://files.pythonhosted.org/packages/4c/fe/93ecac273d3738939d023612ad12cca9a3740a5345d69fda04134c43fd96/cryptography-49.0.0-cp314-cp314t-win_amd64.whl": "33cd0565932807baddb67b96dbee92f2c374b5c89dee09fd74079aeb8c8dba61", - "https://files.pythonhosted.org/packages/4f/01/339573cf1023163a400b0b5d16f6d507de413b9f60be6fd1b77feeaf6737/cryptography-49.0.0-cp311-abi3-musllinux_1_2_aarch64.whl": "b87e65d263b3e5d3bb92a57e2a6638e2f31110fa7aa890c7b2dbba42248d0a3f", - "https://files.pythonhosted.org/packages/58/39/2d51306721330c486495853eda1c567880ff036de15a14c4b74f399934af/cryptography-49.0.0-cp314-cp314t-manylinux_2_28_x86_64.whl": "c2bc30226390d60ea19d9f82b19db005fe0452154a23c1c410c12ea801e43561", - "https://files.pythonhosted.org/packages/62/99/a2c95cf8293f07491e9e27c20cc4dcd18176d944e674679adeb1d0173fd6/cryptography-49.0.0-cp311-abi3-manylinux_2_28_ppc64le.whl": "2eda353d8a27bcbcaa4cbed18994a74ab4d19a2ca897db188ea269ab9b71419b", - "https://files.pythonhosted.org/packages/63/d3/4a83af35d65e3fad632c926fad684c193ea4398569ccb0bbbc7fe8f5dc9a/cryptography-49.0.0-pp311-pypy311_pp73-macosx_11_0_arm64.whl": "fc1e275c2f1d97b1a6450b8b0ea3ebfa6e087a611c2b26cb2404d48588abab7b", - "https://files.pythonhosted.org/packages/67/d0/a5fcd3515f0bae49a7b6d0413cc1bdccdcc1fc0047037a0d480642cdc5d6/cryptography-49.0.0-cp39-abi3-manylinux_2_34_ppc64le.whl": "6fc361c34fb6aac015ce19435876635e5c6d21db31998b0920f675f131e043b8", - "https://files.pythonhosted.org/packages/68/28/8a3ad4653662c93fc44dc4e5d8fd374c25c42e07b34bbfbadf49cf57a5a8/cryptography-49.0.0-cp314-cp314t-manylinux2014_x86_64.manylinux_2_17_x86_64.whl": "7abcee80084cda3f7691f3eb1ce480d8df49cec637b429aa35986c1de71738aa", - "https://files.pythonhosted.org/packages/6c/72/3e798c064bc39e471008075d0f9bc9daf77a80879c092e4a8e170c585ed4/cryptography-49.0.0-cp39-abi3-manylinux_2_31_armv7l.whl": "8c25ceb16df5b9435f3f6a9829204985b0e0cbee3b48aacd432c7d2c850b44d9", - "https://files.pythonhosted.org/packages/6c/a0/db537264e234f7273a73ec020873d6d6b39dfd8a53db78b550ca8320440e/cryptography-49.0.0-cp39-abi3-musllinux_1_2_aarch64.whl": "67e1d20ad9ef3a563c59ef22e7a8a0b8210bd26604369ea4a30a7c66aefe504e", - "https://files.pythonhosted.org/packages/6d/88/05563c7fe2e914e87d1a536d06fe83e66b4e1d95cb593e05aea375531da8/cryptography-49.0.0-cp311-abi3-manylinux_2_34_aarch64.whl": "ccac2bfebc306b862133e3bb71f3f6ee8bb525240089b2d952e4144b3a6d5da7", - "https://files.pythonhosted.org/packages/71/fd/577302e213a1be9468f92d1afef66fcf1ef83d516819d9992ca547f592bd/cryptography-49.0.0-cp311-abi3-musllinux_1_2_x86_64.whl": "66ec79c3904820572d7e987abdf304281f141d37ad9a489b8e97066e7b9b6459", - "https://files.pythonhosted.org/packages/86/12/c48a424f38db03027be9f7ed5c7dc5de9933dbee992865f98b13727a009d/cryptography-49.0.0-cp314-cp314t-manylinux2014_aarch64.manylinux_2_17_aarch64.whl": "196ecd6a36e4e9aa10270393bb98d8df88fccee0bf1e5128b91ae4eb4375896d", - "https://files.pythonhosted.org/packages/93/77/8df9eb486495979bccecd1062e2eaf435250e84437040295b57d09048b0b/cryptography-49.0.0-cp39-abi3-musllinux_1_2_x86_64.whl": "42b0684e0e40cf26122427802486f6d93aea593612603a94fbf260c7eb1e9c1b", - "https://files.pythonhosted.org/packages/94/64/2923570ac1c0bd3a737aa366ac3abbbbde273042308b8cde95e2364a6e6a/cryptography-49.0.0-pp311-pypy311_pp73-manylinux_2_34_aarch64.whl": "b47db11c2c3525083296069b98ac5221907455e989ae0c2e3008bde851921615", - "https://files.pythonhosted.org/packages/9b/22/adf66990e63584a68dfb50c24f48a125c07b1699899381c8151e63ed458c/cryptography-49.0.0-cp311-abi3-macosx_11_0_arm64.whl": "966fe0e9c67490071f14c0d2b1cb2dfb3023c5ce39457343931415f08382f2db", - "https://files.pythonhosted.org/packages/a0/84/84fe36f19caf857d61cb7fc9c63035a47ffabd84ea12d1d393148efa3615/cryptography-49.0.0-cp39-abi3-manylinux_2_34_x86_64.whl": "2400ef9c9e2299a25614eb1dea3db54a69b1349efd043bfac9c67630d136df36", - "https://files.pythonhosted.org/packages/a3/5b/c5246635d5fd3b64e0d45ae10e99fd32fe9676a79915ccfe5a61ba9af1a5/cryptography-49.0.0-cp311-abi3-manylinux_2_31_armv7l.whl": "0b82e28ee398a386f0807bba7884d30f25218855690f45115831bcce5d90822c", - "https://files.pythonhosted.org/packages/a5/4d/9c0cd02f95e2602dd5e563da149ee0830abef3537be8b34dc56281ebe27a/cryptography-49.0.0-cp39-abi3-manylinux_2_28_aarch64.whl": "0f21641cf4b30fca7aee061ced0ec7ad7b073518088b7c9969a297c0ae796c69", - "https://files.pythonhosted.org/packages/a7/f5/8f571d7e27c55bce9f76f026143bcb1e040a4233149ecca0bea5fa5dd5f7/cryptography-49.0.0-cp314-cp314t-manylinux_2_34_aarch64.whl": "b20133d204d2bb56ba047642199603876c872026ca53e79c35b83772ab2cc505", - "https://files.pythonhosted.org/packages/a8/b2/2193fc74f81aee4f9b62733133b73b5176718932ed8f2e4b03fa040480a6/cryptography-49.0.0-cp314-cp314t-manylinux_2_28_aarch64.whl": "4ae387c9cb68ea569ca17e490d66d8142b81c3cc814bf179974b7d146e490bbb", - "https://files.pythonhosted.org/packages/a9/3c/f3ad17eecc1a57b0ba236dc01f90e783c51f4a2f35f64777cc4f47a184b2/cryptography-49.0.0-cp311-abi3-manylinux_2_34_x86_64.whl": "cbc77da8c523d5abd028635ba850a6966fcee2c82e2bf65a41d1d8afe0f98be9", - "https://files.pythonhosted.org/packages/aa/50/a9caea39ad19c431c1a3f8a31114df65b260cdfe67786b6c7e7c040c4c44/cryptography-49.0.0-pp311-pypy311_pp73-win_amd64.whl": "be9fcb48a55f023493482827d4f459bd263cc20efde64f204b97c123201850c6", - "https://files.pythonhosted.org/packages/ab/f8/614dc7e051418cfe53d55173c1e24c6b0085e89996fe90508c2fdf769aef/cryptography-49.0.0-pp311-pypy311_pp73-manylinux_2_34_x86_64.whl": "084ef1af862eb07ec46d25f68689f2102a9fc0e05ce7b80f14f5fe51e4eef0f6", - "https://files.pythonhosted.org/packages/b8/7b/62cbbab75d0659865bf0273790031544a0b16c8072d258f9428dcd8190dc/cryptography-49.0.0-cp39-abi3-manylinux_2_28_x86_64.whl": "6f2debedf9ca60cf1d5bd466475638af5130f89965605cd818484d19987d3a21", - "https://files.pythonhosted.org/packages/b9/26/814681d14248d95d73d5c3eea0c39a94eb8302df966f670a2c60de90974b/cryptography-49.0.0-cp314-cp314t-musllinux_1_2_x86_64.whl": "32703d93296f5c1f4b53349ad3a250c2cae0fdecd3a3dd5d47e616d8d616af27", - "https://files.pythonhosted.org/packages/c2/e6/f60198ea8d9dfa15fff9ed4ca02ce362f6eadd9ba757dcc50634c4257b63/cryptography-49.0.0-cp39-abi3-win_amd64.whl": "026ac7423e6fa66872d3bf889be5974507da3944f866f704fa200eadacd00001", - "https://files.pythonhosted.org/packages/c4/b6/d7696e4e890d6ae1469935164c9e5215c557671cb78d6e3f458ccceaa632/cryptography-49.0.0-cp311-abi3-manylinux_2_34_ppc64le.whl": "d0527ce944105f257f605a827d6ebead966c752038b6e8656abb9c5edee6fc68", - "https://files.pythonhosted.org/packages/d6/a7/f9dac0ab7f80368c56993a7bf638ef9935f825c91902798481fac0898138/cryptography-49.0.0-pp311-pypy311_pp73-manylinux_2_28_aarch64.whl": "c83782480a4a9da4d0feb51950131ba32e12e70813848b3343f6e18c28a66838", - "https://files.pythonhosted.org/packages/d7/70/2ba3769dd0ae167e2f33dfa9592d45db6ff9a61d62ca1a5b3d1bdd09068f/cryptography-49.0.0-pp311-pypy311_pp73-manylinux_2_28_x86_64.whl": "b39efa323140595abd3ecca8529d321ae50f55f3aa3ba9cc81ea56a6011953d5", - "https://files.pythonhosted.org/packages/e4/c0/bff5a02ee731d207d6a1ed51732549d8c53d2bc8da1d10ec6f2844201d68/cryptography-49.0.0-cp314-cp314t-musllinux_1_2_aarch64.whl": "e3fb64c420688e5319ae25113a354015abbd8dffbfbc41781a1ea66fc7622ac3", - "https://files.pythonhosted.org/packages/e6/8b/43011f7ebe515a8aa20d61f290a326cd890c2e738e16e59eaff8d9c3a412/cryptography-49.0.0-cp311-abi3-manylinux2014_x86_64.manylinux_2_17_x86_64.whl": "0e959b578856a3924bc0cbb710fc12c387b9412a951389f3ca61704a9e25f325", - "https://files.pythonhosted.org/packages/e7/84/0e27016a6fc5a0886f797018b26aa42f40c09a82332bff77822a451deaaa/cryptography-49.0.0-cp314-cp314t-manylinux_2_34_ppc64le.whl": "b970c6da94d5bb18629db453d14f2a1300f6bf59b61e9b82377931ef95504866", - "https://files.pythonhosted.org/packages/ec/9e/db72b3ae7fc9cfad53e630e56c6ae83b9b6ff0bf3718ffb8012d20b3aabf/cryptography-49.0.0-cp314-cp314t-macosx_11_0_arm64.whl": "73a205dce83953d131a4aa1e0fd917a2fd1c5b1eef251e9d7152efefcbf5caf7", - "https://files.pythonhosted.org/packages/f0/ee/6fca21d1ac73e06f8bef71940abfd4d2f6472b4bca284d770f32bd4086f6/cryptography-49.0.0-cp39-abi3-manylinux_2_34_aarch64.whl": "28d8b15e6275f12c8a207dc309dfa957903c927d08d0cc937ee3f63f200693cc" + "https://files.pythonhosted.org/packages/09/41/3797cfaf69cae04a13ee78ebd83f0678d9c02b4779d21ce24445326f1a69/cryptography-49.0.0-cp311-abi3-manylinux2014_aarch64.manylinux_2_17_aarch64.whl": "sha256:36d1709f992593689b45bda411498d62c6e365f2ca00b84657d4dadd24de16db", + "https://files.pythonhosted.org/packages/11/2d/5e1fb307cb5931881516b464c98774b3f2c36b5d4bb9a2830253cf553cad/cryptography-49.0.0-cp314-cp314t-manylinux_2_34_x86_64.whl": "sha256:d8ecde755e2e91bf773fc94e8c9d730cd7f2007004cb492263a794ec3899a1c8", + "https://files.pythonhosted.org/packages/17/50/983e838c7fd0d87fd8c969bcdd328edaf5f756e38df5281637424c155873/cryptography-49.0.0-cp314-cp314t-manylinux_2_31_armv7l.whl": "sha256:07cab27cc7b7e0fd28e5e26bb9eeedde5c135c868b46de4a27845abe94af6122", + "https://files.pythonhosted.org/packages/19/2a/5bb823f5bedcf80718cea7fbc95ec5515cca3769633c4b01a32be7f30e7c/cryptography-49.0.0-cp39-abi3-macosx_11_0_arm64.whl": "sha256:ec5e529fb80935c94fe7b729f9972b50e351a0e6b50aa294fd5cabb109fcc29a", + "https://files.pythonhosted.org/packages/1f/09/f42b1d190c5ba75f72062a387f8030d1d75f6ab035788f1d9c4b01de6525/cryptography-49.0.0-cp311-abi3-win_amd64.whl": "sha256:e5dfc1e64de5677cec922ffa8da89c546d0415bf6efdf081842e5d44c84e1f0e", + "https://files.pythonhosted.org/packages/1f/99/d1c90d6041656cc6ee229dc99cd67fd0cd5aec3c5f7d72fffc27cc750054/cryptography-49.0.0.tar.gz": "sha256:f89660a348f4f78a92366240a61404e337586ef7f5909a2fef59ca88ef505493", + "https://files.pythonhosted.org/packages/20/2c/0622f20ff02b2ef32558733443805dc82fd4c275be01b2d19d14676f3a1b/cryptography-49.0.0-cp311-abi3-manylinux_2_28_x86_64.whl": "sha256:2afe9051da7ae7bd5905da5a949280c7d2bb75682e188f650a9d0f2756b834c6", + "https://files.pythonhosted.org/packages/24/01/186c825898477d77e2324d5360fefe622ff1d8d1963ec0554e2cada8ec77/cryptography-49.0.0-cp39-abi3-manylinux_2_28_ppc64le.whl": "sha256:9e82dcc8e56052715fb18b2429e3bca4823b1629136a2084fc45a9a5cecb9b64", + "https://files.pythonhosted.org/packages/2c/99/2d13299eb3dd27b02dcfaafcc91d6b5cb3329f7cbd6d8f51921acd566c1a/cryptography-49.0.0-cp39-abi3-manylinux2014_x86_64.manylinux_2_17_x86_64.whl": "sha256:35b151772baff2c74cba7fa290ceaff4c3b11c0c881eb93eb5dbc05a7cfbba18", + "https://files.pythonhosted.org/packages/3d/df/40577043ca124e17012f408ddddaeb213b856336ac82ddb3bc915f39e29f/cryptography-49.0.0-cp39-abi3-manylinux2014_aarch64.manylinux_2_17_aarch64.whl": "sha256:f78ff2c9ed8dc2d036b0f4d640e22522213d047c1b14e61205a7e55c80a494d4", + "https://files.pythonhosted.org/packages/47/f1/1d3eaa243bfc5de4a187b22aa8c048b3e4980bfbe830ac46e6bac2e66947/cryptography-49.0.0-cp314-cp314t-manylinux_2_28_ppc64le.whl": "sha256:f37d847238971164fdbc68ade6f6574aecc9c0af714190e2083429ff68f4ce9d", + "https://files.pythonhosted.org/packages/4a/91/01ce7303a4579e6d3a6abef01bd322848e9ea7a219adcabc5048b9033571/cryptography-49.0.0-cp311-abi3-manylinux_2_28_aarch64.whl": "sha256:53ecee2e23f7169b6117e99fc8a944e5e50f79e69758a83b52a00cb98ab2b2d2", + "https://files.pythonhosted.org/packages/4c/fe/93ecac273d3738939d023612ad12cca9a3740a5345d69fda04134c43fd96/cryptography-49.0.0-cp314-cp314t-win_amd64.whl": "sha256:33cd0565932807baddb67b96dbee92f2c374b5c89dee09fd74079aeb8c8dba61", + "https://files.pythonhosted.org/packages/4f/01/339573cf1023163a400b0b5d16f6d507de413b9f60be6fd1b77feeaf6737/cryptography-49.0.0-cp311-abi3-musllinux_1_2_aarch64.whl": "sha256:b87e65d263b3e5d3bb92a57e2a6638e2f31110fa7aa890c7b2dbba42248d0a3f", + "https://files.pythonhosted.org/packages/58/39/2d51306721330c486495853eda1c567880ff036de15a14c4b74f399934af/cryptography-49.0.0-cp314-cp314t-manylinux_2_28_x86_64.whl": "sha256:c2bc30226390d60ea19d9f82b19db005fe0452154a23c1c410c12ea801e43561", + "https://files.pythonhosted.org/packages/62/99/a2c95cf8293f07491e9e27c20cc4dcd18176d944e674679adeb1d0173fd6/cryptography-49.0.0-cp311-abi3-manylinux_2_28_ppc64le.whl": "sha256:2eda353d8a27bcbcaa4cbed18994a74ab4d19a2ca897db188ea269ab9b71419b", + "https://files.pythonhosted.org/packages/63/d3/4a83af35d65e3fad632c926fad684c193ea4398569ccb0bbbc7fe8f5dc9a/cryptography-49.0.0-pp311-pypy311_pp73-macosx_11_0_arm64.whl": "sha256:fc1e275c2f1d97b1a6450b8b0ea3ebfa6e087a611c2b26cb2404d48588abab7b", + "https://files.pythonhosted.org/packages/67/d0/a5fcd3515f0bae49a7b6d0413cc1bdccdcc1fc0047037a0d480642cdc5d6/cryptography-49.0.0-cp39-abi3-manylinux_2_34_ppc64le.whl": "sha256:6fc361c34fb6aac015ce19435876635e5c6d21db31998b0920f675f131e043b8", + "https://files.pythonhosted.org/packages/68/28/8a3ad4653662c93fc44dc4e5d8fd374c25c42e07b34bbfbadf49cf57a5a8/cryptography-49.0.0-cp314-cp314t-manylinux2014_x86_64.manylinux_2_17_x86_64.whl": "sha256:7abcee80084cda3f7691f3eb1ce480d8df49cec637b429aa35986c1de71738aa", + "https://files.pythonhosted.org/packages/6c/72/3e798c064bc39e471008075d0f9bc9daf77a80879c092e4a8e170c585ed4/cryptography-49.0.0-cp39-abi3-manylinux_2_31_armv7l.whl": "sha256:8c25ceb16df5b9435f3f6a9829204985b0e0cbee3b48aacd432c7d2c850b44d9", + "https://files.pythonhosted.org/packages/6c/a0/db537264e234f7273a73ec020873d6d6b39dfd8a53db78b550ca8320440e/cryptography-49.0.0-cp39-abi3-musllinux_1_2_aarch64.whl": "sha256:67e1d20ad9ef3a563c59ef22e7a8a0b8210bd26604369ea4a30a7c66aefe504e", + "https://files.pythonhosted.org/packages/6d/88/05563c7fe2e914e87d1a536d06fe83e66b4e1d95cb593e05aea375531da8/cryptography-49.0.0-cp311-abi3-manylinux_2_34_aarch64.whl": "sha256:ccac2bfebc306b862133e3bb71f3f6ee8bb525240089b2d952e4144b3a6d5da7", + "https://files.pythonhosted.org/packages/71/fd/577302e213a1be9468f92d1afef66fcf1ef83d516819d9992ca547f592bd/cryptography-49.0.0-cp311-abi3-musllinux_1_2_x86_64.whl": "sha256:66ec79c3904820572d7e987abdf304281f141d37ad9a489b8e97066e7b9b6459", + "https://files.pythonhosted.org/packages/86/12/c48a424f38db03027be9f7ed5c7dc5de9933dbee992865f98b13727a009d/cryptography-49.0.0-cp314-cp314t-manylinux2014_aarch64.manylinux_2_17_aarch64.whl": "sha256:196ecd6a36e4e9aa10270393bb98d8df88fccee0bf1e5128b91ae4eb4375896d", + "https://files.pythonhosted.org/packages/93/77/8df9eb486495979bccecd1062e2eaf435250e84437040295b57d09048b0b/cryptography-49.0.0-cp39-abi3-musllinux_1_2_x86_64.whl": "sha256:42b0684e0e40cf26122427802486f6d93aea593612603a94fbf260c7eb1e9c1b", + "https://files.pythonhosted.org/packages/94/64/2923570ac1c0bd3a737aa366ac3abbbbde273042308b8cde95e2364a6e6a/cryptography-49.0.0-pp311-pypy311_pp73-manylinux_2_34_aarch64.whl": "sha256:b47db11c2c3525083296069b98ac5221907455e989ae0c2e3008bde851921615", + "https://files.pythonhosted.org/packages/9b/22/adf66990e63584a68dfb50c24f48a125c07b1699899381c8151e63ed458c/cryptography-49.0.0-cp311-abi3-macosx_11_0_arm64.whl": "sha256:966fe0e9c67490071f14c0d2b1cb2dfb3023c5ce39457343931415f08382f2db", + "https://files.pythonhosted.org/packages/a0/84/84fe36f19caf857d61cb7fc9c63035a47ffabd84ea12d1d393148efa3615/cryptography-49.0.0-cp39-abi3-manylinux_2_34_x86_64.whl": "sha256:2400ef9c9e2299a25614eb1dea3db54a69b1349efd043bfac9c67630d136df36", + "https://files.pythonhosted.org/packages/a3/5b/c5246635d5fd3b64e0d45ae10e99fd32fe9676a79915ccfe5a61ba9af1a5/cryptography-49.0.0-cp311-abi3-manylinux_2_31_armv7l.whl": "sha256:0b82e28ee398a386f0807bba7884d30f25218855690f45115831bcce5d90822c", + "https://files.pythonhosted.org/packages/a5/4d/9c0cd02f95e2602dd5e563da149ee0830abef3537be8b34dc56281ebe27a/cryptography-49.0.0-cp39-abi3-manylinux_2_28_aarch64.whl": "sha256:0f21641cf4b30fca7aee061ced0ec7ad7b073518088b7c9969a297c0ae796c69", + "https://files.pythonhosted.org/packages/a7/f5/8f571d7e27c55bce9f76f026143bcb1e040a4233149ecca0bea5fa5dd5f7/cryptography-49.0.0-cp314-cp314t-manylinux_2_34_aarch64.whl": "sha256:b20133d204d2bb56ba047642199603876c872026ca53e79c35b83772ab2cc505", + "https://files.pythonhosted.org/packages/a8/b2/2193fc74f81aee4f9b62733133b73b5176718932ed8f2e4b03fa040480a6/cryptography-49.0.0-cp314-cp314t-manylinux_2_28_aarch64.whl": "sha256:4ae387c9cb68ea569ca17e490d66d8142b81c3cc814bf179974b7d146e490bbb", + "https://files.pythonhosted.org/packages/a9/3c/f3ad17eecc1a57b0ba236dc01f90e783c51f4a2f35f64777cc4f47a184b2/cryptography-49.0.0-cp311-abi3-manylinux_2_34_x86_64.whl": "sha256:cbc77da8c523d5abd028635ba850a6966fcee2c82e2bf65a41d1d8afe0f98be9", + "https://files.pythonhosted.org/packages/aa/50/a9caea39ad19c431c1a3f8a31114df65b260cdfe67786b6c7e7c040c4c44/cryptography-49.0.0-pp311-pypy311_pp73-win_amd64.whl": "sha256:be9fcb48a55f023493482827d4f459bd263cc20efde64f204b97c123201850c6", + "https://files.pythonhosted.org/packages/ab/f8/614dc7e051418cfe53d55173c1e24c6b0085e89996fe90508c2fdf769aef/cryptography-49.0.0-pp311-pypy311_pp73-manylinux_2_34_x86_64.whl": "sha256:084ef1af862eb07ec46d25f68689f2102a9fc0e05ce7b80f14f5fe51e4eef0f6", + "https://files.pythonhosted.org/packages/b8/7b/62cbbab75d0659865bf0273790031544a0b16c8072d258f9428dcd8190dc/cryptography-49.0.0-cp39-abi3-manylinux_2_28_x86_64.whl": "sha256:6f2debedf9ca60cf1d5bd466475638af5130f89965605cd818484d19987d3a21", + "https://files.pythonhosted.org/packages/b9/26/814681d14248d95d73d5c3eea0c39a94eb8302df966f670a2c60de90974b/cryptography-49.0.0-cp314-cp314t-musllinux_1_2_x86_64.whl": "sha256:32703d93296f5c1f4b53349ad3a250c2cae0fdecd3a3dd5d47e616d8d616af27", + "https://files.pythonhosted.org/packages/c2/e6/f60198ea8d9dfa15fff9ed4ca02ce362f6eadd9ba757dcc50634c4257b63/cryptography-49.0.0-cp39-abi3-win_amd64.whl": "sha256:026ac7423e6fa66872d3bf889be5974507da3944f866f704fa200eadacd00001", + "https://files.pythonhosted.org/packages/c4/b6/d7696e4e890d6ae1469935164c9e5215c557671cb78d6e3f458ccceaa632/cryptography-49.0.0-cp311-abi3-manylinux_2_34_ppc64le.whl": "sha256:d0527ce944105f257f605a827d6ebead966c752038b6e8656abb9c5edee6fc68", + "https://files.pythonhosted.org/packages/d6/a7/f9dac0ab7f80368c56993a7bf638ef9935f825c91902798481fac0898138/cryptography-49.0.0-pp311-pypy311_pp73-manylinux_2_28_aarch64.whl": "sha256:c83782480a4a9da4d0feb51950131ba32e12e70813848b3343f6e18c28a66838", + "https://files.pythonhosted.org/packages/d7/70/2ba3769dd0ae167e2f33dfa9592d45db6ff9a61d62ca1a5b3d1bdd09068f/cryptography-49.0.0-pp311-pypy311_pp73-manylinux_2_28_x86_64.whl": "sha256:b39efa323140595abd3ecca8529d321ae50f55f3aa3ba9cc81ea56a6011953d5", + "https://files.pythonhosted.org/packages/e4/c0/bff5a02ee731d207d6a1ed51732549d8c53d2bc8da1d10ec6f2844201d68/cryptography-49.0.0-cp314-cp314t-musllinux_1_2_aarch64.whl": "sha256:e3fb64c420688e5319ae25113a354015abbd8dffbfbc41781a1ea66fc7622ac3", + "https://files.pythonhosted.org/packages/e6/8b/43011f7ebe515a8aa20d61f290a326cd890c2e738e16e59eaff8d9c3a412/cryptography-49.0.0-cp311-abi3-manylinux2014_x86_64.manylinux_2_17_x86_64.whl": "sha256:0e959b578856a3924bc0cbb710fc12c387b9412a951389f3ca61704a9e25f325", + "https://files.pythonhosted.org/packages/e7/84/0e27016a6fc5a0886f797018b26aa42f40c09a82332bff77822a451deaaa/cryptography-49.0.0-cp314-cp314t-manylinux_2_34_ppc64le.whl": "sha256:b970c6da94d5bb18629db453d14f2a1300f6bf59b61e9b82377931ef95504866", + "https://files.pythonhosted.org/packages/ec/9e/db72b3ae7fc9cfad53e630e56c6ae83b9b6ff0bf3718ffb8012d20b3aabf/cryptography-49.0.0-cp314-cp314t-macosx_11_0_arm64.whl": "sha256:73a205dce83953d131a4aa1e0fd917a2fd1c5b1eef251e9d7152efefcbf5caf7", + "https://files.pythonhosted.org/packages/f0/ee/6fca21d1ac73e06f8bef71940abfd4d2f6472b4bca284d770f32bd4086f6/cryptography-49.0.0-cp39-abi3-manylinux_2_34_aarch64.whl": "sha256:28d8b15e6275f12c8a207dc309dfa957903c927d08d0cc937ee3f63f200693cc" }, "docutils": { - "https://files.pythonhosted.org/packages/32/91/30151a39f7570f448ed84529390628a651d7f27c87d73c9b887f8189695e/docutils-0.23-py3-none-any.whl": "25d013af9bf23bc1c7b2b093dff4208166c53a94786c9e447808335ef1185fea", - "https://files.pythonhosted.org/packages/39/a4/5180d9afc57e8fca05601dd652bdff19604c218814037fe90ffc7625a50a/docutils-0.23.tar.gz": "746f5060322511280a1e50eb76846ed6bf2342984b2ac04dc42caa1a8d78799e" + "https://files.pythonhosted.org/packages/32/91/30151a39f7570f448ed84529390628a651d7f27c87d73c9b887f8189695e/docutils-0.23-py3-none-any.whl": "sha256:25d013af9bf23bc1c7b2b093dff4208166c53a94786c9e447808335ef1185fea", + "https://files.pythonhosted.org/packages/39/a4/5180d9afc57e8fca05601dd652bdff19604c218814037fe90ffc7625a50a/docutils-0.23.tar.gz": "sha256:746f5060322511280a1e50eb76846ed6bf2342984b2ac04dc42caa1a8d78799e" }, "id": { - "https://files.pythonhosted.org/packages/42/77/de194443bf38daed9452139e960c632b0ef9f9a5dd9ce605fdf18ca9f1b1/id-1.6.1-py3-none-any.whl": "f5ec41ed2629a508f5d0988eda142e190c9c6da971100612c4de9ad9f9b237ca", - "https://files.pythonhosted.org/packages/6d/04/c2156091427636080787aac190019dc64096e56a23b7364d3c1764ee3a06/id-1.6.1.tar.gz": "d0732d624fb46fd4e7bc4e5152f00214450953b9e772c182c1c22964def1a069" + "https://files.pythonhosted.org/packages/42/77/de194443bf38daed9452139e960c632b0ef9f9a5dd9ce605fdf18ca9f1b1/id-1.6.1-py3-none-any.whl": "sha256:f5ec41ed2629a508f5d0988eda142e190c9c6da971100612c4de9ad9f9b237ca", + "https://files.pythonhosted.org/packages/6d/04/c2156091427636080787aac190019dc64096e56a23b7364d3c1764ee3a06/id-1.6.1.tar.gz": "sha256:d0732d624fb46fd4e7bc4e5152f00214450953b9e772c182c1c22964def1a069" }, "idna": { - "https://files.pythonhosted.org/packages/1e/5e/d4e9f1a599fb8e573b7b87160658329fbf28d19eac2718f51fc3def3aa5a/idna-3.18-py3-none-any.whl": "7f952cbe720b688055e3f87de14f5c3e5fdaa8bc3928985c4077ca689de849a2", - "https://files.pythonhosted.org/packages/cd/63/9496c57188a2ee585e0f1db071d75089a11e98aa86eb99d9d7618fc1edce/idna-3.18.tar.gz": "ffb385a7e039654cef1ab9ef32c6fafe283c0c0467bba1d9029738ce4a14a848" + "https://files.pythonhosted.org/packages/1e/5e/d4e9f1a599fb8e573b7b87160658329fbf28d19eac2718f51fc3def3aa5a/idna-3.18-py3-none-any.whl": "sha256:7f952cbe720b688055e3f87de14f5c3e5fdaa8bc3928985c4077ca689de849a2", + "https://files.pythonhosted.org/packages/cd/63/9496c57188a2ee585e0f1db071d75089a11e98aa86eb99d9d7618fc1edce/idna-3.18.tar.gz": "sha256:ffb385a7e039654cef1ab9ef32c6fafe283c0c0467bba1d9029738ce4a14a848" }, "importlib-metadata": { - "https://files.pythonhosted.org/packages/38/3d/2d244233ac4f76e38533cfcb2991c9eb4c7bf688ae0a036d30725b8faafe/importlib_metadata-9.0.0-py3-none-any.whl": "2d21d1cc5a017bd0559e36150c21c830ab1dc304dedd1b7ea85d20f45ef3edd7", - "https://files.pythonhosted.org/packages/a9/01/15bb152d77b21318514a96f43af312635eb2500c96b55398d020c93d86ea/importlib_metadata-9.0.0.tar.gz": "a4f57ab599e6a2e3016d7595cfd72eb4661a5106e787a95bcc90c7105b831efc" + "https://files.pythonhosted.org/packages/38/3d/2d244233ac4f76e38533cfcb2991c9eb4c7bf688ae0a036d30725b8faafe/importlib_metadata-9.0.0-py3-none-any.whl": "sha256:2d21d1cc5a017bd0559e36150c21c830ab1dc304dedd1b7ea85d20f45ef3edd7", + "https://files.pythonhosted.org/packages/a9/01/15bb152d77b21318514a96f43af312635eb2500c96b55398d020c93d86ea/importlib_metadata-9.0.0.tar.gz": "sha256:a4f57ab599e6a2e3016d7595cfd72eb4661a5106e787a95bcc90c7105b831efc" }, "jaraco-classes": { - "https://files.pythonhosted.org/packages/06/c0/ed4a27bc5571b99e3cff68f8a9fa5b56ff7df1c2251cc715a652ddd26402/jaraco.classes-3.4.0.tar.gz": "47a024b51d0239c0dd8c8540c6c7f484be3b8fcf0b2d85c13825780d3b3f3acd", - "https://files.pythonhosted.org/packages/7f/66/b15ce62552d84bbfcec9a4873ab79d993a1dd4edb922cbfccae192bd5b5f/jaraco.classes-3.4.0-py3-none-any.whl": "f662826b6bed8cace05e7ff873ce0f9283b5c924470fe664fff1c2f00f581790" + "https://files.pythonhosted.org/packages/06/c0/ed4a27bc5571b99e3cff68f8a9fa5b56ff7df1c2251cc715a652ddd26402/jaraco.classes-3.4.0.tar.gz": "sha256:47a024b51d0239c0dd8c8540c6c7f484be3b8fcf0b2d85c13825780d3b3f3acd", + "https://files.pythonhosted.org/packages/7f/66/b15ce62552d84bbfcec9a4873ab79d993a1dd4edb922cbfccae192bd5b5f/jaraco.classes-3.4.0-py3-none-any.whl": "sha256:f662826b6bed8cace05e7ff873ce0f9283b5c924470fe664fff1c2f00f581790" }, "jaraco-context": { - "https://files.pythonhosted.org/packages/af/50/4763cd07e722bb6285316d390a164bc7e479db9d90daa769f22578f698b4/jaraco_context-6.1.2.tar.gz": "f1a6c9d391e661cc5b8d39861ff077a7dc24dc23833ccee564b234b81c82dfe3", - "https://files.pythonhosted.org/packages/f2/58/bc8954bda5fcda97bd7c19be11b85f91973d67a706ed4a3aec33e7de22db/jaraco_context-6.1.2-py3-none-any.whl": "bf8150b79a2d5d91ae48629d8b427a8f7ba0e1097dd6202a9059f29a36379535" + "https://files.pythonhosted.org/packages/af/50/4763cd07e722bb6285316d390a164bc7e479db9d90daa769f22578f698b4/jaraco_context-6.1.2.tar.gz": "sha256:f1a6c9d391e661cc5b8d39861ff077a7dc24dc23833ccee564b234b81c82dfe3", + "https://files.pythonhosted.org/packages/f2/58/bc8954bda5fcda97bd7c19be11b85f91973d67a706ed4a3aec33e7de22db/jaraco_context-6.1.2-py3-none-any.whl": "sha256:bf8150b79a2d5d91ae48629d8b427a8f7ba0e1097dd6202a9059f29a36379535" }, "jaraco-functools": { - "https://files.pythonhosted.org/packages/02/36/ecc85bc96c273dc8a11273ed4782272975e6338d4a3e9228621175edf0e3/jaraco_functools-4.6.0-py3-none-any.whl": "99e3dc0060c5cbe8fcd1cdb36258e2a65ca40f1566b2033b12abb1bb44dd3c30", - "https://files.pythonhosted.org/packages/6c/1f/c23395957d41ccf27c4e535c3d334c4051e5395b3752057ba4cbaec35c56/jaraco_functools-4.6.0.tar.gz": "880c577ec9720b3a052d5bc611fb9f2269b3d87902ef42440df443b88e443280" + "https://files.pythonhosted.org/packages/02/36/ecc85bc96c273dc8a11273ed4782272975e6338d4a3e9228621175edf0e3/jaraco_functools-4.6.0-py3-none-any.whl": "sha256:99e3dc0060c5cbe8fcd1cdb36258e2a65ca40f1566b2033b12abb1bb44dd3c30", + "https://files.pythonhosted.org/packages/6c/1f/c23395957d41ccf27c4e535c3d334c4051e5395b3752057ba4cbaec35c56/jaraco_functools-4.6.0.tar.gz": "sha256:880c577ec9720b3a052d5bc611fb9f2269b3d87902ef42440df443b88e443280" }, "jeepney": { - "https://files.pythonhosted.org/packages/7b/6f/357efd7602486741aa73ffc0617fb310a29b588ed0fd69c2399acbb85b0c/jeepney-0.9.0.tar.gz": "cf0e9e845622b81e4a28df94c40345400256ec608d0e55bb8a3feaa9163f5732", - "https://files.pythonhosted.org/packages/b2/a3/e137168c9c44d18eff0376253da9f1e9234d0239e0ee230d2fee6cea8e55/jeepney-0.9.0-py3-none-any.whl": "97e5714520c16fc0a45695e5365a2e11b81ea79bba796e26f9f1d178cb182683" + "https://files.pythonhosted.org/packages/7b/6f/357efd7602486741aa73ffc0617fb310a29b588ed0fd69c2399acbb85b0c/jeepney-0.9.0.tar.gz": "sha256:cf0e9e845622b81e4a28df94c40345400256ec608d0e55bb8a3feaa9163f5732", + "https://files.pythonhosted.org/packages/b2/a3/e137168c9c44d18eff0376253da9f1e9234d0239e0ee230d2fee6cea8e55/jeepney-0.9.0-py3-none-any.whl": "sha256:97e5714520c16fc0a45695e5365a2e11b81ea79bba796e26f9f1d178cb182683" }, "keyring": { - "https://files.pythonhosted.org/packages/43/4b/674af6ef2f97d56f0ab5153bf0bfa28ccb6c3ed4d1babf4305449668807b/keyring-25.7.0.tar.gz": "fe01bd85eb3f8fb3dd0405defdeac9a5b4f6f0439edbb3149577f244a2e8245b", - "https://files.pythonhosted.org/packages/81/db/e655086b7f3a705df045bf0933bdd9c2f79bb3c97bfef1384598bb79a217/keyring-25.7.0-py3-none-any.whl": "be4a0b195f149690c166e850609a477c532ddbfbaed96a404d4e43f8d5e2689f" + "https://files.pythonhosted.org/packages/43/4b/674af6ef2f97d56f0ab5153bf0bfa28ccb6c3ed4d1babf4305449668807b/keyring-25.7.0.tar.gz": "sha256:fe01bd85eb3f8fb3dd0405defdeac9a5b4f6f0439edbb3149577f244a2e8245b", + "https://files.pythonhosted.org/packages/81/db/e655086b7f3a705df045bf0933bdd9c2f79bb3c97bfef1384598bb79a217/keyring-25.7.0-py3-none-any.whl": "sha256:be4a0b195f149690c166e850609a477c532ddbfbaed96a404d4e43f8d5e2689f" }, "markdown-it-py": { - "https://files.pythonhosted.org/packages/06/ff/7841249c247aa650a76b9ee4bbaeae59370dc8bfd2f6c01f3630c35eb134/markdown_it_py-4.2.0.tar.gz": "04a21681d6fbb623de53f6f364d352309d4094dd4194040a10fd51833e418d49", - "https://files.pythonhosted.org/packages/b3/81/4da04ced5a082363ecfa159c010d200ecbd959ae410c10c0264a38cac0f5/markdown_it_py-4.2.0-py3-none-any.whl": "9f7ebbcd14fe59494226453aed97c1070d83f8d24b6fc3a3bcf9a38092641c4a" + "https://files.pythonhosted.org/packages/06/ff/7841249c247aa650a76b9ee4bbaeae59370dc8bfd2f6c01f3630c35eb134/markdown_it_py-4.2.0.tar.gz": "sha256:04a21681d6fbb623de53f6f364d352309d4094dd4194040a10fd51833e418d49", + "https://files.pythonhosted.org/packages/b3/81/4da04ced5a082363ecfa159c010d200ecbd959ae410c10c0264a38cac0f5/markdown_it_py-4.2.0-py3-none-any.whl": "sha256:9f7ebbcd14fe59494226453aed97c1070d83f8d24b6fc3a3bcf9a38092641c4a" }, "mdurl": { - "https://files.pythonhosted.org/packages/b3/38/89ba8ad64ae25be8de66a6d463314cf1eb366222074cfda9ee839c56a4b4/mdurl-0.1.2-py3-none-any.whl": "84008a41e51615a49fc9966191ff91509e3c40b939176e643fd50a5c2196b8f8", - "https://files.pythonhosted.org/packages/d6/54/cfe61301667036ec958cb99bd3efefba235e65cdeb9c84d24a8293ba1d90/mdurl-0.1.2.tar.gz": "bb413d29f5eea38f31dd4754dd7377d4465116fb207585f97bf925588687c1ba" + "https://files.pythonhosted.org/packages/b3/38/89ba8ad64ae25be8de66a6d463314cf1eb366222074cfda9ee839c56a4b4/mdurl-0.1.2-py3-none-any.whl": "sha256:84008a41e51615a49fc9966191ff91509e3c40b939176e643fd50a5c2196b8f8", + "https://files.pythonhosted.org/packages/d6/54/cfe61301667036ec958cb99bd3efefba235e65cdeb9c84d24a8293ba1d90/mdurl-0.1.2.tar.gz": "sha256:bb413d29f5eea38f31dd4754dd7377d4465116fb207585f97bf925588687c1ba" }, "more-itertools": { - "https://files.pythonhosted.org/packages/de/1d/f4da6f02cdffe04d6362210b807146a26044c88d839208aec273bb0d9184/more_itertools-11.1.0.tar.gz": "48e8f4d9e7e5878571ecf6f2b4e57634f93cd474cc8cfbd2376f2d11b396e30d", - "https://files.pythonhosted.org/packages/e8/3d/1087453384dbde46a8c7f9356eead2c58be8a7bf156bca40243377c85715/more_itertools-11.1.0-py3-none-any.whl": "4b65538ae22f6fed0ce4874efd317463a7489796a0939fa66824dd542125a192" + "https://files.pythonhosted.org/packages/de/1d/f4da6f02cdffe04d6362210b807146a26044c88d839208aec273bb0d9184/more_itertools-11.1.0.tar.gz": "sha256:48e8f4d9e7e5878571ecf6f2b4e57634f93cd474cc8cfbd2376f2d11b396e30d", + "https://files.pythonhosted.org/packages/e8/3d/1087453384dbde46a8c7f9356eead2c58be8a7bf156bca40243377c85715/more_itertools-11.1.0-py3-none-any.whl": "sha256:4b65538ae22f6fed0ce4874efd317463a7489796a0939fa66824dd542125a192" }, "nh3": { - "https://files.pythonhosted.org/packages/09/a1/ea83abe738a3fbaa203dfdb836ca7cbab0e7e9609faaee4fe1d4652599c0/nh3-0.3.6-cp314-cp314t-musllinux_1_2_i686.whl": "edb2b4a1a27523e6cc7c417f8d21ce3d005243548b93e56b762b66b0c7f589f9", - "https://files.pythonhosted.org/packages/0a/13/6f1e302ca674ac74362e150848ad56a1be5145391204f74facdb8e94df12/nh3-0.3.6-cp38-abi3-manylinux_2_17_s390x.manylinux2014_s390x.whl": "455469a29951edc92bc48b47ac2281c3f2609e6c4f6a047056449f8c2c23facf", - "https://files.pythonhosted.org/packages/0e/24/a0d80182a18919665fefd19c1c06f1d1df1c9a6455d0252de40c034a0bc3/nh3-0.3.6-cp38-abi3-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl": "e6b7beece07525dc6e6b0fc2f104442de2ba328360ad00e50cbe2e1fd620447d", - "https://files.pythonhosted.org/packages/11/f9/3966c61455668c08853bf5e33b4bed93c421f3194ce4de896dc248d6f6ce/nh3-0.3.6-cp38-abi3-musllinux_1_2_armv7l.whl": "f5ed5fe84aee7f39db95c214a7421bf0499fbf500fec6d86a4e29bfc37971438", - "https://files.pythonhosted.org/packages/17/0c/6cdb5ee1e127be50dc8391e54bddc1f64e87bf4bfad0c55633320e2e02db/nh3-0.3.6-cp38-abi3-musllinux_1_2_x86_64.whl": "36d06341bd501240d320f5942481ed5e6846136b666e1ba4faf802b78ebc875f", - "https://files.pythonhosted.org/packages/19/d3/479cb4ae440424825735d60525b53e3c77fd60fd6e6afc0e984f00eb0178/nh3-0.3.6-cp38-abi3-musllinux_1_2_i686.whl": "082675ff87b9385ec430ffe6d5847ba7456cc39b73720cd4add472f9f4cffd56", - "https://files.pythonhosted.org/packages/25/bb/431615ba1d1d3eb63cde0f974f2114edf863a8a3f6049a12fed23fc241d3/nh3-0.3.6-cp38-abi3-manylinux_2_17_ppc64.manylinux2014_ppc64.whl": "44673b27010051ab5a5e438a86ec31bbda61d4a77d7e900af6b7be3037c1abae", - "https://files.pythonhosted.org/packages/30/a8/fb2c38845efb703a9173bffdfc745fc64d2b0e55cfc73a3647d2f028250c/nh3-0.3.6-cp38-abi3-manylinux_2_5_i686.manylinux1_i686.whl": "2f90d9a0cfdbee218994fdaaeeb5a0fde62d08f35e4eef0378ec1e2200172fd0", - "https://files.pythonhosted.org/packages/36/ea/5542f3c45da4c00290d9d67a65e996702e23e613c4b627de3e09cb9fe357/nh3-0.3.6-cp314-cp314t-win_amd64.whl": "4713502748f564fee0633b37b3403783ce0a3af3a3d148ad91025a5bdadb7bc6", - "https://files.pythonhosted.org/packages/3c/a6/bfaa00046e58603507dcfc266c4778e3ab7adf68a5dedd73b6274b8d9314/nh3-0.3.6-cp38-abi3-manylinux_2_31_riscv64.whl": "25c733bee928530556b1db0ea46c52cf5aa686146e38e60a6fc7cb801ef91cec", - "https://files.pythonhosted.org/packages/41/21/e1084ab18eb589506335c7c7576f2d4643e9a0c0e33983ef0e549a256b96/nh3-0.3.6-cp38-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl": "e1b160831c9cdb06a6c79c2f9cdb11386602938f9af260d1c457a85add4f6f69", - "https://files.pythonhosted.org/packages/49/09/0d8e3101636d9ad88cdefb2914e764cb8e876ebdbb4286bfc251277d9c67/nh3-0.3.6-cp314-cp314t-musllinux_1_2_armv7l.whl": "889932a97fb4abb6f95fef1914c0d269ebfb60011e67121c1163059b9449dbb4", - "https://files.pythonhosted.org/packages/4b/4a/526f199626bfcb496bc01a268051b44737962005553b158e985ed7e64865/nh3-0.3.6-cp314-cp314t-musllinux_1_2_aarch64.whl": "f2f14b7ae1fca99c4a66c981aac3974e7fbc1ca30a12673d223ae1df76680917", - "https://files.pythonhosted.org/packages/59/62/5b6108bedaef2b2637fed04c87bdbcb5967b9961758b41f0e466ef22a022/nh3-0.3.6-cp314-cp314t-manylinux_2_5_i686.manylinux1_i686.whl": "34d2b0d934156b87ee114f599a3ba9b8b9e17b5d79652ba3a13fa50903de965e", - "https://files.pythonhosted.org/packages/5b/67/314f6151bad77a93d751978a344033e1fc890822f05f0416079338e34231/nh3-0.3.6-cp38-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl": "905f877dc66dd7aea4a76e54bcb26acb5ff8216f720c0017ccf63e0e6035698e", - "https://files.pythonhosted.org/packages/5e/1b/ef84624f14954d270f74060a19fc550dd4f06656399447569afb584d8c06/nh3-0.3.6.tar.gz": "f3736c9dd3d1856f80cd031715b84ca75cda2bbb1ac802c3da26bfce590838d7", - "https://files.pythonhosted.org/packages/66/35/26bd47e6af5915a628281dccdac354ddf4e32f7397047894270acd8c9870/nh3-0.3.6-cp314-cp314t-win_arm64.whl": "69bbb92865a693d909db3a700d3c01537533844d0948c1e9323561ce06ecda41", - "https://files.pythonhosted.org/packages/66/69/0654482b8635012fbae67826bd6c381abb05d841ac7388b9b4666300fdad/nh3-0.3.6-cp314-cp314t-musllinux_1_2_x86_64.whl": "43bc1ed3fa0716295fabee29ba42b2667e4a51d140b0a68e092170a765474fa6", - "https://files.pythonhosted.org/packages/68/17/06e72a18ee9b572914447338237ca7eb164c0df901f141bc10d1282247a2/nh3-0.3.6-cp38-abi3-musllinux_1_2_aarch64.whl": "82ca5bf427ad1b216b65ede1a2e2d87dc49bec417ceba0f297213107d3cd9d78", - "https://files.pythonhosted.org/packages/7b/e5/7cafee2f0413ca4cb0ef3bd111e94d408a48810008b283ad8aee00dd1809/nh3-0.3.6-cp38-abi3-win_arm64.whl": "69f365963f63a1e9bff53bdbb3c542c7c2efed3e163c9d5d83a772a2ac468c21", - "https://files.pythonhosted.org/packages/82/fa/2b5d684e3edf1e81bfd02d298c78c3e3da77ca1d8a2be3183a79544a7548/nh3-0.3.6-cp38-abi3-win_amd64.whl": "f338ac7d594c067679f1e99b4f5ec3906842979560f9d8f15d6bdfa39a353b10", - "https://files.pythonhosted.org/packages/99/3e/6506aa4f23dc7b7993a2d0a45dca3ce864ec48380adfe15a173e643c63e8/nh3-0.3.6-cp314-cp314t-macosx_10_12_x86_64.macosx_11_0_arm64.macosx_10_12_universal2.whl": "2411e8c3cee81a1ddd62c2a5d50585c28aa5566d373ad1db92536b95ddb24ef2", - "https://files.pythonhosted.org/packages/b0/94/f48d08e6f72a406300fa11d8acd929fea1a80d4bf750fa292cb10785f126/nh3-0.3.6-cp38-abi3-manylinux_2_17_armv7l.manylinux2014_armv7l.whl": "d14bf7982e7a77c0c775634c29c07ce08b38a046df73e1c1f139b3e82f18a38e", - "https://files.pythonhosted.org/packages/e3/e1/e96e7864a7a53bd6b6fab7e9632467382a2a2c1f3fed951918ad131542fb/nh3-0.3.6-cp314-cp314t-manylinux_2_17_x86_64.manylinux2014_x86_64.whl": "e196fa70c2ff2eb4de7d3df3108f8f358c1d69dff20d45b11f20a5aa227ffb6d", - "https://files.pythonhosted.org/packages/e9/55/9de666ad975d6ccd77d799ea0add55ee2347aa81286ce21b2a97c070746b/nh3-0.3.6-cp38-abi3-win32.whl": "5276ef17bdba9ad8040575c74072008b13aae429436e9d0429e718bb5f90f4da", - "https://files.pythonhosted.org/packages/ed/a6/1f7285ffadc8307c4dbeb08d21b920536d5117785056d1079e998c4dfa44/nh3-0.3.6-cp314-cp314t-win32.whl": "597a8e843bea00b2eb5520658dc24a9bb032e7fc9e7c2c0c4cd29420220c9796", - "https://files.pythonhosted.org/packages/f3/ab/a7653bce9a3b204be6a6931767a9e23595807bb84790ce6685e4d7e5bd08/nh3-0.3.6-cp38-abi3-macosx_10_12_x86_64.macosx_11_0_arm64.macosx_10_12_universal2.whl": "a43ebd7543555c3ac1bc353023d0794e75cb76f6f18f19c32e95441496c0cc25" + "https://files.pythonhosted.org/packages/09/a1/ea83abe738a3fbaa203dfdb836ca7cbab0e7e9609faaee4fe1d4652599c0/nh3-0.3.6-cp314-cp314t-musllinux_1_2_i686.whl": "sha256:edb2b4a1a27523e6cc7c417f8d21ce3d005243548b93e56b762b66b0c7f589f9", + "https://files.pythonhosted.org/packages/0a/13/6f1e302ca674ac74362e150848ad56a1be5145391204f74facdb8e94df12/nh3-0.3.6-cp38-abi3-manylinux_2_17_s390x.manylinux2014_s390x.whl": "sha256:455469a29951edc92bc48b47ac2281c3f2609e6c4f6a047056449f8c2c23facf", + "https://files.pythonhosted.org/packages/0e/24/a0d80182a18919665fefd19c1c06f1d1df1c9a6455d0252de40c034a0bc3/nh3-0.3.6-cp38-abi3-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl": "sha256:e6b7beece07525dc6e6b0fc2f104442de2ba328360ad00e50cbe2e1fd620447d", + "https://files.pythonhosted.org/packages/11/f9/3966c61455668c08853bf5e33b4bed93c421f3194ce4de896dc248d6f6ce/nh3-0.3.6-cp38-abi3-musllinux_1_2_armv7l.whl": "sha256:f5ed5fe84aee7f39db95c214a7421bf0499fbf500fec6d86a4e29bfc37971438", + "https://files.pythonhosted.org/packages/17/0c/6cdb5ee1e127be50dc8391e54bddc1f64e87bf4bfad0c55633320e2e02db/nh3-0.3.6-cp38-abi3-musllinux_1_2_x86_64.whl": "sha256:36d06341bd501240d320f5942481ed5e6846136b666e1ba4faf802b78ebc875f", + "https://files.pythonhosted.org/packages/19/d3/479cb4ae440424825735d60525b53e3c77fd60fd6e6afc0e984f00eb0178/nh3-0.3.6-cp38-abi3-musllinux_1_2_i686.whl": "sha256:082675ff87b9385ec430ffe6d5847ba7456cc39b73720cd4add472f9f4cffd56", + "https://files.pythonhosted.org/packages/25/bb/431615ba1d1d3eb63cde0f974f2114edf863a8a3f6049a12fed23fc241d3/nh3-0.3.6-cp38-abi3-manylinux_2_17_ppc64.manylinux2014_ppc64.whl": "sha256:44673b27010051ab5a5e438a86ec31bbda61d4a77d7e900af6b7be3037c1abae", + "https://files.pythonhosted.org/packages/30/a8/fb2c38845efb703a9173bffdfc745fc64d2b0e55cfc73a3647d2f028250c/nh3-0.3.6-cp38-abi3-manylinux_2_5_i686.manylinux1_i686.whl": "sha256:2f90d9a0cfdbee218994fdaaeeb5a0fde62d08f35e4eef0378ec1e2200172fd0", + "https://files.pythonhosted.org/packages/36/ea/5542f3c45da4c00290d9d67a65e996702e23e613c4b627de3e09cb9fe357/nh3-0.3.6-cp314-cp314t-win_amd64.whl": "sha256:4713502748f564fee0633b37b3403783ce0a3af3a3d148ad91025a5bdadb7bc6", + "https://files.pythonhosted.org/packages/3c/a6/bfaa00046e58603507dcfc266c4778e3ab7adf68a5dedd73b6274b8d9314/nh3-0.3.6-cp38-abi3-manylinux_2_31_riscv64.whl": "sha256:25c733bee928530556b1db0ea46c52cf5aa686146e38e60a6fc7cb801ef91cec", + "https://files.pythonhosted.org/packages/41/21/e1084ab18eb589506335c7c7576f2d4643e9a0c0e33983ef0e549a256b96/nh3-0.3.6-cp38-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl": "sha256:e1b160831c9cdb06a6c79c2f9cdb11386602938f9af260d1c457a85add4f6f69", + "https://files.pythonhosted.org/packages/49/09/0d8e3101636d9ad88cdefb2914e764cb8e876ebdbb4286bfc251277d9c67/nh3-0.3.6-cp314-cp314t-musllinux_1_2_armv7l.whl": "sha256:889932a97fb4abb6f95fef1914c0d269ebfb60011e67121c1163059b9449dbb4", + "https://files.pythonhosted.org/packages/4b/4a/526f199626bfcb496bc01a268051b44737962005553b158e985ed7e64865/nh3-0.3.6-cp314-cp314t-musllinux_1_2_aarch64.whl": "sha256:f2f14b7ae1fca99c4a66c981aac3974e7fbc1ca30a12673d223ae1df76680917", + "https://files.pythonhosted.org/packages/59/62/5b6108bedaef2b2637fed04c87bdbcb5967b9961758b41f0e466ef22a022/nh3-0.3.6-cp314-cp314t-manylinux_2_5_i686.manylinux1_i686.whl": "sha256:34d2b0d934156b87ee114f599a3ba9b8b9e17b5d79652ba3a13fa50903de965e", + "https://files.pythonhosted.org/packages/5b/67/314f6151bad77a93d751978a344033e1fc890822f05f0416079338e34231/nh3-0.3.6-cp38-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl": "sha256:905f877dc66dd7aea4a76e54bcb26acb5ff8216f720c0017ccf63e0e6035698e", + "https://files.pythonhosted.org/packages/5e/1b/ef84624f14954d270f74060a19fc550dd4f06656399447569afb584d8c06/nh3-0.3.6.tar.gz": "sha256:f3736c9dd3d1856f80cd031715b84ca75cda2bbb1ac802c3da26bfce590838d7", + "https://files.pythonhosted.org/packages/66/35/26bd47e6af5915a628281dccdac354ddf4e32f7397047894270acd8c9870/nh3-0.3.6-cp314-cp314t-win_arm64.whl": "sha256:69bbb92865a693d909db3a700d3c01537533844d0948c1e9323561ce06ecda41", + "https://files.pythonhosted.org/packages/66/69/0654482b8635012fbae67826bd6c381abb05d841ac7388b9b4666300fdad/nh3-0.3.6-cp314-cp314t-musllinux_1_2_x86_64.whl": "sha256:43bc1ed3fa0716295fabee29ba42b2667e4a51d140b0a68e092170a765474fa6", + "https://files.pythonhosted.org/packages/68/17/06e72a18ee9b572914447338237ca7eb164c0df901f141bc10d1282247a2/nh3-0.3.6-cp38-abi3-musllinux_1_2_aarch64.whl": "sha256:82ca5bf427ad1b216b65ede1a2e2d87dc49bec417ceba0f297213107d3cd9d78", + "https://files.pythonhosted.org/packages/7b/e5/7cafee2f0413ca4cb0ef3bd111e94d408a48810008b283ad8aee00dd1809/nh3-0.3.6-cp38-abi3-win_arm64.whl": "sha256:69f365963f63a1e9bff53bdbb3c542c7c2efed3e163c9d5d83a772a2ac468c21", + "https://files.pythonhosted.org/packages/82/fa/2b5d684e3edf1e81bfd02d298c78c3e3da77ca1d8a2be3183a79544a7548/nh3-0.3.6-cp38-abi3-win_amd64.whl": "sha256:f338ac7d594c067679f1e99b4f5ec3906842979560f9d8f15d6bdfa39a353b10", + "https://files.pythonhosted.org/packages/99/3e/6506aa4f23dc7b7993a2d0a45dca3ce864ec48380adfe15a173e643c63e8/nh3-0.3.6-cp314-cp314t-macosx_10_12_x86_64.macosx_11_0_arm64.macosx_10_12_universal2.whl": "sha256:2411e8c3cee81a1ddd62c2a5d50585c28aa5566d373ad1db92536b95ddb24ef2", + "https://files.pythonhosted.org/packages/b0/94/f48d08e6f72a406300fa11d8acd929fea1a80d4bf750fa292cb10785f126/nh3-0.3.6-cp38-abi3-manylinux_2_17_armv7l.manylinux2014_armv7l.whl": "sha256:d14bf7982e7a77c0c775634c29c07ce08b38a046df73e1c1f139b3e82f18a38e", + "https://files.pythonhosted.org/packages/e3/e1/e96e7864a7a53bd6b6fab7e9632467382a2a2c1f3fed951918ad131542fb/nh3-0.3.6-cp314-cp314t-manylinux_2_17_x86_64.manylinux2014_x86_64.whl": "sha256:e196fa70c2ff2eb4de7d3df3108f8f358c1d69dff20d45b11f20a5aa227ffb6d", + "https://files.pythonhosted.org/packages/e9/55/9de666ad975d6ccd77d799ea0add55ee2347aa81286ce21b2a97c070746b/nh3-0.3.6-cp38-abi3-win32.whl": "sha256:5276ef17bdba9ad8040575c74072008b13aae429436e9d0429e718bb5f90f4da", + "https://files.pythonhosted.org/packages/ed/a6/1f7285ffadc8307c4dbeb08d21b920536d5117785056d1079e998c4dfa44/nh3-0.3.6-cp314-cp314t-win32.whl": "sha256:597a8e843bea00b2eb5520658dc24a9bb032e7fc9e7c2c0c4cd29420220c9796", + "https://files.pythonhosted.org/packages/f3/ab/a7653bce9a3b204be6a6931767a9e23595807bb84790ce6685e4d7e5bd08/nh3-0.3.6-cp38-abi3-macosx_10_12_x86_64.macosx_11_0_arm64.macosx_10_12_universal2.whl": "sha256:a43ebd7543555c3ac1bc353023d0794e75cb76f6f18f19c32e95441496c0cc25" }, "packaging": { - "https://files.pythonhosted.org/packages/d7/f1/e7a6dd94a8d4a5626c03e4e99c87f241ba9e350cd9e6d75123f992427270/packaging-26.2.tar.gz": "ff452ff5a3e828ce110190feff1178bb1f2ea2281fa2075aadb987c2fb221661", - "https://files.pythonhosted.org/packages/df/b2/87e62e8c3e2f4b32e5fe99e0b86d576da1312593b39f47d8ceef365e95ed/packaging-26.2-py3-none-any.whl": "5fc45236b9446107ff2415ce77c807cee2862cb6fac22b8a73826d0693b0980e" + "https://files.pythonhosted.org/packages/d7/f1/e7a6dd94a8d4a5626c03e4e99c87f241ba9e350cd9e6d75123f992427270/packaging-26.2.tar.gz": "sha256:ff452ff5a3e828ce110190feff1178bb1f2ea2281fa2075aadb987c2fb221661", + "https://files.pythonhosted.org/packages/df/b2/87e62e8c3e2f4b32e5fe99e0b86d576da1312593b39f47d8ceef365e95ed/packaging-26.2-py3-none-any.whl": "sha256:5fc45236b9446107ff2415ce77c807cee2862cb6fac22b8a73826d0693b0980e" }, "pycparser": { - "https://files.pythonhosted.org/packages/0c/c3/44f3fbbfa403ea2a7c779186dc20772604442dde72947e7d01069cbe98e3/pycparser-3.0-py3-none-any.whl": "b727414169a36b7d524c1c3e31839a521725078d7b2ff038656844266160a992", - "https://files.pythonhosted.org/packages/1b/7d/92392ff7815c21062bea51aa7b87d45576f649f16458d78b7cf94b9ab2e6/pycparser-3.0.tar.gz": "600f49d217304a5902ac3c37e1281c9fe94e4d0489de643a9504c5cdfdfc6b29" + "https://files.pythonhosted.org/packages/0c/c3/44f3fbbfa403ea2a7c779186dc20772604442dde72947e7d01069cbe98e3/pycparser-3.0-py3-none-any.whl": "sha256:b727414169a36b7d524c1c3e31839a521725078d7b2ff038656844266160a992", + "https://files.pythonhosted.org/packages/1b/7d/92392ff7815c21062bea51aa7b87d45576f649f16458d78b7cf94b9ab2e6/pycparser-3.0.tar.gz": "sha256:600f49d217304a5902ac3c37e1281c9fe94e4d0489de643a9504c5cdfdfc6b29" }, "pygments": { - "https://files.pythonhosted.org/packages/c3/b2/bc9c9196916376152d655522fdcebac55e66de6603a76a02bca1b6414f6c/pygments-2.20.0.tar.gz": "6757cd03768053ff99f3039c1a36d6c0aa0b263438fcab17520b30a303a82b5f", - "https://files.pythonhosted.org/packages/f4/7e/a72dd26f3b0f4f2bf1dd8923c85f7ceb43172af56d63c7383eb62b332364/pygments-2.20.0-py3-none-any.whl": "81a9e26dd42fd28a23a2d169d86d7ac03b46e2f8b59ed4698fb4785f946d0176" + "https://files.pythonhosted.org/packages/c3/b2/bc9c9196916376152d655522fdcebac55e66de6603a76a02bca1b6414f6c/pygments-2.20.0.tar.gz": "sha256:6757cd03768053ff99f3039c1a36d6c0aa0b263438fcab17520b30a303a82b5f", + "https://files.pythonhosted.org/packages/f4/7e/a72dd26f3b0f4f2bf1dd8923c85f7ceb43172af56d63c7383eb62b332364/pygments-2.20.0-py3-none-any.whl": "sha256:81a9e26dd42fd28a23a2d169d86d7ac03b46e2f8b59ed4698fb4785f946d0176" }, "pywin32-ctypes": { - "https://files.pythonhosted.org/packages/85/9f/01a1a99704853cb63f253eea009390c88e7131c67e66a0a02099a8c917cb/pywin32-ctypes-0.2.3.tar.gz": "d162dc04946d704503b2edc4d55f3dba5c1d539ead017afa00142c38b9885755", - "https://files.pythonhosted.org/packages/de/3d/8161f7711c017e01ac9f008dfddd9410dff3674334c233bde66e7ba65bbf/pywin32_ctypes-0.2.3-py3-none-any.whl": "8a1513379d709975552d202d942d9837758905c8d01eb82b8bcc30918929e7b8" + "https://files.pythonhosted.org/packages/85/9f/01a1a99704853cb63f253eea009390c88e7131c67e66a0a02099a8c917cb/pywin32-ctypes-0.2.3.tar.gz": "sha256:d162dc04946d704503b2edc4d55f3dba5c1d539ead017afa00142c38b9885755", + "https://files.pythonhosted.org/packages/de/3d/8161f7711c017e01ac9f008dfddd9410dff3674334c233bde66e7ba65bbf/pywin32_ctypes-0.2.3-py3-none-any.whl": "sha256:8a1513379d709975552d202d942d9837758905c8d01eb82b8bcc30918929e7b8" }, "readme-renderer": { - "https://files.pythonhosted.org/packages/02/51/d3a6ea424652c60f05600d8c2e01a55c913755e7cdad64afabbd1aa16f44/readme_renderer-45.0.tar.gz": "030a8fac74904f8fba11ad1bb6964e3f76e896dc7e5e71f16af190c9056696d1", - "https://files.pythonhosted.org/packages/97/1b/295bf2fa3e740131778065e5ffa2c481f0e7210182d408e9a2c244ff5b0c/readme_renderer-45.0-py3-none-any.whl": "3385ed220117104a2bceb4a9dac8c5fdf6d1f96890d7ea2a9c7174fd5c84091f" + "https://files.pythonhosted.org/packages/02/51/d3a6ea424652c60f05600d8c2e01a55c913755e7cdad64afabbd1aa16f44/readme_renderer-45.0.tar.gz": "sha256:030a8fac74904f8fba11ad1bb6964e3f76e896dc7e5e71f16af190c9056696d1", + "https://files.pythonhosted.org/packages/97/1b/295bf2fa3e740131778065e5ffa2c481f0e7210182d408e9a2c244ff5b0c/readme_renderer-45.0-py3-none-any.whl": "sha256:3385ed220117104a2bceb4a9dac8c5fdf6d1f96890d7ea2a9c7174fd5c84091f" }, "requests": { - "https://files.pythonhosted.org/packages/a0/f4/c67b0b3f1b9245e8d266f0f112c500d50e5b4e83cb6f3b71b6528104182a/requests-2.34.2-py3-none-any.whl": "2a0d60c172f83ac6ab31e4554906c0f3b3588d37b5cb939b1c061f4907e278e0", - "https://files.pythonhosted.org/packages/ac/c3/e2a2b89f2d3e2179abd6d00ebd70bff6273f37fb3e0cc209f48b39d00cbf/requests-2.34.2.tar.gz": "f288924cae4e29463698d6d60bc6a4da69c89185ad1e0bcc4104f584e960b9ed" + "https://files.pythonhosted.org/packages/a0/f4/c67b0b3f1b9245e8d266f0f112c500d50e5b4e83cb6f3b71b6528104182a/requests-2.34.2-py3-none-any.whl": "sha256:2a0d60c172f83ac6ab31e4554906c0f3b3588d37b5cb939b1c061f4907e278e0", + "https://files.pythonhosted.org/packages/ac/c3/e2a2b89f2d3e2179abd6d00ebd70bff6273f37fb3e0cc209f48b39d00cbf/requests-2.34.2.tar.gz": "sha256:f288924cae4e29463698d6d60bc6a4da69c89185ad1e0bcc4104f584e960b9ed" }, "requests-toolbelt": { - "https://files.pythonhosted.org/packages/3f/51/d4db610ef29373b879047326cbf6fa98b6c1969d6f6dc423279de2b1be2c/requests_toolbelt-1.0.0-py2.py3-none-any.whl": "cccfdd665f0a24fcf4726e690f65639d272bb0637b9b92dfd91a5568ccf6bd06", - "https://files.pythonhosted.org/packages/f3/61/d7545dafb7ac2230c70d38d31cbfe4cc64f7144dc41f6e4e4b78ecd9f5bb/requests-toolbelt-1.0.0.tar.gz": "7681a0a3d047012b5bdc0ee37d7f8f07ebe76ab08caeccfc3921ce23c88d5bc6" + "https://files.pythonhosted.org/packages/3f/51/d4db610ef29373b879047326cbf6fa98b6c1969d6f6dc423279de2b1be2c/requests_toolbelt-1.0.0-py2.py3-none-any.whl": "sha256:cccfdd665f0a24fcf4726e690f65639d272bb0637b9b92dfd91a5568ccf6bd06", + "https://files.pythonhosted.org/packages/f3/61/d7545dafb7ac2230c70d38d31cbfe4cc64f7144dc41f6e4e4b78ecd9f5bb/requests-toolbelt-1.0.0.tar.gz": "sha256:7681a0a3d047012b5bdc0ee37d7f8f07ebe76ab08caeccfc3921ce23c88d5bc6" }, "rfc3986": { - "https://files.pythonhosted.org/packages/85/40/1520d68bfa07ab5a6f065a186815fb6610c86fe957bc065754e47f7b0840/rfc3986-2.0.0.tar.gz": "97aacf9dbd4bfd829baad6e6309fa6573aaf1be3f6fa735c8ab05e46cecb261c", - "https://files.pythonhosted.org/packages/ff/9a/9afaade874b2fa6c752c36f1548f718b5b83af81ed9b76628329dab81c1b/rfc3986-2.0.0-py2.py3-none-any.whl": "50b1502b60e289cb37883f3dfd34532b8873c7de9f49bb546641ce9cbd256ebd" + "https://files.pythonhosted.org/packages/85/40/1520d68bfa07ab5a6f065a186815fb6610c86fe957bc065754e47f7b0840/rfc3986-2.0.0.tar.gz": "sha256:97aacf9dbd4bfd829baad6e6309fa6573aaf1be3f6fa735c8ab05e46cecb261c", + "https://files.pythonhosted.org/packages/ff/9a/9afaade874b2fa6c752c36f1548f718b5b83af81ed9b76628329dab81c1b/rfc3986-2.0.0-py2.py3-none-any.whl": "sha256:50b1502b60e289cb37883f3dfd34532b8873c7de9f49bb546641ce9cbd256ebd" }, "rich": { - "https://files.pythonhosted.org/packages/82/3b/64d4899d73f91ba49a8c18a8ff3f0ea8f1c1d75481760df8c68ef5235bf5/rich-15.0.0-py3-none-any.whl": "33bd4ef74232fb73fe9279a257718407f169c09b78a87ad3d296f548e27de0bb", - "https://files.pythonhosted.org/packages/c0/8f/0722ca900cc807c13a6a0c696dacf35430f72e0ec571c4275d2371fca3e9/rich-15.0.0.tar.gz": "edd07a4824c6b40189fb7ac9bc4c52536e9780fbbfbddf6f1e2502c31b068c36" + "https://files.pythonhosted.org/packages/82/3b/64d4899d73f91ba49a8c18a8ff3f0ea8f1c1d75481760df8c68ef5235bf5/rich-15.0.0-py3-none-any.whl": "sha256:33bd4ef74232fb73fe9279a257718407f169c09b78a87ad3d296f548e27de0bb", + "https://files.pythonhosted.org/packages/c0/8f/0722ca900cc807c13a6a0c696dacf35430f72e0ec571c4275d2371fca3e9/rich-15.0.0.tar.gz": "sha256:edd07a4824c6b40189fb7ac9bc4c52536e9780fbbfbddf6f1e2502c31b068c36" }, "secretstorage": { - "https://files.pythonhosted.org/packages/1c/03/e834bcd866f2f8a49a85eaff47340affa3bfa391ee9912a952a1faa68c7b/secretstorage-3.5.0.tar.gz": "f04b8e4689cbce351744d5537bf6b1329c6fc68f91fa666f60a380edddcd11be", - "https://files.pythonhosted.org/packages/b7/46/f5af3402b579fd5e11573ce652019a67074317e18c1935cc0b4ba9b35552/secretstorage-3.5.0-py3-none-any.whl": "0ce65888c0725fcb2c5bc0fdb8e5438eece02c523557ea40ce0703c266248137" + "https://files.pythonhosted.org/packages/1c/03/e834bcd866f2f8a49a85eaff47340affa3bfa391ee9912a952a1faa68c7b/secretstorage-3.5.0.tar.gz": "sha256:f04b8e4689cbce351744d5537bf6b1329c6fc68f91fa666f60a380edddcd11be", + "https://files.pythonhosted.org/packages/b7/46/f5af3402b579fd5e11573ce652019a67074317e18c1935cc0b4ba9b35552/secretstorage-3.5.0-py3-none-any.whl": "sha256:0ce65888c0725fcb2c5bc0fdb8e5438eece02c523557ea40ce0703c266248137" }, "six": { - "https://files.pythonhosted.org/packages/94/e7/b2c673351809dca68a0e064b6af791aa332cf192da575fd474ed7d6f16a2/six-1.17.0.tar.gz": "ff70335d468e7eb6ec65b95b99d3a2836546063f63acc5171de367e834932a81", - "https://files.pythonhosted.org/packages/b7/ce/149a00dd41f10bc29e5921b496af8b574d8413afcd5e30dfa0ed46c2cc5e/six-1.17.0-py2.py3-none-any.whl": "4721f391ed90541fddacab5acf947aa0d3dc7d27b2e1e8eda2be8970586c3274" + "https://files.pythonhosted.org/packages/94/e7/b2c673351809dca68a0e064b6af791aa332cf192da575fd474ed7d6f16a2/six-1.17.0.tar.gz": "sha256:ff70335d468e7eb6ec65b95b99d3a2836546063f63acc5171de367e834932a81", + "https://files.pythonhosted.org/packages/b7/ce/149a00dd41f10bc29e5921b496af8b574d8413afcd5e30dfa0ed46c2cc5e/six-1.17.0-py2.py3-none-any.whl": "sha256:4721f391ed90541fddacab5acf947aa0d3dc7d27b2e1e8eda2be8970586c3274" }, "twine": { - "https://files.pythonhosted.org/packages/92/3c/58f808a359700f39a967dffede33efeac809262c03303fa3eec6afff8f49/twine-7.0.0.tar.gz": "85cdb29c518efef867360ae4acd4b0dfd61c8654a22fca08e6f8539f05022177", - "https://files.pythonhosted.org/packages/96/08/ddcdc06225eaad6de0e48e1002b06d919dbde20582d0662c7af51308e5d6/twine-7.0.0-py3-none-any.whl": "b854164df26db268af05f49aa5c0344b10e27a494343ff05b1e0bad3b135f5a7" + "https://files.pythonhosted.org/packages/92/3c/58f808a359700f39a967dffede33efeac809262c03303fa3eec6afff8f49/twine-7.0.0.tar.gz": "sha256:85cdb29c518efef867360ae4acd4b0dfd61c8654a22fca08e6f8539f05022177", + "https://files.pythonhosted.org/packages/96/08/ddcdc06225eaad6de0e48e1002b06d919dbde20582d0662c7af51308e5d6/twine-7.0.0-py3-none-any.whl": "sha256:b854164df26db268af05f49aa5c0344b10e27a494343ff05b1e0bad3b135f5a7" }, "urllib3": { - "https://files.pythonhosted.org/packages/53/0c/06f8b233b8fd13b9e5ee11424ef85419ba0d8ba0b3138bf360be2ff56953/urllib3-2.7.0.tar.gz": "231e0ec3b63ceb14667c67be60f2f2c40a518cb38b03af60abc813da26505f4c", - "https://files.pythonhosted.org/packages/7f/3e/5db95bcf282c52709639744ca2a8b149baccf648e39c8cc87553df9eae0c/urllib3-2.7.0-py3-none-any.whl": "9fb4c81ebbb1ce9531cce37674bbc6f1360472bc18ca9a553ede278ef7276897" + "https://files.pythonhosted.org/packages/53/0c/06f8b233b8fd13b9e5ee11424ef85419ba0d8ba0b3138bf360be2ff56953/urllib3-2.7.0.tar.gz": "sha256:231e0ec3b63ceb14667c67be60f2f2c40a518cb38b03af60abc813da26505f4c", + "https://files.pythonhosted.org/packages/7f/3e/5db95bcf282c52709639744ca2a8b149baccf648e39c8cc87553df9eae0c/urllib3-2.7.0-py3-none-any.whl": "sha256:9fb4c81ebbb1ce9531cce37674bbc6f1360472bc18ca9a553ede278ef7276897" }, "zipp": { - "https://files.pythonhosted.org/packages/3a/13/547360d81e6d88d58492968ffda9f9542854f11310ee556fef14260cc886/zipp-4.1.0-py3-none-any.whl": "25ad4e16390cd314347dd8f1de67a2ac538ae658ed4ab9db16029c07c188e97f", - "https://files.pythonhosted.org/packages/b9/d8/eab98a517c14134c0b2eb4e2387bc5f457334293ec5d2dd3857ec2966802/zipp-4.1.0.tar.gz": "4cb57381f544315db7688e976e922a2b18cdb513d21cc194eb42232ba2a3e602" + "https://files.pythonhosted.org/packages/3a/13/547360d81e6d88d58492968ffda9f9542854f11310ee556fef14260cc886/zipp-4.1.0-py3-none-any.whl": "sha256:25ad4e16390cd314347dd8f1de67a2ac538ae658ed4ab9db16029c07c188e97f", + "https://files.pythonhosted.org/packages/b9/d8/eab98a517c14134c0b2eb4e2387bc5f457334293ec5d2dd3857ec2966802/zipp-4.1.0.tar.gz": "sha256:4cb57381f544315db7688e976e922a2b18cdb513d21cc194eb42232ba2a3e602" } } }, - "fact_version": "v1" + "fact_version": "v2" } } } diff --git a/tests/pypi/hash/BUILD.bazel b/tests/pypi/hash/BUILD.bazel new file mode 100644 index 0000000000..1e052af141 --- /dev/null +++ b/tests/pypi/hash/BUILD.bazel @@ -0,0 +1,3 @@ +load(":hash_tests.bzl", "hash_test_suite") + +hash_test_suite(name = "hash_tests") diff --git a/tests/pypi/hash/hash_tests.bzl b/tests/pypi/hash/hash_tests.bzl new file mode 100644 index 0000000000..6866a79d07 --- /dev/null +++ b/tests/pypi/hash/hash_tests.bzl @@ -0,0 +1,102 @@ +# Copyright 2026 The Bazel Authors. All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +"" + +load("@rules_testing//lib:test_suite.bzl", "test_suite") +load("//python/private/pypi:hash.bzl", "hash") # buildifier: disable=bzl-visibility + +_tests = [] + +# The digests of b"rules_python" with the SRI values calculated with: +# python3 -c 'import base64, hashlib; h = hashlib.sha512(b"rules_python").digest(); print(base64.b64encode(h))' +_SHA256 = "a90bb21ddf552d508565aa2f0d78ac13297e6c407e39eb578eeac09e62c5da3a" +_SHA256_SRI = "sha256-qQuyHd9VLVCFZaovDXisEyl+bEB+OetXjurAnmLF2jo=" +_SHA384 = "7407ad30f83623cabf9139bd80edb332516880321ab2a20fa0f281767675bfc10b53555eb5042aa58de4b9345f5f70b6" +_SHA384_SRI = "sha384-dAetMPg2I8q/kTm9gO2zMlFogDIasqIPoPKBdnZ1v8ELU1VetQQqpY3kuTRfX3C2" +_SHA512 = "127af31fba52ce5ac96a4bc0f1b31c31c12367feace2b7dbb81066df479883add9dd6166495b126b76ac8c624d254b6a4ba3fbb628e8dcf0a34e1b0b065f619f" +_SHA512_SRI = "sha512-EnrzH7pSzlrJakvA8bMcMcEjZ/6s4rfbuBBm30eYg63Z3WFmSVsSa3asjGJNJUtqS6P7tijo3PCjThsLBl9hnw==" + +def _test_digest(env): + env.expect.that_str(hash.digest("sha256", _SHA256)).equals("sha256:" + _SHA256) + + # The algorithm name is normalized to lower case. + env.expect.that_str(hash.digest("SHA256", _SHA256)).equals("sha256:" + _SHA256) + + # Unknown algorithms and empty digests yield nothing. + env.expect.that_str(hash.digest("egg", "foo")).equals("") + env.expect.that_str(hash.digest("sha256", "")).equals("") + env.expect.that_str(hash.digest("", "")).equals("") + +_tests.append(_test_digest) + +def _test_hex_to_sri(env): + env.expect.that_str(hash.hex_to_sri("sha256", _SHA256)).equals(_SHA256_SRI) + env.expect.that_str(hash.hex_to_sri("sha384", _SHA384)).equals(_SHA384_SRI) + env.expect.that_str(hash.hex_to_sri("sha512", _SHA512)).equals(_SHA512_SRI) + + # Upper case digests are accepted. + env.expect.that_str(hash.hex_to_sri("sha512", _SHA512.upper())).equals(_SHA512_SRI) + + # Algorithms that SRI does not support yield nothing. + env.expect.that_str(hash.hex_to_sri("md5", "0" * 32)).equals("") + env.expect.that_str(hash.hex_to_sri("blake2b", "0" * 128)).equals("") + + # Invalid digests yield nothing. + env.expect.that_str(hash.hex_to_sri("sha256", "")).equals("") + env.expect.that_str(hash.hex_to_sri("sha256", "abc")).equals("") + env.expect.that_str(hash.hex_to_sri("sha256", "not-hex!")).equals("") + +_tests.append(_test_hex_to_sri) + +def _test_integrity(env): + env.expect.that_str(hash.integrity("")).equals("") + env.expect.that_str(hash.integrity("md5:" + "0" * 32)).equals("") + env.expect.that_str(hash.integrity("sha256:" + _SHA256)).equals(_SHA256_SRI) + env.expect.that_str(hash.integrity("sha384:" + _SHA384)).equals(_SHA384_SRI) + env.expect.that_str(hash.integrity("sha512:" + _SHA512)).equals(_SHA512_SRI) + +_tests.append(_test_integrity) + +def _test_preferred_digest(env): + env.expect.that_str(hash.preferred_digest([])).equals("") + env.expect.that_str(hash.preferred_digest([""])).equals("") + + # sha256 stays preferred so that the repo names remain stable. + env.expect.that_str(hash.preferred_digest([ + "sha256:" + _SHA256, + "sha512:" + _SHA512, + ])).equals("sha256:" + _SHA256) + + # Otherwise the strongest SRI supported algorithm wins. + env.expect.that_str(hash.preferred_digest([ + "sha384:" + _SHA384, + "sha512:" + _SHA512, + ])).equals("sha512:" + _SHA512) + + # And anything else is picked alphabetically for determinism. + env.expect.that_str(hash.preferred_digest([ + "md5:deadb00f", + "blake2b:deadbeef", + ])).equals("blake2b:deadbeef") + +_tests.append(_test_preferred_digest) + +def hash_test_suite(name): + """Create the test suite. + + Args: + name: the name of the test suite + """ + test_suite(name = name, basic_tests = _tests) diff --git a/tests/pypi/hub_builder/hub_builder_tests.bzl b/tests/pypi/hub_builder/hub_builder_tests.bzl index 4651342dd4..f421c7e351 100644 --- a/tests/pypi/hub_builder/hub_builder_tests.bzl +++ b/tests/pypi/hub_builder/hub_builder_tests.bzl @@ -340,8 +340,8 @@ def _test_simple_extras_vs_no_extras_simpleapi(env): "dep_template": "@pypi//{name}:{target}", "filename": "simple-0.0.1-py3-none-any.whl", "index_url": "https://example.com/simple/", + "integrity": "sha256-3q2+7w==", "requirement": "simple[foo]==0.0.1", - "sha256": "deadbeef", "urls": ["/simple-0.0.1-py3-none-any.whl"], }, "pypi_315_simple_py3_none_any_deadbeef_windows_aarch64": { @@ -349,8 +349,8 @@ def _test_simple_extras_vs_no_extras_simpleapi(env): "dep_template": "@pypi//{name}:{target}", "filename": "simple-0.0.1-py3-none-any.whl", "index_url": "https://example.com/simple/", + "integrity": "sha256-3q2+7w==", "requirement": "simple==0.0.1", - "sha256": "deadbeef", "urls": ["/simple-0.0.1-py3-none-any.whl"], }, }) @@ -358,6 +358,77 @@ def _test_simple_extras_vs_no_extras_simpleapi(env): _tests.append(_test_simple_extras_vs_no_extras_simpleapi) +def _test_simple_sha512_simpleapi(env): + """Non-sha256 pins match the index digests and are downloaded with `integrity`.""" + + def mockread_simpleapi(*_, parse_index, **__): + if parse_index: + content = """\ + simple-0.0.1-py3-none-any.whl
+""" + return struct( + output = parse_simpleapi_html( + content = content, + parse_index = parse_index, + ), + success = True, + ) + + builder = hub_builder( + env, + simpleapi_download_fn = lambda *args, **kwargs: simpleapi_download( + read_simpleapi = mockread_simpleapi, + *args, + **kwargs + ), + ) + builder.pip_parse( + mocks.mctx( + mock_files = { + "win.txt": "simple==0.0.1 --hash=sha512:deadbeef", + }, + ), + _parse( + hub_name = "pypi", + python_version = "3.15", + requirements_windows = "win.txt", + experimental_index_url = "https://example.com", + target_platforms = ["windows_aarch64"], + ), + ) + pypi = builder.build() + + pypi.exposed_packages().contains_exactly(["simple"]) + pypi.whl_map().contains_exactly({ + "simple": { + "pypi_315_simple_py3_none_any_deadbeef": [ + whl_config_setting( + target_platforms = [ + "cp315_windows_aarch64", + ], + version = "3.15", + ), + ], + }, + }) + pypi.whl_libraries().contains_exactly({ + "pypi_315_simple_py3_none_any_deadbeef": { + "config_load": "@pypi//:config.bzl", + "dep_template": "@pypi//{name}:{target}", + "filename": "simple-0.0.1-py3-none-any.whl", + "index_url": "https://example.com/simple/", + "integrity": "sha512-3q2+7w==", + "requirement": "simple==0.0.1", + "urls": ["/simple-0.0.1-py3-none-any.whl"], + }, + }) + +_tests.append(_test_simple_sha512_simpleapi) + def _test_simple_multiple_python_versions(env): builder = hub_builder( env, @@ -672,8 +743,8 @@ torch==2.4.1+cpu ; platform_machine == 'x86_64' \ "dep_template": "@pypi//{name}:{target}", "filename": "torch-2.4.1+cpu-cp312-cp312-linux_x86_64.whl", "index_url": "https://torch.index/torch/", + "integrity": "sha256-iADe7wAmAR1QLAwlbMS2fQAjR/Y8OjjNjkXx9EXGE2Q=", "requirement": "torch==2.4.1+cpu", - "sha256": "8800deef0026011d502c0c256cc4b67d002347f63c3a38cd8e45f1f445c61364", "urls": ["/whl/cpu/torch-2.4.1%2Bcpu-cp312-cp312-linux_x86_64.whl"], }, "pypi_312_torch_cp312_cp312_manylinux_2_17_aarch64_36109432_linux_aarch64": { @@ -681,8 +752,8 @@ torch==2.4.1+cpu ; platform_machine == 'x86_64' \ "dep_template": "@pypi//{name}:{target}", "filename": "torch-2.4.1-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", "index_url": "https://torch.index/torch/", + "integrity": "sha256-NhCUMrEL1xY8mzDOiW88LMobhrl2X5VqFZTw/0MJHio=", "requirement": "torch==2.4.1", - "sha256": "36109432b10bd7163c9b30ce896f3c2cca1b86b9765f956a1594f0ff43091e2a", "urls": ["/whl/cpu/torch-2.4.1-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl"], }, "pypi_312_torch_cp312_cp312_win_amd64_3a570e5c_windows_x86_64": { @@ -690,8 +761,8 @@ torch==2.4.1+cpu ; platform_machine == 'x86_64' \ "dep_template": "@pypi//{name}:{target}", "filename": "torch-2.4.1+cpu-cp312-cp312-win_amd64.whl", "index_url": "https://torch.index/torch/", + "integrity": "sha256-OlcOXFU0Fc293+Z5IHMns6OAayHGreoU+6d2hNFhnpc=", "requirement": "torch==2.4.1+cpu", - "sha256": "3a570e5c553415cdbddfe679207327b3a3806b21c6adea14fba77684d1619e97", "urls": ["/whl/cpu/torch-2.4.1%2Bcpu-cp312-cp312-win_amd64.whl"], }, "pypi_312_torch_cp312_none_macosx_11_0_arm64_72b484d5_osx_aarch64": { @@ -699,8 +770,8 @@ torch==2.4.1+cpu ; platform_machine == 'x86_64' \ "dep_template": "@pypi//{name}:{target}", "filename": "torch-2.4.1-cp312-none-macosx_11_0_arm64.whl", "index_url": "https://torch.index/torch/", + "integrity": "sha256-crSE1bbOwac1vz+locSIPQF0hpjF6c/b60/6t8eYfg0=", "requirement": "torch==2.4.1", - "sha256": "72b484d5b6cec1a735bf3fa5a1c4883d01748698c5e9cfdbeb4ffab7c7987e0d", "urls": ["/whl/cpu/torch-2.4.1-cp312-none-macosx_11_0_arm64.whl"], }, }) @@ -787,15 +858,15 @@ simple==0.0.1 --hash=sha256:deadb00f return { "simple": struct( whls = { - "deadb00f": struct( + "sha256:deadb00f": struct( yanked = None, filename = "simple-0.0.1-py3-none-any.whl", - sha256 = "deadb00f", + digest = "sha256:deadb00f", url = test.expect_url, ), }, sdists = {}, - sha256s_by_version = {}, + hashes_by_version = {}, index_url = test.expect_index_url, ), } @@ -842,8 +913,8 @@ simple==0.0.1 --hash=sha256:deadb00f "dep_template": "@pypi//{name}:{target}", "filename": "simple-0.0.1-py3-none-any.whl", "index_url": test.expect_index_url, + "integrity": "sha256-3q2wDw==", "requirement": "simple==0.0.1", - "sha256": "deadb00f", "urls": [test.expect_url], } if getattr(test, "envsubst", []): @@ -977,33 +1048,33 @@ def _test_simple_get_index(env): return { "plat_pkg": struct( whls = { - "deadb44f": struct( + "sha256:deadb44f": struct( yanked = None, filename = "plat-pkg-0.0.4-py3-none-linux_x86_64.whl", - sha256 = "deadb44f", + digest = "sha256:deadb44f", url = "example2.org/index/plat_pkg/", ), }, sdists = {}, - sha256s_by_version = { - "0.0.4": ["deadb44f"], + hashes_by_version = { + "0.0.4": ["sha256:deadb44f"], }, index_url = "https://pypi.org/simple", ), "simple": struct( whls = { - "deadb00f": struct( + "sha256:deadb00f": struct( yanked = None, filename = "simple-0.0.1-py3-none-any.whl", - sha256 = "deadb00f", + digest = "sha256:deadb00f", url = "example2.org", ), }, sdists = { - "deadbeef": struct( + "sha256:deadbeef": struct( yanked = None, filename = "simple-0.0.1.tar.gz", - sha256 = "deadbeef", + digest = "sha256:deadbeef", url = "example.org", ), }, @@ -1011,17 +1082,17 @@ def _test_simple_get_index(env): ), "some_other_pkg": struct( whls = { - "deadb33f": struct( + "sha256:deadb33f": struct( yanked = None, filename = "some-other-pkg-0.0.1-py3-none-any.whl", - sha256 = "deadb33f", + digest = "sha256:deadb33f", url = "example2.org/index/some_other_pkg/", ), }, sdists = {}, - sha256s_by_version = { - "0.0.1": ["deadb33f"], - "0.0.3": ["deadbeef"], + hashes_by_version = { + "0.0.1": ["sha256:deadb33f"], + "0.0.3": ["sha256:deadbeef"], }, index_url = "https://with_index_url", ), @@ -1186,17 +1257,17 @@ git_dep @ git+https://git.server/repo/project@deadbeefdeadbeef "dep_template": "@pypi//{name}:{target}", "extra_pip_args": ["--extra-args-for-sdist-building"], "filename": "any-name.tar.gz", + "integrity": "", "python_interpreter_target": "unit_test_interpreter_target", "requirement": "direct_sdist_without_sha @ some-archive/any-name.tar.gz", - "sha256": "", "urls": ["some-archive/any-name.tar.gz"], }, "pypi_315_direct_without_sha_0_0_1_py3_none_any": { "config_load": "@pypi//:config.bzl", "dep_template": "@pypi//{name}:{target}", "filename": "direct_without_sha-0.0.1-py3-none-any.whl", + "integrity": "", "requirement": "direct_without_sha==0.0.1", - "sha256": "", "urls": ["example-direct.org/direct_without_sha-0.0.1-py3-none-any.whl"], "whl_patches": {"my_patch": "1"}, }, @@ -1219,8 +1290,8 @@ git_dep @ git+https://git.server/repo/project@deadbeefdeadbeef "dep_template": "@pypi//{name}:{target}", "filename": "plat-pkg-0.0.4-py3-none-linux_x86_64.whl", "index_url": "https://pypi.org/simple", + "integrity": "sha256-3q20Tw==", "requirement": "plat_pkg==0.0.4", - "sha256": "deadb44f", "urls": ["example2.org/index/plat_pkg/"], }, "pypi_315_simple_py3_none_any_deadb00f": { @@ -1228,16 +1299,16 @@ git_dep @ git+https://git.server/repo/project@deadbeefdeadbeef "dep_template": "@pypi//{name}:{target}", "filename": "simple-0.0.1-py3-none-any.whl", "index_url": "https://pypi.org/simple", + "integrity": "sha256-3q2wDw==", "requirement": "simple==0.0.1", - "sha256": "deadb00f", "urls": ["example2.org"], }, "pypi_315_some_pkg_py3_none_any_deadbaaf": { "config_load": "@pypi//:config.bzl", "dep_template": "@pypi//{name}:{target}", "filename": "some_pkg-0.0.1-py3-none-any.whl", + "integrity": "sha256-3q26rw==", "requirement": "some_pkg==0.0.1", - "sha256": "deadbaaf", "urls": ["example-direct.org/some_pkg-0.0.1-py3-none-any.whl"], }, "pypi_315_some_py3_none_any_deadb33f": { @@ -1245,8 +1316,8 @@ git_dep @ git+https://git.server/repo/project@deadbeefdeadbeef "dep_template": "@pypi//{name}:{target}", "filename": "some-other-pkg-0.0.1-py3-none-any.whl", "index_url": "https://with_index_url", + "integrity": "sha256-3q2zPw==", "requirement": "some_other_pkg==0.0.1", - "sha256": "deadb33f", "urls": ["example2.org/index/some_other_pkg/"], }, }) diff --git a/tests/pypi/index_sources/index_sources_tests.bzl b/tests/pypi/index_sources/index_sources_tests.bzl index 7aa22d164a..3a605b9eb9 100644 --- a/tests/pypi/index_sources/index_sources_tests.bzl +++ b/tests/pypi/index_sources/index_sources_tests.bzl @@ -26,7 +26,7 @@ def _test_no_simple_api_sources(env): requirement_line = "foo @ git+https://github.com/org/foo.git@deadbeef", marker = "", url = "git+https://github.com/org/foo.git@deadbeef", - shas = [], + hashes = [], version = "", filename = "", ), @@ -59,7 +59,7 @@ def _test_no_simple_api_sources(env): requirement_line = "foo==0.0.1 @ https://someurl.org/package.whl --hash=sha256:deadbeef", marker = "", url = "https://someurl.org/package.whl", - shas = ["deadbeef"], + hashes = ["sha256:deadbeef"], version = "0.0.1", filename = "package.whl", ), @@ -68,7 +68,7 @@ def _test_no_simple_api_sources(env): requirement_line = "foo==0.0.1 @ https://someurl.org/package.whl --hash=sha256:deadbeef", marker = "python_version < \"2.7\"", url = "https://someurl.org/package.whl", - shas = ["deadbeef"], + hashes = ["sha256:deadbeef"], version = "0.0.1", filename = "package.whl", ), @@ -80,7 +80,7 @@ def _test_no_simple_api_sources(env): requirement_line = "foo[extra] @ https://example.org/foo-1.0.tar.gz --hash=sha256:deadbe0f", marker = "", url = "https://example.org/foo-1.0.tar.gz", - shas = ["deadbe0f"], + hashes = ["sha256:deadbe0f"], version = "", filename = "foo-1.0.tar.gz", ), @@ -89,14 +89,14 @@ def _test_no_simple_api_sources(env): requirement_line = "torch @ https://download.pytorch.org/whl/cpu/torch-2.6.0%2Bcpu-cp311-cp311-linux_x86_64.whl#sha256=deadbeef", marker = "", url = "https://download.pytorch.org/whl/cpu/torch-2.6.0%2Bcpu-cp311-cp311-linux_x86_64.whl", - shas = ["deadbeef"], + hashes = ["sha256:deadbeef"], version = "", filename = "torch-2.6.0+cpu-cp311-cp311-linux_x86_64.whl", ), } for input, want in inputs.items(): got = index_sources(input) - env.expect.that_collection(got.shas).contains_exactly(want.shas if hasattr(want, "shas") else []) + env.expect.that_collection(got.hashes).contains_exactly(want.hashes if hasattr(want, "hashes") else []) env.expect.that_str(got.version).equals(want.version) env.expect.that_str(got.requirement).equals(want.requirement) env.expect.that_str(got.requirement_line).equals(got.requirement_line) @@ -109,9 +109,9 @@ _tests.append(_test_no_simple_api_sources) def _test_simple_api_sources(env): tests = { "foo==0.0.2 --hash=sha256:deafbeef --hash=sha256:deadbeef": struct( - shas = [ - "deadbeef", - "deafbeef", + hashes = [ + "sha256:deadbeef", + "sha256:deafbeef", ], marker = "", requirement = "foo==0.0.2", @@ -119,9 +119,9 @@ def _test_simple_api_sources(env): url = "", ), "foo[extra]==0.0.2; (python_version < 2.7 or extra == \"@\") --hash=sha256:deafbeef --hash=sha256:deadbeef": struct( - shas = [ - "deadbeef", - "deafbeef", + hashes = [ + "sha256:deadbeef", + "sha256:deafbeef", ], marker = "(python_version < 2.7 or extra == \"@\")", requirement = "foo[extra]==0.0.2", @@ -131,7 +131,7 @@ def _test_simple_api_sources(env): } for input, want in tests.items(): got = index_sources(input) - env.expect.that_collection(got.shas).contains_exactly(want.shas) + env.expect.that_collection(got.hashes).contains_exactly(want.hashes) env.expect.that_str(got.version).equals("0.0.2") env.expect.that_str(got.requirement).equals(want.requirement) env.expect.that_str(got.requirement_line).equals(want.requirement_line) @@ -140,6 +140,62 @@ def _test_simple_api_sources(env): _tests.append(_test_simple_api_sources) +def _test_non_sha256_hashes(env): + tests = { + # A `#sha512=` URL fragment, as any hash algorithm is allowed by PEP 503. + "foo @ https://example.org/foo-0.0.1-py3-none-any.whl#sha512=deadbeef": struct( + hashes = ["sha512:deadbeef"], + marker = "", + requirement = "foo", + requirement_line = "foo @ https://example.org/foo-0.0.1-py3-none-any.whl#sha512=deadbeef", + url = "https://example.org/foo-0.0.1-py3-none-any.whl", + version = "", + filename = "foo-0.0.1-py3-none-any.whl", + ), + # Unknown algorithms are dropped. + "foo==0.0.2 --hash=egg:deadbeef": struct( + hashes = [], + marker = "", + requirement = "foo==0.0.2", + requirement_line = "foo==0.0.2", + url = "", + version = "0.0.2", + filename = "", + ), + "foo==0.0.2 --hash=sha512:deadbeef": struct( + hashes = ["sha512:deadbeef"], + marker = "", + requirement = "foo==0.0.2", + requirement_line = "foo==0.0.2 --hash=sha512:deadbeef", + url = "", + version = "0.0.2", + filename = "", + ), + "foo==0.0.2 --hash=sha512:deafbeef --hash=sha256:deadbeef": struct( + hashes = [ + "sha256:deadbeef", + "sha512:deafbeef", + ], + marker = "", + requirement = "foo==0.0.2", + requirement_line = "foo==0.0.2 --hash=sha512:deafbeef --hash=sha256:deadbeef", + url = "", + version = "0.0.2", + filename = "", + ), + } + for input, want in tests.items(): + got = index_sources(input) + env.expect.that_collection(got.hashes).contains_exactly(want.hashes) + env.expect.that_str(got.version).equals(want.version) + env.expect.that_str(got.requirement).equals(want.requirement) + env.expect.that_str(got.requirement_line).equals(want.requirement_line) + env.expect.that_str(got.marker).equals(want.marker) + env.expect.that_str(got.url).equals(want.url) + env.expect.that_str(got.filename).equals(want.filename) + +_tests.append(_test_non_sha256_hashes) + def index_sources_test_suite(name): """Create the test suite. diff --git a/tests/pypi/parse_requirements/parse_requirements_tests.bzl b/tests/pypi/parse_requirements/parse_requirements_tests.bzl index 57b3ea5d3e..54f906929a 100644 --- a/tests/pypi/parse_requirements/parse_requirements_tests.bzl +++ b/tests/pypi/parse_requirements/parse_requirements_tests.bzl @@ -92,6 +92,9 @@ foo==0.0.3 --hash=sha256:deadbaaf --hash=sha256:deadb11f --hash=sha256:5d15t --abi=cp39 foo==0.0.3 --hash=sha256:deadbaaf +""", + "requirements_sha512": """\ +foo==0.0.1 --hash=sha512:deadbeef """, "requirements_windows": """\ foo[extra]==0.0.2 --hash=sha256:deadbeef @@ -108,6 +111,7 @@ bar==0.0.1 --hash=sha256:deadb00f "uv_lock_foo_requires_dist_extras": """{"package":[{"name":"foo","version":"0.0.1","source":{"registry":"https://pypi.org/simple"},"wheels":[{"hash":"sha256:deadbeef","url":"https://files.pythonhosted.org/packages/foo-0.0.1-py3-none-any.whl"}]},{"name":"root-pkg","source":{"virtual":"."},"version":"0.0.0","dependencies":[{"name":"foo"}],"metadata":{"requires-dist":[{"name":"foo","extras":["all"]}]}}]}""", "uv_lock_foo_resolution_markers_dedup": """{"package":[{"name":"foo","source":{"registry":"https://pypi.org/simple"},"version":"0.0.1","resolution-markers":["sys_platform == 'linux'"],"wheels":[{"hash":"sha256:aaa","url":"https://files.pythonhosted.org/packages/foo-0.0.1-cp39-cp39-manylinux_2_17_x86_64.whl"},{"hash":"sha256:bbb","url":"https://files.pythonhosted.org/packages/foo-0.0.1-py3-none-any.whl"}]},{"name":"foo","source":{"registry":"https://pypi.org/simple"},"version":"0.0.2","resolution-markers":["sys_platform == 'darwin'"],"wheels":[{"hash":"sha256:ccc","url":"https://files.pythonhosted.org/packages/foo-0.0.2-cp39-cp39-macosx_11_0_arm64.whl"},{"hash":"sha256:ddd","url":"https://files.pythonhosted.org/packages/foo-0.0.2-py3-none-any.whl"}]}]}""", "uv_lock_foo_sdist": """{"package":[{"name":"foo","sdist":{"hash":"sha256:feedcafe","url":"https://files.pythonhosted.org/packages/foo-0.0.1.tar.gz"},"source":{"registry":"https://pypi.org/simple"},"version":"0.0.1","wheels":[{"hash":"sha256:deadbeef","url":"https://files.pythonhosted.org/packages/foo-0.0.1-py3-none-any.whl"}]}]}""", + "uv_lock_foo_sha512": """{"package":[{"name":"foo","source":{"registry":"https://pypi.org/simple"},"version":"0.0.1","wheels":[{"hash":"sha512:deadbeef","url":"https://files.pythonhosted.org/packages/foo-0.0.1-py3-none-any.whl"}]}]}""", "uv_lock_foo_virtual": """{"package":[{"name":"foo","source":{"registry":"https://pypi.org/simple"},"version":"0.0.1","wheels":[{"hash":"sha256:deadbeef","url":"https://files.pythonhosted.org/packages/foo-0.0.1-py3-none-any.whl"}]},{"name":"virtual-pkg","source":{"virtual":true},"version":"0.0.0"}]}""", "uv_lock_foo_with_extras": """{"package":[{"name":"foo","provides-extras":["extra"],"source":{"registry":"https://pypi.org/simple"},"version":"0.0.1","wheels":[{"hash":"sha256:deadbeef","url":"https://files.pythonhosted.org/packages/foo-0.0.1-py3-none-any.whl"}]}]}""", "uv_lock_git_vcs": """{"package":[{"name":"foo","source":{"git":"https://github.com/org/foo.git"},"version":"0.1.0"}]}""", @@ -192,7 +196,7 @@ def _test_simple(env): ], url = "", filename = "", - sha256 = "", + digest = "", yanked = None, ), ], @@ -221,7 +225,7 @@ def _test_direct_urls_integration(env): extra_pip_args = [], filename = "foo-1.1.tar.gz", requirement_line = "foo @ https://github.com/org/foo/downloads/foo-1.1.tar.gz", - sha256 = "", + digest = "", target_platforms = ["osx_x86_64"], url = "https://github.com/org/foo/downloads/foo-1.1.tar.gz", yanked = None, @@ -231,7 +235,7 @@ def _test_direct_urls_integration(env): extra_pip_args = [], filename = "package.whl", requirement_line = "foo[extra]", - sha256 = "", + digest = "", target_platforms = ["linux_x86_64"], url = "https://some-url/package.whl", yanked = None, @@ -263,7 +267,7 @@ def _test_direct_urls_no_extract(env): extra_pip_args = [], filename = "", requirement_line = "foo @ https://github.com/org/foo/downloads/foo-1.1.tar.gz", - sha256 = "", + digest = "", target_platforms = ["osx_x86_64"], url = "", yanked = None, @@ -273,7 +277,7 @@ def _test_direct_urls_no_extract(env): extra_pip_args = [], filename = "", requirement_line = "foo[extra] @ https://some-url/package.whl", - sha256 = "", + digest = "", target_platforms = ["linux_x86_64"], url = "", yanked = None, @@ -308,7 +312,7 @@ def _test_extra_pip_args(env): ], url = "", filename = "", - sha256 = "", + digest = "", yanked = None, ), ], @@ -338,7 +342,7 @@ def _test_dupe_requirements(env): target_platforms = ["linux_x86_64"], url = "", filename = "", - sha256 = "", + digest = "", yanked = None, ), ], @@ -370,7 +374,7 @@ def _test_multi_os(env): target_platforms = ["windows_x86_64"], url = "", filename = "", - sha256 = "", + digest = "", yanked = None, ), ], @@ -388,7 +392,7 @@ def _test_multi_os(env): target_platforms = ["linux_x86_64"], url = "", filename = "", - sha256 = "", + digest = "", yanked = None, ), struct( @@ -398,7 +402,7 @@ def _test_multi_os(env): target_platforms = ["windows_x86_64"], url = "", filename = "", - sha256 = "", + digest = "", yanked = None, ), ], @@ -436,7 +440,7 @@ def _test_multi_os_legacy(env): target_platforms = ["cp39_linux_x86_64"], url = "", filename = "", - sha256 = "", + digest = "", yanked = None, ), ], @@ -454,7 +458,7 @@ def _test_multi_os_legacy(env): target_platforms = ["cp39_linux_x86_64"], url = "", filename = "", - sha256 = "", + digest = "", yanked = None, ), struct( @@ -464,7 +468,7 @@ def _test_multi_os_legacy(env): target_platforms = ["cp39_osx_aarch64"], url = "", filename = "", - sha256 = "", + digest = "", yanked = None, ), ], @@ -522,7 +526,7 @@ def _test_env_marker_resolution(env): target_platforms = ["cp311_linux_super_exotic", "cp311_windows_x86_64"], url = "", filename = "", - sha256 = "", + digest = "", yanked = None, ), ], @@ -540,7 +544,7 @@ def _test_env_marker_resolution(env): target_platforms = ["cp311_windows_x86_64"], url = "", filename = "", - sha256 = "", + digest = "", yanked = None, ), ], @@ -571,7 +575,7 @@ def _test_different_package_version(env): target_platforms = ["linux_aarch64"], url = "", filename = "", - sha256 = "", + digest = "", yanked = None, ), struct( @@ -581,7 +585,7 @@ def _test_different_package_version(env): target_platforms = ["linux_x86_64"], url = "", filename = "", - sha256 = "", + digest = "", yanked = None, ), ], @@ -612,7 +616,7 @@ def _test_different_package_extras(env): target_platforms = ["linux_aarch64"], url = "", filename = "", - sha256 = "", + digest = "", yanked = None, ), struct( @@ -622,7 +626,7 @@ def _test_different_package_extras(env): target_platforms = ["linux_x86_64"], url = "", filename = "", - sha256 = "", + digest = "", yanked = None, ), ], @@ -652,7 +656,7 @@ def _test_optional_hash(env): target_platforms = ["linux_x86_64"], url = "https://example.org/bar-0.0.4.whl", filename = "bar-0.0.4.whl", - sha256 = "", + digest = "", yanked = None, ), ], @@ -670,7 +674,7 @@ def _test_optional_hash(env): target_platforms = ["linux_x86_64"], url = "https://example.org/foo-0.0.5.whl", filename = "foo-0.0.5.whl", - sha256 = "deadbeef", + digest = "sha256:deadbeef", yanked = None, ), ], @@ -700,7 +704,7 @@ def _test_git_sources(env): target_platforms = ["linux_x86_64"], url = "", filename = "", - sha256 = "", + digest = "", yanked = None, ), ], @@ -740,23 +744,23 @@ def _test_overlapping_shas_with_index_results(env): "foo": struct( index_url = "https://example.com", sdists = { - "5d15t": struct( + "sha256:5d15t": struct( url = "sdist", - sha256 = "5d15t", + digest = "sha256:5d15t", filename = "foo-0.0.1.tar.gz", yanked = None, ), }, whls = { - "deadb11f": struct( + "sha256:deadb11f": struct( url = "super2", - sha256 = "deadb11f", + digest = "sha256:deadb11f", filename = "foo-0.0.1-py3-none-macosx_14_0_x86_64.whl", yanked = None, ), - "deadbaaf": struct( + "sha256:deadbaaf": struct( url = "super2", - sha256 = "deadbaaf", + digest = "sha256:deadbaaf", filename = "foo-0.0.1-py3-none-any.whl", yanked = None, ), @@ -777,7 +781,7 @@ def _test_overlapping_shas_with_index_results(env): extra_pip_args = [], filename = "foo-0.0.1-py3-none-any.whl", requirement_line = "foo==0.0.3", - sha256 = "deadbaaf", + digest = "sha256:deadbaaf", target_platforms = ["cp39_linux_x86_64"], url = "super2", yanked = None, @@ -787,7 +791,7 @@ def _test_overlapping_shas_with_index_results(env): extra_pip_args = [], filename = "foo-0.0.1-py3-none-macosx_14_0_x86_64.whl", requirement_line = "foo==0.0.3", - sha256 = "deadb11f", + digest = "sha256:deadb11f", target_platforms = ["cp39_osx_x86_64"], url = "super2", yanked = None, @@ -798,6 +802,62 @@ def _test_overlapping_shas_with_index_results(env): _tests.append(_test_overlapping_shas_with_index_results) +def _test_non_sha256_hash_matching(env): + """Test that non-sha256 pins are matched against the index digests.""" + got = parse_requirements( + requirements_by_platform = { + "requirements_sha512": ["cp311_linux_x86_64"], + }, + platforms = { + "cp311_linux_x86_64": struct( + env = pep508_env( + python_version = "3.11.0", + os = "linux", + arch = "x86_64", + ), + whl_abi_tags = ["none"], + whl_platform_tags = ["any"], + ), + }, + get_index_urls = lambda _, __, **kwargs: { + "foo": struct( + index_url = "https://example.com", + sdists = {}, + whls = { + "sha512:deadbeef": struct( + url = "https://example.com/foo-0.0.1-py3-none-any.whl", + digest = "sha512:deadbeef", + filename = "foo-0.0.1-py3-none-any.whl", + yanked = None, + ), + }, + ), + }, + ) + + env.expect.that_collection(got).contains_exactly([ + struct( + name = "foo", + index_url = "https://example.com", + is_exposed = True, + is_multiple_versions = False, + srcs = [ + struct( + distribution = "foo", + extra_pip_args = [], + filename = "foo-0.0.1-py3-none-any.whl", + requirement_line = "foo==0.0.1", + digest = "sha512:deadbeef", + target_platforms = ["cp311_linux_x86_64"], + url = "https://example.com/foo-0.0.1-py3-none-any.whl", + yanked = None, + ), + ], + ), + ]) + +_tests.append(_test_non_sha256_hash_matching) + def _test_get_index_urls_different_versions(env): """Test that different versions from index URLs are matched correctly per platform.""" got = parse_requirements( @@ -832,15 +892,15 @@ def _test_get_index_urls_different_versions(env): index_url = "", sdists = {}, whls = { - "deadb11f": struct( + "sha256:deadb11f": struct( url = "super2", - sha256 = "deadb11f", + digest = "sha256:deadb11f", filename = "foo-0.0.2-py3-none-any.whl", yanked = None, ), - "deadbaaf": struct( + "sha256:deadbaaf": struct( url = "super2", - sha256 = "deadbaaf", + digest = "sha256:deadbaaf", filename = "foo-0.0.1-py3-none-any.whl", yanked = None, ), @@ -861,7 +921,7 @@ def _test_get_index_urls_different_versions(env): extra_pip_args = [], filename = "", requirement_line = "boo==0.0.4 --hash=sha256:deadbaaf", - sha256 = "", + digest = "", target_platforms = ["cp39_linux_x86_64"], url = "", yanked = None, @@ -879,7 +939,7 @@ def _test_get_index_urls_different_versions(env): extra_pip_args = [], filename = "", requirement_line = "foo==0.0.1 --hash=sha256:deadbeef", - sha256 = "", + digest = "", target_platforms = ["cp39_linux_x86_64"], url = "", yanked = None, @@ -889,7 +949,7 @@ def _test_get_index_urls_different_versions(env): extra_pip_args = [], filename = "foo-0.0.2-py3-none-any.whl", requirement_line = "foo==0.0.2", - sha256 = "deadb11f", + digest = "sha256:deadb11f", target_platforms = ["cp310_linux_x86_64"], url = "super2", yanked = None, @@ -969,9 +1029,9 @@ def _test_get_index_urls_single_py_version(env): index_url = "", sdists = {}, whls = { - "deadb11f": struct( + "sha256:deadb11f": struct( url = "super2", - sha256 = "deadb11f", + digest = "sha256:deadb11f", filename = "foo-0.0.2-py3-none-any.whl", yanked = None, ), @@ -992,7 +1052,7 @@ def _test_get_index_urls_single_py_version(env): extra_pip_args = [], filename = "foo-0.0.2-py3-none-any.whl", requirement_line = "foo==0.0.2", - sha256 = "deadb11f", + digest = "sha256:deadb11f", target_platforms = ["cp310_linux_x86_64"], url = "super2", yanked = None, @@ -1062,7 +1122,7 @@ def _test_uv_lock_consistent(env): requirement_line = "foo[extra]==0.0.1", target_platforms = ["linux_x86_64", "windows_x86_64"], filename = "foo-0.0.1-py3-none-any.whl", - sha256 = "deadbeef", + digest = "sha256:deadbeef", url = "https://files.pythonhosted.org/packages/foo-0.0.1-py3-none-any.whl", yanked = None, ), @@ -1090,7 +1150,7 @@ def _test_uv_lock_primary_source(env): requirement_line = "foo==0.0.1", target_platforms = ["linux_x86_64"], filename = "foo-0.0.1-py3-none-any.whl", - sha256 = "deadbeef", + digest = "sha256:deadbeef", url = "https://files.pythonhosted.org/packages/foo-0.0.1-py3-none-any.whl", yanked = None, ), @@ -1100,6 +1160,34 @@ def _test_uv_lock_primary_source(env): _tests.append(_test_uv_lock_primary_source) +def _test_uv_lock_non_sha256_hash(env): + """Test that non-sha256 uv.lock hashes are propagated to the sources.""" + got = parse_requirements( + uv_lock = "uv_lock_foo_sha512", + ) + env.expect.that_collection(got).contains_exactly([ + struct( + name = "foo", + index_url = "https://pypi.org/simple/foo", + is_exposed = True, + is_multiple_versions = False, + srcs = [ + struct( + distribution = "foo", + extra_pip_args = [], + requirement_line = "foo==0.0.1", + target_platforms = ["linux_x86_64"], + filename = "foo-0.0.1-py3-none-any.whl", + digest = "sha512:deadbeef", + url = "https://files.pythonhosted.org/packages/foo-0.0.1-py3-none-any.whl", + yanked = None, + ), + ], + ), + ]) + +_tests.append(_test_uv_lock_non_sha256_hash) + def _test_uv_lock_primary_source_multiple_versions(env): """Test that uv.lock with multiple versions of the same package works.""" got = parse_requirements( @@ -1118,7 +1206,7 @@ def _test_uv_lock_primary_source_multiple_versions(env): requirement_line = "foo==0.0.1", target_platforms = ["linux_x86_64"], filename = "foo-0.0.1-py3-none-any.whl", - sha256 = "deadbeef", + digest = "sha256:deadbeef", url = "https://files.pythonhosted.org/packages/foo-0.0.1-py3-none-any.whl", yanked = None, ), @@ -1128,7 +1216,7 @@ def _test_uv_lock_primary_source_multiple_versions(env): requirement_line = "foo==0.0.2", target_platforms = ["linux_x86_64"], filename = "foo-0.0.2-py3-none-any.whl", - sha256 = "deadb11f", + digest = "sha256:deadb11f", url = "https://files.pythonhosted.org/packages/foo-0.0.2-py3-none-any.whl", yanked = None, ), @@ -1156,7 +1244,7 @@ def _test_uv_lock_primary_source_with_extras(env): requirement_line = "foo[extra]==0.0.1", target_platforms = ["linux_x86_64"], filename = "foo-0.0.1-py3-none-any.whl", - sha256 = "deadbeef", + digest = "sha256:deadbeef", url = "https://files.pythonhosted.org/packages/foo-0.0.1-py3-none-any.whl", yanked = None, ), @@ -1184,7 +1272,7 @@ def _test_uv_lock_primary_source_includes_virtual(env): requirement_line = "foo==0.0.1", target_platforms = ["linux_x86_64"], filename = "foo-0.0.1-py3-none-any.whl", - sha256 = "deadbeef", + digest = "sha256:deadbeef", url = "https://files.pythonhosted.org/packages/foo-0.0.1-py3-none-any.whl", yanked = None, ), @@ -1222,7 +1310,7 @@ def _test_uv_lock_cross_consistent(env): requirement_line = "foo[extra]==0.0.1", target_platforms = ["linux_x86_64", "windows_x86_64"], filename = "foo-0.0.1-py3-none-any.whl", - sha256 = "deadbeef", + digest = "sha256:deadbeef", url = "https://files.pythonhosted.org/packages/foo-0.0.1-py3-none-any.whl", yanked = None, ), @@ -1250,7 +1338,7 @@ def _test_uv_lock_vcs_entry(env): requirement_line = "foo==0.1.0", target_platforms = ["linux_x86_64"], filename = "foo.git", - sha256 = "", + digest = "", url = "https://github.com/org/foo.git", yanked = None, ), @@ -1278,7 +1366,7 @@ def _test_uv_lock_rules_python_pkg_not_skipped(env): requirement_line = "rules_python==0.0.1", target_platforms = ["linux_x86_64"], filename = "rules_python-0.0.1-py3-none-any.whl", - sha256 = "deadbeef", + digest = "sha256:deadbeef", url = "https://files.pythonhosted.org/packages/rules_python-0.0.1-py3-none-any.whl", yanked = None, ), @@ -1311,7 +1399,7 @@ def _test_uv_lock_no_consistency_check(env): requirement_line = "foo==0.0.1", target_platforms = ["linux_x86_64"], filename = "foo-0.0.1-py3-none-any.whl", - sha256 = "deadbeef", + digest = "sha256:deadbeef", url = "https://files.pythonhosted.org/packages/foo-0.0.1-py3-none-any.whl", yanked = None, ), @@ -1339,7 +1427,7 @@ def _test_uv_lock_multiple_packages(env): requirement_line = "bar==0.0.1", target_platforms = ["linux_x86_64"], filename = "bar-0.0.1.tar.gz", - sha256 = "deadb00f", + digest = "sha256:deadb00f", url = "https://files.pythonhosted.org/packages/bar-0.0.1.tar.gz", yanked = None, ), @@ -1357,7 +1445,7 @@ def _test_uv_lock_multiple_packages(env): requirement_line = "foo==0.0.1", target_platforms = ["linux_x86_64"], filename = "foo-0.0.1-py3-none-any.whl", - sha256 = "deadbeef", + digest = "sha256:deadbeef", url = "https://files.pythonhosted.org/packages/foo-0.0.1-py3-none-any.whl", yanked = None, ), @@ -1386,7 +1474,7 @@ def _test_uv_lock_with_extra_pip_args(env): requirement_line = "foo==0.0.1", target_platforms = ["linux_x86_64"], filename = "foo-0.0.1-py3-none-any.whl", - sha256 = "deadbeef", + digest = "sha256:deadbeef", url = "https://files.pythonhosted.org/packages/foo-0.0.1-py3-none-any.whl", yanked = None, ), @@ -1418,7 +1506,7 @@ def _test_uv_lock_multi_os_with_requirements(env): requirement_line = "foo==0.0.1", target_platforms = ["linux_aarch64", "linux_x86_64", "windows_x86_64"], filename = "foo-0.0.1-py3-none-any.whl", - sha256 = "deadbeef", + digest = "sha256:deadbeef", url = "https://files.pythonhosted.org/packages/foo-0.0.1-py3-none-any.whl", yanked = None, ), @@ -1446,7 +1534,7 @@ def _test_uv_lock_extras_optional_deps(env): requirement_line = "foo[extra1,extra2]==0.0.1", target_platforms = ["linux_x86_64"], filename = "foo-0.0.1-py3-none-any.whl", - sha256 = "deadbeef", + digest = "sha256:deadbeef", url = "https://files.pythonhosted.org/packages/foo-0.0.1-py3-none-any.whl", yanked = None, ), @@ -1474,7 +1562,7 @@ def _test_uv_lock_extras_dep_edge(env): requirement_line = "bar[extra1]==0.0.2", target_platforms = ["linux_x86_64"], filename = "bar-0.0.2-py3-none-any.whl", - sha256 = "deadbeef", + digest = "sha256:deadbeef", url = "https://files.pythonhosted.org/packages/bar-0.0.2-py3-none-any.whl", yanked = None, ), @@ -1492,7 +1580,7 @@ def _test_uv_lock_extras_dep_edge(env): requirement_line = "foo==0.0.1", target_platforms = ["linux_x86_64"], filename = "foo-0.0.1-py3-none-any.whl", - sha256 = "baadbeef", + digest = "sha256:baadbeef", url = "https://files.pythonhosted.org/packages/foo-0.0.1-py3-none-any.whl", yanked = None, ), @@ -1527,7 +1615,7 @@ def _test_uv_lock_wheel_dedup_single_version(env): requirement_line = "foo==0.0.1", target_platforms = ["cp39_linux_x86_64"], filename = "foo-0.0.1-cp39-cp39-manylinux_2_17_x86_64.whl", - sha256 = "aaa", + digest = "sha256:aaa", url = "https://files.pythonhosted.org/packages/foo-0.0.1-cp39-cp39-manylinux_2_17_x86_64.whl", yanked = None, ), @@ -1573,7 +1661,7 @@ def _test_uv_lock_wheel_dedup_resolution_markers(env): requirement_line = "foo==0.0.1", target_platforms = ["cp39_linux_x86_64"], filename = "foo-0.0.1-cp39-cp39-manylinux_2_17_x86_64.whl", - sha256 = "aaa", + digest = "sha256:aaa", url = "https://files.pythonhosted.org/packages/foo-0.0.1-cp39-cp39-manylinux_2_17_x86_64.whl", yanked = None, ), @@ -1583,7 +1671,7 @@ def _test_uv_lock_wheel_dedup_resolution_markers(env): requirement_line = "foo==0.0.2", target_platforms = ["cp39_osx_aarch64"], filename = "foo-0.0.2-cp39-cp39-macosx_11_0_arm64.whl", - sha256 = "ccc", + digest = "sha256:ccc", url = "https://files.pythonhosted.org/packages/foo-0.0.2-cp39-cp39-macosx_11_0_arm64.whl", yanked = None, ), @@ -1611,7 +1699,7 @@ def _test_uv_lock_requires_dist_extras(env): requirement_line = "foo[all]==0.0.1", target_platforms = ["linux_x86_64"], filename = "foo-0.0.1-py3-none-any.whl", - sha256 = "deadbeef", + digest = "sha256:deadbeef", url = "https://files.pythonhosted.org/packages/foo-0.0.1-py3-none-any.whl", yanked = None, ), diff --git a/tests/pypi/parse_simpleapi_html/parse_simpleapi_html_tests.bzl b/tests/pypi/parse_simpleapi_html/parse_simpleapi_html_tests.bzl index c84140f459..975fcd0ee6 100644 --- a/tests/pypi/parse_simpleapi_html/parse_simpleapi_html_tests.bzl +++ b/tests/pypi/parse_simpleapi_html/parse_simpleapi_html_tests.bzl @@ -78,7 +78,7 @@ def _test_sdist(env): ), struct( filename = "foo-0.0.1.tar.gz", - sha256 = "deadbeefasource", + digest = "sha256:deadbeefasource", url = "https://example.org/full-url/foo-0.0.1.tar.gz", yanked = None, version = "0.0.1", @@ -95,7 +95,7 @@ def _test_sdist(env): ), struct( filename = "foo-0.0.1.tar.gz", - sha256 = "deadbeefasource", + digest = "sha256:deadbeefasource", url = "https://example.org/full-url/foo-0.0.1.tar.gz", version = "0.0.1", yanked = "", @@ -112,7 +112,7 @@ def _test_sdist(env): ), struct( filename = "foo-0.0.1.tar.gz", - sha256 = "deadbeefasource", + digest = "sha256:deadbeefasource", url = "https://example.org/full-url/foo-0.0.1.tar.gz", version = "0.0.1", # NOTE @aignas 2026-03-09: we preserve the white space @@ -130,7 +130,7 @@ def _test_sdist(env): ), struct( filename = "foo-0.0.1.tar.gz", - sha256 = "deadbeefasource", + digest = "sha256:deadbeefasource", url = "https://example.org/full-url/foo-0.0.1.tar.gz", version = "0.0.1", yanked = "", @@ -143,22 +143,22 @@ def _test_sdist(env): got = parse_simpleapi_html(content = html) env.expect.that_collection(got.sdists).has_size(1) env.expect.that_collection(got.whls).has_size(0) - env.expect.that_collection(got.sha256s_by_version).has_size(1) + env.expect.that_collection(got.hashes_by_version).has_size(1) if not got: fail("expected at least one element, but did not get anything from:\n{}".format(html)) actual = env.expect.that_struct( - got.sdists[want.sha256], + got.sdists[want.digest], attrs = dict( filename = subjects.str, - sha256 = subjects.str, + digest = subjects.str, url = subjects.str, yanked = subjects.str, version = subjects.str, ), ) actual.filename().equals(want.filename) - actual.sha256().equals(want.sha256) + actual.digest().equals(want.digest) actual.url().equals(want.url) actual.yanked().equals(want.yanked) actual.version().equals(want.version) @@ -182,7 +182,7 @@ def _test_whls(env): filename = "foo-0.0.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", metadata_sha256 = "deadb00f", metadata_url = "https://example.org/full-url/foo-0.0.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata", - sha256 = "deadbeef", + digest = "sha256:deadbeef", url = "https://example.org/full-url/foo-0.0.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", version = "0.0.2", yanked = None, @@ -202,7 +202,7 @@ def _test_whls(env): filename = "foo-0.0.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", metadata_sha256 = "deadb00f", metadata_url = "https://example.org/full-url/foo-0.0.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata", - sha256 = "deadbeef", + digest = "sha256:deadbeef", url = "https://example.org/full-url/foo-0.0.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", version = "0.0.2", yanked = None, @@ -221,7 +221,7 @@ def _test_whls(env): filename = "foo-0.0.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", metadata_sha256 = "deadb00f", metadata_url = "https://example.org/full-url/foo-0.0.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata", - sha256 = "deadbeef", + digest = "sha256:deadbeef", version = "0.0.2", url = "https://example.org/full-url/foo-0.0.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", yanked = None, @@ -240,7 +240,7 @@ def _test_whls(env): filename = "foo-0.0.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", metadata_sha256 = "deadb00f", metadata_url = "https://example.org/full-url/foo-0.0.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata", - sha256 = "deadbeef", + digest = "sha256:deadbeef", version = "0.0.2", url = "https://example.org/full-url/foo-0.0.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", yanked = None, @@ -258,7 +258,7 @@ def _test_whls(env): filename = "foo-0.0.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", metadata_sha256 = "", metadata_url = "", - sha256 = "deadbeef", + digest = "sha256:deadbeef", url = "https://example.org/full-url/foo-0.0.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", version = "0.0.2", yanked = None, @@ -275,12 +275,12 @@ def _test_whls(env): fail("expected at least one element, but did not get anything from:\n{}".format(html)) actual = env.expect.that_struct( - got.whls[want.sha256], + got.whls[want.digest], attrs = dict( filename = subjects.str, metadata_sha256 = subjects.str, metadata_url = subjects.str, - sha256 = subjects.str, + digest = subjects.str, url = subjects.str, yanked = subjects.str, version = subjects.str, @@ -289,13 +289,75 @@ def _test_whls(env): actual.filename().equals(want.filename) actual.metadata_sha256().equals(want.metadata_sha256) actual.metadata_url().equals(want.metadata_url) - actual.sha256().equals(want.sha256) + actual.digest().equals(want.digest) actual.url().equals(want.url) actual.yanked().equals(want.yanked) actual.version().equals(want.version) _tests.append(_test_whls) +def _test_non_sha256_fragment(env): + """The index may advertise digests calculated with any hash algorithm (PEP 503).""" + + html = _generate_html( + struct( + attrs = [ + 'href="https://example.org/full-url/foo-0.0.1-py3-none-any.whl#sha512=deadbeef"', + ], + filename = "foo-0.0.1-py3-none-any.whl", + ), + struct( + attrs = [ + 'href="https://example.org/full-url/foo-0.0.1.tar.gz#sha512=deadb00f"', + ], + filename = "foo-0.0.1.tar.gz", + ), + struct( + # A non-hash fragment is kept as part of the URL. + attrs = [ + 'href="https://example.org/full-url/foo-0.0.2-py3-none-any.whl#egg=foo"', + ], + filename = "foo-0.0.2-py3-none-any.whl", + ), + ) + got = parse_simpleapi_html(content = html) + + env.expect.that_collection(got.whls).has_size(2) + env.expect.that_collection(got.sdists).has_size(1) + env.expect.that_dict(got.hashes_by_version).contains_exactly({ + "0.0.1": ["sha512:deadbeef", "sha512:deadb00f"], + "0.0.2": [""], + }) + + whl = got.whls["sha512:deadbeef"] + env.expect.that_str(whl.digest).equals("sha512:deadbeef") + env.expect.that_str(whl.url).equals("https://example.org/full-url/foo-0.0.1-py3-none-any.whl") + + sdist = got.sdists["sha512:deadb00f"] + env.expect.that_str(sdist.digest).equals("sha512:deadb00f") + + no_hash_whl = got.whls[""] + env.expect.that_str(no_hash_whl.digest).equals("") + env.expect.that_str(no_hash_whl.url).equals("https://example.org/full-url/foo-0.0.2-py3-none-any.whl#egg=foo") + +_tests.append(_test_non_sha256_fragment) + +def _test_sha256_fragment_digest(env): + html = _generate_html( + struct( + attrs = [ + 'href="https://example.org/full-url/foo-0.0.1-py3-none-any.whl#sha256=deadbeef"', + ], + filename = "foo-0.0.1-py3-none-any.whl", + ), + ) + got = parse_simpleapi_html(content = html) + + whl = got.whls["sha256:deadbeef"] + env.expect.that_str(whl.digest).equals("sha256:deadbeef") + +_tests.append(_test_sha256_fragment_digest) + def parse_simpleapi_html_test_suite(name): """Create the test suite. diff --git a/tests/pypi/pypi_cache/pypi_cache_tests.bzl b/tests/pypi/pypi_cache/pypi_cache_tests.bzl index 14c12ae6d2..21c2f5bd8c 100644 --- a/tests/pypi/pypi_cache/pypi_cache_tests.bzl +++ b/tests/pypi/pypi_cache/pypi_cache_tests.bzl @@ -11,8 +11,8 @@ def _cache(env, **kwargs): cache = pypi_cache(**kwargs) attrs = { + "hashes_by_version": subjects.dict, "sdists": subjects.dict, - "sha256s_by_version": subjects.dict, "whls": subjects.dict, } @@ -48,14 +48,14 @@ def _test_memory_cache_hit(env): # Mocked parsed result from a PyPI-like index fake_result = struct( sdists = { - "sha_1": struct(version = "1.0.0", filename = "pkg-1.0.0.tar.gz"), + "sha256:sha_1": struct(version = "1.0.0", filename = "pkg-1.0.0.tar.gz"), }, whls = { - "sha_2": struct(version = "1.1.0", filename = "pkg-1.1.0-py3-none-any.whl"), + "sha256:sha_2": struct(version = "1.1.0", filename = "pkg-1.1.0-py3-none-any.whl"), }, - sha256s_by_version = { - "1.0.0": ["sha_1"], - "1.1.0": ["sha_2"], + hashes_by_version = { + "1.0.0": ["sha256:sha_1"], + "1.1.0": ["sha256:sha_2"], }, ) @@ -70,7 +70,7 @@ def _test_memory_cache_hit(env): got.sdists().contains_exactly(fake_result.sdists) got.whls().contains_exactly(fake_result.whls) - got.sha256s_by_version().contains_exactly(fake_result.sha256s_by_version) + got.hashes_by_version().contains_exactly(fake_result.hashes_by_version) # A different key with fewer versions key = ("https://{PYPI_INDEX_URL}/pkg", "https://pypi.org/simple/pkg", ["1.0.0"]) @@ -78,7 +78,7 @@ def _test_memory_cache_hit(env): got = cache.get(key) got.sdists().contains_exactly(fake_result.sdists) got.whls().contains_exactly({}) - got.sha256s_by_version().contains_exactly({"1.0.0": ["sha_1"]}) + got.hashes_by_version().contains_exactly({"1.0.0": ["sha256:sha_1"]}) # A key with no matches key = ("https://{PYPI_INDEX_URL}/pkg", "https://pypi.org/simple/pkg", ["1.2.0"]) @@ -94,31 +94,34 @@ def _test_pypi_cache_writes_to_facts(env): fake_result = struct( sdists = { - "sha_sdist": struct( + "sha256:sha_sdist": struct( version = "1.0.0", filename = "pkg-1.0.0.tar.gz", url = "https://pypi.org/files/pkg-1.0.0.tar.gz", + digest = "sha256:sha_sdist", yanked = "", ), }, whls = { - "sha_whl": struct( + "sha256:sha_whl": struct( version = "1.0.0", filename = "pkg-1.0.0-py3-none-any.whl", url = "https://pypi.org/files/pkg-1.0.0-py3-none-any.whl", + digest = "sha256:sha_whl", yanked = "Security issue", ), # This won't get stored - "sha_whl_2": struct( + "sha256:sha_whl_2": struct( version = "1.1.0", filename = "pkg-1.1.0-py3-none-any.whl", url = "https://pypi.org/files/pkg-1.1.0-py3-none-any.whl", + digest = "sha256:sha_whl_2", yanked = None, ), }, - sha256s_by_version = { - "1.0.0": ["sha_sdist", "sha_whl"], - "1.1.0": ["sha_whl_2"], + hashes_by_version = { + "1.0.0": ["sha256:sha_sdist", "sha256:sha_whl"], + "1.1.0": ["sha256:sha_whl_2"], }, ) @@ -130,11 +133,11 @@ def _test_pypi_cache_writes_to_facts(env): # Then the key returns us the same items got = cache.get(key) got.whls().contains_exactly({ - "sha_whl": fake_result.whls["sha_whl"], + "sha256:sha_whl": fake_result.whls["sha256:sha_whl"], }) got.sdists().contains_exactly(fake_result.sdists) - got.sha256s_by_version().contains_exactly({ - "1.0.0": fake_result.sha256s_by_version["1.0.0"], + got.hashes_by_version().contains_exactly({ + "1.0.0": fake_result.hashes_by_version["1.0.0"], }) # Then when we get facts at the end @@ -143,30 +146,30 @@ def _test_pypi_cache_writes_to_facts(env): # We are not using the real index URL, because we may have credentials in here "https://{PYPI_INDEX_URL}": { "pkg": { - "https://pypi.org/files/pkg-1.0.0-py3-none-any.whl": "sha_whl", - "https://pypi.org/files/pkg-1.0.0.tar.gz": "sha_sdist", + "https://pypi.org/files/pkg-1.0.0-py3-none-any.whl": "sha256:sha_whl", + "https://pypi.org/files/pkg-1.0.0.tar.gz": "sha256:sha_sdist", }, }, }, "dist_yanked": { "https://{PYPI_INDEX_URL}": { "pkg": { - "sha_sdist": "", - "sha_whl": "Security issue", + "sha256:sha_sdist": "", + "sha256:sha_whl": "Security issue", }, }, }, - "fact_version": "v1", # Facts version + "fact_version": "v2", # Facts version }) # When we get the other items cached in memory, they get written to facts got = cache.get((key[0], key[1], ["1.1.0"])) got.whls().contains_exactly({ - "sha_whl_2": fake_result.whls["sha_whl_2"], + "sha256:sha_whl_2": fake_result.whls["sha256:sha_whl_2"], }) got.sdists().contains_exactly({}) - got.sha256s_by_version().contains_exactly({ - "1.1.0": fake_result.sha256s_by_version["1.1.0"], + got.hashes_by_version().contains_exactly({ + "1.1.0": fake_result.hashes_by_version["1.1.0"], }) # Then when we get facts at the end @@ -175,21 +178,21 @@ def _test_pypi_cache_writes_to_facts(env): # We are not using the real index URL, because we may have credentials in here "https://{PYPI_INDEX_URL}": { "pkg": { - "https://pypi.org/files/pkg-1.0.0-py3-none-any.whl": "sha_whl", - "https://pypi.org/files/pkg-1.0.0.tar.gz": "sha_sdist", - "https://pypi.org/files/pkg-1.1.0-py3-none-any.whl": "sha_whl_2", + "https://pypi.org/files/pkg-1.0.0-py3-none-any.whl": "sha256:sha_whl", + "https://pypi.org/files/pkg-1.0.0.tar.gz": "sha256:sha_sdist", + "https://pypi.org/files/pkg-1.1.0-py3-none-any.whl": "sha256:sha_whl_2", }, }, }, "dist_yanked": { "https://{PYPI_INDEX_URL}": { "pkg": { - "sha_sdist": "", - "sha_whl": "Security issue", + "sha256:sha_sdist": "", + "sha256:sha_whl": "Security issue", }, }, }, - "fact_version": "v1", # Facts version + "fact_version": "v2", # Facts version }) _tests.append(_test_pypi_cache_writes_to_facts) @@ -201,20 +204,20 @@ def _test_pypi_cache_reads_from_facts(env): # We are not using the real index URL, because we may have credentials in here "https://{PYPI_INDEX_URL}": { "pkg": { - "https://pypi.org/files/pkg-1.0.0-py3-none-any.whl": "sha_whl", - "https://pypi.org/files/pkg-1.0.0.tar.gz": "sha_sdist", + "https://pypi.org/files/pkg-1.0.0-py3-none-any.whl": "sha256:sha_whl", + "https://pypi.org/files/pkg-1.0.0.tar.gz": "sha256:sha_sdist", }, }, }, "dist_yanked": { "https://{PYPI_INDEX_URL}": { "pkg": { - "sha_sdist": "", - "sha_whl": "Security issue", + "sha256:sha_sdist": "", + "sha256:sha_whl": "Security issue", }, }, }, - "fact_version": "v1", # Facts version + "fact_version": "v2", # Facts version }) cache = _cache(env, mctx = mock_ctx) @@ -229,8 +232,8 @@ def _test_pypi_cache_reads_from_facts(env): expected_result = struct( sdists = { - "sha_sdist": struct( - sha256 = "sha_sdist", + "sha256:sha_sdist": struct( + digest = "sha256:sha_sdist", version = "1.0.0", filename = "pkg-1.0.0.tar.gz", metadata_url = "", @@ -240,8 +243,8 @@ def _test_pypi_cache_reads_from_facts(env): ), }, whls = { - "sha_whl": struct( - sha256 = "sha_whl", + "sha256:sha_whl": struct( + digest = "sha256:sha_whl", version = "1.0.0", filename = "pkg-1.0.0-py3-none-any.whl", url = "https://pypi.org/files/pkg-1.0.0-py3-none-any.whl", @@ -250,14 +253,14 @@ def _test_pypi_cache_reads_from_facts(env): yanked = "Security issue", ), }, - sha256s_by_version = { - "1.0.0": ["sha_sdist", "sha_whl"], + hashes_by_version = { + "1.0.0": ["sha256:sha_sdist", "sha256:sha_whl"], }, ) got.whls().contains_exactly(expected_result.whls) got.sdists().contains_exactly(expected_result.sdists) - got.sha256s_by_version().contains_exactly(expected_result.sha256s_by_version) + got.hashes_by_version().contains_exactly(expected_result.hashes_by_version) # Then when we store the same facts back again, because we accessed the cached keys. cache.get_facts().contains_exactly(mock_ctx.facts) @@ -275,14 +278,14 @@ def _test_pypi_cache_reads_from_facts_drops_unaccessed_dists(env): "dist_hashes": { "https://{PYPI_INDEX_URL}": { "pkg": { - "https://pypi.org/files/pkg-1.0.0-py3-none-any.whl": "sha_whl_1.0.0", - "https://pypi.org/files/pkg-1.0.0.tar.gz": "sha_sdist_1.0.0", - "https://pypi.org/files/pkg-1.1.0-py3-none-any.whl": "sha_whl_1.1.0", - "https://pypi.org/files/pkg-1.1.0.tar.gz": "sha_sdist_1.1.0", + "https://pypi.org/files/pkg-1.0.0-py3-none-any.whl": "sha256:sha_whl_1.0.0", + "https://pypi.org/files/pkg-1.0.0.tar.gz": "sha256:sha_sdist_1.0.0", + "https://pypi.org/files/pkg-1.1.0-py3-none-any.whl": "sha256:sha_whl_1.1.0", + "https://pypi.org/files/pkg-1.1.0.tar.gz": "sha256:sha_sdist_1.1.0", }, }, }, - "fact_version": "v1", + "fact_version": "v2", }) cache = _cache(env, mctx = mock_ctx) @@ -292,8 +295,8 @@ def _test_pypi_cache_reads_from_facts_drops_unaccessed_dists(env): expected = struct( sdists = { - "sha_sdist_1.0.0": struct( - sha256 = "sha_sdist_1.0.0", + "sha256:sha_sdist_1.0.0": struct( + digest = "sha256:sha_sdist_1.0.0", version = "1.0.0", filename = "pkg-1.0.0.tar.gz", metadata_url = "", @@ -303,8 +306,8 @@ def _test_pypi_cache_reads_from_facts_drops_unaccessed_dists(env): ), }, whls = { - "sha_whl_1.0.0": struct( - sha256 = "sha_whl_1.0.0", + "sha256:sha_whl_1.0.0": struct( + digest = "sha256:sha_whl_1.0.0", version = "1.0.0", filename = "pkg-1.0.0-py3-none-any.whl", metadata_url = "", @@ -313,29 +316,93 @@ def _test_pypi_cache_reads_from_facts_drops_unaccessed_dists(env): yanked = None, ), }, - sha256s_by_version = { - "1.0.0": ["sha_sdist_1.0.0", "sha_whl_1.0.0"], + hashes_by_version = { + "1.0.0": ["sha256:sha_sdist_1.0.0", "sha256:sha_whl_1.0.0"], }, ) got.whls().contains_exactly(expected.whls) got.sdists().contains_exactly(expected.sdists) - got.sha256s_by_version().contains_exactly(expected.sha256s_by_version) + got.hashes_by_version().contains_exactly(expected.hashes_by_version) # get_facts() must only contain version 1.0.0 data; 1.1.0 is dropped cache.get_facts().contains_exactly({ "dist_hashes": { "https://{PYPI_INDEX_URL}": { "pkg": { - "https://pypi.org/files/pkg-1.0.0-py3-none-any.whl": "sha_whl_1.0.0", - "https://pypi.org/files/pkg-1.0.0.tar.gz": "sha_sdist_1.0.0", + "https://pypi.org/files/pkg-1.0.0-py3-none-any.whl": "sha256:sha_whl_1.0.0", + "https://pypi.org/files/pkg-1.0.0.tar.gz": "sha256:sha_sdist_1.0.0", }, }, }, - "fact_version": "v1", + "fact_version": "v2", }) _tests.append(_test_pypi_cache_reads_from_facts_drops_unaccessed_dists) +def _test_pypi_cache_facts_non_sha256_round_trip(env): + """Verifies that digests of other hash algorithms survive the facts round trip.""" + mock_ctx = mocks.mctx(facts = {}) + cache = _cache(env, mctx = mock_ctx) + + fake_result = struct( + sdists = {}, + whls = { + "sha512:whl_digest": struct( + version = "1.0.0", + filename = "pkg-1.0.0-py3-none-any.whl", + url = "https://pypi.org/files/pkg-1.0.0-py3-none-any.whl", + digest = "sha512:whl_digest", + yanked = None, + ), + }, + hashes_by_version = { + "1.0.0": ["sha512:whl_digest"], + }, + ) + + key = ("https://{PYPI_INDEX_URL}/pkg/", "https://pypi.org/simple/pkg/", ["1.0.0"]) + cache.setdefault(key, fake_result) + + # The hash algorithm is stored in the facts + cache.get_facts().contains_exactly({ + "dist_hashes": { + "https://{PYPI_INDEX_URL}": { + "pkg": { + "https://pypi.org/files/pkg-1.0.0-py3-none-any.whl": "sha512:whl_digest", + }, + }, + }, + "fact_version": "v2", # Facts version + }) + + # And a cache that only has the facts reconstructs the digests + cache = _cache(env, mctx = mocks.mctx(facts = { + "dist_hashes": { + "https://{PYPI_INDEX_URL}": { + "pkg": { + "https://pypi.org/files/pkg-1.0.0-py3-none-any.whl": "sha512:whl_digest", + }, + }, + }, + "fact_version": "v2", # Facts version + })) + + got = cache.get(key) + got.whls().contains_exactly({ + "sha512:whl_digest": struct( + digest = "sha512:whl_digest", + version = "1.0.0", + filename = "pkg-1.0.0-py3-none-any.whl", + metadata_url = "", + metadata_sha256 = "", + url = "https://pypi.org/files/pkg-1.0.0-py3-none-any.whl", + yanked = None, + ), + }) + got.hashes_by_version().contains_exactly({"1.0.0": ["sha512:whl_digest"]}) + +_tests.append(_test_pypi_cache_facts_non_sha256_round_trip) + def _test_memory_cache_index_urls(env): """Verifies that the cache returns stored values for index_urls.""" store = {} @@ -377,7 +444,7 @@ def _test_pypi_cache_writes_index_urls_to_facts(env): cache.setdefault(key, fake_result) cache.get_facts().contains_exactly({ - "fact_version": "v1", + "fact_version": "v2", "index_urls": { "https://pypi.org/simple/": { "pkg-a": "https://pypi.org/simple/pkg-a/", @@ -389,7 +456,7 @@ def _test_pypi_cache_writes_index_urls_to_facts(env): cache.setdefault(key, fake_result) cache.get_facts().contains_exactly({ - "fact_version": "v1", + "fact_version": "v2", "index_urls": { "https://pypi.org/simple/": { "pkg-a": "https://pypi.org/simple/pkg-a/", @@ -403,7 +470,7 @@ _tests.append(_test_pypi_cache_writes_index_urls_to_facts) def _test_pypi_cache_reads_index_urls_from_facts(env): """Verifies that reading index_urls from facts works correctly.""" mock_ctx = mocks.mctx(facts = { - "fact_version": "v1", + "fact_version": "v2", "index_urls": { "https://pypi.org/simple/": { "pkg-a": "https://pypi.org/simple/pkg-a/", @@ -434,7 +501,7 @@ _tests.append(_test_pypi_cache_reads_index_urls_from_facts) def _test_pypi_cache_reads_index_urls_from_facts_incomplete(env): """Verifies that incomplete index_urls facts returns None (forces fresh download).""" mock_ctx = mocks.mctx(facts = { - "fact_version": "v1", + "fact_version": "v2", "index_urls": { "https://pypi.org/simple/": { "pkg-a": "https://pypi.org/simple/pkg-a/", @@ -457,7 +524,7 @@ def _test_pypi_cache_reads_index_urls_from_facts_drops_unaccessed(env): removed from all requirements files) get cleaned up from the lockfile. """ mock_ctx = mocks.mctx(facts = { - "fact_version": "v1", + "fact_version": "v2", "index_urls": { "https://pypi.org/simple/": { "pkg-a": "https://pypi.org/simple/pkg-a/", @@ -478,7 +545,7 @@ def _test_pypi_cache_reads_index_urls_from_facts_drops_unaccessed(env): # get_facts() must only return the requested (accessed) subset; pkg-c is dropped cache.get_facts().contains_exactly({ - "fact_version": "v1", + "fact_version": "v2", "index_urls": { "https://pypi.org/simple/": { "pkg-a": "https://pypi.org/simple/pkg-a/", diff --git a/tests/pypi/simpleapi_download/simpleapi_download_tests.bzl b/tests/pypi/simpleapi_download/simpleapi_download_tests.bzl index 4e86b76e10..dfa33ebe65 100644 --- a/tests/pypi/simpleapi_download/simpleapi_download_tests.bzl +++ b/tests/pypi/simpleapi_download/simpleapi_download_tests.bzl @@ -43,7 +43,7 @@ def _test_simple(env): output = struct( sdists = {"deadbeef": url.strip("/").split("/")[-1]}, whls = {"deadb33f": url.strip("/").split("/")[-1]}, - sha256s_by_version = {"fizz": url.strip("/").split("/")[-1]}, + hashes_by_version = {"fizz": url.strip("/").split("/")[-1]}, ), success = True, ) @@ -71,19 +71,19 @@ def _test_simple(env): "bar": struct( index_url = "https://main.com/bar/", sdists = {"deadbeef": "bar"}, - sha256s_by_version = {"fizz": "bar"}, + hashes_by_version = {"fizz": "bar"}, whls = {"deadb33f": "bar"}, ), "baz": struct( index_url = "https://main.com/baz/", sdists = {"deadbeef": "baz"}, - sha256s_by_version = {"fizz": "baz"}, + hashes_by_version = {"fizz": "baz"}, whls = {"deadb33f": "baz"}, ), "foo": struct( index_url = "https://extra.com/foo/", sdists = {"deadbeef": "foo"}, - sha256s_by_version = {"fizz": "foo"}, + hashes_by_version = {"fizz": "foo"}, whls = {"deadb33f": "foo"}, ), }) @@ -115,7 +115,7 @@ def _test_index_overrides(env): output = struct( sdists = {"deadbeef": url.strip("/").split("/")[-1]}, whls = {"deadb33f": url.strip("/").split("/")[-1]}, - sha256s_by_version = {"fizz": url.strip("/").split("/")[-1]}, + hashes_by_version = {"fizz": url.strip("/").split("/")[-1]}, ), success = True, ) @@ -147,19 +147,19 @@ def _test_index_overrides(env): "ba_z": struct( index_url = "https://main.com/ba-z/", sdists = {"deadbeef": "ba-z"}, - sha256s_by_version = {"fizz": "ba-z"}, + hashes_by_version = {"fizz": "ba-z"}, whls = {"deadb33f": "ba-z"}, ), "bar": struct( index_url = "https://main.com/bar/", sdists = {"deadbeef": "bar"}, - sha256s_by_version = {"fizz": "bar"}, + hashes_by_version = {"fizz": "bar"}, whls = {"deadb33f": "bar"}, ), "foo": struct( index_url = "https://extra.com/foo/", sdists = {"deadbeef": "foo"}, - sha256s_by_version = {"fizz": "foo"}, + hashes_by_version = {"fizz": "foo"}, whls = {"deadb33f": "foo"}, ), }) diff --git a/tests/pypi/whl_repo_name/whl_repo_name_tests.bzl b/tests/pypi/whl_repo_name/whl_repo_name_tests.bzl index 35e6bcdf9f..968bdc3d2a 100644 --- a/tests/pypi/whl_repo_name/whl_repo_name_tests.bzl +++ b/tests/pypi/whl_repo_name/whl_repo_name_tests.bzl @@ -25,6 +25,18 @@ def _test_simple(env): _tests.append(_test_simple) +def _test_simple_canonical_digest(env): + got = whl_repo_name("foo-1.2.3-py3-none-any.whl", "sha256:deadbeef") + env.expect.that_str(got).equals("foo_py3_none_any_deadbeef") + +_tests.append(_test_simple_canonical_digest) + +def _test_simple_canonical_digest_other_algo(env): + got = whl_repo_name("foo-1.2.3-py3-none-any.whl", "sha512:deadbeef000deadbeef") + env.expect.that_str(got).equals("foo_py3_none_any_deadbeef") + +_tests.append(_test_simple_canonical_digest_other_algo) + def _test_simple_no_sha(env): got = whl_repo_name("foo-1.2.3-py3-none-any.whl", "") env.expect.that_str(got).equals("foo_1_2_3_py3_none_any") From fbb48ff22bfb3c77fb05f6e43fd5113a16226927 Mon Sep 17 00:00:00 2001 From: Cal Jacobson Date: Sat, 1 Aug 2026 10:41:36 -0400 Subject: [PATCH 883/922] fix(gazelle)!: use per-version stdlib lists for Python 3.13 and 3.14 (#3978) The gazelle Python extension picks its stdlib list by `python_version`. The select had no branch for `3.13` or `3.14`, so those configurations fell through to the `3.11` list and gazelle misses everything that changed since `3.11` (`3.14` added `compression.zstd`, `3.13` removed `telnetlib`). Bump `python_stdlib_list` to `stdlib-list` `0.12.0`, the first release shipping `lists/3.14.txt`, and add the two matching branches. Point `//conditions:default` at the newest list, so configurations with no matching branch stop landing on one four releases behind. Fetch the archive from the PyPI sdist rather than a GitHub release asset. Unlike `0.11.0`, `0.12.0` published no GitHub release assets. GitHub's auto-generated tag archives are not checksum-stable. PyPI never reuses an uploaded filename. ## BREAKING CHANGE: Gazelle plugin now requires rules_python 1.5.0 Bazel resolves every `select()` key at analysis time, so `is_python_3.14` must exist even for builds that never select it. `rules_python` generates that target from `MINOR_MAPPING`, which lists `3.14` starting in `1.5.0`. Older versions fail analysis, and that broke the `Gazelle: BCR` jobs on the first push. Two pins constrain resolution, and both move to `1.5.0`: the plugin's own `bazel_dep` (was `0.18.0`) and `gazelle/examples/bzlmod_build_file_generation` (was `1.4.0`). The example's `other_module` keeps its placeholder pin, which the root module overrides. ## Testing I added no automated coverage and would appreciate maintainer input on how, if at all, to test this. I didn't see an obvious way given the existing test infra. --- gazelle/MODULE.bazel | 2 +- gazelle/deps.bzl | 6 +++--- gazelle/examples/bzlmod_build_file_generation/MODULE.bazel | 2 +- gazelle/python/BUILD.bazel | 6 ++++-- news/3978.changed.md | 3 +++ news/3978.fixed.md | 5 +++++ 6 files changed, 17 insertions(+), 7 deletions(-) create mode 100644 news/3978.changed.md create mode 100644 news/3978.fixed.md diff --git a/gazelle/MODULE.bazel b/gazelle/MODULE.bazel index cff6341a2b..b6eed23c6f 100644 --- a/gazelle/MODULE.bazel +++ b/gazelle/MODULE.bazel @@ -5,7 +5,7 @@ module( ) bazel_dep(name = "bazel_skylib", version = "1.8.2") -bazel_dep(name = "rules_python", version = "0.18.0") +bazel_dep(name = "rules_python", version = "1.5.0") bazel_dep(name = "rules_go", version = "0.59.0", repo_name = "io_bazel_rules_go") bazel_dep(name = "gazelle", version = "0.47.0", repo_name = "bazel_gazelle") bazel_dep(name = "rules_cc", version = "0.0.16") diff --git a/gazelle/deps.bzl b/gazelle/deps.bzl index 7072c6a372..c57d5193b8 100644 --- a/gazelle/deps.bzl +++ b/gazelle/deps.bzl @@ -26,9 +26,9 @@ def python_stdlib_list_deps(): http_archive( name = "python_stdlib_list", build_file_content = """exports_files(glob(["stdlib_list/lists/*.txt"]))""", - sha256 = "aa21a4f219530e85ecc364f0bbff2df4e6097a8954c63652af060f4e64afa65d", - strip_prefix = "stdlib-list-0.11.0", - url = "https://github.com/pypi/stdlib-list/releases/download/v0.11.0/v0.11.0.tar.gz", + sha256 = "517824f27ee89e591d8ae7c1dd9ff34f672eae50ee886ea31bb8816d77535675", + strip_prefix = "stdlib_list-0.12.0", + url = "https://files.pythonhosted.org/packages/8c/25/f1540879c8815387980e56f973e54605bd924612399ace31487f7444171c/stdlib_list-0.12.0.tar.gz", ) def gazelle_deps(): diff --git a/gazelle/examples/bzlmod_build_file_generation/MODULE.bazel b/gazelle/examples/bzlmod_build_file_generation/MODULE.bazel index 1f92ea3826..81719be01f 100644 --- a/gazelle/examples/bzlmod_build_file_generation/MODULE.bazel +++ b/gazelle/examples/bzlmod_build_file_generation/MODULE.bazel @@ -13,7 +13,7 @@ module( # For typical setups you set the version. # See the releases page for available versions. # https://github.com/bazel-contrib/rules_python/releases -bazel_dep(name = "rules_python", version = "1.4.0") +bazel_dep(name = "rules_python", version = "1.5.0") # The following stanza defines the dependency rules_python_gazelle_plugin. # For typical setups you set the version. diff --git a/gazelle/python/BUILD.bazel b/gazelle/python/BUILD.bazel index 8218d0b6d9..1ffa2890e1 100644 --- a/gazelle/python/BUILD.bazel +++ b/gazelle/python/BUILD.bazel @@ -54,10 +54,12 @@ copy_file( "@rules_python//python/config_settings:is_python_3.10": "@python_stdlib_list//:stdlib_list/lists/3.10.txt", "@rules_python//python/config_settings:is_python_3.11": "@python_stdlib_list//:stdlib_list/lists/3.11.txt", "@rules_python//python/config_settings:is_python_3.12": "@python_stdlib_list//:stdlib_list/lists/3.12.txt", + "@rules_python//python/config_settings:is_python_3.13": "@python_stdlib_list//:stdlib_list/lists/3.13.txt", + "@rules_python//python/config_settings:is_python_3.14": "@python_stdlib_list//:stdlib_list/lists/3.14.txt", "@rules_python//python/config_settings:is_python_3.8": "@python_stdlib_list//:stdlib_list/lists/3.8.txt", "@rules_python//python/config_settings:is_python_3.9": "@python_stdlib_list//:stdlib_list/lists/3.9.txt", - # This is the same behaviour as previously - "//conditions:default": "@python_stdlib_list//:stdlib_list/lists/3.11.txt", + # Fall back to the latest stdlib list + "//conditions:default": "@python_stdlib_list//:stdlib_list/lists/3.14.txt", }, ), out = "stdlib_list.txt", diff --git a/news/3978.changed.md b/news/3978.changed.md new file mode 100644 index 0000000000..8418abd0b3 --- /dev/null +++ b/news/3978.changed.md @@ -0,0 +1,3 @@ +(gazelle) **BREAKING** rules_python 1.5.0 or higher is now required. The Python +extension selects its standard library list on `is_python_3.14`, which earlier +versions do not define. diff --git a/news/3978.fixed.md b/news/3978.fixed.md new file mode 100644 index 0000000000..2c5cbb5f5d --- /dev/null +++ b/news/3978.fixed.md @@ -0,0 +1,5 @@ +(gazelle) The Python extension now uses the correct standard library module list for +`python_version` 3.13 and 3.14; previously both fell back to the 3.11 list, so modules +added or removed since then (e.g. `compression.zstd`, `telnetlib`) were misclassified. The +fallback list for unrecognized versions is now the newest available one rather than 3.11 +([#3978](https://github.com/bazel-contrib/rules_python/pull/3978)). From 0d6016ffe5307e2e7c30a90319f14f8efa801d2b Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Sat, 1 Aug 2026 08:22:08 -0700 Subject: [PATCH 884/922] feat(cc): introduce py_extension rule for C/C++ Python extension modules (#3973) Building C/C++ Python extension modules currently requires manually configuring complex `cc_binary` or `cc_shared_library` targets and setting up platform, ABI3, and extension tags. The new `py_extension` rule provides a high-level Starlark API for compiling native extensions that integrate seamlessly with `rules_python` toolchains and `py_library` dependencies. At a high level, the implementation is fairly simple: Users can provide raw C/C++ `srcs`, which are put into a `cc_library`. That `cc_library` is then given to `cc_shared_library`, which performs the linking to create e.g. `libfoo.so`. That artifact is then given to a custom rule that handles rename it according to Python needs, e.g. `foo.abi3.so`. `cc_shared_library` is used because it provides various facilities to control creation of a shared library, namely, control over what is and isn't linked into the shared library. Credit to @rsartor-cmd for finishing the prototype, expanding test coverage, and verifying behavior. Closes #3283. --------- Signed-off-by: dependabot[bot] Co-authored-by: rsartor-cmd Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> --- .../scripts/analyze_ci_failure.py | 3 + news/3973.added.md | 6 + python/cc/BUILD.bazel | 7 + python/cc/py_extension.bzl | 20 ++ python/features.bzl | 1 + python/private/BUILD.bazel | 2 + python/private/cc/BUILD.bazel | 45 ++++ python/private/cc/py_extension_macro.bzl | 191 ++++++++++++++ python/private/cc/py_extension_rule.bzl | 135 ++++++++++ python/private/common_labels.bzl | 1 + python/private/current_py_cc_libs.bzl | 34 ++- python/private/flags.bzl | 4 + python/private/py_cc_toolchain_info.bzl | 28 +++ python/private/py_cc_toolchain_macro.bzl | 7 + python/private/py_cc_toolchain_rule.bzl | 105 ++++++++ python/private/py_executable.bzl | 4 +- python/private/toolchain_types.bzl | 1 + .../py_cc_toolchain/py_cc_toolchain_tests.bzl | 31 +++ tests/cc/py_extension/BUILD.bazel | 194 ++++++++++++++ tests/cc/py_extension/add_one.c | 7 + tests/cc/py_extension/add_one.h | 6 + tests/cc/py_extension/add_one_helper.c | 7 + tests/cc/py_extension/add_one_helper.h | 2 + .../py_extension/dependency_graph/BUILD.bazel | 12 + .../dependency_graph_tests.bzl | 236 ++++++++++++++++++ .../dependency_graph/test_lib_a.c | 6 + .../dependency_graph/test_lib_b.c | 5 + .../dependency_graph/test_lib_c.c | 6 + .../dependency_graph/test_symbols.h | 8 + tests/cc/py_extension/ext_init_in_dep.c | 20 ++ tests/cc/py_extension/ext_limited.c | 22 ++ tests/cc/py_extension/ext_pkg_test.c | 22 ++ tests/cc/py_extension/ext_shared.c | 33 +++ tests/cc/py_extension/ext_source.c | 33 +++ tests/cc/py_extension/ext_static.c | 1 + .../cc/py_extension/py_extension/BUILD.bazel | 12 + .../py_extension/py_extension_tests.bzl | 102 ++++++++ .../py_extension/py_extension_build_test.bzl | 15 ++ .../cc/py_extension/py_extension_pkg_test.py | 16 ++ tests/cc/py_extension/py_extension_test.py | 47 ++++ .../py_extension/py_limited_api/BUILD.bazel | 12 + .../py_limited_api/py_limited_api_tests.bzl | 140 +++++++++++ tests/cc/py_extension/some_data.txt | 1 + tests/cc/py_extension/static_dep.c | 5 + tests/cc/py_extension/static_dep.h | 1 + .../bzlmod_lockfile/MODULE.bazel.lock | 2 +- tests/support/cc_toolchains/BUILD.bazel | 2 + .../support/py_cc_toolchain_info_subject.bzl | 28 +++ 48 files changed, 1624 insertions(+), 4 deletions(-) create mode 100644 news/3973.added.md create mode 100644 python/cc/py_extension.bzl create mode 100644 python/private/cc/py_extension_macro.bzl create mode 100644 python/private/cc/py_extension_rule.bzl create mode 100644 tests/cc/py_extension/BUILD.bazel create mode 100644 tests/cc/py_extension/add_one.c create mode 100644 tests/cc/py_extension/add_one.h create mode 100644 tests/cc/py_extension/add_one_helper.c create mode 100644 tests/cc/py_extension/add_one_helper.h create mode 100644 tests/cc/py_extension/dependency_graph/BUILD.bazel create mode 100644 tests/cc/py_extension/dependency_graph/dependency_graph_tests.bzl create mode 100644 tests/cc/py_extension/dependency_graph/test_lib_a.c create mode 100644 tests/cc/py_extension/dependency_graph/test_lib_b.c create mode 100644 tests/cc/py_extension/dependency_graph/test_lib_c.c create mode 100644 tests/cc/py_extension/dependency_graph/test_symbols.h create mode 100644 tests/cc/py_extension/ext_init_in_dep.c create mode 100644 tests/cc/py_extension/ext_limited.c create mode 100644 tests/cc/py_extension/ext_pkg_test.c create mode 100644 tests/cc/py_extension/ext_shared.c create mode 100644 tests/cc/py_extension/ext_source.c create mode 100644 tests/cc/py_extension/ext_static.c create mode 100644 tests/cc/py_extension/py_extension/BUILD.bazel create mode 100644 tests/cc/py_extension/py_extension/py_extension_tests.bzl create mode 100644 tests/cc/py_extension/py_extension_build_test.bzl create mode 100644 tests/cc/py_extension/py_extension_pkg_test.py create mode 100644 tests/cc/py_extension/py_extension_test.py create mode 100644 tests/cc/py_extension/py_limited_api/BUILD.bazel create mode 100644 tests/cc/py_extension/py_limited_api/py_limited_api_tests.bzl create mode 100644 tests/cc/py_extension/some_data.txt create mode 100644 tests/cc/py_extension/static_dep.c create mode 100644 tests/cc/py_extension/static_dep.h diff --git a/.agents/skills/analyze-ci-failure/scripts/analyze_ci_failure.py b/.agents/skills/analyze-ci-failure/scripts/analyze_ci_failure.py index 1f976f3e79..661a427cc7 100644 --- a/.agents/skills/analyze-ci-failure/scripts/analyze_ci_failure.py +++ b/.agents/skills/analyze-ci-failure/scripts/analyze_ci_failure.py @@ -74,6 +74,9 @@ def parse_log(log_path): "no such package", "no such target", "exit code", + "exit-code", + "fatal:", + "fatal", "##[error]", "Would reformat:", "would be reformatted", diff --git a/news/3973.added.md b/news/3973.added.md new file mode 100644 index 0000000000..ef3796afca --- /dev/null +++ b/news/3973.added.md @@ -0,0 +1,6 @@ +(cc) Added experimental {obj}`py_extension` macro for creating C/C++ Python +extension modules +([#3283](https://github.com/bazel-contrib/rules_python/issues/3283)). +(cc) Added `abi_flags`, `abi_tag`, `libc`, `platform_machine`, `platform_tag`, +and `sys_platform` attributes and info fields to {obj}`py_cc_toolchain` / +{obj}`PyCcToolchainInfo`. diff --git a/python/cc/BUILD.bazel b/python/cc/BUILD.bazel index f17805d92f..2fa03dcb5a 100644 --- a/python/cc/BUILD.bazel +++ b/python/cc/BUILD.bazel @@ -79,3 +79,10 @@ alias( deprecation = "Use //python/cc:py_cc_toolchain_info instead", visibility = ["//visibility:public"], ) + +bzl_library( + name = "py_extension", + srcs = ["py_extension.bzl"], + visibility = ["//visibility:public"], + deps = ["//python/private/cc:py_extension_macro"], +) diff --git a/python/cc/py_extension.bzl b/python/cc/py_extension.bzl new file mode 100644 index 0000000000..2d5570b546 --- /dev/null +++ b/python/cc/py_extension.bzl @@ -0,0 +1,20 @@ +"""Rules for creating Python C extension modules. + +This module provides `py_extension` for building Python C extension modules +that can be imported by Python targets (`py_binary`, `py_test`, `py_library`). +It manages dynamic linking, symbol exports (`PyInit_*`), platform-specific link +flags, and target dependencies cleanly. + +See the [Python C API documentation](https://docs.python.org/3/c-api/index.html) +for information on writing C extension modules. + +:::{include} /_includes/experimental_api.md +::: +""" + +load( + "//python/private/cc:py_extension_macro.bzl", + _py_extension = "py_extension", +) + +py_extension = _py_extension diff --git a/python/features.bzl b/python/features.bzl index 33323b8b65..9339d7889c 100644 --- a/python/features.bzl +++ b/python/features.bzl @@ -102,6 +102,7 @@ _TARGETS = { "//python/cc:current_py_cc_headers_abi3": True, "//python/cc:py_cc_toolchain": True, "//python/cc:py_cc_toolchain_info": True, + "//python/cc:py_extension": True, "//python/config_settings:venv": True, "//python/entry_points:py_console_script_binary": True, "//python/local_toolchains:repos": True, diff --git a/python/private/BUILD.bazel b/python/private/BUILD.bazel index a5f4d2da3e..0b0dc97f66 100644 --- a/python/private/BUILD.bazel +++ b/python/private/BUILD.bazel @@ -506,6 +506,7 @@ bzl_library( deps = [ ":py_cc_toolchain_rule", ":util", + "//python/private/pypi:pep508_env", ], ) @@ -514,6 +515,7 @@ bzl_library( srcs = ["py_cc_toolchain_rule.bzl"], deps = [ ":common_labels", + ":flags", ":py_cc_toolchain_info", ":sentinel_impl", "@bazel_skylib//rules:common_settings", diff --git a/python/private/cc/BUILD.bazel b/python/private/cc/BUILD.bazel index 8f4fb468a4..a89c47c58f 100644 --- a/python/private/cc/BUILD.bazel +++ b/python/private/cc/BUILD.bazel @@ -1,3 +1,4 @@ +load("@bazel_skylib//:bzl_library.bzl", "bzl_library") load("@rules_cc//cc:cc_library.bzl", "cc_library") load("//python/private:visibility.bzl", "NOT_ACTUALLY_PUBLIC") @@ -18,3 +19,47 @@ cc_library( name = "empty", visibility = NOT_ACTUALLY_PUBLIC, ) + +# Private alias to avoid "duplicate dependency label" errors when users +# explicitly pass //python/cc:current_py_cc_headers or //python/cc:current_py_cc_libs +# in the deps attribute (including when deps is a select() expression). +alias( + name = "current_py_cc_headers_private_alias", + actual = "//python/cc:current_py_cc_headers", + visibility = NOT_ACTUALLY_PUBLIC, +) + +# Private alias to avoid "duplicate dependency label" errors when users +# explicitly pass //python/cc:current_py_cc_headers or //python/cc:current_py_cc_libs +# in the deps attribute (including when deps is a select() expression). +alias( + name = "current_py_cc_libs_private_alias", + actual = "//python/cc:current_py_cc_libs", + visibility = NOT_ACTUALLY_PUBLIC, +) + +bzl_library( + name = "py_extension_macro", + srcs = ["py_extension_macro.bzl"], + deps = [ + ":py_extension_rule", + "//python/private:util", + "@rules_cc//cc:core_rules", + ], +) + +bzl_library( + name = "py_extension_rule", + srcs = ["py_extension_rule.bzl"], + deps = [ + "//python/private:attr_builders", + "//python/private:attributes", + "//python/private:builders", + "//python/private:py_info", + "//python/private:reexports", + "//python/private:rule_builders", + "//python/private:toolchain_types", + "@bazel_skylib//lib:dicts", + "@rules_cc//cc/common", + ], +) diff --git a/python/private/cc/py_extension_macro.bzl b/python/private/cc/py_extension_macro.bzl new file mode 100644 index 0000000000..7845a09ea2 --- /dev/null +++ b/python/private/cc/py_extension_macro.bzl @@ -0,0 +1,191 @@ +"""Macro for creating Python extensions. + +:::{include} /_includes/experimental_api.md +::: +""" + +load("@rules_cc//cc:cc_library.bzl", "cc_library") +load("@rules_cc//cc:cc_shared_library.bzl", "cc_shared_library") +load("//python/private:common_labels.bzl", "labels") +load("//python/private:util.bzl", "add_tag", "copy_propagating_kwargs") +load(":py_extension_rule.bzl", "py_extension_wrapper") + +_EMPTY_CANONICAL_TARGET = str(Label("//python/private/cc:empty")) + +_PY_CC_HEADERS_ALIAS_BASE_TARGET = "//python/private/cc:current_py_cc_headers_private_alias" +_PY_CC_HEADERS_ALIAS_CANONICAL_TARGET = str(Label(_PY_CC_HEADERS_ALIAS_BASE_TARGET)) + +_PY_CC_LIBS_ALIAS_BASE_TARGET = "//python/private/cc:current_py_cc_libs_private_alias" +_PY_CC_LIBS_ALIAS_CANONICAL_TARGET = str(Label(_PY_CC_LIBS_ALIAS_BASE_TARGET)) + +_PY_CC_LIBS_ACTUAL_BASE_TARGET = "//python/cc:current_py_cc_libs" +_PY_CC_LIBS_ACTUAL_CANONICAL_TARGET = str(Label(_PY_CC_LIBS_ACTUAL_BASE_TARGET)) + +def py_extension( + name, + srcs = None, + hdrs = None, + copts = None, + defines = None, + local_defines = None, + includes = None, + linkopts = None, + deps = None, + dynamic_deps = None, + exports_filter = None, + user_link_flags = None, + additional_linker_inputs = None, + module_name = None, + py_limited_api = None, + **kwargs): + """Creates a Python extension module. + + :::{include} /_includes/experimental_api.md + ::: + + By default, extensions are created within their workspace package directory + (e.g., `pkg/ext.so`) and imported using standard Python package paths + (e.g., `from pkg import ext`). + + To customize import path behavior: + - `imports`: Pass `imports = ["..."]` to append custom search directories to + `sys.path` (matching {attr}`py_library.imports`). + - `module_name`: Pass `module_name = "custom_name"` to override the base + module filename. + + Args: + name: {type}`str` Target name. + srcs: {type}`list[Label | str] | None` C/C++ source files to compile + directly for this extension. + hdrs: {type}`list[Label | str] | None` Header files for `srcs`. + copts: {type}`list[str] | None` Compiler flags for `srcs`. + defines: {type}`list[str] | None` Preprocessor defines for `srcs`. + local_defines: {type}`list[str] | None` Preprocessor defines for `srcs` + passed to internal `cc_library`. + includes: {type}`list[str] | None` Header include search paths passed + to internal `cc_library`. + linkopts: {type}`list[str] | None` Link options passed to internal + `cc_library` created for `srcs`/`hdrs`. To pass linker flags to + `cc_shared_library`, use `user_link_flags`. + deps: {type}`list[Label | str] | None` `cc_library` targets to + statically link into the extension. + dynamic_deps: {type}`list[Label | str] | None` `cc_shared_library` + targets to dynamically link. + exports_filter: {type}`list[str] | None` Filter for exported symbols + passed to `cc_shared_library`. + user_link_flags: {type}`list[str] | None` Additional link flags passed + to `cc_shared_library`. To pass linker flags that apply to `srcs`, + use `linkopts`. + additional_linker_inputs: {type}`list[Label | str] | None` Additional + linker inputs passed to `cc_shared_library`. + module_name: {type}`str | None` Custom Python module name. If not set, + defaults to `name`. + py_limited_api: {type}`str | None` Python limited API version string + (e.g., `"3.8"`). + **kwargs: {type}`dict` Additional arguments passed to the underlying + wrapper rule. + """ + add_tag(kwargs, "@rules_python//python/cc:py_extension") + additional_linker_inputs = additional_linker_inputs or [] + copts = copts or [] + deps = deps or [] + user_link_flags = user_link_flags or [] + + csl_deps = [] + + copts = copts + select({ + # -fPIC (Position Independent Code) is required when compiling C/C++ sources into + # dynamic/shared libraries (.so/.dylib/.pyd) so code can be loaded at arbitrary addresses. + # MSVC on Windows does not support or require -fPIC. + labels.PLATFORMS_OS_WINDOWS: [], + "//conditions:default": ["-fPIC"], + }) + + # Private alias targets are appended to avoid "duplicate dependency label" errors + # if a user explicitly passes //python/cc:current_py_cc_headers or //python/cc:current_py_cc_libs + # in their deps attribute (including when deps is a select() expression). + deps = deps + [_PY_CC_HEADERS_ALIAS_CANONICAL_TARGET] + + # 1. If srcs or hdrs are specified, create an implicit cc_library for them + if srcs or hdrs: + impl_lib_name = "_" + name + "_impl" + impl_lib_kwargs = copy_propagating_kwargs(kwargs) + if includes: + impl_lib_kwargs["includes"] = includes + if linkopts: + impl_lib_kwargs["linkopts"] = linkopts + cc_library( + name = impl_lib_name, + srcs = srcs, + hdrs = hdrs, + copts = copts, + defines = defines, + local_defines = local_defines, + deps = deps, + visibility = ["//visibility:private"], + **impl_lib_kwargs + ) + csl_deps.append(":" + impl_lib_name) + + if not csl_deps: + csl_deps = deps + if not csl_deps: + # cc_shared_library requires a dependency, so use an empty library when none are given. + csl_deps = [_EMPTY_CANONICAL_TARGET] + + # 4. Create the underlying cc_shared_library + csl_name = "_" + name + "_csl" + csl_kwargs = copy_propagating_kwargs(kwargs) + + if exports_filter != None: + csl_kwargs["exports_filter"] = exports_filter + + user_link_flags = user_link_flags + select({ + # On macOS, Apple's ld64 linker requires '-undefined dynamic_lookup' so CPython + # C-API symbols (e.g. PyModule_Create) remain unresolved at link time and are + # dynamically resolved at runtime when CPython loads the shared library (.so). + labels.PLATFORMS_OS_MACOS: ["-undefined", "dynamic_lookup"], + "//conditions:default": [], + }) + + # Windows-specific CPython linking requirements: + # 1. Windows requires .lib files when linking, so they must be added to deps. + # 2. CPython import libraries (python3xx.lib) are declared with system_provided = True + # in cc_import, suppressing automatic propagation of the .lib file path to link.exe. + # We explicitly pass $(locations ...) to provide the path of the CPython import library to MSVC link.exe. + # 3. We pass current_py_cc_libs as an additional linker input to ensure the .lib file is available to the link action. + deps = deps + select({ + labels.PLATFORMS_OS_WINDOWS: [_PY_CC_LIBS_ALIAS_CANONICAL_TARGET], + "//conditions:default": [], + }) + user_link_flags = user_link_flags + select({ + labels.PLATFORMS_OS_WINDOWS: ["$(locations " + _PY_CC_LIBS_ACTUAL_BASE_TARGET + ")"], + "//conditions:default": [], + }) + additional_linker_inputs = additional_linker_inputs + select({ + labels.PLATFORMS_OS_WINDOWS: [_PY_CC_LIBS_ACTUAL_CANONICAL_TARGET], + "//conditions:default": [], + }) + + cc_shared_library( + name = csl_name, + deps = csl_deps, + additional_linker_inputs = additional_linker_inputs, + dynamic_deps = dynamic_deps, + user_link_flags = user_link_flags, + visibility = ["//visibility:private"], + **csl_kwargs + ) + + # 5. Filter out C++ specific compilation/linking attributes before invoking wrapper rule + for cc_attr in ("includes", "linkopts", "linkshared", "linkstatic", "features"): + kwargs.pop(cc_attr, None) + + # 6. Wrap with py_extension_wrapper for PEP 3149 naming & PyInfo + py_extension_wrapper( + name = name, + src = ":" + csl_name, + module_name = module_name, + py_limited_api = py_limited_api, + **kwargs + ) diff --git a/python/private/cc/py_extension_rule.bzl b/python/private/cc/py_extension_rule.bzl new file mode 100644 index 0000000000..1c5ee11a06 --- /dev/null +++ b/python/private/cc/py_extension_rule.bzl @@ -0,0 +1,135 @@ +"""Implementation of rules supporting py_extension. + +:::{include} /_includes/experimental_api.md +::: +""" + +load("@bazel_skylib//lib:dicts.bzl", "dicts") +load("@rules_cc//cc/common:cc_shared_library_info.bzl", "CcSharedLibraryInfo") +load("//python/private:attr_builders.bzl", "attrb") +load("//python/private:attributes.bzl", "COMMON_ATTRS", "IMPORTS_ATTRS", "WINDOWS_CONSTRAINTS_ATTRS") +load("//python/private:builders.bzl", "builders") +load("//python/private:common.bzl", "get_imports", "is_windows_platform") +load("//python/private:py_info.bzl", "PyInfo", "PyInfoBuilder") +load("//python/private:rule_builders.bzl", "ruleb") +load("//python/private:toolchain_types.bzl", "CC_TOOLCHAIN_TYPE", "PY_CC_TOOLCHAIN_TYPE") + +def _py_extension_wrapper_impl(ctx): + module_name = ctx.attr.module_name or ctx.label.name + + ext = _get_extension(ctx) + use_py_limited_api = bool(ctx.attr.py_limited_api) + if use_py_limited_api: + output_filename = "{module_name}.abi3.{ext}".format( + module_name = module_name, + ext = ext, + ) + else: + py_toolchain = ctx.toolchains[PY_CC_TOOLCHAIN_TYPE] + py_cc_toolchain = py_toolchain.py_cc_toolchain + platform_tag = _get_platform(ctx) + output_filename = "{module_name}.{abi_tag}-{platform}.{ext}".format( + module_name = module_name, + abi_tag = py_cc_toolchain.abi_tag, + platform = platform_tag, + ext = ext, + ) + + py_dso = ctx.actions.declare_file(output_filename) + + # Symlink the cc_shared_library output to the PEP 3149 / abi3 filename + csl_target = ctx.attr.src + csl_file = csl_target[DefaultInfo].files.to_list()[0] + ctx.actions.symlink( + output = py_dso, + target_file = csl_file, + ) + + runfiles_builder = builders.RunfilesBuilder() + runfiles_builder.add(py_dso) + runfiles_builder.add(ctx.files.data) + runfiles_builder.add_targets(ctx.attr.data) + runfiles_builder.add(csl_target[DefaultInfo].default_runfiles) + runfiles = runfiles_builder.build(ctx) + + py_info_builder = PyInfoBuilder.new() + py_info_builder.transitive_sources.add(py_dso) + py_info_builder.imports.add(get_imports(ctx)) + + return [ + DefaultInfo( + files = depset([py_dso]), + runfiles = runfiles, + ), + py_info_builder.build(), + ] + +PY_EXTENSION_WRAPPER_ATTRS = dicts.add( + COMMON_ATTRS, + IMPORTS_ATTRS, + WINDOWS_CONSTRAINTS_ATTRS, + { + "module_name": lambda: attrb.String( + doc = "Custom Python module name. If not set, defaults to name.", + ), + "py_limited_api": lambda: attrb.String( + default = "", + doc = "Python limited API version string (e.g., '3.8').", + ), + "src": lambda: attrb.Label( + mandatory = True, + providers = [CcSharedLibraryInfo], + doc = "The cc_shared_library target to wrap.", + ), + }, +) + +def create_py_extension_wrapper_rule_builder(): + """Create a rule builder for the private internal wrapper rule.""" + return ruleb.Rule( + doc = "Private internal helper rule for py_extension targets.", + implementation = _py_extension_wrapper_impl, + attrs = PY_EXTENSION_WRAPPER_ATTRS, + provides = [PyInfo], + toolchains = [ + ruleb.ToolchainType(PY_CC_TOOLCHAIN_TYPE), + ruleb.ToolchainType(CC_TOOLCHAIN_TYPE), + ], + fragments = ["cpp"], + ) + +py_extension_wrapper = create_py_extension_wrapper_rule_builder().build() + +def _get_extension(ctx): + """Derives the appropriate file extension for C extensions from target platform. + + Note: On macOS, CPython C extensions use .so (PEP 3149), not .dylib. + Windows uses .pyd. + + Args: + ctx: The rule context. + + Returns: + The extension, e.g. "so" or "pyd" + """ + return "pyd" if is_windows_platform(ctx) else "so" + +def _get_platform(ctx): + """Derives the PEP 3149 platform tag from the active Python C++ toolchain. + + Args: + ctx: The rule context. + + Returns: + The platform tag, e.g. "x86_64-linux-gnu" or "win_amd64" + """ + py_toolchain = ctx.toolchains[PY_CC_TOOLCHAIN_TYPE] + py_cc_toolchain = py_toolchain.py_cc_toolchain + if not py_cc_toolchain.platform_tag: + fail( + ("ERROR: Unable to resolve platform_tag from Python C++ toolchain for {self}. " + + "Please ensure the active py_cc_toolchain provides a non-empty platform_tag.").format( + self = ctx.label, + ), + ) + return py_cc_toolchain.platform_tag diff --git a/python/private/common_labels.bzl b/python/private/common_labels.bzl index 6cc42ecf8d..db4a00ba0a 100644 --- a/python/private/common_labels.bzl +++ b/python/private/common_labels.bzl @@ -18,6 +18,7 @@ labels = struct( NONE = str(Label("//python:none")), PIP_ENV_MARKER_CONFIG = str(Label("//python/config_settings:pip_env_marker_config")), PIP_WHL_OSX_VERSION = str(Label("//python/config_settings:pip_whl_osx_version")), + PLATFORMS_OS_MACOS = str(Label("@platforms//os:macos")), PLATFORMS_OS_WINDOWS = str(Label("@platforms//os:windows")), PRECOMPILE = str(Label("//python/config_settings:precompile")), PRECOMPILE_SOURCE_RETENTION = str(Label("//python/config_settings:precompile_source_retention")), diff --git a/python/private/current_py_cc_libs.bzl b/python/private/current_py_cc_libs.bzl index ca68346bcb..58ab4b1bd8 100644 --- a/python/private/current_py_cc_libs.bzl +++ b/python/private/current_py_cc_libs.bzl @@ -18,7 +18,39 @@ load("@rules_cc//cc/common:cc_info.bzl", "CcInfo") def _current_py_cc_libs_impl(ctx): py_cc_toolchain = ctx.toolchains["//python/cc:toolchain_type"].py_cc_toolchain - return py_cc_toolchain.libs.providers_map.values() + providers = [p for p in py_cc_toolchain.libs.providers_map.values() if not hasattr(p, "data_runfiles")] + default_runfiles = None + data_runfiles = None + files = [] + for p in py_cc_toolchain.libs.providers_map.values(): + if hasattr(p, "data_runfiles"): + default_runfiles = p.default_runfiles + data_runfiles = p.data_runfiles + + cc_infos = [p for p in py_cc_toolchain.libs.providers_map.values() if hasattr(p, "linking_context")] + if cc_infos: + cc_info = cc_infos[0] + for input in cc_info.linking_context.linker_inputs.to_list(): + for lib in input.libraries: + if lib.static_library: + files.append(lib.static_library) + if lib.interface_library: + files.append(lib.interface_library) + elif lib.dynamic_library: + files.append(lib.dynamic_library) + + # On Windows MSVC, user_link_flags passes $(locations @rules_python//python/cc:current_py_cc_libs) + # to link.exe. MSVC link.exe accepts import libraries (.lib) but fails with + # LNK1107 if passed raw DLL binaries (.dll). We filter out .dll files so + # DefaultInfo.files only contains linkable library files (.lib / .a). + link_files = [f for f in files if not f.path.endswith(".dll")] + + providers.append(DefaultInfo( + files = depset(link_files), + default_runfiles = default_runfiles, + data_runfiles = data_runfiles, + )) + return providers current_py_cc_libs = rule( implementation = _current_py_cc_libs_impl, diff --git a/python/private/flags.bzl b/python/private/flags.bzl index 32769e4084..042e4e9838 100644 --- a/python/private/flags.bzl +++ b/python/private/flags.bzl @@ -247,6 +247,9 @@ FreeThreadedFlag = enum( NO = "no", ) +def _libc_flag_get_value(ctx): + return ctx.attr._py_linux_libc_flag[BuildSettingInfo].value + # Determines which libc flavor is preferred when selecting the toolchain and # linux whl distributions. # @@ -256,4 +259,5 @@ LibcFlag = FlagEnum( GLIBC = "glibc", # Prefer musl wheels (e.g. musllinux_2_17_x86_64) MUSL = "musl", + get_value = _libc_flag_get_value, ) diff --git a/python/private/py_cc_toolchain_info.bzl b/python/private/py_cc_toolchain_info.bzl index 34d4acf305..fcf407516c 100644 --- a/python/private/py_cc_toolchain_info.bzl +++ b/python/private/py_cc_toolchain_info.bzl @@ -17,6 +17,16 @@ PyCcToolchainInfo = provider( doc = "C/C++ information about the Python runtime.", fields = { + "abi_flags": """\ +:type: str + +The runtime's ABI flags, i.e. `sys.abiflags` (e.g. 't' for free-threaded builds). +""", + "abi_tag": """\ +:type: str + +The ABI tag for extension modules, e.g. 'cpython-311' or 'cpython-313t'. +""", "headers": """\ :type: struct @@ -91,11 +101,29 @@ If available, information about C libraries, struct with fields: considered private and should be forward along as-is (this better allows e.g. `:current_py_cc_headers` to act as the underlying headers target it represents). +""", + "platform_machine": """ +:type: str + +The [PEP 508](https://peps.python.org/pep-0508/) `platform_machine` marker +value for the target architecture, e.g. 'x86_64', 'aarch64'. +""", + "platform_tag": """\ +:type: str | None + +The PEP 3149 / PEP 425 platform tag for extension modules, e.g. +'x86_64-linux-gnu', 'darwin', or 'win_amd64'. """, "python_version": """ :type: str The Python Major.Minor version. +""", + "sys_platform": """ +:type: str + +The [PEP 508](https://peps.python.org/pep-0508/) `sys_platform` marker value +for the target OS, e.g. 'linux', 'darwin', 'win32'. """, }, ) diff --git a/python/private/py_cc_toolchain_macro.bzl b/python/private/py_cc_toolchain_macro.bzl index 416caac2ab..88223c747c 100644 --- a/python/private/py_cc_toolchain_macro.bzl +++ b/python/private/py_cc_toolchain_macro.bzl @@ -14,6 +14,7 @@ """Fronting macro for the py_cc_toolchain rule.""" +load("//python/private/pypi:pep508_env.bzl", "platform_machine_select_map", "sys_platform_select_map") load(":py_cc_toolchain_rule.bzl", _py_cc_toolchain = "py_cc_toolchain") load(":util.bzl", "add_tag") @@ -30,4 +31,10 @@ def py_cc_toolchain(**kwargs): # This tag is added to easily identify usages through other macros. add_tag(kwargs, "@rules_python//python:py_cc_toolchain") + + if kwargs.get("sys_platform") == None: + kwargs["sys_platform"] = select(sys_platform_select_map) + if kwargs.get("platform_machine") == None: + kwargs["platform_machine"] = select(platform_machine_select_map) + _py_cc_toolchain(**kwargs) diff --git a/python/private/py_cc_toolchain_rule.bzl b/python/private/py_cc_toolchain_rule.bzl index 315de2d3f2..194ce7be78 100644 --- a/python/private/py_cc_toolchain_rule.bzl +++ b/python/private/py_cc_toolchain_rule.bzl @@ -21,9 +21,46 @@ https://github.com/bazel-contrib/rules_python/issues/824 is considered done. load("@bazel_skylib//rules:common_settings.bzl", "BuildSettingInfo") load("@rules_cc//cc/common:cc_info.bzl", "CcInfo") load(":common_labels.bzl", "labels") +load(":flags.bzl", "FreeThreadedFlag", "LibcFlag") load(":py_cc_toolchain_info.bzl", "PyCcToolchainInfo") load(":sentinel_impl.bzl", "SentinelInfo") +def _get_platform_tag(sys_platform, platform_machine, libc): + """Derives the PEP 3149 platform tag string. + + Note that these are platform tags for C extension filenames, not + PEP 425 tags for wheels. + + Linux platform tags are standardized here: + - https://peps.python.org/pep-3149/ + Windows platform tags, such as they are, are defined in this issue and + commit (treated as a de facto standard): + - https://github.com/python/cpython/issues/67169 + - https://github.com/python/cpython/commit/03a144bb6ac3d7631a3bdb895e2a1f2d021fb08b + Apple platform tag is always just "darwin", discussed briefly here: + - https://github.com/python/cpython/commit/3b8124884c3655b4cf2629d741b18c1a38181805 + + Args: + sys_platform: Target PEP 508 OS marker, e.g. "win32", "darwin", "linux" + platform_machine: Target PEP 508 CPU marker, e.g. "x86_64", "aarch64", "x86_32" + libc: Target C library variant, e.g. "glibc", "musl" + + Returns: + The platform tag, e.g. "x86_64-linux-gnu", "darwin", or "win_amd64" + """ + if sys_platform == "win32": + if platform_machine in ("x86_64", "amd64"): + return "win_amd64" + if platform_machine in ("aarch64", "arm64"): + return "win_arm64" + return "win32" + if sys_platform == "darwin": + return "darwin" + + machine_val = platform_machine if platform_machine else "x86_64" + libc_val = "gnu" if libc == LibcFlag.GLIBC else libc + return "{}-linux-{}".format(machine_val, libc_val) + def _py_cc_toolchain_impl(ctx): if ctx.attr.libs: libs = struct( @@ -45,7 +82,36 @@ def _py_cc_toolchain_impl(ctx): else: headers_abi3 = None + abi_flags = ctx.attr.abi_flags + if abi_flags == "": + abi_flags = "" + if ctx.attr._py_freethreaded_flag[BuildSettingInfo].value == FreeThreadedFlag.YES: + abi_flags += "t" + + abi_tag = ctx.attr.abi_tag + if not abi_tag: + # Derive default ABI tag: + # On POSIX: cpython-XX[t] (PEP 3149 / PEP 703) + # On Windows: cpXX[t] (PEP 3149 / PEP 703, CPython issue & commit): + # - https://peps.python.org/pep-3149/ + # - https://peps.python.org/pep-0703/ + # - https://github.com/python/cpython/issues/67169 + # - https://github.com/python/cpython/commit/03a144bb6ac3d7631a3bdb895e2a1f2d021fb08b + version_parts = ctx.attr.python_version.split(".") + prefix = "cp" if ctx.attr.sys_platform == "win32" else "cpython-" + abi_tag = "{}{}{}{}".format(prefix, version_parts[0], version_parts[1], abi_flags) + + libc = ctx.attr.libc or LibcFlag.get_value(ctx) + + platform_tag = _get_platform_tag( + sys_platform = ctx.attr.sys_platform, + platform_machine = ctx.attr.platform_machine, + libc = libc, + ) + py_cc_toolchain = PyCcToolchainInfo( + abi_flags = abi_flags, + abi_tag = abi_tag, headers = struct( providers_map = { "CcInfo": ctx.attr.headers[CcInfo], @@ -54,7 +120,10 @@ def _py_cc_toolchain_impl(ctx): ), headers_abi3 = headers_abi3, libs = libs, + platform_machine = ctx.attr.platform_machine, + platform_tag = platform_tag, python_version = ctx.attr.python_version, + sys_platform = ctx.attr.sys_platform, ) extra_kwargs = {} if ctx.attr._visible_for_testing[BuildSettingInfo].value: @@ -67,6 +136,20 @@ def _py_cc_toolchain_impl(ctx): py_cc_toolchain = rule( implementation = _py_cc_toolchain_impl, attrs = { + "abi_flags": attr.string( + default = "", + doc = """ +The runtime's ABI flags, i.e. `sys.abiflags`. + +If not set, or set to ``, the ABI flags are automatically derived +from `--//python/config_settings:py_freethreaded` (e.g., `'t'` when +free-threaded is enabled, or `''` otherwise). +""", + ), + "abi_tag": attr.string( + doc = "The ABI tag for extension modules, e.g. 'cpython-311'", + default = "", + ), "headers": attr.label( doc = ("Target that provides the Python headers. Typically this " + "is a cc_library target."), @@ -87,15 +170,37 @@ attribute is available or not. default = "//python:none", providers = [[SentinelInfo], [CcInfo]], ), + "libc": attr.string( + doc = "Target C library variant, e.g. 'glibc', 'musl'", + default = "", + ), "libs": attr.label( doc = ("Target that provides the Python runtime libraries for linking. " + "Typically this is a cc_library target of `.so` files."), providers = [CcInfo], ), + "platform_machine": attr.string( + doc = """ +Target architecture as a PEP 508 `platform_machine` marker, e.g. 'x86_64', 'aarch64', 'x86_32'. +""", + default = "", + ), "python_version": attr.string( doc = "The Major.minor Python version, e.g. 3.11", mandatory = True, ), + "sys_platform": attr.string( + doc = """ +Target OS as a PEP 508 `sys_platform` marker, e.g. 'linux', 'darwin', 'win32'. +""", + default = "", + ), + "_py_freethreaded_flag": attr.label( + default = labels.PY_FREETHREADED, + ), + "_py_linux_libc_flag": attr.label( + default = labels.PY_LINUX_LIBC, + ), "_visible_for_testing": attr.label( default = labels.VISIBLE_FOR_TESTING, ), diff --git a/python/private/py_executable.bzl b/python/private/py_executable.bzl index 242b09344e..1ee6fd7c99 100644 --- a/python/private/py_executable.bzl +++ b/python/private/py_executable.bzl @@ -68,7 +68,7 @@ load(":py_interpreter_program.bzl", "PyInterpreterProgramInfo") load(":py_runtime_info.bzl", "DEFAULT_STUB_SHEBANG") load(":reexports.bzl", "BuiltinPyInfo", "BuiltinPyRuntimeInfo") load(":rule_builders.bzl", "ruleb") -load(":toolchain_types.bzl", "EXEC_TOOLS_TOOLCHAIN_TYPE", "LAUNCHER_MAKER_TOOLCHAIN_TYPE", TOOLCHAIN_TYPE = "TARGET_TOOLCHAIN_TYPE") +load(":toolchain_types.bzl", "CC_TOOLCHAIN_TYPE", "EXEC_TOOLS_TOOLCHAIN_TYPE", "LAUNCHER_MAKER_TOOLCHAIN_TYPE", TOOLCHAIN_TYPE = "TARGET_TOOLCHAIN_TYPE") load(":transition_labels.bzl", "TRANSITION_LABELS") load(":venv_runfiles.bzl", "create_venv_app_files") @@ -2204,7 +2204,7 @@ def create_executable_rule_builder(implementation, **kwargs): toolchains = [ ruleb.ToolchainType(TOOLCHAIN_TYPE), ruleb.ToolchainType(EXEC_TOOLS_TOOLCHAIN_TYPE, mandatory = False), - ruleb.ToolchainType("@bazel_tools//tools/cpp:toolchain_type", mandatory = False), + ruleb.ToolchainType(CC_TOOLCHAIN_TYPE, mandatory = False), ] + ([ruleb.ToolchainType(LAUNCHER_MAKER_TOOLCHAIN_TYPE)] if rp_config.bazel_9_or_later else []), cfg = dict( implementation = _transition_executable_impl, diff --git a/python/private/toolchain_types.bzl b/python/private/toolchain_types.bzl index 5b5cce90ee..a3a52409a6 100644 --- a/python/private/toolchain_types.bzl +++ b/python/private/toolchain_types.bzl @@ -22,3 +22,4 @@ TARGET_TOOLCHAIN_TYPE = Label("//python:toolchain_type") EXEC_TOOLS_TOOLCHAIN_TYPE = Label("//python:exec_tools_toolchain_type") PY_CC_TOOLCHAIN_TYPE = Label("//python/cc:toolchain_type") LAUNCHER_MAKER_TOOLCHAIN_TYPE = Label("@bazel_tools//tools/launcher:launcher_maker_toolchain_type") +CC_TOOLCHAIN_TYPE = Label("@bazel_tools//tools/cpp:toolchain_type") diff --git a/tests/cc/py_cc_toolchain/py_cc_toolchain_tests.bzl b/tests/cc/py_cc_toolchain/py_cc_toolchain_tests.bzl index 975f0d56b5..5b57810d0d 100644 --- a/tests/cc/py_cc_toolchain/py_cc_toolchain_tests.bzl +++ b/tests/cc/py_cc_toolchain/py_cc_toolchain_tests.bzl @@ -47,6 +47,7 @@ def _test_py_cc_toolchain_impl(env, target): meta = env.expect.meta.derive(expr = "py_cc_toolchain_info"), ) toolchain.python_version().equals("3.999") + toolchain.abi_flags().equals("") # ===== Verify headers info ===== headers_providers = toolchain.headers().providers_map() @@ -118,8 +119,38 @@ def _test_py_cc_toolchain_impl(env, target): matching.str_matches("/libpython3."), ) + # ===== Verify PEP 508 platform markers ===== + toolchain.sys_platform().equals("linux") + toolchain.platform_machine().equals("x86_64") + toolchain.platform_tag().equals("x86_64-linux-gnu") + _tests.append(_test_py_cc_toolchain) +def _test_custom_pep508_markers(name): + py_cc_toolchain( + name = name + "_subject", + headers = "//tests/support/cc_toolchains:py_headers", + platform_machine = "arm64", + python_version = "3.11", + sys_platform = "darwin", + ) + analysis_test( + name = name, + target = name + "_subject", + impl = _test_custom_pep508_markers_impl, + ) + +def _test_custom_pep508_markers_impl(env, target): + toolchain = PyCcToolchainInfoSubject.new( + target[platform_common.ToolchainInfo].py_cc_toolchain, + meta = env.expect.meta.derive(expr = "py_cc_toolchain_info"), + ) + toolchain.sys_platform().equals("darwin") + toolchain.platform_machine().equals("arm64") + toolchain.platform_tag().equals("darwin") + +_tests.append(_test_custom_pep508_markers) + def _test_libs_optional(name): py_cc_toolchain( name = name + "_subject", diff --git a/tests/cc/py_extension/BUILD.bazel b/tests/cc/py_extension/BUILD.bazel new file mode 100644 index 0000000000..e6edde36e6 --- /dev/null +++ b/tests/cc/py_extension/BUILD.bazel @@ -0,0 +1,194 @@ +load("@rules_cc//cc:cc_library.bzl", "cc_library") +load("@rules_cc//cc:cc_shared_library.bzl", "cc_shared_library") +load("//python:py_test.bzl", "py_test") + +# buildifier: disable=bzl-visibility +load("//python/cc:py_extension.bzl", "py_extension") +load(":py_extension_build_test.bzl", "py_extension_build_test") + +package( + default_visibility = ["//tests/cc/py_extension:__subpackages__"], +) + +licenses(["notice"]) + +##### + +py_extension_build_test( + # An extension defined solely by source files, with no deps + name = "ext_source", + srcs = ["ext_source.c"], +) + +##### + +py_extension_build_test( + # A python extension that explicitly includes current_py_cc_headers in deps. + # Verifies that explicit current_py_cc_headers does not cause a duplicate label error. + name = "ext_explicit_headers", + srcs = ["ext_source.c"], + deps = [ + "@rules_python//python/cc:current_py_cc_headers", + ], +) + +##### + +py_extension_build_test( + # A python extension that includes current_py_cc_headers inside a select() in deps. + # Verifies that select() in deps works cleanly without duplicate label or type error. + name = "ext_explicit_headers_select", + srcs = ["ext_source.c"], + deps = select({ + "//conditions:default": ["@rules_python//python/cc:current_py_cc_headers"], + }), +) + +##### + +py_extension_build_test( + # A python extension that gets its code from a statically-linked library + name = "ext_static", + deps = [":static_dep"], +) + +cc_library( + name = "static_dep", + srcs = ["static_dep.c"], + hdrs = ["static_dep.h"], +) + +##### + +py_extension_build_test( + # An extension that also depends on a data file + name = "ext_with_data", + data = ["some_data.txt"], + deps = [":static_dep"], +) + +##### + +py_extension_build_test( + # A python extension that dynamically links to another shared library + name = "ext_shared", + dynamic_deps = [ + ":add_one_shared", + ], + imports = ["."], + deps = [ + # + ":ext_shared_impl", + ], +) + +cc_library( + name = "ext_shared_impl", + srcs = ["ext_shared.c"], + copts = [ + # Gemini says PIC is needed + "-fPIC", + "-fvisibility=hidden", + ], + deps = [ + #":add_one_headers", + # todo: if we put this here, we statically link add_one into the + # extension. + ":add_one_impl", + "@rules_python//python/cc:current_py_cc_headers", + ], +) + +cc_shared_library( + name = "add_one_shared", + user_link_flags = select({ + "@platforms//os:windows": ["/EXPORT:add_one"], + "//conditions:default": [], + }), + deps = [":add_one_impl"], +) + +cc_library( + name = "add_one_headers", + hdrs = ["add_one.h"], +) + +cc_library( + name = "add_one_impl", + srcs = ["add_one.c"], + deps = [ + ":add_one_headers", + ":add_one_helper", + ], +) + +cc_library( + name = "add_one_helper", + srcs = ["add_one_helper.c"], + hdrs = ["add_one_helper.h"], +) + +##### + +py_extension_build_test( + # An extension that uses the Python limited API + name = "ext_limited", + py_limited_api = "3.8", + deps = [":ext_limited_impl"], +) + +cc_library( + name = "ext_limited_impl", + srcs = ["ext_limited.c"], + copts = [ + "-fPIC", + "-fvisibility=hidden", + ], + defines = ["Py_LIMITED_API=0x3080000"], + deps = [ + "@rules_python//python/cc:current_py_cc_headers", + ], +) + +##### + +py_extension_build_test( + # An extension with its PyInit_* function in a static dependency + name = "ext_init_in_dep", + deps = [":ext_init_in_dep_impl"], +) + +cc_library( + name = "ext_init_in_dep_impl", + srcs = ["ext_init_in_dep.c"], + deps = [ + "@rules_python//python/cc:current_py_cc_headers", + ], +) + +##### + +py_test( + name = "py_extension_test", + srcs = ["py_extension_test.py"], + deps = [ + ":ext_shared", + "@dev_pip//pyelftools", + "@rules_python//python/runfiles", + ], +) + +##### + +py_extension( + name = "ext_pkg_test", + srcs = ["ext_pkg_test.c"], +) + +py_test( + name = "py_extension_pkg_test", + srcs = ["py_extension_pkg_test.py"], + deps = [ + ":ext_pkg_test", + ], +) diff --git a/tests/cc/py_extension/add_one.c b/tests/cc/py_extension/add_one.c new file mode 100644 index 0000000000..7da8f6796e --- /dev/null +++ b/tests/cc/py_extension/add_one.c @@ -0,0 +1,7 @@ + +#include "add_one_helper.h" + +int add_one(int x) { + x = add_one_helper(x); + return x + 1; +} diff --git a/tests/cc/py_extension/add_one.h b/tests/cc/py_extension/add_one.h new file mode 100644 index 0000000000..eda0abf7e5 --- /dev/null +++ b/tests/cc/py_extension/add_one.h @@ -0,0 +1,6 @@ +#ifndef TESTS_CC_PY_EXTENSION_DYN_DEP_A_H_ +#define TESTS_CC_PY_EXTENSION_DYN_DEP_A_H_ + +int add_one(int x); + +#endif // TESTS_CC_PY_EXTENSION_DYN_DEP_A_H_ diff --git a/tests/cc/py_extension/add_one_helper.c b/tests/cc/py_extension/add_one_helper.c new file mode 100644 index 0000000000..21d19a5823 --- /dev/null +++ b/tests/cc/py_extension/add_one_helper.c @@ -0,0 +1,7 @@ + + +#include "add_one_helper.h" + +int add_one_helper(int i) { + return i + 1; +} diff --git a/tests/cc/py_extension/add_one_helper.h b/tests/cc/py_extension/add_one_helper.h new file mode 100644 index 0000000000..5524d3077f --- /dev/null +++ b/tests/cc/py_extension/add_one_helper.h @@ -0,0 +1,2 @@ + +int add_one_helper(int i); diff --git a/tests/cc/py_extension/dependency_graph/BUILD.bazel b/tests/cc/py_extension/dependency_graph/BUILD.bazel new file mode 100644 index 0000000000..47bb5062a0 --- /dev/null +++ b/tests/cc/py_extension/dependency_graph/BUILD.bazel @@ -0,0 +1,12 @@ +load(":dependency_graph_tests.bzl", "dependency_graph_test_suite") + +package( + default_testonly = True, + default_visibility = ["//visibility:private"], +) + +licenses(["notice"]) + +dependency_graph_test_suite( + name = "dependency_graph_tests", +) diff --git a/tests/cc/py_extension/dependency_graph/dependency_graph_tests.bzl b/tests/cc/py_extension/dependency_graph/dependency_graph_tests.bzl new file mode 100644 index 0000000000..269c29eacb --- /dev/null +++ b/tests/cc/py_extension/dependency_graph/dependency_graph_tests.bzl @@ -0,0 +1,236 @@ +# Copyright 2025 The Bazel Authors. All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +"""Parity tests comparing cc_shared_library and py_extension behavior.""" + +load("@rules_cc//cc:cc_library.bzl", "cc_library") +load("@rules_cc//cc:cc_shared_library.bzl", "cc_shared_library") +load("@rules_cc//cc/common:cc_shared_library_info.bzl", "CcSharedLibraryInfo") +load("@rules_testing//lib:analysis_test.bzl", "analysis_test", "test_suite") +load("@rules_testing//lib:util.bzl", "util") + +_tests = [] + +# For tests 1 and 2 +def _create_dynamic_deps_helpers(name): + util.helper_target( + cc_library, + name = name + "_libC", + srcs = ["test_lib_c.c"], + hdrs = ["test_symbols.h"], + copts = ["-fPIC"], + ) + util.helper_target( + cc_shared_library, + name = name + "_cslC", + deps = [":" + name + "_libC"], + ) + util.helper_target( + cc_library, + name = name + "_libB", + srcs = ["test_lib_b.c"], + hdrs = ["test_symbols.h"], + copts = ["-fPIC"], + deps = [":" + name + "_libC"], + ) + util.helper_target( + cc_shared_library, + name = name + "_cslB", + deps = [":" + name + "_libB"], + dynamic_deps = [":" + name + "_cslC"], + ) + util.helper_target( + cc_library, + name = name + "_libA", + srcs = ["test_lib_a.c"], + hdrs = ["test_symbols.h"], + copts = ["-fPIC"], + deps = [":" + name + "_libB", ":" + name + "_libC"], + ) + +# Test 1: CSL A -> CSL B -> CSL C (Dynamic deps) +def _test_csl_dynamic_deps_top(name): + _create_dynamic_deps_helpers(name) + util.helper_target( + cc_shared_library, + name = name + "_cslA", + deps = [":" + name + "_libA"], + dynamic_deps = [":" + name + "_cslB", ":" + name + "_cslC"], + ) + analysis_test( + name = name, + target = name + "_cslA", + impl = _csl_dynamic_deps_test_impl, + ) + +_tests.append(_test_csl_dynamic_deps_top) + +def _csl_dynamic_deps_test_impl(env, target): + env.expect.that_target(target).has_provider(CcSharedLibraryInfo) + csl_info = target[CcSharedLibraryInfo] + + # Derive labels + test_name = target.label.name[:-5] # remove "_cslA" + lib_a_label = target.label.same_package_label(test_name + "_libA") + lib_b_label = target.label.same_package_label(test_name + "_libB") + lib_c_label = target.label.same_package_label(test_name + "_libC") + + env.expect.that_collection([str(e) for e in csl_info.exports]).contains_exactly([str(lib_a_label)]) + static_libs = [str(lbl) for lbl in csl_info.link_once_static_libs] + env.expect.that_collection(static_libs).contains(str(lib_a_label)) + env.expect.that_collection(static_libs).contains_none_of([str(lib_b_label), str(lib_c_label)]) + +# Test 2: py_extension A -> CSL B -> CSL C (Dynamic deps) +def _test_pyext_dynamic_deps_cslB(name): + _create_dynamic_deps_helpers(name) + analysis_test( + name = name, + target = name + "_cslB", + impl = _cslB_deps_test_impl, + ) + +_tests.append(_test_pyext_dynamic_deps_cslB) + +def _test_pyext_dynamic_deps_cslC(name): + _create_dynamic_deps_helpers(name) + analysis_test( + name = name, + target = name + "_cslC", + impl = _cslC_deps_test_impl, + ) + +_tests.append(_test_pyext_dynamic_deps_cslC) + +def _cslC_deps_test_impl(env, target): + env.expect.that_target(target).has_provider(CcSharedLibraryInfo) + csl_info = target[CcSharedLibraryInfo] + + # Derive labels + test_name = target.label.name[:-5] # remove "_cslC" + lib_c_label = target.label.same_package_label(test_name + "_libC") + + env.expect.that_collection([str(e) for e in csl_info.exports]).contains_exactly([str(lib_c_label)]) + env.expect.that_collection([str(lbl) for lbl in csl_info.link_once_static_libs]).contains_exactly([str(lib_c_label)]) + +def _cslB_deps_test_impl(env, target): + env.expect.that_target(target).has_provider(CcSharedLibraryInfo) + csl_info = target[CcSharedLibraryInfo] + + # Derive labels + test_name = target.label.name[:-5] # remove "_cslB" + lib_b_label = target.label.same_package_label(test_name + "_libB") + lib_c_label = target.label.same_package_label(test_name + "_libC") + csl_c_label = target.label.same_package_label(test_name + "_cslC") + + env.expect.that_collection([str(e) for e in csl_info.exports]).contains_exactly([str(lib_b_label)]) + static_libs = [str(lbl) for lbl in csl_info.link_once_static_libs] + env.expect.that_collection(static_libs).contains(str(lib_b_label)) + env.expect.that_collection(static_libs).contains_none_of([str(lib_c_label)]) + + dynamic_deps = [str(d.linker_input.owner) for d in csl_info.dynamic_deps.to_list()] + env.expect.that_collection(dynamic_deps).contains(str(csl_c_label)) + +# For tests 3 and 4 +def _create_static_sharing_helpers(name): + util.helper_target( + cc_library, + name = name + "_libC", + srcs = ["test_lib_c.c"], + hdrs = ["test_symbols.h"], + copts = ["-fPIC"], + ) + util.helper_target( + cc_library, + name = name + "_libB", + srcs = ["test_lib_b.c"], + hdrs = ["test_symbols.h"], + copts = ["-fPIC"], + deps = [":" + name + "_libC"], + ) + util.helper_target( + cc_shared_library, + name = name + "_cslB", + deps = [":" + name + "_libB", ":" + name + "_libC"], + ) + util.helper_target( + cc_library, + name = name + "_libA", + srcs = ["test_lib_a.c"], + hdrs = ["test_symbols.h"], + copts = ["-fPIC"], + deps = [":" + name + "_libB", ":" + name + "_libC"], + ) + +# Test 3: CSL A -> CSL B, CL C (Static sharing) +def _test_csl_static_sharing_top(name): + _create_static_sharing_helpers(name) + util.helper_target( + cc_shared_library, + name = name + "_cslA", + deps = [":" + name + "_libA"], + dynamic_deps = [":" + name + "_cslB"], + ) + analysis_test( + name = name, + target = name + "_cslA", + impl = _csl_static_sharing_test_impl, + ) + +_tests.append(_test_csl_static_sharing_top) + +def _csl_static_sharing_test_impl(env, target): + env.expect.that_target(target).has_provider(CcSharedLibraryInfo) + csl_info = target[CcSharedLibraryInfo] + + # Derive labels + test_name = target.label.name[:-5] # remove "_cslA" + lib_a_label = target.label.same_package_label(test_name + "_libA") + lib_b_label = target.label.same_package_label(test_name + "_libB") + lib_c_label = target.label.same_package_label(test_name + "_libC") + + env.expect.that_collection([str(e) for e in csl_info.exports]).contains_exactly([str(lib_a_label)]) + static_libs = [str(lbl) for lbl in csl_info.link_once_static_libs] + env.expect.that_collection(static_libs).contains(str(lib_a_label)) + env.expect.that_collection(static_libs).contains_none_of([str(lib_b_label), str(lib_c_label)]) + +# Test 4: Same as 3, but A is py_extension + +def _test_pyext_static_sharing_cslB(name): + _create_static_sharing_helpers(name) + analysis_test( + name = name, + target = name + "_cslB", + impl = _cslB_static_sharing_test_impl, + ) + +_tests.append(_test_pyext_static_sharing_cslB) + +def _cslB_static_sharing_test_impl(env, target): + env.expect.that_target(target).has_provider(CcSharedLibraryInfo) + csl_info = target[CcSharedLibraryInfo] + + # Derive labels + test_name = target.label.name[:-5] # remove "_cslB" + lib_b_label = target.label.same_package_label(test_name + "_libB") + lib_c_label = target.label.same_package_label(test_name + "_libC") + + env.expect.that_collection([str(e) for e in csl_info.exports]).contains_exactly([str(lib_b_label), str(lib_c_label)]) + static_libs = [str(lbl) for lbl in csl_info.link_once_static_libs] + env.expect.that_collection(static_libs).contains_exactly([str(lib_b_label), str(lib_c_label)]) + +def dependency_graph_test_suite(name): + test_suite( + name = name, + tests = _tests, + ) diff --git a/tests/cc/py_extension/dependency_graph/test_lib_a.c b/tests/cc/py_extension/dependency_graph/test_lib_a.c new file mode 100644 index 0000000000..19e2f489bf --- /dev/null +++ b/tests/cc/py_extension/dependency_graph/test_lib_a.c @@ -0,0 +1,6 @@ +#include "test_symbols.h" + +void fnA() { + fnB(); + fnC(); +} diff --git a/tests/cc/py_extension/dependency_graph/test_lib_b.c b/tests/cc/py_extension/dependency_graph/test_lib_b.c new file mode 100644 index 0000000000..3621587c1c --- /dev/null +++ b/tests/cc/py_extension/dependency_graph/test_lib_b.c @@ -0,0 +1,5 @@ +#include "test_symbols.h" + +void fnB() { + fnC(); +} diff --git a/tests/cc/py_extension/dependency_graph/test_lib_c.c b/tests/cc/py_extension/dependency_graph/test_lib_c.c new file mode 100644 index 0000000000..99941576dd --- /dev/null +++ b/tests/cc/py_extension/dependency_graph/test_lib_c.c @@ -0,0 +1,6 @@ +#include "test_symbols.h" +#include + +void fnC() { + printf("fnC\n"); +} diff --git a/tests/cc/py_extension/dependency_graph/test_symbols.h b/tests/cc/py_extension/dependency_graph/test_symbols.h new file mode 100644 index 0000000000..59ab3b02e7 --- /dev/null +++ b/tests/cc/py_extension/dependency_graph/test_symbols.h @@ -0,0 +1,8 @@ +#ifndef TEST_SYMBOLS_H +#define TEST_SYMBOLS_H + +void fnC(); +void fnB(); +void fnA(); + +#endif // TEST_SYMBOLS_H diff --git a/tests/cc/py_extension/ext_init_in_dep.c b/tests/cc/py_extension/ext_init_in_dep.c new file mode 100644 index 0000000000..a6d32ba0b2 --- /dev/null +++ b/tests/cc/py_extension/ext_init_in_dep.c @@ -0,0 +1,20 @@ + +#include + +// No methods defined; we're just testing the init function. +static PyMethodDef ModuleMethods[] = { + {NULL, NULL, 0, NULL} /* Sentinel */ +}; + +static struct PyModuleDef ext_init_in_dep_module = { + PyModuleDef_HEAD_INIT, + "ext_init_in_dep", /* name of module */ + NULL, /* module documentation, may be NULL */ + -1, /* size of per-interpreter state of the module, + or -1 if the module keeps state in global variables. */ + ModuleMethods +}; + +PyMODINIT_FUNC PyInit_ext_init_in_dep(void) { + return PyModule_Create(&ext_init_in_dep_module); +} diff --git a/tests/cc/py_extension/ext_limited.c b/tests/cc/py_extension/ext_limited.c new file mode 100644 index 0000000000..f3622c5824 --- /dev/null +++ b/tests/cc/py_extension/ext_limited.c @@ -0,0 +1,22 @@ +#include + +static PyObject* get_limited_api_version(PyObject* self, PyObject* args) { + return PyUnicode_FromFormat("0x%08x", Py_LIMITED_API); +} + +static PyMethodDef ModuleMethods[] = { + {"get_limited_api_version", get_limited_api_version, METH_NOARGS, "Get the version of the limited API this extension was compiled against."}, + {NULL, NULL, 0, NULL} +}; + +static struct PyModuleDef ext_limited_module = { + PyModuleDef_HEAD_INIT, + "ext_limited", + NULL, + -1, + ModuleMethods +}; + +PyMODINIT_FUNC PyInit_ext_limited(void) { + return PyModule_Create(&ext_limited_module); +} diff --git a/tests/cc/py_extension/ext_pkg_test.c b/tests/cc/py_extension/ext_pkg_test.c new file mode 100644 index 0000000000..c151f0162b --- /dev/null +++ b/tests/cc/py_extension/ext_pkg_test.c @@ -0,0 +1,22 @@ +#include + +static PyObject* get_magic_number(PyObject* self, PyObject* args) { + return PyLong_FromLong(42); +} + +static PyMethodDef ModuleMethods[] = { + {"get_magic_number", get_magic_number, METH_NOARGS, "Returns 42."}, + {NULL, NULL, 0, NULL} +}; + +static struct PyModuleDef ext_pkg_test_module = { + PyModuleDef_HEAD_INIT, + "ext_pkg_test", + NULL, + -1, + ModuleMethods +}; + +PyMODINIT_FUNC PyInit_ext_pkg_test(void) { + return PyModule_Create(&ext_pkg_test_module); +} diff --git a/tests/cc/py_extension/ext_shared.c b/tests/cc/py_extension/ext_shared.c new file mode 100644 index 0000000000..4b79a6da2f --- /dev/null +++ b/tests/cc/py_extension/ext_shared.c @@ -0,0 +1,33 @@ +#include + +#include "tests/cc/py_extension/add_one.h" + +// A simple function that returns a Python integer. +static PyObject* do_alpha(PyObject* self, PyObject* args) { + return PyLong_FromLong(add_one(41)); +} + +// Method definition object for this extension, these are the functions +// that will be available in the module. +static PyMethodDef ModuleMethods[] = { + {"do_alpha", do_alpha, METH_NOARGS, "A simple C function."}, + {NULL, NULL, 0, NULL} /* Sentinel */ +}; + +// Module definition +// The arguments of this structure tell Python what to call your extension, +// what its methods are and where to look for its method definitions. +static struct PyModuleDef ext_shared_module = { + PyModuleDef_HEAD_INIT, + "ext_shared", /* name of module */ + NULL, /* module documentation, may be NULL */ + -1, /* size of per-interpreter state of the module, + or -1 if the module keeps state in global variables. */ + ModuleMethods +}; + +// The module init function. This must be exported and retained in the +// shared library output. +PyMODINIT_FUNC PyInit_ext_shared(void) { + return PyModule_Create(&ext_shared_module); +} diff --git a/tests/cc/py_extension/ext_source.c b/tests/cc/py_extension/ext_source.c new file mode 100644 index 0000000000..df0239df3a --- /dev/null +++ b/tests/cc/py_extension/ext_source.c @@ -0,0 +1,33 @@ + +#include + + +static PyObject* calc_one_plus_two(PyObject* self, PyObject* args) { + return PyLong_FromLong(1 + 2); +} + + +// Method definition object for this extension, these are the functions +// that will be available in the module. +static PyMethodDef ModuleMethods[] = { + {"calc_one_plus_two", calc_one_plus_two, METH_NOARGS, "A simple C function."}, + {NULL, NULL, 0, NULL} /* Sentinel */ +}; + +// Module definition +// The arguments of this structure tell Python what to call your extension, +// what its methods are and where to look for its method definitions. +static struct PyModuleDef ext_source_module = { + PyModuleDef_HEAD_INIT, + "ext_source", /* name of module */ + NULL, /* module documentation, may be NULL */ + -1, /* size of per-interpreter state of the module, + or -1 if the module keeps state in global variables. */ + ModuleMethods +}; + +// The module init function. This must be exported and retained in the +// shared library output. +PyMODINIT_FUNC PyInit_ext_source(void) { + return PyModule_Create(&ext_source_module); +} diff --git a/tests/cc/py_extension/ext_static.c b/tests/cc/py_extension/ext_static.c new file mode 100644 index 0000000000..500c52db00 --- /dev/null +++ b/tests/cc/py_extension/ext_static.c @@ -0,0 +1 @@ +/* A no-op C extension for static linking tests. */ diff --git a/tests/cc/py_extension/py_extension/BUILD.bazel b/tests/cc/py_extension/py_extension/BUILD.bazel new file mode 100644 index 0000000000..0d64c2bf9d --- /dev/null +++ b/tests/cc/py_extension/py_extension/BUILD.bazel @@ -0,0 +1,12 @@ +load(":py_extension_tests.bzl", "py_extension_analysis_test_suite") + +package( + default_testonly = True, + default_visibility = ["//visibility:private"], +) + +licenses(["notice"]) + +py_extension_analysis_test_suite( + name = "py_extension_analysis_tests", +) diff --git a/tests/cc/py_extension/py_extension/py_extension_tests.bzl b/tests/cc/py_extension/py_extension/py_extension_tests.bzl new file mode 100644 index 0000000000..ef2a2d8b6e --- /dev/null +++ b/tests/cc/py_extension/py_extension/py_extension_tests.bzl @@ -0,0 +1,102 @@ +# Copyright 2025 The Bazel Authors. All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +"""Tests for py_extension.""" + +load("@rules_testing//lib:analysis_test.bzl", "analysis_test", "test_suite") +load("@rules_testing//lib:truth.bzl", "matching") +load("//python/private:py_info.bzl", "PyInfo") # buildifier: disable=bzl-visibility + +_tests = [] + +def _test_static_deps_impl(env, target): + env.expect.that_target(target).has_provider(PyInfo) + py_info = target[PyInfo] + + # The extension should be in PyInfo + env.expect.that_collection(py_info.transitive_sources.to_list()).has_size(1) + env.expect.that_depset_of_files(py_info.transitive_sources).contains_predicate( + matching.file_path_matches("ext_static.*311-*"), + ) + +def _test_static_deps(name): + analysis_test( + name = name, + impl = _test_static_deps_impl, + target = "//tests/cc/py_extension:ext_static", + ) + +_tests.append(_test_static_deps) + +def _test_data_deps_impl(env, target): + env.expect.that_target(target).has_provider(PyInfo) + + # Check that data file is in runfiles + default_info = target[DefaultInfo] + env.expect.that_depset_of_files(default_info.default_runfiles.files).contains_predicate( + matching.file_basename_equals("some_data.txt"), + ) + +def _test_data_deps(name): + analysis_test( + name = name, + impl = _test_data_deps_impl, + target = "//tests/cc/py_extension:ext_with_data", + ) + +_tests.append(_test_data_deps) + +def _test_dynamic_deps_impl(env, target): + env.expect.that_target(target).has_provider(PyInfo) + py_info = target[PyInfo] + + # The extension should be in PyInfo + env.expect.that_collection(py_info.transitive_sources.to_list()).has_size(1) + env.expect.that_depset_of_files(py_info.transitive_sources).contains_predicate( + matching.file_path_matches("ext_shared.*311-*"), + ) + +def _test_dynamic_deps(name): + analysis_test( + name = name, + impl = _test_dynamic_deps_impl, + target = "//tests/cc/py_extension:ext_shared", + ) + +_tests.append(_test_dynamic_deps) + +def _test_musl_platform_impl(env, target): + env.expect.that_target(target).has_provider(PyInfo) + py_info = target[PyInfo] + env.expect.that_depset_of_files(py_info.transitive_sources).contains_predicate( + matching.file_path_matches("ext_static.*311-*"), + ) + +def _test_musl_platform(name): + analysis_test( + name = name, + impl = _test_musl_platform_impl, + target = "//tests/cc/py_extension:ext_static", + config_settings = { + str(Label("//python/config_settings:py_linux_libc")): "musl", + }, + ) + +_tests.append(_test_musl_platform) + +def py_extension_analysis_test_suite(name): + test_suite( + name = name, + tests = _tests, + ) diff --git a/tests/cc/py_extension/py_extension_build_test.bzl b/tests/cc/py_extension/py_extension_build_test.bzl new file mode 100644 index 0000000000..18cca2204b --- /dev/null +++ b/tests/cc/py_extension/py_extension_build_test.bzl @@ -0,0 +1,15 @@ +"""Macro helper for py_extension build testing.""" + +load("@bazel_skylib//rules:build_test.bzl", "build_test") +load("//python/cc:py_extension.bzl", "py_extension") + +def py_extension_build_test(name, **kwargs): + """Creates a py_extension target and a build_test verifying it builds.""" + py_extension( + name = name, + **kwargs + ) + build_test( + name = name + "_build_test", + targets = [":" + name], + ) diff --git a/tests/cc/py_extension/py_extension_pkg_test.py b/tests/cc/py_extension/py_extension_pkg_test.py new file mode 100644 index 0000000000..e3176d6a6c --- /dev/null +++ b/tests/cc/py_extension/py_extension_pkg_test.py @@ -0,0 +1,16 @@ +import unittest + +from tests.cc.py_extension import ext_pkg_test + + +class PyExtensionPkgTest(unittest.TestCase): + def test_import_via_package(self): + self.assertEqual(ext_pkg_test.get_magic_number(), 42) + + def test_direct_import(self): + with self.assertRaises(ModuleNotFoundError): + import ext_pkg_test # buildifier: disable=g-import-not-at-top # noqa: F401 + + +if __name__ == "__main__": + unittest.main() diff --git a/tests/cc/py_extension/py_extension_test.py b/tests/cc/py_extension/py_extension_test.py new file mode 100644 index 0000000000..d82fe22bcc --- /dev/null +++ b/tests/cc/py_extension/py_extension_test.py @@ -0,0 +1,47 @@ +import os +import sys +import unittest + +import ext_shared +from elftools.elf.dynamic import DynamicSection +from elftools.elf.elffile import ELFFile + + +class PyExtensionTest(unittest.TestCase): + @unittest.skipIf( + sys.platform != "linux", "ELF inspection is only supported on Linux" + ) + def test_inspect_elf(self): + ext_path = ext_shared.__file__ + self.assertTrue( + os.path.exists(ext_path), f"Could not find ext_shared.so at {ext_path}" + ) + + with open(ext_path, "rb") as f: + elf = ELFFile(f) + + # Check for DT_NEEDED entry for the dynamic library + dynamic_section = elf.get_section_by_name(".dynamic") + self.assertIsNotNone(dynamic_section) + self.assertTrue(isinstance(dynamic_section, DynamicSection)) + + needed_libs = [ + tag.needed + for tag in dynamic_section.iter_tags() + if tag.entry.d_tag == "DT_NEEDED" + ] + self.assertIn("libadd_one_shared.so", needed_libs) + + # Check for the PyInit symbol + dynsym_section = elf.get_section_by_name(".dynsym") + self.assertIsNotNone(dynsym_section) + + symbols = [s.name for s in dynsym_section.iter_symbols()] + self.assertIn("PyInit_ext_shared", symbols) + + def test_import_and_call(self): + self.assertEqual(ext_shared.do_alpha(), 43) + + +if __name__ == "__main__": + unittest.main() diff --git a/tests/cc/py_extension/py_limited_api/BUILD.bazel b/tests/cc/py_extension/py_limited_api/BUILD.bazel new file mode 100644 index 0000000000..f548f3040f --- /dev/null +++ b/tests/cc/py_extension/py_limited_api/BUILD.bazel @@ -0,0 +1,12 @@ +load(":py_limited_api_tests.bzl", "py_limited_api_test_suite") + +package( + default_testonly = True, + default_visibility = ["//visibility:private"], +) + +licenses(["notice"]) + +py_limited_api_test_suite( + name = "py_limited_api_tests", +) diff --git a/tests/cc/py_extension/py_limited_api/py_limited_api_tests.bzl b/tests/cc/py_extension/py_limited_api/py_limited_api_tests.bzl new file mode 100644 index 0000000000..fbe92b936a --- /dev/null +++ b/tests/cc/py_extension/py_limited_api/py_limited_api_tests.bzl @@ -0,0 +1,140 @@ +# Copyright 2025 The Bazel Authors. All rights reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +"""Tests for the py_limited_api attribute for py_extension.""" + +load("@rules_cc//cc:cc_library.bzl", "cc_library") +load("@rules_testing//lib:analysis_test.bzl", "analysis_test", "test_suite") +load("@rules_testing//lib:truth.bzl", "matching") +load("@rules_testing//lib:util.bzl", "util") +load("//python/cc:py_extension.bzl", "py_extension") + +def _test_limited_pass_impl(env, target): + env.expect.that_depset_of_files(target[DefaultInfo].files).contains_predicate( + matching.file_path_matches("tests/cc/py_extension/py_limited_api/{}.abi3.*".format(target.label.name)), + ) + +def _test_limited_same_version(name): + util.helper_target( + cc_library, + name = name + "_csl", + defines = ["Py_LIMITED_API=0x03080000"], + deps = [ + "@rules_python//python/cc:current_py_cc_headers", + ], + ) + py_extension( + name = name + "_pyext", + deps = [":" + name + "_csl"], + py_limited_api = "3.8", + ) + analysis_test( + name = name, + target = name + "_pyext", + impl = _test_limited_pass_impl, + ) + +def _test_limited_older_dep(name): + util.helper_target( + cc_library, + name = name + "_csl", + defines = ["Py_LIMITED_API=0x03080000"], # 3.8 + deps = [ + "@rules_python//python/cc:current_py_cc_headers", + ], + ) + py_extension( + name = name + "_pyext", + deps = [":" + name + "_csl"], + py_limited_api = "3.9", # 3.9 + ) + analysis_test( + name = name, + target = name + "_pyext", + impl = _test_limited_pass_impl, + ) + +def _test_no_limited_api(name): + util.helper_target( + cc_library, + name = name + "_csl", + deps = [ + "@rules_python//python/cc:current_py_cc_headers", + ], + ) + py_extension( + name = name + "_pyext", + deps = [":" + name + "_csl"], + ) + analysis_test( + name = name, + target = name + "_pyext", + impl = _test_no_limited_api_impl, + ) + +def _test_no_limited_api_impl(env, target): + # Should pass, nothing to assert on filename since it is platform-specific + _ = env # @unused + _ = target # @unused + +def _test_no_limited_api_dep_has_limited(name): + util.helper_target( + cc_library, + name = name + "_csl", + defines = ["Py_LIMITED_API=0x03080000"], + deps = [ + "@rules_python//python/cc:current_py_cc_headers", + ], + ) + py_extension( + name = name + "_pyext", + deps = [":" + name + "_csl"], + ) + analysis_test( + name = name, + target = name + "_pyext", + impl = _test_no_limited_api_dep_has_limited_impl, + ) + +def _test_no_limited_api_dep_has_limited_impl(env, target): + _ = env # @unused + _ = target # @unused + +def _test_limited_api_dep_has_no_python(name): + util.helper_target( + cc_library, + name = name + "_csl", + ) + py_extension( + name = name + "_pyext", + deps = [":" + name + "_csl"], + py_limited_api = "3.8", + ) + analysis_test( + name = name, + target = name + "_pyext", + impl = _test_limited_pass_impl, + ) + +def py_limited_api_test_suite(name): + test_suite( + name = name, + tests = [ + _test_limited_same_version, + _test_limited_older_dep, + _test_no_limited_api, + _test_no_limited_api_dep_has_limited, + _test_limited_api_dep_has_no_python, + ], + ) diff --git a/tests/cc/py_extension/some_data.txt b/tests/cc/py_extension/some_data.txt new file mode 100644 index 0000000000..4b5dc1d64c --- /dev/null +++ b/tests/cc/py_extension/some_data.txt @@ -0,0 +1 @@ +This is a data file diff --git a/tests/cc/py_extension/static_dep.c b/tests/cc/py_extension/static_dep.c new file mode 100644 index 0000000000..fa95e4dacc --- /dev/null +++ b/tests/cc/py_extension/static_dep.c @@ -0,0 +1,5 @@ +#include "static_dep.h" + +int my_lib_func() { + return 42; +} diff --git a/tests/cc/py_extension/static_dep.h b/tests/cc/py_extension/static_dep.h new file mode 100644 index 0000000000..d0f272abd7 --- /dev/null +++ b/tests/cc/py_extension/static_dep.h @@ -0,0 +1 @@ +int my_lib_func(); diff --git a/tests/integration/bzlmod_lockfile/MODULE.bazel.lock b/tests/integration/bzlmod_lockfile/MODULE.bazel.lock index 686ee0ee40..6b293283ae 100644 --- a/tests/integration/bzlmod_lockfile/MODULE.bazel.lock +++ b/tests/integration/bzlmod_lockfile/MODULE.bazel.lock @@ -252,7 +252,7 @@ }, "@@rules_python+//python/uv:uv.bzl%uv": { "general": { - "bzlTransitiveDigest": "OC4ZhWl8jX9wvFicn83AioC9J7Dx6Us2/+EzoGDlPzU=", + "bzlTransitiveDigest": "DafUArm1CsjJZcemT0FvUq9rUch+pq38+JqfOBJCpm8=", "usagesDigest": "6yXGw7XDyXjOfqBL0SBu1YBEMMYPQzCE3jTzUCkxPgg=", "recordedInputs": [ "REPO_MAPPING:rules_python+,bazel_tools bazel_tools", diff --git a/tests/support/cc_toolchains/BUILD.bazel b/tests/support/cc_toolchains/BUILD.bazel index 1c1a714626..a74a8016fc 100644 --- a/tests/support/cc_toolchains/BUILD.bazel +++ b/tests/support/cc_toolchains/BUILD.bazel @@ -57,7 +57,9 @@ py_cc_toolchain( headers = ":py_headers", headers_abi3 = ":py_headers_abi3", libs = ":fake_libs", + platform_machine = "x86_64", python_version = "3.999", + sys_platform = "linux", tags = PREVENT_IMPLICIT_BUILDING_TAGS, ) diff --git a/tests/support/py_cc_toolchain_info_subject.bzl b/tests/support/py_cc_toolchain_info_subject.bzl index 3820e04e90..fbf5a1a0ad 100644 --- a/tests/support/py_cc_toolchain_info_subject.bzl +++ b/tests/support/py_cc_toolchain_info_subject.bzl @@ -18,15 +18,25 @@ load("@rules_testing//lib:truth.bzl", "subjects") def _py_cc_toolchain_info_subject_new(info, *, meta): # buildifier: disable=uninitialized public = struct( + abi_flags = lambda *a, **k: _py_cc_toolchain_info_subject_abi_flags(self, *a, **k), headers = lambda *a, **k: _py_cc_toolchain_info_subject_headers(self, *a, **k), headers_abi3 = lambda *a, **k: _py_cc_toolchain_info_subject_headers_abi3(self, *a, **k), libs = lambda *a, **k: _py_cc_toolchain_info_subject_libs(self, *a, **k), + platform_machine = lambda *a, **k: _py_cc_toolchain_info_subject_platform_machine(self, *a, **k), + platform_tag = lambda *a, **k: _py_cc_toolchain_info_subject_platform_tag(self, *a, **k), python_version = lambda *a, **k: _py_cc_toolchain_info_subject_python_version(self, *a, **k), + sys_platform = lambda *a, **k: _py_cc_toolchain_info_subject_sys_platform(self, *a, **k), actual = info, ) self = struct(actual = info, meta = meta) return public +def _py_cc_toolchain_info_subject_abi_flags(self): + return subjects.str( + self.actual.abi_flags, + meta = self.meta.derive("abi_flags()"), + ) + def _py_cc_toolchain_info_subject_headers(self): return subjects.struct( self.actual.headers, @@ -54,12 +64,30 @@ def _py_cc_toolchain_info_subject_libs(self): ), ) +def _py_cc_toolchain_info_subject_platform_machine(self): + return subjects.str( + self.actual.platform_machine, + meta = self.meta.derive("platform_machine()"), + ) + +def _py_cc_toolchain_info_subject_platform_tag(self): + return subjects.str( + self.actual.platform_tag, + meta = self.meta.derive("platform_tag()"), + ) + def _py_cc_toolchain_info_subject_python_version(self): return subjects.str( self.actual.python_version, meta = self.meta.derive("python_version()"), ) +def _py_cc_toolchain_info_subject_sys_platform(self): + return subjects.str( + self.actual.sys_platform, + meta = self.meta.derive("sys_platform()"), + ) + # Disable this to aid doc generation # buildifier: disable=name-conventions PyCcToolchainInfoSubject = struct( From 2b482dad110470a28dc3e407d6165fba8257c92e Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Sat, 1 Aug 2026 11:52:14 -0700 Subject: [PATCH 885/922] agents: add technical rules for Starlark, Windows, C++, and testing (#3986) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit To support upcoming C/C++ Python extension features (`py_extension`), autonomous agents require explicit domain-specific guardrails and invariants. Without these rules, agents frequently encounter recurring pitfalls—such as target analysis errors from duplicate dependency labels, invalid use of `--config=fast-tests` on documentation targets, non-canonical macro target strings, or breaking review threads by amending PR commits. This change modularizes agent guidance into dedicated rule files under `.agents/rules/` and updates the `create-pr` skill to enforce PR and news entry conventions across autonomous workflows. ### Detailed Breakdown of Rule Additions * `.agents/rules/cc.md`: Documents C/C++ target rules and Python extension module compilation and linking guardrails. * `.agents/rules/news.md`: Establishes news entry conventions, requiring Sphinx MyST cross-reference syntax (`{obj}`) for code symbols and markdown issue links. * `.agents/rules/pr.md`: Enforces PR description conventions, mandatory `@CONTRIBUTING.md` inclusion, and the commit invariant (never amend or rebase active PR commits). Replaces `.agents/rules/pr_and_news.md`. * `.agents/rules/python.md`: Defines pytest fixture registration rules via `pytest_plugins` and `fixture_` function naming conventions. * `.agents/rules/starlark.md`: Specifies Starlark macro target canonicalization using `str(Label(...))`, private alias patterns for internal helper `deps`, input attribute normalization at function entry, iterative algorithm loop restrictions, 80-column line wrapping, and `@rules_testing` test structures. * `.agents/rules/testing.md`: Enforces `--config=fast-tests` usage guardrails (never use on non-test targets like `//docs:docs`), lockfile updates, and retry behavior for Sphinx documentation build flakes. * `.agents/rules/windows.md`: Documents Windows MSVC toolchain embedded `#pragma comment(lib, ...)` library linking, PEP 3149 ABI tag differences (`cp` vs `cpython-`), and platform tags. * `.agents/skills/create-pr/SKILL.md`: Updates frontmatter to support propose and draft PR workflows, and instructs subagents to include `@/.agents/rules/pr.md`. --- .agents/rules/cc.md | 22 ++++++++++++++ .agents/rules/news.md | 20 +++++++++++++ .agents/rules/pr.md | 20 +++++++++++++ .agents/rules/pr_and_news.md | 12 -------- .agents/rules/python.md | 8 ++++++ .agents/rules/starlark.md | 48 +++++++++++++++++++++++++++++++ .agents/rules/testing.md | 19 ++++++++++++ .agents/rules/windows.md | 21 ++++++++++++++ .agents/skills/create-pr/SKILL.md | 8 ++++-- 9 files changed, 163 insertions(+), 15 deletions(-) create mode 100644 .agents/rules/cc.md create mode 100644 .agents/rules/news.md create mode 100644 .agents/rules/pr.md delete mode 100644 .agents/rules/pr_and_news.md create mode 100644 .agents/rules/python.md create mode 100644 .agents/rules/starlark.md create mode 100644 .agents/rules/testing.md create mode 100644 .agents/rules/windows.md diff --git a/.agents/rules/cc.md b/.agents/rules/cc.md new file mode 100644 index 0000000000..5f2c2b43c2 --- /dev/null +++ b/.agents/rules/cc.md @@ -0,0 +1,22 @@ +# C/C++ Rules Design & Bazel rules_cc Integration + +## Concise `cc_shared_library` Interface & Export Rules +* In `cc_shared_library`: + * Targets directly placed in `deps` are **automatically exported** by Bazel; + they do not need to be listed in `exports_filter`. + * `exports_filter` is defined in `rules_cc` as an `attr.string_list()`, **not** + a `Label` attribute. Passing Starlark `Label` objects directly causes an + immediate Bazel analysis type error. + * **Citation**: [rules_cc `cc_shared_library.bzl`](https://github.com/bazelbuild/rules_cc/blob/main/cc/private/rules_impl/cc_shared_library.bzl) + (*"exports_filter is a list of strings attribute"*). + +## macOS Dynamic Lookup Link Flag +* Apple's `ld64`/`lld` linkers require `-undefined dynamic_lookup` in + `user_link_flags` on macOS so CPython C-API symbols remain unresolved at + link time and resolve dynamically at runtime. + +## Linux Platform Tag Composition +* Linux platform tags are formatted as `{platform_machine}-linux-{libc}` (e.g., + `x86_64-linux-gnu` or `aarch64-linux-musl`). +* **Citation**: [PEP 600 — Perennial manylinux](https://peps.python.org/pep-0600/) + & [PEP 656 — musllinux](https://peps.python.org/pep-0656/). diff --git a/.agents/rules/news.md b/.agents/rules/news.md new file mode 100644 index 0000000000..ce8a844439 --- /dev/null +++ b/.agents/rules/news.md @@ -0,0 +1,20 @@ +--- +trigger: news/*.md +description: Apply when drafting news entries. +--- + +@CONTRIBUTING.md + +# News Entry Conventions + +Before drafting any news entry, strictly adhere to the rules in `CONTRIBUTING.md` +above. + +## Sphinx MyST Cross-Reference Syntax (`{obj}`) +* Use `{obj}\`\`` in news entries for rules, macros, targets, providers, + attributes, args, and any other cross-referencable Starlark or Python + objects. + +## GitHub Issue Link Formatting +* Append GitHub issue cross-references at the end of news entries in markdown + link format: `([#3283](https://github.com/bazel-contrib/rules_python/issues/3283))`. diff --git a/.agents/rules/pr.md b/.agents/rules/pr.md new file mode 100644 index 0000000000..ba8236d072 --- /dev/null +++ b/.agents/rules/pr.md @@ -0,0 +1,20 @@ +--- +trigger: model_decision +description: Apply when drafting pull request descriptions. +--- + +@CONTRIBUTING.md + +# Pull Request Conventions + +Before drafting any pull request description, strictly adhere to the rules in +`CONTRIBUTING.md` above. + +## Prevent Agent Oversight of `CONTRIBUTING.md` +* Always include `@CONTRIBUTING.md` in rule definitions so that PR formatting + rules are injected into context whenever PR workflows run. + +## PR Commit Workflow Invariant +* Once a Pull Request is created, always make new commits or merge commits. +* **NEVER** amend or rebase commits on an active PR branch to avoid breaking + code review threads. diff --git a/.agents/rules/pr_and_news.md b/.agents/rules/pr_and_news.md deleted file mode 100644 index 0eb7726efc..0000000000 --- a/.agents/rules/pr_and_news.md +++ /dev/null @@ -1,12 +0,0 @@ ---- -trigger: model_decision -description: Apply when drafting pull request descriptions or news entries. ---- - -# PR Descriptions and News Entries - -Before drafting any pull request description or news entry, you MUST read -`CONTRIBUTING.md` (specifically the sections on **Commit messages and PR -descriptions** and **Documenting changes**) and strictly adhere to its style, -formatting, and structure rules. Do not generate PR descriptions or news -entries without consulting `CONTRIBUTING.md` first. diff --git a/.agents/rules/python.md b/.agents/rules/python.md new file mode 100644 index 0000000000..523cc923cd --- /dev/null +++ b/.agents/rules/python.md @@ -0,0 +1,8 @@ +# Python Conventions + +## pytest +* **Fixture Registration via `pytest_plugins`**: When registering pytest helper + modules in test files, use `pytest_plugins = [""]`. +* **Fixture Naming Conventions**: Name fixture functions with a `fixture_` prefix + (e.g. `def fixture_foo():`), and pass the public fixture name using the `name` + parameter in `@pytest.fixture(name="foo")`. diff --git a/.agents/rules/starlark.md b/.agents/rules/starlark.md new file mode 100644 index 0000000000..430284bcd5 --- /dev/null +++ b/.agents/rules/starlark.md @@ -0,0 +1,48 @@ +# Starlark Language & Macro Invariants + +## Macro Target Canonicalization +* In macro implementations, internal repository target references **MUST** be + canonicalized using `str(Label("//path/to:target"))` so they resolve in the + macro's module context rather than the caller's repository context. +* Note that `python/private/common_labels.bzl` defines `labels`, a struct + containing common canonicalized label strings used across the project. + +## Private Alias Pattern when Appending to User `deps` +* When macros append internal helper targets to user-provided dependency lists + (`deps`), use private alias targets (e.g. + `//python/private/cc:current_py_cc_headers_private_alias`) to prevent + "duplicate dependency label" analysis errors if the user also explicitly + passes the public target label in their `deps` (including in `select()` + expressions). + +## Attribute Normalization at Function Entry +* Always normalize input list attributes at the start of macro definitions + (e.g., `deps = deps or []`, `copts = copts or []`). +* **Why**: This avoids inlining the normalization as part of a complex + expression later in the macro expansion. + +## Control Flow & Algorithmic Restrictions +* **Iterative Algorithms Only (No Recursion)**: Starlark does not support + recursive function calls; always implement iterative algorithms using bounded + loops. +* **Iterable `for` Loops Only (No `while` Loops)**: Starlark does not support + `while` loops; iterate over fixed-size ranges or explicit collections. + +## Code Style & Conventions +* **Docstring Formatting Invariants**: Use triple-quoted strings for multi-line + docstrings without trailing backslashes (`\`) for line continuation. +* **No Bazel Copyright Headers**: Do not add Bazel copyright headers to new or + existing files unless explicitly directed by the user. +* **Line Length & Wrapping**: Wrap Markdown and Starlark lines to 80 columns in + accordance with `.editorconfig`. + +## Starlark Testing (`rules_testing`) +* **`rules_testing` over `bazel_skylib`**: Always use `@rules_testing` (analysis + tests with `env.expect.that_...`) rather than `bazel_skylib` for Starlark + rule analysis tests. +* **Analysis Test Two-Part Structure**: Separate tests into a setup target + function `def _test_foo(name)` calling `analysis_test` and an implementation + function `def _test_foo_impl(env, target)`. +* **Test Suite Registration**: Collect test setup functions in a private + `_tests` list and register them cleanly via `test_suite(name = name, tests = + _tests)`. diff --git a/.agents/rules/testing.md b/.agents/rules/testing.md new file mode 100644 index 0000000000..0f369b3861 --- /dev/null +++ b/.agents/rules/testing.md @@ -0,0 +1,19 @@ +# Testing & Validation Guardrails + +## `--config=fast-tests` for Test Targets +* Always pass `--config=fast-tests` when running or building test targets to + avoid running expensive, flaky integration tests. + +## CRITICAL: Never use `--config=fast-tests` on Non-Test Targets +* `--config=fast-tests` sets `--build_tests_only=true`, which silently ignores + non-test targets (such as `//docs:docs` or package libraries), resulting in 0 + targets built! + +## Lockfile Testing (`MODULE.bazel.lock`) +* Changes to transitive module extension dependencies or `.bzl` files loaded by + extensions update Bazel 9 lockfile hashes, requiring `bazel mod deps + --lockfile_mode=update` in integration test workspaces. + +## Documentation Flake Handling +* When building `//docs:docs` fails with exit code 2, treat it as a known + Sphinx/Bazel flake and retry the build. diff --git a/.agents/rules/windows.md b/.agents/rules/windows.md new file mode 100644 index 0000000000..5948092902 --- /dev/null +++ b/.agents/rules/windows.md @@ -0,0 +1,21 @@ +# Windows MSVC Toolchains & Linking + +## MSVC `#pragma comment(lib, ...)` Embedded References +* CPython's MSVC headers embed `#pragma comment(lib, "python3xx.lib")` into + generated `.obj` files. MSVC `link.exe` automatically searches for + `python3xx.lib` in library search paths during linking. +* **Citation**: [Microsoft Learn `comment` pragma](https://learn.microsoft.com/en-us/cpp/preprocessor/comment-c-cpp) + (*"Places a library-search record in the object file... The linker searches for + this library the same way as if you had named it on the command line"*). + +## PEP 3149 ABI Tag Prefix (`cp` vs. `cpython-`) +* Windows CPython ABI tags use the `cp` prefix (e.g., `cp311`, `cp312t`) + whereas POSIX platforms use `cpython-` (e.g., `cpython-311`). +* **Citation**: [PEP 3149 — ABI version tagged .so files](https://peps.python.org/pep-3149/) + (*"The tag starts with `cpython-` followed by the Python major and minor version + without dots... followed by build flags"*). + +## Windows Platform Tag Derivation +* Windows platform tags for Python C extensions evaluate to `win_amd64` + (x86_64/amd64), `win_arm64` (aarch64/arm64), or `win32` (32-bit x86). +* **Citation**: [PEP 425 — Compatibility Tags for Built Distributions](https://peps.python.org/pep-0425/). diff --git a/.agents/skills/create-pr/SKILL.md b/.agents/skills/create-pr/SKILL.md index 822ae1bdf8..9c2f454000 100644 --- a/.agents/skills/create-pr/SKILL.md +++ b/.agents/skills/create-pr/SKILL.md @@ -1,6 +1,7 @@ --- name: create-pr -description: Create a pull request by delegating to a subagent +description: Propose, draft, or create a pull request by delegating to a + subagent --- When creating a Pull Request for local changes or a branch, invoke a subagent @@ -11,8 +12,9 @@ to handle PR creation or description drafting. 1. Launch a subagent using `invoke_subagent` with `TypeName: "self"` (or `agentapi new-conversation`). 2. Provide a prompt to the subagent directing it to: - - Read `CONTRIBUTING.md` (specifically the sections on **Commit messages - and PR descriptions** and **Documenting changes**) before drafting. + - Include `@/.agents/rules/pr.md` and read `CONTRIBUTING.md` (specifically + the sections on **Commit messages and PR descriptions** and + **Documenting changes**) before drafting. - Strictly adhere to `CONTRIBUTING.md` rules for: - **PR Title**: Follow conventional commit style and title formatting. For agent rules, skills, and system updates, use `agents:` prefix. From 3c9423a076b6fb1aeade213d77aeaf6020da3b29 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Sat, 1 Aug 2026 14:06:03 -0700 Subject: [PATCH 886/922] docs: convert raw URLs in generated docs and docstrings to hyperlinks (#3987) Plain text URLs in documentation Markdown files and Starlark/Python docstrings were rendered as non-clickable text in the generated HTML output, making navigation cumbersome. To fix this, raw URLs are converted into MyST autolinks (), Markdown links, and {gh-issue} roles. A custom {pep} Sphinx role is also introduced in docs/conf.py to dynamically resolve PEP numbers to https://peps.python.org/pep-XXXX/, and existing PEP references in docstrings are updated to use {pep}. --- CHANGELOG.md | 12 ++++++------ docs/conf.py | 13 +++++++++++++ docs/extending.md | 2 +- docs/toolchains.md | 2 +- gazelle/docs/directives.md | 2 +- python/packaging.bzl | 10 +++++----- python/private/attributes.bzl | 6 +++--- python/private/py_cc_toolchain_info.bzl | 4 ++-- python/private/py_cc_toolchain_rule.bzl | 2 +- python/private/py_exec_tools_info.bzl | 2 +- python/private/py_executable.bzl | 4 ++-- python/private/py_runtime_rule.bzl | 2 +- python/private/py_wheel.bzl | 8 ++++---- python/private/pypi/BUILD.bazel | 12 ++++++------ python/private/pypi/extension.bzl | 6 +++--- python/private/pypi/pip_compile.bzl | 4 ++-- python/private/pypi/pip_repository.bzl | 4 ++-- python/private/zipapp/py_zipapp_rule.bzl | 2 +- python/runfiles/runfiles.py | 2 +- 19 files changed, 56 insertions(+), 43 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 17f176e5b2..f6dc36ee73 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -721,7 +721,7 @@ Other changes: * (core) `#!/usr/bin/env bash` is now used as a shebang in the stage1 bootstrap template. * (gazelle:docs) The Gazelle docs have been migrated from {gh-path}`gazelle/README.md` to {gh-path}`gazelle/docs` and are now available on the primary documentation site - at https://rules-python.readthedocs.io/en/latest/gazelle/docs/index.html + at [20250808]: https://github.com/astral-sh/python-build-standalone/releases/tag/20250808 @@ -1133,7 +1133,7 @@ Other changes: {#v1-2-0-changed} ### Changed * (rules) `py_proto_library` is deprecated in favour of the - implementation in https://github.com/protocolbuffers/protobuf. It will be + implementation in . It will be removed in the future release. * (pypi) {obj}`pip.override` will now be ignored instead of raising an error, fixes [#2550](https://github.com/bazel-contrib/rules_python/issues/2550). @@ -2117,7 +2117,7 @@ Other changes: `common --@rules_python//python/config_settings:python_version=X.Y.Z`. * New Python versions available: `3.11.7`, `3.12.1` using - https://github.com/indygreg/python-build-standalone/releases/tag/20240107. + . * (toolchain) Allow setting `x.y` as the `python_version` parameter in the version-aware `py_binary` and `py_test` rules. This allows users to @@ -2269,7 +2269,7 @@ Other changes: * (docs) bzlmod extensions are now documented on rules-python.readthedocs.io * (docs) Support and backwards compatibility policies have been documented. - See https://rules-python.readthedocs.io/en/latest/support.html + See * (gazelle) `file` generation mode can now also add `__init__.py` to the srcs attribute for every target in the package. This is enabled through a separate directive `python_generation_mode_per_file_include_init`. @@ -2401,7 +2401,7 @@ Breaking changes: the `py_binary` rule used to build it. * New Python versions available: `3.8.17`, `3.11.5` using - https://github.com/indygreg/python-build-standalone/releases/tag/20230826. + . * (gazelle) New `# gazelle:python_generation_mode file` directive to support generating one `py_library` per file. @@ -2421,7 +2421,7 @@ Breaking changes: time being. * New Python versions available: `3.8.18`, `3.9.18`, `3.10.13`, `3.11.6`, `3.12.0` using - https://github.com/indygreg/python-build-standalone/releases/tag/20231002. + . `3.12.0` support is considered beta and may have issues. ### Removed diff --git a/docs/conf.py b/docs/conf.py index 17b3c17106..750e344285 100644 --- a/docs/conf.py +++ b/docs/conf.py @@ -1,6 +1,9 @@ # Configuration file for the Sphinx documentation builder. import os +import re + +from docutils import nodes # -- Project information project = "rules_python" @@ -219,9 +222,19 @@ ] +def _pep_role(name, rawtext, text, lineno, inliner, options={}, content=[]): + match = re.search(r"\d+", text) + pep_num = match.group(0) if match else text + pep_url = f"https://peps.python.org/pep-{pep_num.zfill(4)}/" + display_text = text if text.startswith("PEP") else f"PEP {pep_num}" + node = nodes.reference(rawtext, display_text, refuri=pep_url, **options) + return [node], [] + + def setup(app): # Pygments says it supports starlark, but it doesn't seem to actually # recognize `starlark` as a name. So just manually map it to python. from sphinx.highlighting import lexer_classes app.add_lexer("starlark", lexer_classes["python"]) + app.add_role("pep", _pep_role) diff --git a/docs/extending.md b/docs/extending.md index 00018fbd74..1c91b01795 100644 --- a/docs/extending.md +++ b/docs/extending.md @@ -23,7 +23,7 @@ Extending the core rules is most useful when you want all or most of the behavior of a core rule. ::: -Follow or comment on https://github.com/bazel-contrib/rules_python/issues/1647 +Follow or comment on {gh-issue}`1647` for the development of APIs to support custom derived rules. ## Creating custom rules diff --git a/docs/toolchains.md b/docs/toolchains.md index 98534aee63..f154fc29ed 100644 --- a/docs/toolchains.md +++ b/docs/toolchains.md @@ -473,7 +473,7 @@ pip_parse( ``` After registration, your Python targets will use the toolchain's interpreter during execution, but a system-installed interpreter -is still used to "bootstrap" Python targets (see https://github.com/bazel-contrib/rules_python/issues/691). +is still used to "bootstrap" Python targets (see {gh-issue}`691`). You may also find some quirks while using this toolchain. Please refer to [python-build-standalone documentation's _Quirks_ section](https://gregoryszorc.com/docs/python-build-standalone/main/quirks.html). ## Local toolchain diff --git a/gazelle/docs/directives.md b/gazelle/docs/directives.md index ce8c03b9cd..60f948e4af 100644 --- a/gazelle/docs/directives.md +++ b/gazelle/docs/directives.md @@ -837,7 +837,7 @@ Version 1.9.0 includes a fix ({gh-pr}`3498`) for a long-standing issue ({gh-issue}`3497`) where ancestor `conftest.py` files were not automatically added as dependencies of {bzl:obj}`py_test` targets. -However, some people may not want this behavior (see https://xkcd.com/1172/). +However, some people may not want this behavior (see ). Thus the `python_include_ancestor_conftest` directive controls this behavior. It defaults to `true`, which causes all ancestor `conftest.py` files to be included as dependencies for {bzl:obj}`py_test` targets. diff --git a/python/packaging.bzl b/python/packaging.bzl index 537aa6090f..fb333ca72f 100644 --- a/python/packaging.bzl +++ b/python/packaging.bzl @@ -63,7 +63,7 @@ py_wheel_dist = rule( doc = """\ Prepare a dist/ folder, following Python's packaging standard practice. -See https://packaging.python.org/en/latest/tutorials/packaging-projects/#generating-distribution-archives +See which recommends a dist/ folder containing the wheel file(s), source distributions, etc. This also has the advantage that stamping information is included in the wheel's filename. @@ -94,7 +94,7 @@ def py_wheel( **kwargs): """Builds a Python Wheel. - Wheels are Python distribution format defined in https://www.python.org/dev/peps/pep-0427/. + Wheels are Python distribution format defined in {pep}`PEP 427`. This macro packages a set of targets into a single wheel. It wraps the [py_wheel rule](#py_wheel_rule). @@ -144,7 +144,7 @@ def py_wheel( To publish the wheel to PyPI, the twine package is required and it is installed by default on `bzlmod` setups. On legacy `WORKSPACE`, `rules_python` doesn't provide `twine` itself - (see https://github.com/bazel-contrib/rules_python/issues/1016), but + (see {gh-issue}`1016`), but you can install it with `pip_parse`, just like we do any other dependencies. Once you've installed twine, you can pass its label to the `twine` @@ -160,7 +160,7 @@ def py_wheel( ) ``` - Now you can run a command like the following, which publishes to https://test.pypi.org/ + Now you can run a command like the following, which publishes to ```sh % TWINE_USERNAME=__token__ TWINE_PASSWORD=pypi-*** \\ @@ -172,7 +172,7 @@ def py_wheel( name: A unique name for this target. twine: A label of the external location of the py_library target for twine twine_binary: A label of the external location of a binary target for twine. - publish_args: arguments passed to twine, e.g. ["--repository-url", "https://pypi.my.org/simple/"]. + publish_args: arguments passed to twine, e.g. `["--repository-url", "https://pypi.my.org/simple/"]`. These are subject to make var expansion, as with the `args` attribute. Note that you can also pass additional args to the bazel run command as in the example above. **kwargs: other named parameters passed to the underlying [py_wheel rule](#py_wheel_rule) diff --git a/python/private/attributes.bzl b/python/private/attributes.bzl index 61b41d527c..5909d8872b 100644 --- a/python/private/attributes.bzl +++ b/python/private/attributes.bzl @@ -271,7 +271,7 @@ source files. Possible values are: the source file. This is most useful when the code won't be modified. For more information on pyc invalidation modes, see -https://docs.python.org/3/library/py_compile.html#py_compile.PycInvalidationMode + """, default = PrecompileInvalidationModeAttr.AUTO, values = sorted(PrecompileInvalidationModeAttr.__members__.values()), @@ -281,7 +281,7 @@ https://docs.python.org/3/library/py_compile.html#py_compile.PycInvalidationMode The optimization level for precompiled files. For more information about optimization levels, see the `compile()` function's -`optimize` arg docs at https://docs.python.org/3/library/functions.html#compile +`optimize` arg docs at NOTE: The value `-1` means "current interpreter", which will be the interpreter used _at build time when pycs are generated_, not the interpreter used at @@ -422,7 +422,7 @@ These values are transitioned on, so will affect the analysis graph and the associated memory overhead. The more unique configurations in your overall build, the more memory and (often unnecessary) re-analysis and re-building can occur. See -https://bazel.build/extending/config#memory-performance-considerations for + for more information about risks and considerations. ::: diff --git a/python/private/py_cc_toolchain_info.bzl b/python/private/py_cc_toolchain_info.bzl index fcf407516c..7f443d05a9 100644 --- a/python/private/py_cc_toolchain_info.bzl +++ b/python/private/py_cc_toolchain_info.bzl @@ -105,7 +105,7 @@ If available, information about C libraries, struct with fields: "platform_machine": """ :type: str -The [PEP 508](https://peps.python.org/pep-0508/) `platform_machine` marker +The {pep}`PEP 508` `platform_machine` marker value for the target architecture, e.g. 'x86_64', 'aarch64'. """, "platform_tag": """\ @@ -122,7 +122,7 @@ The Python Major.Minor version. "sys_platform": """ :type: str -The [PEP 508](https://peps.python.org/pep-0508/) `sys_platform` marker value +The {pep}`0508` `sys_platform` marker value for the target OS, e.g. 'linux', 'darwin', 'win32'. """, }, diff --git a/python/private/py_cc_toolchain_rule.bzl b/python/private/py_cc_toolchain_rule.bzl index 194ce7be78..76528654b1 100644 --- a/python/private/py_cc_toolchain_rule.bzl +++ b/python/private/py_cc_toolchain_rule.bzl @@ -15,7 +15,7 @@ """Implementation of py_cc_toolchain rule. NOTE: This is a beta-quality feature. APIs subject to change until -https://github.com/bazel-contrib/rules_python/issues/824 is considered done. +{gh-issue}`824` is considered done. """ load("@bazel_skylib//rules:common_settings.bzl", "BuildSettingInfo") diff --git a/python/private/py_exec_tools_info.bzl b/python/private/py_exec_tools_info.bzl index 470c0c1bf0..43213d5a13 100644 --- a/python/private/py_exec_tools_info.bzl +++ b/python/private/py_exec_tools_info.bzl @@ -41,7 +41,7 @@ toolchain. :::{warning} This does not work correctly with RBE. Use {obj}`exec_runtime` instead. -Once https://github.com/bazelbuild/bazel/issues/23620 is resolved this warning +Once [bazelbuild/bazel#23620](https://github.com/bazelbuild/bazel/issues/23620) is resolved this warning may be removed. ::: """, diff --git a/python/private/py_executable.bzl b/python/private/py_executable.bzl index 1ee6fd7c99..2ac2423b86 100644 --- a/python/private/py_executable.bzl +++ b/python/private/py_executable.bzl @@ -98,7 +98,7 @@ EXECUTABLE_ATTRS = dicts.add( Arguments that are only applicable to the interpreter. The args an interpreter supports are specific to the interpreter. For -CPython, see https://docs.python.org/3/using/cmdline.html. +CPython, see . :::{note} Only supported for {obj}`--bootstrap_impl=script`. Ignored otherwise. @@ -147,7 +147,7 @@ Module name to execute as the main program. When set, `srcs` is not required, and it is assumed the module is provided by a dependency. -See https://docs.python.org/3/using/cmdline.html#cmdoption-m for more +See for more information about running modules as the main program. This is mutually exclusive with {obj}`main`. diff --git a/python/private/py_runtime_rule.bzl b/python/private/py_runtime_rule.bzl index b3399a24fb..4f450c4c9b 100644 --- a/python/private/py_runtime_rule.bzl +++ b/python/private/py_runtime_rule.bzl @@ -422,7 +422,7 @@ The template to use when two stage bootstrapping is enabled "Shebang" expression prepended to the bootstrapping Python stub script used when executing {rule}`py_binary` targets. -See https://github.com/bazelbuild/bazel/issues/8685 for +See [bazelbuild/bazel#8685](https://github.com/bazelbuild/bazel/issues/8685) for motivation. Does not apply to Windows. diff --git a/python/private/py_wheel.bzl b/python/private/py_wheel.bzl index b622411c56..c28aefff48 100644 --- a/python/private/py_wheel.bzl +++ b/python/private/py_wheel.bzl @@ -54,7 +54,7 @@ Workspace status keys are expanded using `{NAME}` format, for example: - `distribution = "package.{CLASSIFIER}"` - `distribution = "{DISTRIBUTION}"` -For the available keys, see https://bazel.build/docs/user-manual#workspace-status +For the available keys, see """, ), "platform": attr.string( @@ -196,7 +196,7 @@ The {attr}`add_path_prefix` attribute was added. default = "", ), "classifiers": attr.string_list( - doc = "A list of strings describing the categories for the package. For valid classifiers see https://pypi.org/classifiers", + doc = "A list of strings describing the categories for the package. For valid classifiers see ", ), "data_files": attr.label_keyed_string_dict( doc = (""" @@ -233,7 +233,7 @@ be moved under that directory. "description_content_type": attr.string( doc = ("The type of contents in description_file. " + "If not provided, the type will be inferred from the extension of description_file. " + - "Also see https://packaging.python.org/en/latest/specifications/core-metadata/#description-content-type"), + "Also see "), ), "description_file": attr.label( doc = "A file containing text describing the package.", @@ -332,7 +332,7 @@ def _escape_filename_distribution_name(name): def _escape_filename_segment(segment): """Escape a segment of the wheel filename. - See https://www.python.org/dev/peps/pep-0427/#escaping-and-unicode + See {pep}`PEP 427` """ # TODO: this is wrong, isalnum replaces non-ascii letters, while we should diff --git a/python/private/pypi/BUILD.bazel b/python/private/pypi/BUILD.bazel index 775aa0d9d2..bb84ff9280 100644 --- a/python/private/pypi/BUILD.bazel +++ b/python/private/pypi/BUILD.bazel @@ -522,6 +522,12 @@ bzl_library( deps = ["//python/private:envsubst"], ) +bzl_library( + name = "index_sources", + srcs = ["index_sources.bzl"], + deps = [":hash"], +) + bzl_library( name = "argparse", srcs = ["argparse.bzl"], @@ -542,12 +548,6 @@ bzl_library( srcs = ["hash.bzl"], ) -bzl_library( - name = "index_sources", - srcs = ["index_sources.bzl"], - deps = [":hash"], -) - bzl_library( name = "labels", srcs = ["labels.bzl"], diff --git a/python/private/pypi/extension.bzl b/python/private/pypi/extension.bzl index 210e169d4d..caca92a5f7 100644 --- a/python/private/pypi/extension.bzl +++ b/python/private/pypi/extension.bzl @@ -690,7 +690,7 @@ This value is going to be subject to `envsubst` substitutions if necessary, look {attr}`pip.parse.envsubst` documentation for more information.. The indexes must support Simple API as described here: -https://packaging.python.org/en/latest/specifications/simple-repository-api/ + Index metadata will be used to get the hash digest values for packages even if the `--hash` values are not present in the requirements.txt lock file. @@ -867,10 +867,10 @@ index URL. This design pattern has been chosen in order to be fully deterministic about which packages come from which source. We want to avoid issues similar to what happened in -https://pytorch.org/blog/compromised-nightly-dependency/. +. The indexes must support Simple API as described here: -https://packaging.python.org/en/latest/specifications/simple-repository-api/ + """, ), "hub_name": attr.string( diff --git a/python/private/pypi/pip_compile.bzl b/python/private/pypi/pip_compile.bzl index 58f7ba3a59..1bec3435a4 100644 --- a/python/private/pypi/pip_compile.bzl +++ b/python/private/pypi/pip_compile.bzl @@ -69,12 +69,12 @@ def pip_compile( defaults to `["pyproject.toml"]`. Supported formats are: * a requirements text file, usually named `requirements.in` * A `.toml` file, where the `project.dependencies` list is used as per - [PEP621](https://peps.python.org/pep-0621/). + {pep}`PEP 621`. src: file containing inputs to dependency resolution. If not specified, defaults to `pyproject.toml`. Supported formats are: * a requirements text file, usually named `requirements.in` * A `.toml` file, where the `project.dependencies` list is used as per - [PEP621](https://peps.python.org/pep-0621/). + {pep}`PEP 621`. extra_args: passed to pip-compile (aka `piptools`). See the [pip-compile docs](https://pip-tools.readthedocs.io/en/latest/cli/pip-compile) for args and meaning (passing `-h` and/or `--version` can help diff --git a/python/private/pypi/pip_repository.bzl b/python/private/pypi/pip_repository.bzl index 5fcd351958..436af73c8d 100644 --- a/python/private/pypi/pip_repository.bzl +++ b/python/private/pypi/pip_repository.bzl @@ -360,13 +360,13 @@ In some cases you may not want to generate the requirements.bzl file as a reposi while Bazel is fetching dependencies. For example, if you produce a reusable Bazel module such as a ruleset, you may want to include the requirements.bzl file rather than make your users install the WORKSPACE setup to generate it. -See https://github.com/bazel-contrib/rules_python/issues/608 +See {gh-issue}`608` This is the same workflow as Gazelle, which creates `go_repository` rules with [`update-repos`](https://github.com/bazelbuild/bazel-gazelle#update-repos) To do this, use the "write to source file" pattern documented in -https://blog.aspect.dev/bazel-can-write-to-the-source-folder + to put a copy of the generated requirements.bzl into your project. Then load the requirements.bzl file directly rather than from the generated repository. See the example in rules_python/examples/pip_parse_vendored. diff --git a/python/private/zipapp/py_zipapp_rule.bzl b/python/private/zipapp/py_zipapp_rule.bzl index 4d31c99311..b1a9399e90 100644 --- a/python/private/zipapp/py_zipapp_rule.bzl +++ b/python/private/zipapp/py_zipapp_rule.bzl @@ -344,7 +344,7 @@ These values are transitioned on, so will affect the analysis graph and the associated memory overhead. The more unique configurations in your overall build, the more memory and (often unnecessary) re-analysis and re-building can occur. See -https://bazel.build/extending/config#memory-performance-considerations for + for more information about risks and considerations. ::: """, diff --git a/python/runfiles/runfiles.py b/python/runfiles/runfiles.py index 7236d4b851..16afeea47c 100644 --- a/python/runfiles/runfiles.py +++ b/python/runfiles/runfiles.py @@ -614,7 +614,7 @@ def CurrentRepository(self, frame: int = 1) -> str: More information about the difference between canonical repository names and the `@repo` part of labels is available at: - https://bazel.build/build/bzlmod#repository-names + NOTE: This function inspects the callstack to determine where in the runfiles the caller is located to determine which repository it came From 61955f2413e6a1c63a6458b0dfdb23e01ed081a0 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Sat, 1 Aug 2026 19:49:48 -0700 Subject: [PATCH 887/922] docs(cc): clarify abi_tag is equivalent to SOABI and cite PEP 3149 (#3985) Clarify the documentation for `abi_tag` in C++ Python toolchains so that rule maintainers and users can easily map Bazel's toolchain attributes to standard Python distribution environment variables. Update docstrings in `python/private/py_cc_toolchain_info.bzl` and `python/private/py_cc_toolchain_rule.bzl` to explicitly note that `abi_tag` is the equivalent of the Python `sysconfig` variable `SOABI`, accompanied by a reference citation to PEP 3149. Also add `:::{versionadded} VERSION_NEXT_FEATURE` directives to all newly introduced toolchain fields and rule attributes. --- python/private/py_cc_toolchain_info.bzl | 19 ++++++++++++++++++- python/private/py_cc_toolchain_rule.bzl | 25 +++++++++++++++++++++++-- 2 files changed, 41 insertions(+), 3 deletions(-) diff --git a/python/private/py_cc_toolchain_info.bzl b/python/private/py_cc_toolchain_info.bzl index 7f443d05a9..19dc5de482 100644 --- a/python/private/py_cc_toolchain_info.bzl +++ b/python/private/py_cc_toolchain_info.bzl @@ -21,11 +21,19 @@ PyCcToolchainInfo = provider( :type: str The runtime's ABI flags, i.e. `sys.abiflags` (e.g. 't' for free-threaded builds). + +:::{versionadded} VERSION_NEXT_FEATURE +::: """, "abi_tag": """\ :type: str -The ABI tag for extension modules, e.g. 'cpython-311' or 'cpython-313t'. +The ABI tag for extension modules, equivalent to the `SOABI` sysconfig var +(see [PEP 3149](https://peps.python.org/pep-3149/)), e.g. 'cpython-311' or +'cpython-313t'. + +:::{versionadded} VERSION_NEXT_FEATURE +::: """, "headers": """\ :type: struct @@ -107,12 +115,18 @@ If available, information about C libraries, struct with fields: The {pep}`PEP 508` `platform_machine` marker value for the target architecture, e.g. 'x86_64', 'aarch64'. + +:::{versionadded} VERSION_NEXT_FEATURE +::: """, "platform_tag": """\ :type: str | None The PEP 3149 / PEP 425 platform tag for extension modules, e.g. 'x86_64-linux-gnu', 'darwin', or 'win_amd64'. + +:::{versionadded} VERSION_NEXT_FEATURE +::: """, "python_version": """ :type: str @@ -124,6 +138,9 @@ The Python Major.Minor version. The {pep}`0508` `sys_platform` marker value for the target OS, e.g. 'linux', 'darwin', 'win32'. + +:::{versionadded} 2.2.0 +::: """, }, ) diff --git a/python/private/py_cc_toolchain_rule.bzl b/python/private/py_cc_toolchain_rule.bzl index 76528654b1..a6cb29635c 100644 --- a/python/private/py_cc_toolchain_rule.bzl +++ b/python/private/py_cc_toolchain_rule.bzl @@ -144,10 +144,20 @@ The runtime's ABI flags, i.e. `sys.abiflags`. If not set, or set to ``, the ABI flags are automatically derived from `--//python/config_settings:py_freethreaded` (e.g., `'t'` when free-threaded is enabled, or `''` otherwise). + +:::{versionadded} VERSION_NEXT_FEATURE +::: """, ), "abi_tag": attr.string( - doc = "The ABI tag for extension modules, e.g. 'cpython-311'", + doc = """\ +The ABI tag for extension modules, equivalent to the `SOABI` sysconfig var +(see [PEP 3149](https://peps.python.org/pep-3149/)), e.g. 'cpython-311' or +'cpython-313t'. + +:::{versionadded} VERSION_NEXT_FEATURE +::: +""", default = "", ), "headers": attr.label( @@ -171,7 +181,12 @@ attribute is available or not. providers = [[SentinelInfo], [CcInfo]], ), "libc": attr.string( - doc = "Target C library variant, e.g. 'glibc', 'musl'", + doc = """\ +Target C library variant, e.g. 'glibc', 'musl'. + +:::{versionadded} VERSION_NEXT_FEATURE +::: +""", default = "", ), "libs": attr.label( @@ -182,6 +197,9 @@ attribute is available or not. "platform_machine": attr.string( doc = """ Target architecture as a PEP 508 `platform_machine` marker, e.g. 'x86_64', 'aarch64', 'x86_32'. + +:::{versionadded} VERSION_NEXT_FEATURE +::: """, default = "", ), @@ -192,6 +210,9 @@ Target architecture as a PEP 508 `platform_machine` marker, e.g. 'x86_64', 'aarc "sys_platform": attr.string( doc = """ Target OS as a PEP 508 `sys_platform` marker, e.g. 'linux', 'darwin', 'win32'. + +:::{versionadded} 2.2.0 +::: """, default = "", ), From 80867901ff4f472d5df97c2e74efdae377e34d6c Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Sun, 2 Aug 2026 17:09:17 -0700 Subject: [PATCH 888/922] agents(monitor-ci-results): ignore superseded retried Buildkite jobs in CI monitor (#3991) When a Buildkite job fails and is subsequently retried, both the original failed job and the retried green job remain in `data/jobs.json`. This betrayal by superseded failed jobs causes false-positive failure reporting on builds that are actually green. To fix this, update the job aggregation logic in the CI monitor: * Skip jobs in `data/jobs.json` where `retried_in_job_uuid` or `retried_at` is set so that only active, non-superseded jobs are evaluated. * Add `bucket` to the `--json` output fields in `gh pr checks` to provide additional check state metadata. --- .../skills/monitor-ci-results/scripts/monitor_remote_ci.py | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/.agents/skills/monitor-ci-results/scripts/monitor_remote_ci.py b/.agents/skills/monitor-ci-results/scripts/monitor_remote_ci.py index 9b5970757e..57fde0ce8d 100755 --- a/.agents/skills/monitor-ci-results/scripts/monitor_remote_ci.py +++ b/.agents/skills/monitor-ci-results/scripts/monitor_remote_ci.py @@ -27,7 +27,7 @@ def get_pr_checks(pr_number): if not check_cli("gh"): print("❌ 'gh' CLI not installed.", file=sys.stderr) return [] - cmd = ["gh", "pr", "checks", str(pr_number), "--json", "name,link,state"] + cmd = ["gh", "pr", "checks", str(pr_number), "--json", "name,link,state,bucket"] try: res = subprocess.run(cmd, capture_output=True, text=True) out = res.stdout @@ -157,6 +157,8 @@ def main(): other = 0 for job in jobs: + if job.get("retried_in_job_uuid") or job.get("retried_at"): + continue jstate = job.get("state", "unknown") exit_status = job.get("exit_status") is_soft_failed = job.get("soft_failed") is True @@ -194,6 +196,8 @@ def main(): ) for job in jobs: + if job.get("retried_in_job_uuid") or job.get("retried_at"): + continue jname = job.get("name", "unknown_job") jstate = job.get("state", "unknown") jid = job.get("id", "") From 18846e31aa3b8d68d2069665596f625d1d8fde50 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Sun, 2 Aug 2026 22:22:17 -0700 Subject: [PATCH 889/922] agents: add concurrent multi-subagent pre-review audit to create-pr skill (#3992) To catch oversights early and prevent low-quality pull requests from reaching reviewers, the PR creation process needs an automated quality gate. Without pre-review validation, common formatting, documentation, and testing errors can easily escape detection and cause unnecessary review iterations. This change integrates an automated, concurrent multi-subagent pre-review quality audit directly into the PR creation workflow: * Adds `.agents/rules/docs.md` and updates `.agents/rules/python.md` to define explicit domain standards for documentation and Python code. * Introduces `.agents/skills/pre-review-audit/SKILL.md` along with 6 specialized domain audit prompt files to inspect diffs across distinct technical areas concurrently. * Updates `.agents/skills/create-pr/SKILL.md` so that `create-pr` automatically spawns concurrent subagents to execute the pre-review audit before a PR is opened, strictly wrapping PR body text at 72 columns max. --- .agents/rules/docs.md | 15 +++++++ .agents/skills/create-pr/SKILL.md | 18 ++++++--- .agents/skills/pre-review-audit/SKILL.md | 40 +++++++++++++++++++ .../pre-review-audit/review-agents-prompt.md | 10 +++++ .../review-contributing-prompt.md | 9 +++++ .../pre-review-audit/review-docs-prompt.md | 12 ++++++ .../review-pr-standards-prompt.md | 9 +++++ .../pre-review-audit/review-python-prompt.md | 9 +++++ .../review-starlark-prompt.md | 11 +++++ 9 files changed, 128 insertions(+), 5 deletions(-) create mode 100644 .agents/rules/docs.md create mode 100644 .agents/skills/pre-review-audit/SKILL.md create mode 100644 .agents/skills/pre-review-audit/review-agents-prompt.md create mode 100644 .agents/skills/pre-review-audit/review-contributing-prompt.md create mode 100644 .agents/skills/pre-review-audit/review-docs-prompt.md create mode 100644 .agents/skills/pre-review-audit/review-pr-standards-prompt.md create mode 100644 .agents/skills/pre-review-audit/review-python-prompt.md create mode 100644 .agents/skills/pre-review-audit/review-starlark-prompt.md diff --git a/.agents/rules/docs.md b/.agents/rules/docs.md new file mode 100644 index 0000000000..99ce5608ac --- /dev/null +++ b/.agents/rules/docs.md @@ -0,0 +1,15 @@ +--- +trigger: glob +description: Documentation formatting, Sphinx MyST style rules, and documentation build correctness +globs: docs/*.md +--- + +# Documentation Rules + +* Act as an expert in tech writing, Sphinx, MyST, and markdown. +* Wrap lines at 80 columns. +* Use hyphens (`-`) in file names instead of underscores (`_`). +* In Sphinx MyST markup, outer directives must have more colons than inner directives. +* When adding `{versionadded}` or `{versionchanged}` sections, add them at the end of the documentation text. +* For unreleased features or attributes, use `VERSION_NEXT_FEATURE` in `{versionadded}` / `{versionchanged}` directives. +* **Documentation Build Correctness**: Ensure new `.bzl` files or user-facing APIs are properly registered in documentation build targets (e.g. `//docs:docs` or relevant Starlark API reference generation configs). diff --git a/.agents/skills/create-pr/SKILL.md b/.agents/skills/create-pr/SKILL.md index 9c2f454000..3510b15d07 100644 --- a/.agents/skills/create-pr/SKILL.md +++ b/.agents/skills/create-pr/SKILL.md @@ -9,9 +9,14 @@ to handle PR creation or description drafting. ### Instructions -1. Launch a subagent using `invoke_subagent` with `TypeName: "self"` (or +1. **Pre-Review Audit Subagent**: Before drafting or creating a PR, launch a + subagent to run the `pre-review-audit` skill on local changes (`git diff`). + Verify all checks pass (e.g., `VERSION_NEXT_FEATURE` directives, no Bazel + copyright headers, line wrapping, Starlark formatting). Fix any issues + found before proceeding. +2. Launch a subagent using `invoke_subagent` with `TypeName: "self"` (or `agentapi new-conversation`). -2. Provide a prompt to the subagent directing it to: +3. Provide a prompt to the subagent directing it to: - Include `@/.agents/rules/pr.md` and read `CONTRIBUTING.md` (specifically the sections on **Commit messages and PR descriptions** and **Documenting changes**) before drafting. @@ -19,6 +24,9 @@ to handle PR creation or description drafting. - **PR Title**: Follow conventional commit style and title formatting. For agent rules, skills, and system updates, use `agents:` prefix. - **PR Body**: Include rationale, high-level summary, and structure. + **CRITICAL**: Strictly wrap all PR body text at 72 columns max + (GitHub uses the PR description as the commit message upon merge, + which reflows text at 72 columns). - **Formatting**: Follow repository style guidelines and structure. - Create a Markdown artifact (`pr_info.md`) containing the PR title, body, and link/metadata so the user can review and comment on it. @@ -29,12 +37,12 @@ to handle PR creation or description drafting. - **Targeting Upstream Repo**: When executing `gh pr create`, always target the upstream repository by passing `--repo bazel-contrib/rules_python` and `--head :`. -3. **Return Status**: Direct the subagent to communicate the PR number or draft +4. **Return Status**: Direct the subagent to communicate the PR number or draft status back using `send_message` (or `agentapi send-message`) with the parent conversation ID, or include it in its final completion response. -4. **Publish Artifact**: Upon receiving the subagent completion message, the +5. **Publish Artifact**: Upon receiving the subagent completion message, the main agent must publish `pr_info.md` to display the artifact directly in the primary user UI. -5. **Interactive Actions**: To present custom action choices to the user +6. **Interactive Actions**: To present custom action choices to the user (e.g., "Create PR", "Create Draft PR"), the main agent can use the `ask_question` tool with custom options. diff --git a/.agents/skills/pre-review-audit/SKILL.md b/.agents/skills/pre-review-audit/SKILL.md new file mode 100644 index 0000000000..1c0b16e273 --- /dev/null +++ b/.agents/skills/pre-review-audit/SKILL.md @@ -0,0 +1,40 @@ +--- +name: pre-review-audit +description: High-level pre-review audit of local changes when preparing to send a PR for review +trigger: model_decision +--- + +When preparing to send a Pull Request for review, invoke **separate, concurrent sub-agents** (`invoke_subagent`), where each sub-agent focuses exclusively on its assigned checklist category. + +**CRITICAL**: Do NOT use a single sub-agent to validate multiple or all dimensions at once. Each sub-agent must be launched with its own distinct prompt file from `.agents/skills/pre-review-audit/`: + +### Audit Checklist & Sub-Agent Prompts + +1. **Starlark / Bazel Sub-Agent** (`Role: "Starlark Code Auditor"`): + - Prompt file: `.agents/skills/pre-review-audit/review-starlark-prompt.md` + - Focus: Audits Starlark / Bazel changes (`*.bzl`, `BUILD`, `*.bazel` files) in `git diff` against `.agents/rules/bzl.md` and Starlark rules in `AGENTS.md`. + +2. **Python Code Sub-Agent** (`Role: "Python Code Auditor"`): + - Prompt file: `.agents/skills/pre-review-audit/review-python-prompt.md` + - Focus: Audits Python source and test changes in `git diff` against `.agents/rules/python.md` and Python and pytest conventions in `AGENTS.md`. + +3. **Documentation Sub-Agent** (`Role: "Documentation Auditor"`): + - Prompt file: `.agents/skills/pre-review-audit/review-docs-prompt.md` + - Focus: Audits documentation (`.md`) changes and docs build targets against `.agents/rules/docs.md` and `AGENTS.md`. + +4. **Contribution Guidelines Sub-Agent** (`Role: "Contributing Auditor"`): + - Prompt file: `.agents/skills/pre-review-audit/review-contributing-prompt.md` + - Focus: Audits changes, requirements updates, and directives against `CONTRIBUTING.md`. + +5. **Project Conventions Sub-Agent** (`Role: "Project Conventions Auditor"`): + - Prompt file: `.agents/skills/pre-review-audit/review-agents-prompt.md` + - Focus: Audits overall workspace compliance against `AGENTS.md`. + +6. **Pull Request Standards Sub-Agent** (`Role: "PR Standards Auditor"`): + - Prompt file: `.agents/skills/pre-review-audit/review-pr-standards-prompt.md` + - Focus: Audits PR titles and descriptions against Conventional Commits formatting and PR update rules in `CONTRIBUTING.md`. + +### Action Instructions +- Launch all sub-agents concurrently using `invoke_subagent`. +- Collect the reports from each sub-agent and summarize any violations clearly with suggested fixes for the user. +- If all domain audits pass, confirm that the PR is ready for review. diff --git a/.agents/skills/pre-review-audit/review-agents-prompt.md b/.agents/skills/pre-review-audit/review-agents-prompt.md new file mode 100644 index 0000000000..f81924224c --- /dev/null +++ b/.agents/skills/pre-review-audit/review-agents-prompt.md @@ -0,0 +1,10 @@ +You are a specialized Project Conventions (`AGENTS.md`) Auditor sub-agent. +Your sole task is to audit all local changes (`git diff`) and workspace state against `AGENTS.md`: + +1. Read and strictly enforce `AGENTS.md` and all `.agents/rules/*.md` files without exception. +2. Verify NO Bazel copyright headers (`# Copyright ... The Bazel Authors`) were added to new or existing files, unless explicitly instructed by the user. +3. Verify that tests were executed using `bazel test --config=fast-tests` and non-test build targets did not use `--config=fast-tests`. +4. Ensure public config settings in `python/config_settings/BUILD.bazel` were not modified unless explicitly instructed. +5. Check that all repo rules and macro conventions described in `AGENTS.md` are respected. + +Report any violations found clearly with actionable suggested fixes, or report that the changes pass project conventions audit. diff --git a/.agents/skills/pre-review-audit/review-contributing-prompt.md b/.agents/skills/pre-review-audit/review-contributing-prompt.md new file mode 100644 index 0000000000..10a719284a --- /dev/null +++ b/.agents/skills/pre-review-audit/review-contributing-prompt.md @@ -0,0 +1,9 @@ +You are a specialized Contribution Guidelines Auditor sub-agent. +Your sole task is to audit all local changes (`git diff`), commit messages, and PR metadata against `CONTRIBUTING.md`: + +1. Read and strictly enforce `CONTRIBUTING.md`, all `.agents/rules/*.md` files, and `AGENTS.md`. +2. Verify that `{versionadded}` and `{versionchanged}` directives use `VERSION_NEXT_FEATURE` for unreleased features. +3. If locked/resolved requirements files (`requirements.txt`, `pyproject.toml`, `requirements.in`) were modified, verify that the associated `requirements.update` target was executed to keep locked requirement files in sync. +4. Ensure style and conventions described in `CONTRIBUTING.md` are respected across the changes. + +Report any violations found clearly with actionable suggested fixes, or report that the changes pass contribution audit. diff --git a/.agents/skills/pre-review-audit/review-docs-prompt.md b/.agents/skills/pre-review-audit/review-docs-prompt.md new file mode 100644 index 0000000000..02867940ed --- /dev/null +++ b/.agents/skills/pre-review-audit/review-docs-prompt.md @@ -0,0 +1,12 @@ +You are a specialized Documentation & Sphinx/MyST Auditor sub-agent. +Your sole task is to audit all documentation (`.md`) changes and new `.bzl` APIs in `git diff` against the project's documentation rules: + +1. Read and strictly enforce `.agents/rules/docs.md`, `AGENTS.md`, and `CONTRIBUTING.md`. +2. Check that lines wrap at 80 columns. +3. Ensure markdown filenames use hyphens (`-`) rather than underscores (`_`). +4. Verify Sphinx MyST colon indentation hierarchy (outer directives must have more colons than inner directives). +5. Verify `{versionadded}` and `{versionchanged}` sections are placed at the end of the documentation text. +6. For unreleased features or attributes, ensure `{versionadded}` / `{versionchanged}` directives use `VERSION_NEXT_FEATURE` (not hardcoded version numbers). +7. Check documentation build correctness: ensure new `.bzl` files or public APIs are included in `//docs:docs` or relevant docs build targets. + +Report any violations found clearly with actionable suggested fixes, or report that the documentation changes pass audit. diff --git a/.agents/skills/pre-review-audit/review-pr-standards-prompt.md b/.agents/skills/pre-review-audit/review-pr-standards-prompt.md new file mode 100644 index 0000000000..6120b412af --- /dev/null +++ b/.agents/skills/pre-review-audit/review-pr-standards-prompt.md @@ -0,0 +1,9 @@ +You are a specialized PR Standards Auditor sub-agent. +Your sole task is to audit PR titles and PR descriptions against `CONTRIBUTING.md` and project rules: + +1. Read and strictly enforce `.agents/rules/pr.md`, `.agents/rules/news.md`, `AGENTS.md`, and `CONTRIBUTING.md`. +2. Check PR title formatting: must follow Conventional Commits format (e.g. `feat(cc): ...`, `docs(python): ...`). For agent rules/skills, use `agents:` prefix. +3. Ensure PR descriptions explain *why* a change is made and provide a high-level overview of *how*, following advice in `CONTRIBUTING.md`. +4. If a PR has already been created, enforce PR update rules: do NOT amend or rebase existing commits (create new commits and merges instead, to preserve code review comment threads). + +Report any violations found clearly with actionable suggested fixes, or report that the PR standards pass audit. diff --git a/.agents/skills/pre-review-audit/review-python-prompt.md b/.agents/skills/pre-review-audit/review-python-prompt.md new file mode 100644 index 0000000000..aa2d726228 --- /dev/null +++ b/.agents/skills/pre-review-audit/review-python-prompt.md @@ -0,0 +1,9 @@ +You are a specialized Python & pytest Auditor sub-agent. +Your sole task is to audit all Python source (`.py`) and test changes in `git diff` against the project's Python conventions: + +1. Read and strictly enforce `.agents/rules/python.md`, `AGENTS.md`, and `CONTRIBUTING.md`. +2. Check Python pytest conventions: when registering pytest fixtures from helper modules in test files, use `pytest_plugins = [""]`. +3. Name fixture functions with a `fixture_` prefix (e.g. `def fixture_foo():`) and pass the public fixture name using `@pytest.fixture(name="foo")`. +4. Verify that tests were executed using Bazel (`bazel test --config=fast-tests`) and passed. + +Report any violations found clearly with actionable suggested fixes, or report that the Python changes pass audit. diff --git a/.agents/skills/pre-review-audit/review-starlark-prompt.md b/.agents/skills/pre-review-audit/review-starlark-prompt.md new file mode 100644 index 0000000000..8421219a36 --- /dev/null +++ b/.agents/skills/pre-review-audit/review-starlark-prompt.md @@ -0,0 +1,11 @@ +You are a specialized Starlark & Bazel Code Auditor sub-agent. +Your sole task is to audit all Starlark (`.bzl`, `BUILD`, `*.bazel`) changes in `git diff` against the project's Starlark coding rules and conventions: + +1. Read and strictly enforce `.agents/rules/starlark.md`, `.agents/rules/bzl.md`, `AGENTS.md`, and `CONTRIBUTING.md`. +2. Verify iterative algorithms are used (no recursion, no `while` loops). +3. Ensure every `.bzl` file outside `tests/` has a corresponding `bzl_library` target in its `BUILD` file with proper dependencies. +4. Ensure loads from `/private/` in test files have `# buildifier: disable=bzl-visibility`. +5. Check multi-line rule/macro doc arguments: use triple-quoted strings (`"""`), and do NOT use trailing backslashes (`\`) on opening triple-quotes. +6. Verify analysis tests use `rules_testing`, not `bazel_skylib`. + +Report any violations found clearly with actionable suggested fixes, or report that the Starlark changes pass audit. From 16e167a5b269930f2f5000c4d9dc99b72f930f2b Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Mon, 3 Aug 2026 06:09:45 -0700 Subject: [PATCH 890/922] fix(sphinxdocs): include stderr in Sphinx worker response output (#3994) When running Sphinx in persistent worker mode, warnings and error messages emitted to stderr during build execution were not included in the worker response output sent back to Bazel. This made it difficult to diagnose build warnings or diagnostic messages printed to stderr when the worker succeeded with exit code 0. To fix this, include both stdout and stderr in the worker response's output dictionary with clear `--- STDOUT ---` and `--- STDERR ---` separators when stderr is present, and document why stderr is included. Work towards #3977. --- sphinxdocs/sphinxdocs/private/sphinx_build.py | 12 +++++++++++- 1 file changed, 11 insertions(+), 1 deletion(-) diff --git a/sphinxdocs/sphinxdocs/private/sphinx_build.py b/sphinxdocs/sphinxdocs/private/sphinx_build.py index 790d4dff84..8b0eec8fd3 100644 --- a/sphinxdocs/sphinxdocs/private/sphinx_build.py +++ b/sphinxdocs/sphinxdocs/private/sphinx_build.py @@ -267,9 +267,19 @@ def _process_request(self, request: "WorkRequest") -> "WorkResponse | None": # implicily bring along what the symlinks point to. shutil.copytree(worker_outdir, bazel_outdir, dirs_exist_ok=True) + # Include both stdout and stderr in the response output so that Sphinx + # warnings or diagnostic messages written to stderr are reported to the + # Bazel console even when the build succeeds. + stdout_output = stdout.getvalue() + stderr_output = stderr.getvalue() + if stderr_output: + output = f"--- STDOUT ---\n{stdout_output}\n--- STDERR ---\n{stderr_output}" + else: + output = stdout_output + response = { "requestId": request.get("requestId", 0), - "output": stdout.getvalue(), + "output": output, "exitCode": 0, } return response From 8b460f82a5379a0fc1a2fe9d93f8c6905e88a49d Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Mon, 3 Aug 2026 16:33:27 -0700 Subject: [PATCH 891/922] agents(pr): prohibit per-file edit lists in PR descriptions (#3996) Update the pull request rules to explicitly prohibit per-file edit lists or individual file changelog items in PR descriptions and commit messages. Reinforce that PR descriptions must provide a high-level overview explaining why a change is made and how at a conceptual level, and mandate linking related issues. --- .agents/rules/pr.md | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/.agents/rules/pr.md b/.agents/rules/pr.md index ba8236d072..caacb07519 100644 --- a/.agents/rules/pr.md +++ b/.agents/rules/pr.md @@ -1,6 +1,6 @@ --- trigger: model_decision -description: Apply when drafting pull request descriptions. +description: rules to apply to pull request descriptions --- @CONTRIBUTING.md @@ -18,3 +18,7 @@ Before drafting any pull request description, strictly adhere to the rules in * Once a Pull Request is created, always make new commits or merge commits. * **NEVER** amend or rebase commits on an active PR branch to avoid breaking code review threads. +* **NEVER** include a list of per-file edits or changelog bullet points of + individual file modifications in PR descriptions or commit messages. +* High-level overview only: state *why* the change is made and *how* at a + conceptual level. Link related issues (e.g. `Work towards #`). From 874a6832c3bb20d623e91e1db516e9baa3c31e49 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Mon, 3 Aug 2026 19:56:21 -0700 Subject: [PATCH 892/922] agents: explain force-merging while requiring user consent (#3998) The merge-pr skill tells agents how to force merge pull requests using gh pr merge --admin, but also explicitly requires that agents obtain user consent before bypassing merge queues or required checks. --- .agents/skills/merge-pr/SKILL.md | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/.agents/skills/merge-pr/SKILL.md b/.agents/skills/merge-pr/SKILL.md index 73086759ed..30b8f38af4 100644 --- a/.agents/skills/merge-pr/SKILL.md +++ b/.agents/skills/merge-pr/SKILL.md @@ -6,6 +6,11 @@ description: Merge a pull request into main, monitoring the merge queue, retryin When the user asks to merge a pull request (e.g., "merge PR ", "merge this PR", or monitor its merge): 1. **Enqueue for Merge**: Run `gh pr merge --auto --squash` to enable auto-merge or add the pull request to the merge queue. + - **Force / Admin Merge**: **CRITICAL**: Passing `--admin` to bypass the + merge queue or required checks requires explicit user permission or + consent. Only pass `--admin` (e.g., `gh pr merge --admin + --squash`) if the user has explicitly requested or approved a force/admin + merge. Never invoke `--admin` autonomously. 2. **Invoke a Background Shepherd**: Launch a background subagent with the role `Merge PR Shepherd` to continuously watch the PR until it merges. 3. **Leverage Existing CI Skills**: - Have the subagent use the **`monitor-ci-results`** skill to watch for CI check failures and generate analysis reports. From c73f93199b7d99091b3cca8822b324f942080f89 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Mon, 3 Aug 2026 21:44:54 -0700 Subject: [PATCH 893/922] feat(cc): populate abi and platform settings on local py_cc_toolchain (#3990) feat(cc): populate abi and platform settings on local py_cc_toolchain The introduction of `py_extension` added `abi_flags`, `abi_tag`, `platform_machine`, and `sys_platform` to `py_cc_toolchain`. Local Python runtime toolchains did not populate these fields, causing `py_extension` builds to fail when using local toolchains. Extract `sys_platform`, `platform_machine`, and `abi_tag` (from `sysconfig` `SOABI`) in `get_local_runtime_info.py`, pass them through `local_runtime_repo.bzl`, and set them on `py_cc_toolchain` in `local_runtime_repo_setup.bzl`. --- python/private/get_local_runtime_info.py | 4 ++++ python/private/local_runtime_repo.bzl | 9 +++++++++ python/private/local_runtime_repo_setup.bzl | 12 +++++++++++- 3 files changed, 24 insertions(+), 1 deletion(-) diff --git a/python/private/get_local_runtime_info.py b/python/private/get_local_runtime_info.py index 787fad5635..340ef934f7 100644 --- a/python/private/get_local_runtime_info.py +++ b/python/private/get_local_runtime_info.py @@ -16,6 +16,7 @@ import glob import json import os +import platform import sys import sysconfig from typing import Any @@ -247,6 +248,7 @@ def _unique_basenames(inputs: dict[str, None]) -> list[str]: "abi_dynamic_libraries": _unique_basenames(abi_dynamic_libraries), "abi_interface_libraries": _unique_basenames(abi_interface_libraries), "abi_flags": abi_flags, + "abi_tag": config_vars.get("SOABI") or "", "shlib_suffix": ".dylib" if _IS_DARWIN else "", "additional_dlls": dlls, "defines": defines, @@ -265,6 +267,8 @@ def _get_base_executable() -> str: "include": sysconfig.get_path("include"), "implementation_name": sys.implementation.name, "base_executable": _get_base_executable(), + "sys_platform": sys.platform, + "platform_machine": platform.machine(), } data.update(_get_python_library_info(_get_base_executable())) print(json.dumps(data)) diff --git a/python/private/local_runtime_repo.bzl b/python/private/local_runtime_repo.bzl index 37b7d2b130..48fc576bb7 100644 --- a/python/private/local_runtime_repo.bzl +++ b/python/private/local_runtime_repo.bzl @@ -35,6 +35,7 @@ define_local_runtime_toolchain_impl( minor = "{minor}", micro = "{micro}", abi_flags = "{abi_flags}", + abi_tag = "{abi_tag}", os = "{os}", implementation_name = "{implementation_name}", interpreter_path = "{interpreter_path}", @@ -44,6 +45,8 @@ define_local_runtime_toolchain_impl( abi3_interface_library = {abi3_interface_library}, abi3_libraries = {abi3_libraries}, additional_dlls = {additional_dlls}, + sys_platform = "{sys_platform}", + platform_machine = "{platform_machine}", ) """ @@ -53,6 +56,7 @@ def _expand_incompatible_template(): minor = "0", micro = "0", abi_flags = "", + abi_tag = "", os = "@platforms//:incompatible", implementation_name = "incompatible", interpreter_path = "/incompatible", @@ -62,6 +66,8 @@ def _expand_incompatible_template(): abi3_interface_library = "None", abi3_libraries = "[]", additional_dlls = "[]", + sys_platform = "", + platform_machine = "", ) def _norm_path(path): @@ -210,6 +216,7 @@ def _local_runtime_repo_impl(rctx): minor = info["minor"], micro = info["micro"], abi_flags = info["abi_flags"], + abi_tag = info["abi_tag"], os = "@platforms//os:{}".format(repo_utils.get_platforms_os_name(rctx)), implementation_name = info["implementation_name"], interpreter_path = _norm_path(interpreter_path), @@ -219,6 +226,8 @@ def _local_runtime_repo_impl(rctx): abi3_interface_library = repr(abi3_interface_library), abi3_libraries = repr(abi3_libraries), additional_dlls = repr(additional_dlls), + sys_platform = info["sys_platform"], + platform_machine = info["platform_machine"], ) logger.debug(lambda: "BUILD.bazel\n{}".format(build_bazel)) diff --git a/python/private/local_runtime_repo_setup.bzl b/python/private/local_runtime_repo_setup.bzl index 5cb7bda200..78c8bd0093 100644 --- a/python/private/local_runtime_repo_setup.bzl +++ b/python/private/local_runtime_repo_setup.bzl @@ -37,7 +37,10 @@ def define_local_runtime_toolchain_impl( defines, abi3_interface_library, abi3_libraries, - additional_dlls): + additional_dlls, + sys_platform = "", + platform_machine = "", + abi_tag = ""): """Defines a toolchain implementation for a local Python runtime. Generates public targets: @@ -71,6 +74,9 @@ def define_local_runtime_toolchain_impl( e.g. ["lib/python3.dll"] or ["lib/python3.so"] additional_dlls: `list[str]` Path[s] to additional DLLs. e.g. ["lib/msvcrt123.dll"] + sys_platform: `str` The PEP 508 `sys_platform` marker, e.g. 'linux', 'darwin', 'win32'. + platform_machine: `str` The PEP 508 `platform_machine` marker, e.g. 'x86_64', 'aarch64'. + abi_tag: `str` The ABI tag for extension modules, e.g. 'cpython-311'. """ major_minor = "{}.{}".format(major, minor) major_minor_micro = "{}.{}".format(major_minor, micro) @@ -183,10 +189,14 @@ def define_local_runtime_toolchain_impl( py_cc_toolchain( name = "py_cc_toolchain", + abi_flags = abi_flags, + abi_tag = abi_tag, headers = ":python_headers", headers_abi3 = ":python_headers_abi3", libs = ":libpython", + platform_machine = platform_machine, python_version = major_minor_micro, + sys_platform = sys_platform, visibility = ["//visibility:public"], ) From 7eb4a6bb3c3afb7800680905b9a92d9f9d352014 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Tue, 4 Aug 2026 06:01:46 -0700 Subject: [PATCH 894/922] refactor(cc): simplify _current_py_cc_libs_impl (#3993) Commit 0d6016ffe5307e2e7c30a90319f14f8efa801d2b accidentally added complex filtering logic for static libraries and DLLs to `_current_py_cc_libs_impl` in `python/private/current_py_cc_libs.bzl`. That extra filtering logic was actually a hack to support another hack in `py_extension`, and shouldn't have been added. To simplify, update `_current_py_cc_libs_impl` to return `py_cc_toolchain.libs.providers_map.values()` directly. Since Windows was relying on that hack, move it into a helper rule in the py_extension implementation. --- python/private/cc/py_extension_macro.bzl | 23 ++++++++++------ python/private/cc/py_extension_rule.bzl | 24 +++++++++++++++++ python/private/current_py_cc_libs.bzl | 34 +----------------------- 3 files changed, 40 insertions(+), 41 deletions(-) diff --git a/python/private/cc/py_extension_macro.bzl b/python/private/cc/py_extension_macro.bzl index 7845a09ea2..b57e9a438e 100644 --- a/python/private/cc/py_extension_macro.bzl +++ b/python/private/cc/py_extension_macro.bzl @@ -8,7 +8,7 @@ load("@rules_cc//cc:cc_library.bzl", "cc_library") load("@rules_cc//cc:cc_shared_library.bzl", "cc_shared_library") load("//python/private:common_labels.bzl", "labels") load("//python/private:util.bzl", "add_tag", "copy_propagating_kwargs") -load(":py_extension_rule.bzl", "py_extension_wrapper") +load(":py_extension_rule.bzl", "py_extension_libs", "py_extension_wrapper") _EMPTY_CANONICAL_TARGET = str(Label("//python/private/cc:empty")) @@ -18,9 +18,6 @@ _PY_CC_HEADERS_ALIAS_CANONICAL_TARGET = str(Label(_PY_CC_HEADERS_ALIAS_BASE_TARG _PY_CC_LIBS_ALIAS_BASE_TARGET = "//python/private/cc:current_py_cc_libs_private_alias" _PY_CC_LIBS_ALIAS_CANONICAL_TARGET = str(Label(_PY_CC_LIBS_ALIAS_BASE_TARGET)) -_PY_CC_LIBS_ACTUAL_BASE_TARGET = "//python/cc:current_py_cc_libs" -_PY_CC_LIBS_ACTUAL_CANONICAL_TARGET = str(Label(_PY_CC_LIBS_ACTUAL_BASE_TARGET)) - def py_extension( name, srcs = None, @@ -148,22 +145,32 @@ def py_extension( "//conditions:default": [], }) + win_libs_name = "_" + name + "_win_libs" + + # On Windows, create a private target using toolchain resolution to extract .lib files + # from CcInfo in py_cc_toolchain because system_provided=True in cc_import leaves DefaultInfo empty. + py_extension_libs( + name = win_libs_name, + tags = ["manual"], + visibility = ["//visibility:private"], + ) + # Windows-specific CPython linking requirements: # 1. Windows requires .lib files when linking, so they must be added to deps. # 2. CPython import libraries (python3xx.lib) are declared with system_provided = True # in cc_import, suppressing automatic propagation of the .lib file path to link.exe. - # We explicitly pass $(locations ...) to provide the path of the CPython import library to MSVC link.exe. - # 3. We pass current_py_cc_libs as an additional linker input to ensure the .lib file is available to the link action. + # We explicitly pass $(locations ...) from win_libs_name to provide the path to MSVC link.exe. + # 3. We pass win_libs_name as an additional linker input to ensure the .lib file is available to the link action. deps = deps + select({ labels.PLATFORMS_OS_WINDOWS: [_PY_CC_LIBS_ALIAS_CANONICAL_TARGET], "//conditions:default": [], }) user_link_flags = user_link_flags + select({ - labels.PLATFORMS_OS_WINDOWS: ["$(locations " + _PY_CC_LIBS_ACTUAL_BASE_TARGET + ")"], + labels.PLATFORMS_OS_WINDOWS: ["$(locations :" + win_libs_name + ")"], "//conditions:default": [], }) additional_linker_inputs = additional_linker_inputs + select({ - labels.PLATFORMS_OS_WINDOWS: [_PY_CC_LIBS_ACTUAL_CANONICAL_TARGET], + labels.PLATFORMS_OS_WINDOWS: [":" + win_libs_name], "//conditions:default": [], }) diff --git a/python/private/cc/py_extension_rule.bzl b/python/private/cc/py_extension_rule.bzl index 1c5ee11a06..2b3a8284b7 100644 --- a/python/private/cc/py_extension_rule.bzl +++ b/python/private/cc/py_extension_rule.bzl @@ -133,3 +133,27 @@ def _get_platform(ctx): ), ) return py_cc_toolchain.platform_tag + +def _py_extension_libs_impl(ctx): + py_toolchain = ctx.toolchains[PY_CC_TOOLCHAIN_TYPE] + py_cc_toolchain = py_toolchain.py_cc_toolchain + cc_info = py_cc_toolchain.libs.providers_map["CcInfo"] + files = [] + for input in cc_info.linking_context.linker_inputs.to_list(): + for lib in input.libraries: + if lib.interface_library: + files.append(lib.interface_library) + elif lib.static_library: + files.append(lib.static_library) + elif lib.dynamic_library: + files.append(lib.dynamic_library) + link_files = [f for f in files if not f.path.endswith(".dll")] + return [DefaultInfo(files = depset(link_files))] + +py_extension_libs = rule( + implementation = _py_extension_libs_impl, + toolchains = [PY_CC_TOOLCHAIN_TYPE], + doc = """\ +Private internal helper rule for extracting Windows C/C++ library files from toolchain. +""", +) diff --git a/python/private/current_py_cc_libs.bzl b/python/private/current_py_cc_libs.bzl index 58ab4b1bd8..ca68346bcb 100644 --- a/python/private/current_py_cc_libs.bzl +++ b/python/private/current_py_cc_libs.bzl @@ -18,39 +18,7 @@ load("@rules_cc//cc/common:cc_info.bzl", "CcInfo") def _current_py_cc_libs_impl(ctx): py_cc_toolchain = ctx.toolchains["//python/cc:toolchain_type"].py_cc_toolchain - providers = [p for p in py_cc_toolchain.libs.providers_map.values() if not hasattr(p, "data_runfiles")] - default_runfiles = None - data_runfiles = None - files = [] - for p in py_cc_toolchain.libs.providers_map.values(): - if hasattr(p, "data_runfiles"): - default_runfiles = p.default_runfiles - data_runfiles = p.data_runfiles - - cc_infos = [p for p in py_cc_toolchain.libs.providers_map.values() if hasattr(p, "linking_context")] - if cc_infos: - cc_info = cc_infos[0] - for input in cc_info.linking_context.linker_inputs.to_list(): - for lib in input.libraries: - if lib.static_library: - files.append(lib.static_library) - if lib.interface_library: - files.append(lib.interface_library) - elif lib.dynamic_library: - files.append(lib.dynamic_library) - - # On Windows MSVC, user_link_flags passes $(locations @rules_python//python/cc:current_py_cc_libs) - # to link.exe. MSVC link.exe accepts import libraries (.lib) but fails with - # LNK1107 if passed raw DLL binaries (.dll). We filter out .dll files so - # DefaultInfo.files only contains linkable library files (.lib / .a). - link_files = [f for f in files if not f.path.endswith(".dll")] - - providers.append(DefaultInfo( - files = depset(link_files), - default_runfiles = default_runfiles, - data_runfiles = data_runfiles, - )) - return providers + return py_cc_toolchain.libs.providers_map.values() current_py_cc_libs = rule( implementation = _current_py_cc_libs_impl, From 0118a70aedbb749133758ca553761cd55e44fa3c Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Tue, 4 Aug 2026 06:04:16 -0700 Subject: [PATCH 895/922] docs(cc): add py_extension to doc generation and update versionadded markup (#3989) Add py_extension to Sphinx Stardoc generation in docs/BUILD.bazel and resolve transitive bzl_library dependencies in python/private/cc/BUILD.bazel so API reference generation succeeds. Update py_extension docstrings in python/cc/py_extension.bzl and python/private/cc/py_extension_macro.bzl to use `:::{versionadded} VERSION_NEXT_FEATURE` per CONTRIBUTING.md. --- docs/BUILD.bazel | 1 + python/cc/py_extension.bzl | 3 +++ python/private/cc/BUILD.bazel | 3 ++- python/private/cc/py_extension_macro.bzl | 3 +++ python/private/cc/py_extension_rule.bzl | 2 -- tests/BUILD.bazel | 1 + 6 files changed, 10 insertions(+), 3 deletions(-) diff --git a/docs/BUILD.bazel b/docs/BUILD.bazel index dd304c410a..baa1a1e3db 100644 --- a/docs/BUILD.bazel +++ b/docs/BUILD.bazel @@ -128,6 +128,7 @@ sphinx_stardocs( "//python/api:rule_builders", "//python/cc:py_cc_toolchain", "//python/cc:py_cc_toolchain_info", + "//python/cc:py_extension", "//python/entry_points:py_console_script_binary", "//python/extensions:config", "//python/extensions:python", diff --git a/python/cc/py_extension.bzl b/python/cc/py_extension.bzl index 2d5570b546..d81877a702 100644 --- a/python/cc/py_extension.bzl +++ b/python/cc/py_extension.bzl @@ -10,6 +10,9 @@ for information on writing C extension modules. :::{include} /_includes/experimental_api.md ::: + +:::{versionadded} VERSION_NEXT_FEATURE +::: """ load( diff --git a/python/private/cc/BUILD.bazel b/python/private/cc/BUILD.bazel index a89c47c58f..087d406932 100644 --- a/python/private/cc/BUILD.bazel +++ b/python/private/cc/BUILD.bazel @@ -43,6 +43,7 @@ bzl_library( srcs = ["py_extension_macro.bzl"], deps = [ ":py_extension_rule", + "//python/private:common_labels", "//python/private:util", "@rules_cc//cc:core_rules", ], @@ -55,11 +56,11 @@ bzl_library( "//python/private:attr_builders", "//python/private:attributes", "//python/private:builders", + "//python/private:common", "//python/private:py_info", "//python/private:reexports", "//python/private:rule_builders", "//python/private:toolchain_types", "@bazel_skylib//lib:dicts", - "@rules_cc//cc/common", ], ) diff --git a/python/private/cc/py_extension_macro.bzl b/python/private/cc/py_extension_macro.bzl index b57e9a438e..6b697ab503 100644 --- a/python/private/cc/py_extension_macro.bzl +++ b/python/private/cc/py_extension_macro.bzl @@ -50,6 +50,9 @@ def py_extension( - `module_name`: Pass `module_name = "custom_name"` to override the base module filename. + :::{versionadded} VERSION_NEXT_FEATURE + ::: + Args: name: {type}`str` Target name. srcs: {type}`list[Label | str] | None` C/C++ source files to compile diff --git a/python/private/cc/py_extension_rule.bzl b/python/private/cc/py_extension_rule.bzl index 2b3a8284b7..dc5a5e0c59 100644 --- a/python/private/cc/py_extension_rule.bzl +++ b/python/private/cc/py_extension_rule.bzl @@ -5,7 +5,6 @@ """ load("@bazel_skylib//lib:dicts.bzl", "dicts") -load("@rules_cc//cc/common:cc_shared_library_info.bzl", "CcSharedLibraryInfo") load("//python/private:attr_builders.bzl", "attrb") load("//python/private:attributes.bzl", "COMMON_ATTRS", "IMPORTS_ATTRS", "WINDOWS_CONSTRAINTS_ATTRS") load("//python/private:builders.bzl", "builders") @@ -78,7 +77,6 @@ PY_EXTENSION_WRAPPER_ATTRS = dicts.add( ), "src": lambda: attrb.Label( mandatory = True, - providers = [CcSharedLibraryInfo], doc = "The cc_shared_library target to wrap.", ), }, diff --git a/tests/BUILD.bazel b/tests/BUILD.bazel index e7dbef65d8..927fc30566 100644 --- a/tests/BUILD.bazel +++ b/tests/BUILD.bazel @@ -22,6 +22,7 @@ build_test( "//python:py_test_bzl", "//python/cc:py_cc_toolchain_bzl", "//python/cc:py_cc_toolchain_info_bzl", + "//python/cc:py_extension", "//python/entry_points:py_console_script_binary_bzl", ], ) From 5f268ab0dae6fdc6fb4f8fd3a698863eeaacc8c5 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Tue, 4 Aug 2026 18:44:57 -0700 Subject: [PATCH 896/922] agents(monitor-ci-results): autonomous flake retries via subagent and forbid file lists in PRs (#4001) agents(monitor-ci-results): autonomous flake retries via subagent and forbid file lists in PRs Scopes process checks (`pgrep -f "monitor_remote_ci.py "`) specifically to the target PR number to prevent cross-PR monitoring conflicts when multiple agent conversations run concurrently. Automates recovery from transient infrastructure or network flakes without requiring manual human intervention. Updates `monitor-ci-results` workflow steps to require PR-scoped process lookups and introduces an autonomous retry step that spawns a dedicated retrier subagent via a prompt template (`retry-job-prompt.md`) to execute the `buildkite-retry-job` skill. --- .agents/skills/monitor-ci-results/SKILL.md | 15 +++++++++++++-- .../skills/monitor-ci-results/retry-job-prompt.md | 3 +++ 2 files changed, 16 insertions(+), 2 deletions(-) create mode 100644 .agents/skills/monitor-ci-results/retry-job-prompt.md diff --git a/.agents/skills/monitor-ci-results/SKILL.md b/.agents/skills/monitor-ci-results/SKILL.md index 3a1fab866b..e2475b55ad 100644 --- a/.agents/skills/monitor-ci-results/SKILL.md +++ b/.agents/skills/monitor-ci-results/SKILL.md @@ -12,8 +12,10 @@ or when monitoring CI after PR updates: > launch duplicate monitoring jobs for the same PR. 1. **Check Existing Process**: Check if a monitor script is already running for - the PR (e.g., `pgrep -f "monitor_remote_ci.py "`). If one is - already running, do not start another instance. + **this specific PR** using `pgrep -f "monitor_remote_ci.py "`. Do + NOT use a generic grep without ``, as other active agent + conversations may be monitoring different PRs concurrently. If one is + already running for this PR, do not start another instance. 2. **Launch Monitoring Script**: If no monitor process is active for ``, launch the script in the background: ```bash @@ -35,3 +37,12 @@ or when monitoring CI after PR updates: the monitoring script, immediately launch a separate subagent using the `invoke_subagent` tool with the role "CI Failure Analyzer" to run the `analyze-ci-failure` skill on the reported failure. +5. **Autonomous Flake Retry**: If the CI Failure Analyzer subagent confirms + that the failure is a transient infrastructure or network flake (e.g. disk + I/O error, 504 gateway, sandbox initialization failure), **immediately and + autonomously** launch a separate subagent using `invoke_subagent` with the + role "CI Job Retrier" and the prompt template in + `.agents/skills/monitor-ci-results/retry-job-prompt.md` to run the + `buildkite-retry-job` skill for ``. Do not execute the script + directly; use the subagent and skill orchestration instead. Continue + monitoring without pausing to ask the user. diff --git a/.agents/skills/monitor-ci-results/retry-job-prompt.md b/.agents/skills/monitor-ci-results/retry-job-prompt.md new file mode 100644 index 0000000000..4d2edad650 --- /dev/null +++ b/.agents/skills/monitor-ci-results/retry-job-prompt.md @@ -0,0 +1,3 @@ +Use the `buildkite-retry-job` skill to retry the failed Buildkite job(s) for PR {pr_number}. +Follow the instructions in `.agents/skills/buildkite-retry-job/SKILL.md` to retry the failing jobs using the skill. +Once the retry request completes, report back to the parent conversation. From 39b8c3c1da239b86fa23ccf70a2821cd55b60cee Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Tue, 4 Aug 2026 18:46:40 -0700 Subject: [PATCH 897/922] agents(rules): add manual tag rule and frontmatter to starlark.md (#4002) agents(rules): add manual tag rule and frontmatter to starlark.md Add YAML frontmatter triggers (`*.bzl,BUILD,BUILD.bazel,*.bazel`) to `.agents/rules/starlark.md` so Bazel rules load automatically, and document that internal macro helper targets must use `tags = ["manual"]`) to avoid being built during wildcard expansion. --- .agents/rules/starlark.md | 12 ++++++++++++ 1 file changed, 12 insertions(+) diff --git a/.agents/rules/starlark.md b/.agents/rules/starlark.md index 430284bcd5..4517b06aec 100644 --- a/.agents/rules/starlark.md +++ b/.agents/rules/starlark.md @@ -1,3 +1,9 @@ +--- +trigger: glob +description: Starlark Language, Macro, and Testing Invariants +globs: "*.bzl,BUILD,BUILD.bazel,*.bazel" +--- + # Starlark Language & Macro Invariants ## Macro Target Canonicalization @@ -7,6 +13,12 @@ * Note that `python/private/common_labels.bzl` defines `labels`, a struct containing common canonicalized label strings used across the project. +## Manual Tag on Internal Macro Helper Targets +* When macros instantiate internal helper targets (such as private rule targets + for artifact extraction or linking support), always include `tags = ["manual"]`. +* **Why**: This prevents internal helper targets from being implicitly built when + wildcard target patterns (e.g., `//...`) are expanded. + ## Private Alias Pattern when Appending to User `deps` * When macros append internal helper targets to user-provided dependency lists (`deps`), use private alias targets (e.g. From e3291d3a17fdd3042bf58e15c3f3db55fa6fdd2f Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Tue, 4 Aug 2026 19:53:48 -0700 Subject: [PATCH 898/922] feat(python): add py_extension to features loadable_symbols (#4000) Add py_extension to features.loadable_symbols. This allows callers to use feature detection and optionally load it, if desired. --- python/features.bzl | 12 ++++++++++++ 1 file changed, 12 insertions(+) diff --git a/python/features.bzl b/python/features.bzl index 9339d7889c..99050a7ca5 100644 --- a/python/features.bzl +++ b/python/features.bzl @@ -26,6 +26,10 @@ def _features_typedef(): A map of public API targets available in rules_python for feature detection purposes. + :::{seealso} + * {obj}`features.loadable_symbols` + ::: + :::{versionadded} 1.9.0 ::: :::: @@ -54,6 +58,10 @@ def _features_typedef(): A map of bzl paths to the list of public symbols they export. + :::{seealso} + * {obj}`features.targets` + ::: + :::{versionadded} 2.2.0 ::: :::: @@ -128,6 +136,10 @@ _TARGETS = { } _LOADABLE_SYMBOLS = { + "//python/cc:py_extension.bzl": [ + # keep sorted + "py_extension", + ], "//python:py_info.bzl": [ # keep sorted "PyInfo", From b84447fed5da40853fcf09d4b976b1647d5da44d Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Tue, 4 Aug 2026 21:21:26 -0700 Subject: [PATCH 899/922] agents: update create-pr skill with interactive artifact requirements (#4004) Update the create-pr skill to specify interactive user-facing artifact requirements and explicit user decision options when proposing PRs. Previously, the create-pr skill instructions lacked explicit requirements for user-facing markdown artifacts with interactive feedback enabled, as well as defined options for user decision workflows. To address this, the SKILL.md documentation is updated to require user-facing artifacts with interactive feedback enabled and to outline the four decision choices for user review. --- .agents/skills/create-pr/SKILL.md | 10 ++++++++-- 1 file changed, 8 insertions(+), 2 deletions(-) diff --git a/.agents/skills/create-pr/SKILL.md b/.agents/skills/create-pr/SKILL.md index 3510b15d07..88b687b39f 100644 --- a/.agents/skills/create-pr/SKILL.md +++ b/.agents/skills/create-pr/SKILL.md @@ -28,8 +28,14 @@ to handle PR creation or description drafting. (GitHub uses the PR description as the commit message upon merge, which reflows text at 72 columns). - **Formatting**: Follow repository style guidelines and structure. - - Create a Markdown artifact (`pr_info.md`) containing the PR title, body, - and link/metadata so the user can review and comment on it. + - Create a Markdown artifact (`pr_info.md`) meeting the following requirements: + - **User-facing**: Published so it is presented directly in the user interface. + - **Interactive feedback enabled**: Allows the user to select lines and leave inline comments on the draft. + - **User decision choices**: Ask the user if they want to: + 1. Create a regular PR + 2. Create a draft PR + 3. Provide feedback on the draft text + 4. Discard the draft text - **Propose vs. Create**: If the user requested to propose or draft a PR description, **do not** run `gh pr create`—just create the `pr_info.md` artifact for the user to review. Otherwise, execute `gh pr create` with From a2bdf6bebede105a7e4ea9aea5732c7866a17312 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Wed, 5 Aug 2026 03:10:24 -0700 Subject: [PATCH 900/922] docs(howto): add optional feature loading guide (#4003) Macro and rule authors need clear guidance on writing Bazel code that remains compatible across multiple rules_python releases as new APIs are introduced. Previously, using features.loadable_symbols, targets, and version lacked dedicated documentation, making it harder for developers to know how to use these feature-detection APIs. Add a how-to guide (docs/howto/optional-feature-loading.md) showing how to inspect //python:features.bzl and generate compatibility repo rules, BUILD files, and macro version checks. --- docs/howto/optional-feature-loading.md | 180 +++++++++++++++++++++++++ 1 file changed, 180 insertions(+) create mode 100644 docs/howto/optional-feature-loading.md diff --git a/docs/howto/optional-feature-loading.md b/docs/howto/optional-feature-loading.md new file mode 100644 index 0000000000..144c0f65cb --- /dev/null +++ b/docs/howto/optional-feature-loading.md @@ -0,0 +1,180 @@ +:::{default-domain} bzl +::: + +# How to optionally load symbols using `features.loadable_symbols` + +When writing Bazel rules, macros, or repository extensions that support +multiple versions of `rules_python`, you may want to detect whether a public +symbol (such as {obj}`py_extension` in `//python/cc:py_extension.bzl`) is +available before attempting to load or use it. + +Because Starlark `load()` statements are evaluated at parse time and must be at +the top level of a `.bzl` file, unconditionally loading a symbol that does not +exist in older versions of `rules_python` will cause a build error. + +The {bzl:obj}`features.loadable_symbols` dictionary in `//python:features.bzl` +allows you to programmatically inspect which symbols are exported by `.bzl` +files in the current `rules_python` version. + +## The `features.loadable_symbols` structure + +{bzl:obj}`features.loadable_symbols` is a `dict[str, list[str]]` mapping label +strings of `.bzl` files to the list of public symbols they export: + +```starlark +load("@rules_python//python:features.bzl", "features") + +# Example structure of features.loadable_symbols: +# { +# "//python/cc:py_extension.bzl": [ +# "py_extension", +# ], +# "//python:py_info.bzl": [ +# "PyInfo", +# ], +# } +``` + +## Using load() with optional symbols + +In repository rules or Bazel module extensions (`repository_ctx` or +`module_ctx`), you generate `.bzl` files dynamically. You can inspect +`features.loadable_symbols` to determine which `load()` statements to write into +a generated compatibility repository. + +Re-export the symbol under its standard name if available, or set it to `None` +if it is absent. By generating compatibility files and empty `BUILD.bazel` +files at the exact same relative package paths as `rules_python`, the only +difference in downstream `load()` statements is the repository name: + +```starlark +load("@rules_python//python:features.bzl", "features") + +def _rules_python_compat_impl(rctx): + for bzl, symbol_list in rctx.attr.symbols.items(): + loadable = features.loadable_symbols.get(bzl, []) + lines = [] + for symbol in symbol_list: + if symbol in loadable: + lines.append( + 'load("{}", _{} = "{}")'.format(bzl, symbol, symbol), + ) + lines.append("{} = _{}".format(symbol, symbol)) + else: + lines.append("{} = None".format(symbol)) + + package, _, filename = bzl.lstrip("/").partition(":") + path = package + "/" + filename if package else filename + build_path = package + "/BUILD.bazel" if package else "BUILD.bazel" + + rctx.file(path, content = "\n".join(lines) + "\n") + rctx.file(build_path, content = "") + +rules_python_compat = repository_rule( + implementation = _rules_python_compat_impl, + attrs = { + "symbols": attr.string_list_dict( + mandatory = True, + doc = "Map of bzl paths to lists of symbols to optionally load", + ), + }, +) +``` + +Instantiate the repository rule by providing a mapping of `.bzl` paths to their +symbols of interest: + +```starlark +rules_python_compat( + name = "rules_python_compat", + symbols = { + "//python/cc:py_extension.bzl": ["py_extension"], + }, +) +``` + +### Using the generated compatibility files + +Your macros and rules can load from `@rules_python_compat` using the same +file path as `@rules_python`, testing whether the symbol is `None` before +using it: + +```starlark +load("@rules_python_compat//python/cc:py_extension.bzl", "py_extension") + +def my_macro(name, **kwargs): + if py_extension != None: + py_extension( + name = name + "_ext", + **kwargs + ) + else: + # Fall back to default behavior for older rules_python versions + pass +``` + +## Handling optional targets + +In addition to symbol loading, you may need to check whether a specific Bazel +target exists in `rules_python` before referencing its label in dependencies, +toolchains, or attribute defaults. + +The {bzl:obj}`features.targets` dictionary in `//python:features.bzl` is a +`dict[str, bool]` mapping public API target labels to `True` when available. + +In a macro: + +```starlark +load("@rules_python//python:features.bzl", "features") + +def my_cc_extension_macro(name, deps = [], **kwargs): + if features.targets.get("//python/cc:current_py_cc_headers_abi3"): + deps = deps + ["@rules_python//python/cc:current_py_cc_headers_abi3"] + + # ... define target with deps +``` + +In a `BUILD` file: + +```starlark +load("@rules_python//python:features.bzl", "features") +load("@rules_python//python:py_library.bzl", "py_library") + +py_library( + name = "my_lib", + srcs = ["my_lib.py"], + deps = [ + "//my/app:base_lib", + ] + ( + ["@rules_python//python/cc:current_py_cc_headers_abi3"] + if features.targets.get("//python/cc:current_py_cc_headers_abi3") + else [] + ), +) +``` + +## Checking versions with `features.version` + +When a behavioral change or capability is not directly reflected by a public +target or loadable symbol, you can inspect {bzl:obj}`features.version` in +`//python:features.bzl`. + +{bzl:obj}`features.version` returns a semver-formatted version string (such as +`"1.0.0"`, `"2.0.0-rc2"`, or `""` for unreleased development builds): + +```starlark +load("@rules_python//python:features.bzl", "features") + +def _to_tuple(v): + return tuple([ + int(x) if x.isdigit() else x + for x in v.replace("-", ".").split(".") + ]) + +def has_foo(): + # If version is empty, it is an unreleased build from main which includes + # all features. + if not features.version: + return True + return _to_tuple(features.version) >= _to_tuple("0.38.0") +``` From 20773d6e46dae6e3ffea736dbbbcc251bd03dd63 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Wed, 5 Aug 2026 20:48:13 -0700 Subject: [PATCH 901/922] fix(sphinxdocs): materialize source symlinks to resolve relative cross-references (#3977) (#4005) In Sphinx with MyST, relative cross-references (`myst.xref_missing`) break when resolving relative paths across symlinked source files inside Bazel's `_sources` directory because Bazel symlinks resolve canonical paths that escape the source tree. To fix this, replaced input symlinking in `_relocate` with physical file materialization in `sphinx_build.py`. Implemented `DirectorySyncer` to concurrently synchronize files into a work-private `{srcdir}.worker-in.d` directory using SHA-based change detection for worker requests and full recursive copying for non-worker requests. Fixes #3977 --- sphinxdocs/sphinxdocs/private/BUILD.bazel | 6 + sphinxdocs/sphinxdocs/private/sphinx.bzl | 7 +- sphinxdocs/sphinxdocs/private/sphinx_build.py | 224 +++++++++++++++--- sphinxdocs/tests/sphinx_build/BUILD.bazel | 11 + .../sphinx_build/directory_syncer_test.py | 133 +++++++++++ sphinxdocs/tests/sphinx_docs/BUILD.bazel | 18 ++ sphinxdocs/tests/sphinx_docs/defs.bzl | 4 +- sphinxdocs/tests/sphinx_docs/index.md | 1 + .../sphinx_docs/sphinx_docs_output_test.py | 28 +++ .../sphinx_stardoc/sphinx_output_test.py | 6 +- sphinxdocs/tests/sphinx_stardoc/xrefs.md | 1 + 11 files changed, 407 insertions(+), 32 deletions(-) create mode 100644 sphinxdocs/tests/sphinx_build/BUILD.bazel create mode 100644 sphinxdocs/tests/sphinx_build/directory_syncer_test.py create mode 100644 sphinxdocs/tests/sphinx_docs/sphinx_docs_output_test.py diff --git a/sphinxdocs/sphinxdocs/private/BUILD.bazel b/sphinxdocs/sphinxdocs/private/BUILD.bazel index fa5ded15f1..823f07fe73 100644 --- a/sphinxdocs/sphinxdocs/private/BUILD.bazel +++ b/sphinxdocs/sphinxdocs/private/BUILD.bazel @@ -129,6 +129,12 @@ py_binary( deps = [":proto_to_markdown_lib"], ) +py_library( + name = "sphinx_build_lib", + srcs = ["sphinx_build.py"], + visibility = ["//:__subpackages__"], +) + py_library( name = "proto_to_markdown_lib", srcs = ["proto_to_markdown.py"], diff --git a/sphinxdocs/sphinxdocs/private/sphinx.bzl b/sphinxdocs/sphinxdocs/private/sphinx.bzl index b7c051a154..c42b42ebc6 100644 --- a/sphinxdocs/sphinxdocs/private/sphinx.bzl +++ b/sphinxdocs/sphinxdocs/private/sphinx.bzl @@ -360,12 +360,17 @@ def _sphinx_source_tree_impl(ctx): dest_path = paths.join(source_prefix, dest_path) if source_file.is_directory: dest_file = ctx.actions.declare_directory(dest_path) + progress_message = "Symlinking Sphinx source directory %{input} to %{output}" else: dest_file = ctx.actions.declare_file(dest_path) + progress_message = "Symlinking Sphinx source %{input} to %{output}" + + # NOTE: Sphinx/MyST will read through symlinks, which can break relative + # xref lookup. Files are copied during the action phase to prevent this. ctx.actions.symlink( output = dest_file, target_file = source_file, - progress_message = "Symlinking Sphinx source %{input} to %{output}", + progress_message = progress_message, ) sphinx_source_files.append(dest_file) return dest_file diff --git a/sphinxdocs/sphinxdocs/private/sphinx_build.py b/sphinxdocs/sphinxdocs/private/sphinx_build.py index 8b0eec8fd3..52a334d9b9 100644 --- a/sphinxdocs/sphinxdocs/private/sphinx_build.py +++ b/sphinxdocs/sphinxdocs/private/sphinx_build.py @@ -1,10 +1,14 @@ +import concurrent.futures import contextlib import io import json import logging import os +import pathlib import shutil +import stat import sys +import threading import traceback import typing @@ -29,7 +33,171 @@ def __init__(self, message, exit_code): _REQUEST_INFO_CONFIG_NAME = "bazel_worker_request_info_path" +class DirectorySyncerError(Exception): + """Raised when one or more errors occur during directory synchronization.""" + + def __init__(self, errors: typing.List[BaseException]): + self.errors = errors + message = f"Encountered {len(errors)} error(s) during sync:\n" + "\n".join( + f" - {e}" for e in errors + ) + super().__init__(message) + + +class DirectorySyncer: + """Synchronizes a working destination directory from a source directory. + + Supports concurrent SHA-aware incremental updates (via sync()) for worker + mode and concurrent full directory copying (via copytree()) for non-worker + mode, ensuring physical file materialization to prevent relative + cross-reference resolution failures. + """ + + def __init__( + self, + srcdir: pathlib.Path, + destdir: pathlib.Path, + max_workers: typing.Optional[int] = None, + ): + self._srcdir = srcdir + self._destdir = destdir + self._max_workers = max_workers or min(32, (os.cpu_count() or 4) + 4) + self._current_shas: typing.Dict[str, str] = {} + self._lock = threading.Lock() + self._finished_cond = threading.Condition(self._lock) + self._remaining = 0 + self._errors: typing.List[BaseException] = [] + self._executor: typing.Optional[concurrent.futures.ThreadPoolExecutor] = None + + def _reset_state(self) -> None: + with self._lock: + self._errors.clear() + self._remaining = 0 + + def _wait_for_completion(self) -> None: + with self._lock: + while self._remaining > 0: + self._finished_cond.wait() + if self._errors: + raise DirectorySyncerError(list(self._errors)) + + def _submit_task(self, fn, *args) -> None: + with self._lock: + self._remaining += 1 + future = self._executor.submit(fn, *args) + future.add_done_callback(self._handle_task_done) + + def _handle_task_done(self, future: concurrent.futures.Future) -> None: + exc = future.exception() + if exc: + with self._lock: + self._errors.append(exc) + + def _task_finished(self) -> None: + with self._lock: + self._remaining -= 1 + if self._remaining == 0: + self._finished_cond.notify_all() + + @contextlib.contextmanager + def _create_executor(self): + with concurrent.futures.ThreadPoolExecutor( + max_workers=self._max_workers + ) as executor: + self._executor = executor + try: + yield + finally: + self._executor = None + + def copytree(self) -> None: + """Concurrently copies srcdir to destdir without SHA tracking.""" + self._reset_state() + shutil.rmtree(self._destdir, ignore_errors=True) + with self._create_executor(): + self._submit_task(self._copy_dir, self._srcdir, self._destdir) + self._wait_for_completion() + + def sync(self, entries: typing.Dict[str, str]) -> None: + """Synchronizes destdir to match entries {relative_path: sha} concurrently.""" + self._reset_state() + + to_remove = set(self._current_shas.keys()) - set(entries.keys()) + to_copy = { + path: sha + for path, sha in entries.items() + if self._current_shas.get(path) != sha + } + + if not to_remove and not to_copy: + self._current_shas = dict(entries) + return + + with self._create_executor(): + # 1. Submit stale path removals concurrently ASAP + for rel_path in to_remove: + dest_path = self._destdir / rel_path + self._submit_task(self._remove_path, dest_path) + + # 2. Submit created/updated item copies concurrently ASAP + for rel_path in to_copy: + src_path = self._srcdir / rel_path + dest_path = self._destdir / rel_path + if src_path.is_dir(): + self._submit_task(self._copy_dir, src_path, dest_path) + else: + self._submit_task(self._copy_file, src_path, dest_path) + + self._wait_for_completion() + + self._current_shas = dict(entries) + + def _remove_path(self, dest_path: pathlib.Path) -> None: + try: + if dest_path.is_dir() and not dest_path.is_symlink(): + shutil.rmtree(dest_path) + else: + dest_path.unlink(missing_ok=True) + except BaseException as e: + e.add_note(f"Failed removing path: dest_path={dest_path}") + raise + finally: + self._task_finished() + + def _copy_file(self, src: pathlib.Path, dest: pathlib.Path) -> None: + try: + dest.parent.mkdir(parents=True, exist_ok=True) + shutil.copy2(src, dest) + mode = dest.stat().st_mode + dest.chmod(mode | stat.S_IWUSR) + except BaseException as e: + e.add_note(f"Failed copying file: src={src}, dest={dest}") + raise + finally: + self._task_finished() + + def _copy_dir(self, src: pathlib.Path, dest: pathlib.Path) -> None: + """Recursively creates destination directory and submits tasks for its entries.""" + try: + dest.mkdir(parents=True, exist_ok=True) + with os.scandir(src) as scanner: + for entry in scanner: + c_dest = dest / entry.name + c_src = pathlib.Path(entry.path) + if entry.is_dir(): + self._submit_task(self._copy_dir, c_src, c_dest) + else: + self._submit_task(self._copy_file, c_src, c_dest) + except BaseException as e: + e.add_note(f"Failed copying directory: src={src}, dest={dest}") + raise + finally: + self._task_finished() + + class Worker: + """A Bazel persistent worker for Sphinx builds.""" + def __init__( self, instream: "typing.TextIO", outstream: "typing.TextIO", exec_root: str ): @@ -51,6 +219,7 @@ def __init__( # dict[str srcdir, dict[str path, str digest]] self._digests = {} + self._syncers: typing.Dict[pathlib.Path, DirectorySyncer] = {} # Internal output directories the worker gives to Sphinx that need # to be cleaned up upon exit. @@ -124,18 +293,25 @@ def _send_response(self, response: "WorkResponse") -> None: def _prepare_sphinx(self, request): sphinx_args = request["arguments"] - srcdir = sphinx_args[0] + srcdir = pathlib.Path(sphinx_args[0]) + destdir = pathlib.Path(f"{srcdir}.worker-in.d") incoming_digests = {} - current_digests = self._digests.setdefault(srcdir, {}) + current_digests = self._digests.setdefault(str(srcdir), {}) is_first_request = not current_digests changed_paths = [] request_info = {"exec_root": self._exec_root, "inputs": request["inputs"]} + srcdir_prefix = str(srcdir) + "/" for entry in request["inputs"]: path = entry["path"] + # In persistent worker mode, request["inputs"] includes action-level + # tools (e.g. sphinx-build, sphinx_build.py) and params files that + # live outside srcdir. Only synchronize documentation sources inside srcdir. + if not path.startswith(srcdir_prefix): + continue digest = entry["digest"] # Make the path srcdir-relative so Sphinx understands it. - path = path.removeprefix(srcdir + "/") + path = path.removeprefix(srcdir_prefix) incoming_digests[path] = digest if path not in current_digests: @@ -145,31 +321,7 @@ def _prepare_sphinx(self, request): logger.info("path %s changed", path) changed_paths.append(path) - # Remove any source files that were tracked in the previous request (`current_digests`) - # but are missing from the current `request["inputs"]` (`incoming_digests`). - # Across incremental branch switches or file removals, if these stale symlinks - # remain in `srcdir` on disk, Sphinx will discover broken/unreadable files during - # `find_files()` and abort with "WARNING: Ignored unreadable document" (fatal with -W). - for path in set(current_digests) - set(incoming_digests): - removed_path = os.path.join(srcdir, path) - if os.path.exists(removed_path) or os.path.islink(removed_path): - logger.info("removing stale source file %s", removed_path) - try: - if os.path.islink(removed_path): - try: - os.remove(removed_path) - except OSError: - os.rmdir(removed_path) - elif os.path.isdir(removed_path): - shutil.rmtree(removed_path) - else: - os.remove(removed_path) - except OSError as e: - logger.warning( - "failed to remove stale source %s: %s", removed_path, e - ) - - self._digests[srcdir] = incoming_digests + self._digests[str(srcdir)] = incoming_digests self._extension.changed_paths = changed_paths request_info["changed_sources"] = changed_paths @@ -184,13 +336,22 @@ def _prepare_sphinx(self, request): # failures. So on the first request start from a clean slate. if is_first_request: shutil.rmtree(worker_outdir, ignore_errors=True) + shutil.rmtree(destdir, ignore_errors=True) for arg in sphinx_args: if arg.startswith("--doctree-dir="): shutil.rmtree(arg.partition("=")[2], ignore_errors=True) self._worker_outdirs.add(worker_outdir) sphinx_args[1] = worker_outdir - request_info_path = os.path.join(srcdir, "_bazel_worker_request_info.json") + if srcdir not in self._syncers: + self._syncers[srcdir] = DirectorySyncer(srcdir, destdir) + syncer = self._syncers[srcdir] + syncer.sync(incoming_digests) + + sphinx_args[0] = str(destdir) + request_info_path = os.path.join( + sphinx_args[0], "_bazel_worker_request_info.json" + ) with open(request_info_path, "w") as fp: json.dump(request_info, fp) sphinx_args.append(f"--define={_REQUEST_INFO_CONFIG_NAME}={request_info_path}") @@ -341,6 +502,11 @@ def _non_worker_main(): args.extend(lines) else: args.append(arg) + if len(args) > 1: + srcdir = pathlib.Path(args[1]) + destdir = pathlib.Path(f"{srcdir}.worker-in.d") + DirectorySyncer(srcdir, destdir).copytree() + args[1] = str(destdir) sys.argv[:] = args return main() diff --git a/sphinxdocs/tests/sphinx_build/BUILD.bazel b/sphinxdocs/tests/sphinx_build/BUILD.bazel new file mode 100644 index 0000000000..b9e77220df --- /dev/null +++ b/sphinxdocs/tests/sphinx_build/BUILD.bazel @@ -0,0 +1,11 @@ +load("@rules_python//python:py_test.bzl", "py_test") + +py_test( + name = "directory_syncer_test", + srcs = ["directory_syncer_test.py"], + deps = [ + "//sphinxdocs/private:sphinx_build_lib", + "@dev_pip//absl_py", + "@dev_pip//sphinx", + ], +) diff --git a/sphinxdocs/tests/sphinx_build/directory_syncer_test.py b/sphinxdocs/tests/sphinx_build/directory_syncer_test.py new file mode 100644 index 0000000000..1365e2b3ce --- /dev/null +++ b/sphinxdocs/tests/sphinx_build/directory_syncer_test.py @@ -0,0 +1,133 @@ +import pathlib +import shutil +import stat +import tempfile + +from absl.testing import absltest +from sphinxdocs.private.sphinx_build import DirectorySyncer, DirectorySyncerError + + +class DirectorySyncerTest(absltest.TestCase): + def setUp(self): + super().setUp() + self.test_dir = pathlib.Path(tempfile.mkdtemp()) + self.addCleanup(shutil.rmtree, self.test_dir, ignore_errors=True) + self.srcdir = self.test_dir / "src" + self.destdir = self.test_dir / "dest" + self.srcdir.mkdir() + + def _write_src(self, rel_path, content, mode=None): + path = self.srcdir / rel_path + path.parent.mkdir(parents=True, exist_ok=True) + if path.exists(): + path.chmod(stat.S_IRUSR | stat.S_IWUSR | stat.S_IRGRP | stat.S_IROTH) + path.write_text(content) + if mode is not None: + path.chmod(mode) + return path + + def assert_dest_equals(self, rel_path, expected_content): + dest_file = self.destdir / rel_path + self.assertTrue(dest_file.exists(), f"Expected {dest_file} to exist") + self.assertEqual(expected_content, dest_file.read_text()) + + def assert_dest_not_exists(self, rel_path): + dest_file = self.destdir / rel_path + self.assertFalse(dest_file.exists(), f"Expected {dest_file} to not exist") + + def test_copytree(self): + self._write_src("file1.txt", "hello") + self._write_src("sub/file2.txt", "world") + + syncer = DirectorySyncer(self.srcdir, self.destdir) + syncer.copytree() + + self.assert_dest_equals("file1.txt", "hello") + self.assert_dest_equals("sub/file2.txt", "world") + + def test_sync_initial_and_incremental(self): + self._write_src("doc1.md", "v1") + self._write_src("doc2.md", "v1") + self._write_src("doc3.md", "v1") + + syncer = DirectorySyncer(self.srcdir, self.destdir) + # Initial sync + syncer.sync( + { + "doc1.md": "sha-doc1-v1", + "doc2.md": "sha-doc2-v1", + "doc3.md": "sha-doc3-v1", + } + ) + + self.assert_dest_equals("doc1.md", "v1") + self.assert_dest_equals("doc2.md", "v1") + self.assert_dest_equals("doc3.md", "v1") + + # Incremental sync: + # - doc1.md: unchanged SHA + # - doc2.md: updated SHA & content + # - doc3.md: removed from entries + # - doc4.md: newly created file + self._write_src("doc2.md", "v2") + self._write_src("doc4.md", "v1") + + syncer.sync( + { + "doc1.md": "sha-doc1-v1", + "doc2.md": "sha-doc2-v2", + "doc4.md": "sha-doc4-v1", + } + ) + + self.assert_dest_equals("doc1.md", "v1") + self.assert_dest_equals("doc2.md", "v2") + self.assert_dest_not_exists("doc3.md") + self.assert_dest_equals("doc4.md", "v1") + + def test_read_only_file_becomes_writable(self): + read_only_mode = stat.S_IRUSR | stat.S_IRGRP | stat.S_IROTH + self._write_src("readonly.txt", "version 1", mode=read_only_mode) + syncer = DirectorySyncer(self.srcdir, self.destdir) + syncer.sync({"readonly.txt": "sha-v1"}) + + dest_file = self.destdir / "readonly.txt" + self.assert_dest_equals("readonly.txt", "version 1") + self.assertTrue( + dest_file.stat().st_mode & stat.S_IWUSR, + "Destination file should be writable", + ) + + # Ensure subsequent incremental updates can overwrite the file without permission error + self._write_src("readonly.txt", "version 2", mode=read_only_mode) + syncer.sync({"readonly.txt": "sha-v2"}) + self.assert_dest_equals("readonly.txt", "version 2") + + def test_sync_directory_artifact(self): + dir_artifact = self.srcdir / "tree_art" + dir_artifact.mkdir() + (dir_artifact / "page.md").write_text("content") + + syncer = DirectorySyncer(self.srcdir, self.destdir) + syncer.sync( + { + "tree_art": "sha-dir-art", + } + ) + + self.assert_dest_equals("tree_art/page.md", "content") + + def test_errors_bubble_up(self): + syncer = DirectorySyncer(self.srcdir, self.destdir) + with self.assertRaises(DirectorySyncerError) as cm: + syncer.sync( + { + "non_existent_1.txt": "sha1", + "non_existent_2.txt": "sha2", + } + ) + self.assertGreaterEqual(len(cm.exception.errors), 2) + + +if __name__ == "__main__": + absltest.main() diff --git a/sphinxdocs/tests/sphinx_docs/BUILD.bazel b/sphinxdocs/tests/sphinx_docs/BUILD.bazel index 33b98ec585..4bbaf90691 100644 --- a/sphinxdocs/tests/sphinx_docs/BUILD.bazel +++ b/sphinxdocs/tests/sphinx_docs/BUILD.bazel @@ -1,4 +1,5 @@ load("@bazel_skylib//rules:build_test.bzl", "build_test") +load("@rules_python//python:py_test.bzl", "py_test") load("//sphinxdocs:sphinx.bzl", "sphinx_build_binary", "sphinx_docs") load(":defs.bzl", "gen_directory") @@ -21,10 +22,20 @@ sphinx_docs( ], config = "conf.py", formats = ["html"], + renamed_srcs = { + ":gen_binary_asset": "binary_asset.bin", + }, sphinx = ":sphinx-build", + strip_prefix = package_name() + "/", target_compatible_with = _TARGET_COMPATIBLE_WITH, ) +genrule( + name = "gen_binary_asset", + outs = ["binary_asset.bin"], + cmd = "printf '\\x00\\xff\\xfe\\xfd\\x80\\x90\\x00\\x01\\x02\\x03\\x04' > $@", +) + gen_directory( name = "generated_directory", ) @@ -42,3 +53,10 @@ build_test( name = "docs_build_test", targets = [":docs"], ) + +py_test( + name = "sphinx_docs_output_test", + srcs = ["sphinx_docs_output_test.py"], + data = [":docs"], + deps = ["@dev_pip//absl_py"], +) diff --git a/sphinxdocs/tests/sphinx_docs/defs.bzl b/sphinxdocs/tests/sphinx_docs/defs.bzl index 2e47ecc0f7..36fd1dba4f 100644 --- a/sphinxdocs/tests/sphinx_docs/defs.bzl +++ b/sphinxdocs/tests/sphinx_docs/defs.bzl @@ -6,7 +6,9 @@ def _gen_directory_impl(ctx): ctx.actions.run_shell( outputs = [out], command = """ -echo "# Hello" > {outdir}/index.md +printf '# Hello\\n' > {outdir}/index.md +printf '# Dir Page 1\\n\\n[Dir Page 2](dir_page2.md)\\n' > {outdir}/dir_page1.md +printf '# Dir Page 2\\n' > {outdir}/dir_page2.md """.format( outdir = out.path, ), diff --git a/sphinxdocs/tests/sphinx_docs/index.md b/sphinxdocs/tests/sphinx_docs/index.md index cdce641fa1..68a5fb38c3 100644 --- a/sphinxdocs/tests/sphinx_docs/index.md +++ b/sphinxdocs/tests/sphinx_docs/index.md @@ -3,6 +3,7 @@ :::{toctree} :glob: +generated_directory/dir_page1 ** genindex ::: diff --git a/sphinxdocs/tests/sphinx_docs/sphinx_docs_output_test.py b/sphinxdocs/tests/sphinx_docs/sphinx_docs_output_test.py new file mode 100644 index 0000000000..5d00817926 --- /dev/null +++ b/sphinxdocs/tests/sphinx_docs/sphinx_docs_output_test.py @@ -0,0 +1,28 @@ +import importlib.resources +import os +from xml.etree import ElementTree + +import tests.sphinx_docs as sphinx_docs +from absl.testing import absltest + + +class SphinxDocsOutputTest(absltest.TestCase): + def test_directory_artifact_relative_xref(self): + page1_path = importlib.resources.files(sphinx_docs).joinpath( + "docs/_build/html/generated_directory/dir_page1.html" + ) + self.assertTrue(os.path.exists(str(page1_path)), f"Not found at {page1_path}") + with open(str(page1_path)) as f: + xml = f.read() + doc_elem = ElementTree.fromstring(xml) + actual = None + for elem in doc_elem.iter(): + if "href" in elem.attrib: + if "".join(elem.itertext()).strip() == "Dir Page 2": + actual = elem.attrib["href"] + break + self.assertEqual("dir_page2.html", actual) + + +if __name__ == "__main__": + absltest.main() diff --git a/sphinxdocs/tests/sphinx_stardoc/sphinx_output_test.py b/sphinxdocs/tests/sphinx_stardoc/sphinx_output_test.py index 650e0134d3..e6653b1656 100644 --- a/sphinxdocs/tests/sphinx_stardoc/sphinx_output_test.py +++ b/sphinxdocs/tests/sphinx_stardoc/sphinx_output_test.py @@ -13,7 +13,10 @@ def setUp(self): self._xmls = {} def assert_xref(self, doc, *, text, href): - match = self._doc_element(doc).find(f".//*[.='{text}']") + # Find an element with an 'href' attribute whose string content (including + # descendants like ) equals `text`. [@href] filters out ancestor + # elements like
  • or

    which also match [.='{text}']. + match = self._doc_element(doc).find(f".//*[@href][.='{text}']") if not match: self.fail(f"No element found with {text=}") actual = match.attrib.get("href", "") @@ -113,6 +116,7 @@ def _doc_element(self, doc): ("file_with_repo", "@testrepo//lang:rule.bzl", "rule.html"), ("package_absolute", "//lang", "target.html"), ("package_basename", "lang", "target.html"), + ("relative_doc_link", "Rule documentation", "rule.html"), # fmt: on ) def test_xrefs(self, text, href): diff --git a/sphinxdocs/tests/sphinx_stardoc/xrefs.md b/sphinxdocs/tests/sphinx_stardoc/xrefs.md index 9893c32023..e3a7d7b21e 100644 --- a/sphinxdocs/tests/sphinx_stardoc/xrefs.md +++ b/sphinxdocs/tests/sphinx_stardoc/xrefs.md @@ -7,6 +7,7 @@ Various tests of cross referencing support ## Short name +* [Rule documentation](rule.md) * function: {obj}`myfunc` * function arg: {obj}`myfunc.arg1` * rule: {obj}`my_rule` From 71a7e319c35296e11d3d80415c257cc2c7ec2507 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Thu, 6 Aug 2026 00:55:21 -0700 Subject: [PATCH 902/922] fix(sphinxdocs): fix path resolution for conf.py in separate directory (#4006) Sphinx documentation builds failed when conf.py was generated or located in a separate directory because _relocate used the relative short_path of config, setting sphinx_source_dir_path to the config subdirectory rather than the source tree root. Relocate ctx.file.config using paths.basename(ctx.file.config.path) so it is always placed at the root of the generated _sources directory. Work towards #3977. Fixes #3999 --- sphinxdocs/sphinxdocs/private/sphinx.bzl | 19 ++++++---- .../sphinx_docs_conf_in_other_dir/BUILD.bazel | 38 +++++++++++++++++++ .../sphinx_docs_conf_in_other_dir/index.md | 3 ++ .../sphinx_docs_conf_in_other_dir/src_conf.py | 4 ++ 4 files changed, 57 insertions(+), 7 deletions(-) create mode 100644 sphinxdocs/tests/sphinx_docs_conf_in_other_dir/BUILD.bazel create mode 100644 sphinxdocs/tests/sphinx_docs_conf_in_other_dir/index.md create mode 100644 sphinxdocs/tests/sphinx_docs_conf_in_other_dir/src_conf.py diff --git a/sphinxdocs/sphinxdocs/private/sphinx.bzl b/sphinxdocs/sphinxdocs/private/sphinx.bzl index c42b42ebc6..24c5e9f9d1 100644 --- a/sphinxdocs/sphinxdocs/private/sphinx.bzl +++ b/sphinxdocs/sphinxdocs/private/sphinx.bzl @@ -100,7 +100,7 @@ def sphinx_docs( sphinx, config, formats, - strip_prefix = "", + strip_prefix = None, extra_opts = [], tools = [], allow_persistent_workers = True, @@ -133,10 +133,10 @@ def sphinx_docs( config: {type}`label` the Sphinx config file (`conf.py`) to use. formats: (list of str) the formats (`-b` flag) to generate documentation in. Each format will become an output group. - strip_prefix: {type}`str` A prefix to remove from the file paths of the - source files. e.g., given `//sphinxdocs/docs:foo.md`, stripping `docs/` makes - Sphinx see `foo.md` in its generated source directory. If not - specified, then {any}`native.package_name` is used. + strip_prefix: {type}`str | None` A prefix to remove from the file paths of the + source files. An empty string (`""`) means no prefix stripping; `None` + means {any}`native.package_name` is used. e.g., given `//docs:foo.md`, + stripping `docs/` makes Sphinx see `foo.md` in its generated source directory. extra_opts: {type}`list[str]` Additional options to pass onto Sphinx building. On each provided option, a location expansion is performed. See {any}`ctx.expand_location`. @@ -148,6 +148,7 @@ def sphinx_docs( This can improve incremental building of docs. **kwargs: {type}`dict` Common attributes to pass onto rules. """ + strip_prefix = strip_prefix if strip_prefix != None else native.package_name() add_tag(kwargs, "//sphinxdocs:sphinx_docs") common_kwargs = copy_propagating_kwargs(kwargs) @@ -355,7 +356,9 @@ def _sphinx_source_tree_impl(ctx): # Materialize a file under the `_sources` dir def _relocate(source_file, dest_path = None): if not dest_path: - dest_path = source_file.short_path.removeprefix(ctx.attr.strip_prefix) + # Strip leading slash if strip_prefix lacks a trailing slash, + # preventing paths.join from treating it as an absolute root. + dest_path = source_file.short_path.removeprefix(ctx.attr.strip_prefix).lstrip("/") dest_path = paths.join(source_prefix, dest_path) if source_file.is_directory: @@ -378,7 +381,9 @@ def _sphinx_source_tree_impl(ctx): # Though Sphinx has a -c flag, we move the config file into the sources # directory to make the config more intuitive because some configuration # options are relative to the config location, not the sources directory. - source_conf_file = _relocate(ctx.file.config) + # Sphinx requires the configuration file to be named conf.py: + # https://www.sphinx-doc.org/en/master/usage/configuration.html#module-conf + source_conf_file = _relocate(ctx.file.config, "conf.py") sphinx_source_dir_path = paths.dirname(source_conf_file.path) for src in ctx.attr.srcs: diff --git a/sphinxdocs/tests/sphinx_docs_conf_in_other_dir/BUILD.bazel b/sphinxdocs/tests/sphinx_docs_conf_in_other_dir/BUILD.bazel new file mode 100644 index 0000000000..eecbb90897 --- /dev/null +++ b/sphinxdocs/tests/sphinx_docs_conf_in_other_dir/BUILD.bazel @@ -0,0 +1,38 @@ +load("@bazel_skylib//rules:build_test.bzl", "build_test") +load("//sphinxdocs:sphinx.bzl", "sphinx_build_binary", "sphinx_docs") + +_TARGET_COMPATIBLE_WITH = select({ + "@platforms//os:linux": [], + "@platforms//os:macos": [], + "//conditions:default": ["@platforms//:incompatible"], +}) + +genrule( + name = "gen_conf", + srcs = ["src_conf.py"], + outs = ["other_dir/conf.py"], + cmd = "cp $(location src_conf.py) $@", +) + +sphinx_docs( + name = "docs", + srcs = ["index.md"], + config = ":gen_conf", + formats = ["html"], + sphinx = ":sphinx-build", + target_compatible_with = _TARGET_COMPATIBLE_WITH, +) + +sphinx_build_binary( + name = "sphinx-build", + tags = ["manual"], + deps = [ + "@dev_pip//myst_parser", + "@dev_pip//sphinx", + ], +) + +build_test( + name = "docs_build_test", + targets = [":docs"], +) diff --git a/sphinxdocs/tests/sphinx_docs_conf_in_other_dir/index.md b/sphinxdocs/tests/sphinx_docs_conf_in_other_dir/index.md new file mode 100644 index 0000000000..cd7d8b4188 --- /dev/null +++ b/sphinxdocs/tests/sphinx_docs_conf_in_other_dir/index.md @@ -0,0 +1,3 @@ +# Test Documentation + +This is a test document for reproducing conf.py in a separate directory. diff --git a/sphinxdocs/tests/sphinx_docs_conf_in_other_dir/src_conf.py b/sphinxdocs/tests/sphinx_docs_conf_in_other_dir/src_conf.py new file mode 100644 index 0000000000..d4f2f45d31 --- /dev/null +++ b/sphinxdocs/tests/sphinx_docs_conf_in_other_dir/src_conf.py @@ -0,0 +1,4 @@ +project = "Repro Test" +copyright = "2026, Test" +author = "Test" +extensions = ["myst_parser"] From c2b3648f9274af4aa59f138b6ec4d63be5cda548 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Thu, 6 Aug 2026 19:26:23 -0700 Subject: [PATCH 903/922] refactor(toolchain): rename abi_tag to soabi to eliminate PEP ambiguity (#4008) Rename `abi_tag` to `soabi` in `PyCcToolchainInfo` and associated toolchain implementation and macro parameters. I found the `abi_tag` term a bit ambiguous with PEP 425 wheel ABI tags and PEP 3149 C extension filename SOABI tags. So rename it to `soabi` to directly reflect the SOABI sysconfig var that it is traditionally derived from. This also uncovered a bug where the value was _just_ the Python ABI-ness (`cpython-311`), while it should be the full ABI infix that includes the platform (`cpython-311-x86_64-linux-gnu`). This also helped simplify some of the Windows code paths. --- news/3973.added.md | 4 +- python/private/cc/py_extension_rule.bzl | 26 +--------- python/private/get_local_runtime_info.py | 2 +- python/private/local_runtime_repo.bzl | 6 +-- python/private/local_runtime_repo_setup.bzl | 6 +-- python/private/py_cc_toolchain_info.bzl | 20 ++++---- python/private/py_cc_toolchain_rule.bzl | 47 +++++++++---------- .../py_cc_toolchain/py_cc_toolchain_tests.bzl | 4 +- .../support/py_cc_toolchain_info_subject.bzl | 7 +++ 9 files changed, 53 insertions(+), 69 deletions(-) diff --git a/news/3973.added.md b/news/3973.added.md index ef3796afca..20d136179a 100644 --- a/news/3973.added.md +++ b/news/3973.added.md @@ -1,6 +1,6 @@ (cc) Added experimental {obj}`py_extension` macro for creating C/C++ Python extension modules ([#3283](https://github.com/bazel-contrib/rules_python/issues/3283)). -(cc) Added `abi_flags`, `abi_tag`, `libc`, `platform_machine`, `platform_tag`, -and `sys_platform` attributes and info fields to {obj}`py_cc_toolchain` / +(cc) Added `libc`, `platform_machine`, `platform_tag`, `soabi`, and +`sys_platform` attributes and info fields to {obj}`py_cc_toolchain` / {obj}`PyCcToolchainInfo`. diff --git a/python/private/cc/py_extension_rule.bzl b/python/private/cc/py_extension_rule.bzl index dc5a5e0c59..e167969d5b 100644 --- a/python/private/cc/py_extension_rule.bzl +++ b/python/private/cc/py_extension_rule.bzl @@ -26,11 +26,9 @@ def _py_extension_wrapper_impl(ctx): else: py_toolchain = ctx.toolchains[PY_CC_TOOLCHAIN_TYPE] py_cc_toolchain = py_toolchain.py_cc_toolchain - platform_tag = _get_platform(ctx) - output_filename = "{module_name}.{abi_tag}-{platform}.{ext}".format( + output_filename = "{module_name}.{soabi}.{ext}".format( module_name = module_name, - abi_tag = py_cc_toolchain.abi_tag, - platform = platform_tag, + soabi = py_cc_toolchain.soabi, ext = ext, ) @@ -112,26 +110,6 @@ def _get_extension(ctx): """ return "pyd" if is_windows_platform(ctx) else "so" -def _get_platform(ctx): - """Derives the PEP 3149 platform tag from the active Python C++ toolchain. - - Args: - ctx: The rule context. - - Returns: - The platform tag, e.g. "x86_64-linux-gnu" or "win_amd64" - """ - py_toolchain = ctx.toolchains[PY_CC_TOOLCHAIN_TYPE] - py_cc_toolchain = py_toolchain.py_cc_toolchain - if not py_cc_toolchain.platform_tag: - fail( - ("ERROR: Unable to resolve platform_tag from Python C++ toolchain for {self}. " + - "Please ensure the active py_cc_toolchain provides a non-empty platform_tag.").format( - self = ctx.label, - ), - ) - return py_cc_toolchain.platform_tag - def _py_extension_libs_impl(ctx): py_toolchain = ctx.toolchains[PY_CC_TOOLCHAIN_TYPE] py_cc_toolchain = py_toolchain.py_cc_toolchain diff --git a/python/private/get_local_runtime_info.py b/python/private/get_local_runtime_info.py index 340ef934f7..e0eab962d6 100644 --- a/python/private/get_local_runtime_info.py +++ b/python/private/get_local_runtime_info.py @@ -248,7 +248,7 @@ def _unique_basenames(inputs: dict[str, None]) -> list[str]: "abi_dynamic_libraries": _unique_basenames(abi_dynamic_libraries), "abi_interface_libraries": _unique_basenames(abi_interface_libraries), "abi_flags": abi_flags, - "abi_tag": config_vars.get("SOABI") or "", + "soabi": config_vars.get("SOABI") or "", "shlib_suffix": ".dylib" if _IS_DARWIN else "", "additional_dlls": dlls, "defines": defines, diff --git a/python/private/local_runtime_repo.bzl b/python/private/local_runtime_repo.bzl index 48fc576bb7..f5597db613 100644 --- a/python/private/local_runtime_repo.bzl +++ b/python/private/local_runtime_repo.bzl @@ -35,7 +35,7 @@ define_local_runtime_toolchain_impl( minor = "{minor}", micro = "{micro}", abi_flags = "{abi_flags}", - abi_tag = "{abi_tag}", + soabi = "{soabi}", os = "{os}", implementation_name = "{implementation_name}", interpreter_path = "{interpreter_path}", @@ -56,7 +56,7 @@ def _expand_incompatible_template(): minor = "0", micro = "0", abi_flags = "", - abi_tag = "", + soabi = "", os = "@platforms//:incompatible", implementation_name = "incompatible", interpreter_path = "/incompatible", @@ -216,7 +216,7 @@ def _local_runtime_repo_impl(rctx): minor = info["minor"], micro = info["micro"], abi_flags = info["abi_flags"], - abi_tag = info["abi_tag"], + soabi = info["soabi"], os = "@platforms//os:{}".format(repo_utils.get_platforms_os_name(rctx)), implementation_name = info["implementation_name"], interpreter_path = _norm_path(interpreter_path), diff --git a/python/private/local_runtime_repo_setup.bzl b/python/private/local_runtime_repo_setup.bzl index 78c8bd0093..62e46e4b98 100644 --- a/python/private/local_runtime_repo_setup.bzl +++ b/python/private/local_runtime_repo_setup.bzl @@ -40,7 +40,7 @@ def define_local_runtime_toolchain_impl( additional_dlls, sys_platform = "", platform_machine = "", - abi_tag = ""): + soabi = ""): """Defines a toolchain implementation for a local Python runtime. Generates public targets: @@ -76,7 +76,7 @@ def define_local_runtime_toolchain_impl( e.g. ["lib/msvcrt123.dll"] sys_platform: `str` The PEP 508 `sys_platform` marker, e.g. 'linux', 'darwin', 'win32'. platform_machine: `str` The PEP 508 `platform_machine` marker, e.g. 'x86_64', 'aarch64'. - abi_tag: `str` The ABI tag for extension modules, e.g. 'cpython-311'. + soabi: `str` The SOABI tag for extension modules, e.g. 'cpython-311-x86_64-linux-gnu'. """ major_minor = "{}.{}".format(major, minor) major_minor_micro = "{}.{}".format(major_minor, micro) @@ -190,7 +190,7 @@ def define_local_runtime_toolchain_impl( py_cc_toolchain( name = "py_cc_toolchain", abi_flags = abi_flags, - abi_tag = abi_tag, + soabi = soabi, headers = ":python_headers", headers_abi3 = ":python_headers_abi3", libs = ":libpython", diff --git a/python/private/py_cc_toolchain_info.bzl b/python/private/py_cc_toolchain_info.bzl index 19dc5de482..208a9589ba 100644 --- a/python/private/py_cc_toolchain_info.bzl +++ b/python/private/py_cc_toolchain_info.bzl @@ -22,16 +22,6 @@ PyCcToolchainInfo = provider( The runtime's ABI flags, i.e. `sys.abiflags` (e.g. 't' for free-threaded builds). -:::{versionadded} VERSION_NEXT_FEATURE -::: -""", - "abi_tag": """\ -:type: str - -The ABI tag for extension modules, equivalent to the `SOABI` sysconfig var -(see [PEP 3149](https://peps.python.org/pep-3149/)), e.g. 'cpython-311' or -'cpython-313t'. - :::{versionadded} VERSION_NEXT_FEATURE ::: """, @@ -132,6 +122,16 @@ The PEP 3149 / PEP 425 platform tag for extension modules, e.g. :type: str The Python Major.Minor version. +""", + "soabi": """\ +:type: str + +The SOABI tag for extension modules (see +[PEP 3149](https://peps.python.org/pep-3149/)), e.g. +'cpython-311-x86_64-linux-gnu' or 'cp311'. + +:::{versionadded} VERSION_NEXT_FEATURE +::: """, "sys_platform": """ :type: str diff --git a/python/private/py_cc_toolchain_rule.bzl b/python/private/py_cc_toolchain_rule.bzl index a6cb29635c..02d4222fc8 100644 --- a/python/private/py_cc_toolchain_rule.bzl +++ b/python/private/py_cc_toolchain_rule.bzl @@ -88,19 +88,6 @@ def _py_cc_toolchain_impl(ctx): if ctx.attr._py_freethreaded_flag[BuildSettingInfo].value == FreeThreadedFlag.YES: abi_flags += "t" - abi_tag = ctx.attr.abi_tag - if not abi_tag: - # Derive default ABI tag: - # On POSIX: cpython-XX[t] (PEP 3149 / PEP 703) - # On Windows: cpXX[t] (PEP 3149 / PEP 703, CPython issue & commit): - # - https://peps.python.org/pep-3149/ - # - https://peps.python.org/pep-0703/ - # - https://github.com/python/cpython/issues/67169 - # - https://github.com/python/cpython/commit/03a144bb6ac3d7631a3bdb895e2a1f2d021fb08b - version_parts = ctx.attr.python_version.split(".") - prefix = "cp" if ctx.attr.sys_platform == "win32" else "cpython-" - abi_tag = "{}{}{}{}".format(prefix, version_parts[0], version_parts[1], abi_flags) - libc = ctx.attr.libc or LibcFlag.get_value(ctx) platform_tag = _get_platform_tag( @@ -109,9 +96,19 @@ def _py_cc_toolchain_impl(ctx): libc = libc, ) + soabi = ctx.attr.soabi + if not soabi: + # Derive default SOABI tag according to PEP 3149: + # On POSIX: cpython-XX[t]- + # On Windows: cpXX[t]- (CPython issue #67169) + version_parts = ctx.attr.python_version.split(".") + prefix = "cp" if ctx.attr.sys_platform == "win32" else "cpython-" + soabi = "{}{}{}{}".format(prefix, version_parts[0], version_parts[1], abi_flags) + if platform_tag: + soabi = "{}-{}".format(soabi, platform_tag) + py_cc_toolchain = PyCcToolchainInfo( abi_flags = abi_flags, - abi_tag = abi_tag, headers = struct( providers_map = { "CcInfo": ctx.attr.headers[CcInfo], @@ -123,6 +120,7 @@ def _py_cc_toolchain_impl(ctx): platform_machine = ctx.attr.platform_machine, platform_tag = platform_tag, python_version = ctx.attr.python_version, + soabi = soabi, sys_platform = ctx.attr.sys_platform, ) extra_kwargs = {} @@ -149,17 +147,6 @@ free-threaded is enabled, or `''` otherwise). ::: """, ), - "abi_tag": attr.string( - doc = """\ -The ABI tag for extension modules, equivalent to the `SOABI` sysconfig var -(see [PEP 3149](https://peps.python.org/pep-3149/)), e.g. 'cpython-311' or -'cpython-313t'. - -:::{versionadded} VERSION_NEXT_FEATURE -::: -""", - default = "", - ), "headers": attr.label( doc = ("Target that provides the Python headers. Typically this " + "is a cc_library target."), @@ -207,6 +194,16 @@ Target architecture as a PEP 508 `platform_machine` marker, e.g. 'x86_64', 'aarc doc = "The Major.minor Python version, e.g. 3.11", mandatory = True, ), + "soabi": attr.string( + doc = """\ +The SOABI tag for extension modules (see PEP 3149), e.g. +'cpython-311-x86_64-linux-gnu' or 'cp311'. + +:::{versionadded} VERSION_NEXT_FEATURE +::: +""", + default = "", + ), "sys_platform": attr.string( doc = """ Target OS as a PEP 508 `sys_platform` marker, e.g. 'linux', 'darwin', 'win32'. diff --git a/tests/cc/py_cc_toolchain/py_cc_toolchain_tests.bzl b/tests/cc/py_cc_toolchain/py_cc_toolchain_tests.bzl index 5b57810d0d..5b6f8e151d 100644 --- a/tests/cc/py_cc_toolchain/py_cc_toolchain_tests.bzl +++ b/tests/cc/py_cc_toolchain/py_cc_toolchain_tests.bzl @@ -119,10 +119,11 @@ def _test_py_cc_toolchain_impl(env, target): matching.str_matches("/libpython3."), ) - # ===== Verify PEP 508 platform markers ===== + # ===== Verify PEP 508 platform markers & SOABI ===== toolchain.sys_platform().equals("linux") toolchain.platform_machine().equals("x86_64") toolchain.platform_tag().equals("x86_64-linux-gnu") + toolchain.soabi().equals("cpython-3999-x86_64-linux-gnu") _tests.append(_test_py_cc_toolchain) @@ -148,6 +149,7 @@ def _test_custom_pep508_markers_impl(env, target): toolchain.sys_platform().equals("darwin") toolchain.platform_machine().equals("arm64") toolchain.platform_tag().equals("darwin") + toolchain.soabi().equals("cpython-311-darwin") _tests.append(_test_custom_pep508_markers) diff --git a/tests/support/py_cc_toolchain_info_subject.bzl b/tests/support/py_cc_toolchain_info_subject.bzl index fbf5a1a0ad..d6aa258a64 100644 --- a/tests/support/py_cc_toolchain_info_subject.bzl +++ b/tests/support/py_cc_toolchain_info_subject.bzl @@ -25,6 +25,7 @@ def _py_cc_toolchain_info_subject_new(info, *, meta): platform_machine = lambda *a, **k: _py_cc_toolchain_info_subject_platform_machine(self, *a, **k), platform_tag = lambda *a, **k: _py_cc_toolchain_info_subject_platform_tag(self, *a, **k), python_version = lambda *a, **k: _py_cc_toolchain_info_subject_python_version(self, *a, **k), + soabi = lambda *a, **k: _py_cc_toolchain_info_subject_soabi(self, *a, **k), sys_platform = lambda *a, **k: _py_cc_toolchain_info_subject_sys_platform(self, *a, **k), actual = info, ) @@ -82,6 +83,12 @@ def _py_cc_toolchain_info_subject_python_version(self): meta = self.meta.derive("python_version()"), ) +def _py_cc_toolchain_info_subject_soabi(self): + return subjects.str( + self.actual.soabi, + meta = self.meta.derive("soabi()"), + ) + def _py_cc_toolchain_info_subject_sys_platform(self): return subjects.str( self.actual.sys_platform, From 81b32605b2987697145c41b3fb59ca91a5b6e5a1 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Thu, 6 Aug 2026 20:24:50 -0700 Subject: [PATCH 904/922] build: add static type checking for runfiles with rules_pyrefly (#4009) Enable static type verification for the runfiles library using the rules_pyrefly ruleset. To ensure hermetic type safety across Bazel runfiles utilities, this configures rules_pyrefly's Bzlmod dev toolchain, tags the runfiles target with `tags = ["pyrefly"]`, and enables global aspect evaluation in .bazelrc via `--aspects=//tests/support/pyrefly:pyrefly.bzl%pyrefly_aspect`. To support WORKSPACE mode builds without breaking dependency resolution, a no-op stub repository for rules_pyrefly is defined under tests/modules/rules_pyrefly_stub and registered in internal_dev_deps.bzl. Under Bzlmod, the aspect performs static type checking on tagged targets, while under WORKSPACE mode, the stub aspect evaluates cleanly as a no-op. --- .agents/rules/no_gh_auth_token_in_cmdline.md | 11 ++++++++++ .bazelrc | 1 + .bazelrc.deleted_packages | 1 + MODULE.bazel | 14 +++++++++++++ internal_dev_deps.bzl | 7 +++++++ python/runfiles/BUILD.bazel | 1 + python/runfiles/runfiles.py | 2 +- tests/modules/rules_pyrefly_stub/WORKSPACE | 1 + .../rules_pyrefly_stub/pyrefly/BUILD.bazel | 3 +++ .../rules_pyrefly_stub/pyrefly/pyrefly.bzl | 21 +++++++++++++++++++ tests/support/pyrefly/BUILD.bazel | 8 +++++++ tests/support/pyrefly/pyrefly.bzl | 7 +++++++ 12 files changed, 76 insertions(+), 1 deletion(-) create mode 100644 .agents/rules/no_gh_auth_token_in_cmdline.md create mode 100644 tests/modules/rules_pyrefly_stub/WORKSPACE create mode 100644 tests/modules/rules_pyrefly_stub/pyrefly/BUILD.bazel create mode 100644 tests/modules/rules_pyrefly_stub/pyrefly/pyrefly.bzl create mode 100644 tests/support/pyrefly/BUILD.bazel create mode 100644 tests/support/pyrefly/pyrefly.bzl diff --git a/.agents/rules/no_gh_auth_token_in_cmdline.md b/.agents/rules/no_gh_auth_token_in_cmdline.md new file mode 100644 index 0000000000..e8cb7794fb --- /dev/null +++ b/.agents/rules/no_gh_auth_token_in_cmdline.md @@ -0,0 +1,11 @@ +--- +trigger: always_on +--- + +# No Auth Token in Cmdline Rule + +* NEVER pass `gh auth token` or embed authentication tokens in command line + arguments or URLs. Doing so can leak credentials in process tables (`ps`), + shell history, or logs. +* Use configured SSH keys, standard `git push`, or `gh` commands natively + instead. diff --git a/.bazelrc b/.bazelrc index 044990fdb1..90f3bc8fdb 100644 --- a/.bazelrc +++ b/.bazelrc @@ -19,6 +19,7 @@ test --test_output=errors # Python targets as required. build --incompatible_default_to_explicit_init_py build --//python/config_settings:incompatible_default_to_explicit_init_py=True +build --aspects=//tests/support/pyrefly:pyrefly.bzl%pyrefly_aspect # Ensure ongoing compatibility with this flag. common --incompatible_disallow_struct_provider_syntax diff --git a/.bazelrc.deleted_packages b/.bazelrc.deleted_packages index 9c4745c178..da79f11058 100644 --- a/.bazelrc.deleted_packages +++ b/.bazelrc.deleted_packages @@ -53,3 +53,4 @@ common --deleted_packages=tests/modules/other/nspkg_single common --deleted_packages=tests/modules/other/simple_v1 common --deleted_packages=tests/modules/other/simple_v2 common --deleted_packages=tests/modules/other/with_external_data +common --deleted_packages=tests/modules/rules_pyrefly_stub/pyrefly diff --git a/MODULE.bazel b/MODULE.bazel index 1c2042152e..027277ab64 100644 --- a/MODULE.bazel +++ b/MODULE.bazel @@ -98,6 +98,20 @@ bazel_dep(name = "another_module", version = "0", dev_dependency = True) bazel_dep(name = "rules_go", version = "0.60.0", dev_dependency = True, repo_name = "io_bazel_rules_go") bazel_dep(name = "gazelle", version = "0.40.0", dev_dependency = True, repo_name = "bazel_gazelle") bazel_dep(name = "bazel_skylib_gazelle_plugin", version = "1.8.2", dev_dependency = True) +bazel_dep(name = "rules_pyrefly", version = "0.1.0", dev_dependency = True) + +pyrefly = use_extension( + "@rules_pyrefly//pyrefly:extensions.bzl", + "pyrefly", + dev_dependency = True, +) +pyrefly.toolchain(version = "1.2.0") +use_repo(pyrefly, "pyrefly_toolchains") + +register_toolchains( + "@pyrefly_toolchains//:all", + dev_dependency = True, +) internal_dev_deps = use_extension( "//python/private:internal_dev_deps.bzl", diff --git a/internal_dev_deps.bzl b/internal_dev_deps.bzl index 7ab4717236..6d4d7656b7 100644 --- a/internal_dev_deps.bzl +++ b/internal_dev_deps.bzl @@ -107,6 +107,13 @@ def rules_python_internal_deps(): ], ) + # Stub repository for rules_pyrefly in WORKSPACE mode so that load() + # statements for @rules_pyrefly resolve without requiring full Pyrefly. + local_repository( + name = "rules_pyrefly", + path = "tests/modules/rules_pyrefly_stub", + ) + # The below two deps are required for the integration test with bazel # gazelle. Maybe the test should be moved to the `gazelle` workspace? http_archive( diff --git a/python/runfiles/BUILD.bazel b/python/runfiles/BUILD.bazel index 73663472dc..4e119eddbe 100644 --- a/python/runfiles/BUILD.bazel +++ b/python/runfiles/BUILD.bazel @@ -40,6 +40,7 @@ py_library( # to the --experimental_python_import_all_repositories setting. "../..", ], + tags = ["pyrefly"], visibility = ["//visibility:public"], ) diff --git a/python/runfiles/runfiles.py b/python/runfiles/runfiles.py index 16afeea47c..6fe2e8f28b 100644 --- a/python/runfiles/runfiles.py +++ b/python/runfiles/runfiles.py @@ -323,7 +323,7 @@ def is_socket(self) -> bool: return self._as_path().is_socket() # override - def open( + def open( # pyrefly: ignore[bad-override] self, mode: str = "r", buffering: int = -1, diff --git a/tests/modules/rules_pyrefly_stub/WORKSPACE b/tests/modules/rules_pyrefly_stub/WORKSPACE new file mode 100644 index 0000000000..48a0b3083d --- /dev/null +++ b/tests/modules/rules_pyrefly_stub/WORKSPACE @@ -0,0 +1 @@ +workspace(name = "rules_pyrefly") diff --git a/tests/modules/rules_pyrefly_stub/pyrefly/BUILD.bazel b/tests/modules/rules_pyrefly_stub/pyrefly/BUILD.bazel new file mode 100644 index 0000000000..2f14f71b3b --- /dev/null +++ b/tests/modules/rules_pyrefly_stub/pyrefly/BUILD.bazel @@ -0,0 +1,3 @@ +package(default_visibility = ["//visibility:public"]) + +exports_files(["pyrefly.bzl"]) diff --git a/tests/modules/rules_pyrefly_stub/pyrefly/pyrefly.bzl b/tests/modules/rules_pyrefly_stub/pyrefly/pyrefly.bzl new file mode 100644 index 0000000000..fc1a44ed31 --- /dev/null +++ b/tests/modules/rules_pyrefly_stub/pyrefly/pyrefly.bzl @@ -0,0 +1,21 @@ +"""Stub implementation of rules_pyrefly for WORKSPACE mode.""" + +# buildifier: disable=unused-variable +def _noop_aspect_impl(_target, _ctx): + return [] + +_noop_aspect = aspect( + implementation = _noop_aspect_impl, + doc = "No-op Pyrefly aspect stub for WORKSPACE mode.", +) + +def pyrefly(**_kwargs): + """Stub pyrefly aspect constructor. + + Args: + **_kwargs: Ignored keyword arguments. + + Returns: + A no-op aspect. + """ + return _noop_aspect diff --git a/tests/support/pyrefly/BUILD.bazel b/tests/support/pyrefly/BUILD.bazel new file mode 100644 index 0000000000..447af06f8f --- /dev/null +++ b/tests/support/pyrefly/BUILD.bazel @@ -0,0 +1,8 @@ +load("@bazel_skylib//:bzl_library.bzl", "bzl_library") + +package(default_visibility = ["//:__subpackages__"]) + +bzl_library( + name = "pyrefly", + srcs = ["pyrefly.bzl"], +) diff --git a/tests/support/pyrefly/pyrefly.bzl b/tests/support/pyrefly/pyrefly.bzl new file mode 100644 index 0000000000..bcb791403f --- /dev/null +++ b/tests/support/pyrefly/pyrefly.bzl @@ -0,0 +1,7 @@ +"""Aspect definition for Pyrefly static type checking.""" + +load("@rules_pyrefly//pyrefly:pyrefly.bzl", "pyrefly") + +pyrefly_aspect = pyrefly( + opt_in_tags = ["pyrefly"], +) From 753795d388e9e24b2e72e26e1c7ae0e4855b4703 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Fri, 7 Aug 2026 00:54:27 -0700 Subject: [PATCH 905/922] chore(release): ignore .agents directory in version marker replacement (#4012) The release tool scans repository files for version placeholder markers (`VERSION_NEXT_FEATURE` / `VERSION_NEXT_PATCH`) to determine the next semantic version and replace them with the release version. Documentation and rule guidance within `.agents/` may reference these placeholder strings as examples or instructions. Previously, the `.agents/` directory was not excluded, allowing placeholder markers in agent configurations to falsely affect version calculations and undergo unwanted replacements during release. To fix this, add `.agents/` to the excluded directory patterns in the release utilities, and add tests verifying that placeholder markers in `.agents` are ignored during version determination and replacement. --- tests/tools/private/release/utils_test.py | 21 +++++++++++++++++++++ tools/private/release/utils.py | 1 + 2 files changed, 22 insertions(+) diff --git a/tests/tools/private/release/utils_test.py b/tests/tools/private/release/utils_test.py index f161308b60..d39b5e108e 100644 --- a/tests/tools/private/release/utils_test.py +++ b/tests/tools/private/release/utils_test.py @@ -264,6 +264,10 @@ def test_replace_version_next_excludes_bazel_dirs(release_tool_env): blabla ::: """ + agents_dir = release_tool_env.git_root / ".agents" + agents_dir.mkdir() + (agents_dir / "mock_file.md").write_text(mock_file_content) + bazel_dir = release_tool_env.git_root / "bazel-rules_python" bazel_dir.mkdir() (bazel_dir / "mock_file.bzl").write_text(mock_file_content) @@ -282,6 +286,9 @@ def test_replace_version_next_excludes_bazel_dirs(release_tool_env): utils.replace_version_next(version) # Assert + new_content = (agents_dir / "mock_file.md").read_text() + assert "VERSION_NEXT_FEATURE" in new_content + new_content = (bazel_dir / "mock_file.bzl").read_text() assert "VERSION_NEXT_FEATURE" in new_content @@ -290,3 +297,17 @@ def test_replace_version_next_excludes_bazel_dirs(release_tool_env): new_content = (tests_dir / "mock_file.bzl").read_text() assert "VERSION_NEXT_FEATURE" in new_content + + +def test_determine_next_version_ignores_agents_markers(mocker, release_tool_env): + mocker.patch( + "tools.private.release.git.Git.get_current_branch", return_value="main" + ) + mocker.patch("tools.private.release.utils.get_latest_version", return_value="1.2.3") + agents_dir = release_tool_env.git_root / ".agents" + agents_dir.mkdir() + (agents_dir / "mock_file.md").write_text(":::{versionadded} VERSION_NEXT_FEATURE") + + next_version = utils.determine_next_version() + + assert next_version == "1.2.4" diff --git a/tools/private/release/utils.py b/tools/private/release/utils.py index cdd1b7724d..8cbbe89a41 100644 --- a/tools/private/release/utils.py +++ b/tools/private/release/utils.py @@ -22,6 +22,7 @@ def semver_type(value): _EXCLUDE_PATTERNS = [ + "./.agents/*", "./.git/*", "./.github/*", "./.bazelci/*", From 1a0095889a891734dc9274b2efdd9482777432ca Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Fri, 7 Aug 2026 01:13:09 -0700 Subject: [PATCH 906/922] chore(release): indent multi-line bullet points in changelog (#4013) When assembling changelog entries from news files during release preparation, multi-line descriptions had continuation lines unindented. In Markdown, unindented continuation lines break list item hierarchy and cause improper formatting. Format news entries so continuation lines are indented with two spaces, preserving proper list item structure in the generated changelog. --- .../private/release/changelog_news_test.py | 58 +++++++++++++++++++ tools/private/release/changelog_news.py | 37 ++++++++++-- 2 files changed, 91 insertions(+), 4 deletions(-) diff --git a/tests/tools/private/release/changelog_news_test.py b/tests/tools/private/release/changelog_news_test.py index 38bba1eecf..dd2bd6df79 100644 --- a/tests/tools/private/release/changelog_news_test.py +++ b/tests/tools/private/release/changelog_news_test.py @@ -495,3 +495,61 @@ def test_update_changelog_insertion_point_too_small(tmp_path): changelog_path=changelog_path, news_dir=news_dir, ) + + +def test_format_entry(): + # Multi-line without leading bullet + raw = "(foo) bla bla\nblabl bla\nblabla" + expected = "* (foo) bla bla\n blabl bla\n blabla" + assert changelog_news._format_entry(raw) == expected + + # Multi-line with leading bullet + raw_with_bullet = "* (foo) bla bla\nblabl bla\nblabla" + assert changelog_news._format_entry(raw_with_bullet) == expected + + # Multi-line already indented + raw_indented = "* (foo) bla bla\n blabl bla\n blabla" + assert changelog_news._format_entry(raw_indented) == expected + + # Empty string + assert changelog_news._format_entry("") == "" + + +def test_update_changelog_multiline_indentation(tmp_path): + # Arrange + changelog = """# Changelog + +{#unreleased} +## Unreleased + +[unreleased]: https://github.com/bazel-contrib/rules_python/releases/tag/unreleased + +{#v2-0-0} +## [2.0.0] - 2026-04-09 + +[2.0.0]: https://github.com/bazel-contrib/rules_python/releases/tag/2.0.0 +""" + changelog_path = tmp_path / "CHANGELOG.md" + changelog_path.write_text(changelog) + + news_dir = tmp_path / "news" + news_dir.mkdir() + (news_dir / "1.fixed.md").write_text("(foo) bla bla\nblabl bla\nblabla") + (news_dir / "2.added.md").write_text("* (bar) feature one\nsecond line of feature") + + # Act + changelog_news.update_changelog( + "2.1.0", + "2026-06-17", + changelog_path=changelog_path, + news_dir=news_dir, + ) + + # Assert + new_content = changelog_path.read_text() + + expected_fixed = "### Fixed\n* (foo) bla bla\n blabl bla\n blabla" + expected_added = "### Added\n* (bar) feature one\n second line of feature" + + assert expected_fixed in new_content + assert expected_added in new_content diff --git a/tools/private/release/changelog_news.py b/tools/private/release/changelog_news.py index e09b082ade..6381aacb34 100644 --- a/tools/private/release/changelog_news.py +++ b/tools/private/release/changelog_news.py @@ -34,6 +34,35 @@ def _get_news_files(news_dir): return [p for p in news_path.iterdir() if is_news_file(p)] +def _format_entry(content): + """Formats news entry content as a markdown bullet list item. + + Continuation lines are indented with two spaces. + """ + lines = content.strip().splitlines() + if not lines: + return "" + + formatted_lines = [] + first_line = lines[0] + if not (first_line.startswith("* ") or first_line.startswith("- ")): + formatted_lines.append(f"* {first_line}") + else: + formatted_lines.append(first_line) + + for line in lines[1:]: + if not line: + formatted_lines.append("") + elif line.startswith("* ") or line.startswith("- "): + formatted_lines.append(line) + elif line.startswith(" "): + formatted_lines.append(line) + else: + formatted_lines.append(f" {line}") + + return "\n".join(formatted_lines) + + def _parse_new_files(news_files): """Parses news files and groups them by category.""" entries = {} @@ -48,13 +77,13 @@ def _parse_new_files(news_files): if not content: continue - # Format as list item if not already - if not (content.startswith("* ") or content.startswith("- ")): - content = f"* {content}" + formatted_content = _format_entry(content) + if not formatted_content: + continue if category not in entries: entries[category] = [] - entries[category].append(content) + entries[category].append(formatted_content) return entries From e84926a2a32732179e8d8b2f85af5dd1ae00a6be Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Fri, 7 Aug 2026 01:49:47 -0700 Subject: [PATCH 907/922] chore(release): fix duplicate argument in NamedTemporaryFile (#4014) In GitHub.update_issue_body, tempfile.NamedTemporaryFile was called with both a positional argument and a keyword argument for the mode, raising a TypeError for multiple values of 'mode'. Additionally, standard output was not flushed before writing fatal error messages to standard error upon release tool failures. Pass mode only as a keyword argument with UTF-8 encoding when writing temporary files for issue body updates, flush stdout prior to logging fatal exceptions, and add unit test coverage for update_issue_body. --- tests/tools/private/release/gh_test.py | 17 +++++++++++++++++ tools/private/release/gh.py | 2 +- tools/private/release/release.py | 1 + 3 files changed, 19 insertions(+), 1 deletion(-) diff --git a/tests/tools/private/release/gh_test.py b/tests/tools/private/release/gh_test.py index b8c346004b..7e19b2232c 100644 --- a/tests/tools/private/release/gh_test.py +++ b/tests/tools/private/release/gh_test.py @@ -77,3 +77,20 @@ def test_auto_patched_helpers_prevent_real_execution(auto_patch_cmd_helpers): gh_obj = GitHub("foo/bar") gh_obj._run_gh("issue", "list") auto_patch_cmd_helpers.run_gh.assert_called_with("issue", "list") + + +def test_update_issue_body(gh, auto_patch_cmd_helpers): + captured_body = {} + + def mock_run(*args, **kwargs): + for arg in args: + if isinstance(arg, str) and arg.startswith("--body-file="): + path = arg.split("=", 1)[1] + with open(path, encoding="utf-8") as f: + captured_body["content"] = f.read() + return None + + auto_patch_cmd_helpers.run_gh.side_effect = mock_run + gh.update_issue_body(123, "new body content") + auto_patch_cmd_helpers.run_gh.assert_called_once() + assert captured_body["content"] == "new body content" diff --git a/tools/private/release/gh.py b/tools/private/release/gh.py index 9d157f31d6..17fcd59a69 100644 --- a/tools/private/release/gh.py +++ b/tools/private/release/gh.py @@ -307,7 +307,7 @@ def update_issue_body(self, issue_num: int, body: str) -> None: issue_num: The issue number. body: The new body content. """ - with tempfile.NamedTemporaryFile("w", delete=False, mode="w") as f: + with tempfile.NamedTemporaryFile(mode="w", delete=False, encoding="utf-8") as f: f.write(body) f.flush() temp_path = f.name diff --git a/tools/private/release/release.py b/tools/private/release/release.py index 72c427a392..7f1896ecf4 100644 --- a/tools/private/release/release.py +++ b/tools/private/release/release.py @@ -64,6 +64,7 @@ def main(): # args.command is the run_from_args classmethod of the selected command exit_code = args.command(args) except Exception as e: + sys.stdout.flush() print(f"Fatal error: {e}", file=sys.stderr) if hasattr(e, "__notes__"): for note in e.__notes__: From d8bb5ebb35d8b7dd6e22792e38fe853df98e49d6 Mon Sep 17 00:00:00 2001 From: "github-actions[bot]" <41898282+github-actions[bot]@users.noreply.github.com> Date: Fri, 7 Aug 2026 01:53:00 -0700 Subject: [PATCH 908/922] Prepare release v2.3.0 (#4011) Work towards #4010 Co-authored-by: Richard Levasseur --- CHANGELOG.md | 76 +++++++++++++++++++ .../python/config_settings/index.md | 2 +- news/3514.added.md | 1 - news/3825.added.md | 6 -- news/3858.fixed.md | 3 - news/3906.fixed.md | 3 - news/3919.fixed.md | 3 - news/3932.added.md | 3 - news/3934.fixed.md | 7 -- news/3935.fixed.md | 4 - news/3950.fixed.md | 3 - news/3952.fixed.md | 3 - news/3972.fixed.md | 11 --- news/3973.added.md | 6 -- news/3978.changed.md | 3 - news/3978.fixed.md | 5 -- python/cc/py_extension.bzl | 2 +- python/private/cc/py_extension_macro.bzl | 2 +- python/private/py_cc_toolchain_info.bzl | 8 +- python/private/py_cc_toolchain_rule.bzl | 8 +- python/private/pypi/extension.bzl | 6 +- python/private/pypi/whl_library.bzl | 4 +- python/private/python.bzl | 2 +- 23 files changed, 93 insertions(+), 78 deletions(-) delete mode 100644 news/3514.added.md delete mode 100644 news/3825.added.md delete mode 100644 news/3858.fixed.md delete mode 100644 news/3906.fixed.md delete mode 100644 news/3919.fixed.md delete mode 100644 news/3932.added.md delete mode 100644 news/3934.fixed.md delete mode 100644 news/3935.fixed.md delete mode 100644 news/3950.fixed.md delete mode 100644 news/3952.fixed.md delete mode 100644 news/3972.fixed.md delete mode 100644 news/3973.added.md delete mode 100644 news/3978.changed.md delete mode 100644 news/3978.fixed.md diff --git a/CHANGELOG.md b/CHANGELOG.md index f6dc36ee73..ede22fc6db 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -29,6 +29,82 @@ Unreleased changes are tracked as individual files in the [news/](./news) directory, or view the [latest generated changelog](https://rules-python.readthedocs.io/en/latest/changelog.html). +{#v2-3-0} +## [2.3.0] - 2026-08-07 + +[2.3.0]: https://github.com/bazel-contrib/rules_python/releases/tag/2.3.0 + +{#v2-3-0-changed} +### Changed +* (gazelle) **BREAKING** rules_python 1.5.0 or higher is now required. The Python + extension selects its standard library list on `is_python_3.14`, which earlier + versions do not define. + +{#v2-3-0-fixed} +### Fixed +* Fixed `py_binary_rule_builder()` / `py_test_rule_builder()` (from `python/api/executables.bzl`) + failing at analysis time with a visibility error when used to construct a custom rule from an + external module. +* (compile_pip_requirements) Add the explicit `data` attribute and forward it + directly to the generated `py_binary`, so files passed via `data` can be + referenced from `extra_args` using `$(location ...)`. +* (coverage) The warning about a missing bundled `coverage.py` wheel is no longer + emitted as we are now falling back to a pure python wheel + ([#3950](https://github.com/bazel-contrib/rules_python/issues/3950)). +* (gazelle) The Python extension now uses the correct standard library module list for + `python_version` 3.13 and 3.14; previously both fell back to the 3.11 list, so modules + added or removed since then (e.g. `compression.zstd`, `telnetlib`) were misclassified. The + fallback list for unrecognized versions is now the newest available one rather than 3.11 + ([#3978](https://github.com/bazel-contrib/rules_python/pull/3978)). +* (pypi) Allow `uv_lock` to be specified in `pip.parse` without requiring + `requirements_lock` (or other os-specific requirement file attributes) to be + set. +* (pypi) Requirement `--hash=:` pins and Simple API + `#=` URL fragments are now parsed for all hash algorithms + instead of silently dropping everything except `sha256`. Non-sha256 pins are + matched against the digests advertised by the index and downloads are verified + using the corresponding Subresource Integrity value, and the pins are kept in + the requirement line when falling back to `pip` + ([#3972](https://github.com/bazel-contrib/rules_python/issues/3972)). + As part of this, `whl_library` repos created by `pip.parse` now always pass + the digest via the `integrity` attribute (SRI format) instead of `sha256`, + and the lock file facts store digests as `:` values (the facts + version was bumped, so cached index information is refreshed once). +* (pypi) `pip.parse(uv_lock = ...)` no longer exposes uv workspace/root members + that resolve to no wheel or sdist (e.g. `source = { virtual = "." }` or editable + installs). Previously these source-less packages were added to the hub's + `all_requirements` / `all_whl_requirements` with an alias to a subpackage that + does not exist, breaking analysis for anything enumerating the full set such as + `modules_mapping(wheels = all_whl_requirements)` + ([#3934](https://github.com/bazel-contrib/rules_python/issues/3934)). +* (pypi) correctly parse the `index_url` for each wheel so that the source registry is forwarded to + the {obj}`whl_library`. This is so that the `purl` for `package_metadata` can be correctly + constructed. +* (pypi) fixed the URL normalization function to correctly handle local paths + enabling wheel sources files to point to an absolute path. Currently it supports + the `file://` for linux and windows like paths. We also support + envsubst for the said paths from now on. + +{#v2-3-0-added} +### Added +* (bzlmod) Added MODULE.bazel flag aliases for Starlark-defined flags: + `build_python_zip`, `incompatible_default_to_explicit_init_py`, + `python_path`, and `experimental_python_import_all_repositories`. +* (cc) Added experimental {obj}`py_extension` macro for creating C/C++ Python + extension modules + ([#3283](https://github.com/bazel-contrib/rules_python/issues/3283)). + (cc) Added `libc`, `platform_machine`, `platform_tag`, `soabi`, and + `sys_platform` attributes and info fields to {obj}`py_cc_toolchain` / + {obj}`PyCcToolchainInfo`. +* (pip,python) Added `pyproject_toml` attribute to {obj}`pip.default`, {obj}`pip.parse` and {obj}`python.defaults` to read the default Python version from the `requires-python` field of `pyproject.toml`. +* (py_test) Added an opt-in safeguard against `py_test` targets that silently + pass without running any tests. Set + {obj}`--@rules_python//python/config_settings:validate_test_main=enabled` to + fail the build when a test's main module only contains inert top-level + statements (definitions, imports, assignments) and never invokes a test + runner ([#3824](https://github.com/bazel-contrib/rules_python/issues/3824)). + + {#v2-2-0} ## [2.2.0] - 2026-06-30 diff --git a/docs/api/rules_python/python/config_settings/index.md b/docs/api/rules_python/python/config_settings/index.md index 78f8937a4d..9ae0665da2 100644 --- a/docs/api/rules_python/python/config_settings/index.md +++ b/docs/api/rules_python/python/config_settings/index.md @@ -276,7 +276,7 @@ Enabling this requires the exec tools toolchain (with an exec interpreter) to be registered, which is the case for the default hermetic toolchains. ::: -:::{versionadded} VERSION_NEXT_FEATURE +:::{versionadded} 2.3.0 ::: :::: diff --git a/news/3514.added.md b/news/3514.added.md deleted file mode 100644 index bf2a119380..0000000000 --- a/news/3514.added.md +++ /dev/null @@ -1 +0,0 @@ -(pip,python) Added `pyproject_toml` attribute to {obj}`pip.default`, {obj}`pip.parse` and {obj}`python.defaults` to read the default Python version from the `requires-python` field of `pyproject.toml`. diff --git a/news/3825.added.md b/news/3825.added.md deleted file mode 100644 index 71b20aca91..0000000000 --- a/news/3825.added.md +++ /dev/null @@ -1,6 +0,0 @@ -(py_test) Added an opt-in safeguard against `py_test` targets that silently -pass without running any tests. Set -{obj}`--@rules_python//python/config_settings:validate_test_main=enabled` to -fail the build when a test's main module only contains inert top-level -statements (definitions, imports, assignments) and never invokes a test -runner ([#3824](https://github.com/bazel-contrib/rules_python/issues/3824)). diff --git a/news/3858.fixed.md b/news/3858.fixed.md deleted file mode 100644 index dd991e4654..0000000000 --- a/news/3858.fixed.md +++ /dev/null @@ -1,3 +0,0 @@ -(compile_pip_requirements) Add the explicit `data` attribute and forward it -directly to the generated `py_binary`, so files passed via `data` can be -referenced from `extra_args` using `$(location ...)`. diff --git a/news/3906.fixed.md b/news/3906.fixed.md deleted file mode 100644 index 64a16fd9c8..0000000000 --- a/news/3906.fixed.md +++ /dev/null @@ -1,3 +0,0 @@ -(pypi) correctly parse the `index_url` for each wheel so that the source registry is forwarded to -the {obj}`whl_library`. This is so that the `purl` for `package_metadata` can be correctly -constructed. diff --git a/news/3919.fixed.md b/news/3919.fixed.md deleted file mode 100644 index 0e45d3ce9c..0000000000 --- a/news/3919.fixed.md +++ /dev/null @@ -1,3 +0,0 @@ -Fixed `py_binary_rule_builder()` / `py_test_rule_builder()` (from `python/api/executables.bzl`) -failing at analysis time with a visibility error when used to construct a custom rule from an -external module. diff --git a/news/3932.added.md b/news/3932.added.md deleted file mode 100644 index 579d1b5f59..0000000000 --- a/news/3932.added.md +++ /dev/null @@ -1,3 +0,0 @@ -(bzlmod) Added MODULE.bazel flag aliases for Starlark-defined flags: -`build_python_zip`, `incompatible_default_to_explicit_init_py`, -`python_path`, and `experimental_python_import_all_repositories`. diff --git a/news/3934.fixed.md b/news/3934.fixed.md deleted file mode 100644 index a30f475770..0000000000 --- a/news/3934.fixed.md +++ /dev/null @@ -1,7 +0,0 @@ -(pypi) `pip.parse(uv_lock = ...)` no longer exposes uv workspace/root members -that resolve to no wheel or sdist (e.g. `source = { virtual = "." }` or editable -installs). Previously these source-less packages were added to the hub's -`all_requirements` / `all_whl_requirements` with an alias to a subpackage that -does not exist, breaking analysis for anything enumerating the full set such as -`modules_mapping(wheels = all_whl_requirements)` -([#3934](https://github.com/bazel-contrib/rules_python/issues/3934)). diff --git a/news/3935.fixed.md b/news/3935.fixed.md deleted file mode 100644 index 83cd0473e8..0000000000 --- a/news/3935.fixed.md +++ /dev/null @@ -1,4 +0,0 @@ -(pypi) fixed the URL normalization function to correctly handle local paths -enabling wheel sources files to point to an absolute path. Currently it supports -the `file://` for linux and windows like paths. We also support -envsubst for the said paths from now on. diff --git a/news/3950.fixed.md b/news/3950.fixed.md deleted file mode 100644 index 87e0bd91a6..0000000000 --- a/news/3950.fixed.md +++ /dev/null @@ -1,3 +0,0 @@ -(coverage) The warning about a missing bundled `coverage.py` wheel is no longer -emitted as we are now falling back to a pure python wheel -([#3950](https://github.com/bazel-contrib/rules_python/issues/3950)). diff --git a/news/3952.fixed.md b/news/3952.fixed.md deleted file mode 100644 index 70558dd986..0000000000 --- a/news/3952.fixed.md +++ /dev/null @@ -1,3 +0,0 @@ -(pypi) Allow `uv_lock` to be specified in `pip.parse` without requiring -`requirements_lock` (or other os-specific requirement file attributes) to be -set. diff --git a/news/3972.fixed.md b/news/3972.fixed.md deleted file mode 100644 index f96f803988..0000000000 --- a/news/3972.fixed.md +++ /dev/null @@ -1,11 +0,0 @@ -(pypi) Requirement `--hash=:` pins and Simple API -`#=` URL fragments are now parsed for all hash algorithms -instead of silently dropping everything except `sha256`. Non-sha256 pins are -matched against the digests advertised by the index and downloads are verified -using the corresponding Subresource Integrity value, and the pins are kept in -the requirement line when falling back to `pip` -([#3972](https://github.com/bazel-contrib/rules_python/issues/3972)). -As part of this, `whl_library` repos created by `pip.parse` now always pass -the digest via the `integrity` attribute (SRI format) instead of `sha256`, -and the lock file facts store digests as `:` values (the facts -version was bumped, so cached index information is refreshed once). diff --git a/news/3973.added.md b/news/3973.added.md deleted file mode 100644 index 20d136179a..0000000000 --- a/news/3973.added.md +++ /dev/null @@ -1,6 +0,0 @@ -(cc) Added experimental {obj}`py_extension` macro for creating C/C++ Python -extension modules -([#3283](https://github.com/bazel-contrib/rules_python/issues/3283)). -(cc) Added `libc`, `platform_machine`, `platform_tag`, `soabi`, and -`sys_platform` attributes and info fields to {obj}`py_cc_toolchain` / -{obj}`PyCcToolchainInfo`. diff --git a/news/3978.changed.md b/news/3978.changed.md deleted file mode 100644 index 8418abd0b3..0000000000 --- a/news/3978.changed.md +++ /dev/null @@ -1,3 +0,0 @@ -(gazelle) **BREAKING** rules_python 1.5.0 or higher is now required. The Python -extension selects its standard library list on `is_python_3.14`, which earlier -versions do not define. diff --git a/news/3978.fixed.md b/news/3978.fixed.md deleted file mode 100644 index 2c5cbb5f5d..0000000000 --- a/news/3978.fixed.md +++ /dev/null @@ -1,5 +0,0 @@ -(gazelle) The Python extension now uses the correct standard library module list for -`python_version` 3.13 and 3.14; previously both fell back to the 3.11 list, so modules -added or removed since then (e.g. `compression.zstd`, `telnetlib`) were misclassified. The -fallback list for unrecognized versions is now the newest available one rather than 3.11 -([#3978](https://github.com/bazel-contrib/rules_python/pull/3978)). diff --git a/python/cc/py_extension.bzl b/python/cc/py_extension.bzl index d81877a702..8e90517789 100644 --- a/python/cc/py_extension.bzl +++ b/python/cc/py_extension.bzl @@ -11,7 +11,7 @@ for information on writing C extension modules. :::{include} /_includes/experimental_api.md ::: -:::{versionadded} VERSION_NEXT_FEATURE +:::{versionadded} 2.3.0 ::: """ diff --git a/python/private/cc/py_extension_macro.bzl b/python/private/cc/py_extension_macro.bzl index 6b697ab503..b76685113a 100644 --- a/python/private/cc/py_extension_macro.bzl +++ b/python/private/cc/py_extension_macro.bzl @@ -50,7 +50,7 @@ def py_extension( - `module_name`: Pass `module_name = "custom_name"` to override the base module filename. - :::{versionadded} VERSION_NEXT_FEATURE + :::{versionadded} 2.3.0 ::: Args: diff --git a/python/private/py_cc_toolchain_info.bzl b/python/private/py_cc_toolchain_info.bzl index 208a9589ba..a56ae7c960 100644 --- a/python/private/py_cc_toolchain_info.bzl +++ b/python/private/py_cc_toolchain_info.bzl @@ -22,7 +22,7 @@ PyCcToolchainInfo = provider( The runtime's ABI flags, i.e. `sys.abiflags` (e.g. 't' for free-threaded builds). -:::{versionadded} VERSION_NEXT_FEATURE +:::{versionadded} 2.3.0 ::: """, "headers": """\ @@ -106,7 +106,7 @@ If available, information about C libraries, struct with fields: The {pep}`PEP 508` `platform_machine` marker value for the target architecture, e.g. 'x86_64', 'aarch64'. -:::{versionadded} VERSION_NEXT_FEATURE +:::{versionadded} 2.3.0 ::: """, "platform_tag": """\ @@ -115,7 +115,7 @@ value for the target architecture, e.g. 'x86_64', 'aarch64'. The PEP 3149 / PEP 425 platform tag for extension modules, e.g. 'x86_64-linux-gnu', 'darwin', or 'win_amd64'. -:::{versionadded} VERSION_NEXT_FEATURE +:::{versionadded} 2.3.0 ::: """, "python_version": """ @@ -130,7 +130,7 @@ The SOABI tag for extension modules (see [PEP 3149](https://peps.python.org/pep-3149/)), e.g. 'cpython-311-x86_64-linux-gnu' or 'cp311'. -:::{versionadded} VERSION_NEXT_FEATURE +:::{versionadded} 2.3.0 ::: """, "sys_platform": """ diff --git a/python/private/py_cc_toolchain_rule.bzl b/python/private/py_cc_toolchain_rule.bzl index 02d4222fc8..405a8afe56 100644 --- a/python/private/py_cc_toolchain_rule.bzl +++ b/python/private/py_cc_toolchain_rule.bzl @@ -143,7 +143,7 @@ If not set, or set to ``, the ABI flags are automatically derived from `--//python/config_settings:py_freethreaded` (e.g., `'t'` when free-threaded is enabled, or `''` otherwise). -:::{versionadded} VERSION_NEXT_FEATURE +:::{versionadded} 2.3.0 ::: """, ), @@ -171,7 +171,7 @@ attribute is available or not. doc = """\ Target C library variant, e.g. 'glibc', 'musl'. -:::{versionadded} VERSION_NEXT_FEATURE +:::{versionadded} 2.3.0 ::: """, default = "", @@ -185,7 +185,7 @@ Target C library variant, e.g. 'glibc', 'musl'. doc = """ Target architecture as a PEP 508 `platform_machine` marker, e.g. 'x86_64', 'aarch64', 'x86_32'. -:::{versionadded} VERSION_NEXT_FEATURE +:::{versionadded} 2.3.0 ::: """, default = "", @@ -199,7 +199,7 @@ Target architecture as a PEP 508 `platform_machine` marker, e.g. 'x86_64', 'aarc The SOABI tag for extension modules (see PEP 3149), e.g. 'cpython-311-x86_64-linux-gnu' or 'cp311'. -:::{versionadded} VERSION_NEXT_FEATURE +:::{versionadded} 2.3.0 ::: """, default = "", diff --git a/python/private/pypi/extension.bzl b/python/private/pypi/extension.bzl index caca92a5f7..ed1fa28664 100644 --- a/python/private/pypi/extension.bzl +++ b/python/private/pypi/extension.bzl @@ -751,7 +751,7 @@ The version must be specified as `==X.Y.Z` (exact version with full semver). This is designed to work with dependency management tools like Renovate. ::: -:::{versionadded} VERSION_NEXT_FEATURE +:::{versionadded} 2.3.0 ::: """, ), @@ -934,7 +934,7 @@ for this `pip.parse()` call, unless `python_version` is set explicitly. The version must be specified as `==X.Y.Z` (exact version with full semver). ::: -:::{versionadded} VERSION_NEXT_FEATURE +:::{versionadded} 2.3.0 ::: """, ), @@ -952,7 +952,7 @@ a corresponding `python.toolchain()` configured. The {obj}`pyproject_toml` attribute for getting the version from a project file. ::: -:::{versionchanged} VERSION_NEXT_FEATURE +:::{versionchanged} 2.3.0 No longer mandatory if the {obj}`pyproject_toml` attribute or {obj}`pip.default.pyproject_toml` is specified. ::: diff --git a/python/private/pypi/whl_library.bzl b/python/private/pypi/whl_library.bzl index 640a6518a6..47b05468ef 100644 --- a/python/private/pypi/whl_library.bzl +++ b/python/private/pypi/whl_library.bzl @@ -587,7 +587,7 @@ The expected checksum of the downloaded whl in Subresource Integrity format (e.g. `sha256-...` or `sha512-...`). Only used when `urls` is passed. If `sha256` is also set, it takes precedence over this attribute. -:::{versionadded} VERSION_NEXT_FEATURE +:::{versionadded} 2.3.0 ::: """, ), @@ -673,7 +673,7 @@ The `whl_library` is marked as reproducible if using starlark to extract and par wheel contents without building an `sdist` first. ::: -:::{versionchanged} VERSION_NEXT_FEATURE +:::{versionchanged} 2.3.0 The whl-only pure Starlark operations have been refactored into {obj}`whl_archive` and the previously named {obj}`whl_library` repository became renamed to `pip_archive`. ::: diff --git a/python/private/python.bzl b/python/private/python.bzl index 0d7d2baf40..8932d00e1b 100644 --- a/python/private/python.bzl +++ b/python/private/python.bzl @@ -1117,7 +1117,7 @@ Label pointing to pyproject.toml file to read the default Python version from. When specified, reads the `requires-python` field from pyproject.toml. The version must be specified as `==X.Y.Z` (exact version with full semver). -:::{versionadded} VERSION_NEXT_FEATURE +:::{versionadded} 2.3.0 ::: """, ), From 8293c76f9235b2d67ff286e4a297084a3366b812 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Fri, 7 Aug 2026 16:27:40 -0700 Subject: [PATCH 909/922] chore(release): add release-prepared label to prepare PRs (#4015) Release preparation pull requests created by the release prepare tool did not have the `release-prepared` label applied upon creation. This prevented downstream automation and CI workflows that filter or trigger on this label from detecting newly prepared release PRs. To address this, define the `release-prepared` label constant in GitHub helpers, attach the label when creating prepare PRs, and update test assertions to verify label assignment. --- tests/tools/private/release/prepare_test.py | 4 ++++ tools/private/release/gh.py | 1 + tools/private/release/prepare.py | 2 ++ 3 files changed, 7 insertions(+) diff --git a/tests/tools/private/release/prepare_test.py b/tests/tools/private/release/prepare_test.py index 3a3e886f45..64275df96f 100644 --- a/tests/tools/private/release/prepare_test.py +++ b/tests/tools/private/release/prepare_test.py @@ -1,5 +1,6 @@ import argparse +from tools.private.release.gh import RELEASE_PREPARED_LABEL from tools.private.release.prepare import Prepare pytest_plugins = ["tests.tools.private.release.release_test_helper"] @@ -27,6 +28,7 @@ def test_prepare_success_existing_issue(mocker, release_tool_env, mock_git, mock assert 1002 in mock_gh.prs assert mock_gh.prs[1002]["title"] == "Prepare release v2.0.0" assert mock_gh.prs[1002]["body"] == "Work towards #1001" + assert mock_gh.prs[1002]["labels"] == [RELEASE_PREPARED_LABEL] mock_git.add_modified_and_deleted.assert_called_once() @@ -49,6 +51,7 @@ def test_prepare_success_create_issue(mocker, release_tool_env, mock_git, mock_g assert 1002 in mock_gh.prs assert mock_gh.prs[1002]["title"] == "Prepare release v2.0.0" assert mock_gh.prs[1002]["body"] == "Work towards #1001" + assert mock_gh.prs[1002]["labels"] == [RELEASE_PREPARED_LABEL] mock_git.add_modified_and_deleted.assert_called_once() @@ -154,6 +157,7 @@ def test_prepare_create_pr_when_none_associated( ) updated_body = mock_gh.get_issue_body(1001) assert "pr=#1002" in updated_body + assert mock_gh.prs[1002]["labels"] == [RELEASE_PREPARED_LABEL] def test_prepare_reuse_existing_pr(mocker, release_tool_env, mock_git, mock_gh): diff --git a/tools/private/release/gh.py b/tools/private/release/gh.py index 17fcd59a69..c21041384e 100644 --- a/tools/private/release/gh.py +++ b/tools/private/release/gh.py @@ -13,6 +13,7 @@ # GitHub label types RELEASE_LABEL = "type: release" BACKPORT_LABEL = "type: backport-pr" +RELEASE_PREPARED_LABEL = "release-prepared" # GitHub reaction types # See: https://docs.github.com/en/rest/reactions/reactions?apiVersion=2022-11-28#about-reactions diff --git a/tools/private/release/prepare.py b/tools/private/release/prepare.py index b01f135ff8..00412a8600 100644 --- a/tools/private/release/prepare.py +++ b/tools/private/release/prepare.py @@ -6,6 +6,7 @@ from tools.private.release import changelog_news from tools.private.release.gh import ( + RELEASE_PREPARED_LABEL, GitHub, MultipleTrackingIssuesError, NoTrackingIssueError, @@ -177,6 +178,7 @@ def run(self) -> int: title=f"Prepare release v{version}", body=f"Work towards #{issue_num}", base="main", + labels=[RELEASE_PREPARED_LABEL], ) pr_num = pr_url.split("/")[-1] print(f"Created Pull Request: {pr_url} (PR #{pr_num})") From caa22bd9229cc9e4b246334bb1b2d0c064cedf2f Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Fri, 7 Aug 2026 20:12:45 -0700 Subject: [PATCH 910/922] refactor(tests): convert legacy py_extension in local_toolchains to official rule (#3988) The local_toolchains integration test workspace previously relied on a custom, partial implementation of py_extension that predated the official rule. Replace this legacy test implementation with the official @rules_python//python/cc:py_extension.bzl rule, simplifying the target definition to use sources directly. --- python/private/get_local_runtime_info.py | 4 +- python/private/local_runtime_repo_setup.bzl | 8 + .../integration/local_toolchains/BUILD.bazel | 21 +-- .../integration/local_toolchains/echo_test.py | 2 +- .../local_toolchains/py_extension.bzl | 154 ------------------ 5 files changed, 20 insertions(+), 169 deletions(-) delete mode 100644 tests/integration/local_toolchains/py_extension.bzl diff --git a/python/private/get_local_runtime_info.py b/python/private/get_local_runtime_info.py index e0eab962d6..a4f622700b 100644 --- a/python/private/get_local_runtime_info.py +++ b/python/private/get_local_runtime_info.py @@ -268,7 +268,9 @@ def _get_base_executable() -> str: "implementation_name": sys.implementation.name, "base_executable": _get_base_executable(), "sys_platform": sys.platform, - "platform_machine": platform.machine(), + # Normalize to lowercase: on Windows, platform.machine() returns uppercase + # "AMD64" / "ARM64", whereas PEP 508 and toolchains expect lowercase. + "platform_machine": platform.machine().lower(), } data.update(_get_python_library_info(_get_base_executable())) print(json.dumps(data)) diff --git a/python/private/local_runtime_repo_setup.bzl b/python/private/local_runtime_repo_setup.bzl index 62e46e4b98..05ca6228e0 100644 --- a/python/private/local_runtime_repo_setup.bzl +++ b/python/private/local_runtime_repo_setup.bzl @@ -137,6 +137,10 @@ def define_local_runtime_toolchain_impl( hdrs = [":includes"], defines = defines, # NOTE: Users should define Py_LIMITED_API=3 srcs = abi3_libraries + additional_dlls, + deps = select({ + "@bazel_tools//src/conditions:windows": [":abi3_interface"], + "//conditions:default": [], + }), ) cc_library( @@ -144,6 +148,10 @@ def define_local_runtime_toolchain_impl( hdrs = [":includes"], defines = defines, srcs = libraries + additional_dlls, + deps = select({ + "@bazel_tools//src/conditions:windows": [":interface"], + "//conditions:default": [], + }), ) # runtime configuration diff --git a/tests/integration/local_toolchains/BUILD.bazel b/tests/integration/local_toolchains/BUILD.bazel index 20ee7bcfe6..ae99e7cf2f 100644 --- a/tests/integration/local_toolchains/BUILD.bazel +++ b/tests/integration/local_toolchains/BUILD.bazel @@ -13,9 +13,8 @@ # limitations under the License. load("@bazel_skylib//rules:common_settings.bzl", "string_flag") -load("@rules_cc//cc:cc_library.bzl", "cc_library") load("@rules_python//python:py_test.bzl", "py_test") -load(":py_extension.bzl", "py_extension") +load("@rules_python//python/cc:py_extension.bzl", "py_extension") py_test( name = "local_runtime_test", @@ -60,28 +59,24 @@ string_flag( ) # Build rules to generate a python extension. -cc_library( - name = "echo_ext_cc", - testonly = True, - srcs = ["echo_ext.cc"], - deps = [ - "@rules_python//python/cc:current_py_cc_headers", - ], - alwayslink = True, -) - py_extension( name = "echo_ext", testonly = True, + srcs = ["echo_ext.cc"], copts = select({ "@rules_cc//cc/compiler:msvc-cl": [], "//conditions:default": ["-fvisibility=hidden"], }), - deps = [":echo_ext_cc"], + imports = ["."], ) py_test( name = "echo_test", srcs = ["echo_test.py"], + # Make this test better respect pyenv/local Python DLLs on Windows + env_inherit = [ + "PYENV_VERSION", + "PATH", + ], deps = [":echo_ext"], ) diff --git a/tests/integration/local_toolchains/echo_test.py b/tests/integration/local_toolchains/echo_test.py index 17121e0f17..655205fe29 100644 --- a/tests/integration/local_toolchains/echo_test.py +++ b/tests/integration/local_toolchains/echo_test.py @@ -5,4 +5,4 @@ class ExtensionTest(unittest.TestCase): def test_echo_extension(self): - self.assertEqual(echo_ext.echo(42, "str"), tuple(42, "str")) + self.assertEqual(echo_ext.echo(42, "str"), (42, "str")) diff --git a/tests/integration/local_toolchains/py_extension.bzl b/tests/integration/local_toolchains/py_extension.bzl deleted file mode 100644 index 5d37fd7824..0000000000 --- a/tests/integration/local_toolchains/py_extension.bzl +++ /dev/null @@ -1,154 +0,0 @@ -# Copyright 2025 The Bazel Authors. All rights reserved. -# -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License. - -"""Macro to build a python C/C++ extension. - -There are variants of py_extension in many other projects, such as: -* https://github.com/protocolbuffers/protobuf/tree/main/python/py_extension.bzl -* https://github.com/google/riegeli/blob/master/python/riegeli/py_extension.bzl -* https://github.com/pybind/pybind11_bazel/blob/master/build_defs.bzl - -The issue for a generic verion is: -* https://github.com/bazel-contrib/rules_python/issues/824 -""" - -load("@bazel_skylib//rules:copy_file.bzl", "copy_file") -load("@rules_cc//cc:cc_binary.bzl", "cc_binary") -load("@rules_python//python:defs.bzl", "py_library") - -def py_extension( - *, - name, - deps = None, - linkopts = None, - imports = None, - visibility = None, - **kwargs): - """Creates a Python module implemented in C++. - - A Python extension has 2 essential parts: - 1. An internal shared object / pyd package for the extension, `name.pyd`/`name.so` - 2. The py_library target for the extension.` - - Python modules can depend on a py_extension. - - Args: - name: `str`. Name for this target. This is typically the module name. - deps: `list`. Required. C++ libraries to link into the module. - linkopts: `list`. Linking options for the shared library. - imports: `list`. Additional imports for the py_library rule. - visibility: `str`. Visibility for target. - **kwargs: Additional options for the cc_library rule. - """ - if not name: - fail("py_extension requires a name") - if not deps: - fail("py_extension requires a non-empty deps attribute") - if "linkshared" in kwargs: - fail("py_extension attribute linkshared not allowed") - - if not linkopts: - linkopts = [] - - testonly = kwargs.get("testonly") - tags = kwargs.pop("tags", []) - - cc_binary_so_name = name + ".so" - cc_binary_dll_name = name + ".dll" - cc_binary_pyd_name = name + ".pyd" - linker_script_name = name + ".lds" - linker_script_name_rule = name + "_lds" - shared_objects_name = name + "__shared_objects" - - # On Unix, restrict symbol visibility. - exported_symbol = "PyInit_" + name - - # Generate linker script used on non-macOS unix platforms. - native.genrule( - name = linker_script_name_rule, - outs = [linker_script_name], - cmd = "\n".join([ - "cat <<'EOF' >$@", - "{", - " global: " + exported_symbol + ";", - " local: *;", - "};", - "EOF", - ]), - ) - - for cc_binary_name in [cc_binary_dll_name, cc_binary_so_name]: - cur_linkopts = linkopts - cur_deps = deps - if cc_binary_name == cc_binary_so_name: - cur_linkopts = linkopts + select({ - "@platforms//os:macos": [ - # Avoid undefined symbol errors for CPython symbols that - # will be resolved at runtime. - "-undefined", - "dynamic_lookup", - # On macOS, the linker does not support version scripts. Use - # the `-exported_symbol` option instead to restrict symbol - # visibility. - "-Wl,-exported_symbol", - # On macOS, the symbol starts with an underscore. - "-Wl,_" + exported_symbol, - ], - # On non-macOS unix, use a version script to restrict symbol - # visibility. - "//conditions:default": [ - "-Wl,--version-script", - "-Wl,$(location :" + linker_script_name + ")", - ], - }) - cur_deps = cur_deps + select({ - "@platforms//os:macos": [], - "//conditions:default": [linker_script_name], - }) - - cc_binary( - name = cc_binary_name, - linkshared = True, - visibility = ["//visibility:private"], - deps = cur_deps, - tags = tags + ["manual"], - linkopts = cur_linkopts, - **kwargs - ) - - copy_file( - name = cc_binary_pyd_name + "__pyd_copy", - src = ":" + cc_binary_dll_name, - out = cc_binary_pyd_name, - visibility = visibility, - tags = ["manual"], - testonly = testonly, - ) - - native.filegroup( - name = shared_objects_name, - data = select({ - "@platforms//os:windows": [":" + cc_binary_pyd_name], - "//conditions:default": [":" + cc_binary_so_name], - }), - testonly = testonly, - ) - py_library( - name = name, - data = [":" + shared_objects_name], - imports = imports, - tags = tags, - testonly = testonly, - visibility = visibility, - ) From 0e6d24a83abc0410ab9329f3ec4099333c5a987f Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Sat, 8 Aug 2026 13:10:04 -0700 Subject: [PATCH 911/922] chore(release): support manual dispatch and /prepare-complete for complete prepare workflow (#4016) Previously, the release preparation completion workflow only triggered automatically when a pull request with the `release-prepared` label was closed as merged. If the automatic trigger was missed or failed, maintainers had no mechanism to trigger completion on demand or update the release tracking issue checklist state. Support manual dispatch and comment-driven triggers with bidirectional issue/PR resolution. The workflow can now be invoked directly via manual dispatch, reusable workflow calls, or the `/prepare-complete` comment command on tracking issues and PRs. The release CLI tool resolves tracking issues and PRs from each other to mark the task complete in the tracking issue. --- .../release_tracking_template.md | 1 + .github/workflows/on_comment.yaml | 19 +++ .../workflows/release_complete_prepare.yaml | 49 ++++++-- RELEASING.md | 10 +- tests/tools/private/release/BUILD.bazel | 11 ++ .../private/release/complete_prepare_test.py | 109 ++++++++++++++++++ tools/private/release/complete_prepare.py | 75 ++++++++---- 7 files changed, 241 insertions(+), 33 deletions(-) create mode 100644 tests/tools/private/release/complete_prepare_test.py diff --git a/.github/ISSUE_TEMPLATE/release_tracking_template.md b/.github/ISSUE_TEMPLATE/release_tracking_template.md index 57a51a16ef..349de8c506 100644 --- a/.github/ISSUE_TEMPLATE/release_tracking_template.md +++ b/.github/ISSUE_TEMPLATE/release_tracking_template.md @@ -23,6 +23,7 @@ To manually control the release flow, see the [RELEASING.md: Manual Editing](htt Comment commands: - `/prepare`: Determines version, creates tracking issue and preparation PR. +- `/prepare-complete [PR]`: Marks preparation task as complete. - `/create-rc`: Tags and publishes a new release candidate (RC). - `/process-backports`: Cherry-picks pending backports. - `/add-backports `: Adds PRs to the backports and processes backports. diff --git a/.github/workflows/on_comment.yaml b/.github/workflows/on_comment.yaml index 7e4645f3fe..e2f742ba8c 100644 --- a/.github/workflows/on_comment.yaml +++ b/.github/workflows/on_comment.yaml @@ -50,6 +50,13 @@ jobs: # Handle /create-rc comment if echo "$COMMENT_BODY" | grep -qE '^[[:space:]]*/create-rc([[:space:]]|$)'; then echo "command=create-rc" >> "$GITHUB_OUTPUT" + # Handle /prepare-complete comment + elif echo "$COMMENT_BODY" | grep -qE '^[[:space:]]*/prepare-complete([[:space:]]|$)'; then + echo "command=prepare-complete" >> "$GITHUB_OUTPUT" + pr_arg=$(echo "$COMMENT_BODY" | grep -E '^[[:space:]]*/prepare-complete([[:space:]]|$)' | sed -E 's/^[[:space:]]*\/prepare-complete[[:space:]]*//' | tr -d '[:space:]#') + if [ -n "$pr_arg" ]; then + echo "pr_number=$pr_arg" >> "$GITHUB_OUTPUT" + fi # Handle /prepare comment elif echo "$COMMENT_BODY" | grep -qE '^[[:space:]]*/prepare([[:space:]]|$)'; then echo "command=prepare" >> "$GITHUB_OUTPUT" @@ -100,6 +107,9 @@ jobs: if echo "$COMMENT_BODY" | grep -qE '^[[:space:]]*/backport([[:space:]]|$)'; then echo "command=pr-backport" >> "$GITHUB_OUTPUT" echo "pr_number=$pr_number" >> "$GITHUB_OUTPUT" + elif echo "$COMMENT_BODY" | grep -qE '^[[:space:]]*/prepare-complete([[:space:]]|$)'; then + echo "command=prepare-complete" >> "$GITHUB_OUTPUT" + echo "pr_number=$pr_number" >> "$GITHUB_OUTPUT" else echo "command=none" >> "$GITHUB_OUTPUT" fi @@ -116,6 +126,15 @@ jobs: comment_id: "${{ github.event.comment.id }}" secrets: inherit + call_prepare_complete: + needs: parse_comment + if: needs.parse_comment.outputs.command == 'prepare-complete' + uses: ./.github/workflows/release_complete_prepare.yaml + with: + pr: ${{ needs.parse_comment.outputs.pr_number }} + issue: ${{ needs.parse_comment.outputs.issue_number }} + secrets: inherit + call_prepare: needs: parse_comment if: needs.parse_comment.outputs.command == 'prepare' diff --git a/.github/workflows/release_complete_prepare.yaml b/.github/workflows/release_complete_prepare.yaml index 6d8bc8fd03..a084b6f682 100644 --- a/.github/workflows/release_complete_prepare.yaml +++ b/.github/workflows/release_complete_prepare.yaml @@ -1,19 +1,41 @@ -name: "Release: Complete Prepare" +name: "Release: Prepare: Complete" on: pull_request: types: [closed] + workflow_dispatch: + inputs: + pr: + description: 'The merged preparation PR number (optional if issue provided)' + required: false + type: string + issue: + description: 'The Release Tracking Issue Number (optional if pr provided)' + required: false + type: string + workflow_call: + inputs: + pr: + description: 'The merged preparation PR number (optional if issue provided)' + required: false + type: string + issue: + description: 'The Release Tracking Issue Number (optional if pr provided)' + required: false + type: string permissions: contents: write issues: write jobs: - on_pr_merged: - # Run only if the release-prepared PR was merged + complete_prepare: + # Run if triggered manually or if the release-prepared PR was merged if: | - github.event.pull_request.merged == true && - contains(github.event.pull_request.labels.*.name, 'release-prepared') + github.event_name == 'workflow_dispatch' || + github.event_name == 'workflow_call' || + (github.event.pull_request.merged == true && + contains(github.event.pull_request.labels.*.name, 'release-prepared')) runs-on: ubuntu-latest steps: - name: Checkout repository @@ -27,9 +49,20 @@ jobs: bazelisk-version: 1.20.0 - name: Mark Prepare Release Complete + env: + GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} + PR_NUMBER: ${{ inputs.pr || github.event.pull_request.number }} + ISSUE: ${{ inputs.issue }} run: | + ARGS=() + if [ -n "$PR_NUMBER" ]; then + PR_NUMBER="${PR_NUMBER#\#}" + ARGS+=("--pr=$PR_NUMBER") + fi + if [ -n "$ISSUE" ]; then + ISSUE="${ISSUE#\#}" + ARGS+=("--issue=$ISSUE") + fi # Run the complete-prepare subcommand in the release tool to cleanly update checklist metadata bazel run //tools/private/release -- \ - complete-prepare --pr ${{ github.event.pull_request.number }} - env: - GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} + complete-prepare "${ARGS[@]}" diff --git a/RELEASING.md b/RELEASING.md index 2b3d5859c1..badf33fb7c 100644 --- a/RELEASING.md +++ b/RELEASING.md @@ -27,8 +27,12 @@ Release Tracking Issue and automated workflows triggered by comments or issue ed This will automatically determine the next version, create a release tracking issue, and send a preparation PR. -2. **Approve and Merge**: Approve and merge the PR. Once merged, a release - branch will be created automatically. +2. **Approve and Merge**: Approve and merge the PR. Once merged, the + [Release: Prepare: Complete](https://github.com/bazel-contrib/rules_python/actions/workflows/release_complete_prepare.yaml) + workflow automatically updates the tracking issue checklist, and a release + branch will be created. If the automatic trigger is missed, comment + `/prepare-complete` on the tracking issue or on the PR, or run the workflow + manually with the PR number. 3. **Add Backports (if needed)**: If there are backports, add them following the [How to add backports](#how-to-add-backports) steps. @@ -253,6 +257,8 @@ You can manually edit the Release Tracking Issue to control the release flow. The checklist items use metadata suffix: `| key=value key2=value2`. * **Retry Prepare Release**: Reset the task to `- [ ] Prepare Release | status=awaiting-preparation`. +* **Complete Prepare Release**: Comment `/prepare-complete` on the tracking + issue (or on the preparation PR) to mark the preparation task as done. * **Force Task Done**: Check the box `- [x]` and add appropriate metadata (e.g. `status=done`). ## Secrets diff --git a/tests/tools/private/release/BUILD.bazel b/tests/tools/private/release/BUILD.bazel index b666cf8b54..dcef2ab53a 100644 --- a/tests/tools/private/release/BUILD.bazel +++ b/tests/tools/private/release/BUILD.bazel @@ -46,6 +46,17 @@ pytest_test( ], ) +pytest_test( + name = "complete_prepare_test", + srcs = ["complete_prepare_test.py"], + target_compatible_with = NOT_WINDOWS, + deps = [ + ":conftest", + ":release_test_helper", + "//tools/private/release:release_lib", + ], +) + pytest_test( name = "complete_sync_changelog_test", srcs = ["complete_sync_changelog_test.py"], diff --git a/tests/tools/private/release/complete_prepare_test.py b/tests/tools/private/release/complete_prepare_test.py new file mode 100644 index 0000000000..4f28cce66e --- /dev/null +++ b/tests/tools/private/release/complete_prepare_test.py @@ -0,0 +1,109 @@ +import argparse + +from tools.private.release.complete_prepare import CompletePrepare + +pytest_plugins = ["tests.tools.private.release.release_test_helper"] + + +def test_complete_prepare_with_pr_success(mock_gh): + args = argparse.Namespace(pr=456, issue=None) + mock_gh.prs[456] = { + "state": "MERGED", + "body": "Prepare release for v2.0.0\n\nWork towards #123", + "mergeCommit": {"oid": "abcdef1234567890"}, + } + issue_body = """ +## Checklist +- [ ] Prepare Release | status=pending pr=#456 +- [ ] Create Release branch +- [ ] Tag Final +""" + mock_gh.issues[123] = { + "title": "Release 2.0.0", + "body": issue_body, + "labels": ["type: release"], + "number": 123, + "url": "https://github.com/bazel-contrib/rules_python/issues/123", + } + + result = CompletePrepare(args, mock_gh).run() + + assert result == 0 + updated_body = mock_gh.get_issue_body(123) + assert ( + "- [x] Prepare Release | status=done pr=#456 commit= abcdef12" in updated_body + ) + + +def test_complete_prepare_with_issue_success(mock_gh): + args = argparse.Namespace(pr=None, issue=123) + mock_gh.prs[456] = { + "state": "MERGED", + "body": "Prepare release for v2.0.0", + "mergeCommit": {"oid": "1234567890abcdef"}, + } + issue_body = """ +## Checklist +- [ ] Prepare Release | status=pending pr=#456 +- [ ] Create Release branch +- [ ] Tag Final +""" + mock_gh.issues[123] = { + "title": "Release 2.0.0", + "body": issue_body, + "labels": ["type: release"], + "number": 123, + "url": "https://github.com/bazel-contrib/rules_python/issues/123", + } + + result = CompletePrepare(args, mock_gh).run() + + assert result == 0 + updated_body = mock_gh.get_issue_body(123) + assert ( + "- [x] Prepare Release | status=done pr=#456 commit= 12345678" in updated_body + ) + + +def test_complete_prepare_no_args(mock_gh): + args = argparse.Namespace(pr=None, issue=None) + result = CompletePrepare(args, mock_gh).run() + assert result == 1 + + +def test_complete_prepare_not_merged(mock_gh): + args = argparse.Namespace(pr=456, issue=123) + mock_gh.prs[456] = { + "state": "OPEN", + "body": "Prepare release for v2.0.0", + } + issue_body = """ +## Checklist +- [ ] Prepare Release | status=pending pr=#456 +""" + mock_gh.issues[123] = { + "title": "Release 2.0.0", + "body": issue_body, + "labels": ["type: release"], + "number": 123, + } + + result = CompletePrepare(args, mock_gh).run() + assert result == 1 + + +def test_complete_prepare_issue_missing_pr_task(mock_gh): + args = argparse.Namespace(pr=None, issue=123) + issue_body = """ +## Checklist +- [ ] Create Release branch +""" + mock_gh.issues[123] = { + "title": "Release 2.0.0", + "body": issue_body, + "labels": ["type: release"], + "number": 123, + } + + result = CompletePrepare(args, mock_gh).run() + assert result == 1 diff --git a/tools/private/release/complete_prepare.py b/tools/private/release/complete_prepare.py index 8a60cbf0f3..23adba9b86 100644 --- a/tools/private/release/complete_prepare.py +++ b/tools/private/release/complete_prepare.py @@ -3,7 +3,10 @@ import re from tools.private.release.gh import GitHub -from tools.private.release.release_issue import update_task_in_body +from tools.private.release.release_issue import ( + parse_checklist_state, + update_task_in_body, +) class CompletePrepare: @@ -16,46 +19,66 @@ def __init__(self, args, gh: GitHub): def run(self) -> int: """Executes the complete-prepare subcommand (Phase 2 PR merged).""" args = self.args - print(f"Completing preparation for PR #{args.pr}...") + pr_number = args.pr + issue_number = args.issue - pr_info = self.gh.get_pr_info(args.pr) - if not pr_info or pr_info.get("state") != "MERGED": - state = pr_info.get("state", "UNKNOWN") - print(f"Error: PR #{args.pr} is not merged yet (state: {state}).") + if not pr_number and not issue_number: + print("Error: Either --pr or --issue must be provided.") return 1 - # Resolve issue number from PR body - pr_body = pr_info.get("body") or "" - match = re.search(r"Work towards #(\d+)", pr_body) - if not match: - match = re.search(r"#(\d+)", pr_body) - if not match: - print( - f"Error: Could not determine tracking issue number from PR" - f" #{args.pr} body: {pr_body}" - ) + if not pr_number and issue_number: + issue_body = self.gh.get_issue_body(issue_number) + state = parse_checklist_state(issue_body) + prep_task = state.get("prepare_release") + if not prep_task or not prep_task.pr: + print( + f"Error: Could not find PR reference for 'Prepare Release'" + f" in issue #{issue_number}." + ) + return 1 + pr_number = int(prep_task.pr.lstrip("#")) + + print(f"Completing preparation for PR #{pr_number}...") + + pr_info = self.gh.get_pr_info(pr_number) + if not pr_info or pr_info.get("state") != "MERGED": + state = pr_info.get("state", "UNKNOWN") if pr_info else "NOT_FOUND" + print(f"Error: PR #{pr_number} is not merged yet (state: {state}).") return 1 - issue_num = int(match.group(1)) - print(f"Resolved tracking issue #{issue_num} from PR #{args.pr} body.") + if not issue_number: + # Resolve issue number from PR body + pr_body = pr_info.get("body") or "" + match = re.search(r"Work towards #(\d+)", pr_body) + if not match: + match = re.search(r"#(\d+)", pr_body) + if not match: + print( + f"Error: Could not determine tracking issue number from PR" + f" #{pr_number} body: {pr_body}" + ) + return 1 + + issue_number = int(match.group(1)) + print(f"Resolved tracking issue #{issue_number} from PR #{pr_number} body.") commit_sha = pr_info["mergeCommit"]["oid"] short_commit = commit_sha[:8] print( - f"PR #{args.pr} merged at commit {commit_sha}. Updating tracking issue..." + f"PR #{pr_number} merged at commit {commit_sha}. Updating tracking issue..." ) # Update checklist: mark Prepare Release as done (checked) and set SUCCESS - body = self.gh.get_issue_body(issue_num) + body = self.gh.get_issue_body(issue_number) metadata = { "status": "done", - "pr": f"#{args.pr}", + "pr": f"#{pr_number}", "commit": short_commit, } updated_body = update_task_in_body( body, "Prepare Release", checked=True, metadata=metadata ) - self.gh.update_issue_body(issue_num, updated_body) + self.gh.update_issue_body(issue_number, updated_body) print("Prepare Release task marked complete successfully!") return 0 @@ -69,9 +92,15 @@ def add_parser(cls, subparsers): parser.add_argument( "--pr", type=int, - required=True, + required=False, help="The merged preparation PR number.", ) + parser.add_argument( + "--issue", + type=int, + required=False, + help="The release tracking issue number.", + ) parser.set_defaults(command=cls.run_from_args) @classmethod From f33feb60789748bb9ca90880b92eb5d43bde6505 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Sat, 8 Aug 2026 13:32:32 -0700 Subject: [PATCH 912/922] chore(release): invoke create release branch from prepare complete and support /create-release-branch (#4017) Triggering the create release branch workflow on issue edits fired unnecessarily on unrelated edits and failed to trigger when other GitHub Actions modified the issue due to token recursion limits. Maintainers also lacked a direct comment command to manually cut the release branch from a release tracking issue if needed. Invoke the create release branch workflow automatically upon completion of the prepare workflow, support manual and reusable workflow dispatch with a tracking issue input, add `/create-release-branch` comment command support, and introduce a shared helper for writing GitHub Actions step outputs across release tools. --- .../release_tracking_template.md | 1 + .github/workflows/on_comment.yaml | 11 +++++++ .../workflows/release_complete_prepare.yaml | 10 +++++++ .../release_create_release_branch.yaml | 23 ++++++++++----- RELEASING.md | 2 ++ .../private/release/complete_prepare_test.py | 29 +++++++++++++++++++ .../private/release/release_test_helper.py | 6 +++- tools/private/release/complete_prepare.py | 4 +++ tools/private/release/create_rc.py | 8 ++--- tools/private/release/promote.py | 5 ++-- tools/private/release/utils.py | 7 +++++ 11 files changed, 90 insertions(+), 16 deletions(-) diff --git a/.github/ISSUE_TEMPLATE/release_tracking_template.md b/.github/ISSUE_TEMPLATE/release_tracking_template.md index 349de8c506..ab634e75f3 100644 --- a/.github/ISSUE_TEMPLATE/release_tracking_template.md +++ b/.github/ISSUE_TEMPLATE/release_tracking_template.md @@ -24,6 +24,7 @@ To manually control the release flow, see the [RELEASING.md: Manual Editing](htt Comment commands: - `/prepare`: Determines version, creates tracking issue and preparation PR. - `/prepare-complete [PR]`: Marks preparation task as complete. +- `/create-release-branch`: Cuts and pushes the release branch. - `/create-rc`: Tags and publishes a new release candidate (RC). - `/process-backports`: Cherry-picks pending backports. - `/add-backports `: Adds PRs to the backports and processes backports. diff --git a/.github/workflows/on_comment.yaml b/.github/workflows/on_comment.yaml index e2f742ba8c..a4330a0d6f 100644 --- a/.github/workflows/on_comment.yaml +++ b/.github/workflows/on_comment.yaml @@ -57,6 +57,9 @@ jobs: if [ -n "$pr_arg" ]; then echo "pr_number=$pr_arg" >> "$GITHUB_OUTPUT" fi + # Handle /create-release-branch comment + elif echo "$COMMENT_BODY" | grep -qE '^[[:space:]]*/create-release-branch([[:space:]]|$)'; then + echo "command=create-release-branch" >> "$GITHUB_OUTPUT" # Handle /prepare comment elif echo "$COMMENT_BODY" | grep -qE '^[[:space:]]*/prepare([[:space:]]|$)'; then echo "command=prepare" >> "$GITHUB_OUTPUT" @@ -135,6 +138,14 @@ jobs: issue: ${{ needs.parse_comment.outputs.issue_number }} secrets: inherit + call_create_release_branch: + needs: parse_comment + if: needs.parse_comment.outputs.command == 'create-release-branch' + uses: ./.github/workflows/release_create_release_branch.yaml + with: + issue: ${{ needs.parse_comment.outputs.issue_number }} + secrets: inherit + call_prepare: needs: parse_comment if: needs.parse_comment.outputs.command == 'prepare' diff --git a/.github/workflows/release_complete_prepare.yaml b/.github/workflows/release_complete_prepare.yaml index a084b6f682..2496dd9cd2 100644 --- a/.github/workflows/release_complete_prepare.yaml +++ b/.github/workflows/release_complete_prepare.yaml @@ -37,6 +37,8 @@ jobs: (github.event.pull_request.merged == true && contains(github.event.pull_request.labels.*.name, 'release-prepared')) runs-on: ubuntu-latest + outputs: + issue: ${{ steps.complete_prepare.outputs.issue }} steps: - name: Checkout repository uses: actions/checkout@v7 @@ -49,6 +51,7 @@ jobs: bazelisk-version: 1.20.0 - name: Mark Prepare Release Complete + id: complete_prepare env: GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} PR_NUMBER: ${{ inputs.pr || github.event.pull_request.number }} @@ -66,3 +69,10 @@ jobs: # Run the complete-prepare subcommand in the release tool to cleanly update checklist metadata bazel run //tools/private/release -- \ complete-prepare "${ARGS[@]}" + + call_create_release_branch: + needs: complete_prepare + uses: ./.github/workflows/release_create_release_branch.yaml + with: + issue: ${{ needs.complete_prepare.outputs.issue }} + secrets: inherit diff --git a/.github/workflows/release_create_release_branch.yaml b/.github/workflows/release_create_release_branch.yaml index 4cbabc7d12..d251b1e394 100644 --- a/.github/workflows/release_create_release_branch.yaml +++ b/.github/workflows/release_create_release_branch.yaml @@ -1,8 +1,18 @@ name: "Release: Create Release Branch" on: - issues: - types: [edited] + workflow_dispatch: + inputs: + issue: + description: 'The Release Tracking Issue Number (e.g., 142)' + required: true + type: string + workflow_call: + inputs: + issue: + description: 'The Release Tracking Issue Number (e.g., 142)' + required: true + type: string permissions: contents: write @@ -10,8 +20,6 @@ permissions: jobs: cut_branch: - # Run only if the issue has the type: release label - if: "contains(github.event.issue.labels.*.name, 'type: release')" runs-on: ubuntu-latest steps: - name: Checkout repository @@ -30,11 +38,11 @@ jobs: git config --global user.email "41898282+github-actions[bot]@users.noreply.github.com" - name: Attempt Branch Creation - run: | - bazel run //tools/private/release -- \ - create-release-branch --issue ${{ github.event.issue.number }} --remote origin env: GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} + run: | + bazel run //tools/private/release -- \ + create-release-branch --issue "${{ inputs.issue }}" --remote origin # A no-op job that always runs to prevent "no jobs ran" failures # when the main job is skipped. @@ -44,3 +52,4 @@ jobs: - name: Echo Success run: echo "Success" + diff --git a/RELEASING.md b/RELEASING.md index badf33fb7c..f16b13e582 100644 --- a/RELEASING.md +++ b/RELEASING.md @@ -259,6 +259,8 @@ The checklist items use metadata suffix: `| key=value key2=value2`. * **Retry Prepare Release**: Reset the task to `- [ ] Prepare Release | status=awaiting-preparation`. * **Complete Prepare Release**: Comment `/prepare-complete` on the tracking issue (or on the preparation PR) to mark the preparation task as done. +* **Create Release Branch**: Comment `/create-release-branch` on the tracking + issue to cut and push the release branch. * **Force Task Done**: Check the box `- [x]` and add appropriate metadata (e.g. `status=done`). ## Secrets diff --git a/tests/tools/private/release/complete_prepare_test.py b/tests/tools/private/release/complete_prepare_test.py index 4f28cce66e..b710c0d0c9 100644 --- a/tests/tools/private/release/complete_prepare_test.py +++ b/tests/tools/private/release/complete_prepare_test.py @@ -35,6 +35,35 @@ def test_complete_prepare_with_pr_success(mock_gh): ) +def test_complete_prepare_writes_github_output(release_tool_env, mock_gh): + args = argparse.Namespace(pr=456, issue=None) + mock_gh.prs[456] = { + "state": "MERGED", + "body": "Prepare release for v2.0.0\n\nWork towards #123", + "mergeCommit": {"oid": "abcdef1234567890"}, + } + issue_body = """ +## Checklist +- [ ] Prepare Release | status=pending pr=#456 +- [ ] Create Release branch +- [ ] Tag Final +""" + mock_gh.issues[123] = { + "title": "Release 2.0.0", + "body": issue_body, + "labels": ["type: release"], + "number": 123, + "url": "https://github.com/bazel-contrib/rules_python/issues/123", + } + + result = CompletePrepare(args, mock_gh).run() + assert result == 0 + assert release_tool_env.github_output_file.exists() + assert ( + release_tool_env.github_output_file.read_text(encoding="utf-8") == "issue=123\n" + ) + + def test_complete_prepare_with_issue_success(mock_gh): args = argparse.Namespace(pr=None, issue=123) mock_gh.prs[456] = { diff --git a/tests/tools/private/release/release_test_helper.py b/tests/tools/private/release/release_test_helper.py index c738d834f9..0b1c4e0351 100644 --- a/tests/tools/private/release/release_test_helper.py +++ b/tests/tools/private/release/release_test_helper.py @@ -13,9 +13,11 @@ class ReleaseToolEnv: Attributes: git_root: The root path of the temporary Git repository workspace. + github_output_file: Path to the mocked GITHUB_OUTPUT file. """ git_root: Path + github_output_file: Path DEFAULT_RELEASE_TEMPLATE_CONTENT = ( @@ -55,4 +57,6 @@ def fixture_release_tool_env(tmp_path, monkeypatch): template_dir.mkdir(parents=True, exist_ok=True) template_file = template_dir / "release_tracking_template.md" template_file.write_text(DEFAULT_RELEASE_TEMPLATE_CONTENT, encoding="utf-8") - yield ReleaseToolEnv(git_root=tmp_path) + github_output_file = tmp_path / "github_output" + monkeypatch.setenv("GITHUB_OUTPUT", str(github_output_file)) + yield ReleaseToolEnv(git_root=tmp_path, github_output_file=github_output_file) diff --git a/tools/private/release/complete_prepare.py b/tools/private/release/complete_prepare.py index 23adba9b86..e585a53e8c 100644 --- a/tools/private/release/complete_prepare.py +++ b/tools/private/release/complete_prepare.py @@ -7,6 +7,7 @@ parse_checklist_state, update_task_in_body, ) +from tools.private.release.utils import set_github_output class CompletePrepare: @@ -80,6 +81,9 @@ def run(self) -> int: ) self.gh.update_issue_body(issue_number, updated_body) print("Prepare Release task marked complete successfully!") + + set_github_output("issue", str(issue_number)) + return 0 @classmethod diff --git a/tools/private/release/create_rc.py b/tools/private/release/create_rc.py index 2e61e7d80a..0987bb3e43 100644 --- a/tools/private/release/create_rc.py +++ b/tools/private/release/create_rc.py @@ -1,5 +1,6 @@ """Subcommand to tag and push the next release candidate.""" +import os import traceback from argparse import Namespace @@ -16,6 +17,7 @@ from tools.private.release.utils import ( REPO_URL, get_latest_rc_tag, + set_github_output, ) @@ -142,11 +144,7 @@ def _run_internal(self) -> int: self.git.tag(next_rc, target_ref) self.git.push(args.remote, next_rc) - import os - - if "GITHUB_OUTPUT" in os.environ: - with open(os.environ["GITHUB_OUTPUT"], "a") as f: - f.write(f"tag_name={next_rc}\n") + set_github_output("tag_name", next_rc) # Check off the appropriate "Tag RC{N}" task in the checklist print(f"Checking off Tag RC{next_rc_num} task...") diff --git a/tools/private/release/promote.py b/tools/private/release/promote.py index 66d5b74324..560c531383 100644 --- a/tools/private/release/promote.py +++ b/tools/private/release/promote.py @@ -15,6 +15,7 @@ determine_next_version, get_latest_rc_tag, semver_type, + set_github_output, ) @@ -170,9 +171,7 @@ def run(self) -> int: self.git.tag(version, commit_sha) self.git.push(args.remote, version) - if github_output := os.environ.get("GITHUB_OUTPUT"): - with open(github_output, "a", encoding="utf-8") as f: - f.write(f"version={version}\n") + set_github_output("version", version) print(f"Updating tracking issue #{issue_num} checklist...") self.gh.update_issue_body(issue_num, updated_body) diff --git a/tools/private/release/utils.py b/tools/private/release/utils.py index 8cbbe89a41..ab1e0acef3 100644 --- a/tools/private/release/utils.py +++ b/tools/private/release/utils.py @@ -184,3 +184,10 @@ def parse_pr_list(value: str) -> list[str]: return [] # Split by space and/or comma return [p for p in re.split(r"[\s,]+", value.strip()) if p] + + +def set_github_output(name: str, value: str) -> None: + """Sets a GitHub Actions output parameter if GITHUB_OUTPUT is set.""" + if github_output := os.environ.get("GITHUB_OUTPUT"): + with open(github_output, "a", encoding="utf-8") as f: + f.write(f"{name}={value}\n") From 028812a1297d4df4b3c69449aa9e08f9b291171f Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Sat, 8 Aug 2026 13:54:33 -0700 Subject: [PATCH 913/922] agents: optimize rules and clarify create-pr subagent requirements (#4018) Agent rule and skill definitions contained redundant prose and missed actionable guardrails for argparse attribute access, workflow input handling, and interactive PR draft linking. Streamline python.md and github_actions_workflows.md rules for token efficiency, add explicit conventions for argparse attributes and workflow inputs, and strengthen create-pr skill delegation instructions while ensuring draft artifacts are linked before requesting confirmation. --- .agents/rules/github_actions_workflows.md | 6 +- .agents/rules/python.md | 13 +++-- .agents/skills/create-pr/SKILL.md | 67 +++++++++-------------- 3 files changed, 38 insertions(+), 48 deletions(-) diff --git a/.agents/rules/github_actions_workflows.md b/.agents/rules/github_actions_workflows.md index 29bf628a24..525d2a2699 100644 --- a/.agents/rules/github_actions_workflows.md +++ b/.agents/rules/github_actions_workflows.md @@ -6,5 +6,7 @@ globs: [".github/workflows/*.yml", ".github/workflows/*.yaml", ".github/*.yaml"] # GitHub Actions Workflows Rule -* When creating files in `.github/workflows/` (such as `.yml` or `.yaml` - files), always use the latest version of the referenced GitHub Actions. +* Use the latest version of referenced actions in `.github/workflows/`. +* Preserve `suppress-no-jobs-ran-error` fallback jobs in conditional workflows. +* Pass inputs (e.g. `${{ inputs.issue }}`) directly to commands without + redundant shell parameter stripping or conversions. diff --git a/.agents/rules/python.md b/.agents/rules/python.md index 523cc923cd..22221fe3a1 100644 --- a/.agents/rules/python.md +++ b/.agents/rules/python.md @@ -1,8 +1,11 @@ # Python Conventions ## pytest -* **Fixture Registration via `pytest_plugins`**: When registering pytest helper - modules in test files, use `pytest_plugins = [""]`. -* **Fixture Naming Conventions**: Name fixture functions with a `fixture_` prefix - (e.g. `def fixture_foo():`), and pass the public fixture name using the `name` - parameter in `@pytest.fixture(name="foo")`. +* Register helper fixtures using `pytest_plugins = [""]`. +* Name fixture functions with `fixture_` prefix and pass public name via + `@pytest.fixture(name="foo")`. + +## CLI & Arguments +* Use direct attribute access (e.g. `args.foo`) on `argparse.Namespace` with + well-defined shapes. Avoid defensive `getattr()`. + diff --git a/.agents/skills/create-pr/SKILL.md b/.agents/skills/create-pr/SKILL.md index 88b687b39f..97abf1d467 100644 --- a/.agents/skills/create-pr/SKILL.md +++ b/.agents/skills/create-pr/SKILL.md @@ -4,51 +4,36 @@ description: Propose, draft, or create a pull request by delegating to a subagent --- -When creating a Pull Request for local changes or a branch, invoke a subagent -to handle PR creation or description drafting. +When proposing, drafting, or creating a Pull Request, you MUST ALWAYS delegate +to a subagent. NEVER create or draft PRs directly in the main conversation. ### Instructions -1. **Pre-Review Audit Subagent**: Before drafting or creating a PR, launch a - subagent to run the `pre-review-audit` skill on local changes (`git diff`). - Verify all checks pass (e.g., `VERSION_NEXT_FEATURE` directives, no Bazel - copyright headers, line wrapping, Starlark formatting). Fix any issues - found before proceeding. -2. Launch a subagent using `invoke_subagent` with `TypeName: "self"` (or - `agentapi new-conversation`). -3. Provide a prompt to the subagent directing it to: - - Include `@/.agents/rules/pr.md` and read `CONTRIBUTING.md` (specifically - the sections on **Commit messages and PR descriptions** and - **Documenting changes**) before drafting. - - Strictly adhere to `CONTRIBUTING.md` rules for: - - **PR Title**: Follow conventional commit style and title formatting. - For agent rules, skills, and system updates, use `agents:` prefix. - - **PR Body**: Include rationale, high-level summary, and structure. - **CRITICAL**: Strictly wrap all PR body text at 72 columns max - (GitHub uses the PR description as the commit message upon merge, - which reflows text at 72 columns). - - **Formatting**: Follow repository style guidelines and structure. - - Create a Markdown artifact (`pr_info.md`) meeting the following requirements: - - **User-facing**: Published so it is presented directly in the user interface. - - **Interactive feedback enabled**: Allows the user to select lines and leave inline comments on the draft. - - **User decision choices**: Ask the user if they want to: +1. **Pre-Review Audit**: Launch a subagent with `pre-review-audit` to verify + `git diff` conforms to all rules (line wrapping, copyright, conventions). +2. **Launch Subagent**: Use `invoke_subagent` (`TypeName: "self"`). +3. **Subagent Prompt Instructions**: + - Follow `CONTRIBUTING.md` and `@/.agents/rules/pr.md`. + - **PR Title**: Conventional commits format (`agents:` prefix for agent + rules/skills). + - **PR Body**: Explain *why* and conceptual *how*. Wrap strictly at 72 + columns max. Omit TAG/CONV. + - **Artifact Requirements**: Create `pr_info.md` with: + - **User-facing**: Published directly in the user interface. + - **Interactive feedback enabled**: Allows selecting lines and leaving + inline comments on the draft (`RequestFeedback: true`). + - **User decision choices**: Present choices: 1. Create a regular PR 2. Create a draft PR 3. Provide feedback on the draft text 4. Discard the draft text - - **Propose vs. Create**: If the user requested to propose or draft a PR - description, **do not** run `gh pr create`—just create the `pr_info.md` - artifact for the user to review. Otherwise, execute `gh pr create` with - the formatted title and body. - - **Targeting Upstream Repo**: When executing `gh pr create`, always target - the upstream repository by passing `--repo bazel-contrib/rules_python` and - `--head :`. -4. **Return Status**: Direct the subagent to communicate the PR number or draft - status back using `send_message` (or `agentapi send-message`) with the - parent conversation ID, or include it in its final completion response. -5. **Publish Artifact**: Upon receiving the subagent completion message, the - main agent must publish `pr_info.md` to display the artifact directly in - the primary user UI. -6. **Interactive Actions**: To present custom action choices to the user - (e.g., "Create PR", "Create Draft PR"), the main agent can use the - `ask_question` tool with custom options. + - **Propose vs. Create**: If proposing or drafting a PR, do NOT run + `gh pr create`—only create the `pr_info.md` artifact. Only execute + `gh pr create` when explicitly requested to create the PR. + - **Targeting Upstream Repo**: When creating, target upstream using + `--repo bazel-contrib/rules_python` and `--head :`. +4. **Return Status**: Direct subagent to report PR number/draft status to caller. +5. **Link Artifact Before Asking**: Upon subagent completion, output a + clickable markdown link to `pr_info.md` before prompting for confirmation. +6. **Interactive Actions**: When presenting choices via `ask_question`, always + include the clickable markdown link to `pr_info.md` in the `question` prompt. From a2da0a9c8c15d996d329bbcc20919ef20b47c0f3 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Sat, 8 Aug 2026 15:27:41 -0700 Subject: [PATCH 914/922] agents: document Windows machine normalization and SOABI support (#4019) Agents resolving platform tags on Windows may encounter uppercase strings from platform.machine(), leading to mismatch errors when matching PEP 508 tags. Additionally, agents lacked guidance on Windows CPython native support for SOABI ABI infix tags. Document the requirement to normalize platform.machine() to lowercase and clarify native SOABI platform tag support for Windows CPython. --- .agents/rules/windows.md | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/.agents/rules/windows.md b/.agents/rules/windows.md index 5948092902..2eea9b2df5 100644 --- a/.agents/rules/windows.md +++ b/.agents/rules/windows.md @@ -18,4 +18,15 @@ ## Windows Platform Tag Derivation * Windows platform tags for Python C extensions evaluate to `win_amd64` (x86_64/amd64), `win_arm64` (aarch64/arm64), or `win32` (32-bit x86). +* **`platform.machine()` Normalization**: On Windows, + `platform.machine()` returns uppercase (`"AMD64"`, `"ARM64"`). Always + normalize with `.lower()` when deriving PEP 508 markers or resolving platform + tags. * **Citation**: [PEP 425 — Compatibility Tags for Built Distributions](https://peps.python.org/pep-0425/). + +## Windows CPython SOABI & ABI Infix Support +* CPython on Windows natively supports loading ABI-tagged `.pyd` files + containing platform tags (e.g., `foo.cp314-win_amd64.pyd`, + `foo.cp311-win_amd64.pyd`). +* SOABI on Windows includes both the ABI prefix and platform tag (e.g., + `cp311-win_amd64`). From 13e21f26c23b0c0bc209bde0d9e42cdecb3a1fd3 Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Sat, 8 Aug 2026 18:06:58 -0700 Subject: [PATCH 915/922] build(deps): bump cryptography from 49.0.0 to 50.0.0 in /tools/publish (#3995) Bumps [cryptography](https://github.com/pyca/cryptography) from 49.0.0 to 50.0.0.

    Changelog

    Sourced from cryptography's changelog.

    50.0.0 - 2026-07-31

    
    * **SECURITY ISSUE**:
    
    :func:`~cryptography.hazmat.primitives.serialization.pkcs7.pkcs7_decrypt_der`
    and its PEM and S/MIME variants no longer expose distinguishable errors
    or
    timing when unwrapping a ``RecipientInfo``'s ``encryptedKey``, which
    could
    act as a Bleichenbacher oracle for callers that decrypt untrusted
    messages.
    A random key is now substituted on failure, as described in :rfc:`3218`.
      Credit to **@X1AOxiang** for reporting the issue
    * Deprecated Diffie-Hellman key exchange over finite fields (FFDH).
      Everything FFDH is deprecated, including the types in
    ``cryptography.hazmat.primitives.asymmetric.dh`` and loading FFDH keys
    or
      parameters with the key loading APIs. Users should migrate to a more
      modern key exchange algorithm.
    * Added ``xof()`` class methods to
      :class:`~cryptography.hazmat.primitives.hashes.SHAKE128` and
    :class:`~cryptography.hazmat.primitives.hashes.SHAKE256` for
    constructing
      algorithm instances configured for use with
      :class:`~cryptography.hazmat.primitives.hashes.XOFHash`.
    * The :mod:`X.509 verification <cryptography.x509.verification>`
    APIs are now
      considered stable and are subject to our API stability policy.
    * Added the :doc:`/cobblestone` recipe, an implementation of the
      Cobblestone-128 and Cobblestone-256 instantiations of the `C2SP
      chunked-encryption specification
    <https://c2sp.org/chunked-encryption>`_ for streaming
    authenticated
      encryption of large messages.
    * Parsing a Signed Certificate Timestamp list now rejects encodings that
    carry trailing bytes after the list or after an individual SCT, instead
    of
      silently ignoring them.
    * Added support for using :class:`~cryptography.x509.Name` as a field
    type in
      the :doc:`/hazmat/asn1/index` module.
    * Loading a public key or an EC private key now rejects DER where the
    ``subjectPublicKey`` (or EC ``publicKey``) ``BIT STRING`` declares a
    non-zero
      number of unused bits, instead of silently ignoring it.
    * Parsing a CRL entry's ``InvalidityDate`` extension now rejects a
    ``GeneralizedTime`` that carries fractional seconds or another non-DER
    form,
    matching the strict encoding already required for every other X.509 time
      field.
    * :func:`~cryptography.x509.ocsp.load_der_ocsp_request` and
    :func:`~cryptography.x509.ocsp.load_der_ocsp_response` now reject a
    request
    or response whose ``version`` field is not ``v1``, the only version
    defined
    by RFC 6960, matching the version validation already performed when
    loading
      certificates, CSRs and CRLs.
    * :class:`~cryptography.hazmat.primitives.hashes.XOFHash` is now
    supported
      when building against AWS-LC.
    * HMAC (and therefore PBKDF2-HMAC) with SHA-3 hashes is now supported
    when
      building against AWS-LC.
    * Diffie-Hellman (:doc:`/hazmat/primitives/asymmetric/dh`) is now
    supported
      when building against AWS-LC.
    </tr></table>
    

    ... (truncated)

    Commits

    [![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=cryptography&package-manager=pip&previous-version=49.0.0&new-version=50.0.0)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) ---
    Dependabot commands and options
    You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot show ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> --- tools/publish/requirements_linux.txt | 94 ++++++++++++------------ tools/publish/requirements_universal.txt | 94 ++++++++++++------------ 2 files changed, 94 insertions(+), 94 deletions(-) diff --git a/tools/publish/requirements_linux.txt b/tools/publish/requirements_linux.txt index 41b0755b12..b786f5d029 100644 --- a/tools/publish/requirements_linux.txt +++ b/tools/publish/requirements_linux.txt @@ -207,53 +207,53 @@ charset-normalizer==3.4.9 \ --hash=sha256:fa36ec09ef71d158186bc79e359ff5fdd6e7996fe8ab638f00d6b93139ba4fcf \ --hash=sha256:fe2c7201c642b7c308f1675355ad7ff7b66acfe3541625efe5a3ad38f29d6115 # via requests -cryptography==49.0.0 \ - --hash=sha256:026ac7423e6fa66872d3bf889be5974507da3944f866f704fa200eadacd00001 \ - --hash=sha256:07cab27cc7b7e0fd28e5e26bb9eeedde5c135c868b46de4a27845abe94af6122 \ - --hash=sha256:084ef1af862eb07ec46d25f68689f2102a9fc0e05ce7b80f14f5fe51e4eef0f6 \ - --hash=sha256:0b82e28ee398a386f0807bba7884d30f25218855690f45115831bcce5d90822c \ - --hash=sha256:0e959b578856a3924bc0cbb710fc12c387b9412a951389f3ca61704a9e25f325 \ - --hash=sha256:0f21641cf4b30fca7aee061ced0ec7ad7b073518088b7c9969a297c0ae796c69 \ - --hash=sha256:196ecd6a36e4e9aa10270393bb98d8df88fccee0bf1e5128b91ae4eb4375896d \ - --hash=sha256:2400ef9c9e2299a25614eb1dea3db54a69b1349efd043bfac9c67630d136df36 \ - --hash=sha256:28d8b15e6275f12c8a207dc309dfa957903c927d08d0cc937ee3f63f200693cc \ - --hash=sha256:2afe9051da7ae7bd5905da5a949280c7d2bb75682e188f650a9d0f2756b834c6 \ - --hash=sha256:2eda353d8a27bcbcaa4cbed18994a74ab4d19a2ca897db188ea269ab9b71419b \ - --hash=sha256:32703d93296f5c1f4b53349ad3a250c2cae0fdecd3a3dd5d47e616d8d616af27 \ - --hash=sha256:33cd0565932807baddb67b96dbee92f2c374b5c89dee09fd74079aeb8c8dba61 \ - --hash=sha256:35b151772baff2c74cba7fa290ceaff4c3b11c0c881eb93eb5dbc05a7cfbba18 \ - --hash=sha256:36d1709f992593689b45bda411498d62c6e365f2ca00b84657d4dadd24de16db \ - --hash=sha256:42b0684e0e40cf26122427802486f6d93aea593612603a94fbf260c7eb1e9c1b \ - --hash=sha256:4ae387c9cb68ea569ca17e490d66d8142b81c3cc814bf179974b7d146e490bbb \ - --hash=sha256:53ecee2e23f7169b6117e99fc8a944e5e50f79e69758a83b52a00cb98ab2b2d2 \ - --hash=sha256:66ec79c3904820572d7e987abdf304281f141d37ad9a489b8e97066e7b9b6459 \ - --hash=sha256:67e1d20ad9ef3a563c59ef22e7a8a0b8210bd26604369ea4a30a7c66aefe504e \ - --hash=sha256:6f2debedf9ca60cf1d5bd466475638af5130f89965605cd818484d19987d3a21 \ - --hash=sha256:6fc361c34fb6aac015ce19435876635e5c6d21db31998b0920f675f131e043b8 \ - --hash=sha256:73a205dce83953d131a4aa1e0fd917a2fd1c5b1eef251e9d7152efefcbf5caf7 \ - --hash=sha256:7abcee80084cda3f7691f3eb1ce480d8df49cec637b429aa35986c1de71738aa \ - --hash=sha256:8c25ceb16df5b9435f3f6a9829204985b0e0cbee3b48aacd432c7d2c850b44d9 \ - --hash=sha256:966fe0e9c67490071f14c0d2b1cb2dfb3023c5ce39457343931415f08382f2db \ - --hash=sha256:9e82dcc8e56052715fb18b2429e3bca4823b1629136a2084fc45a9a5cecb9b64 \ - --hash=sha256:b20133d204d2bb56ba047642199603876c872026ca53e79c35b83772ab2cc505 \ - --hash=sha256:b39efa323140595abd3ecca8529d321ae50f55f3aa3ba9cc81ea56a6011953d5 \ - --hash=sha256:b47db11c2c3525083296069b98ac5221907455e989ae0c2e3008bde851921615 \ - --hash=sha256:b87e65d263b3e5d3bb92a57e2a6638e2f31110fa7aa890c7b2dbba42248d0a3f \ - --hash=sha256:b970c6da94d5bb18629db453d14f2a1300f6bf59b61e9b82377931ef95504866 \ - --hash=sha256:be9fcb48a55f023493482827d4f459bd263cc20efde64f204b97c123201850c6 \ - --hash=sha256:c2bc30226390d60ea19d9f82b19db005fe0452154a23c1c410c12ea801e43561 \ - --hash=sha256:c83782480a4a9da4d0feb51950131ba32e12e70813848b3343f6e18c28a66838 \ - --hash=sha256:cbc77da8c523d5abd028635ba850a6966fcee2c82e2bf65a41d1d8afe0f98be9 \ - --hash=sha256:ccac2bfebc306b862133e3bb71f3f6ee8bb525240089b2d952e4144b3a6d5da7 \ - --hash=sha256:d0527ce944105f257f605a827d6ebead966c752038b6e8656abb9c5edee6fc68 \ - --hash=sha256:d8ecde755e2e91bf773fc94e8c9d730cd7f2007004cb492263a794ec3899a1c8 \ - --hash=sha256:e3fb64c420688e5319ae25113a354015abbd8dffbfbc41781a1ea66fc7622ac3 \ - --hash=sha256:e5dfc1e64de5677cec922ffa8da89c546d0415bf6efdf081842e5d44c84e1f0e \ - --hash=sha256:ec5e529fb80935c94fe7b729f9972b50e351a0e6b50aa294fd5cabb109fcc29a \ - --hash=sha256:f37d847238971164fdbc68ade6f6574aecc9c0af714190e2083429ff68f4ce9d \ - --hash=sha256:f78ff2c9ed8dc2d036b0f4d640e22522213d047c1b14e61205a7e55c80a494d4 \ - --hash=sha256:f89660a348f4f78a92366240a61404e337586ef7f5909a2fef59ca88ef505493 \ - --hash=sha256:fc1e275c2f1d97b1a6450b8b0ea3ebfa6e087a611c2b26cb2404d48588abab7b +cryptography==50.0.0 \ + --hash=sha256:031e2d5dd4bb9caa3ca9c82e5a197fd8ae680232cee62603d1a813f3f07e3d03 \ + --hash=sha256:06a32a980526a6ab9a4b9bf8f7385800791e2bb960903cb6b530e4817509a3b7 \ + --hash=sha256:07479a1cb08219ab719147e742e76090c9c773321959bb94946fffdd397a6437 \ + --hash=sha256:07949c449a1abcf60d1ee6e88956d89404c7df3c8258f46589e912988e551987 \ + --hash=sha256:105110f43a471dbd0060b9c9516cb8a6a79233631a04cc2ba16f28323ac6e025 \ + --hash=sha256:11b74db56cdbe3cdee6e3f6982ecb70334fa10dce99ed58bf7894aaaa3b2a037 \ + --hash=sha256:12b9c6996425c76ea6c457ace4f3073e715b8c545add07cd1a8f3a4f90691269 \ + --hash=sha256:1489e263a8048bb8b6a8bac662eb2d402ea5d2b7b4699b72f385f1e2772db105 \ + --hash=sha256:19736989797678c6af1e55cd49055cdbcb55d8f6b5583ac5335f933aba9101dc \ + --hash=sha256:1b4a266766514614f8aa60416e71f2fc6e575d36e7bdc90f644fadb2f4b75b95 \ + --hash=sha256:2a8183b489dc1f7f80f135780fadc1108f14b31b8a40411c7a5b17425f65f28b \ + --hash=sha256:37fdb0d0111f1e2ff07139dfb79f1b49531f8e213c46f1163dd7642979b58c47 \ + --hash=sha256:3f5735ffe4996d28b809371756219f5354864902a3b9e7c0b9ee87041209fc9c \ + --hash=sha256:49e7d93abdbd2990caced757e5fade25302f719c3c8fb6e6fff2dde98999fc41 \ + --hash=sha256:5e34edd123674534acd70147f0ca331eaa2c74e6325fb2028c886aa26ba0b68c \ + --hash=sha256:62598a8a57f815db4c6259a4e97d857dab56697e7de8e8ab02352ab74da1995d \ + --hash=sha256:65c2c3add92b45fd0709db8594536aea39c2a67af0e27ffcf049c498501140b7 \ + --hash=sha256:6ba6a53445bd3cfa809ef3ef5f1589aa6ba08784a1d962bf47d0940e871dab1c \ + --hash=sha256:6e7d61120573a7f2cd94cc095f9e81f6967c61ccdf194285aa143ecec8e0b708 \ + --hash=sha256:7cec5b856506da6defb290f30c9ee687d5f5e8cb0bd3f6459dde43b0b4fa40ef \ + --hash=sha256:80b63928fa35083b33966ce1efb70e5b9607181e49dcd1c22c8c005e319f667f \ + --hash=sha256:82148ec5bddac30b51a5b3c1945075f896fa022cb93f8e4a01e9f6ee95292c5f \ + --hash=sha256:828743d939e9629bc267b8e2d08d8bb67cd4319c771a33d4b18b22dd8fb7440a \ + --hash=sha256:8d89f3976b10b4ce31118de72329025f70d2c6ead14a8217c5514dd2c6d5a78f \ + --hash=sha256:8eb5e1172eb569ea8a872796576e6a67c276351728b6455d5beb01242b027c6a \ + --hash=sha256:900131fafd8aead39ac7dd3a7e833be754c17a95cfd91221636949fe4eb0aa8a \ + --hash=sha256:910d11e1a385c654bf738bf3e6b8e6ed5de0f5610fcae2be9e5b398d8081d20e \ + --hash=sha256:910e1d2668e7de9648f2bcee30e180db2a6b15c30f887d7c4c93ddf96e3992e3 \ + --hash=sha256:9aa87839c383bdbab6ef865787a1fb877af8dd03464c4400322726feaaadfc6d \ + --hash=sha256:a1b30560f2acc95aa8b2e06e716a13dbfc97314747b80d9707e307f77b40d6b3 \ + --hash=sha256:a91296cb61e8df6f86d0c19cc4068228da256bf59bf86049fbd821084565327f \ + --hash=sha256:b42a28c1844fd9de8f3f7d540e36b66f3a9c83fceac7170ebc7a6a19edd9dcae \ + --hash=sha256:bd1c592e4d5974f0d08d4888e432157adba757c66da0246918e43677fafa2d30 \ + --hash=sha256:c87f62a3d3b9888ed0fdde100ec06aa61ca9cd44bad9057d1dff9a516b5f5bb9 \ + --hash=sha256:c99c003e088647b8a5b7c145d6f78c335f6348332b62e142d411c4b63d1460b9 \ + --hash=sha256:ccdc4a71a4dabae05de219404f9f4abc38e3b58422177ff93d0da05967dafa07 \ + --hash=sha256:d24fead1d4d076e1bfb006dcec392074a3cd8d7b4fc8a595aa64073b2b7a96ba \ + --hash=sha256:d58c3db7cd6eed54e6c06744db55456b65ebd7492ddeae9c1e93cfca7aa857d3 \ + --hash=sha256:d764dcf130c428ef66786f866dd750f53182bc608813489915e9fc106bb0c82f \ + --hash=sha256:df2a58a472f332225671c35b0a830208b86d004f82baa8530fa3782c85646533 \ + --hash=sha256:e722f16708d854fe924790e051061f6704a472c3bac347b6fd88033ea8dd0dc5 \ + --hash=sha256:ecfed7367f965a0328cfbdd70da860f15441f002f613185668c6e6ebf5a0ac11 \ + --hash=sha256:eeac2acb5a20ed25e0ad6d1df9891a520b78b404266b6d11778f25d5d691a6c9 \ + --hash=sha256:f59e38625469987d7ef6d495323c55e7db6c212eaf6112267e0d3b565a2e9c9f \ + --hash=sha256:f89831ef99dd7dd169ab06d63a831adb9e20a87aac6d380266bbda5823349169 \ + --hash=sha256:fd9192b7b70c573d7f214eb1ae35e00d359f6f5e4b27c7e21e30de1fc6204645 # via secretstorage docutils==0.23 \ --hash=sha256:25d013af9bf23bc1c7b2b093dff4208166c53a94786c9e447808335ef1185fea \ diff --git a/tools/publish/requirements_universal.txt b/tools/publish/requirements_universal.txt index bd17c90a54..99d317f1a7 100644 --- a/tools/publish/requirements_universal.txt +++ b/tools/publish/requirements_universal.txt @@ -207,53 +207,53 @@ charset-normalizer==3.4.9 \ --hash=sha256:fa36ec09ef71d158186bc79e359ff5fdd6e7996fe8ab638f00d6b93139ba4fcf \ --hash=sha256:fe2c7201c642b7c308f1675355ad7ff7b66acfe3541625efe5a3ad38f29d6115 # via requests -cryptography==49.0.0 ; platform_machine != 'ppc64le' and platform_machine != 's390x' and sys_platform == 'linux' \ - --hash=sha256:026ac7423e6fa66872d3bf889be5974507da3944f866f704fa200eadacd00001 \ - --hash=sha256:07cab27cc7b7e0fd28e5e26bb9eeedde5c135c868b46de4a27845abe94af6122 \ - --hash=sha256:084ef1af862eb07ec46d25f68689f2102a9fc0e05ce7b80f14f5fe51e4eef0f6 \ - --hash=sha256:0b82e28ee398a386f0807bba7884d30f25218855690f45115831bcce5d90822c \ - --hash=sha256:0e959b578856a3924bc0cbb710fc12c387b9412a951389f3ca61704a9e25f325 \ - --hash=sha256:0f21641cf4b30fca7aee061ced0ec7ad7b073518088b7c9969a297c0ae796c69 \ - --hash=sha256:196ecd6a36e4e9aa10270393bb98d8df88fccee0bf1e5128b91ae4eb4375896d \ - --hash=sha256:2400ef9c9e2299a25614eb1dea3db54a69b1349efd043bfac9c67630d136df36 \ - --hash=sha256:28d8b15e6275f12c8a207dc309dfa957903c927d08d0cc937ee3f63f200693cc \ - --hash=sha256:2afe9051da7ae7bd5905da5a949280c7d2bb75682e188f650a9d0f2756b834c6 \ - --hash=sha256:2eda353d8a27bcbcaa4cbed18994a74ab4d19a2ca897db188ea269ab9b71419b \ - --hash=sha256:32703d93296f5c1f4b53349ad3a250c2cae0fdecd3a3dd5d47e616d8d616af27 \ - --hash=sha256:33cd0565932807baddb67b96dbee92f2c374b5c89dee09fd74079aeb8c8dba61 \ - --hash=sha256:35b151772baff2c74cba7fa290ceaff4c3b11c0c881eb93eb5dbc05a7cfbba18 \ - --hash=sha256:36d1709f992593689b45bda411498d62c6e365f2ca00b84657d4dadd24de16db \ - --hash=sha256:42b0684e0e40cf26122427802486f6d93aea593612603a94fbf260c7eb1e9c1b \ - --hash=sha256:4ae387c9cb68ea569ca17e490d66d8142b81c3cc814bf179974b7d146e490bbb \ - --hash=sha256:53ecee2e23f7169b6117e99fc8a944e5e50f79e69758a83b52a00cb98ab2b2d2 \ - --hash=sha256:66ec79c3904820572d7e987abdf304281f141d37ad9a489b8e97066e7b9b6459 \ - --hash=sha256:67e1d20ad9ef3a563c59ef22e7a8a0b8210bd26604369ea4a30a7c66aefe504e \ - --hash=sha256:6f2debedf9ca60cf1d5bd466475638af5130f89965605cd818484d19987d3a21 \ - --hash=sha256:6fc361c34fb6aac015ce19435876635e5c6d21db31998b0920f675f131e043b8 \ - --hash=sha256:73a205dce83953d131a4aa1e0fd917a2fd1c5b1eef251e9d7152efefcbf5caf7 \ - --hash=sha256:7abcee80084cda3f7691f3eb1ce480d8df49cec637b429aa35986c1de71738aa \ - --hash=sha256:8c25ceb16df5b9435f3f6a9829204985b0e0cbee3b48aacd432c7d2c850b44d9 \ - --hash=sha256:966fe0e9c67490071f14c0d2b1cb2dfb3023c5ce39457343931415f08382f2db \ - --hash=sha256:9e82dcc8e56052715fb18b2429e3bca4823b1629136a2084fc45a9a5cecb9b64 \ - --hash=sha256:b20133d204d2bb56ba047642199603876c872026ca53e79c35b83772ab2cc505 \ - --hash=sha256:b39efa323140595abd3ecca8529d321ae50f55f3aa3ba9cc81ea56a6011953d5 \ - --hash=sha256:b47db11c2c3525083296069b98ac5221907455e989ae0c2e3008bde851921615 \ - --hash=sha256:b87e65d263b3e5d3bb92a57e2a6638e2f31110fa7aa890c7b2dbba42248d0a3f \ - --hash=sha256:b970c6da94d5bb18629db453d14f2a1300f6bf59b61e9b82377931ef95504866 \ - --hash=sha256:be9fcb48a55f023493482827d4f459bd263cc20efde64f204b97c123201850c6 \ - --hash=sha256:c2bc30226390d60ea19d9f82b19db005fe0452154a23c1c410c12ea801e43561 \ - --hash=sha256:c83782480a4a9da4d0feb51950131ba32e12e70813848b3343f6e18c28a66838 \ - --hash=sha256:cbc77da8c523d5abd028635ba850a6966fcee2c82e2bf65a41d1d8afe0f98be9 \ - --hash=sha256:ccac2bfebc306b862133e3bb71f3f6ee8bb525240089b2d952e4144b3a6d5da7 \ - --hash=sha256:d0527ce944105f257f605a827d6ebead966c752038b6e8656abb9c5edee6fc68 \ - --hash=sha256:d8ecde755e2e91bf773fc94e8c9d730cd7f2007004cb492263a794ec3899a1c8 \ - --hash=sha256:e3fb64c420688e5319ae25113a354015abbd8dffbfbc41781a1ea66fc7622ac3 \ - --hash=sha256:e5dfc1e64de5677cec922ffa8da89c546d0415bf6efdf081842e5d44c84e1f0e \ - --hash=sha256:ec5e529fb80935c94fe7b729f9972b50e351a0e6b50aa294fd5cabb109fcc29a \ - --hash=sha256:f37d847238971164fdbc68ade6f6574aecc9c0af714190e2083429ff68f4ce9d \ - --hash=sha256:f78ff2c9ed8dc2d036b0f4d640e22522213d047c1b14e61205a7e55c80a494d4 \ - --hash=sha256:f89660a348f4f78a92366240a61404e337586ef7f5909a2fef59ca88ef505493 \ - --hash=sha256:fc1e275c2f1d97b1a6450b8b0ea3ebfa6e087a611c2b26cb2404d48588abab7b +cryptography==50.0.0 ; platform_machine != 'ppc64le' and platform_machine != 's390x' and sys_platform == 'linux' \ + --hash=sha256:031e2d5dd4bb9caa3ca9c82e5a197fd8ae680232cee62603d1a813f3f07e3d03 \ + --hash=sha256:06a32a980526a6ab9a4b9bf8f7385800791e2bb960903cb6b530e4817509a3b7 \ + --hash=sha256:07479a1cb08219ab719147e742e76090c9c773321959bb94946fffdd397a6437 \ + --hash=sha256:07949c449a1abcf60d1ee6e88956d89404c7df3c8258f46589e912988e551987 \ + --hash=sha256:105110f43a471dbd0060b9c9516cb8a6a79233631a04cc2ba16f28323ac6e025 \ + --hash=sha256:11b74db56cdbe3cdee6e3f6982ecb70334fa10dce99ed58bf7894aaaa3b2a037 \ + --hash=sha256:12b9c6996425c76ea6c457ace4f3073e715b8c545add07cd1a8f3a4f90691269 \ + --hash=sha256:1489e263a8048bb8b6a8bac662eb2d402ea5d2b7b4699b72f385f1e2772db105 \ + --hash=sha256:19736989797678c6af1e55cd49055cdbcb55d8f6b5583ac5335f933aba9101dc \ + --hash=sha256:1b4a266766514614f8aa60416e71f2fc6e575d36e7bdc90f644fadb2f4b75b95 \ + --hash=sha256:2a8183b489dc1f7f80f135780fadc1108f14b31b8a40411c7a5b17425f65f28b \ + --hash=sha256:37fdb0d0111f1e2ff07139dfb79f1b49531f8e213c46f1163dd7642979b58c47 \ + --hash=sha256:3f5735ffe4996d28b809371756219f5354864902a3b9e7c0b9ee87041209fc9c \ + --hash=sha256:49e7d93abdbd2990caced757e5fade25302f719c3c8fb6e6fff2dde98999fc41 \ + --hash=sha256:5e34edd123674534acd70147f0ca331eaa2c74e6325fb2028c886aa26ba0b68c \ + --hash=sha256:62598a8a57f815db4c6259a4e97d857dab56697e7de8e8ab02352ab74da1995d \ + --hash=sha256:65c2c3add92b45fd0709db8594536aea39c2a67af0e27ffcf049c498501140b7 \ + --hash=sha256:6ba6a53445bd3cfa809ef3ef5f1589aa6ba08784a1d962bf47d0940e871dab1c \ + --hash=sha256:6e7d61120573a7f2cd94cc095f9e81f6967c61ccdf194285aa143ecec8e0b708 \ + --hash=sha256:7cec5b856506da6defb290f30c9ee687d5f5e8cb0bd3f6459dde43b0b4fa40ef \ + --hash=sha256:80b63928fa35083b33966ce1efb70e5b9607181e49dcd1c22c8c005e319f667f \ + --hash=sha256:82148ec5bddac30b51a5b3c1945075f896fa022cb93f8e4a01e9f6ee95292c5f \ + --hash=sha256:828743d939e9629bc267b8e2d08d8bb67cd4319c771a33d4b18b22dd8fb7440a \ + --hash=sha256:8d89f3976b10b4ce31118de72329025f70d2c6ead14a8217c5514dd2c6d5a78f \ + --hash=sha256:8eb5e1172eb569ea8a872796576e6a67c276351728b6455d5beb01242b027c6a \ + --hash=sha256:900131fafd8aead39ac7dd3a7e833be754c17a95cfd91221636949fe4eb0aa8a \ + --hash=sha256:910d11e1a385c654bf738bf3e6b8e6ed5de0f5610fcae2be9e5b398d8081d20e \ + --hash=sha256:910e1d2668e7de9648f2bcee30e180db2a6b15c30f887d7c4c93ddf96e3992e3 \ + --hash=sha256:9aa87839c383bdbab6ef865787a1fb877af8dd03464c4400322726feaaadfc6d \ + --hash=sha256:a1b30560f2acc95aa8b2e06e716a13dbfc97314747b80d9707e307f77b40d6b3 \ + --hash=sha256:a91296cb61e8df6f86d0c19cc4068228da256bf59bf86049fbd821084565327f \ + --hash=sha256:b42a28c1844fd9de8f3f7d540e36b66f3a9c83fceac7170ebc7a6a19edd9dcae \ + --hash=sha256:bd1c592e4d5974f0d08d4888e432157adba757c66da0246918e43677fafa2d30 \ + --hash=sha256:c87f62a3d3b9888ed0fdde100ec06aa61ca9cd44bad9057d1dff9a516b5f5bb9 \ + --hash=sha256:c99c003e088647b8a5b7c145d6f78c335f6348332b62e142d411c4b63d1460b9 \ + --hash=sha256:ccdc4a71a4dabae05de219404f9f4abc38e3b58422177ff93d0da05967dafa07 \ + --hash=sha256:d24fead1d4d076e1bfb006dcec392074a3cd8d7b4fc8a595aa64073b2b7a96ba \ + --hash=sha256:d58c3db7cd6eed54e6c06744db55456b65ebd7492ddeae9c1e93cfca7aa857d3 \ + --hash=sha256:d764dcf130c428ef66786f866dd750f53182bc608813489915e9fc106bb0c82f \ + --hash=sha256:df2a58a472f332225671c35b0a830208b86d004f82baa8530fa3782c85646533 \ + --hash=sha256:e722f16708d854fe924790e051061f6704a472c3bac347b6fd88033ea8dd0dc5 \ + --hash=sha256:ecfed7367f965a0328cfbdd70da860f15441f002f613185668c6e6ebf5a0ac11 \ + --hash=sha256:eeac2acb5a20ed25e0ad6d1df9891a520b78b404266b6d11778f25d5d691a6c9 \ + --hash=sha256:f59e38625469987d7ef6d495323c55e7db6c212eaf6112267e0d3b565a2e9c9f \ + --hash=sha256:f89831ef99dd7dd169ab06d63a831adb9e20a87aac6d380266bbda5823349169 \ + --hash=sha256:fd9192b7b70c573d7f214eb1ae35e00d359f6f5e4b27c7e21e30de1fc6204645 # via secretstorage docutils==0.23 \ --hash=sha256:25d013af9bf23bc1c7b2b093dff4208166c53a94786c9e447808335ef1185fea \ From 39a37b8eeb384bc19152f3e42d512108c18477e2 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Sat, 8 Aug 2026 18:29:07 -0700 Subject: [PATCH 916/922] agents: add merge queue CI monitoring and lockfile sync rules (#4021) CI monitoring previously missed runs executed on GitHub merge queue temporary branches (`gh-readonly-queue/...`), and integration lockfiles became out of sync when requirements were bumped. Update merge shepherd guidance to discover and monitor merge queue branch statuses, and add testing rules to synchronize Bazel lockfiles in integration test workspaces whenever dependencies or requirements files change. --- .agents/rules/testing.md | 5 +++++ .agents/skills/merge-pr/SKILL.md | 12 ++++++++++-- 2 files changed, 15 insertions(+), 2 deletions(-) diff --git a/.agents/rules/testing.md b/.agents/rules/testing.md index 0f369b3861..769197f968 100644 --- a/.agents/rules/testing.md +++ b/.agents/rules/testing.md @@ -13,6 +13,11 @@ * Changes to transitive module extension dependencies or `.bzl` files loaded by extensions update Bazel 9 lockfile hashes, requiring `bazel mod deps --lockfile_mode=update` in integration test workspaces. +* When requirements files (e.g., in `//tools/publish` or root pip parses) are + modified or bumped by Dependabot, update the integration lockfile by running + `bazel mod deps --lockfile_mode=update` in `tests/integration/bzlmod_lockfile` + and verify with + `bazel test //tests/integration:bzlmod_lockfile_test_bazel_9.1.0`. ## Documentation Flake Handling * When building `//docs:docs` fails with exit code 2, treat it as a known diff --git a/.agents/skills/merge-pr/SKILL.md b/.agents/skills/merge-pr/SKILL.md index 30b8f38af4..406590e49f 100644 --- a/.agents/skills/merge-pr/SKILL.md +++ b/.agents/skills/merge-pr/SKILL.md @@ -13,7 +13,15 @@ When the user asks to merge a pull request (e.g., "merge PR ", "merge th merge. Never invoke `--admin` autonomously. 2. **Invoke a Background Shepherd**: Launch a background subagent with the role `Merge PR Shepherd` to continuously watch the PR until it merges. 3. **Leverage Existing CI Skills**: - - Have the subagent use the **`monitor-ci-results`** skill to watch for CI check failures and generate analysis reports. - - Have the subagent use the **`buildkite-retry-job`** skill (`retry_buildkite_jobs.py `) to automatically retry any transient network flakes (e.g., HTTP 504 gateway timeouts, downloader errors). + - Have the subagent use the **`monitor-ci-results`** skill to watch for CI + check failures and generate analysis reports. + - Have the subagent use the **`buildkite-retry-job`** skill + (`retry_buildkite_jobs.py `) to automatically retry any + transient network flakes (e.g., HTTP 504 gateway timeouts, downloader + errors). + - When the PR is queued, actively discover the merge queue branch via + `gh api repos/:owner/:repo/branches --jq '.[].name | select(test("gh-readonly-queue/.*/pr--"))'` + and monitor commit statuses/Buildkite builds running on that temporary + branch. 4. **Queue Shepherding**: Periodically check `gh pr view --json state,autoMergeRequest`. While `state` is `"OPEN"`, ensure auto-merge is enabled / queued by running `gh pr merge --auto --squash`. If `autoMergeRequest` is null (e.g., ejected from the merge queue due to a CI flake in the temporary queue branch), re-enqueue it for merge by running `gh pr merge --auto --squash` once checks are retried or green. 5. **Completion Notification**: Once `state` becomes `"MERGED"`, send a high-priority message back to the parent conversation. From 88c24b677d1fe5525a0b252f5c945e9cf79d8daf Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Sat, 8 Aug 2026 19:33:59 -0700 Subject: [PATCH 917/922] agents(rules): document linking external objects in TypedDict docstrings (#4022) When defining TypedDict structures that represent external objects or schemas, agents and developers need clear provenance and context for the fields and their semantics. Update the agent Python rules to establish a convention requiring TypedDict docstrings to include a link to the external object's original definition. --- .agents/rules/python.md | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/.agents/rules/python.md b/.agents/rules/python.md index 22221fe3a1..200295e650 100644 --- a/.agents/rules/python.md +++ b/.agents/rules/python.md @@ -9,3 +9,7 @@ * Use direct attribute access (e.g. `args.foo`) on `argparse.Namespace` with well-defined shapes. Avoid defensive `getattr()`. +## TypedDict +* **External Objects**: When defining a `TypedDict` for an external object, + link to its definition in the docstring. + From 99766be442009d5f805824bbd6a183539082657b Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Sun, 9 Aug 2026 00:59:16 -0700 Subject: [PATCH 918/922] agents: rename pre-review-audit to review-code and add news file checks (#4027) The pre-review audit skill lacked checks for news fragment entries, which allowed user-visible changes to slip through without proper changelog documentation. Rename the pre-review-audit skill to review-code to better reflect its scope, and add dedicated news entry file validation across the auditor sub-agent prompts. --- .agents/skills/create-pr/SKILL.md | 12 ++-- .agents/skills/pre-review-audit/SKILL.md | 40 ------------- .../review-contributing-prompt.md | 9 --- .../pre-review-audit/review-docs-prompt.md | 12 ---- .../review-pr-standards-prompt.md | 9 --- .../pre-review-audit/review-python-prompt.md | 9 --- .../review-starlark-prompt.md | 11 ---- .agents/skills/review-code/SKILL.md | 59 +++++++++++++++++++ .../review-agents-prompt.md | 22 ++++--- .../review-code/review-contributing-prompt.md | 37 ++++++++++++ .../skills/review-code/review-docs-prompt.md | 24 ++++++++ .../review-code/review-pr-standards-prompt.md | 17 ++++++ .../review-code/review-python-prompt.md | 15 +++++ .../review-code/review-starlark-prompt.md | 17 ++++++ 14 files changed, 192 insertions(+), 101 deletions(-) delete mode 100644 .agents/skills/pre-review-audit/SKILL.md delete mode 100644 .agents/skills/pre-review-audit/review-contributing-prompt.md delete mode 100644 .agents/skills/pre-review-audit/review-docs-prompt.md delete mode 100644 .agents/skills/pre-review-audit/review-pr-standards-prompt.md delete mode 100644 .agents/skills/pre-review-audit/review-python-prompt.md delete mode 100644 .agents/skills/pre-review-audit/review-starlark-prompt.md create mode 100644 .agents/skills/review-code/SKILL.md rename .agents/skills/{pre-review-audit => review-code}/review-agents-prompt.md (57%) create mode 100644 .agents/skills/review-code/review-contributing-prompt.md create mode 100644 .agents/skills/review-code/review-docs-prompt.md create mode 100644 .agents/skills/review-code/review-pr-standards-prompt.md create mode 100644 .agents/skills/review-code/review-python-prompt.md create mode 100644 .agents/skills/review-code/review-starlark-prompt.md diff --git a/.agents/skills/create-pr/SKILL.md b/.agents/skills/create-pr/SKILL.md index 97abf1d467..1c73c66424 100644 --- a/.agents/skills/create-pr/SKILL.md +++ b/.agents/skills/create-pr/SKILL.md @@ -9,8 +9,10 @@ to a subagent. NEVER create or draft PRs directly in the main conversation. ### Instructions -1. **Pre-Review Audit**: Launch a subagent with `pre-review-audit` to verify - `git diff` conforms to all rules (line wrapping, copyright, conventions). +1. **Code Review Audit**: Launch a subagent with `review-code` to verify + `git diff` conforms to all rules (news entry in `news/`, line wrapping, + copyright, conventions, Starlark formatting). Fix any issues before + proceeding. 2. **Launch Subagent**: Use `invoke_subagent` (`TypeName: "self"`). 3. **Subagent Prompt Instructions**: - Follow `CONTRIBUTING.md` and `@/.agents/rules/pr.md`. @@ -32,8 +34,10 @@ to a subagent. NEVER create or draft PRs directly in the main conversation. `gh pr create` when explicitly requested to create the PR. - **Targeting Upstream Repo**: When creating, target upstream using `--repo bazel-contrib/rules_python` and `--head :`. -4. **Return Status**: Direct subagent to report PR number/draft status to caller. +4. **Return Status**: Direct subagent to report PR number/draft status to + caller. 5. **Link Artifact Before Asking**: Upon subagent completion, output a clickable markdown link to `pr_info.md` before prompting for confirmation. 6. **Interactive Actions**: When presenting choices via `ask_question`, always - include the clickable markdown link to `pr_info.md` in the `question` prompt. + include the clickable markdown link to `pr_info.md` in the `question` + prompt. diff --git a/.agents/skills/pre-review-audit/SKILL.md b/.agents/skills/pre-review-audit/SKILL.md deleted file mode 100644 index 1c0b16e273..0000000000 --- a/.agents/skills/pre-review-audit/SKILL.md +++ /dev/null @@ -1,40 +0,0 @@ ---- -name: pre-review-audit -description: High-level pre-review audit of local changes when preparing to send a PR for review -trigger: model_decision ---- - -When preparing to send a Pull Request for review, invoke **separate, concurrent sub-agents** (`invoke_subagent`), where each sub-agent focuses exclusively on its assigned checklist category. - -**CRITICAL**: Do NOT use a single sub-agent to validate multiple or all dimensions at once. Each sub-agent must be launched with its own distinct prompt file from `.agents/skills/pre-review-audit/`: - -### Audit Checklist & Sub-Agent Prompts - -1. **Starlark / Bazel Sub-Agent** (`Role: "Starlark Code Auditor"`): - - Prompt file: `.agents/skills/pre-review-audit/review-starlark-prompt.md` - - Focus: Audits Starlark / Bazel changes (`*.bzl`, `BUILD`, `*.bazel` files) in `git diff` against `.agents/rules/bzl.md` and Starlark rules in `AGENTS.md`. - -2. **Python Code Sub-Agent** (`Role: "Python Code Auditor"`): - - Prompt file: `.agents/skills/pre-review-audit/review-python-prompt.md` - - Focus: Audits Python source and test changes in `git diff` against `.agents/rules/python.md` and Python and pytest conventions in `AGENTS.md`. - -3. **Documentation Sub-Agent** (`Role: "Documentation Auditor"`): - - Prompt file: `.agents/skills/pre-review-audit/review-docs-prompt.md` - - Focus: Audits documentation (`.md`) changes and docs build targets against `.agents/rules/docs.md` and `AGENTS.md`. - -4. **Contribution Guidelines Sub-Agent** (`Role: "Contributing Auditor"`): - - Prompt file: `.agents/skills/pre-review-audit/review-contributing-prompt.md` - - Focus: Audits changes, requirements updates, and directives against `CONTRIBUTING.md`. - -5. **Project Conventions Sub-Agent** (`Role: "Project Conventions Auditor"`): - - Prompt file: `.agents/skills/pre-review-audit/review-agents-prompt.md` - - Focus: Audits overall workspace compliance against `AGENTS.md`. - -6. **Pull Request Standards Sub-Agent** (`Role: "PR Standards Auditor"`): - - Prompt file: `.agents/skills/pre-review-audit/review-pr-standards-prompt.md` - - Focus: Audits PR titles and descriptions against Conventional Commits formatting and PR update rules in `CONTRIBUTING.md`. - -### Action Instructions -- Launch all sub-agents concurrently using `invoke_subagent`. -- Collect the reports from each sub-agent and summarize any violations clearly with suggested fixes for the user. -- If all domain audits pass, confirm that the PR is ready for review. diff --git a/.agents/skills/pre-review-audit/review-contributing-prompt.md b/.agents/skills/pre-review-audit/review-contributing-prompt.md deleted file mode 100644 index 10a719284a..0000000000 --- a/.agents/skills/pre-review-audit/review-contributing-prompt.md +++ /dev/null @@ -1,9 +0,0 @@ -You are a specialized Contribution Guidelines Auditor sub-agent. -Your sole task is to audit all local changes (`git diff`), commit messages, and PR metadata against `CONTRIBUTING.md`: - -1. Read and strictly enforce `CONTRIBUTING.md`, all `.agents/rules/*.md` files, and `AGENTS.md`. -2. Verify that `{versionadded}` and `{versionchanged}` directives use `VERSION_NEXT_FEATURE` for unreleased features. -3. If locked/resolved requirements files (`requirements.txt`, `pyproject.toml`, `requirements.in`) were modified, verify that the associated `requirements.update` target was executed to keep locked requirement files in sync. -4. Ensure style and conventions described in `CONTRIBUTING.md` are respected across the changes. - -Report any violations found clearly with actionable suggested fixes, or report that the changes pass contribution audit. diff --git a/.agents/skills/pre-review-audit/review-docs-prompt.md b/.agents/skills/pre-review-audit/review-docs-prompt.md deleted file mode 100644 index 02867940ed..0000000000 --- a/.agents/skills/pre-review-audit/review-docs-prompt.md +++ /dev/null @@ -1,12 +0,0 @@ -You are a specialized Documentation & Sphinx/MyST Auditor sub-agent. -Your sole task is to audit all documentation (`.md`) changes and new `.bzl` APIs in `git diff` against the project's documentation rules: - -1. Read and strictly enforce `.agents/rules/docs.md`, `AGENTS.md`, and `CONTRIBUTING.md`. -2. Check that lines wrap at 80 columns. -3. Ensure markdown filenames use hyphens (`-`) rather than underscores (`_`). -4. Verify Sphinx MyST colon indentation hierarchy (outer directives must have more colons than inner directives). -5. Verify `{versionadded}` and `{versionchanged}` sections are placed at the end of the documentation text. -6. For unreleased features or attributes, ensure `{versionadded}` / `{versionchanged}` directives use `VERSION_NEXT_FEATURE` (not hardcoded version numbers). -7. Check documentation build correctness: ensure new `.bzl` files or public APIs are included in `//docs:docs` or relevant docs build targets. - -Report any violations found clearly with actionable suggested fixes, or report that the documentation changes pass audit. diff --git a/.agents/skills/pre-review-audit/review-pr-standards-prompt.md b/.agents/skills/pre-review-audit/review-pr-standards-prompt.md deleted file mode 100644 index 6120b412af..0000000000 --- a/.agents/skills/pre-review-audit/review-pr-standards-prompt.md +++ /dev/null @@ -1,9 +0,0 @@ -You are a specialized PR Standards Auditor sub-agent. -Your sole task is to audit PR titles and PR descriptions against `CONTRIBUTING.md` and project rules: - -1. Read and strictly enforce `.agents/rules/pr.md`, `.agents/rules/news.md`, `AGENTS.md`, and `CONTRIBUTING.md`. -2. Check PR title formatting: must follow Conventional Commits format (e.g. `feat(cc): ...`, `docs(python): ...`). For agent rules/skills, use `agents:` prefix. -3. Ensure PR descriptions explain *why* a change is made and provide a high-level overview of *how*, following advice in `CONTRIBUTING.md`. -4. If a PR has already been created, enforce PR update rules: do NOT amend or rebase existing commits (create new commits and merges instead, to preserve code review comment threads). - -Report any violations found clearly with actionable suggested fixes, or report that the PR standards pass audit. diff --git a/.agents/skills/pre-review-audit/review-python-prompt.md b/.agents/skills/pre-review-audit/review-python-prompt.md deleted file mode 100644 index aa2d726228..0000000000 --- a/.agents/skills/pre-review-audit/review-python-prompt.md +++ /dev/null @@ -1,9 +0,0 @@ -You are a specialized Python & pytest Auditor sub-agent. -Your sole task is to audit all Python source (`.py`) and test changes in `git diff` against the project's Python conventions: - -1. Read and strictly enforce `.agents/rules/python.md`, `AGENTS.md`, and `CONTRIBUTING.md`. -2. Check Python pytest conventions: when registering pytest fixtures from helper modules in test files, use `pytest_plugins = [""]`. -3. Name fixture functions with a `fixture_` prefix (e.g. `def fixture_foo():`) and pass the public fixture name using `@pytest.fixture(name="foo")`. -4. Verify that tests were executed using Bazel (`bazel test --config=fast-tests`) and passed. - -Report any violations found clearly with actionable suggested fixes, or report that the Python changes pass audit. diff --git a/.agents/skills/pre-review-audit/review-starlark-prompt.md b/.agents/skills/pre-review-audit/review-starlark-prompt.md deleted file mode 100644 index 8421219a36..0000000000 --- a/.agents/skills/pre-review-audit/review-starlark-prompt.md +++ /dev/null @@ -1,11 +0,0 @@ -You are a specialized Starlark & Bazel Code Auditor sub-agent. -Your sole task is to audit all Starlark (`.bzl`, `BUILD`, `*.bazel`) changes in `git diff` against the project's Starlark coding rules and conventions: - -1. Read and strictly enforce `.agents/rules/starlark.md`, `.agents/rules/bzl.md`, `AGENTS.md`, and `CONTRIBUTING.md`. -2. Verify iterative algorithms are used (no recursion, no `while` loops). -3. Ensure every `.bzl` file outside `tests/` has a corresponding `bzl_library` target in its `BUILD` file with proper dependencies. -4. Ensure loads from `/private/` in test files have `# buildifier: disable=bzl-visibility`. -5. Check multi-line rule/macro doc arguments: use triple-quoted strings (`"""`), and do NOT use trailing backslashes (`\`) on opening triple-quotes. -6. Verify analysis tests use `rules_testing`, not `bazel_skylib`. - -Report any violations found clearly with actionable suggested fixes, or report that the Starlark changes pass audit. diff --git a/.agents/skills/review-code/SKILL.md b/.agents/skills/review-code/SKILL.md new file mode 100644 index 0000000000..8646ff2451 --- /dev/null +++ b/.agents/skills/review-code/SKILL.md @@ -0,0 +1,59 @@ +--- +name: review-code +description: High-level code review audit of local changes when preparing to + send a PR for review +trigger: model_decision +--- + +When preparing to send a Pull Request for review, invoke **separate, concurrent +sub-agents** (`invoke_subagent`), where each sub-agent focuses exclusively on +its assigned checklist category. + +**CRITICAL**: Do NOT use a single sub-agent to validate multiple or all +dimensions at once. Each sub-agent must be launched with its own distinct prompt +file from `.agents/skills/review-code/`: + +### Audit Checklist & Sub-Agent Prompts + +1. **Starlark / Bazel Sub-Agent** (`Role: "Starlark Code Auditor"`): + - Prompt file: + `.agents/skills/review-code/review-starlark-prompt.md` + - Focus: Audits Starlark / Bazel changes (`*.bzl`, `BUILD`, `*.bazel` files) + in `git diff` against `.agents/rules/bzl.md` and Starlark rules in + `AGENTS.md`. + +2. **Python Code Sub-Agent** (`Role: "Python Code Auditor"`): + - Prompt file: + `.agents/skills/review-code/review-python-prompt.md` + - Focus: Audits Python source and test changes in `git diff` against + `.agents/rules/python.md` and Python and pytest conventions in `AGENTS.md`. + +3. **Documentation Sub-Agent** (`Role: "Documentation Auditor"`): + - Prompt file: + `.agents/skills/review-code/review-docs-prompt.md` + - Focus: Audits documentation (`.md`) changes and docs build targets against + `.agents/rules/docs.md`, `.agents/rules/news.md`, and `AGENTS.md`. + +4. **Contribution Guidelines Sub-Agent** (`Role: "Contributing Auditor"`): + - Prompt file: + `.agents/skills/review-code/review-contributing-prompt.md` + - Focus: Audits changes, requirements updates, directives, and news entry + files (`news/..md`) against `CONTRIBUTING.md` and + `.agents/rules/news.md`. + +5. **Project Conventions Sub-Agent** (`Role: "Project Conventions Auditor"`): + - Prompt file: + `.agents/skills/review-code/review-agents-prompt.md` + - Focus: Audits overall workspace compliance against `AGENTS.md`. + +6. **Pull Request Standards Sub-Agent** (`Role: "PR Standards Auditor"`): + - Prompt file: + `.agents/skills/review-code/review-pr-standards-prompt.md` + - Focus: Audits PR titles and descriptions against Conventional Commits + formatting and PR update rules in `CONTRIBUTING.md`. + +### Action Instructions +- Launch all sub-agents concurrently using `invoke_subagent`. +- Collect the reports from each sub-agent and report all violations and + suggested improvements clearly with suggested fixes for the user. +- If all domain audits pass, confirm that the PR is ready for review. diff --git a/.agents/skills/pre-review-audit/review-agents-prompt.md b/.agents/skills/review-code/review-agents-prompt.md similarity index 57% rename from .agents/skills/pre-review-audit/review-agents-prompt.md rename to .agents/skills/review-code/review-agents-prompt.md index f81924224c..45651fd0de 100644 --- a/.agents/skills/pre-review-audit/review-agents-prompt.md +++ b/.agents/skills/review-code/review-agents-prompt.md @@ -1,10 +1,18 @@ You are a specialized Project Conventions (`AGENTS.md`) Auditor sub-agent. -Your sole task is to audit all local changes (`git diff`) and workspace state against `AGENTS.md`: +Your sole task is to audit all local changes (`git diff`) and workspace state +against `AGENTS.md`: -1. Read and strictly enforce `AGENTS.md` and all `.agents/rules/*.md` files without exception. -2. Verify NO Bazel copyright headers (`# Copyright ... The Bazel Authors`) were added to new or existing files, unless explicitly instructed by the user. -3. Verify that tests were executed using `bazel test --config=fast-tests` and non-test build targets did not use `--config=fast-tests`. -4. Ensure public config settings in `python/config_settings/BUILD.bazel` were not modified unless explicitly instructed. -5. Check that all repo rules and macro conventions described in `AGENTS.md` are respected. +1. Read and strictly enforce `AGENTS.md` and all `.agents/rules/*.md` files + without exception. +2. Verify NO Bazel copyright headers (`# Copyright ... The Bazel Authors`) + were added to new or existing files, unless explicitly instructed by the + user. +3. Verify that tests were executed using `bazel test --config=fast-tests` and + non-test build targets did not use `--config=fast-tests`. +4. Ensure public config settings in `python/config_settings/BUILD.bazel` were + not modified unless explicitly instructed. +5. Check that all repo rules and macro conventions described in `AGENTS.md` + are respected. -Report any violations found clearly with actionable suggested fixes, or report that the changes pass project conventions audit. +Report any violations found clearly with actionable suggested fixes, or report +that the changes pass project conventions audit. diff --git a/.agents/skills/review-code/review-contributing-prompt.md b/.agents/skills/review-code/review-contributing-prompt.md new file mode 100644 index 0000000000..51fe58db00 --- /dev/null +++ b/.agents/skills/review-code/review-contributing-prompt.md @@ -0,0 +1,37 @@ +You are a specialized Contribution Guidelines Auditor sub-agent. +Your sole task is to audit all local changes (`git diff`), commit messages, +and PR metadata against `CONTRIBUTING.md` and project rules: + +1. Read and strictly enforce `CONTRIBUTING.md`, all `.agents/rules/*.md` files + (especially `.agents/rules/news.md`), and `AGENTS.md`. +2. **News Entry File Audit**: + - Check if the PR introduces user-visible features (`feat:`), bug fixes + (`fix:`), behavioral changes (`changed:`), breaking changes, or removals + (`removed:`). If so, verify that a news fragment file is added under + `news/..md`. + - Verify `CHANGELOG.md` is NOT modified directly for unreleased changes. + - Verify news filename format: `news/..md` where `` is the + PR or issue number (or placeholder ID) and `` is strictly one of + `added`, `changed`, `fixed`, or `removed`. + - Verify news entry content rules: + - Brief, human-friendly description without leading bullet points (`*` + or `-`). + - Subsystem prefix in parentheses when applicable (e.g. `(gazelle) ...`, + `(cc) ...`). + - Use Sphinx MyST cross-reference syntax `{obj}\`\`` for rules, + macros, targets, providers, attributes, and args. + - Append GitHub issue cross-references at the end in markdown link format: + `([#1234](https://github.com/bazel-contrib/rules_python/issues/1234))`. + - Lines wrapped at 80 columns. +3. Verify that `{versionadded}` and `{versionchanged}` directives use + `VERSION_NEXT_FEATURE` for unreleased features and are placed at the end of + the documentation text. +4. If locked/resolved requirements files (`requirements.txt`, `pyproject.toml`, + `requirements.in`) were modified, verify that the associated + `requirements.update` target was executed to keep locked requirement files + in sync. +5. Ensure style and conventions described in `CONTRIBUTING.md` are respected + across all changes. + +Report any violations found clearly with actionable suggested fixes, or report +that the changes pass contribution audit. diff --git a/.agents/skills/review-code/review-docs-prompt.md b/.agents/skills/review-code/review-docs-prompt.md new file mode 100644 index 0000000000..eaeb91232d --- /dev/null +++ b/.agents/skills/review-code/review-docs-prompt.md @@ -0,0 +1,24 @@ +You are a specialized Documentation & Sphinx/MyST Auditor sub-agent. +Your sole task is to audit all documentation (`.md`) changes and new `.bzl` APIs +in `git diff` against the project's documentation rules: + +1. Read and strictly enforce `.agents/rules/docs.md`, `.agents/rules/news.md`, + `AGENTS.md`, and `CONTRIBUTING.md`. +2. Check that lines wrap at 80 columns. +3. Ensure markdown filenames use hyphens (`-`) rather than underscores (`_`) + (except news entry files under `news/`, which follow `..md`). +4. Verify Sphinx MyST colon indentation hierarchy (outer directives must have + more colons than inner directives). +5. Verify `{versionadded}` and `{versionchanged}` sections are placed at the + end of the documentation text. +6. For unreleased features or attributes, ensure `{versionadded}` / + `{versionchanged}` directives use `VERSION_NEXT_FEATURE` (not hardcoded + version numbers). +7. Check documentation build correctness: ensure new `.bzl` files or public + APIs are included in `//docs:docs` or relevant docs build targets. +8. For any added or modified news entries in `news/`, verify they adhere to + `.agents/rules/news.md` and `CONTRIBUTING.md` (proper `..md` + name, no leading bullets, subsystem prefix, `{obj}` refs, issue links). + +Report any violations found clearly with actionable suggested fixes, or report +that the documentation changes pass audit. diff --git a/.agents/skills/review-code/review-pr-standards-prompt.md b/.agents/skills/review-code/review-pr-standards-prompt.md new file mode 100644 index 0000000000..a4de4296a1 --- /dev/null +++ b/.agents/skills/review-code/review-pr-standards-prompt.md @@ -0,0 +1,17 @@ +You are a specialized PR Standards Auditor sub-agent. +Your sole task is to audit PR titles and PR descriptions against +`CONTRIBUTING.md` and project rules: + +1. Read and strictly enforce `.agents/rules/pr.md`, `.agents/rules/news.md`, + `AGENTS.md`, and `CONTRIBUTING.md`. +2. Check PR title formatting: must follow Conventional Commits format (e.g. + `feat(cc): ...`, `docs(python): ...`). For agent rules/skills, use `agents:` + prefix. +3. Ensure PR descriptions explain *why* a change is made and provide a + high-level overview of *how*, following advice in `CONTRIBUTING.md`. +4. If a PR has already been created, enforce PR update rules: do NOT amend or + rebase existing commits (create new commits and merges instead, to preserve + code review comment threads). + +Report any violations found clearly with actionable suggested fixes, or report +that the PR standards pass audit. diff --git a/.agents/skills/review-code/review-python-prompt.md b/.agents/skills/review-code/review-python-prompt.md new file mode 100644 index 0000000000..8e477ab3a7 --- /dev/null +++ b/.agents/skills/review-code/review-python-prompt.md @@ -0,0 +1,15 @@ +You are a specialized Python & pytest Auditor sub-agent. +Your sole task is to audit all Python source (`.py`) and test changes in +`git diff` against the project's Python conventions: + +1. Read and strictly enforce `.agents/rules/python.md`, `AGENTS.md`, and + `CONTRIBUTING.md`. +2. Check Python pytest conventions: when registering pytest fixtures from helper + modules in test files, use `pytest_plugins = [""]`. +3. Name fixture functions with a `fixture_` prefix (e.g. `def fixture_foo():`) + and pass the public fixture name using `@pytest.fixture(name="foo")`. +4. Verify that tests were executed using Bazel + (`bazel test --config=fast-tests`) and passed. + +Report any violations found clearly with actionable suggested fixes, or report +that the Python changes pass audit. diff --git a/.agents/skills/review-code/review-starlark-prompt.md b/.agents/skills/review-code/review-starlark-prompt.md new file mode 100644 index 0000000000..8bd8cdbe21 --- /dev/null +++ b/.agents/skills/review-code/review-starlark-prompt.md @@ -0,0 +1,17 @@ +You are a specialized Starlark & Bazel Code Auditor sub-agent. +Your sole task is to audit all Starlark (`.bzl`, `BUILD`, `*.bazel`) changes +in `git diff` against the project's Starlark coding rules and conventions: + +1. Read and strictly enforce `.agents/rules/starlark.md`, + `.agents/rules/bzl.md`, `AGENTS.md`, and `CONTRIBUTING.md`. +2. Verify iterative algorithms are used (no recursion, no `while` loops). +3. Ensure every `.bzl` file outside `tests/` has a corresponding `bzl_library` + target in its `BUILD` file with proper dependencies. +4. Ensure loads from `/private/` in test files have + `# buildifier: disable=bzl-visibility`. +5. Check multi-line rule/macro doc arguments: use triple-quoted strings + (`"""`), and do NOT use trailing backslashes (`\`) on opening triple-quotes. +6. Verify analysis tests use `rules_testing`, not `bazel_skylib`. + +Report any violations found clearly with actionable suggested fixes, or report +that the Starlark changes pass audit. From 05795bedfeda5a36efbc487ff23fb247a57294b7 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Sun, 9 Aug 2026 16:45:37 -0700 Subject: [PATCH 919/922] feat(runfiles): add CreateOrRaise API (#4028) Currently, Runfiles.Create returns None when runfiles environment variables ($RUNFILES_MANIFEST_FILE and $RUNFILES_DIR) are unset or empty, requiring callers to write repetitive None checks and handle missing runfiles manually. This adds Runfiles.CreateOrRaise and module-level CreateOrRaise, which return the initialized Runfiles instance or raise a RuntimeError if runfiles cannot be found. --- news/add_runfiles_raise_api.added.md | 2 + python/runfiles/README.md | 12 +++++- python/runfiles/runfiles.py | 49 ++++++++++++++++++++++++ tests/runfiles/runfiles_test.py | 56 ++++++++++++++++++++++++++++ 4 files changed, 118 insertions(+), 1 deletion(-) create mode 100644 news/add_runfiles_raise_api.added.md diff --git a/news/add_runfiles_raise_api.added.md b/news/add_runfiles_raise_api.added.md new file mode 100644 index 0000000000..5b2ff574e6 --- /dev/null +++ b/news/add_runfiles_raise_api.added.md @@ -0,0 +1,2 @@ +(runfiles) Added {obj}`Runfiles.CreateOrRaise` to return a `Runfiles` instance +or raise an error if runfiles cannot be found. diff --git a/python/runfiles/README.md b/python/runfiles/README.md index b5315a48f5..4ed51a3b12 100644 --- a/python/runfiles/README.md +++ b/python/runfiles/README.md @@ -61,7 +61,17 @@ with open(r.Rlocation("my_workspace/path/to/my/data.txt"), "r") as f: Here `my_workspace` is the name you specified via `module(name = "...")` in your `MODULE.bazel` file (with `--enable_bzlmod`, default as of Bazel 7) or `workspace(name = "...")` in `WORKSPACE` (with `--noenable_bzlmod`). -The code above creates a manifest- or directory-based implementation based on the environment variables in `os.environ`. See `Runfiles.Create()` for more info. +The code above creates a manifest- or directory-based implementation based on +the environment variables in `os.environ`. See `Runfiles.Create()` for more +info. + +Alternatively, `Runfiles.CreateOrRaise()` can be used to raise an error +instead of returning `None` if runfiles cannot be found: + +```python +r = Runfiles.CreateOrRaise() +``` + If you want to explicitly create a manifest- or directory-based implementation, you can do so as follows: diff --git a/python/runfiles/runfiles.py b/python/runfiles/runfiles.py index 6fe2e8f28b..02fceb3020 100644 --- a/python/runfiles/runfiles.py +++ b/python/runfiles/runfiles.py @@ -728,6 +728,46 @@ def Create(env: Optional[Dict[str, str]] = None) -> Optional["Runfiles"]: return None + # TODO: Update return type to Self when 3.11 is the min version + # https://peps.python.org/pep-0673/ + @staticmethod + def CreateOrRaise(env: Optional[Dict[str, str]] = None) -> "Runfiles": + """Returns a new `Runfiles` instance, or raises an error. + + The returned object is either: + - manifest-based, meaning it looks up runfile paths from a manifest + file, or + - directory-based, meaning it looks up runfile paths under a given + directory path + + If `env` contains "RUNFILES_MANIFEST_FILE" with non-empty value, this + method returns a manifest-based implementation. The object eagerly + reads and caches the whole manifest file upon instantiation; this may + be relevant for performance consideration. + + Otherwise, if `env` contains "RUNFILES_DIR" with non-empty value + (checked in this priority order), this method returns a directory-based + implementation. + + If neither cases apply, this method raises a `RuntimeError`. + + Args: + env: {string: string}; optional; the map of environment variables. If + None, this function uses the environment variable map of this + process. + Raises: + RuntimeError: if runfiles cannot be found. + + :::{versionadded} VERSION_NEXT_FEATURE + ::: + """ + runfiles = Runfiles.Create(env=env) + if runfiles is None: + raise RuntimeError( + "Cannot create Runfiles: $RUNFILES_MANIFEST_FILE and $RUNFILES_DIR are both unset or empty" + ) + return runfiles + # Support legacy imports by defining a private symbol. _Runfiles = Runfiles @@ -743,3 +783,12 @@ def CreateDirectoryBased(runfiles_dir_path: str) -> Runfiles: def Create(env: Optional[Dict[str, str]] = None) -> Optional[Runfiles]: return Runfiles.Create(env) + + +def CreateOrRaise(env: Optional[Dict[str, str]] = None) -> Runfiles: + """Refer to `Runfiles.CreateOrRaise`. + + :::{versionadded} VERSION_NEXT_FEATURE + ::: + """ + return Runfiles.CreateOrRaise(env) diff --git a/tests/runfiles/runfiles_test.py b/tests/runfiles/runfiles_test.py index 47c964631f..38a89ede7e 100644 --- a/tests/runfiles/runfiles_test.py +++ b/tests/runfiles/runfiles_test.py @@ -195,6 +195,62 @@ def testFailsToCreateAnyRunfilesBecauseEnvvarsAreNotDefined(self) -> None: self.assertIsNone(runfiles.Create({"TEST_SRCDIR": "always ignored"})) self.assertIsNone(runfiles.Create({"FOO": "bar"})) + def testCreatesManifestBasedRunfilesWithCreateOrRaise(self) -> None: + with _MockFile(contents=["a/b c/d"]) as mf: + r = runfiles.CreateOrRaise( + { + "RUNFILES_MANIFEST_FILE": mf.Path(), + "RUNFILES_DIR": "ignored when RUNFILES_MANIFEST_FILE has a value", + "TEST_SRCDIR": "always ignored", + } + ) + self.assertEqual(r.Rlocation("a/b"), "c/d") + self.assertIsNone(r.Rlocation("foo")) + + r_class = runfiles.Runfiles.CreateOrRaise( + { + "RUNFILES_MANIFEST_FILE": mf.Path(), + } + ) + self.assertEqual(r_class.Rlocation("a/b"), "c/d") + + def testCreatesDirectoryBasedRunfilesWithCreateOrRaise(self) -> None: + r = runfiles.CreateOrRaise( + { + "RUNFILES_DIR": "runfiles/dir", + "TEST_SRCDIR": "always ignored", + } + ) + self.assertEqual(r.Rlocation("a/b"), "runfiles/dir/a/b") + self.assertEqual(r.Rlocation("foo"), "runfiles/dir/foo") + + r_class = runfiles.Runfiles.CreateOrRaise( + { + "RUNFILES_DIR": "runfiles/dir", + } + ) + self.assertEqual(r_class.Rlocation("a/b"), "runfiles/dir/a/b") + + def testFailsToCreateManifestBasedBecauseManifestDoesNotExistWithCreateOrRaise( + self, + ) -> None: + def _Run(): + runfiles.CreateOrRaise({"RUNFILES_MANIFEST_FILE": "non-existing path"}) + + self.assertRaisesRegex(IOError, "non-existing path", _Run) + + def testFailsToCreateAnyRunfilesWithCreateOrRaise(self) -> None: + with self.assertRaises(RuntimeError): + runfiles.CreateOrRaise({"TEST_SRCDIR": "always ignored"}) + with self.assertRaises(RuntimeError): + runfiles.CreateOrRaise({"FOO": "bar"}) + with self.assertRaises(RuntimeError): + runfiles.CreateOrRaise({}) + with self.assertRaises(RuntimeError): + runfiles.Runfiles.CreateOrRaise({"TEST_SRCDIR": "always ignored"}) + with self.assertRaises(RuntimeError): + runfiles.Runfiles.CreateOrRaise({}) + def testManifestBasedRlocation(self) -> None: with _MockFile( contents=[ From 8f63d1e18f953c761e778cfd7d7896e81b25d78b Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Sun, 9 Aug 2026 22:38:50 -0700 Subject: [PATCH 920/922] build(dev): bump dev requirements python version to 3.10 (#4024) Bump the minimum Python version target for dev requirements and lockfiles from Python 3.9 to Python 3.10. Python 3.9 compatibility constraints in dev dependencies are no longer needed and trigger dependency audit and Dependabot alerts. Updating the baseline resolves these warnings and keeps development tooling aligned with supported Python runtimes. --- dev/BUILD.bazel | 9 +- dev/requirements.txt | 90 ++------- dev/uv.lock | 435 +++++++------------------------------------ 3 files changed, 84 insertions(+), 450 deletions(-) diff --git a/dev/BUILD.bazel b/dev/BUILD.bazel index 91b2a9f566..22fe5bb918 100644 --- a/dev/BUILD.bazel +++ b/dev/BUILD.bazel @@ -12,9 +12,10 @@ lock( "--universal", "--upgrade", ], - # NOTE @aignas 2025-08-17: here we select the lowest actively supported version so that the - # requirements file is generated to be compatible with Python version 3.9 or greater. - python_version = "3.9", + # NOTE @aignas 2025-08-17: here we select the lowest actively supported + # version so that the requirements file is generated to be compatible with + # Python version 3.10 or greater. + python_version = "3.10", visibility = ["//:__subpackages__"], ) @@ -23,6 +24,6 @@ lock( name = "uv_lock", srcs = ["pyproject.toml"], out = "uv.lock", - python_version = "3.9", + python_version = "3.10", visibility = ["//:__subpackages__"], ) diff --git a/dev/requirements.txt b/dev/requirements.txt index 943fce2dcf..6efb1b8bdb 100644 --- a/dev/requirements.txt +++ b/dev/requirements.txt @@ -2,19 +2,11 @@ # bazel run //dev:requirements.update --index-url https://pypi.org/simple -absl-py==2.3.1 ; python_full_version < '3.10' \ - --hash=sha256:a97820526f7fbfd2ec1bce83f3f25e3a14840dac0d8e02a0b71cd75db3f77fc9 \ - --hash=sha256:eeecf07f0c2a93ace0772c92e596ace6d3d3996c042b2128459aaae2a76de11d - # via rules-python-dev (dev/pyproject.toml) -absl-py==2.5.0 ; python_full_version >= '3.10' \ +absl-py==2.5.0 \ --hash=sha256:0c996f25c0490700fadabe6351630f6111534fa0ae252cc6d2014ea3b141135f \ --hash=sha256:0f17b89f2a4eaaedc4f28c622998aa690564b3012a396a4ffad0821007fe03ba # via rules-python-dev (dev/pyproject.toml) -alabaster==0.7.16 ; python_full_version < '3.10' \ - --hash=sha256:75a8b99c28a5dad50dd7f8ccdd447a121ddb3892da9e53d1ca5cca3106d58d65 \ - --hash=sha256:b46733c07dce03ae4e150330b975c75737fa60f0a7c591b6c8bf4928a28e2c92 - # via sphinx -alabaster==1.0.0 ; python_full_version >= '3.10' \ +alabaster==1.0.0 \ --hash=sha256:c00dca57bca26fa62a6d7d0a9fcce65f3e026e9bfe33e9c538fd3fbb2144fd9e \ --hash=sha256:fc6786402dc3fcb2de3cabd5fe455a2db534b371124f1f21de8731783dec828b # via sphinx @@ -157,23 +149,11 @@ idna==3.18 \ --hash=sha256:7f952cbe720b688055e3f87de14f5c3e5fdaa8bc3928985c4077ca689de849a2 \ --hash=sha256:ffb385a7e039654cef1ab9ef32c6fafe283c0c0467bba1d9029738ce4a14a848 # via requests -imagesize==1.5.0 ; python_full_version < '3.10' \ - --hash=sha256:32677681b3f434c2cb496f00e89c5a291247b35b1f527589909e008057da5899 \ - --hash=sha256:8bfc5363a7f2133a89f0098451e0bcb1cd71aba4dc02bbcecb39d99d40e1b94f - # via sphinx -imagesize==2.0.0 ; python_full_version >= '3.10' \ +imagesize==2.0.0 \ --hash=sha256:5667c5bbb57ab3f1fa4bc366f4fbc971db3d5ed011fd2715fd8001f782718d96 \ --hash=sha256:8e8358c4a05c304f1fccf7ff96f036e7243a189e9e42e90851993c558cfe9ee3 # via sphinx -importlib-metadata==8.7.1 ; python_full_version < '3.10' \ - --hash=sha256:49fef1ae6440c182052f407c8d34a68f72efc36db9ca90dc0113398f2fdde8bb \ - --hash=sha256:5a1f80bf1daa489495071efbb095d75a634cf28a8bc299581244063b53176151 - # via sphinx -iniconfig==2.1.0 ; python_full_version < '3.10' \ - --hash=sha256:3abbd2e30b36733fee78f9c7f7308f2d0050e88f0087fd25c2645f63c773e1c7 \ - --hash=sha256:9deba5723312380e77435581c6bf4935c94cbfab9b1ed33ef8d238ea168eb760 - # via pytest -iniconfig==2.3.0 ; python_full_version >= '3.10' \ +iniconfig==2.3.0 \ --hash=sha256:c76315c77db068650d49c5b56314774a7804df16fee4402c1f19d6d15d8c4730 \ --hash=sha256:f631c04d2c48c52b84d0d0549c99ff3859c98df65b3101406327ecc7d53fbf12 # via pytest @@ -293,11 +273,7 @@ markupsafe==3.0.3 \ # via # rules-python-dev (dev/pyproject.toml) # jinja2 -mdit-py-plugins==0.4.2 ; python_full_version < '3.10' \ - --hash=sha256:0c673c3f889399a33b95e88d2f0d111b4447bdfea7f237dab2d488f459835636 \ - --hash=sha256:5f2cd1fdb606ddf152d37ec30e46101a60512bc0e5fa1a7002c36647b09e26b5 - # via myst-parser -mdit-py-plugins==0.6.1 ; python_full_version >= '3.10' \ +mdit-py-plugins==0.6.1 \ --hash=sha256:214c82fb2ac524472ab6a5bcab1de80f73b50443e187f401bfd77efbc7c6481d \ --hash=sha256:a2bca0f039f39dbd35fb74ae1b5f998608c437463371f0ff7f49a19a17a114d0 # via myst-parser @@ -305,11 +281,7 @@ mdurl==0.1.2 \ --hash=sha256:84008a41e51615a49fc9966191ff91509e3c40b939176e643fd50a5c2196b8f8 \ --hash=sha256:bb413d29f5eea38f31dd4754dd7377d4465116fb207585f97bf925588687c1ba # via markdown-it-py -myst-parser==3.0.1 ; python_full_version < '3.10' \ - --hash=sha256:6457aaa33a5d474aca678b8ead9b3dc298e89c68e67012e73146ea6fd54babf1 \ - --hash=sha256:88f0cb406cb363b077d176b51c476f62d60604d68a8dcdf4832e080441301a87 - # via rules-python-dev (dev/pyproject.toml) -myst-parser==4.0.1 ; python_full_version == '3.10.*' \ +myst-parser==4.0.1 ; python_full_version < '3.11' \ --hash=sha256:5cfea715e4f3574138aecbf7d54132296bfd72bb614d31168f48c477a830a7c4 \ --hash=sha256:9134e88959ec3b5780aedf8a99680ea242869d012e8821db3126d427edc9c95d # via rules-python-dev (dev/pyproject.toml) @@ -317,9 +289,9 @@ myst-parser==5.1.0 ; python_full_version >= '3.11' \ --hash=sha256:9c91c52b3cdb4d94a6506e4fab4e2f296c7623a0da0dcbe6de1565c3dad67a8a \ --hash=sha256:ab69322dc6719dcc7f296479dbb70181b66df6ed315064f92dbc85c0e1bf2f02 # via rules-python-dev (dev/pyproject.toml) -packaging==26.2 \ - --hash=sha256:5fc45236b9446107ff2415ce77c807cee2862cb6fac22b8a73826d0693b0980e \ - --hash=sha256:ff452ff5a3e828ce110190feff1178bb1f2ea2281fa2075aadb987c2fb221661 +packaging==26.3 \ + --hash=sha256:94edc256424af38762eb31306eed28beb9f0efc50a8837492c9d6fd6004aed79 \ + --hash=sha256:d7193f7c8e4e93f444fde0262bf90af30e16fa0ad0ad44cb553c87339b23cd1c # via # pytest # readthedocs-sphinx-ext @@ -332,11 +304,7 @@ pluggy==1.6.0 \ --hash=sha256:7dcc130b76258d33b90f61b658791dede3486c3e6bfb003ee5c9bfb396dd22f3 \ --hash=sha256:e920276dd6813095e9377c0bc5566d94c932c33b27a3e3945d8389c374dd4746 # via pytest -pyelftools==0.32 ; python_full_version < '3.10' \ - --hash=sha256:013df952a006db5e138b1edf6d8a68ecc50630adbd0d83a2d41e7f846163d738 \ - --hash=sha256:6de90ee7b8263e740c8715a925382d4099b354f29ac48ea40d840cf7aa14ace5 - # via rules-python-dev (dev/pyproject.toml) -pyelftools==0.33 ; python_full_version >= '3.10' \ +pyelftools==0.33 \ --hash=sha256:660d82dcbeb8e83d1702bd97f223f761625da06111c0cc988eac6b8ab0c1b61f \ --hash=sha256:f215ad5f47d3f1373a21496a6c9e0707c622840d0622f23ff7ce08678b020036 # via rules-python-dev (dev/pyproject.toml) @@ -346,14 +314,7 @@ pygments==2.20.0 \ # via # pytest # sphinx -pytest==8.4.2 ; python_full_version < '3.10' \ - --hash=sha256:86c0d0b93306b961d58d62a4db4879f27fe25513d4b969df351abdddb3c30e01 \ - --hash=sha256:872f880de3fc3a5bdc88a11b39c9710c3497a547cfa9320bc3c5e62fbf272e79 - # via - # rules-python-dev (dev/pyproject.toml) - # pytest-bazel - # pytest-mock -pytest==9.1.1 ; python_full_version >= '3.10' \ +pytest==9.1.1 \ --hash=sha256:1088fbde8f2b49d95a549a195707afa7a76a3ce9bcadc26b6d71f0ffda5fe313 \ --hash=sha256:37a86b45efb9a47a61a36449063e8e18d0cab3161329fc099eb21783169c4f0c # via @@ -446,13 +407,7 @@ readthedocs-sphinx-ext==2.2.5 \ --hash=sha256:ee5fd5b99db9f0c180b2396cbce528aa36671951b9526bb0272dbfce5517bd27 \ --hash=sha256:f8c56184ea011c972dd45a90122568587cc85b0127bc9cf064d17c68bc809daa # via rules-python-dev (dev/pyproject.toml) -requests==2.32.5 ; python_full_version < '3.10' \ - --hash=sha256:2462f94637a34fd532264295e186976db0f5d453d1cdd31473c85a6a161affb6 \ - --hash=sha256:dbba0bac56e100853db0ea71b82b4dfd5fe2bf6d3754a8893c3af500cec7d7cf - # via - # readthedocs-sphinx-ext - # sphinx -requests==2.34.2 ; python_full_version >= '3.10' \ +requests==2.34.2 \ --hash=sha256:2a0d60c172f83ac6ab31e4554906c0f3b3588d37b5cb939b1c061f4907e278e0 \ --hash=sha256:f288924cae4e29463698d6d60bc6a4da69c89185ad1e0bcc4104f584e960b9ed # via @@ -466,16 +421,7 @@ snowballstemmer==3.1.1 \ --hash=sha256:7e207fa178741da09cdee59d3ecec3827ad5f92b1fc5c9ff3755b639f71f5752 \ --hash=sha256:e07bbc54a0d798fe6010a12398422e62a8bfbba95c394fd0956ef58cb4d3e260 # via sphinx -sphinx==7.4.7 ; python_full_version < '3.10' \ - --hash=sha256:242f92a7ea7e6c5b406fdc2615413890ba9f699114a9c09192d7dfead2ee9cfe \ - --hash=sha256:c2419e2135d11f1951cd994d6eb18a1835bd8fdd8429f9ca375dc1f3281bd239 - # via - # rules-python-dev (dev/pyproject.toml) - # myst-parser - # sphinx-reredirects - # sphinx-rtd-theme - # sphinxcontrib-jquery -sphinx==8.1.3 ; python_full_version == '3.10.*' \ +sphinx==8.1.3 ; python_full_version < '3.11' \ --hash=sha256:09719015511837b76bf6e03e42eb7595ac8c2e41eeb9c29c5b755c6b677992a2 \ --hash=sha256:43c1911eecb0d3e161ad78611bc905d1ad0e523e4ddc202a58a821773dc4c927 # via @@ -606,15 +552,7 @@ typing-extensions==4.16.0 \ # astroid # exceptiongroup # sphinx-autodoc2 -urllib3==2.6.3 ; python_full_version < '3.10' \ - --hash=sha256:1b62b6884944a57dbe321509ab94fd4d3b307075e0c2eae991ac71ee15ad38ed \ - --hash=sha256:bf272323e553dfb2e87d9bfd225ca7b0f467b919d7bbd355436d3fd37cb0acd4 - # via requests -urllib3==2.7.0 ; python_full_version >= '3.10' \ +urllib3==2.7.0 \ --hash=sha256:231e0ec3b63ceb14667c67be60f2f2c40a518cb38b03af60abc813da26505f4c \ --hash=sha256:9fb4c81ebbb1ce9531cce37674bbc6f1360472bc18ca9a553ede278ef7276897 # via requests -zipp==3.23.1 ; python_full_version < '3.10' \ - --hash=sha256:0b3596c50a5c700c9cb40ba8d86d9f2cc4807e9bedb06bcdf7fac85633e444dc \ - --hash=sha256:32120e378d32cd9714ad503c1d024619063ec28aad2248dc6672ad13edfa5110 - # via importlib-metadata diff --git a/dev/uv.lock b/dev/uv.lock index 842e831dec..b524eb1f61 100644 --- a/dev/uv.lock +++ b/dev/uv.lock @@ -1,66 +1,25 @@ version = 1 revision = 3 -requires-python = ">=3.9" +requires-python = ">=3.10" resolution-markers = [ - "python_full_version >= '3.14'", - "python_full_version == '3.13.*'", - "python_full_version == '3.12.*'", + "python_full_version >= '3.12'", "python_full_version == '3.11.*'", - "python_full_version == '3.10.*'", - "python_full_version < '3.10'", -] - -[[package]] -name = "absl-py" -version = "2.3.1" -source = { registry = "https://pypi.org/simple" } -resolution-markers = [ - "python_full_version < '3.10'", -] -sdist = { url = "https://files.pythonhosted.org/packages/10/2a/c93173ffa1b39c1d0395b7e842bbdc62e556ca9d8d3b5572926f3e4ca752/absl_py-2.3.1.tar.gz", hash = "sha256:a97820526f7fbfd2ec1bce83f3f25e3a14840dac0d8e02a0b71cd75db3f77fc9", size = 116588, upload-time = "2025-07-03T09:31:44.05Z" } -wheels = [ - { url = "https://files.pythonhosted.org/packages/8f/aa/ba0014cc4659328dc818a28827be78e6d97312ab0cb98105a770924dc11e/absl_py-2.3.1-py3-none-any.whl", hash = "sha256:eeecf07f0c2a93ace0772c92e596ace6d3d3996c042b2128459aaae2a76de11d", size = 135811, upload-time = "2025-07-03T09:31:42.253Z" }, + "python_full_version < '3.11'", ] [[package]] name = "absl-py" version = "2.4.0" source = { registry = "https://pypi.org/simple" } -resolution-markers = [ - "python_full_version >= '3.14'", - "python_full_version == '3.13.*'", - "python_full_version == '3.12.*'", - "python_full_version == '3.11.*'", - "python_full_version == '3.10.*'", -] sdist = { url = "https://files.pythonhosted.org/packages/64/c7/8de93764ad66968d19329a7e0c147a2bb3c7054c554d4a119111b8f9440f/absl_py-2.4.0.tar.gz", hash = "sha256:8c6af82722b35cf71e0f4d1d47dcaebfff286e27110a99fc359349b247dfb5d4", size = 116543, upload-time = "2026-01-28T10:17:05.322Z" } wheels = [ { url = "https://files.pythonhosted.org/packages/18/a6/907a406bb7d359e6a63f99c313846d9eec4f7e6f7437809e03aa00fa3074/absl_py-2.4.0-py3-none-any.whl", hash = "sha256:88476fd881ca8aab94ffa78b7b6c632a782ab3ba1cd19c9bd423abc4fb4cd28d", size = 135750, upload-time = "2026-01-28T10:17:04.19Z" }, ] -[[package]] -name = "alabaster" -version = "0.7.16" -source = { registry = "https://pypi.org/simple" } -resolution-markers = [ - "python_full_version < '3.10'", -] -sdist = { url = "https://files.pythonhosted.org/packages/c9/3e/13dd8e5ed9094e734ac430b5d0eb4f2bb001708a8b7856cbf8e084e001ba/alabaster-0.7.16.tar.gz", hash = "sha256:75a8b99c28a5dad50dd7f8ccdd447a121ddb3892da9e53d1ca5cca3106d58d65", size = 23776, upload-time = "2024-01-10T00:56:10.189Z" } -wheels = [ - { url = "https://files.pythonhosted.org/packages/32/34/d4e1c02d3bee589efb5dfa17f88ea08bdb3e3eac12bc475462aec52ed223/alabaster-0.7.16-py3-none-any.whl", hash = "sha256:b46733c07dce03ae4e150330b975c75737fa60f0a7c591b6c8bf4928a28e2c92", size = 13511, upload-time = "2024-01-10T00:56:08.388Z" }, -] - [[package]] name = "alabaster" version = "1.0.0" source = { registry = "https://pypi.org/simple" } -resolution-markers = [ - "python_full_version >= '3.14'", - "python_full_version == '3.13.*'", - "python_full_version == '3.12.*'", - "python_full_version == '3.11.*'", - "python_full_version == '3.10.*'", -] sdist = { url = "https://files.pythonhosted.org/packages/a6/f8/d9c74d0daf3f742840fd818d69cfae176fa332022fd44e3469487d5a9420/alabaster-1.0.0.tar.gz", hash = "sha256:c00dca57bca26fa62a6d7d0a9fcce65f3e026e9bfe33e9c538fd3fbb2144fd9e", size = 24210, upload-time = "2024-07-26T18:15:03.762Z" } wheels = [ { url = "https://files.pythonhosted.org/packages/7e/b3/6b4067be973ae96ba0d615946e314c5ae35f9f993eca561b356540bb0c2b/alabaster-1.0.0-py3-none-any.whl", hash = "sha256:fc6786402dc3fcb2de3cabd5fe455a2db534b371124f1f21de8731783dec828b", size = 13929, upload-time = "2024-07-26T18:15:02.05Z" }, @@ -207,22 +166,6 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/c5/a7/0e0ab3e0b5bc1219bd80a6a0d4d72ca74d9250cb2382b7c699c147e06017/charset_normalizer-3.4.7-cp314-cp314t-win32.whl", hash = "sha256:c03a41a8784091e67a39648f70c5f97b5b6a37f216896d44d2cdcb82615339a0", size = 159827, upload-time = "2026-04-02T09:27:48.053Z" }, { url = "https://files.pythonhosted.org/packages/7a/1d/29d32e0fb40864b1f878c7f5a0b343ae676c6e2b271a2d55cc3a152391da/charset_normalizer-3.4.7-cp314-cp314t-win_amd64.whl", hash = "sha256:03853ed82eeebbce3c2abfdbc98c96dc205f32a79627688ac9a27370ea61a49c", size = 174168, upload-time = "2026-04-02T09:27:49.795Z" }, { url = "https://files.pythonhosted.org/packages/de/32/d92444ad05c7a6e41fb2036749777c163baf7a0301a040cb672d6b2b1ae9/charset_normalizer-3.4.7-cp314-cp314t-win_arm64.whl", hash = "sha256:c35abb8bfff0185efac5878da64c45dafd2b37fb0383add1be155a763c1f083d", size = 153018, upload-time = "2026-04-02T09:27:51.116Z" }, - { url = "https://files.pythonhosted.org/packages/01/1b/ef725f8eb19b5a261b30f78efa9252ef9d017985cb499102f6f49834cd12/charset_normalizer-3.4.7-cp39-cp39-macosx_10_9_universal2.whl", hash = "sha256:177a0ba5f0211d488e295aaf82707237e331c24788d8d76c96c5a41594723217", size = 299121, upload-time = "2026-04-02T09:28:14.372Z" }, - { url = "https://files.pythonhosted.org/packages/a3/22/2f12878fbc680fbbb52386cd39a379801f62eaca74fc8b323381325f0f04/charset_normalizer-3.4.7-cp39-cp39-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:6e0d51f618228538a3e8f46bd246f87a6cd030565e015803691603f55e12afb5", size = 200612, upload-time = "2026-04-02T09:28:16.162Z" }, - { url = "https://files.pythonhosted.org/packages/bc/b6/10c84e789126ca97d4a7228863a30481e786980a8b8cfcbf4f30658ca63c/charset_normalizer-3.4.7-cp39-cp39-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:14265bfe1f09498b9d8ec91e9ec9fa52775edf90fcbde092b25f4a33d444fea9", size = 221041, upload-time = "2026-04-02T09:28:17.554Z" }, - { url = "https://files.pythonhosted.org/packages/21/7b/c414866a138400b2e81973d006da7f694cfeaf895ef07d2cba9a8743841a/charset_normalizer-3.4.7-cp39-cp39-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:87fad7d9ba98c86bcb41b2dc8dbb326619be2562af1f8ff50776a39e55721c5a", size = 216323, upload-time = "2026-04-02T09:28:18.863Z" }, - { url = "https://files.pythonhosted.org/packages/2e/92/bdcf94997e06b223d826df3abed45a5ad6e17f609b7df9d25cd23b5bde30/charset_normalizer-3.4.7-cp39-cp39-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:f22dec1690b584cea26fade98b2435c132c1b5f68e39f5a0b7627cd7ae31f1dc", size = 208419, upload-time = "2026-04-02T09:28:20.332Z" }, - { url = "https://files.pythonhosted.org/packages/1a/64/3f9142293c88b1b10e199649ed1330f070c2a68e305335a5819fa7f25fa7/charset_normalizer-3.4.7-cp39-cp39-manylinux_2_31_armv7l.whl", hash = "sha256:d61f00a0869d77422d9b2aba989e2d24afa6ffd552af442e0e58de4f35ea6d00", size = 195016, upload-time = "2026-04-02T09:28:21.657Z" }, - { url = "https://files.pythonhosted.org/packages/c1/d1/d8a6b7dd5c5636b76ce0d080bc57d8e56c7bbd6bc2ac941529a35e41d84a/charset_normalizer-3.4.7-cp39-cp39-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:6370e8686f662e6a3941ee48ed4742317cafbe5707e36406e9df792cdb535776", size = 206115, upload-time = "2026-04-02T09:28:23.259Z" }, - { url = "https://files.pythonhosted.org/packages/dd/8c/60ebe912379627d023eb96995b40bc50308729f210f43d66109ca0a7bbd2/charset_normalizer-3.4.7-cp39-cp39-musllinux_1_2_aarch64.whl", hash = "sha256:a6c5863edfbe888d9eff9c8b8087354e27618d9da76425c119293f11712a6319", size = 204022, upload-time = "2026-04-02T09:28:24.779Z" }, - { url = "https://files.pythonhosted.org/packages/d5/2a/41816ceda78a551cbfdfbeab6f3891152b0e3f758ce6580c2c18c829f774/charset_normalizer-3.4.7-cp39-cp39-musllinux_1_2_armv7l.whl", hash = "sha256:ed065083d0898c9d5b4bbec7b026fd755ff7454e6e8b73a67f8c744b13986e24", size = 195914, upload-time = "2026-04-02T09:28:26.181Z" }, - { url = "https://files.pythonhosted.org/packages/8f/9b/7c7f4b7f11525fcbdfba752455314ac60646bae91cdd671d531c1f7a97c6/charset_normalizer-3.4.7-cp39-cp39-musllinux_1_2_ppc64le.whl", hash = "sha256:2cd4a60d0e2fb04537162c62bbbb4182f53541fe0ede35cdf270a1c1e723cc42", size = 222159, upload-time = "2026-04-02T09:28:27.504Z" }, - { url = "https://files.pythonhosted.org/packages/9f/57/301682e7469bdbfa2ce219a804f0668b2266ab8520570d85d3b3ef483ea3/charset_normalizer-3.4.7-cp39-cp39-musllinux_1_2_riscv64.whl", hash = "sha256:813c0e0132266c08eb87469a642cb30aaff57c5f426255419572aaeceeaa7bf4", size = 206154, upload-time = "2026-04-02T09:28:28.848Z" }, - { url = "https://files.pythonhosted.org/packages/20/ec/90339ff5cdc598b265748c1f231c7d7fbd9123a92cee10f757e0b1448de4/charset_normalizer-3.4.7-cp39-cp39-musllinux_1_2_s390x.whl", hash = "sha256:07d9e39b01743c3717745f4c530a6349eadbfa043c7577eef86c502c15df2c67", size = 217423, upload-time = "2026-04-02T09:28:30.248Z" }, - { url = "https://files.pythonhosted.org/packages/2e/e7/a7a6147f8e3375676309cf584b25c72a3bab784ea4085b0011fa07b23aeb/charset_normalizer-3.4.7-cp39-cp39-musllinux_1_2_x86_64.whl", hash = "sha256:c0f081d69a6e58272819b70288d3221a6ee64b98df852631c80f293514d3b274", size = 210604, upload-time = "2026-04-02T09:28:31.736Z" }, - { url = "https://files.pythonhosted.org/packages/1a/62/d9340c7a79c393e57807d7fb6c57e82060687891f81b74d3201958b919c1/charset_normalizer-3.4.7-cp39-cp39-win32.whl", hash = "sha256:8751d2787c9131302398b11e6c8068053dcb55d5a8964e114b6e196cf16cb366", size = 144631, upload-time = "2026-04-02T09:28:33.158Z" }, - { url = "https://files.pythonhosted.org/packages/21/e7/92901117e2ddc8facfe8235a3ecd4eb482185b2ad5d5b6606b37c1afea06/charset_normalizer-3.4.7-cp39-cp39-win_amd64.whl", hash = "sha256:12a6fff75f6bc66711b73a2f0addfc4c8c15a20e805146a02d147a318962c444", size = 154710, upload-time = "2026-04-02T09:28:34.557Z" }, - { url = "https://files.pythonhosted.org/packages/cc/4f/e1fb138201ad9a32499dd9a98aa4a5a5441fbf7f56b52b619a54b7ee8777/charset_normalizer-3.4.7-cp39-cp39-win_arm64.whl", hash = "sha256:bb8cc7534f51d9a017b93e3e85b260924f909601c3df002bcdb58ddb4dc41a5c", size = 143716, upload-time = "2026-04-02T09:28:35.908Z" }, { url = "https://files.pythonhosted.org/packages/db/8f/61959034484a4a7c527811f4721e75d02d653a35afb0b6054474d8185d4c/charset_normalizer-3.4.7-py3-none-any.whl", hash = "sha256:3dce51d0f5e7951f8bb4900c257dad282f49190fdbebecd4ba99bcc41fef404d", size = 61958, upload-time = "2026-04-02T09:28:37.794Z" }, ] @@ -240,8 +183,7 @@ name = "docutils" version = "0.21.2" source = { registry = "https://pypi.org/simple" } resolution-markers = [ - "python_full_version == '3.10.*'", - "python_full_version < '3.10'", + "python_full_version < '3.11'", ] sdist = { url = "https://files.pythonhosted.org/packages/ae/ed/aefcc8cd0ba62a0560c3c18c33925362d46c6075480bfa4df87b28e169a9/docutils-0.21.2.tar.gz", hash = "sha256:3a6b18732edf182daa3cd12775bbb338cf5691468f91eeeb109deff6ebfa986f", size = 2204444, upload-time = "2024-04-23T18:57:18.24Z" } wheels = [ @@ -253,9 +195,7 @@ name = "docutils" version = "0.22.4" source = { registry = "https://pypi.org/simple" } resolution-markers = [ - "python_full_version >= '3.14'", - "python_full_version == '3.13.*'", - "python_full_version == '3.12.*'", + "python_full_version >= '3.12'", "python_full_version == '3.11.*'", ] sdist = { url = "https://files.pythonhosted.org/packages/ae/b6/03bb70946330e88ffec97aefd3ea75ba575cb2e762061e0e62a213befee8/docutils-0.22.4.tar.gz", hash = "sha256:4db53b1fde9abecbb74d91230d32ab626d94f6badfc575d6db9194a49df29968", size = 2291750, upload-time = "2025-12-18T19:00:26.443Z" } @@ -284,69 +224,19 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/d2/23/408243171aa9aaba178d3e2559159c24c1171a641aa83b67bdd3394ead8e/idna-3.15-py3-none-any.whl", hash = "sha256:048adeaf8c2d788c40fee287673ccaa74c24ffd8dcf09ffa555a2fbb59f10ac8", size = 72340, upload-time = "2026-05-12T22:45:55.733Z" }, ] -[[package]] -name = "imagesize" -version = "1.5.0" -source = { registry = "https://pypi.org/simple" } -resolution-markers = [ - "python_full_version < '3.10'", -] -sdist = { url = "https://files.pythonhosted.org/packages/cf/59/4b0dd64676aa6fb4986a755790cb6fc558559cf0084effad516820208ec3/imagesize-1.5.0.tar.gz", hash = "sha256:8bfc5363a7f2133a89f0098451e0bcb1cd71aba4dc02bbcecb39d99d40e1b94f", size = 1281127, upload-time = "2026-03-03T01:59:54.651Z" } -wheels = [ - { url = "https://files.pythonhosted.org/packages/1e/b1/a0662b03103c66cf77101a187f396ea91167cd9b7d5d3a2e465ad2c7ee9b/imagesize-1.5.0-py2.py3-none-any.whl", hash = "sha256:32677681b3f434c2cb496f00e89c5a291247b35b1f527589909e008057da5899", size = 5763, upload-time = "2026-03-03T01:59:52.343Z" }, -] - [[package]] name = "imagesize" version = "2.0.0" source = { registry = "https://pypi.org/simple" } -resolution-markers = [ - "python_full_version >= '3.14'", - "python_full_version == '3.13.*'", - "python_full_version == '3.12.*'", - "python_full_version == '3.11.*'", - "python_full_version == '3.10.*'", -] sdist = { url = "https://files.pythonhosted.org/packages/6c/e6/7bf14eeb8f8b7251141944835abd42eb20a658d89084b7e1f3e5fe394090/imagesize-2.0.0.tar.gz", hash = "sha256:8e8358c4a05c304f1fccf7ff96f036e7243a189e9e42e90851993c558cfe9ee3", size = 1773045, upload-time = "2026-03-03T14:18:29.941Z" } wheels = [ { url = "https://files.pythonhosted.org/packages/5f/53/fb7122b71361a0d121b669dcf3d31244ef75badbbb724af388948de543e2/imagesize-2.0.0-py2.py3-none-any.whl", hash = "sha256:5667c5bbb57ab3f1fa4bc366f4fbc971db3d5ed011fd2715fd8001f782718d96", size = 9441, upload-time = "2026-03-03T14:18:27.892Z" }, ] -[[package]] -name = "importlib-metadata" -version = "8.7.1" -source = { registry = "https://pypi.org/simple" } -dependencies = [ - { name = "zipp", marker = "python_full_version < '3.10'" }, -] -sdist = { url = "https://files.pythonhosted.org/packages/f3/49/3b30cad09e7771a4982d9975a8cbf64f00d4a1ececb53297f1d9a7be1b10/importlib_metadata-8.7.1.tar.gz", hash = "sha256:49fef1ae6440c182052f407c8d34a68f72efc36db9ca90dc0113398f2fdde8bb", size = 57107, upload-time = "2025-12-21T10:00:19.278Z" } -wheels = [ - { url = "https://files.pythonhosted.org/packages/fa/5e/f8e9a1d23b9c20a551a8a02ea3637b4642e22c2626e3a13a9a29cdea99eb/importlib_metadata-8.7.1-py3-none-any.whl", hash = "sha256:5a1f80bf1daa489495071efbb095d75a634cf28a8bc299581244063b53176151", size = 27865, upload-time = "2025-12-21T10:00:18.329Z" }, -] - -[[package]] -name = "iniconfig" -version = "2.1.0" -source = { registry = "https://pypi.org/simple" } -resolution-markers = [ - "python_full_version < '3.10'", -] -sdist = { url = "https://files.pythonhosted.org/packages/f2/97/ebf4da567aa6827c909642694d71c9fcf53e5b504f2d96afea02718862f3/iniconfig-2.1.0.tar.gz", hash = "sha256:3abbd2e30b36733fee78f9c7f7308f2d0050e88f0087fd25c2645f63c773e1c7", size = 4793, upload-time = "2025-03-19T20:09:59.721Z" } -wheels = [ - { url = "https://files.pythonhosted.org/packages/2c/e1/e6716421ea10d38022b952c159d5161ca1193197fb744506875fbb87ea7b/iniconfig-2.1.0-py3-none-any.whl", hash = "sha256:9deba5723312380e77435581c6bf4935c94cbfab9b1ed33ef8d238ea168eb760", size = 6050, upload-time = "2025-03-19T20:10:01.071Z" }, -] - [[package]] name = "iniconfig" version = "2.3.0" source = { registry = "https://pypi.org/simple" } -resolution-markers = [ - "python_full_version >= '3.14'", - "python_full_version == '3.13.*'", - "python_full_version == '3.12.*'", - "python_full_version == '3.11.*'", - "python_full_version == '3.10.*'", -] sdist = { url = "https://files.pythonhosted.org/packages/72/34/14ca021ce8e5dfedc35312d08ba8bf51fdd999c576889fc2c24cb97f4f10/iniconfig-2.3.0.tar.gz", hash = "sha256:c76315c77db068650d49c5b56314774a7804df16fee4402c1f19d6d15d8c4730", size = 20503, upload-time = "2025-10-18T21:55:43.219Z" } wheels = [ { url = "https://files.pythonhosted.org/packages/cb/b1/3846dd7f199d53cb17f49cba7e651e9ce294d8497c8c150530ed11865bb8/iniconfig-2.3.0-py3-none-any.whl", hash = "sha256:f631c04d2c48c52b84d0d0549c99ff3859c98df65b3101406327ecc7d53fbf12", size = 7484, upload-time = "2025-10-18T21:55:41.639Z" }, @@ -381,8 +271,7 @@ name = "markdown-it-py" version = "3.0.0" source = { registry = "https://pypi.org/simple" } resolution-markers = [ - "python_full_version == '3.10.*'", - "python_full_version < '3.10'", + "python_full_version < '3.11'", ] dependencies = [ { name = "mdurl", marker = "python_full_version < '3.11'" }, @@ -397,9 +286,7 @@ name = "markdown-it-py" version = "4.2.0" source = { registry = "https://pypi.org/simple" } resolution-markers = [ - "python_full_version >= '3.14'", - "python_full_version == '3.13.*'", - "python_full_version == '3.12.*'", + "python_full_version >= '3.12'", "python_full_version == '3.11.*'", ] dependencies = [ @@ -493,47 +380,14 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/fb/df/5bd7a48c256faecd1d36edc13133e51397e41b73bb77e1a69deab746ebac/markupsafe-3.0.3-cp314-cp314t-win32.whl", hash = "sha256:915c04ba3851909ce68ccc2b8e2cd691618c4dc4c4232fb7982bca3f41fd8c3d", size = 14819, upload-time = "2025-09-27T18:37:26.285Z" }, { url = "https://files.pythonhosted.org/packages/1a/8a/0402ba61a2f16038b48b39bccca271134be00c5c9f0f623208399333c448/markupsafe-3.0.3-cp314-cp314t-win_amd64.whl", hash = "sha256:4faffd047e07c38848ce017e8725090413cd80cbc23d86e55c587bf979e579c9", size = 15426, upload-time = "2025-09-27T18:37:27.316Z" }, { url = "https://files.pythonhosted.org/packages/70/bc/6f1c2f612465f5fa89b95bead1f44dcb607670fd42891d8fdcd5d039f4f4/markupsafe-3.0.3-cp314-cp314t-win_arm64.whl", hash = "sha256:32001d6a8fc98c8cb5c947787c5d08b0a50663d139f1305bac5885d98d9b40fa", size = 14146, upload-time = "2025-09-27T18:37:28.327Z" }, - { url = "https://files.pythonhosted.org/packages/56/23/0d8c13a44bde9154821586520840643467aee574d8ce79a17da539ee7fed/markupsafe-3.0.3-cp39-cp39-macosx_10_9_x86_64.whl", hash = "sha256:15d939a21d546304880945ca1ecb8a039db6b4dc49b2c5a400387cdae6a62e26", size = 11623, upload-time = "2025-09-27T18:37:29.296Z" }, - { url = "https://files.pythonhosted.org/packages/fd/23/07a2cb9a8045d5f3f0890a8c3bc0859d7a47bfd9a560b563899bec7b72ed/markupsafe-3.0.3-cp39-cp39-macosx_11_0_arm64.whl", hash = "sha256:f71a396b3bf33ecaa1626c255855702aca4d3d9fea5e051b41ac59a9c1c41edc", size = 12049, upload-time = "2025-09-27T18:37:30.234Z" }, - { url = "https://files.pythonhosted.org/packages/bc/e4/6be85eb81503f8e11b61c0b6369b6e077dcf0a74adbd9ebf6b349937b4e9/markupsafe-3.0.3-cp39-cp39-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:0f4b68347f8c5eab4a13419215bdfd7f8c9b19f2b25520968adfad23eb0ce60c", size = 21923, upload-time = "2025-09-27T18:37:31.177Z" }, - { url = "https://files.pythonhosted.org/packages/6f/bc/4dc914ead3fe6ddaef035341fee0fc956949bbd27335b611829292b89ee2/markupsafe-3.0.3-cp39-cp39-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:e8fc20152abba6b83724d7ff268c249fa196d8259ff481f3b1476383f8f24e42", size = 20543, upload-time = "2025-09-27T18:37:32.168Z" }, - { url = "https://files.pythonhosted.org/packages/89/6e/5fe81fbcfba4aef4093d5f856e5c774ec2057946052d18d168219b7bd9f9/markupsafe-3.0.3-cp39-cp39-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:949b8d66bc381ee8b007cd945914c721d9aba8e27f71959d750a46f7c282b20b", size = 20585, upload-time = "2025-09-27T18:37:33.166Z" }, - { url = "https://files.pythonhosted.org/packages/f6/f6/e0e5a3d3ae9c4020f696cd055f940ef86b64fe88de26f3a0308b9d3d048c/markupsafe-3.0.3-cp39-cp39-musllinux_1_2_aarch64.whl", hash = "sha256:3537e01efc9d4dccdf77221fb1cb3b8e1a38d5428920e0657ce299b20324d758", size = 21387, upload-time = "2025-09-27T18:37:34.185Z" }, - { url = "https://files.pythonhosted.org/packages/c8/25/651753ef4dea08ea790f4fbb65146a9a44a014986996ca40102e237aa49a/markupsafe-3.0.3-cp39-cp39-musllinux_1_2_riscv64.whl", hash = "sha256:591ae9f2a647529ca990bc681daebdd52c8791ff06c2bfa05b65163e28102ef2", size = 20133, upload-time = "2025-09-27T18:37:35.138Z" }, - { url = "https://files.pythonhosted.org/packages/dc/0a/c3cf2b4fef5f0426e8a6d7fce3cb966a17817c568ce59d76b92a233fdbec/markupsafe-3.0.3-cp39-cp39-musllinux_1_2_x86_64.whl", hash = "sha256:a320721ab5a1aba0a233739394eb907f8c8da5c98c9181d1161e77a0c8e36f2d", size = 20588, upload-time = "2025-09-27T18:37:36.096Z" }, - { url = "https://files.pythonhosted.org/packages/cd/1b/a7782984844bd519ad4ffdbebbba2671ec5d0ebbeac34736c15fb86399e8/markupsafe-3.0.3-cp39-cp39-win32.whl", hash = "sha256:df2449253ef108a379b8b5d6b43f4b1a8e81a061d6537becd5582fba5f9196d7", size = 14566, upload-time = "2025-09-27T18:37:37.09Z" }, - { url = "https://files.pythonhosted.org/packages/18/1f/8d9c20e1c9440e215a44be5ab64359e207fcb4f675543f1cf9a2a7f648d0/markupsafe-3.0.3-cp39-cp39-win_amd64.whl", hash = "sha256:7c3fb7d25180895632e5d3148dbdc29ea38ccb7fd210aa27acbd1201a1902c6e", size = 15053, upload-time = "2025-09-27T18:37:38.054Z" }, - { url = "https://files.pythonhosted.org/packages/4e/d3/fe08482b5cd995033556d45041a4f4e76e7f0521112a9c9991d40d39825f/markupsafe-3.0.3-cp39-cp39-win_arm64.whl", hash = "sha256:38664109c14ffc9e7437e86b4dceb442b0096dfe3541d7864d9cbe1da4cf36c8", size = 13928, upload-time = "2025-09-27T18:37:39.037Z" }, -] - -[[package]] -name = "mdit-py-plugins" -version = "0.4.2" -source = { registry = "https://pypi.org/simple" } -resolution-markers = [ - "python_full_version < '3.10'", -] -dependencies = [ - { name = "markdown-it-py", version = "3.0.0", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.10'" }, -] -sdist = { url = "https://files.pythonhosted.org/packages/19/03/a2ecab526543b152300717cf232bb4bb8605b6edb946c845016fa9c9c9fd/mdit_py_plugins-0.4.2.tar.gz", hash = "sha256:5f2cd1fdb606ddf152d37ec30e46101a60512bc0e5fa1a7002c36647b09e26b5", size = 43542, upload-time = "2024-09-09T20:27:49.564Z" } -wheels = [ - { url = "https://files.pythonhosted.org/packages/a7/f7/7782a043553ee469c1ff49cfa1cdace2d6bf99a1f333cf38676b3ddf30da/mdit_py_plugins-0.4.2-py3-none-any.whl", hash = "sha256:0c673c3f889399a33b95e88d2f0d111b4447bdfea7f237dab2d488f459835636", size = 55316, upload-time = "2024-09-09T20:27:48.397Z" }, ] [[package]] name = "mdit-py-plugins" version = "0.6.1" source = { registry = "https://pypi.org/simple" } -resolution-markers = [ - "python_full_version >= '3.14'", - "python_full_version == '3.13.*'", - "python_full_version == '3.12.*'", - "python_full_version == '3.11.*'", - "python_full_version == '3.10.*'", -] dependencies = [ - { name = "markdown-it-py", version = "3.0.0", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version == '3.10.*'" }, + { name = "markdown-it-py", version = "3.0.0", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.11'" }, { name = "markdown-it-py", version = "4.2.0", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version >= '3.11'" }, ] sdist = { url = "https://files.pythonhosted.org/packages/59/fc/f8d0863f8862f25602c0404d75568e89fb6b4109804645e5cdfb1be5cf56/mdit_py_plugins-0.6.1.tar.gz", hash = "sha256:a2bca0f039f39dbd35fb74ae1b5f998608c437463371f0ff7f49a19a17a114d0", size = 56114, upload-time = "2026-05-13T09:03:38.91Z" } @@ -550,40 +404,20 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/b3/38/89ba8ad64ae25be8de66a6d463314cf1eb366222074cfda9ee839c56a4b4/mdurl-0.1.2-py3-none-any.whl", hash = "sha256:84008a41e51615a49fc9966191ff91509e3c40b939176e643fd50a5c2196b8f8", size = 9979, upload-time = "2022-08-14T12:40:09.779Z" }, ] -[[package]] -name = "myst-parser" -version = "3.0.1" -source = { registry = "https://pypi.org/simple" } -resolution-markers = [ - "python_full_version < '3.10'", -] -dependencies = [ - { name = "docutils", version = "0.21.2", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.10'" }, - { name = "jinja2", marker = "python_full_version < '3.10'" }, - { name = "markdown-it-py", version = "3.0.0", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.10'" }, - { name = "mdit-py-plugins", version = "0.4.2", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.10'" }, - { name = "pyyaml", marker = "python_full_version < '3.10'" }, - { name = "sphinx", version = "7.4.7", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.10'" }, -] -sdist = { url = "https://files.pythonhosted.org/packages/49/64/e2f13dac02f599980798c01156393b781aec983b52a6e4057ee58f07c43a/myst_parser-3.0.1.tar.gz", hash = "sha256:88f0cb406cb363b077d176b51c476f62d60604d68a8dcdf4832e080441301a87", size = 92392, upload-time = "2024-04-28T20:22:42.116Z" } -wheels = [ - { url = "https://files.pythonhosted.org/packages/e2/de/21aa8394f16add8f7427f0a1326ccd2b3a2a8a3245c9252bc5ac034c6155/myst_parser-3.0.1-py3-none-any.whl", hash = "sha256:6457aaa33a5d474aca678b8ead9b3dc298e89c68e67012e73146ea6fd54babf1", size = 83163, upload-time = "2024-04-28T20:22:39.985Z" }, -] - [[package]] name = "myst-parser" version = "4.0.1" source = { registry = "https://pypi.org/simple" } resolution-markers = [ - "python_full_version == '3.10.*'", + "python_full_version < '3.11'", ] dependencies = [ - { name = "docutils", version = "0.21.2", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version == '3.10.*'" }, - { name = "jinja2", marker = "python_full_version == '3.10.*'" }, - { name = "markdown-it-py", version = "3.0.0", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version == '3.10.*'" }, - { name = "mdit-py-plugins", version = "0.6.1", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version == '3.10.*'" }, - { name = "pyyaml", marker = "python_full_version == '3.10.*'" }, - { name = "sphinx", version = "8.1.3", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version == '3.10.*'" }, + { name = "docutils", version = "0.21.2", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.11'" }, + { name = "jinja2", marker = "python_full_version < '3.11'" }, + { name = "markdown-it-py", version = "3.0.0", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.11'" }, + { name = "mdit-py-plugins", marker = "python_full_version < '3.11'" }, + { name = "pyyaml", marker = "python_full_version < '3.11'" }, + { name = "sphinx", version = "8.1.3", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.11'" }, ] sdist = { url = "https://files.pythonhosted.org/packages/66/a5/9626ba4f73555b3735ad86247a8077d4603aa8628537687c839ab08bfe44/myst_parser-4.0.1.tar.gz", hash = "sha256:5cfea715e4f3574138aecbf7d54132296bfd72bb614d31168f48c477a830a7c4", size = 93985, upload-time = "2025-02-12T10:53:03.833Z" } wheels = [ @@ -595,16 +429,14 @@ name = "myst-parser" version = "5.1.0" source = { registry = "https://pypi.org/simple" } resolution-markers = [ - "python_full_version >= '3.14'", - "python_full_version == '3.13.*'", - "python_full_version == '3.12.*'", + "python_full_version >= '3.12'", "python_full_version == '3.11.*'", ] dependencies = [ { name = "docutils", version = "0.22.4", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version >= '3.11'" }, { name = "jinja2", marker = "python_full_version >= '3.11'" }, { name = "markdown-it-py", version = "4.2.0", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version >= '3.11'" }, - { name = "mdit-py-plugins", version = "0.6.1", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version >= '3.11'" }, + { name = "mdit-py-plugins", marker = "python_full_version >= '3.11'" }, { name = "pyyaml", marker = "python_full_version >= '3.11'" }, { name = "sphinx", version = "9.0.4", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version == '3.11.*'" }, { name = "sphinx", version = "9.1.0", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version >= '3.12'" }, @@ -659,46 +491,18 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/f4/7e/a72dd26f3b0f4f2bf1dd8923c85f7ceb43172af56d63c7383eb62b332364/pygments-2.20.0-py3-none-any.whl", hash = "sha256:81a9e26dd42fd28a23a2d169d86d7ac03b46e2f8b59ed4698fb4785f946d0176", size = 1231151, upload-time = "2026-03-29T13:29:30.038Z" }, ] -[[package]] -name = "pytest" -version = "8.4.2" -source = { registry = "https://pypi.org/simple" } -resolution-markers = [ - "python_full_version < '3.10'", -] -dependencies = [ - { name = "colorama", marker = "python_full_version < '3.10' and sys_platform == 'win32'" }, - { name = "exceptiongroup", marker = "python_full_version < '3.10'" }, - { name = "iniconfig", version = "2.1.0", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.10'" }, - { name = "packaging", marker = "python_full_version < '3.10'" }, - { name = "pluggy", marker = "python_full_version < '3.10'" }, - { name = "pygments", marker = "python_full_version < '3.10'" }, - { name = "tomli", marker = "python_full_version < '3.10'" }, -] -sdist = { url = "https://files.pythonhosted.org/packages/a3/5c/00a0e072241553e1a7496d638deababa67c5058571567b92a7eaa258397c/pytest-8.4.2.tar.gz", hash = "sha256:86c0d0b93306b961d58d62a4db4879f27fe25513d4b969df351abdddb3c30e01", size = 1519618, upload-time = "2025-09-04T14:34:22.711Z" } -wheels = [ - { url = "https://files.pythonhosted.org/packages/a8/a4/20da314d277121d6534b3a980b29035dcd51e6744bd79075a6ce8fa4eb8d/pytest-8.4.2-py3-none-any.whl", hash = "sha256:872f880de3fc3a5bdc88a11b39c9710c3497a547cfa9320bc3c5e62fbf272e79", size = 365750, upload-time = "2025-09-04T14:34:20.226Z" }, -] - [[package]] name = "pytest" version = "9.1.1" source = { registry = "https://pypi.org/simple" } -resolution-markers = [ - "python_full_version >= '3.14'", - "python_full_version == '3.13.*'", - "python_full_version == '3.12.*'", - "python_full_version == '3.11.*'", - "python_full_version == '3.10.*'", -] dependencies = [ - { name = "colorama", marker = "python_full_version >= '3.10' and sys_platform == 'win32'" }, - { name = "exceptiongroup", marker = "python_full_version == '3.10.*'" }, - { name = "iniconfig", version = "2.3.0", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version >= '3.10'" }, - { name = "packaging", marker = "python_full_version >= '3.10'" }, - { name = "pluggy", marker = "python_full_version >= '3.10'" }, - { name = "pygments", marker = "python_full_version >= '3.10'" }, - { name = "tomli", marker = "python_full_version == '3.10.*'" }, + { name = "colorama", marker = "sys_platform == 'win32'" }, + { name = "exceptiongroup", marker = "python_full_version < '3.11'" }, + { name = "iniconfig" }, + { name = "packaging" }, + { name = "pluggy" }, + { name = "pygments" }, + { name = "tomli", marker = "python_full_version < '3.11'" }, ] sdist = { url = "https://files.pythonhosted.org/packages/e4/47/b9efed96c114afcfa3c9d3fe98a76a1d14c74a9e266d397cf6eb64be5e01/pytest-9.1.1.tar.gz", hash = "sha256:1088fbde8f2b49d95a549a195707afa7a76a3ce9bcadc26b6d71f0ffda5fe313", size = 1636369, upload-time = "2026-06-19T10:58:32.857Z" } wheels = [ @@ -710,8 +514,7 @@ name = "pytest-bazel" version = "0.1.6" source = { registry = "https://pypi.org/simple" } dependencies = [ - { name = "pytest", version = "8.4.2", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.10'" }, - { name = "pytest", version = "9.1.1", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version >= '3.10'" }, + { name = "pytest" }, ] wheels = [ { url = "https://files.pythonhosted.org/packages/c2/90/4726a39728fb5a5f77a773c5e9d7bfd3a0120aa2029963a88fd06e2c5fb2/pytest_bazel-0.1.6-py3-none-any.whl", hash = "sha256:a29e80e1d67c3db801bdd4d0b6b742f2bfb48cd6841caa33401458e5c4e29c21", size = 10126, upload-time = "2025-10-31T08:41:17.165Z" }, @@ -722,8 +525,7 @@ name = "pytest-mock" version = "3.15.1" source = { registry = "https://pypi.org/simple" } dependencies = [ - { name = "pytest", version = "8.4.2", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.10'" }, - { name = "pytest", version = "9.1.1", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version >= '3.10'" }, + { name = "pytest" }, ] sdist = { url = "https://files.pythonhosted.org/packages/68/14/eb014d26be205d38ad5ad20d9a80f7d201472e08167f0bb4361e251084a9/pytest_mock-3.15.1.tar.gz", hash = "sha256:1849a238f6f396da19762269de72cb1814ab44416fa73a8686deac10b0d87a0f", size = 34036, upload-time = "2025-09-16T16:37:27.081Z" } wheels = [ @@ -792,15 +594,6 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/da/92/1446574745d74df0c92e6aa4a7b0b3130706a4142b2d1a5869f2eaa423c6/pyyaml-6.0.3-cp314-cp314t-musllinux_1_2_x86_64.whl", hash = "sha256:16249ee61e95f858e83976573de0f5b2893b3677ba71c9dd36b9cf8be9ac6d65", size = 829923, upload-time = "2025-09-25T21:32:54.537Z" }, { url = "https://files.pythonhosted.org/packages/f0/7a/1c7270340330e575b92f397352af856a8c06f230aa3e76f86b39d01b416a/pyyaml-6.0.3-cp314-cp314t-win_amd64.whl", hash = "sha256:4ad1906908f2f5ae4e5a8ddfce73c320c2a1429ec52eafd27138b7f1cbe341c9", size = 174062, upload-time = "2025-09-25T21:32:55.767Z" }, { url = "https://files.pythonhosted.org/packages/f1/12/de94a39c2ef588c7e6455cfbe7343d3b2dc9d6b6b2f40c4c6565744c873d/pyyaml-6.0.3-cp314-cp314t-win_arm64.whl", hash = "sha256:ebc55a14a21cb14062aa4162f906cd962b28e2e9ea38f9b4391244cd8de4ae0b", size = 149341, upload-time = "2025-09-25T21:32:56.828Z" }, - { url = "https://files.pythonhosted.org/packages/9f/62/67fc8e68a75f738c9200422bf65693fb79a4cd0dc5b23310e5202e978090/pyyaml-6.0.3-cp39-cp39-macosx_10_13_x86_64.whl", hash = "sha256:b865addae83924361678b652338317d1bd7e79b1f4596f96b96c77a5a34b34da", size = 184450, upload-time = "2025-09-25T21:33:00.618Z" }, - { url = "https://files.pythonhosted.org/packages/ae/92/861f152ce87c452b11b9d0977952259aa7df792d71c1053365cc7b09cc08/pyyaml-6.0.3-cp39-cp39-macosx_11_0_arm64.whl", hash = "sha256:c3355370a2c156cffb25e876646f149d5d68f5e0a3ce86a5084dd0b64a994917", size = 174319, upload-time = "2025-09-25T21:33:02.086Z" }, - { url = "https://files.pythonhosted.org/packages/d0/cd/f0cfc8c74f8a030017a2b9c771b7f47e5dd702c3e28e5b2071374bda2948/pyyaml-6.0.3-cp39-cp39-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:3c5677e12444c15717b902a5798264fa7909e41153cdf9ef7ad571b704a63dd9", size = 737631, upload-time = "2025-09-25T21:33:03.25Z" }, - { url = "https://files.pythonhosted.org/packages/ef/b2/18f2bd28cd2055a79a46c9b0895c0b3d987ce40ee471cecf58a1a0199805/pyyaml-6.0.3-cp39-cp39-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:5ed875a24292240029e4483f9d4a4b8a1ae08843b9c54f43fcc11e404532a8a5", size = 836795, upload-time = "2025-09-25T21:33:05.014Z" }, - { url = "https://files.pythonhosted.org/packages/73/b9/793686b2d54b531203c160ef12bec60228a0109c79bae6c1277961026770/pyyaml-6.0.3-cp39-cp39-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:0150219816b6a1fa26fb4699fb7daa9caf09eb1999f3b70fb6e786805e80375a", size = 750767, upload-time = "2025-09-25T21:33:06.398Z" }, - { url = "https://files.pythonhosted.org/packages/a9/86/a137b39a611def2ed78b0e66ce2fe13ee701a07c07aebe55c340ed2a050e/pyyaml-6.0.3-cp39-cp39-musllinux_1_2_aarch64.whl", hash = "sha256:fa160448684b4e94d80416c0fa4aac48967a969efe22931448d853ada8baf926", size = 727982, upload-time = "2025-09-25T21:33:08.708Z" }, - { url = "https://files.pythonhosted.org/packages/dd/62/71c27c94f457cf4418ef8ccc71735324c549f7e3ea9d34aba50874563561/pyyaml-6.0.3-cp39-cp39-musllinux_1_2_x86_64.whl", hash = "sha256:27c0abcb4a5dac13684a37f76e701e054692a9b2d3064b70f5e4eb54810553d7", size = 755677, upload-time = "2025-09-25T21:33:09.876Z" }, - { url = "https://files.pythonhosted.org/packages/29/3d/6f5e0d58bd924fb0d06c3a6bad00effbdae2de5adb5cda5648006ffbd8d3/pyyaml-6.0.3-cp39-cp39-win32.whl", hash = "sha256:1ebe39cb5fc479422b83de611d14e2c0d3bb2a18bbcb01f229ab3cfbd8fee7a0", size = 142592, upload-time = "2025-09-25T21:33:10.983Z" }, - { url = "https://files.pythonhosted.org/packages/f0/0c/25113e0b5e103d7f1490c0e947e303fe4a696c10b501dea7a9f49d4e876c/pyyaml-6.0.3-cp39-cp39-win_amd64.whl", hash = "sha256:2e71d11abed7344e42a8849600193d15b6def118602c4c176f748e4583246007", size = 158777, upload-time = "2025-09-25T21:33:15.55Z" }, ] [[package]] @@ -810,48 +603,22 @@ source = { registry = "https://pypi.org/simple" } dependencies = [ { name = "jinja2" }, { name = "packaging" }, - { name = "requests", version = "2.32.5", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.10'" }, - { name = "requests", version = "2.34.2", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version >= '3.10'" }, + { name = "requests" }, ] sdist = { url = "https://files.pythonhosted.org/packages/e8/ce/38130d8dec600bf5413eb89a3413dd38f204c7c728c4947e12ff8cb793b7/readthedocs-sphinx-ext-2.2.5.tar.gz", hash = "sha256:ee5fd5b99db9f0c180b2396cbce528aa36671951b9526bb0272dbfce5517bd27", size = 12303, upload-time = "2023-12-19T10:00:49.573Z" } wheels = [ { url = "https://files.pythonhosted.org/packages/64/71/c89e7709a0d4f93af1848e9855112299a820b470d84f917b4dd5998bdd07/readthedocs_sphinx_ext-2.2.5-py2.py3-none-any.whl", hash = "sha256:f8c56184ea011c972dd45a90122568587cc85b0127bc9cf064d17c68bc809daa", size = 11332, upload-time = "2023-12-19T10:00:43.972Z" }, ] -[[package]] -name = "requests" -version = "2.32.5" -source = { registry = "https://pypi.org/simple" } -resolution-markers = [ - "python_full_version < '3.10'", -] -dependencies = [ - { name = "certifi", marker = "python_full_version < '3.10'" }, - { name = "charset-normalizer", marker = "python_full_version < '3.10'" }, - { name = "idna", marker = "python_full_version < '3.10'" }, - { name = "urllib3", version = "2.6.3", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.10'" }, -] -sdist = { url = "https://files.pythonhosted.org/packages/c9/74/b3ff8e6c8446842c3f5c837e9c3dfcfe2018ea6ecef224c710c85ef728f4/requests-2.32.5.tar.gz", hash = "sha256:dbba0bac56e100853db0ea71b82b4dfd5fe2bf6d3754a8893c3af500cec7d7cf", size = 134517, upload-time = "2025-08-18T20:46:02.573Z" } -wheels = [ - { url = "https://files.pythonhosted.org/packages/1e/db/4254e3eabe8020b458f1a747140d32277ec7a271daf1d235b70dc0b4e6e3/requests-2.32.5-py3-none-any.whl", hash = "sha256:2462f94637a34fd532264295e186976db0f5d453d1cdd31473c85a6a161affb6", size = 64738, upload-time = "2025-08-18T20:46:00.542Z" }, -] - [[package]] name = "requests" version = "2.34.2" source = { registry = "https://pypi.org/simple" } -resolution-markers = [ - "python_full_version >= '3.14'", - "python_full_version == '3.13.*'", - "python_full_version == '3.12.*'", - "python_full_version == '3.11.*'", - "python_full_version == '3.10.*'", -] dependencies = [ - { name = "certifi", marker = "python_full_version >= '3.10'" }, - { name = "charset-normalizer", marker = "python_full_version >= '3.10'" }, - { name = "idna", marker = "python_full_version >= '3.10'" }, - { name = "urllib3", version = "2.7.0", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version >= '3.10'" }, + { name = "certifi" }, + { name = "charset-normalizer" }, + { name = "idna" }, + { name = "urllib3" }, ] sdist = { url = "https://files.pythonhosted.org/packages/ac/c3/e2a2b89f2d3e2179abd6d00ebd70bff6273f37fb3e0cc209f48b39d00cbf/requests-2.34.2.tar.gz", hash = "sha256:f288924cae4e29463698d6d60bc6a4da69c89185ad1e0bcc4104f584e960b9ed", size = 142856, upload-time = "2026-05-14T19:25:27.735Z" } wheels = [ @@ -872,22 +639,18 @@ name = "rules-python-dev" version = "0.0.0" source = { virtual = "." } dependencies = [ - { name = "absl-py", version = "2.3.1", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.10'" }, - { name = "absl-py", version = "2.4.0", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version >= '3.10'" }, + { name = "absl-py" }, { name = "macholib" }, { name = "markupsafe" }, - { name = "myst-parser", version = "3.0.1", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.10'" }, - { name = "myst-parser", version = "4.0.1", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version == '3.10.*'" }, + { name = "myst-parser", version = "4.0.1", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.11'" }, { name = "myst-parser", version = "5.1.0", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version >= '3.11'" }, { name = "pefile" }, { name = "pyelftools" }, - { name = "pytest", version = "8.4.2", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.10'" }, - { name = "pytest", version = "9.1.1", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version >= '3.10'" }, + { name = "pytest" }, { name = "pytest-bazel" }, { name = "pytest-mock" }, { name = "readthedocs-sphinx-ext" }, - { name = "sphinx", version = "7.4.7", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.10'" }, - { name = "sphinx", version = "8.1.3", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version == '3.10.*'" }, + { name = "sphinx", version = "8.1.3", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.11'" }, { name = "sphinx", version = "9.0.4", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version == '3.11.*'" }, { name = "sphinx", version = "9.1.0", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version >= '3.12'" }, { name = "sphinx-autodoc2" }, @@ -925,63 +688,31 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/c8/78/3565d011c61f5a43488987ee32b6f3f656e7f107ac2782dd57bdd7d91d9a/snowballstemmer-3.0.1-py3-none-any.whl", hash = "sha256:6cd7b3897da8d6c9ffb968a6781fa6532dce9c3618a4b127d920dab764a19064", size = 103274, upload-time = "2025-05-09T16:34:50.371Z" }, ] -[[package]] -name = "sphinx" -version = "7.4.7" -source = { registry = "https://pypi.org/simple" } -resolution-markers = [ - "python_full_version < '3.10'", -] -dependencies = [ - { name = "alabaster", version = "0.7.16", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.10'" }, - { name = "babel", marker = "python_full_version < '3.10'" }, - { name = "colorama", marker = "python_full_version < '3.10' and sys_platform == 'win32'" }, - { name = "docutils", version = "0.21.2", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.10'" }, - { name = "imagesize", version = "1.5.0", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.10'" }, - { name = "importlib-metadata", marker = "python_full_version < '3.10'" }, - { name = "jinja2", marker = "python_full_version < '3.10'" }, - { name = "packaging", marker = "python_full_version < '3.10'" }, - { name = "pygments", marker = "python_full_version < '3.10'" }, - { name = "requests", version = "2.32.5", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.10'" }, - { name = "snowballstemmer", marker = "python_full_version < '3.10'" }, - { name = "sphinxcontrib-applehelp", marker = "python_full_version < '3.10'" }, - { name = "sphinxcontrib-devhelp", marker = "python_full_version < '3.10'" }, - { name = "sphinxcontrib-htmlhelp", marker = "python_full_version < '3.10'" }, - { name = "sphinxcontrib-jsmath", marker = "python_full_version < '3.10'" }, - { name = "sphinxcontrib-qthelp", marker = "python_full_version < '3.10'" }, - { name = "sphinxcontrib-serializinghtml", marker = "python_full_version < '3.10'" }, - { name = "tomli", marker = "python_full_version < '3.10'" }, -] -sdist = { url = "https://files.pythonhosted.org/packages/5b/be/50e50cb4f2eff47df05673d361095cafd95521d2a22521b920c67a372dcb/sphinx-7.4.7.tar.gz", hash = "sha256:242f92a7ea7e6c5b406fdc2615413890ba9f699114a9c09192d7dfead2ee9cfe", size = 8067911, upload-time = "2024-07-20T14:46:56.059Z" } -wheels = [ - { url = "https://files.pythonhosted.org/packages/0d/ef/153f6803c5d5f8917dbb7f7fcf6d34a871ede3296fa89c2c703f5f8a6c8e/sphinx-7.4.7-py3-none-any.whl", hash = "sha256:c2419e2135d11f1951cd994d6eb18a1835bd8fdd8429f9ca375dc1f3281bd239", size = 3401624, upload-time = "2024-07-20T14:46:52.142Z" }, -] - [[package]] name = "sphinx" version = "8.1.3" source = { registry = "https://pypi.org/simple" } resolution-markers = [ - "python_full_version == '3.10.*'", + "python_full_version < '3.11'", ] dependencies = [ - { name = "alabaster", version = "1.0.0", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version == '3.10.*'" }, - { name = "babel", marker = "python_full_version == '3.10.*'" }, - { name = "colorama", marker = "python_full_version == '3.10.*' and sys_platform == 'win32'" }, - { name = "docutils", version = "0.21.2", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version == '3.10.*'" }, - { name = "imagesize", version = "2.0.0", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version == '3.10.*'" }, - { name = "jinja2", marker = "python_full_version == '3.10.*'" }, - { name = "packaging", marker = "python_full_version == '3.10.*'" }, - { name = "pygments", marker = "python_full_version == '3.10.*'" }, - { name = "requests", version = "2.34.2", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version == '3.10.*'" }, - { name = "snowballstemmer", marker = "python_full_version == '3.10.*'" }, - { name = "sphinxcontrib-applehelp", marker = "python_full_version == '3.10.*'" }, - { name = "sphinxcontrib-devhelp", marker = "python_full_version == '3.10.*'" }, - { name = "sphinxcontrib-htmlhelp", marker = "python_full_version == '3.10.*'" }, - { name = "sphinxcontrib-jsmath", marker = "python_full_version == '3.10.*'" }, - { name = "sphinxcontrib-qthelp", marker = "python_full_version == '3.10.*'" }, - { name = "sphinxcontrib-serializinghtml", marker = "python_full_version == '3.10.*'" }, - { name = "tomli", marker = "python_full_version == '3.10.*'" }, + { name = "alabaster", marker = "python_full_version < '3.11'" }, + { name = "babel", marker = "python_full_version < '3.11'" }, + { name = "colorama", marker = "python_full_version < '3.11' and sys_platform == 'win32'" }, + { name = "docutils", version = "0.21.2", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.11'" }, + { name = "imagesize", marker = "python_full_version < '3.11'" }, + { name = "jinja2", marker = "python_full_version < '3.11'" }, + { name = "packaging", marker = "python_full_version < '3.11'" }, + { name = "pygments", marker = "python_full_version < '3.11'" }, + { name = "requests", marker = "python_full_version < '3.11'" }, + { name = "snowballstemmer", marker = "python_full_version < '3.11'" }, + { name = "sphinxcontrib-applehelp", marker = "python_full_version < '3.11'" }, + { name = "sphinxcontrib-devhelp", marker = "python_full_version < '3.11'" }, + { name = "sphinxcontrib-htmlhelp", marker = "python_full_version < '3.11'" }, + { name = "sphinxcontrib-jsmath", marker = "python_full_version < '3.11'" }, + { name = "sphinxcontrib-qthelp", marker = "python_full_version < '3.11'" }, + { name = "sphinxcontrib-serializinghtml", marker = "python_full_version < '3.11'" }, + { name = "tomli", marker = "python_full_version < '3.11'" }, ] sdist = { url = "https://files.pythonhosted.org/packages/6f/6d/be0b61178fe2cdcb67e2a92fc9ebb488e3c51c4f74a36a7824c0adf23425/sphinx-8.1.3.tar.gz", hash = "sha256:43c1911eecb0d3e161ad78611bc905d1ad0e523e4ddc202a58a821773dc4c927", size = 8184611, upload-time = "2024-10-13T20:27:13.93Z" } wheels = [ @@ -996,15 +727,15 @@ resolution-markers = [ "python_full_version == '3.11.*'", ] dependencies = [ - { name = "alabaster", version = "1.0.0", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version == '3.11.*'" }, + { name = "alabaster", marker = "python_full_version == '3.11.*'" }, { name = "babel", marker = "python_full_version == '3.11.*'" }, { name = "colorama", marker = "python_full_version == '3.11.*' and sys_platform == 'win32'" }, { name = "docutils", version = "0.22.4", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version == '3.11.*'" }, - { name = "imagesize", version = "2.0.0", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version == '3.11.*'" }, + { name = "imagesize", marker = "python_full_version == '3.11.*'" }, { name = "jinja2", marker = "python_full_version == '3.11.*'" }, { name = "packaging", marker = "python_full_version == '3.11.*'" }, { name = "pygments", marker = "python_full_version == '3.11.*'" }, - { name = "requests", version = "2.34.2", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version == '3.11.*'" }, + { name = "requests", marker = "python_full_version == '3.11.*'" }, { name = "roman-numerals", marker = "python_full_version == '3.11.*'" }, { name = "snowballstemmer", marker = "python_full_version == '3.11.*'" }, { name = "sphinxcontrib-applehelp", marker = "python_full_version == '3.11.*'" }, @@ -1024,20 +755,18 @@ name = "sphinx" version = "9.1.0" source = { registry = "https://pypi.org/simple" } resolution-markers = [ - "python_full_version >= '3.14'", - "python_full_version == '3.13.*'", - "python_full_version == '3.12.*'", + "python_full_version >= '3.12'", ] dependencies = [ - { name = "alabaster", version = "1.0.0", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version >= '3.12'" }, + { name = "alabaster", marker = "python_full_version >= '3.12'" }, { name = "babel", marker = "python_full_version >= '3.12'" }, { name = "colorama", marker = "python_full_version >= '3.12' and sys_platform == 'win32'" }, { name = "docutils", version = "0.22.4", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version >= '3.12'" }, - { name = "imagesize", version = "2.0.0", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version >= '3.12'" }, + { name = "imagesize", marker = "python_full_version >= '3.12'" }, { name = "jinja2", marker = "python_full_version >= '3.12'" }, { name = "packaging", marker = "python_full_version >= '3.12'" }, { name = "pygments", marker = "python_full_version >= '3.12'" }, - { name = "requests", version = "2.34.2", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version >= '3.12'" }, + { name = "requests", marker = "python_full_version >= '3.12'" }, { name = "roman-numerals", marker = "python_full_version >= '3.12'" }, { name = "snowballstemmer", marker = "python_full_version >= '3.12'" }, { name = "sphinxcontrib-applehelp", marker = "python_full_version >= '3.12'" }, @@ -1071,12 +800,10 @@ name = "sphinx-reredirects" version = "0.1.6" source = { registry = "https://pypi.org/simple" } resolution-markers = [ - "python_full_version == '3.10.*'", - "python_full_version < '3.10'", + "python_full_version < '3.11'", ] dependencies = [ - { name = "sphinx", version = "7.4.7", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.10'" }, - { name = "sphinx", version = "8.1.3", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version == '3.10.*'" }, + { name = "sphinx", version = "8.1.3", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.11'" }, ] sdist = { url = "https://files.pythonhosted.org/packages/16/6b/bcca2785de4071f604a722444d4d7ba8a9d40de3c14ad52fce93e6d92694/sphinx_reredirects-0.1.6.tar.gz", hash = "sha256:c491cba545f67be9697508727818d8626626366245ae64456fe29f37e9bbea64", size = 7080, upload-time = "2025-03-22T10:52:30.271Z" } wheels = [ @@ -1088,9 +815,7 @@ name = "sphinx-reredirects" version = "1.1.0" source = { registry = "https://pypi.org/simple" } resolution-markers = [ - "python_full_version >= '3.14'", - "python_full_version == '3.13.*'", - "python_full_version == '3.12.*'", + "python_full_version >= '3.12'", "python_full_version == '3.11.*'", ] dependencies = [ @@ -1109,8 +834,7 @@ source = { registry = "https://pypi.org/simple" } dependencies = [ { name = "docutils", version = "0.21.2", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.11'" }, { name = "docutils", version = "0.22.4", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version >= '3.11'" }, - { name = "sphinx", version = "7.4.7", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.10'" }, - { name = "sphinx", version = "8.1.3", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version == '3.10.*'" }, + { name = "sphinx", version = "8.1.3", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.11'" }, { name = "sphinx", version = "9.0.4", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version == '3.11.*'" }, { name = "sphinx", version = "9.1.0", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version >= '3.12'" }, { name = "sphinxcontrib-jquery" }, @@ -1152,8 +876,7 @@ name = "sphinxcontrib-jquery" version = "4.1" source = { registry = "https://pypi.org/simple" } dependencies = [ - { name = "sphinx", version = "7.4.7", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.10'" }, - { name = "sphinx", version = "8.1.3", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version == '3.10.*'" }, + { name = "sphinx", version = "8.1.3", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.11'" }, { name = "sphinx", version = "9.0.4", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version == '3.11.*'" }, { name = "sphinx", version = "9.1.0", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version >= '3.12'" }, ] @@ -1252,39 +975,11 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/18/67/36e9267722cc04a6b9f15c7f3441c2363321a3ea07da7ae0c0707beb2a9c/typing_extensions-4.15.0-py3-none-any.whl", hash = "sha256:f0fa19c6845758ab08074a0cfa8b7aecb71c999ca73d62883bc25cc018c4e548", size = 44614, upload-time = "2025-08-25T13:49:24.86Z" }, ] -[[package]] -name = "urllib3" -version = "2.6.3" -source = { registry = "https://pypi.org/simple" } -resolution-markers = [ - "python_full_version < '3.10'", -] -sdist = { url = "https://files.pythonhosted.org/packages/c7/24/5f1b3bdffd70275f6661c76461e25f024d5a38a46f04aaca912426a2b1d3/urllib3-2.6.3.tar.gz", hash = "sha256:1b62b6884944a57dbe321509ab94fd4d3b307075e0c2eae991ac71ee15ad38ed", size = 435556, upload-time = "2026-01-07T16:24:43.925Z" } -wheels = [ - { url = "https://files.pythonhosted.org/packages/39/08/aaaad47bc4e9dc8c725e68f9d04865dbcb2052843ff09c97b08904852d84/urllib3-2.6.3-py3-none-any.whl", hash = "sha256:bf272323e553dfb2e87d9bfd225ca7b0f467b919d7bbd355436d3fd37cb0acd4", size = 131584, upload-time = "2026-01-07T16:24:42.685Z" }, -] - [[package]] name = "urllib3" version = "2.7.0" source = { registry = "https://pypi.org/simple" } -resolution-markers = [ - "python_full_version >= '3.14'", - "python_full_version == '3.13.*'", - "python_full_version == '3.12.*'", - "python_full_version == '3.11.*'", - "python_full_version == '3.10.*'", -] sdist = { url = "https://files.pythonhosted.org/packages/53/0c/06f8b233b8fd13b9e5ee11424ef85419ba0d8ba0b3138bf360be2ff56953/urllib3-2.7.0.tar.gz", hash = "sha256:231e0ec3b63ceb14667c67be60f2f2c40a518cb38b03af60abc813da26505f4c", size = 433602, upload-time = "2026-05-07T16:13:18.596Z" } wheels = [ { url = "https://files.pythonhosted.org/packages/7f/3e/5db95bcf282c52709639744ca2a8b149baccf648e39c8cc87553df9eae0c/urllib3-2.7.0-py3-none-any.whl", hash = "sha256:9fb4c81ebbb1ce9531cce37674bbc6f1360472bc18ca9a553ede278ef7276897", size = 131087, upload-time = "2026-05-07T16:13:17.151Z" }, ] - -[[package]] -name = "zipp" -version = "3.23.1" -source = { registry = "https://pypi.org/simple" } -sdist = { url = "https://files.pythonhosted.org/packages/30/21/093488dfc7cc8964ded15ab726fad40f25fd3d788fd741cc1c5a17d78ee8/zipp-3.23.1.tar.gz", hash = "sha256:32120e378d32cd9714ad503c1d024619063ec28aad2248dc6672ad13edfa5110", size = 25965, upload-time = "2026-04-13T23:21:46.6Z" } -wheels = [ - { url = "https://files.pythonhosted.org/packages/08/8a/0861bec20485572fbddf3dfba2910e38fe249796cb73ecdeb74e07eeb8d3/zipp-3.23.1-py3-none-any.whl", hash = "sha256:0b3596c50a5c700c9cb40ba8d86d9f2cc4807e9bedb06bcdf7fac85633e444dc", size = 10378, upload-time = "2026-04-13T23:21:45.386Z" }, -] From 0d9eb231eb9d4c8f68fdb61aefa282049f28cc80 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Sun, 9 Aug 2026 23:00:36 -0700 Subject: [PATCH 921/922] agents(rules): document git push -u guardrail and delegating function docstrings (#4030) Using `git push -u` or `--set-upstream` disrupts triangle branch tracking by repointing the local tracking branch away from upstream. Additionally, delegating helper functions need consistent docstring references to their underlying implementations. Update workspace and Python agent rules to prohibit setting upstream on push and establish a docstring reference convention for delegating functions. --- .agents/rules/python.md | 5 +++++ .agents/rules/workspace.md | 2 ++ 2 files changed, 7 insertions(+) diff --git a/.agents/rules/python.md b/.agents/rules/python.md index 200295e650..d683152814 100644 --- a/.agents/rules/python.md +++ b/.agents/rules/python.md @@ -13,3 +13,8 @@ * **External Objects**: When defining a `TypedDict` for an external object, link to its definition in the docstring. +## Delegating Functions +* Module-level functions delegating to class methods should have a docstring + referring to the class method (e.g. `"""Refer to \`Class.method\`."""`). + + diff --git a/.agents/rules/workspace.md b/.agents/rules/workspace.md index b819925f83..bc64d4779c 100644 --- a/.agents/rules/workspace.md +++ b/.agents/rules/workspace.md @@ -16,3 +16,5 @@ basis: ```bash .agents/scripts/setup_triangle_branch.sh ``` +* Never use `git push -u` or `--set-upstream` (it breaks upstream tracking). + From cba8cc96586a126e5fec35400bad8ee45c0ec779 Mon Sep 17 00:00:00 2001 From: Richard Levasseur Date: Sun, 9 Aug 2026 23:21:14 -0700 Subject: [PATCH 922/922] agents: document dev python version bump and merge queue fields (#4031) Updating dev tooling Python versions requires updating multiple lock configurations and regenerating lockfiles, but this process was not documented in agent instructions. In addition, the merge-pr skill needed to query additional GitHub PR state fields to reliably inspect merge queue status and mergeability. Add developer guidance for bumping dev tooling Python versions and running lockfile update commands, and expand the gh pr view fields queried during merge queue shepherding. --- .agents/skills/merge-pr/SKILL.md | 8 +++++++- AGENTS.md | 8 ++++++++ 2 files changed, 15 insertions(+), 1 deletion(-) diff --git a/.agents/skills/merge-pr/SKILL.md b/.agents/skills/merge-pr/SKILL.md index 406590e49f..ae3e2e3ad5 100644 --- a/.agents/skills/merge-pr/SKILL.md +++ b/.agents/skills/merge-pr/SKILL.md @@ -23,5 +23,11 @@ When the user asks to merge a pull request (e.g., "merge PR ", "merge th `gh api repos/:owner/:repo/branches --jq '.[].name | select(test("gh-readonly-queue/.*/pr--"))'` and monitor commit statuses/Buildkite builds running on that temporary branch. -4. **Queue Shepherding**: Periodically check `gh pr view --json state,autoMergeRequest`. While `state` is `"OPEN"`, ensure auto-merge is enabled / queued by running `gh pr merge --auto --squash`. If `autoMergeRequest` is null (e.g., ejected from the merge queue due to a CI flake in the temporary queue branch), re-enqueue it for merge by running `gh pr merge --auto --squash` once checks are retried or green. +4. **Queue Shepherding**: Periodically check `gh pr view --json + state,autoMergeRequest,mergeStateStatus,mergeable`. While `state` is + `"OPEN"`, ensure auto-merge is enabled / queued by running `gh pr merge + --auto --squash`. If `autoMergeRequest` is null (e.g., ejected + from the merge queue due to a CI flake in the temporary queue branch), + re-enqueue it for merge by running `gh pr merge --auto --squash` + once checks are retried or green. 5. **Completion Notification**: Once `state` becomes `"MERGED"`, send a high-priority message back to the parent conversation. diff --git a/AGENTS.md b/AGENTS.md index d66e18ca11..07b523e410 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -211,6 +211,14 @@ When modifying locked/resolved requirements files: the requirements.txt file. That will update the locked/resolved requirements.txt file. +When updating the minimum Python version for dev/docs tooling (e.g. following +Dependabot alerts): + * Update `python_version` in both `lock(name = "requirements", ...)` and + `lock(name = "uv_lock", ...)` within `dev/BUILD.bazel`. + * Regenerate lockfiles by running: + `bazel run //dev:requirements.update && bazel run //dev:uv_lock.update` + * Verify docs build via `bazel build //docs:docs`. + ## rules_python idiosyncrasies When building `//docs:docs`, ignore an error about exit code 2; this is a flake,